2024-04-25T09:51:31.865 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T09:51:31.865 DEBUG:teuthology.run:Teuthology command: teuthology --name neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --archive /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673046 --verbose --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} --owner scheduled_neesingh@teuthology -- /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673046/orig.config.yaml 2024-04-25T09:51:31.909 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T09:51:32.004 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673046 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7673046' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1124 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOJo8U9+QQ9DHgmYAF/GqBfHPmk+1neS6TI80vf0Oy9sk8FrajK2LBP3Z0GtQqlKRKvdMo3Gjk1i+7Zp4VLb/bc= smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBW1PWiQ3NBSQTdGIckR1SFVrrJkj/jfHuCtXxMjY0uP/fUXMQeULwdHmHqKw36gQ1tp2IIvchRJK0G6iSztMHg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_09:36:50 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T09:51:32.004 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa; will attempt to use it 2024-04-25T09:51:32.005 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa/tasks 2024-04-25T09:51:32.005 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T09:51:32.007 INFO:teuthology.task.internal:Checking packages... 2024-04-25T09:51:32.030 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T09:51:32.030 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T09:51:32.030 INFO:teuthology.packaging:ref: None 2024-04-25T09:51:32.030 INFO:teuthology.packaging:tag: None 2024-04-25T09:51:32.030 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T09:51:32.031 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:51:32.031 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T09:51:32.174 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T09:51:32.175 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T09:51:32.182 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T09:51:32.182 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T09:51:32.189 INFO:teuthology.task.internal:Saving configuration 2024-04-25T09:51:32.205 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T09:51:32.212 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:51:32.237 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673046', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:45:44.528114', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILS/A4ZlUMGWmZoMUZlN3UHIzBxyRSQaCWe/RBNuKPk7'} 2024-04-25T09:51:32.259 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673046', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:45:44.529796', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJKKRb/xvEYzHexxe7xKD7TzdgUry3OZVe8V12ro3F+x'} 2024-04-25T09:51:32.259 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T09:51:32.265 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T09:51:32.265 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T09:51:32.265 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T09:51:32.340 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f42e8ce60d0>, signals=[15]) 2024-04-25T09:51:32.341 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T09:51:32.347 INFO:teuthology.task.internal:Opening connections... 2024-04-25T09:51:32.347 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T09:51:32.349 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:51:32.433 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T09:51:32.434 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:51:32.513 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T09:51:32.519 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2024-04-25T09:51:32.525 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2024-04-25T09:51:32.525 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2024-04-25T09:51:32.572 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T09:51:32.572 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="Ubuntu" 2024-04-25T09:51:32.573 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="22.04" 2024-04-25T09:51:32.573 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T09:51:32.573 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_CODENAME=jammy 2024-04-25T09:51:32.573 INFO:teuthology.orchestra.run.smithi060.stdout:ID=ubuntu 2024-04-25T09:51:32.573 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE=debian 2024-04-25T09:51:32.573 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T09:51:32.573 INFO:teuthology.orchestra.run.smithi060.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T09:51:32.573 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T09:51:32.573 INFO:teuthology.orchestra.run.smithi060.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T09:51:32.573 INFO:teuthology.orchestra.run.smithi060.stdout:UBUNTU_CODENAME=jammy 2024-04-25T09:51:32.574 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2024-04-25T09:51:32.611 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-04-25T09:51:32.616 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-04-25T09:51:32.616 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-04-25T09:51:32.668 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T09:51:32.668 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="Ubuntu" 2024-04-25T09:51:32.668 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="22.04" 2024-04-25T09:51:32.668 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T09:51:32.668 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_CODENAME=jammy 2024-04-25T09:51:32.668 INFO:teuthology.orchestra.run.smithi100.stdout:ID=ubuntu 2024-04-25T09:51:32.668 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE=debian 2024-04-25T09:51:32.669 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T09:51:32.669 INFO:teuthology.orchestra.run.smithi100.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T09:51:32.669 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T09:51:32.669 INFO:teuthology.orchestra.run.smithi100.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T09:51:32.669 INFO:teuthology.orchestra.run.smithi100.stdout:UBUNTU_CODENAME=jammy 2024-04-25T09:51:32.669 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-04-25T09:51:32.696 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T09:51:32.856 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T09:51:32.910 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T09:51:32.910 DEBUG:teuthology.orchestra.run.smithi060:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:51:32.913 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:51:32.917 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T09:51:32.958 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T09:51:32.958 DEBUG:teuthology.orchestra.run.smithi060:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:51:32.961 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:51:32.969 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T09:51:33.024 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T09:51:33.039 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T09:51:33.039 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T09:51:33.040 DEBUG:teuthology.orchestra.run.smithi060:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:51:33.040 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:51:33.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:51:33.045 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-04-25T09:51:33.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:51:33.046 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-04-25T09:51:33.093 INFO:teuthology.orchestra.run.smithi060.stdout:5.15.0-56-generic 2024-04-25T09:51:33.093 INFO:teuthology.task.kernel:Running kernel on smithi060: 5.15.0-56-generic 2024-04-25T09:51:33.093 DEBUG:teuthology.orchestra.run.smithi060:> sudo apt-get clean 2024-04-25T09:51:33.094 INFO:teuthology.orchestra.run.smithi100.stdout:5.15.0-56-generic 2024-04-25T09:51:33.094 INFO:teuthology.task.kernel:Running kernel on smithi100: 5.15.0-56-generic 2024-04-25T09:51:33.095 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get clean 2024-04-25T09:51:33.219 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get update 2024-04-25T09:51:33.237 DEBUG:teuthology.orchestra.run.smithi060:> sudo apt-get update 2024-04-25T09:51:33.376 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T09:51:33.402 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T09:51:33.480 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T09:51:33.487 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T09:51:33.513 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T09:51:33.513 INFO:teuthology.orchestra.run.smithi060.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T09:51:33.568 INFO:teuthology.orchestra.run.smithi060.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T09:51:33.607 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T09:51:33.714 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [619 kB] 2024-04-25T09:51:33.719 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T09:51:33.723 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T09:51:33.723 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T09:51:33.724 INFO:teuthology.orchestra.run.smithi100.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T09:51:33.744 INFO:teuthology.orchestra.run.smithi100.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T09:51:33.748 INFO:teuthology.orchestra.run.smithi100.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T09:51:33.748 INFO:teuthology.orchestra.run.smithi100.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T09:51:33.759 INFO:teuthology.orchestra.run.smithi100.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T09:51:33.765 INFO:teuthology.orchestra.run.smithi100.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T09:51:33.767 INFO:teuthology.orchestra.run.smithi100.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T09:51:33.768 INFO:teuthology.orchestra.run.smithi100.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T09:51:33.768 INFO:teuthology.orchestra.run.smithi100.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T09:51:33.801 INFO:teuthology.orchestra.run.smithi100.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T09:51:33.801 INFO:teuthology.orchestra.run.smithi100.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T09:51:33.801 INFO:teuthology.orchestra.run.smithi100.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T09:51:33.802 INFO:teuthology.orchestra.run.smithi100.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T09:51:33.803 INFO:teuthology.orchestra.run.smithi100.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T09:51:33.803 INFO:teuthology.orchestra.run.smithi100.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T09:51:33.804 INFO:teuthology.orchestra.run.smithi100.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T09:51:33.804 INFO:teuthology.orchestra.run.smithi100.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T09:51:33.822 INFO:teuthology.orchestra.run.smithi100.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T09:51:33.823 INFO:teuthology.orchestra.run.smithi100.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T09:51:33.926 INFO:teuthology.orchestra.run.smithi060.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T09:51:33.938 INFO:teuthology.orchestra.run.smithi060.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T09:51:33.954 INFO:teuthology.orchestra.run.smithi100.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T09:51:34.081 INFO:teuthology.orchestra.run.smithi060.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [619 kB] 2024-04-25T09:51:34.241 INFO:teuthology.orchestra.run.smithi060.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T09:51:34.307 INFO:teuthology.orchestra.run.smithi060.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T09:51:34.333 INFO:teuthology.orchestra.run.smithi100.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T09:51:34.457 INFO:teuthology.orchestra.run.smithi060.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T09:51:34.458 INFO:teuthology.orchestra.run.smithi060.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T09:51:34.471 INFO:teuthology.orchestra.run.smithi060.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T09:51:34.472 INFO:teuthology.orchestra.run.smithi060.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T09:51:34.479 INFO:teuthology.orchestra.run.smithi060.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T09:51:34.480 INFO:teuthology.orchestra.run.smithi060.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T09:51:34.485 INFO:teuthology.orchestra.run.smithi060.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T09:51:34.535 INFO:teuthology.orchestra.run.smithi060.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T09:51:34.535 INFO:teuthology.orchestra.run.smithi060.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T09:51:34.543 INFO:teuthology.orchestra.run.smithi060.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T09:51:34.618 INFO:teuthology.orchestra.run.smithi060.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T09:51:34.624 INFO:teuthology.orchestra.run.smithi060.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T09:51:34.625 INFO:teuthology.orchestra.run.smithi060.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T09:51:34.640 INFO:teuthology.orchestra.run.smithi060.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T09:51:34.646 INFO:teuthology.orchestra.run.smithi100.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T09:51:34.674 INFO:teuthology.orchestra.run.smithi100.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T09:51:34.675 INFO:teuthology.orchestra.run.smithi100.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T09:51:34.710 INFO:teuthology.orchestra.run.smithi060.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T09:51:34.819 INFO:teuthology.orchestra.run.smithi060.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T09:51:34.821 INFO:teuthology.orchestra.run.smithi100.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T09:51:34.822 INFO:teuthology.orchestra.run.smithi100.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T09:51:34.843 INFO:teuthology.orchestra.run.smithi100.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T09:51:34.843 INFO:teuthology.orchestra.run.smithi100.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T09:51:34.889 INFO:teuthology.orchestra.run.smithi100.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T09:51:35.079 INFO:teuthology.orchestra.run.smithi100.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T09:51:35.221 INFO:teuthology.orchestra.run.smithi060.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T09:51:35.227 INFO:teuthology.orchestra.run.smithi060.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T09:51:35.258 INFO:teuthology.orchestra.run.smithi060.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T09:51:35.258 INFO:teuthology.orchestra.run.smithi060.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T09:51:35.260 INFO:teuthology.orchestra.run.smithi060.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T09:51:35.345 INFO:teuthology.orchestra.run.smithi060.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T09:51:35.383 INFO:teuthology.orchestra.run.smithi060.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T09:51:35.401 INFO:teuthology.orchestra.run.smithi060.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T09:51:35.402 INFO:teuthology.orchestra.run.smithi060.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T09:51:35.404 INFO:teuthology.orchestra.run.smithi060.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T09:51:35.405 INFO:teuthology.orchestra.run.smithi060.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T09:51:35.406 INFO:teuthology.orchestra.run.smithi060.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T09:51:35.406 INFO:teuthology.orchestra.run.smithi060.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T09:51:35.429 INFO:teuthology.orchestra.run.smithi060.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T09:51:35.434 INFO:teuthology.orchestra.run.smithi060.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T09:51:35.486 INFO:teuthology.orchestra.run.smithi100.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T09:51:35.493 INFO:teuthology.orchestra.run.smithi100.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T09:51:35.493 INFO:teuthology.orchestra.run.smithi100.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T09:51:35.510 INFO:teuthology.orchestra.run.smithi100.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T09:51:35.512 INFO:teuthology.orchestra.run.smithi100.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T09:51:35.598 INFO:teuthology.orchestra.run.smithi060.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T09:51:35.681 INFO:teuthology.orchestra.run.smithi060.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T09:51:35.788 INFO:teuthology.orchestra.run.smithi060.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T09:51:35.846 INFO:teuthology.orchestra.run.smithi060.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T09:51:35.879 INFO:teuthology.orchestra.run.smithi060.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T09:51:39.972 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 13.3 MB in 2s (5,964 kB/s) 2024-04-25T09:51:40.549 INFO:teuthology.orchestra.run.smithi060.stdout:Fetched 13.3 MB in 3s (5,166 kB/s) 2024-04-25T09:51:41.338 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-25T09:51:41.356 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T09:51:41.418 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-25T09:51:41.620 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-04-25T09:51:41.620 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-04-25T09:51:41.795 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:51:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:51:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:51:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:51:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T09:51:41.796 INFO:teuthology.orchestra.run.smithi100.stdout: python2.7-minimal 2024-04-25T09:51:41.796 INFO:teuthology.orchestra.run.smithi100.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:51:41.797 INFO:teuthology.orchestra.run.smithi100.stdout:The following additional packages will be installed: 2024-04-25T09:51:41.797 INFO:teuthology.orchestra.run.smithi100.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T09:51:41.798 INFO:teuthology.orchestra.run.smithi100.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T09:51:41.798 INFO:teuthology.orchestra.run.smithi100.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T09:51:41.799 INFO:teuthology.orchestra.run.smithi100.stdout:Suggested packages: 2024-04-25T09:51:41.799 INFO:teuthology.orchestra.run.smithi100.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T09:51:41.799 INFO:teuthology.orchestra.run.smithi100.stdout:Recommended packages: 2024-04-25T09:51:41.799 INFO:teuthology.orchestra.run.smithi100.stdout: thermald 2024-04-25T09:51:41.851 INFO:teuthology.orchestra.run.smithi100.stdout:The following NEW packages will be installed: 2024-04-25T09:51:41.851 INFO:teuthology.orchestra.run.smithi100.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T09:51:41.851 INFO:teuthology.orchestra.run.smithi100.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T09:51:41.852 INFO:teuthology.orchestra.run.smithi100.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T09:51:41.853 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages will be upgraded: 2024-04-25T09:51:41.853 INFO:teuthology.orchestra.run.smithi100.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T09:51:42.041 INFO:teuthology.orchestra.run.smithi100.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T09:51:42.041 INFO:teuthology.orchestra.run.smithi100.stdout:Need to get 113 MB of archives. 2024-04-25T09:51:42.041 INFO:teuthology.orchestra.run.smithi100.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T09:51:42.041 INFO:teuthology.orchestra.run.smithi100.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T09:51:42.071 INFO:teuthology.orchestra.run.smithi060.stdout:Reading package lists... 2024-04-25T09:51:42.088 DEBUG:teuthology.orchestra.run.smithi060:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T09:51:42.141 INFO:teuthology.orchestra.run.smithi060.stdout:Reading package lists... 2024-04-25T09:51:42.330 INFO:teuthology.orchestra.run.smithi060.stdout:Building dependency tree... 2024-04-25T09:51:42.330 INFO:teuthology.orchestra.run.smithi060.stdout:Reading state information... 2024-04-25T09:51:42.498 INFO:teuthology.orchestra.run.smithi060.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:51:42.498 INFO:teuthology.orchestra.run.smithi060.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:51:42.499 INFO:teuthology.orchestra.run.smithi060.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:51:42.499 INFO:teuthology.orchestra.run.smithi060.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:51:42.499 INFO:teuthology.orchestra.run.smithi060.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T09:51:42.499 INFO:teuthology.orchestra.run.smithi060.stdout: python2.7-minimal 2024-04-25T09:51:42.499 INFO:teuthology.orchestra.run.smithi060.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:51:42.500 INFO:teuthology.orchestra.run.smithi060.stdout:The following additional packages will be installed: 2024-04-25T09:51:42.501 INFO:teuthology.orchestra.run.smithi060.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T09:51:42.501 INFO:teuthology.orchestra.run.smithi060.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T09:51:42.501 INFO:teuthology.orchestra.run.smithi060.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T09:51:42.502 INFO:teuthology.orchestra.run.smithi060.stdout:Suggested packages: 2024-04-25T09:51:42.502 INFO:teuthology.orchestra.run.smithi060.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T09:51:42.502 INFO:teuthology.orchestra.run.smithi060.stdout:Recommended packages: 2024-04-25T09:51:42.503 INFO:teuthology.orchestra.run.smithi060.stdout: thermald 2024-04-25T09:51:42.550 INFO:teuthology.orchestra.run.smithi060.stdout:The following NEW packages will be installed: 2024-04-25T09:51:42.550 INFO:teuthology.orchestra.run.smithi060.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T09:51:42.550 INFO:teuthology.orchestra.run.smithi060.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T09:51:42.550 INFO:teuthology.orchestra.run.smithi060.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T09:51:42.551 INFO:teuthology.orchestra.run.smithi060.stdout:The following packages will be upgraded: 2024-04-25T09:51:42.552 INFO:teuthology.orchestra.run.smithi060.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T09:51:42.733 INFO:teuthology.orchestra.run.smithi060.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T09:51:42.733 INFO:teuthology.orchestra.run.smithi060.stdout:Need to get 113 MB of archives. 2024-04-25T09:51:42.733 INFO:teuthology.orchestra.run.smithi060.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T09:51:42.733 INFO:teuthology.orchestra.run.smithi060.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T09:51:44.027 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T09:51:44.338 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T09:51:44.368 INFO:teuthology.orchestra.run.smithi060.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T09:51:44.706 INFO:teuthology.orchestra.run.smithi060.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T09:51:46.516 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T09:51:46.516 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T09:51:46.516 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T09:51:46.920 INFO:teuthology.orchestra.run.smithi060.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T09:51:46.976 INFO:teuthology.orchestra.run.smithi060.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T09:51:46.981 INFO:teuthology.orchestra.run.smithi060.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T09:51:47.471 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T09:51:47.565 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T09:51:47.701 INFO:teuthology.orchestra.run.smithi060.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T09:51:47.769 INFO:teuthology.orchestra.run.smithi060.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T09:51:47.978 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 113 MB in 6s (19.9 MB/s) 2024-04-25T09:51:48.141 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T09:51:48.185 INFO:teuthology.orchestra.run.smithi060.stdout:Fetched 113 MB in 5s (21.7 MB/s) 2024-04-25T09:51:48.391 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T09:51:50.164 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T09:51:50.168 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:51:50.230 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:51:50.454 INFO:teuthology.orchestra.run.smithi060.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T09:51:50.459 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:51:50.581 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:51:52.798 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T09:51:52.816 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T09:51:52.819 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:51:52.837 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:51:52.906 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:51:52.915 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:51:53.376 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T09:51:53.396 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:51:53.411 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T09:51:53.432 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:51:53.433 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:51:53.468 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:51:59.726 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:51:59.893 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:52:00.204 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:52:00.330 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:52:00.682 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:52:00.698 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T09:52:00.721 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T09:52:00.766 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T09:52:00.798 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:52:01.165 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:52:01.318 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:52:01.621 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T09:52:01.644 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T09:52:01.688 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T09:52:05.883 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T09:52:05.920 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:52:05.958 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:52:07.569 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T09:52:07.606 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:52:07.644 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:52:09.981 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:52:10.103 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:52:10.444 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T09:52:10.558 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:52:10.675 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T09:52:10.804 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:52:11.382 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:52:11.513 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:52:11.896 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T09:52:12.035 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:52:12.111 INFO:teuthology.orchestra.run.smithi100.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:52:12.112 INFO:teuthology.orchestra.run.smithi100.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T09:52:12.112 INFO:teuthology.orchestra.run.smithi100.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:52:12.112 INFO:teuthology.orchestra.run.smithi100.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T09:52:12.161 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T09:52:12.273 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:52:12.306 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:52:13.683 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:52:13.703 INFO:teuthology.orchestra.run.smithi060.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:52:13.703 INFO:teuthology.orchestra.run.smithi060.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T09:52:13.703 INFO:teuthology.orchestra.run.smithi060.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:52:13.703 INFO:teuthology.orchestra.run.smithi060.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T09:52:13.944 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:52:15.526 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T09:52:15.652 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T09:52:15.769 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:52:15.828 INFO:teuthology.orchestra.run.smithi100.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T09:52:15.828 INFO:teuthology.orchestra.run.smithi100.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:52:15.954 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:52:17.413 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T09:52:17.555 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T09:52:17.715 INFO:teuthology.orchestra.run.smithi060.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:52:17.789 INFO:teuthology.orchestra.run.smithi060.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T09:52:17.789 INFO:teuthology.orchestra.run.smithi060.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:52:29.840 INFO:teuthology.orchestra.run.smithi100.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T09:52:29.840 INFO:teuthology.orchestra.run.smithi100.stdout:Sourcing file `/etc/default/grub' 2024-04-25T09:52:29.850 INFO:teuthology.orchestra.run.smithi100.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T09:52:29.876 INFO:teuthology.orchestra.run.smithi100.stdout:Generating grub configuration file ... 2024-04-25T09:52:30.127 INFO:teuthology.orchestra.run.smithi100.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:52:30.145 INFO:teuthology.orchestra.run.smithi100.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:52:30.413 INFO:teuthology.orchestra.run.smithi100.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:52:30.419 INFO:teuthology.orchestra.run.smithi100.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T09:52:30.484 INFO:teuthology.orchestra.run.smithi100.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T09:52:30.489 INFO:teuthology.orchestra.run.smithi100.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T09:52:30.672 INFO:teuthology.orchestra.run.smithi100.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T09:52:30.672 INFO:teuthology.orchestra.run.smithi100.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T09:52:30.672 INFO:teuthology.orchestra.run.smithi100.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T09:52:30.710 INFO:teuthology.orchestra.run.smithi100.stdout:done 2024-04-25T09:52:31.162 DEBUG:teuthology.orchestra.run.smithi100:> dpkg -s linux-image-generic 2024-04-25T09:52:31.183 INFO:teuthology.orchestra.run.smithi100.stdout:Package: linux-image-generic 2024-04-25T09:52:31.183 INFO:teuthology.orchestra.run.smithi100.stdout:Status: install ok installed 2024-04-25T09:52:31.183 INFO:teuthology.orchestra.run.smithi100.stdout:Priority: optional 2024-04-25T09:52:31.183 INFO:teuthology.orchestra.run.smithi100.stdout:Section: kernel 2024-04-25T09:52:31.183 INFO:teuthology.orchestra.run.smithi100.stdout:Installed-Size: 21 2024-04-25T09:52:31.183 INFO:teuthology.orchestra.run.smithi100.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T09:52:31.183 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: amd64 2024-04-25T09:52:31.184 INFO:teuthology.orchestra.run.smithi100.stdout:Source: linux-meta 2024-04-25T09:52:31.184 INFO:teuthology.orchestra.run.smithi100.stdout:Version: 5.15.0.105.102 2024-04-25T09:52:31.184 INFO:teuthology.orchestra.run.smithi100.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T09:52:31.184 INFO:teuthology.orchestra.run.smithi100.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T09:52:31.184 INFO:teuthology.orchestra.run.smithi100.stdout:Recommends: thermald 2024-04-25T09:52:31.184 INFO:teuthology.orchestra.run.smithi100.stdout:Description: Generic Linux kernel image 2024-04-25T09:52:31.184 INFO:teuthology.orchestra.run.smithi100.stdout: This package will always depend on the latest generic kernel image 2024-04-25T09:52:31.184 INFO:teuthology.orchestra.run.smithi100.stdout: available. 2024-04-25T09:52:31.184 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T09:52:31.184 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T09:52:31.184 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T09:52:31.185 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-25T09:52:31.185 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T09:52:31.185 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get clean 2024-04-25T09:52:31.397 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get update 2024-04-25T09:52:31.581 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T09:52:31.644 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T09:52:31.725 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T09:52:31.806 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T09:52:32.597 INFO:teuthology.orchestra.run.smithi060.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T09:52:32.598 INFO:teuthology.orchestra.run.smithi060.stdout:Sourcing file `/etc/default/grub' 2024-04-25T09:52:32.620 INFO:teuthology.orchestra.run.smithi060.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T09:52:32.641 INFO:teuthology.orchestra.run.smithi060.stdout:Generating grub configuration file ... 2024-04-25T09:52:32.932 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-25T09:52:32.949 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T09:52:32.977 INFO:teuthology.orchestra.run.smithi060.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:52:32.993 INFO:teuthology.orchestra.run.smithi060.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:52:33.001 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-25T09:52:33.178 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-04-25T09:52:33.178 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-04-25T09:52:33.339 INFO:teuthology.orchestra.run.smithi100.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T09:52:33.339 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:52:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:52:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:52:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:52:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T09:52:33.340 INFO:teuthology.orchestra.run.smithi100.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T09:52:33.340 INFO:teuthology.orchestra.run.smithi100.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:52:33.349 INFO:teuthology.orchestra.run.smithi060.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:52:33.355 INFO:teuthology.orchestra.run.smithi060.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T09:52:33.378 INFO:teuthology.orchestra.run.smithi100.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T09:52:33.380 DEBUG:teuthology.orchestra.run.smithi100:> dpkg -s linux-image-generic 2024-04-25T09:52:33.435 INFO:teuthology.orchestra.run.smithi060.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T09:52:33.436 INFO:teuthology.orchestra.run.smithi100.stdout:Package: linux-image-generic 2024-04-25T09:52:33.436 INFO:teuthology.orchestra.run.smithi100.stdout:Status: install ok installed 2024-04-25T09:52:33.436 INFO:teuthology.orchestra.run.smithi100.stdout:Priority: optional 2024-04-25T09:52:33.437 INFO:teuthology.orchestra.run.smithi100.stdout:Section: kernel 2024-04-25T09:52:33.437 INFO:teuthology.orchestra.run.smithi100.stdout:Installed-Size: 21 2024-04-25T09:52:33.437 INFO:teuthology.orchestra.run.smithi100.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T09:52:33.437 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: amd64 2024-04-25T09:52:33.437 INFO:teuthology.orchestra.run.smithi100.stdout:Source: linux-meta 2024-04-25T09:52:33.437 INFO:teuthology.orchestra.run.smithi100.stdout:Version: 5.15.0.105.102 2024-04-25T09:52:33.437 INFO:teuthology.orchestra.run.smithi100.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T09:52:33.437 INFO:teuthology.orchestra.run.smithi100.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T09:52:33.437 INFO:teuthology.orchestra.run.smithi100.stdout:Recommends: thermald 2024-04-25T09:52:33.437 INFO:teuthology.orchestra.run.smithi100.stdout:Description: Generic Linux kernel image 2024-04-25T09:52:33.437 INFO:teuthology.orchestra.run.smithi100.stdout: This package will always depend on the latest generic kernel image 2024-04-25T09:52:33.437 INFO:teuthology.orchestra.run.smithi100.stdout: available. 2024-04-25T09:52:33.438 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-04-25T09:52:33.442 INFO:teuthology.orchestra.run.smithi060.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T09:52:33.481 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.Sannp69War 2024-04-25T09:52:33.481 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Sannp69War 2024-04-25T09:52:33.537 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.Sannp69War 2024-04-25T09:52:33.600 INFO:teuthology.orchestra.run.smithi060.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T09:52:33.600 INFO:teuthology.orchestra.run.smithi060.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T09:52:33.600 INFO:teuthology.orchestra.run.smithi060.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T09:52:33.641 INFO:teuthology.orchestra.run.smithi060.stdout:done 2024-04-25T09:52:33.651 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.Sannp69War is 10KB 2024-04-25T09:52:33.663 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.Sannp69War 2024-04-25T09:52:33.668 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T09:52:33.721 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T09:52:33.721 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T09:52:33.721 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T09:52:33.788 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T09:52:33.789 DEBUG:teuthology.orchestra.run.smithi100:> sudo update-grub 2024-04-25T09:52:34.131 DEBUG:teuthology.orchestra.run.smithi060:> dpkg -s linux-image-generic 2024-04-25T09:52:34.154 INFO:teuthology.orchestra.run.smithi060.stdout:Package: linux-image-generic 2024-04-25T09:52:34.154 INFO:teuthology.orchestra.run.smithi060.stdout:Status: install ok installed 2024-04-25T09:52:34.154 INFO:teuthology.orchestra.run.smithi060.stdout:Priority: optional 2024-04-25T09:52:34.154 INFO:teuthology.orchestra.run.smithi060.stdout:Section: kernel 2024-04-25T09:52:34.154 INFO:teuthology.orchestra.run.smithi060.stdout:Installed-Size: 21 2024-04-25T09:52:34.154 INFO:teuthology.orchestra.run.smithi060.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T09:52:34.155 INFO:teuthology.orchestra.run.smithi060.stdout:Architecture: amd64 2024-04-25T09:52:34.155 INFO:teuthology.orchestra.run.smithi060.stdout:Source: linux-meta 2024-04-25T09:52:34.155 INFO:teuthology.orchestra.run.smithi060.stdout:Version: 5.15.0.105.102 2024-04-25T09:52:34.155 INFO:teuthology.orchestra.run.smithi060.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T09:52:34.155 INFO:teuthology.orchestra.run.smithi060.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T09:52:34.155 INFO:teuthology.orchestra.run.smithi060.stdout:Recommends: thermald 2024-04-25T09:52:34.155 INFO:teuthology.orchestra.run.smithi060.stdout:Description: Generic Linux kernel image 2024-04-25T09:52:34.155 INFO:teuthology.orchestra.run.smithi060.stdout: This package will always depend on the latest generic kernel image 2024-04-25T09:52:34.155 INFO:teuthology.orchestra.run.smithi060.stdout: available. 2024-04-25T09:52:34.155 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T09:52:34.155 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T09:52:34.155 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T09:52:34.156 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-25T09:52:34.156 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi060.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T09:52:34.156 DEBUG:teuthology.orchestra.run.smithi060:> sudo apt-get clean 2024-04-25T09:52:34.381 DEBUG:teuthology.orchestra.run.smithi060:> sudo apt-get update 2024-04-25T09:52:34.648 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T09:52:34.648 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T09:52:34.719 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T09:52:34.800 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T09:52:34.927 INFO:teuthology.orchestra.run.smithi100.stderr:Sourcing file `/etc/default/grub' 2024-04-25T09:52:34.928 INFO:teuthology.orchestra.run.smithi100.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T09:52:34.931 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-04-25T09:52:35.099 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:52:35.105 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:52:35.319 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:52:35.324 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T09:52:35.366 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T09:52:35.371 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T09:52:35.499 INFO:teuthology.orchestra.run.smithi100.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T09:52:35.499 INFO:teuthology.orchestra.run.smithi100.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T09:52:35.499 INFO:teuthology.orchestra.run.smithi100.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T09:52:35.511 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-04-25T09:52:35.530 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-04-25T09:52:36.085 INFO:teuthology.orchestra.run.smithi060.stdout:Reading package lists... 2024-04-25T09:52:36.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T09:52:36.170 INFO:teuthology.orchestra.run.smithi060.stdout:Reading package lists... 2024-04-25T09:52:36.375 INFO:teuthology.orchestra.run.smithi060.stdout:Building dependency tree... 2024-04-25T09:52:36.376 INFO:teuthology.orchestra.run.smithi060.stdout:Reading state information... 2024-04-25T09:52:36.562 INFO:teuthology.orchestra.run.smithi060.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T09:52:36.562 INFO:teuthology.orchestra.run.smithi060.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:52:36.562 INFO:teuthology.orchestra.run.smithi060.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:52:36.562 INFO:teuthology.orchestra.run.smithi060.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:52:36.562 INFO:teuthology.orchestra.run.smithi060.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:52:36.563 INFO:teuthology.orchestra.run.smithi060.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T09:52:36.563 INFO:teuthology.orchestra.run.smithi060.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T09:52:36.563 INFO:teuthology.orchestra.run.smithi060.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:52:36.606 INFO:teuthology.orchestra.run.smithi060.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T09:52:36.608 DEBUG:teuthology.orchestra.run.smithi060:> dpkg -s linux-image-generic 2024-04-25T09:52:36.621 INFO:teuthology.orchestra.run.smithi060.stdout:Package: linux-image-generic 2024-04-25T09:52:36.621 INFO:teuthology.orchestra.run.smithi060.stdout:Status: install ok installed 2024-04-25T09:52:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:Priority: optional 2024-04-25T09:52:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:Section: kernel 2024-04-25T09:52:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:Installed-Size: 21 2024-04-25T09:52:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T09:52:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:Architecture: amd64 2024-04-25T09:52:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:Source: linux-meta 2024-04-25T09:52:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:Version: 5.15.0.105.102 2024-04-25T09:52:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T09:52:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T09:52:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:Recommends: thermald 2024-04-25T09:52:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:Description: Generic Linux kernel image 2024-04-25T09:52:36.622 INFO:teuthology.orchestra.run.smithi060.stdout: This package will always depend on the latest generic kernel image 2024-04-25T09:52:36.622 INFO:teuthology.orchestra.run.smithi060.stdout: available. 2024-04-25T09:52:36.623 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2024-04-25T09:52:36.668 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.L72VMdvFRn 2024-04-25T09:52:36.669 DEBUG:teuthology.orchestra.run.smithi060:> sudo cp /boot/grub/grub.cfg /tmp/tmp.L72VMdvFRn 2024-04-25T09:52:36.723 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.L72VMdvFRn 2024-04-25T09:52:36.837 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.L72VMdvFRn is 10KB 2024-04-25T09:52:36.850 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.L72VMdvFRn 2024-04-25T09:52:36.856 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T09:52:36.908 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T09:52:36.908 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T09:52:36.908 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T09:52:37.000 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T09:52:37.000 DEBUG:teuthology.orchestra.run.smithi060:> sudo update-grub 2024-04-25T09:52:38.423 INFO:teuthology.orchestra.run.smithi060.stderr:Sourcing file `/etc/default/grub' 2024-04-25T09:52:38.427 INFO:teuthology.orchestra.run.smithi060.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T09:52:38.438 INFO:teuthology.orchestra.run.smithi060.stderr:Generating grub configuration file ... 2024-04-25T09:52:38.624 INFO:teuthology.orchestra.run.smithi060.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:52:38.629 INFO:teuthology.orchestra.run.smithi060.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:52:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:52:38.906 INFO:teuthology.orchestra.run.smithi060.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T09:52:38.951 INFO:teuthology.orchestra.run.smithi060.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T09:52:38.957 INFO:teuthology.orchestra.run.smithi060.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T09:52:39.079 INFO:teuthology.orchestra.run.smithi060.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T09:52:39.079 INFO:teuthology.orchestra.run.smithi060.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T09:52:39.079 INFO:teuthology.orchestra.run.smithi060.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T09:52:39.095 INFO:teuthology.orchestra.run.smithi060.stderr:done 2024-04-25T09:52:39.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo shutdown -r now 2024-04-25T09:53:05.559 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-25T09:53:05.560 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-25T09:53:05.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:53:09.158 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-25T09:53:09.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-04-25T09:53:09.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:54:05.598 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T09:54:09.162 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T09:54:14.604 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-25T09:54:14.605 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:54:15.207 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-04-25T09:54:16.165 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-25T09:54:16.165 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-25T09:54:16.166 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-04-25T09:54:16.171 INFO:teuthology.orchestra.run.smithi100.stdout:5.15.0-105-generic 2024-04-25T09:54:16.171 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T09:54:16.171 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T09:54:16.171 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T09:54:17.172 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T09:54:17.172 DEBUG:teuthology.orchestra.run.smithi100:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T09:54:17.293 INFO:teuthology.orchestra.run.smithi100.stdout:ttyS1 2024-04-25T09:54:17.317 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:18.163 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-04-25T09:54:18.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:54:18.721 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-04-25T09:54:19.548 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-04-25T09:54:19.548 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-25T09:54:19.548 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-04-25T09:54:19.595 INFO:teuthology.orchestra.run.smithi060.stdout:5.15.0-105-generic 2024-04-25T09:54:19.595 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T09:54:19.596 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T09:54:19.596 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T09:54:20.597 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T09:54:20.597 DEBUG:teuthology.orchestra.run.smithi060:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T09:54:20.925 INFO:teuthology.orchestra.run.smithi060.stdout:ttyS1 2024-04-25T09:54:20.941 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:20.941 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T09:54:20.948 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T09:54:20.948 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:54:20.951 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:54:20.957 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T09:54:20.973 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T09:54:20.982 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T09:54:20.983 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:54:21.000 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:54:21.125 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T09:54:21.132 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T09:54:21.132 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:54:21.134 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:54:21.154 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:54:21.161 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:54:21.254 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:54:21.264 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:54:21.266 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T09:54:21.274 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T09:54:21.274 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:54:21.276 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:54:21.306 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T09:54:21.316 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T09:54:21.316 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:54:21.332 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:54:21.351 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:54:21.387 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:54:21.435 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T09:54:21.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:54:21.496 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:54:21.501 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:54:21.548 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T09:54:21.548 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:54:21.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo service rsyslog restart 2024-04-25T09:54:21.612 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-04-25T09:54:21.683 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T09:54:21.691 INFO:teuthology.task.internal:Starting timer... 2024-04-25T09:54:21.691 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T09:54:21.702 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T09:54:21.709 DEBUG:teuthology.task.selinux:Excluding smithi060: OS 'ubuntu' does not support SELinux 2024-04-25T09:54:21.710 DEBUG:teuthology.task.selinux:Excluding smithi100: OS 'ubuntu' does not support SELinux 2024-04-25T09:54:21.710 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T09:54:21.710 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T09:54:21.710 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T09:54:21.710 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T09:54:21.719 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T09:54:21.725 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T09:54:21.846 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T09:54:21.863 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T09:54:21.865 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi060.front.sepia.ceph.com,smithi100.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T10:01:25.796 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi060.front.sepia.ceph.com'), Remote(name='ubuntu@smithi100.front.sepia.ceph.com')] 2024-04-25T10:01:25.797 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-04-25T10:01:25.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:01:25.883 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-04-25T10:01:25.959 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-04-25T10:01:25.959 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-25T10:01:25.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:01:26.038 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-04-25T10:01:26.114 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-25T10:01:26.115 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T10:01:26.126 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T10:01:26.127 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:01:26.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:01:26.129 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:01:26.130 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:01:26.158 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T10:01:26.159 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: Command line: ntpd -gq 2024-04-25T10:01:26.159 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: ---------------------------------------------------- 2024-04-25T10:01:26.159 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T10:01:26.159 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T10:01:26.159 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: corporation. Support and training for ntp-4 are 2024-04-25T10:01:26.159 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: available at https://www.nwtime.org/support 2024-04-25T10:01:26.159 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: ---------------------------------------------------- 2024-04-25T10:01:26.160 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: proto: precision = 0.046 usec (-24) 2024-04-25T10:01:26.160 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: basedate set to 2022-02-04 2024-04-25T10:01:26.160 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: gps base set to 2022-02-06 (week 2196) 2024-04-25T10:01:26.160 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T10:01:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T10:01:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T10:01:26.161 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T10:01:26.161 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T10:01:26.161 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T10:01:26.162 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T10:01:26.162 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: Listen normally on 3 ens1f0 172.21.15.60:123 2024-04-25T10:01:26.162 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: Listen normally on 4 lo [::1]:123 2024-04-25T10:01:26.162 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1560%4]:123 2024-04-25T10:01:26.162 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:26 ntpd[17157]: Listening on routing socket on fd #22 for interface updates 2024-04-25T10:01:26.178 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T10:01:26.179 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: Command line: ntpd -gq 2024-04-25T10:01:26.179 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: ---------------------------------------------------- 2024-04-25T10:01:26.179 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T10:01:26.179 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T10:01:26.179 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: corporation. Support and training for ntp-4 are 2024-04-25T10:01:26.179 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: available at https://www.nwtime.org/support 2024-04-25T10:01:26.179 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: ---------------------------------------------------- 2024-04-25T10:01:26.179 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: proto: precision = 0.075 usec (-24) 2024-04-25T10:01:26.180 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: basedate set to 2022-02-04 2024-04-25T10:01:26.180 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: gps base set to 2022-02-06 (week 2196) 2024-04-25T10:01:26.180 INFO:teuthology.orchestra.run.smithi100.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T10:01:26.180 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T10:01:26.180 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T10:01:26.181 INFO:teuthology.orchestra.run.smithi100.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T10:01:26.181 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T10:01:26.181 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T10:01:26.182 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T10:01:26.182 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: Listen normally on 3 enp3s0f1 172.21.15.100:123 2024-04-25T10:01:26.182 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: Listen normally on 4 lo [::1]:123 2024-04-25T10:01:26.182 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d733%5]:123 2024-04-25T10:01:26.182 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:26 ntpd[17214]: Listening on routing socket on fd #22 for interface updates 2024-04-25T10:01:27.159 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:27 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:27.159 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:27 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:27.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:27 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:27.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:27 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:27.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:27 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:27.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:27 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:27.179 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:27 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:27.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:27 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:27.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:27 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:27.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:27 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:27.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:27 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:27.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:27 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:27.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:27 ntpd[17214]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:27.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:27 ntpd[17214]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:28.159 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:28 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:28.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:28 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:28.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:28 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:28.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:28 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:28.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:28 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:28.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:28 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:28 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:28 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:28 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:28.179 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:28 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:28 ntpd[17214]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:28 ntpd[17214]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:29 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:29 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:29 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:29 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:29 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:29 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:29 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:29 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:29.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:29 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:29.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:29 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:29.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:29 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:29.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:29 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:29.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:29 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:29.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:29 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:29.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:29 ntpd[17214]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:29.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:29 ntpd[17214]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:29.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:29 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:29.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:29 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:29.181 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:29 ntpd[17214]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:29.181 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:29 ntpd[17214]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:30 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:30 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:30 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:30 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:30 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:30 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:30.161 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:30 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:30.161 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:30 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:30.161 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:30 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:30.161 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:30 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:30.161 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:30 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:30.161 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:30 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:30.179 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:30 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:30.179 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:30 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:30.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:30 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:30.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:30 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:30.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:30 ntpd[17214]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:30.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:30 ntpd[17214]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:31.159 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:31 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:31.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:31 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:31.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:31 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:31.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:31 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:31.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:31 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:31.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:31 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:31.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:31 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:31.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:31 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:31.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:31 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:31.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:31 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:31.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:31 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:31.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:31 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:31.181 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:31 ntpd[17214]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:31.181 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:31 ntpd[17214]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:31.181 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:31 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:31.181 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:31 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:31.181 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:31 ntpd[17214]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:31.181 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:31 ntpd[17214]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:32.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:32 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:32.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:32 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:32.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:32 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:32.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:32 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:32.161 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:32 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:32.161 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:32 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:32.161 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:32 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:32.161 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:32 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:32.161 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:32 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:32.161 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:32 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:32.161 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:32 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:32.161 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:32 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:32.179 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:32 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:32.179 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:32 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:32.179 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:32 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:32.179 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:32 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:32.179 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:32 ntpd[17214]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:32.179 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:32 ntpd[17214]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:33.159 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:33 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:33.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:33 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:33.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:33 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:33.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:33 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:33.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:33 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:33.160 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:33 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:33.160 INFO:teuthology.orchestra.run.smithi060.stdout:25 Apr 10:01:33 ntpd[17157]: ntpd: time slew -0.001009 s 2024-04-25T10:01:33.160 INFO:teuthology.orchestra.run.smithi060.stdout:ntpd: time slew -0.001009s 2024-04-25T10:01:33.161 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:33 ntpd[17157]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T10:01:33.161 INFO:teuthology.orchestra.run.smithi060.stderr:25 Apr 10:01:33 ntpd[17157]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T10:01:33.179 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:33 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:33 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:33 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:33 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:33 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:33 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:33 ntpd[17214]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:33 ntpd[17214]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:33 ntpd[17214]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:01:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:33 ntpd[17214]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:01:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:33 ntpd[17214]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:01:33.180 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:33 ntpd[17214]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:01:33.181 INFO:teuthology.orchestra.run.smithi100.stdout:25 Apr 10:01:33 ntpd[17214]: ntpd: time slew -0.003567 s 2024-04-25T10:01:33.181 INFO:teuthology.orchestra.run.smithi100.stdout:ntpd: time slew -0.003567s 2024-04-25T10:01:33.181 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:33 ntpd[17214]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T10:01:33.181 INFO:teuthology.orchestra.run.smithi100.stderr:25 Apr 10:01:33 ntpd[17214]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T10:01:33.215 INFO:teuthology.orchestra.run.smithi060.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T10:01:33.215 INFO:teuthology.orchestra.run.smithi060.stdout:============================================================================== 2024-04-25T10:01:33.215 INFO:teuthology.orchestra.run.smithi060.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:01:33.215 INFO:teuthology.orchestra.run.smithi060.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:01:33.215 INFO:teuthology.orchestra.run.smithi060.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:01:33.215 INFO:teuthology.orchestra.run.smithi060.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:01:33.241 INFO:teuthology.orchestra.run.smithi100.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T10:01:33.242 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================== 2024-04-25T10:01:33.242 INFO:teuthology.orchestra.run.smithi100.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:01:33.242 INFO:teuthology.orchestra.run.smithi100.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:01:33.242 INFO:teuthology.orchestra.run.smithi100.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:01:33.242 INFO:teuthology.orchestra.run.smithi100.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:01:33.242 INFO:teuthology.run_tasks:Running task install... 2024-04-25T10:01:33.252 DEBUG:teuthology.task.install:project ceph 2024-04-25T10:01:33.252 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T10:01:33.253 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T10:01:33.253 INFO:teuthology.task.install:Using flavor: default 2024-04-25T10:01:33.259 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T10:01:33.259 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T10:01:33.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo apt-key list | grep Ceph 2024-04-25T10:01:33.260 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-key list | grep Ceph 2024-04-25T10:01:33.322 INFO:teuthology.orchestra.run.smithi060.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T10:01:33.339 INFO:teuthology.orchestra.run.smithi100.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T10:01:33.354 INFO:teuthology.orchestra.run.smithi060.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T10:01:33.354 INFO:teuthology.orchestra.run.smithi060.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T10:01:33.355 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T10:01:33.355 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T10:01:33.355 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:01:33.364 INFO:teuthology.orchestra.run.smithi100.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T10:01:33.364 INFO:teuthology.orchestra.run.smithi100.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T10:01:33.364 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T10:01:33.364 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T10:01:33.365 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:01:33.514 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-25T10:01:33.514 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:01:33.533 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-25T10:01:33.534 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:01:33.670 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:01:33.670 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T10:01:33.684 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get update 2024-04-25T10:01:33.691 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:01:33.691 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T10:01:33.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo apt-get update 2024-04-25T10:01:33.854 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T10:01:33.869 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T10:01:33.882 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T10:01:33.907 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T10:01:33.929 INFO:teuthology.orchestra.run.smithi060.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T10:01:33.932 INFO:teuthology.orchestra.run.smithi100.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T10:01:33.950 INFO:teuthology.orchestra.run.smithi060.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T10:01:33.962 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T10:01:33.968 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T10:01:33.972 INFO:teuthology.orchestra.run.smithi060.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T10:01:33.987 INFO:teuthology.orchestra.run.smithi100.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T10:01:33.993 INFO:teuthology.orchestra.run.smithi060.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T10:01:34.001 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T10:01:34.011 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-25T10:01:34.022 INFO:teuthology.orchestra.run.smithi060.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-25T10:01:34.041 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T10:01:34.076 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T10:01:34.158 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T10:01:34.197 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 28.9 kB in 0s (85.3 kB/s) 2024-04-25T10:01:34.301 INFO:teuthology.orchestra.run.smithi060.stdout:Fetched 28.9 kB in 0s (69.7 kB/s) 2024-04-25T10:01:35.176 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-25T10:01:35.191 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:01:35.251 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-25T10:01:35.280 INFO:teuthology.orchestra.run.smithi060.stdout:Reading package lists... 2024-04-25T10:01:35.297 DEBUG:teuthology.orchestra.run.smithi060:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:01:35.349 INFO:teuthology.orchestra.run.smithi060.stdout:Reading package lists... 2024-04-25T10:01:35.452 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-04-25T10:01:35.452 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-04-25T10:01:35.532 INFO:teuthology.orchestra.run.smithi060.stdout:Building dependency tree... 2024-04-25T10:01:35.532 INFO:teuthology.orchestra.run.smithi060.stdout:Reading state information... 2024-04-25T10:01:35.645 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T10:01:35.645 INFO:teuthology.orchestra.run.smithi100.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T10:01:35.645 INFO:teuthology.orchestra.run.smithi100.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T10:01:35.645 INFO:teuthology.orchestra.run.smithi100.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T10:01:35.645 INFO:teuthology.orchestra.run.smithi100.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T10:01:35.646 INFO:teuthology.orchestra.run.smithi100.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T10:01:35.646 INFO:teuthology.orchestra.run.smithi100.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T10:01:35.647 INFO:teuthology.orchestra.run.smithi100.stdout:The following additional packages will be installed: 2024-04-25T10:01:35.647 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T10:01:35.647 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T10:01:35.647 INFO:teuthology.orchestra.run.smithi100.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T10:01:35.647 INFO:teuthology.orchestra.run.smithi100.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T10:01:35.648 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T10:01:35.648 INFO:teuthology.orchestra.run.smithi100.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T10:01:35.648 INFO:teuthology.orchestra.run.smithi100.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T10:01:35.648 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T10:01:35.648 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T10:01:35.648 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T10:01:35.649 INFO:teuthology.orchestra.run.smithi100.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T10:01:35.649 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T10:01:35.649 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T10:01:35.649 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T10:01:35.649 INFO:teuthology.orchestra.run.smithi100.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T10:01:35.649 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T10:01:35.649 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T10:01:35.649 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T10:01:35.649 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T10:01:35.649 INFO:teuthology.orchestra.run.smithi100.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T10:01:35.649 INFO:teuthology.orchestra.run.smithi100.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T10:01:35.649 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T10:01:35.649 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T10:01:35.650 INFO:teuthology.orchestra.run.smithi100.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T10:01:35.650 INFO:teuthology.orchestra.run.smithi100.stdout:Suggested packages: 2024-04-25T10:01:35.650 INFO:teuthology.orchestra.run.smithi100.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T10:01:35.650 INFO:teuthology.orchestra.run.smithi100.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T10:01:35.650 INFO:teuthology.orchestra.run.smithi100.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T10:01:35.650 INFO:teuthology.orchestra.run.smithi100.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T10:01:35.650 INFO:teuthology.orchestra.run.smithi100.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T10:01:35.650 INFO:teuthology.orchestra.run.smithi100.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T10:01:35.650 INFO:teuthology.orchestra.run.smithi100.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T10:01:35.651 INFO:teuthology.orchestra.run.smithi100.stdout:Recommended packages: 2024-04-25T10:01:35.651 INFO:teuthology.orchestra.run.smithi100.stdout: btrfs-tools 2024-04-25T10:01:35.714 INFO:teuthology.orchestra.run.smithi060.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T10:01:35.714 INFO:teuthology.orchestra.run.smithi060.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T10:01:35.714 INFO:teuthology.orchestra.run.smithi060.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T10:01:35.714 INFO:teuthology.orchestra.run.smithi060.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T10:01:35.714 INFO:teuthology.orchestra.run.smithi060.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T10:01:35.714 INFO:teuthology.orchestra.run.smithi060.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T10:01:35.715 INFO:teuthology.orchestra.run.smithi060.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T10:01:35.716 INFO:teuthology.orchestra.run.smithi060.stdout:The following additional packages will be installed: 2024-04-25T10:01:35.716 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T10:01:35.716 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T10:01:35.716 INFO:teuthology.orchestra.run.smithi060.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T10:01:35.716 INFO:teuthology.orchestra.run.smithi060.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T10:01:35.717 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T10:01:35.717 INFO:teuthology.orchestra.run.smithi060.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T10:01:35.717 INFO:teuthology.orchestra.run.smithi060.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T10:01:35.717 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T10:01:35.717 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T10:01:35.717 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T10:01:35.717 INFO:teuthology.orchestra.run.smithi060.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T10:01:35.717 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T10:01:35.717 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T10:01:35.717 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T10:01:35.717 INFO:teuthology.orchestra.run.smithi060.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T10:01:35.718 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T10:01:35.718 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T10:01:35.718 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T10:01:35.718 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T10:01:35.718 INFO:teuthology.orchestra.run.smithi060.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T10:01:35.718 INFO:teuthology.orchestra.run.smithi060.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T10:01:35.718 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T10:01:35.718 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T10:01:35.718 INFO:teuthology.orchestra.run.smithi060.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T10:01:35.719 INFO:teuthology.orchestra.run.smithi060.stdout:Suggested packages: 2024-04-25T10:01:35.719 INFO:teuthology.orchestra.run.smithi060.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T10:01:35.719 INFO:teuthology.orchestra.run.smithi060.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T10:01:35.719 INFO:teuthology.orchestra.run.smithi060.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T10:01:35.719 INFO:teuthology.orchestra.run.smithi060.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T10:01:35.719 INFO:teuthology.orchestra.run.smithi060.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T10:01:35.719 INFO:teuthology.orchestra.run.smithi060.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T10:01:35.719 INFO:teuthology.orchestra.run.smithi060.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T10:01:35.719 INFO:teuthology.orchestra.run.smithi060.stdout:Recommended packages: 2024-04-25T10:01:35.719 INFO:teuthology.orchestra.run.smithi060.stdout: btrfs-tools 2024-04-25T10:01:35.756 INFO:teuthology.orchestra.run.smithi100.stdout:The following NEW packages will be installed: 2024-04-25T10:01:35.756 INFO:teuthology.orchestra.run.smithi100.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T10:01:35.756 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T10:01:35.756 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T10:01:35.756 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T10:01:35.758 INFO:teuthology.orchestra.run.smithi100.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T10:01:35.758 INFO:teuthology.orchestra.run.smithi100.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T10:01:35.758 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T10:01:35.758 INFO:teuthology.orchestra.run.smithi100.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T10:01:35.758 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T10:01:35.758 INFO:teuthology.orchestra.run.smithi100.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T10:01:35.758 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T10:01:35.758 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T10:01:35.758 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T10:01:35.758 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T10:01:35.759 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T10:01:35.759 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T10:01:35.759 INFO:teuthology.orchestra.run.smithi100.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T10:01:35.759 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T10:01:35.759 INFO:teuthology.orchestra.run.smithi100.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T10:01:35.759 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T10:01:35.759 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T10:01:35.759 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T10:01:35.759 INFO:teuthology.orchestra.run.smithi100.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T10:01:35.759 INFO:teuthology.orchestra.run.smithi100.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T10:01:35.759 INFO:teuthology.orchestra.run.smithi100.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T10:01:35.760 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T10:01:35.760 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T10:01:35.760 INFO:teuthology.orchestra.run.smithi100.stdout: socat unzip xmlstarlet zip 2024-04-25T10:01:35.823 INFO:teuthology.orchestra.run.smithi100.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T10:01:35.823 INFO:teuthology.orchestra.run.smithi100.stdout:Need to get 176 MB of archives. 2024-04-25T10:01:35.823 INFO:teuthology.orchestra.run.smithi100.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-25T10:01:35.823 INFO:teuthology.orchestra.run.smithi100.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T10:01:35.825 INFO:teuthology.orchestra.run.smithi060.stdout:The following NEW packages will be installed: 2024-04-25T10:01:35.825 INFO:teuthology.orchestra.run.smithi060.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T10:01:35.825 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T10:01:35.825 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T10:01:35.825 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T10:01:35.827 INFO:teuthology.orchestra.run.smithi060.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T10:01:35.827 INFO:teuthology.orchestra.run.smithi060.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T10:01:35.827 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T10:01:35.827 INFO:teuthology.orchestra.run.smithi060.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T10:01:35.827 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T10:01:35.828 INFO:teuthology.orchestra.run.smithi060.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T10:01:35.828 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T10:01:35.828 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T10:01:35.828 INFO:teuthology.orchestra.run.smithi060.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T10:01:35.828 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T10:01:35.828 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T10:01:35.828 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T10:01:35.828 INFO:teuthology.orchestra.run.smithi060.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T10:01:35.828 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T10:01:35.828 INFO:teuthology.orchestra.run.smithi060.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T10:01:35.828 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T10:01:35.828 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T10:01:35.828 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T10:01:35.828 INFO:teuthology.orchestra.run.smithi060.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T10:01:35.829 INFO:teuthology.orchestra.run.smithi060.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T10:01:35.829 INFO:teuthology.orchestra.run.smithi060.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T10:01:35.829 INFO:teuthology.orchestra.run.smithi060.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T10:01:35.829 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T10:01:35.829 INFO:teuthology.orchestra.run.smithi060.stdout: socat unzip xmlstarlet zip 2024-04-25T10:01:35.844 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T10:01:35.857 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T10:01:35.894 INFO:teuthology.orchestra.run.smithi060.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T10:01:35.895 INFO:teuthology.orchestra.run.smithi060.stdout:Need to get 176 MB of archives. 2024-04-25T10:01:35.895 INFO:teuthology.orchestra.run.smithi060.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-25T10:01:35.895 INFO:teuthology.orchestra.run.smithi060.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T10:01:35.899 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-25T10:01:35.906 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T10:01:35.916 INFO:teuthology.orchestra.run.smithi060.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T10:01:35.927 INFO:teuthology.orchestra.run.smithi060.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T10:01:35.931 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T10:01:35.935 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T10:01:35.949 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T10:01:35.973 INFO:teuthology.orchestra.run.smithi060.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-25T10:01:35.977 INFO:teuthology.orchestra.run.smithi060.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T10:01:36.001 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T10:01:36.003 INFO:teuthology.orchestra.run.smithi060.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T10:01:36.006 INFO:teuthology.orchestra.run.smithi100.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T10:01:36.007 INFO:teuthology.orchestra.run.smithi060.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T10:01:36.017 INFO:teuthology.orchestra.run.smithi100.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T10:01:36.021 INFO:teuthology.orchestra.run.smithi100.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T10:01:36.022 INFO:teuthology.orchestra.run.smithi060.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T10:01:36.022 INFO:teuthology.orchestra.run.smithi100.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T10:01:36.030 INFO:teuthology.orchestra.run.smithi100.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T10:01:36.033 INFO:teuthology.orchestra.run.smithi100.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T10:01:36.035 INFO:teuthology.orchestra.run.smithi100.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T10:01:36.036 INFO:teuthology.orchestra.run.smithi100.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T10:01:36.036 INFO:teuthology.orchestra.run.smithi100.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T10:01:36.048 INFO:teuthology.orchestra.run.smithi100.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T10:01:36.050 INFO:teuthology.orchestra.run.smithi100.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T10:01:36.052 INFO:teuthology.orchestra.run.smithi100.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T10:01:36.053 INFO:teuthology.orchestra.run.smithi100.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T10:01:36.078 INFO:teuthology.orchestra.run.smithi100.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T10:01:36.080 INFO:teuthology.orchestra.run.smithi060.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T10:01:36.085 INFO:teuthology.orchestra.run.smithi100.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T10:01:36.086 INFO:teuthology.orchestra.run.smithi060.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T10:01:36.086 INFO:teuthology.orchestra.run.smithi100.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T10:01:36.086 INFO:teuthology.orchestra.run.smithi100.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T10:01:36.087 INFO:teuthology.orchestra.run.smithi100.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T10:01:36.088 INFO:teuthology.orchestra.run.smithi100.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T10:01:36.098 INFO:teuthology.orchestra.run.smithi060.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T10:01:36.099 INFO:teuthology.orchestra.run.smithi100.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T10:01:36.099 INFO:teuthology.orchestra.run.smithi100.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T10:01:36.102 INFO:teuthology.orchestra.run.smithi060.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T10:01:36.103 INFO:teuthology.orchestra.run.smithi100.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T10:01:36.103 INFO:teuthology.orchestra.run.smithi060.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T10:01:36.104 INFO:teuthology.orchestra.run.smithi100.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T10:01:36.112 INFO:teuthology.orchestra.run.smithi060.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T10:01:36.114 INFO:teuthology.orchestra.run.smithi060.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T10:01:36.114 INFO:teuthology.orchestra.run.smithi100.stdout:Get:33 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-25T10:01:36.116 INFO:teuthology.orchestra.run.smithi060.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T10:01:36.118 INFO:teuthology.orchestra.run.smithi060.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T10:01:36.118 INFO:teuthology.orchestra.run.smithi060.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T10:01:36.119 INFO:teuthology.orchestra.run.smithi100.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T10:01:36.120 INFO:teuthology.orchestra.run.smithi100.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T10:01:36.121 INFO:teuthology.orchestra.run.smithi100.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T10:01:36.122 INFO:teuthology.orchestra.run.smithi100.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T10:01:36.123 INFO:teuthology.orchestra.run.smithi100.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T10:01:36.125 INFO:teuthology.orchestra.run.smithi100.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T10:01:36.127 INFO:teuthology.orchestra.run.smithi060.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T10:01:36.129 INFO:teuthology.orchestra.run.smithi060.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T10:01:36.131 INFO:teuthology.orchestra.run.smithi060.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T10:01:36.140 INFO:teuthology.orchestra.run.smithi100.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T10:01:36.140 INFO:teuthology.orchestra.run.smithi100.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T10:01:36.147 INFO:teuthology.orchestra.run.smithi060.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T10:01:36.148 INFO:teuthology.orchestra.run.smithi060.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T10:01:36.149 INFO:teuthology.orchestra.run.smithi100.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T10:01:36.149 INFO:teuthology.orchestra.run.smithi060.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T10:01:36.150 INFO:teuthology.orchestra.run.smithi100.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T10:01:36.150 INFO:teuthology.orchestra.run.smithi060.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T10:01:36.150 INFO:teuthology.orchestra.run.smithi060.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T10:01:36.150 INFO:teuthology.orchestra.run.smithi060.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T10:01:36.151 INFO:teuthology.orchestra.run.smithi060.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T10:01:36.160 INFO:teuthology.orchestra.run.smithi100.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T10:01:36.161 INFO:teuthology.orchestra.run.smithi100.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T10:01:36.163 INFO:teuthology.orchestra.run.smithi100.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T10:01:36.166 INFO:teuthology.orchestra.run.smithi100.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T10:01:36.168 INFO:teuthology.orchestra.run.smithi100.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T10:01:36.168 INFO:teuthology.orchestra.run.smithi060.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T10:01:36.169 INFO:teuthology.orchestra.run.smithi060.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T10:01:36.170 INFO:teuthology.orchestra.run.smithi100.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T10:01:36.171 INFO:teuthology.orchestra.run.smithi060.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T10:01:36.181 INFO:teuthology.orchestra.run.smithi100.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T10:01:36.185 INFO:teuthology.orchestra.run.smithi100.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T10:01:36.186 INFO:teuthology.orchestra.run.smithi100.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T10:01:36.187 INFO:teuthology.orchestra.run.smithi100.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T10:01:36.188 INFO:teuthology.orchestra.run.smithi060.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T10:01:36.189 INFO:teuthology.orchestra.run.smithi060.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T10:01:36.189 INFO:teuthology.orchestra.run.smithi060.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T10:01:36.190 INFO:teuthology.orchestra.run.smithi060.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T10:01:36.191 INFO:teuthology.orchestra.run.smithi060.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T10:01:36.191 INFO:teuthology.orchestra.run.smithi060.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T10:01:36.192 INFO:teuthology.orchestra.run.smithi060.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T10:01:36.202 INFO:teuthology.orchestra.run.smithi100.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T10:01:36.219 INFO:teuthology.orchestra.run.smithi060.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T10:01:36.220 INFO:teuthology.orchestra.run.smithi060.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T10:01:36.225 INFO:teuthology.orchestra.run.smithi100.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-25T10:01:36.231 INFO:teuthology.orchestra.run.smithi060.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T10:01:36.249 INFO:teuthology.orchestra.run.smithi100.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-25T10:01:36.254 INFO:teuthology.orchestra.run.smithi060.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T10:01:36.255 INFO:teuthology.orchestra.run.smithi060.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T10:01:36.257 INFO:teuthology.orchestra.run.smithi060.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T10:01:36.259 INFO:teuthology.orchestra.run.smithi060.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T10:01:36.261 INFO:teuthology.orchestra.run.smithi100.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-25T10:01:36.262 INFO:teuthology.orchestra.run.smithi100.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-25T10:01:36.263 INFO:teuthology.orchestra.run.smithi060.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T10:01:36.264 INFO:teuthology.orchestra.run.smithi060.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T10:01:36.267 INFO:teuthology.orchestra.run.smithi060.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T10:01:36.268 INFO:teuthology.orchestra.run.smithi100.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-25T10:01:36.270 INFO:teuthology.orchestra.run.smithi100.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T10:01:36.271 INFO:teuthology.orchestra.run.smithi100.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-25T10:01:36.271 INFO:teuthology.orchestra.run.smithi100.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T10:01:36.272 INFO:teuthology.orchestra.run.smithi100.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T10:01:36.273 INFO:teuthology.orchestra.run.smithi100.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T10:01:36.274 INFO:teuthology.orchestra.run.smithi100.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T10:01:36.285 INFO:teuthology.orchestra.run.smithi100.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-25T10:01:36.292 INFO:teuthology.orchestra.run.smithi060.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T10:01:36.296 INFO:teuthology.orchestra.run.smithi060.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T10:01:36.297 INFO:teuthology.orchestra.run.smithi060.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T10:01:36.300 INFO:teuthology.orchestra.run.smithi100.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T10:01:36.302 INFO:teuthology.orchestra.run.smithi100.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T10:01:36.303 INFO:teuthology.orchestra.run.smithi100.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T10:01:36.322 INFO:teuthology.orchestra.run.smithi100.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T10:01:36.322 INFO:teuthology.orchestra.run.smithi100.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T10:01:36.323 INFO:teuthology.orchestra.run.smithi060.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T10:01:36.328 INFO:teuthology.orchestra.run.smithi060.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T10:01:36.356 INFO:teuthology.orchestra.run.smithi100.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T10:01:36.357 INFO:teuthology.orchestra.run.smithi100.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T10:01:36.357 INFO:teuthology.orchestra.run.smithi100.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T10:01:36.358 INFO:teuthology.orchestra.run.smithi100.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T10:01:36.358 INFO:teuthology.orchestra.run.smithi100.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T10:01:36.359 INFO:teuthology.orchestra.run.smithi100.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T10:01:36.363 INFO:teuthology.orchestra.run.smithi100.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T10:01:36.393 INFO:teuthology.orchestra.run.smithi100.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T10:01:36.395 INFO:teuthology.orchestra.run.smithi100.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T10:01:36.395 INFO:teuthology.orchestra.run.smithi100.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T10:01:36.411 INFO:teuthology.orchestra.run.smithi060.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T10:01:36.411 INFO:teuthology.orchestra.run.smithi060.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T10:01:36.413 INFO:teuthology.orchestra.run.smithi060.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T10:01:36.414 INFO:teuthology.orchestra.run.smithi060.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T10:01:36.414 INFO:teuthology.orchestra.run.smithi060.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T10:01:36.430 INFO:teuthology.orchestra.run.smithi100.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T10:01:36.435 INFO:teuthology.orchestra.run.smithi100.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T10:01:36.435 INFO:teuthology.orchestra.run.smithi100.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T10:01:36.437 INFO:teuthology.orchestra.run.smithi100.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T10:01:36.438 INFO:teuthology.orchestra.run.smithi100.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T10:01:36.445 INFO:teuthology.orchestra.run.smithi100.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T10:01:36.446 INFO:teuthology.orchestra.run.smithi100.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T10:01:36.449 INFO:teuthology.orchestra.run.smithi060.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T10:01:36.452 INFO:teuthology.orchestra.run.smithi060.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T10:01:36.452 INFO:teuthology.orchestra.run.smithi060.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T10:01:36.475 INFO:teuthology.orchestra.run.smithi060.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T10:01:36.475 INFO:teuthology.orchestra.run.smithi060.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T10:01:36.476 INFO:teuthology.orchestra.run.smithi100.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T10:01:36.476 INFO:teuthology.orchestra.run.smithi100.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T10:01:36.478 INFO:teuthology.orchestra.run.smithi100.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T10:01:36.480 INFO:teuthology.orchestra.run.smithi060.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T10:01:36.481 INFO:teuthology.orchestra.run.smithi060.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T10:01:36.481 INFO:teuthology.orchestra.run.smithi060.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T10:01:36.482 INFO:teuthology.orchestra.run.smithi060.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T10:01:36.482 INFO:teuthology.orchestra.run.smithi060.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T10:01:36.483 INFO:teuthology.orchestra.run.smithi060.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T10:01:36.488 INFO:teuthology.orchestra.run.smithi060.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T10:01:36.496 INFO:teuthology.orchestra.run.smithi100.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T10:01:36.499 INFO:teuthology.orchestra.run.smithi100.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T10:01:36.501 INFO:teuthology.orchestra.run.smithi100.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T10:01:36.506 INFO:teuthology.orchestra.run.smithi060.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T10:01:36.508 INFO:teuthology.orchestra.run.smithi060.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T10:01:36.509 INFO:teuthology.orchestra.run.smithi060.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T10:01:36.513 INFO:teuthology.orchestra.run.smithi100.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T10:01:36.513 INFO:teuthology.orchestra.run.smithi100.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T10:01:36.514 INFO:teuthology.orchestra.run.smithi100.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T10:01:36.517 INFO:teuthology.orchestra.run.smithi100.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T10:01:36.519 INFO:teuthology.orchestra.run.smithi100.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T10:01:36.527 INFO:teuthology.orchestra.run.smithi100.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-25T10:01:36.530 INFO:teuthology.orchestra.run.smithi100.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-25T10:01:36.534 INFO:teuthology.orchestra.run.smithi060.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T10:01:36.538 INFO:teuthology.orchestra.run.smithi060.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T10:01:36.538 INFO:teuthology.orchestra.run.smithi060.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T10:01:36.539 INFO:teuthology.orchestra.run.smithi060.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T10:01:36.540 INFO:teuthology.orchestra.run.smithi060.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T10:01:36.544 INFO:teuthology.orchestra.run.smithi100.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-25T10:01:36.546 INFO:teuthology.orchestra.run.smithi060.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T10:01:36.547 INFO:teuthology.orchestra.run.smithi060.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T10:01:36.555 INFO:teuthology.orchestra.run.smithi060.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T10:01:36.555 INFO:teuthology.orchestra.run.smithi060.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T10:01:36.556 INFO:teuthology.orchestra.run.smithi060.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T10:01:36.576 INFO:teuthology.orchestra.run.smithi060.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T10:01:36.578 INFO:teuthology.orchestra.run.smithi060.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T10:01:36.579 INFO:teuthology.orchestra.run.smithi060.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T10:01:36.588 INFO:teuthology.orchestra.run.smithi060.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T10:01:36.588 INFO:teuthology.orchestra.run.smithi060.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T10:01:36.589 INFO:teuthology.orchestra.run.smithi060.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T10:01:36.591 INFO:teuthology.orchestra.run.smithi060.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T10:01:36.597 INFO:teuthology.orchestra.run.smithi060.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T10:01:37.002 INFO:teuthology.orchestra.run.smithi060.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-25T10:01:37.544 INFO:teuthology.orchestra.run.smithi100.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-25T10:01:37.654 INFO:teuthology.orchestra.run.smithi100.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-25T10:01:37.659 INFO:teuthology.orchestra.run.smithi100.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-25T10:01:37.663 INFO:teuthology.orchestra.run.smithi100.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-25T10:01:37.720 INFO:teuthology.orchestra.run.smithi100.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-25T10:01:37.735 INFO:teuthology.orchestra.run.smithi060.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-25T10:01:37.899 INFO:teuthology.orchestra.run.smithi060.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-25T10:01:37.963 INFO:teuthology.orchestra.run.smithi100.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-25T10:01:38.008 INFO:teuthology.orchestra.run.smithi060.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-25T10:01:38.010 INFO:teuthology.orchestra.run.smithi060.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-25T10:01:38.064 INFO:teuthology.orchestra.run.smithi060.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-25T10:01:38.066 INFO:teuthology.orchestra.run.smithi060.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-25T10:01:38.126 INFO:teuthology.orchestra.run.smithi060.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-25T10:01:38.920 INFO:teuthology.orchestra.run.smithi100.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-25T10:01:38.921 INFO:teuthology.orchestra.run.smithi100.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-25T10:01:38.938 INFO:teuthology.orchestra.run.smithi100.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-25T10:01:38.975 INFO:teuthology.orchestra.run.smithi100.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-25T10:01:38.986 INFO:teuthology.orchestra.run.smithi100.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-25T10:01:38.987 INFO:teuthology.orchestra.run.smithi100.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-25T10:01:39.054 INFO:teuthology.orchestra.run.smithi100.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-25T10:01:39.379 INFO:teuthology.orchestra.run.smithi100.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-25T10:01:39.380 INFO:teuthology.orchestra.run.smithi100.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-25T10:01:39.738 INFO:teuthology.orchestra.run.smithi060.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-25T10:01:39.740 INFO:teuthology.orchestra.run.smithi060.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-25T10:01:39.875 INFO:teuthology.orchestra.run.smithi060.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-25T10:01:41.510 INFO:teuthology.orchestra.run.smithi100.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-25T10:01:41.512 INFO:teuthology.orchestra.run.smithi100.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-25T10:01:41.517 INFO:teuthology.orchestra.run.smithi100.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-25T10:01:41.518 INFO:teuthology.orchestra.run.smithi100.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-25T10:01:41.519 INFO:teuthology.orchestra.run.smithi100.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-25T10:01:41.520 INFO:teuthology.orchestra.run.smithi100.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-25T10:01:41.880 INFO:teuthology.orchestra.run.smithi100.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-25T10:01:42.491 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 176 MB in 6s (28.8 MB/s) 2024-04-25T10:01:42.626 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T10:01:42.720 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T10:01:42.725 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T10:01:42.766 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T10:01:43.060 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T10:01:43.087 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T10:01:43.119 INFO:teuthology.orchestra.run.smithi060.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-25T10:01:43.127 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T10:01:43.405 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T10:01:43.431 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T10:01:43.443 INFO:teuthology.orchestra.run.smithi060.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-25T10:01:43.463 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T10:01:43.464 INFO:teuthology.orchestra.run.smithi060.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-25T10:01:43.479 INFO:teuthology.orchestra.run.smithi060.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-25T10:01:43.570 INFO:teuthology.orchestra.run.smithi060.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-25T10:01:43.774 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T10:01:43.801 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T10:01:43.875 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T10:01:43.992 INFO:teuthology.orchestra.run.smithi060.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-25T10:01:44.178 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T10:01:44.204 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T10:01:44.245 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T10:01:44.589 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T10:01:44.616 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T10:01:44.656 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T10:01:45.101 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T10:01:45.128 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:01:45.175 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:01:45.497 INFO:teuthology.orchestra.run.smithi060.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-25T10:01:45.497 INFO:teuthology.orchestra.run.smithi060.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-25T10:01:45.519 INFO:teuthology.orchestra.run.smithi060.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-25T10:01:45.596 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T10:01:45.622 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:01:45.632 INFO:teuthology.orchestra.run.smithi060.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-25T10:01:45.671 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:01:45.687 INFO:teuthology.orchestra.run.smithi060.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-25T10:01:45.690 INFO:teuthology.orchestra.run.smithi060.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-25T10:01:45.796 INFO:teuthology.orchestra.run.smithi060.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-25T10:01:46.016 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T10:01:46.041 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:01:46.082 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:01:46.266 INFO:teuthology.orchestra.run.smithi060.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-25T10:01:46.267 INFO:teuthology.orchestra.run.smithi060.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-25T10:01:46.393 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T10:01:46.408 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T10:01:46.443 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T10:01:46.746 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librados2. 2024-04-25T10:01:46.772 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:46.821 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:47.299 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librbd1. 2024-04-25T10:01:47.325 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:47.365 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:47.977 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T10:01:48.003 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:48.110 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:48.371 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rados. 2024-04-25T10:01:48.386 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:48.421 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:48.765 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T10:01:48.791 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:01:48.832 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:48.918 INFO:teuthology.orchestra.run.smithi060.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-25T10:01:48.920 INFO:teuthology.orchestra.run.smithi060.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-25T10:01:48.925 INFO:teuthology.orchestra.run.smithi060.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-25T10:01:48.926 INFO:teuthology.orchestra.run.smithi060.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-25T10:01:48.926 INFO:teuthology.orchestra.run.smithi060.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-25T10:01:48.927 INFO:teuthology.orchestra.run.smithi060.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-25T10:01:49.076 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T10:01:49.102 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:49.142 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:49.420 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T10:01:49.428 INFO:teuthology.orchestra.run.smithi060.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-25T10:01:49.446 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:01:49.486 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:49.815 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T10:01:49.841 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T10:01:49.881 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T10:01:50.070 INFO:teuthology.orchestra.run.smithi060.stdout:Fetched 176 MB in 14s (13.0 MB/s) 2024-04-25T10:01:50.142 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T10:01:50.169 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:50.208 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T10:01:50.209 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:50.264 INFO:teuthology.orchestra.run.smithi060.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T10:01:50.269 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T10:01:50.315 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T10:01:50.562 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T10:01:50.588 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T10:01:50.628 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T10:01:50.726 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T10:01:50.752 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T10:01:50.793 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T10:01:50.889 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T10:01:50.904 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T10:01:50.947 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T10:01:51.079 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T10:01:51.105 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T10:01:51.137 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T10:01:51.242 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T10:01:51.257 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T10:01:51.300 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T10:01:51.448 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T10:01:51.475 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T10:01:51.540 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T10:01:51.652 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package lua5.1. 2024-04-25T10:01:51.678 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T10:01:51.720 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T10:01:51.894 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T10:01:51.921 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T10:01:51.977 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T10:01:52.015 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package lua-any. 2024-04-25T10:01:52.041 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T10:01:52.073 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T10:01:52.292 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package zip. 2024-04-25T10:01:52.307 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T10:01:52.313 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T10:01:52.340 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T10:01:52.350 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T10:01:52.380 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T10:01:52.670 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package unzip. 2024-04-25T10:01:52.696 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T10:01:52.737 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T10:01:52.825 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T10:01:52.852 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:01:52.900 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:01:53.132 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package luarocks. 2024-04-25T10:01:53.159 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T10:01:53.216 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T10:01:53.237 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T10:01:53.260 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:01:53.303 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:01:53.565 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librgw2. 2024-04-25T10:01:53.592 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:53.623 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:53.624 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T10:01:53.650 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:01:53.691 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:01:54.051 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T10:01:54.078 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T10:01:54.118 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T10:01:54.134 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T10:01:54.149 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:54.192 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:54.396 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package librados2. 2024-04-25T10:01:54.423 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:54.454 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:54.537 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T10:01:54.563 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T10:01:54.595 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T10:01:54.915 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T10:01:54.941 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:54.981 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:55.049 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package librbd1. 2024-04-25T10:01:55.075 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:55.118 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:55.426 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-common. 2024-04-25T10:01:55.453 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:55.493 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:55.594 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T10:01:55.620 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:55.660 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:55.921 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-rados. 2024-04-25T10:01:55.937 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:55.971 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:56.257 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T10:01:56.283 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:01:56.323 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:56.585 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T10:01:56.611 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:56.651 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:56.898 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-base. 2024-04-25T10:01:56.924 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:56.962 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T10:01:56.982 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:56.989 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:01:57.029 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:57.340 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T10:01:57.367 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T10:01:57.407 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T10:01:57.611 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T10:01:57.626 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T10:01:57.637 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T10:01:57.642 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:01:57.677 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T10:01:57.685 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:01:57.996 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T10:01:58.022 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T10:01:58.037 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T10:01:58.064 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T10:01:58.071 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T10:01:58.104 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T10:01:58.390 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T10:01:58.406 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T10:01:58.448 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T10:01:58.616 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T10:01:58.643 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T10:01:58.683 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T10:01:58.743 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T10:01:58.758 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T10:01:58.801 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T10:01:59.036 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T10:01:59.062 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T10:01:59.110 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T10:01:59.138 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package lua5.1. 2024-04-25T10:01:59.164 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T10:01:59.204 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T10:01:59.413 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T10:01:59.440 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T10:01:59.488 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T10:01:59.524 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package lua-any. 2024-04-25T10:01:59.540 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T10:01:59.582 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T10:01:59.741 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T10:01:59.768 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T10:01:59.808 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T10:01:59.819 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package zip. 2024-04-25T10:01:59.845 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T10:01:59.885 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T10:02:00.139 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package unzip. 2024-04-25T10:02:00.144 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-portend. 2024-04-25T10:02:00.165 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T10:02:00.170 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T10:02:00.197 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T10:02:00.219 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T10:02:00.567 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package luarocks. 2024-04-25T10:02:00.580 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T10:02:00.594 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T10:02:00.606 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T10:02:00.634 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T10:02:00.663 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T10:02:00.983 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T10:02:01.009 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T10:02:01.016 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package librgw2. 2024-04-25T10:02:01.043 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:01.049 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T10:02:01.083 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:01.386 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T10:02:01.412 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T10:02:01.453 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T10:02:01.619 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T10:02:01.635 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:01.669 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:01.764 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T10:02:01.791 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T10:02:01.831 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T10:02:01.972 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T10:02:01.999 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T10:02:02.038 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T10:02:02.092 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T10:02:02.107 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T10:02:02.142 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T10:02:02.291 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T10:02:02.319 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:02.349 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:02.444 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-mako. 2024-04-25T10:02:02.471 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T10:02:02.511 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T10:02:02.803 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-common. 2024-04-25T10:02:02.831 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:02.848 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T10:02:02.869 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:02.874 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T10:02:02.914 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T10:02:03.217 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T10:02:03.243 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T10:02:03.284 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T10:02:03.544 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-webob. 2024-04-25T10:02:03.560 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T10:02:03.594 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T10:02:03.964 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T10:02:03.991 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T10:02:04.034 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T10:02:04.217 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-base. 2024-04-25T10:02:04.244 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:04.292 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:04.342 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T10:02:04.368 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T10:02:04.409 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T10:02:04.695 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-paste. 2024-04-25T10:02:04.721 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T10:02:04.761 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T10:02:04.848 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T10:02:04.864 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T10:02:04.898 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T10:02:05.090 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T10:02:05.105 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T10:02:05.148 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T10:02:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T10:02:05.428 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T10:02:05.450 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T10:02:05.467 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T10:02:05.477 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T10:02:05.517 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T10:02:05.770 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T10:02:05.797 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T10:02:05.820 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T10:02:05.845 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T10:02:05.846 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T10:02:05.886 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T10:02:06.149 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T10:02:06.173 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T10:02:06.175 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T10:02:06.188 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T10:02:06.215 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T10:02:06.223 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T10:02:06.493 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T10:02:06.520 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T10:02:06.559 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T10:02:06.577 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T10:02:06.604 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T10:02:06.644 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T10:02:06.871 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T10:02:06.898 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T10:02:06.938 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T10:02:06.972 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-idna. 2024-04-25T10:02:06.998 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T10:02:07.046 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T10:02:07.241 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-portend. 2024-04-25T10:02:07.267 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T10:02:07.315 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T10:02:07.375 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T10:02:07.401 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T10:02:07.441 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T10:02:07.577 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T10:02:07.603 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T10:02:07.635 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T10:02:07.728 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-requests. 2024-04-25T10:02:07.754 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T10:02:07.786 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T10:02:07.921 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T10:02:07.948 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T10:02:07.988 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T10:02:08.030 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T10:02:08.045 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T10:02:08.089 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T10:02:08.317 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T10:02:08.332 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T10:02:08.358 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T10:02:08.366 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T10:02:08.374 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:02:08.408 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:08.661 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T10:02:08.688 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T10:02:08.728 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T10:02:08.969 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T10:02:08.995 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:09.039 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T10:02:09.067 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T10:02:09.093 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:09.489 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T10:02:10.167 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T10:02:10.193 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-mako. 2024-04-25T10:02:10.193 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T10:02:10.220 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T10:02:10.225 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T10:02:10.259 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T10:02:10.494 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T10:02:10.520 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T10:02:10.554 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T10:02:10.560 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T10:02:10.581 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T10:02:10.629 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T10:02:10.880 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T10:02:10.890 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T10:02:10.907 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:10.917 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T10:02:10.946 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:10.957 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T10:02:11.260 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-webob. 2024-04-25T10:02:11.286 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T10:02:11.317 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T10:02:11.326 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T10:02:11.343 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:11.375 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:11.671 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T10:02:11.698 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T10:02:11.741 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T10:02:11.995 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T10:02:12.021 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:12.057 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T10:02:12.061 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:12.084 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T10:02:12.124 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T10:02:12.377 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-paste. 2024-04-25T10:02:12.392 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T10:02:12.435 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T10:02:12.755 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T10:02:12.782 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T10:02:12.822 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T10:02:13.056 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph. 2024-04-25T10:02:13.066 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T10:02:13.071 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:13.093 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T10:02:13.114 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:13.133 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T10:02:13.417 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T10:02:13.444 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:13.461 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T10:02:13.475 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:13.489 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T10:02:13.527 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T10:02:13.828 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T10:02:13.839 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T10:02:13.855 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:13.866 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T10:02:13.894 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:13.905 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T10:02:14.298 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package cephadm. 2024-04-25T10:02:14.302 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T10:02:14.325 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:14.329 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T10:02:14.364 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:14.368 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T10:02:14.684 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T10:02:14.705 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-idna. 2024-04-25T10:02:14.710 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T10:02:14.732 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T10:02:14.750 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T10:02:14.780 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T10:02:15.061 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T10:02:15.076 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T10:02:15.091 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T10:02:15.118 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T10:02:15.119 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T10:02:15.158 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T10:02:15.511 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-requests. 2024-04-25T10:02:15.538 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T10:02:15.578 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T10:02:15.839 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T10:02:15.866 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T10:02:15.897 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T10:02:16.091 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-babel. 2024-04-25T10:02:16.106 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T10:02:16.141 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T10:02:16.184 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T10:02:16.200 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:02:16.234 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:16.469 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T10:02:16.495 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T10:02:16.527 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T10:02:16.614 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T10:02:16.641 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:16.680 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:16.813 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T10:02:16.828 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:02:16.863 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:17.001 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T10:02:17.027 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T10:02:17.075 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T10:02:17.175 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T10:02:17.201 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T10:02:17.233 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T10:02:17.336 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T10:02:17.363 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T10:02:17.394 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T10:02:17.502 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-routes. 2024-04-25T10:02:17.529 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T10:02:17.569 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T10:02:17.739 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T10:02:17.766 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:17.806 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:17.863 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T10:02:17.876 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:02:17.913 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:18.201 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T10:02:18.228 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:18.267 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:18.311 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T10:02:18.326 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T10:02:18.360 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T10:02:18.797 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T10:02:18.804 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T10:02:18.824 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T10:02:18.831 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:18.863 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:18.864 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T10:02:19.226 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T10:02:19.252 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T10:02:19.561 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T10:02:19.872 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T10:02:19.899 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T10:02:19.938 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T10:02:19.949 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph. 2024-04-25T10:02:19.976 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:20.016 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:20.368 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T10:02:20.396 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:20.435 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:20.648 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:02:20.714 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:20.988 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T10:02:21.004 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:21.038 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:21.408 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package cephadm. 2024-04-25T10:02:21.435 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:21.474 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:21.634 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T10:02:21.661 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T10:02:21.700 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T10:02:21.821 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T10:02:21.860 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T10:02:21.970 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T10:02:21.997 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T10:02:22.037 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T10:02:22.180 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T10:02:22.207 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T10:02:22.246 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T10:02:22.374 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T10:02:22.401 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T10:02:22.440 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T10:02:22.735 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T10:02:22.762 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T10:02:22.801 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T10:02:23.097 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T10:02:23.124 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T10:02:23.163 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T10:02:23.243 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-babel. 2024-04-25T10:02:23.259 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T10:02:23.301 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T10:02:23.475 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T10:02:23.501 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T10:02:23.541 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T10:02:23.604 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T10:02:23.621 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T10:02:23.654 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T10:02:23.828 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T10:02:23.855 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T10:02:23.894 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T10:02:23.941 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T10:02:23.968 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:02:23.999 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:24.214 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T10:02:24.241 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T10:02:24.280 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T10:02:24.345 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T10:02:24.360 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T10:02:24.394 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T10:02:24.680 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-routes. 2024-04-25T10:02:24.707 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T10:02:24.746 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T10:02:24.986 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T10:02:25.014 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:02:25.053 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:25.059 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T10:02:25.086 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:02:25.125 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:25.381 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T10:02:25.408 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T10:02:25.472 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T10:02:25.604 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T10:02:25.621 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T10:02:25.662 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T10:02:25.784 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T10:02:25.811 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T10:02:25.842 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T10:02:26.112 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package jq. 2024-04-25T10:02:26.139 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T10:02:26.178 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T10:02:26.341 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T10:02:26.368 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T10:02:26.407 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T10:02:26.431 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package socat. 2024-04-25T10:02:26.458 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T10:02:26.497 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T10:02:26.719 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T10:02:26.736 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T10:02:26.809 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T10:02:26.836 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T10:02:26.871 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T10:02:26.875 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T10:02:27.175 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T10:02:27.196 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-test. 2024-04-25T10:02:27.202 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T10:02:27.212 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:27.240 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T10:02:27.246 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:27.951 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T10:02:27.978 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:02:28.017 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:28.987 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T10:02:29.014 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T10:02:29.053 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T10:02:29.356 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T10:02:29.384 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T10:02:29.423 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T10:02:29.492 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T10:02:29.519 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:02:29.559 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:29.718 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T10:02:29.746 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T10:02:29.776 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T10:02:30.037 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T10:02:30.054 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T10:02:30.095 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T10:02:30.129 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T10:02:30.157 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:30.196 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:30.416 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T10:02:30.444 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T10:02:30.483 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T10:02:30.516 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T10:02:30.543 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T10:02:30.698 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T10:02:30.968 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T10:02:30.969 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T10:02:30.984 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:02:30.997 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T10:02:31.018 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:31.036 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T10:02:31.287 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T10:02:31.303 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:02:31.306 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T10:02:31.333 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T10:02:31.345 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:31.397 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T10:02:31.614 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T10:02:31.642 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:31.676 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T10:02:31.681 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:31.704 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T10:02:31.742 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T10:02:31.984 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T10:02:32.011 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T10:02:32.042 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T10:02:32.330 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T10:02:32.357 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T10:02:32.396 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T10:02:32.398 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T10:02:32.426 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:02:32.464 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:32.775 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T10:02:32.801 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T10:02:32.802 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T10:02:32.828 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T10:02:32.850 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T10:02:32.867 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T10:02:33.212 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T10:02:33.240 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T10:02:33.255 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package pkg-config. 2024-04-25T10:02:33.279 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T10:02:33.283 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T10:02:33.322 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T10:02:33.523 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package jq. 2024-04-25T10:02:33.540 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T10:02:33.573 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T10:02:33.575 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T10:02:33.591 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T10:02:33.633 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T10:02:33.793 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package socat. 2024-04-25T10:02:33.820 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T10:02:33.859 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T10:02:34.053 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T10:02:34.081 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T10:02:34.119 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T10:02:34.171 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T10:02:34.187 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T10:02:34.229 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T10:02:34.380 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T10:02:34.396 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T10:02:34.430 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T10:02:34.600 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-test. 2024-04-25T10:02:34.628 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:34.666 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:34.750 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T10:02:34.778 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T10:02:34.817 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T10:02:35.129 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T10:02:35.156 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T10:02:35.203 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T10:02:35.531 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-py. 2024-04-25T10:02:35.559 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T10:02:35.597 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T10:02:35.926 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T10:02:35.954 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T10:02:35.992 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T10:02:36.464 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T10:02:36.491 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T10:02:36.530 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T10:02:36.808 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-toml. 2024-04-25T10:02:36.835 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T10:02:36.866 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T10:02:37.004 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T10:02:37.032 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:02:37.070 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:37.152 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T10:02:37.179 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T10:02:37.218 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T10:02:37.467 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T10:02:37.495 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:37.522 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T10:02:37.533 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:37.550 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T10:02:37.580 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T10:02:37.820 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T10:02:37.836 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T10:02:37.849 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T10:02:37.877 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T10:02:37.878 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T10:02:37.916 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T10:02:38.088 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T10:02:38.105 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:02:38.138 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:38.408 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T10:02:38.436 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:02:38.474 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:38.695 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package radosgw. 2024-04-25T10:02:38.724 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:38.760 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T10:02:38.761 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:38.788 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:38.827 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:39.146 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T10:02:39.174 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T10:02:39.213 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T10:02:39.531 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T10:02:39.534 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T10:02:39.547 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:39.562 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T10:02:39.581 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:39.592 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T10:02:39.878 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T10:02:39.963 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T10:02:39.991 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T10:02:40.030 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T10:02:40.128 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T10:02:40.229 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T10:02:40.460 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package pkg-config. 2024-04-25T10:02:40.462 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T10:02:40.488 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T10:02:40.527 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T10:02:40.786 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T10:02:40.788 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T10:02:40.804 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T10:02:40.846 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T10:02:41.188 INFO:teuthology.orchestra.run.smithi100.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T10:02:41.256 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:41.276 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T10:02:41.304 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T10:02:41.343 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T10:02:41.745 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T10:02:41.773 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T10:02:41.780 INFO:teuthology.orchestra.run.smithi100.stdout:Adding system user cephadm....done 2024-04-25T10:02:41.912 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T10:02:41.928 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T10:02:42.214 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T10:02:42.240 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T10:02:42.269 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T10:02:42.307 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T10:02:42.448 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T10:02:42.574 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T10:02:42.627 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T10:02:42.655 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T10:02:42.693 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T10:02:42.825 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T10:02:43.021 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-py. 2024-04-25T10:02:43.049 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T10:02:43.087 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T10:02:43.093 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T10:02:43.219 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T10:02:43.408 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T10:02:43.436 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T10:02:43.474 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T10:02:43.512 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T10:02:43.898 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T10:02:43.945 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T10:02:43.973 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T10:02:44.012 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T10:02:44.140 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T10:02:44.298 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-toml. 2024-04-25T10:02:44.326 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T10:02:44.356 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T10:02:44.400 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T10:02:44.617 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T10:02:44.645 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T10:02:44.683 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T10:02:44.693 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T10:02:44.936 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T10:02:45.004 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T10:02:45.032 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T10:02:45.070 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T10:02:45.170 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T10:02:45.323 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T10:02:45.340 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T10:02:45.373 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T10:02:45.422 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T10:02:45.707 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:46.049 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T10:02:46.094 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package radosgw. 2024-04-25T10:02:46.123 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:46.152 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:46.175 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T10:02:46.309 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T10:02:46.569 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T10:02:46.670 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T10:02:46.921 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T10:02:46.998 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T10:02:47.025 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:02:47.064 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:47.312 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T10:02:47.344 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T10:02:47.369 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T10:02:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T10:02:47.534 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T10:02:47.677 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T10:02:47.803 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T10:02:47.918 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T10:02:48.044 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T10:02:48.070 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T10:02:48.304 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T10:02:48.434 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T10:02:48.546 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T10:02:48.831 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T10:02:48.870 INFO:teuthology.orchestra.run.smithi060.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T10:02:48.939 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:48.957 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T10:02:49.488 INFO:teuthology.orchestra.run.smithi060.stdout:Adding system user cephadm....done 2024-04-25T10:02:49.618 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T10:02:49.645 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T10:02:49.886 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T10:02:49.921 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T10:02:49.986 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T10:02:50.114 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T10:02:50.164 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T10:02:50.274 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T10:02:50.516 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T10:02:50.614 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T10:02:50.776 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T10:02:50.877 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T10:02:50.998 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T10:02:51.162 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T10:02:51.249 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T10:02:51.539 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T10:02:51.660 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T10:02:51.799 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T10:02:51.953 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T10:02:52.050 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T10:02:52.254 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T10:02:52.344 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T10:02:52.595 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T10:02:52.596 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T10:02:52.839 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T10:02:52.888 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T10:02:52.965 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:53.247 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T10:02:53.267 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T10:02:53.498 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:02:53.576 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T10:02:53.766 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T10:02:53.892 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T10:02:54.027 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T10:02:54.278 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T10:02:54.404 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T10:02:54.663 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T10:02:54.716 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T10:02:54.967 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:02:55.018 INFO:teuthology.orchestra.run.smithi060.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T10:02:55.053 INFO:teuthology.orchestra.run.smithi060.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T10:02:55.102 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T10:02:55.108 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T10:02:55.234 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T10:02:55.345 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T10:02:55.471 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T10:02:55.572 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T10:02:55.610 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T10:02:55.736 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T10:02:55.873 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T10:02:56.013 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T10:02:56.132 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T10:02:56.281 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T10:02:56.442 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:02:56.568 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T10:02:56.590 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T10:02:56.691 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T10:02:56.853 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T10:02:57.095 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T10:02:57.338 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T10:02:57.535 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T10:02:57.581 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T10:02:57.778 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T10:02:57.841 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T10:02:57.904 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T10:02:58.030 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T10:02:58.100 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T10:02:58.226 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T10:02:58.347 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T10:02:58.511 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T10:02:58.637 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T10:02:58.740 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T10:02:58.913 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T10:02:58.983 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T10:02:59.214 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T10:02:59.368 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T10:02:59.473 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T10:02:59.599 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T10:02:59.653 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T10:02:59.734 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T10:02:59.824 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T10:02:59.971 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T10:03:00.094 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T10:03:00.195 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T10:03:00.405 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T10:03:00.555 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T10:03:00.640 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T10:03:00.741 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:00.798 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T10:03:00.924 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T10:03:01.034 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T10:03:01.175 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:03:01.301 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T10:03:01.376 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T10:03:01.602 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T10:03:01.911 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T10:03:02.180 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T10:03:02.383 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T10:03:02.625 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T10:03:02.661 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T10:03:02.884 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:03:02.912 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T10:03:03.003 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T10:03:03.155 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T10:03:03.262 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T10:03:03.388 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T10:03:03.514 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T10:03:03.523 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T10:03:03.799 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T10:03:03.800 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T10:03:04.058 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T10:03:04.152 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T10:03:04.279 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:04.309 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:03:04.380 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:04.419 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T10:03:04.481 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T10:03:04.703 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T10:03:04.817 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:04.943 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:04.963 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T10:03:05.069 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T10:03:05.222 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T10:03:05.337 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:05.465 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T10:03:05.480 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:05.606 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:05.716 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T10:03:05.818 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T10:03:05.818 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T10:03:05.992 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T10:03:06.110 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T10:03:06.249 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:06.375 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:06.378 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T10:03:06.501 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:06.504 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T10:03:06.627 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:06.736 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T10:03:06.805 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T10:03:07.131 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T10:03:07.407 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T10:03:07.534 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T10:03:07.659 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T10:03:07.766 INFO:teuthology.orchestra.run.smithi060.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T10:03:07.898 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:08.003 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T10:03:08.024 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:08.129 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T10:03:08.151 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:08.260 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:08.369 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:08.504 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:08.514 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T10:03:08.773 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T10:03:08.874 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T10:03:09.046 INFO:teuthology.orchestra.run.smithi100.stdout:Adding group ceph....done 2024-04-25T10:03:09.150 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:03:09.277 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T10:03:09.373 INFO:teuthology.orchestra.run.smithi100.stdout:Adding system user ceph....done 2024-04-25T10:03:09.517 INFO:teuthology.orchestra.run.smithi100.stdout:Setting system user ceph properties....done 2024-04-25T10:03:09.537 INFO:teuthology.orchestra.run.smithi100.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T10:03:09.586 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T10:03:09.673 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T10:03:09.887 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T10:03:10.012 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T10:03:10.172 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T10:03:10.367 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T10:03:10.447 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:10.643 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T10:03:10.656 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:03:10.656 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:03:10.920 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T10:03:11.148 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:11.196 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T10:03:11.274 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:11.581 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T10:03:11.728 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:03:11.729 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:03:11.866 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T10:03:12.167 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:12.218 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T10:03:12.345 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:12.471 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:12.493 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T10:03:12.597 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T10:03:12.900 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:12.978 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:13.026 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:13.160 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T10:03:13.218 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T10:03:13.218 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T10:03:13.445 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:13.571 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:13.696 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:13.697 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:13.915 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T10:03:13.915 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T10:03:13.935 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T10:03:13.935 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T10:03:14.406 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:14.491 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:14.609 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:14.681 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T10:03:14.681 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T10:03:15.085 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:15.174 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:15.220 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:15.300 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:15.346 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T10:03:15.435 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:15.640 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T10:03:15.641 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T10:03:16.315 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:16.474 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:16.600 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:16.641 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:16.743 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:16.767 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:16.876 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:16.978 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:17.079 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:17.079 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:03:17.172 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:03:17.204 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:17.324 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:03:17.748 INFO:teuthology.orchestra.run.smithi060.stdout:Adding group ceph....done 2024-04-25T10:03:18.416 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:03:18.434 INFO:teuthology.orchestra.run.smithi060.stdout:Adding system user ceph....done 2024-04-25T10:03:18.569 INFO:teuthology.orchestra.run.smithi060.stdout:Setting system user ceph properties....done 2024-04-25T10:03:18.586 INFO:teuthology.orchestra.run.smithi060.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T10:03:18.727 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T10:03:19.063 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T10:03:19.077 INFO:teuthology.orchestra.run.smithi100.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:03:19.081 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T10:03:19.134 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-04-25T10:03:19.319 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-04-25T10:03:19.320 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-04-25T10:03:19.490 INFO:teuthology.orchestra.run.smithi100.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T10:03:19.490 INFO:teuthology.orchestra.run.smithi100.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T10:03:19.490 INFO:teuthology.orchestra.run.smithi100.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T10:03:19.490 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T10:03:19.490 INFO:teuthology.orchestra.run.smithi100.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T10:03:19.490 INFO:teuthology.orchestra.run.smithi100.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T10:03:19.490 INFO:teuthology.orchestra.run.smithi100.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T10:03:19.490 INFO:teuthology.orchestra.run.smithi100.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T10:03:19.491 INFO:teuthology.orchestra.run.smithi100.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T10:03:19.491 INFO:teuthology.orchestra.run.smithi100.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T10:03:19.491 INFO:teuthology.orchestra.run.smithi100.stdout:The following additional packages will be installed: 2024-04-25T10:03:19.491 INFO:teuthology.orchestra.run.smithi100.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T10:03:19.491 INFO:teuthology.orchestra.run.smithi100.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T10:03:19.492 INFO:teuthology.orchestra.run.smithi100.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T10:03:19.492 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T10:03:19.492 INFO:teuthology.orchestra.run.smithi100.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:03:19.493 INFO:teuthology.orchestra.run.smithi100.stdout:Suggested packages: 2024-04-25T10:03:19.493 INFO:teuthology.orchestra.run.smithi100.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T10:03:19.493 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T10:03:19.493 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T10:03:19.516 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:19.533 INFO:teuthology.orchestra.run.smithi100.stdout:The following NEW packages will be installed: 2024-04-25T10:03:19.533 INFO:teuthology.orchestra.run.smithi100.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T10:03:19.533 INFO:teuthology.orchestra.run.smithi100.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T10:03:19.534 INFO:teuthology.orchestra.run.smithi100.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T10:03:19.535 INFO:teuthology.orchestra.run.smithi100.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T10:03:19.535 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T10:03:19.535 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T10:03:19.535 INFO:teuthology.orchestra.run.smithi100.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:03:19.536 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages will be upgraded: 2024-04-25T10:03:19.536 INFO:teuthology.orchestra.run.smithi100.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T10:03:19.717 INFO:teuthology.orchestra.run.smithi100.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T10:03:19.717 INFO:teuthology.orchestra.run.smithi100.stdout:Need to get 55.7 MB of archives. 2024-04-25T10:03:19.717 INFO:teuthology.orchestra.run.smithi100.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T10:03:19.717 INFO:teuthology.orchestra.run.smithi100.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T10:03:19.725 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:03:19.725 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:03:20.184 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:20.310 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:20.364 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T10:03:20.384 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T10:03:20.513 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T10:03:20.514 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T10:03:20.515 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T10:03:20.543 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T10:03:20.544 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T10:03:20.548 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T10:03:20.550 INFO:teuthology.orchestra.run.smithi100.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T10:03:20.551 INFO:teuthology.orchestra.run.smithi100.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T10:03:20.718 INFO:teuthology.orchestra.run.smithi100.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T10:03:20.800 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:03:20.801 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:03:21.011 INFO:teuthology.orchestra.run.smithi100.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T10:03:21.021 INFO:teuthology.orchestra.run.smithi100.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T10:03:21.026 INFO:teuthology.orchestra.run.smithi100.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T10:03:21.111 INFO:teuthology.orchestra.run.smithi100.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T10:03:21.263 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:21.628 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T10:03:22.081 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:22.188 INFO:teuthology.orchestra.run.smithi100.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T10:03:22.193 INFO:teuthology.orchestra.run.smithi100.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T10:03:22.193 INFO:teuthology.orchestra.run.smithi100.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T10:03:22.199 INFO:teuthology.orchestra.run.smithi100.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T10:03:22.200 INFO:teuthology.orchestra.run.smithi100.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T10:03:22.211 INFO:teuthology.orchestra.run.smithi100.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T10:03:22.293 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T10:03:22.293 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T10:03:22.378 INFO:teuthology.orchestra.run.smithi100.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T10:03:22.766 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:22.977 INFO:teuthology.orchestra.run.smithi100.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T10:03:22.978 INFO:teuthology.orchestra.run.smithi100.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T10:03:22.985 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T10:03:22.985 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T10:03:23.160 INFO:teuthology.orchestra.run.smithi100.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T10:03:23.161 INFO:teuthology.orchestra.run.smithi100.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T10:03:23.196 INFO:teuthology.orchestra.run.smithi100.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T10:03:23.261 INFO:teuthology.orchestra.run.smithi100.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T10:03:23.269 INFO:teuthology.orchestra.run.smithi100.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T10:03:23.269 INFO:teuthology.orchestra.run.smithi100.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T10:03:23.270 INFO:teuthology.orchestra.run.smithi100.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T10:03:23.271 INFO:teuthology.orchestra.run.smithi100.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T10:03:23.478 INFO:teuthology.orchestra.run.smithi100.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T10:03:23.588 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:23.861 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T10:03:23.861 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T10:03:23.889 INFO:teuthology.orchestra.run.smithi100.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T10:03:24.348 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:24.449 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:24.583 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:24.701 INFO:teuthology.orchestra.run.smithi100.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T10:03:24.701 INFO:teuthology.orchestra.run.smithi100.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T10:03:24.701 INFO:teuthology.orchestra.run.smithi100.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T10:03:24.744 INFO:teuthology.orchestra.run.smithi100.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T10:03:24.744 INFO:teuthology.orchestra.run.smithi100.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T10:03:24.745 INFO:teuthology.orchestra.run.smithi100.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T10:03:24.778 INFO:teuthology.orchestra.run.smithi100.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T10:03:24.778 INFO:teuthology.orchestra.run.smithi100.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T10:03:24.804 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T10:03:24.804 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T10:03:25.182 INFO:teuthology.orchestra.run.smithi100.stdout:Preconfiguring packages ... 2024-04-25T10:03:25.251 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:25.346 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 55.7 MB in 5s (10.6 MB/s) 2024-04-25T10:03:25.411 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:25.537 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:25.664 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T10:03:25.669 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:03:25.688 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:03:25.816 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:03:26.033 INFO:teuthology.orchestra.run.smithi060.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:03:26.133 INFO:teuthology.orchestra.run.smithi060.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:03:26.277 INFO:teuthology.orchestra.run.smithi060.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:03:26.454 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T10:03:27.394 INFO:teuthology.orchestra.run.smithi100.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T10:03:27.429 INFO:teuthology.orchestra.run.smithi060.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:03:27.664 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package flex. 2024-04-25T10:03:27.767 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T10:03:27.772 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T10:03:27.820 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T10:03:28.075 INFO:teuthology.orchestra.run.smithi060.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:03:28.078 DEBUG:teuthology.orchestra.run.smithi060:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T10:03:28.174 INFO:teuthology.orchestra.run.smithi060.stdout:Reading package lists... 2024-04-25T10:03:28.344 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:03:28.379 INFO:teuthology.orchestra.run.smithi060.stdout:Building dependency tree... 2024-04-25T10:03:28.380 INFO:teuthology.orchestra.run.smithi060.stdout:Reading state information... 2024-04-25T10:03:28.517 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T10:03:28.583 INFO:teuthology.orchestra.run.smithi060.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T10:03:28.583 INFO:teuthology.orchestra.run.smithi060.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T10:03:28.583 INFO:teuthology.orchestra.run.smithi060.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T10:03:28.583 INFO:teuthology.orchestra.run.smithi060.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T10:03:28.583 INFO:teuthology.orchestra.run.smithi060.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T10:03:28.583 INFO:teuthology.orchestra.run.smithi060.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T10:03:28.584 INFO:teuthology.orchestra.run.smithi060.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T10:03:28.584 INFO:teuthology.orchestra.run.smithi060.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T10:03:28.584 INFO:teuthology.orchestra.run.smithi060.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T10:03:28.584 INFO:teuthology.orchestra.run.smithi060.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T10:03:28.585 INFO:teuthology.orchestra.run.smithi060.stdout:The following additional packages will be installed: 2024-04-25T10:03:28.585 INFO:teuthology.orchestra.run.smithi060.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T10:03:28.585 INFO:teuthology.orchestra.run.smithi060.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T10:03:28.586 INFO:teuthology.orchestra.run.smithi060.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T10:03:28.586 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T10:03:28.586 INFO:teuthology.orchestra.run.smithi060.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:03:28.587 INFO:teuthology.orchestra.run.smithi060.stdout:Suggested packages: 2024-04-25T10:03:28.587 INFO:teuthology.orchestra.run.smithi060.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T10:03:28.587 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T10:03:28.587 INFO:teuthology.orchestra.run.smithi060.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T10:03:28.630 INFO:teuthology.orchestra.run.smithi060.stdout:The following NEW packages will be installed: 2024-04-25T10:03:28.631 INFO:teuthology.orchestra.run.smithi060.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T10:03:28.631 INFO:teuthology.orchestra.run.smithi060.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T10:03:28.631 INFO:teuthology.orchestra.run.smithi060.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T10:03:28.631 INFO:teuthology.orchestra.run.smithi060.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T10:03:28.631 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T10:03:28.631 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T10:03:28.632 INFO:teuthology.orchestra.run.smithi060.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:03:28.632 INFO:teuthology.orchestra.run.smithi060.stdout:The following packages will be upgraded: 2024-04-25T10:03:28.632 INFO:teuthology.orchestra.run.smithi060.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T10:03:28.689 INFO:teuthology.orchestra.run.smithi060.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T10:03:28.690 INFO:teuthology.orchestra.run.smithi060.stdout:Need to get 55.7 MB of archives. 2024-04-25T10:03:28.690 INFO:teuthology.orchestra.run.smithi060.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T10:03:28.690 INFO:teuthology.orchestra.run.smithi060.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T10:03:28.841 INFO:teuthology.orchestra.run.smithi060.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T10:03:28.849 INFO:teuthology.orchestra.run.smithi060.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T10:03:28.889 INFO:teuthology.orchestra.run.smithi060.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T10:03:28.890 INFO:teuthology.orchestra.run.smithi060.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T10:03:28.891 INFO:teuthology.orchestra.run.smithi060.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T10:03:28.902 INFO:teuthology.orchestra.run.smithi060.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T10:03:28.903 INFO:teuthology.orchestra.run.smithi060.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T10:03:28.905 INFO:teuthology.orchestra.run.smithi060.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T10:03:28.907 INFO:teuthology.orchestra.run.smithi060.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T10:03:28.907 INFO:teuthology.orchestra.run.smithi060.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T10:03:28.908 INFO:teuthology.orchestra.run.smithi060.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T10:03:28.910 INFO:teuthology.orchestra.run.smithi060.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T10:03:28.930 INFO:teuthology.orchestra.run.smithi060.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T10:03:28.930 INFO:teuthology.orchestra.run.smithi060.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T10:03:28.931 INFO:teuthology.orchestra.run.smithi060.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T10:03:28.944 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:03:29.345 INFO:teuthology.orchestra.run.smithi060.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T10:03:29.349 INFO:teuthology.orchestra.run.smithi060.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T10:03:29.352 INFO:teuthology.orchestra.run.smithi060.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T10:03:29.359 INFO:teuthology.orchestra.run.smithi060.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T10:03:29.362 INFO:teuthology.orchestra.run.smithi060.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T10:03:29.373 INFO:teuthology.orchestra.run.smithi060.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T10:03:29.377 INFO:teuthology.orchestra.run.smithi060.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T10:03:29.394 INFO:teuthology.orchestra.run.smithi060.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T10:03:29.395 INFO:teuthology.orchestra.run.smithi060.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T10:03:29.433 INFO:teuthology.orchestra.run.smithi060.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T10:03:29.433 INFO:teuthology.orchestra.run.smithi060.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T10:03:29.446 INFO:teuthology.orchestra.run.smithi060.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T10:03:29.467 INFO:teuthology.orchestra.run.smithi060.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T10:03:29.470 INFO:teuthology.orchestra.run.smithi060.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T10:03:29.471 INFO:teuthology.orchestra.run.smithi060.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T10:03:29.472 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T10:03:29.472 INFO:teuthology.orchestra.run.smithi060.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T10:03:29.472 INFO:teuthology.orchestra.run.smithi060.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T10:03:29.477 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:03:29.482 INFO:teuthology.orchestra.run.smithi060.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T10:03:29.484 INFO:teuthology.orchestra.run.smithi060.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T10:03:29.575 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:03:29.814 INFO:teuthology.orchestra.run.smithi060.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T10:03:29.814 INFO:teuthology.orchestra.run.smithi060.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T10:03:29.814 INFO:teuthology.orchestra.run.smithi060.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T10:03:29.814 INFO:teuthology.orchestra.run.smithi060.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T10:03:29.815 INFO:teuthology.orchestra.run.smithi060.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T10:03:29.815 INFO:teuthology.orchestra.run.smithi060.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T10:03:29.829 INFO:teuthology.orchestra.run.smithi060.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T10:03:29.829 INFO:teuthology.orchestra.run.smithi060.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T10:03:30.001 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:03:30.132 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:03:30.255 INFO:teuthology.orchestra.run.smithi060.stdout:Preconfiguring packages ... 2024-04-25T10:03:30.415 INFO:teuthology.orchestra.run.smithi060.stdout:Fetched 55.7 MB in 1s (46.8 MB/s) 2024-04-25T10:03:30.459 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:03:30.736 INFO:teuthology.orchestra.run.smithi060.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T10:03:30.742 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:03:30.858 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package bison. 2024-04-25T10:03:30.885 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:03:30.962 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T10:03:30.968 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T10:03:31.018 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:03:31.377 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T10:03:31.405 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T10:03:31.436 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:03:31.524 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T10:03:31.755 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package dump. 2024-04-25T10:03:31.783 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T10:03:31.822 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T10:03:32.118 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package indent. 2024-04-25T10:03:32.145 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:03:32.184 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:03:32.437 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T10:03:32.453 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T10:03:32.487 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:03:32.498 INFO:teuthology.orchestra.run.smithi060.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T10:03:32.759 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package flex. 2024-04-25T10:03:32.774 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T10:03:32.790 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T10:03:32.832 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:03:32.857 INFO:teuthology.orchestra.run.smithi060.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T10:03:32.863 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T10:03:32.916 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T10:03:33.118 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T10:03:33.134 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T10:03:33.176 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:03:33.450 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:03:33.472 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T10:03:33.500 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T10:03:33.538 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:03:33.630 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T10:03:33.741 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T10:03:33.757 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T10:03:33.799 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:03:34.050 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:03:34.052 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T10:03:34.076 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T10:03:34.119 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T10:03:34.455 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T10:03:34.483 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T10:03:34.522 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:03:34.552 INFO:teuthology.orchestra.run.smithi060.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T10:03:34.558 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:03:34.738 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:03:35.214 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:03:35.353 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:03:35.690 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:03:36.096 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package bison. 2024-04-25T10:03:36.203 INFO:teuthology.orchestra.run.smithi060.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T10:03:36.209 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T10:03:36.250 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T10:03:36.256 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:03:36.266 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:03:36.300 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:03:36.570 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T10:03:36.597 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:03:36.644 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:03:36.674 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T10:03:36.702 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T10:03:36.740 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:03:37.010 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package dump. 2024-04-25T10:03:37.038 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T10:03:37.068 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T10:03:37.114 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T10:03:37.141 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:03:37.181 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:03:37.356 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package indent. 2024-04-25T10:03:37.384 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:03:37.422 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:03:37.492 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T10:03:37.519 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T10:03:37.550 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:03:37.691 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T10:03:37.708 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T10:03:37.750 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:03:37.870 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T10:03:37.898 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:03:37.936 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:03:38.029 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T10:03:38.045 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T10:03:38.095 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:03:38.290 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T10:03:38.317 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:03:38.356 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:03:38.357 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T10:03:38.373 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T10:03:38.415 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:03:38.667 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T10:03:38.695 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:03:38.726 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:03:38.752 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T10:03:38.780 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T10:03:38.818 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:03:39.028 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T10:03:39.044 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:03:39.087 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:03:39.121 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T10:03:39.138 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T10:03:39.188 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:03:39.364 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T10:03:39.392 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:03:39.431 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:03:39.482 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T10:03:39.510 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T10:03:39.549 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T10:03:39.910 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T10:03:39.937 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T10:03:39.960 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T10:03:39.968 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:03:39.989 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T10:03:40.027 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:03:40.330 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package modemmanager. 2024-04-25T10:03:40.357 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:03:40.396 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:03:40.936 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package network-manager. 2024-04-25T10:03:40.964 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:03:41.011 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:03:41.652 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ppp. 2024-04-25T10:03:41.680 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T10:03:41.772 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T10:03:41.800 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:03:41.838 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:03:41.948 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:03:42.100 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T10:03:42.116 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:03:42.158 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:03:42.407 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T10:03:42.435 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T10:03:42.482 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T10:03:42.485 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T10:03:42.514 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:03:42.552 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:03:42.727 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T10:03:42.754 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T10:03:42.785 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T10:03:42.855 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T10:03:42.883 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T10:03:42.913 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:03:43.071 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T10:03:43.099 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T10:03:43.138 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T10:03:43.275 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T10:03:43.303 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:03:43.341 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:03:43.450 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T10:03:43.478 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:03:43.516 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:03:43.694 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T10:03:43.722 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:03:43.761 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:03:44.071 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T10:03:44.097 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T10:03:44.099 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T10:03:44.125 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:03:44.148 INFO:teuthology.orchestra.run.smithi100.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T10:03:44.156 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:03:44.169 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking postgresql-common (238) ... 2024-04-25T10:03:44.450 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T10:03:44.468 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:03:44.509 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:03:44.612 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T10:03:44.640 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:03:44.682 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:03:44.769 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T10:03:44.786 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:03:44.828 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:03:45.349 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T10:03:45.377 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T10:03:45.415 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:03:45.760 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package modemmanager. 2024-04-25T10:03:45.788 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:03:45.826 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:03:45.865 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package postgresql. 2024-04-25T10:03:45.882 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T10:03:45.915 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking postgresql (14+238) ... 2024-04-25T10:03:46.142 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T10:03:46.159 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T10:03:46.200 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T10:03:46.453 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T10:03:46.469 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T10:03:46.475 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package network-manager. 2024-04-25T10:03:46.503 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:03:46.511 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T10:03:46.551 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:03:46.839 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T10:03:46.867 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T10:03:46.905 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:03:47.141 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package tcl. 2024-04-25T10:03:47.170 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T10:03:47.191 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ppp. 2024-04-25T10:03:47.203 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T10:03:47.219 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T10:03:47.492 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:03:47.552 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T10:03:47.581 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T10:03:47.619 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:03:47.829 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T10:03:47.858 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T10:03:47.896 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T10:03:47.974 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T10:03:48.002 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T10:03:48.040 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T10:03:48.141 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T10:03:48.158 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T10:03:48.199 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T10:03:48.352 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T10:03:48.380 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T10:03:48.427 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:03:48.510 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T10:03:48.538 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T10:03:48.585 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T10:03:48.724 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T10:03:48.926 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T10:03:49.022 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T10:03:49.051 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:03:49.052 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:03:49.178 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:03:49.304 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:03:49.333 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:03:49.430 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:03:49.540 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:03:49.927 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T10:03:49.956 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T10:03:50.014 INFO:teuthology.orchestra.run.smithi060.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T10:03:50.022 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T10:03:50.042 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking postgresql-common (238) ... 2024-04-25T10:03:50.112 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:03:50.505 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:03:50.505 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:03:50.526 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T10:03:50.555 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:03:50.605 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:03:50.982 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:03:51.083 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:03:51.200 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:03:51.326 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T10:03:51.661 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:03:51.946 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package postgresql. 2024-04-25T10:03:51.964 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T10:03:52.004 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking postgresql (14+238) ... 2024-04-25T10:03:52.037 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:03:52.163 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:03:52.223 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T10:03:52.240 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T10:03:52.273 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:03:52.273 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T10:03:52.500 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T10:03:52.517 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T10:03:52.559 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T10:03:52.878 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T10:03:52.894 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:03:52.907 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T10:03:52.945 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:03:53.021 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T10:03:53.114 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T10:03:53.181 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:03:53.205 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package tcl. 2024-04-25T10:03:53.222 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T10:03:53.268 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T10:03:53.282 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:03:53.383 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T10:03:53.501 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T10:03:53.550 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T10:03:53.579 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T10:03:53.617 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:03:53.626 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:03:53.717 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T10:03:53.787 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:03:53.912 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:03:54.012 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T10:03:54.029 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T10:03:54.071 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T10:03:54.089 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:03:54.190 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:03:54.291 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T10:03:54.434 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T10:03:54.508 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T10:03:54.536 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T10:03:54.586 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:03:54.587 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:03:54.872 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T10:03:55.057 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:03:55.057 INFO:teuthology.orchestra.run.smithi100.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T10:03:55.057 INFO:teuthology.orchestra.run.smithi100.stdout:which means they are currently configured by ifupdown: 2024-04-25T10:03:55.057 INFO:teuthology.orchestra.run.smithi100.stdout:- enp3s0f1 2024-04-25T10:03:55.057 INFO:teuthology.orchestra.run.smithi100.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T10:03:55.057 INFO:teuthology.orchestra.run.smithi100.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T10:03:55.058 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:03:55.080 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T10:03:55.174 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T10:03:55.206 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:03:55.324 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:03:55.442 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:03:55.477 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T10:03:55.568 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:03:55.693 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:03:55.754 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T10:03:56.232 INFO:teuthology.orchestra.run.smithi060.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T10:03:56.237 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:03:56.341 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:03:56.381 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:03:56.482 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T10:03:56.583 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up postgresql-common (238) ... 2024-04-25T10:03:56.753 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:03:56.753 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:03:57.277 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:03:57.420 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:03:57.546 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:03:57.681 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T10:03:57.823 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:03:57.872 INFO:teuthology.orchestra.run.smithi100.stdout:Adding user postgres to group ssl-cert 2024-04-25T10:03:57.983 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:03:58.097 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:03:58.097 INFO:teuthology.orchestra.run.smithi100.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T10:03:58.134 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:03:58.260 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:03:58.271 INFO:teuthology.orchestra.run.smithi100.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T10:03:58.271 INFO:teuthology.orchestra.run.smithi100.stdout:Removing obsolete dictionary files: 2024-04-25T10:03:58.848 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T10:03:58.997 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:03:59.175 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T10:03:59.218 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:03:59.286 INFO:teuthology.orchestra.run.smithi060.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T10:03:59.352 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:03:59.535 INFO:teuthology.orchestra.run.smithi100.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T10:03:59.551 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T10:03:59.599 INFO:teuthology.orchestra.run.smithi100.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T10:03:59.599 INFO:teuthology.orchestra.run.smithi100.stdout:This user must also own the server process. 2024-04-25T10:03:59.599 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:03:59.599 INFO:teuthology.orchestra.run.smithi100.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T10:03:59.599 INFO:teuthology.orchestra.run.smithi100.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T10:03:59.599 INFO:teuthology.orchestra.run.smithi100.stdout:The default text search configuration will be set to "english". 2024-04-25T10:03:59.599 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:03:59.599 INFO:teuthology.orchestra.run.smithi100.stdout:Data page checksums are disabled. 2024-04-25T10:03:59.600 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:03:59.600 INFO:teuthology.orchestra.run.smithi100.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T10:03:59.600 INFO:teuthology.orchestra.run.smithi100.stdout:creating subdirectories ... ok 2024-04-25T10:03:59.600 INFO:teuthology.orchestra.run.smithi100.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T10:03:59.612 INFO:teuthology.orchestra.run.smithi100.stdout:selecting default max_connections ... 100 2024-04-25T10:03:59.630 INFO:teuthology.orchestra.run.smithi100.stdout:selecting default shared_buffers ... 128MB 2024-04-25T10:03:59.640 INFO:teuthology.orchestra.run.smithi100.stdout:selecting default time zone ... Etc/UTC 2024-04-25T10:03:59.641 INFO:teuthology.orchestra.run.smithi100.stdout:creating configuration files ... ok 2024-04-25T10:03:59.750 INFO:teuthology.orchestra.run.smithi100.stdout:running bootstrap script ... ok 2024-04-25T10:03:59.870 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:03:59.996 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T10:04:00.114 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T10:04:00.215 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:04:00.288 INFO:teuthology.orchestra.run.smithi060.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T10:04:00.347 INFO:teuthology.orchestra.run.smithi100.stdout:performing post-bootstrap initialization ... ok 2024-04-25T10:04:00.350 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:04:00.476 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:04:00.652 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:04:00.778 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:04:00.904 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T10:04:01.012 INFO:teuthology.orchestra.run.smithi100.stdout:syncing data to disk ... ok 2024-04-25T10:04:01.089 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T10:04:01.207 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:04:01.398 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T10:04:01.684 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:01.684 INFO:teuthology.orchestra.run.smithi060.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T10:04:01.684 INFO:teuthology.orchestra.run.smithi060.stdout:which means they are currently configured by ifupdown: 2024-04-25T10:04:01.684 INFO:teuthology.orchestra.run.smithi060.stdout:- ens1f0 2024-04-25T10:04:01.684 INFO:teuthology.orchestra.run.smithi060.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T10:04:01.684 INFO:teuthology.orchestra.run.smithi060.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T10:04:01.684 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:01.821 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T10:04:02.160 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T10:04:02.492 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T10:04:02.976 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:04:03.111 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:04:03.237 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T10:04:03.363 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up postgresql-common (238) ... 2024-04-25T10:04:04.249 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:04:04.270 INFO:teuthology.orchestra.run.smithi060.stdout:Adding user postgres to group ssl-cert 2024-04-25T10:04:04.375 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T10:04:04.501 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up postgresql (14+238) ... 2024-04-25T10:04:04.502 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:04.503 INFO:teuthology.orchestra.run.smithi060.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T10:04:04.627 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:04:04.687 INFO:teuthology.orchestra.run.smithi060.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T10:04:04.687 INFO:teuthology.orchestra.run.smithi060.stdout:Removing obsolete dictionary files: 2024-04-25T10:04:04.744 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:04:04.974 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:04:04.974 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:04:05.220 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T10:04:05.513 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:04:05.606 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T10:04:05.615 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:04:05.908 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:04:05.960 INFO:teuthology.orchestra.run.smithi060.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T10:04:05.975 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T10:04:06.027 INFO:teuthology.orchestra.run.smithi060.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T10:04:06.027 INFO:teuthology.orchestra.run.smithi060.stdout:This user must also own the server process. 2024-04-25T10:04:06.027 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:06.027 INFO:teuthology.orchestra.run.smithi060.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T10:04:06.027 INFO:teuthology.orchestra.run.smithi060.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T10:04:06.028 INFO:teuthology.orchestra.run.smithi060.stdout:The default text search configuration will be set to "english". 2024-04-25T10:04:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:06.028 INFO:teuthology.orchestra.run.smithi060.stdout:Data page checksums are disabled. 2024-04-25T10:04:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:06.028 INFO:teuthology.orchestra.run.smithi060.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T10:04:06.028 INFO:teuthology.orchestra.run.smithi060.stdout:creating subdirectories ... ok 2024-04-25T10:04:06.028 INFO:teuthology.orchestra.run.smithi060.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T10:04:06.042 INFO:teuthology.orchestra.run.smithi060.stdout:selecting default max_connections ... 100 2024-04-25T10:04:06.052 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:04:06.065 INFO:teuthology.orchestra.run.smithi060.stdout:selecting default shared_buffers ... 128MB 2024-04-25T10:04:06.072 INFO:teuthology.orchestra.run.smithi060.stdout:selecting default time zone ... Etc/UTC 2024-04-25T10:04:06.073 INFO:teuthology.orchestra.run.smithi060.stdout:creating configuration files ... ok 2024-04-25T10:04:06.187 INFO:teuthology.orchestra.run.smithi060.stdout:running bootstrap script ... ok 2024-04-25T10:04:06.908 INFO:teuthology.orchestra.run.smithi060.stdout:performing post-bootstrap initialization ... ok 2024-04-25T10:04:07.446 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T10:04:07.546 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:04:07.583 INFO:teuthology.orchestra.run.smithi060.stdout:syncing data to disk ... ok 2024-04-25T10:04:08.002 INFO:teuthology.orchestra.run.smithi060.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T10:04:08.032 INFO:teuthology.orchestra.run.smithi100.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:04:08.038 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:10.738 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:04:10.889 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T10:04:11.049 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up postgresql (14+238) ... 2024-04-25T10:04:11.200 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:04:11.334 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:04:11.598 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:04:11.598 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:04:12.213 INFO:teuthology.orchestra.run.smithi060.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:04:12.304 INFO:teuthology.orchestra.run.smithi060.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T10:04:12.615 INFO:teuthology.orchestra.run.smithi060.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:04:12.767 INFO:teuthology.orchestra.run.smithi060.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:04:14.187 INFO:teuthology.orchestra.run.smithi060.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T10:04:14.320 INFO:teuthology.orchestra.run.smithi060.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:04:14.833 INFO:teuthology.orchestra.run.smithi060.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:04:14.838 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:14.839 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:04:14.985 DEBUG:teuthology.orchestra.run.smithi060:> dpkg-query -W -f '${Version}' ceph 2024-04-25T10:04:15.005 INFO:teuthology.orchestra.run.smithi060.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:04:15.006 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:04:15.006 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-25T10:04:15.007 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:04:15.157 DEBUG:teuthology.orchestra.run.smithi100:> dpkg-query -W -f '${Version}' ceph 2024-04-25T10:04:15.176 INFO:teuthology.orchestra.run.smithi100.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:04:15.177 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:04:15.177 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-25T10:04:15.178 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T10:04:15.179 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:15.179 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:04:15.194 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:15.194 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:04:15.232 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T10:04:15.232 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:15.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:04:15.250 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:04:15.305 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:15.305 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:04:15.318 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:04:15.374 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T10:04:15.374 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:15.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:04:15.387 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:04:15.440 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:15.440 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:04:15.454 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:04:15.506 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T10:04:15.507 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:15.507 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:04:15.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:04:15.577 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:15.577 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:04:15.591 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:04:15.646 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T10:04:15.738 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T10:04:15.738 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 777 /var/log/ceph 2024-04-25T10:04:15.740 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /var/log/ceph 2024-04-25T10:04:15.753 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T10:04:15.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:04:15.795 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:04:15.807 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T10:04:15.807 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:04:15.850 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:04:15.868 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T10:04:15.869 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T10:04:15.869 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T10:04:15.869 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:15.869 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:04:15.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:15.986 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:15.986 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:04:15.991 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:16.072 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T10:04:16.081 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T10:04:16.081 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673046', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7673046', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1124, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi060.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOJo8U9+QQ9DHgmYAF/GqBfHPmk+1neS6TI80vf0Oy9sk8FrajK2LBP3Z0GtQqlKRKvdMo3Gjk1i+7Zp4VLb/bc=', 'smithi100.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBW1PWiQ3NBSQTdGIckR1SFVrrJkj/jfHuCtXxMjY0uP/fUXMQeULwdHmHqKw36gQ1tp2IIvchRJK0G6iSztMHg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_09:36:50', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T10:04:16.082 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:04:16.084 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:04:16.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:04:16.130 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:04:16.175 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:16.175 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:04:16.189 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:04:16.189 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_1 2024-04-25T10:04:16.238 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:04:16.238 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:04:16.238 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 888 Links: 1 2024-04-25T10:04:16.238 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:04:16.238 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-04-25 09:58:08.310886274 +0000 2024-04-25T10:04:16.238 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-04-25 09:58:08.306886310 +0000 2024-04-25T10:04:16.238 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-04-25 09:58:08.306886310 +0000 2024-04-25T10:04:16.238 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: - 2024-04-25T10:04:16.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:04:16.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T10:04:16.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T10:04:16.292 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000298893 s, 1.7 MB/s 2024-04-25T10:04:16.293 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:04:16.343 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_2 2024-04-25T10:04:16.389 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:04:16.389 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:04:16.389 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-04-25T10:04:16.390 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:04:16.390 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-04-25 09:58:08.814881772 +0000 2024-04-25T10:04:16.390 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-04-25 09:58:08.810881807 +0000 2024-04-25T10:04:16.390 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-04-25 09:58:08.810881807 +0000 2024-04-25T10:04:16.390 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: - 2024-04-25T10:04:16.390 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:04:16.444 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T10:04:16.444 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T10:04:16.444 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000311241 s, 1.6 MB/s 2024-04-25T10:04:16.446 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:04:16.494 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_3 2024-04-25T10:04:16.541 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:04:16.542 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:04:16.542 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-04-25T10:04:16.542 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:04:16.542 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-04-25 09:58:09.550875198 +0000 2024-04-25T10:04:16.542 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-04-25 09:58:09.290877520 +0000 2024-04-25T10:04:16.542 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-04-25 09:58:09.290877520 +0000 2024-04-25T10:04:16.542 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: - 2024-04-25T10:04:16.542 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:04:16.596 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T10:04:16.596 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T10:04:16.596 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000300451 s, 1.7 MB/s 2024-04-25T10:04:16.597 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:04:16.647 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_4 2024-04-25T10:04:16.693 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:04:16.693 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:04:16.693 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 937 Links: 1 2024-04-25T10:04:16.694 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:04:16.694 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-04-25 09:58:10.074870518 +0000 2024-04-25T10:04:16.694 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-04-25 09:58:09.806872911 +0000 2024-04-25T10:04:16.694 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-04-25 09:58:09.806872911 +0000 2024-04-25T10:04:16.694 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: - 2024-04-25T10:04:16.694 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:04:16.747 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T10:04:16.748 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T10:04:16.748 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000248696 s, 2.1 MB/s 2024-04-25T10:04:16.749 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:04:16.799 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:16.799 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:16.799 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:04:16.805 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:04:16.805 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-04-25T10:04:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:04:16.851 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:04:16.851 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 864 Links: 1 2024-04-25T10:04:16.852 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:04:16.852 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-25 09:58:03.109714287 +0000 2024-04-25T10:04:16.852 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-25 09:58:02.861716502 +0000 2024-04-25T10:04:16.852 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-25 09:58:02.861716502 +0000 2024-04-25T10:04:16.852 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-04-25T10:04:16.852 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:04:16.906 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-25T10:04:16.906 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-25T10:04:16.906 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000440537 s, 1.2 MB/s 2024-04-25T10:04:16.907 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:04:16.956 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-04-25T10:04:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:04:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:04:17.003 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 878 Links: 1 2024-04-25T10:04:17.003 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:04:17.003 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-25 09:58:03.613709784 +0000 2024-04-25T10:04:17.003 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-25 09:58:03.337712250 +0000 2024-04-25T10:04:17.003 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-25 09:58:03.337712250 +0000 2024-04-25T10:04:17.003 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-04-25T10:04:17.003 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:04:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-25T10:04:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-25T10:04:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000417344 s, 1.2 MB/s 2024-04-25T10:04:17.059 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:04:17.108 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-04-25T10:04:17.155 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:04:17.156 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:04:17.156 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 900 Links: 1 2024-04-25T10:04:17.156 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:04:17.156 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-25 09:58:04.129705176 +0000 2024-04-25T10:04:17.156 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-25 09:58:03.849707676 +0000 2024-04-25T10:04:17.156 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-25 09:58:03.849707676 +0000 2024-04-25T10:04:17.156 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-04-25T10:04:17.156 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:04:17.209 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-25T10:04:17.209 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-25T10:04:17.209 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000434176 s, 1.2 MB/s 2024-04-25T10:04:17.210 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:04:17.260 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-25 09:58:04.597700996 +0000 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-25 09:58:04.345703246 +0000 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-25 09:58:04.345703246 +0000 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-04-25T10:04:17.308 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:04:17.361 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-25T10:04:17.361 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-25T10:04:17.362 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000293074 s, 1.7 MB/s 2024-04-25T10:04:17.363 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:04:17.412 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:17.412 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:04:17.412 INFO:tasks.ceph:Generating config... 2024-04-25T10:04:17.414 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T10:04:17.414 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T10:04:17.414 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T10:04:17.414 INFO:tasks.ceph:[client] pid file = 2024-04-25T10:04:17.414 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T10:04:17.414 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T10:04:17.414 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T10:04:17.414 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T10:04:17.414 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T10:04:17.415 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T10:04:17.415 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T10:04:17.415 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T10:04:17.415 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T10:04:17.415 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T10:04:17.415 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T10:04:17.415 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T10:04:17.415 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T10:04:17.415 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T10:04:17.415 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T10:04:17.415 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T10:04:17.415 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T10:04:17.415 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T10:04:17.415 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T10:04:17.415 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T10:04:17.415 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T10:04:17.416 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T10:04:17.416 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T10:04:17.416 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T10:04:17.416 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T10:04:17.416 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T10:04:17.416 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T10:04:17.416 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T10:04:17.416 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T10:04:17.416 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T10:04:17.416 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T10:04:17.416 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T10:04:17.416 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T10:04:17.416 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T10:04:17.416 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T10:04:17.417 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T10:04:17.453 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T10:04:17.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T10:04:17.550 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:04:17.606 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.60'), ('mon.b', '172.21.15.100'), ('mon.c', '[v2:172.21.15.100:3301,v1:172.21.15.100:6790]')] 2024-04-25T10:04:17.607 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T10:04:17.608 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:17.608 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:04:17.654 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.60 --add b 172.21.15.100 --addv c '[v2:172.21.15.100:3301,v1:172.21.15.100:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:17.741 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:17.741 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: generated fsid 0ea93863-a079-45c4-abed-6e83540ea9fe 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi060.stdout:setting min_mon_release = quincy 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 0 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi060.stdout:fsid 0ea93863-a079-45c4-abed-6e83540ea9fe 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi060.stdout:last_changed 2024-04-25T10:04:17.736562+0000 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi060.stdout:created 2024-04-25T10:04:17.736562+0000 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi060.stdout:min_mon_release 17 (quincy) 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi060.stdout:election_strategy: 1 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi060.stdout:0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi060.stdout:1: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.b 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi060.stdout:2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T10:04:17.744 DEBUG:teuthology.orchestra.run.smithi060:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:04:17.749 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0ea93863-a079-45c4-abed-6e83540ea9fe... 2024-04-25T10:04:17.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:04:17.794 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:04:17.823 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2024-04-25T10:04:17.823 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2024-04-25T10:04:17.823 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:04:17.823 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2024-04-25T10:04:17.823 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.823 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2024-04-25T10:04:17.823 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.823 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2024-04-25T10:04:17.823 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.823 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2024-04-25T10:04:17.823 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2024-04-25T10:04:17.823 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.824 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2024-04-25T10:04:17.824 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2024-04-25T10:04:17.824 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.824 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:04:17.824 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2024-04-25T10:04:17.824 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.824 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:04:17.824 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:04:17.824 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.824 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2024-04-25T10:04:17.824 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.824 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2024-04-25T10:04:17.824 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2024-04-25T10:04:17.824 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:04:17.825 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:04:17.825 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2024-04-25T10:04:17.825 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:04:17.825 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2024-04-25T10:04:17.825 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:04:17.825 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:04:17.825 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2024-04-25T10:04:17.825 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.825 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:04:17.825 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.825 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2024-04-25T10:04:17.825 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.825 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2024-04-25T10:04:17.825 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.825 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2024-04-25T10:04:17.826 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2024-04-25T10:04:17.826 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2024-04-25T10:04:17.826 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = "172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2024-04-25T10:04:17.826 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = 0ea93863-a079-45c4-abed-6e83540ea9fe 2024-04-25T10:04:17.826 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.826 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2024-04-25T10:04:17.826 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2024-04-25T10:04:17.826 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.826 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2024-04-25T10:04:17.826 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2024-04-25T10:04:17.826 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:04:17.826 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.826 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2024-04-25T10:04:17.827 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2024-04-25T10:04:17.827 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.827 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2024-04-25T10:04:17.827 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2024-04-25T10:04:17.827 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2024-04-25T10:04:17.827 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug trim objects = true 2024-04-25T10:04:17.827 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.827 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2024-04-25T10:04:17.827 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2024-04-25T10:04:17.827 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.827 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:04:17.827 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.827 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2024-04-25T10:04:17.827 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.827 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2024-04-25T10:04:17.828 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.828 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:04:17.828 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2024-04-25T10:04:17.828 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.828 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2024-04-25T10:04:17.828 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2024-04-25T10:04:17.828 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2024-04-25T10:04:17.828 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2024-04-25T10:04:17.828 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2024-04-25T10:04:17.828 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2024-04-25T10:04:17.828 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2024-04-25T10:04:17.828 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2024-04-25T10:04:17.828 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2024-04-25T10:04:17.828 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:04:17.828 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2024-04-25T10:04:17.829 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2024-04-25T10:04:17.829 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:04:17.829 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2024-04-25T10:04:17.829 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:04:17.829 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:04:17.829 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2024-04-25T10:04:17.829 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2024-04-25T10:04:17.829 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.829 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2024-04-25T10:04:17.829 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:04:17.829 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2024-04-25T10:04:17.829 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-04-25T10:04:17.829 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-04-25T10:04:17.829 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:04:17.829 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:04:17.830 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2024-04-25T10:04:17.830 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.830 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2024-04-25T10:04:17.830 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:04:17.830 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-04-25T10:04:17.830 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2024-04-25T10:04:17.830 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-04-25T10:04:17.830 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2024-04-25T10:04:17.830 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2024-04-25T10:04:17.830 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:04:17.830 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2024-04-25T10:04:17.830 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2024-04-25T10:04:17.830 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:04:17.831 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.831 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:04:17.831 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:04:17.831 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:04:17.831 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.831 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:04:17.831 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:04:17.831 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:04:17.831 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.831 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2024-04-25T10:04:17.831 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:04:17.831 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.831 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:04:17.831 INFO:teuthology.orchestra.run.smithi060.stdout: mon op complaint time = 120 2024-04-25T10:04:17.832 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:17.832 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2024-04-25T10:04:17.832 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2024-04-25T10:04:17.832 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2024-04-25T10:04:17.832 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2024-04-25T10:04:17.832 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:04:17.832 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:04:17.832 INFO:teuthology.orchestra.run.smithi060.stdout: client mount timeout = 600 2024-04-25T10:04:17.832 INFO:teuthology.orchestra.run.smithi060.stdout: debug client = 20 2024-04-25T10:04:17.832 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:04:17.832 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = "" 2024-04-25T10:04:17.832 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2024-04-25T10:04:17.832 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2024-04-25T10:04:17.832 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2024-04-25T10:04:17.832 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.b] 2024-04-25T10:04:17.833 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.c] 2024-04-25T10:04:17.833 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2024-04-25T10:04:17.833 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2024-04-25T10:04:17.833 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2024-04-25T10:04:17.833 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2024-04-25T10:04:17.833 INFO:teuthology.orchestra.run.smithi060.stdout:[mds] 2024-04-25T10:04:17.833 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds = 20 2024-04-25T10:04:17.833 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds balancer = 20 2024-04-25T10:04:17.833 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:04:17.833 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug frag = True 2024-04-25T10:04:17.833 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug scatterstat = True 2024-04-25T10:04:17.833 INFO:teuthology.orchestra.run.smithi060.stdout: mds op complaint time = 180 2024-04-25T10:04:17.833 INFO:teuthology.orchestra.run.smithi060.stdout: mds verify scatter = True 2024-04-25T10:04:17.833 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2024-04-25T10:04:17.833 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2024-04-25T10:04:17.834 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2024-04-25T10:04:17.835 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-04-25T10:04:17.835 INFO:teuthology.orchestra.run.smithi100.stdout: chdir = "" 2024-04-25T10:04:17.835 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:04:17.835 INFO:teuthology.orchestra.run.smithi100.stdout: auth supported = cephx 2024-04-25T10:04:17.835 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.835 INFO:teuthology.orchestra.run.smithi100.stdout: filestore xattr use omap = true 2024-04-25T10:04:17.836 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.836 INFO:teuthology.orchestra.run.smithi100.stdout: mon clock drift allowed = 1.000 2024-04-25T10:04:17.836 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.836 INFO:teuthology.orchestra.run.smithi100.stdout: osd crush chooseleaf type = 0 2024-04-25T10:04:17.836 INFO:teuthology.orchestra.run.smithi100.stdout: auth debug = true 2024-04-25T10:04:17.836 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.836 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on old message = true 2024-04-25T10:04:17.836 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on bug = true 2024-04-25T10:04:17.836 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.836 INFO:teuthology.orchestra.run.smithi100.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:04:17.836 INFO:teuthology.orchestra.run.smithi100.stdout: mon pg warn max object skew = 0 2024-04-25T10:04:17.836 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.836 INFO:teuthology.orchestra.run.smithi100.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:04:17.837 INFO:teuthology.orchestra.run.smithi100.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:04:17.837 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.837 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default size = 2 2024-04-25T10:04:17.837 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.837 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow primary affinity = true 2024-04-25T10:04:17.837 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow pg remap = true 2024-04-25T10:04:17.837 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:04:17.837 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:04:17.837 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on no sortbitwise = false 2024-04-25T10:04:17.837 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:04:17.837 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on too few osds = false 2024-04-25T10:04:17.837 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:04:17.837 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:04:17.838 INFO:teuthology.orchestra.run.smithi100.stdout: mon_allow_pool_size_one = true 2024-04-25T10:04:17.838 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.838 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:04:17.838 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.838 INFO:teuthology.orchestra.run.smithi100.stdout: osd default data pool replay window = 5 2024-04-25T10:04:17.838 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.838 INFO:teuthology.orchestra.run.smithi100.stdout: mon allow pool delete = true 2024-04-25T10:04:17.838 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.838 INFO:teuthology.orchestra.run.smithi100.stdout: mon cluster log file level = debug 2024-04-25T10:04:17.838 INFO:teuthology.orchestra.run.smithi100.stdout: debug asserts on shutdown = true 2024-04-25T10:04:17.838 INFO:teuthology.orchestra.run.smithi100.stdout: mon health detail to clog = false 2024-04-25T10:04:17.838 INFO:teuthology.orchestra.run.smithi100.stdout: mon host = "172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2024-04-25T10:04:17.838 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = 0ea93863-a079-45c4-abed-6e83540ea9fe 2024-04-25T10:04:17.839 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.839 INFO:teuthology.orchestra.run.smithi100.stdout:[osd] 2024-04-25T10:04:17.839 INFO:teuthology.orchestra.run.smithi100.stdout: osd journal size = 100 2024-04-25T10:04:17.839 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.839 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub load threshold = 5.0 2024-04-25T10:04:17.839 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub max interval = 600 2024-04-25T10:04:17.839 INFO:teuthology.orchestra.run.smithi100.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:04:17.839 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.839 INFO:teuthology.orchestra.run.smithi100.stdout: osd recover clone overlap = true 2024-04-25T10:04:17.839 INFO:teuthology.orchestra.run.smithi100.stdout: osd recovery max chunk = 1048576 2024-04-25T10:04:17.839 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.839 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug shutdown = true 2024-04-25T10:04:17.839 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug op order = true 2024-04-25T10:04:17.840 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify stray on activate = true 2024-04-25T10:04:17.840 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug trim objects = true 2024-04-25T10:04:17.840 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.840 INFO:teuthology.orchestra.run.smithi100.stdout: osd open classes on start = true 2024-04-25T10:04:17.840 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug pg log writeout = true 2024-04-25T10:04:17.840 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.840 INFO:teuthology.orchestra.run.smithi100.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:04:17.840 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.840 INFO:teuthology.orchestra.run.smithi100.stdout: osd map max advance = 10 2024-04-25T10:04:17.840 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.840 INFO:teuthology.orchestra.run.smithi100.stdout: journal zero on create = true 2024-04-25T10:04:17.840 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.840 INFO:teuthology.orchestra.run.smithi100.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:04:17.840 INFO:teuthology.orchestra.run.smithi100.stdout: filestore apply finisher threads = 3 2024-04-25T10:04:17.841 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.841 INFO:teuthology.orchestra.run.smithi100.stdout: bdev debug aio = true 2024-04-25T10:04:17.841 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug misdirected ops = true 2024-04-25T10:04:17.841 INFO:teuthology.orchestra.run.smithi100.stdout: bdev async discard = True 2024-04-25T10:04:17.841 INFO:teuthology.orchestra.run.smithi100.stdout: bdev enable discard = True 2024-04-25T10:04:17.841 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore allocator = bitmap 2024-04-25T10:04:17.841 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore block size = 96636764160 2024-04-25T10:04:17.841 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore fsck on mount = True 2024-04-25T10:04:17.841 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluefs = 1/20 2024-04-25T10:04:17.841 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluestore = 1/20 2024-04-25T10:04:17.841 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:04:17.841 INFO:teuthology.orchestra.run.smithi100.stdout: debug osd = 20 2024-04-25T10:04:17.841 INFO:teuthology.orchestra.run.smithi100.stdout: debug rocksdb = 4/10 2024-04-25T10:04:17.842 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:04:17.842 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd full ratio = 0.9 2024-04-25T10:04:17.842 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:04:17.842 INFO:teuthology.orchestra.run.smithi100.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:04:17.842 INFO:teuthology.orchestra.run.smithi100.stdout: osd objectstore = bluestore 2024-04-25T10:04:17.842 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-04-25T10:04:17.842 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.842 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr] 2024-04-25T10:04:17.842 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:04:17.842 INFO:teuthology.orchestra.run.smithi100.stdout: debug mgr = 20 2024-04-25T10:04:17.842 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-04-25T10:04:17.842 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-04-25T10:04:17.842 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:04:17.842 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:04:17.843 INFO:teuthology.orchestra.run.smithi100.stdout: mgr/telemetry/nag = false 2024-04-25T10:04:17.843 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.843 INFO:teuthology.orchestra.run.smithi100.stdout:[mon] 2024-04-25T10:04:17.843 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:04:17.843 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-04-25T10:04:17.843 INFO:teuthology.orchestra.run.smithi100.stdout: debug paxos = 20 2024-04-25T10:04:17.843 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-04-25T10:04:17.843 INFO:teuthology.orchestra.run.smithi100.stdout: mon data avail warn = 5 2024-04-25T10:04:17.843 INFO:teuthology.orchestra.run.smithi100.stdout: mon mgr mkfs grace = 240 2024-04-25T10:04:17.843 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:04:17.843 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd reporter subtree level = osd 2024-04-25T10:04:17.843 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd prime pg temp = true 2024-04-25T10:04:17.843 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:04:17.843 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.844 INFO:teuthology.orchestra.run.smithi100.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:04:17.844 INFO:teuthology.orchestra.run.smithi100.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:04:17.844 INFO:teuthology.orchestra.run.smithi100.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:04:17.844 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.844 INFO:teuthology.orchestra.run.smithi100.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:04:17.844 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:04:17.844 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:04:17.844 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.844 INFO:teuthology.orchestra.run.smithi100.stdout: # 1m isn't quite enough 2024-04-25T10:04:17.844 INFO:teuthology.orchestra.run.smithi100.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:04:17.844 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.844 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:04:17.844 INFO:teuthology.orchestra.run.smithi100.stdout: mon op complaint time = 120 2024-04-25T10:04:17.844 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:04:17.844 INFO:teuthology.orchestra.run.smithi100.stdout:[client] 2024-04-25T10:04:17.845 INFO:teuthology.orchestra.run.smithi100.stdout: rgw cache enabled = true 2024-04-25T10:04:17.845 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable ops log = true 2024-04-25T10:04:17.845 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable usage log = true 2024-04-25T10:04:17.845 INFO:teuthology.orchestra.run.smithi100.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:04:17.845 INFO:teuthology.orchestra.run.smithi100.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:04:17.845 INFO:teuthology.orchestra.run.smithi100.stdout: client mount timeout = 600 2024-04-25T10:04:17.845 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2024-04-25T10:04:17.845 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:04:17.845 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = "" 2024-04-25T10:04:17.845 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-04-25T10:04:17.845 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-04-25T10:04:17.845 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.a] 2024-04-25T10:04:17.845 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.b] 2024-04-25T10:04:17.845 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.c] 2024-04-25T10:04:17.845 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-04-25T10:04:17.846 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-04-25T10:04:17.846 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-04-25T10:04:17.846 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-04-25T10:04:17.846 INFO:teuthology.orchestra.run.smithi100.stdout:[mds] 2024-04-25T10:04:17.846 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds = 20 2024-04-25T10:04:17.846 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds balancer = 20 2024-04-25T10:04:17.846 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:04:17.846 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug frag = True 2024-04-25T10:04:17.846 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug scatterstat = True 2024-04-25T10:04:17.846 INFO:teuthology.orchestra.run.smithi100.stdout: mds op complaint time = 180 2024-04-25T10:04:17.846 INFO:teuthology.orchestra.run.smithi100.stdout: mds verify scatter = True 2024-04-25T10:04:17.846 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-04-25T10:04:17.846 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-04-25T10:04:17.847 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-04-25T10:04:17.847 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T10:04:17.847 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T10:04:17.928 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T10:04:17.928 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:17.928 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T10:04:17.934 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:17.934 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T10:04:17.981 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:04:17.981 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:17.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:04:17.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:04:18.046 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:18.047 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:18.095 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:04:18.095 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:18.095 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:04:18.095 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:04:18.119 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:18.119 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:18.167 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T10:04:18.168 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T10:04:18.168 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:04:18.212 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:04:18.215 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:04:18.259 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:04:18.270 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T10:04:18.270 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:04:18.314 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:04:18.317 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T10:04:18.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:04:18.460 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:04:18.463 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T10:04:18.517 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:04:18.563 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:04:18.566 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T10:04:18.620 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:04:18.704 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:04:18.707 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T10:04:18.762 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T10:04:18.763 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T10:04:18.799 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T10:04:18.810 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T10:04:18.886 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T10:04:18.898 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T10:04:18.934 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T10:04:18.946 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T10:04:19.017 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T10:04:19.027 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T10:04:19.028 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:04:19.028 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T10:04:19.042 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:19.042 INFO:tasks.ceph:role: osd.0 2024-04-25T10:04:19.042 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:04:19.042 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:04:19.098 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:04:19.098 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:04:19.098 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:04:19.098 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:04:19.098 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:04:19.099 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:04:19.099 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:04:19.099 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:04:19.099 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:04:19.099 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:04:19.123 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-04-25T10:04:19.125 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-04-25T10:04:19.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T10:04:19.183 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T10:04:19.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:19.239 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:04:19.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T10:04:19.291 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:19.291 INFO:tasks.ceph:role: osd.1 2024-04-25T10:04:19.291 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:04:19.291 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:04:19.350 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:04:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:04:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:04:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:04:19.350 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:04:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:04:19.350 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:04:19.350 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:04:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:04:19.350 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:04:19.374 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-04-25T10:04:19.375 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-04-25T10:04:19.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T10:04:19.432 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T10:04:19.485 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:04:19.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:19.486 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T10:04:19.539 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:19.539 INFO:tasks.ceph:role: osd.2 2024-04-25T10:04:19.539 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:04:19.539 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:04:19.596 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:04:19.596 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:04:19.596 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:04:19.596 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:04:19.596 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:04:19.596 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:04:19.596 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:04:19.597 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:04:19.597 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:04:19.597 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:04:19.620 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-04-25T10:04:19.621 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-04-25T10:04:19.622 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T10:04:19.680 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T10:04:19.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:19.730 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:04:19.730 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T10:04:19.783 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:19.783 INFO:tasks.ceph:role: osd.3 2024-04-25T10:04:19.783 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:04:19.785 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:04:19.836 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:04:19.836 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:04:19.836 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:04:19.837 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:04:19.837 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:04:19.837 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:04:19.837 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:04:19.837 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:04:19.837 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:04:19.837 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:04:19.860 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-04-25T10:04:19.862 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-04-25T10:04:19.862 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T10:04:19.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T10:04:19.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:19.978 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:04:19.978 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:20.083+0000 7f047b48e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T10:04:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:20.083+0000 7f047b48e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T10:04:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:20.083+0000 7f047b48e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:04:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:20.083+0000 7f047b48e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:04:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:20.083+0000 7f047b48e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:04:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:20.083+0000 7f047b48e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T10:04:23.054 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T10:04:23.071 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:23.147+0000 7f8889f857c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T10:04:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:23.147+0000 7f8889f857c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T10:04:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:23.147+0000 7f8889f857c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:04:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:23.147+0000 7f8889f857c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:04:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:23.147+0000 7f8889f857c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:04:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:23.147+0000 7f8889f857c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T10:04:26.122 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T10:04:26.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:26.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:26.255+0000 7f76793087c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T10:04:26.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:26.255+0000 7f76793087c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T10:04:26.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:26.255+0000 7f76793087c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:04:26.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:26.255+0000 7f76793087c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:04:26.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:26.255+0000 7f76793087c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:04:26.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:26.255+0000 7f76793087c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T10:04:29.264 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T10:04:29.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:29.395+0000 7feed99717c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T10:04:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:29.395+0000 7feed99717c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T10:04:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:29.395+0000 7feed99717c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:04:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:29.395+0000 7feed99717c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:04:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:29.395+0000 7feed99717c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:04:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:29.395+0000 7feed99717c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T10:04:32.373 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T10:04:32.430 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T10:04:32.445 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:32.446 INFO:tasks.ceph:role: osd.4 2024-04-25T10:04:32.446 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:04:32.446 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:04:32.507 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:04:32.507 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:04:32.507 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:04:32.507 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:04:32.507 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:04:32.507 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:04:32.507 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:04:32.507 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:04:32.507 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:04:32.507 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:04:32.968 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-04-25T10:04:32.970 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-04-25T10:04:32.970 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T10:04:33.069 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T10:04:33.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:33.080 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:04:33.081 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T10:04:33.134 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:33.134 INFO:tasks.ceph:role: osd.5 2024-04-25T10:04:33.134 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:04:33.134 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:04:33.203 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:04:33.203 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:04:33.203 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:04:33.203 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:04:33.203 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:04:33.203 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:04:33.203 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:04:33.203 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:04:33.203 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:04:33.204 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:04:33.669 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-04-25T10:04:33.670 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-04-25T10:04:33.671 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T10:04:33.782 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T10:04:33.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:33.795 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:04:33.795 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T10:04:33.850 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:33.850 INFO:tasks.ceph:role: osd.6 2024-04-25T10:04:33.851 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:04:33.851 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:04:33.915 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:04:33.915 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:04:33.915 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:04:33.915 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:04:33.915 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:04:33.915 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:04:33.915 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:04:33.916 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:04:33.916 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:04:33.916 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:04:34.417 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-04-25T10:04:34.419 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-04-25T10:04:34.420 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T10:04:34.563 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T10:04:34.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:34.575 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:04:34.575 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T10:04:34.631 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:04:34.631 INFO:tasks.ceph:role: osd.7 2024-04-25T10:04:34.631 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:04:34.631 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:04:34.694 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:04:34.695 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:04:34.695 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:04:34.695 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:04:34.695 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:04:34.695 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:04:34.695 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:04:34.695 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:04:34.695 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:04:34.695 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:04:35.142 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-04-25T10:04:35.144 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-04-25T10:04:35.144 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T10:04:35.243 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T10:04:35.251 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:04:35.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:35.252 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:35.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:35.359+0000 7fc32b8ad7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T10:04:35.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:35.359+0000 7fc32b8ad7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T10:04:35.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:35.359+0000 7fc32b8ad7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:04:35.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:35.359+0000 7fc32b8ad7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:04:35.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:35.359+0000 7fc32b8ad7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:04:35.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:35.363+0000 7fc32b8ad7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T10:04:38.344 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T10:04:38.357 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:38.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:38.431+0000 7f317ef5a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T10:04:38.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:38.431+0000 7f317ef5a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T10:04:38.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:38.431+0000 7f317ef5a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:04:38.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:38.431+0000 7f317ef5a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:04:38.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:38.431+0000 7f317ef5a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:04:38.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:38.431+0000 7f317ef5a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T10:04:41.406 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T10:04:41.463 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:41.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:41.539+0000 7efe891937c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T10:04:41.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:41.539+0000 7efe891937c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T10:04:41.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:41.539+0000 7efe891937c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:04:41.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:41.539+0000 7efe891937c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:04:41.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:41.539+0000 7efe891937c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:04:41.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:41.539+0000 7efe891937c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T10:04:44.529 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T10:04:44.582 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:44.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:44.659+0000 7f7e9cecc7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T10:04:44.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:44.659+0000 7f7e9cecc7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T10:04:44.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:44.659+0000 7f7e9cecc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:04:44.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:44.659+0000 7f7e9cecc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:04:44.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:44.659+0000 7f7e9cecc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:04:44.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:44.659+0000 7f7e9cecc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T10:04:46.083 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:46.086 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:47.639 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T10:04:47.652 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T10:04:47.653 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:47.653 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T10:04:47.667 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:47.667 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T10:04:47.722 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:47.722 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T10:04:47.777 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:47.777 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T10:04:47.833 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:47.833 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T10:04:47.888 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:47.888 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T10:04:47.946 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:47.946 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T10:04:48.000 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:48.001 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T10:04:48.014 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:48.014 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T10:04:48.069 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:48.069 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T10:04:48.121 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:48.121 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T10:04:48.173 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:48.174 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T10:04:48.229 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:48.229 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T10:04:48.281 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:48.282 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T10:04:48.335 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:48.335 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T10:04:48.341 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:48.341 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T10:04:48.390 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:48.390 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T10:04:48.395 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:48.395 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T10:04:48.442 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T10:04:48.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:04:48.445 DEBUG:teuthology.orchestra.run.smithi100:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:04:48.488 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr.y] 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCiKipmpl1cDBAAGqx2kIak9NoWpkcdHczcFA== 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCiKipmKr12EhAAnHHZ5VjG0cywMrIv6VElVg== 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCiKipmVsshGxAAttXv0jYEGKgo0sPhIRTuqQ== 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.0] 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCkKipmE+UIBRAAc8ctpHoL3PkkbkMRCAJUGw== 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.1] 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCnKipmbAnYCBAALLsPH2FjUpjoIZwTw+8aOA== 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.2] 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCqKipmqrNSDxAAZdgFrcyeVwRQMKJJYIM98g== 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.3] 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCtKipm1L+hFxAAPnPWlCm1Fmi0mN3tyReeQw== 2024-04-25T10:04:48.489 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr.x] 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCiKipmhxlLDxAAegBrJWdMNz6ZC3oXT2Qo0A== 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCiKipmU39rIRAAk6Ca0tKG4LpmR4pNxjzv0w== 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCiKipmQtXUKRAAI3Iq02h3KH/PZbWKvJD9Jg== 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.4] 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCzKipm2WyDFRAAmyfg6qu5drE9yMaR2FKd/A== 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.5] 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC2KipmZLDhGRAAcgaz8YaPGUMIfxOLtWObyA== 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.6] 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC5Kipm/3VMIBAA+rhGCyA/7d+s8m7ZWTzX/Q== 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.7] 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC8KipmFNCCJxAAv0OSTpAG+xIHhLqYXAZazQ== 2024-04-25T10:04:48.490 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T10:04:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCiKipmv65dLxAA6/QpPrqE9aT+CtZqkyutog== 2024-04-25T10:04:48.491 INFO:teuthology.orchestra.run.smithi060.stdout:[client.2] 2024-04-25T10:04:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCiKipmZRONNBAAa+XTv5YIBJWeUvf5dfoWxw== 2024-04-25T10:04:48.491 INFO:teuthology.orchestra.run.smithi060.stdout:[client.1] 2024-04-25T10:04:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCiKipm2NiONxAAd3X4zx1JnBXKHJvdZ/zMdQ== 2024-04-25T10:04:48.491 INFO:teuthology.orchestra.run.smithi060.stdout:[client.3] 2024-04-25T10:04:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCjKipmgvfiABAApLvTC9U6HzeAefCvX4tiSQ== 2024-04-25T10:04:48.497 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.y] 2024-04-25T10:04:48.497 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCiKipmpl1cDBAAGqx2kIak9NoWpkcdHczcFA== 2024-04-25T10:04:48.497 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-04-25T10:04:48.497 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCiKipmKr12EhAAnHHZ5VjG0cywMrIv6VElVg== 2024-04-25T10:04:48.497 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-04-25T10:04:48.497 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCiKipmVsshGxAAttXv0jYEGKgo0sPhIRTuqQ== 2024-04-25T10:04:48.497 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.0] 2024-04-25T10:04:48.497 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCkKipmE+UIBRAAc8ctpHoL3PkkbkMRCAJUGw== 2024-04-25T10:04:48.497 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.1] 2024-04-25T10:04:48.497 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCnKipmbAnYCBAALLsPH2FjUpjoIZwTw+8aOA== 2024-04-25T10:04:48.497 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.2] 2024-04-25T10:04:48.497 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCqKipmqrNSDxAAZdgFrcyeVwRQMKJJYIM98g== 2024-04-25T10:04:48.498 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.3] 2024-04-25T10:04:48.498 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCtKipm1L+hFxAAPnPWlCm1Fmi0mN3tyReeQw== 2024-04-25T10:04:48.498 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.x] 2024-04-25T10:04:48.498 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCiKipmhxlLDxAAegBrJWdMNz6ZC3oXT2Qo0A== 2024-04-25T10:04:48.498 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-04-25T10:04:48.498 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCiKipmU39rIRAAk6Ca0tKG4LpmR4pNxjzv0w== 2024-04-25T10:04:48.498 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-04-25T10:04:48.498 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCiKipmQtXUKRAAI3Iq02h3KH/PZbWKvJD9Jg== 2024-04-25T10:04:48.498 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.4] 2024-04-25T10:04:48.498 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCzKipm2WyDFRAAmyfg6qu5drE9yMaR2FKd/A== 2024-04-25T10:04:48.498 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.5] 2024-04-25T10:04:48.498 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQC2KipmZLDhGRAAcgaz8YaPGUMIfxOLtWObyA== 2024-04-25T10:04:48.498 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.6] 2024-04-25T10:04:48.499 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQC5Kipm/3VMIBAA+rhGCyA/7d+s8m7ZWTzX/Q== 2024-04-25T10:04:48.499 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.7] 2024-04-25T10:04:48.499 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQC8KipmFNCCJxAAv0OSTpAG+xIHhLqYXAZazQ== 2024-04-25T10:04:48.499 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-04-25T10:04:48.499 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCiKipmv65dLxAA6/QpPrqE9aT+CtZqkyutog== 2024-04-25T10:04:48.499 INFO:teuthology.orchestra.run.smithi100.stdout:[client.2] 2024-04-25T10:04:48.499 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCiKipmZRONNBAAa+XTv5YIBJWeUvf5dfoWxw== 2024-04-25T10:04:48.499 INFO:teuthology.orchestra.run.smithi100.stdout:[client.1] 2024-04-25T10:04:48.499 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCiKipm2NiONxAAd3X4zx1JnBXKHJvdZ/zMdQ== 2024-04-25T10:04:48.499 INFO:teuthology.orchestra.run.smithi100.stdout:[client.3] 2024-04-25T10:04:48.499 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCjKipmgvfiABAApLvTC9U6HzeAefCvX4tiSQ== 2024-04-25T10:04:48.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:04:48.534 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:04:48.586 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:04:48.589 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:04:48.635 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:04:48.679 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:04:48.722 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:48.766 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:48.806 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:48.850 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:48.893 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:48.934 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:48.978 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:48.981 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:49.055 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:04:49.062 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:04:49.137 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:04:49.146 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:04:49.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:04:49.222 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:04:49.310 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:49.313 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:49.352 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:49.394 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:49.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:49.478 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:49.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:49.563 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:04:49.603 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:04:49.647 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:04:49.693 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:04:49.695 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:04:49.737 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:04:49.778 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:04:49.824 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:04:49.827 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:04:49.906 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T10:04:49.907 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T10:04:49.923 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:04:50.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T10:04:50.025 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T10:04:50.039 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:04:50.133 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T10:04:50.145 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T10:04:50.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:04:50.289 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T10:04:50.300 DEBUG:teuthology.orchestra.run.smithi060:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:50.302 DEBUG:teuthology.orchestra.run.smithi100:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:04:50.346 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T10:04:50.347 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T10:04:50.347 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T10:04:50.350 INFO:tasks.ceph.mon.a:Started 2024-04-25T10:04:50.350 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T10:04:50.350 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T10:04:50.392 INFO:tasks.ceph.mon.b:Started 2024-04-25T10:04:50.392 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T10:04:50.392 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T10:04:50.394 INFO:tasks.ceph.mon.c:Started 2024-04-25T10:04:50.394 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T10:04:50.394 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T10:04:50.395 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T10:04:50.397 INFO:tasks.ceph.mgr.y:Started 2024-04-25T10:04:50.397 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T10:04:50.397 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T10:04:50.400 INFO:tasks.ceph.mgr.x:Started 2024-04-25T10:04:50.400 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T10:04:50.400 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T10:04:50.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.511+0000 7f3b37c08640 1 Processor -- start 2024-04-25T10:04:50.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.511+0000 7f3b37c08640 1 -- start start 2024-04-25T10:04:50.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.511+0000 7f3b37c08640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b300ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:50.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.511+0000 7f3b37c08640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b30070b30 con 0x7f3b300b5870 2024-04-25T10:04:50.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.511+0000 7f3b37c08640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b30161790 con 0x7f3b3015dc80 2024-04-25T10:04:50.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.511+0000 7f3b37c08640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b30161a70 con 0x7f3b3015d8a0 2024-04-25T10:04:50.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.511+0000 7f3b361a5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3b300b5870 0x7f3b300ce5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57176/0 (socket says 172.21.15.60:57176) 2024-04-25T10:04:50.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.511+0000 7f3b361a5640 1 -- 172.21.15.60:0/384420819 learned_addr learned my addr 172.21.15.60:0/384420819 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:04:50.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.511+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b300ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:50.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.511+0000 7f3b359a4640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 msgr2=0x7f3b300ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:50.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.511+0000 7f3b359a4640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 msgr2=0x7f3b300ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:50.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.511+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b300ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:50.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.511+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b300ccd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:50.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.715+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b300ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:50.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.715+0000 7f3b359a4640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 msgr2=0x7f3b300ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:50.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.715+0000 7f3b359a4640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 msgr2=0x7f3b300ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:50.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.715+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b300ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:50.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:50.715+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b300ccd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:04:51.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:51.119+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b300ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:51.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:51.119+0000 7f3b359a4640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 msgr2=0x7f3b300ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:51.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:51.119+0000 7f3b359a4640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 msgr2=0x7f3b300ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:51.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:51.119+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b300ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:51.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:51.119+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b300ccd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:04:51.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:51.923+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b300ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:51.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:51.923+0000 7f3b359a4640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 msgr2=0x7f3b300ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:51.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:51.923+0000 7f3b359a4640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 msgr2=0x7f3b300ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:51.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:51.923+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b300ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:51.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:51.923+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b300ccd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:04:53.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b277fe640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 msgr2=0x7f3b300ccd00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:04:53.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b277fe640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b300ccd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:53.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b277fe640 1 -- 172.21.15.60:0/384420819 >> v1:172.21.15.100:6789/0 conn(0x7f3b3015dc80 legacy=0x7f3b300b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:53.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b277fe640 1 -- 172.21.15.60:0/384420819 >> v1:172.21.15.60:6789/0 conn(0x7f3b300b5870 legacy=0x7f3b300ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:53.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b277fe640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 0x7f3b14005b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:53.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b277fe640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 0x7f3b140086c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b277fe640 1 -- 172.21.15.60:0/384420819 --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b30161a70 con 0x7f3b14006200 2024-04-25T10:04:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b277fe640 1 -- 172.21.15.60:0/384420819 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14008d80 con 0x7f3b14003720 2024-04-25T10:04:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b351a3640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 0x7f3b140086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 0x7f3b14005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b351a3640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 msgr2=0x7f3b140086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:04:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b351a3640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 msgr2=0x7f3b140086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b351a3640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 0x7f3b140086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b359a4640 1 -- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 msgr2=0x7f3b14005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b359a4640 1 -- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 msgr2=0x7f3b14005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b351a3640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 0x7f3b140086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 0x7f3b14005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:53.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.511+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 0x7f3b14005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:04:53.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.715+0000 7f3b351a3640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 0x7f3b140086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:53.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.715+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 0x7f3b14005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:53.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.715+0000 7f3b351a3640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 msgr2=0x7f3b140086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:53.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.715+0000 7f3b351a3640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 msgr2=0x7f3b140086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:53.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.715+0000 7f3b351a3640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 0x7f3b140086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:53.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.715+0000 7f3b351a3640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 0x7f3b140086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:04:53.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.715+0000 7f3b359a4640 1 -- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 msgr2=0x7f3b14005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:04:53.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.715+0000 7f3b359a4640 1 -- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 msgr2=0x7f3b14005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:53.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.715+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 0x7f3b14005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:53.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:53.715+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 0x7f3b14005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:04:54.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.115+0000 7f3b351a3640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 0x7f3b140086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:54.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.115+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 0x7f3b14005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:54.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.115+0000 7f3b351a3640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 msgr2=0x7f3b140086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:04:54.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.115+0000 7f3b351a3640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 msgr2=0x7f3b140086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:54.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.115+0000 7f3b351a3640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 0x7f3b140086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:54.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.115+0000 7f3b359a4640 1 -- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 msgr2=0x7f3b14005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:54.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.115+0000 7f3b359a4640 1 -- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 msgr2=0x7f3b14005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:54.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.115+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 0x7f3b14005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:54.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.115+0000 7f3b351a3640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 0x7f3b140086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:04:54.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.115+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 0x7f3b14005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:04:54.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.919+0000 7f3b351a3640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 0x7f3b140086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:54.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.919+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 0x7f3b14005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:54.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.919+0000 7f3b351a3640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 msgr2=0x7f3b140086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:04:54.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.919+0000 7f3b351a3640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 msgr2=0x7f3b140086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:54.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.919+0000 7f3b351a3640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 0x7f3b140086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:54.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.919+0000 7f3b359a4640 1 -- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 msgr2=0x7f3b14005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:04:54.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.919+0000 7f3b351a3640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 0x7f3b140086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:04:54.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.919+0000 7f3b359a4640 1 -- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 msgr2=0x7f3b14005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:04:54.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.919+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 0x7f3b14005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:04:54.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:54.919+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 0x7f3b14005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:04:55.990 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:55.983+0000 7f7287c22180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:04:56.064 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:56.058+0000 7f2f389e0180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:04:56.490 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:56.483+0000 7f7287c22180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b277fe640 1 -- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 msgr2=0x7f3b14005b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b277fe640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14003720 0x7f3b14005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b277fe640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 msgr2=0x7f3b140086c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b277fe640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14006200 0x7f3b140086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b277fe640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b1400b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b361a5640 1 -- 172.21.15.60:0/384420819 reap_dead start 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b277fe640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14003b60 0x7f3b14006630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b277fe640 1 -- 172.21.15.60:0/384420819 --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b30070800 con 0x7f3b14006b70 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b277fe640 1 -- 172.21.15.60:0/384420819 --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b30161a70 con 0x7f3b14003b60 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b277fe640 1 -- 172.21.15.60:0/384420819 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b14008d80 con 0x7f3b3015d8a0 2024-04-25T10:04:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b351a3640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.60:3300/0 conn(0x7f3b14003b60 0x7f3b14006630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b359a4640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b1400b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b349a2640 1 -- 172.21.15.60:0/384420819 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240058737 0 0) 0x7f3b30070800 con 0x7f3b14006b70 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b351a3640 1 -- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 msgr2=0x7f3b1400b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b351a3640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b1400b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b351a3640 1 -- 172.21.15.60:0/384420819 >> v1:172.21.15.60:6789/0 conn(0x7f3b14006b70 legacy=0x7f3b14007020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.511+0000 7f3b351a3640 1 -- 172.21.15.60:0/384420819 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b14008940 con 0x7f3b14003b60 2024-04-25T10:04:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.519+0000 7f3b351a3640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b14003b60 0x7f3b14006630 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3b30070750 tx=0x7f3b2c01cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d6f6c6d06b8294 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.519+0000 7f3b349a2640 1 -- 172.21.15.60:0/384420819 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c0ab070 con 0x7f3b14003b60 2024-04-25T10:04:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.519+0000 7f3b349a2640 1 -- 172.21.15.60:0/384420819 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c0a9070 con 0x7f3b14003b60 2024-04-25T10:04:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.519+0000 7f3b349a2640 1 -- 172.21.15.60:0/384420819 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c0a5d50 con 0x7f3b14003b60 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.519+0000 7f3b37c08640 1 -- 172.21.15.60:0/384420819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b14003b60 msgr2=0x7f3b14006630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.519+0000 7f3b37c08640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b14003b60 0x7f3b14006630 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3b30070750 tx=0x7f3b2c01cc70 comp rx=0 tx=0).stop 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.519+0000 7f3b37c08640 1 -- 172.21.15.60:0/384420819 shutdown_connections 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.519+0000 7f3b37c08640 1 --2- 172.21.15.60:0/384420819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b14003b60 0x7f3b14006630 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.519+0000 7f3b37c08640 1 --2- 172.21.15.60:0/384420819 >> v2:172.21.15.100:3300/0 conn(0x7f3b3015d8a0 0x7f3b1400b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.519+0000 7f3b37c08640 1 -- 172.21.15.60:0/384420819 >> 172.21.15.60:0/384420819 conn(0x7f3b3006e400 msgr2=0x7f3b3006e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.519+0000 7f3b37c08640 1 -- 172.21.15.60:0/384420819 shutdown_connections 2024-04-25T10:04:56.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.519+0000 7f3b37c08640 1 -- 172.21.15.60:0/384420819 wait complete. 2024-04-25T10:04:56.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b37c08640 1 Processor -- start 2024-04-25T10:04:56.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b37c08640 1 -- start start 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b37c08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b14003b60 0x7f3b30158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b37c08640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14006b70 0x7f3b3014f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b37c08640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3015d8a0 0x7f3b301543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b37c08640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b30162270 con 0x7f3b14003b60 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b37c08640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b300aff70 con 0x7f3b3015d8a0 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b37c08640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b301623f0 con 0x7f3b14006b70 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b359a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b14003b60 0x7f3b30158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b359a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b14003b60 0x7f3b30158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38328/0 (socket says 172.21.15.60:38328) 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b359a4640 1 -- 172.21.15.60:0/2192466861 learned_addr learned my addr 172.21.15.60:0/2192466861 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b361a5640 1 --2- 172.21.15.60:0/2192466861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3015d8a0 0x7f3b301543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b351a3640 1 --2- 172.21.15.60:0/2192466861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14006b70 0x7f3b3014f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b359a4640 1 -- 172.21.15.60:0/2192466861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14006b70 msgr2=0x7f3b3014f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b359a4640 1 --2- 172.21.15.60:0/2192466861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14006b70 0x7f3b3014f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b359a4640 1 -- 172.21.15.60:0/2192466861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3015d8a0 msgr2=0x7f3b301543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b359a4640 1 --2- 172.21.15.60:0/2192466861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3015d8a0 0x7f3b301543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.523+0000 7f3b359a4640 1 -- 172.21.15.60:0/2192466861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b30153300 con 0x7f3b14003b60 2024-04-25T10:04:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.527+0000 7f3b359a4640 1 --2- 172.21.15.60:0/2192466861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b14003b60 0x7f3b30158bc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3b200186a0 tx=0x7f3b20018b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.527+0000 7f3b267fc640 1 -- 172.21.15.60:0/2192466861 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20028070 con 0x7f3b14003b60 2024-04-25T10:04:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.527+0000 7f3b267fc640 1 -- 172.21.15.60:0/2192466861 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b20016d60 con 0x7f3b14003b60 2024-04-25T10:04:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.527+0000 7f3b267fc640 1 -- 172.21.15.60:0/2192466861 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2000f040 con 0x7f3b14003b60 2024-04-25T10:04:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.527+0000 7f3b37c08640 1 -- 172.21.15.60:0/2192466861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b301535e0 con 0x7f3b14003b60 2024-04-25T10:04:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.527+0000 7f3b37c08640 1 -- 172.21.15.60:0/2192466861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b30153ba0 con 0x7f3b14003b60 2024-04-25T10:04:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.527+0000 7f3b267fc640 1 -- 172.21.15.60:0/2192466861 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3b2002b070 con 0x7f3b14003b60 2024-04-25T10:04:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.527+0000 7f3b267fc640 1 -- 172.21.15.60:0/2192466861 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3b2002db60 con 0x7f3b14003b60 2024-04-25T10:04:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.527+0000 7f3b37c08640 1 -- 172.21.15.60:0/2192466861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b30074770 con 0x7f3b14003b60 2024-04-25T10:04:56.542 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:56.534+0000 7f2f389e0180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:04:56.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.543+0000 7f3b267fc640 1 -- 172.21.15.60:0/2192466861 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3b20020070 con 0x7f3b14003b60 2024-04-25T10:04:56.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.607+0000 7f3b37c08640 1 -- 172.21.15.60:0/2192466861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3b300700d0 con 0x7f3b14003b60 2024-04-25T10:04:56.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.611+0000 7f3b267fc640 1 -- 172.21.15.60:0/2192466861 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3b200059a0 con 0x7f3b14003b60 2024-04-25T10:04:56.617 INFO:teuthology.orchestra.run.smithi060.stderr:adjusted tunables profile to default 2024-04-25T10:04:56.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.611+0000 7f3b37c08640 1 -- 172.21.15.60:0/2192466861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b14003b60 msgr2=0x7f3b30158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:56.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.611+0000 7f3b37c08640 1 --2- 172.21.15.60:0/2192466861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b14003b60 0x7f3b30158bc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3b200186a0 tx=0x7f3b20018b60 comp rx=0 tx=0).stop 2024-04-25T10:04:56.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.611+0000 7f3b37c08640 1 -- 172.21.15.60:0/2192466861 shutdown_connections 2024-04-25T10:04:56.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.611+0000 7f3b37c08640 1 --2- 172.21.15.60:0/2192466861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3015d8a0 0x7f3b301543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.611+0000 7f3b37c08640 1 --2- 172.21.15.60:0/2192466861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b14006b70 0x7f3b3014f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.611+0000 7f3b37c08640 1 --2- 172.21.15.60:0/2192466861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b14003b60 0x7f3b30158bc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:56.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.611+0000 7f3b37c08640 1 -- 172.21.15.60:0/2192466861 >> 172.21.15.60:0/2192466861 conn(0x7f3b3006e400 msgr2=0x7f3b300b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:56.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.611+0000 7f3b37c08640 1 -- 172.21.15.60:0/2192466861 shutdown_connections 2024-04-25T10:04:56.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:56.611+0000 7f3b37c08640 1 -- 172.21.15.60:0/2192466861 wait complete. 2024-04-25T10:04:56.635 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T10:04:56.635 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T10:04:56.636 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:56.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T10:04:56.646 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:56.647 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T10:04:56.677 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:56.671+0000 7f7287c22180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:04:56.687 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:56.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T10:04:56.740 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:04:56.740 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T10:04:56.745 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:56.738+0000 7f2f389e0180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:04:56.792 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:56.792 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T10:04:56.804 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:56.804 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T10:04:56.858 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:56.858 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T10:04:56.915 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:04:56.915 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T10:04:56.970 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new e461b596-4289-46c7-baac-b1127e3f8b0f 0 2024-04-25T10:04:57.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.126+0000 7f3079825640 1 Processor -- start 2024-04-25T10:04:57.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.126+0000 7f3079825640 1 -- start start 2024-04-25T10:04:57.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f3079825640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f307406f0c0 0x7f30740cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f3079825640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30740b4a20 con 0x7f307415aad0 2024-04-25T10:04:57.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f3079825640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30740727a0 con 0x7f307406bc20 2024-04-25T10:04:57.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f3079825640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30740b47d0 con 0x7f307406f0c0 2024-04-25T10:04:57.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f3072ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f307406bc20 0x7f307406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57346/0 (socket says 172.21.15.100:57346) 2024-04-25T10:04:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f3072ffd640 1 -- 172.21.15.100:0/4112808634 learned_addr learned my addr 172.21.15.100:0/4112808634 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-25T10:04:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30727fc640 1 -- 172.21.15.100:0/4112808634 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084514922 0 0) 0x7f30740727a0 con 0x7f307406bc20 2024-04-25T10:04:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30737fe640 1 --2- 172.21.15.100:0/4112808634 >> v2:172.21.15.60:3300/0 conn(0x7f307406f0c0 0x7f30740cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30727fc640 1 -- 172.21.15.100:0/4112808634 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3060005740 con 0x7f307406bc20 2024-04-25T10:04:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30727fc640 1 -- 172.21.15.100:0/4112808634 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3260185116 0 0) 0x7f3060005740 con 0x7f307406bc20 2024-04-25T10:04:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30727fc640 1 -- 172.21.15.100:0/4112808634 >> v2:172.21.15.60:3300/0 conn(0x7f307406f0c0 msgr2=0x7f30740cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30727fc640 1 --2- 172.21.15.100:0/4112808634 >> v2:172.21.15.60:3300/0 conn(0x7f307406f0c0 0x7f30740cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30727fc640 1 -- 172.21.15.100:0/4112808634 >> v1:172.21.15.60:6789/0 conn(0x7f307415aad0 legacy=0x7f30740b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30727fc640 1 -- 172.21.15.100:0/4112808634 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074072a80 con 0x7f307406bc20 2024-04-25T10:04:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30727fc640 1 -- 172.21.15.100:0/4112808634 <== mon.1 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f3064001680 con 0x7f307406bc20 2024-04-25T10:04:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30727fc640 1 -- 172.21.15.100:0/4112808634 >> v1:172.21.15.100:6789/0 conn(0x7f307406bc20 legacy=0x7f307406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30727fc640 1 --2- 172.21.15.100:0/4112808634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3060005bd0 0x7f3060005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30727fc640 1 --2- 172.21.15.100:0/4112808634 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3060006a20 0x7f3060006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30727fc640 1 --2- 172.21.15.100:0/4112808634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30600073e0 0x7f306000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30727fc640 1 -- 172.21.15.100:0/4112808634 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30740b47d0 con 0x7f3060005bd0 2024-04-25T10:04:57.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30727fc640 1 -- 172.21.15.100:0/4112808634 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306000e380 con 0x7f30600073e0 2024-04-25T10:04:57.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30727fc640 1 -- 172.21.15.100:0/4112808634 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306000e580 con 0x7f3060006a20 2024-04-25T10:04:57.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30737fe640 1 --2- 172.21.15.100:0/4112808634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30600073e0 0x7f306000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f306bfff640 1 --2- 172.21.15.100:0/4112808634 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3060006a20 0x7f3060006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f3072ffd640 1 --2- 172.21.15.100:0/4112808634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3060005bd0 0x7f3060005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30737fe640 1 -- 172.21.15.100:0/4112808634 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3060006a20 msgr2=0x7f3060006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:57.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30737fe640 1 --2- 172.21.15.100:0/4112808634 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3060006a20 0x7f3060006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30737fe640 1 -- 172.21.15.100:0/4112808634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3060005bd0 msgr2=0x7f3060005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:57.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30737fe640 1 --2- 172.21.15.100:0/4112808634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3060005bd0 0x7f3060005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.130+0000 7f30737fe640 1 -- 172.21.15.100:0/4112808634 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3060005740 con 0x7f30600073e0 2024-04-25T10:04:57.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f30737fe640 1 -- 172.21.15.100:0/4112808634 reap_dead start 2024-04-25T10:04:57.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f30737fe640 1 --2- 172.21.15.100:0/4112808634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30600073e0 0x7f306000dbe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f306c009fc0 tx=0x7f306c01cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66e16cf417ec0c4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:57.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f30727fc640 1 -- 172.21.15.100:0/4112808634 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c009c10 con 0x7f30600073e0 2024-04-25T10:04:57.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f30727fc640 1 -- 172.21.15.100:0/4112808634 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c0a5ce0 con 0x7f30600073e0 2024-04-25T10:04:57.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f30727fc640 1 -- 172.21.15.100:0/4112808634 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c003be0 con 0x7f30600073e0 2024-04-25T10:04:57.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3079825640 1 -- 172.21.15.100:0/4112808634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30600073e0 msgr2=0x7f306000dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3079825640 1 --2- 172.21.15.100:0/4112808634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30600073e0 0x7f306000dbe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f306c009fc0 tx=0x7f306c01cc90 comp rx=0 tx=0).stop 2024-04-25T10:04:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3079825640 1 -- 172.21.15.100:0/4112808634 shutdown_connections 2024-04-25T10:04:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3079825640 1 --2- 172.21.15.100:0/4112808634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30600073e0 0x7f306000dbe0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3079825640 1 -- 172.21.15.100:0/4112808634 >> 172.21.15.100:0/4112808634 conn(0x7f3074071a00 msgr2=0x7f3074071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3079825640 1 -- 172.21.15.100:0/4112808634 shutdown_connections 2024-04-25T10:04:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3079825640 1 -- 172.21.15.100:0/4112808634 wait complete. 2024-04-25T10:04:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3079825640 1 Processor -- start 2024-04-25T10:04:57.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3079825640 1 -- start start 2024-04-25T10:04:57.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3079825640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30600073e0 0x7f3074156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3079825640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30740afe20 0x7f30740b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3079825640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f307415c070 0x7f307415c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3079825640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30740ae980 con 0x7f30740afe20 2024-04-25T10:04:57.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3079825640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30740ae800 con 0x7f30600073e0 2024-04-25T10:04:57.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3079825640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30740cc110 con 0x7f307415c070 2024-04-25T10:04:57.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3072ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30600073e0 0x7f3074156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3072ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30600073e0 0x7f3074156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35944/0 (socket says 172.21.15.100:35944) 2024-04-25T10:04:57.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f30737fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f307415c070 0x7f307415c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f3072ffd640 1 -- 172.21.15.100:0/3324069621 learned_addr learned my addr 172.21.15.100:0/3324069621 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T10:04:57.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.134+0000 7f306bfff640 1 --2- 172.21.15.100:0/3324069621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30740afe20 0x7f30740b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.138+0000 7f30737fe640 1 -- 172.21.15.100:0/3324069621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30600073e0 msgr2=0x7f3074156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.138+0000 7f30737fe640 1 --2- 172.21.15.100:0/3324069621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30600073e0 0x7f3074156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.138+0000 7f30737fe640 1 -- 172.21.15.100:0/3324069621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30740afe20 msgr2=0x7f30740b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.138+0000 7f30737fe640 1 --2- 172.21.15.100:0/3324069621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30740afe20 0x7f30740b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.138+0000 7f30737fe640 1 -- 172.21.15.100:0/3324069621 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30740cc620 con 0x7f307415c070 2024-04-25T10:04:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.138+0000 7f3072ffd640 1 --2- 172.21.15.100:0/3324069621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30600073e0 0x7f3074156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.138+0000 7f30737fe640 1 --2- 172.21.15.100:0/3324069621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f307415c070 0x7f307415c500 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f307415f490 tx=0x7f306c009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.138+0000 7f3070ff9640 1 -- 172.21.15.100:0/3324069621 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c0be070 con 0x7f307415c070 2024-04-25T10:04:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.138+0000 7f3070ff9640 1 -- 172.21.15.100:0/3324069621 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c0039a0 con 0x7f307415c070 2024-04-25T10:04:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.138+0000 7f3079825640 1 -- 172.21.15.100:0/3324069621 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30740cc900 con 0x7f307415c070 2024-04-25T10:04:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.138+0000 7f3070ff9640 1 -- 172.21.15.100:0/3324069621 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c003d50 con 0x7f307415c070 2024-04-25T10:04:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.138+0000 7f3079825640 1 -- 172.21.15.100:0/3324069621 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30740ccec0 con 0x7f307415c070 2024-04-25T10:04:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.138+0000 7f3070ff9640 1 -- 172.21.15.100:0/3324069621 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f306c0c1070 con 0x7f307415c070 2024-04-25T10:04:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.138+0000 7f3070ff9640 1 -- 172.21.15.100:0/3324069621 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f306c0047a0 con 0x7f307415c070 2024-04-25T10:04:57.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.138+0000 7f3079825640 1 -- 172.21.15.100:0/3324069621 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3074066d90 con 0x7f307415c070 2024-04-25T10:04:57.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.150+0000 7f3070ff9640 1 -- 172.21.15.100:0/3324069621 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f306c0c8020 con 0x7f307415c070 2024-04-25T10:04:57.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.218+0000 7f3079825640 1 -- 172.21.15.100:0/3324069621 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e461b596-4289-46c7-baac-b1127e3f8b0f", "id": 0} v 0) v1 -- 0x7f307415e410 con 0x7f307415c070 2024-04-25T10:04:57.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.222+0000 7f3070ff9640 1 -- 172.21.15.100:0/3324069621 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e461b596-4289-46c7-baac-b1127e3f8b0f", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f306c0169d0 con 0x7f307415c070 2024-04-25T10:04:57.229 INFO:teuthology.orchestra.run.smithi100.stdout:0 2024-04-25T10:04:57.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.222+0000 7f3079825640 1 -- 172.21.15.100:0/3324069621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f307415c070 msgr2=0x7f307415c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.222+0000 7f3079825640 1 --2- 172.21.15.100:0/3324069621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f307415c070 0x7f307415c500 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f307415f490 tx=0x7f306c009750 comp rx=0 tx=0).stop 2024-04-25T10:04:57.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.226+0000 7f3079825640 1 -- 172.21.15.100:0/3324069621 shutdown_connections 2024-04-25T10:04:57.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.226+0000 7f3079825640 1 --2- 172.21.15.100:0/3324069621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f307415c070 0x7f307415c500 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.226+0000 7f3079825640 1 --2- 172.21.15.100:0/3324069621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30740afe20 0x7f30740b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.226+0000 7f3079825640 1 --2- 172.21.15.100:0/3324069621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30600073e0 0x7f3074156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.226+0000 7f3079825640 1 -- 172.21.15.100:0/3324069621 >> 172.21.15.100:0/3324069621 conn(0x7f3074071a00 msgr2=0x7f30741545c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:57.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.226+0000 7f3079825640 1 -- 172.21.15.100:0/3324069621 shutdown_connections 2024-04-25T10:04:57.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.226+0000 7f3079825640 1 -- 172.21.15.100:0/3324069621 wait complete. 2024-04-25T10:04:57.246 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 4a2d4b54-a6f8-41f5-a25c-f06886b279dc 1 2024-04-25T10:04:57.247 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:57.239+0000 7f7287c22180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:04:57.301 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:57.294+0000 7f2f389e0180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:04:57.303 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:57.295+0000 7f7287c22180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:04:57.356 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:57.350+0000 7f2f389e0180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:04:57.358 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:57.351+0000 7f7287c22180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:04:57.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.358+0000 7f18162b5640 1 Processor -- start 2024-04-25T10:04:57.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.358+0000 7f18162b5640 1 -- start start 2024-04-25T10:04:57.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.358+0000 7f18162b5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18101528d0 0x7f18100cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.358+0000 7f18162b5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f181006b320 0x7f181006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.358+0000 7f18162b5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f181015c290 con 0x7f18100cd380 2024-04-25T10:04:57.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.358+0000 7f18162b5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18100cd7c0 con 0x7f181006b320 2024-04-25T10:04:57.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.358+0000 7f18162b5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18100cf620 con 0x7f18101528d0 2024-04-25T10:04:57.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.358+0000 7f180ffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18101528d0 0x7f18100cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.358+0000 7f180ffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18101528d0 0x7f18100cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:41264/0 (socket says 172.21.15.100:41264) 2024-04-25T10:04:57.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.358+0000 7f180ffff640 1 -- 172.21.15.100:0/3050493645 learned_addr learned my addr 172.21.15.100:0/3050493645 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T10:04:57.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.358+0000 7f1814852640 1 --2- 172.21.15.100:0/3050493645 >> v2:172.21.15.60:3300/0 conn(0x7f181006b320 0x7f181006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.358+0000 7f180ffff640 1 -- 172.21.15.100:0/3050493645 >> v2:172.21.15.60:3300/0 conn(0x7f181006b320 msgr2=0x7f181006fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:57.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.358+0000 7f180ffff640 1 --2- 172.21.15.100:0/3050493645 >> v2:172.21.15.60:3300/0 conn(0x7f181006b320 0x7f181006fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.358+0000 7f180ffff640 1 -- 172.21.15.100:0/3050493645 >> v1:172.21.15.60:6789/0 conn(0x7f18100cd380 legacy=0x7f181006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:57.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.358+0000 7f180ffff640 1 -- 172.21.15.100:0/3050493645 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181015c570 con 0x7f18101528d0 2024-04-25T10:04:57.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.362+0000 7f180ffff640 1 --2- 172.21.15.100:0/3050493645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18101528d0 0x7f18100cce40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1808009e50 tx=0x7f180801c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b529b4b5b58a25a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:57.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.362+0000 7f180effd640 1 -- 172.21.15.100:0/3050493645 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18080a5760 con 0x7f18101528d0 2024-04-25T10:04:57.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.362+0000 7f180effd640 1 -- 172.21.15.100:0/3050493645 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18080a5900 con 0x7f18101528d0 2024-04-25T10:04:57.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.362+0000 7f180effd640 1 -- 172.21.15.100:0/3050493645 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18080a5cd0 con 0x7f18101528d0 2024-04-25T10:04:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.362+0000 7f18162b5640 1 -- 172.21.15.100:0/3050493645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18101528d0 msgr2=0x7f18100cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.362+0000 7f18162b5640 1 --2- 172.21.15.100:0/3050493645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18101528d0 0x7f18100cce40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1808009e50 tx=0x7f180801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.362+0000 7f18162b5640 1 -- 172.21.15.100:0/3050493645 shutdown_connections 2024-04-25T10:04:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.362+0000 7f18162b5640 1 --2- 172.21.15.100:0/3050493645 >> v2:172.21.15.60:3300/0 conn(0x7f181006b320 0x7f181006fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.362+0000 7f18162b5640 1 --2- 172.21.15.100:0/3050493645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18101528d0 0x7f18100cce40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.362+0000 7f18162b5640 1 -- 172.21.15.100:0/3050493645 >> 172.21.15.100:0/3050493645 conn(0x7f18100b2360 msgr2=0x7f18100b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.362+0000 7f18162b5640 1 -- 172.21.15.100:0/3050493645 shutdown_connections 2024-04-25T10:04:57.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.362+0000 7f18162b5640 1 -- 172.21.15.100:0/3050493645 wait complete. 2024-04-25T10:04:57.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.362+0000 7f18162b5640 1 Processor -- start 2024-04-25T10:04:57.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.362+0000 7f18162b5640 1 -- start start 2024-04-25T10:04:57.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f18162b5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f181006b320 0x7f181014c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f18162b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18100cd380 0x7f1810141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f18162b5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18101528d0 0x7f1810142c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f18162b5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1810152e30 con 0x7f181006b320 2024-04-25T10:04:57.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f18162b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1810152cb0 con 0x7f18100cd380 2024-04-25T10:04:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f18162b5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18100708b0 con 0x7f18101528d0 2024-04-25T10:04:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f180f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18100cd380 0x7f1810141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f1814852640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18101528d0 0x7f1810142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f180f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18100cd380 0x7f1810141a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35950/0 (socket says 172.21.15.100:35950) 2024-04-25T10:04:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f180f7fe640 1 -- 172.21.15.100:0/2566004707 learned_addr learned my addr 172.21.15.100:0/2566004707 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T10:04:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f180ffff640 1 --2- 172.21.15.100:0/2566004707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f181006b320 0x7f181014c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f180f7fe640 1 -- 172.21.15.100:0/2566004707 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18101528d0 msgr2=0x7f1810142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f180f7fe640 1 --2- 172.21.15.100:0/2566004707 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18101528d0 0x7f1810142c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f180f7fe640 1 -- 172.21.15.100:0/2566004707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f181006b320 msgr2=0x7f181014c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f180f7fe640 1 --2- 172.21.15.100:0/2566004707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f181006b320 0x7f181014c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f180f7fe640 1 -- 172.21.15.100:0/2566004707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181014c210 con 0x7f18100cd380 2024-04-25T10:04:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f1814852640 1 --2- 172.21.15.100:0/2566004707 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18101528d0 0x7f1810142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f180f7fe640 1 --2- 172.21.15.100:0/2566004707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18100cd380 0x7f1810141a20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f180400c490 tx=0x7f1804001e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:57.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f180cff9640 1 -- 172.21.15.100:0/2566004707 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804017070 con 0x7f18100cd380 2024-04-25T10:04:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f180cff9640 1 -- 172.21.15.100:0/2566004707 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1804003600 con 0x7f18100cd380 2024-04-25T10:04:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f180cff9640 1 -- 172.21.15.100:0/2566004707 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804003a30 con 0x7f18100cd380 2024-04-25T10:04:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f18162b5640 1 -- 172.21.15.100:0/2566004707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18101421b0 con 0x7f18100cd380 2024-04-25T10:04:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f18162b5640 1 -- 172.21.15.100:0/2566004707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1810142770 con 0x7f18100cd380 2024-04-25T10:04:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f18162b5640 1 -- 172.21.15.100:0/2566004707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1810074770 con 0x7f18100cd380 2024-04-25T10:04:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f180cff9640 1 -- 172.21.15.100:0/2566004707 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1804003bd0 con 0x7f18100cd380 2024-04-25T10:04:57.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.366+0000 7f180cff9640 1 -- 172.21.15.100:0/2566004707 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f18040127b0 con 0x7f18100cd380 2024-04-25T10:04:57.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.370+0000 7f180cff9640 1 -- 172.21.15.100:0/2566004707 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f18040067a0 con 0x7f18100cd380 2024-04-25T10:04:57.411 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:57.406+0000 7f2f389e0180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:04:57.439 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:57.431+0000 7f7287c22180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:04:57.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.438+0000 7f18162b5640 1 -- 172.21.15.100:0/2566004707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4a2d4b54-a6f8-41f5-a25c-f06886b279dc", "id": 1} v 0) v1 -- 0x7f1810142940 con 0x7f18100cd380 2024-04-25T10:04:57.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.442+0000 7f180cff9640 1 -- 172.21.15.100:0/2566004707 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4a2d4b54-a6f8-41f5-a25c-f06886b279dc", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1804006cd0 con 0x7f18100cd380 2024-04-25T10:04:57.448 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-25T10:04:57.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.442+0000 7f18162b5640 1 -- 172.21.15.100:0/2566004707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18100cd380 msgr2=0x7f1810141a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.442+0000 7f18162b5640 1 --2- 172.21.15.100:0/2566004707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18100cd380 0x7f1810141a20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f180400c490 tx=0x7f1804001e80 comp rx=0 tx=0).stop 2024-04-25T10:04:57.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.442+0000 7f18162b5640 1 -- 172.21.15.100:0/2566004707 shutdown_connections 2024-04-25T10:04:57.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.442+0000 7f18162b5640 1 --2- 172.21.15.100:0/2566004707 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18101528d0 0x7f1810142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.442+0000 7f18162b5640 1 --2- 172.21.15.100:0/2566004707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18100cd380 0x7f1810141a20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.442+0000 7f18162b5640 1 --2- 172.21.15.100:0/2566004707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f181006b320 0x7f181014c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.442+0000 7f18162b5640 1 -- 172.21.15.100:0/2566004707 >> 172.21.15.100:0/2566004707 conn(0x7f18100b2360 msgr2=0x7f181006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:57.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.442+0000 7f18162b5640 1 -- 172.21.15.100:0/2566004707 shutdown_connections 2024-04-25T10:04:57.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.442+0000 7f18162b5640 1 -- 172.21.15.100:0/2566004707 wait complete. 2024-04-25T10:04:57.464 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 316be201-bf30-46fc-a685-ddd534dec41d 2 2024-04-25T10:04:57.491 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:57.486+0000 7f2f389e0180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:04:57.495 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:57.487+0000 7f7287c22180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:04:57.545 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:57.538+0000 7f2f389e0180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:04:57.576 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:57.567+0000 7f7287c22180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:04:57.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.574+0000 7fdea734f640 1 Processor -- start 2024-04-25T10:04:57.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.574+0000 7fdea734f640 1 -- start start 2024-04-25T10:04:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.574+0000 7fdea734f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdea0145320 0x7fdea0070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.574+0000 7fdea734f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdea00cf140 0x7fdea00b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.574+0000 7fdea734f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea0145700 con 0x7fdea0071210 2024-04-25T10:04:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.574+0000 7fdea734f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea0071670 con 0x7fdea00cf140 2024-04-25T10:04:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.574+0000 7fdea734f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea00b4800 con 0x7fdea0145320 2024-04-25T10:04:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.574+0000 7fdea50eb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdea0145320 0x7fdea0070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.574+0000 7fdea50eb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdea0145320 0x7fdea0070cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35958/0 (socket says 172.21.15.100:35958) 2024-04-25T10:04:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.574+0000 7fdea50eb640 1 -- 172.21.15.100:0/2274809223 learned_addr learned my addr 172.21.15.100:0/2274809223 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T10:04:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.574+0000 7fdea58ec640 1 --2- 172.21.15.100:0/2274809223 >> v2:172.21.15.60:3300/0 conn(0x7fdea00cf140 0x7fdea00b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.574+0000 7fdea50eb640 1 -- 172.21.15.100:0/2274809223 >> v2:172.21.15.60:3300/0 conn(0x7fdea00cf140 msgr2=0x7fdea00b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.574+0000 7fdea50eb640 1 --2- 172.21.15.100:0/2274809223 >> v2:172.21.15.60:3300/0 conn(0x7fdea00cf140 0x7fdea00b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.574+0000 7fdea50eb640 1 -- 172.21.15.100:0/2274809223 >> v1:172.21.15.60:6789/0 conn(0x7fdea0071210 legacy=0x7fdea00cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.574+0000 7fdea50eb640 1 -- 172.21.15.100:0/2274809223 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea00d0bd0 con 0x7fdea0145320 2024-04-25T10:04:57.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.578+0000 7fdea50eb640 1 --2- 172.21.15.100:0/2274809223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdea0145320 0x7fdea0070cd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdea0073150 tx=0x7fde9c01c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b17ec433ad0dd4a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:57.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.578+0000 7fde97fff640 1 -- 172.21.15.100:0/2274809223 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c0ab620 con 0x7fdea0145320 2024-04-25T10:04:57.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.578+0000 7fde97fff640 1 -- 172.21.15.100:0/2274809223 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9c0b0070 con 0x7fdea0145320 2024-04-25T10:04:57.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.578+0000 7fde97fff640 1 -- 172.21.15.100:0/2274809223 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c0ab9e0 con 0x7fdea0145320 2024-04-25T10:04:57.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.578+0000 7fdea734f640 1 -- 172.21.15.100:0/2274809223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdea0145320 msgr2=0x7fdea0070cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.578+0000 7fdea734f640 1 --2- 172.21.15.100:0/2274809223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdea0145320 0x7fdea0070cd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdea0073150 tx=0x7fde9c01c420 comp rx=0 tx=0).stop 2024-04-25T10:04:57.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.578+0000 7fdea734f640 1 -- 172.21.15.100:0/2274809223 shutdown_connections 2024-04-25T10:04:57.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.578+0000 7fdea734f640 1 --2- 172.21.15.100:0/2274809223 >> v2:172.21.15.60:3300/0 conn(0x7fdea00cf140 0x7fdea00b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.578+0000 7fdea734f640 1 --2- 172.21.15.100:0/2274809223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdea0145320 0x7fdea0070cd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.578+0000 7fdea734f640 1 -- 172.21.15.100:0/2274809223 >> 172.21.15.100:0/2274809223 conn(0x7fdea006f530 msgr2=0x7fdea006f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:57.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.578+0000 7fdea734f640 1 -- 172.21.15.100:0/2274809223 shutdown_connections 2024-04-25T10:04:57.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.578+0000 7fdea734f640 1 -- 172.21.15.100:0/2274809223 wait complete. 2024-04-25T10:04:57.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea734f640 1 Processor -- start 2024-04-25T10:04:57.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea734f640 1 -- start start 2024-04-25T10:04:57.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea734f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdea0071210 0x7fdea015af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea734f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdea00cf140 0x7fdea0143a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea734f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea0145320 0x7fdea015c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea734f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea00d0970 con 0x7fdea0145320 2024-04-25T10:04:57.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea734f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea00d07f0 con 0x7fdea00cf140 2024-04-25T10:04:57.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea734f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea00b59e0 con 0x7fdea0071210 2024-04-25T10:04:57.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea48ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdea00cf140 0x7fdea0143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea50eb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdea0071210 0x7fdea015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea48ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdea00cf140 0x7fdea0143a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59966/0 (socket says 172.21.15.100:59966) 2024-04-25T10:04:57.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea48ea640 1 -- 172.21.15.100:0/3313714066 learned_addr learned my addr 172.21.15.100:0/3313714066 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T10:04:57.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea58ec640 1 --2- 172.21.15.100:0/3313714066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea0145320 0x7fdea015c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea48ea640 1 -- 172.21.15.100:0/3313714066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdea0071210 msgr2=0x7fdea015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea48ea640 1 --2- 172.21.15.100:0/3313714066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdea0071210 0x7fdea015af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea48ea640 1 -- 172.21.15.100:0/3313714066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea0145320 msgr2=0x7fdea015c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea48ea640 1 --2- 172.21.15.100:0/3313714066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea0145320 0x7fdea015c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea48ea640 1 -- 172.21.15.100:0/3313714066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea013ba90 con 0x7fdea00cf140 2024-04-25T10:04:57.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea50eb640 1 --2- 172.21.15.100:0/3313714066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdea0071210 0x7fdea015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:57.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea58ec640 1 --2- 172.21.15.100:0/3313714066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea0145320 0x7fdea015c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:57.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea48ea640 1 --2- 172.21.15.100:0/3313714066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdea00cf140 0x7fdea0143a70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fde9000c480 tx=0x7fde9000c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:57.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fde95ffb640 1 -- 172.21.15.100:0/3313714066 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90012830 con 0x7fdea00cf140 2024-04-25T10:04:57.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fde95ffb640 1 -- 172.21.15.100:0/3313714066 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde90017070 con 0x7fdea00cf140 2024-04-25T10:04:57.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fde95ffb640 1 -- 172.21.15.100:0/3313714066 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90012c30 con 0x7fdea00cf140 2024-04-25T10:04:57.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea734f640 1 -- 172.21.15.100:0/3313714066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea013bd70 con 0x7fdea00cf140 2024-04-25T10:04:57.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea734f640 1 -- 172.21.15.100:0/3313714066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea013c330 con 0x7fdea00cf140 2024-04-25T10:04:57.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fde95ffb640 1 -- 172.21.15.100:0/3313714066 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fde9001c460 con 0x7fdea00cf140 2024-04-25T10:04:57.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fde95ffb640 1 -- 172.21.15.100:0/3313714066 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fde90012dd0 con 0x7fdea00cf140 2024-04-25T10:04:57.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.582+0000 7fdea734f640 1 -- 172.21.15.100:0/3313714066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea0071c90 con 0x7fdea00cf140 2024-04-25T10:04:57.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.586+0000 7fde95ffb640 1 -- 172.21.15.100:0/3313714066 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fde9001b730 con 0x7fdea00cf140 2024-04-25T10:04:57.626 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:57.618+0000 7f2f389e0180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:04:57.633 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:57.627+0000 7f7287c22180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:04:57.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.654+0000 7fdea734f640 1 -- 172.21.15.100:0/3313714066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "316be201-bf30-46fc-a685-ddd534dec41d", "id": 2} v 0) v1 -- 0x7fdea0074770 con 0x7fdea00cf140 2024-04-25T10:04:57.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.658+0000 7fde95ffb640 1 -- 172.21.15.100:0/3313714066 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "316be201-bf30-46fc-a685-ddd534dec41d", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fde9001ba10 con 0x7fdea00cf140 2024-04-25T10:04:57.666 INFO:teuthology.orchestra.run.smithi100.stdout:2 2024-04-25T10:04:57.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.662+0000 7fdea734f640 1 -- 172.21.15.100:0/3313714066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdea00cf140 msgr2=0x7fdea0143a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.662+0000 7fdea734f640 1 --2- 172.21.15.100:0/3313714066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdea00cf140 0x7fdea0143a70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fde9000c480 tx=0x7fde9000c940 comp rx=0 tx=0).stop 2024-04-25T10:04:57.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.662+0000 7fdea734f640 1 -- 172.21.15.100:0/3313714066 shutdown_connections 2024-04-25T10:04:57.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.662+0000 7fdea734f640 1 --2- 172.21.15.100:0/3313714066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdea0145320 0x7fdea015c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.662+0000 7fdea734f640 1 --2- 172.21.15.100:0/3313714066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdea00cf140 0x7fdea0143a70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.662+0000 7fdea734f640 1 --2- 172.21.15.100:0/3313714066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdea0071210 0x7fdea015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.662+0000 7fdea734f640 1 -- 172.21.15.100:0/3313714066 >> 172.21.15.100:0/3313714066 conn(0x7fdea006f530 msgr2=0x7fdea00b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:57.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.662+0000 7fdea734f640 1 -- 172.21.15.100:0/3313714066 shutdown_connections 2024-04-25T10:04:57.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.662+0000 7fdea734f640 1 -- 172.21.15.100:0/3313714066 wait complete. 2024-04-25T10:04:57.682 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new db5b8962-b524-423a-8c08-c644f9598b50 3 2024-04-25T10:04:57.683 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:57.678+0000 7f2f389e0180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:04:57.790 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:57.783+0000 7f7287c22180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:04:57.835 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:57.830+0000 7f2f389e0180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:04:57.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.842+0000 7fda057a4640 1 Processor -- start 2024-04-25T10:04:57.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.842+0000 7fda057a4640 1 -- start start 2024-04-25T10:04:57.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.842+0000 7fda057a4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0015b880 0x7fda000ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.842+0000 7fda057a4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fda000cf960 0x7fda0006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fda057a4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda000cfe60 con 0x7fda0006b150 2024-04-25T10:04:57.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fda057a4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda000cc230 con 0x7fda000cf960 2024-04-25T10:04:57.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fda057a4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0006b5a0 con 0x7fda0015b880 2024-04-25T10:04:57.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fd9feffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0015b880 0x7fda000ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fd9feffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0015b880 0x7fda000ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:32956/0 (socket says 172.21.15.100:32956) 2024-04-25T10:04:57.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fd9feffd640 1 -- 172.21.15.100:0/3848298683 learned_addr learned my addr 172.21.15.100:0/3848298683 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T10:04:57.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fd9ff7fe640 1 --1- 172.21.15.100:0/3848298683 >> v1:172.21.15.100:6789/0 conn(0x7fda0006b150 0x7fda0015f460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:39874/0 (socket says 172.21.15.100:39874) 2024-04-25T10:04:57.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fd9fe7fc640 1 --2- 172.21.15.100:0/3848298683 >> v2:172.21.15.60:3300/0 conn(0x7fda000cf960 0x7fda0006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fd9feffd640 1 -- 172.21.15.100:0/3848298683 >> v2:172.21.15.60:3300/0 conn(0x7fda000cf960 msgr2=0x7fda0006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:57.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fd9feffd640 1 --2- 172.21.15.100:0/3848298683 >> v2:172.21.15.60:3300/0 conn(0x7fda000cf960 0x7fda0006ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fd9feffd640 1 -- 172.21.15.100:0/3848298683 >> v1:172.21.15.100:6789/0 conn(0x7fda0006b150 legacy=0x7fda0015f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fd9feffd640 1 -- 172.21.15.100:0/3848298683 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda000d0140 con 0x7fda0015b880 2024-04-25T10:04:57.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fd9fe7fc640 1 --2- 172.21.15.100:0/3848298683 >> v2:172.21.15.60:3300/0 conn(0x7fda000cf960 0x7fda0006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:57.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fd9feffd640 1 --2- 172.21.15.100:0/3848298683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0015b880 0x7fda000ceb90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009870 tx=0x7fd9f401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5705dbb52758689 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:57.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fd9fdffb640 1 -- 172.21.15.100:0/3848298683 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40b0070 con 0x7fda0015b880 2024-04-25T10:04:57.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fd9fdffb640 1 -- 172.21.15.100:0/3848298683 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f40ab600 con 0x7fda0015b880 2024-04-25T10:04:57.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fd9fdffb640 1 -- 172.21.15.100:0/3848298683 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40ab9d0 con 0x7fda0015b880 2024-04-25T10:04:57.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fda057a4640 1 -- 172.21.15.100:0/3848298683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0015b880 msgr2=0x7fda000ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fda057a4640 1 --2- 172.21.15.100:0/3848298683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0015b880 0x7fda000ceb90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009870 tx=0x7fd9f401c560 comp rx=0 tx=0).stop 2024-04-25T10:04:57.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fda057a4640 1 -- 172.21.15.100:0/3848298683 shutdown_connections 2024-04-25T10:04:57.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fda057a4640 1 --2- 172.21.15.100:0/3848298683 >> v2:172.21.15.60:3300/0 conn(0x7fda000cf960 0x7fda0006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fda057a4640 1 --2- 172.21.15.100:0/3848298683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0015b880 0x7fda000ceb90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fda057a4640 1 -- 172.21.15.100:0/3848298683 >> 172.21.15.100:0/3848298683 conn(0x7fda00099a30 msgr2=0x7fda000d0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:57.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fda057a4640 1 -- 172.21.15.100:0/3848298683 shutdown_connections 2024-04-25T10:04:57.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.846+0000 7fda057a4640 1 -- 172.21.15.100:0/3848298683 wait complete. 2024-04-25T10:04:57.855 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:57.847+0000 7f7287c22180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:04:57.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fda057a4640 1 Processor -- start 2024-04-25T10:04:57.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fda057a4640 1 -- start start 2024-04-25T10:04:57.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fda057a4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0006b150 0x7fda00150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fda057a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda000cf960 0x7fda001596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fda057a4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda0015b880 0x7fda00159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:57.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fda057a4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda000cddf0 con 0x7fda000cf960 2024-04-25T10:04:57.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fda057a4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda000cdc70 con 0x7fda0015b880 2024-04-25T10:04:57.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fda057a4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda00168800 con 0x7fda0006b150 2024-04-25T10:04:57.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9feffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0006b150 0x7fda00150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9feffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0006b150 0x7fda00150c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:32966/0 (socket says 172.21.15.100:32966) 2024-04-25T10:04:57.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9feffd640 1 -- 172.21.15.100:0/2011885537 learned_addr learned my addr 172.21.15.100:0/2011885537 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T10:04:57.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9ff7fe640 1 --2- 172.21.15.100:0/2011885537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda0015b880 0x7fda00159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9fe7fc640 1 --2- 172.21.15.100:0/2011885537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda000cf960 0x7fda001596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:57.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9feffd640 1 -- 172.21.15.100:0/2011885537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda0015b880 msgr2=0x7fda00159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9feffd640 1 --2- 172.21.15.100:0/2011885537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda0015b880 0x7fda00159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9feffd640 1 -- 172.21.15.100:0/2011885537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda000cf960 msgr2=0x7fda001596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9feffd640 1 --2- 172.21.15.100:0/2011885537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda000cf960 0x7fda001596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9feffd640 1 -- 172.21.15.100:0/2011885537 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0006f320 con 0x7fda0006b150 2024-04-25T10:04:57.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9fe7fc640 1 --2- 172.21.15.100:0/2011885537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda000cf960 0x7fda001596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:57.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9feffd640 1 --2- 172.21.15.100:0/2011885537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0006b150 0x7fda00150c10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40ae040 tx=0x7fd9f4019300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:57.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9e77fe640 1 -- 172.21.15.100:0/2011885537 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40ba020 con 0x7fda0006b150 2024-04-25T10:04:57.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9e77fe640 1 -- 172.21.15.100:0/2011885537 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f40b6660 con 0x7fda0006b150 2024-04-25T10:04:57.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fda057a4640 1 -- 172.21.15.100:0/2011885537 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0006f600 con 0x7fda0006b150 2024-04-25T10:04:57.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9e77fe640 1 -- 172.21.15.100:0/2011885537 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40b0040 con 0x7fda0006b150 2024-04-25T10:04:57.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fda057a4640 1 -- 172.21.15.100:0/2011885537 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0006fbc0 con 0x7fda0006b150 2024-04-25T10:04:57.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9e77fe640 1 -- 172.21.15.100:0/2011885537 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd9f40bd020 con 0x7fda0006b150 2024-04-25T10:04:57.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fd9e77fe640 1 -- 172.21.15.100:0/2011885537 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fd9f40bfc20 con 0x7fda0006b150 2024-04-25T10:04:57.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.850+0000 7fda057a4640 1 -- 172.21.15.100:0/2011885537 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda00074770 con 0x7fda0006b150 2024-04-25T10:04:57.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.854+0000 7fd9e77fe640 1 -- 172.21.15.100:0/2011885537 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd9f40c4030 con 0x7fda0006b150 2024-04-25T10:04:57.899 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:57.894+0000 7f2f389e0180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:04:57.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.922+0000 7fda057a4640 1 -- 172.21.15.100:0/2011885537 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "db5b8962-b524-423a-8c08-c644f9598b50", "id": 3} v 0) v1 -- 0x7fda0006b860 con 0x7fda0006b150 2024-04-25T10:04:57.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.926+0000 7fd9e77fe640 1 -- 172.21.15.100:0/2011885537 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "db5b8962-b524-423a-8c08-c644f9598b50", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd9f4016150 con 0x7fda0006b150 2024-04-25T10:04:57.932 INFO:teuthology.orchestra.run.smithi100.stdout:3 2024-04-25T10:04:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.926+0000 7fda057a4640 1 -- 172.21.15.100:0/2011885537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0006b150 msgr2=0x7fda00150c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.926+0000 7fda057a4640 1 --2- 172.21.15.100:0/2011885537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0006b150 0x7fda00150c10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40ae040 tx=0x7fd9f4019300 comp rx=0 tx=0).stop 2024-04-25T10:04:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.926+0000 7fda057a4640 1 -- 172.21.15.100:0/2011885537 shutdown_connections 2024-04-25T10:04:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.926+0000 7fda057a4640 1 --2- 172.21.15.100:0/2011885537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda0015b880 0x7fda00159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.926+0000 7fda057a4640 1 --2- 172.21.15.100:0/2011885537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda000cf960 0x7fda001596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.926+0000 7fda057a4640 1 --2- 172.21.15.100:0/2011885537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0006b150 0x7fda00150c10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.926+0000 7fda057a4640 1 -- 172.21.15.100:0/2011885537 >> 172.21.15.100:0/2011885537 conn(0x7fda00099a30 msgr2=0x7fda000ad400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.926+0000 7fda057a4640 1 -- 172.21.15.100:0/2011885537 shutdown_connections 2024-04-25T10:04:57.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:57.926+0000 7fda057a4640 1 -- 172.21.15.100:0/2011885537 wait complete. 2024-04-25T10:04:57.948 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 37f3785c-f929-4fb1-8d93-cf297404e837 4 2024-04-25T10:04:58.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.050+0000 7f1606a46640 1 Processor -- start 2024-04-25T10:04:58.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.050+0000 7f1606a46640 1 -- start start 2024-04-25T10:04:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.050+0000 7f1606a46640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f16000713e0 0x7f16000cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.050+0000 7f1606a46640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16000cef70 0x7f16000b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.050+0000 7f1606a46640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1600145700 con 0x7f1600145320 2024-04-25T10:04:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.050+0000 7f1606a46640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16000b4800 con 0x7f16000713e0 2024-04-25T10:04:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.050+0000 7f1606a46640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16000d0bd0 con 0x7f16000cef70 2024-04-25T10:04:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.050+0000 7f15fffff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1600145320 0x7f1600070cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:39884/0 (socket says 172.21.15.100:39884) 2024-04-25T10:04:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f15fffff640 1 -- 172.21.15.100:0/1945034280 learned_addr learned my addr 172.21.15.100:0/1945034280 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-25T10:04:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f1604fe3640 1 --2- 172.21.15.100:0/1945034280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16000cef70 0x7f16000b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f15feffd640 1 -- 172.21.15.100:0/1945034280 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217842339 0 0) 0x7f1600145700 con 0x7f1600145320 2024-04-25T10:04:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f15ff7fe640 1 --2- 172.21.15.100:0/1945034280 >> v2:172.21.15.60:3300/0 conn(0x7f16000713e0 0x7f16000cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f15feffd640 1 -- 172.21.15.100:0/1945034280 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15e8003530 con 0x7f1600145320 2024-04-25T10:04:58.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f1604fe3640 1 -- 172.21.15.100:0/1945034280 >> v2:172.21.15.60:3300/0 conn(0x7f16000713e0 msgr2=0x7f16000cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f1604fe3640 1 --2- 172.21.15.100:0/1945034280 >> v2:172.21.15.60:3300/0 conn(0x7f16000713e0 0x7f16000cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f1604fe3640 1 -- 172.21.15.100:0/1945034280 >> v1:172.21.15.100:6789/0 conn(0x7f1600145320 legacy=0x7f1600070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f1604fe3640 1 -- 172.21.15.100:0/1945034280 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16000d0eb0 con 0x7f16000cef70 2024-04-25T10:04:58.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f15ff7fe640 1 --2- 172.21.15.100:0/1945034280 >> v2:172.21.15.60:3300/0 conn(0x7f16000713e0 0x7f16000cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f1604fe3640 1 --2- 172.21.15.100:0/1945034280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16000cef70 0x7f16000b4c90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009d90 tx=0x7f15f801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb14c34493f04292 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f15feffd640 1 -- 172.21.15.100:0/1945034280 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f80b2070 con 0x7f16000cef70 2024-04-25T10:04:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f15feffd640 1 -- 172.21.15.100:0/1945034280 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f80a5890 con 0x7f16000cef70 2024-04-25T10:04:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f15feffd640 1 -- 172.21.15.100:0/1945034280 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f80a5c60 con 0x7f16000cef70 2024-04-25T10:04:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f1606a46640 1 -- 172.21.15.100:0/1945034280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16000cef70 msgr2=0x7f16000b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f1606a46640 1 --2- 172.21.15.100:0/1945034280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16000cef70 0x7f16000b4c90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009d90 tx=0x7f15f801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f1606a46640 1 -- 172.21.15.100:0/1945034280 shutdown_connections 2024-04-25T10:04:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f1606a46640 1 --2- 172.21.15.100:0/1945034280 >> v2:172.21.15.60:3300/0 conn(0x7f16000713e0 0x7f16000cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f1606a46640 1 --2- 172.21.15.100:0/1945034280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16000cef70 0x7f16000b4c90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f1606a46640 1 -- 172.21.15.100:0/1945034280 >> 172.21.15.100:0/1945034280 conn(0x7f160006f530 msgr2=0x7f160006f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f1606a46640 1 -- 172.21.15.100:0/1945034280 shutdown_connections 2024-04-25T10:04:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.054+0000 7f1606a46640 1 -- 172.21.15.100:0/1945034280 wait complete. 2024-04-25T10:04:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f1606a46640 1 Processor -- start 2024-04-25T10:04:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f1606a46640 1 -- start start 2024-04-25T10:04:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f1606a46640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16000713e0 0x7f1600136040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f1606a46640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16000cef70 0x7f160015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f1606a46640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1600145320 0x7f1600137f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f1606a46640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16000b5b60 con 0x7f16000cef70 2024-04-25T10:04:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f1606a46640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16000b59e0 con 0x7f1600145320 2024-04-25T10:04:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f1606a46640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16000b5ce0 con 0x7f16000713e0 2024-04-25T10:04:58.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f15fffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16000713e0 0x7f1600136040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f15fffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16000713e0 0x7f1600136040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:32976/0 (socket says 172.21.15.100:32976) 2024-04-25T10:04:58.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f15fffff640 1 -- 172.21.15.100:0/662657252 learned_addr learned my addr 172.21.15.100:0/662657252 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T10:04:58.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f1604fe3640 1 --2- 172.21.15.100:0/662657252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1600145320 0x7f1600137f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f15fffff640 1 -- 172.21.15.100:0/662657252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1600145320 msgr2=0x7f1600137f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f15fffff640 1 --2- 172.21.15.100:0/662657252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1600145320 0x7f1600137f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f15ff7fe640 1 --2- 172.21.15.100:0/662657252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16000cef70 0x7f160015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f15fffff640 1 -- 172.21.15.100:0/662657252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16000cef70 msgr2=0x7f160015a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f15fffff640 1 --2- 172.21.15.100:0/662657252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16000cef70 0x7f160015a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f15fffff640 1 -- 172.21.15.100:0/662657252 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1600134fc0 con 0x7f16000713e0 2024-04-25T10:04:58.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f15ff7fe640 1 --2- 172.21.15.100:0/662657252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16000cef70 0x7f160015a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f15fffff640 1 --2- 172.21.15.100:0/662657252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16000713e0 0x7f1600136040 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f15f400c3c0 tx=0x7f15f400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f15fcff9640 1 -- 172.21.15.100:0/662657252 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f4017070 con 0x7f16000713e0 2024-04-25T10:04:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f15fcff9640 1 -- 172.21.15.100:0/662657252 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f4005cf0 con 0x7f16000713e0 2024-04-25T10:04:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f1606a46640 1 -- 172.21.15.100:0/662657252 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16001352a0 con 0x7f16000713e0 2024-04-25T10:04:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f15fcff9640 1 -- 172.21.15.100:0/662657252 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f40072f0 con 0x7f16000713e0 2024-04-25T10:04:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f1606a46640 1 -- 172.21.15.100:0/662657252 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1600135860 con 0x7f16000713e0 2024-04-25T10:04:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f15fcff9640 1 -- 172.21.15.100:0/662657252 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f15f401a070 con 0x7f16000713e0 2024-04-25T10:04:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f15fcff9640 1 -- 172.21.15.100:0/662657252 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f15f4012910 con 0x7f16000713e0 2024-04-25T10:04:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.058+0000 7f1606a46640 1 -- 172.21.15.100:0/662657252 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1600074770 con 0x7f16000713e0 2024-04-25T10:04:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.062+0000 7f15fcff9640 1 -- 172.21.15.100:0/662657252 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f15f4021070 con 0x7f16000713e0 2024-04-25T10:04:58.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.130+0000 7f1606a46640 1 -- 172.21.15.100:0/662657252 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "37f3785c-f929-4fb1-8d93-cf297404e837", "id": 4} v 0) v1 -- 0x7f16000b1730 con 0x7f16000713e0 2024-04-25T10:04:58.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.134+0000 7f15fcff9640 1 -- 172.21.15.100:0/662657252 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "37f3785c-f929-4fb1-8d93-cf297404e837", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f15f40076c0 con 0x7f16000713e0 2024-04-25T10:04:58.141 INFO:teuthology.orchestra.run.smithi100.stdout:4 2024-04-25T10:04:58.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.138+0000 7f1606a46640 1 -- 172.21.15.100:0/662657252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16000713e0 msgr2=0x7f1600136040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.138+0000 7f1606a46640 1 --2- 172.21.15.100:0/662657252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16000713e0 0x7f1600136040 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f15f400c3c0 tx=0x7f15f400c880 comp rx=0 tx=0).stop 2024-04-25T10:04:58.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.138+0000 7f1606a46640 1 -- 172.21.15.100:0/662657252 shutdown_connections 2024-04-25T10:04:58.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.138+0000 7f1606a46640 1 --2- 172.21.15.100:0/662657252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1600145320 0x7f1600137f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.138+0000 7f1606a46640 1 --2- 172.21.15.100:0/662657252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16000cef70 0x7f160015a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.138+0000 7f1606a46640 1 --2- 172.21.15.100:0/662657252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16000713e0 0x7f1600136040 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.138+0000 7f1606a46640 1 -- 172.21.15.100:0/662657252 >> 172.21.15.100:0/662657252 conn(0x7f160006f530 msgr2=0x7f16000ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:58.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.138+0000 7f1606a46640 1 -- 172.21.15.100:0/662657252 shutdown_connections 2024-04-25T10:04:58.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.138+0000 7f1606a46640 1 -- 172.21.15.100:0/662657252 wait complete. 2024-04-25T10:04:58.161 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 452236ce-e60a-4cc1-8a06-3a550d19c0d3 5 2024-04-25T10:04:58.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb94b519640 1 Processor -- start 2024-04-25T10:04:58.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb94b519640 1 -- start start 2024-04-25T10:04:58.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb94b519640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb94415a530 0x7fb9440b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb94b519640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94415ebf0 con 0x7fb9440afe70 2024-04-25T10:04:58.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb94b519640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94415f150 con 0x7fb94415a910 2024-04-25T10:04:58.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb94b519640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9440ce510 con 0x7fb94415a530 2024-04-25T10:04:58.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb948ab4640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb94415a910 0x7fb9440cef00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:39896/0 (socket says 172.21.15.100:39896) 2024-04-25T10:04:58.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb948ab4640 1 -- 172.21.15.100:0/3173370262 learned_addr learned my addr 172.21.15.100:0/3173370262 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-25T10:04:58.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb943fff640 1 -- 172.21.15.100:0/3173370262 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3572062243 0 0) 0x7fb94415f150 con 0x7fb94415a910 2024-04-25T10:04:58.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb9492b5640 1 --2- 172.21.15.100:0/3173370262 >> v2:172.21.15.60:3300/0 conn(0x7fb94415a530 0x7fb9440b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb943fff640 1 -- 172.21.15.100:0/3173370262 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb92c003530 con 0x7fb94415a910 2024-04-25T10:04:58.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb943fff640 1 -- 172.21.15.100:0/3173370262 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 600902576 0 0) 0x7fb92c003530 con 0x7fb94415a910 2024-04-25T10:04:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb943fff640 1 -- 172.21.15.100:0/3173370262 >> v2:172.21.15.60:3300/0 conn(0x7fb94415a530 msgr2=0x7fb9440b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb943fff640 1 --2- 172.21.15.100:0/3173370262 >> v2:172.21.15.60:3300/0 conn(0x7fb94415a530 0x7fb9440b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb943fff640 1 -- 172.21.15.100:0/3173370262 >> v1:172.21.15.60:6789/0 conn(0x7fb9440afe70 legacy=0x7fb9440d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb943fff640 1 -- 172.21.15.100:0/3173370262 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9440ce740 con 0x7fb94415a910 2024-04-25T10:04:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb9492b5640 1 --2- 172.21.15.100:0/3173370262 >> v2:172.21.15.60:3300/0 conn(0x7fb94415a530 0x7fb9440b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb943fff640 1 -- 172.21.15.100:0/3173370262 <== mon.1 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fb938001680 con 0x7fb94415a910 2024-04-25T10:04:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb943fff640 1 -- 172.21.15.100:0/3173370262 >> v1:172.21.15.100:6789/0 conn(0x7fb94415a910 legacy=0x7fb9440cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb943fff640 1 --2- 172.21.15.100:0/3173370262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb92c0039c0 0x7fb92c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb943fff640 1 --2- 172.21.15.100:0/3173370262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb92c004810 0x7fb92c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb943fff640 1 --2- 172.21.15.100:0/3173370262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c0052f0 0x7fb92c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb943fff640 1 -- 172.21.15.100:0/3173370262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c00c190 con 0x7fb92c0039c0 2024-04-25T10:04:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb943fff640 1 -- 172.21.15.100:0/3173370262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c00c340 con 0x7fb92c0052f0 2024-04-25T10:04:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb943fff640 1 -- 172.21.15.100:0/3173370262 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c00c540 con 0x7fb92c004810 2024-04-25T10:04:58.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb948ab4640 1 --2- 172.21.15.100:0/3173370262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb92c004810 0x7fb92c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb949ab6640 1 --2- 172.21.15.100:0/3173370262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c0052f0 0x7fb92c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb9492b5640 1 --2- 172.21.15.100:0/3173370262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb92c0039c0 0x7fb92c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb948ab4640 1 -- 172.21.15.100:0/3173370262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c0052f0 msgr2=0x7fb92c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb948ab4640 1 --2- 172.21.15.100:0/3173370262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c0052f0 0x7fb92c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb948ab4640 1 -- 172.21.15.100:0/3173370262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb92c0039c0 msgr2=0x7fb92c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb948ab4640 1 --2- 172.21.15.100:0/3173370262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb92c0039c0 0x7fb92c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb948ab4640 1 -- 172.21.15.100:0/3173370262 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c003530 con 0x7fb92c004810 2024-04-25T10:04:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb9492b5640 1 --2- 172.21.15.100:0/3173370262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb92c0039c0 0x7fb92c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb949ab6640 1 --2- 172.21.15.100:0/3173370262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c0052f0 0x7fb92c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:58.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.306+0000 7fb949ab6640 1 -- 172.21.15.100:0/3173370262 reap_dead start 2024-04-25T10:04:58.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb948ab4640 1 --2- 172.21.15.100:0/3173370262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb92c004810 0x7fb92c004c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb938001ec0 tx=0x7fb93801cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9323e401660f0f68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb943fff640 1 -- 172.21.15.100:0/3173370262 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938007d40 con 0x7fb92c004810 2024-04-25T10:04:58.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb943fff640 1 -- 172.21.15.100:0/3173370262 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9380a9070 con 0x7fb92c004810 2024-04-25T10:04:58.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb943fff640 1 -- 172.21.15.100:0/3173370262 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938005d80 con 0x7fb92c004810 2024-04-25T10:04:58.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb94b519640 1 -- 172.21.15.100:0/3173370262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb92c004810 msgr2=0x7fb92c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb94b519640 1 --2- 172.21.15.100:0/3173370262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb92c004810 0x7fb92c004c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb938001ec0 tx=0x7fb93801cbd0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb94b519640 1 -- 172.21.15.100:0/3173370262 shutdown_connections 2024-04-25T10:04:58.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb94b519640 1 --2- 172.21.15.100:0/3173370262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb92c004810 0x7fb92c004c90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb94b519640 1 -- 172.21.15.100:0/3173370262 >> 172.21.15.100:0/3173370262 conn(0x7fb9440b2360 msgr2=0x7fb9440b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:58.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb94b519640 1 -- 172.21.15.100:0/3173370262 shutdown_connections 2024-04-25T10:04:58.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb94b519640 1 -- 172.21.15.100:0/3173370262 wait complete. 2024-04-25T10:04:58.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb94b519640 1 Processor -- start 2024-04-25T10:04:58.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb94b519640 1 -- start start 2024-04-25T10:04:58.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb94b519640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c004810 0x7fb94414ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb94b519640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb94414e020 0x7fb9441f3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb94b519640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb94414e4a0 0x7fb9441f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb94b519640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440cef00 con 0x7fb94414e020 2024-04-25T10:04:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb94b519640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440afac0 con 0x7fb92c004810 2024-04-25T10:04:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb94b519640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440cf080 con 0x7fb94414e4a0 2024-04-25T10:04:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb9492b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c004810 0x7fb94414ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb9492b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c004810 0x7fb94414ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60000/0 (socket says 172.21.15.100:60000) 2024-04-25T10:04:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb949ab6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb94414e4a0 0x7fb9441f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb9492b5640 1 -- 172.21.15.100:0/2518916406 learned_addr learned my addr 172.21.15.100:0/2518916406 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T10:04:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.310+0000 7fb948ab4640 1 --2- 172.21.15.100:0/2518916406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb94414e020 0x7fb9441f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.314+0000 7fb9492b5640 1 -- 172.21.15.100:0/2518916406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb94414e4a0 msgr2=0x7fb9441f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.314+0000 7fb9492b5640 1 --2- 172.21.15.100:0/2518916406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb94414e4a0 0x7fb9441f3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.314+0000 7fb9492b5640 1 -- 172.21.15.100:0/2518916406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb94414e020 msgr2=0x7fb9441f3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.314+0000 7fb9492b5640 1 --2- 172.21.15.100:0/2518916406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb94414e020 0x7fb9441f3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.314+0000 7fb9492b5640 1 -- 172.21.15.100:0/2518916406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94414b230 con 0x7fb92c004810 2024-04-25T10:04:58.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.314+0000 7fb948ab4640 1 --2- 172.21.15.100:0/2518916406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb94414e020 0x7fb9441f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:58.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.314+0000 7fb9492b5640 1 --2- 172.21.15.100:0/2518916406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c004810 0x7fb94414ff10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0046e0 tx=0x7fb93c007700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.314+0000 7fb941ffb640 1 -- 172.21.15.100:0/2518916406 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c014610 con 0x7fb92c004810 2024-04-25T10:04:58.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.314+0000 7fb941ffb640 1 -- 172.21.15.100:0/2518916406 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c019070 con 0x7fb92c004810 2024-04-25T10:04:58.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.314+0000 7fb94b519640 1 -- 172.21.15.100:0/2518916406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94414a190 con 0x7fb92c004810 2024-04-25T10:04:58.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.314+0000 7fb941ffb640 1 -- 172.21.15.100:0/2518916406 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0149b0 con 0x7fb92c004810 2024-04-25T10:04:58.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.314+0000 7fb94b519640 1 -- 172.21.15.100:0/2518916406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94414a750 con 0x7fb92c004810 2024-04-25T10:04:58.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.314+0000 7fb941ffb640 1 -- 172.21.15.100:0/2518916406 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb93c014b50 con 0x7fb92c004810 2024-04-25T10:04:58.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.314+0000 7fb941ffb640 1 -- 172.21.15.100:0/2518916406 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fb93c01d940 con 0x7fb92c004810 2024-04-25T10:04:58.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.314+0000 7fb94b519640 1 -- 172.21.15.100:0/2518916406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb944074770 con 0x7fb92c004810 2024-04-25T10:04:58.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.314+0000 7fb941ffb640 1 -- 172.21.15.100:0/2518916406 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb93c01dc20 con 0x7fb92c004810 2024-04-25T10:04:58.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.382+0000 7fb94b519640 1 -- 172.21.15.100:0/2518916406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "452236ce-e60a-4cc1-8a06-3a550d19c0d3", "id": 5} v 0) v1 -- 0x7fb9440700e0 con 0x7fb92c004810 2024-04-25T10:04:58.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.390+0000 7fb941ffb640 1 -- 172.21.15.100:0/2518916406 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "452236ce-e60a-4cc1-8a06-3a550d19c0d3", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb93c01de00 con 0x7fb92c004810 2024-04-25T10:04:58.395 INFO:teuthology.orchestra.run.smithi100.stdout:5 2024-04-25T10:04:58.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.390+0000 7fb94b519640 1 -- 172.21.15.100:0/2518916406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c004810 msgr2=0x7fb94414ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.390+0000 7fb94b519640 1 --2- 172.21.15.100:0/2518916406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c004810 0x7fb94414ff10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0046e0 tx=0x7fb93c007700 comp rx=0 tx=0).stop 2024-04-25T10:04:58.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.390+0000 7fb94b519640 1 -- 172.21.15.100:0/2518916406 shutdown_connections 2024-04-25T10:04:58.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.390+0000 7fb94b519640 1 --2- 172.21.15.100:0/2518916406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb94414e4a0 0x7fb9441f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.390+0000 7fb94b519640 1 --2- 172.21.15.100:0/2518916406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb94414e020 0x7fb9441f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.390+0000 7fb94b519640 1 --2- 172.21.15.100:0/2518916406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb92c004810 0x7fb94414ff10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.390+0000 7fb94b519640 1 -- 172.21.15.100:0/2518916406 >> 172.21.15.100:0/2518916406 conn(0x7fb9440b2360 msgr2=0x7fb9440cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:58.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.390+0000 7fb94b519640 1 -- 172.21.15.100:0/2518916406 shutdown_connections 2024-04-25T10:04:58.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.390+0000 7fb94b519640 1 -- 172.21.15.100:0/2518916406 wait complete. 2024-04-25T10:04:58.411 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 99bd25b7-66bc-4987-ab26-8cdef85f643f 6 2024-04-25T10:04:58.463 INFO:tasks.ceph.mgr.x.smithi100.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T10:04:58.463 INFO:tasks.ceph.mgr.x.smithi100.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T10:04:58.463 INFO:tasks.ceph.mgr.x.smithi100.stderr: from numpy import show_config as show_numpy_config 2024-04-25T10:04:58.483 INFO:tasks.ceph.mgr.y.smithi060.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T10:04:58.483 INFO:tasks.ceph.mgr.y.smithi060.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T10:04:58.483 INFO:tasks.ceph.mgr.y.smithi060.stderr: from numpy import show_config as show_numpy_config 2024-04-25T10:04:58.504 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:58.498+0000 7f2f389e0180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:04:58.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.514+0000 7f8cc5c24640 1 Processor -- start 2024-04-25T10:04:58.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.518+0000 7f8cc5c24640 1 -- start start 2024-04-25T10:04:58.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.518+0000 7f8cc5c24640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cc00adfa0 0x7f8cc006e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.518+0000 7f8cc5c24640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8cc015e0a0 0x7f8cc00ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.518+0000 7f8cc5c24640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc006ebf0 con 0x7f8cc00ae380 2024-04-25T10:04:58.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.518+0000 7f8cc5c24640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc006eed0 con 0x7f8cc015e0a0 2024-04-25T10:04:58.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.518+0000 7f8cc5c24640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc006f1b0 con 0x7f8cc00adfa0 2024-04-25T10:04:58.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.518+0000 7f8cbf7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cc00adfa0 0x7f8cc006e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.518+0000 7f8cb7fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f8cc00ae380 0x7f8cc015d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:39904/0 (socket says 172.21.15.100:39904) 2024-04-25T10:04:58.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.518+0000 7f8cb7fff640 1 -- 172.21.15.100:0/441321819 learned_addr learned my addr 172.21.15.100:0/441321819 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-25T10:04:58.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.518+0000 7f8cbf7fe640 1 -- 172.21.15.100:0/441321819 >> v2:172.21.15.60:3300/0 conn(0x7f8cc015e0a0 msgr2=0x7f8cc00ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:58.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.518+0000 7f8cbf7fe640 1 --2- 172.21.15.100:0/441321819 >> v2:172.21.15.60:3300/0 conn(0x7f8cc015e0a0 0x7f8cc00ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.518+0000 7f8cbf7fe640 1 -- 172.21.15.100:0/441321819 >> v1:172.21.15.100:6789/0 conn(0x7f8cc00ae380 legacy=0x7f8cc015d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.518+0000 7f8cbeffd640 1 -- 172.21.15.100:0/441321819 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630241865 0 0) 0x7f8cc006ebf0 con 0x7f8cc00ae380 2024-04-25T10:04:58.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.518+0000 7f8cbf7fe640 1 -- 172.21.15.100:0/441321819 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc006f490 con 0x7f8cc00adfa0 2024-04-25T10:04:58.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.518+0000 7f8cbf7fe640 1 --2- 172.21.15.100:0/441321819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cc00adfa0 0x7f8cc006e4f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00097c0 tx=0x7f8cb001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dfd8ff02bf57680 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.518+0000 7f8cbeffd640 1 -- 172.21.15.100:0/441321819 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00ab640 con 0x7f8cc00adfa0 2024-04-25T10:04:58.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.522+0000 7f8cbeffd640 1 -- 172.21.15.100:0/441321819 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb00b0070 con 0x7f8cc00adfa0 2024-04-25T10:04:58.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.522+0000 7f8cbeffd640 1 -- 172.21.15.100:0/441321819 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00aba00 con 0x7f8cc00adfa0 2024-04-25T10:04:58.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.522+0000 7f8cc5c24640 1 -- 172.21.15.100:0/441321819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cc00adfa0 msgr2=0x7f8cc006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.522+0000 7f8cc5c24640 1 --2- 172.21.15.100:0/441321819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cc00adfa0 0x7f8cc006e4f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00097c0 tx=0x7f8cb001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.522+0000 7f8cc5c24640 1 -- 172.21.15.100:0/441321819 shutdown_connections 2024-04-25T10:04:58.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.522+0000 7f8cc5c24640 1 --2- 172.21.15.100:0/441321819 >> v2:172.21.15.60:3300/0 conn(0x7f8cc015e0a0 0x7f8cc00ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.522+0000 7f8cc5c24640 1 --2- 172.21.15.100:0/441321819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cc00adfa0 0x7f8cc006e4f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.522+0000 7f8cc5c24640 1 -- 172.21.15.100:0/441321819 >> 172.21.15.100:0/441321819 conn(0x7f8cc00cfdd0 msgr2=0x7f8cc00d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:58.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.522+0000 7f8cc5c24640 1 -- 172.21.15.100:0/441321819 shutdown_connections 2024-04-25T10:04:58.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.522+0000 7f8cc5c24640 1 -- 172.21.15.100:0/441321819 wait complete. 2024-04-25T10:04:58.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.522+0000 7f8cc5c24640 1 Processor -- start 2024-04-25T10:04:58.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.522+0000 7f8cc5c24640 1 -- start start 2024-04-25T10:04:58.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cc5c24640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc00adfa0 0x7f8cc0142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cc5c24640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc00ae380 0x7f8cc01f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cc5c24640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cc015e0a0 0x7f8cc01f8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cc5c24640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0157c30 con 0x7f8cc00ae380 2024-04-25T10:04:58.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cc5c24640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0157ab0 con 0x7f8cc00adfa0 2024-04-25T10:04:58.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cc5c24640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0157db0 con 0x7f8cc015e0a0 2024-04-25T10:04:58.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cbf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc00adfa0 0x7f8cc0142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cbffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cc015e0a0 0x7f8cc01f8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cbf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc00adfa0 0x7f8cc0142520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60014/0 (socket says 172.21.15.100:60014) 2024-04-25T10:04:58.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cbf7fe640 1 -- 172.21.15.100:0/3037237561 learned_addr learned my addr 172.21.15.100:0/3037237561 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T10:04:58.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cbf7fe640 1 -- 172.21.15.100:0/3037237561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cc015e0a0 msgr2=0x7f8cc01f8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cbf7fe640 1 --2- 172.21.15.100:0/3037237561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cc015e0a0 0x7f8cc01f8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cbf7fe640 1 -- 172.21.15.100:0/3037237561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc00ae380 msgr2=0x7f8cc01f7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cb7fff640 1 --2- 172.21.15.100:0/3037237561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc00ae380 0x7f8cc01f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cbf7fe640 1 --2- 172.21.15.100:0/3037237561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc00ae380 0x7f8cc01f7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cbf7fe640 1 -- 172.21.15.100:0/3037237561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc0149e10 con 0x7f8cc00adfa0 2024-04-25T10:04:58.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cbffff640 1 --2- 172.21.15.100:0/3037237561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cc015e0a0 0x7f8cc01f8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:58.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cbf7fe640 1 --2- 172.21.15.100:0/3037237561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc00adfa0 0x7f8cc0142520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00097c0 tx=0x7f8cb001ace0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cbcff9640 1 -- 172.21.15.100:0/3037237561 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0004220 con 0x7f8cc00adfa0 2024-04-25T10:04:58.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cbcff9640 1 -- 172.21.15.100:0/3037237561 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb00043c0 con 0x7f8cc00adfa0 2024-04-25T10:04:58.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cc5c24640 1 -- 172.21.15.100:0/3037237561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc014a0f0 con 0x7f8cc00adfa0 2024-04-25T10:04:58.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cbcff9640 1 -- 172.21.15.100:0/3037237561 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0003820 con 0x7f8cc00adfa0 2024-04-25T10:04:58.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cc5c24640 1 -- 172.21.15.100:0/3037237561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc0142a60 con 0x7f8cc00adfa0 2024-04-25T10:04:58.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cbcff9640 1 -- 172.21.15.100:0/3037237561 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8cb00bf420 con 0x7f8cc00adfa0 2024-04-25T10:04:58.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cbcff9640 1 -- 172.21.15.100:0/3037237561 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f8cb00b49f0 con 0x7f8cc00adfa0 2024-04-25T10:04:58.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.526+0000 7f8cc5c24640 1 -- 172.21.15.100:0/3037237561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc0074770 con 0x7f8cc00adfa0 2024-04-25T10:04:58.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.530+0000 7f8cbcff9640 1 -- 172.21.15.100:0/3037237561 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8cb0016090 con 0x7f8cc00adfa0 2024-04-25T10:04:58.563 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:58.555+0000 7f7287c22180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:04:58.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.594+0000 7f8cc5c24640 1 -- 172.21.15.100:0/3037237561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "99bd25b7-66bc-4987-ab26-8cdef85f643f", "id": 6} v 0) v1 -- 0x7f8cc015cd80 con 0x7f8cc00adfa0 2024-04-25T10:04:58.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.602+0000 7f8cbcff9640 1 -- 172.21.15.100:0/3037237561 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "99bd25b7-66bc-4987-ab26-8cdef85f643f", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8cb00b4c40 con 0x7f8cc00adfa0 2024-04-25T10:04:58.607 INFO:teuthology.orchestra.run.smithi100.stdout:6 2024-04-25T10:04:58.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.602+0000 7f8cc5c24640 1 -- 172.21.15.100:0/3037237561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc00adfa0 msgr2=0x7f8cc0142520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.602+0000 7f8cc5c24640 1 --2- 172.21.15.100:0/3037237561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc00adfa0 0x7f8cc0142520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00097c0 tx=0x7f8cb001ace0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.602+0000 7f8cc5c24640 1 -- 172.21.15.100:0/3037237561 shutdown_connections 2024-04-25T10:04:58.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.602+0000 7f8cc5c24640 1 --2- 172.21.15.100:0/3037237561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cc015e0a0 0x7f8cc01f8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.602+0000 7f8cc5c24640 1 --2- 172.21.15.100:0/3037237561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc00ae380 0x7f8cc01f7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.602+0000 7f8cc5c24640 1 --2- 172.21.15.100:0/3037237561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc00adfa0 0x7f8cc0142520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.602+0000 7f8cc5c24640 1 -- 172.21.15.100:0/3037237561 >> 172.21.15.100:0/3037237561 conn(0x7f8cc00cfdd0 msgr2=0x7f8cc00b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:58.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.602+0000 7f8cc5c24640 1 -- 172.21.15.100:0/3037237561 shutdown_connections 2024-04-25T10:04:58.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.602+0000 7f8cc5c24640 1 -- 172.21.15.100:0/3037237561 wait complete. 2024-04-25T10:04:58.623 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new ffd96246-26fa-4837-a994-a1df9369b36e 7 2024-04-25T10:04:58.648 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:58.642+0000 7f2f389e0180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:04:58.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.734+0000 7fda91b19640 1 Processor -- start 2024-04-25T10:04:58.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.734+0000 7fda91b19640 1 -- start start 2024-04-25T10:04:58.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.734+0000 7fda91b19640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fda8c072620 0x7fda8c15d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.734+0000 7fda91b19640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda8c06c330 0x7fda8c15d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.734+0000 7fda91b19640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda8c15c0f0 con 0x7fda8c15f160 2024-04-25T10:04:58.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.734+0000 7fda91b19640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda8c15c3b0 con 0x7fda8c072620 2024-04-25T10:04:58.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.734+0000 7fda91b19640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c15c690 con 0x7fda8c06c330 2024-04-25T10:04:58.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.734+0000 7fda8affd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fda8c072620 0x7fda8c15d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda8bfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda8c06c330 0x7fda8c15d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda8affd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fda8c072620 0x7fda8c15d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60026/0 (socket says 172.21.15.100:60026) 2024-04-25T10:04:58.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda8affd640 1 -- 172.21.15.100:0/2185519845 learned_addr learned my addr 172.21.15.100:0/2185519845 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T10:04:58.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda8bfff640 1 -- 172.21.15.100:0/2185519845 >> v2:172.21.15.100:3300/0 conn(0x7fda8c072620 msgr2=0x7fda8c15d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:58.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda8bfff640 1 --2- 172.21.15.100:0/2185519845 >> v2:172.21.15.100:3300/0 conn(0x7fda8c072620 0x7fda8c15d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda8bfff640 1 -- 172.21.15.100:0/2185519845 >> v1:172.21.15.60:6789/0 conn(0x7fda8c15f160 legacy=0x7fda8c06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda8bfff640 1 -- 172.21.15.100:0/2185519845 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c15c970 con 0x7fda8c06c330 2024-04-25T10:04:58.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda8affd640 1 --2- 172.21.15.100:0/2185519845 >> v2:172.21.15.100:3300/0 conn(0x7fda8c072620 0x7fda8c15d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:58.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda8bfff640 1 --2- 172.21.15.100:0/2185519845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda8c06c330 0x7fda8c15d9a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0097c0 tx=0x7fda7c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4605e77ec32bf47d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda8a7fc640 1 -- 172.21.15.100:0/2185519845 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c0b0070 con 0x7fda8c06c330 2024-04-25T10:04:58.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda8a7fc640 1 -- 172.21.15.100:0/2185519845 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7c0ab600 con 0x7fda8c06c330 2024-04-25T10:04:58.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda8a7fc640 1 -- 172.21.15.100:0/2185519845 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c0ab9d0 con 0x7fda8c06c330 2024-04-25T10:04:58.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda91b19640 1 -- 172.21.15.100:0/2185519845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda8c06c330 msgr2=0x7fda8c15d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda91b19640 1 --2- 172.21.15.100:0/2185519845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda8c06c330 0x7fda8c15d9a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0097c0 tx=0x7fda7c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda91b19640 1 -- 172.21.15.100:0/2185519845 shutdown_connections 2024-04-25T10:04:58.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda91b19640 1 --2- 172.21.15.100:0/2185519845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda8c06c330 0x7fda8c15d9a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda91b19640 1 --2- 172.21.15.100:0/2185519845 >> v2:172.21.15.100:3300/0 conn(0x7fda8c072620 0x7fda8c15d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda91b19640 1 -- 172.21.15.100:0/2185519845 >> 172.21.15.100:0/2185519845 conn(0x7fda8c071b00 msgr2=0x7fda8c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:58.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda91b19640 1 -- 172.21.15.100:0/2185519845 shutdown_connections 2024-04-25T10:04:58.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.738+0000 7fda91b19640 1 -- 172.21.15.100:0/2185519845 wait complete. 2024-04-25T10:04:58.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda91b19640 1 Processor -- start 2024-04-25T10:04:58.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda91b19640 1 -- start start 2024-04-25T10:04:58.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda91b19640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda8c06c330 0x7fda8c146b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda91b19640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda8c072620 0x7fda8c099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda91b19640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda8c15f160 0x7fda8c149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:58.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda91b19640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c168e10 con 0x7fda8c072620 2024-04-25T10:04:58.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda91b19640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c168c90 con 0x7fda8c15f160 2024-04-25T10:04:58.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda91b19640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c168f90 con 0x7fda8c06c330 2024-04-25T10:04:58.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda8b7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda8c06c330 0x7fda8c146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda8b7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda8c06c330 0x7fda8c146b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:33026/0 (socket says 172.21.15.100:33026) 2024-04-25T10:04:58.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda8bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda8c15f160 0x7fda8c149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda8b7fe640 1 -- 172.21.15.100:0/238869215 learned_addr learned my addr 172.21.15.100:0/238869215 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T10:04:58.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda8affd640 1 --2- 172.21.15.100:0/238869215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda8c072620 0x7fda8c099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:58.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda8b7fe640 1 -- 172.21.15.100:0/238869215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda8c15f160 msgr2=0x7fda8c149060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda8b7fe640 1 --2- 172.21.15.100:0/238869215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda8c15f160 0x7fda8c149060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda8b7fe640 1 -- 172.21.15.100:0/238869215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda8c072620 msgr2=0x7fda8c099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda8b7fe640 1 --2- 172.21.15.100:0/238869215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda8c072620 0x7fda8c099a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda8b7fe640 1 -- 172.21.15.100:0/238869215 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c1470c0 con 0x7fda8c06c330 2024-04-25T10:04:58.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda8bfff640 1 --2- 172.21.15.100:0/238869215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda8c15f160 0x7fda8c149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:58.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda8b7fe640 1 --2- 172.21.15.100:0/238869215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda8c06c330 0x7fda8c146b80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fda8400c790 tx=0x7fda8400cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:58.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda73fff640 1 -- 172.21.15.100:0/238869215 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda84016020 con 0x7fda8c06c330 2024-04-25T10:04:58.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda73fff640 1 -- 172.21.15.100:0/238869215 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda84006d60 con 0x7fda8c06c330 2024-04-25T10:04:58.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda91b19640 1 -- 172.21.15.100:0/238869215 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda8c152ca0 con 0x7fda8c06c330 2024-04-25T10:04:58.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda73fff640 1 -- 172.21.15.100:0/238869215 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda84003bc0 con 0x7fda8c06c330 2024-04-25T10:04:58.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda91b19640 1 -- 172.21.15.100:0/238869215 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda8c14b520 con 0x7fda8c06c330 2024-04-25T10:04:58.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda73fff640 1 -- 172.21.15.100:0/238869215 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fda8400abe0 con 0x7fda8c06c330 2024-04-25T10:04:58.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda73fff640 1 -- 172.21.15.100:0/238869215 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fda84012ad0 con 0x7fda8c06c330 2024-04-25T10:04:58.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.742+0000 7fda91b19640 1 -- 172.21.15.100:0/238869215 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda8c14b030 con 0x7fda8c06c330 2024-04-25T10:04:58.751 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:58.739+0000 7f7287c22180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:04:58.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.746+0000 7fda73fff640 1 -- 172.21.15.100:0/238869215 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fda84020020 con 0x7fda8c06c330 2024-04-25T10:04:58.813 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:58.806+0000 7f2f389e0180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:04:58.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.814+0000 7fda91b19640 1 -- 172.21.15.100:0/238869215 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ffd96246-26fa-4837-a994-a1df9369b36e", "id": 7} v 0) v1 -- 0x7fda8c0af680 con 0x7fda8c06c330 2024-04-25T10:04:58.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.822+0000 7fda73fff640 1 -- 172.21.15.100:0/238869215 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ffd96246-26fa-4837-a994-a1df9369b36e", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fda84012d10 con 0x7fda8c06c330 2024-04-25T10:04:58.827 INFO:teuthology.orchestra.run.smithi100.stdout:7 2024-04-25T10:04:58.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.822+0000 7fda91b19640 1 -- 172.21.15.100:0/238869215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda8c06c330 msgr2=0x7fda8c146b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:58.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.822+0000 7fda91b19640 1 --2- 172.21.15.100:0/238869215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda8c06c330 0x7fda8c146b80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fda8400c790 tx=0x7fda8400cc50 comp rx=0 tx=0).stop 2024-04-25T10:04:58.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.822+0000 7fda91b19640 1 -- 172.21.15.100:0/238869215 shutdown_connections 2024-04-25T10:04:58.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.822+0000 7fda91b19640 1 --2- 172.21.15.100:0/238869215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda8c15f160 0x7fda8c149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.822+0000 7fda91b19640 1 --2- 172.21.15.100:0/238869215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda8c072620 0x7fda8c099a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.822+0000 7fda91b19640 1 --2- 172.21.15.100:0/238869215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda8c06c330 0x7fda8c146b80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:58.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.822+0000 7fda91b19640 1 -- 172.21.15.100:0/238869215 >> 172.21.15.100:0/238869215 conn(0x7fda8c071b00 msgr2=0x7fda8c0b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:58.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.822+0000 7fda91b19640 1 -- 172.21.15.100:0/238869215 shutdown_connections 2024-04-25T10:04:58.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:04:58.822+0000 7fda91b19640 1 -- 172.21.15.100:0/238869215 wait complete. 2024-04-25T10:04:58.843 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T10:04:58.843 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T10:04:58.846 INFO:tasks.ceph.osd.0:Started 2024-04-25T10:04:58.846 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T10:04:58.846 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T10:04:58.848 INFO:tasks.ceph.osd.1:Started 2024-04-25T10:04:58.848 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T10:04:58.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T10:04:58.851 INFO:tasks.ceph.osd.2:Started 2024-04-25T10:04:58.851 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T10:04:58.851 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T10:04:58.853 INFO:tasks.ceph.osd.3:Started 2024-04-25T10:04:58.853 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T10:04:58.853 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T10:04:58.880 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:58.874+0000 7f2f389e0180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:04:58.881 INFO:tasks.ceph.osd.4:Started 2024-04-25T10:04:58.881 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T10:04:58.881 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T10:04:58.883 INFO:tasks.ceph.osd.5:Started 2024-04-25T10:04:58.883 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T10:04:58.883 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T10:04:58.885 INFO:tasks.ceph.osd.6:Started 2024-04-25T10:04:58.885 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T10:04:58.885 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T10:04:58.887 INFO:tasks.ceph.osd.7:Started 2024-04-25T10:04:58.888 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:58.933 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:58.927+0000 7f7287c22180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:04:59.006 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:58.999+0000 7f7287c22180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:04:59.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.007+0000 7f3748c7e640 1 Processor -- start 2024-04-25T10:04:59.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.007+0000 7f3748c7e640 1 -- start start 2024-04-25T10:04:59.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f3748c7e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f37440cda70 0x7f37440cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:59.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f3748c7e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37440ce080 con 0x7f374406ac10 2024-04-25T10:04:59.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f3748c7e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37440ce360 con 0x7f37440ad580 2024-04-25T10:04:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f3748c7e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37440ce640 con 0x7f37440cda70 2024-04-25T10:04:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f374259c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f374406ac10 0x7f374406aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53150/0 (socket says 172.21.15.60:53150) 2024-04-25T10:04:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f374259c640 1 -- 172.21.15.60:0/1529931660 learned_addr learned my addr 172.21.15.60:0/1529931660 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:04:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f3742d9d640 1 --2- 172.21.15.60:0/1529931660 >> v2:172.21.15.100:3300/0 conn(0x7f37440cda70 0x7f37440cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f374159a640 1 -- 172.21.15.60:0/1529931660 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2070089086 0 0) 0x7f37440ce080 con 0x7f374406ac10 2024-04-25T10:04:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f374159a640 1 -- 172.21.15.60:0/1529931660 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3730003530 con 0x7f374406ac10 2024-04-25T10:04:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f3742d9d640 1 -- 172.21.15.60:0/1529931660 >> v1:172.21.15.100:6789/0 conn(0x7f37440ad580 legacy=0x7f37440cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f3742d9d640 1 -- 172.21.15.60:0/1529931660 >> v1:172.21.15.60:6789/0 conn(0x7f374406ac10 legacy=0x7f374406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f3742d9d640 1 -- 172.21.15.60:0/1529931660 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37440ce920 con 0x7f37440cda70 2024-04-25T10:04:59.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f3742d9d640 1 --2- 172.21.15.60:0/1529931660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37440cda70 0x7f37440cf830 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f373c009d90 tx=0x7f373c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17b0344e03b4278 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:59.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f374159a640 1 -- 172.21.15.60:0/1529931660 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c0b2070 con 0x7f37440cda70 2024-04-25T10:04:59.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f374159a640 1 -- 172.21.15.60:0/1529931660 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373c0a5940 con 0x7f37440cda70 2024-04-25T10:04:59.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f374159a640 1 -- 172.21.15.60:0/1529931660 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c0a5d10 con 0x7f37440cda70 2024-04-25T10:04:59.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f3748c7e640 1 -- 172.21.15.60:0/1529931660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37440cda70 msgr2=0x7f37440cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.011+0000 7f3748c7e640 1 --2- 172.21.15.60:0/1529931660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37440cda70 0x7f37440cf830 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f373c009d90 tx=0x7f373c01c970 comp rx=0 tx=0).stop 2024-04-25T10:04:59.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3748c7e640 1 -- 172.21.15.60:0/1529931660 shutdown_connections 2024-04-25T10:04:59.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3748c7e640 1 --2- 172.21.15.60:0/1529931660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37440cda70 0x7f37440cf830 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3748c7e640 1 -- 172.21.15.60:0/1529931660 >> 172.21.15.60:0/1529931660 conn(0x7f3744072920 msgr2=0x7f3744099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:59.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3748c7e640 1 -- 172.21.15.60:0/1529931660 shutdown_connections 2024-04-25T10:04:59.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3748c7e640 1 -- 172.21.15.60:0/1529931660 wait complete. 2024-04-25T10:04:59.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3748c7e640 1 Processor -- start 2024-04-25T10:04:59.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3748c7e640 1 -- start start 2024-04-25T10:04:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3748c7e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f374406ac10 0x7f37440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3748c7e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37440ad580 0x7f374415e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3748c7e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37440cda70 0x7f374413bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3748c7e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37441455e0 con 0x7f374406ac10 2024-04-25T10:04:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3748c7e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3744145460 con 0x7f37440ad580 2024-04-25T10:04:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3748c7e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3744169c60 con 0x7f37440cda70 2024-04-25T10:04:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f374259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f374406ac10 0x7f37440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f374259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f374406ac10 0x7f37440b4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33998/0 (socket says 172.21.15.60:33998) 2024-04-25T10:04:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f374259c640 1 -- 172.21.15.60:0/2389409329 learned_addr learned my addr 172.21.15.60:0/2389409329 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:04:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3741d9b640 1 --2- 172.21.15.60:0/2389409329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37440ad580 0x7f374415e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3742d9d640 1 --2- 172.21.15.60:0/2389409329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37440cda70 0x7f374413bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f374259c640 1 -- 172.21.15.60:0/2389409329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37440cda70 msgr2=0x7f374413bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f374259c640 1 --2- 172.21.15.60:0/2389409329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37440cda70 0x7f374413bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f374259c640 1 -- 172.21.15.60:0/2389409329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37440ad580 msgr2=0x7f374415e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f374259c640 1 --2- 172.21.15.60:0/2389409329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37440ad580 0x7f374415e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f374259c640 1 -- 172.21.15.60:0/2389409329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374415dfe0 con 0x7f374406ac10 2024-04-25T10:04:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f374259c640 1 --2- 172.21.15.60:0/2389409329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f374406ac10 0x7f37440b4dc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f372c00c4d0 tx=0x7f372c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f373affd640 1 -- 172.21.15.60:0/2389409329 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c017070 con 0x7f374406ac10 2024-04-25T10:04:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f373affd640 1 -- 172.21.15.60:0/2389409329 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c005cf0 con 0x7f374406ac10 2024-04-25T10:04:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f373affd640 1 -- 172.21.15.60:0/2389409329 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c007270 con 0x7f374406ac10 2024-04-25T10:04:59.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3748c7e640 1 -- 172.21.15.60:0/2389409329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37440b5700 con 0x7f374406ac10 2024-04-25T10:04:59.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f3748c7e640 1 -- 172.21.15.60:0/2389409329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37440cad20 con 0x7f374406ac10 2024-04-25T10:04:59.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f373affd640 1 -- 172.21.15.60:0/2389409329 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f372c01a070 con 0x7f374406ac10 2024-04-25T10:04:59.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.015+0000 7f373affd640 1 -- 172.21.15.60:0/2389409329 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f372c01cb70 con 0x7f374406ac10 2024-04-25T10:04:59.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.019+0000 7f3748c7e640 1 -- 172.21.15.60:0/2389409329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37440ade80 con 0x7f374406ac10 2024-04-25T10:04:59.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.023+0000 7f373affd640 1 -- 172.21.15.60:0/2389409329 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f372c007430 con 0x7f374406ac10 2024-04-25T10:04:59.041 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:59.034+0000 7f2f389e0180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:04:59.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.079+0000 7f3748c7e640 1 -- 172.21.15.60:0/2389409329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3744074770 con 0x7f374406ac10 2024-04-25T10:04:59.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.079+0000 7f373affd640 1 -- 172.21.15.60:0/2389409329 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f372c007670 con 0x7f374406ac10 2024-04-25T10:04:59.089 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:04:59.089 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":10,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:04:58.818141+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:59.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.083+0000 7f3748c7e640 1 -- 172.21.15.60:0/2389409329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f374406ac10 msgr2=0x7f37440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:59.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.083+0000 7f3748c7e640 1 --2- 172.21.15.60:0/2389409329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f374406ac10 0x7f37440b4dc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f372c00c4d0 tx=0x7f372c00c990 comp rx=0 tx=0).stop 2024-04-25T10:04:59.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.083+0000 7f3748c7e640 1 -- 172.21.15.60:0/2389409329 shutdown_connections 2024-04-25T10:04:59.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.083+0000 7f3748c7e640 1 --2- 172.21.15.60:0/2389409329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37440cda70 0x7f374413bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.083+0000 7f3748c7e640 1 --2- 172.21.15.60:0/2389409329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37440ad580 0x7f374415e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.083+0000 7f3748c7e640 1 --2- 172.21.15.60:0/2389409329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f374406ac10 0x7f37440b4dc0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:59.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.083+0000 7f3748c7e640 1 -- 172.21.15.60:0/2389409329 >> 172.21.15.60:0/2389409329 conn(0x7f3744072920 msgr2=0x7f374406ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:59.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.083+0000 7f3748c7e640 1 -- 172.21.15.60:0/2389409329 shutdown_connections 2024-04-25T10:04:59.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:04:59.083+0000 7f3748c7e640 1 -- 172.21.15.60:0/2389409329 wait complete. 2024-04-25T10:04:59.108 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T10:04:59.108 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T10:04:59.162 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:59.155+0000 7f7287c22180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:04:59.484 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-25T10:04:59.478+0000 7f7cb14837c0 -1 Falling back to public interface 2024-04-25T10:04:59.494 INFO:tasks.ceph.osd.0.smithi060.stderr:2024-04-25T10:04:59.487+0000 7f197d0a27c0 -1 Falling back to public interface 2024-04-25T10:04:59.494 INFO:tasks.ceph.osd.2.smithi060.stderr:2024-04-25T10:04:59.487+0000 7fc1c8d2c7c0 -1 Falling back to public interface 2024-04-25T10:04:59.494 INFO:tasks.ceph.osd.0.smithi060.stderr:2024-04-25T10:04:59.487+0000 7f197c062640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-04-25T10:04:59.495 INFO:tasks.ceph.osd.3.smithi060.stderr:2024-04-25T10:04:59.487+0000 7f3103d187c0 -1 Falling back to public interface 2024-04-25T10:04:59.495 INFO:tasks.ceph.osd.2.smithi060.stderr:2024-04-25T10:04:59.487+0000 7fc1c7cec640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6812/0: (98) Address already in use 2024-04-25T10:04:59.496 INFO:tasks.ceph.osd.3.smithi060.stderr:2024-04-25T10:04:59.487+0000 7f3102cd8640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6812/0: (98) Address already in use 2024-04-25T10:04:59.499 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-25T10:04:59.494+0000 7ff0d94bd7c0 -1 Falling back to public interface 2024-04-25T10:04:59.503 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-25T10:04:59.498+0000 7f77dc8d17c0 -1 Falling back to public interface 2024-04-25T10:04:59.509 INFO:tasks.ceph.osd.1.smithi060.stderr:2024-04-25T10:04:59.503+0000 7f3b982ee7c0 -1 Falling back to public interface 2024-04-25T10:04:59.515 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-25T10:04:59.510+0000 7f5fdeff37c0 -1 Falling back to public interface 2024-04-25T10:04:59.669 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:59.662+0000 7f2f389e0180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:04:59.745 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:59.738+0000 7f2f389e0180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:04:59.814 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:59.807+0000 7f7287c22180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:04:59.894 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:04:59.887+0000 7f7287c22180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:04:59.914 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:59.926 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:59.918+0000 7f2f389e0180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:04:59.999 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:04:59.994+0000 7f2f389e0180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:05:00.028 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.019+0000 7febd3607640 1 Processor -- start 2024-04-25T10:05:00.028 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.019+0000 7febd3607640 1 -- start start 2024-04-25T10:05:00.029 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd3607640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7febcc0cf220 0x7febcc0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.030 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd3607640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febcc0b4de0 0x7febcc0ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.030 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd3607640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febcc0cd780 con 0x7febcc0cc300 2024-04-25T10:05:00.030 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd3607640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febcc072de0 con 0x7febcc0cf220 2024-04-25T10:05:00.030 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd3607640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febcc15beb0 con 0x7febcc0b4de0 2024-04-25T10:05:00.030 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd13a3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7febcc0cf220 0x7febcc0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.030 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd13a3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7febcc0cf220 0x7febcc0cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34000/0 (socket says 172.21.15.60:34000) 2024-04-25T10:05:00.030 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd13a3640 1 -- 172.21.15.60:0/1227684104 learned_addr learned my addr 172.21.15.60:0/1227684104 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:00.030 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd1ba4640 1 --2- 172.21.15.60:0/1227684104 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febcc0b4de0 0x7febcc0ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.030 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd13a3640 1 -- 172.21.15.60:0/1227684104 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febcc0b4de0 msgr2=0x7febcc0ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.030 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd13a3640 1 --2- 172.21.15.60:0/1227684104 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febcc0b4de0 0x7febcc0ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.030 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd13a3640 1 -- 172.21.15.60:0/1227684104 >> v1:172.21.15.100:6789/0 conn(0x7febcc0cc300 legacy=0x7febcc0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.030 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd13a3640 1 -- 172.21.15.60:0/1227684104 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc0cfb40 con 0x7febcc0cf220 2024-04-25T10:05:00.032 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd13a3640 1 --2- 172.21.15.60:0/1227684104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febcc0cf220 0x7febcc0cf600 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7febc8009870 tx=0x7febc801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1b3ccd19fd7de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.032 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febc3fff640 1 -- 172.21.15.60:0/1227684104 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc80b0070 con 0x7febcc0cf220 2024-04-25T10:05:00.033 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febc3fff640 1 -- 172.21.15.60:0/1227684104 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc80ab600 con 0x7febcc0cf220 2024-04-25T10:05:00.033 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febc3fff640 1 -- 172.21.15.60:0/1227684104 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc80ab9d0 con 0x7febcc0cf220 2024-04-25T10:05:00.033 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd3607640 1 -- 172.21.15.60:0/1227684104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febcc0cf220 msgr2=0x7febcc0cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.033 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd3607640 1 --2- 172.21.15.60:0/1227684104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febcc0cf220 0x7febcc0cf600 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7febc8009870 tx=0x7febc801c460 comp rx=0 tx=0).stop 2024-04-25T10:05:00.033 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd3607640 1 -- 172.21.15.60:0/1227684104 shutdown_connections 2024-04-25T10:05:00.033 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd3607640 1 --2- 172.21.15.60:0/1227684104 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febcc0b4de0 0x7febcc0ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.033 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.023+0000 7febd3607640 1 --2- 172.21.15.60:0/1227684104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febcc0cf220 0x7febcc0cf600 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.033 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd3607640 1 -- 172.21.15.60:0/1227684104 >> 172.21.15.60:0/1227684104 conn(0x7febcc0cfda0 msgr2=0x7febcc0d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:00.033 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd3607640 1 -- 172.21.15.60:0/1227684104 shutdown_connections 2024-04-25T10:05:00.033 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd3607640 1 -- 172.21.15.60:0/1227684104 wait complete. 2024-04-25T10:05:00.034 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd3607640 1 Processor -- start 2024-04-25T10:05:00.034 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd3607640 1 -- start start 2024-04-25T10:05:00.036 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd3607640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febcc0b4de0 0x7febcc0ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.036 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd3607640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febcc0cc300 0x7febcc09a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.036 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd3607640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febcc0cf220 0x7febcc09a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.036 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd3607640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc15ccb0 con 0x7febcc0cf220 2024-04-25T10:05:00.036 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd3607640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc15c9d0 con 0x7febcc0b4de0 2024-04-25T10:05:00.036 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd3607640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febcc06ac70 con 0x7febcc0cc300 2024-04-25T10:05:00.036 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd1ba4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febcc0cf220 0x7febcc09a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.037 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd1ba4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febcc0cf220 0x7febcc09a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34014/0 (socket says 172.21.15.60:34014) 2024-04-25T10:05:00.037 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd1ba4640 1 -- 172.21.15.60:0/860255019 learned_addr learned my addr 172.21.15.60:0/860255019 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:00.037 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd13a3640 1 --2- 172.21.15.60:0/860255019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febcc0b4de0 0x7febcc0ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.037 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd0ba2640 1 --2- 172.21.15.60:0/860255019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febcc0cc300 0x7febcc09a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.037 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd1ba4640 1 -- 172.21.15.60:0/860255019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febcc0cc300 msgr2=0x7febcc09a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.037 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd1ba4640 1 --2- 172.21.15.60:0/860255019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febcc0cc300 0x7febcc09a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.037 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd1ba4640 1 -- 172.21.15.60:0/860255019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febcc0b4de0 msgr2=0x7febcc0ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.037 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd1ba4640 1 --2- 172.21.15.60:0/860255019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febcc0b4de0 0x7febcc0ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.037 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd1ba4640 1 -- 172.21.15.60:0/860255019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc0ae710 con 0x7febcc0cf220 2024-04-25T10:05:00.037 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd1ba4640 1 --2- 172.21.15.60:0/860255019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febcc0cf220 0x7febcc09a650 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7febc4007e90 tx=0x7febc400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.037 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febc1ffb640 1 -- 172.21.15.60:0/860255019 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4014420 con 0x7febcc0cf220 2024-04-25T10:05:00.037 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febc1ffb640 1 -- 172.21.15.60:0/860255019 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc4019070 con 0x7febcc0cf220 2024-04-25T10:05:00.037 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd3607640 1 -- 172.21.15.60:0/860255019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febcc0ade70 con 0x7febcc0cf220 2024-04-25T10:05:00.037 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febc1ffb640 1 -- 172.21.15.60:0/860255019 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4014800 con 0x7febcc0cf220 2024-04-25T10:05:00.038 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd3607640 1 -- 172.21.15.60:0/860255019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc0ae430 con 0x7febcc0cf220 2024-04-25T10:05:00.038 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febc1ffb640 1 -- 172.21.15.60:0/860255019 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7febc40149a0 con 0x7febcc0cf220 2024-04-25T10:05:00.038 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febc1ffb640 1 -- 172.21.15.60:0/860255019 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7febc4014c90 con 0x7febcc0cf220 2024-04-25T10:05:00.038 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.027+0000 7febd3607640 1 -- 172.21.15.60:0/860255019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febcc04ef30 con 0x7febcc0cf220 2024-04-25T10:05:00.038 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.031+0000 7febc1ffb640 1 -- 172.21.15.60:0/860255019 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7febc401e510 con 0x7febcc0cf220 2024-04-25T10:05:00.096 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.087+0000 7febd3607640 1 -- 172.21.15.60:0/860255019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febcc0b3070 con 0x7febcc0cf220 2024-04-25T10:05:00.097 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.091+0000 7febc1ffb640 1 -- 172.21.15.60:0/860255019 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7febc401e7a0 con 0x7febcc0cf220 2024-04-25T10:05:00.097 INFO:teuthology.misc.health.smithi060.stdout: 2024-04-25T10:05:00.097 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":10,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:04:58.818141+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:00.099 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.091+0000 7febd3607640 1 -- 172.21.15.60:0/860255019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febcc0cf220 msgr2=0x7febcc09a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.099 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.091+0000 7febd3607640 1 --2- 172.21.15.60:0/860255019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febcc0cf220 0x7febcc09a650 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7febc4007e90 tx=0x7febc400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.099 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.091+0000 7febd3607640 1 -- 172.21.15.60:0/860255019 shutdown_connections 2024-04-25T10:05:00.099 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.091+0000 7febd3607640 1 --2- 172.21.15.60:0/860255019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febcc0cf220 0x7febcc09a650 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.099 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.091+0000 7febd3607640 1 --2- 172.21.15.60:0/860255019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febcc0cc300 0x7febcc09a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.099 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.091+0000 7febd3607640 1 --2- 172.21.15.60:0/860255019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febcc0b4de0 0x7febcc0ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.099 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.091+0000 7febd3607640 1 -- 172.21.15.60:0/860255019 >> 172.21.15.60:0/860255019 conn(0x7febcc0cfda0 msgr2=0x7febcc15e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:00.099 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.091+0000 7febd3607640 1 -- 172.21.15.60:0/860255019 shutdown_connections 2024-04-25T10:05:00.100 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:00.091+0000 7febd3607640 1 -- 172.21.15.60:0/860255019 wait complete. 2024-04-25T10:05:00.108 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:05:00.099+0000 7f7287c22180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:05:00.113 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:05:00.188 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:05:00.179+0000 7f7287c22180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:05:00.290 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:05:00.286+0000 7f2f389e0180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:05:00.353 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:05:00.346+0000 7f2f389e0180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:05:00.498 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:05:00.491+0000 7f7287c22180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:05:00.563 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T10:05:00.555+0000 7f7287c22180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:05:01.396 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:05:01.390+0000 7f2edfe29640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T10:05:01.396 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T10:05:01.390+0000 7f2edfe29640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T10:05:03.244 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-25T10:05:03.238+0000 7f7cb14837c0 -1 osd.5 0 log_to_monitors true 2024-04-25T10:05:03.330 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-25T10:05:03.322+0000 7f5fdeff37c0 -1 osd.6 0 log_to_monitors true 2024-04-25T10:05:03.333 INFO:tasks.ceph.osd.2.smithi060.stderr:2024-04-25T10:05:03.327+0000 7fc1c8d2c7c0 -1 osd.2 0 log_to_monitors true 2024-04-25T10:05:03.370 INFO:tasks.ceph.osd.3.smithi060.stderr:2024-04-25T10:05:03.363+0000 7f3103d187c0 -1 osd.3 0 log_to_monitors true 2024-04-25T10:05:03.371 INFO:tasks.ceph.osd.0.smithi060.stderr:2024-04-25T10:05:03.363+0000 7f197d0a27c0 -1 osd.0 0 log_to_monitors true 2024-04-25T10:05:03.388 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-25T10:05:03.382+0000 7ff0d94bd7c0 -1 osd.4 0 log_to_monitors true 2024-04-25T10:05:03.426 INFO:tasks.ceph.osd.1.smithi060.stderr:2024-04-25T10:05:03.419+0000 7f3b982ee7c0 -1 osd.1 0 log_to_monitors true 2024-04-25T10:05:03.431 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-25T10:05:03.426+0000 7f77dc8d17c0 -1 osd.7 0 log_to_monitors true 2024-04-25T10:05:05.654 INFO:tasks.ceph.osd.2.smithi060.stderr:2024-04-25T10:05:05.647+0000 7fc1c44b2640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T10:05:05.663 INFO:tasks.ceph.osd.3.smithi060.stderr:2024-04-25T10:05:05.655+0000 7f3100cd4640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T10:05:05.678 INFO:tasks.ceph.osd.2.smithi060.stderr:2024-04-25T10:05:05.671+0000 7fc1bcaf9640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:05:05.691 INFO:tasks.ceph.osd.3.smithi060.stderr:2024-04-25T10:05:05.683+0000 7f30f7ae5640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:05:05.696 INFO:tasks.ceph.osd.0.smithi060.stderr:2024-04-25T10:05:05.687+0000 7f1978828640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T10:05:05.739 INFO:tasks.ceph.osd.0.smithi060.stderr:2024-04-25T10:05:05.731+0000 7f1970e6f640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:05:05.816 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-25T10:05:05.810+0000 7f5fdbfaf640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T10:05:05.865 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-25T10:05:05.858+0000 7f5fd2dc0640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:05:05.926 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-25T10:05:05.918+0000 7f7cacc09640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T10:05:05.971 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-25T10:05:05.966+0000 7f7ca5250640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:05:06.251 INFO:tasks.ceph.osd.1.smithi060.stderr:2024-04-25T10:05:06.243+0000 7f3b93a74640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T10:05:06.278 INFO:tasks.ceph.osd.1.smithi060.stderr:2024-04-25T10:05:06.271+0000 7f3b8c0bb640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:05:06.812 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-25T10:05:06.806+0000 7ff0d4c43640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T10:05:06.822 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-25T10:05:06.814+0000 7f77d988d640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T10:05:06.847 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-25T10:05:06.842+0000 7ff0cd28a640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:05:06.856 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-25T10:05:06.850+0000 7f77d069e640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:05:06.919 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:07.041 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.035+0000 7f5aec94f640 1 Processor -- start 2024-04-25T10:05:07.041 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.035+0000 7f5aec94f640 1 -- start start 2024-04-25T10:05:07.042 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.035+0000 7f5aec94f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ae8145120 0x7f5ae806dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.043 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.035+0000 7f5aec94f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae80cdcf0 con 0x7f5ae8071190 2024-04-25T10:05:07.043 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.035+0000 7f5aec94f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae806d420 con 0x7f5ae80cccc0 2024-04-25T10:05:07.043 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.035+0000 7f5aec94f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae806da30 con 0x7f5ae8145120 2024-04-25T10:05:07.043 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.035+0000 7f5ae659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ae8145120 0x7f5ae806dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.043 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.035+0000 7f5ae6d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5ae8071190 0x7f5ae815bdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53154/0 (socket says 172.21.15.60:53154) 2024-04-25T10:05:07.043 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.035+0000 7f5ae6d9d640 1 -- 172.21.15.60:0/3419347554 learned_addr learned my addr 172.21.15.60:0/3419347554 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:07.043 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.035+0000 7f5ae659c640 1 -- 172.21.15.60:0/3419347554 >> v1:172.21.15.100:6789/0 conn(0x7f5ae80cccc0 legacy=0x7f5ae80cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:07.043 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.035+0000 7f5ae659c640 1 -- 172.21.15.60:0/3419347554 >> v1:172.21.15.60:6789/0 conn(0x7f5ae8071190 legacy=0x7f5ae815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.043 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.035+0000 7f5ae659c640 1 -- 172.21.15.60:0/3419347554 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae8145500 con 0x7f5ae8145120 2024-04-25T10:05:07.043 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.035+0000 7f5ae559a640 1 -- 172.21.15.60:0/3419347554 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491588923 0 0) 0x7f5ae80cdcf0 con 0x7f5ae8071190 2024-04-25T10:05:07.045 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5ae659c640 1 --2- 172.21.15.60:0/3419347554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8145120 0x7f5ae806dec0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0097c0 tx=0x7f5adc01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2143c5db4bffef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.045 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5ae559a640 1 -- 172.21.15.60:0/3419347554 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0b1070 con 0x7f5ae8145120 2024-04-25T10:05:07.045 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5ae559a640 1 -- 172.21.15.60:0/3419347554 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5adc0ac600 con 0x7f5ae8145120 2024-04-25T10:05:07.045 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5ae559a640 1 -- 172.21.15.60:0/3419347554 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0ac9d0 con 0x7f5ae8145120 2024-04-25T10:05:07.046 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5aec94f640 1 -- 172.21.15.60:0/3419347554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8145120 msgr2=0x7f5ae806dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.046 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5aec94f640 1 --2- 172.21.15.60:0/3419347554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8145120 0x7f5ae806dec0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0097c0 tx=0x7f5adc01c3a0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.046 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5aec94f640 1 -- 172.21.15.60:0/3419347554 shutdown_connections 2024-04-25T10:05:07.046 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5aec94f640 1 --2- 172.21.15.60:0/3419347554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8145120 0x7f5ae806dec0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.046 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5aec94f640 1 -- 172.21.15.60:0/3419347554 >> 172.21.15.60:0/3419347554 conn(0x7f5ae80670f0 msgr2=0x7f5ae806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:07.046 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5aec94f640 1 -- 172.21.15.60:0/3419347554 shutdown_connections 2024-04-25T10:05:07.046 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5aec94f640 1 -- 172.21.15.60:0/3419347554 wait complete. 2024-04-25T10:05:07.047 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5aec94f640 1 Processor -- start 2024-04-25T10:05:07.047 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5aec94f640 1 -- start start 2024-04-25T10:05:07.050 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5aec94f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ae8071190 0x7f5ae813a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.050 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5aec94f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ae80cccc0 0x7f5ae80b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.050 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5aec94f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8145120 0x7f5ae813b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.050 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5aec94f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae81694d0 con 0x7f5ae8145120 2024-04-25T10:05:07.050 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5aec94f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae8071a10 con 0x7f5ae80cccc0 2024-04-25T10:05:07.050 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5aec94f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae8169650 con 0x7f5ae8071190 2024-04-25T10:05:07.051 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5ae6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8145120 0x7f5ae813b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.051 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5ae6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8145120 0x7f5ae813b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34078/0 (socket says 172.21.15.60:34078) 2024-04-25T10:05:07.051 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5ae6d9d640 1 -- 172.21.15.60:0/4230870948 learned_addr learned my addr 172.21.15.60:0/4230870948 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:07.051 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5ae5d9b640 1 --2- 172.21.15.60:0/4230870948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ae80cccc0 0x7f5ae80b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.051 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5ae659c640 1 --2- 172.21.15.60:0/4230870948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ae8071190 0x7f5ae813a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.051 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5ae6d9d640 1 -- 172.21.15.60:0/4230870948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ae8071190 msgr2=0x7f5ae813a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.051 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5ae6d9d640 1 --2- 172.21.15.60:0/4230870948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ae8071190 0x7f5ae813a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.051 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5ae6d9d640 1 -- 172.21.15.60:0/4230870948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ae80cccc0 msgr2=0x7f5ae80b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.051 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5ae6d9d640 1 --2- 172.21.15.60:0/4230870948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ae80cccc0 0x7f5ae80b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.051 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.039+0000 7f5ae6d9d640 1 -- 172.21.15.60:0/4230870948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae8159a60 con 0x7f5ae8145120 2024-04-25T10:05:07.051 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.043+0000 7f5ae6d9d640 1 --2- 172.21.15.60:0/4230870948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8145120 0x7f5ae813b990 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0001570 tx=0x7f5ae000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.051 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.043+0000 7f5ad6ffd640 1 -- 172.21.15.60:0/4230870948 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0018070 con 0x7f5ae8145120 2024-04-25T10:05:07.051 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.043+0000 7f5ad6ffd640 1 -- 172.21.15.60:0/4230870948 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae000a640 con 0x7f5ae8145120 2024-04-25T10:05:07.052 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.043+0000 7f5aec94f640 1 -- 172.21.15.60:0/4230870948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae8158d80 con 0x7f5ae8145120 2024-04-25T10:05:07.052 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.043+0000 7f5ad6ffd640 1 -- 172.21.15.60:0/4230870948 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000aa30 con 0x7f5ae8145120 2024-04-25T10:05:07.052 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.043+0000 7f5aec94f640 1 -- 172.21.15.60:0/4230870948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae8159340 con 0x7f5ae8145120 2024-04-25T10:05:07.052 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.043+0000 7f5aec94f640 1 -- 172.21.15.60:0/4230870948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae804ef30 con 0x7f5ae8145120 2024-04-25T10:05:07.052 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.043+0000 7f5ad6ffd640 1 -- 172.21.15.60:0/4230870948 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 86671+0+0 (secure 0 0 0) 0x7f5ae001b070 con 0x7f5ae8145120 2024-04-25T10:05:07.052 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.043+0000 7f5ad6ffd640 1 --2- 172.21.15.60:0/4230870948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5ac8066ed0 0x7f5ac8069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.053 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.043+0000 7f5ad6ffd640 1 -- 172.21.15.60:0/4230870948 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3911+0+0 (secure 0 0 0) 0x7f5ae0083e40 con 0x7f5ae8145120 2024-04-25T10:05:07.053 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.043+0000 7f5ae659c640 1 --2- 172.21.15.60:0/4230870948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5ac8066ed0 0x7f5ac8069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.053 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.043+0000 7f5ae659c640 1 --2- 172.21.15.60:0/4230870948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5ac8066ed0 0x7f5ac8069390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0ae000 tx=0x7f5adc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.055 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.047+0000 7f5ad6ffd640 1 -- 172.21.15.60:0/4230870948 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ae0055630 con 0x7f5ae8145120 2024-04-25T10:05:07.214 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.207+0000 7f5aec94f640 1 -- 172.21.15.60:0/4230870948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ae8159c30 con 0x7f5ae8145120 2024-04-25T10:05:07.215 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.207+0000 7f5ad6ffd640 1 -- 172.21.15.60:0/4230870948 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+9949 (secure 0 0 0) 0x7f5ae0054f30 con 0x7f5ae8145120 2024-04-25T10:05:07.215 INFO:teuthology.misc.health.smithi060.stdout: 2024-04-25T10:05:07.215 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":14,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:06.671348+0000","last_up_change":"2024-04-25T10:05:06.671348+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:07.218 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.211+0000 7f5aec94f640 1 -- 172.21.15.60:0/4230870948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5ac8066ed0 msgr2=0x7f5ac8069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.218 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.211+0000 7f5aec94f640 1 --2- 172.21.15.60:0/4230870948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5ac8066ed0 0x7f5ac8069390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0ae000 tx=0x7f5adc008040 comp rx=0 tx=0).stop 2024-04-25T10:05:07.218 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.211+0000 7f5aec94f640 1 -- 172.21.15.60:0/4230870948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8145120 msgr2=0x7f5ae813b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.218 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.211+0000 7f5aec94f640 1 --2- 172.21.15.60:0/4230870948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8145120 0x7f5ae813b990 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0001570 tx=0x7f5ae000cca0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.218 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.211+0000 7f5aec94f640 1 -- 172.21.15.60:0/4230870948 shutdown_connections 2024-04-25T10:05:07.218 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.211+0000 7f5aec94f640 1 --2- 172.21.15.60:0/4230870948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5ac8066ed0 0x7f5ac8069390 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.218 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.211+0000 7f5aec94f640 1 --2- 172.21.15.60:0/4230870948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae8145120 0x7f5ae813b990 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.218 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.211+0000 7f5aec94f640 1 --2- 172.21.15.60:0/4230870948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ae80cccc0 0x7f5ae80b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.218 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.211+0000 7f5aec94f640 1 --2- 172.21.15.60:0/4230870948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ae8071190 0x7f5ae813a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.218 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.211+0000 7f5aec94f640 1 -- 172.21.15.60:0/4230870948 >> 172.21.15.60:0/4230870948 conn(0x7f5ae80670f0 msgr2=0x7f5ae80cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:07.219 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.211+0000 7f5aec94f640 1 -- 172.21.15.60:0/4230870948 shutdown_connections 2024-04-25T10:05:07.219 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:07.211+0000 7f5aec94f640 1 -- 172.21.15.60:0/4230870948 wait complete. 2024-04-25T10:05:07.234 DEBUG:teuthology.misc:6 of 8 OSDs are up 2024-04-25T10:05:14.041 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:14.165 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.159+0000 7fdfccaeb640 1 Processor -- start 2024-04-25T10:05:14.165 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.159+0000 7fdfccaeb640 1 -- start start 2024-04-25T10:05:14.167 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.159+0000 7fdfccaeb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdfc806c390 0x7fdfc815df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.167 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.159+0000 7fdfccaeb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdfc80acd60 0x7fdfc80adf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.167 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.159+0000 7fdfccaeb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc815e5c0 con 0x7fdfc80adb20 2024-04-25T10:05:14.167 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.159+0000 7fdfccaeb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc815e8a0 con 0x7fdfc80acd60 2024-04-25T10:05:14.167 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.159+0000 7fdfccaeb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc815eb80 con 0x7fdfc806c390 2024-04-25T10:05:14.167 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.159+0000 7fdfc6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdfc80acd60 0x7fdfc80adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.167 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.159+0000 7fdfc6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdfc80acd60 0x7fdfc80adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38682/0 (socket says 172.21.15.60:38682) 2024-04-25T10:05:14.167 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.159+0000 7fdfc6d9d640 1 -- 172.21.15.60:0/3389276605 learned_addr learned my addr 172.21.15.60:0/3389276605 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:14.167 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.159+0000 7fdfc5d9b640 1 --2- 172.21.15.60:0/3389276605 >> v2:172.21.15.100:3300/0 conn(0x7fdfc806c390 0x7fdfc815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.167 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.159+0000 7fdfc6d9d640 1 -- 172.21.15.60:0/3389276605 >> v2:172.21.15.100:3300/0 conn(0x7fdfc806c390 msgr2=0x7fdfc815df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.168 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.159+0000 7fdfc6d9d640 1 --2- 172.21.15.60:0/3389276605 >> v2:172.21.15.100:3300/0 conn(0x7fdfc806c390 0x7fdfc815df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.168 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.159+0000 7fdfc6d9d640 1 -- 172.21.15.60:0/3389276605 >> v1:172.21.15.100:6789/0 conn(0x7fdfc80adb20 legacy=0x7fdfc806bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.168 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.159+0000 7fdfc6d9d640 1 -- 172.21.15.60:0/3389276605 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc815ee60 con 0x7fdfc80acd60 2024-04-25T10:05:14.169 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfc6d9d640 1 --2- 172.21.15.60:0/3389276605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfc80acd60 0x7fdfc80adf00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdfc806d8d0 tx=0x7fdfb801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fc104f1f37df63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.169 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfc559a640 1 -- 172.21.15.60:0/3389276605 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80ab640 con 0x7fdfc80acd60 2024-04-25T10:05:14.169 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfc559a640 1 -- 172.21.15.60:0/3389276605 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb80b0070 con 0x7fdfc80acd60 2024-04-25T10:05:14.169 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfc559a640 1 -- 172.21.15.60:0/3389276605 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80aba00 con 0x7fdfc80acd60 2024-04-25T10:05:14.170 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfccaeb640 1 -- 172.21.15.60:0/3389276605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfc80acd60 msgr2=0x7fdfc80adf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.170 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfccaeb640 1 --2- 172.21.15.60:0/3389276605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfc80acd60 0x7fdfc80adf00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdfc806d8d0 tx=0x7fdfb801c420 comp rx=0 tx=0).stop 2024-04-25T10:05:14.170 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfccaeb640 1 -- 172.21.15.60:0/3389276605 shutdown_connections 2024-04-25T10:05:14.170 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfccaeb640 1 --2- 172.21.15.60:0/3389276605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfc80acd60 0x7fdfc80adf00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.171 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfccaeb640 1 --2- 172.21.15.60:0/3389276605 >> v2:172.21.15.100:3300/0 conn(0x7fdfc806c390 0x7fdfc815df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.171 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfccaeb640 1 -- 172.21.15.60:0/3389276605 >> 172.21.15.60:0/3389276605 conn(0x7fdfc8071a00 msgr2=0x7fdfc8071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.171 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfccaeb640 1 -- 172.21.15.60:0/3389276605 shutdown_connections 2024-04-25T10:05:14.171 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfccaeb640 1 -- 172.21.15.60:0/3389276605 wait complete. 2024-04-25T10:05:14.171 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfccaeb640 1 Processor -- start 2024-04-25T10:05:14.171 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfccaeb640 1 -- start start 2024-04-25T10:05:14.173 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfccaeb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfc806c390 0x7fdfc8155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.173 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfccaeb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfc80acd60 0x7fdfc8157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.174 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfccaeb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc80adb20 0x7fdfc8148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.174 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfccaeb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc8160250 con 0x7fdfc806c390 2024-04-25T10:05:14.174 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfccaeb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc81600d0 con 0x7fdfc80adb20 2024-04-25T10:05:14.174 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfccaeb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc81603d0 con 0x7fdfc80acd60 2024-04-25T10:05:14.174 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfc659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfc806c390 0x7fdfc8155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.174 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfc659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfc806c390 0x7fdfc8155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38690/0 (socket says 172.21.15.60:38690) 2024-04-25T10:05:14.174 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.163+0000 7fdfc659c640 1 -- 172.21.15.60:0/1741837462 learned_addr learned my addr 172.21.15.60:0/1741837462 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:14.174 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfc6d9d640 1 --2- 172.21.15.60:0/1741837462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc80adb20 0x7fdfc8148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.174 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfc5d9b640 1 --2- 172.21.15.60:0/1741837462 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfc80acd60 0x7fdfc8157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.174 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfc659c640 1 -- 172.21.15.60:0/1741837462 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfc80acd60 msgr2=0x7fdfc8157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.174 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfc659c640 1 --2- 172.21.15.60:0/1741837462 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfc80acd60 0x7fdfc8157340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.174 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfc659c640 1 -- 172.21.15.60:0/1741837462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc80adb20 msgr2=0x7fdfc8148ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.175 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfc659c640 1 --2- 172.21.15.60:0/1741837462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc80adb20 0x7fdfc8148ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.175 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfc659c640 1 -- 172.21.15.60:0/1741837462 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc80b2a70 con 0x7fdfc806c390 2024-04-25T10:05:14.175 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfc6d9d640 1 --2- 172.21.15.60:0/1741837462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc80adb20 0x7fdfc8148ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:14.175 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfc659c640 1 --2- 172.21.15.60:0/1741837462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfc806c390 0x7fdfc8155a00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000ba50 tx=0x7fdfc000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.175 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfb6ffd640 1 -- 172.21.15.60:0/1741837462 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0019070 con 0x7fdfc806c390 2024-04-25T10:05:14.175 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfb6ffd640 1 -- 172.21.15.60:0/1741837462 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc000cc20 con 0x7fdfc806c390 2024-04-25T10:05:14.175 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfccaeb640 1 -- 172.21.15.60:0/1741837462 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc8157ab0 con 0x7fdfc806c390 2024-04-25T10:05:14.175 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfb6ffd640 1 -- 172.21.15.60:0/1741837462 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0014680 con 0x7fdfc806c390 2024-04-25T10:05:14.175 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfccaeb640 1 -- 172.21.15.60:0/1741837462 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc8158070 con 0x7fdfc806c390 2024-04-25T10:05:14.175 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfccaeb640 1 -- 172.21.15.60:0/1741837462 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc804ef30 con 0x7fdfc806c390 2024-04-25T10:05:14.176 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfb6ffd640 1 -- 172.21.15.60:0/1741837462 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdfc001c070 con 0x7fdfc806c390 2024-04-25T10:05:14.176 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfb6ffd640 1 --2- 172.21.15.60:0/1741837462 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdfa8066f60 0x7fdfa8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.176 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfb6ffd640 1 -- 172.21.15.60:0/1741837462 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdfc0084ac0 con 0x7fdfc806c390 2024-04-25T10:05:14.176 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfc5d9b640 1 --2- 172.21.15.60:0/1741837462 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdfa8066f60 0x7fdfa8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.176 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.167+0000 7fdfc5d9b640 1 --2- 172.21.15.60:0/1741837462 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdfa8066f60 0x7fdfa8069420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8167880 tx=0x7fdfbc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.179 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.171+0000 7fdfb6ffd640 1 -- 172.21.15.60:0/1741837462 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdfc00556a0 con 0x7fdfc806c390 2024-04-25T10:05:14.340 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.331+0000 7fdfccaeb640 1 -- 172.21.15.60:0/1741837462 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfc8160db0 con 0x7fdfc806c390 2024-04-25T10:05:14.340 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.335+0000 7fdfb6ffd640 1 -- 172.21.15.60:0/1741837462 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12896 (secure 0 0 0) 0x7fdfc0059740 con 0x7fdfc806c390 2024-04-25T10:05:14.341 INFO:teuthology.misc.health.smithi060.stdout: 2024-04-25T10:05:14.341 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":18,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:10.833211+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:14.343 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.335+0000 7fdfccaeb640 1 -- 172.21.15.60:0/1741837462 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdfa8066f60 msgr2=0x7fdfa8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.343 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.335+0000 7fdfccaeb640 1 --2- 172.21.15.60:0/1741837462 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdfa8066f60 0x7fdfa8069420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8167880 tx=0x7fdfbc008040 comp rx=0 tx=0).stop 2024-04-25T10:05:14.343 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.335+0000 7fdfccaeb640 1 -- 172.21.15.60:0/1741837462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfc806c390 msgr2=0x7fdfc8155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.343 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.335+0000 7fdfccaeb640 1 --2- 172.21.15.60:0/1741837462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfc806c390 0x7fdfc8155a00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000ba50 tx=0x7fdfc000bf10 comp rx=0 tx=0).stop 2024-04-25T10:05:14.343 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.335+0000 7fdfccaeb640 1 -- 172.21.15.60:0/1741837462 shutdown_connections 2024-04-25T10:05:14.343 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.335+0000 7fdfccaeb640 1 --2- 172.21.15.60:0/1741837462 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdfa8066f60 0x7fdfa8069420 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.343 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.335+0000 7fdfccaeb640 1 --2- 172.21.15.60:0/1741837462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc80adb20 0x7fdfc8148ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.343 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.335+0000 7fdfccaeb640 1 --2- 172.21.15.60:0/1741837462 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfc80acd60 0x7fdfc8157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.343 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.335+0000 7fdfccaeb640 1 --2- 172.21.15.60:0/1741837462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfc806c390 0x7fdfc8155a00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.344 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.335+0000 7fdfccaeb640 1 -- 172.21.15.60:0/1741837462 >> 172.21.15.60:0/1741837462 conn(0x7fdfc8071a00 msgr2=0x7fdfc806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.344 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.335+0000 7fdfccaeb640 1 -- 172.21.15.60:0/1741837462 shutdown_connections 2024-04-25T10:05:14.344 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T10:05:14.335+0000 7fdfccaeb640 1 -- 172.21.15.60:0/1741837462 wait complete. 2024-04-25T10:05:14.358 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T10:05:14.358 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T10:05:14.358 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:05:14.358 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:05:14.360 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:05:14.361 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:05:14.361 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:05:14.363 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:05:14.363 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:05:14.363 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:05:14.366 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:05:14.366 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:05:14.366 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:05:14.368 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:05:14.368 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T10:05:14.371 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:14.436 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:05:14.436 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T10:05:14.469 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T10:05:14.469 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T10:05:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.471+0000 7f3d30127640 1 Processor -- start 2024-04-25T10:05:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.471+0000 7f3d30127640 1 -- start start 2024-04-25T10:05:14.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.471+0000 7f3d30127640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d2815deb0 0x7f3d280b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.471+0000 7f3d30127640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3d280ce9c0 0x7f3d280d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.471+0000 7f3d30127640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d2806d200 con 0x7f3d280d0340 2024-04-25T10:05:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.471+0000 7f3d30127640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d280cf2c0 con 0x7f3d280ce9c0 2024-04-25T10:05:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.471+0000 7f3d30127640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d280cefe0 con 0x7f3d2815deb0 2024-04-25T10:05:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.471+0000 7f3d2e6c4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3d280ce9c0 0x7f3d280d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.471+0000 7f3d2d6c2640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3d280d0340 0x7f3d280ce2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48274/0 (socket says 172.21.15.60:48274) 2024-04-25T10:05:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.471+0000 7f3d2d6c2640 1 -- 172.21.15.60:0/4044636137 learned_addr learned my addr 172.21.15.60:0/4044636137 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.471+0000 7f3d2dec3640 1 --2- 172.21.15.60:0/4044636137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d2815deb0 0x7f3d280b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.471+0000 7f3d2e6c4640 1 -- 172.21.15.60:0/4044636137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d2815deb0 msgr2=0x7f3d280b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.471+0000 7f3d2e6c4640 1 --2- 172.21.15.60:0/4044636137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d2815deb0 0x7f3d280b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.471+0000 7f3d2e6c4640 1 -- 172.21.15.60:0/4044636137 >> v1:172.21.15.60:6789/0 conn(0x7f3d280d0340 legacy=0x7f3d280ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.475+0000 7f3d2e6c4640 1 -- 172.21.15.60:0/4044636137 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d280d1d70 con 0x7f3d280ce9c0 2024-04-25T10:05:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.475+0000 7f3d2cec1640 1 -- 172.21.15.60:0/4044636137 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604665569 0 0) 0x7f3d2806d200 con 0x7f3d280d0340 2024-04-25T10:05:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.475+0000 7f3d2e6c4640 1 --2- 172.21.15.60:0/4044636137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d280ce9c0 0x7f3d280d07c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3d200097c0 tx=0x7f3d2001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b365a14add02d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.475+0000 7f3d2cec1640 1 -- 172.21.15.60:0/4044636137 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d200b0070 con 0x7f3d280ce9c0 2024-04-25T10:05:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.475+0000 7f3d2cec1640 1 -- 172.21.15.60:0/4044636137 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d200ab600 con 0x7f3d280ce9c0 2024-04-25T10:05:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.475+0000 7f3d2cec1640 1 -- 172.21.15.60:0/4044636137 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d200ab9d0 con 0x7f3d280ce9c0 2024-04-25T10:05:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.475+0000 7f3d30127640 1 -- 172.21.15.60:0/4044636137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d280ce9c0 msgr2=0x7f3d280d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.475+0000 7f3d30127640 1 --2- 172.21.15.60:0/4044636137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d280ce9c0 0x7f3d280d07c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3d200097c0 tx=0x7f3d2001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.475+0000 7f3d30127640 1 -- 172.21.15.60:0/4044636137 shutdown_connections 2024-04-25T10:05:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.475+0000 7f3d30127640 1 --2- 172.21.15.60:0/4044636137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d280ce9c0 0x7f3d280d07c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.475+0000 7f3d30127640 1 --2- 172.21.15.60:0/4044636137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d2815deb0 0x7f3d280b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.475+0000 7f3d30127640 1 -- 172.21.15.60:0/4044636137 >> 172.21.15.60:0/4044636137 conn(0x7f3d2806c090 msgr2=0x7f3d2806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.475+0000 7f3d30127640 1 -- 172.21.15.60:0/4044636137 shutdown_connections 2024-04-25T10:05:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.475+0000 7f3d30127640 1 -- 172.21.15.60:0/4044636137 wait complete. 2024-04-25T10:05:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.475+0000 7f3d30127640 1 Processor -- start 2024-04-25T10:05:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.475+0000 7f3d30127640 1 -- start start 2024-04-25T10:05:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d30127640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d280ce9c0 0x7f3d280d3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d30127640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d280d0340 0x7f3d280d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d30127640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d2815deb0 0x7f3d2814f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d30127640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d280d3430 con 0x7f3d280d0340 2024-04-25T10:05:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d30127640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d280d32b0 con 0x7f3d2815deb0 2024-04-25T10:05:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d30127640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d280d35b0 con 0x7f3d280ce9c0 2024-04-25T10:05:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d2d6c2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d280d0340 0x7f3d280d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d2d6c2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d280d0340 0x7f3d280d3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38754/0 (socket says 172.21.15.60:38754) 2024-04-25T10:05:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d2d6c2640 1 -- 172.21.15.60:0/365764357 learned_addr learned my addr 172.21.15.60:0/365764357 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d2dec3640 1 --2- 172.21.15.60:0/365764357 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d280ce9c0 0x7f3d280d3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d2e6c4640 1 --2- 172.21.15.60:0/365764357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d2815deb0 0x7f3d2814f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d2d6c2640 1 -- 172.21.15.60:0/365764357 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d280ce9c0 msgr2=0x7f3d280d3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d2d6c2640 1 --2- 172.21.15.60:0/365764357 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d280ce9c0 0x7f3d280d3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d2d6c2640 1 -- 172.21.15.60:0/365764357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d2815deb0 msgr2=0x7f3d2814f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d2d6c2640 1 --2- 172.21.15.60:0/365764357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d2815deb0 0x7f3d2814f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d2d6c2640 1 -- 172.21.15.60:0/365764357 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d28148ce0 con 0x7f3d280d0340 2024-04-25T10:05:14.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d2d6c2640 1 --2- 172.21.15.60:0/365764357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d280d0340 0x7f3d280d3e70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00c520 tx=0x7f3d1c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d267fc640 1 -- 172.21.15.60:0/365764357 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c018070 con 0x7f3d280d0340 2024-04-25T10:05:14.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d267fc640 1 -- 172.21.15.60:0/365764357 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1c0074e0 con 0x7f3d280d0340 2024-04-25T10:05:14.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d30127640 1 -- 172.21.15.60:0/365764357 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d28148fc0 con 0x7f3d280d0340 2024-04-25T10:05:14.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d267fc640 1 -- 172.21.15.60:0/365764357 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c0049f0 con 0x7f3d280d0340 2024-04-25T10:05:14.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d30127640 1 -- 172.21.15.60:0/365764357 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d28149580 con 0x7f3d280d0340 2024-04-25T10:05:14.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d30127640 1 -- 172.21.15.60:0/365764357 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d28074770 con 0x7f3d280d0340 2024-04-25T10:05:14.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d267fc640 1 -- 172.21.15.60:0/365764357 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d1c01b070 con 0x7f3d280d0340 2024-04-25T10:05:14.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d267fc640 1 --2- 172.21.15.60:0/365764357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3d14066f60 0x7f3d14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d267fc640 1 -- 172.21.15.60:0/365764357 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3d1c0842b0 con 0x7f3d280d0340 2024-04-25T10:05:14.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.479+0000 7f3d2dec3640 1 --2- 172.21.15.60:0/365764357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3d14066f60 0x7f3d14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.483+0000 7f3d2dec3640 1 --2- 172.21.15.60:0/365764357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3d14066f60 0x7f3d14069420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009fd0 tx=0x7f3d18009800 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.483+0000 7f3d267fc640 1 -- 172.21.15.60:0/365764357 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d1c022080 con 0x7f3d280d0340 2024-04-25T10:05:14.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.643+0000 7f3d30127640 1 -- 172.21.15.60:0/365764357 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d280cf950 con 0x7f3d280d0340 2024-04-25T10:05:14.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.643+0000 7f3d267fc640 1 -- 172.21.15.60:0/365764357 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12896 (secure 0 0 0) 0x7f3d1c055130 con 0x7f3d280d0340 2024-04-25T10:05:14.651 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:14.651 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":18,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:10.833211+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.647+0000 7f3d30127640 1 -- 172.21.15.60:0/365764357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3d14066f60 msgr2=0x7f3d14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.647+0000 7f3d30127640 1 --2- 172.21.15.60:0/365764357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3d14066f60 0x7f3d14069420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009fd0 tx=0x7f3d18009800 comp rx=0 tx=0).stop 2024-04-25T10:05:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.647+0000 7f3d30127640 1 -- 172.21.15.60:0/365764357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d280d0340 msgr2=0x7f3d280d3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.647+0000 7f3d30127640 1 --2- 172.21.15.60:0/365764357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d280d0340 0x7f3d280d3e70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00c520 tx=0x7f3d1c00c9e0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.647+0000 7f3d30127640 1 -- 172.21.15.60:0/365764357 shutdown_connections 2024-04-25T10:05:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.647+0000 7f3d30127640 1 --2- 172.21.15.60:0/365764357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3d14066f60 0x7f3d14069420 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.647+0000 7f3d30127640 1 --2- 172.21.15.60:0/365764357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d2815deb0 0x7f3d2814f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.647+0000 7f3d30127640 1 --2- 172.21.15.60:0/365764357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d280d0340 0x7f3d280d3e70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.647+0000 7f3d30127640 1 --2- 172.21.15.60:0/365764357 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d280ce9c0 0x7f3d280d3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.647+0000 7f3d30127640 1 -- 172.21.15.60:0/365764357 >> 172.21.15.60:0/365764357 conn(0x7f3d2806c090 msgr2=0x7f3d2815caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.647+0000 7f3d30127640 1 -- 172.21.15.60:0/365764357 shutdown_connections 2024-04-25T10:05:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.647+0000 7f3d30127640 1 -- 172.21.15.60:0/365764357 wait complete. 2024-04-25T10:05:14.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:14.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:14.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.783+0000 7f8435b80640 1 Processor -- start 2024-04-25T10:05:14.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f8435b80640 1 -- start start 2024-04-25T10:05:14.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f8435b80640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f84300cf8a0 0x7f843015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f8435b80640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84300cd890 con 0x7f84300cd3d0 2024-04-25T10:05:14.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f8435b80640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84300cdb70 con 0x7f843015dd30 2024-04-25T10:05:14.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f8435b80640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84300ce4a0 con 0x7f84300cf8a0 2024-04-25T10:05:14.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f84300cd3d0 0x7f843015be20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48286/0 (socket says 172.21.15.60:48286) 2024-04-25T10:05:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842ffff640 1 -- 172.21.15.60:0/988047438 learned_addr learned my addr 172.21.15.60:0/988047438 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842effd640 1 --2- 172.21.15.60:0/988047438 >> v2:172.21.15.100:3300/0 conn(0x7f84300cf8a0 0x7f843015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842e7fc640 1 -- 172.21.15.60:0/988047438 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607985225 0 0) 0x7f84300cd890 con 0x7f84300cd3d0 2024-04-25T10:05:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842e7fc640 1 -- 172.21.15.60:0/988047438 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8418003530 con 0x7f84300cd3d0 2024-04-25T10:05:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842e7fc640 1 -- 172.21.15.60:0/988047438 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3754856134 0 0) 0x7f8418003530 con 0x7f84300cd3d0 2024-04-25T10:05:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842e7fc640 1 -- 172.21.15.60:0/988047438 >> v2:172.21.15.100:3300/0 conn(0x7f84300cf8a0 msgr2=0x7f843015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842e7fc640 1 --2- 172.21.15.60:0/988047438 >> v2:172.21.15.100:3300/0 conn(0x7f84300cf8a0 0x7f843015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842e7fc640 1 -- 172.21.15.60:0/988047438 >> v1:172.21.15.100:6789/0 conn(0x7f843015dd30 legacy=0x7f84300cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842e7fc640 1 -- 172.21.15.60:0/988047438 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84300cde50 con 0x7f84300cd3d0 2024-04-25T10:05:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842e7fc640 1 -- 172.21.15.60:0/988047438 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f8428001a00 con 0x7f84300cd3d0 2024-04-25T10:05:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842e7fc640 1 -- 172.21.15.60:0/988047438 >> v1:172.21.15.60:6789/0 conn(0x7f84300cd3d0 legacy=0x7f843015be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842e7fc640 1 --2- 172.21.15.60:0/988047438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8418003a10 0x7f8418003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842e7fc640 1 --2- 172.21.15.60:0/988047438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8418004810 0x7f8418004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842e7fc640 1 --2- 172.21.15.60:0/988047438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8418005220 0x7f841800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842e7fc640 1 -- 172.21.15.60:0/988047438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84300ce4a0 con 0x7f8418004810 2024-04-25T10:05:14.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842e7fc640 1 -- 172.21.15.60:0/988047438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841800c170 con 0x7f8418005220 2024-04-25T10:05:14.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842e7fc640 1 -- 172.21.15.60:0/988047438 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841800c370 con 0x7f8418003a10 2024-04-25T10:05:14.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842effd640 1 --2- 172.21.15.60:0/988047438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8418004810 0x7f8418004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842f7fe640 1 --2- 172.21.15.60:0/988047438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8418003a10 0x7f8418003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842effd640 1 -- 172.21.15.60:0/988047438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8418003a10 msgr2=0x7f8418003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842effd640 1 --2- 172.21.15.60:0/988047438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8418003a10 0x7f8418003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842effd640 1 -- 172.21.15.60:0/988047438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8418005220 msgr2=0x7f841800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842effd640 1 --2- 172.21.15.60:0/988047438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8418005220 0x7f841800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842effd640 1 -- 172.21.15.60:0/988047438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8418003530 con 0x7f8418004810 2024-04-25T10:05:14.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.787+0000 7f842ffff640 1 -- 172.21.15.60:0/988047438 reap_dead start 2024-04-25T10:05:14.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f842effd640 1 --2- 172.21.15.60:0/988047438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8418004810 0x7f8418004c90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8424009ab0 tx=0x7f842401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d5b5166d7247fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f842e7fc640 1 -- 172.21.15.60:0/988047438 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84240b2070 con 0x7f8418004810 2024-04-25T10:05:14.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f842e7fc640 1 -- 172.21.15.60:0/988047438 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84240a5630 con 0x7f8418004810 2024-04-25T10:05:14.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f842e7fc640 1 -- 172.21.15.60:0/988047438 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84240096b0 con 0x7f8418004810 2024-04-25T10:05:14.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f8435b80640 1 -- 172.21.15.60:0/988047438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8418004810 msgr2=0x7f8418004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f8435b80640 1 --2- 172.21.15.60:0/988047438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8418004810 0x7f8418004c90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8424009ab0 tx=0x7f842401ca40 comp rx=0 tx=0).stop 2024-04-25T10:05:14.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f8435b80640 1 -- 172.21.15.60:0/988047438 shutdown_connections 2024-04-25T10:05:14.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f8435b80640 1 --2- 172.21.15.60:0/988047438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8418004810 0x7f8418004c90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f8435b80640 1 -- 172.21.15.60:0/988047438 >> 172.21.15.60:0/988047438 conn(0x7f84300670b0 msgr2=0x7f843006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f8435b80640 1 -- 172.21.15.60:0/988047438 shutdown_connections 2024-04-25T10:05:14.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f8435b80640 1 -- 172.21.15.60:0/988047438 wait complete. 2024-04-25T10:05:14.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f8435b80640 1 Processor -- start 2024-04-25T10:05:14.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f8435b80640 1 -- start start 2024-04-25T10:05:14.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f8435b80640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8418004810 0x7f84300add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f8435b80640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843014d4e0 0x7f84301f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f8435b80640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f843015c620 0x7f843015caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f8435b80640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843006dbe0 con 0x7f843015c620 2024-04-25T10:05:14.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f8435b80640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843006da60 con 0x7f8418004810 2024-04-25T10:05:14.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f8435b80640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843006dd60 con 0x7f843014d4e0 2024-04-25T10:05:14.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f842ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f843015c620 0x7f843015caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f842ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f843015c620 0x7f843015caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38778/0 (socket says 172.21.15.60:38778) 2024-04-25T10:05:14.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.791+0000 7f842ffff640 1 -- 172.21.15.60:0/2625849539 learned_addr learned my addr 172.21.15.60:0/2625849539 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:14.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f842f7fe640 1 --2- 172.21.15.60:0/2625849539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8418004810 0x7f84300add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f842effd640 1 --2- 172.21.15.60:0/2625849539 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843014d4e0 0x7f84301f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f842ffff640 1 -- 172.21.15.60:0/2625849539 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843014d4e0 msgr2=0x7f84301f8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f842ffff640 1 --2- 172.21.15.60:0/2625849539 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843014d4e0 0x7f84301f8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f842ffff640 1 -- 172.21.15.60:0/2625849539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8418004810 msgr2=0x7f84300add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f842ffff640 1 --2- 172.21.15.60:0/2625849539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8418004810 0x7f84300add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f842ffff640 1 -- 172.21.15.60:0/2625849539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84301f98c0 con 0x7f843015c620 2024-04-25T10:05:14.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f842ffff640 1 --2- 172.21.15.60:0/2625849539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f843015c620 0x7f843015caa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8430072500 tx=0x7f842800cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f8417fff640 1 -- 172.21.15.60:0/2625849539 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8428007070 con 0x7f843015c620 2024-04-25T10:05:14.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f8417fff640 1 -- 172.21.15.60:0/2625849539 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8428007210 con 0x7f843015c620 2024-04-25T10:05:14.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f8435b80640 1 -- 172.21.15.60:0/2625849539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84301f9ba0 con 0x7f843015c620 2024-04-25T10:05:14.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f8417fff640 1 -- 172.21.15.60:0/2625849539 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8428007600 con 0x7f843015c620 2024-04-25T10:05:14.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f8435b80640 1 -- 172.21.15.60:0/2625849539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84300ae270 con 0x7f843015c620 2024-04-25T10:05:14.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f8435b80640 1 -- 172.21.15.60:0/2625849539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8430074770 con 0x7f843015c620 2024-04-25T10:05:14.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f8417fff640 1 -- 172.21.15.60:0/2625849539 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8428011070 con 0x7f843015c620 2024-04-25T10:05:14.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f8417fff640 1 --2- 172.21.15.60:0/2625849539 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f841c066f90 0x7f841c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f8417fff640 1 -- 172.21.15.60:0/2625849539 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8428084580 con 0x7f843015c620 2024-04-25T10:05:14.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f842f7fe640 1 --2- 172.21.15.60:0/2625849539 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f841c066f90 0x7f841c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.795+0000 7f842f7fe640 1 --2- 172.21.15.60:0/2625849539 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f841c066f90 0x7f841c069450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f84200016e0 tx=0x7f8420009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.799+0000 7f8417fff640 1 -- 172.21.15.60:0/2625849539 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8428019060 con 0x7f843015c620 2024-04-25T10:05:14.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.959+0000 7f8435b80640 1 -- 172.21.15.60:0/2625849539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8430071a40 con 0x7f843015c620 2024-04-25T10:05:14.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.959+0000 7f8417fff640 1 -- 172.21.15.60:0/2625849539 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f842805b030 con 0x7f843015c620 2024-04-25T10:05:14.968 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:05:14.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.963+0000 7f8435b80640 1 -- 172.21.15.60:0/2625849539 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f841c066f90 msgr2=0x7f841c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.963+0000 7f8435b80640 1 --2- 172.21.15.60:0/2625849539 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f841c066f90 0x7f841c069450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f84200016e0 tx=0x7f8420009040 comp rx=0 tx=0).stop 2024-04-25T10:05:14.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.963+0000 7f8435b80640 1 -- 172.21.15.60:0/2625849539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f843015c620 msgr2=0x7f843015caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.963+0000 7f8435b80640 1 --2- 172.21.15.60:0/2625849539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f843015c620 0x7f843015caa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8430072500 tx=0x7f842800cad0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.963+0000 7f8435b80640 1 -- 172.21.15.60:0/2625849539 shutdown_connections 2024-04-25T10:05:14.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.963+0000 7f8435b80640 1 --2- 172.21.15.60:0/2625849539 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f841c066f90 0x7f841c069450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.963+0000 7f8435b80640 1 --2- 172.21.15.60:0/2625849539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f843015c620 0x7f843015caa0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.963+0000 7f8435b80640 1 --2- 172.21.15.60:0/2625849539 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843014d4e0 0x7f84301f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.963+0000 7f8435b80640 1 --2- 172.21.15.60:0/2625849539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8418004810 0x7f84300add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.963+0000 7f8435b80640 1 -- 172.21.15.60:0/2625849539 >> 172.21.15.60:0/2625849539 conn(0x7f84300670b0 msgr2=0x7f84300b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.963+0000 7f8435b80640 1 -- 172.21.15.60:0/2625849539 shutdown_connections 2024-04-25T10:05:14.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:14.963+0000 7f8435b80640 1 -- 172.21.15.60:0/2625849539 wait complete. 2024-04-25T10:05:14.987 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:15.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71ede4f640 1 Processor -- start 2024-04-25T10:05:15.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71ede4f640 1 -- start start 2024-04-25T10:05:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71ede4f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f71e81549f0 0x7f71e80ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71ede4f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71e80ad770 0x7f71e806bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71ede4f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e80ae4b0 con 0x7f71e806c160 2024-04-25T10:05:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71ede4f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e80adc10 con 0x7f71e81549f0 2024-04-25T10:05:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71ede4f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80adef0 con 0x7f71e80ad770 2024-04-25T10:05:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f71e806c160 0x7f71e8161550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48292/0 (socket says 172.21.15.60:48292) 2024-04-25T10:05:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e7fff640 1 -- 172.21.15.60:0/1545311232 learned_addr learned my addr 172.21.15.60:0/1545311232 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e6ffd640 1 --2- 172.21.15.60:0/1545311232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71e80ad770 0x7f71e806bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e77fe640 1 --2- 172.21.15.60:0/1545311232 >> v2:172.21.15.100:3300/0 conn(0x7f71e81549f0 0x7f71e80ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e67fc640 1 -- 172.21.15.60:0/1545311232 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1337677533 0 0) 0x7f71e80ae4b0 con 0x7f71e806c160 2024-04-25T10:05:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e67fc640 1 -- 172.21.15.60:0/1545311232 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71d0003530 con 0x7f71e806c160 2024-04-25T10:05:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e67fc640 1 -- 172.21.15.60:0/1545311232 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2256695841 0 0) 0x7f71d0003530 con 0x7f71e806c160 2024-04-25T10:05:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e67fc640 1 -- 172.21.15.60:0/1545311232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71e80ad770 msgr2=0x7f71e806bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e67fc640 1 --2- 172.21.15.60:0/1545311232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71e80ad770 0x7f71e806bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e67fc640 1 -- 172.21.15.60:0/1545311232 >> v2:172.21.15.100:3300/0 conn(0x7f71e81549f0 msgr2=0x7f71e80ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e67fc640 1 --2- 172.21.15.60:0/1545311232 >> v2:172.21.15.100:3300/0 conn(0x7f71e81549f0 0x7f71e80ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e67fc640 1 -- 172.21.15.60:0/1545311232 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e80ae1d0 con 0x7f71e806c160 2024-04-25T10:05:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e67fc640 1 -- 172.21.15.60:0/1545311232 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f71e0001a00 con 0x7f71e806c160 2024-04-25T10:05:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e67fc640 1 -- 172.21.15.60:0/1545311232 >> v1:172.21.15.60:6789/0 conn(0x7f71e806c160 legacy=0x7f71e8161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e67fc640 1 --2- 172.21.15.60:0/1545311232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71d00039c0 0x7f71d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e67fc640 1 --2- 172.21.15.60:0/1545311232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71e80ad770 0x7f71d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e67fc640 1 --2- 172.21.15.60:0/1545311232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71d0004d50 0x7f71d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e67fc640 1 -- 172.21.15.60:0/1545311232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80adc10 con 0x7f71d00039c0 2024-04-25T10:05:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e67fc640 1 -- 172.21.15.60:0/1545311232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80adef0 con 0x7f71d0004d50 2024-04-25T10:05:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e67fc640 1 -- 172.21.15.60:0/1545311232 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d000b9d0 con 0x7f71e80ad770 2024-04-25T10:05:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e77fe640 1 --2- 172.21.15.60:0/1545311232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71d00039c0 0x7f71d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e6ffd640 1 --2- 172.21.15.60:0/1545311232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71e80ad770 0x7f71d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e7fff640 1 --2- 172.21.15.60:0/1545311232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71d0004d50 0x7f71d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e77fe640 1 -- 172.21.15.60:0/1545311232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71e80ad770 msgr2=0x7f71d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e77fe640 1 --2- 172.21.15.60:0/1545311232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71e80ad770 0x7f71d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e77fe640 1 -- 172.21.15.60:0/1545311232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71d0004d50 msgr2=0x7f71d00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e77fe640 1 --2- 172.21.15.60:0/1545311232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71d0004d50 0x7f71d00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e77fe640 1 -- 172.21.15.60:0/1545311232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d0003530 con 0x7f71d00039c0 2024-04-25T10:05:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.099+0000 7f71e7fff640 1 --2- 172.21.15.60:0/1545311232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71d0004d50 0x7f71d00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:15.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.103+0000 7f71e77fe640 1 --2- 172.21.15.60:0/1545311232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71d00039c0 0x7f71d0003de0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009ae0 tx=0x7f71d801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8963db61d05b952 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.103+0000 7f71e67fc640 1 -- 172.21.15.60:0/1545311232 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d80b2070 con 0x7f71d00039c0 2024-04-25T10:05:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.103+0000 7f71e67fc640 1 -- 172.21.15.60:0/1545311232 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d80a5590 con 0x7f71d00039c0 2024-04-25T10:05:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.103+0000 7f71e67fc640 1 -- 172.21.15.60:0/1545311232 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d80096b0 con 0x7f71d00039c0 2024-04-25T10:05:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.103+0000 7f71ede4f640 1 -- 172.21.15.60:0/1545311232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71d00039c0 msgr2=0x7f71d0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.103+0000 7f71ede4f640 1 --2- 172.21.15.60:0/1545311232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71d00039c0 0x7f71d0003de0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009ae0 tx=0x7f71d801cab0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.103+0000 7f71e7fff640 1 -- 172.21.15.60:0/1545311232 reap_dead start 2024-04-25T10:05:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.103+0000 7f71ede4f640 1 -- 172.21.15.60:0/1545311232 shutdown_connections 2024-04-25T10:05:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.103+0000 7f71ede4f640 1 -- 172.21.15.60:0/1545311232 >> 172.21.15.60:0/1545311232 conn(0x7f71e8071b90 msgr2=0x7f71e8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.103+0000 7f71ede4f640 1 -- 172.21.15.60:0/1545311232 shutdown_connections 2024-04-25T10:05:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.103+0000 7f71ede4f640 1 -- 172.21.15.60:0/1545311232 wait complete. 2024-04-25T10:05:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.103+0000 7f71ede4f640 1 Processor -- start 2024-04-25T10:05:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.103+0000 7f71ede4f640 1 -- start start 2024-04-25T10:05:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71ede4f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71d00039c0 0x7f71e806f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71ede4f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71e806e680 0x7f71e814d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71ede4f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71e80b0780 0x7f71e80b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71ede4f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e8168a60 con 0x7f71d00039c0 2024-04-25T10:05:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71ede4f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e81688e0 con 0x7f71e80b0780 2024-04-25T10:05:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71ede4f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e8168be0 con 0x7f71e806e680 2024-04-25T10:05:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71e77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71d00039c0 0x7f71e806f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71e77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71d00039c0 0x7f71e806f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38796/0 (socket says 172.21.15.60:38796) 2024-04-25T10:05:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71e77fe640 1 -- 172.21.15.60:0/471867822 learned_addr learned my addr 172.21.15.60:0/471867822 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71e6ffd640 1 --2- 172.21.15.60:0/471867822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71e806e680 0x7f71e814d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71e7fff640 1 --2- 172.21.15.60:0/471867822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71e80b0780 0x7f71e80b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71e77fe640 1 -- 172.21.15.60:0/471867822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71e806e680 msgr2=0x7f71e814d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71e77fe640 1 --2- 172.21.15.60:0/471867822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71e806e680 0x7f71e814d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71e77fe640 1 -- 172.21.15.60:0/471867822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71e80b0780 msgr2=0x7f71e80b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71e77fe640 1 --2- 172.21.15.60:0/471867822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71e80b0780 0x7f71e80b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71e77fe640 1 -- 172.21.15.60:0/471867822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e80b1140 con 0x7f71d00039c0 2024-04-25T10:05:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71e77fe640 1 --2- 172.21.15.60:0/471867822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71d00039c0 0x7f71e806f4c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f71d801c8e0 tx=0x7f71d0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71cffff640 1 -- 172.21.15.60:0/471867822 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d80b2040 con 0x7f71d00039c0 2024-04-25T10:05:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71cffff640 1 -- 172.21.15.60:0/471867822 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d800f000 con 0x7f71d00039c0 2024-04-25T10:05:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71ede4f640 1 -- 172.21.15.60:0/471867822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e814ddc0 con 0x7f71d00039c0 2024-04-25T10:05:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71cffff640 1 -- 172.21.15.60:0/471867822 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8004020 con 0x7f71d00039c0 2024-04-25T10:05:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71ede4f640 1 -- 172.21.15.60:0/471867822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e806dba0 con 0x7f71d00039c0 2024-04-25T10:05:15.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71ede4f640 1 -- 172.21.15.60:0/471867822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e8074770 con 0x7f71d00039c0 2024-04-25T10:05:15.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71cffff640 1 -- 172.21.15.60:0/471867822 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f71d80c0020 con 0x7f71d00039c0 2024-04-25T10:05:15.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71cffff640 1 --2- 172.21.15.60:0/471867822 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f71d4066f90 0x7f71d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71cffff640 1 -- 172.21.15.60:0/471867822 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f71d811ebe0 con 0x7f71d00039c0 2024-04-25T10:05:15.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.107+0000 7f71e6ffd640 1 --2- 172.21.15.60:0/471867822 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f71d4066f90 0x7f71d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.111+0000 7f71e6ffd640 1 --2- 172.21.15.60:0/471867822 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f71d4066f90 0x7f71d4069450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f71e81fa3c0 tx=0x7f71dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.111+0000 7f71cffff640 1 -- 172.21.15.60:0/471867822 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f71d80167c0 con 0x7f71d00039c0 2024-04-25T10:05:15.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.307+0000 7f71ede4f640 1 -- 172.21.15.60:0/471867822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71e8072780 con 0x7f71d00039c0 2024-04-25T10:05:15.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.307+0000 7f71cffff640 1 -- 172.21.15.60:0/471867822 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f71d80eff40 con 0x7f71d00039c0 2024-04-25T10:05:15.314 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:15.314 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3591836867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3591836867},{"type":"v1","addr":"172.21.15.60:6835","nonce":3591836867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:05:15.315 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 2 2024-04-25T10:05:15.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.311+0000 7f71ede4f640 1 -- 172.21.15.60:0/471867822 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f71d4066f90 msgr2=0x7f71d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.311+0000 7f71ede4f640 1 --2- 172.21.15.60:0/471867822 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f71d4066f90 0x7f71d4069450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f71e81fa3c0 tx=0x7f71dc008040 comp rx=0 tx=0).stop 2024-04-25T10:05:15.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.311+0000 7f71ede4f640 1 -- 172.21.15.60:0/471867822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71d00039c0 msgr2=0x7f71e806f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.311+0000 7f71ede4f640 1 --2- 172.21.15.60:0/471867822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71d00039c0 0x7f71e806f4c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f71d801c8e0 tx=0x7f71d0005840 comp rx=0 tx=0).stop 2024-04-25T10:05:15.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.311+0000 7f71ede4f640 1 -- 172.21.15.60:0/471867822 shutdown_connections 2024-04-25T10:05:15.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.311+0000 7f71ede4f640 1 --2- 172.21.15.60:0/471867822 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f71d4066f90 0x7f71d4069450 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.311+0000 7f71ede4f640 1 --2- 172.21.15.60:0/471867822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71e80b0780 0x7f71e80b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.311+0000 7f71ede4f640 1 --2- 172.21.15.60:0/471867822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71e806e680 0x7f71e814d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.311+0000 7f71ede4f640 1 --2- 172.21.15.60:0/471867822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71d00039c0 0x7f71e806f4c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.311+0000 7f71ede4f640 1 -- 172.21.15.60:0/471867822 >> 172.21.15.60:0/471867822 conn(0x7f71e8071b90 msgr2=0x7f71e80afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.311+0000 7f71ede4f640 1 -- 172.21.15.60:0/471867822 shutdown_connections 2024-04-25T10:05:15.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:15.311+0000 7f71ede4f640 1 -- 172.21.15.60:0/471867822 wait complete. 2024-04-25T10:05:16.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:16.160 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:17.333 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:17.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.463+0000 7f39a5688640 1 Processor -- start 2024-04-25T10:05:17.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.463+0000 7f39a5688640 1 -- start start 2024-04-25T10:05:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.463+0000 7f39a5688640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39a00d0a50 0x7f39a00b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.463+0000 7f39a5688640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a00ce2b0 0x7f39a00ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.463+0000 7f39a5688640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a006d200 con 0x7f39a015deb0 2024-04-25T10:05:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.463+0000 7f39a5688640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a00cf2c0 con 0x7f39a00d0a50 2024-04-25T10:05:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.463+0000 7f39a5688640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00cefe0 con 0x7f39a00ce2b0 2024-04-25T10:05:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.463+0000 7f3997fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39a00d0a50 0x7f39a00b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.463+0000 7f3997fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39a00d0a50 0x7f39a00b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38806/0 (socket says 172.21.15.60:38806) 2024-04-25T10:05:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.463+0000 7f3997fff640 1 -- 172.21.15.60:0/15237765 learned_addr learned my addr 172.21.15.60:0/15237765 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.463+0000 7f399f7fe640 1 --2- 172.21.15.60:0/15237765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a00ce2b0 0x7f39a00ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.463+0000 7f3997fff640 1 -- 172.21.15.60:0/15237765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a00ce2b0 msgr2=0x7f39a00ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.463+0000 7f3997fff640 1 --2- 172.21.15.60:0/15237765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a00ce2b0 0x7f39a00ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.463+0000 7f3997fff640 1 -- 172.21.15.60:0/15237765 >> v1:172.21.15.100:6789/0 conn(0x7f39a015deb0 legacy=0x7f39a00d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.463+0000 7f3997fff640 1 -- 172.21.15.60:0/15237765 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a00d1d70 con 0x7f39a00d0a50 2024-04-25T10:05:17.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.467+0000 7f3997fff640 1 --2- 172.21.15.60:0/15237765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00d0a50 0x7f39a00b6c10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3988009870 tx=0x7f398801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e58a628c2a7740b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.467+0000 7f399e7fc640 1 -- 172.21.15.60:0/15237765 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39880ab650 con 0x7f39a00d0a50 2024-04-25T10:05:17.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.467+0000 7f399e7fc640 1 -- 172.21.15.60:0/15237765 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39880b0070 con 0x7f39a00d0a50 2024-04-25T10:05:17.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.467+0000 7f399e7fc640 1 -- 172.21.15.60:0/15237765 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39880aba10 con 0x7f39a00d0a50 2024-04-25T10:05:17.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.467+0000 7f39a5688640 1 -- 172.21.15.60:0/15237765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00d0a50 msgr2=0x7f39a00b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.467+0000 7f39a5688640 1 --2- 172.21.15.60:0/15237765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00d0a50 0x7f39a00b6c10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3988009870 tx=0x7f398801c560 comp rx=0 tx=0).stop 2024-04-25T10:05:17.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.467+0000 7f39a5688640 1 -- 172.21.15.60:0/15237765 shutdown_connections 2024-04-25T10:05:17.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.467+0000 7f39a5688640 1 --2- 172.21.15.60:0/15237765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00d0a50 0x7f39a00b6c10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.467+0000 7f39a5688640 1 --2- 172.21.15.60:0/15237765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a00ce2b0 0x7f39a00ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.467+0000 7f39a5688640 1 -- 172.21.15.60:0/15237765 >> 172.21.15.60:0/15237765 conn(0x7f39a006c090 msgr2=0x7f39a006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.467+0000 7f39a5688640 1 -- 172.21.15.60:0/15237765 shutdown_connections 2024-04-25T10:05:17.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.467+0000 7f39a5688640 1 -- 172.21.15.60:0/15237765 wait complete. 2024-04-25T10:05:17.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.467+0000 7f39a5688640 1 Processor -- start 2024-04-25T10:05:17.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.467+0000 7f39a5688640 1 -- start start 2024-04-25T10:05:17.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f39a5688640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00ce2b0 0x7f39a014c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f39a5688640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a00d0a50 0x7f39a00b8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f39a5688640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a015deb0 0x7f39a014e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f39a5688640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00d31e0 con 0x7f39a00ce2b0 2024-04-25T10:05:17.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f39a5688640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00d3060 con 0x7f39a015deb0 2024-04-25T10:05:17.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f39a5688640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00d3360 con 0x7f39a00d0a50 2024-04-25T10:05:17.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f399effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00ce2b0 0x7f39a014c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f399effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00ce2b0 0x7f39a014c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38808/0 (socket says 172.21.15.60:38808) 2024-04-25T10:05:17.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f399effd640 1 -- 172.21.15.60:0/1238565233 learned_addr learned my addr 172.21.15.60:0/1238565233 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:17.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f3997fff640 1 --2- 172.21.15.60:0/1238565233 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a00d0a50 0x7f39a00b8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f399f7fe640 1 --2- 172.21.15.60:0/1238565233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a015deb0 0x7f39a014e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f399effd640 1 -- 172.21.15.60:0/1238565233 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a00d0a50 msgr2=0x7f39a00b8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f399effd640 1 --2- 172.21.15.60:0/1238565233 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a00d0a50 0x7f39a00b8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f399effd640 1 -- 172.21.15.60:0/1238565233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a015deb0 msgr2=0x7f39a014e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f399effd640 1 --2- 172.21.15.60:0/1238565233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a015deb0 0x7f39a014e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f399effd640 1 -- 172.21.15.60:0/1238565233 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a00cea90 con 0x7f39a00ce2b0 2024-04-25T10:05:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f399f7fe640 1 --2- 172.21.15.60:0/1238565233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a015deb0 0x7f39a014e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f399effd640 1 --2- 172.21.15.60:0/1238565233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00ce2b0 0x7f39a014c7a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f399000c3f0 tx=0x7f399000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f39977fe640 1 -- 172.21.15.60:0/1238565233 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990017070 con 0x7f39a00ce2b0 2024-04-25T10:05:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f39977fe640 1 -- 172.21.15.60:0/1238565233 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39900052d0 con 0x7f39a00ce2b0 2024-04-25T10:05:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f39977fe640 1 -- 172.21.15.60:0/1238565233 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990003840 con 0x7f39a00ce2b0 2024-04-25T10:05:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f39a5688640 1 -- 172.21.15.60:0/1238565233 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a00b8e90 con 0x7f39a00ce2b0 2024-04-25T10:05:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f39a5688640 1 -- 172.21.15.60:0/1238565233 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a014bfa0 con 0x7f39a00ce2b0 2024-04-25T10:05:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f39a5688640 1 -- 172.21.15.60:0/1238565233 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a01612c0 con 0x7f39a00ce2b0 2024-04-25T10:05:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f39977fe640 1 -- 172.21.15.60:0/1238565233 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f399001a070 con 0x7f39a00ce2b0 2024-04-25T10:05:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f39977fe640 1 --2- 172.21.15.60:0/1238565233 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3980066f60 0x7f3980069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f39977fe640 1 -- 172.21.15.60:0/1238565233 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3990084060 con 0x7f39a00ce2b0 2024-04-25T10:05:17.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.471+0000 7f3997fff640 1 --2- 172.21.15.60:0/1238565233 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3980066f60 0x7f3980069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.475+0000 7f3997fff640 1 --2- 172.21.15.60:0/1238565233 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3980066f60 0x7f3980069420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f39880af000 tx=0x7f39880a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.475+0000 7f39977fe640 1 -- 172.21.15.60:0/1238565233 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f39a00b8e90 con 0x7f39a00ce2b0 2024-04-25T10:05:17.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.675+0000 7f39a5688640 1 -- 172.21.15.60:0/1238565233 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39a014cce0 con 0x7f39a00ce2b0 2024-04-25T10:05:17.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.675+0000 7f39977fe640 1 -- 172.21.15.60:0/1238565233 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7f3990005480 con 0x7f39a00ce2b0 2024-04-25T10:05:17.682 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:17.682 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/622088871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":622088871},{"type":"v1","addr":"172.21.15.100:6836","nonce":622088871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3988204202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3988204202},{"type":"v1","addr":"172.21.15.60:6833","nonce":3988204202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3591836867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3591836867},{"type":"v1","addr":"172.21.15.60:6835","nonce":3591836867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4254,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/690070228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":690070228},{"type":"v1","addr":"172.21.15.100:6837","nonce":690070228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T10:05:17.683 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 3 2024-04-25T10:05:17.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.679+0000 7f39a5688640 1 -- 172.21.15.60:0/1238565233 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3980066f60 msgr2=0x7f3980069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.679+0000 7f39a5688640 1 --2- 172.21.15.60:0/1238565233 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3980066f60 0x7f3980069420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f39880af000 tx=0x7f39880a7000 comp rx=0 tx=0).stop 2024-04-25T10:05:17.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.679+0000 7f39a5688640 1 -- 172.21.15.60:0/1238565233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00ce2b0 msgr2=0x7f39a014c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.679+0000 7f39a5688640 1 --2- 172.21.15.60:0/1238565233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00ce2b0 0x7f39a014c7a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f399000c3f0 tx=0x7f399000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.679+0000 7f39a5688640 1 -- 172.21.15.60:0/1238565233 shutdown_connections 2024-04-25T10:05:17.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.679+0000 7f39a5688640 1 --2- 172.21.15.60:0/1238565233 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3980066f60 0x7f3980069420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.679+0000 7f39a5688640 1 --2- 172.21.15.60:0/1238565233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a015deb0 0x7f39a014e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.679+0000 7f39a5688640 1 --2- 172.21.15.60:0/1238565233 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a00d0a50 0x7f39a00b8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.679+0000 7f39a5688640 1 --2- 172.21.15.60:0/1238565233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00ce2b0 0x7f39a014c7a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.679+0000 7f39a5688640 1 -- 172.21.15.60:0/1238565233 >> 172.21.15.60:0/1238565233 conn(0x7f39a006c090 msgr2=0x7f39a00c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.679+0000 7f39a5688640 1 -- 172.21.15.60:0/1238565233 shutdown_connections 2024-04-25T10:05:17.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.679+0000 7f39a5688640 1 -- 172.21.15.60:0/1238565233 wait complete. 2024-04-25T10:05:17.703 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:17.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.811+0000 7fec31844640 1 Processor -- start 2024-04-25T10:05:17.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.811+0000 7fec31844640 1 -- start start 2024-04-25T10:05:17.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec31844640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fec2c15a7a0 0x7fec2c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec31844640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2c0b37e0 0x7fec2c0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec31844640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec2c06f130 con 0x7fec2c06ec30 2024-04-25T10:05:17.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec31844640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2c0cdaa0 con 0x7fec2c15a7a0 2024-04-25T10:05:17.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec31844640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c06d2f0 con 0x7fec2c0b37e0 2024-04-25T10:05:17.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec2a7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fec2c06ec30 0x7fec2c071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48308/0 (socket says 172.21.15.60:48308) 2024-04-25T10:05:17.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec2a7fc640 1 -- 172.21.15.60:0/908614487 learned_addr learned my addr 172.21.15.60:0/908614487 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:17.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec2affd640 1 --2- 172.21.15.60:0/908614487 >> v2:172.21.15.100:3300/0 conn(0x7fec2c15a7a0 0x7fec2c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec2b7fe640 1 --2- 172.21.15.60:0/908614487 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2c0b37e0 0x7fec2c0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec29ffb640 1 -- 172.21.15.60:0/908614487 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 358854804 0 0) 0x7fec2c06f130 con 0x7fec2c06ec30 2024-04-25T10:05:17.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec29ffb640 1 -- 172.21.15.60:0/908614487 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec14003530 con 0x7fec2c06ec30 2024-04-25T10:05:17.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec29ffb640 1 -- 172.21.15.60:0/908614487 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3518477588 0 0) 0x7fec14003530 con 0x7fec2c06ec30 2024-04-25T10:05:17.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec29ffb640 1 -- 172.21.15.60:0/908614487 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2c0b37e0 msgr2=0x7fec2c0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec29ffb640 1 --2- 172.21.15.60:0/908614487 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2c0b37e0 0x7fec2c0b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec29ffb640 1 -- 172.21.15.60:0/908614487 >> v2:172.21.15.100:3300/0 conn(0x7fec2c15a7a0 msgr2=0x7fec2c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec29ffb640 1 --2- 172.21.15.60:0/908614487 >> v2:172.21.15.100:3300/0 conn(0x7fec2c15a7a0 0x7fec2c0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec29ffb640 1 -- 172.21.15.60:0/908614487 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c06f410 con 0x7fec2c06ec30 2024-04-25T10:05:17.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec29ffb640 1 -- 172.21.15.60:0/908614487 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fec20001680 con 0x7fec2c06ec30 2024-04-25T10:05:17.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec29ffb640 1 -- 172.21.15.60:0/908614487 >> v1:172.21.15.60:6789/0 conn(0x7fec2c06ec30 legacy=0x7fec2c071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec29ffb640 1 --2- 172.21.15.60:0/908614487 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2c0b37e0 0x7fec140039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec29ffb640 1 --2- 172.21.15.60:0/908614487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec14003f40 0x7fec14004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec29ffb640 1 --2- 172.21.15.60:0/908614487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14004d50 0x7fec140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec29ffb640 1 -- 172.21.15.60:0/908614487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c06d2f0 con 0x7fec14003f40 2024-04-25T10:05:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec29ffb640 1 -- 172.21.15.60:0/908614487 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1400b9d0 con 0x7fec14004d50 2024-04-25T10:05:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec29ffb640 1 -- 172.21.15.60:0/908614487 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1400bb50 con 0x7fec2c0b37e0 2024-04-25T10:05:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec2a7fc640 1 --2- 172.21.15.60:0/908614487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec14003f40 0x7fec14004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec2affd640 1 --2- 172.21.15.60:0/908614487 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2c0b37e0 0x7fec140039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec2b7fe640 1 --2- 172.21.15.60:0/908614487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14004d50 0x7fec140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec2a7fc640 1 -- 172.21.15.60:0/908614487 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2c0b37e0 msgr2=0x7fec140039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec2a7fc640 1 --2- 172.21.15.60:0/908614487 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2c0b37e0 0x7fec140039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec2a7fc640 1 -- 172.21.15.60:0/908614487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14004d50 msgr2=0x7fec140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec2a7fc640 1 --2- 172.21.15.60:0/908614487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14004d50 0x7fec140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec2a7fc640 1 -- 172.21.15.60:0/908614487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec14003530 con 0x7fec14003f40 2024-04-25T10:05:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.815+0000 7fec2affd640 1 --2- 172.21.15.60:0/908614487 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2c0b37e0 0x7fec140039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec2a7fc640 1 --2- 172.21.15.60:0/908614487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec14003f40 0x7fec14004810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fec2000c210 tx=0x7fec2001f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79ff7bef00723c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec29ffb640 1 -- 172.21.15.60:0/908614487 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20007d20 con 0x7fec14003f40 2024-04-25T10:05:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec29ffb640 1 -- 172.21.15.60:0/908614487 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec20005ae0 con 0x7fec14003f40 2024-04-25T10:05:17.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec29ffb640 1 -- 172.21.15.60:0/908614487 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20007070 con 0x7fec14003f40 2024-04-25T10:05:17.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec31844640 1 -- 172.21.15.60:0/908614487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec14003f40 msgr2=0x7fec14004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec31844640 1 --2- 172.21.15.60:0/908614487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec14003f40 0x7fec14004810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fec2000c210 tx=0x7fec2001f350 comp rx=0 tx=0).stop 2024-04-25T10:05:17.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec2b7fe640 1 -- 172.21.15.60:0/908614487 reap_dead start 2024-04-25T10:05:17.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec31844640 1 -- 172.21.15.60:0/908614487 shutdown_connections 2024-04-25T10:05:17.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec31844640 1 -- 172.21.15.60:0/908614487 >> 172.21.15.60:0/908614487 conn(0x7fec2c0b12d0 msgr2=0x7fec2c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec31844640 1 -- 172.21.15.60:0/908614487 shutdown_connections 2024-04-25T10:05:17.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec31844640 1 -- 172.21.15.60:0/908614487 wait complete. 2024-04-25T10:05:17.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec31844640 1 Processor -- start 2024-04-25T10:05:17.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec31844640 1 -- start start 2024-04-25T10:05:17.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec31844640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14003f40 0x7fec2c145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec31844640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2c15cc30 0x7fec2c1410c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec31844640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec2c143a90 0x7fec2c143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec31844640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c15ab80 con 0x7fec2c143a90 2024-04-25T10:05:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec31844640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c06d8b0 con 0x7fec14003f40 2024-04-25T10:05:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.819+0000 7fec31844640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c15ad00 con 0x7fec2c15cc30 2024-04-25T10:05:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec2b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec2c143a90 0x7fec2c143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec2affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14003f40 0x7fec2c145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec2a7fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2c15cc30 0x7fec2c1410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec2b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec2c143a90 0x7fec2c143f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38818/0 (socket says 172.21.15.60:38818) 2024-04-25T10:05:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec2b7fe640 1 -- 172.21.15.60:0/3531873182 learned_addr learned my addr 172.21.15.60:0/3531873182 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec2b7fe640 1 -- 172.21.15.60:0/3531873182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2c15cc30 msgr2=0x7fec2c1410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec2b7fe640 1 --2- 172.21.15.60:0/3531873182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2c15cc30 0x7fec2c1410c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec2b7fe640 1 -- 172.21.15.60:0/3531873182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14003f40 msgr2=0x7fec2c145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec2b7fe640 1 --2- 172.21.15.60:0/3531873182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14003f40 0x7fec2c145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec2b7fe640 1 -- 172.21.15.60:0/3531873182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c144450 con 0x7fec2c143a90 2024-04-25T10:05:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec2b7fe640 1 --2- 172.21.15.60:0/3531873182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec2c143a90 0x7fec2c143f10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fec2c068640 tx=0x7fec240115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec13fff640 1 -- 172.21.15.60:0/3531873182 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24011e40 con 0x7fec2c143a90 2024-04-25T10:05:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec13fff640 1 -- 172.21.15.60:0/3531873182 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2401b070 con 0x7fec2c143a90 2024-04-25T10:05:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec31844640 1 -- 172.21.15.60:0/3531873182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec2c15d0b0 con 0x7fec2c143a90 2024-04-25T10:05:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec13fff640 1 -- 172.21.15.60:0/3531873182 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec240039a0 con 0x7fec2c143a90 2024-04-25T10:05:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec31844640 1 -- 172.21.15.60:0/3531873182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2c146470 con 0x7fec2c143a90 2024-04-25T10:05:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec31844640 1 -- 172.21.15.60:0/3531873182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec2c074770 con 0x7fec2c143a90 2024-04-25T10:05:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec13fff640 1 -- 172.21.15.60:0/3531873182 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec24003b40 con 0x7fec2c143a90 2024-04-25T10:05:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec13fff640 1 --2- 172.21.15.60:0/3531873182 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec18066f90 0x7fec18069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec13fff640 1 -- 172.21.15.60:0/3531873182 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fec24085210 con 0x7fec2c143a90 2024-04-25T10:05:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec2affd640 1 --2- 172.21.15.60:0/3531873182 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec18066f90 0x7fec18069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.823+0000 7fec2affd640 1 --2- 172.21.15.60:0/3531873182 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec18066f90 0x7fec18069450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fec1c006f90 tx=0x7fec1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:17.827+0000 7fec13fff640 1 -- 172.21.15.60:0/3531873182 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec24055e90 con 0x7fec2c143a90 2024-04-25T10:05:18.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.003+0000 7fec31844640 1 -- 172.21.15.60:0/3531873182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec2c0b0df0 con 0x7fec2c143a90 2024-04-25T10:05:18.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.003+0000 7fec13fff640 1 -- 172.21.15.60:0/3531873182 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12896 (secure 0 0 0) 0x7fec24059f30 con 0x7fec2c143a90 2024-04-25T10:05:18.012 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:18.012 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":18,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:10.833211+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.007+0000 7fec31844640 1 -- 172.21.15.60:0/3531873182 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec18066f90 msgr2=0x7fec18069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.007+0000 7fec31844640 1 --2- 172.21.15.60:0/3531873182 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec18066f90 0x7fec18069450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fec1c006f90 tx=0x7fec1c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.007+0000 7fec31844640 1 -- 172.21.15.60:0/3531873182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec2c143a90 msgr2=0x7fec2c143f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.007+0000 7fec31844640 1 --2- 172.21.15.60:0/3531873182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec2c143a90 0x7fec2c143f10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fec2c068640 tx=0x7fec240115a0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.007+0000 7fec31844640 1 -- 172.21.15.60:0/3531873182 shutdown_connections 2024-04-25T10:05:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.007+0000 7fec31844640 1 --2- 172.21.15.60:0/3531873182 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec18066f90 0x7fec18069450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.007+0000 7fec31844640 1 --2- 172.21.15.60:0/3531873182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec2c143a90 0x7fec2c143f10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.007+0000 7fec31844640 1 --2- 172.21.15.60:0/3531873182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec2c15cc30 0x7fec2c1410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.007+0000 7fec31844640 1 --2- 172.21.15.60:0/3531873182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec14003f40 0x7fec2c145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.007+0000 7fec31844640 1 -- 172.21.15.60:0/3531873182 >> 172.21.15.60:0/3531873182 conn(0x7fec2c0b12d0 msgr2=0x7fec2c0ce4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.007+0000 7fec31844640 1 -- 172.21.15.60:0/3531873182 shutdown_connections 2024-04-25T10:05:18.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.007+0000 7fec31844640 1 -- 172.21.15.60:0/3531873182 wait complete. 2024-04-25T10:05:18.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:18.029 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:18.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f578295e640 1 Processor -- start 2024-04-25T10:05:18.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f578295e640 1 -- start start 2024-04-25T10:05:18.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f578295e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f577c15b6f0 0x7f577c1620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f578295e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f577c15e0d0 0x7f577c15e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f578295e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f577c15bad0 con 0x7f577c0ce7e0 2024-04-25T10:05:18.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f578295e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577c162610 con 0x7f577c15b6f0 2024-04-25T10:05:18.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f578295e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577c161c40 con 0x7f577c15e0d0 2024-04-25T10:05:18.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f5780efb640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f577c0ce7e0 0x7f577c0d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48310/0 (socket says 172.21.15.60:48310) 2024-04-25T10:05:18.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f5780efb640 1 -- 172.21.15.60:0/2420367871 learned_addr learned my addr 172.21.15.60:0/2420367871 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:18.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f577bfff640 1 --2- 172.21.15.60:0/2420367871 >> v2:172.21.15.60:3300/0 conn(0x7f577c15b6f0 0x7f577c1620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f577affd640 1 -- 172.21.15.60:0/2420367871 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156937779 0 0) 0x7f577c15bad0 con 0x7f577c0ce7e0 2024-04-25T10:05:18.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f577b7fe640 1 --2- 172.21.15.60:0/2420367871 >> v2:172.21.15.100:3300/0 conn(0x7f577c15e0d0 0x7f577c15e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f577bfff640 1 -- 172.21.15.60:0/2420367871 >> v2:172.21.15.100:3300/0 conn(0x7f577c15e0d0 msgr2=0x7f577c15e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f577bfff640 1 --2- 172.21.15.60:0/2420367871 >> v2:172.21.15.100:3300/0 conn(0x7f577c15e0d0 0x7f577c15e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f577bfff640 1 -- 172.21.15.60:0/2420367871 >> v1:172.21.15.60:6789/0 conn(0x7f577c0ce7e0 legacy=0x7f577c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f577bfff640 1 -- 172.21.15.60:0/2420367871 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c15ebe0 con 0x7f577c15b6f0 2024-04-25T10:05:18.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.139+0000 7f577bfff640 1 --2- 172.21.15.60:0/2420367871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f577c15b6f0 0x7f577c1620d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f577c0b6600 tx=0x7f576c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb5b1d7a4895fce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f577affd640 1 -- 172.21.15.60:0/2420367871 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c0b0070 con 0x7f577c15b6f0 2024-04-25T10:05:18.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f577affd640 1 -- 172.21.15.60:0/2420367871 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576c0ab600 con 0x7f577c15b6f0 2024-04-25T10:05:18.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f577affd640 1 -- 172.21.15.60:0/2420367871 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c0ab9d0 con 0x7f577c15b6f0 2024-04-25T10:05:18.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 -- 172.21.15.60:0/2420367871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f577c15b6f0 msgr2=0x7f577c1620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 --2- 172.21.15.60:0/2420367871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f577c15b6f0 0x7f577c1620d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f577c0b6600 tx=0x7f576c01c480 comp rx=0 tx=0).stop 2024-04-25T10:05:18.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 -- 172.21.15.60:0/2420367871 shutdown_connections 2024-04-25T10:05:18.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 --2- 172.21.15.60:0/2420367871 >> v2:172.21.15.100:3300/0 conn(0x7f577c15e0d0 0x7f577c15e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 --2- 172.21.15.60:0/2420367871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f577c15b6f0 0x7f577c1620d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 -- 172.21.15.60:0/2420367871 >> 172.21.15.60:0/2420367871 conn(0x7f577c072340 msgr2=0x7f577c072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 -- 172.21.15.60:0/2420367871 shutdown_connections 2024-04-25T10:05:18.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 -- 172.21.15.60:0/2420367871 wait complete. 2024-04-25T10:05:18.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 Processor -- start 2024-04-25T10:05:18.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 -- start start 2024-04-25T10:05:18.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f577c0ce7e0 0x7f577c0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f577c15b6f0 0x7f577c0b7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f577c15e0d0 0x7f577c1503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c0b96d0 con 0x7f577c15b6f0 2024-04-25T10:05:18.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c0b9550 con 0x7f577c0ce7e0 2024-04-25T10:05:18.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577c0cf000 con 0x7f577c15e0d0 2024-04-25T10:05:18.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f577b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f577c15b6f0 0x7f577c0b7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f577b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f577c15b6f0 0x7f577c0b7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38838/0 (socket says 172.21.15.60:38838) 2024-04-25T10:05:18.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f577b7fe640 1 -- 172.21.15.60:0/1520900246 learned_addr learned my addr 172.21.15.60:0/1520900246 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:18.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f577bfff640 1 --2- 172.21.15.60:0/1520900246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f577c0ce7e0 0x7f577c0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f5780efb640 1 --2- 172.21.15.60:0/1520900246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f577c15e0d0 0x7f577c1503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f577b7fe640 1 -- 172.21.15.60:0/1520900246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f577c15e0d0 msgr2=0x7f577c1503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f577b7fe640 1 --2- 172.21.15.60:0/1520900246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f577c15e0d0 0x7f577c1503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f577b7fe640 1 -- 172.21.15.60:0/1520900246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f577c0ce7e0 msgr2=0x7f577c0c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f577b7fe640 1 --2- 172.21.15.60:0/1520900246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f577c0ce7e0 0x7f577c0c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f577b7fe640 1 -- 172.21.15.60:0/1520900246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c0c78a0 con 0x7f577c15b6f0 2024-04-25T10:05:18.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f577b7fe640 1 --2- 172.21.15.60:0/1520900246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f577c15b6f0 0x7f577c0b7e60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f577000c7f0 tx=0x7f577000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f5778ff9640 1 -- 172.21.15.60:0/1520900246 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770015070 con 0x7f577c15b6f0 2024-04-25T10:05:18.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f5778ff9640 1 -- 172.21.15.60:0/1520900246 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5770007820 con 0x7f577c15b6f0 2024-04-25T10:05:18.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f5778ff9640 1 -- 172.21.15.60:0/1520900246 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770007c10 con 0x7f577c15b6f0 2024-04-25T10:05:18.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 -- 172.21.15.60:0/1520900246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577c0cbcc0 con 0x7f577c15b6f0 2024-04-25T10:05:18.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.143+0000 7f578295e640 1 -- 172.21.15.60:0/1520900246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577c0cc280 con 0x7f577c15b6f0 2024-04-25T10:05:18.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.147+0000 7f578295e640 1 -- 172.21.15.60:0/1520900246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577c074770 con 0x7f577c15b6f0 2024-04-25T10:05:18.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.147+0000 7f5778ff9640 1 -- 172.21.15.60:0/1520900246 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5770018070 con 0x7f577c15b6f0 2024-04-25T10:05:18.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.147+0000 7f5778ff9640 1 --2- 172.21.15.60:0/1520900246 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5768066f90 0x7f5768069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.147+0000 7f5778ff9640 1 -- 172.21.15.60:0/1520900246 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5770084c80 con 0x7f577c15b6f0 2024-04-25T10:05:18.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.147+0000 7f577bfff640 1 --2- 172.21.15.60:0/1520900246 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5768066f90 0x7f5768069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.147+0000 7f577bfff640 1 --2- 172.21.15.60:0/1520900246 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5768066f90 0x7f5768069450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f576c0ad000 tx=0x7f576c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.151+0000 7f5778ff9640 1 -- 172.21.15.60:0/1520900246 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5770056070 con 0x7f577c15b6f0 2024-04-25T10:05:18.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.303+0000 7f578295e640 1 -- 172.21.15.60:0/1520900246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f577c0d16e0 con 0x7f577c15b6f0 2024-04-25T10:05:18.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.303+0000 7f5778ff9640 1 -- 172.21.15.60:0/1520900246 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5770055970 con 0x7f577c15b6f0 2024-04-25T10:05:18.312 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:05:18.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.307+0000 7f578295e640 1 -- 172.21.15.60:0/1520900246 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5768066f90 msgr2=0x7f5768069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.307+0000 7f578295e640 1 --2- 172.21.15.60:0/1520900246 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5768066f90 0x7f5768069450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f576c0ad000 tx=0x7f576c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:05:18.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.307+0000 7f578295e640 1 -- 172.21.15.60:0/1520900246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f577c15b6f0 msgr2=0x7f577c0b7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.307+0000 7f578295e640 1 --2- 172.21.15.60:0/1520900246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f577c15b6f0 0x7f577c0b7e60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f577000c7f0 tx=0x7f577000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.307+0000 7f578295e640 1 -- 172.21.15.60:0/1520900246 shutdown_connections 2024-04-25T10:05:18.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.307+0000 7f578295e640 1 --2- 172.21.15.60:0/1520900246 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5768066f90 0x7f5768069450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.307+0000 7f578295e640 1 --2- 172.21.15.60:0/1520900246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f577c15e0d0 0x7f577c1503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.307+0000 7f578295e640 1 --2- 172.21.15.60:0/1520900246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f577c15b6f0 0x7f577c0b7e60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.307+0000 7f578295e640 1 --2- 172.21.15.60:0/1520900246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f577c0ce7e0 0x7f577c0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.307+0000 7f578295e640 1 -- 172.21.15.60:0/1520900246 >> 172.21.15.60:0/1520900246 conn(0x7f577c072340 msgr2=0x7f577c0d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.307+0000 7f578295e640 1 -- 172.21.15.60:0/1520900246 shutdown_connections 2024-04-25T10:05:18.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.307+0000 7f578295e640 1 -- 172.21.15.60:0/1520900246 wait complete. 2024-04-25T10:05:18.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:05:18.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1694b4640 1 Processor -- start 2024-04-25T10:05:18.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1694b4640 1 -- start start 2024-04-25T10:05:18.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1694b4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff164070300 0x7ff16415ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1694b4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff16406bc20 0x7ff16406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1694b4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1640ad670 con 0x7ff164161590 2024-04-25T10:05:18.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1694b4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff16406c5c0 con 0x7ff164070300 2024-04-25T10:05:18.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1694b4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1640ad950 con 0x7ff16406bc20 2024-04-25T10:05:18.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1637fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff164161590 0x7ff164161a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48326/0 (socket says 172.21.15.60:48326) 2024-04-25T10:05:18.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1637fe640 1 -- 172.21.15.60:0/4275420513 learned_addr learned my addr 172.21.15.60:0/4275420513 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:18.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1627fc640 1 --2- 172.21.15.60:0/4275420513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff16406bc20 0x7ff16406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff161ffb640 1 -- 172.21.15.60:0/4275420513 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801093968 0 0) 0x7ff1640ad670 con 0x7ff164161590 2024-04-25T10:05:18.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff161ffb640 1 -- 172.21.15.60:0/4275420513 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff14c003530 con 0x7ff164161590 2024-04-25T10:05:18.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff162ffd640 1 --2- 172.21.15.60:0/4275420513 >> v2:172.21.15.100:3300/0 conn(0x7ff164070300 0x7ff16415ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff161ffb640 1 -- 172.21.15.60:0/4275420513 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 490676760 0 0) 0x7ff14c003530 con 0x7ff164161590 2024-04-25T10:05:18.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff161ffb640 1 -- 172.21.15.60:0/4275420513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff16406bc20 msgr2=0x7ff16406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff161ffb640 1 --2- 172.21.15.60:0/4275420513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff16406bc20 0x7ff16406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff161ffb640 1 -- 172.21.15.60:0/4275420513 >> v2:172.21.15.100:3300/0 conn(0x7ff164070300 msgr2=0x7ff16415ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff161ffb640 1 --2- 172.21.15.60:0/4275420513 >> v2:172.21.15.100:3300/0 conn(0x7ff164070300 0x7ff16415ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff161ffb640 1 -- 172.21.15.60:0/4275420513 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1640adc30 con 0x7ff164161590 2024-04-25T10:05:18.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff161ffb640 1 -- 172.21.15.60:0/4275420513 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7ff15c001a00 con 0x7ff164161590 2024-04-25T10:05:18.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff161ffb640 1 -- 172.21.15.60:0/4275420513 >> v1:172.21.15.60:6789/0 conn(0x7ff164161590 legacy=0x7ff164161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff161ffb640 1 --2- 172.21.15.60:0/4275420513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff14c0039c0 0x7ff14c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff161ffb640 1 --2- 172.21.15.60:0/4275420513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff16406bc20 0x7ff14c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff161ffb640 1 --2- 172.21.15.60:0/4275420513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff14c004d50 0x7ff14c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff161ffb640 1 -- 172.21.15.60:0/4275420513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1640ad950 con 0x7ff14c004d50 2024-04-25T10:05:18.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff161ffb640 1 -- 172.21.15.60:0/4275420513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14c00b9d0 con 0x7ff14c0039c0 2024-04-25T10:05:18.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff161ffb640 1 -- 172.21.15.60:0/4275420513 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14c00bb80 con 0x7ff16406bc20 2024-04-25T10:05:18.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1637fe640 1 --2- 172.21.15.60:0/4275420513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff14c004d50 0x7ff14c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff162ffd640 1 --2- 172.21.15.60:0/4275420513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff14c0039c0 0x7ff14c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1627fc640 1 --2- 172.21.15.60:0/4275420513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff16406bc20 0x7ff14c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1637fe640 1 -- 172.21.15.60:0/4275420513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff16406bc20 msgr2=0x7ff14c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1637fe640 1 --2- 172.21.15.60:0/4275420513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff16406bc20 0x7ff14c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1637fe640 1 -- 172.21.15.60:0/4275420513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff14c0039c0 msgr2=0x7ff14c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1637fe640 1 --2- 172.21.15.60:0/4275420513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff14c0039c0 0x7ff14c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.435+0000 7ff1637fe640 1 -- 172.21.15.60:0/4275420513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14c003530 con 0x7ff14c004d50 2024-04-25T10:05:18.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1637fe640 1 --2- 172.21.15.60:0/4275420513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff14c004d50 0x7ff14c0051d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00a7b0 tx=0x7ff15c01fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d565305477146d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff161ffb640 1 -- 172.21.15.60:0/4275420513 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00a3d0 con 0x7ff14c004d50 2024-04-25T10:05:18.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff161ffb640 1 -- 172.21.15.60:0/4275420513 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15c0aa690 con 0x7ff14c004d50 2024-04-25T10:05:18.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff161ffb640 1 -- 172.21.15.60:0/4275420513 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0aaa80 con 0x7ff14c004d50 2024-04-25T10:05:18.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1694b4640 1 -- 172.21.15.60:0/4275420513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff14c004d50 msgr2=0x7ff14c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1694b4640 1 --2- 172.21.15.60:0/4275420513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff14c004d50 0x7ff14c0051d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00a7b0 tx=0x7ff15c01fa40 comp rx=0 tx=0).stop 2024-04-25T10:05:18.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1637fe640 1 -- 172.21.15.60:0/4275420513 reap_dead start 2024-04-25T10:05:18.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1694b4640 1 -- 172.21.15.60:0/4275420513 shutdown_connections 2024-04-25T10:05:18.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1694b4640 1 -- 172.21.15.60:0/4275420513 >> 172.21.15.60:0/4275420513 conn(0x7ff164071b90 msgr2=0x7ff164071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1694b4640 1 -- 172.21.15.60:0/4275420513 shutdown_connections 2024-04-25T10:05:18.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1694b4640 1 -- 172.21.15.60:0/4275420513 wait complete. 2024-04-25T10:05:18.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1694b4640 1 Processor -- start 2024-04-25T10:05:18.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1694b4640 1 -- start start 2024-04-25T10:05:18.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1694b4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff14c004d50 0x7ff16414c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1694b4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff16414fa00 0x7ff16415f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1694b4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff16415de30 0x7ff16415e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1694b4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1640ad2d0 con 0x7ff16415de30 2024-04-25T10:05:18.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1694b4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1640ae1f0 con 0x7ff14c004d50 2024-04-25T10:05:18.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1694b4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1640ad450 con 0x7ff16414fa00 2024-04-25T10:05:18.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1637fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff16415de30 0x7ff16415e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1637fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff16415de30 0x7ff16415e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38860/0 (socket says 172.21.15.60:38860) 2024-04-25T10:05:18.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1637fe640 1 -- 172.21.15.60:0/4151001509 learned_addr learned my addr 172.21.15.60:0/4151001509 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:18.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff162ffd640 1 --2- 172.21.15.60:0/4151001509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff14c004d50 0x7ff16414c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1627fc640 1 --2- 172.21.15.60:0/4151001509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff16414fa00 0x7ff16415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1637fe640 1 -- 172.21.15.60:0/4151001509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff16414fa00 msgr2=0x7ff16415f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1637fe640 1 --2- 172.21.15.60:0/4151001509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff16414fa00 0x7ff16415f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1637fe640 1 -- 172.21.15.60:0/4151001509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff14c004d50 msgr2=0x7ff16414c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1637fe640 1 --2- 172.21.15.60:0/4151001509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff14c004d50 0x7ff16414c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1637fe640 1 -- 172.21.15.60:0/4151001509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1641600f0 con 0x7ff16415de30 2024-04-25T10:05:18.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff162ffd640 1 --2- 172.21.15.60:0/4151001509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff14c004d50 0x7ff16414c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:18.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.439+0000 7ff1637fe640 1 --2- 172.21.15.60:0/4151001509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff16415de30 0x7ff16415e2c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff15c01f870 tx=0x7ff15c01c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.443+0000 7ff14b7fe640 1 -- 172.21.15.60:0/4151001509 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0c0070 con 0x7ff16415de30 2024-04-25T10:05:18.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.443+0000 7ff14b7fe640 1 -- 172.21.15.60:0/4151001509 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15c0aa840 con 0x7ff16415de30 2024-04-25T10:05:18.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.443+0000 7ff1694b4640 1 -- 172.21.15.60:0/4151001509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1641603d0 con 0x7ff16415de30 2024-04-25T10:05:18.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.443+0000 7ff14b7fe640 1 -- 172.21.15.60:0/4151001509 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c001f30 con 0x7ff16415de30 2024-04-25T10:05:18.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.443+0000 7ff1694b4640 1 -- 172.21.15.60:0/4151001509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff164160990 con 0x7ff16415de30 2024-04-25T10:05:18.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.443+0000 7ff1694b4640 1 -- 172.21.15.60:0/4151001509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff164074770 con 0x7ff16415de30 2024-04-25T10:05:18.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.443+0000 7ff14b7fe640 1 -- 172.21.15.60:0/4151001509 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff15c0c3070 con 0x7ff16415de30 2024-04-25T10:05:18.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.443+0000 7ff14b7fe640 1 --2- 172.21.15.60:0/4151001509 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff150066f60 0x7ff150069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.443+0000 7ff14b7fe640 1 -- 172.21.15.60:0/4151001509 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff15c122300 con 0x7ff16415de30 2024-04-25T10:05:18.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.443+0000 7ff162ffd640 1 --2- 172.21.15.60:0/4151001509 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff150066f60 0x7ff150069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.443+0000 7ff162ffd640 1 --2- 172.21.15.60:0/4151001509 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff150066f60 0x7ff150069420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff1640cf770 tx=0x7ff158008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.447+0000 7ff14b7fe640 1 -- 172.21.15.60:0/4151001509 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff15c0af020 con 0x7ff16415de30 2024-04-25T10:05:18.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.607+0000 7ff1694b4640 1 -- 172.21.15.60:0/4151001509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff16406f7a0 con 0x7ff16415de30 2024-04-25T10:05:18.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.611+0000 7ff14b7fe640 1 -- 172.21.15.60:0/4151001509 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff15c019750 con 0x7ff16415de30 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:05:18.617 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T10:05:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.611+0000 7ff1694b4640 1 -- 172.21.15.60:0/4151001509 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff150066f60 msgr2=0x7ff150069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.611+0000 7ff1694b4640 1 --2- 172.21.15.60:0/4151001509 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff150066f60 0x7ff150069420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff1640cf770 tx=0x7ff158008040 comp rx=0 tx=0).stop 2024-04-25T10:05:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.611+0000 7ff1694b4640 1 -- 172.21.15.60:0/4151001509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff16415de30 msgr2=0x7ff16415e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.611+0000 7ff1694b4640 1 --2- 172.21.15.60:0/4151001509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff16415de30 0x7ff16415e2c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff15c01f870 tx=0x7ff15c01c7e0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.611+0000 7ff1694b4640 1 -- 172.21.15.60:0/4151001509 shutdown_connections 2024-04-25T10:05:18.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.611+0000 7ff1694b4640 1 --2- 172.21.15.60:0/4151001509 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff150066f60 0x7ff150069420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.611+0000 7ff1694b4640 1 --2- 172.21.15.60:0/4151001509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff16415de30 0x7ff16415e2c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.611+0000 7ff1694b4640 1 --2- 172.21.15.60:0/4151001509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff16414fa00 0x7ff16415f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.611+0000 7ff1694b4640 1 --2- 172.21.15.60:0/4151001509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff14c004d50 0x7ff16414c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.611+0000 7ff1694b4640 1 -- 172.21.15.60:0/4151001509 >> 172.21.15.60:0/4151001509 conn(0x7ff164071b90 msgr2=0x7ff1640b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.611+0000 7ff1694b4640 1 -- 172.21.15.60:0/4151001509 shutdown_connections 2024-04-25T10:05:18.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.611+0000 7ff1694b4640 1 -- 172.21.15.60:0/4151001509 wait complete. 2024-04-25T10:05:18.636 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:05:18.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:05:18.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.763+0000 7f6b77d50640 1 Processor -- start 2024-04-25T10:05:18.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.763+0000 7f6b77d50640 1 -- start start 2024-04-25T10:05:18.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.763+0000 7f6b77d50640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6b70144e40 0x7f6b700ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.763+0000 7f6b77d50640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b700cf810 con 0x7f6b700cf0a0 2024-04-25T10:05:18.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.763+0000 7f6b77d50640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b700cfaf0 con 0x7f6b70071980 2024-04-25T10:05:18.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.763+0000 7f6b77d50640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b700b3060 con 0x7f6b70144e40 2024-04-25T10:05:18.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.763+0000 7f6b75aec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6b70144e40 0x7f6b700ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.763+0000 7f6b752eb640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6b700cf0a0 0x7f6b70071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48332/0 (socket says 172.21.15.60:48332) 2024-04-25T10:05:18.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.763+0000 7f6b752eb640 1 -- 172.21.15.60:0/501000533 learned_addr learned my addr 172.21.15.60:0/501000533 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:18.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.763+0000 7f6b75aec640 1 -- 172.21.15.60:0/501000533 >> v1:172.21.15.100:6789/0 conn(0x7f6b70071980 legacy=0x7f6b700b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.763+0000 7f6b75aec640 1 -- 172.21.15.60:0/501000533 >> v1:172.21.15.60:6789/0 conn(0x7f6b700cf0a0 legacy=0x7f6b70071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.763+0000 7f6b74aea640 1 -- 172.21.15.60:0/501000533 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3771862343 0 0) 0x7f6b700cf810 con 0x7f6b700cf0a0 2024-04-25T10:05:18.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.763+0000 7f6b75aec640 1 -- 172.21.15.60:0/501000533 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b700cfdd0 con 0x7f6b70144e40 2024-04-25T10:05:18.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b75aec640 1 --2- 172.21.15.60:0/501000533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b70144e40 0x7f6b700ccc60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6b700b31e0 tx=0x7f6b6801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c3c2b323399fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b74aea640 1 -- 172.21.15.60:0/501000533 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680b2070 con 0x7f6b70144e40 2024-04-25T10:05:18.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b74aea640 1 -- 172.21.15.60:0/501000533 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b70145430 con 0x7f6b70144e40 2024-04-25T10:05:18.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b74aea640 1 -- 172.21.15.60:0/501000533 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680a5bb0 con 0x7f6b70144e40 2024-04-25T10:05:18.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b77d50640 1 -- 172.21.15.60:0/501000533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b70144e40 msgr2=0x7f6b700ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b77d50640 1 --2- 172.21.15.60:0/501000533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b70144e40 0x7f6b700ccc60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6b700b31e0 tx=0x7f6b6801c8b0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b77d50640 1 -- 172.21.15.60:0/501000533 shutdown_connections 2024-04-25T10:05:18.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b77d50640 1 --2- 172.21.15.60:0/501000533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b70144e40 0x7f6b700ccc60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b77d50640 1 -- 172.21.15.60:0/501000533 >> 172.21.15.60:0/501000533 conn(0x7f6b70066fa0 msgr2=0x7f6b7006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b77d50640 1 -- 172.21.15.60:0/501000533 shutdown_connections 2024-04-25T10:05:18.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b77d50640 1 -- 172.21.15.60:0/501000533 wait complete. 2024-04-25T10:05:18.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b77d50640 1 Processor -- start 2024-04-25T10:05:18.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b77d50640 1 -- start start 2024-04-25T10:05:18.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b77d50640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b70071980 0x7f6b70159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b77d50640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b700cf0a0 0x7f6b70159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b77d50640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b70144e40 0x7f6b701f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b77d50640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b700b47a0 con 0x7f6b70071980 2024-04-25T10:05:18.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b77d50640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b700b4620 con 0x7f6b700cf0a0 2024-04-25T10:05:18.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b77d50640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b701696f0 con 0x7f6b70144e40 2024-04-25T10:05:18.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b75aec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b70071980 0x7f6b70159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b75aec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b70071980 0x7f6b70159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38888/0 (socket says 172.21.15.60:38888) 2024-04-25T10:05:18.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.767+0000 7f6b75aec640 1 -- 172.21.15.60:0/3438755175 learned_addr learned my addr 172.21.15.60:0/3438755175 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:18.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b752eb640 1 --2- 172.21.15.60:0/3438755175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b700cf0a0 0x7f6b70159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b762ed640 1 --2- 172.21.15.60:0/3438755175 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b70144e40 0x7f6b701f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b75aec640 1 -- 172.21.15.60:0/3438755175 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b70144e40 msgr2=0x7f6b701f7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b75aec640 1 --2- 172.21.15.60:0/3438755175 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b70144e40 0x7f6b701f7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b75aec640 1 -- 172.21.15.60:0/3438755175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b700cf0a0 msgr2=0x7f6b70159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b75aec640 1 --2- 172.21.15.60:0/3438755175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b700cf0a0 0x7f6b70159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b75aec640 1 -- 172.21.15.60:0/3438755175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b70140cd0 con 0x7f6b70071980 2024-04-25T10:05:18.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b752eb640 1 --2- 172.21.15.60:0/3438755175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b700cf0a0 0x7f6b70159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:18.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b762ed640 1 --2- 172.21.15.60:0/3438755175 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b70144e40 0x7f6b701f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:18.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b75aec640 1 --2- 172.21.15.60:0/3438755175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b70071980 0x7f6b70159760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b680b0040 tx=0x7f6b680196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b667fc640 1 -- 172.21.15.60:0/3438755175 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680b2040 con 0x7f6b70071980 2024-04-25T10:05:18.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b667fc640 1 -- 172.21.15.60:0/3438755175 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b680bf070 con 0x7f6b70071980 2024-04-25T10:05:18.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b77d50640 1 -- 172.21.15.60:0/3438755175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7013c760 con 0x7f6b70071980 2024-04-25T10:05:18.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b667fc640 1 -- 172.21.15.60:0/3438755175 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680b88d0 con 0x7f6b70071980 2024-04-25T10:05:18.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b77d50640 1 -- 172.21.15.60:0/3438755175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b70140f70 con 0x7f6b70071980 2024-04-25T10:05:18.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b77d50640 1 -- 172.21.15.60:0/3438755175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b70074770 con 0x7f6b70071980 2024-04-25T10:05:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b667fc640 1 -- 172.21.15.60:0/3438755175 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b680b8a70 con 0x7f6b70071980 2024-04-25T10:05:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b667fc640 1 --2- 172.21.15.60:0/3438755175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6b54066f60 0x7f6b54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b667fc640 1 -- 172.21.15.60:0/3438755175 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6b6811ffc0 con 0x7f6b70071980 2024-04-25T10:05:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b752eb640 1 --2- 172.21.15.60:0/3438755175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6b54066f60 0x7f6b54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.771+0000 7f6b752eb640 1 --2- 172.21.15.60:0/3438755175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6b54066f60 0x7f6b54069420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6b70157fe0 tx=0x7f6b60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.775+0000 7f6b667fc640 1 -- 172.21.15.60:0/3438755175 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6b68016500 con 0x7f6b70071980 2024-04-25T10:05:18.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:18.935+0000 7f6b77d50640 1 -- 172.21.15.60:0/3438755175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6b700b0b60 con 0x7f6b70071980 2024-04-25T10:05:19.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.027+0000 7f6b667fc640 1 -- 172.21.15.60:0/3438755175 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6b680ab340 con 0x7f6b70071980 2024-04-25T10:05:19.035 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T10:05:19.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.031+0000 7f6b77d50640 1 -- 172.21.15.60:0/3438755175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6b54066f60 msgr2=0x7f6b54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.031+0000 7f6b77d50640 1 --2- 172.21.15.60:0/3438755175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6b54066f60 0x7f6b54069420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6b70157fe0 tx=0x7f6b60009040 comp rx=0 tx=0).stop 2024-04-25T10:05:19.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.031+0000 7f6b77d50640 1 -- 172.21.15.60:0/3438755175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b70071980 msgr2=0x7f6b70159760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.031+0000 7f6b77d50640 1 --2- 172.21.15.60:0/3438755175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b70071980 0x7f6b70159760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b680b0040 tx=0x7f6b680196b0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.031+0000 7f6b77d50640 1 -- 172.21.15.60:0/3438755175 shutdown_connections 2024-04-25T10:05:19.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.031+0000 7f6b77d50640 1 --2- 172.21.15.60:0/3438755175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6b54066f60 0x7f6b54069420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.031+0000 7f6b77d50640 1 --2- 172.21.15.60:0/3438755175 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b70144e40 0x7f6b701f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.031+0000 7f6b77d50640 1 --2- 172.21.15.60:0/3438755175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b700cf0a0 0x7f6b70159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.031+0000 7f6b77d50640 1 --2- 172.21.15.60:0/3438755175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b70071980 0x7f6b70159760 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.031+0000 7f6b77d50640 1 -- 172.21.15.60:0/3438755175 >> 172.21.15.60:0/3438755175 conn(0x7f6b70066fa0 msgr2=0x7f6b70140390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:19.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.031+0000 7f6b77d50640 1 -- 172.21.15.60:0/3438755175 shutdown_connections 2024-04-25T10:05:19.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.031+0000 7f6b77d50640 1 -- 172.21.15.60:0/3438755175 wait complete. 2024-04-25T10:05:19.061 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:05:19.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.187+0000 7f9f5c792640 1 Processor -- start 2024-04-25T10:05:19.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.187+0000 7f9f5c792640 1 -- start start 2024-04-25T10:05:19.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.187+0000 7f9f5c792640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f540ce530 0x7f9f5406c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.187+0000 7f9f5c792640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f540cefd0 con 0x7f9f540cea10 2024-04-25T10:05:19.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.187+0000 7f9f5c792640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f540cf2b0 con 0x7f9f5415ddd0 2024-04-25T10:05:19.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.187+0000 7f9f5c792640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f540d1f90 con 0x7f9f540ce530 2024-04-25T10:05:19.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.187+0000 7f9f5ad2f640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9f540cea10 0x7f9f540d29e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55702/0 (socket says 172.21.15.60:55702) 2024-04-25T10:05:19.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.187+0000 7f9f5ad2f640 1 -- 172.21.15.60:0/3967794686 learned_addr learned my addr 172.21.15.60:0/3967794686 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:19.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f59d2d640 1 --2- 172.21.15.60:0/3967794686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f540ce530 0x7f9f5406c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 -- 172.21.15.60:0/3967794686 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277597907 0 0) 0x7f9f540cefd0 con 0x7f9f540cea10 2024-04-25T10:05:19.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 -- 172.21.15.60:0/3967794686 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f40003530 con 0x7f9f540cea10 2024-04-25T10:05:19.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 -- 172.21.15.60:0/3967794686 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2046708212 0 0) 0x7f9f40003530 con 0x7f9f540cea10 2024-04-25T10:05:19.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 -- 172.21.15.60:0/3967794686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f540ce530 msgr2=0x7f9f5406c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:19.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 --2- 172.21.15.60:0/3967794686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f540ce530 0x7f9f5406c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 -- 172.21.15.60:0/3967794686 >> v1:172.21.15.100:6789/0 conn(0x7f9f5415ddd0 legacy=0x7f9f540b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 -- 172.21.15.60:0/3967794686 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f540d21e0 con 0x7f9f540cea10 2024-04-25T10:05:19.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 -- 172.21.15.60:0/3967794686 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f9f4c001680 con 0x7f9f540cea10 2024-04-25T10:05:19.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 -- 172.21.15.60:0/3967794686 >> v1:172.21.15.60:6789/0 conn(0x7f9f540cea10 legacy=0x7f9f540d29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 --2- 172.21.15.60:0/3967794686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f400039f0 0x7f9f40003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 --2- 172.21.15.60:0/3967794686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f540ce530 0x7f9f40004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 --2- 172.21.15.60:0/3967794686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f40004d50 0x7f9f400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 -- 172.21.15.60:0/3967794686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f540d1f90 con 0x7f9f40004d50 2024-04-25T10:05:19.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 -- 172.21.15.60:0/3967794686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4000ba80 con 0x7f9f400039f0 2024-04-25T10:05:19.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 -- 172.21.15.60:0/3967794686 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4000bc80 con 0x7f9f540ce530 2024-04-25T10:05:19.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5ad2f640 1 --2- 172.21.15.60:0/3967794686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f40004d50 0x7f9f400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f59d2d640 1 --2- 172.21.15.60:0/3967794686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f540ce530 0x7f9f40004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5a52e640 1 --2- 172.21.15.60:0/3967794686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f400039f0 0x7f9f40003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5ad2f640 1 -- 172.21.15.60:0/3967794686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f540ce530 msgr2=0x7f9f40004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:19.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5ad2f640 1 --2- 172.21.15.60:0/3967794686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f540ce530 0x7f9f40004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5ad2f640 1 -- 172.21.15.60:0/3967794686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f400039f0 msgr2=0x7f9f40003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:19.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5ad2f640 1 --2- 172.21.15.60:0/3967794686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f400039f0 0x7f9f40003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5ad2f640 1 -- 172.21.15.60:0/3967794686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f40003530 con 0x7f9f40004d50 2024-04-25T10:05:19.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5ad2f640 1 --2- 172.21.15.60:0/3967794686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f40004d50 0x7f9f400051d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00c190 tx=0x7f9f4c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a959b5e3dc1a855 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 -- 172.21.15.60:0/3967794686 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c007d00 con 0x7f9f40004d50 2024-04-25T10:05:19.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 -- 172.21.15.60:0/3967794686 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c0a9020 con 0x7f9f40004d50 2024-04-25T10:05:19.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.191+0000 7f9f5952c640 1 -- 172.21.15.60:0/3967794686 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0a5b60 con 0x7f9f40004d50 2024-04-25T10:05:19.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 -- 172.21.15.60:0/3967794686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f40004d50 msgr2=0x7f9f400051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 --2- 172.21.15.60:0/3967794686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f40004d50 0x7f9f400051d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00c190 tx=0x7f9f4c01cc40 comp rx=0 tx=0).stop 2024-04-25T10:05:19.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5ad2f640 1 -- 172.21.15.60:0/3967794686 reap_dead start 2024-04-25T10:05:19.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 -- 172.21.15.60:0/3967794686 shutdown_connections 2024-04-25T10:05:19.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 -- 172.21.15.60:0/3967794686 >> 172.21.15.60:0/3967794686 conn(0x7f9f540d04c0 msgr2=0x7f9f540d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:19.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 -- 172.21.15.60:0/3967794686 shutdown_connections 2024-04-25T10:05:19.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 -- 172.21.15.60:0/3967794686 wait complete. 2024-04-25T10:05:19.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 Processor -- start 2024-04-25T10:05:19.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 -- start start 2024-04-25T10:05:19.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f541f6b80 0x7f9f5413f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f54157e50 0x7f9f541582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f540cbad0 0x7f9f540cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5406ce80 con 0x7f9f54157e50 2024-04-25T10:05:19.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f540d2550 con 0x7f9f541f6b80 2024-04-25T10:05:19.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5406d000 con 0x7f9f540cbad0 2024-04-25T10:05:19.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f59d2d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f54157e50 0x7f9f541582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f59d2d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f54157e50 0x7f9f541582d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45054/0 (socket says 172.21.15.60:45054) 2024-04-25T10:05:19.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f59d2d640 1 -- 172.21.15.60:0/3832594452 learned_addr learned my addr 172.21.15.60:0/3832594452 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:19.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5ad2f640 1 --2- 172.21.15.60:0/3832594452 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f540cbad0 0x7f9f540cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5a52e640 1 --2- 172.21.15.60:0/3832594452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f541f6b80 0x7f9f5413f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f59d2d640 1 -- 172.21.15.60:0/3832594452 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f540cbad0 msgr2=0x7f9f540cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f59d2d640 1 --2- 172.21.15.60:0/3832594452 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f540cbad0 0x7f9f540cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f59d2d640 1 -- 172.21.15.60:0/3832594452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f541f6b80 msgr2=0x7f9f5413f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f59d2d640 1 --2- 172.21.15.60:0/3832594452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f541f6b80 0x7f9f5413f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f59d2d640 1 -- 172.21.15.60:0/3832594452 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5413faf0 con 0x7f9f54157e50 2024-04-25T10:05:19.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5a52e640 1 --2- 172.21.15.60:0/3832594452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f541f6b80 0x7f9f5413f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:19.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f59d2d640 1 --2- 172.21.15.60:0/3832594452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f54157e50 0x7f9f541582d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9f440115c0 tx=0x7f9f44011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f4affd640 1 -- 172.21.15.60:0/3832594452 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44019070 con 0x7f9f54157e50 2024-04-25T10:05:19.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f4affd640 1 -- 172.21.15.60:0/3832594452 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f44003620 con 0x7f9f54157e50 2024-04-25T10:05:19.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 -- 172.21.15.60:0/3832594452 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5413fd90 con 0x7f9f54157e50 2024-04-25T10:05:19.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f4affd640 1 -- 172.21.15.60:0/3832594452 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44003a10 con 0x7f9f54157e50 2024-04-25T10:05:19.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 -- 172.21.15.60:0/3832594452 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f54140350 con 0x7f9f54157e50 2024-04-25T10:05:19.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.195+0000 7f9f5c792640 1 -- 172.21.15.60:0/3832594452 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f54074770 con 0x7f9f54157e50 2024-04-25T10:05:19.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.199+0000 7f9f4affd640 1 -- 172.21.15.60:0/3832594452 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f4401c070 con 0x7f9f54157e50 2024-04-25T10:05:19.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.199+0000 7f9f4affd640 1 --2- 172.21.15.60:0/3832594452 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9f38066f90 0x7f9f38069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.199+0000 7f9f4affd640 1 -- 172.21.15.60:0/3832594452 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9f44085540 con 0x7f9f54157e50 2024-04-25T10:05:19.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.199+0000 7f9f5a52e640 1 --2- 172.21.15.60:0/3832594452 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9f38066f90 0x7f9f38069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.199+0000 7f9f5a52e640 1 --2- 172.21.15.60:0/3832594452 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9f38066f90 0x7f9f38069450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f50001dd0 tx=0x7f9f50009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.203+0000 7f9f4affd640 1 -- 172.21.15.60:0/3832594452 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f440568a0 con 0x7f9f54157e50 2024-04-25T10:05:19.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:19.363+0000 7f9f5c792640 1 -- 172.21.15.60:0/3832594452 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9f540d0970 con 0x7f9f54157e50 2024-04-25T10:05:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.031+0000 7f9f4affd640 1 -- 172.21.15.60:0/3832594452 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9f440561a0 con 0x7f9f54157e50 2024-04-25T10:05:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T10:05:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.035+0000 7f9f5c792640 1 -- 172.21.15.60:0/3832594452 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9f38066f90 msgr2=0x7f9f38069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.035+0000 7f9f5c792640 1 --2- 172.21.15.60:0/3832594452 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9f38066f90 0x7f9f38069450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f50001dd0 tx=0x7f9f50009040 comp rx=0 tx=0).stop 2024-04-25T10:05:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.035+0000 7f9f5c792640 1 -- 172.21.15.60:0/3832594452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f54157e50 msgr2=0x7f9f541582d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.035+0000 7f9f5c792640 1 --2- 172.21.15.60:0/3832594452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f54157e50 0x7f9f541582d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9f440115c0 tx=0x7f9f44011a80 comp rx=0 tx=0).stop 2024-04-25T10:05:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.035+0000 7f9f5c792640 1 -- 172.21.15.60:0/3832594452 shutdown_connections 2024-04-25T10:05:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.035+0000 7f9f5c792640 1 --2- 172.21.15.60:0/3832594452 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9f38066f90 0x7f9f38069450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.035+0000 7f9f5c792640 1 --2- 172.21.15.60:0/3832594452 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f540cbad0 0x7f9f540cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.035+0000 7f9f5c792640 1 --2- 172.21.15.60:0/3832594452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f54157e50 0x7f9f541582d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.035+0000 7f9f5c792640 1 --2- 172.21.15.60:0/3832594452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f541f6b80 0x7f9f5413f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.035+0000 7f9f5c792640 1 -- 172.21.15.60:0/3832594452 >> 172.21.15.60:0/3832594452 conn(0x7f9f540d04c0 msgr2=0x7f9f540c7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.035+0000 7f9f5c792640 1 -- 172.21.15.60:0/3832594452 shutdown_connections 2024-04-25T10:05:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.035+0000 7f9f5c792640 1 -- 172.21.15.60:0/3832594452 wait complete. 2024-04-25T10:05:20.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:05:20.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.191+0000 7f9dcd91e640 1 Processor -- start 2024-04-25T10:05:20.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.191+0000 7f9dcd91e640 1 -- start start 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dcd91e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9dc815fbf0 0x7f9dc8166870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dcd91e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc8144a40 con 0x7f9dc8160080 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dcd91e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc80cc9f0 con 0x7f9dc80cbcf0 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dcd91e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc80b2dc0 con 0x7f9dc815fbf0 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc67fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9dc8160080 0x7f9dc815f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55714/0 (socket says 172.21.15.60:55714) 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc67fc640 1 -- 172.21.15.60:0/1048611211 learned_addr learned my addr 172.21.15.60:0/1048611211 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc77fe640 1 --2- 172.21.15.60:0/1048611211 >> v2:172.21.15.100:3300/0 conn(0x7f9dc815fbf0 0x7f9dc8166870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc5ffb640 1 -- 172.21.15.60:0/1048611211 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705194929 0 0) 0x7f9dc8144a40 con 0x7f9dc8160080 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc5ffb640 1 -- 172.21.15.60:0/1048611211 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9db0003530 con 0x7f9dc8160080 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc5ffb640 1 -- 172.21.15.60:0/1048611211 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2533887335 0 0) 0x7f9db0003530 con 0x7f9dc8160080 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc5ffb640 1 -- 172.21.15.60:0/1048611211 >> v2:172.21.15.100:3300/0 conn(0x7f9dc815fbf0 msgr2=0x7f9dc8166870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc5ffb640 1 --2- 172.21.15.60:0/1048611211 >> v2:172.21.15.100:3300/0 conn(0x7f9dc815fbf0 0x7f9dc8166870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc5ffb640 1 -- 172.21.15.60:0/1048611211 >> v1:172.21.15.100:6789/0 conn(0x7f9dc80cbcf0 legacy=0x7f9dc8071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc5ffb640 1 -- 172.21.15.60:0/1048611211 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc80cccd0 con 0x7f9dc8160080 2024-04-25T10:05:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc5ffb640 1 -- 172.21.15.60:0/1048611211 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f9dbc001680 con 0x7f9dc8160080 2024-04-25T10:05:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc5ffb640 1 -- 172.21.15.60:0/1048611211 >> v1:172.21.15.60:6789/0 conn(0x7f9dc8160080 legacy=0x7f9dc815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc5ffb640 1 --2- 172.21.15.60:0/1048611211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db00039c0 0x7f9db0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc5ffb640 1 --2- 172.21.15.60:0/1048611211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9db0004810 0x7f9db0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc5ffb640 1 --2- 172.21.15.60:0/1048611211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9db00051d0 0x7f9db000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc5ffb640 1 -- 172.21.15.60:0/1048611211 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db000c040 con 0x7f9db0004810 2024-04-25T10:05:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc5ffb640 1 -- 172.21.15.60:0/1048611211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db000c1f0 con 0x7f9db00039c0 2024-04-25T10:05:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc5ffb640 1 -- 172.21.15.60:0/1048611211 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db000c3f0 con 0x7f9db00051d0 2024-04-25T10:05:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc67fc640 1 --2- 172.21.15.60:0/1048611211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9db0004810 0x7f9db0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc6ffd640 1 --2- 172.21.15.60:0/1048611211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db00039c0 0x7f9db0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc77fe640 1 --2- 172.21.15.60:0/1048611211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9db00051d0 0x7f9db000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc67fc640 1 -- 172.21.15.60:0/1048611211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9db00051d0 msgr2=0x7f9db000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc67fc640 1 --2- 172.21.15.60:0/1048611211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9db00051d0 0x7f9db000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc67fc640 1 -- 172.21.15.60:0/1048611211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db00039c0 msgr2=0x7f9db0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc67fc640 1 --2- 172.21.15.60:0/1048611211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db00039c0 0x7f9db0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc67fc640 1 -- 172.21.15.60:0/1048611211 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db0003530 con 0x7f9db0004810 2024-04-25T10:05:20.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc77fe640 1 --2- 172.21.15.60:0/1048611211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9db00051d0 0x7f9db000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:20.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.195+0000 7f9dc77fe640 1 -- 172.21.15.60:0/1048611211 reap_dead start 2024-04-25T10:05:20.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dc67fc640 1 --2- 172.21.15.60:0/1048611211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9db0004810 0x7f9db0004c90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00c1c0 tx=0x7f9dbc01f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da183ec1732298a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dc5ffb640 1 -- 172.21.15.60:0/1048611211 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc007d00 con 0x7f9db0004810 2024-04-25T10:05:20.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dc5ffb640 1 -- 172.21.15.60:0/1048611211 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc0ac020 con 0x7f9db0004810 2024-04-25T10:05:20.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dc5ffb640 1 -- 172.21.15.60:0/1048611211 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc0058e0 con 0x7f9db0004810 2024-04-25T10:05:20.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dcd91e640 1 -- 172.21.15.60:0/1048611211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9db0004810 msgr2=0x7f9db0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dcd91e640 1 --2- 172.21.15.60:0/1048611211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9db0004810 0x7f9db0004c90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00c1c0 tx=0x7f9dbc01f2d0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dcd91e640 1 -- 172.21.15.60:0/1048611211 shutdown_connections 2024-04-25T10:05:20.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dcd91e640 1 --2- 172.21.15.60:0/1048611211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9db0004810 0x7f9db0004c90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dcd91e640 1 -- 172.21.15.60:0/1048611211 >> 172.21.15.60:0/1048611211 conn(0x7f9dc80670b0 msgr2=0x7f9dc806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dcd91e640 1 -- 172.21.15.60:0/1048611211 shutdown_connections 2024-04-25T10:05:20.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dcd91e640 1 -- 172.21.15.60:0/1048611211 wait complete. 2024-04-25T10:05:20.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dcd91e640 1 Processor -- start 2024-04-25T10:05:20.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dcd91e640 1 -- start start 2024-04-25T10:05:20.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dcd91e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db0004810 0x7f9dc814d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dcd91e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dc8150880 0x7f9dc815e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dcd91e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc813bb40 0x7f9dc813bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dcd91e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80cc0d0 con 0x7f9dc813bb40 2024-04-25T10:05:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dcd91e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80ce9a0 con 0x7f9db0004810 2024-04-25T10:05:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dcd91e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80ce5c0 con 0x7f9dc8150880 2024-04-25T10:05:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dc77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc813bb40 0x7f9dc813bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dc77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc813bb40 0x7f9dc813bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45082/0 (socket says 172.21.15.60:45082) 2024-04-25T10:05:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dc77fe640 1 -- 172.21.15.60:0/1360045268 learned_addr learned my addr 172.21.15.60:0/1360045268 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dc67fc640 1 --2- 172.21.15.60:0/1360045268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dc8150880 0x7f9dc815e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dc6ffd640 1 --2- 172.21.15.60:0/1360045268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db0004810 0x7f9dc814d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dc77fe640 1 -- 172.21.15.60:0/1360045268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dc8150880 msgr2=0x7f9dc815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dc77fe640 1 --2- 172.21.15.60:0/1360045268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dc8150880 0x7f9dc815e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dc77fe640 1 -- 172.21.15.60:0/1360045268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db0004810 msgr2=0x7f9dc814d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dc77fe640 1 --2- 172.21.15.60:0/1360045268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db0004810 0x7f9dc814d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dc77fe640 1 -- 172.21.15.60:0/1360045268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc814dc30 con 0x7f9dc813bb40 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dc6ffd640 1 --2- 172.21.15.60:0/1360045268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db0004810 0x7f9dc814d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.199+0000 7f9dc77fe640 1 --2- 172.21.15.60:0/1360045268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc813bb40 0x7f9dc813bfb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80cde50 tx=0x7f9dc000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.203+0000 7f9daffff640 1 -- 172.21.15.60:0/1360045268 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0016620 con 0x7f9dc813bb40 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.203+0000 7f9daffff640 1 -- 172.21.15.60:0/1360045268 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc001b070 con 0x7f9dc813bb40 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.203+0000 7f9dcd91e640 1 -- 172.21.15.60:0/1360045268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc813c4f0 con 0x7f9dc813bb40 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.203+0000 7f9daffff640 1 -- 172.21.15.60:0/1360045268 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc00169c0 con 0x7f9dc813bb40 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.203+0000 7f9dcd91e640 1 -- 172.21.15.60:0/1360045268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc81f4e10 con 0x7f9dc813bb40 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.203+0000 7f9dcd91e640 1 -- 172.21.15.60:0/1360045268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc806bba0 con 0x7f9dc813bb40 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.203+0000 7f9daffff640 1 -- 172.21.15.60:0/1360045268 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9dc0016b60 con 0x7f9dc813bb40 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.203+0000 7f9daffff640 1 --2- 172.21.15.60:0/1360045268 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9db4066f90 0x7f9db4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.203+0000 7f9daffff640 1 -- 172.21.15.60:0/1360045268 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f9dc0085c70 con 0x7f9dc813bb40 2024-04-25T10:05:20.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.203+0000 7f9dc6ffd640 1 --2- 172.21.15.60:0/1360045268 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9db4066f90 0x7f9db4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.203+0000 7f9dc6ffd640 1 --2- 172.21.15.60:0/1360045268 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9db4066f90 0x7f9db4069450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9db8007fd0 tx=0x7f9db8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.207+0000 7f9daffff640 1 -- 172.21.15.60:0/1360045268 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9dc00122d0 con 0x7f9dc813bb40 2024-04-25T10:05:20.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:20.403+0000 7f9dcd91e640 1 -- 172.21.15.60:0/1360045268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9dc80b2970 con 0x7f9dc813bb40 2024-04-25T10:05:21.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.063+0000 7f9daffff640 1 -- 172.21.15.60:0/1360045268 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9dc0056ce0 con 0x7f9dc813bb40 2024-04-25T10:05:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T10:05:21.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.075+0000 7f9dcd91e640 1 -- 172.21.15.60:0/1360045268 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9db4066f90 msgr2=0x7f9db4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.075+0000 7f9dcd91e640 1 --2- 172.21.15.60:0/1360045268 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9db4066f90 0x7f9db4069450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9db8007fd0 tx=0x7f9db8009040 comp rx=0 tx=0).stop 2024-04-25T10:05:21.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.075+0000 7f9dcd91e640 1 -- 172.21.15.60:0/1360045268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc813bb40 msgr2=0x7f9dc813bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.075+0000 7f9dcd91e640 1 --2- 172.21.15.60:0/1360045268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc813bb40 0x7f9dc813bfb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80cde50 tx=0x7f9dc000f520 comp rx=0 tx=0).stop 2024-04-25T10:05:21.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.075+0000 7f9dcd91e640 1 -- 172.21.15.60:0/1360045268 shutdown_connections 2024-04-25T10:05:21.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.075+0000 7f9dcd91e640 1 --2- 172.21.15.60:0/1360045268 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9db4066f90 0x7f9db4069450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.075+0000 7f9dcd91e640 1 --2- 172.21.15.60:0/1360045268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc813bb40 0x7f9dc813bfb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.075+0000 7f9dcd91e640 1 --2- 172.21.15.60:0/1360045268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dc8150880 0x7f9dc815e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.075+0000 7f9dcd91e640 1 --2- 172.21.15.60:0/1360045268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db0004810 0x7f9dc814d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.075+0000 7f9dcd91e640 1 -- 172.21.15.60:0/1360045268 >> 172.21.15.60:0/1360045268 conn(0x7f9dc80670b0 msgr2=0x7f9dc806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.075+0000 7f9dcd91e640 1 -- 172.21.15.60:0/1360045268 shutdown_connections 2024-04-25T10:05:21.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.075+0000 7f9dcd91e640 1 -- 172.21.15.60:0/1360045268 wait complete. 2024-04-25T10:05:21.113 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.223+0000 7f91acb24640 1 Processor -- start 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.223+0000 7f91acb24640 1 -- start start 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.223+0000 7f91acb24640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f91a80d0a50 0x7f91a80b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.223+0000 7f91acb24640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91a80ce2b0 0x7f91a80ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.223+0000 7f91acb24640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91a806d200 con 0x7f91a815deb0 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.223+0000 7f91acb24640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91a80cf2c0 con 0x7f91a80d0a50 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.223+0000 7f91acb24640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a80cefe0 con 0x7f91a80ce2b0 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.223+0000 7f91a659c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f91a815deb0 0x7f91a80d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55716/0 (socket says 172.21.15.60:55716) 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.223+0000 7f91a659c640 1 -- 172.21.15.60:0/1902903567 learned_addr learned my addr 172.21.15.60:0/1902903567 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.223+0000 7f91a5d9b640 1 --2- 172.21.15.60:0/1902903567 >> v2:172.21.15.100:3300/0 conn(0x7f91a80d0a50 0x7f91a80b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.223+0000 7f91a559a640 1 -- 172.21.15.60:0/1902903567 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568835188 0 0) 0x7f91a806d200 con 0x7f91a815deb0 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.223+0000 7f91a6d9d640 1 --2- 172.21.15.60:0/1902903567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91a80ce2b0 0x7f91a80ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.223+0000 7f91a559a640 1 -- 172.21.15.60:0/1902903567 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9190003530 con 0x7f91a815deb0 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 -- 172.21.15.60:0/1902903567 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 458225231 0 0) 0x7f9190003530 con 0x7f91a815deb0 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 -- 172.21.15.60:0/1902903567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91a80ce2b0 msgr2=0x7f91a80ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 --2- 172.21.15.60:0/1902903567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91a80ce2b0 0x7f91a80ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 -- 172.21.15.60:0/1902903567 >> v2:172.21.15.100:3300/0 conn(0x7f91a80d0a50 msgr2=0x7f91a80b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 --2- 172.21.15.60:0/1902903567 >> v2:172.21.15.100:3300/0 conn(0x7f91a80d0a50 0x7f91a80b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 -- 172.21.15.60:0/1902903567 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a80d1d70 con 0x7f91a815deb0 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 -- 172.21.15.60:0/1902903567 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f9198001680 con 0x7f91a815deb0 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 -- 172.21.15.60:0/1902903567 >> v1:172.21.15.60:6789/0 conn(0x7f91a815deb0 legacy=0x7f91a80d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 --2- 172.21.15.60:0/1902903567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91900039c0 0x7f9190003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 --2- 172.21.15.60:0/1902903567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9190004810 0x7f9190004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 --2- 172.21.15.60:0/1902903567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91a80ce2b0 0x7f91900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 -- 172.21.15.60:0/1902903567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a80cf2c0 con 0x7f9190004810 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 -- 172.21.15.60:0/1902903567 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a80cefe0 con 0x7f91900039c0 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 -- 172.21.15.60:0/1902903567 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919000b9d0 con 0x7f91a80ce2b0 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a5d9b640 1 --2- 172.21.15.60:0/1902903567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9190004810 0x7f9190004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a659c640 1 --2- 172.21.15.60:0/1902903567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91900039c0 0x7f9190003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a6d9d640 1 --2- 172.21.15.60:0/1902903567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91a80ce2b0 0x7f91900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a5d9b640 1 -- 172.21.15.60:0/1902903567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91a80ce2b0 msgr2=0x7f91900051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a5d9b640 1 --2- 172.21.15.60:0/1902903567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91a80ce2b0 0x7f91900051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a5d9b640 1 -- 172.21.15.60:0/1902903567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91900039c0 msgr2=0x7f9190003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a5d9b640 1 --2- 172.21.15.60:0/1902903567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91900039c0 0x7f9190003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a5d9b640 1 -- 172.21.15.60:0/1902903567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9190003530 con 0x7f9190004810 2024-04-25T10:05:21.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a5d9b640 1 --2- 172.21.15.60:0/1902903567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9190004810 0x7f9190004c90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f919c009ae0 tx=0x7f919c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950d4a5162e1947e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 -- 172.21.15.60:0/1902903567 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c0096b0 con 0x7f9190004810 2024-04-25T10:05:21.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 -- 172.21.15.60:0/1902903567 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919c0a64c0 con 0x7f9190004810 2024-04-25T10:05:21.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a559a640 1 -- 172.21.15.60:0/1902903567 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c0a68b0 con 0x7f9190004810 2024-04-25T10:05:21.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91acb24640 1 -- 172.21.15.60:0/1902903567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9190004810 msgr2=0x7f9190004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91acb24640 1 --2- 172.21.15.60:0/1902903567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9190004810 0x7f9190004c90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f919c009ae0 tx=0x7f919c01cab0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91a6d9d640 1 -- 172.21.15.60:0/1902903567 reap_dead start 2024-04-25T10:05:21.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91acb24640 1 -- 172.21.15.60:0/1902903567 shutdown_connections 2024-04-25T10:05:21.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91acb24640 1 -- 172.21.15.60:0/1902903567 >> 172.21.15.60:0/1902903567 conn(0x7f91a806c090 msgr2=0x7f91a806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91acb24640 1 -- 172.21.15.60:0/1902903567 shutdown_connections 2024-04-25T10:05:21.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.227+0000 7f91acb24640 1 -- 172.21.15.60:0/1902903567 wait complete. 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91acb24640 1 Processor -- start 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91acb24640 1 -- start start 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91acb24640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9190004810 0x7f91a81f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91acb24640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91a80caa00 0x7f91a815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91acb24640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91a80cae80 0x7f91a815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91acb24640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a80ce760 con 0x7f91a80caa00 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91acb24640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a80d2330 con 0x7f91a80cae80 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91acb24640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a80ce8e0 con 0x7f9190004810 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91a5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91a80caa00 0x7f91a815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91a5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91a80caa00 0x7f91a815ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45094/0 (socket says 172.21.15.60:45094) 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91a5d9b640 1 -- 172.21.15.60:0/2203841813 learned_addr learned my addr 172.21.15.60:0/2203841813 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:21.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91a6d9d640 1 --2- 172.21.15.60:0/2203841813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91a80cae80 0x7f91a815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91a659c640 1 --2- 172.21.15.60:0/2203841813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9190004810 0x7f91a81f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91a5d9b640 1 -- 172.21.15.60:0/2203841813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9190004810 msgr2=0x7f91a81f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91a5d9b640 1 --2- 172.21.15.60:0/2203841813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9190004810 0x7f91a81f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91a5d9b640 1 -- 172.21.15.60:0/2203841813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91a80cae80 msgr2=0x7f91a815bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91a5d9b640 1 --2- 172.21.15.60:0/2203841813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91a80cae80 0x7f91a815bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91a5d9b640 1 -- 172.21.15.60:0/2203841813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a80cbf90 con 0x7f91a80caa00 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91a5d9b640 1 --2- 172.21.15.60:0/2203841813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91a80caa00 0x7f91a815ba50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f919c009ae0 tx=0x7f9190005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f9196ffd640 1 -- 172.21.15.60:0/2203841813 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c004020 con 0x7f91a80caa00 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f9196ffd640 1 -- 172.21.15.60:0/2203841813 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919c0041c0 con 0x7f91a80caa00 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f9196ffd640 1 -- 172.21.15.60:0/2203841813 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c0043b0 con 0x7f91a80caa00 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91acb24640 1 -- 172.21.15.60:0/2203841813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a80cb410 con 0x7f91a80caa00 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91acb24640 1 -- 172.21.15.60:0/2203841813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a80cb9d0 con 0x7f91a80caa00 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.231+0000 7f91acb24640 1 -- 172.21.15.60:0/2203841813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a8074770 con 0x7f91a80caa00 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.235+0000 7f9196ffd640 1 -- 172.21.15.60:0/2203841813 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f919c0c1020 con 0x7f91a80caa00 2024-04-25T10:05:21.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.235+0000 7f9196ffd640 1 --2- 172.21.15.60:0/2203841813 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9188066f90 0x7f9188069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.235+0000 7f9196ffd640 1 -- 172.21.15.60:0/2203841813 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f919c11f8c0 con 0x7f91a80caa00 2024-04-25T10:05:21.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.235+0000 7f91a659c640 1 --2- 172.21.15.60:0/2203841813 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9188066f90 0x7f9188069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.235+0000 7f91a659c640 1 --2- 172.21.15.60:0/2203841813 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9188066f90 0x7f9188069450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9198005260 tx=0x7f9198009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.239+0000 7f9196ffd640 1 -- 172.21.15.60:0/2203841813 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f919c0167c0 con 0x7f91a80caa00 2024-04-25T10:05:21.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.399+0000 7f91acb24640 1 -- 172.21.15.60:0/2203841813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91a80ccbf0 con 0x7f91a80caa00 2024-04-25T10:05:21.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.403+0000 7f9196ffd640 1 -- 172.21.15.60:0/2203841813 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16630 (secure 0 0 0) 0x7f919c0f0830 con 0x7f91a80caa00 2024-04-25T10:05:21.410 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:21.410 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:20.405012+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:21.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.403+0000 7f91acb24640 1 -- 172.21.15.60:0/2203841813 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9188066f90 msgr2=0x7f9188069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.403+0000 7f91acb24640 1 --2- 172.21.15.60:0/2203841813 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9188066f90 0x7f9188069450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9198005260 tx=0x7f9198009040 comp rx=0 tx=0).stop 2024-04-25T10:05:21.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.403+0000 7f91acb24640 1 -- 172.21.15.60:0/2203841813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91a80caa00 msgr2=0x7f91a815ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.403+0000 7f91acb24640 1 --2- 172.21.15.60:0/2203841813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91a80caa00 0x7f91a815ba50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f919c009ae0 tx=0x7f9190005840 comp rx=0 tx=0).stop 2024-04-25T10:05:21.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.403+0000 7f91acb24640 1 -- 172.21.15.60:0/2203841813 shutdown_connections 2024-04-25T10:05:21.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.403+0000 7f91acb24640 1 --2- 172.21.15.60:0/2203841813 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9188066f90 0x7f9188069450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.403+0000 7f91acb24640 1 --2- 172.21.15.60:0/2203841813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91a80cae80 0x7f91a815bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.403+0000 7f91acb24640 1 --2- 172.21.15.60:0/2203841813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91a80caa00 0x7f91a815ba50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.403+0000 7f91acb24640 1 --2- 172.21.15.60:0/2203841813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9190004810 0x7f91a81f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.403+0000 7f91acb24640 1 -- 172.21.15.60:0/2203841813 >> 172.21.15.60:0/2203841813 conn(0x7f91a806c090 msgr2=0x7f91a815cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.403+0000 7f91acb24640 1 -- 172.21.15.60:0/2203841813 shutdown_connections 2024-04-25T10:05:21.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.403+0000 7f91acb24640 1 -- 172.21.15.60:0/2203841813 wait complete. 2024-04-25T10:05:21.429 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:21.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.543+0000 7f00c023a640 1 Processor -- start 2024-04-25T10:05:21.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.543+0000 7f00c023a640 1 -- start start 2024-04-25T10:05:21.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.543+0000 7f00c023a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f00b815dea0 0x7f00b806c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.543+0000 7f00c023a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f00b80ce9d0 0x7f00b80cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.543+0000 7f00c023a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b806c680 con 0x7f00b80ce4f0 2024-04-25T10:05:21.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.543+0000 7f00c023a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b815e280 con 0x7f00b815dea0 2024-04-25T10:05:21.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.543+0000 7f00c023a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b80cf380 con 0x7f00b80ce9d0 2024-04-25T10:05:21.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.543+0000 7f00bdfd6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f00b815dea0 0x7f00b806c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.543+0000 7f00bd7d5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f00b80ce4f0 0x7f00b80d2bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55724/0 (socket says 172.21.15.60:55724) 2024-04-25T10:05:21.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.543+0000 7f00bd7d5640 1 -- 172.21.15.60:0/1554640077 learned_addr learned my addr 172.21.15.60:0/1554640077 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:21.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.543+0000 7f00be7d7640 1 --2- 172.21.15.60:0/1554640077 >> v2:172.21.15.100:3300/0 conn(0x7f00b80ce9d0 0x7f00b80cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.543+0000 7f00bdfd6640 1 -- 172.21.15.60:0/1554640077 >> v2:172.21.15.100:3300/0 conn(0x7f00b80ce9d0 msgr2=0x7f00b80cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.543+0000 7f00bdfd6640 1 --2- 172.21.15.60:0/1554640077 >> v2:172.21.15.100:3300/0 conn(0x7f00b80ce9d0 0x7f00b80cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.543+0000 7f00bdfd6640 1 -- 172.21.15.60:0/1554640077 >> v1:172.21.15.60:6789/0 conn(0x7f00b80ce4f0 legacy=0x7f00b80d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.543+0000 7f00bdfd6640 1 -- 172.21.15.60:0/1554640077 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b806c930 con 0x7f00b815dea0 2024-04-25T10:05:21.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.543+0000 7f00be7d7640 1 --2- 172.21.15.60:0/1554640077 >> v2:172.21.15.100:3300/0 conn(0x7f00b80ce9d0 0x7f00b80cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:21.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00bdfd6640 1 --2- 172.21.15.60:0/1554640077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b815dea0 0x7f00b806c140 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f00b80b9840 tx=0x7f00b401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55eb7122d0f4fbd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00bcfd4640 1 -- 172.21.15.60:0/1554640077 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b40b0070 con 0x7f00b815dea0 2024-04-25T10:05:21.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00bcfd4640 1 -- 172.21.15.60:0/1554640077 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b40ab600 con 0x7f00b815dea0 2024-04-25T10:05:21.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00bcfd4640 1 -- 172.21.15.60:0/1554640077 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b40ab9d0 con 0x7f00b815dea0 2024-04-25T10:05:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00c023a640 1 -- 172.21.15.60:0/1554640077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b815dea0 msgr2=0x7f00b806c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00c023a640 1 --2- 172.21.15.60:0/1554640077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b815dea0 0x7f00b806c140 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f00b80b9840 tx=0x7f00b401c480 comp rx=0 tx=0).stop 2024-04-25T10:05:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00c023a640 1 -- 172.21.15.60:0/1554640077 shutdown_connections 2024-04-25T10:05:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00c023a640 1 --2- 172.21.15.60:0/1554640077 >> v2:172.21.15.100:3300/0 conn(0x7f00b80ce9d0 0x7f00b80cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00c023a640 1 --2- 172.21.15.60:0/1554640077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b815dea0 0x7f00b806c140 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00c023a640 1 -- 172.21.15.60:0/1554640077 >> 172.21.15.60:0/1554640077 conn(0x7f00b80d0480 msgr2=0x7f00b80d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00c023a640 1 -- 172.21.15.60:0/1554640077 shutdown_connections 2024-04-25T10:05:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00c023a640 1 -- 172.21.15.60:0/1554640077 wait complete. 2024-04-25T10:05:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00c023a640 1 Processor -- start 2024-04-25T10:05:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00c023a640 1 -- start start 2024-04-25T10:05:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00c023a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b80ce4f0 0x7f00b80b8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00c023a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00b80ce9d0 0x7f00b80b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00c023a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00b815dea0 0x7f00b80cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00c023a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b80b7620 con 0x7f00b80ce4f0 2024-04-25T10:05:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00c023a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b80d32e0 con 0x7f00b80ce9d0 2024-04-25T10:05:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00c023a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b80b77a0 con 0x7f00b815dea0 2024-04-25T10:05:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00bdfd6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b80ce4f0 0x7f00b80b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00bdfd6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b80ce4f0 0x7f00b80b8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45110/0 (socket says 172.21.15.60:45110) 2024-04-25T10:05:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00bdfd6640 1 -- 172.21.15.60:0/979310704 learned_addr learned my addr 172.21.15.60:0/979310704 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00bdfd6640 1 -- 172.21.15.60:0/979310704 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00b815dea0 msgr2=0x7f00b80cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00bdfd6640 1 --2- 172.21.15.60:0/979310704 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00b815dea0 0x7f00b80cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00bd7d5640 1 --2- 172.21.15.60:0/979310704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00b80ce9d0 0x7f00b80b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00bdfd6640 1 -- 172.21.15.60:0/979310704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00b80ce9d0 msgr2=0x7f00b80b87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00bdfd6640 1 --2- 172.21.15.60:0/979310704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00b80ce9d0 0x7f00b80b87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00bdfd6640 1 -- 172.21.15.60:0/979310704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b80cac90 con 0x7f00b80ce4f0 2024-04-25T10:05:21.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00bdfd6640 1 --2- 172.21.15.60:0/979310704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b80ce4f0 0x7f00b80b8280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f00b40ae040 tx=0x7f00b401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.547+0000 7f00ae7fc640 1 -- 172.21.15.60:0/979310704 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b40b0050 con 0x7f00b80ce4f0 2024-04-25T10:05:21.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.551+0000 7f00ae7fc640 1 -- 172.21.15.60:0/979310704 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b40bc070 con 0x7f00b80ce4f0 2024-04-25T10:05:21.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.551+0000 7f00c023a640 1 -- 172.21.15.60:0/979310704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b80caf70 con 0x7f00b80ce4f0 2024-04-25T10:05:21.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.551+0000 7f00ae7fc640 1 -- 172.21.15.60:0/979310704 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b40b5a70 con 0x7f00b80ce4f0 2024-04-25T10:05:21.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.551+0000 7f00c023a640 1 -- 172.21.15.60:0/979310704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b80cb530 con 0x7f00b80ce4f0 2024-04-25T10:05:21.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.551+0000 7f00c023a640 1 -- 172.21.15.60:0/979310704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b8074770 con 0x7f00b80ce4f0 2024-04-25T10:05:21.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.551+0000 7f00ae7fc640 1 -- 172.21.15.60:0/979310704 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00b40ab480 con 0x7f00b80ce4f0 2024-04-25T10:05:21.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.551+0000 7f00ae7fc640 1 --2- 172.21.15.60:0/979310704 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f009c066f60 0x7f009c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.551+0000 7f00ae7fc640 1 -- 172.21.15.60:0/979310704 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00b411f910 con 0x7f00b80ce4f0 2024-04-25T10:05:21.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.551+0000 7f00bd7d5640 1 --2- 172.21.15.60:0/979310704 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f009c066f60 0x7f009c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.551+0000 7f00bd7d5640 1 --2- 172.21.15.60:0/979310704 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f009c066f60 0x7f009c069420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f00b815f4c0 tx=0x7f00a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.555+0000 7f00ae7fc640 1 -- 172.21.15.60:0/979310704 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f00b4016070 con 0x7f00b80ce4f0 2024-04-25T10:05:21.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.715+0000 7f00c023a640 1 -- 172.21.15.60:0/979310704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00b80d15e0 con 0x7f00b80ce4f0 2024-04-25T10:05:21.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.715+0000 7f00ae7fc640 1 -- 172.21.15.60:0/979310704 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16630 (secure 0 0 0) 0x7f00b40f09b0 con 0x7f00b80ce4f0 2024-04-25T10:05:21.724 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:21.724 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:20.405012+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:21.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.719+0000 7f00c023a640 1 -- 172.21.15.60:0/979310704 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f009c066f60 msgr2=0x7f009c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.719+0000 7f00c023a640 1 --2- 172.21.15.60:0/979310704 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f009c066f60 0x7f009c069420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f00b815f4c0 tx=0x7f00a8009040 comp rx=0 tx=0).stop 2024-04-25T10:05:21.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.719+0000 7f00c023a640 1 -- 172.21.15.60:0/979310704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b80ce4f0 msgr2=0x7f00b80b8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.719+0000 7f00c023a640 1 --2- 172.21.15.60:0/979310704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b80ce4f0 0x7f00b80b8280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f00b40ae040 tx=0x7f00b401acc0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.719+0000 7f00c023a640 1 -- 172.21.15.60:0/979310704 shutdown_connections 2024-04-25T10:05:21.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.719+0000 7f00c023a640 1 --2- 172.21.15.60:0/979310704 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f009c066f60 0x7f009c069420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.719+0000 7f00c023a640 1 --2- 172.21.15.60:0/979310704 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00b815dea0 0x7f00b80cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.719+0000 7f00c023a640 1 --2- 172.21.15.60:0/979310704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00b80ce9d0 0x7f00b80b87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.719+0000 7f00c023a640 1 --2- 172.21.15.60:0/979310704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b80ce4f0 0x7f00b80b8280 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.719+0000 7f00c023a640 1 -- 172.21.15.60:0/979310704 >> 172.21.15.60:0/979310704 conn(0x7f00b80d0480 msgr2=0x7f00b815fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.719+0000 7f00c023a640 1 -- 172.21.15.60:0/979310704 shutdown_connections 2024-04-25T10:05:21.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.719+0000 7f00c023a640 1 -- 172.21.15.60:0/979310704 wait complete. 2024-04-25T10:05:21.743 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:05:21.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2d1bb5640 1 Processor -- start 2024-04-25T10:05:21.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2d1bb5640 1 -- start start 2024-04-25T10:05:21.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2d1bb5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc2cc15dbf0 0x7fc2cc070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2d1bb5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2cc071210 0x7fc2cc0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2d1bb5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc2cc0b4c10 0x7fc2cc1616d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2d1bb5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2cc0b58e0 con 0x7fc2cc0b4c10 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2d1bb5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2cc0b50a0 con 0x7fc2cc15dbf0 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2d1bb5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc15dfd0 con 0x7fc2cc071210 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2cbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc2cc0b4c10 0x7fc2cc1616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2cbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc2cc0b4c10 0x7fc2cc1616d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45120/0 (socket says 172.21.15.60:45120) 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2cbfff640 1 -- 172.21.15.60:0/2082684761 learned_addr learned my addr 172.21.15.60:0/2082684761 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2c2ffd640 1 --2- 172.21.15.60:0/2082684761 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2cc071210 0x7fc2cc0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2cbfff640 1 -- 172.21.15.60:0/2082684761 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2cc071210 msgr2=0x7fc2cc0b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2cb7fe640 1 --2- 172.21.15.60:0/2082684761 >> v2:172.21.15.100:3300/0 conn(0x7fc2cc15dbf0 0x7fc2cc070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2cbfff640 1 --2- 172.21.15.60:0/2082684761 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2cc071210 0x7fc2cc0b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2cbfff640 1 -- 172.21.15.60:0/2082684761 >> v2:172.21.15.100:3300/0 conn(0x7fc2cc15dbf0 msgr2=0x7fc2cc070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2cbfff640 1 --2- 172.21.15.60:0/2082684761 >> v2:172.21.15.100:3300/0 conn(0x7fc2cc15dbf0 0x7fc2cc070cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2cbfff640 1 -- 172.21.15.60:0/2082684761 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2cc0b5320 con 0x7fc2cc0b4c10 2024-04-25T10:05:21.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2c2ffd640 1 --2- 172.21.15.60:0/2082684761 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2cc071210 0x7fc2cc0b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:21.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2cb7fe640 1 --2- 172.21.15.60:0/2082684761 >> v2:172.21.15.100:3300/0 conn(0x7fc2cc15dbf0 0x7fc2cc070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:21.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.863+0000 7fc2cbfff640 1 --2- 172.21.15.60:0/2082684761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2cc0b4c10 0x7fc2cc1616d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc073060 tx=0x7fc2c401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cbc61b51486612 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2caffd640 1 -- 172.21.15.60:0/2082684761 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c40a55d0 con 0x7fc2cc0b4c10 2024-04-25T10:05:21.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2caffd640 1 -- 172.21.15.60:0/2082684761 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c40a5770 con 0x7fc2cc0b4c10 2024-04-25T10:05:21.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2caffd640 1 -- 172.21.15.60:0/2082684761 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c40a5b40 con 0x7fc2cc0b4c10 2024-04-25T10:05:21.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 -- 172.21.15.60:0/2082684761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2cc0b4c10 msgr2=0x7fc2cc1616d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 --2- 172.21.15.60:0/2082684761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2cc0b4c10 0x7fc2cc1616d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc073060 tx=0x7fc2c401c730 comp rx=0 tx=0).stop 2024-04-25T10:05:21.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 -- 172.21.15.60:0/2082684761 shutdown_connections 2024-04-25T10:05:21.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 --2- 172.21.15.60:0/2082684761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2cc0b4c10 0x7fc2cc1616d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 --2- 172.21.15.60:0/2082684761 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2cc071210 0x7fc2cc0b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 --2- 172.21.15.60:0/2082684761 >> v2:172.21.15.100:3300/0 conn(0x7fc2cc15dbf0 0x7fc2cc070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 -- 172.21.15.60:0/2082684761 >> 172.21.15.60:0/2082684761 conn(0x7fc2cc06f530 msgr2=0x7fc2cc06f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 -- 172.21.15.60:0/2082684761 shutdown_connections 2024-04-25T10:05:21.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 -- 172.21.15.60:0/2082684761 wait complete. 2024-04-25T10:05:21.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 Processor -- start 2024-04-25T10:05:21.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 -- start start 2024-04-25T10:05:21.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2cc071210 0x7fc2cc15d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2cc0b4c10 0x7fc2cc1512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2cc15dbf0 0x7fc2cc14e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc06c6a0 con 0x7fc2cc15dbf0 2024-04-25T10:05:21.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc06c180 con 0x7fc2cc0b4c10 2024-04-25T10:05:21.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc06c820 con 0x7fc2cc071210 2024-04-25T10:05:21.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2cbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2cc15dbf0 0x7fc2cc14e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2cbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2cc15dbf0 0x7fc2cc14e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45124/0 (socket says 172.21.15.60:45124) 2024-04-25T10:05:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2cbfff640 1 -- 172.21.15.60:0/1238783500 learned_addr learned my addr 172.21.15.60:0/1238783500 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2cb7fe640 1 --2- 172.21.15.60:0/1238783500 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2cc071210 0x7fc2cc15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2c2ffd640 1 --2- 172.21.15.60:0/1238783500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2cc0b4c10 0x7fc2cc1512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2cbfff640 1 -- 172.21.15.60:0/1238783500 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2cc071210 msgr2=0x7fc2cc15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2cbfff640 1 --2- 172.21.15.60:0/1238783500 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2cc071210 0x7fc2cc15d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2cbfff640 1 -- 172.21.15.60:0/1238783500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2cc0b4c10 msgr2=0x7fc2cc1512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2cbfff640 1 --2- 172.21.15.60:0/1238783500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2cc0b4c10 0x7fc2cc1512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2cbfff640 1 -- 172.21.15.60:0/1238783500 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2cc150540 con 0x7fc2cc15dbf0 2024-04-25T10:05:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2cbfff640 1 --2- 172.21.15.60:0/1238783500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2cc15dbf0 0x7fc2cc14e7c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc2c401ade0 tx=0x7fc2c40194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2c8ff9640 1 -- 172.21.15.60:0/1238783500 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c40bc020 con 0x7fc2cc15dbf0 2024-04-25T10:05:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2c8ff9640 1 -- 172.21.15.60:0/1238783500 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c40b8670 con 0x7fc2cc15dbf0 2024-04-25T10:05:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 -- 172.21.15.60:0/1238783500 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2cc14f290 con 0x7fc2cc15dbf0 2024-04-25T10:05:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2c8ff9640 1 -- 172.21.15.60:0/1238783500 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4009510 con 0x7fc2cc15dbf0 2024-04-25T10:05:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.867+0000 7fc2d1bb5640 1 -- 172.21.15.60:0/1238783500 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2cc1507e0 con 0x7fc2cc15dbf0 2024-04-25T10:05:21.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.871+0000 7fc2d1bb5640 1 -- 172.21.15.60:0/1238783500 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2cc074770 con 0x7fc2cc15dbf0 2024-04-25T10:05:21.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.871+0000 7fc2c8ff9640 1 -- 172.21.15.60:0/1238783500 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc2c40bf020 con 0x7fc2cc15dbf0 2024-04-25T10:05:21.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.871+0000 7fc2c8ff9640 1 --2- 172.21.15.60:0/1238783500 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2ac066f60 0x7fc2ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.871+0000 7fc2c8ff9640 1 -- 172.21.15.60:0/1238783500 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2c411ff30 con 0x7fc2cc15dbf0 2024-04-25T10:05:21.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.871+0000 7fc2cb7fe640 1 --2- 172.21.15.60:0/1238783500 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2ac066f60 0x7fc2ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.871+0000 7fc2cb7fe640 1 --2- 172.21.15.60:0/1238783500 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2ac066f60 0x7fc2ac069420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc006fd0 tx=0x7fc2bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:21.875+0000 7fc2c8ff9640 1 -- 172.21.15.60:0/1238783500 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc2c40c6030 con 0x7fc2cc15dbf0 2024-04-25T10:05:22.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:22.075+0000 7fc2d1bb5640 1 -- 172.21.15.60:0/1238783500 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc2cc0702d0 con 0x7fc2cc15dbf0 2024-04-25T10:05:23.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.083+0000 7fc2c8ff9640 1 -- 172.21.15.60:0/1238783500 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc2c40165e0 con 0x7fc2cc15dbf0 2024-04-25T10:05:23.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.083+0000 7fc2d1bb5640 1 -- 172.21.15.60:0/1238783500 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2ac066f60 msgr2=0x7fc2ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.083+0000 7fc2d1bb5640 1 --2- 172.21.15.60:0/1238783500 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2ac066f60 0x7fc2ac069420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc006fd0 tx=0x7fc2bc008040 comp rx=0 tx=0).stop 2024-04-25T10:05:23.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.083+0000 7fc2d1bb5640 1 -- 172.21.15.60:0/1238783500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2cc15dbf0 msgr2=0x7fc2cc14e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.083+0000 7fc2d1bb5640 1 --2- 172.21.15.60:0/1238783500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2cc15dbf0 0x7fc2cc14e7c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc2c401ade0 tx=0x7fc2c40194d0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.083+0000 7fc2d1bb5640 1 -- 172.21.15.60:0/1238783500 shutdown_connections 2024-04-25T10:05:23.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.083+0000 7fc2d1bb5640 1 --2- 172.21.15.60:0/1238783500 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2ac066f60 0x7fc2ac069420 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.083+0000 7fc2d1bb5640 1 --2- 172.21.15.60:0/1238783500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2cc15dbf0 0x7fc2cc14e7c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.083+0000 7fc2d1bb5640 1 --2- 172.21.15.60:0/1238783500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2cc0b4c10 0x7fc2cc1512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.083+0000 7fc2d1bb5640 1 --2- 172.21.15.60:0/1238783500 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2cc071210 0x7fc2cc15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.083+0000 7fc2d1bb5640 1 -- 172.21.15.60:0/1238783500 >> 172.21.15.60:0/1238783500 conn(0x7fc2cc06f530 msgr2=0x7fc2cc15b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.087+0000 7fc2d1bb5640 1 -- 172.21.15.60:0/1238783500 shutdown_connections 2024-04-25T10:05:23.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.087+0000 7fc2d1bb5640 1 -- 172.21.15.60:0/1238783500 wait complete. 2024-04-25T10:05:23.116 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:05:23.117 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:05:23.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.243+0000 7fb097a00640 1 Processor -- start 2024-04-25T10:05:23.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.243+0000 7fb097a00640 1 -- start start 2024-04-25T10:05:23.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.243+0000 7fb097a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0900cbcf0 0x7fb090160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.243+0000 7fb097a00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb09015fbf0 0x7fb0900ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.243+0000 7fb097a00640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb090144a40 con 0x7fb090071c30 2024-04-25T10:05:23.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.243+0000 7fb097a00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0900b2dc0 con 0x7fb09015fbf0 2024-04-25T10:05:23.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.243+0000 7fb097a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900cc9f0 con 0x7fb0900cbcf0 2024-04-25T10:05:23.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.243+0000 7fb095f9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb09015fbf0 0x7fb0900ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.243+0000 7fb09579c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0900cbcf0 0x7fb090160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.243+0000 7fb094f9b640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb090071c30 0x7fb09015f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:48626/0 (socket says 172.21.15.60:48626) 2024-04-25T10:05:23.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.243+0000 7fb094f9b640 1 -- 172.21.15.60:0/2897520371 learned_addr learned my addr 172.21.15.60:0/2897520371 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:23.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.243+0000 7fb09579c640 1 -- 172.21.15.60:0/2897520371 >> v2:172.21.15.100:3300/0 conn(0x7fb09015fbf0 msgr2=0x7fb0900ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.243+0000 7fb09579c640 1 --2- 172.21.15.60:0/2897520371 >> v2:172.21.15.100:3300/0 conn(0x7fb09015fbf0 0x7fb0900ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.243+0000 7fb09579c640 1 -- 172.21.15.60:0/2897520371 >> v1:172.21.15.100:6789/0 conn(0x7fb090071c30 legacy=0x7fb09015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.243+0000 7fb09579c640 1 -- 172.21.15.60:0/2897520371 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0900cccd0 con 0x7fb0900cbcf0 2024-04-25T10:05:23.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.247+0000 7fb09579c640 1 --2- 172.21.15.60:0/2897520371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0900cbcf0 0x7fb090160080 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb0800097c0 tx=0x7fb08001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9c52c86d8cb63e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.247+0000 7fb087fff640 1 -- 172.21.15.60:0/2897520371 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0800a7070 con 0x7fb0900cbcf0 2024-04-25T10:05:23.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.247+0000 7fb087fff640 1 -- 172.21.15.60:0/2897520371 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0800b0070 con 0x7fb0900cbcf0 2024-04-25T10:05:23.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.247+0000 7fb087fff640 1 -- 172.21.15.60:0/2897520371 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0800ab890 con 0x7fb0900cbcf0 2024-04-25T10:05:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.247+0000 7fb097a00640 1 -- 172.21.15.60:0/2897520371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0900cbcf0 msgr2=0x7fb090160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.247+0000 7fb097a00640 1 --2- 172.21.15.60:0/2897520371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0900cbcf0 0x7fb090160080 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb0800097c0 tx=0x7fb08001c470 comp rx=0 tx=0).stop 2024-04-25T10:05:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.247+0000 7fb097a00640 1 -- 172.21.15.60:0/2897520371 shutdown_connections 2024-04-25T10:05:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.247+0000 7fb097a00640 1 --2- 172.21.15.60:0/2897520371 >> v2:172.21.15.100:3300/0 conn(0x7fb09015fbf0 0x7fb0900ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.247+0000 7fb097a00640 1 --2- 172.21.15.60:0/2897520371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0900cbcf0 0x7fb090160080 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.247+0000 7fb097a00640 1 -- 172.21.15.60:0/2897520371 >> 172.21.15.60:0/2897520371 conn(0x7fb0900670b0 msgr2=0x7fb09006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.247+0000 7fb097a00640 1 -- 172.21.15.60:0/2897520371 shutdown_connections 2024-04-25T10:05:23.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.247+0000 7fb097a00640 1 -- 172.21.15.60:0/2897520371 wait complete. 2024-04-25T10:05:23.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb097a00640 1 Processor -- start 2024-04-25T10:05:23.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb097a00640 1 -- start start 2024-04-25T10:05:23.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb097a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb090071c30 0x7fb090141e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb097a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0900cbcf0 0x7fb0901595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb097a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb09015fbf0 0x7fb09015a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb097a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900ce740 con 0x7fb0900cbcf0 2024-04-25T10:05:23.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb097a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900ce5c0 con 0x7fb090071c30 2024-04-25T10:05:23.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb097a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb090072030 con 0x7fb09015fbf0 2024-04-25T10:05:23.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb094f9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0900cbcf0 0x7fb0901595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb094f9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0900cbcf0 0x7fb0901595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45140/0 (socket says 172.21.15.60:45140) 2024-04-25T10:05:23.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb094f9b640 1 -- 172.21.15.60:0/1970644169 learned_addr learned my addr 172.21.15.60:0/1970644169 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:23.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb09579c640 1 --2- 172.21.15.60:0/1970644169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb090071c30 0x7fb090141e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb095f9d640 1 --2- 172.21.15.60:0/1970644169 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb09015fbf0 0x7fb09015a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb094f9b640 1 -- 172.21.15.60:0/1970644169 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb09015fbf0 msgr2=0x7fb09015a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb094f9b640 1 --2- 172.21.15.60:0/1970644169 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb09015fbf0 0x7fb09015a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb094f9b640 1 -- 172.21.15.60:0/1970644169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb090071c30 msgr2=0x7fb090141e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb094f9b640 1 --2- 172.21.15.60:0/1970644169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb090071c30 0x7fb090141e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb094f9b640 1 -- 172.21.15.60:0/1970644169 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09015b5c0 con 0x7fb0900cbcf0 2024-04-25T10:05:23.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb094f9b640 1 --2- 172.21.15.60:0/1970644169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0900cbcf0 0x7fb0901595a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00c370 tx=0x7fb08c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb085ffb640 1 -- 172.21.15.60:0/1970644169 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c017070 con 0x7fb0900cbcf0 2024-04-25T10:05:23.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb085ffb640 1 -- 172.21.15.60:0/1970644169 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c005cf0 con 0x7fb0900cbcf0 2024-04-25T10:05:23.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb097a00640 1 -- 172.21.15.60:0/1970644169 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb090138c40 con 0x7fb0900cbcf0 2024-04-25T10:05:23.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb085ffb640 1 -- 172.21.15.60:0/1970644169 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0072f0 con 0x7fb0900cbcf0 2024-04-25T10:05:23.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb097a00640 1 -- 172.21.15.60:0/1970644169 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0901394e0 con 0x7fb0900cbcf0 2024-04-25T10:05:23.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb097a00640 1 -- 172.21.15.60:0/1970644169 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb090074770 con 0x7fb0900cbcf0 2024-04-25T10:05:23.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.251+0000 7fb085ffb640 1 -- 172.21.15.60:0/1970644169 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb08c01a070 con 0x7fb0900cbcf0 2024-04-25T10:05:23.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.255+0000 7fb085ffb640 1 --2- 172.21.15.60:0/1970644169 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb074066f60 0x7fb074069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.255+0000 7fb085ffb640 1 -- 172.21.15.60:0/1970644169 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb08c084320 con 0x7fb0900cbcf0 2024-04-25T10:05:23.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.255+0000 7fb09579c640 1 --2- 172.21.15.60:0/1970644169 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb074066f60 0x7fb074069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.255+0000 7fb09579c640 1 --2- 172.21.15.60:0/1970644169 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb074066f60 0x7fb074069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb0800af000 tx=0x7fb0800a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.259+0000 7fb085ffb640 1 -- 172.21.15.60:0/1970644169 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb08c054ce0 con 0x7fb0900cbcf0 2024-04-25T10:05:23.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:23.451+0000 7fb097a00640 1 -- 172.21.15.60:0/1970644169 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb0900b2970 con 0x7fb0900cbcf0 2024-04-25T10:05:24.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.091+0000 7fb085ffb640 1 -- 172.21.15.60:0/1970644169 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb08c058d80 con 0x7fb0900cbcf0 2024-04-25T10:05:24.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.099+0000 7fb097a00640 1 -- 172.21.15.60:0/1970644169 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb074066f60 msgr2=0x7fb074069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.099+0000 7fb097a00640 1 --2- 172.21.15.60:0/1970644169 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb074066f60 0x7fb074069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb0800af000 tx=0x7fb0800a7000 comp rx=0 tx=0).stop 2024-04-25T10:05:24.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.099+0000 7fb097a00640 1 -- 172.21.15.60:0/1970644169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0900cbcf0 msgr2=0x7fb0901595a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.099+0000 7fb097a00640 1 --2- 172.21.15.60:0/1970644169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0900cbcf0 0x7fb0901595a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00c370 tx=0x7fb08c00c830 comp rx=0 tx=0).stop 2024-04-25T10:05:24.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.099+0000 7fb097a00640 1 -- 172.21.15.60:0/1970644169 shutdown_connections 2024-04-25T10:05:24.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.099+0000 7fb097a00640 1 --2- 172.21.15.60:0/1970644169 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb074066f60 0x7fb074069420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.099+0000 7fb097a00640 1 --2- 172.21.15.60:0/1970644169 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb09015fbf0 0x7fb09015a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.099+0000 7fb097a00640 1 --2- 172.21.15.60:0/1970644169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0900cbcf0 0x7fb0901595a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.099+0000 7fb097a00640 1 --2- 172.21.15.60:0/1970644169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb090071c30 0x7fb090141e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.099+0000 7fb097a00640 1 -- 172.21.15.60:0/1970644169 >> 172.21.15.60:0/1970644169 conn(0x7fb0900670b0 msgr2=0x7fb09013e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.099+0000 7fb097a00640 1 -- 172.21.15.60:0/1970644169 shutdown_connections 2024-04-25T10:05:24.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.099+0000 7fb097a00640 1 -- 172.21.15.60:0/1970644169 wait complete. 2024-04-25T10:05:24.129 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:05:24.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:24.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.251+0000 7fad5a5e1640 1 Processor -- start 2024-04-25T10:05:24.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.251+0000 7fad5a5e1640 1 -- start start 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad5a5e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad5415a7a0 0x7fad540cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad5a5e1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fad5406ec30 0x7fad54071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad5a5e1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fad540b37e0 0x7fad540b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad5a5e1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad540cdaa0 con 0x7fad5406ec30 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad5a5e1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5406d2f0 con 0x7fad540b37e0 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad5a5e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5406d8b0 con 0x7fad5415a7a0 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad537fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fad5406ec30 0x7fad54071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad537fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fad5406ec30 0x7fad54071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45144/0 (socket says 172.21.15.60:45144) 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad537fe640 1 -- 172.21.15.60:0/1342920319 learned_addr learned my addr 172.21.15.60:0/1342920319 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad58b7e640 1 --2- 172.21.15.60:0/1342920319 >> v2:172.21.15.100:3300/0 conn(0x7fad540b37e0 0x7fad540b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad53fff640 1 --2- 172.21.15.60:0/1342920319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad5415a7a0 0x7fad540cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad537fe640 1 -- 172.21.15.60:0/1342920319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad5415a7a0 msgr2=0x7fad540cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad537fe640 1 --2- 172.21.15.60:0/1342920319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad5415a7a0 0x7fad540cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad537fe640 1 -- 172.21.15.60:0/1342920319 >> v2:172.21.15.100:3300/0 conn(0x7fad540b37e0 msgr2=0x7fad540b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad537fe640 1 --2- 172.21.15.60:0/1342920319 >> v2:172.21.15.100:3300/0 conn(0x7fad540b37e0 0x7fad540b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad537fe640 1 -- 172.21.15.60:0/1342920319 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad540721e0 con 0x7fad5406ec30 2024-04-25T10:05:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.255+0000 7fad537fe640 1 --2- 172.21.15.60:0/1342920319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad5406ec30 0x7fad54071ca0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fad540b2660 tx=0x7fad4801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad31a436778de6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad52ffd640 1 -- 172.21.15.60:0/1342920319 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad480b4070 con 0x7fad5406ec30 2024-04-25T10:05:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad52ffd640 1 -- 172.21.15.60:0/1342920319 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad480a7b90 con 0x7fad5406ec30 2024-04-25T10:05:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad52ffd640 1 -- 172.21.15.60:0/1342920319 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad480af440 con 0x7fad5406ec30 2024-04-25T10:05:24.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 -- 172.21.15.60:0/1342920319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad5406ec30 msgr2=0x7fad54071ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 --2- 172.21.15.60:0/1342920319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad5406ec30 0x7fad54071ca0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fad540b2660 tx=0x7fad4801ebf0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 -- 172.21.15.60:0/1342920319 shutdown_connections 2024-04-25T10:05:24.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 --2- 172.21.15.60:0/1342920319 >> v2:172.21.15.100:3300/0 conn(0x7fad540b37e0 0x7fad540b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 --2- 172.21.15.60:0/1342920319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad5406ec30 0x7fad54071ca0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 --2- 172.21.15.60:0/1342920319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad5415a7a0 0x7fad540cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 -- 172.21.15.60:0/1342920319 >> 172.21.15.60:0/1342920319 conn(0x7fad540b12d0 msgr2=0x7fad540b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 -- 172.21.15.60:0/1342920319 shutdown_connections 2024-04-25T10:05:24.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 -- 172.21.15.60:0/1342920319 wait complete. 2024-04-25T10:05:24.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 Processor -- start 2024-04-25T10:05:24.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 -- start start 2024-04-25T10:05:24.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad5406ec30 0x7fad5413eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad540b37e0 0x7fad5413f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad5415a7a0 0x7fad541409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5415ad00 con 0x7fad5406ec30 2024-04-25T10:05:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5415ab80 con 0x7fad5415a7a0 2024-04-25T10:05:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5406e530 con 0x7fad540b37e0 2024-04-25T10:05:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad53fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad5406ec30 0x7fad5413eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad53fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad5406ec30 0x7fad5413eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45158/0 (socket says 172.21.15.60:45158) 2024-04-25T10:05:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad53fff640 1 -- 172.21.15.60:0/3263754029 learned_addr learned my addr 172.21.15.60:0/3263754029 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad537fe640 1 --2- 172.21.15.60:0/3263754029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad540b37e0 0x7fad5413f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad58b7e640 1 --2- 172.21.15.60:0/3263754029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad5415a7a0 0x7fad541409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad53fff640 1 -- 172.21.15.60:0/3263754029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad540b37e0 msgr2=0x7fad5413f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad53fff640 1 --2- 172.21.15.60:0/3263754029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad540b37e0 0x7fad5413f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad53fff640 1 -- 172.21.15.60:0/3263754029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad5415a7a0 msgr2=0x7fad541409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad53fff640 1 --2- 172.21.15.60:0/3263754029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad5415a7a0 0x7fad541409c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad53fff640 1 -- 172.21.15.60:0/3263754029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5413f920 con 0x7fad5406ec30 2024-04-25T10:05:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad58b7e640 1 --2- 172.21.15.60:0/3263754029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad5415a7a0 0x7fad541409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad53fff640 1 --2- 172.21.15.60:0/3263754029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad5406ec30 0x7fad5413eac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad4400d5f0 tx=0x7fad4400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad50ff9640 1 -- 172.21.15.60:0/3263754029 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad440125f0 con 0x7fad5406ec30 2024-04-25T10:05:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad50ff9640 1 -- 172.21.15.60:0/3263754029 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad44019070 con 0x7fad5406ec30 2024-04-25T10:05:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 -- 172.21.15.60:0/3263754029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5413fc00 con 0x7fad5406ec30 2024-04-25T10:05:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad50ff9640 1 -- 172.21.15.60:0/3263754029 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad440129b0 con 0x7fad5406ec30 2024-04-25T10:05:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.259+0000 7fad5a5e1640 1 -- 172.21.15.60:0/3263754029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad541401c0 con 0x7fad5406ec30 2024-04-25T10:05:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.263+0000 7fad5a5e1640 1 -- 172.21.15.60:0/3263754029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad54074770 con 0x7fad5406ec30 2024-04-25T10:05:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.263+0000 7fad50ff9640 1 -- 172.21.15.60:0/3263754029 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad44012b50 con 0x7fad5406ec30 2024-04-25T10:05:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.263+0000 7fad50ff9640 1 --2- 172.21.15.60:0/3263754029 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fad40066f60 0x7fad40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.263+0000 7fad50ff9640 1 -- 172.21.15.60:0/3263754029 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad44085320 con 0x7fad5406ec30 2024-04-25T10:05:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.263+0000 7fad537fe640 1 --2- 172.21.15.60:0/3263754029 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fad40066f60 0x7fad40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.263+0000 7fad537fe640 1 --2- 172.21.15.60:0/3263754029 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fad40066f60 0x7fad40069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fad480b1000 tx=0x7fad48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.267+0000 7fad50ff9640 1 -- 172.21.15.60:0/3263754029 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad44011340 con 0x7fad5406ec30 2024-04-25T10:05:24.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.459+0000 7fad5a5e1640 1 -- 172.21.15.60:0/3263754029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad540b30f0 con 0x7fad5406ec30 2024-04-25T10:05:24.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.463+0000 7fad50ff9640 1 -- 172.21.15.60:0/3263754029 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7fad44056330 con 0x7fad5406ec30 2024-04-25T10:05:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:24.469 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/622088871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":622088871},{"type":"v1","addr":"172.21.15.100:6836","nonce":622088871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3988204202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3988204202},{"type":"v1","addr":"172.21.15.60:6833","nonce":3988204202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3591836867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3591836867},{"type":"v1","addr":"172.21.15.60:6835","nonce":3591836867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:20.405486+0000","modified":"2024-04-25T10:05:23.453258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/690070228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":690070228},{"type":"v1","addr":"172.21.15.100:6837","nonce":690070228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:05:24.469 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-04-25T10:05:24.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.463+0000 7fad5a5e1640 1 -- 172.21.15.60:0/3263754029 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fad40066f60 msgr2=0x7fad40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.463+0000 7fad5a5e1640 1 --2- 172.21.15.60:0/3263754029 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fad40066f60 0x7fad40069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fad480b1000 tx=0x7fad48008040 comp rx=0 tx=0).stop 2024-04-25T10:05:24.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.463+0000 7fad5a5e1640 1 -- 172.21.15.60:0/3263754029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad5406ec30 msgr2=0x7fad5413eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.463+0000 7fad5a5e1640 1 --2- 172.21.15.60:0/3263754029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad5406ec30 0x7fad5413eac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad4400d5f0 tx=0x7fad4400dab0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.463+0000 7fad5a5e1640 1 -- 172.21.15.60:0/3263754029 shutdown_connections 2024-04-25T10:05:24.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.463+0000 7fad5a5e1640 1 --2- 172.21.15.60:0/3263754029 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fad40066f60 0x7fad40069420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.463+0000 7fad5a5e1640 1 --2- 172.21.15.60:0/3263754029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad5415a7a0 0x7fad541409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.463+0000 7fad5a5e1640 1 --2- 172.21.15.60:0/3263754029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad540b37e0 0x7fad5413f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.463+0000 7fad5a5e1640 1 --2- 172.21.15.60:0/3263754029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad5406ec30 0x7fad5413eac0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.463+0000 7fad5a5e1640 1 -- 172.21.15.60:0/3263754029 >> 172.21.15.60:0/3263754029 conn(0x7fad540b12d0 msgr2=0x7fad540cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.463+0000 7fad5a5e1640 1 -- 172.21.15.60:0/3263754029 shutdown_connections 2024-04-25T10:05:24.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.463+0000 7fad5a5e1640 1 -- 172.21.15.60:0/3263754029 wait complete. 2024-04-25T10:05:24.488 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:24.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.591+0000 7f8291755640 1 Processor -- start 2024-04-25T10:05:24.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.591+0000 7f8291755640 1 -- start start 2024-04-25T10:05:24.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.591+0000 7f8291755640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f828c15bc90 0x7f828c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.591+0000 7f8291755640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828c071ca0 con 0x7f828c0b4660 2024-04-25T10:05:24.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.591+0000 7f8291755640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828c0cca40 con 0x7f828c0ce7a0 2024-04-25T10:05:24.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.591+0000 7f8291755640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f828c0cc370 con 0x7f828c15bc90 2024-04-25T10:05:24.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.591+0000 7f82827fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f828c15bc90 0x7f828c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.591+0000 7f82827fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f828c15bc90 0x7f828c15c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45166/0 (socket says 172.21.15.60:45166) 2024-04-25T10:05:24.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.591+0000 7f82827fc640 1 -- 172.21.15.60:0/2656301699 learned_addr learned my addr 172.21.15.60:0/2656301699 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:24.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.591+0000 7f828b7fe640 1 --1- 172.21.15.60:0/2656301699 >> v1:172.21.15.60:6789/0 conn(0x7f828c0b4660 0x7f828c15c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55734/0 (socket says 172.21.15.60:55734) 2024-04-25T10:05:24.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.595+0000 7f82827fc640 1 -- 172.21.15.60:0/2656301699 >> v1:172.21.15.100:6789/0 conn(0x7f828c0ce7a0 legacy=0x7f828c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.595+0000 7f82827fc640 1 -- 172.21.15.60:0/2656301699 >> v1:172.21.15.60:6789/0 conn(0x7f828c0b4660 legacy=0x7f828c15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.595+0000 7f82827fc640 1 -- 172.21.15.60:0/2656301699 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c0ccd20 con 0x7f828c15bc90 2024-04-25T10:05:24.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.595+0000 7f828a7fc640 1 -- 172.21.15.60:0/2656301699 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600598219 0 0) 0x7f828c071ca0 con 0x7f828c0b4660 2024-04-25T10:05:24.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.595+0000 7f82827fc640 1 --2- 172.21.15.60:0/2656301699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f828c15bc90 0x7f828c15c220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8274009870 tx=0x7f827401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bfd23c100836da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.595+0000 7f828a7fc640 1 -- 172.21.15.60:0/2656301699 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82740b0070 con 0x7f828c15bc90 2024-04-25T10:05:24.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.595+0000 7f828a7fc640 1 -- 172.21.15.60:0/2656301699 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82740ab600 con 0x7f828c15bc90 2024-04-25T10:05:24.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.595+0000 7f828a7fc640 1 -- 172.21.15.60:0/2656301699 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82740ab9d0 con 0x7f828c15bc90 2024-04-25T10:05:24.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.595+0000 7f8291755640 1 -- 172.21.15.60:0/2656301699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f828c15bc90 msgr2=0x7f828c15c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.595+0000 7f8291755640 1 --2- 172.21.15.60:0/2656301699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f828c15bc90 0x7f828c15c220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8274009870 tx=0x7f827401c520 comp rx=0 tx=0).stop 2024-04-25T10:05:24.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.595+0000 7f8291755640 1 -- 172.21.15.60:0/2656301699 shutdown_connections 2024-04-25T10:05:24.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.595+0000 7f8291755640 1 --2- 172.21.15.60:0/2656301699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f828c15bc90 0x7f828c15c220 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.595+0000 7f8291755640 1 -- 172.21.15.60:0/2656301699 >> 172.21.15.60:0/2656301699 conn(0x7f828c0b12d0 msgr2=0x7f828c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.595+0000 7f8291755640 1 -- 172.21.15.60:0/2656301699 shutdown_connections 2024-04-25T10:05:24.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.595+0000 7f8291755640 1 -- 172.21.15.60:0/2656301699 wait complete. 2024-04-25T10:05:24.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f8291755640 1 Processor -- start 2024-04-25T10:05:24.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f8291755640 1 -- start start 2024-04-25T10:05:24.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f8291755640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f828c0ce7a0 0x7f828c14fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f8291755640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f828c15bc90 0x7f828c14e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f8291755640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c150430 0x7f828c1524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f8291755640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c1695d0 con 0x7f828c15bc90 2024-04-25T10:05:24.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f8291755640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0b3db0 con 0x7f828c150430 2024-04-25T10:05:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f8291755640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c169750 con 0x7f828c0ce7a0 2024-04-25T10:05:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f82827fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f828c15bc90 0x7f828c14e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f82827fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f828c15bc90 0x7f828c14e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45170/0 (socket says 172.21.15.60:45170) 2024-04-25T10:05:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f82827fc640 1 -- 172.21.15.60:0/1832433212 learned_addr learned my addr 172.21.15.60:0/1832433212 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f828affd640 1 --2- 172.21.15.60:0/1832433212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f828c0ce7a0 0x7f828c14fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f828b7fe640 1 --2- 172.21.15.60:0/1832433212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c150430 0x7f828c1524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f82827fc640 1 -- 172.21.15.60:0/1832433212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f828c0ce7a0 msgr2=0x7f828c14fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f82827fc640 1 --2- 172.21.15.60:0/1832433212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f828c0ce7a0 0x7f828c14fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f82827fc640 1 -- 172.21.15.60:0/1832433212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c150430 msgr2=0x7f828c1524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f82827fc640 1 --2- 172.21.15.60:0/1832433212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c150430 0x7f828c1524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f82827fc640 1 -- 172.21.15.60:0/1832433212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c135230 con 0x7f828c15bc90 2024-04-25T10:05:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f828affd640 1 --2- 172.21.15.60:0/1832433212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f828c0ce7a0 0x7f828c14fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f828b7fe640 1 --2- 172.21.15.60:0/1832433212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c150430 0x7f828c1524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f82827fc640 1 --2- 172.21.15.60:0/1832433212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f828c15bc90 0x7f828c14e230 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82740ae040 tx=0x7f828c15d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f8283fff640 1 -- 172.21.15.60:0/1832433212 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82740b0040 con 0x7f828c15bc90 2024-04-25T10:05:24.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f8283fff640 1 -- 172.21.15.60:0/1832433212 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82740b6420 con 0x7f828c15bc90 2024-04-25T10:05:24.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f8291755640 1 -- 172.21.15.60:0/1832433212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828c1508d0 con 0x7f828c15bc90 2024-04-25T10:05:24.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f8283fff640 1 -- 172.21.15.60:0/1832433212 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82740b6830 con 0x7f828c15bc90 2024-04-25T10:05:24.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f8291755640 1 -- 172.21.15.60:0/1832433212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c150e90 con 0x7f828c15bc90 2024-04-25T10:05:24.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.599+0000 7f8291755640 1 -- 172.21.15.60:0/1832433212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828c074770 con 0x7f828c15bc90 2024-04-25T10:05:24.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.603+0000 7f8283fff640 1 -- 172.21.15.60:0/1832433212 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82740be020 con 0x7f828c15bc90 2024-04-25T10:05:24.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.603+0000 7f8283fff640 1 --2- 172.21.15.60:0/1832433212 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f826c066f60 0x7f826c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.603+0000 7f8283fff640 1 -- 172.21.15.60:0/1832433212 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f827411eb60 con 0x7f828c15bc90 2024-04-25T10:05:24.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.603+0000 7f828affd640 1 --2- 172.21.15.60:0/1832433212 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f826c066f60 0x7f826c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.603+0000 7f828affd640 1 --2- 172.21.15.60:0/1832433212 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f826c066f60 0x7f826c069420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f827c00a850 tx=0x7f827c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.607+0000 7f8283fff640 1 -- 172.21.15.60:0/1832433212 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f82740c5020 con 0x7f828c15bc90 2024-04-25T10:05:24.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.767+0000 7f8291755640 1 -- 172.21.15.60:0/1832433212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f828c153d50 con 0x7f828c15bc90 2024-04-25T10:05:24.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.771+0000 7f8283fff640 1 -- 172.21.15.60:0/1832433212 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f8274016110 con 0x7f828c15bc90 2024-04-25T10:05:24.776 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:24.777 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:22.036582+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:24.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.771+0000 7f8291755640 1 -- 172.21.15.60:0/1832433212 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f826c066f60 msgr2=0x7f826c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.771+0000 7f8291755640 1 --2- 172.21.15.60:0/1832433212 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f826c066f60 0x7f826c069420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f827c00a850 tx=0x7f827c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:24.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.771+0000 7f8291755640 1 -- 172.21.15.60:0/1832433212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f828c15bc90 msgr2=0x7f828c14e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.771+0000 7f8291755640 1 --2- 172.21.15.60:0/1832433212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f828c15bc90 0x7f828c14e230 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82740ae040 tx=0x7f828c15d680 comp rx=0 tx=0).stop 2024-04-25T10:05:24.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.771+0000 7f8291755640 1 -- 172.21.15.60:0/1832433212 shutdown_connections 2024-04-25T10:05:24.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.771+0000 7f8291755640 1 --2- 172.21.15.60:0/1832433212 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f826c066f60 0x7f826c069420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.771+0000 7f8291755640 1 --2- 172.21.15.60:0/1832433212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c150430 0x7f828c1524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.771+0000 7f8291755640 1 --2- 172.21.15.60:0/1832433212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f828c15bc90 0x7f828c14e230 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.771+0000 7f8291755640 1 --2- 172.21.15.60:0/1832433212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f828c0ce7a0 0x7f828c14fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.771+0000 7f8291755640 1 -- 172.21.15.60:0/1832433212 >> 172.21.15.60:0/1832433212 conn(0x7f828c0b12d0 msgr2=0x7f828c0b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.771+0000 7f8291755640 1 -- 172.21.15.60:0/1832433212 shutdown_connections 2024-04-25T10:05:24.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.771+0000 7f8291755640 1 -- 172.21.15.60:0/1832433212 wait complete. 2024-04-25T10:05:24.796 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:05:24.796 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.903+0000 7f62719af640 1 Processor -- start 2024-04-25T10:05:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.903+0000 7f62719af640 1 -- start start 2024-04-25T10:05:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f62719af640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f626c06ec30 0x7f626c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f62719af640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626c06f130 con 0x7f626c15a7a0 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f62719af640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626c06f410 con 0x7f626c0b37e0 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f62719af640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f626c0cdaa0 con 0x7f626c06ec30 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f626affd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f626c15a7a0 0x7f626c071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55740/0 (socket says 172.21.15.60:55740) 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f626affd640 1 -- 172.21.15.60:0/2580197782 learned_addr learned my addr 172.21.15.60:0/2580197782 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f626a7fc640 1 --2- 172.21.15.60:0/2580197782 >> v2:172.21.15.100:3300/0 conn(0x7f626c06ec30 0x7f626c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f6269ffb640 1 -- 172.21.15.60:0/2580197782 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068405928 0 0) 0x7f626c06f130 con 0x7f626c15a7a0 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f6269ffb640 1 -- 172.21.15.60:0/2580197782 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6254003530 con 0x7f626c15a7a0 2024-04-25T10:05:24.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f6269ffb640 1 -- 172.21.15.60:0/2580197782 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 642197697 0 0) 0x7f6254003530 con 0x7f626c15a7a0 2024-04-25T10:05:24.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f6269ffb640 1 -- 172.21.15.60:0/2580197782 >> v2:172.21.15.100:3300/0 conn(0x7f626c06ec30 msgr2=0x7f626c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f6269ffb640 1 --2- 172.21.15.60:0/2580197782 >> v2:172.21.15.100:3300/0 conn(0x7f626c06ec30 0x7f626c0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f6269ffb640 1 -- 172.21.15.60:0/2580197782 >> v1:172.21.15.100:6789/0 conn(0x7f626c0b37e0 legacy=0x7f626c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f6269ffb640 1 -- 172.21.15.60:0/2580197782 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c06d540 con 0x7f626c15a7a0 2024-04-25T10:05:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f6269ffb640 1 -- 172.21.15.60:0/2580197782 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f6260001680 con 0x7f626c15a7a0 2024-04-25T10:05:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f6269ffb640 1 -- 172.21.15.60:0/2580197782 >> v1:172.21.15.60:6789/0 conn(0x7f626c15a7a0 legacy=0x7f626c071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f6269ffb640 1 --2- 172.21.15.60:0/2580197782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6254003a40 0x7f6254003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f6269ffb640 1 --2- 172.21.15.60:0/2580197782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6254004810 0x7f6254004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f6269ffb640 1 --2- 172.21.15.60:0/2580197782 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540051d0 0x7f625400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f6269ffb640 1 -- 172.21.15.60:0/2580197782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625400c0d0 con 0x7f6254003a40 2024-04-25T10:05:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f6269ffb640 1 -- 172.21.15.60:0/2580197782 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625400c280 con 0x7f6254004810 2024-04-25T10:05:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f6269ffb640 1 -- 172.21.15.60:0/2580197782 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625400c480 con 0x7f62540051d0 2024-04-25T10:05:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f626affd640 1 --2- 172.21.15.60:0/2580197782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6254003a40 0x7f6254003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f626a7fc640 1 --2- 172.21.15.60:0/2580197782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6254004810 0x7f6254004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f626b7fe640 1 --2- 172.21.15.60:0/2580197782 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540051d0 0x7f625400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f626affd640 1 -- 172.21.15.60:0/2580197782 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540051d0 msgr2=0x7f625400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f626affd640 1 --2- 172.21.15.60:0/2580197782 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540051d0 0x7f625400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f626affd640 1 -- 172.21.15.60:0/2580197782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6254004810 msgr2=0x7f6254004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f626affd640 1 --2- 172.21.15.60:0/2580197782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6254004810 0x7f6254004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f626b7fe640 1 -- 172.21.15.60:0/2580197782 reap_dead start 2024-04-25T10:05:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.907+0000 7f626affd640 1 -- 172.21.15.60:0/2580197782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6254003530 con 0x7f6254003a40 2024-04-25T10:05:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f626affd640 1 --2- 172.21.15.60:0/2580197782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6254003a40 0x7f6254003e60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f626000c110 tx=0x7f626001cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb46983c4251c447 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f6269ffb640 1 -- 172.21.15.60:0/2580197782 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260007c50 con 0x7f6254003a40 2024-04-25T10:05:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f6269ffb640 1 -- 172.21.15.60:0/2580197782 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62600a9070 con 0x7f6254003a40 2024-04-25T10:05:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f6269ffb640 1 -- 172.21.15.60:0/2580197782 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260005d80 con 0x7f6254003a40 2024-04-25T10:05:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f62719af640 1 -- 172.21.15.60:0/2580197782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6254003a40 msgr2=0x7f6254003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f62719af640 1 --2- 172.21.15.60:0/2580197782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6254003a40 0x7f6254003e60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f626000c110 tx=0x7f626001cbc0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f62719af640 1 -- 172.21.15.60:0/2580197782 shutdown_connections 2024-04-25T10:05:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f62719af640 1 --2- 172.21.15.60:0/2580197782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6254003a40 0x7f6254003e60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f62719af640 1 -- 172.21.15.60:0/2580197782 >> 172.21.15.60:0/2580197782 conn(0x7f626c0b12d0 msgr2=0x7f626c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f62719af640 1 -- 172.21.15.60:0/2580197782 shutdown_connections 2024-04-25T10:05:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f62719af640 1 -- 172.21.15.60:0/2580197782 wait complete. 2024-04-25T10:05:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f62719af640 1 Processor -- start 2024-04-25T10:05:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f62719af640 1 -- start start 2024-04-25T10:05:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f62719af640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6254003a40 0x7f626c0a8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f62719af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f626c15c150 0x7f626c145af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f62719af640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f626c15c5b0 0x7f626c146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f62719af640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c06e8c0 con 0x7f6254003a40 2024-04-25T10:05:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f62719af640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c06e740 con 0x7f626c15c150 2024-04-25T10:05:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.911+0000 7f62719af640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626c06d2f0 con 0x7f626c15c5b0 2024-04-25T10:05:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f626affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6254003a40 0x7f626c0a8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f626affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6254003a40 0x7f626c0a8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45200/0 (socket says 172.21.15.60:45200) 2024-04-25T10:05:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f626affd640 1 -- 172.21.15.60:0/2374650592 learned_addr learned my addr 172.21.15.60:0/2374650592 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f626a7fc640 1 --2- 172.21.15.60:0/2374650592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f626c15c150 0x7f626c145af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f626b7fe640 1 --2- 172.21.15.60:0/2374650592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f626c15c5b0 0x7f626c146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f626affd640 1 -- 172.21.15.60:0/2374650592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f626c15c5b0 msgr2=0x7f626c146030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f626affd640 1 --2- 172.21.15.60:0/2374650592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f626c15c5b0 0x7f626c146030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f626affd640 1 -- 172.21.15.60:0/2374650592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f626c15c150 msgr2=0x7f626c145af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f626affd640 1 --2- 172.21.15.60:0/2374650592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f626c15c150 0x7f626c145af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f626affd640 1 -- 172.21.15.60:0/2374650592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c157b30 con 0x7f6254003a40 2024-04-25T10:05:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f626affd640 1 --2- 172.21.15.60:0/2374650592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6254003a40 0x7f626c0a8490 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f626000c110 tx=0x7f6254005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f6253fff640 1 -- 172.21.15.60:0/2374650592 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260004a00 con 0x7f6254003a40 2024-04-25T10:05:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f6253fff640 1 -- 172.21.15.60:0/2374650592 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6260004ba0 con 0x7f6254003a40 2024-04-25T10:05:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f62719af640 1 -- 172.21.15.60:0/2374650592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626c0a89d0 con 0x7f6254003a40 2024-04-25T10:05:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f6253fff640 1 -- 172.21.15.60:0/2374650592 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62600070d0 con 0x7f6254003a40 2024-04-25T10:05:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f62719af640 1 -- 172.21.15.60:0/2374650592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626c157200 con 0x7f6254003a40 2024-04-25T10:05:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f62719af640 1 -- 172.21.15.60:0/2374650592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626c074770 con 0x7f6254003a40 2024-04-25T10:05:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f6253fff640 1 -- 172.21.15.60:0/2374650592 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6260007270 con 0x7f6254003a40 2024-04-25T10:05:24.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f6253fff640 1 --2- 172.21.15.60:0/2374650592 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6258066f60 0x7f6258069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f6253fff640 1 -- 172.21.15.60:0/2374650592 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f626011e7c0 con 0x7f6254003a40 2024-04-25T10:05:24.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.915+0000 7f626a7fc640 1 --2- 172.21.15.60:0/2374650592 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6258066f60 0x7f6258069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.919+0000 7f626a7fc640 1 --2- 172.21.15.60:0/2374650592 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6258066f60 0x7f6258069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f626c0b0390 tx=0x7f625c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:24.919+0000 7f6253fff640 1 -- 172.21.15.60:0/2374650592 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62600168d0 con 0x7f6254003a40 2024-04-25T10:05:25.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.075+0000 7f62719af640 1 -- 172.21.15.60:0/2374650592 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f626c15eb40 con 0x7f6258066f60 2024-04-25T10:05:25.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.095+0000 7f6253fff640 1 -- 172.21.15.60:0/2374650592 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251946 (secure 0 0 0) 0x7f626c15eb40 con 0x7f6258066f60 2024-04-25T10:05:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:25.106 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:05:25.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.099+0000 7f62719af640 1 -- 172.21.15.60:0/2374650592 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6258066f60 msgr2=0x7f6258069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.099+0000 7f62719af640 1 --2- 172.21.15.60:0/2374650592 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6258066f60 0x7f6258069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f626c0b0390 tx=0x7f625c009190 comp rx=0 tx=0).stop 2024-04-25T10:05:25.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.099+0000 7f62719af640 1 -- 172.21.15.60:0/2374650592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6254003a40 msgr2=0x7f626c0a8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.099+0000 7f62719af640 1 --2- 172.21.15.60:0/2374650592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6254003a40 0x7f626c0a8490 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f626000c110 tx=0x7f6254005840 comp rx=0 tx=0).stop 2024-04-25T10:05:25.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.099+0000 7f62719af640 1 -- 172.21.15.60:0/2374650592 shutdown_connections 2024-04-25T10:05:25.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.099+0000 7f62719af640 1 --2- 172.21.15.60:0/2374650592 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6258066f60 0x7f6258069420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.099+0000 7f62719af640 1 --2- 172.21.15.60:0/2374650592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f626c15c5b0 0x7f626c146030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.099+0000 7f62719af640 1 --2- 172.21.15.60:0/2374650592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f626c15c150 0x7f626c145af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.099+0000 7f62719af640 1 --2- 172.21.15.60:0/2374650592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6254003a40 0x7f626c0a8490 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.099+0000 7f62719af640 1 -- 172.21.15.60:0/2374650592 >> 172.21.15.60:0/2374650592 conn(0x7f626c0b12d0 msgr2=0x7f626c0ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.099+0000 7f62719af640 1 -- 172.21.15.60:0/2374650592 shutdown_connections 2024-04-25T10:05:25.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.099+0000 7f62719af640 1 -- 172.21.15.60:0/2374650592 wait complete. 2024-04-25T10:05:25.124 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T10:05:24.386913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235768,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738952,"statfs":{"total":773094113280,"available":772852686848,"internally_reserved":0,"allocated":3022848,"data_stored":1811156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.353600"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137405+0000","last_change":"2024-04-25T10:05:21.093662+0000","last_active":"2024-04-25T10:05:22.137405+0000","last_peered":"2024-04-25T10:05:22.137405+0000","last_clean":"2024-04-25T10:05:22.137405+0000","last_became_active":"2024-04-25T10:05:21.093342+0000","last_became_peered":"2024-04-25T10:05:21.093342+0000","last_unstale":"2024-04-25T10:05:22.137405+0000","last_undegraded":"2024-04-25T10:05:22.137405+0000","last_fullsized":"2024-04-25T10:05:22.137405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:38.452914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043321+0000","last_change":"2024-04-25T10:05:21.076738+0000","last_active":"2024-04-25T10:05:22.043321+0000","last_peered":"2024-04-25T10:05:22.043321+0000","last_clean":"2024-04-25T10:05:22.043321+0000","last_became_active":"2024-04-25T10:05:21.076085+0000","last_became_peered":"2024-04-25T10:05:21.076085+0000","last_unstale":"2024-04-25T10:05:22.043321+0000","last_undegraded":"2024-04-25T10:05:22.043321+0000","last_fullsized":"2024-04-25T10:05:22.043321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:36.826815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136181+0000","last_change":"2024-04-25T10:05:21.072290+0000","last_active":"2024-04-25T10:05:22.136181+0000","last_peered":"2024-04-25T10:05:22.136181+0000","last_clean":"2024-04-25T10:05:22.136181+0000","last_became_active":"2024-04-25T10:05:21.069763+0000","last_became_peered":"2024-04-25T10:05:21.069763+0000","last_unstale":"2024-04-25T10:05:22.136181+0000","last_undegraded":"2024-04-25T10:05:22.136181+0000","last_fullsized":"2024-04-25T10:05:22.136181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:17.144179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048606+0000","last_change":"2024-04-25T10:05:21.083631+0000","last_active":"2024-04-25T10:05:22.048606+0000","last_peered":"2024-04-25T10:05:22.048606+0000","last_clean":"2024-04-25T10:05:22.048606+0000","last_became_active":"2024-04-25T10:05:21.082783+0000","last_became_peered":"2024-04-25T10:05:21.082783+0000","last_unstale":"2024-04-25T10:05:22.048606+0000","last_undegraded":"2024-04-25T10:05:22.048606+0000","last_fullsized":"2024-04-25T10:05:22.048606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:02.040066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136043+0000","last_change":"2024-04-25T10:05:21.092602+0000","last_active":"2024-04-25T10:05:22.136043+0000","last_peered":"2024-04-25T10:05:22.136043+0000","last_clean":"2024-04-25T10:05:22.136043+0000","last_became_active":"2024-04-25T10:05:21.089559+0000","last_became_peered":"2024-04-25T10:05:21.089559+0000","last_unstale":"2024-04-25T10:05:22.136043+0000","last_undegraded":"2024-04-25T10:05:22.136043+0000","last_fullsized":"2024-04-25T10:05:22.136043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:57.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046962+0000","last_change":"2024-04-25T10:05:21.078938+0000","last_active":"2024-04-25T10:05:22.046962+0000","last_peered":"2024-04-25T10:05:22.046962+0000","last_clean":"2024-04-25T10:05:22.046962+0000","last_became_active":"2024-04-25T10:05:21.078310+0000","last_became_peered":"2024-04-25T10:05:21.078310+0000","last_unstale":"2024-04-25T10:05:22.046962+0000","last_undegraded":"2024-04-25T10:05:22.046962+0000","last_fullsized":"2024-04-25T10:05:22.046962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:30.299159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135741+0000","last_change":"2024-04-25T10:05:21.086161+0000","last_active":"2024-04-25T10:05:22.135741+0000","last_peered":"2024-04-25T10:05:22.135741+0000","last_clean":"2024-04-25T10:05:22.135741+0000","last_became_active":"2024-04-25T10:05:21.084217+0000","last_became_peered":"2024-04-25T10:05:21.084217+0000","last_unstale":"2024-04-25T10:05:22.135741+0000","last_undegraded":"2024-04-25T10:05:22.135741+0000","last_fullsized":"2024-04-25T10:05:22.135741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:29.451932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.133417+0000","last_change":"2024-04-25T10:05:21.071738+0000","last_active":"2024-04-25T10:05:22.133417+0000","last_peered":"2024-04-25T10:05:22.133417+0000","last_clean":"2024-04-25T10:05:22.133417+0000","last_became_active":"2024-04-25T10:05:21.071239+0000","last_became_peered":"2024-04-25T10:05:21.071239+0000","last_unstale":"2024-04-25T10:05:22.133417+0000","last_undegraded":"2024-04-25T10:05:22.133417+0000","last_fullsized":"2024-04-25T10:05:22.133417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:13.318906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137424+0000","last_change":"2024-04-25T10:05:21.089854+0000","last_active":"2024-04-25T10:05:22.137424+0000","last_peered":"2024-04-25T10:05:22.137424+0000","last_clean":"2024-04-25T10:05:22.137424+0000","last_became_active":"2024-04-25T10:05:21.089145+0000","last_became_peered":"2024-04-25T10:05:21.089145+0000","last_unstale":"2024-04-25T10:05:22.137424+0000","last_undegraded":"2024-04-25T10:05:22.137424+0000","last_fullsized":"2024-04-25T10:05:22.137424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:17.168463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.081237+0000","last_change":"2024-04-25T10:05:21.081763+0000","last_active":"2024-04-25T10:05:22.081237+0000","last_peered":"2024-04-25T10:05:22.081237+0000","last_clean":"2024-04-25T10:05:22.081237+0000","last_became_active":"2024-04-25T10:05:21.081398+0000","last_became_peered":"2024-04-25T10:05:21.081398+0000","last_unstale":"2024-04-25T10:05:22.081237+0000","last_undegraded":"2024-04-25T10:05:22.081237+0000","last_fullsized":"2024-04-25T10:05:22.081237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:23.519531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080566+0000","last_change":"2024-04-25T10:05:21.081165+0000","last_active":"2024-04-25T10:05:22.080566+0000","last_peered":"2024-04-25T10:05:22.080566+0000","last_clean":"2024-04-25T10:05:22.080566+0000","last_became_active":"2024-04-25T10:05:21.076196+0000","last_became_peered":"2024-04-25T10:05:21.076196+0000","last_unstale":"2024-04-25T10:05:22.080566+0000","last_undegraded":"2024-04-25T10:05:22.080566+0000","last_fullsized":"2024-04-25T10:05:22.080566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:36.529873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048072+0000","last_change":"2024-04-25T10:05:21.082653+0000","last_active":"2024-04-25T10:05:22.048072+0000","last_peered":"2024-04-25T10:05:22.048072+0000","last_clean":"2024-04-25T10:05:22.048072+0000","last_became_active":"2024-04-25T10:05:21.082004+0000","last_became_peered":"2024-04-25T10:05:21.082004+0000","last_unstale":"2024-04-25T10:05:22.048072+0000","last_undegraded":"2024-04-25T10:05:22.048072+0000","last_fullsized":"2024-04-25T10:05:22.048072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:56.267445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136198+0000","last_change":"2024-04-25T10:05:21.071024+0000","last_active":"2024-04-25T10:05:22.136198+0000","last_peered":"2024-04-25T10:05:22.136198+0000","last_clean":"2024-04-25T10:05:22.136198+0000","last_became_active":"2024-04-25T10:05:21.070048+0000","last_became_peered":"2024-04-25T10:05:21.070048+0000","last_unstale":"2024-04-25T10:05:22.136198+0000","last_undegraded":"2024-04-25T10:05:22.136198+0000","last_fullsized":"2024-04-25T10:05:22.136198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:00.583322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134119+0000","last_change":"2024-04-25T10:05:21.080794+0000","last_active":"2024-04-25T10:05:22.134119+0000","last_peered":"2024-04-25T10:05:22.134119+0000","last_clean":"2024-04-25T10:05:22.134119+0000","last_became_active":"2024-04-25T10:05:21.079418+0000","last_became_peered":"2024-04-25T10:05:21.079418+0000","last_unstale":"2024-04-25T10:05:22.134119+0000","last_undegraded":"2024-04-25T10:05:22.134119+0000","last_fullsized":"2024-04-25T10:05:22.134119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:09.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042828+0000","last_change":"2024-04-25T10:05:21.055371+0000","last_active":"2024-04-25T10:05:22.042828+0000","last_peered":"2024-04-25T10:05:22.042828+0000","last_clean":"2024-04-25T10:05:22.042828+0000","last_became_active":"2024-04-25T10:05:21.054565+0000","last_became_peered":"2024-04-25T10:05:21.054565+0000","last_unstale":"2024-04-25T10:05:22.042828+0000","last_undegraded":"2024-04-25T10:05:22.042828+0000","last_fullsized":"2024-04-25T10:05:22.042828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:31.511781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046177+0000","last_change":"2024-04-25T10:05:21.079756+0000","last_active":"2024-04-25T10:05:22.046177+0000","last_peered":"2024-04-25T10:05:22.046177+0000","last_clean":"2024-04-25T10:05:22.046177+0000","last_became_active":"2024-04-25T10:05:21.078988+0000","last_became_peered":"2024-04-25T10:05:21.078988+0000","last_unstale":"2024-04-25T10:05:22.046177+0000","last_undegraded":"2024-04-25T10:05:22.046177+0000","last_fullsized":"2024-04-25T10:05:22.046177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:47.724037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.049473+0000","last_change":"2024-04-25T10:05:21.087275+0000","last_active":"2024-04-25T10:05:22.049473+0000","last_peered":"2024-04-25T10:05:22.049473+0000","last_clean":"2024-04-25T10:05:22.049473+0000","last_became_active":"2024-04-25T10:05:21.086912+0000","last_became_peered":"2024-04-25T10:05:21.086912+0000","last_unstale":"2024-04-25T10:05:22.049473+0000","last_undegraded":"2024-04-25T10:05:22.049473+0000","last_fullsized":"2024-04-25T10:05:22.049473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:01.067797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134822+0000","last_change":"2024-04-25T10:05:21.085565+0000","last_active":"2024-04-25T10:05:22.134822+0000","last_peered":"2024-04-25T10:05:22.134822+0000","last_clean":"2024-04-25T10:05:22.134822+0000","last_became_active":"2024-04-25T10:05:21.084852+0000","last_became_peered":"2024-04-25T10:05:21.084852+0000","last_unstale":"2024-04-25T10:05:22.134822+0000","last_undegraded":"2024-04-25T10:05:22.134822+0000","last_fullsized":"2024-04-25T10:05:22.134822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:51.076186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136820+0000","last_change":"2024-04-25T10:05:21.089683+0000","last_active":"2024-04-25T10:05:22.136820+0000","last_peered":"2024-04-25T10:05:22.136820+0000","last_clean":"2024-04-25T10:05:22.136820+0000","last_became_active":"2024-04-25T10:05:21.089192+0000","last_became_peered":"2024-04-25T10:05:21.089192+0000","last_unstale":"2024-04-25T10:05:22.136820+0000","last_undegraded":"2024-04-25T10:05:22.136820+0000","last_fullsized":"2024-04-25T10:05:22.136820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:50.393309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080406+0000","last_change":"2024-04-25T10:05:21.072923+0000","last_active":"2024-04-25T10:05:22.080406+0000","last_peered":"2024-04-25T10:05:22.080406+0000","last_clean":"2024-04-25T10:05:22.080406+0000","last_became_active":"2024-04-25T10:05:21.072600+0000","last_became_peered":"2024-04-25T10:05:21.072600+0000","last_unstale":"2024-04-25T10:05:22.080406+0000","last_undegraded":"2024-04-25T10:05:22.080406+0000","last_fullsized":"2024-04-25T10:05:22.080406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:33.904915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135126+0000","last_change":"2024-04-25T10:05:21.092396+0000","last_active":"2024-04-25T10:05:22.135126+0000","last_peered":"2024-04-25T10:05:22.135126+0000","last_clean":"2024-04-25T10:05:22.135126+0000","last_became_active":"2024-04-25T10:05:21.092118+0000","last_became_peered":"2024-04-25T10:05:21.092118+0000","last_unstale":"2024-04-25T10:05:22.135126+0000","last_undegraded":"2024-04-25T10:05:22.135126+0000","last_fullsized":"2024-04-25T10:05:22.135126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:33.904915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047430+0000","last_change":"2024-04-25T10:05:21.079563+0000","last_active":"2024-04-25T10:05:22.047430+0000","last_peered":"2024-04-25T10:05:22.047430+0000","last_clean":"2024-04-25T10:05:22.047430+0000","last_became_active":"2024-04-25T10:05:21.079032+0000","last_became_peered":"2024-04-25T10:05:21.079032+0000","last_unstale":"2024-04-25T10:05:22.047430+0000","last_undegraded":"2024-04-25T10:05:22.047430+0000","last_fullsized":"2024-04-25T10:05:22.047430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:02.817716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135891+0000","last_change":"2024-04-25T10:05:21.071825+0000","last_active":"2024-04-25T10:05:22.135891+0000","last_peered":"2024-04-25T10:05:22.135891+0000","last_clean":"2024-04-25T10:05:22.135891+0000","last_became_active":"2024-04-25T10:05:21.071207+0000","last_became_peered":"2024-04-25T10:05:21.071207+0000","last_unstale":"2024-04-25T10:05:22.135891+0000","last_undegraded":"2024-04-25T10:05:22.135891+0000","last_fullsized":"2024-04-25T10:05:22.135891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:23.519531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046222+0000","last_change":"2024-04-25T10:05:21.083347+0000","last_active":"2024-04-25T10:05:22.046222+0000","last_peered":"2024-04-25T10:05:22.046222+0000","last_clean":"2024-04-25T10:05:22.046222+0000","last_became_active":"2024-04-25T10:05:21.082648+0000","last_became_peered":"2024-04-25T10:05:21.082648+0000","last_unstale":"2024-04-25T10:05:22.046222+0000","last_undegraded":"2024-04-25T10:05:22.046222+0000","last_fullsized":"2024-04-25T10:05:22.046222+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135886+0000","last_change":"2024-04-25T10:05:21.069597+0000","last_active":"2024-04-25T10:05:22.135886+0000","last_peered":"2024-04-25T10:05:22.135886+0000","last_clean":"2024-04-25T10:05:22.135886+0000","last_became_active":"2024-04-25T10:05:21.064771+0000","last_became_peered":"2024-04-25T10:05:21.064771+0000","last_unstale":"2024-04-25T10:05:22.135886+0000","last_undegraded":"2024-04-25T10:05:22.135886+0000","last_fullsized":"2024-04-25T10:05:22.135886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:36.529873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.081502+0000","last_change":"2024-04-25T10:05:21.090548+0000","last_active":"2024-04-25T10:05:22.081502+0000","last_peered":"2024-04-25T10:05:22.081502+0000","last_clean":"2024-04-25T10:05:22.081502+0000","last_became_active":"2024-04-25T10:05:21.090195+0000","last_became_peered":"2024-04-25T10:05:21.090195+0000","last_unstale":"2024-04-25T10:05:22.081502+0000","last_undegraded":"2024-04-25T10:05:22.081502+0000","last_fullsized":"2024-04-25T10:05:22.081502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:00.583322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043216+0000","last_change":"2024-04-25T10:05:21.078928+0000","last_active":"2024-04-25T10:05:22.043216+0000","last_peered":"2024-04-25T10:05:22.043216+0000","last_clean":"2024-04-25T10:05:22.043216+0000","last_became_active":"2024-04-25T10:05:21.078521+0000","last_became_peered":"2024-04-25T10:05:21.078521+0000","last_unstale":"2024-04-25T10:05:22.043216+0000","last_undegraded":"2024-04-25T10:05:22.043216+0000","last_fullsized":"2024-04-25T10:05:22.043216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:03.085314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136663+0000","last_change":"2024-04-25T10:05:21.063167+0000","last_active":"2024-04-25T10:05:22.136663+0000","last_peered":"2024-04-25T10:05:22.136663+0000","last_clean":"2024-04-25T10:05:22.136663+0000","last_became_active":"2024-04-25T10:05:21.061503+0000","last_became_peered":"2024-04-25T10:05:21.061503+0000","last_unstale":"2024-04-25T10:05:22.136663+0000","last_undegraded":"2024-04-25T10:05:22.136663+0000","last_fullsized":"2024-04-25T10:05:22.136663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:30.299159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135212+0000","last_change":"2024-04-25T10:05:21.080730+0000","last_active":"2024-04-25T10:05:22.135212+0000","last_peered":"2024-04-25T10:05:22.135212+0000","last_clean":"2024-04-25T10:05:22.135212+0000","last_became_active":"2024-04-25T10:05:21.080248+0000","last_became_peered":"2024-04-25T10:05:21.080248+0000","last_unstale":"2024-04-25T10:05:22.135212+0000","last_undegraded":"2024-04-25T10:05:22.135212+0000","last_fullsized":"2024-04-25T10:05:22.135212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:36.529873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047524+0000","last_change":"2024-04-25T10:05:21.083357+0000","last_active":"2024-04-25T10:05:22.047524+0000","last_peered":"2024-04-25T10:05:22.047524+0000","last_clean":"2024-04-25T10:05:22.047524+0000","last_became_active":"2024-04-25T10:05:21.082923+0000","last_became_peered":"2024-04-25T10:05:21.082923+0000","last_unstale":"2024-04-25T10:05:22.047524+0000","last_undegraded":"2024-04-25T10:05:22.047524+0000","last_fullsized":"2024-04-25T10:05:22.047524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:50.797661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135432+0000","last_change":"2024-04-25T10:05:21.075647+0000","last_active":"2024-04-25T10:05:22.135432+0000","last_peered":"2024-04-25T10:05:22.135432+0000","last_clean":"2024-04-25T10:05:22.135432+0000","last_became_active":"2024-04-25T10:05:21.074954+0000","last_became_peered":"2024-04-25T10:05:21.074954+0000","last_unstale":"2024-04-25T10:05:22.135432+0000","last_undegraded":"2024-04-25T10:05:22.135432+0000","last_fullsized":"2024-04-25T10:05:22.135432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:55.339069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.079882+0000","last_change":"2024-04-25T10:05:21.084060+0000","last_active":"2024-04-25T10:05:22.079882+0000","last_peered":"2024-04-25T10:05:22.079882+0000","last_clean":"2024-04-25T10:05:22.079882+0000","last_became_active":"2024-04-25T10:05:21.083506+0000","last_became_peered":"2024-04-25T10:05:21.083506+0000","last_unstale":"2024-04-25T10:05:22.079882+0000","last_undegraded":"2024-04-25T10:05:22.079882+0000","last_fullsized":"2024-04-25T10:05:22.079882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:19.908741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046660+0000","last_change":"2024-04-25T10:05:21.065678+0000","last_active":"2024-04-25T10:05:22.046660+0000","last_peered":"2024-04-25T10:05:22.046660+0000","last_clean":"2024-04-25T10:05:22.046660+0000","last_became_active":"2024-04-25T10:05:21.064718+0000","last_became_peered":"2024-04-25T10:05:21.064718+0000","last_unstale":"2024-04-25T10:05:22.046660+0000","last_undegraded":"2024-04-25T10:05:22.046660+0000","last_fullsized":"2024-04-25T10:05:22.046660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:03.887937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136502+0000","last_change":"2024-04-25T10:05:20.083026+0000","last_active":"2024-04-25T10:05:22.136502+0000","last_peered":"2024-04-25T10:05:22.136502+0000","last_clean":"2024-04-25T10:05:22.136502+0000","last_became_active":"2024-04-25T10:05:20.082702+0000","last_became_peered":"2024-04-25T10:05:20.082702+0000","last_unstale":"2024-04-25T10:05:22.136502+0000","last_undegraded":"2024-04-25T10:05:22.136502+0000","last_fullsized":"2024-04-25T10:05:22.136502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:35.252363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135477+0000","last_change":"2024-04-25T10:05:21.086317+0000","last_active":"2024-04-25T10:05:22.135477+0000","last_peered":"2024-04-25T10:05:22.135477+0000","last_clean":"2024-04-25T10:05:22.135477+0000","last_became_active":"2024-04-25T10:05:21.085802+0000","last_became_peered":"2024-04-25T10:05:21.085802+0000","last_unstale":"2024-04-25T10:05:22.135477+0000","last_undegraded":"2024-04-25T10:05:22.135477+0000","last_fullsized":"2024-04-25T10:05:22.135477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:00.583322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080039+0000","last_change":"2024-04-25T10:05:20.080501+0000","last_active":"2024-04-25T10:05:22.080039+0000","last_peered":"2024-04-25T10:05:22.080039+0000","last_clean":"2024-04-25T10:05:22.080039+0000","last_became_active":"2024-04-25T10:05:20.079392+0000","last_became_peered":"2024-04-25T10:05:20.079392+0000","last_unstale":"2024-04-25T10:05:22.080039+0000","last_undegraded":"2024-04-25T10:05:22.080039+0000","last_fullsized":"2024-04-25T10:05:22.080039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:44.794815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136074+0000","last_change":"2024-04-25T10:05:20.072870+0000","last_active":"2024-04-25T10:05:22.136074+0000","last_peered":"2024-04-25T10:05:22.136074+0000","last_clean":"2024-04-25T10:05:22.136074+0000","last_became_active":"2024-04-25T10:05:20.072608+0000","last_became_peered":"2024-04-25T10:05:20.072608+0000","last_unstale":"2024-04-25T10:05:22.136074+0000","last_undegraded":"2024-04-25T10:05:22.136074+0000","last_fullsized":"2024-04-25T10:05:22.136074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.133679+0000","last_change":"2024-04-25T10:05:21.078027+0000","last_active":"2024-04-25T10:05:22.133679+0000","last_peered":"2024-04-25T10:05:22.133679+0000","last_clean":"2024-04-25T10:05:22.133679+0000","last_became_active":"2024-04-25T10:05:21.076660+0000","last_became_peered":"2024-04-25T10:05:21.076660+0000","last_unstale":"2024-04-25T10:05:22.133679+0000","last_undegraded":"2024-04-25T10:05:22.133679+0000","last_fullsized":"2024-04-25T10:05:22.133679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:21.545921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046449+0000","last_change":"2024-04-25T10:05:20.071927+0000","last_active":"2024-04-25T10:05:22.046449+0000","last_peered":"2024-04-25T10:05:22.046449+0000","last_clean":"2024-04-25T10:05:22.046449+0000","last_became_active":"2024-04-25T10:05:20.071228+0000","last_became_peered":"2024-04-25T10:05:20.071228+0000","last_unstale":"2024-04-25T10:05:22.046449+0000","last_undegraded":"2024-04-25T10:05:22.046449+0000","last_fullsized":"2024-04-25T10:05:22.046449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:54.336058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042801+0000","last_change":"2024-04-25T10:05:21.073018+0000","last_active":"2024-04-25T10:05:22.042801+0000","last_peered":"2024-04-25T10:05:22.042801+0000","last_clean":"2024-04-25T10:05:22.042801+0000","last_became_active":"2024-04-25T10:05:21.072257+0000","last_became_peered":"2024-04-25T10:05:21.072257+0000","last_unstale":"2024-04-25T10:05:22.042801+0000","last_undegraded":"2024-04-25T10:05:22.042801+0000","last_fullsized":"2024-04-25T10:05:22.042801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:53.558925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080605+0000","last_change":"2024-04-25T10:05:20.080134+0000","last_active":"2024-04-25T10:05:22.080605+0000","last_peered":"2024-04-25T10:05:22.080605+0000","last_clean":"2024-04-25T10:05:22.080605+0000","last_became_active":"2024-04-25T10:05:20.079484+0000","last_became_peered":"2024-04-25T10:05:20.079484+0000","last_unstale":"2024-04-25T10:05:22.080605+0000","last_undegraded":"2024-04-25T10:05:22.080605+0000","last_fullsized":"2024-04-25T10:05:22.080605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048538+0000","last_change":"2024-04-25T10:05:21.079513+0000","last_active":"2024-04-25T10:05:22.048538+0000","last_peered":"2024-04-25T10:05:22.048538+0000","last_clean":"2024-04-25T10:05:22.048538+0000","last_became_active":"2024-04-25T10:05:21.078962+0000","last_became_peered":"2024-04-25T10:05:21.078962+0000","last_unstale":"2024-04-25T10:05:22.048538+0000","last_undegraded":"2024-04-25T10:05:22.048538+0000","last_fullsized":"2024-04-25T10:05:22.048538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:18.289495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.049115+0000","last_change":"2024-04-25T10:05:20.075486+0000","last_active":"2024-04-25T10:05:22.049115+0000","last_peered":"2024-04-25T10:05:22.049115+0000","last_clean":"2024-04-25T10:05:22.049115+0000","last_became_active":"2024-04-25T10:05:20.074735+0000","last_became_peered":"2024-04-25T10:05:20.074735+0000","last_unstale":"2024-04-25T10:05:22.049115+0000","last_undegraded":"2024-04-25T10:05:22.049115+0000","last_fullsized":"2024-04-25T10:05:22.049115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:34.961990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134412+0000","last_change":"2024-04-25T10:05:21.085752+0000","last_active":"2024-04-25T10:05:22.134412+0000","last_peered":"2024-04-25T10:05:22.134412+0000","last_clean":"2024-04-25T10:05:22.134412+0000","last_became_active":"2024-04-25T10:05:21.081588+0000","last_became_peered":"2024-04-25T10:05:21.081588+0000","last_unstale":"2024-04-25T10:05:22.134412+0000","last_undegraded":"2024-04-25T10:05:22.134412+0000","last_fullsized":"2024-04-25T10:05:22.134412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:13.318906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043378+0000","last_change":"2024-04-25T10:05:20.065094+0000","last_active":"2024-04-25T10:05:22.043378+0000","last_peered":"2024-04-25T10:05:22.043378+0000","last_clean":"2024-04-25T10:05:22.043378+0000","last_became_active":"2024-04-25T10:05:20.063987+0000","last_became_peered":"2024-04-25T10:05:20.063987+0000","last_unstale":"2024-04-25T10:05:22.043378+0000","last_undegraded":"2024-04-25T10:05:22.043378+0000","last_fullsized":"2024-04-25T10:05:22.043378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:16.201265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048473+0000","last_change":"2024-04-25T10:05:21.063419+0000","last_active":"2024-04-25T10:05:22.048473+0000","last_peered":"2024-04-25T10:05:22.048473+0000","last_clean":"2024-04-25T10:05:22.048473+0000","last_became_active":"2024-04-25T10:05:21.060544+0000","last_became_peered":"2024-04-25T10:05:21.060544+0000","last_unstale":"2024-04-25T10:05:22.048473+0000","last_undegraded":"2024-04-25T10:05:22.048473+0000","last_fullsized":"2024-04-25T10:05:22.048473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:07.266916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080356+0000","last_change":"2024-04-25T10:05:20.070471+0000","last_active":"2024-04-25T10:05:22.080356+0000","last_peered":"2024-04-25T10:05:22.080356+0000","last_clean":"2024-04-25T10:05:22.080356+0000","last_became_active":"2024-04-25T10:05:20.069692+0000","last_became_peered":"2024-04-25T10:05:20.069692+0000","last_unstale":"2024-04-25T10:05:22.080356+0000","last_undegraded":"2024-04-25T10:05:22.080356+0000","last_fullsized":"2024-04-25T10:05:22.080356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:56.779127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136463+0000","last_change":"2024-04-25T10:05:21.086200+0000","last_active":"2024-04-25T10:05:22.136463+0000","last_peered":"2024-04-25T10:05:22.136463+0000","last_clean":"2024-04-25T10:05:22.136463+0000","last_became_active":"2024-04-25T10:05:21.085624+0000","last_became_peered":"2024-04-25T10:05:21.085624+0000","last_unstale":"2024-04-25T10:05:22.136463+0000","last_undegraded":"2024-04-25T10:05:22.136463+0000","last_fullsized":"2024-04-25T10:05:22.136463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:44.226635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135516+0000","last_change":"2024-04-25T10:05:20.077783+0000","last_active":"2024-04-25T10:05:22.135516+0000","last_peered":"2024-04-25T10:05:22.135516+0000","last_clean":"2024-04-25T10:05:22.135516+0000","last_became_active":"2024-04-25T10:05:20.077301+0000","last_became_peered":"2024-04-25T10:05:20.077301+0000","last_unstale":"2024-04-25T10:05:22.135516+0000","last_undegraded":"2024-04-25T10:05:22.135516+0000","last_fullsized":"2024-04-25T10:05:22.135516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:44.794815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134343+0000","last_change":"2024-04-25T10:05:21.086058+0000","last_active":"2024-04-25T10:05:22.134343+0000","last_peered":"2024-04-25T10:05:22.134343+0000","last_clean":"2024-04-25T10:05:22.134343+0000","last_became_active":"2024-04-25T10:05:21.085660+0000","last_became_peered":"2024-04-25T10:05:21.085660+0000","last_unstale":"2024-04-25T10:05:22.134343+0000","last_undegraded":"2024-04-25T10:05:22.134343+0000","last_fullsized":"2024-04-25T10:05:22.134343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:19.908741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047701+0000","last_change":"2024-04-25T10:05:20.062806+0000","last_active":"2024-04-25T10:05:22.047701+0000","last_peered":"2024-04-25T10:05:22.047701+0000","last_clean":"2024-04-25T10:05:22.047701+0000","last_became_active":"2024-04-25T10:05:20.061013+0000","last_became_peered":"2024-04-25T10:05:20.061013+0000","last_unstale":"2024-04-25T10:05:22.047701+0000","last_undegraded":"2024-04-25T10:05:22.047701+0000","last_fullsized":"2024-04-25T10:05:22.047701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:07.439088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136199+0000","last_change":"2024-04-25T10:05:21.092958+0000","last_active":"2024-04-25T10:05:22.136199+0000","last_peered":"2024-04-25T10:05:22.136199+0000","last_clean":"2024-04-25T10:05:22.136199+0000","last_became_active":"2024-04-25T10:05:21.089856+0000","last_became_peered":"2024-04-25T10:05:21.089856+0000","last_unstale":"2024-04-25T10:05:22.136199+0000","last_undegraded":"2024-04-25T10:05:22.136199+0000","last_fullsized":"2024-04-25T10:05:22.136199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:03.477184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.079832+0000","last_change":"2024-04-25T10:05:20.080086+0000","last_active":"2024-04-25T10:05:22.079832+0000","last_peered":"2024-04-25T10:05:22.079832+0000","last_clean":"2024-04-25T10:05:22.079832+0000","last_became_active":"2024-04-25T10:05:20.079302+0000","last_became_peered":"2024-04-25T10:05:20.079302+0000","last_unstale":"2024-04-25T10:05:22.079832+0000","last_undegraded":"2024-04-25T10:05:22.079832+0000","last_fullsized":"2024-04-25T10:05:22.079832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:34.961990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.079883+0000","last_change":"2024-04-25T10:05:21.091338+0000","last_active":"2024-04-25T10:05:22.079883+0000","last_peered":"2024-04-25T10:05:22.079883+0000","last_clean":"2024-04-25T10:05:22.079883+0000","last_became_active":"2024-04-25T10:05:21.090998+0000","last_became_peered":"2024-04-25T10:05:21.090998+0000","last_unstale":"2024-04-25T10:05:22.079883+0000","last_undegraded":"2024-04-25T10:05:22.079883+0000","last_fullsized":"2024-04-25T10:05:22.079883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:13.318906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.133875+0000","last_change":"2024-04-25T10:05:20.063163+0000","last_active":"2024-04-25T10:05:22.133875+0000","last_peered":"2024-04-25T10:05:22.133875+0000","last_clean":"2024-04-25T10:05:22.133875+0000","last_became_active":"2024-04-25T10:05:20.062707+0000","last_became_peered":"2024-04-25T10:05:20.062707+0000","last_unstale":"2024-04-25T10:05:22.133875+0000","last_undegraded":"2024-04-25T10:05:22.133875+0000","last_fullsized":"2024-04-25T10:05:22.133875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:12.228316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135958+0000","last_change":"2024-04-25T10:05:21.088835+0000","last_active":"2024-04-25T10:05:22.135958+0000","last_peered":"2024-04-25T10:05:22.135958+0000","last_clean":"2024-04-25T10:05:22.135958+0000","last_became_active":"2024-04-25T10:05:21.088476+0000","last_became_peered":"2024-04-25T10:05:21.088476+0000","last_unstale":"2024-04-25T10:05:22.135958+0000","last_undegraded":"2024-04-25T10:05:22.135958+0000","last_fullsized":"2024-04-25T10:05:22.135958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:54.535451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048102+0000","last_change":"2024-04-25T10:05:20.066666+0000","last_active":"2024-04-25T10:05:22.048102+0000","last_peered":"2024-04-25T10:05:22.048102+0000","last_clean":"2024-04-25T10:05:22.048102+0000","last_became_active":"2024-04-25T10:05:20.065812+0000","last_became_peered":"2024-04-25T10:05:20.065812+0000","last_unstale":"2024-04-25T10:05:22.048102+0000","last_undegraded":"2024-04-25T10:05:22.048102+0000","last_fullsized":"2024-04-25T10:05:22.048102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:45.802025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136886+0000","last_change":"2024-04-25T10:05:21.089032+0000","last_active":"2024-04-25T10:05:22.136886+0000","last_peered":"2024-04-25T10:05:22.136886+0000","last_clean":"2024-04-25T10:05:22.136886+0000","last_became_active":"2024-04-25T10:05:21.088707+0000","last_became_peered":"2024-04-25T10:05:21.088707+0000","last_unstale":"2024-04-25T10:05:22.136886+0000","last_undegraded":"2024-04-25T10:05:22.136886+0000","last_fullsized":"2024-04-25T10:05:22.136886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:29.342928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135030+0000","last_change":"2024-04-25T10:05:20.056234+0000","last_active":"2024-04-25T10:05:22.135030+0000","last_peered":"2024-04-25T10:05:22.135030+0000","last_clean":"2024-04-25T10:05:22.135030+0000","last_became_active":"2024-04-25T10:05:20.055711+0000","last_became_peered":"2024-04-25T10:05:20.055711+0000","last_unstale":"2024-04-25T10:05:22.135030+0000","last_undegraded":"2024-04-25T10:05:22.135030+0000","last_fullsized":"2024-04-25T10:05:22.135030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:44.794815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137621+0000","last_change":"2024-04-25T10:05:21.097805+0000","last_active":"2024-04-25T10:05:22.137621+0000","last_peered":"2024-04-25T10:05:22.137621+0000","last_clean":"2024-04-25T10:05:22.137621+0000","last_became_active":"2024-04-25T10:05:21.097066+0000","last_became_peered":"2024-04-25T10:05:21.097066+0000","last_unstale":"2024-04-25T10:05:22.137621+0000","last_undegraded":"2024-04-25T10:05:22.137621+0000","last_fullsized":"2024-04-25T10:05:22.137621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:10.508421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043379+0000","last_change":"2024-04-25T10:05:20.063861+0000","last_active":"2024-04-25T10:05:22.043379+0000","last_peered":"2024-04-25T10:05:22.043379+0000","last_clean":"2024-04-25T10:05:22.043379+0000","last_became_active":"2024-04-25T10:05:20.062501+0000","last_became_peered":"2024-04-25T10:05:20.062501+0000","last_unstale":"2024-04-25T10:05:22.043379+0000","last_undegraded":"2024-04-25T10:05:22.043379+0000","last_fullsized":"2024-04-25T10:05:22.043379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:02.884927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.049234+0000","last_change":"2024-04-25T10:05:21.083111+0000","last_active":"2024-04-25T10:05:22.049234+0000","last_peered":"2024-04-25T10:05:22.049234+0000","last_clean":"2024-04-25T10:05:22.049234+0000","last_became_active":"2024-04-25T10:05:21.082321+0000","last_became_peered":"2024-04-25T10:05:21.082321+0000","last_unstale":"2024-04-25T10:05:22.049234+0000","last_undegraded":"2024-04-25T10:05:22.049234+0000","last_fullsized":"2024-04-25T10:05:22.049234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:02.305355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135030+0000","last_change":"2024-04-25T10:05:20.078355+0000","last_active":"2024-04-25T10:05:22.135030+0000","last_peered":"2024-04-25T10:05:22.135030+0000","last_clean":"2024-04-25T10:05:22.135030+0000","last_became_active":"2024-04-25T10:05:20.077677+0000","last_became_peered":"2024-04-25T10:05:20.077677+0000","last_unstale":"2024-04-25T10:05:22.135030+0000","last_undegraded":"2024-04-25T10:05:22.135030+0000","last_fullsized":"2024-04-25T10:05:22.135030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:35.252363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136520+0000","last_change":"2024-04-25T10:05:21.089183+0000","last_active":"2024-04-25T10:05:22.136520+0000","last_peered":"2024-04-25T10:05:22.136520+0000","last_clean":"2024-04-25T10:05:22.136520+0000","last_became_active":"2024-04-25T10:05:21.088768+0000","last_became_peered":"2024-04-25T10:05:21.088768+0000","last_unstale":"2024-04-25T10:05:22.136520+0000","last_undegraded":"2024-04-25T10:05:22.136520+0000","last_fullsized":"2024-04-25T10:05:22.136520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:06.419537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135576+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:05:22.135576+0000","last_peered":"2024-04-25T10:05:22.135576+0000","last_clean":"2024-04-25T10:05:22.135576+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:05:22.135576+0000","last_undegraded":"2024-04-25T10:05:22.135576+0000","last_fullsized":"2024-04-25T10:05:22.135576+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047506+0000","last_change":"2024-04-25T10:05:20.069430+0000","last_active":"2024-04-25T10:05:22.047506+0000","last_peered":"2024-04-25T10:05:22.047506+0000","last_clean":"2024-04-25T10:05:22.047506+0000","last_became_active":"2024-04-25T10:05:20.068755+0000","last_became_peered":"2024-04-25T10:05:20.068755+0000","last_unstale":"2024-04-25T10:05:22.047506+0000","last_undegraded":"2024-04-25T10:05:22.047506+0000","last_fullsized":"2024-04-25T10:05:22.047506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:56.779127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136220+0000","last_change":"2024-04-25T10:05:21.071861+0000","last_active":"2024-04-25T10:05:22.136220+0000","last_peered":"2024-04-25T10:05:22.136220+0000","last_clean":"2024-04-25T10:05:22.136220+0000","last_became_active":"2024-04-25T10:05:21.071182+0000","last_became_peered":"2024-04-25T10:05:21.071182+0000","last_unstale":"2024-04-25T10:05:22.136220+0000","last_undegraded":"2024-04-25T10:05:22.136220+0000","last_fullsized":"2024-04-25T10:05:22.136220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:17.144179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135408+0000","last_change":"2024-04-25T10:05:20.084051+0000","last_active":"2024-04-25T10:05:22.135408+0000","last_peered":"2024-04-25T10:05:22.135408+0000","last_clean":"2024-04-25T10:05:22.135408+0000","last_became_active":"2024-04-25T10:05:20.083563+0000","last_became_peered":"2024-04-25T10:05:20.083563+0000","last_unstale":"2024-04-25T10:05:22.135408+0000","last_undegraded":"2024-04-25T10:05:22.135408+0000","last_fullsized":"2024-04-25T10:05:22.135408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:20.779254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135004+0000","last_change":"2024-04-25T10:05:21.072824+0000","last_active":"2024-04-25T10:05:22.135004+0000","last_peered":"2024-04-25T10:05:22.135004+0000","last_clean":"2024-04-25T10:05:22.135004+0000","last_became_active":"2024-04-25T10:05:21.071691+0000","last_became_peered":"2024-04-25T10:05:21.071691+0000","last_unstale":"2024-04-25T10:05:22.135004+0000","last_undegraded":"2024-04-25T10:05:22.135004+0000","last_fullsized":"2024-04-25T10:05:22.135004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:21.545921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135509+0000","last_change":"2024-04-25T10:05:20.056953+0000","last_active":"2024-04-25T10:05:22.135509+0000","last_peered":"2024-04-25T10:05:22.135509+0000","last_clean":"2024-04-25T10:05:22.135509+0000","last_became_active":"2024-04-25T10:05:20.056056+0000","last_became_peered":"2024-04-25T10:05:20.056056+0000","last_unstale":"2024-04-25T10:05:22.135509+0000","last_undegraded":"2024-04-25T10:05:22.135509+0000","last_fullsized":"2024-04-25T10:05:22.135509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042576+0000","last_change":"2024-04-25T10:05:21.056503+0000","last_active":"2024-04-25T10:05:22.042576+0000","last_peered":"2024-04-25T10:05:22.042576+0000","last_clean":"2024-04-25T10:05:22.042576+0000","last_became_active":"2024-04-25T10:05:21.055664+0000","last_became_peered":"2024-04-25T10:05:21.055664+0000","last_unstale":"2024-04-25T10:05:22.042576+0000","last_undegraded":"2024-04-25T10:05:22.042576+0000","last_fullsized":"2024-04-25T10:05:22.042576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:20.349799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047779+0000","last_change":"2024-04-25T10:05:20.061869+0000","last_active":"2024-04-25T10:05:22.047779+0000","last_peered":"2024-04-25T10:05:22.047779+0000","last_clean":"2024-04-25T10:05:22.047779+0000","last_became_active":"2024-04-25T10:05:20.060232+0000","last_became_peered":"2024-04-25T10:05:20.060232+0000","last_unstale":"2024-04-25T10:05:22.047779+0000","last_undegraded":"2024-04-25T10:05:22.047779+0000","last_fullsized":"2024-04-25T10:05:22.047779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:50.073175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047317+0000","last_change":"2024-04-25T10:05:21.083202+0000","last_active":"2024-04-25T10:05:22.047317+0000","last_peered":"2024-04-25T10:05:22.047317+0000","last_clean":"2024-04-25T10:05:22.047317+0000","last_became_active":"2024-04-25T10:05:21.082857+0000","last_became_peered":"2024-04-25T10:05:21.082857+0000","last_unstale":"2024-04-25T10:05:22.047317+0000","last_undegraded":"2024-04-25T10:05:22.047317+0000","last_fullsized":"2024-04-25T10:05:22.047317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:49.496977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042931+0000","last_change":"2024-04-25T10:05:20.060703+0000","last_active":"2024-04-25T10:05:22.042931+0000","last_peered":"2024-04-25T10:05:22.042931+0000","last_clean":"2024-04-25T10:05:22.042931+0000","last_became_active":"2024-04-25T10:05:20.056864+0000","last_became_peered":"2024-04-25T10:05:20.056864+0000","last_unstale":"2024-04-25T10:05:22.042931+0000","last_undegraded":"2024-04-25T10:05:22.042931+0000","last_fullsized":"2024-04-25T10:05:22.042931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:58.129770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.049114+0000","last_change":"2024-04-25T10:05:21.081253+0000","last_active":"2024-04-25T10:05:22.049114+0000","last_peered":"2024-04-25T10:05:22.049114+0000","last_clean":"2024-04-25T10:05:22.049114+0000","last_became_active":"2024-04-25T10:05:21.080917+0000","last_became_peered":"2024-04-25T10:05:21.080917+0000","last_unstale":"2024-04-25T10:05:22.049114+0000","last_undegraded":"2024-04-25T10:05:22.049114+0000","last_fullsized":"2024-04-25T10:05:22.049114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:41.708275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136822+0000","last_change":"2024-04-25T10:05:20.089660+0000","last_active":"2024-04-25T10:05:22.136822+0000","last_peered":"2024-04-25T10:05:22.136822+0000","last_clean":"2024-04-25T10:05:22.136822+0000","last_became_active":"2024-04-25T10:05:20.088996+0000","last_became_peered":"2024-04-25T10:05:20.088996+0000","last_unstale":"2024-04-25T10:05:22.136822+0000","last_undegraded":"2024-04-25T10:05:22.136822+0000","last_fullsized":"2024-04-25T10:05:22.136822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:55.264435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042818+0000","last_change":"2024-04-25T10:05:21.061203+0000","last_active":"2024-04-25T10:05:22.042818+0000","last_peered":"2024-04-25T10:05:22.042818+0000","last_clean":"2024-04-25T10:05:22.042818+0000","last_became_active":"2024-04-25T10:05:21.060108+0000","last_became_peered":"2024-04-25T10:05:21.060108+0000","last_unstale":"2024-04-25T10:05:22.042818+0000","last_undegraded":"2024-04-25T10:05:22.042818+0000","last_fullsized":"2024-04-25T10:05:22.042818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:11.969876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.077357+0000","last_change":"2024-04-25T10:05:20.071770+0000","last_active":"2024-04-25T10:05:22.077357+0000","last_peered":"2024-04-25T10:05:22.077357+0000","last_clean":"2024-04-25T10:05:22.077357+0000","last_became_active":"2024-04-25T10:05:20.071460+0000","last_became_peered":"2024-04-25T10:05:20.071460+0000","last_unstale":"2024-04-25T10:05:22.077357+0000","last_undegraded":"2024-04-25T10:05:22.077357+0000","last_fullsized":"2024-04-25T10:05:22.077357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:20.779254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136359+0000","last_change":"2024-04-25T10:05:21.063585+0000","last_active":"2024-04-25T10:05:22.136359+0000","last_peered":"2024-04-25T10:05:22.136359+0000","last_clean":"2024-04-25T10:05:22.136359+0000","last_became_active":"2024-04-25T10:05:21.062481+0000","last_became_peered":"2024-04-25T10:05:21.062481+0000","last_unstale":"2024-04-25T10:05:22.136359+0000","last_undegraded":"2024-04-25T10:05:22.136359+0000","last_fullsized":"2024-04-25T10:05:22.136359+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:47.724037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080795+0000","last_change":"2024-04-25T10:05:20.086795+0000","last_active":"2024-04-25T10:05:22.080795+0000","last_peered":"2024-04-25T10:05:22.080795+0000","last_clean":"2024-04-25T10:05:22.080795+0000","last_became_active":"2024-04-25T10:05:20.086157+0000","last_became_peered":"2024-04-25T10:05:20.086157+0000","last_unstale":"2024-04-25T10:05:22.080795+0000","last_undegraded":"2024-04-25T10:05:22.080795+0000","last_fullsized":"2024-04-25T10:05:22.080795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:35.252363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135349+0000","last_change":"2024-04-25T10:05:21.080072+0000","last_active":"2024-04-25T10:05:22.135349+0000","last_peered":"2024-04-25T10:05:22.135349+0000","last_clean":"2024-04-25T10:05:22.135349+0000","last_became_active":"2024-04-25T10:05:21.077807+0000","last_became_peered":"2024-04-25T10:05:21.077807+0000","last_unstale":"2024-04-25T10:05:22.135349+0000","last_undegraded":"2024-04-25T10:05:22.135349+0000","last_fullsized":"2024-04-25T10:05:22.135349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:23.519531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135461+0000","last_change":"2024-04-25T10:05:20.074066+0000","last_active":"2024-04-25T10:05:22.135461+0000","last_peered":"2024-04-25T10:05:22.135461+0000","last_clean":"2024-04-25T10:05:22.135461+0000","last_became_active":"2024-04-25T10:05:20.073313+0000","last_became_peered":"2024-04-25T10:05:20.073313+0000","last_unstale":"2024-04-25T10:05:22.135461+0000","last_undegraded":"2024-04-25T10:05:22.135461+0000","last_fullsized":"2024-04-25T10:05:22.135461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:35.144583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047063+0000","last_change":"2024-04-25T10:05:21.070227+0000","last_active":"2024-04-25T10:05:22.047063+0000","last_peered":"2024-04-25T10:05:22.047063+0000","last_clean":"2024-04-25T10:05:22.047063+0000","last_became_active":"2024-04-25T10:05:21.069473+0000","last_became_peered":"2024-04-25T10:05:21.069473+0000","last_unstale":"2024-04-25T10:05:22.047063+0000","last_undegraded":"2024-04-25T10:05:22.047063+0000","last_fullsized":"2024-04-25T10:05:22.047063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:17.204276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043069+0000","last_change":"2024-04-25T10:05:20.073767+0000","last_active":"2024-04-25T10:05:22.043069+0000","last_peered":"2024-04-25T10:05:22.043069+0000","last_clean":"2024-04-25T10:05:22.043069+0000","last_became_active":"2024-04-25T10:05:20.073333+0000","last_became_peered":"2024-04-25T10:05:20.073333+0000","last_unstale":"2024-04-25T10:05:22.043069+0000","last_undegraded":"2024-04-25T10:05:22.043069+0000","last_fullsized":"2024-04-25T10:05:22.043069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:48.493967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135540+0000","last_change":"2024-04-25T10:05:21.070624+0000","last_active":"2024-04-25T10:05:22.135540+0000","last_peered":"2024-04-25T10:05:22.135540+0000","last_clean":"2024-04-25T10:05:22.135540+0000","last_became_active":"2024-04-25T10:05:21.065398+0000","last_became_peered":"2024-04-25T10:05:21.065398+0000","last_unstale":"2024-04-25T10:05:22.135540+0000","last_undegraded":"2024-04-25T10:05:22.135540+0000","last_fullsized":"2024-04-25T10:05:22.135540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:33.904915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043074+0000","last_change":"2024-04-25T10:05:20.055362+0000","last_active":"2024-04-25T10:05:22.043074+0000","last_peered":"2024-04-25T10:05:22.043074+0000","last_clean":"2024-04-25T10:05:22.043074+0000","last_became_active":"2024-04-25T10:05:20.052279+0000","last_became_peered":"2024-04-25T10:05:20.052279+0000","last_unstale":"2024-04-25T10:05:22.043074+0000","last_undegraded":"2024-04-25T10:05:22.043074+0000","last_fullsized":"2024-04-25T10:05:22.043074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:35.144583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137237+0000","last_change":"2024-04-25T10:05:21.075800+0000","last_active":"2024-04-25T10:05:22.137237+0000","last_peered":"2024-04-25T10:05:22.137237+0000","last_clean":"2024-04-25T10:05:22.137237+0000","last_became_active":"2024-04-25T10:05:21.075425+0000","last_became_peered":"2024-04-25T10:05:21.075425+0000","last_unstale":"2024-04-25T10:05:22.137237+0000","last_undegraded":"2024-04-25T10:05:22.137237+0000","last_fullsized":"2024-04-25T10:05:22.137237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:02.817716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136402+0000","last_change":"2024-04-25T10:05:20.085669+0000","last_active":"2024-04-25T10:05:22.136402+0000","last_peered":"2024-04-25T10:05:22.136402+0000","last_clean":"2024-04-25T10:05:22.136402+0000","last_became_active":"2024-04-25T10:05:20.084890+0000","last_became_peered":"2024-04-25T10:05:20.084890+0000","last_unstale":"2024-04-25T10:05:22.136402+0000","last_undegraded":"2024-04-25T10:05:22.136402+0000","last_fullsized":"2024-04-25T10:05:22.136402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:16.201265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080121+0000","last_change":"2024-04-25T10:05:21.072582+0000","last_active":"2024-04-25T10:05:22.080121+0000","last_peered":"2024-04-25T10:05:22.080121+0000","last_clean":"2024-04-25T10:05:22.080121+0000","last_became_active":"2024-04-25T10:05:21.071786+0000","last_became_peered":"2024-04-25T10:05:21.071786+0000","last_unstale":"2024-04-25T10:05:22.080121+0000","last_undegraded":"2024-04-25T10:05:22.080121+0000","last_fullsized":"2024-04-25T10:05:22.080121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:21.545921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048737+0000","last_change":"2024-04-25T10:05:21.069632+0000","last_active":"2024-04-25T10:05:22.048737+0000","last_peered":"2024-04-25T10:05:22.048737+0000","last_clean":"2024-04-25T10:05:22.048737+0000","last_became_active":"2024-04-25T10:05:21.068870+0000","last_became_peered":"2024-04-25T10:05:21.068870+0000","last_unstale":"2024-04-25T10:05:22.048737+0000","last_undegraded":"2024-04-25T10:05:22.048737+0000","last_fullsized":"2024-04-25T10:05:22.048737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:09.103889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136270+0000","last_change":"2024-04-25T10:05:20.085837+0000","last_active":"2024-04-25T10:05:22.136270+0000","last_peered":"2024-04-25T10:05:22.136270+0000","last_clean":"2024-04-25T10:05:22.136270+0000","last_became_active":"2024-04-25T10:05:20.085079+0000","last_became_peered":"2024-04-25T10:05:20.085079+0000","last_unstale":"2024-04-25T10:05:22.136270+0000","last_undegraded":"2024-04-25T10:05:22.136270+0000","last_fullsized":"2024-04-25T10:05:22.136270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:02.884927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048913+0000","last_change":"2024-04-25T10:05:21.073527+0000","last_active":"2024-04-25T10:05:22.048913+0000","last_peered":"2024-04-25T10:05:22.048913+0000","last_clean":"2024-04-25T10:05:22.048913+0000","last_became_active":"2024-04-25T10:05:21.072780+0000","last_became_peered":"2024-04-25T10:05:21.072780+0000","last_unstale":"2024-04-25T10:05:22.048913+0000","last_undegraded":"2024-04-25T10:05:22.048913+0000","last_fullsized":"2024-04-25T10:05:22.048913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:58.306898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136676+0000","last_change":"2024-04-25T10:05:20.088969+0000","last_active":"2024-04-25T10:05:22.136676+0000","last_peered":"2024-04-25T10:05:22.136676+0000","last_clean":"2024-04-25T10:05:22.136676+0000","last_became_active":"2024-04-25T10:05:20.088074+0000","last_became_peered":"2024-04-25T10:05:20.088074+0000","last_unstale":"2024-04-25T10:05:22.136676+0000","last_undegraded":"2024-04-25T10:05:22.136676+0000","last_fullsized":"2024-04-25T10:05:22.136676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:48.493967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134241+0000","last_change":"2024-04-25T10:05:21.081805+0000","last_active":"2024-04-25T10:05:22.134241+0000","last_peered":"2024-04-25T10:05:22.134241+0000","last_clean":"2024-04-25T10:05:22.134241+0000","last_became_active":"2024-04-25T10:05:21.081062+0000","last_became_peered":"2024-04-25T10:05:21.081062+0000","last_unstale":"2024-04-25T10:05:22.134241+0000","last_undegraded":"2024-04-25T10:05:22.134241+0000","last_fullsized":"2024-04-25T10:05:22.134241+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:55.635372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137045+0000","last_change":"2024-04-25T10:05:20.060475+0000","last_active":"2024-04-25T10:05:22.137045+0000","last_peered":"2024-04-25T10:05:22.137045+0000","last_clean":"2024-04-25T10:05:22.137045+0000","last_became_active":"2024-04-25T10:05:20.060153+0000","last_became_peered":"2024-04-25T10:05:20.060153+0000","last_unstale":"2024-04-25T10:05:22.137045+0000","last_undegraded":"2024-04-25T10:05:22.137045+0000","last_fullsized":"2024-04-25T10:05:22.137045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:12.228316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136396+0000","last_change":"2024-04-25T10:05:21.085394+0000","last_active":"2024-04-25T10:05:22.136396+0000","last_peered":"2024-04-25T10:05:22.136396+0000","last_clean":"2024-04-25T10:05:22.136396+0000","last_became_active":"2024-04-25T10:05:21.085122+0000","last_became_peered":"2024-04-25T10:05:21.085122+0000","last_unstale":"2024-04-25T10:05:22.136396+0000","last_undegraded":"2024-04-25T10:05:22.136396+0000","last_fullsized":"2024-04-25T10:05:22.136396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134727+0000","last_change":"2024-04-25T10:05:20.085225+0000","last_active":"2024-04-25T10:05:22.134727+0000","last_peered":"2024-04-25T10:05:22.134727+0000","last_clean":"2024-04-25T10:05:22.134727+0000","last_became_active":"2024-04-25T10:05:20.084907+0000","last_became_peered":"2024-04-25T10:05:20.084907+0000","last_unstale":"2024-04-25T10:05:22.134727+0000","last_undegraded":"2024-04-25T10:05:22.134727+0000","last_fullsized":"2024-04-25T10:05:22.134727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":233472,"data_stored":79812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":221184,"data_stored":78200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":78360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":827392,"data_stored":669206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:25.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:25.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67cb383640 1 Processor -- start 2024-04-25T10:05:25.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67cb383640 1 -- start start 2024-04-25T10:05:25.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67cb383640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67c415dd00 0x7f67c40b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67cb383640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f67c406ca60 0x7f67c40cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67cb383640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c406d080 con 0x7f67c40b3c90 2024-04-25T10:05:25.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67cb383640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c406d360 con 0x7f67c406ca60 2024-04-25T10:05:25.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67cb383640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c406d640 con 0x7f67c415dd00 2024-04-25T10:05:25.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67c9920640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f67c406ca60 0x7f67c40cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67c891e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f67c40b3c90 0x7f67c406c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55750/0 (socket says 172.21.15.60:55750) 2024-04-25T10:05:25.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67c891e640 1 -- 172.21.15.60:0/664481092 learned_addr learned my addr 172.21.15.60:0/664481092 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67c9920640 1 --2- 172.21.15.60:0/664481092 >> v2:172.21.15.60:3300/0 conn(0x7f67c406ca60 0x7f67c40cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45206/0 (socket says 172.21.15.60:45206) 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67c911f640 1 --2- 172.21.15.60:0/664481092 >> v2:172.21.15.100:3300/0 conn(0x7f67c415dd00 0x7f67c40b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67c9920640 1 -- 172.21.15.60:0/664481092 >> v2:172.21.15.100:3300/0 conn(0x7f67c415dd00 msgr2=0x7f67c40b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67c9920640 1 --2- 172.21.15.60:0/664481092 >> v2:172.21.15.100:3300/0 conn(0x7f67c415dd00 0x7f67c40b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67c9920640 1 -- 172.21.15.60:0/664481092 >> v1:172.21.15.60:6789/0 conn(0x7f67c40b3c90 legacy=0x7f67c406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67c9920640 1 -- 172.21.15.60:0/664481092 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c406d920 con 0x7f67c406ca60 2024-04-25T10:05:25.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.287+0000 7f67bbfff640 1 -- 172.21.15.60:0/664481092 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249353021 0 0) 0x7f67c406d080 con 0x7f67c40b3c90 2024-04-25T10:05:25.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.291+0000 7f67c9920640 1 --2- 172.21.15.60:0/664481092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c406ca60 0x7f67c40cc2e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f67c40cf0d0 tx=0x7f67bc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572fb5c09dcd6b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.291+0000 7f67bbfff640 1 -- 172.21.15.60:0/664481092 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc0b2070 con 0x7f67c406ca60 2024-04-25T10:05:25.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.291+0000 7f67bbfff640 1 -- 172.21.15.60:0/664481092 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67bc0a57f0 con 0x7f67c406ca60 2024-04-25T10:05:25.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.291+0000 7f67bbfff640 1 -- 172.21.15.60:0/664481092 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc0a5bc0 con 0x7f67c406ca60 2024-04-25T10:05:25.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.291+0000 7f67cb383640 1 -- 172.21.15.60:0/664481092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c406ca60 msgr2=0x7f67c40cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.291+0000 7f67cb383640 1 --2- 172.21.15.60:0/664481092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c406ca60 0x7f67c40cc2e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f67c40cf0d0 tx=0x7f67bc01c930 comp rx=0 tx=0).stop 2024-04-25T10:05:25.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.291+0000 7f67cb383640 1 -- 172.21.15.60:0/664481092 shutdown_connections 2024-04-25T10:05:25.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.291+0000 7f67cb383640 1 --2- 172.21.15.60:0/664481092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c406ca60 0x7f67c40cc2e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.291+0000 7f67cb383640 1 --2- 172.21.15.60:0/664481092 >> v2:172.21.15.100:3300/0 conn(0x7f67c415dd00 0x7f67c40b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.291+0000 7f67cb383640 1 -- 172.21.15.60:0/664481092 >> 172.21.15.60:0/664481092 conn(0x7f67c4066da0 msgr2=0x7f67c40671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.291+0000 7f67cb383640 1 -- 172.21.15.60:0/664481092 shutdown_connections 2024-04-25T10:05:25.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.291+0000 7f67cb383640 1 -- 172.21.15.60:0/664481092 wait complete. 2024-04-25T10:05:25.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.291+0000 7f67cb383640 1 Processor -- start 2024-04-25T10:05:25.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.291+0000 7f67cb383640 1 -- start start 2024-04-25T10:05:25.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67cb383640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c406ca60 0x7f67c40cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67cb383640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c40b3c90 0x7f67c41541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67cb383640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c415dd00 0x7f67c4154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67cb383640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c40cd400 con 0x7f67c40b3c90 2024-04-25T10:05:25.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67cb383640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c40cd280 con 0x7f67c406ca60 2024-04-25T10:05:25.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67cb383640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c40cd580 con 0x7f67c415dd00 2024-04-25T10:05:25.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67c891e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c40b3c90 0x7f67c41541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67c891e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c40b3c90 0x7f67c41541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45214/0 (socket says 172.21.15.60:45214) 2024-04-25T10:05:25.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67c891e640 1 -- 172.21.15.60:0/3804240168 learned_addr learned my addr 172.21.15.60:0/3804240168 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:25.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67c911f640 1 --2- 172.21.15.60:0/3804240168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c406ca60 0x7f67c40cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67c9920640 1 --2- 172.21.15.60:0/3804240168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c415dd00 0x7f67c4154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67c891e640 1 -- 172.21.15.60:0/3804240168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c415dd00 msgr2=0x7f67c4154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67c891e640 1 --2- 172.21.15.60:0/3804240168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c415dd00 0x7f67c4154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67c891e640 1 -- 172.21.15.60:0/3804240168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c406ca60 msgr2=0x7f67c40cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67c891e640 1 --2- 172.21.15.60:0/3804240168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c406ca60 0x7f67c40cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67c891e640 1 -- 172.21.15.60:0/3804240168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c4160ca0 con 0x7f67c40b3c90 2024-04-25T10:05:25.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67c9920640 1 --2- 172.21.15.60:0/3804240168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c415dd00 0x7f67c4154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:25.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67c891e640 1 --2- 172.21.15.60:0/3804240168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c40b3c90 0x7f67c41541b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f67b4005890 tx=0x7f67b400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67b9ffb640 1 -- 172.21.15.60:0/3804240168 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4017070 con 0x7f67c40b3c90 2024-04-25T10:05:25.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67b9ffb640 1 -- 172.21.15.60:0/3804240168 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b4005d10 con 0x7f67c40b3c90 2024-04-25T10:05:25.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67cb383640 1 -- 172.21.15.60:0/3804240168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c4160f80 con 0x7f67c40b3c90 2024-04-25T10:05:25.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67b9ffb640 1 -- 172.21.15.60:0/3804240168 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4007310 con 0x7f67c40b3c90 2024-04-25T10:05:25.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67cb383640 1 -- 172.21.15.60:0/3804240168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c40c6a10 con 0x7f67c40b3c90 2024-04-25T10:05:25.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67cb383640 1 -- 172.21.15.60:0/3804240168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c4074770 con 0x7f67c40b3c90 2024-04-25T10:05:25.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67b9ffb640 1 -- 172.21.15.60:0/3804240168 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67b401a070 con 0x7f67c40b3c90 2024-04-25T10:05:25.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67b9ffb640 1 --2- 172.21.15.60:0/3804240168 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67a8066f60 0x7f67a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67b9ffb640 1 -- 172.21.15.60:0/3804240168 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67b4084370 con 0x7f67c40b3c90 2024-04-25T10:05:25.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.295+0000 7f67c911f640 1 --2- 172.21.15.60:0/3804240168 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67a8066f60 0x7f67a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.299+0000 7f67c911f640 1 --2- 172.21.15.60:0/3804240168 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67a8066f60 0x7f67a8069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f67c000b930 tx=0x7f67c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.299+0000 7f67b9ffb640 1 -- 172.21.15.60:0/3804240168 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67b4055410 con 0x7f67c40b3c90 2024-04-25T10:05:25.454 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-04-25T10:05:25.445+0000 7f3a152e0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:05:25.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.451+0000 7f67cb383640 1 -- 172.21.15.60:0/3804240168 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67c40ce570 con 0x7f67a8066f60 2024-04-25T10:05:25.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.471+0000 7f67b9ffb640 1 -- 172.21.15.60:0/3804240168 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251946 (secure 0 0 0) 0x7f67c40ce570 con 0x7f67a8066f60 2024-04-25T10:05:25.480 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:25.482 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:05:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.475+0000 7f67cb383640 1 -- 172.21.15.60:0/3804240168 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67a8066f60 msgr2=0x7f67a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.475+0000 7f67cb383640 1 --2- 172.21.15.60:0/3804240168 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67a8066f60 0x7f67a8069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f67c000b930 tx=0x7f67c0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.475+0000 7f67cb383640 1 -- 172.21.15.60:0/3804240168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c40b3c90 msgr2=0x7f67c41541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.475+0000 7f67cb383640 1 --2- 172.21.15.60:0/3804240168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c40b3c90 0x7f67c41541b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f67b4005890 tx=0x7f67b400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.475+0000 7f67cb383640 1 -- 172.21.15.60:0/3804240168 shutdown_connections 2024-04-25T10:05:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.475+0000 7f67cb383640 1 --2- 172.21.15.60:0/3804240168 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67a8066f60 0x7f67a8069420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.475+0000 7f67cb383640 1 --2- 172.21.15.60:0/3804240168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c415dd00 0x7f67c4154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.475+0000 7f67cb383640 1 --2- 172.21.15.60:0/3804240168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c40b3c90 0x7f67c41541b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.475+0000 7f67cb383640 1 --2- 172.21.15.60:0/3804240168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c406ca60 0x7f67c40cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.475+0000 7f67cb383640 1 -- 172.21.15.60:0/3804240168 >> 172.21.15.60:0/3804240168 conn(0x7f67c4066da0 msgr2=0x7f67c40b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.475+0000 7f67cb383640 1 -- 172.21.15.60:0/3804240168 shutdown_connections 2024-04-25T10:05:25.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.475+0000 7f67cb383640 1 -- 172.21.15.60:0/3804240168 wait complete. 2024-04-25T10:05:25.497 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T10:05:24.386913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235768,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738952,"statfs":{"total":773094113280,"available":772852686848,"internally_reserved":0,"allocated":3022848,"data_stored":1811156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.353600"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137405+0000","last_change":"2024-04-25T10:05:21.093662+0000","last_active":"2024-04-25T10:05:22.137405+0000","last_peered":"2024-04-25T10:05:22.137405+0000","last_clean":"2024-04-25T10:05:22.137405+0000","last_became_active":"2024-04-25T10:05:21.093342+0000","last_became_peered":"2024-04-25T10:05:21.093342+0000","last_unstale":"2024-04-25T10:05:22.137405+0000","last_undegraded":"2024-04-25T10:05:22.137405+0000","last_fullsized":"2024-04-25T10:05:22.137405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:38.452914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043321+0000","last_change":"2024-04-25T10:05:21.076738+0000","last_active":"2024-04-25T10:05:22.043321+0000","last_peered":"2024-04-25T10:05:22.043321+0000","last_clean":"2024-04-25T10:05:22.043321+0000","last_became_active":"2024-04-25T10:05:21.076085+0000","last_became_peered":"2024-04-25T10:05:21.076085+0000","last_unstale":"2024-04-25T10:05:22.043321+0000","last_undegraded":"2024-04-25T10:05:22.043321+0000","last_fullsized":"2024-04-25T10:05:22.043321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:36.826815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136181+0000","last_change":"2024-04-25T10:05:21.072290+0000","last_active":"2024-04-25T10:05:22.136181+0000","last_peered":"2024-04-25T10:05:22.136181+0000","last_clean":"2024-04-25T10:05:22.136181+0000","last_became_active":"2024-04-25T10:05:21.069763+0000","last_became_peered":"2024-04-25T10:05:21.069763+0000","last_unstale":"2024-04-25T10:05:22.136181+0000","last_undegraded":"2024-04-25T10:05:22.136181+0000","last_fullsized":"2024-04-25T10:05:22.136181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:17.144179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048606+0000","last_change":"2024-04-25T10:05:21.083631+0000","last_active":"2024-04-25T10:05:22.048606+0000","last_peered":"2024-04-25T10:05:22.048606+0000","last_clean":"2024-04-25T10:05:22.048606+0000","last_became_active":"2024-04-25T10:05:21.082783+0000","last_became_peered":"2024-04-25T10:05:21.082783+0000","last_unstale":"2024-04-25T10:05:22.048606+0000","last_undegraded":"2024-04-25T10:05:22.048606+0000","last_fullsized":"2024-04-25T10:05:22.048606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:02.040066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136043+0000","last_change":"2024-04-25T10:05:21.092602+0000","last_active":"2024-04-25T10:05:22.136043+0000","last_peered":"2024-04-25T10:05:22.136043+0000","last_clean":"2024-04-25T10:05:22.136043+0000","last_became_active":"2024-04-25T10:05:21.089559+0000","last_became_peered":"2024-04-25T10:05:21.089559+0000","last_unstale":"2024-04-25T10:05:22.136043+0000","last_undegraded":"2024-04-25T10:05:22.136043+0000","last_fullsized":"2024-04-25T10:05:22.136043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:57.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046962+0000","last_change":"2024-04-25T10:05:21.078938+0000","last_active":"2024-04-25T10:05:22.046962+0000","last_peered":"2024-04-25T10:05:22.046962+0000","last_clean":"2024-04-25T10:05:22.046962+0000","last_became_active":"2024-04-25T10:05:21.078310+0000","last_became_peered":"2024-04-25T10:05:21.078310+0000","last_unstale":"2024-04-25T10:05:22.046962+0000","last_undegraded":"2024-04-25T10:05:22.046962+0000","last_fullsized":"2024-04-25T10:05:22.046962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:30.299159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135741+0000","last_change":"2024-04-25T10:05:21.086161+0000","last_active":"2024-04-25T10:05:22.135741+0000","last_peered":"2024-04-25T10:05:22.135741+0000","last_clean":"2024-04-25T10:05:22.135741+0000","last_became_active":"2024-04-25T10:05:21.084217+0000","last_became_peered":"2024-04-25T10:05:21.084217+0000","last_unstale":"2024-04-25T10:05:22.135741+0000","last_undegraded":"2024-04-25T10:05:22.135741+0000","last_fullsized":"2024-04-25T10:05:22.135741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:29.451932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.133417+0000","last_change":"2024-04-25T10:05:21.071738+0000","last_active":"2024-04-25T10:05:22.133417+0000","last_peered":"2024-04-25T10:05:22.133417+0000","last_clean":"2024-04-25T10:05:22.133417+0000","last_became_active":"2024-04-25T10:05:21.071239+0000","last_became_peered":"2024-04-25T10:05:21.071239+0000","last_unstale":"2024-04-25T10:05:22.133417+0000","last_undegraded":"2024-04-25T10:05:22.133417+0000","last_fullsized":"2024-04-25T10:05:22.133417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:13.318906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137424+0000","last_change":"2024-04-25T10:05:21.089854+0000","last_active":"2024-04-25T10:05:22.137424+0000","last_peered":"2024-04-25T10:05:22.137424+0000","last_clean":"2024-04-25T10:05:22.137424+0000","last_became_active":"2024-04-25T10:05:21.089145+0000","last_became_peered":"2024-04-25T10:05:21.089145+0000","last_unstale":"2024-04-25T10:05:22.137424+0000","last_undegraded":"2024-04-25T10:05:22.137424+0000","last_fullsized":"2024-04-25T10:05:22.137424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:17.168463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.081237+0000","last_change":"2024-04-25T10:05:21.081763+0000","last_active":"2024-04-25T10:05:22.081237+0000","last_peered":"2024-04-25T10:05:22.081237+0000","last_clean":"2024-04-25T10:05:22.081237+0000","last_became_active":"2024-04-25T10:05:21.081398+0000","last_became_peered":"2024-04-25T10:05:21.081398+0000","last_unstale":"2024-04-25T10:05:22.081237+0000","last_undegraded":"2024-04-25T10:05:22.081237+0000","last_fullsized":"2024-04-25T10:05:22.081237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:23.519531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080566+0000","last_change":"2024-04-25T10:05:21.081165+0000","last_active":"2024-04-25T10:05:22.080566+0000","last_peered":"2024-04-25T10:05:22.080566+0000","last_clean":"2024-04-25T10:05:22.080566+0000","last_became_active":"2024-04-25T10:05:21.076196+0000","last_became_peered":"2024-04-25T10:05:21.076196+0000","last_unstale":"2024-04-25T10:05:22.080566+0000","last_undegraded":"2024-04-25T10:05:22.080566+0000","last_fullsized":"2024-04-25T10:05:22.080566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:36.529873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048072+0000","last_change":"2024-04-25T10:05:21.082653+0000","last_active":"2024-04-25T10:05:22.048072+0000","last_peered":"2024-04-25T10:05:22.048072+0000","last_clean":"2024-04-25T10:05:22.048072+0000","last_became_active":"2024-04-25T10:05:21.082004+0000","last_became_peered":"2024-04-25T10:05:21.082004+0000","last_unstale":"2024-04-25T10:05:22.048072+0000","last_undegraded":"2024-04-25T10:05:22.048072+0000","last_fullsized":"2024-04-25T10:05:22.048072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:56.267445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136198+0000","last_change":"2024-04-25T10:05:21.071024+0000","last_active":"2024-04-25T10:05:22.136198+0000","last_peered":"2024-04-25T10:05:22.136198+0000","last_clean":"2024-04-25T10:05:22.136198+0000","last_became_active":"2024-04-25T10:05:21.070048+0000","last_became_peered":"2024-04-25T10:05:21.070048+0000","last_unstale":"2024-04-25T10:05:22.136198+0000","last_undegraded":"2024-04-25T10:05:22.136198+0000","last_fullsized":"2024-04-25T10:05:22.136198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:00.583322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134119+0000","last_change":"2024-04-25T10:05:21.080794+0000","last_active":"2024-04-25T10:05:22.134119+0000","last_peered":"2024-04-25T10:05:22.134119+0000","last_clean":"2024-04-25T10:05:22.134119+0000","last_became_active":"2024-04-25T10:05:21.079418+0000","last_became_peered":"2024-04-25T10:05:21.079418+0000","last_unstale":"2024-04-25T10:05:22.134119+0000","last_undegraded":"2024-04-25T10:05:22.134119+0000","last_fullsized":"2024-04-25T10:05:22.134119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:09.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042828+0000","last_change":"2024-04-25T10:05:21.055371+0000","last_active":"2024-04-25T10:05:22.042828+0000","last_peered":"2024-04-25T10:05:22.042828+0000","last_clean":"2024-04-25T10:05:22.042828+0000","last_became_active":"2024-04-25T10:05:21.054565+0000","last_became_peered":"2024-04-25T10:05:21.054565+0000","last_unstale":"2024-04-25T10:05:22.042828+0000","last_undegraded":"2024-04-25T10:05:22.042828+0000","last_fullsized":"2024-04-25T10:05:22.042828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:31.511781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046177+0000","last_change":"2024-04-25T10:05:21.079756+0000","last_active":"2024-04-25T10:05:22.046177+0000","last_peered":"2024-04-25T10:05:22.046177+0000","last_clean":"2024-04-25T10:05:22.046177+0000","last_became_active":"2024-04-25T10:05:21.078988+0000","last_became_peered":"2024-04-25T10:05:21.078988+0000","last_unstale":"2024-04-25T10:05:22.046177+0000","last_undegraded":"2024-04-25T10:05:22.046177+0000","last_fullsized":"2024-04-25T10:05:22.046177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:47.724037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.049473+0000","last_change":"2024-04-25T10:05:21.087275+0000","last_active":"2024-04-25T10:05:22.049473+0000","last_peered":"2024-04-25T10:05:22.049473+0000","last_clean":"2024-04-25T10:05:22.049473+0000","last_became_active":"2024-04-25T10:05:21.086912+0000","last_became_peered":"2024-04-25T10:05:21.086912+0000","last_unstale":"2024-04-25T10:05:22.049473+0000","last_undegraded":"2024-04-25T10:05:22.049473+0000","last_fullsized":"2024-04-25T10:05:22.049473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:01.067797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134822+0000","last_change":"2024-04-25T10:05:21.085565+0000","last_active":"2024-04-25T10:05:22.134822+0000","last_peered":"2024-04-25T10:05:22.134822+0000","last_clean":"2024-04-25T10:05:22.134822+0000","last_became_active":"2024-04-25T10:05:21.084852+0000","last_became_peered":"2024-04-25T10:05:21.084852+0000","last_unstale":"2024-04-25T10:05:22.134822+0000","last_undegraded":"2024-04-25T10:05:22.134822+0000","last_fullsized":"2024-04-25T10:05:22.134822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:51.076186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136820+0000","last_change":"2024-04-25T10:05:21.089683+0000","last_active":"2024-04-25T10:05:22.136820+0000","last_peered":"2024-04-25T10:05:22.136820+0000","last_clean":"2024-04-25T10:05:22.136820+0000","last_became_active":"2024-04-25T10:05:21.089192+0000","last_became_peered":"2024-04-25T10:05:21.089192+0000","last_unstale":"2024-04-25T10:05:22.136820+0000","last_undegraded":"2024-04-25T10:05:22.136820+0000","last_fullsized":"2024-04-25T10:05:22.136820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:50.393309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080406+0000","last_change":"2024-04-25T10:05:21.072923+0000","last_active":"2024-04-25T10:05:22.080406+0000","last_peered":"2024-04-25T10:05:22.080406+0000","last_clean":"2024-04-25T10:05:22.080406+0000","last_became_active":"2024-04-25T10:05:21.072600+0000","last_became_peered":"2024-04-25T10:05:21.072600+0000","last_unstale":"2024-04-25T10:05:22.080406+0000","last_undegraded":"2024-04-25T10:05:22.080406+0000","last_fullsized":"2024-04-25T10:05:22.080406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:33.904915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135126+0000","last_change":"2024-04-25T10:05:21.092396+0000","last_active":"2024-04-25T10:05:22.135126+0000","last_peered":"2024-04-25T10:05:22.135126+0000","last_clean":"2024-04-25T10:05:22.135126+0000","last_became_active":"2024-04-25T10:05:21.092118+0000","last_became_peered":"2024-04-25T10:05:21.092118+0000","last_unstale":"2024-04-25T10:05:22.135126+0000","last_undegraded":"2024-04-25T10:05:22.135126+0000","last_fullsized":"2024-04-25T10:05:22.135126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:33.904915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047430+0000","last_change":"2024-04-25T10:05:21.079563+0000","last_active":"2024-04-25T10:05:22.047430+0000","last_peered":"2024-04-25T10:05:22.047430+0000","last_clean":"2024-04-25T10:05:22.047430+0000","last_became_active":"2024-04-25T10:05:21.079032+0000","last_became_peered":"2024-04-25T10:05:21.079032+0000","last_unstale":"2024-04-25T10:05:22.047430+0000","last_undegraded":"2024-04-25T10:05:22.047430+0000","last_fullsized":"2024-04-25T10:05:22.047430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:02.817716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135891+0000","last_change":"2024-04-25T10:05:21.071825+0000","last_active":"2024-04-25T10:05:22.135891+0000","last_peered":"2024-04-25T10:05:22.135891+0000","last_clean":"2024-04-25T10:05:22.135891+0000","last_became_active":"2024-04-25T10:05:21.071207+0000","last_became_peered":"2024-04-25T10:05:21.071207+0000","last_unstale":"2024-04-25T10:05:22.135891+0000","last_undegraded":"2024-04-25T10:05:22.135891+0000","last_fullsized":"2024-04-25T10:05:22.135891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:23.519531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046222+0000","last_change":"2024-04-25T10:05:21.083347+0000","last_active":"2024-04-25T10:05:22.046222+0000","last_peered":"2024-04-25T10:05:22.046222+0000","last_clean":"2024-04-25T10:05:22.046222+0000","last_became_active":"2024-04-25T10:05:21.082648+0000","last_became_peered":"2024-04-25T10:05:21.082648+0000","last_unstale":"2024-04-25T10:05:22.046222+0000","last_undegraded":"2024-04-25T10:05:22.046222+0000","last_fullsized":"2024-04-25T10:05:22.046222+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135886+0000","last_change":"2024-04-25T10:05:21.069597+0000","last_active":"2024-04-25T10:05:22.135886+0000","last_peered":"2024-04-25T10:05:22.135886+0000","last_clean":"2024-04-25T10:05:22.135886+0000","last_became_active":"2024-04-25T10:05:21.064771+0000","last_became_peered":"2024-04-25T10:05:21.064771+0000","last_unstale":"2024-04-25T10:05:22.135886+0000","last_undegraded":"2024-04-25T10:05:22.135886+0000","last_fullsized":"2024-04-25T10:05:22.135886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:36.529873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.081502+0000","last_change":"2024-04-25T10:05:21.090548+0000","last_active":"2024-04-25T10:05:22.081502+0000","last_peered":"2024-04-25T10:05:22.081502+0000","last_clean":"2024-04-25T10:05:22.081502+0000","last_became_active":"2024-04-25T10:05:21.090195+0000","last_became_peered":"2024-04-25T10:05:21.090195+0000","last_unstale":"2024-04-25T10:05:22.081502+0000","last_undegraded":"2024-04-25T10:05:22.081502+0000","last_fullsized":"2024-04-25T10:05:22.081502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:00.583322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043216+0000","last_change":"2024-04-25T10:05:21.078928+0000","last_active":"2024-04-25T10:05:22.043216+0000","last_peered":"2024-04-25T10:05:22.043216+0000","last_clean":"2024-04-25T10:05:22.043216+0000","last_became_active":"2024-04-25T10:05:21.078521+0000","last_became_peered":"2024-04-25T10:05:21.078521+0000","last_unstale":"2024-04-25T10:05:22.043216+0000","last_undegraded":"2024-04-25T10:05:22.043216+0000","last_fullsized":"2024-04-25T10:05:22.043216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:03.085314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136663+0000","last_change":"2024-04-25T10:05:21.063167+0000","last_active":"2024-04-25T10:05:22.136663+0000","last_peered":"2024-04-25T10:05:22.136663+0000","last_clean":"2024-04-25T10:05:22.136663+0000","last_became_active":"2024-04-25T10:05:21.061503+0000","last_became_peered":"2024-04-25T10:05:21.061503+0000","last_unstale":"2024-04-25T10:05:22.136663+0000","last_undegraded":"2024-04-25T10:05:22.136663+0000","last_fullsized":"2024-04-25T10:05:22.136663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:30.299159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135212+0000","last_change":"2024-04-25T10:05:21.080730+0000","last_active":"2024-04-25T10:05:22.135212+0000","last_peered":"2024-04-25T10:05:22.135212+0000","last_clean":"2024-04-25T10:05:22.135212+0000","last_became_active":"2024-04-25T10:05:21.080248+0000","last_became_peered":"2024-04-25T10:05:21.080248+0000","last_unstale":"2024-04-25T10:05:22.135212+0000","last_undegraded":"2024-04-25T10:05:22.135212+0000","last_fullsized":"2024-04-25T10:05:22.135212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:36.529873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047524+0000","last_change":"2024-04-25T10:05:21.083357+0000","last_active":"2024-04-25T10:05:22.047524+0000","last_peered":"2024-04-25T10:05:22.047524+0000","last_clean":"2024-04-25T10:05:22.047524+0000","last_became_active":"2024-04-25T10:05:21.082923+0000","last_became_peered":"2024-04-25T10:05:21.082923+0000","last_unstale":"2024-04-25T10:05:22.047524+0000","last_undegraded":"2024-04-25T10:05:22.047524+0000","last_fullsized":"2024-04-25T10:05:22.047524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:50.797661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135432+0000","last_change":"2024-04-25T10:05:21.075647+0000","last_active":"2024-04-25T10:05:22.135432+0000","last_peered":"2024-04-25T10:05:22.135432+0000","last_clean":"2024-04-25T10:05:22.135432+0000","last_became_active":"2024-04-25T10:05:21.074954+0000","last_became_peered":"2024-04-25T10:05:21.074954+0000","last_unstale":"2024-04-25T10:05:22.135432+0000","last_undegraded":"2024-04-25T10:05:22.135432+0000","last_fullsized":"2024-04-25T10:05:22.135432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:55.339069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.079882+0000","last_change":"2024-04-25T10:05:21.084060+0000","last_active":"2024-04-25T10:05:22.079882+0000","last_peered":"2024-04-25T10:05:22.079882+0000","last_clean":"2024-04-25T10:05:22.079882+0000","last_became_active":"2024-04-25T10:05:21.083506+0000","last_became_peered":"2024-04-25T10:05:21.083506+0000","last_unstale":"2024-04-25T10:05:22.079882+0000","last_undegraded":"2024-04-25T10:05:22.079882+0000","last_fullsized":"2024-04-25T10:05:22.079882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:19.908741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046660+0000","last_change":"2024-04-25T10:05:21.065678+0000","last_active":"2024-04-25T10:05:22.046660+0000","last_peered":"2024-04-25T10:05:22.046660+0000","last_clean":"2024-04-25T10:05:22.046660+0000","last_became_active":"2024-04-25T10:05:21.064718+0000","last_became_peered":"2024-04-25T10:05:21.064718+0000","last_unstale":"2024-04-25T10:05:22.046660+0000","last_undegraded":"2024-04-25T10:05:22.046660+0000","last_fullsized":"2024-04-25T10:05:22.046660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:03.887937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136502+0000","last_change":"2024-04-25T10:05:20.083026+0000","last_active":"2024-04-25T10:05:22.136502+0000","last_peered":"2024-04-25T10:05:22.136502+0000","last_clean":"2024-04-25T10:05:22.136502+0000","last_became_active":"2024-04-25T10:05:20.082702+0000","last_became_peered":"2024-04-25T10:05:20.082702+0000","last_unstale":"2024-04-25T10:05:22.136502+0000","last_undegraded":"2024-04-25T10:05:22.136502+0000","last_fullsized":"2024-04-25T10:05:22.136502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:35.252363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135477+0000","last_change":"2024-04-25T10:05:21.086317+0000","last_active":"2024-04-25T10:05:22.135477+0000","last_peered":"2024-04-25T10:05:22.135477+0000","last_clean":"2024-04-25T10:05:22.135477+0000","last_became_active":"2024-04-25T10:05:21.085802+0000","last_became_peered":"2024-04-25T10:05:21.085802+0000","last_unstale":"2024-04-25T10:05:22.135477+0000","last_undegraded":"2024-04-25T10:05:22.135477+0000","last_fullsized":"2024-04-25T10:05:22.135477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:00.583322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080039+0000","last_change":"2024-04-25T10:05:20.080501+0000","last_active":"2024-04-25T10:05:22.080039+0000","last_peered":"2024-04-25T10:05:22.080039+0000","last_clean":"2024-04-25T10:05:22.080039+0000","last_became_active":"2024-04-25T10:05:20.079392+0000","last_became_peered":"2024-04-25T10:05:20.079392+0000","last_unstale":"2024-04-25T10:05:22.080039+0000","last_undegraded":"2024-04-25T10:05:22.080039+0000","last_fullsized":"2024-04-25T10:05:22.080039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:44.794815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136074+0000","last_change":"2024-04-25T10:05:20.072870+0000","last_active":"2024-04-25T10:05:22.136074+0000","last_peered":"2024-04-25T10:05:22.136074+0000","last_clean":"2024-04-25T10:05:22.136074+0000","last_became_active":"2024-04-25T10:05:20.072608+0000","last_became_peered":"2024-04-25T10:05:20.072608+0000","last_unstale":"2024-04-25T10:05:22.136074+0000","last_undegraded":"2024-04-25T10:05:22.136074+0000","last_fullsized":"2024-04-25T10:05:22.136074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.133679+0000","last_change":"2024-04-25T10:05:21.078027+0000","last_active":"2024-04-25T10:05:22.133679+0000","last_peered":"2024-04-25T10:05:22.133679+0000","last_clean":"2024-04-25T10:05:22.133679+0000","last_became_active":"2024-04-25T10:05:21.076660+0000","last_became_peered":"2024-04-25T10:05:21.076660+0000","last_unstale":"2024-04-25T10:05:22.133679+0000","last_undegraded":"2024-04-25T10:05:22.133679+0000","last_fullsized":"2024-04-25T10:05:22.133679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:21.545921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046449+0000","last_change":"2024-04-25T10:05:20.071927+0000","last_active":"2024-04-25T10:05:22.046449+0000","last_peered":"2024-04-25T10:05:22.046449+0000","last_clean":"2024-04-25T10:05:22.046449+0000","last_became_active":"2024-04-25T10:05:20.071228+0000","last_became_peered":"2024-04-25T10:05:20.071228+0000","last_unstale":"2024-04-25T10:05:22.046449+0000","last_undegraded":"2024-04-25T10:05:22.046449+0000","last_fullsized":"2024-04-25T10:05:22.046449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:54.336058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042801+0000","last_change":"2024-04-25T10:05:21.073018+0000","last_active":"2024-04-25T10:05:22.042801+0000","last_peered":"2024-04-25T10:05:22.042801+0000","last_clean":"2024-04-25T10:05:22.042801+0000","last_became_active":"2024-04-25T10:05:21.072257+0000","last_became_peered":"2024-04-25T10:05:21.072257+0000","last_unstale":"2024-04-25T10:05:22.042801+0000","last_undegraded":"2024-04-25T10:05:22.042801+0000","last_fullsized":"2024-04-25T10:05:22.042801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:53.558925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080605+0000","last_change":"2024-04-25T10:05:20.080134+0000","last_active":"2024-04-25T10:05:22.080605+0000","last_peered":"2024-04-25T10:05:22.080605+0000","last_clean":"2024-04-25T10:05:22.080605+0000","last_became_active":"2024-04-25T10:05:20.079484+0000","last_became_peered":"2024-04-25T10:05:20.079484+0000","last_unstale":"2024-04-25T10:05:22.080605+0000","last_undegraded":"2024-04-25T10:05:22.080605+0000","last_fullsized":"2024-04-25T10:05:22.080605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048538+0000","last_change":"2024-04-25T10:05:21.079513+0000","last_active":"2024-04-25T10:05:22.048538+0000","last_peered":"2024-04-25T10:05:22.048538+0000","last_clean":"2024-04-25T10:05:22.048538+0000","last_became_active":"2024-04-25T10:05:21.078962+0000","last_became_peered":"2024-04-25T10:05:21.078962+0000","last_unstale":"2024-04-25T10:05:22.048538+0000","last_undegraded":"2024-04-25T10:05:22.048538+0000","last_fullsized":"2024-04-25T10:05:22.048538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:18.289495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.049115+0000","last_change":"2024-04-25T10:05:20.075486+0000","last_active":"2024-04-25T10:05:22.049115+0000","last_peered":"2024-04-25T10:05:22.049115+0000","last_clean":"2024-04-25T10:05:22.049115+0000","last_became_active":"2024-04-25T10:05:20.074735+0000","last_became_peered":"2024-04-25T10:05:20.074735+0000","last_unstale":"2024-04-25T10:05:22.049115+0000","last_undegraded":"2024-04-25T10:05:22.049115+0000","last_fullsized":"2024-04-25T10:05:22.049115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:34.961990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134412+0000","last_change":"2024-04-25T10:05:21.085752+0000","last_active":"2024-04-25T10:05:22.134412+0000","last_peered":"2024-04-25T10:05:22.134412+0000","last_clean":"2024-04-25T10:05:22.134412+0000","last_became_active":"2024-04-25T10:05:21.081588+0000","last_became_peered":"2024-04-25T10:05:21.081588+0000","last_unstale":"2024-04-25T10:05:22.134412+0000","last_undegraded":"2024-04-25T10:05:22.134412+0000","last_fullsized":"2024-04-25T10:05:22.134412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:13.318906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043378+0000","last_change":"2024-04-25T10:05:20.065094+0000","last_active":"2024-04-25T10:05:22.043378+0000","last_peered":"2024-04-25T10:05:22.043378+0000","last_clean":"2024-04-25T10:05:22.043378+0000","last_became_active":"2024-04-25T10:05:20.063987+0000","last_became_peered":"2024-04-25T10:05:20.063987+0000","last_unstale":"2024-04-25T10:05:22.043378+0000","last_undegraded":"2024-04-25T10:05:22.043378+0000","last_fullsized":"2024-04-25T10:05:22.043378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:16.201265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048473+0000","last_change":"2024-04-25T10:05:21.063419+0000","last_active":"2024-04-25T10:05:22.048473+0000","last_peered":"2024-04-25T10:05:22.048473+0000","last_clean":"2024-04-25T10:05:22.048473+0000","last_became_active":"2024-04-25T10:05:21.060544+0000","last_became_peered":"2024-04-25T10:05:21.060544+0000","last_unstale":"2024-04-25T10:05:22.048473+0000","last_undegraded":"2024-04-25T10:05:22.048473+0000","last_fullsized":"2024-04-25T10:05:22.048473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:07.266916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080356+0000","last_change":"2024-04-25T10:05:20.070471+0000","last_active":"2024-04-25T10:05:22.080356+0000","last_peered":"2024-04-25T10:05:22.080356+0000","last_clean":"2024-04-25T10:05:22.080356+0000","last_became_active":"2024-04-25T10:05:20.069692+0000","last_became_peered":"2024-04-25T10:05:20.069692+0000","last_unstale":"2024-04-25T10:05:22.080356+0000","last_undegraded":"2024-04-25T10:05:22.080356+0000","last_fullsized":"2024-04-25T10:05:22.080356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:56.779127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136463+0000","last_change":"2024-04-25T10:05:21.086200+0000","last_active":"2024-04-25T10:05:22.136463+0000","last_peered":"2024-04-25T10:05:22.136463+0000","last_clean":"2024-04-25T10:05:22.136463+0000","last_became_active":"2024-04-25T10:05:21.085624+0000","last_became_peered":"2024-04-25T10:05:21.085624+0000","last_unstale":"2024-04-25T10:05:22.136463+0000","last_undegraded":"2024-04-25T10:05:22.136463+0000","last_fullsized":"2024-04-25T10:05:22.136463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:44.226635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135516+0000","last_change":"2024-04-25T10:05:20.077783+0000","last_active":"2024-04-25T10:05:22.135516+0000","last_peered":"2024-04-25T10:05:22.135516+0000","last_clean":"2024-04-25T10:05:22.135516+0000","last_became_active":"2024-04-25T10:05:20.077301+0000","last_became_peered":"2024-04-25T10:05:20.077301+0000","last_unstale":"2024-04-25T10:05:22.135516+0000","last_undegraded":"2024-04-25T10:05:22.135516+0000","last_fullsized":"2024-04-25T10:05:22.135516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:44.794815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134343+0000","last_change":"2024-04-25T10:05:21.086058+0000","last_active":"2024-04-25T10:05:22.134343+0000","last_peered":"2024-04-25T10:05:22.134343+0000","last_clean":"2024-04-25T10:05:22.134343+0000","last_became_active":"2024-04-25T10:05:21.085660+0000","last_became_peered":"2024-04-25T10:05:21.085660+0000","last_unstale":"2024-04-25T10:05:22.134343+0000","last_undegraded":"2024-04-25T10:05:22.134343+0000","last_fullsized":"2024-04-25T10:05:22.134343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:19.908741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047701+0000","last_change":"2024-04-25T10:05:20.062806+0000","last_active":"2024-04-25T10:05:22.047701+0000","last_peered":"2024-04-25T10:05:22.047701+0000","last_clean":"2024-04-25T10:05:22.047701+0000","last_became_active":"2024-04-25T10:05:20.061013+0000","last_became_peered":"2024-04-25T10:05:20.061013+0000","last_unstale":"2024-04-25T10:05:22.047701+0000","last_undegraded":"2024-04-25T10:05:22.047701+0000","last_fullsized":"2024-04-25T10:05:22.047701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:07.439088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136199+0000","last_change":"2024-04-25T10:05:21.092958+0000","last_active":"2024-04-25T10:05:22.136199+0000","last_peered":"2024-04-25T10:05:22.136199+0000","last_clean":"2024-04-25T10:05:22.136199+0000","last_became_active":"2024-04-25T10:05:21.089856+0000","last_became_peered":"2024-04-25T10:05:21.089856+0000","last_unstale":"2024-04-25T10:05:22.136199+0000","last_undegraded":"2024-04-25T10:05:22.136199+0000","last_fullsized":"2024-04-25T10:05:22.136199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:03.477184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.079832+0000","last_change":"2024-04-25T10:05:20.080086+0000","last_active":"2024-04-25T10:05:22.079832+0000","last_peered":"2024-04-25T10:05:22.079832+0000","last_clean":"2024-04-25T10:05:22.079832+0000","last_became_active":"2024-04-25T10:05:20.079302+0000","last_became_peered":"2024-04-25T10:05:20.079302+0000","last_unstale":"2024-04-25T10:05:22.079832+0000","last_undegraded":"2024-04-25T10:05:22.079832+0000","last_fullsized":"2024-04-25T10:05:22.079832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:34.961990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.079883+0000","last_change":"2024-04-25T10:05:21.091338+0000","last_active":"2024-04-25T10:05:22.079883+0000","last_peered":"2024-04-25T10:05:22.079883+0000","last_clean":"2024-04-25T10:05:22.079883+0000","last_became_active":"2024-04-25T10:05:21.090998+0000","last_became_peered":"2024-04-25T10:05:21.090998+0000","last_unstale":"2024-04-25T10:05:22.079883+0000","last_undegraded":"2024-04-25T10:05:22.079883+0000","last_fullsized":"2024-04-25T10:05:22.079883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:13.318906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.133875+0000","last_change":"2024-04-25T10:05:20.063163+0000","last_active":"2024-04-25T10:05:22.133875+0000","last_peered":"2024-04-25T10:05:22.133875+0000","last_clean":"2024-04-25T10:05:22.133875+0000","last_became_active":"2024-04-25T10:05:20.062707+0000","last_became_peered":"2024-04-25T10:05:20.062707+0000","last_unstale":"2024-04-25T10:05:22.133875+0000","last_undegraded":"2024-04-25T10:05:22.133875+0000","last_fullsized":"2024-04-25T10:05:22.133875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:12.228316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135958+0000","last_change":"2024-04-25T10:05:21.088835+0000","last_active":"2024-04-25T10:05:22.135958+0000","last_peered":"2024-04-25T10:05:22.135958+0000","last_clean":"2024-04-25T10:05:22.135958+0000","last_became_active":"2024-04-25T10:05:21.088476+0000","last_became_peered":"2024-04-25T10:05:21.088476+0000","last_unstale":"2024-04-25T10:05:22.135958+0000","last_undegraded":"2024-04-25T10:05:22.135958+0000","last_fullsized":"2024-04-25T10:05:22.135958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:54.535451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048102+0000","last_change":"2024-04-25T10:05:20.066666+0000","last_active":"2024-04-25T10:05:22.048102+0000","last_peered":"2024-04-25T10:05:22.048102+0000","last_clean":"2024-04-25T10:05:22.048102+0000","last_became_active":"2024-04-25T10:05:20.065812+0000","last_became_peered":"2024-04-25T10:05:20.065812+0000","last_unstale":"2024-04-25T10:05:22.048102+0000","last_undegraded":"2024-04-25T10:05:22.048102+0000","last_fullsized":"2024-04-25T10:05:22.048102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:45.802025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136886+0000","last_change":"2024-04-25T10:05:21.089032+0000","last_active":"2024-04-25T10:05:22.136886+0000","last_peered":"2024-04-25T10:05:22.136886+0000","last_clean":"2024-04-25T10:05:22.136886+0000","last_became_active":"2024-04-25T10:05:21.088707+0000","last_became_peered":"2024-04-25T10:05:21.088707+0000","last_unstale":"2024-04-25T10:05:22.136886+0000","last_undegraded":"2024-04-25T10:05:22.136886+0000","last_fullsized":"2024-04-25T10:05:22.136886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:29.342928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135030+0000","last_change":"2024-04-25T10:05:20.056234+0000","last_active":"2024-04-25T10:05:22.135030+0000","last_peered":"2024-04-25T10:05:22.135030+0000","last_clean":"2024-04-25T10:05:22.135030+0000","last_became_active":"2024-04-25T10:05:20.055711+0000","last_became_peered":"2024-04-25T10:05:20.055711+0000","last_unstale":"2024-04-25T10:05:22.135030+0000","last_undegraded":"2024-04-25T10:05:22.135030+0000","last_fullsized":"2024-04-25T10:05:22.135030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:44.794815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137621+0000","last_change":"2024-04-25T10:05:21.097805+0000","last_active":"2024-04-25T10:05:22.137621+0000","last_peered":"2024-04-25T10:05:22.137621+0000","last_clean":"2024-04-25T10:05:22.137621+0000","last_became_active":"2024-04-25T10:05:21.097066+0000","last_became_peered":"2024-04-25T10:05:21.097066+0000","last_unstale":"2024-04-25T10:05:22.137621+0000","last_undegraded":"2024-04-25T10:05:22.137621+0000","last_fullsized":"2024-04-25T10:05:22.137621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:10.508421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043379+0000","last_change":"2024-04-25T10:05:20.063861+0000","last_active":"2024-04-25T10:05:22.043379+0000","last_peered":"2024-04-25T10:05:22.043379+0000","last_clean":"2024-04-25T10:05:22.043379+0000","last_became_active":"2024-04-25T10:05:20.062501+0000","last_became_peered":"2024-04-25T10:05:20.062501+0000","last_unstale":"2024-04-25T10:05:22.043379+0000","last_undegraded":"2024-04-25T10:05:22.043379+0000","last_fullsized":"2024-04-25T10:05:22.043379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:02.884927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.049234+0000","last_change":"2024-04-25T10:05:21.083111+0000","last_active":"2024-04-25T10:05:22.049234+0000","last_peered":"2024-04-25T10:05:22.049234+0000","last_clean":"2024-04-25T10:05:22.049234+0000","last_became_active":"2024-04-25T10:05:21.082321+0000","last_became_peered":"2024-04-25T10:05:21.082321+0000","last_unstale":"2024-04-25T10:05:22.049234+0000","last_undegraded":"2024-04-25T10:05:22.049234+0000","last_fullsized":"2024-04-25T10:05:22.049234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:02.305355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135030+0000","last_change":"2024-04-25T10:05:20.078355+0000","last_active":"2024-04-25T10:05:22.135030+0000","last_peered":"2024-04-25T10:05:22.135030+0000","last_clean":"2024-04-25T10:05:22.135030+0000","last_became_active":"2024-04-25T10:05:20.077677+0000","last_became_peered":"2024-04-25T10:05:20.077677+0000","last_unstale":"2024-04-25T10:05:22.135030+0000","last_undegraded":"2024-04-25T10:05:22.135030+0000","last_fullsized":"2024-04-25T10:05:22.135030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:35.252363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136520+0000","last_change":"2024-04-25T10:05:21.089183+0000","last_active":"2024-04-25T10:05:22.136520+0000","last_peered":"2024-04-25T10:05:22.136520+0000","last_clean":"2024-04-25T10:05:22.136520+0000","last_became_active":"2024-04-25T10:05:21.088768+0000","last_became_peered":"2024-04-25T10:05:21.088768+0000","last_unstale":"2024-04-25T10:05:22.136520+0000","last_undegraded":"2024-04-25T10:05:22.136520+0000","last_fullsized":"2024-04-25T10:05:22.136520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:06.419537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135576+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:05:22.135576+0000","last_peered":"2024-04-25T10:05:22.135576+0000","last_clean":"2024-04-25T10:05:22.135576+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:05:22.135576+0000","last_undegraded":"2024-04-25T10:05:22.135576+0000","last_fullsized":"2024-04-25T10:05:22.135576+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047506+0000","last_change":"2024-04-25T10:05:20.069430+0000","last_active":"2024-04-25T10:05:22.047506+0000","last_peered":"2024-04-25T10:05:22.047506+0000","last_clean":"2024-04-25T10:05:22.047506+0000","last_became_active":"2024-04-25T10:05:20.068755+0000","last_became_peered":"2024-04-25T10:05:20.068755+0000","last_unstale":"2024-04-25T10:05:22.047506+0000","last_undegraded":"2024-04-25T10:05:22.047506+0000","last_fullsized":"2024-04-25T10:05:22.047506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:56.779127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136220+0000","last_change":"2024-04-25T10:05:21.071861+0000","last_active":"2024-04-25T10:05:22.136220+0000","last_peered":"2024-04-25T10:05:22.136220+0000","last_clean":"2024-04-25T10:05:22.136220+0000","last_became_active":"2024-04-25T10:05:21.071182+0000","last_became_peered":"2024-04-25T10:05:21.071182+0000","last_unstale":"2024-04-25T10:05:22.136220+0000","last_undegraded":"2024-04-25T10:05:22.136220+0000","last_fullsized":"2024-04-25T10:05:22.136220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:17.144179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135408+0000","last_change":"2024-04-25T10:05:20.084051+0000","last_active":"2024-04-25T10:05:22.135408+0000","last_peered":"2024-04-25T10:05:22.135408+0000","last_clean":"2024-04-25T10:05:22.135408+0000","last_became_active":"2024-04-25T10:05:20.083563+0000","last_became_peered":"2024-04-25T10:05:20.083563+0000","last_unstale":"2024-04-25T10:05:22.135408+0000","last_undegraded":"2024-04-25T10:05:22.135408+0000","last_fullsized":"2024-04-25T10:05:22.135408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:20.779254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135004+0000","last_change":"2024-04-25T10:05:21.072824+0000","last_active":"2024-04-25T10:05:22.135004+0000","last_peered":"2024-04-25T10:05:22.135004+0000","last_clean":"2024-04-25T10:05:22.135004+0000","last_became_active":"2024-04-25T10:05:21.071691+0000","last_became_peered":"2024-04-25T10:05:21.071691+0000","last_unstale":"2024-04-25T10:05:22.135004+0000","last_undegraded":"2024-04-25T10:05:22.135004+0000","last_fullsized":"2024-04-25T10:05:22.135004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:21.545921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135509+0000","last_change":"2024-04-25T10:05:20.056953+0000","last_active":"2024-04-25T10:05:22.135509+0000","last_peered":"2024-04-25T10:05:22.135509+0000","last_clean":"2024-04-25T10:05:22.135509+0000","last_became_active":"2024-04-25T10:05:20.056056+0000","last_became_peered":"2024-04-25T10:05:20.056056+0000","last_unstale":"2024-04-25T10:05:22.135509+0000","last_undegraded":"2024-04-25T10:05:22.135509+0000","last_fullsized":"2024-04-25T10:05:22.135509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042576+0000","last_change":"2024-04-25T10:05:21.056503+0000","last_active":"2024-04-25T10:05:22.042576+0000","last_peered":"2024-04-25T10:05:22.042576+0000","last_clean":"2024-04-25T10:05:22.042576+0000","last_became_active":"2024-04-25T10:05:21.055664+0000","last_became_peered":"2024-04-25T10:05:21.055664+0000","last_unstale":"2024-04-25T10:05:22.042576+0000","last_undegraded":"2024-04-25T10:05:22.042576+0000","last_fullsized":"2024-04-25T10:05:22.042576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:20.349799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047779+0000","last_change":"2024-04-25T10:05:20.061869+0000","last_active":"2024-04-25T10:05:22.047779+0000","last_peered":"2024-04-25T10:05:22.047779+0000","last_clean":"2024-04-25T10:05:22.047779+0000","last_became_active":"2024-04-25T10:05:20.060232+0000","last_became_peered":"2024-04-25T10:05:20.060232+0000","last_unstale":"2024-04-25T10:05:22.047779+0000","last_undegraded":"2024-04-25T10:05:22.047779+0000","last_fullsized":"2024-04-25T10:05:22.047779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:50.073175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047317+0000","last_change":"2024-04-25T10:05:21.083202+0000","last_active":"2024-04-25T10:05:22.047317+0000","last_peered":"2024-04-25T10:05:22.047317+0000","last_clean":"2024-04-25T10:05:22.047317+0000","last_became_active":"2024-04-25T10:05:21.082857+0000","last_became_peered":"2024-04-25T10:05:21.082857+0000","last_unstale":"2024-04-25T10:05:22.047317+0000","last_undegraded":"2024-04-25T10:05:22.047317+0000","last_fullsized":"2024-04-25T10:05:22.047317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:49.496977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042931+0000","last_change":"2024-04-25T10:05:20.060703+0000","last_active":"2024-04-25T10:05:22.042931+0000","last_peered":"2024-04-25T10:05:22.042931+0000","last_clean":"2024-04-25T10:05:22.042931+0000","last_became_active":"2024-04-25T10:05:20.056864+0000","last_became_peered":"2024-04-25T10:05:20.056864+0000","last_unstale":"2024-04-25T10:05:22.042931+0000","last_undegraded":"2024-04-25T10:05:22.042931+0000","last_fullsized":"2024-04-25T10:05:22.042931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:58.129770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.049114+0000","last_change":"2024-04-25T10:05:21.081253+0000","last_active":"2024-04-25T10:05:22.049114+0000","last_peered":"2024-04-25T10:05:22.049114+0000","last_clean":"2024-04-25T10:05:22.049114+0000","last_became_active":"2024-04-25T10:05:21.080917+0000","last_became_peered":"2024-04-25T10:05:21.080917+0000","last_unstale":"2024-04-25T10:05:22.049114+0000","last_undegraded":"2024-04-25T10:05:22.049114+0000","last_fullsized":"2024-04-25T10:05:22.049114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:41.708275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136822+0000","last_change":"2024-04-25T10:05:20.089660+0000","last_active":"2024-04-25T10:05:22.136822+0000","last_peered":"2024-04-25T10:05:22.136822+0000","last_clean":"2024-04-25T10:05:22.136822+0000","last_became_active":"2024-04-25T10:05:20.088996+0000","last_became_peered":"2024-04-25T10:05:20.088996+0000","last_unstale":"2024-04-25T10:05:22.136822+0000","last_undegraded":"2024-04-25T10:05:22.136822+0000","last_fullsized":"2024-04-25T10:05:22.136822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:55.264435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042818+0000","last_change":"2024-04-25T10:05:21.061203+0000","last_active":"2024-04-25T10:05:22.042818+0000","last_peered":"2024-04-25T10:05:22.042818+0000","last_clean":"2024-04-25T10:05:22.042818+0000","last_became_active":"2024-04-25T10:05:21.060108+0000","last_became_peered":"2024-04-25T10:05:21.060108+0000","last_unstale":"2024-04-25T10:05:22.042818+0000","last_undegraded":"2024-04-25T10:05:22.042818+0000","last_fullsized":"2024-04-25T10:05:22.042818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:11.969876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.077357+0000","last_change":"2024-04-25T10:05:20.071770+0000","last_active":"2024-04-25T10:05:22.077357+0000","last_peered":"2024-04-25T10:05:22.077357+0000","last_clean":"2024-04-25T10:05:22.077357+0000","last_became_active":"2024-04-25T10:05:20.071460+0000","last_became_peered":"2024-04-25T10:05:20.071460+0000","last_unstale":"2024-04-25T10:05:22.077357+0000","last_undegraded":"2024-04-25T10:05:22.077357+0000","last_fullsized":"2024-04-25T10:05:22.077357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:20.779254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136359+0000","last_change":"2024-04-25T10:05:21.063585+0000","last_active":"2024-04-25T10:05:22.136359+0000","last_peered":"2024-04-25T10:05:22.136359+0000","last_clean":"2024-04-25T10:05:22.136359+0000","last_became_active":"2024-04-25T10:05:21.062481+0000","last_became_peered":"2024-04-25T10:05:21.062481+0000","last_unstale":"2024-04-25T10:05:22.136359+0000","last_undegraded":"2024-04-25T10:05:22.136359+0000","last_fullsized":"2024-04-25T10:05:22.136359+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:47.724037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080795+0000","last_change":"2024-04-25T10:05:20.086795+0000","last_active":"2024-04-25T10:05:22.080795+0000","last_peered":"2024-04-25T10:05:22.080795+0000","last_clean":"2024-04-25T10:05:22.080795+0000","last_became_active":"2024-04-25T10:05:20.086157+0000","last_became_peered":"2024-04-25T10:05:20.086157+0000","last_unstale":"2024-04-25T10:05:22.080795+0000","last_undegraded":"2024-04-25T10:05:22.080795+0000","last_fullsized":"2024-04-25T10:05:22.080795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:35.252363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135349+0000","last_change":"2024-04-25T10:05:21.080072+0000","last_active":"2024-04-25T10:05:22.135349+0000","last_peered":"2024-04-25T10:05:22.135349+0000","last_clean":"2024-04-25T10:05:22.135349+0000","last_became_active":"2024-04-25T10:05:21.077807+0000","last_became_peered":"2024-04-25T10:05:21.077807+0000","last_unstale":"2024-04-25T10:05:22.135349+0000","last_undegraded":"2024-04-25T10:05:22.135349+0000","last_fullsized":"2024-04-25T10:05:22.135349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:23.519531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135461+0000","last_change":"2024-04-25T10:05:20.074066+0000","last_active":"2024-04-25T10:05:22.135461+0000","last_peered":"2024-04-25T10:05:22.135461+0000","last_clean":"2024-04-25T10:05:22.135461+0000","last_became_active":"2024-04-25T10:05:20.073313+0000","last_became_peered":"2024-04-25T10:05:20.073313+0000","last_unstale":"2024-04-25T10:05:22.135461+0000","last_undegraded":"2024-04-25T10:05:22.135461+0000","last_fullsized":"2024-04-25T10:05:22.135461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:35.144583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047063+0000","last_change":"2024-04-25T10:05:21.070227+0000","last_active":"2024-04-25T10:05:22.047063+0000","last_peered":"2024-04-25T10:05:22.047063+0000","last_clean":"2024-04-25T10:05:22.047063+0000","last_became_active":"2024-04-25T10:05:21.069473+0000","last_became_peered":"2024-04-25T10:05:21.069473+0000","last_unstale":"2024-04-25T10:05:22.047063+0000","last_undegraded":"2024-04-25T10:05:22.047063+0000","last_fullsized":"2024-04-25T10:05:22.047063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:17.204276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043069+0000","last_change":"2024-04-25T10:05:20.073767+0000","last_active":"2024-04-25T10:05:22.043069+0000","last_peered":"2024-04-25T10:05:22.043069+0000","last_clean":"2024-04-25T10:05:22.043069+0000","last_became_active":"2024-04-25T10:05:20.073333+0000","last_became_peered":"2024-04-25T10:05:20.073333+0000","last_unstale":"2024-04-25T10:05:22.043069+0000","last_undegraded":"2024-04-25T10:05:22.043069+0000","last_fullsized":"2024-04-25T10:05:22.043069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:48.493967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135540+0000","last_change":"2024-04-25T10:05:21.070624+0000","last_active":"2024-04-25T10:05:22.135540+0000","last_peered":"2024-04-25T10:05:22.135540+0000","last_clean":"2024-04-25T10:05:22.135540+0000","last_became_active":"2024-04-25T10:05:21.065398+0000","last_became_peered":"2024-04-25T10:05:21.065398+0000","last_unstale":"2024-04-25T10:05:22.135540+0000","last_undegraded":"2024-04-25T10:05:22.135540+0000","last_fullsized":"2024-04-25T10:05:22.135540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:33.904915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043074+0000","last_change":"2024-04-25T10:05:20.055362+0000","last_active":"2024-04-25T10:05:22.043074+0000","last_peered":"2024-04-25T10:05:22.043074+0000","last_clean":"2024-04-25T10:05:22.043074+0000","last_became_active":"2024-04-25T10:05:20.052279+0000","last_became_peered":"2024-04-25T10:05:20.052279+0000","last_unstale":"2024-04-25T10:05:22.043074+0000","last_undegraded":"2024-04-25T10:05:22.043074+0000","last_fullsized":"2024-04-25T10:05:22.043074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:35.144583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137237+0000","last_change":"2024-04-25T10:05:21.075800+0000","last_active":"2024-04-25T10:05:22.137237+0000","last_peered":"2024-04-25T10:05:22.137237+0000","last_clean":"2024-04-25T10:05:22.137237+0000","last_became_active":"2024-04-25T10:05:21.075425+0000","last_became_peered":"2024-04-25T10:05:21.075425+0000","last_unstale":"2024-04-25T10:05:22.137237+0000","last_undegraded":"2024-04-25T10:05:22.137237+0000","last_fullsized":"2024-04-25T10:05:22.137237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:02.817716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136402+0000","last_change":"2024-04-25T10:05:20.085669+0000","last_active":"2024-04-25T10:05:22.136402+0000","last_peered":"2024-04-25T10:05:22.136402+0000","last_clean":"2024-04-25T10:05:22.136402+0000","last_became_active":"2024-04-25T10:05:20.084890+0000","last_became_peered":"2024-04-25T10:05:20.084890+0000","last_unstale":"2024-04-25T10:05:22.136402+0000","last_undegraded":"2024-04-25T10:05:22.136402+0000","last_fullsized":"2024-04-25T10:05:22.136402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:16.201265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080121+0000","last_change":"2024-04-25T10:05:21.072582+0000","last_active":"2024-04-25T10:05:22.080121+0000","last_peered":"2024-04-25T10:05:22.080121+0000","last_clean":"2024-04-25T10:05:22.080121+0000","last_became_active":"2024-04-25T10:05:21.071786+0000","last_became_peered":"2024-04-25T10:05:21.071786+0000","last_unstale":"2024-04-25T10:05:22.080121+0000","last_undegraded":"2024-04-25T10:05:22.080121+0000","last_fullsized":"2024-04-25T10:05:22.080121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:21.545921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048737+0000","last_change":"2024-04-25T10:05:21.069632+0000","last_active":"2024-04-25T10:05:22.048737+0000","last_peered":"2024-04-25T10:05:22.048737+0000","last_clean":"2024-04-25T10:05:22.048737+0000","last_became_active":"2024-04-25T10:05:21.068870+0000","last_became_peered":"2024-04-25T10:05:21.068870+0000","last_unstale":"2024-04-25T10:05:22.048737+0000","last_undegraded":"2024-04-25T10:05:22.048737+0000","last_fullsized":"2024-04-25T10:05:22.048737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:09.103889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136270+0000","last_change":"2024-04-25T10:05:20.085837+0000","last_active":"2024-04-25T10:05:22.136270+0000","last_peered":"2024-04-25T10:05:22.136270+0000","last_clean":"2024-04-25T10:05:22.136270+0000","last_became_active":"2024-04-25T10:05:20.085079+0000","last_became_peered":"2024-04-25T10:05:20.085079+0000","last_unstale":"2024-04-25T10:05:22.136270+0000","last_undegraded":"2024-04-25T10:05:22.136270+0000","last_fullsized":"2024-04-25T10:05:22.136270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:02.884927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048913+0000","last_change":"2024-04-25T10:05:21.073527+0000","last_active":"2024-04-25T10:05:22.048913+0000","last_peered":"2024-04-25T10:05:22.048913+0000","last_clean":"2024-04-25T10:05:22.048913+0000","last_became_active":"2024-04-25T10:05:21.072780+0000","last_became_peered":"2024-04-25T10:05:21.072780+0000","last_unstale":"2024-04-25T10:05:22.048913+0000","last_undegraded":"2024-04-25T10:05:22.048913+0000","last_fullsized":"2024-04-25T10:05:22.048913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:58.306898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136676+0000","last_change":"2024-04-25T10:05:20.088969+0000","last_active":"2024-04-25T10:05:22.136676+0000","last_peered":"2024-04-25T10:05:22.136676+0000","last_clean":"2024-04-25T10:05:22.136676+0000","last_became_active":"2024-04-25T10:05:20.088074+0000","last_became_peered":"2024-04-25T10:05:20.088074+0000","last_unstale":"2024-04-25T10:05:22.136676+0000","last_undegraded":"2024-04-25T10:05:22.136676+0000","last_fullsized":"2024-04-25T10:05:22.136676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:48.493967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134241+0000","last_change":"2024-04-25T10:05:21.081805+0000","last_active":"2024-04-25T10:05:22.134241+0000","last_peered":"2024-04-25T10:05:22.134241+0000","last_clean":"2024-04-25T10:05:22.134241+0000","last_became_active":"2024-04-25T10:05:21.081062+0000","last_became_peered":"2024-04-25T10:05:21.081062+0000","last_unstale":"2024-04-25T10:05:22.134241+0000","last_undegraded":"2024-04-25T10:05:22.134241+0000","last_fullsized":"2024-04-25T10:05:22.134241+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:55.635372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137045+0000","last_change":"2024-04-25T10:05:20.060475+0000","last_active":"2024-04-25T10:05:22.137045+0000","last_peered":"2024-04-25T10:05:22.137045+0000","last_clean":"2024-04-25T10:05:22.137045+0000","last_became_active":"2024-04-25T10:05:20.060153+0000","last_became_peered":"2024-04-25T10:05:20.060153+0000","last_unstale":"2024-04-25T10:05:22.137045+0000","last_undegraded":"2024-04-25T10:05:22.137045+0000","last_fullsized":"2024-04-25T10:05:22.137045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:12.228316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136396+0000","last_change":"2024-04-25T10:05:21.085394+0000","last_active":"2024-04-25T10:05:22.136396+0000","last_peered":"2024-04-25T10:05:22.136396+0000","last_clean":"2024-04-25T10:05:22.136396+0000","last_became_active":"2024-04-25T10:05:21.085122+0000","last_became_peered":"2024-04-25T10:05:21.085122+0000","last_unstale":"2024-04-25T10:05:22.136396+0000","last_undegraded":"2024-04-25T10:05:22.136396+0000","last_fullsized":"2024-04-25T10:05:22.136396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134727+0000","last_change":"2024-04-25T10:05:20.085225+0000","last_active":"2024-04-25T10:05:22.134727+0000","last_peered":"2024-04-25T10:05:22.134727+0000","last_clean":"2024-04-25T10:05:22.134727+0000","last_became_active":"2024-04-25T10:05:20.084907+0000","last_became_peered":"2024-04-25T10:05:20.084907+0000","last_unstale":"2024-04-25T10:05:22.134727+0000","last_undegraded":"2024-04-25T10:05:22.134727+0000","last_fullsized":"2024-04-25T10:05:22.134727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":233472,"data_stored":79812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":221184,"data_stored":78200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":78360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":827392,"data_stored":669206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:25.501 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:05:25.501 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T10:05:25.501 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T10:05:25.501 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T10:05:25.502 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T10:05:25.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb833d640 1 Processor -- start 2024-04-25T10:05:25.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb833d640 1 -- start start 2024-04-25T10:05:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb833d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bb00af8e0 0x7f0bb006b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb833d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb00aff30 con 0x7f0bb006ac10 2024-04-25T10:05:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb833d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb00b0210 con 0x7f0bb00b0fd0 2024-04-25T10:05:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb833d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb015bf10 con 0x7f0bb00af8e0 2024-04-25T10:05:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb60d9640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0bb006ac10 0x7f0bb006aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55766/0 (socket says 172.21.15.60:55766) 2024-04-25T10:05:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb60d9640 1 -- 172.21.15.60:0/4291790390 learned_addr learned my addr 172.21.15.60:0/4291790390 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb58d8640 1 --2- 172.21.15.60:0/4291790390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bb00af8e0 0x7f0bb006b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb50d7640 1 -- 172.21.15.60:0/4291790390 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414440339 0 0) 0x7f0bb00aff30 con 0x7f0bb006ac10 2024-04-25T10:05:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb50d7640 1 -- 172.21.15.60:0/4291790390 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b9c003530 con 0x7f0bb006ac10 2024-04-25T10:05:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb50d7640 1 -- 172.21.15.60:0/4291790390 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 720361220 0 0) 0x7f0b9c003530 con 0x7f0bb006ac10 2024-04-25T10:05:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb50d7640 1 -- 172.21.15.60:0/4291790390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bb00af8e0 msgr2=0x7f0bb006b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb50d7640 1 --2- 172.21.15.60:0/4291790390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bb00af8e0 0x7f0bb006b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb50d7640 1 -- 172.21.15.60:0/4291790390 >> v1:172.21.15.100:6789/0 conn(0x7f0bb00b0fd0 legacy=0x7f0bb00b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb50d7640 1 -- 172.21.15.60:0/4291790390 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb00b04f0 con 0x7f0bb006ac10 2024-04-25T10:05:25.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb50d7640 1 -- 172.21.15.60:0/4291790390 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f0bac001680 con 0x7f0bb006ac10 2024-04-25T10:05:25.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb50d7640 1 -- 172.21.15.60:0/4291790390 >> v1:172.21.15.60:6789/0 conn(0x7f0bb006ac10 legacy=0x7f0bb006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb50d7640 1 --2- 172.21.15.60:0/4291790390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bb00af8e0 0x7f0b9c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb50d7640 1 --2- 172.21.15.60:0/4291790390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b9c003f70 0x7f0b9c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb50d7640 1 --2- 172.21.15.60:0/4291790390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b9c004d50 0x7f0b9c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb50d7640 1 -- 172.21.15.60:0/4291790390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c00ba80 con 0x7f0b9c004d50 2024-04-25T10:05:25.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb50d7640 1 -- 172.21.15.60:0/4291790390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c00bc30 con 0x7f0b9c003f70 2024-04-25T10:05:25.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb50d7640 1 -- 172.21.15.60:0/4291790390 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c00be30 con 0x7f0bb00af8e0 2024-04-25T10:05:25.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb68da640 1 --2- 172.21.15.60:0/4291790390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b9c004d50 0x7f0b9c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb60d9640 1 --2- 172.21.15.60:0/4291790390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bb00af8e0 0x7f0b9c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb58d8640 1 --2- 172.21.15.60:0/4291790390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b9c003f70 0x7f0b9c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb68da640 1 -- 172.21.15.60:0/4291790390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bb00af8e0 msgr2=0x7f0b9c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb68da640 1 --2- 172.21.15.60:0/4291790390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bb00af8e0 0x7f0b9c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb68da640 1 -- 172.21.15.60:0/4291790390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b9c003f70 msgr2=0x7f0b9c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb68da640 1 --2- 172.21.15.60:0/4291790390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b9c003f70 0x7f0b9c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb68da640 1 -- 172.21.15.60:0/4291790390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c003530 con 0x7f0b9c004d50 2024-04-25T10:05:25.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.659+0000 7f0bb60d9640 1 --2- 172.21.15.60:0/4291790390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bb00af8e0 0x7f0b9c0039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:25.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.663+0000 7f0bb68da640 1 --2- 172.21.15.60:0/4291790390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b9c004d50 0x7f0b9c0051d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8001e70 tx=0x7f0ba801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78044b6663d5027b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.663+0000 7f0bb50d7640 1 -- 172.21.15.60:0/4291790390 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8001a90 con 0x7f0b9c004d50 2024-04-25T10:05:25.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.663+0000 7f0bb50d7640 1 -- 172.21.15.60:0/4291790390 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba8003b80 con 0x7f0b9c004d50 2024-04-25T10:05:25.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.663+0000 7f0bb50d7640 1 -- 172.21.15.60:0/4291790390 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8005080 con 0x7f0b9c004d50 2024-04-25T10:05:25.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.663+0000 7f0bb833d640 1 -- 172.21.15.60:0/4291790390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b9c004d50 msgr2=0x7f0b9c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.663+0000 7f0bb833d640 1 --2- 172.21.15.60:0/4291790390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b9c004d50 0x7f0b9c0051d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8001e70 tx=0x7f0ba801c780 comp rx=0 tx=0).stop 2024-04-25T10:05:25.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.663+0000 7f0bb68da640 1 -- 172.21.15.60:0/4291790390 reap_dead start 2024-04-25T10:05:25.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.663+0000 7f0bb833d640 1 -- 172.21.15.60:0/4291790390 shutdown_connections 2024-04-25T10:05:25.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.663+0000 7f0bb833d640 1 -- 172.21.15.60:0/4291790390 >> 172.21.15.60:0/4291790390 conn(0x7f0bb0072920 msgr2=0x7f0bb0099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.663+0000 7f0bb833d640 1 -- 172.21.15.60:0/4291790390 shutdown_connections 2024-04-25T10:05:25.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.663+0000 7f0bb833d640 1 -- 172.21.15.60:0/4291790390 wait complete. 2024-04-25T10:05:25.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.663+0000 7f0bb833d640 1 Processor -- start 2024-04-25T10:05:25.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.663+0000 7f0bb833d640 1 -- start start 2024-04-25T10:05:25.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb833d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b9c004d50 0x7f0bb009c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb833d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb00d03d0 0x7f0bb00ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb833d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bb00cf040 0x7f0bb014e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb833d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb0145510 con 0x7f0bb00cf040 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb833d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb00b0ab0 con 0x7f0bb00d03d0 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb833d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb006b700 con 0x7f0b9c004d50 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb68da640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bb00cf040 0x7f0bb014e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb68da640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bb00cf040 0x7f0bb014e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45228/0 (socket says 172.21.15.60:45228) 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb68da640 1 -- 172.21.15.60:0/253056828 learned_addr learned my addr 172.21.15.60:0/253056828 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb58d8640 1 --2- 172.21.15.60:0/253056828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb00d03d0 0x7f0bb00ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb60d9640 1 --2- 172.21.15.60:0/253056828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b9c004d50 0x7f0bb009c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb68da640 1 -- 172.21.15.60:0/253056828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b9c004d50 msgr2=0x7f0bb009c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb68da640 1 --2- 172.21.15.60:0/253056828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b9c004d50 0x7f0bb009c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb68da640 1 -- 172.21.15.60:0/253056828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb00d03d0 msgr2=0x7f0bb00ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb68da640 1 --2- 172.21.15.60:0/253056828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb00d03d0 0x7f0bb00ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb68da640 1 -- 172.21.15.60:0/253056828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb00cda60 con 0x7f0bb00cf040 2024-04-25T10:05:25.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb60d9640 1 --2- 172.21.15.60:0/253056828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b9c004d50 0x7f0bb009c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:25.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb68da640 1 --2- 172.21.15.60:0/253056828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bb00cf040 0x7f0bb014e820 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0ba801ccf0 tx=0x7f0b9c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0ba2ffd640 1 -- 172.21.15.60:0/253056828 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba80ba070 con 0x7f0bb00cf040 2024-04-25T10:05:25.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0ba2ffd640 1 -- 172.21.15.60:0/253056828 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba8005830 con 0x7f0bb00cf040 2024-04-25T10:05:25.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0ba2ffd640 1 -- 172.21.15.60:0/253056828 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8001a90 con 0x7f0bb00cf040 2024-04-25T10:05:25.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb833d640 1 -- 172.21.15.60:0/253056828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb00cdd40 con 0x7f0bb00cf040 2024-04-25T10:05:25.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb833d640 1 -- 172.21.15.60:0/253056828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb00ce300 con 0x7f0bb00cf040 2024-04-25T10:05:25.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0bb833d640 1 -- 172.21.15.60:0/253056828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb0074770 con 0x7f0bb00cf040 2024-04-25T10:05:25.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0ba2ffd640 1 -- 172.21.15.60:0/253056828 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ba80a6480 con 0x7f0bb00cf040 2024-04-25T10:05:25.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.667+0000 7f0ba2ffd640 1 --2- 172.21.15.60:0/253056828 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0b94066f60 0x7f0b94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.671+0000 7f0ba2ffd640 1 -- 172.21.15.60:0/253056828 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ba811e980 con 0x7f0bb00cf040 2024-04-25T10:05:25.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.671+0000 7f0bb60d9640 1 --2- 172.21.15.60:0/253056828 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0b94066f60 0x7f0b94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.671+0000 7f0bb60d9640 1 --2- 172.21.15.60:0/253056828 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0b94066f60 0x7f0b94069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0bac001a20 tx=0x7f0bac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.675+0000 7f0ba2ffd640 1 -- 172.21.15.60:0/253056828 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ba80ac020 con 0x7f0bb00cf040 2024-04-25T10:05:25.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.867+0000 7f0bb833d640 1 -- 172.21.15.60:0/253056828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0bb00d0960 con 0x7f0bb00cf040 2024-04-25T10:05:25.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.871+0000 7f0ba2ffd640 1 -- 172.21.15.60:0/253056828 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+165740 (secure 0 0 0) 0x7f0ba8016490 con 0x7f0bb00cf040 2024-04-25T10:05:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:25.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.875+0000 7f0bb833d640 1 -- 172.21.15.60:0/253056828 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0b94066f60 msgr2=0x7f0b94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.875+0000 7f0bb833d640 1 --2- 172.21.15.60:0/253056828 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0b94066f60 0x7f0b94069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0bac001a20 tx=0x7f0bac009040 comp rx=0 tx=0).stop 2024-04-25T10:05:25.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.875+0000 7f0bb833d640 1 -- 172.21.15.60:0/253056828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bb00cf040 msgr2=0x7f0bb014e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.875+0000 7f0bb833d640 1 --2- 172.21.15.60:0/253056828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bb00cf040 0x7f0bb014e820 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0ba801ccf0 tx=0x7f0b9c005840 comp rx=0 tx=0).stop 2024-04-25T10:05:25.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.875+0000 7f0bb833d640 1 -- 172.21.15.60:0/253056828 shutdown_connections 2024-04-25T10:05:25.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.875+0000 7f0bb833d640 1 --2- 172.21.15.60:0/253056828 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0b94066f60 0x7f0b94069420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.875+0000 7f0bb833d640 1 --2- 172.21.15.60:0/253056828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bb00cf040 0x7f0bb014e820 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.875+0000 7f0bb833d640 1 --2- 172.21.15.60:0/253056828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb00d03d0 0x7f0bb00ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.875+0000 7f0bb833d640 1 --2- 172.21.15.60:0/253056828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b9c004d50 0x7f0bb009c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.875+0000 7f0bb833d640 1 -- 172.21.15.60:0/253056828 >> 172.21.15.60:0/253056828 conn(0x7f0bb0072920 msgr2=0x7f0bb006e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.879+0000 7f0bb833d640 1 -- 172.21.15.60:0/253056828 shutdown_connections 2024-04-25T10:05:25.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:25.879+0000 7f0bb833d640 1 -- 172.21.15.60:0/253056828 wait complete. 2024-04-25T10:05:25.900 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3699412842},{"type":"v1","addr":"172.21.15.100:6833","nonce":3699412842}]},"active_addr":"172.21.15.100:6833/3699412842","active_change":"2024-04-25T10:05:00.369002+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1001252218}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":2403314895}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1320685480}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1946761738}]}]} 2024-04-25T10:05:25.904 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T10:05:25.904 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T10:05:25.904 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:26.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.019+0000 7f0793f48640 1 Processor -- start 2024-04-25T10:05:26.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.019+0000 7f0793f48640 1 -- start start 2024-04-25T10:05:26.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.019+0000 7f0793f48640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f078c15a7a0 0x7f078c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.019+0000 7f0793f48640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f078c06ec30 0x7f078c071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.019+0000 7f0793f48640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f078c0721e0 con 0x7f078c0b37e0 2024-04-25T10:05:26.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.019+0000 7f0793f48640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f078c0cdaa0 con 0x7f078c15a7a0 2024-04-25T10:05:26.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.019+0000 7f0793f48640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f078c06d2f0 con 0x7f078c06ec30 2024-04-25T10:05:26.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.019+0000 7f0791ce4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f078c15a7a0 0x7f078c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.019+0000 7f07924e5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f078c0b37e0 0x7f078c06dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55768/0 (socket says 172.21.15.60:55768) 2024-04-25T10:05:26.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.019+0000 7f07924e5640 1 -- 172.21.15.60:0/3808264976 learned_addr learned my addr 172.21.15.60:0/3808264976 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:26.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.019+0000 7f07914e3640 1 --2- 172.21.15.60:0/3808264976 >> v2:172.21.15.100:3300/0 conn(0x7f078c06ec30 0x7f078c071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.019+0000 7f0791ce4640 1 -- 172.21.15.60:0/3808264976 >> v2:172.21.15.100:3300/0 conn(0x7f078c06ec30 msgr2=0x7f078c071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.019+0000 7f0791ce4640 1 --2- 172.21.15.60:0/3808264976 >> v2:172.21.15.100:3300/0 conn(0x7f078c06ec30 0x7f078c071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.019+0000 7f0791ce4640 1 -- 172.21.15.60:0/3808264976 >> v1:172.21.15.60:6789/0 conn(0x7f078c0b37e0 legacy=0x7f078c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.019+0000 7f0791ce4640 1 -- 172.21.15.60:0/3808264976 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c06f130 con 0x7f078c15a7a0 2024-04-25T10:05:26.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0791ce4640 1 --2- 172.21.15.60:0/3808264976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f078c15a7a0 0x7f078c0cd560 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f078c072bf0 tx=0x7f078801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c85ea25258edf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0790ce2640 1 -- 172.21.15.60:0/3808264976 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880b1070 con 0x7f078c15a7a0 2024-04-25T10:05:26.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0790ce2640 1 -- 172.21.15.60:0/3808264976 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07880ac5f0 con 0x7f078c15a7a0 2024-04-25T10:05:26.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0790ce2640 1 -- 172.21.15.60:0/3808264976 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880ac9c0 con 0x7f078c15a7a0 2024-04-25T10:05:26.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0793f48640 1 -- 172.21.15.60:0/3808264976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f078c15a7a0 msgr2=0x7f078c0cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0793f48640 1 --2- 172.21.15.60:0/3808264976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f078c15a7a0 0x7f078c0cd560 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f078c072bf0 tx=0x7f078801c480 comp rx=0 tx=0).stop 2024-04-25T10:05:26.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0793f48640 1 -- 172.21.15.60:0/3808264976 shutdown_connections 2024-04-25T10:05:26.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0793f48640 1 --2- 172.21.15.60:0/3808264976 >> v2:172.21.15.100:3300/0 conn(0x7f078c06ec30 0x7f078c071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0793f48640 1 --2- 172.21.15.60:0/3808264976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f078c15a7a0 0x7f078c0cd560 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0793f48640 1 -- 172.21.15.60:0/3808264976 >> 172.21.15.60:0/3808264976 conn(0x7f078c0b12d0 msgr2=0x7f078c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0793f48640 1 -- 172.21.15.60:0/3808264976 shutdown_connections 2024-04-25T10:05:26.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0793f48640 1 -- 172.21.15.60:0/3808264976 wait complete. 2024-04-25T10:05:26.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0793f48640 1 Processor -- start 2024-04-25T10:05:26.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0793f48640 1 -- start start 2024-04-25T10:05:26.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0793f48640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f078c06ec30 0x7f078c152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0793f48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f078c0b37e0 0x7f078c14b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0793f48640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f078c15a7a0 0x7f078c1638a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0793f48640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c15ad00 con 0x7f078c0b37e0 2024-04-25T10:05:26.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0793f48640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c15ab80 con 0x7f078c06ec30 2024-04-25T10:05:26.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0793f48640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078c06e860 con 0x7f078c15a7a0 2024-04-25T10:05:26.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f07914e3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f078c0b37e0 0x7f078c14b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f07914e3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f078c0b37e0 0x7f078c14b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45256/0 (socket says 172.21.15.60:45256) 2024-04-25T10:05:26.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f07914e3640 1 -- 172.21.15.60:0/321473803 learned_addr learned my addr 172.21.15.60:0/321473803 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:26.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f0791ce4640 1 --2- 172.21.15.60:0/321473803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f078c06ec30 0x7f078c152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f07924e5640 1 --2- 172.21.15.60:0/321473803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f078c15a7a0 0x7f078c1638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f07914e3640 1 -- 172.21.15.60:0/321473803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f078c15a7a0 msgr2=0x7f078c1638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f07914e3640 1 --2- 172.21.15.60:0/321473803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f078c15a7a0 0x7f078c1638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f07914e3640 1 -- 172.21.15.60:0/321473803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f078c06ec30 msgr2=0x7f078c152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f07914e3640 1 --2- 172.21.15.60:0/321473803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f078c06ec30 0x7f078c152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.023+0000 7f07914e3640 1 -- 172.21.15.60:0/321473803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c1530b0 con 0x7f078c0b37e0 2024-04-25T10:05:26.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.027+0000 7f07914e3640 1 --2- 172.21.15.60:0/321473803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f078c0b37e0 0x7f078c14b8d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f077c00d7d0 tx=0x7f077c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.027+0000 7f07827fc640 1 -- 172.21.15.60:0/321473803 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c00b6c0 con 0x7f078c0b37e0 2024-04-25T10:05:26.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.027+0000 7f07827fc640 1 -- 172.21.15.60:0/321473803 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077c019070 con 0x7f078c0b37e0 2024-04-25T10:05:26.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.027+0000 7f0793f48640 1 -- 172.21.15.60:0/321473803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c1537c0 con 0x7f078c0b37e0 2024-04-25T10:05:26.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.027+0000 7f07827fc640 1 -- 172.21.15.60:0/321473803 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c00baa0 con 0x7f078c0b37e0 2024-04-25T10:05:26.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.027+0000 7f0793f48640 1 -- 172.21.15.60:0/321473803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c1515a0 con 0x7f078c0b37e0 2024-04-25T10:05:26.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.027+0000 7f0793f48640 1 -- 172.21.15.60:0/321473803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078c074770 con 0x7f078c0b37e0 2024-04-25T10:05:26.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.027+0000 7f07827fc640 1 -- 172.21.15.60:0/321473803 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f077c00bc40 con 0x7f078c0b37e0 2024-04-25T10:05:26.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.027+0000 7f07827fc640 1 --2- 172.21.15.60:0/321473803 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0770066f60 0x7f0770069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.027+0000 7f07827fc640 1 -- 172.21.15.60:0/321473803 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f077c0854c0 con 0x7f078c0b37e0 2024-04-25T10:05:26.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.027+0000 7f0791ce4640 1 --2- 172.21.15.60:0/321473803 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0770066f60 0x7f0770069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.027+0000 7f0791ce4640 1 --2- 172.21.15.60:0/321473803 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0770066f60 0x7f0770069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f07880ae000 tx=0x7f0788008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.031+0000 7f07827fc640 1 -- 172.21.15.60:0/321473803 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f077c0564d0 con 0x7f078c0b37e0 2024-04-25T10:05:26.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.195+0000 7f0793f48640 1 -- 172.21.15.60:0/321473803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f078c165090 con 0x7f078c0b37e0 2024-04-25T10:05:26.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.195+0000 7f07827fc640 1 -- 172.21.15.60:0/321473803 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f077c055dd0 con 0x7f078c0b37e0 2024-04-25T10:05:26.204 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:26.204 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:22.036582+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:26.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.199+0000 7f0793f48640 1 -- 172.21.15.60:0/321473803 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0770066f60 msgr2=0x7f0770069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.199+0000 7f0793f48640 1 --2- 172.21.15.60:0/321473803 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0770066f60 0x7f0770069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f07880ae000 tx=0x7f0788008040 comp rx=0 tx=0).stop 2024-04-25T10:05:26.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.199+0000 7f0793f48640 1 -- 172.21.15.60:0/321473803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f078c0b37e0 msgr2=0x7f078c14b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.199+0000 7f0793f48640 1 --2- 172.21.15.60:0/321473803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f078c0b37e0 0x7f078c14b8d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f077c00d7d0 tx=0x7f077c00dc90 comp rx=0 tx=0).stop 2024-04-25T10:05:26.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.199+0000 7f0793f48640 1 -- 172.21.15.60:0/321473803 shutdown_connections 2024-04-25T10:05:26.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.199+0000 7f0793f48640 1 --2- 172.21.15.60:0/321473803 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0770066f60 0x7f0770069420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.199+0000 7f0793f48640 1 --2- 172.21.15.60:0/321473803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f078c15a7a0 0x7f078c1638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.199+0000 7f0793f48640 1 --2- 172.21.15.60:0/321473803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f078c0b37e0 0x7f078c14b8d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.199+0000 7f0793f48640 1 --2- 172.21.15.60:0/321473803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f078c06ec30 0x7f078c152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.199+0000 7f0793f48640 1 -- 172.21.15.60:0/321473803 >> 172.21.15.60:0/321473803 conn(0x7f078c0b12d0 msgr2=0x7f078c15d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.199+0000 7f0793f48640 1 -- 172.21.15.60:0/321473803 shutdown_connections 2024-04-25T10:05:26.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.199+0000 7f0793f48640 1 -- 172.21.15.60:0/321473803 wait complete. 2024-04-25T10:05:26.224 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T10:05:26.224 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.335+0000 7fbfba45b640 1 Processor -- start 2024-04-25T10:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.335+0000 7fbfba45b640 1 -- start start 2024-04-25T10:05:26.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfba45b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbfb40b3ef0 0x7fbfb40cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfba45b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb406f130 con 0x7fbfb406ec30 2024-04-25T10:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfba45b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb406f410 con 0x7fbfb415a7a0 2024-04-25T10:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfba45b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb40cdaa0 con 0x7fbfb40b3ef0 2024-04-25T10:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbfb406ec30 0x7fbfb40b37e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55772/0 (socket says 172.21.15.60:55772) 2024-04-25T10:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb37fe640 1 -- 172.21.15.60:0/2945159617 learned_addr learned my addr 172.21.15.60:0/2945159617 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb89f8640 1 --2- 172.21.15.60:0/2945159617 >> v2:172.21.15.100:3300/0 conn(0x7fbfb40b3ef0 0x7fbfb40cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb2ffd640 1 -- 172.21.15.60:0/2945159617 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830613435 0 0) 0x7fbfb406f130 con 0x7fbfb406ec30 2024-04-25T10:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb2ffd640 1 -- 172.21.15.60:0/2945159617 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf9c003530 con 0x7fbfb406ec30 2024-04-25T10:05:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb2ffd640 1 -- 172.21.15.60:0/2945159617 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3115235661 0 0) 0x7fbf9c003530 con 0x7fbfb406ec30 2024-04-25T10:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb2ffd640 1 -- 172.21.15.60:0/2945159617 >> v2:172.21.15.100:3300/0 conn(0x7fbfb40b3ef0 msgr2=0x7fbfb40cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb2ffd640 1 --2- 172.21.15.60:0/2945159617 >> v2:172.21.15.100:3300/0 conn(0x7fbfb40b3ef0 0x7fbfb40cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb2ffd640 1 -- 172.21.15.60:0/2945159617 >> v1:172.21.15.100:6789/0 conn(0x7fbfb415a7a0 legacy=0x7fbfb4071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb2ffd640 1 -- 172.21.15.60:0/2945159617 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb406d540 con 0x7fbfb406ec30 2024-04-25T10:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb2ffd640 1 -- 172.21.15.60:0/2945159617 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fbfa8001a00 con 0x7fbfb406ec30 2024-04-25T10:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb2ffd640 1 -- 172.21.15.60:0/2945159617 >> v1:172.21.15.60:6789/0 conn(0x7fbfb406ec30 legacy=0x7fbfb40b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb2ffd640 1 --2- 172.21.15.60:0/2945159617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf9c0039c0 0x7fbf9c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb2ffd640 1 --2- 172.21.15.60:0/2945159617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf9c004810 0x7fbf9c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb2ffd640 1 --2- 172.21.15.60:0/2945159617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9c0051d0 0x7fbf9c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb2ffd640 1 -- 172.21.15.60:0/2945159617 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c00c040 con 0x7fbf9c0051d0 2024-04-25T10:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb2ffd640 1 -- 172.21.15.60:0/2945159617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c00c1f0 con 0x7fbf9c004810 2024-04-25T10:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb2ffd640 1 -- 172.21.15.60:0/2945159617 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c00c3f0 con 0x7fbf9c0039c0 2024-04-25T10:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb89f8640 1 --2- 172.21.15.60:0/2945159617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9c0051d0 0x7fbf9c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb37fe640 1 --2- 172.21.15.60:0/2945159617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf9c004810 0x7fbf9c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb3fff640 1 --2- 172.21.15.60:0/2945159617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf9c0039c0 0x7fbf9c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb89f8640 1 -- 172.21.15.60:0/2945159617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf9c0039c0 msgr2=0x7fbf9c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb89f8640 1 --2- 172.21.15.60:0/2945159617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf9c0039c0 0x7fbf9c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb89f8640 1 -- 172.21.15.60:0/2945159617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf9c004810 msgr2=0x7fbf9c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb89f8640 1 --2- 172.21.15.60:0/2945159617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf9c004810 0x7fbf9c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb89f8640 1 -- 172.21.15.60:0/2945159617 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c003530 con 0x7fbf9c0051d0 2024-04-25T10:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb37fe640 1 --2- 172.21.15.60:0/2945159617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf9c004810 0x7fbf9c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.339+0000 7fbfb3fff640 1 --2- 172.21.15.60:0/2945159617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf9c0039c0 0x7fbf9c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb89f8640 1 -- 172.21.15.60:0/2945159617 reap_dead start 2024-04-25T10:05:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb89f8640 1 --2- 172.21.15.60:0/2945159617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9c0051d0 0x7fbf9c00b9d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40b2680 tx=0x7fbfa401ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4437a55a31de34f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb2ffd640 1 -- 172.21.15.60:0/2945159617 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4009690 con 0x7fbf9c0051d0 2024-04-25T10:05:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb2ffd640 1 -- 172.21.15.60:0/2945159617 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa40a7840 con 0x7fbf9c0051d0 2024-04-25T10:05:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb2ffd640 1 -- 172.21.15.60:0/2945159617 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40a7c10 con 0x7fbf9c0051d0 2024-04-25T10:05:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfba45b640 1 -- 172.21.15.60:0/2945159617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9c0051d0 msgr2=0x7fbf9c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfba45b640 1 --2- 172.21.15.60:0/2945159617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9c0051d0 0x7fbf9c00b9d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40b2680 tx=0x7fbfa401ec40 comp rx=0 tx=0).stop 2024-04-25T10:05:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfba45b640 1 -- 172.21.15.60:0/2945159617 shutdown_connections 2024-04-25T10:05:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfba45b640 1 --2- 172.21.15.60:0/2945159617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9c0051d0 0x7fbf9c00b9d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfba45b640 1 -- 172.21.15.60:0/2945159617 >> 172.21.15.60:0/2945159617 conn(0x7fbfb40b12d0 msgr2=0x7fbfb40b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfba45b640 1 -- 172.21.15.60:0/2945159617 shutdown_connections 2024-04-25T10:05:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfba45b640 1 -- 172.21.15.60:0/2945159617 wait complete. 2024-04-25T10:05:26.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfba45b640 1 Processor -- start 2024-04-25T10:05:26.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfba45b640 1 -- start start 2024-04-25T10:05:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfba45b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9c0051d0 0x7fbfb414a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfba45b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbfb414bcd0 0x7fbfb415c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfba45b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfb40af7c0 0x7fbfb40afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfba45b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb406d8b0 con 0x7fbf9c0051d0 2024-04-25T10:05:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfba45b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb406d2f0 con 0x7fbfb40af7c0 2024-04-25T10:05:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfba45b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb40cd560 con 0x7fbfb414bcd0 2024-04-25T10:05:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9c0051d0 0x7fbfb414a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9c0051d0 0x7fbfb414a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45270/0 (socket says 172.21.15.60:45270) 2024-04-25T10:05:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb3fff640 1 -- 172.21.15.60:0/492633835 learned_addr learned my addr 172.21.15.60:0/492633835 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb37fe640 1 --2- 172.21.15.60:0/492633835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbfb414bcd0 0x7fbfb415c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb89f8640 1 --2- 172.21.15.60:0/492633835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfb40af7c0 0x7fbfb40afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb3fff640 1 -- 172.21.15.60:0/492633835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbfb414bcd0 msgr2=0x7fbfb415c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb3fff640 1 --2- 172.21.15.60:0/492633835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbfb414bcd0 0x7fbfb415c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb3fff640 1 -- 172.21.15.60:0/492633835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfb40af7c0 msgr2=0x7fbfb40afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb3fff640 1 --2- 172.21.15.60:0/492633835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfb40af7c0 0x7fbfb40afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb3fff640 1 -- 172.21.15.60:0/492633835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb415cb10 con 0x7fbf9c0051d0 2024-04-25T10:05:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb89f8640 1 --2- 172.21.15.60:0/492633835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfb40af7c0 0x7fbfb40afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb3fff640 1 --2- 172.21.15.60:0/492633835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9c0051d0 0x7fbfb414a270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00d5a0 tx=0x7fbfac00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb0ff9640 1 -- 172.21.15.60:0/492633835 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac005480 con 0x7fbf9c0051d0 2024-04-25T10:05:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.343+0000 7fbfb0ff9640 1 -- 172.21.15.60:0/492633835 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac017070 con 0x7fbf9c0051d0 2024-04-25T10:05:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.347+0000 7fbfba45b640 1 -- 172.21.15.60:0/492633835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb415cdf0 con 0x7fbf9c0051d0 2024-04-25T10:05:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.347+0000 7fbfb0ff9640 1 -- 172.21.15.60:0/492633835 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac003840 con 0x7fbf9c0051d0 2024-04-25T10:05:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.347+0000 7fbfba45b640 1 -- 172.21.15.60:0/492633835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb415d3b0 con 0x7fbf9c0051d0 2024-04-25T10:05:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.347+0000 7fbfba45b640 1 -- 172.21.15.60:0/492633835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb4074770 con 0x7fbf9c0051d0 2024-04-25T10:05:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.347+0000 7fbfb0ff9640 1 -- 172.21.15.60:0/492633835 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbfac0039e0 con 0x7fbf9c0051d0 2024-04-25T10:05:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.347+0000 7fbfb0ff9640 1 --2- 172.21.15.60:0/492633835 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbfa0066f60 0x7fbfa0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.347+0000 7fbfb0ff9640 1 -- 172.21.15.60:0/492633835 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfac084e20 con 0x7fbf9c0051d0 2024-04-25T10:05:26.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.347+0000 7fbfb37fe640 1 --2- 172.21.15.60:0/492633835 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbfa0066f60 0x7fbfa0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.347+0000 7fbfb37fe640 1 --2- 172.21.15.60:0/492633835 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbfa0066f60 0x7fbfa0069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8001dc0 tx=0x7fbfa8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.351+0000 7fbfb0ff9640 1 -- 172.21.15.60:0/492633835 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbfac055e30 con 0x7fbf9c0051d0 2024-04-25T10:05:26.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.511+0000 7fbfba45b640 1 -- 172.21.15.60:0/492633835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfb41552a0 con 0x7fbf9c0051d0 2024-04-25T10:05:26.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.515+0000 7fbfb0ff9640 1 -- 172.21.15.60:0/492633835 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16617 (secure 0 0 0) 0x7fbfac055730 con 0x7fbf9c0051d0 2024-04-25T10:05:26.522 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:26.522 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:22.036582+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.519+0000 7fbfba45b640 1 -- 172.21.15.60:0/492633835 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbfa0066f60 msgr2=0x7fbfa0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.519+0000 7fbfba45b640 1 --2- 172.21.15.60:0/492633835 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbfa0066f60 0x7fbfa0069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8001dc0 tx=0x7fbfa8009040 comp rx=0 tx=0).stop 2024-04-25T10:05:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.519+0000 7fbfba45b640 1 -- 172.21.15.60:0/492633835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9c0051d0 msgr2=0x7fbfb414a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.519+0000 7fbfba45b640 1 --2- 172.21.15.60:0/492633835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9c0051d0 0x7fbfb414a270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00d5a0 tx=0x7fbfac00da60 comp rx=0 tx=0).stop 2024-04-25T10:05:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.519+0000 7fbfba45b640 1 -- 172.21.15.60:0/492633835 shutdown_connections 2024-04-25T10:05:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.519+0000 7fbfba45b640 1 --2- 172.21.15.60:0/492633835 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbfa0066f60 0x7fbfa0069420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.519+0000 7fbfba45b640 1 --2- 172.21.15.60:0/492633835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfb40af7c0 0x7fbfb40afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.519+0000 7fbfba45b640 1 --2- 172.21.15.60:0/492633835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbfb414bcd0 0x7fbfb415c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.519+0000 7fbfba45b640 1 --2- 172.21.15.60:0/492633835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9c0051d0 0x7fbfb414a270 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.519+0000 7fbfba45b640 1 -- 172.21.15.60:0/492633835 >> 172.21.15.60:0/492633835 conn(0x7fbfb40b12d0 msgr2=0x7fbfb40aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.519+0000 7fbfba45b640 1 -- 172.21.15.60:0/492633835 shutdown_connections 2024-04-25T10:05:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.519+0000 7fbfba45b640 1 -- 172.21.15.60:0/492633835 wait complete. 2024-04-25T10:05:26.542 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T10:05:26.543 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T10:05:26.543 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T10:05:26.543 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T10:05:26.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T10:05:26.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T10:05:26.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T10:05:26.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T10:05:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.723+0000 7fcfb1802640 1 Processor -- start 2024-04-25T10:05:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.723+0000 7fcfb1802640 1 -- start start 2024-04-25T10:05:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe937e640 1 Processor -- start 2024-04-25T10:05:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f6aae66b640 1 Processor -- start 2024-04-25T10:05:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7fcfb1802640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfa40501b0 0x7fcfa4006540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7fcfb1802640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa4054120 con 0x7fcfa4033d00 2024-04-25T10:05:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7fcfb1802640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa4054400 con 0x7fcfa40df2b0 2024-04-25T10:05:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7fcfb1802640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa40df690 con 0x7fcfa40501b0 2024-04-25T10:05:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7fcfab7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fcfa4033d00 0x7fcfa4006a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55774/0 (socket says 172.21.15.60:55774) 2024-04-25T10:05:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7fcfab7fe640 1 -- 172.21.15.60:0/3726093468 learned_addr learned my addr 172.21.15.60:0/3726093468 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7fcfaa7fc640 1 --2- 172.21.15.60:0/3726093468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfa40501b0 0x7fcfa4006540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f6aae66b640 1 -- start start 2024-04-25T10:05:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe937e640 1 -- start start 2024-04-25T10:05:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe937e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbdc0df2b0 0x7ffbdc04faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe937e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffbdc007550 0x7ffbdc04ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe937e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbdc054120 con 0x7ffbdc033d00 2024-04-25T10:05:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe937e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbdc0df690 con 0x7ffbdc007550 2024-04-25T10:05:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe937e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc054400 con 0x7ffbdc0df2b0 2024-04-25T10:05:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f6aae66b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6aa0047c20 0x7f6aa00eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f6aae66b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6aa0005ea0 0x7f6aa00062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f6aae66b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa0034220 con 0x7f6aa0033e40 2024-04-25T10:05:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f6aae66b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa0050210 con 0x7f6aa0005ea0 2024-04-25T10:05:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f6aae66b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0037bc0 con 0x7f6aa0047c20 2024-04-25T10:05:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f6aade6a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6aa0005ea0 0x7f6aa00062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f6aade6a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6aa0005ea0 0x7f6aa00062f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45280/0 (socket says 172.21.15.60:45280) 2024-04-25T10:05:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f6aade6a640 1 -- 172.21.15.60:0/3008404563 learned_addr learned my addr 172.21.15.60:0/3008404563 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:26.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7fcfa9ffb640 1 -- 172.21.15.60:0/3726093468 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982749147 0 0) 0x7fcfa4054120 con 0x7fcfa4033d00 2024-04-25T10:05:26.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7fcfa9ffb640 1 -- 172.21.15.60:0/3726093468 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf94003530 con 0x7fcfa4033d00 2024-04-25T10:05:26.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe8b7d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffbdc007550 0x7ffbdc04ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7fcfa9ffb640 1 -- 172.21.15.60:0/3726093468 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678253984 0 0) 0x7fcfa4054400 con 0x7fcfa40df2b0 2024-04-25T10:05:26.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7fcfaa7fc640 1 -- 172.21.15.60:0/3726093468 >> v1:172.21.15.100:6789/0 conn(0x7fcfa40df2b0 legacy=0x7fcfa404faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7fcfaa7fc640 1 -- 172.21.15.60:0/3726093468 >> v1:172.21.15.60:6789/0 conn(0x7fcfa4033d00 legacy=0x7fcfa4006a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7fcfaa7fc640 1 -- 172.21.15.60:0/3726093468 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa40546e0 con 0x7fcfa40501b0 2024-04-25T10:05:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe8b7d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffbdc007550 0x7ffbdc04ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45276/0 (socket says 172.21.15.60:45276) 2024-04-25T10:05:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe3fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbdc0df2b0 0x7ffbdc04faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe3fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbdc0df2b0 0x7ffbdc04faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:37084/0 (socket says 172.21.15.60:37084) 2024-04-25T10:05:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ffbdc033d00 0x7ffbdc0e2d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55786/0 (socket says 172.21.15.60:55786) 2024-04-25T10:05:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe37fe640 1 -- 172.21.15.60:0/3251593163 learned_addr learned my addr 172.21.15.60:0/3251593163 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f6a9ffff640 1 -- 172.21.15.60:0/3008404563 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421944937 0 0) 0x7f6aa0034220 con 0x7f6aa0033e40 2024-04-25T10:05:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f6a9ffff640 1 -- 172.21.15.60:0/3008404563 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a94003530 con 0x7f6aa0033e40 2024-04-25T10:05:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f6aade6a640 1 -- 172.21.15.60:0/3008404563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6aa0047c20 msgr2=0x7f6aa00eb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f6aade6a640 1 --2- 172.21.15.60:0/3008404563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6aa0047c20 0x7f6aa00eb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f6aade6a640 1 -- 172.21.15.60:0/3008404563 >> v1:172.21.15.100:6789/0 conn(0x7f6aa0033e40 legacy=0x7f6aa00eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f6aade6a640 1 -- 172.21.15.60:0/3008404563 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa004fa30 con 0x7f6aa0005ea0 2024-04-25T10:05:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f0967fff640 1 Processor -- start 2024-04-25T10:05:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f0967fff640 1 -- start start 2024-04-25T10:05:26.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe8b7d640 1 -- 172.21.15.60:0/3251593163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbdc0df2b0 msgr2=0x7ffbdc04faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe8b7d640 1 --2- 172.21.15.60:0/3251593163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbdc0df2b0 0x7ffbdc04faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe8b7d640 1 -- 172.21.15.60:0/3251593163 >> v1:172.21.15.60:6789/0 conn(0x7ffbdc033d00 legacy=0x7ffbdc0e2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7ffbe8b7d640 1 -- 172.21.15.60:0/3251593163 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbdc0546e0 con 0x7ffbdc007550 2024-04-25T10:05:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f0967fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f09600e5060 0x7f096003a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f0967fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0960007430 0x7f09600078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f0967fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f096004f4e0 0x7f096004f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f0967fff640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09600e5440 con 0x7f09600e5060 2024-04-25T10:05:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f0967fff640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0960006390 con 0x7f0960007430 2024-04-25T10:05:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f0967fff640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0960006670 con 0x7f096004f4e0 2024-04-25T10:05:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f09677fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f096004f4e0 0x7f096004f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.727+0000 7f09667fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0960007430 0x7f09600078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f09677fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f096004f4e0 0x7f096004f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:37090/0 (socket says 172.21.15.60:37090) 2024-04-25T10:05:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f09677fe640 1 -- 172.21.15.60:0/2920550986 learned_addr learned my addr 172.21.15.60:0/2920550986 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d6c726640 1 Processor -- start 2024-04-25T10:05:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d6c726640 1 -- start start 2024-04-25T10:05:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f09667fc640 1 -- 172.21.15.60:0/2920550986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f096004f4e0 msgr2=0x7f096004f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f09667fc640 1 --2- 172.21.15.60:0/2920550986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f096004f4e0 0x7f096004f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f09667fc640 1 -- 172.21.15.60:0/2920550986 >> v2:172.21.15.60:3300/0 conn(0x7f09600e5060 msgr2=0x7f096003a930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:05:26.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f09667fc640 1 --2- 172.21.15.60:0/2920550986 >> v2:172.21.15.60:3300/0 conn(0x7f09600e5060 0x7f096003a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f09667fc640 1 -- 172.21.15.60:0/2920550986 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0960006950 con 0x7f0960007430 2024-04-25T10:05:26.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d6c726640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d641597c0 0x7f2d6406f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d6c726640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d640cdbb0 con 0x7f2d6415daa0 2024-04-25T10:05:26.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d6c726640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d640cde90 con 0x7f2d6406f870 2024-04-25T10:05:26.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d6c726640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d640ce170 con 0x7f2d641597c0 2024-04-25T10:05:26.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d6acc3640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2d6415daa0 0x7f2d640b2460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55798/0 (socket says 172.21.15.60:55798) 2024-04-25T10:05:26.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d6acc3640 1 -- 172.21.15.60:0/2870053596 learned_addr learned my addr 172.21.15.60:0/2870053596 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:26.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7fcfaa7fc640 1 --2- 172.21.15.60:0/3726093468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfa40501b0 0x7fcfa4006540 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00097f0 tx=0x7fcfa001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed89acdd1d44d4ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7fcfa9ffb640 1 -- 172.21.15.60:0/3726093468 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00b0070 con 0x7fcfa40501b0 2024-04-25T10:05:26.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7fcfa9ffb640 1 -- 172.21.15.60:0/3726093468 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa00ab600 con 0x7fcfa40501b0 2024-04-25T10:05:26.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7fcfa9ffb640 1 -- 172.21.15.60:0/3726093468 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00ab9b0 con 0x7fcfa40501b0 2024-04-25T10:05:26.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 -- 172.21.15.60:0/2870053596 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476004915 0 0) 0x7f2d640cdbb0 con 0x7f2d6415daa0 2024-04-25T10:05:26.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 -- 172.21.15.60:0/2870053596 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d50003530 con 0x7f2d6415daa0 2024-04-25T10:05:26.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 -- 172.21.15.60:0/2870053596 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432172571 0 0) 0x7f2d640cde90 con 0x7f2d6406f870 2024-04-25T10:05:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 -- 172.21.15.60:0/2870053596 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d640cdbb0 con 0x7f2d6406f870 2024-04-25T10:05:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 -- 172.21.15.60:0/2870053596 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 207741067 0 0) 0x7f2d50003530 con 0x7f2d6415daa0 2024-04-25T10:05:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 -- 172.21.15.60:0/2870053596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d641597c0 msgr2=0x7f2d6406f330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:05:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 --2- 172.21.15.60:0/2870053596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d641597c0 0x7f2d6406f330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 -- 172.21.15.60:0/2870053596 >> v1:172.21.15.100:6789/0 conn(0x7f2d6406f870 legacy=0x7f2d6415d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 -- 172.21.15.60:0/2870053596 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d640ce450 con 0x7f2d6415daa0 2024-04-25T10:05:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 -- 172.21.15.60:0/2870053596 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f2d5c001a30 con 0x7f2d6415daa0 2024-04-25T10:05:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7fedb2d2f640 1 Processor -- start 2024-04-25T10:05:26.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 -- 172.21.15.60:0/2870053596 >> v1:172.21.15.60:6789/0 conn(0x7f2d6415daa0 legacy=0x7f2d640b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 --2- 172.21.15.60:0/2870053596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d50003b80 0x7f2d50003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 --2- 172.21.15.60:0/2870053596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d50004810 0x7f2d50004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7fedb2d2f640 1 -- start start 2024-04-25T10:05:26.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 --2- 172.21.15.60:0/2870053596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d500052f0 0x7f2d500059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 -- 172.21.15.60:0/2870053596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d640ce170 con 0x7f2d500052f0 2024-04-25T10:05:26.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 -- 172.21.15.60:0/2870053596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d500061c0 con 0x7f2d50004810 2024-04-25T10:05:26.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d694c0640 1 -- 172.21.15.60:0/2870053596 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d500063c0 con 0x7f2d50003b80 2024-04-25T10:05:26.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f6aade6a640 1 --2- 172.21.15.60:0/3008404563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6aa0005ea0 0x7f6aa00062f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009870 tx=0x7f6aa401ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf3637d6a476de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d69cc1640 1 --2- 172.21.15.60:0/2870053596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d50004810 0x7f2d50004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d6acc3640 1 --2- 172.21.15.60:0/2870053596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d500052f0 0x7f2d500059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7fec2110f640 1 Processor -- start 2024-04-25T10:05:26.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f87d848f640 1 Processor -- start 2024-04-25T10:05:26.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7fec2110f640 1 -- start start 2024-04-25T10:05:26.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7fcfb1802640 1 -- 172.21.15.60:0/3726093468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfa40501b0 msgr2=0x7fcfa4006540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7fcfb1802640 1 --2- 172.21.15.60:0/3726093468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfa40501b0 0x7fcfa4006540 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00097f0 tx=0x7fcfa001c3e0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7ffbe8b7d640 1 --2- 172.21.15.60:0/3251593163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbdc007550 0x7ffbdc04ffe0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4068340 tx=0x7ffbe4098790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108cab114433ba30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7fcfb1802640 1 -- 172.21.15.60:0/3726093468 shutdown_connections 2024-04-25T10:05:26.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7fcfb1802640 1 --2- 172.21.15.60:0/3726093468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfa40501b0 0x7fcfa4006540 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7fcfb1802640 1 -- 172.21.15.60:0/3726093468 >> 172.21.15.60:0/3726093468 conn(0x7fcfa4009fc0 msgr2=0x7fcfa400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d6a4c2640 1 --2- 172.21.15.60:0/2870053596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d50003b80 0x7f2d50003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7ffbe2ffd640 1 -- 172.21.15.60:0/3251593163 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4125720 con 0x7ffbdc007550 2024-04-25T10:05:26.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7ffbe2ffd640 1 -- 172.21.15.60:0/3251593163 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe41258c0 con 0x7ffbdc007550 2024-04-25T10:05:26.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7ffbe2ffd640 1 -- 172.21.15.60:0/3251593163 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4125d30 con 0x7ffbdc007550 2024-04-25T10:05:26.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f6a9ffff640 1 -- 172.21.15.60:0/3008404563 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40b5070 con 0x7f6aa0005ea0 2024-04-25T10:05:26.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f6a9ffff640 1 -- 172.21.15.60:0/3008404563 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa40a8a70 con 0x7f6aa0005ea0 2024-04-25T10:05:26.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f6a9ffff640 1 -- 172.21.15.60:0/3008404563 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40a8e40 con 0x7f6aa0005ea0 2024-04-25T10:05:26.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d69cc1640 1 -- 172.21.15.60:0/2870053596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d50003b80 msgr2=0x7f2d50003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d69cc1640 1 --2- 172.21.15.60:0/2870053596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d50003b80 0x7f2d50003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d69cc1640 1 -- 172.21.15.60:0/2870053596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d500052f0 msgr2=0x7f2d500059a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d69cc1640 1 --2- 172.21.15.60:0/2870053596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d500052f0 0x7f2d500059a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f2d69cc1640 1 -- 172.21.15.60:0/2870053596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d640ce450 con 0x7f2d50004810 2024-04-25T10:05:26.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7ffbe937e640 1 -- 172.21.15.60:0/3251593163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbdc007550 msgr2=0x7ffbdc04ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7ffbe937e640 1 --2- 172.21.15.60:0/3251593163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbdc007550 0x7ffbdc04ffe0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4068340 tx=0x7ffbe4098790 comp rx=0 tx=0).stop 2024-04-25T10:05:26.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fec2110f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fec1c06f480 0x7fec1c15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f09667fc640 1 --2- 172.21.15.60:0/2920550986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0960007430 0x7f09600078b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f096000b760 tx=0x7f095c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53b74b9d7eaed07f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7ffbe937e640 1 -- 172.21.15.60:0/3251593163 shutdown_connections 2024-04-25T10:05:26.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7ffbe937e640 1 --2- 172.21.15.60:0/3251593163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbdc007550 0x7ffbdc04ffe0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7ffbe937e640 1 --2- 172.21.15.60:0/3251593163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbdc0df2b0 0x7ffbdc04faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7ffbe937e640 1 -- 172.21.15.60:0/3251593163 >> 172.21.15.60:0/3251593163 conn(0x7ffbdc009fc0 msgr2=0x7ffbdc00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fec2110f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fec1c06d900 0x7fec1c06b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f6aae66b640 1 -- 172.21.15.60:0/3008404563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6aa0005ea0 msgr2=0x7f6aa00062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f6aae66b640 1 --2- 172.21.15.60:0/3008404563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6aa0005ea0 0x7f6aa00062f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009870 tx=0x7f6aa401ed70 comp rx=0 tx=0).stop 2024-04-25T10:05:26.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fec2110f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec1c157c20 con 0x7fec1c157840 2024-04-25T10:05:26.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fec2110f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1c06ad40 con 0x7fec1c06d900 2024-04-25T10:05:26.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fec2110f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1c06dd40 con 0x7fec1c06f480 2024-04-25T10:05:26.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fec1bfff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fec1c157840 0x7fec1c06ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:48700/0 (socket says 172.21.15.60:48700) 2024-04-25T10:05:26.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fec1bfff640 1 -- 172.21.15.60:0/1904415827 learned_addr learned my addr 172.21.15.60:0/1904415827 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:26.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fec2090e640 1 --2- 172.21.15.60:0/1904415827 >> v2:172.21.15.60:3300/0 conn(0x7fec1c06d900 0x7fec1c06b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fec1b7fe640 1 --2- 172.21.15.60:0/1904415827 >> v2:172.21.15.100:3300/0 conn(0x7fec1c06f480 0x7fec1c15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fec2090e640 1 -- 172.21.15.60:0/1904415827 >> v2:172.21.15.100:3300/0 conn(0x7fec1c06f480 msgr2=0x7fec1c15a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fec2090e640 1 --2- 172.21.15.60:0/1904415827 >> v2:172.21.15.100:3300/0 conn(0x7fec1c06f480 0x7fec1c15a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fec2090e640 1 -- 172.21.15.60:0/1904415827 >> v1:172.21.15.100:6789/0 conn(0x7fec1c157840 legacy=0x7fec1c06ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fec2090e640 1 -- 172.21.15.60:0/1904415827 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c06df90 con 0x7fec1c06d900 2024-04-25T10:05:26.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fcfb1802640 1 -- 172.21.15.60:0/3726093468 shutdown_connections 2024-04-25T10:05:26.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fcfb1802640 1 -- 172.21.15.60:0/3726093468 wait complete. 2024-04-25T10:05:26.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f0965ffb640 1 -- 172.21.15.60:0/2920550986 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c0b0070 con 0x7f0960007430 2024-04-25T10:05:26.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f0965ffb640 1 -- 172.21.15.60:0/2920550986 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095c0ab600 con 0x7f0960007430 2024-04-25T10:05:26.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f0965ffb640 1 -- 172.21.15.60:0/2920550986 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c0ab9d0 con 0x7f0960007430 2024-04-25T10:05:26.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f0967fff640 1 -- 172.21.15.60:0/2920550986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0960007430 msgr2=0x7f09600078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f0967fff640 1 --2- 172.21.15.60:0/2920550986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0960007430 0x7f09600078b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f096000b760 tx=0x7f095c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f0967fff640 1 -- 172.21.15.60:0/2920550986 shutdown_connections 2024-04-25T10:05:26.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f0967fff640 1 --2- 172.21.15.60:0/2920550986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f096004f4e0 0x7f096004f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f0967fff640 1 --2- 172.21.15.60:0/2920550986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0960007430 0x7f09600078b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f0967fff640 1 --2- 172.21.15.60:0/2920550986 >> v2:172.21.15.60:3300/0 conn(0x7f09600e5060 0x7f096003a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f0967fff640 1 -- 172.21.15.60:0/2920550986 >> 172.21.15.60:0/2920550986 conn(0x7f096000a410 msgr2=0x7f096000a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f0967fff640 1 -- 172.21.15.60:0/2920550986 shutdown_connections 2024-04-25T10:05:26.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7ffbe937e640 1 -- 172.21.15.60:0/3251593163 shutdown_connections 2024-04-25T10:05:26.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.731+0000 7f87d848f640 1 -- start start 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f87d848f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f87c8033e40 0x7f87c80eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f87d848f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8047c20 0x7f87c80eb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f87d848f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c8034220 con 0x7f87c80ebc00 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f87d848f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c8050210 con 0x7f87c8033e40 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f87d848f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c8037bc0 con 0x7f87c8047c20 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f87d622b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f87c8033e40 0x7f87c80eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f87d5a2a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8047c20 0x7f87c80eb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f87d5a2a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8047c20 0x7f87c80eb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:37118/0 (socket says 172.21.15.60:37118) 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f87d5a2a640 1 -- 172.21.15.60:0/2816410609 learned_addr learned my addr 172.21.15.60:0/2816410609 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f87d622b640 1 -- 172.21.15.60:0/2816410609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8047c20 msgr2=0x7f87c80eb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f87d622b640 1 --2- 172.21.15.60:0/2816410609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8047c20 0x7f87c80eb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f87d622b640 1 -- 172.21.15.60:0/2816410609 >> v1:172.21.15.100:6789/0 conn(0x7f87c80ebc00 legacy=0x7f87c8052f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f87d622b640 1 -- 172.21.15.60:0/2816410609 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c804fa30 con 0x7f87c8033e40 2024-04-25T10:05:26.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f87d5a2a640 1 --2- 172.21.15.60:0/2816410609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8047c20 0x7f87c80eb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fedb2d2f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feda4053c20 0x7feda4054080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fedb2d2f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feda4006800 0x7feda4006c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fedb152c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feda4053c20 0x7feda4054080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fedb252e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feda4006800 0x7feda4006c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fedb1d2d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7feda40e32e0 0x7feda403e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:48726/0 (socket says 172.21.15.60:48726) 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fedb1d2d640 1 -- 172.21.15.60:0/445873174 learned_addr learned my addr 172.21.15.60:0/445873174 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fedb2d2f640 1 -- 172.21.15.60:0/445873174 --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda4057df0 con 0x7feda40e32e0 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fedb2d2f640 1 -- 172.21.15.60:0/445873174 --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda403bfd0 con 0x7feda4006800 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fedb2d2f640 1 -- 172.21.15.60:0/445873174 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda40545c0 con 0x7feda4053c20 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fedb0d2b640 1 -- 172.21.15.60:0/445873174 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585533784 0 0) 0x7feda4057df0 con 0x7feda40e32e0 2024-04-25T10:05:26.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fedb0d2b640 1 -- 172.21.15.60:0/445873174 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed98003530 con 0x7feda40e32e0 2024-04-25T10:05:26.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fedb152c640 1 -- 172.21.15.60:0/445873174 >> v2:172.21.15.60:3300/0 conn(0x7feda4006800 msgr2=0x7feda4006c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fedb152c640 1 --2- 172.21.15.60:0/445873174 >> v2:172.21.15.60:3300/0 conn(0x7feda4006800 0x7feda4006c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fedb152c640 1 -- 172.21.15.60:0/445873174 >> v1:172.21.15.100:6789/0 conn(0x7feda40e32e0 legacy=0x7feda403e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fedb152c640 1 -- 172.21.15.60:0/445873174 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda40580d0 con 0x7feda4053c20 2024-04-25T10:05:26.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f2d69cc1640 1 --2- 172.21.15.60:0/2870053596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d50004810 0x7f2d50004c90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2d540018c0 tx=0x7f2d5401ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2ffa86a31184526 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f2d694c0640 1 -- 172.21.15.60:0/2870053596 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d54001510 con 0x7f2d50004810 2024-04-25T10:05:26.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f2d694c0640 1 -- 172.21.15.60:0/2870053596 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d540052d0 con 0x7f2d50004810 2024-04-25T10:05:26.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f2d694c0640 1 -- 172.21.15.60:0/2870053596 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d54007260 con 0x7f2d50004810 2024-04-25T10:05:26.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f2d6c726640 1 -- 172.21.15.60:0/2870053596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d50004810 msgr2=0x7f2d50004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f2d6c726640 1 --2- 172.21.15.60:0/2870053596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d50004810 0x7f2d50004c90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2d540018c0 tx=0x7f2d5401ecd0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f2d6acc3640 1 -- 172.21.15.60:0/2870053596 reap_dead start 2024-04-25T10:05:26.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f2d6c726640 1 -- 172.21.15.60:0/2870053596 shutdown_connections 2024-04-25T10:05:26.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f2d6c726640 1 -- 172.21.15.60:0/2870053596 >> 172.21.15.60:0/2870053596 conn(0x7f2d6406ac10 msgr2=0x7f2d6406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f6aae66b640 1 -- 172.21.15.60:0/3008404563 shutdown_connections 2024-04-25T10:05:26.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f6aae66b640 1 --2- 172.21.15.60:0/3008404563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6aa0005ea0 0x7f6aa00062f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f6aae66b640 1 --2- 172.21.15.60:0/3008404563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6aa0047c20 0x7f6aa00eb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f6aae66b640 1 -- 172.21.15.60:0/3008404563 >> 172.21.15.60:0/3008404563 conn(0x7f6aa0009fc0 msgr2=0x7f6aa000a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fcfb1802640 1 Processor -- start 2024-04-25T10:05:26.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fcfb1802640 1 -- start start 2024-04-25T10:05:26.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f2d6c726640 1 -- 172.21.15.60:0/2870053596 shutdown_connections 2024-04-25T10:05:26.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f2d6c726640 1 -- 172.21.15.60:0/2870053596 wait complete. 2024-04-25T10:05:26.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f6aae66b640 1 -- 172.21.15.60:0/3008404563 shutdown_connections 2024-04-25T10:05:26.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfb1802640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfa4033d00 0x7fcfa40dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfb1802640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfa40501b0 0x7fcfa40cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfb1802640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfa40df2b0 0x7fcfa40d2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfb1802640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa404f7e0 con 0x7fcfa4033d00 2024-04-25T10:05:26.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfb1802640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa404f660 con 0x7fcfa40df2b0 2024-04-25T10:05:26.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfb1802640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa40ed060 con 0x7fcfa40501b0 2024-04-25T10:05:26.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7ffbe937e640 1 -- 172.21.15.60:0/3251593163 wait complete. 2024-04-25T10:05:26.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7fedb252e640 1 --2- 172.21.15.60:0/445873174 >> v2:172.21.15.60:3300/0 conn(0x7feda4006800 0x7feda4006c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfaa7fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfa40501b0 0x7fcfa40cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfaa7fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfa40501b0 0x7fcfa40cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:37144/0 (socket says 172.21.15.60:37144) 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfaa7fc640 1 -- 172.21.15.60:0/2373592610 learned_addr learned my addr 172.21.15.60:0/2373592610 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfab7fe640 1 --2- 172.21.15.60:0/2373592610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfa40df2b0 0x7fcfa40d2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.735+0000 7f0967fff640 1 -- 172.21.15.60:0/2920550986 wait complete. 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f0967fff640 1 Processor -- start 2024-04-25T10:05:26.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfaaffd640 1 --2- 172.21.15.60:0/2373592610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfa4033d00 0x7fcfa40dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d6c726640 1 Processor -- start 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d6c726640 1 -- start start 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f87d622b640 1 --2- 172.21.15.60:0/2816410609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87c8033e40 0x7f87c80eb180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0097c0 tx=0x7f87cc01ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65b8246c21df7bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f87d5229640 1 -- 172.21.15.60:0/2816410609 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0b5070 con 0x7f87c8033e40 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f87d5229640 1 -- 172.21.15.60:0/2816410609 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc0a8990 con 0x7f87c8033e40 2024-04-25T10:05:26.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f87d5229640 1 -- 172.21.15.60:0/2816410609 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0a8d60 con 0x7f87c8033e40 2024-04-25T10:05:26.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7ffbe937e640 1 Processor -- start 2024-04-25T10:05:26.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f6aae66b640 1 -- 172.21.15.60:0/3008404563 wait complete. 2024-04-25T10:05:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfaa7fc640 1 -- 172.21.15.60:0/2373592610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfa40df2b0 msgr2=0x7fcfa40d2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfaa7fc640 1 --2- 172.21.15.60:0/2373592610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfa40df2b0 0x7fcfa40d2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfaa7fc640 1 -- 172.21.15.60:0/2373592610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfa4033d00 msgr2=0x7fcfa40dfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfaa7fc640 1 --2- 172.21.15.60:0/2373592610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfa4033d00 0x7fcfa40dfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfaa7fc640 1 -- 172.21.15.60:0/2373592610 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa40cd430 con 0x7fcfa40501b0 2024-04-25T10:05:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcfaa7fc640 1 --2- 172.21.15.60:0/2373592610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfa40501b0 0x7fcfa40cc5b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00ae040 tx=0x7fcfa0019180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcf93fff640 1 -- 172.21.15.60:0/2373592610 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00b9020 con 0x7fcfa40501b0 2024-04-25T10:05:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f0967fff640 1 -- start start 2024-04-25T10:05:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f0967fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0960007430 0x7f09600df870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7ffbe937e640 1 -- start start 2024-04-25T10:05:26.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f87d848f640 1 -- 172.21.15.60:0/2816410609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87c8033e40 msgr2=0x7f87c80eb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f87d848f640 1 --2- 172.21.15.60:0/2816410609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87c8033e40 0x7f87c80eb180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0097c0 tx=0x7f87cc01ec90 comp rx=0 tx=0).stop 2024-04-25T10:05:26.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f0967fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f096004f4e0 0x7f0960036bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fedb152c640 1 --2- 172.21.15.60:0/445873174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feda4053c20 0x7feda4054080 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feda80097c0 tx=0x7feda801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=373b2164392ed081 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcf93fff640 1 -- 172.21.15.60:0/2373592610 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa00ab9b0 con 0x7fcfa40501b0 2024-04-25T10:05:26.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fcf93fff640 1 -- 172.21.15.60:0/2373592610 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00b0040 con 0x7fcfa40501b0 2024-04-25T10:05:26.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fedb0d2b640 1 -- 172.21.15.60:0/445873174 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda80b0070 con 0x7feda4053c20 2024-04-25T10:05:26.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fedb0d2b640 1 -- 172.21.15.60:0/445873174 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda80ab600 con 0x7feda4053c20 2024-04-25T10:05:26.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7fedb0d2b640 1 -- 172.21.15.60:0/445873174 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda80ab9d0 con 0x7feda4053c20 2024-04-25T10:05:26.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f0967fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09600e5060 0x7f096004d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f0966ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0960007430 0x7f09600df870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d6c726640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d50004810 0x7f2d6406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d6c726640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d6415d360 0x7f2d6415b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d6c726640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6415bad0 0x7f2d641f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d6c726640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d640cec50 con 0x7f2d50004810 2024-04-25T10:05:26.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d6c726640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d64159ba0 con 0x7f2d6415d360 2024-04-25T10:05:26.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d6c726640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d640cedd0 con 0x7f2d6415bad0 2024-04-25T10:05:26.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f87d848f640 1 -- 172.21.15.60:0/2816410609 shutdown_connections 2024-04-25T10:05:26.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f87d848f640 1 --2- 172.21.15.60:0/2816410609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8047c20 0x7f87c80eb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f87d848f640 1 --2- 172.21.15.60:0/2816410609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87c8033e40 0x7f87c80eb180 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f87d848f640 1 -- 172.21.15.60:0/2816410609 >> 172.21.15.60:0/2816410609 conn(0x7f87c8009fc0 msgr2=0x7f87c800a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f0966ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0960007430 0x7f09600df870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:38996/0 (socket says 172.21.15.60:38996) 2024-04-25T10:05:26.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7ffbe937e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbdc007550 0x7ffbdc180740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f6aae66b640 1 Processor -- start 2024-04-25T10:05:26.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f6aae66b640 1 -- start start 2024-04-25T10:05:26.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f6aae66b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa0005ea0 0x7f6aa0049c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f6aae66b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6aa0033e40 0x7f6aa00482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f6aae66b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6aa0047c20 0x7f6aa017d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f6aae66b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa004fd80 con 0x7f6aa0033e40 2024-04-25T10:05:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f6aae66b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa004f090 con 0x7f6aa0005ea0 2024-04-25T10:05:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f6aae66b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa004f5c0 con 0x7f6aa0047c20 2024-04-25T10:05:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f6aad669640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa0005ea0 0x7f6aa0049c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f6aad669640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa0005ea0 0x7f6aa0049c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:39024/0 (socket says 172.21.15.60:39024) 2024-04-25T10:05:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f6aad669640 1 -- 172.21.15.60:0/1208109695 learned_addr learned my addr 172.21.15.60:0/1208109695 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f09667fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f096004f4e0 0x7f0960036bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f0966ffd640 1 -- 172.21.15.60:0/970433770 learned_addr learned my addr 172.21.15.60:0/970433770 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f0967fff640 1 -- 172.21.15.60:0/970433770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0960034530 con 0x7f09600e5060 2024-04-25T10:05:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f0967fff640 1 -- 172.21.15.60:0/970433770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09600343b0 con 0x7f0960007430 2024-04-25T10:05:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f0967fff640 1 -- 172.21.15.60:0/970433770 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09600346b0 con 0x7f096004f4e0 2024-04-25T10:05:26.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f09677fe640 1 --2- 172.21.15.60:0/970433770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09600e5060 0x7f096004d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f09677fe640 1 -- 172.21.15.60:0/970433770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f096004f4e0 msgr2=0x7f0960036bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f09677fe640 1 --2- 172.21.15.60:0/970433770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f096004f4e0 0x7f0960036bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f09677fe640 1 -- 172.21.15.60:0/970433770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0960007430 msgr2=0x7f09600df870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f09677fe640 1 --2- 172.21.15.60:0/970433770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0960007430 0x7f09600df870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f09677fe640 1 -- 172.21.15.60:0/970433770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09600355a0 con 0x7f09600e5060 2024-04-25T10:05:26.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7ffbe937e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbdc033d00 0x7ffbdc0d5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7ffbe937e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbdc0df2b0 0x7ffbdc0d63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7ffbe937e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc04f7e0 con 0x7ffbdc0df2b0 2024-04-25T10:05:26.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7ffbe937e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc04f660 con 0x7ffbdc007550 2024-04-25T10:05:26.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7ffbe937e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc0ec050 con 0x7ffbdc033d00 2024-04-25T10:05:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7ffbe3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbdc007550 0x7ffbdc180740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7ffbe37fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbdc033d00 0x7ffbdc0d5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7ffbe3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbdc007550 0x7ffbdc180740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:39016/0 (socket says 172.21.15.60:39016) 2024-04-25T10:05:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7ffbe3fff640 1 -- 172.21.15.60:0/138666357 learned_addr learned my addr 172.21.15.60:0/138666357 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7ffbe8b7d640 1 --2- 172.21.15.60:0/138666357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbdc0df2b0 0x7ffbdc0d63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7ffbe3fff640 1 -- 172.21.15.60:0/138666357 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbdc033d00 msgr2=0x7ffbdc0d5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7ffbe3fff640 1 --2- 172.21.15.60:0/138666357 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbdc033d00 0x7ffbdc0d5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7ffbe3fff640 1 -- 172.21.15.60:0/138666357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbdc0df2b0 msgr2=0x7ffbdc0d63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7ffbe3fff640 1 --2- 172.21.15.60:0/138666357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbdc0df2b0 0x7ffbdc0d63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7ffbe3fff640 1 -- 172.21.15.60:0/138666357 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbdc04a130 con 0x7ffbdc007550 2024-04-25T10:05:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f87d848f640 1 -- 172.21.15.60:0/2816410609 shutdown_connections 2024-04-25T10:05:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f87d848f640 1 -- 172.21.15.60:0/2816410609 wait complete. 2024-04-25T10:05:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d848f640 1 Processor -- start 2024-04-25T10:05:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d69cc1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d6415d360 0x7f2d6415b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d69cc1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d6415d360 0x7f2d6415b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:39002/0 (socket says 172.21.15.60:39002) 2024-04-25T10:05:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d69cc1640 1 -- 172.21.15.60:0/3840804343 learned_addr learned my addr 172.21.15.60:0/3840804343 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d6acc3640 1 --2- 172.21.15.60:0/3840804343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6415bad0 0x7f2d641f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d6a4c2640 1 --2- 172.21.15.60:0/3840804343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d50004810 0x7f2d6406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d69cc1640 1 -- 172.21.15.60:0/3840804343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6415bad0 msgr2=0x7f2d641f7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d69cc1640 1 --2- 172.21.15.60:0/3840804343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6415bad0 0x7f2d641f7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d69cc1640 1 -- 172.21.15.60:0/3840804343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d50004810 msgr2=0x7f2d6406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d69cc1640 1 --2- 172.21.15.60:0/3840804343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d50004810 0x7f2d6406e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.739+0000 7f2d69cc1640 1 -- 172.21.15.60:0/3840804343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6415c2e0 con 0x7f2d6415d360 2024-04-25T10:05:26.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f2d69cc1640 1 --2- 172.21.15.60:0/3840804343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d6415d360 0x7f2d6415b590 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2d54005040 tx=0x7f2d50005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d848f640 1 -- start start 2024-04-25T10:05:26.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7ffbe3fff640 1 --2- 172.21.15.60:0/138666357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbdc007550 0x7ffbdc180740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800d5f0 tx=0x7ffbd800dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f6aad669640 1 -- 172.21.15.60:0/1208109695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6aa0047c20 msgr2=0x7f6aa017d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f6aad669640 1 --2- 172.21.15.60:0/1208109695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6aa0047c20 0x7f6aa017d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f6aad669640 1 -- 172.21.15.60:0/1208109695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6aa0033e40 msgr2=0x7f6aa00482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f6aad669640 1 --2- 172.21.15.60:0/1208109695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6aa0033e40 0x7f6aa00482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f6aad669640 1 -- 172.21.15.60:0/1208109695 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa0048ea0 con 0x7f6aa0005ea0 2024-04-25T10:05:26.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f09677fe640 1 --2- 172.21.15.60:0/970433770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09600e5060 0x7f096004d860 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f095800e7f0 tx=0x7f095800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7ffbe0ff9640 1 -- 172.21.15.60:0/138666357 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd80125f0 con 0x7ffbdc007550 2024-04-25T10:05:26.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7ffbe0ff9640 1 -- 172.21.15.60:0/138666357 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd8019070 con 0x7ffbdc007550 2024-04-25T10:05:26.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7ffbe937e640 1 -- 172.21.15.60:0/138666357 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbdc04a410 con 0x7ffbdc007550 2024-04-25T10:05:26.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7ffbe0ff9640 1 -- 172.21.15.60:0/138666357 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd80129b0 con 0x7ffbdc007550 2024-04-25T10:05:26.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7ffbe937e640 1 -- 172.21.15.60:0/138666357 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbdc181b20 con 0x7ffbdc007550 2024-04-25T10:05:26.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f2d5affd640 1 -- 172.21.15.60:0/3840804343 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d540bf070 con 0x7f2d6415d360 2024-04-25T10:05:26.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f2d6c726640 1 -- 172.21.15.60:0/3840804343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6415c5c0 con 0x7f2d6415d360 2024-04-25T10:05:26.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f2d6c726640 1 -- 172.21.15.60:0/3840804343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6415cb80 con 0x7f2d6415d360 2024-04-25T10:05:26.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f2d5affd640 1 -- 172.21.15.60:0/3840804343 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d54007640 con 0x7f2d6415d360 2024-04-25T10:05:26.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f2d5affd640 1 -- 172.21.15.60:0/3840804343 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d540050d0 con 0x7f2d6415d360 2024-04-25T10:05:26.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d848f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8033e40 0x7f87c80d4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d848f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87c8047c20 0x7f87c80d5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d848f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87c80ebc00 0x7f87c80e9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d848f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c804fd80 con 0x7f87c8047c20 2024-04-25T10:05:26.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d848f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c804f090 con 0x7f87c80ebc00 2024-04-25T10:05:26.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d848f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c804f5c0 con 0x7f87c8033e40 2024-04-25T10:05:26.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d6a2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87c80ebc00 0x7f87c80e9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d622b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8033e40 0x7f87c80d4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d622b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8033e40 0x7f87c80d4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:37196/0 (socket says 172.21.15.60:37196) 2024-04-25T10:05:26.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d622b640 1 -- 172.21.15.60:0/243358901 learned_addr learned my addr 172.21.15.60:0/243358901 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:26.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fec2090e640 1 --2- 172.21.15.60:0/1904415827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec1c06d900 0x7fec1c06b1d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fec1c06aec0 tx=0x7fec1402d7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db39d1b275086675 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7ffbe937e640 1 -- 172.21.15.60:0/138666357 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffbdc0053a0 con 0x7ffbdc007550 2024-04-25T10:05:26.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fedb2d2f640 1 -- 172.21.15.60:0/445873174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feda4053c20 msgr2=0x7feda4054080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fedb2d2f640 1 --2- 172.21.15.60:0/445873174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feda4053c20 0x7feda4054080 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feda80097c0 tx=0x7feda801c4c0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fedb2d2f640 1 -- 172.21.15.60:0/445873174 shutdown_connections 2024-04-25T10:05:26.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fedb2d2f640 1 --2- 172.21.15.60:0/445873174 >> v2:172.21.15.60:3300/0 conn(0x7feda4006800 0x7feda4006c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fedb2d2f640 1 --2- 172.21.15.60:0/445873174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feda4053c20 0x7feda4054080 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fedb2d2f640 1 -- 172.21.15.60:0/445873174 >> 172.21.15.60:0/445873174 conn(0x7feda40559f0 msgr2=0x7feda4055e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d5a2a640 1 --2- 172.21.15.60:0/243358901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87c8047c20 0x7f87c80d5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f6aad669640 1 --2- 172.21.15.60:0/1208109695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa0005ea0 0x7f6aa0049c60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8069a80 tx=0x7f6aa80742b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f2d6c726640 1 -- 172.21.15.60:0/3840804343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2d4c001040 con 0x7f2d6415d360 2024-04-25T10:05:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f094f7fe640 1 -- 172.21.15.60:0/970433770 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0958017070 con 0x7f09600e5060 2024-04-25T10:05:26.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f094f7fe640 1 -- 172.21.15.60:0/970433770 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095800c6d0 con 0x7f09600e5060 2024-04-25T10:05:26.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fcfb1802640 1 -- 172.21.15.60:0/2373592610 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa40cd030 con 0x7fcfa40501b0 2024-04-25T10:05:26.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fcfb1802640 1 -- 172.21.15.60:0/2373592610 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa40d1c40 con 0x7fcfa40501b0 2024-04-25T10:05:26.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f094f7fe640 1 -- 172.21.15.60:0/970433770 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095800caa0 con 0x7f09600e5060 2024-04-25T10:05:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fedb2d2f640 1 -- 172.21.15.60:0/445873174 shutdown_connections 2024-04-25T10:05:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fcfb1802640 1 -- 172.21.15.60:0/2373592610 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcfa40053a0 con 0x7fcfa40501b0 2024-04-25T10:05:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fec1affd640 1 -- 172.21.15.60:0/1904415827 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec140c3070 con 0x7fec1c06d900 2024-04-25T10:05:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fec1affd640 1 -- 172.21.15.60:0/1904415827 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec140b6770 con 0x7fec1c06d900 2024-04-25T10:05:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fec1affd640 1 -- 172.21.15.60:0/1904415827 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec140b6b40 con 0x7fec1c06d900 2024-04-25T10:05:26.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d6a2c640 1 -- 172.21.15.60:0/243358901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8033e40 msgr2=0x7f87c80d4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d6a2c640 1 --2- 172.21.15.60:0/243358901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8033e40 0x7f87c80d4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d6a2c640 1 -- 172.21.15.60:0/243358901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87c8047c20 msgr2=0x7f87c80d5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d6a2c640 1 --2- 172.21.15.60:0/243358901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87c8047c20 0x7f87c80d5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fedb2d2f640 1 -- 172.21.15.60:0/445873174 wait complete. 2024-04-25T10:05:26.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d6a2c640 1 -- 172.21.15.60:0/243358901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c804bc40 con 0x7f87c80ebc00 2024-04-25T10:05:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f0967fff640 1 -- 172.21.15.60:0/970433770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0960035880 con 0x7f09600e5060 2024-04-25T10:05:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f0967fff640 1 -- 172.21.15.60:0/970433770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0960035e40 con 0x7f09600e5060 2024-04-25T10:05:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f87d6a2c640 1 --2- 172.21.15.60:0/243358901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87c80ebc00 0x7f87c80e9930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f87d00693a0 tx=0x7f87d00742b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f6a9dffb640 1 -- 172.21.15.60:0/1208109695 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8068650 con 0x7f6aa0005ea0 2024-04-25T10:05:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f6a9dffb640 1 -- 172.21.15.60:0/1208109695 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa8068810 con 0x7f6aa0005ea0 2024-04-25T10:05:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f6a9dffb640 1 -- 172.21.15.60:0/1208109695 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8092640 con 0x7f6aa0005ea0 2024-04-25T10:05:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f6aae66b640 1 -- 172.21.15.60:0/1208109695 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa0049180 con 0x7f6aa0005ea0 2024-04-25T10:05:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f6aae66b640 1 -- 172.21.15.60:0/1208109695 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa0049740 con 0x7f6aa0005ea0 2024-04-25T10:05:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fec2110f640 1 -- 172.21.15.60:0/1904415827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec1c06d900 msgr2=0x7fec1c06b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7fec2110f640 1 --2- 172.21.15.60:0/1904415827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec1c06d900 0x7fec1c06b1d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fec1c06aec0 tx=0x7fec1402d7c0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.743+0000 7f0967fff640 1 -- 172.21.15.60:0/970433770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0960035350 con 0x7f09600e5060 2024-04-25T10:05:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f87c6ffd640 1 -- 172.21.15.60:0/243358901 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d008e070 con 0x7f87c80ebc00 2024-04-25T10:05:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f87c6ffd640 1 -- 172.21.15.60:0/243358901 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d00731c0 con 0x7f87c80ebc00 2024-04-25T10:05:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f87c6ffd640 1 -- 172.21.15.60:0/243358901 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0062260 con 0x7f87c80ebc00 2024-04-25T10:05:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f6aae66b640 1 -- 172.21.15.60:0/1208109695 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a88001040 con 0x7f6aa0005ea0 2024-04-25T10:05:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb2d2f640 1 Processor -- start 2024-04-25T10:05:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb2d2f640 1 -- start start 2024-04-25T10:05:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb2d2f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feda4006800 0x7feda404ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb2d2f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feda4053c20 0x7feda40dbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f094f7fe640 1 -- 172.21.15.60:0/970433770 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f095801a070 con 0x7f09600e5060 2024-04-25T10:05:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f094f7fe640 1 --2- 172.21.15.60:0/970433770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0954067060 0x7f0954069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f094f7fe640 1 -- 172.21.15.60:0/970433770 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0958084ca0 con 0x7f09600e5060 2024-04-25T10:05:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f094f7fe640 1 --2- 172.21.15.60:0/970433770 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x7f095406f200 0x7f0954071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f094f7fe640 1 -- 172.21.15.60:0/970433770 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0958008eb0 con 0x7f095406f200 2024-04-25T10:05:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f09667fc640 1 --2- 172.21.15.60:0/970433770 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x7f095406f200 0x7f0954071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb2d2f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feda40dc200 0x7feda40dad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb2d2f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda40efe80 con 0x7feda4053c20 2024-04-25T10:05:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb2d2f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda40efd00 con 0x7feda40dc200 2024-04-25T10:05:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb2d2f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda40f0000 con 0x7feda4006800 2024-04-25T10:05:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb1d2d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feda4006800 0x7feda404ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb1d2d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feda4006800 0x7feda404ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:37208/0 (socket says 172.21.15.60:37208) 2024-04-25T10:05:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb1d2d640 1 -- 172.21.15.60:0/3693494926 learned_addr learned my addr 172.21.15.60:0/3693494926 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb252e640 1 --2- 172.21.15.60:0/3693494926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feda40dc200 0x7feda40dad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f6a9dffb640 1 -- 172.21.15.60:0/1208109695 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6aa80928a0 con 0x7f6aa0005ea0 2024-04-25T10:05:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f2d5affd640 1 -- 172.21.15.60:0/3840804343 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d540c2070 con 0x7f2d6415d360 2024-04-25T10:05:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f2d5affd640 1 --2- 172.21.15.60:0/3840804343 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d48066f60 0x7f2d48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f2d5affd640 1 -- 172.21.15.60:0/3840804343 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d541218f0 con 0x7f2d6415d360 2024-04-25T10:05:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f2d5affd640 1 --2- 172.21.15.60:0/3840804343 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x7f2d4806f160 0x7f2d48071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f2d5affd640 1 -- 172.21.15.60:0/3840804343 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d540a8e50 con 0x7f2d4806f160 2024-04-25T10:05:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f2d5affd640 1 -- 172.21.15.60:0/3840804343 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d54121d80 con 0x7f2d6415d360 2024-04-25T10:05:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fcf93fff640 1 -- 172.21.15.60:0/2373592610 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfa00bc020 con 0x7fcfa40501b0 2024-04-25T10:05:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fcf93fff640 1 --2- 172.21.15.60:0/2373592610 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcf98066f60 0x7fcf98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fcf93fff640 1 -- 172.21.15.60:0/2373592610 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfa011f470 con 0x7fcfa40501b0 2024-04-25T10:05:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fcf93fff640 1 --2- 172.21.15.60:0/2373592610 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x7fcf9806f160 0x7fcf98071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fcf93fff640 1 -- 172.21.15.60:0/2373592610 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcfa00ace50 con 0x7fcf9806f160 2024-04-25T10:05:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fcf93fff640 1 -- 172.21.15.60:0/2373592610 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfa011f860 con 0x7fcfa40501b0 2024-04-25T10:05:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f2d6a4c2640 1 --2- 172.21.15.60:0/3840804343 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d48066f60 0x7f2d48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f2d6acc3640 1 --2- 172.21.15.60:0/3840804343 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x7f2d4806f160 0x7f2d48071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fcfab7fe640 1 --2- 172.21.15.60:0/2373592610 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x7fcf9806f160 0x7fcf98071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fcfaaffd640 1 --2- 172.21.15.60:0/2373592610 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcf98066f60 0x7fcf98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f87d848f640 1 -- 172.21.15.60:0/243358901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c804beb0 con 0x7f87c80ebc00 2024-04-25T10:05:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f87d848f640 1 -- 172.21.15.60:0/243358901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c804c470 con 0x7f87c80ebc00 2024-04-25T10:05:26.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f6a9dffb640 1 --2- 172.21.15.60:0/1208109695 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6a8c066fb0 0x7f6a8c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f6a9dffb640 1 -- 172.21.15.60:0/1208109695 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6aa80ffa20 con 0x7f6aa0005ea0 2024-04-25T10:05:26.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f6a9dffb640 1 --2- 172.21.15.60:0/1208109695 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x7f6a8c06f1d0 0x7f6a8c0715f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f6a9dffb640 1 -- 172.21.15.60:0/1208109695 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6aa8071e50 con 0x7f6a8c06f1d0 2024-04-25T10:05:26.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f6a9dffb640 1 -- 172.21.15.60:0/1208109695 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6aa80fff00 con 0x7f6aa0005ea0 2024-04-25T10:05:26.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fec2110f640 1 -- 172.21.15.60:0/1904415827 shutdown_connections 2024-04-25T10:05:26.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fec2110f640 1 --2- 172.21.15.60:0/1904415827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec1c06d900 0x7fec1c06b1d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fec2110f640 1 --2- 172.21.15.60:0/1904415827 >> v2:172.21.15.100:3300/0 conn(0x7fec1c06f480 0x7fec1c15a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fec2110f640 1 -- 172.21.15.60:0/1904415827 >> 172.21.15.60:0/1904415827 conn(0x7fec1c0cffb0 msgr2=0x7fec1c0d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f2d6a4c2640 1 --2- 172.21.15.60:0/3840804343 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d48066f60 0x7f2d48069420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2d60006fd0 tx=0x7f2d60008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb152c640 1 --2- 172.21.15.60:0/3693494926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feda4053c20 0x7feda40dbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb1d2d640 1 -- 172.21.15.60:0/3693494926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feda40dc200 msgr2=0x7feda40dad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb1d2d640 1 --2- 172.21.15.60:0/3693494926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feda40dc200 0x7feda40dad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fec2110f640 1 -- 172.21.15.60:0/1904415827 shutdown_connections 2024-04-25T10:05:26.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fec2110f640 1 -- 172.21.15.60:0/1904415827 wait complete. 2024-04-25T10:05:26.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fcfab7fe640 1 --2- 172.21.15.60:0/2373592610 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x7fcf9806f160 0x7fcf98071580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb1d2d640 1 -- 172.21.15.60:0/3693494926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feda4053c20 msgr2=0x7feda40dbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb1d2d640 1 --2- 172.21.15.60:0/3693494926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feda4053c20 0x7feda40dbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb1d2d640 1 -- 172.21.15.60:0/3693494926 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda40e3870 con 0x7feda4006800 2024-04-25T10:05:26.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f0966ffd640 1 --2- 172.21.15.60:0/970433770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0954067060 0x7f0954069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f09667fc640 1 --2- 172.21.15.60:0/970433770 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x7f095406f200 0x7f0954071620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f6aace68640 1 --2- 172.21.15.60:0/1208109695 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6a8c066fb0 0x7f6a8c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fedb1d2d640 1 --2- 172.21.15.60:0/3693494926 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feda4006800 0x7feda404ef40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fedac06a6f0 tx=0x7fedac068650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7fcfaaffd640 1 --2- 172.21.15.60:0/2373592610 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcf98066f60 0x7fcf98069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40daf50 tx=0x7fcf9c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f87d848f640 1 -- 172.21.15.60:0/243358901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f87b8001040 con 0x7f87c80ebc00 2024-04-25T10:05:26.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f094f7fe640 1 -- 172.21.15.60:0/970433770 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0958051970 con 0x7f09600e5060 2024-04-25T10:05:26.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f0966ffd640 1 --2- 172.21.15.60:0/970433770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0954067060 0x7f0954069520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0968069de0 tx=0x7f096806aa20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7f6aade6a640 1 --2- 172.21.15.60:0/1208109695 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x7f6a8c06f1d0 0x7f6a8c0715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7feda27fc640 1 -- 172.21.15.60:0/3693494926 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac0731c0 con 0x7feda4006800 2024-04-25T10:05:26.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7feda27fc640 1 -- 172.21.15.60:0/3693494926 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedac08f020 con 0x7feda4006800 2024-04-25T10:05:26.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fcf93fff640 1 -- 172.21.15.60:0/2373592610 <== osd.2 v2:172.21.15.60:6800/1771147646 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcfa00ace50 con 0x7fcf9806f160 2024-04-25T10:05:26.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7f094f7fe640 1 -- 172.21.15.60:0/970433770 <== osd.3 v2:172.21.15.60:6814/1867957156 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0958008eb0 con 0x7f095406f200 2024-04-25T10:05:26.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7f6aade6a640 1 --2- 172.21.15.60:0/1208109695 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x7f6a8c06f1d0 0x7f6a8c0715f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7f6aace68640 1 --2- 172.21.15.60:0/1208109695 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6a8c066fb0 0x7f6a8c069470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0053e10 tx=0x7f6a98006c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.747+0000 7ffbe0ff9640 1 -- 172.21.15.60:0/138666357 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffbd8012b50 con 0x7ffbdc007550 2024-04-25T10:05:26.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7ffbe0ff9640 1 --2- 172.21.15.60:0/138666357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffbc8066f60 0x7ffbc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7ffbe0ff9640 1 -- 172.21.15.60:0/138666357 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffbd8085dc0 con 0x7ffbdc007550 2024-04-25T10:05:26.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7ffbe0ff9640 1 --2- 172.21.15.60:0/138666357 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x7ffbc806f180 0x7ffbc80715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7ffbe0ff9640 1 -- 172.21.15.60:0/138666357 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffbd8004480 con 0x7ffbc806f180 2024-04-25T10:05:26.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7ffbe0ff9640 1 -- 172.21.15.60:0/138666357 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbd80861b0 con 0x7ffbdc007550 2024-04-25T10:05:26.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2110f640 1 Processor -- start 2024-04-25T10:05:26.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7f2d6acc3640 1 --2- 172.21.15.60:0/3840804343 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x7f2d4806f160 0x7f2d48071580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7feda27fc640 1 -- 172.21.15.60:0/3693494926 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac0746d0 con 0x7feda4006800 2024-04-25T10:05:26.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7ffbe37fe640 1 --2- 172.21.15.60:0/138666357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffbc8066f60 0x7ffbc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7ffbe8b7d640 1 --2- 172.21.15.60:0/138666357 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x7ffbc806f180 0x7ffbc80715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fedb2d2f640 1 -- 172.21.15.60:0/3693494926 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda40e3b50 con 0x7feda4006800 2024-04-25T10:05:26.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fedb2d2f640 1 -- 172.21.15.60:0/3693494926 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda40e4110 con 0x7feda4006800 2024-04-25T10:05:26.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2110f640 1 -- start start 2024-04-25T10:05:26.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7ffbe37fe640 1 --2- 172.21.15.60:0/138666357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffbc8066f60 0x7ffbc8069420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0005890 tx=0x7ffbd0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fed8ffff640 1 -- 172.21.15.60:0/3693494926 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fed88001040 con 0x7feda4006800 2024-04-25T10:05:26.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2110f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c06d900 0x7fec1c1580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2110f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1c06f480 0x7fec1c0af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2110f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec1c157840 0x7fec1c0afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2110f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c06b710 con 0x7fec1c157840 2024-04-25T10:05:26.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2110f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0b43e0 con 0x7fec1c06d900 2024-04-25T10:05:26.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2110f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c06b890 con 0x7fec1c06f480 2024-04-25T10:05:26.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2090e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec1c157840 0x7fec1c0afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2090e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec1c157840 0x7fec1c0afd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45390/0 (socket says 172.21.15.60:45390) 2024-04-25T10:05:26.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2090e640 1 -- 172.21.15.60:0/2715317855 learned_addr learned my addr 172.21.15.60:0/2715317855 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:26.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec1bfff640 1 --2- 172.21.15.60:0/2715317855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c06d900 0x7fec1c1580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec1b7fe640 1 --2- 172.21.15.60:0/2715317855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1c06f480 0x7fec1c0af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2090e640 1 -- 172.21.15.60:0/2715317855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1c06f480 msgr2=0x7fec1c0af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2090e640 1 --2- 172.21.15.60:0/2715317855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1c06f480 0x7fec1c0af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2090e640 1 -- 172.21.15.60:0/2715317855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c06d900 msgr2=0x7fec1c1580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2090e640 1 --2- 172.21.15.60:0/2715317855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1c06d900 0x7fec1c1580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2090e640 1 -- 172.21.15.60:0/2715317855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c159b10 con 0x7fec1c157840 2024-04-25T10:05:26.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec1b7fe640 1 --2- 172.21.15.60:0/2715317855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec1c06f480 0x7fec1c0af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:26.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7f87c6ffd640 1 -- 172.21.15.60:0/243358901 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87d0062400 con 0x7f87c80ebc00 2024-04-25T10:05:26.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7f87c6ffd640 1 --2- 172.21.15.60:0/243358901 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87b4067030 0x7f87b40694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7f87c6ffd640 1 -- 172.21.15.60:0/243358901 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87d00fef40 con 0x7f87c80ebc00 2024-04-25T10:05:26.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7f87c6ffd640 1 --2- 172.21.15.60:0/243358901 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x7f87b406f250 0x7f87b4071670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7f87c6ffd640 1 -- 172.21.15.60:0/243358901 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87b4071d20 con 0x7f87b406f250 2024-04-25T10:05:26.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7f87c6ffd640 1 -- 172.21.15.60:0/243358901 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87d00ff330 con 0x7f87c80ebc00 2024-04-25T10:05:26.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2090e640 1 --2- 172.21.15.60:0/2715317855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec1c157840 0x7fec1c0afd10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fec1402be70 tx=0x7fec1402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7f87d622b640 1 --2- 172.21.15.60:0/243358901 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87b4067030 0x7f87b40694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec18ff9640 1 -- 172.21.15.60:0/2715317855 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec140c3040 con 0x7fec1c157840 2024-04-25T10:05:26.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7f87d5a2a640 1 --2- 172.21.15.60:0/243358901 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x7f87b406f250 0x7f87b4071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec18ff9640 1 -- 172.21.15.60:0/2715317855 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec140ce070 con 0x7fec1c157840 2024-04-25T10:05:26.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2110f640 1 -- 172.21.15.60:0/2715317855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c0b0f70 con 0x7fec1c157840 2024-04-25T10:05:26.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec18ff9640 1 -- 172.21.15.60:0/2715317855 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec140be660 con 0x7fec1c157840 2024-04-25T10:05:26.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7fec2110f640 1 -- 172.21.15.60:0/2715317855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1c0b14c0 con 0x7fec1c157840 2024-04-25T10:05:26.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.755+0000 7f87d5a2a640 1 --2- 172.21.15.60:0/243358901 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x7f87b406f250 0x7f87b4071670 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.755+0000 7f87d622b640 1 --2- 172.21.15.60:0/243358901 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87b4067030 0x7f87b40694f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0097c0 tx=0x7f87cc0aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.755+0000 7f87c6ffd640 1 -- 172.21.15.60:0/243358901 <== osd.5 v2:172.21.15.100:6800/3542120824 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f87b4071d20 con 0x7f87b406f250 2024-04-25T10:05:26.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.755+0000 7f6a9dffb640 1 -- 172.21.15.60:0/1208109695 <== osd.0 v2:172.21.15.60:6801/4056167511 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6aa8071e50 con 0x7f6a8c06f1d0 2024-04-25T10:05:26.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.755+0000 7fec2110f640 1 -- 172.21.15.60:0/2715317855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec1c074770 con 0x7fec1c157840 2024-04-25T10:05:26.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.755+0000 7fec18ff9640 1 -- 172.21.15.60:0/2715317855 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec140be800 con 0x7fec1c157840 2024-04-25T10:05:26.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.755+0000 7fec18ff9640 1 --2- 172.21.15.60:0/2715317855 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec08066f60 0x7fec08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.755+0000 7fec18ff9640 1 -- 172.21.15.60:0/2715317855 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec14130690 con 0x7fec1c157840 2024-04-25T10:05:26.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.755+0000 7fec18ff9640 1 --2- 172.21.15.60:0/2715317855 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x7fec0806f100 0x7fec08071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.755+0000 7fec18ff9640 1 -- 172.21.15.60:0/2715317855 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec140c8e50 con 0x7fec0806f100 2024-04-25T10:05:26.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.755+0000 7fec18ff9640 1 -- 172.21.15.60:0/2715317855 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec14137050 con 0x7fec1c157840 2024-04-25T10:05:26.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.751+0000 7f2d5affd640 1 -- 172.21.15.60:0/3840804343 <== osd.6 v2:172.21.15.100:6824/4086390550 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2d540a8e50 con 0x7f2d4806f160 2024-04-25T10:05:26.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.759+0000 7fec1b7fe640 1 --2- 172.21.15.60:0/2715317855 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x7fec0806f100 0x7fec08071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.759+0000 7fec1bfff640 1 --2- 172.21.15.60:0/2715317855 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec08066f60 0x7fec08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.759+0000 7ffbe8b7d640 1 --2- 172.21.15.60:0/138666357 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x7ffbc806f180 0x7ffbc80715a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.759+0000 7fec1b7fe640 1 --2- 172.21.15.60:0/2715317855 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x7fec0806f100 0x7fec08071520 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.759+0000 7fec1bfff640 1 --2- 172.21.15.60:0/2715317855 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec08066f60 0x7fec08069420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fec10000f70 tx=0x7fec10007e30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.759+0000 7fec18ff9640 1 -- 172.21.15.60:0/2715317855 <== osd.1 v2:172.21.15.60:6824/3313552458 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fec140c8e50 con 0x7fec0806f100 2024-04-25T10:05:26.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.759+0000 7ffbe0ff9640 1 -- 172.21.15.60:0/138666357 <== osd.4 v2:172.21.15.100:6808/3965541654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ffbd8004480 con 0x7ffbc806f180 2024-04-25T10:05:26.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.759+0000 7feda27fc640 1 -- 172.21.15.60:0/3693494926 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fedac074870 con 0x7feda4006800 2024-04-25T10:05:26.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.759+0000 7feda27fc640 1 --2- 172.21.15.60:0/3693494926 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed90066f60 0x7fed90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.759+0000 7feda27fc640 1 -- 172.21.15.60:0/3693494926 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedac0ffd20 con 0x7feda4006800 2024-04-25T10:05:26.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.759+0000 7feda27fc640 1 --2- 172.21.15.60:0/3693494926 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x7fed9006f180 0x7fed900715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.759+0000 7feda27fc640 1 -- 172.21.15.60:0/3693494926 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fedac09be50 con 0x7fed9006f180 2024-04-25T10:05:26.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.759+0000 7feda27fc640 1 -- 172.21.15.60:0/3693494926 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedac100110 con 0x7feda4006800 2024-04-25T10:05:26.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.759+0000 7fedb252e640 1 --2- 172.21.15.60:0/3693494926 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x7fed9006f180 0x7fed900715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.759+0000 7fedb252e640 1 --2- 172.21.15.60:0/3693494926 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x7fed9006f180 0x7fed900715a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.759+0000 7feda27fc640 1 -- 172.21.15.60:0/3693494926 <== osd.7 v2:172.21.15.100:6816/2771485684 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fedac09be50 con 0x7fed9006f180 2024-04-25T10:05:26.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.767+0000 7fedb152c640 1 --2- 172.21.15.60:0/3693494926 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed90066f60 0x7fed90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.767+0000 7fedb152c640 1 --2- 172.21.15.60:0/3693494926 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed90066f60 0x7fed90069420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feda80ad000 tx=0x7feda80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.775+0000 7fcfb1802640 1 -- 172.21.15.60:0/2373592610 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcfa4052cf0 con 0x7fcf9806f160 2024-04-25T10:05:26.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.775+0000 7f2d6c726640 1 -- 172.21.15.60:0/3840804343 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2d4c002de0 con 0x7f2d4806f160 2024-04-25T10:05:26.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.775+0000 7fcf93fff640 1 -- 172.21.15.60:0/2373592610 <== osd.2 v2:172.21.15.60:6800/1771147646 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcfa4052cf0 con 0x7fcf9806f160 2024-04-25T10:05:26.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.775+0000 7fec2110f640 1 -- 172.21.15.60:0/2715317855 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fec1c0cd110 con 0x7fec0806f100 2024-04-25T10:05:26.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.775+0000 7f2d5affd640 1 -- 172.21.15.60:0/3840804343 <== osd.6 v2:172.21.15.100:6824/4086390550 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2d4c002de0 con 0x7f2d4806f160 2024-04-25T10:05:26.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7fec18ff9640 1 -- 172.21.15.60:0/2715317855 <== osd.1 v2:172.21.15.60:6824/3313552458 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fec0c011320 con 0x7fec0806f100 2024-04-25T10:05:26.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7f2d6c726640 1 -- 172.21.15.60:0/3840804343 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x7f2d4806f160 msgr2=0x7f2d48071580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7f2d6c726640 1 --2- 172.21.15.60:0/3840804343 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x7f2d4806f160 0x7f2d48071580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7f2d6c726640 1 -- 172.21.15.60:0/3840804343 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d48066f60 msgr2=0x7f2d48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7f2d6c726640 1 --2- 172.21.15.60:0/3840804343 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d48066f60 0x7f2d48069420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2d60006fd0 tx=0x7f2d60008040 comp rx=0 tx=0).stop 2024-04-25T10:05:26.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7f2d6c726640 1 -- 172.21.15.60:0/3840804343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d6415d360 msgr2=0x7f2d6415b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7f2d6c726640 1 --2- 172.21.15.60:0/3840804343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d6415d360 0x7f2d6415b590 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2d54005040 tx=0x7f2d50005840 comp rx=0 tx=0).stop 2024-04-25T10:05:26.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7f2d6acc3640 1 -- 172.21.15.60:0/3840804343 reap_dead start 2024-04-25T10:05:26.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7f2d6c726640 1 -- 172.21.15.60:0/3840804343 shutdown_connections 2024-04-25T10:05:26.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7f2d6c726640 1 -- 172.21.15.60:0/3840804343 >> 172.21.15.60:0/3840804343 conn(0x7f2d6406ac10 msgr2=0x7f2d6406efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7fec2110f640 1 -- 172.21.15.60:0/2715317855 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x7fec0806f100 msgr2=0x7fec08071520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7fec2110f640 1 --2- 172.21.15.60:0/2715317855 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x7fec0806f100 0x7fec08071520 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7f2d6c726640 1 -- 172.21.15.60:0/3840804343 shutdown_connections 2024-04-25T10:05:26.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7f2d6c726640 1 -- 172.21.15.60:0/3840804343 wait complete. 2024-04-25T10:05:26.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7fec2110f640 1 -- 172.21.15.60:0/2715317855 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec08066f60 msgr2=0x7fec08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7fec2110f640 1 --2- 172.21.15.60:0/2715317855 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec08066f60 0x7fec08069420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fec10000f70 tx=0x7fec10007e30 comp rx=0 tx=0).stop 2024-04-25T10:05:26.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7fec2110f640 1 -- 172.21.15.60:0/2715317855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec1c157840 msgr2=0x7fec1c0afd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7fec2110f640 1 --2- 172.21.15.60:0/2715317855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec1c157840 0x7fec1c0afd10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fec1402be70 tx=0x7fec1402a560 comp rx=0 tx=0).stop 2024-04-25T10:05:26.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7fec2090e640 1 -- 172.21.15.60:0/2715317855 reap_dead start 2024-04-25T10:05:26.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7fec2110f640 1 -- 172.21.15.60:0/2715317855 shutdown_connections 2024-04-25T10:05:26.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7fec2110f640 1 -- 172.21.15.60:0/2715317855 >> 172.21.15.60:0/2715317855 conn(0x7fec1c0cffb0 msgr2=0x7fec1c15a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7fec2110f640 1 -- 172.21.15.60:0/2715317855 shutdown_connections 2024-04-25T10:05:26.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.779+0000 7fec2110f640 1 -- 172.21.15.60:0/2715317855 wait complete. 2024-04-25T10:05:26.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f0967fff640 1 -- 172.21.15.60:0/970433770 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f09600057f0 con 0x7f095406f200 2024-04-25T10:05:26.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f6aae66b640 1 -- 172.21.15.60:0/1208109695 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6a88002d60 con 0x7f6a8c06f1d0 2024-04-25T10:05:26.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f094f7fe640 1 -- 172.21.15.60:0/970433770 <== osd.3 v2:172.21.15.60:6814/1867957156 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f09600057f0 con 0x7f095406f200 2024-04-25T10:05:26.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f6a9dffb640 1 -- 172.21.15.60:0/1208109695 <== osd.0 v2:172.21.15.60:6801/4056167511 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6a88002d60 con 0x7f6a8c06f1d0 2024-04-25T10:05:26.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f0967fff640 1 -- 172.21.15.60:0/970433770 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x7f095406f200 msgr2=0x7f0954071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f0967fff640 1 --2- 172.21.15.60:0/970433770 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x7f095406f200 0x7f0954071620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f6aae66b640 1 -- 172.21.15.60:0/1208109695 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x7f6a8c06f1d0 msgr2=0x7f6a8c0715f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f6aae66b640 1 --2- 172.21.15.60:0/1208109695 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x7f6a8c06f1d0 0x7f6a8c0715f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f6aae66b640 1 -- 172.21.15.60:0/1208109695 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6a8c066fb0 msgr2=0x7f6a8c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f6aae66b640 1 --2- 172.21.15.60:0/1208109695 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6a8c066fb0 0x7f6a8c069470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0053e10 tx=0x7f6a98006c00 comp rx=0 tx=0).stop 2024-04-25T10:05:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f6aae66b640 1 -- 172.21.15.60:0/1208109695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa0005ea0 msgr2=0x7f6aa0049c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f6aae66b640 1 --2- 172.21.15.60:0/1208109695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa0005ea0 0x7f6aa0049c60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8069a80 tx=0x7f6aa80742b0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f6aade6a640 1 -- 172.21.15.60:0/1208109695 reap_dead start 2024-04-25T10:05:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f6aae66b640 1 -- 172.21.15.60:0/1208109695 shutdown_connections 2024-04-25T10:05:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f6aae66b640 1 -- 172.21.15.60:0/1208109695 >> 172.21.15.60:0/1208109695 conn(0x7f6aa0009fc0 msgr2=0x7f6aa00e5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f0967fff640 1 -- 172.21.15.60:0/970433770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0954067060 msgr2=0x7f0954069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f0967fff640 1 --2- 172.21.15.60:0/970433770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0954067060 0x7f0954069520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0968069de0 tx=0x7f096806aa20 comp rx=0 tx=0).stop 2024-04-25T10:05:26.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f0967fff640 1 -- 172.21.15.60:0/970433770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09600e5060 msgr2=0x7f096004d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f0967fff640 1 --2- 172.21.15.60:0/970433770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09600e5060 0x7f096004d860 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f095800e7f0 tx=0x7f095800ecb0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f09677fe640 1 -- 172.21.15.60:0/970433770 reap_dead start 2024-04-25T10:05:26.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f6aae66b640 1 -- 172.21.15.60:0/1208109695 shutdown_connections 2024-04-25T10:05:26.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.783+0000 7f6aae66b640 1 -- 172.21.15.60:0/1208109695 wait complete. 2024-04-25T10:05:26.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7f0967fff640 1 -- 172.21.15.60:0/970433770 shutdown_connections 2024-04-25T10:05:26.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7ffbe937e640 1 -- 172.21.15.60:0/138666357 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffbdc181d60 con 0x7ffbc806f180 2024-04-25T10:05:26.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7f0967fff640 1 -- 172.21.15.60:0/970433770 >> 172.21.15.60:0/970433770 conn(0x7f096000a410 msgr2=0x7f09600396b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7f0967fff640 1 -- 172.21.15.60:0/970433770 shutdown_connections 2024-04-25T10:05:26.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7f0967fff640 1 -- 172.21.15.60:0/970433770 wait complete. 2024-04-25T10:05:26.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7fcfb1802640 1 -- 172.21.15.60:0/2373592610 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x7fcf9806f160 msgr2=0x7fcf98071580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7fcfb1802640 1 --2- 172.21.15.60:0/2373592610 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x7fcf9806f160 0x7fcf98071580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7ffbe0ff9640 1 -- 172.21.15.60:0/138666357 <== osd.4 v2:172.21.15.100:6808/3965541654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffbe40669a0 con 0x7ffbc806f180 2024-04-25T10:05:26.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7fcfb1802640 1 -- 172.21.15.60:0/2373592610 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcf98066f60 msgr2=0x7fcf98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7fcfb1802640 1 --2- 172.21.15.60:0/2373592610 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcf98066f60 0x7fcf98069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40daf50 tx=0x7fcf9c009040 comp rx=0 tx=0).stop 2024-04-25T10:05:26.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7fcfb1802640 1 -- 172.21.15.60:0/2373592610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfa40501b0 msgr2=0x7fcfa40cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7fcfb1802640 1 --2- 172.21.15.60:0/2373592610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfa40501b0 0x7fcfa40cc5b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00ae040 tx=0x7fcfa0019180 comp rx=0 tx=0).stop 2024-04-25T10:05:26.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7fcfab7fe640 1 -- 172.21.15.60:0/2373592610 reap_dead start 2024-04-25T10:05:26.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7fcfb1802640 1 -- 172.21.15.60:0/2373592610 shutdown_connections 2024-04-25T10:05:26.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7fcfb1802640 1 -- 172.21.15.60:0/2373592610 >> 172.21.15.60:0/2373592610 conn(0x7fcfa4009fc0 msgr2=0x7fcfa40365a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7fcfb1802640 1 -- 172.21.15.60:0/2373592610 shutdown_connections 2024-04-25T10:05:26.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7fcfb1802640 1 -- 172.21.15.60:0/2373592610 wait complete. 2024-04-25T10:05:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7ffbe937e640 1 -- 172.21.15.60:0/138666357 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x7ffbc806f180 msgr2=0x7ffbc80715a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7ffbe937e640 1 --2- 172.21.15.60:0/138666357 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x7ffbc806f180 0x7ffbc80715a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7ffbe937e640 1 -- 172.21.15.60:0/138666357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffbc8066f60 msgr2=0x7ffbc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7ffbe937e640 1 --2- 172.21.15.60:0/138666357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffbc8066f60 0x7ffbc8069420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0005890 tx=0x7ffbd0009040 comp rx=0 tx=0).stop 2024-04-25T10:05:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7ffbe937e640 1 -- 172.21.15.60:0/138666357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbdc007550 msgr2=0x7ffbdc180740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7ffbe937e640 1 --2- 172.21.15.60:0/138666357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbdc007550 0x7ffbdc180740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800d5f0 tx=0x7ffbd800dab0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7ffbe8b7d640 1 -- 172.21.15.60:0/138666357 reap_dead start 2024-04-25T10:05:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7ffbe937e640 1 -- 172.21.15.60:0/138666357 shutdown_connections 2024-04-25T10:05:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7ffbe937e640 1 -- 172.21.15.60:0/138666357 >> 172.21.15.60:0/138666357 conn(0x7ffbdc009fc0 msgr2=0x7ffbdc036e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7ffbe937e640 1 -- 172.21.15.60:0/138666357 shutdown_connections 2024-04-25T10:05:26.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.791+0000 7ffbe937e640 1 -- 172.21.15.60:0/138666357 wait complete. 2024-04-25T10:05:26.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.795+0000 7fed8ffff640 1 -- 172.21.15.60:0/3693494926 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fed88002de0 con 0x7fed9006f180 2024-04-25T10:05:26.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.795+0000 7f87d848f640 1 -- 172.21.15.60:0/243358901 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f87b8002de0 con 0x7f87b406f250 2024-04-25T10:05:26.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.795+0000 7feda27fc640 1 -- 172.21.15.60:0/3693494926 <== osd.7 v2:172.21.15.100:6816/2771485684 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fed88002de0 con 0x7fed9006f180 2024-04-25T10:05:26.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.795+0000 7f87c6ffd640 1 -- 172.21.15.60:0/243358901 <== osd.5 v2:172.21.15.100:6800/3542120824 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f87b8002de0 con 0x7f87b406f250 2024-04-25T10:05:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.795+0000 7fed8ffff640 1 -- 172.21.15.60:0/3693494926 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x7fed9006f180 msgr2=0x7fed900715a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.795+0000 7fed8ffff640 1 --2- 172.21.15.60:0/3693494926 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x7fed9006f180 0x7fed900715a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.795+0000 7fed8ffff640 1 -- 172.21.15.60:0/3693494926 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed90066f60 msgr2=0x7fed90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.795+0000 7fed8ffff640 1 --2- 172.21.15.60:0/3693494926 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed90066f60 0x7fed90069420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feda80ad000 tx=0x7feda80a6000 comp rx=0 tx=0).stop 2024-04-25T10:05:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.795+0000 7fed8ffff640 1 -- 172.21.15.60:0/3693494926 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feda4006800 msgr2=0x7feda404ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.795+0000 7fed8ffff640 1 --2- 172.21.15.60:0/3693494926 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feda4006800 0x7feda404ef40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fedac06a6f0 tx=0x7fedac068650 comp rx=0 tx=0).stop 2024-04-25T10:05:26.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.795+0000 7fedb252e640 1 -- 172.21.15.60:0/3693494926 reap_dead start 2024-04-25T10:05:26.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.795+0000 7f87d848f640 1 -- 172.21.15.60:0/243358901 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x7f87b406f250 msgr2=0x7f87b4071670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.795+0000 7f87d848f640 1 --2- 172.21.15.60:0/243358901 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x7f87b406f250 0x7f87b4071670 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.799+0000 7f87d848f640 1 -- 172.21.15.60:0/243358901 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87b4067030 msgr2=0x7f87b40694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.799+0000 7f87d848f640 1 --2- 172.21.15.60:0/243358901 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87b4067030 0x7f87b40694f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0097c0 tx=0x7f87cc0aa070 comp rx=0 tx=0).stop 2024-04-25T10:05:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.799+0000 7f87d848f640 1 -- 172.21.15.60:0/243358901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87c80ebc00 msgr2=0x7f87c80e9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.799+0000 7f87d848f640 1 --2- 172.21.15.60:0/243358901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87c80ebc00 0x7f87c80e9930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f87d00693a0 tx=0x7f87d00742b0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.799+0000 7f87d6a2c640 1 -- 172.21.15.60:0/243358901 reap_dead start 2024-04-25T10:05:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.803+0000 7f87d848f640 1 -- 172.21.15.60:0/243358901 shutdown_connections 2024-04-25T10:05:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.803+0000 7f87d848f640 1 -- 172.21.15.60:0/243358901 >> 172.21.15.60:0/243358901 conn(0x7f87c8009fc0 msgr2=0x7f87c8046ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.803+0000 7f87d848f640 1 -- 172.21.15.60:0/243358901 shutdown_connections 2024-04-25T10:05:26.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.803+0000 7f87d848f640 1 -- 172.21.15.60:0/243358901 wait complete. 2024-04-25T10:05:26.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.807+0000 7fed8ffff640 1 -- 172.21.15.60:0/3693494926 shutdown_connections 2024-04-25T10:05:26.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.807+0000 7fed8ffff640 1 -- 172.21.15.60:0/3693494926 >> 172.21.15.60:0/3693494926 conn(0x7feda40559f0 msgr2=0x7feda403d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.807+0000 7fed8ffff640 1 -- 172.21.15.60:0/3693494926 shutdown_connections 2024-04-25T10:05:26.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:26.807+0000 7fed8ffff640 1 -- 172.21.15.60:0/3693494926 wait complete. 2024-04-25T10:05:26.846 INFO:teuthology.orchestra.run.smithi060.stdout:60129542149 2024-04-25T10:05:26.846 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:05:26.846 INFO:teuthology.orchestra.run.smithi060.stdout:60129542151 2024-04-25T10:05:26.847 INFO:teuthology.orchestra.run.smithi060.stdout:60129542150 2024-04-25T10:05:26.847 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:05:26.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:05:26.849 INFO:teuthology.orchestra.run.smithi060.stdout:60129542150 2024-04-25T10:05:26.849 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:05:26.850 INFO:teuthology.orchestra.run.smithi060.stdout:60129542150 2024-04-25T10:05:26.850 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:05:26.851 INFO:teuthology.orchestra.run.smithi060.stdout:64424509446 2024-04-25T10:05:26.851 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:05:26.852 INFO:teuthology.orchestra.run.smithi060.stdout:60129542150 2024-04-25T10:05:26.852 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:05:26.853 INFO:teuthology.orchestra.run.smithi060.stdout:64424509447 2024-04-25T10:05:26.853 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:05:27.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.059+0000 7f93421fc640 1 Processor -- start 2024-04-25T10:05:27.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.059+0000 7f93421fc640 1 -- start start 2024-04-25T10:05:27.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f6792c4b640 1 Processor -- start 2024-04-25T10:05:27.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f6792c4b640 1 -- start start 2024-04-25T10:05:27.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f93421fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f933c0cbeb0 0x7f933c0cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f93421fc640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933c0b26c0 con 0x7f933c0b3e70 2024-04-25T10:05:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f93421fc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933c0b29a0 con 0x7f933c0af150 2024-04-25T10:05:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f93421fc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933c1433d0 con 0x7f933c0cbeb0 2024-04-25T10:05:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f2cc0e6c640 1 Processor -- start 2024-04-25T10:05:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f2cc0e6c640 1 -- start start 2024-04-25T10:05:27.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f933bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f933c0b3e70 0x7f933c0cd7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55810/0 (socket says 172.21.15.60:55810) 2024-04-25T10:05:27.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f933bfff640 1 -- 172.21.15.60:0/1297538890 learned_addr learned my addr 172.21.15.60:0/1297538890 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:27.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f4d1ba95640 1 Processor -- start 2024-04-25T10:05:27.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7fc49759e640 1 Processor -- start 2024-04-25T10:05:27.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f933b7fe640 1 --2- 172.21.15.60:0/1297538890 >> v2:172.21.15.100:3300/0 conn(0x7f933c0cbeb0 0x7f933c0cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7fc49759e640 1 -- start start 2024-04-25T10:05:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f076ef2d640 1 Processor -- start 2024-04-25T10:05:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7fc49759e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc498161b30 0x7fc4980b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7fc49759e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc49806e600 0x7fc49806ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7fc49759e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49806d2b0 0x7fc49806d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7fc49759e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4980b3860 con 0x7fc49806e600 2024-04-25T10:05:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7fc49759e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49806e020 con 0x7fc498161b30 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7fc49759e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980b3ab0 con 0x7fc49806d2b0 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7fc495d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc49806e600 0x7fc49806ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7fc495d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc49806e600 0x7fc49806ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45402/0 (socket says 172.21.15.60:45402) 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7fc495d9b640 1 -- 172.21.15.60:0/2868671635 learned_addr learned my addr 172.21.15.60:0/2868671635 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f4d1ba95640 1 -- start start 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f4d1ba95640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d1406d920 0x7f4d140cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f4d1ba95640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d140af150 0x7f4d140cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f4d1ba95640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4d140af5b0 0x7f4d140b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f4d1ba95640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d141433d0 con 0x7f4d1406d920 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f4d1ba95640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d14142960 con 0x7f4d140af5b0 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f4d1ba95640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d14160540 con 0x7f4d140af150 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f4d19831640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d1406d920 0x7f4d140cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f4d19831640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d1406d920 0x7f4d140cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45412/0 (socket says 172.21.15.60:45412) 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f4d19831640 1 -- 172.21.15.60:0/3814101717 learned_addr learned my addr 172.21.15.60:0/3814101717 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f4d19030640 1 --2- 172.21.15.60:0/3814101717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d140af150 0x7f4d140cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f2cc0e6c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2cbc06db40 0x7f2cbc0aec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f2cc0e6c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2cbc071100 0x7f2cbc0715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f2cc0e6c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc06e0c0 con 0x7f2cbc1652c0 2024-04-25T10:05:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f2cc0e6c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc0b3860 con 0x7f2cbc06db40 2024-04-25T10:05:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f2cc0e6c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc071af0 con 0x7f2cbc071100 2024-04-25T10:05:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f2cbaffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2cbc06db40 0x7f2cbc0aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f2cbaffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2cbc06db40 0x7f2cbc0aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45392/0 (socket says 172.21.15.60:45392) 2024-04-25T10:05:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f2cbaffd640 1 -- 172.21.15.60:0/36421756 learned_addr learned my addr 172.21.15.60:0/36421756 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f2cbaffd640 1 -- 172.21.15.60:0/36421756 >> v2:172.21.15.100:3300/0 conn(0x7f2cbc071100 msgr2=0x7f2cbc0715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f2cbaffd640 1 --2- 172.21.15.60:0/36421756 >> v2:172.21.15.100:3300/0 conn(0x7f2cbc071100 0x7f2cbc0715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f2cbaffd640 1 -- 172.21.15.60:0/36421756 >> v1:172.21.15.60:6789/0 conn(0x7f2cbc1652c0 legacy=0x7f2cbc0cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f2cbaffd640 1 -- 172.21.15.60:0/36421756 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc06e8a0 con 0x7f2cbc06db40 2024-04-25T10:05:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7fba48ffe640 1 Processor -- start 2024-04-25T10:05:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7fba48ffe640 1 -- start start 2024-04-25T10:05:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f076ef2d640 1 -- start start 2024-04-25T10:05:27.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f076ef2d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f07680b1c80 0x7f07680af920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f076ef2d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07680b0950 con 0x7f0768165000 2024-04-25T10:05:27.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f076ef2d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07680b4220 con 0x7f07680cd390 2024-04-25T10:05:27.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f076ef2d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07680b0670 con 0x7f07680b1c80 2024-04-25T10:05:27.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f6792c4b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6784056790 0x7f6784054280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f6792c4b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678403cc80 con 0x7f67840e4800 2024-04-25T10:05:27.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f6792c4b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678403c9a0 con 0x7f67840521c0 2024-04-25T10:05:27.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f6792c4b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67840559f0 con 0x7f6784056790 2024-04-25T10:05:27.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f67911e8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6784056790 0x7f6784054280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f67911e8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6784056790 0x7f6784054280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:39062/0 (socket says 172.21.15.60:39062) 2024-04-25T10:05:27.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f67911e8640 1 -- 172.21.15.60:0/2343568372 learned_addr learned my addr 172.21.15.60:0/2343568372 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:27.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f933a7fc640 1 -- 172.21.15.60:0/1297538890 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737809615 0 0) 0x7f933c0b29a0 con 0x7f933c0af150 2024-04-25T10:05:27.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f933a7fc640 1 -- 172.21.15.60:0/1297538890 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9324003530 con 0x7f933c0af150 2024-04-25T10:05:27.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f933b7fe640 1 -- 172.21.15.60:0/1297538890 >> v1:172.21.15.100:6789/0 conn(0x7f933c0af150 legacy=0x7f933c0b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f933b7fe640 1 -- 172.21.15.60:0/1297538890 >> v1:172.21.15.60:6789/0 conn(0x7f933c0b3e70 legacy=0x7f933c0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7f933b7fe640 1 -- 172.21.15.60:0/1297538890 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c0b2c80 con 0x7f933c0cbeb0 2024-04-25T10:05:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.063+0000 7fc496d9d640 1 --2- 172.21.15.60:0/2868671635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49806d2b0 0x7fc49806d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f076d4ca640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f07680cd390 0x7f07680afe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:48744/0 (socket says 172.21.15.60:48744) 2024-04-25T10:05:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f076d4ca640 1 -- 172.21.15.60:0/860394519 learned_addr learned my addr 172.21.15.60:0/860394519 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f0767fff640 1 --2- 172.21.15.60:0/860394519 >> v2:172.21.15.60:3300/0 conn(0x7f07680b1c80 0x7f07680af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fc49659c640 1 --2- 172.21.15.60:0/2868671635 >> v2:172.21.15.100:3300/0 conn(0x7fc498161b30 0x7fc4980b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f678b7fe640 1 -- 172.21.15.60:0/2343568372 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384992716 0 0) 0x7f678403c9a0 con 0x7f67840521c0 2024-04-25T10:05:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fc496d9d640 1 -- 172.21.15.60:0/2868671635 >> v2:172.21.15.100:3300/0 conn(0x7fc498161b30 msgr2=0x7fc4980b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fc496d9d640 1 --2- 172.21.15.60:0/2868671635 >> v2:172.21.15.100:3300/0 conn(0x7fc498161b30 0x7fc4980b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fc496d9d640 1 -- 172.21.15.60:0/2868671635 >> v2:172.21.15.60:3300/0 conn(0x7fc49806e600 msgr2=0x7fc49806ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fc496d9d640 1 --2- 172.21.15.60:0/2868671635 >> v2:172.21.15.60:3300/0 conn(0x7fc49806e600 0x7fc49806ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fc496d9d640 1 -- 172.21.15.60:0/2868671635 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4980b3d90 con 0x7fc49806d2b0 2024-04-25T10:05:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fc495d9b640 1 --2- 172.21.15.60:0/2868671635 >> v2:172.21.15.60:3300/0 conn(0x7fc49806e600 0x7fc49806ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f67911e8640 1 -- 172.21.15.60:0/2343568372 >> v1:172.21.15.100:6789/0 conn(0x7f67840521c0 legacy=0x7f6784039df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f4d19030640 1 -- 172.21.15.60:0/3814101717 >> v2:172.21.15.100:3300/0 conn(0x7f4d140af5b0 msgr2=0x7f4d140b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f4d19030640 1 --2- 172.21.15.60:0/3814101717 >> v2:172.21.15.100:3300/0 conn(0x7f4d140af5b0 0x7f4d140b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f4d19030640 1 -- 172.21.15.60:0/3814101717 >> v2:172.21.15.60:3300/0 conn(0x7f4d1406d920 msgr2=0x7f4d140cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f4d19030640 1 --2- 172.21.15.60:0/3814101717 >> v2:172.21.15.60:3300/0 conn(0x7f4d1406d920 0x7f4d140cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f4d19030640 1 -- 172.21.15.60:0/3814101717 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d140b29a0 con 0x7f4d140af150 2024-04-25T10:05:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fba48ffe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba44153cd0 0x7fba4415a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fba48ffe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fba440afa10 0x7fba440afe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fba48ffe640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba4415ad60 con 0x7fba440cb700 2024-04-25T10:05:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fba48ffe640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba440cebb0 con 0x7fba440afa10 2024-04-25T10:05:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fba48ffe640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba440cdb10 con 0x7fba44153cd0 2024-04-25T10:05:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f0767fff640 1 -- 172.21.15.60:0/860394519 >> v1:172.21.15.100:6789/0 conn(0x7f07680cd390 legacy=0x7f07680afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f0767fff640 1 -- 172.21.15.60:0/860394519 >> v1:172.21.15.60:6789/0 conn(0x7f0768165000 legacy=0x7f07680b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f0767fff640 1 -- 172.21.15.60:0/860394519 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07680b2730 con 0x7f07680b1c80 2024-04-25T10:05:27.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f933b7fe640 1 --2- 172.21.15.60:0/1297538890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933c0cbeb0 0x7f933c0cefc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f93300097c0 tx=0x7f933001c4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4dc100a63d5cfd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f67911e8640 1 -- 172.21.15.60:0/2343568372 >> v1:172.21.15.60:6789/0 conn(0x7f67840e4800 legacy=0x7f6784051ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f67911e8640 1 -- 172.21.15.60:0/2343568372 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6784055cd0 con 0x7f6784056790 2024-04-25T10:05:27.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fba41d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fba440afa10 0x7fba440afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fba41d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fba440afa10 0x7fba440afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45418/0 (socket says 172.21.15.60:45418) 2024-04-25T10:05:27.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fba41d9b640 1 -- 172.21.15.60:0/2570704514 learned_addr learned my addr 172.21.15.60:0/2570704514 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:27.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fba41d9b640 1 -- 172.21.15.60:0/2570704514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba44153cd0 msgr2=0x7fba4415a820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:05:27.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fba41d9b640 1 --2- 172.21.15.60:0/2570704514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba44153cd0 0x7fba4415a820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fba41d9b640 1 -- 172.21.15.60:0/2570704514 >> v1:172.21.15.60:6789/0 conn(0x7fba440cb700 legacy=0x7fba440cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fba41d9b640 1 -- 172.21.15.60:0/2570704514 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba440cddf0 con 0x7fba440afa10 2024-04-25T10:05:27.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f2cbaffd640 1 --2- 172.21.15.60:0/36421756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cbc06db40 0x7f2cbc0aec50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc06cf90 tx=0x7f2cb001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499a850658bdd7d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f67911e8640 1 --2- 172.21.15.60:0/2343568372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6784056790 0x7f6784054280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f678c069ba0 tx=0x7f678c0975c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8010242f4765e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f2cba7fc640 1 -- 172.21.15.60:0/36421756 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00b0070 con 0x7f2cbc06db40 2024-04-25T10:05:27.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f678b7fe640 1 -- 172.21.15.60:0/2343568372 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c12b070 con 0x7f6784056790 2024-04-25T10:05:27.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f678b7fe640 1 -- 172.21.15.60:0/2343568372 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c126640 con 0x7f6784056790 2024-04-25T10:05:27.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f678b7fe640 1 -- 172.21.15.60:0/2343568372 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c126a30 con 0x7f6784056790 2024-04-25T10:05:27.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f2cba7fc640 1 -- 172.21.15.60:0/36421756 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb00ab600 con 0x7f2cbc06db40 2024-04-25T10:05:27.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f933a7fc640 1 -- 172.21.15.60:0/1297538890 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93300b1070 con 0x7f933c0cbeb0 2024-04-25T10:05:27.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f933a7fc640 1 -- 172.21.15.60:0/1297538890 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93300ac5f0 con 0x7f933c0cbeb0 2024-04-25T10:05:27.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f933a7fc640 1 -- 172.21.15.60:0/1297538890 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93300ac9c0 con 0x7f933c0cbeb0 2024-04-25T10:05:27.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fc496d9d640 1 --2- 172.21.15.60:0/2868671635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49806d2b0 0x7fc49806d740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc4980b1dd0 tx=0x7fc49001c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f1c6f006b06af64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f93421fc640 1 -- 172.21.15.60:0/1297538890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933c0cbeb0 msgr2=0x7f933c0cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7f93421fc640 1 --2- 172.21.15.60:0/1297538890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933c0cbeb0 0x7f933c0cefc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f93300097c0 tx=0x7f933001c4c0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.067+0000 7fc49559a640 1 -- 172.21.15.60:0/2868671635 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4900b2070 con 0x7fc49806d2b0 2024-04-25T10:05:27.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fc49559a640 1 -- 172.21.15.60:0/2868671635 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4900a5740 con 0x7fc49806d2b0 2024-04-25T10:05:27.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fc49559a640 1 -- 172.21.15.60:0/2868671635 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4900a5b10 con 0x7fc49806d2b0 2024-04-25T10:05:27.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f93421fc640 1 -- 172.21.15.60:0/1297538890 shutdown_connections 2024-04-25T10:05:27.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f93421fc640 1 --2- 172.21.15.60:0/1297538890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933c0cbeb0 0x7f933c0cefc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f93421fc640 1 -- 172.21.15.60:0/1297538890 >> 172.21.15.60:0/1297538890 conn(0x7f933c06b4c0 msgr2=0x7f933c06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f0767fff640 1 --2- 172.21.15.60:0/860394519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07680b1c80 0x7f07680af920 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f07540097c0 tx=0x7f075401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83cf08b30b4e6f7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cba7fc640 1 -- 172.21.15.60:0/36421756 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00ab9d0 con 0x7f2cbc06db40 2024-04-25T10:05:27.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f6792c4b640 1 -- 172.21.15.60:0/2343568372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6784056790 msgr2=0x7f6784054280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f6792c4b640 1 --2- 172.21.15.60:0/2343568372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6784056790 0x7f6784054280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f678c069ba0 tx=0x7f678c0975c0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f93421fc640 1 -- 172.21.15.60:0/1297538890 shutdown_connections 2024-04-25T10:05:27.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cc0e6c640 1 -- 172.21.15.60:0/36421756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cbc06db40 msgr2=0x7f2cbc0aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cc0e6c640 1 --2- 172.21.15.60:0/36421756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cbc06db40 0x7f2cbc0aec50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc06cf90 tx=0x7f2cb001c380 comp rx=0 tx=0).stop 2024-04-25T10:05:27.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cc0e6c640 1 -- 172.21.15.60:0/36421756 shutdown_connections 2024-04-25T10:05:27.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cc0e6c640 1 --2- 172.21.15.60:0/36421756 >> v2:172.21.15.100:3300/0 conn(0x7f2cbc071100 0x7f2cbc0715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cc0e6c640 1 --2- 172.21.15.60:0/36421756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cbc06db40 0x7f2cbc0aec50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cc0e6c640 1 -- 172.21.15.60:0/36421756 >> 172.21.15.60:0/36421756 conn(0x7f2cbc0b1240 msgr2=0x7f2cbc0b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f07677fe640 1 -- 172.21.15.60:0/860394519 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07540b0070 con 0x7f07680b1c80 2024-04-25T10:05:27.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f07677fe640 1 -- 172.21.15.60:0/860394519 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07540ab600 con 0x7f07680b1c80 2024-04-25T10:05:27.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f07677fe640 1 -- 172.21.15.60:0/860394519 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07540ab9d0 con 0x7f07680b1c80 2024-04-25T10:05:27.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f6792c4b640 1 -- 172.21.15.60:0/2343568372 shutdown_connections 2024-04-25T10:05:27.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f6792c4b640 1 --2- 172.21.15.60:0/2343568372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6784056790 0x7f6784054280 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f6792c4b640 1 -- 172.21.15.60:0/2343568372 >> 172.21.15.60:0/2343568372 conn(0x7f6784053760 msgr2=0x7f6784053b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f6792c4b640 1 -- 172.21.15.60:0/2343568372 shutdown_connections 2024-04-25T10:05:27.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cc0e6c640 1 -- 172.21.15.60:0/36421756 shutdown_connections 2024-04-25T10:05:27.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fc49759e640 1 -- 172.21.15.60:0/2868671635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49806d2b0 msgr2=0x7fc49806d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fc49759e640 1 --2- 172.21.15.60:0/2868671635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49806d2b0 0x7fc49806d740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc4980b1dd0 tx=0x7fc49001c8a0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f93421fc640 1 -- 172.21.15.60:0/1297538890 wait complete. 2024-04-25T10:05:27.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fba41d9b640 1 --2- 172.21.15.60:0/2570704514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba440afa10 0x7fba440afe90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fba3800a910 tx=0x7fba38025080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b494925a3db5f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f076ef2d640 1 -- 172.21.15.60:0/860394519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07680b1c80 msgr2=0x7f07680af920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f076ef2d640 1 --2- 172.21.15.60:0/860394519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07680b1c80 0x7f07680af920 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f07540097c0 tx=0x7f075401c480 comp rx=0 tx=0).stop 2024-04-25T10:05:27.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fc49759e640 1 -- 172.21.15.60:0/2868671635 shutdown_connections 2024-04-25T10:05:27.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fc49759e640 1 --2- 172.21.15.60:0/2868671635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49806d2b0 0x7fc49806d740 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fc49759e640 1 --2- 172.21.15.60:0/2868671635 >> v2:172.21.15.60:3300/0 conn(0x7fc49806e600 0x7fc49806ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fc49759e640 1 --2- 172.21.15.60:0/2868671635 >> v2:172.21.15.100:3300/0 conn(0x7fc498161b30 0x7fc4980b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fc49759e640 1 -- 172.21.15.60:0/2868671635 >> 172.21.15.60:0/2868671635 conn(0x7fc4980b1240 msgr2=0x7fc4980b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fba4159a640 1 -- 172.21.15.60:0/2570704514 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba380bb070 con 0x7fba440afa10 2024-04-25T10:05:27.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cc0e6c640 1 -- 172.21.15.60:0/36421756 wait complete. 2024-04-25T10:05:27.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fba4159a640 1 -- 172.21.15.60:0/2570704514 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba380b6410 con 0x7fba440afa10 2024-04-25T10:05:27.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fba4159a640 1 -- 172.21.15.60:0/2570704514 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba380b67e0 con 0x7fba440afa10 2024-04-25T10:05:27.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fc49759e640 1 -- 172.21.15.60:0/2868671635 shutdown_connections 2024-04-25T10:05:27.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f6792c4b640 1 -- 172.21.15.60:0/2343568372 wait complete. 2024-04-25T10:05:27.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fba48ffe640 1 -- 172.21.15.60:0/2570704514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba440afa10 msgr2=0x7fba440afe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fba48ffe640 1 --2- 172.21.15.60:0/2570704514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba440afa10 0x7fba440afe90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fba3800a910 tx=0x7fba38025080 comp rx=0 tx=0).stop 2024-04-25T10:05:27.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f076ef2d640 1 -- 172.21.15.60:0/860394519 shutdown_connections 2024-04-25T10:05:27.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f076ef2d640 1 --2- 172.21.15.60:0/860394519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07680b1c80 0x7f07680af920 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f076ef2d640 1 -- 172.21.15.60:0/860394519 >> 172.21.15.60:0/860394519 conn(0x7f0768066fa0 msgr2=0x7f076806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f4d19030640 1 --2- 172.21.15.60:0/3814101717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d140af150 0x7f4d140cbc50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4d140ced60 tx=0x7f4d0c01f110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bbf951d2eafa18e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f93421fc640 1 Processor -- start 2024-04-25T10:05:27.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f076ef2d640 1 -- 172.21.15.60:0/860394519 shutdown_connections 2024-04-25T10:05:27.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f076ef2d640 1 -- 172.21.15.60:0/860394519 wait complete. 2024-04-25T10:05:27.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cc0e6c640 1 Processor -- start 2024-04-25T10:05:27.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cc0e6c640 1 -- start start 2024-04-25T10:05:27.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f4d1882f640 1 -- 172.21.15.60:0/3814101717 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c0b5070 con 0x7f4d140af150 2024-04-25T10:05:27.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f4d1882f640 1 -- 172.21.15.60:0/3814101717 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c0b0440 con 0x7f4d140af150 2024-04-25T10:05:27.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f4d1882f640 1 -- 172.21.15.60:0/3814101717 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c0b0810 con 0x7f4d140af150 2024-04-25T10:05:27.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fc49759e640 1 -- 172.21.15.60:0/2868671635 wait complete. 2024-04-25T10:05:27.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fba48ffe640 1 -- 172.21.15.60:0/2570704514 shutdown_connections 2024-04-25T10:05:27.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fba48ffe640 1 --2- 172.21.15.60:0/2570704514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba440afa10 0x7fba440afe90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fba48ffe640 1 --2- 172.21.15.60:0/2570704514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba44153cd0 0x7fba4415a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fba48ffe640 1 -- 172.21.15.60:0/2570704514 >> 172.21.15.60:0/2570704514 conn(0x7fba44066fa0 msgr2=0x7fba4406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fba48ffe640 1 -- 172.21.15.60:0/2570704514 shutdown_connections 2024-04-25T10:05:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f6792c4b640 1 Processor -- start 2024-04-25T10:05:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f6792c4b640 1 -- start start 2024-04-25T10:05:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fc49759e640 1 Processor -- start 2024-04-25T10:05:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fc49759e640 1 -- start start 2024-04-25T10:05:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc49759e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49806d2b0 0x7fc498163fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc49759e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc49806e600 0x7fc498202820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc49759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc498161b30 0x7fc4981648f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc49759e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980cdba0 con 0x7fc498161b30 2024-04-25T10:05:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc49759e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980cda20 con 0x7fc49806e600 2024-04-25T10:05:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc49759e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980cdd20 con 0x7fc49806d2b0 2024-04-25T10:05:27.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f4d1ba95640 1 -- 172.21.15.60:0/3814101717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d140af150 msgr2=0x7f4d140cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f4d1ba95640 1 --2- 172.21.15.60:0/3814101717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d140af150 0x7f4d140cbc50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4d140ced60 tx=0x7f4d0c01f110 comp rx=0 tx=0).stop 2024-04-25T10:05:27.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cc0e6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cbc06db40 0x7f2cbc15fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cc0e6c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cbc071100 0x7f2cbc15b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cc0e6c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cbc1652c0 0x7f2cbc15cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cc0e6c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0b4680 con 0x7f2cbc1652c0 2024-04-25T10:05:27.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cc0e6c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0b4500 con 0x7f2cbc06db40 2024-04-25T10:05:27.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f2cc0e6c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0b4800 con 0x7f2cbc071100 2024-04-25T10:05:27.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f2cbbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cbc1652c0 0x7f2cbc15cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f2cbbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cbc1652c0 0x7f2cbc15cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45442/0 (socket says 172.21.15.60:45442) 2024-04-25T10:05:27.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f2cbbfff640 1 -- 172.21.15.60:0/4126784033 learned_addr learned my addr 172.21.15.60:0/4126784033 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:27.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f2cbb7fe640 1 --2- 172.21.15.60:0/4126784033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cbc06db40 0x7f2cbc15fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f2cbbfff640 1 -- 172.21.15.60:0/4126784033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cbc071100 msgr2=0x7f2cbc15b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f2cbbfff640 1 --2- 172.21.15.60:0/4126784033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cbc071100 0x7f2cbc15b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f2cbbfff640 1 -- 172.21.15.60:0/4126784033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cbc06db40 msgr2=0x7f2cbc15fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f2cbbfff640 1 --2- 172.21.15.60:0/4126784033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cbc06db40 0x7f2cbc15fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f2cbbfff640 1 -- 172.21.15.60:0/4126784033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc159dc0 con 0x7f2cbc1652c0 2024-04-25T10:05:27.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7fba48ffe640 1 -- 172.21.15.60:0/2570704514 wait complete. 2024-04-25T10:05:27.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba48ffe640 1 Processor -- start 2024-04-25T10:05:27.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba48ffe640 1 -- start start 2024-04-25T10:05:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f076ef2d640 1 Processor -- start 2024-04-25T10:05:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f076ef2d640 1 -- start start 2024-04-25T10:05:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f6792c4b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67840521c0 0x7f6784188780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f6792c4b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6784056790 0x7f67840493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f6792c4b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67840e4800 0x7f67840d57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f6792c4b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678403ae90 con 0x7f6784056790 2024-04-25T10:05:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f6792c4b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678403ad10 con 0x7f67840521c0 2024-04-25T10:05:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f678bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6784056790 0x7f67840493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f678bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6784056790 0x7f67840493f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45452/0 (socket says 172.21.15.60:45452) 2024-04-25T10:05:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f678bfff640 1 -- 172.21.15.60:0/644113762 learned_addr learned my addr 172.21.15.60:0/644113762 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f6792c4b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678403b010 con 0x7f67840e4800 2024-04-25T10:05:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f67909e7640 1 --2- 172.21.15.60:0/644113762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67840521c0 0x7f6784188780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f67911e8640 1 --2- 172.21.15.60:0/644113762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67840e4800 0x7f67840d57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f93421fc640 1 -- start start 2024-04-25T10:05:27.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f93421fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f933c0af150 0x7f933c1fd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.071+0000 7f93421fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933c0b3e70 0x7f933c0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f933b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f933c0af150 0x7f933c1fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f933b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f933c0af150 0x7f933c1fd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45426/0 (socket says 172.21.15.60:45426) 2024-04-25T10:05:27.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f93421fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f933c0cbeb0 0x7f933c0aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f93421fc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c167580 con 0x7f933c0af150 2024-04-25T10:05:27.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f93421fc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0ce700 con 0x7f933c0b3e70 2024-04-25T10:05:27.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f93421fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933c167700 con 0x7f933c0cbeb0 2024-04-25T10:05:27.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f933b7fe640 1 -- 172.21.15.60:0/2611697872 learned_addr learned my addr 172.21.15.60:0/2611697872 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:27.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f4d1ba95640 1 -- 172.21.15.60:0/3814101717 shutdown_connections 2024-04-25T10:05:27.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f4d1ba95640 1 --2- 172.21.15.60:0/3814101717 >> v2:172.21.15.100:3300/0 conn(0x7f4d140af5b0 0x7f4d140b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f4d1ba95640 1 --2- 172.21.15.60:0/3814101717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d140af150 0x7f4d140cbc50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f4d1ba95640 1 --2- 172.21.15.60:0/3814101717 >> v2:172.21.15.60:3300/0 conn(0x7f4d1406d920 0x7f4d140cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f4d1ba95640 1 -- 172.21.15.60:0/3814101717 >> 172.21.15.60:0/3814101717 conn(0x7f4d1406b4c0 msgr2=0x7f4d1406d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f4d1ba95640 1 -- 172.21.15.60:0/3814101717 shutdown_connections 2024-04-25T10:05:27.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f4d1ba95640 1 -- 172.21.15.60:0/3814101717 wait complete. 2024-04-25T10:05:27.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba48ffe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba440afa10 0x7fba441456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba48ffe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba440cb700 0x7fba44149830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc495d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc49806e600 0x7fc498202820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc495d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc49806e600 0x7fc498202820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:39122/0 (socket says 172.21.15.60:39122) 2024-04-25T10:05:27.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc495d9b640 1 -- 172.21.15.60:0/2493574349 learned_addr learned my addr 172.21.15.60:0/2493574349 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:27.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f2cbbfff640 1 --2- 172.21.15.60:0/4126784033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cbc1652c0 0x7f2cbc15cdd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00b460 tx=0x7f2cac00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba48ffe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba44153cd0 0x7fba44071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba48ffe640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba440cd7c0 con 0x7fba440cb700 2024-04-25T10:05:27.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba48ffe640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba440cc2a0 con 0x7fba440afa10 2024-04-25T10:05:27.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba48ffe640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba440cf160 con 0x7fba44153cd0 2024-04-25T10:05:27.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba42d9d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba44153cd0 0x7fba44071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba42d9d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba44153cd0 0x7fba44071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:37308/0 (socket says 172.21.15.60:37308) 2024-04-25T10:05:27.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba42d9d640 1 -- 172.21.15.60:0/3863539390 learned_addr learned my addr 172.21.15.60:0/3863539390 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:27.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f076ef2d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07680b1c80 0x7f0768151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f076ef2d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07680cd390 0x7f07681621d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f076ef2d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0768165000 0x7f0768162710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f076ef2d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076816f9b0 con 0x7f07680cd390 2024-04-25T10:05:27.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f076ef2d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07681655f0 con 0x7f0768165000 2024-04-25T10:05:27.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f076ef2d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076816fb30 con 0x7f07680b1c80 2024-04-25T10:05:27.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f933affd640 1 --2- 172.21.15.60:0/2611697872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933c0b3e70 0x7f933c0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f933b7fe640 1 -- 172.21.15.60:0/2611697872 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f933c0cbeb0 msgr2=0x7f933c0aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f933b7fe640 1 --2- 172.21.15.60:0/2611697872 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f933c0cbeb0 0x7f933c0aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f933b7fe640 1 -- 172.21.15.60:0/2611697872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933c0b3e70 msgr2=0x7f933c0afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f933b7fe640 1 --2- 172.21.15.60:0/2611697872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933c0b3e70 0x7f933c0afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f933b7fe640 1 -- 172.21.15.60:0/2611697872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c1fe130 con 0x7f933c0af150 2024-04-25T10:05:27.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f933affd640 1 --2- 172.21.15.60:0/2611697872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933c0b3e70 0x7f933c0afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:27.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc496d9d640 1 --2- 172.21.15.60:0/2493574349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc498161b30 0x7fc4981648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc49659c640 1 --2- 172.21.15.60:0/2493574349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49806d2b0 0x7fc498163fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f933b7fe640 1 --2- 172.21.15.60:0/2611697872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f933c0af150 0x7f933c1fd900 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f933001a0f0 tx=0x7f9330019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f2ca3fff640 1 -- 172.21.15.60:0/4126784033 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac015020 con 0x7f2cbc1652c0 2024-04-25T10:05:27.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f2ca3fff640 1 -- 172.21.15.60:0/4126784033 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cac0118b0 con 0x7f2cbc1652c0 2024-04-25T10:05:27.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f2ca3fff640 1 -- 172.21.15.60:0/4126784033 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac011ca0 con 0x7f2cbc1652c0 2024-04-25T10:05:27.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba4259c640 1 --2- 172.21.15.60:0/3863539390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba440afa10 0x7fba441456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba41d9b640 1 --2- 172.21.15.60:0/3863539390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba440cb700 0x7fba44149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f0767fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07680cd390 0x7f07681621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f076d4ca640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0768165000 0x7f0768162710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f076d4ca640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0768165000 0x7f0768162710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:39158/0 (socket says 172.21.15.60:39158) 2024-04-25T10:05:27.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f076d4ca640 1 -- 172.21.15.60:0/4221542741 learned_addr learned my addr 172.21.15.60:0/4221542741 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:27.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f076ccc9640 1 --2- 172.21.15.60:0/4221542741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07680b1c80 0x7f0768151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f67911e8640 1 -- 172.21.15.60:0/644113762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67840521c0 msgr2=0x7f6784188780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f67911e8640 1 --2- 172.21.15.60:0/644113762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67840521c0 0x7f6784188780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f67911e8640 1 -- 172.21.15.60:0/644113762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6784056790 msgr2=0x7f67840493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f67911e8640 1 --2- 172.21.15.60:0/644113762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6784056790 0x7f67840493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f67911e8640 1 -- 172.21.15.60:0/644113762 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6784008870 con 0x7f67840e4800 2024-04-25T10:05:27.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f67911e8640 1 --2- 172.21.15.60:0/644113762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67840e4800 0x7f67840d57b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f678c069de0 tx=0x7f678c094360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f9323fff640 1 -- 172.21.15.60:0/2611697872 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93300ba070 con 0x7f933c0af150 2024-04-25T10:05:27.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f9323fff640 1 -- 172.21.15.60:0/2611697872 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93300b6480 con 0x7f933c0af150 2024-04-25T10:05:27.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f9323fff640 1 -- 172.21.15.60:0/2611697872 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93300b1040 con 0x7f933c0af150 2024-04-25T10:05:27.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f93421fc640 1 -- 172.21.15.60:0/2611697872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c13d170 con 0x7f933c0af150 2024-04-25T10:05:27.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f93421fc640 1 -- 172.21.15.60:0/2611697872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933c1fe6d0 con 0x7f933c0af150 2024-04-25T10:05:27.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f4d1ba95640 1 Processor -- start 2024-04-25T10:05:27.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f2cc0e6c640 1 -- 172.21.15.60:0/4126784033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc15a0a0 con 0x7f2cbc1652c0 2024-04-25T10:05:27.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f2cc0e6c640 1 -- 172.21.15.60:0/4126784033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc15a660 con 0x7f2cbc1652c0 2024-04-25T10:05:27.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba41d9b640 1 -- 172.21.15.60:0/3863539390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba44153cd0 msgr2=0x7fba44071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba41d9b640 1 --2- 172.21.15.60:0/3863539390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba44153cd0 0x7fba44071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba41d9b640 1 -- 172.21.15.60:0/3863539390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba440afa10 msgr2=0x7fba441456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba41d9b640 1 --2- 172.21.15.60:0/3863539390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba440afa10 0x7fba441456a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba41d9b640 1 -- 172.21.15.60:0/3863539390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4414c7a0 con 0x7fba440cb700 2024-04-25T10:05:27.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc496d9d640 1 -- 172.21.15.60:0/2493574349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49806d2b0 msgr2=0x7fc498163fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc496d9d640 1 --2- 172.21.15.60:0/2493574349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49806d2b0 0x7fc498163fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc496d9d640 1 -- 172.21.15.60:0/2493574349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc49806e600 msgr2=0x7fc498202820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc496d9d640 1 --2- 172.21.15.60:0/2493574349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc49806e600 0x7fc498202820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc496d9d640 1 -- 172.21.15.60:0/2493574349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc498162f10 con 0x7fc498161b30 2024-04-25T10:05:27.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fc496d9d640 1 --2- 172.21.15.60:0/2493574349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc498161b30 0x7fc4981648f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc4980b3c30 tx=0x7fc490019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f2cc0e6c640 1 -- 172.21.15.60:0/4126784033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc074f60 con 0x7f2cbc1652c0 2024-04-25T10:05:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7fba41d9b640 1 --2- 172.21.15.60:0/3863539390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba440cb700 0x7fba44149830 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fba380b5000 tx=0x7fba380238c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f0767fff640 1 -- 172.21.15.60:0/4221542741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07680b1c80 msgr2=0x7f0768151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f0767fff640 1 --2- 172.21.15.60:0/4221542741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07680b1c80 0x7f0768151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f0767fff640 1 -- 172.21.15.60:0/4221542741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0768165000 msgr2=0x7f0768162710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f0767fff640 1 --2- 172.21.15.60:0/4221542741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0768165000 0x7f0768162710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f0767fff640 1 -- 172.21.15.60:0/4221542741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0768162d50 con 0x7f07680cd390 2024-04-25T10:05:27.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f67897fa640 1 -- 172.21.15.60:0/644113762 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c12b040 con 0x7f67840e4800 2024-04-25T10:05:27.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f67897fa640 1 -- 172.21.15.60:0/644113762 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c1268b0 con 0x7f67840e4800 2024-04-25T10:05:27.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f67897fa640 1 -- 172.21.15.60:0/644113762 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c126ca0 con 0x7f67840e4800 2024-04-25T10:05:27.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f6792c4b640 1 -- 172.21.15.60:0/644113762 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6784008b50 con 0x7f67840e4800 2024-04-25T10:05:27.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.075+0000 7f6792c4b640 1 -- 172.21.15.60:0/644113762 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6784009110 con 0x7f67840e4800 2024-04-25T10:05:27.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f93421fc640 1 -- 172.21.15.60:0/2611697872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931c0052a0 con 0x7f933c0af150 2024-04-25T10:05:27.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d1ba95640 1 -- start start 2024-04-25T10:05:27.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7fc486ffd640 1 -- 172.21.15.60:0/2493574349 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4900b2050 con 0x7fc498161b30 2024-04-25T10:05:27.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7fc486ffd640 1 -- 172.21.15.60:0/2493574349 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4900be070 con 0x7fc498161b30 2024-04-25T10:05:27.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7fc486ffd640 1 -- 172.21.15.60:0/2493574349 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc490009450 con 0x7fc498161b30 2024-04-25T10:05:27.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7fc49759e640 1 -- 172.21.15.60:0/2493574349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4981631f0 con 0x7fc498161b30 2024-04-25T10:05:27.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7fc49759e640 1 -- 172.21.15.60:0/2493574349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4981637b0 con 0x7fc498161b30 2024-04-25T10:05:27.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d1ba95640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d1406d920 0x7f4d140ab410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d1ba95640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d140af150 0x7f4d140a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d1ba95640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d140af5b0 0x7f4d140712d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d1ba95640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d140b3240 con 0x7f4d140af150 2024-04-25T10:05:27.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d1ba95640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d140b2f60 con 0x7f4d140af5b0 2024-04-25T10:05:27.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d1ba95640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d140cc190 con 0x7f4d1406d920 2024-04-25T10:05:27.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f2ca3fff640 1 -- 172.21.15.60:0/4126784033 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2cac018020 con 0x7f2cbc1652c0 2024-04-25T10:05:27.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f2ca3fff640 1 --2- 172.21.15.60:0/4126784033 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ca8066f60 0x7f2ca8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7fba32ffd640 1 -- 172.21.15.60:0/3863539390 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba380bb040 con 0x7fba440cb700 2024-04-25T10:05:27.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7fba32ffd640 1 -- 172.21.15.60:0/3863539390 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba380c1420 con 0x7fba440cb700 2024-04-25T10:05:27.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7fba32ffd640 1 -- 172.21.15.60:0/3863539390 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba380c1830 con 0x7fba440cb700 2024-04-25T10:05:27.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f0767fff640 1 --2- 172.21.15.60:0/4221542741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07680cd390 0x7f07681621d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f07540ab000 tx=0x7f0754019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f6792c4b640 1 -- 172.21.15.60:0/644113762 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67840057f0 con 0x7f67840e4800 2024-04-25T10:05:27.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f2ca3fff640 1 -- 172.21.15.60:0/4126784033 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cac085780 con 0x7f2cbc1652c0 2024-04-25T10:05:27.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d19030640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d140af150 0x7f4d140a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d19030640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d140af150 0x7f4d140a96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45486/0 (socket says 172.21.15.60:45486) 2024-04-25T10:05:27.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d19030640 1 -- 172.21.15.60:0/1055699694 learned_addr learned my addr 172.21.15.60:0/1055699694 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f9323fff640 1 -- 172.21.15.60:0/2611697872 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f93300bd070 con 0x7f933c0af150 2024-04-25T10:05:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7fba48ffe640 1 -- 172.21.15.60:0/3863539390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba4414a760 con 0x7fba440cb700 2024-04-25T10:05:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7fba48ffe640 1 -- 172.21.15.60:0/3863539390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba4414a480 con 0x7fba440cb700 2024-04-25T10:05:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f2cbb7fe640 1 --2- 172.21.15.60:0/4126784033 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ca8066f60 0x7f2ca8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7fc49759e640 1 -- 172.21.15.60:0/2493574349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc498074f60 con 0x7fc498161b30 2024-04-25T10:05:27.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d19030640 1 -- 172.21.15.60:0/1055699694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d1406d920 msgr2=0x7f4d140ab410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d19831640 1 --2- 172.21.15.60:0/1055699694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d1406d920 0x7f4d140ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f9323fff640 1 --2- 172.21.15.60:0/2611697872 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9328067030 0x7f93280694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f9323fff640 1 -- 172.21.15.60:0/2611697872 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f933011f4e0 con 0x7f933c0af150 2024-04-25T10:05:27.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d19030640 1 --2- 172.21.15.60:0/1055699694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d1406d920 0x7f4d140ab410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d19030640 1 -- 172.21.15.60:0/1055699694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d140af5b0 msgr2=0x7f4d140712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d19030640 1 --2- 172.21.15.60:0/1055699694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d140af5b0 0x7f4d140712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d19030640 1 -- 172.21.15.60:0/1055699694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d140aa650 con 0x7f4d140af150 2024-04-25T10:05:27.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f933affd640 1 --2- 172.21.15.60:0/2611697872 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9328067030 0x7f93280694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f4d19030640 1 --2- 172.21.15.60:0/1055699694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d140af150 0x7f4d140a96a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c01d7c0 tx=0x7f4d0c01beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f2cbb7fe640 1 --2- 172.21.15.60:0/4126784033 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ca8066f60 0x7f2ca8069420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40012f0 tx=0x7f2cb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f67897fa640 1 -- 172.21.15.60:0/644113762 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f678c136030 con 0x7f67840e4800 2024-04-25T10:05:27.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f67897fa640 1 --2- 172.21.15.60:0/644113762 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6770067030 0x7f67700694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f67897fa640 1 -- 172.21.15.60:0/644113762 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f678c1990d0 con 0x7f67840e4800 2024-04-25T10:05:27.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.079+0000 7f67909e7640 1 --2- 172.21.15.60:0/644113762 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6770067030 0x7f67700694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f933affd640 1 --2- 172.21.15.60:0/2611697872 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9328067030 0x7f93280694f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f933c13cb80 tx=0x7f9334009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7fba48ffe640 1 -- 172.21.15.60:0/3863539390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba4406bba0 con 0x7fba440cb700 2024-04-25T10:05:27.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f67909e7640 1 --2- 172.21.15.60:0/644113762 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6770067030 0x7f67700694f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6780007fd0 tx=0x7f6780009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f6f5ea4d640 1 Processor -- start 2024-04-25T10:05:27.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f6f5ea4d640 1 -- start start 2024-04-25T10:05:27.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f6f5ea4d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6f50050230 0x7f6f50054d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f6f5ea4d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f50055270 0x7f6f50038d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f6f5ea4d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f50007fa0 con 0x7f6f500dfee0 2024-04-25T10:05:27.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f6f5ea4d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f500556e0 con 0x7f6f50050230 2024-04-25T10:05:27.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f6f5ea4d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f50008bb0 con 0x7f6f50055270 2024-04-25T10:05:27.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f6f57fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6f500dfee0 0x7f6f50009040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55842/0 (socket says 172.21.15.60:55842) 2024-04-25T10:05:27.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f6f57fff640 1 -- 172.21.15.60:0/3828765333 learned_addr learned my addr 172.21.15.60:0/3828765333 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:27.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f6f577fe640 1 --2- 172.21.15.60:0/3828765333 >> v2:172.21.15.100:3300/0 conn(0x7f6f50050230 0x7f6f50054d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f6f5cfea640 1 --2- 172.21.15.60:0/3828765333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f50055270 0x7f6f50038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f6f5cfea640 1 -- 172.21.15.60:0/3828765333 >> v2:172.21.15.100:3300/0 conn(0x7f6f50050230 msgr2=0x7f6f50054d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f6f5cfea640 1 --2- 172.21.15.60:0/3828765333 >> v2:172.21.15.100:3300/0 conn(0x7f6f50050230 0x7f6f50054d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f6f5cfea640 1 -- 172.21.15.60:0/3828765333 >> v1:172.21.15.60:6789/0 conn(0x7f6f500dfee0 legacy=0x7f6f50009040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f6f5cfea640 1 -- 172.21.15.60:0/3828765333 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f50008280 con 0x7f6f50055270 2024-04-25T10:05:27.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f6f577fe640 1 --2- 172.21.15.60:0/3828765333 >> v2:172.21.15.100:3300/0 conn(0x7f6f50050230 0x7f6f50054d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:27.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.083+0000 7f2ca3fff640 1 -- 172.21.15.60:0/4126784033 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2cac01f020 con 0x7f2cbc1652c0 2024-04-25T10:05:27.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.087+0000 7f6f5cfea640 1 --2- 172.21.15.60:0/3828765333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f50055270 0x7f6f50038d80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6f58050240 tx=0x7f6f58099c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66aa18cbb7dcfee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.087+0000 7f9323fff640 1 -- 172.21.15.60:0/2611697872 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f93300c4030 con 0x7f933c0af150 2024-04-25T10:05:27.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7f07657fa640 1 -- 172.21.15.60:0/4221542741 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07540b0040 con 0x7f07680cd390 2024-04-25T10:05:27.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7f076ef2d640 1 -- 172.21.15.60:0/4221542741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07681fc5e0 con 0x7f07680cd390 2024-04-25T10:05:27.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7f076ef2d640 1 -- 172.21.15.60:0/4221542741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0768161590 con 0x7f07680cd390 2024-04-25T10:05:27.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7f4d0a7fc640 1 -- 172.21.15.60:0/1055699694 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c0b5050 con 0x7f4d140af150 2024-04-25T10:05:27.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7f4d1ba95640 1 -- 172.21.15.60:0/1055699694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d140aa930 con 0x7f4d140af150 2024-04-25T10:05:27.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7f4d1ba95640 1 -- 172.21.15.60:0/1055699694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d140aaef0 con 0x7f4d140af150 2024-04-25T10:05:27.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7f4d0a7fc640 1 -- 172.21.15.60:0/1055699694 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c0c1070 con 0x7f4d140af150 2024-04-25T10:05:27.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7fba32ffd640 1 -- 172.21.15.60:0/3863539390 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba380c9030 con 0x7fba440cb700 2024-04-25T10:05:27.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7fba32ffd640 1 --2- 172.21.15.60:0/3863539390 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fba24067030 0x7fba240694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7f07657fa640 1 -- 172.21.15.60:0/4221542741 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07540bc070 con 0x7f07680cd390 2024-04-25T10:05:27.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7f076ef2d640 1 -- 172.21.15.60:0/4221542741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0768074770 con 0x7f07680cd390 2024-04-25T10:05:27.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7fba32ffd640 1 -- 172.21.15.60:0/3863539390 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba38128600 con 0x7fba440cb700 2024-04-25T10:05:27.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7fc486ffd640 1 -- 172.21.15.60:0/2493574349 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4900ad850 con 0x7fc498161b30 2024-04-25T10:05:27.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7fc486ffd640 1 --2- 172.21.15.60:0/2493574349 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc478066f60 0x7fc478069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7fc486ffd640 1 -- 172.21.15.60:0/2493574349 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc490120a20 con 0x7fc498161b30 2024-04-25T10:05:27.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7f4d0a7fc640 1 -- 172.21.15.60:0/1055699694 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c009450 con 0x7f4d140af150 2024-04-25T10:05:27.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7f07657fa640 1 -- 172.21.15.60:0/4221542741 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07540b5680 con 0x7f07680cd390 2024-04-25T10:05:27.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7fba4259c640 1 --2- 172.21.15.60:0/3863539390 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fba24067030 0x7fba240694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7fc49659c640 1 --2- 172.21.15.60:0/2493574349 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc478066f60 0x7fc478069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7f4d1ba95640 1 -- 172.21.15.60:0/1055699694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cfc0052a0 con 0x7f4d140af150 2024-04-25T10:05:27.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7fc49659c640 1 --2- 172.21.15.60:0/2493574349 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc478066f60 0x7fc478069420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc48800b950 tx=0x7fc488008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.091+0000 7fba4259c640 1 --2- 172.21.15.60:0/3863539390 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fba24067030 0x7fba240694f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba340098e0 tx=0x7fba34009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.095+0000 7f67897fa640 1 -- 172.21.15.60:0/644113762 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f678c13d020 con 0x7f67840e4800 2024-04-25T10:05:27.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.095+0000 7f6f56ffd640 1 -- 172.21.15.60:0/3828765333 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f58123970 con 0x7f6f50055270 2024-04-25T10:05:27.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.095+0000 7f6f56ffd640 1 -- 172.21.15.60:0/3828765333 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f58123b10 con 0x7f6f50055270 2024-04-25T10:05:27.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.095+0000 7f6f56ffd640 1 -- 172.21.15.60:0/3828765333 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5812b440 con 0x7f6f50055270 2024-04-25T10:05:27.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.095+0000 7fc486ffd640 1 -- 172.21.15.60:0/2493574349 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc490016520 con 0x7fc498161b30 2024-04-25T10:05:27.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.095+0000 7f6f5ea4d640 1 -- 172.21.15.60:0/3828765333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f50055270 msgr2=0x7f6f50038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.095+0000 7f6f5ea4d640 1 --2- 172.21.15.60:0/3828765333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f50055270 0x7f6f50038d80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6f58050240 tx=0x7f6f58099c90 comp rx=0 tx=0).stop 2024-04-25T10:05:27.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.095+0000 7f6f5ea4d640 1 -- 172.21.15.60:0/3828765333 shutdown_connections 2024-04-25T10:05:27.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.095+0000 7f6f5ea4d640 1 --2- 172.21.15.60:0/3828765333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f50055270 0x7f6f50038d80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.095+0000 7f6f5ea4d640 1 --2- 172.21.15.60:0/3828765333 >> v2:172.21.15.100:3300/0 conn(0x7f6f50050230 0x7f6f50054d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.095+0000 7f6f5ea4d640 1 -- 172.21.15.60:0/3828765333 >> 172.21.15.60:0/3828765333 conn(0x7f6f50055be0 msgr2=0x7f6f50055ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.099+0000 7f6f5ea4d640 1 -- 172.21.15.60:0/3828765333 shutdown_connections 2024-04-25T10:05:27.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.099+0000 7f6f5ea4d640 1 -- 172.21.15.60:0/3828765333 wait complete. 2024-04-25T10:05:27.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.099+0000 7f6f5ea4d640 1 Processor -- start 2024-04-25T10:05:27.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.099+0000 7f6f5ea4d640 1 -- start start 2024-04-25T10:05:27.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.099+0000 7fba32ffd640 1 -- 172.21.15.60:0/3863539390 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba380d0060 con 0x7fba440cb700 2024-04-25T10:05:27.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.099+0000 7f6f5ea4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f50050230 0x7f6f500368b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.099+0000 7f6f5ea4d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50055270 0x7f6f5004b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.099+0000 7f6f5ea4d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f500dfee0 0x7f6f5004b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.099+0000 7f6f5ea4d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f500e02c0 con 0x7f6f50050230 2024-04-25T10:05:27.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.099+0000 7f6f5ea4d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f50008840 con 0x7f6f50055270 2024-04-25T10:05:27.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.099+0000 7f6f5ea4d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f50051120 con 0x7f6f500dfee0 2024-04-25T10:05:27.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.099+0000 7f07657fa640 1 -- 172.21.15.60:0/4221542741 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f07540b3070 con 0x7f07680cd390 2024-04-25T10:05:27.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f6f577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50055270 0x7f6f5004b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f6f57fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f50050230 0x7f6f500368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f6f577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50055270 0x7f6f5004b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:39172/0 (socket says 172.21.15.60:39172) 2024-04-25T10:05:27.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f6f577fe640 1 -- 172.21.15.60:0/1586947771 learned_addr learned my addr 172.21.15.60:0/1586947771 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:27.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f6f5cfea640 1 --2- 172.21.15.60:0/1586947771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f500dfee0 0x7f6f5004b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f07657fa640 1 --2- 172.21.15.60:0/4221542741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f074c066f60 0x7f074c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f6f57fff640 1 -- 172.21.15.60:0/1586947771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f500dfee0 msgr2=0x7f6f5004b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f6f57fff640 1 --2- 172.21.15.60:0/1586947771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f500dfee0 0x7f6f5004b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f6f57fff640 1 -- 172.21.15.60:0/1586947771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50055270 msgr2=0x7f6f5004b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f6f57fff640 1 --2- 172.21.15.60:0/1586947771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50055270 0x7f6f5004b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f6f57fff640 1 -- 172.21.15.60:0/1586947771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f500e08d0 con 0x7f6f50050230 2024-04-25T10:05:27.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f6f57fff640 1 --2- 172.21.15.60:0/1586947771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f50050230 0x7f6f500368b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00c3c0 tx=0x7f6f4c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f4d0a7fc640 1 -- 172.21.15.60:0/1055699694 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d0c0bc890 con 0x7f4d140af150 2024-04-25T10:05:27.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f4d0a7fc640 1 --2- 172.21.15.60:0/1055699694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4cf8066f60 0x7f4cf8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f4d0a7fc640 1 -- 172.21.15.60:0/1055699694 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d0c124650 con 0x7f4d140af150 2024-04-25T10:05:27.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f076ccc9640 1 --2- 172.21.15.60:0/4221542741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f074c066f60 0x7f074c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f07657fa640 1 -- 172.21.15.60:0/4221542741 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07540ee370 con 0x7f07680cd390 2024-04-25T10:05:27.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.103+0000 7f4d19831640 1 --2- 172.21.15.60:0/1055699694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4cf8066f60 0x7f4cf8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.107+0000 7f076ccc9640 1 --2- 172.21.15.60:0/4221542741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f074c066f60 0x7f074c069420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f07680cbae0 tx=0x7f075c009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.107+0000 7f4d19831640 1 --2- 172.21.15.60:0/1055699694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4cf8066f60 0x7f4cf8069420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009a20 tx=0x7f4d04008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.107+0000 7f07657fa640 1 -- 172.21.15.60:0/4221542741 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0754016070 con 0x7f07680cd390 2024-04-25T10:05:27.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.111+0000 7f6f54ff9640 1 -- 172.21.15.60:0/1586947771 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c017070 con 0x7f6f50050230 2024-04-25T10:05:27.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.111+0000 7f6f54ff9640 1 -- 172.21.15.60:0/1586947771 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4c005cf0 con 0x7f6f50050230 2024-04-25T10:05:27.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.111+0000 7f6f54ff9640 1 -- 172.21.15.60:0/1586947771 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c0072f0 con 0x7f6f50050230 2024-04-25T10:05:27.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.111+0000 7f6f5ea4d640 1 -- 172.21.15.60:0/1586947771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f500e0bb0 con 0x7f6f50050230 2024-04-25T10:05:27.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.111+0000 7f6f5ea4d640 1 -- 172.21.15.60:0/1586947771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f500e1170 con 0x7f6f50050230 2024-04-25T10:05:27.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.119+0000 7f4d0a7fc640 1 -- 172.21.15.60:0/1055699694 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d0c018c90 con 0x7f4d140af150 2024-04-25T10:05:27.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.119+0000 7f6f54ff9640 1 -- 172.21.15.60:0/1586947771 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6f4c01a070 con 0x7f6f50050230 2024-04-25T10:05:27.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.119+0000 7f6f54ff9640 1 --2- 172.21.15.60:0/1586947771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f3c067030 0x7f6f3c0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.119+0000 7f6f54ff9640 1 -- 172.21.15.60:0/1586947771 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f4c0842b0 con 0x7f6f50050230 2024-04-25T10:05:27.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.119+0000 7f6f577fe640 1 --2- 172.21.15.60:0/1586947771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f3c067030 0x7f6f3c0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.119+0000 7f6f577fe640 1 --2- 172.21.15.60:0/1586947771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f3c067030 0x7f6f3c0694f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009580 tx=0x7f6f44009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.131+0000 7f6f427fc640 1 -- 172.21.15.60:0/1586947771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f500057f0 con 0x7f6f50050230 2024-04-25T10:05:27.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.143+0000 7f6f54ff9640 1 -- 172.21.15.60:0/1586947771 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6f4c054c50 con 0x7f6f50050230 2024-04-25T10:05:27.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.403+0000 7f2cc0e6c640 1 -- 172.21.15.60:0/4126784033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2cbc0b1620 con 0x7f2cbc1652c0 2024-04-25T10:05:27.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.403+0000 7f2ca3fff640 1 -- 172.21.15.60:0/4126784033 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2cac00d040 con 0x7f2cbc1652c0 2024-04-25T10:05:27.411 INFO:teuthology.orchestra.run.smithi060.stdout:60129542149 2024-04-25T10:05:27.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.407+0000 7f2ca1ffb640 1 -- 172.21.15.60:0/4126784033 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ca8066f60 msgr2=0x7f2ca8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.407+0000 7f2ca1ffb640 1 --2- 172.21.15.60:0/4126784033 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ca8066f60 0x7f2ca8069420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40012f0 tx=0x7f2cb4009040 comp rx=0 tx=0).stop 2024-04-25T10:05:27.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.407+0000 7f2ca1ffb640 1 -- 172.21.15.60:0/4126784033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cbc1652c0 msgr2=0x7f2cbc15cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.407+0000 7f2ca1ffb640 1 --2- 172.21.15.60:0/4126784033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cbc1652c0 0x7f2cbc15cdd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00b460 tx=0x7f2cac00b920 comp rx=0 tx=0).stop 2024-04-25T10:05:27.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.407+0000 7f2ca1ffb640 1 -- 172.21.15.60:0/4126784033 shutdown_connections 2024-04-25T10:05:27.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.407+0000 7f2ca1ffb640 1 --2- 172.21.15.60:0/4126784033 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ca8066f60 0x7f2ca8069420 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.407+0000 7f2ca1ffb640 1 --2- 172.21.15.60:0/4126784033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cbc1652c0 0x7f2cbc15cdd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.407+0000 7f2ca1ffb640 1 --2- 172.21.15.60:0/4126784033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cbc071100 0x7f2cbc15b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.407+0000 7f2ca1ffb640 1 --2- 172.21.15.60:0/4126784033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cbc06db40 0x7f2cbc15fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.407+0000 7f2ca1ffb640 1 -- 172.21.15.60:0/4126784033 >> 172.21.15.60:0/4126784033 conn(0x7f2cbc0b1240 msgr2=0x7f2cbc0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.407+0000 7f2ca1ffb640 1 -- 172.21.15.60:0/4126784033 shutdown_connections 2024-04-25T10:05:27.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.407+0000 7f2ca1ffb640 1 -- 172.21.15.60:0/4126784033 wait complete. 2024-04-25T10:05:27.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.411+0000 7fc49759e640 1 -- 172.21.15.60:0/2493574349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc4980cc350 con 0x7fc498161b30 2024-04-25T10:05:27.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.415+0000 7fc486ffd640 1 -- 172.21.15.60:0/2493574349 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc4900f1990 con 0x7fc498161b30 2024-04-25T10:05:27.423 INFO:teuthology.orchestra.run.smithi060.stdout:60129542149 2024-04-25T10:05:27.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.419+0000 7fc484ff9640 1 -- 172.21.15.60:0/2493574349 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc478066f60 msgr2=0x7fc478069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.419+0000 7fc484ff9640 1 --2- 172.21.15.60:0/2493574349 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc478066f60 0x7fc478069420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc48800b950 tx=0x7fc488008040 comp rx=0 tx=0).stop 2024-04-25T10:05:27.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.419+0000 7fc484ff9640 1 -- 172.21.15.60:0/2493574349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc498161b30 msgr2=0x7fc4981648f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.419+0000 7fc484ff9640 1 --2- 172.21.15.60:0/2493574349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc498161b30 0x7fc4981648f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc4980b3c30 tx=0x7fc490019640 comp rx=0 tx=0).stop 2024-04-25T10:05:27.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.419+0000 7fc484ff9640 1 -- 172.21.15.60:0/2493574349 shutdown_connections 2024-04-25T10:05:27.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.419+0000 7fc484ff9640 1 --2- 172.21.15.60:0/2493574349 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc478066f60 0x7fc478069420 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.419+0000 7fc484ff9640 1 --2- 172.21.15.60:0/2493574349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc498161b30 0x7fc4981648f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.419+0000 7fc484ff9640 1 --2- 172.21.15.60:0/2493574349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc49806e600 0x7fc498202820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.419+0000 7fc484ff9640 1 --2- 172.21.15.60:0/2493574349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49806d2b0 0x7fc498163fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.419+0000 7fc484ff9640 1 -- 172.21.15.60:0/2493574349 >> 172.21.15.60:0/2493574349 conn(0x7fc4980b1240 msgr2=0x7fc498160e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.419+0000 7fc484ff9640 1 -- 172.21.15.60:0/2493574349 shutdown_connections 2024-04-25T10:05:27.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.419+0000 7fc484ff9640 1 -- 172.21.15.60:0/2493574349 wait complete. 2024-04-25T10:05:27.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.419+0000 7f93421fc640 1 -- 172.21.15.60:0/2611697872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f931c005860 con 0x7f933c0af150 2024-04-25T10:05:27.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.419+0000 7f6792c4b640 1 -- 172.21.15.60:0/644113762 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f67840d67a0 con 0x7f67840e4800 2024-04-25T10:05:27.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.423+0000 7f67897fa640 1 -- 172.21.15.60:0/644113762 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f678c091280 con 0x7f67840e4800 2024-04-25T10:05:27.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.423+0000 7f9323fff640 1 -- 172.21.15.60:0/2611697872 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f93300a9070 con 0x7f933c0af150 2024-04-25T10:05:27.429 INFO:teuthology.orchestra.run.smithi060.stdout:64424509446 2024-04-25T10:05:27.431 INFO:teuthology.orchestra.run.smithi060.stdout:60129542149 2024-04-25T10:05:27.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.423+0000 7fba48ffe640 1 -- 172.21.15.60:0/3863539390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fba44074770 con 0x7fba440cb700 2024-04-25T10:05:27.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.423+0000 7fba32ffd640 1 -- 172.21.15.60:0/3863539390 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba3801ec70 con 0x7fba440cb700 2024-04-25T10:05:27.432 INFO:teuthology.orchestra.run.smithi060.stdout:60129542149 2024-04-25T10:05:27.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.423+0000 7f9321ffb640 1 -- 172.21.15.60:0/2611697872 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9328067030 msgr2=0x7f93280694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.423+0000 7f9321ffb640 1 --2- 172.21.15.60:0/2611697872 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9328067030 0x7f93280694f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f933c13cb80 tx=0x7f9334009040 comp rx=0 tx=0).stop 2024-04-25T10:05:27.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.423+0000 7f9321ffb640 1 -- 172.21.15.60:0/2611697872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f933c0af150 msgr2=0x7f933c1fd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.423+0000 7f9321ffb640 1 --2- 172.21.15.60:0/2611697872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f933c0af150 0x7f933c1fd900 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f933001a0f0 tx=0x7f9330019260 comp rx=0 tx=0).stop 2024-04-25T10:05:27.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7f9321ffb640 1 -- 172.21.15.60:0/2611697872 shutdown_connections 2024-04-25T10:05:27.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7f9321ffb640 1 --2- 172.21.15.60:0/2611697872 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9328067030 0x7f93280694f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7f9321ffb640 1 --2- 172.21.15.60:0/2611697872 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f933c0cbeb0 0x7f933c0aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7f9321ffb640 1 --2- 172.21.15.60:0/2611697872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f933c0b3e70 0x7f933c0afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7f9321ffb640 1 --2- 172.21.15.60:0/2611697872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f933c0af150 0x7f933c1fd900 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7f9321ffb640 1 -- 172.21.15.60:0/2611697872 >> 172.21.15.60:0/2611697872 conn(0x7f933c06b4c0 msgr2=0x7f933c13f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7f9321ffb640 1 -- 172.21.15.60:0/2611697872 shutdown_connections 2024-04-25T10:05:27.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7f9321ffb640 1 -- 172.21.15.60:0/2611697872 wait complete. 2024-04-25T10:05:27.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7fba30ff9640 1 -- 172.21.15.60:0/3863539390 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fba24067030 msgr2=0x7fba240694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7fba30ff9640 1 --2- 172.21.15.60:0/3863539390 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fba24067030 0x7fba240694f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba340098e0 tx=0x7fba34009060 comp rx=0 tx=0).stop 2024-04-25T10:05:27.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7fba30ff9640 1 -- 172.21.15.60:0/3863539390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba440cb700 msgr2=0x7fba44149830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7fba30ff9640 1 --2- 172.21.15.60:0/3863539390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba440cb700 0x7fba44149830 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fba380b5000 tx=0x7fba380238c0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7fba30ff9640 1 -- 172.21.15.60:0/3863539390 shutdown_connections 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7fba30ff9640 1 --2- 172.21.15.60:0/3863539390 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fba24067030 0x7fba240694f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7fba30ff9640 1 --2- 172.21.15.60:0/3863539390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba44153cd0 0x7fba44071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7fba30ff9640 1 --2- 172.21.15.60:0/3863539390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba440cb700 0x7fba44149830 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7fba30ff9640 1 --2- 172.21.15.60:0/3863539390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba440afa10 0x7fba441456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.427+0000 7fba30ff9640 1 -- 172.21.15.60:0/3863539390 >> 172.21.15.60:0/3863539390 conn(0x7fba44066fa0 msgr2=0x7fba4414f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.431+0000 7f6776ffd640 1 -- 172.21.15.60:0/644113762 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6770067030 msgr2=0x7f67700694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.431+0000 7f6776ffd640 1 --2- 172.21.15.60:0/644113762 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6770067030 0x7f67700694f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6780007fd0 tx=0x7f6780009040 comp rx=0 tx=0).stop 2024-04-25T10:05:27.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.431+0000 7f6776ffd640 1 -- 172.21.15.60:0/644113762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67840e4800 msgr2=0x7f67840d57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.431+0000 7f6776ffd640 1 --2- 172.21.15.60:0/644113762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67840e4800 0x7f67840d57b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f678c069de0 tx=0x7f678c094360 comp rx=0 tx=0).stop 2024-04-25T10:05:27.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.431+0000 7f6776ffd640 1 -- 172.21.15.60:0/644113762 shutdown_connections 2024-04-25T10:05:27.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.431+0000 7f6776ffd640 1 --2- 172.21.15.60:0/644113762 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6770067030 0x7f67700694f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.431+0000 7f6776ffd640 1 --2- 172.21.15.60:0/644113762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67840e4800 0x7f67840d57b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.431+0000 7f6776ffd640 1 --2- 172.21.15.60:0/644113762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6784056790 0x7f67840493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.431+0000 7f6776ffd640 1 --2- 172.21.15.60:0/644113762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67840521c0 0x7f6784188780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.431+0000 7f6776ffd640 1 -- 172.21.15.60:0/644113762 >> 172.21.15.60:0/644113762 conn(0x7f6784053760 msgr2=0x7f67840d3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.431+0000 7f6776ffd640 1 -- 172.21.15.60:0/644113762 shutdown_connections 2024-04-25T10:05:27.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.431+0000 7f6776ffd640 1 -- 172.21.15.60:0/644113762 wait complete. 2024-04-25T10:05:27.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.431+0000 7fba30ff9640 1 -- 172.21.15.60:0/3863539390 shutdown_connections 2024-04-25T10:05:27.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.431+0000 7fba30ff9640 1 -- 172.21.15.60:0/3863539390 wait complete. 2024-04-25T10:05:27.441 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.3 2024-04-25T10:05:27.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.431+0000 7f076ef2d640 1 -- 172.21.15.60:0/4221542741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f0768160ae0 con 0x7f07680cd390 2024-04-25T10:05:27.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.435+0000 7f07657fa640 1 -- 172.21.15.60:0/4221542741 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f07540f09d0 con 0x7f07680cd390 2024-04-25T10:05:27.441 INFO:teuthology.orchestra.run.smithi060.stdout:64424509445 2024-04-25T10:05:27.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.439+0000 7f076ef2d640 1 -- 172.21.15.60:0/4221542741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f074c066f60 msgr2=0x7f074c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.439+0000 7f076ef2d640 1 --2- 172.21.15.60:0/4221542741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f074c066f60 0x7f074c069420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f07680cbae0 tx=0x7f075c009060 comp rx=0 tx=0).stop 2024-04-25T10:05:27.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.439+0000 7f076ef2d640 1 -- 172.21.15.60:0/4221542741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07680cd390 msgr2=0x7f07681621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.439+0000 7f076ef2d640 1 --2- 172.21.15.60:0/4221542741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07680cd390 0x7f07681621d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f07540ab000 tx=0x7f0754019220 comp rx=0 tx=0).stop 2024-04-25T10:05:27.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.439+0000 7f076ef2d640 1 -- 172.21.15.60:0/4221542741 shutdown_connections 2024-04-25T10:05:27.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.439+0000 7f076ef2d640 1 --2- 172.21.15.60:0/4221542741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f074c066f60 0x7f074c069420 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.439+0000 7f076ef2d640 1 --2- 172.21.15.60:0/4221542741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0768165000 0x7f0768162710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.439+0000 7f076ef2d640 1 --2- 172.21.15.60:0/4221542741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07680cd390 0x7f07681621d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.439+0000 7f076ef2d640 1 --2- 172.21.15.60:0/4221542741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07680b1c80 0x7f0768151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.439+0000 7f076ef2d640 1 -- 172.21.15.60:0/4221542741 >> 172.21.15.60:0/4221542741 conn(0x7f0768066fa0 msgr2=0x7f07680711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.439+0000 7f076ef2d640 1 -- 172.21.15.60:0/4221542741 shutdown_connections 2024-04-25T10:05:27.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.439+0000 7f076ef2d640 1 -- 172.21.15.60:0/4221542741 wait complete. 2024-04-25T10:05:27.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.439+0000 7f4d1ba95640 1 -- 172.21.15.60:0/1055699694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f4cfc005860 con 0x7f4d140af150 2024-04-25T10:05:27.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.439+0000 7f4d0a7fc640 1 -- 172.21.15.60:0/1055699694 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4d0c0f5640 con 0x7f4d140af150 2024-04-25T10:05:27.447 INFO:teuthology.orchestra.run.smithi060.stdout:60129542148 2024-04-25T10:05:27.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.447+0000 7f4d1ba95640 1 -- 172.21.15.60:0/1055699694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4cf8066f60 msgr2=0x7f4cf8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.447+0000 7f4d1ba95640 1 --2- 172.21.15.60:0/1055699694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4cf8066f60 0x7f4cf8069420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009a20 tx=0x7f4d04008040 comp rx=0 tx=0).stop 2024-04-25T10:05:27.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.447+0000 7f4d1ba95640 1 -- 172.21.15.60:0/1055699694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d140af150 msgr2=0x7f4d140a96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.447+0000 7f4d1ba95640 1 --2- 172.21.15.60:0/1055699694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d140af150 0x7f4d140a96a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c01d7c0 tx=0x7f4d0c01beb0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.447+0000 7f4d1ba95640 1 -- 172.21.15.60:0/1055699694 shutdown_connections 2024-04-25T10:05:27.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.447+0000 7f4d1ba95640 1 --2- 172.21.15.60:0/1055699694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4cf8066f60 0x7f4cf8069420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.447+0000 7f4d1ba95640 1 --2- 172.21.15.60:0/1055699694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d140af5b0 0x7f4d140712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.447+0000 7f4d1ba95640 1 --2- 172.21.15.60:0/1055699694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d140af150 0x7f4d140a96a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.447+0000 7f4d1ba95640 1 --2- 172.21.15.60:0/1055699694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d1406d920 0x7f4d140ab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.447+0000 7f4d1ba95640 1 -- 172.21.15.60:0/1055699694 >> 172.21.15.60:0/1055699694 conn(0x7f4d1406b4c0 msgr2=0x7f4d1406d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.447+0000 7f4d1ba95640 1 -- 172.21.15.60:0/1055699694 shutdown_connections 2024-04-25T10:05:27.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.447+0000 7f4d1ba95640 1 -- 172.21.15.60:0/1055699694 wait complete. 2024-04-25T10:05:27.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.1 2024-04-25T10:05:27.466 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.7 2024-04-25T10:05:27.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.459+0000 7f6f427fc640 1 -- 172.21.15.60:0/1586947771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6f50007560 con 0x7f6f50050230 2024-04-25T10:05:27.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.459+0000 7f6f54ff9640 1 -- 172.21.15.60:0/1586947771 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6f4c058cf0 con 0x7f6f50050230 2024-04-25T10:05:27.468 INFO:teuthology.orchestra.run.smithi060.stdout:60129542150 2024-04-25T10:05:27.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.463+0000 7f6f427fc640 1 -- 172.21.15.60:0/1586947771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f3c067030 msgr2=0x7f6f3c0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.463+0000 7f6f427fc640 1 --2- 172.21.15.60:0/1586947771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f3c067030 0x7f6f3c0694f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009580 tx=0x7f6f44009190 comp rx=0 tx=0).stop 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.463+0000 7f6f427fc640 1 -- 172.21.15.60:0/1586947771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f50050230 msgr2=0x7f6f500368b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.463+0000 7f6f427fc640 1 --2- 172.21.15.60:0/1586947771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f50050230 0x7f6f500368b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00c3c0 tx=0x7f6f4c00c880 comp rx=0 tx=0).stop 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.463+0000 7f6f427fc640 1 -- 172.21.15.60:0/1586947771 shutdown_connections 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.463+0000 7f6f427fc640 1 --2- 172.21.15.60:0/1586947771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f3c067030 0x7f6f3c0694f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.463+0000 7f6f427fc640 1 --2- 172.21.15.60:0/1586947771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f500dfee0 0x7f6f5004b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.463+0000 7f6f427fc640 1 --2- 172.21.15.60:0/1586947771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50055270 0x7f6f5004b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.463+0000 7f6f427fc640 1 --2- 172.21.15.60:0/1586947771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f50050230 0x7f6f500368b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.463+0000 7f6f427fc640 1 -- 172.21.15.60:0/1586947771 >> 172.21.15.60:0/1586947771 conn(0x7f6f50055be0 msgr2=0x7f6f5004ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.463+0000 7f6f427fc640 1 -- 172.21.15.60:0/1586947771 shutdown_connections 2024-04-25T10:05:27.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:27.463+0000 7f6f427fc640 1 -- 172.21.15.60:0/1586947771 wait complete. 2024-04-25T10:05:27.472 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.5 2024-04-25T10:05:27.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.2 2024-04-25T10:05:27.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2024-04-25T10:05:27.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2024-04-25T10:05:27.487 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.0 2024-04-25T10:05:28.442 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:05:28.460 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:05:28.466 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:05:28.473 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:05:28.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:05:28.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:05:28.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:05:28.488 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:05:28.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.619+0000 7f3a80529640 1 Processor -- start 2024-04-25T10:05:28.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.623+0000 7f3a80529640 1 -- start start 2024-04-25T10:05:28.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.623+0000 7f3a80529640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3a780cefc0 0x7f3a780b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.623+0000 7f3a80529640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a780af860 con 0x7f3a780b3ca0 2024-04-25T10:05:28.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.623+0000 7f3a80529640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a780b26c0 con 0x7f3a780cbeb0 2024-04-25T10:05:28.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.623+0000 7f3a80529640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a781433d0 con 0x7f3a780cefc0 2024-04-25T10:05:28.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.623+0000 7f3a7eac6640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3a780b3ca0 0x7f3a780cd7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55852/0 (socket says 172.21.15.60:55852) 2024-04-25T10:05:28.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.623+0000 7f3a7eac6640 1 -- 172.21.15.60:0/2829495038 learned_addr learned my addr 172.21.15.60:0/2829495038 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:28.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.623+0000 7f3a7dac4640 1 --2- 172.21.15.60:0/2829495038 >> v2:172.21.15.100:3300/0 conn(0x7f3a780cefc0 0x7f3a780b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.623+0000 7f3a7d2c3640 1 -- 172.21.15.60:0/2829495038 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251388687 0 0) 0x7f3a780af860 con 0x7f3a780b3ca0 2024-04-25T10:05:28.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.623+0000 7f3a7d2c3640 1 -- 172.21.15.60:0/2829495038 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a64004640 con 0x7f3a780b3ca0 2024-04-25T10:05:28.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.623+0000 7f3a7dac4640 1 -- 172.21.15.60:0/2829495038 >> v1:172.21.15.100:6789/0 conn(0x7f3a780cbeb0 legacy=0x7f3a780af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.623+0000 7f3a7dac4640 1 -- 172.21.15.60:0/2829495038 >> v1:172.21.15.60:6789/0 conn(0x7f3a780b3ca0 legacy=0x7f3a780cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.623+0000 7f3a7dac4640 1 -- 172.21.15.60:0/2829495038 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a780b29a0 con 0x7f3a780cefc0 2024-04-25T10:05:28.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.627+0000 7f3a7dac4640 1 --2- 172.21.15.60:0/2829495038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a780cefc0 0x7f3a780b3760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009910 tx=0x7f3a7001c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b801034fb52395e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.627+0000 7f3a7d2c3640 1 -- 172.21.15.60:0/2829495038 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a700b2070 con 0x7f3a780cefc0 2024-04-25T10:05:28.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.627+0000 7f3a7d2c3640 1 -- 172.21.15.60:0/2829495038 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a700a5980 con 0x7f3a780cefc0 2024-04-25T10:05:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.627+0000 7f3a80529640 1 -- 172.21.15.60:0/2829495038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a780cefc0 msgr2=0x7f3a780b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.627+0000 7f3a80529640 1 --2- 172.21.15.60:0/2829495038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a780cefc0 0x7f3a780b3760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009910 tx=0x7f3a7001c9b0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.627+0000 7f3a80529640 1 -- 172.21.15.60:0/2829495038 shutdown_connections 2024-04-25T10:05:28.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.627+0000 7f3a80529640 1 --2- 172.21.15.60:0/2829495038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a780cefc0 0x7f3a780b3760 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.627+0000 7f3a80529640 1 -- 172.21.15.60:0/2829495038 >> 172.21.15.60:0/2829495038 conn(0x7f3a7806b4c0 msgr2=0x7f3a7806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.631+0000 7f3a80529640 1 -- 172.21.15.60:0/2829495038 shutdown_connections 2024-04-25T10:05:28.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.631+0000 7f3a80529640 1 -- 172.21.15.60:0/2829495038 wait complete. 2024-04-25T10:05:28.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.631+0000 7f3a80529640 1 Processor -- start 2024-04-25T10:05:28.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.631+0000 7f3a80529640 1 -- start start 2024-04-25T10:05:28.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.631+0000 7f3a80529640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a780b3ca0 0x7f3a781595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.631+0000 7f3a80529640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a780cbeb0 0x7f3a7813e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.631+0000 7f3a80529640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a780cefc0 0x7f3a7813a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.631+0000 7f3a80529640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a780cc290 con 0x7f3a780b3ca0 2024-04-25T10:05:28.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.631+0000 7f3a80529640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a780b3240 con 0x7f3a780cbeb0 2024-04-25T10:05:28.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.631+0000 7f3a80529640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a78167580 con 0x7f3a780cefc0 2024-04-25T10:05:28.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.631+0000 7f3a7e2c5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a780b3ca0 0x7f3a781595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.631+0000 7f3a7e2c5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a780b3ca0 0x7f3a781595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45502/0 (socket says 172.21.15.60:45502) 2024-04-25T10:05:28.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.631+0000 7f3a7e2c5640 1 -- 172.21.15.60:0/4199027122 learned_addr learned my addr 172.21.15.60:0/4199027122 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:28.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.635+0000 7f3a7eac6640 1 --2- 172.21.15.60:0/4199027122 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a780cefc0 0x7f3a7813a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.635+0000 7f3a7dac4640 1 --2- 172.21.15.60:0/4199027122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a780cbeb0 0x7f3a7813e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.635+0000 7f3a7e2c5640 1 -- 172.21.15.60:0/4199027122 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a780cefc0 msgr2=0x7f3a7813a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.635+0000 7f3a7e2c5640 1 --2- 172.21.15.60:0/4199027122 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a780cefc0 0x7f3a7813a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.635+0000 7f3a7e2c5640 1 -- 172.21.15.60:0/4199027122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a780cbeb0 msgr2=0x7f3a7813e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.635+0000 7f3a7e2c5640 1 --2- 172.21.15.60:0/4199027122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a780cbeb0 0x7f3a7813e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.635+0000 7f3a7e2c5640 1 -- 172.21.15.60:0/4199027122 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7813f5b0 con 0x7f3a780b3ca0 2024-04-25T10:05:28.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.635+0000 7f3a7e2c5640 1 --2- 172.21.15.60:0/4199027122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a780b3ca0 0x7f3a781595e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800c780 tx=0x7f3a6800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.635+0000 7f3a6effd640 1 -- 172.21.15.60:0/4199027122 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a68017070 con 0x7f3a780b3ca0 2024-04-25T10:05:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.635+0000 7f3a80529640 1 -- 172.21.15.60:0/4199027122 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7813f890 con 0x7f3a780b3ca0 2024-04-25T10:05:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.635+0000 7f3a80529640 1 -- 172.21.15.60:0/4199027122 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7815cb70 con 0x7f3a780b3ca0 2024-04-25T10:05:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.635+0000 7f3a6effd640 1 -- 172.21.15.60:0/4199027122 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a68005480 con 0x7f3a780b3ca0 2024-04-25T10:05:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.635+0000 7f3a6effd640 1 -- 172.21.15.60:0/4199027122 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800a880 con 0x7f3a780b3ca0 2024-04-25T10:05:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.635+0000 7f3a6effd640 1 -- 172.21.15.60:0/4199027122 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a6801a070 con 0x7f3a780b3ca0 2024-04-25T10:05:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.635+0000 7f3a6effd640 1 --2- 172.21.15.60:0/4199027122 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a64067400 0x7f3a640698c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.639+0000 7f3a7dac4640 1 --2- 172.21.15.60:0/4199027122 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a64067400 0x7f3a640698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.639+0000 7f3a6effd640 1 -- 172.21.15.60:0/4199027122 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a68084520 con 0x7f3a780b3ca0 2024-04-25T10:05:28.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.639+0000 7f3a7dac4640 1 --2- 172.21.15.60:0/4199027122 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a64067400 0x7f3a640698c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a700af000 tx=0x7f3a700a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.647+0000 7f3a80529640 1 -- 172.21.15.60:0/4199027122 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a600052a0 con 0x7f3a780b3ca0 2024-04-25T10:05:28.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.655+0000 7f3a6effd640 1 -- 172.21.15.60:0/4199027122 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a68054e40 con 0x7f3a780b3ca0 2024-04-25T10:05:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.663+0000 7f1a9a520640 1 Processor -- start 2024-04-25T10:05:28.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.663+0000 7f1a9a520640 1 -- start start 2024-04-25T10:05:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.663+0000 7f1a9a520640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1a94140220 0x7f1a9406e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.663+0000 7f1a9a520640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a940cf100 0x7f1a94071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.663+0000 7f1a9a520640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a9415b260 con 0x7f1a9406ef20 2024-04-25T10:05:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.663+0000 7f1a9a520640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a9415be70 con 0x7f1a94140220 2024-04-25T10:05:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.663+0000 7f1a9a520640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9415b540 con 0x7f1a940cf100 2024-04-25T10:05:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.663+0000 7f1a98abd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1a9406ef20 0x7f1a9406d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55862/0 (socket says 172.21.15.60:55862) 2024-04-25T10:05:28.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.663+0000 7f1a93fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1a94140220 0x7f1a9406e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.663+0000 7f1a98abd640 1 -- 172.21.15.60:0/2756874040 learned_addr learned my addr 172.21.15.60:0/2756874040 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:28.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.663+0000 7f1a937fe640 1 --2- 172.21.15.60:0/2756874040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a940cf100 0x7f1a94071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.663+0000 7f1a93fff640 1 -- 172.21.15.60:0/2756874040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a940cf100 msgr2=0x7f1a94071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.663+0000 7f1a93fff640 1 --2- 172.21.15.60:0/2756874040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a940cf100 0x7f1a94071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.663+0000 7f1a93fff640 1 -- 172.21.15.60:0/2756874040 >> v1:172.21.15.60:6789/0 conn(0x7f1a9406ef20 legacy=0x7f1a9406d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.663+0000 7f1a93fff640 1 -- 172.21.15.60:0/2756874040 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9415b820 con 0x7f1a94140220 2024-04-25T10:05:28.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.667+0000 7f1a93fff640 1 --2- 172.21.15.60:0/2756874040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a94140220 0x7f1a9406e9e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1a84006200 tx=0x7f1a840209d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c078f77c099f53c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.667+0000 7f1a92ffd640 1 -- 172.21.15.60:0/2756874040 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8400c2d0 con 0x7f1a94140220 2024-04-25T10:05:28.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.667+0000 7f1a92ffd640 1 -- 172.21.15.60:0/2756874040 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a84004030 con 0x7f1a94140220 2024-04-25T10:05:28.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.667+0000 7f1a92ffd640 1 -- 172.21.15.60:0/2756874040 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84004440 con 0x7f1a94140220 2024-04-25T10:05:28.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.667+0000 7fbee6a33640 1 Processor -- start 2024-04-25T10:05:28.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.667+0000 7fbee6a33640 1 -- start start 2024-04-25T10:05:28.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.667+0000 7f364e1ca640 1 Processor -- start 2024-04-25T10:05:28.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f1a9a520640 1 -- 172.21.15.60:0/2756874040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a94140220 msgr2=0x7f1a9406e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f1a9a520640 1 --2- 172.21.15.60:0/2756874040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a94140220 0x7f1a9406e9e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1a84006200 tx=0x7f1a840209d0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.667+0000 7f8801003640 1 Processor -- start 2024-04-25T10:05:28.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.667+0000 7f8801003640 1 -- start start 2024-04-25T10:05:28.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f1a9a520640 1 -- 172.21.15.60:0/2756874040 shutdown_connections 2024-04-25T10:05:28.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f1a9a520640 1 --2- 172.21.15.60:0/2756874040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a940cf100 0x7f1a94071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f1a9a520640 1 --2- 172.21.15.60:0/2756874040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a94140220 0x7f1a9406e9e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f1a9a520640 1 -- 172.21.15.60:0/2756874040 >> 172.21.15.60:0/2756874040 conn(0x7f1a940b12d0 msgr2=0x7f1a940b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f1a9a520640 1 -- 172.21.15.60:0/2756874040 shutdown_connections 2024-04-25T10:05:28.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f1a9a520640 1 -- 172.21.15.60:0/2756874040 wait complete. 2024-04-25T10:05:28.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f8801003640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f87fc15a860 0x7f87fc15ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f8801003640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87fc0b0240 con 0x7f87fc0afcd0 2024-04-25T10:05:28.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f8801003640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87fc0b0520 con 0x7f87fc0b19f0 2024-04-25T10:05:28.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f8801003640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87fc0b0800 con 0x7f87fc15a860 2024-04-25T10:05:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f87fb7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f87fc0afcd0 0x7f87fc0b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55866/0 (socket says 172.21.15.60:55866) 2024-04-25T10:05:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f87fb7fe640 1 -- 172.21.15.60:0/3296973748 learned_addr learned my addr 172.21.15.60:0/3296973748 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7fbee6a33640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbee00cec50 0x7fbee00b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7fbee6a33640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee00b4e10 con 0x7fbee00aec30 2024-04-25T10:05:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7fbee6a33640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee00b50f0 con 0x7fbee0161ac0 2024-04-25T10:05:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7fbee6a33640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee00cf030 con 0x7fbee00cec50 2024-04-25T10:05:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7fbee5a31640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbee00cec50 0x7fbee00b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f364e1ca640 1 -- start start 2024-04-25T10:05:28.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f364e1ca640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f36480b3610 0x7f364815c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f364e1ca640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364815b670 con 0x7f36481583d0 2024-04-25T10:05:28.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f364e1ca640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364815b950 con 0x7f36480b14a0 2024-04-25T10:05:28.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f364e1ca640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364815bc30 con 0x7f36480b3610 2024-04-25T10:05:28.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7fbee5a31640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbee00cec50 0x7fbee00b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45518/0 (socket says 172.21.15.60:45518) 2024-04-25T10:05:28.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7fbee5a31640 1 -- 172.21.15.60:0/1805924263 learned_addr learned my addr 172.21.15.60:0/1805924263 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:28.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f87fbfff640 1 --2- 172.21.15.60:0/3296973748 >> v2:172.21.15.100:3300/0 conn(0x7f87fc15a860 0x7f87fc15ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f3646ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f36480b3610 0x7f364815c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f3646ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f36480b3610 0x7f364815c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45520/0 (socket says 172.21.15.60:45520) 2024-04-25T10:05:28.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f3646ffd640 1 -- 172.21.15.60:0/2192517848 learned_addr learned my addr 172.21.15.60:0/2192517848 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:28.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f87fbfff640 1 -- 172.21.15.60:0/3296973748 >> v1:172.21.15.100:6789/0 conn(0x7f87fc0b19f0 legacy=0x7f87fc06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f87fbfff640 1 -- 172.21.15.60:0/3296973748 >> v1:172.21.15.60:6789/0 conn(0x7f87fc0afcd0 legacy=0x7f87fc0b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7fbee5a31640 1 -- 172.21.15.60:0/1805924263 >> v1:172.21.15.100:6789/0 conn(0x7fbee0161ac0 legacy=0x7fbee00ad0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7fbee5a31640 1 -- 172.21.15.60:0/1805924263 >> v1:172.21.15.60:6789/0 conn(0x7fbee00aec30 legacy=0x7fbee01613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f3646ffd640 1 -- 172.21.15.60:0/2192517848 >> v1:172.21.15.100:6789/0 conn(0x7f36480b14a0 legacy=0x7f36480d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f3646ffd640 1 -- 172.21.15.60:0/2192517848 >> v1:172.21.15.60:6789/0 conn(0x7f36481583d0 legacy=0x7f364815ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f3646ffd640 1 -- 172.21.15.60:0/2192517848 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36480d0590 con 0x7f36480b3610 2024-04-25T10:05:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7fbee5a31640 1 -- 172.21.15.60:0/1805924263 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee00b53d0 con 0x7fbee00cec50 2024-04-25T10:05:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f1a9a520640 1 Processor -- start 2024-04-25T10:05:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f1a9a520640 1 -- start start 2024-04-25T10:05:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f1a9a520640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a9406ef20 0x7f1a94130650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f1a9a520640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a940cf100 0x7f1a94133160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f1a9a520640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a94131db0 0x7f1a940b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f1a9a520640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94166640 con 0x7f1a9406ef20 2024-04-25T10:05:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f1a9a520640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94071be0 con 0x7f1a940cf100 2024-04-25T10:05:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f1a9a520640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a941667c0 con 0x7f1a94131db0 2024-04-25T10:05:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a940cf100 0x7f1a94133160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a93fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a9406ef20 0x7f1a94130650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a940cf100 0x7f1a94133160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:39212/0 (socket says 172.21.15.60:39212) 2024-04-25T10:05:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a937fe640 1 -- 172.21.15.60:0/1480195413 learned_addr learned my addr 172.21.15.60:0/1480195413 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:28.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58359e640 1 Processor -- start 2024-04-25T10:05:28.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a93fff640 1 -- 172.21.15.60:0/1480195413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a94131db0 msgr2=0x7f1a940b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a93fff640 1 --2- 172.21.15.60:0/1480195413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a94131db0 0x7f1a940b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a93fff640 1 -- 172.21.15.60:0/1480195413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a940cf100 msgr2=0x7f1a94133160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a93fff640 1 --2- 172.21.15.60:0/1480195413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a940cf100 0x7f1a94133160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a93fff640 1 -- 172.21.15.60:0/1480195413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a94130d10 con 0x7f1a9406ef20 2024-04-25T10:05:28.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a93fff640 1 --2- 172.21.15.60:0/1480195413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a9406ef20 0x7f1a94130650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1a9406bf70 tx=0x7f1a8401d770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fe29ae39640 1 Processor -- start 2024-04-25T10:05:28.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58359e640 1 -- start start 2024-04-25T10:05:28.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58359e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa5840b35a0 0x7fa5840cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58359e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5840cd730 con 0x7fa5840b4d90 2024-04-25T10:05:28.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58359e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5840b3ad0 con 0x7fa5841597e0 2024-04-25T10:05:28.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58359e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5840b3db0 con 0x7fa5840b35a0 2024-04-25T10:05:28.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.671+0000 7f87fbfff640 1 -- 172.21.15.60:0/3296973748 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc0b0ae0 con 0x7f87fc15a860 2024-04-25T10:05:28.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f87fbfff640 1 --2- 172.21.15.60:0/3296973748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87fc15a860 0x7f87fc15ac40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f87f00097f0 tx=0x7f87f001ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b08b1a4036c0bda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fbee5a31640 1 --2- 172.21.15.60:0/1805924263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbee00cec50 0x7fbee00b2620 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009870 tx=0x7fbedc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff12c439d5da3e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f3646ffd640 1 --2- 172.21.15.60:0/2192517848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36480b3610 0x7f364815c000 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3638009870 tx=0x7f363801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9645d68d2d33b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a90ff9640 1 -- 172.21.15.60:0/1480195413 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a840a9a40 con 0x7f1a9406ef20 2024-04-25T10:05:28.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a90ff9640 1 -- 172.21.15.60:0/1480195413 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a840a9be0 con 0x7f1a9406ef20 2024-04-25T10:05:28.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a90ff9640 1 -- 172.21.15.60:0/1480195413 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84003620 con 0x7f1a9406ef20 2024-04-25T10:05:28.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fbee4a2f640 1 -- 172.21.15.60:0/1805924263 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0b2070 con 0x7fbee00cec50 2024-04-25T10:05:28.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a9a520640 1 -- 172.21.15.60:0/1480195413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a94130ff0 con 0x7f1a9406ef20 2024-04-25T10:05:28.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a9a520640 1 -- 172.21.15.60:0/1480195413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a941315b0 con 0x7f1a9406ef20 2024-04-25T10:05:28.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa581d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa5840b35a0 0x7fa5840cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa581d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa5840b35a0 0x7fa5840cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:39220/0 (socket says 172.21.15.60:39220) 2024-04-25T10:05:28.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa581d9b640 1 -- 172.21.15.60:0/3690429875 learned_addr learned my addr 172.21.15.60:0/3690429875 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:28.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fbee4a2f640 1 -- 172.21.15.60:0/1805924263 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc0a9070 con 0x7fbee00cec50 2024-04-25T10:05:28.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fbee4a2f640 1 -- 172.21.15.60:0/1805924263 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0ad880 con 0x7fbee00cec50 2024-04-25T10:05:28.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fe29ae39640 1 -- start start 2024-04-25T10:05:28.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58159a640 1 -- 172.21.15.60:0/3690429875 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297998189 0 0) 0x7fa5840cd730 con 0x7fa5840b4d90 2024-04-25T10:05:28.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58159a640 1 -- 172.21.15.60:0/3690429875 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa56c003530 con 0x7fa5840b4d90 2024-04-25T10:05:28.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a9a520640 1 -- 172.21.15.60:0/1480195413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a94074770 con 0x7f1a9406ef20 2024-04-25T10:05:28.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fbee6a33640 1 -- 172.21.15.60:0/1805924263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbee00cec50 msgr2=0x7fbee00b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fbee6a33640 1 --2- 172.21.15.60:0/1805924263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbee00cec50 0x7fbee00b2620 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009870 tx=0x7fbedc01c450 comp rx=0 tx=0).stop 2024-04-25T10:05:28.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f36467fc640 1 -- 172.21.15.60:0/2192517848 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36380aa660 con 0x7f36480b3610 2024-04-25T10:05:28.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f36467fc640 1 -- 172.21.15.60:0/2192517848 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36380af070 con 0x7f36480b3610 2024-04-25T10:05:28.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f36467fc640 1 -- 172.21.15.60:0/2192517848 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36380aaa20 con 0x7f36480b3610 2024-04-25T10:05:28.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f364e1ca640 1 -- 172.21.15.60:0/2192517848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36480b3610 msgr2=0x7f364815c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f364e1ca640 1 --2- 172.21.15.60:0/2192517848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36480b3610 0x7f364815c000 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3638009870 tx=0x7f363801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58159a640 1 -- 172.21.15.60:0/3690429875 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2855478010 0 0) 0x7fa56c003530 con 0x7fa5840b4d90 2024-04-25T10:05:28.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f364e1ca640 1 -- 172.21.15.60:0/2192517848 shutdown_connections 2024-04-25T10:05:28.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f364e1ca640 1 --2- 172.21.15.60:0/2192517848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36480b3610 0x7f364815c000 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3638009870 tx=0x7f363801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f364e1ca640 1 -- 172.21.15.60:0/2192517848 >> 172.21.15.60:0/2192517848 conn(0x7f3648072920 msgr2=0x7f3648099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58159a640 1 -- 172.21.15.60:0/3690429875 >> v2:172.21.15.100:3300/0 conn(0x7fa5840b35a0 msgr2=0x7fa5840cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58159a640 1 --2- 172.21.15.60:0/3690429875 >> v2:172.21.15.100:3300/0 conn(0x7fa5840b35a0 0x7fa5840cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58159a640 1 -- 172.21.15.60:0/3690429875 >> v1:172.21.15.100:6789/0 conn(0x7fa5841597e0 legacy=0x7fa5840ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58159a640 1 -- 172.21.15.60:0/3690429875 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5840cd9d0 con 0x7fa5840b4d90 2024-04-25T10:05:28.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58159a640 1 -- 172.21.15.60:0/3690429875 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fa57c001a20 con 0x7fa5840b4d90 2024-04-25T10:05:28.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58159a640 1 -- 172.21.15.60:0/3690429875 >> v1:172.21.15.60:6789/0 conn(0x7fa5840b4d90 legacy=0x7fa5840b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fbee6a33640 1 -- 172.21.15.60:0/1805924263 shutdown_connections 2024-04-25T10:05:28.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fbee6a33640 1 --2- 172.21.15.60:0/1805924263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbee00cec50 0x7fbee00b2620 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fbee6a33640 1 -- 172.21.15.60:0/1805924263 >> 172.21.15.60:0/1805924263 conn(0x7fbee006e870 msgr2=0x7fbee006ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58159a640 1 --2- 172.21.15.60:0/3690429875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa56c0039c0 0x7fa56c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58159a640 1 --2- 172.21.15.60:0/3690429875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa56c004810 0x7fa56c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58159a640 1 --2- 172.21.15.60:0/3690429875 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa56c0051d0 0x7fa56c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58159a640 1 -- 172.21.15.60:0/3690429875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840b3db0 con 0x7fa56c0039c0 2024-04-25T10:05:28.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58159a640 1 -- 172.21.15.60:0/3690429875 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c00c050 con 0x7fa56c004810 2024-04-25T10:05:28.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fa58159a640 1 -- 172.21.15.60:0/3690429875 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c00c250 con 0x7fa56c0051d0 2024-04-25T10:05:28.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f364e1ca640 1 -- 172.21.15.60:0/2192517848 shutdown_connections 2024-04-25T10:05:28.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f1a90ff9640 1 -- 172.21.15.60:0/1480195413 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a840037c0 con 0x7f1a9406ef20 2024-04-25T10:05:28.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f364e1ca640 1 -- 172.21.15.60:0/2192517848 wait complete. 2024-04-25T10:05:28.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7fbee6a33640 1 -- 172.21.15.60:0/1805924263 shutdown_connections 2024-04-25T10:05:28.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f1a90ff9640 1 --2- 172.21.15.60:0/1480195413 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1a80067030 0x7f1a800694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f1a90ff9640 1 -- 172.21.15.60:0/1480195413 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a8411eee0 con 0x7f1a9406ef20 2024-04-25T10:05:28.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f364e1ca640 1 Processor -- start 2024-04-25T10:05:28.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7fbee6a33640 1 -- 172.21.15.60:0/1805924263 wait complete. 2024-04-25T10:05:28.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7fa581d9b640 1 --2- 172.21.15.60:0/3690429875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa56c004810 0x7fa56c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f364e1ca640 1 -- start start 2024-04-25T10:05:28.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f3d02939640 1 Processor -- start 2024-04-25T10:05:28.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f3d02939640 1 -- start start 2024-04-25T10:05:28.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f1a937fe640 1 --2- 172.21.15.60:0/1480195413 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1a80067030 0x7f1a800694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7fa581d9b640 1 -- 172.21.15.60:0/3690429875 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa56c0051d0 msgr2=0x7fa56c00b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:05:28.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7fa581d9b640 1 --2- 172.21.15.60:0/3690429875 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa56c0051d0 0x7fa56c00b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7fa581d9b640 1 -- 172.21.15.60:0/3690429875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa56c0039c0 msgr2=0x7fa56c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7fa581d9b640 1 --2- 172.21.15.60:0/3690429875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa56c0039c0 0x7fa56c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7fa581d9b640 1 -- 172.21.15.60:0/3690429875 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5840cd9d0 con 0x7fa56c004810 2024-04-25T10:05:28.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f364e1ca640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36480b14a0 0x7f364815a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f364e1ca640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36481583d0 0x7f36481fec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f364e1ca640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3648148fe0 0x7f3648159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f364e1ca640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364806ad90 con 0x7f3648148fe0 2024-04-25T10:05:28.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f364e1ca640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364806ac10 con 0x7f36480b14a0 2024-04-25T10:05:28.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f364e1ca640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364806af10 con 0x7f36481583d0 2024-04-25T10:05:28.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f36477fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36480b14a0 0x7f364815a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7f87faffd640 1 -- 172.21.15.60:0/3296973748 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f00b5070 con 0x7f87fc15a860 2024-04-25T10:05:28.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f87faffd640 1 -- 172.21.15.60:0/3296973748 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f00a8950 con 0x7f87fc15a860 2024-04-25T10:05:28.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f87faffd640 1 -- 172.21.15.60:0/3296973748 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f00a8d20 con 0x7f87fc15a860 2024-04-25T10:05:28.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f8801003640 1 -- 172.21.15.60:0/3296973748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87fc15a860 msgr2=0x7f87fc15ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f8801003640 1 --2- 172.21.15.60:0/3296973748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87fc15a860 0x7f87fc15ac40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f87f00097f0 tx=0x7f87f001ec50 comp rx=0 tx=0).stop 2024-04-25T10:05:28.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f36477fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36480b14a0 0x7f364815a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:39234/0 (socket says 172.21.15.60:39234) 2024-04-25T10:05:28.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f36477fe640 1 -- 172.21.15.60:0/3628356002 learned_addr learned my addr 172.21.15.60:0/3628356002 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:28.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f8801003640 1 -- 172.21.15.60:0/3296973748 shutdown_connections 2024-04-25T10:05:28.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f8801003640 1 --2- 172.21.15.60:0/3296973748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87fc15a860 0x7f87fc15ac40 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f87f00097f0 tx=0x7f87f001ec50 comp rx=0 tx=0).stop 2024-04-25T10:05:28.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f8801003640 1 -- 172.21.15.60:0/3296973748 >> 172.21.15.60:0/3296973748 conn(0x7f87fc0b6120 msgr2=0x7f87fc099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7fa582d9d640 1 -- 172.21.15.60:0/3690429875 reap_dead start 2024-04-25T10:05:28.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f3646ffd640 1 --2- 172.21.15.60:0/3628356002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36481583d0 0x7f36481fec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f8801003640 1 -- 172.21.15.60:0/3296973748 shutdown_connections 2024-04-25T10:05:28.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f3647fff640 1 --2- 172.21.15.60:0/3628356002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3648148fe0 0x7f3648159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7fbee6a33640 1 Processor -- start 2024-04-25T10:05:28.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7fbee6a33640 1 -- start start 2024-04-25T10:05:28.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f3cfb7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3cf404f7c0 0x7f3cf404fc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55906/0 (socket says 172.21.15.60:55906) 2024-04-25T10:05:28.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f3cfbfff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f3cf404f3e0 0x7f3cf4053420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:48790/0 (socket says 172.21.15.60:48790) 2024-04-25T10:05:28.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f3cfb7fe640 1 -- 172.21.15.60:0/1222464097 learned_addr learned my addr 172.21.15.60:0/1222464097 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:28.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f36477fe640 1 -- 172.21.15.60:0/3628356002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36481583d0 msgr2=0x7f36481fec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f36477fe640 1 --2- 172.21.15.60:0/3628356002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36481583d0 0x7f36481fec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f36477fe640 1 -- 172.21.15.60:0/3628356002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3648148fe0 msgr2=0x7f3648159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f36477fe640 1 --2- 172.21.15.60:0/3628356002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3648148fe0 0x7f3648159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f3d02939640 1 --2- 172.21.15.60:0/1222464097 >> v2:172.21.15.100:3300/0 conn(0x7f3cf4109b70 0x7f3cf410bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f36477fe640 1 -- 172.21.15.60:0/3628356002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364815ccf0 con 0x7f36480b14a0 2024-04-25T10:05:28.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f1a937fe640 1 --2- 172.21.15.60:0/1480195413 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1a80067030 0x7f1a800694f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1a940acd00 tx=0x7f1a8c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fe29ae39640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe28c0dffe0 0x7fe28c0e4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fe29ae39640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe28c051130 0x7fe28c054dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fe29ae39640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe28c055510 con 0x7fe28c038e10 2024-04-25T10:05:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fe29ae39640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe28c0e03c0 con 0x7fe28c0dffe0 2024-04-25T10:05:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fe29ae39640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c0095c0 con 0x7fe28c051130 2024-04-25T10:05:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fe299e37640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe28c0dffe0 0x7fe28c0e4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fe299e37640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe28c0dffe0 0x7fe28c0e4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45536/0 (socket says 172.21.15.60:45536) 2024-04-25T10:05:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fe299e37640 1 -- 172.21.15.60:0/2475816820 learned_addr learned my addr 172.21.15.60:0/2475816820 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fe298e35640 1 -- 172.21.15.60:0/2475816820 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706607542 0 0) 0x7fe28c055510 con 0x7fe28c038e10 2024-04-25T10:05:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fe298e35640 1 -- 172.21.15.60:0/2475816820 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe280003530 con 0x7fe28c038e10 2024-04-25T10:05:28.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fe299e37640 1 -- 172.21.15.60:0/2475816820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe28c051130 msgr2=0x7fe28c054dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:05:28.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fe299e37640 1 --2- 172.21.15.60:0/2475816820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe28c051130 0x7fe28c054dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fe299e37640 1 -- 172.21.15.60:0/2475816820 >> v1:172.21.15.60:6789/0 conn(0x7fe28c038e10 legacy=0x7fe28c050a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.675+0000 7fe299e37640 1 -- 172.21.15.60:0/2475816820 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c0557f0 con 0x7fe28c0dffe0 2024-04-25T10:05:28.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fe299e37640 1 --2- 172.21.15.60:0/2475816820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe28c0dffe0 0x7fe28c0e4510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe29402d380 tx=0x7fe29409fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753aceeefae9341 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f3d02939640 1 -- 172.21.15.60:0/1222464097 --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf4101180 con 0x7f3cf404f7c0 2024-04-25T10:05:28.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f3d00ed6640 1 --2- 172.21.15.60:0/1222464097 >> v2:172.21.15.100:3300/0 conn(0x7f3cf4109b70 0x7f3cf410bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f3d02939640 1 -- 172.21.15.60:0/1222464097 --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf403a5c0 con 0x7f3cf404f3e0 2024-04-25T10:05:28.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f36477fe640 1 --2- 172.21.15.60:0/3628356002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36480b14a0 0x7f364815a2b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f363c00b3e0 tx=0x7f363c00b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.679+0000 7f8801003640 1 -- 172.21.15.60:0/3296973748 wait complete. 2024-04-25T10:05:28.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f3d02939640 1 -- 172.21.15.60:0/1222464097 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cf4005ea0 con 0x7f3cf4109b70 2024-04-25T10:05:28.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee6a33640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbee00aec30 0x7fbee0154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee6a33640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbee00cec50 0x7fbee0133bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee6a33640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbee0161ac0 0x7fbee0134100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee6a33640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee016b970 con 0x7fbee00cec50 2024-04-25T10:05:28.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee6a33640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00b56b0 con 0x7fbee00aec30 2024-04-25T10:05:28.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee6a33640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee016baf0 con 0x7fbee0161ac0 2024-04-25T10:05:28.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f362ffff640 1 -- 172.21.15.60:0/3628356002 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c015020 con 0x7f36480b14a0 2024-04-25T10:05:28.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f364e1ca640 1 -- 172.21.15.60:0/3628356002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36481ffb30 con 0x7f36480b14a0 2024-04-25T10:05:28.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f364e1ca640 1 -- 172.21.15.60:0/3628356002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36481ff850 con 0x7f36480b14a0 2024-04-25T10:05:28.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8801003640 1 Processor -- start 2024-04-25T10:05:28.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f3cfaffd640 1 -- 172.21.15.60:0/1222464097 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280229816 0 0) 0x7f3cf403a5c0 con 0x7f3cf404f3e0 2024-04-25T10:05:28.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f3cfaffd640 1 -- 172.21.15.60:0/1222464097 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cec003530 con 0x7f3cf404f3e0 2024-04-25T10:05:28.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee5a31640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbee00aec30 0x7fbee0154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee5a31640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbee00aec30 0x7fbee0154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:39252/0 (socket says 172.21.15.60:39252) 2024-04-25T10:05:28.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee5a31640 1 -- 172.21.15.60:0/54693008 learned_addr learned my addr 172.21.15.60:0/54693008 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:28.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee6232640 1 --2- 172.21.15.60:0/54693008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbee0161ac0 0x7fbee0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee5a31640 1 -- 172.21.15.60:0/54693008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbee0161ac0 msgr2=0x7fbee0134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee5a31640 1 --2- 172.21.15.60:0/54693008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbee0161ac0 0x7fbee0134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee5a31640 1 -- 172.21.15.60:0/54693008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbee00cec50 msgr2=0x7fbee0133bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee5a31640 1 --2- 172.21.15.60:0/54693008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbee00cec50 0x7fbee0133bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee5a31640 1 -- 172.21.15.60:0/54693008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee0156430 con 0x7fbee00aec30 2024-04-25T10:05:28.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f362ffff640 1 -- 172.21.15.60:0/3628356002 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c011850 con 0x7f36480b14a0 2024-04-25T10:05:28.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f362ffff640 1 -- 172.21.15.60:0/3628356002 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c011c40 con 0x7f36480b14a0 2024-04-25T10:05:28.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f3d00ed6640 1 -- 172.21.15.60:0/1222464097 >> v1:172.21.15.100:6789/0 conn(0x7f3cf404f3e0 legacy=0x7f3cf4053420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f3d00ed6640 1 -- 172.21.15.60:0/1222464097 >> v1:172.21.15.60:6789/0 conn(0x7f3cf404f7c0 legacy=0x7f3cf404fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f3d00ed6640 1 -- 172.21.15.60:0/1222464097 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf403a8a0 con 0x7f3cf4109b70 2024-04-25T10:05:28.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f364e1ca640 1 -- 172.21.15.60:0/3628356002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36280052a0 con 0x7f36480b14a0 2024-04-25T10:05:28.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee5a31640 1 --2- 172.21.15.60:0/54693008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbee00aec30 0x7fbee0154730 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0b0040 tx=0x7fbedc01ac90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fa581d9b640 1 --2- 172.21.15.60:0/3690429875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa56c004810 0x7fa56c004c90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa574010240 tx=0x7fa57401eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18e4d8314cc82dff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8801003640 1 -- start start 2024-04-25T10:05:28.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8801003640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87fc0afcd0 0x7f87fc0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8801003640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87fc0b19f0 0x7f87fc09a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8801003640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87fc146430 0x7f87fc09a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8801003640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc0aee90 con 0x7f87fc146430 2024-04-25T10:05:28.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8801003640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc0aed10 con 0x7f87fc0afcd0 2024-04-25T10:05:28.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8801003640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc15b180 con 0x7f87fc0b19f0 2024-04-25T10:05:28.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbed67fc640 1 -- 172.21.15.60:0/54693008 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0b2040 con 0x7fbee00aec30 2024-04-25T10:05:28.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbed67fc640 1 -- 172.21.15.60:0/54693008 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc0adcf0 con 0x7fbee00aec30 2024-04-25T10:05:28.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbed67fc640 1 -- 172.21.15.60:0/54693008 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0b8680 con 0x7fbee00aec30 2024-04-25T10:05:28.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8800802640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87fc146430 0x7f87fc09a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee6a33640 1 -- 172.21.15.60:0/54693008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee0156710 con 0x7fbee00aec30 2024-04-25T10:05:28.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8800802640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87fc146430 0x7f87fc09a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45570/0 (socket says 172.21.15.60:45570) 2024-04-25T10:05:28.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8800802640 1 -- 172.21.15.60:0/3371663290 learned_addr learned my addr 172.21.15.60:0/3371663290 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:28.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f87fb7fe640 1 --2- 172.21.15.60:0/3371663290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87fc0b19f0 0x7f87fc09a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fbee6a33640 1 -- 172.21.15.60:0/54693008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee0156cd0 con 0x7fbee00aec30 2024-04-25T10:05:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fa58159a640 1 -- 172.21.15.60:0/3690429875 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5740ad070 con 0x7fa56c004810 2024-04-25T10:05:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fa58159a640 1 -- 172.21.15.60:0/3690429875 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5740a7c40 con 0x7fa56c004810 2024-04-25T10:05:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fa58159a640 1 -- 172.21.15.60:0/3690429875 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5740086f0 con 0x7fa56c004810 2024-04-25T10:05:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8800802640 1 -- 172.21.15.60:0/3371663290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87fc0b19f0 msgr2=0x7f87fc09a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8800802640 1 --2- 172.21.15.60:0/3371663290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87fc0b19f0 0x7f87fc09a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8800802640 1 -- 172.21.15.60:0/3371663290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87fc0afcd0 msgr2=0x7f87fc0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f87fbfff640 1 --2- 172.21.15.60:0/3371663290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87fc0afcd0 0x7f87fc0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8800802640 1 --2- 172.21.15.60:0/3371663290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87fc0afcd0 0x7f87fc0b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8800802640 1 -- 172.21.15.60:0/3371663290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc146930 con 0x7f87fc146430 2024-04-25T10:05:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f87fbfff640 1 --2- 172.21.15.60:0/3371663290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87fc0afcd0 0x7f87fc0b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fa58359e640 1 -- 172.21.15.60:0/3690429875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa56c004810 msgr2=0x7fa56c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fa58359e640 1 --2- 172.21.15.60:0/3690429875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa56c004810 0x7fa56c004c90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa574010240 tx=0x7fa57401eda0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fa58359e640 1 -- 172.21.15.60:0/3690429875 shutdown_connections 2024-04-25T10:05:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fa58359e640 1 --2- 172.21.15.60:0/3690429875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa56c004810 0x7fa56c004c90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fa58359e640 1 -- 172.21.15.60:0/3690429875 >> 172.21.15.60:0/3690429875 conn(0x7fa58406e870 msgr2=0x7fa58406ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7f8800802640 1 --2- 172.21.15.60:0/3371663290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87fc146430 0x7f87fc09a9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f87f400c390 tx=0x7f87f400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fa58359e640 1 -- 172.21.15.60:0/3690429875 shutdown_connections 2024-04-25T10:05:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.683+0000 7fa58359e640 1 -- 172.21.15.60:0/3690429875 wait complete. 2024-04-25T10:05:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7f3d00ed6640 1 --2- 172.21.15.60:0/1222464097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cf4109b70 0x7f3cf410bf60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4006020 tx=0x7f3cfc088870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44a126794d4656f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7f362ffff640 1 -- 172.21.15.60:0/3628356002 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f363c018020 con 0x7f36480b14a0 2024-04-25T10:05:28.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fbee6a33640 1 -- 172.21.15.60:0/54693008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec80052a0 con 0x7fbee00aec30 2024-04-25T10:05:28.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa58359e640 1 Processor -- start 2024-04-25T10:05:28.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fe298e35640 1 -- 172.21.15.60:0/2475816820 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294135070 con 0x7fe28c0dffe0 2024-04-25T10:05:28.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7f362ffff640 1 --2- 172.21.15.60:0/3628356002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3634066f60 0x7f3634069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7f1a90ff9640 1 -- 172.21.15.60:0/1480195413 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a8401a5f0 con 0x7f1a9406ef20 2024-04-25T10:05:28.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fe298e35640 1 -- 172.21.15.60:0/2475816820 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe294128ca0 con 0x7fe28c0dffe0 2024-04-25T10:05:28.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fe298e35640 1 -- 172.21.15.60:0/2475816820 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294130640 con 0x7fe28c0dffe0 2024-04-25T10:05:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fe29ae39640 1 -- 172.21.15.60:0/2475816820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe28c0dffe0 msgr2=0x7fe28c0e4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fe29ae39640 1 --2- 172.21.15.60:0/2475816820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe28c0dffe0 0x7fe28c0e4510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe29402d380 tx=0x7fe29409fef0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fe29ae39640 1 -- 172.21.15.60:0/2475816820 shutdown_connections 2024-04-25T10:05:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fe29ae39640 1 --2- 172.21.15.60:0/2475816820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe28c051130 0x7fe28c054dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fe29ae39640 1 --2- 172.21.15.60:0/2475816820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe28c0dffe0 0x7fe28c0e4510 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fe29ae39640 1 -- 172.21.15.60:0/2475816820 >> 172.21.15.60:0/2475816820 conn(0x7fe28c055be0 msgr2=0x7fe28c055ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa58359e640 1 -- start start 2024-04-25T10:05:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa58359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa56c004810 0x7fa584202ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa58359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5840a8ea0 0x7fa58415c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa58359e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5840aa150 0x7fa5840aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa58359e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840b0b20 con 0x7fa56c004810 2024-04-25T10:05:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa58359e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840b4ad0 con 0x7fa5840a8ea0 2024-04-25T10:05:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa58359e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840b0ca0 con 0x7fa5840aa150 2024-04-25T10:05:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa581d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5840a8ea0 0x7fa58415c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa58259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa56c004810 0x7fa584202ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa581d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5840a8ea0 0x7fa58415c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:39284/0 (socket says 172.21.15.60:39284) 2024-04-25T10:05:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa581d9b640 1 -- 172.21.15.60:0/3013931816 learned_addr learned my addr 172.21.15.60:0/3013931816 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7f87f8ff9640 1 -- 172.21.15.60:0/3371663290 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f40052d0 con 0x7f87fc146430 2024-04-25T10:05:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7f87f8ff9640 1 -- 172.21.15.60:0/3371663290 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f4005470 con 0x7f87fc146430 2024-04-25T10:05:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7f87f8ff9640 1 -- 172.21.15.60:0/3371663290 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4007240 con 0x7f87fc146430 2024-04-25T10:05:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7f8801003640 1 -- 172.21.15.60:0/3371663290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87fc146c10 con 0x7f87fc146430 2024-04-25T10:05:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7f8801003640 1 -- 172.21.15.60:0/3371663290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87fc1471d0 con 0x7f87fc146430 2024-04-25T10:05:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fe29ae39640 1 -- 172.21.15.60:0/2475816820 shutdown_connections 2024-04-25T10:05:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7f362ffff640 1 -- 172.21.15.60:0/3628356002 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f363c0854f0 con 0x7f36480b14a0 2024-04-25T10:05:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7f3646ffd640 1 --2- 172.21.15.60:0/3628356002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3634066f60 0x7f3634069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7f3646ffd640 1 --2- 172.21.15.60:0/3628356002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3634066f60 0x7f3634069420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f36380ae000 tx=0x7f36380a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa582d9d640 1 --2- 172.21.15.60:0/3013931816 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5840aa150 0x7fa5840aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa58259c640 1 -- 172.21.15.60:0/3013931816 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5840aa150 msgr2=0x7fa5840aa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa58259c640 1 --2- 172.21.15.60:0/3013931816 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5840aa150 0x7fa5840aa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa58259c640 1 -- 172.21.15.60:0/3013931816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5840a8ea0 msgr2=0x7fa58415c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa58259c640 1 --2- 172.21.15.60:0/3013931816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5840a8ea0 0x7fa58415c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa58259c640 1 -- 172.21.15.60:0/3013931816 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa584203000 con 0x7fa56c004810 2024-04-25T10:05:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fe29ae39640 1 -- 172.21.15.60:0/2475816820 wait complete. 2024-04-25T10:05:28.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa58259c640 1 --2- 172.21.15.60:0/3013931816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa56c004810 0x7fa584202ac0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa58415d2f0 tx=0x7fa57800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7f8801003640 1 -- 172.21.15.60:0/3371663290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87fc074770 con 0x7f87fc146430 2024-04-25T10:05:28.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa572ffd640 1 -- 172.21.15.60:0/3013931816 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa578017070 con 0x7fa56c004810 2024-04-25T10:05:28.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa572ffd640 1 -- 172.21.15.60:0/3013931816 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa578003600 con 0x7fa56c004810 2024-04-25T10:05:28.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa58359e640 1 -- 172.21.15.60:0/3013931816 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5840a9330 con 0x7fa56c004810 2024-04-25T10:05:28.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.687+0000 7fa572ffd640 1 -- 172.21.15.60:0/3013931816 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa578003a30 con 0x7fa56c004810 2024-04-25T10:05:28.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fa58359e640 1 -- 172.21.15.60:0/3013931816 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa584159e70 con 0x7fa56c004810 2024-04-25T10:05:28.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29ae39640 1 Processor -- start 2024-04-25T10:05:28.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fa58359e640 1 -- 172.21.15.60:0/3013931816 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa584074770 con 0x7fa56c004810 2024-04-25T10:05:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29ae39640 1 -- start start 2024-04-25T10:05:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29ae39640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe28c038e10 0x7fe28c0dc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29ae39640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe28c051130 0x7fe28c04ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29ae39640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe28c0dffe0 0x7fe28c0d9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29ae39640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c009190 con 0x7fe28c051130 2024-04-25T10:05:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29ae39640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c009010 con 0x7fe28c038e10 2024-04-25T10:05:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29ae39640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c009310 con 0x7fe28c0dffe0 2024-04-25T10:05:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29a638640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe28c0dffe0 0x7fe28c0d9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fa572ffd640 1 -- 172.21.15.60:0/3013931816 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa57801a070 con 0x7fa56c004810 2024-04-25T10:05:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29a638640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe28c0dffe0 0x7fe28c0d9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:37418/0 (socket says 172.21.15.60:37418) 2024-04-25T10:05:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29a638640 1 -- 172.21.15.60:0/941350858 learned_addr learned my addr 172.21.15.60:0/941350858 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7f87f8ff9640 1 -- 172.21.15.60:0/3371663290 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87f40073e0 con 0x7f87fc146430 2024-04-25T10:05:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe299636640 1 --2- 172.21.15.60:0/941350858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe28c051130 0x7fe28c04ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe299e37640 1 --2- 172.21.15.60:0/941350858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe28c038e10 0x7fe28c0dc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fa572ffd640 1 --2- 172.21.15.60:0/3013931816 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 0x7fa564069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7f87f8ff9640 1 --2- 172.21.15.60:0/3371663290 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87e8066f60 0x7f87e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fa572ffd640 1 -- 172.21.15.60:0/3013931816 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5780841f0 con 0x7fa56c004810 2024-04-25T10:05:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7f87f8ff9640 1 -- 172.21.15.60:0/3371663290 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87f4084400 con 0x7f87fc146430 2024-04-25T10:05:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29a638640 1 -- 172.21.15.60:0/941350858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe28c038e10 msgr2=0x7fe28c0dc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29a638640 1 --2- 172.21.15.60:0/941350858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe28c038e10 0x7fe28c0dc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29a638640 1 -- 172.21.15.60:0/941350858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe28c051130 msgr2=0x7fe28c04ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29a638640 1 --2- 172.21.15.60:0/941350858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe28c051130 0x7fe28c04ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29a638640 1 -- 172.21.15.60:0/941350858 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c04c290 con 0x7fe28c0dffe0 2024-04-25T10:05:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fa581d9b640 1 --2- 172.21.15.60:0/3013931816 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 0x7fa564069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29a638640 1 --2- 172.21.15.60:0/941350858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe28c0dffe0 0x7fe28c0d9a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe28400b410 tx=0x7fe28400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe28a7fc640 1 -- 172.21.15.60:0/941350858 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284015020 con 0x7fe28c0dffe0 2024-04-25T10:05:28.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe28a7fc640 1 -- 172.21.15.60:0/941350858 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe284011830 con 0x7fe28c0dffe0 2024-04-25T10:05:28.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29ae39640 1 -- 172.21.15.60:0/941350858 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe28c035510 con 0x7fe28c0dffe0 2024-04-25T10:05:28.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe28a7fc640 1 -- 172.21.15.60:0/941350858 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284011c20 con 0x7fe28c0dffe0 2024-04-25T10:05:28.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fe29ae39640 1 -- 172.21.15.60:0/941350858 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28c04bfa0 con 0x7fe28c0dffe0 2024-04-25T10:05:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.691+0000 7fa581d9b640 1 --2- 172.21.15.60:0/3013931816 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 0x7fa564069420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa57401ede0 tx=0x7fa574004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.695+0000 7f87fbfff640 1 --2- 172.21.15.60:0/3371663290 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87e8066f60 0x7f87e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.695+0000 7fe29ae39640 1 -- 172.21.15.60:0/941350858 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe28c0057f0 con 0x7fe28c0dffe0 2024-04-25T10:05:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.695+0000 7f87fbfff640 1 --2- 172.21.15.60:0/3371663290 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87e8066f60 0x7f87e8069420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87f00b2000 tx=0x7f87f00aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.695+0000 7fe28a7fc640 1 -- 172.21.15.60:0/941350858 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe284018020 con 0x7fe28c0dffe0 2024-04-25T10:05:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.695+0000 7fe28a7fc640 1 --2- 172.21.15.60:0/941350858 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe278066f60 0x7fe278069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.695+0000 7fe28a7fc640 1 -- 172.21.15.60:0/941350858 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe284085350 con 0x7fe28c0dffe0 2024-04-25T10:05:28.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.695+0000 7fe299e37640 1 --2- 172.21.15.60:0/941350858 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe278066f60 0x7fe278069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.695+0000 7fe299e37640 1 --2- 172.21.15.60:0/941350858 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe278066f60 0x7fe278069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe28c035440 tx=0x7fe294073bb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.695+0000 7f87f8ff9640 1 -- 172.21.15.60:0/3371663290 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87f4001010 con 0x7f87fc146430 2024-04-25T10:05:28.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.699+0000 7f362ffff640 1 -- 172.21.15.60:0/3628356002 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f363c01f020 con 0x7f36480b14a0 2024-04-25T10:05:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.699+0000 7f3cfaffd640 1 -- 172.21.15.60:0/1222464097 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc1164f0 con 0x7f3cf4109b70 2024-04-25T10:05:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.699+0000 7f3cfaffd640 1 -- 172.21.15.60:0/1222464097 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cfc116690 con 0x7f3cf4109b70 2024-04-25T10:05:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.699+0000 7f3d02939640 1 -- 172.21.15.60:0/1222464097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cf4109b70 msgr2=0x7f3cf410bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.699+0000 7f3d02939640 1 --2- 172.21.15.60:0/1222464097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cf4109b70 0x7f3cf410bf60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4006020 tx=0x7f3cfc088870 comp rx=0 tx=0).stop 2024-04-25T10:05:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7f3d02939640 1 -- 172.21.15.60:0/1222464097 shutdown_connections 2024-04-25T10:05:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7f3d02939640 1 --2- 172.21.15.60:0/1222464097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cf4109b70 0x7f3cf410bf60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7f3d02939640 1 -- 172.21.15.60:0/1222464097 >> 172.21.15.60:0/1222464097 conn(0x7f3cf4009fc0 msgr2=0x7f3cf400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7f3d02939640 1 -- 172.21.15.60:0/1222464097 shutdown_connections 2024-04-25T10:05:28.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7f3d02939640 1 -- 172.21.15.60:0/1222464097 wait complete. 2024-04-25T10:05:28.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7fbed67fc640 1 -- 172.21.15.60:0/54693008 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbedc0c0030 con 0x7fbee00aec30 2024-04-25T10:05:28.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7fbed67fc640 1 --2- 172.21.15.60:0/54693008 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbec4066f60 0x7fbec4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7fbed67fc640 1 -- 172.21.15.60:0/54693008 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbedc11ee50 con 0x7fbee00aec30 2024-04-25T10:05:28.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7fbee5230640 1 --2- 172.21.15.60:0/54693008 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbec4066f60 0x7fbec4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7f3d02939640 1 Processor -- start 2024-04-25T10:05:28.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7fbee5230640 1 --2- 172.21.15.60:0/54693008 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbec4066f60 0x7fbec4069420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbee0074450 tx=0x7fbed00079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7f3d02939640 1 -- start start 2024-04-25T10:05:28.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7f3d02939640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cf404f3e0 0x7f3cf40f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7f3d02939640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3cf404f7c0 0x7f3cf40fe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7f3d02939640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cf4109b70 0x7f3cf404d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7f3cfbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cf404f3e0 0x7f3cf40f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7f3cfbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cf404f3e0 0x7f3cf40f0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:39306/0 (socket says 172.21.15.60:39306) 2024-04-25T10:05:28.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7f3cfbfff640 1 -- 172.21.15.60:0/2806913261 learned_addr learned my addr 172.21.15.60:0/2806913261 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:28.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.703+0000 7f3cfb7fe640 1 --2- 172.21.15.60:0/2806913261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3cf404f7c0 0x7f3cf40fe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7f3d00ed6640 1 --2- 172.21.15.60:0/2806913261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cf4109b70 0x7f3cf404d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7f3d02939640 1 -- 172.21.15.60:0/2806913261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf410eed0 con 0x7f3cf4109b70 2024-04-25T10:05:28.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7f3d02939640 1 -- 172.21.15.60:0/2806913261 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf410f050 con 0x7f3cf404f3e0 2024-04-25T10:05:28.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7f3d02939640 1 -- 172.21.15.60:0/2806913261 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf410f1d0 con 0x7f3cf404f7c0 2024-04-25T10:05:28.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7f3d00ed6640 1 -- 172.21.15.60:0/2806913261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3cf404f7c0 msgr2=0x7f3cf40fe8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7f3d00ed6640 1 --2- 172.21.15.60:0/2806913261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3cf404f7c0 0x7f3cf40fe8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7f3d00ed6640 1 -- 172.21.15.60:0/2806913261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cf404f3e0 msgr2=0x7f3cf40f0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7f3d00ed6640 1 --2- 172.21.15.60:0/2806913261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cf404f3e0 0x7f3cf40f0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7f3d00ed6640 1 -- 172.21.15.60:0/2806913261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf400a3a0 con 0x7f3cf4109b70 2024-04-25T10:05:28.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7f3d00ed6640 1 --2- 172.21.15.60:0/2806913261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cf4109b70 0x7f3cf404d2a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc069de0 tx=0x7f3cfc0870b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7f3cf8ff9640 1 -- 172.21.15.60:0/2806913261 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc066c70 con 0x7f3cf4109b70 2024-04-25T10:05:28.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7f3d02939640 1 -- 172.21.15.60:0/2806913261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf4034a80 con 0x7f3cf4109b70 2024-04-25T10:05:28.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7f3d02939640 1 -- 172.21.15.60:0/2806913261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf4035040 con 0x7f3cf4109b70 2024-04-25T10:05:28.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7fa572ffd640 1 -- 172.21.15.60:0/3013931816 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa578054c40 con 0x7fa56c004810 2024-04-25T10:05:28.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7fe28a7fc640 1 -- 172.21.15.60:0/941350858 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe28401f020 con 0x7fe28c0dffe0 2024-04-25T10:05:28.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7f3cf8ff9640 1 -- 172.21.15.60:0/2806913261 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cfc066e10 con 0x7f3cf4109b70 2024-04-25T10:05:28.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7f3cf8ff9640 1 -- 172.21.15.60:0/2806913261 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc067040 con 0x7f3cf4109b70 2024-04-25T10:05:28.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.707+0000 7f3ce67fc640 1 -- 172.21.15.60:0/2806913261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cdc0052a0 con 0x7f3cf4109b70 2024-04-25T10:05:28.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.715+0000 7fbed67fc640 1 -- 172.21.15.60:0/54693008 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbedc0c7060 con 0x7fbee00aec30 2024-04-25T10:05:28.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.723+0000 7f3cf8ff9640 1 -- 172.21.15.60:0/2806913261 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3cfc125030 con 0x7f3cf4109b70 2024-04-25T10:05:28.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.723+0000 7f3cf8ff9640 1 --2- 172.21.15.60:0/2806913261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3cec067400 0x7f3cec0698c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.723+0000 7f3cf8ff9640 1 -- 172.21.15.60:0/2806913261 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3cfc188ae0 con 0x7f3cf4109b70 2024-04-25T10:05:28.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.723+0000 7f3cfbfff640 1 --2- 172.21.15.60:0/2806913261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3cec067400 0x7f3cec0698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.723+0000 7f3cf8ff9640 1 -- 172.21.15.60:0/2806913261 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3cfc12c020 con 0x7f3cf4109b70 2024-04-25T10:05:28.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.731+0000 7f3cfbfff640 1 --2- 172.21.15.60:0/2806913261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3cec067400 0x7f3cec0698c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00011f0 tx=0x7f3cf0007e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.987+0000 7f3a80529640 1 -- 172.21.15.60:0/4199027122 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3a60005590 con 0x7f3a780b3ca0 2024-04-25T10:05:28.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.987+0000 7f3a6effd640 1 -- 172.21.15.60:0/4199027122 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a68058ee0 con 0x7f3a780b3ca0 2024-04-25T10:05:28.994 INFO:teuthology.orchestra.run.smithi060.stdout:60129542149 2024-04-25T10:05:29.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.995+0000 7f3a6cff9640 1 -- 172.21.15.60:0/4199027122 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a64067400 msgr2=0x7f3a640698c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.995+0000 7f3a6cff9640 1 --2- 172.21.15.60:0/4199027122 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a64067400 0x7f3a640698c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a700af000 tx=0x7f3a700a7070 comp rx=0 tx=0).stop 2024-04-25T10:05:29.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.995+0000 7f3a6cff9640 1 -- 172.21.15.60:0/4199027122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a780b3ca0 msgr2=0x7f3a781595e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.995+0000 7f3a6cff9640 1 --2- 172.21.15.60:0/4199027122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a780b3ca0 0x7f3a781595e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800c780 tx=0x7f3a6800cc40 comp rx=0 tx=0).stop 2024-04-25T10:05:29.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.995+0000 7f3a6cff9640 1 -- 172.21.15.60:0/4199027122 shutdown_connections 2024-04-25T10:05:29.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.995+0000 7f3a6cff9640 1 --2- 172.21.15.60:0/4199027122 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a64067400 0x7f3a640698c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.995+0000 7f3a6cff9640 1 --2- 172.21.15.60:0/4199027122 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a780cefc0 0x7f3a7813a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.995+0000 7f3a6cff9640 1 --2- 172.21.15.60:0/4199027122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a780cbeb0 0x7f3a7813e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.995+0000 7f3a6cff9640 1 --2- 172.21.15.60:0/4199027122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a780b3ca0 0x7f3a781595e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.995+0000 7f3a6cff9640 1 -- 172.21.15.60:0/4199027122 >> 172.21.15.60:0/4199027122 conn(0x7f3a7806b4c0 msgr2=0x7f3a780728e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.995+0000 7f3a6cff9640 1 -- 172.21.15.60:0/4199027122 shutdown_connections 2024-04-25T10:05:29.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:28.995+0000 7f3a6cff9640 1 -- 172.21.15.60:0/4199027122 wait complete. 2024-04-25T10:05:29.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.011+0000 7f1a9a520640 1 -- 172.21.15.60:0/1480195413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1a940b3620 con 0x7f1a9406ef20 2024-04-25T10:05:29.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.011+0000 7f8801003640 1 -- 172.21.15.60:0/3371663290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f87fc072770 con 0x7f87fc146430 2024-04-25T10:05:29.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.011+0000 7f1a90ff9640 1 -- 172.21.15.60:0/1480195413 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1a840eff60 con 0x7f1a9406ef20 2024-04-25T10:05:29.019 INFO:teuthology.orchestra.run.smithi060.stdout:60129542150 2024-04-25T10:05:29.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.011+0000 7f87f8ff9640 1 -- 172.21.15.60:0/3371663290 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f87f405b080 con 0x7f87fc146430 2024-04-25T10:05:29.019 INFO:teuthology.orchestra.run.smithi060.stdout:64424509446 2024-04-25T10:05:29.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f8801003640 1 -- 172.21.15.60:0/3371663290 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87e8066f60 msgr2=0x7f87e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f8801003640 1 --2- 172.21.15.60:0/3371663290 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87e8066f60 0x7f87e8069420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87f00b2000 tx=0x7f87f00aa070 comp rx=0 tx=0).stop 2024-04-25T10:05:29.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f8801003640 1 -- 172.21.15.60:0/3371663290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87fc146430 msgr2=0x7f87fc09a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f8801003640 1 --2- 172.21.15.60:0/3371663290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87fc146430 0x7f87fc09a9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f87f400c390 tx=0x7f87f400c850 comp rx=0 tx=0).stop 2024-04-25T10:05:29.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f8801003640 1 -- 172.21.15.60:0/3371663290 shutdown_connections 2024-04-25T10:05:29.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f8801003640 1 --2- 172.21.15.60:0/3371663290 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87e8066f60 0x7f87e8069420 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f8801003640 1 --2- 172.21.15.60:0/3371663290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87fc146430 0x7f87fc09a9c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f8801003640 1 --2- 172.21.15.60:0/3371663290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87fc0b19f0 0x7f87fc09a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f8801003640 1 --2- 172.21.15.60:0/3371663290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87fc0afcd0 0x7f87fc0b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f8801003640 1 -- 172.21.15.60:0/3371663290 >> 172.21.15.60:0/3371663290 conn(0x7f87fc0b6120 msgr2=0x7f87fc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.3 2024-04-25T10:05:29.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f8801003640 1 -- 172.21.15.60:0/3371663290 shutdown_connections 2024-04-25T10:05:29.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f1a9a520640 1 -- 172.21.15.60:0/1480195413 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1a80067030 msgr2=0x7f1a800694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f1a9a520640 1 --2- 172.21.15.60:0/1480195413 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1a80067030 0x7f1a800694f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1a940acd00 tx=0x7f1a8c00b040 comp rx=0 tx=0).stop 2024-04-25T10:05:29.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f1a9a520640 1 -- 172.21.15.60:0/1480195413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a9406ef20 msgr2=0x7f1a94130650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f1a9a520640 1 --2- 172.21.15.60:0/1480195413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a9406ef20 0x7f1a94130650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1a9406bf70 tx=0x7f1a8401d770 comp rx=0 tx=0).stop 2024-04-25T10:05:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f8801003640 1 -- 172.21.15.60:0/3371663290 wait complete. 2024-04-25T10:05:29.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f1a9a520640 1 -- 172.21.15.60:0/1480195413 shutdown_connections 2024-04-25T10:05:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f1a9a520640 1 --2- 172.21.15.60:0/1480195413 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1a80067030 0x7f1a800694f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f1a9a520640 1 --2- 172.21.15.60:0/1480195413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a94131db0 0x7f1a940b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f1a9a520640 1 --2- 172.21.15.60:0/1480195413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a940cf100 0x7f1a94133160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f1a9a520640 1 --2- 172.21.15.60:0/1480195413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a9406ef20 0x7f1a94130650 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f1a9a520640 1 -- 172.21.15.60:0/1480195413 >> 172.21.15.60:0/1480195413 conn(0x7f1a940b12d0 msgr2=0x7f1a940cb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f1a9a520640 1 -- 172.21.15.60:0/1480195413 shutdown_connections 2024-04-25T10:05:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.015+0000 7f1a9a520640 1 -- 172.21.15.60:0/1480195413 wait complete. 2024-04-25T10:05:29.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.019+0000 7f364e1ca640 1 -- 172.21.15.60:0/3628356002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3628005860 con 0x7f36480b14a0 2024-04-25T10:05:29.027 INFO:teuthology.orchestra.run.smithi060.stdout:60129542148 2024-04-25T10:05:29.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.019+0000 7f362ffff640 1 -- 172.21.15.60:0/3628356002 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f363c0564e0 con 0x7f36480b14a0 2024-04-25T10:05:29.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.023+0000 7f364e1ca640 1 -- 172.21.15.60:0/3628356002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3634066f60 msgr2=0x7f3634069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.023+0000 7f364e1ca640 1 --2- 172.21.15.60:0/3628356002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3634066f60 0x7f3634069420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f36380ae000 tx=0x7f36380a7000 comp rx=0 tx=0).stop 2024-04-25T10:05:29.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.023+0000 7f364e1ca640 1 -- 172.21.15.60:0/3628356002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36480b14a0 msgr2=0x7f364815a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.023+0000 7f364e1ca640 1 --2- 172.21.15.60:0/3628356002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36480b14a0 0x7f364815a2b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f363c00b3e0 tx=0x7f363c00b8a0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.023+0000 7f364e1ca640 1 -- 172.21.15.60:0/3628356002 shutdown_connections 2024-04-25T10:05:29.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.023+0000 7f364e1ca640 1 --2- 172.21.15.60:0/3628356002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3634066f60 0x7f3634069420 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.023+0000 7f364e1ca640 1 --2- 172.21.15.60:0/3628356002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3648148fe0 0x7f3648159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.023+0000 7f364e1ca640 1 --2- 172.21.15.60:0/3628356002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36481583d0 0x7f36481fec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.023+0000 7f364e1ca640 1 --2- 172.21.15.60:0/3628356002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36480b14a0 0x7f364815a2b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.023+0000 7f364e1ca640 1 -- 172.21.15.60:0/3628356002 >> 172.21.15.60:0/3628356002 conn(0x7f3648072920 msgr2=0x7f36480d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.023+0000 7f364e1ca640 1 -- 172.21.15.60:0/3628356002 shutdown_connections 2024-04-25T10:05:29.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.023+0000 7f364e1ca640 1 -- 172.21.15.60:0/3628356002 wait complete. 2024-04-25T10:05:29.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.035+0000 7fe29ae39640 1 -- 172.21.15.60:0/941350858 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe28c053960 con 0x7fe28c0dffe0 2024-04-25T10:05:29.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.035+0000 7fe28a7fc640 1 -- 172.21.15.60:0/941350858 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe284056440 con 0x7fe28c0dffe0 2024-04-25T10:05:29.041 INFO:teuthology.orchestra.run.smithi060.stdout:60129542149 2024-04-25T10:05:29.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.039+0000 7fe29ae39640 1 -- 172.21.15.60:0/941350858 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe278066f60 msgr2=0x7fe278069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.039+0000 7fe29ae39640 1 --2- 172.21.15.60:0/941350858 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe278066f60 0x7fe278069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe28c035440 tx=0x7fe294073bb0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.039+0000 7fe29ae39640 1 -- 172.21.15.60:0/941350858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe28c0dffe0 msgr2=0x7fe28c0d9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.039+0000 7fe29ae39640 1 --2- 172.21.15.60:0/941350858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe28c0dffe0 0x7fe28c0d9a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe28400b410 tx=0x7fe28400b8d0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.039+0000 7fe29ae39640 1 -- 172.21.15.60:0/941350858 shutdown_connections 2024-04-25T10:05:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.039+0000 7fe29ae39640 1 --2- 172.21.15.60:0/941350858 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe278066f60 0x7fe278069420 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.039+0000 7fe29ae39640 1 --2- 172.21.15.60:0/941350858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe28c0dffe0 0x7fe28c0d9a30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.039+0000 7fe29ae39640 1 --2- 172.21.15.60:0/941350858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe28c051130 0x7fe28c04ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.039+0000 7fe29ae39640 1 --2- 172.21.15.60:0/941350858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe28c038e10 0x7fe28c0dc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.039+0000 7fe29ae39640 1 -- 172.21.15.60:0/941350858 >> 172.21.15.60:0/941350858 conn(0x7fe28c055be0 msgr2=0x7fe28c03b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.039+0000 7fe29ae39640 1 -- 172.21.15.60:0/941350858 shutdown_connections 2024-04-25T10:05:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.039+0000 7fe29ae39640 1 -- 172.21.15.60:0/941350858 wait complete. 2024-04-25T10:05:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.039+0000 7fa58359e640 1 -- 172.21.15.60:0/3013931816 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa5840ac290 con 0x7fa56c004810 2024-04-25T10:05:29.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.043+0000 7fa572ffd640 1 -- 172.21.15.60:0/3013931816 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa578058ce0 con 0x7fa56c004810 2024-04-25T10:05:29.048 INFO:teuthology.orchestra.run.smithi060.stdout:60129542149 2024-04-25T10:05:29.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2024-04-25T10:05:29.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.0 2024-04-25T10:05:29.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.043+0000 7fa58359e640 1 -- 172.21.15.60:0/3013931816 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 msgr2=0x7fa564069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.043+0000 7fa58359e640 1 --2- 172.21.15.60:0/3013931816 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 0x7fa564069420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa57401ede0 tx=0x7fa574004740 comp rx=0 tx=0).stop 2024-04-25T10:05:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.043+0000 7fa58359e640 1 -- 172.21.15.60:0/3013931816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa56c004810 msgr2=0x7fa584202ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.043+0000 7fa58359e640 1 --2- 172.21.15.60:0/3013931816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa56c004810 0x7fa584202ac0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa58415d2f0 tx=0x7fa57800c820 comp rx=0 tx=0).stop 2024-04-25T10:05:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.043+0000 7fa58359e640 1 -- 172.21.15.60:0/3013931816 shutdown_connections 2024-04-25T10:05:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.043+0000 7fa58359e640 1 --2- 172.21.15.60:0/3013931816 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 0x7fa564069420 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.043+0000 7fa58359e640 1 --2- 172.21.15.60:0/3013931816 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5840aa150 0x7fa5840aa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.043+0000 7fa58359e640 1 --2- 172.21.15.60:0/3013931816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5840a8ea0 0x7fa58415c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.043+0000 7fa58359e640 1 --2- 172.21.15.60:0/3013931816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa56c004810 0x7fa584202ac0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.043+0000 7fa58359e640 1 -- 172.21.15.60:0/3013931816 >> 172.21.15.60:0/3013931816 conn(0x7fa58406e870 msgr2=0x7fa58406ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.043+0000 7fa58359e640 1 -- 172.21.15.60:0/3013931816 shutdown_connections 2024-04-25T10:05:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.043+0000 7fa58359e640 1 -- 172.21.15.60:0/3013931816 wait complete. 2024-04-25T10:05:29.054 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.7 2024-04-25T10:05:29.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.051+0000 7fbee6a33640 1 -- 172.21.15.60:0/54693008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fbec8005860 con 0x7fbee00aec30 2024-04-25T10:05:29.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.051+0000 7fbed67fc640 1 -- 172.21.15.60:0/54693008 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbedc0160d0 con 0x7fbee00aec30 2024-04-25T10:05:29.058 INFO:teuthology.orchestra.run.smithi060.stdout:60129542149 2024-04-25T10:05:29.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.051+0000 7f3ce67fc640 1 -- 172.21.15.60:0/2806913261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f3cdc005860 con 0x7f3cf4109b70 2024-04-25T10:05:29.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.051+0000 7f3cf8ff9640 1 -- 172.21.15.60:0/2806913261 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3cfc0824c0 con 0x7f3cf4109b70 2024-04-25T10:05:29.059 INFO:teuthology.orchestra.run.smithi060.stdout:64424509445 2024-04-25T10:05:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7fbee6a33640 1 -- 172.21.15.60:0/54693008 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbec4066f60 msgr2=0x7fbec4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7fbee6a33640 1 --2- 172.21.15.60:0/54693008 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbec4066f60 0x7fbec4069420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbee0074450 tx=0x7fbed00079c0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7fbee6a33640 1 -- 172.21.15.60:0/54693008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbee00aec30 msgr2=0x7fbee0154730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7fbee6a33640 1 --2- 172.21.15.60:0/54693008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbee00aec30 0x7fbee0154730 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0b0040 tx=0x7fbedc01ac90 comp rx=0 tx=0).stop 2024-04-25T10:05:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7fbee6a33640 1 -- 172.21.15.60:0/54693008 shutdown_connections 2024-04-25T10:05:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7fbee6a33640 1 --2- 172.21.15.60:0/54693008 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbec4066f60 0x7fbec4069420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7fbee6a33640 1 --2- 172.21.15.60:0/54693008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbee0161ac0 0x7fbee0134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7fbee6a33640 1 --2- 172.21.15.60:0/54693008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbee00cec50 0x7fbee0133bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7fbee6a33640 1 --2- 172.21.15.60:0/54693008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbee00aec30 0x7fbee0154730 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7fbee6a33640 1 -- 172.21.15.60:0/54693008 >> 172.21.15.60:0/54693008 conn(0x7fbee006e870 msgr2=0x7fbee00b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7fbee6a33640 1 -- 172.21.15.60:0/54693008 shutdown_connections 2024-04-25T10:05:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7fbee6a33640 1 -- 172.21.15.60:0/54693008 wait complete. 2024-04-25T10:05:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7f3ce67fc640 1 -- 172.21.15.60:0/2806913261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3cec067400 msgr2=0x7f3cec0698c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7f3ce67fc640 1 --2- 172.21.15.60:0/2806913261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3cec067400 0x7f3cec0698c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00011f0 tx=0x7f3cf0007e00 comp rx=0 tx=0).stop 2024-04-25T10:05:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7f3ce67fc640 1 -- 172.21.15.60:0/2806913261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cf4109b70 msgr2=0x7f3cf404d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7f3ce67fc640 1 --2- 172.21.15.60:0/2806913261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cf4109b70 0x7f3cf404d2a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc069de0 tx=0x7f3cfc0870b0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7f3ce67fc640 1 -- 172.21.15.60:0/2806913261 shutdown_connections 2024-04-25T10:05:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7f3ce67fc640 1 --2- 172.21.15.60:0/2806913261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3cec067400 0x7f3cec0698c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7f3ce67fc640 1 --2- 172.21.15.60:0/2806913261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cf4109b70 0x7f3cf404d2a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7f3ce67fc640 1 --2- 172.21.15.60:0/2806913261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3cf404f7c0 0x7f3cf40fe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7f3ce67fc640 1 --2- 172.21.15.60:0/2806913261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cf404f3e0 0x7f3cf40f0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7f3ce67fc640 1 -- 172.21.15.60:0/2806913261 >> 172.21.15.60:0/2806913261 conn(0x7f3cf4009fc0 msgr2=0x7f3cf404cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7f3ce67fc640 1 -- 172.21.15.60:0/2806913261 shutdown_connections 2024-04-25T10:05:29.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:29.055+0000 7f3ce67fc640 1 -- 172.21.15.60:0/2806913261 wait complete. 2024-04-25T10:05:29.068 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.2 2024-04-25T10:05:29.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.1 2024-04-25T10:05:29.081 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.5 2024-04-25T10:05:29.085 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2024-04-25T10:05:30.025 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:05:30.051 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:05:30.052 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:05:30.055 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:05:30.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:05:30.075 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:05:30.082 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:05:30.087 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:05:30.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.199+0000 7ff940d8c640 1 Processor -- start 2024-04-25T10:05:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.199+0000 7ff940d8c640 1 -- start start 2024-04-25T10:05:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.199+0000 7ff940d8c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff93c0cefc0 0x7ff93c0b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.199+0000 7ff940d8c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff93c0b3ca0 0x7ff93c0cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.199+0000 7ff940d8c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93c0af860 con 0x7ff93c0cbeb0 2024-04-25T10:05:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.199+0000 7ff940d8c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93c1433d0 con 0x7ff93c0cefc0 2024-04-25T10:05:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.199+0000 7ff940d8c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93c142960 con 0x7ff93c0b3ca0 2024-04-25T10:05:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.199+0000 7ff939d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff93c0cefc0 0x7ff93c0b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.199+0000 7ff939d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff93c0cefc0 0x7ff93c0b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46398/0 (socket says 172.21.15.60:46398) 2024-04-25T10:05:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.199+0000 7ff939d9b640 1 -- 172.21.15.60:0/1114980874 learned_addr learned my addr 172.21.15.60:0/1114980874 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.199+0000 7ff93ad9d640 1 --2- 172.21.15.60:0/1114980874 >> v2:172.21.15.100:3300/0 conn(0x7ff93c0b3ca0 0x7ff93c0cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.199+0000 7ff939d9b640 1 -- 172.21.15.60:0/1114980874 >> v2:172.21.15.100:3300/0 conn(0x7ff93c0b3ca0 msgr2=0x7ff93c0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.199+0000 7ff939d9b640 1 --2- 172.21.15.60:0/1114980874 >> v2:172.21.15.100:3300/0 conn(0x7ff93c0b3ca0 0x7ff93c0cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.199+0000 7ff939d9b640 1 -- 172.21.15.60:0/1114980874 >> v1:172.21.15.60:6789/0 conn(0x7ff93c0cbeb0 legacy=0x7ff93c0af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.199+0000 7ff939d9b640 1 -- 172.21.15.60:0/1114980874 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c0b26c0 con 0x7ff93c0cefc0 2024-04-25T10:05:30.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.203+0000 7ff939d9b640 1 --2- 172.21.15.60:0/1114980874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff93c0cefc0 0x7ff93c0b3760 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0b23b0 tx=0x7ff93401ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b764f03e2a2036e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.203+0000 7ff93959a640 1 -- 172.21.15.60:0/1114980874 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9340ad070 con 0x7ff93c0cefc0 2024-04-25T10:05:30.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.207+0000 7ff93959a640 1 -- 172.21.15.60:0/1114980874 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9340a7b60 con 0x7ff93c0cefc0 2024-04-25T10:05:30.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.207+0000 7ff940d8c640 1 -- 172.21.15.60:0/1114980874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff93c0cefc0 msgr2=0x7ff93c0b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.207+0000 7ff940d8c640 1 --2- 172.21.15.60:0/1114980874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff93c0cefc0 0x7ff93c0b3760 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0b23b0 tx=0x7ff93401ec20 comp rx=0 tx=0).stop 2024-04-25T10:05:30.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.207+0000 7ff940d8c640 1 -- 172.21.15.60:0/1114980874 shutdown_connections 2024-04-25T10:05:30.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.207+0000 7ff940d8c640 1 --2- 172.21.15.60:0/1114980874 >> v2:172.21.15.100:3300/0 conn(0x7ff93c0b3ca0 0x7ff93c0cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.207+0000 7ff940d8c640 1 --2- 172.21.15.60:0/1114980874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff93c0cefc0 0x7ff93c0b3760 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.207+0000 7ff940d8c640 1 -- 172.21.15.60:0/1114980874 >> 172.21.15.60:0/1114980874 conn(0x7ff93c06b4c0 msgr2=0x7ff93c06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.207+0000 7ff940d8c640 1 -- 172.21.15.60:0/1114980874 shutdown_connections 2024-04-25T10:05:30.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.207+0000 7ff940d8c640 1 -- 172.21.15.60:0/1114980874 wait complete. 2024-04-25T10:05:30.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.207+0000 7ff940d8c640 1 Processor -- start 2024-04-25T10:05:30.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.207+0000 7ff940d8c640 1 -- start start 2024-04-25T10:05:30.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff940d8c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff93c0b3ca0 0x7ff93c159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff940d8c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff93c0cbeb0 0x7ff93c13d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff940d8c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff93c13de20 0x7ff93c139eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff940d8c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c0cc290 con 0x7ff93c0cbeb0 2024-04-25T10:05:30.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff940d8c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c0b3240 con 0x7ff93c13de20 2024-04-25T10:05:30.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff940d8c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c166570 con 0x7ff93c0b3ca0 2024-04-25T10:05:30.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff939d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff93c0cbeb0 0x7ff93c13d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff939d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff93c0cbeb0 0x7ff93c13d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46412/0 (socket says 172.21.15.60:46412) 2024-04-25T10:05:30.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff939d9b640 1 -- 172.21.15.60:0/538658678 learned_addr learned my addr 172.21.15.60:0/538658678 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:30.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff93a59c640 1 --2- 172.21.15.60:0/538658678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff93c0b3ca0 0x7ff93c159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff93ad9d640 1 --2- 172.21.15.60:0/538658678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff93c13de20 0x7ff93c139eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff939d9b640 1 -- 172.21.15.60:0/538658678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff93c0b3ca0 msgr2=0x7ff93c159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff939d9b640 1 --2- 172.21.15.60:0/538658678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff93c0b3ca0 0x7ff93c159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff939d9b640 1 -- 172.21.15.60:0/538658678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff93c13de20 msgr2=0x7ff93c139eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff939d9b640 1 --2- 172.21.15.60:0/538658678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff93c13de20 0x7ff93c139eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff939d9b640 1 -- 172.21.15.60:0/538658678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c13e2c0 con 0x7ff93c0cbeb0 2024-04-25T10:05:30.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff939d9b640 1 --2- 172.21.15.60:0/538658678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff93c0cbeb0 0x7ff93c13d8e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff93401b830 tx=0x7ff93401b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff92affd640 1 -- 172.21.15.60:0/538658678 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9340ad040 con 0x7ff93c0cbeb0 2024-04-25T10:05:30.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff940d8c640 1 -- 172.21.15.60:0/538658678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93c13e5a0 con 0x7ff93c0cbeb0 2024-04-25T10:05:30.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff940d8c640 1 -- 172.21.15.60:0/538658678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93c13eb60 con 0x7ff93c0cbeb0 2024-04-25T10:05:30.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.211+0000 7ff92affd640 1 -- 172.21.15.60:0/538658678 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9340a7cb0 con 0x7ff93c0cbeb0 2024-04-25T10:05:30.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.215+0000 7ff92affd640 1 -- 172.21.15.60:0/538658678 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934008040 con 0x7ff93c0cbeb0 2024-04-25T10:05:30.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.215+0000 7ff92affd640 1 -- 172.21.15.60:0/538658678 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9340c2070 con 0x7ff93c0cbeb0 2024-04-25T10:05:30.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.215+0000 7ff940d8c640 1 -- 172.21.15.60:0/538658678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9200052a0 con 0x7ff93c0cbeb0 2024-04-25T10:05:30.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.215+0000 7ff92affd640 1 --2- 172.21.15.60:0/538658678 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff91c066f90 0x7ff91c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.215+0000 7ff92affd640 1 -- 172.21.15.60:0/538658678 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff93411f220 con 0x7ff93c0cbeb0 2024-04-25T10:05:30.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.215+0000 7ff93a59c640 1 --2- 172.21.15.60:0/538658678 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff91c066f90 0x7ff91c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.219+0000 7ff93a59c640 1 --2- 172.21.15.60:0/538658678 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff91c066f90 0x7ff91c069450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff930000c10 tx=0x7ff930009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.223+0000 7ff92affd640 1 -- 172.21.15.60:0/538658678 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff9340188a0 con 0x7ff93c0cbeb0 2024-04-25T10:05:30.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f3a97bd8640 1 Processor -- start 2024-04-25T10:05:30.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f7ae8055640 1 Processor -- start 2024-04-25T10:05:30.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f3a97bd8640 1 -- start start 2024-04-25T10:05:30.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f7ae8055640 1 -- start start 2024-04-25T10:05:30.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f3a97bd8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a90140220 0x7f3a9006e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f3a97bd8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3a900719a0 0x7f3a9015c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f3a97bd8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9015b260 con 0x7f3a900cf100 2024-04-25T10:05:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f3a97bd8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9015be70 con 0x7f3a900719a0 2024-04-25T10:05:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f3a97bd8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9015b540 con 0x7f3a90140220 2024-04-25T10:05:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f3a95974640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a90140220 0x7f3a9006e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f3a96175640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3a900719a0 0x7f3a9015c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f3a95974640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a90140220 0x7f3a9006e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:51624/0 (socket says 172.21.15.60:51624) 2024-04-25T10:05:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f3a95974640 1 -- 172.21.15.60:0/4062862394 learned_addr learned my addr 172.21.15.60:0/4062862394 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:30.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f7ae8055640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7ae015f480 0x7f7ae015f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f7ae8055640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ae00cd390 0x7f7ae009a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f7ae8055640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae015fda0 con 0x7f7ae009a930 2024-04-25T10:05:30.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f7ae8055640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae00cc5d0 con 0x7f7ae015f480 2024-04-25T10:05:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f7ae8055640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae00cc8b0 con 0x7f7ae00cd390 2024-04-25T10:05:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f7ae65f2640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7ae009a930 0x7f7ae00cf380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37252/0 (socket says 172.21.15.60:37252) 2024-04-25T10:05:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f7ae65f2640 1 -- 172.21.15.60:0/1571590219 learned_addr learned my addr 172.21.15.60:0/1571590219 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f7ae5df1640 1 --2- 172.21.15.60:0/1571590219 >> v2:172.21.15.60:3300/0 conn(0x7f7ae015f480 0x7f7ae015f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f7ae55f0640 1 --2- 172.21.15.60:0/1571590219 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ae00cd390 0x7f7ae009a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f3a95974640 1 -- 172.21.15.60:0/4062862394 >> v2:172.21.15.100:3300/0 conn(0x7f3a900719a0 msgr2=0x7f3a9015c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f3a95974640 1 --2- 172.21.15.60:0/4062862394 >> v2:172.21.15.100:3300/0 conn(0x7f3a900719a0 0x7f3a9015c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f3a95974640 1 -- 172.21.15.60:0/4062862394 >> v1:172.21.15.100:6789/0 conn(0x7f3a900cf100 legacy=0x7f3a90071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.235+0000 7f3a95974640 1 -- 172.21.15.60:0/4062862394 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9015b820 con 0x7f3a90140220 2024-04-25T10:05:30.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f7ae5df1640 1 -- 172.21.15.60:0/1571590219 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ae00cd390 msgr2=0x7f7ae009a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f7ae5df1640 1 --2- 172.21.15.60:0/1571590219 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ae00cd390 0x7f7ae009a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f7ae5df1640 1 -- 172.21.15.60:0/1571590219 >> v1:172.21.15.60:6789/0 conn(0x7f7ae009a930 legacy=0x7f7ae00cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f7ae5df1640 1 -- 172.21.15.60:0/1571590219 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae00ccb90 con 0x7f7ae015f480 2024-04-25T10:05:30.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f7ae5df1640 1 --2- 172.21.15.60:0/1571590219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae015f480 0x7f7ae015f860 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0097c0 tx=0x7f7adc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7176f136aa0e5ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f7ae4def640 1 -- 172.21.15.60:0/1571590219 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0ab610 con 0x7f7ae015f480 2024-04-25T10:05:30.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f7ae4def640 1 -- 172.21.15.60:0/1571590219 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7adc0b0070 con 0x7f7ae015f480 2024-04-25T10:05:30.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f7ae4def640 1 -- 172.21.15.60:0/1571590219 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0aba10 con 0x7f7ae015f480 2024-04-25T10:05:30.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f3a95974640 1 --2- 172.21.15.60:0/4062862394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a90140220 0x7f3a9006e9e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800ce20 tx=0x7f3a8801ca70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f9251079fc2f6c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f3a94972640 1 -- 172.21.15.60:0/4062862394 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8800c2d0 con 0x7f3a90140220 2024-04-25T10:05:30.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f3a94972640 1 -- 172.21.15.60:0/4062862394 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a880a5920 con 0x7f3a90140220 2024-04-25T10:05:30.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f7ae8055640 1 -- 172.21.15.60:0/1571590219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae015f480 msgr2=0x7f7ae015f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f7ae8055640 1 --2- 172.21.15.60:0/1571590219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae015f480 0x7f7ae015f860 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0097c0 tx=0x7f7adc01c490 comp rx=0 tx=0).stop 2024-04-25T10:05:30.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f7ae8055640 1 -- 172.21.15.60:0/1571590219 shutdown_connections 2024-04-25T10:05:30.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f7ae8055640 1 --2- 172.21.15.60:0/1571590219 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ae00cd390 0x7f7ae009a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f7ae8055640 1 --2- 172.21.15.60:0/1571590219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae015f480 0x7f7ae015f860 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f7ae8055640 1 -- 172.21.15.60:0/1571590219 >> 172.21.15.60:0/1571590219 conn(0x7f7ae00b32a0 msgr2=0x7f7ae00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f7ae8055640 1 -- 172.21.15.60:0/1571590219 shutdown_connections 2024-04-25T10:05:30.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.239+0000 7f7ae8055640 1 -- 172.21.15.60:0/1571590219 wait complete. 2024-04-25T10:05:30.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f3a97bd8640 1 -- 172.21.15.60:0/4062862394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a90140220 msgr2=0x7f3a9006e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f3a97bd8640 1 --2- 172.21.15.60:0/4062862394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a90140220 0x7f3a9006e9e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800ce20 tx=0x7f3a8801ca70 comp rx=0 tx=0).stop 2024-04-25T10:05:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f3a97bd8640 1 -- 172.21.15.60:0/4062862394 shutdown_connections 2024-04-25T10:05:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f3a97bd8640 1 --2- 172.21.15.60:0/4062862394 >> v2:172.21.15.100:3300/0 conn(0x7f3a900719a0 0x7f3a9015c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f3a97bd8640 1 --2- 172.21.15.60:0/4062862394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a90140220 0x7f3a9006e9e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f3a97bd8640 1 -- 172.21.15.60:0/4062862394 >> 172.21.15.60:0/4062862394 conn(0x7f3a900b12d0 msgr2=0x7f3a900b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f3a97bd8640 1 -- 172.21.15.60:0/4062862394 shutdown_connections 2024-04-25T10:05:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae8055640 1 Processor -- start 2024-04-25T10:05:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae8055640 1 -- start start 2024-04-25T10:05:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae8055640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ae009a930 0x7f7ae00af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae8055640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae00cd390 0x7f7ae00b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae8055640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ae00b4b40 0x7f7ae00b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae8055640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae015cd40 con 0x7f7ae00cd390 2024-04-25T10:05:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae8055640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae015cbc0 con 0x7f7ae009a930 2024-04-25T10:05:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae8055640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae01687c0 con 0x7f7ae00b4b40 2024-04-25T10:05:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f3a97bd8640 1 -- 172.21.15.60:0/4062862394 wait complete. 2024-04-25T10:05:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f3a97bd8640 1 Processor -- start 2024-04-25T10:05:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae55f0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae00cd390 0x7f7ae00b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae55f0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae00cd390 0x7f7ae00b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46426/0 (socket says 172.21.15.60:46426) 2024-04-25T10:05:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae55f0640 1 -- 172.21.15.60:0/2413798676 learned_addr learned my addr 172.21.15.60:0/2413798676 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae65f2640 1 --2- 172.21.15.60:0/2413798676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ae00b4b40 0x7f7ae00b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae55f0640 1 -- 172.21.15.60:0/2413798676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ae00b4b40 msgr2=0x7f7ae00b6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae55f0640 1 --2- 172.21.15.60:0/2413798676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ae00b4b40 0x7f7ae00b6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae55f0640 1 -- 172.21.15.60:0/2413798676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ae009a930 msgr2=0x7f7ae00af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae55f0640 1 --2- 172.21.15.60:0/2413798676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ae009a930 0x7f7ae00af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae55f0640 1 -- 172.21.15.60:0/2413798676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae00afa80 con 0x7f7ae00cd390 2024-04-25T10:05:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae55f0640 1 --2- 172.21.15.60:0/2413798676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae00cd390 0x7f7ae00b4600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000e930 tx=0x7f7ad000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ad67fc640 1 -- 172.21.15.60:0/2413798676 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0020070 con 0x7f7ae00cd390 2024-04-25T10:05:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f3a97bd8640 1 -- start start 2024-04-25T10:05:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f3a97bd8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a900719a0 0x7f3a901588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f3a97bd8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a900cf100 0x7f3a90153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f3a97bd8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a90140220 0x7f3a90159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae8055640 1 -- 172.21.15.60:0/2413798676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae00afd30 con 0x7f7ae00cd390 2024-04-25T10:05:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.243+0000 7f7ae8055640 1 -- 172.21.15.60:0/2413798676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae01593f0 con 0x7f7ae00cd390 2024-04-25T10:05:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f7ad67fc640 1 -- 172.21.15.60:0/2413798676 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad000ca00 con 0x7f7ae00cd390 2024-04-25T10:05:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a95974640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a900719a0 0x7f3a901588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a95974640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a900719a0 0x7f3a901588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:45968/0 (socket says 172.21.15.60:45968) 2024-04-25T10:05:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a95173640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a900cf100 0x7f3a90153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a95173640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a900cf100 0x7f3a90153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:51632/0 (socket says 172.21.15.60:51632) 2024-04-25T10:05:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a96175640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a90140220 0x7f3a90159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a96175640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a90140220 0x7f3a90159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46428/0 (socket says 172.21.15.60:46428) 2024-04-25T10:05:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a97bd8640 1 -- 172.21.15.60:0/2952863690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9015d3e0 con 0x7f3a90140220 2024-04-25T10:05:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a95974640 1 -- 172.21.15.60:0/2952863690 learned_addr learned my addr 172.21.15.60:0/2952863690 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a97bd8640 1 -- 172.21.15.60:0/2952863690 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9015d260 con 0x7f3a900719a0 2024-04-25T10:05:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a97bd8640 1 -- 172.21.15.60:0/2952863690 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a90166640 con 0x7f3a900cf100 2024-04-25T10:05:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f7ad67fc640 1 -- 172.21.15.60:0/2413798676 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad000cdb0 con 0x7f7ae00cd390 2024-04-25T10:05:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a95173640 1 -- 172.21.15.60:0/2952863690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a900719a0 msgr2=0x7f3a901588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a95173640 1 --2- 172.21.15.60:0/2952863690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a900719a0 0x7f3a901588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a95173640 1 -- 172.21.15.60:0/2952863690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a90140220 msgr2=0x7f3a90159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a95173640 1 --2- 172.21.15.60:0/2952863690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a90140220 0x7f3a90159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a95173640 1 -- 172.21.15.60:0/2952863690 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a901552f0 con 0x7f3a900cf100 2024-04-25T10:05:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a96175640 1 --2- 172.21.15.60:0/2952863690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a90140220 0x7f3a90159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:30.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f7ae8055640 1 -- 172.21.15.60:0/2413798676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac80052a0 con 0x7f7ae00cd390 2024-04-25T10:05:30.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a95173640 1 --2- 172.21.15.60:0/2952863690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a900cf100 0x7f3a90153550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c000f70 tx=0x7f3a8c00c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a867fc640 1 -- 172.21.15.60:0/2952863690 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c017070 con 0x7f3a900cf100 2024-04-25T10:05:30.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a867fc640 1 -- 172.21.15.60:0/2952863690 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8c0052d0 con 0x7f3a900cf100 2024-04-25T10:05:30.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a867fc640 1 -- 172.21.15.60:0/2952863690 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c003840 con 0x7f3a900cf100 2024-04-25T10:05:30.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a97bd8640 1 -- 172.21.15.60:0/2952863690 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a901555d0 con 0x7f3a900cf100 2024-04-25T10:05:30.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a97bd8640 1 -- 172.21.15.60:0/2952863690 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a90155b90 con 0x7f3a900cf100 2024-04-25T10:05:30.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.247+0000 7f3a97bd8640 1 -- 172.21.15.60:0/2952863690 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a90074770 con 0x7f3a900cf100 2024-04-25T10:05:30.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.251+0000 7f7ad67fc640 1 -- 172.21.15.60:0/2413798676 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7ad0023070 con 0x7f7ae00cd390 2024-04-25T10:05:30.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.251+0000 7f7ad67fc640 1 --2- 172.21.15.60:0/2413798676 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ac4067030 0x7f7ac40694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.251+0000 7f7ad67fc640 1 -- 172.21.15.60:0/2413798676 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ad0088dd0 con 0x7f7ae00cd390 2024-04-25T10:05:30.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.251+0000 7f7ae5df1640 1 --2- 172.21.15.60:0/2413798676 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ac4067030 0x7f7ac40694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.251+0000 7f3a867fc640 1 -- 172.21.15.60:0/2952863690 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a8c01a070 con 0x7f3a900cf100 2024-04-25T10:05:30.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.251+0000 7f7ae5df1640 1 --2- 172.21.15.60:0/2413798676 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ac4067030 0x7f7ac40694f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0af000 tx=0x7f7adc004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.251+0000 7f3a867fc640 1 --2- 172.21.15.60:0/2952863690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a74066f90 0x7f3a74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.251+0000 7f3a867fc640 1 -- 172.21.15.60:0/2952863690 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a8c084700 con 0x7f3a900cf100 2024-04-25T10:05:30.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.251+0000 7f3a95974640 1 --2- 172.21.15.60:0/2952863690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a74066f90 0x7f3a74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.251+0000 7f3a95974640 1 --2- 172.21.15.60:0/2952863690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a74066f90 0x7f3a74069450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3a8801b120 tx=0x7f3a88004ac0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7f7ad67fc640 1 -- 172.21.15.60:0/2413798676 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7ad008c030 con 0x7f7ae00cd390 2024-04-25T10:05:30.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d4472640 1 Processor -- start 2024-04-25T10:05:30.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d4472640 1 -- start start 2024-04-25T10:05:30.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d4472640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6cc161aa0 0x7fd6cc0b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d4472640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6cc0b3f60 con 0x7fd6cc0cd370 2024-04-25T10:05:30.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d4472640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6cc0af970 con 0x7fd6cc0b3900 2024-04-25T10:05:30.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d4472640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc0b4240 con 0x7fd6cc161aa0 2024-04-25T10:05:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d1a0d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd6cc0cd370 0x7fd6cc0b31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37256/0 (socket says 172.21.15.60:37256) 2024-04-25T10:05:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d1a0d640 1 -- 172.21.15.60:0/3345489392 learned_addr learned my addr 172.21.15.60:0/3345489392 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d120c640 1 -- 172.21.15.60:0/3345489392 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591567011 0 0) 0x7fd6cc0b3f60 con 0x7fd6cc0cd370 2024-04-25T10:05:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d120c640 1 -- 172.21.15.60:0/3345489392 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6b8003530 con 0x7fd6cc0cd370 2024-04-25T10:05:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d120c640 1 -- 172.21.15.60:0/3345489392 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2180292720 0 0) 0x7fd6b8003530 con 0x7fd6cc0cd370 2024-04-25T10:05:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d120c640 1 -- 172.21.15.60:0/3345489392 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6cc161aa0 msgr2=0x7fd6cc0b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d120c640 1 --2- 172.21.15.60:0/3345489392 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6cc161aa0 0x7fd6cc0b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d120c640 1 -- 172.21.15.60:0/3345489392 >> v1:172.21.15.100:6789/0 conn(0x7fd6cc0b3900 legacy=0x7fd6cc0b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d120c640 1 -- 172.21.15.60:0/3345489392 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc0afc50 con 0x7fd6cc0cd370 2024-04-25T10:05:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d120c640 1 -- 172.21.15.60:0/3345489392 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fd6bc001680 con 0x7fd6cc0cd370 2024-04-25T10:05:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d120c640 1 -- 172.21.15.60:0/3345489392 >> v1:172.21.15.60:6789/0 conn(0x7fd6cc0cd370 legacy=0x7fd6cc0b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d120c640 1 --2- 172.21.15.60:0/3345489392 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6b80039f0 0x7fd6b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d120c640 1 --2- 172.21.15.60:0/3345489392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b8004810 0x7fd6b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d120c640 1 --2- 172.21.15.60:0/3345489392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6b80051d0 0x7fd6b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d120c640 1 -- 172.21.15.60:0/3345489392 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc0b4240 con 0x7fd6b8004810 2024-04-25T10:05:30.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d120c640 1 -- 172.21.15.60:0/3345489392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b802e0e0 con 0x7fd6b80051d0 2024-04-25T10:05:30.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d120c640 1 -- 172.21.15.60:0/3345489392 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b802e2e0 con 0x7fd6b80039f0 2024-04-25T10:05:30.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d1a0d640 1 --2- 172.21.15.60:0/3345489392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b8004810 0x7fd6b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d1a0d640 1 -- 172.21.15.60:0/3345489392 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6b80039f0 msgr2=0x7fd6b8003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:05:30.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d1a0d640 1 --2- 172.21.15.60:0/3345489392 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6b80039f0 0x7fd6b8003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d1a0d640 1 -- 172.21.15.60:0/3345489392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6b80051d0 msgr2=0x7fd6b800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d1a0d640 1 --2- 172.21.15.60:0/3345489392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6b80051d0 0x7fd6b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.255+0000 7fd6d1a0d640 1 -- 172.21.15.60:0/3345489392 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b8003530 con 0x7fd6b8004810 2024-04-25T10:05:30.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.259+0000 7fd6d1a0d640 1 --2- 172.21.15.60:0/3345489392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b8004810 0x7fd6b8004c90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00c240 tx=0x7fd6bc01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288a99792499f4dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.259+0000 7fd6d120c640 1 -- 172.21.15.60:0/3345489392 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc007d40 con 0x7fd6b8004810 2024-04-25T10:05:30.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.259+0000 7f6fd6dfa640 1 Processor -- start 2024-04-25T10:05:30.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.259+0000 7fd6d120c640 1 -- 172.21.15.60:0/3345489392 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6bc0a9020 con 0x7fd6b8004810 2024-04-25T10:05:30.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.259+0000 7fd6d120c640 1 -- 172.21.15.60:0/3345489392 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc005b00 con 0x7fd6b8004810 2024-04-25T10:05:30.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7fd6d4472640 1 -- 172.21.15.60:0/3345489392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b8004810 msgr2=0x7fd6b8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7fd6d4472640 1 --2- 172.21.15.60:0/3345489392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b8004810 0x7fd6b8004c90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00c240 tx=0x7fd6bc01cc60 comp rx=0 tx=0).stop 2024-04-25T10:05:30.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7fd6d2a0f640 1 -- 172.21.15.60:0/3345489392 reap_dead start 2024-04-25T10:05:30.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7f6fd6dfa640 1 -- start start 2024-04-25T10:05:30.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7fd6d4472640 1 -- 172.21.15.60:0/3345489392 shutdown_connections 2024-04-25T10:05:30.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7fd6d4472640 1 -- 172.21.15.60:0/3345489392 >> 172.21.15.60:0/3345489392 conn(0x7fd6cc06b4c0 msgr2=0x7fd6cc06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7fd6d4472640 1 -- 172.21.15.60:0/3345489392 shutdown_connections 2024-04-25T10:05:30.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7f3a867fc640 1 -- 172.21.15.60:0/2952863690 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a8c021020 con 0x7f3a900cf100 2024-04-25T10:05:30.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7f6fd6dfa640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6fd00cefc0 0x7f6fd015bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7f6fd6dfa640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6fd0070e50 0x7f6fd00d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7f6fd6dfa640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd00716c0 con 0x7f6fd00d0ec0 2024-04-25T10:05:30.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7f6fd4b96640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6fd00cefc0 0x7f6fd015bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7f6fd4b96640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6fd00cefc0 0x7f6fd015bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:45986/0 (socket says 172.21.15.60:45986) 2024-04-25T10:05:30.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7f6fd4b96640 1 -- 172.21.15.60:0/2636529543 learned_addr learned my addr 172.21.15.60:0/2636529543 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:30.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7f6fcffff640 1 --2- 172.21.15.60:0/2636529543 >> v2:172.21.15.60:3300/0 conn(0x7f6fd0070e50 0x7f6fd00d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7fd6d4472640 1 -- 172.21.15.60:0/3345489392 wait complete. 2024-04-25T10:05:30.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7fd6d4472640 1 Processor -- start 2024-04-25T10:05:30.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7f6fd6dfa640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd015d4e0 con 0x7f6fd0070e50 2024-04-25T10:05:30.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2636529543 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd00719a0 con 0x7f6fd00cefc0 2024-04-25T10:05:30.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7fd6d4472640 1 -- start start 2024-04-25T10:05:30.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.263+0000 7fd6d4472640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b8004810 0x7fd6cc151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6d220e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b8004810 0x7fd6cc151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6d220e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b8004810 0x7fd6cc151910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46442/0 (socket says 172.21.15.60:46442) 2024-04-25T10:05:30.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7f6fcffff640 1 -- 172.21.15.60:0/2636529543 >> v2:172.21.15.100:3300/0 conn(0x7f6fd00cefc0 msgr2=0x7f6fd015bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7f6fcffff640 1 --2- 172.21.15.60:0/2636529543 >> v2:172.21.15.100:3300/0 conn(0x7f6fd00cefc0 0x7f6fd015bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7f6fcffff640 1 -- 172.21.15.60:0/2636529543 >> v1:172.21.15.60:6789/0 conn(0x7f6fd00d0ec0 legacy=0x7f6fd015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7f6fcffff640 1 -- 172.21.15.60:0/2636529543 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd0071c80 con 0x7f6fd0070e50 2024-04-25T10:05:30.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6d4472640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6cc153700 0x7fd6cc156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6d220e640 1 -- 172.21.15.60:0/3597915271 learned_addr learned my addr 172.21.15.60:0/3597915271 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:30.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6d4472640 1 --2- 172.21.15.60:0/3597915271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6cc14f2c0 0x7fd6cc165490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6d1a0d640 1 --2- 172.21.15.60:0/3597915271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6cc153700 0x7fd6cc156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6d4472640 1 -- 172.21.15.60:0/3597915271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc166c10 con 0x7fd6b8004810 2024-04-25T10:05:30.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6d4472640 1 -- 172.21.15.60:0/3597915271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc166a90 con 0x7fd6cc14f2c0 2024-04-25T10:05:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6d4472640 1 -- 172.21.15.60:0/3597915271 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc166d90 con 0x7fd6cc153700 2024-04-25T10:05:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6d2a0f640 1 --2- 172.21.15.60:0/3597915271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6cc14f2c0 0x7fd6cc165490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6d2a0f640 1 -- 172.21.15.60:0/3597915271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6cc153700 msgr2=0x7fd6cc156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6d2a0f640 1 --2- 172.21.15.60:0/3597915271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6cc153700 0x7fd6cc156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6d2a0f640 1 -- 172.21.15.60:0/3597915271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b8004810 msgr2=0x7fd6cc151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6d2a0f640 1 --2- 172.21.15.60:0/3597915271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b8004810 0x7fd6cc151910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6d2a0f640 1 -- 172.21.15.60:0/3597915271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc153b80 con 0x7fd6cc14f2c0 2024-04-25T10:05:30.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6d2a0f640 1 --2- 172.21.15.60:0/3597915271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6cc14f2c0 0x7fd6cc165490 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc06fa80 tx=0x7fd6c400cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7f6fcffff640 1 --2- 172.21.15.60:0/2636529543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd0070e50 0x7f6fd00d0980 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00750e0 tx=0x7f6fc401ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3811408d8905ce36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6c2ffd640 1 -- 172.21.15.60:0/3597915271 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4017070 con 0x7fd6cc14f2c0 2024-04-25T10:05:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6c2ffd640 1 -- 172.21.15.60:0/3597915271 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c4005d70 con 0x7fd6cc14f2c0 2024-04-25T10:05:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.267+0000 7fd6c2ffd640 1 -- 172.21.15.60:0/3597915271 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c400a7f0 con 0x7fd6cc14f2c0 2024-04-25T10:05:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.271+0000 7fd6d4472640 1 -- 172.21.15.60:0/3597915271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6cc151ef0 con 0x7fd6cc14f2c0 2024-04-25T10:05:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.271+0000 7fd6d4472640 1 -- 172.21.15.60:0/3597915271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6cc1524b0 con 0x7fd6cc14f2c0 2024-04-25T10:05:30.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.271+0000 7fd6d4472640 1 -- 172.21.15.60:0/3597915271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6cc074770 con 0x7fd6cc14f2c0 2024-04-25T10:05:30.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.271+0000 7f6fcf7fe640 1 -- 172.21.15.60:0/2636529543 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc40ab430 con 0x7f6fd0070e50 2024-04-25T10:05:30.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.271+0000 7f6fcf7fe640 1 -- 172.21.15.60:0/2636529543 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc40b0070 con 0x7f6fd0070e50 2024-04-25T10:05:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.271+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2636529543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd0070e50 msgr2=0x7f6fd00d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.271+0000 7f6fd6dfa640 1 --2- 172.21.15.60:0/2636529543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd0070e50 0x7f6fd00d0980 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00750e0 tx=0x7f6fc401ce70 comp rx=0 tx=0).stop 2024-04-25T10:05:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.275+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2636529543 shutdown_connections 2024-04-25T10:05:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.275+0000 7f6fd6dfa640 1 --2- 172.21.15.60:0/2636529543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd0070e50 0x7f6fd00d0980 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.275+0000 7f6fd6dfa640 1 --2- 172.21.15.60:0/2636529543 >> v2:172.21.15.100:3300/0 conn(0x7f6fd00cefc0 0x7f6fd015bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.275+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2636529543 >> 172.21.15.60:0/2636529543 conn(0x7f6fd00b3520 msgr2=0x7f6fd00b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.271+0000 7fd6c2ffd640 1 -- 172.21.15.60:0/3597915271 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6c400a990 con 0x7fd6cc14f2c0 2024-04-25T10:05:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.275+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2636529543 shutdown_connections 2024-04-25T10:05:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.275+0000 7fd6c2ffd640 1 --2- 172.21.15.60:0/3597915271 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd6b0066f60 0x7fd6b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.275+0000 7fd6c2ffd640 1 -- 172.21.15.60:0/3597915271 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6c4084740 con 0x7fd6cc14f2c0 2024-04-25T10:05:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.275+0000 7fd6d220e640 1 --2- 172.21.15.60:0/3597915271 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd6b0066f60 0x7fd6b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.275+0000 7fd6d220e640 1 --2- 172.21.15.60:0/3597915271 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd6b0066f60 0x7fd6b0069420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009920 tx=0x7fd6c8009110 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.275+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2636529543 wait complete. 2024-04-25T10:05:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.275+0000 7f6fd6dfa640 1 Processor -- start 2024-04-25T10:05:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.275+0000 7f6fd6dfa640 1 -- start start 2024-04-25T10:05:30.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.275+0000 7f6fd6dfa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00cefc0 0x7f6fd01434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.275+0000 7f6fd6dfa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd00d0ec0 0x7f6fd014dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd6dfa640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fd01465e0 0x7f6fd014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd6dfa640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00cf520 con 0x7f6fd00cefc0 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd4b96640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00cefc0 0x7f6fd01434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd4b96640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00cefc0 0x7f6fd01434e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46458/0 (socket says 172.21.15.60:46458) 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd4b96640 1 -- 172.21.15.60:0/2752229602 learned_addr learned my addr 172.21.15.60:0/2752229602 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2752229602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00cf3a0 con 0x7f6fd00d0ec0 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fcffff640 1 --2- 172.21.15.60:0/2752229602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd00d0ec0 0x7f6fd014dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd5397640 1 --2- 172.21.15.60:0/2752229602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fd01465e0 0x7f6fd014e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2752229602 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd0168690 con 0x7f6fd01465e0 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd4b96640 1 -- 172.21.15.60:0/2752229602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fd01465e0 msgr2=0x7f6fd014e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd4b96640 1 --2- 172.21.15.60:0/2752229602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fd01465e0 0x7f6fd014e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd4b96640 1 -- 172.21.15.60:0/2752229602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd00d0ec0 msgr2=0x7f6fd014dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd4b96640 1 --2- 172.21.15.60:0/2752229602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd00d0ec0 0x7f6fd014dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd4b96640 1 -- 172.21.15.60:0/2752229602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd014cfe0 con 0x7f6fd00cefc0 2024-04-25T10:05:30.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd4b96640 1 --2- 172.21.15.60:0/2752229602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00cefc0 0x7f6fd01434e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00e890 tx=0x7f6fbc00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fcd7fa640 1 -- 172.21.15.60:0/2752229602 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc017070 con 0x7f6fd00cefc0 2024-04-25T10:05:30.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fcd7fa640 1 -- 172.21.15.60:0/2752229602 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc00c940 con 0x7f6fd00cefc0 2024-04-25T10:05:30.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fcd7fa640 1 -- 172.21.15.60:0/2752229602 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc00ccf0 con 0x7f6fd00cefc0 2024-04-25T10:05:30.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2752229602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd014d2c0 con 0x7f6fd00cefc0 2024-04-25T10:05:30.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2752229602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd014e820 con 0x7f6fd00cefc0 2024-04-25T10:05:30.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.279+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2752229602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd0074770 con 0x7f6fd00cefc0 2024-04-25T10:05:30.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.283+0000 7f6fcd7fa640 1 -- 172.21.15.60:0/2752229602 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6fbc01a070 con 0x7f6fd00cefc0 2024-04-25T10:05:30.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.283+0000 7f6fcd7fa640 1 --2- 172.21.15.60:0/2752229602 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6fb4066f90 0x7f6fb4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.283+0000 7f6fcd7fa640 1 -- 172.21.15.60:0/2752229602 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fbc084f90 con 0x7f6fd00cefc0 2024-04-25T10:05:30.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.283+0000 7f6fcffff640 1 --2- 172.21.15.60:0/2752229602 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6fb4066f90 0x7f6fb4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.283+0000 7f6fcffff640 1 --2- 172.21.15.60:0/2752229602 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6fb4066f90 0x7f6fb4069450 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40af000 tx=0x7f6fc4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.283+0000 7f2b9cc58640 1 Processor -- start 2024-04-25T10:05:30.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.283+0000 7f2b9cc58640 1 -- start start 2024-04-25T10:05:30.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.287+0000 7f2b9cc58640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b90055260 0x7f2b900355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.287+0000 7f2b9cc58640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2b90036fa0 0x7f2b90052590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.287+0000 7f2b9cc58640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b90055640 con 0x7f2b90052ad0 2024-04-25T10:05:30.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.287+0000 7f2b9cc58640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9003b670 con 0x7f2b90036fa0 2024-04-25T10:05:30.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.287+0000 7f2b9cc58640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9003aa60 con 0x7f2b90055260 2024-04-25T10:05:30.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.287+0000 7fd6c2ffd640 1 -- 172.21.15.60:0/3597915271 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6c4020020 con 0x7fd6cc14f2c0 2024-04-25T10:05:30.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.287+0000 7f2b96d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2b90052ad0 0x7f2b900df8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37266/0 (socket says 172.21.15.60:37266) 2024-04-25T10:05:30.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.287+0000 7f2b96d9d640 1 -- 172.21.15.60:0/2770173261 learned_addr learned my addr 172.21.15.60:0/2770173261 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:30.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.287+0000 7f2b95d9b640 1 --2- 172.21.15.60:0/2770173261 >> v2:172.21.15.60:3300/0 conn(0x7f2b90036fa0 0x7f2b90052590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.287+0000 7f2b9659c640 1 --2- 172.21.15.60:0/2770173261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b90055260 0x7f2b900355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.287+0000 7f2b95d9b640 1 -- 172.21.15.60:0/2770173261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b90055260 msgr2=0x7f2b900355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.287+0000 7f2b95d9b640 1 --2- 172.21.15.60:0/2770173261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b90055260 0x7f2b900355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.287+0000 7f2b95d9b640 1 -- 172.21.15.60:0/2770173261 >> v1:172.21.15.60:6789/0 conn(0x7f2b90052ad0 legacy=0x7f2b900df8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.287+0000 7f2b95d9b640 1 -- 172.21.15.60:0/2770173261 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9003ad40 con 0x7f2b90036fa0 2024-04-25T10:05:30.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.287+0000 7f2b95d9b640 1 --2- 172.21.15.60:0/2770173261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b90036fa0 0x7f2b90052590 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009a20 tx=0x7f2b8801c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e82564403d21c9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.291+0000 7f6fcd7fa640 1 -- 172.21.15.60:0/2752229602 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6fbc0558b0 con 0x7f6fd00cefc0 2024-04-25T10:05:30.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.291+0000 7f2b9559a640 1 -- 172.21.15.60:0/2770173261 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b880aa660 con 0x7f2b90036fa0 2024-04-25T10:05:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.291+0000 7f2b9559a640 1 -- 172.21.15.60:0/2770173261 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b880af070 con 0x7f2b90036fa0 2024-04-25T10:05:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.291+0000 7f2b9cc58640 1 -- 172.21.15.60:0/2770173261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b90036fa0 msgr2=0x7f2b90052590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.291+0000 7f2b9559a640 1 -- 172.21.15.60:0/2770173261 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b880aa660 con 0x7f2b90036fa0 2024-04-25T10:05:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.291+0000 7f2b9cc58640 1 --2- 172.21.15.60:0/2770173261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b90036fa0 0x7f2b90052590 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009a20 tx=0x7f2b8801c700 comp rx=0 tx=0).stop 2024-04-25T10:05:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.291+0000 7f2b9cc58640 1 -- 172.21.15.60:0/2770173261 shutdown_connections 2024-04-25T10:05:30.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.291+0000 7f2b9cc58640 1 --2- 172.21.15.60:0/2770173261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b90036fa0 0x7f2b90052590 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.291+0000 7f2b9cc58640 1 --2- 172.21.15.60:0/2770173261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b90055260 0x7f2b900355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.291+0000 7f2b9cc58640 1 -- 172.21.15.60:0/2770173261 >> 172.21.15.60:0/2770173261 conn(0x7f2b9001ef50 msgr2=0x7f2b90055c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.291+0000 7f2b9cc58640 1 -- 172.21.15.60:0/2770173261 shutdown_connections 2024-04-25T10:05:30.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.291+0000 7f2b9cc58640 1 -- 172.21.15.60:0/2770173261 wait complete. 2024-04-25T10:05:30.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.295+0000 7f2b9cc58640 1 Processor -- start 2024-04-25T10:05:30.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.295+0000 7f2b9cc58640 1 -- start start 2024-04-25T10:05:30.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.295+0000 7f2b9cc58640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b90052ad0 0x7f2b900d9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.295+0000 7f2b9cc58640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b90055260 0x7f2b9000c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.295+0000 7f2b9cc58640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b900d7ec0 0x7f2b900d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.295+0000 7f2b9cc58640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900083e0 con 0x7f2b900d7ec0 2024-04-25T10:05:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.295+0000 7f2b9cc58640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900e03f0 con 0x7f2b90055260 2024-04-25T10:05:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.295+0000 7f2b9cc58640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b90008560 con 0x7f2b90052ad0 2024-04-25T10:05:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.295+0000 7f2b96d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b900d7ec0 0x7f2b900d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.295+0000 7f2b96d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b900d7ec0 0x7f2b900d17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46484/0 (socket says 172.21.15.60:46484) 2024-04-25T10:05:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.295+0000 7f2b96d9d640 1 -- 172.21.15.60:0/1193121312 learned_addr learned my addr 172.21.15.60:0/1193121312 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.295+0000 7f2b9659c640 1 --2- 172.21.15.60:0/1193121312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b90052ad0 0x7f2b900d9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.295+0000 7f2b95d9b640 1 --2- 172.21.15.60:0/1193121312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b90055260 0x7f2b9000c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.299+0000 7f2b96d9d640 1 -- 172.21.15.60:0/1193121312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b90052ad0 msgr2=0x7f2b900d9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.299+0000 7f2b96d9d640 1 --2- 172.21.15.60:0/1193121312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b90052ad0 0x7f2b900d9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.299+0000 7f2b96d9d640 1 -- 172.21.15.60:0/1193121312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b90055260 msgr2=0x7f2b9000c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.299+0000 7f2b96d9d640 1 --2- 172.21.15.60:0/1193121312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b90055260 0x7f2b9000c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.299+0000 7f2b96d9d640 1 -- 172.21.15.60:0/1193121312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b900d2420 con 0x7f2b900d7ec0 2024-04-25T10:05:30.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.299+0000 7f2b96d9d640 1 --2- 172.21.15.60:0/1193121312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b900d7ec0 0x7f2b900d17f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2b9806a4c0 tx=0x7f2b9804ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.299+0000 7f2b86ffd640 1 -- 172.21.15.60:0/1193121312 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b98074430 con 0x7f2b900d7ec0 2024-04-25T10:05:30.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.299+0000 7f2b9cc58640 1 -- 172.21.15.60:0/1193121312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b900d8350 con 0x7f2b900d7ec0 2024-04-25T10:05:30.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.299+0000 7f2b9cc58640 1 -- 172.21.15.60:0/1193121312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b900d8910 con 0x7f2b900d7ec0 2024-04-25T10:05:30.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.299+0000 7f2b86ffd640 1 -- 172.21.15.60:0/1193121312 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9808f020 con 0x7f2b900d7ec0 2024-04-25T10:05:30.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.299+0000 7f2b86ffd640 1 -- 172.21.15.60:0/1193121312 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9808e020 con 0x7f2b900d7ec0 2024-04-25T10:05:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.299+0000 7f2b86ffd640 1 -- 172.21.15.60:0/1193121312 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2b98074890 con 0x7f2b900d7ec0 2024-04-25T10:05:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.299+0000 7f2b86ffd640 1 --2- 172.21.15.60:0/1193121312 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b78066f60 0x7f2b78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.299+0000 7f2b86ffd640 1 -- 172.21.15.60:0/1193121312 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b980ff2b0 con 0x7f2b900d7ec0 2024-04-25T10:05:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.299+0000 7f2b9659c640 1 --2- 172.21.15.60:0/1193121312 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b78066f60 0x7f2b78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f2b9659c640 1 --2- 172.21.15.60:0/1193121312 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b78066f60 0x7f2b78069420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009ac0 tx=0x7f2b8c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e83c40640 1 Processor -- start 2024-04-25T10:05:30.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e83c40640 1 -- start start 2024-04-25T10:05:30.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e83c40640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e7c0d0360 0x7f6e7c0b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e83c40640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c0d0860 con 0x7f6e7c0b19e0 2024-04-25T10:05:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e83c40640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c0d0b40 con 0x7f6e7c15a700 2024-04-25T10:05:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e83c40640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0d0e20 con 0x7f6e7c0d0360 2024-04-25T10:05:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e821dd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6e7c0b19e0 0x7f6e7c0cd260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37282/0 (socket says 172.21.15.60:37282) 2024-04-25T10:05:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e821dd640 1 -- 172.21.15.60:0/1277616171 learned_addr learned my addr 172.21.15.60:0/1277616171 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e819dc640 1 --1- 172.21.15.60:0/1277616171 >> v1:172.21.15.100:6789/0 conn(0x7f6e7c15a700 0x7f6e7c0b37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:60292/0 (socket says 172.21.15.60:60292) 2024-04-25T10:05:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e811db640 1 --2- 172.21.15.60:0/1277616171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e7c0d0360 0x7f6e7c0b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 -- 172.21.15.60:0/1277616171 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883756694 0 0) 0x7f6e7c0d0860 con 0x7f6e7c0b19e0 2024-04-25T10:05:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 -- 172.21.15.60:0/1277616171 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e68003530 con 0x7f6e7c0b19e0 2024-04-25T10:05:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 -- 172.21.15.60:0/1277616171 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406694428 0 0) 0x7f6e7c0d0b40 con 0x7f6e7c15a700 2024-04-25T10:05:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 -- 172.21.15.60:0/1277616171 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e7c0d0860 con 0x7f6e7c15a700 2024-04-25T10:05:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 -- 172.21.15.60:0/1277616171 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2587188550 0 0) 0x7f6e68003530 con 0x7f6e7c0b19e0 2024-04-25T10:05:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 -- 172.21.15.60:0/1277616171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e7c0d0360 msgr2=0x7f6e7c0b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 --2- 172.21.15.60:0/1277616171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e7c0d0360 0x7f6e7c0b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 -- 172.21.15.60:0/1277616171 >> v1:172.21.15.100:6789/0 conn(0x7f6e7c15a700 legacy=0x7f6e7c0b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 -- 172.21.15.60:0/1277616171 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c0d1100 con 0x7f6e7c0b19e0 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 -- 172.21.15.60:0/1277616171 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f6e74001a00 con 0x7f6e7c0b19e0 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e811db640 1 --2- 172.21.15.60:0/1277616171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e7c0d0360 0x7f6e7c0b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 -- 172.21.15.60:0/1277616171 >> v1:172.21.15.60:6789/0 conn(0x7f6e7c0b19e0 legacy=0x7f6e7c0cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 --2- 172.21.15.60:0/1277616171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e68004be0 0x7f6e68004fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 --2- 172.21.15.60:0/1277616171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e7c0d0360 0x7f6e680037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 --2- 172.21.15.60:0/1277616171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e68003cf0 0x7f6e68004170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 -- 172.21.15.60:0/1277616171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0d0e20 con 0x7f6e68003cf0 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 -- 172.21.15.60:0/1277616171 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6800ffe0 con 0x7f6e68004be0 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e809da640 1 -- 172.21.15.60:0/1277616171 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e68010160 con 0x7f6e7c0d0360 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e819dc640 1 --2- 172.21.15.60:0/1277616171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e68004be0 0x7f6e68004fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e811db640 1 --2- 172.21.15.60:0/1277616171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e7c0d0360 0x7f6e680037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e821dd640 1 --2- 172.21.15.60:0/1277616171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e68003cf0 0x7f6e68004170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e821dd640 1 -- 172.21.15.60:0/1277616171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e7c0d0360 msgr2=0x7f6e680037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e821dd640 1 --2- 172.21.15.60:0/1277616171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e7c0d0360 0x7f6e680037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e821dd640 1 -- 172.21.15.60:0/1277616171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e68004be0 msgr2=0x7f6e68004fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e821dd640 1 --2- 172.21.15.60:0/1277616171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e68004be0 0x7f6e68004fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.303+0000 7f6e821dd640 1 -- 172.21.15.60:0/1277616171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c0d1100 con 0x7f6e68003cf0 2024-04-25T10:05:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.307+0000 7f6e821dd640 1 --2- 172.21.15.60:0/1277616171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e68003cf0 0x7f6e68004170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400a8b0 tx=0x7f6e7401faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ec733841509484 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.307+0000 7f6e809da640 1 -- 172.21.15.60:0/1277616171 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7400a480 con 0x7f6e68003cf0 2024-04-25T10:05:30.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.307+0000 7f6e809da640 1 -- 172.21.15.60:0/1277616171 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e740aa6d0 con 0x7f6e68003cf0 2024-04-25T10:05:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.307+0000 7f6e809da640 1 -- 172.21.15.60:0/1277616171 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e740aab00 con 0x7f6e68003cf0 2024-04-25T10:05:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.307+0000 7f6e83c40640 1 -- 172.21.15.60:0/1277616171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e68003cf0 msgr2=0x7f6e68004170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.307+0000 7f6e83c40640 1 --2- 172.21.15.60:0/1277616171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e68003cf0 0x7f6e68004170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400a8b0 tx=0x7f6e7401faa0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.307+0000 7f6e821dd640 1 -- 172.21.15.60:0/1277616171 reap_dead start 2024-04-25T10:05:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.307+0000 7f6e83c40640 1 -- 172.21.15.60:0/1277616171 shutdown_connections 2024-04-25T10:05:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.307+0000 7f6e83c40640 1 -- 172.21.15.60:0/1277616171 >> 172.21.15.60:0/1277616171 conn(0x7f6e7c072920 msgr2=0x7f6e7c099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.307+0000 7f6e83c40640 1 -- 172.21.15.60:0/1277616171 shutdown_connections 2024-04-25T10:05:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.307+0000 7f6e83c40640 1 -- 172.21.15.60:0/1277616171 wait complete. 2024-04-25T10:05:30.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.307+0000 7f2b9cc58640 1 -- 172.21.15.60:0/1193121312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7c0052a0 con 0x7f2b900d7ec0 2024-04-25T10:05:30.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e83c40640 1 Processor -- start 2024-04-25T10:05:30.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e83c40640 1 -- start start 2024-04-25T10:05:30.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e83c40640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e68003cf0 0x7f6e7c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e83c40640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7c149e90 0x7f6e7c0ce580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e83c40640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c0adb80 0x7f6e7c0ae010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e821dd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c0adb80 0x7f6e7c0ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e821dd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c0adb80 0x7f6e7c0ae010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46498/0 (socket says 172.21.15.60:46498) 2024-04-25T10:05:30.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e821dd640 1 -- 172.21.15.60:0/2091619977 learned_addr learned my addr 172.21.15.60:0/2091619977 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:30.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e83c40640 1 -- 172.21.15.60:0/2091619977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c15ac60 con 0x7f6e7c0adb80 2024-04-25T10:05:30.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e83c40640 1 -- 172.21.15.60:0/2091619977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c15aae0 con 0x7f6e7c149e90 2024-04-25T10:05:30.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e83c40640 1 -- 172.21.15.60:0/2091619977 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0b14a0 con 0x7f6e68003cf0 2024-04-25T10:05:30.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e811db640 1 --2- 172.21.15.60:0/2091619977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7c149e90 0x7f6e7c0ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e821dd640 1 -- 172.21.15.60:0/2091619977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e68003cf0 msgr2=0x7f6e7c15d630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:05:30.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e821dd640 1 --2- 172.21.15.60:0/2091619977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e68003cf0 0x7f6e7c15d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e821dd640 1 -- 172.21.15.60:0/2091619977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7c149e90 msgr2=0x7f6e7c0ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e821dd640 1 --2- 172.21.15.60:0/2091619977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7c149e90 0x7f6e7c0ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e821dd640 1 -- 172.21.15.60:0/2091619977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c0cf490 con 0x7f6e7c0adb80 2024-04-25T10:05:30.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7f6e821dd640 1 --2- 172.21.15.60:0/2091619977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c0adb80 0x7f6e7c0ae010 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6e7401f8d0 tx=0x7f6e68005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7fba26d17640 1 Processor -- start 2024-04-25T10:05:30.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.311+0000 7fba26d17640 1 -- start start 2024-04-25T10:05:30.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.315+0000 7fba26d17640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba200afcb0 0x7fba2015b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.315+0000 7fba26d17640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fba200b3550 0x7fba200721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.315+0000 7fba26d17640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba200b39e0 con 0x7fba200b21e0 2024-04-25T10:05:30.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.315+0000 7fba26d17640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2015ba60 con 0x7fba200b3550 2024-04-25T10:05:30.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.315+0000 7fba26d17640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba200b3cc0 con 0x7fba200afcb0 2024-04-25T10:05:30.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.315+0000 7fba25514640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fba200b3550 0x7fba200721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.315+0000 7fba25514640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fba200b3550 0x7fba200721a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46500/0 (socket says 172.21.15.60:46500) 2024-04-25T10:05:30.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.315+0000 7fba25514640 1 -- 172.21.15.60:0/2664747921 learned_addr learned my addr 172.21.15.60:0/2664747921 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:30.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.315+0000 7fba25d15640 1 --2- 172.21.15.60:0/2664747921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba200afcb0 0x7fba2015b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.315+0000 7fba25514640 1 -- 172.21.15.60:0/2664747921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba200afcb0 msgr2=0x7fba2015b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.315+0000 7fba25514640 1 --2- 172.21.15.60:0/2664747921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba200afcb0 0x7fba2015b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.315+0000 7fba25514640 1 -- 172.21.15.60:0/2664747921 >> v1:172.21.15.100:6789/0 conn(0x7fba200b21e0 legacy=0x7fba200b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.315+0000 7fba25514640 1 -- 172.21.15.60:0/2664747921 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba200b3fa0 con 0x7fba200b3550 2024-04-25T10:05:30.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.315+0000 7fba25514640 1 --2- 172.21.15.60:0/2664747921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba200b3550 0x7fba200721a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00bb10 tx=0x7fba1c01eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6e4a6767f539b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.319+0000 7f2b86ffd640 1 -- 172.21.15.60:0/1193121312 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2b98099030 con 0x7f2b900d7ec0 2024-04-25T10:05:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.319+0000 7fba24d13640 1 -- 172.21.15.60:0/2664747921 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0b4070 con 0x7fba200b3550 2024-04-25T10:05:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.319+0000 7fba24d13640 1 -- 172.21.15.60:0/2664747921 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1c0a7ae0 con 0x7fba200b3550 2024-04-25T10:05:30.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.319+0000 7fba24d13640 1 -- 172.21.15.60:0/2664747921 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0af440 con 0x7fba200b3550 2024-04-25T10:05:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.319+0000 7fba26d17640 1 -- 172.21.15.60:0/2664747921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba200b3550 msgr2=0x7fba200721a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.319+0000 7fba26d17640 1 --2- 172.21.15.60:0/2664747921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba200b3550 0x7fba200721a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00bb10 tx=0x7fba1c01eb60 comp rx=0 tx=0).stop 2024-04-25T10:05:30.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.319+0000 7fba26d17640 1 -- 172.21.15.60:0/2664747921 shutdown_connections 2024-04-25T10:05:30.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.319+0000 7fba26d17640 1 --2- 172.21.15.60:0/2664747921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba200b3550 0x7fba200721a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.319+0000 7fba26d17640 1 --2- 172.21.15.60:0/2664747921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba200afcb0 0x7fba2015b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.319+0000 7fba26d17640 1 -- 172.21.15.60:0/2664747921 >> 172.21.15.60:0/2664747921 conn(0x7fba2006e390 msgr2=0x7fba2006e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.319+0000 7f6e727fc640 1 -- 172.21.15.60:0/2091619977 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e740011d0 con 0x7f6e7c0adb80 2024-04-25T10:05:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.323+0000 7f6e727fc640 1 -- 172.21.15.60:0/2091619977 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e74001800 con 0x7f6e7c0adb80 2024-04-25T10:05:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.323+0000 7f6e727fc640 1 -- 172.21.15.60:0/2091619977 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e740aaa80 con 0x7f6e7c0adb80 2024-04-25T10:05:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.327+0000 7f6e83c40640 1 -- 172.21.15.60:0/2091619977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c0ceed0 con 0x7f6e7c0adb80 2024-04-25T10:05:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.327+0000 7f6e83c40640 1 -- 172.21.15.60:0/2091619977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7c1563d0 con 0x7f6e7c0adb80 2024-04-25T10:05:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.327+0000 7fba26d17640 1 -- 172.21.15.60:0/2664747921 shutdown_connections 2024-04-25T10:05:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.327+0000 7f6e83c40640 1 -- 172.21.15.60:0/2091619977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7c06ccb0 con 0x7f6e7c0adb80 2024-04-25T10:05:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.327+0000 7fba26d17640 1 -- 172.21.15.60:0/2664747921 wait complete. 2024-04-25T10:05:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.327+0000 7f6e727fc640 1 -- 172.21.15.60:0/2091619977 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e740aac20 con 0x7f6e7c0adb80 2024-04-25T10:05:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.327+0000 7f6e727fc640 1 --2- 172.21.15.60:0/2091619977 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6e60067060 0x7f6e60069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.327+0000 7f6e727fc640 1 -- 172.21.15.60:0/2091619977 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e74122e50 con 0x7f6e7c0adb80 2024-04-25T10:05:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.327+0000 7f6e819dc640 1 --2- 172.21.15.60:0/2091619977 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6e60067060 0x7f6e60069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.331+0000 7f6e819dc640 1 --2- 172.21.15.60:0/2091619977 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6e60067060 0x7f6e60069520 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6e780079e0 tx=0x7f6e78007950 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.335+0000 7fba26d17640 1 Processor -- start 2024-04-25T10:05:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.335+0000 7fba26d17640 1 -- start start 2024-04-25T10:05:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.335+0000 7fba26d17640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba200afcb0 0x7fba2015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.335+0000 7fba26d17640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba200b21e0 0x7fba20145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.335+0000 7fba26d17640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba20157f20 0x7fba20151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.335+0000 7fba26d17640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba200704b0 con 0x7fba20157f20 2024-04-25T10:05:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.335+0000 7fba25d15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba200afcb0 0x7fba2015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.335+0000 7fba25d15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba200afcb0 0x7fba2015a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:51706/0 (socket says 172.21.15.60:51706) 2024-04-25T10:05:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.335+0000 7fba25d15640 1 -- 172.21.15.60:0/271683949 learned_addr learned my addr 172.21.15.60:0/271683949 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.335+0000 7fba26516640 1 --2- 172.21.15.60:0/271683949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba20157f20 0x7fba20151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.335+0000 7fba25514640 1 --2- 172.21.15.60:0/271683949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba200b21e0 0x7fba20145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.335+0000 7fba26d17640 1 -- 172.21.15.60:0/271683949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba200b2a20 con 0x7fba200b21e0 2024-04-25T10:05:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.335+0000 7fba26d17640 1 -- 172.21.15.60:0/271683949 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba20070630 con 0x7fba200afcb0 2024-04-25T10:05:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.339+0000 7fba26516640 1 -- 172.21.15.60:0/271683949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba200afcb0 msgr2=0x7fba2015a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.339+0000 7fba26516640 1 --2- 172.21.15.60:0/271683949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba200afcb0 0x7fba2015a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.339+0000 7fba26516640 1 -- 172.21.15.60:0/271683949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba200b21e0 msgr2=0x7fba20145c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.339+0000 7fba26516640 1 --2- 172.21.15.60:0/271683949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba200b21e0 0x7fba20145c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.339+0000 7fba26516640 1 -- 172.21.15.60:0/271683949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2014e770 con 0x7fba20157f20 2024-04-25T10:05:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.339+0000 7fba26516640 1 --2- 172.21.15.60:0/271683949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba20157f20 0x7fba20151460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fba1800c7b0 tx=0x7fba1800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.339+0000 7fba167fc640 1 -- 172.21.15.60:0/271683949 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba18016020 con 0x7fba20157f20 2024-04-25T10:05:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.339+0000 7f6e727fc640 1 -- 172.21.15.60:0/2091619977 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e740ae030 con 0x7f6e7c0adb80 2024-04-25T10:05:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.339+0000 7fba26d17640 1 -- 172.21.15.60:0/271683949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2014ea50 con 0x7fba20157f20 2024-04-25T10:05:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.339+0000 7fba26d17640 1 -- 172.21.15.60:0/271683949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba20148d00 con 0x7fba20157f20 2024-04-25T10:05:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.339+0000 7fba167fc640 1 -- 172.21.15.60:0/271683949 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1800a9f0 con 0x7fba20157f20 2024-04-25T10:05:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.339+0000 7fba167fc640 1 -- 172.21.15.60:0/271683949 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1800ce60 con 0x7fba20157f20 2024-04-25T10:05:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.343+0000 7fba167fc640 1 -- 172.21.15.60:0/271683949 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba18019020 con 0x7fba20157f20 2024-04-25T10:05:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.343+0000 7fba167fc640 1 --2- 172.21.15.60:0/271683949 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fba0c06b4c0 0x7fba0c06d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.343+0000 7fba25d15640 1 --2- 172.21.15.60:0/271683949 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fba0c06b4c0 0x7fba0c06d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.343+0000 7fba167fc640 1 -- 172.21.15.60:0/271683949 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba18053b00 con 0x7fba20157f20 2024-04-25T10:05:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.343+0000 7fba25d15640 1 --2- 172.21.15.60:0/271683949 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fba0c06b4c0 0x7fba0c06d980 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fba10007a70 tx=0x7fba10008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.347+0000 7fba26d17640 1 -- 172.21.15.60:0/271683949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba080052a0 con 0x7fba20157f20 2024-04-25T10:05:30.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.359+0000 7fba167fc640 1 -- 172.21.15.60:0/271683949 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba18020020 con 0x7fba20157f20 2024-04-25T10:05:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.563+0000 7ff940d8c640 1 -- 172.21.15.60:0/538658678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff920005590 con 0x7ff93c0cbeb0 2024-04-25T10:05:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.563+0000 7ff92affd640 1 -- 172.21.15.60:0/538658678 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff9340efac0 con 0x7ff93c0cbeb0 2024-04-25T10:05:30.572 INFO:teuthology.orchestra.run.smithi060.stdout:60129542151 2024-04-25T10:05:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.567+0000 7ff928ff9640 1 -- 172.21.15.60:0/538658678 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff91c066f90 msgr2=0x7ff91c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.567+0000 7ff928ff9640 1 --2- 172.21.15.60:0/538658678 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff91c066f90 0x7ff91c069450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff930000c10 tx=0x7ff930009040 comp rx=0 tx=0).stop 2024-04-25T10:05:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.567+0000 7ff928ff9640 1 -- 172.21.15.60:0/538658678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff93c0cbeb0 msgr2=0x7ff93c13d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.567+0000 7ff928ff9640 1 --2- 172.21.15.60:0/538658678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff93c0cbeb0 0x7ff93c13d8e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff93401b830 tx=0x7ff93401b9c0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.567+0000 7ff928ff9640 1 -- 172.21.15.60:0/538658678 shutdown_connections 2024-04-25T10:05:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.567+0000 7ff928ff9640 1 --2- 172.21.15.60:0/538658678 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff91c066f90 0x7ff91c069450 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.567+0000 7ff928ff9640 1 --2- 172.21.15.60:0/538658678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff93c13de20 0x7ff93c139eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.567+0000 7ff928ff9640 1 --2- 172.21.15.60:0/538658678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff93c0cbeb0 0x7ff93c13d8e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.567+0000 7ff928ff9640 1 --2- 172.21.15.60:0/538658678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff93c0b3ca0 0x7ff93c159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.567+0000 7ff928ff9640 1 -- 172.21.15.60:0/538658678 >> 172.21.15.60:0/538658678 conn(0x7ff93c06b4c0 msgr2=0x7ff93c141ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.567+0000 7ff928ff9640 1 -- 172.21.15.60:0/538658678 shutdown_connections 2024-04-25T10:05:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.567+0000 7ff928ff9640 1 -- 172.21.15.60:0/538658678 wait complete. 2024-04-25T10:05:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.575+0000 7f7ae8055640 1 -- 172.21.15.60:0/2413798676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7ac8005590 con 0x7f7ae00cd390 2024-04-25T10:05:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.575+0000 7f7ad67fc640 1 -- 172.21.15.60:0/2413798676 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7ad0059800 con 0x7f7ae00cd390 2024-04-25T10:05:30.584 INFO:teuthology.orchestra.run.smithi060.stdout:60129542150 2024-04-25T10:05:30.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.579+0000 7f7ae8055640 1 -- 172.21.15.60:0/2413798676 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ac4067030 msgr2=0x7f7ac40694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.579+0000 7f7ae8055640 1 --2- 172.21.15.60:0/2413798676 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ac4067030 0x7f7ac40694f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0af000 tx=0x7f7adc004740 comp rx=0 tx=0).stop 2024-04-25T10:05:30.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.579+0000 7f7ae8055640 1 -- 172.21.15.60:0/2413798676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae00cd390 msgr2=0x7f7ae00b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.579+0000 7f7ae8055640 1 --2- 172.21.15.60:0/2413798676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae00cd390 0x7f7ae00b4600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000e930 tx=0x7f7ad000edf0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.579+0000 7f7ae8055640 1 -- 172.21.15.60:0/2413798676 shutdown_connections 2024-04-25T10:05:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.579+0000 7f7ae8055640 1 --2- 172.21.15.60:0/2413798676 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ac4067030 0x7f7ac40694f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.579+0000 7f7ae8055640 1 --2- 172.21.15.60:0/2413798676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ae00b4b40 0x7f7ae00b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.579+0000 7f7ae8055640 1 --2- 172.21.15.60:0/2413798676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ae00cd390 0x7f7ae00b4600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.579+0000 7f7ae8055640 1 --2- 172.21.15.60:0/2413798676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ae009a930 0x7f7ae00af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.579+0000 7f7ae8055640 1 -- 172.21.15.60:0/2413798676 >> 172.21.15.60:0/2413798676 conn(0x7f7ae00b32a0 msgr2=0x7f7ae00b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.579+0000 7f7ae8055640 1 -- 172.21.15.60:0/2413798676 shutdown_connections 2024-04-25T10:05:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.579+0000 7f7ae8055640 1 -- 172.21.15.60:0/2413798676 wait complete. 2024-04-25T10:05:30.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.583+0000 7f3a97bd8640 1 -- 172.21.15.60:0/2952863690 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3a90155d60 con 0x7f3a900cf100 2024-04-25T10:05:30.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.583+0000 7f3a867fc640 1 -- 172.21.15.60:0/2952863690 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a8c005480 con 0x7f3a900cf100 2024-04-25T10:05:30.592 INFO:teuthology.orchestra.run.smithi060.stdout:60129542152 2024-04-25T10:05:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.587+0000 7f3a97bd8640 1 -- 172.21.15.60:0/2952863690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a74066f90 msgr2=0x7f3a74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.587+0000 7f3a97bd8640 1 --2- 172.21.15.60:0/2952863690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a74066f90 0x7f3a74069450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3a8801b120 tx=0x7f3a88004ac0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.587+0000 7f3a97bd8640 1 -- 172.21.15.60:0/2952863690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a900cf100 msgr2=0x7f3a90153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.587+0000 7f3a97bd8640 1 --2- 172.21.15.60:0/2952863690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a900cf100 0x7f3a90153550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c000f70 tx=0x7f3a8c00c890 comp rx=0 tx=0).stop 2024-04-25T10:05:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.587+0000 7f3a97bd8640 1 -- 172.21.15.60:0/2952863690 shutdown_connections 2024-04-25T10:05:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.587+0000 7f3a97bd8640 1 --2- 172.21.15.60:0/2952863690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a74066f90 0x7f3a74069450 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.587+0000 7f3a97bd8640 1 --2- 172.21.15.60:0/2952863690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a90140220 0x7f3a90159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.587+0000 7f3a97bd8640 1 --2- 172.21.15.60:0/2952863690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a900cf100 0x7f3a90153550 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.587+0000 7f3a97bd8640 1 --2- 172.21.15.60:0/2952863690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a900719a0 0x7f3a901588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.587+0000 7f3a97bd8640 1 -- 172.21.15.60:0/2952863690 >> 172.21.15.60:0/2952863690 conn(0x7f3a900b12d0 msgr2=0x7f3a9006d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.587+0000 7f3a97bd8640 1 -- 172.21.15.60:0/2952863690 shutdown_connections 2024-04-25T10:05:30.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.587+0000 7f3a97bd8640 1 -- 172.21.15.60:0/2952863690 wait complete. 2024-04-25T10:05:30.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.3 2024-04-25T10:05:30.598 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:30.612 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.6 2024-04-25T10:05:30.612 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:30.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542152 for osd.0 2024-04-25T10:05:30.617 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:30.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.611+0000 7f2b9cc58640 1 -- 172.21.15.60:0/1193121312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2b7c005860 con 0x7f2b900d7ec0 2024-04-25T10:05:30.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.611+0000 7f2b86ffd640 1 -- 172.21.15.60:0/1193121312 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2b980ff0d0 con 0x7f2b900d7ec0 2024-04-25T10:05:30.619 INFO:teuthology.orchestra.run.smithi060.stdout:60129542151 2024-04-25T10:05:30.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.611+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2752229602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f6fd014c120 con 0x7f6fd00cefc0 2024-04-25T10:05:30.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.611+0000 7f6fcd7fa640 1 -- 172.21.15.60:0/2752229602 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6fbc059950 con 0x7f6fd00cefc0 2024-04-25T10:05:30.620 INFO:teuthology.orchestra.run.smithi060.stdout:64424509448 2024-04-25T10:05:30.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f2b9cc58640 1 -- 172.21.15.60:0/1193121312 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b78066f60 msgr2=0x7f2b78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f2b9cc58640 1 --2- 172.21.15.60:0/1193121312 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b78066f60 0x7f2b78069420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009ac0 tx=0x7f2b8c009190 comp rx=0 tx=0).stop 2024-04-25T10:05:30.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f2b9cc58640 1 -- 172.21.15.60:0/1193121312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b900d7ec0 msgr2=0x7f2b900d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f2b9cc58640 1 --2- 172.21.15.60:0/1193121312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b900d7ec0 0x7f2b900d17f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2b9806a4c0 tx=0x7f2b9804ff80 comp rx=0 tx=0).stop 2024-04-25T10:05:30.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f2b9cc58640 1 -- 172.21.15.60:0/1193121312 shutdown_connections 2024-04-25T10:05:30.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f2b9cc58640 1 --2- 172.21.15.60:0/1193121312 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b78066f60 0x7f2b78069420 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f2b9cc58640 1 --2- 172.21.15.60:0/1193121312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b900d7ec0 0x7f2b900d17f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f2b9cc58640 1 --2- 172.21.15.60:0/1193121312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b90055260 0x7f2b9000c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f2b9cc58640 1 --2- 172.21.15.60:0/1193121312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b90052ad0 0x7f2b900d9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f2b9cc58640 1 -- 172.21.15.60:0/1193121312 >> 172.21.15.60:0/1193121312 conn(0x7f2b9001ef50 msgr2=0x7f2b9000b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f2b9cc58640 1 -- 172.21.15.60:0/1193121312 shutdown_connections 2024-04-25T10:05:30.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f2b9cc58640 1 -- 172.21.15.60:0/1193121312 wait complete. 2024-04-25T10:05:30.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2752229602 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6fb4066f90 msgr2=0x7f6fb4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f6fd6dfa640 1 --2- 172.21.15.60:0/2752229602 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6fb4066f90 0x7f6fb4069450 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40af000 tx=0x7f6fc4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:30.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2752229602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00cefc0 msgr2=0x7f6fd01434e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f6fd6dfa640 1 --2- 172.21.15.60:0/2752229602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00cefc0 0x7f6fd01434e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00e890 tx=0x7f6fbc00ed50 comp rx=0 tx=0).stop 2024-04-25T10:05:30.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2752229602 shutdown_connections 2024-04-25T10:05:30.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f6fd6dfa640 1 --2- 172.21.15.60:0/2752229602 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6fb4066f90 0x7f6fb4069450 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f6fd6dfa640 1 --2- 172.21.15.60:0/2752229602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fd01465e0 0x7f6fd014e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f6fd6dfa640 1 --2- 172.21.15.60:0/2752229602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fd00d0ec0 0x7f6fd014dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f6fd6dfa640 1 --2- 172.21.15.60:0/2752229602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fd00cefc0 0x7f6fd01434e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2752229602 >> 172.21.15.60:0/2752229602 conn(0x7f6fd00b3520 msgr2=0x7f6fd00d1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2752229602 shutdown_connections 2024-04-25T10:05:30.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.615+0000 7f6fd6dfa640 1 -- 172.21.15.60:0/2752229602 wait complete. 2024-04-25T10:05:30.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.623+0000 7fd6d4472640 1 -- 172.21.15.60:0/3597915271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd6cc06d600 con 0x7fd6cc14f2c0 2024-04-25T10:05:30.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.623+0000 7fd6c2ffd640 1 -- 172.21.15.60:0/3597915271 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd6cc153b80 con 0x7fd6cc14f2c0 2024-04-25T10:05:30.631 INFO:teuthology.orchestra.run.smithi060.stdout:60129542151 2024-04-25T10:05:30.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.627+0000 7fd6d4472640 1 -- 172.21.15.60:0/3597915271 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd6b0066f60 msgr2=0x7fd6b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.627+0000 7fd6d4472640 1 --2- 172.21.15.60:0/3597915271 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd6b0066f60 0x7fd6b0069420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009920 tx=0x7fd6c8009110 comp rx=0 tx=0).stop 2024-04-25T10:05:30.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.627+0000 7fd6d4472640 1 -- 172.21.15.60:0/3597915271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6cc14f2c0 msgr2=0x7fd6cc165490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.627+0000 7fd6d4472640 1 --2- 172.21.15.60:0/3597915271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6cc14f2c0 0x7fd6cc165490 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc06fa80 tx=0x7fd6c400cb90 comp rx=0 tx=0).stop 2024-04-25T10:05:30.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.627+0000 7fd6d4472640 1 -- 172.21.15.60:0/3597915271 shutdown_connections 2024-04-25T10:05:30.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.627+0000 7fd6d4472640 1 --2- 172.21.15.60:0/3597915271 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd6b0066f60 0x7fd6b0069420 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.627+0000 7fd6d4472640 1 --2- 172.21.15.60:0/3597915271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6cc14f2c0 0x7fd6cc165490 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.627+0000 7fd6d4472640 1 --2- 172.21.15.60:0/3597915271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6cc153700 0x7fd6cc156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.627+0000 7fd6d4472640 1 --2- 172.21.15.60:0/3597915271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b8004810 0x7fd6cc151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.627+0000 7fd6d4472640 1 -- 172.21.15.60:0/3597915271 >> 172.21.15.60:0/3597915271 conn(0x7fd6cc06b4c0 msgr2=0x7fd6cc0cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.627+0000 7fd6d4472640 1 -- 172.21.15.60:0/3597915271 shutdown_connections 2024-04-25T10:05:30.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.627+0000 7fd6d4472640 1 -- 172.21.15.60:0/3597915271 wait complete. 2024-04-25T10:05:30.646 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.5 2024-04-25T10:05:30.647 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:30.647 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.7 2024-04-25T10:05:30.648 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:30.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.639+0000 7f6e83c40640 1 -- 172.21.15.60:0/2091619977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6e7c074770 con 0x7f6e7c0adb80 2024-04-25T10:05:30.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.643+0000 7f6e727fc640 1 -- 172.21.15.60:0/2091619977 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6e74124090 con 0x7f6e7c0adb80 2024-04-25T10:05:30.648 INFO:teuthology.orchestra.run.smithi060.stdout:60129542151 2024-04-25T10:05:30.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.643+0000 7f6e83c40640 1 -- 172.21.15.60:0/2091619977 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6e60067060 msgr2=0x7f6e60069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.643+0000 7f6e83c40640 1 --2- 172.21.15.60:0/2091619977 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6e60067060 0x7f6e60069520 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6e780079e0 tx=0x7f6e78007950 comp rx=0 tx=0).stop 2024-04-25T10:05:30.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.643+0000 7f6e83c40640 1 -- 172.21.15.60:0/2091619977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c0adb80 msgr2=0x7f6e7c0ae010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.643+0000 7f6e83c40640 1 --2- 172.21.15.60:0/2091619977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c0adb80 0x7f6e7c0ae010 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6e7401f8d0 tx=0x7f6e68005840 comp rx=0 tx=0).stop 2024-04-25T10:05:30.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.643+0000 7f6e83c40640 1 -- 172.21.15.60:0/2091619977 shutdown_connections 2024-04-25T10:05:30.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.643+0000 7f6e83c40640 1 --2- 172.21.15.60:0/2091619977 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6e60067060 0x7f6e60069520 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.643+0000 7f6e83c40640 1 --2- 172.21.15.60:0/2091619977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c0adb80 0x7f6e7c0ae010 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.643+0000 7f6e83c40640 1 --2- 172.21.15.60:0/2091619977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7c149e90 0x7f6e7c0ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.643+0000 7f6e83c40640 1 --2- 172.21.15.60:0/2091619977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e68003cf0 0x7f6e7c15d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.643+0000 7f6e83c40640 1 -- 172.21.15.60:0/2091619977 >> 172.21.15.60:0/2091619977 conn(0x7f6e7c072920 msgr2=0x7f6e7c153ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.643+0000 7f6e83c40640 1 -- 172.21.15.60:0/2091619977 shutdown_connections 2024-04-25T10:05:30.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.643+0000 7f6e83c40640 1 -- 172.21.15.60:0/2091619977 wait complete. 2024-04-25T10:05:30.660 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.2 2024-04-25T10:05:30.661 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:30.667 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.1 2024-04-25T10:05:30.667 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:30.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.667+0000 7fba26d17640 1 -- 172.21.15.60:0/271683949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fba08005590 con 0x7fba20157f20 2024-04-25T10:05:30.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.667+0000 7fba167fc640 1 -- 172.21.15.60:0/271683949 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba18055a60 con 0x7fba20157f20 2024-04-25T10:05:30.676 INFO:teuthology.orchestra.run.smithi060.stdout:64424509447 2024-04-25T10:05:30.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.671+0000 7fba26d17640 1 -- 172.21.15.60:0/271683949 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fba0c06b4c0 msgr2=0x7fba0c06d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.671+0000 7fba26d17640 1 --2- 172.21.15.60:0/271683949 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fba0c06b4c0 0x7fba0c06d980 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fba10007a70 tx=0x7fba10008040 comp rx=0 tx=0).stop 2024-04-25T10:05:30.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.671+0000 7fba26d17640 1 -- 172.21.15.60:0/271683949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba20157f20 msgr2=0x7fba20151460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.671+0000 7fba26d17640 1 --2- 172.21.15.60:0/271683949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba20157f20 0x7fba20151460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fba1800c7b0 tx=0x7fba1800cc70 comp rx=0 tx=0).stop 2024-04-25T10:05:30.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.671+0000 7fba26d17640 1 -- 172.21.15.60:0/271683949 shutdown_connections 2024-04-25T10:05:30.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.671+0000 7fba26d17640 1 --2- 172.21.15.60:0/271683949 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fba0c06b4c0 0x7fba0c06d980 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.671+0000 7fba26d17640 1 --2- 172.21.15.60:0/271683949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba20157f20 0x7fba20151460 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.671+0000 7fba26d17640 1 --2- 172.21.15.60:0/271683949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba200b21e0 0x7fba20145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.671+0000 7fba26d17640 1 --2- 172.21.15.60:0/271683949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba200afcb0 0x7fba2015a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.671+0000 7fba26d17640 1 -- 172.21.15.60:0/271683949 >> 172.21.15.60:0/271683949 conn(0x7fba2006e390 msgr2=0x7fba2006e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.671+0000 7fba26d17640 1 -- 172.21.15.60:0/271683949 shutdown_connections 2024-04-25T10:05:30.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.671+0000 7fba26d17640 1 -- 172.21.15.60:0/271683949 wait complete. 2024-04-25T10:05:30.692 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.4 2024-04-25T10:05:30.692 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:30.692 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:05:30.692 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:30.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.835+0000 7f72be2fb640 1 Processor -- start 2024-04-25T10:05:30.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.839+0000 7f72be2fb640 1 -- start start 2024-04-25T10:05:30.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.839+0000 7f72be2fb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f72b80ce7a0 0x7f72b80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.839+0000 7f72be2fb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b80b33f0 con 0x7f72b80b4660 2024-04-25T10:05:30.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.839+0000 7f72be2fb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b80cf0c0 con 0x7f72b815bc90 2024-04-25T10:05:30.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.839+0000 7f72be2fb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b80cc370 con 0x7f72b80ce7a0 2024-04-25T10:05:30.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.839+0000 7f72b7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f72b80ce7a0 0x7f72b80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.839+0000 7f72bc898640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f72b80b4660 0x7f72b80b2ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37292/0 (socket says 172.21.15.60:37292) 2024-04-25T10:05:30.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.839+0000 7f72bc898640 1 -- 172.21.15.60:0/975717666 learned_addr learned my addr 172.21.15.60:0/975717666 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:30.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.839+0000 7f72b7fff640 1 -- 172.21.15.60:0/975717666 >> v1:172.21.15.100:6789/0 conn(0x7f72b815bc90 legacy=0x7f72b80b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.839+0000 7f72b7fff640 1 -- 172.21.15.60:0/975717666 >> v1:172.21.15.60:6789/0 conn(0x7f72b80b4660 legacy=0x7f72b80b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.839+0000 7f72b7fff640 1 -- 172.21.15.60:0/975717666 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b8071ca0 con 0x7f72b80ce7a0 2024-04-25T10:05:30.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.839+0000 7f72b7fff640 1 --2- 172.21.15.60:0/975717666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80ce7a0 0x7f72b80ceb80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f72a80097c0 tx=0x7f72a801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9613f175969153b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.839+0000 7f72b6ffd640 1 -- 172.21.15.60:0/975717666 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a80b0070 con 0x7f72b80ce7a0 2024-04-25T10:05:30.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.839+0000 7f72b6ffd640 1 -- 172.21.15.60:0/975717666 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a80ab600 con 0x7f72b80ce7a0 2024-04-25T10:05:30.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.839+0000 7f72b6ffd640 1 -- 172.21.15.60:0/975717666 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a80ab9d0 con 0x7f72b80ce7a0 2024-04-25T10:05:30.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 -- 172.21.15.60:0/975717666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80ce7a0 msgr2=0x7f72b80ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 --2- 172.21.15.60:0/975717666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80ce7a0 0x7f72b80ceb80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f72a80097c0 tx=0x7f72a801c460 comp rx=0 tx=0).stop 2024-04-25T10:05:30.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 -- 172.21.15.60:0/975717666 shutdown_connections 2024-04-25T10:05:30.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 --2- 172.21.15.60:0/975717666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80ce7a0 0x7f72b80ceb80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 -- 172.21.15.60:0/975717666 >> 172.21.15.60:0/975717666 conn(0x7f72b80b12d0 msgr2=0x7f72b80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 -- 172.21.15.60:0/975717666 shutdown_connections 2024-04-25T10:05:30.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 -- 172.21.15.60:0/975717666 wait complete. 2024-04-25T10:05:30.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 Processor -- start 2024-04-25T10:05:30.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 -- start start 2024-04-25T10:05:30.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72b80ce7a0 0x7f72b80ac440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72b815bc90 0x7f72b815a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b815aa10 0x7f72b815aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b80b3d20 con 0x7f72b815aa10 2024-04-25T10:05:30.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b80b3ba0 con 0x7f72b80ce7a0 2024-04-25T10:05:30.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b80b4280 con 0x7f72b815bc90 2024-04-25T10:05:30.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72bc898640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b815aa10 0x7f72b815aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72bc898640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b815aa10 0x7f72b815aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46516/0 (socket says 172.21.15.60:46516) 2024-04-25T10:05:30.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72bc898640 1 -- 172.21.15.60:0/3508366583 learned_addr learned my addr 172.21.15.60:0/3508366583 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:30.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72bc898640 1 -- 172.21.15.60:0/3508366583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72b815bc90 msgr2=0x7f72b815a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72bc898640 1 --2- 172.21.15.60:0/3508366583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72b815bc90 0x7f72b815a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72b7fff640 1 --2- 172.21.15.60:0/3508366583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72b80ce7a0 0x7f72b80ac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72bc898640 1 -- 172.21.15.60:0/3508366583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72b80ce7a0 msgr2=0x7f72b80ac440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72bc898640 1 --2- 172.21.15.60:0/3508366583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72b80ce7a0 0x7f72b80ac440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72bc898640 1 -- 172.21.15.60:0/3508366583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b80ad3e0 con 0x7f72b815aa10 2024-04-25T10:05:30.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72bc898640 1 --2- 172.21.15.60:0/3508366583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b815aa10 0x7f72b815aec0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f72b000caa0 tx=0x7f72b000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72b4ff9640 1 -- 172.21.15.60:0/3508366583 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0017070 con 0x7f72b815aa10 2024-04-25T10:05:30.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72b4ff9640 1 -- 172.21.15.60:0/3508366583 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b000ac00 con 0x7f72b815aa10 2024-04-25T10:05:30.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 -- 172.21.15.60:0/3508366583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b80acb40 con 0x7f72b815aa10 2024-04-25T10:05:30.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72b4ff9640 1 -- 172.21.15.60:0/3508366583 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0012620 con 0x7f72b815aa10 2024-04-25T10:05:30.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.843+0000 7f72be2fb640 1 -- 172.21.15.60:0/3508366583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b80ad100 con 0x7f72b815aa10 2024-04-25T10:05:30.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.847+0000 7f72be2fb640 1 -- 172.21.15.60:0/3508366583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b8074770 con 0x7f72b815aa10 2024-04-25T10:05:30.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.847+0000 7f72b4ff9640 1 -- 172.21.15.60:0/3508366583 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72b001a070 con 0x7f72b815aa10 2024-04-25T10:05:30.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.847+0000 7f72b4ff9640 1 --2- 172.21.15.60:0/3508366583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f72a4066f60 0x7f72a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.847+0000 7f72b4ff9640 1 -- 172.21.15.60:0/3508366583 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72b0085130 con 0x7f72b815aa10 2024-04-25T10:05:30.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.847+0000 7f72b7fff640 1 --2- 172.21.15.60:0/3508366583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f72a4066f60 0x7f72a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.847+0000 7f72b7fff640 1 --2- 172.21.15.60:0/3508366583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f72a4066f60 0x7f72a4069420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f72a80ad000 tx=0x7f72a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.851+0000 7f72b4ff9640 1 -- 172.21.15.60:0/3508366583 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f72b0021020 con 0x7f72b815aa10 2024-04-25T10:05:31.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:30.999+0000 7f72be2fb640 1 -- 172.21.15.60:0/3508366583 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f72b80ad5b0 con 0x7f72a4066f60 2024-04-25T10:05:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.019+0000 7f72b4ff9640 1 -- 172.21.15.60:0/3508366583 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7f72b80ad5b0 con 0x7f72a4066f60 2024-04-25T10:05:31.029 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:31.032 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:05:31.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.023+0000 7f72be2fb640 1 -- 172.21.15.60:0/3508366583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f72a4066f60 msgr2=0x7f72a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.023+0000 7f72be2fb640 1 --2- 172.21.15.60:0/3508366583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f72a4066f60 0x7f72a4069420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f72a80ad000 tx=0x7f72a8008040 comp rx=0 tx=0).stop 2024-04-25T10:05:31.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.023+0000 7f72be2fb640 1 -- 172.21.15.60:0/3508366583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b815aa10 msgr2=0x7f72b815aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.023+0000 7f72be2fb640 1 --2- 172.21.15.60:0/3508366583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b815aa10 0x7f72b815aec0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f72b000caa0 tx=0x7f72b000cf60 comp rx=0 tx=0).stop 2024-04-25T10:05:31.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.023+0000 7f72be2fb640 1 -- 172.21.15.60:0/3508366583 shutdown_connections 2024-04-25T10:05:31.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.023+0000 7f72be2fb640 1 --2- 172.21.15.60:0/3508366583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f72a4066f60 0x7f72a4069420 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.023+0000 7f72be2fb640 1 --2- 172.21.15.60:0/3508366583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b815aa10 0x7f72b815aec0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.023+0000 7f72be2fb640 1 --2- 172.21.15.60:0/3508366583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72b815bc90 0x7f72b815a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.023+0000 7f72be2fb640 1 --2- 172.21.15.60:0/3508366583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72b80ce7a0 0x7f72b80ac440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.023+0000 7f72be2fb640 1 -- 172.21.15.60:0/3508366583 >> 172.21.15.60:0/3508366583 conn(0x7f72b80b12d0 msgr2=0x7f72b80714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.023+0000 7f72be2fb640 1 -- 172.21.15.60:0/3508366583 shutdown_connections 2024-04-25T10:05:31.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.023+0000 7f72be2fb640 1 -- 172.21.15.60:0/3508366583 wait complete. 2024-04-25T10:05:31.045 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-25T10:05:30.389303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235768,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738952,"statfs":{"total":773094113280,"available":772852686848,"internally_reserved":0,"allocated":3022848,"data_stored":1811156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.347160"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137405+0000","last_change":"2024-04-25T10:05:21.093662+0000","last_active":"2024-04-25T10:05:22.137405+0000","last_peered":"2024-04-25T10:05:22.137405+0000","last_clean":"2024-04-25T10:05:22.137405+0000","last_became_active":"2024-04-25T10:05:21.093342+0000","last_became_peered":"2024-04-25T10:05:21.093342+0000","last_unstale":"2024-04-25T10:05:22.137405+0000","last_undegraded":"2024-04-25T10:05:22.137405+0000","last_fullsized":"2024-04-25T10:05:22.137405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:38.452914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043321+0000","last_change":"2024-04-25T10:05:21.076738+0000","last_active":"2024-04-25T10:05:22.043321+0000","last_peered":"2024-04-25T10:05:22.043321+0000","last_clean":"2024-04-25T10:05:22.043321+0000","last_became_active":"2024-04-25T10:05:21.076085+0000","last_became_peered":"2024-04-25T10:05:21.076085+0000","last_unstale":"2024-04-25T10:05:22.043321+0000","last_undegraded":"2024-04-25T10:05:22.043321+0000","last_fullsized":"2024-04-25T10:05:22.043321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:36.826815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136181+0000","last_change":"2024-04-25T10:05:21.072290+0000","last_active":"2024-04-25T10:05:22.136181+0000","last_peered":"2024-04-25T10:05:22.136181+0000","last_clean":"2024-04-25T10:05:22.136181+0000","last_became_active":"2024-04-25T10:05:21.069763+0000","last_became_peered":"2024-04-25T10:05:21.069763+0000","last_unstale":"2024-04-25T10:05:22.136181+0000","last_undegraded":"2024-04-25T10:05:22.136181+0000","last_fullsized":"2024-04-25T10:05:22.136181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:17.144179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048606+0000","last_change":"2024-04-25T10:05:21.083631+0000","last_active":"2024-04-25T10:05:22.048606+0000","last_peered":"2024-04-25T10:05:22.048606+0000","last_clean":"2024-04-25T10:05:22.048606+0000","last_became_active":"2024-04-25T10:05:21.082783+0000","last_became_peered":"2024-04-25T10:05:21.082783+0000","last_unstale":"2024-04-25T10:05:22.048606+0000","last_undegraded":"2024-04-25T10:05:22.048606+0000","last_fullsized":"2024-04-25T10:05:22.048606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:02.040066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136043+0000","last_change":"2024-04-25T10:05:21.092602+0000","last_active":"2024-04-25T10:05:22.136043+0000","last_peered":"2024-04-25T10:05:22.136043+0000","last_clean":"2024-04-25T10:05:22.136043+0000","last_became_active":"2024-04-25T10:05:21.089559+0000","last_became_peered":"2024-04-25T10:05:21.089559+0000","last_unstale":"2024-04-25T10:05:22.136043+0000","last_undegraded":"2024-04-25T10:05:22.136043+0000","last_fullsized":"2024-04-25T10:05:22.136043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:57.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046962+0000","last_change":"2024-04-25T10:05:21.078938+0000","last_active":"2024-04-25T10:05:22.046962+0000","last_peered":"2024-04-25T10:05:22.046962+0000","last_clean":"2024-04-25T10:05:22.046962+0000","last_became_active":"2024-04-25T10:05:21.078310+0000","last_became_peered":"2024-04-25T10:05:21.078310+0000","last_unstale":"2024-04-25T10:05:22.046962+0000","last_undegraded":"2024-04-25T10:05:22.046962+0000","last_fullsized":"2024-04-25T10:05:22.046962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:30.299159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135741+0000","last_change":"2024-04-25T10:05:21.086161+0000","last_active":"2024-04-25T10:05:22.135741+0000","last_peered":"2024-04-25T10:05:22.135741+0000","last_clean":"2024-04-25T10:05:22.135741+0000","last_became_active":"2024-04-25T10:05:21.084217+0000","last_became_peered":"2024-04-25T10:05:21.084217+0000","last_unstale":"2024-04-25T10:05:22.135741+0000","last_undegraded":"2024-04-25T10:05:22.135741+0000","last_fullsized":"2024-04-25T10:05:22.135741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:29.451932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.133417+0000","last_change":"2024-04-25T10:05:21.071738+0000","last_active":"2024-04-25T10:05:22.133417+0000","last_peered":"2024-04-25T10:05:22.133417+0000","last_clean":"2024-04-25T10:05:22.133417+0000","last_became_active":"2024-04-25T10:05:21.071239+0000","last_became_peered":"2024-04-25T10:05:21.071239+0000","last_unstale":"2024-04-25T10:05:22.133417+0000","last_undegraded":"2024-04-25T10:05:22.133417+0000","last_fullsized":"2024-04-25T10:05:22.133417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:13.318906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137424+0000","last_change":"2024-04-25T10:05:21.089854+0000","last_active":"2024-04-25T10:05:22.137424+0000","last_peered":"2024-04-25T10:05:22.137424+0000","last_clean":"2024-04-25T10:05:22.137424+0000","last_became_active":"2024-04-25T10:05:21.089145+0000","last_became_peered":"2024-04-25T10:05:21.089145+0000","last_unstale":"2024-04-25T10:05:22.137424+0000","last_undegraded":"2024-04-25T10:05:22.137424+0000","last_fullsized":"2024-04-25T10:05:22.137424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:17.168463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.081237+0000","last_change":"2024-04-25T10:05:21.081763+0000","last_active":"2024-04-25T10:05:22.081237+0000","last_peered":"2024-04-25T10:05:22.081237+0000","last_clean":"2024-04-25T10:05:22.081237+0000","last_became_active":"2024-04-25T10:05:21.081398+0000","last_became_peered":"2024-04-25T10:05:21.081398+0000","last_unstale":"2024-04-25T10:05:22.081237+0000","last_undegraded":"2024-04-25T10:05:22.081237+0000","last_fullsized":"2024-04-25T10:05:22.081237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:23.519531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080566+0000","last_change":"2024-04-25T10:05:21.081165+0000","last_active":"2024-04-25T10:05:22.080566+0000","last_peered":"2024-04-25T10:05:22.080566+0000","last_clean":"2024-04-25T10:05:22.080566+0000","last_became_active":"2024-04-25T10:05:21.076196+0000","last_became_peered":"2024-04-25T10:05:21.076196+0000","last_unstale":"2024-04-25T10:05:22.080566+0000","last_undegraded":"2024-04-25T10:05:22.080566+0000","last_fullsized":"2024-04-25T10:05:22.080566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:36.529873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048072+0000","last_change":"2024-04-25T10:05:21.082653+0000","last_active":"2024-04-25T10:05:22.048072+0000","last_peered":"2024-04-25T10:05:22.048072+0000","last_clean":"2024-04-25T10:05:22.048072+0000","last_became_active":"2024-04-25T10:05:21.082004+0000","last_became_peered":"2024-04-25T10:05:21.082004+0000","last_unstale":"2024-04-25T10:05:22.048072+0000","last_undegraded":"2024-04-25T10:05:22.048072+0000","last_fullsized":"2024-04-25T10:05:22.048072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:56.267445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136198+0000","last_change":"2024-04-25T10:05:21.071024+0000","last_active":"2024-04-25T10:05:22.136198+0000","last_peered":"2024-04-25T10:05:22.136198+0000","last_clean":"2024-04-25T10:05:22.136198+0000","last_became_active":"2024-04-25T10:05:21.070048+0000","last_became_peered":"2024-04-25T10:05:21.070048+0000","last_unstale":"2024-04-25T10:05:22.136198+0000","last_undegraded":"2024-04-25T10:05:22.136198+0000","last_fullsized":"2024-04-25T10:05:22.136198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:00.583322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134119+0000","last_change":"2024-04-25T10:05:21.080794+0000","last_active":"2024-04-25T10:05:22.134119+0000","last_peered":"2024-04-25T10:05:22.134119+0000","last_clean":"2024-04-25T10:05:22.134119+0000","last_became_active":"2024-04-25T10:05:21.079418+0000","last_became_peered":"2024-04-25T10:05:21.079418+0000","last_unstale":"2024-04-25T10:05:22.134119+0000","last_undegraded":"2024-04-25T10:05:22.134119+0000","last_fullsized":"2024-04-25T10:05:22.134119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:09.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042828+0000","last_change":"2024-04-25T10:05:21.055371+0000","last_active":"2024-04-25T10:05:22.042828+0000","last_peered":"2024-04-25T10:05:22.042828+0000","last_clean":"2024-04-25T10:05:22.042828+0000","last_became_active":"2024-04-25T10:05:21.054565+0000","last_became_peered":"2024-04-25T10:05:21.054565+0000","last_unstale":"2024-04-25T10:05:22.042828+0000","last_undegraded":"2024-04-25T10:05:22.042828+0000","last_fullsized":"2024-04-25T10:05:22.042828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:31.511781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046177+0000","last_change":"2024-04-25T10:05:21.079756+0000","last_active":"2024-04-25T10:05:22.046177+0000","last_peered":"2024-04-25T10:05:22.046177+0000","last_clean":"2024-04-25T10:05:22.046177+0000","last_became_active":"2024-04-25T10:05:21.078988+0000","last_became_peered":"2024-04-25T10:05:21.078988+0000","last_unstale":"2024-04-25T10:05:22.046177+0000","last_undegraded":"2024-04-25T10:05:22.046177+0000","last_fullsized":"2024-04-25T10:05:22.046177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:47.724037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.049473+0000","last_change":"2024-04-25T10:05:21.087275+0000","last_active":"2024-04-25T10:05:22.049473+0000","last_peered":"2024-04-25T10:05:22.049473+0000","last_clean":"2024-04-25T10:05:22.049473+0000","last_became_active":"2024-04-25T10:05:21.086912+0000","last_became_peered":"2024-04-25T10:05:21.086912+0000","last_unstale":"2024-04-25T10:05:22.049473+0000","last_undegraded":"2024-04-25T10:05:22.049473+0000","last_fullsized":"2024-04-25T10:05:22.049473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:01.067797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134822+0000","last_change":"2024-04-25T10:05:21.085565+0000","last_active":"2024-04-25T10:05:22.134822+0000","last_peered":"2024-04-25T10:05:22.134822+0000","last_clean":"2024-04-25T10:05:22.134822+0000","last_became_active":"2024-04-25T10:05:21.084852+0000","last_became_peered":"2024-04-25T10:05:21.084852+0000","last_unstale":"2024-04-25T10:05:22.134822+0000","last_undegraded":"2024-04-25T10:05:22.134822+0000","last_fullsized":"2024-04-25T10:05:22.134822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:51.076186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136820+0000","last_change":"2024-04-25T10:05:21.089683+0000","last_active":"2024-04-25T10:05:22.136820+0000","last_peered":"2024-04-25T10:05:22.136820+0000","last_clean":"2024-04-25T10:05:22.136820+0000","last_became_active":"2024-04-25T10:05:21.089192+0000","last_became_peered":"2024-04-25T10:05:21.089192+0000","last_unstale":"2024-04-25T10:05:22.136820+0000","last_undegraded":"2024-04-25T10:05:22.136820+0000","last_fullsized":"2024-04-25T10:05:22.136820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:50.393309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080406+0000","last_change":"2024-04-25T10:05:21.072923+0000","last_active":"2024-04-25T10:05:22.080406+0000","last_peered":"2024-04-25T10:05:22.080406+0000","last_clean":"2024-04-25T10:05:22.080406+0000","last_became_active":"2024-04-25T10:05:21.072600+0000","last_became_peered":"2024-04-25T10:05:21.072600+0000","last_unstale":"2024-04-25T10:05:22.080406+0000","last_undegraded":"2024-04-25T10:05:22.080406+0000","last_fullsized":"2024-04-25T10:05:22.080406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:33.904915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135126+0000","last_change":"2024-04-25T10:05:21.092396+0000","last_active":"2024-04-25T10:05:22.135126+0000","last_peered":"2024-04-25T10:05:22.135126+0000","last_clean":"2024-04-25T10:05:22.135126+0000","last_became_active":"2024-04-25T10:05:21.092118+0000","last_became_peered":"2024-04-25T10:05:21.092118+0000","last_unstale":"2024-04-25T10:05:22.135126+0000","last_undegraded":"2024-04-25T10:05:22.135126+0000","last_fullsized":"2024-04-25T10:05:22.135126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:33.904915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047430+0000","last_change":"2024-04-25T10:05:21.079563+0000","last_active":"2024-04-25T10:05:22.047430+0000","last_peered":"2024-04-25T10:05:22.047430+0000","last_clean":"2024-04-25T10:05:22.047430+0000","last_became_active":"2024-04-25T10:05:21.079032+0000","last_became_peered":"2024-04-25T10:05:21.079032+0000","last_unstale":"2024-04-25T10:05:22.047430+0000","last_undegraded":"2024-04-25T10:05:22.047430+0000","last_fullsized":"2024-04-25T10:05:22.047430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:02.817716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135891+0000","last_change":"2024-04-25T10:05:21.071825+0000","last_active":"2024-04-25T10:05:22.135891+0000","last_peered":"2024-04-25T10:05:22.135891+0000","last_clean":"2024-04-25T10:05:22.135891+0000","last_became_active":"2024-04-25T10:05:21.071207+0000","last_became_peered":"2024-04-25T10:05:21.071207+0000","last_unstale":"2024-04-25T10:05:22.135891+0000","last_undegraded":"2024-04-25T10:05:22.135891+0000","last_fullsized":"2024-04-25T10:05:22.135891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:23.519531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046222+0000","last_change":"2024-04-25T10:05:21.083347+0000","last_active":"2024-04-25T10:05:22.046222+0000","last_peered":"2024-04-25T10:05:22.046222+0000","last_clean":"2024-04-25T10:05:22.046222+0000","last_became_active":"2024-04-25T10:05:21.082648+0000","last_became_peered":"2024-04-25T10:05:21.082648+0000","last_unstale":"2024-04-25T10:05:22.046222+0000","last_undegraded":"2024-04-25T10:05:22.046222+0000","last_fullsized":"2024-04-25T10:05:22.046222+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135886+0000","last_change":"2024-04-25T10:05:21.069597+0000","last_active":"2024-04-25T10:05:22.135886+0000","last_peered":"2024-04-25T10:05:22.135886+0000","last_clean":"2024-04-25T10:05:22.135886+0000","last_became_active":"2024-04-25T10:05:21.064771+0000","last_became_peered":"2024-04-25T10:05:21.064771+0000","last_unstale":"2024-04-25T10:05:22.135886+0000","last_undegraded":"2024-04-25T10:05:22.135886+0000","last_fullsized":"2024-04-25T10:05:22.135886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:36.529873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.081502+0000","last_change":"2024-04-25T10:05:21.090548+0000","last_active":"2024-04-25T10:05:22.081502+0000","last_peered":"2024-04-25T10:05:22.081502+0000","last_clean":"2024-04-25T10:05:22.081502+0000","last_became_active":"2024-04-25T10:05:21.090195+0000","last_became_peered":"2024-04-25T10:05:21.090195+0000","last_unstale":"2024-04-25T10:05:22.081502+0000","last_undegraded":"2024-04-25T10:05:22.081502+0000","last_fullsized":"2024-04-25T10:05:22.081502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:00.583322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043216+0000","last_change":"2024-04-25T10:05:21.078928+0000","last_active":"2024-04-25T10:05:22.043216+0000","last_peered":"2024-04-25T10:05:22.043216+0000","last_clean":"2024-04-25T10:05:22.043216+0000","last_became_active":"2024-04-25T10:05:21.078521+0000","last_became_peered":"2024-04-25T10:05:21.078521+0000","last_unstale":"2024-04-25T10:05:22.043216+0000","last_undegraded":"2024-04-25T10:05:22.043216+0000","last_fullsized":"2024-04-25T10:05:22.043216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:03.085314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136663+0000","last_change":"2024-04-25T10:05:21.063167+0000","last_active":"2024-04-25T10:05:22.136663+0000","last_peered":"2024-04-25T10:05:22.136663+0000","last_clean":"2024-04-25T10:05:22.136663+0000","last_became_active":"2024-04-25T10:05:21.061503+0000","last_became_peered":"2024-04-25T10:05:21.061503+0000","last_unstale":"2024-04-25T10:05:22.136663+0000","last_undegraded":"2024-04-25T10:05:22.136663+0000","last_fullsized":"2024-04-25T10:05:22.136663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:30.299159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135212+0000","last_change":"2024-04-25T10:05:21.080730+0000","last_active":"2024-04-25T10:05:22.135212+0000","last_peered":"2024-04-25T10:05:22.135212+0000","last_clean":"2024-04-25T10:05:22.135212+0000","last_became_active":"2024-04-25T10:05:21.080248+0000","last_became_peered":"2024-04-25T10:05:21.080248+0000","last_unstale":"2024-04-25T10:05:22.135212+0000","last_undegraded":"2024-04-25T10:05:22.135212+0000","last_fullsized":"2024-04-25T10:05:22.135212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:36.529873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047524+0000","last_change":"2024-04-25T10:05:21.083357+0000","last_active":"2024-04-25T10:05:22.047524+0000","last_peered":"2024-04-25T10:05:22.047524+0000","last_clean":"2024-04-25T10:05:22.047524+0000","last_became_active":"2024-04-25T10:05:21.082923+0000","last_became_peered":"2024-04-25T10:05:21.082923+0000","last_unstale":"2024-04-25T10:05:22.047524+0000","last_undegraded":"2024-04-25T10:05:22.047524+0000","last_fullsized":"2024-04-25T10:05:22.047524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:50.797661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135432+0000","last_change":"2024-04-25T10:05:21.075647+0000","last_active":"2024-04-25T10:05:22.135432+0000","last_peered":"2024-04-25T10:05:22.135432+0000","last_clean":"2024-04-25T10:05:22.135432+0000","last_became_active":"2024-04-25T10:05:21.074954+0000","last_became_peered":"2024-04-25T10:05:21.074954+0000","last_unstale":"2024-04-25T10:05:22.135432+0000","last_undegraded":"2024-04-25T10:05:22.135432+0000","last_fullsized":"2024-04-25T10:05:22.135432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:55.339069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.079882+0000","last_change":"2024-04-25T10:05:21.084060+0000","last_active":"2024-04-25T10:05:22.079882+0000","last_peered":"2024-04-25T10:05:22.079882+0000","last_clean":"2024-04-25T10:05:22.079882+0000","last_became_active":"2024-04-25T10:05:21.083506+0000","last_became_peered":"2024-04-25T10:05:21.083506+0000","last_unstale":"2024-04-25T10:05:22.079882+0000","last_undegraded":"2024-04-25T10:05:22.079882+0000","last_fullsized":"2024-04-25T10:05:22.079882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:19.908741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046660+0000","last_change":"2024-04-25T10:05:21.065678+0000","last_active":"2024-04-25T10:05:22.046660+0000","last_peered":"2024-04-25T10:05:22.046660+0000","last_clean":"2024-04-25T10:05:22.046660+0000","last_became_active":"2024-04-25T10:05:21.064718+0000","last_became_peered":"2024-04-25T10:05:21.064718+0000","last_unstale":"2024-04-25T10:05:22.046660+0000","last_undegraded":"2024-04-25T10:05:22.046660+0000","last_fullsized":"2024-04-25T10:05:22.046660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:03.887937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136502+0000","last_change":"2024-04-25T10:05:20.083026+0000","last_active":"2024-04-25T10:05:22.136502+0000","last_peered":"2024-04-25T10:05:22.136502+0000","last_clean":"2024-04-25T10:05:22.136502+0000","last_became_active":"2024-04-25T10:05:20.082702+0000","last_became_peered":"2024-04-25T10:05:20.082702+0000","last_unstale":"2024-04-25T10:05:22.136502+0000","last_undegraded":"2024-04-25T10:05:22.136502+0000","last_fullsized":"2024-04-25T10:05:22.136502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:35.252363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135477+0000","last_change":"2024-04-25T10:05:21.086317+0000","last_active":"2024-04-25T10:05:22.135477+0000","last_peered":"2024-04-25T10:05:22.135477+0000","last_clean":"2024-04-25T10:05:22.135477+0000","last_became_active":"2024-04-25T10:05:21.085802+0000","last_became_peered":"2024-04-25T10:05:21.085802+0000","last_unstale":"2024-04-25T10:05:22.135477+0000","last_undegraded":"2024-04-25T10:05:22.135477+0000","last_fullsized":"2024-04-25T10:05:22.135477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:00.583322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080039+0000","last_change":"2024-04-25T10:05:20.080501+0000","last_active":"2024-04-25T10:05:22.080039+0000","last_peered":"2024-04-25T10:05:22.080039+0000","last_clean":"2024-04-25T10:05:22.080039+0000","last_became_active":"2024-04-25T10:05:20.079392+0000","last_became_peered":"2024-04-25T10:05:20.079392+0000","last_unstale":"2024-04-25T10:05:22.080039+0000","last_undegraded":"2024-04-25T10:05:22.080039+0000","last_fullsized":"2024-04-25T10:05:22.080039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:44.794815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136074+0000","last_change":"2024-04-25T10:05:20.072870+0000","last_active":"2024-04-25T10:05:22.136074+0000","last_peered":"2024-04-25T10:05:22.136074+0000","last_clean":"2024-04-25T10:05:22.136074+0000","last_became_active":"2024-04-25T10:05:20.072608+0000","last_became_peered":"2024-04-25T10:05:20.072608+0000","last_unstale":"2024-04-25T10:05:22.136074+0000","last_undegraded":"2024-04-25T10:05:22.136074+0000","last_fullsized":"2024-04-25T10:05:22.136074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.133679+0000","last_change":"2024-04-25T10:05:21.078027+0000","last_active":"2024-04-25T10:05:22.133679+0000","last_peered":"2024-04-25T10:05:22.133679+0000","last_clean":"2024-04-25T10:05:22.133679+0000","last_became_active":"2024-04-25T10:05:21.076660+0000","last_became_peered":"2024-04-25T10:05:21.076660+0000","last_unstale":"2024-04-25T10:05:22.133679+0000","last_undegraded":"2024-04-25T10:05:22.133679+0000","last_fullsized":"2024-04-25T10:05:22.133679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:21.545921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046449+0000","last_change":"2024-04-25T10:05:20.071927+0000","last_active":"2024-04-25T10:05:22.046449+0000","last_peered":"2024-04-25T10:05:22.046449+0000","last_clean":"2024-04-25T10:05:22.046449+0000","last_became_active":"2024-04-25T10:05:20.071228+0000","last_became_peered":"2024-04-25T10:05:20.071228+0000","last_unstale":"2024-04-25T10:05:22.046449+0000","last_undegraded":"2024-04-25T10:05:22.046449+0000","last_fullsized":"2024-04-25T10:05:22.046449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:54.336058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042801+0000","last_change":"2024-04-25T10:05:21.073018+0000","last_active":"2024-04-25T10:05:22.042801+0000","last_peered":"2024-04-25T10:05:22.042801+0000","last_clean":"2024-04-25T10:05:22.042801+0000","last_became_active":"2024-04-25T10:05:21.072257+0000","last_became_peered":"2024-04-25T10:05:21.072257+0000","last_unstale":"2024-04-25T10:05:22.042801+0000","last_undegraded":"2024-04-25T10:05:22.042801+0000","last_fullsized":"2024-04-25T10:05:22.042801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:53.558925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080605+0000","last_change":"2024-04-25T10:05:20.080134+0000","last_active":"2024-04-25T10:05:22.080605+0000","last_peered":"2024-04-25T10:05:22.080605+0000","last_clean":"2024-04-25T10:05:22.080605+0000","last_became_active":"2024-04-25T10:05:20.079484+0000","last_became_peered":"2024-04-25T10:05:20.079484+0000","last_unstale":"2024-04-25T10:05:22.080605+0000","last_undegraded":"2024-04-25T10:05:22.080605+0000","last_fullsized":"2024-04-25T10:05:22.080605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048538+0000","last_change":"2024-04-25T10:05:21.079513+0000","last_active":"2024-04-25T10:05:22.048538+0000","last_peered":"2024-04-25T10:05:22.048538+0000","last_clean":"2024-04-25T10:05:22.048538+0000","last_became_active":"2024-04-25T10:05:21.078962+0000","last_became_peered":"2024-04-25T10:05:21.078962+0000","last_unstale":"2024-04-25T10:05:22.048538+0000","last_undegraded":"2024-04-25T10:05:22.048538+0000","last_fullsized":"2024-04-25T10:05:22.048538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:18.289495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.049115+0000","last_change":"2024-04-25T10:05:20.075486+0000","last_active":"2024-04-25T10:05:22.049115+0000","last_peered":"2024-04-25T10:05:22.049115+0000","last_clean":"2024-04-25T10:05:22.049115+0000","last_became_active":"2024-04-25T10:05:20.074735+0000","last_became_peered":"2024-04-25T10:05:20.074735+0000","last_unstale":"2024-04-25T10:05:22.049115+0000","last_undegraded":"2024-04-25T10:05:22.049115+0000","last_fullsized":"2024-04-25T10:05:22.049115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:34.961990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134412+0000","last_change":"2024-04-25T10:05:21.085752+0000","last_active":"2024-04-25T10:05:22.134412+0000","last_peered":"2024-04-25T10:05:22.134412+0000","last_clean":"2024-04-25T10:05:22.134412+0000","last_became_active":"2024-04-25T10:05:21.081588+0000","last_became_peered":"2024-04-25T10:05:21.081588+0000","last_unstale":"2024-04-25T10:05:22.134412+0000","last_undegraded":"2024-04-25T10:05:22.134412+0000","last_fullsized":"2024-04-25T10:05:22.134412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:13.318906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043378+0000","last_change":"2024-04-25T10:05:20.065094+0000","last_active":"2024-04-25T10:05:22.043378+0000","last_peered":"2024-04-25T10:05:22.043378+0000","last_clean":"2024-04-25T10:05:22.043378+0000","last_became_active":"2024-04-25T10:05:20.063987+0000","last_became_peered":"2024-04-25T10:05:20.063987+0000","last_unstale":"2024-04-25T10:05:22.043378+0000","last_undegraded":"2024-04-25T10:05:22.043378+0000","last_fullsized":"2024-04-25T10:05:22.043378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:16.201265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048473+0000","last_change":"2024-04-25T10:05:21.063419+0000","last_active":"2024-04-25T10:05:22.048473+0000","last_peered":"2024-04-25T10:05:22.048473+0000","last_clean":"2024-04-25T10:05:22.048473+0000","last_became_active":"2024-04-25T10:05:21.060544+0000","last_became_peered":"2024-04-25T10:05:21.060544+0000","last_unstale":"2024-04-25T10:05:22.048473+0000","last_undegraded":"2024-04-25T10:05:22.048473+0000","last_fullsized":"2024-04-25T10:05:22.048473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:07.266916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080356+0000","last_change":"2024-04-25T10:05:20.070471+0000","last_active":"2024-04-25T10:05:22.080356+0000","last_peered":"2024-04-25T10:05:22.080356+0000","last_clean":"2024-04-25T10:05:22.080356+0000","last_became_active":"2024-04-25T10:05:20.069692+0000","last_became_peered":"2024-04-25T10:05:20.069692+0000","last_unstale":"2024-04-25T10:05:22.080356+0000","last_undegraded":"2024-04-25T10:05:22.080356+0000","last_fullsized":"2024-04-25T10:05:22.080356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:56.779127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136463+0000","last_change":"2024-04-25T10:05:21.086200+0000","last_active":"2024-04-25T10:05:22.136463+0000","last_peered":"2024-04-25T10:05:22.136463+0000","last_clean":"2024-04-25T10:05:22.136463+0000","last_became_active":"2024-04-25T10:05:21.085624+0000","last_became_peered":"2024-04-25T10:05:21.085624+0000","last_unstale":"2024-04-25T10:05:22.136463+0000","last_undegraded":"2024-04-25T10:05:22.136463+0000","last_fullsized":"2024-04-25T10:05:22.136463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:44.226635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135516+0000","last_change":"2024-04-25T10:05:20.077783+0000","last_active":"2024-04-25T10:05:22.135516+0000","last_peered":"2024-04-25T10:05:22.135516+0000","last_clean":"2024-04-25T10:05:22.135516+0000","last_became_active":"2024-04-25T10:05:20.077301+0000","last_became_peered":"2024-04-25T10:05:20.077301+0000","last_unstale":"2024-04-25T10:05:22.135516+0000","last_undegraded":"2024-04-25T10:05:22.135516+0000","last_fullsized":"2024-04-25T10:05:22.135516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:44.794815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134343+0000","last_change":"2024-04-25T10:05:21.086058+0000","last_active":"2024-04-25T10:05:22.134343+0000","last_peered":"2024-04-25T10:05:22.134343+0000","last_clean":"2024-04-25T10:05:22.134343+0000","last_became_active":"2024-04-25T10:05:21.085660+0000","last_became_peered":"2024-04-25T10:05:21.085660+0000","last_unstale":"2024-04-25T10:05:22.134343+0000","last_undegraded":"2024-04-25T10:05:22.134343+0000","last_fullsized":"2024-04-25T10:05:22.134343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:19.908741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047701+0000","last_change":"2024-04-25T10:05:20.062806+0000","last_active":"2024-04-25T10:05:22.047701+0000","last_peered":"2024-04-25T10:05:22.047701+0000","last_clean":"2024-04-25T10:05:22.047701+0000","last_became_active":"2024-04-25T10:05:20.061013+0000","last_became_peered":"2024-04-25T10:05:20.061013+0000","last_unstale":"2024-04-25T10:05:22.047701+0000","last_undegraded":"2024-04-25T10:05:22.047701+0000","last_fullsized":"2024-04-25T10:05:22.047701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:07.439088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136199+0000","last_change":"2024-04-25T10:05:21.092958+0000","last_active":"2024-04-25T10:05:22.136199+0000","last_peered":"2024-04-25T10:05:22.136199+0000","last_clean":"2024-04-25T10:05:22.136199+0000","last_became_active":"2024-04-25T10:05:21.089856+0000","last_became_peered":"2024-04-25T10:05:21.089856+0000","last_unstale":"2024-04-25T10:05:22.136199+0000","last_undegraded":"2024-04-25T10:05:22.136199+0000","last_fullsized":"2024-04-25T10:05:22.136199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:03.477184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.079832+0000","last_change":"2024-04-25T10:05:20.080086+0000","last_active":"2024-04-25T10:05:22.079832+0000","last_peered":"2024-04-25T10:05:22.079832+0000","last_clean":"2024-04-25T10:05:22.079832+0000","last_became_active":"2024-04-25T10:05:20.079302+0000","last_became_peered":"2024-04-25T10:05:20.079302+0000","last_unstale":"2024-04-25T10:05:22.079832+0000","last_undegraded":"2024-04-25T10:05:22.079832+0000","last_fullsized":"2024-04-25T10:05:22.079832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:34.961990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.079883+0000","last_change":"2024-04-25T10:05:21.091338+0000","last_active":"2024-04-25T10:05:22.079883+0000","last_peered":"2024-04-25T10:05:22.079883+0000","last_clean":"2024-04-25T10:05:22.079883+0000","last_became_active":"2024-04-25T10:05:21.090998+0000","last_became_peered":"2024-04-25T10:05:21.090998+0000","last_unstale":"2024-04-25T10:05:22.079883+0000","last_undegraded":"2024-04-25T10:05:22.079883+0000","last_fullsized":"2024-04-25T10:05:22.079883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:13.318906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.133875+0000","last_change":"2024-04-25T10:05:20.063163+0000","last_active":"2024-04-25T10:05:22.133875+0000","last_peered":"2024-04-25T10:05:22.133875+0000","last_clean":"2024-04-25T10:05:22.133875+0000","last_became_active":"2024-04-25T10:05:20.062707+0000","last_became_peered":"2024-04-25T10:05:20.062707+0000","last_unstale":"2024-04-25T10:05:22.133875+0000","last_undegraded":"2024-04-25T10:05:22.133875+0000","last_fullsized":"2024-04-25T10:05:22.133875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:12.228316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135958+0000","last_change":"2024-04-25T10:05:21.088835+0000","last_active":"2024-04-25T10:05:22.135958+0000","last_peered":"2024-04-25T10:05:22.135958+0000","last_clean":"2024-04-25T10:05:22.135958+0000","last_became_active":"2024-04-25T10:05:21.088476+0000","last_became_peered":"2024-04-25T10:05:21.088476+0000","last_unstale":"2024-04-25T10:05:22.135958+0000","last_undegraded":"2024-04-25T10:05:22.135958+0000","last_fullsized":"2024-04-25T10:05:22.135958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:54.535451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048102+0000","last_change":"2024-04-25T10:05:20.066666+0000","last_active":"2024-04-25T10:05:22.048102+0000","last_peered":"2024-04-25T10:05:22.048102+0000","last_clean":"2024-04-25T10:05:22.048102+0000","last_became_active":"2024-04-25T10:05:20.065812+0000","last_became_peered":"2024-04-25T10:05:20.065812+0000","last_unstale":"2024-04-25T10:05:22.048102+0000","last_undegraded":"2024-04-25T10:05:22.048102+0000","last_fullsized":"2024-04-25T10:05:22.048102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:45.802025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136886+0000","last_change":"2024-04-25T10:05:21.089032+0000","last_active":"2024-04-25T10:05:22.136886+0000","last_peered":"2024-04-25T10:05:22.136886+0000","last_clean":"2024-04-25T10:05:22.136886+0000","last_became_active":"2024-04-25T10:05:21.088707+0000","last_became_peered":"2024-04-25T10:05:21.088707+0000","last_unstale":"2024-04-25T10:05:22.136886+0000","last_undegraded":"2024-04-25T10:05:22.136886+0000","last_fullsized":"2024-04-25T10:05:22.136886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:29.342928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135030+0000","last_change":"2024-04-25T10:05:20.056234+0000","last_active":"2024-04-25T10:05:22.135030+0000","last_peered":"2024-04-25T10:05:22.135030+0000","last_clean":"2024-04-25T10:05:22.135030+0000","last_became_active":"2024-04-25T10:05:20.055711+0000","last_became_peered":"2024-04-25T10:05:20.055711+0000","last_unstale":"2024-04-25T10:05:22.135030+0000","last_undegraded":"2024-04-25T10:05:22.135030+0000","last_fullsized":"2024-04-25T10:05:22.135030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:44.794815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137621+0000","last_change":"2024-04-25T10:05:21.097805+0000","last_active":"2024-04-25T10:05:22.137621+0000","last_peered":"2024-04-25T10:05:22.137621+0000","last_clean":"2024-04-25T10:05:22.137621+0000","last_became_active":"2024-04-25T10:05:21.097066+0000","last_became_peered":"2024-04-25T10:05:21.097066+0000","last_unstale":"2024-04-25T10:05:22.137621+0000","last_undegraded":"2024-04-25T10:05:22.137621+0000","last_fullsized":"2024-04-25T10:05:22.137621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:10.508421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043379+0000","last_change":"2024-04-25T10:05:20.063861+0000","last_active":"2024-04-25T10:05:22.043379+0000","last_peered":"2024-04-25T10:05:22.043379+0000","last_clean":"2024-04-25T10:05:22.043379+0000","last_became_active":"2024-04-25T10:05:20.062501+0000","last_became_peered":"2024-04-25T10:05:20.062501+0000","last_unstale":"2024-04-25T10:05:22.043379+0000","last_undegraded":"2024-04-25T10:05:22.043379+0000","last_fullsized":"2024-04-25T10:05:22.043379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:02.884927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.049234+0000","last_change":"2024-04-25T10:05:21.083111+0000","last_active":"2024-04-25T10:05:22.049234+0000","last_peered":"2024-04-25T10:05:22.049234+0000","last_clean":"2024-04-25T10:05:22.049234+0000","last_became_active":"2024-04-25T10:05:21.082321+0000","last_became_peered":"2024-04-25T10:05:21.082321+0000","last_unstale":"2024-04-25T10:05:22.049234+0000","last_undegraded":"2024-04-25T10:05:22.049234+0000","last_fullsized":"2024-04-25T10:05:22.049234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:02.305355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135030+0000","last_change":"2024-04-25T10:05:20.078355+0000","last_active":"2024-04-25T10:05:22.135030+0000","last_peered":"2024-04-25T10:05:22.135030+0000","last_clean":"2024-04-25T10:05:22.135030+0000","last_became_active":"2024-04-25T10:05:20.077677+0000","last_became_peered":"2024-04-25T10:05:20.077677+0000","last_unstale":"2024-04-25T10:05:22.135030+0000","last_undegraded":"2024-04-25T10:05:22.135030+0000","last_fullsized":"2024-04-25T10:05:22.135030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:35.252363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136520+0000","last_change":"2024-04-25T10:05:21.089183+0000","last_active":"2024-04-25T10:05:22.136520+0000","last_peered":"2024-04-25T10:05:22.136520+0000","last_clean":"2024-04-25T10:05:22.136520+0000","last_became_active":"2024-04-25T10:05:21.088768+0000","last_became_peered":"2024-04-25T10:05:21.088768+0000","last_unstale":"2024-04-25T10:05:22.136520+0000","last_undegraded":"2024-04-25T10:05:22.136520+0000","last_fullsized":"2024-04-25T10:05:22.136520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:06.419537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135576+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:05:22.135576+0000","last_peered":"2024-04-25T10:05:22.135576+0000","last_clean":"2024-04-25T10:05:22.135576+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:05:22.135576+0000","last_undegraded":"2024-04-25T10:05:22.135576+0000","last_fullsized":"2024-04-25T10:05:22.135576+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047506+0000","last_change":"2024-04-25T10:05:20.069430+0000","last_active":"2024-04-25T10:05:22.047506+0000","last_peered":"2024-04-25T10:05:22.047506+0000","last_clean":"2024-04-25T10:05:22.047506+0000","last_became_active":"2024-04-25T10:05:20.068755+0000","last_became_peered":"2024-04-25T10:05:20.068755+0000","last_unstale":"2024-04-25T10:05:22.047506+0000","last_undegraded":"2024-04-25T10:05:22.047506+0000","last_fullsized":"2024-04-25T10:05:22.047506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:56.779127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136220+0000","last_change":"2024-04-25T10:05:21.071861+0000","last_active":"2024-04-25T10:05:22.136220+0000","last_peered":"2024-04-25T10:05:22.136220+0000","last_clean":"2024-04-25T10:05:22.136220+0000","last_became_active":"2024-04-25T10:05:21.071182+0000","last_became_peered":"2024-04-25T10:05:21.071182+0000","last_unstale":"2024-04-25T10:05:22.136220+0000","last_undegraded":"2024-04-25T10:05:22.136220+0000","last_fullsized":"2024-04-25T10:05:22.136220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:17.144179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135408+0000","last_change":"2024-04-25T10:05:20.084051+0000","last_active":"2024-04-25T10:05:22.135408+0000","last_peered":"2024-04-25T10:05:22.135408+0000","last_clean":"2024-04-25T10:05:22.135408+0000","last_became_active":"2024-04-25T10:05:20.083563+0000","last_became_peered":"2024-04-25T10:05:20.083563+0000","last_unstale":"2024-04-25T10:05:22.135408+0000","last_undegraded":"2024-04-25T10:05:22.135408+0000","last_fullsized":"2024-04-25T10:05:22.135408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:20.779254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135004+0000","last_change":"2024-04-25T10:05:21.072824+0000","last_active":"2024-04-25T10:05:22.135004+0000","last_peered":"2024-04-25T10:05:22.135004+0000","last_clean":"2024-04-25T10:05:22.135004+0000","last_became_active":"2024-04-25T10:05:21.071691+0000","last_became_peered":"2024-04-25T10:05:21.071691+0000","last_unstale":"2024-04-25T10:05:22.135004+0000","last_undegraded":"2024-04-25T10:05:22.135004+0000","last_fullsized":"2024-04-25T10:05:22.135004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:21.545921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135509+0000","last_change":"2024-04-25T10:05:20.056953+0000","last_active":"2024-04-25T10:05:22.135509+0000","last_peered":"2024-04-25T10:05:22.135509+0000","last_clean":"2024-04-25T10:05:22.135509+0000","last_became_active":"2024-04-25T10:05:20.056056+0000","last_became_peered":"2024-04-25T10:05:20.056056+0000","last_unstale":"2024-04-25T10:05:22.135509+0000","last_undegraded":"2024-04-25T10:05:22.135509+0000","last_fullsized":"2024-04-25T10:05:22.135509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042576+0000","last_change":"2024-04-25T10:05:21.056503+0000","last_active":"2024-04-25T10:05:22.042576+0000","last_peered":"2024-04-25T10:05:22.042576+0000","last_clean":"2024-04-25T10:05:22.042576+0000","last_became_active":"2024-04-25T10:05:21.055664+0000","last_became_peered":"2024-04-25T10:05:21.055664+0000","last_unstale":"2024-04-25T10:05:22.042576+0000","last_undegraded":"2024-04-25T10:05:22.042576+0000","last_fullsized":"2024-04-25T10:05:22.042576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:20.349799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047779+0000","last_change":"2024-04-25T10:05:20.061869+0000","last_active":"2024-04-25T10:05:22.047779+0000","last_peered":"2024-04-25T10:05:22.047779+0000","last_clean":"2024-04-25T10:05:22.047779+0000","last_became_active":"2024-04-25T10:05:20.060232+0000","last_became_peered":"2024-04-25T10:05:20.060232+0000","last_unstale":"2024-04-25T10:05:22.047779+0000","last_undegraded":"2024-04-25T10:05:22.047779+0000","last_fullsized":"2024-04-25T10:05:22.047779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:50.073175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047317+0000","last_change":"2024-04-25T10:05:21.083202+0000","last_active":"2024-04-25T10:05:22.047317+0000","last_peered":"2024-04-25T10:05:22.047317+0000","last_clean":"2024-04-25T10:05:22.047317+0000","last_became_active":"2024-04-25T10:05:21.082857+0000","last_became_peered":"2024-04-25T10:05:21.082857+0000","last_unstale":"2024-04-25T10:05:22.047317+0000","last_undegraded":"2024-04-25T10:05:22.047317+0000","last_fullsized":"2024-04-25T10:05:22.047317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:49.496977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042931+0000","last_change":"2024-04-25T10:05:20.060703+0000","last_active":"2024-04-25T10:05:22.042931+0000","last_peered":"2024-04-25T10:05:22.042931+0000","last_clean":"2024-04-25T10:05:22.042931+0000","last_became_active":"2024-04-25T10:05:20.056864+0000","last_became_peered":"2024-04-25T10:05:20.056864+0000","last_unstale":"2024-04-25T10:05:22.042931+0000","last_undegraded":"2024-04-25T10:05:22.042931+0000","last_fullsized":"2024-04-25T10:05:22.042931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:58.129770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.049114+0000","last_change":"2024-04-25T10:05:21.081253+0000","last_active":"2024-04-25T10:05:22.049114+0000","last_peered":"2024-04-25T10:05:22.049114+0000","last_clean":"2024-04-25T10:05:22.049114+0000","last_became_active":"2024-04-25T10:05:21.080917+0000","last_became_peered":"2024-04-25T10:05:21.080917+0000","last_unstale":"2024-04-25T10:05:22.049114+0000","last_undegraded":"2024-04-25T10:05:22.049114+0000","last_fullsized":"2024-04-25T10:05:22.049114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:41.708275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136822+0000","last_change":"2024-04-25T10:05:20.089660+0000","last_active":"2024-04-25T10:05:22.136822+0000","last_peered":"2024-04-25T10:05:22.136822+0000","last_clean":"2024-04-25T10:05:22.136822+0000","last_became_active":"2024-04-25T10:05:20.088996+0000","last_became_peered":"2024-04-25T10:05:20.088996+0000","last_unstale":"2024-04-25T10:05:22.136822+0000","last_undegraded":"2024-04-25T10:05:22.136822+0000","last_fullsized":"2024-04-25T10:05:22.136822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:55.264435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042818+0000","last_change":"2024-04-25T10:05:21.061203+0000","last_active":"2024-04-25T10:05:22.042818+0000","last_peered":"2024-04-25T10:05:22.042818+0000","last_clean":"2024-04-25T10:05:22.042818+0000","last_became_active":"2024-04-25T10:05:21.060108+0000","last_became_peered":"2024-04-25T10:05:21.060108+0000","last_unstale":"2024-04-25T10:05:22.042818+0000","last_undegraded":"2024-04-25T10:05:22.042818+0000","last_fullsized":"2024-04-25T10:05:22.042818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:11.969876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.077357+0000","last_change":"2024-04-25T10:05:20.071770+0000","last_active":"2024-04-25T10:05:22.077357+0000","last_peered":"2024-04-25T10:05:22.077357+0000","last_clean":"2024-04-25T10:05:22.077357+0000","last_became_active":"2024-04-25T10:05:20.071460+0000","last_became_peered":"2024-04-25T10:05:20.071460+0000","last_unstale":"2024-04-25T10:05:22.077357+0000","last_undegraded":"2024-04-25T10:05:22.077357+0000","last_fullsized":"2024-04-25T10:05:22.077357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:20.779254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136359+0000","last_change":"2024-04-25T10:05:21.063585+0000","last_active":"2024-04-25T10:05:22.136359+0000","last_peered":"2024-04-25T10:05:22.136359+0000","last_clean":"2024-04-25T10:05:22.136359+0000","last_became_active":"2024-04-25T10:05:21.062481+0000","last_became_peered":"2024-04-25T10:05:21.062481+0000","last_unstale":"2024-04-25T10:05:22.136359+0000","last_undegraded":"2024-04-25T10:05:22.136359+0000","last_fullsized":"2024-04-25T10:05:22.136359+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:47.724037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080795+0000","last_change":"2024-04-25T10:05:20.086795+0000","last_active":"2024-04-25T10:05:22.080795+0000","last_peered":"2024-04-25T10:05:22.080795+0000","last_clean":"2024-04-25T10:05:22.080795+0000","last_became_active":"2024-04-25T10:05:20.086157+0000","last_became_peered":"2024-04-25T10:05:20.086157+0000","last_unstale":"2024-04-25T10:05:22.080795+0000","last_undegraded":"2024-04-25T10:05:22.080795+0000","last_fullsized":"2024-04-25T10:05:22.080795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:35.252363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135349+0000","last_change":"2024-04-25T10:05:21.080072+0000","last_active":"2024-04-25T10:05:22.135349+0000","last_peered":"2024-04-25T10:05:22.135349+0000","last_clean":"2024-04-25T10:05:22.135349+0000","last_became_active":"2024-04-25T10:05:21.077807+0000","last_became_peered":"2024-04-25T10:05:21.077807+0000","last_unstale":"2024-04-25T10:05:22.135349+0000","last_undegraded":"2024-04-25T10:05:22.135349+0000","last_fullsized":"2024-04-25T10:05:22.135349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:23.519531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135461+0000","last_change":"2024-04-25T10:05:20.074066+0000","last_active":"2024-04-25T10:05:22.135461+0000","last_peered":"2024-04-25T10:05:22.135461+0000","last_clean":"2024-04-25T10:05:22.135461+0000","last_became_active":"2024-04-25T10:05:20.073313+0000","last_became_peered":"2024-04-25T10:05:20.073313+0000","last_unstale":"2024-04-25T10:05:22.135461+0000","last_undegraded":"2024-04-25T10:05:22.135461+0000","last_fullsized":"2024-04-25T10:05:22.135461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:35.144583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047063+0000","last_change":"2024-04-25T10:05:21.070227+0000","last_active":"2024-04-25T10:05:22.047063+0000","last_peered":"2024-04-25T10:05:22.047063+0000","last_clean":"2024-04-25T10:05:22.047063+0000","last_became_active":"2024-04-25T10:05:21.069473+0000","last_became_peered":"2024-04-25T10:05:21.069473+0000","last_unstale":"2024-04-25T10:05:22.047063+0000","last_undegraded":"2024-04-25T10:05:22.047063+0000","last_fullsized":"2024-04-25T10:05:22.047063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:17.204276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043069+0000","last_change":"2024-04-25T10:05:20.073767+0000","last_active":"2024-04-25T10:05:22.043069+0000","last_peered":"2024-04-25T10:05:22.043069+0000","last_clean":"2024-04-25T10:05:22.043069+0000","last_became_active":"2024-04-25T10:05:20.073333+0000","last_became_peered":"2024-04-25T10:05:20.073333+0000","last_unstale":"2024-04-25T10:05:22.043069+0000","last_undegraded":"2024-04-25T10:05:22.043069+0000","last_fullsized":"2024-04-25T10:05:22.043069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:48.493967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135540+0000","last_change":"2024-04-25T10:05:21.070624+0000","last_active":"2024-04-25T10:05:22.135540+0000","last_peered":"2024-04-25T10:05:22.135540+0000","last_clean":"2024-04-25T10:05:22.135540+0000","last_became_active":"2024-04-25T10:05:21.065398+0000","last_became_peered":"2024-04-25T10:05:21.065398+0000","last_unstale":"2024-04-25T10:05:22.135540+0000","last_undegraded":"2024-04-25T10:05:22.135540+0000","last_fullsized":"2024-04-25T10:05:22.135540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:33.904915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043074+0000","last_change":"2024-04-25T10:05:20.055362+0000","last_active":"2024-04-25T10:05:22.043074+0000","last_peered":"2024-04-25T10:05:22.043074+0000","last_clean":"2024-04-25T10:05:22.043074+0000","last_became_active":"2024-04-25T10:05:20.052279+0000","last_became_peered":"2024-04-25T10:05:20.052279+0000","last_unstale":"2024-04-25T10:05:22.043074+0000","last_undegraded":"2024-04-25T10:05:22.043074+0000","last_fullsized":"2024-04-25T10:05:22.043074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:35.144583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137237+0000","last_change":"2024-04-25T10:05:21.075800+0000","last_active":"2024-04-25T10:05:22.137237+0000","last_peered":"2024-04-25T10:05:22.137237+0000","last_clean":"2024-04-25T10:05:22.137237+0000","last_became_active":"2024-04-25T10:05:21.075425+0000","last_became_peered":"2024-04-25T10:05:21.075425+0000","last_unstale":"2024-04-25T10:05:22.137237+0000","last_undegraded":"2024-04-25T10:05:22.137237+0000","last_fullsized":"2024-04-25T10:05:22.137237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:02.817716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136402+0000","last_change":"2024-04-25T10:05:20.085669+0000","last_active":"2024-04-25T10:05:22.136402+0000","last_peered":"2024-04-25T10:05:22.136402+0000","last_clean":"2024-04-25T10:05:22.136402+0000","last_became_active":"2024-04-25T10:05:20.084890+0000","last_became_peered":"2024-04-25T10:05:20.084890+0000","last_unstale":"2024-04-25T10:05:22.136402+0000","last_undegraded":"2024-04-25T10:05:22.136402+0000","last_fullsized":"2024-04-25T10:05:22.136402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:16.201265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080121+0000","last_change":"2024-04-25T10:05:21.072582+0000","last_active":"2024-04-25T10:05:22.080121+0000","last_peered":"2024-04-25T10:05:22.080121+0000","last_clean":"2024-04-25T10:05:22.080121+0000","last_became_active":"2024-04-25T10:05:21.071786+0000","last_became_peered":"2024-04-25T10:05:21.071786+0000","last_unstale":"2024-04-25T10:05:22.080121+0000","last_undegraded":"2024-04-25T10:05:22.080121+0000","last_fullsized":"2024-04-25T10:05:22.080121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:21.545921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048737+0000","last_change":"2024-04-25T10:05:21.069632+0000","last_active":"2024-04-25T10:05:22.048737+0000","last_peered":"2024-04-25T10:05:22.048737+0000","last_clean":"2024-04-25T10:05:22.048737+0000","last_became_active":"2024-04-25T10:05:21.068870+0000","last_became_peered":"2024-04-25T10:05:21.068870+0000","last_unstale":"2024-04-25T10:05:22.048737+0000","last_undegraded":"2024-04-25T10:05:22.048737+0000","last_fullsized":"2024-04-25T10:05:22.048737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:09.103889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136270+0000","last_change":"2024-04-25T10:05:20.085837+0000","last_active":"2024-04-25T10:05:22.136270+0000","last_peered":"2024-04-25T10:05:22.136270+0000","last_clean":"2024-04-25T10:05:22.136270+0000","last_became_active":"2024-04-25T10:05:20.085079+0000","last_became_peered":"2024-04-25T10:05:20.085079+0000","last_unstale":"2024-04-25T10:05:22.136270+0000","last_undegraded":"2024-04-25T10:05:22.136270+0000","last_fullsized":"2024-04-25T10:05:22.136270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:02.884927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048913+0000","last_change":"2024-04-25T10:05:21.073527+0000","last_active":"2024-04-25T10:05:22.048913+0000","last_peered":"2024-04-25T10:05:22.048913+0000","last_clean":"2024-04-25T10:05:22.048913+0000","last_became_active":"2024-04-25T10:05:21.072780+0000","last_became_peered":"2024-04-25T10:05:21.072780+0000","last_unstale":"2024-04-25T10:05:22.048913+0000","last_undegraded":"2024-04-25T10:05:22.048913+0000","last_fullsized":"2024-04-25T10:05:22.048913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:58.306898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136676+0000","last_change":"2024-04-25T10:05:20.088969+0000","last_active":"2024-04-25T10:05:22.136676+0000","last_peered":"2024-04-25T10:05:22.136676+0000","last_clean":"2024-04-25T10:05:22.136676+0000","last_became_active":"2024-04-25T10:05:20.088074+0000","last_became_peered":"2024-04-25T10:05:20.088074+0000","last_unstale":"2024-04-25T10:05:22.136676+0000","last_undegraded":"2024-04-25T10:05:22.136676+0000","last_fullsized":"2024-04-25T10:05:22.136676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:48.493967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134241+0000","last_change":"2024-04-25T10:05:21.081805+0000","last_active":"2024-04-25T10:05:22.134241+0000","last_peered":"2024-04-25T10:05:22.134241+0000","last_clean":"2024-04-25T10:05:22.134241+0000","last_became_active":"2024-04-25T10:05:21.081062+0000","last_became_peered":"2024-04-25T10:05:21.081062+0000","last_unstale":"2024-04-25T10:05:22.134241+0000","last_undegraded":"2024-04-25T10:05:22.134241+0000","last_fullsized":"2024-04-25T10:05:22.134241+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:55.635372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137045+0000","last_change":"2024-04-25T10:05:20.060475+0000","last_active":"2024-04-25T10:05:22.137045+0000","last_peered":"2024-04-25T10:05:22.137045+0000","last_clean":"2024-04-25T10:05:22.137045+0000","last_became_active":"2024-04-25T10:05:20.060153+0000","last_became_peered":"2024-04-25T10:05:20.060153+0000","last_unstale":"2024-04-25T10:05:22.137045+0000","last_undegraded":"2024-04-25T10:05:22.137045+0000","last_fullsized":"2024-04-25T10:05:22.137045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:12.228316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136396+0000","last_change":"2024-04-25T10:05:21.085394+0000","last_active":"2024-04-25T10:05:22.136396+0000","last_peered":"2024-04-25T10:05:22.136396+0000","last_clean":"2024-04-25T10:05:22.136396+0000","last_became_active":"2024-04-25T10:05:21.085122+0000","last_became_peered":"2024-04-25T10:05:21.085122+0000","last_unstale":"2024-04-25T10:05:22.136396+0000","last_undegraded":"2024-04-25T10:05:22.136396+0000","last_fullsized":"2024-04-25T10:05:22.136396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134727+0000","last_change":"2024-04-25T10:05:20.085225+0000","last_active":"2024-04-25T10:05:22.134727+0000","last_peered":"2024-04-25T10:05:22.134727+0000","last_clean":"2024-04-25T10:05:22.134727+0000","last_became_active":"2024-04-25T10:05:20.084907+0000","last_became_peered":"2024-04-25T10:05:20.084907+0000","last_unstale":"2024-04-25T10:05:22.134727+0000","last_undegraded":"2024-04-25T10:05:22.134727+0000","last_fullsized":"2024-04-25T10:05:22.134727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":233472,"data_stored":79812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":221184,"data_stored":78200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":78360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":827392,"data_stored":669206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:31.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:31.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.199+0000 7fe9ffde1640 1 Processor -- start 2024-04-25T10:05:31.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.199+0000 7fe9ffde1640 1 -- start start 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9ffde1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80ce9d0 0x7fe9f80cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9ffde1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f806c930 con 0x7fe9f815dea0 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9ffde1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f80d20f0 con 0x7fe9f80ce4f0 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9ffde1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f815e280 con 0x7fe9f80ce9d0 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fdb7d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe9f815dea0 0x7fe9f806c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37298/0 (socket says 172.21.15.60:37298) 2024-04-25T10:05:31.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fdb7d640 1 -- 172.21.15.60:0/2259178932 learned_addr learned my addr 172.21.15.60:0/2259178932 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:31.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fe37e640 1 --2- 172.21.15.60:0/2259178932 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80ce9d0 0x7fe9f80cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fcb7b640 1 -- 172.21.15.60:0/2259178932 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335213820 0 0) 0x7fe9f806c930 con 0x7fe9f815dea0 2024-04-25T10:05:31.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fcb7b640 1 -- 172.21.15.60:0/2259178932 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9e4003530 con 0x7fe9f815dea0 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fcb7b640 1 -- 172.21.15.60:0/2259178932 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2887774958 0 0) 0x7fe9e4003530 con 0x7fe9f815dea0 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fcb7b640 1 -- 172.21.15.60:0/2259178932 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80ce9d0 msgr2=0x7fe9f80cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fcb7b640 1 --2- 172.21.15.60:0/2259178932 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80ce9d0 0x7fe9f80cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fcb7b640 1 -- 172.21.15.60:0/2259178932 >> v1:172.21.15.100:6789/0 conn(0x7fe9f80ce4f0 legacy=0x7fe9f80d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fcb7b640 1 -- 172.21.15.60:0/2259178932 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f80d23d0 con 0x7fe9f815dea0 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fcb7b640 1 -- 172.21.15.60:0/2259178932 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fe9f0001a00 con 0x7fe9f815dea0 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fcb7b640 1 -- 172.21.15.60:0/2259178932 >> v1:172.21.15.60:6789/0 conn(0x7fe9f815dea0 legacy=0x7fe9f806c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fcb7b640 1 --2- 172.21.15.60:0/2259178932 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80ce9d0 0x7fe9e40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fcb7b640 1 --2- 172.21.15.60:0/2259178932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e4003f70 0x7fe9e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fcb7b640 1 --2- 172.21.15.60:0/2259178932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9e4004d50 0x7fe9e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fcb7b640 1 -- 172.21.15.60:0/2259178932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f815e280 con 0x7fe9e4003f70 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fcb7b640 1 -- 172.21.15.60:0/2259178932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e400bab0 con 0x7fe9e4004d50 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fcb7b640 1 -- 172.21.15.60:0/2259178932 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e400bcb0 con 0x7fe9f80ce9d0 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fd37c640 1 --2- 172.21.15.60:0/2259178932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e4003f70 0x7fe9e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fdb7d640 1 --2- 172.21.15.60:0/2259178932 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80ce9d0 0x7fe9e40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fe37e640 1 --2- 172.21.15.60:0/2259178932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9e4004d50 0x7fe9e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fd37c640 1 -- 172.21.15.60:0/2259178932 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80ce9d0 msgr2=0x7fe9e40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fd37c640 1 --2- 172.21.15.60:0/2259178932 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80ce9d0 0x7fe9e40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fd37c640 1 -- 172.21.15.60:0/2259178932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9e4004d50 msgr2=0x7fe9e40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fd37c640 1 --2- 172.21.15.60:0/2259178932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9e4004d50 0x7fe9e40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fd37c640 1 -- 172.21.15.60:0/2259178932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e4003530 con 0x7fe9e4003f70 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.203+0000 7fe9fe37e640 1 --2- 172.21.15.60:0/2259178932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9e4004d50 0x7fe9e40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:31.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fd37c640 1 --2- 172.21.15.60:0/2259178932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e4003f70 0x7fe9e4004810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8001780 tx=0x7fe9e801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260b1d22af1fbdcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fcb7b640 1 -- 172.21.15.60:0/2259178932 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80013a0 con 0x7fe9e4003f70 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fcb7b640 1 -- 172.21.15.60:0/2259178932 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e80052f0 con 0x7fe9e4003f70 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fcb7b640 1 -- 172.21.15.60:0/2259178932 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8003840 con 0x7fe9e4003f70 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9ffde1640 1 -- 172.21.15.60:0/2259178932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e4003f70 msgr2=0x7fe9e4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9ffde1640 1 --2- 172.21.15.60:0/2259178932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e4003f70 0x7fe9e4004810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8001780 tx=0x7fe9e801c420 comp rx=0 tx=0).stop 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fe37e640 1 -- 172.21.15.60:0/2259178932 reap_dead start 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9ffde1640 1 -- 172.21.15.60:0/2259178932 shutdown_connections 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9ffde1640 1 -- 172.21.15.60:0/2259178932 >> 172.21.15.60:0/2259178932 conn(0x7fe9f80d0480 msgr2=0x7fe9f80d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9ffde1640 1 -- 172.21.15.60:0/2259178932 shutdown_connections 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9ffde1640 1 -- 172.21.15.60:0/2259178932 wait complete. 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9ffde1640 1 Processor -- start 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9ffde1640 1 -- start start 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9ffde1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e4003f70 0x7fe9f814ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9ffde1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f80cb930 0x7fe9f815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9ffde1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f80cbdb0 0x7fe9f815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9ffde1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80cee60 con 0x7fe9f80cbdb0 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9ffde1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80d32e0 con 0x7fe9f80cb930 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9ffde1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80cefe0 con 0x7fe9e4003f70 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fe37e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f80cbdb0 0x7fe9f815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fe37e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f80cbdb0 0x7fe9f815a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46548/0 (socket says 172.21.15.60:46548) 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fe37e640 1 -- 172.21.15.60:0/1595209261 learned_addr learned my addr 172.21.15.60:0/1595209261 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fd37c640 1 --2- 172.21.15.60:0/1595209261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f80cb930 0x7fe9f815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fdb7d640 1 --2- 172.21.15.60:0/1595209261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e4003f70 0x7fe9f814ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fe37e640 1 -- 172.21.15.60:0/1595209261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e4003f70 msgr2=0x7fe9f814ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fe37e640 1 --2- 172.21.15.60:0/1595209261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e4003f70 0x7fe9f814ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fe37e640 1 -- 172.21.15.60:0/1595209261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f80cb930 msgr2=0x7fe9f815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fe37e640 1 --2- 172.21.15.60:0/1595209261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f80cb930 0x7fe9f815a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fe37e640 1 -- 172.21.15.60:0/1595209261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f80cab70 con 0x7fe9f80cbdb0 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fdb7d640 1 --2- 172.21.15.60:0/1595209261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e4003f70 0x7fe9f814ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fd37c640 1 --2- 172.21.15.60:0/1595209261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f80cb930 0x7fe9f815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.207+0000 7fe9fe37e640 1 --2- 172.21.15.60:0/1595209261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f80cbdb0 0x7fe9f815a9b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80b98f0 tx=0x7fe9f4011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.211+0000 7fe9ee7fc640 1 -- 172.21.15.60:0/1595209261 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4004230 con 0x7fe9f80cbdb0 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.211+0000 7fe9ee7fc640 1 -- 172.21.15.60:0/1595209261 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f40043f0 con 0x7fe9f80cbdb0 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.211+0000 7fe9ffde1640 1 -- 172.21.15.60:0/1595209261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f80cae50 con 0x7fe9f80cbdb0 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.211+0000 7fe9ee7fc640 1 -- 172.21.15.60:0/1595209261 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4015970 con 0x7fe9f80cbdb0 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.211+0000 7fe9ffde1640 1 -- 172.21.15.60:0/1595209261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f80cb410 con 0x7fe9f80cbdb0 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.211+0000 7fe9ffde1640 1 -- 172.21.15.60:0/1595209261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f8074770 con 0x7fe9f80cbdb0 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.211+0000 7fe9ee7fc640 1 -- 172.21.15.60:0/1595209261 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9f4015b10 con 0x7fe9f80cbdb0 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.211+0000 7fe9ee7fc640 1 --2- 172.21.15.60:0/1595209261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f60 0x7fe9dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.211+0000 7fe9ee7fc640 1 -- 172.21.15.60:0/1595209261 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9f4085510 con 0x7fe9f80cbdb0 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.211+0000 7fe9fdb7d640 1 --2- 172.21.15.60:0/1595209261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f60 0x7fe9dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.211+0000 7fe9fdb7d640 1 --2- 172.21.15.60:0/1595209261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f60 0x7fe9dc069420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00013e0 tx=0x7fe9f0001370 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.215+0000 7fe9ee7fc640 1 -- 172.21.15.60:0/1595209261 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9f4019020 con 0x7fe9f80cbdb0 2024-04-25T10:05:31.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.367+0000 7fe9ffde1640 1 -- 172.21.15.60:0/1595209261 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe9f81592e0 con 0x7fe9dc066f60 2024-04-25T10:05:31.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.383+0000 7fe9ee7fc640 1 -- 172.21.15.60:0/1595209261 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7fe9f0007a70 con 0x7fe9dc066f60 2024-04-25T10:05:31.394 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:31.396 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:05:31.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.387+0000 7fe9ffde1640 1 -- 172.21.15.60:0/1595209261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f60 msgr2=0x7fe9dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.387+0000 7fe9ffde1640 1 --2- 172.21.15.60:0/1595209261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f60 0x7fe9dc069420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00013e0 tx=0x7fe9f0001370 comp rx=0 tx=0).stop 2024-04-25T10:05:31.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.387+0000 7fe9ffde1640 1 -- 172.21.15.60:0/1595209261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f80cbdb0 msgr2=0x7fe9f815a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.387+0000 7fe9ffde1640 1 --2- 172.21.15.60:0/1595209261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f80cbdb0 0x7fe9f815a9b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80b98f0 tx=0x7fe9f4011920 comp rx=0 tx=0).stop 2024-04-25T10:05:31.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.387+0000 7fe9ffde1640 1 -- 172.21.15.60:0/1595209261 shutdown_connections 2024-04-25T10:05:31.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.387+0000 7fe9ffde1640 1 --2- 172.21.15.60:0/1595209261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f60 0x7fe9dc069420 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.387+0000 7fe9ffde1640 1 --2- 172.21.15.60:0/1595209261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f80cbdb0 0x7fe9f815a9b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.387+0000 7fe9ffde1640 1 --2- 172.21.15.60:0/1595209261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f80cb930 0x7fe9f815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.387+0000 7fe9ffde1640 1 --2- 172.21.15.60:0/1595209261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e4003f70 0x7fe9f814ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.387+0000 7fe9ffde1640 1 -- 172.21.15.60:0/1595209261 >> 172.21.15.60:0/1595209261 conn(0x7fe9f80d0480 msgr2=0x7fe9f815f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.387+0000 7fe9ffde1640 1 -- 172.21.15.60:0/1595209261 shutdown_connections 2024-04-25T10:05:31.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.387+0000 7fe9ffde1640 1 -- 172.21.15.60:0/1595209261 wait complete. 2024-04-25T10:05:31.408 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-25T10:05:30.389303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235768,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738952,"statfs":{"total":773094113280,"available":772852686848,"internally_reserved":0,"allocated":3022848,"data_stored":1811156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.347160"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137405+0000","last_change":"2024-04-25T10:05:21.093662+0000","last_active":"2024-04-25T10:05:22.137405+0000","last_peered":"2024-04-25T10:05:22.137405+0000","last_clean":"2024-04-25T10:05:22.137405+0000","last_became_active":"2024-04-25T10:05:21.093342+0000","last_became_peered":"2024-04-25T10:05:21.093342+0000","last_unstale":"2024-04-25T10:05:22.137405+0000","last_undegraded":"2024-04-25T10:05:22.137405+0000","last_fullsized":"2024-04-25T10:05:22.137405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:38.452914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043321+0000","last_change":"2024-04-25T10:05:21.076738+0000","last_active":"2024-04-25T10:05:22.043321+0000","last_peered":"2024-04-25T10:05:22.043321+0000","last_clean":"2024-04-25T10:05:22.043321+0000","last_became_active":"2024-04-25T10:05:21.076085+0000","last_became_peered":"2024-04-25T10:05:21.076085+0000","last_unstale":"2024-04-25T10:05:22.043321+0000","last_undegraded":"2024-04-25T10:05:22.043321+0000","last_fullsized":"2024-04-25T10:05:22.043321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:36.826815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136181+0000","last_change":"2024-04-25T10:05:21.072290+0000","last_active":"2024-04-25T10:05:22.136181+0000","last_peered":"2024-04-25T10:05:22.136181+0000","last_clean":"2024-04-25T10:05:22.136181+0000","last_became_active":"2024-04-25T10:05:21.069763+0000","last_became_peered":"2024-04-25T10:05:21.069763+0000","last_unstale":"2024-04-25T10:05:22.136181+0000","last_undegraded":"2024-04-25T10:05:22.136181+0000","last_fullsized":"2024-04-25T10:05:22.136181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:17.144179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048606+0000","last_change":"2024-04-25T10:05:21.083631+0000","last_active":"2024-04-25T10:05:22.048606+0000","last_peered":"2024-04-25T10:05:22.048606+0000","last_clean":"2024-04-25T10:05:22.048606+0000","last_became_active":"2024-04-25T10:05:21.082783+0000","last_became_peered":"2024-04-25T10:05:21.082783+0000","last_unstale":"2024-04-25T10:05:22.048606+0000","last_undegraded":"2024-04-25T10:05:22.048606+0000","last_fullsized":"2024-04-25T10:05:22.048606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:02.040066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136043+0000","last_change":"2024-04-25T10:05:21.092602+0000","last_active":"2024-04-25T10:05:22.136043+0000","last_peered":"2024-04-25T10:05:22.136043+0000","last_clean":"2024-04-25T10:05:22.136043+0000","last_became_active":"2024-04-25T10:05:21.089559+0000","last_became_peered":"2024-04-25T10:05:21.089559+0000","last_unstale":"2024-04-25T10:05:22.136043+0000","last_undegraded":"2024-04-25T10:05:22.136043+0000","last_fullsized":"2024-04-25T10:05:22.136043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:57.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046962+0000","last_change":"2024-04-25T10:05:21.078938+0000","last_active":"2024-04-25T10:05:22.046962+0000","last_peered":"2024-04-25T10:05:22.046962+0000","last_clean":"2024-04-25T10:05:22.046962+0000","last_became_active":"2024-04-25T10:05:21.078310+0000","last_became_peered":"2024-04-25T10:05:21.078310+0000","last_unstale":"2024-04-25T10:05:22.046962+0000","last_undegraded":"2024-04-25T10:05:22.046962+0000","last_fullsized":"2024-04-25T10:05:22.046962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:30.299159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135741+0000","last_change":"2024-04-25T10:05:21.086161+0000","last_active":"2024-04-25T10:05:22.135741+0000","last_peered":"2024-04-25T10:05:22.135741+0000","last_clean":"2024-04-25T10:05:22.135741+0000","last_became_active":"2024-04-25T10:05:21.084217+0000","last_became_peered":"2024-04-25T10:05:21.084217+0000","last_unstale":"2024-04-25T10:05:22.135741+0000","last_undegraded":"2024-04-25T10:05:22.135741+0000","last_fullsized":"2024-04-25T10:05:22.135741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:29.451932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.133417+0000","last_change":"2024-04-25T10:05:21.071738+0000","last_active":"2024-04-25T10:05:22.133417+0000","last_peered":"2024-04-25T10:05:22.133417+0000","last_clean":"2024-04-25T10:05:22.133417+0000","last_became_active":"2024-04-25T10:05:21.071239+0000","last_became_peered":"2024-04-25T10:05:21.071239+0000","last_unstale":"2024-04-25T10:05:22.133417+0000","last_undegraded":"2024-04-25T10:05:22.133417+0000","last_fullsized":"2024-04-25T10:05:22.133417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:13.318906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137424+0000","last_change":"2024-04-25T10:05:21.089854+0000","last_active":"2024-04-25T10:05:22.137424+0000","last_peered":"2024-04-25T10:05:22.137424+0000","last_clean":"2024-04-25T10:05:22.137424+0000","last_became_active":"2024-04-25T10:05:21.089145+0000","last_became_peered":"2024-04-25T10:05:21.089145+0000","last_unstale":"2024-04-25T10:05:22.137424+0000","last_undegraded":"2024-04-25T10:05:22.137424+0000","last_fullsized":"2024-04-25T10:05:22.137424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:17.168463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.081237+0000","last_change":"2024-04-25T10:05:21.081763+0000","last_active":"2024-04-25T10:05:22.081237+0000","last_peered":"2024-04-25T10:05:22.081237+0000","last_clean":"2024-04-25T10:05:22.081237+0000","last_became_active":"2024-04-25T10:05:21.081398+0000","last_became_peered":"2024-04-25T10:05:21.081398+0000","last_unstale":"2024-04-25T10:05:22.081237+0000","last_undegraded":"2024-04-25T10:05:22.081237+0000","last_fullsized":"2024-04-25T10:05:22.081237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:23.519531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080566+0000","last_change":"2024-04-25T10:05:21.081165+0000","last_active":"2024-04-25T10:05:22.080566+0000","last_peered":"2024-04-25T10:05:22.080566+0000","last_clean":"2024-04-25T10:05:22.080566+0000","last_became_active":"2024-04-25T10:05:21.076196+0000","last_became_peered":"2024-04-25T10:05:21.076196+0000","last_unstale":"2024-04-25T10:05:22.080566+0000","last_undegraded":"2024-04-25T10:05:22.080566+0000","last_fullsized":"2024-04-25T10:05:22.080566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:36.529873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048072+0000","last_change":"2024-04-25T10:05:21.082653+0000","last_active":"2024-04-25T10:05:22.048072+0000","last_peered":"2024-04-25T10:05:22.048072+0000","last_clean":"2024-04-25T10:05:22.048072+0000","last_became_active":"2024-04-25T10:05:21.082004+0000","last_became_peered":"2024-04-25T10:05:21.082004+0000","last_unstale":"2024-04-25T10:05:22.048072+0000","last_undegraded":"2024-04-25T10:05:22.048072+0000","last_fullsized":"2024-04-25T10:05:22.048072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:56.267445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136198+0000","last_change":"2024-04-25T10:05:21.071024+0000","last_active":"2024-04-25T10:05:22.136198+0000","last_peered":"2024-04-25T10:05:22.136198+0000","last_clean":"2024-04-25T10:05:22.136198+0000","last_became_active":"2024-04-25T10:05:21.070048+0000","last_became_peered":"2024-04-25T10:05:21.070048+0000","last_unstale":"2024-04-25T10:05:22.136198+0000","last_undegraded":"2024-04-25T10:05:22.136198+0000","last_fullsized":"2024-04-25T10:05:22.136198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:00.583322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134119+0000","last_change":"2024-04-25T10:05:21.080794+0000","last_active":"2024-04-25T10:05:22.134119+0000","last_peered":"2024-04-25T10:05:22.134119+0000","last_clean":"2024-04-25T10:05:22.134119+0000","last_became_active":"2024-04-25T10:05:21.079418+0000","last_became_peered":"2024-04-25T10:05:21.079418+0000","last_unstale":"2024-04-25T10:05:22.134119+0000","last_undegraded":"2024-04-25T10:05:22.134119+0000","last_fullsized":"2024-04-25T10:05:22.134119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:09.604912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042828+0000","last_change":"2024-04-25T10:05:21.055371+0000","last_active":"2024-04-25T10:05:22.042828+0000","last_peered":"2024-04-25T10:05:22.042828+0000","last_clean":"2024-04-25T10:05:22.042828+0000","last_became_active":"2024-04-25T10:05:21.054565+0000","last_became_peered":"2024-04-25T10:05:21.054565+0000","last_unstale":"2024-04-25T10:05:22.042828+0000","last_undegraded":"2024-04-25T10:05:22.042828+0000","last_fullsized":"2024-04-25T10:05:22.042828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:31.511781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046177+0000","last_change":"2024-04-25T10:05:21.079756+0000","last_active":"2024-04-25T10:05:22.046177+0000","last_peered":"2024-04-25T10:05:22.046177+0000","last_clean":"2024-04-25T10:05:22.046177+0000","last_became_active":"2024-04-25T10:05:21.078988+0000","last_became_peered":"2024-04-25T10:05:21.078988+0000","last_unstale":"2024-04-25T10:05:22.046177+0000","last_undegraded":"2024-04-25T10:05:22.046177+0000","last_fullsized":"2024-04-25T10:05:22.046177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:47.724037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.049473+0000","last_change":"2024-04-25T10:05:21.087275+0000","last_active":"2024-04-25T10:05:22.049473+0000","last_peered":"2024-04-25T10:05:22.049473+0000","last_clean":"2024-04-25T10:05:22.049473+0000","last_became_active":"2024-04-25T10:05:21.086912+0000","last_became_peered":"2024-04-25T10:05:21.086912+0000","last_unstale":"2024-04-25T10:05:22.049473+0000","last_undegraded":"2024-04-25T10:05:22.049473+0000","last_fullsized":"2024-04-25T10:05:22.049473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:01.067797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134822+0000","last_change":"2024-04-25T10:05:21.085565+0000","last_active":"2024-04-25T10:05:22.134822+0000","last_peered":"2024-04-25T10:05:22.134822+0000","last_clean":"2024-04-25T10:05:22.134822+0000","last_became_active":"2024-04-25T10:05:21.084852+0000","last_became_peered":"2024-04-25T10:05:21.084852+0000","last_unstale":"2024-04-25T10:05:22.134822+0000","last_undegraded":"2024-04-25T10:05:22.134822+0000","last_fullsized":"2024-04-25T10:05:22.134822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:51.076186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136820+0000","last_change":"2024-04-25T10:05:21.089683+0000","last_active":"2024-04-25T10:05:22.136820+0000","last_peered":"2024-04-25T10:05:22.136820+0000","last_clean":"2024-04-25T10:05:22.136820+0000","last_became_active":"2024-04-25T10:05:21.089192+0000","last_became_peered":"2024-04-25T10:05:21.089192+0000","last_unstale":"2024-04-25T10:05:22.136820+0000","last_undegraded":"2024-04-25T10:05:22.136820+0000","last_fullsized":"2024-04-25T10:05:22.136820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:50.393309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080406+0000","last_change":"2024-04-25T10:05:21.072923+0000","last_active":"2024-04-25T10:05:22.080406+0000","last_peered":"2024-04-25T10:05:22.080406+0000","last_clean":"2024-04-25T10:05:22.080406+0000","last_became_active":"2024-04-25T10:05:21.072600+0000","last_became_peered":"2024-04-25T10:05:21.072600+0000","last_unstale":"2024-04-25T10:05:22.080406+0000","last_undegraded":"2024-04-25T10:05:22.080406+0000","last_fullsized":"2024-04-25T10:05:22.080406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:33.904915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135126+0000","last_change":"2024-04-25T10:05:21.092396+0000","last_active":"2024-04-25T10:05:22.135126+0000","last_peered":"2024-04-25T10:05:22.135126+0000","last_clean":"2024-04-25T10:05:22.135126+0000","last_became_active":"2024-04-25T10:05:21.092118+0000","last_became_peered":"2024-04-25T10:05:21.092118+0000","last_unstale":"2024-04-25T10:05:22.135126+0000","last_undegraded":"2024-04-25T10:05:22.135126+0000","last_fullsized":"2024-04-25T10:05:22.135126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:33.904915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047430+0000","last_change":"2024-04-25T10:05:21.079563+0000","last_active":"2024-04-25T10:05:22.047430+0000","last_peered":"2024-04-25T10:05:22.047430+0000","last_clean":"2024-04-25T10:05:22.047430+0000","last_became_active":"2024-04-25T10:05:21.079032+0000","last_became_peered":"2024-04-25T10:05:21.079032+0000","last_unstale":"2024-04-25T10:05:22.047430+0000","last_undegraded":"2024-04-25T10:05:22.047430+0000","last_fullsized":"2024-04-25T10:05:22.047430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:02.817716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135891+0000","last_change":"2024-04-25T10:05:21.071825+0000","last_active":"2024-04-25T10:05:22.135891+0000","last_peered":"2024-04-25T10:05:22.135891+0000","last_clean":"2024-04-25T10:05:22.135891+0000","last_became_active":"2024-04-25T10:05:21.071207+0000","last_became_peered":"2024-04-25T10:05:21.071207+0000","last_unstale":"2024-04-25T10:05:22.135891+0000","last_undegraded":"2024-04-25T10:05:22.135891+0000","last_fullsized":"2024-04-25T10:05:22.135891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:23.519531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046222+0000","last_change":"2024-04-25T10:05:21.083347+0000","last_active":"2024-04-25T10:05:22.046222+0000","last_peered":"2024-04-25T10:05:22.046222+0000","last_clean":"2024-04-25T10:05:22.046222+0000","last_became_active":"2024-04-25T10:05:21.082648+0000","last_became_peered":"2024-04-25T10:05:21.082648+0000","last_unstale":"2024-04-25T10:05:22.046222+0000","last_undegraded":"2024-04-25T10:05:22.046222+0000","last_fullsized":"2024-04-25T10:05:22.046222+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135886+0000","last_change":"2024-04-25T10:05:21.069597+0000","last_active":"2024-04-25T10:05:22.135886+0000","last_peered":"2024-04-25T10:05:22.135886+0000","last_clean":"2024-04-25T10:05:22.135886+0000","last_became_active":"2024-04-25T10:05:21.064771+0000","last_became_peered":"2024-04-25T10:05:21.064771+0000","last_unstale":"2024-04-25T10:05:22.135886+0000","last_undegraded":"2024-04-25T10:05:22.135886+0000","last_fullsized":"2024-04-25T10:05:22.135886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:36.529873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.081502+0000","last_change":"2024-04-25T10:05:21.090548+0000","last_active":"2024-04-25T10:05:22.081502+0000","last_peered":"2024-04-25T10:05:22.081502+0000","last_clean":"2024-04-25T10:05:22.081502+0000","last_became_active":"2024-04-25T10:05:21.090195+0000","last_became_peered":"2024-04-25T10:05:21.090195+0000","last_unstale":"2024-04-25T10:05:22.081502+0000","last_undegraded":"2024-04-25T10:05:22.081502+0000","last_fullsized":"2024-04-25T10:05:22.081502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:00.583322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043216+0000","last_change":"2024-04-25T10:05:21.078928+0000","last_active":"2024-04-25T10:05:22.043216+0000","last_peered":"2024-04-25T10:05:22.043216+0000","last_clean":"2024-04-25T10:05:22.043216+0000","last_became_active":"2024-04-25T10:05:21.078521+0000","last_became_peered":"2024-04-25T10:05:21.078521+0000","last_unstale":"2024-04-25T10:05:22.043216+0000","last_undegraded":"2024-04-25T10:05:22.043216+0000","last_fullsized":"2024-04-25T10:05:22.043216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:03.085314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136663+0000","last_change":"2024-04-25T10:05:21.063167+0000","last_active":"2024-04-25T10:05:22.136663+0000","last_peered":"2024-04-25T10:05:22.136663+0000","last_clean":"2024-04-25T10:05:22.136663+0000","last_became_active":"2024-04-25T10:05:21.061503+0000","last_became_peered":"2024-04-25T10:05:21.061503+0000","last_unstale":"2024-04-25T10:05:22.136663+0000","last_undegraded":"2024-04-25T10:05:22.136663+0000","last_fullsized":"2024-04-25T10:05:22.136663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:30.299159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135212+0000","last_change":"2024-04-25T10:05:21.080730+0000","last_active":"2024-04-25T10:05:22.135212+0000","last_peered":"2024-04-25T10:05:22.135212+0000","last_clean":"2024-04-25T10:05:22.135212+0000","last_became_active":"2024-04-25T10:05:21.080248+0000","last_became_peered":"2024-04-25T10:05:21.080248+0000","last_unstale":"2024-04-25T10:05:22.135212+0000","last_undegraded":"2024-04-25T10:05:22.135212+0000","last_fullsized":"2024-04-25T10:05:22.135212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:36.529873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047524+0000","last_change":"2024-04-25T10:05:21.083357+0000","last_active":"2024-04-25T10:05:22.047524+0000","last_peered":"2024-04-25T10:05:22.047524+0000","last_clean":"2024-04-25T10:05:22.047524+0000","last_became_active":"2024-04-25T10:05:21.082923+0000","last_became_peered":"2024-04-25T10:05:21.082923+0000","last_unstale":"2024-04-25T10:05:22.047524+0000","last_undegraded":"2024-04-25T10:05:22.047524+0000","last_fullsized":"2024-04-25T10:05:22.047524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:50.797661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135432+0000","last_change":"2024-04-25T10:05:21.075647+0000","last_active":"2024-04-25T10:05:22.135432+0000","last_peered":"2024-04-25T10:05:22.135432+0000","last_clean":"2024-04-25T10:05:22.135432+0000","last_became_active":"2024-04-25T10:05:21.074954+0000","last_became_peered":"2024-04-25T10:05:21.074954+0000","last_unstale":"2024-04-25T10:05:22.135432+0000","last_undegraded":"2024-04-25T10:05:22.135432+0000","last_fullsized":"2024-04-25T10:05:22.135432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:55.339069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.079882+0000","last_change":"2024-04-25T10:05:21.084060+0000","last_active":"2024-04-25T10:05:22.079882+0000","last_peered":"2024-04-25T10:05:22.079882+0000","last_clean":"2024-04-25T10:05:22.079882+0000","last_became_active":"2024-04-25T10:05:21.083506+0000","last_became_peered":"2024-04-25T10:05:21.083506+0000","last_unstale":"2024-04-25T10:05:22.079882+0000","last_undegraded":"2024-04-25T10:05:22.079882+0000","last_fullsized":"2024-04-25T10:05:22.079882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:19.908741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046660+0000","last_change":"2024-04-25T10:05:21.065678+0000","last_active":"2024-04-25T10:05:22.046660+0000","last_peered":"2024-04-25T10:05:22.046660+0000","last_clean":"2024-04-25T10:05:22.046660+0000","last_became_active":"2024-04-25T10:05:21.064718+0000","last_became_peered":"2024-04-25T10:05:21.064718+0000","last_unstale":"2024-04-25T10:05:22.046660+0000","last_undegraded":"2024-04-25T10:05:22.046660+0000","last_fullsized":"2024-04-25T10:05:22.046660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:03.887937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136502+0000","last_change":"2024-04-25T10:05:20.083026+0000","last_active":"2024-04-25T10:05:22.136502+0000","last_peered":"2024-04-25T10:05:22.136502+0000","last_clean":"2024-04-25T10:05:22.136502+0000","last_became_active":"2024-04-25T10:05:20.082702+0000","last_became_peered":"2024-04-25T10:05:20.082702+0000","last_unstale":"2024-04-25T10:05:22.136502+0000","last_undegraded":"2024-04-25T10:05:22.136502+0000","last_fullsized":"2024-04-25T10:05:22.136502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:35.252363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135477+0000","last_change":"2024-04-25T10:05:21.086317+0000","last_active":"2024-04-25T10:05:22.135477+0000","last_peered":"2024-04-25T10:05:22.135477+0000","last_clean":"2024-04-25T10:05:22.135477+0000","last_became_active":"2024-04-25T10:05:21.085802+0000","last_became_peered":"2024-04-25T10:05:21.085802+0000","last_unstale":"2024-04-25T10:05:22.135477+0000","last_undegraded":"2024-04-25T10:05:22.135477+0000","last_fullsized":"2024-04-25T10:05:22.135477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:00.583322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080039+0000","last_change":"2024-04-25T10:05:20.080501+0000","last_active":"2024-04-25T10:05:22.080039+0000","last_peered":"2024-04-25T10:05:22.080039+0000","last_clean":"2024-04-25T10:05:22.080039+0000","last_became_active":"2024-04-25T10:05:20.079392+0000","last_became_peered":"2024-04-25T10:05:20.079392+0000","last_unstale":"2024-04-25T10:05:22.080039+0000","last_undegraded":"2024-04-25T10:05:22.080039+0000","last_fullsized":"2024-04-25T10:05:22.080039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:44.794815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136074+0000","last_change":"2024-04-25T10:05:20.072870+0000","last_active":"2024-04-25T10:05:22.136074+0000","last_peered":"2024-04-25T10:05:22.136074+0000","last_clean":"2024-04-25T10:05:22.136074+0000","last_became_active":"2024-04-25T10:05:20.072608+0000","last_became_peered":"2024-04-25T10:05:20.072608+0000","last_unstale":"2024-04-25T10:05:22.136074+0000","last_undegraded":"2024-04-25T10:05:22.136074+0000","last_fullsized":"2024-04-25T10:05:22.136074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.133679+0000","last_change":"2024-04-25T10:05:21.078027+0000","last_active":"2024-04-25T10:05:22.133679+0000","last_peered":"2024-04-25T10:05:22.133679+0000","last_clean":"2024-04-25T10:05:22.133679+0000","last_became_active":"2024-04-25T10:05:21.076660+0000","last_became_peered":"2024-04-25T10:05:21.076660+0000","last_unstale":"2024-04-25T10:05:22.133679+0000","last_undegraded":"2024-04-25T10:05:22.133679+0000","last_fullsized":"2024-04-25T10:05:22.133679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:21.545921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.046449+0000","last_change":"2024-04-25T10:05:20.071927+0000","last_active":"2024-04-25T10:05:22.046449+0000","last_peered":"2024-04-25T10:05:22.046449+0000","last_clean":"2024-04-25T10:05:22.046449+0000","last_became_active":"2024-04-25T10:05:20.071228+0000","last_became_peered":"2024-04-25T10:05:20.071228+0000","last_unstale":"2024-04-25T10:05:22.046449+0000","last_undegraded":"2024-04-25T10:05:22.046449+0000","last_fullsized":"2024-04-25T10:05:22.046449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:54.336058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042801+0000","last_change":"2024-04-25T10:05:21.073018+0000","last_active":"2024-04-25T10:05:22.042801+0000","last_peered":"2024-04-25T10:05:22.042801+0000","last_clean":"2024-04-25T10:05:22.042801+0000","last_became_active":"2024-04-25T10:05:21.072257+0000","last_became_peered":"2024-04-25T10:05:21.072257+0000","last_unstale":"2024-04-25T10:05:22.042801+0000","last_undegraded":"2024-04-25T10:05:22.042801+0000","last_fullsized":"2024-04-25T10:05:22.042801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:53.558925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080605+0000","last_change":"2024-04-25T10:05:20.080134+0000","last_active":"2024-04-25T10:05:22.080605+0000","last_peered":"2024-04-25T10:05:22.080605+0000","last_clean":"2024-04-25T10:05:22.080605+0000","last_became_active":"2024-04-25T10:05:20.079484+0000","last_became_peered":"2024-04-25T10:05:20.079484+0000","last_unstale":"2024-04-25T10:05:22.080605+0000","last_undegraded":"2024-04-25T10:05:22.080605+0000","last_fullsized":"2024-04-25T10:05:22.080605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048538+0000","last_change":"2024-04-25T10:05:21.079513+0000","last_active":"2024-04-25T10:05:22.048538+0000","last_peered":"2024-04-25T10:05:22.048538+0000","last_clean":"2024-04-25T10:05:22.048538+0000","last_became_active":"2024-04-25T10:05:21.078962+0000","last_became_peered":"2024-04-25T10:05:21.078962+0000","last_unstale":"2024-04-25T10:05:22.048538+0000","last_undegraded":"2024-04-25T10:05:22.048538+0000","last_fullsized":"2024-04-25T10:05:22.048538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:18.289495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.049115+0000","last_change":"2024-04-25T10:05:20.075486+0000","last_active":"2024-04-25T10:05:22.049115+0000","last_peered":"2024-04-25T10:05:22.049115+0000","last_clean":"2024-04-25T10:05:22.049115+0000","last_became_active":"2024-04-25T10:05:20.074735+0000","last_became_peered":"2024-04-25T10:05:20.074735+0000","last_unstale":"2024-04-25T10:05:22.049115+0000","last_undegraded":"2024-04-25T10:05:22.049115+0000","last_fullsized":"2024-04-25T10:05:22.049115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:34.961990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134412+0000","last_change":"2024-04-25T10:05:21.085752+0000","last_active":"2024-04-25T10:05:22.134412+0000","last_peered":"2024-04-25T10:05:22.134412+0000","last_clean":"2024-04-25T10:05:22.134412+0000","last_became_active":"2024-04-25T10:05:21.081588+0000","last_became_peered":"2024-04-25T10:05:21.081588+0000","last_unstale":"2024-04-25T10:05:22.134412+0000","last_undegraded":"2024-04-25T10:05:22.134412+0000","last_fullsized":"2024-04-25T10:05:22.134412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:13.318906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043378+0000","last_change":"2024-04-25T10:05:20.065094+0000","last_active":"2024-04-25T10:05:22.043378+0000","last_peered":"2024-04-25T10:05:22.043378+0000","last_clean":"2024-04-25T10:05:22.043378+0000","last_became_active":"2024-04-25T10:05:20.063987+0000","last_became_peered":"2024-04-25T10:05:20.063987+0000","last_unstale":"2024-04-25T10:05:22.043378+0000","last_undegraded":"2024-04-25T10:05:22.043378+0000","last_fullsized":"2024-04-25T10:05:22.043378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:16.201265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048473+0000","last_change":"2024-04-25T10:05:21.063419+0000","last_active":"2024-04-25T10:05:22.048473+0000","last_peered":"2024-04-25T10:05:22.048473+0000","last_clean":"2024-04-25T10:05:22.048473+0000","last_became_active":"2024-04-25T10:05:21.060544+0000","last_became_peered":"2024-04-25T10:05:21.060544+0000","last_unstale":"2024-04-25T10:05:22.048473+0000","last_undegraded":"2024-04-25T10:05:22.048473+0000","last_fullsized":"2024-04-25T10:05:22.048473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:07.266916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080356+0000","last_change":"2024-04-25T10:05:20.070471+0000","last_active":"2024-04-25T10:05:22.080356+0000","last_peered":"2024-04-25T10:05:22.080356+0000","last_clean":"2024-04-25T10:05:22.080356+0000","last_became_active":"2024-04-25T10:05:20.069692+0000","last_became_peered":"2024-04-25T10:05:20.069692+0000","last_unstale":"2024-04-25T10:05:22.080356+0000","last_undegraded":"2024-04-25T10:05:22.080356+0000","last_fullsized":"2024-04-25T10:05:22.080356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:56.779127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136463+0000","last_change":"2024-04-25T10:05:21.086200+0000","last_active":"2024-04-25T10:05:22.136463+0000","last_peered":"2024-04-25T10:05:22.136463+0000","last_clean":"2024-04-25T10:05:22.136463+0000","last_became_active":"2024-04-25T10:05:21.085624+0000","last_became_peered":"2024-04-25T10:05:21.085624+0000","last_unstale":"2024-04-25T10:05:22.136463+0000","last_undegraded":"2024-04-25T10:05:22.136463+0000","last_fullsized":"2024-04-25T10:05:22.136463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:44.226635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135516+0000","last_change":"2024-04-25T10:05:20.077783+0000","last_active":"2024-04-25T10:05:22.135516+0000","last_peered":"2024-04-25T10:05:22.135516+0000","last_clean":"2024-04-25T10:05:22.135516+0000","last_became_active":"2024-04-25T10:05:20.077301+0000","last_became_peered":"2024-04-25T10:05:20.077301+0000","last_unstale":"2024-04-25T10:05:22.135516+0000","last_undegraded":"2024-04-25T10:05:22.135516+0000","last_fullsized":"2024-04-25T10:05:22.135516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:44.794815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134343+0000","last_change":"2024-04-25T10:05:21.086058+0000","last_active":"2024-04-25T10:05:22.134343+0000","last_peered":"2024-04-25T10:05:22.134343+0000","last_clean":"2024-04-25T10:05:22.134343+0000","last_became_active":"2024-04-25T10:05:21.085660+0000","last_became_peered":"2024-04-25T10:05:21.085660+0000","last_unstale":"2024-04-25T10:05:22.134343+0000","last_undegraded":"2024-04-25T10:05:22.134343+0000","last_fullsized":"2024-04-25T10:05:22.134343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:19.908741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047701+0000","last_change":"2024-04-25T10:05:20.062806+0000","last_active":"2024-04-25T10:05:22.047701+0000","last_peered":"2024-04-25T10:05:22.047701+0000","last_clean":"2024-04-25T10:05:22.047701+0000","last_became_active":"2024-04-25T10:05:20.061013+0000","last_became_peered":"2024-04-25T10:05:20.061013+0000","last_unstale":"2024-04-25T10:05:22.047701+0000","last_undegraded":"2024-04-25T10:05:22.047701+0000","last_fullsized":"2024-04-25T10:05:22.047701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:07.439088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136199+0000","last_change":"2024-04-25T10:05:21.092958+0000","last_active":"2024-04-25T10:05:22.136199+0000","last_peered":"2024-04-25T10:05:22.136199+0000","last_clean":"2024-04-25T10:05:22.136199+0000","last_became_active":"2024-04-25T10:05:21.089856+0000","last_became_peered":"2024-04-25T10:05:21.089856+0000","last_unstale":"2024-04-25T10:05:22.136199+0000","last_undegraded":"2024-04-25T10:05:22.136199+0000","last_fullsized":"2024-04-25T10:05:22.136199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:03.477184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.079832+0000","last_change":"2024-04-25T10:05:20.080086+0000","last_active":"2024-04-25T10:05:22.079832+0000","last_peered":"2024-04-25T10:05:22.079832+0000","last_clean":"2024-04-25T10:05:22.079832+0000","last_became_active":"2024-04-25T10:05:20.079302+0000","last_became_peered":"2024-04-25T10:05:20.079302+0000","last_unstale":"2024-04-25T10:05:22.079832+0000","last_undegraded":"2024-04-25T10:05:22.079832+0000","last_fullsized":"2024-04-25T10:05:22.079832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:34.961990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.079883+0000","last_change":"2024-04-25T10:05:21.091338+0000","last_active":"2024-04-25T10:05:22.079883+0000","last_peered":"2024-04-25T10:05:22.079883+0000","last_clean":"2024-04-25T10:05:22.079883+0000","last_became_active":"2024-04-25T10:05:21.090998+0000","last_became_peered":"2024-04-25T10:05:21.090998+0000","last_unstale":"2024-04-25T10:05:22.079883+0000","last_undegraded":"2024-04-25T10:05:22.079883+0000","last_fullsized":"2024-04-25T10:05:22.079883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:13.318906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.133875+0000","last_change":"2024-04-25T10:05:20.063163+0000","last_active":"2024-04-25T10:05:22.133875+0000","last_peered":"2024-04-25T10:05:22.133875+0000","last_clean":"2024-04-25T10:05:22.133875+0000","last_became_active":"2024-04-25T10:05:20.062707+0000","last_became_peered":"2024-04-25T10:05:20.062707+0000","last_unstale":"2024-04-25T10:05:22.133875+0000","last_undegraded":"2024-04-25T10:05:22.133875+0000","last_fullsized":"2024-04-25T10:05:22.133875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:12.228316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135958+0000","last_change":"2024-04-25T10:05:21.088835+0000","last_active":"2024-04-25T10:05:22.135958+0000","last_peered":"2024-04-25T10:05:22.135958+0000","last_clean":"2024-04-25T10:05:22.135958+0000","last_became_active":"2024-04-25T10:05:21.088476+0000","last_became_peered":"2024-04-25T10:05:21.088476+0000","last_unstale":"2024-04-25T10:05:22.135958+0000","last_undegraded":"2024-04-25T10:05:22.135958+0000","last_fullsized":"2024-04-25T10:05:22.135958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:54.535451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048102+0000","last_change":"2024-04-25T10:05:20.066666+0000","last_active":"2024-04-25T10:05:22.048102+0000","last_peered":"2024-04-25T10:05:22.048102+0000","last_clean":"2024-04-25T10:05:22.048102+0000","last_became_active":"2024-04-25T10:05:20.065812+0000","last_became_peered":"2024-04-25T10:05:20.065812+0000","last_unstale":"2024-04-25T10:05:22.048102+0000","last_undegraded":"2024-04-25T10:05:22.048102+0000","last_fullsized":"2024-04-25T10:05:22.048102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:45.802025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136886+0000","last_change":"2024-04-25T10:05:21.089032+0000","last_active":"2024-04-25T10:05:22.136886+0000","last_peered":"2024-04-25T10:05:22.136886+0000","last_clean":"2024-04-25T10:05:22.136886+0000","last_became_active":"2024-04-25T10:05:21.088707+0000","last_became_peered":"2024-04-25T10:05:21.088707+0000","last_unstale":"2024-04-25T10:05:22.136886+0000","last_undegraded":"2024-04-25T10:05:22.136886+0000","last_fullsized":"2024-04-25T10:05:22.136886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:29.342928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135030+0000","last_change":"2024-04-25T10:05:20.056234+0000","last_active":"2024-04-25T10:05:22.135030+0000","last_peered":"2024-04-25T10:05:22.135030+0000","last_clean":"2024-04-25T10:05:22.135030+0000","last_became_active":"2024-04-25T10:05:20.055711+0000","last_became_peered":"2024-04-25T10:05:20.055711+0000","last_unstale":"2024-04-25T10:05:22.135030+0000","last_undegraded":"2024-04-25T10:05:22.135030+0000","last_fullsized":"2024-04-25T10:05:22.135030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:44.794815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137621+0000","last_change":"2024-04-25T10:05:21.097805+0000","last_active":"2024-04-25T10:05:22.137621+0000","last_peered":"2024-04-25T10:05:22.137621+0000","last_clean":"2024-04-25T10:05:22.137621+0000","last_became_active":"2024-04-25T10:05:21.097066+0000","last_became_peered":"2024-04-25T10:05:21.097066+0000","last_unstale":"2024-04-25T10:05:22.137621+0000","last_undegraded":"2024-04-25T10:05:22.137621+0000","last_fullsized":"2024-04-25T10:05:22.137621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:10.508421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043379+0000","last_change":"2024-04-25T10:05:20.063861+0000","last_active":"2024-04-25T10:05:22.043379+0000","last_peered":"2024-04-25T10:05:22.043379+0000","last_clean":"2024-04-25T10:05:22.043379+0000","last_became_active":"2024-04-25T10:05:20.062501+0000","last_became_peered":"2024-04-25T10:05:20.062501+0000","last_unstale":"2024-04-25T10:05:22.043379+0000","last_undegraded":"2024-04-25T10:05:22.043379+0000","last_fullsized":"2024-04-25T10:05:22.043379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:02.884927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.049234+0000","last_change":"2024-04-25T10:05:21.083111+0000","last_active":"2024-04-25T10:05:22.049234+0000","last_peered":"2024-04-25T10:05:22.049234+0000","last_clean":"2024-04-25T10:05:22.049234+0000","last_became_active":"2024-04-25T10:05:21.082321+0000","last_became_peered":"2024-04-25T10:05:21.082321+0000","last_unstale":"2024-04-25T10:05:22.049234+0000","last_undegraded":"2024-04-25T10:05:22.049234+0000","last_fullsized":"2024-04-25T10:05:22.049234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:02.305355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135030+0000","last_change":"2024-04-25T10:05:20.078355+0000","last_active":"2024-04-25T10:05:22.135030+0000","last_peered":"2024-04-25T10:05:22.135030+0000","last_clean":"2024-04-25T10:05:22.135030+0000","last_became_active":"2024-04-25T10:05:20.077677+0000","last_became_peered":"2024-04-25T10:05:20.077677+0000","last_unstale":"2024-04-25T10:05:22.135030+0000","last_undegraded":"2024-04-25T10:05:22.135030+0000","last_fullsized":"2024-04-25T10:05:22.135030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:35.252363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136520+0000","last_change":"2024-04-25T10:05:21.089183+0000","last_active":"2024-04-25T10:05:22.136520+0000","last_peered":"2024-04-25T10:05:22.136520+0000","last_clean":"2024-04-25T10:05:22.136520+0000","last_became_active":"2024-04-25T10:05:21.088768+0000","last_became_peered":"2024-04-25T10:05:21.088768+0000","last_unstale":"2024-04-25T10:05:22.136520+0000","last_undegraded":"2024-04-25T10:05:22.136520+0000","last_fullsized":"2024-04-25T10:05:22.136520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:06.419537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135576+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:05:22.135576+0000","last_peered":"2024-04-25T10:05:22.135576+0000","last_clean":"2024-04-25T10:05:22.135576+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:05:22.135576+0000","last_undegraded":"2024-04-25T10:05:22.135576+0000","last_fullsized":"2024-04-25T10:05:22.135576+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047506+0000","last_change":"2024-04-25T10:05:20.069430+0000","last_active":"2024-04-25T10:05:22.047506+0000","last_peered":"2024-04-25T10:05:22.047506+0000","last_clean":"2024-04-25T10:05:22.047506+0000","last_became_active":"2024-04-25T10:05:20.068755+0000","last_became_peered":"2024-04-25T10:05:20.068755+0000","last_unstale":"2024-04-25T10:05:22.047506+0000","last_undegraded":"2024-04-25T10:05:22.047506+0000","last_fullsized":"2024-04-25T10:05:22.047506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:56.779127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136220+0000","last_change":"2024-04-25T10:05:21.071861+0000","last_active":"2024-04-25T10:05:22.136220+0000","last_peered":"2024-04-25T10:05:22.136220+0000","last_clean":"2024-04-25T10:05:22.136220+0000","last_became_active":"2024-04-25T10:05:21.071182+0000","last_became_peered":"2024-04-25T10:05:21.071182+0000","last_unstale":"2024-04-25T10:05:22.136220+0000","last_undegraded":"2024-04-25T10:05:22.136220+0000","last_fullsized":"2024-04-25T10:05:22.136220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:17.144179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135408+0000","last_change":"2024-04-25T10:05:20.084051+0000","last_active":"2024-04-25T10:05:22.135408+0000","last_peered":"2024-04-25T10:05:22.135408+0000","last_clean":"2024-04-25T10:05:22.135408+0000","last_became_active":"2024-04-25T10:05:20.083563+0000","last_became_peered":"2024-04-25T10:05:20.083563+0000","last_unstale":"2024-04-25T10:05:22.135408+0000","last_undegraded":"2024-04-25T10:05:22.135408+0000","last_fullsized":"2024-04-25T10:05:22.135408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:20.779254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135004+0000","last_change":"2024-04-25T10:05:21.072824+0000","last_active":"2024-04-25T10:05:22.135004+0000","last_peered":"2024-04-25T10:05:22.135004+0000","last_clean":"2024-04-25T10:05:22.135004+0000","last_became_active":"2024-04-25T10:05:21.071691+0000","last_became_peered":"2024-04-25T10:05:21.071691+0000","last_unstale":"2024-04-25T10:05:22.135004+0000","last_undegraded":"2024-04-25T10:05:22.135004+0000","last_fullsized":"2024-04-25T10:05:22.135004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:21.545921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135509+0000","last_change":"2024-04-25T10:05:20.056953+0000","last_active":"2024-04-25T10:05:22.135509+0000","last_peered":"2024-04-25T10:05:22.135509+0000","last_clean":"2024-04-25T10:05:22.135509+0000","last_became_active":"2024-04-25T10:05:20.056056+0000","last_became_peered":"2024-04-25T10:05:20.056056+0000","last_unstale":"2024-04-25T10:05:22.135509+0000","last_undegraded":"2024-04-25T10:05:22.135509+0000","last_fullsized":"2024-04-25T10:05:22.135509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042576+0000","last_change":"2024-04-25T10:05:21.056503+0000","last_active":"2024-04-25T10:05:22.042576+0000","last_peered":"2024-04-25T10:05:22.042576+0000","last_clean":"2024-04-25T10:05:22.042576+0000","last_became_active":"2024-04-25T10:05:21.055664+0000","last_became_peered":"2024-04-25T10:05:21.055664+0000","last_unstale":"2024-04-25T10:05:22.042576+0000","last_undegraded":"2024-04-25T10:05:22.042576+0000","last_fullsized":"2024-04-25T10:05:22.042576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:20.349799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047779+0000","last_change":"2024-04-25T10:05:20.061869+0000","last_active":"2024-04-25T10:05:22.047779+0000","last_peered":"2024-04-25T10:05:22.047779+0000","last_clean":"2024-04-25T10:05:22.047779+0000","last_became_active":"2024-04-25T10:05:20.060232+0000","last_became_peered":"2024-04-25T10:05:20.060232+0000","last_unstale":"2024-04-25T10:05:22.047779+0000","last_undegraded":"2024-04-25T10:05:22.047779+0000","last_fullsized":"2024-04-25T10:05:22.047779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:50.073175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047317+0000","last_change":"2024-04-25T10:05:21.083202+0000","last_active":"2024-04-25T10:05:22.047317+0000","last_peered":"2024-04-25T10:05:22.047317+0000","last_clean":"2024-04-25T10:05:22.047317+0000","last_became_active":"2024-04-25T10:05:21.082857+0000","last_became_peered":"2024-04-25T10:05:21.082857+0000","last_unstale":"2024-04-25T10:05:22.047317+0000","last_undegraded":"2024-04-25T10:05:22.047317+0000","last_fullsized":"2024-04-25T10:05:22.047317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:49.496977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042931+0000","last_change":"2024-04-25T10:05:20.060703+0000","last_active":"2024-04-25T10:05:22.042931+0000","last_peered":"2024-04-25T10:05:22.042931+0000","last_clean":"2024-04-25T10:05:22.042931+0000","last_became_active":"2024-04-25T10:05:20.056864+0000","last_became_peered":"2024-04-25T10:05:20.056864+0000","last_unstale":"2024-04-25T10:05:22.042931+0000","last_undegraded":"2024-04-25T10:05:22.042931+0000","last_fullsized":"2024-04-25T10:05:22.042931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:58.129770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.049114+0000","last_change":"2024-04-25T10:05:21.081253+0000","last_active":"2024-04-25T10:05:22.049114+0000","last_peered":"2024-04-25T10:05:22.049114+0000","last_clean":"2024-04-25T10:05:22.049114+0000","last_became_active":"2024-04-25T10:05:21.080917+0000","last_became_peered":"2024-04-25T10:05:21.080917+0000","last_unstale":"2024-04-25T10:05:22.049114+0000","last_undegraded":"2024-04-25T10:05:22.049114+0000","last_fullsized":"2024-04-25T10:05:22.049114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:41.708275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136822+0000","last_change":"2024-04-25T10:05:20.089660+0000","last_active":"2024-04-25T10:05:22.136822+0000","last_peered":"2024-04-25T10:05:22.136822+0000","last_clean":"2024-04-25T10:05:22.136822+0000","last_became_active":"2024-04-25T10:05:20.088996+0000","last_became_peered":"2024-04-25T10:05:20.088996+0000","last_unstale":"2024-04-25T10:05:22.136822+0000","last_undegraded":"2024-04-25T10:05:22.136822+0000","last_fullsized":"2024-04-25T10:05:22.136822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:55.264435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.042818+0000","last_change":"2024-04-25T10:05:21.061203+0000","last_active":"2024-04-25T10:05:22.042818+0000","last_peered":"2024-04-25T10:05:22.042818+0000","last_clean":"2024-04-25T10:05:22.042818+0000","last_became_active":"2024-04-25T10:05:21.060108+0000","last_became_peered":"2024-04-25T10:05:21.060108+0000","last_unstale":"2024-04-25T10:05:22.042818+0000","last_undegraded":"2024-04-25T10:05:22.042818+0000","last_fullsized":"2024-04-25T10:05:22.042818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:11.969876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.077357+0000","last_change":"2024-04-25T10:05:20.071770+0000","last_active":"2024-04-25T10:05:22.077357+0000","last_peered":"2024-04-25T10:05:22.077357+0000","last_clean":"2024-04-25T10:05:22.077357+0000","last_became_active":"2024-04-25T10:05:20.071460+0000","last_became_peered":"2024-04-25T10:05:20.071460+0000","last_unstale":"2024-04-25T10:05:22.077357+0000","last_undegraded":"2024-04-25T10:05:22.077357+0000","last_fullsized":"2024-04-25T10:05:22.077357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:20.779254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136359+0000","last_change":"2024-04-25T10:05:21.063585+0000","last_active":"2024-04-25T10:05:22.136359+0000","last_peered":"2024-04-25T10:05:22.136359+0000","last_clean":"2024-04-25T10:05:22.136359+0000","last_became_active":"2024-04-25T10:05:21.062481+0000","last_became_peered":"2024-04-25T10:05:21.062481+0000","last_unstale":"2024-04-25T10:05:22.136359+0000","last_undegraded":"2024-04-25T10:05:22.136359+0000","last_fullsized":"2024-04-25T10:05:22.136359+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:47.724037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080795+0000","last_change":"2024-04-25T10:05:20.086795+0000","last_active":"2024-04-25T10:05:22.080795+0000","last_peered":"2024-04-25T10:05:22.080795+0000","last_clean":"2024-04-25T10:05:22.080795+0000","last_became_active":"2024-04-25T10:05:20.086157+0000","last_became_peered":"2024-04-25T10:05:20.086157+0000","last_unstale":"2024-04-25T10:05:22.080795+0000","last_undegraded":"2024-04-25T10:05:22.080795+0000","last_fullsized":"2024-04-25T10:05:22.080795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:35.252363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135349+0000","last_change":"2024-04-25T10:05:21.080072+0000","last_active":"2024-04-25T10:05:22.135349+0000","last_peered":"2024-04-25T10:05:22.135349+0000","last_clean":"2024-04-25T10:05:22.135349+0000","last_became_active":"2024-04-25T10:05:21.077807+0000","last_became_peered":"2024-04-25T10:05:21.077807+0000","last_unstale":"2024-04-25T10:05:22.135349+0000","last_undegraded":"2024-04-25T10:05:22.135349+0000","last_fullsized":"2024-04-25T10:05:22.135349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:23.519531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135461+0000","last_change":"2024-04-25T10:05:20.074066+0000","last_active":"2024-04-25T10:05:22.135461+0000","last_peered":"2024-04-25T10:05:22.135461+0000","last_clean":"2024-04-25T10:05:22.135461+0000","last_became_active":"2024-04-25T10:05:20.073313+0000","last_became_peered":"2024-04-25T10:05:20.073313+0000","last_unstale":"2024-04-25T10:05:22.135461+0000","last_undegraded":"2024-04-25T10:05:22.135461+0000","last_fullsized":"2024-04-25T10:05:22.135461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:35.144583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.047063+0000","last_change":"2024-04-25T10:05:21.070227+0000","last_active":"2024-04-25T10:05:22.047063+0000","last_peered":"2024-04-25T10:05:22.047063+0000","last_clean":"2024-04-25T10:05:22.047063+0000","last_became_active":"2024-04-25T10:05:21.069473+0000","last_became_peered":"2024-04-25T10:05:21.069473+0000","last_unstale":"2024-04-25T10:05:22.047063+0000","last_undegraded":"2024-04-25T10:05:22.047063+0000","last_fullsized":"2024-04-25T10:05:22.047063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:17.204276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043069+0000","last_change":"2024-04-25T10:05:20.073767+0000","last_active":"2024-04-25T10:05:22.043069+0000","last_peered":"2024-04-25T10:05:22.043069+0000","last_clean":"2024-04-25T10:05:22.043069+0000","last_became_active":"2024-04-25T10:05:20.073333+0000","last_became_peered":"2024-04-25T10:05:20.073333+0000","last_unstale":"2024-04-25T10:05:22.043069+0000","last_undegraded":"2024-04-25T10:05:22.043069+0000","last_fullsized":"2024-04-25T10:05:22.043069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:48.493967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.135540+0000","last_change":"2024-04-25T10:05:21.070624+0000","last_active":"2024-04-25T10:05:22.135540+0000","last_peered":"2024-04-25T10:05:22.135540+0000","last_clean":"2024-04-25T10:05:22.135540+0000","last_became_active":"2024-04-25T10:05:21.065398+0000","last_became_peered":"2024-04-25T10:05:21.065398+0000","last_unstale":"2024-04-25T10:05:22.135540+0000","last_undegraded":"2024-04-25T10:05:22.135540+0000","last_fullsized":"2024-04-25T10:05:22.135540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:33.904915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.043074+0000","last_change":"2024-04-25T10:05:20.055362+0000","last_active":"2024-04-25T10:05:22.043074+0000","last_peered":"2024-04-25T10:05:22.043074+0000","last_clean":"2024-04-25T10:05:22.043074+0000","last_became_active":"2024-04-25T10:05:20.052279+0000","last_became_peered":"2024-04-25T10:05:20.052279+0000","last_unstale":"2024-04-25T10:05:22.043074+0000","last_undegraded":"2024-04-25T10:05:22.043074+0000","last_fullsized":"2024-04-25T10:05:22.043074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:35.144583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137237+0000","last_change":"2024-04-25T10:05:21.075800+0000","last_active":"2024-04-25T10:05:22.137237+0000","last_peered":"2024-04-25T10:05:22.137237+0000","last_clean":"2024-04-25T10:05:22.137237+0000","last_became_active":"2024-04-25T10:05:21.075425+0000","last_became_peered":"2024-04-25T10:05:21.075425+0000","last_unstale":"2024-04-25T10:05:22.137237+0000","last_undegraded":"2024-04-25T10:05:22.137237+0000","last_fullsized":"2024-04-25T10:05:22.137237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:02.817716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136402+0000","last_change":"2024-04-25T10:05:20.085669+0000","last_active":"2024-04-25T10:05:22.136402+0000","last_peered":"2024-04-25T10:05:22.136402+0000","last_clean":"2024-04-25T10:05:22.136402+0000","last_became_active":"2024-04-25T10:05:20.084890+0000","last_became_peered":"2024-04-25T10:05:20.084890+0000","last_unstale":"2024-04-25T10:05:22.136402+0000","last_undegraded":"2024-04-25T10:05:22.136402+0000","last_fullsized":"2024-04-25T10:05:22.136402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:16.201265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.080121+0000","last_change":"2024-04-25T10:05:21.072582+0000","last_active":"2024-04-25T10:05:22.080121+0000","last_peered":"2024-04-25T10:05:22.080121+0000","last_clean":"2024-04-25T10:05:22.080121+0000","last_became_active":"2024-04-25T10:05:21.071786+0000","last_became_peered":"2024-04-25T10:05:21.071786+0000","last_unstale":"2024-04-25T10:05:22.080121+0000","last_undegraded":"2024-04-25T10:05:22.080121+0000","last_fullsized":"2024-04-25T10:05:22.080121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:21.545921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048737+0000","last_change":"2024-04-25T10:05:21.069632+0000","last_active":"2024-04-25T10:05:22.048737+0000","last_peered":"2024-04-25T10:05:22.048737+0000","last_clean":"2024-04-25T10:05:22.048737+0000","last_became_active":"2024-04-25T10:05:21.068870+0000","last_became_peered":"2024-04-25T10:05:21.068870+0000","last_unstale":"2024-04-25T10:05:22.048737+0000","last_undegraded":"2024-04-25T10:05:22.048737+0000","last_fullsized":"2024-04-25T10:05:22.048737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:09.103889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136270+0000","last_change":"2024-04-25T10:05:20.085837+0000","last_active":"2024-04-25T10:05:22.136270+0000","last_peered":"2024-04-25T10:05:22.136270+0000","last_clean":"2024-04-25T10:05:22.136270+0000","last_became_active":"2024-04-25T10:05:20.085079+0000","last_became_peered":"2024-04-25T10:05:20.085079+0000","last_unstale":"2024-04-25T10:05:22.136270+0000","last_undegraded":"2024-04-25T10:05:22.136270+0000","last_fullsized":"2024-04-25T10:05:22.136270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:02.884927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.048913+0000","last_change":"2024-04-25T10:05:21.073527+0000","last_active":"2024-04-25T10:05:22.048913+0000","last_peered":"2024-04-25T10:05:22.048913+0000","last_clean":"2024-04-25T10:05:22.048913+0000","last_became_active":"2024-04-25T10:05:21.072780+0000","last_became_peered":"2024-04-25T10:05:21.072780+0000","last_unstale":"2024-04-25T10:05:22.048913+0000","last_undegraded":"2024-04-25T10:05:22.048913+0000","last_fullsized":"2024-04-25T10:05:22.048913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:58.306898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136676+0000","last_change":"2024-04-25T10:05:20.088969+0000","last_active":"2024-04-25T10:05:22.136676+0000","last_peered":"2024-04-25T10:05:22.136676+0000","last_clean":"2024-04-25T10:05:22.136676+0000","last_became_active":"2024-04-25T10:05:20.088074+0000","last_became_peered":"2024-04-25T10:05:20.088074+0000","last_unstale":"2024-04-25T10:05:22.136676+0000","last_undegraded":"2024-04-25T10:05:22.136676+0000","last_fullsized":"2024-04-25T10:05:22.136676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:48.493967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134241+0000","last_change":"2024-04-25T10:05:21.081805+0000","last_active":"2024-04-25T10:05:22.134241+0000","last_peered":"2024-04-25T10:05:22.134241+0000","last_clean":"2024-04-25T10:05:22.134241+0000","last_became_active":"2024-04-25T10:05:21.081062+0000","last_became_peered":"2024-04-25T10:05:21.081062+0000","last_unstale":"2024-04-25T10:05:22.134241+0000","last_undegraded":"2024-04-25T10:05:22.134241+0000","last_fullsized":"2024-04-25T10:05:22.134241+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:55.635372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.137045+0000","last_change":"2024-04-25T10:05:20.060475+0000","last_active":"2024-04-25T10:05:22.137045+0000","last_peered":"2024-04-25T10:05:22.137045+0000","last_clean":"2024-04-25T10:05:22.137045+0000","last_became_active":"2024-04-25T10:05:20.060153+0000","last_became_peered":"2024-04-25T10:05:20.060153+0000","last_unstale":"2024-04-25T10:05:22.137045+0000","last_undegraded":"2024-04-25T10:05:22.137045+0000","last_fullsized":"2024-04-25T10:05:22.137045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:12.228316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.136396+0000","last_change":"2024-04-25T10:05:21.085394+0000","last_active":"2024-04-25T10:05:22.136396+0000","last_peered":"2024-04-25T10:05:22.136396+0000","last_clean":"2024-04-25T10:05:22.136396+0000","last_became_active":"2024-04-25T10:05:21.085122+0000","last_became_peered":"2024-04-25T10:05:21.085122+0000","last_unstale":"2024-04-25T10:05:22.136396+0000","last_undegraded":"2024-04-25T10:05:22.136396+0000","last_fullsized":"2024-04-25T10:05:22.136396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:20.029881+0000","last_clean_scrub_stamp":"2024-04-25T10:05:20.029881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:39.569330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:05:22.134727+0000","last_change":"2024-04-25T10:05:20.085225+0000","last_active":"2024-04-25T10:05:22.134727+0000","last_peered":"2024-04-25T10:05:22.134727+0000","last_clean":"2024-04-25T10:05:22.134727+0000","last_became_active":"2024-04-25T10:05:20.084907+0000","last_became_peered":"2024-04-25T10:05:20.084907+0000","last_unstale":"2024-04-25T10:05:22.134727+0000","last_undegraded":"2024-04-25T10:05:22.134727+0000","last_fullsized":"2024-04-25T10:05:22.134727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:19.026871+0000","last_clean_scrub_stamp":"2024-04-25T10:05:19.026871+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":233472,"data_stored":79812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":221184,"data_stored":78200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":78360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":827392,"data_stored":669206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:31.412 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:05:31.412 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T10:05:31.413 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T10:05:31.413 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:05:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.555+0000 7fce48aa2640 1 Processor -- start 2024-04-25T10:05:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.555+0000 7fce48aa2640 1 -- start start 2024-04-25T10:05:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.555+0000 7fce48aa2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce4415dc90 0x7fce4415e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.555+0000 7fce48aa2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fce4415e5b0 0x7fce44071250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.555+0000 7fce48aa2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4415b3f0 con 0x7fce44071790 2024-04-25T10:05:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.555+0000 7fce48aa2640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce44071c00 con 0x7fce4415e5b0 2024-04-25T10:05:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.555+0000 7fce48aa2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4415a7e0 con 0x7fce4415dc90 2024-04-25T10:05:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce41d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fce4415e5b0 0x7fce44071250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce42d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fce44071790 0x7fce44150740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37314/0 (socket says 172.21.15.60:37314) 2024-04-25T10:05:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce42d9d640 1 -- 172.21.15.60:0/1238557514 learned_addr learned my addr 172.21.15.60:0/1238557514 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce4259c640 1 --2- 172.21.15.60:0/1238557514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce4415dc90 0x7fce4415e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce41d9b640 1 -- 172.21.15.60:0/1238557514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce4415dc90 msgr2=0x7fce4415e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce41d9b640 1 --2- 172.21.15.60:0/1238557514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce4415dc90 0x7fce4415e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce41d9b640 1 -- 172.21.15.60:0/1238557514 >> v1:172.21.15.60:6789/0 conn(0x7fce44071790 legacy=0x7fce44150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce41d9b640 1 -- 172.21.15.60:0/1238557514 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4415aac0 con 0x7fce4415e5b0 2024-04-25T10:05:31.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce4259c640 1 --2- 172.21.15.60:0/1238557514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce4415dc90 0x7fce4415e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:31.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce41d9b640 1 --2- 172.21.15.60:0/1238557514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4415e5b0 0x7fce44071250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fce3800a960 tx=0x7fce3801e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efa91772cde9479 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce4159a640 1 -- 172.21.15.60:0/1238557514 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce380b4070 con 0x7fce4415e5b0 2024-04-25T10:05:31.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce4159a640 1 -- 172.21.15.60:0/1238557514 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce380a78c0 con 0x7fce4415e5b0 2024-04-25T10:05:31.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce4159a640 1 -- 172.21.15.60:0/1238557514 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3800b040 con 0x7fce4415e5b0 2024-04-25T10:05:31.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce48aa2640 1 -- 172.21.15.60:0/1238557514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4415e5b0 msgr2=0x7fce44071250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce48aa2640 1 --2- 172.21.15.60:0/1238557514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4415e5b0 0x7fce44071250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fce3800a960 tx=0x7fce3801e980 comp rx=0 tx=0).stop 2024-04-25T10:05:31.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce48aa2640 1 -- 172.21.15.60:0/1238557514 shutdown_connections 2024-04-25T10:05:31.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce48aa2640 1 --2- 172.21.15.60:0/1238557514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4415e5b0 0x7fce44071250 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce48aa2640 1 --2- 172.21.15.60:0/1238557514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce4415dc90 0x7fce4415e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce48aa2640 1 -- 172.21.15.60:0/1238557514 >> 172.21.15.60:0/1238557514 conn(0x7fce44067080 msgr2=0x7fce4406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce48aa2640 1 -- 172.21.15.60:0/1238557514 shutdown_connections 2024-04-25T10:05:31.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.559+0000 7fce48aa2640 1 -- 172.21.15.60:0/1238557514 wait complete. 2024-04-25T10:05:31.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce48aa2640 1 Processor -- start 2024-04-25T10:05:31.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce48aa2640 1 -- start start 2024-04-25T10:05:31.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce48aa2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce44071790 0x7fce441500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce48aa2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4415dc90 0x7fce44159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce48aa2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce4415e5b0 0x7fce4414dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce48aa2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce441513e0 con 0x7fce4415dc90 2024-04-25T10:05:31.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce48aa2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce44151260 con 0x7fce4415e5b0 2024-04-25T10:05:31.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce48aa2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4406f1a0 con 0x7fce44071790 2024-04-25T10:05:31.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce41d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4415dc90 0x7fce44159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce41d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4415dc90 0x7fce44159380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46570/0 (socket says 172.21.15.60:46570) 2024-04-25T10:05:31.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce41d9b640 1 -- 172.21.15.60:0/1910009423 learned_addr learned my addr 172.21.15.60:0/1910009423 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:31.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce4259c640 1 --2- 172.21.15.60:0/1910009423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce44071790 0x7fce441500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce42d9d640 1 --2- 172.21.15.60:0/1910009423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce4415e5b0 0x7fce4414dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce41d9b640 1 -- 172.21.15.60:0/1910009423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce44071790 msgr2=0x7fce441500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce41d9b640 1 --2- 172.21.15.60:0/1910009423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce44071790 0x7fce441500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce41d9b640 1 -- 172.21.15.60:0/1910009423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce4415e5b0 msgr2=0x7fce4414dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce41d9b640 1 --2- 172.21.15.60:0/1910009423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce4415e5b0 0x7fce4414dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce41d9b640 1 -- 172.21.15.60:0/1910009423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4414e0e0 con 0x7fce4415dc90 2024-04-25T10:05:31.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce41d9b640 1 --2- 172.21.15.60:0/1910009423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4415dc90 0x7fce44159380 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce380b2040 tx=0x7fce3801b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce32ffd640 1 -- 172.21.15.60:0/1910009423 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce380be020 con 0x7fce4415dc90 2024-04-25T10:05:31.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce32ffd640 1 -- 172.21.15.60:0/1910009423 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce380af640 con 0x7fce4415dc90 2024-04-25T10:05:31.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce32ffd640 1 -- 172.21.15.60:0/1910009423 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce380b4040 con 0x7fce4415dc90 2024-04-25T10:05:31.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce48aa2640 1 -- 172.21.15.60:0/1910009423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce441526e0 con 0x7fce4415dc90 2024-04-25T10:05:31.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce48aa2640 1 -- 172.21.15.60:0/1910009423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce44152ca0 con 0x7fce4415dc90 2024-04-25T10:05:31.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.563+0000 7fce48aa2640 1 -- 172.21.15.60:0/1910009423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce44074770 con 0x7fce4415dc90 2024-04-25T10:05:31.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.567+0000 7fce32ffd640 1 -- 172.21.15.60:0/1910009423 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce380afa30 con 0x7fce4415dc90 2024-04-25T10:05:31.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.567+0000 7fce32ffd640 1 --2- 172.21.15.60:0/1910009423 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fce24066fb0 0x7fce24069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.567+0000 7fce32ffd640 1 -- 172.21.15.60:0/1910009423 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce38121390 con 0x7fce4415dc90 2024-04-25T10:05:31.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.567+0000 7fce4259c640 1 --2- 172.21.15.60:0/1910009423 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fce24066fb0 0x7fce24069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.567+0000 7fce4259c640 1 --2- 172.21.15.60:0/1910009423 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fce24066fb0 0x7fce24069470 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fce44159c40 tx=0x7fce34008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.571+0000 7fce32ffd640 1 -- 172.21.15.60:0/1910009423 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce380c8060 con 0x7fce4415dc90 2024-04-25T10:05:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.755+0000 7fce48aa2640 1 -- 172.21.15.60:0/1910009423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fce4415b5e0 con 0x7fce4415dc90 2024-04-25T10:05:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.759+0000 7fce32ffd640 1 -- 172.21.15.60:0/1910009423 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fce38018570 con 0x7fce4415dc90 2024-04-25T10:05:31.764 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:31.765 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:05:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.759+0000 7fce48aa2640 1 -- 172.21.15.60:0/1910009423 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fce24066fb0 msgr2=0x7fce24069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.759+0000 7fce48aa2640 1 --2- 172.21.15.60:0/1910009423 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fce24066fb0 0x7fce24069470 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fce44159c40 tx=0x7fce34008040 comp rx=0 tx=0).stop 2024-04-25T10:05:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.759+0000 7fce48aa2640 1 -- 172.21.15.60:0/1910009423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4415dc90 msgr2=0x7fce44159380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.759+0000 7fce48aa2640 1 --2- 172.21.15.60:0/1910009423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4415dc90 0x7fce44159380 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce380b2040 tx=0x7fce3801b720 comp rx=0 tx=0).stop 2024-04-25T10:05:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.759+0000 7fce48aa2640 1 -- 172.21.15.60:0/1910009423 shutdown_connections 2024-04-25T10:05:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.759+0000 7fce48aa2640 1 --2- 172.21.15.60:0/1910009423 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fce24066fb0 0x7fce24069470 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.759+0000 7fce48aa2640 1 --2- 172.21.15.60:0/1910009423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce4415e5b0 0x7fce4414dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.759+0000 7fce48aa2640 1 --2- 172.21.15.60:0/1910009423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4415dc90 0x7fce44159380 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.759+0000 7fce48aa2640 1 --2- 172.21.15.60:0/1910009423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce44071790 0x7fce441500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.759+0000 7fce48aa2640 1 -- 172.21.15.60:0/1910009423 >> 172.21.15.60:0/1910009423 conn(0x7fce44067080 msgr2=0x7fce4406d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.759+0000 7fce48aa2640 1 -- 172.21.15.60:0/1910009423 shutdown_connections 2024-04-25T10:05:31.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.759+0000 7fce48aa2640 1 -- 172.21.15.60:0/1910009423 wait complete. 2024-04-25T10:05:31.783 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T10:05:31.783 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:31.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8c0def640 1 Processor -- start 2024-04-25T10:05:31.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8c0def640 1 -- start start 2024-04-25T10:05:31.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8c0def640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe8bc0d0a50 0x7fe8bc0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8c0def640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe8bc0ce2b0 0x7fe8bc0ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8c0def640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8bc06d200 con 0x7fe8bc15deb0 2024-04-25T10:05:31.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8c0def640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8bc0cf2c0 con 0x7fe8bc0d0a50 2024-04-25T10:05:31.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8c0def640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8bc0cefe0 con 0x7fe8bc0ce2b0 2024-04-25T10:05:31.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8b9d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe8bc0d0a50 0x7fe8bc0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8ba59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe8bc15deb0 0x7fe8bc0d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37316/0 (socket says 172.21.15.60:37316) 2024-04-25T10:05:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8ba59c640 1 -- 172.21.15.60:0/3953760601 learned_addr learned my addr 172.21.15.60:0/3953760601 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8b9d9b640 1 -- 172.21.15.60:0/3953760601 >> v2:172.21.15.100:3300/0 conn(0x7fe8bc0ce2b0 msgr2=0x7fe8bc0ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8b9d9b640 1 --2- 172.21.15.60:0/3953760601 >> v2:172.21.15.100:3300/0 conn(0x7fe8bc0ce2b0 0x7fe8bc0ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8b9d9b640 1 -- 172.21.15.60:0/3953760601 >> v1:172.21.15.60:6789/0 conn(0x7fe8bc15deb0 legacy=0x7fe8bc0d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8b9d9b640 1 -- 172.21.15.60:0/3953760601 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc0d1d70 con 0x7fe8bc0d0a50 2024-04-25T10:05:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8b959a640 1 -- 172.21.15.60:0/3953760601 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952943202 0 0) 0x7fe8bc06d200 con 0x7fe8bc15deb0 2024-04-25T10:05:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8b9d9b640 1 --2- 172.21.15.60:0/3953760601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8bc0d0a50 0x7fe8bc0b6c10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0d1590 tx=0x7fe8b001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdca29bbcff5507 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8b959a640 1 -- 172.21.15.60:0/3953760601 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00b0070 con 0x7fe8bc0d0a50 2024-04-25T10:05:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8b959a640 1 -- 172.21.15.60:0/3953760601 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b00ab600 con 0x7fe8bc0d0a50 2024-04-25T10:05:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.891+0000 7fe8b959a640 1 -- 172.21.15.60:0/3953760601 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00ab9d0 con 0x7fe8bc0d0a50 2024-04-25T10:05:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 -- 172.21.15.60:0/3953760601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8bc0d0a50 msgr2=0x7fe8bc0b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 --2- 172.21.15.60:0/3953760601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8bc0d0a50 0x7fe8bc0b6c10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0d1590 tx=0x7fe8b001c480 comp rx=0 tx=0).stop 2024-04-25T10:05:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 -- 172.21.15.60:0/3953760601 shutdown_connections 2024-04-25T10:05:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 --2- 172.21.15.60:0/3953760601 >> v2:172.21.15.100:3300/0 conn(0x7fe8bc0ce2b0 0x7fe8bc0ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 --2- 172.21.15.60:0/3953760601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8bc0d0a50 0x7fe8bc0b6c10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 -- 172.21.15.60:0/3953760601 >> 172.21.15.60:0/3953760601 conn(0x7fe8bc06c090 msgr2=0x7fe8bc06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 -- 172.21.15.60:0/3953760601 shutdown_connections 2024-04-25T10:05:31.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 -- 172.21.15.60:0/3953760601 wait complete. 2024-04-25T10:05:31.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 Processor -- start 2024-04-25T10:05:31.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 -- start start 2024-04-25T10:05:31.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8bc0ce2b0 0x7fe8bc0c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8bc0d0a50 0x7fe8bc14fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8bc15deb0 0x7fe8bc15a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0d31e0 con 0x7fe8bc0ce2b0 2024-04-25T10:05:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0d3060 con 0x7fe8bc15deb0 2024-04-25T10:05:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0d3360 con 0x7fe8bc0d0a50 2024-04-25T10:05:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8ba59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8bc0ce2b0 0x7fe8bc0c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8ba59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8bc0ce2b0 0x7fe8bc0c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46594/0 (socket says 172.21.15.60:46594) 2024-04-25T10:05:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8ba59c640 1 -- 172.21.15.60:0/398457336 learned_addr learned my addr 172.21.15.60:0/398457336 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:31.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8bad9d640 1 --2- 172.21.15.60:0/398457336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8bc15deb0 0x7fe8bc15a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8b9d9b640 1 --2- 172.21.15.60:0/398457336 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8bc0d0a50 0x7fe8bc14fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8ba59c640 1 -- 172.21.15.60:0/398457336 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8bc0d0a50 msgr2=0x7fe8bc14fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8ba59c640 1 --2- 172.21.15.60:0/398457336 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8bc0d0a50 0x7fe8bc14fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8ba59c640 1 -- 172.21.15.60:0/398457336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8bc15deb0 msgr2=0x7fe8bc15a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8ba59c640 1 --2- 172.21.15.60:0/398457336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8bc15deb0 0x7fe8bc15a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8ba59c640 1 -- 172.21.15.60:0/398457336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc0c8950 con 0x7fe8bc0ce2b0 2024-04-25T10:05:31.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8bad9d640 1 --2- 172.21.15.60:0/398457336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8bc15deb0 0x7fe8bc15a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:31.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8ba59c640 1 --2- 172.21.15.60:0/398457336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8bc0ce2b0 0x7fe8bc0c7e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00c480 tx=0x7fe8ac00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8aaffd640 1 -- 172.21.15.60:0/398457336 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac018070 con 0x7fe8bc0ce2b0 2024-04-25T10:05:31.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8aaffd640 1 -- 172.21.15.60:0/398457336 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ac007500 con 0x7fe8bc0ce2b0 2024-04-25T10:05:31.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 -- 172.21.15.60:0/398457336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8bc0c8c30 con 0x7fe8bc0ce2b0 2024-04-25T10:05:31.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8aaffd640 1 -- 172.21.15.60:0/398457336 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac004a30 con 0x7fe8bc0ce2b0 2024-04-25T10:05:31.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.895+0000 7fe8c0def640 1 -- 172.21.15.60:0/398457336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8bc0c91f0 con 0x7fe8bc0ce2b0 2024-04-25T10:05:31.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.899+0000 7fe8c0def640 1 -- 172.21.15.60:0/398457336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8bc074770 con 0x7fe8bc0ce2b0 2024-04-25T10:05:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.899+0000 7fe8aaffd640 1 -- 172.21.15.60:0/398457336 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8ac01b070 con 0x7fe8bc0ce2b0 2024-04-25T10:05:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.899+0000 7fe8aaffd640 1 --2- 172.21.15.60:0/398457336 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe89c066f60 0x7fe89c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.899+0000 7fe8aaffd640 1 -- 172.21.15.60:0/398457336 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8ac084370 con 0x7fe8bc0ce2b0 2024-04-25T10:05:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.899+0000 7fe8b9d9b640 1 --2- 172.21.15.60:0/398457336 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe89c066f60 0x7fe89c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.899+0000 7fe8b9d9b640 1 --2- 172.21.15.60:0/398457336 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe89c066f60 0x7fe89c069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00ad000 tx=0x7fe8b00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:31.903+0000 7fe8aaffd640 1 -- 172.21.15.60:0/398457336 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8ac055410 con 0x7fe8bc0ce2b0 2024-04-25T10:05:32.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.055+0000 7fe8c0def640 1 -- 172.21.15.60:0/398457336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8bc0cfc20 con 0x7fe8bc0ce2b0 2024-04-25T10:05:32.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.059+0000 7fe8aaffd640 1 -- 172.21.15.60:0/398457336 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16617 (secure 0 0 0) 0x7fe8ac054d10 con 0x7fe8bc0ce2b0 2024-04-25T10:05:32.065 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:32.066 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:22.036582+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:32.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.059+0000 7fe8c0def640 1 -- 172.21.15.60:0/398457336 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe89c066f60 msgr2=0x7fe89c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.059+0000 7fe8c0def640 1 --2- 172.21.15.60:0/398457336 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe89c066f60 0x7fe89c069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00ad000 tx=0x7fe8b00a6000 comp rx=0 tx=0).stop 2024-04-25T10:05:32.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.059+0000 7fe8c0def640 1 -- 172.21.15.60:0/398457336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8bc0ce2b0 msgr2=0x7fe8bc0c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.059+0000 7fe8c0def640 1 --2- 172.21.15.60:0/398457336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8bc0ce2b0 0x7fe8bc0c7e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00c480 tx=0x7fe8ac00c940 comp rx=0 tx=0).stop 2024-04-25T10:05:32.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.059+0000 7fe8c0def640 1 -- 172.21.15.60:0/398457336 shutdown_connections 2024-04-25T10:05:32.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.059+0000 7fe8c0def640 1 --2- 172.21.15.60:0/398457336 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe89c066f60 0x7fe89c069420 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.059+0000 7fe8c0def640 1 --2- 172.21.15.60:0/398457336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8bc15deb0 0x7fe8bc15a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.059+0000 7fe8c0def640 1 --2- 172.21.15.60:0/398457336 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8bc0d0a50 0x7fe8bc14fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.059+0000 7fe8c0def640 1 --2- 172.21.15.60:0/398457336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8bc0ce2b0 0x7fe8bc0c7e10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.059+0000 7fe8c0def640 1 -- 172.21.15.60:0/398457336 >> 172.21.15.60:0/398457336 conn(0x7fe8bc06c090 msgr2=0x7fe8bc0d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.059+0000 7fe8c0def640 1 -- 172.21.15.60:0/398457336 shutdown_connections 2024-04-25T10:05:32.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.059+0000 7fe8c0def640 1 -- 172.21.15.60:0/398457336 wait complete. 2024-04-25T10:05:32.088 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:18.976643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:19.405302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:32.089 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26f2153640 1 Processor -- start 2024-04-25T10:05:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26f2153640 1 -- start start 2024-04-25T10:05:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26f2153640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f26ec0b2f70 0x7f26ec0b3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26f2153640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ec162910 con 0x7f26ec162450 2024-04-25T10:05:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26f2153640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ec162bf0 con 0x7f26ec0aeaa0 2024-04-25T10:05:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26f2153640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ec162ed0 con 0x7f26ec0b2f70 2024-04-25T10:05:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26eaffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f26ec162450 0x7f26ec06db70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37322/0 (socket says 172.21.15.60:37322) 2024-04-25T10:05:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26eaffd640 1 -- 172.21.15.60:0/1561889243 learned_addr learned my addr 172.21.15.60:0/1561889243 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ebfff640 1 --2- 172.21.15.60:0/1561889243 >> v2:172.21.15.100:3300/0 conn(0x7f26ec0b2f70 0x7f26ec0b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ea7fc640 1 -- 172.21.15.60:0/1561889243 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300392379 0 0) 0x7f26ec162910 con 0x7f26ec162450 2024-04-25T10:05:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ea7fc640 1 -- 172.21.15.60:0/1561889243 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26d8003530 con 0x7f26ec162450 2024-04-25T10:05:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ea7fc640 1 -- 172.21.15.60:0/1561889243 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2215379436 0 0) 0x7f26d8003530 con 0x7f26ec162450 2024-04-25T10:05:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ea7fc640 1 -- 172.21.15.60:0/1561889243 >> v2:172.21.15.100:3300/0 conn(0x7f26ec0b2f70 msgr2=0x7f26ec0b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ea7fc640 1 --2- 172.21.15.60:0/1561889243 >> v2:172.21.15.100:3300/0 conn(0x7f26ec0b2f70 0x7f26ec0b3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ea7fc640 1 -- 172.21.15.60:0/1561889243 >> v1:172.21.15.100:6789/0 conn(0x7f26ec0aeaa0 legacy=0x7f26ec166450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ea7fc640 1 -- 172.21.15.60:0/1561889243 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec1631b0 con 0x7f26ec162450 2024-04-25T10:05:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ea7fc640 1 -- 172.21.15.60:0/1561889243 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f26d4001680 con 0x7f26ec162450 2024-04-25T10:05:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ea7fc640 1 -- 172.21.15.60:0/1561889243 >> v1:172.21.15.60:6789/0 conn(0x7f26ec162450 legacy=0x7f26ec06db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ea7fc640 1 --2- 172.21.15.60:0/1561889243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d8003a10 0x7f26d8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ea7fc640 1 --2- 172.21.15.60:0/1561889243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d8004810 0x7f26d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ea7fc640 1 --2- 172.21.15.60:0/1561889243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26d80051d0 0x7f26d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ea7fc640 1 -- 172.21.15.60:0/1561889243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec162ed0 con 0x7f26d8003a10 2024-04-25T10:05:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ea7fc640 1 -- 172.21.15.60:0/1561889243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d800c0d0 con 0x7f26d8004810 2024-04-25T10:05:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ea7fc640 1 -- 172.21.15.60:0/1561889243 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d800c2d0 con 0x7f26d80051d0 2024-04-25T10:05:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26eb7fe640 1 --2- 172.21.15.60:0/1561889243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d8003a10 0x7f26d8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ebfff640 1 --2- 172.21.15.60:0/1561889243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26d80051d0 0x7f26d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26eaffd640 1 --2- 172.21.15.60:0/1561889243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d8004810 0x7f26d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26eb7fe640 1 -- 172.21.15.60:0/1561889243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26d80051d0 msgr2=0x7f26d800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:32.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26eb7fe640 1 --2- 172.21.15.60:0/1561889243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26d80051d0 0x7f26d800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26eb7fe640 1 -- 172.21.15.60:0/1561889243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d8004810 msgr2=0x7f26d8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:32.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26eb7fe640 1 --2- 172.21.15.60:0/1561889243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d8004810 0x7f26d8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26eb7fe640 1 -- 172.21.15.60:0/1561889243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d8003530 con 0x7f26d8003a10 2024-04-25T10:05:32.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ebfff640 1 --2- 172.21.15.60:0/1561889243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26d80051d0 0x7f26d800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:32.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26eaffd640 1 --2- 172.21.15.60:0/1561889243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d8004810 0x7f26d8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:32.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.207+0000 7f26ebfff640 1 -- 172.21.15.60:0/1561889243 reap_dead start 2024-04-25T10:05:32.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.211+0000 7f26eb7fe640 1 --2- 172.21.15.60:0/1561889243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d8003a10 0x7f26d8003e30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f26dc001e70 tx=0x7f26dc01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4514428556415f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.211+0000 7f26ea7fc640 1 -- 172.21.15.60:0/1561889243 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc0b0070 con 0x7f26d8003a10 2024-04-25T10:05:32.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.211+0000 7f26ea7fc640 1 -- 172.21.15.60:0/1561889243 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26dc003b80 con 0x7f26d8003a10 2024-04-25T10:05:32.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.211+0000 7f26ea7fc640 1 -- 172.21.15.60:0/1561889243 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc001a90 con 0x7f26d8003a10 2024-04-25T10:05:32.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.211+0000 7f26f2153640 1 -- 172.21.15.60:0/1561889243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d8003a10 msgr2=0x7f26d8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.211+0000 7f26f2153640 1 --2- 172.21.15.60:0/1561889243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d8003a10 0x7f26d8003e30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f26dc001e70 tx=0x7f26dc01c760 comp rx=0 tx=0).stop 2024-04-25T10:05:32.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.211+0000 7f26f2153640 1 -- 172.21.15.60:0/1561889243 shutdown_connections 2024-04-25T10:05:32.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.211+0000 7f26f2153640 1 --2- 172.21.15.60:0/1561889243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d8003a10 0x7f26d8003e30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.211+0000 7f26f2153640 1 -- 172.21.15.60:0/1561889243 >> 172.21.15.60:0/1561889243 conn(0x7f26ec0670b0 msgr2=0x7f26ec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.211+0000 7f26f2153640 1 -- 172.21.15.60:0/1561889243 shutdown_connections 2024-04-25T10:05:32.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.211+0000 7f26f2153640 1 -- 172.21.15.60:0/1561889243 wait complete. 2024-04-25T10:05:32.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.211+0000 7f26f2153640 1 Processor -- start 2024-04-25T10:05:32.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.211+0000 7f26f2153640 1 -- start start 2024-04-25T10:05:32.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.211+0000 7f26f2153640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d8003a10 0x7f26ec155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.211+0000 7f26f2153640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26ec158f80 0x7f26ec159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.211+0000 7f26f2153640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26ec15b8d0 0x7f26ec15bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26f2153640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec0af000 con 0x7f26ec158f80 2024-04-25T10:05:32.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26f2153640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec0aee80 con 0x7f26d8003a10 2024-04-25T10:05:32.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26f2153640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec071e20 con 0x7f26ec15b8d0 2024-04-25T10:05:32.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26eaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26ec158f80 0x7f26ec159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26eaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26ec158f80 0x7f26ec159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46608/0 (socket says 172.21.15.60:46608) 2024-04-25T10:05:32.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26eaffd640 1 -- 172.21.15.60:0/3056203345 learned_addr learned my addr 172.21.15.60:0/3056203345 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:32.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26ebfff640 1 --2- 172.21.15.60:0/3056203345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26ec15b8d0 0x7f26ec15bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26eb7fe640 1 --2- 172.21.15.60:0/3056203345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d8003a10 0x7f26ec155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26eaffd640 1 -- 172.21.15.60:0/3056203345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26ec15b8d0 msgr2=0x7f26ec15bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26eaffd640 1 --2- 172.21.15.60:0/3056203345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26ec15b8d0 0x7f26ec15bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26eaffd640 1 -- 172.21.15.60:0/3056203345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d8003a10 msgr2=0x7f26ec155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26eaffd640 1 --2- 172.21.15.60:0/3056203345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d8003a10 0x7f26ec155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26eaffd640 1 -- 172.21.15.60:0/3056203345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec0b0be0 con 0x7f26ec158f80 2024-04-25T10:05:32.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26eb7fe640 1 --2- 172.21.15.60:0/3056203345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d8003a10 0x7f26ec155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:32.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26eaffd640 1 --2- 172.21.15.60:0/3056203345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26ec158f80 0x7f26ec159400 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f26d400d800 tx=0x7f26d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26e3fff640 1 -- 172.21.15.60:0/3056203345 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4019070 con 0x7f26ec158f80 2024-04-25T10:05:32.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26e3fff640 1 -- 172.21.15.60:0/3056203345 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d40072d0 con 0x7f26ec158f80 2024-04-25T10:05:32.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26f2153640 1 -- 172.21.15.60:0/3056203345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ec0b0ec0 con 0x7f26ec158f80 2024-04-25T10:05:32.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26e3fff640 1 -- 172.21.15.60:0/3056203345 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4007680 con 0x7f26ec158f80 2024-04-25T10:05:32.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26f2153640 1 -- 172.21.15.60:0/3056203345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ec0b1480 con 0x7f26ec158f80 2024-04-25T10:05:32.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26f2153640 1 -- 172.21.15.60:0/3056203345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ec06bba0 con 0x7f26ec158f80 2024-04-25T10:05:32.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26e3fff640 1 -- 172.21.15.60:0/3056203345 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26d40047f0 con 0x7f26ec158f80 2024-04-25T10:05:32.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26e3fff640 1 --2- 172.21.15.60:0/3056203345 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f26cc066f90 0x7f26cc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26e3fff640 1 -- 172.21.15.60:0/3056203345 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26d4084900 con 0x7f26ec158f80 2024-04-25T10:05:32.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26eb7fe640 1 --2- 172.21.15.60:0/3056203345 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f26cc066f90 0x7f26cc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.215+0000 7f26eb7fe640 1 --2- 172.21.15.60:0/3056203345 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f26cc066f90 0x7f26cc069450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f26dc0ad000 tx=0x7f26dc001520 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.219+0000 7f26e3fff640 1 -- 172.21.15.60:0/3056203345 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f26d4055870 con 0x7f26ec158f80 2024-04-25T10:05:32.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.383+0000 7f26f2153640 1 -- 172.21.15.60:0/3056203345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26ec074770 con 0x7f26ec158f80 2024-04-25T10:05:32.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.383+0000 7f26e3fff640 1 -- 172.21.15.60:0/3056203345 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f26d4084030 con 0x7f26ec158f80 2024-04-25T10:05:32.390 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:05:32.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.383+0000 7f26f2153640 1 -- 172.21.15.60:0/3056203345 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f26cc066f90 msgr2=0x7f26cc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.383+0000 7f26f2153640 1 --2- 172.21.15.60:0/3056203345 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f26cc066f90 0x7f26cc069450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f26dc0ad000 tx=0x7f26dc001520 comp rx=0 tx=0).stop 2024-04-25T10:05:32.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.383+0000 7f26f2153640 1 -- 172.21.15.60:0/3056203345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26ec158f80 msgr2=0x7f26ec159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.383+0000 7f26f2153640 1 --2- 172.21.15.60:0/3056203345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26ec158f80 0x7f26ec159400 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f26d400d800 tx=0x7f26d400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.383+0000 7f26f2153640 1 -- 172.21.15.60:0/3056203345 shutdown_connections 2024-04-25T10:05:32.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.383+0000 7f26f2153640 1 --2- 172.21.15.60:0/3056203345 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f26cc066f90 0x7f26cc069450 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.383+0000 7f26f2153640 1 --2- 172.21.15.60:0/3056203345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26ec15b8d0 0x7f26ec15bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.383+0000 7f26f2153640 1 --2- 172.21.15.60:0/3056203345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26ec158f80 0x7f26ec159400 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.383+0000 7f26f2153640 1 --2- 172.21.15.60:0/3056203345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26d8003a10 0x7f26ec155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.383+0000 7f26f2153640 1 -- 172.21.15.60:0/3056203345 >> 172.21.15.60:0/3056203345 conn(0x7f26ec0670b0 msgr2=0x7f26ec160430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.383+0000 7f26f2153640 1 -- 172.21.15.60:0/3056203345 shutdown_connections 2024-04-25T10:05:32.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.383+0000 7f26f2153640 1 -- 172.21.15.60:0/3056203345 wait complete. 2024-04-25T10:05:32.408 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:32.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f73950e8640 1 Processor -- start 2024-04-25T10:05:32.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f73950e8640 1 -- start start 2024-04-25T10:05:32.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f73950e8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73900b2560 0x7f73900b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f73950e8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7390099a30 0x7f7390099ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f73950e8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73900cc320 con 0x7f739015ef20 2024-04-25T10:05:32.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f73950e8640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73900cc5c0 con 0x7f7390099a30 2024-04-25T10:05:32.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f73950e8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73900d07b0 con 0x7f73900b2560 2024-04-25T10:05:32.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f738f59e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7390099a30 0x7f7390099ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f738ed9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f739015ef20 0x7f73900cfcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37334/0 (socket says 172.21.15.60:37334) 2024-04-25T10:05:32.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f738f59e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7390099a30 0x7f7390099ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46616/0 (socket says 172.21.15.60:46616) 2024-04-25T10:05:32.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f738ed9d640 1 -- 172.21.15.60:0/581415643 learned_addr learned my addr 172.21.15.60:0/581415643 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:32.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f738e59c640 1 --2- 172.21.15.60:0/581415643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73900b2560 0x7f73900b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f738f59e640 1 -- 172.21.15.60:0/581415643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73900b2560 msgr2=0x7f73900b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:32.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f738f59e640 1 --2- 172.21.15.60:0/581415643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73900b2560 0x7f73900b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f738f59e640 1 -- 172.21.15.60:0/581415643 >> v1:172.21.15.60:6789/0 conn(0x7f739015ef20 legacy=0x7f73900cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f738f59e640 1 -- 172.21.15.60:0/581415643 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73900d04d0 con 0x7f7390099a30 2024-04-25T10:05:32.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.527+0000 7f738dd9b640 1 -- 172.21.15.60:0/581415643 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985959192 0 0) 0x7f73900cc320 con 0x7f739015ef20 2024-04-25T10:05:32.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.531+0000 7f738f59e640 1 --2- 172.21.15.60:0/581415643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7390099a30 0x7f7390099ec0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7388009d90 tx=0x7f738801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec408d6753428f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.531+0000 7f738dd9b640 1 -- 172.21.15.60:0/581415643 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73880b2070 con 0x7f7390099a30 2024-04-25T10:05:32.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.531+0000 7f738dd9b640 1 -- 172.21.15.60:0/581415643 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73880a5830 con 0x7f7390099a30 2024-04-25T10:05:32.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.531+0000 7f738dd9b640 1 -- 172.21.15.60:0/581415643 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73880a5c00 con 0x7f7390099a30 2024-04-25T10:05:32.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.531+0000 7f73950e8640 1 -- 172.21.15.60:0/581415643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7390099a30 msgr2=0x7f7390099ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.531+0000 7f73950e8640 1 --2- 172.21.15.60:0/581415643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7390099a30 0x7f7390099ec0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7388009d90 tx=0x7f738801c970 comp rx=0 tx=0).stop 2024-04-25T10:05:32.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.531+0000 7f73950e8640 1 -- 172.21.15.60:0/581415643 shutdown_connections 2024-04-25T10:05:32.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.531+0000 7f73950e8640 1 --2- 172.21.15.60:0/581415643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7390099a30 0x7f7390099ec0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.531+0000 7f73950e8640 1 --2- 172.21.15.60:0/581415643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73900b2560 0x7f73900b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.531+0000 7f73950e8640 1 -- 172.21.15.60:0/581415643 >> 172.21.15.60:0/581415643 conn(0x7f73900b3200 msgr2=0x7f73900b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.531+0000 7f73950e8640 1 -- 172.21.15.60:0/581415643 shutdown_connections 2024-04-25T10:05:32.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.531+0000 7f73950e8640 1 -- 172.21.15.60:0/581415643 wait complete. 2024-04-25T10:05:32.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.531+0000 7f73950e8640 1 Processor -- start 2024-04-25T10:05:32.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.531+0000 7f73950e8640 1 -- start start 2024-04-25T10:05:32.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f73950e8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7390099a30 0x7f73900c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f73950e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73900b2560 0x7f7390151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f73950e8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f739015ef20 0x7f7390159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f73950e8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739015f300 con 0x7f73900b2560 2024-04-25T10:05:32.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f73950e8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73900b2940 con 0x7f7390099a30 2024-04-25T10:05:32.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f73950e8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739015f480 con 0x7f739015ef20 2024-04-25T10:05:32.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f738e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73900b2560 0x7f7390151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f738e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73900b2560 0x7f7390151cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46630/0 (socket says 172.21.15.60:46630) 2024-04-25T10:05:32.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f738e59c640 1 -- 172.21.15.60:0/2393209230 learned_addr learned my addr 172.21.15.60:0/2393209230 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:32.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f738f59e640 1 --2- 172.21.15.60:0/2393209230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f739015ef20 0x7f7390159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f738ed9d640 1 --2- 172.21.15.60:0/2393209230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7390099a30 0x7f73900c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f738e59c640 1 -- 172.21.15.60:0/2393209230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f739015ef20 msgr2=0x7f7390159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f738e59c640 1 --2- 172.21.15.60:0/2393209230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f739015ef20 0x7f7390159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f738e59c640 1 -- 172.21.15.60:0/2393209230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7390099a30 msgr2=0x7f73900c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f738e59c640 1 --2- 172.21.15.60:0/2393209230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7390099a30 0x7f73900c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f738e59c640 1 -- 172.21.15.60:0/2393209230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73900c9c60 con 0x7f73900b2560 2024-04-25T10:05:32.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f738ed9d640 1 --2- 172.21.15.60:0/2393209230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7390099a30 0x7f73900c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:32.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f738e59c640 1 --2- 172.21.15.60:0/2393209230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73900b2560 0x7f7390151cd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f738000d800 tx=0x7f738000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f73777fe640 1 -- 172.21.15.60:0/2393209230 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738000b6c0 con 0x7f73900b2560 2024-04-25T10:05:32.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f73777fe640 1 -- 172.21.15.60:0/2393209230 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7380019070 con 0x7f73900b2560 2024-04-25T10:05:32.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f73777fe640 1 -- 172.21.15.60:0/2393209230 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738000ba60 con 0x7f73900b2560 2024-04-25T10:05:32.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f73950e8640 1 -- 172.21.15.60:0/2393209230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73900c9f40 con 0x7f73900b2560 2024-04-25T10:05:32.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f73950e8640 1 -- 172.21.15.60:0/2393209230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7390152970 con 0x7f73900b2560 2024-04-25T10:05:32.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f73950e8640 1 -- 172.21.15.60:0/2393209230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7390074770 con 0x7f73900b2560 2024-04-25T10:05:32.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f73777fe640 1 -- 172.21.15.60:0/2393209230 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f738000bc00 con 0x7f73900b2560 2024-04-25T10:05:32.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f73777fe640 1 --2- 172.21.15.60:0/2393209230 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f737c066f90 0x7f737c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f73777fe640 1 -- 172.21.15.60:0/2393209230 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73800854d0 con 0x7f73900b2560 2024-04-25T10:05:32.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.535+0000 7f738ed9d640 1 --2- 172.21.15.60:0/2393209230 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f737c066f90 0x7f737c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.539+0000 7f738ed9d640 1 --2- 172.21.15.60:0/2393209230 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f737c066f90 0x7f737c069450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7384000f70 tx=0x7f7384009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.539+0000 7f73777fe640 1 -- 172.21.15.60:0/2393209230 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f73800564d0 con 0x7f73900b2560 2024-04-25T10:05:32.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.695+0000 7f73950e8640 1 -- 172.21.15.60:0/2393209230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f73900b7120 con 0x7f73900b2560 2024-04-25T10:05:32.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.695+0000 7f73777fe640 1 -- 172.21.15.60:0/2393209230 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7380055dd0 con 0x7f73900b2560 2024-04-25T10:05:32.702 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:05:32.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.695+0000 7f73950e8640 1 -- 172.21.15.60:0/2393209230 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f737c066f90 msgr2=0x7f737c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.695+0000 7f73950e8640 1 --2- 172.21.15.60:0/2393209230 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f737c066f90 0x7f737c069450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7384000f70 tx=0x7f7384009040 comp rx=0 tx=0).stop 2024-04-25T10:05:32.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.695+0000 7f73950e8640 1 -- 172.21.15.60:0/2393209230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73900b2560 msgr2=0x7f7390151cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.695+0000 7f73950e8640 1 --2- 172.21.15.60:0/2393209230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73900b2560 0x7f7390151cd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f738000d800 tx=0x7f738000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.695+0000 7f73950e8640 1 -- 172.21.15.60:0/2393209230 shutdown_connections 2024-04-25T10:05:32.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.695+0000 7f73950e8640 1 --2- 172.21.15.60:0/2393209230 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f737c066f90 0x7f737c069450 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.695+0000 7f73950e8640 1 --2- 172.21.15.60:0/2393209230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f739015ef20 0x7f7390159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.695+0000 7f73950e8640 1 --2- 172.21.15.60:0/2393209230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73900b2560 0x7f7390151cd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.695+0000 7f73950e8640 1 --2- 172.21.15.60:0/2393209230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7390099a30 0x7f73900c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.695+0000 7f73950e8640 1 -- 172.21.15.60:0/2393209230 >> 172.21.15.60:0/2393209230 conn(0x7f73900b3200 msgr2=0x7f7390070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.695+0000 7f73950e8640 1 -- 172.21.15.60:0/2393209230 shutdown_connections 2024-04-25T10:05:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.695+0000 7f73950e8640 1 -- 172.21.15.60:0/2393209230 wait complete. 2024-04-25T10:05:32.720 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:32.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.835+0000 7fc09df2c640 1 Processor -- start 2024-04-25T10:05:32.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.835+0000 7fc09df2c640 1 -- start start 2024-04-25T10:05:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.835+0000 7fc09df2c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0980724b0 0x7fc09806ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.835+0000 7fc09df2c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc09806b150 0x7fc0980cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.835+0000 7fc09df2c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc09815c650 con 0x7fc098156040 2024-04-25T10:05:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.835+0000 7fc09df2c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc09815c930 con 0x7fc09806b150 2024-04-25T10:05:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.835+0000 7fc09df2c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09815cc10 con 0x7fc0980724b0 2024-04-25T10:05:32.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.835+0000 7fc097fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc09806b150 0x7fc0980cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.835+0000 7fc096ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0980724b0 0x7fc09806ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.835+0000 7fc0977fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc098156040 0x7fc09815be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:60342/0 (socket says 172.21.15.60:60342) 2024-04-25T10:05:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.835+0000 7fc0977fe640 1 -- 172.21.15.60:0/37234900 learned_addr learned my addr 172.21.15.60:0/37234900 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.835+0000 7fc096ffd640 1 -- 172.21.15.60:0/37234900 >> v2:172.21.15.100:3300/0 conn(0x7fc09806b150 msgr2=0x7fc0980cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.835+0000 7fc096ffd640 1 --2- 172.21.15.60:0/37234900 >> v2:172.21.15.100:3300/0 conn(0x7fc09806b150 0x7fc0980cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.835+0000 7fc096ffd640 1 -- 172.21.15.60:0/37234900 >> v1:172.21.15.100:6789/0 conn(0x7fc098156040 legacy=0x7fc09815be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.835+0000 7fc096ffd640 1 -- 172.21.15.60:0/37234900 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0980cf900 con 0x7fc0980724b0 2024-04-25T10:05:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.839+0000 7fc096ffd640 1 --2- 172.21.15.60:0/37234900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0980724b0 0x7fc09806ac10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0097c0 tx=0x7fc08c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=670b5b769917c5b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.839+0000 7fc0967fc640 1 -- 172.21.15.60:0/37234900 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0b0070 con 0x7fc0980724b0 2024-04-25T10:05:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.839+0000 7fc0967fc640 1 -- 172.21.15.60:0/37234900 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c0ab600 con 0x7fc0980724b0 2024-04-25T10:05:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.839+0000 7fc0967fc640 1 -- 172.21.15.60:0/37234900 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0ab9d0 con 0x7fc0980724b0 2024-04-25T10:05:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.839+0000 7fc09df2c640 1 -- 172.21.15.60:0/37234900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0980724b0 msgr2=0x7fc09806ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.839+0000 7fc09df2c640 1 --2- 172.21.15.60:0/37234900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0980724b0 0x7fc09806ac10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0097c0 tx=0x7fc08c01c410 comp rx=0 tx=0).stop 2024-04-25T10:05:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.839+0000 7fc09df2c640 1 -- 172.21.15.60:0/37234900 shutdown_connections 2024-04-25T10:05:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.839+0000 7fc09df2c640 1 --2- 172.21.15.60:0/37234900 >> v2:172.21.15.100:3300/0 conn(0x7fc09806b150 0x7fc0980cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.839+0000 7fc09df2c640 1 --2- 172.21.15.60:0/37234900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0980724b0 0x7fc09806ac10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.839+0000 7fc09df2c640 1 -- 172.21.15.60:0/37234900 >> 172.21.15.60:0/37234900 conn(0x7fc09806c6a0 msgr2=0x7fc09806e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.839+0000 7fc09df2c640 1 -- 172.21.15.60:0/37234900 shutdown_connections 2024-04-25T10:05:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.839+0000 7fc09df2c640 1 -- 172.21.15.60:0/37234900 wait complete. 2024-04-25T10:05:32.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.839+0000 7fc09df2c640 1 Processor -- start 2024-04-25T10:05:32.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.839+0000 7fc09df2c640 1 -- start start 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc09df2c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc09806b150 0x7fc0980b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc09df2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0980724b0 0x7fc098146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc09df2c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc098156040 0x7fc098164910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc09df2c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980d0f00 con 0x7fc09806b150 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc09df2c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980d0d80 con 0x7fc0980724b0 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc09df2c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980d1080 con 0x7fc098156040 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc0977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc09806b150 0x7fc0980b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc0977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc09806b150 0x7fc0980b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46644/0 (socket says 172.21.15.60:46644) 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc0977fe640 1 -- 172.21.15.60:0/2266339624 learned_addr learned my addr 172.21.15.60:0/2266339624 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc096ffd640 1 --2- 172.21.15.60:0/2266339624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0980724b0 0x7fc098146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc097fff640 1 --2- 172.21.15.60:0/2266339624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc098156040 0x7fc098164910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc0977fe640 1 -- 172.21.15.60:0/2266339624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc098156040 msgr2=0x7fc098164910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc0977fe640 1 --2- 172.21.15.60:0/2266339624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc098156040 0x7fc098164910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc0977fe640 1 -- 172.21.15.60:0/2266339624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0980724b0 msgr2=0x7fc098146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc0977fe640 1 --2- 172.21.15.60:0/2266339624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0980724b0 0x7fc098146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc0977fe640 1 -- 172.21.15.60:0/2266339624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0980b5730 con 0x7fc09806b150 2024-04-25T10:05:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc097fff640 1 --2- 172.21.15.60:0/2266339624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc098156040 0x7fc098164910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc0977fe640 1 --2- 172.21.15.60:0/2266339624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc09806b150 0x7fc0980b3410 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc08800caa0 tx=0x7fc08800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc07ffff640 1 -- 172.21.15.60:0/2266339624 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088017070 con 0x7fc09806b150 2024-04-25T10:05:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc07ffff640 1 -- 172.21.15.60:0/2266339624 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08800ac00 con 0x7fc09806b150 2024-04-25T10:05:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc07ffff640 1 -- 172.21.15.60:0/2266339624 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088012400 con 0x7fc09806b150 2024-04-25T10:05:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc09df2c640 1 -- 172.21.15.60:0/2266339624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0980b2370 con 0x7fc09806b150 2024-04-25T10:05:32.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc09df2c640 1 -- 172.21.15.60:0/2266339624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0980b2930 con 0x7fc09806b150 2024-04-25T10:05:32.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc09df2c640 1 -- 172.21.15.60:0/2266339624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc098074770 con 0x7fc09806b150 2024-04-25T10:05:32.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc07ffff640 1 -- 172.21.15.60:0/2266339624 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc08801a070 con 0x7fc09806b150 2024-04-25T10:05:32.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc07ffff640 1 --2- 172.21.15.60:0/2266339624 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc084066f90 0x7fc084069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc07ffff640 1 -- 172.21.15.60:0/2266339624 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc088084dc0 con 0x7fc09806b150 2024-04-25T10:05:32.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.843+0000 7fc096ffd640 1 --2- 172.21.15.60:0/2266339624 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc084066f90 0x7fc084069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.847+0000 7fc096ffd640 1 --2- 172.21.15.60:0/2266339624 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc084066f90 0x7fc084069450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0ad000 tx=0x7fc08c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:32.847+0000 7fc07ffff640 1 -- 172.21.15.60:0/2266339624 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc088055d40 con 0x7fc09806b150 2024-04-25T10:05:33.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.003+0000 7fc09df2c640 1 -- 172.21.15.60:0/2266339624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc098147490 con 0x7fc09806b150 2024-04-25T10:05:33.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.003+0000 7fc07ffff640 1 -- 172.21.15.60:0/2266339624 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc088055640 con 0x7fc09806b150 2024-04-25T10:05:33.010 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:05:33.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.003+0000 7fc09df2c640 1 -- 172.21.15.60:0/2266339624 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc084066f90 msgr2=0x7fc084069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.003+0000 7fc09df2c640 1 --2- 172.21.15.60:0/2266339624 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc084066f90 0x7fc084069450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0ad000 tx=0x7fc08c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:33.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.003+0000 7fc09df2c640 1 -- 172.21.15.60:0/2266339624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc09806b150 msgr2=0x7fc0980b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.003+0000 7fc09df2c640 1 --2- 172.21.15.60:0/2266339624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc09806b150 0x7fc0980b3410 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc08800caa0 tx=0x7fc08800cf60 comp rx=0 tx=0).stop 2024-04-25T10:05:33.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.003+0000 7fc09df2c640 1 -- 172.21.15.60:0/2266339624 shutdown_connections 2024-04-25T10:05:33.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.003+0000 7fc09df2c640 1 --2- 172.21.15.60:0/2266339624 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc084066f90 0x7fc084069450 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.003+0000 7fc09df2c640 1 --2- 172.21.15.60:0/2266339624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc098156040 0x7fc098164910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.003+0000 7fc09df2c640 1 --2- 172.21.15.60:0/2266339624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0980724b0 0x7fc098146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.003+0000 7fc09df2c640 1 --2- 172.21.15.60:0/2266339624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc09806b150 0x7fc0980b3410 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.003+0000 7fc09df2c640 1 -- 172.21.15.60:0/2266339624 >> 172.21.15.60:0/2266339624 conn(0x7fc09806c6a0 msgr2=0x7fc09806e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.003+0000 7fc09df2c640 1 -- 172.21.15.60:0/2266339624 shutdown_connections 2024-04-25T10:05:33.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.007+0000 7fc09df2c640 1 -- 172.21.15.60:0/2266339624 wait complete. 2024-04-25T10:05:33.027 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.131+0000 7f1123145640 1 Processor -- start 2024-04-25T10:05:33.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.131+0000 7f1123145640 1 -- start start 2024-04-25T10:05:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.131+0000 7f1123145640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f111c15dc10 0x7f111c1689d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.131+0000 7f1123145640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111c162070 con 0x7f111c0cea20 2024-04-25T10:05:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.131+0000 7f1123145640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111c162350 con 0x7f111c0cee00 2024-04-25T10:05:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.131+0000 7f1123145640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111c0d1cb0 con 0x7f111c15dc10 2024-04-25T10:05:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.131+0000 7f1120ee1640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f111c0cea20 0x7f111c161640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37338/0 (socket says 172.21.15.60:37338) 2024-04-25T10:05:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.131+0000 7f1120ee1640 1 -- 172.21.15.60:0/1909720585 learned_addr learned my addr 172.21.15.60:0/1909720585 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.131+0000 7f11216e2640 1 --2- 172.21.15.60:0/1909720585 >> v2:172.21.15.100:3300/0 conn(0x7f111c15dc10 0x7f111c1689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 -- 172.21.15.60:0/1909720585 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372720716 0 0) 0x7f111c162070 con 0x7f111c0cea20 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 -- 172.21.15.60:0/1909720585 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f110c003530 con 0x7f111c0cea20 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 -- 172.21.15.60:0/1909720585 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2213640438 0 0) 0x7f110c003530 con 0x7f111c0cea20 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 -- 172.21.15.60:0/1909720585 >> v2:172.21.15.100:3300/0 conn(0x7f111c15dc10 msgr2=0x7f111c1689d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 --2- 172.21.15.60:0/1909720585 >> v2:172.21.15.100:3300/0 conn(0x7f111c15dc10 0x7f111c1689d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 -- 172.21.15.60:0/1909720585 >> v1:172.21.15.100:6789/0 conn(0x7f111c0cee00 legacy=0x7f111c15d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 -- 172.21.15.60:0/1909720585 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c0d1f90 con 0x7f111c0cea20 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 -- 172.21.15.60:0/1909720585 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f1110001680 con 0x7f111c0cea20 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 -- 172.21.15.60:0/1909720585 >> v1:172.21.15.60:6789/0 conn(0x7f111c0cea20 legacy=0x7f111c161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 --2- 172.21.15.60:0/1909720585 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f110c0039c0 0x7f110c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 --2- 172.21.15.60:0/1909720585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c004810 0x7f110c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 --2- 172.21.15.60:0/1909720585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110c0051d0 0x7f110c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 -- 172.21.15.60:0/1909720585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0d1cb0 con 0x7f110c0051d0 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 -- 172.21.15.60:0/1909720585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c00c070 con 0x7f110c004810 2024-04-25T10:05:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 -- 172.21.15.60:0/1909720585 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110c00c270 con 0x7f110c0039c0 2024-04-25T10:05:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f11216e2640 1 --2- 172.21.15.60:0/1909720585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110c0051d0 0x7f110c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111bfff640 1 --2- 172.21.15.60:0/1909720585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c004810 0x7f110c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f1120ee1640 1 --2- 172.21.15.60:0/1909720585 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f110c0039c0 0x7f110c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f11216e2640 1 -- 172.21.15.60:0/1909720585 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f110c0039c0 msgr2=0x7f110c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f11216e2640 1 --2- 172.21.15.60:0/1909720585 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f110c0039c0 0x7f110c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f11216e2640 1 -- 172.21.15.60:0/1909720585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c004810 msgr2=0x7f110c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f11216e2640 1 --2- 172.21.15.60:0/1909720585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c004810 0x7f110c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f11216e2640 1 -- 172.21.15.60:0/1909720585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c003530 con 0x7f110c0051d0 2024-04-25T10:05:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f11216e2640 1 -- 172.21.15.60:0/1909720585 reap_dead start 2024-04-25T10:05:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f11216e2640 1 --2- 172.21.15.60:0/1909720585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110c0051d0 0x7f110c00b9d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1114009fc0 tx=0x7f111401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abddf6aa4365089f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 -- 172.21.15.60:0/1909720585 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114009c10 con 0x7f110c0051d0 2024-04-25T10:05:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 -- 172.21.15.60:0/1909720585 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11140a5cd0 con 0x7f110c0051d0 2024-04-25T10:05:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f111b7fe640 1 -- 172.21.15.60:0/1909720585 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114003be0 con 0x7f110c0051d0 2024-04-25T10:05:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f1123145640 1 -- 172.21.15.60:0/1909720585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110c0051d0 msgr2=0x7f110c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f1123145640 1 --2- 172.21.15.60:0/1909720585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110c0051d0 0x7f110c00b9d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1114009fc0 tx=0x7f111401ccc0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f1123145640 1 -- 172.21.15.60:0/1909720585 shutdown_connections 2024-04-25T10:05:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f1123145640 1 --2- 172.21.15.60:0/1909720585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110c0051d0 0x7f110c00b9d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f1123145640 1 -- 172.21.15.60:0/1909720585 >> 172.21.15.60:0/1909720585 conn(0x7f111c072340 msgr2=0x7f111c072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f1123145640 1 -- 172.21.15.60:0/1909720585 shutdown_connections 2024-04-25T10:05:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.135+0000 7f1123145640 1 -- 172.21.15.60:0/1909720585 wait complete. 2024-04-25T10:05:33.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f1123145640 1 Processor -- start 2024-04-25T10:05:33.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f1123145640 1 -- start start 2024-04-25T10:05:33.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f1123145640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c0051d0 0x7f111c14cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f1123145640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f111c156ed0 0x7f111c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f1123145640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f111c1603d0 0x7f111c0aa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f1123145640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c15d680 con 0x7f111c156ed0 2024-04-25T10:05:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f1123145640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c15d500 con 0x7f110c0051d0 2024-04-25T10:05:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f1123145640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c15d800 con 0x7f111c1603d0 2024-04-25T10:05:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f111bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f111c156ed0 0x7f111c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f111bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f111c156ed0 0x7f111c15ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46664/0 (socket says 172.21.15.60:46664) 2024-04-25T10:05:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f111bfff640 1 -- 172.21.15.60:0/1330911141 learned_addr learned my addr 172.21.15.60:0/1330911141 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f1120ee1640 1 --2- 172.21.15.60:0/1330911141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c0051d0 0x7f111c14cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f11216e2640 1 --2- 172.21.15.60:0/1330911141 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f111c1603d0 0x7f111c0aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f111bfff640 1 -- 172.21.15.60:0/1330911141 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f111c1603d0 msgr2=0x7f111c0aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f111bfff640 1 --2- 172.21.15.60:0/1330911141 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f111c1603d0 0x7f111c0aa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f111bfff640 1 -- 172.21.15.60:0/1330911141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c0051d0 msgr2=0x7f111c14cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f111bfff640 1 --2- 172.21.15.60:0/1330911141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c0051d0 0x7f111c14cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f111bfff640 1 -- 172.21.15.60:0/1330911141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c15f330 con 0x7f111c156ed0 2024-04-25T10:05:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f1120ee1640 1 --2- 172.21.15.60:0/1330911141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c0051d0 0x7f111c14cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f11216e2640 1 --2- 172.21.15.60:0/1330911141 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f111c1603d0 0x7f111c0aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f111bfff640 1 --2- 172.21.15.60:0/1330911141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f111c156ed0 0x7f111c15ea30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f110800e4e0 tx=0x7f110800e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f11197fa640 1 -- 172.21.15.60:0/1330911141 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11080052e0 con 0x7f111c156ed0 2024-04-25T10:05:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f11197fa640 1 -- 172.21.15.60:0/1330911141 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1108018070 con 0x7f111c156ed0 2024-04-25T10:05:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f1123145640 1 -- 172.21.15.60:0/1330911141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111c15f610 con 0x7f111c156ed0 2024-04-25T10:05:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f11197fa640 1 -- 172.21.15.60:0/1330911141 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108005480 con 0x7f111c156ed0 2024-04-25T10:05:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f1123145640 1 -- 172.21.15.60:0/1330911141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c15fbd0 con 0x7f111c156ed0 2024-04-25T10:05:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f1123145640 1 -- 172.21.15.60:0/1330911141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111c074770 con 0x7f111c156ed0 2024-04-25T10:05:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.139+0000 7f11197fa640 1 -- 172.21.15.60:0/1330911141 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1108003840 con 0x7f111c156ed0 2024-04-25T10:05:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.143+0000 7f11197fa640 1 --2- 172.21.15.60:0/1330911141 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1100066f60 0x7f1100069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.143+0000 7f11197fa640 1 -- 172.21.15.60:0/1330911141 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1108084c60 con 0x7f111c156ed0 2024-04-25T10:05:33.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.143+0000 7f1120ee1640 1 --2- 172.21.15.60:0/1330911141 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1100066f60 0x7f1100069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.143+0000 7f1120ee1640 1 --2- 172.21.15.60:0/1330911141 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1100066f60 0x7f1100069420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1110000fc0 tx=0x7f1110001180 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.147+0000 7f11197fa640 1 -- 172.21.15.60:0/1330911141 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1108055c70 con 0x7f111c156ed0 2024-04-25T10:05:33.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.331+0000 7f1123145640 1 -- 172.21.15.60:0/1330911141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f111c0d16c0 con 0x7f111c156ed0 2024-04-25T10:05:33.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.331+0000 7f11197fa640 1 -- 172.21.15.60:0/1330911141 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f1108055570 con 0x7f111c156ed0 2024-04-25T10:05:33.339 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:33.339 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/622088871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":622088871},{"type":"v1","addr":"172.21.15.100:6836","nonce":622088871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3988204202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3988204202},{"type":"v1","addr":"172.21.15.60:6833","nonce":3988204202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3591836867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3591836867},{"type":"v1","addr":"172.21.15.60:6835","nonce":3591836867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:20.405486+0000","modified":"2024-04-25T10:05:23.453258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/690070228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":690070228},{"type":"v1","addr":"172.21.15.100:6837","nonce":690070228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:05:33.340 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-04-25T10:05:33.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.335+0000 7f1123145640 1 -- 172.21.15.60:0/1330911141 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1100066f60 msgr2=0x7f1100069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.335+0000 7f1123145640 1 --2- 172.21.15.60:0/1330911141 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1100066f60 0x7f1100069420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1110000fc0 tx=0x7f1110001180 comp rx=0 tx=0).stop 2024-04-25T10:05:33.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.335+0000 7f1123145640 1 -- 172.21.15.60:0/1330911141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f111c156ed0 msgr2=0x7f111c15ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.335+0000 7f1123145640 1 --2- 172.21.15.60:0/1330911141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f111c156ed0 0x7f111c15ea30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f110800e4e0 tx=0x7f110800e9a0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.335+0000 7f1123145640 1 -- 172.21.15.60:0/1330911141 shutdown_connections 2024-04-25T10:05:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.335+0000 7f1123145640 1 --2- 172.21.15.60:0/1330911141 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1100066f60 0x7f1100069420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.335+0000 7f1123145640 1 --2- 172.21.15.60:0/1330911141 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f111c1603d0 0x7f111c0aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.335+0000 7f1123145640 1 --2- 172.21.15.60:0/1330911141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f111c156ed0 0x7f111c15ea30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.335+0000 7f1123145640 1 --2- 172.21.15.60:0/1330911141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110c0051d0 0x7f111c14cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.335+0000 7f1123145640 1 -- 172.21.15.60:0/1330911141 >> 172.21.15.60:0/1330911141 conn(0x7f111c072340 msgr2=0x7f111c072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.335+0000 7f1123145640 1 -- 172.21.15.60:0/1330911141 shutdown_connections 2024-04-25T10:05:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.335+0000 7f1123145640 1 -- 172.21.15.60:0/1330911141 wait complete. 2024-04-25T10:05:33.357 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:33.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.471+0000 7f2d70626640 1 Processor -- start 2024-04-25T10:05:33.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.471+0000 7f2d70626640 1 -- start start 2024-04-25T10:05:33.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d70626640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680ce7a0 0x7f2d680ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d70626640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2d6815bc90 0x7f2d680b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d70626640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d680cf0c0 con 0x7f2d680b4660 2024-04-25T10:05:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d70626640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d680cc370 con 0x7f2d6815bc90 2024-04-25T10:05:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d70626640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d68071ca0 con 0x7f2d680ce7a0 2024-04-25T10:05:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d6dbc1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2d6815bc90 0x7f2d680b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d6dbc1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2d6815bc90 0x7f2d680b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46676/0 (socket says 172.21.15.60:46676) 2024-04-25T10:05:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d6dbc1640 1 -- 172.21.15.60:0/2084179756 learned_addr learned my addr 172.21.15.60:0/2084179756 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:33.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d6e3c2640 1 --2- 172.21.15.60:0/2084179756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680ce7a0 0x7f2d680ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d6dbc1640 1 -- 172.21.15.60:0/2084179756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680ce7a0 msgr2=0x7f2d680ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d6dbc1640 1 --2- 172.21.15.60:0/2084179756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680ce7a0 0x7f2d680ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d6dbc1640 1 -- 172.21.15.60:0/2084179756 >> v1:172.21.15.100:6789/0 conn(0x7f2d680b4660 legacy=0x7f2d680b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d6dbc1640 1 -- 172.21.15.60:0/2084179756 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d680cca40 con 0x7f2d6815bc90 2024-04-25T10:05:33.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d6dbc1640 1 --2- 172.21.15.60:0/2084179756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d6815bc90 0x7f2d680b25d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2d580098a0 tx=0x7f2d5801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2151d4fc81d2cfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d6d3c0640 1 -- 172.21.15.60:0/2084179756 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d580b0070 con 0x7f2d6815bc90 2024-04-25T10:05:33.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d6d3c0640 1 -- 172.21.15.60:0/2084179756 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d580ab600 con 0x7f2d6815bc90 2024-04-25T10:05:33.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d6d3c0640 1 -- 172.21.15.60:0/2084179756 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d580ab9d0 con 0x7f2d6815bc90 2024-04-25T10:05:33.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d70626640 1 -- 172.21.15.60:0/2084179756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d6815bc90 msgr2=0x7f2d680b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d70626640 1 --2- 172.21.15.60:0/2084179756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d6815bc90 0x7f2d680b25d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2d580098a0 tx=0x7f2d5801c500 comp rx=0 tx=0).stop 2024-04-25T10:05:33.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d70626640 1 -- 172.21.15.60:0/2084179756 shutdown_connections 2024-04-25T10:05:33.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d70626640 1 --2- 172.21.15.60:0/2084179756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d6815bc90 0x7f2d680b25d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d70626640 1 --2- 172.21.15.60:0/2084179756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680ce7a0 0x7f2d680ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.475+0000 7f2d70626640 1 -- 172.21.15.60:0/2084179756 >> 172.21.15.60:0/2084179756 conn(0x7f2d680b12d0 msgr2=0x7f2d680b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d70626640 1 -- 172.21.15.60:0/2084179756 shutdown_connections 2024-04-25T10:05:33.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d70626640 1 -- 172.21.15.60:0/2084179756 wait complete. 2024-04-25T10:05:33.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d70626640 1 Processor -- start 2024-04-25T10:05:33.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d70626640 1 -- start start 2024-04-25T10:05:33.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d70626640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680ce7a0 0x7f2d68156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d70626640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d6815bc90 0x7f2d68159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d70626640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d68157c80 0x7f2d6815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d70626640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680b4280 con 0x7f2d68157c80 2024-04-25T10:05:33.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d70626640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680b3db0 con 0x7f2d6815bc90 2024-04-25T10:05:33.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d70626640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680b4400 con 0x7f2d680ce7a0 2024-04-25T10:05:33.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d6ebc3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d68157c80 0x7f2d6815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d6e3c2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680ce7a0 0x7f2d68156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d6ebc3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d68157c80 0x7f2d6815a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46686/0 (socket says 172.21.15.60:46686) 2024-04-25T10:05:33.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d6ebc3640 1 -- 172.21.15.60:0/1574292767 learned_addr learned my addr 172.21.15.60:0/1574292767 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:33.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d6dbc1640 1 --2- 172.21.15.60:0/1574292767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d6815bc90 0x7f2d68159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d6ebc3640 1 -- 172.21.15.60:0/1574292767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680ce7a0 msgr2=0x7f2d68156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d6ebc3640 1 --2- 172.21.15.60:0/1574292767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680ce7a0 0x7f2d68156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d6ebc3640 1 -- 172.21.15.60:0/1574292767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d6815bc90 msgr2=0x7f2d68159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d6ebc3640 1 --2- 172.21.15.60:0/1574292767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d6815bc90 0x7f2d68159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d6ebc3640 1 -- 172.21.15.60:0/1574292767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d681581d0 con 0x7f2d68157c80 2024-04-25T10:05:33.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d6ebc3640 1 --2- 172.21.15.60:0/1574292767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d68157c80 0x7f2d6815a500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000be90 tx=0x7f2d6000bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d5effd640 1 -- 172.21.15.60:0/1574292767 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60019070 con 0x7f2d68157c80 2024-04-25T10:05:33.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d5effd640 1 -- 172.21.15.60:0/1574292767 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d60014440 con 0x7f2d68157c80 2024-04-25T10:05:33.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d70626640 1 -- 172.21.15.60:0/1574292767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d681584b0 con 0x7f2d68157c80 2024-04-25T10:05:33.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d5effd640 1 -- 172.21.15.60:0/1574292767 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60014830 con 0x7f2d68157c80 2024-04-25T10:05:33.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.479+0000 7f2d70626640 1 -- 172.21.15.60:0/1574292767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d68158a70 con 0x7f2d68157c80 2024-04-25T10:05:33.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.483+0000 7f2d70626640 1 -- 172.21.15.60:0/1574292767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d68074770 con 0x7f2d68157c80 2024-04-25T10:05:33.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.483+0000 7f2d5effd640 1 -- 172.21.15.60:0/1574292767 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d6001c070 con 0x7f2d68157c80 2024-04-25T10:05:33.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.483+0000 7f2d5effd640 1 --2- 172.21.15.60:0/1574292767 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d4c066f90 0x7f2d4c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.483+0000 7f2d5effd640 1 -- 172.21.15.60:0/1574292767 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d60084f20 con 0x7f2d68157c80 2024-04-25T10:05:33.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.483+0000 7f2d6e3c2640 1 --2- 172.21.15.60:0/1574292767 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d4c066f90 0x7f2d4c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.483+0000 7f2d6e3c2640 1 --2- 172.21.15.60:0/1574292767 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d4c066f90 0x7f2d4c069450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d640079c0 tx=0x7f2d64008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.487+0000 7f2d5effd640 1 -- 172.21.15.60:0/1574292767 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d600558d0 con 0x7f2d68157c80 2024-04-25T10:05:33.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.643+0000 7f2d70626640 1 -- 172.21.15.60:0/1574292767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d680ac8a0 con 0x7f2d68157c80 2024-04-25T10:05:33.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.647+0000 7f2d5effd640 1 -- 172.21.15.60:0/1574292767 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f2d60059970 con 0x7f2d68157c80 2024-04-25T10:05:33.655 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:33.655 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:22.036582+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:33.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.651+0000 7f2d70626640 1 -- 172.21.15.60:0/1574292767 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d4c066f90 msgr2=0x7f2d4c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.651+0000 7f2d70626640 1 --2- 172.21.15.60:0/1574292767 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d4c066f90 0x7f2d4c069450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d640079c0 tx=0x7f2d64008040 comp rx=0 tx=0).stop 2024-04-25T10:05:33.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.651+0000 7f2d70626640 1 -- 172.21.15.60:0/1574292767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d68157c80 msgr2=0x7f2d6815a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.651+0000 7f2d70626640 1 --2- 172.21.15.60:0/1574292767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d68157c80 0x7f2d6815a500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000be90 tx=0x7f2d6000bf80 comp rx=0 tx=0).stop 2024-04-25T10:05:33.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.651+0000 7f2d70626640 1 -- 172.21.15.60:0/1574292767 shutdown_connections 2024-04-25T10:05:33.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.651+0000 7f2d70626640 1 --2- 172.21.15.60:0/1574292767 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d4c066f90 0x7f2d4c069450 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.651+0000 7f2d70626640 1 --2- 172.21.15.60:0/1574292767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d68157c80 0x7f2d6815a500 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.651+0000 7f2d70626640 1 --2- 172.21.15.60:0/1574292767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d6815bc90 0x7f2d68159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.651+0000 7f2d70626640 1 --2- 172.21.15.60:0/1574292767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680ce7a0 0x7f2d68156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.651+0000 7f2d70626640 1 -- 172.21.15.60:0/1574292767 >> 172.21.15.60:0/1574292767 conn(0x7f2d680b12d0 msgr2=0x7f2d6815d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.651+0000 7f2d70626640 1 -- 172.21.15.60:0/1574292767 shutdown_connections 2024-04-25T10:05:33.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.651+0000 7f2d70626640 1 -- 172.21.15.60:0/1574292767 wait complete. 2024-04-25T10:05:33.673 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:18.976643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:19.405302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:33.674 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:33.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.783+0000 7f1bd239e640 1 Processor -- start 2024-04-25T10:05:33.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.783+0000 7f1bd239e640 1 -- start start 2024-04-25T10:05:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.787+0000 7f1bd239e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1bcc0b2560 0x7f1bcc0b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.787+0000 7f1bd239e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1bcc099a30 0x7f1bcc099ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.787+0000 7f1bd239e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bcc0cc320 con 0x7f1bcc15ef20 2024-04-25T10:05:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.787+0000 7f1bd239e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bcc0cc5c0 con 0x7f1bcc099a30 2024-04-25T10:05:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.787+0000 7f1bd239e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bcc0d07b0 con 0x7f1bcc0b2560 2024-04-25T10:05:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.787+0000 7f1bd093b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1bcc099a30 0x7f1bcc099ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.787+0000 7f1bd093b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1bcc099a30 0x7f1bcc099ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46698/0 (socket says 172.21.15.60:46698) 2024-04-25T10:05:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.787+0000 7f1bd093b640 1 -- 172.21.15.60:0/1840558689 learned_addr learned my addr 172.21.15.60:0/1840558689 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.787+0000 7f1bcbfff640 1 --1- 172.21.15.60:0/1840558689 >> v1:172.21.15.60:6789/0 conn(0x7f1bcc15ef20 0x7f1bcc0cfcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37354/0 (socket says 172.21.15.60:37354) 2024-04-25T10:05:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.787+0000 7f1bcb7fe640 1 --2- 172.21.15.60:0/1840558689 >> v2:172.21.15.100:3300/0 conn(0x7f1bcc0b2560 0x7f1bcc0b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.787+0000 7f1bd093b640 1 -- 172.21.15.60:0/1840558689 >> v2:172.21.15.100:3300/0 conn(0x7f1bcc0b2560 msgr2=0x7f1bcc0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.787+0000 7f1bd093b640 1 --2- 172.21.15.60:0/1840558689 >> v2:172.21.15.100:3300/0 conn(0x7f1bcc0b2560 0x7f1bcc0b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.787+0000 7f1bd093b640 1 -- 172.21.15.60:0/1840558689 >> v1:172.21.15.60:6789/0 conn(0x7f1bcc15ef20 legacy=0x7f1bcc0cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.787+0000 7f1bd093b640 1 -- 172.21.15.60:0/1840558689 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc0d04d0 con 0x7f1bcc099a30 2024-04-25T10:05:33.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.787+0000 7f1bcaffd640 1 -- 172.21.15.60:0/1840558689 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555290476 0 0) 0x7f1bcc0cc320 con 0x7f1bcc15ef20 2024-04-25T10:05:33.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.787+0000 7f1bd093b640 1 --2- 172.21.15.60:0/1840558689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc099a30 0x7f1bcc099ec0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc071ea0 tx=0x7f1bc401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9588d6b53f6f55e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bcaffd640 1 -- 172.21.15.60:0/1840558689 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc40b0070 con 0x7f1bcc099a30 2024-04-25T10:05:33.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bcaffd640 1 -- 172.21.15.60:0/1840558689 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc40ab600 con 0x7f1bcc099a30 2024-04-25T10:05:33.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bcaffd640 1 -- 172.21.15.60:0/1840558689 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc40ab9d0 con 0x7f1bcc099a30 2024-04-25T10:05:33.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 -- 172.21.15.60:0/1840558689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc099a30 msgr2=0x7f1bcc099ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 --2- 172.21.15.60:0/1840558689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc099a30 0x7f1bcc099ec0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc071ea0 tx=0x7f1bc401c520 comp rx=0 tx=0).stop 2024-04-25T10:05:33.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 -- 172.21.15.60:0/1840558689 shutdown_connections 2024-04-25T10:05:33.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 --2- 172.21.15.60:0/1840558689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc099a30 0x7f1bcc099ec0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 --2- 172.21.15.60:0/1840558689 >> v2:172.21.15.100:3300/0 conn(0x7f1bcc0b2560 0x7f1bcc0b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 -- 172.21.15.60:0/1840558689 >> 172.21.15.60:0/1840558689 conn(0x7f1bcc0b3200 msgr2=0x7f1bcc0b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 -- 172.21.15.60:0/1840558689 shutdown_connections 2024-04-25T10:05:33.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 -- 172.21.15.60:0/1840558689 wait complete. 2024-04-25T10:05:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 Processor -- start 2024-04-25T10:05:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 -- start start 2024-04-25T10:05:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc099a30 0x7f1bcc0cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc0b2560 0x7f1bcc15df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bcc15ef20 0x7f1bcc06faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc15f300 con 0x7f1bcc0b2560 2024-04-25T10:05:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc0b2940 con 0x7f1bcc099a30 2024-04-25T10:05:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc15f480 con 0x7f1bcc15ef20 2024-04-25T10:05:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bcb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc0b2560 0x7f1bcc15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bcb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc0b2560 0x7f1bcc15df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46712/0 (socket says 172.21.15.60:46712) 2024-04-25T10:05:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bcb7fe640 1 -- 172.21.15.60:0/1379196567 learned_addr learned my addr 172.21.15.60:0/1379196567 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bcbfff640 1 --2- 172.21.15.60:0/1379196567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc099a30 0x7f1bcc0cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bcb7fe640 1 -- 172.21.15.60:0/1379196567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bcc15ef20 msgr2=0x7f1bcc06faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd093b640 1 --2- 172.21.15.60:0/1379196567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bcc15ef20 0x7f1bcc06faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bcb7fe640 1 --2- 172.21.15.60:0/1379196567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bcc15ef20 0x7f1bcc06faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bcb7fe640 1 -- 172.21.15.60:0/1379196567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc099a30 msgr2=0x7f1bcc0cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bcb7fe640 1 --2- 172.21.15.60:0/1379196567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc099a30 0x7f1bcc0cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bcb7fe640 1 -- 172.21.15.60:0/1379196567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc151470 con 0x7f1bcc0b2560 2024-04-25T10:05:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bcbfff640 1 --2- 172.21.15.60:0/1379196567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc099a30 0x7f1bcc0cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bcb7fe640 1 --2- 172.21.15.60:0/1379196567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc0b2560 0x7f1bcc15df30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000d830 tx=0x7f1bc000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bc8ff9640 1 -- 172.21.15.60:0/1379196567 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc000b6c0 con 0x7f1bcc0b2560 2024-04-25T10:05:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bc8ff9640 1 -- 172.21.15.60:0/1379196567 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc0019070 con 0x7f1bcc0b2560 2024-04-25T10:05:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 -- 172.21.15.60:0/1379196567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc151750 con 0x7f1bcc0b2560 2024-04-25T10:05:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bc8ff9640 1 -- 172.21.15.60:0/1379196567 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc000ba60 con 0x7f1bcc0b2560 2024-04-25T10:05:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.791+0000 7f1bd239e640 1 -- 172.21.15.60:0/1379196567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bcc151d10 con 0x7f1bcc0b2560 2024-04-25T10:05:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.795+0000 7f1bd239e640 1 -- 172.21.15.60:0/1379196567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bcc074770 con 0x7f1bcc0b2560 2024-04-25T10:05:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.795+0000 7f1bc8ff9640 1 -- 172.21.15.60:0/1379196567 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1bc000bc00 con 0x7f1bcc0b2560 2024-04-25T10:05:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.795+0000 7f1bc8ff9640 1 --2- 172.21.15.60:0/1379196567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1bb8066f60 0x7f1bb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.795+0000 7f1bc8ff9640 1 -- 172.21.15.60:0/1379196567 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bc0085480 con 0x7f1bcc0b2560 2024-04-25T10:05:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.795+0000 7f1bcbfff640 1 --2- 172.21.15.60:0/1379196567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1bb8066f60 0x7f1bb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.795+0000 7f1bcbfff640 1 --2- 172.21.15.60:0/1379196567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1bb8066f60 0x7f1bb8069420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc007fd0 tx=0x7f1bbc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.799+0000 7f1bc8ff9640 1 -- 172.21.15.60:0/1379196567 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1bc0056520 con 0x7f1bcc0b2560 2024-04-25T10:05:33.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.951+0000 7f1bd239e640 1 -- 172.21.15.60:0/1379196567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1bcc0b4070 con 0x7f1bcc0b2560 2024-04-25T10:05:33.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.955+0000 7f1bc8ff9640 1 -- 172.21.15.60:0/1379196567 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1bc0055e20 con 0x7f1bcc0b2560 2024-04-25T10:05:33.961 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:05:33.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.955+0000 7f1bd239e640 1 -- 172.21.15.60:0/1379196567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1bb8066f60 msgr2=0x7f1bb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.955+0000 7f1bd239e640 1 --2- 172.21.15.60:0/1379196567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1bb8066f60 0x7f1bb8069420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc007fd0 tx=0x7f1bbc009040 comp rx=0 tx=0).stop 2024-04-25T10:05:33.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.955+0000 7f1bd239e640 1 -- 172.21.15.60:0/1379196567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc0b2560 msgr2=0x7f1bcc15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.955+0000 7f1bd239e640 1 --2- 172.21.15.60:0/1379196567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc0b2560 0x7f1bcc15df30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000d830 tx=0x7f1bc000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.955+0000 7f1bd239e640 1 -- 172.21.15.60:0/1379196567 shutdown_connections 2024-04-25T10:05:33.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.955+0000 7f1bd239e640 1 --2- 172.21.15.60:0/1379196567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1bb8066f60 0x7f1bb8069420 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.955+0000 7f1bd239e640 1 --2- 172.21.15.60:0/1379196567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bcc15ef20 0x7f1bcc06faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.955+0000 7f1bd239e640 1 --2- 172.21.15.60:0/1379196567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc0b2560 0x7f1bcc15df30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.955+0000 7f1bd239e640 1 --2- 172.21.15.60:0/1379196567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc099a30 0x7f1bcc0cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.955+0000 7f1bd239e640 1 -- 172.21.15.60:0/1379196567 >> 172.21.15.60:0/1379196567 conn(0x7f1bcc0b3200 msgr2=0x7f1bcc154590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.955+0000 7f1bd239e640 1 -- 172.21.15.60:0/1379196567 shutdown_connections 2024-04-25T10:05:33.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:33.955+0000 7f1bd239e640 1 -- 172.21.15.60:0/1379196567 wait complete. 2024-04-25T10:05:33.978 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:34.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.099+0000 7efcac400640 1 Processor -- start 2024-04-25T10:05:34.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.099+0000 7efcac400640 1 -- start start 2024-04-25T10:05:34.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcac400640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efca40cbcf0 0x7efca4160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcac400640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efca4071c30 0x7efca415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcac400640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca4144a40 con 0x7efca415fa20 2024-04-25T10:05:34.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcac400640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca40b2dc0 con 0x7efca40cbcf0 2024-04-25T10:05:34.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcac400640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca40cc9f0 con 0x7efca4071c30 2024-04-25T10:05:34.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcaa19c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efca40cbcf0 0x7efca4160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcaa19c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efca40cbcf0 0x7efca4160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46724/0 (socket says 172.21.15.60:46724) 2024-04-25T10:05:34.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcaa19c640 1 -- 172.21.15.60:0/3449039794 learned_addr learned my addr 172.21.15.60:0/3449039794 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:34.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efca999b640 1 --2- 172.21.15.60:0/3449039794 >> v2:172.21.15.100:3300/0 conn(0x7efca4071c30 0x7efca415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcaa19c640 1 -- 172.21.15.60:0/3449039794 >> v2:172.21.15.100:3300/0 conn(0x7efca4071c30 msgr2=0x7efca415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:34.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcaa19c640 1 --2- 172.21.15.60:0/3449039794 >> v2:172.21.15.100:3300/0 conn(0x7efca4071c30 0x7efca415f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcaa19c640 1 -- 172.21.15.60:0/3449039794 >> v1:172.21.15.100:6789/0 conn(0x7efca415fa20 legacy=0x7efca415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcaa19c640 1 -- 172.21.15.60:0/3449039794 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca40cccd0 con 0x7efca40cbcf0 2024-04-25T10:05:34.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcaa19c640 1 --2- 172.21.15.60:0/3449039794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca40cbcf0 0x7efca4160080 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efca40d0570 tx=0x7efca001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45a15739b0703a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efca919a640 1 -- 172.21.15.60:0/3449039794 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca00ab610 con 0x7efca40cbcf0 2024-04-25T10:05:34.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efca919a640 1 -- 172.21.15.60:0/3449039794 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca00b0070 con 0x7efca40cbcf0 2024-04-25T10:05:34.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efca919a640 1 -- 172.21.15.60:0/3449039794 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca00ab9d0 con 0x7efca40cbcf0 2024-04-25T10:05:34.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcac400640 1 -- 172.21.15.60:0/3449039794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca40cbcf0 msgr2=0x7efca4160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcac400640 1 --2- 172.21.15.60:0/3449039794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca40cbcf0 0x7efca4160080 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efca40d0570 tx=0x7efca001c520 comp rx=0 tx=0).stop 2024-04-25T10:05:34.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcac400640 1 -- 172.21.15.60:0/3449039794 shutdown_connections 2024-04-25T10:05:34.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcac400640 1 --2- 172.21.15.60:0/3449039794 >> v2:172.21.15.100:3300/0 conn(0x7efca4071c30 0x7efca415f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcac400640 1 --2- 172.21.15.60:0/3449039794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca40cbcf0 0x7efca4160080 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcac400640 1 -- 172.21.15.60:0/3449039794 >> 172.21.15.60:0/3449039794 conn(0x7efca40670b0 msgr2=0x7efca406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcac400640 1 -- 172.21.15.60:0/3449039794 shutdown_connections 2024-04-25T10:05:34.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.103+0000 7efcac400640 1 -- 172.21.15.60:0/3449039794 wait complete. 2024-04-25T10:05:34.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcac400640 1 Processor -- start 2024-04-25T10:05:34.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcac400640 1 -- start start 2024-04-25T10:05:34.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcac400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca4071c30 0x7efca415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcac400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efca40cbcf0 0x7efca41434d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcac400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efca415fa20 0x7efca4143a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcac400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca40cfed0 con 0x7efca4071c30 2024-04-25T10:05:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcac400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca40ce5c0 con 0x7efca40cbcf0 2024-04-25T10:05:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcac400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca4138ea0 con 0x7efca415fa20 2024-04-25T10:05:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcaa19c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca4071c30 0x7efca415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcaa19c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca4071c30 0x7efca415adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46730/0 (socket says 172.21.15.60:46730) 2024-04-25T10:05:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcaa19c640 1 -- 172.21.15.60:0/3311516567 learned_addr learned my addr 172.21.15.60:0/3311516567 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efca999b640 1 --2- 172.21.15.60:0/3311516567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efca40cbcf0 0x7efca41434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcaa99d640 1 --2- 172.21.15.60:0/3311516567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efca415fa20 0x7efca4143a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcaa19c640 1 -- 172.21.15.60:0/3311516567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efca415fa20 msgr2=0x7efca4143a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcaa19c640 1 --2- 172.21.15.60:0/3311516567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efca415fa20 0x7efca4143a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcaa19c640 1 -- 172.21.15.60:0/3311516567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efca40cbcf0 msgr2=0x7efca41434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcaa19c640 1 --2- 172.21.15.60:0/3311516567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efca40cbcf0 0x7efca41434d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcaa19c640 1 -- 172.21.15.60:0/3311516567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca4138290 con 0x7efca4071c30 2024-04-25T10:05:34.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcaa19c640 1 --2- 172.21.15.60:0/3311516567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca4071c30 0x7efca415adc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efca001abd0 tx=0x7efca001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efc9affd640 1 -- 172.21.15.60:0/3311516567 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca00be5f0 con 0x7efca4071c30 2024-04-25T10:05:34.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efc9affd640 1 -- 172.21.15.60:0/3311516567 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca00be790 con 0x7efca4071c30 2024-04-25T10:05:34.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efc9affd640 1 -- 172.21.15.60:0/3311516567 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca00beb40 con 0x7efca4071c30 2024-04-25T10:05:34.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcac400640 1 -- 172.21.15.60:0/3311516567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca4138570 con 0x7efca4071c30 2024-04-25T10:05:34.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.107+0000 7efcac400640 1 -- 172.21.15.60:0/3311516567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca4138b30 con 0x7efca4071c30 2024-04-25T10:05:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.111+0000 7efcac400640 1 -- 172.21.15.60:0/3311516567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca406bba0 con 0x7efca4071c30 2024-04-25T10:05:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.111+0000 7efc9affd640 1 -- 172.21.15.60:0/3311516567 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efca00b43f0 con 0x7efca4071c30 2024-04-25T10:05:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.111+0000 7efc9affd640 1 --2- 172.21.15.60:0/3311516567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc88066f60 0x7efc88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.111+0000 7efc9affd640 1 -- 172.21.15.60:0/3311516567 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efca011f330 con 0x7efca4071c30 2024-04-25T10:05:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.111+0000 7efca999b640 1 --2- 172.21.15.60:0/3311516567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc88066f60 0x7efc88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.111+0000 7efca999b640 1 --2- 172.21.15.60:0/3311516567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc88066f60 0x7efc88069420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efca40ce070 tx=0x7efc94008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.115+0000 7efc9affd640 1 -- 172.21.15.60:0/3311516567 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efca00a8050 con 0x7efca4071c30 2024-04-25T10:05:34.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.275+0000 7efcac400640 1 -- 172.21.15.60:0/3311516567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efca4074770 con 0x7efca4071c30 2024-04-25T10:05:34.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.275+0000 7efc9affd640 1 -- 172.21.15.60:0/3311516567 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7efca0016110 con 0x7efca4071c30 2024-04-25T10:05:34.284 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:05:34.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.279+0000 7efcac400640 1 -- 172.21.15.60:0/3311516567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc88066f60 msgr2=0x7efc88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.279+0000 7efcac400640 1 --2- 172.21.15.60:0/3311516567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc88066f60 0x7efc88069420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efca40ce070 tx=0x7efc94008040 comp rx=0 tx=0).stop 2024-04-25T10:05:34.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.279+0000 7efcac400640 1 -- 172.21.15.60:0/3311516567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca4071c30 msgr2=0x7efca415adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.279+0000 7efcac400640 1 --2- 172.21.15.60:0/3311516567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca4071c30 0x7efca415adc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efca001abd0 tx=0x7efca001ad60 comp rx=0 tx=0).stop 2024-04-25T10:05:34.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.279+0000 7efcac400640 1 -- 172.21.15.60:0/3311516567 shutdown_connections 2024-04-25T10:05:34.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.279+0000 7efcac400640 1 --2- 172.21.15.60:0/3311516567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc88066f60 0x7efc88069420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.279+0000 7efcac400640 1 --2- 172.21.15.60:0/3311516567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efca415fa20 0x7efca4143a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.279+0000 7efcac400640 1 --2- 172.21.15.60:0/3311516567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efca40cbcf0 0x7efca41434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.279+0000 7efcac400640 1 --2- 172.21.15.60:0/3311516567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efca4071c30 0x7efca415adc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.279+0000 7efcac400640 1 -- 172.21.15.60:0/3311516567 >> 172.21.15.60:0/3311516567 conn(0x7efca40670b0 msgr2=0x7efca406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.279+0000 7efcac400640 1 -- 172.21.15.60:0/3311516567 shutdown_connections 2024-04-25T10:05:34.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.279+0000 7efcac400640 1 -- 172.21.15.60:0/3311516567 wait complete. 2024-04-25T10:05:34.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f48a0634640 1 Processor -- start 2024-04-25T10:05:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f48a0634640 1 -- start start 2024-04-25T10:05:34.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f48a0634640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4898145320 0x7f4898070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f48a0634640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f48980cf140 0x7f48980b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f48a0634640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4898145700 con 0x7f4898071210 2024-04-25T10:05:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f48a0634640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4898071670 con 0x7f48980cf140 2024-04-25T10:05:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f48a0634640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48980b4800 con 0x7f4898145320 2024-04-25T10:05:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489dbcf640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4898071210 0x7f48980cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37356/0 (socket says 172.21.15.60:37356) 2024-04-25T10:05:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489dbcf640 1 -- 172.21.15.60:0/2523716639 learned_addr learned my addr 172.21.15.60:0/2523716639 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489ebd1640 1 --2- 172.21.15.60:0/2523716639 >> v2:172.21.15.100:3300/0 conn(0x7f48980cf140 0x7f48980b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489e3d0640 1 --2- 172.21.15.60:0/2523716639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4898145320 0x7f4898070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489d3ce640 1 -- 172.21.15.60:0/2523716639 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074181582 0 0) 0x7f4898145700 con 0x7f4898071210 2024-04-25T10:05:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489d3ce640 1 -- 172.21.15.60:0/2523716639 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4884003530 con 0x7f4898071210 2024-04-25T10:05:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489d3ce640 1 -- 172.21.15.60:0/2523716639 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3623625514 0 0) 0x7f4884003530 con 0x7f4898071210 2024-04-25T10:05:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489d3ce640 1 -- 172.21.15.60:0/2523716639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4898145320 msgr2=0x7f4898070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489d3ce640 1 --2- 172.21.15.60:0/2523716639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4898145320 0x7f4898070cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489d3ce640 1 -- 172.21.15.60:0/2523716639 >> v2:172.21.15.100:3300/0 conn(0x7f48980cf140 msgr2=0x7f48980b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489d3ce640 1 --2- 172.21.15.60:0/2523716639 >> v2:172.21.15.100:3300/0 conn(0x7f48980cf140 0x7f48980b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489d3ce640 1 -- 172.21.15.60:0/2523716639 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48980d0bd0 con 0x7f4898071210 2024-04-25T10:05:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489d3ce640 1 -- 172.21.15.60:0/2523716639 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f4888001680 con 0x7f4898071210 2024-04-25T10:05:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489e3d0640 1 --2- 172.21.15.60:0/2523716639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4898145320 0x7f4898070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489d3ce640 1 -- 172.21.15.60:0/2523716639 >> v1:172.21.15.60:6789/0 conn(0x7f4898071210 legacy=0x7f48980cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489d3ce640 1 --2- 172.21.15.60:0/2523716639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48840039c0 0x7f4884003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489d3ce640 1 --2- 172.21.15.60:0/2523716639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4884004810 0x7f4884004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489d3ce640 1 --2- 172.21.15.60:0/2523716639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4898145320 0x7f48840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489d3ce640 1 -- 172.21.15.60:0/2523716639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48980b4800 con 0x7f48840039c0 2024-04-25T10:05:34.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489d3ce640 1 -- 172.21.15.60:0/2523716639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488400b9d0 con 0x7f4884004810 2024-04-25T10:05:34.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489d3ce640 1 -- 172.21.15.60:0/2523716639 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488400bbd0 con 0x7f4898145320 2024-04-25T10:05:34.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489e3d0640 1 --2- 172.21.15.60:0/2523716639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48840039c0 0x7f4884003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489dbcf640 1 --2- 172.21.15.60:0/2523716639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4884004810 0x7f4884004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489e3d0640 1 -- 172.21.15.60:0/2523716639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4898145320 msgr2=0x7f48840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:34.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489e3d0640 1 --2- 172.21.15.60:0/2523716639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4898145320 0x7f48840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489e3d0640 1 -- 172.21.15.60:0/2523716639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4884004810 msgr2=0x7f4884004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:34.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489e3d0640 1 --2- 172.21.15.60:0/2523716639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4884004810 0x7f4884004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.423+0000 7f489e3d0640 1 -- 172.21.15.60:0/2523716639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4884003530 con 0x7f48840039c0 2024-04-25T10:05:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.427+0000 7f489e3d0640 1 --2- 172.21.15.60:0/2523716639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48840039c0 0x7f4884003de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f48940098d0 tx=0x7f489401c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409ef325845fc239 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.427+0000 7f489d3ce640 1 -- 172.21.15.60:0/2523716639 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489400f040 con 0x7f48840039c0 2024-04-25T10:05:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.427+0000 7f489d3ce640 1 -- 172.21.15.60:0/2523716639 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48940b0070 con 0x7f48840039c0 2024-04-25T10:05:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.427+0000 7f489d3ce640 1 -- 172.21.15.60:0/2523716639 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48940ab840 con 0x7f48840039c0 2024-04-25T10:05:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.427+0000 7f48a0634640 1 -- 172.21.15.60:0/2523716639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48840039c0 msgr2=0x7f4884003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.427+0000 7f48a0634640 1 --2- 172.21.15.60:0/2523716639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48840039c0 0x7f4884003de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f48940098d0 tx=0x7f489401c5e0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.427+0000 7f489ebd1640 1 -- 172.21.15.60:0/2523716639 reap_dead start 2024-04-25T10:05:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.427+0000 7f48a0634640 1 -- 172.21.15.60:0/2523716639 shutdown_connections 2024-04-25T10:05:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.427+0000 7f48a0634640 1 -- 172.21.15.60:0/2523716639 >> 172.21.15.60:0/2523716639 conn(0x7f489806f530 msgr2=0x7f489806f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.427+0000 7f48a0634640 1 -- 172.21.15.60:0/2523716639 shutdown_connections 2024-04-25T10:05:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.427+0000 7f48a0634640 1 -- 172.21.15.60:0/2523716639 wait complete. 2024-04-25T10:05:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.427+0000 7f48a0634640 1 Processor -- start 2024-04-25T10:05:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f48a0634640 1 -- start start 2024-04-25T10:05:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f48a0634640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48840039c0 0x7f489815a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f48a0634640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48980cb970 0x7f48980cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f48a0634640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4898159270 0x7f48981596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f48a0634640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48980d0970 con 0x7f4898159270 2024-04-25T10:05:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f48a0634640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48980d07f0 con 0x7f48840039c0 2024-04-25T10:05:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f48a0634640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48980b59e0 con 0x7f48980cb970 2024-04-25T10:05:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f489ebd1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4898159270 0x7f48981596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f489ebd1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4898159270 0x7f48981596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46752/0 (socket says 172.21.15.60:46752) 2024-04-25T10:05:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f489ebd1640 1 -- 172.21.15.60:0/157405179 learned_addr learned my addr 172.21.15.60:0/157405179 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f489dbcf640 1 --2- 172.21.15.60:0/157405179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48980cb970 0x7f48980cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f489e3d0640 1 --2- 172.21.15.60:0/157405179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48840039c0 0x7f489815a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f489ebd1640 1 -- 172.21.15.60:0/157405179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48980cb970 msgr2=0x7f48980cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f489ebd1640 1 --2- 172.21.15.60:0/157405179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48980cb970 0x7f48980cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f489ebd1640 1 -- 172.21.15.60:0/157405179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48840039c0 msgr2=0x7f489815a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f489ebd1640 1 --2- 172.21.15.60:0/157405179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48840039c0 0x7f489815a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f489ebd1640 1 -- 172.21.15.60:0/157405179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4898159c30 con 0x7f4898159270 2024-04-25T10:05:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f489ebd1640 1 --2- 172.21.15.60:0/157405179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4898159270 0x7f48981596f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f48980cfcc0 tx=0x7f489000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f488effd640 1 -- 172.21.15.60:0/157405179 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4890018020 con 0x7f4898159270 2024-04-25T10:05:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f488effd640 1 -- 172.21.15.60:0/157405179 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489000bce0 con 0x7f4898159270 2024-04-25T10:05:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f48a0634640 1 -- 172.21.15.60:0/157405179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48980cc3f0 con 0x7f4898159270 2024-04-25T10:05:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f488effd640 1 -- 172.21.15.60:0/157405179 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4890014640 con 0x7f4898159270 2024-04-25T10:05:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f48a0634640 1 -- 172.21.15.60:0/157405179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489806c130 con 0x7f4898159270 2024-04-25T10:05:34.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f48a0634640 1 -- 172.21.15.60:0/157405179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4898074770 con 0x7f4898159270 2024-04-25T10:05:34.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f488effd640 1 -- 172.21.15.60:0/157405179 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f489001b020 con 0x7f4898159270 2024-04-25T10:05:34.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f488effd640 1 --2- 172.21.15.60:0/157405179 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f487c066f60 0x7f487c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f488effd640 1 -- 172.21.15.60:0/157405179 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4890085650 con 0x7f4898159270 2024-04-25T10:05:34.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.431+0000 7f489e3d0640 1 --2- 172.21.15.60:0/157405179 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f487c066f60 0x7f487c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.435+0000 7f489e3d0640 1 --2- 172.21.15.60:0/157405179 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f487c066f60 0x7f487c069420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f48940af000 tx=0x7f48940046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.435+0000 7f488effd640 1 -- 172.21.15.60:0/157405179 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4890022030 con 0x7f4898159270 2024-04-25T10:05:34.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.595+0000 7f48a0634640 1 -- 172.21.15.60:0/157405179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f48980b2d20 con 0x7f4898159270 2024-04-25T10:05:34.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.595+0000 7f488effd640 1 -- 172.21.15.60:0/157405179 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4898159c30 con 0x7f4898159270 2024-04-25T10:05:34.602 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:05:34.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.599+0000 7f48a0634640 1 -- 172.21.15.60:0/157405179 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f487c066f60 msgr2=0x7f487c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.599+0000 7f48a0634640 1 --2- 172.21.15.60:0/157405179 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f487c066f60 0x7f487c069420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f48940af000 tx=0x7f48940046d0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.599+0000 7f48a0634640 1 -- 172.21.15.60:0/157405179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4898159270 msgr2=0x7f48981596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.599+0000 7f48a0634640 1 --2- 172.21.15.60:0/157405179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4898159270 0x7f48981596f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f48980cfcc0 tx=0x7f489000b480 comp rx=0 tx=0).stop 2024-04-25T10:05:34.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.599+0000 7f48a0634640 1 -- 172.21.15.60:0/157405179 shutdown_connections 2024-04-25T10:05:34.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.599+0000 7f48a0634640 1 --2- 172.21.15.60:0/157405179 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f487c066f60 0x7f487c069420 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.599+0000 7f48a0634640 1 --2- 172.21.15.60:0/157405179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4898159270 0x7f48981596f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.599+0000 7f48a0634640 1 --2- 172.21.15.60:0/157405179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48980cb970 0x7f48980cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.599+0000 7f48a0634640 1 --2- 172.21.15.60:0/157405179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48840039c0 0x7f489815a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.599+0000 7f48a0634640 1 -- 172.21.15.60:0/157405179 >> 172.21.15.60:0/157405179 conn(0x7f489806f530 msgr2=0x7f48980b5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.599+0000 7f48a0634640 1 -- 172.21.15.60:0/157405179 shutdown_connections 2024-04-25T10:05:34.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.599+0000 7f48a0634640 1 -- 172.21.15.60:0/157405179 wait complete. 2024-04-25T10:05:34.621 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.727+0000 7fe2bf74a640 1 Processor -- start 2024-04-25T10:05:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.727+0000 7fe2bf74a640 1 -- start start 2024-04-25T10:05:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.727+0000 7fe2bf74a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe2b815a7a0 0x7fe2b80cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.727+0000 7fe2bf74a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b806f130 con 0x7fe2b80b37e0 2024-04-25T10:05:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.727+0000 7fe2bf74a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b806f410 con 0x7fe2b806ec30 2024-04-25T10:05:34.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.727+0000 7fe2bf74a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b80cdaa0 con 0x7fe2b815a7a0 2024-04-25T10:05:34.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2bdce7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe2b80b37e0 0x7fe2b806dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37372/0 (socket says 172.21.15.60:37372) 2024-04-25T10:05:34.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2bdce7640 1 -- 172.21.15.60:0/2352383445 learned_addr learned my addr 172.21.15.60:0/2352383445 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:34.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2bd4e6640 1 --2- 172.21.15.60:0/2352383445 >> v2:172.21.15.60:3300/0 conn(0x7fe2b815a7a0 0x7fe2b80cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2affff640 1 -- 172.21.15.60:0/2352383445 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046335635 0 0) 0x7fe2b806f130 con 0x7fe2b80b37e0 2024-04-25T10:05:34.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2affff640 1 -- 172.21.15.60:0/2352383445 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2a4003530 con 0x7fe2b80b37e0 2024-04-25T10:05:34.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2bd4e6640 1 -- 172.21.15.60:0/2352383445 >> v1:172.21.15.100:6789/0 conn(0x7fe2b806ec30 legacy=0x7fe2b8071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:34.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2bd4e6640 1 -- 172.21.15.60:0/2352383445 >> v1:172.21.15.60:6789/0 conn(0x7fe2b80b37e0 legacy=0x7fe2b806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2bd4e6640 1 -- 172.21.15.60:0/2352383445 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b806d540 con 0x7fe2b815a7a0 2024-04-25T10:05:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2bd4e6640 1 --2- 172.21.15.60:0/2352383445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b815a7a0 0x7fe2b80cd560 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40097f0 tx=0x7fe2b401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32897b0cf5ed540 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2affff640 1 -- 172.21.15.60:0/2352383445 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40b0070 con 0x7fe2b815a7a0 2024-04-25T10:05:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2affff640 1 -- 172.21.15.60:0/2352383445 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b40ab600 con 0x7fe2b815a7a0 2024-04-25T10:05:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2affff640 1 -- 172.21.15.60:0/2352383445 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40ab9d0 con 0x7fe2b815a7a0 2024-04-25T10:05:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2bf74a640 1 -- 172.21.15.60:0/2352383445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b815a7a0 msgr2=0x7fe2b80cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2bf74a640 1 --2- 172.21.15.60:0/2352383445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b815a7a0 0x7fe2b80cd560 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40097f0 tx=0x7fe2b401c460 comp rx=0 tx=0).stop 2024-04-25T10:05:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2bf74a640 1 -- 172.21.15.60:0/2352383445 shutdown_connections 2024-04-25T10:05:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2bf74a640 1 --2- 172.21.15.60:0/2352383445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b815a7a0 0x7fe2b80cd560 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2bf74a640 1 -- 172.21.15.60:0/2352383445 >> 172.21.15.60:0/2352383445 conn(0x7fe2b80b12d0 msgr2=0x7fe2b80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2bf74a640 1 -- 172.21.15.60:0/2352383445 shutdown_connections 2024-04-25T10:05:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.731+0000 7fe2bf74a640 1 -- 172.21.15.60:0/2352383445 wait complete. 2024-04-25T10:05:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bf74a640 1 Processor -- start 2024-04-25T10:05:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bf74a640 1 -- start start 2024-04-25T10:05:34.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bf74a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2b806ec30 0x7fe2b8140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bf74a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b80b37e0 0x7fe2b81fa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bf74a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2b815a7a0 0x7fe2b81fa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bf74a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b806d8b0 con 0x7fe2b80b37e0 2024-04-25T10:05:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bf74a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b806d2f0 con 0x7fe2b806ec30 2024-04-25T10:05:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bf74a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80b26d0 con 0x7fe2b815a7a0 2024-04-25T10:05:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bcce5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b80b37e0 0x7fe2b81fa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bcce5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b80b37e0 0x7fe2b81fa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46774/0 (socket says 172.21.15.60:46774) 2024-04-25T10:05:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bcce5640 1 -- 172.21.15.60:0/707454145 learned_addr learned my addr 172.21.15.60:0/707454145 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bcce5640 1 -- 172.21.15.60:0/707454145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2b815a7a0 msgr2=0x7fe2b81fa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bcce5640 1 --2- 172.21.15.60:0/707454145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2b815a7a0 0x7fe2b81fa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bcce5640 1 -- 172.21.15.60:0/707454145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2b806ec30 msgr2=0x7fe2b8140e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bcce5640 1 --2- 172.21.15.60:0/707454145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2b806ec30 0x7fe2b8140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bcce5640 1 -- 172.21.15.60:0/707454145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b8164500 con 0x7fe2b80b37e0 2024-04-25T10:05:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bcce5640 1 --2- 172.21.15.60:0/707454145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b80b37e0 0x7fe2b81fa410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800c410 tx=0x7fe2a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2adffb640 1 -- 172.21.15.60:0/707454145 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8011790 con 0x7fe2b80b37e0 2024-04-25T10:05:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2adffb640 1 -- 172.21.15.60:0/707454145 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a8017070 con 0x7fe2b80b37e0 2024-04-25T10:05:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bf74a640 1 -- 172.21.15.60:0/707454145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b81647e0 con 0x7fe2b80b37e0 2024-04-25T10:05:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2adffb640 1 -- 172.21.15.60:0/707454145 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8011bb0 con 0x7fe2b80b37e0 2024-04-25T10:05:34.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bf74a640 1 -- 172.21.15.60:0/707454145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b8164da0 con 0x7fe2b80b37e0 2024-04-25T10:05:34.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2bf74a640 1 -- 172.21.15.60:0/707454145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b8074770 con 0x7fe2b80b37e0 2024-04-25T10:05:34.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.735+0000 7fe2adffb640 1 -- 172.21.15.60:0/707454145 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2a801c460 con 0x7fe2b80b37e0 2024-04-25T10:05:34.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.739+0000 7fe2adffb640 1 --2- 172.21.15.60:0/707454145 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe29c066f60 0x7fe29c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.739+0000 7fe2adffb640 1 -- 172.21.15.60:0/707454145 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2a8084d70 con 0x7fe2b80b37e0 2024-04-25T10:05:34.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.739+0000 7fe2bd4e6640 1 --2- 172.21.15.60:0/707454145 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe29c066f60 0x7fe29c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.739+0000 7fe2bd4e6640 1 --2- 172.21.15.60:0/707454145 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe29c066f60 0x7fe29c069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40ad000 tx=0x7fe2b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.743+0000 7fe2adffb640 1 -- 172.21.15.60:0/707454145 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2a8021020 con 0x7fe2b80b37e0 2024-04-25T10:05:34.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.927+0000 7fe2bf74a640 1 -- 172.21.15.60:0/707454145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2b8067c90 con 0x7fe2b80b37e0 2024-04-25T10:05:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.931+0000 7fe2adffb640 1 -- 172.21.15.60:0/707454145 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7fe2a8055ce0 con 0x7fe2b80b37e0 2024-04-25T10:05:34.936 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:34.937 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/622088871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":622088871},{"type":"v1","addr":"172.21.15.100:6836","nonce":622088871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3988204202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3988204202},{"type":"v1","addr":"172.21.15.60:6833","nonce":3988204202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3591836867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3591836867},{"type":"v1","addr":"172.21.15.60:6835","nonce":3591836867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:20.405486+0000","modified":"2024-04-25T10:05:23.453258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/690070228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":690070228},{"type":"v1","addr":"172.21.15.100:6837","nonce":690070228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:05:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-04-25T10:05:34.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.931+0000 7fe2bf74a640 1 -- 172.21.15.60:0/707454145 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe29c066f60 msgr2=0x7fe29c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.931+0000 7fe2bf74a640 1 --2- 172.21.15.60:0/707454145 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe29c066f60 0x7fe29c069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40ad000 tx=0x7fe2b4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.931+0000 7fe2bf74a640 1 -- 172.21.15.60:0/707454145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b80b37e0 msgr2=0x7fe2b81fa410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.931+0000 7fe2bf74a640 1 --2- 172.21.15.60:0/707454145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b80b37e0 0x7fe2b81fa410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800c410 tx=0x7fe2a800c8d0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.931+0000 7fe2bf74a640 1 -- 172.21.15.60:0/707454145 shutdown_connections 2024-04-25T10:05:34.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.931+0000 7fe2bf74a640 1 --2- 172.21.15.60:0/707454145 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe29c066f60 0x7fe29c069420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.931+0000 7fe2bf74a640 1 --2- 172.21.15.60:0/707454145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2b815a7a0 0x7fe2b81fa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.931+0000 7fe2bf74a640 1 --2- 172.21.15.60:0/707454145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b80b37e0 0x7fe2b81fa410 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.931+0000 7fe2bf74a640 1 --2- 172.21.15.60:0/707454145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2b806ec30 0x7fe2b8140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.931+0000 7fe2bf74a640 1 -- 172.21.15.60:0/707454145 >> 172.21.15.60:0/707454145 conn(0x7fe2b80b12d0 msgr2=0x7fe2b8165d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.931+0000 7fe2bf74a640 1 -- 172.21.15.60:0/707454145 shutdown_connections 2024-04-25T10:05:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:34.931+0000 7fe2bf74a640 1 -- 172.21.15.60:0/707454145 wait complete. 2024-04-25T10:05:34.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.071+0000 7ff7bb8a6640 1 Processor -- start 2024-04-25T10:05:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.071+0000 7ff7bb8a6640 1 -- start start 2024-04-25T10:05:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7bb8a6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff7b415deb0 0x7ff7b40b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7bb8a6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b40d0340 0x7ff7b40d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7bb8a6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff7b40ce2b0 0x7ff7b40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7bb8a6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b40cf2c0 con 0x7ff7b40ce2b0 2024-04-25T10:05:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7bb8a6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b40cefe0 con 0x7ff7b415deb0 2024-04-25T10:05:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7bb8a6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40d1d70 con 0x7ff7b40d0340 2024-04-25T10:05:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7b9e43640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff7b40ce2b0 0x7ff7b40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7b9e43640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff7b40ce2b0 0x7ff7b40ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46776/0 (socket says 172.21.15.60:46776) 2024-04-25T10:05:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7b9e43640 1 -- 172.21.15.60:0/2771080887 learned_addr learned my addr 172.21.15.60:0/2771080887 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7b9642640 1 --2- 172.21.15.60:0/2771080887 >> v2:172.21.15.100:3300/0 conn(0x7ff7b415deb0 0x7ff7b40b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7b8e41640 1 --2- 172.21.15.60:0/2771080887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b40d0340 0x7ff7b40d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7b9e43640 1 -- 172.21.15.60:0/2771080887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b40d0340 msgr2=0x7ff7b40d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7b9e43640 1 --2- 172.21.15.60:0/2771080887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b40d0340 0x7ff7b40d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7b9e43640 1 -- 172.21.15.60:0/2771080887 >> v2:172.21.15.100:3300/0 conn(0x7ff7b415deb0 msgr2=0x7ff7b40b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7b9e43640 1 --2- 172.21.15.60:0/2771080887 >> v2:172.21.15.100:3300/0 conn(0x7ff7b415deb0 0x7ff7b40b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7b9e43640 1 -- 172.21.15.60:0/2771080887 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b406d200 con 0x7ff7b40ce2b0 2024-04-25T10:05:35.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7b9e43640 1 --2- 172.21.15.60:0/2771080887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7b40ce2b0 0x7ff7b40ce740 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff7b408bd20 tx=0x7ff7ac01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce94fec3f598c5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.075+0000 7ff7abfff640 1 -- 172.21.15.60:0/2771080887 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac0a5630 con 0x7ff7b40ce2b0 2024-04-25T10:05:35.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7abfff640 1 -- 172.21.15.60:0/2771080887 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac0a57d0 con 0x7ff7b40ce2b0 2024-04-25T10:05:35.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7abfff640 1 -- 172.21.15.60:0/2771080887 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac0a5ba0 con 0x7ff7b40ce2b0 2024-04-25T10:05:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 -- 172.21.15.60:0/2771080887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7b40ce2b0 msgr2=0x7ff7b40ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 --2- 172.21.15.60:0/2771080887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7b40ce2b0 0x7ff7b40ce740 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff7b408bd20 tx=0x7ff7ac01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 -- 172.21.15.60:0/2771080887 shutdown_connections 2024-04-25T10:05:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 --2- 172.21.15.60:0/2771080887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7b40ce2b0 0x7ff7b40ce740 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 --2- 172.21.15.60:0/2771080887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b40d0340 0x7ff7b40d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 --2- 172.21.15.60:0/2771080887 >> v2:172.21.15.100:3300/0 conn(0x7ff7b415deb0 0x7ff7b40b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 -- 172.21.15.60:0/2771080887 >> 172.21.15.60:0/2771080887 conn(0x7ff7b406c090 msgr2=0x7ff7b406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 -- 172.21.15.60:0/2771080887 shutdown_connections 2024-04-25T10:05:35.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 -- 172.21.15.60:0/2771080887 wait complete. 2024-04-25T10:05:35.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 Processor -- start 2024-04-25T10:05:35.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 -- start start 2024-04-25T10:05:35.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b40ce2b0 0x7ff7b40d0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b40d0340 0x7ff7b4160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7b415deb0 0x7ff7b40c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40b5d40 con 0x7ff7b415deb0 2024-04-25T10:05:35.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40b5bc0 con 0x7ff7b40d0340 2024-04-25T10:05:35.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40b5ec0 con 0x7ff7b40ce2b0 2024-04-25T10:05:35.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7b9e43640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7b415deb0 0x7ff7b40c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7b9e43640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7b415deb0 0x7ff7b40c6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46782/0 (socket says 172.21.15.60:46782) 2024-04-25T10:05:35.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7b9e43640 1 -- 172.21.15.60:0/2016321400 learned_addr learned my addr 172.21.15.60:0/2016321400 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:35.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7b9642640 1 --2- 172.21.15.60:0/2016321400 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b40ce2b0 0x7ff7b40d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7b8e41640 1 --2- 172.21.15.60:0/2016321400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b40d0340 0x7ff7b4160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7b9e43640 1 -- 172.21.15.60:0/2016321400 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b40ce2b0 msgr2=0x7ff7b40d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7b9e43640 1 --2- 172.21.15.60:0/2016321400 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b40ce2b0 0x7ff7b40d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7b9e43640 1 -- 172.21.15.60:0/2016321400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b40d0340 msgr2=0x7ff7b4160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7b9e43640 1 --2- 172.21.15.60:0/2016321400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b40d0340 0x7ff7b4160e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7b9e43640 1 -- 172.21.15.60:0/2016321400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b4149e30 con 0x7ff7b415deb0 2024-04-25T10:05:35.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7b9642640 1 --2- 172.21.15.60:0/2016321400 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b40ce2b0 0x7ff7b40d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:35.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7b9e43640 1 --2- 172.21.15.60:0/2016321400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7b415deb0 0x7ff7b40c6890 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac01d290 tx=0x7ff7ac019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7a9ffb640 1 -- 172.21.15.60:0/2016321400 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac009510 con 0x7ff7b415deb0 2024-04-25T10:05:35.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7a9ffb640 1 -- 172.21.15.60:0/2016321400 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac0be070 con 0x7ff7b415deb0 2024-04-25T10:05:35.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 -- 172.21.15.60:0/2016321400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b414a1a0 con 0x7ff7b415deb0 2024-04-25T10:05:35.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7a9ffb640 1 -- 172.21.15.60:0/2016321400 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac0b88f0 con 0x7ff7b415deb0 2024-04-25T10:05:35.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.079+0000 7ff7bb8a6640 1 -- 172.21.15.60:0/2016321400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b414bb40 con 0x7ff7b415deb0 2024-04-25T10:05:35.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.083+0000 7ff7bb8a6640 1 -- 172.21.15.60:0/2016321400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b4074770 con 0x7ff7b415deb0 2024-04-25T10:05:35.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.083+0000 7ff7a9ffb640 1 -- 172.21.15.60:0/2016321400 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7ac0b8a90 con 0x7ff7b415deb0 2024-04-25T10:05:35.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.083+0000 7ff7a9ffb640 1 --2- 172.21.15.60:0/2016321400 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff798066f90 0x7ff798069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.083+0000 7ff7a9ffb640 1 -- 172.21.15.60:0/2016321400 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7ac11fcf0 con 0x7ff7b415deb0 2024-04-25T10:05:35.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.083+0000 7ff7b9642640 1 --2- 172.21.15.60:0/2016321400 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff798066f90 0x7ff798069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.083+0000 7ff7b9642640 1 --2- 172.21.15.60:0/2016321400 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff798066f90 0x7ff798069450 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009a40 tx=0x7ff7b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.087+0000 7ff7a9ffb640 1 -- 172.21.15.60:0/2016321400 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff7ac124070 con 0x7ff7b415deb0 2024-04-25T10:05:35.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.247+0000 7ff7bb8a6640 1 -- 172.21.15.60:0/2016321400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7b40cf840 con 0x7ff7b415deb0 2024-04-25T10:05:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.251+0000 7ff7a9ffb640 1 -- 172.21.15.60:0/2016321400 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16617 (secure 0 0 0) 0x7ff7b4149e30 con 0x7ff7b415deb0 2024-04-25T10:05:35.256 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:35.257 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:22.036582+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.251+0000 7ff7bb8a6640 1 -- 172.21.15.60:0/2016321400 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff798066f90 msgr2=0x7ff798069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.251+0000 7ff7bb8a6640 1 --2- 172.21.15.60:0/2016321400 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff798066f90 0x7ff798069450 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009a40 tx=0x7ff7b0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.251+0000 7ff7bb8a6640 1 -- 172.21.15.60:0/2016321400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7b415deb0 msgr2=0x7ff7b40c6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.251+0000 7ff7bb8a6640 1 --2- 172.21.15.60:0/2016321400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7b415deb0 0x7ff7b40c6890 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac01d290 tx=0x7ff7ac019790 comp rx=0 tx=0).stop 2024-04-25T10:05:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.251+0000 7ff7bb8a6640 1 -- 172.21.15.60:0/2016321400 shutdown_connections 2024-04-25T10:05:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.251+0000 7ff7bb8a6640 1 --2- 172.21.15.60:0/2016321400 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff798066f90 0x7ff798069450 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.251+0000 7ff7bb8a6640 1 --2- 172.21.15.60:0/2016321400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7b415deb0 0x7ff7b40c6890 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.251+0000 7ff7bb8a6640 1 --2- 172.21.15.60:0/2016321400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7b40d0340 0x7ff7b4160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.251+0000 7ff7bb8a6640 1 --2- 172.21.15.60:0/2016321400 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7b40ce2b0 0x7ff7b40d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.251+0000 7ff7bb8a6640 1 -- 172.21.15.60:0/2016321400 >> 172.21.15.60:0/2016321400 conn(0x7ff7b406c090 msgr2=0x7ff7b4155be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.251+0000 7ff7bb8a6640 1 -- 172.21.15.60:0/2016321400 shutdown_connections 2024-04-25T10:05:35.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.251+0000 7ff7bb8a6640 1 -- 172.21.15.60:0/2016321400 wait complete. 2024-04-25T10:05:35.275 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:35.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.379+0000 7f965d0cd640 1 Processor -- start 2024-04-25T10:05:35.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.379+0000 7f965d0cd640 1 -- start start 2024-04-25T10:05:35.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.379+0000 7f965d0cd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f965806e480 0x7f965806e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.379+0000 7f965d0cd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96580cce50 0x7f96580cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.379+0000 7f965d0cd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965815dc10 con 0x7f9658071b20 2024-04-25T10:05:35.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.379+0000 7f965d0cd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f965815def0 con 0x7f965806e480 2024-04-25T10:05:35.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.379+0000 7f965d0cd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965815e1d0 con 0x7f96580cce50 2024-04-25T10:05:35.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.379+0000 7f965659c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f965806e480 0x7f965806e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.379+0000 7f965759e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96580cce50 0x7f96580cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.379+0000 7f9656d9d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9658071b20 0x7f9658071f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:60370/0 (socket says 172.21.15.60:60370) 2024-04-25T10:05:35.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.379+0000 7f9656d9d640 1 -- 172.21.15.60:0/502284671 learned_addr learned my addr 172.21.15.60:0/502284671 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:35.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.379+0000 7f965659c640 1 -- 172.21.15.60:0/502284671 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96580cce50 msgr2=0x7f96580cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:35.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.379+0000 7f965659c640 1 --2- 172.21.15.60:0/502284671 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96580cce50 0x7f96580cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.379+0000 7f965659c640 1 -- 172.21.15.60:0/502284671 >> v1:172.21.15.100:6789/0 conn(0x7f9658071b20 legacy=0x7f9658071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.379+0000 7f965659c640 1 -- 172.21.15.60:0/502284671 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965815e4b0 con 0x7f965806e480 2024-04-25T10:05:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965659c640 1 --2- 172.21.15.60:0/502284671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f965806e480 0x7f965806e910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f964c0097c0 tx=0x7f964c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6be60e9ae9fb40ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f9655d9b640 1 -- 172.21.15.60:0/502284671 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c0b0070 con 0x7f965806e480 2024-04-25T10:05:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f9655d9b640 1 -- 172.21.15.60:0/502284671 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c0ab600 con 0x7f965806e480 2024-04-25T10:05:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f9655d9b640 1 -- 172.21.15.60:0/502284671 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c0ab9d0 con 0x7f965806e480 2024-04-25T10:05:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965d0cd640 1 -- 172.21.15.60:0/502284671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f965806e480 msgr2=0x7f965806e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965d0cd640 1 --2- 172.21.15.60:0/502284671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f965806e480 0x7f965806e910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f964c0097c0 tx=0x7f964c01c410 comp rx=0 tx=0).stop 2024-04-25T10:05:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965d0cd640 1 -- 172.21.15.60:0/502284671 shutdown_connections 2024-04-25T10:05:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965d0cd640 1 --2- 172.21.15.60:0/502284671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f965806e480 0x7f965806e910 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965d0cd640 1 --2- 172.21.15.60:0/502284671 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96580cce50 0x7f96580cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965d0cd640 1 -- 172.21.15.60:0/502284671 >> 172.21.15.60:0/502284671 conn(0x7f96580b12d0 msgr2=0x7f96580b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965d0cd640 1 -- 172.21.15.60:0/502284671 shutdown_connections 2024-04-25T10:05:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965d0cd640 1 -- 172.21.15.60:0/502284671 wait complete. 2024-04-25T10:05:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965d0cd640 1 Processor -- start 2024-04-25T10:05:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965d0cd640 1 -- start start 2024-04-25T10:05:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965d0cd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f965806e480 0x7f9658140120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965d0cd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9658071b20 0x7f9658155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965d0cd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96580cce50 0x7f9658155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965d0cd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96580cd980 con 0x7f96580cce50 2024-04-25T10:05:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965d0cd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96580cd800 con 0x7f965806e480 2024-04-25T10:05:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965d0cd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96580cdb00 con 0x7f9658071b20 2024-04-25T10:05:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96580cce50 0x7f9658155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96580cce50 0x7f9658155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46792/0 (socket says 172.21.15.60:46792) 2024-04-25T10:05:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f965759e640 1 -- 172.21.15.60:0/1952010982 learned_addr learned my addr 172.21.15.60:0/1952010982 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.383+0000 7f9656d9d640 1 --2- 172.21.15.60:0/1952010982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f965806e480 0x7f9658140120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f965659c640 1 --2- 172.21.15.60:0/1952010982 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9658071b20 0x7f9658155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f965759e640 1 -- 172.21.15.60:0/1952010982 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9658071b20 msgr2=0x7f9658155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f965759e640 1 --2- 172.21.15.60:0/1952010982 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9658071b20 0x7f9658155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f965759e640 1 -- 172.21.15.60:0/1952010982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f965806e480 msgr2=0x7f9658140120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f965759e640 1 --2- 172.21.15.60:0/1952010982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f965806e480 0x7f9658140120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f965759e640 1 -- 172.21.15.60:0/1952010982 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965814fac0 con 0x7f96580cce50 2024-04-25T10:05:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f9656d9d640 1 --2- 172.21.15.60:0/1952010982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f965806e480 0x7f9658140120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f965759e640 1 --2- 172.21.15.60:0/1952010982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96580cce50 0x7f9658155650 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9650009db0 tx=0x7f96500075e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f963f7fe640 1 -- 172.21.15.60:0/1952010982 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9650019070 con 0x7f96580cce50 2024-04-25T10:05:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f963f7fe640 1 -- 172.21.15.60:0/1952010982 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96500039a0 con 0x7f96580cce50 2024-04-25T10:05:35.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f965d0cd640 1 -- 172.21.15.60:0/1952010982 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9658153a70 con 0x7f96580cce50 2024-04-25T10:05:35.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f963f7fe640 1 -- 172.21.15.60:0/1952010982 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9650003d90 con 0x7f96580cce50 2024-04-25T10:05:35.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f965d0cd640 1 -- 172.21.15.60:0/1952010982 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9658154030 con 0x7f96580cce50 2024-04-25T10:05:35.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f965d0cd640 1 -- 172.21.15.60:0/1952010982 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965814fc90 con 0x7f96580cce50 2024-04-25T10:05:35.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f963f7fe640 1 -- 172.21.15.60:0/1952010982 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f965001c070 con 0x7f96580cce50 2024-04-25T10:05:35.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f963f7fe640 1 --2- 172.21.15.60:0/1952010982 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9644066f60 0x7f9644069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f963f7fe640 1 -- 172.21.15.60:0/1952010982 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9650084f30 con 0x7f96580cce50 2024-04-25T10:05:35.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f9656d9d640 1 --2- 172.21.15.60:0/1952010982 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9644066f60 0x7f9644069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.387+0000 7f9656d9d640 1 --2- 172.21.15.60:0/1952010982 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9644066f60 0x7f9644069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9648005890 tx=0x7f9648009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.391+0000 7f963f7fe640 1 -- 172.21.15.60:0/1952010982 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9650011070 con 0x7f96580cce50 2024-04-25T10:05:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.579+0000 7f965d0cd640 1 -- 172.21.15.60:0/1952010982 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96580b2010 con 0x7f96580cce50 2024-04-25T10:05:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.579+0000 7f963f7fe640 1 -- 172.21.15.60:0/1952010982 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f9650055f40 con 0x7f96580cce50 2024-04-25T10:05:35.588 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:35.588 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/622088871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":622088871},{"type":"v1","addr":"172.21.15.100:6836","nonce":622088871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3988204202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3988204202},{"type":"v1","addr":"172.21.15.60:6833","nonce":3988204202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3591836867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3591836867},{"type":"v1","addr":"172.21.15.60:6835","nonce":3591836867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:20.405486+0000","modified":"2024-04-25T10:05:23.453258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/690070228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":690070228},{"type":"v1","addr":"172.21.15.100:6837","nonce":690070228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:05:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-04-25T10:05:35.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.583+0000 7f965d0cd640 1 -- 172.21.15.60:0/1952010982 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9644066f60 msgr2=0x7f9644069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.583+0000 7f965d0cd640 1 --2- 172.21.15.60:0/1952010982 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9644066f60 0x7f9644069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9648005890 tx=0x7f9648009040 comp rx=0 tx=0).stop 2024-04-25T10:05:35.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.583+0000 7f965d0cd640 1 -- 172.21.15.60:0/1952010982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96580cce50 msgr2=0x7f9658155650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.583+0000 7f965d0cd640 1 --2- 172.21.15.60:0/1952010982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96580cce50 0x7f9658155650 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9650009db0 tx=0x7f96500075e0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.583+0000 7f965d0cd640 1 -- 172.21.15.60:0/1952010982 shutdown_connections 2024-04-25T10:05:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.583+0000 7f965d0cd640 1 --2- 172.21.15.60:0/1952010982 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9644066f60 0x7f9644069420 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.583+0000 7f965d0cd640 1 --2- 172.21.15.60:0/1952010982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96580cce50 0x7f9658155650 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.583+0000 7f965d0cd640 1 --2- 172.21.15.60:0/1952010982 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9658071b20 0x7f9658155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.583+0000 7f965d0cd640 1 --2- 172.21.15.60:0/1952010982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f965806e480 0x7f9658140120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.583+0000 7f965d0cd640 1 -- 172.21.15.60:0/1952010982 >> 172.21.15.60:0/1952010982 conn(0x7f96580b12d0 msgr2=0x7f965806d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.583+0000 7f965d0cd640 1 -- 172.21.15.60:0/1952010982 shutdown_connections 2024-04-25T10:05:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:35.583+0000 7f965d0cd640 1 -- 172.21.15.60:0/1952010982 wait complete. 2024-04-25T10:05:36.008 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:05:20.405486+0000', 'modified': '2024-04-25T10:05:23.453258+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4254}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4254': {'gid': 4254, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6837/690070228', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6835', 'nonce': 690070228}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 690070228}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:05:36.009 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:05:36.009 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:05:36.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.139+0000 7f71859b9640 1 Processor -- start 2024-04-25T10:05:36.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.139+0000 7f71859b9640 1 -- start start 2024-04-25T10:05:36.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f71859b9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f71800cd390 0x7f718009a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f71859b9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f718009a930 0x7f71800cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f71859b9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71800cc5d0 con 0x7f718015f480 2024-04-25T10:05:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f71859b9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71800cc8b0 con 0x7f71800cd390 2024-04-25T10:05:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f71859b9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71800ccb90 con 0x7f718009a930 2024-04-25T10:05:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f717effd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f718015f480 0x7f718015f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37388/0 (socket says 172.21.15.60:37388) 2024-04-25T10:05:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f717effd640 1 -- 172.21.15.60:0/3151612445 learned_addr learned my addr 172.21.15.60:0/3151612445 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f717e7fc640 1 --2- 172.21.15.60:0/3151612445 >> v2:172.21.15.60:3300/0 conn(0x7f71800cd390 0x7f718009a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f717dffb640 1 -- 172.21.15.60:0/3151612445 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 535831177 0 0) 0x7f71800cc5d0 con 0x7f718015f480 2024-04-25T10:05:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f717f7fe640 1 --2- 172.21.15.60:0/3151612445 >> v2:172.21.15.100:3300/0 conn(0x7f718009a930 0x7f71800cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f717dffb640 1 -- 172.21.15.60:0/3151612445 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f716c003530 con 0x7f718015f480 2024-04-25T10:05:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f717e7fc640 1 -- 172.21.15.60:0/3151612445 >> v2:172.21.15.100:3300/0 conn(0x7f718009a930 msgr2=0x7f71800cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f717e7fc640 1 --2- 172.21.15.60:0/3151612445 >> v2:172.21.15.100:3300/0 conn(0x7f718009a930 0x7f71800cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f717e7fc640 1 -- 172.21.15.60:0/3151612445 >> v1:172.21.15.60:6789/0 conn(0x7f718015f480 legacy=0x7f718015f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f717e7fc640 1 -- 172.21.15.60:0/3151612445 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71800cce70 con 0x7f71800cd390 2024-04-25T10:05:36.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f717e7fc640 1 --2- 172.21.15.60:0/3151612445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71800cd390 0x7f718009a3f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7180071e80 tx=0x7f716801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a564b23c58d40a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f717dffb640 1 -- 172.21.15.60:0/3151612445 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71680b0070 con 0x7f71800cd390 2024-04-25T10:05:36.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f717dffb640 1 -- 172.21.15.60:0/3151612445 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71680ab600 con 0x7f71800cd390 2024-04-25T10:05:36.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f717dffb640 1 -- 172.21.15.60:0/3151612445 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71680ab9d0 con 0x7f71800cd390 2024-04-25T10:05:36.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f71859b9640 1 -- 172.21.15.60:0/3151612445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71800cd390 msgr2=0x7f718009a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f71859b9640 1 --2- 172.21.15.60:0/3151612445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71800cd390 0x7f718009a3f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7180071e80 tx=0x7f716801c420 comp rx=0 tx=0).stop 2024-04-25T10:05:36.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f71859b9640 1 -- 172.21.15.60:0/3151612445 shutdown_connections 2024-04-25T10:05:36.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f71859b9640 1 --2- 172.21.15.60:0/3151612445 >> v2:172.21.15.100:3300/0 conn(0x7f718009a930 0x7f71800cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f71859b9640 1 --2- 172.21.15.60:0/3151612445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71800cd390 0x7f718009a3f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f71859b9640 1 -- 172.21.15.60:0/3151612445 >> 172.21.15.60:0/3151612445 conn(0x7f71800b32a0 msgr2=0x7f71800b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f71859b9640 1 -- 172.21.15.60:0/3151612445 shutdown_connections 2024-04-25T10:05:36.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.143+0000 7f71859b9640 1 -- 172.21.15.60:0/3151612445 wait complete. 2024-04-25T10:05:36.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f71859b9640 1 Processor -- start 2024-04-25T10:05:36.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f71859b9640 1 -- start start 2024-04-25T10:05:36.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f71859b9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f718009a930 0x7f718015a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f71859b9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71800cd390 0x7f7180153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f71859b9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f718015f480 0x7f71800afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f71859b9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71801687c0 con 0x7f718009a930 2024-04-25T10:05:36.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f71859b9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718015cdd0 con 0x7f718015f480 2024-04-25T10:05:36.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f71859b9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7180168940 con 0x7f71800cd390 2024-04-25T10:05:36.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f717effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f718009a930 0x7f718015a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f717effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f718009a930 0x7f718015a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46810/0 (socket says 172.21.15.60:46810) 2024-04-25T10:05:36.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f717effd640 1 -- 172.21.15.60:0/3969443739 learned_addr learned my addr 172.21.15.60:0/3969443739 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:36.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f717effd640 1 -- 172.21.15.60:0/3969443739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71800cd390 msgr2=0x7f7180153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f717effd640 1 --2- 172.21.15.60:0/3969443739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71800cd390 0x7f7180153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f717effd640 1 -- 172.21.15.60:0/3969443739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f718015f480 msgr2=0x7f71800afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f717effd640 1 --2- 172.21.15.60:0/3969443739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f718015f480 0x7f71800afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f717effd640 1 -- 172.21.15.60:0/3969443739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71800b0410 con 0x7f718009a930 2024-04-25T10:05:36.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f717effd640 1 --2- 172.21.15.60:0/3969443739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f718009a930 0x7f718015a9f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f717000c3c0 tx=0x7f717000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f7177fff640 1 -- 172.21.15.60:0/3969443739 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7170017070 con 0x7f718009a930 2024-04-25T10:05:36.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f7177fff640 1 -- 172.21.15.60:0/3969443739 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7170005cf0 con 0x7f718009a930 2024-04-25T10:05:36.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f71859b9640 1 -- 172.21.15.60:0/3969443739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71800b06f0 con 0x7f718009a930 2024-04-25T10:05:36.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f7177fff640 1 -- 172.21.15.60:0/3969443739 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71700072f0 con 0x7f718009a930 2024-04-25T10:05:36.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.147+0000 7f71859b9640 1 -- 172.21.15.60:0/3969443739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7180156c70 con 0x7f718009a930 2024-04-25T10:05:36.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7177fff640 1 -- 172.21.15.60:0/3969443739 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f717001a070 con 0x7f718009a930 2024-04-25T10:05:36.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7177fff640 1 --2- 172.21.15.60:0/3969443739 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7160066f60 0x7f7160069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7177fff640 1 -- 172.21.15.60:0/3969443739 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7170084430 con 0x7f718009a930 2024-04-25T10:05:36.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f717e7fc640 1 --2- 172.21.15.60:0/3969443739 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7160066f60 0x7f7160069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 Processor -- start 2024-04-25T10:05:36.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 -- start start 2024-04-25T10:05:36.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f717e7fc640 1 --2- 172.21.15.60:0/3969443739 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7160066f60 0x7f7160069420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f71680ad000 tx=0x7f7168004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c493d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c494240 0x55e12c4499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c449f30 0x55e12c44a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12c3315f0 con 0x55e12c493920 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12c298da0 con 0x55e12c494240 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e12c221150 con 0x55e12c449f30 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f717effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c493d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f717effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c493d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46820/0 (socket says 172.21.15.60:46820) 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f717effd640 1 -- 172.21.15.60:0/2086448379 learned_addr learned my addr 172.21.15.60:0/2086448379 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f717e7fc640 1 --2- 172.21.15.60:0/2086448379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c494240 0x55e12c4499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f717f7fe640 1 --2- 172.21.15.60:0/2086448379 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c449f30 0x55e12c44a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f717effd640 1 -- 172.21.15.60:0/2086448379 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c449f30 msgr2=0x55e12c44a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f717effd640 1 --2- 172.21.15.60:0/2086448379 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c449f30 0x55e12c44a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f717effd640 1 -- 172.21.15.60:0/2086448379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c494240 msgr2=0x55e12c4499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f717effd640 1 --2- 172.21.15.60:0/2086448379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c494240 0x55e12c4499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f717effd640 1 -- 172.21.15.60:0/2086448379 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e12c23aca0 con 0x55e12c493920 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f717f7fe640 1 --2- 172.21.15.60:0/2086448379 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c449f30 0x55e12c44a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f717effd640 1 --2- 172.21.15.60:0/2086448379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c493d00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7170082980 tx=0x7f7170054e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6144e88861154ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f71757fa640 1 -- 172.21.15.60:0/2086448379 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717005b0c0 con 0x55e12c493920 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f71757fa640 1 -- 172.21.15.60:0/2086448379 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7170053040 con 0x55e12c493920 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f71757fa640 1 -- 172.21.15.60:0/2086448379 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717005cfc0 con 0x55e12c493920 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 -- 172.21.15.60:0/2086448379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 msgr2=0x55e12c493d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 --2- 172.21.15.60:0/2086448379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c493d00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7170082980 tx=0x7f7170054e00 comp rx=0 tx=0).stop 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 -- 172.21.15.60:0/2086448379 shutdown_connections 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 --2- 172.21.15.60:0/2086448379 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c449f30 0x55e12c44a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 --2- 172.21.15.60:0/2086448379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c494240 0x55e12c4499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 --2- 172.21.15.60:0/2086448379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c493d00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 -- 172.21.15.60:0/2086448379 >> 172.21.15.60:0/2086448379 conn(0x55e12c4502a0 msgr2=0x55e12c4482d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 -- 172.21.15.60:0/2086448379 shutdown_connections 2024-04-25T10:05:36.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.151+0000 7f7188b26000 1 -- 172.21.15.60:0/2086448379 wait complete. 2024-04-25T10:05:36.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 1 -- start start 2024-04-25T10:05:36.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 1 Processor -- start 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c449f30 0x55e12c49ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c49b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c494240 0x55e12c44daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12c253990 con 0x55e12c493920 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12c294e40 con 0x55e12c494240 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e12c224900 con 0x55e12c449f30 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 10 client.0 _ll_register_callbacks cb 0x55e12c34c1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f717e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c49b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f717e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c49b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46832/0 (socket says 172.21.15.60:46832) 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f717e7fc640 1 -- 172.21.15.60:0/910935299 learned_addr learned my addr 172.21.15.60:0/910935299 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f717effd640 1 --2- 172.21.15.60:0/910935299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c449f30 0x55e12c49ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f717f7fe640 1 --2- 172.21.15.60:0/910935299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c494240 0x55e12c44daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f717e7fc640 1 -- 172.21.15.60:0/910935299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c449f30 msgr2=0x55e12c49ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f717e7fc640 1 --2- 172.21.15.60:0/910935299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c449f30 0x55e12c49ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f717e7fc640 1 -- 172.21.15.60:0/910935299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c494240 msgr2=0x55e12c44daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f717e7fc640 1 --2- 172.21.15.60:0/910935299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c494240 0x55e12c44daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f717e7fc640 1 -- 172.21.15.60:0/910935299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e12c251630 con 0x55e12c493920 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f717effd640 1 --2- 172.21.15.60:0/910935299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c449f30 0x55e12c49ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f717e7fc640 1 --2- 172.21.15.60:0/910935299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c49b1c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f716801c900 tx=0x7f71680046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0520f7e75a8f81e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f715e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f715e7fc640 1 -- 172.21.15.60:0/910935299 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71680b0040 con 0x55e12c493920 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f715e7fc640 1 -- 172.21.15.60:0/910935299 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71680b6620 con 0x55e12c493920 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 1 -- 172.21.15.60:0/910935299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7170053040 con 0x55e12c493920 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f715e7fc640 1 -- 172.21.15.60:0/910935299 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71680ce050 con 0x55e12c493920 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 10 client.4704 fetch_fsmap learned FSMap version 8 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 1 -- 172.21.15.60:0/910935299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55e12c2213a0 con 0x55e12c493920 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f715e7fc640 1 -- 172.21.15.60:0/910935299 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7170053040 con 0x55e12c493920 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f715e7fc640 1 -- 172.21.15.60:0/910935299 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f71680d1020 con 0x55e12c493920 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 10 client.4704 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 10 client.4704 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 20 client.4704 populate_metadata read hostname 'smithi060' 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 1 --2- 172.21.15.60:0/910935299 >> [v2:172.21.15.100:6835/690070228,v1:172.21.15.100:6837/690070228] conn(0x55e12c44f160 0x55e12c4a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 4 client.4704 mds_command: new command op to 4254 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f7188b26000 1 -- 172.21.15.60:0/910935299 --> [v2:172.21.15.100:6835/690070228,v1:172.21.15.100:6837/690070228] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e12c234a50 con 0x55e12c44f160 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f717effd640 1 --2- 172.21.15.60:0/910935299 >> [v2:172.21.15.100:6835/690070228,v1:172.21.15.100:6837/690070228] conn(0x55e12c44f160 0x55e12c4a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f717effd640 1 --2- 172.21.15.60:0/910935299 >> [v2:172.21.15.100:6835/690070228,v1:172.21.15.100:6837/690070228] conn(0x55e12c44f160 0x55e12c4a4ce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c00fcea19b579aa server_cookie=8b4f48a9d71da1e in_seq=0 out_seq=0 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.155+0000 7f715e7fc640 10 client.4704 ms_handle_connect on v2:172.21.15.100:6835/690070228 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.159+0000 7f715e7fc640 1 -- 172.21.15.60:0/910935299 <== mds.0 v2:172.21.15.100:6835/690070228 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e12c234a50 con 0x55e12c44f160 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.159+0000 7f715e7fc640 10 client.4704 handle_command_reply: tid=0 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.159+0000 7f7188b26000 1 client.4704 shutdown 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.159+0000 7f7188b26000 1 -- 172.21.15.60:0/910935299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 msgr2=0x55e12c49b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.159+0000 7f7188b26000 1 --2- 172.21.15.60:0/910935299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c49b1c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f716801c900 tx=0x7f71680046d0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.159+0000 7f7188b26000 1 -- 172.21.15.60:0/910935299 shutdown_connections 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.159+0000 7f7188b26000 1 --2- 172.21.15.60:0/910935299 >> [v2:172.21.15.100:6835/690070228,v1:172.21.15.100:6837/690070228] conn(0x55e12c44f160 0x55e12c4a4ce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.159+0000 7f7188b26000 1 --2- 172.21.15.60:0/910935299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c494240 0x55e12c44daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.159+0000 7f7188b26000 1 --2- 172.21.15.60:0/910935299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c49b1c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.159+0000 7f7188b26000 1 --2- 172.21.15.60:0/910935299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c449f30 0x55e12c49ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.159+0000 7f715e7fc640 0 client.4704 ms_handle_reset on v2:172.21.15.100:6835/690070228 2024-04-25T10:05:36.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.159+0000 7f7188b26000 1 -- 172.21.15.60:0/910935299 >> 172.21.15.60:0/910935299 conn(0x55e12c4502a0 msgr2=0x55e12c4489e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.159+0000 7f7188b26000 1 -- 172.21.15.60:0/910935299 shutdown_connections 2024-04-25T10:05:36.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.159+0000 7f7188b26000 1 -- 172.21.15.60:0/910935299 wait complete. 2024-04-25T10:05:36.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.159+0000 7f7188b26000 20 client.4704 trim_cache size 0 max 16384 2024-04-25T10:05:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 Processor -- start 2024-04-25T10:05:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 -- start start 2024-04-25T10:05:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c449f30 0x55e12c44d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c493920 0x55e12c44d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c494240 0x55e12c49c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7180165d20 con 0x55e12c494240 2024-04-25T10:05:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718006c7f0 con 0x55e12c449f30 2024-04-25T10:05:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7180165720 con 0x55e12c493920 2024-04-25T10:05:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f717f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c494240 0x55e12c49c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f717f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c494240 0x55e12c49c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46834/0 (socket says 172.21.15.60:46834) 2024-04-25T10:05:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f717f7fe640 1 -- 172.21.15.60:0/1215673319 learned_addr learned my addr 172.21.15.60:0/1215673319 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f717effd640 1 --2- 172.21.15.60:0/1215673319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c449f30 0x55e12c44d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f717e7fc640 1 --2- 172.21.15.60:0/1215673319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c493920 0x55e12c44d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f717f7fe640 1 -- 172.21.15.60:0/1215673319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c493920 msgr2=0x55e12c44d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f717f7fe640 1 --2- 172.21.15.60:0/1215673319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c493920 0x55e12c44d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f717f7fe640 1 -- 172.21.15.60:0/1215673319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c449f30 msgr2=0x55e12c44d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f717f7fe640 1 --2- 172.21.15.60:0/1215673319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c449f30 0x55e12c44d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f717f7fe640 1 -- 172.21.15.60:0/1215673319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e12c494070 con 0x55e12c494240 2024-04-25T10:05:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f717f7fe640 1 --2- 172.21.15.60:0/1215673319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c494240 0x55e12c49c8b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7178004360 tx=0x7f7178004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9dab8ff45e75640 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7174ff9640 1 -- 172.21.15.60:0/1215673319 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178014610 con 0x55e12c494240 2024-04-25T10:05:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7174ff9640 1 -- 172.21.15.60:0/1215673319 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7178019070 con 0x55e12c494240 2024-04-25T10:05:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7174ff9640 1 -- 172.21.15.60:0/1215673319 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71780149f0 con 0x55e12c494240 2024-04-25T10:05:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 -- 172.21.15.60:0/1215673319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c494240 msgr2=0x55e12c49c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 --2- 172.21.15.60:0/1215673319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c494240 0x55e12c49c8b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7178004360 tx=0x7f7178004820 comp rx=0 tx=0).stop 2024-04-25T10:05:36.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 -- 172.21.15.60:0/1215673319 shutdown_connections 2024-04-25T10:05:36.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 --2- 172.21.15.60:0/1215673319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c494240 0x55e12c49c8b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 --2- 172.21.15.60:0/1215673319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c493920 0x55e12c44d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 --2- 172.21.15.60:0/1215673319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c449f30 0x55e12c44d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 -- 172.21.15.60:0/1215673319 >> 172.21.15.60:0/1215673319 conn(0x55e12c4502a0 msgr2=0x55e12c44cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 -- 172.21.15.60:0/1215673319 shutdown_connections 2024-04-25T10:05:36.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.175+0000 7f7188b26000 1 -- 172.21.15.60:0/1215673319 wait complete. 2024-04-25T10:05:36.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 1 -- start start 2024-04-25T10:05:36.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 1 Processor -- start 2024-04-25T10:05:36.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c449f30 0x55e12c49f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c4f4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c494240 0x55e12c4f4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12c29d3d0 con 0x55e12c493920 2024-04-25T10:05:36.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12c354d50 con 0x55e12c494240 2024-04-25T10:05:36.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e12c233980 con 0x55e12c449f30 2024-04-25T10:05:36.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 10 client.0 _ll_register_callbacks cb 0x7f7170051060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:36.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f717e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c4f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f717e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c4f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46850/0 (socket says 172.21.15.60:46850) 2024-04-25T10:05:36.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f717e7fc640 1 -- 172.21.15.60:0/3796330541 learned_addr learned my addr 172.21.15.60:0/3796330541 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:36.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f717e7fc640 1 -- 172.21.15.60:0/3796330541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c449f30 msgr2=0x55e12c49f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f717e7fc640 1 --2- 172.21.15.60:0/3796330541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c449f30 0x55e12c49f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f717e7fc640 1 -- 172.21.15.60:0/3796330541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c494240 msgr2=0x55e12c4f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f717e7fc640 1 --2- 172.21.15.60:0/3796330541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c494240 0x55e12c4f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f717e7fc640 1 -- 172.21.15.60:0/3796330541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e12c31a600 con 0x55e12c493920 2024-04-25T10:05:36.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f717e7fc640 1 --2- 172.21.15.60:0/3796330541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c4f4310 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7168009160 tx=0x7f716801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafba389b170656a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f715e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:05:36.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f715e7fc640 1 -- 172.21.15.60:0/3796330541 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71680d7060 con 0x55e12c493920 2024-04-25T10:05:36.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f715e7fc640 1 -- 172.21.15.60:0/3796330541 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7168004030 con 0x55e12c493920 2024-04-25T10:05:36.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 1 -- 172.21.15.60:0/3796330541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7178019070 con 0x55e12c493920 2024-04-25T10:05:36.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f715e7fc640 1 -- 172.21.15.60:0/3796330541 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71680dd050 con 0x55e12c493920 2024-04-25T10:05:36.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 10 client.4710 fetch_fsmap learned FSMap version 8 2024-04-25T10:05:36.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 1 -- 172.21.15.60:0/3796330541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55e12c22d9b0 con 0x55e12c493920 2024-04-25T10:05:36.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f715e7fc640 1 -- 172.21.15.60:0/3796330541 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7178019070 con 0x55e12c493920 2024-04-25T10:05:36.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f715e7fc640 1 -- 172.21.15.60:0/3796330541 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f71680e0020 con 0x55e12c493920 2024-04-25T10:05:36.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 10 client.4710 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T10:05:36.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 10 client.4710 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:05:36.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 20 client.4710 populate_metadata read hostname 'smithi060' 2024-04-25T10:05:36.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 1 --2- 172.21.15.60:0/3796330541 >> [v2:172.21.15.100:6835/690070228,v1:172.21.15.100:6837/690070228] conn(0x55e12c4f4e00 0x55e12c4ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 4 client.4710 mds_command: new command op to 4254 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:05:36.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f7188b26000 1 -- 172.21.15.60:0/3796330541 --> [v2:172.21.15.100:6835/690070228,v1:172.21.15.100:6837/690070228] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e12c2a62e0 con 0x55e12c4f4e00 2024-04-25T10:05:36.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f717effd640 1 --2- 172.21.15.60:0/3796330541 >> [v2:172.21.15.100:6835/690070228,v1:172.21.15.100:6837/690070228] conn(0x55e12c4f4e00 0x55e12c4ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f717effd640 1 --2- 172.21.15.60:0/3796330541 >> [v2:172.21.15.100:6835/690070228,v1:172.21.15.100:6837/690070228] conn(0x55e12c4f4e00 0x55e12c4ae760 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1132d338fac78cc5 server_cookie=6ba1e3af745055e1 in_seq=0 out_seq=0 2024-04-25T10:05:36.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.179+0000 7f715e7fc640 10 client.4710 ms_handle_connect on v2:172.21.15.100:6835/690070228 2024-04-25T10:05:36.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f715e7fc640 1 -- 172.21.15.60:0/3796330541 <== mds.0 v2:172.21.15.100:6835/690070228 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55e12c2a62e0 con 0x55e12c4f4e00 2024-04-25T10:05:36.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f715e7fc640 10 client.4710 handle_command_reply: tid=0 2024-04-25T10:05:36.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f7188b26000 1 client.4710 shutdown 2024-04-25T10:05:36.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f7188b26000 1 -- 172.21.15.60:0/3796330541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 msgr2=0x55e12c4f4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f7188b26000 1 --2- 172.21.15.60:0/3796330541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c4f4310 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7168009160 tx=0x7f716801acc0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f7188b26000 1 -- 172.21.15.60:0/3796330541 shutdown_connections 2024-04-25T10:05:36.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f7188b26000 1 --2- 172.21.15.60:0/3796330541 >> [v2:172.21.15.100:6835/690070228,v1:172.21.15.100:6837/690070228] conn(0x55e12c4f4e00 0x55e12c4ae760 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f7188b26000 1 --2- 172.21.15.60:0/3796330541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e12c494240 0x55e12c4f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f7188b26000 1 --2- 172.21.15.60:0/3796330541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e12c493920 0x55e12c4f4310 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f7188b26000 1 --2- 172.21.15.60:0/3796330541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e12c449f30 0x55e12c49f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f715e7fc640 0 client.4710 ms_handle_reset on v2:172.21.15.100:6835/690070228 2024-04-25T10:05:36.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f7188b26000 1 -- 172.21.15.60:0/3796330541 >> 172.21.15.60:0/3796330541 conn(0x55e12c4502a0 msgr2=0x55e12c44d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f7188b26000 1 -- 172.21.15.60:0/3796330541 shutdown_connections 2024-04-25T10:05:36.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f7188b26000 1 -- 172.21.15.60:0/3796330541 wait complete. 2024-04-25T10:05:36.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f7188b26000 20 client.4710 trim_cache size 0 max 16384 2024-04-25T10:05:36.200 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:05:36.200 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "0ea93863-a079-45c4-abed-6e83540ea9fe", 2024-04-25T10:05:36.200 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T10:05:36.200 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4254, 2024-04-25T10:05:36.200 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T10:05:36.200 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T10:05:36.201 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T10:05:36.201 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 15.114267357999999, 2024-04-25T10:05:36.201 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 8, 2024-04-25T10:05:36.201 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 22, 2024-04-25T10:05:36.201 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 22, 2024-04-25T10:05:36.201 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 21.717234919999999 2024-04-25T10:05:36.201 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:05:36.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f71859b9640 1 -- 172.21.15.60:0/3969443739 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7160066f60 msgr2=0x7f7160069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f71859b9640 1 --2- 172.21.15.60:0/3969443739 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7160066f60 0x7f7160069420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f71680ad000 tx=0x7f7168004740 comp rx=0 tx=0).stop 2024-04-25T10:05:36.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f71859b9640 1 -- 172.21.15.60:0/3969443739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f718009a930 msgr2=0x7f718015a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f71859b9640 1 --2- 172.21.15.60:0/3969443739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f718009a930 0x7f718015a9f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f717000c3c0 tx=0x7f717000c880 comp rx=0 tx=0).stop 2024-04-25T10:05:36.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f71859b9640 1 -- 172.21.15.60:0/3969443739 shutdown_connections 2024-04-25T10:05:36.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f71859b9640 1 --2- 172.21.15.60:0/3969443739 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7160066f60 0x7f7160069420 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f71859b9640 1 --2- 172.21.15.60:0/3969443739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f718015f480 0x7f71800afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f71859b9640 1 --2- 172.21.15.60:0/3969443739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71800cd390 0x7f7180153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f71859b9640 1 --2- 172.21.15.60:0/3969443739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f718009a930 0x7f718015a9f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f71859b9640 1 -- 172.21.15.60:0/3969443739 >> 172.21.15.60:0/3969443739 conn(0x7f71800b32a0 msgr2=0x7f71800b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f71859b9640 1 -- 172.21.15.60:0/3969443739 shutdown_connections 2024-04-25T10:05:36.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.183+0000 7f71859b9640 1 -- 172.21.15.60:0/3969443739 wait complete. 2024-04-25T10:05:36.206 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T10:05:36.242 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T10:05:36.242 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T10:05:36.242 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T10:05:36.243 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:05:36.243 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:36.243 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-25T10:05:36.243 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:05:36.243 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:05:36.243 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-25T10:05:36.243 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:05:36.243 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:05:36.243 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-25T10:05:36.243 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:05:36.243 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:05:36.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:36.244 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:05:36.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:36.250 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-04-25T10:05:36.306 INFO:teuthology.orchestra.run.smithi060.stderr:Cannot find device "ceph-brx" 2024-04-25T10:05:36.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:05:36.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:36.308 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-25T10:05:36.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:36.315 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-04-25T10:05:36.376 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2024-04-25T10:05:36.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:05:36.377 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T10:05:36.378 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:05:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b545ff640 1 Processor -- start 2024-04-25T10:05:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b545ff640 1 -- start start 2024-04-25T10:05:36.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b545ff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7b4c06e480 0x7f7b4c06e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b545ff640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c15dc10 con 0x7f7b4c0cce50 2024-04-25T10:05:36.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b545ff640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c15def0 con 0x7f7b4c071b20 2024-04-25T10:05:36.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b545ff640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b4c15e1d0 con 0x7f7b4c06e480 2024-04-25T10:05:36.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b52b9c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7b4c0cce50 0x7f7b4c0cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37396/0 (socket says 172.21.15.60:37396) 2024-04-25T10:05:36.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b52b9c640 1 -- 172.21.15.60:0/2540774186 learned_addr learned my addr 172.21.15.60:0/2540774186 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:36.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51b9a640 1 --2- 172.21.15.60:0/2540774186 >> v2:172.21.15.100:3300/0 conn(0x7f7b4c06e480 0x7f7b4c06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 -- 172.21.15.60:0/2540774186 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 575404839 0 0) 0x7f7b4c15dc10 con 0x7f7b4c0cce50 2024-04-25T10:05:36.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 -- 172.21.15.60:0/2540774186 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b38004640 con 0x7f7b4c0cce50 2024-04-25T10:05:36.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 -- 172.21.15.60:0/2540774186 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432217197 0 0) 0x7f7b4c15def0 con 0x7f7b4c071b20 2024-04-25T10:05:36.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 -- 172.21.15.60:0/2540774186 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b4c15dc10 con 0x7f7b4c071b20 2024-04-25T10:05:36.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 -- 172.21.15.60:0/2540774186 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2209489660 0 0) 0x7f7b38004640 con 0x7f7b4c0cce50 2024-04-25T10:05:36.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 -- 172.21.15.60:0/2540774186 >> v2:172.21.15.100:3300/0 conn(0x7f7b4c06e480 msgr2=0x7f7b4c06e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 --2- 172.21.15.60:0/2540774186 >> v2:172.21.15.100:3300/0 conn(0x7f7b4c06e480 0x7f7b4c06e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 -- 172.21.15.60:0/2540774186 >> v1:172.21.15.100:6789/0 conn(0x7f7b4c071b20 legacy=0x7f7b4c071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 -- 172.21.15.60:0/2540774186 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c15e4b0 con 0x7f7b4c0cce50 2024-04-25T10:05:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 -- 172.21.15.60:0/2540774186 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f7b44001a00 con 0x7f7b4c0cce50 2024-04-25T10:05:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 -- 172.21.15.60:0/2540774186 >> v1:172.21.15.60:6789/0 conn(0x7f7b4c0cce50 legacy=0x7f7b4c0cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 --2- 172.21.15.60:0/2540774186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b38004c60 0x7f7b38005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 --2- 172.21.15.60:0/2540774186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b38005920 0x7f7b38005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 --2- 172.21.15.60:0/2540774186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b380062e0 0x7f7b3800cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 -- 172.21.15.60:0/2540774186 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c15e1d0 con 0x7f7b38004c60 2024-04-25T10:05:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 -- 172.21.15.60:0/2540774186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3800d210 con 0x7f7b380062e0 2024-04-25T10:05:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51399640 1 -- 172.21.15.60:0/2540774186 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3800d410 con 0x7f7b38005920 2024-04-25T10:05:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b5239b640 1 --2- 172.21.15.60:0/2540774186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b38004c60 0x7f7b38005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b52b9c640 1 --2- 172.21.15.60:0/2540774186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b380062e0 0x7f7b3800cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b51b9a640 1 --2- 172.21.15.60:0/2540774186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b38005920 0x7f7b38005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b5239b640 1 -- 172.21.15.60:0/2540774186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b38005920 msgr2=0x7f7b38005da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b5239b640 1 --2- 172.21.15.60:0/2540774186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b38005920 0x7f7b38005da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b5239b640 1 -- 172.21.15.60:0/2540774186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b380062e0 msgr2=0x7f7b3800cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b5239b640 1 --2- 172.21.15.60:0/2540774186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b380062e0 0x7f7b3800cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b5239b640 1 -- 172.21.15.60:0/2540774186 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b38004640 con 0x7f7b38004c60 2024-04-25T10:05:36.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b52b9c640 1 --2- 172.21.15.60:0/2540774186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b380062e0 0x7f7b3800cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:36.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.503+0000 7f7b52b9c640 1 -- 172.21.15.60:0/2540774186 reap_dead start 2024-04-25T10:05:36.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.507+0000 7f7b5239b640 1 --2- 172.21.15.60:0/2540774186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b38004c60 0x7f7b38005060 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7b480018d0 tx=0x7f7b4801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101663081b29e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.507+0000 7f7b51399640 1 -- 172.21.15.60:0/2540774186 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48001520 con 0x7f7b38004c60 2024-04-25T10:05:36.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.507+0000 7f7b51399640 1 -- 172.21.15.60:0/2540774186 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b480052f0 con 0x7f7b38004c60 2024-04-25T10:05:36.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.507+0000 7f7b51399640 1 -- 172.21.15.60:0/2540774186 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48003840 con 0x7f7b38004c60 2024-04-25T10:05:36.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.507+0000 7f7b545ff640 1 -- 172.21.15.60:0/2540774186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b38004c60 msgr2=0x7f7b38005060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.507+0000 7f7b545ff640 1 --2- 172.21.15.60:0/2540774186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b38004c60 0x7f7b38005060 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7b480018d0 tx=0x7f7b4801c560 comp rx=0 tx=0).stop 2024-04-25T10:05:36.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.507+0000 7f7b545ff640 1 -- 172.21.15.60:0/2540774186 shutdown_connections 2024-04-25T10:05:36.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.507+0000 7f7b545ff640 1 --2- 172.21.15.60:0/2540774186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b38004c60 0x7f7b38005060 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.507+0000 7f7b545ff640 1 -- 172.21.15.60:0/2540774186 >> 172.21.15.60:0/2540774186 conn(0x7f7b4c0b12d0 msgr2=0x7f7b4c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.507+0000 7f7b545ff640 1 -- 172.21.15.60:0/2540774186 shutdown_connections 2024-04-25T10:05:36.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.507+0000 7f7b545ff640 1 -- 172.21.15.60:0/2540774186 wait complete. 2024-04-25T10:05:36.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.507+0000 7f7b545ff640 1 Processor -- start 2024-04-25T10:05:36.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.507+0000 7f7b545ff640 1 -- start start 2024-04-25T10:05:36.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b545ff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b38004c60 0x7f7b4c159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b545ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b4c13df20 0x7f7b4c1f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b545ff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b4c1f6180 0x7f7b4c12edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b545ff640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c06e9e0 con 0x7f7b38004c60 2024-04-25T10:05:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b545ff640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c06e860 con 0x7f7b4c13df20 2024-04-25T10:05:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b545ff640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c06eb60 con 0x7f7b4c1f6180 2024-04-25T10:05:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b5239b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b38004c60 0x7f7b4c159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b5239b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b38004c60 0x7f7b4c159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46864/0 (socket says 172.21.15.60:46864) 2024-04-25T10:05:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b5239b640 1 -- 172.21.15.60:0/2990393970 learned_addr learned my addr 172.21.15.60:0/2990393970 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b52b9c640 1 --2- 172.21.15.60:0/2990393970 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b4c1f6180 0x7f7b4c12edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b51b9a640 1 --2- 172.21.15.60:0/2990393970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b4c13df20 0x7f7b4c1f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b5239b640 1 -- 172.21.15.60:0/2990393970 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b4c1f6180 msgr2=0x7f7b4c12edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b5239b640 1 --2- 172.21.15.60:0/2990393970 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b4c1f6180 0x7f7b4c12edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b5239b640 1 -- 172.21.15.60:0/2990393970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b4c13df20 msgr2=0x7f7b4c1f5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b5239b640 1 --2- 172.21.15.60:0/2990393970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b4c13df20 0x7f7b4c1f5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b5239b640 1 -- 172.21.15.60:0/2990393970 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c1588c0 con 0x7f7b38004c60 2024-04-25T10:05:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b51b9a640 1 --2- 172.21.15.60:0/2990393970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b4c13df20 0x7f7b4c1f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b5239b640 1 --2- 172.21.15.60:0/2990393970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b38004c60 0x7f7b4c159960 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b48005040 tx=0x7f7b38006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b42ffd640 1 -- 172.21.15.60:0/2990393970 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b480054a0 con 0x7f7b38004c60 2024-04-25T10:05:36.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b42ffd640 1 -- 172.21.15.60:0/2990393970 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b48001500 con 0x7f7b38004c60 2024-04-25T10:05:36.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b545ff640 1 -- 172.21.15.60:0/2990393970 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4c158ba0 con 0x7f7b38004c60 2024-04-25T10:05:36.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b42ffd640 1 -- 172.21.15.60:0/2990393970 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48003ce0 con 0x7f7b38004c60 2024-04-25T10:05:36.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b545ff640 1 -- 172.21.15.60:0/2990393970 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4c159160 con 0x7f7b38004c60 2024-04-25T10:05:36.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b545ff640 1 -- 172.21.15.60:0/2990393970 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b4c074770 con 0x7f7b38004c60 2024-04-25T10:05:36.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b42ffd640 1 -- 172.21.15.60:0/2990393970 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7b480b5480 con 0x7f7b38004c60 2024-04-25T10:05:36.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b42ffd640 1 --2- 172.21.15.60:0/2990393970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7b30066f60 0x7f7b30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b42ffd640 1 -- 172.21.15.60:0/2990393970 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b4811e790 con 0x7f7b38004c60 2024-04-25T10:05:36.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.511+0000 7f7b51b9a640 1 --2- 172.21.15.60:0/2990393970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7b30066f60 0x7f7b30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.515+0000 7f7b51b9a640 1 --2- 172.21.15.60:0/2990393970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7b30066f60 0x7f7b30069420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c142520 tx=0x7f7b3c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.515+0000 7f7b42ffd640 1 -- 172.21.15.60:0/2990393970 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7b48016270 con 0x7f7b38004c60 2024-04-25T10:05:36.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.711+0000 7f7b545ff640 1 -- 172.21.15.60:0/2990393970 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7b4c0b2010 con 0x7f7b38004c60 2024-04-25T10:05:36.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.711+0000 7f7b42ffd640 1 -- 172.21.15.60:0/2990393970 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7b480ef7a0 con 0x7f7b38004c60 2024-04-25T10:05:36.720 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:05:36.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.715+0000 7f7b545ff640 1 -- 172.21.15.60:0/2990393970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7b30066f60 msgr2=0x7f7b30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.715+0000 7f7b545ff640 1 --2- 172.21.15.60:0/2990393970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7b30066f60 0x7f7b30069420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c142520 tx=0x7f7b3c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:36.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.715+0000 7f7b545ff640 1 -- 172.21.15.60:0/2990393970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b38004c60 msgr2=0x7f7b4c159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.715+0000 7f7b545ff640 1 --2- 172.21.15.60:0/2990393970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b38004c60 0x7f7b4c159960 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b48005040 tx=0x7f7b38006950 comp rx=0 tx=0).stop 2024-04-25T10:05:36.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.715+0000 7f7b545ff640 1 -- 172.21.15.60:0/2990393970 shutdown_connections 2024-04-25T10:05:36.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.715+0000 7f7b545ff640 1 --2- 172.21.15.60:0/2990393970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7b30066f60 0x7f7b30069420 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.715+0000 7f7b545ff640 1 --2- 172.21.15.60:0/2990393970 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b4c1f6180 0x7f7b4c12edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.715+0000 7f7b545ff640 1 --2- 172.21.15.60:0/2990393970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b4c13df20 0x7f7b4c1f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.715+0000 7f7b545ff640 1 --2- 172.21.15.60:0/2990393970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b38004c60 0x7f7b4c159960 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.715+0000 7f7b545ff640 1 -- 172.21.15.60:0/2990393970 >> 172.21.15.60:0/2990393970 conn(0x7f7b4c0b12d0 msgr2=0x7f7b4c13ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.715+0000 7f7b545ff640 1 -- 172.21.15.60:0/2990393970 shutdown_connections 2024-04-25T10:05:36.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:36.715+0000 7f7b545ff640 1 -- 172.21.15.60:0/2990393970 wait complete. 2024-04-25T10:05:36.739 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:05:36.740 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:05:36.740 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:05:36.740 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:05:36.740 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:36.740 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:05:36.740 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:05:36.740 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:05:36.740 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:36.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:36.740 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:05:36.746 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:36.746 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:05:36.747 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:05:36.747 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:05:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:05:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:05:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42520sec preferred_lft 42520sec 2024-04-25T10:05:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:05:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:36.747 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:05:36.748 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:05:36.748 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:36.748 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:05:36.748 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:36.748 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:05:36.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:36.748 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:05:36.748 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:05:36.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:05:36.749 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr flush dev ceph-brx 2024-04-25T10:05:36.749 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set ceph-brx up 2024-04-25T10:05:36.749 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:05:36.749 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:05:36.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:36.824 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:05:36.878 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:05:36.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:36.879 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:05:36.942 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:05:36.942 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:05:36.942 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:05:36.942 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:05:36.942 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:05:36.942 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:05:36.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:36.943 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:05:36.943 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:05:36.943 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:05:36.943 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:05:36.943 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:05:36.943 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:05:37.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:37.016 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:05:37.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:37.066 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2024-04-25T10:05:37.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:37.114 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:05:37.114 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:05:37.114 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:37.114 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T10:05:37.114 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:05:37.185 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T10:05:37.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:37.185 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:05:37.186 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:05:37.186 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T10:05:37.186 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:05:37.186 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T10:05:37.186 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T10:05:37.186 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T10:05:37.186 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:05:37.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:37.298 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:05:37.298 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:05:37.298 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T10:05:37.298 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:05:37.298 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:05:37.318 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:05:37.318 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:37.318 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:37.365 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:37.365 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:37.365 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:37.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:05:37.473 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:37.522 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:05:37.522 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:05:37.522 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:05:37.522 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:05:37.522 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:05:37.522 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:05:37.522 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:05:37.522 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:05:37.522 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:05:37.522 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:05:37.522 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:05:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:05:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:05:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:05:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:05:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:05:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:05:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:05:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:05:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:05:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:05:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:05:37.523 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:05:37.524 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:05:37.524 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:05:37.524 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:05:37.524 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:37.524 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:37.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:37.525 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:05:37.573 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:05:37.573 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:05:37.617 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:05:37.632 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:37.681 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:05:37.681 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:05:37.681 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:05:37.681 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:05:37.681 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:05:37.682 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:05:37.682 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:05:37.682 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:05:37.682 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:05:37.682 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:05:37.682 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:05:37.682 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:05:37.682 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:37.682 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:05:37.682 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:05:37.682 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:05:37.682 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:05:37.682 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:05:37.683 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:37.683 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:05:37.683 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:37.683 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:05:37.683 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:05:37.683 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:05:37.683 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:05:37.683 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:05:37.683 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:05:37.683 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:05:37.683 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:05:37.683 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:37.683 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:37.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:37.684 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:05:37.703 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2024-04-25T10:05:37.695+0000 7f8974994580 -1 init, newargv = 0x55740ea0a150 newargc=15 2024-04-25T10:05:37.704 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[33696]: starting ceph client 2024-04-25T10:05:37.711 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[33696]: starting fuse 2024-04-25T10:05:38.810 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:05:38.827 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:38.879 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:05:38.880 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:05:38.880 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:05:38.880 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:05:38.880 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:05:38.880 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:05:38.880 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:05:38.880 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:05:38.880 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:05:38.880 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:05:38.880 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:05:38.880 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:05:38.880 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:38.881 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:05:38.881 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:05:38.881 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:05:38.881 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:05:38.881 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:05:38.881 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:38.881 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:05:38.881 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:38.881 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:05:38.881 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:05:38.881 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:05:38.881 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:05:38.881 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:05:38.881 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:05:38.881 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:05:38.882 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:05:38.882 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:38.882 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:38.882 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:38.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:38.883 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:05:38.929 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:05:38.929 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:05:38.930 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:05:38.930 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:05:38.930 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:05:38.930 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:05:38.930 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:05:38.930 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:05:38.930 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:05:38.930 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:05:38.930 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:05:38.930 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:05:38.930 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:05:38.930 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:05:38.930 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:05:38.930 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:05:38.930 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:05:38.931 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:05:38.931 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:05:38.931 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:05:38.931 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:05:38.931 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:05:38.931 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:05:38.931 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:05:38.931 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:05:38.931 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:05:38.931 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:05:38.931 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2024-04-25T10:05:38.931 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:05:39.057 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.33696.asok 2024-04-25T10:05:39.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:05:39.069 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33696.asok 2024-04-25T10:05:39.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:39.069 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33696.asok status 2024-04-25T10:05:39.225 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:05:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:05:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:05:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:05:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2024-04-25T10:05:39.225 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:05:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:05:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "33696", 2024-04-25T10:05:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:05:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:05:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:05:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:05:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4719, 2024-04-25T10:05:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:05:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:05:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:05:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 4719 2024-04-25T10:05:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:05:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:05:39.226 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:05:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:05:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4138068543 2024-04-25T10:05:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:05:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:05:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:05:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:05:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:05:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4138068543 2024-04-25T10:05:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:05:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.4719 192.168.0.1:0/4138068543", 2024-04-25T10:05:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/4138068543", 2024-04-25T10:05:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:05:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 8, 2024-04-25T10:05:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 22, 2024-04-25T10:05:39.227 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:05:39.228 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:05:39.228 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:05:39.228 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:05:39.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:05:39.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.359+0000 7f694c59c640 1 Processor -- start 2024-04-25T10:05:39.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.359+0000 7f694c59c640 1 -- start start 2024-04-25T10:05:39.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694c59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f69440ce7a0 0x7f69440ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694c59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f694415bc90 0x7f69440b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694c59c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f69440b4660 0x7f69440b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694c59c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69440cc370 con 0x7f69440ce7a0 2024-04-25T10:05:39.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694c59c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69440cf0c0 con 0x7f69440b4660 2024-04-25T10:05:39.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694c59c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6944071ca0 con 0x7f694415bc90 2024-04-25T10:05:39.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694a338640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f69440ce7a0 0x7f69440ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694a338640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f69440ce7a0 0x7f69440ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46870/0 (socket says 172.21.15.60:46870) 2024-04-25T10:05:39.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694a338640 1 -- 172.21.15.60:0/3921443726 learned_addr learned my addr 172.21.15.60:0/3921443726 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:39.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f6949b37640 1 --2- 172.21.15.60:0/3921443726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f694415bc90 0x7f69440b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694ab39640 1 --2- 172.21.15.60:0/3921443726 >> v2:172.21.15.100:3300/0 conn(0x7f69440b4660 0x7f69440b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694a338640 1 -- 172.21.15.60:0/3921443726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f694415bc90 msgr2=0x7f69440b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:39.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694a338640 1 --2- 172.21.15.60:0/3921443726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f694415bc90 0x7f69440b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694a338640 1 -- 172.21.15.60:0/3921443726 >> v2:172.21.15.100:3300/0 conn(0x7f69440b4660 msgr2=0x7f69440b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:39.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694a338640 1 --2- 172.21.15.60:0/3921443726 >> v2:172.21.15.100:3300/0 conn(0x7f69440b4660 0x7f69440b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694a338640 1 -- 172.21.15.60:0/3921443726 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69440cca40 con 0x7f69440ce7a0 2024-04-25T10:05:39.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694ab39640 1 --2- 172.21.15.60:0/3921443726 >> v2:172.21.15.100:3300/0 conn(0x7f69440b4660 0x7f69440b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:39.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f6949b37640 1 --2- 172.21.15.60:0/3921443726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f694415bc90 0x7f69440b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:39.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f694a338640 1 --2- 172.21.15.60:0/3921443726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69440ce7a0 0x7f69440ceb80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6934009e50 tx=0x7f693401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8ff26e2aae74a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f6949336640 1 -- 172.21.15.60:0/3921443726 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69340a5670 con 0x7f69440ce7a0 2024-04-25T10:05:39.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f6949336640 1 -- 172.21.15.60:0/3921443726 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69340a5810 con 0x7f69440ce7a0 2024-04-25T10:05:39.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.363+0000 7f6949336640 1 -- 172.21.15.60:0/3921443726 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69340a5be0 con 0x7f69440ce7a0 2024-04-25T10:05:39.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 -- 172.21.15.60:0/3921443726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69440ce7a0 msgr2=0x7f69440ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 --2- 172.21.15.60:0/3921443726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69440ce7a0 0x7f69440ceb80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6934009e50 tx=0x7f693401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 -- 172.21.15.60:0/3921443726 shutdown_connections 2024-04-25T10:05:39.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 --2- 172.21.15.60:0/3921443726 >> v2:172.21.15.100:3300/0 conn(0x7f69440b4660 0x7f69440b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 --2- 172.21.15.60:0/3921443726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f694415bc90 0x7f69440b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 --2- 172.21.15.60:0/3921443726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69440ce7a0 0x7f69440ceb80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 -- 172.21.15.60:0/3921443726 >> 172.21.15.60:0/3921443726 conn(0x7f69440b12d0 msgr2=0x7f69440b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 -- 172.21.15.60:0/3921443726 shutdown_connections 2024-04-25T10:05:39.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 -- 172.21.15.60:0/3921443726 wait complete. 2024-04-25T10:05:39.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 Processor -- start 2024-04-25T10:05:39.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 -- start start 2024-04-25T10:05:39.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69440ce7a0 0x7f6944145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f694415bc90 0x7f69441f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69441f6ac0 0x7f6944159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69440b3db0 con 0x7f69441f6ac0 2024-04-25T10:05:39.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69440b3460 con 0x7f69440ce7a0 2024-04-25T10:05:39.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69440b4280 con 0x7f694415bc90 2024-04-25T10:05:39.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694ab39640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69441f6ac0 0x7f6944159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694ab39640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69441f6ac0 0x7f6944159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46874/0 (socket says 172.21.15.60:46874) 2024-04-25T10:05:39.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694ab39640 1 -- 172.21.15.60:0/3216647852 learned_addr learned my addr 172.21.15.60:0/3216647852 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:39.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694a338640 1 --2- 172.21.15.60:0/3216647852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69440ce7a0 0x7f6944145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f6949b37640 1 --2- 172.21.15.60:0/3216647852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f694415bc90 0x7f69441f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694ab39640 1 -- 172.21.15.60:0/3216647852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f694415bc90 msgr2=0x7f69441f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694ab39640 1 --2- 172.21.15.60:0/3216647852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f694415bc90 0x7f69441f6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694ab39640 1 -- 172.21.15.60:0/3216647852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69440ce7a0 msgr2=0x7f6944145000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694ab39640 1 --2- 172.21.15.60:0/3216647852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69440ce7a0 0x7f6944145000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694ab39640 1 -- 172.21.15.60:0/3216647852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6944152e00 con 0x7f69441f6ac0 2024-04-25T10:05:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f6949b37640 1 --2- 172.21.15.60:0/3216647852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f694415bc90 0x7f69441f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694ab39640 1 --2- 172.21.15.60:0/3216647852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69441f6ac0 0x7f6944159d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f693c00c5e0 tx=0x7f693c00caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f6942ffd640 1 -- 172.21.15.60:0/3216647852 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c015070 con 0x7f69441f6ac0 2024-04-25T10:05:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f6942ffd640 1 -- 172.21.15.60:0/3216647852 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693c004240 con 0x7f69441f6ac0 2024-04-25T10:05:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f6942ffd640 1 -- 172.21.15.60:0/3216647852 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c003620 con 0x7f69441f6ac0 2024-04-25T10:05:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 -- 172.21.15.60:0/3216647852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6944156a10 con 0x7f69441f6ac0 2024-04-25T10:05:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.367+0000 7f694c59c640 1 -- 172.21.15.60:0/3216647852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6944156fd0 con 0x7f69441f6ac0 2024-04-25T10:05:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.371+0000 7f694c59c640 1 -- 172.21.15.60:0/3216647852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6944074770 con 0x7f69441f6ac0 2024-04-25T10:05:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.371+0000 7f6942ffd640 1 -- 172.21.15.60:0/3216647852 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f693c0037c0 con 0x7f69441f6ac0 2024-04-25T10:05:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.371+0000 7f6942ffd640 1 --2- 172.21.15.60:0/3216647852 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6930066f60 0x7f6930069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.371+0000 7f6942ffd640 1 -- 172.21.15.60:0/3216647852 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f693c084d90 con 0x7f69441f6ac0 2024-04-25T10:05:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.371+0000 7f694a338640 1 --2- 172.21.15.60:0/3216647852 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6930066f60 0x7f6930069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.371+0000 7f694a338640 1 --2- 172.21.15.60:0/3216647852 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6930066f60 0x7f6930069420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f69340ae000 tx=0x7f69340a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.375+0000 7f6942ffd640 1 -- 172.21.15.60:0/3216647852 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f693c055e30 con 0x7f69441f6ac0 2024-04-25T10:05:39.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.575+0000 7f694c59c640 1 -- 172.21.15.60:0/3216647852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6944067c90 con 0x7f69441f6ac0 2024-04-25T10:05:39.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.575+0000 7f6942ffd640 1 -- 172.21.15.60:0/3216647852 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f693c055730 con 0x7f69441f6ac0 2024-04-25T10:05:39.581 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:05:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.575+0000 7f694c59c640 1 -- 172.21.15.60:0/3216647852 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6930066f60 msgr2=0x7f6930069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.575+0000 7f694c59c640 1 --2- 172.21.15.60:0/3216647852 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6930066f60 0x7f6930069420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f69340ae000 tx=0x7f69340a7070 comp rx=0 tx=0).stop 2024-04-25T10:05:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.575+0000 7f694c59c640 1 -- 172.21.15.60:0/3216647852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69441f6ac0 msgr2=0x7f6944159d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.575+0000 7f694c59c640 1 --2- 172.21.15.60:0/3216647852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69441f6ac0 0x7f6944159d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f693c00c5e0 tx=0x7f693c00caa0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.579+0000 7f694c59c640 1 -- 172.21.15.60:0/3216647852 shutdown_connections 2024-04-25T10:05:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.579+0000 7f694c59c640 1 --2- 172.21.15.60:0/3216647852 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6930066f60 0x7f6930069420 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.579+0000 7f694c59c640 1 --2- 172.21.15.60:0/3216647852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69441f6ac0 0x7f6944159d40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.579+0000 7f694c59c640 1 --2- 172.21.15.60:0/3216647852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f694415bc90 0x7f69441f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.579+0000 7f694c59c640 1 --2- 172.21.15.60:0/3216647852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69440ce7a0 0x7f6944145000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.579+0000 7f694c59c640 1 -- 172.21.15.60:0/3216647852 >> 172.21.15.60:0/3216647852 conn(0x7f69440b12d0 msgr2=0x7f694415aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.579+0000 7f694c59c640 1 -- 172.21.15.60:0/3216647852 shutdown_connections 2024-04-25T10:05:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:39.579+0000 7f694c59c640 1 -- 172.21.15.60:0/3216647852 wait complete. 2024-04-25T10:05:39.602 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:05:39.602 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:05:39.602 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:05:39.602 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T10:05:39.602 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:05:39.602 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:05:39.602 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:05:39.602 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:05:39.603 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:05:39.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:39.603 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:05:39.609 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:39.609 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:39.609 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:39.609 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:39.609 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:39.610 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:39.610 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:39.610 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:39.610 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:05:39.610 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:39.610 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:39.610 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:05:39.610 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:05:39.610 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:39.610 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:05:39.610 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:05:39.610 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42517sec preferred_lft 42517sec 2024-04-25T10:05:39.610 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:05:39.611 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:39.611 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:39.611 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:39.611 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:05:39.611 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:05:39.611 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:39.611 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:05:39.611 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:39.611 INFO:teuthology.orchestra.run.smithi060.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:39.611 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:39.611 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:05:39.611 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:39.611 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:05:39.612 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:39.612 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:05:39.612 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:39.612 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:05:39.612 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:39.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:39.612 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:05:39.667 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:05:39.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:39.668 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:05:39.718 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:05:39.719 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:05:39.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:39.719 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:05:39.719 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:05:39.719 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:05:39.719 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:05:39.719 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:05:39.719 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:05:39.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:39.798 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:05:39.845 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:05:39.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:39.846 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2024-04-25T10:05:39.893 INFO:teuthology.orchestra.run.smithi060.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T10:05:39.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:39.894 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:05:39.894 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:05:39.894 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:39.894 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T10:05:39.894 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:05:39.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:39.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:05:40.016 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether ce:72:14:65:8d:fc brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::cc72:14ff:fe65:8dfc/64 scope link 2024-04-25T10:05:40.017 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:40.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:40.018 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:40.077 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether ce:72:14:65:8d:fc brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:05:40.078 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::cc72:14ff:fe65:8dfc/64 scope link 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:40.081 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T10:05:40.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:40.081 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:05:40.081 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:05:40.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T10:05:40.082 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:05:40.082 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T10:05:40.082 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T10:05:40.082 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T10:05:40.082 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:05:40.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:40.178 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:05:40.179 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:05:40.179 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.1 up 2024-04-25T10:05:40.179 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:05:40.179 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:05:40.199 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-25T10:05:40.199 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:40.199 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T10:05:40.244 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:05:40.244 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:40.244 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:05:40.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:05:40.347 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:40.393 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:05:40.394 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:05:40.395 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:40.396 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:40.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:40.397 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:05:40.444 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:05:40.444 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T10:05:40.444 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-25T10:05:40.490 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:05:40.505 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:40.557 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:40.558 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:05:40.559 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:40.560 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:40.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:40.561 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:05:40.569 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi060.stderr:2024-04-25T10:05:40.563+0000 7fdc77dd1580 -1 init, newargv = 0x55dd342be520 newargc=15 2024-04-25T10:05:40.569 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi060.stderr:ceph-fuse[33837]: starting ceph client 2024-04-25T10:05:40.573 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:05:40.575 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi060.stderr:ceph-fuse[33837]: starting fuse 2024-04-25T10:05:41.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:05:41.692 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:41.742 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:05:41.742 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:05:41.742 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:05:41.742 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:05:41.742 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:05:41.742 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:05:41.742 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:05:41.742 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:05:41.743 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:05:41.743 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:05:41.743 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:05:41.743 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:05:41.743 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:41.743 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:05:41.743 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:05:41.743 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:05:41.743 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:05:41.743 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:05:41.743 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:41.743 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:05:41.743 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:41.743 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:05:41.744 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:05:41.744 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:05:41.744 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:05:41.744 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:05:41.744 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:05:41.744 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:05:41.744 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:05:41.744 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:41.744 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:41.744 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:41.744 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:41.744 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:41.744 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.2 2024-04-25T10:05:41.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:41.745 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:05:41.797 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:05:41.798 INFO:teuthology.orchestra.run.smithi060.stdout:46 2024-04-25T10:05:41.798 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T10:05:41.798 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:05:41.798 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:05:41.798 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:05:41.798 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:05:41.798 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:05:41.798 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:05:41.798 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:05:41.798 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-25T10:05:41.799 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:05:41.799 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-25T10:05:41.799 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:05:41.799 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:05:41.799 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:05:41.799 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:05:41.799 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:05:41.799 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:05:41.799 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:05:41.799 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:05:41.799 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:05:41.799 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:05:41.799 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:05:41.799 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:05:41.799 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:05:41.799 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:05:41.800 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:05:41.800 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.2")) 2024-04-25T10:05:41.800 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:05:41.926 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.2.33837.asok 2024-04-25T10:05:41.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:05:41.940 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.33837.asok 2024-04-25T10:05:41.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:41.940 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.33837.asok status 2024-04-25T10:05:42.104 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:05:42.104 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:05:42.104 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:05:42.104 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:05:42.104 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "2", 2024-04-25T10:05:42.104 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:05:42.104 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-25T10:05:42.104 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "33837", 2024-04-25T10:05:42.104 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:05:42.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4731, 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 4731 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.2:0", 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1745272570 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:05:42.105 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.2:0", 2024-04-25T10:05:42.106 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1745272570 2024-04-25T10:05:42.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:05:42.106 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.4731 192.168.0.2:0/1745272570", 2024-04-25T10:05:42.106 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.2:0/1745272570", 2024-04-25T10:05:42.106 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:05:42.106 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 8, 2024-04-25T10:05:42.106 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 22, 2024-04-25T10:05:42.106 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:05:42.106 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:05:42.106 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:05:42.106 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:05:42.118 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:05:42.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.231+0000 7f24363ab640 1 Processor -- start 2024-04-25T10:05:42.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.235+0000 7f24363ab640 1 -- start start 2024-04-25T10:05:42.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.235+0000 7f24363ab640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f243006c0e0 0x7f24300d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.235+0000 7f24363ab640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24300d0930 0x7f24300d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.235+0000 7f24363ab640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243015f5a0 con 0x7f2430099b10 2024-04-25T10:05:42.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.235+0000 7f24363ab640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243006c570 con 0x7f243006c0e0 2024-04-25T10:05:42.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.235+0000 7f24363ab640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243015ea60 con 0x7f24300d0930 2024-04-25T10:05:42.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.235+0000 7f242f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f243006c0e0 0x7f24300d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.235+0000 7f242f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f243006c0e0 0x7f24300d03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46878/0 (socket says 172.21.15.60:46878) 2024-04-25T10:05:42.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.235+0000 7f242f7fe640 1 -- 172.21.15.60:0/385557285 learned_addr learned my addr 172.21.15.60:0/385557285 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:42.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.235+0000 7f2434948640 1 --2- 172.21.15.60:0/385557285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24300d0930 0x7f24300d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.235+0000 7f242f7fe640 1 -- 172.21.15.60:0/385557285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24300d0930 msgr2=0x7f24300d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:42.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.235+0000 7f242f7fe640 1 --2- 172.21.15.60:0/385557285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24300d0930 0x7f24300d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.235+0000 7f242f7fe640 1 -- 172.21.15.60:0/385557285 >> v1:172.21.15.100:6789/0 conn(0x7f2430099b10 legacy=0x7f2430099ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:42.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.235+0000 7f242f7fe640 1 -- 172.21.15.60:0/385557285 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243015ed40 con 0x7f243006c0e0 2024-04-25T10:05:42.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.235+0000 7f242f7fe640 1 --2- 172.21.15.60:0/385557285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f243006c0e0 0x7f24300d03f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2424009870 tx=0x7f242401c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54abfd8c7c78529 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.235+0000 7f242effd640 1 -- 172.21.15.60:0/385557285 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24240b0070 con 0x7f243006c0e0 2024-04-25T10:05:42.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f242effd640 1 -- 172.21.15.60:0/385557285 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24240ab600 con 0x7f243006c0e0 2024-04-25T10:05:42.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f242effd640 1 -- 172.21.15.60:0/385557285 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24240ab9d0 con 0x7f243006c0e0 2024-04-25T10:05:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 -- 172.21.15.60:0/385557285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f243006c0e0 msgr2=0x7f24300d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 --2- 172.21.15.60:0/385557285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f243006c0e0 0x7f24300d03f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2424009870 tx=0x7f242401c280 comp rx=0 tx=0).stop 2024-04-25T10:05:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 -- 172.21.15.60:0/385557285 shutdown_connections 2024-04-25T10:05:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 --2- 172.21.15.60:0/385557285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f243006c0e0 0x7f24300d03f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 --2- 172.21.15.60:0/385557285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24300d0930 0x7f24300d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 -- 172.21.15.60:0/385557285 >> 172.21.15.60:0/385557285 conn(0x7f24300b3320 msgr2=0x7f24300b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 -- 172.21.15.60:0/385557285 shutdown_connections 2024-04-25T10:05:42.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 -- 172.21.15.60:0/385557285 wait complete. 2024-04-25T10:05:42.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 Processor -- start 2024-04-25T10:05:42.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 -- start start 2024-04-25T10:05:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f243006c0e0 0x7f2430152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2430099b10 0x7f24300c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24300d0930 0x7f2430070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2430161510 con 0x7f243006c0e0 2024-04-25T10:05:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2430161390 con 0x7f24300d0930 2024-04-25T10:05:42.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24300d1300 con 0x7f2430099b10 2024-04-25T10:05:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f242ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f243006c0e0 0x7f2430152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f242ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f243006c0e0 0x7f2430152bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46888/0 (socket says 172.21.15.60:46888) 2024-04-25T10:05:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f242ffff640 1 -- 172.21.15.60:0/1576678335 learned_addr learned my addr 172.21.15.60:0/1576678335 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f242f7fe640 1 --2- 172.21.15.60:0/1576678335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2430099b10 0x7f24300c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f2434948640 1 --2- 172.21.15.60:0/1576678335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24300d0930 0x7f2430070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f242ffff640 1 -- 172.21.15.60:0/1576678335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2430099b10 msgr2=0x7f24300c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f242ffff640 1 --2- 172.21.15.60:0/1576678335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2430099b10 0x7f24300c5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f242ffff640 1 -- 172.21.15.60:0/1576678335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24300d0930 msgr2=0x7f2430070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f242ffff640 1 --2- 172.21.15.60:0/1576678335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24300d0930 0x7f2430070320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f242ffff640 1 -- 172.21.15.60:0/1576678335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24300c5e20 con 0x7f243006c0e0 2024-04-25T10:05:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f242ffff640 1 --2- 172.21.15.60:0/1576678335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f243006c0e0 0x7f2430152bb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f242000c7c0 tx=0x7f242000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f242cff9640 1 -- 172.21.15.60:0/1576678335 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420016020 con 0x7f243006c0e0 2024-04-25T10:05:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f242cff9640 1 -- 172.21.15.60:0/1576678335 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242000aa50 con 0x7f243006c0e0 2024-04-25T10:05:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 -- 172.21.15.60:0/1576678335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24300c6100 con 0x7f243006c0e0 2024-04-25T10:05:42.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f242cff9640 1 -- 172.21.15.60:0/1576678335 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420012420 con 0x7f243006c0e0 2024-04-25T10:05:42.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.239+0000 7f24363ab640 1 -- 172.21.15.60:0/1576678335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24300c66c0 con 0x7f243006c0e0 2024-04-25T10:05:42.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.243+0000 7f24363ab640 1 -- 172.21.15.60:0/1576678335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2430074770 con 0x7f243006c0e0 2024-04-25T10:05:42.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.243+0000 7f242cff9640 1 -- 172.21.15.60:0/1576678335 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2420019020 con 0x7f243006c0e0 2024-04-25T10:05:42.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.243+0000 7f242cff9640 1 --2- 172.21.15.60:0/1576678335 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f241c066f60 0x7f241c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.243+0000 7f242cff9640 1 -- 172.21.15.60:0/1576678335 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2420085420 con 0x7f243006c0e0 2024-04-25T10:05:42.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.243+0000 7f242f7fe640 1 --2- 172.21.15.60:0/1576678335 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f241c066f60 0x7f241c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.243+0000 7f242f7fe640 1 --2- 172.21.15.60:0/1576678335 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f241c066f60 0x7f241c069420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f24240ad000 tx=0x7f24240a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.247+0000 7f242cff9640 1 -- 172.21.15.60:0/1576678335 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2420020020 con 0x7f243006c0e0 2024-04-25T10:05:42.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.439+0000 7f24363ab640 1 -- 172.21.15.60:0/1576678335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2430169920 con 0x7f243006c0e0 2024-04-25T10:05:42.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.439+0000 7f242cff9640 1 -- 172.21.15.60:0/1576678335 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2420056430 con 0x7f243006c0e0 2024-04-25T10:05:42.448 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:05:42.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.443+0000 7f24363ab640 1 -- 172.21.15.60:0/1576678335 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f241c066f60 msgr2=0x7f241c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.443+0000 7f24363ab640 1 --2- 172.21.15.60:0/1576678335 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f241c066f60 0x7f241c069420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f24240ad000 tx=0x7f24240a6000 comp rx=0 tx=0).stop 2024-04-25T10:05:42.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.443+0000 7f24363ab640 1 -- 172.21.15.60:0/1576678335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f243006c0e0 msgr2=0x7f2430152bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.443+0000 7f24363ab640 1 --2- 172.21.15.60:0/1576678335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f243006c0e0 0x7f2430152bb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f242000c7c0 tx=0x7f242000cc80 comp rx=0 tx=0).stop 2024-04-25T10:05:42.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.443+0000 7f24363ab640 1 -- 172.21.15.60:0/1576678335 shutdown_connections 2024-04-25T10:05:42.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.443+0000 7f24363ab640 1 --2- 172.21.15.60:0/1576678335 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f241c066f60 0x7f241c069420 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.443+0000 7f24363ab640 1 --2- 172.21.15.60:0/1576678335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24300d0930 0x7f2430070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.443+0000 7f24363ab640 1 --2- 172.21.15.60:0/1576678335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2430099b10 0x7f24300c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.443+0000 7f24363ab640 1 --2- 172.21.15.60:0/1576678335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f243006c0e0 0x7f2430152bb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.443+0000 7f24363ab640 1 -- 172.21.15.60:0/1576678335 >> 172.21.15.60:0/1576678335 conn(0x7f24300b3320 msgr2=0x7f243015f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:42.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.443+0000 7f24363ab640 1 -- 172.21.15.60:0/1576678335 shutdown_connections 2024-04-25T10:05:42.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:42.443+0000 7f24363ab640 1 -- 172.21.15.60:0/1576678335 wait complete. 2024-04-25T10:05:42.466 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:05:42.466 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:05:42.466 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-04-25T10:05:42.466 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T10:05:42.466 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:05:42.466 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:05:42.466 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:05:42.466 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:05:42.466 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:05:42.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:42.467 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-04-25T10:05:42.474 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:42.474 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:42.474 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:42.474 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:42.474 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:42.474 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:42.474 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:42.474 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:42.474 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-04-25T10:05:42.474 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:42.475 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:42.475 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-04-25T10:05:42.475 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:42.475 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:42.475 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:05:42.475 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:42.475 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:05:42.475 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42510sec preferred_lft 42510sec 2024-04-25T10:05:42.475 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-04-25T10:05:42.475 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:42.475 INFO:teuthology.orchestra.run.smithi100.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:05:42.475 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 02:42:99:83:b8:78 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:42.475 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:05:42.475 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:42.476 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:05:42.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:42.476 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T10:05:42.476 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T10:05:42.476 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:05:42.476 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr flush dev ceph-brx 2024-04-25T10:05:42.476 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set ceph-brx up 2024-04-25T10:05:42.476 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:05:42.476 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T10:05:42.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:42.559 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:05:42.613 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-25T10:05:42.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:42.614 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-04-25T10:05:42.672 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-04-25T10:05:42.672 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:05:42.672 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:05:42.672 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:05:42.672 INFO:teuthology.orchestra.run.smithi100.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:05:42.673 INFO:teuthology.orchestra.run.smithi100.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:05:42.673 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:05:42.673 INFO:teuthology.orchestra.run.smithi100.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:05:42.673 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:05:42.673 INFO:teuthology.orchestra.run.smithi100.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:05:42.673 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:05:42.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:42.673 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T10:05:42.673 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T10:05:42.673 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:05:42.673 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:05:42.674 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:05:42.674 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T10:05:42.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:42.748 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-25T10:05:42.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:42.796 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-04-25T10:05:42.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:42.845 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T10:05:42.845 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T10:05:42.845 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:05:42.845 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T10:05:42.845 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T10:05:42.913 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T10:05:42.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:42.914 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T10:05:42.914 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T10:05:42.914 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T10:05:42.914 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:05:42.914 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T10:05:42.914 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T10:05:42.914 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T10:05:42.914 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T10:05:43.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:43.018 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T10:05:43.018 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T10:05:43.018 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-04-25T10:05:43.018 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:05:43.018 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T10:05:43.036 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T10:05:43.037 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:43.037 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T10:05:43.083 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:05:43.084 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:43.084 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:05:43.131 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-25T10:05:43.188 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:43.236 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-25T10:05:43.238 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-25T10:05:43.238 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-25T10:05:43.238 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-25T10:05:43.238 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-25T10:05:43.238 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:43.238 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-25T10:05:43.238 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:43.238 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-25T10:05:43.238 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-25T10:05:43.238 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-25T10:05:43.238 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:05:43.238 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:05:43.238 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:05:43.238 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:05:43.239 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-25T10:05:43.239 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:05:43.239 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:05:43.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:43.240 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-25T10:05:43.287 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:05:43.288 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T10:05:43.333 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-25T10:05:43.348 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:43.396 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-25T10:05:43.396 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-25T10:05:43.396 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-25T10:05:43.396 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-25T10:05:43.396 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-25T10:05:43.396 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-25T10:05:43.397 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-25T10:05:43.397 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-25T10:05:43.397 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-25T10:05:43.397 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-25T10:05:43.397 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-25T10:05:43.397 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-25T10:05:43.397 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:43.397 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-25T10:05:43.397 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-25T10:05:43.397 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-25T10:05:43.398 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-25T10:05:43.398 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-25T10:05:43.398 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:43.398 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-25T10:05:43.398 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:43.398 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-25T10:05:43.398 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-25T10:05:43.398 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-25T10:05:43.398 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:05:43.398 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:05:43.398 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:05:43.398 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:05:43.398 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-25T10:05:43.398 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:05:43.399 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:05:43.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:43.400 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-25T10:05:43.412 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi100.stderr:2024-04-25T10:05:43.405+0000 7f76fa2d4580 -1 init, newargv = 0x556118e0dc40 newargc=15 2024-04-25T10:05:43.412 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi100.stderr:ceph-fuse[33220]: starting ceph client 2024-04-25T10:05:43.418 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi100.stderr:ceph-fuse[33220]: starting fuse 2024-04-25T10:05:44.518 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-25T10:05:44.534 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:44.582 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-25T10:05:44.582 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-25T10:05:44.582 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-25T10:05:44.582 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-25T10:05:44.582 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-25T10:05:44.582 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-25T10:05:44.582 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-25T10:05:44.583 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-25T10:05:44.583 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-25T10:05:44.583 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-25T10:05:44.583 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-25T10:05:44.583 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-25T10:05:44.583 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:44.583 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-25T10:05:44.583 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-25T10:05:44.583 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-25T10:05:44.583 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-25T10:05:44.583 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-25T10:05:44.583 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:44.583 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-25T10:05:44.583 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:44.584 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-25T10:05:44.584 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-25T10:05:44.584 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-25T10:05:44.584 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:05:44.584 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:05:44.584 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:05:44.584 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:05:44.584 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-25T10:05:44.584 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:05:44.584 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:05:44.584 INFO:teuthology.orchestra.run.smithi100.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T10:05:44.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:44.585 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-25T10:05:44.628 INFO:teuthology.orchestra.run.smithi100.stdout:45 2024-04-25T10:05:44.629 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:05:44.629 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:05:44.629 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-04-25T10:05:44.629 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-04-25T10:05:44.629 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-04-25T10:05:44.629 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-04-25T10:05:44.629 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-25T10:05:44.629 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-04-25T10:05:44.629 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T10:05:44.629 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-04-25T10:05:44.629 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T10:05:44.630 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-25T10:05:44.630 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-04-25T10:05:44.630 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-04-25T10:05:44.630 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-04-25T10:05:44.630 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-04-25T10:05:44.630 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-25T10:05:44.630 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-04-25T10:05:44.630 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:05:44.630 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:05:44.630 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:05:44.630 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-04-25T10:05:44.630 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-04-25T10:05:44.630 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-04-25T10:05:44.630 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:05:44.631 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-25T10:05:44.631 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.1")) 2024-04-25T10:05:44.631 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-04-25T10:05:44.752 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.1.33220.asok 2024-04-25T10:05:44.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:05:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:05:44.764 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33220.asok 2024-04-25T10:05:44.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:44.765 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33220.asok status 2024-04-25T10:05:44.919 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-25T10:05:44.919 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-04-25T10:05:44.919 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:05:44.920 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:05:44.920 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "1", 2024-04-25T10:05:44.920 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-04-25T10:05:44.920 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T10:05:44.920 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "33220", 2024-04-25T10:05:44.920 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-04-25T10:05:44.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-25T10:05:44.920 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-04-25T10:05:44.920 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-04-25T10:05:44.920 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4633, 2024-04-25T10:05:44.920 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-04-25T10:05:44.921 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-04-25T10:05:44.921 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-04-25T10:05:44.921 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 4633 2024-04-25T10:05:44.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-25T10:05:44.921 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-25T10:05:44.921 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-25T10:05:44.921 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:05:44.921 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 695897870 2024-04-25T10:05:44.921 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-25T10:05:44.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-25T10:05:44.921 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-25T10:05:44.921 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-25T10:05:44.921 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:05:44.921 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 695897870 2024-04-25T10:05:44.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-25T10:05:44.922 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.4633 192.168.0.1:0/695897870", 2024-04-25T10:05:44.922 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.1:0/695897870", 2024-04-25T10:05:44.922 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-04-25T10:05:44.922 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 8, 2024-04-25T10:05:44.922 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 22, 2024-04-25T10:05:44.922 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:05:44.922 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-04-25T10:05:44.922 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-04-25T10:05:44.922 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-25T10:05:44.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:05:45.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.059+0000 7f487925c640 1 Processor -- start 2024-04-25T10:05:45.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.059+0000 7f487925c640 1 -- start start 2024-04-25T10:05:45.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f487925c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f487406d3f0 0x7f4874168f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f487925c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48740d16c0 con 0x7f487415ddf0 2024-04-25T10:05:45.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f487925c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48740b5c50 con 0x7f4874071560 2024-04-25T10:05:45.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f487925c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48740b53b0 con 0x7f487406d3f0 2024-04-25T10:05:45.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f4872d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f487415ddf0 0x7f4874070e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43872/0 (socket says 172.21.15.60:43872) 2024-04-25T10:05:45.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f4872d9d640 1 -- 172.21.15.60:0/3314496237 learned_addr learned my addr 172.21.15.60:0/3314496237 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:45.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f487359e640 1 --2- 172.21.15.60:0/3314496237 >> v2:172.21.15.60:3300/0 conn(0x7f487406d3f0 0x7f4874168f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f487359e640 1 -- 172.21.15.60:0/3314496237 >> v1:172.21.15.100:6789/0 conn(0x7f4874071560 legacy=0x7f4874165840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:45.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f487359e640 1 -- 172.21.15.60:0/3314496237 >> v1:172.21.15.60:6789/0 conn(0x7f487415ddf0 legacy=0x7f4874070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f487359e640 1 -- 172.21.15.60:0/3314496237 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48740b5690 con 0x7f487406d3f0 2024-04-25T10:05:45.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f487359e640 1 --2- 172.21.15.60:0/3314496237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487406d3f0 0x7f4874168f70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f486c009d90 tx=0x7f486c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e033874b2d44c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f4871d9b640 1 -- 172.21.15.60:0/3314496237 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c0b2070 con 0x7f487406d3f0 2024-04-25T10:05:45.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f4871d9b640 1 -- 172.21.15.60:0/3314496237 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c0a59e0 con 0x7f487406d3f0 2024-04-25T10:05:45.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f4871d9b640 1 -- 172.21.15.60:0/3314496237 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c0a5db0 con 0x7f487406d3f0 2024-04-25T10:05:45.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f487925c640 1 -- 172.21.15.60:0/3314496237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487406d3f0 msgr2=0x7f4874168f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f487925c640 1 --2- 172.21.15.60:0/3314496237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487406d3f0 0x7f4874168f70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f486c009d90 tx=0x7f486c01c900 comp rx=0 tx=0).stop 2024-04-25T10:05:45.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f487925c640 1 -- 172.21.15.60:0/3314496237 shutdown_connections 2024-04-25T10:05:45.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f487925c640 1 --2- 172.21.15.60:0/3314496237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487406d3f0 0x7f4874168f70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f487925c640 1 -- 172.21.15.60:0/3314496237 >> 172.21.15.60:0/3314496237 conn(0x7f48740b3520 msgr2=0x7f48740b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:45.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f487925c640 1 -- 172.21.15.60:0/3314496237 shutdown_connections 2024-04-25T10:05:45.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.063+0000 7f487925c640 1 -- 172.21.15.60:0/3314496237 wait complete. 2024-04-25T10:05:45.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487925c640 1 Processor -- start 2024-04-25T10:05:45.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487925c640 1 -- start start 2024-04-25T10:05:45.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487925c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487406d3f0 0x7f487415ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487925c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4874071560 0x7f4874151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487925c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487415ddf0 0x7f4874143400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487925c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487416c060 con 0x7f487415ddf0 2024-04-25T10:05:45.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487925c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487416bee0 con 0x7f487406d3f0 2024-04-25T10:05:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487925c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487416c1e0 con 0x7f4874071560 2024-04-25T10:05:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487415ddf0 0x7f4874143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487415ddf0 0x7f4874143400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46898/0 (socket says 172.21.15.60:46898) 2024-04-25T10:05:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487359e640 1 -- 172.21.15.60:0/107779893 learned_addr learned my addr 172.21.15.60:0/107779893 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487259c640 1 --2- 172.21.15.60:0/107779893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4874071560 0x7f4874151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f4872d9d640 1 --2- 172.21.15.60:0/107779893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487406d3f0 0x7f487415ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487359e640 1 -- 172.21.15.60:0/107779893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4874071560 msgr2=0x7f4874151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487359e640 1 --2- 172.21.15.60:0/107779893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4874071560 0x7f4874151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487359e640 1 -- 172.21.15.60:0/107779893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487406d3f0 msgr2=0x7f487415ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487359e640 1 --2- 172.21.15.60:0/107779893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487406d3f0 0x7f487415ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487359e640 1 -- 172.21.15.60:0/107779893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4874158230 con 0x7f487415ddf0 2024-04-25T10:05:45.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487359e640 1 --2- 172.21.15.60:0/107779893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487415ddf0 0x7f4874143400 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f486c01a530 tx=0x7f486c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f485b7fe640 1 -- 172.21.15.60:0/107779893 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c0b2050 con 0x7f487415ddf0 2024-04-25T10:05:45.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f485b7fe640 1 -- 172.21.15.60:0/107779893 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c0be070 con 0x7f487415ddf0 2024-04-25T10:05:45.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487925c640 1 -- 172.21.15.60:0/107779893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487415fb00 con 0x7f487415ddf0 2024-04-25T10:05:45.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f485b7fe640 1 -- 172.21.15.60:0/107779893 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c0adbd0 con 0x7f487415ddf0 2024-04-25T10:05:45.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.067+0000 7f487925c640 1 -- 172.21.15.60:0/107779893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4874143940 con 0x7f487415ddf0 2024-04-25T10:05:45.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.071+0000 7f487925c640 1 -- 172.21.15.60:0/107779893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4874074770 con 0x7f487415ddf0 2024-04-25T10:05:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.071+0000 7f485b7fe640 1 -- 172.21.15.60:0/107779893 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f486c0b9450 con 0x7f487415ddf0 2024-04-25T10:05:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.071+0000 7f485b7fe640 1 --2- 172.21.15.60:0/107779893 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4860066f90 0x7f4860069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.071+0000 7f485b7fe640 1 -- 172.21.15.60:0/107779893 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f486c1208f0 con 0x7f487415ddf0 2024-04-25T10:05:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.071+0000 7f4872d9d640 1 --2- 172.21.15.60:0/107779893 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4860066f90 0x7f4860069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.071+0000 7f4872d9d640 1 --2- 172.21.15.60:0/107779893 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4860066f90 0x7f4860069450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4874158880 tx=0x7f4864009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.075+0000 7f485b7fe640 1 -- 172.21.15.60:0/107779893 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f486c016520 con 0x7f487415ddf0 2024-04-25T10:05:45.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.275+0000 7f487925c640 1 -- 172.21.15.60:0/107779893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f487415cb60 con 0x7f487415ddf0 2024-04-25T10:05:45.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.275+0000 7f485b7fe640 1 -- 172.21.15.60:0/107779893 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f486c0f1a70 con 0x7f487415ddf0 2024-04-25T10:05:45.283 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:05:45.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.279+0000 7f487925c640 1 -- 172.21.15.60:0/107779893 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4860066f90 msgr2=0x7f4860069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.279+0000 7f487925c640 1 --2- 172.21.15.60:0/107779893 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4860066f90 0x7f4860069450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4874158880 tx=0x7f4864009040 comp rx=0 tx=0).stop 2024-04-25T10:05:45.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.279+0000 7f487925c640 1 -- 172.21.15.60:0/107779893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487415ddf0 msgr2=0x7f4874143400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.279+0000 7f487925c640 1 --2- 172.21.15.60:0/107779893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487415ddf0 0x7f4874143400 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f486c01a530 tx=0x7f486c0196a0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.279+0000 7f487925c640 1 -- 172.21.15.60:0/107779893 shutdown_connections 2024-04-25T10:05:45.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.279+0000 7f487925c640 1 --2- 172.21.15.60:0/107779893 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4860066f90 0x7f4860069450 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.279+0000 7f487925c640 1 --2- 172.21.15.60:0/107779893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487415ddf0 0x7f4874143400 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.279+0000 7f487925c640 1 --2- 172.21.15.60:0/107779893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4874071560 0x7f4874151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.279+0000 7f487925c640 1 --2- 172.21.15.60:0/107779893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487406d3f0 0x7f487415ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.279+0000 7f487925c640 1 -- 172.21.15.60:0/107779893 >> 172.21.15.60:0/107779893 conn(0x7f48740b3520 msgr2=0x7f487415bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:45.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.279+0000 7f487925c640 1 -- 172.21.15.60:0/107779893 shutdown_connections 2024-04-25T10:05:45.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:45.279+0000 7f487925c640 1 -- 172.21.15.60:0/107779893 wait complete. 2024-04-25T10:05:45.300 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:05:45.301 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:05:45.301 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-04-25T10:05:45.301 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T10:05:45.301 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:05:45.301 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:05:45.301 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:05:45.301 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:05:45.301 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:05:45.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:45.301 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-04-25T10:05:45.308 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:45.308 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:45.308 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:45.309 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:45.309 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:45.309 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:45.309 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:45.309 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:45.309 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-04-25T10:05:45.309 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:45.309 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:45.309 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-04-25T10:05:45.309 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:45.309 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:45.309 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:05:45.309 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:45.309 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:05:45.309 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42507sec preferred_lft 42507sec 2024-04-25T10:05:45.310 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-04-25T10:05:45.310 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:45.310 INFO:teuthology.orchestra.run.smithi100.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:05:45.310 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 02:42:99:83:b8:78 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:45.310 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:05:45.310 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:45.310 INFO:teuthology.orchestra.run.smithi100.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:45.310 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:45.310 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:05:45.310 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:45.310 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:05:45.310 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:45.310 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:05:45.310 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:05:45.311 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:05:45.311 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:45.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:45.311 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:05:45.365 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-25T10:05:45.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:45.368 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-04-25T10:05:45.421 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-04-25T10:05:45.421 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:05:45.421 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:05:45.421 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:05:45.421 INFO:teuthology.orchestra.run.smithi100.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:05:45.422 INFO:teuthology.orchestra.run.smithi100.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:05:45.422 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:05:45.422 INFO:teuthology.orchestra.run.smithi100.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:05:45.422 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:05:45.422 INFO:teuthology.orchestra.run.smithi100.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:05:45.422 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:05:45.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:45.422 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T10:05:45.422 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T10:05:45.422 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:05:45.423 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:05:45.423 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:05:45.423 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T10:05:45.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:45.503 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-25T10:05:45.552 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:05:45.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:45.552 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-04-25T10:05:45.604 INFO:teuthology.orchestra.run.smithi100.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T10:05:45.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:45.605 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T10:05:45.605 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T10:05:45.605 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:05:45.605 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T10:05:45.605 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T10:05:45.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:45.671 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:05:45.725 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:45.726 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:45.726 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:45.726 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:45.726 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:45.726 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:45.726 INFO:teuthology.orchestra.run.smithi100.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:45.726 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether f6:cb:ac:ab:76:88 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:05:45.726 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:05:45.726 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:45.726 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::f4cb:acff:feab:7688/64 scope link 2024-04-25T10:05:45.726 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:45.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:45.727 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:05:45.783 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:45.783 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:45.783 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:45.783 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:45.783 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:45.783 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:45.783 INFO:teuthology.orchestra.run.smithi100.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:45.783 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether f6:cb:ac:ab:76:88 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:05:45.783 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:05:45.783 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:45.784 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::f4cb:acff:feab:7688/64 scope link 2024-04-25T10:05:45.784 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:45.784 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T10:05:45.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:45.785 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T10:05:45.785 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T10:05:45.785 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T10:05:45.785 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:05:45.785 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T10:05:45.785 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T10:05:45.785 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T10:05:45.785 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T10:05:45.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:45.879 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T10:05:45.879 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T10:05:45.879 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.1 up 2024-04-25T10:05:45.879 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:05:45.879 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T10:05:45.900 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-25T10:05:45.901 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:45.901 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T10:05:45.952 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:05:45.953 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:45.953 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:05:46.005 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-25T10:05:46.061 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:46.109 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-25T10:05:46.109 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-25T10:05:46.109 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-25T10:05:46.109 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-25T10:05:46.109 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-25T10:05:46.109 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-25T10:05:46.109 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-25T10:05:46.110 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-25T10:05:46.110 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-25T10:05:46.110 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-25T10:05:46.110 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-25T10:05:46.110 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-25T10:05:46.110 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:46.110 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-25T10:05:46.110 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-25T10:05:46.110 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-25T10:05:46.110 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-25T10:05:46.110 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-25T10:05:46.110 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:46.110 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-25T10:05:46.110 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:46.111 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-25T10:05:46.111 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-25T10:05:46.111 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-25T10:05:46.111 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:05:46.111 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:05:46.111 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:05:46.111 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:05:46.111 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-25T10:05:46.111 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:05:46.111 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:05:46.111 INFO:teuthology.orchestra.run.smithi100.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T10:05:46.111 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:05:46.111 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:05:46.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.112 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-25T10:05:46.160 INFO:teuthology.orchestra.run.smithi100.stdout:45 2024-04-25T10:05:46.161 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T10:05:46.161 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-25T10:05:46.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:46.206 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2024-04-25T10:05:46.206 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:46.221 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:46.281 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi100.stderr:2024-04-25T10:05:46.272+0000 7f6bdf14a580 -1 init, newargv = 0x55d80dedae40 newargc=15 2024-04-25T10:05:46.281 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi100.stderr:ceph-fuse[33337]: starting ceph client 2024-04-25T10:05:46.284 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi100.stdout:/run/lock 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/fs/nfsd 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi100.stdout:/run/rpc_pipefs 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2024-04-25T10:05:46.287 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:05:46.287 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:05:46.287 INFO:teuthology.orchestra.run.smithi100.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T10:05:46.287 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:05:46.287 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:05:46.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.288 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2024-04-25T10:05:46.288 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi100.stderr:ceph-fuse[33337]: starting fuse 2024-04-25T10:05:46.335 INFO:teuthology.orchestra.run.smithi100.stdout:45 2024-04-25T10:05:46.336 INFO:teuthology.orchestra.run.smithi100.stdout:46 2024-04-25T10:05:46.336 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T10:05:46.336 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:05:46.336 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2024-04-25T10:05:46.336 DEBUG:teuthology.orchestra.run.smithi100:> import re 2024-04-25T10:05:46.336 DEBUG:teuthology.orchestra.run.smithi100:> import os 2024-04-25T10:05:46.336 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2024-04-25T10:05:46.336 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-25T10:05:46.336 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi100:> return f 2024-04-25T10:05:46.338 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:05:46.338 DEBUG:teuthology.orchestra.run.smithi100:> 2024-04-25T10:05:46.338 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.3")) 2024-04-25T10:05:46.338 DEBUG:teuthology.orchestra.run.smithi100:> ' 2024-04-25T10:05:46.457 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.3.33337.asok 2024-04-25T10:05:46.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T10:05:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:05:46.468 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33337.asok 2024-04-25T10:05:46.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.468 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33337.asok status 2024-04-25T10:05:46.617 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-04-25T10:05:46.618 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2024-04-25T10:05:46.618 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:05:46.618 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:05:46.618 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "3", 2024-04-25T10:05:46.618 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2024-04-25T10:05:46.618 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-25T10:05:46.618 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "33337", 2024-04-25T10:05:46.618 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2024-04-25T10:05:46.618 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-25T10:05:46.618 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2024-04-25T10:05:46.618 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2024-04-25T10:05:46.618 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4642, 2024-04-25T10:05:46.618 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2024-04-25T10:05:46.618 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 4642 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.2:0", 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3483287216 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.2:0", 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3483287216 2024-04-25T10:05:46.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.4642 192.168.0.2:0/3483287216", 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.2:0/3483287216", 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 8, 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 22, 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2024-04-25T10:05:46.620 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-04-25T10:05:46.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.631 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:05:46.631 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:46.639 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:05:46.640 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:46.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.640 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:46.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.700 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:05:46.700 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-25T10:05:46.750 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:05:46.750 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-25T10:05:46.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.751 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:05:46.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.811 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-04-25T10:05:46.811 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T10:05:46.818 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-04-25T10:05:46.818 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T10:05:46.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.819 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:05:46.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.877 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2024-04-25T10:05:46.877 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-25T10:05:46.924 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2024-04-25T10:05:46.925 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-25T10:05:46.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.925 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:05:46.980 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T10:05:47.000 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:47.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16c104e640 1 Processor -- start 2024-04-25T10:05:47.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16c104e640 1 -- start start 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16c104e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16bc160eb0 0x7f16bc0b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16c104e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f16bc0b9730 0x7f16bc0b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16c104e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16bc0ce9c0 con 0x7f16bc161290 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16c104e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16bc0ceca0 con 0x7f16bc0b9730 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16c104e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc0d2490 con 0x7f16bc160eb0 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16ba59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f16bc161290 0x7f16bc0b6a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43874/0 (socket says 172.21.15.60:43874) 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16bb59e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f16bc0b9730 0x7f16bc0b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16ba59c640 1 -- 172.21.15.60:0/1653645960 learned_addr learned my addr 172.21.15.60:0/1653645960 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16b9d9b640 1 -- 172.21.15.60:0/1653645960 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029119573 0 0) 0x7f16bc0ce9c0 con 0x7f16bc161290 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16bad9d640 1 --2- 172.21.15.60:0/1653645960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16bc160eb0 0x7f16bc0b91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16b9d9b640 1 -- 172.21.15.60:0/1653645960 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16a8003530 con 0x7f16bc161290 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16bb59e640 1 -- 172.21.15.60:0/1653645960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16bc160eb0 msgr2=0x7f16bc0b91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16bb59e640 1 --2- 172.21.15.60:0/1653645960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16bc160eb0 0x7f16bc0b91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16bb59e640 1 -- 172.21.15.60:0/1653645960 >> v1:172.21.15.60:6789/0 conn(0x7f16bc161290 legacy=0x7f16bc0b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.127+0000 7f16bb59e640 1 -- 172.21.15.60:0/1653645960 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc0d2770 con 0x7f16bc0b9730 2024-04-25T10:05:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16bb59e640 1 --2- 172.21.15.60:0/1653645960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16bc0b9730 0x7f16bc0b53f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009d90 tx=0x7f16b401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b48834dde28325e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16b9d9b640 1 -- 172.21.15.60:0/1653645960 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b40b4070 con 0x7f16bc0b9730 2024-04-25T10:05:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16b9d9b640 1 -- 172.21.15.60:0/1653645960 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b40af600 con 0x7f16bc0b9730 2024-04-25T10:05:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16b9d9b640 1 -- 172.21.15.60:0/1653645960 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b40af9d0 con 0x7f16bc0b9730 2024-04-25T10:05:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16c104e640 1 -- 172.21.15.60:0/1653645960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16bc0b9730 msgr2=0x7f16bc0b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16c104e640 1 --2- 172.21.15.60:0/1653645960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16bc0b9730 0x7f16bc0b53f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009d90 tx=0x7f16b401f150 comp rx=0 tx=0).stop 2024-04-25T10:05:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16c104e640 1 -- 172.21.15.60:0/1653645960 shutdown_connections 2024-04-25T10:05:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16c104e640 1 --2- 172.21.15.60:0/1653645960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16bc0b9730 0x7f16bc0b53f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16c104e640 1 --2- 172.21.15.60:0/1653645960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16bc160eb0 0x7f16bc0b91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16c104e640 1 -- 172.21.15.60:0/1653645960 >> 172.21.15.60:0/1653645960 conn(0x7f16bc0d05c0 msgr2=0x7f16bc0d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16c104e640 1 -- 172.21.15.60:0/1653645960 shutdown_connections 2024-04-25T10:05:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16c104e640 1 -- 172.21.15.60:0/1653645960 wait complete. 2024-04-25T10:05:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16c104e640 1 Processor -- start 2024-04-25T10:05:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16c104e640 1 -- start start 2024-04-25T10:05:47.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16c104e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16bc0b9730 0x7f16bc0cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16c104e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16bc160eb0 0x7f16bc1f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16c104e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16bc161290 0x7f16bc1f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16c104e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc0cef80 con 0x7f16bc160eb0 2024-04-25T10:05:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16c104e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc0b7180 con 0x7f16bc0b9730 2024-04-25T10:05:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.131+0000 7f16c104e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc0b5930 con 0x7f16bc161290 2024-04-25T10:05:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16ba59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16bc160eb0 0x7f16bc1f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16ba59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16bc160eb0 0x7f16bc1f6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46920/0 (socket says 172.21.15.60:46920) 2024-04-25T10:05:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16ba59c640 1 -- 172.21.15.60:0/3916667697 learned_addr learned my addr 172.21.15.60:0/3916667697 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16bad9d640 1 --2- 172.21.15.60:0/3916667697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16bc0b9730 0x7f16bc0cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16bb59e640 1 --2- 172.21.15.60:0/3916667697 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16bc161290 0x7f16bc1f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16ba59c640 1 -- 172.21.15.60:0/3916667697 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16bc161290 msgr2=0x7f16bc1f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16ba59c640 1 --2- 172.21.15.60:0/3916667697 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16bc161290 0x7f16bc1f7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16ba59c640 1 -- 172.21.15.60:0/3916667697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16bc0b9730 msgr2=0x7f16bc0cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16ba59c640 1 --2- 172.21.15.60:0/3916667697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16bc0b9730 0x7f16bc0cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16ba59c640 1 -- 172.21.15.60:0/3916667697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc0cc390 con 0x7f16bc160eb0 2024-04-25T10:05:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16bad9d640 1 --2- 172.21.15.60:0/3916667697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16bc0b9730 0x7f16bc0cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16ba59c640 1 --2- 172.21.15.60:0/3916667697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16bc160eb0 0x7f16bc1f6be0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00c3c0 tx=0x7f16ac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16b37fe640 1 -- 172.21.15.60:0/3916667697 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac017070 con 0x7f16bc160eb0 2024-04-25T10:05:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16b37fe640 1 -- 172.21.15.60:0/3916667697 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ac005cf0 con 0x7f16bc160eb0 2024-04-25T10:05:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16b37fe640 1 -- 172.21.15.60:0/3916667697 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac0072f0 con 0x7f16bc160eb0 2024-04-25T10:05:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16c104e640 1 -- 172.21.15.60:0/3916667697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16bc0cc670 con 0x7f16bc160eb0 2024-04-25T10:05:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16c104e640 1 -- 172.21.15.60:0/3916667697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc0ccc30 con 0x7f16bc160eb0 2024-04-25T10:05:47.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16c104e640 1 -- 172.21.15.60:0/3916667697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16bc074770 con 0x7f16bc160eb0 2024-04-25T10:05:47.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16b37fe640 1 -- 172.21.15.60:0/3916667697 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f16ac01a070 con 0x7f16bc160eb0 2024-04-25T10:05:47.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16b37fe640 1 --2- 172.21.15.60:0/3916667697 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f169c066f60 0x7f169c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16b37fe640 1 -- 172.21.15.60:0/3916667697 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16ac084360 con 0x7f16bc160eb0 2024-04-25T10:05:47.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.135+0000 7f16bad9d640 1 --2- 172.21.15.60:0/3916667697 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f169c066f60 0x7f169c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.139+0000 7f16bad9d640 1 --2- 172.21.15.60:0/3916667697 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f169c066f60 0x7f169c069420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0b7a30 tx=0x7f16a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.139+0000 7f16b37fe640 1 -- 172.21.15.60:0/3916667697 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16ac055400 con 0x7f16bc160eb0 2024-04-25T10:05:47.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.303+0000 7f16c104e640 1 -- 172.21.15.60:0/3916667697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16bc0d15b0 con 0x7f16bc160eb0 2024-04-25T10:05:47.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.303+0000 7f16b37fe640 1 -- 172.21.15.60:0/3916667697 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f16ac054d00 con 0x7f16bc160eb0 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:22.036582+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:47.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.307+0000 7f16c104e640 1 -- 172.21.15.60:0/3916667697 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f169c066f60 msgr2=0x7f169c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.307+0000 7f16c104e640 1 --2- 172.21.15.60:0/3916667697 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f169c066f60 0x7f169c069420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0b7a30 tx=0x7f16a4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:47.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.307+0000 7f16c104e640 1 -- 172.21.15.60:0/3916667697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16bc160eb0 msgr2=0x7f16bc1f6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.307+0000 7f16c104e640 1 --2- 172.21.15.60:0/3916667697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16bc160eb0 0x7f16bc1f6be0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00c3c0 tx=0x7f16ac00c880 comp rx=0 tx=0).stop 2024-04-25T10:05:47.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.307+0000 7f16c104e640 1 -- 172.21.15.60:0/3916667697 shutdown_connections 2024-04-25T10:05:47.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.307+0000 7f16c104e640 1 --2- 172.21.15.60:0/3916667697 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f169c066f60 0x7f169c069420 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.307+0000 7f16c104e640 1 --2- 172.21.15.60:0/3916667697 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16bc161290 0x7f16bc1f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.307+0000 7f16c104e640 1 --2- 172.21.15.60:0/3916667697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16bc160eb0 0x7f16bc1f6be0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.307+0000 7f16c104e640 1 --2- 172.21.15.60:0/3916667697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16bc0b9730 0x7f16bc0cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.307+0000 7f16c104e640 1 -- 172.21.15.60:0/3916667697 >> 172.21.15.60:0/3916667697 conn(0x7f16bc0d05c0 msgr2=0x7f16bc160110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:47.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.307+0000 7f16c104e640 1 -- 172.21.15.60:0/3916667697 shutdown_connections 2024-04-25T10:05:47.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.307+0000 7f16c104e640 1 -- 172.21.15.60:0/3916667697 wait complete. 2024-04-25T10:05:47.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:18.976643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:19.405302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:47.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:47.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd514782640 1 Processor -- start 2024-04-25T10:05:47.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd514782640 1 -- start start 2024-04-25T10:05:47.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd514782640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd50c15d9c0 0x7fd50c0d1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd514782640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd50c0b6970 0x7fd50c0b91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd514782640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd50c0b6f30 con 0x7fd50c0ce5b0 2024-04-25T10:05:47.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd514782640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50c0d12d0 con 0x7fd50c15d9c0 2024-04-25T10:05:47.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd514782640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0b7210 con 0x7fd50c0b6970 2024-04-25T10:05:47.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd512d1f640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd50c0ce5b0 0x7fd50c0b7fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43882/0 (socket says 172.21.15.60:43882) 2024-04-25T10:05:47.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd512d1f640 1 -- 172.21.15.60:0/553332048 learned_addr learned my addr 172.21.15.60:0/553332048 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:47.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd511d1d640 1 --2- 172.21.15.60:0/553332048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd50c0b6970 0x7fd50c0b91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51251e640 1 --2- 172.21.15.60:0/553332048 >> v2:172.21.15.100:3300/0 conn(0x7fd50c15d9c0 0x7fd50c0d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51151c640 1 -- 172.21.15.60:0/553332048 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225652373 0 0) 0x7fd50c0b6f30 con 0x7fd50c0ce5b0 2024-04-25T10:05:47.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51151c640 1 -- 172.21.15.60:0/553332048 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4f8003530 con 0x7fd50c0ce5b0 2024-04-25T10:05:47.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51151c640 1 -- 172.21.15.60:0/553332048 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 65359762 0 0) 0x7fd4f8003530 con 0x7fd50c0ce5b0 2024-04-25T10:05:47.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51151c640 1 -- 172.21.15.60:0/553332048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd50c0b6970 msgr2=0x7fd50c0b91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:47.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51151c640 1 --2- 172.21.15.60:0/553332048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd50c0b6970 0x7fd50c0b91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51151c640 1 -- 172.21.15.60:0/553332048 >> v2:172.21.15.100:3300/0 conn(0x7fd50c15d9c0 msgr2=0x7fd50c0d1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:47.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51151c640 1 --2- 172.21.15.60:0/553332048 >> v2:172.21.15.100:3300/0 conn(0x7fd50c15d9c0 0x7fd50c0d1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51151c640 1 -- 172.21.15.60:0/553332048 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c0b74f0 con 0x7fd50c0ce5b0 2024-04-25T10:05:47.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51151c640 1 -- 172.21.15.60:0/553332048 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fd504001a20 con 0x7fd50c0ce5b0 2024-04-25T10:05:47.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51151c640 1 -- 172.21.15.60:0/553332048 >> v1:172.21.15.60:6789/0 conn(0x7fd50c0ce5b0 legacy=0x7fd50c0b7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51151c640 1 --2- 172.21.15.60:0/553332048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f80039c0 0x7fd4f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51151c640 1 --2- 172.21.15.60:0/553332048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8004810 0x7fd4f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51151c640 1 --2- 172.21.15.60:0/553332048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd50c0b6970 0x7fd4f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51151c640 1 -- 172.21.15.60:0/553332048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0d12d0 con 0x7fd4f8004810 2024-04-25T10:05:47.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51151c640 1 -- 172.21.15.60:0/553332048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0b7210 con 0x7fd4f80039c0 2024-04-25T10:05:47.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51151c640 1 -- 172.21.15.60:0/553332048 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f800b9d0 con 0x7fd50c0b6970 2024-04-25T10:05:47.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd511d1d640 1 --2- 172.21.15.60:0/553332048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8004810 0x7fd4f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd51251e640 1 --2- 172.21.15.60:0/553332048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f80039c0 0x7fd4f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd512d1f640 1 --2- 172.21.15.60:0/553332048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd50c0b6970 0x7fd4f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd511d1d640 1 -- 172.21.15.60:0/553332048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd50c0b6970 msgr2=0x7fd4f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:47.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd511d1d640 1 --2- 172.21.15.60:0/553332048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd50c0b6970 0x7fd4f80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd511d1d640 1 -- 172.21.15.60:0/553332048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f80039c0 msgr2=0x7fd4f8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:47.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd511d1d640 1 --2- 172.21.15.60:0/553332048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f80039c0 0x7fd4f8003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd511d1d640 1 -- 172.21.15.60:0/553332048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f8003530 con 0x7fd4f8004810 2024-04-25T10:05:47.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.443+0000 7fd512d1f640 1 --2- 172.21.15.60:0/553332048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd50c0b6970 0x7fd4f80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:47.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.447+0000 7fd511d1d640 1 --2- 172.21.15.60:0/553332048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8004810 0x7fd4f8004c90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd508009ae0 tx=0x7fd50801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557c70ece3c349cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.447+0000 7fd51151c640 1 -- 172.21.15.60:0/553332048 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5080b2070 con 0x7fd4f8004810 2024-04-25T10:05:47.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.447+0000 7fd51151c640 1 -- 172.21.15.60:0/553332048 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5080a5580 con 0x7fd4f8004810 2024-04-25T10:05:47.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.447+0000 7fd51151c640 1 -- 172.21.15.60:0/553332048 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5080096b0 con 0x7fd4f8004810 2024-04-25T10:05:47.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.447+0000 7fd514782640 1 -- 172.21.15.60:0/553332048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8004810 msgr2=0x7fd4f8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.447+0000 7fd514782640 1 --2- 172.21.15.60:0/553332048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8004810 0x7fd4f8004c90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd508009ae0 tx=0x7fd50801ca70 comp rx=0 tx=0).stop 2024-04-25T10:05:47.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.447+0000 7fd512d1f640 1 -- 172.21.15.60:0/553332048 reap_dead start 2024-04-25T10:05:47.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.447+0000 7fd514782640 1 -- 172.21.15.60:0/553332048 shutdown_connections 2024-04-25T10:05:47.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.447+0000 7fd514782640 1 -- 172.21.15.60:0/553332048 >> 172.21.15.60:0/553332048 conn(0x7fd50c072710 msgr2=0x7fd50c072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:47.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.447+0000 7fd514782640 1 -- 172.21.15.60:0/553332048 shutdown_connections 2024-04-25T10:05:47.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.447+0000 7fd514782640 1 -- 172.21.15.60:0/553332048 wait complete. 2024-04-25T10:05:47.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.447+0000 7fd514782640 1 Processor -- start 2024-04-25T10:05:47.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.447+0000 7fd514782640 1 -- start start 2024-04-25T10:05:47.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd514782640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8004810 0x7fd50c1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd514782640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd50c14de10 0x7fd50c157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd514782640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd50c158080 0x7fd50c1f3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd514782640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c15dda0 con 0x7fd4f8004810 2024-04-25T10:05:47.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd514782640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0b7ab0 con 0x7fd50c14de10 2024-04-25T10:05:47.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd514782640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0cea20 con 0x7fd50c158080 2024-04-25T10:05:47.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd51251e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8004810 0x7fd50c1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd51251e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8004810 0x7fd50c1597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46942/0 (socket says 172.21.15.60:46942) 2024-04-25T10:05:47.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd51251e640 1 -- 172.21.15.60:0/3408074595 learned_addr learned my addr 172.21.15.60:0/3408074595 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:47.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd512d1f640 1 --2- 172.21.15.60:0/3408074595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd50c158080 0x7fd50c1f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd511d1d640 1 --2- 172.21.15.60:0/3408074595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd50c14de10 0x7fd50c157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd51251e640 1 -- 172.21.15.60:0/3408074595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd50c158080 msgr2=0x7fd50c1f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd51251e640 1 --2- 172.21.15.60:0/3408074595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd50c158080 0x7fd50c1f3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd51251e640 1 -- 172.21.15.60:0/3408074595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd50c14de10 msgr2=0x7fd50c157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd51251e640 1 --2- 172.21.15.60:0/3408074595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd50c14de10 0x7fd50c157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd51251e640 1 -- 172.21.15.60:0/3408074595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c158750 con 0x7fd4f8004810 2024-04-25T10:05:47.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd511d1d640 1 --2- 172.21.15.60:0/3408074595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd50c14de10 0x7fd50c157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:47.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd512d1f640 1 --2- 172.21.15.60:0/3408074595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd50c158080 0x7fd50c1f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:47.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd51251e640 1 --2- 172.21.15.60:0/3408074595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8004810 0x7fd50c1597f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc010930 tx=0x7fd4fc010df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd502ffd640 1 -- 172.21.15.60:0/3408074595 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc00f040 con 0x7fd4f8004810 2024-04-25T10:05:47.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd502ffd640 1 -- 172.21.15.60:0/3408074595 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc00ec20 con 0x7fd4f8004810 2024-04-25T10:05:47.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd514782640 1 -- 172.21.15.60:0/3408074595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd50c158a30 con 0x7fd4f8004810 2024-04-25T10:05:47.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd502ffd640 1 -- 172.21.15.60:0/3408074595 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc01c400 con 0x7fd4f8004810 2024-04-25T10:05:47.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd514782640 1 -- 172.21.15.60:0/3408074595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50c158ff0 con 0x7fd4f8004810 2024-04-25T10:05:47.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd514782640 1 -- 172.21.15.60:0/3408074595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50c157320 con 0x7fd4f8004810 2024-04-25T10:05:47.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd502ffd640 1 -- 172.21.15.60:0/3408074595 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd4fc01c5a0 con 0x7fd4f8004810 2024-04-25T10:05:47.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd502ffd640 1 --2- 172.21.15.60:0/3408074595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd4f0066f60 0x7fd4f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd502ffd640 1 -- 172.21.15.60:0/3408074595 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4fc085740 con 0x7fd4f8004810 2024-04-25T10:05:47.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.451+0000 7fd511d1d640 1 --2- 172.21.15.60:0/3408074595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd4f0066f60 0x7fd4f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.455+0000 7fd511d1d640 1 --2- 172.21.15.60:0/3408074595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd4f0066f60 0x7fd4f0069420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd5080af000 tx=0x7fd5080046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.455+0000 7fd502ffd640 1 -- 172.21.15.60:0/3408074595 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd4fc018030 con 0x7fd4f8004810 2024-04-25T10:05:47.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.615+0000 7fd514782640 1 -- 172.21.15.60:0/3408074595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd50c074770 con 0x7fd4f8004810 2024-04-25T10:05:47.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.615+0000 7fd502ffd640 1 -- 172.21.15.60:0/3408074595 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4fc056810 con 0x7fd4f8004810 2024-04-25T10:05:47.621 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:05:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.615+0000 7fd514782640 1 -- 172.21.15.60:0/3408074595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd4f0066f60 msgr2=0x7fd4f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.615+0000 7fd514782640 1 --2- 172.21.15.60:0/3408074595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd4f0066f60 0x7fd4f0069420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd5080af000 tx=0x7fd5080046d0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.615+0000 7fd514782640 1 -- 172.21.15.60:0/3408074595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8004810 msgr2=0x7fd50c1597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.615+0000 7fd514782640 1 --2- 172.21.15.60:0/3408074595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8004810 0x7fd50c1597f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc010930 tx=0x7fd4fc010df0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.615+0000 7fd514782640 1 -- 172.21.15.60:0/3408074595 shutdown_connections 2024-04-25T10:05:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.615+0000 7fd514782640 1 --2- 172.21.15.60:0/3408074595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd4f0066f60 0x7fd4f0069420 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.615+0000 7fd514782640 1 --2- 172.21.15.60:0/3408074595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd50c158080 0x7fd50c1f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.615+0000 7fd514782640 1 --2- 172.21.15.60:0/3408074595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd50c14de10 0x7fd50c157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.615+0000 7fd514782640 1 --2- 172.21.15.60:0/3408074595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f8004810 0x7fd50c1597f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.615+0000 7fd514782640 1 -- 172.21.15.60:0/3408074595 >> 172.21.15.60:0/3408074595 conn(0x7fd50c072710 msgr2=0x7fd50c0b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:47.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.615+0000 7fd514782640 1 -- 172.21.15.60:0/3408074595 shutdown_connections 2024-04-25T10:05:47.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.615+0000 7fd514782640 1 -- 172.21.15.60:0/3408074595 wait complete. 2024-04-25T10:05:47.640 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:47.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.751+0000 7ff9ab9eb640 1 Processor -- start 2024-04-25T10:05:47.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.751+0000 7ff9ab9eb640 1 -- start start 2024-04-25T10:05:47.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.751+0000 7ff9ab9eb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff9a4099a30 0x7ff9a4099ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.751+0000 7ff9ab9eb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a40cc320 con 0x7ff9a40b2560 2024-04-25T10:05:47.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.751+0000 7ff9ab9eb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a40cc5c0 con 0x7ff9a415ef20 2024-04-25T10:05:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.751+0000 7ff9ab9eb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a40d07b0 con 0x7ff9a4099a30 2024-04-25T10:05:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.751+0000 7ff9a8f86640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff9a40b2560 0x7ff9a40b4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43890/0 (socket says 172.21.15.60:43890) 2024-04-25T10:05:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.751+0000 7ff9a8f86640 1 -- 172.21.15.60:0/1836533121 learned_addr learned my addr 172.21.15.60:0/1836533121 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.751+0000 7ff9a9f88640 1 --2- 172.21.15.60:0/1836533121 >> v2:172.21.15.60:3300/0 conn(0x7ff9a4099a30 0x7ff9a4099ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.751+0000 7ff99bfff640 1 -- 172.21.15.60:0/1836533121 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871128700 0 0) 0x7ff9a40cc320 con 0x7ff9a40b2560 2024-04-25T10:05:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.751+0000 7ff99bfff640 1 -- 172.21.15.60:0/1836533121 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff990003530 con 0x7ff9a40b2560 2024-04-25T10:05:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.751+0000 7ff9a9f88640 1 -- 172.21.15.60:0/1836533121 >> v1:172.21.15.100:6789/0 conn(0x7ff9a415ef20 legacy=0x7ff9a40cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:47.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.751+0000 7ff9a9f88640 1 -- 172.21.15.60:0/1836533121 >> v1:172.21.15.60:6789/0 conn(0x7ff9a40b2560 legacy=0x7ff9a40b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.751+0000 7ff9a9f88640 1 -- 172.21.15.60:0/1836533121 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a40d04d0 con 0x7ff9a4099a30 2024-04-25T10:05:47.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9a9f88640 1 --2- 172.21.15.60:0/1836533121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a4099a30 0x7ff9a4099ec0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009d90 tx=0x7ff99c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc82bddb64f60919 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff99bfff640 1 -- 172.21.15.60:0/1836533121 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0b2070 con 0x7ff9a4099a30 2024-04-25T10:05:47.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff99bfff640 1 -- 172.21.15.60:0/1836533121 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c0a5830 con 0x7ff9a4099a30 2024-04-25T10:05:47.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff99bfff640 1 -- 172.21.15.60:0/1836533121 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0a5c00 con 0x7ff9a4099a30 2024-04-25T10:05:47.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9ab9eb640 1 -- 172.21.15.60:0/1836533121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a4099a30 msgr2=0x7ff9a4099ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9ab9eb640 1 --2- 172.21.15.60:0/1836533121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a4099a30 0x7ff9a4099ec0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009d90 tx=0x7ff99c01c970 comp rx=0 tx=0).stop 2024-04-25T10:05:47.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9ab9eb640 1 -- 172.21.15.60:0/1836533121 shutdown_connections 2024-04-25T10:05:47.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9ab9eb640 1 --2- 172.21.15.60:0/1836533121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a4099a30 0x7ff9a4099ec0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9ab9eb640 1 -- 172.21.15.60:0/1836533121 >> 172.21.15.60:0/1836533121 conn(0x7ff9a40b3200 msgr2=0x7ff9a40b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:47.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9ab9eb640 1 -- 172.21.15.60:0/1836533121 shutdown_connections 2024-04-25T10:05:47.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9ab9eb640 1 -- 172.21.15.60:0/1836533121 wait complete. 2024-04-25T10:05:47.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9ab9eb640 1 Processor -- start 2024-04-25T10:05:47.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9ab9eb640 1 -- start start 2024-04-25T10:05:47.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9ab9eb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a4099a30 0x7ff9a40cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9ab9eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a40b2560 0x7ff9a4155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9ab9eb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a415ef20 0x7ff9a4155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9ab9eb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a415f510 con 0x7ff9a4099a30 2024-04-25T10:05:47.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9ab9eb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a40b2960 con 0x7ff9a40b2560 2024-04-25T10:05:47.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9ab9eb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a40cd870 con 0x7ff9a415ef20 2024-04-25T10:05:47.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9a9787640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a4099a30 0x7ff9a40cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9a9787640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a4099a30 0x7ff9a40cac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46954/0 (socket says 172.21.15.60:46954) 2024-04-25T10:05:47.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9a9787640 1 -- 172.21.15.60:0/223003948 learned_addr learned my addr 172.21.15.60:0/223003948 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:47.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9a8f86640 1 --2- 172.21.15.60:0/223003948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a40b2560 0x7ff9a4155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9a9f88640 1 --2- 172.21.15.60:0/223003948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a415ef20 0x7ff9a4155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9a9787640 1 -- 172.21.15.60:0/223003948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a415ef20 msgr2=0x7ff9a4155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9a9787640 1 --2- 172.21.15.60:0/223003948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a415ef20 0x7ff9a4155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9a9787640 1 -- 172.21.15.60:0/223003948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a40b2560 msgr2=0x7ff9a4155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9a9787640 1 --2- 172.21.15.60:0/223003948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a40b2560 0x7ff9a4155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9a9787640 1 -- 172.21.15.60:0/223003948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a4155d80 con 0x7ff9a4099a30 2024-04-25T10:05:47.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.755+0000 7ff9a9787640 1 --2- 172.21.15.60:0/223003948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a4099a30 0x7ff9a40cac80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000c3c0 tx=0x7ff9a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.759+0000 7ff999ffb640 1 -- 172.21.15.60:0/223003948 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0017070 con 0x7ff9a4099a30 2024-04-25T10:05:47.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.759+0000 7ff999ffb640 1 -- 172.21.15.60:0/223003948 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a00052d0 con 0x7ff9a4099a30 2024-04-25T10:05:47.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.759+0000 7ff9ab9eb640 1 -- 172.21.15.60:0/223003948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a406d580 con 0x7ff9a4099a30 2024-04-25T10:05:47.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.759+0000 7ff999ffb640 1 -- 172.21.15.60:0/223003948 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0003840 con 0x7ff9a4099a30 2024-04-25T10:05:47.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.759+0000 7ff9ab9eb640 1 -- 172.21.15.60:0/223003948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a40c8da0 con 0x7ff9a4099a30 2024-04-25T10:05:47.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.759+0000 7ff9ab9eb640 1 -- 172.21.15.60:0/223003948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a40c86b0 con 0x7ff9a4099a30 2024-04-25T10:05:47.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.759+0000 7ff999ffb640 1 -- 172.21.15.60:0/223003948 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9a001a070 con 0x7ff9a4099a30 2024-04-25T10:05:47.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.759+0000 7ff999ffb640 1 --2- 172.21.15.60:0/223003948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff988066f60 0x7ff988069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:47.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.759+0000 7ff999ffb640 1 -- 172.21.15.60:0/223003948 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9a00843f0 con 0x7ff9a4099a30 2024-04-25T10:05:47.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.759+0000 7ff9a8f86640 1 --2- 172.21.15.60:0/223003948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff988066f60 0x7ff988069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:47.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.759+0000 7ff9a8f86640 1 --2- 172.21.15.60:0/223003948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff988066f60 0x7ff988069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4167a30 tx=0x7ff994007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:47.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.763+0000 7ff999ffb640 1 -- 172.21.15.60:0/223003948 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff9a0005480 con 0x7ff9a4099a30 2024-04-25T10:05:47.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.915+0000 7ff9ab9eb640 1 -- 172.21.15.60:0/223003948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9a4074770 con 0x7ff9a4099a30 2024-04-25T10:05:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.915+0000 7ff999ffb640 1 -- 172.21.15.60:0/223003948 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9a0054c90 con 0x7ff9a4099a30 2024-04-25T10:05:47.925 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:05:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.919+0000 7ff9ab9eb640 1 -- 172.21.15.60:0/223003948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff988066f60 msgr2=0x7ff988069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.919+0000 7ff9ab9eb640 1 --2- 172.21.15.60:0/223003948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff988066f60 0x7ff988069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4167a30 tx=0x7ff994007d40 comp rx=0 tx=0).stop 2024-04-25T10:05:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.919+0000 7ff9ab9eb640 1 -- 172.21.15.60:0/223003948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a4099a30 msgr2=0x7ff9a40cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.919+0000 7ff9ab9eb640 1 --2- 172.21.15.60:0/223003948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a4099a30 0x7ff9a40cac80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000c3c0 tx=0x7ff9a000c880 comp rx=0 tx=0).stop 2024-04-25T10:05:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.919+0000 7ff9ab9eb640 1 -- 172.21.15.60:0/223003948 shutdown_connections 2024-04-25T10:05:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.919+0000 7ff9ab9eb640 1 --2- 172.21.15.60:0/223003948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff988066f60 0x7ff988069420 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.919+0000 7ff9ab9eb640 1 --2- 172.21.15.60:0/223003948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a415ef20 0x7ff9a4155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.919+0000 7ff9ab9eb640 1 --2- 172.21.15.60:0/223003948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a40b2560 0x7ff9a4155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.919+0000 7ff9ab9eb640 1 --2- 172.21.15.60:0/223003948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a4099a30 0x7ff9a40cac80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.919+0000 7ff9ab9eb640 1 -- 172.21.15.60:0/223003948 >> 172.21.15.60:0/223003948 conn(0x7ff9a40b3200 msgr2=0x7ff9a40cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.919+0000 7ff9ab9eb640 1 -- 172.21.15.60:0/223003948 shutdown_connections 2024-04-25T10:05:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:47.919+0000 7ff9ab9eb640 1 -- 172.21.15.60:0/223003948 wait complete. 2024-04-25T10:05:47.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:48.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40e114d640 1 Processor -- start 2024-04-25T10:05:48.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40e114d640 1 -- start start 2024-04-25T10:05:48.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40e114d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40dc0d0de0 0x7f40dc0d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40e114d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40dc0d1bb0 con 0x7f40dc15dea0 2024-04-25T10:05:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40e114d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40dc0b85e0 con 0x7f40dc0d0960 2024-04-25T10:05:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40e114d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc0b9670 con 0x7f40dc0d0de0 2024-04-25T10:05:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40dad9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f40dc15dea0 0x7f40dc0ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43900/0 (socket says 172.21.15.60:43900) 2024-04-25T10:05:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40dad9d640 1 -- 172.21.15.60:0/3754118967 learned_addr learned my addr 172.21.15.60:0/3754118967 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40db59e640 1 --2- 172.21.15.60:0/3754118967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40dc0d0de0 0x7f40dc0d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40d9d9b640 1 -- 172.21.15.60:0/3754118967 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 723235754 0 0) 0x7f40dc0d1bb0 con 0x7f40dc15dea0 2024-04-25T10:05:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40d9d9b640 1 -- 172.21.15.60:0/3754118967 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40c4003530 con 0x7f40dc15dea0 2024-04-25T10:05:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40d9d9b640 1 -- 172.21.15.60:0/3754118967 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2789846734 0 0) 0x7f40c4003530 con 0x7f40dc15dea0 2024-04-25T10:05:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40d9d9b640 1 -- 172.21.15.60:0/3754118967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40dc0d0de0 msgr2=0x7f40dc0d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40d9d9b640 1 --2- 172.21.15.60:0/3754118967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40dc0d0de0 0x7f40dc0d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40d9d9b640 1 -- 172.21.15.60:0/3754118967 >> v1:172.21.15.100:6789/0 conn(0x7f40dc0d0960 legacy=0x7f40dc0d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40d9d9b640 1 -- 172.21.15.60:0/3754118967 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40dc0b88c0 con 0x7f40dc15dea0 2024-04-25T10:05:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40d9d9b640 1 -- 172.21.15.60:0/3754118967 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f40cc001680 con 0x7f40dc15dea0 2024-04-25T10:05:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40d9d9b640 1 -- 172.21.15.60:0/3754118967 >> v1:172.21.15.60:6789/0 conn(0x7f40dc15dea0 legacy=0x7f40dc0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40d9d9b640 1 --2- 172.21.15.60:0/3754118967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40c40039f0 0x7f40c4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40d9d9b640 1 --2- 172.21.15.60:0/3754118967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40dc0d0de0 0x7f40c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40d9d9b640 1 --2- 172.21.15.60:0/3754118967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40c4004d50 0x7f40c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40d9d9b640 1 -- 172.21.15.60:0/3754118967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc0b9670 con 0x7f40c4004d50 2024-04-25T10:05:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40d9d9b640 1 -- 172.21.15.60:0/3754118967 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c400bab0 con 0x7f40c40039f0 2024-04-25T10:05:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40d9d9b640 1 -- 172.21.15.60:0/3754118967 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c400bcb0 con 0x7f40dc0d0de0 2024-04-25T10:05:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40db59e640 1 --2- 172.21.15.60:0/3754118967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40c4004d50 0x7f40c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40dad9d640 1 --2- 172.21.15.60:0/3754118967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40c40039f0 0x7f40c4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40da59c640 1 --2- 172.21.15.60:0/3754118967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40dc0d0de0 0x7f40c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40db59e640 1 -- 172.21.15.60:0/3754118967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40dc0d0de0 msgr2=0x7f40c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40db59e640 1 --2- 172.21.15.60:0/3754118967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40dc0d0de0 0x7f40c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40db59e640 1 -- 172.21.15.60:0/3754118967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40c40039f0 msgr2=0x7f40c4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40db59e640 1 --2- 172.21.15.60:0/3754118967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40c40039f0 0x7f40c4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40db59e640 1 -- 172.21.15.60:0/3754118967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c4003530 con 0x7f40c4004d50 2024-04-25T10:05:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.067+0000 7f40dad9d640 1 --2- 172.21.15.60:0/3754118967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40c40039f0 0x7f40c4003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.071+0000 7f40db59e640 1 --2- 172.21.15.60:0/3754118967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40c4004d50 0x7f40c40051d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009f70 tx=0x7f40d401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7feafdb239c1788e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.071+0000 7f40d9d9b640 1 -- 172.21.15.60:0/3754118967 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d40b2070 con 0x7f40c4004d50 2024-04-25T10:05:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.071+0000 7f40d9d9b640 1 -- 172.21.15.60:0/3754118967 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d40a5cf0 con 0x7f40c4004d50 2024-04-25T10:05:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.071+0000 7f40d9d9b640 1 -- 172.21.15.60:0/3754118967 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4009bf0 con 0x7f40c4004d50 2024-04-25T10:05:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.071+0000 7f40e114d640 1 -- 172.21.15.60:0/3754118967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40c4004d50 msgr2=0x7f40c40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.071+0000 7f40e114d640 1 --2- 172.21.15.60:0/3754118967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40c4004d50 0x7f40c40051d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009f70 tx=0x7f40d401cc20 comp rx=0 tx=0).stop 2024-04-25T10:05:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.071+0000 7f40db59e640 1 -- 172.21.15.60:0/3754118967 reap_dead start 2024-04-25T10:05:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.071+0000 7f40e114d640 1 -- 172.21.15.60:0/3754118967 shutdown_connections 2024-04-25T10:05:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.071+0000 7f40e114d640 1 -- 172.21.15.60:0/3754118967 >> 172.21.15.60:0/3754118967 conn(0x7f40dc06c4d0 msgr2=0x7f40dc06c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.071+0000 7f40e114d640 1 -- 172.21.15.60:0/3754118967 shutdown_connections 2024-04-25T10:05:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.071+0000 7f40e114d640 1 -- 172.21.15.60:0/3754118967 wait complete. 2024-04-25T10:05:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.071+0000 7f40e114d640 1 Processor -- start 2024-04-25T10:05:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.071+0000 7f40e114d640 1 -- start start 2024-04-25T10:05:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40e114d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40c4004d50 0x7f40dc1559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40e114d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40dc14a720 0x7f40dc1f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40e114d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40dc1f2f70 0x7f40dc1f6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40e114d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc0d1270 con 0x7f40dc14a720 2024-04-25T10:05:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40e114d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc0d1e20 con 0x7f40dc1f2f70 2024-04-25T10:05:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40e114d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc0b8e80 con 0x7f40c4004d50 2024-04-25T10:05:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40da59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40dc14a720 0x7f40dc1f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40da59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40dc14a720 0x7f40dc1f2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46966/0 (socket says 172.21.15.60:46966) 2024-04-25T10:05:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40da59c640 1 -- 172.21.15.60:0/2372490247 learned_addr learned my addr 172.21.15.60:0/2372490247 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40db59e640 1 --2- 172.21.15.60:0/2372490247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40dc1f2f70 0x7f40dc1f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40dad9d640 1 --2- 172.21.15.60:0/2372490247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40c4004d50 0x7f40dc1559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40da59c640 1 -- 172.21.15.60:0/2372490247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40c4004d50 msgr2=0x7f40dc1559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40da59c640 1 --2- 172.21.15.60:0/2372490247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40c4004d50 0x7f40dc1559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40da59c640 1 -- 172.21.15.60:0/2372490247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40dc1f2f70 msgr2=0x7f40dc1f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40da59c640 1 --2- 172.21.15.60:0/2372490247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40dc1f2f70 0x7f40dc1f6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40da59c640 1 -- 172.21.15.60:0/2372490247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40dc156040 con 0x7f40dc14a720 2024-04-25T10:05:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40db59e640 1 --2- 172.21.15.60:0/2372490247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40dc1f2f70 0x7f40dc1f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40dad9d640 1 --2- 172.21.15.60:0/2372490247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40c4004d50 0x7f40dc1559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40da59c640 1 --2- 172.21.15.60:0/2372490247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40dc14a720 0x7f40dc1f2a30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f40d000e550 tx=0x7f40d000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40c37fe640 1 -- 172.21.15.60:0/2372490247 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d00052e0 con 0x7f40dc14a720 2024-04-25T10:05:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40c37fe640 1 -- 172.21.15.60:0/2372490247 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d0018070 con 0x7f40dc14a720 2024-04-25T10:05:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40c37fe640 1 -- 172.21.15.60:0/2372490247 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d0003820 con 0x7f40dc14a720 2024-04-25T10:05:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40e114d640 1 -- 172.21.15.60:0/2372490247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40dc14aba0 con 0x7f40dc14a720 2024-04-25T10:05:48.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40e114d640 1 -- 172.21.15.60:0/2372490247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40dc160170 con 0x7f40dc14a720 2024-04-25T10:05:48.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40e114d640 1 -- 172.21.15.60:0/2372490247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40dc074770 con 0x7f40dc14a720 2024-04-25T10:05:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40c37fe640 1 -- 172.21.15.60:0/2372490247 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40d00039c0 con 0x7f40dc14a720 2024-04-25T10:05:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40c37fe640 1 --2- 172.21.15.60:0/2372490247 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f40c8066f60 0x7f40c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40c37fe640 1 -- 172.21.15.60:0/2372490247 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40d0084d20 con 0x7f40dc14a720 2024-04-25T10:05:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.075+0000 7f40dad9d640 1 --2- 172.21.15.60:0/2372490247 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f40c8066f60 0x7f40c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.079+0000 7f40dad9d640 1 --2- 172.21.15.60:0/2372490247 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f40c8066f60 0x7f40c8069420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f40cc000f70 tx=0x7f40cc000f00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.079+0000 7f40c37fe640 1 -- 172.21.15.60:0/2372490247 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f40dc156040 con 0x7f40dc14a720 2024-04-25T10:05:48.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.243+0000 7f40e114d640 1 -- 172.21.15.60:0/2372490247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f40dc0ca2a0 con 0x7f40dc14a720 2024-04-25T10:05:48.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.243+0000 7f40c37fe640 1 -- 172.21.15.60:0/2372490247 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f40d0055d30 con 0x7f40dc14a720 2024-04-25T10:05:48.249 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:05:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.243+0000 7f40e114d640 1 -- 172.21.15.60:0/2372490247 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f40c8066f60 msgr2=0x7f40c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.243+0000 7f40e114d640 1 --2- 172.21.15.60:0/2372490247 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f40c8066f60 0x7f40c8069420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f40cc000f70 tx=0x7f40cc000f00 comp rx=0 tx=0).stop 2024-04-25T10:05:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.243+0000 7f40e114d640 1 -- 172.21.15.60:0/2372490247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40dc14a720 msgr2=0x7f40dc1f2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.243+0000 7f40e114d640 1 --2- 172.21.15.60:0/2372490247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40dc14a720 0x7f40dc1f2a30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f40d000e550 tx=0x7f40d000ea10 comp rx=0 tx=0).stop 2024-04-25T10:05:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.243+0000 7f40e114d640 1 -- 172.21.15.60:0/2372490247 shutdown_connections 2024-04-25T10:05:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.243+0000 7f40e114d640 1 --2- 172.21.15.60:0/2372490247 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f40c8066f60 0x7f40c8069420 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.243+0000 7f40e114d640 1 --2- 172.21.15.60:0/2372490247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40dc1f2f70 0x7f40dc1f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.243+0000 7f40e114d640 1 --2- 172.21.15.60:0/2372490247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40dc14a720 0x7f40dc1f2a30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.243+0000 7f40e114d640 1 --2- 172.21.15.60:0/2372490247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40c4004d50 0x7f40dc1559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.243+0000 7f40e114d640 1 -- 172.21.15.60:0/2372490247 >> 172.21.15.60:0/2372490247 conn(0x7f40dc06c4d0 msgr2=0x7f40dc15b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.243+0000 7f40e114d640 1 -- 172.21.15.60:0/2372490247 shutdown_connections 2024-04-25T10:05:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.243+0000 7f40e114d640 1 -- 172.21.15.60:0/2372490247 wait complete. 2024-04-25T10:05:48.269 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc3661640 1 Processor -- start 2024-04-25T10:05:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc3661640 1 -- start start 2024-04-25T10:05:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc3661640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4fbc15dc90 0x7f4fbc0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc3661640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fbc071fd0 con 0x7f4fbc0719a0 2024-04-25T10:05:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc3661640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fbc072290 con 0x7f4fbc0b2b10 2024-04-25T10:05:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc3661640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fbc0accc0 con 0x7f4fbc15dc90 2024-04-25T10:05:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc1bfe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4fbc0719a0 0x7f4fbc0ad6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43902/0 (socket says 172.21.15.60:43902) 2024-04-25T10:05:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc1bfe640 1 -- 172.21.15.60:0/4174026501 learned_addr learned my addr 172.21.15.60:0/4174026501 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc13fd640 1 --2- 172.21.15.60:0/4174026501 >> v2:172.21.15.100:3300/0 conn(0x7f4fbc15dc90 0x7f4fbc0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fb3fff640 1 -- 172.21.15.60:0/4174026501 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047363291 0 0) 0x7f4fbc071fd0 con 0x7f4fbc0719a0 2024-04-25T10:05:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fb3fff640 1 -- 172.21.15.60:0/4174026501 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa8003530 con 0x7f4fbc0719a0 2024-04-25T10:05:48.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fb3fff640 1 -- 172.21.15.60:0/4174026501 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2519720837 0 0) 0x7f4fa8003530 con 0x7f4fbc0719a0 2024-04-25T10:05:48.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fb3fff640 1 -- 172.21.15.60:0/4174026501 >> v2:172.21.15.100:3300/0 conn(0x7f4fbc15dc90 msgr2=0x7f4fbc0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fb3fff640 1 --2- 172.21.15.60:0/4174026501 >> v2:172.21.15.100:3300/0 conn(0x7f4fbc15dc90 0x7f4fbc0b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fb3fff640 1 -- 172.21.15.60:0/4174026501 >> v1:172.21.15.100:6789/0 conn(0x7f4fbc0b2b10 legacy=0x7f4fbc071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fb3fff640 1 -- 172.21.15.60:0/4174026501 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc0acef0 con 0x7f4fbc0719a0 2024-04-25T10:05:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fb3fff640 1 -- 172.21.15.60:0/4174026501 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f4fb4001a00 con 0x7f4fbc0719a0 2024-04-25T10:05:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fb3fff640 1 -- 172.21.15.60:0/4174026501 >> v1:172.21.15.60:6789/0 conn(0x7f4fbc0719a0 legacy=0x7f4fbc0ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fb3fff640 1 --2- 172.21.15.60:0/4174026501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa80039c0 0x7f4fa8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fb3fff640 1 --2- 172.21.15.60:0/4174026501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa8004810 0x7f4fa8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fb3fff640 1 --2- 172.21.15.60:0/4174026501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa80051d0 0x7f4fa800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fb3fff640 1 -- 172.21.15.60:0/4174026501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa800c040 con 0x7f4fa80051d0 2024-04-25T10:05:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fb3fff640 1 -- 172.21.15.60:0/4174026501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa800c1f0 con 0x7f4fa8004810 2024-04-25T10:05:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fb3fff640 1 -- 172.21.15.60:0/4174026501 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa800c3f0 con 0x7f4fa80039c0 2024-04-25T10:05:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc1bfe640 1 --2- 172.21.15.60:0/4174026501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa80051d0 0x7f4fa800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc13fd640 1 --2- 172.21.15.60:0/4174026501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa80039c0 0x7f4fa8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc0bfc640 1 --2- 172.21.15.60:0/4174026501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa8004810 0x7f4fa8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc1bfe640 1 -- 172.21.15.60:0/4174026501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa80039c0 msgr2=0x7f4fa8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc1bfe640 1 --2- 172.21.15.60:0/4174026501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa80039c0 0x7f4fa8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc1bfe640 1 -- 172.21.15.60:0/4174026501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa8004810 msgr2=0x7f4fa8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc1bfe640 1 --2- 172.21.15.60:0/4174026501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa8004810 0x7f4fa8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc1bfe640 1 -- 172.21.15.60:0/4174026501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa8003530 con 0x7f4fa80051d0 2024-04-25T10:05:48.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.391+0000 7f4fc13fd640 1 --2- 172.21.15.60:0/4174026501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa80039c0 0x7f4fa8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:48.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc1bfe640 1 -- 172.21.15.60:0/4174026501 reap_dead start 2024-04-25T10:05:48.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc1bfe640 1 --2- 172.21.15.60:0/4174026501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa80051d0 0x7f4fa800b9d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc072c10 tx=0x7f4fb401d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8080984ef352d1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fb3fff640 1 -- 172.21.15.60:0/4174026501 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb400a4a0 con 0x7f4fa80051d0 2024-04-25T10:05:48.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fb3fff640 1 -- 172.21.15.60:0/4174026501 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb40ac640 con 0x7f4fa80051d0 2024-04-25T10:05:48.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fb3fff640 1 -- 172.21.15.60:0/4174026501 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40aca30 con 0x7f4fa80051d0 2024-04-25T10:05:48.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc3661640 1 -- 172.21.15.60:0/4174026501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa80051d0 msgr2=0x7f4fa800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc3661640 1 --2- 172.21.15.60:0/4174026501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa80051d0 0x7f4fa800b9d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc072c10 tx=0x7f4fb401d3a0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc3661640 1 -- 172.21.15.60:0/4174026501 shutdown_connections 2024-04-25T10:05:48.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc3661640 1 --2- 172.21.15.60:0/4174026501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa80051d0 0x7f4fa800b9d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc3661640 1 -- 172.21.15.60:0/4174026501 >> 172.21.15.60:0/4174026501 conn(0x7f4fbc0b12d0 msgr2=0x7f4fbc0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc3661640 1 -- 172.21.15.60:0/4174026501 shutdown_connections 2024-04-25T10:05:48.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc3661640 1 -- 172.21.15.60:0/4174026501 wait complete. 2024-04-25T10:05:48.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc3661640 1 Processor -- start 2024-04-25T10:05:48.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc3661640 1 -- start start 2024-04-25T10:05:48.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc3661640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa80051d0 0x7f4fbc145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc3661640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc152350 0x7f4fbc1585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc3661640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fbc158ae0 0x7f4fbc150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc3661640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0ad260 con 0x7f4fa80051d0 2024-04-25T10:05:48.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc3661640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0b2220 con 0x7f4fbc152350 2024-04-25T10:05:48.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc3661640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0ae7c0 con 0x7f4fbc158ae0 2024-04-25T10:05:48.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc13fd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa80051d0 0x7f4fbc145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc13fd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa80051d0 0x7f4fbc145ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46982/0 (socket says 172.21.15.60:46982) 2024-04-25T10:05:48.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.395+0000 7f4fc13fd640 1 -- 172.21.15.60:0/3633293305 learned_addr learned my addr 172.21.15.60:0/3633293305 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:48.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fc13fd640 1 -- 172.21.15.60:0/3633293305 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fbc158ae0 msgr2=0x7f4fbc150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fc13fd640 1 --2- 172.21.15.60:0/3633293305 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fbc158ae0 0x7f4fbc150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fc13fd640 1 -- 172.21.15.60:0/3633293305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc152350 msgr2=0x7f4fbc1585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fc13fd640 1 --2- 172.21.15.60:0/3633293305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc152350 0x7f4fbc1585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fc13fd640 1 -- 172.21.15.60:0/3633293305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc1512b0 con 0x7f4fa80051d0 2024-04-25T10:05:48.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fc13fd640 1 --2- 172.21.15.60:0/3633293305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa80051d0 0x7f4fbc145ef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8010900 tx=0x7f4fb8010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fb1ffb640 1 -- 172.21.15.60:0/3633293305 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800f040 con 0x7f4fa80051d0 2024-04-25T10:05:48.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fb1ffb640 1 -- 172.21.15.60:0/3633293305 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb800ec20 con 0x7f4fa80051d0 2024-04-25T10:05:48.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fb1ffb640 1 -- 172.21.15.60:0/3633293305 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8004020 con 0x7f4fa80051d0 2024-04-25T10:05:48.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fc3661640 1 -- 172.21.15.60:0/3633293305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fbc151590 con 0x7f4fa80051d0 2024-04-25T10:05:48.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fc3661640 1 -- 172.21.15.60:0/3633293305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fbc151b50 con 0x7f4fa80051d0 2024-04-25T10:05:48.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fc3661640 1 -- 172.21.15.60:0/3633293305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fbc071fd0 con 0x7f4fa80051d0 2024-04-25T10:05:48.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fb1ffb640 1 -- 172.21.15.60:0/3633293305 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4fb80041c0 con 0x7f4fa80051d0 2024-04-25T10:05:48.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fb1ffb640 1 --2- 172.21.15.60:0/3633293305 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fa0066f90 0x7f4fa0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fb1ffb640 1 -- 172.21.15.60:0/3633293305 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fb80857d0 con 0x7f4fa80051d0 2024-04-25T10:05:48.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fc0bfc640 1 --2- 172.21.15.60:0/3633293305 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fa0066f90 0x7f4fa0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.399+0000 7f4fc0bfc640 1 --2- 172.21.15.60:0/3633293305 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fa0066f90 0x7f4fa0069450 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc15fd30 tx=0x7f4fac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.403+0000 7f4fb1ffb640 1 -- 172.21.15.60:0/3633293305 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4fb8018030 con 0x7f4fa80051d0 2024-04-25T10:05:48.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.559+0000 7f4fc3661640 1 -- 172.21.15.60:0/3633293305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fbc158f00 con 0x7f4fa80051d0 2024-04-25T10:05:48.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.563+0000 7f4fb1ffb640 1 -- 172.21.15.60:0/3633293305 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f4fb80567e0 con 0x7f4fa80051d0 2024-04-25T10:05:48.569 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:48.569 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:22.036582+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:48.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.563+0000 7f4fc3661640 1 -- 172.21.15.60:0/3633293305 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fa0066f90 msgr2=0x7f4fa0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.563+0000 7f4fc3661640 1 --2- 172.21.15.60:0/3633293305 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fa0066f90 0x7f4fa0069450 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc15fd30 tx=0x7f4fac009040 comp rx=0 tx=0).stop 2024-04-25T10:05:48.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.563+0000 7f4fc3661640 1 -- 172.21.15.60:0/3633293305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa80051d0 msgr2=0x7f4fbc145ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.563+0000 7f4fc3661640 1 --2- 172.21.15.60:0/3633293305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa80051d0 0x7f4fbc145ef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8010900 tx=0x7f4fb8010dc0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.563+0000 7f4fc3661640 1 -- 172.21.15.60:0/3633293305 shutdown_connections 2024-04-25T10:05:48.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.563+0000 7f4fc3661640 1 --2- 172.21.15.60:0/3633293305 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fa0066f90 0x7f4fa0069450 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.563+0000 7f4fc3661640 1 --2- 172.21.15.60:0/3633293305 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fbc158ae0 0x7f4fbc150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.563+0000 7f4fc3661640 1 --2- 172.21.15.60:0/3633293305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc152350 0x7f4fbc1585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.563+0000 7f4fc3661640 1 --2- 172.21.15.60:0/3633293305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa80051d0 0x7f4fbc145ef0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.563+0000 7f4fc3661640 1 -- 172.21.15.60:0/3633293305 >> 172.21.15.60:0/3633293305 conn(0x7f4fbc0b12d0 msgr2=0x7f4fbc155870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.563+0000 7f4fc3661640 1 -- 172.21.15.60:0/3633293305 shutdown_connections 2024-04-25T10:05:48.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.563+0000 7f4fc3661640 1 -- 172.21.15.60:0/3633293305 wait complete. 2024-04-25T10:05:48.587 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:18.976643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:19.405302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:48.587 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:48.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.695+0000 7f43bf40f640 1 Processor -- start 2024-04-25T10:05:48.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.695+0000 7f43bf40f640 1 -- start start 2024-04-25T10:05:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.695+0000 7f43bf40f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43b80cbcf0 0x7f43b8160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.695+0000 7f43bf40f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f43b8071c30 0x7f43b815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.695+0000 7f43bf40f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b8144a40 con 0x7f43b815fa20 2024-04-25T10:05:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.695+0000 7f43bf40f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b80b2dc0 con 0x7f43b8071c30 2024-04-25T10:05:48.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.695+0000 7f43bf40f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b815fe80 con 0x7f43b80cbcf0 2024-04-25T10:05:48.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43bd9ac640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f43b815fa20 0x7f43b815bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43910/0 (socket says 172.21.15.60:43910) 2024-04-25T10:05:48.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43bd9ac640 1 -- 172.21.15.60:0/3456471153 learned_addr learned my addr 172.21.15.60:0/3456471153 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:48.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 -- 172.21.15.60:0/3456471153 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3468075753 0 0) 0x7f43b8144a40 con 0x7f43b815fa20 2024-04-25T10:05:48.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43bd1ab640 1 --2- 172.21.15.60:0/3456471153 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43b80cbcf0 0x7f43b8160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 -- 172.21.15.60:0/3456471153 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43a4003530 con 0x7f43b815fa20 2024-04-25T10:05:48.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43bc9aa640 1 --2- 172.21.15.60:0/3456471153 >> v2:172.21.15.100:3300/0 conn(0x7f43b8071c30 0x7f43b815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 -- 172.21.15.60:0/3456471153 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 454659608 0 0) 0x7f43a4003530 con 0x7f43b815fa20 2024-04-25T10:05:48.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 -- 172.21.15.60:0/3456471153 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43b80cbcf0 msgr2=0x7f43b8160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 --2- 172.21.15.60:0/3456471153 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43b80cbcf0 0x7f43b8160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 -- 172.21.15.60:0/3456471153 >> v2:172.21.15.100:3300/0 conn(0x7f43b8071c30 msgr2=0x7f43b815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 --2- 172.21.15.60:0/3456471153 >> v2:172.21.15.100:3300/0 conn(0x7f43b8071c30 0x7f43b815f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 -- 172.21.15.60:0/3456471153 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b80cc9f0 con 0x7f43b815fa20 2024-04-25T10:05:48.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 -- 172.21.15.60:0/3456471153 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f43b0001a00 con 0x7f43b815fa20 2024-04-25T10:05:48.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43bc9aa640 1 --2- 172.21.15.60:0/3456471153 >> v2:172.21.15.100:3300/0 conn(0x7f43b8071c30 0x7f43b815f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:48.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 -- 172.21.15.60:0/3456471153 >> v1:172.21.15.60:6789/0 conn(0x7f43b815fa20 legacy=0x7f43b815bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 --2- 172.21.15.60:0/3456471153 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43b80cbcf0 0x7f43a40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 --2- 172.21.15.60:0/3456471153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43a4003f40 0x7f43a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 --2- 172.21.15.60:0/3456471153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a4004d50 0x7f43a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 -- 172.21.15.60:0/3456471153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a400b9d0 con 0x7f43a4004d50 2024-04-25T10:05:48.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 -- 172.21.15.60:0/3456471153 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a400bb50 con 0x7f43a4003f40 2024-04-25T10:05:48.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 -- 172.21.15.60:0/3456471153 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a400bd20 con 0x7f43b80cbcf0 2024-04-25T10:05:48.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43bd9ac640 1 --2- 172.21.15.60:0/3456471153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a4004d50 0x7f43a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43bc9aa640 1 --2- 172.21.15.60:0/3456471153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43a4003f40 0x7f43a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43bd1ab640 1 --2- 172.21.15.60:0/3456471153 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43b80cbcf0 0x7f43a40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43bd9ac640 1 -- 172.21.15.60:0/3456471153 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43b80cbcf0 msgr2=0x7f43a40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43bd9ac640 1 --2- 172.21.15.60:0/3456471153 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43b80cbcf0 0x7f43a40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43bd9ac640 1 -- 172.21.15.60:0/3456471153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43a4003f40 msgr2=0x7f43a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:48.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43bd9ac640 1 --2- 172.21.15.60:0/3456471153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43a4003f40 0x7f43a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43bd9ac640 1 -- 172.21.15.60:0/3456471153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a4003530 con 0x7f43a4004d50 2024-04-25T10:05:48.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43bd9ac640 1 --2- 172.21.15.60:0/3456471153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a4004d50 0x7f43a40051d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f43b000a880 tx=0x7f43b001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fa31158170daf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 -- 172.21.15.60:0/3456471153 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b000a4a0 con 0x7f43a4004d50 2024-04-25T10:05:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 -- 172.21.15.60:0/3456471153 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b00ad650 con 0x7f43a4004d50 2024-04-25T10:05:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43affff640 1 -- 172.21.15.60:0/3456471153 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b00adac0 con 0x7f43a4004d50 2024-04-25T10:05:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.699+0000 7f43bf40f640 1 -- 172.21.15.60:0/3456471153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a4004d50 msgr2=0x7f43a40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bf40f640 1 --2- 172.21.15.60:0/3456471153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a4004d50 0x7f43a40051d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f43b000a880 tx=0x7f43b001d3a0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bd9ac640 1 -- 172.21.15.60:0/3456471153 reap_dead start 2024-04-25T10:05:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bf40f640 1 -- 172.21.15.60:0/3456471153 shutdown_connections 2024-04-25T10:05:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bf40f640 1 -- 172.21.15.60:0/3456471153 >> 172.21.15.60:0/3456471153 conn(0x7f43b80670b0 msgr2=0x7f43b806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bf40f640 1 -- 172.21.15.60:0/3456471153 shutdown_connections 2024-04-25T10:05:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bf40f640 1 -- 172.21.15.60:0/3456471153 wait complete. 2024-04-25T10:05:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bf40f640 1 Processor -- start 2024-04-25T10:05:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bf40f640 1 -- start start 2024-04-25T10:05:48.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bf40f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43b8158310 0x7f43b806dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bf40f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b80b0780 0x7f43b81f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bf40f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43b81f7070 0x7f43b806e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bf40f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80ce9a0 con 0x7f43b80b0780 2024-04-25T10:05:48.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bf40f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80ccfb0 con 0x7f43b81f7070 2024-04-25T10:05:48.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bf40f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80cc0d0 con 0x7f43b8158310 2024-04-25T10:05:48.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bc9aa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b80b0780 0x7f43b81f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bc9aa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b80b0780 0x7f43b81f6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46998/0 (socket says 172.21.15.60:46998) 2024-04-25T10:05:48.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bc9aa640 1 -- 172.21.15.60:0/3574163921 learned_addr learned my addr 172.21.15.60:0/3574163921 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bd1ab640 1 --2- 172.21.15.60:0/3574163921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43b8158310 0x7f43b806dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bc9aa640 1 -- 172.21.15.60:0/3574163921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43b8158310 msgr2=0x7f43b806dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bc9aa640 1 --2- 172.21.15.60:0/3574163921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43b8158310 0x7f43b806dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bc9aa640 1 -- 172.21.15.60:0/3574163921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43b81f7070 msgr2=0x7f43b806e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bc9aa640 1 --2- 172.21.15.60:0/3574163921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43b81f7070 0x7f43b806e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bc9aa640 1 -- 172.21.15.60:0/3574163921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b80b0e60 con 0x7f43b80b0780 2024-04-25T10:05:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bc9aa640 1 --2- 172.21.15.60:0/3574163921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b80b0780 0x7f43b81f6b30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f43a800f510 tx=0x7f43a800f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43adffb640 1 -- 172.21.15.60:0/3574163921 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8018020 con 0x7f43b80b0780 2024-04-25T10:05:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43adffb640 1 -- 172.21.15.60:0/3574163921 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a8014890 con 0x7f43b80b0780 2024-04-25T10:05:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bf40f640 1 -- 172.21.15.60:0/3574163921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b80b1140 con 0x7f43b80b0780 2024-04-25T10:05:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43adffb640 1 -- 172.21.15.60:0/3574163921 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8014c80 con 0x7f43b80b0780 2024-04-25T10:05:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bf40f640 1 -- 172.21.15.60:0/3574163921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b80acd20 con 0x7f43b80b0780 2024-04-25T10:05:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.703+0000 7f43bf40f640 1 -- 172.21.15.60:0/3574163921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b806bba0 con 0x7f43b80b0780 2024-04-25T10:05:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.707+0000 7f43adffb640 1 -- 172.21.15.60:0/3574163921 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43a801b020 con 0x7f43b80b0780 2024-04-25T10:05:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.707+0000 7f43adffb640 1 --2- 172.21.15.60:0/3574163921 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f439c066f60 0x7f439c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.707+0000 7f43adffb640 1 -- 172.21.15.60:0/3574163921 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43a8085280 con 0x7f43b80b0780 2024-04-25T10:05:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.707+0000 7f43bd1ab640 1 --2- 172.21.15.60:0/3574163921 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f439c066f60 0x7f439c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.707+0000 7f43bd1ab640 1 --2- 172.21.15.60:0/3574163921 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f439c066f60 0x7f439c069420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009970 tx=0x7f43b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:48.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.711+0000 7f43adffb640 1 -- 172.21.15.60:0/3574163921 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43a8022020 con 0x7f43b80b0780 2024-04-25T10:05:48.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.867+0000 7f43bf40f640 1 -- 172.21.15.60:0/3574163921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43b81f74c0 con 0x7f43b80b0780 2024-04-25T10:05:48.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.871+0000 7f43adffb640 1 -- 172.21.15.60:0/3574163921 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43a8014e20 con 0x7f43b80b0780 2024-04-25T10:05:48.877 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:05:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.871+0000 7f43bf40f640 1 -- 172.21.15.60:0/3574163921 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f439c066f60 msgr2=0x7f439c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.871+0000 7f43bf40f640 1 --2- 172.21.15.60:0/3574163921 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f439c066f60 0x7f439c069420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009970 tx=0x7f43b4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.871+0000 7f43bf40f640 1 -- 172.21.15.60:0/3574163921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b80b0780 msgr2=0x7f43b81f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.871+0000 7f43bf40f640 1 --2- 172.21.15.60:0/3574163921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b80b0780 0x7f43b81f6b30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f43a800f510 tx=0x7f43a800f9d0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.871+0000 7f43bf40f640 1 -- 172.21.15.60:0/3574163921 shutdown_connections 2024-04-25T10:05:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.871+0000 7f43bf40f640 1 --2- 172.21.15.60:0/3574163921 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f439c066f60 0x7f439c069420 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.871+0000 7f43bf40f640 1 --2- 172.21.15.60:0/3574163921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43b81f7070 0x7f43b806e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.871+0000 7f43bf40f640 1 --2- 172.21.15.60:0/3574163921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b80b0780 0x7f43b81f6b30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.871+0000 7f43bf40f640 1 --2- 172.21.15.60:0/3574163921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43b8158310 0x7f43b806dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.871+0000 7f43bf40f640 1 -- 172.21.15.60:0/3574163921 >> 172.21.15.60:0/3574163921 conn(0x7f43b80670b0 msgr2=0x7f43b815d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.871+0000 7f43bf40f640 1 -- 172.21.15.60:0/3574163921 shutdown_connections 2024-04-25T10:05:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:48.871+0000 7f43bf40f640 1 -- 172.21.15.60:0/3574163921 wait complete. 2024-04-25T10:05:48.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:49.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.015+0000 7f67f88c1640 1 Processor -- start 2024-04-25T10:05:49.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.015+0000 7f67f88c1640 1 -- start start 2024-04-25T10:05:49.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.015+0000 7f67f88c1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67f4160080 0x7f67f415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.015+0000 7f67f88c1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f67f415fa20 0x7f67f40ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.015+0000 7f67f88c1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f4144a40 con 0x7f67f40cbcf0 2024-04-25T10:05:49.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.015+0000 7f67f88c1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f415fe90 con 0x7f67f415fa20 2024-04-25T10:05:49.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.015+0000 7f67f88c1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f40b2dc0 con 0x7f67f4160080 2024-04-25T10:05:49.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.015+0000 7f67f259c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f67f40cbcf0 0x7f67f4071c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45444/0 (socket says 172.21.15.60:45444) 2024-04-25T10:05:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.015+0000 7f67f259c640 1 -- 172.21.15.60:0/1896470996 learned_addr learned my addr 172.21.15.60:0/1896470996 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.015+0000 7f67f2d9d640 1 --2- 172.21.15.60:0/1896470996 >> v2:172.21.15.60:3300/0 conn(0x7f67f415fa20 0x7f67f40ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.015+0000 7f67f1d9b640 1 --2- 172.21.15.60:0/1896470996 >> v2:172.21.15.100:3300/0 conn(0x7f67f4160080 0x7f67f415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.015+0000 7f67f2d9d640 1 -- 172.21.15.60:0/1896470996 >> v2:172.21.15.100:3300/0 conn(0x7f67f4160080 msgr2=0x7f67f415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.015+0000 7f67f2d9d640 1 --2- 172.21.15.60:0/1896470996 >> v2:172.21.15.100:3300/0 conn(0x7f67f4160080 0x7f67f415f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.015+0000 7f67f2d9d640 1 -- 172.21.15.60:0/1896470996 >> v1:172.21.15.60:6789/0 conn(0x7f67f40cbcf0 legacy=0x7f67f4071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.015+0000 7f67f2d9d640 1 -- 172.21.15.60:0/1896470996 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f40cc9f0 con 0x7f67f415fa20 2024-04-25T10:05:49.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f2d9d640 1 --2- 172.21.15.60:0/1896470996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f415fa20 0x7f67f40ceec0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f67f40d0570 tx=0x7f67ec01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136d8b122bd4bec8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f159a640 1 -- 172.21.15.60:0/1896470996 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec0b2070 con 0x7f67f415fa20 2024-04-25T10:05:49.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f159a640 1 -- 172.21.15.60:0/1896470996 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ec0a58e0 con 0x7f67f415fa20 2024-04-25T10:05:49.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f159a640 1 -- 172.21.15.60:0/1896470996 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec0a5cb0 con 0x7f67f415fa20 2024-04-25T10:05:49.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f88c1640 1 -- 172.21.15.60:0/1896470996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f415fa20 msgr2=0x7f67f40ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f88c1640 1 --2- 172.21.15.60:0/1896470996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f415fa20 0x7f67f40ceec0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f67f40d0570 tx=0x7f67ec01c930 comp rx=0 tx=0).stop 2024-04-25T10:05:49.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f88c1640 1 -- 172.21.15.60:0/1896470996 shutdown_connections 2024-04-25T10:05:49.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f88c1640 1 --2- 172.21.15.60:0/1896470996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f415fa20 0x7f67f40ceec0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f88c1640 1 --2- 172.21.15.60:0/1896470996 >> v2:172.21.15.100:3300/0 conn(0x7f67f4160080 0x7f67f415f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f88c1640 1 -- 172.21.15.60:0/1896470996 >> 172.21.15.60:0/1896470996 conn(0x7f67f40670b0 msgr2=0x7f67f406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:49.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f88c1640 1 -- 172.21.15.60:0/1896470996 shutdown_connections 2024-04-25T10:05:49.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f88c1640 1 -- 172.21.15.60:0/1896470996 wait complete. 2024-04-25T10:05:49.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f88c1640 1 Processor -- start 2024-04-25T10:05:49.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f88c1640 1 -- start start 2024-04-25T10:05:49.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f88c1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67f40cbcf0 0x7f67f40b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f88c1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f415fa20 0x7f67f4140810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f88c1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f4160080 0x7f67f4158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f88c1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f40ce5c0 con 0x7f67f415fa20 2024-04-25T10:05:49.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f88c1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f40cc0d0 con 0x7f67f4160080 2024-04-25T10:05:49.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f88c1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f40ce740 con 0x7f67f40cbcf0 2024-04-25T10:05:49.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f415fa20 0x7f67f4140810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f415fa20 0x7f67f4140810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54144/0 (socket says 172.21.15.60:54144) 2024-04-25T10:05:49.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f1d9b640 1 -- 172.21.15.60:0/2142302401 learned_addr learned my addr 172.21.15.60:0/2142302401 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:49.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f259c640 1 --2- 172.21.15.60:0/2142302401 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67f40cbcf0 0x7f67f40b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f2d9d640 1 --2- 172.21.15.60:0/2142302401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f4160080 0x7f67f4158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f1d9b640 1 -- 172.21.15.60:0/2142302401 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67f40cbcf0 msgr2=0x7f67f40b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f1d9b640 1 --2- 172.21.15.60:0/2142302401 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67f40cbcf0 0x7f67f40b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.019+0000 7f67f1d9b640 1 -- 172.21.15.60:0/2142302401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f4160080 msgr2=0x7f67f4158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.023+0000 7f67f1d9b640 1 --2- 172.21.15.60:0/2142302401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f4160080 0x7f67f4158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.023+0000 7f67f1d9b640 1 -- 172.21.15.60:0/2142302401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f4159de0 con 0x7f67f415fa20 2024-04-25T10:05:49.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.023+0000 7f67f1d9b640 1 --2- 172.21.15.60:0/2142302401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f415fa20 0x7f67f4140810 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f67e800d5f0 tx=0x7f67e800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.023+0000 7f67e2ffd640 1 -- 172.21.15.60:0/2142302401 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e80125f0 con 0x7f67f415fa20 2024-04-25T10:05:49.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.023+0000 7f67e2ffd640 1 -- 172.21.15.60:0/2142302401 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e8019070 con 0x7f67f415fa20 2024-04-25T10:05:49.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.023+0000 7f67e2ffd640 1 -- 172.21.15.60:0/2142302401 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e80129b0 con 0x7f67f415fa20 2024-04-25T10:05:49.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.023+0000 7f67f88c1640 1 -- 172.21.15.60:0/2142302401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f415a0c0 con 0x7f67f415fa20 2024-04-25T10:05:49.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.023+0000 7f67f88c1640 1 -- 172.21.15.60:0/2142302401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f415a960 con 0x7f67f415fa20 2024-04-25T10:05:49.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.023+0000 7f67f88c1640 1 -- 172.21.15.60:0/2142302401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f415a290 con 0x7f67f415fa20 2024-04-25T10:05:49.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.023+0000 7f67e2ffd640 1 -- 172.21.15.60:0/2142302401 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67e8012b50 con 0x7f67f415fa20 2024-04-25T10:05:49.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.023+0000 7f67e2ffd640 1 --2- 172.21.15.60:0/2142302401 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67d4066f60 0x7f67d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.023+0000 7f67e2ffd640 1 -- 172.21.15.60:0/2142302401 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67e8085340 con 0x7f67f415fa20 2024-04-25T10:05:49.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.023+0000 7f67f259c640 1 --2- 172.21.15.60:0/2142302401 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67d4066f60 0x7f67d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.023+0000 7f67f259c640 1 --2- 172.21.15.60:0/2142302401 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67d4066f60 0x7f67d4069420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f67e4000f70 tx=0x7f67e4007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.027+0000 7f67e2ffd640 1 -- 172.21.15.60:0/2142302401 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67e8056350 con 0x7f67f415fa20 2024-04-25T10:05:49.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.191+0000 7f67f88c1640 1 -- 172.21.15.60:0/2142302401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67f406bba0 con 0x7f67f415fa20 2024-04-25T10:05:49.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.191+0000 7f67e2ffd640 1 -- 172.21.15.60:0/2142302401 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f67e8055c50 con 0x7f67f415fa20 2024-04-25T10:05:49.197 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:05:49.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.191+0000 7f67f88c1640 1 -- 172.21.15.60:0/2142302401 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67d4066f60 msgr2=0x7f67d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.191+0000 7f67f88c1640 1 --2- 172.21.15.60:0/2142302401 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67d4066f60 0x7f67d4069420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f67e4000f70 tx=0x7f67e4007e90 comp rx=0 tx=0).stop 2024-04-25T10:05:49.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.191+0000 7f67f88c1640 1 -- 172.21.15.60:0/2142302401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f415fa20 msgr2=0x7f67f4140810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.191+0000 7f67f88c1640 1 --2- 172.21.15.60:0/2142302401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f415fa20 0x7f67f4140810 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f67e800d5f0 tx=0x7f67e800dab0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.191+0000 7f67f88c1640 1 -- 172.21.15.60:0/2142302401 shutdown_connections 2024-04-25T10:05:49.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.191+0000 7f67f88c1640 1 --2- 172.21.15.60:0/2142302401 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67d4066f60 0x7f67d4069420 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.191+0000 7f67f88c1640 1 --2- 172.21.15.60:0/2142302401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f4160080 0x7f67f4158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.191+0000 7f67f88c1640 1 --2- 172.21.15.60:0/2142302401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f415fa20 0x7f67f4140810 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.191+0000 7f67f88c1640 1 --2- 172.21.15.60:0/2142302401 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67f40cbcf0 0x7f67f40b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.191+0000 7f67f88c1640 1 -- 172.21.15.60:0/2142302401 >> 172.21.15.60:0/2142302401 conn(0x7f67f40670b0 msgr2=0x7f67f415d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:49.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.191+0000 7f67f88c1640 1 -- 172.21.15.60:0/2142302401 shutdown_connections 2024-04-25T10:05:49.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.191+0000 7f67f88c1640 1 -- 172.21.15.60:0/2142302401 wait complete. 2024-04-25T10:05:49.217 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:49.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.331+0000 7f81b5ee9640 1 Processor -- start 2024-04-25T10:05:49.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81b5ee9640 1 -- start start 2024-04-25T10:05:49.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81b5ee9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f81b01583d0 0x7f81b015ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81b5ee9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f81b00b3610 0x7f81b015c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81b5ee9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b015b670 con 0x7f81b00b14a0 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81b5ee9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b015b950 con 0x7f81b00b3610 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81b5ee9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b015bc30 con 0x7f81b01583d0 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81aeffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f81b00b3610 0x7f81b015c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81aeffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f81b00b3610 0x7f81b015c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54146/0 (socket says 172.21.15.60:54146) 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81aeffd640 1 -- 172.21.15.60:0/2982652382 learned_addr learned my addr 172.21.15.60:0/2982652382 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81affff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f81b00b14a0 0x7f81b00d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45448/0 (socket says 172.21.15.60:45448) 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81af7fe640 1 --2- 172.21.15.60:0/2982652382 >> v2:172.21.15.100:3300/0 conn(0x7f81b01583d0 0x7f81b015ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81aeffd640 1 -- 172.21.15.60:0/2982652382 >> v2:172.21.15.100:3300/0 conn(0x7f81b01583d0 msgr2=0x7f81b015ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81aeffd640 1 --2- 172.21.15.60:0/2982652382 >> v2:172.21.15.100:3300/0 conn(0x7f81b01583d0 0x7f81b015ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81aeffd640 1 -- 172.21.15.60:0/2982652382 >> v1:172.21.15.60:6789/0 conn(0x7f81b00b14a0 legacy=0x7f81b00d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81aeffd640 1 -- 172.21.15.60:0/2982652382 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b00d0590 con 0x7f81b00b3610 2024-04-25T10:05:49.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81af7fe640 1 --2- 172.21.15.60:0/2982652382 >> v2:172.21.15.100:3300/0 conn(0x7f81b01583d0 0x7f81b015ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:49.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81aeffd640 1 --2- 172.21.15.60:0/2982652382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81b00b3610 0x7f81b015c000 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f81b0071cf0 tx=0x7f81a401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b908947ee3d7d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.335+0000 7f81ae7fc640 1 -- 172.21.15.60:0/2982652382 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a40b4070 con 0x7f81b00b3610 2024-04-25T10:05:49.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81ae7fc640 1 -- 172.21.15.60:0/2982652382 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a40a7be0 con 0x7f81b00b3610 2024-04-25T10:05:49.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81ae7fc640 1 -- 172.21.15.60:0/2982652382 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a40af440 con 0x7f81b00b3610 2024-04-25T10:05:49.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81b5ee9640 1 -- 172.21.15.60:0/2982652382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81b00b3610 msgr2=0x7f81b015c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81b5ee9640 1 --2- 172.21.15.60:0/2982652382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81b00b3610 0x7f81b015c000 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f81b0071cf0 tx=0x7f81a401ebf0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81b5ee9640 1 -- 172.21.15.60:0/2982652382 shutdown_connections 2024-04-25T10:05:49.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81b5ee9640 1 --2- 172.21.15.60:0/2982652382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81b00b3610 0x7f81b015c000 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81b5ee9640 1 --2- 172.21.15.60:0/2982652382 >> v2:172.21.15.100:3300/0 conn(0x7f81b01583d0 0x7f81b015ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81b5ee9640 1 -- 172.21.15.60:0/2982652382 >> 172.21.15.60:0/2982652382 conn(0x7f81b0072920 msgr2=0x7f81b0099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:49.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81b5ee9640 1 -- 172.21.15.60:0/2982652382 shutdown_connections 2024-04-25T10:05:49.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81b5ee9640 1 -- 172.21.15.60:0/2982652382 wait complete. 2024-04-25T10:05:49.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81b5ee9640 1 Processor -- start 2024-04-25T10:05:49.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81b5ee9640 1 -- start start 2024-04-25T10:05:49.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81b5ee9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81b00b14a0 0x7f81b006e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81b5ee9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81b00b3610 0x7f81b006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81b5ee9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81b01583d0 0x7f81b015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81b5ee9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b015c6c0 con 0x7f81b01583d0 2024-04-25T10:05:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81b5ee9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b015c540 con 0x7f81b00b3610 2024-04-25T10:05:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81b5ee9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b006b420 con 0x7f81b00b14a0 2024-04-25T10:05:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81affff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81b01583d0 0x7f81b015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81affff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81b01583d0 0x7f81b015e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54160/0 (socket says 172.21.15.60:54160) 2024-04-25T10:05:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81affff640 1 -- 172.21.15.60:0/1734428209 learned_addr learned my addr 172.21.15.60:0/1734428209 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81af7fe640 1 --2- 172.21.15.60:0/1734428209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81b00b14a0 0x7f81b006e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.339+0000 7f81aeffd640 1 --2- 172.21.15.60:0/1734428209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81b00b3610 0x7f81b006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f81affff640 1 -- 172.21.15.60:0/1734428209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81b00b14a0 msgr2=0x7f81b006e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f81affff640 1 --2- 172.21.15.60:0/1734428209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81b00b14a0 0x7f81b006e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f81affff640 1 -- 172.21.15.60:0/1734428209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81b00b3610 msgr2=0x7f81b006f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f81affff640 1 --2- 172.21.15.60:0/1734428209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81b00b3610 0x7f81b006f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f81affff640 1 -- 172.21.15.60:0/1734428209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b01f7090 con 0x7f81b01583d0 2024-04-25T10:05:49.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f81af7fe640 1 --2- 172.21.15.60:0/1734428209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81b00b14a0 0x7f81b006e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:49.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f81affff640 1 --2- 172.21.15.60:0/1734428209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81b01583d0 0x7f81b015e3c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f81a800c7a0 tx=0x7f81a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f8197fff640 1 -- 172.21.15.60:0/1734428209 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8017070 con 0x7f81b01583d0 2024-04-25T10:05:49.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f8197fff640 1 -- 172.21.15.60:0/1734428209 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a800a640 con 0x7f81b01583d0 2024-04-25T10:05:49.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f8197fff640 1 -- 172.21.15.60:0/1734428209 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a800a9f0 con 0x7f81b01583d0 2024-04-25T10:05:49.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f81b5ee9640 1 -- 172.21.15.60:0/1734428209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b006fcf0 con 0x7f81b01583d0 2024-04-25T10:05:49.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f81b5ee9640 1 -- 172.21.15.60:0/1734428209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b01f7630 con 0x7f81b01583d0 2024-04-25T10:05:49.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f81b5ee9640 1 -- 172.21.15.60:0/1734428209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b0074770 con 0x7f81b01583d0 2024-04-25T10:05:49.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f8197fff640 1 -- 172.21.15.60:0/1734428209 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f81a801a070 con 0x7f81b01583d0 2024-04-25T10:05:49.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f8197fff640 1 --2- 172.21.15.60:0/1734428209 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f819c066f60 0x7f819c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f8197fff640 1 -- 172.21.15.60:0/1734428209 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81a8084530 con 0x7f81b01583d0 2024-04-25T10:05:49.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f81af7fe640 1 --2- 172.21.15.60:0/1734428209 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f819c066f60 0x7f819c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.343+0000 7f81af7fe640 1 --2- 172.21.15.60:0/1734428209 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f819c066f60 0x7f819c069420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f81a000a9b0 tx=0x7f81a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.347+0000 7f8197fff640 1 -- 172.21.15.60:0/1734428209 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f81a8054f80 con 0x7f81b01583d0 2024-04-25T10:05:49.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.511+0000 7f81b5ee9640 1 -- 172.21.15.60:0/1734428209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f81b006ff20 con 0x7f81b01583d0 2024-04-25T10:05:49.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.511+0000 7f8197fff640 1 -- 172.21.15.60:0/1734428209 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f81a8059020 con 0x7f81b01583d0 2024-04-25T10:05:49.519 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:05:49.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.515+0000 7f81b5ee9640 1 -- 172.21.15.60:0/1734428209 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f819c066f60 msgr2=0x7f819c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.515+0000 7f81b5ee9640 1 --2- 172.21.15.60:0/1734428209 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f819c066f60 0x7f819c069420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f81a000a9b0 tx=0x7f81a0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:49.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.515+0000 7f81b5ee9640 1 -- 172.21.15.60:0/1734428209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81b01583d0 msgr2=0x7f81b015e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.515+0000 7f81b5ee9640 1 --2- 172.21.15.60:0/1734428209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81b01583d0 0x7f81b015e3c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f81a800c7a0 tx=0x7f81a800cc60 comp rx=0 tx=0).stop 2024-04-25T10:05:49.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.515+0000 7f81b5ee9640 1 -- 172.21.15.60:0/1734428209 shutdown_connections 2024-04-25T10:05:49.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.515+0000 7f81b5ee9640 1 --2- 172.21.15.60:0/1734428209 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f819c066f60 0x7f819c069420 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.515+0000 7f81b5ee9640 1 --2- 172.21.15.60:0/1734428209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81b01583d0 0x7f81b015e3c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.515+0000 7f81b5ee9640 1 --2- 172.21.15.60:0/1734428209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81b00b3610 0x7f81b006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.515+0000 7f81b5ee9640 1 --2- 172.21.15.60:0/1734428209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81b00b14a0 0x7f81b006e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.515+0000 7f81b5ee9640 1 -- 172.21.15.60:0/1734428209 >> 172.21.15.60:0/1734428209 conn(0x7f81b0072920 msgr2=0x7f81b0099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:49.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.515+0000 7f81b5ee9640 1 -- 172.21.15.60:0/1734428209 shutdown_connections 2024-04-25T10:05:49.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.515+0000 7f81b5ee9640 1 -- 172.21.15.60:0/1734428209 wait complete. 2024-04-25T10:05:49.539 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:49.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.651+0000 7f4e8c9c0640 1 Processor -- start 2024-04-25T10:05:49.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.651+0000 7f4e8c9c0640 1 -- start start 2024-04-25T10:05:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.651+0000 7f4e8c9c0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4e8806ec30 0x7f4e880cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.651+0000 7f4e8c9c0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e880b37e0 0x7f4e880b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.651+0000 7f4e8c9c0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e8806f130 con 0x7f4e8815a7a0 2024-04-25T10:05:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.651+0000 7f4e8c9c0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e880b4190 con 0x7f4e880b37e0 2024-04-25T10:05:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.651+0000 7f4e8c9c0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e880cdaa0 con 0x7f4e8806ec30 2024-04-25T10:05:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.651+0000 7f4e86d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e880b37e0 0x7f4e880b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.651+0000 7f4e86d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e880b37e0 0x7f4e880b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54166/0 (socket says 172.21.15.60:54166) 2024-04-25T10:05:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.651+0000 7f4e86d9d640 1 -- 172.21.15.60:0/2314018657 learned_addr learned my addr 172.21.15.60:0/2314018657 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.651+0000 7f4e85d9b640 1 --2- 172.21.15.60:0/2314018657 >> v2:172.21.15.100:3300/0 conn(0x7f4e8806ec30 0x7f4e880cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.651+0000 7f4e86d9d640 1 -- 172.21.15.60:0/2314018657 >> v2:172.21.15.100:3300/0 conn(0x7f4e8806ec30 msgr2=0x7f4e880cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.651+0000 7f4e86d9d640 1 --2- 172.21.15.60:0/2314018657 >> v2:172.21.15.100:3300/0 conn(0x7f4e8806ec30 0x7f4e880cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.651+0000 7f4e86d9d640 1 -- 172.21.15.60:0/2314018657 >> v1:172.21.15.100:6789/0 conn(0x7f4e8815a7a0 legacy=0x7f4e88071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.651+0000 7f4e86d9d640 1 -- 172.21.15.60:0/2314018657 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8806f410 con 0x7f4e880b37e0 2024-04-25T10:05:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e86d9d640 1 --2- 172.21.15.60:0/2314018657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e880b37e0 0x7f4e880b3c50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000bb10 tx=0x7f4e8001ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c09bdf2f73e83d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8559a640 1 -- 172.21.15.60:0/2314018657 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e800b4070 con 0x7f4e880b37e0 2024-04-25T10:05:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8559a640 1 -- 172.21.15.60:0/2314018657 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e800a7b90 con 0x7f4e880b37e0 2024-04-25T10:05:49.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8559a640 1 -- 172.21.15.60:0/2314018657 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e800af440 con 0x7f4e880b37e0 2024-04-25T10:05:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8c9c0640 1 -- 172.21.15.60:0/2314018657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e880b37e0 msgr2=0x7f4e880b3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8c9c0640 1 --2- 172.21.15.60:0/2314018657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e880b37e0 0x7f4e880b3c50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000bb10 tx=0x7f4e8001ebb0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8c9c0640 1 -- 172.21.15.60:0/2314018657 shutdown_connections 2024-04-25T10:05:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8c9c0640 1 --2- 172.21.15.60:0/2314018657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e880b37e0 0x7f4e880b3c50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8c9c0640 1 --2- 172.21.15.60:0/2314018657 >> v2:172.21.15.100:3300/0 conn(0x7f4e8806ec30 0x7f4e880cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8c9c0640 1 -- 172.21.15.60:0/2314018657 >> 172.21.15.60:0/2314018657 conn(0x7f4e880b12d0 msgr2=0x7f4e880b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8c9c0640 1 -- 172.21.15.60:0/2314018657 shutdown_connections 2024-04-25T10:05:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8c9c0640 1 -- 172.21.15.60:0/2314018657 wait complete. 2024-04-25T10:05:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8c9c0640 1 Processor -- start 2024-04-25T10:05:49.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8c9c0640 1 -- start start 2024-04-25T10:05:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8c9c0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e8806ec30 0x7f4e8814a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8c9c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e880b37e0 0x7f4e8815c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8c9c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e8815a7a0 0x7f4e8815c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8c9c0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8806e8e0 con 0x7f4e8806ec30 2024-04-25T10:05:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8c9c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8806e760 con 0x7f4e8815a7a0 2024-04-25T10:05:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.655+0000 7f4e8c9c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8806d8b0 con 0x7f4e880b37e0 2024-04-25T10:05:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e8659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e8806ec30 0x7f4e8814a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e8659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e8806ec30 0x7f4e8814a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54178/0 (socket says 172.21.15.60:54178) 2024-04-25T10:05:49.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e8659c640 1 -- 172.21.15.60:0/808726113 learned_addr learned my addr 172.21.15.60:0/808726113 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:49.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e86d9d640 1 --2- 172.21.15.60:0/808726113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e8815a7a0 0x7f4e8815c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e8659c640 1 -- 172.21.15.60:0/808726113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e880b37e0 msgr2=0x7f4e8815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e8659c640 1 --2- 172.21.15.60:0/808726113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e880b37e0 0x7f4e8815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e8659c640 1 -- 172.21.15.60:0/808726113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e8815a7a0 msgr2=0x7f4e8815c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e8659c640 1 --2- 172.21.15.60:0/808726113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e8815a7a0 0x7f4e8815c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e8659c640 1 -- 172.21.15.60:0/808726113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e88150e50 con 0x7f4e8806ec30 2024-04-25T10:05:49.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e86d9d640 1 --2- 172.21.15.60:0/808726113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e8815a7a0 0x7f4e8815c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:49.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e8659c640 1 --2- 172.21.15.60:0/808726113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e8806ec30 0x7f4e8814a150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800ba50 tx=0x7f4e7800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e76ffd640 1 -- 172.21.15.60:0/808726113 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e78019070 con 0x7f4e8806ec30 2024-04-25T10:05:49.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e76ffd640 1 -- 172.21.15.60:0/808726113 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7800cc20 con 0x7f4e8806ec30 2024-04-25T10:05:49.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e8c9c0640 1 -- 172.21.15.60:0/808726113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e88151130 con 0x7f4e8806ec30 2024-04-25T10:05:49.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e76ffd640 1 -- 172.21.15.60:0/808726113 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e78014680 con 0x7f4e8806ec30 2024-04-25T10:05:49.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e8c9c0640 1 -- 172.21.15.60:0/808726113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e881516f0 con 0x7f4e8806ec30 2024-04-25T10:05:49.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e8c9c0640 1 -- 172.21.15.60:0/808726113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e88074770 con 0x7f4e8806ec30 2024-04-25T10:05:49.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e76ffd640 1 -- 172.21.15.60:0/808726113 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4e7801c070 con 0x7f4e8806ec30 2024-04-25T10:05:49.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e76ffd640 1 --2- 172.21.15.60:0/808726113 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e68066f60 0x7f4e68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e76ffd640 1 -- 172.21.15.60:0/808726113 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e78084c60 con 0x7f4e8806ec30 2024-04-25T10:05:49.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.659+0000 7f4e85d9b640 1 --2- 172.21.15.60:0/808726113 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e68066f60 0x7f4e68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.663+0000 7f4e85d9b640 1 --2- 172.21.15.60:0/808726113 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e68066f60 0x7f4e68069420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4e8814e520 tx=0x7f4e7c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.663+0000 7f4e76ffd640 1 -- 172.21.15.60:0/808726113 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4e780556b0 con 0x7f4e8806ec30 2024-04-25T10:05:49.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.827+0000 7f4e8c9c0640 1 -- 172.21.15.60:0/808726113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e8815db70 con 0x7f4e8806ec30 2024-04-25T10:05:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.827+0000 7f4e76ffd640 1 -- 172.21.15.60:0/808726113 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f4e78059750 con 0x7f4e8806ec30 2024-04-25T10:05:49.836 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:49.836 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:22.036582+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:49.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.831+0000 7f4e8c9c0640 1 -- 172.21.15.60:0/808726113 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e68066f60 msgr2=0x7f4e68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.831+0000 7f4e8c9c0640 1 --2- 172.21.15.60:0/808726113 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e68066f60 0x7f4e68069420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4e8814e520 tx=0x7f4e7c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:49.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.831+0000 7f4e8c9c0640 1 -- 172.21.15.60:0/808726113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e8806ec30 msgr2=0x7f4e8814a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.831+0000 7f4e8c9c0640 1 --2- 172.21.15.60:0/808726113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e8806ec30 0x7f4e8814a150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800ba50 tx=0x7f4e7800bf10 comp rx=0 tx=0).stop 2024-04-25T10:05:49.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.831+0000 7f4e8c9c0640 1 -- 172.21.15.60:0/808726113 shutdown_connections 2024-04-25T10:05:49.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.831+0000 7f4e8c9c0640 1 --2- 172.21.15.60:0/808726113 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e68066f60 0x7f4e68069420 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4e8814e520 tx=0x7f4e7c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:49.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.831+0000 7f4e8c9c0640 1 --2- 172.21.15.60:0/808726113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e8815a7a0 0x7f4e8815c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.831+0000 7f4e8c9c0640 1 --2- 172.21.15.60:0/808726113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e880b37e0 0x7f4e8815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.831+0000 7f4e8c9c0640 1 --2- 172.21.15.60:0/808726113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e8806ec30 0x7f4e8814a150 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.831+0000 7f4e8c9c0640 1 -- 172.21.15.60:0/808726113 >> 172.21.15.60:0/808726113 conn(0x7f4e880b12d0 msgr2=0x7f4e880cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:49.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.831+0000 7f4e8c9c0640 1 -- 172.21.15.60:0/808726113 shutdown_connections 2024-04-25T10:05:49.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.831+0000 7f4e8c9c0640 1 -- 172.21.15.60:0/808726113 wait complete. 2024-04-25T10:05:49.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:18.976643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:19.405302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:49.856 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:49.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.971+0000 7fabc5c5c640 1 Processor -- start 2024-04-25T10:05:49.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.975+0000 7fabc5c5c640 1 -- start start 2024-04-25T10:05:49.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.975+0000 7fabc5c5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabc00cbcf0 0x7fabc0160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.975+0000 7fabc5c5c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fabc0071c30 0x7fabc015f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.975+0000 7fabc5c5c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc0144a40 con 0x7fabc015fa20 2024-04-25T10:05:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.975+0000 7fabc5c5c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc00b2dc0 con 0x7fabc0071c30 2024-04-25T10:05:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.975+0000 7fabc5c5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc015fe80 con 0x7fabc00cbcf0 2024-04-25T10:05:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.975+0000 7fabbeffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fabc0071c30 0x7fabc015f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.975+0000 7fabbeffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fabc0071c30 0x7fabc015f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54192/0 (socket says 172.21.15.60:54192) 2024-04-25T10:05:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.975+0000 7fabbeffd640 1 -- 172.21.15.60:0/2605368072 learned_addr learned my addr 172.21.15.60:0/2605368072 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.975+0000 7fabbf7fe640 1 --2- 172.21.15.60:0/2605368072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabc00cbcf0 0x7fabc0160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.975+0000 7fabbeffd640 1 -- 172.21.15.60:0/2605368072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabc00cbcf0 msgr2=0x7fabc0160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.975+0000 7fabbeffd640 1 --2- 172.21.15.60:0/2605368072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabc00cbcf0 0x7fabc0160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.975+0000 7fabbeffd640 1 -- 172.21.15.60:0/2605368072 >> v1:172.21.15.60:6789/0 conn(0x7fabc015fa20 legacy=0x7fabc015bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.975+0000 7fabbe7fc640 1 -- 172.21.15.60:0/2605368072 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948794599 0 0) 0x7fabc0144a40 con 0x7fabc015fa20 2024-04-25T10:05:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.975+0000 7fabbeffd640 1 -- 172.21.15.60:0/2605368072 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc00cc9f0 con 0x7fabc0071c30 2024-04-25T10:05:49.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.975+0000 7fabbeffd640 1 --2- 172.21.15.60:0/2605368072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabc0071c30 0x7fabc015f4e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009870 tx=0x7fabb401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eabbf1fc07406f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabbe7fc640 1 -- 172.21.15.60:0/2605368072 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb40a95f0 con 0x7fabc0071c30 2024-04-25T10:05:49.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabbe7fc640 1 -- 172.21.15.60:0/2605368072 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb40a9790 con 0x7fabc0071c30 2024-04-25T10:05:49.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabbe7fc640 1 -- 172.21.15.60:0/2605368072 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb40a9b60 con 0x7fabc0071c30 2024-04-25T10:05:49.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 -- 172.21.15.60:0/2605368072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabc0071c30 msgr2=0x7fabc015f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 --2- 172.21.15.60:0/2605368072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabc0071c30 0x7fabc015f4e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009870 tx=0x7fabb401c530 comp rx=0 tx=0).stop 2024-04-25T10:05:49.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 -- 172.21.15.60:0/2605368072 shutdown_connections 2024-04-25T10:05:49.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 --2- 172.21.15.60:0/2605368072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabc0071c30 0x7fabc015f4e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 --2- 172.21.15.60:0/2605368072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabc00cbcf0 0x7fabc0160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 -- 172.21.15.60:0/2605368072 >> 172.21.15.60:0/2605368072 conn(0x7fabc00670b0 msgr2=0x7fabc006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:49.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 -- 172.21.15.60:0/2605368072 shutdown_connections 2024-04-25T10:05:49.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 -- 172.21.15.60:0/2605368072 wait complete. 2024-04-25T10:05:49.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 Processor -- start 2024-04-25T10:05:49.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 -- start start 2024-04-25T10:05:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabc0071c30 0x7fabc013aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabc00cbcf0 0x7fabc0137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabc015fa20 0x7fabc0138210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00cfed0 con 0x7fabc00cbcf0 2024-04-25T10:05:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00720b0 con 0x7fabc015fa20 2024-04-25T10:05:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc013b530 con 0x7fabc0071c30 2024-04-25T10:05:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabbeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabc00cbcf0 0x7fabc0137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabbeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabc00cbcf0 0x7fabc0137cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54204/0 (socket says 172.21.15.60:54204) 2024-04-25T10:05:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabbeffd640 1 -- 172.21.15.60:0/4083951306 learned_addr learned my addr 172.21.15.60:0/4083951306 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabbf7fe640 1 --2- 172.21.15.60:0/4083951306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabc0071c30 0x7fabc013aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabbeffd640 1 -- 172.21.15.60:0/4083951306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabc0071c30 msgr2=0x7fabc013aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabbeffd640 1 --2- 172.21.15.60:0/4083951306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabc0071c30 0x7fabc013aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabbeffd640 1 -- 172.21.15.60:0/4083951306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabc015fa20 msgr2=0x7fabc0138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:49.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabbeffd640 1 --2- 172.21.15.60:0/4083951306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabc015fa20 0x7fabc0138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:49.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabbeffd640 1 -- 172.21.15.60:0/4083951306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc013b760 con 0x7fabc00cbcf0 2024-04-25T10:05:49.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabbeffd640 1 --2- 172.21.15.60:0/4083951306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabc00cbcf0 0x7fabc0137cd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fabb40ab040 tx=0x7fabb401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7faba7fff640 1 -- 172.21.15.60:0/4083951306 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb40b1800 con 0x7fabc00cbcf0 2024-04-25T10:05:49.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7faba7fff640 1 -- 172.21.15.60:0/4083951306 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb40b19c0 con 0x7fabc00cbcf0 2024-04-25T10:05:49.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 -- 172.21.15.60:0/4083951306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc013ba40 con 0x7fabc00cbcf0 2024-04-25T10:05:49.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7faba7fff640 1 -- 172.21.15.60:0/4083951306 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb40b1db0 con 0x7fabc00cbcf0 2024-04-25T10:05:49.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.979+0000 7fabc5c5c640 1 -- 172.21.15.60:0/4083951306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc013c000 con 0x7fabc00cbcf0 2024-04-25T10:05:49.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.983+0000 7fabc5c5c640 1 -- 172.21.15.60:0/4083951306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc006bba0 con 0x7fabc00cbcf0 2024-04-25T10:05:49.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.983+0000 7faba7fff640 1 -- 172.21.15.60:0/4083951306 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fabb40c53f0 con 0x7fabc00cbcf0 2024-04-25T10:05:49.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.983+0000 7faba7fff640 1 --2- 172.21.15.60:0/4083951306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabac066f60 0x7fabac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:49.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.983+0000 7faba7fff640 1 -- 172.21.15.60:0/4083951306 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabb411fbd0 con 0x7fabc00cbcf0 2024-04-25T10:05:49.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.983+0000 7fabbf7fe640 1 --2- 172.21.15.60:0/4083951306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabac066f60 0x7fabac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:49.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.983+0000 7fabbf7fe640 1 --2- 172.21.15.60:0/4083951306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabac066f60 0x7fabac069420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fabc0144400 tx=0x7fabb0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:49.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:49.987+0000 7faba7fff640 1 -- 172.21.15.60:0/4083951306 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fabb40ad020 con 0x7fabc00cbcf0 2024-04-25T10:05:50.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.151+0000 7fabc5c5c640 1 -- 172.21.15.60:0/4083951306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fabc0074770 con 0x7fabc00cbcf0 2024-04-25T10:05:50.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.151+0000 7faba7fff640 1 -- 172.21.15.60:0/4083951306 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fabb4016150 con 0x7fabc00cbcf0 2024-04-25T10:05:50.157 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:05:50.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.151+0000 7fabc5c5c640 1 -- 172.21.15.60:0/4083951306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabac066f60 msgr2=0x7fabac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.151+0000 7fabc5c5c640 1 --2- 172.21.15.60:0/4083951306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabac066f60 0x7fabac069420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fabc0144400 tx=0x7fabb0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.151+0000 7fabc5c5c640 1 -- 172.21.15.60:0/4083951306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabc00cbcf0 msgr2=0x7fabc0137cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.151+0000 7fabc5c5c640 1 --2- 172.21.15.60:0/4083951306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabc00cbcf0 0x7fabc0137cd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fabb40ab040 tx=0x7fabb401ad70 comp rx=0 tx=0).stop 2024-04-25T10:05:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.151+0000 7fabc5c5c640 1 -- 172.21.15.60:0/4083951306 shutdown_connections 2024-04-25T10:05:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.151+0000 7fabc5c5c640 1 --2- 172.21.15.60:0/4083951306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabac066f60 0x7fabac069420 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.151+0000 7fabc5c5c640 1 --2- 172.21.15.60:0/4083951306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabc015fa20 0x7fabc0138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.151+0000 7fabc5c5c640 1 --2- 172.21.15.60:0/4083951306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabc00cbcf0 0x7fabc0137cd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.151+0000 7fabc5c5c640 1 --2- 172.21.15.60:0/4083951306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabc0071c30 0x7fabc013aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.151+0000 7fabc5c5c640 1 -- 172.21.15.60:0/4083951306 >> 172.21.15.60:0/4083951306 conn(0x7fabc00670b0 msgr2=0x7fabc00b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.151+0000 7fabc5c5c640 1 -- 172.21.15.60:0/4083951306 shutdown_connections 2024-04-25T10:05:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.151+0000 7fabc5c5c640 1 -- 172.21.15.60:0/4083951306 wait complete. 2024-04-25T10:05:50.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:50.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa926f640 1 Processor -- start 2024-04-25T10:05:50.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa926f640 1 -- start start 2024-04-25T10:05:50.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa926f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa406b070 0x7f4fa4070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa926f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa40b4880 con 0x7f4fa415d930 2024-04-25T10:05:50.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa926f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa415e180 con 0x7f4fa406ac10 2024-04-25T10:05:50.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa926f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa415e460 con 0x7f4fa406b070 2024-04-25T10:05:50.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa2ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4fa415d930 0x7f4fa40b4170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45472/0 (socket says 172.21.15.60:45472) 2024-04-25T10:05:50.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa2ffd640 1 -- 172.21.15.60:0/4023267564 learned_addr learned my addr 172.21.15.60:0/4023267564 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:50.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa37fe640 1 --2- 172.21.15.60:0/4023267564 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa406b070 0x7f4fa4070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa1ffb640 1 -- 172.21.15.60:0/4023267564 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564575563 0 0) 0x7f4fa40b4880 con 0x7f4fa415d930 2024-04-25T10:05:50.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa1ffb640 1 -- 172.21.15.60:0/4023267564 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f8c003530 con 0x7f4fa415d930 2024-04-25T10:05:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa1ffb640 1 -- 172.21.15.60:0/4023267564 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3679569940 0 0) 0x7f4f8c003530 con 0x7f4fa415d930 2024-04-25T10:05:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa1ffb640 1 -- 172.21.15.60:0/4023267564 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa406b070 msgr2=0x7f4fa4070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa1ffb640 1 --2- 172.21.15.60:0/4023267564 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa406b070 0x7f4fa4070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa1ffb640 1 -- 172.21.15.60:0/4023267564 >> v1:172.21.15.100:6789/0 conn(0x7f4fa406ac10 legacy=0x7f4fa406fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa1ffb640 1 -- 172.21.15.60:0/4023267564 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa415e740 con 0x7f4fa415d930 2024-04-25T10:05:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa1ffb640 1 -- 172.21.15.60:0/4023267564 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f4f94001680 con 0x7f4fa415d930 2024-04-25T10:05:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa1ffb640 1 -- 172.21.15.60:0/4023267564 >> v1:172.21.15.60:6789/0 conn(0x7f4fa415d930 legacy=0x7f4fa40b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa1ffb640 1 --2- 172.21.15.60:0/4023267564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f8c0039f0 0x7f4f8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa1ffb640 1 --2- 172.21.15.60:0/4023267564 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa406b070 0x7f4f8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa1ffb640 1 --2- 172.21.15.60:0/4023267564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f8c004d50 0x7f4f8c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa1ffb640 1 -- 172.21.15.60:0/4023267564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa415e460 con 0x7f4f8c0039f0 2024-04-25T10:05:50.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa1ffb640 1 -- 172.21.15.60:0/4023267564 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c00ba80 con 0x7f4f8c004d50 2024-04-25T10:05:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa1ffb640 1 -- 172.21.15.60:0/4023267564 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c00bc30 con 0x7f4fa406b070 2024-04-25T10:05:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa2ffd640 1 --2- 172.21.15.60:0/4023267564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f8c0039f0 0x7f4f8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa27fc640 1 --2- 172.21.15.60:0/4023267564 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa406b070 0x7f4f8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.299+0000 7f4fa37fe640 1 --2- 172.21.15.60:0/4023267564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f8c004d50 0x7f4f8c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.303+0000 7f4fa2ffd640 1 -- 172.21.15.60:0/4023267564 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa406b070 msgr2=0x7f4f8c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.303+0000 7f4fa2ffd640 1 --2- 172.21.15.60:0/4023267564 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa406b070 0x7f4f8c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.303+0000 7f4fa2ffd640 1 -- 172.21.15.60:0/4023267564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f8c004d50 msgr2=0x7f4f8c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.303+0000 7f4fa2ffd640 1 --2- 172.21.15.60:0/4023267564 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f8c004d50 0x7f4f8c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.303+0000 7f4fa2ffd640 1 -- 172.21.15.60:0/4023267564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c003530 con 0x7f4f8c0039f0 2024-04-25T10:05:50.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.303+0000 7f4fa2ffd640 1 --2- 172.21.15.60:0/4023267564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f8c0039f0 0x7f4f8c003dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400c160 tx=0x7f4f9401cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c4c36cdae45e36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.303+0000 7f4fa1ffb640 1 -- 172.21.15.60:0/4023267564 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f94007c90 con 0x7f4f8c0039f0 2024-04-25T10:05:50.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.303+0000 7f4fa1ffb640 1 -- 172.21.15.60:0/4023267564 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f940a9070 con 0x7f4f8c0039f0 2024-04-25T10:05:50.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.303+0000 7f4fa1ffb640 1 -- 172.21.15.60:0/4023267564 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f94005d40 con 0x7f4f8c0039f0 2024-04-25T10:05:50.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.303+0000 7f4fa926f640 1 -- 172.21.15.60:0/4023267564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f8c0039f0 msgr2=0x7f4f8c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.303+0000 7f4fa926f640 1 --2- 172.21.15.60:0/4023267564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f8c0039f0 0x7f4f8c003dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400c160 tx=0x7f4f9401cbb0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.303+0000 7f4fa37fe640 1 -- 172.21.15.60:0/4023267564 reap_dead start 2024-04-25T10:05:50.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.303+0000 7f4fa926f640 1 -- 172.21.15.60:0/4023267564 shutdown_connections 2024-04-25T10:05:50.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.303+0000 7f4fa926f640 1 -- 172.21.15.60:0/4023267564 >> 172.21.15.60:0/4023267564 conn(0x7f4fa40b2360 msgr2=0x7f4fa40b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.303+0000 7f4fa926f640 1 -- 172.21.15.60:0/4023267564 shutdown_connections 2024-04-25T10:05:50.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.303+0000 7f4fa926f640 1 -- 172.21.15.60:0/4023267564 wait complete. 2024-04-25T10:05:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa926f640 1 Processor -- start 2024-04-25T10:05:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa926f640 1 -- start start 2024-04-25T10:05:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa926f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f8c0039f0 0x7f4fa4156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa926f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa4153480 0x7f4fa415aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa926f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa41538c0 0x7f4fa41f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa926f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa40b37b0 con 0x7f4f8c0039f0 2024-04-25T10:05:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa926f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa406b450 con 0x7f4fa4153480 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa926f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa40b3930 con 0x7f4fa41538c0 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f8c0039f0 0x7f4fa4156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f8c0039f0 0x7f4fa4156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54224/0 (socket says 172.21.15.60:54224) 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa2ffd640 1 -- 172.21.15.60:0/3664102103 learned_addr learned my addr 172.21.15.60:0/3664102103 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa27fc640 1 --2- 172.21.15.60:0/3664102103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa4153480 0x7f4fa415aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa37fe640 1 --2- 172.21.15.60:0/3664102103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa41538c0 0x7f4fa41f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa2ffd640 1 -- 172.21.15.60:0/3664102103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa41538c0 msgr2=0x7f4fa41f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa2ffd640 1 --2- 172.21.15.60:0/3664102103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa41538c0 0x7f4fa41f6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa2ffd640 1 -- 172.21.15.60:0/3664102103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa4153480 msgr2=0x7f4fa415aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa2ffd640 1 --2- 172.21.15.60:0/3664102103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa4153480 0x7f4fa415aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa2ffd640 1 -- 172.21.15.60:0/3664102103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa4159bb0 con 0x7f4f8c0039f0 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa2ffd640 1 --2- 172.21.15.60:0/3664102103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f8c0039f0 0x7f4fa4156790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f9401c620 tx=0x7f4f8c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4f8b7fe640 1 -- 172.21.15.60:0/3664102103 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f94005d20 con 0x7f4f8c0039f0 2024-04-25T10:05:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4f8b7fe640 1 -- 172.21.15.60:0/3664102103 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f940a58b0 con 0x7f4f8c0039f0 2024-04-25T10:05:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa926f640 1 -- 172.21.15.60:0/3664102103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa41598d0 con 0x7f4f8c0039f0 2024-04-25T10:05:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4f8b7fe640 1 -- 172.21.15.60:0/3664102103 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f940a5c60 con 0x7f4f8c0039f0 2024-04-25T10:05:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa926f640 1 -- 172.21.15.60:0/3664102103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa4155f90 con 0x7f4f8c0039f0 2024-04-25T10:05:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4fa926f640 1 -- 172.21.15.60:0/3664102103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa4074770 con 0x7f4f8c0039f0 2024-04-25T10:05:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4f8b7fe640 1 -- 172.21.15.60:0/3664102103 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f940047f0 con 0x7f4f8c0039f0 2024-04-25T10:05:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.307+0000 7f4f8b7fe640 1 --2- 172.21.15.60:0/3664102103 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4f90066f60 0x7f4f90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.311+0000 7f4f8b7fe640 1 -- 172.21.15.60:0/3664102103 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f9411e630 con 0x7f4f8c0039f0 2024-04-25T10:05:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.311+0000 7f4fa27fc640 1 --2- 172.21.15.60:0/3664102103 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4f90066f60 0x7f4f90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.311+0000 7f4fa27fc640 1 --2- 172.21.15.60:0/3664102103 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4f90066f60 0x7f4f90069420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4fa415d670 tx=0x7f4f98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.315+0000 7f4f8b7fe640 1 -- 172.21.15.60:0/3664102103 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4f940168f0 con 0x7f4f8c0039f0 2024-04-25T10:05:50.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.475+0000 7f4fa926f640 1 -- 172.21.15.60:0/3664102103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4fa40b2810 con 0x7f4f8c0039f0 2024-04-25T10:05:50.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.479+0000 7f4f8b7fe640 1 -- 172.21.15.60:0/3664102103 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4f940ef510 con 0x7f4f8c0039f0 2024-04-25T10:05:50.484 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:05:50.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.479+0000 7f4fa926f640 1 -- 172.21.15.60:0/3664102103 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4f90066f60 msgr2=0x7f4f90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.479+0000 7f4fa926f640 1 --2- 172.21.15.60:0/3664102103 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4f90066f60 0x7f4f90069420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4fa415d670 tx=0x7f4f98009040 comp rx=0 tx=0).stop 2024-04-25T10:05:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.479+0000 7f4fa926f640 1 -- 172.21.15.60:0/3664102103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f8c0039f0 msgr2=0x7f4fa4156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.479+0000 7f4fa926f640 1 --2- 172.21.15.60:0/3664102103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f8c0039f0 0x7f4fa4156790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f9401c620 tx=0x7f4f8c005840 comp rx=0 tx=0).stop 2024-04-25T10:05:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.479+0000 7f4fa926f640 1 -- 172.21.15.60:0/3664102103 shutdown_connections 2024-04-25T10:05:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.479+0000 7f4fa926f640 1 --2- 172.21.15.60:0/3664102103 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4f90066f60 0x7f4f90069420 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.479+0000 7f4fa926f640 1 --2- 172.21.15.60:0/3664102103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa41538c0 0x7f4fa41f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.479+0000 7f4fa926f640 1 --2- 172.21.15.60:0/3664102103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa4153480 0x7f4fa415aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.479+0000 7f4fa926f640 1 --2- 172.21.15.60:0/3664102103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f8c0039f0 0x7f4fa4156790 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.479+0000 7f4fa926f640 1 -- 172.21.15.60:0/3664102103 >> 172.21.15.60:0/3664102103 conn(0x7f4fa40b2360 msgr2=0x7f4fa40afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.479+0000 7f4fa926f640 1 -- 172.21.15.60:0/3664102103 shutdown_connections 2024-04-25T10:05:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.479+0000 7f4fa926f640 1 -- 172.21.15.60:0/3664102103 wait complete. 2024-04-25T10:05:50.503 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:50.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.615+0000 7f4e70c77640 1 Processor -- start 2024-04-25T10:05:50.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.615+0000 7f4e70c77640 1 -- start start 2024-04-25T10:05:50.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e70c77640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e6c1620d0 0x7f4e6c0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e70c77640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e6c0ced20 0x7f4e6c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e70c77640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c15bad0 con 0x7f4e6c15b6f0 2024-04-25T10:05:50.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e70c77640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6c0cf190 con 0x7f4e6c1620d0 2024-04-25T10:05:50.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e70c77640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c161c40 con 0x7f4e6c0ced20 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e63fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e6c1620d0 0x7f4e6c0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e63fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e6c1620d0 0x7f4e6c0ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54234/0 (socket says 172.21.15.60:54234) 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e63fff640 1 -- 172.21.15.60:0/55425763 learned_addr learned my addr 172.21.15.60:0/55425763 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e6ad9d640 1 --2- 172.21.15.60:0/55425763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e6c0ced20 0x7f4e6c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e63fff640 1 -- 172.21.15.60:0/55425763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e6c0ced20 msgr2=0x7f4e6c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e63fff640 1 --2- 172.21.15.60:0/55425763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e6c0ced20 0x7f4e6c0d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e63fff640 1 -- 172.21.15.60:0/55425763 >> v1:172.21.15.100:6789/0 conn(0x7f4e6c15b6f0 legacy=0x7f4e6c15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e63fff640 1 -- 172.21.15.60:0/55425763 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c15ebe0 con 0x7f4e6c1620d0 2024-04-25T10:05:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e6ad9d640 1 --2- 172.21.15.60:0/55425763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e6c0ced20 0x7f4e6c0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:50.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e63fff640 1 --2- 172.21.15.60:0/55425763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e6c1620d0 0x7f4e6c0ce7e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009870 tx=0x7f4e5401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2905e0965da09c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e69d9b640 1 -- 172.21.15.60:0/55425763 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e540b2070 con 0x7f4e6c1620d0 2024-04-25T10:05:50.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e69d9b640 1 -- 172.21.15.60:0/55425763 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e540ad640 con 0x7f4e6c1620d0 2024-04-25T10:05:50.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.619+0000 7f4e69d9b640 1 -- 172.21.15.60:0/55425763 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e540ada10 con 0x7f4e6c1620d0 2024-04-25T10:05:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 -- 172.21.15.60:0/55425763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e6c1620d0 msgr2=0x7f4e6c0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 --2- 172.21.15.60:0/55425763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e6c1620d0 0x7f4e6c0ce7e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009870 tx=0x7f4e5401c520 comp rx=0 tx=0).stop 2024-04-25T10:05:50.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 -- 172.21.15.60:0/55425763 shutdown_connections 2024-04-25T10:05:50.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 --2- 172.21.15.60:0/55425763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e6c1620d0 0x7f4e6c0ce7e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 --2- 172.21.15.60:0/55425763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e6c0ced20 0x7f4e6c0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 -- 172.21.15.60:0/55425763 >> 172.21.15.60:0/55425763 conn(0x7f4e6c072340 msgr2=0x7f4e6c072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 -- 172.21.15.60:0/55425763 shutdown_connections 2024-04-25T10:05:50.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 -- 172.21.15.60:0/55425763 wait complete. 2024-04-25T10:05:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 Processor -- start 2024-04-25T10:05:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 -- start start 2024-04-25T10:05:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e6c0ced20 0x7f4e6c14fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e6c15b6f0 0x7f4e6c0c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e6c1620d0 0x7f4e6c0c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0b96d0 con 0x7f4e6c15b6f0 2024-04-25T10:05:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0b9550 con 0x7f4e6c1620d0 2024-04-25T10:05:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c15e7e0 con 0x7f4e6c0ced20 2024-04-25T10:05:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e63fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e6c15b6f0 0x7f4e6c0c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e63fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e6c15b6f0 0x7f4e6c0c90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54250/0 (socket says 172.21.15.60:54250) 2024-04-25T10:05:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e63fff640 1 -- 172.21.15.60:0/1619020628 learned_addr learned my addr 172.21.15.60:0/1619020628 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e6ad9d640 1 --2- 172.21.15.60:0/1619020628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e6c1620d0 0x7f4e6c0c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e6a59c640 1 --2- 172.21.15.60:0/1619020628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e6c0ced20 0x7f4e6c14fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e63fff640 1 -- 172.21.15.60:0/1619020628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e6c0ced20 msgr2=0x7f4e6c14fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e63fff640 1 --2- 172.21.15.60:0/1619020628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e6c0ced20 0x7f4e6c14fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e63fff640 1 -- 172.21.15.60:0/1619020628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e6c1620d0 msgr2=0x7f4e6c0c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e63fff640 1 --2- 172.21.15.60:0/1619020628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e6c1620d0 0x7f4e6c0c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e63fff640 1 -- 172.21.15.60:0/1619020628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c0b3fb0 con 0x7f4e6c15b6f0 2024-04-25T10:05:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e6a59c640 1 --2- 172.21.15.60:0/1619020628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e6c0ced20 0x7f4e6c14fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e6ad9d640 1 --2- 172.21.15.60:0/1619020628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e6c1620d0 0x7f4e6c0c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:50.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e63fff640 1 --2- 172.21.15.60:0/1619020628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e6c15b6f0 0x7f4e6c0c90b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4e540b0040 tx=0x7f4e540192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e62ffd640 1 -- 172.21.15.60:0/1619020628 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e540b2040 con 0x7f4e6c15b6f0 2024-04-25T10:05:50.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e62ffd640 1 -- 172.21.15.60:0/1619020628 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e540bd070 con 0x7f4e6c15b6f0 2024-04-25T10:05:50.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 -- 172.21.15.60:0/1619020628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6c0b4290 con 0x7f4e6c15b6f0 2024-04-25T10:05:50.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e62ffd640 1 -- 172.21.15.60:0/1619020628 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e540b88c0 con 0x7f4e6c15b6f0 2024-04-25T10:05:50.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 -- 172.21.15.60:0/1619020628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6c0b4850 con 0x7f4e6c15b6f0 2024-04-25T10:05:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.623+0000 7f4e70c77640 1 -- 172.21.15.60:0/1619020628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6c0b4a90 con 0x7f4e6c15b6f0 2024-04-25T10:05:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.627+0000 7f4e62ffd640 1 -- 172.21.15.60:0/1619020628 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4e540b8a60 con 0x7f4e6c15b6f0 2024-04-25T10:05:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.627+0000 7f4e62ffd640 1 --2- 172.21.15.60:0/1619020628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e4c066f60 0x7f4e4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.627+0000 7f4e62ffd640 1 -- 172.21.15.60:0/1619020628 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e5411f810 con 0x7f4e6c15b6f0 2024-04-25T10:05:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.627+0000 7f4e6a59c640 1 --2- 172.21.15.60:0/1619020628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e4c066f60 0x7f4e4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.627+0000 7f4e6a59c640 1 --2- 172.21.15.60:0/1619020628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e4c066f60 0x7f4e4c069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c003e40 tx=0x7f4e5c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.631+0000 7f4e62ffd640 1 -- 172.21.15.60:0/1619020628 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4e6c0b4a90 con 0x7f4e6c15b6f0 2024-04-25T10:05:50.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.791+0000 7f4e70c77640 1 -- 172.21.15.60:0/1619020628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e6c074770 con 0x7f4e6c15b6f0 2024-04-25T10:05:50.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.791+0000 7f4e62ffd640 1 -- 172.21.15.60:0/1619020628 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4e54016110 con 0x7f4e6c15b6f0 2024-04-25T10:05:50.799 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:05:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.795+0000 7f4e70c77640 1 -- 172.21.15.60:0/1619020628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e4c066f60 msgr2=0x7f4e4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.795+0000 7f4e70c77640 1 --2- 172.21.15.60:0/1619020628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e4c066f60 0x7f4e4c069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c003e40 tx=0x7f4e5c009040 comp rx=0 tx=0).stop 2024-04-25T10:05:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.795+0000 7f4e70c77640 1 -- 172.21.15.60:0/1619020628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e6c15b6f0 msgr2=0x7f4e6c0c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.795+0000 7f4e70c77640 1 --2- 172.21.15.60:0/1619020628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e6c15b6f0 0x7f4e6c0c90b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4e540b0040 tx=0x7f4e540192c0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.795+0000 7f4e70c77640 1 -- 172.21.15.60:0/1619020628 shutdown_connections 2024-04-25T10:05:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.795+0000 7f4e70c77640 1 --2- 172.21.15.60:0/1619020628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e4c066f60 0x7f4e4c069420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.795+0000 7f4e70c77640 1 --2- 172.21.15.60:0/1619020628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e6c1620d0 0x7f4e6c0c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.795+0000 7f4e70c77640 1 --2- 172.21.15.60:0/1619020628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e6c15b6f0 0x7f4e6c0c90b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.795+0000 7f4e70c77640 1 --2- 172.21.15.60:0/1619020628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e6c0ced20 0x7f4e6c14fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.795+0000 7f4e70c77640 1 -- 172.21.15.60:0/1619020628 >> 172.21.15.60:0/1619020628 conn(0x7f4e6c072340 msgr2=0x7f4e6c1547d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.795+0000 7f4e70c77640 1 -- 172.21.15.60:0/1619020628 shutdown_connections 2024-04-25T10:05:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.795+0000 7f4e70c77640 1 -- 172.21.15.60:0/1619020628 wait complete. 2024-04-25T10:05:50.818 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:50.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.927+0000 7f640eae2640 1 Processor -- start 2024-04-25T10:05:50.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.927+0000 7f640eae2640 1 -- start start 2024-04-25T10:05:50.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f640eae2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f640815f480 0x7f640815f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f640eae2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64080cd390 0x7f640809a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f640eae2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f640809a930 0x7f64080cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f640eae2640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640815fda0 con 0x7f64080cd390 2024-04-25T10:05:50.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f640eae2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64080cc5d0 con 0x7f640815f480 2024-04-25T10:05:50.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f640eae2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64080cc8b0 con 0x7f640809a930 2024-04-25T10:05:50.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f6407fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64080cd390 0x7f640809a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f640c87e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f640815f480 0x7f640815f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f640d07f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f640809a930 0x7f64080cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f6407fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64080cd390 0x7f640809a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54254/0 (socket says 172.21.15.60:54254) 2024-04-25T10:05:50.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f6407fff640 1 -- 172.21.15.60:0/1495305777 learned_addr learned my addr 172.21.15.60:0/1495305777 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:50.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f6407fff640 1 -- 172.21.15.60:0/1495305777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f640809a930 msgr2=0x7f64080cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:50.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f6407fff640 1 --2- 172.21.15.60:0/1495305777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f640809a930 0x7f64080cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f6407fff640 1 -- 172.21.15.60:0/1495305777 >> v2:172.21.15.100:3300/0 conn(0x7f640815f480 msgr2=0x7f640815f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:50.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f6407fff640 1 --2- 172.21.15.60:0/1495305777 >> v2:172.21.15.100:3300/0 conn(0x7f640815f480 0x7f640815f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f6407fff640 1 -- 172.21.15.60:0/1495305777 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64080ccb90 con 0x7f64080cd390 2024-04-25T10:05:50.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f6407fff640 1 --2- 172.21.15.60:0/1495305777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080cd390 0x7f640809a3f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f640806b860 tx=0x7f63fc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ce70fe85814c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f64077fe640 1 -- 172.21.15.60:0/1495305777 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0b1070 con 0x7f64080cd390 2024-04-25T10:05:50.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f64077fe640 1 -- 172.21.15.60:0/1495305777 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63fc0ac620 con 0x7f64080cd390 2024-04-25T10:05:50.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.931+0000 7f64077fe640 1 -- 172.21.15.60:0/1495305777 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0ac9f0 con 0x7f64080cd390 2024-04-25T10:05:50.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 -- 172.21.15.60:0/1495305777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080cd390 msgr2=0x7f640809a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 --2- 172.21.15.60:0/1495305777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080cd390 0x7f640809a3f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f640806b860 tx=0x7f63fc01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 -- 172.21.15.60:0/1495305777 shutdown_connections 2024-04-25T10:05:50.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 --2- 172.21.15.60:0/1495305777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f640809a930 0x7f64080cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 --2- 172.21.15.60:0/1495305777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080cd390 0x7f640809a3f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 --2- 172.21.15.60:0/1495305777 >> v2:172.21.15.100:3300/0 conn(0x7f640815f480 0x7f640815f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 -- 172.21.15.60:0/1495305777 >> 172.21.15.60:0/1495305777 conn(0x7f64080b32a0 msgr2=0x7f64080b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 -- 172.21.15.60:0/1495305777 shutdown_connections 2024-04-25T10:05:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 -- 172.21.15.60:0/1495305777 wait complete. 2024-04-25T10:05:50.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 Processor -- start 2024-04-25T10:05:50.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 -- start start 2024-04-25T10:05:50.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640809a930 0x7f640815c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080cd390 0x7f64080cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f640815f480 0x7f6408150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640815bd30 con 0x7f64080cd390 2024-04-25T10:05:50.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640815bbb0 con 0x7f640809a930 2024-04-25T10:05:50.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640815beb0 con 0x7f640815f480 2024-04-25T10:05:50.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f6407fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080cd390 0x7f64080cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f6407fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080cd390 0x7f64080cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54262/0 (socket says 172.21.15.60:54262) 2024-04-25T10:05:50.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f6407fff640 1 -- 172.21.15.60:0/82264848 learned_addr learned my addr 172.21.15.60:0/82264848 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:50.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640c87e640 1 --2- 172.21.15.60:0/82264848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640809a930 0x7f640815c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640d07f640 1 --2- 172.21.15.60:0/82264848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f640815f480 0x7f6408150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f6407fff640 1 -- 172.21.15.60:0/82264848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f640815f480 msgr2=0x7f6408150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f6407fff640 1 --2- 172.21.15.60:0/82264848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f640815f480 0x7f6408150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f6407fff640 1 -- 172.21.15.60:0/82264848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640809a930 msgr2=0x7f640815c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f6407fff640 1 --2- 172.21.15.60:0/82264848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640809a930 0x7f640815c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f6407fff640 1 -- 172.21.15.60:0/82264848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6408151300 con 0x7f64080cd390 2024-04-25T10:05:50.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f6407fff640 1 --2- 172.21.15.60:0/82264848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080cd390 0x7f64080cae50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f63fc01aba0 tx=0x7f63fc019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f64057fa640 1 -- 172.21.15.60:0/82264848 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0b1040 con 0x7f64080cd390 2024-04-25T10:05:50.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f64057fa640 1 -- 172.21.15.60:0/82264848 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63fc0ba070 con 0x7f64080cd390 2024-04-25T10:05:50.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 -- 172.21.15.60:0/82264848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640815c9b0 con 0x7f64080cd390 2024-04-25T10:05:50.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f64057fa640 1 -- 172.21.15.60:0/82264848 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0b5660 con 0x7f64080cd390 2024-04-25T10:05:50.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.935+0000 7f640eae2640 1 -- 172.21.15.60:0/82264848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640815cf70 con 0x7f64080cd390 2024-04-25T10:05:50.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.939+0000 7f640eae2640 1 -- 172.21.15.60:0/82264848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6408074770 con 0x7f64080cd390 2024-04-25T10:05:50.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.939+0000 7f64057fa640 1 -- 172.21.15.60:0/82264848 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63fc0b5800 con 0x7f64080cd390 2024-04-25T10:05:50.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.939+0000 7f64057fa640 1 --2- 172.21.15.60:0/82264848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63f4066f60 0x7f63f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:50.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.939+0000 7f64057fa640 1 -- 172.21.15.60:0/82264848 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63fc11ed70 con 0x7f64080cd390 2024-04-25T10:05:50.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.939+0000 7f640c87e640 1 --2- 172.21.15.60:0/82264848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63f4066f60 0x7f63f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:50.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.939+0000 7f640c87e640 1 --2- 172.21.15.60:0/82264848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63f4066f60 0x7f63f4069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f64080c8130 tx=0x7f63f800f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:50.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:50.943+0000 7f64057fa640 1 -- 172.21.15.60:0/82264848 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f63fc016110 con 0x7f64080cd390 2024-04-25T10:05:51.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.099+0000 7f640eae2640 1 -- 172.21.15.60:0/82264848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64080b40a0 con 0x7f64080cd390 2024-04-25T10:05:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.103+0000 7f64057fa640 1 -- 172.21.15.60:0/82264848 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f63fc0efe10 con 0x7f64080cd390 2024-04-25T10:05:51.108 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:51.108 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:22.036582+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:51.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.103+0000 7f640eae2640 1 -- 172.21.15.60:0/82264848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63f4066f60 msgr2=0x7f63f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.103+0000 7f640eae2640 1 --2- 172.21.15.60:0/82264848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63f4066f60 0x7f63f4069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f64080c8130 tx=0x7f63f800f040 comp rx=0 tx=0).stop 2024-04-25T10:05:51.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.103+0000 7f640eae2640 1 -- 172.21.15.60:0/82264848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080cd390 msgr2=0x7f64080cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.103+0000 7f640eae2640 1 --2- 172.21.15.60:0/82264848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080cd390 0x7f64080cae50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f63fc01aba0 tx=0x7f63fc019290 comp rx=0 tx=0).stop 2024-04-25T10:05:51.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.103+0000 7f640eae2640 1 -- 172.21.15.60:0/82264848 shutdown_connections 2024-04-25T10:05:51.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.103+0000 7f640eae2640 1 --2- 172.21.15.60:0/82264848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63f4066f60 0x7f63f4069420 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.103+0000 7f640eae2640 1 --2- 172.21.15.60:0/82264848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f640815f480 0x7f6408150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.103+0000 7f640eae2640 1 --2- 172.21.15.60:0/82264848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080cd390 0x7f64080cae50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.103+0000 7f640eae2640 1 --2- 172.21.15.60:0/82264848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640809a930 0x7f640815c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.103+0000 7f640eae2640 1 -- 172.21.15.60:0/82264848 >> 172.21.15.60:0/82264848 conn(0x7f64080b32a0 msgr2=0x7f64080b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.103+0000 7f640eae2640 1 -- 172.21.15.60:0/82264848 shutdown_connections 2024-04-25T10:05:51.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.103+0000 7f640eae2640 1 -- 172.21.15.60:0/82264848 wait complete. 2024-04-25T10:05:51.126 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:18.976643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:19.405302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:51.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:51.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.243+0000 7f28730be640 1 Processor -- start 2024-04-25T10:05:51.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f28730be640 1 -- start start 2024-04-25T10:05:51.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f28730be640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f286c1549f0 0x7f286c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f28730be640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286c0ad770 0x7f286c06bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f28730be640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f286c0ae4b0 con 0x7f286c06c160 2024-04-25T10:05:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f28730be640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f286c0adc10 con 0x7f286c1549f0 2024-04-25T10:05:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f28730be640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286c0adef0 con 0x7f286c0ad770 2024-04-25T10:05:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f2870e5a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f286c1549f0 0x7f286c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f2870e5a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f286c1549f0 0x7f286c0ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54270/0 (socket says 172.21.15.60:54270) 2024-04-25T10:05:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f2870e5a640 1 -- 172.21.15.60:0/879568049 learned_addr learned my addr 172.21.15.60:0/879568049 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f286bfff640 1 --2- 172.21.15.60:0/879568049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286c0ad770 0x7f286c06bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f2870e5a640 1 -- 172.21.15.60:0/879568049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286c0ad770 msgr2=0x7f286c06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f2870e5a640 1 --2- 172.21.15.60:0/879568049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286c0ad770 0x7f286c06bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f2870e5a640 1 -- 172.21.15.60:0/879568049 >> v1:172.21.15.100:6789/0 conn(0x7f286c06c160 legacy=0x7f286c161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f2870e5a640 1 -- 172.21.15.60:0/879568049 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286c0ae1d0 con 0x7f286c1549f0 2024-04-25T10:05:51.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f2870e5a640 1 --2- 172.21.15.60:0/879568049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c1549f0 0x7f286c0ad230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2854009870 tx=0x7f285401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584971249a21c335 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f286b7fe640 1 -- 172.21.15.60:0/879568049 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28540a7070 con 0x7f286c1549f0 2024-04-25T10:05:51.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f286b7fe640 1 -- 172.21.15.60:0/879568049 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28540b0070 con 0x7f286c1549f0 2024-04-25T10:05:51.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.247+0000 7f286b7fe640 1 -- 172.21.15.60:0/879568049 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28540ab890 con 0x7f286c1549f0 2024-04-25T10:05:51.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 -- 172.21.15.60:0/879568049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c1549f0 msgr2=0x7f286c0ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 --2- 172.21.15.60:0/879568049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c1549f0 0x7f286c0ad230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2854009870 tx=0x7f285401c560 comp rx=0 tx=0).stop 2024-04-25T10:05:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 -- 172.21.15.60:0/879568049 shutdown_connections 2024-04-25T10:05:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 --2- 172.21.15.60:0/879568049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286c0ad770 0x7f286c06bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 --2- 172.21.15.60:0/879568049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c1549f0 0x7f286c0ad230 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 -- 172.21.15.60:0/879568049 >> 172.21.15.60:0/879568049 conn(0x7f286c071b90 msgr2=0x7f286c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 -- 172.21.15.60:0/879568049 shutdown_connections 2024-04-25T10:05:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 -- 172.21.15.60:0/879568049 wait complete. 2024-04-25T10:05:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 Processor -- start 2024-04-25T10:05:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 -- start start 2024-04-25T10:05:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f286c06c160 0x7f286c14fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c0ad770 0x7f286c15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286c1549f0 0x7f286c156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c16aa70 con 0x7f286c0ad770 2024-04-25T10:05:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c16a8f0 con 0x7f286c06c160 2024-04-25T10:05:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286c16abf0 con 0x7f286c1549f0 2024-04-25T10:05:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f286bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c0ad770 0x7f286c15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f286bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c0ad770 0x7f286c15fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54276/0 (socket says 172.21.15.60:54276) 2024-04-25T10:05:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f286bfff640 1 -- 172.21.15.60:0/3223477220 learned_addr learned my addr 172.21.15.60:0/3223477220 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:51.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f2870e5a640 1 --2- 172.21.15.60:0/3223477220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f286c06c160 0x7f286c14fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f287165b640 1 --2- 172.21.15.60:0/3223477220 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286c1549f0 0x7f286c156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f286bfff640 1 -- 172.21.15.60:0/3223477220 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286c1549f0 msgr2=0x7f286c156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f286bfff640 1 --2- 172.21.15.60:0/3223477220 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286c1549f0 0x7f286c156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f286bfff640 1 -- 172.21.15.60:0/3223477220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f286c06c160 msgr2=0x7f286c14fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f286bfff640 1 --2- 172.21.15.60:0/3223477220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f286c06c160 0x7f286c14fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f286bfff640 1 -- 172.21.15.60:0/3223477220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286c14c9e0 con 0x7f286c0ad770 2024-04-25T10:05:51.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f286bfff640 1 --2- 172.21.15.60:0/3223477220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c0ad770 0x7f286c15fd40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f285c00d800 tx=0x7f285c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28697fa640 1 -- 172.21.15.60:0/3223477220 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c00b6c0 con 0x7f286c0ad770 2024-04-25T10:05:51.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28697fa640 1 -- 172.21.15.60:0/3223477220 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c019070 con 0x7f286c0ad770 2024-04-25T10:05:51.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 -- 172.21.15.60:0/3223477220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286c14ccc0 con 0x7f286c0ad770 2024-04-25T10:05:51.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28697fa640 1 -- 172.21.15.60:0/3223477220 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c00ba60 con 0x7f286c0ad770 2024-04-25T10:05:51.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.251+0000 7f28730be640 1 -- 172.21.15.60:0/3223477220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286c14d280 con 0x7f286c0ad770 2024-04-25T10:05:51.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.255+0000 7f28730be640 1 -- 172.21.15.60:0/3223477220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286c074770 con 0x7f286c0ad770 2024-04-25T10:05:51.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.255+0000 7f28697fa640 1 -- 172.21.15.60:0/3223477220 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f285c00bc00 con 0x7f286c0ad770 2024-04-25T10:05:51.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.255+0000 7f28697fa640 1 --2- 172.21.15.60:0/3223477220 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f284c066f60 0x7f284c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.255+0000 7f28697fa640 1 -- 172.21.15.60:0/3223477220 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f285c085430 con 0x7f286c0ad770 2024-04-25T10:05:51.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.255+0000 7f2870e5a640 1 --2- 172.21.15.60:0/3223477220 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f284c066f60 0x7f284c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.255+0000 7f2870e5a640 1 --2- 172.21.15.60:0/3223477220 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f284c066f60 0x7f284c069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f28540af000 tx=0x7f28540a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.259+0000 7f28697fa640 1 -- 172.21.15.60:0/3223477220 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f285c056440 con 0x7f286c0ad770 2024-04-25T10:05:51.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.415+0000 7f28730be640 1 -- 172.21.15.60:0/3223477220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f286c072780 con 0x7f286c0ad770 2024-04-25T10:05:51.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.415+0000 7f28697fa640 1 -- 172.21.15.60:0/3223477220 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f285c055d40 con 0x7f286c0ad770 2024-04-25T10:05:51.422 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:05:51.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.415+0000 7f28730be640 1 -- 172.21.15.60:0/3223477220 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f284c066f60 msgr2=0x7f284c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.415+0000 7f28730be640 1 --2- 172.21.15.60:0/3223477220 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f284c066f60 0x7f284c069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f28540af000 tx=0x7f28540a7000 comp rx=0 tx=0).stop 2024-04-25T10:05:51.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.415+0000 7f28730be640 1 -- 172.21.15.60:0/3223477220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c0ad770 msgr2=0x7f286c15fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.415+0000 7f28730be640 1 --2- 172.21.15.60:0/3223477220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c0ad770 0x7f286c15fd40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f285c00d800 tx=0x7f285c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.415+0000 7f28730be640 1 -- 172.21.15.60:0/3223477220 shutdown_connections 2024-04-25T10:05:51.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.415+0000 7f28730be640 1 --2- 172.21.15.60:0/3223477220 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f284c066f60 0x7f284c069420 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.415+0000 7f28730be640 1 --2- 172.21.15.60:0/3223477220 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286c1549f0 0x7f286c156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.415+0000 7f28730be640 1 --2- 172.21.15.60:0/3223477220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c0ad770 0x7f286c15fd40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.415+0000 7f28730be640 1 --2- 172.21.15.60:0/3223477220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f286c06c160 0x7f286c14fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.415+0000 7f28730be640 1 -- 172.21.15.60:0/3223477220 >> 172.21.15.60:0/3223477220 conn(0x7f286c071b90 msgr2=0x7f286c15f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.415+0000 7f28730be640 1 -- 172.21.15.60:0/3223477220 shutdown_connections 2024-04-25T10:05:51.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.415+0000 7f28730be640 1 -- 172.21.15.60:0/3223477220 wait complete. 2024-04-25T10:05:51.439 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:51.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.559+0000 7fd91e1f8640 1 Processor -- start 2024-04-25T10:05:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.559+0000 7fd91e1f8640 1 -- start start 2024-04-25T10:05:51.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.563+0000 7fd91e1f8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd91815dd30 0x7fd91815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.563+0000 7fd91e1f8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180cf8a0 0x7fd91815be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.563+0000 7fd91e1f8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9180cd890 con 0x7fd9180cd3d0 2024-04-25T10:05:51.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.563+0000 7fd91e1f8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9180ce4a0 con 0x7fd91815dd30 2024-04-25T10:05:51.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.563+0000 7fd91e1f8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180cdb70 con 0x7fd9180cf8a0 2024-04-25T10:05:51.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.563+0000 7fd9177fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd91815dd30 0x7fd91815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.563+0000 7fd917fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180cf8a0 0x7fd91815be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.563+0000 7fd9177fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd91815dd30 0x7fd91815ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:55282/0 (socket says 172.21.15.60:55282) 2024-04-25T10:05:51.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.563+0000 7fd9177fe640 1 -- 172.21.15.60:0/2876051734 learned_addr learned my addr 172.21.15.60:0/2876051734 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:51.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.563+0000 7fd9177fe640 1 -- 172.21.15.60:0/2876051734 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180cf8a0 msgr2=0x7fd91815be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:51.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.563+0000 7fd9177fe640 1 --2- 172.21.15.60:0/2876051734 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180cf8a0 0x7fd91815be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.563+0000 7fd9177fe640 1 -- 172.21.15.60:0/2876051734 >> v1:172.21.15.100:6789/0 conn(0x7fd9180cd3d0 legacy=0x7fd9180cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.563+0000 7fd9177fe640 1 -- 172.21.15.60:0/2876051734 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9180cde50 con 0x7fd91815dd30 2024-04-25T10:05:51.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd9177fe640 1 --2- 172.21.15.60:0/2876051734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd91815dd30 0x7fd91815ac30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd908009870 tx=0x7fd90801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=886c9fa48519aa88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd9167fc640 1 -- 172.21.15.60:0/2876051734 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080aa620 con 0x7fd91815dd30 2024-04-25T10:05:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd9167fc640 1 -- 172.21.15.60:0/2876051734 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9080af070 con 0x7fd91815dd30 2024-04-25T10:05:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd9167fc640 1 -- 172.21.15.60:0/2876051734 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080aa9e0 con 0x7fd91815dd30 2024-04-25T10:05:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd91e1f8640 1 -- 172.21.15.60:0/2876051734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd91815dd30 msgr2=0x7fd91815ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd91e1f8640 1 --2- 172.21.15.60:0/2876051734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd91815dd30 0x7fd91815ac30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd908009870 tx=0x7fd90801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd91e1f8640 1 -- 172.21.15.60:0/2876051734 shutdown_connections 2024-04-25T10:05:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd91e1f8640 1 --2- 172.21.15.60:0/2876051734 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180cf8a0 0x7fd91815be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd91e1f8640 1 --2- 172.21.15.60:0/2876051734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd91815dd30 0x7fd91815ac30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd91e1f8640 1 -- 172.21.15.60:0/2876051734 >> 172.21.15.60:0/2876051734 conn(0x7fd9180670b0 msgr2=0x7fd91806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd91e1f8640 1 -- 172.21.15.60:0/2876051734 shutdown_connections 2024-04-25T10:05:51.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd91e1f8640 1 -- 172.21.15.60:0/2876051734 wait complete. 2024-04-25T10:05:51.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd91e1f8640 1 Processor -- start 2024-04-25T10:05:51.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd91e1f8640 1 -- start start 2024-04-25T10:05:51.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd91e1f8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9180cd3d0 0x7fd9181490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd91e1f8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9180cf8a0 0x7fd9180b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd91e1f8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd91815dd30 0x7fd9180af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd91e1f8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91815cf00 con 0x7fd9180cf8a0 2024-04-25T10:05:51.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd91e1f8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91815cd80 con 0x7fd9180cd3d0 2024-04-25T10:05:51.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd91e1f8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91815d080 con 0x7fd91815dd30 2024-04-25T10:05:51.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd916ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9180cf8a0 0x7fd9180b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd916ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9180cf8a0 0x7fd9180b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54292/0 (socket says 172.21.15.60:54292) 2024-04-25T10:05:51.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd916ffd640 1 -- 172.21.15.60:0/2962147134 learned_addr learned my addr 172.21.15.60:0/2962147134 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:51.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd9177fe640 1 --2- 172.21.15.60:0/2962147134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9180cd3d0 0x7fd9181490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd917fff640 1 --2- 172.21.15.60:0/2962147134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd91815dd30 0x7fd9180af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd916ffd640 1 -- 172.21.15.60:0/2962147134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd91815dd30 msgr2=0x7fd9180af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd916ffd640 1 --2- 172.21.15.60:0/2962147134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd91815dd30 0x7fd9180af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd916ffd640 1 -- 172.21.15.60:0/2962147134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9180cd3d0 msgr2=0x7fd9181490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd916ffd640 1 --2- 172.21.15.60:0/2962147134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9180cd3d0 0x7fd9181490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd916ffd640 1 -- 172.21.15.60:0/2962147134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd918149600 con 0x7fd9180cf8a0 2024-04-25T10:05:51.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd916ffd640 1 --2- 172.21.15.60:0/2962147134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9180cf8a0 0x7fd9180b3e30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00c370 tx=0x7fd90c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd8fffff640 1 -- 172.21.15.60:0/2962147134 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0052d0 con 0x7fd9180cf8a0 2024-04-25T10:05:51.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.567+0000 7fd8fffff640 1 -- 172.21.15.60:0/2962147134 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c005470 con 0x7fd9180cf8a0 2024-04-25T10:05:51.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.571+0000 7fd91e1f8640 1 -- 172.21.15.60:0/2962147134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd918159a40 con 0x7fd9180cf8a0 2024-04-25T10:05:51.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.571+0000 7fd8fffff640 1 -- 172.21.15.60:0/2962147134 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c007260 con 0x7fd9180cf8a0 2024-04-25T10:05:51.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.571+0000 7fd91e1f8640 1 -- 172.21.15.60:0/2962147134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd918158c80 con 0x7fd9180cf8a0 2024-04-25T10:05:51.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.571+0000 7fd91e1f8640 1 -- 172.21.15.60:0/2962147134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd918074770 con 0x7fd9180cf8a0 2024-04-25T10:05:51.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.571+0000 7fd8fffff640 1 -- 172.21.15.60:0/2962147134 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd90c007400 con 0x7fd9180cf8a0 2024-04-25T10:05:51.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.571+0000 7fd8fffff640 1 --2- 172.21.15.60:0/2962147134 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd904066f60 0x7fd904069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.571+0000 7fd8fffff640 1 -- 172.21.15.60:0/2962147134 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd90c084430 con 0x7fd9180cf8a0 2024-04-25T10:05:51.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.571+0000 7fd9177fe640 1 --2- 172.21.15.60:0/2962147134 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd904066f60 0x7fd904069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.571+0000 7fd9177fe640 1 --2- 172.21.15.60:0/2962147134 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd904066f60 0x7fd904069420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd9080ae000 tx=0x7fd9080a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.575+0000 7fd8fffff640 1 -- 172.21.15.60:0/2962147134 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd90c0554d0 con 0x7fd9180cf8a0 2024-04-25T10:05:51.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.735+0000 7fd91e1f8640 1 -- 172.21.15.60:0/2962147134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd91815b4b0 con 0x7fd9180cf8a0 2024-04-25T10:05:51.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.735+0000 7fd8fffff640 1 -- 172.21.15.60:0/2962147134 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd90c054dd0 con 0x7fd9180cf8a0 2024-04-25T10:05:51.744 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:05:51.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.739+0000 7fd91e1f8640 1 -- 172.21.15.60:0/2962147134 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd904066f60 msgr2=0x7fd904069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.739+0000 7fd91e1f8640 1 --2- 172.21.15.60:0/2962147134 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd904066f60 0x7fd904069420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd9080ae000 tx=0x7fd9080a7000 comp rx=0 tx=0).stop 2024-04-25T10:05:51.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.739+0000 7fd91e1f8640 1 -- 172.21.15.60:0/2962147134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9180cf8a0 msgr2=0x7fd9180b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.739+0000 7fd91e1f8640 1 --2- 172.21.15.60:0/2962147134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9180cf8a0 0x7fd9180b3e30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00c370 tx=0x7fd90c00c830 comp rx=0 tx=0).stop 2024-04-25T10:05:51.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.739+0000 7fd91e1f8640 1 -- 172.21.15.60:0/2962147134 shutdown_connections 2024-04-25T10:05:51.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.739+0000 7fd91e1f8640 1 --2- 172.21.15.60:0/2962147134 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd904066f60 0x7fd904069420 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.739+0000 7fd91e1f8640 1 --2- 172.21.15.60:0/2962147134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd91815dd30 0x7fd9180af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.739+0000 7fd91e1f8640 1 --2- 172.21.15.60:0/2962147134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9180cf8a0 0x7fd9180b3e30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.739+0000 7fd91e1f8640 1 --2- 172.21.15.60:0/2962147134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9180cd3d0 0x7fd9181490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.739+0000 7fd91e1f8640 1 -- 172.21.15.60:0/2962147134 >> 172.21.15.60:0/2962147134 conn(0x7fd9180670b0 msgr2=0x7fd9180cb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.739+0000 7fd91e1f8640 1 -- 172.21.15.60:0/2962147134 shutdown_connections 2024-04-25T10:05:51.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.739+0000 7fd91e1f8640 1 -- 172.21.15.60:0/2962147134 wait complete. 2024-04-25T10:05:51.763 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:51.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.887+0000 7efcc2a41640 1 Processor -- start 2024-04-25T10:05:51.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.887+0000 7efcc2a41640 1 -- start start 2024-04-25T10:05:51.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.887+0000 7efcc2a41640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efcbc15fbf0 0x7efcbc166870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.887+0000 7efcc2a41640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcbc144a40 con 0x7efcbc160080 2024-04-25T10:05:51.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.887+0000 7efcc2a41640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcbc0cc9f0 con 0x7efcbc0cbcf0 2024-04-25T10:05:51.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.887+0000 7efcc2a41640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcbc0b2dc0 con 0x7efcbc15fbf0 2024-04-25T10:05:51.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.887+0000 7efcbb7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efcbc160080 0x7efcbc15f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45474/0 (socket says 172.21.15.60:45474) 2024-04-25T10:05:51.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.887+0000 7efcbb7fe640 1 -- 172.21.15.60:0/521790921 learned_addr learned my addr 172.21.15.60:0/521790921 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:51.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.887+0000 7efcc0fde640 1 --2- 172.21.15.60:0/521790921 >> v2:172.21.15.100:3300/0 conn(0x7efcbc15fbf0 0x7efcbc166870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.887+0000 7efcbaffd640 1 -- 172.21.15.60:0/521790921 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813341446 0 0) 0x7efcbc144a40 con 0x7efcbc160080 2024-04-25T10:05:51.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.887+0000 7efcbaffd640 1 -- 172.21.15.60:0/521790921 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efca4003530 con 0x7efcbc160080 2024-04-25T10:05:51.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.887+0000 7efcc0fde640 1 -- 172.21.15.60:0/521790921 >> v1:172.21.15.100:6789/0 conn(0x7efcbc0cbcf0 legacy=0x7efcbc071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.887+0000 7efcc0fde640 1 -- 172.21.15.60:0/521790921 >> v1:172.21.15.60:6789/0 conn(0x7efcbc160080 legacy=0x7efcbc15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.887+0000 7efcc0fde640 1 -- 172.21.15.60:0/521790921 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc0cccd0 con 0x7efcbc15fbf0 2024-04-25T10:05:51.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.891+0000 7efcc0fde640 1 --2- 172.21.15.60:0/521790921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcbc15fbf0 0x7efcbc166870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009d90 tx=0x7efcb401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f062f5dca06eaff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.891+0000 7efcbaffd640 1 -- 172.21.15.60:0/521790921 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb40b2070 con 0x7efcbc15fbf0 2024-04-25T10:05:51.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.891+0000 7efcbaffd640 1 -- 172.21.15.60:0/521790921 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb40a5920 con 0x7efcbc15fbf0 2024-04-25T10:05:51.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.891+0000 7efcbaffd640 1 -- 172.21.15.60:0/521790921 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb40a5cd0 con 0x7efcbc15fbf0 2024-04-25T10:05:51.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.891+0000 7efcc2a41640 1 -- 172.21.15.60:0/521790921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcbc15fbf0 msgr2=0x7efcbc166870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.891+0000 7efcc2a41640 1 --2- 172.21.15.60:0/521790921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcbc15fbf0 0x7efcbc166870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009d90 tx=0x7efcb401c970 comp rx=0 tx=0).stop 2024-04-25T10:05:51.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.891+0000 7efcc2a41640 1 -- 172.21.15.60:0/521790921 shutdown_connections 2024-04-25T10:05:51.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.891+0000 7efcc2a41640 1 --2- 172.21.15.60:0/521790921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcbc15fbf0 0x7efcbc166870 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.891+0000 7efcc2a41640 1 -- 172.21.15.60:0/521790921 >> 172.21.15.60:0/521790921 conn(0x7efcbc0670b0 msgr2=0x7efcbc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:51.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.891+0000 7efcc2a41640 1 -- 172.21.15.60:0/521790921 shutdown_connections 2024-04-25T10:05:51.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.891+0000 7efcc2a41640 1 -- 172.21.15.60:0/521790921 wait complete. 2024-04-25T10:05:51.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcc2a41640 1 Processor -- start 2024-04-25T10:05:51.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcc2a41640 1 -- start start 2024-04-25T10:05:51.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcc2a41640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcbc0cbcf0 0x7efcbc13fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcc2a41640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcbc15fbf0 0x7efcbc142f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcc2a41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcbc160080 0x7efcbc15b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcc2a41640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0cfed0 con 0x7efcbc0cbcf0 2024-04-25T10:05:51.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcc2a41640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0ce5c0 con 0x7efcbc160080 2024-04-25T10:05:51.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcc2a41640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc163ab0 con 0x7efcbc15fbf0 2024-04-25T10:05:51.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcbbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcbc0cbcf0 0x7efcbc13fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcbbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcbc0cbcf0 0x7efcbc13fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54302/0 (socket says 172.21.15.60:54302) 2024-04-25T10:05:51.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcbbfff640 1 -- 172.21.15.60:0/1008109324 learned_addr learned my addr 172.21.15.60:0/1008109324 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:51.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcbb7fe640 1 --2- 172.21.15.60:0/1008109324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcbc15fbf0 0x7efcbc142f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcc0fde640 1 --2- 172.21.15.60:0/1008109324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcbc160080 0x7efcbc15b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcbbfff640 1 -- 172.21.15.60:0/1008109324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcbc15fbf0 msgr2=0x7efcbc142f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcbbfff640 1 --2- 172.21.15.60:0/1008109324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcbc15fbf0 0x7efcbc142f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcbbfff640 1 -- 172.21.15.60:0/1008109324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcbc160080 msgr2=0x7efcbc15b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:51.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcbbfff640 1 --2- 172.21.15.60:0/1008109324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcbc160080 0x7efcbc15b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:51.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcbbfff640 1 -- 172.21.15.60:0/1008109324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc163d90 con 0x7efcbc0cbcf0 2024-04-25T10:05:51.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcc0fde640 1 --2- 172.21.15.60:0/1008109324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcbc160080 0x7efcbc15b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:51.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcbbfff640 1 --2- 172.21.15.60:0/1008109324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcbc0cbcf0 0x7efcbc13fb60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efcb000c370 tx=0x7efcb000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcb8ff9640 1 -- 172.21.15.60:0/1008109324 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb00052d0 con 0x7efcbc0cbcf0 2024-04-25T10:05:51.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcb8ff9640 1 -- 172.21.15.60:0/1008109324 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb0005470 con 0x7efcbc0cbcf0 2024-04-25T10:05:51.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcc2a41640 1 -- 172.21.15.60:0/1008109324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcbc164070 con 0x7efcbc0cbcf0 2024-04-25T10:05:51.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcb8ff9640 1 -- 172.21.15.60:0/1008109324 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb0007260 con 0x7efcbc0cbcf0 2024-04-25T10:05:51.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcc2a41640 1 -- 172.21.15.60:0/1008109324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcbc1646c0 con 0x7efcbc0cbcf0 2024-04-25T10:05:51.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.895+0000 7efcc2a41640 1 -- 172.21.15.60:0/1008109324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcbc06bba0 con 0x7efcbc0cbcf0 2024-04-25T10:05:51.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.899+0000 7efcb8ff9640 1 -- 172.21.15.60:0/1008109324 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efcb0007400 con 0x7efcbc0cbcf0 2024-04-25T10:05:51.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.899+0000 7efcb8ff9640 1 --2- 172.21.15.60:0/1008109324 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efca8066f60 0x7efca8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:51.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.899+0000 7efcb8ff9640 1 -- 172.21.15.60:0/1008109324 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efcb0084490 con 0x7efcbc0cbcf0 2024-04-25T10:05:51.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.899+0000 7efcbb7fe640 1 --2- 172.21.15.60:0/1008109324 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efca8066f60 0x7efca8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:51.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.899+0000 7efcbb7fe640 1 --2- 172.21.15.60:0/1008109324 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efca8066f60 0x7efca8069420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efcbc13d610 tx=0x7efcac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:51.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:51.903+0000 7efcb8ff9640 1 -- 172.21.15.60:0/1008109324 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efcb00554a0 con 0x7efcbc0cbcf0 2024-04-25T10:05:52.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.055+0000 7efcc2a41640 1 -- 172.21.15.60:0/1008109324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efcbc0b2970 con 0x7efcbc0cbcf0 2024-04-25T10:05:52.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.055+0000 7efcb8ff9640 1 -- 172.21.15.60:0/1008109324 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7efcb0054da0 con 0x7efcbc0cbcf0 2024-04-25T10:05:52.062 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:05:52.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.059+0000 7efcc2a41640 1 -- 172.21.15.60:0/1008109324 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efca8066f60 msgr2=0x7efca8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.059+0000 7efcc2a41640 1 --2- 172.21.15.60:0/1008109324 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efca8066f60 0x7efca8069420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efcbc13d610 tx=0x7efcac009040 comp rx=0 tx=0).stop 2024-04-25T10:05:52.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.059+0000 7efcc2a41640 1 -- 172.21.15.60:0/1008109324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcbc0cbcf0 msgr2=0x7efcbc13fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.059+0000 7efcc2a41640 1 --2- 172.21.15.60:0/1008109324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcbc0cbcf0 0x7efcbc13fb60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efcb000c370 tx=0x7efcb000c830 comp rx=0 tx=0).stop 2024-04-25T10:05:52.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.059+0000 7efcc2a41640 1 -- 172.21.15.60:0/1008109324 shutdown_connections 2024-04-25T10:05:52.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.059+0000 7efcc2a41640 1 --2- 172.21.15.60:0/1008109324 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efca8066f60 0x7efca8069420 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.059+0000 7efcc2a41640 1 --2- 172.21.15.60:0/1008109324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcbc160080 0x7efcbc15b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.059+0000 7efcc2a41640 1 --2- 172.21.15.60:0/1008109324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcbc15fbf0 0x7efcbc142f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.059+0000 7efcc2a41640 1 --2- 172.21.15.60:0/1008109324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcbc0cbcf0 0x7efcbc13fb60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.059+0000 7efcc2a41640 1 -- 172.21.15.60:0/1008109324 >> 172.21.15.60:0/1008109324 conn(0x7efcbc0670b0 msgr2=0x7efcbc15e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.059+0000 7efcc2a41640 1 -- 172.21.15.60:0/1008109324 shutdown_connections 2024-04-25T10:05:52.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.059+0000 7efcc2a41640 1 -- 172.21.15.60:0/1008109324 wait complete. 2024-04-25T10:05:52.089 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:05:52.090 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:52.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f1930059640 1 Processor -- start 2024-04-25T10:05:52.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f1930059640 1 -- start start 2024-04-25T10:05:52.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f1930059640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f19280ce7a0 0x7f19280ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f1930059640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f192815bc90 0x7f19280b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f1930059640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19280cf0c0 con 0x7f19280b4660 2024-04-25T10:05:52.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f1930059640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19280cc370 con 0x7f19280ce7a0 2024-04-25T10:05:52.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f1930059640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1928071ca0 con 0x7f192815bc90 2024-04-25T10:05:52.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f192e5f6640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f19280b4660 0x7f19280b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45478/0 (socket says 172.21.15.60:45478) 2024-04-25T10:05:52.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f192ddf5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f19280ce7a0 0x7f19280ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f192e5f6640 1 -- 172.21.15.60:0/2720699790 learned_addr learned my addr 172.21.15.60:0/2720699790 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:52.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f192d5f4640 1 --2- 172.21.15.60:0/2720699790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f192815bc90 0x7f19280b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f192cdf3640 1 -- 172.21.15.60:0/2720699790 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823762104 0 0) 0x7f19280cf0c0 con 0x7f19280b4660 2024-04-25T10:05:52.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f192cdf3640 1 -- 172.21.15.60:0/2720699790 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1914003530 con 0x7f19280b4660 2024-04-25T10:05:52.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f192ddf5640 1 -- 172.21.15.60:0/2720699790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f192815bc90 msgr2=0x7f19280b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f192ddf5640 1 --2- 172.21.15.60:0/2720699790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f192815bc90 0x7f19280b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f192ddf5640 1 -- 172.21.15.60:0/2720699790 >> v1:172.21.15.60:6789/0 conn(0x7f19280b4660 legacy=0x7f19280b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.207+0000 7f192ddf5640 1 -- 172.21.15.60:0/2720699790 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19280cca40 con 0x7f19280ce7a0 2024-04-25T10:05:52.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f192ddf5640 1 --2- 172.21.15.60:0/2720699790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19280ce7a0 0x7f19280ceb80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f19240097f0 tx=0x7f192401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6196bfe3d9c68e9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f192cdf3640 1 -- 172.21.15.60:0/2720699790 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19240b0070 con 0x7f19280ce7a0 2024-04-25T10:05:52.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f192cdf3640 1 -- 172.21.15.60:0/2720699790 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19240ab600 con 0x7f19280ce7a0 2024-04-25T10:05:52.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f192cdf3640 1 -- 172.21.15.60:0/2720699790 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19240ab9d0 con 0x7f19280ce7a0 2024-04-25T10:05:52.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f1930059640 1 -- 172.21.15.60:0/2720699790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19280ce7a0 msgr2=0x7f19280ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f1930059640 1 --2- 172.21.15.60:0/2720699790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19280ce7a0 0x7f19280ceb80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f19240097f0 tx=0x7f192401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f1930059640 1 -- 172.21.15.60:0/2720699790 shutdown_connections 2024-04-25T10:05:52.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f1930059640 1 --2- 172.21.15.60:0/2720699790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f192815bc90 0x7f19280b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f1930059640 1 --2- 172.21.15.60:0/2720699790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19280ce7a0 0x7f19280ceb80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f1930059640 1 -- 172.21.15.60:0/2720699790 >> 172.21.15.60:0/2720699790 conn(0x7f19280b12d0 msgr2=0x7f19280b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f1930059640 1 -- 172.21.15.60:0/2720699790 shutdown_connections 2024-04-25T10:05:52.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f1930059640 1 -- 172.21.15.60:0/2720699790 wait complete. 2024-04-25T10:05:52.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f1930059640 1 Processor -- start 2024-04-25T10:05:52.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f1930059640 1 -- start start 2024-04-25T10:05:52.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f1930059640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19280ce7a0 0x7f19280adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f1930059640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f192815bc90 0x7f19280cf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f1930059640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19280cfa40 0x7f192813a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f1930059640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19280b4280 con 0x7f19280ce7a0 2024-04-25T10:05:52.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f1930059640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19280b3db0 con 0x7f192815bc90 2024-04-25T10:05:52.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f1930059640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19280b4400 con 0x7f19280cfa40 2024-04-25T10:05:52.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f192ddf5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19280ce7a0 0x7f19280adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f192ddf5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19280ce7a0 0x7f19280adcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54324/0 (socket says 172.21.15.60:54324) 2024-04-25T10:05:52.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.211+0000 7f192ddf5640 1 -- 172.21.15.60:0/2377802765 learned_addr learned my addr 172.21.15.60:0/2377802765 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:52.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f192d5f4640 1 --2- 172.21.15.60:0/2377802765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f192815bc90 0x7f19280cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f192e5f6640 1 --2- 172.21.15.60:0/2377802765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19280cfa40 0x7f192813a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f192ddf5640 1 -- 172.21.15.60:0/2377802765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19280cfa40 msgr2=0x7f192813a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f192ddf5640 1 --2- 172.21.15.60:0/2377802765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19280cfa40 0x7f192813a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f192ddf5640 1 -- 172.21.15.60:0/2377802765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f192815bc90 msgr2=0x7f19280cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f192ddf5640 1 --2- 172.21.15.60:0/2377802765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f192815bc90 0x7f19280cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f192ddf5640 1 -- 172.21.15.60:0/2377802765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19281509a0 con 0x7f19280ce7a0 2024-04-25T10:05:52.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f192d5f4640 1 --2- 172.21.15.60:0/2377802765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f192815bc90 0x7f19280cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:52.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f192ddf5640 1 --2- 172.21.15.60:0/2377802765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19280ce7a0 0x7f19280adcc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f192401a0d0 tx=0x7f1924019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f191e7fc640 1 -- 172.21.15.60:0/2377802765 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19240b0040 con 0x7f19280ce7a0 2024-04-25T10:05:52.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f191e7fc640 1 -- 172.21.15.60:0/2377802765 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19240ba070 con 0x7f19280ce7a0 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f1930059640 1 -- 172.21.15.60:0/2377802765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1928150c40 con 0x7f19280ce7a0 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f191e7fc640 1 -- 172.21.15.60:0/2377802765 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19240b4630 con 0x7f19280ce7a0 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f1930059640 1 -- 172.21.15.60:0/2377802765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1928151200 con 0x7f19280ce7a0 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f1930059640 1 -- 172.21.15.60:0/2377802765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1928074770 con 0x7f19280ce7a0 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f191e7fc640 1 -- 172.21.15.60:0/2377802765 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19240b47d0 con 0x7f19280ce7a0 2024-04-25T10:05:52.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f191e7fc640 1 --2- 172.21.15.60:0/2377802765 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f190c066f60 0x7f190c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f191e7fc640 1 -- 172.21.15.60:0/2377802765 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f192411ec60 con 0x7f19280ce7a0 2024-04-25T10:05:52.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f192d5f4640 1 --2- 172.21.15.60:0/2377802765 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f190c066f60 0x7f190c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.215+0000 7f192d5f4640 1 --2- 172.21.15.60:0/2377802765 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f190c066f60 0x7f190c069420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f192814ebc0 tx=0x7f1918008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.219+0000 7f191e7fc640 1 -- 172.21.15.60:0/2377802765 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1924016090 con 0x7f19280ce7a0 2024-04-25T10:05:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.375+0000 7f1930059640 1 -- 172.21.15.60:0/2377802765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19280aea80 con 0x7f19280ce7a0 2024-04-25T10:05:52.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.375+0000 7f191e7fc640 1 -- 172.21.15.60:0/2377802765 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f19240efc70 con 0x7f19280ce7a0 2024-04-25T10:05:52.384 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:52.384 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:22.036582+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:52.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.379+0000 7f1930059640 1 -- 172.21.15.60:0/2377802765 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f190c066f60 msgr2=0x7f190c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.379+0000 7f1930059640 1 --2- 172.21.15.60:0/2377802765 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f190c066f60 0x7f190c069420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f192814ebc0 tx=0x7f1918008040 comp rx=0 tx=0).stop 2024-04-25T10:05:52.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.379+0000 7f1930059640 1 -- 172.21.15.60:0/2377802765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19280ce7a0 msgr2=0x7f19280adcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.379+0000 7f1930059640 1 --2- 172.21.15.60:0/2377802765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19280ce7a0 0x7f19280adcc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f192401a0d0 tx=0x7f1924019240 comp rx=0 tx=0).stop 2024-04-25T10:05:52.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.379+0000 7f1930059640 1 -- 172.21.15.60:0/2377802765 shutdown_connections 2024-04-25T10:05:52.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.379+0000 7f1930059640 1 --2- 172.21.15.60:0/2377802765 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f190c066f60 0x7f190c069420 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.379+0000 7f1930059640 1 --2- 172.21.15.60:0/2377802765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19280cfa40 0x7f192813a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.379+0000 7f1930059640 1 --2- 172.21.15.60:0/2377802765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f192815bc90 0x7f19280cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.379+0000 7f1930059640 1 --2- 172.21.15.60:0/2377802765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19280ce7a0 0x7f19280adcc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.379+0000 7f1930059640 1 -- 172.21.15.60:0/2377802765 >> 172.21.15.60:0/2377802765 conn(0x7f19280b12d0 msgr2=0x7f19281545e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.379+0000 7f1930059640 1 -- 172.21.15.60:0/2377802765 shutdown_connections 2024-04-25T10:05:52.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.379+0000 7f1930059640 1 -- 172.21.15.60:0/2377802765 wait complete. 2024-04-25T10:05:52.402 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:18.976643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:19.405302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:52.402 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.507+0000 7fdd4abd9640 1 Processor -- start 2024-04-25T10:05:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.507+0000 7fdd4abd9640 1 -- start start 2024-04-25T10:05:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.507+0000 7fdd4abd9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd44160080 0x7fdd4415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.507+0000 7fdd4abd9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd44144a40 con 0x7fdd4415fa20 2024-04-25T10:05:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.507+0000 7fdd4abd9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd440cc9f0 con 0x7fdd440cbcf0 2024-04-25T10:05:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.507+0000 7fdd4abd9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd440b2dc0 con 0x7fdd44160080 2024-04-25T10:05:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.507+0000 7fdd49176640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdd4415fa20 0x7fdd440ceec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45490/0 (socket says 172.21.15.60:45490) 2024-04-25T10:05:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.507+0000 7fdd49176640 1 -- 172.21.15.60:0/3820387807 learned_addr learned my addr 172.21.15.60:0/3820387807 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.507+0000 7fdd3bfff640 1 --2- 172.21.15.60:0/3820387807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd44160080 0x7fdd4415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.507+0000 7fdd3b7fe640 1 -- 172.21.15.60:0/3820387807 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710398763 0 0) 0x7fdd44144a40 con 0x7fdd4415fa20 2024-04-25T10:05:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.507+0000 7fdd3b7fe640 1 -- 172.21.15.60:0/3820387807 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd2c003530 con 0x7fdd4415fa20 2024-04-25T10:05:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3b7fe640 1 -- 172.21.15.60:0/3820387807 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 140208162 0 0) 0x7fdd2c003530 con 0x7fdd4415fa20 2024-04-25T10:05:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3b7fe640 1 -- 172.21.15.60:0/3820387807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd44160080 msgr2=0x7fdd4415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3b7fe640 1 --2- 172.21.15.60:0/3820387807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd44160080 0x7fdd4415f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3b7fe640 1 -- 172.21.15.60:0/3820387807 >> v1:172.21.15.100:6789/0 conn(0x7fdd440cbcf0 legacy=0x7fdd44071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3b7fe640 1 -- 172.21.15.60:0/3820387807 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd440cccd0 con 0x7fdd4415fa20 2024-04-25T10:05:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3b7fe640 1 -- 172.21.15.60:0/3820387807 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fdd3c001a00 con 0x7fdd4415fa20 2024-04-25T10:05:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3b7fe640 1 -- 172.21.15.60:0/3820387807 >> v1:172.21.15.60:6789/0 conn(0x7fdd4415fa20 legacy=0x7fdd440ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3b7fe640 1 --2- 172.21.15.60:0/3820387807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd2c0039f0 0x7fdd2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3b7fe640 1 --2- 172.21.15.60:0/3820387807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd2c0043a0 0x7fdd2c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3b7fe640 1 --2- 172.21.15.60:0/3820387807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd44160080 0x7fdd2c0050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3b7fe640 1 -- 172.21.15.60:0/3820387807 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c00b9d0 con 0x7fdd2c0039f0 2024-04-25T10:05:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3b7fe640 1 -- 172.21.15.60:0/3820387807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c00bb50 con 0x7fdd2c0043a0 2024-04-25T10:05:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3b7fe640 1 -- 172.21.15.60:0/3820387807 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c00bd50 con 0x7fdd44160080 2024-04-25T10:05:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd48975640 1 --2- 172.21.15.60:0/3820387807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd2c0039f0 0x7fdd2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3bfff640 1 --2- 172.21.15.60:0/3820387807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd2c0043a0 0x7fdd2c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd49176640 1 --2- 172.21.15.60:0/3820387807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd44160080 0x7fdd2c0050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd48975640 1 -- 172.21.15.60:0/3820387807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd44160080 msgr2=0x7fdd2c0050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd48975640 1 --2- 172.21.15.60:0/3820387807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd44160080 0x7fdd2c0050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd48975640 1 -- 172.21.15.60:0/3820387807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd2c0043a0 msgr2=0x7fdd2c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd48975640 1 --2- 172.21.15.60:0/3820387807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd2c0043a0 0x7fdd2c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd48975640 1 -- 172.21.15.60:0/3820387807 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c003530 con 0x7fdd2c0039f0 2024-04-25T10:05:52.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3bfff640 1 --2- 172.21.15.60:0/3820387807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd2c0043a0 0x7fdd2c004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:52.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd48975640 1 --2- 172.21.15.60:0/3820387807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd2c0039f0 0x7fdd2c003dd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdd40001860 tx=0x7fdd4001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6a4aa504edbb2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3b7fe640 1 -- 172.21.15.60:0/3820387807 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd400b0070 con 0x7fdd2c0039f0 2024-04-25T10:05:52.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3b7fe640 1 -- 172.21.15.60:0/3820387807 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd400ab600 con 0x7fdd2c0039f0 2024-04-25T10:05:52.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd3b7fe640 1 -- 172.21.15.60:0/3820387807 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd40001460 con 0x7fdd2c0039f0 2024-04-25T10:05:52.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd4abd9640 1 -- 172.21.15.60:0/3820387807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd2c0039f0 msgr2=0x7fdd2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd4abd9640 1 --2- 172.21.15.60:0/3820387807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd2c0039f0 0x7fdd2c003dd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdd40001860 tx=0x7fdd4001c520 comp rx=0 tx=0).stop 2024-04-25T10:05:52.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd49176640 1 -- 172.21.15.60:0/3820387807 reap_dead start 2024-04-25T10:05:52.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd4abd9640 1 -- 172.21.15.60:0/3820387807 shutdown_connections 2024-04-25T10:05:52.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd4abd9640 1 -- 172.21.15.60:0/3820387807 >> 172.21.15.60:0/3820387807 conn(0x7fdd440670b0 msgr2=0x7fdd4406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd4abd9640 1 -- 172.21.15.60:0/3820387807 shutdown_connections 2024-04-25T10:05:52.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.511+0000 7fdd4abd9640 1 -- 172.21.15.60:0/3820387807 wait complete. 2024-04-25T10:05:52.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd4abd9640 1 Processor -- start 2024-04-25T10:05:52.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd4abd9640 1 -- start start 2024-04-25T10:05:52.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd4abd9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd2c0039f0 0x7fdd4413edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd4abd9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd44142cc0 0x7fdd4415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd4abd9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd44141230 0x7fdd441f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd4abd9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd440ce5c0 con 0x7fdd44142cc0 2024-04-25T10:05:52.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd4abd9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4415fe30 con 0x7fdd2c0039f0 2024-04-25T10:05:52.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd4abd9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd440ce740 con 0x7fdd44141230 2024-04-25T10:05:52.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd3bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd44142cc0 0x7fdd4415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd3bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd44142cc0 0x7fdd4415b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54342/0 (socket says 172.21.15.60:54342) 2024-04-25T10:05:52.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd3bfff640 1 -- 172.21.15.60:0/1660796942 learned_addr learned my addr 172.21.15.60:0/1660796942 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:52.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd48975640 1 --2- 172.21.15.60:0/1660796942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd2c0039f0 0x7fdd4413edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd49176640 1 --2- 172.21.15.60:0/1660796942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd44141230 0x7fdd441f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd3bfff640 1 -- 172.21.15.60:0/1660796942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd44141230 msgr2=0x7fdd441f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd3bfff640 1 --2- 172.21.15.60:0/1660796942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd44141230 0x7fdd441f4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd3bfff640 1 -- 172.21.15.60:0/1660796942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd2c0039f0 msgr2=0x7fdd4413edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd3bfff640 1 --2- 172.21.15.60:0/1660796942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd2c0039f0 0x7fdd4413edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd3bfff640 1 -- 172.21.15.60:0/1660796942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd44140190 con 0x7fdd44142cc0 2024-04-25T10:05:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd3bfff640 1 --2- 172.21.15.60:0/1660796942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd44142cc0 0x7fdd4415b310 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009fd0 tx=0x7fdd34011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd397fa640 1 -- 172.21.15.60:0/1660796942 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34004220 con 0x7fdd44142cc0 2024-04-25T10:05:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd397fa640 1 -- 172.21.15.60:0/1660796942 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd340043c0 con 0x7fdd44142cc0 2024-04-25T10:05:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd4abd9640 1 -- 172.21.15.60:0/1660796942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd44140470 con 0x7fdd44142cc0 2024-04-25T10:05:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd397fa640 1 -- 172.21.15.60:0/1660796942 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3400f9e0 con 0x7fdd44142cc0 2024-04-25T10:05:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd4abd9640 1 -- 172.21.15.60:0/1660796942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd44140a30 con 0x7fdd44142cc0 2024-04-25T10:05:52.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd4abd9640 1 -- 172.21.15.60:0/1660796942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4406bba0 con 0x7fdd44142cc0 2024-04-25T10:05:52.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.515+0000 7fdd397fa640 1 -- 172.21.15.60:0/1660796942 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd3400fb80 con 0x7fdd44142cc0 2024-04-25T10:05:52.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.519+0000 7fdd397fa640 1 --2- 172.21.15.60:0/1660796942 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdd30066f60 0x7fdd30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.519+0000 7fdd397fa640 1 -- 172.21.15.60:0/1660796942 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd34085610 con 0x7fdd44142cc0 2024-04-25T10:05:52.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.519+0000 7fdd48975640 1 --2- 172.21.15.60:0/1660796942 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdd30066f60 0x7fdd30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.519+0000 7fdd48975640 1 --2- 172.21.15.60:0/1660796942 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdd30066f60 0x7fdd30069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdd40001860 tx=0x7fdd400a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.523+0000 7fdd397fa640 1 -- 172.21.15.60:0/1660796942 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdd34019030 con 0x7fdd44142cc0 2024-04-25T10:05:52.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.683+0000 7fdd4abd9640 1 -- 172.21.15.60:0/1660796942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd44074770 con 0x7fdd44142cc0 2024-04-25T10:05:52.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.683+0000 7fdd397fa640 1 -- 172.21.15.60:0/1660796942 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd34055f50 con 0x7fdd44142cc0 2024-04-25T10:05:52.689 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:05:52.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.683+0000 7fdd4abd9640 1 -- 172.21.15.60:0/1660796942 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdd30066f60 msgr2=0x7fdd30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.683+0000 7fdd4abd9640 1 --2- 172.21.15.60:0/1660796942 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdd30066f60 0x7fdd30069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdd40001860 tx=0x7fdd400a6000 comp rx=0 tx=0).stop 2024-04-25T10:05:52.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.683+0000 7fdd4abd9640 1 -- 172.21.15.60:0/1660796942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd44142cc0 msgr2=0x7fdd4415b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.683+0000 7fdd4abd9640 1 --2- 172.21.15.60:0/1660796942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd44142cc0 0x7fdd4415b310 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009fd0 tx=0x7fdd34011b30 comp rx=0 tx=0).stop 2024-04-25T10:05:52.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.683+0000 7fdd4abd9640 1 -- 172.21.15.60:0/1660796942 shutdown_connections 2024-04-25T10:05:52.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.683+0000 7fdd4abd9640 1 --2- 172.21.15.60:0/1660796942 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdd30066f60 0x7fdd30069420 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.683+0000 7fdd4abd9640 1 --2- 172.21.15.60:0/1660796942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd44141230 0x7fdd441f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.683+0000 7fdd4abd9640 1 --2- 172.21.15.60:0/1660796942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd44142cc0 0x7fdd4415b310 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.683+0000 7fdd4abd9640 1 --2- 172.21.15.60:0/1660796942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd2c0039f0 0x7fdd4413edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.683+0000 7fdd4abd9640 1 -- 172.21.15.60:0/1660796942 >> 172.21.15.60:0/1660796942 conn(0x7fdd440670b0 msgr2=0x7fdd44167060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.683+0000 7fdd4abd9640 1 -- 172.21.15.60:0/1660796942 shutdown_connections 2024-04-25T10:05:52.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.683+0000 7fdd4abd9640 1 -- 172.21.15.60:0/1660796942 wait complete. 2024-04-25T10:05:52.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:52.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.819+0000 7f7f2df6a640 1 Processor -- start 2024-04-25T10:05:52.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.819+0000 7f7f2df6a640 1 -- start start 2024-04-25T10:05:52.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f2df6a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7f280b3f60 0x7f7f2815a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f2df6a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f2806f030 0x7f7f280b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f2df6a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f28071250 0x7f7f280716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f2df6a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f2815ad30 con 0x7f7f2806f030 2024-04-25T10:05:52.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f2df6a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f2815b010 con 0x7f7f280b3f60 2024-04-25T10:05:52.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f2df6a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2815b2f0 con 0x7f7f28071250 2024-04-25T10:05:52.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f1ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f2806f030 0x7f7f280b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f1ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f2806f030 0x7f7f280b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54348/0 (socket says 172.21.15.60:54348) 2024-04-25T10:05:52.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f1ffff640 1 -- 172.21.15.60:0/3323486946 learned_addr learned my addr 172.21.15.60:0/3323486946 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:52.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f277fe640 1 --2- 172.21.15.60:0/3323486946 >> v2:172.21.15.100:3300/0 conn(0x7f7f280b3f60 0x7f7f2815a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f27fff640 1 --2- 172.21.15.60:0/3323486946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f28071250 0x7f7f280716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f1ffff640 1 -- 172.21.15.60:0/3323486946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f28071250 msgr2=0x7f7f280716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f1ffff640 1 --2- 172.21.15.60:0/3323486946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f28071250 0x7f7f280716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f1ffff640 1 -- 172.21.15.60:0/3323486946 >> v2:172.21.15.100:3300/0 conn(0x7f7f280b3f60 msgr2=0x7f7f2815a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:52.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f1ffff640 1 --2- 172.21.15.60:0/3323486946 >> v2:172.21.15.100:3300/0 conn(0x7f7f280b3f60 0x7f7f2815a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f1ffff640 1 -- 172.21.15.60:0/3323486946 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f280b2600 con 0x7f7f2806f030 2024-04-25T10:05:52.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f277fe640 1 --2- 172.21.15.60:0/3323486946 >> v2:172.21.15.100:3300/0 conn(0x7f7f280b3f60 0x7f7f2815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:52.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f27fff640 1 --2- 172.21.15.60:0/3323486946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f28071250 0x7f7f280716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:52.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f1ffff640 1 --2- 172.21.15.60:0/3323486946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2806f030 0x7f7f280b2e00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7f280b36e0 tx=0x7f7f1001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e0c7636f03e11b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f26ffd640 1 -- 172.21.15.60:0/3323486946 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f100b4070 con 0x7f7f2806f030 2024-04-25T10:05:52.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f26ffd640 1 -- 172.21.15.60:0/3323486946 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f100a7970 con 0x7f7f2806f030 2024-04-25T10:05:52.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f26ffd640 1 -- 172.21.15.60:0/3323486946 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f100a7d40 con 0x7f7f2806f030 2024-04-25T10:05:52.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f2df6a640 1 -- 172.21.15.60:0/3323486946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2806f030 msgr2=0x7f7f280b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.823+0000 7f7f2df6a640 1 --2- 172.21.15.60:0/3323486946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2806f030 0x7f7f280b2e00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7f280b36e0 tx=0x7f7f1001eb60 comp rx=0 tx=0).stop 2024-04-25T10:05:52.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 -- 172.21.15.60:0/3323486946 shutdown_connections 2024-04-25T10:05:52.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 --2- 172.21.15.60:0/3323486946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f28071250 0x7f7f280716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 --2- 172.21.15.60:0/3323486946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2806f030 0x7f7f280b2e00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 --2- 172.21.15.60:0/3323486946 >> v2:172.21.15.100:3300/0 conn(0x7f7f280b3f60 0x7f7f2815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 -- 172.21.15.60:0/3323486946 >> 172.21.15.60:0/3323486946 conn(0x7f7f280670b0 msgr2=0x7f7f2806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:52.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 -- 172.21.15.60:0/3323486946 shutdown_connections 2024-04-25T10:05:52.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 -- 172.21.15.60:0/3323486946 wait complete. 2024-04-25T10:05:52.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 Processor -- start 2024-04-25T10:05:52.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 -- start start 2024-04-25T10:05:52.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2806f030 0x7f7f280ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f28071250 0x7f7f281f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f280b3f60 0x7f7f281f7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2815b5a0 con 0x7f7f2806f030 2024-04-25T10:05:52.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2815b720 con 0x7f7f280b3f60 2024-04-25T10:05:52.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2815b8a0 con 0x7f7f28071250 2024-04-25T10:05:52.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f277fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2806f030 0x7f7f280ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f277fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2806f030 0x7f7f280ab9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54364/0 (socket says 172.21.15.60:54364) 2024-04-25T10:05:52.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f277fe640 1 -- 172.21.15.60:0/2907864103 learned_addr learned my addr 172.21.15.60:0/2907864103 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:52.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f1ffff640 1 --2- 172.21.15.60:0/2907864103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f28071250 0x7f7f281f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f27fff640 1 --2- 172.21.15.60:0/2907864103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f280b3f60 0x7f7f281f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f277fe640 1 -- 172.21.15.60:0/2907864103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f28071250 msgr2=0x7f7f281f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f277fe640 1 --2- 172.21.15.60:0/2907864103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f28071250 0x7f7f281f7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f277fe640 1 -- 172.21.15.60:0/2907864103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f280b3f60 msgr2=0x7f7f281f7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:52.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f277fe640 1 --2- 172.21.15.60:0/2907864103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f280b3f60 0x7f7f281f7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:52.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f277fe640 1 -- 172.21.15.60:0/2907864103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f280ac920 con 0x7f7f2806f030 2024-04-25T10:05:52.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f27fff640 1 --2- 172.21.15.60:0/2907864103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f280b3f60 0x7f7f281f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:52.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f277fe640 1 --2- 172.21.15.60:0/2907864103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2806f030 0x7f7f280ab9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800c7f0 tx=0x7f7f1800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f24ff9640 1 -- 172.21.15.60:0/2907864103 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18015070 con 0x7f7f2806f030 2024-04-25T10:05:52.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f24ff9640 1 -- 172.21.15.60:0/2907864103 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f18007820 con 0x7f7f2806f030 2024-04-25T10:05:52.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 -- 172.21.15.60:0/2907864103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f280ac080 con 0x7f7f2806f030 2024-04-25T10:05:52.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f24ff9640 1 -- 172.21.15.60:0/2907864103 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18007c10 con 0x7f7f2806f030 2024-04-25T10:05:52.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 -- 172.21.15.60:0/2907864103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f280ac640 con 0x7f7f2806f030 2024-04-25T10:05:52.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.827+0000 7f7f2df6a640 1 -- 172.21.15.60:0/2907864103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f28074770 con 0x7f7f2806f030 2024-04-25T10:05:52.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.831+0000 7f7f24ff9640 1 -- 172.21.15.60:0/2907864103 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f18018070 con 0x7f7f2806f030 2024-04-25T10:05:52.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.831+0000 7f7f24ff9640 1 --2- 172.21.15.60:0/2907864103 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7f08066f60 0x7f7f08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:52.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.831+0000 7f7f24ff9640 1 -- 172.21.15.60:0/2907864103 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f180851b0 con 0x7f7f2806f030 2024-04-25T10:05:52.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.831+0000 7f7f1ffff640 1 --2- 172.21.15.60:0/2907864103 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7f08066f60 0x7f7f08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:52.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.831+0000 7f7f1ffff640 1 --2- 172.21.15.60:0/2907864103 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7f08066f60 0x7f7f08069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7f100b1000 tx=0x7f7f10008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:52.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.835+0000 7f7f24ff9640 1 -- 172.21.15.60:0/2907864103 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f180561b0 con 0x7f7f2806f030 2024-04-25T10:05:53.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.991+0000 7f7f2df6a640 1 -- 172.21.15.60:0/2907864103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7f280ab7a0 con 0x7f7f2806f030 2024-04-25T10:05:53.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.995+0000 7f7f24ff9640 1 -- 172.21.15.60:0/2907864103 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7f18055ab0 con 0x7f7f2806f030 2024-04-25T10:05:53.001 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:05:53.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.995+0000 7f7f2df6a640 1 -- 172.21.15.60:0/2907864103 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7f08066f60 msgr2=0x7f7f08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.995+0000 7f7f2df6a640 1 --2- 172.21.15.60:0/2907864103 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7f08066f60 0x7f7f08069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7f100b1000 tx=0x7f7f10008040 comp rx=0 tx=0).stop 2024-04-25T10:05:53.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.995+0000 7f7f2df6a640 1 -- 172.21.15.60:0/2907864103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2806f030 msgr2=0x7f7f280ab9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.995+0000 7f7f2df6a640 1 --2- 172.21.15.60:0/2907864103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2806f030 0x7f7f280ab9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800c7f0 tx=0x7f7f1800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.995+0000 7f7f2df6a640 1 -- 172.21.15.60:0/2907864103 shutdown_connections 2024-04-25T10:05:53.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.995+0000 7f7f2df6a640 1 --2- 172.21.15.60:0/2907864103 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7f08066f60 0x7f7f08069420 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.995+0000 7f7f2df6a640 1 --2- 172.21.15.60:0/2907864103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f280b3f60 0x7f7f281f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.995+0000 7f7f2df6a640 1 --2- 172.21.15.60:0/2907864103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f28071250 0x7f7f281f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.995+0000 7f7f2df6a640 1 --2- 172.21.15.60:0/2907864103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2806f030 0x7f7f280ab9c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.995+0000 7f7f2df6a640 1 -- 172.21.15.60:0/2907864103 >> 172.21.15.60:0/2907864103 conn(0x7f7f280670b0 msgr2=0x7f7f2806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:53.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.995+0000 7f7f2df6a640 1 -- 172.21.15.60:0/2907864103 shutdown_connections 2024-04-25T10:05:53.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:52.995+0000 7f7f2df6a640 1 -- 172.21.15.60:0/2907864103 wait complete. 2024-04-25T10:05:53.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:53.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.143+0000 7f93eadfd640 1 Processor -- start 2024-04-25T10:05:53.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.147+0000 7f93eadfd640 1 -- start start 2024-04-25T10:05:53.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.147+0000 7f93eadfd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93e40cf8a0 0x7f93e415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.147+0000 7f93eadfd640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e40cd890 con 0x7f93e415dd30 2024-04-25T10:05:53.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.147+0000 7f93eadfd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e40cdb70 con 0x7f93e40cd3d0 2024-04-25T10:05:53.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.147+0000 7f93eadfd640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e40ce4a0 con 0x7f93e40cf8a0 2024-04-25T10:05:53.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.147+0000 7f93dbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93e40cf8a0 0x7f93e415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.147+0000 7f93e8b99640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f93e415dd30 0x7f93e40cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45494/0 (socket says 172.21.15.60:45494) 2024-04-25T10:05:53.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.147+0000 7f93e8b99640 1 -- 172.21.15.60:0/2526165741 learned_addr learned my addr 172.21.15.60:0/2526165741 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:53.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.147+0000 7f93dbfff640 1 -- 172.21.15.60:0/2526165741 >> v1:172.21.15.100:6789/0 conn(0x7f93e40cd3d0 legacy=0x7f93e415be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:53.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.147+0000 7f93dbfff640 1 -- 172.21.15.60:0/2526165741 >> v1:172.21.15.60:6789/0 conn(0x7f93e415dd30 legacy=0x7f93e40cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.147+0000 7f93dbfff640 1 -- 172.21.15.60:0/2526165741 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e40cde50 con 0x7f93e40cf8a0 2024-04-25T10:05:53.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.147+0000 7f93db7fe640 1 -- 172.21.15.60:0/2526165741 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2262542329 0 0) 0x7f93e40cd890 con 0x7f93e415dd30 2024-04-25T10:05:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.147+0000 7f93dbfff640 1 --2- 172.21.15.60:0/2526165741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e40cf8a0 0x7f93e415ac30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f93d40097c0 tx=0x7f93d401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0581050e9d09a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.147+0000 7f93db7fe640 1 -- 172.21.15.60:0/2526165741 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d40b1070 con 0x7f93e40cf8a0 2024-04-25T10:05:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.147+0000 7f93db7fe640 1 -- 172.21.15.60:0/2526165741 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d40ac620 con 0x7f93e40cf8a0 2024-04-25T10:05:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93db7fe640 1 -- 172.21.15.60:0/2526165741 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d40ac9f0 con 0x7f93e40cf8a0 2024-04-25T10:05:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 -- 172.21.15.60:0/2526165741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e40cf8a0 msgr2=0x7f93e415ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 --2- 172.21.15.60:0/2526165741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e40cf8a0 0x7f93e415ac30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f93d40097c0 tx=0x7f93d401c3a0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 -- 172.21.15.60:0/2526165741 shutdown_connections 2024-04-25T10:05:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 --2- 172.21.15.60:0/2526165741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e40cf8a0 0x7f93e415ac30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 -- 172.21.15.60:0/2526165741 >> 172.21.15.60:0/2526165741 conn(0x7f93e40670b0 msgr2=0x7f93e406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 -- 172.21.15.60:0/2526165741 shutdown_connections 2024-04-25T10:05:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 -- 172.21.15.60:0/2526165741 wait complete. 2024-04-25T10:05:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 Processor -- start 2024-04-25T10:05:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 -- start start 2024-04-25T10:05:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93e40cd3d0 0x7f93e415a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e40cf8a0 0x7f93e41514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e415dd30 0x7f93e40a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e41682b0 con 0x7f93e415dd30 2024-04-25T10:05:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e415cfd0 con 0x7f93e40cf8a0 2024-04-25T10:05:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e4168430 con 0x7f93e40cd3d0 2024-04-25T10:05:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93e939a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e415dd30 0x7f93e40a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93e939a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e415dd30 0x7f93e40a8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54382/0 (socket says 172.21.15.60:54382) 2024-04-25T10:05:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93e939a640 1 -- 172.21.15.60:0/718765605 learned_addr learned my addr 172.21.15.60:0/718765605 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93e8b99640 1 --2- 172.21.15.60:0/718765605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93e40cd3d0 0x7f93e415a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93dbfff640 1 --2- 172.21.15.60:0/718765605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e40cf8a0 0x7f93e41514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93e939a640 1 -- 172.21.15.60:0/718765605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93e40cd3d0 msgr2=0x7f93e415a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93e939a640 1 --2- 172.21.15.60:0/718765605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93e40cd3d0 0x7f93e415a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93e939a640 1 -- 172.21.15.60:0/718765605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e40cf8a0 msgr2=0x7f93e41514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93e939a640 1 --2- 172.21.15.60:0/718765605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e40cf8a0 0x7f93e41514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93e939a640 1 -- 172.21.15.60:0/718765605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e40a8530 con 0x7f93e415dd30 2024-04-25T10:05:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93e939a640 1 --2- 172.21.15.60:0/718765605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e415dd30 0x7f93e40a8fa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00b410 tx=0x7f93dc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93d97fa640 1 -- 172.21.15.60:0/718765605 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc015020 con 0x7f93e415dd30 2024-04-25T10:05:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93d97fa640 1 -- 172.21.15.60:0/718765605 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93dc011870 con 0x7f93e415dd30 2024-04-25T10:05:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 -- 172.21.15.60:0/718765605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e40aaf70 con 0x7f93e415dd30 2024-04-25T10:05:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93d97fa640 1 -- 172.21.15.60:0/718765605 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc011c60 con 0x7f93e415dd30 2024-04-25T10:05:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.151+0000 7f93eadfd640 1 -- 172.21.15.60:0/718765605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e40a8a80 con 0x7f93e415dd30 2024-04-25T10:05:53.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.155+0000 7f93eadfd640 1 -- 172.21.15.60:0/718765605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e4074770 con 0x7f93e415dd30 2024-04-25T10:05:53.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.155+0000 7f93d97fa640 1 -- 172.21.15.60:0/718765605 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f93dc018020 con 0x7f93e415dd30 2024-04-25T10:05:53.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.155+0000 7f93d97fa640 1 --2- 172.21.15.60:0/718765605 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f93d0066f60 0x7f93d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.155+0000 7f93d97fa640 1 -- 172.21.15.60:0/718765605 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93dc084c10 con 0x7f93e415dd30 2024-04-25T10:05:53.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.155+0000 7f93e8b99640 1 --2- 172.21.15.60:0/718765605 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f93d0066f60 0x7f93d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.155+0000 7f93e8b99640 1 --2- 172.21.15.60:0/718765605 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f93d0066f60 0x7f93d0069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f93e00012f0 tx=0x7f93e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.159+0000 7f93d97fa640 1 -- 172.21.15.60:0/718765605 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f93dc01f030 con 0x7f93e415dd30 2024-04-25T10:05:53.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.311+0000 7f93eadfd640 1 -- 172.21.15.60:0/718765605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f93e4156a70 con 0x7f93e415dd30 2024-04-25T10:05:53.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.311+0000 7f93d97fa640 1 -- 172.21.15.60:0/718765605 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f93dc055b80 con 0x7f93e415dd30 2024-04-25T10:05:53.347 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:05:53.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.311+0000 7f93eadfd640 1 -- 172.21.15.60:0/718765605 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f93d0066f60 msgr2=0x7f93d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.311+0000 7f93eadfd640 1 --2- 172.21.15.60:0/718765605 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f93d0066f60 0x7f93d0069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f93e00012f0 tx=0x7f93e0009040 comp rx=0 tx=0).stop 2024-04-25T10:05:53.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.311+0000 7f93eadfd640 1 -- 172.21.15.60:0/718765605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e415dd30 msgr2=0x7f93e40a8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.311+0000 7f93eadfd640 1 --2- 172.21.15.60:0/718765605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e415dd30 0x7f93e40a8fa0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00b410 tx=0x7f93dc00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.311+0000 7f93eadfd640 1 -- 172.21.15.60:0/718765605 shutdown_connections 2024-04-25T10:05:53.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.311+0000 7f93eadfd640 1 --2- 172.21.15.60:0/718765605 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f93d0066f60 0x7f93d0069420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.311+0000 7f93eadfd640 1 --2- 172.21.15.60:0/718765605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e415dd30 0x7f93e40a8fa0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.311+0000 7f93eadfd640 1 --2- 172.21.15.60:0/718765605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e40cf8a0 0x7f93e41514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.311+0000 7f93eadfd640 1 --2- 172.21.15.60:0/718765605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93e40cd3d0 0x7f93e415a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.311+0000 7f93eadfd640 1 -- 172.21.15.60:0/718765605 >> 172.21.15.60:0/718765605 conn(0x7f93e40670b0 msgr2=0x7f93e40af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:53.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.311+0000 7f93eadfd640 1 -- 172.21.15.60:0/718765605 shutdown_connections 2024-04-25T10:05:53.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.311+0000 7f93eadfd640 1 -- 172.21.15.60:0/718765605 wait complete. 2024-04-25T10:05:53.350 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-25T10:05:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.467+0000 7f252ff89640 1 Processor -- start 2024-04-25T10:05:53.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.467+0000 7f252ff89640 1 -- start start 2024-04-25T10:05:53.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.467+0000 7f252ff89640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f25280ce7a0 0x7f25280ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.467+0000 7f252ff89640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25280b33f0 con 0x7f25280b4660 2024-04-25T10:05:53.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.467+0000 7f252ff89640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25280cf0c0 con 0x7f252815bc90 2024-04-25T10:05:53.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.467+0000 7f252ff89640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25280cc370 con 0x7f25280ce7a0 2024-04-25T10:05:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.467+0000 7f252e526640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f25280b4660 0x7f25280b2ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45502/0 (socket says 172.21.15.60:45502) 2024-04-25T10:05:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.467+0000 7f252e526640 1 -- 172.21.15.60:0/824348894 learned_addr learned my addr 172.21.15.60:0/824348894 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.471+0000 7f252dd25640 1 --2- 172.21.15.60:0/824348894 >> v2:172.21.15.100:3300/0 conn(0x7f25280ce7a0 0x7f25280ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.471+0000 7f252cd23640 1 -- 172.21.15.60:0/824348894 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772445848 0 0) 0x7f25280b33f0 con 0x7f25280b4660 2024-04-25T10:05:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.471+0000 7f252cd23640 1 -- 172.21.15.60:0/824348894 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2514003530 con 0x7f25280b4660 2024-04-25T10:05:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.471+0000 7f252dd25640 1 -- 172.21.15.60:0/824348894 >> v1:172.21.15.100:6789/0 conn(0x7f252815bc90 legacy=0x7f25280b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.471+0000 7f252dd25640 1 -- 172.21.15.60:0/824348894 >> v1:172.21.15.60:6789/0 conn(0x7f25280b4660 legacy=0x7f25280b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.471+0000 7f252dd25640 1 -- 172.21.15.60:0/824348894 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2528071ca0 con 0x7f25280ce7a0 2024-04-25T10:05:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.471+0000 7f252dd25640 1 --2- 172.21.15.60:0/824348894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25280ce7a0 0x7f25280ceb80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f25240097c0 tx=0x7f252401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad363bbdd2423dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.471+0000 7f252cd23640 1 -- 172.21.15.60:0/824348894 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25240b0070 con 0x7f25280ce7a0 2024-04-25T10:05:53.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.471+0000 7f252cd23640 1 -- 172.21.15.60:0/824348894 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25240ab600 con 0x7f25280ce7a0 2024-04-25T10:05:53.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.471+0000 7f252cd23640 1 -- 172.21.15.60:0/824348894 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25240ab9b0 con 0x7f25280ce7a0 2024-04-25T10:05:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252ff89640 1 -- 172.21.15.60:0/824348894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25280ce7a0 msgr2=0x7f25280ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252ff89640 1 --2- 172.21.15.60:0/824348894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25280ce7a0 0x7f25280ceb80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f25240097c0 tx=0x7f252401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252ff89640 1 -- 172.21.15.60:0/824348894 shutdown_connections 2024-04-25T10:05:53.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252ff89640 1 --2- 172.21.15.60:0/824348894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25280ce7a0 0x7f25280ceb80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252ff89640 1 -- 172.21.15.60:0/824348894 >> 172.21.15.60:0/824348894 conn(0x7f25280b12d0 msgr2=0x7f25280b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:53.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252ff89640 1 -- 172.21.15.60:0/824348894 shutdown_connections 2024-04-25T10:05:53.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252ff89640 1 -- 172.21.15.60:0/824348894 wait complete. 2024-04-25T10:05:53.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252ff89640 1 Processor -- start 2024-04-25T10:05:53.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252ff89640 1 -- start start 2024-04-25T10:05:53.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252ff89640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25280ce7a0 0x7f252814cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252ff89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f252815bc90 0x7f2528149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252ff89640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25281522a0 0x7f2528159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252ff89640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25280b3d20 con 0x7f252815bc90 2024-04-25T10:05:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252ff89640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25280b3ba0 con 0x7f25281522a0 2024-04-25T10:05:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252ff89640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25280b4280 con 0x7f25280ce7a0 2024-04-25T10:05:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252d524640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f252815bc90 0x7f2528149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252d524640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f252815bc90 0x7f2528149f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54384/0 (socket says 172.21.15.60:54384) 2024-04-25T10:05:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252d524640 1 -- 172.21.15.60:0/730854731 learned_addr learned my addr 172.21.15.60:0/730854731 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252dd25640 1 --2- 172.21.15.60:0/730854731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25280ce7a0 0x7f252814cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252e526640 1 --2- 172.21.15.60:0/730854731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25281522a0 0x7f2528159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252d524640 1 -- 172.21.15.60:0/730854731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25280ce7a0 msgr2=0x7f252814cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252d524640 1 --2- 172.21.15.60:0/730854731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25280ce7a0 0x7f252814cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252d524640 1 -- 172.21.15.60:0/730854731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25281522a0 msgr2=0x7f2528159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252d524640 1 --2- 172.21.15.60:0/730854731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25281522a0 0x7f2528159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252d524640 1 -- 172.21.15.60:0/730854731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2528151b10 con 0x7f252815bc90 2024-04-25T10:05:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252e526640 1 --2- 172.21.15.60:0/730854731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25281522a0 0x7f2528159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:53.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.475+0000 7f252d524640 1 --2- 172.21.15.60:0/730854731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f252815bc90 0x7f2528149f30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f251c00c370 tx=0x7f251c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.479+0000 7f251a7fc640 1 -- 172.21.15.60:0/730854731 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c0052d0 con 0x7f252815bc90 2024-04-25T10:05:53.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.479+0000 7f252ff89640 1 -- 172.21.15.60:0/730854731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252814ed40 con 0x7f252815bc90 2024-04-25T10:05:53.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.479+0000 7f251a7fc640 1 -- 172.21.15.60:0/730854731 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251c005470 con 0x7f252815bc90 2024-04-25T10:05:53.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.479+0000 7f251a7fc640 1 -- 172.21.15.60:0/730854731 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c007260 con 0x7f252815bc90 2024-04-25T10:05:53.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.479+0000 7f252ff89640 1 -- 172.21.15.60:0/730854731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2528151d80 con 0x7f252815bc90 2024-04-25T10:05:53.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.479+0000 7f252ff89640 1 -- 172.21.15.60:0/730854731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2528151f80 con 0x7f252815bc90 2024-04-25T10:05:53.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.479+0000 7f251a7fc640 1 -- 172.21.15.60:0/730854731 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f251c007400 con 0x7f252815bc90 2024-04-25T10:05:53.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.479+0000 7f251a7fc640 1 --2- 172.21.15.60:0/730854731 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f250c066f60 0x7f250c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:53.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.479+0000 7f251a7fc640 1 -- 172.21.15.60:0/730854731 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f251c084550 con 0x7f252815bc90 2024-04-25T10:05:53.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.479+0000 7f252dd25640 1 --2- 172.21.15.60:0/730854731 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f250c066f60 0x7f250c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:53.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.479+0000 7f252dd25640 1 --2- 172.21.15.60:0/730854731 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f250c066f60 0x7f250c069420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f25240ad000 tx=0x7f25240a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:53.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.483+0000 7f251a7fc640 1 -- 172.21.15.60:0/730854731 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2528151f80 con 0x7f252815bc90 2024-04-25T10:05:53.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:53.707+0000 7f252ff89640 1 -- 172.21.15.60:0/730854731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f2528074770 con 0x7f252815bc90 2024-04-25T10:05:54.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.539+0000 7f251a7fc640 1 -- 172.21.15.60:0/730854731 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v55) v1 ==== 145+0+0 (secure 0 0 0) 0x7f251c054df0 con 0x7f252815bc90 2024-04-25T10:05:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.539+0000 7f252ff89640 1 -- 172.21.15.60:0/730854731 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f250c066f60 msgr2=0x7f250c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.539+0000 7f252ff89640 1 --2- 172.21.15.60:0/730854731 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f250c066f60 0x7f250c069420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f25240ad000 tx=0x7f25240a6000 comp rx=0 tx=0).stop 2024-04-25T10:05:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.539+0000 7f252ff89640 1 -- 172.21.15.60:0/730854731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f252815bc90 msgr2=0x7f2528149f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.539+0000 7f252ff89640 1 --2- 172.21.15.60:0/730854731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f252815bc90 0x7f2528149f30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f251c00c370 tx=0x7f251c00c830 comp rx=0 tx=0).stop 2024-04-25T10:05:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.539+0000 7f252ff89640 1 -- 172.21.15.60:0/730854731 shutdown_connections 2024-04-25T10:05:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.539+0000 7f252ff89640 1 --2- 172.21.15.60:0/730854731 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f250c066f60 0x7f250c069420 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.539+0000 7f252ff89640 1 --2- 172.21.15.60:0/730854731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25281522a0 0x7f2528159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.539+0000 7f252ff89640 1 --2- 172.21.15.60:0/730854731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f252815bc90 0x7f2528149f30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.539+0000 7f252ff89640 1 --2- 172.21.15.60:0/730854731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25280ce7a0 0x7f252814cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.539+0000 7f252ff89640 1 -- 172.21.15.60:0/730854731 >> 172.21.15.60:0/730854731 conn(0x7f25280b12d0 msgr2=0x7f252815ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.539+0000 7f252ff89640 1 -- 172.21.15.60:0/730854731 shutdown_connections 2024-04-25T10:05:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.539+0000 7f252ff89640 1 -- 172.21.15.60:0/730854731 wait complete. 2024-04-25T10:05:54.567 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:05:54.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.695+0000 7fd112627640 1 Processor -- start 2024-04-25T10:05:54.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.695+0000 7fd112627640 1 -- start start 2024-04-25T10:05:54.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.695+0000 7fd112627640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd10c0cbcf0 0x7fd10c160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:54.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.695+0000 7fd112627640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd10c071c30 0x7fd10c15f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:54.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.695+0000 7fd112627640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10c144a40 con 0x7fd10c15fa20 2024-04-25T10:05:54.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.695+0000 7fd112627640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd10c0b2dc0 con 0x7fd10c071c30 2024-04-25T10:05:54.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.695+0000 7fd112627640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd10c0cc9f0 con 0x7fd10c0cbcf0 2024-04-25T10:05:54.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.695+0000 7fd103fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd10c071c30 0x7fd10c15f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:54.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.695+0000 7fd103fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd10c071c30 0x7fd10c15f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54396/0 (socket says 172.21.15.60:54396) 2024-04-25T10:05:54.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.695+0000 7fd103fff640 1 -- 172.21.15.60:0/3156441635 learned_addr learned my addr 172.21.15.60:0/3156441635 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:54.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.695+0000 7fd10bfff640 1 --2- 172.21.15.60:0/3156441635 >> v2:172.21.15.100:3300/0 conn(0x7fd10c0cbcf0 0x7fd10c160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:54.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.695+0000 7fd103fff640 1 -- 172.21.15.60:0/3156441635 >> v2:172.21.15.100:3300/0 conn(0x7fd10c0cbcf0 msgr2=0x7fd10c160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:54.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.695+0000 7fd103fff640 1 --2- 172.21.15.60:0/3156441635 >> v2:172.21.15.100:3300/0 conn(0x7fd10c0cbcf0 0x7fd10c160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.695+0000 7fd103fff640 1 -- 172.21.15.60:0/3156441635 >> v1:172.21.15.100:6789/0 conn(0x7fd10c15fa20 legacy=0x7fd10c15bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.695+0000 7fd103fff640 1 -- 172.21.15.60:0/3156441635 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c0cccd0 con 0x7fd10c071c30 2024-04-25T10:05:54.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd103fff640 1 --2- 172.21.15.60:0/3156441635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd10c071c30 0x7fd10c15f4e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd10c0b4c50 tx=0x7fd0f401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86f34a398179bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:54.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd10b7fe640 1 -- 172.21.15.60:0/3156441635 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f40b1070 con 0x7fd10c071c30 2024-04-25T10:05:54.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd10b7fe640 1 -- 172.21.15.60:0/3156441635 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f40ac630 con 0x7fd10c071c30 2024-04-25T10:05:54.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd10b7fe640 1 -- 172.21.15.60:0/3156441635 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f40a8050 con 0x7fd10c071c30 2024-04-25T10:05:54.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd112627640 1 -- 172.21.15.60:0/3156441635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd10c071c30 msgr2=0x7fd10c15f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:54.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd112627640 1 --2- 172.21.15.60:0/3156441635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd10c071c30 0x7fd10c15f4e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd10c0b4c50 tx=0x7fd0f401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd112627640 1 -- 172.21.15.60:0/3156441635 shutdown_connections 2024-04-25T10:05:54.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd112627640 1 --2- 172.21.15.60:0/3156441635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd10c071c30 0x7fd10c15f4e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd112627640 1 --2- 172.21.15.60:0/3156441635 >> v2:172.21.15.100:3300/0 conn(0x7fd10c0cbcf0 0x7fd10c160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd112627640 1 -- 172.21.15.60:0/3156441635 >> 172.21.15.60:0/3156441635 conn(0x7fd10c0670b0 msgr2=0x7fd10c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd112627640 1 -- 172.21.15.60:0/3156441635 shutdown_connections 2024-04-25T10:05:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd112627640 1 -- 172.21.15.60:0/3156441635 wait complete. 2024-04-25T10:05:54.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd112627640 1 Processor -- start 2024-04-25T10:05:54.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd112627640 1 -- start start 2024-04-25T10:05:54.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd112627640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd10c071c30 0x7fd10c13fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:54.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd112627640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd10c0cbcf0 0x7fd10c1618e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:54.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd112627640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd10c15fa20 0x7fd10c142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd112627640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c0cfed0 con 0x7fd10c0cbcf0 2024-04-25T10:05:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd112627640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c0720b0 con 0x7fd10c071c30 2024-04-25T10:05:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd112627640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c160840 con 0x7fd10c15fa20 2024-04-25T10:05:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd103fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd10c0cbcf0 0x7fd10c1618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd103fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd10c0cbcf0 0x7fd10c1618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54402/0 (socket says 172.21.15.60:54402) 2024-04-25T10:05:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd103fff640 1 -- 172.21.15.60:0/2823415322 learned_addr learned my addr 172.21.15.60:0/2823415322 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd10bfff640 1 --2- 172.21.15.60:0/2823415322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd10c071c30 0x7fd10c13fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd110bc4640 1 --2- 172.21.15.60:0/2823415322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd10c15fa20 0x7fd10c142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd103fff640 1 -- 172.21.15.60:0/2823415322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd10c15fa20 msgr2=0x7fd10c142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd103fff640 1 --2- 172.21.15.60:0/2823415322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd10c15fa20 0x7fd10c142770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd103fff640 1 -- 172.21.15.60:0/2823415322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd10c071c30 msgr2=0x7fd10c13fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd103fff640 1 --2- 172.21.15.60:0/2823415322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd10c071c30 0x7fd10c13fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd103fff640 1 -- 172.21.15.60:0/2823415322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c06b150 con 0x7fd10c0cbcf0 2024-04-25T10:05:54.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd110bc4640 1 --2- 172.21.15.60:0/2823415322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd10c15fa20 0x7fd10c142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:54.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd103fff640 1 --2- 172.21.15.60:0/2823415322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd10c0cbcf0 0x7fd10c1618e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd0f401a110 tx=0x7fd0f4019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:54.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.699+0000 7fd1097fa640 1 -- 172.21.15.60:0/2823415322 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f40bc070 con 0x7fd10c0cbcf0 2024-04-25T10:05:54.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.703+0000 7fd1097fa640 1 -- 172.21.15.60:0/2823415322 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f40ab5e0 con 0x7fd10c0cbcf0 2024-04-25T10:05:54.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.703+0000 7fd112627640 1 -- 172.21.15.60:0/2823415322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10c160b20 con 0x7fd10c0cbcf0 2024-04-25T10:05:54.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.703+0000 7fd1097fa640 1 -- 172.21.15.60:0/2823415322 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f40b1070 con 0x7fd10c0cbcf0 2024-04-25T10:05:54.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.703+0000 7fd112627640 1 -- 172.21.15.60:0/2823415322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10c1610e0 con 0x7fd10c0cbcf0 2024-04-25T10:05:54.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.703+0000 7fd112627640 1 -- 172.21.15.60:0/2823415322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd10c06bba0 con 0x7fd10c0cbcf0 2024-04-25T10:05:54.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.703+0000 7fd1097fa640 1 -- 172.21.15.60:0/2823415322 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0f40bf070 con 0x7fd10c0cbcf0 2024-04-25T10:05:54.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.703+0000 7fd1097fa640 1 --2- 172.21.15.60:0/2823415322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd0ec066f60 0x7fd0ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:54.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.703+0000 7fd1097fa640 1 -- 172.21.15.60:0/2823415322 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0f411ea50 con 0x7fd10c0cbcf0 2024-04-25T10:05:54.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.703+0000 7fd10bfff640 1 --2- 172.21.15.60:0/2823415322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd0ec066f60 0x7fd0ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:54.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.703+0000 7fd10bfff640 1 --2- 172.21.15.60:0/2823415322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd0ec066f60 0x7fd0ec069420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00b9d0 tx=0x7fd0fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:54.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.707+0000 7fd1097fa640 1 -- 172.21.15.60:0/2823415322 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd0f40160d0 con 0x7fd10c0cbcf0 2024-04-25T10:05:54.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.867+0000 7fd112627640 1 -- 172.21.15.60:0/2823415322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd10c0b2970 con 0x7fd10c0cbcf0 2024-04-25T10:05:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.871+0000 7fd1097fa640 1 -- 172.21.15.60:0/2823415322 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd0f40efa50 con 0x7fd10c0cbcf0 2024-04-25T10:05:54.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.875+0000 7fd112627640 1 -- 172.21.15.60:0/2823415322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd0ec066f60 msgr2=0x7fd0ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:54.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.875+0000 7fd112627640 1 --2- 172.21.15.60:0/2823415322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd0ec066f60 0x7fd0ec069420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00b9d0 tx=0x7fd0fc008040 comp rx=0 tx=0).stop 2024-04-25T10:05:54.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.875+0000 7fd112627640 1 -- 172.21.15.60:0/2823415322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd10c0cbcf0 msgr2=0x7fd10c1618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:54.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.875+0000 7fd112627640 1 --2- 172.21.15.60:0/2823415322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd10c0cbcf0 0x7fd10c1618e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd0f401a110 tx=0x7fd0f4019280 comp rx=0 tx=0).stop 2024-04-25T10:05:54.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.875+0000 7fd112627640 1 -- 172.21.15.60:0/2823415322 shutdown_connections 2024-04-25T10:05:54.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.875+0000 7fd112627640 1 --2- 172.21.15.60:0/2823415322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd0ec066f60 0x7fd0ec069420 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.875+0000 7fd112627640 1 --2- 172.21.15.60:0/2823415322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd10c15fa20 0x7fd10c142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.875+0000 7fd112627640 1 --2- 172.21.15.60:0/2823415322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd10c0cbcf0 0x7fd10c1618e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.875+0000 7fd112627640 1 --2- 172.21.15.60:0/2823415322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd10c071c30 0x7fd10c13fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:54.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.875+0000 7fd112627640 1 -- 172.21.15.60:0/2823415322 >> 172.21.15.60:0/2823415322 conn(0x7fd10c0670b0 msgr2=0x7fd10c15e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:54.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.875+0000 7fd112627640 1 -- 172.21.15.60:0/2823415322 shutdown_connections 2024-04-25T10:05:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:54.875+0000 7fd112627640 1 -- 172.21.15.60:0/2823415322 wait complete. 2024-04-25T10:05:54.898 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:05:54.899 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:54.958 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:55.013 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:05:55.014 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:05:55.014 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:55.020 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:05:55.020 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:55.069 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:55.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c758e640 1 Processor -- start 2024-04-25T10:05:55.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c758e640 1 -- start start 2024-04-25T10:05:55.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c758e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f88c00d0a50 0x7f88c00b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c758e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c00ce2b0 0x7f88c00ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c758e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c006d200 con 0x7f88c015deb0 2024-04-25T10:05:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c758e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c00cf2c0 con 0x7f88c00d0a50 2024-04-25T10:05:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c758e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c00cefe0 con 0x7f88c00ce2b0 2024-04-25T10:05:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c532a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f88c015deb0 0x7f88c00d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45518/0 (socket says 172.21.15.60:45518) 2024-04-25T10:05:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c532a640 1 -- 172.21.15.60:0/2765710358 learned_addr learned my addr 172.21.15.60:0/2765710358 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c5b2b640 1 --2- 172.21.15.60:0/2765710358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c00ce2b0 0x7f88c00ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c4b29640 1 --2- 172.21.15.60:0/2765710358 >> v2:172.21.15.100:3300/0 conn(0x7f88c00d0a50 0x7f88c00b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88b3fff640 1 -- 172.21.15.60:0/2765710358 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040080315 0 0) 0x7f88c006d200 con 0x7f88c015deb0 2024-04-25T10:05:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88b3fff640 1 -- 172.21.15.60:0/2765710358 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88ac003530 con 0x7f88c015deb0 2024-04-25T10:05:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88b3fff640 1 -- 172.21.15.60:0/2765710358 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2830846790 0 0) 0x7f88ac003530 con 0x7f88c015deb0 2024-04-25T10:05:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88b3fff640 1 -- 172.21.15.60:0/2765710358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c00ce2b0 msgr2=0x7f88c00ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88b3fff640 1 --2- 172.21.15.60:0/2765710358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c00ce2b0 0x7f88c00ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88b3fff640 1 -- 172.21.15.60:0/2765710358 >> v2:172.21.15.100:3300/0 conn(0x7f88c00d0a50 msgr2=0x7f88c00b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:55.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88b3fff640 1 --2- 172.21.15.60:0/2765710358 >> v2:172.21.15.100:3300/0 conn(0x7f88c00d0a50 0x7f88c00b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88b3fff640 1 -- 172.21.15.60:0/2765710358 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c00d1d70 con 0x7f88c015deb0 2024-04-25T10:05:55.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88b3fff640 1 -- 172.21.15.60:0/2765710358 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f88bc001a00 con 0x7f88c015deb0 2024-04-25T10:05:55.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88b3fff640 1 -- 172.21.15.60:0/2765710358 >> v1:172.21.15.60:6789/0 conn(0x7f88c015deb0 legacy=0x7f88c00d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88b3fff640 1 --2- 172.21.15.60:0/2765710358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac0039c0 0x7f88ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88b3fff640 1 --2- 172.21.15.60:0/2765710358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c00ce2b0 0x7f88ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88b3fff640 1 --2- 172.21.15.60:0/2765710358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88ac004d50 0x7f88ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88b3fff640 1 -- 172.21.15.60:0/2765710358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c00cf2c0 con 0x7f88ac004d50 2024-04-25T10:05:55.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88b3fff640 1 -- 172.21.15.60:0/2765710358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c00cefe0 con 0x7f88ac0039c0 2024-04-25T10:05:55.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88b3fff640 1 -- 172.21.15.60:0/2765710358 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac00b9d0 con 0x7f88c00ce2b0 2024-04-25T10:05:55.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c5b2b640 1 --2- 172.21.15.60:0/2765710358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88ac004d50 0x7f88ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c532a640 1 --2- 172.21.15.60:0/2765710358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac0039c0 0x7f88ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c5b2b640 1 -- 172.21.15.60:0/2765710358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c00ce2b0 msgr2=0x7f88ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:55.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c5b2b640 1 --2- 172.21.15.60:0/2765710358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c00ce2b0 0x7f88ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c5b2b640 1 -- 172.21.15.60:0/2765710358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac0039c0 msgr2=0x7f88ac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:55.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c5b2b640 1 --2- 172.21.15.60:0/2765710358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac0039c0 0x7f88ac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c5b2b640 1 -- 172.21.15.60:0/2765710358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac003530 con 0x7f88ac004d50 2024-04-25T10:05:55.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.227+0000 7f88c532a640 1 --2- 172.21.15.60:0/2765710358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac0039c0 0x7f88ac003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:55.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.231+0000 7f88c5b2b640 1 --2- 172.21.15.60:0/2765710358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88ac004d50 0x7f88ac0051d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009ae0 tx=0x7f88b801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda5854c05124248 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:55.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.231+0000 7f88b3fff640 1 -- 172.21.15.60:0/2765710358 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80b2070 con 0x7f88ac004d50 2024-04-25T10:05:55.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.231+0000 7f88b3fff640 1 -- 172.21.15.60:0/2765710358 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b80a56a0 con 0x7f88ac004d50 2024-04-25T10:05:55.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.231+0000 7f88b3fff640 1 -- 172.21.15.60:0/2765710358 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80096b0 con 0x7f88ac004d50 2024-04-25T10:05:55.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.231+0000 7f88c758e640 1 -- 172.21.15.60:0/2765710358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88ac004d50 msgr2=0x7f88ac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.231+0000 7f88c758e640 1 --2- 172.21.15.60:0/2765710358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88ac004d50 0x7f88ac0051d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009ae0 tx=0x7f88b801cab0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.231+0000 7f88c5b2b640 1 -- 172.21.15.60:0/2765710358 reap_dead start 2024-04-25T10:05:55.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.231+0000 7f88c758e640 1 -- 172.21.15.60:0/2765710358 shutdown_connections 2024-04-25T10:05:55.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.231+0000 7f88c758e640 1 -- 172.21.15.60:0/2765710358 >> 172.21.15.60:0/2765710358 conn(0x7f88c006c090 msgr2=0x7f88c006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:55.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.231+0000 7f88c758e640 1 -- 172.21.15.60:0/2765710358 shutdown_connections 2024-04-25T10:05:55.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.231+0000 7f88c758e640 1 -- 172.21.15.60:0/2765710358 wait complete. 2024-04-25T10:05:55.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.231+0000 7f88c758e640 1 Processor -- start 2024-04-25T10:05:55.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.231+0000 7f88c758e640 1 -- start start 2024-04-25T10:05:55.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c758e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac004d50 0x7f88c014daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c758e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c00cd770 0x7f88c01573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c758e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88c00cb0f0 0x7f88c0158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c758e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c00ce760 con 0x7f88c00cb0f0 2024-04-25T10:05:55.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c758e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c00d2330 con 0x7f88ac004d50 2024-04-25T10:05:55.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c758e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c00ce8e0 con 0x7f88c00cd770 2024-04-25T10:05:55.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c5b2b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88c00cb0f0 0x7f88c0158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c5b2b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88c00cb0f0 0x7f88c0158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54414/0 (socket says 172.21.15.60:54414) 2024-04-25T10:05:55.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c5b2b640 1 -- 172.21.15.60:0/3957967306 learned_addr learned my addr 172.21.15.60:0/3957967306 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c532a640 1 --2- 172.21.15.60:0/3957967306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac004d50 0x7f88c014daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c4b29640 1 --2- 172.21.15.60:0/3957967306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c00cd770 0x7f88c01573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c5b2b640 1 -- 172.21.15.60:0/3957967306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c00cd770 msgr2=0x7f88c01573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c5b2b640 1 --2- 172.21.15.60:0/3957967306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c00cd770 0x7f88c01573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c5b2b640 1 -- 172.21.15.60:0/3957967306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac004d50 msgr2=0x7f88c014daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c5b2b640 1 --2- 172.21.15.60:0/3957967306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac004d50 0x7f88c014daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c5b2b640 1 -- 172.21.15.60:0/3957967306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c0156330 con 0x7f88c00cb0f0 2024-04-25T10:05:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c5b2b640 1 --2- 172.21.15.60:0/3957967306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88c00cb0f0 0x7f88c0158510 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f88b80b0040 tx=0x7f88ac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88b1ffb640 1 -- 172.21.15.60:0/3957967306 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80b2040 con 0x7f88c00cb0f0 2024-04-25T10:05:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88b1ffb640 1 -- 172.21.15.60:0/3957967306 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b8000e50 con 0x7f88c00cb0f0 2024-04-25T10:05:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c758e640 1 -- 172.21.15.60:0/3957967306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c0156610 con 0x7f88c00cb0f0 2024-04-25T10:05:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88b1ffb640 1 -- 172.21.15.60:0/3957967306 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80096b0 con 0x7f88c00cb0f0 2024-04-25T10:05:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c758e640 1 -- 172.21.15.60:0/3957967306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c0156bd0 con 0x7f88c00cb0f0 2024-04-25T10:05:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c758e640 1 -- 172.21.15.60:0/3957967306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c0074770 con 0x7f88c00cb0f0 2024-04-25T10:05:55.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88b1ffb640 1 -- 172.21.15.60:0/3957967306 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88b8004240 con 0x7f88c00cb0f0 2024-04-25T10:05:55.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88b1ffb640 1 --2- 172.21.15.60:0/3957967306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f88a4066f90 0x7f88a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88b1ffb640 1 -- 172.21.15.60:0/3957967306 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88b811e600 con 0x7f88c00cb0f0 2024-04-25T10:05:55.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.235+0000 7f88c532a640 1 --2- 172.21.15.60:0/3957967306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f88a4066f90 0x7f88a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.239+0000 7f88c532a640 1 --2- 172.21.15.60:0/3957967306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f88a4066f90 0x7f88a4069450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f88bc005fd0 tx=0x7f88bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:55.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.239+0000 7f88b1ffb640 1 -- 172.21.15.60:0/3957967306 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f88b80167c0 con 0x7f88c00cb0f0 2024-04-25T10:05:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.403+0000 7f88c758e640 1 -- 172.21.15.60:0/3957967306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88c00d17a0 con 0x7f88c00cb0f0 2024-04-25T10:05:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.403+0000 7f88b1ffb640 1 -- 172.21.15.60:0/3957967306 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f88b80eefc0 con 0x7f88c00cb0f0 2024-04-25T10:05:55.410 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:55.410 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:22.036582+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:55.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.407+0000 7f88c758e640 1 -- 172.21.15.60:0/3957967306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f88a4066f90 msgr2=0x7f88a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.407+0000 7f88c758e640 1 --2- 172.21.15.60:0/3957967306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f88a4066f90 0x7f88a4069450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f88bc005fd0 tx=0x7f88bc009040 comp rx=0 tx=0).stop 2024-04-25T10:05:55.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.407+0000 7f88c758e640 1 -- 172.21.15.60:0/3957967306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88c00cb0f0 msgr2=0x7f88c0158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.407+0000 7f88c758e640 1 --2- 172.21.15.60:0/3957967306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88c00cb0f0 0x7f88c0158510 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f88b80b0040 tx=0x7f88ac005840 comp rx=0 tx=0).stop 2024-04-25T10:05:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.407+0000 7f88c758e640 1 -- 172.21.15.60:0/3957967306 shutdown_connections 2024-04-25T10:05:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.407+0000 7f88c758e640 1 --2- 172.21.15.60:0/3957967306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f88a4066f90 0x7f88a4069450 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.407+0000 7f88c758e640 1 --2- 172.21.15.60:0/3957967306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88c00cb0f0 0x7f88c0158510 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.407+0000 7f88c758e640 1 --2- 172.21.15.60:0/3957967306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c00cd770 0x7f88c01573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.407+0000 7f88c758e640 1 --2- 172.21.15.60:0/3957967306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac004d50 0x7f88c014daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.407+0000 7f88c758e640 1 -- 172.21.15.60:0/3957967306 >> 172.21.15.60:0/3957967306 conn(0x7f88c006c090 msgr2=0x7f88c01616c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.407+0000 7f88c758e640 1 -- 172.21.15.60:0/3957967306 shutdown_connections 2024-04-25T10:05:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.407+0000 7f88c758e640 1 -- 172.21.15.60:0/3957967306 wait complete. 2024-04-25T10:05:55.430 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:18.976643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:19.405302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:55.431 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f889cb98640 1 Processor -- start 2024-04-25T10:05:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f889cb98640 1 -- start start 2024-04-25T10:05:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f889cb98640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f889806c150 0x7f88980af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f889cb98640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88980b43a0 0x7f889815c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f889cb98640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f88980acff0 0x7f88980ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f889cb98640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88980ad9a0 con 0x7f889806c150 2024-04-25T10:05:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f889cb98640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88980b3650 con 0x7f88980acff0 2024-04-25T10:05:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f889cb98640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889815cd60 con 0x7f88980b43a0 2024-04-25T10:05:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f889659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f889806c150 0x7f88980af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f889659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f889806c150 0x7f88980af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54422/0 (socket says 172.21.15.60:54422) 2024-04-25T10:05:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f889659c640 1 -- 172.21.15.60:0/898412035 learned_addr learned my addr 172.21.15.60:0/898412035 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f8895d9b640 1 --2- 172.21.15.60:0/898412035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88980b43a0 0x7f889815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f8896d9d640 1 --2- 172.21.15.60:0/898412035 >> v2:172.21.15.100:3300/0 conn(0x7f88980acff0 0x7f88980ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f889659c640 1 -- 172.21.15.60:0/898412035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88980b43a0 msgr2=0x7f889815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:55.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f889659c640 1 --2- 172.21.15.60:0/898412035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88980b43a0 0x7f889815c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f889659c640 1 -- 172.21.15.60:0/898412035 >> v2:172.21.15.100:3300/0 conn(0x7f88980acff0 msgr2=0x7f88980ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:55.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f889659c640 1 --2- 172.21.15.60:0/898412035 >> v2:172.21.15.100:3300/0 conn(0x7f88980acff0 0x7f88980ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.547+0000 7f889659c640 1 -- 172.21.15.60:0/898412035 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889806c530 con 0x7f889806c150 2024-04-25T10:05:55.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.551+0000 7f889659c640 1 --2- 172.21.15.60:0/898412035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806c150 0x7f88980af1b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f88980b5470 tx=0x7f888801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f7fea140ffab27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:55.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.551+0000 7f889559a640 1 -- 172.21.15.60:0/898412035 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88880ab650 con 0x7f889806c150 2024-04-25T10:05:55.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.551+0000 7f889559a640 1 -- 172.21.15.60:0/898412035 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88880b0070 con 0x7f889806c150 2024-04-25T10:05:55.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.551+0000 7f889559a640 1 -- 172.21.15.60:0/898412035 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88880aba10 con 0x7f889806c150 2024-04-25T10:05:55.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.551+0000 7f889cb98640 1 -- 172.21.15.60:0/898412035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806c150 msgr2=0x7f88980af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.551+0000 7f889cb98640 1 --2- 172.21.15.60:0/898412035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806c150 0x7f88980af1b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f88980b5470 tx=0x7f888801c490 comp rx=0 tx=0).stop 2024-04-25T10:05:55.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.551+0000 7f889cb98640 1 -- 172.21.15.60:0/898412035 shutdown_connections 2024-04-25T10:05:55.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.551+0000 7f889cb98640 1 --2- 172.21.15.60:0/898412035 >> v2:172.21.15.100:3300/0 conn(0x7f88980acff0 0x7f88980ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.551+0000 7f889cb98640 1 --2- 172.21.15.60:0/898412035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88980b43a0 0x7f889815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.551+0000 7f889cb98640 1 --2- 172.21.15.60:0/898412035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806c150 0x7f88980af1b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.551+0000 7f889cb98640 1 -- 172.21.15.60:0/898412035 >> 172.21.15.60:0/898412035 conn(0x7f88980cfdd0 msgr2=0x7f88980d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:55.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.551+0000 7f889cb98640 1 -- 172.21.15.60:0/898412035 shutdown_connections 2024-04-25T10:05:55.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.551+0000 7f889cb98640 1 -- 172.21.15.60:0/898412035 wait complete. 2024-04-25T10:05:55.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889cb98640 1 Processor -- start 2024-04-25T10:05:55.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889cb98640 1 -- start start 2024-04-25T10:05:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889cb98640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806c150 0x7f88981501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889cb98640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88980acff0 0x7f889814f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889cb98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88980b43a0 0x7f889814f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889cb98640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889815fbd0 con 0x7f889806c150 2024-04-25T10:05:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889cb98640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88980af7c0 con 0x7f88980b43a0 2024-04-25T10:05:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889cb98640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889815fd50 con 0x7f88980acff0 2024-04-25T10:05:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806c150 0x7f88981501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806c150 0x7f88981501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54438/0 (socket says 172.21.15.60:54438) 2024-04-25T10:05:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889659c640 1 -- 172.21.15.60:0/1891574928 learned_addr learned my addr 172.21.15.60:0/1891574928 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f8895d9b640 1 --2- 172.21.15.60:0/1891574928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88980acff0 0x7f889814f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f8896d9d640 1 --2- 172.21.15.60:0/1891574928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88980b43a0 0x7f889814f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889659c640 1 -- 172.21.15.60:0/1891574928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88980acff0 msgr2=0x7f889814f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889659c640 1 --2- 172.21.15.60:0/1891574928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88980acff0 0x7f889814f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889659c640 1 -- 172.21.15.60:0/1891574928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88980b43a0 msgr2=0x7f889814f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889659c640 1 --2- 172.21.15.60:0/1891574928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88980b43a0 0x7f889814f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889659c640 1 -- 172.21.15.60:0/1891574928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8898146db0 con 0x7f889806c150 2024-04-25T10:05:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f8895d9b640 1 --2- 172.21.15.60:0/1891574928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88980acff0 0x7f889814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889659c640 1 --2- 172.21.15.60:0/1891574928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806c150 0x7f88981501e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f88880ad040 tx=0x7f888801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f8886ffd640 1 -- 172.21.15.60:0/1891574928 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88880be5f0 con 0x7f889806c150 2024-04-25T10:05:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f8886ffd640 1 -- 172.21.15.60:0/1891574928 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88880be790 con 0x7f889806c150 2024-04-25T10:05:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889cb98640 1 -- 172.21.15.60:0/1891574928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8898147090 con 0x7f889806c150 2024-04-25T10:05:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f8886ffd640 1 -- 172.21.15.60:0/1891574928 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88880beb80 con 0x7f889806c150 2024-04-25T10:05:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889cb98640 1 -- 172.21.15.60:0/1891574928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8898144b10 con 0x7f889806c150 2024-04-25T10:05:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f889cb98640 1 -- 172.21.15.60:0/1891574928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8898074770 con 0x7f889806c150 2024-04-25T10:05:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f8886ffd640 1 -- 172.21.15.60:0/1891574928 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88880b43f0 con 0x7f889806c150 2024-04-25T10:05:55.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f8886ffd640 1 --2- 172.21.15.60:0/1891574928 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8878066f60 0x7f8878069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.555+0000 7f8886ffd640 1 -- 172.21.15.60:0/1891574928 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f888811fe10 con 0x7f889806c150 2024-04-25T10:05:55.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.559+0000 7f8895d9b640 1 --2- 172.21.15.60:0/1891574928 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8878066f60 0x7f8878069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.559+0000 7f8895d9b640 1 --2- 172.21.15.60:0/1891574928 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8878066f60 0x7f8878069420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f88981493d0 tx=0x7f888c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:55.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.563+0000 7f8886ffd640 1 -- 172.21.15.60:0/1891574928 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8888016110 con 0x7f889806c150 2024-04-25T10:05:55.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.719+0000 7f889cb98640 1 -- 172.21.15.60:0/1891574928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8898147260 con 0x7f889806c150 2024-04-25T10:05:55.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.719+0000 7f8886ffd640 1 -- 172.21.15.60:0/1891574928 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88880f0fd0 con 0x7f889806c150 2024-04-25T10:05:55.727 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:05:55.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.723+0000 7f889cb98640 1 -- 172.21.15.60:0/1891574928 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8878066f60 msgr2=0x7f8878069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.723+0000 7f889cb98640 1 --2- 172.21.15.60:0/1891574928 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8878066f60 0x7f8878069420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f88981493d0 tx=0x7f888c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:55.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.723+0000 7f889cb98640 1 -- 172.21.15.60:0/1891574928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806c150 msgr2=0x7f88981501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.723+0000 7f889cb98640 1 --2- 172.21.15.60:0/1891574928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806c150 0x7f88981501e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f88880ad040 tx=0x7f888801acd0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.723+0000 7f889cb98640 1 -- 172.21.15.60:0/1891574928 shutdown_connections 2024-04-25T10:05:55.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.723+0000 7f889cb98640 1 --2- 172.21.15.60:0/1891574928 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8878066f60 0x7f8878069420 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.723+0000 7f889cb98640 1 --2- 172.21.15.60:0/1891574928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88980b43a0 0x7f889814f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.723+0000 7f889cb98640 1 --2- 172.21.15.60:0/1891574928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88980acff0 0x7f889814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.723+0000 7f889cb98640 1 --2- 172.21.15.60:0/1891574928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806c150 0x7f88981501e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.723+0000 7f889cb98640 1 -- 172.21.15.60:0/1891574928 >> 172.21.15.60:0/1891574928 conn(0x7f88980cfdd0 msgr2=0x7f88980addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:55.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.723+0000 7f889cb98640 1 -- 172.21.15.60:0/1891574928 shutdown_connections 2024-04-25T10:05:55.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.723+0000 7f889cb98640 1 -- 172.21.15.60:0/1891574928 wait complete. 2024-04-25T10:05:55.744 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:55.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced771f640 1 Processor -- start 2024-04-25T10:05:55.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced771f640 1 -- start start 2024-04-25T10:05:55.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced771f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fced015dd30 0x7fced015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced771f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fced00cf8a0 0x7fced015be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced771f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced00cd890 con 0x7fced00cd3d0 2024-04-25T10:05:55.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced771f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced00ce4a0 con 0x7fced00cf8a0 2024-04-25T10:05:55.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced771f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced00cdb70 con 0x7fced015dd30 2024-04-25T10:05:55.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced4cba640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fced00cd3d0 0x7fced00cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45524/0 (socket says 172.21.15.60:45524) 2024-04-25T10:05:55.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced4cba640 1 -- 172.21.15.60:0/1816206726 learned_addr learned my addr 172.21.15.60:0/1816206726 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:55.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced54bb640 1 --2- 172.21.15.60:0/1816206726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fced015dd30 0x7fced015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced5cbc640 1 --2- 172.21.15.60:0/1816206726 >> v2:172.21.15.100:3300/0 conn(0x7fced00cf8a0 0x7fced015be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fcec7fff640 1 -- 172.21.15.60:0/1816206726 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697793768 0 0) 0x7fced00cd890 con 0x7fced00cd3d0 2024-04-25T10:05:55.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fcec7fff640 1 -- 172.21.15.60:0/1816206726 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcebc003530 con 0x7fced00cd3d0 2024-04-25T10:05:55.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fcec7fff640 1 -- 172.21.15.60:0/1816206726 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4196140855 0 0) 0x7fcebc003530 con 0x7fced00cd3d0 2024-04-25T10:05:55.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fcec7fff640 1 -- 172.21.15.60:0/1816206726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fced015dd30 msgr2=0x7fced015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:55.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fcec7fff640 1 --2- 172.21.15.60:0/1816206726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fced015dd30 0x7fced015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fcec7fff640 1 -- 172.21.15.60:0/1816206726 >> v2:172.21.15.100:3300/0 conn(0x7fced00cf8a0 msgr2=0x7fced015be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:55.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fcec7fff640 1 --2- 172.21.15.60:0/1816206726 >> v2:172.21.15.100:3300/0 conn(0x7fced00cf8a0 0x7fced015be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fcec7fff640 1 -- 172.21.15.60:0/1816206726 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced00cde50 con 0x7fced00cd3d0 2024-04-25T10:05:55.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fcec7fff640 1 -- 172.21.15.60:0/1816206726 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fcec0001680 con 0x7fced00cd3d0 2024-04-25T10:05:55.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fcec7fff640 1 -- 172.21.15.60:0/1816206726 >> v1:172.21.15.60:6789/0 conn(0x7fced00cd3d0 legacy=0x7fced00cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fcec7fff640 1 --2- 172.21.15.60:0/1816206726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fced015dd30 0x7fcebc0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fcec7fff640 1 --2- 172.21.15.60:0/1816206726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcebc003f40 0x7fcebc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fcec7fff640 1 --2- 172.21.15.60:0/1816206726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcebc004d50 0x7fcebc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fcec7fff640 1 -- 172.21.15.60:0/1816206726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced00ce4a0 con 0x7fcebc004d50 2024-04-25T10:05:55.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fcec7fff640 1 -- 172.21.15.60:0/1816206726 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced00cdb70 con 0x7fcebc003f40 2024-04-25T10:05:55.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fcec7fff640 1 -- 172.21.15.60:0/1816206726 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc00ba20 con 0x7fced015dd30 2024-04-25T10:05:55.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced5cbc640 1 --2- 172.21.15.60:0/1816206726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcebc004d50 0x7fcebc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced4cba640 1 --2- 172.21.15.60:0/1816206726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcebc003f40 0x7fcebc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced54bb640 1 --2- 172.21.15.60:0/1816206726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fced015dd30 0x7fcebc0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced5cbc640 1 -- 172.21.15.60:0/1816206726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fced015dd30 msgr2=0x7fcebc0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:55.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced5cbc640 1 --2- 172.21.15.60:0/1816206726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fced015dd30 0x7fcebc0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced5cbc640 1 -- 172.21.15.60:0/1816206726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcebc003f40 msgr2=0x7fcebc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:55.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced5cbc640 1 --2- 172.21.15.60:0/1816206726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcebc003f40 0x7fcebc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.863+0000 7fced5cbc640 1 -- 172.21.15.60:0/1816206726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc003530 con 0x7fcebc004d50 2024-04-25T10:05:55.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.867+0000 7fced5cbc640 1 --2- 172.21.15.60:0/1816206726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcebc004d50 0x7fcebc0051d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcec800e170 tx=0x7fcec801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa739e3f8f72c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:55.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.867+0000 7fcec7fff640 1 -- 172.21.15.60:0/1816206726 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec80b2070 con 0x7fcebc004d50 2024-04-25T10:05:55.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.867+0000 7fcec7fff640 1 -- 172.21.15.60:0/1816206726 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec80a5ba0 con 0x7fcebc004d50 2024-04-25T10:05:55.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.867+0000 7fcec7fff640 1 -- 172.21.15.60:0/1816206726 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec8009c10 con 0x7fcebc004d50 2024-04-25T10:05:55.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.867+0000 7fced771f640 1 -- 172.21.15.60:0/1816206726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcebc004d50 msgr2=0x7fcebc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.867+0000 7fced771f640 1 --2- 172.21.15.60:0/1816206726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcebc004d50 0x7fcebc0051d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcec800e170 tx=0x7fcec801cca0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.867+0000 7fced5cbc640 1 -- 172.21.15.60:0/1816206726 reap_dead start 2024-04-25T10:05:55.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.867+0000 7fced771f640 1 -- 172.21.15.60:0/1816206726 shutdown_connections 2024-04-25T10:05:55.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.867+0000 7fced771f640 1 -- 172.21.15.60:0/1816206726 >> 172.21.15.60:0/1816206726 conn(0x7fced00670b0 msgr2=0x7fced006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:55.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.867+0000 7fced771f640 1 -- 172.21.15.60:0/1816206726 shutdown_connections 2024-04-25T10:05:55.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.867+0000 7fced771f640 1 -- 172.21.15.60:0/1816206726 wait complete. 2024-04-25T10:05:55.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.867+0000 7fced771f640 1 Processor -- start 2024-04-25T10:05:55.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.867+0000 7fced771f640 1 -- start start 2024-04-25T10:05:55.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced771f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fced0153d00 0x7fced01f1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced771f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fced006d250 0x7fced006d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced771f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fced00adeb0 0x7fced00ae330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced771f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced006f490 con 0x7fced0153d00 2024-04-25T10:05:55.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced771f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced006f310 con 0x7fced006d250 2024-04-25T10:05:55.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced771f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced006f610 con 0x7fced00adeb0 2024-04-25T10:05:55.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced54bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fced0153d00 0x7fced01f1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced54bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fced0153d00 0x7fced01f1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54470/0 (socket says 172.21.15.60:54470) 2024-04-25T10:05:55.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced54bb640 1 -- 172.21.15.60:0/3947900653 learned_addr learned my addr 172.21.15.60:0/3947900653 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:55.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced4cba640 1 --2- 172.21.15.60:0/3947900653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fced006d250 0x7fced006d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced5cbc640 1 --2- 172.21.15.60:0/3947900653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fced00adeb0 0x7fced00ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced54bb640 1 -- 172.21.15.60:0/3947900653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fced00adeb0 msgr2=0x7fced00ae330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced54bb640 1 --2- 172.21.15.60:0/3947900653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fced00adeb0 0x7fced00ae330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced54bb640 1 -- 172.21.15.60:0/3947900653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fced006d250 msgr2=0x7fced006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:55.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced54bb640 1 --2- 172.21.15.60:0/3947900653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fced006d250 0x7fced006d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:55.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced54bb640 1 -- 172.21.15.60:0/3947900653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced01f1780 con 0x7fced0153d00 2024-04-25T10:05:55.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced5cbc640 1 --2- 172.21.15.60:0/3947900653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fced00adeb0 0x7fced00ae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:55.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced54bb640 1 --2- 172.21.15.60:0/3947900653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fced0153d00 0x7fced01f1240 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0115e0 tx=0x7fcecc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:55.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fcec5ffb640 1 -- 172.21.15.60:0/3947900653 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc015870 con 0x7fced0153d00 2024-04-25T10:05:55.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fcec5ffb640 1 -- 172.21.15.60:0/3947900653 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc015a10 con 0x7fced0153d00 2024-04-25T10:05:55.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fcec5ffb640 1 -- 172.21.15.60:0/3947900653 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc015dc0 con 0x7fced0153d00 2024-04-25T10:05:55.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced771f640 1 -- 172.21.15.60:0/3947900653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced00a9640 con 0x7fced0153d00 2024-04-25T10:05:55.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced771f640 1 -- 172.21.15.60:0/3947900653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced00a9c00 con 0x7fced0153d00 2024-04-25T10:05:55.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fced771f640 1 -- 172.21.15.60:0/3947900653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced0074770 con 0x7fced0153d00 2024-04-25T10:05:55.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fcec5ffb640 1 -- 172.21.15.60:0/3947900653 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcecc00f480 con 0x7fced0153d00 2024-04-25T10:05:55.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.871+0000 7fcec5ffb640 1 --2- 172.21.15.60:0/3947900653 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fceb4066f90 0x7fceb4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:55.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.875+0000 7fcec5ffb640 1 -- 172.21.15.60:0/3947900653 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcecc0859a0 con 0x7fced0153d00 2024-04-25T10:05:55.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.875+0000 7fced4cba640 1 --2- 172.21.15.60:0/3947900653 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fceb4066f90 0x7fceb4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:55.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.875+0000 7fced4cba640 1 --2- 172.21.15.60:0/3947900653 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fceb4066f90 0x7fceb4069450 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fced0159290 tx=0x7fcec0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:55.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:55.879+0000 7fcec5ffb640 1 -- 172.21.15.60:0/3947900653 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcecc019030 con 0x7fced0153d00 2024-04-25T10:05:56.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.043+0000 7fced771f640 1 -- 172.21.15.60:0/3947900653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fced015aec0 con 0x7fced0153d00 2024-04-25T10:05:56.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.043+0000 7fcec5ffb640 1 -- 172.21.15.60:0/3947900653 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcecc056990 con 0x7fced0153d00 2024-04-25T10:05:56.051 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:05:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.047+0000 7fced771f640 1 -- 172.21.15.60:0/3947900653 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fceb4066f90 msgr2=0x7fceb4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.047+0000 7fced771f640 1 --2- 172.21.15.60:0/3947900653 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fceb4066f90 0x7fceb4069450 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fced0159290 tx=0x7fcec0009040 comp rx=0 tx=0).stop 2024-04-25T10:05:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.047+0000 7fced771f640 1 -- 172.21.15.60:0/3947900653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fced0153d00 msgr2=0x7fced01f1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.047+0000 7fced771f640 1 --2- 172.21.15.60:0/3947900653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fced0153d00 0x7fced01f1240 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0115e0 tx=0x7fcecc011aa0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.047+0000 7fced771f640 1 -- 172.21.15.60:0/3947900653 shutdown_connections 2024-04-25T10:05:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.047+0000 7fced771f640 1 --2- 172.21.15.60:0/3947900653 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fceb4066f90 0x7fceb4069450 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.047+0000 7fced771f640 1 --2- 172.21.15.60:0/3947900653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fced00adeb0 0x7fced00ae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.047+0000 7fced771f640 1 --2- 172.21.15.60:0/3947900653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fced006d250 0x7fced006d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.047+0000 7fced771f640 1 --2- 172.21.15.60:0/3947900653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fced0153d00 0x7fced01f1240 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.047+0000 7fced771f640 1 -- 172.21.15.60:0/3947900653 >> 172.21.15.60:0/3947900653 conn(0x7fced00670b0 msgr2=0x7fced00af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.047+0000 7fced771f640 1 -- 172.21.15.60:0/3947900653 shutdown_connections 2024-04-25T10:05:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.047+0000 7fced771f640 1 -- 172.21.15.60:0/3947900653 wait complete. 2024-04-25T10:05:56.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:56.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.183+0000 7f2505bca640 1 Processor -- start 2024-04-25T10:05:56.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.183+0000 7f2505bca640 1 -- start start 2024-04-25T10:05:56.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f2505bca640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25000cbcf0 0x7f2500160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f2505bca640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2500144a40 con 0x7f2500071c30 2024-04-25T10:05:56.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f2505bca640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25000cc9f0 con 0x7f250015fbf0 2024-04-25T10:05:56.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f2505bca640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25000b2dc0 con 0x7f25000cbcf0 2024-04-25T10:05:56.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24feffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2500071c30 0x7f250015f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45540/0 (socket says 172.21.15.60:45540) 2024-04-25T10:05:56.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24feffd640 1 -- 172.21.15.60:0/2562727652 learned_addr learned my addr 172.21.15.60:0/2562727652 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:56.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24ff7fe640 1 --2- 172.21.15.60:0/2562727652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25000cbcf0 0x7f2500160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24fe7fc640 1 -- 172.21.15.60:0/2562727652 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190646538 0 0) 0x7f2500144a40 con 0x7f2500071c30 2024-04-25T10:05:56.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24fe7fc640 1 -- 172.21.15.60:0/2562727652 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24ec003530 con 0x7f2500071c30 2024-04-25T10:05:56.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24fe7fc640 1 -- 172.21.15.60:0/2562727652 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 151644409 0 0) 0x7f24ec003530 con 0x7f2500071c30 2024-04-25T10:05:56.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24fe7fc640 1 -- 172.21.15.60:0/2562727652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25000cbcf0 msgr2=0x7f2500160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:56.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24fe7fc640 1 --2- 172.21.15.60:0/2562727652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25000cbcf0 0x7f2500160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24fe7fc640 1 -- 172.21.15.60:0/2562727652 >> v1:172.21.15.100:6789/0 conn(0x7f250015fbf0 legacy=0x7f25000ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24fe7fc640 1 -- 172.21.15.60:0/2562727652 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25000cccd0 con 0x7f2500071c30 2024-04-25T10:05:56.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24fe7fc640 1 -- 172.21.15.60:0/2562727652 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f24e8001680 con 0x7f2500071c30 2024-04-25T10:05:56.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24fe7fc640 1 -- 172.21.15.60:0/2562727652 >> v1:172.21.15.60:6789/0 conn(0x7f2500071c30 legacy=0x7f250015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24fe7fc640 1 --2- 172.21.15.60:0/2562727652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0039f0 0x7f24ec003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24fe7fc640 1 --2- 172.21.15.60:0/2562727652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec004810 0x7f24ec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24fe7fc640 1 --2- 172.21.15.60:0/2562727652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25000cbcf0 0x7f24ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24fe7fc640 1 -- 172.21.15.60:0/2562727652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec00ba80 con 0x7f24ec0039f0 2024-04-25T10:05:56.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24fe7fc640 1 -- 172.21.15.60:0/2562727652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec00bc30 con 0x7f24ec004810 2024-04-25T10:05:56.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24fe7fc640 1 -- 172.21.15.60:0/2562727652 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec00be30 con 0x7f25000cbcf0 2024-04-25T10:05:56.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24ff7fe640 1 --2- 172.21.15.60:0/2562727652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0039f0 0x7f24ec003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24feffd640 1 --2- 172.21.15.60:0/2562727652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec004810 0x7f24ec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24fffff640 1 --2- 172.21.15.60:0/2562727652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25000cbcf0 0x7f24ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24ff7fe640 1 -- 172.21.15.60:0/2562727652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25000cbcf0 msgr2=0x7f24ec0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:56.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24ff7fe640 1 --2- 172.21.15.60:0/2562727652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25000cbcf0 0x7f24ec0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24ff7fe640 1 -- 172.21.15.60:0/2562727652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec004810 msgr2=0x7f24ec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:56.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24ff7fe640 1 --2- 172.21.15.60:0/2562727652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec004810 0x7f24ec004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.187+0000 7f24ff7fe640 1 -- 172.21.15.60:0/2562727652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec003530 con 0x7f24ec0039f0 2024-04-25T10:05:56.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f24ff7fe640 1 --2- 172.21.15.60:0/2562727652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0039f0 0x7f24ec003e10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009ab0 tx=0x7f24f001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c21df70b747181 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:56.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f24fe7fc640 1 -- 172.21.15.60:0/2562727652 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f00096b0 con 0x7f24ec0039f0 2024-04-25T10:05:56.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f24fe7fc640 1 -- 172.21.15.60:0/2562727652 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f00a64f0 con 0x7f24ec0039f0 2024-04-25T10:05:56.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f24fe7fc640 1 -- 172.21.15.60:0/2562727652 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f00a68e0 con 0x7f24ec0039f0 2024-04-25T10:05:56.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f2505bca640 1 -- 172.21.15.60:0/2562727652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0039f0 msgr2=0x7f24ec003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f2505bca640 1 --2- 172.21.15.60:0/2562727652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0039f0 0x7f24ec003e10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009ab0 tx=0x7f24f001ca00 comp rx=0 tx=0).stop 2024-04-25T10:05:56.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f24fffff640 1 -- 172.21.15.60:0/2562727652 reap_dead start 2024-04-25T10:05:56.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f2505bca640 1 -- 172.21.15.60:0/2562727652 shutdown_connections 2024-04-25T10:05:56.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f2505bca640 1 -- 172.21.15.60:0/2562727652 >> 172.21.15.60:0/2562727652 conn(0x7f25000670b0 msgr2=0x7f250006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:56.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f2505bca640 1 -- 172.21.15.60:0/2562727652 shutdown_connections 2024-04-25T10:05:56.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f2505bca640 1 -- 172.21.15.60:0/2562727652 wait complete. 2024-04-25T10:05:56.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f2505bca640 1 Processor -- start 2024-04-25T10:05:56.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f2505bca640 1 -- start start 2024-04-25T10:05:56.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f2505bca640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0039f0 0x7f2500139bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f2505bca640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f250013db10 0x7f25001390c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f2505bca640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f250013c0c0 0x7f2500142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f2505bca640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25000ce740 con 0x7f24ec0039f0 2024-04-25T10:05:56.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f2505bca640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25000ce5c0 con 0x7f250013db10 2024-04-25T10:05:56.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f2505bca640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2500072030 con 0x7f250013c0c0 2024-04-25T10:05:56.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f24ff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0039f0 0x7f2500139bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f24ff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0039f0 0x7f2500139bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54484/0 (socket says 172.21.15.60:54484) 2024-04-25T10:05:56.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.191+0000 7f24ff7fe640 1 -- 172.21.15.60:0/427803575 learned_addr learned my addr 172.21.15.60:0/427803575 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:56.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24feffd640 1 --2- 172.21.15.60:0/427803575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f250013db10 0x7f25001390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24fffff640 1 --2- 172.21.15.60:0/427803575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f250013c0c0 0x7f2500142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24ff7fe640 1 -- 172.21.15.60:0/427803575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f250013c0c0 msgr2=0x7f2500142dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24ff7fe640 1 --2- 172.21.15.60:0/427803575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f250013c0c0 0x7f2500142dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24ff7fe640 1 -- 172.21.15.60:0/427803575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f250013db10 msgr2=0x7f25001390c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24ff7fe640 1 --2- 172.21.15.60:0/427803575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f250013db10 0x7f25001390c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24ff7fe640 1 -- 172.21.15.60:0/427803575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250013ca10 con 0x7f24ec0039f0 2024-04-25T10:05:56.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24feffd640 1 --2- 172.21.15.60:0/427803575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f250013db10 0x7f25001390c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:56.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24ff7fe640 1 --2- 172.21.15.60:0/427803575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0039f0 0x7f2500139bd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f24f00b0040 tx=0x7f24ec005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:56.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24f7fff640 1 -- 172.21.15.60:0/427803575 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f00bf020 con 0x7f24ec0039f0 2024-04-25T10:05:56.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24f7fff640 1 -- 172.21.15.60:0/427803575 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f0004020 con 0x7f24ec0039f0 2024-04-25T10:05:56.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24f7fff640 1 -- 172.21.15.60:0/427803575 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f00043f0 con 0x7f24ec0039f0 2024-04-25T10:05:56.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f2505bca640 1 -- 172.21.15.60:0/427803575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250013d030 con 0x7f24ec0039f0 2024-04-25T10:05:56.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f2505bca640 1 -- 172.21.15.60:0/427803575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250013d5f0 con 0x7f24ec0039f0 2024-04-25T10:05:56.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f2505bca640 1 -- 172.21.15.60:0/427803575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2500074770 con 0x7f24ec0039f0 2024-04-25T10:05:56.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24f7fff640 1 -- 172.21.15.60:0/427803575 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24f00c2070 con 0x7f24ec0039f0 2024-04-25T10:05:56.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24f7fff640 1 --2- 172.21.15.60:0/427803575 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f24e0066f60 0x7f24e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24f7fff640 1 -- 172.21.15.60:0/427803575 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24f011f290 con 0x7f24ec0039f0 2024-04-25T10:05:56.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24feffd640 1 --2- 172.21.15.60:0/427803575 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f24e0066f60 0x7f24e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.195+0000 7f24feffd640 1 --2- 172.21.15.60:0/427803575 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f24e0066f60 0x7f24e0069420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2500143a80 tx=0x7f24e8000f00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:56.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.199+0000 7f24f7fff640 1 -- 172.21.15.60:0/427803575 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f24f0016740 con 0x7f24ec0039f0 2024-04-25T10:05:56.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.363+0000 7f2505bca640 1 -- 172.21.15.60:0/427803575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2500139600 con 0x7f24ec0039f0 2024-04-25T10:05:56.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.363+0000 7f24f7fff640 1 -- 172.21.15.60:0/427803575 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f24f00f0210 con 0x7f24ec0039f0 2024-04-25T10:05:56.369 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:05:56.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.367+0000 7f2505bca640 1 -- 172.21.15.60:0/427803575 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f24e0066f60 msgr2=0x7f24e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.367+0000 7f2505bca640 1 --2- 172.21.15.60:0/427803575 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f24e0066f60 0x7f24e0069420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2500143a80 tx=0x7f24e8000f00 comp rx=0 tx=0).stop 2024-04-25T10:05:56.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.367+0000 7f2505bca640 1 -- 172.21.15.60:0/427803575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0039f0 msgr2=0x7f2500139bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.367+0000 7f2505bca640 1 --2- 172.21.15.60:0/427803575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0039f0 0x7f2500139bd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f24f00b0040 tx=0x7f24ec005840 comp rx=0 tx=0).stop 2024-04-25T10:05:56.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.367+0000 7f2505bca640 1 -- 172.21.15.60:0/427803575 shutdown_connections 2024-04-25T10:05:56.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.367+0000 7f2505bca640 1 --2- 172.21.15.60:0/427803575 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f24e0066f60 0x7f24e0069420 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.367+0000 7f2505bca640 1 --2- 172.21.15.60:0/427803575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f250013c0c0 0x7f2500142dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.367+0000 7f2505bca640 1 --2- 172.21.15.60:0/427803575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f250013db10 0x7f25001390c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.367+0000 7f2505bca640 1 --2- 172.21.15.60:0/427803575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0039f0 0x7f2500139bd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.367+0000 7f2505bca640 1 -- 172.21.15.60:0/427803575 >> 172.21.15.60:0/427803575 conn(0x7f25000670b0 msgr2=0x7f250015d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:56.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.367+0000 7f2505bca640 1 -- 172.21.15.60:0/427803575 shutdown_connections 2024-04-25T10:05:56.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.367+0000 7f2505bca640 1 -- 172.21.15.60:0/427803575 wait complete. 2024-04-25T10:05:56.390 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:05:56.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.503+0000 7f194dd61640 1 Processor -- start 2024-04-25T10:05:56.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.503+0000 7f194dd61640 1 -- start start 2024-04-25T10:05:56.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.503+0000 7f194dd61640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f19480cdc30 0x7f194806ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.503+0000 7f194dd61640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f19480cff30 0x7f19480cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.503+0000 7f194dd61640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19480d03b0 con 0x7f19480cfae0 2024-04-25T10:05:56.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.503+0000 7f194dd61640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19480d0690 con 0x7f19480cff30 2024-04-25T10:05:56.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.503+0000 7f194dd61640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19480d0970 con 0x7f19480cdc30 2024-04-25T10:05:56.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.503+0000 7f1947fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f19480cff30 0x7f19480cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.503+0000 7f1947fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f19480cff30 0x7f19480cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54500/0 (socket says 172.21.15.60:54500) 2024-04-25T10:05:56.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.503+0000 7f1947fff640 1 -- 172.21.15.60:0/1597178910 learned_addr learned my addr 172.21.15.60:0/1597178910 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:56.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.503+0000 7f19477fe640 1 --2- 172.21.15.60:0/1597178910 >> v2:172.21.15.100:3300/0 conn(0x7f19480cdc30 0x7f194806ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.503+0000 7f1947fff640 1 -- 172.21.15.60:0/1597178910 >> v2:172.21.15.100:3300/0 conn(0x7f19480cdc30 msgr2=0x7f194806ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:56.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.503+0000 7f1947fff640 1 --2- 172.21.15.60:0/1597178910 >> v2:172.21.15.100:3300/0 conn(0x7f19480cdc30 0x7f194806ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.503+0000 7f1947fff640 1 -- 172.21.15.60:0/1597178910 >> v1:172.21.15.100:6789/0 conn(0x7f19480cfae0 legacy=0x7f1948099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:56.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.503+0000 7f1947fff640 1 -- 172.21.15.60:0/1597178910 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19480d0c50 con 0x7f19480cff30 2024-04-25T10:05:56.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.507+0000 7f1947fff640 1 --2- 172.21.15.60:0/1597178910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19480cff30 0x7f19480cbe00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1940009e50 tx=0x7f194001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779d287e7f283301 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:56.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.507+0000 7f19467fc640 1 -- 172.21.15.60:0/1597178910 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19400a5670 con 0x7f19480cff30 2024-04-25T10:05:56.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.507+0000 7f19467fc640 1 -- 172.21.15.60:0/1597178910 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19400a5810 con 0x7f19480cff30 2024-04-25T10:05:56.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.507+0000 7f19467fc640 1 -- 172.21.15.60:0/1597178910 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19400a5be0 con 0x7f19480cff30 2024-04-25T10:05:56.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.507+0000 7f194dd61640 1 -- 172.21.15.60:0/1597178910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19480cff30 msgr2=0x7f19480cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.507+0000 7f194dd61640 1 --2- 172.21.15.60:0/1597178910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19480cff30 0x7f19480cbe00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1940009e50 tx=0x7f194001c9b0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.507+0000 7f194dd61640 1 -- 172.21.15.60:0/1597178910 shutdown_connections 2024-04-25T10:05:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.507+0000 7f194dd61640 1 --2- 172.21.15.60:0/1597178910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19480cff30 0x7f19480cbe00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.507+0000 7f194dd61640 1 --2- 172.21.15.60:0/1597178910 >> v2:172.21.15.100:3300/0 conn(0x7f19480cdc30 0x7f194806ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.507+0000 7f194dd61640 1 -- 172.21.15.60:0/1597178910 >> 172.21.15.60:0/1597178910 conn(0x7f19480b32a0 msgr2=0x7f19480b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.507+0000 7f194dd61640 1 -- 172.21.15.60:0/1597178910 shutdown_connections 2024-04-25T10:05:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.507+0000 7f194dd61640 1 -- 172.21.15.60:0/1597178910 wait complete. 2024-04-25T10:05:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.507+0000 7f194dd61640 1 Processor -- start 2024-04-25T10:05:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.507+0000 7f194dd61640 1 -- start start 2024-04-25T10:05:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f194dd61640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19480cdc30 0x7f19481465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f194dd61640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19480cfae0 0x7f194815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f194dd61640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19480cff30 0x7f194815b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f194dd61640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19480ccb50 con 0x7f19480cff30 2024-04-25T10:05:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f194dd61640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194809a350 con 0x7f19480cfae0 2024-04-25T10:05:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f194dd61640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19480cccd0 con 0x7f19480cdc30 2024-04-25T10:05:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f1947fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19480cff30 0x7f194815b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f1947fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19480cff30 0x7f194815b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54504/0 (socket says 172.21.15.60:54504) 2024-04-25T10:05:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f1947fff640 1 -- 172.21.15.60:0/1249842866 learned_addr learned my addr 172.21.15.60:0/1249842866 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f1946ffd640 1 --2- 172.21.15.60:0/1249842866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19480cfae0 0x7f194815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f19477fe640 1 --2- 172.21.15.60:0/1249842866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19480cdc30 0x7f19481465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f1947fff640 1 -- 172.21.15.60:0/1249842866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19480cdc30 msgr2=0x7f19481465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f1947fff640 1 --2- 172.21.15.60:0/1249842866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19480cdc30 0x7f19481465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f1947fff640 1 -- 172.21.15.60:0/1249842866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19480cfae0 msgr2=0x7f194815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f1947fff640 1 --2- 172.21.15.60:0/1249842866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19480cfae0 0x7f194815d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f1947fff640 1 -- 172.21.15.60:0/1249842866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194815dfb0 con 0x7f19480cff30 2024-04-25T10:05:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f1947fff640 1 --2- 172.21.15.60:0/1249842866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19480cff30 0x7f194815b040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f194001a5e0 tx=0x7f1940019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f192ffff640 1 -- 172.21.15.60:0/1249842866 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19400a5670 con 0x7f19480cff30 2024-04-25T10:05:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f192ffff640 1 -- 172.21.15.60:0/1249842866 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19400be070 con 0x7f19480cff30 2024-04-25T10:05:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f194dd61640 1 -- 172.21.15.60:0/1249842866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194815e290 con 0x7f19480cff30 2024-04-25T10:05:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f192ffff640 1 -- 172.21.15.60:0/1249842866 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940009510 con 0x7f19480cff30 2024-04-25T10:05:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f194dd61640 1 -- 172.21.15.60:0/1249842866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194815be70 con 0x7f19480cff30 2024-04-25T10:05:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f194dd61640 1 -- 172.21.15.60:0/1249842866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1948074770 con 0x7f19480cff30 2024-04-25T10:05:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f192ffff640 1 -- 172.21.15.60:0/1249842866 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19400b8aa0 con 0x7f19480cff30 2024-04-25T10:05:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f192ffff640 1 --2- 172.21.15.60:0/1249842866 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1934066f60 0x7f1934069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f192ffff640 1 -- 172.21.15.60:0/1249842866 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19401203b0 con 0x7f19480cff30 2024-04-25T10:05:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.511+0000 7f19477fe640 1 --2- 172.21.15.60:0/1249842866 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1934066f60 0x7f1934069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.515+0000 7f19477fe640 1 --2- 172.21.15.60:0/1249842866 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1934066f60 0x7f1934069420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f193800b9d0 tx=0x7f1938008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.515+0000 7f192ffff640 1 -- 172.21.15.60:0/1249842866 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f19400165a0 con 0x7f19480cff30 2024-04-25T10:05:56.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.687+0000 7f194dd61640 1 -- 172.21.15.60:0/1249842866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f194806d9d0 con 0x7f19480cff30 2024-04-25T10:05:56.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.687+0000 7f192ffff640 1 -- 172.21.15.60:0/1249842866 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f19400f1450 con 0x7f19480cff30 2024-04-25T10:05:56.696 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:05:56.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.691+0000 7f194dd61640 1 -- 172.21.15.60:0/1249842866 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1934066f60 msgr2=0x7f1934069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.691+0000 7f194dd61640 1 --2- 172.21.15.60:0/1249842866 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1934066f60 0x7f1934069420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f193800b9d0 tx=0x7f1938008040 comp rx=0 tx=0).stop 2024-04-25T10:05:56.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.691+0000 7f194dd61640 1 -- 172.21.15.60:0/1249842866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19480cff30 msgr2=0x7f194815b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.691+0000 7f194dd61640 1 --2- 172.21.15.60:0/1249842866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19480cff30 0x7f194815b040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f194001a5e0 tx=0x7f1940019750 comp rx=0 tx=0).stop 2024-04-25T10:05:56.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.691+0000 7f194dd61640 1 -- 172.21.15.60:0/1249842866 shutdown_connections 2024-04-25T10:05:56.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.691+0000 7f194dd61640 1 --2- 172.21.15.60:0/1249842866 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1934066f60 0x7f1934069420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.691+0000 7f194dd61640 1 --2- 172.21.15.60:0/1249842866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19480cff30 0x7f194815b040 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.691+0000 7f194dd61640 1 --2- 172.21.15.60:0/1249842866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19480cfae0 0x7f194815d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.691+0000 7f194dd61640 1 --2- 172.21.15.60:0/1249842866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19480cdc30 0x7f19481465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.691+0000 7f194dd61640 1 -- 172.21.15.60:0/1249842866 >> 172.21.15.60:0/1249842866 conn(0x7f19480b32a0 msgr2=0x7f194806f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:56.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.691+0000 7f194dd61640 1 -- 172.21.15.60:0/1249842866 shutdown_connections 2024-04-25T10:05:56.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.691+0000 7f194dd61640 1 -- 172.21.15.60:0/1249842866 wait complete. 2024-04-25T10:05:56.715 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:05:56.715 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:56.721 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:56.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.871+0000 7fe4f7cc8640 1 Processor -- start 2024-04-25T10:05:56.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.871+0000 7fe4f7cc8640 1 -- start start 2024-04-25T10:05:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f7cc8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4f00ce7a0 0x7fe4f00ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f7cc8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe4f00b4660 0x7fe4f00b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f7cc8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f00cf0c0 con 0x7fe4f015bc90 2024-04-25T10:05:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f7cc8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f00cc370 con 0x7fe4f00b4660 2024-04-25T10:05:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f7cc8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f0071ca0 con 0x7fe4f00ce7a0 2024-04-25T10:05:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f5263640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe4f015bc90 0x7fe4f00b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45542/0 (socket says 172.21.15.60:45542) 2024-04-25T10:05:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f5263640 1 -- 172.21.15.60:0/123081635 learned_addr learned my addr 172.21.15.60:0/123081635 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:05:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f5a64640 1 --2- 172.21.15.60:0/123081635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4f00ce7a0 0x7fe4f00ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f6265640 1 --2- 172.21.15.60:0/123081635 >> v2:172.21.15.100:3300/0 conn(0x7fe4f00b4660 0x7fe4f00b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f4a62640 1 -- 172.21.15.60:0/123081635 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453896417 0 0) 0x7fe4f00cf0c0 con 0x7fe4f015bc90 2024-04-25T10:05:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f4a62640 1 -- 172.21.15.60:0/123081635 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4dc003530 con 0x7fe4f015bc90 2024-04-25T10:05:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f4a62640 1 -- 172.21.15.60:0/123081635 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 741613924 0 0) 0x7fe4dc003530 con 0x7fe4f015bc90 2024-04-25T10:05:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f4a62640 1 -- 172.21.15.60:0/123081635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4f00ce7a0 msgr2=0x7fe4f00ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f4a62640 1 --2- 172.21.15.60:0/123081635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4f00ce7a0 0x7fe4f00ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f4a62640 1 -- 172.21.15.60:0/123081635 >> v2:172.21.15.100:3300/0 conn(0x7fe4f00b4660 msgr2=0x7fe4f00b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f4a62640 1 --2- 172.21.15.60:0/123081635 >> v2:172.21.15.100:3300/0 conn(0x7fe4f00b4660 0x7fe4f00b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f4a62640 1 -- 172.21.15.60:0/123081635 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f00cca40 con 0x7fe4f015bc90 2024-04-25T10:05:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f4a62640 1 -- 172.21.15.60:0/123081635 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fe4e0001680 con 0x7fe4f015bc90 2024-04-25T10:05:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f4a62640 1 -- 172.21.15.60:0/123081635 >> v1:172.21.15.60:6789/0 conn(0x7fe4f015bc90 legacy=0x7fe4f00b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f4a62640 1 --2- 172.21.15.60:0/123081635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0039c0 0x7fe4dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f4a62640 1 --2- 172.21.15.60:0/123081635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4dc004810 0x7fe4dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f4a62640 1 --2- 172.21.15.60:0/123081635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4f00ce7a0 0x7fe4dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f4a62640 1 -- 172.21.15.60:0/123081635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f0071ca0 con 0x7fe4dc0039c0 2024-04-25T10:05:56.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f4a62640 1 -- 172.21.15.60:0/123081635 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc00b9d0 con 0x7fe4dc004810 2024-04-25T10:05:56.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f4a62640 1 -- 172.21.15.60:0/123081635 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc00bb50 con 0x7fe4f00ce7a0 2024-04-25T10:05:56.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f5a64640 1 --2- 172.21.15.60:0/123081635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0039c0 0x7fe4dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f6265640 1 --2- 172.21.15.60:0/123081635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4f00ce7a0 0x7fe4dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f5263640 1 --2- 172.21.15.60:0/123081635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4dc004810 0x7fe4dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f5a64640 1 -- 172.21.15.60:0/123081635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4f00ce7a0 msgr2=0x7fe4dc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:56.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f5a64640 1 --2- 172.21.15.60:0/123081635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4f00ce7a0 0x7fe4dc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f5a64640 1 -- 172.21.15.60:0/123081635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4dc004810 msgr2=0x7fe4dc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:56.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f5a64640 1 --2- 172.21.15.60:0/123081635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4dc004810 0x7fe4dc004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f5a64640 1 -- 172.21.15.60:0/123081635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc003530 con 0x7fe4dc0039c0 2024-04-25T10:05:56.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f6265640 1 --2- 172.21.15.60:0/123081635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4f00ce7a0 0x7fe4dc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:56.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.875+0000 7fe4f5a64640 1 --2- 172.21.15.60:0/123081635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0039c0 0x7fe4dc003de0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009ae0 tx=0x7fe4ec01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2464cacc1c97e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:56.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f4a62640 1 -- 172.21.15.60:0/123081635 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0096b0 con 0x7fe4dc0039c0 2024-04-25T10:05:56.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f4a62640 1 -- 172.21.15.60:0/123081635 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec0a9470 con 0x7fe4dc0039c0 2024-04-25T10:05:56.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f4a62640 1 -- 172.21.15.60:0/123081635 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0a9860 con 0x7fe4dc0039c0 2024-04-25T10:05:56.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f7cc8640 1 -- 172.21.15.60:0/123081635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0039c0 msgr2=0x7fe4dc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f7cc8640 1 --2- 172.21.15.60:0/123081635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0039c0 0x7fe4dc003de0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009ae0 tx=0x7fe4ec01ca70 comp rx=0 tx=0).stop 2024-04-25T10:05:56.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f6265640 1 -- 172.21.15.60:0/123081635 reap_dead start 2024-04-25T10:05:56.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f7cc8640 1 -- 172.21.15.60:0/123081635 shutdown_connections 2024-04-25T10:05:56.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f7cc8640 1 -- 172.21.15.60:0/123081635 >> 172.21.15.60:0/123081635 conn(0x7fe4f00b12d0 msgr2=0x7fe4f00b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:56.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f7cc8640 1 -- 172.21.15.60:0/123081635 shutdown_connections 2024-04-25T10:05:56.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f7cc8640 1 -- 172.21.15.60:0/123081635 wait complete. 2024-04-25T10:05:56.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f7cc8640 1 Processor -- start 2024-04-25T10:05:56.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f7cc8640 1 -- start start 2024-04-25T10:05:56.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f7cc8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0039c0 0x7fe4f01f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f7cc8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4f00b0b20 0x7fe4f00b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f7cc8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4f01f8330 0x7fe4f0151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f7cc8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00b4280 con 0x7fe4dc0039c0 2024-04-25T10:05:56.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f7cc8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00b3db0 con 0x7fe4f01f8330 2024-04-25T10:05:56.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f7cc8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00b4400 con 0x7fe4f00b0b20 2024-04-25T10:05:56.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f5a64640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0039c0 0x7fe4f01f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f5a64640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0039c0 0x7fe4f01f45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54524/0 (socket says 172.21.15.60:54524) 2024-04-25T10:05:56.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f5a64640 1 -- 172.21.15.60:0/3994787359 learned_addr learned my addr 172.21.15.60:0/3994787359 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:56.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f6265640 1 --2- 172.21.15.60:0/3994787359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4f01f8330 0x7fe4f0151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f5263640 1 --2- 172.21.15.60:0/3994787359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4f00b0b20 0x7fe4f00b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f5a64640 1 -- 172.21.15.60:0/3994787359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4f00b0b20 msgr2=0x7fe4f00b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f5a64640 1 --2- 172.21.15.60:0/3994787359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4f00b0b20 0x7fe4f00b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f5a64640 1 -- 172.21.15.60:0/3994787359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4f01f8330 msgr2=0x7fe4f0151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f5a64640 1 --2- 172.21.15.60:0/3994787359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4f01f8330 0x7fe4f0151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f5a64640 1 -- 172.21.15.60:0/3994787359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f01f4b30 con 0x7fe4dc0039c0 2024-04-25T10:05:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f5a64640 1 --2- 172.21.15.60:0/3994787359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0039c0 0x7fe4f01f45f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009ae0 tx=0x7fe4dc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4e67fc640 1 -- 172.21.15.60:0/3994787359 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec00f000 con 0x7fe4dc0039c0 2024-04-25T10:05:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4e67fc640 1 -- 172.21.15.60:0/3994787359 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec0a9b20 con 0x7fe4dc0039c0 2024-04-25T10:05:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f7cc8640 1 -- 172.21.15.60:0/3994787359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f01f4dd0 con 0x7fe4dc0039c0 2024-04-25T10:05:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4e67fc640 1 -- 172.21.15.60:0/3994787359 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec004020 con 0x7fe4dc0039c0 2024-04-25T10:05:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.879+0000 7fe4f7cc8640 1 -- 172.21.15.60:0/3994787359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f01f5390 con 0x7fe4dc0039c0 2024-04-25T10:05:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.883+0000 7fe4f7cc8640 1 -- 172.21.15.60:0/3994787359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f0074770 con 0x7fe4dc0039c0 2024-04-25T10:05:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.883+0000 7fe4e67fc640 1 -- 172.21.15.60:0/3994787359 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe4ec0c3060 con 0x7fe4dc0039c0 2024-04-25T10:05:56.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.883+0000 7fe4e67fc640 1 --2- 172.21.15.60:0/3994787359 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe4d4066f90 0x7fe4d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:56.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.883+0000 7fe4e67fc640 1 -- 172.21.15.60:0/3994787359 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4ec11f180 con 0x7fe4dc0039c0 2024-04-25T10:05:56.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.883+0000 7fe4f5263640 1 --2- 172.21.15.60:0/3994787359 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe4d4066f90 0x7fe4d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:56.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.883+0000 7fe4f5263640 1 --2- 172.21.15.60:0/3994787359 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe4d4066f90 0x7fe4d4069450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00b34d0 tx=0x7fe4e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:56.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:56.887+0000 7fe4e67fc640 1 -- 172.21.15.60:0/3994787359 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe4ec0ad020 con 0x7fe4dc0039c0 2024-04-25T10:05:57.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.043+0000 7fe4f7cc8640 1 -- 172.21.15.60:0/3994787359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4f015aab0 con 0x7fe4dc0039c0 2024-04-25T10:05:57.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.043+0000 7fe4e67fc640 1 -- 172.21.15.60:0/3994787359 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16617 (secure 0 0 0) 0x7fe4ec016780 con 0x7fe4dc0039c0 2024-04-25T10:05:57.051 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:57.051 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:22.036582+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:57.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.047+0000 7fe4f7cc8640 1 -- 172.21.15.60:0/3994787359 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe4d4066f90 msgr2=0x7fe4d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.047+0000 7fe4f7cc8640 1 --2- 172.21.15.60:0/3994787359 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe4d4066f90 0x7fe4d4069450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00b34d0 tx=0x7fe4e0009040 comp rx=0 tx=0).stop 2024-04-25T10:05:57.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.047+0000 7fe4f7cc8640 1 -- 172.21.15.60:0/3994787359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0039c0 msgr2=0x7fe4f01f45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.047+0000 7fe4f7cc8640 1 --2- 172.21.15.60:0/3994787359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0039c0 0x7fe4f01f45f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009ae0 tx=0x7fe4dc005840 comp rx=0 tx=0).stop 2024-04-25T10:05:57.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.047+0000 7fe4f7cc8640 1 -- 172.21.15.60:0/3994787359 shutdown_connections 2024-04-25T10:05:57.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.047+0000 7fe4f7cc8640 1 --2- 172.21.15.60:0/3994787359 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe4d4066f90 0x7fe4d4069450 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.047+0000 7fe4f7cc8640 1 --2- 172.21.15.60:0/3994787359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4f01f8330 0x7fe4f0151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.047+0000 7fe4f7cc8640 1 --2- 172.21.15.60:0/3994787359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4f00b0b20 0x7fe4f00b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.047+0000 7fe4f7cc8640 1 --2- 172.21.15.60:0/3994787359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0039c0 0x7fe4f01f45f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.047+0000 7fe4f7cc8640 1 -- 172.21.15.60:0/3994787359 >> 172.21.15.60:0/3994787359 conn(0x7fe4f00b12d0 msgr2=0x7fe4f015aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.047+0000 7fe4f7cc8640 1 -- 172.21.15.60:0/3994787359 shutdown_connections 2024-04-25T10:05:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.047+0000 7fe4f7cc8640 1 -- 172.21.15.60:0/3994787359 wait complete. 2024-04-25T10:05:57.069 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:18.976643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:19.405302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:57.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:57.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.191+0000 7faa7fc04640 1 Processor -- start 2024-04-25T10:05:57.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.191+0000 7faa7fc04640 1 -- start start 2024-04-25T10:05:57.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.195+0000 7faa7fc04640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faa7815db10 0x7faa780b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.195+0000 7faa7fc04640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa780ae140 con 0x7faa780cfcf0 2024-04-25T10:05:57.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.195+0000 7faa7fc04640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7815def0 con 0x7faa780aec10 2024-04-25T10:05:57.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.195+0000 7faa7fc04640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa780accc0 con 0x7faa7815db10 2024-04-25T10:05:57.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.195+0000 7faa7d9a0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faa7815db10 0x7faa780b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.195+0000 7faa7d9a0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faa7815db10 0x7faa780b2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54528/0 (socket says 172.21.15.60:54528) 2024-04-25T10:05:57.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.195+0000 7faa7d9a0640 1 -- 172.21.15.60:0/3286939293 learned_addr learned my addr 172.21.15.60:0/3286939293 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:57.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.195+0000 7faa7e1a1640 1 --1- 172.21.15.60:0/3286939293 >> v1:172.21.15.60:6789/0 conn(0x7faa780cfcf0 0x7faa78161d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45556/0 (socket says 172.21.15.60:45556) 2024-04-25T10:05:57.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.195+0000 7faa7d9a0640 1 -- 172.21.15.60:0/3286939293 >> v1:172.21.15.100:6789/0 conn(0x7faa780aec10 legacy=0x7faa78161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.195+0000 7faa7d9a0640 1 -- 172.21.15.60:0/3286939293 >> v1:172.21.15.60:6789/0 conn(0x7faa780cfcf0 legacy=0x7faa78161d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.195+0000 7faa7d9a0640 1 -- 172.21.15.60:0/3286939293 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa78162480 con 0x7faa7815db10 2024-04-25T10:05:57.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.195+0000 7faa7d9a0640 1 --2- 172.21.15.60:0/3286939293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa7815db10 0x7faa780b2ee0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faa70009870 tx=0x7faa7001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ed70273f0760a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.195+0000 7faa7c99e640 1 -- 172.21.15.60:0/3286939293 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa700ab430 con 0x7faa7815db10 2024-04-25T10:05:57.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.195+0000 7faa7c99e640 1 -- 172.21.15.60:0/3286939293 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa700b0070 con 0x7faa7815db10 2024-04-25T10:05:57.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.195+0000 7faa7c99e640 1 -- 172.21.15.60:0/3286939293 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa700ab7f0 con 0x7faa7815db10 2024-04-25T10:05:57.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 -- 172.21.15.60:0/3286939293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa7815db10 msgr2=0x7faa780b2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 --2- 172.21.15.60:0/3286939293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa7815db10 0x7faa780b2ee0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faa70009870 tx=0x7faa7001c520 comp rx=0 tx=0).stop 2024-04-25T10:05:57.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 -- 172.21.15.60:0/3286939293 shutdown_connections 2024-04-25T10:05:57.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 --2- 172.21.15.60:0/3286939293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa7815db10 0x7faa780b2ee0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 -- 172.21.15.60:0/3286939293 >> 172.21.15.60:0/3286939293 conn(0x7faa780670b0 msgr2=0x7faa7806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 -- 172.21.15.60:0/3286939293 shutdown_connections 2024-04-25T10:05:57.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 -- 172.21.15.60:0/3286939293 wait complete. 2024-04-25T10:05:57.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 Processor -- start 2024-04-25T10:05:57.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 -- start start 2024-04-25T10:05:57.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa780aec10 0x7faa7814bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa780cfcf0 0x7faa7814c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa7815db10 0x7faa7815f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7806db70 con 0x7faa780aec10 2024-04-25T10:05:57.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa78071ed0 con 0x7faa7815db10 2024-04-25T10:05:57.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7806dcf0 con 0x7faa780cfcf0 2024-04-25T10:05:57.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7d9a0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa780aec10 0x7faa7814bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7d9a0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa780aec10 0x7faa7814bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54538/0 (socket says 172.21.15.60:54538) 2024-04-25T10:05:57.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7d9a0640 1 -- 172.21.15.60:0/453379295 learned_addr learned my addr 172.21.15.60:0/453379295 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:57.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7d19f640 1 --2- 172.21.15.60:0/453379295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa780cfcf0 0x7faa7814c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7d9a0640 1 -- 172.21.15.60:0/453379295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa780cfcf0 msgr2=0x7faa7814c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7e1a1640 1 --2- 172.21.15.60:0/453379295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa7815db10 0x7faa7815f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7d9a0640 1 --2- 172.21.15.60:0/453379295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa780cfcf0 0x7faa7814c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7d9a0640 1 -- 172.21.15.60:0/453379295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa7815db10 msgr2=0x7faa7815f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7d9a0640 1 --2- 172.21.15.60:0/453379295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa7815db10 0x7faa7815f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7d9a0640 1 -- 172.21.15.60:0/453379295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa780afb00 con 0x7faa780aec10 2024-04-25T10:05:57.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7d9a0640 1 --2- 172.21.15.60:0/453379295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa780aec10 0x7faa7814bea0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7faa70019130 tx=0x7faa7001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa6e7fc640 1 -- 172.21.15.60:0/453379295 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa700b4800 con 0x7faa780aec10 2024-04-25T10:05:57.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa6e7fc640 1 -- 172.21.15.60:0/453379295 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa700b49a0 con 0x7faa780aec10 2024-04-25T10:05:57.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 -- 172.21.15.60:0/453379295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa78151370 con 0x7faa780aec10 2024-04-25T10:05:57.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa6e7fc640 1 -- 172.21.15.60:0/453379295 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa700b4d90 con 0x7faa780aec10 2024-04-25T10:05:57.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.199+0000 7faa7fc04640 1 -- 172.21.15.60:0/453379295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa78151930 con 0x7faa780aec10 2024-04-25T10:05:57.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.203+0000 7faa7fc04640 1 -- 172.21.15.60:0/453379295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7806bba0 con 0x7faa780aec10 2024-04-25T10:05:57.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.203+0000 7faa6e7fc640 1 -- 172.21.15.60:0/453379295 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa700b3380 con 0x7faa780aec10 2024-04-25T10:05:57.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.203+0000 7faa6e7fc640 1 --2- 172.21.15.60:0/453379295 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faa5c066f60 0x7faa5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.203+0000 7faa6e7fc640 1 -- 172.21.15.60:0/453379295 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa700a8070 con 0x7faa780aec10 2024-04-25T10:05:57.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.203+0000 7faa7d19f640 1 --2- 172.21.15.60:0/453379295 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faa5c066f60 0x7faa5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.203+0000 7faa7d19f640 1 --2- 172.21.15.60:0/453379295 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faa5c066f60 0x7faa5c069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa78072410 tx=0x7faa68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.207+0000 7faa6e7fc640 1 -- 172.21.15.60:0/453379295 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa70016110 con 0x7faa780aec10 2024-04-25T10:05:57.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.367+0000 7faa7fc04640 1 -- 172.21.15.60:0/453379295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa7806bba0 con 0x7faa780aec10 2024-04-25T10:05:57.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.367+0000 7faa6e7fc640 1 -- 172.21.15.60:0/453379295 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa700f0470 con 0x7faa780aec10 2024-04-25T10:05:57.374 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:05:57.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.371+0000 7faa7fc04640 1 -- 172.21.15.60:0/453379295 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faa5c066f60 msgr2=0x7faa5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.371+0000 7faa7fc04640 1 --2- 172.21.15.60:0/453379295 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faa5c066f60 0x7faa5c069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa78072410 tx=0x7faa68009040 comp rx=0 tx=0).stop 2024-04-25T10:05:57.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.371+0000 7faa7fc04640 1 -- 172.21.15.60:0/453379295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa780aec10 msgr2=0x7faa7814bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.371+0000 7faa7fc04640 1 --2- 172.21.15.60:0/453379295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa780aec10 0x7faa7814bea0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7faa70019130 tx=0x7faa7001ad60 comp rx=0 tx=0).stop 2024-04-25T10:05:57.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.371+0000 7faa7fc04640 1 -- 172.21.15.60:0/453379295 shutdown_connections 2024-04-25T10:05:57.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.371+0000 7faa7fc04640 1 --2- 172.21.15.60:0/453379295 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faa5c066f60 0x7faa5c069420 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.371+0000 7faa7fc04640 1 --2- 172.21.15.60:0/453379295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa7815db10 0x7faa7815f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.371+0000 7faa7fc04640 1 --2- 172.21.15.60:0/453379295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa780cfcf0 0x7faa7814c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.371+0000 7faa7fc04640 1 --2- 172.21.15.60:0/453379295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa780aec10 0x7faa7814bea0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.371+0000 7faa7fc04640 1 -- 172.21.15.60:0/453379295 >> 172.21.15.60:0/453379295 conn(0x7faa780670b0 msgr2=0x7faa780ad910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.371+0000 7faa7fc04640 1 -- 172.21.15.60:0/453379295 shutdown_connections 2024-04-25T10:05:57.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.371+0000 7faa7fc04640 1 -- 172.21.15.60:0/453379295 wait complete. 2024-04-25T10:05:57.394 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:57.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.515+0000 7f30d003f640 1 Processor -- start 2024-04-25T10:05:57.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.515+0000 7f30d003f640 1 -- start start 2024-04-25T10:05:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.515+0000 7f30d003f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f30c80adfa0 0x7f30c806e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.515+0000 7f30d003f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f30c815e0a0 0x7f30c80ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.515+0000 7f30d003f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c806ebf0 con 0x7f30c80ae380 2024-04-25T10:05:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.515+0000 7f30d003f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c806eed0 con 0x7f30c815e0a0 2024-04-25T10:05:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.515+0000 7f30d003f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c806f1b0 con 0x7f30c80adfa0 2024-04-25T10:05:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.515+0000 7f30ce5dc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f30c815e0a0 0x7f30c80ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.515+0000 7f30ce5dc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f30c815e0a0 0x7f30c80ae7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54546/0 (socket says 172.21.15.60:54546) 2024-04-25T10:05:57.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.515+0000 7f30ce5dc640 1 -- 172.21.15.60:0/3005112573 learned_addr learned my addr 172.21.15.60:0/3005112573 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:57.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.515+0000 7f30cdddb640 1 --2- 172.21.15.60:0/3005112573 >> v2:172.21.15.100:3300/0 conn(0x7f30c80adfa0 0x7f30c806e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.515+0000 7f30ce5dc640 1 -- 172.21.15.60:0/3005112573 >> v2:172.21.15.100:3300/0 conn(0x7f30c80adfa0 msgr2=0x7f30c806e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.515+0000 7f30ce5dc640 1 --2- 172.21.15.60:0/3005112573 >> v2:172.21.15.100:3300/0 conn(0x7f30c80adfa0 0x7f30c806e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.515+0000 7f30ce5dc640 1 -- 172.21.15.60:0/3005112573 >> v1:172.21.15.60:6789/0 conn(0x7f30c80ae380 legacy=0x7f30c815d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.515+0000 7f30ce5dc640 1 -- 172.21.15.60:0/3005112573 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c806f490 con 0x7f30c815e0a0 2024-04-25T10:05:57.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.519+0000 7f30ce5dc640 1 --2- 172.21.15.60:0/3005112573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c815e0a0 0x7f30c80ae7f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f30c80d0a00 tx=0x7f30c001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8ce43f2388e602 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.519+0000 7f30ccdd9640 1 -- 172.21.15.60:0/3005112573 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c00aa620 con 0x7f30c815e0a0 2024-04-25T10:05:57.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.519+0000 7f30ccdd9640 1 -- 172.21.15.60:0/3005112573 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c00af070 con 0x7f30c815e0a0 2024-04-25T10:05:57.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.519+0000 7f30ccdd9640 1 -- 172.21.15.60:0/3005112573 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c00aa9e0 con 0x7f30c815e0a0 2024-04-25T10:05:57.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.519+0000 7f30d003f640 1 -- 172.21.15.60:0/3005112573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c815e0a0 msgr2=0x7f30c80ae7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.519+0000 7f30d003f640 1 --2- 172.21.15.60:0/3005112573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c815e0a0 0x7f30c80ae7f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f30c80d0a00 tx=0x7f30c001c520 comp rx=0 tx=0).stop 2024-04-25T10:05:57.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.519+0000 7f30d003f640 1 -- 172.21.15.60:0/3005112573 shutdown_connections 2024-04-25T10:05:57.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.519+0000 7f30d003f640 1 --2- 172.21.15.60:0/3005112573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c815e0a0 0x7f30c80ae7f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.519+0000 7f30d003f640 1 --2- 172.21.15.60:0/3005112573 >> v2:172.21.15.100:3300/0 conn(0x7f30c80adfa0 0x7f30c806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.519+0000 7f30d003f640 1 -- 172.21.15.60:0/3005112573 >> 172.21.15.60:0/3005112573 conn(0x7f30c80cfdd0 msgr2=0x7f30c80d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.519+0000 7f30d003f640 1 -- 172.21.15.60:0/3005112573 shutdown_connections 2024-04-25T10:05:57.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.519+0000 7f30d003f640 1 -- 172.21.15.60:0/3005112573 wait complete. 2024-04-25T10:05:57.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.519+0000 7f30d003f640 1 Processor -- start 2024-04-25T10:05:57.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.519+0000 7f30d003f640 1 -- start start 2024-04-25T10:05:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30d003f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c80adfa0 0x7f30c813aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30d003f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c80ae380 0x7f30c81599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30d003f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30c815e0a0 0x7f30c8159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30d003f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8157c30 con 0x7f30c80ae380 2024-04-25T10:05:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30d003f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8157ab0 con 0x7f30c80adfa0 2024-04-25T10:05:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30d003f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8157db0 con 0x7f30c815e0a0 2024-04-25T10:05:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30cd5da640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c80ae380 0x7f30c81599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30cd5da640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c80ae380 0x7f30c81599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54562/0 (socket says 172.21.15.60:54562) 2024-04-25T10:05:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30cd5da640 1 -- 172.21.15.60:0/3478774217 learned_addr learned my addr 172.21.15.60:0/3478774217 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30cdddb640 1 --2- 172.21.15.60:0/3478774217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c80adfa0 0x7f30c813aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30ce5dc640 1 --2- 172.21.15.60:0/3478774217 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30c815e0a0 0x7f30c8159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30cd5da640 1 -- 172.21.15.60:0/3478774217 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30c815e0a0 msgr2=0x7f30c8159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30cd5da640 1 --2- 172.21.15.60:0/3478774217 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30c815e0a0 0x7f30c8159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30cd5da640 1 -- 172.21.15.60:0/3478774217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c80adfa0 msgr2=0x7f30c813aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30cd5da640 1 --2- 172.21.15.60:0/3478774217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c80adfa0 0x7f30c813aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30cd5da640 1 -- 172.21.15.60:0/3478774217 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c8167090 con 0x7f30c80ae380 2024-04-25T10:05:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30cd5da640 1 --2- 172.21.15.60:0/3478774217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c80ae380 0x7f30c81599d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00c420 tx=0x7f30bc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30c67fc640 1 -- 172.21.15.60:0/3478774217 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc0052d0 con 0x7f30c80ae380 2024-04-25T10:05:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30c67fc640 1 -- 172.21.15.60:0/3478774217 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30bc005470 con 0x7f30c80ae380 2024-04-25T10:05:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30c67fc640 1 -- 172.21.15.60:0/3478774217 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc007260 con 0x7f30c80ae380 2024-04-25T10:05:57.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30d003f640 1 -- 172.21.15.60:0/3478774217 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c8167370 con 0x7f30c80ae380 2024-04-25T10:05:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30d003f640 1 -- 172.21.15.60:0/3478774217 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c8167930 con 0x7f30c80ae380 2024-04-25T10:05:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30d003f640 1 -- 172.21.15.60:0/3478774217 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c8167b30 con 0x7f30c80ae380 2024-04-25T10:05:57.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30c67fc640 1 -- 172.21.15.60:0/3478774217 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f30bc007400 con 0x7f30c80ae380 2024-04-25T10:05:57.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30c67fc640 1 --2- 172.21.15.60:0/3478774217 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f30b4066f90 0x7f30b4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30c67fc640 1 -- 172.21.15.60:0/3478774217 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30bc084540 con 0x7f30c80ae380 2024-04-25T10:05:57.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.523+0000 7f30cdddb640 1 --2- 172.21.15.60:0/3478774217 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f30b4066f90 0x7f30b4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.527+0000 7f30cdddb640 1 --2- 172.21.15.60:0/3478774217 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f30b4066f90 0x7f30b4069450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f30b8003fd0 tx=0x7f30b80039a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.527+0000 7f30c67fc640 1 -- 172.21.15.60:0/3478774217 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f30c8167b30 con 0x7f30c80ae380 2024-04-25T10:05:57.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.683+0000 7f30d003f640 1 -- 172.21.15.60:0/3478774217 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30c8074770 con 0x7f30c80ae380 2024-04-25T10:05:57.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.683+0000 7f30c67fc640 1 -- 172.21.15.60:0/3478774217 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f30bc055550 con 0x7f30c80ae380 2024-04-25T10:05:57.690 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:05:57.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.683+0000 7f30d003f640 1 -- 172.21.15.60:0/3478774217 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f30b4066f90 msgr2=0x7f30b4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.683+0000 7f30d003f640 1 --2- 172.21.15.60:0/3478774217 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f30b4066f90 0x7f30b4069450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f30b8003fd0 tx=0x7f30b80039a0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.683+0000 7f30d003f640 1 -- 172.21.15.60:0/3478774217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c80ae380 msgr2=0x7f30c81599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.683+0000 7f30d003f640 1 --2- 172.21.15.60:0/3478774217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c80ae380 0x7f30c81599d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00c420 tx=0x7f30bc00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.683+0000 7f30d003f640 1 -- 172.21.15.60:0/3478774217 shutdown_connections 2024-04-25T10:05:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.683+0000 7f30d003f640 1 --2- 172.21.15.60:0/3478774217 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f30b4066f90 0x7f30b4069450 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.687+0000 7f30d003f640 1 --2- 172.21.15.60:0/3478774217 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30c815e0a0 0x7f30c8159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.687+0000 7f30d003f640 1 --2- 172.21.15.60:0/3478774217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c80ae380 0x7f30c81599d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.687+0000 7f30d003f640 1 --2- 172.21.15.60:0/3478774217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c80adfa0 0x7f30c813aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.687+0000 7f30d003f640 1 -- 172.21.15.60:0/3478774217 >> 172.21.15.60:0/3478774217 conn(0x7f30c80cfdd0 msgr2=0x7f30c80b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.687+0000 7f30d003f640 1 -- 172.21.15.60:0/3478774217 shutdown_connections 2024-04-25T10:05:57.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.687+0000 7f30d003f640 1 -- 172.21.15.60:0/3478774217 wait complete. 2024-04-25T10:05:57.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:57.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbd0203f640 1 Processor -- start 2024-04-25T10:05:57.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbd0203f640 1 -- start start 2024-04-25T10:05:57.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbd0203f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbcfc15ef20 0x7fbcfc0b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbd0203f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcfc0cfcf0 0x7fbcfc0b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbd0203f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbcfc099a30 0x7fbcfc099ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbd0203f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcfc0cc320 con 0x7fbcfc099a30 2024-04-25T10:05:57.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbd0203f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcfc0cc5c0 con 0x7fbcfc15ef20 2024-04-25T10:05:57.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbd0203f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc0d07b0 con 0x7fbcfc0cfcf0 2024-04-25T10:05:57.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbcfbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbcfc099a30 0x7fbcfc099ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbcfbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbcfc099a30 0x7fbcfc099ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54576/0 (socket says 172.21.15.60:54576) 2024-04-25T10:05:57.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbcfbfff640 1 -- 172.21.15.60:0/2770823613 learned_addr learned my addr 172.21.15.60:0/2770823613 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:57.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbcfaffd640 1 --2- 172.21.15.60:0/2770823613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcfc0cfcf0 0x7fbcfc0b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbcfb7fe640 1 --2- 172.21.15.60:0/2770823613 >> v2:172.21.15.100:3300/0 conn(0x7fbcfc15ef20 0x7fbcfc0b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbcfbfff640 1 -- 172.21.15.60:0/2770823613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcfc0cfcf0 msgr2=0x7fbcfc0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbcfbfff640 1 --2- 172.21.15.60:0/2770823613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcfc0cfcf0 0x7fbcfc0b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbcfbfff640 1 -- 172.21.15.60:0/2770823613 >> v2:172.21.15.100:3300/0 conn(0x7fbcfc15ef20 msgr2=0x7fbcfc0b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:57.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbcfbfff640 1 --2- 172.21.15.60:0/2770823613 >> v2:172.21.15.100:3300/0 conn(0x7fbcfc15ef20 0x7fbcfc0b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.819+0000 7fbcfbfff640 1 -- 172.21.15.60:0/2770823613 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc0d04d0 con 0x7fbcfc099a30 2024-04-25T10:05:57.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbcfbfff640 1 --2- 172.21.15.60:0/2770823613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcfc099a30 0x7fbcfc099ec0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0b3da0 tx=0x7fbcf401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a237400129bd40d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbcfa7fc640 1 -- 172.21.15.60:0/2770823613 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40a5670 con 0x7fbcfc099a30 2024-04-25T10:05:57.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbcfa7fc640 1 -- 172.21.15.60:0/2770823613 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf40a5810 con 0x7fbcfc099a30 2024-04-25T10:05:57.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbcfa7fc640 1 -- 172.21.15.60:0/2770823613 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40a5be0 con 0x7fbcfc099a30 2024-04-25T10:05:57.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 -- 172.21.15.60:0/2770823613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcfc099a30 msgr2=0x7fbcfc099ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 --2- 172.21.15.60:0/2770823613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcfc099a30 0x7fbcfc099ec0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0b3da0 tx=0x7fbcf401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 -- 172.21.15.60:0/2770823613 shutdown_connections 2024-04-25T10:05:57.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 --2- 172.21.15.60:0/2770823613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcfc099a30 0x7fbcfc099ec0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 --2- 172.21.15.60:0/2770823613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcfc0cfcf0 0x7fbcfc0b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 --2- 172.21.15.60:0/2770823613 >> v2:172.21.15.100:3300/0 conn(0x7fbcfc15ef20 0x7fbcfc0b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 -- 172.21.15.60:0/2770823613 >> 172.21.15.60:0/2770823613 conn(0x7fbcfc0b3200 msgr2=0x7fbcfc0b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:57.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 -- 172.21.15.60:0/2770823613 shutdown_connections 2024-04-25T10:05:57.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 -- 172.21.15.60:0/2770823613 wait complete. 2024-04-25T10:05:57.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 Processor -- start 2024-04-25T10:05:57.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 -- start start 2024-04-25T10:05:57.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcfc099a30 0x7fbcfc0afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcfc0cfcf0 0x7fbcfc0b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcfc15ef20 0x7fbcfc0b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc0d0250 con 0x7fbcfc0cfcf0 2024-04-25T10:05:57.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc0d00d0 con 0x7fbcfc099a30 2024-04-25T10:05:57.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.823+0000 7fbd0203f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc15f300 con 0x7fbcfc15ef20 2024-04-25T10:05:57.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbcfaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcfc0cfcf0 0x7fbcfc0b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbcfaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcfc0cfcf0 0x7fbcfc0b18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54584/0 (socket says 172.21.15.60:54584) 2024-04-25T10:05:57.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbcfaffd640 1 -- 172.21.15.60:0/279472861 learned_addr learned my addr 172.21.15.60:0/279472861 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:57.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbcfb7fe640 1 --2- 172.21.15.60:0/279472861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcfc099a30 0x7fbcfc0afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbcfbfff640 1 --2- 172.21.15.60:0/279472861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcfc15ef20 0x7fbcfc0b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbcfaffd640 1 -- 172.21.15.60:0/279472861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcfc15ef20 msgr2=0x7fbcfc0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbcfaffd640 1 --2- 172.21.15.60:0/279472861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcfc15ef20 0x7fbcfc0b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbcfaffd640 1 -- 172.21.15.60:0/279472861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcfc099a30 msgr2=0x7fbcfc0afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbcfaffd640 1 --2- 172.21.15.60:0/279472861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcfc099a30 0x7fbcfc0afeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:57.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbcfaffd640 1 -- 172.21.15.60:0/279472861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc13e320 con 0x7fbcfc0cfcf0 2024-04-25T10:05:57.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbcfb7fe640 1 --2- 172.21.15.60:0/279472861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcfc099a30 0x7fbcfc0afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:57.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbcfaffd640 1 --2- 172.21.15.60:0/279472861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcfc0cfcf0 0x7fbcfc0b18f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000d800 tx=0x7fbcf000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbce3fff640 1 -- 172.21.15.60:0/279472861 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0018020 con 0x7fbcfc0cfcf0 2024-04-25T10:05:57.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbce3fff640 1 -- 172.21.15.60:0/279472861 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf00096a0 con 0x7fbcfc0cfcf0 2024-04-25T10:05:57.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbce3fff640 1 -- 172.21.15.60:0/279472861 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0009a50 con 0x7fbcfc0cfcf0 2024-04-25T10:05:57.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbd0203f640 1 -- 172.21.15.60:0/279472861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcfc13e600 con 0x7fbcfc0cfcf0 2024-04-25T10:05:57.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbd0203f640 1 -- 172.21.15.60:0/279472861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcfc13ebc0 con 0x7fbcfc0cfcf0 2024-04-25T10:05:57.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbd0203f640 1 -- 172.21.15.60:0/279472861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcfc074770 con 0x7fbcfc0cfcf0 2024-04-25T10:05:57.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbce3fff640 1 -- 172.21.15.60:0/279472861 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbcf001b020 con 0x7fbcfc0cfcf0 2024-04-25T10:05:57.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbce3fff640 1 --2- 172.21.15.60:0/279472861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbce8066f90 0x7fbce8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:57.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbce3fff640 1 -- 172.21.15.60:0/279472861 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbcf0085510 con 0x7fbcfc0cfcf0 2024-04-25T10:05:57.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbcfb7fe640 1 --2- 172.21.15.60:0/279472861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbce8066f90 0x7fbce8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:57.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.827+0000 7fbcfb7fe640 1 --2- 172.21.15.60:0/279472861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbce8066f90 0x7fbce8069450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc074350 tx=0x7fbcec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:57.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.831+0000 7fbce3fff640 1 -- 172.21.15.60:0/279472861 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbcf0022020 con 0x7fbcfc0cfcf0 2024-04-25T10:05:57.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.991+0000 7fbd0203f640 1 -- 172.21.15.60:0/279472861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbcfc0b4070 con 0x7fbcfc0cfcf0 2024-04-25T10:05:57.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.991+0000 7fbce3fff640 1 -- 172.21.15.60:0/279472861 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbcf00565b0 con 0x7fbcfc0cfcf0 2024-04-25T10:05:57.997 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:05:57.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.991+0000 7fbd0203f640 1 -- 172.21.15.60:0/279472861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbce8066f90 msgr2=0x7fbce8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:57.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.991+0000 7fbd0203f640 1 --2- 172.21.15.60:0/279472861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbce8066f90 0x7fbce8069450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc074350 tx=0x7fbcec008040 comp rx=0 tx=0).stop 2024-04-25T10:05:58.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.991+0000 7fbd0203f640 1 -- 172.21.15.60:0/279472861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcfc0cfcf0 msgr2=0x7fbcfc0b18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.991+0000 7fbd0203f640 1 --2- 172.21.15.60:0/279472861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcfc0cfcf0 0x7fbcfc0b18f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000d800 tx=0x7fbcf000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.991+0000 7fbd0203f640 1 -- 172.21.15.60:0/279472861 shutdown_connections 2024-04-25T10:05:58.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.991+0000 7fbd0203f640 1 --2- 172.21.15.60:0/279472861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbce8066f90 0x7fbce8069450 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.991+0000 7fbd0203f640 1 --2- 172.21.15.60:0/279472861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbcfc15ef20 0x7fbcfc0b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.991+0000 7fbd0203f640 1 --2- 172.21.15.60:0/279472861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbcfc0cfcf0 0x7fbcfc0b18f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.991+0000 7fbd0203f640 1 --2- 172.21.15.60:0/279472861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcfc099a30 0x7fbcfc0afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.991+0000 7fbd0203f640 1 -- 172.21.15.60:0/279472861 >> 172.21.15.60:0/279472861 conn(0x7fbcfc0b3200 msgr2=0x7fbcfc0cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.991+0000 7fbd0203f640 1 -- 172.21.15.60:0/279472861 shutdown_connections 2024-04-25T10:05:58.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:57.991+0000 7fbd0203f640 1 -- 172.21.15.60:0/279472861 wait complete. 2024-04-25T10:05:58.017 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:05:58.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.135+0000 7fc15dc93640 1 Processor -- start 2024-04-25T10:05:58.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.135+0000 7fc15dc93640 1 -- start start 2024-04-25T10:05:58.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.135+0000 7fc15dc93640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1580cd390 0x7fc15809a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.135+0000 7fc15dc93640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc15809a930 0x7fc1580cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.135+0000 7fc15dc93640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1580cc5d0 con 0x7fc15815f480 2024-04-25T10:05:58.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.135+0000 7fc15dc93640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1580cc8b0 con 0x7fc15809a930 2024-04-25T10:05:58.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.135+0000 7fc15dc93640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1580ccb90 con 0x7fc1580cd390 2024-04-25T10:05:58.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.135+0000 7fc157fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc15809a930 0x7fc1580cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.135+0000 7fc157fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc15809a930 0x7fc1580cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54600/0 (socket says 172.21.15.60:54600) 2024-04-25T10:05:58.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.135+0000 7fc157fff640 1 -- 172.21.15.60:0/521310343 learned_addr learned my addr 172.21.15.60:0/521310343 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:58.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.135+0000 7fc156ffd640 1 --2- 172.21.15.60:0/521310343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1580cd390 0x7fc15809a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.135+0000 7fc157fff640 1 -- 172.21.15.60:0/521310343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1580cd390 msgr2=0x7fc15809a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.135+0000 7fc157fff640 1 --2- 172.21.15.60:0/521310343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1580cd390 0x7fc15809a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.135+0000 7fc157fff640 1 -- 172.21.15.60:0/521310343 >> v1:172.21.15.100:6789/0 conn(0x7fc15815f480 legacy=0x7fc15815f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.135+0000 7fc157fff640 1 -- 172.21.15.60:0/521310343 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1580cce70 con 0x7fc15809a930 2024-04-25T10:05:58.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.139+0000 7fc157fff640 1 --2- 172.21.15.60:0/521310343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc15809a930 0x7fc1580cf380 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc150009e50 tx=0x7fc15001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fae5309bce6ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.139+0000 7fc1567fc640 1 -- 172.21.15.60:0/521310343 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1500a58b0 con 0x7fc15809a930 2024-04-25T10:05:58.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.139+0000 7fc1567fc640 1 -- 172.21.15.60:0/521310343 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1500a5a50 con 0x7fc15809a930 2024-04-25T10:05:58.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.139+0000 7fc1567fc640 1 -- 172.21.15.60:0/521310343 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1500a5e20 con 0x7fc15809a930 2024-04-25T10:05:58.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.139+0000 7fc15dc93640 1 -- 172.21.15.60:0/521310343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc15809a930 msgr2=0x7fc1580cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.139+0000 7fc15dc93640 1 --2- 172.21.15.60:0/521310343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc15809a930 0x7fc1580cf380 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc150009e50 tx=0x7fc15001c9d0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.139+0000 7fc15dc93640 1 -- 172.21.15.60:0/521310343 shutdown_connections 2024-04-25T10:05:58.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.139+0000 7fc15dc93640 1 --2- 172.21.15.60:0/521310343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc15809a930 0x7fc1580cf380 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.139+0000 7fc15dc93640 1 --2- 172.21.15.60:0/521310343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1580cd390 0x7fc15809a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.139+0000 7fc15dc93640 1 -- 172.21.15.60:0/521310343 >> 172.21.15.60:0/521310343 conn(0x7fc1580b32a0 msgr2=0x7fc1580b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.139+0000 7fc15dc93640 1 -- 172.21.15.60:0/521310343 shutdown_connections 2024-04-25T10:05:58.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.139+0000 7fc15dc93640 1 -- 172.21.15.60:0/521310343 wait complete. 2024-04-25T10:05:58.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.139+0000 7fc15dc93640 1 Processor -- start 2024-04-25T10:05:58.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.139+0000 7fc15dc93640 1 -- start start 2024-04-25T10:05:58.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc15dc93640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc15809a930 0x7fc158154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc15dc93640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1580cd390 0x7fc15806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc15dc93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc15815f480 0x7fc15815e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc15dc93640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1581687c0 con 0x7fc1580cd390 2024-04-25T10:05:58.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc15dc93640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15815cdd0 con 0x7fc15815f480 2024-04-25T10:05:58.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc15dc93640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc158168940 con 0x7fc15809a930 2024-04-25T10:05:58.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc156ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1580cd390 0x7fc15806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc156ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1580cd390 0x7fc15806c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54602/0 (socket says 172.21.15.60:54602) 2024-04-25T10:05:58.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc156ffd640 1 -- 172.21.15.60:0/2378646902 learned_addr learned my addr 172.21.15.60:0/2378646902 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:58.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc157fff640 1 --2- 172.21.15.60:0/2378646902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc15815f480 0x7fc15815e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc1577fe640 1 --2- 172.21.15.60:0/2378646902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc15809a930 0x7fc158154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc156ffd640 1 -- 172.21.15.60:0/2378646902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc15809a930 msgr2=0x7fc158154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc156ffd640 1 --2- 172.21.15.60:0/2378646902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc15809a930 0x7fc158154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc156ffd640 1 -- 172.21.15.60:0/2378646902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc15815f480 msgr2=0x7fc15815e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc156ffd640 1 --2- 172.21.15.60:0/2378646902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc15815f480 0x7fc15815e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc156ffd640 1 -- 172.21.15.60:0/2378646902 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1581550d0 con 0x7fc1580cd390 2024-04-25T10:05:58.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc156ffd640 1 --2- 172.21.15.60:0/2378646902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1580cd390 0x7fc15806c510 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00e930 tx=0x7fc14c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc13ffff640 1 -- 172.21.15.60:0/2378646902 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c017070 con 0x7fc1580cd390 2024-04-25T10:05:58.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc13ffff640 1 -- 172.21.15.60:0/2378646902 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14c00ca00 con 0x7fc1580cd390 2024-04-25T10:05:58.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc15dc93640 1 -- 172.21.15.60:0/2378646902 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15806cbd0 con 0x7fc1580cd390 2024-04-25T10:05:58.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc13ffff640 1 -- 172.21.15.60:0/2378646902 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c00cdb0 con 0x7fc1580cd390 2024-04-25T10:05:58.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc15dc93640 1 -- 172.21.15.60:0/2378646902 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15815dce0 con 0x7fc1580cd390 2024-04-25T10:05:58.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc15dc93640 1 -- 172.21.15.60:0/2378646902 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc158074770 con 0x7fc1580cd390 2024-04-25T10:05:58.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc13ffff640 1 -- 172.21.15.60:0/2378646902 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc14c01a070 con 0x7fc1580cd390 2024-04-25T10:05:58.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc13ffff640 1 --2- 172.21.15.60:0/2378646902 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc144066f90 0x7fc144069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc13ffff640 1 -- 172.21.15.60:0/2378646902 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc14c085070 con 0x7fc1580cd390 2024-04-25T10:05:58.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.143+0000 7fc1577fe640 1 --2- 172.21.15.60:0/2378646902 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc144066f90 0x7fc144069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.147+0000 7fc1577fe640 1 --2- 172.21.15.60:0/2378646902 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc144066f90 0x7fc144069450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc1480015b0 tx=0x7fc148009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.147+0000 7fc13ffff640 1 -- 172.21.15.60:0/2378646902 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc1581550d0 con 0x7fc1580cd390 2024-04-25T10:05:58.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.319+0000 7fc15dc93640 1 -- 172.21.15.60:0/2378646902 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc1580b7730 con 0x7fc1580cd390 2024-04-25T10:05:58.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.323+0000 7fc13ffff640 1 -- 172.21.15.60:0/2378646902 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc14c056140 con 0x7fc1580cd390 2024-04-25T10:05:58.328 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:05:58.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.323+0000 7fc15dc93640 1 -- 172.21.15.60:0/2378646902 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc144066f90 msgr2=0x7fc144069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.323+0000 7fc15dc93640 1 --2- 172.21.15.60:0/2378646902 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc144066f90 0x7fc144069450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc1480015b0 tx=0x7fc148009040 comp rx=0 tx=0).stop 2024-04-25T10:05:58.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.323+0000 7fc15dc93640 1 -- 172.21.15.60:0/2378646902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1580cd390 msgr2=0x7fc15806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.323+0000 7fc15dc93640 1 --2- 172.21.15.60:0/2378646902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1580cd390 0x7fc15806c510 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00e930 tx=0x7fc14c00edf0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.323+0000 7fc15dc93640 1 -- 172.21.15.60:0/2378646902 shutdown_connections 2024-04-25T10:05:58.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.323+0000 7fc15dc93640 1 --2- 172.21.15.60:0/2378646902 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc144066f90 0x7fc144069450 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.323+0000 7fc15dc93640 1 --2- 172.21.15.60:0/2378646902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc15815f480 0x7fc15815e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.323+0000 7fc15dc93640 1 --2- 172.21.15.60:0/2378646902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1580cd390 0x7fc15806c510 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.323+0000 7fc15dc93640 1 --2- 172.21.15.60:0/2378646902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc15809a930 0x7fc158154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.323+0000 7fc15dc93640 1 -- 172.21.15.60:0/2378646902 >> 172.21.15.60:0/2378646902 conn(0x7fc1580b32a0 msgr2=0x7fc158158080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.323+0000 7fc15dc93640 1 -- 172.21.15.60:0/2378646902 shutdown_connections 2024-04-25T10:05:58.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.323+0000 7fc15dc93640 1 -- 172.21.15.60:0/2378646902 wait complete. 2024-04-25T10:05:58.347 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:05:58.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:58.347 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:58.372 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:58.373 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:05:58.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:58.373 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:05:58.426 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:05:58.426 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:58.432 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:05:58.433 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:58.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:58.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.599+0000 7fe10cf3b640 1 Processor -- start 2024-04-25T10:05:58.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.599+0000 7fe10cf3b640 1 -- start start 2024-04-25T10:05:58.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe10cf3b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1080ce7a0 0x7fe1080ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe10cf3b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe10815bc90 0x7fe1080b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe10cf3b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe1080b4660 0x7fe1080b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe10cf3b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1080cc370 con 0x7fe1080b4660 2024-04-25T10:05:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe10cf3b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1080cf0c0 con 0x7fe10815bc90 2024-04-25T10:05:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe10cf3b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe108071ca0 con 0x7fe1080ce7a0 2024-04-25T10:05:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe106d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe1080b4660 0x7fe1080b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe106d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe1080b4660 0x7fe1080b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54604/0 (socket says 172.21.15.60:54604) 2024-04-25T10:05:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe106d9d640 1 -- 172.21.15.60:0/1058044750 learned_addr learned my addr 172.21.15.60:0/1058044750 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe10659c640 1 --2- 172.21.15.60:0/1058044750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1080ce7a0 0x7fe1080ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe105d9b640 1 --2- 172.21.15.60:0/1058044750 >> v2:172.21.15.100:3300/0 conn(0x7fe10815bc90 0x7fe1080b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe106d9d640 1 -- 172.21.15.60:0/1058044750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1080ce7a0 msgr2=0x7fe1080ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe106d9d640 1 --2- 172.21.15.60:0/1058044750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1080ce7a0 0x7fe1080ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe106d9d640 1 -- 172.21.15.60:0/1058044750 >> v2:172.21.15.100:3300/0 conn(0x7fe10815bc90 msgr2=0x7fe1080b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe106d9d640 1 --2- 172.21.15.60:0/1058044750 >> v2:172.21.15.100:3300/0 conn(0x7fe10815bc90 0x7fe1080b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe106d9d640 1 -- 172.21.15.60:0/1058044750 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1080cca40 con 0x7fe1080b4660 2024-04-25T10:05:58.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe10659c640 1 --2- 172.21.15.60:0/1058044750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1080ce7a0 0x7fe1080ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:58.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe105d9b640 1 --2- 172.21.15.60:0/1058044750 >> v2:172.21.15.100:3300/0 conn(0x7fe10815bc90 0x7fe1080b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:58.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe106d9d640 1 --2- 172.21.15.60:0/1058044750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1080b4660 0x7fe1080b2b10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe100009e50 tx=0x7fe10001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa9d39f4e6395cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe10559a640 1 -- 172.21.15.60:0/1058044750 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1000a5670 con 0x7fe1080b4660 2024-04-25T10:05:58.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe10559a640 1 -- 172.21.15.60:0/1058044750 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1000a5810 con 0x7fe1080b4660 2024-04-25T10:05:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.603+0000 7fe10559a640 1 -- 172.21.15.60:0/1058044750 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1000a5be0 con 0x7fe1080b4660 2024-04-25T10:05:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 -- 172.21.15.60:0/1058044750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1080b4660 msgr2=0x7fe1080b2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 --2- 172.21.15.60:0/1058044750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1080b4660 0x7fe1080b2b10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe100009e50 tx=0x7fe10001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 -- 172.21.15.60:0/1058044750 shutdown_connections 2024-04-25T10:05:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 --2- 172.21.15.60:0/1058044750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1080b4660 0x7fe1080b2b10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 --2- 172.21.15.60:0/1058044750 >> v2:172.21.15.100:3300/0 conn(0x7fe10815bc90 0x7fe1080b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 --2- 172.21.15.60:0/1058044750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1080ce7a0 0x7fe1080ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 -- 172.21.15.60:0/1058044750 >> 172.21.15.60:0/1058044750 conn(0x7fe1080b12d0 msgr2=0x7fe1080b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 -- 172.21.15.60:0/1058044750 shutdown_connections 2024-04-25T10:05:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 -- 172.21.15.60:0/1058044750 wait complete. 2024-04-25T10:05:58.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 Processor -- start 2024-04-25T10:05:58.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 -- start start 2024-04-25T10:05:58.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1080ce7a0 0x7fe108145f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe10815bc90 0x7fe108159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe108156950 0x7fe10815dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1080b3db0 con 0x7fe1080ce7a0 2024-04-25T10:05:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1080b3460 con 0x7fe10815bc90 2024-04-25T10:05:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10cf3b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1080b4280 con 0x7fe108156950 2024-04-25T10:05:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1080ce7a0 0x7fe108145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1080ce7a0 0x7fe108145f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54616/0 (socket says 172.21.15.60:54616) 2024-04-25T10:05:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10659c640 1 -- 172.21.15.60:0/1829898831 learned_addr learned my addr 172.21.15.60:0/1829898831 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe105d9b640 1 --2- 172.21.15.60:0/1829898831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe10815bc90 0x7fe108159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe106d9d640 1 --2- 172.21.15.60:0/1829898831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe108156950 0x7fe10815dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10659c640 1 -- 172.21.15.60:0/1829898831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe108156950 msgr2=0x7fe10815dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10659c640 1 --2- 172.21.15.60:0/1829898831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe108156950 0x7fe10815dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10659c640 1 -- 172.21.15.60:0/1829898831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe10815bc90 msgr2=0x7fe108159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10659c640 1 --2- 172.21.15.60:0/1829898831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe10815bc90 0x7fe108159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe10659c640 1 -- 172.21.15.60:0/1829898831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe108152d50 con 0x7fe1080ce7a0 2024-04-25T10:05:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.607+0000 7fe105d9b640 1 --2- 172.21.15.60:0/1829898831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe10815bc90 0x7fe108159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.611+0000 7fe10659c640 1 --2- 172.21.15.60:0/1829898831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1080ce7a0 0x7fe108145f10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800e810 tx=0x7fe0f800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.611+0000 7fe0f6ffd640 1 -- 172.21.15.60:0/1829898831 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f800f040 con 0x7fe1080ce7a0 2024-04-25T10:05:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.611+0000 7fe0f6ffd640 1 -- 172.21.15.60:0/1829898831 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f8004030 con 0x7fe1080ce7a0 2024-04-25T10:05:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.611+0000 7fe10cf3b640 1 -- 172.21.15.60:0/1829898831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe108156e20 con 0x7fe1080ce7a0 2024-04-25T10:05:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.611+0000 7fe0f6ffd640 1 -- 172.21.15.60:0/1829898831 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8004400 con 0x7fe1080ce7a0 2024-04-25T10:05:58.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.611+0000 7fe10cf3b640 1 -- 172.21.15.60:0/1829898831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1081564e0 con 0x7fe1080ce7a0 2024-04-25T10:05:58.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.611+0000 7fe10cf3b640 1 -- 172.21.15.60:0/1829898831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe108155ff0 con 0x7fe1080ce7a0 2024-04-25T10:05:58.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.611+0000 7fe0f6ffd640 1 -- 172.21.15.60:0/1829898831 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0f800c6d0 con 0x7fe1080ce7a0 2024-04-25T10:05:58.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.611+0000 7fe0f6ffd640 1 --2- 172.21.15.60:0/1829898831 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe0e8066f60 0x7fe0e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.611+0000 7fe0f6ffd640 1 -- 172.21.15.60:0/1829898831 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0f8084c30 con 0x7fe1080ce7a0 2024-04-25T10:05:58.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.611+0000 7fe105d9b640 1 --2- 172.21.15.60:0/1829898831 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe0e8066f60 0x7fe0e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.611+0000 7fe105d9b640 1 --2- 172.21.15.60:0/1829898831 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe0e8066f60 0x7fe0e8069420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe1080b03a0 tx=0x7fe0fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.615+0000 7fe0f6ffd640 1 -- 172.21.15.60:0/1829898831 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe0f808b080 con 0x7fe1080ce7a0 2024-04-25T10:05:58.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.775+0000 7fe10cf3b640 1 -- 172.21.15.60:0/1829898831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe108074770 con 0x7fe1080ce7a0 2024-04-25T10:05:58.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.775+0000 7fe0f6ffd640 1 -- 172.21.15.60:0/1829898831 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16617 (secure 0 0 0) 0x7fe108152d50 con 0x7fe1080ce7a0 2024-04-25T10:05:58.784 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:05:58.784 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:22.036582+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:58.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.779+0000 7fe10cf3b640 1 -- 172.21.15.60:0/1829898831 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe0e8066f60 msgr2=0x7fe0e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.779+0000 7fe10cf3b640 1 --2- 172.21.15.60:0/1829898831 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe0e8066f60 0x7fe0e8069420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe1080b03a0 tx=0x7fe0fc008040 comp rx=0 tx=0).stop 2024-04-25T10:05:58.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.779+0000 7fe10cf3b640 1 -- 172.21.15.60:0/1829898831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1080ce7a0 msgr2=0x7fe108145f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.779+0000 7fe10cf3b640 1 --2- 172.21.15.60:0/1829898831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1080ce7a0 0x7fe108145f10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800e810 tx=0x7fe0f800ecd0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.779+0000 7fe10cf3b640 1 -- 172.21.15.60:0/1829898831 shutdown_connections 2024-04-25T10:05:58.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.779+0000 7fe10cf3b640 1 --2- 172.21.15.60:0/1829898831 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe0e8066f60 0x7fe0e8069420 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.779+0000 7fe10cf3b640 1 --2- 172.21.15.60:0/1829898831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe108156950 0x7fe10815dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.779+0000 7fe10cf3b640 1 --2- 172.21.15.60:0/1829898831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe10815bc90 0x7fe108159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.779+0000 7fe10cf3b640 1 --2- 172.21.15.60:0/1829898831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1080ce7a0 0x7fe108145f10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.779+0000 7fe10cf3b640 1 -- 172.21.15.60:0/1829898831 >> 172.21.15.60:0/1829898831 conn(0x7fe1080b12d0 msgr2=0x7fe1080b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.779+0000 7fe10cf3b640 1 -- 172.21.15.60:0/1829898831 shutdown_connections 2024-04-25T10:05:58.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.779+0000 7fe10cf3b640 1 -- 172.21.15.60:0/1829898831 wait complete. 2024-04-25T10:05:58.802 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:18.976643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:19.405302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:58.802 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:58.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.919+0000 7fd3420ad640 1 Processor -- start 2024-04-25T10:05:58.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.919+0000 7fd3420ad640 1 -- start start 2024-04-25T10:05:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd3420ad640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd33c0cbcf0 0x7fd33c160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd3420ad640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd33c071c30 0x7fd33c15f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd3420ad640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd33c15fa20 0x7fd33c15bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd3420ad640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd33c0b2dc0 con 0x7fd33c15fa20 2024-04-25T10:05:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd3420ad640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd33c15fe80 con 0x7fd33c0cbcf0 2024-04-25T10:05:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd3420ad640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c144a40 con 0x7fd33c071c30 2024-04-25T10:05:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd33bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd33c15fa20 0x7fd33c15bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd33bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd33c15fa20 0x7fd33c15bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45162/0 (socket says 172.21.15.60:45162) 2024-04-25T10:05:58.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd33bfff640 1 -- 172.21.15.60:0/2735290843 learned_addr learned my addr 172.21.15.60:0/2735290843 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:58.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd33b7fe640 1 --2- 172.21.15.60:0/2735290843 >> v2:172.21.15.100:3300/0 conn(0x7fd33c0cbcf0 0x7fd33c160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd33affd640 1 --2- 172.21.15.60:0/2735290843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd33c071c30 0x7fd33c15f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd33bfff640 1 -- 172.21.15.60:0/2735290843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd33c071c30 msgr2=0x7fd33c15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd33bfff640 1 --2- 172.21.15.60:0/2735290843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd33c071c30 0x7fd33c15f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd33bfff640 1 -- 172.21.15.60:0/2735290843 >> v2:172.21.15.100:3300/0 conn(0x7fd33c0cbcf0 msgr2=0x7fd33c160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd33bfff640 1 --2- 172.21.15.60:0/2735290843 >> v2:172.21.15.100:3300/0 conn(0x7fd33c0cbcf0 0x7fd33c160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd33bfff640 1 -- 172.21.15.60:0/2735290843 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c0cc9f0 con 0x7fd33c15fa20 2024-04-25T10:05:58.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd33bfff640 1 --2- 172.21.15.60:0/2735290843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd33c15fa20 0x7fd33c15bf10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0b4c50 tx=0x7fd33401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378677d9dc33ef45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.923+0000 7fd33a7fc640 1 -- 172.21.15.60:0/2735290843 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3340aa610 con 0x7fd33c15fa20 2024-04-25T10:05:58.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd33a7fc640 1 -- 172.21.15.60:0/2735290843 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3340af070 con 0x7fd33c15fa20 2024-04-25T10:05:58.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd33a7fc640 1 -- 172.21.15.60:0/2735290843 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3340aa9d0 con 0x7fd33c15fa20 2024-04-25T10:05:58.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 -- 172.21.15.60:0/2735290843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd33c15fa20 msgr2=0x7fd33c15bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 --2- 172.21.15.60:0/2735290843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd33c15fa20 0x7fd33c15bf10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0b4c50 tx=0x7fd33401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 -- 172.21.15.60:0/2735290843 shutdown_connections 2024-04-25T10:05:58.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 --2- 172.21.15.60:0/2735290843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd33c15fa20 0x7fd33c15bf10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 --2- 172.21.15.60:0/2735290843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd33c071c30 0x7fd33c15f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 --2- 172.21.15.60:0/2735290843 >> v2:172.21.15.100:3300/0 conn(0x7fd33c0cbcf0 0x7fd33c160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 -- 172.21.15.60:0/2735290843 >> 172.21.15.60:0/2735290843 conn(0x7fd33c0670b0 msgr2=0x7fd33c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 -- 172.21.15.60:0/2735290843 shutdown_connections 2024-04-25T10:05:58.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 -- 172.21.15.60:0/2735290843 wait complete. 2024-04-25T10:05:58.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 Processor -- start 2024-04-25T10:05:58.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 -- start start 2024-04-25T10:05:58.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd33c071c30 0x7fd33c13cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33c0cbcf0 0x7fd33c143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd33c15fa20 0x7fd33c143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c0ce740 con 0x7fd33c071c30 2024-04-25T10:05:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c0ce5c0 con 0x7fd33c0cbcf0 2024-04-25T10:05:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c072010 con 0x7fd33c15fa20 2024-04-25T10:05:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd33b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd33c071c30 0x7fd33c13cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd33b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd33c071c30 0x7fd33c13cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45168/0 (socket says 172.21.15.60:45168) 2024-04-25T10:05:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd33b7fe640 1 -- 172.21.15.60:0/977798869 learned_addr learned my addr 172.21.15.60:0/977798869 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd33affd640 1 --2- 172.21.15.60:0/977798869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33c0cbcf0 0x7fd33c143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd33bfff640 1 --2- 172.21.15.60:0/977798869 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd33c15fa20 0x7fd33c143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd33b7fe640 1 -- 172.21.15.60:0/977798869 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd33c15fa20 msgr2=0x7fd33c143fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd33b7fe640 1 --2- 172.21.15.60:0/977798869 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd33c15fa20 0x7fd33c143fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd33b7fe640 1 -- 172.21.15.60:0/977798869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33c0cbcf0 msgr2=0x7fd33c143a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd33b7fe640 1 --2- 172.21.15.60:0/977798869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33c0cbcf0 0x7fd33c143a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd33b7fe640 1 -- 172.21.15.60:0/977798869 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c160840 con 0x7fd33c071c30 2024-04-25T10:05:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd33b7fe640 1 --2- 172.21.15.60:0/977798869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd33c071c30 0x7fd33c13cf50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd32400d800 tx=0x7fd32400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd333fff640 1 -- 172.21.15.60:0/977798869 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32400b6c0 con 0x7fd33c071c30 2024-04-25T10:05:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd333fff640 1 -- 172.21.15.60:0/977798869 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd324019070 con 0x7fd33c071c30 2024-04-25T10:05:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 -- 172.21.15.60:0/977798869 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33c160b20 con 0x7fd33c071c30 2024-04-25T10:05:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd333fff640 1 -- 172.21.15.60:0/977798869 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32400ba60 con 0x7fd33c071c30 2024-04-25T10:05:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.927+0000 7fd3420ad640 1 -- 172.21.15.60:0/977798869 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33c1610e0 con 0x7fd33c071c30 2024-04-25T10:05:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.931+0000 7fd3420ad640 1 -- 172.21.15.60:0/977798869 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd33c06bba0 con 0x7fd33c071c30 2024-04-25T10:05:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.931+0000 7fd333fff640 1 -- 172.21.15.60:0/977798869 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd32400bc00 con 0x7fd33c071c30 2024-04-25T10:05:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.931+0000 7fd333fff640 1 --2- 172.21.15.60:0/977798869 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd31c066f90 0x7fd31c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.931+0000 7fd333fff640 1 -- 172.21.15.60:0/977798869 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd324085490 con 0x7fd33c071c30 2024-04-25T10:05:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.931+0000 7fd33affd640 1 --2- 172.21.15.60:0/977798869 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd31c066f90 0x7fd31c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.931+0000 7fd33affd640 1 --2- 172.21.15.60:0/977798869 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd31c066f90 0x7fd31c069450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd32c003fd0 tx=0x7fd32c003a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:58.935+0000 7fd333fff640 1 -- 172.21.15.60:0/977798869 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd3240564a0 con 0x7fd33c071c30 2024-04-25T10:05:59.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.095+0000 7fd3420ad640 1 -- 172.21.15.60:0/977798869 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd33c074770 con 0x7fd33c071c30 2024-04-25T10:05:59.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.095+0000 7fd333fff640 1 -- 172.21.15.60:0/977798869 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd324055da0 con 0x7fd33c071c30 2024-04-25T10:05:59.101 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:05:59.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.095+0000 7fd3420ad640 1 -- 172.21.15.60:0/977798869 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd31c066f90 msgr2=0x7fd31c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.095+0000 7fd3420ad640 1 --2- 172.21.15.60:0/977798869 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd31c066f90 0x7fd31c069450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd32c003fd0 tx=0x7fd32c003a00 comp rx=0 tx=0).stop 2024-04-25T10:05:59.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.095+0000 7fd3420ad640 1 -- 172.21.15.60:0/977798869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd33c071c30 msgr2=0x7fd33c13cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.095+0000 7fd3420ad640 1 --2- 172.21.15.60:0/977798869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd33c071c30 0x7fd33c13cf50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd32400d800 tx=0x7fd32400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.095+0000 7fd3420ad640 1 -- 172.21.15.60:0/977798869 shutdown_connections 2024-04-25T10:05:59.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.095+0000 7fd3420ad640 1 --2- 172.21.15.60:0/977798869 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd31c066f90 0x7fd31c069450 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.095+0000 7fd3420ad640 1 --2- 172.21.15.60:0/977798869 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd33c15fa20 0x7fd33c143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.095+0000 7fd3420ad640 1 --2- 172.21.15.60:0/977798869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd33c0cbcf0 0x7fd33c143a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.095+0000 7fd3420ad640 1 --2- 172.21.15.60:0/977798869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd33c071c30 0x7fd33c13cf50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.095+0000 7fd3420ad640 1 -- 172.21.15.60:0/977798869 >> 172.21.15.60:0/977798869 conn(0x7fd33c0670b0 msgr2=0x7fd33c071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.095+0000 7fd3420ad640 1 -- 172.21.15.60:0/977798869 shutdown_connections 2024-04-25T10:05:59.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.095+0000 7fd3420ad640 1 -- 172.21.15.60:0/977798869 wait complete. 2024-04-25T10:05:59.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:59.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.231+0000 7fe384407640 1 Processor -- start 2024-04-25T10:05:59.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.231+0000 7fe384407640 1 -- start start 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.231+0000 7fe384407640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe37c06b320 0x7fe37c0b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.231+0000 7fe384407640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe37c0afdb0 0x7fe37c0b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.231+0000 7fe384407640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37c0b5620 con 0x7fe37c1620b0 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.231+0000 7fe384407640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe37c0b3840 con 0x7fe37c06b320 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.231+0000 7fe384407640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0aefa0 con 0x7fe37c0afdb0 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.231+0000 7fe3819a2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe37c06b320 0x7fe37c0b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.231+0000 7fe3819a2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe37c06b320 0x7fe37c0b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45180/0 (socket says 172.21.15.60:45180) 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.231+0000 7fe3819a2640 1 -- 172.21.15.60:0/3437438285 learned_addr learned my addr 172.21.15.60:0/3437438285 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.231+0000 7fe3821a3640 1 --1- 172.21.15.60:0/3437438285 >> v1:172.21.15.60:6789/0 conn(0x7fe37c1620b0 0x7fe37c06ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48812/0 (socket says 172.21.15.60:48812) 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.231+0000 7fe3829a4640 1 --2- 172.21.15.60:0/3437438285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe37c0afdb0 0x7fe37c0b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.231+0000 7fe3819a2640 1 -- 172.21.15.60:0/3437438285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe37c0afdb0 msgr2=0x7fe37c0b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.231+0000 7fe3819a2640 1 --2- 172.21.15.60:0/3437438285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe37c0afdb0 0x7fe37c0b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.231+0000 7fe3819a2640 1 -- 172.21.15.60:0/3437438285 >> v1:172.21.15.60:6789/0 conn(0x7fe37c1620b0 legacy=0x7fe37c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.231+0000 7fe3819a2640 1 -- 172.21.15.60:0/3437438285 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c0b5900 con 0x7fe37c06b320 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe3819a2640 1 --2- 172.21.15.60:0/3437438285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe37c06b320 0x7fe37c0b1d40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe370009870 tx=0x7fe37001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa29c999b26e447 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe3811a1640 1 -- 172.21.15.60:0/3437438285 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3700a7070 con 0x7fe37c06b320 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe3811a1640 1 -- 172.21.15.60:0/3437438285 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3700b0070 con 0x7fe37c06b320 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe3811a1640 1 -- 172.21.15.60:0/3437438285 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3700ab890 con 0x7fe37c06b320 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe384407640 1 -- 172.21.15.60:0/3437438285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe37c06b320 msgr2=0x7fe37c0b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe384407640 1 --2- 172.21.15.60:0/3437438285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe37c06b320 0x7fe37c0b1d40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe370009870 tx=0x7fe37001c560 comp rx=0 tx=0).stop 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe384407640 1 -- 172.21.15.60:0/3437438285 shutdown_connections 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe384407640 1 --2- 172.21.15.60:0/3437438285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe37c0afdb0 0x7fe37c0b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe384407640 1 --2- 172.21.15.60:0/3437438285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe37c06b320 0x7fe37c0b1d40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe384407640 1 -- 172.21.15.60:0/3437438285 >> 172.21.15.60:0/3437438285 conn(0x7fe37c072920 msgr2=0x7fe37c099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe384407640 1 -- 172.21.15.60:0/3437438285 shutdown_connections 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe384407640 1 -- 172.21.15.60:0/3437438285 wait complete. 2024-04-25T10:05:59.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe384407640 1 Processor -- start 2024-04-25T10:05:59.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe384407640 1 -- start start 2024-04-25T10:05:59.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe384407640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe37c06b320 0x7fe37c164da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe384407640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe37c0afdb0 0x7fe37c1652e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe384407640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe37c1620b0 0x7fe37c144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe384407640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c16ea80 con 0x7fe37c1620b0 2024-04-25T10:05:59.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe384407640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0b11c0 con 0x7fe37c06b320 2024-04-25T10:05:59.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe384407640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c16ec00 con 0x7fe37c0afdb0 2024-04-25T10:05:59.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe3829a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe37c1620b0 0x7fe37c144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe3829a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe37c1620b0 0x7fe37c144380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45190/0 (socket says 172.21.15.60:45190) 2024-04-25T10:05:59.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.235+0000 7fe3829a4640 1 -- 172.21.15.60:0/610943948 learned_addr learned my addr 172.21.15.60:0/610943948 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:59.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe3821a3640 1 --2- 172.21.15.60:0/610943948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe37c06b320 0x7fe37c164da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe3819a2640 1 --2- 172.21.15.60:0/610943948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe37c0afdb0 0x7fe37c1652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe3829a4640 1 -- 172.21.15.60:0/610943948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe37c0afdb0 msgr2=0x7fe37c1652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe3829a4640 1 --2- 172.21.15.60:0/610943948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe37c0afdb0 0x7fe37c1652e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe3829a4640 1 -- 172.21.15.60:0/610943948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe37c06b320 msgr2=0x7fe37c164da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe3829a4640 1 --2- 172.21.15.60:0/610943948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe37c06b320 0x7fe37c164da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe3829a4640 1 -- 172.21.15.60:0/610943948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c1423f0 con 0x7fe37c1620b0 2024-04-25T10:05:59.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe3821a3640 1 --2- 172.21.15.60:0/610943948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe37c06b320 0x7fe37c164da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:59.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe3819a2640 1 --2- 172.21.15.60:0/610943948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe37c0afdb0 0x7fe37c1652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:59.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe3829a4640 1 --2- 172.21.15.60:0/610943948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe37c1620b0 0x7fe37c144380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe37800d560 tx=0x7fe37800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe36effd640 1 -- 172.21.15.60:0/610943948 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3780125f0 con 0x7fe37c1620b0 2024-04-25T10:05:59.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe36effd640 1 -- 172.21.15.60:0/610943948 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe378019070 con 0x7fe37c1620b0 2024-04-25T10:05:59.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe36effd640 1 -- 172.21.15.60:0/610943948 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3780129f0 con 0x7fe37c1620b0 2024-04-25T10:05:59.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe384407640 1 -- 172.21.15.60:0/610943948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37c142690 con 0x7fe37c1620b0 2024-04-25T10:05:59.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe384407640 1 -- 172.21.15.60:0/610943948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37c142c50 con 0x7fe37c1620b0 2024-04-25T10:05:59.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe384407640 1 -- 172.21.15.60:0/610943948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37c074770 con 0x7fe37c1620b0 2024-04-25T10:05:59.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe36effd640 1 -- 172.21.15.60:0/610943948 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe378012b90 con 0x7fe37c1620b0 2024-04-25T10:05:59.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe36effd640 1 --2- 172.21.15.60:0/610943948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe360066f60 0x7fe360069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe36effd640 1 -- 172.21.15.60:0/610943948 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3780854a0 con 0x7fe37c1620b0 2024-04-25T10:05:59.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe3821a3640 1 --2- 172.21.15.60:0/610943948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe360066f60 0x7fe360069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.239+0000 7fe3821a3640 1 --2- 172.21.15.60:0/610943948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe360066f60 0x7fe360069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3740012f0 tx=0x7fe374009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.243+0000 7fe36effd640 1 -- 172.21.15.60:0/610943948 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe378056410 con 0x7fe37c1620b0 2024-04-25T10:05:59.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.399+0000 7fe384407640 1 -- 172.21.15.60:0/610943948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe37c06f0a0 con 0x7fe37c1620b0 2024-04-25T10:05:59.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.399+0000 7fe36effd640 1 -- 172.21.15.60:0/610943948 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe378055d10 con 0x7fe37c1620b0 2024-04-25T10:05:59.406 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:05:59.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.403+0000 7fe384407640 1 -- 172.21.15.60:0/610943948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe360066f60 msgr2=0x7fe360069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.403+0000 7fe384407640 1 --2- 172.21.15.60:0/610943948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe360066f60 0x7fe360069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3740012f0 tx=0x7fe374009040 comp rx=0 tx=0).stop 2024-04-25T10:05:59.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.403+0000 7fe384407640 1 -- 172.21.15.60:0/610943948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe37c1620b0 msgr2=0x7fe37c144380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.403+0000 7fe384407640 1 --2- 172.21.15.60:0/610943948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe37c1620b0 0x7fe37c144380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe37800d560 tx=0x7fe37800da20 comp rx=0 tx=0).stop 2024-04-25T10:05:59.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.403+0000 7fe384407640 1 -- 172.21.15.60:0/610943948 shutdown_connections 2024-04-25T10:05:59.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.403+0000 7fe384407640 1 --2- 172.21.15.60:0/610943948 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe360066f60 0x7fe360069420 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.403+0000 7fe384407640 1 --2- 172.21.15.60:0/610943948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe37c1620b0 0x7fe37c144380 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.403+0000 7fe384407640 1 --2- 172.21.15.60:0/610943948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe37c0afdb0 0x7fe37c1652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.403+0000 7fe384407640 1 --2- 172.21.15.60:0/610943948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe37c06b320 0x7fe37c164da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.403+0000 7fe384407640 1 -- 172.21.15.60:0/610943948 >> 172.21.15.60:0/610943948 conn(0x7fe37c072920 msgr2=0x7fe37c15b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.403+0000 7fe384407640 1 -- 172.21.15.60:0/610943948 shutdown_connections 2024-04-25T10:05:59.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.403+0000 7fe384407640 1 -- 172.21.15.60:0/610943948 wait complete. 2024-04-25T10:05:59.424 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:59.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc8eae8640 1 Processor -- start 2024-04-25T10:05:59.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc8eae8640 1 -- start start 2024-04-25T10:05:59.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc8eae8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc881450b0 0x7fdc8815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc8eae8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdc880b2b90 0x7fdc88071250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc8eae8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdc88071790 0x7fdc8815f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc8eae8640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc88071be0 con 0x7fdc880b2b90 2024-04-25T10:05:59.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc8eae8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc880b3c50 con 0x7fdc88071790 2024-04-25T10:05:59.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc8eae8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc88145490 con 0x7fdc881450b0 2024-04-25T10:05:59.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc87fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdc880b2b90 0x7fdc88071250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc87fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdc880b2b90 0x7fdc88071250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45196/0 (socket says 172.21.15.60:45196) 2024-04-25T10:05:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc87fff640 1 -- 172.21.15.60:0/859466328 learned_addr learned my addr 172.21.15.60:0/859466328 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc8d085640 1 --2- 172.21.15.60:0/859466328 >> v2:172.21.15.100:3300/0 conn(0x7fdc88071790 0x7fdc8815f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc8c884640 1 --2- 172.21.15.60:0/859466328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc881450b0 0x7fdc8815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc87fff640 1 -- 172.21.15.60:0/859466328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc881450b0 msgr2=0x7fdc8815c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc87fff640 1 --2- 172.21.15.60:0/859466328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc881450b0 0x7fdc8815c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc87fff640 1 -- 172.21.15.60:0/859466328 >> v2:172.21.15.100:3300/0 conn(0x7fdc88071790 msgr2=0x7fdc8815f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc87fff640 1 --2- 172.21.15.60:0/859466328 >> v2:172.21.15.100:3300/0 conn(0x7fdc88071790 0x7fdc8815f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.535+0000 7fdc87fff640 1 -- 172.21.15.60:0/859466328 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc880b3110 con 0x7fdc880b2b90 2024-04-25T10:05:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc87fff640 1 --2- 172.21.15.60:0/859466328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc880b2b90 0x7fdc88071250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdc880b2680 tx=0x7fdc7c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e1574202cc53f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc877fe640 1 -- 172.21.15.60:0/859466328 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c0b0070 con 0x7fdc880b2b90 2024-04-25T10:05:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc877fe640 1 -- 172.21.15.60:0/859466328 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7c0ab600 con 0x7fdc880b2b90 2024-04-25T10:05:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc877fe640 1 -- 172.21.15.60:0/859466328 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c0ab9d0 con 0x7fdc880b2b90 2024-04-25T10:05:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 -- 172.21.15.60:0/859466328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc880b2b90 msgr2=0x7fdc88071250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 --2- 172.21.15.60:0/859466328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc880b2b90 0x7fdc88071250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdc880b2680 tx=0x7fdc7c01c260 comp rx=0 tx=0).stop 2024-04-25T10:05:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 -- 172.21.15.60:0/859466328 shutdown_connections 2024-04-25T10:05:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 --2- 172.21.15.60:0/859466328 >> v2:172.21.15.100:3300/0 conn(0x7fdc88071790 0x7fdc8815f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 --2- 172.21.15.60:0/859466328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc880b2b90 0x7fdc88071250 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 --2- 172.21.15.60:0/859466328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc881450b0 0x7fdc8815c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 -- 172.21.15.60:0/859466328 >> 172.21.15.60:0/859466328 conn(0x7fdc880670b0 msgr2=0x7fdc8806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 -- 172.21.15.60:0/859466328 shutdown_connections 2024-04-25T10:05:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 -- 172.21.15.60:0/859466328 wait complete. 2024-04-25T10:05:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 Processor -- start 2024-04-25T10:05:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 -- start start 2024-04-25T10:05:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88071790 0x7fdc88156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc880b2b90 0x7fdc88137790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc881450b0 0x7fdc88135d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc880cbb90 con 0x7fdc881450b0 2024-04-25T10:05:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc880cba10 con 0x7fdc88071790 2024-04-25T10:05:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8eae8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc880cbd10 con 0x7fdc880b2b90 2024-04-25T10:05:59.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8d085640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc881450b0 0x7fdc88135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8d085640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc881450b0 0x7fdc88135d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45212/0 (socket says 172.21.15.60:45212) 2024-04-25T10:05:59.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8d085640 1 -- 172.21.15.60:0/2422724248 learned_addr learned my addr 172.21.15.60:0/2422724248 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:59.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8c884640 1 --2- 172.21.15.60:0/2422724248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88071790 0x7fdc88156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc87fff640 1 --2- 172.21.15.60:0/2422724248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc880b2b90 0x7fdc88137790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8d085640 1 -- 172.21.15.60:0/2422724248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc880b2b90 msgr2=0x7fdc88137790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8d085640 1 --2- 172.21.15.60:0/2422724248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc880b2b90 0x7fdc88137790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8d085640 1 -- 172.21.15.60:0/2422724248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88071790 msgr2=0x7fdc88156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8d085640 1 --2- 172.21.15.60:0/2422724248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88071790 0x7fdc88156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8d085640 1 -- 172.21.15.60:0/2422724248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc881366f0 con 0x7fdc881450b0 2024-04-25T10:05:59.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.539+0000 7fdc8d085640 1 --2- 172.21.15.60:0/2422724248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc881450b0 0x7fdc88135d80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdc80007e60 tx=0x7fdc8000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.543+0000 7fdc857fa640 1 -- 172.21.15.60:0/2422724248 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc80014620 con 0x7fdc881450b0 2024-04-25T10:05:59.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.543+0000 7fdc857fa640 1 -- 172.21.15.60:0/2422724248 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc80019070 con 0x7fdc881450b0 2024-04-25T10:05:59.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.543+0000 7fdc8eae8640 1 -- 172.21.15.60:0/2422724248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc881369d0 con 0x7fdc881450b0 2024-04-25T10:05:59.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.543+0000 7fdc857fa640 1 -- 172.21.15.60:0/2422724248 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc80014a20 con 0x7fdc881450b0 2024-04-25T10:05:59.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.543+0000 7fdc8eae8640 1 -- 172.21.15.60:0/2422724248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc88136f90 con 0x7fdc881450b0 2024-04-25T10:05:59.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.543+0000 7fdc8eae8640 1 -- 172.21.15.60:0/2422724248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc88074770 con 0x7fdc881450b0 2024-04-25T10:05:59.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.543+0000 7fdc857fa640 1 -- 172.21.15.60:0/2422724248 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc80014bc0 con 0x7fdc881450b0 2024-04-25T10:05:59.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.543+0000 7fdc857fa640 1 --2- 172.21.15.60:0/2422724248 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc74066f60 0x7fdc74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.543+0000 7fdc857fa640 1 -- 172.21.15.60:0/2422724248 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc80085900 con 0x7fdc881450b0 2024-04-25T10:05:59.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.543+0000 7fdc8c884640 1 --2- 172.21.15.60:0/2422724248 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc74066f60 0x7fdc74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.543+0000 7fdc8c884640 1 --2- 172.21.15.60:0/2422724248 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc74066f60 0x7fdc74069420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdc880d0400 tx=0x7fdc78008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.547+0000 7fdc857fa640 1 -- 172.21.15.60:0/2422724248 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc80056900 con 0x7fdc881450b0 2024-04-25T10:05:59.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.699+0000 7fdc8eae8640 1 -- 172.21.15.60:0/2422724248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc8815a710 con 0x7fdc881450b0 2024-04-25T10:05:59.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.699+0000 7fdc857fa640 1 -- 172.21.15.60:0/2422724248 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc80085020 con 0x7fdc881450b0 2024-04-25T10:05:59.707 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:05:59.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.703+0000 7fdc8eae8640 1 -- 172.21.15.60:0/2422724248 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc74066f60 msgr2=0x7fdc74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.703+0000 7fdc8eae8640 1 --2- 172.21.15.60:0/2422724248 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc74066f60 0x7fdc74069420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdc880d0400 tx=0x7fdc78008040 comp rx=0 tx=0).stop 2024-04-25T10:05:59.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.703+0000 7fdc8eae8640 1 -- 172.21.15.60:0/2422724248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc881450b0 msgr2=0x7fdc88135d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.703+0000 7fdc8eae8640 1 --2- 172.21.15.60:0/2422724248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc881450b0 0x7fdc88135d80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdc80007e60 tx=0x7fdc8000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.703+0000 7fdc8eae8640 1 -- 172.21.15.60:0/2422724248 shutdown_connections 2024-04-25T10:05:59.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.703+0000 7fdc8eae8640 1 --2- 172.21.15.60:0/2422724248 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc74066f60 0x7fdc74069420 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.703+0000 7fdc8eae8640 1 --2- 172.21.15.60:0/2422724248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc881450b0 0x7fdc88135d80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.703+0000 7fdc8eae8640 1 --2- 172.21.15.60:0/2422724248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc880b2b90 0x7fdc88137790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.703+0000 7fdc8eae8640 1 --2- 172.21.15.60:0/2422724248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc88071790 0x7fdc88156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.703+0000 7fdc8eae8640 1 -- 172.21.15.60:0/2422724248 >> 172.21.15.60:0/2422724248 conn(0x7fdc880670b0 msgr2=0x7fdc8815e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.703+0000 7fdc8eae8640 1 -- 172.21.15.60:0/2422724248 shutdown_connections 2024-04-25T10:05:59.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.703+0000 7fdc8eae8640 1 -- 172.21.15.60:0/2422724248 wait complete. 2024-04-25T10:05:59.725 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:05:59.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.843+0000 7f4ab7cb7640 1 Processor -- start 2024-04-25T10:05:59.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.843+0000 7f4ab7cb7640 1 -- start start 2024-04-25T10:05:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.843+0000 7f4ab7cb7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4ab00b9270 0x7f4ab00ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.843+0000 7f4ab7cb7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4ab00b96d0 0x7f4ab00cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.843+0000 7f4ab7cb7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab00d1af0 con 0x7f4ab015db60 2024-04-25T10:05:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.843+0000 7f4ab7cb7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab015df40 con 0x7f4ab00b96d0 2024-04-25T10:05:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.843+0000 7f4ab7cb7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab00cf140 con 0x7f4ab00b9270 2024-04-25T10:05:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.843+0000 7f4ab6254640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4ab00b96d0 0x7f4ab00cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.843+0000 7f4ab6254640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4ab00b96d0 0x7f4ab00cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45220/0 (socket says 172.21.15.60:45220) 2024-04-25T10:05:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.843+0000 7f4ab6254640 1 -- 172.21.15.60:0/2884365779 learned_addr learned my addr 172.21.15.60:0/2884365779 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.843+0000 7f4ab5252640 1 --2- 172.21.15.60:0/2884365779 >> v2:172.21.15.100:3300/0 conn(0x7f4ab00b9270 0x7f4ab00ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.843+0000 7f4ab6254640 1 -- 172.21.15.60:0/2884365779 >> v2:172.21.15.100:3300/0 conn(0x7f4ab00b9270 msgr2=0x7f4ab00ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.843+0000 7f4ab6254640 1 --2- 172.21.15.60:0/2884365779 >> v2:172.21.15.100:3300/0 conn(0x7f4ab00b9270 0x7f4ab00ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.843+0000 7f4ab6254640 1 -- 172.21.15.60:0/2884365779 >> v1:172.21.15.60:6789/0 conn(0x7f4ab015db60 legacy=0x7f4ab00b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.843+0000 7f4ab6254640 1 -- 172.21.15.60:0/2884365779 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab00d1dd0 con 0x7f4ab00b96d0 2024-04-25T10:05:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.847+0000 7f4ab6254640 1 --2- 172.21.15.60:0/2884365779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ab00b96d0 0x7f4ab00cec00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00ce360 tx=0x7f4aa801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a200b82f37467 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.847+0000 7f4ab4a51640 1 -- 172.21.15.60:0/2884365779 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80a5710 con 0x7f4ab00b96d0 2024-04-25T10:05:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.847+0000 7f4ab4a51640 1 -- 172.21.15.60:0/2884365779 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa80a58b0 con 0x7f4ab00b96d0 2024-04-25T10:05:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.847+0000 7f4ab4a51640 1 -- 172.21.15.60:0/2884365779 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80a5c80 con 0x7f4ab00b96d0 2024-04-25T10:05:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.847+0000 7f4ab7cb7640 1 -- 172.21.15.60:0/2884365779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ab00b96d0 msgr2=0x7f4ab00cec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.847+0000 7f4ab7cb7640 1 --2- 172.21.15.60:0/2884365779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ab00b96d0 0x7f4ab00cec00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00ce360 tx=0x7f4aa801c960 comp rx=0 tx=0).stop 2024-04-25T10:05:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.847+0000 7f4ab7cb7640 1 -- 172.21.15.60:0/2884365779 shutdown_connections 2024-04-25T10:05:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.847+0000 7f4ab7cb7640 1 --2- 172.21.15.60:0/2884365779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ab00b96d0 0x7f4ab00cec00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.847+0000 7f4ab7cb7640 1 --2- 172.21.15.60:0/2884365779 >> v2:172.21.15.100:3300/0 conn(0x7f4ab00b9270 0x7f4ab00ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.847+0000 7f4ab7cb7640 1 -- 172.21.15.60:0/2884365779 >> 172.21.15.60:0/2884365779 conn(0x7f4ab00d0470 msgr2=0x7f4ab00d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.847+0000 7f4ab7cb7640 1 -- 172.21.15.60:0/2884365779 shutdown_connections 2024-04-25T10:05:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.847+0000 7f4ab7cb7640 1 -- 172.21.15.60:0/2884365779 wait complete. 2024-04-25T10:05:59.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.847+0000 7f4ab7cb7640 1 Processor -- start 2024-04-25T10:05:59.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.847+0000 7f4ab7cb7640 1 -- start start 2024-04-25T10:05:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab7cb7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ab00b9270 0x7f4ab015b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab7cb7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ab00b96d0 0x7f4ab014b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab7cb7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ab015db60 0x7f4ab014cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab7cb7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00d3500 con 0x7f4ab00b9270 2024-04-25T10:05:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab7cb7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00d3380 con 0x7f4ab00b96d0 2024-04-25T10:05:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab7cb7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00d3680 con 0x7f4ab015db60 2024-04-25T10:05:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab5a53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ab00b9270 0x7f4ab015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab5a53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ab00b9270 0x7f4ab015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45232/0 (socket says 172.21.15.60:45232) 2024-04-25T10:05:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab5a53640 1 -- 172.21.15.60:0/1556454125 learned_addr learned my addr 172.21.15.60:0/1556454125 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:05:59.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab6254640 1 --2- 172.21.15.60:0/1556454125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ab015db60 0x7f4ab014cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab5252640 1 --2- 172.21.15.60:0/1556454125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ab00b96d0 0x7f4ab014b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab5a53640 1 -- 172.21.15.60:0/1556454125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ab015db60 msgr2=0x7f4ab014cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab5a53640 1 --2- 172.21.15.60:0/1556454125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ab015db60 0x7f4ab014cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab5a53640 1 -- 172.21.15.60:0/1556454125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ab00b96d0 msgr2=0x7f4ab014b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab5a53640 1 --2- 172.21.15.60:0/1556454125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ab00b96d0 0x7f4ab014b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab5a53640 1 -- 172.21.15.60:0/1556454125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab014bef0 con 0x7f4ab00b9270 2024-04-25T10:05:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab6254640 1 --2- 172.21.15.60:0/1556454125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ab015db60 0x7f4ab014cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab5252640 1 --2- 172.21.15.60:0/1556454125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ab00b96d0 0x7f4ab014b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab5a53640 1 --2- 172.21.15.60:0/1556454125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ab00b9270 0x7f4ab015b200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00c370 tx=0x7f4aac00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4aa67fc640 1 -- 172.21.15.60:0/1556454125 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0052d0 con 0x7f4ab00b9270 2024-04-25T10:05:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4aa67fc640 1 -- 172.21.15.60:0/1556454125 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aac005470 con 0x7f4ab00b9270 2024-04-25T10:05:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4aa67fc640 1 -- 172.21.15.60:0/1556454125 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac007260 con 0x7f4ab00b9270 2024-04-25T10:05:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab7cb7640 1 -- 172.21.15.60:0/1556454125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab015b7e0 con 0x7f4ab00b9270 2024-04-25T10:05:59.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab7cb7640 1 -- 172.21.15.60:0/1556454125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab015ec20 con 0x7f4ab00b9270 2024-04-25T10:05:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4ab7cb7640 1 -- 172.21.15.60:0/1556454125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab0074770 con 0x7f4ab00b9270 2024-04-25T10:05:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4aa67fc640 1 -- 172.21.15.60:0/1556454125 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4aac007400 con 0x7f4ab00b9270 2024-04-25T10:05:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4aa67fc640 1 --2- 172.21.15.60:0/1556454125 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a94066f60 0x7f4a94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.851+0000 7f4aa67fc640 1 -- 172.21.15.60:0/1556454125 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4aac084530 con 0x7f4ab00b9270 2024-04-25T10:05:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.855+0000 7f4ab5252640 1 --2- 172.21.15.60:0/1556454125 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a94066f60 0x7f4a94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.855+0000 7f4ab5252640 1 --2- 172.21.15.60:0/1556454125 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a94066f60 0x7f4a94069420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00ca110 tx=0x7f4aa0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:05:59.859+0000 7f4aa67fc640 1 -- 172.21.15.60:0/1556454125 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4aac0554a0 con 0x7f4ab00b9270 2024-04-25T10:06:00.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.027+0000 7f4ab7cb7640 1 -- 172.21.15.60:0/1556454125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4ab015a8a0 con 0x7f4ab00b9270 2024-04-25T10:06:00.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.027+0000 7f4aa67fc640 1 -- 172.21.15.60:0/1556454125 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4aac054da0 con 0x7f4ab00b9270 2024-04-25T10:06:00.035 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:06:00.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.031+0000 7f4ab7cb7640 1 -- 172.21.15.60:0/1556454125 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a94066f60 msgr2=0x7f4a94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.031+0000 7f4ab7cb7640 1 --2- 172.21.15.60:0/1556454125 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a94066f60 0x7f4a94069420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00ca110 tx=0x7f4aa0008040 comp rx=0 tx=0).stop 2024-04-25T10:06:00.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.031+0000 7f4ab7cb7640 1 -- 172.21.15.60:0/1556454125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ab00b9270 msgr2=0x7f4ab015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.031+0000 7f4ab7cb7640 1 --2- 172.21.15.60:0/1556454125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ab00b9270 0x7f4ab015b200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00c370 tx=0x7f4aac00c830 comp rx=0 tx=0).stop 2024-04-25T10:06:00.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.031+0000 7f4ab7cb7640 1 -- 172.21.15.60:0/1556454125 shutdown_connections 2024-04-25T10:06:00.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.031+0000 7f4ab7cb7640 1 --2- 172.21.15.60:0/1556454125 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a94066f60 0x7f4a94069420 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.031+0000 7f4ab7cb7640 1 --2- 172.21.15.60:0/1556454125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ab015db60 0x7f4ab014cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.031+0000 7f4ab7cb7640 1 --2- 172.21.15.60:0/1556454125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ab00b96d0 0x7f4ab014b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.031+0000 7f4ab7cb7640 1 --2- 172.21.15.60:0/1556454125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ab00b9270 0x7f4ab015b200 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.031+0000 7f4ab7cb7640 1 -- 172.21.15.60:0/1556454125 >> 172.21.15.60:0/1556454125 conn(0x7f4ab00d0470 msgr2=0x7f4ab00b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:00.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.031+0000 7f4ab7cb7640 1 -- 172.21.15.60:0/1556454125 shutdown_connections 2024-04-25T10:06:00.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.031+0000 7f4ab7cb7640 1 -- 172.21.15.60:0/1556454125 wait complete. 2024-04-25T10:06:00.055 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:06:00.055 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:00.062 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:00.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c03c23640 1 Processor -- start 2024-04-25T10:06:00.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c03c23640 1 -- start start 2024-04-25T10:06:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c03c23640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bfc0b1cc0 0x7f2bfc0b0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c03c23640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bfc0b5310 con 0x7f2bfc15dcc0 2024-04-25T10:06:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c03c23640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bfc0b55b0 con 0x7f2bfc06ac10 2024-04-25T10:06:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c03c23640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0b5b70 con 0x7f2bfc0b1cc0 2024-04-25T10:06:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c011be640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2bfc15dcc0 0x7f2bfc0b0180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48818/0 (socket says 172.21.15.60:48818) 2024-04-25T10:06:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c011be640 1 -- 172.21.15.60:0/3317388552 learned_addr learned my addr 172.21.15.60:0/3317388552 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c021c0640 1 --2- 172.21.15.60:0/3317388552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bfc0b1cc0 0x7f2bfc0b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c009bd640 1 -- 172.21.15.60:0/3317388552 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203829718 0 0) 0x7f2bfc0b5310 con 0x7f2bfc15dcc0 2024-04-25T10:06:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c009bd640 1 -- 172.21.15.60:0/3317388552 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2be8003530 con 0x7f2bfc15dcc0 2024-04-25T10:06:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c009bd640 1 -- 172.21.15.60:0/3317388552 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1683692054 0 0) 0x7f2be8003530 con 0x7f2bfc15dcc0 2024-04-25T10:06:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c009bd640 1 -- 172.21.15.60:0/3317388552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bfc0b1cc0 msgr2=0x7f2bfc0b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c009bd640 1 --2- 172.21.15.60:0/3317388552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bfc0b1cc0 0x7f2bfc0b0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c009bd640 1 -- 172.21.15.60:0/3317388552 >> v1:172.21.15.100:6789/0 conn(0x7f2bfc06ac10 legacy=0x7f2bfc06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c009bd640 1 -- 172.21.15.60:0/3317388552 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc0b5890 con 0x7f2bfc15dcc0 2024-04-25T10:06:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c009bd640 1 -- 172.21.15.60:0/3317388552 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f2bf00016a0 con 0x7f2bfc15dcc0 2024-04-25T10:06:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c009bd640 1 -- 172.21.15.60:0/3317388552 >> v1:172.21.15.60:6789/0 conn(0x7f2bfc15dcc0 legacy=0x7f2bfc0b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c009bd640 1 --2- 172.21.15.60:0/3317388552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2be80039f0 0x7f2be8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c009bd640 1 --2- 172.21.15.60:0/3317388552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bfc0b1cc0 0x7f2be8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c009bd640 1 --2- 172.21.15.60:0/3317388552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be8004d50 0x7f2be80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c009bd640 1 -- 172.21.15.60:0/3317388552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0b5b70 con 0x7f2be8004d50 2024-04-25T10:06:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c009bd640 1 -- 172.21.15.60:0/3317388552 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be800ba80 con 0x7f2be80039f0 2024-04-25T10:06:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c009bd640 1 -- 172.21.15.60:0/3317388552 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be800bc80 con 0x7f2bfc0b1cc0 2024-04-25T10:06:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c021c0640 1 --2- 172.21.15.60:0/3317388552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be8004d50 0x7f2be80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c019bf640 1 --2- 172.21.15.60:0/3317388552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2be80039f0 0x7f2be8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c011be640 1 --2- 172.21.15.60:0/3317388552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bfc0b1cc0 0x7f2be8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c021c0640 1 -- 172.21.15.60:0/3317388552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bfc0b1cc0 msgr2=0x7f2be8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c021c0640 1 --2- 172.21.15.60:0/3317388552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bfc0b1cc0 0x7f2be8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c021c0640 1 -- 172.21.15.60:0/3317388552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2be80039f0 msgr2=0x7f2be8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c021c0640 1 --2- 172.21.15.60:0/3317388552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2be80039f0 0x7f2be8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c021c0640 1 -- 172.21.15.60:0/3317388552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be8003530 con 0x7f2be8004d50 2024-04-25T10:06:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.179+0000 7f2c019bf640 1 --2- 172.21.15.60:0/3317388552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2be80039f0 0x7f2be8003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:00.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c021c0640 1 --2- 172.21.15.60:0/3317388552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be8004d50 0x7f2be80051d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc067600 tx=0x7f2bf801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f7c39100b4235d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c009bd640 1 -- 172.21.15.60:0/3317388552 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf80b2070 con 0x7f2be8004d50 2024-04-25T10:06:00.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c009bd640 1 -- 172.21.15.60:0/3317388552 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf80a55a0 con 0x7f2be8004d50 2024-04-25T10:06:00.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c009bd640 1 -- 172.21.15.60:0/3317388552 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf80096b0 con 0x7f2be8004d50 2024-04-25T10:06:00.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c03c23640 1 -- 172.21.15.60:0/3317388552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be8004d50 msgr2=0x7f2be80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c03c23640 1 --2- 172.21.15.60:0/3317388552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be8004d50 0x7f2be80051d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc067600 tx=0x7f2bf801c9b0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c021c0640 1 -- 172.21.15.60:0/3317388552 reap_dead start 2024-04-25T10:06:00.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c03c23640 1 -- 172.21.15.60:0/3317388552 shutdown_connections 2024-04-25T10:06:00.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c03c23640 1 -- 172.21.15.60:0/3317388552 >> 172.21.15.60:0/3317388552 conn(0x7f2bfc072920 msgr2=0x7f2bfc099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:00.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c03c23640 1 -- 172.21.15.60:0/3317388552 shutdown_connections 2024-04-25T10:06:00.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c03c23640 1 -- 172.21.15.60:0/3317388552 wait complete. 2024-04-25T10:06:00.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c03c23640 1 Processor -- start 2024-04-25T10:06:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c03c23640 1 -- start start 2024-04-25T10:06:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c03c23640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be8004d50 0x7f2bfc14af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c03c23640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bfc09a690 0x7f2bfc155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c03c23640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bfc09c8c0 0x7f2bfc09cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c03c23640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc06b840 con 0x7f2be8004d50 2024-04-25T10:06:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c03c23640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0aefa0 con 0x7f2bfc09c8c0 2024-04-25T10:06:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c03c23640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc06b9c0 con 0x7f2bfc09a690 2024-04-25T10:06:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c019bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be8004d50 0x7f2bfc14af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c019bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be8004d50 0x7f2bfc14af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45248/0 (socket says 172.21.15.60:45248) 2024-04-25T10:06:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.183+0000 7f2c019bf640 1 -- 172.21.15.60:0/3415335632 learned_addr learned my addr 172.21.15.60:0/3415335632 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2c011be640 1 --2- 172.21.15.60:0/3415335632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bfc09a690 0x7f2bfc155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2c021c0640 1 --2- 172.21.15.60:0/3415335632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bfc09c8c0 0x7f2bfc09cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2c019bf640 1 -- 172.21.15.60:0/3415335632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bfc09a690 msgr2=0x7f2bfc155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2c019bf640 1 --2- 172.21.15.60:0/3415335632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bfc09a690 0x7f2bfc155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2c019bf640 1 -- 172.21.15.60:0/3415335632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bfc09c8c0 msgr2=0x7f2bfc09cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2c019bf640 1 --2- 172.21.15.60:0/3415335632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bfc09c8c0 0x7f2bfc09cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2c019bf640 1 -- 172.21.15.60:0/3415335632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc1562a0 con 0x7f2be8004d50 2024-04-25T10:06:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2c021c0640 1 --2- 172.21.15.60:0/3415335632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bfc09c8c0 0x7f2bfc09cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2c019bf640 1 --2- 172.21.15.60:0/3415335632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be8004d50 0x7f2bfc14af40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400eb50 tx=0x7f2bf400c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2bee7fc640 1 -- 172.21.15.60:0/3415335632 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf400ee50 con 0x7f2be8004d50 2024-04-25T10:06:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2bee7fc640 1 -- 172.21.15.60:0/3415335632 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf4018070 con 0x7f2be8004d50 2024-04-25T10:06:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2bee7fc640 1 -- 172.21.15.60:0/3415335632 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf400ce00 con 0x7f2be8004d50 2024-04-25T10:06:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2c03c23640 1 -- 172.21.15.60:0/3415335632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bfc156580 con 0x7f2be8004d50 2024-04-25T10:06:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2c03c23640 1 -- 172.21.15.60:0/3415335632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bfc156b40 con 0x7f2be8004d50 2024-04-25T10:06:00.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2c03c23640 1 -- 172.21.15.60:0/3415335632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bfc074770 con 0x7f2be8004d50 2024-04-25T10:06:00.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2bee7fc640 1 -- 172.21.15.60:0/3415335632 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2bf4003980 con 0x7f2be8004d50 2024-04-25T10:06:00.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2bee7fc640 1 --2- 172.21.15.60:0/3415335632 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2be0066f90 0x7f2be0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2bee7fc640 1 -- 172.21.15.60:0/3415335632 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bf4085150 con 0x7f2be8004d50 2024-04-25T10:06:00.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.187+0000 7f2c011be640 1 --2- 172.21.15.60:0/3415335632 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2be0066f90 0x7f2be0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.191+0000 7f2c011be640 1 --2- 172.21.15.60:0/3415335632 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2be0066f90 0x7f2be0069450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0703e0 tx=0x7f2bf0007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.191+0000 7f2bee7fc640 1 -- 172.21.15.60:0/3415335632 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2bf4003c70 con 0x7f2be8004d50 2024-04-25T10:06:00.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.351+0000 7f2c03c23640 1 -- 172.21.15.60:0/3415335632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bfc09c200 con 0x7f2be8004d50 2024-04-25T10:06:00.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.355+0000 7f2bee7fc640 1 -- 172.21.15.60:0/3415335632 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f2bf40560d0 con 0x7f2be8004d50 2024-04-25T10:06:00.361 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:00.361 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:05:22.036582+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:00.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.355+0000 7f2c03c23640 1 -- 172.21.15.60:0/3415335632 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2be0066f90 msgr2=0x7f2be0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.355+0000 7f2c03c23640 1 --2- 172.21.15.60:0/3415335632 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2be0066f90 0x7f2be0069450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0703e0 tx=0x7f2bf0007040 comp rx=0 tx=0).stop 2024-04-25T10:06:00.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.355+0000 7f2c03c23640 1 -- 172.21.15.60:0/3415335632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be8004d50 msgr2=0x7f2bfc14af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.355+0000 7f2c03c23640 1 --2- 172.21.15.60:0/3415335632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be8004d50 0x7f2bfc14af40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400eb50 tx=0x7f2bf400c4b0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.355+0000 7f2c03c23640 1 -- 172.21.15.60:0/3415335632 shutdown_connections 2024-04-25T10:06:00.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.355+0000 7f2c03c23640 1 --2- 172.21.15.60:0/3415335632 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2be0066f90 0x7f2be0069450 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.355+0000 7f2c03c23640 1 --2- 172.21.15.60:0/3415335632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bfc09c8c0 0x7f2bfc09cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.355+0000 7f2c03c23640 1 --2- 172.21.15.60:0/3415335632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bfc09a690 0x7f2bfc155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.355+0000 7f2c03c23640 1 --2- 172.21.15.60:0/3415335632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2be8004d50 0x7f2bfc14af40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.355+0000 7f2c03c23640 1 -- 172.21.15.60:0/3415335632 >> 172.21.15.60:0/3415335632 conn(0x7f2bfc072920 msgr2=0x7f2bfc0ade40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:00.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.355+0000 7f2c03c23640 1 -- 172.21.15.60:0/3415335632 shutdown_connections 2024-04-25T10:06:00.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.355+0000 7f2c03c23640 1 -- 172.21.15.60:0/3415335632 wait complete. 2024-04-25T10:06:00.380 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:18.976643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:19.405302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:00.380 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:00.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff442c14640 1 Processor -- start 2024-04-25T10:06:00.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff442c14640 1 -- start start 2024-04-25T10:06:00.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff442c14640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff43c06b320 0x7ff43c0b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff442c14640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43c0b3bd0 con 0x7ff43c0b3610 2024-04-25T10:06:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff442c14640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43c0b55b0 con 0x7ff43c0d11b0 2024-04-25T10:06:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff442c14640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff43c0b5b70 con 0x7ff43c06b320 2024-04-25T10:06:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff43c0b3610 0x7ff43c06ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48824/0 (socket says 172.21.15.60:48824) 2024-04-25T10:06:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43bfff640 1 -- 172.21.15.60:0/291890240 learned_addr learned my addr 172.21.15.60:0/291890240 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff4411b1640 1 --2- 172.21.15.60:0/291890240 >> v2:172.21.15.100:3300/0 conn(0x7ff43c06b320 0x7ff43c0b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43b7fe640 1 -- 172.21.15.60:0/291890240 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250698047 0 0) 0x7ff43c0b3bd0 con 0x7ff43c0b3610 2024-04-25T10:06:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43b7fe640 1 -- 172.21.15.60:0/291890240 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff424003530 con 0x7ff43c0b3610 2024-04-25T10:06:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43b7fe640 1 -- 172.21.15.60:0/291890240 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 845095293 0 0) 0x7ff424003530 con 0x7ff43c0b3610 2024-04-25T10:06:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43b7fe640 1 -- 172.21.15.60:0/291890240 >> v2:172.21.15.100:3300/0 conn(0x7ff43c06b320 msgr2=0x7ff43c0b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43b7fe640 1 --2- 172.21.15.60:0/291890240 >> v2:172.21.15.100:3300/0 conn(0x7ff43c06b320 0x7ff43c0b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43b7fe640 1 -- 172.21.15.60:0/291890240 >> v1:172.21.15.100:6789/0 conn(0x7ff43c0d11b0 legacy=0x7ff43c15ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43b7fe640 1 -- 172.21.15.60:0/291890240 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0b5890 con 0x7ff43c0b3610 2024-04-25T10:06:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43b7fe640 1 -- 172.21.15.60:0/291890240 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7ff430001680 con 0x7ff43c0b3610 2024-04-25T10:06:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43b7fe640 1 -- 172.21.15.60:0/291890240 >> v1:172.21.15.60:6789/0 conn(0x7ff43c0b3610 legacy=0x7ff43c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43b7fe640 1 --2- 172.21.15.60:0/291890240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4240039c0 0x7ff424003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43b7fe640 1 --2- 172.21.15.60:0/291890240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff424004810 0x7ff424004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43b7fe640 1 --2- 172.21.15.60:0/291890240 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4240051d0 0x7ff42400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43b7fe640 1 -- 172.21.15.60:0/291890240 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0b5b70 con 0x7ff4240039c0 2024-04-25T10:06:00.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43b7fe640 1 -- 172.21.15.60:0/291890240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42400c170 con 0x7ff424004810 2024-04-25T10:06:00.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43b7fe640 1 -- 172.21.15.60:0/291890240 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42400c370 con 0x7ff4240051d0 2024-04-25T10:06:00.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff4409b0640 1 --2- 172.21.15.60:0/291890240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4240039c0 0x7ff424003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff43bfff640 1 --2- 172.21.15.60:0/291890240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff424004810 0x7ff424004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff4411b1640 1 --2- 172.21.15.60:0/291890240 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4240051d0 0x7ff42400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff4409b0640 1 -- 172.21.15.60:0/291890240 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4240051d0 msgr2=0x7ff42400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff4409b0640 1 --2- 172.21.15.60:0/291890240 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4240051d0 0x7ff42400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff4409b0640 1 -- 172.21.15.60:0/291890240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff424004810 msgr2=0x7ff424004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff4409b0640 1 --2- 172.21.15.60:0/291890240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff424004810 0x7ff424004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff4409b0640 1 -- 172.21.15.60:0/291890240 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff424003530 con 0x7ff4240039c0 2024-04-25T10:06:00.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff4411b1640 1 --2- 172.21.15.60:0/291890240 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4240051d0 0x7ff42400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:00.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.503+0000 7ff4411b1640 1 -- 172.21.15.60:0/291890240 reap_dead start 2024-04-25T10:06:00.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.507+0000 7ff4409b0640 1 --2- 172.21.15.60:0/291890240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4240039c0 0x7ff424003de0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff42c001780 tx=0x7ff42c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3604c1538c4e5758 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.507+0000 7ff43b7fe640 1 -- 172.21.15.60:0/291890240 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c0013a0 con 0x7ff4240039c0 2024-04-25T10:06:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.507+0000 7ff43b7fe640 1 -- 172.21.15.60:0/291890240 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42c0052f0 con 0x7ff4240039c0 2024-04-25T10:06:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.507+0000 7ff43b7fe640 1 -- 172.21.15.60:0/291890240 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c003840 con 0x7ff4240039c0 2024-04-25T10:06:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.507+0000 7ff442c14640 1 -- 172.21.15.60:0/291890240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4240039c0 msgr2=0x7ff424003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.507+0000 7ff442c14640 1 --2- 172.21.15.60:0/291890240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4240039c0 0x7ff424003de0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff42c001780 tx=0x7ff42c01c400 comp rx=0 tx=0).stop 2024-04-25T10:06:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.507+0000 7ff442c14640 1 -- 172.21.15.60:0/291890240 shutdown_connections 2024-04-25T10:06:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.507+0000 7ff442c14640 1 --2- 172.21.15.60:0/291890240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4240039c0 0x7ff424003de0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.507+0000 7ff442c14640 1 -- 172.21.15.60:0/291890240 >> 172.21.15.60:0/291890240 conn(0x7ff43c072900 msgr2=0x7ff43c099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.507+0000 7ff442c14640 1 -- 172.21.15.60:0/291890240 shutdown_connections 2024-04-25T10:06:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.507+0000 7ff442c14640 1 -- 172.21.15.60:0/291890240 wait complete. 2024-04-25T10:06:00.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.507+0000 7ff442c14640 1 Processor -- start 2024-04-25T10:06:00.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.507+0000 7ff442c14640 1 -- start start 2024-04-25T10:06:00.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff442c14640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4240039c0 0x7ff43c1452a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff442c14640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff43c15dbc0 0x7ff43c0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff442c14640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff43c15e020 0x7ff43c0af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff442c14640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0b0f40 con 0x7ff43c15dbc0 2024-04-25T10:06:00.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff442c14640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0b0dc0 con 0x7ff4240039c0 2024-04-25T10:06:00.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff442c14640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0b10c0 con 0x7ff43c15e020 2024-04-25T10:06:00.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff43bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff43c15dbc0 0x7ff43c0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff43bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff43c15dbc0 0x7ff43c0aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45266/0 (socket says 172.21.15.60:45266) 2024-04-25T10:06:00.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff43bfff640 1 -- 172.21.15.60:0/2758941727 learned_addr learned my addr 172.21.15.60:0/2758941727 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:00.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff4411b1640 1 --2- 172.21.15.60:0/2758941727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff43c15e020 0x7ff43c0af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff4409b0640 1 --2- 172.21.15.60:0/2758941727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4240039c0 0x7ff43c1452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff43bfff640 1 -- 172.21.15.60:0/2758941727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff43c15e020 msgr2=0x7ff43c0af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff43bfff640 1 --2- 172.21.15.60:0/2758941727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff43c15e020 0x7ff43c0af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff43bfff640 1 -- 172.21.15.60:0/2758941727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4240039c0 msgr2=0x7ff43c1452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff43bfff640 1 --2- 172.21.15.60:0/2758941727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4240039c0 0x7ff43c1452a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff43bfff640 1 -- 172.21.15.60:0/2758941727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c15d360 con 0x7ff43c15dbc0 2024-04-25T10:06:00.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff4409b0640 1 --2- 172.21.15.60:0/2758941727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4240039c0 0x7ff43c1452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:00.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff4411b1640 1 --2- 172.21.15.60:0/2758941727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff43c15e020 0x7ff43c0af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:00.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff43bfff640 1 --2- 172.21.15.60:0/2758941727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff43c15dbc0 0x7ff43c0aef40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff4300072c0 tx=0x7ff430007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff4397fa640 1 -- 172.21.15.60:0/2758941727 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff430007e10 con 0x7ff43c15dbc0 2024-04-25T10:06:00.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff4397fa640 1 -- 172.21.15.60:0/2758941727 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff430018070 con 0x7ff43c15dbc0 2024-04-25T10:06:00.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff442c14640 1 -- 172.21.15.60:0/2758941727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43c15d640 con 0x7ff43c15dbc0 2024-04-25T10:06:00.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff4397fa640 1 -- 172.21.15.60:0/2758941727 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43000fb20 con 0x7ff43c15dbc0 2024-04-25T10:06:00.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff442c14640 1 -- 172.21.15.60:0/2758941727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43c1fb2f0 con 0x7ff43c15dbc0 2024-04-25T10:06:00.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff442c14640 1 -- 172.21.15.60:0/2758941727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43c06ccb0 con 0x7ff43c15dbc0 2024-04-25T10:06:00.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff4397fa640 1 -- 172.21.15.60:0/2758941727 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff43000fcc0 con 0x7ff43c15dbc0 2024-04-25T10:06:00.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff4397fa640 1 --2- 172.21.15.60:0/2758941727 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff428066f60 0x7ff428069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff4397fa640 1 -- 172.21.15.60:0/2758941727 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff430084eb0 con 0x7ff43c15dbc0 2024-04-25T10:06:00.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.511+0000 7ff4409b0640 1 --2- 172.21.15.60:0/2758941727 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff428066f60 0x7ff428069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.515+0000 7ff4409b0640 1 --2- 172.21.15.60:0/2758941727 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff428066f60 0x7ff428069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff42c001750 tx=0x7ff42c0011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.515+0000 7ff4397fa640 1 -- 172.21.15.60:0/2758941727 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff430055f50 con 0x7ff43c15dbc0 2024-04-25T10:06:00.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.683+0000 7ff442c14640 1 -- 172.21.15.60:0/2758941727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff43c074770 con 0x7ff43c15dbc0 2024-04-25T10:06:00.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.683+0000 7ff4397fa640 1 -- 172.21.15.60:0/2758941727 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff430055850 con 0x7ff43c15dbc0 2024-04-25T10:06:00.689 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:06:00.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.683+0000 7ff442c14640 1 -- 172.21.15.60:0/2758941727 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff428066f60 msgr2=0x7ff428069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.683+0000 7ff442c14640 1 --2- 172.21.15.60:0/2758941727 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff428066f60 0x7ff428069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff42c001750 tx=0x7ff42c0011a0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.683+0000 7ff442c14640 1 -- 172.21.15.60:0/2758941727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff43c15dbc0 msgr2=0x7ff43c0aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.683+0000 7ff442c14640 1 --2- 172.21.15.60:0/2758941727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff43c15dbc0 0x7ff43c0aef40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff4300072c0 tx=0x7ff430007780 comp rx=0 tx=0).stop 2024-04-25T10:06:00.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.683+0000 7ff442c14640 1 -- 172.21.15.60:0/2758941727 shutdown_connections 2024-04-25T10:06:00.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.683+0000 7ff442c14640 1 --2- 172.21.15.60:0/2758941727 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff428066f60 0x7ff428069420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.683+0000 7ff442c14640 1 --2- 172.21.15.60:0/2758941727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff43c15e020 0x7ff43c0af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.683+0000 7ff442c14640 1 --2- 172.21.15.60:0/2758941727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff43c15dbc0 0x7ff43c0aef40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.683+0000 7ff442c14640 1 --2- 172.21.15.60:0/2758941727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4240039c0 0x7ff43c1452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.683+0000 7ff442c14640 1 -- 172.21.15.60:0/2758941727 >> 172.21.15.60:0/2758941727 conn(0x7ff43c072900 msgr2=0x7ff43c0b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:00.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.683+0000 7ff442c14640 1 -- 172.21.15.60:0/2758941727 shutdown_connections 2024-04-25T10:06:00.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.683+0000 7ff442c14640 1 -- 172.21.15.60:0/2758941727 wait complete. 2024-04-25T10:06:00.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:00.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.823+0000 7fe500c82640 1 Processor -- start 2024-04-25T10:06:00.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.823+0000 7fe500c82640 1 -- start start 2024-04-25T10:06:00.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.823+0000 7fe500c82640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4fc0b1f30 0x7fe4fc15f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.823+0000 7fe500c82640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe4fc15f920 0x7fe4fc06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.823+0000 7fe500c82640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc160230 con 0x7fe4fc06b150 2024-04-25T10:06:00.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.823+0000 7fe500c82640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4fc15fd60 con 0x7fe4fc15f920 2024-04-25T10:06:00.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.823+0000 7fe500c82640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc06bad0 con 0x7fe4fc0b1f30 2024-04-25T10:06:00.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.823+0000 7fe4f9d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe4fc15f920 0x7fe4fc06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.823+0000 7fe4f9d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe4fc15f920 0x7fe4fc06ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45278/0 (socket says 172.21.15.60:45278) 2024-04-25T10:06:00.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.823+0000 7fe4fad9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe4fc06b150 0x7fe4fc15bdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48836/0 (socket says 172.21.15.60:48836) 2024-04-25T10:06:00.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.823+0000 7fe4f9d9b640 1 -- 172.21.15.60:0/120609709 learned_addr learned my addr 172.21.15.60:0/120609709 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:00.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.823+0000 7fe4f9d9b640 1 -- 172.21.15.60:0/120609709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4fc0b1f30 msgr2=0x7fe4fc15f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.823+0000 7fe4f9d9b640 1 --2- 172.21.15.60:0/120609709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4fc0b1f30 0x7fe4fc15f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.823+0000 7fe4f9d9b640 1 -- 172.21.15.60:0/120609709 >> v1:172.21.15.60:6789/0 conn(0x7fe4fc06b150 legacy=0x7fe4fc15bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.823+0000 7fe4f9d9b640 1 -- 172.21.15.60:0/120609709 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc06b7f0 con 0x7fe4fc15f920 2024-04-25T10:06:00.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe4f9d9b640 1 --2- 172.21.15.60:0/120609709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4fc15f920 0x7fe4fc06ac10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009870 tx=0x7fe4f001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559057b0989a08e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe4f959a640 1 -- 172.21.15.60:0/120609709 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f00ab650 con 0x7fe4fc15f920 2024-04-25T10:06:00.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe4f959a640 1 -- 172.21.15.60:0/120609709 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f00b0070 con 0x7fe4fc15f920 2024-04-25T10:06:00.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe4f959a640 1 -- 172.21.15.60:0/120609709 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f00aba10 con 0x7fe4fc15f920 2024-04-25T10:06:00.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe500c82640 1 -- 172.21.15.60:0/120609709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4fc15f920 msgr2=0x7fe4fc06ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe500c82640 1 --2- 172.21.15.60:0/120609709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4fc15f920 0x7fe4fc06ac10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009870 tx=0x7fe4f001c560 comp rx=0 tx=0).stop 2024-04-25T10:06:00.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe500c82640 1 -- 172.21.15.60:0/120609709 shutdown_connections 2024-04-25T10:06:00.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe500c82640 1 --2- 172.21.15.60:0/120609709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4fc15f920 0x7fe4fc06ac10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe500c82640 1 --2- 172.21.15.60:0/120609709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4fc0b1f30 0x7fe4fc15f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe500c82640 1 -- 172.21.15.60:0/120609709 >> 172.21.15.60:0/120609709 conn(0x7fe4fc072900 msgr2=0x7fe4fc099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:00.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe500c82640 1 -- 172.21.15.60:0/120609709 shutdown_connections 2024-04-25T10:06:00.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe500c82640 1 -- 172.21.15.60:0/120609709 wait complete. 2024-04-25T10:06:00.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe500c82640 1 Processor -- start 2024-04-25T10:06:00.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe500c82640 1 -- start start 2024-04-25T10:06:00.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe500c82640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4fc06b150 0x7fe4fc156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe500c82640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4fc0b1f30 0x7fe4fc142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe500c82640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4fc15f920 0x7fe4fc1503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe500c82640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc168660 con 0x7fe4fc0b1f30 2024-04-25T10:06:00.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe500c82640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc1684e0 con 0x7fe4fc06b150 2024-04-25T10:06:00.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe500c82640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc1687e0 con 0x7fe4fc15f920 2024-04-25T10:06:00.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe4f9d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4fc0b1f30 0x7fe4fc142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe4f9d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4fc0b1f30 0x7fe4fc142fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45292/0 (socket says 172.21.15.60:45292) 2024-04-25T10:06:00.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.827+0000 7fe4f9d9b640 1 -- 172.21.15.60:0/3114537819 learned_addr learned my addr 172.21.15.60:0/3114537819 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:00.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe4fa59c640 1 --2- 172.21.15.60:0/3114537819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4fc06b150 0x7fe4fc156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe4f9d9b640 1 -- 172.21.15.60:0/3114537819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4fc15f920 msgr2=0x7fe4fc1503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe4f9d9b640 1 --2- 172.21.15.60:0/3114537819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4fc15f920 0x7fe4fc1503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe4f9d9b640 1 -- 172.21.15.60:0/3114537819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4fc06b150 msgr2=0x7fe4fc156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe4f9d9b640 1 --2- 172.21.15.60:0/3114537819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4fc06b150 0x7fe4fc156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe4f9d9b640 1 -- 172.21.15.60:0/3114537819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc0ae670 con 0x7fe4fc0b1f30 2024-04-25T10:06:00.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe4fa59c640 1 --2- 172.21.15.60:0/3114537819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4fc06b150 0x7fe4fc156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:00.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe4f9d9b640 1 --2- 172.21.15.60:0/3114537819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4fc0b1f30 0x7fe4fc142fc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00ad040 tx=0x7fe4f001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe4eaffd640 1 -- 172.21.15.60:0/3114537819 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f00b4800 con 0x7fe4fc0b1f30 2024-04-25T10:06:00.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe4eaffd640 1 -- 172.21.15.60:0/3114537819 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f00b49c0 con 0x7fe4fc0b1f30 2024-04-25T10:06:00.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe500c82640 1 -- 172.21.15.60:0/3114537819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4fc0b5480 con 0x7fe4fc0b1f30 2024-04-25T10:06:00.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe4eaffd640 1 -- 172.21.15.60:0/3114537819 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f00b4db0 con 0x7fe4fc0b1f30 2024-04-25T10:06:00.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe500c82640 1 -- 172.21.15.60:0/3114537819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc158d10 con 0x7fe4fc0b1f30 2024-04-25T10:06:00.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe500c82640 1 -- 172.21.15.60:0/3114537819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4fc06ccb0 con 0x7fe4fc0b1f30 2024-04-25T10:06:00.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe4eaffd640 1 -- 172.21.15.60:0/3114537819 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe4f00c7390 con 0x7fe4fc0b1f30 2024-04-25T10:06:00.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe4eaffd640 1 --2- 172.21.15.60:0/3114537819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe4dc066f90 0x7fe4dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe4eaffd640 1 -- 172.21.15.60:0/3114537819 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fe4f011fa90 con 0x7fe4fc0b1f30 2024-04-25T10:06:00.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe4fa59c640 1 --2- 172.21.15.60:0/3114537819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe4dc066f90 0x7fe4dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.831+0000 7fe4fa59c640 1 --2- 172.21.15.60:0/3114537819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe4dc066f90 0x7fe4dc069450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec007eb0 tx=0x7fe4ec0073c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.835+0000 7fe4eaffd640 1 -- 172.21.15.60:0/3114537819 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe4f00c4060 con 0x7fe4fc0b1f30 2024-04-25T10:06:01.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.995+0000 7fe500c82640 1 -- 172.21.15.60:0/3114537819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe4fc074770 con 0x7fe4fc0b1f30 2024-04-25T10:06:01.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.995+0000 7fe4eaffd640 1 -- 172.21.15.60:0/3114537819 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe4f0016150 con 0x7fe4fc0b1f30 2024-04-25T10:06:01.001 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:06:01.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.995+0000 7fe500c82640 1 -- 172.21.15.60:0/3114537819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe4dc066f90 msgr2=0x7fe4dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.995+0000 7fe500c82640 1 --2- 172.21.15.60:0/3114537819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe4dc066f90 0x7fe4dc069450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec007eb0 tx=0x7fe4ec0073c0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.995+0000 7fe500c82640 1 -- 172.21.15.60:0/3114537819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4fc0b1f30 msgr2=0x7fe4fc142fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.995+0000 7fe500c82640 1 --2- 172.21.15.60:0/3114537819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4fc0b1f30 0x7fe4fc142fc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00ad040 tx=0x7fe4f001ada0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.995+0000 7fe500c82640 1 -- 172.21.15.60:0/3114537819 shutdown_connections 2024-04-25T10:06:01.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.995+0000 7fe500c82640 1 --2- 172.21.15.60:0/3114537819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe4dc066f90 0x7fe4dc069450 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec007eb0 tx=0x7fe4ec0073c0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.995+0000 7fe500c82640 1 --2- 172.21.15.60:0/3114537819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4fc15f920 0x7fe4fc1503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.995+0000 7fe500c82640 1 --2- 172.21.15.60:0/3114537819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4fc0b1f30 0x7fe4fc142fc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.995+0000 7fe500c82640 1 --2- 172.21.15.60:0/3114537819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4fc06b150 0x7fe4fc156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.995+0000 7fe500c82640 1 -- 172.21.15.60:0/3114537819 >> 172.21.15.60:0/3114537819 conn(0x7fe4fc072900 msgr2=0x7fe4fc06f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:01.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.995+0000 7fe500c82640 1 -- 172.21.15.60:0/3114537819 shutdown_connections 2024-04-25T10:06:01.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:00.995+0000 7fe500c82640 1 -- 172.21.15.60:0/3114537819 wait complete. 2024-04-25T10:06:01.020 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:01.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc50730a640 1 Processor -- start 2024-04-25T10:06:01.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc50730a640 1 -- start start 2024-04-25T10:06:01.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc50730a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc50015dd30 0x7fc50015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc50730a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5000cd890 con 0x7fc5000cf8a0 2024-04-25T10:06:01.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc50730a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5000cdb70 con 0x7fc5000cd3d0 2024-04-25T10:06:01.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc50730a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5000ce4a0 con 0x7fc50015dd30 2024-04-25T10:06:01.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc5058a7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc5000cf8a0 0x7fc50015be20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48846/0 (socket says 172.21.15.60:48846) 2024-04-25T10:06:01.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc5058a7640 1 -- 172.21.15.60:0/3139923647 learned_addr learned my addr 172.21.15.60:0/3139923647 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:01.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc5050a6640 1 --2- 172.21.15.60:0/3139923647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc50015dd30 0x7fc50015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc4f7fff640 1 -- 172.21.15.60:0/3139923647 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065947227 0 0) 0x7fc5000cd890 con 0x7fc5000cf8a0 2024-04-25T10:06:01.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc4f7fff640 1 -- 172.21.15.60:0/3139923647 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4ec003530 con 0x7fc5000cf8a0 2024-04-25T10:06:01.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc4f7fff640 1 -- 172.21.15.60:0/3139923647 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1745053289 0 0) 0x7fc4ec003530 con 0x7fc5000cf8a0 2024-04-25T10:06:01.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc4f7fff640 1 -- 172.21.15.60:0/3139923647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc50015dd30 msgr2=0x7fc50015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc4f7fff640 1 --2- 172.21.15.60:0/3139923647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc50015dd30 0x7fc50015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc4f7fff640 1 -- 172.21.15.60:0/3139923647 >> v1:172.21.15.100:6789/0 conn(0x7fc5000cd3d0 legacy=0x7fc5000cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc4f7fff640 1 -- 172.21.15.60:0/3139923647 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5000cde50 con 0x7fc5000cf8a0 2024-04-25T10:06:01.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc4f7fff640 1 -- 172.21.15.60:0/3139923647 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fc4f8001680 con 0x7fc5000cf8a0 2024-04-25T10:06:01.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc4f7fff640 1 -- 172.21.15.60:0/3139923647 >> v1:172.21.15.60:6789/0 conn(0x7fc5000cf8a0 legacy=0x7fc50015be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc4f7fff640 1 --2- 172.21.15.60:0/3139923647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc50015dd30 0x7fc4ec0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc4f7fff640 1 --2- 172.21.15.60:0/3139923647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4ec003f70 0x7fc4ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc4f7fff640 1 --2- 172.21.15.60:0/3139923647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4ec004d50 0x7fc4ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc4f7fff640 1 -- 172.21.15.60:0/3139923647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5000ce4a0 con 0x7fc4ec004d50 2024-04-25T10:06:01.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc4f7fff640 1 -- 172.21.15.60:0/3139923647 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec00bb60 con 0x7fc4ec003f70 2024-04-25T10:06:01.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc4f7fff640 1 -- 172.21.15.60:0/3139923647 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec00bd60 con 0x7fc50015dd30 2024-04-25T10:06:01.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc5058a7640 1 --2- 172.21.15.60:0/3139923647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4ec004d50 0x7fc4ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc5050a6640 1 --2- 172.21.15.60:0/3139923647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc50015dd30 0x7fc4ec0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc5048a5640 1 --2- 172.21.15.60:0/3139923647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4ec003f70 0x7fc4ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc5058a7640 1 -- 172.21.15.60:0/3139923647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc50015dd30 msgr2=0x7fc4ec0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc5058a7640 1 --2- 172.21.15.60:0/3139923647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc50015dd30 0x7fc4ec0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc5058a7640 1 -- 172.21.15.60:0/3139923647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4ec003f70 msgr2=0x7fc4ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc5058a7640 1 --2- 172.21.15.60:0/3139923647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4ec003f70 0x7fc4ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.135+0000 7fc5058a7640 1 -- 172.21.15.60:0/3139923647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec003530 con 0x7fc4ec004d50 2024-04-25T10:06:01.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.139+0000 7fc5058a7640 1 --2- 172.21.15.60:0/3139923647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4ec004d50 0x7fc4ec0051d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800c1c0 tx=0x7fc4f801f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782791577ad18d49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.139+0000 7fc4f7fff640 1 -- 172.21.15.60:0/3139923647 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8007d00 con 0x7fc4ec004d50 2024-04-25T10:06:01.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.139+0000 7fc4f7fff640 1 -- 172.21.15.60:0/3139923647 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f8005ae0 con 0x7fc4ec004d50 2024-04-25T10:06:01.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.139+0000 7fc4f7fff640 1 -- 172.21.15.60:0/3139923647 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8007070 con 0x7fc4ec004d50 2024-04-25T10:06:01.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.139+0000 7fc50730a640 1 -- 172.21.15.60:0/3139923647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4ec004d50 msgr2=0x7fc4ec0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.139+0000 7fc50730a640 1 --2- 172.21.15.60:0/3139923647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4ec004d50 0x7fc4ec0051d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800c1c0 tx=0x7fc4f801f3d0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.139+0000 7fc5058a7640 1 -- 172.21.15.60:0/3139923647 reap_dead start 2024-04-25T10:06:01.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.139+0000 7fc50730a640 1 -- 172.21.15.60:0/3139923647 shutdown_connections 2024-04-25T10:06:01.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.139+0000 7fc50730a640 1 -- 172.21.15.60:0/3139923647 >> 172.21.15.60:0/3139923647 conn(0x7fc5000670b0 msgr2=0x7fc50006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:01.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.139+0000 7fc50730a640 1 -- 172.21.15.60:0/3139923647 shutdown_connections 2024-04-25T10:06:01.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.139+0000 7fc50730a640 1 -- 172.21.15.60:0/3139923647 wait complete. 2024-04-25T10:06:01.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.139+0000 7fc50730a640 1 Processor -- start 2024-04-25T10:06:01.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.139+0000 7fc50730a640 1 -- start start 2024-04-25T10:06:01.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc50730a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4ec004d50 0x7fc5001588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc50730a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc500157370 0x7fc50014a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc50730a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5000ae7b0 0x7fc5001e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc50730a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50006f490 con 0x7fc5000ae7b0 2024-04-25T10:06:01.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc50730a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50006f310 con 0x7fc4ec004d50 2024-04-25T10:06:01.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc50730a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50006f610 con 0x7fc500157370 2024-04-25T10:06:01.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc5058a7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5000ae7b0 0x7fc5001e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc5058a7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5000ae7b0 0x7fc5001e3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45316/0 (socket says 172.21.15.60:45316) 2024-04-25T10:06:01.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc5058a7640 1 -- 172.21.15.60:0/1486966667 learned_addr learned my addr 172.21.15.60:0/1486966667 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:01.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc5048a5640 1 --2- 172.21.15.60:0/1486966667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc500157370 0x7fc50014a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc5058a7640 1 -- 172.21.15.60:0/1486966667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc500157370 msgr2=0x7fc50014a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc5050a6640 1 --2- 172.21.15.60:0/1486966667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4ec004d50 0x7fc5001588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc5058a7640 1 --2- 172.21.15.60:0/1486966667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc500157370 0x7fc50014a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc5058a7640 1 -- 172.21.15.60:0/1486966667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4ec004d50 msgr2=0x7fc5001588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc5058a7640 1 --2- 172.21.15.60:0/1486966667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4ec004d50 0x7fc5001588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc5058a7640 1 -- 172.21.15.60:0/1486966667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc500157810 con 0x7fc5000ae7b0 2024-04-25T10:06:01.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc5058a7640 1 --2- 172.21.15.60:0/1486966667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5000ae7b0 0x7fc5001e3f10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc4f801fc70 tx=0x7fc4ec005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc4f5ffb640 1 -- 172.21.15.60:0/1486966667 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8001c40 con 0x7fc5000ae7b0 2024-04-25T10:06:01.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc4f5ffb640 1 -- 172.21.15.60:0/1486966667 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f8005c90 con 0x7fc5000ae7b0 2024-04-25T10:06:01.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc50730a640 1 -- 172.21.15.60:0/1486966667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc500157af0 con 0x7fc5000ae7b0 2024-04-25T10:06:01.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc4f5ffb640 1 -- 172.21.15.60:0/1486966667 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f80049f0 con 0x7fc5000ae7b0 2024-04-25T10:06:01.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc50730a640 1 -- 172.21.15.60:0/1486966667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5001580b0 con 0x7fc5000ae7b0 2024-04-25T10:06:01.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc50730a640 1 -- 172.21.15.60:0/1486966667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc500074770 con 0x7fc5000ae7b0 2024-04-25T10:06:01.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc4f5ffb640 1 -- 172.21.15.60:0/1486966667 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4f80ba030 con 0x7fc5000ae7b0 2024-04-25T10:06:01.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc4f5ffb640 1 --2- 172.21.15.60:0/1486966667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4e4066f60 0x7fc4e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc4f5ffb640 1 -- 172.21.15.60:0/1486966667 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fc4f8121930 con 0x7fc5000ae7b0 2024-04-25T10:06:01.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.143+0000 7fc5050a6640 1 --2- 172.21.15.60:0/1486966667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4e4066f60 0x7fc4e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.147+0000 7fc5050a6640 1 --2- 172.21.15.60:0/1486966667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4e4066f60 0x7fc4e4069420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc007380 tx=0x7fc4fc00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.147+0000 7fc4f5ffb640 1 -- 172.21.15.60:0/1486966667 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4f80190e0 con 0x7fc5000ae7b0 2024-04-25T10:06:01.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.307+0000 7fc50730a640 1 -- 172.21.15.60:0/1486966667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc50015be00 con 0x7fc5000ae7b0 2024-04-25T10:06:01.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.307+0000 7fc4f5ffb640 1 -- 172.21.15.60:0/1486966667 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc4f80f2940 con 0x7fc5000ae7b0 2024-04-25T10:06:01.315 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:06:01.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.311+0000 7fc50730a640 1 -- 172.21.15.60:0/1486966667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4e4066f60 msgr2=0x7fc4e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.311+0000 7fc50730a640 1 --2- 172.21.15.60:0/1486966667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4e4066f60 0x7fc4e4069420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc007380 tx=0x7fc4fc00b040 comp rx=0 tx=0).stop 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.311+0000 7fc50730a640 1 -- 172.21.15.60:0/1486966667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5000ae7b0 msgr2=0x7fc5001e3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.311+0000 7fc50730a640 1 --2- 172.21.15.60:0/1486966667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5000ae7b0 0x7fc5001e3f10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc4f801fc70 tx=0x7fc4ec005840 comp rx=0 tx=0).stop 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.311+0000 7fc50730a640 1 -- 172.21.15.60:0/1486966667 shutdown_connections 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.311+0000 7fc50730a640 1 --2- 172.21.15.60:0/1486966667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4e4066f60 0x7fc4e4069420 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.311+0000 7fc50730a640 1 --2- 172.21.15.60:0/1486966667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5000ae7b0 0x7fc5001e3f10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.311+0000 7fc50730a640 1 --2- 172.21.15.60:0/1486966667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc500157370 0x7fc50014a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.311+0000 7fc50730a640 1 --2- 172.21.15.60:0/1486966667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4ec004d50 0x7fc5001588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.311+0000 7fc50730a640 1 -- 172.21.15.60:0/1486966667 >> 172.21.15.60:0/1486966667 conn(0x7fc5000670b0 msgr2=0x7fc50015cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.311+0000 7fc50730a640 1 -- 172.21.15.60:0/1486966667 shutdown_connections 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.311+0000 7fc50730a640 1 -- 172.21.15.60:0/1486966667 wait complete. 2024-04-25T10:06:01.335 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:06:01.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.447+0000 7f7dd503a640 1 Processor -- start 2024-04-25T10:06:01.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.447+0000 7f7dd503a640 1 -- start start 2024-04-25T10:06:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.447+0000 7f7dd503a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7dd015a7a0 0x7f7dd00cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.447+0000 7f7dd503a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7dd006ec30 0x7f7dd0071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.447+0000 7f7dd503a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd00721e0 con 0x7f7dd00b37e0 2024-04-25T10:06:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.447+0000 7f7dd503a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd00cdaa0 con 0x7f7dd015a7a0 2024-04-25T10:06:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.447+0000 7f7dd503a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd006d2f0 con 0x7f7dd006ec30 2024-04-25T10:06:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.447+0000 7f7dced9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7dd015a7a0 0x7f7dd00cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.447+0000 7f7dced9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7dd015a7a0 0x7f7dd00cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45326/0 (socket says 172.21.15.60:45326) 2024-04-25T10:06:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.447+0000 7f7dced9d640 1 -- 172.21.15.60:0/245939002 learned_addr learned my addr 172.21.15.60:0/245939002 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dce59c640 1 --2- 172.21.15.60:0/245939002 >> v2:172.21.15.100:3300/0 conn(0x7f7dd006ec30 0x7f7dd0071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dced9d640 1 -- 172.21.15.60:0/245939002 >> v2:172.21.15.100:3300/0 conn(0x7f7dd006ec30 msgr2=0x7f7dd0071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dced9d640 1 --2- 172.21.15.60:0/245939002 >> v2:172.21.15.100:3300/0 conn(0x7f7dd006ec30 0x7f7dd0071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dced9d640 1 -- 172.21.15.60:0/245939002 >> v1:172.21.15.60:6789/0 conn(0x7f7dd00b37e0 legacy=0x7f7dd006dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dced9d640 1 -- 172.21.15.60:0/245939002 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd006f130 con 0x7f7dd015a7a0 2024-04-25T10:06:01.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dced9d640 1 --2- 172.21.15.60:0/245939002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dd015a7a0 0x7f7dd00cd560 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0072bf0 tx=0x7f7dc001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22093ee711dd09fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dcdd9b640 1 -- 172.21.15.60:0/245939002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00ab430 con 0x7f7dd015a7a0 2024-04-25T10:06:01.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dcdd9b640 1 -- 172.21.15.60:0/245939002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc00b0070 con 0x7f7dd015a7a0 2024-04-25T10:06:01.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dcdd9b640 1 -- 172.21.15.60:0/245939002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00ab7f0 con 0x7f7dd015a7a0 2024-04-25T10:06:01.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dd503a640 1 -- 172.21.15.60:0/245939002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dd015a7a0 msgr2=0x7f7dd00cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dd503a640 1 --2- 172.21.15.60:0/245939002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dd015a7a0 0x7f7dd00cd560 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0072bf0 tx=0x7f7dc001c520 comp rx=0 tx=0).stop 2024-04-25T10:06:01.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dd503a640 1 -- 172.21.15.60:0/245939002 shutdown_connections 2024-04-25T10:06:01.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dd503a640 1 --2- 172.21.15.60:0/245939002 >> v2:172.21.15.100:3300/0 conn(0x7f7dd006ec30 0x7f7dd0071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dd503a640 1 --2- 172.21.15.60:0/245939002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dd015a7a0 0x7f7dd00cd560 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dd503a640 1 -- 172.21.15.60:0/245939002 >> 172.21.15.60:0/245939002 conn(0x7f7dd00b12d0 msgr2=0x7f7dd00b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dd503a640 1 -- 172.21.15.60:0/245939002 shutdown_connections 2024-04-25T10:06:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.451+0000 7f7dd503a640 1 -- 172.21.15.60:0/245939002 wait complete. 2024-04-25T10:06:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dd503a640 1 Processor -- start 2024-04-25T10:06:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dd503a640 1 -- start start 2024-04-25T10:06:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dd503a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dd006ec30 0x7f7dd014ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dd503a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dd00b37e0 0x7f7dd0152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dd503a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dd015a7a0 0x7f7dd015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dd503a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd015ad00 con 0x7f7dd006ec30 2024-04-25T10:06:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dd503a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd015ab80 con 0x7f7dd015a7a0 2024-04-25T10:06:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dd503a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd006e860 con 0x7f7dd00b37e0 2024-04-25T10:06:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dced9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dd006ec30 0x7f7dd014ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dced9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dd006ec30 0x7f7dd014ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45340/0 (socket says 172.21.15.60:45340) 2024-04-25T10:06:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dced9d640 1 -- 172.21.15.60:0/2867587063 learned_addr learned my addr 172.21.15.60:0/2867587063 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dce59c640 1 --2- 172.21.15.60:0/2867587063 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dd00b37e0 0x7f7dd0152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dcf59e640 1 --2- 172.21.15.60:0/2867587063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dd015a7a0 0x7f7dd015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dced9d640 1 -- 172.21.15.60:0/2867587063 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dd00b37e0 msgr2=0x7f7dd0152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dced9d640 1 --2- 172.21.15.60:0/2867587063 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dd00b37e0 0x7f7dd0152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dced9d640 1 -- 172.21.15.60:0/2867587063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dd015a7a0 msgr2=0x7f7dd015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dced9d640 1 --2- 172.21.15.60:0/2867587063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dd015a7a0 0x7f7dd015b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dced9d640 1 -- 172.21.15.60:0/2867587063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd015c010 con 0x7f7dd006ec30 2024-04-25T10:06:01.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dcf59e640 1 --2- 172.21.15.60:0/2867587063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dd015a7a0 0x7f7dd015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:01.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dce59c640 1 --2- 172.21.15.60:0/2867587063 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dd00b37e0 0x7f7dd0152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:01.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dced9d640 1 --2- 172.21.15.60:0/2867587063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dd006ec30 0x7f7dd014ab60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00ad040 tx=0x7f7dc001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7db77fe640 1 -- 172.21.15.60:0/2867587063 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00be5f0 con 0x7f7dd006ec30 2024-04-25T10:06:01.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7db77fe640 1 -- 172.21.15.60:0/2867587063 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc00be7b0 con 0x7f7dd006ec30 2024-04-25T10:06:01.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7db77fe640 1 -- 172.21.15.60:0/2867587063 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00beba0 con 0x7f7dd006ec30 2024-04-25T10:06:01.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dd503a640 1 -- 172.21.15.60:0/2867587063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd015c2f0 con 0x7f7dd006ec30 2024-04-25T10:06:01.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dd503a640 1 -- 172.21.15.60:0/2867587063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd014b0a0 con 0x7f7dd006ec30 2024-04-25T10:06:01.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7dd503a640 1 -- 172.21.15.60:0/2867587063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd0074770 con 0x7f7dd006ec30 2024-04-25T10:06:01.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7db77fe640 1 -- 172.21.15.60:0/2867587063 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7dc00b43f0 con 0x7f7dd006ec30 2024-04-25T10:06:01.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.455+0000 7f7db77fe640 1 --2- 172.21.15.60:0/2867587063 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7dbc066f60 0x7f7dbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.459+0000 7f7db77fe640 1 -- 172.21.15.60:0/2867587063 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7dc00a8030 con 0x7f7dd006ec30 2024-04-25T10:06:01.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.459+0000 7f7dce59c640 1 --2- 172.21.15.60:0/2867587063 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7dbc066f60 0x7f7dbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.459+0000 7f7dce59c640 1 --2- 172.21.15.60:0/2867587063 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7dbc066f60 0x7f7dbc069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0152420 tx=0x7f7dc4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.463+0000 7f7db77fe640 1 -- 172.21.15.60:0/2867587063 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7dc0016110 con 0x7f7dd006ec30 2024-04-25T10:06:01.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.635+0000 7f7dd503a640 1 -- 172.21.15.60:0/2867587063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7dd0165040 con 0x7f7dd006ec30 2024-04-25T10:06:01.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.639+0000 7f7db77fe640 1 -- 172.21.15.60:0/2867587063 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7dc00f0810 con 0x7f7dd006ec30 2024-04-25T10:06:01.644 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:06:01.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.639+0000 7f7dd503a640 1 -- 172.21.15.60:0/2867587063 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7dbc066f60 msgr2=0x7f7dbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.639+0000 7f7dd503a640 1 --2- 172.21.15.60:0/2867587063 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7dbc066f60 0x7f7dbc069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0152420 tx=0x7f7dc4008040 comp rx=0 tx=0).stop 2024-04-25T10:06:01.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.639+0000 7f7dd503a640 1 -- 172.21.15.60:0/2867587063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dd006ec30 msgr2=0x7f7dd014ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.639+0000 7f7dd503a640 1 --2- 172.21.15.60:0/2867587063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dd006ec30 0x7f7dd014ab60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00ad040 tx=0x7f7dc001ad60 comp rx=0 tx=0).stop 2024-04-25T10:06:01.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.639+0000 7f7dd503a640 1 -- 172.21.15.60:0/2867587063 shutdown_connections 2024-04-25T10:06:01.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.639+0000 7f7dd503a640 1 --2- 172.21.15.60:0/2867587063 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7dbc066f60 0x7f7dbc069420 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.639+0000 7f7dd503a640 1 --2- 172.21.15.60:0/2867587063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dd015a7a0 0x7f7dd015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.639+0000 7f7dd503a640 1 --2- 172.21.15.60:0/2867587063 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dd00b37e0 0x7f7dd0152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.639+0000 7f7dd503a640 1 --2- 172.21.15.60:0/2867587063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dd006ec30 0x7f7dd014ab60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.639+0000 7f7dd503a640 1 -- 172.21.15.60:0/2867587063 >> 172.21.15.60:0/2867587063 conn(0x7f7dd00b12d0 msgr2=0x7f7dd015cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:01.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.639+0000 7f7dd503a640 1 -- 172.21.15.60:0/2867587063 shutdown_connections 2024-04-25T10:06:01.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.639+0000 7f7dd503a640 1 -- 172.21.15.60:0/2867587063 wait complete. 2024-04-25T10:06:01.662 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-04-25T10:06:01.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:01.662 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T10:06:01.690 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:01.691 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:06:01.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:01.691 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:06:01.743 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:06:01.743 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:01.749 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:06:01.749 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:01.797 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.959+0000 7f325e6f2640 1 Processor -- start 2024-04-25T10:06:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.959+0000 7f325e6f2640 1 -- start start 2024-04-25T10:06:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f325e6f2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f32580cdc30 0x7f325806ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f325e6f2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32580cfae0 0x7f3258099a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f325e6f2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32580d03b0 con 0x7f32580cff00 2024-04-25T10:06:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f325e6f2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32580d0690 con 0x7f32580cdc30 2024-04-25T10:06:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f325e6f2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32580d0970 con 0x7f32580cfae0 2024-04-25T10:06:01.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f325cc8f640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f32580cff00 0x7f3258161720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48870/0 (socket says 172.21.15.60:48870) 2024-04-25T10:06:01.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f325cc8f640 1 -- 172.21.15.60:0/2454490560 learned_addr learned my addr 172.21.15.60:0/2454490560 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:01.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3257fff640 1 --2- 172.21.15.60:0/2454490560 >> v2:172.21.15.100:3300/0 conn(0x7f32580cdc30 0x7f325806ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f32577fe640 1 --2- 172.21.15.60:0/2454490560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32580cfae0 0x7f3258099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3256ffd640 1 -- 172.21.15.60:0/2454490560 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915398628 0 0) 0x7f32580d03b0 con 0x7f32580cff00 2024-04-25T10:06:01.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3256ffd640 1 -- 172.21.15.60:0/2454490560 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3244003530 con 0x7f32580cff00 2024-04-25T10:06:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3256ffd640 1 -- 172.21.15.60:0/2454490560 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2755374380 0 0) 0x7f3244003530 con 0x7f32580cff00 2024-04-25T10:06:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3256ffd640 1 -- 172.21.15.60:0/2454490560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32580cfae0 msgr2=0x7f3258099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3256ffd640 1 --2- 172.21.15.60:0/2454490560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32580cfae0 0x7f3258099a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3256ffd640 1 -- 172.21.15.60:0/2454490560 >> v2:172.21.15.100:3300/0 conn(0x7f32580cdc30 msgr2=0x7f325806ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3256ffd640 1 --2- 172.21.15.60:0/2454490560 >> v2:172.21.15.100:3300/0 conn(0x7f32580cdc30 0x7f325806ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3256ffd640 1 -- 172.21.15.60:0/2454490560 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32580d0c50 con 0x7f32580cff00 2024-04-25T10:06:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3256ffd640 1 -- 172.21.15.60:0/2454490560 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f3250001a00 con 0x7f32580cff00 2024-04-25T10:06:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3256ffd640 1 -- 172.21.15.60:0/2454490560 >> v1:172.21.15.60:6789/0 conn(0x7f32580cff00 legacy=0x7f3258161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3256ffd640 1 --2- 172.21.15.60:0/2454490560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32440039c0 0x7f3244003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3256ffd640 1 --2- 172.21.15.60:0/2454490560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32580cfae0 0x7f3244004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3256ffd640 1 --2- 172.21.15.60:0/2454490560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3244004d50 0x7f32440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3256ffd640 1 -- 172.21.15.60:0/2454490560 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32580d0690 con 0x7f32440039c0 2024-04-25T10:06:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3256ffd640 1 -- 172.21.15.60:0/2454490560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32580d0970 con 0x7f3244004d50 2024-04-25T10:06:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3256ffd640 1 -- 172.21.15.60:0/2454490560 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324400b9d0 con 0x7f32580cfae0 2024-04-25T10:06:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3257fff640 1 --2- 172.21.15.60:0/2454490560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32440039c0 0x7f3244003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f325cc8f640 1 --2- 172.21.15.60:0/2454490560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3244004d50 0x7f32440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f32577fe640 1 --2- 172.21.15.60:0/2454490560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32580cfae0 0x7f3244004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3257fff640 1 -- 172.21.15.60:0/2454490560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32580cfae0 msgr2=0x7f3244004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3257fff640 1 --2- 172.21.15.60:0/2454490560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32580cfae0 0x7f3244004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3257fff640 1 -- 172.21.15.60:0/2454490560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3244004d50 msgr2=0x7f32440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3257fff640 1 --2- 172.21.15.60:0/2454490560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3244004d50 0x7f32440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f3257fff640 1 -- 172.21.15.60:0/2454490560 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3244003530 con 0x7f32440039c0 2024-04-25T10:06:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.963+0000 7f32577fe640 1 --2- 172.21.15.60:0/2454490560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32580cfae0 0x7f3244004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f3257fff640 1 --2- 172.21.15.60:0/2454490560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32440039c0 0x7f3244003de0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3240009b10 tx=0x7f324001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f23c92b905119b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f3256ffd640 1 -- 172.21.15.60:0/2454490560 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32400096b0 con 0x7f32440039c0 2024-04-25T10:06:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f3256ffd640 1 -- 172.21.15.60:0/2454490560 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32400a8410 con 0x7f32440039c0 2024-04-25T10:06:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f3256ffd640 1 -- 172.21.15.60:0/2454490560 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32400a8800 con 0x7f32440039c0 2024-04-25T10:06:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f325e6f2640 1 -- 172.21.15.60:0/2454490560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32440039c0 msgr2=0x7f3244003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f325e6f2640 1 --2- 172.21.15.60:0/2454490560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32440039c0 0x7f3244003de0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3240009b10 tx=0x7f324001cab0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f325cc8f640 1 -- 172.21.15.60:0/2454490560 reap_dead start 2024-04-25T10:06:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f325e6f2640 1 -- 172.21.15.60:0/2454490560 shutdown_connections 2024-04-25T10:06:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f325e6f2640 1 -- 172.21.15.60:0/2454490560 >> 172.21.15.60:0/2454490560 conn(0x7f32580b32a0 msgr2=0x7f32580b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f325e6f2640 1 -- 172.21.15.60:0/2454490560 shutdown_connections 2024-04-25T10:06:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f325e6f2640 1 -- 172.21.15.60:0/2454490560 wait complete. 2024-04-25T10:06:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f325e6f2640 1 Processor -- start 2024-04-25T10:06:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f325e6f2640 1 -- start start 2024-04-25T10:06:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f325e6f2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32440039c0 0x7f32581554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f325e6f2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3258154820 0x7f32581541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f325e6f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32581571c0 0x7f3258157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f325e6f2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3258099dc0 con 0x7f32440039c0 2024-04-25T10:06:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f325e6f2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3258099c40 con 0x7f32581571c0 2024-04-25T10:06:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f325e6f2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3258099f40 con 0x7f3258154820 2024-04-25T10:06:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f3257fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32440039c0 0x7f32581554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f3257fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32440039c0 0x7f32581554c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45354/0 (socket says 172.21.15.60:45354) 2024-04-25T10:06:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f3257fff640 1 -- 172.21.15.60:0/266782508 learned_addr learned my addr 172.21.15.60:0/266782508 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f32577fe640 1 --2- 172.21.15.60:0/266782508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3258154820 0x7f32581541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f325cc8f640 1 --2- 172.21.15.60:0/266782508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32581571c0 0x7f3258157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f3257fff640 1 -- 172.21.15.60:0/266782508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3258154820 msgr2=0x7f32581541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f3257fff640 1 --2- 172.21.15.60:0/266782508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3258154820 0x7f32581541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f3257fff640 1 -- 172.21.15.60:0/266782508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32581571c0 msgr2=0x7f3258157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f3257fff640 1 --2- 172.21.15.60:0/266782508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32581571c0 0x7f3258157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.967+0000 7f3257fff640 1 -- 172.21.15.60:0/266782508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3258155aa0 con 0x7f32440039c0 2024-04-25T10:06:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.971+0000 7f3257fff640 1 --2- 172.21.15.60:0/266782508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32440039c0 0x7f32581554c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f324001c8e0 tx=0x7f3240019850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.971+0000 7f3254ff9640 1 -- 172.21.15.60:0/266782508 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32400a8c40 con 0x7f32440039c0 2024-04-25T10:06:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.971+0000 7f3254ff9640 1 -- 172.21.15.60:0/266782508 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32400a8de0 con 0x7f32440039c0 2024-04-25T10:06:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.971+0000 7f325e6f2640 1 -- 172.21.15.60:0/266782508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3258155d80 con 0x7f32440039c0 2024-04-25T10:06:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.971+0000 7f3254ff9640 1 -- 172.21.15.60:0/266782508 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32400bc670 con 0x7f32440039c0 2024-04-25T10:06:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.971+0000 7f325e6f2640 1 -- 172.21.15.60:0/266782508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3258156340 con 0x7f32440039c0 2024-04-25T10:06:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.971+0000 7f325e6f2640 1 -- 172.21.15.60:0/266782508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3258074770 con 0x7f32440039c0 2024-04-25T10:06:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.971+0000 7f3254ff9640 1 -- 172.21.15.60:0/266782508 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32400bc810 con 0x7f32440039c0 2024-04-25T10:06:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.971+0000 7f3254ff9640 1 --2- 172.21.15.60:0/266782508 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3238066f60 0x7f3238069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.971+0000 7f3254ff9640 1 -- 172.21.15.60:0/266782508 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f324011f450 con 0x7f32440039c0 2024-04-25T10:06:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.971+0000 7f32577fe640 1 --2- 172.21.15.60:0/266782508 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3238066f60 0x7f3238069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.971+0000 7f32577fe640 1 --2- 172.21.15.60:0/266782508 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3238066f60 0x7f3238069420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f32581566d0 tx=0x7f3248008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:01.975+0000 7f3254ff9640 1 -- 172.21.15.60:0/266782508 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f32400ac030 con 0x7f32440039c0 2024-04-25T10:06:02.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.135+0000 7f325e6f2640 1 -- 172.21.15.60:0/266782508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f325815f820 con 0x7f32440039c0 2024-04-25T10:06:02.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.135+0000 7f3254ff9640 1 -- 172.21.15.60:0/266782508 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16857 (secure 0 0 0) 0x7f3240016800 con 0x7f32440039c0 2024-04-25T10:06:02.144 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:02.144 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:01.568957+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:02.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.139+0000 7f325e6f2640 1 -- 172.21.15.60:0/266782508 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3238066f60 msgr2=0x7f3238069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.139+0000 7f325e6f2640 1 --2- 172.21.15.60:0/266782508 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3238066f60 0x7f3238069420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f32581566d0 tx=0x7f3248008040 comp rx=0 tx=0).stop 2024-04-25T10:06:02.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.139+0000 7f325e6f2640 1 -- 172.21.15.60:0/266782508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32440039c0 msgr2=0x7f32581554c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.139+0000 7f325e6f2640 1 --2- 172.21.15.60:0/266782508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32440039c0 0x7f32581554c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f324001c8e0 tx=0x7f3240019850 comp rx=0 tx=0).stop 2024-04-25T10:06:02.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.139+0000 7f325e6f2640 1 -- 172.21.15.60:0/266782508 shutdown_connections 2024-04-25T10:06:02.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.139+0000 7f325e6f2640 1 --2- 172.21.15.60:0/266782508 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3238066f60 0x7f3238069420 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.139+0000 7f325e6f2640 1 --2- 172.21.15.60:0/266782508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32581571c0 0x7f3258157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.139+0000 7f325e6f2640 1 --2- 172.21.15.60:0/266782508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3258154820 0x7f32581541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.139+0000 7f325e6f2640 1 --2- 172.21.15.60:0/266782508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32440039c0 0x7f32581554c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.139+0000 7f325e6f2640 1 -- 172.21.15.60:0/266782508 >> 172.21.15.60:0/266782508 conn(0x7f32580b32a0 msgr2=0x7f32580ce1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.139+0000 7f325e6f2640 1 -- 172.21.15.60:0/266782508 shutdown_connections 2024-04-25T10:06:02.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.139+0000 7f325e6f2640 1 -- 172.21.15.60:0/266782508 wait complete. 2024-04-25T10:06:02.164 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:18.976643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:19.405302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:02.164 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.287+0000 7f49e568d640 1 Processor -- start 2024-04-25T10:06:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.287+0000 7f49e568d640 1 -- start start 2024-04-25T10:06:02.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.287+0000 7f49e568d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f49e00b3f60 0x7f49e015a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.287+0000 7f49e568d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f49e0071250 0x7f49e00716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.287+0000 7f49e568d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e00b3510 con 0x7f49e006f030 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.287+0000 7f49e568d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e015ad30 con 0x7f49e0071250 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.287+0000 7f49e568d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e015b010 con 0x7f49e00b3f60 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.287+0000 7f49df7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f49e0071250 0x7f49e00716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.287+0000 7f49df7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f49e0071250 0x7f49e00716e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45364/0 (socket says 172.21.15.60:45364) 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.287+0000 7f49df7fe640 1 -- 172.21.15.60:0/4098783135 learned_addr learned my addr 172.21.15.60:0/4098783135 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.287+0000 7f49deffd640 1 --2- 172.21.15.60:0/4098783135 >> v2:172.21.15.100:3300/0 conn(0x7f49e00b3f60 0x7f49e015a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.287+0000 7f49df7fe640 1 -- 172.21.15.60:0/4098783135 >> v2:172.21.15.100:3300/0 conn(0x7f49e00b3f60 msgr2=0x7f49e015a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.287+0000 7f49df7fe640 1 --2- 172.21.15.60:0/4098783135 >> v2:172.21.15.100:3300/0 conn(0x7f49e00b3f60 0x7f49e015a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.287+0000 7f49df7fe640 1 -- 172.21.15.60:0/4098783135 >> v1:172.21.15.100:6789/0 conn(0x7f49e006f030 legacy=0x7f49e00b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.287+0000 7f49df7fe640 1 -- 172.21.15.60:0/4098783135 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e015b2f0 con 0x7f49e0071250 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.291+0000 7f49df7fe640 1 --2- 172.21.15.60:0/4098783135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e0071250 0x7f49e00716e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f49e00b36e0 tx=0x7f49d801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2549d41c0ee24da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.291+0000 7f49ddffb640 1 -- 172.21.15.60:0/4098783135 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d80a5560 con 0x7f49e0071250 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.291+0000 7f49ddffb640 1 -- 172.21.15.60:0/4098783135 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d80a5700 con 0x7f49e0071250 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.291+0000 7f49ddffb640 1 -- 172.21.15.60:0/4098783135 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d80a5ad0 con 0x7f49e0071250 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.291+0000 7f49e568d640 1 -- 172.21.15.60:0/4098783135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e0071250 msgr2=0x7f49e00716e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.291+0000 7f49e568d640 1 --2- 172.21.15.60:0/4098783135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e0071250 0x7f49e00716e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f49e00b36e0 tx=0x7f49d801c920 comp rx=0 tx=0).stop 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.291+0000 7f49e568d640 1 -- 172.21.15.60:0/4098783135 shutdown_connections 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.291+0000 7f49e568d640 1 --2- 172.21.15.60:0/4098783135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e0071250 0x7f49e00716e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.291+0000 7f49e568d640 1 --2- 172.21.15.60:0/4098783135 >> v2:172.21.15.100:3300/0 conn(0x7f49e00b3f60 0x7f49e015a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.291+0000 7f49e568d640 1 -- 172.21.15.60:0/4098783135 >> 172.21.15.60:0/4098783135 conn(0x7f49e00670b0 msgr2=0x7f49e006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.291+0000 7f49e568d640 1 -- 172.21.15.60:0/4098783135 shutdown_connections 2024-04-25T10:06:02.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.291+0000 7f49e568d640 1 -- 172.21.15.60:0/4098783135 wait complete. 2024-04-25T10:06:02.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.291+0000 7f49e568d640 1 Processor -- start 2024-04-25T10:06:02.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.291+0000 7f49e568d640 1 -- start start 2024-04-25T10:06:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49e568d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e006f030 0x7f49e0158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49e568d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e0071250 0x7f49e0153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49e568d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49e00b3f60 0x7f49e0153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49e568d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e015b720 con 0x7f49e0071250 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49e568d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e015b5a0 con 0x7f49e00b3f60 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49e568d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e015b8a0 con 0x7f49e006f030 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49de7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e0071250 0x7f49e0153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49de7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e0071250 0x7f49e0153e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45378/0 (socket says 172.21.15.60:45378) 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49de7fc640 1 -- 172.21.15.60:0/2137097437 learned_addr learned my addr 172.21.15.60:0/2137097437 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49df7fe640 1 --2- 172.21.15.60:0/2137097437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49e00b3f60 0x7f49e0153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49deffd640 1 --2- 172.21.15.60:0/2137097437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e006f030 0x7f49e0158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49de7fc640 1 -- 172.21.15.60:0/2137097437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e006f030 msgr2=0x7f49e0158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49de7fc640 1 --2- 172.21.15.60:0/2137097437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e006f030 0x7f49e0158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49de7fc640 1 -- 172.21.15.60:0/2137097437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49e00b3f60 msgr2=0x7f49e0153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49de7fc640 1 --2- 172.21.15.60:0/2137097437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49e00b3f60 0x7f49e0153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49de7fc640 1 -- 172.21.15.60:0/2137097437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e0159250 con 0x7f49e0071250 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49deffd640 1 --2- 172.21.15.60:0/2137097437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e006f030 0x7f49e0158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49df7fe640 1 --2- 172.21.15.60:0/2137097437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49e00b3f60 0x7f49e0153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49de7fc640 1 --2- 172.21.15.60:0/2137097437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e0071250 0x7f49e0153e80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f49d400c450 tx=0x7f49d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49c77fe640 1 -- 172.21.15.60:0/2137097437 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4012830 con 0x7f49e0071250 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49c77fe640 1 -- 172.21.15.60:0/2137097437 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d4017070 con 0x7f49e0071250 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49e568d640 1 -- 172.21.15.60:0/2137097437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e0157f50 con 0x7f49e0071250 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49c77fe640 1 -- 172.21.15.60:0/2137097437 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4012c30 con 0x7f49e0071250 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49e568d640 1 -- 172.21.15.60:0/2137097437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e01587f0 con 0x7f49e0071250 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49e568d640 1 -- 172.21.15.60:0/2137097437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e0158340 con 0x7f49e0071250 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49c77fe640 1 -- 172.21.15.60:0/2137097437 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49d401c460 con 0x7f49e0071250 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49c77fe640 1 --2- 172.21.15.60:0/2137097437 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f49cc066f60 0x7f49cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49c77fe640 1 -- 172.21.15.60:0/2137097437 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f49d4084e70 con 0x7f49e0071250 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.295+0000 7f49deffd640 1 --2- 172.21.15.60:0/2137097437 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f49cc066f60 0x7f49cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.299+0000 7f49deffd640 1 --2- 172.21.15.60:0/2137097437 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f49cc066f60 0x7f49cc069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f49d00097a0 tx=0x7f49d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.299+0000 7f49c77fe640 1 -- 172.21.15.60:0/2137097437 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f49d4055e80 con 0x7f49e0071250 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.463+0000 7f49e568d640 1 -- 172.21.15.60:0/2137097437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49e0074770 con 0x7f49e0071250 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.463+0000 7f49c77fe640 1 -- 172.21.15.60:0/2137097437 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49d4055780 con 0x7f49e0071250 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:06:02.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.463+0000 7f49e568d640 1 -- 172.21.15.60:0/2137097437 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f49cc066f60 msgr2=0x7f49cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.463+0000 7f49e568d640 1 --2- 172.21.15.60:0/2137097437 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f49cc066f60 0x7f49cc069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f49d00097a0 tx=0x7f49d0008040 comp rx=0 tx=0).stop 2024-04-25T10:06:02.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.463+0000 7f49e568d640 1 -- 172.21.15.60:0/2137097437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e0071250 msgr2=0x7f49e0153e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.463+0000 7f49e568d640 1 --2- 172.21.15.60:0/2137097437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e0071250 0x7f49e0153e80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f49d400c450 tx=0x7f49d400c910 comp rx=0 tx=0).stop 2024-04-25T10:06:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.463+0000 7f49e568d640 1 -- 172.21.15.60:0/2137097437 shutdown_connections 2024-04-25T10:06:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.463+0000 7f49e568d640 1 --2- 172.21.15.60:0/2137097437 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f49cc066f60 0x7f49cc069420 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.463+0000 7f49e568d640 1 --2- 172.21.15.60:0/2137097437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49e00b3f60 0x7f49e0153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.463+0000 7f49e568d640 1 --2- 172.21.15.60:0/2137097437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e0071250 0x7f49e0153e80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.463+0000 7f49e568d640 1 --2- 172.21.15.60:0/2137097437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e006f030 0x7f49e0158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.463+0000 7f49e568d640 1 -- 172.21.15.60:0/2137097437 >> 172.21.15.60:0/2137097437 conn(0x7f49e00670b0 msgr2=0x7f49e00af3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.463+0000 7f49e568d640 1 -- 172.21.15.60:0/2137097437 shutdown_connections 2024-04-25T10:06:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.463+0000 7f49e568d640 1 -- 172.21.15.60:0/2137097437 wait complete. 2024-04-25T10:06:02.488 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:02.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.599+0000 7f721060d640 1 Processor -- start 2024-04-25T10:06:02.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.599+0000 7f721060d640 1 -- start start 2024-04-25T10:06:02.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f721060d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7208160080 0x7f720815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f721060d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f720815fa20 0x7f72080ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f721060d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7208144a40 con 0x7f72080cbcf0 2024-04-25T10:06:02.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f721060d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720815fe90 con 0x7f7208160080 2024-04-25T10:06:02.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f721060d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72080b2dc0 con 0x7f720815fa20 2024-04-25T10:06:02.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f720e3a9640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f72080cbcf0 0x7f7208071c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48874/0 (socket says 172.21.15.60:48874) 2024-04-25T10:06:02.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f720dba8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7208160080 0x7f720815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f720e3a9640 1 -- 172.21.15.60:0/531361236 learned_addr learned my addr 172.21.15.60:0/531361236 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:02.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f720dba8640 1 -- 172.21.15.60:0/531361236 >> v2:172.21.15.100:3300/0 conn(0x7f720815fa20 msgr2=0x7f72080ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f720dba8640 1 --2- 172.21.15.60:0/531361236 >> v2:172.21.15.100:3300/0 conn(0x7f720815fa20 0x7f72080ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f720dba8640 1 -- 172.21.15.60:0/531361236 >> v1:172.21.15.60:6789/0 conn(0x7f72080cbcf0 legacy=0x7f7208071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f720dba8640 1 -- 172.21.15.60:0/531361236 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72080cc9f0 con 0x7f7208160080 2024-04-25T10:06:02.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f720d3a7640 1 -- 172.21.15.60:0/531361236 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957638713 0 0) 0x7f7208144a40 con 0x7f72080cbcf0 2024-04-25T10:06:02.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f720dba8640 1 --2- 172.21.15.60:0/531361236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7208160080 0x7f720815f4e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f72080d03a0 tx=0x7f71f801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8744d8be54d23737 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f720d3a7640 1 -- 172.21.15.60:0/531361236 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f80b0070 con 0x7f7208160080 2024-04-25T10:06:02.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f720d3a7640 1 -- 172.21.15.60:0/531361236 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f80ab600 con 0x7f7208160080 2024-04-25T10:06:02.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f720d3a7640 1 -- 172.21.15.60:0/531361236 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f80ab9d0 con 0x7f7208160080 2024-04-25T10:06:02.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f721060d640 1 -- 172.21.15.60:0/531361236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7208160080 msgr2=0x7f720815f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f721060d640 1 --2- 172.21.15.60:0/531361236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7208160080 0x7f720815f4e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f72080d03a0 tx=0x7f71f801c480 comp rx=0 tx=0).stop 2024-04-25T10:06:02.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f721060d640 1 -- 172.21.15.60:0/531361236 shutdown_connections 2024-04-25T10:06:02.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f721060d640 1 --2- 172.21.15.60:0/531361236 >> v2:172.21.15.100:3300/0 conn(0x7f720815fa20 0x7f72080ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f721060d640 1 --2- 172.21.15.60:0/531361236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7208160080 0x7f720815f4e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.603+0000 7f721060d640 1 -- 172.21.15.60:0/531361236 >> 172.21.15.60:0/531361236 conn(0x7f72080670b0 msgr2=0x7f720806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f721060d640 1 -- 172.21.15.60:0/531361236 shutdown_connections 2024-04-25T10:06:02.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f721060d640 1 -- 172.21.15.60:0/531361236 wait complete. 2024-04-25T10:06:02.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f721060d640 1 Processor -- start 2024-04-25T10:06:02.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f721060d640 1 -- start start 2024-04-25T10:06:02.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f721060d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72080cbcf0 0x7f72081614b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f721060d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f720815fa20 0x7f7208159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f721060d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208160080 0x7f720813f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f721060d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72080ce5c0 con 0x7f72080cbcf0 2024-04-25T10:06:02.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f721060d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72080cea30 con 0x7f7208160080 2024-04-25T10:06:02.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f721060d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72080ce740 con 0x7f720815fa20 2024-04-25T10:06:02.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f720e3a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72080cbcf0 0x7f72081614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f720e3a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72080cbcf0 0x7f72081614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45394/0 (socket says 172.21.15.60:45394) 2024-04-25T10:06:02.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f720e3a9640 1 -- 172.21.15.60:0/2998100120 learned_addr learned my addr 172.21.15.60:0/2998100120 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:02.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f720e3a9640 1 -- 172.21.15.60:0/2998100120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f720815fa20 msgr2=0x7f7208159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f720e3a9640 1 --2- 172.21.15.60:0/2998100120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f720815fa20 0x7f7208159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f720e3a9640 1 -- 172.21.15.60:0/2998100120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208160080 msgr2=0x7f720813f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f720ebaa640 1 --2- 172.21.15.60:0/2998100120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208160080 0x7f720813f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f720e3a9640 1 --2- 172.21.15.60:0/2998100120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208160080 0x7f720813f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f720e3a9640 1 -- 172.21.15.60:0/2998100120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720815a000 con 0x7f72080cbcf0 2024-04-25T10:06:02.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f720e3a9640 1 --2- 172.21.15.60:0/2998100120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72080cbcf0 0x7f72081614b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f72040011f0 tx=0x7f720400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f71feffd640 1 -- 172.21.15.60:0/2998100120 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204018070 con 0x7f72080cbcf0 2024-04-25T10:06:02.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f71feffd640 1 -- 172.21.15.60:0/2998100120 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72040074e0 con 0x7f72080cbcf0 2024-04-25T10:06:02.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f721060d640 1 -- 172.21.15.60:0/2998100120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720815a2e0 con 0x7f72080cbcf0 2024-04-25T10:06:02.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f71feffd640 1 -- 172.21.15.60:0/2998100120 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204004a10 con 0x7f72080cbcf0 2024-04-25T10:06:02.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f721060d640 1 -- 172.21.15.60:0/2998100120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720815a8a0 con 0x7f72080cbcf0 2024-04-25T10:06:02.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.607+0000 7f721060d640 1 -- 172.21.15.60:0/2998100120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7208074770 con 0x7f72080cbcf0 2024-04-25T10:06:02.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.611+0000 7f71feffd640 1 -- 172.21.15.60:0/2998100120 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f720401b070 con 0x7f72080cbcf0 2024-04-25T10:06:02.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.611+0000 7f71feffd640 1 --2- 172.21.15.60:0/2998100120 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f71ec066f90 0x7f71ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.611+0000 7f71feffd640 1 -- 172.21.15.60:0/2998100120 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7204084370 con 0x7f72080cbcf0 2024-04-25T10:06:02.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.611+0000 7f720dba8640 1 --2- 172.21.15.60:0/2998100120 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f71ec066f90 0x7f71ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.611+0000 7f720dba8640 1 --2- 172.21.15.60:0/2998100120 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f71ec066f90 0x7f71ec069450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f71f80ad000 tx=0x7f71f8004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.615+0000 7f71feffd640 1 -- 172.21.15.60:0/2998100120 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f72040552f0 con 0x7f72080cbcf0 2024-04-25T10:06:02.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.771+0000 7f721060d640 1 -- 172.21.15.60:0/2998100120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f720815ecc0 con 0x7f72080cbcf0 2024-04-25T10:06:02.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.775+0000 7f71feffd640 1 -- 172.21.15.60:0/2998100120 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7204054bf0 con 0x7f72080cbcf0 2024-04-25T10:06:02.780 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:06:02.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.779+0000 7f721060d640 1 -- 172.21.15.60:0/2998100120 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f71ec066f90 msgr2=0x7f71ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.779+0000 7f721060d640 1 --2- 172.21.15.60:0/2998100120 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f71ec066f90 0x7f71ec069450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f71f80ad000 tx=0x7f71f8004740 comp rx=0 tx=0).stop 2024-04-25T10:06:02.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.779+0000 7f721060d640 1 -- 172.21.15.60:0/2998100120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72080cbcf0 msgr2=0x7f72081614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.779+0000 7f721060d640 1 --2- 172.21.15.60:0/2998100120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72080cbcf0 0x7f72081614b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f72040011f0 tx=0x7f720400c9b0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.779+0000 7f721060d640 1 -- 172.21.15.60:0/2998100120 shutdown_connections 2024-04-25T10:06:02.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.779+0000 7f721060d640 1 --2- 172.21.15.60:0/2998100120 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f71ec066f90 0x7f71ec069450 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.779+0000 7f721060d640 1 --2- 172.21.15.60:0/2998100120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7208160080 0x7f720813f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.779+0000 7f721060d640 1 --2- 172.21.15.60:0/2998100120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f720815fa20 0x7f7208159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.779+0000 7f721060d640 1 --2- 172.21.15.60:0/2998100120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72080cbcf0 0x7f72081614b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.779+0000 7f721060d640 1 -- 172.21.15.60:0/2998100120 >> 172.21.15.60:0/2998100120 conn(0x7f72080670b0 msgr2=0x7f72080b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.779+0000 7f721060d640 1 -- 172.21.15.60:0/2998100120 shutdown_connections 2024-04-25T10:06:02.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.779+0000 7f721060d640 1 -- 172.21.15.60:0/2998100120 wait complete. 2024-04-25T10:06:02.801 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:02.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.923+0000 7fd40cae6640 1 Processor -- start 2024-04-25T10:06:02.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.923+0000 7fd40cae6640 1 -- start start 2024-04-25T10:06:02.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.923+0000 7fd40cae6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd40815dbf0 0x7fd408070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.923+0000 7fd40cae6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd408071210 0x7fd4080b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40cae6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4080b50a0 con 0x7fd4080b4c10 2024-04-25T10:06:02.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40cae6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4080b58e0 con 0x7fd40815dbf0 2024-04-25T10:06:02.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40cae6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd40815dfd0 con 0x7fd408071210 2024-04-25T10:06:02.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd40815dbf0 0x7fd408070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd40815dbf0 0x7fd408070cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45396/0 (socket says 172.21.15.60:45396) 2024-04-25T10:06:02.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40659c640 1 -- 172.21.15.60:0/2194311457 learned_addr learned my addr 172.21.15.60:0/2194311457 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:02.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd405d9b640 1 --2- 172.21.15.60:0/2194311457 >> v2:172.21.15.100:3300/0 conn(0x7fd408071210 0x7fd4080b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40659c640 1 -- 172.21.15.60:0/2194311457 >> v2:172.21.15.100:3300/0 conn(0x7fd408071210 msgr2=0x7fd4080b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40659c640 1 --2- 172.21.15.60:0/2194311457 >> v2:172.21.15.100:3300/0 conn(0x7fd408071210 0x7fd4080b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40659c640 1 -- 172.21.15.60:0/2194311457 >> v1:172.21.15.100:6789/0 conn(0x7fd4080b4c10 legacy=0x7fd4081616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40659c640 1 -- 172.21.15.60:0/2194311457 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4080b5320 con 0x7fd40815dbf0 2024-04-25T10:06:02.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd405d9b640 1 --2- 172.21.15.60:0/2194311457 >> v2:172.21.15.100:3300/0 conn(0x7fd408071210 0x7fd4080b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:02.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40659c640 1 --2- 172.21.15.60:0/2194311457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd40815dbf0 0x7fd408070cd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd40806f0c0 tx=0x7fd3f801c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a0f7b9cd4f40ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40559a640 1 -- 172.21.15.60:0/2194311457 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f80b0070 con 0x7fd40815dbf0 2024-04-25T10:06:02.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40559a640 1 -- 172.21.15.60:0/2194311457 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f80ab600 con 0x7fd40815dbf0 2024-04-25T10:06:02.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40559a640 1 -- 172.21.15.60:0/2194311457 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f80ab9d0 con 0x7fd40815dbf0 2024-04-25T10:06:02.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40cae6640 1 -- 172.21.15.60:0/2194311457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd40815dbf0 msgr2=0x7fd408070cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40cae6640 1 --2- 172.21.15.60:0/2194311457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd40815dbf0 0x7fd408070cd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd40806f0c0 tx=0x7fd3f801c300 comp rx=0 tx=0).stop 2024-04-25T10:06:02.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40cae6640 1 -- 172.21.15.60:0/2194311457 shutdown_connections 2024-04-25T10:06:02.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40cae6640 1 --2- 172.21.15.60:0/2194311457 >> v2:172.21.15.100:3300/0 conn(0x7fd408071210 0x7fd4080b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40cae6640 1 --2- 172.21.15.60:0/2194311457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd40815dbf0 0x7fd408070cd0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40cae6640 1 -- 172.21.15.60:0/2194311457 >> 172.21.15.60:0/2194311457 conn(0x7fd40806f530 msgr2=0x7fd40806f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40cae6640 1 -- 172.21.15.60:0/2194311457 shutdown_connections 2024-04-25T10:06:02.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.927+0000 7fd40cae6640 1 -- 172.21.15.60:0/2194311457 wait complete. 2024-04-25T10:06:02.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd40cae6640 1 Processor -- start 2024-04-25T10:06:02.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd40cae6640 1 -- start start 2024-04-25T10:06:02.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd40cae6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd408071210 0x7fd408160400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd40cae6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4080b4c10 0x7fd40814bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd40cae6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd40815dbf0 0x7fd408139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd40cae6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40816aa10 con 0x7fd40815dbf0 2024-04-25T10:06:02.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd40cae6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40806c180 con 0x7fd4080b4c10 2024-04-25T10:06:02.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd40cae6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40816ab90 con 0x7fd408071210 2024-04-25T10:06:02.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd406d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd40815dbf0 0x7fd408139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd406d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd40815dbf0 0x7fd408139cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45404/0 (socket says 172.21.15.60:45404) 2024-04-25T10:06:02.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd406d9d640 1 -- 172.21.15.60:0/4190872659 learned_addr learned my addr 172.21.15.60:0/4190872659 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:02.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd40659c640 1 --2- 172.21.15.60:0/4190872659 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd408071210 0x7fd408160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd405d9b640 1 --2- 172.21.15.60:0/4190872659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4080b4c10 0x7fd40814bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd406d9d640 1 -- 172.21.15.60:0/4190872659 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd408071210 msgr2=0x7fd408160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd406d9d640 1 --2- 172.21.15.60:0/4190872659 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd408071210 0x7fd408160400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd406d9d640 1 -- 172.21.15.60:0/4190872659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4080b4c10 msgr2=0x7fd40814bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd406d9d640 1 --2- 172.21.15.60:0/4190872659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4080b4c10 0x7fd40814bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd406d9d640 1 -- 172.21.15.60:0/4190872659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4080d04a0 con 0x7fd40815dbf0 2024-04-25T10:06:02.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd406d9d640 1 --2- 172.21.15.60:0/4190872659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd40815dbf0 0x7fd408139cc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd40000c780 tx=0x7fd40000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd3f6ffd640 1 -- 172.21.15.60:0/4190872659 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400004b70 con 0x7fd40815dbf0 2024-04-25T10:06:02.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd3f6ffd640 1 -- 172.21.15.60:0/4190872659 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd400004d10 con 0x7fd40815dbf0 2024-04-25T10:06:02.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd3f6ffd640 1 -- 172.21.15.60:0/4190872659 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40000aa90 con 0x7fd40815dbf0 2024-04-25T10:06:02.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd40cae6640 1 -- 172.21.15.60:0/4190872659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40814c730 con 0x7fd40815dbf0 2024-04-25T10:06:02.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd40cae6640 1 -- 172.21.15.60:0/4190872659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40814cfd0 con 0x7fd40815dbf0 2024-04-25T10:06:02.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.931+0000 7fd40cae6640 1 -- 172.21.15.60:0/4190872659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd408074770 con 0x7fd40815dbf0 2024-04-25T10:06:02.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.935+0000 7fd3f6ffd640 1 -- 172.21.15.60:0/4190872659 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd40000ac30 con 0x7fd40815dbf0 2024-04-25T10:06:02.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.935+0000 7fd3f6ffd640 1 --2- 172.21.15.60:0/4190872659 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd3e8066f60 0x7fd3e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.935+0000 7fd3f6ffd640 1 -- 172.21.15.60:0/4190872659 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd400085360 con 0x7fd40815dbf0 2024-04-25T10:06:02.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.935+0000 7fd40659c640 1 --2- 172.21.15.60:0/4190872659 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd3e8066f60 0x7fd3e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.935+0000 7fd40659c640 1 --2- 172.21.15.60:0/4190872659 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd3e8066f60 0x7fd3e8069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80ad000 tx=0x7fd3f80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:02.939+0000 7fd3f6ffd640 1 -- 172.21.15.60:0/4190872659 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd400020070 con 0x7fd40815dbf0 2024-04-25T10:06:03.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.099+0000 7fd40cae6640 1 -- 172.21.15.60:0/4190872659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd4080702d0 con 0x7fd40815dbf0 2024-04-25T10:06:03.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.099+0000 7fd3f6ffd640 1 -- 172.21.15.60:0/4190872659 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd4000562e0 con 0x7fd40815dbf0 2024-04-25T10:06:03.105 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:06:03.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.099+0000 7fd40cae6640 1 -- 172.21.15.60:0/4190872659 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd3e8066f60 msgr2=0x7fd3e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.099+0000 7fd40cae6640 1 --2- 172.21.15.60:0/4190872659 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd3e8066f60 0x7fd3e8069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80ad000 tx=0x7fd3f80a6000 comp rx=0 tx=0).stop 2024-04-25T10:06:03.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.099+0000 7fd40cae6640 1 -- 172.21.15.60:0/4190872659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd40815dbf0 msgr2=0x7fd408139cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.099+0000 7fd40cae6640 1 --2- 172.21.15.60:0/4190872659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd40815dbf0 0x7fd408139cc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd40000c780 tx=0x7fd40000cc40 comp rx=0 tx=0).stop 2024-04-25T10:06:03.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.099+0000 7fd40cae6640 1 -- 172.21.15.60:0/4190872659 shutdown_connections 2024-04-25T10:06:03.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.099+0000 7fd40cae6640 1 --2- 172.21.15.60:0/4190872659 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd3e8066f60 0x7fd3e8069420 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.099+0000 7fd40cae6640 1 --2- 172.21.15.60:0/4190872659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd40815dbf0 0x7fd408139cc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.099+0000 7fd40cae6640 1 --2- 172.21.15.60:0/4190872659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4080b4c10 0x7fd40814bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.099+0000 7fd40cae6640 1 --2- 172.21.15.60:0/4190872659 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd408071210 0x7fd408160400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.099+0000 7fd40cae6640 1 -- 172.21.15.60:0/4190872659 >> 172.21.15.60:0/4190872659 conn(0x7fd40806f530 msgr2=0x7fd4080b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.099+0000 7fd40cae6640 1 -- 172.21.15.60:0/4190872659 shutdown_connections 2024-04-25T10:06:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.099+0000 7fd40cae6640 1 -- 172.21.15.60:0/4190872659 wait complete. 2024-04-25T10:06:03.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:06:03.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf1a760640 1 Processor -- start 2024-04-25T10:06:03.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf1a760640 1 -- start start 2024-04-25T10:06:03.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf1a760640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf1415bc90 0x7fdf1415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf1a760640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf14071ca0 con 0x7fdf140b4660 2024-04-25T10:06:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf1a760640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf140cca40 con 0x7fdf140ce7a0 2024-04-25T10:06:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf1a760640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140cc370 con 0x7fdf1415bc90 2024-04-25T10:06:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf18cfd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdf140b4660 0x7fdf1415c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48886/0 (socket says 172.21.15.60:48886) 2024-04-25T10:06:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf18cfd640 1 -- 172.21.15.60:0/1339348289 learned_addr learned my addr 172.21.15.60:0/1339348289 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf137fe640 1 --2- 172.21.15.60:0/1339348289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf1415bc90 0x7fdf1415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf12ffd640 1 -- 172.21.15.60:0/1339348289 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3144096925 0 0) 0x7fdf14071ca0 con 0x7fdf140b4660 2024-04-25T10:06:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf12ffd640 1 -- 172.21.15.60:0/1339348289 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdefc003530 con 0x7fdf140b4660 2024-04-25T10:06:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf12ffd640 1 -- 172.21.15.60:0/1339348289 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2115487612 0 0) 0x7fdefc003530 con 0x7fdf140b4660 2024-04-25T10:06:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf12ffd640 1 -- 172.21.15.60:0/1339348289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf1415bc90 msgr2=0x7fdf1415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf12ffd640 1 --2- 172.21.15.60:0/1339348289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf1415bc90 0x7fdf1415c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf12ffd640 1 -- 172.21.15.60:0/1339348289 >> v1:172.21.15.100:6789/0 conn(0x7fdf140ce7a0 legacy=0x7fdf140ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf12ffd640 1 -- 172.21.15.60:0/1339348289 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf140ccd20 con 0x7fdf140b4660 2024-04-25T10:06:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf12ffd640 1 -- 172.21.15.60:0/1339348289 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fdf0c001a00 con 0x7fdf140b4660 2024-04-25T10:06:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf12ffd640 1 -- 172.21.15.60:0/1339348289 >> v1:172.21.15.60:6789/0 conn(0x7fdf140b4660 legacy=0x7fdf1415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf12ffd640 1 --2- 172.21.15.60:0/1339348289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdefc0039f0 0x7fdefc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf12ffd640 1 --2- 172.21.15.60:0/1339348289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf1415bc90 0x7fdefc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf12ffd640 1 --2- 172.21.15.60:0/1339348289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdefc004d50 0x7fdefc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf12ffd640 1 -- 172.21.15.60:0/1339348289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc00b9d0 con 0x7fdefc004d50 2024-04-25T10:06:03.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf12ffd640 1 -- 172.21.15.60:0/1339348289 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc00bb80 con 0x7fdefc0039f0 2024-04-25T10:06:03.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf12ffd640 1 -- 172.21.15.60:0/1339348289 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc00bd80 con 0x7fdf1415bc90 2024-04-25T10:06:03.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf18cfd640 1 --2- 172.21.15.60:0/1339348289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdefc004d50 0x7fdefc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.239+0000 7fdf13fff640 1 --2- 172.21.15.60:0/1339348289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdefc0039f0 0x7fdefc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf137fe640 1 --2- 172.21.15.60:0/1339348289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf1415bc90 0x7fdefc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf18cfd640 1 -- 172.21.15.60:0/1339348289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf1415bc90 msgr2=0x7fdefc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:03.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf18cfd640 1 --2- 172.21.15.60:0/1339348289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf1415bc90 0x7fdefc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf18cfd640 1 -- 172.21.15.60:0/1339348289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdefc0039f0 msgr2=0x7fdefc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:03.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf18cfd640 1 --2- 172.21.15.60:0/1339348289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdefc0039f0 0x7fdefc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf18cfd640 1 -- 172.21.15.60:0/1339348289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc003530 con 0x7fdefc004d50 2024-04-25T10:06:03.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf13fff640 1 --2- 172.21.15.60:0/1339348289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdefc0039f0 0x7fdefc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:03.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf18cfd640 1 --2- 172.21.15.60:0/1339348289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdefc004d50 0x7fdefc0051d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdf140b4010 tx=0x7fdf0c01fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4164c7557cfeb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf12ffd640 1 -- 172.21.15.60:0/1339348289 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c00a4a0 con 0x7fdefc004d50 2024-04-25T10:06:03.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf12ffd640 1 -- 172.21.15.60:0/1339348289 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0c0aa6d0 con 0x7fdefc004d50 2024-04-25T10:06:03.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf12ffd640 1 -- 172.21.15.60:0/1339348289 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c0aaac0 con 0x7fdefc004d50 2024-04-25T10:06:03.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf1a760640 1 -- 172.21.15.60:0/1339348289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdefc004d50 msgr2=0x7fdefc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf1a760640 1 --2- 172.21.15.60:0/1339348289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdefc004d50 0x7fdefc0051d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdf140b4010 tx=0x7fdf0c01fa80 comp rx=0 tx=0).stop 2024-04-25T10:06:03.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf18cfd640 1 -- 172.21.15.60:0/1339348289 reap_dead start 2024-04-25T10:06:03.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf1a760640 1 -- 172.21.15.60:0/1339348289 shutdown_connections 2024-04-25T10:06:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf1a760640 1 -- 172.21.15.60:0/1339348289 >> 172.21.15.60:0/1339348289 conn(0x7fdf140b12d0 msgr2=0x7fdf140b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf1a760640 1 -- 172.21.15.60:0/1339348289 shutdown_connections 2024-04-25T10:06:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.243+0000 7fdf1a760640 1 -- 172.21.15.60:0/1339348289 wait complete. 2024-04-25T10:06:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf1a760640 1 Processor -- start 2024-04-25T10:06:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf1a760640 1 -- start start 2024-04-25T10:06:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf1a760640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf141525f0 0x7fdf14154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf1a760640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf141529d0 0x7fdf14154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf1a760640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf1415df80 0x7fdf1415e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf1a760640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140b3d20 con 0x7fdf141525f0 2024-04-25T10:06:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf1a760640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140b3ba0 con 0x7fdf1415df80 2024-04-25T10:06:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf1a760640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140b4280 con 0x7fdf141529d0 2024-04-25T10:06:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf13fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf141525f0 0x7fdf14154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf13fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf141525f0 0x7fdf14154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45420/0 (socket says 172.21.15.60:45420) 2024-04-25T10:06:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf13fff640 1 -- 172.21.15.60:0/1949012232 learned_addr learned my addr 172.21.15.60:0/1949012232 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:03.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf137fe640 1 --2- 172.21.15.60:0/1949012232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf141529d0 0x7fdf14154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf18cfd640 1 --2- 172.21.15.60:0/1949012232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf1415df80 0x7fdf1415e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf13fff640 1 -- 172.21.15.60:0/1949012232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf141529d0 msgr2=0x7fdf14154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf13fff640 1 --2- 172.21.15.60:0/1949012232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf141529d0 0x7fdf14154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf13fff640 1 -- 172.21.15.60:0/1949012232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf1415df80 msgr2=0x7fdf1415e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf13fff640 1 --2- 172.21.15.60:0/1949012232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf1415df80 0x7fdf1415e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf13fff640 1 -- 172.21.15.60:0/1949012232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf141f3b20 con 0x7fdf141525f0 2024-04-25T10:06:03.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf137fe640 1 --2- 172.21.15.60:0/1949012232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf141529d0 0x7fdf14154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:03.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf13fff640 1 --2- 172.21.15.60:0/1949012232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf141525f0 0x7fdf14154450 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400e5d0 tx=0x7fdf0400ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf10ff9640 1 -- 172.21.15.60:0/1949012232 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf040052e0 con 0x7fdf141525f0 2024-04-25T10:06:03.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf10ff9640 1 -- 172.21.15.60:0/1949012232 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf04018070 con 0x7fdf141525f0 2024-04-25T10:06:03.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf10ff9640 1 -- 172.21.15.60:0/1949012232 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04005480 con 0x7fdf141525f0 2024-04-25T10:06:03.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf1a760640 1 -- 172.21.15.60:0/1949012232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf141f3e00 con 0x7fdf141525f0 2024-04-25T10:06:03.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf1a760640 1 -- 172.21.15.60:0/1949012232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf141f43c0 con 0x7fdf141525f0 2024-04-25T10:06:03.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf1a760640 1 -- 172.21.15.60:0/1949012232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf14074770 con 0x7fdf141525f0 2024-04-25T10:06:03.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf10ff9640 1 -- 172.21.15.60:0/1949012232 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf04003820 con 0x7fdf141525f0 2024-04-25T10:06:03.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.247+0000 7fdf10ff9640 1 --2- 172.21.15.60:0/1949012232 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf00066f90 0x7fdf00069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.251+0000 7fdf10ff9640 1 -- 172.21.15.60:0/1949012232 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fdf04084b00 con 0x7fdf141525f0 2024-04-25T10:06:03.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.251+0000 7fdf137fe640 1 --2- 172.21.15.60:0/1949012232 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf00066f90 0x7fdf00069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.251+0000 7fdf137fe640 1 --2- 172.21.15.60:0/1949012232 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf00066f90 0x7fdf00069450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdf1415eca0 tx=0x7fdf08008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.255+0000 7fdf10ff9640 1 -- 172.21.15.60:0/1949012232 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf04055b10 con 0x7fdf141525f0 2024-04-25T10:06:03.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.419+0000 7fdf1a760640 1 -- 172.21.15.60:0/1949012232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdf140ce0a0 con 0x7fdf141525f0 2024-04-25T10:06:03.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.423+0000 7fdf10ff9640 1 -- 172.21.15.60:0/1949012232 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdf04055410 con 0x7fdf141525f0 2024-04-25T10:06:03.428 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:06:03.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.423+0000 7fdf1a760640 1 -- 172.21.15.60:0/1949012232 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf00066f90 msgr2=0x7fdf00069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.423+0000 7fdf1a760640 1 --2- 172.21.15.60:0/1949012232 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf00066f90 0x7fdf00069450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdf1415eca0 tx=0x7fdf08008040 comp rx=0 tx=0).stop 2024-04-25T10:06:03.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.423+0000 7fdf1a760640 1 -- 172.21.15.60:0/1949012232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf141525f0 msgr2=0x7fdf14154450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.423+0000 7fdf1a760640 1 --2- 172.21.15.60:0/1949012232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf141525f0 0x7fdf14154450 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400e5d0 tx=0x7fdf0400ea90 comp rx=0 tx=0).stop 2024-04-25T10:06:03.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.423+0000 7fdf1a760640 1 -- 172.21.15.60:0/1949012232 shutdown_connections 2024-04-25T10:06:03.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.423+0000 7fdf1a760640 1 --2- 172.21.15.60:0/1949012232 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf00066f90 0x7fdf00069450 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.423+0000 7fdf1a760640 1 --2- 172.21.15.60:0/1949012232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf1415df80 0x7fdf1415e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.423+0000 7fdf1a760640 1 --2- 172.21.15.60:0/1949012232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf141529d0 0x7fdf14154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.423+0000 7fdf1a760640 1 --2- 172.21.15.60:0/1949012232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf141525f0 0x7fdf14154450 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.423+0000 7fdf1a760640 1 -- 172.21.15.60:0/1949012232 >> 172.21.15.60:0/1949012232 conn(0x7fdf140b12d0 msgr2=0x7fdf14151180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:03.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.423+0000 7fdf1a760640 1 -- 172.21.15.60:0/1949012232 shutdown_connections 2024-04-25T10:06:03.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.423+0000 7fdf1a760640 1 -- 172.21.15.60:0/1949012232 wait complete. 2024-04-25T10:06:03.446 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:06:03.447 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:03.451 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:03.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.607+0000 7f1fc8516640 1 Processor -- start 2024-04-25T10:06:03.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.607+0000 7f1fc8516640 1 -- start start 2024-04-25T10:06:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.607+0000 7f1fc8516640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1fc015bc90 0x7f1fc015c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.607+0000 7f1fc8516640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc0071ca0 con 0x7f1fc00ce7a0 2024-04-25T10:06:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.607+0000 7f1fc8516640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc00cca40 con 0x7f1fc00b4660 2024-04-25T10:06:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.607+0000 7f1fc8516640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc00cc370 con 0x7f1fc015bc90 2024-04-25T10:06:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.607+0000 7f1fc62b2640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1fc00ce7a0 0x7f1fc00ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48892/0 (socket says 172.21.15.60:48892) 2024-04-25T10:06:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.607+0000 7f1fc62b2640 1 -- 172.21.15.60:0/3774762958 learned_addr learned my addr 172.21.15.60:0/3774762958 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.607+0000 7f1fc52b0640 1 -- 172.21.15.60:0/3774762958 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3882429812 0 0) 0x7f1fc0071ca0 con 0x7f1fc00ce7a0 2024-04-25T10:06:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.607+0000 7f1fc52b0640 1 -- 172.21.15.60:0/3774762958 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fac003530 con 0x7f1fc00ce7a0 2024-04-25T10:06:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.607+0000 7f1fc5ab1640 1 --2- 172.21.15.60:0/3774762958 >> v2:172.21.15.100:3300/0 conn(0x7f1fc015bc90 0x7f1fc015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.607+0000 7f1fc52b0640 1 -- 172.21.15.60:0/3774762958 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1622020017 0 0) 0x7f1fac003530 con 0x7f1fc00ce7a0 2024-04-25T10:06:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.607+0000 7f1fc52b0640 1 -- 172.21.15.60:0/3774762958 >> v2:172.21.15.100:3300/0 conn(0x7f1fc015bc90 msgr2=0x7f1fc015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.607+0000 7f1fc52b0640 1 --2- 172.21.15.60:0/3774762958 >> v2:172.21.15.100:3300/0 conn(0x7f1fc015bc90 0x7f1fc015c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.607+0000 7f1fc52b0640 1 -- 172.21.15.60:0/3774762958 >> v1:172.21.15.100:6789/0 conn(0x7f1fc00b4660 legacy=0x7f1fc015c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.607+0000 7f1fc52b0640 1 -- 172.21.15.60:0/3774762958 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc00ccd20 con 0x7f1fc00ce7a0 2024-04-25T10:06:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc52b0640 1 -- 172.21.15.60:0/3774762958 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f1fbc001680 con 0x7f1fc00ce7a0 2024-04-25T10:06:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc52b0640 1 -- 172.21.15.60:0/3774762958 >> v1:172.21.15.60:6789/0 conn(0x7f1fc00ce7a0 legacy=0x7f1fc00ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc52b0640 1 --2- 172.21.15.60:0/3774762958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fac0039c0 0x7f1fac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc52b0640 1 --2- 172.21.15.60:0/3774762958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fac004810 0x7f1fac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc52b0640 1 --2- 172.21.15.60:0/3774762958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fac0051d0 0x7f1fac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc52b0640 1 -- 172.21.15.60:0/3774762958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac00c0d0 con 0x7f1fac0039c0 2024-04-25T10:06:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc52b0640 1 -- 172.21.15.60:0/3774762958 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac00c280 con 0x7f1fac004810 2024-04-25T10:06:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc52b0640 1 -- 172.21.15.60:0/3774762958 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac00c480 con 0x7f1fac0051d0 2024-04-25T10:06:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc62b2640 1 --2- 172.21.15.60:0/3774762958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fac0039c0 0x7f1fac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc5ab1640 1 --2- 172.21.15.60:0/3774762958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fac004810 0x7f1fac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc6ab3640 1 --2- 172.21.15.60:0/3774762958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fac0051d0 0x7f1fac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc62b2640 1 -- 172.21.15.60:0/3774762958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fac0051d0 msgr2=0x7f1fac00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc62b2640 1 --2- 172.21.15.60:0/3774762958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fac0051d0 0x7f1fac00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc62b2640 1 -- 172.21.15.60:0/3774762958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fac004810 msgr2=0x7f1fac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc62b2640 1 --2- 172.21.15.60:0/3774762958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fac004810 0x7f1fac004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc6ab3640 1 -- 172.21.15.60:0/3774762958 reap_dead start 2024-04-25T10:06:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc62b2640 1 -- 172.21.15.60:0/3774762958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac003530 con 0x7f1fac0039c0 2024-04-25T10:06:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc62b2640 1 --2- 172.21.15.60:0/3774762958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fac0039c0 0x7f1fac003dc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00c210 tx=0x7f1fbc01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616f4b7f1dcf7be6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc52b0640 1 -- 172.21.15.60:0/3774762958 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc007d40 con 0x7f1fac0039c0 2024-04-25T10:06:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc52b0640 1 -- 172.21.15.60:0/3774762958 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc0a9070 con 0x7f1fac0039c0 2024-04-25T10:06:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc52b0640 1 -- 172.21.15.60:0/3774762958 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc005d80 con 0x7f1fac0039c0 2024-04-25T10:06:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc8516640 1 -- 172.21.15.60:0/3774762958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fac0039c0 msgr2=0x7f1fac003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc8516640 1 --2- 172.21.15.60:0/3774762958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fac0039c0 0x7f1fac003dc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00c210 tx=0x7f1fbc01cc60 comp rx=0 tx=0).stop 2024-04-25T10:06:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc8516640 1 -- 172.21.15.60:0/3774762958 shutdown_connections 2024-04-25T10:06:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc8516640 1 --2- 172.21.15.60:0/3774762958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fac0039c0 0x7f1fac003dc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc8516640 1 -- 172.21.15.60:0/3774762958 >> 172.21.15.60:0/3774762958 conn(0x7f1fc00b12d0 msgr2=0x7f1fc00b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc8516640 1 -- 172.21.15.60:0/3774762958 shutdown_connections 2024-04-25T10:06:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.611+0000 7f1fc8516640 1 -- 172.21.15.60:0/3774762958 wait complete. 2024-04-25T10:06:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc8516640 1 Processor -- start 2024-04-25T10:06:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc8516640 1 -- start start 2024-04-25T10:06:03.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc8516640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fac0039c0 0x7f1fc00adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc8516640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fc0157240 0x7f1fc0155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc8516640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc01504e0 0x7f1fc0150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc8516640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc00b4280 con 0x7f1fc0157240 2024-04-25T10:06:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc8516640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc00b3db0 con 0x7f1fc01504e0 2024-04-25T10:06:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc8516640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc00b4400 con 0x7f1fac0039c0 2024-04-25T10:06:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc5ab1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fc0157240 0x7f1fc0155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc5ab1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fc0157240 0x7f1fc0155770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45434/0 (socket says 172.21.15.60:45434) 2024-04-25T10:06:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc5ab1640 1 -- 172.21.15.60:0/2243313175 learned_addr learned my addr 172.21.15.60:0/2243313175 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc62b2640 1 --2- 172.21.15.60:0/2243313175 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fac0039c0 0x7f1fc00adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc6ab3640 1 --2- 172.21.15.60:0/2243313175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc01504e0 0x7f1fc0150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc5ab1640 1 -- 172.21.15.60:0/2243313175 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fac0039c0 msgr2=0x7f1fc00adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc5ab1640 1 --2- 172.21.15.60:0/2243313175 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fac0039c0 0x7f1fc00adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc5ab1640 1 -- 172.21.15.60:0/2243313175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc01504e0 msgr2=0x7f1fc0150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc5ab1640 1 --2- 172.21.15.60:0/2243313175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc01504e0 0x7f1fc0150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc5ab1640 1 -- 172.21.15.60:0/2243313175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc014c310 con 0x7f1fc0157240 2024-04-25T10:06:03.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc6ab3640 1 --2- 172.21.15.60:0/2243313175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc01504e0 0x7f1fc0150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:03.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc5ab1640 1 --2- 172.21.15.60:0/2243313175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fc0157240 0x7f1fc0155770 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000f590 tx=0x7f1fb000fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fb6ffd640 1 -- 172.21.15.60:0/2243313175 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0019030 con 0x7f1fc0157240 2024-04-25T10:06:03.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fb6ffd640 1 -- 172.21.15.60:0/2243313175 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb0004020 con 0x7f1fc0157240 2024-04-25T10:06:03.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fb6ffd640 1 -- 172.21.15.60:0/2243313175 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb00043f0 con 0x7f1fc0157240 2024-04-25T10:06:03.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc8516640 1 -- 172.21.15.60:0/2243313175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc0157920 con 0x7f1fc0157240 2024-04-25T10:06:03.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc8516640 1 -- 172.21.15.60:0/2243313175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc0157ee0 con 0x7f1fc0157240 2024-04-25T10:06:03.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.615+0000 7f1fc8516640 1 -- 172.21.15.60:0/2243313175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc0074770 con 0x7f1fc0157240 2024-04-25T10:06:03.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.619+0000 7f1fb6ffd640 1 -- 172.21.15.60:0/2243313175 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1fb001c070 con 0x7f1fc0157240 2024-04-25T10:06:03.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.619+0000 7f1fb6ffd640 1 --2- 172.21.15.60:0/2243313175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1fa4066f60 0x7f1fa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.619+0000 7f1fb6ffd640 1 -- 172.21.15.60:0/2243313175 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f1fb00851d0 con 0x7f1fc0157240 2024-04-25T10:06:03.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.619+0000 7f1fc62b2640 1 --2- 172.21.15.60:0/2243313175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1fa4066f60 0x7f1fa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.619+0000 7f1fc62b2640 1 --2- 172.21.15.60:0/2243313175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1fa4066f60 0x7f1fa4069420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc0ab000 tx=0x7f1fbc0b0040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.623+0000 7f1fb6ffd640 1 -- 172.21.15.60:0/2243313175 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1fb00561e0 con 0x7f1fc0157240 2024-04-25T10:06:03.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.783+0000 7f1fc8516640 1 -- 172.21.15.60:0/2243313175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fc00b0240 con 0x7f1fc0157240 2024-04-25T10:06:03.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.783+0000 7f1fb6ffd640 1 -- 172.21.15.60:0/2243313175 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16857 (secure 0 0 0) 0x7f1fb0055ae0 con 0x7f1fc0157240 2024-04-25T10:06:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:03.791 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:01.568957+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.787+0000 7f1fc8516640 1 -- 172.21.15.60:0/2243313175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1fa4066f60 msgr2=0x7f1fa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.787+0000 7f1fc8516640 1 --2- 172.21.15.60:0/2243313175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1fa4066f60 0x7f1fa4069420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc0ab000 tx=0x7f1fbc0b0040 comp rx=0 tx=0).stop 2024-04-25T10:06:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.787+0000 7f1fc8516640 1 -- 172.21.15.60:0/2243313175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fc0157240 msgr2=0x7f1fc0155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.787+0000 7f1fc8516640 1 --2- 172.21.15.60:0/2243313175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fc0157240 0x7f1fc0155770 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000f590 tx=0x7f1fb000fa50 comp rx=0 tx=0).stop 2024-04-25T10:06:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.787+0000 7f1fc8516640 1 -- 172.21.15.60:0/2243313175 shutdown_connections 2024-04-25T10:06:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.787+0000 7f1fc8516640 1 --2- 172.21.15.60:0/2243313175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1fa4066f60 0x7f1fa4069420 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.787+0000 7f1fc8516640 1 --2- 172.21.15.60:0/2243313175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc01504e0 0x7f1fc0150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.787+0000 7f1fc8516640 1 --2- 172.21.15.60:0/2243313175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fc0157240 0x7f1fc0155770 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.787+0000 7f1fc8516640 1 --2- 172.21.15.60:0/2243313175 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fac0039c0 0x7f1fc00adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.787+0000 7f1fc8516640 1 -- 172.21.15.60:0/2243313175 >> 172.21.15.60:0/2243313175 conn(0x7f1fc00b12d0 msgr2=0x7f1fc00b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.787+0000 7f1fc8516640 1 -- 172.21.15.60:0/2243313175 shutdown_connections 2024-04-25T10:06:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.787+0000 7f1fc8516640 1 -- 172.21.15.60:0/2243313175 wait complete. 2024-04-25T10:06:03.810 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:18.976643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:19.405302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:03.810 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:03.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.919+0000 7f4063c23640 1 Processor -- start 2024-04-25T10:06:03.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.919+0000 7f4063c23640 1 -- start start 2024-04-25T10:06:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f4063c23640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f405c06f030 0x7f405c0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f4063c23640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f405c071250 0x7f405c0716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f4063c23640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f405c15ad30 con 0x7f405c0b3f60 2024-04-25T10:06:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f4063c23640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f405c15aaf0 con 0x7f405c071250 2024-04-25T10:06:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f4063c23640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405c15b010 con 0x7f405c06f030 2024-04-25T10:06:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f40619bf640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f405c0b3f60 0x7f405c15a3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48894/0 (socket says 172.21.15.60:48894) 2024-04-25T10:06:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f40619bf640 1 -- 172.21.15.60:0/1811711183 learned_addr learned my addr 172.21.15.60:0/1811711183 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f40621c0640 1 --2- 172.21.15.60:0/1811711183 >> v2:172.21.15.60:3300/0 conn(0x7f405c071250 0x7f405c0716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f40611be640 1 --2- 172.21.15.60:0/1811711183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f405c06f030 0x7f405c0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f40621c0640 1 -- 172.21.15.60:0/1811711183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f405c06f030 msgr2=0x7f405c0b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f40621c0640 1 --2- 172.21.15.60:0/1811711183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f405c06f030 0x7f405c0b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f40621c0640 1 -- 172.21.15.60:0/1811711183 >> v1:172.21.15.60:6789/0 conn(0x7f405c0b3f60 legacy=0x7f405c15a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f40621c0640 1 -- 172.21.15.60:0/1811711183 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c15b2f0 con 0x7f405c071250 2024-04-25T10:06:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f40609bd640 1 -- 172.21.15.60:0/1811711183 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2185617180 0 0) 0x7f405c15ad30 con 0x7f405c0b3f60 2024-04-25T10:06:03.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f40621c0640 1 --2- 172.21.15.60:0/1811711183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c071250 0x7f405c0716e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4054009d90 tx=0x7f405401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7ef5ec889206e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f40609bd640 1 -- 172.21.15.60:0/1811711183 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40540b2070 con 0x7f405c071250 2024-04-25T10:06:03.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f40609bd640 1 -- 172.21.15.60:0/1811711183 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40540a5720 con 0x7f405c071250 2024-04-25T10:06:03.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.923+0000 7f40609bd640 1 -- 172.21.15.60:0/1811711183 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40540a5af0 con 0x7f405c071250 2024-04-25T10:06:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 -- 172.21.15.60:0/1811711183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c071250 msgr2=0x7f405c0716e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 --2- 172.21.15.60:0/1811711183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c071250 0x7f405c0716e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4054009d90 tx=0x7f405401c8e0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 -- 172.21.15.60:0/1811711183 shutdown_connections 2024-04-25T10:06:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 --2- 172.21.15.60:0/1811711183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c071250 0x7f405c0716e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 --2- 172.21.15.60:0/1811711183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f405c06f030 0x7f405c0b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 -- 172.21.15.60:0/1811711183 >> 172.21.15.60:0/1811711183 conn(0x7f405c0670b0 msgr2=0x7f405c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 -- 172.21.15.60:0/1811711183 shutdown_connections 2024-04-25T10:06:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 -- 172.21.15.60:0/1811711183 wait complete. 2024-04-25T10:06:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 Processor -- start 2024-04-25T10:06:03.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 -- start start 2024-04-25T10:06:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c06f030 0x7f405c152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f405c071250 0x7f405c136d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f405c0b3f60 0x7f405c1484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c15b5a0 con 0x7f405c06f030 2024-04-25T10:06:03.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c15b720 con 0x7f405c0b3f60 2024-04-25T10:06:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405c15b8a0 con 0x7f405c071250 2024-04-25T10:06:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f40619bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c06f030 0x7f405c152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f40619bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c06f030 0x7f405c152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45454/0 (socket says 172.21.15.60:45454) 2024-04-25T10:06:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f40619bf640 1 -- 172.21.15.60:0/1859339780 learned_addr learned my addr 172.21.15.60:0/1859339780 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f40619bf640 1 -- 172.21.15.60:0/1859339780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f405c071250 msgr2=0x7f405c136d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f40619bf640 1 --2- 172.21.15.60:0/1859339780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f405c071250 0x7f405c136d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f40619bf640 1 -- 172.21.15.60:0/1859339780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f405c0b3f60 msgr2=0x7f405c1484f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f40619bf640 1 --2- 172.21.15.60:0/1859339780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f405c0b3f60 0x7f405c1484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f40619bf640 1 -- 172.21.15.60:0/1859339780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c151d50 con 0x7f405c06f030 2024-04-25T10:06:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f40619bf640 1 --2- 172.21.15.60:0/1859339780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c06f030 0x7f405c152df0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f405800c4b0 tx=0x7f405800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f40527fc640 1 -- 172.21.15.60:0/1859339780 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4058018070 con 0x7f405c06f030 2024-04-25T10:06:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f40527fc640 1 -- 172.21.15.60:0/1859339780 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4058007500 con 0x7f405c06f030 2024-04-25T10:06:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 -- 172.21.15.60:0/1859339780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405c152030 con 0x7f405c06f030 2024-04-25T10:06:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f40527fc640 1 -- 172.21.15.60:0/1859339780 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4058004a30 con 0x7f405c06f030 2024-04-25T10:06:03.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 -- 172.21.15.60:0/1859339780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405c1525f0 con 0x7f405c06f030 2024-04-25T10:06:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.927+0000 7f4063c23640 1 -- 172.21.15.60:0/1859339780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405c153330 con 0x7f405c06f030 2024-04-25T10:06:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.931+0000 7f40527fc640 1 -- 172.21.15.60:0/1859339780 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f405801b070 con 0x7f405c06f030 2024-04-25T10:06:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.931+0000 7f40527fc640 1 --2- 172.21.15.60:0/1859339780 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4040066f60 0x7f4040069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.931+0000 7f40527fc640 1 -- 172.21.15.60:0/1859339780 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f40580843d0 con 0x7f405c06f030 2024-04-25T10:06:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.931+0000 7f40611be640 1 --2- 172.21.15.60:0/1859339780 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4040066f60 0x7f4040069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.931+0000 7f40611be640 1 --2- 172.21.15.60:0/1859339780 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4040066f60 0x7f4040069420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f405c0cbbd0 tx=0x7f404c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:03.935+0000 7f40527fc640 1 -- 172.21.15.60:0/1859339780 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f405c153330 con 0x7f405c06f030 2024-04-25T10:06:04.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.095+0000 7f4063c23640 1 -- 172.21.15.60:0/1859339780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f405c0b3840 con 0x7f405c06f030 2024-04-25T10:06:04.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.095+0000 7f40527fc640 1 -- 172.21.15.60:0/1859339780 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4058055350 con 0x7f405c06f030 2024-04-25T10:06:04.103 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:06:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.099+0000 7f4063c23640 1 -- 172.21.15.60:0/1859339780 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4040066f60 msgr2=0x7f4040069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.099+0000 7f4063c23640 1 --2- 172.21.15.60:0/1859339780 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4040066f60 0x7f4040069420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f405c0cbbd0 tx=0x7f404c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.099+0000 7f4063c23640 1 -- 172.21.15.60:0/1859339780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c06f030 msgr2=0x7f405c152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.099+0000 7f4063c23640 1 --2- 172.21.15.60:0/1859339780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c06f030 0x7f405c152df0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f405800c4b0 tx=0x7f405800c970 comp rx=0 tx=0).stop 2024-04-25T10:06:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.099+0000 7f4063c23640 1 -- 172.21.15.60:0/1859339780 shutdown_connections 2024-04-25T10:06:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.099+0000 7f4063c23640 1 --2- 172.21.15.60:0/1859339780 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4040066f60 0x7f4040069420 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.099+0000 7f4063c23640 1 --2- 172.21.15.60:0/1859339780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f405c0b3f60 0x7f405c1484f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.099+0000 7f4063c23640 1 --2- 172.21.15.60:0/1859339780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f405c071250 0x7f405c136d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.099+0000 7f4063c23640 1 --2- 172.21.15.60:0/1859339780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f405c06f030 0x7f405c152df0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.099+0000 7f4063c23640 1 -- 172.21.15.60:0/1859339780 >> 172.21.15.60:0/1859339780 conn(0x7f405c0670b0 msgr2=0x7f405c15a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.099+0000 7f4063c23640 1 -- 172.21.15.60:0/1859339780 shutdown_connections 2024-04-25T10:06:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.099+0000 7f4063c23640 1 -- 172.21.15.60:0/1859339780 wait complete. 2024-04-25T10:06:04.122 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:04.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.243+0000 7effe6c8a640 1 Processor -- start 2024-04-25T10:06:04.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.243+0000 7effe6c8a640 1 -- start start 2024-04-25T10:06:04.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effe6c8a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7effe00cf8a0 0x7effe015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effe6c8a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe00cd3d0 0x7effe015be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effe6c8a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe00cd890 con 0x7effe015dd30 2024-04-25T10:06:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effe6c8a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe00ce4a0 con 0x7effe00cf8a0 2024-04-25T10:06:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effe6c8a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe00cdb70 con 0x7effe00cd3d0 2024-04-25T10:06:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effe4a26640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7effe015dd30 0x7effe00cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48900/0 (socket says 172.21.15.60:48900) 2024-04-25T10:06:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effe4a26640 1 -- 172.21.15.60:0/4095290208 learned_addr learned my addr 172.21.15.60:0/4095290208 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdf7fe640 1 -- 172.21.15.60:0/4095290208 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425173702 0 0) 0x7effe00cd890 con 0x7effe015dd30 2024-04-25T10:06:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdf7fe640 1 -- 172.21.15.60:0/4095290208 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effc8003530 con 0x7effe015dd30 2024-04-25T10:06:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effe5227640 1 --2- 172.21.15.60:0/4095290208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe00cd3d0 0x7effe015be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdffff640 1 --2- 172.21.15.60:0/4095290208 >> v2:172.21.15.100:3300/0 conn(0x7effe00cf8a0 0x7effe015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdf7fe640 1 -- 172.21.15.60:0/4095290208 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 15098549 0 0) 0x7effc8003530 con 0x7effe015dd30 2024-04-25T10:06:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdf7fe640 1 -- 172.21.15.60:0/4095290208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe00cd3d0 msgr2=0x7effe015be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdf7fe640 1 --2- 172.21.15.60:0/4095290208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe00cd3d0 0x7effe015be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdf7fe640 1 -- 172.21.15.60:0/4095290208 >> v2:172.21.15.100:3300/0 conn(0x7effe00cf8a0 msgr2=0x7effe015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdf7fe640 1 --2- 172.21.15.60:0/4095290208 >> v2:172.21.15.100:3300/0 conn(0x7effe00cf8a0 0x7effe015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdf7fe640 1 -- 172.21.15.60:0/4095290208 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe00cde50 con 0x7effe015dd30 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdf7fe640 1 -- 172.21.15.60:0/4095290208 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7effd0001680 con 0x7effe015dd30 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdffff640 1 --2- 172.21.15.60:0/4095290208 >> v2:172.21.15.100:3300/0 conn(0x7effe00cf8a0 0x7effe015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdf7fe640 1 -- 172.21.15.60:0/4095290208 >> v1:172.21.15.60:6789/0 conn(0x7effe015dd30 legacy=0x7effe00cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdf7fe640 1 --2- 172.21.15.60:0/4095290208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe00cd3d0 0x7effc80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdf7fe640 1 --2- 172.21.15.60:0/4095290208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc8003f40 0x7effc8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdf7fe640 1 --2- 172.21.15.60:0/4095290208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc8004d50 0x7effc80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdf7fe640 1 -- 172.21.15.60:0/4095290208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe00ce4a0 con 0x7effc8003f40 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdf7fe640 1 -- 172.21.15.60:0/4095290208 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe00cdb70 con 0x7effc8004d50 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdf7fe640 1 -- 172.21.15.60:0/4095290208 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc800bad0 con 0x7effe00cd3d0 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdffff640 1 --2- 172.21.15.60:0/4095290208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc8003f40 0x7effc8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effe4a26640 1 --2- 172.21.15.60:0/4095290208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe00cd3d0 0x7effc80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effe5227640 1 --2- 172.21.15.60:0/4095290208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc8004d50 0x7effc80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdffff640 1 -- 172.21.15.60:0/4095290208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe00cd3d0 msgr2=0x7effc80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdffff640 1 --2- 172.21.15.60:0/4095290208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe00cd3d0 0x7effc80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdffff640 1 -- 172.21.15.60:0/4095290208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc8004d50 msgr2=0x7effc80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdffff640 1 --2- 172.21.15.60:0/4095290208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc8004d50 0x7effc80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.247+0000 7effdffff640 1 -- 172.21.15.60:0/4095290208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc8003530 con 0x7effc8003f40 2024-04-25T10:06:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.251+0000 7effdffff640 1 --2- 172.21.15.60:0/4095290208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc8003f40 0x7effc8004810 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7effd4009900 tx=0x7effd401c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3def9b7c83d46a51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.251+0000 7effdf7fe640 1 -- 172.21.15.60:0/4095290208 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd400f040 con 0x7effc8003f40 2024-04-25T10:06:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.251+0000 7effdf7fe640 1 -- 172.21.15.60:0/4095290208 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd40a9610 con 0x7effc8003f40 2024-04-25T10:06:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.251+0000 7effdf7fe640 1 -- 172.21.15.60:0/4095290208 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd40a99c0 con 0x7effc8003f40 2024-04-25T10:06:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.251+0000 7effe6c8a640 1 -- 172.21.15.60:0/4095290208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc8003f40 msgr2=0x7effc8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.251+0000 7effe6c8a640 1 --2- 172.21.15.60:0/4095290208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc8003f40 0x7effc8004810 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7effd4009900 tx=0x7effd401c600 comp rx=0 tx=0).stop 2024-04-25T10:06:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.251+0000 7effe5227640 1 -- 172.21.15.60:0/4095290208 reap_dead start 2024-04-25T10:06:04.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.251+0000 7effe6c8a640 1 -- 172.21.15.60:0/4095290208 shutdown_connections 2024-04-25T10:06:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.251+0000 7effe6c8a640 1 -- 172.21.15.60:0/4095290208 >> 172.21.15.60:0/4095290208 conn(0x7effe00670b0 msgr2=0x7effe006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.251+0000 7effe6c8a640 1 -- 172.21.15.60:0/4095290208 shutdown_connections 2024-04-25T10:06:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.251+0000 7effe6c8a640 1 -- 172.21.15.60:0/4095290208 wait complete. 2024-04-25T10:06:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.251+0000 7effe6c8a640 1 Processor -- start 2024-04-25T10:06:04.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.251+0000 7effe6c8a640 1 -- start start 2024-04-25T10:06:04.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effe6c8a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe014a690 0x7effe01538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effe6c8a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effe014b980 0x7effe014df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effe6c8a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effe014be00 0x7effe014e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effe6c8a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe0071df0 con 0x7effe014b980 2024-04-25T10:06:04.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effe6c8a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe0071c70 con 0x7effe014be00 2024-04-25T10:06:04.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effe6c8a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe0071f70 con 0x7effe014a690 2024-04-25T10:06:04.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effdffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effe014b980 0x7effe014df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effdffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effe014b980 0x7effe014df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45484/0 (socket says 172.21.15.60:45484) 2024-04-25T10:06:04.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effdffff640 1 -- 172.21.15.60:0/66015325 learned_addr learned my addr 172.21.15.60:0/66015325 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:04.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effe4a26640 1 --2- 172.21.15.60:0/66015325 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe014a690 0x7effe01538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effe5227640 1 --2- 172.21.15.60:0/66015325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effe014be00 0x7effe014e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effdffff640 1 -- 172.21.15.60:0/66015325 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe014a690 msgr2=0x7effe01538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effdffff640 1 --2- 172.21.15.60:0/66015325 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe014a690 0x7effe01538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effdffff640 1 -- 172.21.15.60:0/66015325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effe014be00 msgr2=0x7effe014e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effdffff640 1 --2- 172.21.15.60:0/66015325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effe014be00 0x7effe014e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effdffff640 1 -- 172.21.15.60:0/66015325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe014b090 con 0x7effe014b980 2024-04-25T10:06:04.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effdffff640 1 --2- 172.21.15.60:0/66015325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effe014b980 0x7effe014df90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7effd4009900 tx=0x7effd40193a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effdd7fa640 1 -- 172.21.15.60:0/66015325 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd40c1070 con 0x7effe014b980 2024-04-25T10:06:04.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effdd7fa640 1 -- 172.21.15.60:0/66015325 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd400f000 con 0x7effe014b980 2024-04-25T10:06:04.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effe6c8a640 1 -- 172.21.15.60:0/66015325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe014c310 con 0x7effe014b980 2024-04-25T10:06:04.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effdd7fa640 1 -- 172.21.15.60:0/66015325 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd40bc630 con 0x7effe014b980 2024-04-25T10:06:04.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effe6c8a640 1 -- 172.21.15.60:0/66015325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe014c8d0 con 0x7effe014b980 2024-04-25T10:06:04.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effe6c8a640 1 -- 172.21.15.60:0/66015325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe0153e20 con 0x7effe014b980 2024-04-25T10:06:04.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effdd7fa640 1 -- 172.21.15.60:0/66015325 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7effd40c4070 con 0x7effe014b980 2024-04-25T10:06:04.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effdd7fa640 1 --2- 172.21.15.60:0/66015325 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7effcc066f60 0x7effcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effdd7fa640 1 -- 172.21.15.60:0/66015325 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7effd411f5c0 con 0x7effe014b980 2024-04-25T10:06:04.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.255+0000 7effe4a26640 1 --2- 172.21.15.60:0/66015325 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7effcc066f60 0x7effcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.259+0000 7effe4a26640 1 --2- 172.21.15.60:0/66015325 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7effcc066f60 0x7effcc069420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7effd0001060 tx=0x7effd0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.259+0000 7effdd7fa640 1 -- 172.21.15.60:0/66015325 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7effd40ad020 con 0x7effe014b980 2024-04-25T10:06:04.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.411+0000 7effe6c8a640 1 -- 172.21.15.60:0/66015325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7effe0074770 con 0x7effe014b980 2024-04-25T10:06:04.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.415+0000 7effdd7fa640 1 -- 172.21.15.60:0/66015325 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7effd4016340 con 0x7effe014b980 2024-04-25T10:06:04.420 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:06:04.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.415+0000 7effe6c8a640 1 -- 172.21.15.60:0/66015325 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7effcc066f60 msgr2=0x7effcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.415+0000 7effe6c8a640 1 --2- 172.21.15.60:0/66015325 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7effcc066f60 0x7effcc069420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7effd0001060 tx=0x7effd0009040 comp rx=0 tx=0).stop 2024-04-25T10:06:04.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.415+0000 7effe6c8a640 1 -- 172.21.15.60:0/66015325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effe014b980 msgr2=0x7effe014df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.415+0000 7effe6c8a640 1 --2- 172.21.15.60:0/66015325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effe014b980 0x7effe014df90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7effd4009900 tx=0x7effd40193a0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.415+0000 7effe6c8a640 1 -- 172.21.15.60:0/66015325 shutdown_connections 2024-04-25T10:06:04.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.415+0000 7effe6c8a640 1 --2- 172.21.15.60:0/66015325 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7effcc066f60 0x7effcc069420 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.415+0000 7effe6c8a640 1 --2- 172.21.15.60:0/66015325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effe014be00 0x7effe014e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.415+0000 7effe6c8a640 1 --2- 172.21.15.60:0/66015325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effe014b980 0x7effe014df90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.415+0000 7effe6c8a640 1 --2- 172.21.15.60:0/66015325 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe014a690 0x7effe01538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.415+0000 7effe6c8a640 1 -- 172.21.15.60:0/66015325 >> 172.21.15.60:0/66015325 conn(0x7effe00670b0 msgr2=0x7effe015ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.415+0000 7effe6c8a640 1 -- 172.21.15.60:0/66015325 shutdown_connections 2024-04-25T10:06:04.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.415+0000 7effe6c8a640 1 -- 172.21.15.60:0/66015325 wait complete. 2024-04-25T10:06:04.438 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:04.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb1e3c0640 1 Processor -- start 2024-04-25T10:06:04.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb1e3c0640 1 -- start start 2024-04-25T10:06:04.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb1e3c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb180b35f0 0x7fcb180ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb1e3c0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcb180aec10 0x7fcb180ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb1e3c0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb1815def0 con 0x7fcb1815db10 2024-04-25T10:06:04.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb1e3c0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb180accc0 con 0x7fcb180aec10 2024-04-25T10:06:04.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb1e3c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb18071630 con 0x7fcb180b35f0 2024-04-25T10:06:04.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb17fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fcb1815db10 0x7fcb180b2ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48916/0 (socket says 172.21.15.60:48916) 2024-04-25T10:06:04.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb17fff640 1 -- 172.21.15.60:0/761823302 learned_addr learned my addr 172.21.15.60:0/761823302 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:04.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb177fe640 1 --2- 172.21.15.60:0/761823302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb180b35f0 0x7fcb180ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb1c95d640 1 --2- 172.21.15.60:0/761823302 >> v2:172.21.15.100:3300/0 conn(0x7fcb180aec10 0x7fcb180ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb16ffd640 1 -- 172.21.15.60:0/761823302 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708259347 0 0) 0x7fcb1815def0 con 0x7fcb1815db10 2024-04-25T10:06:04.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb16ffd640 1 -- 172.21.15.60:0/761823302 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb00003530 con 0x7fcb1815db10 2024-04-25T10:06:04.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb16ffd640 1 -- 172.21.15.60:0/761823302 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2241196471 0 0) 0x7fcb00003530 con 0x7fcb1815db10 2024-04-25T10:06:04.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb16ffd640 1 -- 172.21.15.60:0/761823302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb180b35f0 msgr2=0x7fcb180ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb16ffd640 1 --2- 172.21.15.60:0/761823302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb180b35f0 0x7fcb180ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb16ffd640 1 -- 172.21.15.60:0/761823302 >> v2:172.21.15.100:3300/0 conn(0x7fcb180aec10 msgr2=0x7fcb180ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb16ffd640 1 --2- 172.21.15.60:0/761823302 >> v2:172.21.15.100:3300/0 conn(0x7fcb180aec10 0x7fcb180ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb16ffd640 1 -- 172.21.15.60:0/761823302 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb18071910 con 0x7fcb1815db10 2024-04-25T10:06:04.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb16ffd640 1 -- 172.21.15.60:0/761823302 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fcb0c001680 con 0x7fcb1815db10 2024-04-25T10:06:04.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb16ffd640 1 -- 172.21.15.60:0/761823302 >> v1:172.21.15.60:6789/0 conn(0x7fcb1815db10 legacy=0x7fcb180b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb16ffd640 1 --2- 172.21.15.60:0/761823302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb000039c0 0x7fcb00003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb16ffd640 1 --2- 172.21.15.60:0/761823302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb180b35f0 0x7fcb00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb16ffd640 1 --2- 172.21.15.60:0/761823302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb00004d50 0x7fcb000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb16ffd640 1 -- 172.21.15.60:0/761823302 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb18071630 con 0x7fcb00004d50 2024-04-25T10:06:04.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb16ffd640 1 -- 172.21.15.60:0/761823302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0000b9d0 con 0x7fcb000039c0 2024-04-25T10:06:04.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb16ffd640 1 -- 172.21.15.60:0/761823302 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0000bb50 con 0x7fcb180b35f0 2024-04-25T10:06:04.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb1c95d640 1 --2- 172.21.15.60:0/761823302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb00004d50 0x7fcb000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb17fff640 1 --2- 172.21.15.60:0/761823302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb000039c0 0x7fcb00003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb177fe640 1 --2- 172.21.15.60:0/761823302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb180b35f0 0x7fcb00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb1c95d640 1 -- 172.21.15.60:0/761823302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb180b35f0 msgr2=0x7fcb00004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb1c95d640 1 --2- 172.21.15.60:0/761823302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb180b35f0 0x7fcb00004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb1c95d640 1 -- 172.21.15.60:0/761823302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb000039c0 msgr2=0x7fcb00003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb1c95d640 1 --2- 172.21.15.60:0/761823302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb000039c0 0x7fcb00003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb1c95d640 1 -- 172.21.15.60:0/761823302 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb00003530 con 0x7fcb00004d50 2024-04-25T10:06:04.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.551+0000 7fcb177fe640 1 --2- 172.21.15.60:0/761823302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb180b35f0 0x7fcb00004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:04.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.555+0000 7fcb1c95d640 1 --2- 172.21.15.60:0/761823302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb00004d50 0x7fcb000051d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009fc0 tx=0x7fcb1001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e3c78eb6f8a627 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.555+0000 7fcb16ffd640 1 -- 172.21.15.60:0/761823302 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb100b2070 con 0x7fcb00004d50 2024-04-25T10:06:04.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.555+0000 7fcb16ffd640 1 -- 172.21.15.60:0/761823302 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb100a5cd0 con 0x7fcb00004d50 2024-04-25T10:06:04.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.555+0000 7fcb16ffd640 1 -- 172.21.15.60:0/761823302 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10009c10 con 0x7fcb00004d50 2024-04-25T10:06:04.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.555+0000 7fcb1e3c0640 1 -- 172.21.15.60:0/761823302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb00004d50 msgr2=0x7fcb000051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.555+0000 7fcb1e3c0640 1 --2- 172.21.15.60:0/761823302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb00004d50 0x7fcb000051d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009fc0 tx=0x7fcb1001ccc0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.555+0000 7fcb1c95d640 1 -- 172.21.15.60:0/761823302 reap_dead start 2024-04-25T10:06:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.555+0000 7fcb1e3c0640 1 -- 172.21.15.60:0/761823302 shutdown_connections 2024-04-25T10:06:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.555+0000 7fcb1e3c0640 1 -- 172.21.15.60:0/761823302 >> 172.21.15.60:0/761823302 conn(0x7fcb180670b0 msgr2=0x7fcb1806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.555+0000 7fcb1e3c0640 1 -- 172.21.15.60:0/761823302 shutdown_connections 2024-04-25T10:06:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.555+0000 7fcb1e3c0640 1 -- 172.21.15.60:0/761823302 wait complete. 2024-04-25T10:06:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.555+0000 7fcb1e3c0640 1 Processor -- start 2024-04-25T10:06:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.555+0000 7fcb1e3c0640 1 -- start start 2024-04-25T10:06:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1e3c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb00004d50 0x7fcb18160480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1e3c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb1815ef40 0x7fcb18151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1e3c0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb1814a940 0x7fcb1814f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1e3c0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1806db70 con 0x7fcb1814a940 2024-04-25T10:06:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1e3c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb18071ed0 con 0x7fcb1815ef40 2024-04-25T10:06:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1e3c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1806dcf0 con 0x7fcb00004d50 2024-04-25T10:06:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1c95d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb1814a940 0x7fcb1814f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1c95d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb1814a940 0x7fcb1814f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45496/0 (socket says 172.21.15.60:45496) 2024-04-25T10:06:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1c95d640 1 -- 172.21.15.60:0/2259546770 learned_addr learned my addr 172.21.15.60:0/2259546770 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1c95d640 1 -- 172.21.15.60:0/2259546770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb00004d50 msgr2=0x7fcb18160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1c95d640 1 --2- 172.21.15.60:0/2259546770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb00004d50 0x7fcb18160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1c95d640 1 -- 172.21.15.60:0/2259546770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb1815ef40 msgr2=0x7fcb18151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1c95d640 1 --2- 172.21.15.60:0/2259546770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb1815ef40 0x7fcb18151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1c95d640 1 -- 172.21.15.60:0/2259546770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1815f3e0 con 0x7fcb1814a940 2024-04-25T10:06:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1c95d640 1 --2- 172.21.15.60:0/2259546770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb1814a940 0x7fcb1814f050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcb100b0040 tx=0x7fcb10009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb14ff9640 1 -- 172.21.15.60:0/2259546770 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb100b2040 con 0x7fcb1814a940 2024-04-25T10:06:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb14ff9640 1 -- 172.21.15.60:0/2259546770 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb100ad440 con 0x7fcb1814a940 2024-04-25T10:06:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1e3c0640 1 -- 172.21.15.60:0/2259546770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1815f6c0 con 0x7fcb1814a940 2024-04-25T10:06:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb14ff9640 1 -- 172.21.15.60:0/2259546770 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10009c10 con 0x7fcb1814a940 2024-04-25T10:06:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1e3c0640 1 -- 172.21.15.60:0/2259546770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1815fc80 con 0x7fcb1814a940 2024-04-25T10:06:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb1e3c0640 1 -- 172.21.15.60:0/2259546770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb18074770 con 0x7fcb1814a940 2024-04-25T10:06:04.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb14ff9640 1 -- 172.21.15.60:0/2259546770 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb100be020 con 0x7fcb1814a940 2024-04-25T10:06:04.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb14ff9640 1 --2- 172.21.15.60:0/2259546770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcb04066f90 0x7fcb04069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb14ff9640 1 -- 172.21.15.60:0/2259546770 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fcb1011f320 con 0x7fcb1814a940 2024-04-25T10:06:04.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb17fff640 1 --2- 172.21.15.60:0/2259546770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcb04066f90 0x7fcb04069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.559+0000 7fcb17fff640 1 --2- 172.21.15.60:0/2259546770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcb04066f90 0x7fcb04069450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c001ec0 tx=0x7fcb0c001440 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.563+0000 7fcb14ff9640 1 -- 172.21.15.60:0/2259546770 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb100169d0 con 0x7fcb1814a940 2024-04-25T10:06:04.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.727+0000 7fcb1e3c0640 1 -- 172.21.15.60:0/2259546770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb1806ea20 con 0x7fcb1814a940 2024-04-25T10:06:04.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.727+0000 7fcb14ff9640 1 -- 172.21.15.60:0/2259546770 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb100f0330 con 0x7fcb1814a940 2024-04-25T10:06:04.735 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:06:04.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.731+0000 7fcb1e3c0640 1 -- 172.21.15.60:0/2259546770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcb04066f90 msgr2=0x7fcb04069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.731+0000 7fcb1e3c0640 1 --2- 172.21.15.60:0/2259546770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcb04066f90 0x7fcb04069450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c001ec0 tx=0x7fcb0c001440 comp rx=0 tx=0).stop 2024-04-25T10:06:04.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.731+0000 7fcb1e3c0640 1 -- 172.21.15.60:0/2259546770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb1814a940 msgr2=0x7fcb1814f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.731+0000 7fcb1e3c0640 1 --2- 172.21.15.60:0/2259546770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb1814a940 0x7fcb1814f050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcb100b0040 tx=0x7fcb10009750 comp rx=0 tx=0).stop 2024-04-25T10:06:04.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.731+0000 7fcb1e3c0640 1 -- 172.21.15.60:0/2259546770 shutdown_connections 2024-04-25T10:06:04.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.731+0000 7fcb1e3c0640 1 --2- 172.21.15.60:0/2259546770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcb04066f90 0x7fcb04069450 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.731+0000 7fcb1e3c0640 1 --2- 172.21.15.60:0/2259546770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb1814a940 0x7fcb1814f050 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.731+0000 7fcb1e3c0640 1 --2- 172.21.15.60:0/2259546770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb1815ef40 0x7fcb18151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.731+0000 7fcb1e3c0640 1 --2- 172.21.15.60:0/2259546770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb00004d50 0x7fcb18160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.731+0000 7fcb1e3c0640 1 -- 172.21.15.60:0/2259546770 >> 172.21.15.60:0/2259546770 conn(0x7fcb180670b0 msgr2=0x7fcb18157fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.731+0000 7fcb1e3c0640 1 -- 172.21.15.60:0/2259546770 shutdown_connections 2024-04-25T10:06:04.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.731+0000 7fcb1e3c0640 1 -- 172.21.15.60:0/2259546770 wait complete. 2024-04-25T10:06:04.754 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:06:04.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f9901640640 1 Processor -- start 2024-04-25T10:06:04.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f9901640640 1 -- start start 2024-04-25T10:06:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f9901640640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98fc06c330 0x7f98fc06ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f9901640640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98fc0b3370 con 0x7f98fc15b8d0 2024-04-25T10:06:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f9901640640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98fc0b3650 con 0x7f98fc0b2d90 2024-04-25T10:06:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f9901640640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc15c3c0 con 0x7f98fc06c330 2024-04-25T10:06:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98faffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f98fc15b8d0 0x7f98fc15bcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48924/0 (socket says 172.21.15.60:48924) 2024-04-25T10:06:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98faffd640 1 -- 172.21.15.60:0/3552695470 learned_addr learned my addr 172.21.15.60:0/3552695470 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98fb7fe640 1 --2- 172.21.15.60:0/3552695470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98fc06c330 0x7f98fc06ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98f9ffb640 1 -- 172.21.15.60:0/3552695470 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626051869 0 0) 0x7f98fc0b3370 con 0x7f98fc15b8d0 2024-04-25T10:06:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98f9ffb640 1 -- 172.21.15.60:0/3552695470 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98e4003530 con 0x7f98fc15b8d0 2024-04-25T10:06:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98f9ffb640 1 -- 172.21.15.60:0/3552695470 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2627253001 0 0) 0x7f98e4003530 con 0x7f98fc15b8d0 2024-04-25T10:06:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98f9ffb640 1 -- 172.21.15.60:0/3552695470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98fc06c330 msgr2=0x7f98fc06ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98f9ffb640 1 --2- 172.21.15.60:0/3552695470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98fc06c330 0x7f98fc06ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98f9ffb640 1 -- 172.21.15.60:0/3552695470 >> v1:172.21.15.100:6789/0 conn(0x7f98fc0b2d90 legacy=0x7f98fc06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98f9ffb640 1 -- 172.21.15.60:0/3552695470 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc0b3930 con 0x7f98fc15b8d0 2024-04-25T10:06:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98f9ffb640 1 -- 172.21.15.60:0/3552695470 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f98ec001680 con 0x7f98fc15b8d0 2024-04-25T10:06:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98f9ffb640 1 -- 172.21.15.60:0/3552695470 >> v1:172.21.15.60:6789/0 conn(0x7f98fc15b8d0 legacy=0x7f98fc15bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98f9ffb640 1 --2- 172.21.15.60:0/3552695470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e40039f0 0x7f98e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98f9ffb640 1 --2- 172.21.15.60:0/3552695470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e4004810 0x7f98e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98f9ffb640 1 --2- 172.21.15.60:0/3552695470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98fc06c330 0x7f98e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98f9ffb640 1 -- 172.21.15.60:0/3552695470 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e400ba80 con 0x7f98e4004810 2024-04-25T10:06:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98f9ffb640 1 -- 172.21.15.60:0/3552695470 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e400bc30 con 0x7f98e40039f0 2024-04-25T10:06:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98f9ffb640 1 -- 172.21.15.60:0/3552695470 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e400be30 con 0x7f98fc06c330 2024-04-25T10:06:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98fa7fc640 1 --2- 172.21.15.60:0/3552695470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e4004810 0x7f98e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98faffd640 1 --2- 172.21.15.60:0/3552695470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e40039f0 0x7f98e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98fb7fe640 1 --2- 172.21.15.60:0/3552695470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98fc06c330 0x7f98e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98fa7fc640 1 -- 172.21.15.60:0/3552695470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98fc06c330 msgr2=0x7f98e40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98fa7fc640 1 --2- 172.21.15.60:0/3552695470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98fc06c330 0x7f98e40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98fa7fc640 1 -- 172.21.15.60:0/3552695470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e40039f0 msgr2=0x7f98e4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.875+0000 7f98fa7fc640 1 --2- 172.21.15.60:0/3552695470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e40039f0 0x7f98e4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.879+0000 7f98fa7fc640 1 -- 172.21.15.60:0/3552695470 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e4003530 con 0x7f98e4004810 2024-04-25T10:06:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.879+0000 7f98fa7fc640 1 --2- 172.21.15.60:0/3552695470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e4004810 0x7f98e4004c90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f98f0001750 tx=0x7f98f001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926d9b306a1d1c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.879+0000 7f98f9ffb640 1 -- 172.21.15.60:0/3552695470 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f00013a0 con 0x7f98e4004810 2024-04-25T10:06:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.879+0000 7f98f9ffb640 1 -- 172.21.15.60:0/3552695470 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f00052f0 con 0x7f98e4004810 2024-04-25T10:06:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.879+0000 7f98f9ffb640 1 -- 172.21.15.60:0/3552695470 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0003840 con 0x7f98e4004810 2024-04-25T10:06:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.879+0000 7f9901640640 1 -- 172.21.15.60:0/3552695470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e4004810 msgr2=0x7f98e4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.879+0000 7f9901640640 1 --2- 172.21.15.60:0/3552695470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e4004810 0x7f98e4004c90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f98f0001750 tx=0x7f98f001c3c0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.879+0000 7f98fb7fe640 1 -- 172.21.15.60:0/3552695470 reap_dead start 2024-04-25T10:06:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.879+0000 7f9901640640 1 -- 172.21.15.60:0/3552695470 shutdown_connections 2024-04-25T10:06:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.879+0000 7f9901640640 1 -- 172.21.15.60:0/3552695470 >> 172.21.15.60:0/3552695470 conn(0x7f98fc071b70 msgr2=0x7f98fc071f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.879+0000 7f9901640640 1 -- 172.21.15.60:0/3552695470 shutdown_connections 2024-04-25T10:06:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.879+0000 7f9901640640 1 -- 172.21.15.60:0/3552695470 wait complete. 2024-04-25T10:06:04.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f9901640640 1 Processor -- start 2024-04-25T10:06:04.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f9901640640 1 -- start start 2024-04-25T10:06:04.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f9901640640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e4004810 0x7f98fc0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f9901640640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98fc1f3fc0 0x7f98fc15cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f9901640640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc15d240 0x7f98fc0acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f9901640640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc0af2e0 con 0x7f98e4004810 2024-04-25T10:06:04.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f9901640640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc0b3ef0 con 0x7f98fc15d240 2024-04-25T10:06:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f9901640640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc1504b0 con 0x7f98fc1f3fc0 2024-04-25T10:06:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98faffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e4004810 0x7f98fc0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98faffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e4004810 0x7f98fc0ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45506/0 (socket says 172.21.15.60:45506) 2024-04-25T10:06:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98faffd640 1 -- 172.21.15.60:0/3729219322 learned_addr learned my addr 172.21.15.60:0/3729219322 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98fa7fc640 1 --2- 172.21.15.60:0/3729219322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98fc1f3fc0 0x7f98fc15cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98fb7fe640 1 --2- 172.21.15.60:0/3729219322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc15d240 0x7f98fc0acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98faffd640 1 -- 172.21.15.60:0/3729219322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98fc1f3fc0 msgr2=0x7f98fc15cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98faffd640 1 --2- 172.21.15.60:0/3729219322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98fc1f3fc0 0x7f98fc15cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98faffd640 1 -- 172.21.15.60:0/3729219322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc15d240 msgr2=0x7f98fc0acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98faffd640 1 --2- 172.21.15.60:0/3729219322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc15d240 0x7f98fc0acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98faffd640 1 -- 172.21.15.60:0/3729219322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc15d6e0 con 0x7f98e4004810 2024-04-25T10:06:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98fa7fc640 1 --2- 172.21.15.60:0/3729219322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98fc1f3fc0 0x7f98fc15cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98fb7fe640 1 --2- 172.21.15.60:0/3729219322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc15d240 0x7f98fc0acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98faffd640 1 --2- 172.21.15.60:0/3729219322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e4004810 0x7f98fc0ce940 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f98ec011790 tx=0x7f98ec011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98e37fe640 1 -- 172.21.15.60:0/3729219322 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec010040 con 0x7f98e4004810 2024-04-25T10:06:04.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98e37fe640 1 -- 172.21.15.60:0/3729219322 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec01a070 con 0x7f98e4004810 2024-04-25T10:06:04.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98e37fe640 1 -- 172.21.15.60:0/3729219322 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec011ea0 con 0x7f98e4004810 2024-04-25T10:06:04.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f9901640640 1 -- 172.21.15.60:0/3729219322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98fc150700 con 0x7f98e4004810 2024-04-25T10:06:04.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f9901640640 1 -- 172.21.15.60:0/3729219322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc150cc0 con 0x7f98e4004810 2024-04-25T10:06:04.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f9901640640 1 -- 172.21.15.60:0/3729219322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98fc074770 con 0x7f98e4004810 2024-04-25T10:06:04.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.883+0000 7f98e37fe640 1 -- 172.21.15.60:0/3729219322 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98ec01f460 con 0x7f98e4004810 2024-04-25T10:06:04.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.887+0000 7f98e37fe640 1 --2- 172.21.15.60:0/3729219322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f98e8066f60 0x7f98e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.887+0000 7f98e37fe640 1 -- 172.21.15.60:0/3729219322 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f98ec088260 con 0x7f98e4004810 2024-04-25T10:06:04.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.887+0000 7f98fa7fc640 1 --2- 172.21.15.60:0/3729219322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f98e8066f60 0x7f98e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.887+0000 7f98fa7fc640 1 --2- 172.21.15.60:0/3729219322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f98e8066f60 0x7f98e8069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f98e40057f0 tx=0x7f98f00011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:04.891+0000 7f98e37fe640 1 -- 172.21.15.60:0/3729219322 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f98ec0591e0 con 0x7f98e4004810 2024-04-25T10:06:05.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.063+0000 7f9901640640 1 -- 172.21.15.60:0/3729219322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f98fc0afd40 con 0x7f98e4004810 2024-04-25T10:06:05.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.063+0000 7f98e37fe640 1 -- 172.21.15.60:0/3729219322 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f98ec058ae0 con 0x7f98e4004810 2024-04-25T10:06:05.070 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:06:05.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.063+0000 7f9901640640 1 -- 172.21.15.60:0/3729219322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f98e8066f60 msgr2=0x7f98e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.063+0000 7f9901640640 1 --2- 172.21.15.60:0/3729219322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f98e8066f60 0x7f98e8069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f98e40057f0 tx=0x7f98f00011a0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.063+0000 7f9901640640 1 -- 172.21.15.60:0/3729219322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e4004810 msgr2=0x7f98fc0ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.063+0000 7f9901640640 1 --2- 172.21.15.60:0/3729219322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e4004810 0x7f98fc0ce940 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f98ec011790 tx=0x7f98ec011c50 comp rx=0 tx=0).stop 2024-04-25T10:06:05.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.067+0000 7f9901640640 1 -- 172.21.15.60:0/3729219322 shutdown_connections 2024-04-25T10:06:05.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.067+0000 7f9901640640 1 --2- 172.21.15.60:0/3729219322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f98e8066f60 0x7f98e8069420 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.067+0000 7f9901640640 1 --2- 172.21.15.60:0/3729219322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc15d240 0x7f98fc0acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.067+0000 7f9901640640 1 --2- 172.21.15.60:0/3729219322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98fc1f3fc0 0x7f98fc15cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.067+0000 7f9901640640 1 --2- 172.21.15.60:0/3729219322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e4004810 0x7f98fc0ce940 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.067+0000 7f9901640640 1 -- 172.21.15.60:0/3729219322 >> 172.21.15.60:0/3729219322 conn(0x7f98fc071b70 msgr2=0x7f98fc0b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:05.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.067+0000 7f9901640640 1 -- 172.21.15.60:0/3729219322 shutdown_connections 2024-04-25T10:06:05.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.067+0000 7f9901640640 1 -- 172.21.15.60:0/3729219322 wait complete. 2024-04-25T10:06:05.089 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:06:05.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:05.089 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-25T10:06:05.101 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:05.102 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:06:05.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:05.102 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:06:05.151 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:06:05.151 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:05.158 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:06:05.158 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:05.204 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:05.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f43dc5640 1 Processor -- start 2024-04-25T10:06:05.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f43dc5640 1 -- start start 2024-04-25T10:06:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f43dc5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f3c15a7a0 0x7f5f3c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f43dc5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f3c06ec30 0x7f5f3c071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f43dc5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3c0721e0 con 0x7f5f3c0b37e0 2024-04-25T10:06:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f43dc5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3c0cdaa0 con 0x7f5f3c15a7a0 2024-04-25T10:06:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f43dc5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c06d2f0 con 0x7f5f3c06ec30 2024-04-25T10:06:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f42362640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5f3c0b37e0 0x7f5f3c06dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48936/0 (socket says 172.21.15.60:48936) 2024-04-25T10:06:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f41b61640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f3c15a7a0 0x7f5f3c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f42362640 1 -- 172.21.15.60:0/2769696830 learned_addr learned my addr 172.21.15.60:0/2769696830 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f41360640 1 --2- 172.21.15.60:0/2769696830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f3c06ec30 0x7f5f3c071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f41b61640 1 -- 172.21.15.60:0/2769696830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f3c06ec30 msgr2=0x7f5f3c071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f41b61640 1 --2- 172.21.15.60:0/2769696830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f3c06ec30 0x7f5f3c071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f41b61640 1 -- 172.21.15.60:0/2769696830 >> v1:172.21.15.60:6789/0 conn(0x7f5f3c0b37e0 legacy=0x7f5f3c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f41b61640 1 -- 172.21.15.60:0/2769696830 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c06f130 con 0x7f5f3c15a7a0 2024-04-25T10:06:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f40b5f640 1 -- 172.21.15.60:0/2769696830 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106281217 0 0) 0x7f5f3c0721e0 con 0x7f5f3c0b37e0 2024-04-25T10:06:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f41360640 1 --2- 172.21.15.60:0/2769696830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f3c06ec30 0x7f5f3c071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:05.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f41b61640 1 --2- 172.21.15.60:0/2769696830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c15a7a0 0x7f5f3c0cd560 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5f380097c0 tx=0x7f5f3801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14d3786c3bee515 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.331+0000 7f5f40b5f640 1 -- 172.21.15.60:0/2769696830 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f380b1070 con 0x7f5f3c15a7a0 2024-04-25T10:06:05.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f40b5f640 1 -- 172.21.15.60:0/2769696830 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f380ac410 con 0x7f5f3c15a7a0 2024-04-25T10:06:05.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f40b5f640 1 -- 172.21.15.60:0/2769696830 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f380ac7e0 con 0x7f5f3c15a7a0 2024-04-25T10:06:05.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 -- 172.21.15.60:0/2769696830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c15a7a0 msgr2=0x7f5f3c0cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 --2- 172.21.15.60:0/2769696830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c15a7a0 0x7f5f3c0cd560 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5f380097c0 tx=0x7f5f3801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 -- 172.21.15.60:0/2769696830 shutdown_connections 2024-04-25T10:06:05.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 --2- 172.21.15.60:0/2769696830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f3c06ec30 0x7f5f3c071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 --2- 172.21.15.60:0/2769696830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c15a7a0 0x7f5f3c0cd560 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 -- 172.21.15.60:0/2769696830 >> 172.21.15.60:0/2769696830 conn(0x7f5f3c0b12d0 msgr2=0x7f5f3c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:05.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 -- 172.21.15.60:0/2769696830 shutdown_connections 2024-04-25T10:06:05.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 -- 172.21.15.60:0/2769696830 wait complete. 2024-04-25T10:06:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 Processor -- start 2024-04-25T10:06:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 -- start start 2024-04-25T10:06:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f3c06ec30 0x7f5f3c150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c0b37e0 0x7f5f3c0ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f3c15a7a0 0x7f5f3c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c15ad00 con 0x7f5f3c0b37e0 2024-04-25T10:06:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c15ab80 con 0x7f5f3c06ec30 2024-04-25T10:06:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c06e860 con 0x7f5f3c15a7a0 2024-04-25T10:06:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f41360640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c0b37e0 0x7f5f3c0ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f41360640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c0b37e0 0x7f5f3c0ce260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45524/0 (socket says 172.21.15.60:45524) 2024-04-25T10:06:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f41360640 1 -- 172.21.15.60:0/747101420 learned_addr learned my addr 172.21.15.60:0/747101420 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f41b61640 1 --2- 172.21.15.60:0/747101420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f3c06ec30 0x7f5f3c150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f42362640 1 --2- 172.21.15.60:0/747101420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f3c15a7a0 0x7f5f3c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f41360640 1 -- 172.21.15.60:0/747101420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f3c15a7a0 msgr2=0x7f5f3c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f41360640 1 --2- 172.21.15.60:0/747101420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f3c15a7a0 0x7f5f3c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f41360640 1 -- 172.21.15.60:0/747101420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f3c06ec30 msgr2=0x7f5f3c150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f41360640 1 --2- 172.21.15.60:0/747101420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f3c06ec30 0x7f5f3c150420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f41360640 1 -- 172.21.15.60:0/747101420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c152ef0 con 0x7f5f3c0b37e0 2024-04-25T10:06:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f41360640 1 --2- 172.21.15.60:0/747101420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c0b37e0 0x7f5f3c0ce260 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00c7f0 tx=0x7f5f2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f327fc640 1 -- 172.21.15.60:0/747101420 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c015070 con 0x7f5f3c0b37e0 2024-04-25T10:06:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f327fc640 1 -- 172.21.15.60:0/747101420 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c007820 con 0x7f5f3c0b37e0 2024-04-25T10:06:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 -- 172.21.15.60:0/747101420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3c151e50 con 0x7f5f3c0b37e0 2024-04-25T10:06:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f43dc5640 1 -- 172.21.15.60:0/747101420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3c152410 con 0x7f5f3c0b37e0 2024-04-25T10:06:05.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.335+0000 7f5f327fc640 1 -- 172.21.15.60:0/747101420 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c007c10 con 0x7f5f3c0b37e0 2024-04-25T10:06:05.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.339+0000 7f5f43dc5640 1 -- 172.21.15.60:0/747101420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f240052a0 con 0x7f5f3c0b37e0 2024-04-25T10:06:05.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.339+0000 7f5f327fc640 1 -- 172.21.15.60:0/747101420 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f2c018070 con 0x7f5f3c0b37e0 2024-04-25T10:06:05.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.339+0000 7f5f327fc640 1 --2- 172.21.15.60:0/747101420 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5f20066f90 0x7f5f20069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.339+0000 7f5f327fc640 1 -- 172.21.15.60:0/747101420 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f5f2c085150 con 0x7f5f3c0b37e0 2024-04-25T10:06:05.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.339+0000 7f5f41b61640 1 --2- 172.21.15.60:0/747101420 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5f20066f90 0x7f5f20069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.339+0000 7f5f41b61640 1 --2- 172.21.15.60:0/747101420 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5f20066f90 0x7f5f20069450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5f380ae000 tx=0x7f5f38008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.343+0000 7f5f327fc640 1 -- 172.21.15.60:0/747101420 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f2c056040 con 0x7f5f3c0b37e0 2024-04-25T10:06:05.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.503+0000 7f5f43dc5640 1 -- 172.21.15.60:0/747101420 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f24005860 con 0x7f5f3c0b37e0 2024-04-25T10:06:05.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.503+0000 7f5f327fc640 1 -- 172.21.15.60:0/747101420 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16857 (secure 0 0 0) 0x7f5f2c055940 con 0x7f5f3c0b37e0 2024-04-25T10:06:05.512 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:05.512 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:01.568957+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:05.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.507+0000 7f5f43dc5640 1 -- 172.21.15.60:0/747101420 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5f20066f90 msgr2=0x7f5f20069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.507+0000 7f5f43dc5640 1 --2- 172.21.15.60:0/747101420 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5f20066f90 0x7f5f20069450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5f380ae000 tx=0x7f5f38008040 comp rx=0 tx=0).stop 2024-04-25T10:06:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.507+0000 7f5f43dc5640 1 -- 172.21.15.60:0/747101420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c0b37e0 msgr2=0x7f5f3c0ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.507+0000 7f5f43dc5640 1 --2- 172.21.15.60:0/747101420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c0b37e0 0x7f5f3c0ce260 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00c7f0 tx=0x7f5f2c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.507+0000 7f5f43dc5640 1 -- 172.21.15.60:0/747101420 shutdown_connections 2024-04-25T10:06:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.507+0000 7f5f43dc5640 1 --2- 172.21.15.60:0/747101420 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5f20066f90 0x7f5f20069450 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.507+0000 7f5f43dc5640 1 --2- 172.21.15.60:0/747101420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f3c15a7a0 0x7f5f3c0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.507+0000 7f5f43dc5640 1 --2- 172.21.15.60:0/747101420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c0b37e0 0x7f5f3c0ce260 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.507+0000 7f5f43dc5640 1 --2- 172.21.15.60:0/747101420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f3c06ec30 0x7f5f3c150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.507+0000 7f5f43dc5640 1 -- 172.21.15.60:0/747101420 >> 172.21.15.60:0/747101420 conn(0x7f5f3c0b12d0 msgr2=0x7f5f3c0b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.507+0000 7f5f43dc5640 1 -- 172.21.15.60:0/747101420 shutdown_connections 2024-04-25T10:06:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.507+0000 7f5f43dc5640 1 -- 172.21.15.60:0/747101420 wait complete. 2024-04-25T10:06:05.532 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:18.976643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:19.405302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:05.532 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:05.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.647+0000 7f296ea15640 1 Processor -- start 2024-04-25T10:06:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.647+0000 7f296ea15640 1 -- start start 2024-04-25T10:06:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.647+0000 7f296ea15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29680aeaa0 0x7f2968166450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.647+0000 7f296ea15640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f29680b2f70 0x7f29680b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.647+0000 7f296ea15640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29680b38d0 con 0x7f2968162450 2024-04-25T10:06:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.647+0000 7f296ea15640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2968166990 con 0x7f29680b2f70 2024-04-25T10:06:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.647+0000 7f296ea15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2968162910 con 0x7f29680aeaa0 2024-04-25T10:06:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.647+0000 7f296cfb2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f29680b2f70 0x7f29680b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.647+0000 7f295f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2968162450 0x7f296806db70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48942/0 (socket says 172.21.15.60:48942) 2024-04-25T10:06:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.647+0000 7f295f7fe640 1 -- 172.21.15.60:0/381542146 learned_addr learned my addr 172.21.15.60:0/381542146 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.647+0000 7f296cfb2640 1 -- 172.21.15.60:0/381542146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29680aeaa0 msgr2=0x7f2968166450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.647+0000 7f296cfb2640 1 --2- 172.21.15.60:0/381542146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29680aeaa0 0x7f2968166450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.647+0000 7f296cfb2640 1 -- 172.21.15.60:0/381542146 >> v1:172.21.15.60:6789/0 conn(0x7f2968162450 legacy=0x7f296806db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.647+0000 7f296cfb2640 1 -- 172.21.15.60:0/381542146 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2968162bf0 con 0x7f29680b2f70 2024-04-25T10:06:05.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.647+0000 7f29677fe640 1 -- 172.21.15.60:0/381542146 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824633895 0 0) 0x7f29680b38d0 con 0x7f2968162450 2024-04-25T10:06:05.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.651+0000 7f296cfb2640 1 --2- 172.21.15.60:0/381542146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29680b2f70 0x7f29680b3390 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2960009d90 tx=0x7f296001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67215545de7d462 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.651+0000 7f29677fe640 1 -- 172.21.15.60:0/381542146 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29600b2070 con 0x7f29680b2f70 2024-04-25T10:06:05.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.651+0000 7f29677fe640 1 -- 172.21.15.60:0/381542146 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29600a5920 con 0x7f29680b2f70 2024-04-25T10:06:05.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.651+0000 7f29677fe640 1 -- 172.21.15.60:0/381542146 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29600a5cf0 con 0x7f29680b2f70 2024-04-25T10:06:05.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.651+0000 7f296ea15640 1 -- 172.21.15.60:0/381542146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29680b2f70 msgr2=0x7f29680b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.651+0000 7f296ea15640 1 --2- 172.21.15.60:0/381542146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29680b2f70 0x7f29680b3390 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2960009d90 tx=0x7f296001c970 comp rx=0 tx=0).stop 2024-04-25T10:06:05.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.651+0000 7f296ea15640 1 -- 172.21.15.60:0/381542146 shutdown_connections 2024-04-25T10:06:05.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.651+0000 7f296ea15640 1 --2- 172.21.15.60:0/381542146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29680b2f70 0x7f29680b3390 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.651+0000 7f296ea15640 1 --2- 172.21.15.60:0/381542146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29680aeaa0 0x7f2968166450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.651+0000 7f296ea15640 1 -- 172.21.15.60:0/381542146 >> 172.21.15.60:0/381542146 conn(0x7f29680670b0 msgr2=0x7f296806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:05.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.651+0000 7f296ea15640 1 -- 172.21.15.60:0/381542146 shutdown_connections 2024-04-25T10:06:05.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.651+0000 7f296ea15640 1 -- 172.21.15.60:0/381542146 wait complete. 2024-04-25T10:06:05.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296ea15640 1 Processor -- start 2024-04-25T10:06:05.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296ea15640 1 -- start start 2024-04-25T10:06:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296ea15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29680aeaa0 0x7f2968157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296ea15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29680b2f70 0x7f296815dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296ea15640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2968162450 0x7f296815e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296ea15640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29680af000 con 0x7f2968162450 2024-04-25T10:06:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296ea15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29680aee80 con 0x7f29680aeaa0 2024-04-25T10:06:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296ea15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2968071e20 con 0x7f29680b2f70 2024-04-25T10:06:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296cfb2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2968162450 0x7f296815e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296cfb2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2968162450 0x7f296815e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45542/0 (socket says 172.21.15.60:45542) 2024-04-25T10:06:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296cfb2640 1 -- 172.21.15.60:0/3420937921 learned_addr learned my addr 172.21.15.60:0/3420937921 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f2967fff640 1 --2- 172.21.15.60:0/3420937921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29680aeaa0 0x7f2968157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f295f7fe640 1 --2- 172.21.15.60:0/3420937921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29680b2f70 0x7f296815dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296cfb2640 1 -- 172.21.15.60:0/3420937921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29680b2f70 msgr2=0x7f296815dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296cfb2640 1 --2- 172.21.15.60:0/3420937921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29680b2f70 0x7f296815dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296cfb2640 1 -- 172.21.15.60:0/3420937921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29680aeaa0 msgr2=0x7f2968157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296cfb2640 1 --2- 172.21.15.60:0/3420937921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29680aeaa0 0x7f2968157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296cfb2640 1 -- 172.21.15.60:0/3420937921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2968158390 con 0x7f2968162450 2024-04-25T10:06:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f2967fff640 1 --2- 172.21.15.60:0/3420937921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29680aeaa0 0x7f2968157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296cfb2640 1 --2- 172.21.15.60:0/3420937921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2968162450 0x7f296815e180 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f29600b0040 tx=0x7f2960019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f29657fa640 1 -- 172.21.15.60:0/3420937921 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29600b2040 con 0x7f2968162450 2024-04-25T10:06:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f29657fa640 1 -- 172.21.15.60:0/3420937921 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29600bf070 con 0x7f2968162450 2024-04-25T10:06:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296ea15640 1 -- 172.21.15.60:0/3420937921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2968157af0 con 0x7f2968162450 2024-04-25T10:06:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f29657fa640 1 -- 172.21.15.60:0/3420937921 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29600ad9a0 con 0x7f2968162450 2024-04-25T10:06:05.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296ea15640 1 -- 172.21.15.60:0/3420937921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29681580b0 con 0x7f2968162450 2024-04-25T10:06:05.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.655+0000 7f296ea15640 1 -- 172.21.15.60:0/3420937921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296806bba0 con 0x7f2968162450 2024-04-25T10:06:05.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.659+0000 7f29657fa640 1 -- 172.21.15.60:0/3420937921 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29600adb40 con 0x7f2968162450 2024-04-25T10:06:05.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.659+0000 7f29657fa640 1 --2- 172.21.15.60:0/3420937921 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2948066f90 0x7f2948069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.659+0000 7f29657fa640 1 -- 172.21.15.60:0/3420937921 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f29601206e0 con 0x7f2968162450 2024-04-25T10:06:05.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.659+0000 7f2967fff640 1 --2- 172.21.15.60:0/3420937921 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2948066f90 0x7f2948069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.659+0000 7f2967fff640 1 --2- 172.21.15.60:0/3420937921 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2948066f90 0x7f2948069450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f295800a8b0 tx=0x7f2958008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.663+0000 7f29657fa640 1 -- 172.21.15.60:0/3420937921 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f29600c8020 con 0x7f2968162450 2024-04-25T10:06:05.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.819+0000 7f296ea15640 1 -- 172.21.15.60:0/3420937921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f296806b0d0 con 0x7f2968162450 2024-04-25T10:06:05.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.823+0000 7f29657fa640 1 -- 172.21.15.60:0/3420937921 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2960016560 con 0x7f2968162450 2024-04-25T10:06:05.830 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.823+0000 7f296ea15640 1 -- 172.21.15.60:0/3420937921 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2948066f90 msgr2=0x7f2948069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.823+0000 7f296ea15640 1 --2- 172.21.15.60:0/3420937921 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2948066f90 0x7f2948069450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f295800a8b0 tx=0x7f2958008040 comp rx=0 tx=0).stop 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.823+0000 7f296ea15640 1 -- 172.21.15.60:0/3420937921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2968162450 msgr2=0x7f296815e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.823+0000 7f296ea15640 1 --2- 172.21.15.60:0/3420937921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2968162450 0x7f296815e180 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f29600b0040 tx=0x7f2960019710 comp rx=0 tx=0).stop 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.823+0000 7f296ea15640 1 -- 172.21.15.60:0/3420937921 shutdown_connections 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.823+0000 7f296ea15640 1 --2- 172.21.15.60:0/3420937921 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2948066f90 0x7f2948069450 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.823+0000 7f296ea15640 1 --2- 172.21.15.60:0/3420937921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2968162450 0x7f296815e180 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.823+0000 7f296ea15640 1 --2- 172.21.15.60:0/3420937921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29680b2f70 0x7f296815dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.823+0000 7f296ea15640 1 --2- 172.21.15.60:0/3420937921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29680aeaa0 0x7f2968157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.823+0000 7f296ea15640 1 -- 172.21.15.60:0/3420937921 >> 172.21.15.60:0/3420937921 conn(0x7f29680670b0 msgr2=0x7f296815cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.823+0000 7f296ea15640 1 -- 172.21.15.60:0/3420937921 shutdown_connections 2024-04-25T10:06:05.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.823+0000 7f296ea15640 1 -- 172.21.15.60:0/3420937921 wait complete. 2024-04-25T10:06:05.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:05.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.963+0000 7fe8b923f640 1 Processor -- start 2024-04-25T10:06:05.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.967+0000 7fe8b923f640 1 -- start start 2024-04-25T10:06:05.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.967+0000 7fe8b923f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe8b40cf8a0 0x7fe8b415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.967+0000 7fe8b923f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8b40cd3d0 0x7fe8b415be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.967+0000 7fe8b923f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b40cd890 con 0x7fe8b415dd30 2024-04-25T10:06:05.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.967+0000 7fe8b923f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b40ce4a0 con 0x7fe8b40cf8a0 2024-04-25T10:06:05.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.967+0000 7fe8b923f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40cdb70 con 0x7fe8b40cd3d0 2024-04-25T10:06:05.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.967+0000 7fe8b259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe8b40cf8a0 0x7fe8b415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.967+0000 7fe8b2d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe8b415dd30 0x7fe8b40cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48946/0 (socket says 172.21.15.60:48946) 2024-04-25T10:06:05.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.967+0000 7fe8b2d9d640 1 -- 172.21.15.60:0/5374338 learned_addr learned my addr 172.21.15.60:0/5374338 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:05.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.967+0000 7fe8b359e640 1 --2- 172.21.15.60:0/5374338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8b40cd3d0 0x7fe8b415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.967+0000 7fe8b259c640 1 -- 172.21.15.60:0/5374338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8b40cd3d0 msgr2=0x7fe8b415be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:05.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.967+0000 7fe8b259c640 1 --2- 172.21.15.60:0/5374338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8b40cd3d0 0x7fe8b415be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.967+0000 7fe8b259c640 1 -- 172.21.15.60:0/5374338 >> v1:172.21.15.60:6789/0 conn(0x7fe8b415dd30 legacy=0x7fe8b40cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.967+0000 7fe8b259c640 1 -- 172.21.15.60:0/5374338 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b40cde50 con 0x7fe8b40cf8a0 2024-04-25T10:06:05.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.967+0000 7fe8b259c640 1 --2- 172.21.15.60:0/5374338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8b40cf8a0 0x7fe8b415ac30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40097c0 tx=0x7fe8a401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278fda848c67a3ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b1d9b640 1 -- 172.21.15.60:0/5374338 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40ab600 con 0x7fe8b40cf8a0 2024-04-25T10:06:05.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b1d9b640 1 -- 172.21.15.60:0/5374338 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a40b0070 con 0x7fe8b40cf8a0 2024-04-25T10:06:05.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b1d9b640 1 -- 172.21.15.60:0/5374338 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40ab9c0 con 0x7fe8b40cf8a0 2024-04-25T10:06:05.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 -- 172.21.15.60:0/5374338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8b40cf8a0 msgr2=0x7fe8b415ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 --2- 172.21.15.60:0/5374338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8b40cf8a0 0x7fe8b415ac30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40097c0 tx=0x7fe8a401c3e0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 -- 172.21.15.60:0/5374338 shutdown_connections 2024-04-25T10:06:05.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 --2- 172.21.15.60:0/5374338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8b40cd3d0 0x7fe8b415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 --2- 172.21.15.60:0/5374338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8b40cf8a0 0x7fe8b415ac30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 -- 172.21.15.60:0/5374338 >> 172.21.15.60:0/5374338 conn(0x7fe8b40670b0 msgr2=0x7fe8b406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:05.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 -- 172.21.15.60:0/5374338 shutdown_connections 2024-04-25T10:06:05.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 -- 172.21.15.60:0/5374338 wait complete. 2024-04-25T10:06:05.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 Processor -- start 2024-04-25T10:06:05.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 -- start start 2024-04-25T10:06:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8b40cd3d0 0x7fe8b4144e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8b40cf8a0 0x7fe8b41592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8b415dd30 0x7fe8b40ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b415cf40 con 0x7fe8b40cd3d0 2024-04-25T10:06:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b415cdc0 con 0x7fe8b415dd30 2024-04-25T10:06:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b413ea70 con 0x7fe8b40cf8a0 2024-04-25T10:06:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8b40cd3d0 0x7fe8b4144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8b40cd3d0 0x7fe8b4144e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45564/0 (socket says 172.21.15.60:45564) 2024-04-25T10:06:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b2d9d640 1 -- 172.21.15.60:0/3999566054 learned_addr learned my addr 172.21.15.60:0/3999566054 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b359e640 1 --2- 172.21.15.60:0/3999566054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8b415dd30 0x7fe8b40ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b259c640 1 --2- 172.21.15.60:0/3999566054 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8b40cf8a0 0x7fe8b41592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b2d9d640 1 -- 172.21.15.60:0/3999566054 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8b40cf8a0 msgr2=0x7fe8b41592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b2d9d640 1 --2- 172.21.15.60:0/3999566054 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8b40cf8a0 0x7fe8b41592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b2d9d640 1 -- 172.21.15.60:0/3999566054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8b415dd30 msgr2=0x7fe8b40ab9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b2d9d640 1 --2- 172.21.15.60:0/3999566054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8b415dd30 0x7fe8b40ab9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b2d9d640 1 -- 172.21.15.60:0/3999566054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b413eda0 con 0x7fe8b40cd3d0 2024-04-25T10:06:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b2d9d640 1 --2- 172.21.15.60:0/3999566054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8b40cd3d0 0x7fe8b4144e00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800c420 tx=0x7fe8a800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe89b7fe640 1 -- 172.21.15.60:0/3999566054 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8017070 con 0x7fe8b40cd3d0 2024-04-25T10:06:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe89b7fe640 1 -- 172.21.15.60:0/3999566054 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a8005cf0 con 0x7fe8b40cd3d0 2024-04-25T10:06:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 -- 172.21.15.60:0/3999566054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b413f080 con 0x7fe8b40cd3d0 2024-04-25T10:06:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe89b7fe640 1 -- 172.21.15.60:0/3999566054 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80072f0 con 0x7fe8b40cd3d0 2024-04-25T10:06:05.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.971+0000 7fe8b923f640 1 -- 172.21.15.60:0/3999566054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b41421f0 con 0x7fe8b40cd3d0 2024-04-25T10:06:05.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.975+0000 7fe8b923f640 1 -- 172.21.15.60:0/3999566054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b4074770 con 0x7fe8b40cd3d0 2024-04-25T10:06:05.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.975+0000 7fe89b7fe640 1 -- 172.21.15.60:0/3999566054 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8a801a070 con 0x7fe8b40cd3d0 2024-04-25T10:06:05.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.975+0000 7fe89b7fe640 1 --2- 172.21.15.60:0/3999566054 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8a0066f60 0x7fe8a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.975+0000 7fe89b7fe640 1 -- 172.21.15.60:0/3999566054 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fe8a8084400 con 0x7fe8b40cd3d0 2024-04-25T10:06:05.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.975+0000 7fe8b259c640 1 --2- 172.21.15.60:0/3999566054 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8a0066f60 0x7fe8a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.975+0000 7fe8b259c640 1 --2- 172.21.15.60:0/3999566054 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8a0066f60 0x7fe8a0069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40af000 tx=0x7fe8a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:05.979+0000 7fe89b7fe640 1 -- 172.21.15.60:0/3999566054 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8a8055410 con 0x7fe8b40cd3d0 2024-04-25T10:06:06.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.135+0000 7fe8b923f640 1 -- 172.21.15.60:0/3999566054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe8b415aaa0 con 0x7fe8b40cd3d0 2024-04-25T10:06:06.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.135+0000 7fe89b7fe640 1 -- 172.21.15.60:0/3999566054 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe8a8054d10 con 0x7fe8b40cd3d0 2024-04-25T10:06:06.142 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:06:06.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.139+0000 7fe8b923f640 1 -- 172.21.15.60:0/3999566054 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8a0066f60 msgr2=0x7fe8a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.139+0000 7fe8b923f640 1 --2- 172.21.15.60:0/3999566054 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8a0066f60 0x7fe8a0069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40af000 tx=0x7fe8a4008040 comp rx=0 tx=0).stop 2024-04-25T10:06:06.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.139+0000 7fe8b923f640 1 -- 172.21.15.60:0/3999566054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8b40cd3d0 msgr2=0x7fe8b4144e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.139+0000 7fe8b923f640 1 --2- 172.21.15.60:0/3999566054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8b40cd3d0 0x7fe8b4144e00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800c420 tx=0x7fe8a800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.139+0000 7fe8b923f640 1 -- 172.21.15.60:0/3999566054 shutdown_connections 2024-04-25T10:06:06.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.139+0000 7fe8b923f640 1 --2- 172.21.15.60:0/3999566054 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8a0066f60 0x7fe8a0069420 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.139+0000 7fe8b923f640 1 --2- 172.21.15.60:0/3999566054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8b415dd30 0x7fe8b40ab9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.139+0000 7fe8b923f640 1 --2- 172.21.15.60:0/3999566054 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8b40cf8a0 0x7fe8b41592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.139+0000 7fe8b923f640 1 --2- 172.21.15.60:0/3999566054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8b40cd3d0 0x7fe8b4144e00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.139+0000 7fe8b923f640 1 -- 172.21.15.60:0/3999566054 >> 172.21.15.60:0/3999566054 conn(0x7fe8b40670b0 msgr2=0x7fe8b406d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.139+0000 7fe8b923f640 1 -- 172.21.15.60:0/3999566054 shutdown_connections 2024-04-25T10:06:06.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.139+0000 7fe8b923f640 1 -- 172.21.15.60:0/3999566054 wait complete. 2024-04-25T10:06:06.160 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:06.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e9650d640 1 Processor -- start 2024-04-25T10:06:06.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e9650d640 1 -- start start 2024-04-25T10:06:06.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e9650d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e9006ac10 0x7f5e90070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e9650d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9006fb70 con 0x7f5e900d05a0 2024-04-25T10:06:06.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e9650d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9006b1d0 con 0x7f5e900b4dd0 2024-04-25T10:06:06.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e9650d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9006b4b0 con 0x7f5e9006ac10 2024-04-25T10:06:06.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5e900d05a0 0x7f5e900b51b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48958/0 (socket says 172.21.15.60:48958) 2024-04-25T10:06:06.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8f7fe640 1 -- 172.21.15.60:0/3982619873 learned_addr learned my addr 172.21.15.60:0/3982619873 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:06.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e94aaa640 1 --2- 172.21.15.60:0/3982619873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e9006ac10 0x7f5e90070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8effd640 1 -- 172.21.15.60:0/3982619873 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893715554 0 0) 0x7f5e9006fb70 con 0x7f5e900d05a0 2024-04-25T10:06:06.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8effd640 1 -- 172.21.15.60:0/3982619873 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e70003530 con 0x7f5e900d05a0 2024-04-25T10:06:06.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8effd640 1 -- 172.21.15.60:0/3982619873 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 975748365 0 0) 0x7f5e70003530 con 0x7f5e900d05a0 2024-04-25T10:06:06.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8effd640 1 -- 172.21.15.60:0/3982619873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e9006ac10 msgr2=0x7f5e90070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8effd640 1 --2- 172.21.15.60:0/3982619873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e9006ac10 0x7f5e90070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8effd640 1 -- 172.21.15.60:0/3982619873 >> v1:172.21.15.100:6789/0 conn(0x7f5e900b4dd0 legacy=0x7f5e9015be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8effd640 1 -- 172.21.15.60:0/3982619873 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9006b790 con 0x7f5e900d05a0 2024-04-25T10:06:06.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8effd640 1 -- 172.21.15.60:0/3982619873 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f5e78001680 con 0x7f5e900d05a0 2024-04-25T10:06:06.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8effd640 1 -- 172.21.15.60:0/3982619873 >> v1:172.21.15.60:6789/0 conn(0x7f5e900d05a0 legacy=0x7f5e900b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8effd640 1 --2- 172.21.15.60:0/3982619873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e9006ac10 0x7f5e700039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8effd640 1 --2- 172.21.15.60:0/3982619873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e70003f70 0x7f5e70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8effd640 1 --2- 172.21.15.60:0/3982619873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e70004d50 0x7f5e700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8effd640 1 -- 172.21.15.60:0/3982619873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9006b4b0 con 0x7f5e70004d50 2024-04-25T10:06:06.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8effd640 1 -- 172.21.15.60:0/3982619873 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7000bb60 con 0x7f5e70003f70 2024-04-25T10:06:06.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8effd640 1 -- 172.21.15.60:0/3982619873 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7000bd60 con 0x7f5e9006ac10 2024-04-25T10:06:06.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e94aaa640 1 --2- 172.21.15.60:0/3982619873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e70004d50 0x7f5e700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8ffff640 1 --2- 172.21.15.60:0/3982619873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e9006ac10 0x7f5e700039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e8f7fe640 1 --2- 172.21.15.60:0/3982619873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e70003f70 0x7f5e70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e94aaa640 1 -- 172.21.15.60:0/3982619873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e9006ac10 msgr2=0x7f5e700039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e94aaa640 1 --2- 172.21.15.60:0/3982619873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e9006ac10 0x7f5e700039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e94aaa640 1 -- 172.21.15.60:0/3982619873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e70003f70 msgr2=0x7f5e70004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e94aaa640 1 --2- 172.21.15.60:0/3982619873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e70003f70 0x7f5e70004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.283+0000 7f5e94aaa640 1 -- 172.21.15.60:0/3982619873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e70003530 con 0x7f5e70004d50 2024-04-25T10:06:06.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.287+0000 7f5e94aaa640 1 --2- 172.21.15.60:0/3982619873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e70004d50 0x7f5e700051d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009fc0 tx=0x7f5e8001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9150c9ca589f54a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.287+0000 7f5e8effd640 1 -- 172.21.15.60:0/3982619873 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e800b2070 con 0x7f5e70004d50 2024-04-25T10:06:06.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.287+0000 7f5e8effd640 1 -- 172.21.15.60:0/3982619873 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e800a5b80 con 0x7f5e70004d50 2024-04-25T10:06:06.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.287+0000 7f5e8effd640 1 -- 172.21.15.60:0/3982619873 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e80009c10 con 0x7f5e70004d50 2024-04-25T10:06:06.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.287+0000 7f5e9650d640 1 -- 172.21.15.60:0/3982619873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e70004d50 msgr2=0x7f5e700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.287+0000 7f5e9650d640 1 --2- 172.21.15.60:0/3982619873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e70004d50 0x7f5e700051d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009fc0 tx=0x7f5e8001cc80 comp rx=0 tx=0).stop 2024-04-25T10:06:06.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.287+0000 7f5e94aaa640 1 -- 172.21.15.60:0/3982619873 reap_dead start 2024-04-25T10:06:06.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.287+0000 7f5e9650d640 1 -- 172.21.15.60:0/3982619873 shutdown_connections 2024-04-25T10:06:06.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.287+0000 7f5e9650d640 1 -- 172.21.15.60:0/3982619873 >> 172.21.15.60:0/3982619873 conn(0x7f5e900b2360 msgr2=0x7f5e900b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.287+0000 7f5e9650d640 1 -- 172.21.15.60:0/3982619873 shutdown_connections 2024-04-25T10:06:06.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.287+0000 7f5e9650d640 1 -- 172.21.15.60:0/3982619873 wait complete. 2024-04-25T10:06:06.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.287+0000 7f5e9650d640 1 Processor -- start 2024-04-25T10:06:06.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.287+0000 7f5e9650d640 1 -- start start 2024-04-25T10:06:06.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e9650d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e70004d50 0x7f5e9013b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e9650d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9013d6e0 0x7f5e9013db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e9650d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e90139c50 0x7f5e9013a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e9650d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e90145390 con 0x7f5e9013d6e0 2024-04-25T10:06:06.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e9650d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9006ba70 con 0x7f5e70004d50 2024-04-25T10:06:06.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e9650d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e90145510 con 0x7f5e90139c50 2024-04-25T10:06:06.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9013d6e0 0x7f5e9013db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9013d6e0 0x7f5e9013db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45580/0 (socket says 172.21.15.60:45580) 2024-04-25T10:06:06.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8f7fe640 1 -- 172.21.15.60:0/539541194 learned_addr learned my addr 172.21.15.60:0/539541194 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:06.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e94aaa640 1 --2- 172.21.15.60:0/539541194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e90139c50 0x7f5e9013a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8ffff640 1 --2- 172.21.15.60:0/539541194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e70004d50 0x7f5e9013b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8f7fe640 1 -- 172.21.15.60:0/539541194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e90139c50 msgr2=0x7f5e9013a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8f7fe640 1 --2- 172.21.15.60:0/539541194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e90139c50 0x7f5e9013a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8f7fe640 1 -- 172.21.15.60:0/539541194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e70004d50 msgr2=0x7f5e9013b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8f7fe640 1 --2- 172.21.15.60:0/539541194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e70004d50 0x7f5e9013b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8f7fe640 1 -- 172.21.15.60:0/539541194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e90138bb0 con 0x7f5e9013d6e0 2024-04-25T10:06:06.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8f7fe640 1 --2- 172.21.15.60:0/539541194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9013d6e0 0x7f5e9013db60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5e78011980 tx=0x7f5e78011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8cff9640 1 -- 172.21.15.60:0/539541194 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78010040 con 0x7f5e9013d6e0 2024-04-25T10:06:06.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8cff9640 1 -- 172.21.15.60:0/539541194 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7801a070 con 0x7f5e9013d6e0 2024-04-25T10:06:06.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e9650d640 1 -- 172.21.15.60:0/539541194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e90138e90 con 0x7f5e9013d6e0 2024-04-25T10:06:06.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8cff9640 1 -- 172.21.15.60:0/539541194 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800fe40 con 0x7f5e9013d6e0 2024-04-25T10:06:06.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e9650d640 1 -- 172.21.15.60:0/539541194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e90139450 con 0x7f5e9013d6e0 2024-04-25T10:06:06.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e9650d640 1 -- 172.21.15.60:0/539541194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e90074770 con 0x7f5e9013d6e0 2024-04-25T10:06:06.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8cff9640 1 -- 172.21.15.60:0/539541194 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e7801f460 con 0x7f5e9013d6e0 2024-04-25T10:06:06.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8cff9640 1 --2- 172.21.15.60:0/539541194 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e74066f90 0x7f5e74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8cff9640 1 -- 172.21.15.60:0/539541194 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f5e78088300 con 0x7f5e9013d6e0 2024-04-25T10:06:06.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.291+0000 7f5e8ffff640 1 --2- 172.21.15.60:0/539541194 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e74066f90 0x7f5e74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.295+0000 7f5e8ffff640 1 --2- 172.21.15.60:0/539541194 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e74066f90 0x7f5e74069450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5e88005510 tx=0x7f5e88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.295+0000 7f5e8cff9640 1 -- 172.21.15.60:0/539541194 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5e78059280 con 0x7f5e9013d6e0 2024-04-25T10:06:06.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.451+0000 7f5e9650d640 1 -- 172.21.15.60:0/539541194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e90072d10 con 0x7f5e9013d6e0 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.451+0000 7f5e8cff9640 1 -- 172.21.15.60:0/539541194 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5e78058b80 con 0x7f5e9013d6e0 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:06:06.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.451+0000 7f5e9650d640 1 -- 172.21.15.60:0/539541194 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e74066f90 msgr2=0x7f5e74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.451+0000 7f5e9650d640 1 --2- 172.21.15.60:0/539541194 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e74066f90 0x7f5e74069450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5e88005510 tx=0x7f5e88009040 comp rx=0 tx=0).stop 2024-04-25T10:06:06.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.451+0000 7f5e9650d640 1 -- 172.21.15.60:0/539541194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9013d6e0 msgr2=0x7f5e9013db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.451+0000 7f5e9650d640 1 --2- 172.21.15.60:0/539541194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9013d6e0 0x7f5e9013db60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5e78011980 tx=0x7f5e78011e40 comp rx=0 tx=0).stop 2024-04-25T10:06:06.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.451+0000 7f5e9650d640 1 -- 172.21.15.60:0/539541194 shutdown_connections 2024-04-25T10:06:06.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.451+0000 7f5e9650d640 1 --2- 172.21.15.60:0/539541194 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e74066f90 0x7f5e74069450 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.451+0000 7f5e9650d640 1 --2- 172.21.15.60:0/539541194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e90139c50 0x7f5e9013a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.451+0000 7f5e9650d640 1 --2- 172.21.15.60:0/539541194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e9013d6e0 0x7f5e9013db60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.451+0000 7f5e9650d640 1 --2- 172.21.15.60:0/539541194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e70004d50 0x7f5e9013b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.451+0000 7f5e9650d640 1 -- 172.21.15.60:0/539541194 >> 172.21.15.60:0/539541194 conn(0x7f5e900b2360 msgr2=0x7f5e900b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.451+0000 7f5e9650d640 1 -- 172.21.15.60:0/539541194 shutdown_connections 2024-04-25T10:06:06.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.451+0000 7f5e9650d640 1 -- 172.21.15.60:0/539541194 wait complete. 2024-04-25T10:06:06.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:06:06.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.583+0000 7f9d4df5e640 1 Processor -- start 2024-04-25T10:06:06.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.583+0000 7f9d4df5e640 1 -- start start 2024-04-25T10:06:06.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.583+0000 7f9d4df5e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9d4815bc90 0x7f9d4815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.583+0000 7f9d4df5e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d48071ca0 con 0x7f9d480ce7a0 2024-04-25T10:06:06.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.583+0000 7f9d4df5e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d480cca40 con 0x7f9d480b4660 2024-04-25T10:06:06.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.583+0000 7f9d4df5e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d480cc370 con 0x7f9d4815bc90 2024-04-25T10:06:06.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.583+0000 7f9d46ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9d4815bc90 0x7f9d4815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.583+0000 7f9d477fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9d480ce7a0 0x7f9d480ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48964/0 (socket says 172.21.15.60:48964) 2024-04-25T10:06:06.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.583+0000 7f9d477fe640 1 -- 172.21.15.60:0/1923740009 learned_addr learned my addr 172.21.15.60:0/1923740009 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:06.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.583+0000 7f9d46ffd640 1 -- 172.21.15.60:0/1923740009 >> v1:172.21.15.100:6789/0 conn(0x7f9d480b4660 legacy=0x7f9d4815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.583+0000 7f9d46ffd640 1 -- 172.21.15.60:0/1923740009 >> v1:172.21.15.60:6789/0 conn(0x7f9d480ce7a0 legacy=0x7f9d480ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.583+0000 7f9d46ffd640 1 -- 172.21.15.60:0/1923740009 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d480ccd20 con 0x7f9d4815bc90 2024-04-25T10:06:06.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.587+0000 7f9d46ffd640 1 --2- 172.21.15.60:0/1923740009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d4815bc90 0x7f9d4815c220 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0097c0 tx=0x7f9d3c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9624d3993eef1c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.587+0000 7f9d467fc640 1 -- 172.21.15.60:0/1923740009 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0b0070 con 0x7f9d4815bc90 2024-04-25T10:06:06.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.587+0000 7f9d467fc640 1 -- 172.21.15.60:0/1923740009 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c0ab600 con 0x7f9d4815bc90 2024-04-25T10:06:06.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.587+0000 7f9d467fc640 1 -- 172.21.15.60:0/1923740009 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0ab9d0 con 0x7f9d4815bc90 2024-04-25T10:06:06.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.587+0000 7f9d4df5e640 1 -- 172.21.15.60:0/1923740009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d4815bc90 msgr2=0x7f9d4815c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.587+0000 7f9d4df5e640 1 --2- 172.21.15.60:0/1923740009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d4815bc90 0x7f9d4815c220 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0097c0 tx=0x7f9d3c01c460 comp rx=0 tx=0).stop 2024-04-25T10:06:06.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.587+0000 7f9d4df5e640 1 -- 172.21.15.60:0/1923740009 shutdown_connections 2024-04-25T10:06:06.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.587+0000 7f9d4df5e640 1 --2- 172.21.15.60:0/1923740009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d4815bc90 0x7f9d4815c220 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.587+0000 7f9d4df5e640 1 -- 172.21.15.60:0/1923740009 >> 172.21.15.60:0/1923740009 conn(0x7f9d480b12d0 msgr2=0x7f9d480b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.587+0000 7f9d4df5e640 1 -- 172.21.15.60:0/1923740009 shutdown_connections 2024-04-25T10:06:06.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.587+0000 7f9d4df5e640 1 -- 172.21.15.60:0/1923740009 wait complete. 2024-04-25T10:06:06.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d4df5e640 1 Processor -- start 2024-04-25T10:06:06.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d4df5e640 1 -- start start 2024-04-25T10:06:06.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d4df5e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d480ce7a0 0x7f9d48146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d4df5e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d4815bc90 0x7f9d4815b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d4df5e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d4806d9f0 0x7f9d480b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d4df5e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d481695d0 con 0x7f9d4806d9f0 2024-04-25T10:06:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d4df5e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d480b3db0 con 0x7f9d480ce7a0 2024-04-25T10:06:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d4df5e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d48169750 con 0x7f9d4815bc90 2024-04-25T10:06:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d47fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d4806d9f0 0x7f9d480b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d47fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d4806d9f0 0x7f9d480b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45600/0 (socket says 172.21.15.60:45600) 2024-04-25T10:06:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d47fff640 1 -- 172.21.15.60:0/1592244483 learned_addr learned my addr 172.21.15.60:0/1592244483 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d477fe640 1 --2- 172.21.15.60:0/1592244483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d480ce7a0 0x7f9d48146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d46ffd640 1 --2- 172.21.15.60:0/1592244483 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d4815bc90 0x7f9d4815b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d47fff640 1 -- 172.21.15.60:0/1592244483 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d4815bc90 msgr2=0x7f9d4815b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d47fff640 1 --2- 172.21.15.60:0/1592244483 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d4815bc90 0x7f9d4815b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d47fff640 1 -- 172.21.15.60:0/1592244483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d480ce7a0 msgr2=0x7f9d48146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d47fff640 1 --2- 172.21.15.60:0/1592244483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d480ce7a0 0x7f9d48146d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d47fff640 1 -- 172.21.15.60:0/1592244483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d48157ee0 con 0x7f9d4806d9f0 2024-04-25T10:06:06.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d47fff640 1 --2- 172.21.15.60:0/1592244483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d4806d9f0 0x7f9d480b0130 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000b9e0 tx=0x7f9d4000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d2ffff640 1 -- 172.21.15.60:0/1592244483 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d40019070 con 0x7f9d4806d9f0 2024-04-25T10:06:06.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d2ffff640 1 -- 172.21.15.60:0/1592244483 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4000cb20 con 0x7f9d4806d9f0 2024-04-25T10:06:06.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d2ffff640 1 -- 172.21.15.60:0/1592244483 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d40014440 con 0x7f9d4806d9f0 2024-04-25T10:06:06.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d4df5e640 1 -- 172.21.15.60:0/1592244483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4806de80 con 0x7f9d4806d9f0 2024-04-25T10:06:06.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d4df5e640 1 -- 172.21.15.60:0/1592244483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d48149840 con 0x7f9d4806d9f0 2024-04-25T10:06:06.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d4df5e640 1 -- 172.21.15.60:0/1592244483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d48074770 con 0x7f9d4806d9f0 2024-04-25T10:06:06.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.591+0000 7f9d2ffff640 1 -- 172.21.15.60:0/1592244483 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d4001c070 con 0x7f9d4806d9f0 2024-04-25T10:06:06.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.595+0000 7f9d2ffff640 1 --2- 172.21.15.60:0/1592244483 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d34066f60 0x7f9d34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.595+0000 7f9d2ffff640 1 -- 172.21.15.60:0/1592244483 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9d400851f0 con 0x7f9d4806d9f0 2024-04-25T10:06:06.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.595+0000 7f9d477fe640 1 --2- 172.21.15.60:0/1592244483 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d34066f60 0x7f9d34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.595+0000 7f9d477fe640 1 --2- 172.21.15.60:0/1592244483 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d34066f60 0x7f9d34069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9d38000f70 tx=0x7f9d38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.599+0000 7f9d2ffff640 1 -- 172.21.15.60:0/1592244483 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d40023020 con 0x7f9d4806d9f0 2024-04-25T10:06:06.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.767+0000 7f9d4df5e640 1 -- 172.21.15.60:0/1592244483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9d480cef00 con 0x7f9d4806d9f0 2024-04-25T10:06:06.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.771+0000 7f9d2ffff640 1 -- 172.21.15.60:0/1592244483 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9d400561e0 con 0x7f9d4806d9f0 2024-04-25T10:06:06.776 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:06:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.771+0000 7f9d4df5e640 1 -- 172.21.15.60:0/1592244483 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d34066f60 msgr2=0x7f9d34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.771+0000 7f9d4df5e640 1 --2- 172.21.15.60:0/1592244483 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d34066f60 0x7f9d34069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9d38000f70 tx=0x7f9d38009040 comp rx=0 tx=0).stop 2024-04-25T10:06:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.771+0000 7f9d4df5e640 1 -- 172.21.15.60:0/1592244483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d4806d9f0 msgr2=0x7f9d480b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.771+0000 7f9d4df5e640 1 --2- 172.21.15.60:0/1592244483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d4806d9f0 0x7f9d480b0130 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000b9e0 tx=0x7f9d4000bea0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.771+0000 7f9d4df5e640 1 -- 172.21.15.60:0/1592244483 shutdown_connections 2024-04-25T10:06:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.771+0000 7f9d4df5e640 1 --2- 172.21.15.60:0/1592244483 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d34066f60 0x7f9d34069420 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.771+0000 7f9d4df5e640 1 --2- 172.21.15.60:0/1592244483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d4806d9f0 0x7f9d480b0130 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.771+0000 7f9d4df5e640 1 --2- 172.21.15.60:0/1592244483 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d4815bc90 0x7f9d4815b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.771+0000 7f9d4df5e640 1 --2- 172.21.15.60:0/1592244483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d480ce7a0 0x7f9d48146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.771+0000 7f9d4df5e640 1 -- 172.21.15.60:0/1592244483 >> 172.21.15.60:0/1592244483 conn(0x7f9d480b12d0 msgr2=0x7f9d48157010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.771+0000 7f9d4df5e640 1 -- 172.21.15.60:0/1592244483 shutdown_connections 2024-04-25T10:06:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.771+0000 7f9d4df5e640 1 -- 172.21.15.60:0/1592244483 wait complete. 2024-04-25T10:06:06.795 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:06:06.795 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:06.802 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:06.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.907+0000 7fe45d342640 1 Processor -- start 2024-04-25T10:06:06.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.907+0000 7fe45d342640 1 -- start start 2024-04-25T10:06:06.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.911+0000 7fe45d342640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe45815dc90 0x7fe4580b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.911+0000 7fe45d342640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4580b2b10 0x7fe458071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.911+0000 7fe45d342640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe458071fd0 con 0x7fe4580ae7c0 2024-04-25T10:06:06.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.911+0000 7fe45d342640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4580accc0 con 0x7fe45815dc90 2024-04-25T10:06:06.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.911+0000 7fe45d342640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580ad260 con 0x7fe4580b2b10 2024-04-25T10:06:06.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.911+0000 7fe456ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe45815dc90 0x7fe4580b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.911+0000 7fe4567fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4580b2b10 0x7fe458071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.911+0000 7fe4577fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe4580ae7c0 0x7fe4580ad6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:35956/0 (socket says 172.21.15.60:35956) 2024-04-25T10:06:06.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.911+0000 7fe4577fe640 1 -- 172.21.15.60:0/2200424809 learned_addr learned my addr 172.21.15.60:0/2200424809 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:06.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.911+0000 7fe456ffd640 1 --2- 172.21.15.60:0/2200424809 >> v2:172.21.15.100:3300/0 conn(0x7fe45815dc90 0x7fe4580b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:35832/0 (socket says 172.21.15.60:35832) 2024-04-25T10:06:06.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.911+0000 7fe456ffd640 1 -- 172.21.15.60:0/2200424809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4580b2b10 msgr2=0x7fe458071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.911+0000 7fe456ffd640 1 --2- 172.21.15.60:0/2200424809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4580b2b10 0x7fe458071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.911+0000 7fe456ffd640 1 -- 172.21.15.60:0/2200424809 >> v1:172.21.15.100:6789/0 conn(0x7fe4580ae7c0 legacy=0x7fe4580ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.911+0000 7fe456ffd640 1 -- 172.21.15.60:0/2200424809 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe458072290 con 0x7fe45815dc90 2024-04-25T10:06:06.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.911+0000 7fe456ffd640 1 --2- 172.21.15.60:0/2200424809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe45815dc90 0x7fe4580b25d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe4480097c0 tx=0x7fe44801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8a6a193c4dc8d71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe455ffb640 1 -- 172.21.15.60:0/2200424809 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4480b1070 con 0x7fe45815dc90 2024-04-25T10:06:06.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe455ffb640 1 -- 172.21.15.60:0/2200424809 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4480ac410 con 0x7fe45815dc90 2024-04-25T10:06:06.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe455ffb640 1 -- 172.21.15.60:0/2200424809 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4480ac7e0 con 0x7fe45815dc90 2024-04-25T10:06:06.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 -- 172.21.15.60:0/2200424809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe45815dc90 msgr2=0x7fe4580b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 --2- 172.21.15.60:0/2200424809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe45815dc90 0x7fe4580b25d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe4480097c0 tx=0x7fe44801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 -- 172.21.15.60:0/2200424809 shutdown_connections 2024-04-25T10:06:06.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 --2- 172.21.15.60:0/2200424809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4580b2b10 0x7fe458071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 --2- 172.21.15.60:0/2200424809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe45815dc90 0x7fe4580b25d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 -- 172.21.15.60:0/2200424809 >> 172.21.15.60:0/2200424809 conn(0x7fe4580b12d0 msgr2=0x7fe4580b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 -- 172.21.15.60:0/2200424809 shutdown_connections 2024-04-25T10:06:06.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 -- 172.21.15.60:0/2200424809 wait complete. 2024-04-25T10:06:06.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 Processor -- start 2024-04-25T10:06:06.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 -- start start 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4580ae7c0 0x7fe458153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe45815dc90 0x7fe45814f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe45814f560 0x7fe45814c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe458071ba0 con 0x7fe45815dc90 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe458071a20 con 0x7fe4580ae7c0 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580b2220 con 0x7fe45814f560 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe4567fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe45815dc90 0x7fe45814f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe4567fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe45815dc90 0x7fe45814f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45604/0 (socket says 172.21.15.60:45604) 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe4567fc640 1 -- 172.21.15.60:0/3084451425 learned_addr learned my addr 172.21.15.60:0/3084451425 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe456ffd640 1 --2- 172.21.15.60:0/3084451425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4580ae7c0 0x7fe458153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe4577fe640 1 --2- 172.21.15.60:0/3084451425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe45814f560 0x7fe45814c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe4567fc640 1 -- 172.21.15.60:0/3084451425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe45814f560 msgr2=0x7fe45814c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe4567fc640 1 --2- 172.21.15.60:0/3084451425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe45814f560 0x7fe45814c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe4567fc640 1 -- 172.21.15.60:0/3084451425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4580ae7c0 msgr2=0x7fe458153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe4567fc640 1 --2- 172.21.15.60:0/3084451425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4580ae7c0 0x7fe458153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe4567fc640 1 -- 172.21.15.60:0/3084451425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45814fae0 con 0x7fe45815dc90 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe4567fc640 1 --2- 172.21.15.60:0/3084451425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe45815dc90 0x7fe45814f020 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe44000e870 tx=0x7fe44000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe44f7fe640 1 -- 172.21.15.60:0/3084451425 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440017070 con 0x7fe45815dc90 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe44f7fe640 1 -- 172.21.15.60:0/3084451425 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44000c8e0 con 0x7fe45815dc90 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 -- 172.21.15.60:0/3084451425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45815ec80 con 0x7fe45815dc90 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe44f7fe640 1 -- 172.21.15.60:0/3084451425 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44000cc90 con 0x7fe45815dc90 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.915+0000 7fe45d342640 1 -- 172.21.15.60:0/3084451425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45815f240 con 0x7fe45815dc90 2024-04-25T10:06:06.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.919+0000 7fe45d342640 1 -- 172.21.15.60:0/3084451425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe458074770 con 0x7fe45815dc90 2024-04-25T10:06:06.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.919+0000 7fe44f7fe640 1 -- 172.21.15.60:0/3084451425 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe44001a070 con 0x7fe45815dc90 2024-04-25T10:06:06.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.919+0000 7fe44f7fe640 1 --2- 172.21.15.60:0/3084451425 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe438066f60 0x7fe438069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.919+0000 7fe44f7fe640 1 -- 172.21.15.60:0/3084451425 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fe440085220 con 0x7fe45815dc90 2024-04-25T10:06:06.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.919+0000 7fe456ffd640 1 --2- 172.21.15.60:0/3084451425 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe438066f60 0x7fe438069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.919+0000 7fe456ffd640 1 --2- 172.21.15.60:0/3084451425 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe438066f60 0x7fe438069420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe4480ae000 tx=0x7fe448008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:06.923+0000 7fe44f7fe640 1 -- 172.21.15.60:0/3084451425 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe440055b50 con 0x7fe45815dc90 2024-04-25T10:06:07.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.075+0000 7fe45d342640 1 -- 172.21.15.60:0/3084451425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe45806c250 con 0x7fe45815dc90 2024-04-25T10:06:07.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.079+0000 7fe44f7fe640 1 -- 172.21.15.60:0/3084451425 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16857 (secure 0 0 0) 0x7fe440059bf0 con 0x7fe45815dc90 2024-04-25T10:06:07.086 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:07.086 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:01.568957+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:07.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.079+0000 7fe45d342640 1 -- 172.21.15.60:0/3084451425 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe438066f60 msgr2=0x7fe438069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.079+0000 7fe45d342640 1 --2- 172.21.15.60:0/3084451425 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe438066f60 0x7fe438069420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe4480ae000 tx=0x7fe448008040 comp rx=0 tx=0).stop 2024-04-25T10:06:07.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.079+0000 7fe45d342640 1 -- 172.21.15.60:0/3084451425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe45815dc90 msgr2=0x7fe45814f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.079+0000 7fe45d342640 1 --2- 172.21.15.60:0/3084451425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe45815dc90 0x7fe45814f020 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe44000e870 tx=0x7fe44000ec70 comp rx=0 tx=0).stop 2024-04-25T10:06:07.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.079+0000 7fe45d342640 1 -- 172.21.15.60:0/3084451425 shutdown_connections 2024-04-25T10:06:07.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.079+0000 7fe45d342640 1 --2- 172.21.15.60:0/3084451425 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe438066f60 0x7fe438069420 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.079+0000 7fe45d342640 1 --2- 172.21.15.60:0/3084451425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe45814f560 0x7fe45814c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.079+0000 7fe45d342640 1 --2- 172.21.15.60:0/3084451425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe45815dc90 0x7fe45814f020 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.079+0000 7fe45d342640 1 --2- 172.21.15.60:0/3084451425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4580ae7c0 0x7fe458153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.083+0000 7fe45d342640 1 -- 172.21.15.60:0/3084451425 >> 172.21.15.60:0/3084451425 conn(0x7fe4580b12d0 msgr2=0x7fe4580b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.083+0000 7fe45d342640 1 -- 172.21.15.60:0/3084451425 shutdown_connections 2024-04-25T10:06:07.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.083+0000 7fe45d342640 1 -- 172.21.15.60:0/3084451425 wait complete. 2024-04-25T10:06:07.103 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:18.976643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:19.405302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:07.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:07.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.215+0000 7f30f9349640 1 Processor -- start 2024-04-25T10:06:07.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.215+0000 7f30f9349640 1 -- start start 2024-04-25T10:06:07.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.215+0000 7f30f9349640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f30f40cbcf0 0x7f30f4160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.215+0000 7f30f9349640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f4144a40 con 0x7f30f415fbf0 2024-04-25T10:06:07.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.215+0000 7f30f9349640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f40cc9f0 con 0x7f30f4071c30 2024-04-25T10:06:07.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.215+0000 7f30f9349640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30f40b2dc0 con 0x7f30f40cbcf0 2024-04-25T10:06:07.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.215+0000 7f30f37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f30f415fbf0 0x7f30f40ceec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48974/0 (socket says 172.21.15.60:48974) 2024-04-25T10:06:07.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.215+0000 7f30f37fe640 1 -- 172.21.15.60:0/359175314 learned_addr learned my addr 172.21.15.60:0/359175314 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:07.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.215+0000 7f30f2ffd640 1 --2- 172.21.15.60:0/359175314 >> v2:172.21.15.100:3300/0 conn(0x7f30f40cbcf0 0x7f30f4160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f27fc640 1 -- 172.21.15.60:0/359175314 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473190405 0 0) 0x7f30f4144a40 con 0x7f30f415fbf0 2024-04-25T10:06:07.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f27fc640 1 -- 172.21.15.60:0/359175314 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30e0003530 con 0x7f30f415fbf0 2024-04-25T10:06:07.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f27fc640 1 -- 172.21.15.60:0/359175314 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3094668252 0 0) 0x7f30e0003530 con 0x7f30f415fbf0 2024-04-25T10:06:07.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f2ffd640 1 -- 172.21.15.60:0/359175314 >> v1:172.21.15.100:6789/0 conn(0x7f30f4071c30 legacy=0x7f30f415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f2ffd640 1 -- 172.21.15.60:0/359175314 >> v1:172.21.15.60:6789/0 conn(0x7f30f415fbf0 legacy=0x7f30f40ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f2ffd640 1 -- 172.21.15.60:0/359175314 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f40cccd0 con 0x7f30f40cbcf0 2024-04-25T10:06:07.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f2ffd640 1 --2- 172.21.15.60:0/359175314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f40cbcf0 0x7f30f4160080 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f30e40097c0 tx=0x7f30e401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db99bbc225883f78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f27fc640 1 -- 172.21.15.60:0/359175314 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e40b1070 con 0x7f30f40cbcf0 2024-04-25T10:06:07.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f27fc640 1 -- 172.21.15.60:0/359175314 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e40ac440 con 0x7f30f40cbcf0 2024-04-25T10:06:07.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f27fc640 1 -- 172.21.15.60:0/359175314 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec001a00 con 0x7f30f40cbcf0 2024-04-25T10:06:07.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f9349640 1 -- 172.21.15.60:0/359175314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f40cbcf0 msgr2=0x7f30f4160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f9349640 1 --2- 172.21.15.60:0/359175314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f40cbcf0 0x7f30f4160080 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f30e40097c0 tx=0x7f30e401c390 comp rx=0 tx=0).stop 2024-04-25T10:06:07.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f9349640 1 -- 172.21.15.60:0/359175314 shutdown_connections 2024-04-25T10:06:07.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f9349640 1 --2- 172.21.15.60:0/359175314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f40cbcf0 0x7f30f4160080 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f9349640 1 -- 172.21.15.60:0/359175314 >> 172.21.15.60:0/359175314 conn(0x7f30f40670b0 msgr2=0x7f30f406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f9349640 1 -- 172.21.15.60:0/359175314 shutdown_connections 2024-04-25T10:06:07.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.219+0000 7f30f9349640 1 -- 172.21.15.60:0/359175314 wait complete. 2024-04-25T10:06:07.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f9349640 1 Processor -- start 2024-04-25T10:06:07.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f9349640 1 -- start start 2024-04-25T10:06:07.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f9349640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f4071c30 0x7f30f40aa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f9349640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30f415fbf0 0x7f30f415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f9349640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f415b110 0x7f30f40aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f9349640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f40ce740 con 0x7f30f4071c30 2024-04-25T10:06:07.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f9349640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f40ce5c0 con 0x7f30f415b110 2024-04-25T10:06:07.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f9349640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f4072030 con 0x7f30f415fbf0 2024-04-25T10:06:07.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f4071c30 0x7f30f40aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f4071c30 0x7f30f40aa220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45608/0 (socket says 172.21.15.60:45608) 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f2ffd640 1 -- 172.21.15.60:0/4276763715 learned_addr learned my addr 172.21.15.60:0/4276763715 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f37fe640 1 --2- 172.21.15.60:0/4276763715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f415b110 0x7f30f40aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30ebfff640 1 --2- 172.21.15.60:0/4276763715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30f415fbf0 0x7f30f415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f2ffd640 1 -- 172.21.15.60:0/4276763715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30f415fbf0 msgr2=0x7f30f415a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f2ffd640 1 --2- 172.21.15.60:0/4276763715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30f415fbf0 0x7f30f415a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f2ffd640 1 -- 172.21.15.60:0/4276763715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f415b110 msgr2=0x7f30f40aba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f2ffd640 1 --2- 172.21.15.60:0/4276763715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f415b110 0x7f30f40aba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f2ffd640 1 -- 172.21.15.60:0/4276763715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f40a9180 con 0x7f30f4071c30 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30ebfff640 1 --2- 172.21.15.60:0/4276763715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30f415fbf0 0x7f30f415a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f2ffd640 1 --2- 172.21.15.60:0/4276763715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f4071c30 0x7f30f40aa220 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f30e40af040 tx=0x7f30e401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30eb7fe640 1 -- 172.21.15.60:0/4276763715 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e40b1050 con 0x7f30f4071c30 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30eb7fe640 1 -- 172.21.15.60:0/4276763715 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e40bd070 con 0x7f30f4071c30 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f9349640 1 -- 172.21.15.60:0/4276763715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f40a9460 con 0x7f30f4071c30 2024-04-25T10:06:07.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30eb7fe640 1 -- 172.21.15.60:0/4276763715 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e4006e20 con 0x7f30f4071c30 2024-04-25T10:06:07.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f9349640 1 -- 172.21.15.60:0/4276763715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f40a9a20 con 0x7f30f4071c30 2024-04-25T10:06:07.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.223+0000 7f30f9349640 1 -- 172.21.15.60:0/4276763715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f4074770 con 0x7f30f4071c30 2024-04-25T10:06:07.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.227+0000 7f30eb7fe640 1 -- 172.21.15.60:0/4276763715 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f30e40b8890 con 0x7f30f4071c30 2024-04-25T10:06:07.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.227+0000 7f30eb7fe640 1 --2- 172.21.15.60:0/4276763715 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f30d4066f60 0x7f30d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.227+0000 7f30eb7fe640 1 -- 172.21.15.60:0/4276763715 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f30e411f9d0 con 0x7f30f4071c30 2024-04-25T10:06:07.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.227+0000 7f30ebfff640 1 --2- 172.21.15.60:0/4276763715 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f30d4066f60 0x7f30d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.227+0000 7f30ebfff640 1 --2- 172.21.15.60:0/4276763715 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f30d4066f60 0x7f30d4069420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f30f415bb20 tx=0x7f30dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.231+0000 7f30eb7fe640 1 -- 172.21.15.60:0/4276763715 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f30e4015fb0 con 0x7f30f4071c30 2024-04-25T10:06:07.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.391+0000 7f30f9349640 1 -- 172.21.15.60:0/4276763715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30f41410f0 con 0x7f30f4071c30 2024-04-25T10:06:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.391+0000 7f30eb7fe640 1 -- 172.21.15.60:0/4276763715 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30e40f0820 con 0x7f30f4071c30 2024-04-25T10:06:07.399 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:06:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.395+0000 7f30f9349640 1 -- 172.21.15.60:0/4276763715 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f30d4066f60 msgr2=0x7f30d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.395+0000 7f30f9349640 1 --2- 172.21.15.60:0/4276763715 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f30d4066f60 0x7f30d4069420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f30f415bb20 tx=0x7f30dc009040 comp rx=0 tx=0).stop 2024-04-25T10:06:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.395+0000 7f30f9349640 1 -- 172.21.15.60:0/4276763715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f4071c30 msgr2=0x7f30f40aa220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.395+0000 7f30f9349640 1 --2- 172.21.15.60:0/4276763715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f4071c30 0x7f30f40aa220 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f30e40af040 tx=0x7f30e401abd0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.395+0000 7f30f9349640 1 -- 172.21.15.60:0/4276763715 shutdown_connections 2024-04-25T10:06:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.395+0000 7f30f9349640 1 --2- 172.21.15.60:0/4276763715 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f30d4066f60 0x7f30d4069420 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.395+0000 7f30f9349640 1 --2- 172.21.15.60:0/4276763715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f415b110 0x7f30f40aba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.395+0000 7f30f9349640 1 --2- 172.21.15.60:0/4276763715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30f415fbf0 0x7f30f415a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.395+0000 7f30f9349640 1 --2- 172.21.15.60:0/4276763715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f4071c30 0x7f30f40aa220 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.395+0000 7f30f9349640 1 -- 172.21.15.60:0/4276763715 >> 172.21.15.60:0/4276763715 conn(0x7f30f40670b0 msgr2=0x7f30f40b3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.395+0000 7f30f9349640 1 -- 172.21.15.60:0/4276763715 shutdown_connections 2024-04-25T10:06:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.395+0000 7f30f9349640 1 -- 172.21.15.60:0/4276763715 wait complete. 2024-04-25T10:06:07.420 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:07.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.543+0000 7fdfa5255640 1 Processor -- start 2024-04-25T10:06:07.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.543+0000 7fdfa5255640 1 -- start start 2024-04-25T10:06:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.543+0000 7fdfa5255640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfa015dd30 0x7fdfa015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.543+0000 7fdfa5255640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdfa00cd3d0 0x7fdfa00cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.543+0000 7fdfa5255640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa00cd890 con 0x7fdfa006f310 2024-04-25T10:06:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.543+0000 7fdfa5255640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa00ce4a0 con 0x7fdfa00cd3d0 2024-04-25T10:06:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.543+0000 7fdfa5255640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00cdb70 con 0x7fdfa015dd30 2024-04-25T10:06:07.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.543+0000 7fdf9e59c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdfa00cd3d0 0x7fdfa00cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.543+0000 7fdf9f59e640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fdfa006f310 0x7fdfa006d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:35974/0 (socket says 172.21.15.60:35974) 2024-04-25T10:06:07.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.543+0000 7fdf9ed9d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfa015dd30 0x7fdfa015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.543+0000 7fdf9f59e640 1 -- 172.21.15.60:0/1765770794 learned_addr learned my addr 172.21.15.60:0/1765770794 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:07.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.543+0000 7fdf9e59c640 1 -- 172.21.15.60:0/1765770794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfa015dd30 msgr2=0x7fdfa015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:07.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.543+0000 7fdf9e59c640 1 --2- 172.21.15.60:0/1765770794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfa015dd30 0x7fdfa015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.543+0000 7fdf9e59c640 1 -- 172.21.15.60:0/1765770794 >> v1:172.21.15.100:6789/0 conn(0x7fdfa006f310 legacy=0x7fdfa006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.543+0000 7fdf9e59c640 1 -- 172.21.15.60:0/1765770794 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa00cde50 con 0x7fdfa00cd3d0 2024-04-25T10:06:07.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.547+0000 7fdf9e59c640 1 --2- 172.21.15.60:0/1765770794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfa00cd3d0 0x7fdfa00cf190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00ce620 tx=0x7fdf9401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccdd00204ef755d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.547+0000 7fdf9dd9b640 1 -- 172.21.15.60:0/1765770794 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf940ab600 con 0x7fdfa00cd3d0 2024-04-25T10:06:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.547+0000 7fdf9dd9b640 1 -- 172.21.15.60:0/1765770794 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf940b0070 con 0x7fdfa00cd3d0 2024-04-25T10:06:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.547+0000 7fdf9dd9b640 1 -- 172.21.15.60:0/1765770794 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf940ab9c0 con 0x7fdfa00cd3d0 2024-04-25T10:06:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.547+0000 7fdfa5255640 1 -- 172.21.15.60:0/1765770794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfa00cd3d0 msgr2=0x7fdfa00cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.547+0000 7fdfa5255640 1 --2- 172.21.15.60:0/1765770794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfa00cd3d0 0x7fdfa00cf190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00ce620 tx=0x7fdf9401c3e0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.547+0000 7fdfa5255640 1 -- 172.21.15.60:0/1765770794 shutdown_connections 2024-04-25T10:06:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.547+0000 7fdfa5255640 1 --2- 172.21.15.60:0/1765770794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfa00cd3d0 0x7fdfa00cf190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.547+0000 7fdfa5255640 1 --2- 172.21.15.60:0/1765770794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfa015dd30 0x7fdfa015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.547+0000 7fdfa5255640 1 -- 172.21.15.60:0/1765770794 >> 172.21.15.60:0/1765770794 conn(0x7fdfa00670b0 msgr2=0x7fdfa006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.547+0000 7fdfa5255640 1 -- 172.21.15.60:0/1765770794 shutdown_connections 2024-04-25T10:06:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.547+0000 7fdfa5255640 1 -- 172.21.15.60:0/1765770794 wait complete. 2024-04-25T10:06:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.547+0000 7fdfa5255640 1 Processor -- start 2024-04-25T10:06:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.547+0000 7fdfa5255640 1 -- start start 2024-04-25T10:06:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdfa5255640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfa006f310 0x7fdfa00b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdfa5255640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfa00cd3d0 0x7fdfa01448a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdfa5255640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfa015dd30 0x7fdfa0144de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdfa5255640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa01674b0 con 0x7fdfa006f310 2024-04-25T10:06:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdfa5255640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00cfae0 con 0x7fdfa015dd30 2024-04-25T10:06:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdfa5255640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa0167630 con 0x7fdfa00cd3d0 2024-04-25T10:06:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf9ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfa006f310 0x7fdfa00b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf9ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfa006f310 0x7fdfa00b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45624/0 (socket says 172.21.15.60:45624) 2024-04-25T10:06:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf9ed9d640 1 -- 172.21.15.60:0/454706042 learned_addr learned my addr 172.21.15.60:0/454706042 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf9f59e640 1 --2- 172.21.15.60:0/454706042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfa015dd30 0x7fdfa0144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf9e59c640 1 --2- 172.21.15.60:0/454706042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfa00cd3d0 0x7fdfa01448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf9ed9d640 1 -- 172.21.15.60:0/454706042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfa00cd3d0 msgr2=0x7fdfa01448a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf9ed9d640 1 --2- 172.21.15.60:0/454706042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfa00cd3d0 0x7fdfa01448a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf9ed9d640 1 -- 172.21.15.60:0/454706042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfa015dd30 msgr2=0x7fdfa0144de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf9ed9d640 1 --2- 172.21.15.60:0/454706042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfa015dd30 0x7fdfa0144de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf9ed9d640 1 -- 172.21.15.60:0/454706042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa0143800 con 0x7fdfa006f310 2024-04-25T10:06:07.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf9e59c640 1 --2- 172.21.15.60:0/454706042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfa00cd3d0 0x7fdfa01448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:07.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf9ed9d640 1 --2- 172.21.15.60:0/454706042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfa006f310 0x7fdfa00b14a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800efc0 tx=0x7fdf9800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf877fe640 1 -- 172.21.15.60:0/454706042 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf98018020 con 0x7fdfa006f310 2024-04-25T10:06:07.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf877fe640 1 -- 172.21.15.60:0/454706042 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf98014480 con 0x7fdfa006f310 2024-04-25T10:06:07.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdfa5255640 1 -- 172.21.15.60:0/454706042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa0143ae0 con 0x7fdfa006f310 2024-04-25T10:06:07.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf877fe640 1 -- 172.21.15.60:0/454706042 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf98014850 con 0x7fdfa006f310 2024-04-25T10:06:07.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdfa5255640 1 -- 172.21.15.60:0/454706042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa01440a0 con 0x7fdfa006f310 2024-04-25T10:06:07.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdfa5255640 1 -- 172.21.15.60:0/454706042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa0074770 con 0x7fdfa006f310 2024-04-25T10:06:07.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf877fe640 1 -- 172.21.15.60:0/454706042 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf9801b020 con 0x7fdfa006f310 2024-04-25T10:06:07.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf877fe640 1 --2- 172.21.15.60:0/454706042 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf8c066f60 0x7fdf8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf877fe640 1 -- 172.21.15.60:0/454706042 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fdf98085ba0 con 0x7fdfa006f310 2024-04-25T10:06:07.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.551+0000 7fdf9e59c640 1 --2- 172.21.15.60:0/454706042 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf8c066f60 0x7fdf8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.555+0000 7fdf9e59c640 1 --2- 172.21.15.60:0/454706042 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf8c066f60 0x7fdf8c069420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdf940af000 tx=0x7fdf94008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.555+0000 7fdf877fe640 1 -- 172.21.15.60:0/454706042 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf98022070 con 0x7fdfa006f310 2024-04-25T10:06:07.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.711+0000 7fdfa5255640 1 -- 172.21.15.60:0/454706042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdfa006c250 con 0x7fdfa006f310 2024-04-25T10:06:07.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.711+0000 7fdf877fe640 1 -- 172.21.15.60:0/454706042 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf98010040 con 0x7fdfa006f310 2024-04-25T10:06:07.717 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:06:07.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.711+0000 7fdfa5255640 1 -- 172.21.15.60:0/454706042 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf8c066f60 msgr2=0x7fdf8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.711+0000 7fdfa5255640 1 --2- 172.21.15.60:0/454706042 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf8c066f60 0x7fdf8c069420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdf940af000 tx=0x7fdf94008040 comp rx=0 tx=0).stop 2024-04-25T10:06:07.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.711+0000 7fdfa5255640 1 -- 172.21.15.60:0/454706042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfa006f310 msgr2=0x7fdfa00b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.711+0000 7fdfa5255640 1 --2- 172.21.15.60:0/454706042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfa006f310 0x7fdfa00b14a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800efc0 tx=0x7fdf9800c490 comp rx=0 tx=0).stop 2024-04-25T10:06:07.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.711+0000 7fdfa5255640 1 -- 172.21.15.60:0/454706042 shutdown_connections 2024-04-25T10:06:07.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.711+0000 7fdfa5255640 1 --2- 172.21.15.60:0/454706042 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf8c066f60 0x7fdf8c069420 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.711+0000 7fdfa5255640 1 --2- 172.21.15.60:0/454706042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfa015dd30 0x7fdfa0144de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.711+0000 7fdfa5255640 1 --2- 172.21.15.60:0/454706042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfa00cd3d0 0x7fdfa01448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.711+0000 7fdfa5255640 1 --2- 172.21.15.60:0/454706042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfa006f310 0x7fdfa00b14a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.711+0000 7fdfa5255640 1 -- 172.21.15.60:0/454706042 >> 172.21.15.60:0/454706042 conn(0x7fdfa00670b0 msgr2=0x7fdfa00713e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.711+0000 7fdfa5255640 1 -- 172.21.15.60:0/454706042 shutdown_connections 2024-04-25T10:06:07.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.711+0000 7fdfa5255640 1 -- 172.21.15.60:0/454706042 wait complete. 2024-04-25T10:06:07.735 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:07.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb678b17640 1 Processor -- start 2024-04-25T10:06:07.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb678b17640 1 -- start start 2024-04-25T10:06:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb678b17640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb6740b2860 0x7fb6740d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb678b17640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6740d0fe0 0x7fb6740cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb678b17640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67406d220 con 0x7fb67415dcd0 2024-04-25T10:06:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb678b17640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6740cfa40 con 0x7fb6740b2860 2024-04-25T10:06:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb678b17640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6740cfce0 con 0x7fb6740d0fe0 2024-04-25T10:06:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb671d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb6740b2860 0x7fb6740d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb67259c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb67415dcd0 0x7fb67406c940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:35986/0 (socket says 172.21.15.60:35986) 2024-04-25T10:06:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb67259c640 1 -- 172.21.15.60:0/1361482048 learned_addr learned my addr 172.21.15.60:0/1361482048 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:07.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb672d9d640 1 --2- 172.21.15.60:0/1361482048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6740d0fe0 0x7fb6740cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb671d9b640 1 -- 172.21.15.60:0/1361482048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6740d0fe0 msgr2=0x7fb6740cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb671d9b640 1 --2- 172.21.15.60:0/1361482048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6740d0fe0 0x7fb6740cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb671d9b640 1 -- 172.21.15.60:0/1361482048 >> v1:172.21.15.100:6789/0 conn(0x7fb67415dcd0 legacy=0x7fb67406c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb671d9b640 1 -- 172.21.15.60:0/1361482048 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6740cffc0 con 0x7fb6740b2860 2024-04-25T10:06:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb672d9d640 1 --2- 172.21.15.60:0/1361482048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6740d0fe0 0x7fb6740cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:07.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.843+0000 7fb67159a640 1 -- 172.21.15.60:0/1361482048 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534438338 0 0) 0x7fb67406d220 con 0x7fb67415dcd0 2024-04-25T10:06:07.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.847+0000 7fb671d9b640 1 --2- 172.21.15.60:0/1361482048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6740b2860 0x7fb6740d0aa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb6680097c0 tx=0x7fb66801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3a352f900ae2c6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.847+0000 7fb67159a640 1 -- 172.21.15.60:0/1361482048 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6680b0070 con 0x7fb6740b2860 2024-04-25T10:06:07.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.847+0000 7fb67159a640 1 -- 172.21.15.60:0/1361482048 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6680ab600 con 0x7fb6740b2860 2024-04-25T10:06:07.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.847+0000 7fb67159a640 1 -- 172.21.15.60:0/1361482048 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6680ab9d0 con 0x7fb6740b2860 2024-04-25T10:06:07.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.847+0000 7fb678b17640 1 -- 172.21.15.60:0/1361482048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6740b2860 msgr2=0x7fb6740d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.847+0000 7fb678b17640 1 --2- 172.21.15.60:0/1361482048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6740b2860 0x7fb6740d0aa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb6680097c0 tx=0x7fb66801c410 comp rx=0 tx=0).stop 2024-04-25T10:06:07.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.847+0000 7fb678b17640 1 -- 172.21.15.60:0/1361482048 shutdown_connections 2024-04-25T10:06:07.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.847+0000 7fb678b17640 1 --2- 172.21.15.60:0/1361482048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6740b2860 0x7fb6740d0aa0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.847+0000 7fb678b17640 1 --2- 172.21.15.60:0/1361482048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6740d0fe0 0x7fb6740cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.847+0000 7fb678b17640 1 -- 172.21.15.60:0/1361482048 >> 172.21.15.60:0/1361482048 conn(0x7fb6740b3200 msgr2=0x7fb6740b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.847+0000 7fb678b17640 1 -- 172.21.15.60:0/1361482048 shutdown_connections 2024-04-25T10:06:07.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.847+0000 7fb678b17640 1 -- 172.21.15.60:0/1361482048 wait complete. 2024-04-25T10:06:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.847+0000 7fb678b17640 1 Processor -- start 2024-04-25T10:06:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.847+0000 7fb678b17640 1 -- start start 2024-04-25T10:06:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.847+0000 7fb678b17640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6740b2860 0x7fb674157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.847+0000 7fb678b17640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6740d0fe0 0x7fb6740b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb678b17640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb67415dcd0 0x7fb674151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb678b17640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67416adc0 con 0x7fb67415dcd0 2024-04-25T10:06:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb678b17640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67416ac40 con 0x7fb6740d0fe0 2024-04-25T10:06:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb678b17640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67416af40 con 0x7fb6740b2860 2024-04-25T10:06:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb672d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb67415dcd0 0x7fb674151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb672d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb67415dcd0 0x7fb674151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45626/0 (socket says 172.21.15.60:45626) 2024-04-25T10:06:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb672d9d640 1 -- 172.21.15.60:0/2116861545 learned_addr learned my addr 172.21.15.60:0/2116861545 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb67259c640 1 --2- 172.21.15.60:0/2116861545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6740b2860 0x7fb674157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb671d9b640 1 --2- 172.21.15.60:0/2116861545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6740d0fe0 0x7fb6740b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb672d9d640 1 -- 172.21.15.60:0/2116861545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6740b2860 msgr2=0x7fb674157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb672d9d640 1 --2- 172.21.15.60:0/2116861545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6740b2860 0x7fb674157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb672d9d640 1 -- 172.21.15.60:0/2116861545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6740d0fe0 msgr2=0x7fb6740b6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb672d9d640 1 --2- 172.21.15.60:0/2116861545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6740d0fe0 0x7fb6740b6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb672d9d640 1 -- 172.21.15.60:0/2116861545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6741599e0 con 0x7fb67415dcd0 2024-04-25T10:06:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb67259c640 1 --2- 172.21.15.60:0/2116861545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6740b2860 0x7fb674157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb672d9d640 1 --2- 172.21.15.60:0/2116861545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb67415dcd0 0x7fb674151fc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00efc0 tx=0x7fb66c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb662ffd640 1 -- 172.21.15.60:0/2116861545 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c018030 con 0x7fb67415dcd0 2024-04-25T10:06:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb662ffd640 1 -- 172.21.15.60:0/2116861545 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66c00cdf0 con 0x7fb67415dcd0 2024-04-25T10:06:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb678b17640 1 -- 172.21.15.60:0/2116861545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb674158560 con 0x7fb67415dcd0 2024-04-25T10:06:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb662ffd640 1 -- 172.21.15.60:0/2116861545 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c003bc0 con 0x7fb67415dcd0 2024-04-25T10:06:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb678b17640 1 -- 172.21.15.60:0/2116861545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6741535d0 con 0x7fb67415dcd0 2024-04-25T10:06:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb678b17640 1 -- 172.21.15.60:0/2116861545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb674074770 con 0x7fb67415dcd0 2024-04-25T10:06:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb662ffd640 1 -- 172.21.15.60:0/2116861545 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb66c01b070 con 0x7fb67415dcd0 2024-04-25T10:06:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb662ffd640 1 --2- 172.21.15.60:0/2116861545 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb654066f90 0x7fb654069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb662ffd640 1 -- 172.21.15.60:0/2116861545 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb66c085980 con 0x7fb67415dcd0 2024-04-25T10:06:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.851+0000 7fb67259c640 1 --2- 172.21.15.60:0/2116861545 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb654066f90 0x7fb654069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.855+0000 7fb67259c640 1 --2- 172.21.15.60:0/2116861545 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb654066f90 0x7fb654069450 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb664000f70 tx=0x7fb664007cb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:07.855+0000 7fb662ffd640 1 -- 172.21.15.60:0/2116861545 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb66c022070 con 0x7fb67415dcd0 2024-04-25T10:06:08.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.011+0000 7fb678b17640 1 -- 172.21.15.60:0/2116861545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb6740b4070 con 0x7fb67415dcd0 2024-04-25T10:06:08.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.011+0000 7fb662ffd640 1 -- 172.21.15.60:0/2116861545 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb6741599e0 con 0x7fb67415dcd0 2024-04-25T10:06:08.017 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:06:08.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.011+0000 7fb678b17640 1 -- 172.21.15.60:0/2116861545 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb654066f90 msgr2=0x7fb654069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.011+0000 7fb678b17640 1 --2- 172.21.15.60:0/2116861545 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb654066f90 0x7fb654069450 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb664000f70 tx=0x7fb664007cb0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.011+0000 7fb678b17640 1 -- 172.21.15.60:0/2116861545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb67415dcd0 msgr2=0x7fb674151fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.011+0000 7fb678b17640 1 --2- 172.21.15.60:0/2116861545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb67415dcd0 0x7fb674151fc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00efc0 tx=0x7fb66c00c490 comp rx=0 tx=0).stop 2024-04-25T10:06:08.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.011+0000 7fb678b17640 1 -- 172.21.15.60:0/2116861545 shutdown_connections 2024-04-25T10:06:08.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.011+0000 7fb678b17640 1 --2- 172.21.15.60:0/2116861545 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb654066f90 0x7fb654069450 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.011+0000 7fb678b17640 1 --2- 172.21.15.60:0/2116861545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb67415dcd0 0x7fb674151fc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.011+0000 7fb678b17640 1 --2- 172.21.15.60:0/2116861545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6740d0fe0 0x7fb6740b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.011+0000 7fb678b17640 1 --2- 172.21.15.60:0/2116861545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6740b2860 0x7fb674157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.011+0000 7fb678b17640 1 -- 172.21.15.60:0/2116861545 >> 172.21.15.60:0/2116861545 conn(0x7fb6740b3200 msgr2=0x7fb674073fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.011+0000 7fb678b17640 1 -- 172.21.15.60:0/2116861545 shutdown_connections 2024-04-25T10:06:08.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.011+0000 7fb678b17640 1 -- 172.21.15.60:0/2116861545 wait complete. 2024-04-25T10:06:08.035 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:06:08.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.147+0000 7f1069581640 1 Processor -- start 2024-04-25T10:06:08.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.147+0000 7f1069581640 1 -- start start 2024-04-25T10:06:08.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.147+0000 7f1069581640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f106406bc20 0x7f106406c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.147+0000 7f1069581640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10640acf80 con 0x7f10640b4500 2024-04-25T10:06:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.147+0000 7f1069581640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10640b3fd0 con 0x7f1064161d60 2024-04-25T10:06:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.147+0000 7f1069581640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106406c590 con 0x7f106406bc20 2024-04-25T10:06:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.147+0000 7f10637fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f106406bc20 0x7f106406c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.147+0000 7f10627fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f10640b4500 0x7f106406dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48978/0 (socket says 172.21.15.60:48978) 2024-04-25T10:06:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.147+0000 7f10637fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f106406bc20 0x7f106406c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45636/0 (socket says 172.21.15.60:45636) 2024-04-25T10:06:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.147+0000 7f10627fc640 1 -- 172.21.15.60:0/802344947 learned_addr learned my addr 172.21.15.60:0/802344947 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.147+0000 7f10637fe640 1 -- 172.21.15.60:0/802344947 >> v1:172.21.15.100:6789/0 conn(0x7f1064161d60 legacy=0x7f10640b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.147+0000 7f10637fe640 1 -- 172.21.15.60:0/802344947 >> v1:172.21.15.60:6789/0 conn(0x7f10640b4500 legacy=0x7f106406dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.147+0000 7f10637fe640 1 -- 172.21.15.60:0/802344947 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10640ad220 con 0x7f106406bc20 2024-04-25T10:06:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.147+0000 7f1061ffb640 1 -- 172.21.15.60:0/802344947 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331421343 0 0) 0x7f10640acf80 con 0x7f10640b4500 2024-04-25T10:06:08.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f10637fe640 1 --2- 172.21.15.60:0/802344947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f106406bc20 0x7f106406c050 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f10540097f0 tx=0x7f105401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ebaf88a4f44334 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1061ffb640 1 -- 172.21.15.60:0/802344947 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540ab430 con 0x7f106406bc20 2024-04-25T10:06:08.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1061ffb640 1 -- 172.21.15.60:0/802344947 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10540b0070 con 0x7f106406bc20 2024-04-25T10:06:08.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1061ffb640 1 -- 172.21.15.60:0/802344947 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540ab7f0 con 0x7f106406bc20 2024-04-25T10:06:08.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1069581640 1 -- 172.21.15.60:0/802344947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f106406bc20 msgr2=0x7f106406c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1069581640 1 --2- 172.21.15.60:0/802344947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f106406bc20 0x7f106406c050 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f10540097f0 tx=0x7f105401c430 comp rx=0 tx=0).stop 2024-04-25T10:06:08.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1069581640 1 -- 172.21.15.60:0/802344947 shutdown_connections 2024-04-25T10:06:08.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1069581640 1 --2- 172.21.15.60:0/802344947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f106406bc20 0x7f106406c050 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1069581640 1 -- 172.21.15.60:0/802344947 >> 172.21.15.60:0/802344947 conn(0x7f1064071b90 msgr2=0x7f1064071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1069581640 1 -- 172.21.15.60:0/802344947 shutdown_connections 2024-04-25T10:06:08.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1069581640 1 -- 172.21.15.60:0/802344947 wait complete. 2024-04-25T10:06:08.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1069581640 1 Processor -- start 2024-04-25T10:06:08.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1069581640 1 -- start start 2024-04-25T10:06:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1069581640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f106406bc20 0x7f106409a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1069581640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10640b4500 0x7f1064149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1069581640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1064161d60 0x7f1064160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1069581640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106416f970 con 0x7f1064161d60 2024-04-25T10:06:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1069581640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106416faf0 con 0x7f10640b4500 2024-04-25T10:06:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1069581640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1064149570 con 0x7f106406bc20 2024-04-25T10:06:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f10637fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1064161d60 0x7f1064160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f10637fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1064161d60 0x7f1064160600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45648/0 (socket says 172.21.15.60:45648) 2024-04-25T10:06:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f10637fe640 1 -- 172.21.15.60:0/1408448470 learned_addr learned my addr 172.21.15.60:0/1408448470 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f10637fe640 1 -- 172.21.15.60:0/1408448470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f106406bc20 msgr2=0x7f106409a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f1062ffd640 1 --2- 172.21.15.60:0/1408448470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f106406bc20 0x7f106409a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f10627fc640 1 --2- 172.21.15.60:0/1408448470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10640b4500 0x7f1064149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f10637fe640 1 --2- 172.21.15.60:0/1408448470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f106406bc20 0x7f106409a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f10637fe640 1 -- 172.21.15.60:0/1408448470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10640b4500 msgr2=0x7f1064149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f10637fe640 1 --2- 172.21.15.60:0/1408448470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10640b4500 0x7f1064149030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f10637fe640 1 -- 172.21.15.60:0/1408448470 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106409ab60 con 0x7f1064161d60 2024-04-25T10:06:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.151+0000 7f10637fe640 1 --2- 172.21.15.60:0/1408448470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1064161d60 0x7f1064160600 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1054019040 tx=0x7f105401ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.155+0000 7f105b7fe640 1 -- 172.21.15.60:0/1408448470 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540bf610 con 0x7f1064161d60 2024-04-25T10:06:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.155+0000 7f105b7fe640 1 -- 172.21.15.60:0/1408448470 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10540bf7b0 con 0x7f1064161d60 2024-04-25T10:06:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.155+0000 7f1069581640 1 -- 172.21.15.60:0/1408448470 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106409ae40 con 0x7f1064161d60 2024-04-25T10:06:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.155+0000 7f105b7fe640 1 -- 172.21.15.60:0/1408448470 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540bfb80 con 0x7f1064161d60 2024-04-25T10:06:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.155+0000 7f1069581640 1 -- 172.21.15.60:0/1408448470 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106409b400 con 0x7f1064161d60 2024-04-25T10:06:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.155+0000 7f1069581640 1 -- 172.21.15.60:0/1408448470 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1064074770 con 0x7f1064161d60 2024-04-25T10:06:08.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.155+0000 7f105b7fe640 1 -- 172.21.15.60:0/1408448470 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10540bd020 con 0x7f1064161d60 2024-04-25T10:06:08.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.155+0000 7f105b7fe640 1 --2- 172.21.15.60:0/1408448470 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1044066f60 0x7f1044069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.155+0000 7f105b7fe640 1 -- 172.21.15.60:0/1408448470 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f10540a8030 con 0x7f1064161d60 2024-04-25T10:06:08.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.155+0000 7f1062ffd640 1 --2- 172.21.15.60:0/1408448470 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1044066f60 0x7f1044069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.155+0000 7f1062ffd640 1 --2- 172.21.15.60:0/1408448470 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1044066f60 0x7f1044069420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f105c007bf0 tx=0x7f105c007240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.159+0000 7f105b7fe640 1 -- 172.21.15.60:0/1408448470 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1054016050 con 0x7f1064161d60 2024-04-25T10:06:08.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.327+0000 7f1069581640 1 -- 172.21.15.60:0/1408448470 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1064159de0 con 0x7f1064161d60 2024-04-25T10:06:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.327+0000 7f105b7fe640 1 -- 172.21.15.60:0/1408448470 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f10540f07f0 con 0x7f1064161d60 2024-04-25T10:06:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:06:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.327+0000 7f1069581640 1 -- 172.21.15.60:0/1408448470 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1044066f60 msgr2=0x7f1044069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.327+0000 7f1069581640 1 --2- 172.21.15.60:0/1408448470 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1044066f60 0x7f1044069420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f105c007bf0 tx=0x7f105c007240 comp rx=0 tx=0).stop 2024-04-25T10:06:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.327+0000 7f1069581640 1 -- 172.21.15.60:0/1408448470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1064161d60 msgr2=0x7f1064160600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.327+0000 7f1069581640 1 --2- 172.21.15.60:0/1408448470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1064161d60 0x7f1064160600 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1054019040 tx=0x7f105401ac70 comp rx=0 tx=0).stop 2024-04-25T10:06:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.327+0000 7f1069581640 1 -- 172.21.15.60:0/1408448470 shutdown_connections 2024-04-25T10:06:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.327+0000 7f1069581640 1 --2- 172.21.15.60:0/1408448470 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1044066f60 0x7f1044069420 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.327+0000 7f1069581640 1 --2- 172.21.15.60:0/1408448470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1064161d60 0x7f1064160600 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.327+0000 7f1069581640 1 --2- 172.21.15.60:0/1408448470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10640b4500 0x7f1064149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.327+0000 7f1069581640 1 --2- 172.21.15.60:0/1408448470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f106406bc20 0x7f106409a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.327+0000 7f1069581640 1 -- 172.21.15.60:0/1408448470 >> 172.21.15.60:0/1408448470 conn(0x7f1064071b90 msgr2=0x7f1064162390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.327+0000 7f1069581640 1 -- 172.21.15.60:0/1408448470 shutdown_connections 2024-04-25T10:06:08.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.327+0000 7f1069581640 1 -- 172.21.15.60:0/1408448470 wait complete. 2024-04-25T10:06:08.350 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2024-04-25T10:06:08.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:08.351 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-25T10:06:08.366 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:08.366 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:06:08.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:08.367 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:06:08.414 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:06:08.414 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:08.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.531+0000 7fb21cb5f640 1 Processor -- start 2024-04-25T10:06:08.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.535+0000 7fb21cb5f640 1 -- start start 2024-04-25T10:06:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.535+0000 7fb21cb5f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb21815bc90 0x7fb21815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.535+0000 7fb21cb5f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2180b4660 0x7fb21815c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.535+0000 7fb21cb5f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb218071ca0 con 0x7fb2180ce7a0 2024-04-25T10:06:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.535+0000 7fb21cb5f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2180cc370 con 0x7fb21815bc90 2024-04-25T10:06:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.535+0000 7fb21cb5f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2180cca40 con 0x7fb2180b4660 2024-04-25T10:06:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.535+0000 7fb20dd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb21815bc90 0x7fb21815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.535+0000 7fb20dd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb21815bc90 0x7fb21815c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45652/0 (socket says 172.21.15.60:45652) 2024-04-25T10:06:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.535+0000 7fb20dd9b640 1 -- 172.21.15.60:0/3495860368 learned_addr learned my addr 172.21.15.60:0/3495860368 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.535+0000 7fb216d9d640 1 --2- 172.21.15.60:0/3495860368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2180b4660 0x7fb21815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.535+0000 7fb20dd9b640 1 -- 172.21.15.60:0/3495860368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2180b4660 msgr2=0x7fb21815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.535+0000 7fb20dd9b640 1 --2- 172.21.15.60:0/3495860368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2180b4660 0x7fb21815c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.535+0000 7fb20dd9b640 1 -- 172.21.15.60:0/3495860368 >> v1:172.21.15.100:6789/0 conn(0x7fb2180ce7a0 legacy=0x7fb2180ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.535+0000 7fb20dd9b640 1 -- 172.21.15.60:0/3495860368 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2180ccd20 con 0x7fb21815bc90 2024-04-25T10:06:08.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.535+0000 7fb216d9d640 1 --2- 172.21.15.60:0/3495860368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2180b4660 0x7fb21815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:08.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb20dd9b640 1 --2- 172.21.15.60:0/3495860368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb21815bc90 0x7fb21815c220 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb200009870 tx=0x7fb20001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bd8e7268b3827 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb215d9b640 1 -- 172.21.15.60:0/3495860368 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2000b0070 con 0x7fb21815bc90 2024-04-25T10:06:08.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb215d9b640 1 -- 172.21.15.60:0/3495860368 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2000ab600 con 0x7fb21815bc90 2024-04-25T10:06:08.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb215d9b640 1 -- 172.21.15.60:0/3495860368 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2000ab9d0 con 0x7fb21815bc90 2024-04-25T10:06:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21cb5f640 1 -- 172.21.15.60:0/3495860368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb21815bc90 msgr2=0x7fb21815c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21cb5f640 1 --2- 172.21.15.60:0/3495860368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb21815bc90 0x7fb21815c220 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb200009870 tx=0x7fb20001c520 comp rx=0 tx=0).stop 2024-04-25T10:06:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21cb5f640 1 -- 172.21.15.60:0/3495860368 shutdown_connections 2024-04-25T10:06:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21cb5f640 1 --2- 172.21.15.60:0/3495860368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb21815bc90 0x7fb21815c220 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21cb5f640 1 --2- 172.21.15.60:0/3495860368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2180b4660 0x7fb21815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21cb5f640 1 -- 172.21.15.60:0/3495860368 >> 172.21.15.60:0/3495860368 conn(0x7fb2180b12d0 msgr2=0x7fb2180b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21cb5f640 1 -- 172.21.15.60:0/3495860368 shutdown_connections 2024-04-25T10:06:08.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21cb5f640 1 -- 172.21.15.60:0/3495860368 wait complete. 2024-04-25T10:06:08.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21cb5f640 1 Processor -- start 2024-04-25T10:06:08.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21cb5f640 1 -- start start 2024-04-25T10:06:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21cb5f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2180ce7a0 0x7fb218147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21cb5f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb21815bc90 0x7fb218148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21cb5f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb218149260 0x7fb2180b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21cb5f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2181685c0 con 0x7fb2180ce7a0 2024-04-25T10:06:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21cb5f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2180b3db0 con 0x7fb218149260 2024-04-25T10:06:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21cb5f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb218168740 con 0x7fb21815bc90 2024-04-25T10:06:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2180ce7a0 0x7fb218147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2180ce7a0 0x7fb218147790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45654/0 (socket says 172.21.15.60:45654) 2024-04-25T10:06:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb21659c640 1 -- 172.21.15.60:0/954924700 learned_addr learned my addr 172.21.15.60:0/954924700 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb216d9d640 1 --2- 172.21.15.60:0/954924700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb218149260 0x7fb2180b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.539+0000 7fb20dd9b640 1 --2- 172.21.15.60:0/954924700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb21815bc90 0x7fb218148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb21659c640 1 -- 172.21.15.60:0/954924700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb21815bc90 msgr2=0x7fb218148730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb21659c640 1 --2- 172.21.15.60:0/954924700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb21815bc90 0x7fb218148730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb21659c640 1 -- 172.21.15.60:0/954924700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb218149260 msgr2=0x7fb2180b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb21659c640 1 --2- 172.21.15.60:0/954924700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb218149260 0x7fb2180b2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb21659c640 1 -- 172.21.15.60:0/954924700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb218153a20 con 0x7fb2180ce7a0 2024-04-25T10:06:08.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb21659c640 1 --2- 172.21.15.60:0/954924700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2180ce7a0 0x7fb218147790 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb20800b410 tx=0x7fb20800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb20f7fe640 1 -- 172.21.15.60:0/954924700 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208015030 con 0x7fb2180ce7a0 2024-04-25T10:06:08.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb20f7fe640 1 -- 172.21.15.60:0/954924700 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb208004020 con 0x7fb2180ce7a0 2024-04-25T10:06:08.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb21cb5f640 1 -- 172.21.15.60:0/954924700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb218148e10 con 0x7fb2180ce7a0 2024-04-25T10:06:08.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb20f7fe640 1 -- 172.21.15.60:0/954924700 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208004450 con 0x7fb2180ce7a0 2024-04-25T10:06:08.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb21cb5f640 1 -- 172.21.15.60:0/954924700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb218159660 con 0x7fb2180ce7a0 2024-04-25T10:06:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb21cb5f640 1 -- 172.21.15.60:0/954924700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb218074770 con 0x7fb2180ce7a0 2024-04-25T10:06:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb20f7fe640 1 -- 172.21.15.60:0/954924700 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb208018070 con 0x7fb2180ce7a0 2024-04-25T10:06:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb20f7fe640 1 --2- 172.21.15.60:0/954924700 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb1f8066f60 0x7fb1f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb20f7fe640 1 -- 172.21.15.60:0/954924700 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb208084c00 con 0x7fb2180ce7a0 2024-04-25T10:06:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb20dd9b640 1 --2- 172.21.15.60:0/954924700 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb1f8066f60 0x7fb1f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.543+0000 7fb20dd9b640 1 --2- 172.21.15.60:0/954924700 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb1f8066f60 0x7fb1f8069420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb2000ad000 tx=0x7fb2000a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.547+0000 7fb20f7fe640 1 -- 172.21.15.60:0/954924700 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb20808b080 con 0x7fb2180ce7a0 2024-04-25T10:06:08.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.743+0000 7fb21cb5f640 1 -- 172.21.15.60:0/954924700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb218071890 con 0x7fb2180ce7a0 2024-04-25T10:06:08.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.747+0000 7fb20f7fe640 1 -- 172.21.15.60:0/954924700 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7fb208055b80 con 0x7fb2180ce7a0 2024-04-25T10:06:08.753 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:08.753 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/622088871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":622088871},{"type":"v1","addr":"172.21.15.100:6836","nonce":622088871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3988204202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3988204202},{"type":"v1","addr":"172.21.15.60:6833","nonce":3988204202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3591836867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3591836867},{"type":"v1","addr":"172.21.15.60:6835","nonce":3591836867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:20.405486+0000","modified":"2024-04-25T10:05:23.453258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/690070228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":690070228},{"type":"v1","addr":"172.21.15.100:6837","nonce":690070228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:06:08.753 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-04-25T10:06:08.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.747+0000 7fb21cb5f640 1 -- 172.21.15.60:0/954924700 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb1f8066f60 msgr2=0x7fb1f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.747+0000 7fb21cb5f640 1 --2- 172.21.15.60:0/954924700 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb1f8066f60 0x7fb1f8069420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb2000ad000 tx=0x7fb2000a6000 comp rx=0 tx=0).stop 2024-04-25T10:06:08.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.747+0000 7fb21cb5f640 1 -- 172.21.15.60:0/954924700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2180ce7a0 msgr2=0x7fb218147790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.747+0000 7fb21cb5f640 1 --2- 172.21.15.60:0/954924700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2180ce7a0 0x7fb218147790 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb20800b410 tx=0x7fb20800b8d0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.747+0000 7fb21cb5f640 1 -- 172.21.15.60:0/954924700 shutdown_connections 2024-04-25T10:06:08.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.747+0000 7fb21cb5f640 1 --2- 172.21.15.60:0/954924700 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb1f8066f60 0x7fb1f8069420 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.747+0000 7fb21cb5f640 1 --2- 172.21.15.60:0/954924700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb218149260 0x7fb2180b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.747+0000 7fb21cb5f640 1 --2- 172.21.15.60:0/954924700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb21815bc90 0x7fb218148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.747+0000 7fb21cb5f640 1 --2- 172.21.15.60:0/954924700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2180ce7a0 0x7fb218147790 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.747+0000 7fb21cb5f640 1 -- 172.21.15.60:0/954924700 >> 172.21.15.60:0/954924700 conn(0x7fb2180b12d0 msgr2=0x7fb218155110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.747+0000 7fb21cb5f640 1 -- 172.21.15.60:0/954924700 shutdown_connections 2024-04-25T10:06:08.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.747+0000 7fb21cb5f640 1 -- 172.21.15.60:0/954924700 wait complete. 2024-04-25T10:06:08.774 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:08.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.891+0000 7f817e5dd640 1 Processor -- start 2024-04-25T10:06:08.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.891+0000 7f817e5dd640 1 -- start start 2024-04-25T10:06:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.895+0000 7f817e5dd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f817815a7a0 0x7f81780cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.895+0000 7f817e5dd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81780b37e0 0x7f81780b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.895+0000 7f817e5dd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817806f130 con 0x7f817806ec30 2024-04-25T10:06:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.895+0000 7f817e5dd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81780cdaa0 con 0x7f817815a7a0 2024-04-25T10:06:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.895+0000 7f817e5dd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817806d2f0 con 0x7f81780b37e0 2024-04-25T10:06:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.895+0000 7f817cb7a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81780b37e0 0x7f81780b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.895+0000 7f8177fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f817815a7a0 0x7f81780cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.895+0000 7f8177fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f817815a7a0 0x7f81780cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:45184/0 (socket says 172.21.15.60:45184) 2024-04-25T10:06:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.895+0000 7f817cb7a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81780b37e0 0x7f81780b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:38386/0 (socket says 172.21.15.60:38386) 2024-04-25T10:06:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.895+0000 7f8177fff640 1 -- 172.21.15.60:0/3541749010 learned_addr learned my addr 172.21.15.60:0/3541749010 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.895+0000 7f816f7fe640 1 --1- 172.21.15.60:0/3541749010 >> v1:172.21.15.100:6789/0 conn(0x7f817806ec30 0x7f8178071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:55356/0 (socket says 172.21.15.60:55356) 2024-04-25T10:06:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.895+0000 7f8177fff640 1 -- 172.21.15.60:0/3541749010 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81780b37e0 msgr2=0x7f81780b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.895+0000 7f8177fff640 1 --2- 172.21.15.60:0/3541749010 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81780b37e0 0x7f81780b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.895+0000 7f8177fff640 1 -- 172.21.15.60:0/3541749010 >> v1:172.21.15.100:6789/0 conn(0x7f817806ec30 legacy=0x7f8178071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.895+0000 7f8177fff640 1 -- 172.21.15.60:0/3541749010 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817806f410 con 0x7f817815a7a0 2024-04-25T10:06:08.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.899+0000 7f8177fff640 1 --2- 172.21.15.60:0/3541749010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f817815a7a0 0x7f81780cd560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f81680098a0 tx=0x7f816801c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f08ae01498f6f32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.899+0000 7f81777fe640 1 -- 172.21.15.60:0/3541749010 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81680ab650 con 0x7f817815a7a0 2024-04-25T10:06:08.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.899+0000 7f81777fe640 1 -- 172.21.15.60:0/3541749010 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81680b0070 con 0x7f817815a7a0 2024-04-25T10:06:08.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.899+0000 7f81777fe640 1 -- 172.21.15.60:0/3541749010 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81680aba10 con 0x7f817815a7a0 2024-04-25T10:06:08.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.899+0000 7f817e5dd640 1 -- 172.21.15.60:0/3541749010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f817815a7a0 msgr2=0x7f81780cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.899+0000 7f817e5dd640 1 --2- 172.21.15.60:0/3541749010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f817815a7a0 0x7f81780cd560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f81680098a0 tx=0x7f816801c540 comp rx=0 tx=0).stop 2024-04-25T10:06:08.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.899+0000 7f817e5dd640 1 -- 172.21.15.60:0/3541749010 shutdown_connections 2024-04-25T10:06:08.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.899+0000 7f817e5dd640 1 --2- 172.21.15.60:0/3541749010 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81780b37e0 0x7f81780b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.899+0000 7f817e5dd640 1 --2- 172.21.15.60:0/3541749010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f817815a7a0 0x7f81780cd560 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.899+0000 7f817e5dd640 1 -- 172.21.15.60:0/3541749010 >> 172.21.15.60:0/3541749010 conn(0x7f81780b12d0 msgr2=0x7f81780b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.899+0000 7f817e5dd640 1 -- 172.21.15.60:0/3541749010 shutdown_connections 2024-04-25T10:06:08.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.899+0000 7f817e5dd640 1 -- 172.21.15.60:0/3541749010 wait complete. 2024-04-25T10:06:08.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f817e5dd640 1 Processor -- start 2024-04-25T10:06:08.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f817e5dd640 1 -- start start 2024-04-25T10:06:08.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f817e5dd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f817806ec30 0x7f8178156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f817e5dd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81780b37e0 0x7f817814e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f817e5dd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f817815a7a0 0x7f8178154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f817e5dd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817806e8e0 con 0x7f817806ec30 2024-04-25T10:06:08.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f817e5dd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817806e760 con 0x7f817815a7a0 2024-04-25T10:06:08.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f817e5dd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817806d8b0 con 0x7f81780b37e0 2024-04-25T10:06:08.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f8177fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f817806ec30 0x7f8178156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f8177fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f817806ec30 0x7f8178156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40820/0 (socket says 172.21.15.60:40820) 2024-04-25T10:06:08.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f8177fff640 1 -- 172.21.15.60:0/3319478661 learned_addr learned my addr 172.21.15.60:0/3319478661 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:08.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f817cb7a640 1 --2- 172.21.15.60:0/3319478661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f817815a7a0 0x7f8178154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f816f7fe640 1 --2- 172.21.15.60:0/3319478661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81780b37e0 0x7f817814e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f8177fff640 1 -- 172.21.15.60:0/3319478661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81780b37e0 msgr2=0x7f817814e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f8177fff640 1 --2- 172.21.15.60:0/3319478661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81780b37e0 0x7f817814e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f8177fff640 1 -- 172.21.15.60:0/3319478661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f817815a7a0 msgr2=0x7f8178154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f8177fff640 1 --2- 172.21.15.60:0/3319478661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f817815a7a0 0x7f8178154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f8177fff640 1 -- 172.21.15.60:0/3319478661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81781544c0 con 0x7f817806ec30 2024-04-25T10:06:08.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f8177fff640 1 --2- 172.21.15.60:0/3319478661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f817806ec30 0x7f8178156600 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f81680ad040 tx=0x7f816801ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f81757fa640 1 -- 172.21.15.60:0/3319478661 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81680be5f0 con 0x7f817806ec30 2024-04-25T10:06:08.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f81757fa640 1 -- 172.21.15.60:0/3319478661 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81680be790 con 0x7f817806ec30 2024-04-25T10:06:08.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f817e5dd640 1 -- 172.21.15.60:0/3319478661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8178162b60 con 0x7f817806ec30 2024-04-25T10:06:08.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f81757fa640 1 -- 172.21.15.60:0/3319478661 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81680beb60 con 0x7f817806ec30 2024-04-25T10:06:08.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f817e5dd640 1 -- 172.21.15.60:0/3319478661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817814ec00 con 0x7f817806ec30 2024-04-25T10:06:08.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f817e5dd640 1 -- 172.21.15.60:0/3319478661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8178074770 con 0x7f817806ec30 2024-04-25T10:06:08.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.903+0000 7f81757fa640 1 -- 172.21.15.60:0/3319478661 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f81680b43f0 con 0x7f817806ec30 2024-04-25T10:06:08.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.907+0000 7f81757fa640 1 --2- 172.21.15.60:0/3319478661 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8158066f60 0x7f8158069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.907+0000 7f81757fa640 1 -- 172.21.15.60:0/3319478661 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f816811f9e0 con 0x7f817806ec30 2024-04-25T10:06:08.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.907+0000 7f816f7fe640 1 --2- 172.21.15.60:0/3319478661 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8158066f60 0x7f8158069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.907+0000 7f816f7fe640 1 --2- 172.21.15.60:0/3319478661 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8158066f60 0x7f8158069420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f817815dec0 tx=0x7f8160007cd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:08.911+0000 7f81757fa640 1 -- 172.21.15.60:0/3319478661 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8168016130 con 0x7f817806ec30 2024-04-25T10:06:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.071+0000 7f817e5dd640 1 -- 172.21.15.60:0/3319478661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f817806c250 con 0x7f817806ec30 2024-04-25T10:06:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.075+0000 7f81757fa640 1 -- 172.21.15.60:0/3319478661 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16857 (secure 0 0 0) 0x7f81680f04c0 con 0x7f817806ec30 2024-04-25T10:06:09.082 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:09.082 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:01.568957+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:09.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.079+0000 7f817e5dd640 1 -- 172.21.15.60:0/3319478661 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8158066f60 msgr2=0x7f8158069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.079+0000 7f817e5dd640 1 --2- 172.21.15.60:0/3319478661 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8158066f60 0x7f8158069420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f817815dec0 tx=0x7f8160007cd0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.079+0000 7f817e5dd640 1 -- 172.21.15.60:0/3319478661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f817806ec30 msgr2=0x7f8178156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.079+0000 7f817e5dd640 1 --2- 172.21.15.60:0/3319478661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f817806ec30 0x7f8178156600 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f81680ad040 tx=0x7f816801ad80 comp rx=0 tx=0).stop 2024-04-25T10:06:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.079+0000 7f817e5dd640 1 -- 172.21.15.60:0/3319478661 shutdown_connections 2024-04-25T10:06:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.079+0000 7f817e5dd640 1 --2- 172.21.15.60:0/3319478661 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8158066f60 0x7f8158069420 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.079+0000 7f817e5dd640 1 --2- 172.21.15.60:0/3319478661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f817815a7a0 0x7f8178154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.079+0000 7f817e5dd640 1 --2- 172.21.15.60:0/3319478661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81780b37e0 0x7f817814e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.079+0000 7f817e5dd640 1 --2- 172.21.15.60:0/3319478661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f817806ec30 0x7f8178156600 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.079+0000 7f817e5dd640 1 -- 172.21.15.60:0/3319478661 >> 172.21.15.60:0/3319478661 conn(0x7f81780b12d0 msgr2=0x7f817814fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.079+0000 7f817e5dd640 1 -- 172.21.15.60:0/3319478661 shutdown_connections 2024-04-25T10:06:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.079+0000 7f817e5dd640 1 -- 172.21.15.60:0/3319478661 wait complete. 2024-04-25T10:06:09.102 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:18.976643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:19.405302+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:09.103 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:09.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b818cf640 1 Processor -- start 2024-04-25T10:06:09.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b818cf640 1 -- start start 2024-04-25T10:06:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b818cf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c15ef20 0x7f3b7c0b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b818cf640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b7c0cc320 con 0x7f3b7c099a30 2024-04-25T10:06:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b818cf640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b7c0cc5c0 con 0x7f3b7c0cfcf0 2024-04-25T10:06:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b818cf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c0d07b0 con 0x7f3b7c15ef20 2024-04-25T10:06:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b7b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3b7c099a30 0x7f3b7c0cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42538/0 (socket says 172.21.15.60:42538) 2024-04-25T10:06:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b7b7fe640 1 -- 172.21.15.60:0/4159468131 learned_addr learned my addr 172.21.15.60:0/4159468131 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b7affd640 1 --2- 172.21.15.60:0/4159468131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c15ef20 0x7f3b7c0b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b79ffb640 1 -- 172.21.15.60:0/4159468131 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912556540 0 0) 0x7f3b7c0cc320 con 0x7f3b7c099a30 2024-04-25T10:06:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b79ffb640 1 -- 172.21.15.60:0/4159468131 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b64003530 con 0x7f3b7c099a30 2024-04-25T10:06:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b79ffb640 1 -- 172.21.15.60:0/4159468131 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3926329813 0 0) 0x7f3b64003530 con 0x7f3b7c099a30 2024-04-25T10:06:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b79ffb640 1 -- 172.21.15.60:0/4159468131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c15ef20 msgr2=0x7f3b7c0b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b79ffb640 1 --2- 172.21.15.60:0/4159468131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c15ef20 0x7f3b7c0b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b79ffb640 1 -- 172.21.15.60:0/4159468131 >> v1:172.21.15.100:6789/0 conn(0x7f3b7c0cfcf0 legacy=0x7f3b7c0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b79ffb640 1 -- 172.21.15.60:0/4159468131 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c0d04d0 con 0x7f3b7c099a30 2024-04-25T10:06:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b79ffb640 1 -- 172.21.15.60:0/4159468131 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f3b74001a00 con 0x7f3b7c099a30 2024-04-25T10:06:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b79ffb640 1 -- 172.21.15.60:0/4159468131 >> v1:172.21.15.60:6789/0 conn(0x7f3b7c099a30 legacy=0x7f3b7c0cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b79ffb640 1 --2- 172.21.15.60:0/4159468131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b640039f0 0x7f3b64003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b79ffb640 1 --2- 172.21.15.60:0/4159468131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c15ef20 0x7f3b64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b79ffb640 1 --2- 172.21.15.60:0/4159468131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b64004d50 0x7f3b640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b79ffb640 1 -- 172.21.15.60:0/4159468131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c0d07b0 con 0x7f3b640039f0 2024-04-25T10:06:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b79ffb640 1 -- 172.21.15.60:0/4159468131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6400ba80 con 0x7f3b64004d50 2024-04-25T10:06:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b79ffb640 1 -- 172.21.15.60:0/4159468131 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6400bc80 con 0x7f3b7c15ef20 2024-04-25T10:06:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b7affd640 1 --2- 172.21.15.60:0/4159468131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b640039f0 0x7f3b64003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b7a7fc640 1 --2- 172.21.15.60:0/4159468131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c15ef20 0x7f3b64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b7b7fe640 1 --2- 172.21.15.60:0/4159468131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b64004d50 0x7f3b640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b7affd640 1 -- 172.21.15.60:0/4159468131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c15ef20 msgr2=0x7f3b64004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b7affd640 1 --2- 172.21.15.60:0/4159468131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c15ef20 0x7f3b64004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b7affd640 1 -- 172.21.15.60:0/4159468131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b64004d50 msgr2=0x7f3b640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b7affd640 1 --2- 172.21.15.60:0/4159468131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b64004d50 0x7f3b640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b7affd640 1 -- 172.21.15.60:0/4159468131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b64003530 con 0x7f3b640039f0 2024-04-25T10:06:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b7a7fc640 1 --2- 172.21.15.60:0/4159468131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c15ef20 0x7f3b64004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:09.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.227+0000 7f3b7b7fe640 1 --2- 172.21.15.60:0/4159468131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b64004d50 0x7f3b640051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:09.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.231+0000 7f3b7affd640 1 --2- 172.21.15.60:0/4159468131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b640039f0 0x7f3b64003e10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009ab0 tx=0x7f3b6c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd3f4b62181a2a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.231+0000 7f3b79ffb640 1 -- 172.21.15.60:0/4159468131 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0096b0 con 0x7f3b640039f0 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.231+0000 7f3b79ffb640 1 -- 172.21.15.60:0/4159468131 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c0a6430 con 0x7f3b640039f0 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.231+0000 7f3b79ffb640 1 -- 172.21.15.60:0/4159468131 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0a6820 con 0x7f3b640039f0 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.231+0000 7f3b818cf640 1 -- 172.21.15.60:0/4159468131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b640039f0 msgr2=0x7f3b64003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.231+0000 7f3b818cf640 1 --2- 172.21.15.60:0/4159468131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b640039f0 0x7f3b64003e10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009ab0 tx=0x7f3b6c01ca50 comp rx=0 tx=0).stop 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.231+0000 7f3b7b7fe640 1 -- 172.21.15.60:0/4159468131 reap_dead start 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.231+0000 7f3b818cf640 1 -- 172.21.15.60:0/4159468131 shutdown_connections 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.231+0000 7f3b818cf640 1 -- 172.21.15.60:0/4159468131 >> 172.21.15.60:0/4159468131 conn(0x7f3b7c0b3200 msgr2=0x7f3b7c0b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.231+0000 7f3b818cf640 1 -- 172.21.15.60:0/4159468131 shutdown_connections 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.231+0000 7f3b818cf640 1 -- 172.21.15.60:0/4159468131 wait complete. 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.231+0000 7f3b818cf640 1 Processor -- start 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.231+0000 7f3b818cf640 1 -- start start 2024-04-25T10:06:09.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b818cf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c0b60a0 0x7f3b7c0c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b818cf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c1568b0 0x7f3b7c155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b818cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c156e00 0x7f3b7c156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b818cf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c15f300 con 0x7f3b7c156e00 2024-04-25T10:06:09.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b818cf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c0cdb80 con 0x7f3b7c1568b0 2024-04-25T10:06:09.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b818cf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c15f480 con 0x7f3b7c0b60a0 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b7b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c156e00 0x7f3b7c156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b7b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c156e00 0x7f3b7c156270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40834/0 (socket says 172.21.15.60:40834) 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b7b7fe640 1 -- 172.21.15.60:0/417776885 learned_addr learned my addr 172.21.15.60:0/417776885 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b7a7fc640 1 --2- 172.21.15.60:0/417776885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c1568b0 0x7f3b7c155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b7affd640 1 --2- 172.21.15.60:0/417776885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c0b60a0 0x7f3b7c0c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b7b7fe640 1 -- 172.21.15.60:0/417776885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c0b60a0 msgr2=0x7f3b7c0c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b7b7fe640 1 --2- 172.21.15.60:0/417776885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c0b60a0 0x7f3b7c0c6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b7b7fe640 1 -- 172.21.15.60:0/417776885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c1568b0 msgr2=0x7f3b7c155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b7b7fe640 1 --2- 172.21.15.60:0/417776885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c1568b0 0x7f3b7c155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b7b7fe640 1 -- 172.21.15.60:0/417776885 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c0c5490 con 0x7f3b7c156e00 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b7b7fe640 1 --2- 172.21.15.60:0/417776885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c156e00 0x7f3b7c156270 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0b3da0 tx=0x7f3b7400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b63fff640 1 -- 172.21.15.60:0/417776885 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b74004d70 con 0x7f3b7c156e00 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b63fff640 1 -- 172.21.15.60:0/417776885 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7400bb80 con 0x7f3b7c156e00 2024-04-25T10:06:09.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b818cf640 1 -- 172.21.15.60:0/417776885 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7c0c5770 con 0x7f3b7c156e00 2024-04-25T10:06:09.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b63fff640 1 -- 172.21.15.60:0/417776885 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b74007110 con 0x7f3b7c156e00 2024-04-25T10:06:09.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b818cf640 1 -- 172.21.15.60:0/417776885 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7c0c5d30 con 0x7f3b7c156e00 2024-04-25T10:06:09.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b818cf640 1 -- 172.21.15.60:0/417776885 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7c074770 con 0x7f3b7c156e00 2024-04-25T10:06:09.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b63fff640 1 -- 172.21.15.60:0/417776885 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b7401b060 con 0x7f3b7c156e00 2024-04-25T10:06:09.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b63fff640 1 --2- 172.21.15.60:0/417776885 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3b68066f90 0x7f3b68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b63fff640 1 -- 172.21.15.60:0/417776885 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f3b74084d40 con 0x7f3b7c156e00 2024-04-25T10:06:09.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.235+0000 7f3b7affd640 1 --2- 172.21.15.60:0/417776885 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3b68066f90 0x7f3b68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.239+0000 7f3b7affd640 1 --2- 172.21.15.60:0/417776885 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3b68066f90 0x7f3b68069450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0ad000 tx=0x7f3b6c0b2040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.239+0000 7f3b63fff640 1 -- 172.21.15.60:0/417776885 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3b74055d50 con 0x7f3b7c156e00 2024-04-25T10:06:09.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.399+0000 7f3b818cf640 1 -- 172.21.15.60:0/417776885 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3b7c0b4070 con 0x7f3b7c156e00 2024-04-25T10:06:09.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.399+0000 7f3b63fff640 1 -- 172.21.15.60:0/417776885 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3b74055650 con 0x7f3b7c156e00 2024-04-25T10:06:09.407 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:06:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.403+0000 7f3b818cf640 1 -- 172.21.15.60:0/417776885 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3b68066f90 msgr2=0x7f3b68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.403+0000 7f3b818cf640 1 --2- 172.21.15.60:0/417776885 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3b68066f90 0x7f3b68069450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0ad000 tx=0x7f3b6c0b2040 comp rx=0 tx=0).stop 2024-04-25T10:06:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.403+0000 7f3b818cf640 1 -- 172.21.15.60:0/417776885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c156e00 msgr2=0x7f3b7c156270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.403+0000 7f3b818cf640 1 --2- 172.21.15.60:0/417776885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c156e00 0x7f3b7c156270 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0b3da0 tx=0x7f3b7400b4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.403+0000 7f3b818cf640 1 -- 172.21.15.60:0/417776885 shutdown_connections 2024-04-25T10:06:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.403+0000 7f3b818cf640 1 --2- 172.21.15.60:0/417776885 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3b68066f90 0x7f3b68069450 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.403+0000 7f3b818cf640 1 --2- 172.21.15.60:0/417776885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c156e00 0x7f3b7c156270 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.403+0000 7f3b818cf640 1 --2- 172.21.15.60:0/417776885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c1568b0 0x7f3b7c155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.403+0000 7f3b818cf640 1 --2- 172.21.15.60:0/417776885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c0b60a0 0x7f3b7c0c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.403+0000 7f3b818cf640 1 -- 172.21.15.60:0/417776885 >> 172.21.15.60:0/417776885 conn(0x7f3b7c0b3200 msgr2=0x7f3b7c0cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.403+0000 7f3b818cf640 1 -- 172.21.15.60:0/417776885 shutdown_connections 2024-04-25T10:06:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.403+0000 7f3b818cf640 1 -- 172.21.15.60:0/417776885 wait complete. 2024-04-25T10:06:09.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:09.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.555+0000 7f8c1ea9f640 1 Processor -- start 2024-04-25T10:06:09.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.555+0000 7f8c1ea9f640 1 -- start start 2024-04-25T10:06:09.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.555+0000 7f8c1ea9f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8c1815b6f0 0x7f8c181620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.555+0000 7f8c1ea9f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c1815e0d0 0x7f8c1815e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.555+0000 7f8c1ea9f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1815bad0 con 0x7f8c180ce7e0 2024-04-25T10:06:09.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.555+0000 7f8c1ea9f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c18162610 con 0x7f8c1815b6f0 2024-04-25T10:06:09.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.555+0000 7f8c1ea9f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18161c40 con 0x7f8c1815e0d0 2024-04-25T10:06:09.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.555+0000 7f8c1c83b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8c1815b6f0 0x7f8c181620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.555+0000 7f8c1c83b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8c1815b6f0 0x7f8c181620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40842/0 (socket says 172.21.15.60:40842) 2024-04-25T10:06:09.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.555+0000 7f8c1c83b640 1 -- 172.21.15.60:0/2398133478 learned_addr learned my addr 172.21.15.60:0/2398133478 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:09.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.555+0000 7f8c17fff640 1 --2- 172.21.15.60:0/2398133478 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c1815e0d0 0x7f8c1815e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.555+0000 7f8c1c83b640 1 -- 172.21.15.60:0/2398133478 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c1815e0d0 msgr2=0x7f8c1815e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.555+0000 7f8c1c83b640 1 --2- 172.21.15.60:0/2398133478 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c1815e0d0 0x7f8c1815e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.555+0000 7f8c1c83b640 1 -- 172.21.15.60:0/2398133478 >> v1:172.21.15.100:6789/0 conn(0x7f8c180ce7e0 legacy=0x7f8c180d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.555+0000 7f8c1c83b640 1 -- 172.21.15.60:0/2398133478 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1815ebe0 con 0x7f8c1815b6f0 2024-04-25T10:06:09.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.559+0000 7f8c1c83b640 1 --2- 172.21.15.60:0/2398133478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c1815b6f0 0x7f8c181620d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009870 tx=0x7f8c0801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739c140ec963382b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.559+0000 7f8c177fe640 1 -- 172.21.15.60:0/2398133478 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c080ab650 con 0x7f8c1815b6f0 2024-04-25T10:06:09.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.559+0000 7f8c177fe640 1 -- 172.21.15.60:0/2398133478 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c080b0070 con 0x7f8c1815b6f0 2024-04-25T10:06:09.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.559+0000 7f8c177fe640 1 -- 172.21.15.60:0/2398133478 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c080aba10 con 0x7f8c1815b6f0 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.559+0000 7f8c1ea9f640 1 -- 172.21.15.60:0/2398133478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c1815b6f0 msgr2=0x7f8c181620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.559+0000 7f8c1ea9f640 1 --2- 172.21.15.60:0/2398133478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c1815b6f0 0x7f8c181620d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009870 tx=0x7f8c0801c560 comp rx=0 tx=0).stop 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.559+0000 7f8c1ea9f640 1 -- 172.21.15.60:0/2398133478 shutdown_connections 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.559+0000 7f8c1ea9f640 1 --2- 172.21.15.60:0/2398133478 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c1815e0d0 0x7f8c1815e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.559+0000 7f8c1ea9f640 1 --2- 172.21.15.60:0/2398133478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c1815b6f0 0x7f8c181620d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.559+0000 7f8c1ea9f640 1 -- 172.21.15.60:0/2398133478 >> 172.21.15.60:0/2398133478 conn(0x7f8c18072340 msgr2=0x7f8c18072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.559+0000 7f8c1ea9f640 1 -- 172.21.15.60:0/2398133478 shutdown_connections 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.559+0000 7f8c1ea9f640 1 -- 172.21.15.60:0/2398133478 wait complete. 2024-04-25T10:06:09.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.559+0000 7f8c1ea9f640 1 Processor -- start 2024-04-25T10:06:09.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.559+0000 7f8c1ea9f640 1 -- start start 2024-04-25T10:06:09.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c1ea9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c180ce7e0 0x7f8c180c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c1ea9f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c1815b6f0 0x7f8c1814b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c1ea9f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c1815e0d0 0x7f8c1814b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c1ea9f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c180b96d0 con 0x7f8c1815b6f0 2024-04-25T10:06:09.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c1ea9f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c180b9550 con 0x7f8c180ce7e0 2024-04-25T10:06:09.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c1ea9f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c180cf000 con 0x7f8c1815e0d0 2024-04-25T10:06:09.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c17fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c1815b6f0 0x7f8c1814b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c17fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c1815b6f0 0x7f8c1814b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40848/0 (socket says 172.21.15.60:40848) 2024-04-25T10:06:09.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c17fff640 1 -- 172.21.15.60:0/2193973175 learned_addr learned my addr 172.21.15.60:0/2193973175 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:09.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c1d03c640 1 --2- 172.21.15.60:0/2193973175 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c1815e0d0 0x7f8c1814b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c1c83b640 1 --2- 172.21.15.60:0/2193973175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c180ce7e0 0x7f8c180c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c17fff640 1 -- 172.21.15.60:0/2193973175 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c1815e0d0 msgr2=0x7f8c1814b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c17fff640 1 --2- 172.21.15.60:0/2193973175 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c1815e0d0 0x7f8c1814b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c17fff640 1 -- 172.21.15.60:0/2193973175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c180ce7e0 msgr2=0x7f8c180c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c17fff640 1 --2- 172.21.15.60:0/2193973175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c180ce7e0 0x7f8c180c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c17fff640 1 -- 172.21.15.60:0/2193973175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c180c7190 con 0x7f8c1815b6f0 2024-04-25T10:06:09.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c1d03c640 1 --2- 172.21.15.60:0/2193973175 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c1815e0d0 0x7f8c1814b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:09.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c1c83b640 1 --2- 172.21.15.60:0/2193973175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c180ce7e0 0x7f8c180c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:09.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c17fff640 1 --2- 172.21.15.60:0/2193973175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c1815b6f0 0x7f8c1814b2e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8c00007e40 tx=0x7f8c0000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c157fa640 1 -- 172.21.15.60:0/2193973175 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00014620 con 0x7f8c1815b6f0 2024-04-25T10:06:09.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c157fa640 1 -- 172.21.15.60:0/2193973175 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c00019070 con 0x7f8c1815b6f0 2024-04-25T10:06:09.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c1ea9f640 1 -- 172.21.15.60:0/2193973175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c18148ce0 con 0x7f8c1815b6f0 2024-04-25T10:06:09.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c157fa640 1 -- 172.21.15.60:0/2193973175 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00014a00 con 0x7f8c1815b6f0 2024-04-25T10:06:09.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c1ea9f640 1 -- 172.21.15.60:0/2193973175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c180c6eb0 con 0x7f8c1815b6f0 2024-04-25T10:06:09.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c1ea9f640 1 -- 172.21.15.60:0/2193973175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c18074770 con 0x7f8c1815b6f0 2024-04-25T10:06:09.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c157fa640 1 -- 172.21.15.60:0/2193973175 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c00014ba0 con 0x7f8c1815b6f0 2024-04-25T10:06:09.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.563+0000 7f8c157fa640 1 --2- 172.21.15.60:0/2193973175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8bf8066f60 0x7f8bf8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.567+0000 7f8c157fa640 1 -- 172.21.15.60:0/2193973175 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8c00086840 con 0x7f8c1815b6f0 2024-04-25T10:06:09.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.567+0000 7f8c1c83b640 1 --2- 172.21.15.60:0/2193973175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8bf8066f60 0x7f8bf8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.567+0000 7f8c1c83b640 1 --2- 172.21.15.60:0/2193973175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8bf8066f60 0x7f8bf8069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8c080af000 tx=0x7f8c080a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.567+0000 7f8c157fa640 1 -- 172.21.15.60:0/2193973175 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c000577a0 con 0x7f8c1815b6f0 2024-04-25T10:06:09.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.727+0000 7f8c1ea9f640 1 -- 172.21.15.60:0/2193973175 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c180d16e0 con 0x7f8c1815b6f0 2024-04-25T10:06:09.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.731+0000 7f8c157fa640 1 -- 172.21.15.60:0/2193973175 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8c000570a0 con 0x7f8c1815b6f0 2024-04-25T10:06:09.737 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:06:09.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.731+0000 7f8c1ea9f640 1 -- 172.21.15.60:0/2193973175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8bf8066f60 msgr2=0x7f8bf8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.731+0000 7f8c1ea9f640 1 --2- 172.21.15.60:0/2193973175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8bf8066f60 0x7f8bf8069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8c080af000 tx=0x7f8c080a7000 comp rx=0 tx=0).stop 2024-04-25T10:06:09.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.731+0000 7f8c1ea9f640 1 -- 172.21.15.60:0/2193973175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c1815b6f0 msgr2=0x7f8c1814b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.731+0000 7f8c1ea9f640 1 --2- 172.21.15.60:0/2193973175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c1815b6f0 0x7f8c1814b2e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8c00007e40 tx=0x7f8c0000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.731+0000 7f8c1ea9f640 1 -- 172.21.15.60:0/2193973175 shutdown_connections 2024-04-25T10:06:09.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.731+0000 7f8c1ea9f640 1 --2- 172.21.15.60:0/2193973175 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8bf8066f60 0x7f8bf8069420 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.731+0000 7f8c1ea9f640 1 --2- 172.21.15.60:0/2193973175 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c1815e0d0 0x7f8c1814b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.731+0000 7f8c1ea9f640 1 --2- 172.21.15.60:0/2193973175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c1815b6f0 0x7f8c1814b2e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.731+0000 7f8c1ea9f640 1 --2- 172.21.15.60:0/2193973175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c180ce7e0 0x7f8c180c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.731+0000 7f8c1ea9f640 1 -- 172.21.15.60:0/2193973175 >> 172.21.15.60:0/2193973175 conn(0x7f8c18072340 msgr2=0x7f8c180d2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.731+0000 7f8c1ea9f640 1 -- 172.21.15.60:0/2193973175 shutdown_connections 2024-04-25T10:06:09.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.731+0000 7f8c1ea9f640 1 -- 172.21.15.60:0/2193973175 wait complete. 2024-04-25T10:06:09.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:09.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb080c42640 1 Processor -- start 2024-04-25T10:06:09.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb080c42640 1 -- start start 2024-04-25T10:06:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb080c42640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb07c06f030 0x7fb07c0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb080c42640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb07c071250 0x7fb07c0716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb080c42640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb07c15ad30 con 0x7fb07c0b3f60 2024-04-25T10:06:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb080c42640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb07c15aaf0 con 0x7fb07c06f030 2024-04-25T10:06:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb080c42640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb07c15b010 con 0x7fb07c071250 2024-04-25T10:06:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb079d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb07c06f030 0x7fb07c0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb07a59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb07c0b3f60 0x7fb07c15a3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42552/0 (socket says 172.21.15.60:42552) 2024-04-25T10:06:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb07a59c640 1 -- 172.21.15.60:0/2823414919 learned_addr learned my addr 172.21.15.60:0/2823414919 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb07ad9d640 1 --2- 172.21.15.60:0/2823414919 >> v2:172.21.15.100:3300/0 conn(0x7fb07c071250 0x7fb07c0716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb07959a640 1 -- 172.21.15.60:0/2823414919 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500354434 0 0) 0x7fb07c15ad30 con 0x7fb07c0b3f60 2024-04-25T10:06:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb07959a640 1 -- 172.21.15.60:0/2823414919 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb068003530 con 0x7fb07c0b3f60 2024-04-25T10:06:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb079d9b640 1 -- 172.21.15.60:0/2823414919 >> v2:172.21.15.100:3300/0 conn(0x7fb07c071250 msgr2=0x7fb07c0716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb079d9b640 1 --2- 172.21.15.60:0/2823414919 >> v2:172.21.15.100:3300/0 conn(0x7fb07c071250 0x7fb07c0716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb079d9b640 1 -- 172.21.15.60:0/2823414919 >> v1:172.21.15.60:6789/0 conn(0x7fb07c0b3f60 legacy=0x7fb07c15a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb079d9b640 1 -- 172.21.15.60:0/2823414919 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c15b2f0 con 0x7fb07c06f030 2024-04-25T10:06:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.871+0000 7fb07ad9d640 1 --2- 172.21.15.60:0/2823414919 >> v2:172.21.15.100:3300/0 conn(0x7fb07c071250 0x7fb07c0716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb079d9b640 1 --2- 172.21.15.60:0/2823414919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb07c06f030 0x7fb07c0b2e00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0cc3d0 tx=0x7fb06401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de4c52135018080 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb07959a640 1 -- 172.21.15.60:0/2823414919 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0640b0070 con 0x7fb07c06f030 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb07959a640 1 -- 172.21.15.60:0/2823414919 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0640ab400 con 0x7fb07c06f030 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb07959a640 1 -- 172.21.15.60:0/2823414919 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0640ab7d0 con 0x7fb07c06f030 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb080c42640 1 -- 172.21.15.60:0/2823414919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb07c06f030 msgr2=0x7fb07c0b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb080c42640 1 --2- 172.21.15.60:0/2823414919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb07c06f030 0x7fb07c0b2e00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0cc3d0 tx=0x7fb06401c3f0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb080c42640 1 -- 172.21.15.60:0/2823414919 shutdown_connections 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb080c42640 1 --2- 172.21.15.60:0/2823414919 >> v2:172.21.15.100:3300/0 conn(0x7fb07c071250 0x7fb07c0716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb080c42640 1 --2- 172.21.15.60:0/2823414919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb07c06f030 0x7fb07c0b2e00 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb080c42640 1 -- 172.21.15.60:0/2823414919 >> 172.21.15.60:0/2823414919 conn(0x7fb07c0670b0 msgr2=0x7fb07c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb080c42640 1 -- 172.21.15.60:0/2823414919 shutdown_connections 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb080c42640 1 -- 172.21.15.60:0/2823414919 wait complete. 2024-04-25T10:06:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb080c42640 1 Processor -- start 2024-04-25T10:06:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb080c42640 1 -- start start 2024-04-25T10:06:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb080c42640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb07c06f030 0x7fb07c0ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb080c42640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb07c071250 0x7fb07c14cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb080c42640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb07c0b3f60 0x7fb07c06d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb080c42640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c15b5a0 con 0x7fb07c06f030 2024-04-25T10:06:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb080c42640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c15b720 con 0x7fb07c071250 2024-04-25T10:06:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb080c42640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c15b8a0 con 0x7fb07c0b3f60 2024-04-25T10:06:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb07a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb07c06f030 0x7fb07c0ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb07a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb07c06f030 0x7fb07c0ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40866/0 (socket says 172.21.15.60:40866) 2024-04-25T10:06:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb07a59c640 1 -- 172.21.15.60:0/632310830 learned_addr learned my addr 172.21.15.60:0/632310830 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb079d9b640 1 --2- 172.21.15.60:0/632310830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb07c071250 0x7fb07c14cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb07ad9d640 1 --2- 172.21.15.60:0/632310830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb07c0b3f60 0x7fb07c06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb07a59c640 1 -- 172.21.15.60:0/632310830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb07c0b3f60 msgr2=0x7fb07c06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb07a59c640 1 --2- 172.21.15.60:0/632310830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb07c0b3f60 0x7fb07c06d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb07a59c640 1 -- 172.21.15.60:0/632310830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb07c071250 msgr2=0x7fb07c14cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb07a59c640 1 --2- 172.21.15.60:0/632310830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb07c071250 0x7fb07c14cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb07a59c640 1 -- 172.21.15.60:0/632310830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c1f9a10 con 0x7fb07c06f030 2024-04-25T10:06:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb07a59c640 1 --2- 172.21.15.60:0/632310830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb07c06f030 0x7fb07c0ade40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb06c00c3c0 tx=0x7fb06c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.875+0000 7fb072ffd640 1 -- 172.21.15.60:0/632310830 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c017070 con 0x7fb07c06f030 2024-04-25T10:06:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.879+0000 7fb072ffd640 1 -- 172.21.15.60:0/632310830 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06c005cf0 con 0x7fb07c06f030 2024-04-25T10:06:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.879+0000 7fb080c42640 1 -- 172.21.15.60:0/632310830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c1f9cb0 con 0x7fb07c06f030 2024-04-25T10:06:09.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.879+0000 7fb072ffd640 1 -- 172.21.15.60:0/632310830 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c0072f0 con 0x7fb07c06f030 2024-04-25T10:06:09.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.879+0000 7fb080c42640 1 -- 172.21.15.60:0/632310830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c1fa270 con 0x7fb07c06f030 2024-04-25T10:06:09.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.879+0000 7fb080c42640 1 -- 172.21.15.60:0/632310830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07c1f6470 con 0x7fb07c06f030 2024-04-25T10:06:09.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.879+0000 7fb072ffd640 1 -- 172.21.15.60:0/632310830 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb06c01a070 con 0x7fb07c06f030 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.879+0000 7fb072ffd640 1 --2- 172.21.15.60:0/632310830 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb05c066f60 0x7fb05c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.879+0000 7fb072ffd640 1 -- 172.21.15.60:0/632310830 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb06c0844b0 con 0x7fb07c06f030 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.879+0000 7fb079d9b640 1 --2- 172.21.15.60:0/632310830 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb05c066f60 0x7fb05c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.879+0000 7fb079d9b640 1 --2- 172.21.15.60:0/632310830 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb05c066f60 0x7fb05c069420 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb0640ad000 tx=0x7fb064008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:09.883+0000 7fb072ffd640 1 -- 172.21.15.60:0/632310830 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb06c055430 con 0x7fb07c06f030 2024-04-25T10:06:10.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.043+0000 7fb080c42640 1 -- 172.21.15.60:0/632310830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb07c074770 con 0x7fb07c06f030 2024-04-25T10:06:10.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.043+0000 7fb072ffd640 1 -- 172.21.15.60:0/632310830 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb06c054d30 con 0x7fb07c06f030 2024-04-25T10:06:10.050 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.047+0000 7fb080c42640 1 -- 172.21.15.60:0/632310830 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb05c066f60 msgr2=0x7fb05c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.047+0000 7fb080c42640 1 --2- 172.21.15.60:0/632310830 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb05c066f60 0x7fb05c069420 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb0640ad000 tx=0x7fb064008040 comp rx=0 tx=0).stop 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.047+0000 7fb080c42640 1 -- 172.21.15.60:0/632310830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb07c06f030 msgr2=0x7fb07c0ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.047+0000 7fb080c42640 1 --2- 172.21.15.60:0/632310830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb07c06f030 0x7fb07c0ade40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb06c00c3c0 tx=0x7fb06c00c880 comp rx=0 tx=0).stop 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.047+0000 7fb080c42640 1 -- 172.21.15.60:0/632310830 shutdown_connections 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.047+0000 7fb080c42640 1 --2- 172.21.15.60:0/632310830 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb05c066f60 0x7fb05c069420 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.047+0000 7fb080c42640 1 --2- 172.21.15.60:0/632310830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb07c0b3f60 0x7fb07c06d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.047+0000 7fb080c42640 1 --2- 172.21.15.60:0/632310830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb07c071250 0x7fb07c14cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.047+0000 7fb080c42640 1 --2- 172.21.15.60:0/632310830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb07c06f030 0x7fb07c0ade40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.047+0000 7fb080c42640 1 -- 172.21.15.60:0/632310830 >> 172.21.15.60:0/632310830 conn(0x7fb07c0670b0 msgr2=0x7fb07c1510d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.047+0000 7fb080c42640 1 -- 172.21.15.60:0/632310830 shutdown_connections 2024-04-25T10:06:10.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.047+0000 7fb080c42640 1 -- 172.21.15.60:0/632310830 wait complete. 2024-04-25T10:06:10.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:10.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.203+0000 7f9c88ad0640 1 Processor -- start 2024-04-25T10:06:10.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.203+0000 7f9c88ad0640 1 -- start start 2024-04-25T10:06:10.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.203+0000 7f9c88ad0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9c8415bc90 0x7f9c8415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.203+0000 7f9c88ad0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c84071ca0 con 0x7f9c840b4660 2024-04-25T10:06:10.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.203+0000 7f9c88ad0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c840cca40 con 0x7f9c840ce7a0 2024-04-25T10:06:10.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.203+0000 7f9c88ad0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c840cc370 con 0x7f9c8415bc90 2024-04-25T10:06:10.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.203+0000 7f9c82d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9c840b4660 0x7f9c8415c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42554/0 (socket says 172.21.15.60:42554) 2024-04-25T10:06:10.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.203+0000 7f9c82d9d640 1 -- 172.21.15.60:0/1740356975 learned_addr learned my addr 172.21.15.60:0/1740356975 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:10.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.203+0000 7f9c81d9b640 1 --2- 172.21.15.60:0/1740356975 >> v2:172.21.15.100:3300/0 conn(0x7f9c8415bc90 0x7f9c8415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.203+0000 7f9c8159a640 1 -- 172.21.15.60:0/1740356975 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2482140613 0 0) 0x7f9c84071ca0 con 0x7f9c840b4660 2024-04-25T10:06:10.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.203+0000 7f9c8159a640 1 -- 172.21.15.60:0/1740356975 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c6c003530 con 0x7f9c840b4660 2024-04-25T10:06:10.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.203+0000 7f9c81d9b640 1 -- 172.21.15.60:0/1740356975 >> v1:172.21.15.100:6789/0 conn(0x7f9c840ce7a0 legacy=0x7f9c840ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.203+0000 7f9c8159a640 1 -- 172.21.15.60:0/1740356975 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1105650177 0 0) 0x7f9c6c003530 con 0x7f9c840b4660 2024-04-25T10:06:10.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.203+0000 7f9c81d9b640 1 -- 172.21.15.60:0/1740356975 >> v1:172.21.15.60:6789/0 conn(0x7f9c840b4660 legacy=0x7f9c8415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.203+0000 7f9c81d9b640 1 -- 172.21.15.60:0/1740356975 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c840ccd20 con 0x7f9c8415bc90 2024-04-25T10:06:10.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.207+0000 7f9c81d9b640 1 --2- 172.21.15.60:0/1740356975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c8415bc90 0x7f9c8415c220 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9c780097c0 tx=0x7f9c7801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4bdbfba37f84ff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.207+0000 7f9c8159a640 1 -- 172.21.15.60:0/1740356975 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c780b0070 con 0x7f9c8415bc90 2024-04-25T10:06:10.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.207+0000 7f9c8159a640 1 -- 172.21.15.60:0/1740356975 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c780ab400 con 0x7f9c8415bc90 2024-04-25T10:06:10.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.207+0000 7f9c8159a640 1 -- 172.21.15.60:0/1740356975 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c001a00 con 0x7f9c8415bc90 2024-04-25T10:06:10.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.207+0000 7f9c88ad0640 1 -- 172.21.15.60:0/1740356975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c8415bc90 msgr2=0x7f9c8415c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.207+0000 7f9c88ad0640 1 --2- 172.21.15.60:0/1740356975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c8415bc90 0x7f9c8415c220 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9c780097c0 tx=0x7f9c7801c3c0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.207+0000 7f9c88ad0640 1 -- 172.21.15.60:0/1740356975 shutdown_connections 2024-04-25T10:06:10.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.207+0000 7f9c88ad0640 1 --2- 172.21.15.60:0/1740356975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c8415bc90 0x7f9c8415c220 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.207+0000 7f9c88ad0640 1 -- 172.21.15.60:0/1740356975 >> 172.21.15.60:0/1740356975 conn(0x7f9c840b12d0 msgr2=0x7f9c840b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.207+0000 7f9c88ad0640 1 -- 172.21.15.60:0/1740356975 shutdown_connections 2024-04-25T10:06:10.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.207+0000 7f9c88ad0640 1 -- 172.21.15.60:0/1740356975 wait complete. 2024-04-25T10:06:10.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.207+0000 7f9c88ad0640 1 Processor -- start 2024-04-25T10:06:10.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.207+0000 7f9c88ad0640 1 -- start start 2024-04-25T10:06:10.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.207+0000 7f9c88ad0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c840ce7a0 0x7f9c84157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c88ad0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c8415bc90 0x7f9c8414a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c88ad0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c841559e0 0x7f9c84155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c88ad0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c841695d0 con 0x7f9c841559e0 2024-04-25T10:06:10.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c88ad0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c840b3db0 con 0x7f9c8415bc90 2024-04-25T10:06:10.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c88ad0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c84169750 con 0x7f9c840ce7a0 2024-04-25T10:06:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c82d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c841559e0 0x7f9c84155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c82d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c841559e0 0x7f9c84155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40880/0 (socket says 172.21.15.60:40880) 2024-04-25T10:06:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c82d9d640 1 -- 172.21.15.60:0/3911233533 learned_addr learned my addr 172.21.15.60:0/3911233533 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c81d9b640 1 --2- 172.21.15.60:0/3911233533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c8415bc90 0x7f9c8414a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c8259c640 1 --2- 172.21.15.60:0/3911233533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c840ce7a0 0x7f9c84157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c82d9d640 1 -- 172.21.15.60:0/3911233533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c840ce7a0 msgr2=0x7f9c84157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c82d9d640 1 --2- 172.21.15.60:0/3911233533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c840ce7a0 0x7f9c84157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c82d9d640 1 -- 172.21.15.60:0/3911233533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c8415bc90 msgr2=0x7f9c8414a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c82d9d640 1 --2- 172.21.15.60:0/3911233533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c8415bc90 0x7f9c8414a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c82d9d640 1 -- 172.21.15.60:0/3911233533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c841585a0 con 0x7f9c841559e0 2024-04-25T10:06:10.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c82d9d640 1 --2- 172.21.15.60:0/3911233533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c841559e0 0x7f9c84155e70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00cc00 tx=0x7f9c7c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c72ffd640 1 -- 172.21.15.60:0/3911233533 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c017070 con 0x7f9c841559e0 2024-04-25T10:06:10.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c72ffd640 1 -- 172.21.15.60:0/3911233533 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c00ad40 con 0x7f9c841559e0 2024-04-25T10:06:10.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c88ad0640 1 -- 172.21.15.60:0/3911233533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c84158880 con 0x7f9c841559e0 2024-04-25T10:06:10.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c72ffd640 1 -- 172.21.15.60:0/3911233533 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c012640 con 0x7f9c841559e0 2024-04-25T10:06:10.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c88ad0640 1 -- 172.21.15.60:0/3911233533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c84158e40 con 0x7f9c841559e0 2024-04-25T10:06:10.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c88ad0640 1 -- 172.21.15.60:0/3911233533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c84074770 con 0x7f9c841559e0 2024-04-25T10:06:10.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c72ffd640 1 -- 172.21.15.60:0/3911233533 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9c7c01a070 con 0x7f9c841559e0 2024-04-25T10:06:10.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c72ffd640 1 --2- 172.21.15.60:0/3911233533 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9c64066f90 0x7f9c64069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c72ffd640 1 -- 172.21.15.60:0/3911233533 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9c7c084ea0 con 0x7f9c841559e0 2024-04-25T10:06:10.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.211+0000 7f9c8259c640 1 --2- 172.21.15.60:0/3911233533 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9c64066f90 0x7f9c64069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.215+0000 7f9c8259c640 1 --2- 172.21.15.60:0/3911233533 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9c64066f90 0x7f9c64069450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9c74000f70 tx=0x7f9c74009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.215+0000 7f9c72ffd640 1 -- 172.21.15.60:0/3911233533 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9c7c055e90 con 0x7f9c841559e0 2024-04-25T10:06:10.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.407+0000 7f9c88ad0640 1 -- 172.21.15.60:0/3911233533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c84157360 con 0x7f9c841559e0 2024-04-25T10:06:10.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.407+0000 7f9c72ffd640 1 -- 172.21.15.60:0/3911233533 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f9c7c055790 con 0x7f9c841559e0 2024-04-25T10:06:10.414 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:10.415 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/622088871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":622088871},{"type":"v1","addr":"172.21.15.100:6836","nonce":622088871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3988204202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3988204202},{"type":"v1","addr":"172.21.15.60:6833","nonce":3988204202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3591836867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3591836867},{"type":"v1","addr":"172.21.15.60:6835","nonce":3591836867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:20.405486+0000","modified":"2024-04-25T10:05:23.453258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/690070228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":690070228},{"type":"v1","addr":"172.21.15.100:6837","nonce":690070228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:06:10.415 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-04-25T10:06:10.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.411+0000 7f9c88ad0640 1 -- 172.21.15.60:0/3911233533 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9c64066f90 msgr2=0x7f9c64069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.411+0000 7f9c88ad0640 1 --2- 172.21.15.60:0/3911233533 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9c64066f90 0x7f9c64069450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9c74000f70 tx=0x7f9c74009040 comp rx=0 tx=0).stop 2024-04-25T10:06:10.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.411+0000 7f9c88ad0640 1 -- 172.21.15.60:0/3911233533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c841559e0 msgr2=0x7f9c84155e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.411+0000 7f9c88ad0640 1 --2- 172.21.15.60:0/3911233533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c841559e0 0x7f9c84155e70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00cc00 tx=0x7f9c7c00a430 comp rx=0 tx=0).stop 2024-04-25T10:06:10.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.411+0000 7f9c88ad0640 1 -- 172.21.15.60:0/3911233533 shutdown_connections 2024-04-25T10:06:10.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.411+0000 7f9c88ad0640 1 --2- 172.21.15.60:0/3911233533 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9c64066f90 0x7f9c64069450 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.411+0000 7f9c88ad0640 1 --2- 172.21.15.60:0/3911233533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c841559e0 0x7f9c84155e70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.411+0000 7f9c88ad0640 1 --2- 172.21.15.60:0/3911233533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c8415bc90 0x7f9c8414a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.411+0000 7f9c88ad0640 1 --2- 172.21.15.60:0/3911233533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c840ce7a0 0x7f9c84157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.411+0000 7f9c88ad0640 1 -- 172.21.15.60:0/3911233533 >> 172.21.15.60:0/3911233533 conn(0x7f9c840b12d0 msgr2=0x7f9c84071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.411+0000 7f9c88ad0640 1 -- 172.21.15.60:0/3911233533 shutdown_connections 2024-04-25T10:06:10.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.411+0000 7f9c88ad0640 1 -- 172.21.15.60:0/3911233533 wait complete. 2024-04-25T10:06:10.432 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:10.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.551+0000 7efe08129640 1 Processor -- start 2024-04-25T10:06:10.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.551+0000 7efe08129640 1 -- start start 2024-04-25T10:06:10.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.551+0000 7efe08129640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efe0015f480 0x7efe0015f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.551+0000 7efe08129640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe000cd390 0x7efe0009a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.551+0000 7efe08129640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe0015fda0 con 0x7efe0009a930 2024-04-25T10:06:10.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.551+0000 7efe08129640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe000cc5d0 con 0x7efe0015f480 2024-04-25T10:06:10.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.551+0000 7efe08129640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe000cc8b0 con 0x7efe000cd390 2024-04-25T10:06:10.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.551+0000 7efe05ec5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efe0015f480 0x7efe0015f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.551+0000 7efe056c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe000cd390 0x7efe0009a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.551+0000 7efe05ec5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efe0015f480 0x7efe0015f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:45234/0 (socket says 172.21.15.60:45234) 2024-04-25T10:06:10.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.551+0000 7efe05ec5640 1 -- 172.21.15.60:0/2009729569 learned_addr learned my addr 172.21.15.60:0/2009729569 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:10.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe05ec5640 1 -- 172.21.15.60:0/2009729569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe000cd390 msgr2=0x7efe0009a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe05ec5640 1 --2- 172.21.15.60:0/2009729569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe000cd390 0x7efe0009a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe05ec5640 1 -- 172.21.15.60:0/2009729569 >> v1:172.21.15.100:6789/0 conn(0x7efe0009a930 legacy=0x7efe000cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe05ec5640 1 -- 172.21.15.60:0/2009729569 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe000ccb90 con 0x7efe0015f480 2024-04-25T10:06:10.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe056c4640 1 --2- 172.21.15.60:0/2009729569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe000cd390 0x7efe0009a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:10.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe05ec5640 1 --2- 172.21.15.60:0/2009729569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe0015f480 0x7efe0015f860 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009870 tx=0x7efdfc01c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6303c0060a5a418b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe04ec3640 1 -- 172.21.15.60:0/2009729569 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0aa670 con 0x7efe0015f480 2024-04-25T10:06:10.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe04ec3640 1 -- 172.21.15.60:0/2009729569 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdfc0af070 con 0x7efe0015f480 2024-04-25T10:06:10.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe04ec3640 1 -- 172.21.15.60:0/2009729569 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0aaa30 con 0x7efe0015f480 2024-04-25T10:06:10.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe08129640 1 -- 172.21.15.60:0/2009729569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe0015f480 msgr2=0x7efe0015f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe08129640 1 --2- 172.21.15.60:0/2009729569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe0015f480 0x7efe0015f860 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009870 tx=0x7efdfc01c530 comp rx=0 tx=0).stop 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe08129640 1 -- 172.21.15.60:0/2009729569 shutdown_connections 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe08129640 1 --2- 172.21.15.60:0/2009729569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe000cd390 0x7efe0009a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe08129640 1 --2- 172.21.15.60:0/2009729569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe0015f480 0x7efe0015f860 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe08129640 1 -- 172.21.15.60:0/2009729569 >> 172.21.15.60:0/2009729569 conn(0x7efe000b32a0 msgr2=0x7efe000b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe08129640 1 -- 172.21.15.60:0/2009729569 shutdown_connections 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.555+0000 7efe08129640 1 -- 172.21.15.60:0/2009729569 wait complete. 2024-04-25T10:06:10.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe08129640 1 Processor -- start 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe08129640 1 -- start start 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe08129640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe0009a930 0x7efe000b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe08129640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe000cd390 0x7efe0014c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe08129640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe0015f480 0x7efe0014cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe08129640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0015cd40 con 0x7efe0015f480 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe08129640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0015cbc0 con 0x7efe0009a930 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe08129640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe001687c0 con 0x7efe000cd390 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe066c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe0015f480 0x7efe0014cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe066c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe0015f480 0x7efe0014cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40892/0 (socket says 172.21.15.60:40892) 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe066c6640 1 -- 172.21.15.60:0/2619700289 learned_addr learned my addr 172.21.15.60:0/2619700289 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe056c4640 1 --2- 172.21.15.60:0/2619700289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe000cd390 0x7efe0014c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe05ec5640 1 --2- 172.21.15.60:0/2619700289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe0009a930 0x7efe000b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe066c6640 1 -- 172.21.15.60:0/2619700289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe000cd390 msgr2=0x7efe0014c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe066c6640 1 --2- 172.21.15.60:0/2619700289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe000cd390 0x7efe0014c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe066c6640 1 -- 172.21.15.60:0/2619700289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe0009a930 msgr2=0x7efe000b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe066c6640 1 --2- 172.21.15.60:0/2619700289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe0009a930 0x7efe000b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe066c6640 1 -- 172.21.15.60:0/2619700289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0015df70 con 0x7efe0015f480 2024-04-25T10:06:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe05ec5640 1 --2- 172.21.15.60:0/2619700289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe0009a930 0x7efe000b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe066c6640 1 --2- 172.21.15.60:0/2619700289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe0015f480 0x7efe0014cd30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7efdf800c730 tx=0x7efdf800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efdf67fc640 1 -- 172.21.15.60:0/2619700289 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8003b80 con 0x7efe0015f480 2024-04-25T10:06:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efdf67fc640 1 -- 172.21.15.60:0/2619700289 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf8003d20 con 0x7efe0015f480 2024-04-25T10:06:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe08129640 1 -- 172.21.15.60:0/2619700289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0015e250 con 0x7efe0015f480 2024-04-25T10:06:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efdf67fc640 1 -- 172.21.15.60:0/2619700289 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8005310 con 0x7efe0015f480 2024-04-25T10:06:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe08129640 1 -- 172.21.15.60:0/2619700289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0015e810 con 0x7efe0015f480 2024-04-25T10:06:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.559+0000 7efe08129640 1 -- 172.21.15.60:0/2619700289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe00074770 con 0x7efe0015f480 2024-04-25T10:06:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.563+0000 7efdf67fc640 1 -- 172.21.15.60:0/2619700289 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efdf80054b0 con 0x7efe0015f480 2024-04-25T10:06:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.563+0000 7efdf67fc640 1 --2- 172.21.15.60:0/2619700289 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efde4066f60 0x7efde4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.563+0000 7efdf67fc640 1 -- 172.21.15.60:0/2619700289 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7efdf8084f30 con 0x7efe0015f480 2024-04-25T10:06:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.563+0000 7efe05ec5640 1 --2- 172.21.15.60:0/2619700289 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efde4066f60 0x7efde4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.563+0000 7efe05ec5640 1 --2- 172.21.15.60:0/2619700289 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efde4066f60 0x7efde4069420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0ae000 tx=0x7efdfc0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.567+0000 7efdf67fc640 1 -- 172.21.15.60:0/2619700289 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efdf8055eb0 con 0x7efe0015f480 2024-04-25T10:06:10.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.723+0000 7efe08129640 1 -- 172.21.15.60:0/2619700289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe000b7730 con 0x7efe0015f480 2024-04-25T10:06:10.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.727+0000 7efdf67fc640 1 -- 172.21.15.60:0/2619700289 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16857 (secure 0 0 0) 0x7efdf80557b0 con 0x7efe0015f480 2024-04-25T10:06:10.734 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:10.735 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:01.568957+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:10.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.731+0000 7efe08129640 1 -- 172.21.15.60:0/2619700289 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efde4066f60 msgr2=0x7efde4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.731+0000 7efe08129640 1 --2- 172.21.15.60:0/2619700289 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efde4066f60 0x7efde4069420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0ae000 tx=0x7efdfc0a7000 comp rx=0 tx=0).stop 2024-04-25T10:06:10.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.731+0000 7efe08129640 1 -- 172.21.15.60:0/2619700289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe0015f480 msgr2=0x7efe0014cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.731+0000 7efe08129640 1 --2- 172.21.15.60:0/2619700289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe0015f480 0x7efe0014cd30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7efdf800c730 tx=0x7efdf800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.731+0000 7efe08129640 1 -- 172.21.15.60:0/2619700289 shutdown_connections 2024-04-25T10:06:10.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.731+0000 7efe08129640 1 --2- 172.21.15.60:0/2619700289 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efde4066f60 0x7efde4069420 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.731+0000 7efe08129640 1 --2- 172.21.15.60:0/2619700289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe0015f480 0x7efe0014cd30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.731+0000 7efe08129640 1 --2- 172.21.15.60:0/2619700289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe000cd390 0x7efe0014c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.731+0000 7efe08129640 1 --2- 172.21.15.60:0/2619700289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe0009a930 0x7efe000b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.731+0000 7efe08129640 1 -- 172.21.15.60:0/2619700289 >> 172.21.15.60:0/2619700289 conn(0x7efe000b32a0 msgr2=0x7efe0015a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.731+0000 7efe08129640 1 -- 172.21.15.60:0/2619700289 shutdown_connections 2024-04-25T10:06:10.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.731+0000 7efe08129640 1 -- 172.21.15.60:0/2619700289 wait complete. 2024-04-25T10:06:10.752 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:06:10.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:10.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f672246c640 1 Processor -- start 2024-04-25T10:06:10.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f672246c640 1 -- start start 2024-04-25T10:06:10.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f672246c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f671c06b320 0x7f671c06ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f672246c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671c0b3880 con 0x7f671c145300 2024-04-25T10:06:10.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f672246c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671c0b4cd0 con 0x7f671c15bed0 2024-04-25T10:06:10.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f672246c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f671c06faf0 con 0x7f671c06b320 2024-04-25T10:06:10.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f671c145300 0x7f671c072d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42562/0 (socket says 172.21.15.60:42562) 2024-04-25T10:06:10.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671bfff640 1 -- 172.21.15.60:0/2764975005 learned_addr learned my addr 172.21.15.60:0/2764975005 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:10.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f6720a09640 1 --2- 172.21.15.60:0/2764975005 >> v2:172.21.15.100:3300/0 conn(0x7f671c06b320 0x7f671c06ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671affd640 1 -- 172.21.15.60:0/2764975005 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160174967 0 0) 0x7f671c0b3880 con 0x7f671c145300 2024-04-25T10:06:10.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671affd640 1 -- 172.21.15.60:0/2764975005 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6704003530 con 0x7f671c145300 2024-04-25T10:06:10.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671affd640 1 -- 172.21.15.60:0/2764975005 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3588099787 0 0) 0x7f6704003530 con 0x7f671c145300 2024-04-25T10:06:10.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671affd640 1 -- 172.21.15.60:0/2764975005 >> v2:172.21.15.100:3300/0 conn(0x7f671c06b320 msgr2=0x7f671c06ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671affd640 1 --2- 172.21.15.60:0/2764975005 >> v2:172.21.15.100:3300/0 conn(0x7f671c06b320 0x7f671c06ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671affd640 1 -- 172.21.15.60:0/2764975005 >> v1:172.21.15.100:6789/0 conn(0x7f671c15bed0 legacy=0x7f671c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671affd640 1 -- 172.21.15.60:0/2764975005 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c0b4f70 con 0x7f671c145300 2024-04-25T10:06:10.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671affd640 1 -- 172.21.15.60:0/2764975005 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f670c001680 con 0x7f671c145300 2024-04-25T10:06:10.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671affd640 1 -- 172.21.15.60:0/2764975005 >> v1:172.21.15.60:6789/0 conn(0x7f671c145300 legacy=0x7f671c072d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671affd640 1 --2- 172.21.15.60:0/2764975005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67040039c0 0x7f6704003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671affd640 1 --2- 172.21.15.60:0/2764975005 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6704004810 0x7f6704004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671affd640 1 --2- 172.21.15.60:0/2764975005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67040051d0 0x7f670400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671affd640 1 -- 172.21.15.60:0/2764975005 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c06faf0 con 0x7f67040051d0 2024-04-25T10:06:10.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671affd640 1 -- 172.21.15.60:0/2764975005 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670400c070 con 0x7f67040039c0 2024-04-25T10:06:10.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671affd640 1 -- 172.21.15.60:0/2764975005 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670400c270 con 0x7f6704004810 2024-04-25T10:06:10.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f6720a09640 1 --2- 172.21.15.60:0/2764975005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67040051d0 0x7f670400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671bfff640 1 --2- 172.21.15.60:0/2764975005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67040039c0 0x7f6704003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f671b7fe640 1 --2- 172.21.15.60:0/2764975005 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6704004810 0x7f6704004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f6720a09640 1 -- 172.21.15.60:0/2764975005 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6704004810 msgr2=0x7f6704004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f6720a09640 1 --2- 172.21.15.60:0/2764975005 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6704004810 0x7f6704004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f6720a09640 1 -- 172.21.15.60:0/2764975005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67040039c0 msgr2=0x7f6704003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f6720a09640 1 --2- 172.21.15.60:0/2764975005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67040039c0 0x7f6704003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.867+0000 7f6720a09640 1 -- 172.21.15.60:0/2764975005 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6704003530 con 0x7f67040051d0 2024-04-25T10:06:10.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.871+0000 7f6720a09640 1 -- 172.21.15.60:0/2764975005 reap_dead start 2024-04-25T10:06:10.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.871+0000 7f6720a09640 1 --2- 172.21.15.60:0/2764975005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67040051d0 0x7f670400b9d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6714009fa0 tx=0x7f671401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25056f64fb938b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.871+0000 7f671affd640 1 -- 172.21.15.60:0/2764975005 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67140b2070 con 0x7f67040051d0 2024-04-25T10:06:10.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.871+0000 7f671affd640 1 -- 172.21.15.60:0/2764975005 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67140a5d50 con 0x7f67040051d0 2024-04-25T10:06:10.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.871+0000 7f671affd640 1 -- 172.21.15.60:0/2764975005 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714009bf0 con 0x7f67040051d0 2024-04-25T10:06:10.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.871+0000 7f672246c640 1 -- 172.21.15.60:0/2764975005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67040051d0 msgr2=0x7f670400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.871+0000 7f672246c640 1 --2- 172.21.15.60:0/2764975005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67040051d0 0x7f670400b9d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6714009fa0 tx=0x7f671401cc80 comp rx=0 tx=0).stop 2024-04-25T10:06:10.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.871+0000 7f672246c640 1 -- 172.21.15.60:0/2764975005 shutdown_connections 2024-04-25T10:06:10.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.871+0000 7f672246c640 1 --2- 172.21.15.60:0/2764975005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67040051d0 0x7f670400b9d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.871+0000 7f672246c640 1 -- 172.21.15.60:0/2764975005 >> 172.21.15.60:0/2764975005 conn(0x7f671c0b2360 msgr2=0x7f671c0b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.871+0000 7f672246c640 1 -- 172.21.15.60:0/2764975005 shutdown_connections 2024-04-25T10:06:10.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.871+0000 7f672246c640 1 -- 172.21.15.60:0/2764975005 wait complete. 2024-04-25T10:06:10.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.871+0000 7f672246c640 1 Processor -- start 2024-04-25T10:06:10.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.871+0000 7f672246c640 1 -- start start 2024-04-25T10:06:10.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f672246c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67040051d0 0x7f671c13ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f672246c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f671c0ce980 0x7f671c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f672246c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f671c140c10 0x7f671c1410c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f672246c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c0b5530 con 0x7f67040051d0 2024-04-25T10:06:10.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f672246c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c15c330 con 0x7f671c0ce980 2024-04-25T10:06:10.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f672246c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671c0b5810 con 0x7f671c140c10 2024-04-25T10:06:10.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f671bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67040051d0 0x7f671c13ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f671bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67040051d0 0x7f671c13ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40902/0 (socket says 172.21.15.60:40902) 2024-04-25T10:06:10.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f671bfff640 1 -- 172.21.15.60:0/3576442634 learned_addr learned my addr 172.21.15.60:0/3576442634 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:10.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f671bfff640 1 -- 172.21.15.60:0/3576442634 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f671c140c10 msgr2=0x7f671c1410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f671bfff640 1 --2- 172.21.15.60:0/3576442634 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f671c140c10 0x7f671c1410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f671bfff640 1 -- 172.21.15.60:0/3576442634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f671c0ce980 msgr2=0x7f671c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f671bfff640 1 --2- 172.21.15.60:0/3576442634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f671c0ce980 0x7f671c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f671bfff640 1 -- 172.21.15.60:0/3576442634 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c0ceef0 con 0x7f67040051d0 2024-04-25T10:06:10.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f671bfff640 1 --2- 172.21.15.60:0/3576442634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67040051d0 0x7f671c13ffe0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f670c0011f0 tx=0x7f670c007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f6718ff9640 1 -- 172.21.15.60:0/3576442634 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c007e20 con 0x7f67040051d0 2024-04-25T10:06:10.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f6718ff9640 1 -- 172.21.15.60:0/3576442634 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670c018070 con 0x7f67040051d0 2024-04-25T10:06:10.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f672246c640 1 -- 172.21.15.60:0/3576442634 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671c0cf1d0 con 0x7f67040051d0 2024-04-25T10:06:10.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f6718ff9640 1 -- 172.21.15.60:0/3576442634 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c00fb80 con 0x7f67040051d0 2024-04-25T10:06:10.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f672246c640 1 -- 172.21.15.60:0/3576442634 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671c0cf790 con 0x7f67040051d0 2024-04-25T10:06:10.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f672246c640 1 -- 172.21.15.60:0/3576442634 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f671c06ccb0 con 0x7f67040051d0 2024-04-25T10:06:10.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f6718ff9640 1 -- 172.21.15.60:0/3576442634 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f670c00fd20 con 0x7f67040051d0 2024-04-25T10:06:10.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f6718ff9640 1 --2- 172.21.15.60:0/3576442634 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6708066f90 0x7f6708069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f6718ff9640 1 -- 172.21.15.60:0/3576442634 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f670c084ff0 con 0x7f67040051d0 2024-04-25T10:06:10.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.875+0000 7f671b7fe640 1 --2- 172.21.15.60:0/3576442634 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6708066f90 0x7f6708069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.879+0000 7f671b7fe640 1 --2- 172.21.15.60:0/3576442634 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6708066f90 0x7f6708069450 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6710001060 tx=0x7f6710009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:10.879+0000 7f6718ff9640 1 -- 172.21.15.60:0/3576442634 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f670c055f30 con 0x7f67040051d0 2024-04-25T10:06:11.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.075+0000 7f672246c640 1 -- 172.21.15.60:0/3576442634 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f671c0cd100 con 0x7f67040051d0 2024-04-25T10:06:11.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.079+0000 7f6718ff9640 1 -- 172.21.15.60:0/3576442634 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f670c055830 con 0x7f67040051d0 2024-04-25T10:06:11.085 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:11.086 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/622088871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":622088871},{"type":"v1","addr":"172.21.15.100:6836","nonce":622088871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3988204202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3988204202},{"type":"v1","addr":"172.21.15.60:6833","nonce":3988204202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3591836867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3591836867},{"type":"v1","addr":"172.21.15.60:6835","nonce":3591836867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:20.405486+0000","modified":"2024-04-25T10:05:23.453258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/690070228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":690070228},{"type":"v1","addr":"172.21.15.100:6837","nonce":690070228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:06:11.086 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-04-25T10:06:11.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.079+0000 7f672246c640 1 -- 172.21.15.60:0/3576442634 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6708066f90 msgr2=0x7f6708069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.079+0000 7f672246c640 1 --2- 172.21.15.60:0/3576442634 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6708066f90 0x7f6708069450 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6710001060 tx=0x7f6710009040 comp rx=0 tx=0).stop 2024-04-25T10:06:11.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.079+0000 7f672246c640 1 -- 172.21.15.60:0/3576442634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67040051d0 msgr2=0x7f671c13ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.079+0000 7f672246c640 1 --2- 172.21.15.60:0/3576442634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67040051d0 0x7f671c13ffe0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f670c0011f0 tx=0x7f670c007720 comp rx=0 tx=0).stop 2024-04-25T10:06:11.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.079+0000 7f672246c640 1 -- 172.21.15.60:0/3576442634 shutdown_connections 2024-04-25T10:06:11.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.079+0000 7f672246c640 1 --2- 172.21.15.60:0/3576442634 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6708066f90 0x7f6708069450 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.079+0000 7f672246c640 1 --2- 172.21.15.60:0/3576442634 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f671c140c10 0x7f671c1410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.079+0000 7f672246c640 1 --2- 172.21.15.60:0/3576442634 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f671c0ce980 0x7f671c0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.079+0000 7f672246c640 1 --2- 172.21.15.60:0/3576442634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67040051d0 0x7f671c13ffe0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.079+0000 7f672246c640 1 -- 172.21.15.60:0/3576442634 >> 172.21.15.60:0/3576442634 conn(0x7f671c0b2360 msgr2=0x7f671c144020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.079+0000 7f672246c640 1 -- 172.21.15.60:0/3576442634 shutdown_connections 2024-04-25T10:06:11.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.079+0000 7f672246c640 1 -- 172.21.15.60:0/3576442634 wait complete. 2024-04-25T10:06:11.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:11.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a34fa640 1 Processor -- start 2024-04-25T10:06:11.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a34fa640 1 -- start start 2024-04-25T10:06:11.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a34fa640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f979c071720 0x7f979c0cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a34fa640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f979c0b3aa0 con 0x7f979c162250 2024-04-25T10:06:11.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a34fa640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f979c0b3d80 con 0x7f979c071290 2024-04-25T10:06:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a34fa640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979c0af0c0 con 0x7f979c071720 2024-04-25T10:06:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a1296640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f979c162250 0x7f979c0b2fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42574/0 (socket says 172.21.15.60:42574) 2024-04-25T10:06:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a1296640 1 -- 172.21.15.60:0/2650498807 learned_addr learned my addr 172.21.15.60:0/2650498807 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a1a97640 1 --2- 172.21.15.60:0/2650498807 >> v2:172.21.15.100:3300/0 conn(0x7f979c071720 0x7f979c0cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f9793fff640 1 -- 172.21.15.60:0/2650498807 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229116748 0 0) 0x7f979c0b3aa0 con 0x7f979c162250 2024-04-25T10:06:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f9793fff640 1 -- 172.21.15.60:0/2650498807 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9788003530 con 0x7f979c162250 2024-04-25T10:06:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f9793fff640 1 -- 172.21.15.60:0/2650498807 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2980853642 0 0) 0x7f9788003530 con 0x7f979c162250 2024-04-25T10:06:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f9793fff640 1 -- 172.21.15.60:0/2650498807 >> v2:172.21.15.100:3300/0 conn(0x7f979c071720 msgr2=0x7f979c0cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f9793fff640 1 --2- 172.21.15.60:0/2650498807 >> v2:172.21.15.100:3300/0 conn(0x7f979c071720 0x7f979c0cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f9793fff640 1 -- 172.21.15.60:0/2650498807 >> v1:172.21.15.100:6789/0 conn(0x7f979c071290 legacy=0x7f979c0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f9793fff640 1 -- 172.21.15.60:0/2650498807 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c0b4060 con 0x7f979c162250 2024-04-25T10:06:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f9793fff640 1 -- 172.21.15.60:0/2650498807 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f9798001a00 con 0x7f979c162250 2024-04-25T10:06:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f9793fff640 1 -- 172.21.15.60:0/2650498807 >> v1:172.21.15.60:6789/0 conn(0x7f979c162250 legacy=0x7f979c0b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f9793fff640 1 --2- 172.21.15.60:0/2650498807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97880039c0 0x7f9788003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f9793fff640 1 --2- 172.21.15.60:0/2650498807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9788004810 0x7f9788004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f9793fff640 1 --2- 172.21.15.60:0/2650498807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97880051d0 0x7f978800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f9793fff640 1 -- 172.21.15.60:0/2650498807 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c0af0c0 con 0x7f97880051d0 2024-04-25T10:06:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f9793fff640 1 -- 172.21.15.60:0/2650498807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978800c080 con 0x7f97880039c0 2024-04-25T10:06:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f9793fff640 1 -- 172.21.15.60:0/2650498807 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978800c280 con 0x7f9788004810 2024-04-25T10:06:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a1a97640 1 --2- 172.21.15.60:0/2650498807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97880051d0 0x7f978800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a1296640 1 --2- 172.21.15.60:0/2650498807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97880039c0 0x7f9788003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a0a95640 1 --2- 172.21.15.60:0/2650498807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9788004810 0x7f9788004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a1a97640 1 -- 172.21.15.60:0/2650498807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9788004810 msgr2=0x7f9788004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a1a97640 1 --2- 172.21.15.60:0/2650498807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9788004810 0x7f9788004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a1a97640 1 -- 172.21.15.60:0/2650498807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97880039c0 msgr2=0x7f9788003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a1a97640 1 --2- 172.21.15.60:0/2650498807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97880039c0 0x7f9788003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a1a97640 1 -- 172.21.15.60:0/2650498807 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9788003530 con 0x7f97880051d0 2024-04-25T10:06:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.219+0000 7f97a1296640 1 --2- 172.21.15.60:0/2650498807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97880039c0 0x7f9788003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.223+0000 7f97a1a97640 1 -- 172.21.15.60:0/2650498807 reap_dead start 2024-04-25T10:06:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.223+0000 7f97a1a97640 1 --2- 172.21.15.60:0/2650498807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97880051d0 0x7f978800b9d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9794009ac0 tx=0x7f979401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3830575e7aaf59c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.223+0000 7f9793fff640 1 -- 172.21.15.60:0/2650498807 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794009690 con 0x7f97880051d0 2024-04-25T10:06:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.223+0000 7f9793fff640 1 -- 172.21.15.60:0/2650498807 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97940a5680 con 0x7f97880051d0 2024-04-25T10:06:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.223+0000 7f9793fff640 1 -- 172.21.15.60:0/2650498807 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97940a5a50 con 0x7f97880051d0 2024-04-25T10:06:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.223+0000 7f97a34fa640 1 -- 172.21.15.60:0/2650498807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97880051d0 msgr2=0x7f978800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.223+0000 7f97a34fa640 1 --2- 172.21.15.60:0/2650498807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97880051d0 0x7f978800b9d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9794009ac0 tx=0x7f979401ca90 comp rx=0 tx=0).stop 2024-04-25T10:06:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.223+0000 7f97a34fa640 1 -- 172.21.15.60:0/2650498807 shutdown_connections 2024-04-25T10:06:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.223+0000 7f97a34fa640 1 --2- 172.21.15.60:0/2650498807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97880051d0 0x7f978800b9d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.223+0000 7f97a34fa640 1 -- 172.21.15.60:0/2650498807 >> 172.21.15.60:0/2650498807 conn(0x7f979c0b12d0 msgr2=0x7f979c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.223+0000 7f97a34fa640 1 -- 172.21.15.60:0/2650498807 shutdown_connections 2024-04-25T10:06:11.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.223+0000 7f97a34fa640 1 -- 172.21.15.60:0/2650498807 wait complete. 2024-04-25T10:06:11.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a34fa640 1 Processor -- start 2024-04-25T10:06:11.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a34fa640 1 -- start start 2024-04-25T10:06:11.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a34fa640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97880051d0 0x7f979c14eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a34fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c14f400 0x7f979c0acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a34fa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979c0ad120 0x7f979c0a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a34fa640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c0cc440 con 0x7f979c14f400 2024-04-25T10:06:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a34fa640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c06e5a0 con 0x7f979c0ad120 2024-04-25T10:06:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a34fa640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979c071b50 con 0x7f97880051d0 2024-04-25T10:06:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a0a95640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c14f400 0x7f979c0acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a0a95640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c14f400 0x7f979c0acbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40928/0 (socket says 172.21.15.60:40928) 2024-04-25T10:06:11.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a0a95640 1 -- 172.21.15.60:0/2090623285 learned_addr learned my addr 172.21.15.60:0/2090623285 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:11.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a1a97640 1 --2- 172.21.15.60:0/2090623285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979c0ad120 0x7f979c0a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a1296640 1 --2- 172.21.15.60:0/2090623285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97880051d0 0x7f979c14eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a0a95640 1 -- 172.21.15.60:0/2090623285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97880051d0 msgr2=0x7f979c14eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a0a95640 1 --2- 172.21.15.60:0/2090623285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97880051d0 0x7f979c14eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a0a95640 1 -- 172.21.15.60:0/2090623285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979c0ad120 msgr2=0x7f979c0a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a0a95640 1 --2- 172.21.15.60:0/2090623285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979c0ad120 0x7f979c0a8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a0a95640 1 -- 172.21.15.60:0/2090623285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c0a9410 con 0x7f979c14f400 2024-04-25T10:06:11.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a1296640 1 --2- 172.21.15.60:0/2090623285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97880051d0 0x7f979c14eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:11.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a0a95640 1 --2- 172.21.15.60:0/2090623285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c14f400 0x7f979c0acbe0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f978c00e4a0 tx=0x7f978c00e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f9791ffb640 1 -- 172.21.15.60:0/2090623285 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c0052e0 con 0x7f979c14f400 2024-04-25T10:06:11.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f9791ffb640 1 -- 172.21.15.60:0/2090623285 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c018070 con 0x7f979c14f400 2024-04-25T10:06:11.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a34fa640 1 -- 172.21.15.60:0/2090623285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979c0a96f0 con 0x7f979c14f400 2024-04-25T10:06:11.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f9791ffb640 1 -- 172.21.15.60:0/2090623285 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c003820 con 0x7f979c14f400 2024-04-25T10:06:11.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a34fa640 1 -- 172.21.15.60:0/2090623285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979c0a9cb0 con 0x7f979c14f400 2024-04-25T10:06:11.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.227+0000 7f97a34fa640 1 -- 172.21.15.60:0/2090623285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979c074770 con 0x7f979c14f400 2024-04-25T10:06:11.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.231+0000 7f9791ffb640 1 -- 172.21.15.60:0/2090623285 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f978c0039c0 con 0x7f979c14f400 2024-04-25T10:06:11.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.231+0000 7f9791ffb640 1 --2- 172.21.15.60:0/2090623285 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9780066f60 0x7f9780069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.231+0000 7f9791ffb640 1 -- 172.21.15.60:0/2090623285 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f978c084cb0 con 0x7f979c14f400 2024-04-25T10:06:11.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.231+0000 7f97a1296640 1 --2- 172.21.15.60:0/2090623285 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9780066f60 0x7f9780069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.231+0000 7f97a1296640 1 --2- 172.21.15.60:0/2090623285 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9780066f60 0x7f9780069420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f97980012f0 tx=0x7f97980014e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.235+0000 7f9791ffb640 1 -- 172.21.15.60:0/2090623285 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f978c055c10 con 0x7f979c14f400 2024-04-25T10:06:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.423+0000 7f97a34fa640 1 -- 172.21.15.60:0/2090623285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f979c14fa60 con 0x7f979c14f400 2024-04-25T10:06:11.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.427+0000 7f9791ffb640 1 -- 172.21.15.60:0/2090623285 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f978c005480 con 0x7f979c14f400 2024-04-25T10:06:11.433 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:11.434 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/622088871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":622088871},{"type":"v1","addr":"172.21.15.100:6836","nonce":622088871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3988204202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3988204202},{"type":"v1","addr":"172.21.15.60:6833","nonce":3988204202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3591836867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3591836867},{"type":"v1","addr":"172.21.15.60:6835","nonce":3591836867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:20.405486+0000","modified":"2024-04-25T10:05:23.453258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/690070228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":690070228},{"type":"v1","addr":"172.21.15.100:6837","nonce":690070228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:06:11.434 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-04-25T10:06:11.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.427+0000 7f97a34fa640 1 -- 172.21.15.60:0/2090623285 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9780066f60 msgr2=0x7f9780069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.427+0000 7f97a34fa640 1 --2- 172.21.15.60:0/2090623285 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9780066f60 0x7f9780069420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f97980012f0 tx=0x7f97980014e0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.427+0000 7f97a34fa640 1 -- 172.21.15.60:0/2090623285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c14f400 msgr2=0x7f979c0acbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.427+0000 7f97a34fa640 1 --2- 172.21.15.60:0/2090623285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c14f400 0x7f979c0acbe0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f978c00e4a0 tx=0x7f978c00e960 comp rx=0 tx=0).stop 2024-04-25T10:06:11.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.427+0000 7f97a34fa640 1 -- 172.21.15.60:0/2090623285 shutdown_connections 2024-04-25T10:06:11.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.427+0000 7f97a34fa640 1 --2- 172.21.15.60:0/2090623285 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9780066f60 0x7f9780069420 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.427+0000 7f97a34fa640 1 --2- 172.21.15.60:0/2090623285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979c0ad120 0x7f979c0a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.427+0000 7f97a34fa640 1 --2- 172.21.15.60:0/2090623285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c14f400 0x7f979c0acbe0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.427+0000 7f97a34fa640 1 --2- 172.21.15.60:0/2090623285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97880051d0 0x7f979c14eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.427+0000 7f97a34fa640 1 -- 172.21.15.60:0/2090623285 >> 172.21.15.60:0/2090623285 conn(0x7f979c0b12d0 msgr2=0x7f979c164640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.427+0000 7f97a34fa640 1 -- 172.21.15.60:0/2090623285 shutdown_connections 2024-04-25T10:06:11.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.427+0000 7f97a34fa640 1 -- 172.21.15.60:0/2090623285 wait complete. 2024-04-25T10:06:11.451 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f1212b3d640 1 Processor -- start 2024-04-25T10:06:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f1212b3d640 1 -- start start 2024-04-25T10:06:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f1212b3d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f120c06ec30 0x7f120c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f1212b3d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f120c0b37e0 0x7f120c0b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f1212b3d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120c06f130 con 0x7f120c15a7a0 2024-04-25T10:06:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f1212b3d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f120c0b4190 con 0x7f120c06ec30 2024-04-25T10:06:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f1212b3d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f120c0cdaa0 con 0x7f120c0b37e0 2024-04-25T10:06:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f120bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f120c06ec30 0x7f120c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f12108d9640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f120c15a7a0 0x7f120c071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42584/0 (socket says 172.21.15.60:42584) 2024-04-25T10:06:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f12108d9640 1 -- 172.21.15.60:0/1622187549 learned_addr learned my addr 172.21.15.60:0/1622187549 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f12110da640 1 --2- 172.21.15.60:0/1622187549 >> v2:172.21.15.100:3300/0 conn(0x7f120c0b37e0 0x7f120c0b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f120bfff640 1 -- 172.21.15.60:0/1622187549 >> v2:172.21.15.100:3300/0 conn(0x7f120c0b37e0 msgr2=0x7f120c0b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f120bfff640 1 --2- 172.21.15.60:0/1622187549 >> v2:172.21.15.100:3300/0 conn(0x7f120c0b37e0 0x7f120c0b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f120bfff640 1 -- 172.21.15.60:0/1622187549 >> v1:172.21.15.60:6789/0 conn(0x7f120c15a7a0 legacy=0x7f120c071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f120b7fe640 1 -- 172.21.15.60:0/1622187549 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182614154 0 0) 0x7f120c06f130 con 0x7f120c15a7a0 2024-04-25T10:06:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f120bfff640 1 -- 172.21.15.60:0/1622187549 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c06f410 con 0x7f120c06ec30 2024-04-25T10:06:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.563+0000 7f12110da640 1 --2- 172.21.15.60:0/1622187549 >> v2:172.21.15.100:3300/0 conn(0x7f120c0b37e0 0x7f120c0b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f120bfff640 1 --2- 172.21.15.60:0/1622187549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c06ec30 0x7f120c0cd560 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f120000ba60 tx=0x7f120001eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6338588383968e88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f120b7fe640 1 -- 172.21.15.60:0/1622187549 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12000b4070 con 0x7f120c06ec30 2024-04-25T10:06:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f120b7fe640 1 -- 172.21.15.60:0/1622187549 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12000a7b10 con 0x7f120c06ec30 2024-04-25T10:06:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f120b7fe640 1 -- 172.21.15.60:0/1622187549 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12000af440 con 0x7f120c06ec30 2024-04-25T10:06:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f1212b3d640 1 -- 172.21.15.60:0/1622187549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c06ec30 msgr2=0x7f120c0cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f1212b3d640 1 --2- 172.21.15.60:0/1622187549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c06ec30 0x7f120c0cd560 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f120000ba60 tx=0x7f120001eb30 comp rx=0 tx=0).stop 2024-04-25T10:06:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f1212b3d640 1 -- 172.21.15.60:0/1622187549 shutdown_connections 2024-04-25T10:06:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f1212b3d640 1 --2- 172.21.15.60:0/1622187549 >> v2:172.21.15.100:3300/0 conn(0x7f120c0b37e0 0x7f120c0b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f1212b3d640 1 --2- 172.21.15.60:0/1622187549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c06ec30 0x7f120c0cd560 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f1212b3d640 1 -- 172.21.15.60:0/1622187549 >> 172.21.15.60:0/1622187549 conn(0x7f120c0b12d0 msgr2=0x7f120c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f1212b3d640 1 -- 172.21.15.60:0/1622187549 shutdown_connections 2024-04-25T10:06:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f1212b3d640 1 -- 172.21.15.60:0/1622187549 wait complete. 2024-04-25T10:06:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f1212b3d640 1 Processor -- start 2024-04-25T10:06:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f1212b3d640 1 -- start start 2024-04-25T10:06:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f1212b3d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f120c06ec30 0x7f120c1f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f1212b3d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0b37e0 0x7f120c156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f1212b3d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f120c15a7a0 0x7f120c1638a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f1212b3d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c06e8e0 con 0x7f120c0b37e0 2024-04-25T10:06:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f1212b3d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c06e760 con 0x7f120c15a7a0 2024-04-25T10:06:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f1212b3d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120c06d8b0 con 0x7f120c06ec30 2024-04-25T10:06:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f120bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0b37e0 0x7f120c156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f120bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0b37e0 0x7f120c156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40956/0 (socket says 172.21.15.60:40956) 2024-04-25T10:06:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f120bfff640 1 -- 172.21.15.60:0/3865571641 learned_addr learned my addr 172.21.15.60:0/3865571641 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f120bfff640 1 -- 172.21.15.60:0/3865571641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f120c06ec30 msgr2=0x7f120c1f78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f12110da640 1 --2- 172.21.15.60:0/3865571641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f120c15a7a0 0x7f120c1638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f120bfff640 1 --2- 172.21.15.60:0/3865571641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f120c06ec30 0x7f120c1f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f120bfff640 1 -- 172.21.15.60:0/3865571641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f120c15a7a0 msgr2=0x7f120c1638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f120bfff640 1 --2- 172.21.15.60:0/3865571641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f120c15a7a0 0x7f120c1638a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f120bfff640 1 -- 172.21.15.60:0/3865571641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c156a30 con 0x7f120c0b37e0 2024-04-25T10:06:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.567+0000 7f120bfff640 1 --2- 172.21.15.60:0/3865571641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0b37e0 0x7f120c156060 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f120001b740 tx=0x7f120001b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.571+0000 7f12097fa640 1 -- 172.21.15.60:0/3865571641 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12000be070 con 0x7f120c0b37e0 2024-04-25T10:06:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.571+0000 7f12097fa640 1 -- 172.21.15.60:0/3865571641 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12000b8420 con 0x7f120c0b37e0 2024-04-25T10:06:11.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.571+0000 7f1212b3d640 1 -- 172.21.15.60:0/3865571641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120c158420 con 0x7f120c0b37e0 2024-04-25T10:06:11.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.571+0000 7f12097fa640 1 -- 172.21.15.60:0/3865571641 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12000b4040 con 0x7f120c0b37e0 2024-04-25T10:06:11.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.571+0000 7f1212b3d640 1 -- 172.21.15.60:0/3865571641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120c1589e0 con 0x7f120c0b37e0 2024-04-25T10:06:11.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.571+0000 7f1212b3d640 1 -- 172.21.15.60:0/3865571641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120c074770 con 0x7f120c0b37e0 2024-04-25T10:06:11.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.571+0000 7f12097fa640 1 -- 172.21.15.60:0/3865571641 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12000c1070 con 0x7f120c0b37e0 2024-04-25T10:06:11.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.571+0000 7f12097fa640 1 --2- 172.21.15.60:0/3865571641 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f11f8066f60 0x7f11f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.571+0000 7f12097fa640 1 -- 172.21.15.60:0/3865571641 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f120011f1d0 con 0x7f120c0b37e0 2024-04-25T10:06:11.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.571+0000 7f12108d9640 1 --2- 172.21.15.60:0/3865571641 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f11f8066f60 0x7f11f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.571+0000 7f12108d9640 1 --2- 172.21.15.60:0/3865571641 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f11f8066f60 0x7f11f8069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f120c153850 tx=0x7f11fc007970 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.575+0000 7f12097fa640 1 -- 172.21.15.60:0/3865571641 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1200018720 con 0x7f120c0b37e0 2024-04-25T10:06:11.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.735+0000 7f1212b3d640 1 -- 172.21.15.60:0/3865571641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f120c15b630 con 0x7f120c0b37e0 2024-04-25T10:06:11.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.735+0000 7f12097fa640 1 -- 172.21.15.60:0/3865571641 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16857 (secure 0 0 0) 0x7f12000f0150 con 0x7f120c0b37e0 2024-04-25T10:06:11.744 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:11.744 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:01.568957+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:18.976643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:19.405302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.739+0000 7f1212b3d640 1 -- 172.21.15.60:0/3865571641 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f11f8066f60 msgr2=0x7f11f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.739+0000 7f1212b3d640 1 --2- 172.21.15.60:0/3865571641 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f11f8066f60 0x7f11f8069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f120c153850 tx=0x7f11fc007970 comp rx=0 tx=0).stop 2024-04-25T10:06:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.739+0000 7f1212b3d640 1 -- 172.21.15.60:0/3865571641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0b37e0 msgr2=0x7f120c156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.739+0000 7f1212b3d640 1 --2- 172.21.15.60:0/3865571641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0b37e0 0x7f120c156060 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f120001b740 tx=0x7f120001b8d0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.739+0000 7f1212b3d640 1 -- 172.21.15.60:0/3865571641 shutdown_connections 2024-04-25T10:06:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.739+0000 7f1212b3d640 1 --2- 172.21.15.60:0/3865571641 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f11f8066f60 0x7f11f8069420 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.739+0000 7f1212b3d640 1 --2- 172.21.15.60:0/3865571641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f120c15a7a0 0x7f120c1638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.739+0000 7f1212b3d640 1 --2- 172.21.15.60:0/3865571641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0b37e0 0x7f120c156060 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.739+0000 7f1212b3d640 1 --2- 172.21.15.60:0/3865571641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f120c06ec30 0x7f120c1f78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.739+0000 7f1212b3d640 1 -- 172.21.15.60:0/3865571641 >> 172.21.15.60:0/3865571641 conn(0x7f120c0b12d0 msgr2=0x7f120c15adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.739+0000 7f1212b3d640 1 -- 172.21.15.60:0/3865571641 shutdown_connections 2024-04-25T10:06:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.739+0000 7f1212b3d640 1 -- 172.21.15.60:0/3865571641 wait complete. 2024-04-25T10:06:11.763 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:06:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.875+0000 7fa628acc640 1 Processor -- start 2024-04-25T10:06:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.875+0000 7fa628acc640 1 -- start start 2024-04-25T10:06:11.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.875+0000 7fa628acc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa624071b20 0x7fa624071f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa628acc640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa624072440 con 0x7fa62406e480 2024-04-25T10:06:11.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa628acc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62415dc10 con 0x7fa6240cce50 2024-04-25T10:06:11.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa628acc640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa62415def0 con 0x7fa624071b20 2024-04-25T10:06:11.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa62259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa624071b20 0x7fa624071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa62259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa624071b20 0x7fa624071f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40968/0 (socket says 172.21.15.60:40968) 2024-04-25T10:06:11.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa62259c640 1 -- 172.21.15.60:0/3195822181 learned_addr learned my addr 172.21.15.60:0/3195822181 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:11.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa619d9b640 1 --1- 172.21.15.60:0/3195822181 >> v1:172.21.15.60:6789/0 conn(0x7fa62406e480 0x7fa62406e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42600/0 (socket says 172.21.15.60:42600) 2024-04-25T10:06:11.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa62259c640 1 -- 172.21.15.60:0/3195822181 >> v1:172.21.15.100:6789/0 conn(0x7fa6240cce50 legacy=0x7fa6240cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa62259c640 1 -- 172.21.15.60:0/3195822181 >> v1:172.21.15.60:6789/0 conn(0x7fa62406e480 legacy=0x7fa62406e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa62259c640 1 -- 172.21.15.60:0/3195822181 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62415e1d0 con 0x7fa624071b20 2024-04-25T10:06:11.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa62259c640 1 --2- 172.21.15.60:0/3195822181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624071b20 0x7fa624071f00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa60c006fd0 tx=0x7fa60c01e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50025d3156609d7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa621d9b640 1 -- 172.21.15.60:0/3195822181 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0ad070 con 0x7fa624071b20 2024-04-25T10:06:11.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa621d9b640 1 -- 172.21.15.60:0/3195822181 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60c004230 con 0x7fa624071b20 2024-04-25T10:06:11.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa621d9b640 1 -- 172.21.15.60:0/3195822181 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c003620 con 0x7fa624071b20 2024-04-25T10:06:11.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa628acc640 1 -- 172.21.15.60:0/3195822181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624071b20 msgr2=0x7fa624071f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa628acc640 1 --2- 172.21.15.60:0/3195822181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624071b20 0x7fa624071f00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa60c006fd0 tx=0x7fa60c01e920 comp rx=0 tx=0).stop 2024-04-25T10:06:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa628acc640 1 -- 172.21.15.60:0/3195822181 shutdown_connections 2024-04-25T10:06:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa628acc640 1 --2- 172.21.15.60:0/3195822181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa624071b20 0x7fa624071f00 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa628acc640 1 -- 172.21.15.60:0/3195822181 >> 172.21.15.60:0/3195822181 conn(0x7fa6240b12d0 msgr2=0x7fa6240b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa628acc640 1 -- 172.21.15.60:0/3195822181 shutdown_connections 2024-04-25T10:06:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.879+0000 7fa628acc640 1 -- 172.21.15.60:0/3195822181 wait complete. 2024-04-25T10:06:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa628acc640 1 Processor -- start 2024-04-25T10:06:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa628acc640 1 -- start start 2024-04-25T10:06:11.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa628acc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa62406e480 0x7fa6241392f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa628acc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa624071b20 0x7fa62413c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa628acc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6240cce50 0x7fa6241422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa628acc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6240cdaf0 con 0x7fa6240cce50 2024-04-25T10:06:11.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa628acc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6240cd970 con 0x7fa62406e480 2024-04-25T10:06:11.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa628acc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6240cdc70 con 0x7fa624071b20 2024-04-25T10:06:11.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa622d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6240cce50 0x7fa6241422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa622d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6240cce50 0x7fa6241422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40984/0 (socket says 172.21.15.60:40984) 2024-04-25T10:06:11.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa622d9d640 1 -- 172.21.15.60:0/3046117436 learned_addr learned my addr 172.21.15.60:0/3046117436 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:11.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa62259c640 1 --2- 172.21.15.60:0/3046117436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa62406e480 0x7fa6241392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa619d9b640 1 --2- 172.21.15.60:0/3046117436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa624071b20 0x7fa62413c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa622d9d640 1 -- 172.21.15.60:0/3046117436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa624071b20 msgr2=0x7fa62413c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa622d9d640 1 --2- 172.21.15.60:0/3046117436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa624071b20 0x7fa62413c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa622d9d640 1 -- 172.21.15.60:0/3046117436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa62406e480 msgr2=0x7fa6241392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa622d9d640 1 --2- 172.21.15.60:0/3046117436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa62406e480 0x7fa6241392f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa622d9d640 1 -- 172.21.15.60:0/3046117436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62413cbd0 con 0x7fa6240cce50 2024-04-25T10:06:11.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa62259c640 1 --2- 172.21.15.60:0/3046117436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa62406e480 0x7fa6241392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:11.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa622d9d640 1 --2- 172.21.15.60:0/3046117436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6240cce50 0x7fa6241422d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00b970 tx=0x7fa61c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa61b7fe640 1 -- 172.21.15.60:0/3046117436 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c019070 con 0x7fa6240cce50 2024-04-25T10:06:11.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa61b7fe640 1 -- 172.21.15.60:0/3046117436 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61c00cb40 con 0x7fa6240cce50 2024-04-25T10:06:11.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa61b7fe640 1 -- 172.21.15.60:0/3046117436 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c014440 con 0x7fa6240cce50 2024-04-25T10:06:11.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa628acc640 1 -- 172.21.15.60:0/3046117436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62413bd00 con 0x7fa6240cce50 2024-04-25T10:06:11.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa628acc640 1 -- 172.21.15.60:0/3046117436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62413b740 con 0x7fa6240cce50 2024-04-25T10:06:11.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.883+0000 7fa628acc640 1 -- 172.21.15.60:0/3046117436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa624074770 con 0x7fa6240cce50 2024-04-25T10:06:11.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.887+0000 7fa61b7fe640 1 -- 172.21.15.60:0/3046117436 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa61c01c070 con 0x7fa6240cce50 2024-04-25T10:06:11.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.887+0000 7fa61b7fe640 1 --2- 172.21.15.60:0/3046117436 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa604066f60 0x7fa604069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.887+0000 7fa61b7fe640 1 -- 172.21.15.60:0/3046117436 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa61c084d30 con 0x7fa6240cce50 2024-04-25T10:06:11.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.887+0000 7fa62259c640 1 --2- 172.21.15.60:0/3046117436 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa604066f60 0x7fa604069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.887+0000 7fa62259c640 1 --2- 172.21.15.60:0/3046117436 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa604066f60 0x7fa604069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0b1000 tx=0x7fa60c0a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:11.891+0000 7fa61b7fe640 1 -- 172.21.15.60:0/3046117436 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa61c055c20 con 0x7fa6240cce50 2024-04-25T10:06:12.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.079+0000 7fa628acc640 1 -- 172.21.15.60:0/3046117436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa624143030 con 0x7fa6240cce50 2024-04-25T10:06:12.088 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:06:12.083+0000 7f3a792d6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:06:12.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.083+0000 7fa61b7fe640 1 -- 172.21.15.60:0/3046117436 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa61c055520 con 0x7fa6240cce50 2024-04-25T10:06:12.091 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:06:12.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.087+0000 7fa628acc640 1 -- 172.21.15.60:0/3046117436 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa604066f60 msgr2=0x7fa604069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.087+0000 7fa628acc640 1 --2- 172.21.15.60:0/3046117436 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa604066f60 0x7fa604069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0b1000 tx=0x7fa60c0a9070 comp rx=0 tx=0).stop 2024-04-25T10:06:12.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.087+0000 7fa628acc640 1 -- 172.21.15.60:0/3046117436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6240cce50 msgr2=0x7fa6241422d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.087+0000 7fa628acc640 1 --2- 172.21.15.60:0/3046117436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6240cce50 0x7fa6241422d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00b970 tx=0x7fa61c00be30 comp rx=0 tx=0).stop 2024-04-25T10:06:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.087+0000 7fa628acc640 1 -- 172.21.15.60:0/3046117436 shutdown_connections 2024-04-25T10:06:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.087+0000 7fa628acc640 1 --2- 172.21.15.60:0/3046117436 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa604066f60 0x7fa604069420 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.087+0000 7fa628acc640 1 --2- 172.21.15.60:0/3046117436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6240cce50 0x7fa6241422d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.087+0000 7fa628acc640 1 --2- 172.21.15.60:0/3046117436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa624071b20 0x7fa62413c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.087+0000 7fa628acc640 1 --2- 172.21.15.60:0/3046117436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa62406e480 0x7fa6241392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.087+0000 7fa628acc640 1 -- 172.21.15.60:0/3046117436 >> 172.21.15.60:0/3046117436 conn(0x7fa6240b12d0 msgr2=0x7fa6240cb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.087+0000 7fa628acc640 1 -- 172.21.15.60:0/3046117436 shutdown_connections 2024-04-25T10:06:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.087+0000 7fa628acc640 1 -- 172.21.15.60:0/3046117436 wait complete. 2024-04-25T10:06:12.103 INFO:tasks.ceph.mds.d.smithi100.stderr: -3205> 2024-04-25T10:05:25.445+0000 7f3a152e0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:06:12.110 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:06:12.156 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T10:06:12.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.227+0000 7ff1a3491640 1 Processor -- start 2024-04-25T10:06:12.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.227+0000 7ff1a3491640 1 -- start start 2024-04-25T10:06:12.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a3491640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff19c0cbcf0 0x7ff19c160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a3491640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff19c15fbf0 0x7ff19c0ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a3491640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c144a40 con 0x7ff19c071c30 2024-04-25T10:06:12.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a3491640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19c0b2dc0 con 0x7ff19c15fbf0 2024-04-25T10:06:12.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a3491640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0cc9f0 con 0x7ff19c0cbcf0 2024-04-25T10:06:12.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a1a2e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff19c15fbf0 0x7ff19c0ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a1a2e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff19c15fbf0 0x7ff19c0ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41000/0 (socket says 172.21.15.60:41000) 2024-04-25T10:06:12.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a1a2e640 1 -- 172.21.15.60:0/294089368 learned_addr learned my addr 172.21.15.60:0/294089368 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a122d640 1 --2- 172.21.15.60:0/294089368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff19c0cbcf0 0x7ff19c160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a1a2e640 1 -- 172.21.15.60:0/294089368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff19c0cbcf0 msgr2=0x7ff19c160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a1a2e640 1 --2- 172.21.15.60:0/294089368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff19c0cbcf0 0x7ff19c160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a1a2e640 1 -- 172.21.15.60:0/294089368 >> v1:172.21.15.60:6789/0 conn(0x7ff19c071c30 legacy=0x7ff19c15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a1a2e640 1 -- 172.21.15.60:0/294089368 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c0cccd0 con 0x7ff19c15fbf0 2024-04-25T10:06:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a122d640 1 --2- 172.21.15.60:0/294089368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff19c0cbcf0 0x7ff19c160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a1a2e640 1 --2- 172.21.15.60:0/294089368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff19c15fbf0 0x7ff19c0ceec0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff194009e50 tx=0x7ff19401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9816b7bf96ce507 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff193fff640 1 -- 172.21.15.60:0/294089368 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1940a58d0 con 0x7ff19c15fbf0 2024-04-25T10:06:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff193fff640 1 -- 172.21.15.60:0/294089368 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1940a5a70 con 0x7ff19c15fbf0 2024-04-25T10:06:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff193fff640 1 -- 172.21.15.60:0/294089368 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1940a5e40 con 0x7ff19c15fbf0 2024-04-25T10:06:12.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a3491640 1 -- 172.21.15.60:0/294089368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff19c15fbf0 msgr2=0x7ff19c0ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.231+0000 7ff1a3491640 1 --2- 172.21.15.60:0/294089368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff19c15fbf0 0x7ff19c0ceec0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff194009e50 tx=0x7ff19401ca00 comp rx=0 tx=0).stop 2024-04-25T10:06:12.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a3491640 1 -- 172.21.15.60:0/294089368 shutdown_connections 2024-04-25T10:06:12.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a3491640 1 --2- 172.21.15.60:0/294089368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff19c15fbf0 0x7ff19c0ceec0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a3491640 1 --2- 172.21.15.60:0/294089368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff19c0cbcf0 0x7ff19c160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a3491640 1 -- 172.21.15.60:0/294089368 >> 172.21.15.60:0/294089368 conn(0x7ff19c0670b0 msgr2=0x7ff19c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a3491640 1 -- 172.21.15.60:0/294089368 shutdown_connections 2024-04-25T10:06:12.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a3491640 1 -- 172.21.15.60:0/294089368 wait complete. 2024-04-25T10:06:12.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a3491640 1 Processor -- start 2024-04-25T10:06:12.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a3491640 1 -- start start 2024-04-25T10:06:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a3491640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff19c071c30 0x7ff19c1f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a3491640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff19c0cbcf0 0x7ff19c06dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a3491640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff19c15fbf0 0x7ff19c1f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a3491640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0ce740 con 0x7ff19c15fbf0 2024-04-25T10:06:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a3491640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0ce5c0 con 0x7ff19c071c30 2024-04-25T10:06:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a3491640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c072030 con 0x7ff19c0cbcf0 2024-04-25T10:06:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a1a2e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff19c15fbf0 0x7ff19c1f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a1a2e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff19c15fbf0 0x7ff19c1f6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41008/0 (socket says 172.21.15.60:41008) 2024-04-25T10:06:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a1a2e640 1 -- 172.21.15.60:0/2029473799 learned_addr learned my addr 172.21.15.60:0/2029473799 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a122d640 1 --2- 172.21.15.60:0/2029473799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff19c071c30 0x7ff19c1f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a0a2c640 1 --2- 172.21.15.60:0/2029473799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff19c0cbcf0 0x7ff19c06dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a1a2e640 1 -- 172.21.15.60:0/2029473799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff19c0cbcf0 msgr2=0x7ff19c06dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a1a2e640 1 --2- 172.21.15.60:0/2029473799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff19c0cbcf0 0x7ff19c06dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a1a2e640 1 -- 172.21.15.60:0/2029473799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff19c071c30 msgr2=0x7ff19c1f8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a1a2e640 1 --2- 172.21.15.60:0/2029473799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff19c071c30 0x7ff19c1f8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a1a2e640 1 -- 172.21.15.60:0/2029473799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c1655f0 con 0x7ff19c15fbf0 2024-04-25T10:06:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a0a2c640 1 --2- 172.21.15.60:0/2029473799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff19c0cbcf0 0x7ff19c06dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:12.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a1a2e640 1 --2- 172.21.15.60:0/2029473799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff19c15fbf0 0x7ff19c1f6230 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff19401a630 tx=0x7ff1940197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff191ffb640 1 -- 172.21.15.60:0/2029473799 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1940bc070 con 0x7ff19c15fbf0 2024-04-25T10:06:12.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff191ffb640 1 -- 172.21.15.60:0/2029473799 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1940b8440 con 0x7ff19c15fbf0 2024-04-25T10:06:12.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff191ffb640 1 -- 172.21.15.60:0/2029473799 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1940a58d0 con 0x7ff19c15fbf0 2024-04-25T10:06:12.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a3491640 1 -- 172.21.15.60:0/2029473799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c165890 con 0x7ff19c15fbf0 2024-04-25T10:06:12.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.235+0000 7ff1a3491640 1 -- 172.21.15.60:0/2029473799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c165e50 con 0x7ff19c15fbf0 2024-04-25T10:06:12.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.239+0000 7ff1a3491640 1 -- 172.21.15.60:0/2029473799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19c06bba0 con 0x7ff19c15fbf0 2024-04-25T10:06:12.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.239+0000 7ff191ffb640 1 -- 172.21.15.60:0/2029473799 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1940bf070 con 0x7ff19c15fbf0 2024-04-25T10:06:12.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.239+0000 7ff191ffb640 1 --2- 172.21.15.60:0/2029473799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff180066f90 0x7ff180069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.239+0000 7ff191ffb640 1 -- 172.21.15.60:0/2029473799 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7ff1941203a0 con 0x7ff19c15fbf0 2024-04-25T10:06:12.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.239+0000 7ff1a122d640 1 --2- 172.21.15.60:0/2029473799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff180066f90 0x7ff180069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.239+0000 7ff1a122d640 1 --2- 172.21.15.60:0/2029473799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff180066f90 0x7ff180069450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff19800ab60 tx=0x7ff198008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.243+0000 7ff191ffb640 1 -- 172.21.15.60:0/2029473799 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff1940c6030 con 0x7ff19c15fbf0 2024-04-25T10:06:12.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:12.443+0000 7ff1a3491640 1 -- 172.21.15.60:0/2029473799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff19c0b2970 con 0x7ff19c15fbf0 2024-04-25T10:06:13.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.091+0000 7ff191ffb640 1 -- 172.21.15.60:0/2029473799 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff194016620 con 0x7ff19c15fbf0 2024-04-25T10:06:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.095+0000 7ff1a3491640 1 -- 172.21.15.60:0/2029473799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff180066f90 msgr2=0x7ff180069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.095+0000 7ff1a3491640 1 --2- 172.21.15.60:0/2029473799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff180066f90 0x7ff180069450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff19800ab60 tx=0x7ff198008040 comp rx=0 tx=0).stop 2024-04-25T10:06:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.095+0000 7ff1a3491640 1 -- 172.21.15.60:0/2029473799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff19c15fbf0 msgr2=0x7ff19c1f6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.095+0000 7ff1a3491640 1 --2- 172.21.15.60:0/2029473799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff19c15fbf0 0x7ff19c1f6230 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff19401a630 tx=0x7ff1940197a0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.095+0000 7ff1a3491640 1 -- 172.21.15.60:0/2029473799 shutdown_connections 2024-04-25T10:06:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.095+0000 7ff1a3491640 1 --2- 172.21.15.60:0/2029473799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff180066f90 0x7ff180069450 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.095+0000 7ff1a3491640 1 --2- 172.21.15.60:0/2029473799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff19c15fbf0 0x7ff19c1f6230 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.095+0000 7ff1a3491640 1 --2- 172.21.15.60:0/2029473799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff19c0cbcf0 0x7ff19c06dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.095+0000 7ff1a3491640 1 --2- 172.21.15.60:0/2029473799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff19c071c30 0x7ff19c1f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.095+0000 7ff1a3491640 1 -- 172.21.15.60:0/2029473799 >> 172.21.15.60:0/2029473799 conn(0x7ff19c0670b0 msgr2=0x7ff19c15e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.095+0000 7ff1a3491640 1 -- 172.21.15.60:0/2029473799 shutdown_connections 2024-04-25T10:06:13.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.095+0000 7ff1a3491640 1 -- 172.21.15.60:0/2029473799 wait complete. 2024-04-25T10:06:13.124 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:06:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f9388a2c640 1 Processor -- start 2024-04-25T10:06:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f9388a2c640 1 -- start start 2024-04-25T10:06:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f9388a2c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93840adfd0 0x7f938415df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f9388a2c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f93840729f0 0x7f938406bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f9388a2c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938415bb90 con 0x7f938406c160 2024-04-25T10:06:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f9388a2c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93840ae3b0 con 0x7f93840729f0 2024-04-25T10:06:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f9388a2c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938415bee0 con 0x7f93840adfd0 2024-04-25T10:06:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f9382d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f938406c160 0x7f9384161540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42618/0 (socket says 172.21.15.60:42618) 2024-04-25T10:06:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f9382d9d640 1 -- 172.21.15.60:0/494811211 learned_addr learned my addr 172.21.15.60:0/494811211 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f9381d9b640 1 --2- 172.21.15.60:0/494811211 >> v2:172.21.15.100:3300/0 conn(0x7f93840729f0 0x7f938406bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f938259c640 1 --2- 172.21.15.60:0/494811211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93840adfd0 0x7f938415df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f938159a640 1 -- 172.21.15.60:0/494811211 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530194603 0 0) 0x7f938415bb90 con 0x7f938406c160 2024-04-25T10:06:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f938159a640 1 -- 172.21.15.60:0/494811211 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9364003530 con 0x7f938406c160 2024-04-25T10:06:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f9381d9b640 1 -- 172.21.15.60:0/494811211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93840adfd0 msgr2=0x7f938415df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f9381d9b640 1 --2- 172.21.15.60:0/494811211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93840adfd0 0x7f938415df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f9381d9b640 1 -- 172.21.15.60:0/494811211 >> v1:172.21.15.60:6789/0 conn(0x7f938406c160 legacy=0x7f9384161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.251+0000 7f9381d9b640 1 -- 172.21.15.60:0/494811211 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938415e7a0 con 0x7f93840729f0 2024-04-25T10:06:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.255+0000 7f9381d9b640 1 --2- 172.21.15.60:0/494811211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93840729f0 0x7f938406bc20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f936c0097c0 tx=0x7f936c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a733cc29d67b627 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.255+0000 7f938159a640 1 -- 172.21.15.60:0/494811211 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c0b2070 con 0x7f93840729f0 2024-04-25T10:06:13.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.255+0000 7f938159a640 1 -- 172.21.15.60:0/494811211 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936c0ad640 con 0x7f93840729f0 2024-04-25T10:06:13.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.255+0000 7f938159a640 1 -- 172.21.15.60:0/494811211 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c0ad9f0 con 0x7f93840729f0 2024-04-25T10:06:13.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.255+0000 7f9388a2c640 1 -- 172.21.15.60:0/494811211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93840729f0 msgr2=0x7f938406bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.255+0000 7f9388a2c640 1 --2- 172.21.15.60:0/494811211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93840729f0 0x7f938406bc20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f936c0097c0 tx=0x7f936c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.255+0000 7f9388a2c640 1 -- 172.21.15.60:0/494811211 shutdown_connections 2024-04-25T10:06:13.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.255+0000 7f9388a2c640 1 --2- 172.21.15.60:0/494811211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93840729f0 0x7f938406bc20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.255+0000 7f9388a2c640 1 --2- 172.21.15.60:0/494811211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93840adfd0 0x7f938415df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.255+0000 7f9388a2c640 1 -- 172.21.15.60:0/494811211 >> 172.21.15.60:0/494811211 conn(0x7f9384071b00 msgr2=0x7f9384071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.255+0000 7f9388a2c640 1 -- 172.21.15.60:0/494811211 shutdown_connections 2024-04-25T10:06:13.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.255+0000 7f9388a2c640 1 -- 172.21.15.60:0/494811211 wait complete. 2024-04-25T10:06:13.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.255+0000 7f9388a2c640 1 Processor -- start 2024-04-25T10:06:13.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.255+0000 7f9388a2c640 1 -- start start 2024-04-25T10:06:13.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9388a2c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f938406c160 0x7f938414b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9388a2c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93840729f0 0x7f938415c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9388a2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93840adfd0 0x7f938415c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9388a2c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938416aa60 con 0x7f93840729f0 2024-04-25T10:06:13.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9388a2c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938416a8e0 con 0x7f93840adfd0 2024-04-25T10:06:13.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9388a2c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938416abe0 con 0x7f938406c160 2024-04-25T10:06:13.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9381d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93840729f0 0x7f938415c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9381d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93840729f0 0x7f938415c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41010/0 (socket says 172.21.15.60:41010) 2024-04-25T10:06:13.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9381d9b640 1 -- 172.21.15.60:0/3993257628 learned_addr learned my addr 172.21.15.60:0/3993257628 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:13.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f938259c640 1 --2- 172.21.15.60:0/3993257628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f938406c160 0x7f938414b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9382d9d640 1 --2- 172.21.15.60:0/3993257628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93840adfd0 0x7f938415c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9381d9b640 1 -- 172.21.15.60:0/3993257628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f938406c160 msgr2=0x7f938414b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9381d9b640 1 --2- 172.21.15.60:0/3993257628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f938406c160 0x7f938414b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9381d9b640 1 -- 172.21.15.60:0/3993257628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93840adfd0 msgr2=0x7f938415c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9381d9b640 1 --2- 172.21.15.60:0/3993257628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93840adfd0 0x7f938415c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9381d9b640 1 -- 172.21.15.60:0/3993257628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938415f450 con 0x7f93840729f0 2024-04-25T10:06:13.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9382d9d640 1 --2- 172.21.15.60:0/3993257628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93840adfd0 0x7f938415c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:13.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9381d9b640 1 --2- 172.21.15.60:0/3993257628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93840729f0 0x7f938415c370 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f936c01a0d0 tx=0x7f936c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f937affd640 1 -- 172.21.15.60:0/3993257628 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c0bc070 con 0x7f93840729f0 2024-04-25T10:06:13.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f937affd640 1 -- 172.21.15.60:0/3993257628 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936c0ad9f0 con 0x7f93840729f0 2024-04-25T10:06:13.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9388a2c640 1 -- 172.21.15.60:0/3993257628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938415f730 con 0x7f93840729f0 2024-04-25T10:06:13.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f937affd640 1 -- 172.21.15.60:0/3993257628 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c0b2040 con 0x7f93840729f0 2024-04-25T10:06:13.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9388a2c640 1 -- 172.21.15.60:0/3993257628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938415fcf0 con 0x7f93840729f0 2024-04-25T10:06:13.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f9388a2c640 1 -- 172.21.15.60:0/3993257628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9384074770 con 0x7f93840729f0 2024-04-25T10:06:13.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f937affd640 1 -- 172.21.15.60:0/3993257628 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f936c0bf070 con 0x7f93840729f0 2024-04-25T10:06:13.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f937affd640 1 --2- 172.21.15.60:0/3993257628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9368066f60 0x7f9368069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.259+0000 7f937affd640 1 -- 172.21.15.60:0/3993257628 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f936c11e6c0 con 0x7f93840729f0 2024-04-25T10:06:13.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.263+0000 7f938259c640 1 --2- 172.21.15.60:0/3993257628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9368066f60 0x7f9368069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.263+0000 7f938259c640 1 --2- 172.21.15.60:0/3993257628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9368066f60 0x7f9368069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f937c007380 tx=0x7f937c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.263+0000 7f937affd640 1 -- 172.21.15.60:0/3993257628 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f936c0c6080 con 0x7f93840729f0 2024-04-25T10:06:13.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:13.427+0000 7f9388a2c640 1 -- 172.21.15.60:0/3993257628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f93840b5290 con 0x7f93840729f0 2024-04-25T10:06:14.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.095+0000 7f937affd640 1 -- 172.21.15.60:0/3993257628 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 181+0+0 (secure 0 0 0) 0x7f936c016090 con 0x7f93840729f0 2024-04-25T10:06:14.102 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:06:14.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.099+0000 7f9388a2c640 1 -- 172.21.15.60:0/3993257628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9368066f60 msgr2=0x7f9368069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.099+0000 7f9388a2c640 1 --2- 172.21.15.60:0/3993257628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9368066f60 0x7f9368069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f937c007380 tx=0x7f937c00b040 comp rx=0 tx=0).stop 2024-04-25T10:06:14.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.099+0000 7f9388a2c640 1 -- 172.21.15.60:0/3993257628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93840729f0 msgr2=0x7f938415c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.099+0000 7f9388a2c640 1 --2- 172.21.15.60:0/3993257628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93840729f0 0x7f938415c370 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f936c01a0d0 tx=0x7f936c019240 comp rx=0 tx=0).stop 2024-04-25T10:06:14.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.099+0000 7f9388a2c640 1 -- 172.21.15.60:0/3993257628 shutdown_connections 2024-04-25T10:06:14.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.099+0000 7f9388a2c640 1 --2- 172.21.15.60:0/3993257628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9368066f60 0x7f9368069420 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.099+0000 7f9388a2c640 1 --2- 172.21.15.60:0/3993257628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93840adfd0 0x7f938415c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.099+0000 7f9388a2c640 1 --2- 172.21.15.60:0/3993257628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93840729f0 0x7f938415c370 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.099+0000 7f9388a2c640 1 --2- 172.21.15.60:0/3993257628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f938406c160 0x7f938414b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.099+0000 7f9388a2c640 1 -- 172.21.15.60:0/3993257628 >> 172.21.15.60:0/3993257628 conn(0x7f9384071b00 msgr2=0x7f938415d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.099+0000 7f9388a2c640 1 -- 172.21.15.60:0/3993257628 shutdown_connections 2024-04-25T10:06:14.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.103+0000 7f9388a2c640 1 -- 172.21.15.60:0/3993257628 wait complete. 2024-04-25T10:06:14.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:06:14.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.251+0000 7f7789ba7640 1 Processor -- start 2024-04-25T10:06:14.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.251+0000 7f7789ba7640 1 -- start start 2024-04-25T10:06:14.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f7789ba7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7784159f90 0x7f77840ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f7789ba7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7784071980 0x7f77840b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f7789ba7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778415a370 con 0x7f778406f140 2024-04-25T10:06:14.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f7789ba7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f778406d7d0 con 0x7f7784071980 2024-04-25T10:06:14.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f7789ba7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f778406dab0 con 0x7f7784159f90 2024-04-25T10:06:14.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f7783fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7784071980 0x7f77840b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f7783fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7784071980 0x7f77840b2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41012/0 (socket says 172.21.15.60:41012) 2024-04-25T10:06:14.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f7783fff640 1 -- 172.21.15.60:0/1493219985 learned_addr learned my addr 172.21.15.60:0/1493219985 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:14.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f77837fe640 1 --2- 172.21.15.60:0/1493219985 >> v2:172.21.15.100:3300/0 conn(0x7f7784159f90 0x7f77840ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f7783fff640 1 -- 172.21.15.60:0/1493219985 >> v2:172.21.15.100:3300/0 conn(0x7f7784159f90 msgr2=0x7f77840ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f7783fff640 1 --2- 172.21.15.60:0/1493219985 >> v2:172.21.15.100:3300/0 conn(0x7f7784159f90 0x7f77840ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f7783fff640 1 -- 172.21.15.60:0/1493219985 >> v1:172.21.15.60:6789/0 conn(0x7f778406f140 legacy=0x7f7784071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f7783fff640 1 -- 172.21.15.60:0/1493219985 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778406dd90 con 0x7f7784071980 2024-04-25T10:06:14.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f7783fff640 1 --2- 172.21.15.60:0/1493219985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7784071980 0x7f77840b2a20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f778406bb50 tx=0x7f777c01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c54c74ab2b7606 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f77827fc640 1 -- 172.21.15.60:0/1493219985 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77840cd330 con 0x7f7784071980 2024-04-25T10:06:14.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f77827fc640 1 -- 172.21.15.60:0/1493219985 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c00b2d0 con 0x7f7784071980 2024-04-25T10:06:14.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.255+0000 7f77827fc640 1 -- 172.21.15.60:0/1493219985 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c00b480 con 0x7f7784071980 2024-04-25T10:06:14.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7789ba7640 1 -- 172.21.15.60:0/1493219985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7784071980 msgr2=0x7f77840b2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7789ba7640 1 --2- 172.21.15.60:0/1493219985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7784071980 0x7f77840b2a20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f778406bb50 tx=0x7f777c01ebf0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7789ba7640 1 -- 172.21.15.60:0/1493219985 shutdown_connections 2024-04-25T10:06:14.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7789ba7640 1 --2- 172.21.15.60:0/1493219985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7784071980 0x7f77840b2a20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7789ba7640 1 --2- 172.21.15.60:0/1493219985 >> v2:172.21.15.100:3300/0 conn(0x7f7784159f90 0x7f77840ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7789ba7640 1 -- 172.21.15.60:0/1493219985 >> 172.21.15.60:0/1493219985 conn(0x7f7784066fa0 msgr2=0x7f778406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7789ba7640 1 -- 172.21.15.60:0/1493219985 shutdown_connections 2024-04-25T10:06:14.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7789ba7640 1 -- 172.21.15.60:0/1493219985 wait complete. 2024-04-25T10:06:14.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7789ba7640 1 Processor -- start 2024-04-25T10:06:14.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7789ba7640 1 -- start start 2024-04-25T10:06:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7789ba7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f778406f140 0x7f77841f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7789ba7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7784071980 0x7f7784153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7789ba7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7784159f90 0x7f7784154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7789ba7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77840b3330 con 0x7f778406f140 2024-04-25T10:06:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7789ba7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77840b31b0 con 0x7f7784071980 2024-04-25T10:06:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7789ba7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77840b34b0 con 0x7f7784159f90 2024-04-25T10:06:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f77837fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f778406f140 0x7f77841f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f77837fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f778406f140 0x7f77841f6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41026/0 (socket says 172.21.15.60:41026) 2024-04-25T10:06:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f77837fe640 1 -- 172.21.15.60:0/1255741994 learned_addr learned my addr 172.21.15.60:0/1255741994 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7782ffd640 1 --2- 172.21.15.60:0/1255741994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7784071980 0x7f7784153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f7783fff640 1 --2- 172.21.15.60:0/1255741994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7784159f90 0x7f7784154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f77837fe640 1 -- 172.21.15.60:0/1255741994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7784159f90 msgr2=0x7f7784154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f77837fe640 1 --2- 172.21.15.60:0/1255741994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7784159f90 0x7f7784154670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f77837fe640 1 -- 172.21.15.60:0/1255741994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7784071980 msgr2=0x7f7784153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f77837fe640 1 --2- 172.21.15.60:0/1255741994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7784071980 0x7f7784153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f77837fe640 1 -- 172.21.15.60:0/1255741994 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7784157490 con 0x7f778406f140 2024-04-25T10:06:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f77837fe640 1 --2- 172.21.15.60:0/1255741994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f778406f140 0x7f77841f6f20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f777400d5f0 tx=0x7f777400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f776bfff640 1 -- 172.21.15.60:0/1255741994 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77740125f0 con 0x7f778406f140 2024-04-25T10:06:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f776bfff640 1 -- 172.21.15.60:0/1255741994 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7774019070 con 0x7f778406f140 2024-04-25T10:06:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.259+0000 7f776bfff640 1 -- 172.21.15.60:0/1255741994 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77740129b0 con 0x7f778406f140 2024-04-25T10:06:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.263+0000 7f7789ba7640 1 -- 172.21.15.60:0/1255741994 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7784157770 con 0x7f778406f140 2024-04-25T10:06:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.263+0000 7f7789ba7640 1 -- 172.21.15.60:0/1255741994 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7784157d30 con 0x7f778406f140 2024-04-25T10:06:14.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.263+0000 7f7789ba7640 1 -- 172.21.15.60:0/1255741994 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7784074770 con 0x7f778406f140 2024-04-25T10:06:14.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.263+0000 7f776bfff640 1 -- 172.21.15.60:0/1255741994 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7774012b50 con 0x7f778406f140 2024-04-25T10:06:14.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.263+0000 7f776bfff640 1 --2- 172.21.15.60:0/1255741994 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7770066f60 0x7f7770069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.263+0000 7f776bfff640 1 -- 172.21.15.60:0/1255741994 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f7774085350 con 0x7f778406f140 2024-04-25T10:06:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.263+0000 7f7782ffd640 1 --2- 172.21.15.60:0/1255741994 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7770066f60 0x7f7770069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.263+0000 7f7782ffd640 1 --2- 172.21.15.60:0/1255741994 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7770066f60 0x7f7770069420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7784157140 tx=0x7f7778009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.267+0000 7f776bfff640 1 -- 172.21.15.60:0/1255741994 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7774056420 con 0x7f778406f140 2024-04-25T10:06:14.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:14.443+0000 7f7789ba7640 1 -- 172.21.15.60:0/1255741994 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f77840722d0 con 0x7f778406f140 2024-04-25T10:06:15.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.099+0000 7f776bfff640 1 -- 172.21.15.60:0/1255741994 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7774055d20 con 0x7f778406f140 2024-04-25T10:06:15.105 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T10:06:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.103+0000 7f7789ba7640 1 -- 172.21.15.60:0/1255741994 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7770066f60 msgr2=0x7f7770069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.103+0000 7f7789ba7640 1 --2- 172.21.15.60:0/1255741994 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7770066f60 0x7f7770069420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7784157140 tx=0x7f7778009040 comp rx=0 tx=0).stop 2024-04-25T10:06:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.103+0000 7f7789ba7640 1 -- 172.21.15.60:0/1255741994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f778406f140 msgr2=0x7f77841f6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.103+0000 7f7789ba7640 1 --2- 172.21.15.60:0/1255741994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f778406f140 0x7f77841f6f20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f777400d5f0 tx=0x7f777400dab0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.103+0000 7f7789ba7640 1 -- 172.21.15.60:0/1255741994 shutdown_connections 2024-04-25T10:06:15.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.103+0000 7f7789ba7640 1 --2- 172.21.15.60:0/1255741994 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7770066f60 0x7f7770069420 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.103+0000 7f7789ba7640 1 --2- 172.21.15.60:0/1255741994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7784159f90 0x7f7784154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.103+0000 7f7789ba7640 1 --2- 172.21.15.60:0/1255741994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7784071980 0x7f7784153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.103+0000 7f7789ba7640 1 --2- 172.21.15.60:0/1255741994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f778406f140 0x7f77841f6f20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.103+0000 7f7789ba7640 1 -- 172.21.15.60:0/1255741994 >> 172.21.15.60:0/1255741994 conn(0x7f7784066fa0 msgr2=0x7f77840af430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.103+0000 7f7789ba7640 1 -- 172.21.15.60:0/1255741994 shutdown_connections 2024-04-25T10:06:15.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.103+0000 7f7789ba7640 1 -- 172.21.15.60:0/1255741994 wait complete. 2024-04-25T10:06:15.137 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:06:15.137 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:06:15.137 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:06:15.137 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:15.137 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:06:15.137 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:06:15.138 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:06:15.138 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:15.138 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:06:15.138 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:06:15.138 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:06:15.138 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:15.138 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:06:15.139 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:06:15.139 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:06:15.139 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:15.148 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:06:15.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:06:15.149 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:06:15.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:06:15.152 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:06:15.152 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:06:15.155 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:06:15.155 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:06:15.193 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:06:15.193 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:15.194 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:06:15.194 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:15.200 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:06:15.201 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:15.202 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:06:15.202 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:15.202 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:06:15.263 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T10:06:15.264 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:06:15.272 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T10:06:15.273 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T10:06:15.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83cd35e640 1 Processor -- start 2024-04-25T10:06:15.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83cd35e640 1 -- start start 2024-04-25T10:06:15.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83cd35e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f83c806c330 0x7f83c815ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83cd35e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83c80b2ed0 0x7f83c80b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83cd35e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c80ad670 con 0x7f83c8070300 2024-04-25T10:06:15.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83cd35e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c80ad950 con 0x7f83c80b2ed0 2024-04-25T10:06:15.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83cd35e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c80adc30 con 0x7f83c806c330 2024-04-25T10:06:15.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83c77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83c80b2ed0 0x7f83c80b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83c77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83c80b2ed0 0x7f83c80b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41074/0 (socket says 172.21.15.60:41074) 2024-04-25T10:06:15.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83c77fe640 1 -- 172.21.15.60:0/1754963177 learned_addr learned my addr 172.21.15.60:0/1754963177 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:15.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83c67fc640 1 --2- 172.21.15.60:0/1754963177 >> v2:172.21.15.100:3300/0 conn(0x7f83c806c330 0x7f83c815ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83c77fe640 1 -- 172.21.15.60:0/1754963177 >> v2:172.21.15.100:3300/0 conn(0x7f83c806c330 msgr2=0x7f83c815ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83c77fe640 1 --2- 172.21.15.60:0/1754963177 >> v2:172.21.15.100:3300/0 conn(0x7f83c806c330 0x7f83c815ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83c77fe640 1 -- 172.21.15.60:0/1754963177 >> v1:172.21.15.100:6789/0 conn(0x7f83c8070300 legacy=0x7f83c806bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83c77fe640 1 -- 172.21.15.60:0/1754963177 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c80adf10 con 0x7f83c80b2ed0 2024-04-25T10:06:15.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83c77fe640 1 --2- 172.21.15.60:0/1754963177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c80b2ed0 0x7f83c80b32e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f83c8071700 tx=0x7f83b801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab52b66ed97b19a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83c5ffb640 1 -- 172.21.15.60:0/1754963177 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b80b0070 con 0x7f83c80b2ed0 2024-04-25T10:06:15.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83c5ffb640 1 -- 172.21.15.60:0/1754963177 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b80ab600 con 0x7f83c80b2ed0 2024-04-25T10:06:15.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.315+0000 7f83c5ffb640 1 -- 172.21.15.60:0/1754963177 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b80ab9d0 con 0x7f83c80b2ed0 2024-04-25T10:06:15.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 -- 172.21.15.60:0/1754963177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c80b2ed0 msgr2=0x7f83c80b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 --2- 172.21.15.60:0/1754963177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c80b2ed0 0x7f83c80b32e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f83c8071700 tx=0x7f83b801c490 comp rx=0 tx=0).stop 2024-04-25T10:06:15.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 -- 172.21.15.60:0/1754963177 shutdown_connections 2024-04-25T10:06:15.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 --2- 172.21.15.60:0/1754963177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c80b2ed0 0x7f83c80b32e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 --2- 172.21.15.60:0/1754963177 >> v2:172.21.15.100:3300/0 conn(0x7f83c806c330 0x7f83c815ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 -- 172.21.15.60:0/1754963177 >> 172.21.15.60:0/1754963177 conn(0x7f83c8071b90 msgr2=0x7f83c8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 -- 172.21.15.60:0/1754963177 shutdown_connections 2024-04-25T10:06:15.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 -- 172.21.15.60:0/1754963177 wait complete. 2024-04-25T10:06:15.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 Processor -- start 2024-04-25T10:06:15.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 -- start start 2024-04-25T10:06:15.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83c806c330 0x7f83c815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83c8070300 0x7f83c81f9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c80b2ed0 0x7f83c81fa320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c816a860 con 0x7f83c80b2ed0 2024-04-25T10:06:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c816a6e0 con 0x7f83c8070300 2024-04-25T10:06:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c816a9e0 con 0x7f83c806c330 2024-04-25T10:06:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83c77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c80b2ed0 0x7f83c81fa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83c77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c80b2ed0 0x7f83c81fa320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41088/0 (socket says 172.21.15.60:41088) 2024-04-25T10:06:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83c77fe640 1 -- 172.21.15.60:0/614810577 learned_addr learned my addr 172.21.15.60:0/614810577 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83c6ffd640 1 --2- 172.21.15.60:0/614810577 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83c806c330 0x7f83c815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83c67fc640 1 --2- 172.21.15.60:0/614810577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83c8070300 0x7f83c81f9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83c77fe640 1 -- 172.21.15.60:0/614810577 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83c806c330 msgr2=0x7f83c815c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83c77fe640 1 --2- 172.21.15.60:0/614810577 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83c806c330 0x7f83c815c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83c77fe640 1 -- 172.21.15.60:0/614810577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83c8070300 msgr2=0x7f83c81f9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83c77fe640 1 --2- 172.21.15.60:0/614810577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83c8070300 0x7f83c81f9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83c77fe640 1 -- 172.21.15.60:0/614810577 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c8151360 con 0x7f83c80b2ed0 2024-04-25T10:06:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83c77fe640 1 --2- 172.21.15.60:0/614810577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c80b2ed0 0x7f83c81fa320 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f83b80190a0 tx=0x7f83b8019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83bf7fe640 1 -- 172.21.15.60:0/614810577 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b80b0040 con 0x7f83c80b2ed0 2024-04-25T10:06:15.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83bf7fe640 1 -- 172.21.15.60:0/614810577 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b80b4420 con 0x7f83c80b2ed0 2024-04-25T10:06:15.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 -- 172.21.15.60:0/614810577 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c8151640 con 0x7f83c80b2ed0 2024-04-25T10:06:15.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83bf7fe640 1 -- 172.21.15.60:0/614810577 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b80b47d0 con 0x7f83c80b2ed0 2024-04-25T10:06:15.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.319+0000 7f83cd35e640 1 -- 172.21.15.60:0/614810577 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c8155fd0 con 0x7f83c80b2ed0 2024-04-25T10:06:15.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.323+0000 7f83cd35e640 1 -- 172.21.15.60:0/614810577 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c81561d0 con 0x7f83c80b2ed0 2024-04-25T10:06:15.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.323+0000 7f83bf7fe640 1 -- 172.21.15.60:0/614810577 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83b80b4970 con 0x7f83c80b2ed0 2024-04-25T10:06:15.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.323+0000 7f83bf7fe640 1 --2- 172.21.15.60:0/614810577 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83a8066f60 0x7f83a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.323+0000 7f83bf7fe640 1 -- 172.21.15.60:0/614810577 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f83b811e6e0 con 0x7f83c80b2ed0 2024-04-25T10:06:15.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.323+0000 7f83c6ffd640 1 --2- 172.21.15.60:0/614810577 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83a8066f60 0x7f83a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.323+0000 7f83c6ffd640 1 --2- 172.21.15.60:0/614810577 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83a8066f60 0x7f83a8069420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f83c0001ea0 tx=0x7f83c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.327+0000 7f83bf7fe640 1 -- 172.21.15.60:0/614810577 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83b8016110 con 0x7f83c80b2ed0 2024-04-25T10:06:15.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:15.499+0000 7f83cd35e640 1 -- 172.21.15.60:0/614810577 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f83c8074770 con 0x7f83c80b2ed0 2024-04-25T10:06:16.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.135+0000 7f83bf7fe640 1 -- 172.21.15.60:0/614810577 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) v1 ==== 97+0+0 (secure 0 0 0) 0x7f83b80ef430 con 0x7f83c80b2ed0 2024-04-25T10:06:16.142 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-04-25T10:06:16.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.139+0000 7f83cd35e640 1 -- 172.21.15.60:0/614810577 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83a8066f60 msgr2=0x7f83a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.139+0000 7f83cd35e640 1 --2- 172.21.15.60:0/614810577 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83a8066f60 0x7f83a8069420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f83c0001ea0 tx=0x7f83c0009040 comp rx=0 tx=0).stop 2024-04-25T10:06:16.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.139+0000 7f83cd35e640 1 -- 172.21.15.60:0/614810577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c80b2ed0 msgr2=0x7f83c81fa320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.139+0000 7f83cd35e640 1 --2- 172.21.15.60:0/614810577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c80b2ed0 0x7f83c81fa320 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f83b80190a0 tx=0x7f83b8019230 comp rx=0 tx=0).stop 2024-04-25T10:06:16.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.139+0000 7f83cd35e640 1 -- 172.21.15.60:0/614810577 shutdown_connections 2024-04-25T10:06:16.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.139+0000 7f83cd35e640 1 --2- 172.21.15.60:0/614810577 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83a8066f60 0x7f83a8069420 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.139+0000 7f83cd35e640 1 --2- 172.21.15.60:0/614810577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83c80b2ed0 0x7f83c81fa320 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.139+0000 7f83cd35e640 1 --2- 172.21.15.60:0/614810577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83c8070300 0x7f83c81f9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.139+0000 7f83cd35e640 1 --2- 172.21.15.60:0/614810577 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83c806c330 0x7f83c815c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.139+0000 7f83cd35e640 1 -- 172.21.15.60:0/614810577 >> 172.21.15.60:0/614810577 conn(0x7f83c8071b90 msgr2=0x7f83c80b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:16.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.139+0000 7f83cd35e640 1 -- 172.21.15.60:0/614810577 shutdown_connections 2024-04-25T10:06:16.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.139+0000 7f83cd35e640 1 -- 172.21.15.60:0/614810577 wait complete. 2024-04-25T10:06:16.171 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:06:16.253 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:16.255 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:16.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.291+0000 7fefddb5f640 1 Processor -- start 2024-04-25T10:06:16.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.291+0000 7fefddb5f640 1 -- start start 2024-04-25T10:06:16.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.291+0000 7fefddb5f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fefd80ce7a0 0x7fefd80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.291+0000 7fefddb5f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fefd815bc90 0x7fefd80b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.291+0000 7fefddb5f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd80cf0c0 con 0x7fefd80b4660 2024-04-25T10:06:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.291+0000 7fefddb5f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd80cc370 con 0x7fefd815bc90 2024-04-25T10:06:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.291+0000 7fefddb5f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd8071ca0 con 0x7fefd80ce7a0 2024-04-25T10:06:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.291+0000 7fefd6ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fefd815bc90 0x7fefd80b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.291+0000 7fefd6ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fefd815bc90 0x7fefd80b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41102/0 (socket says 172.21.15.60:41102) 2024-04-25T10:06:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.291+0000 7fefd6ffd640 1 -- 172.21.15.60:0/2367603066 learned_addr learned my addr 172.21.15.60:0/2367603066 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.291+0000 7fefd77fe640 1 --2- 172.21.15.60:0/2367603066 >> v2:172.21.15.100:3300/0 conn(0x7fefd80ce7a0 0x7fefd80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.291+0000 7fefd6ffd640 1 -- 172.21.15.60:0/2367603066 >> v2:172.21.15.100:3300/0 conn(0x7fefd80ce7a0 msgr2=0x7fefd80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.291+0000 7fefd6ffd640 1 --2- 172.21.15.60:0/2367603066 >> v2:172.21.15.100:3300/0 conn(0x7fefd80ce7a0 0x7fefd80ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.291+0000 7fefd6ffd640 1 -- 172.21.15.60:0/2367603066 >> v1:172.21.15.100:6789/0 conn(0x7fefd80b4660 legacy=0x7fefd80b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.291+0000 7fefd6ffd640 1 -- 172.21.15.60:0/2367603066 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd80cca40 con 0x7fefd815bc90 2024-04-25T10:06:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.291+0000 7fefd77fe640 1 --2- 172.21.15.60:0/2367603066 >> v2:172.21.15.100:3300/0 conn(0x7fefd80ce7a0 0x7fefd80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefd6ffd640 1 --2- 172.21.15.60:0/2367603066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd815bc90 0x7fefd80b25d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fefd804f5d0 tx=0x7fefcc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f150a7c80a647f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefd67fc640 1 -- 172.21.15.60:0/2367603066 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc0b0070 con 0x7fefd815bc90 2024-04-25T10:06:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefd67fc640 1 -- 172.21.15.60:0/2367603066 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefcc0ab600 con 0x7fefd815bc90 2024-04-25T10:06:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefd67fc640 1 -- 172.21.15.60:0/2367603066 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc0ab9d0 con 0x7fefd815bc90 2024-04-25T10:06:16.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefddb5f640 1 -- 172.21.15.60:0/2367603066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd815bc90 msgr2=0x7fefd80b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefddb5f640 1 --2- 172.21.15.60:0/2367603066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd815bc90 0x7fefd80b25d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fefd804f5d0 tx=0x7fefcc01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefddb5f640 1 -- 172.21.15.60:0/2367603066 shutdown_connections 2024-04-25T10:06:16.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefddb5f640 1 --2- 172.21.15.60:0/2367603066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd815bc90 0x7fefd80b25d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefddb5f640 1 --2- 172.21.15.60:0/2367603066 >> v2:172.21.15.100:3300/0 conn(0x7fefd80ce7a0 0x7fefd80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefddb5f640 1 -- 172.21.15.60:0/2367603066 >> 172.21.15.60:0/2367603066 conn(0x7fefd80b12d0 msgr2=0x7fefd80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:16.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefddb5f640 1 -- 172.21.15.60:0/2367603066 shutdown_connections 2024-04-25T10:06:16.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefddb5f640 1 -- 172.21.15.60:0/2367603066 wait complete. 2024-04-25T10:06:16.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefddb5f640 1 Processor -- start 2024-04-25T10:06:16.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefddb5f640 1 -- start start 2024-04-25T10:06:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefddb5f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefd80ce7a0 0x7fefd81542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefddb5f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fefd815bc90 0x7fefd8137fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefddb5f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd8152aa0 0x7fefd8156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefddb5f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd80b4400 con 0x7fefd8152aa0 2024-04-25T10:06:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefddb5f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd80b4280 con 0x7fefd80ce7a0 2024-04-25T10:06:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefddb5f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd80b3220 con 0x7fefd815bc90 2024-04-25T10:06:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefd7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd8152aa0 0x7fefd8156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefd7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd8152aa0 0x7fefd8156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41114/0 (socket says 172.21.15.60:41114) 2024-04-25T10:06:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefd7fff640 1 -- 172.21.15.60:0/2680193928 learned_addr learned my addr 172.21.15.60:0/2680193928 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefd6ffd640 1 --2- 172.21.15.60:0/2680193928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fefd815bc90 0x7fefd8137fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.295+0000 7fefd77fe640 1 --2- 172.21.15.60:0/2680193928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefd80ce7a0 0x7fefd81542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefd7fff640 1 -- 172.21.15.60:0/2680193928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fefd815bc90 msgr2=0x7fefd8137fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefd7fff640 1 --2- 172.21.15.60:0/2680193928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fefd815bc90 0x7fefd8137fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefd7fff640 1 -- 172.21.15.60:0/2680193928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefd80ce7a0 msgr2=0x7fefd81542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefd7fff640 1 --2- 172.21.15.60:0/2680193928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefd80ce7a0 0x7fefd81542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefd7fff640 1 -- 172.21.15.60:0/2680193928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd814e2c0 con 0x7fefd8152aa0 2024-04-25T10:06:16.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefd77fe640 1 --2- 172.21.15.60:0/2680193928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefd80ce7a0 0x7fefd81542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:16.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefd7fff640 1 --2- 172.21.15.60:0/2680193928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd8152aa0 0x7fefd8156660 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fefd000c780 tx=0x7fefd000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:16.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefbffff640 1 -- 172.21.15.60:0/2680193928 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0004b70 con 0x7fefd8152aa0 2024-04-25T10:06:16.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefbffff640 1 -- 172.21.15.60:0/2680193928 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd0004d10 con 0x7fefd8152aa0 2024-04-25T10:06:16.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefbffff640 1 -- 172.21.15.60:0/2680193928 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd000aa30 con 0x7fefd8152aa0 2024-04-25T10:06:16.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefddb5f640 1 -- 172.21.15.60:0/2680193928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd8154830 con 0x7fefd8152aa0 2024-04-25T10:06:16.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefddb5f640 1 -- 172.21.15.60:0/2680193928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd8154db0 con 0x7fefd8152aa0 2024-04-25T10:06:16.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefddb5f640 1 -- 172.21.15.60:0/2680193928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd8074770 con 0x7fefd8152aa0 2024-04-25T10:06:16.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefbffff640 1 -- 172.21.15.60:0/2680193928 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fefd000abd0 con 0x7fefd8152aa0 2024-04-25T10:06:16.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefbffff640 1 --2- 172.21.15.60:0/2680193928 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fefc4066f60 0x7fefc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefbffff640 1 -- 172.21.15.60:0/2680193928 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fefd0084fa0 con 0x7fefd8152aa0 2024-04-25T10:06:16.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefd77fe640 1 --2- 172.21.15.60:0/2680193928 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fefc4066f60 0x7fefc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.299+0000 7fefd77fe640 1 --2- 172.21.15.60:0/2680193928 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fefc4066f60 0x7fefc4069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fefc8006fd0 tx=0x7fefc8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:16.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.303+0000 7fefbffff640 1 -- 172.21.15.60:0/2680193928 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fefd0020070 con 0x7fefd8152aa0 2024-04-25T10:06:16.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.523+0000 7fefddb5f640 1 -- 172.21.15.60:0/2680193928 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fefd80b37a0 con 0x7fefd8152aa0 2024-04-25T10:06:16.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.523+0000 7fefbffff640 1 -- 172.21.15.60:0/2680193928 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fefd814e2c0 con 0x7fefd8152aa0 2024-04-25T10:06:16.532 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:16.532 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:06:16.532 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-04-25T10:06:16.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.532 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-04-25T10:06:16.532 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmKr12EhAAnHHZ5VjG0cywMrIv6VElVg==", 2024-04-25T10:06:16.532 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.532 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:06:16.533 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:06:16.533 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:06:16.533 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:06:16.533 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.533 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-04-25T10:06:16.533 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmU39rIRAAk6Ca0tKG4LpmR4pNxjzv0w==", 2024-04-25T10:06:16.533 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.533 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:06:16.533 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:06:16.533 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:06:16.533 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:06:16.533 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.534 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-04-25T10:06:16.534 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmVsshGxAAttXv0jYEGKgo0sPhIRTuqQ==", 2024-04-25T10:06:16.534 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.534 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:06:16.534 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:06:16.534 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:06:16.534 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:06:16.534 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.534 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-04-25T10:06:16.534 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmQtXUKRAAI3Iq02h3KH/PZbWKvJD9Jg==", 2024-04-25T10:06:16.535 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.535 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:06:16.535 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:06:16.535 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:06:16.535 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:06:16.535 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.535 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-04-25T10:06:16.535 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCkKipmE+UIBRAAc8ctpHoL3PkkbkMRCAJUGw==", 2024-04-25T10:06:16.535 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.535 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:16.535 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:06:16.535 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:06:16.536 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.536 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-04-25T10:06:16.536 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCnKipmbAnYCBAALLsPH2FjUpjoIZwTw+8aOA==", 2024-04-25T10:06:16.536 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.536 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:16.536 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:06:16.536 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:06:16.536 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.536 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-04-25T10:06:16.537 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCqKipmqrNSDxAAZdgFrcyeVwRQMKJJYIM98g==", 2024-04-25T10:06:16.537 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.537 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:16.537 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:06:16.537 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:06:16.537 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.537 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.537 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-04-25T10:06:16.537 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCtKipm1L+hFxAAPnPWlCm1Fmi0mN3tyReeQw==", 2024-04-25T10:06:16.537 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.537 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:16.537 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:06:16.537 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:06:16.538 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.538 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-04-25T10:06:16.538 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCzKipm2WyDFRAAmyfg6qu5drE9yMaR2FKd/A==", 2024-04-25T10:06:16.538 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.538 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:16.538 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:06:16.538 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:06:16.538 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.538 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-04-25T10:06:16.538 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC2KipmZLDhGRAAcgaz8YaPGUMIfxOLtWObyA==", 2024-04-25T10:06:16.539 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.539 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:16.539 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:06:16.539 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:06:16.539 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.539 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.539 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-04-25T10:06:16.539 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC5Kipm/3VMIBAA+rhGCyA/7d+s8m7ZWTzX/Q==", 2024-04-25T10:06:16.539 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.539 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:16.539 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:06:16.539 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:06:16.539 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.540 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.540 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-04-25T10:06:16.540 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8KipmFNCCJxAAv0OSTpAG+xIHhLqYXAZazQ==", 2024-04-25T10:06:16.540 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.540 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:16.540 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:06:16.540 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:06:16.540 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.540 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.540 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.540 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-04-25T10:06:16.540 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmv65dLxAA6/QpPrqE9aT+CtZqkyutog==", 2024-04-25T10:06:16.540 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.540 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:06:16.540 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:06:16.541 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:06:16.541 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:06:16.541 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.541 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.541 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-04-25T10:06:16.541 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipm2NiONxAAd3X4zx1JnBXKHJvdZ/zMdQ==", 2024-04-25T10:06:16.541 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.541 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:06:16.541 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:06:16.541 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:06:16.541 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:06:16.541 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.541 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.542 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-04-25T10:06:16.542 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmZRONNBAAa+XTv5YIBJWeUvf5dfoWxw==", 2024-04-25T10:06:16.542 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.542 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:06:16.542 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:06:16.542 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:06:16.542 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:06:16.542 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.542 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.542 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-04-25T10:06:16.542 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCjKipmgvfiABAApLvTC9U6HzeAefCvX4tiSQ==", 2024-04-25T10:06:16.542 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.542 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:06:16.543 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:06:16.543 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:06:16.543 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:06:16.543 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.543 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-04-25T10:06:16.543 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQChKipmumzYNhAAlCRYMpSNaTo7bz8P6x0W9g==", 2024-04-25T10:06:16.543 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.543 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:06:16.543 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:06:16.543 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:06:16.543 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:06:16.543 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-mds", 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDHKipmu7D+LBAAsmW7Vo2PRVTNsFBeeT8PIQ==", 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDHKipmqeH+LBAAK0W4NkqlDzo32cTHJjX0Rw==", 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-osd", 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDHKipm3Qb/LBAAORM7DSBBqdStVbIq0QagsA==", 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDHKipm2DD/LBAAlLrPv0tFkpNHXOU5S7IgJg==", 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.546 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.546 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T10:06:16.546 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDHKipmMlj/LBAA0OjZvTtRALP69E4HD57R+w==", 2024-04-25T10:06:16.546 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.546 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T10:06:16.546 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.546 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.546 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T10:06:16.546 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDHKipmRoT/LBAAFDzEVJgj5yeqlg0U8wYMwg==", 2024-04-25T10:06:16.546 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.546 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T10:06:16.546 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.547 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.547 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-04-25T10:06:16.547 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmhxlLDxAAegBrJWdMNz6ZC3oXT2Qo0A==", 2024-04-25T10:06:16.547 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.547 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:06:16.547 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T10:06:16.547 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:06:16.547 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:16.547 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:16.547 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-04-25T10:06:16.547 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmpl1cDBAAGqx2kIak9NoWpkcdHczcFA==", 2024-04-25T10:06:16.547 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:06:16.547 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:06:16.548 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T10:06:16.548 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:06:16.548 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.548 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:16.548 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T10:06:16.548 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:06:16.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.527+0000 7fefddb5f640 1 -- 172.21.15.60:0/2680193928 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fefc4066f60 msgr2=0x7fefc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.527+0000 7fefddb5f640 1 --2- 172.21.15.60:0/2680193928 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fefc4066f60 0x7fefc4069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fefc8006fd0 tx=0x7fefc8008040 comp rx=0 tx=0).stop 2024-04-25T10:06:16.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.527+0000 7fefddb5f640 1 -- 172.21.15.60:0/2680193928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd8152aa0 msgr2=0x7fefd8156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.527+0000 7fefddb5f640 1 --2- 172.21.15.60:0/2680193928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd8152aa0 0x7fefd8156660 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fefd000c780 tx=0x7fefd000cc40 comp rx=0 tx=0).stop 2024-04-25T10:06:16.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.527+0000 7fefddb5f640 1 -- 172.21.15.60:0/2680193928 shutdown_connections 2024-04-25T10:06:16.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.527+0000 7fefddb5f640 1 --2- 172.21.15.60:0/2680193928 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fefc4066f60 0x7fefc4069420 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.527+0000 7fefddb5f640 1 --2- 172.21.15.60:0/2680193928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd8152aa0 0x7fefd8156660 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.527+0000 7fefddb5f640 1 --2- 172.21.15.60:0/2680193928 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fefd815bc90 0x7fefd8137fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.527+0000 7fefddb5f640 1 --2- 172.21.15.60:0/2680193928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefd80ce7a0 0x7fefd81542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.527+0000 7fefddb5f640 1 -- 172.21.15.60:0/2680193928 >> 172.21.15.60:0/2680193928 conn(0x7fefd80b12d0 msgr2=0x7fefd81579d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:16.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.527+0000 7fefddb5f640 1 -- 172.21.15.60:0/2680193928 shutdown_connections 2024-04-25T10:06:16.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.527+0000 7fefddb5f640 1 -- 172.21.15.60:0/2680193928 wait complete. 2024-04-25T10:06:16.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T10:06:16.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f90216ac640 1 Processor -- start 2024-04-25T10:06:16.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f90216ac640 1 -- start start 2024-04-25T10:06:16.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f90216ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f901c1620b0 0x7f901c0b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f90216ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901c0b5620 con 0x7f901c0b0dc0 2024-04-25T10:06:16.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f90216ac640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901c0b5900 con 0x7f901c06ac10 2024-04-25T10:06:16.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f90216ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0b3840 con 0x7f901c1620b0 2024-04-25T10:06:16.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f901b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f901c0b0dc0 0x7f901c0ae6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42658/0 (socket says 172.21.15.60:42658) 2024-04-25T10:06:16.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f901b7fe640 1 -- 172.21.15.60:0/709766200 learned_addr learned my addr 172.21.15.60:0/709766200 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:16.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f901affd640 1 --2- 172.21.15.60:0/709766200 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f901c1620b0 0x7f901c0b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f9019ffb640 1 -- 172.21.15.60:0/709766200 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1047533715 0 0) 0x7f901c0b5620 con 0x7f901c0b0dc0 2024-04-25T10:06:16.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f9019ffb640 1 -- 172.21.15.60:0/709766200 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9008003530 con 0x7f901c0b0dc0 2024-04-25T10:06:16.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f9019ffb640 1 -- 172.21.15.60:0/709766200 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2673842168 0 0) 0x7f9008003530 con 0x7f901c0b0dc0 2024-04-25T10:06:16.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f9019ffb640 1 -- 172.21.15.60:0/709766200 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f901c1620b0 msgr2=0x7f901c0b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:16.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f9019ffb640 1 --2- 172.21.15.60:0/709766200 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f901c1620b0 0x7f901c0b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f9019ffb640 1 -- 172.21.15.60:0/709766200 >> v1:172.21.15.100:6789/0 conn(0x7f901c06ac10 legacy=0x7f901c0adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f9019ffb640 1 -- 172.21.15.60:0/709766200 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c0b1230 con 0x7f901c0b0dc0 2024-04-25T10:06:16.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f9019ffb640 1 -- 172.21.15.60:0/709766200 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f9014001680 con 0x7f901c0b0dc0 2024-04-25T10:06:16.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f9019ffb640 1 -- 172.21.15.60:0/709766200 >> v1:172.21.15.60:6789/0 conn(0x7f901c0b0dc0 legacy=0x7f901c0ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f9019ffb640 1 --2- 172.21.15.60:0/709766200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90080039f0 0x7f9008003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f9019ffb640 1 --2- 172.21.15.60:0/709766200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9008004810 0x7f9008004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f9019ffb640 1 --2- 172.21.15.60:0/709766200 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f901c1620b0 0x7f90080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f9019ffb640 1 -- 172.21.15.60:0/709766200 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0b3840 con 0x7f90080039f0 2024-04-25T10:06:16.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f9019ffb640 1 -- 172.21.15.60:0/709766200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900800ba80 con 0x7f9008004810 2024-04-25T10:06:16.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f9019ffb640 1 -- 172.21.15.60:0/709766200 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900800bc80 con 0x7f901c1620b0 2024-04-25T10:06:16.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f901affd640 1 --2- 172.21.15.60:0/709766200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90080039f0 0x7f9008003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f901affd640 1 -- 172.21.15.60:0/709766200 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f901c1620b0 msgr2=0x7f90080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:16.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f901affd640 1 --2- 172.21.15.60:0/709766200 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f901c1620b0 0x7f90080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f901affd640 1 -- 172.21.15.60:0/709766200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9008004810 msgr2=0x7f9008004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:16.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f901a7fc640 1 --2- 172.21.15.60:0/709766200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9008004810 0x7f9008004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f901affd640 1 --2- 172.21.15.60:0/709766200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9008004810 0x7f9008004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f901affd640 1 -- 172.21.15.60:0/709766200 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9008003530 con 0x7f90080039f0 2024-04-25T10:06:16.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.723+0000 7f901a7fc640 1 --2- 172.21.15.60:0/709766200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9008004810 0x7f9008004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:16.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.727+0000 7f901affd640 1 --2- 172.21.15.60:0/709766200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90080039f0 0x7f9008003e10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9004009ab0 tx=0x7f900401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca04af4acb116c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:16.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.727+0000 7f9019ffb640 1 -- 172.21.15.60:0/709766200 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90040096b0 con 0x7f90080039f0 2024-04-25T10:06:16.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.727+0000 7f9019ffb640 1 -- 172.21.15.60:0/709766200 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90040a6450 con 0x7f90080039f0 2024-04-25T10:06:16.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.727+0000 7f9019ffb640 1 -- 172.21.15.60:0/709766200 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90040a6840 con 0x7f90080039f0 2024-04-25T10:06:16.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.727+0000 7f90216ac640 1 -- 172.21.15.60:0/709766200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90080039f0 msgr2=0x7f9008003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.727+0000 7f90216ac640 1 --2- 172.21.15.60:0/709766200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90080039f0 0x7f9008003e10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9004009ab0 tx=0x7f900401ca70 comp rx=0 tx=0).stop 2024-04-25T10:06:16.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.727+0000 7f901b7fe640 1 -- 172.21.15.60:0/709766200 reap_dead start 2024-04-25T10:06:16.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.727+0000 7f90216ac640 1 -- 172.21.15.60:0/709766200 shutdown_connections 2024-04-25T10:06:16.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.727+0000 7f90216ac640 1 -- 172.21.15.60:0/709766200 >> 172.21.15.60:0/709766200 conn(0x7f901c072920 msgr2=0x7f901c099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:16.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.727+0000 7f90216ac640 1 -- 172.21.15.60:0/709766200 shutdown_connections 2024-04-25T10:06:16.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.727+0000 7f90216ac640 1 -- 172.21.15.60:0/709766200 wait complete. 2024-04-25T10:06:16.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f90216ac640 1 Processor -- start 2024-04-25T10:06:16.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f90216ac640 1 -- start start 2024-04-25T10:06:16.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f90216ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90080039f0 0x7f901c1530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f90216ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f901c15d250 0x7f901c15d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f90216ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f901c0cec50 0x7f901c0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f90216ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0aef30 con 0x7f901c15d250 2024-04-25T10:06:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f90216ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0aedb0 con 0x7f901c0cec50 2024-04-25T10:06:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f90216ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901c16e290 con 0x7f90080039f0 2024-04-25T10:06:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f901a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f901c15d250 0x7f901c15d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f901a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f901c15d250 0x7f901c15d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41122/0 (socket says 172.21.15.60:41122) 2024-04-25T10:06:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f901a7fc640 1 -- 172.21.15.60:0/3175138482 learned_addr learned my addr 172.21.15.60:0/3175138482 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f901affd640 1 --2- 172.21.15.60:0/3175138482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90080039f0 0x7f901c1530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f901b7fe640 1 --2- 172.21.15.60:0/3175138482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f901c0cec50 0x7f901c0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f901a7fc640 1 -- 172.21.15.60:0/3175138482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90080039f0 msgr2=0x7f901c1530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f901a7fc640 1 --2- 172.21.15.60:0/3175138482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90080039f0 0x7f901c1530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f901a7fc640 1 -- 172.21.15.60:0/3175138482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f901c0cec50 msgr2=0x7f901c0cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f901a7fc640 1 --2- 172.21.15.60:0/3175138482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f901c0cec50 0x7f901c0cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f901a7fc640 1 -- 172.21.15.60:0/3175138482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c1535e0 con 0x7f901c15d250 2024-04-25T10:06:16.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f901affd640 1 --2- 172.21.15.60:0/3175138482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90080039f0 0x7f901c1530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:16.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f901b7fe640 1 --2- 172.21.15.60:0/3175138482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f901c0cec50 0x7f901c0cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:16.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f901a7fc640 1 --2- 172.21.15.60:0/3175138482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f901c15d250 0x7f901c15d6d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f900c00dae0 tx=0x7f900c0020d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:16.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f90137fe640 1 -- 172.21.15.60:0/3175138482 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c003bb0 con 0x7f901c15d250 2024-04-25T10:06:16.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f90137fe640 1 -- 172.21.15.60:0/3175138482 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900c003d70 con 0x7f901c15d250 2024-04-25T10:06:16.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f90216ac640 1 -- 172.21.15.60:0/3175138482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c0cff00 con 0x7f901c15d250 2024-04-25T10:06:16.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f90137fe640 1 -- 172.21.15.60:0/3175138482 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c006da0 con 0x7f901c15d250 2024-04-25T10:06:16.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f90216ac640 1 -- 172.21.15.60:0/3175138482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c15e260 con 0x7f901c15d250 2024-04-25T10:06:16.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f90216ac640 1 -- 172.21.15.60:0/3175138482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901c0cea50 con 0x7f901c15d250 2024-04-25T10:06:16.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f90137fe640 1 -- 172.21.15.60:0/3175138482 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f900c019020 con 0x7f901c15d250 2024-04-25T10:06:16.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.731+0000 7f90137fe640 1 --2- 172.21.15.60:0/3175138482 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ffc066f60 0x7f8ffc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.735+0000 7f90137fe640 1 -- 172.21.15.60:0/3175138482 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f900c0849a0 con 0x7f901c15d250 2024-04-25T10:06:16.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.735+0000 7f901affd640 1 --2- 172.21.15.60:0/3175138482 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ffc066f60 0x7f8ffc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.735+0000 7f901affd640 1 --2- 172.21.15.60:0/3175138482 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ffc066f60 0x7f8ffc069420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f90040ad000 tx=0x7f90040046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:16.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.739+0000 7f90137fe640 1 -- 172.21.15.60:0/3175138482 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f901c0cea50 con 0x7f901c15d250 2024-04-25T10:06:16.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.951+0000 7f90216ac640 1 -- 172.21.15.60:0/3175138482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f901c06f840 con 0x7f901c15d250 2024-04-25T10:06:16.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.955+0000 7f90137fe640 1 -- 172.21.15.60:0/3175138482 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f900c055c60 con 0x7f901c15d250 2024-04-25T10:06:16.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.959+0000 7f90216ac640 1 -- 172.21.15.60:0/3175138482 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ffc066f60 msgr2=0x7f8ffc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.959+0000 7f90216ac640 1 --2- 172.21.15.60:0/3175138482 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ffc066f60 0x7f8ffc069420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f90040ad000 tx=0x7f90040046d0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.959+0000 7f90216ac640 1 -- 172.21.15.60:0/3175138482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f901c15d250 msgr2=0x7f901c15d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.959+0000 7f90216ac640 1 --2- 172.21.15.60:0/3175138482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f901c15d250 0x7f901c15d6d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f900c00dae0 tx=0x7f900c0020d0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.959+0000 7f90216ac640 1 -- 172.21.15.60:0/3175138482 shutdown_connections 2024-04-25T10:06:16.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.959+0000 7f90216ac640 1 --2- 172.21.15.60:0/3175138482 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ffc066f60 0x7f8ffc069420 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.959+0000 7f90216ac640 1 --2- 172.21.15.60:0/3175138482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f901c0cec50 0x7f901c0cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.959+0000 7f90216ac640 1 --2- 172.21.15.60:0/3175138482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f901c15d250 0x7f901c15d6d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.959+0000 7f90216ac640 1 --2- 172.21.15.60:0/3175138482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90080039f0 0x7f901c1530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.959+0000 7f90216ac640 1 -- 172.21.15.60:0/3175138482 >> 172.21.15.60:0/3175138482 conn(0x7f901c072920 msgr2=0x7f901c06fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:16.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.959+0000 7f90216ac640 1 -- 172.21.15.60:0/3175138482 shutdown_connections 2024-04-25T10:06:16.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:16.959+0000 7f90216ac640 1 -- 172.21.15.60:0/3175138482 wait complete. 2024-04-25T10:06:16.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T10:06:17.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.099+0000 7fe41e7e3640 1 Processor -- start 2024-04-25T10:06:17.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.099+0000 7fe41e7e3640 1 -- start start 2024-04-25T10:06:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.099+0000 7fe41e7e3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe41815dc10 0x7fe4181689d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.099+0000 7fe41e7e3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe418162070 con 0x7fe4180cee00 2024-04-25T10:06:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.099+0000 7fe41e7e3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe418162350 con 0x7fe4180cea20 2024-04-25T10:06:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.099+0000 7fe41e7e3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4180d1cb0 con 0x7fe41815dc10 2024-04-25T10:06:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.099+0000 7fe41cd80640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe41815dc10 0x7fe4181689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.099+0000 7fe41cd80640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe41815dc10 0x7fe4181689d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41138/0 (socket says 172.21.15.60:41138) 2024-04-25T10:06:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.099+0000 7fe4177fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe4180cee00 0x7fe41815d500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42666/0 (socket says 172.21.15.60:42666) 2024-04-25T10:06:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.099+0000 7fe41cd80640 1 -- 172.21.15.60:0/2591205276 learned_addr learned my addr 172.21.15.60:0/2591205276 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.099+0000 7fe41cd80640 1 -- 172.21.15.60:0/2591205276 >> v1:172.21.15.100:6789/0 conn(0x7fe4180cea20 legacy=0x7fe418161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:17.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.099+0000 7fe41cd80640 1 -- 172.21.15.60:0/2591205276 >> v1:172.21.15.60:6789/0 conn(0x7fe4180cee00 legacy=0x7fe41815d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.099+0000 7fe416ffd640 1 -- 172.21.15.60:0/2591205276 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298272034 0 0) 0x7fe418162070 con 0x7fe4180cee00 2024-04-25T10:06:17.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.099+0000 7fe41cd80640 1 -- 172.21.15.60:0/2591205276 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4180d1f90 con 0x7fe41815dc10 2024-04-25T10:06:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.103+0000 7fe41cd80640 1 --2- 172.21.15.60:0/2591205276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe41815dc10 0x7fe4181689d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe410009e50 tx=0x7fe41001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57daf88e727e2568 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.103+0000 7fe416ffd640 1 -- 172.21.15.60:0/2591205276 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4100a5810 con 0x7fe41815dc10 2024-04-25T10:06:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.103+0000 7fe416ffd640 1 -- 172.21.15.60:0/2591205276 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4100a59b0 con 0x7fe41815dc10 2024-04-25T10:06:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.103+0000 7fe416ffd640 1 -- 172.21.15.60:0/2591205276 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4100a5d80 con 0x7fe41815dc10 2024-04-25T10:06:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.103+0000 7fe41e7e3640 1 -- 172.21.15.60:0/2591205276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe41815dc10 msgr2=0x7fe4181689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.103+0000 7fe41e7e3640 1 --2- 172.21.15.60:0/2591205276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe41815dc10 0x7fe4181689d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe410009e50 tx=0x7fe41001ca30 comp rx=0 tx=0).stop 2024-04-25T10:06:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.103+0000 7fe41e7e3640 1 -- 172.21.15.60:0/2591205276 shutdown_connections 2024-04-25T10:06:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.103+0000 7fe41e7e3640 1 --2- 172.21.15.60:0/2591205276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe41815dc10 0x7fe4181689d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.103+0000 7fe41e7e3640 1 -- 172.21.15.60:0/2591205276 >> 172.21.15.60:0/2591205276 conn(0x7fe418072340 msgr2=0x7fe418072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.103+0000 7fe41e7e3640 1 -- 172.21.15.60:0/2591205276 shutdown_connections 2024-04-25T10:06:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.103+0000 7fe41e7e3640 1 -- 172.21.15.60:0/2591205276 wait complete. 2024-04-25T10:06:17.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.103+0000 7fe41e7e3640 1 Processor -- start 2024-04-25T10:06:17.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.103+0000 7fe41e7e3640 1 -- start start 2024-04-25T10:06:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe41e7e3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4180cea20 0x7fe4181519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe41e7e3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4180cee00 0x7fe41815f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe41e7e3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41815dc10 0x7fe41815e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe41e7e3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41816bac0 con 0x7fe4180cea20 2024-04-25T10:06:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe41e7e3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41816b940 con 0x7fe41815dc10 2024-04-25T10:06:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe41e7e3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41816bc40 con 0x7fe4180cee00 2024-04-25T10:06:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe417fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4180cea20 0x7fe4181519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe417fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4180cea20 0x7fe4181519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41146/0 (socket says 172.21.15.60:41146) 2024-04-25T10:06:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe417fff640 1 -- 172.21.15.60:0/3721560837 learned_addr learned my addr 172.21.15.60:0/3721560837 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe4177fe640 1 --2- 172.21.15.60:0/3721560837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4180cee00 0x7fe41815f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe417fff640 1 -- 172.21.15.60:0/3721560837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4180cee00 msgr2=0x7fe41815f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe41cd80640 1 --2- 172.21.15.60:0/3721560837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41815dc10 0x7fe41815e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe417fff640 1 --2- 172.21.15.60:0/3721560837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4180cee00 0x7fe41815f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe417fff640 1 -- 172.21.15.60:0/3721560837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41815dc10 msgr2=0x7fe41815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe417fff640 1 --2- 172.21.15.60:0/3721560837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41815dc10 0x7fe41815e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe417fff640 1 -- 172.21.15.60:0/3721560837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41815e8d0 con 0x7fe4180cea20 2024-04-25T10:06:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe417fff640 1 --2- 172.21.15.60:0/3721560837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4180cea20 0x7fe4181519d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe40800c530 tx=0x7fe40800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe414ff9640 1 -- 172.21.15.60:0/3721560837 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408016020 con 0x7fe4180cea20 2024-04-25T10:06:17.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe414ff9640 1 -- 172.21.15.60:0/3721560837 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe408005220 con 0x7fe4180cea20 2024-04-25T10:06:17.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe41e7e3640 1 -- 172.21.15.60:0/3721560837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe41815ebb0 con 0x7fe4180cea20 2024-04-25T10:06:17.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe414ff9640 1 -- 172.21.15.60:0/3721560837 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408007030 con 0x7fe4180cea20 2024-04-25T10:06:17.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe41e7e3640 1 -- 172.21.15.60:0/3721560837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe41815f170 con 0x7fe4180cea20 2024-04-25T10:06:17.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe41e7e3640 1 -- 172.21.15.60:0/3721560837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe418074770 con 0x7fe4180cea20 2024-04-25T10:06:17.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe414ff9640 1 -- 172.21.15.60:0/3721560837 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe4080071d0 con 0x7fe4180cea20 2024-04-25T10:06:17.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe414ff9640 1 --2- 172.21.15.60:0/3721560837 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe404066f60 0x7fe404069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe414ff9640 1 -- 172.21.15.60:0/3721560837 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe408084710 con 0x7fe4180cea20 2024-04-25T10:06:17.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.107+0000 7fe4177fe640 1 --2- 172.21.15.60:0/3721560837 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe404066f60 0x7fe404069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.111+0000 7fe4177fe640 1 --2- 172.21.15.60:0/3721560837 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe404066f60 0x7fe404069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe40c000f70 tx=0x7fe40c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.111+0000 7fe414ff9640 1 -- 172.21.15.60:0/3721560837 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe408020020 con 0x7fe4180cea20 2024-04-25T10:06:17.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.335+0000 7fe41e7e3640 1 -- 172.21.15.60:0/3721560837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fe4180d16c0 con 0x7fe4180cea20 2024-04-25T10:06:17.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.335+0000 7fe414ff9640 1 -- 172.21.15.60:0/3721560837 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe4080074c0 con 0x7fe4180cea20 2024-04-25T10:06:17.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.339+0000 7fe41e7e3640 1 -- 172.21.15.60:0/3721560837 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe404066f60 msgr2=0x7fe404069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.339+0000 7fe41e7e3640 1 --2- 172.21.15.60:0/3721560837 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe404066f60 0x7fe404069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe40c000f70 tx=0x7fe40c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:17.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.339+0000 7fe41e7e3640 1 -- 172.21.15.60:0/3721560837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4180cea20 msgr2=0x7fe4181519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.339+0000 7fe41e7e3640 1 --2- 172.21.15.60:0/3721560837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4180cea20 0x7fe4181519d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe40800c530 tx=0x7fe40800c9f0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.339+0000 7fe41e7e3640 1 -- 172.21.15.60:0/3721560837 shutdown_connections 2024-04-25T10:06:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.339+0000 7fe41e7e3640 1 --2- 172.21.15.60:0/3721560837 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe404066f60 0x7fe404069420 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.339+0000 7fe41e7e3640 1 --2- 172.21.15.60:0/3721560837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41815dc10 0x7fe41815e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.339+0000 7fe41e7e3640 1 --2- 172.21.15.60:0/3721560837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4180cee00 0x7fe41815f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.339+0000 7fe41e7e3640 1 --2- 172.21.15.60:0/3721560837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4180cea20 0x7fe4181519d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.339+0000 7fe41e7e3640 1 -- 172.21.15.60:0/3721560837 >> 172.21.15.60:0/3721560837 conn(0x7fe418072340 msgr2=0x7fe4180b8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.339+0000 7fe41e7e3640 1 -- 172.21.15.60:0/3721560837 shutdown_connections 2024-04-25T10:06:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.339+0000 7fe41e7e3640 1 -- 172.21.15.60:0/3721560837 wait complete. 2024-04-25T10:06:17.362 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T10:06:17.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.479+0000 7f6e275b3640 1 Processor -- start 2024-04-25T10:06:17.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.479+0000 7f6e275b3640 1 -- start start 2024-04-25T10:06:17.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.479+0000 7f6e275b3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e200ce7a0 0x7f6e200ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.479+0000 7f6e275b3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6e200b4660 0x7f6e200b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.479+0000 7f6e275b3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e200cf0c0 con 0x7f6e2015bc90 2024-04-25T10:06:17.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.479+0000 7f6e275b3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e200cc370 con 0x7f6e200b4660 2024-04-25T10:06:17.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.479+0000 7f6e275b3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e20071ca0 con 0x7f6e200ce7a0 2024-04-25T10:06:17.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.479+0000 7f6e25b50640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6e200b4660 0x7f6e200b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.479+0000 7f6e24b4e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6e2015bc90 0x7f6e200b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42672/0 (socket says 172.21.15.60:42672) 2024-04-25T10:06:17.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.479+0000 7f6e24b4e640 1 -- 172.21.15.60:0/2790796027 learned_addr learned my addr 172.21.15.60:0/2790796027 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:17.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.479+0000 7f6e25b50640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6e200b4660 0x7f6e200b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41162/0 (socket says 172.21.15.60:41162) 2024-04-25T10:06:17.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.479+0000 7f6e2534f640 1 --2- 172.21.15.60:0/2790796027 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e200ce7a0 0x7f6e200ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.483+0000 7f6e25b50640 1 -- 172.21.15.60:0/2790796027 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e200ce7a0 msgr2=0x7f6e200ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:17.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.483+0000 7f6e25b50640 1 --2- 172.21.15.60:0/2790796027 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e200ce7a0 0x7f6e200ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.483+0000 7f6e25b50640 1 -- 172.21.15.60:0/2790796027 >> v1:172.21.15.60:6789/0 conn(0x7f6e2015bc90 legacy=0x7f6e200b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.483+0000 7f6e25b50640 1 -- 172.21.15.60:0/2790796027 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e200cca40 con 0x7f6e200b4660 2024-04-25T10:06:17.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.483+0000 7f6e25b50640 1 --2- 172.21.15.60:0/2790796027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e200b4660 0x7f6e200b2ce0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009dc0 tx=0x7f6e1801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9431fcd66b02520 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.483+0000 7f6e17fff640 1 -- 172.21.15.60:0/2790796027 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e180b2070 con 0x7f6e200b4660 2024-04-25T10:06:17.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.483+0000 7f6e17fff640 1 -- 172.21.15.60:0/2790796027 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e180a5810 con 0x7f6e200b4660 2024-04-25T10:06:17.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.483+0000 7f6e17fff640 1 -- 172.21.15.60:0/2790796027 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e180a5be0 con 0x7f6e200b4660 2024-04-25T10:06:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.483+0000 7f6e275b3640 1 -- 172.21.15.60:0/2790796027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e200b4660 msgr2=0x7f6e200b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.483+0000 7f6e275b3640 1 --2- 172.21.15.60:0/2790796027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e200b4660 0x7f6e200b2ce0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009dc0 tx=0x7f6e1801c950 comp rx=0 tx=0).stop 2024-04-25T10:06:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.483+0000 7f6e275b3640 1 -- 172.21.15.60:0/2790796027 shutdown_connections 2024-04-25T10:06:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.483+0000 7f6e275b3640 1 --2- 172.21.15.60:0/2790796027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e200b4660 0x7f6e200b2ce0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.483+0000 7f6e275b3640 1 --2- 172.21.15.60:0/2790796027 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e200ce7a0 0x7f6e200ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.483+0000 7f6e275b3640 1 -- 172.21.15.60:0/2790796027 >> 172.21.15.60:0/2790796027 conn(0x7f6e200b12d0 msgr2=0x7f6e200b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:17.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.483+0000 7f6e275b3640 1 -- 172.21.15.60:0/2790796027 shutdown_connections 2024-04-25T10:06:17.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.483+0000 7f6e275b3640 1 -- 172.21.15.60:0/2790796027 wait complete. 2024-04-25T10:06:17.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e275b3640 1 Processor -- start 2024-04-25T10:06:17.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e275b3640 1 -- start start 2024-04-25T10:06:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e275b3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e200ce7a0 0x7f6e2015d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e275b3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e2015bc90 0x7f6e20156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e275b3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20156b40 0x7f6e201f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e275b3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e200b4280 con 0x7f6e20156b40 2024-04-25T10:06:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e275b3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e200b3db0 con 0x7f6e200ce7a0 2024-04-25T10:06:17.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e275b3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e200b4400 con 0x7f6e2015bc90 2024-04-25T10:06:17.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e25b50640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20156b40 0x7f6e201f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e25b50640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20156b40 0x7f6e201f4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41174/0 (socket says 172.21.15.60:41174) 2024-04-25T10:06:17.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e25b50640 1 -- 172.21.15.60:0/4069777191 learned_addr learned my addr 172.21.15.60:0/4069777191 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:17.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e24b4e640 1 --2- 172.21.15.60:0/4069777191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e2015bc90 0x7f6e20156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e2534f640 1 --2- 172.21.15.60:0/4069777191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e200ce7a0 0x7f6e2015d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e25b50640 1 -- 172.21.15.60:0/4069777191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e2015bc90 msgr2=0x7f6e20156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e25b50640 1 --2- 172.21.15.60:0/4069777191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e2015bc90 0x7f6e20156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e25b50640 1 -- 172.21.15.60:0/4069777191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e200ce7a0 msgr2=0x7f6e2015d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e25b50640 1 --2- 172.21.15.60:0/4069777191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e200ce7a0 0x7f6e2015d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e25b50640 1 -- 172.21.15.60:0/4069777191 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e20157000 con 0x7f6e20156b40 2024-04-25T10:06:17.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e25b50640 1 --2- 172.21.15.60:0/4069777191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20156b40 0x7f6e201f4e90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6e1801a580 tx=0x7f6e180196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e15ffb640 1 -- 172.21.15.60:0/4069777191 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e180b2050 con 0x7f6e20156b40 2024-04-25T10:06:17.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e15ffb640 1 -- 172.21.15.60:0/4069777191 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e180be070 con 0x7f6e20156b40 2024-04-25T10:06:17.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e15ffb640 1 -- 172.21.15.60:0/4069777191 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e180ada90 con 0x7f6e20156b40 2024-04-25T10:06:17.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e275b3640 1 -- 172.21.15.60:0/4069777191 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e201572e0 con 0x7f6e20156b40 2024-04-25T10:06:17.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e275b3640 1 -- 172.21.15.60:0/4069777191 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e201596c0 con 0x7f6e20156b40 2024-04-25T10:06:17.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e275b3640 1 -- 172.21.15.60:0/4069777191 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e20074770 con 0x7f6e20156b40 2024-04-25T10:06:17.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e15ffb640 1 -- 172.21.15.60:0/4069777191 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e180a56d0 con 0x7f6e20156b40 2024-04-25T10:06:17.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e15ffb640 1 --2- 172.21.15.60:0/4069777191 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6e04066f60 0x7f6e04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e15ffb640 1 -- 172.21.15.60:0/4069777191 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6e1811fcc0 con 0x7f6e20156b40 2024-04-25T10:06:17.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.487+0000 7f6e2534f640 1 --2- 172.21.15.60:0/4069777191 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6e04066f60 0x7f6e04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.491+0000 7f6e2534f640 1 --2- 172.21.15.60:0/4069777191 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6e04066f60 0x7f6e04069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0096f0 tx=0x7f6e1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.491+0000 7f6e15ffb640 1 -- 172.21.15.60:0/4069777191 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e18016540 con 0x7f6e20156b40 2024-04-25T10:06:17.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.699+0000 7f6e275b3640 1 -- 172.21.15.60:0/4069777191 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f6e200afdf0 con 0x7f6e20156b40 2024-04-25T10:06:17.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.703+0000 7f6e15ffb640 1 -- 172.21.15.60:0/4069777191 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6e180ab340 con 0x7f6e20156b40 2024-04-25T10:06:17.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.707+0000 7f6e275b3640 1 -- 172.21.15.60:0/4069777191 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6e04066f60 msgr2=0x7f6e04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.707+0000 7f6e275b3640 1 --2- 172.21.15.60:0/4069777191 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6e04066f60 0x7f6e04069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0096f0 tx=0x7f6e1c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:17.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.707+0000 7f6e275b3640 1 -- 172.21.15.60:0/4069777191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20156b40 msgr2=0x7f6e201f4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.707+0000 7f6e275b3640 1 --2- 172.21.15.60:0/4069777191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20156b40 0x7f6e201f4e90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6e1801a580 tx=0x7f6e180196f0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.707+0000 7f6e275b3640 1 -- 172.21.15.60:0/4069777191 shutdown_connections 2024-04-25T10:06:17.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.707+0000 7f6e275b3640 1 --2- 172.21.15.60:0/4069777191 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6e04066f60 0x7f6e04069420 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.707+0000 7f6e275b3640 1 --2- 172.21.15.60:0/4069777191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e20156b40 0x7f6e201f4e90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.707+0000 7f6e275b3640 1 --2- 172.21.15.60:0/4069777191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e2015bc90 0x7f6e20156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.707+0000 7f6e275b3640 1 --2- 172.21.15.60:0/4069777191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e200ce7a0 0x7f6e2015d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.707+0000 7f6e275b3640 1 -- 172.21.15.60:0/4069777191 >> 172.21.15.60:0/4069777191 conn(0x7f6e200b12d0 msgr2=0x7f6e2015c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:17.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.707+0000 7f6e275b3640 1 -- 172.21.15.60:0/4069777191 shutdown_connections 2024-04-25T10:06:17.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.707+0000 7f6e275b3640 1 -- 172.21.15.60:0/4069777191 wait complete. 2024-04-25T10:06:17.728 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T10:06:17.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.843+0000 7f2d7386a640 1 Processor -- start 2024-04-25T10:06:17.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.843+0000 7f2d7386a640 1 -- start start 2024-04-25T10:06:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d7386a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6c0ce7a0 0x7f2d6c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d7386a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2d6c0b4660 0x7f2d6c0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d7386a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c0cf0c0 con 0x7f2d6c15bc90 2024-04-25T10:06:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d7386a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6c0cc370 con 0x7f2d6c0b4660 2024-04-25T10:06:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d7386a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c071ca0 con 0x7f2d6c0ce7a0 2024-04-25T10:06:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d70e05640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2d6c15bc90 0x7f2d6c0b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42682/0 (socket says 172.21.15.60:42682) 2024-04-25T10:06:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d70e05640 1 -- 172.21.15.60:0/1883441960 learned_addr learned my addr 172.21.15.60:0/1883441960 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d71606640 1 --2- 172.21.15.60:0/1883441960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6c0ce7a0 0x7f2d6c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d71e07640 1 --2- 172.21.15.60:0/1883441960 >> v2:172.21.15.100:3300/0 conn(0x7f2d6c0b4660 0x7f2d6c0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d63fff640 1 -- 172.21.15.60:0/1883441960 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212607527 0 0) 0x7f2d6c0cf0c0 con 0x7f2d6c15bc90 2024-04-25T10:06:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d63fff640 1 -- 172.21.15.60:0/1883441960 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d58003530 con 0x7f2d6c15bc90 2024-04-25T10:06:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d63fff640 1 -- 172.21.15.60:0/1883441960 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1472908400 0 0) 0x7f2d58003530 con 0x7f2d6c15bc90 2024-04-25T10:06:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d63fff640 1 -- 172.21.15.60:0/1883441960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6c0ce7a0 msgr2=0x7f2d6c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d63fff640 1 --2- 172.21.15.60:0/1883441960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6c0ce7a0 0x7f2d6c0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d63fff640 1 -- 172.21.15.60:0/1883441960 >> v2:172.21.15.100:3300/0 conn(0x7f2d6c0b4660 msgr2=0x7f2d6c0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d63fff640 1 --2- 172.21.15.60:0/1883441960 >> v2:172.21.15.100:3300/0 conn(0x7f2d6c0b4660 0x7f2d6c0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d63fff640 1 -- 172.21.15.60:0/1883441960 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0cca40 con 0x7f2d6c15bc90 2024-04-25T10:06:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d63fff640 1 -- 172.21.15.60:0/1883441960 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f2d5c001680 con 0x7f2d6c15bc90 2024-04-25T10:06:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d63fff640 1 -- 172.21.15.60:0/1883441960 >> v1:172.21.15.60:6789/0 conn(0x7f2d6c15bc90 legacy=0x7f2d6c0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d63fff640 1 --2- 172.21.15.60:0/1883441960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d580039c0 0x7f2d58003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d63fff640 1 --2- 172.21.15.60:0/1883441960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6c0ce7a0 0x7f2d58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d63fff640 1 --2- 172.21.15.60:0/1883441960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d58004d50 0x7f2d580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d63fff640 1 -- 172.21.15.60:0/1883441960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c071ca0 con 0x7f2d58004d50 2024-04-25T10:06:17.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d63fff640 1 -- 172.21.15.60:0/1883441960 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5800b9d0 con 0x7f2d580039c0 2024-04-25T10:06:17.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d63fff640 1 -- 172.21.15.60:0/1883441960 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5800bb50 con 0x7f2d6c0ce7a0 2024-04-25T10:06:17.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d71e07640 1 --2- 172.21.15.60:0/1883441960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d58004d50 0x7f2d580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d70e05640 1 --2- 172.21.15.60:0/1883441960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6c0ce7a0 0x7f2d58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d71e07640 1 -- 172.21.15.60:0/1883441960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6c0ce7a0 msgr2=0x7f2d58004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:17.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d71e07640 1 --2- 172.21.15.60:0/1883441960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6c0ce7a0 0x7f2d58004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d71e07640 1 -- 172.21.15.60:0/1883441960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d580039c0 msgr2=0x7f2d58003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:17.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d71e07640 1 --2- 172.21.15.60:0/1883441960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d580039c0 0x7f2d58003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.847+0000 7f2d71e07640 1 -- 172.21.15.60:0/1883441960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d58003530 con 0x7f2d58004d50 2024-04-25T10:06:17.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d71e07640 1 --2- 172.21.15.60:0/1883441960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d58004d50 0x7f2d580051d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c06cf70 tx=0x7f2d6401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c6c7bd6bad7a6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d63fff640 1 -- 172.21.15.60:0/1883441960 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d640b2070 con 0x7f2d58004d50 2024-04-25T10:06:17.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d63fff640 1 -- 172.21.15.60:0/1883441960 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d640a5c70 con 0x7f2d58004d50 2024-04-25T10:06:17.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d63fff640 1 -- 172.21.15.60:0/1883441960 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64009c10 con 0x7f2d58004d50 2024-04-25T10:06:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d7386a640 1 -- 172.21.15.60:0/1883441960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d58004d50 msgr2=0x7f2d580051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d7386a640 1 --2- 172.21.15.60:0/1883441960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d58004d50 0x7f2d580051d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c06cf70 tx=0x7f2d6401cc60 comp rx=0 tx=0).stop 2024-04-25T10:06:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d71e07640 1 -- 172.21.15.60:0/1883441960 reap_dead start 2024-04-25T10:06:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d7386a640 1 -- 172.21.15.60:0/1883441960 shutdown_connections 2024-04-25T10:06:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d7386a640 1 -- 172.21.15.60:0/1883441960 >> 172.21.15.60:0/1883441960 conn(0x7f2d6c0b12d0 msgr2=0x7f2d6c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d7386a640 1 -- 172.21.15.60:0/1883441960 shutdown_connections 2024-04-25T10:06:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d7386a640 1 -- 172.21.15.60:0/1883441960 wait complete. 2024-04-25T10:06:17.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d7386a640 1 Processor -- start 2024-04-25T10:06:17.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d7386a640 1 -- start start 2024-04-25T10:06:17.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d7386a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d58004d50 0x7f2d6c1f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d7386a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6c0afc30 0x7f2d6c151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d7386a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d6c0b00b0 0x7f2d6c152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d7386a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0b3ba0 con 0x7f2d6c0b00b0 2024-04-25T10:06:17.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d7386a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c071f80 con 0x7f2d58004d50 2024-04-25T10:06:17.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d7386a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0b3d20 con 0x7f2d6c0afc30 2024-04-25T10:06:17.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d71e07640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d6c0b00b0 0x7f2d6c152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d71e07640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d6c0b00b0 0x7f2d6c152420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41200/0 (socket says 172.21.15.60:41200) 2024-04-25T10:06:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d71e07640 1 -- 172.21.15.60:0/3243203342 learned_addr learned my addr 172.21.15.60:0/3243203342 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d71606640 1 --2- 172.21.15.60:0/3243203342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d58004d50 0x7f2d6c1f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d70e05640 1 --2- 172.21.15.60:0/3243203342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6c0afc30 0x7f2d6c151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d71e07640 1 -- 172.21.15.60:0/3243203342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6c0afc30 msgr2=0x7f2d6c151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d71e07640 1 --2- 172.21.15.60:0/3243203342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6c0afc30 0x7f2d6c151ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d71e07640 1 -- 172.21.15.60:0/3243203342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d58004d50 msgr2=0x7f2d6c1f6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d71e07640 1 --2- 172.21.15.60:0/3243203342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d58004d50 0x7f2d6c1f6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d71e07640 1 -- 172.21.15.60:0/3243203342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c15aac0 con 0x7f2d6c0b00b0 2024-04-25T10:06:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d71606640 1 --2- 172.21.15.60:0/3243203342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d58004d50 0x7f2d6c1f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d71e07640 1 --2- 172.21.15.60:0/3243203342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d6c0b00b0 0x7f2d6c152420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2d640b0040 tx=0x7f2d64009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d61ffb640 1 -- 172.21.15.60:0/3243203342 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d640b2040 con 0x7f2d6c0b00b0 2024-04-25T10:06:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d61ffb640 1 -- 172.21.15.60:0/3243203342 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d640bc070 con 0x7f2d6c0b00b0 2024-04-25T10:06:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d7386a640 1 -- 172.21.15.60:0/3243203342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c1567d0 con 0x7f2d6c0b00b0 2024-04-25T10:06:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d61ffb640 1 -- 172.21.15.60:0/3243203342 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64009c10 con 0x7f2d6c0b00b0 2024-04-25T10:06:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.851+0000 7f2d7386a640 1 -- 172.21.15.60:0/3243203342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c155050 con 0x7f2d6c0b00b0 2024-04-25T10:06:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.855+0000 7f2d7386a640 1 -- 172.21.15.60:0/3243203342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c1569a0 con 0x7f2d6c0b00b0 2024-04-25T10:06:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.855+0000 7f2d61ffb640 1 -- 172.21.15.60:0/3243203342 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d640aeb40 con 0x7f2d6c0b00b0 2024-04-25T10:06:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.855+0000 7f2d61ffb640 1 --2- 172.21.15.60:0/3243203342 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d50066f60 0x7f2d50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.855+0000 7f2d61ffb640 1 -- 172.21.15.60:0/3243203342 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2d6411f7b0 con 0x7f2d6c0b00b0 2024-04-25T10:06:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.855+0000 7f2d71606640 1 --2- 172.21.15.60:0/3243203342 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d50066f60 0x7f2d50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.855+0000 7f2d71606640 1 --2- 172.21.15.60:0/3243203342 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d50066f60 0x7f2d50069420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2d680047c0 tx=0x7f2d68008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:17.859+0000 7f2d61ffb640 1 -- 172.21.15.60:0/3243203342 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d640a5e20 con 0x7f2d6c0b00b0 2024-04-25T10:06:18.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.067+0000 7f2d7386a640 1 -- 172.21.15.60:0/3243203342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f2d6c074770 con 0x7f2d6c0b00b0 2024-04-25T10:06:18.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.071+0000 7f2d61ffb640 1 -- 172.21.15.60:0/3243203342 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2d64016970 con 0x7f2d6c0b00b0 2024-04-25T10:06:18.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.075+0000 7f2d7386a640 1 -- 172.21.15.60:0/3243203342 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d50066f60 msgr2=0x7f2d50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.075+0000 7f2d7386a640 1 --2- 172.21.15.60:0/3243203342 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d50066f60 0x7f2d50069420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2d680047c0 tx=0x7f2d68008040 comp rx=0 tx=0).stop 2024-04-25T10:06:18.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.075+0000 7f2d7386a640 1 -- 172.21.15.60:0/3243203342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d6c0b00b0 msgr2=0x7f2d6c152420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.075+0000 7f2d7386a640 1 --2- 172.21.15.60:0/3243203342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d6c0b00b0 0x7f2d6c152420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2d640b0040 tx=0x7f2d64009750 comp rx=0 tx=0).stop 2024-04-25T10:06:18.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.075+0000 7f2d7386a640 1 -- 172.21.15.60:0/3243203342 shutdown_connections 2024-04-25T10:06:18.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.075+0000 7f2d7386a640 1 --2- 172.21.15.60:0/3243203342 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d50066f60 0x7f2d50069420 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.075+0000 7f2d7386a640 1 --2- 172.21.15.60:0/3243203342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d6c0b00b0 0x7f2d6c152420 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.075+0000 7f2d7386a640 1 --2- 172.21.15.60:0/3243203342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6c0afc30 0x7f2d6c151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.075+0000 7f2d7386a640 1 --2- 172.21.15.60:0/3243203342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d58004d50 0x7f2d6c1f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.075+0000 7f2d7386a640 1 -- 172.21.15.60:0/3243203342 >> 172.21.15.60:0/3243203342 conn(0x7f2d6c0b12d0 msgr2=0x7f2d6c0b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:18.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.075+0000 7f2d7386a640 1 -- 172.21.15.60:0/3243203342 shutdown_connections 2024-04-25T10:06:18.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.075+0000 7f2d7386a640 1 -- 172.21.15.60:0/3243203342 wait complete. 2024-04-25T10:06:18.097 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T10:06:18.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b576d6640 1 Processor -- start 2024-04-25T10:06:18.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b576d6640 1 -- start start 2024-04-25T10:06:18.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b576d6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8b500b37b0 0x7f8b5006af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b576d6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8b5006f750 0x7f8b5006b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b576d6640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5015e410 con 0x7f8b500af3d0 2024-04-25T10:06:18.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b576d6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b5015e6f0 con 0x7f8b5006f750 2024-04-25T10:06:18.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b576d6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b5015e9d0 con 0x7f8b500b37b0 2024-04-25T10:06:18.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b55472640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8b500af3d0 0x7f8b500b30a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42686/0 (socket says 172.21.15.60:42686) 2024-04-25T10:06:18.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b55472640 1 -- 172.21.15.60:0/3017065646 learned_addr learned my addr 172.21.15.60:0/3017065646 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:18.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b55c73640 1 --2- 172.21.15.60:0/3017065646 >> v2:172.21.15.60:3300/0 conn(0x7f8b5006f750 0x7f8b5006b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b54c71640 1 --2- 172.21.15.60:0/3017065646 >> v2:172.21.15.100:3300/0 conn(0x7f8b500b37b0 0x7f8b5006af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b47fff640 1 -- 172.21.15.60:0/3017065646 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236791324 0 0) 0x7f8b5015e410 con 0x7f8b500af3d0 2024-04-25T10:06:18.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b47fff640 1 -- 172.21.15.60:0/3017065646 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b3c003530 con 0x7f8b500af3d0 2024-04-25T10:06:18.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b55c73640 1 -- 172.21.15.60:0/3017065646 >> v2:172.21.15.100:3300/0 conn(0x7f8b500b37b0 msgr2=0x7f8b5006af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:18.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b55c73640 1 --2- 172.21.15.60:0/3017065646 >> v2:172.21.15.100:3300/0 conn(0x7f8b500b37b0 0x7f8b5006af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b55c73640 1 -- 172.21.15.60:0/3017065646 >> v1:172.21.15.60:6789/0 conn(0x7f8b500af3d0 legacy=0x7f8b500b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.215+0000 7f8b55c73640 1 -- 172.21.15.60:0/3017065646 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5015ecb0 con 0x7f8b5006f750 2024-04-25T10:06:18.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b55c73640 1 --2- 172.21.15.60:0/3017065646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b5006f750 0x7f8b5006b4a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8b5006c7e0 tx=0x7f8b4c01c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d74335079da6d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b47fff640 1 -- 172.21.15.60:0/3017065646 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c0b2070 con 0x7f8b5006f750 2024-04-25T10:06:18.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b47fff640 1 -- 172.21.15.60:0/3017065646 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4c0ad680 con 0x7f8b5006f750 2024-04-25T10:06:18.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b47fff640 1 -- 172.21.15.60:0/3017065646 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5015e1d0 con 0x7f8b5006f750 2024-04-25T10:06:18.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b576d6640 1 -- 172.21.15.60:0/3017065646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b5006f750 msgr2=0x7f8b5006b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b576d6640 1 --2- 172.21.15.60:0/3017065646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b5006f750 0x7f8b5006b4a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8b5006c7e0 tx=0x7f8b4c01c320 comp rx=0 tx=0).stop 2024-04-25T10:06:18.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b576d6640 1 -- 172.21.15.60:0/3017065646 shutdown_connections 2024-04-25T10:06:18.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b576d6640 1 --2- 172.21.15.60:0/3017065646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b5006f750 0x7f8b5006b4a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b576d6640 1 --2- 172.21.15.60:0/3017065646 >> v2:172.21.15.100:3300/0 conn(0x7f8b500b37b0 0x7f8b5006af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b576d6640 1 -- 172.21.15.60:0/3017065646 >> 172.21.15.60:0/3017065646 conn(0x7f8b500b23a0 msgr2=0x7f8b500cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:18.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b576d6640 1 -- 172.21.15.60:0/3017065646 shutdown_connections 2024-04-25T10:06:18.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b576d6640 1 -- 172.21.15.60:0/3017065646 wait complete. 2024-04-25T10:06:18.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b576d6640 1 Processor -- start 2024-04-25T10:06:18.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b576d6640 1 -- start start 2024-04-25T10:06:18.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b576d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b5006f750 0x7f8b50154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b576d6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b500af3d0 0x7f8b50153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b576d6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b500b37b0 0x7f8b5014bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b576d6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5016ace0 con 0x7f8b500b37b0 2024-04-25T10:06:18.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b576d6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5016ab60 con 0x7f8b5006f750 2024-04-25T10:06:18.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b576d6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5016ae60 con 0x7f8b500af3d0 2024-04-25T10:06:18.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b55c73640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b500b37b0 0x7f8b5014bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b55c73640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b500b37b0 0x7f8b5014bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41218/0 (socket says 172.21.15.60:41218) 2024-04-25T10:06:18.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.219+0000 7f8b55c73640 1 -- 172.21.15.60:0/2392250131 learned_addr learned my addr 172.21.15.60:0/2392250131 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:18.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b54c71640 1 --2- 172.21.15.60:0/2392250131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b500af3d0 0x7f8b50153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b55c73640 1 -- 172.21.15.60:0/2392250131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b500af3d0 msgr2=0x7f8b50153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b55472640 1 --2- 172.21.15.60:0/2392250131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b5006f750 0x7f8b50154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b55c73640 1 --2- 172.21.15.60:0/2392250131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b500af3d0 0x7f8b50153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b55c73640 1 -- 172.21.15.60:0/2392250131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b5006f750 msgr2=0x7f8b50154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b55c73640 1 --2- 172.21.15.60:0/2392250131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b5006f750 0x7f8b50154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b55c73640 1 -- 172.21.15.60:0/2392250131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b500ce7c0 con 0x7f8b500b37b0 2024-04-25T10:06:18.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b54c71640 1 --2- 172.21.15.60:0/2392250131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b500af3d0 0x7f8b50153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:18.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b55c73640 1 --2- 172.21.15.60:0/2392250131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b500b37b0 0x7f8b5014bd60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c019f50 tx=0x7f8b4c01ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b45ffb640 1 -- 172.21.15.60:0/2392250131 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c0b2040 con 0x7f8b500b37b0 2024-04-25T10:06:18.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b45ffb640 1 -- 172.21.15.60:0/2392250131 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4c0bd070 con 0x7f8b500b37b0 2024-04-25T10:06:18.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b576d6640 1 -- 172.21.15.60:0/2392250131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b500ceaa0 con 0x7f8b500b37b0 2024-04-25T10:06:18.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b45ffb640 1 -- 172.21.15.60:0/2392250131 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c0adcf0 con 0x7f8b500b37b0 2024-04-25T10:06:18.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b576d6640 1 -- 172.21.15.60:0/2392250131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5014c2a0 con 0x7f8b500b37b0 2024-04-25T10:06:18.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b576d6640 1 -- 172.21.15.60:0/2392250131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b50074770 con 0x7f8b500b37b0 2024-04-25T10:06:18.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b45ffb640 1 -- 172.21.15.60:0/2392250131 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b4c0c2460 con 0x7f8b500b37b0 2024-04-25T10:06:18.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b45ffb640 1 --2- 172.21.15.60:0/2392250131 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8b34066f60 0x7f8b34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b45ffb640 1 -- 172.21.15.60:0/2392250131 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8b4c11ef40 con 0x7f8b500b37b0 2024-04-25T10:06:18.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b55472640 1 --2- 172.21.15.60:0/2392250131 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8b34066f60 0x7f8b34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.223+0000 7f8b55472640 1 --2- 172.21.15.60:0/2392250131 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8b34066f60 0x7f8b34069420 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8b50161200 tx=0x7f8b48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.227+0000 7f8b45ffb640 1 -- 172.21.15.60:0/2392250131 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b4c016030 con 0x7f8b500b37b0 2024-04-25T10:06:18.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.443+0000 7f8b576d6640 1 -- 172.21.15.60:0/2392250131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f8b50154640 con 0x7f8b500b37b0 2024-04-25T10:06:18.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.447+0000 7f8b45ffb640 1 -- 172.21.15.60:0/2392250131 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8b4c0f02a0 con 0x7f8b500b37b0 2024-04-25T10:06:18.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.447+0000 7f8b576d6640 1 -- 172.21.15.60:0/2392250131 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8b34066f60 msgr2=0x7f8b34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.447+0000 7f8b576d6640 1 --2- 172.21.15.60:0/2392250131 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8b34066f60 0x7f8b34069420 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8b50161200 tx=0x7f8b48009040 comp rx=0 tx=0).stop 2024-04-25T10:06:18.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.447+0000 7f8b576d6640 1 -- 172.21.15.60:0/2392250131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b500b37b0 msgr2=0x7f8b5014bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.447+0000 7f8b576d6640 1 --2- 172.21.15.60:0/2392250131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b500b37b0 0x7f8b5014bd60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c019f50 tx=0x7f8b4c01ab60 comp rx=0 tx=0).stop 2024-04-25T10:06:18.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.447+0000 7f8b576d6640 1 -- 172.21.15.60:0/2392250131 shutdown_connections 2024-04-25T10:06:18.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.447+0000 7f8b576d6640 1 --2- 172.21.15.60:0/2392250131 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8b34066f60 0x7f8b34069420 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.451+0000 7f8b576d6640 1 --2- 172.21.15.60:0/2392250131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b500b37b0 0x7f8b5014bd60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.451+0000 7f8b576d6640 1 --2- 172.21.15.60:0/2392250131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b500af3d0 0x7f8b50153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.451+0000 7f8b576d6640 1 --2- 172.21.15.60:0/2392250131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b5006f750 0x7f8b50154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.451+0000 7f8b576d6640 1 -- 172.21.15.60:0/2392250131 >> 172.21.15.60:0/2392250131 conn(0x7f8b500b23a0 msgr2=0x7f8b5015f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:18.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.451+0000 7f8b576d6640 1 -- 172.21.15.60:0/2392250131 shutdown_connections 2024-04-25T10:06:18.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.451+0000 7f8b576d6640 1 -- 172.21.15.60:0/2392250131 wait complete. 2024-04-25T10:06:18.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T10:06:18.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.579+0000 7f82c7ff6640 1 Processor -- start 2024-04-25T10:06:18.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.579+0000 7f82c7ff6640 1 -- start start 2024-04-25T10:06:18.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.579+0000 7f82c7ff6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f82c00ce7a0 0x7f82c00ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.579+0000 7f82c7ff6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82c015bc90 0x7f82c00b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.579+0000 7f82c7ff6640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c00cf0c0 con 0x7f82c00b4660 2024-04-25T10:06:18.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.579+0000 7f82c7ff6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c00cc370 con 0x7f82c00ce7a0 2024-04-25T10:06:18.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.579+0000 7f82c7ff6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0071ca0 con 0x7f82c015bc90 2024-04-25T10:06:18.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.579+0000 7f82c6593640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f82c00b4660 0x7f82c00b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42698/0 (socket says 172.21.15.60:42698) 2024-04-25T10:06:18.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.579+0000 7f82c6593640 1 -- 172.21.15.60:0/2444194028 learned_addr learned my addr 172.21.15.60:0/2444194028 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:18.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c5591640 1 --2- 172.21.15.60:0/2444194028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82c015bc90 0x7f82c00b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c5d92640 1 --2- 172.21.15.60:0/2444194028 >> v2:172.21.15.100:3300/0 conn(0x7f82c00ce7a0 0x7f82c00ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 -- 172.21.15.60:0/2444194028 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721415732 0 0) 0x7f82c00cf0c0 con 0x7f82c00b4660 2024-04-25T10:06:18.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 -- 172.21.15.60:0/2444194028 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82ac003530 con 0x7f82c00b4660 2024-04-25T10:06:18.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 -- 172.21.15.60:0/2444194028 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2846193854 0 0) 0x7f82ac003530 con 0x7f82c00b4660 2024-04-25T10:06:18.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 -- 172.21.15.60:0/2444194028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82c015bc90 msgr2=0x7f82c00b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:18.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 --2- 172.21.15.60:0/2444194028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82c015bc90 0x7f82c00b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 -- 172.21.15.60:0/2444194028 >> v2:172.21.15.100:3300/0 conn(0x7f82c00ce7a0 msgr2=0x7f82c00ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:18.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 --2- 172.21.15.60:0/2444194028 >> v2:172.21.15.100:3300/0 conn(0x7f82c00ce7a0 0x7f82c00ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 -- 172.21.15.60:0/2444194028 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c00cca40 con 0x7f82c00b4660 2024-04-25T10:06:18.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 -- 172.21.15.60:0/2444194028 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f82b8001a00 con 0x7f82c00b4660 2024-04-25T10:06:18.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 -- 172.21.15.60:0/2444194028 >> v1:172.21.15.60:6789/0 conn(0x7f82c00b4660 legacy=0x7f82c00b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 --2- 172.21.15.60:0/2444194028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82ac0039c0 0x7f82ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 --2- 172.21.15.60:0/2444194028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82c015bc90 0x7f82ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 --2- 172.21.15.60:0/2444194028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82ac004d50 0x7f82ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 -- 172.21.15.60:0/2444194028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0071ca0 con 0x7f82ac004d50 2024-04-25T10:06:18.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 -- 172.21.15.60:0/2444194028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac00b9d0 con 0x7f82ac0039c0 2024-04-25T10:06:18.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 -- 172.21.15.60:0/2444194028 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac00bb50 con 0x7f82c015bc90 2024-04-25T10:06:18.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c6593640 1 --2- 172.21.15.60:0/2444194028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82ac004d50 0x7f82ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c5d92640 1 --2- 172.21.15.60:0/2444194028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82ac0039c0 0x7f82ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c5591640 1 --2- 172.21.15.60:0/2444194028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82c015bc90 0x7f82ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c6593640 1 -- 172.21.15.60:0/2444194028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82c015bc90 msgr2=0x7f82ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:18.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c6593640 1 --2- 172.21.15.60:0/2444194028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82c015bc90 0x7f82ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c6593640 1 -- 172.21.15.60:0/2444194028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82ac0039c0 msgr2=0x7f82ac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:18.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c6593640 1 --2- 172.21.15.60:0/2444194028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82ac0039c0 0x7f82ac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c6593640 1 -- 172.21.15.60:0/2444194028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ac003530 con 0x7f82ac004d50 2024-04-25T10:06:18.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c5591640 1 --2- 172.21.15.60:0/2444194028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82c015bc90 0x7f82ac004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:18.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c6593640 1 --2- 172.21.15.60:0/2444194028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82ac004d50 0x7f82ac0051d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f82b800a830 tx=0x7f82b801d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af82517a6b5ffe21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 -- 172.21.15.60:0/2444194028 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b800a480 con 0x7f82ac004d50 2024-04-25T10:06:18.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 -- 172.21.15.60:0/2444194028 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b80ad630 con 0x7f82ac004d50 2024-04-25T10:06:18.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c4d90640 1 -- 172.21.15.60:0/2444194028 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b80ada60 con 0x7f82ac004d50 2024-04-25T10:06:18.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c7ff6640 1 -- 172.21.15.60:0/2444194028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82ac004d50 msgr2=0x7f82ac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c7ff6640 1 --2- 172.21.15.60:0/2444194028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82ac004d50 0x7f82ac0051d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f82b800a830 tx=0x7f82b801d320 comp rx=0 tx=0).stop 2024-04-25T10:06:18.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c6593640 1 -- 172.21.15.60:0/2444194028 reap_dead start 2024-04-25T10:06:18.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c7ff6640 1 -- 172.21.15.60:0/2444194028 shutdown_connections 2024-04-25T10:06:18.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.583+0000 7f82c7ff6640 1 -- 172.21.15.60:0/2444194028 >> 172.21.15.60:0/2444194028 conn(0x7f82c00b12d0 msgr2=0x7f82c00b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:18.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c7ff6640 1 -- 172.21.15.60:0/2444194028 shutdown_connections 2024-04-25T10:06:18.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c7ff6640 1 -- 172.21.15.60:0/2444194028 wait complete. 2024-04-25T10:06:18.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c7ff6640 1 Processor -- start 2024-04-25T10:06:18.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c7ff6640 1 -- start start 2024-04-25T10:06:18.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c7ff6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82c0155fd0 0x7f82c01e46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c7ff6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c01563b0 0x7f82c01f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c7ff6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82c01f4d50 0x7f82c00b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c7ff6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c00b3ba0 con 0x7f82c01563b0 2024-04-25T10:06:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c7ff6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0071f80 con 0x7f82c01f4d50 2024-04-25T10:06:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c7ff6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c00b3d20 con 0x7f82c0155fd0 2024-04-25T10:06:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c5591640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c01563b0 0x7f82c01f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c5591640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c01563b0 0x7f82c01f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41240/0 (socket says 172.21.15.60:41240) 2024-04-25T10:06:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c5591640 1 -- 172.21.15.60:0/1551465715 learned_addr learned my addr 172.21.15.60:0/1551465715 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c5d92640 1 --2- 172.21.15.60:0/1551465715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82c0155fd0 0x7f82c01e46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c6593640 1 --2- 172.21.15.60:0/1551465715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82c01f4d50 0x7f82c00b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c5591640 1 -- 172.21.15.60:0/1551465715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82c0155fd0 msgr2=0x7f82c01e46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c5591640 1 --2- 172.21.15.60:0/1551465715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82c0155fd0 0x7f82c01e46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c5591640 1 -- 172.21.15.60:0/1551465715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82c01f4d50 msgr2=0x7f82c00b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c5591640 1 --2- 172.21.15.60:0/1551465715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82c01f4d50 0x7f82c00b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c5591640 1 -- 172.21.15.60:0/1551465715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c01e1a10 con 0x7f82c01563b0 2024-04-25T10:06:18.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c5591640 1 --2- 172.21.15.60:0/1551465715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c01563b0 0x7f82c01f4810 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009fd0 tx=0x7f82bc011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82b67fc640 1 -- 172.21.15.60:0/1551465715 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc004220 con 0x7f82c01563b0 2024-04-25T10:06:18.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82b67fc640 1 -- 172.21.15.60:0/1551465715 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc0043c0 con 0x7f82c01563b0 2024-04-25T10:06:18.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c7ff6640 1 -- 172.21.15.60:0/1551465715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c01e1cf0 con 0x7f82c01563b0 2024-04-25T10:06:18.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82b67fc640 1 -- 172.21.15.60:0/1551465715 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00f9e0 con 0x7f82c01563b0 2024-04-25T10:06:18.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c7ff6640 1 -- 172.21.15.60:0/1551465715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c01e22b0 con 0x7f82c01563b0 2024-04-25T10:06:18.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.587+0000 7f82c7ff6640 1 -- 172.21.15.60:0/1551465715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c0074770 con 0x7f82c01563b0 2024-04-25T10:06:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.591+0000 7f82b67fc640 1 -- 172.21.15.60:0/1551465715 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82bc00fb80 con 0x7f82c01563b0 2024-04-25T10:06:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.591+0000 7f82b67fc640 1 --2- 172.21.15.60:0/1551465715 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f82a4066f60 0x7f82a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.591+0000 7f82b67fc640 1 -- 172.21.15.60:0/1551465715 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f82bc0853b0 con 0x7f82c01563b0 2024-04-25T10:06:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.591+0000 7f82c5d92640 1 --2- 172.21.15.60:0/1551465715 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f82a4066f60 0x7f82a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.591+0000 7f82c5d92640 1 --2- 172.21.15.60:0/1551465715 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f82a4066f60 0x7f82a4069420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f82b00099b0 tx=0x7f82b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.595+0000 7f82b67fc640 1 -- 172.21.15.60:0/1551465715 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f82bc019030 con 0x7f82c01563b0 2024-04-25T10:06:18.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.811+0000 7f82c7ff6640 1 -- 172.21.15.60:0/1551465715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f82c006f1d0 con 0x7f82c01563b0 2024-04-25T10:06:18.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.815+0000 7f82b67fc640 1 -- 172.21.15.60:0/1551465715 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f82bc056710 con 0x7f82c01563b0 2024-04-25T10:06:18.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.819+0000 7f82c7ff6640 1 -- 172.21.15.60:0/1551465715 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f82a4066f60 msgr2=0x7f82a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.819+0000 7f82c7ff6640 1 --2- 172.21.15.60:0/1551465715 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f82a4066f60 0x7f82a4069420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f82b00099b0 tx=0x7f82b0008040 comp rx=0 tx=0).stop 2024-04-25T10:06:18.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.819+0000 7f82c7ff6640 1 -- 172.21.15.60:0/1551465715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c01563b0 msgr2=0x7f82c01f4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.819+0000 7f82c7ff6640 1 --2- 172.21.15.60:0/1551465715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c01563b0 0x7f82c01f4810 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009fd0 tx=0x7f82bc011b30 comp rx=0 tx=0).stop 2024-04-25T10:06:18.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.819+0000 7f82c7ff6640 1 -- 172.21.15.60:0/1551465715 shutdown_connections 2024-04-25T10:06:18.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.819+0000 7f82c7ff6640 1 --2- 172.21.15.60:0/1551465715 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f82a4066f60 0x7f82a4069420 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.819+0000 7f82c7ff6640 1 --2- 172.21.15.60:0/1551465715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82c01f4d50 0x7f82c00b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.819+0000 7f82c7ff6640 1 --2- 172.21.15.60:0/1551465715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82c01563b0 0x7f82c01f4810 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.819+0000 7f82c7ff6640 1 --2- 172.21.15.60:0/1551465715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82c0155fd0 0x7f82c01e46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.819+0000 7f82c7ff6640 1 -- 172.21.15.60:0/1551465715 >> 172.21.15.60:0/1551465715 conn(0x7f82c00b12d0 msgr2=0x7f82c00cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:18.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.819+0000 7f82c7ff6640 1 -- 172.21.15.60:0/1551465715 shutdown_connections 2024-04-25T10:06:18.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.819+0000 7f82c7ff6640 1 -- 172.21.15.60:0/1551465715 wait complete. 2024-04-25T10:06:18.842 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:18.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc38ccb9640 1 Processor -- start 2024-04-25T10:06:18.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc38ccb9640 1 -- start start 2024-04-25T10:06:18.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc38ccb9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc38815a7a0 0x7fc3880cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc38ccb9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38806f130 con 0x7fc3880b37e0 2024-04-25T10:06:18.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc38ccb9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38806f410 con 0x7fc38806ec30 2024-04-25T10:06:18.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc38ccb9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880cdaa0 con 0x7fc38815a7a0 2024-04-25T10:06:18.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc386d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc3880b37e0 0x7fc38806dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45790/0 (socket says 172.21.15.60:45790) 2024-04-25T10:06:18.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc386d9d640 1 -- 172.21.15.60:0/231282321 learned_addr learned my addr 172.21.15.60:0/231282321 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:18.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc38659c640 1 --2- 172.21.15.60:0/231282321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc38815a7a0 0x7fc3880cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc385d9b640 1 -- 172.21.15.60:0/231282321 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2752159107 0 0) 0x7fc38806f130 con 0x7fc3880b37e0 2024-04-25T10:06:18.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc385d9b640 1 -- 172.21.15.60:0/231282321 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc374003530 con 0x7fc3880b37e0 2024-04-25T10:06:18.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc385d9b640 1 -- 172.21.15.60:0/231282321 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1521278713 0 0) 0x7fc374003530 con 0x7fc3880b37e0 2024-04-25T10:06:18.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc385d9b640 1 -- 172.21.15.60:0/231282321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc38815a7a0 msgr2=0x7fc3880cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:18.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc385d9b640 1 --2- 172.21.15.60:0/231282321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc38815a7a0 0x7fc3880cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc385d9b640 1 -- 172.21.15.60:0/231282321 >> v1:172.21.15.100:6789/0 conn(0x7fc38806ec30 legacy=0x7fc388071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc385d9b640 1 -- 172.21.15.60:0/231282321 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38806d540 con 0x7fc3880b37e0 2024-04-25T10:06:18.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc385d9b640 1 -- 172.21.15.60:0/231282321 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fc380001a00 con 0x7fc3880b37e0 2024-04-25T10:06:18.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc385d9b640 1 -- 172.21.15.60:0/231282321 >> v1:172.21.15.60:6789/0 conn(0x7fc3880b37e0 legacy=0x7fc38806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc385d9b640 1 --2- 172.21.15.60:0/231282321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3740039f0 0x7fc374003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc385d9b640 1 --2- 172.21.15.60:0/231282321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc374004810 0x7fc374004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc385d9b640 1 --2- 172.21.15.60:0/231282321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc38815a7a0 0x7fc3740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc385d9b640 1 -- 172.21.15.60:0/231282321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37400ba80 con 0x7fc3740039f0 2024-04-25T10:06:18.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc385d9b640 1 -- 172.21.15.60:0/231282321 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37400bc00 con 0x7fc374004810 2024-04-25T10:06:18.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc385d9b640 1 -- 172.21.15.60:0/231282321 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37400e010 con 0x7fc38815a7a0 2024-04-25T10:06:18.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc38659c640 1 --2- 172.21.15.60:0/231282321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3740039f0 0x7fc374003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc37dd9b640 1 --2- 172.21.15.60:0/231282321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc374004810 0x7fc374004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc386d9d640 1 --2- 172.21.15.60:0/231282321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc38815a7a0 0x7fc3740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc38659c640 1 -- 172.21.15.60:0/231282321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc38815a7a0 msgr2=0x7fc3740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:18.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc38659c640 1 --2- 172.21.15.60:0/231282321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc38815a7a0 0x7fc3740051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc38659c640 1 -- 172.21.15.60:0/231282321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc374004810 msgr2=0x7fc374004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:18.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc38659c640 1 --2- 172.21.15.60:0/231282321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc374004810 0x7fc374004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.955+0000 7fc38659c640 1 -- 172.21.15.60:0/231282321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc374003530 con 0x7fc3740039f0 2024-04-25T10:06:18.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.959+0000 7fc38659c640 1 --2- 172.21.15.60:0/231282321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3740039f0 0x7fc374003e10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc378009ab0 tx=0x7fc37801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9e672c5297531a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.959+0000 7fc385d9b640 1 -- 172.21.15.60:0/231282321 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3780096b0 con 0x7fc3740039f0 2024-04-25T10:06:18.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.959+0000 7fc385d9b640 1 -- 172.21.15.60:0/231282321 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3780a6420 con 0x7fc3740039f0 2024-04-25T10:06:18.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.959+0000 7fc385d9b640 1 -- 172.21.15.60:0/231282321 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3780a6810 con 0x7fc3740039f0 2024-04-25T10:06:18.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.959+0000 7fc38ccb9640 1 -- 172.21.15.60:0/231282321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3740039f0 msgr2=0x7fc374003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.959+0000 7fc38ccb9640 1 --2- 172.21.15.60:0/231282321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3740039f0 0x7fc374003e10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc378009ab0 tx=0x7fc37801ca30 comp rx=0 tx=0).stop 2024-04-25T10:06:18.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.959+0000 7fc386d9d640 1 -- 172.21.15.60:0/231282321 reap_dead start 2024-04-25T10:06:18.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.959+0000 7fc38ccb9640 1 -- 172.21.15.60:0/231282321 shutdown_connections 2024-04-25T10:06:18.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.959+0000 7fc38ccb9640 1 -- 172.21.15.60:0/231282321 >> 172.21.15.60:0/231282321 conn(0x7fc3880b12d0 msgr2=0x7fc3880b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:18.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.959+0000 7fc38ccb9640 1 -- 172.21.15.60:0/231282321 shutdown_connections 2024-04-25T10:06:18.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.959+0000 7fc38ccb9640 1 -- 172.21.15.60:0/231282321 wait complete. 2024-04-25T10:06:18.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.959+0000 7fc38ccb9640 1 Processor -- start 2024-04-25T10:06:18.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.959+0000 7fc38ccb9640 1 -- start start 2024-04-25T10:06:18.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc38ccb9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3740039f0 0x7fc388147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc38ccb9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc38815af00 0x7fc38815b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc38ccb9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc388149e80 0x7fc38814a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc38ccb9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38806e7e0 con 0x7fc38815af00 2024-04-25T10:06:18.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc38ccb9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38806e660 con 0x7fc3740039f0 2024-04-25T10:06:18.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc38ccb9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38806e960 con 0x7fc388149e80 2024-04-25T10:06:18.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc37dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc38815af00 0x7fc38815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc37dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc38815af00 0x7fc38815b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37612/0 (socket says 172.21.15.60:37612) 2024-04-25T10:06:18.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc37dd9b640 1 -- 172.21.15.60:0/356502596 learned_addr learned my addr 172.21.15.60:0/356502596 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:18.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc386d9d640 1 --2- 172.21.15.60:0/356502596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc388149e80 0x7fc38814a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc38659c640 1 --2- 172.21.15.60:0/356502596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3740039f0 0x7fc388147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc37dd9b640 1 -- 172.21.15.60:0/356502596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc388149e80 msgr2=0x7fc38814a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc37dd9b640 1 --2- 172.21.15.60:0/356502596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc388149e80 0x7fc38814a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc37dd9b640 1 -- 172.21.15.60:0/356502596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3740039f0 msgr2=0x7fc388147340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc37dd9b640 1 --2- 172.21.15.60:0/356502596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3740039f0 0x7fc388147340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc37dd9b640 1 -- 172.21.15.60:0/356502596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38814abb0 con 0x7fc38815af00 2024-04-25T10:06:18.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc37dd9b640 1 --2- 172.21.15.60:0/356502596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc38815af00 0x7fc38815b380 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc37000e530 tx=0x7fc37000e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc37f7fe640 1 -- 172.21.15.60:0/356502596 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3700053a0 con 0x7fc38815af00 2024-04-25T10:06:18.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc37f7fe640 1 -- 172.21.15.60:0/356502596 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc370018070 con 0x7fc38815af00 2024-04-25T10:06:18.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc37f7fe640 1 -- 172.21.15.60:0/356502596 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc370003820 con 0x7fc38815af00 2024-04-25T10:06:18.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc38ccb9640 1 -- 172.21.15.60:0/356502596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc388147cd0 con 0x7fc38815af00 2024-04-25T10:06:18.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc38ccb9640 1 -- 172.21.15.60:0/356502596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc388148290 con 0x7fc38815af00 2024-04-25T10:06:18.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc38ccb9640 1 -- 172.21.15.60:0/356502596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc38815b8c0 con 0x7fc38815af00 2024-04-25T10:06:18.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc37f7fe640 1 -- 172.21.15.60:0/356502596 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3700039c0 con 0x7fc38815af00 2024-04-25T10:06:18.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc37f7fe640 1 --2- 172.21.15.60:0/356502596 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc368066f60 0x7fc368069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc37f7fe640 1 -- 172.21.15.60:0/356502596 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc3700849b0 con 0x7fc38815af00 2024-04-25T10:06:18.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.963+0000 7fc38659c640 1 --2- 172.21.15.60:0/356502596 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc368066f60 0x7fc368069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.967+0000 7fc38659c640 1 --2- 172.21.15.60:0/356502596 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc368066f60 0x7fc368069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc3780ad000 tx=0x7fc3780046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:18.967+0000 7fc37f7fe640 1 -- 172.21.15.60:0/356502596 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc370055d10 con 0x7fc38815af00 2024-04-25T10:06:19.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.123+0000 7fc38ccb9640 1 -- 172.21.15.60:0/356502596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc38815be60 con 0x7fc38815af00 2024-04-25T10:06:19.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.123+0000 7fc37f7fe640 1 -- 172.21.15.60:0/356502596 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+12903 (secure 0 0 0) 0x7fc370055610 con 0x7fc38815af00 2024-04-25T10:06:19.132 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:19.132 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":31,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:16.154308+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:19.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.127+0000 7fc38ccb9640 1 -- 172.21.15.60:0/356502596 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc368066f60 msgr2=0x7fc368069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.127+0000 7fc38ccb9640 1 --2- 172.21.15.60:0/356502596 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc368066f60 0x7fc368069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc3780ad000 tx=0x7fc3780046d0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.127+0000 7fc38ccb9640 1 -- 172.21.15.60:0/356502596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc38815af00 msgr2=0x7fc38815b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.127+0000 7fc38ccb9640 1 --2- 172.21.15.60:0/356502596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc38815af00 0x7fc38815b380 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc37000e530 tx=0x7fc37000e9f0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.127+0000 7fc38ccb9640 1 -- 172.21.15.60:0/356502596 shutdown_connections 2024-04-25T10:06:19.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.127+0000 7fc38ccb9640 1 --2- 172.21.15.60:0/356502596 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc368066f60 0x7fc368069420 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.127+0000 7fc38ccb9640 1 --2- 172.21.15.60:0/356502596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc388149e80 0x7fc38814a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.127+0000 7fc38ccb9640 1 --2- 172.21.15.60:0/356502596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc38815af00 0x7fc38815b380 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.127+0000 7fc38ccb9640 1 --2- 172.21.15.60:0/356502596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3740039f0 0x7fc388147340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.127+0000 7fc38ccb9640 1 -- 172.21.15.60:0/356502596 >> 172.21.15.60:0/356502596 conn(0x7fc3880b12d0 msgr2=0x7fc388073ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.127+0000 7fc38ccb9640 1 -- 172.21.15.60:0/356502596 shutdown_connections 2024-04-25T10:06:19.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.127+0000 7fc38ccb9640 1 -- 172.21.15.60:0/356502596 wait complete. 2024-04-25T10:06:19.150 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:19.150 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:19.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c96ad640 1 Processor -- start 2024-04-25T10:06:19.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c96ad640 1 -- start start 2024-04-25T10:06:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c96ad640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f62c41595a0 0x7f62c40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c96ad640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f62c415c800 0x7f62c40c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c96ad640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c415f5c0 con 0x7f62c40b3950 2024-04-25T10:06:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c96ad640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62c4160380 con 0x7f62c41595a0 2024-04-25T10:06:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c96ad640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62c415cc30 con 0x7f62c415c800 2024-04-25T10:06:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c27fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f62c40b3950 0x7f62c415c0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45796/0 (socket says 172.21.15.60:45796) 2024-04-25T10:06:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c27fc640 1 -- 172.21.15.60:0/2765716451 learned_addr learned my addr 172.21.15.60:0/2765716451 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c2ffd640 1 --2- 172.21.15.60:0/2765716451 >> v2:172.21.15.60:3300/0 conn(0x7f62c41595a0 0x7f62c40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c1ffb640 1 -- 172.21.15.60:0/2765716451 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926271475 0 0) 0x7f62c415f5c0 con 0x7f62c40b3950 2024-04-25T10:06:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c1ffb640 1 -- 172.21.15.60:0/2765716451 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62b0003530 con 0x7f62c40b3950 2024-04-25T10:06:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c37fe640 1 --2- 172.21.15.60:0/2765716451 >> v2:172.21.15.100:3300/0 conn(0x7f62c415c800 0x7f62c40c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c2ffd640 1 -- 172.21.15.60:0/2765716451 >> v2:172.21.15.100:3300/0 conn(0x7f62c415c800 msgr2=0x7f62c40c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c2ffd640 1 --2- 172.21.15.60:0/2765716451 >> v2:172.21.15.100:3300/0 conn(0x7f62c415c800 0x7f62c40c6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c2ffd640 1 -- 172.21.15.60:0/2765716451 >> v1:172.21.15.60:6789/0 conn(0x7f62c40b3950 legacy=0x7f62c415c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c2ffd640 1 -- 172.21.15.60:0/2765716451 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c415f8a0 con 0x7f62c41595a0 2024-04-25T10:06:19.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.267+0000 7f62c37fe640 1 --2- 172.21.15.60:0/2765716451 >> v2:172.21.15.100:3300/0 conn(0x7f62c415c800 0x7f62c40c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:19.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c2ffd640 1 --2- 172.21.15.60:0/2765716451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62c41595a0 0x7f62c40b3410 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f62c40c6990 tx=0x7f62bc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1454eb4554f1efc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c1ffb640 1 -- 172.21.15.60:0/2765716451 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc0b2070 con 0x7f62c41595a0 2024-04-25T10:06:19.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c1ffb640 1 -- 172.21.15.60:0/2765716451 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62bc0a57f0 con 0x7f62c41595a0 2024-04-25T10:06:19.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c1ffb640 1 -- 172.21.15.60:0/2765716451 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc0a5bc0 con 0x7f62c41595a0 2024-04-25T10:06:19.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c96ad640 1 -- 172.21.15.60:0/2765716451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62c41595a0 msgr2=0x7f62c40b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c96ad640 1 --2- 172.21.15.60:0/2765716451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62c41595a0 0x7f62c40b3410 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f62c40c6990 tx=0x7f62bc01c930 comp rx=0 tx=0).stop 2024-04-25T10:06:19.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c96ad640 1 -- 172.21.15.60:0/2765716451 shutdown_connections 2024-04-25T10:06:19.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c96ad640 1 --2- 172.21.15.60:0/2765716451 >> v2:172.21.15.100:3300/0 conn(0x7f62c415c800 0x7f62c40c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c96ad640 1 --2- 172.21.15.60:0/2765716451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62c41595a0 0x7f62c40b3410 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c96ad640 1 -- 172.21.15.60:0/2765716451 >> 172.21.15.60:0/2765716451 conn(0x7f62c40669a0 msgr2=0x7f62c4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c96ad640 1 -- 172.21.15.60:0/2765716451 shutdown_connections 2024-04-25T10:06:19.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c96ad640 1 -- 172.21.15.60:0/2765716451 wait complete. 2024-04-25T10:06:19.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c96ad640 1 Processor -- start 2024-04-25T10:06:19.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c96ad640 1 -- start start 2024-04-25T10:06:19.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c96ad640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62c40b3950 0x7f62c406d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c96ad640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62c41595a0 0x7f62c406bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c96ad640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62c415c800 0x7f62c415a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.271+0000 7f62c96ad640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c40c7bc0 con 0x7f62c41595a0 2024-04-25T10:06:19.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c96ad640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c40c7a40 con 0x7f62c40b3950 2024-04-25T10:06:19.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c96ad640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c4168750 con 0x7f62c415c800 2024-04-25T10:06:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62c41595a0 0x7f62c406bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62c41595a0 0x7f62c406bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37642/0 (socket says 172.21.15.60:37642) 2024-04-25T10:06:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c27fc640 1 -- 172.21.15.60:0/1077834084 learned_addr learned my addr 172.21.15.60:0/1077834084 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c37fe640 1 --2- 172.21.15.60:0/1077834084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62c415c800 0x7f62c415a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c2ffd640 1 --2- 172.21.15.60:0/1077834084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62c40b3950 0x7f62c406d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c27fc640 1 -- 172.21.15.60:0/1077834084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62c415c800 msgr2=0x7f62c415a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c27fc640 1 --2- 172.21.15.60:0/1077834084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62c415c800 0x7f62c415a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c27fc640 1 -- 172.21.15.60:0/1077834084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62c40b3950 msgr2=0x7f62c406d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c27fc640 1 --2- 172.21.15.60:0/1077834084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62c40b3950 0x7f62c406d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c27fc640 1 -- 172.21.15.60:0/1077834084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c406c390 con 0x7f62c41595a0 2024-04-25T10:06:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c27fc640 1 --2- 172.21.15.60:0/1077834084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62c41595a0 0x7f62c406bd20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00c3c0 tx=0x7f62ac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62bb7fe640 1 -- 172.21.15.60:0/1077834084 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac017070 con 0x7f62c41595a0 2024-04-25T10:06:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62bb7fe640 1 -- 172.21.15.60:0/1077834084 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac005cf0 con 0x7f62c41595a0 2024-04-25T10:06:19.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c96ad640 1 -- 172.21.15.60:0/1077834084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c406c670 con 0x7f62c41595a0 2024-04-25T10:06:19.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62bb7fe640 1 -- 172.21.15.60:0/1077834084 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0072f0 con 0x7f62c41595a0 2024-04-25T10:06:19.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c96ad640 1 -- 172.21.15.60:0/1077834084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c406cc30 con 0x7f62c41595a0 2024-04-25T10:06:19.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c96ad640 1 -- 172.21.15.60:0/1077834084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c4074770 con 0x7f62c41595a0 2024-04-25T10:06:19.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62bb7fe640 1 -- 172.21.15.60:0/1077834084 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62ac01a070 con 0x7f62c41595a0 2024-04-25T10:06:19.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62bb7fe640 1 --2- 172.21.15.60:0/1077834084 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62a4066f60 0x7f62a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62bb7fe640 1 -- 172.21.15.60:0/1077834084 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f62ac084020 con 0x7f62c41595a0 2024-04-25T10:06:19.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.275+0000 7f62c2ffd640 1 --2- 172.21.15.60:0/1077834084 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62a4066f60 0x7f62a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.279+0000 7f62c2ffd640 1 --2- 172.21.15.60:0/1077834084 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62a4066f60 0x7f62a4069420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f62bc0af000 tx=0x7f62bc0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.279+0000 7f62bb7fe640 1 -- 172.21.15.60:0/1077834084 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62ac055410 con 0x7f62c41595a0 2024-04-25T10:06:19.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.435+0000 7f62c96ad640 1 -- 172.21.15.60:0/1077834084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f62c40cef70 con 0x7f62c41595a0 2024-04-25T10:06:19.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.439+0000 7f62bb7fe640 1 -- 172.21.15.60:0/1077834084 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7f62ac054d10 con 0x7f62c41595a0 2024-04-25T10:06:19.445 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:06:19.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.439+0000 7f62c96ad640 1 -- 172.21.15.60:0/1077834084 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62a4066f60 msgr2=0x7f62a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.439+0000 7f62c96ad640 1 --2- 172.21.15.60:0/1077834084 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62a4066f60 0x7f62a4069420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f62bc0af000 tx=0x7f62bc0a7070 comp rx=0 tx=0).stop 2024-04-25T10:06:19.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.439+0000 7f62c96ad640 1 -- 172.21.15.60:0/1077834084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62c41595a0 msgr2=0x7f62c406bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.439+0000 7f62c96ad640 1 --2- 172.21.15.60:0/1077834084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62c41595a0 0x7f62c406bd20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00c3c0 tx=0x7f62ac00c880 comp rx=0 tx=0).stop 2024-04-25T10:06:19.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.439+0000 7f62c96ad640 1 -- 172.21.15.60:0/1077834084 shutdown_connections 2024-04-25T10:06:19.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.439+0000 7f62c96ad640 1 --2- 172.21.15.60:0/1077834084 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62a4066f60 0x7f62a4069420 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.439+0000 7f62c96ad640 1 --2- 172.21.15.60:0/1077834084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62c415c800 0x7f62c415a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.439+0000 7f62c96ad640 1 --2- 172.21.15.60:0/1077834084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62c41595a0 0x7f62c406bd20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.439+0000 7f62c96ad640 1 --2- 172.21.15.60:0/1077834084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62c40b3950 0x7f62c406d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.439+0000 7f62c96ad640 1 -- 172.21.15.60:0/1077834084 >> 172.21.15.60:0/1077834084 conn(0x7f62c40669a0 msgr2=0x7f62c40d1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.439+0000 7f62c96ad640 1 -- 172.21.15.60:0/1077834084 shutdown_connections 2024-04-25T10:06:19.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.439+0000 7f62c96ad640 1 -- 172.21.15.60:0/1077834084 wait complete. 2024-04-25T10:06:19.462 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:06:19.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07d2eb8640 1 Processor -- start 2024-04-25T10:06:19.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07d2eb8640 1 -- start start 2024-04-25T10:06:19.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07d2eb8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15a7a0 0x7f07cc0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07d2eb8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f07cc06ec30 0x7f07cc071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07d2eb8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07cc0721e0 con 0x7f07cc0b37e0 2024-04-25T10:06:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07d2eb8640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07cc0cdaa0 con 0x7f07cc06ec30 2024-04-25T10:06:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07d2eb8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc06d2f0 con 0x7f07cc15a7a0 2024-04-25T10:06:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07cbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f07cc06ec30 0x7f07cc071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07d1455640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f07cc0b37e0 0x7f07cc06dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45800/0 (socket says 172.21.15.60:45800) 2024-04-25T10:06:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07cbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f07cc06ec30 0x7f07cc071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37658/0 (socket says 172.21.15.60:37658) 2024-04-25T10:06:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07d1455640 1 -- 172.21.15.60:0/978207006 learned_addr learned my addr 172.21.15.60:0/978207006 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07d0c54640 1 --2- 172.21.15.60:0/978207006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15a7a0 0x7f07cc0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07cbfff640 1 -- 172.21.15.60:0/978207006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15a7a0 msgr2=0x7f07cc0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07cbfff640 1 --2- 172.21.15.60:0/978207006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15a7a0 0x7f07cc0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07cbfff640 1 -- 172.21.15.60:0/978207006 >> v1:172.21.15.60:6789/0 conn(0x7f07cc0b37e0 legacy=0x7f07cc06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07cb7fe640 1 -- 172.21.15.60:0/978207006 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903238249 0 0) 0x7f07cc0721e0 con 0x7f07cc0b37e0 2024-04-25T10:06:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07cbfff640 1 -- 172.21.15.60:0/978207006 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc06f130 con 0x7f07cc06ec30 2024-04-25T10:06:19.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.567+0000 7f07cbfff640 1 --2- 172.21.15.60:0/978207006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc06ec30 0x7f07cc071ca0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f07c00097c0 tx=0x7f07c001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d867154b4db538ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07cb7fe640 1 -- 172.21.15.60:0/978207006 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c00b1070 con 0x7f07cc06ec30 2024-04-25T10:06:19.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07cb7fe640 1 -- 172.21.15.60:0/978207006 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c00ac410 con 0x7f07cc06ec30 2024-04-25T10:06:19.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07cb7fe640 1 -- 172.21.15.60:0/978207006 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c00ac7e0 con 0x7f07cc06ec30 2024-04-25T10:06:19.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 -- 172.21.15.60:0/978207006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc06ec30 msgr2=0x7f07cc071ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 --2- 172.21.15.60:0/978207006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc06ec30 0x7f07cc071ca0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f07c00097c0 tx=0x7f07c001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 -- 172.21.15.60:0/978207006 shutdown_connections 2024-04-25T10:06:19.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 --2- 172.21.15.60:0/978207006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc06ec30 0x7f07cc071ca0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 --2- 172.21.15.60:0/978207006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15a7a0 0x7f07cc0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 -- 172.21.15.60:0/978207006 >> 172.21.15.60:0/978207006 conn(0x7f07cc0b12d0 msgr2=0x7f07cc0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 -- 172.21.15.60:0/978207006 shutdown_connections 2024-04-25T10:06:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 -- 172.21.15.60:0/978207006 wait complete. 2024-04-25T10:06:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 Processor -- start 2024-04-25T10:06:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 -- start start 2024-04-25T10:06:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc06ec30 0x7f07cc136fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc0b37e0 0x7f07cc152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15a7a0 0x7f07cc152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc15ad00 con 0x7f07cc0b37e0 2024-04-25T10:06:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc15ab80 con 0x7f07cc06ec30 2024-04-25T10:06:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc06e860 con 0x7f07cc15a7a0 2024-04-25T10:06:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07cbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc0b37e0 0x7f07cc152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07cbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc0b37e0 0x7f07cc152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37660/0 (socket says 172.21.15.60:37660) 2024-04-25T10:06:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07cbfff640 1 -- 172.21.15.60:0/2547910930 learned_addr learned my addr 172.21.15.60:0/2547910930 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d1455640 1 --2- 172.21.15.60:0/2547910930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15a7a0 0x7f07cc152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d0c54640 1 --2- 172.21.15.60:0/2547910930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc06ec30 0x7f07cc136fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07cbfff640 1 -- 172.21.15.60:0/2547910930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15a7a0 msgr2=0x7f07cc152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07cbfff640 1 --2- 172.21.15.60:0/2547910930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15a7a0 0x7f07cc152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07cbfff640 1 -- 172.21.15.60:0/2547910930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc06ec30 msgr2=0x7f07cc136fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07cbfff640 1 --2- 172.21.15.60:0/2547910930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc06ec30 0x7f07cc136fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07cbfff640 1 -- 172.21.15.60:0/2547910930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc13a700 con 0x7f07cc0b37e0 2024-04-25T10:06:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d0c54640 1 --2- 172.21.15.60:0/2547910930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc06ec30 0x7f07cc136fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07cbfff640 1 --2- 172.21.15.60:0/2547910930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc0b37e0 0x7f07cc152790 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f07c00097c0 tx=0x7f07c0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07c97fa640 1 -- 172.21.15.60:0/2547910930 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c00b9020 con 0x7f07cc0b37e0 2024-04-25T10:06:19.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07c97fa640 1 -- 172.21.15.60:0/2547910930 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c00ace40 con 0x7f07cc0b37e0 2024-04-25T10:06:19.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 -- 172.21.15.60:0/2547910930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07cc137b20 con 0x7f07cc0b37e0 2024-04-25T10:06:19.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07c97fa640 1 -- 172.21.15.60:0/2547910930 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c00b1040 con 0x7f07cc0b37e0 2024-04-25T10:06:19.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.571+0000 7f07d2eb8640 1 -- 172.21.15.60:0/2547910930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07cc1380e0 con 0x7f07cc0b37e0 2024-04-25T10:06:19.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.575+0000 7f07d2eb8640 1 -- 172.21.15.60:0/2547910930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07cc139760 con 0x7f07cc0b37e0 2024-04-25T10:06:19.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.575+0000 7f07c97fa640 1 -- 172.21.15.60:0/2547910930 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f07c00bc020 con 0x7f07cc0b37e0 2024-04-25T10:06:19.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.575+0000 7f07c97fa640 1 --2- 172.21.15.60:0/2547910930 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f07b8066f60 0x7f07b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.575+0000 7f07c97fa640 1 -- 172.21.15.60:0/2547910930 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f07c011e5e0 con 0x7f07cc0b37e0 2024-04-25T10:06:19.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.575+0000 7f07d0c54640 1 --2- 172.21.15.60:0/2547910930 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f07b8066f60 0x7f07b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.575+0000 7f07d0c54640 1 --2- 172.21.15.60:0/2547910930 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f07b8066f60 0x7f07b8069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f07bc007330 tx=0x7f07bc00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.579+0000 7f07c97fa640 1 -- 172.21.15.60:0/2547910930 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07cc139760 con 0x7f07cc0b37e0 2024-04-25T10:06:19.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.735+0000 7f07d2eb8640 1 -- 172.21.15.60:0/2547910930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f07cc074770 con 0x7f07cc0b37e0 2024-04-25T10:06:19.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.735+0000 7f07c97fa640 1 -- 172.21.15.60:0/2547910930 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+65 (secure 0 0 0) 0x7f07c00c3030 con 0x7f07cc0b37e0 2024-04-25T10:06:19.744 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:19.744 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T10:06:19.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:06:19.744 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T10:06:19.744 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T10:06:19.744 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:19.744 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T10:06:19.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.739+0000 7f07d2eb8640 1 -- 172.21.15.60:0/2547910930 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f07b8066f60 msgr2=0x7f07b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.739+0000 7f07d2eb8640 1 --2- 172.21.15.60:0/2547910930 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f07b8066f60 0x7f07b8069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f07bc007330 tx=0x7f07bc00b040 comp rx=0 tx=0).stop 2024-04-25T10:06:19.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.739+0000 7f07d2eb8640 1 -- 172.21.15.60:0/2547910930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc0b37e0 msgr2=0x7f07cc152790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.739+0000 7f07d2eb8640 1 --2- 172.21.15.60:0/2547910930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc0b37e0 0x7f07cc152790 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f07c00097c0 tx=0x7f07c0019240 comp rx=0 tx=0).stop 2024-04-25T10:06:19.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.739+0000 7f07d2eb8640 1 -- 172.21.15.60:0/2547910930 shutdown_connections 2024-04-25T10:06:19.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.739+0000 7f07d2eb8640 1 --2- 172.21.15.60:0/2547910930 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f07b8066f60 0x7f07b8069420 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.739+0000 7f07d2eb8640 1 --2- 172.21.15.60:0/2547910930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15a7a0 0x7f07cc152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.739+0000 7f07d2eb8640 1 --2- 172.21.15.60:0/2547910930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc0b37e0 0x7f07cc152790 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.739+0000 7f07d2eb8640 1 --2- 172.21.15.60:0/2547910930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc06ec30 0x7f07cc136fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.739+0000 7f07d2eb8640 1 -- 172.21.15.60:0/2547910930 >> 172.21.15.60:0/2547910930 conn(0x7f07cc0b12d0 msgr2=0x7f07cc15d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.739+0000 7f07d2eb8640 1 -- 172.21.15.60:0/2547910930 shutdown_connections 2024-04-25T10:06:19.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.739+0000 7f07d2eb8640 1 -- 172.21.15.60:0/2547910930 wait complete. 2024-04-25T10:06:19.761 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:06:19.761 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:06:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.879+0000 7fa6c750c640 1 Processor -- start 2024-04-25T10:06:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.879+0000 7fa6c750c640 1 -- start start 2024-04-25T10:06:19.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c750c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa6c015b450 0x7fa6c00727e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c750c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c015bd20 con 0x7fa6c0072370 2024-04-25T10:06:19.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c750c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c00cd170 con 0x7fa6c01583f0 2024-04-25T10:06:19.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c750c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c0072d20 con 0x7fa6c015b450 2024-04-25T10:06:19.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c4aa7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa6c0072370 0x7fa6c015ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45802/0 (socket says 172.21.15.60:45802) 2024-04-25T10:06:19.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c4aa7640 1 -- 172.21.15.60:0/882704144 learned_addr learned my addr 172.21.15.60:0/882704144 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:19.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c5aa9640 1 --2- 172.21.15.60:0/882704144 >> v2:172.21.15.100:3300/0 conn(0x7fa6c015b450 0x7fa6c00727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6b7fff640 1 -- 172.21.15.60:0/882704144 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 326842153 0 0) 0x7fa6c015bd20 con 0x7fa6c0072370 2024-04-25T10:06:19.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6b7fff640 1 -- 172.21.15.60:0/882704144 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6ac003530 con 0x7fa6c0072370 2024-04-25T10:06:19.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6b7fff640 1 -- 172.21.15.60:0/882704144 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2172326239 0 0) 0x7fa6ac003530 con 0x7fa6c0072370 2024-04-25T10:06:19.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6b7fff640 1 -- 172.21.15.60:0/882704144 >> v2:172.21.15.100:3300/0 conn(0x7fa6c015b450 msgr2=0x7fa6c00727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6b7fff640 1 --2- 172.21.15.60:0/882704144 >> v2:172.21.15.100:3300/0 conn(0x7fa6c015b450 0x7fa6c00727e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6b7fff640 1 -- 172.21.15.60:0/882704144 >> v1:172.21.15.100:6789/0 conn(0x7fa6c01583f0 legacy=0x7fa6c00cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6b7fff640 1 -- 172.21.15.60:0/882704144 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c00ce320 con 0x7fa6c0072370 2024-04-25T10:06:19.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6b7fff640 1 -- 172.21.15.60:0/882704144 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fa6b0001680 con 0x7fa6c0072370 2024-04-25T10:06:19.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6b7fff640 1 -- 172.21.15.60:0/882704144 >> v1:172.21.15.60:6789/0 conn(0x7fa6c0072370 legacy=0x7fa6c015ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6b7fff640 1 --2- 172.21.15.60:0/882704144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6ac0039c0 0x7fa6ac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6b7fff640 1 --2- 172.21.15.60:0/882704144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6ac004810 0x7fa6ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6b7fff640 1 --2- 172.21.15.60:0/882704144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6ac005220 0x7fa6ac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6b7fff640 1 -- 172.21.15.60:0/882704144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c0072d20 con 0x7fa6ac005220 2024-04-25T10:06:19.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6b7fff640 1 -- 172.21.15.60:0/882704144 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac00c150 con 0x7fa6ac004810 2024-04-25T10:06:19.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6b7fff640 1 -- 172.21.15.60:0/882704144 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac00c350 con 0x7fa6ac0039c0 2024-04-25T10:06:19.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c5aa9640 1 --2- 172.21.15.60:0/882704144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6ac005220 0x7fa6ac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c52a8640 1 --2- 172.21.15.60:0/882704144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6ac0039c0 0x7fa6ac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c4aa7640 1 --2- 172.21.15.60:0/882704144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6ac004810 0x7fa6ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c5aa9640 1 -- 172.21.15.60:0/882704144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6ac0039c0 msgr2=0x7fa6ac003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c5aa9640 1 --2- 172.21.15.60:0/882704144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6ac0039c0 0x7fa6ac003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c5aa9640 1 -- 172.21.15.60:0/882704144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6ac004810 msgr2=0x7fa6ac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c5aa9640 1 --2- 172.21.15.60:0/882704144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6ac004810 0x7fa6ac004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c5aa9640 1 -- 172.21.15.60:0/882704144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac003530 con 0x7fa6ac005220 2024-04-25T10:06:19.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c4aa7640 1 --2- 172.21.15.60:0/882704144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6ac004810 0x7fa6ac004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:19.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c5aa9640 1 -- 172.21.15.60:0/882704144 reap_dead start 2024-04-25T10:06:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.883+0000 7fa6c5aa9640 1 --2- 172.21.15.60:0/882704144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6ac005220 0x7fa6ac00b9d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009fa0 tx=0x7fa6b801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50db0057707c3954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6b7fff640 1 -- 172.21.15.60:0/882704144 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80b2070 con 0x7fa6ac005220 2024-04-25T10:06:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6b7fff640 1 -- 172.21.15.60:0/882704144 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b80a5d50 con 0x7fa6ac005220 2024-04-25T10:06:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6b7fff640 1 -- 172.21.15.60:0/882704144 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8009bf0 con 0x7fa6ac005220 2024-04-25T10:06:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 -- 172.21.15.60:0/882704144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6ac005220 msgr2=0x7fa6ac00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 --2- 172.21.15.60:0/882704144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6ac005220 0x7fa6ac00b9d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009fa0 tx=0x7fa6b801cc80 comp rx=0 tx=0).stop 2024-04-25T10:06:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 -- 172.21.15.60:0/882704144 shutdown_connections 2024-04-25T10:06:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 --2- 172.21.15.60:0/882704144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6ac005220 0x7fa6ac00b9d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 -- 172.21.15.60:0/882704144 >> 172.21.15.60:0/882704144 conn(0x7fa6c006e400 msgr2=0x7fa6c006e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 -- 172.21.15.60:0/882704144 shutdown_connections 2024-04-25T10:06:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 -- 172.21.15.60:0/882704144 wait complete. 2024-04-25T10:06:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 Processor -- start 2024-04-25T10:06:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 -- start start 2024-04-25T10:06:19.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6ac005220 0x7fa6c0144930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6c00b1110 0x7fa6c00b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6c0147240 0x7fa6c01476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c00cd6f0 con 0x7fa6c00b1110 2024-04-25T10:06:19.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c01587d0 con 0x7fa6ac005220 2024-04-25T10:06:19.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c015b880 con 0x7fa6c0147240 2024-04-25T10:06:19.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c4aa7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6c00b1110 0x7fa6c00b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c4aa7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6c00b1110 0x7fa6c00b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37682/0 (socket says 172.21.15.60:37682) 2024-04-25T10:06:19.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c4aa7640 1 -- 172.21.15.60:0/1232564962 learned_addr learned my addr 172.21.15.60:0/1232564962 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:19.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c52a8640 1 --2- 172.21.15.60:0/1232564962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6ac005220 0x7fa6c0144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c5aa9640 1 --2- 172.21.15.60:0/1232564962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6c0147240 0x7fa6c01476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c4aa7640 1 -- 172.21.15.60:0/1232564962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6c0147240 msgr2=0x7fa6c01476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c4aa7640 1 --2- 172.21.15.60:0/1232564962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6c0147240 0x7fa6c01476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c4aa7640 1 -- 172.21.15.60:0/1232564962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6ac005220 msgr2=0x7fa6c0144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c4aa7640 1 --2- 172.21.15.60:0/1232564962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6ac005220 0x7fa6c0144930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c4aa7640 1 -- 172.21.15.60:0/1232564962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c0147f50 con 0x7fa6c00b1110 2024-04-25T10:06:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c5aa9640 1 --2- 172.21.15.60:0/1232564962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6c0147240 0x7fa6c01476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c4aa7640 1 --2- 172.21.15.60:0/1232564962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6c00b1110 0x7fa6c00b1590 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0010aa0 tx=0x7fa6b0010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6b5ffb640 1 -- 172.21.15.60:0/1232564962 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b001b070 con 0x7fa6c00b1110 2024-04-25T10:06:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6b5ffb640 1 -- 172.21.15.60:0/1232564962 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b0011040 con 0x7fa6c00b1110 2024-04-25T10:06:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6b5ffb640 1 -- 172.21.15.60:0/1232564962 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b000ee40 con 0x7fa6c00b1110 2024-04-25T10:06:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 -- 172.21.15.60:0/1232564962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c0145350 con 0x7fa6c00b1110 2024-04-25T10:06:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.887+0000 7fa6c750c640 1 -- 172.21.15.60:0/1232564962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c0145910 con 0x7fa6c00b1110 2024-04-25T10:06:19.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.891+0000 7fa6c750c640 1 -- 172.21.15.60:0/1232564962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c0074770 con 0x7fa6c00b1110 2024-04-25T10:06:19.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.891+0000 7fa6b5ffb640 1 -- 172.21.15.60:0/1232564962 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6b00143a0 con 0x7fa6c00b1110 2024-04-25T10:06:19.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.891+0000 7fa6b5ffb640 1 --2- 172.21.15.60:0/1232564962 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa6a4066f90 0x7fa6a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.891+0000 7fa6b5ffb640 1 -- 172.21.15.60:0/1232564962 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa6b0087660 con 0x7fa6c00b1110 2024-04-25T10:06:19.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.891+0000 7fa6c52a8640 1 --2- 172.21.15.60:0/1232564962 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa6a4066f90 0x7fa6a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.891+0000 7fa6c52a8640 1 --2- 172.21.15.60:0/1232564962 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa6a4066f90 0x7fa6a4069450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc005490 tx=0x7fa6bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:19.895+0000 7fa6b5ffb640 1 -- 172.21.15.60:0/1232564962 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6b0058b10 con 0x7fa6c00b1110 2024-04-25T10:06:20.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.047+0000 7fa6c750c640 1 -- 172.21.15.60:0/1232564962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa6c0154740 con 0x7fa6c00b1110 2024-04-25T10:06:20.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.143+0000 7fa6b5ffb640 1 -- 172.21.15.60:0/1232564962 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa6b0058410 con 0x7fa6c00b1110 2024-04-25T10:06:20.149 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T10:06:20.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.147+0000 7fa6c750c640 1 -- 172.21.15.60:0/1232564962 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa6a4066f90 msgr2=0x7fa6a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.147+0000 7fa6c750c640 1 --2- 172.21.15.60:0/1232564962 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa6a4066f90 0x7fa6a4069450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc005490 tx=0x7fa6bc009040 comp rx=0 tx=0).stop 2024-04-25T10:06:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.147+0000 7fa6c750c640 1 -- 172.21.15.60:0/1232564962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6c00b1110 msgr2=0x7fa6c00b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.147+0000 7fa6c750c640 1 --2- 172.21.15.60:0/1232564962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6c00b1110 0x7fa6c00b1590 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0010aa0 tx=0x7fa6b0010f60 comp rx=0 tx=0).stop 2024-04-25T10:06:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.147+0000 7fa6c750c640 1 -- 172.21.15.60:0/1232564962 shutdown_connections 2024-04-25T10:06:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.147+0000 7fa6c750c640 1 --2- 172.21.15.60:0/1232564962 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa6a4066f90 0x7fa6a4069450 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.147+0000 7fa6c750c640 1 --2- 172.21.15.60:0/1232564962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6c0147240 0x7fa6c01476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.147+0000 7fa6c750c640 1 --2- 172.21.15.60:0/1232564962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6c00b1110 0x7fa6c00b1590 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.147+0000 7fa6c750c640 1 --2- 172.21.15.60:0/1232564962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6ac005220 0x7fa6c0144930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.147+0000 7fa6c750c640 1 -- 172.21.15.60:0/1232564962 >> 172.21.15.60:0/1232564962 conn(0x7fa6c006e400 msgr2=0x7fa6c00b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.147+0000 7fa6c750c640 1 -- 172.21.15.60:0/1232564962 shutdown_connections 2024-04-25T10:06:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.147+0000 7fa6c750c640 1 -- 172.21.15.60:0/1232564962 wait complete. 2024-04-25T10:06:20.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:06:20.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f8261f1b640 1 Processor -- start 2024-04-25T10:06:20.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f8261f1b640 1 -- start start 2024-04-25T10:06:20.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f8261f1b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f825c159ed0 0x7f825c0cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f8261f1b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f825c0aea20 0x7f825c071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f8261f1b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825c15a2b0 con 0x7f825c06d380 2024-04-25T10:06:20.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f8261f1b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825c0ce090 con 0x7f825c159ed0 2024-04-25T10:06:20.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f8261f1b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825c0ce370 con 0x7f825c0aea20 2024-04-25T10:06:20.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f825b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f825c159ed0 0x7f825c0cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f825b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f825c159ed0 0x7f825c0cf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37698/0 (socket says 172.21.15.60:37698) 2024-04-25T10:06:20.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f825b7fe640 1 -- 172.21.15.60:0/1849436681 learned_addr learned my addr 172.21.15.60:0/1849436681 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:20.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f825bfff640 1 --1- 172.21.15.60:0/1849436681 >> v1:172.21.15.60:6789/0 conn(0x7f825c06d380 0x7f825c0b2d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45812/0 (socket says 172.21.15.60:45812) 2024-04-25T10:06:20.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f825b7fe640 1 -- 172.21.15.60:0/1849436681 >> v2:172.21.15.100:3300/0 conn(0x7f825c0aea20 msgr2=0x7f825c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:20.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f825affd640 1 --2- 172.21.15.60:0/1849436681 >> v2:172.21.15.100:3300/0 conn(0x7f825c0aea20 0x7f825c071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f825a7fc640 1 -- 172.21.15.60:0/1849436681 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229695683 0 0) 0x7f825c15a2b0 con 0x7f825c06d380 2024-04-25T10:06:20.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f825b7fe640 1 --2- 172.21.15.60:0/1849436681 >> v2:172.21.15.100:3300/0 conn(0x7f825c0aea20 0x7f825c071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f825b7fe640 1 -- 172.21.15.60:0/1849436681 >> v1:172.21.15.60:6789/0 conn(0x7f825c06d380 legacy=0x7f825c0b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f825b7fe640 1 -- 172.21.15.60:0/1849436681 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c0ce650 con 0x7f825c159ed0 2024-04-25T10:06:20.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.299+0000 7f825affd640 1 --2- 172.21.15.60:0/1849436681 >> v2:172.21.15.100:3300/0 conn(0x7f825c0aea20 0x7f825c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:20.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f825b7fe640 1 --2- 172.21.15.60:0/1849436681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825c159ed0 0x7f825c0cf130 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f825c06ebd0 tx=0x7f825c0b3c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7c73c51f37f8ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f825a7fc640 1 -- 172.21.15.60:0/1849436681 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c0ad070 con 0x7f825c159ed0 2024-04-25T10:06:20.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f825a7fc640 1 -- 172.21.15.60:0/1849436681 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824c004230 con 0x7f825c159ed0 2024-04-25T10:06:20.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f825a7fc640 1 -- 172.21.15.60:0/1849436681 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c0a7780 con 0x7f825c159ed0 2024-04-25T10:06:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f8261f1b640 1 -- 172.21.15.60:0/1849436681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825c159ed0 msgr2=0x7f825c0cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f8261f1b640 1 --2- 172.21.15.60:0/1849436681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825c159ed0 0x7f825c0cf130 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f825c06ebd0 tx=0x7f825c0b3c40 comp rx=0 tx=0).stop 2024-04-25T10:06:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f8261f1b640 1 -- 172.21.15.60:0/1849436681 shutdown_connections 2024-04-25T10:06:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f8261f1b640 1 --2- 172.21.15.60:0/1849436681 >> v2:172.21.15.100:3300/0 conn(0x7f825c0aea20 0x7f825c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f8261f1b640 1 --2- 172.21.15.60:0/1849436681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825c159ed0 0x7f825c0cf130 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f8261f1b640 1 -- 172.21.15.60:0/1849436681 >> 172.21.15.60:0/1849436681 conn(0x7f825c067060 msgr2=0x7f825c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f8261f1b640 1 -- 172.21.15.60:0/1849436681 shutdown_connections 2024-04-25T10:06:20.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f8261f1b640 1 -- 172.21.15.60:0/1849436681 wait complete. 2024-04-25T10:06:20.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f8261f1b640 1 Processor -- start 2024-04-25T10:06:20.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f8261f1b640 1 -- start start 2024-04-25T10:06:20.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f8261f1b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f825c06d380 0x7f825c0b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f8261f1b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f825c0aea20 0x7f825c14a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f8261f1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825c159ed0 0x7f825c0aa910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f8261f1b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0714a0 con 0x7f825c159ed0 2024-04-25T10:06:20.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f8261f1b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c071320 con 0x7f825c0aea20 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f8261f1b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c071620 con 0x7f825c06d380 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.303+0000 7f825bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825c159ed0 0x7f825c0aa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f825bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825c159ed0 0x7f825c0aa910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37702/0 (socket says 172.21.15.60:37702) 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f825bfff640 1 -- 172.21.15.60:0/1391233058 learned_addr learned my addr 172.21.15.60:0/1391233058 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f825affd640 1 --2- 172.21.15.60:0/1391233058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f825c0aea20 0x7f825c14a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f825b7fe640 1 --2- 172.21.15.60:0/1391233058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f825c06d380 0x7f825c0b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f825bfff640 1 -- 172.21.15.60:0/1391233058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f825c06d380 msgr2=0x7f825c0b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f825bfff640 1 --2- 172.21.15.60:0/1391233058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f825c06d380 0x7f825c0b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f825bfff640 1 -- 172.21.15.60:0/1391233058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f825c0aea20 msgr2=0x7f825c14a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f825bfff640 1 --2- 172.21.15.60:0/1391233058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f825c0aea20 0x7f825c14a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f825bfff640 1 -- 172.21.15.60:0/1391233058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c1478f0 con 0x7f825c159ed0 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f825b7fe640 1 --2- 172.21.15.60:0/1391233058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f825c06d380 0x7f825c0b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f825bfff640 1 --2- 172.21.15.60:0/1391233058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825c159ed0 0x7f825c0aa910 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f825400c7d0 tx=0x7f825400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f8243fff640 1 -- 172.21.15.60:0/1391233058 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8254018070 con 0x7f825c159ed0 2024-04-25T10:06:20.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f8243fff640 1 -- 172.21.15.60:0/1391233058 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825400a640 con 0x7f825c159ed0 2024-04-25T10:06:20.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f8261f1b640 1 -- 172.21.15.60:0/1391233058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825c147e80 con 0x7f825c159ed0 2024-04-25T10:06:20.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f8243fff640 1 -- 172.21.15.60:0/1391233058 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825400aa70 con 0x7f825c159ed0 2024-04-25T10:06:20.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f8261f1b640 1 -- 172.21.15.60:0/1391233058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f825c148440 con 0x7f825c159ed0 2024-04-25T10:06:20.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f8261f1b640 1 -- 172.21.15.60:0/1391233058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f825c149030 con 0x7f825c159ed0 2024-04-25T10:06:20.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f8243fff640 1 -- 172.21.15.60:0/1391233058 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f825401b070 con 0x7f825c159ed0 2024-04-25T10:06:20.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f8243fff640 1 --2- 172.21.15.60:0/1391233058 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8248066f60 0x7f8248069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f8243fff640 1 -- 172.21.15.60:0/1391233058 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f82540845a0 con 0x7f825c159ed0 2024-04-25T10:06:20.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.307+0000 7f825b7fe640 1 --2- 172.21.15.60:0/1391233058 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8248066f60 0x7f8248069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.311+0000 7f825b7fe640 1 --2- 172.21.15.60:0/1391233058 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8248066f60 0x7f8248069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f824c0b1000 tx=0x7f824c004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.311+0000 7f8243fff640 1 -- 172.21.15.60:0/1391233058 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f82540552b0 con 0x7f825c159ed0 2024-04-25T10:06:20.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:20.463+0000 7f8261f1b640 1 -- 172.21.15.60:0/1391233058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f825c074770 con 0x7f825c159ed0 2024-04-25T10:06:21.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.147+0000 7f8243fff640 1 -- 172.21.15.60:0/1391233058 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8254059350 con 0x7f825c159ed0 2024-04-25T10:06:21.154 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T10:06:21.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.151+0000 7f8261f1b640 1 -- 172.21.15.60:0/1391233058 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8248066f60 msgr2=0x7f8248069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.151+0000 7f8261f1b640 1 --2- 172.21.15.60:0/1391233058 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8248066f60 0x7f8248069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f824c0b1000 tx=0x7f824c004740 comp rx=0 tx=0).stop 2024-04-25T10:06:21.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.151+0000 7f8261f1b640 1 -- 172.21.15.60:0/1391233058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825c159ed0 msgr2=0x7f825c0aa910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.151+0000 7f8261f1b640 1 --2- 172.21.15.60:0/1391233058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825c159ed0 0x7f825c0aa910 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f825400c7d0 tx=0x7f825400cc90 comp rx=0 tx=0).stop 2024-04-25T10:06:21.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.151+0000 7f8261f1b640 1 -- 172.21.15.60:0/1391233058 shutdown_connections 2024-04-25T10:06:21.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.151+0000 7f8261f1b640 1 --2- 172.21.15.60:0/1391233058 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8248066f60 0x7f8248069420 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.151+0000 7f8261f1b640 1 --2- 172.21.15.60:0/1391233058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825c159ed0 0x7f825c0aa910 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.151+0000 7f8261f1b640 1 --2- 172.21.15.60:0/1391233058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f825c0aea20 0x7f825c14a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.151+0000 7f8261f1b640 1 --2- 172.21.15.60:0/1391233058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f825c06d380 0x7f825c0b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.151+0000 7f8261f1b640 1 -- 172.21.15.60:0/1391233058 >> 172.21.15.60:0/1391233058 conn(0x7f825c067060 msgr2=0x7f825c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:21.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.151+0000 7f8261f1b640 1 -- 172.21.15.60:0/1391233058 shutdown_connections 2024-04-25T10:06:21.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.151+0000 7f8261f1b640 1 -- 172.21.15.60:0/1391233058 wait complete. 2024-04-25T10:06:21.196 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:06:21.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.311+0000 7febecce5640 1 Processor -- start 2024-04-25T10:06:21.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.315+0000 7febecce5640 1 -- start start 2024-04-25T10:06:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.315+0000 7febecce5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febe80aecc0 0x7febe80afe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.315+0000 7febecce5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7febe815bbf0 0x7febe80b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.315+0000 7febecce5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe80af6d0 con 0x7febe815c050 2024-04-25T10:06:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.315+0000 7febecce5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe815cba0 con 0x7febe815bbf0 2024-04-25T10:06:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.315+0000 7febecce5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe815c5e0 con 0x7febe80aecc0 2024-04-25T10:06:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.315+0000 7febdffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7febe815bbf0 0x7febe80b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.315+0000 7febe77fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febe80aecc0 0x7febe80afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.315+0000 7febe7fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7febe815c050 0x7febe815cd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:37458/0 (socket says 172.21.15.60:37458) 2024-04-25T10:06:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.315+0000 7febe7fff640 1 -- 172.21.15.60:0/2325880363 learned_addr learned my addr 172.21.15.60:0/2325880363 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.315+0000 7febdffff640 1 -- 172.21.15.60:0/2325880363 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febe80aecc0 msgr2=0x7febe80afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.315+0000 7febdffff640 1 --2- 172.21.15.60:0/2325880363 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febe80aecc0 0x7febe80afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.315+0000 7febdffff640 1 -- 172.21.15.60:0/2325880363 >> v1:172.21.15.100:6789/0 conn(0x7febe815c050 legacy=0x7febe815cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.315+0000 7febdffff640 1 -- 172.21.15.60:0/2325880363 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe815c8c0 con 0x7febe815bbf0 2024-04-25T10:06:21.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febdffff640 1 --2- 172.21.15.60:0/2325880363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febe815bbf0 0x7febe80b4930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7febd00097c0 tx=0x7febd001ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42843970639e3533 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:21.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febe6ffd640 1 -- 172.21.15.60:0/2325880363 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd00b5070 con 0x7febe815bbf0 2024-04-25T10:06:21.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febe6ffd640 1 -- 172.21.15.60:0/2325880363 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd00a8990 con 0x7febe815bbf0 2024-04-25T10:06:21.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febe6ffd640 1 -- 172.21.15.60:0/2325880363 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd00a8d60 con 0x7febe815bbf0 2024-04-25T10:06:21.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febecce5640 1 -- 172.21.15.60:0/2325880363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febe815bbf0 msgr2=0x7febe80b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febecce5640 1 --2- 172.21.15.60:0/2325880363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febe815bbf0 0x7febe80b4930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7febd00097c0 tx=0x7febd001ec90 comp rx=0 tx=0).stop 2024-04-25T10:06:21.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febecce5640 1 -- 172.21.15.60:0/2325880363 shutdown_connections 2024-04-25T10:06:21.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febecce5640 1 --2- 172.21.15.60:0/2325880363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febe815bbf0 0x7febe80b4930 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febecce5640 1 --2- 172.21.15.60:0/2325880363 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febe80aecc0 0x7febe80afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febecce5640 1 -- 172.21.15.60:0/2325880363 >> 172.21.15.60:0/2325880363 conn(0x7febe806afe0 msgr2=0x7febe806b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:21.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febecce5640 1 -- 172.21.15.60:0/2325880363 shutdown_connections 2024-04-25T10:06:21.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febecce5640 1 -- 172.21.15.60:0/2325880363 wait complete. 2024-04-25T10:06:21.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febecce5640 1 Processor -- start 2024-04-25T10:06:21.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febecce5640 1 -- start start 2024-04-25T10:06:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febecce5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febe80aecc0 0x7febe8144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febecce5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febe815bbf0 0x7febe80a8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febecce5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febe815c050 0x7febe815a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febecce5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe80cd450 con 0x7febe815bbf0 2024-04-25T10:06:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febecce5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe80af0a0 con 0x7febe80aecc0 2024-04-25T10:06:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febecce5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe80cd5d0 con 0x7febe815c050 2024-04-25T10:06:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febdffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febe815bbf0 0x7febe80a8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febdffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febe815bbf0 0x7febe80a8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37706/0 (socket says 172.21.15.60:37706) 2024-04-25T10:06:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.319+0000 7febdffff640 1 -- 172.21.15.60:0/3875992969 learned_addr learned my addr 172.21.15.60:0/3875992969 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:21.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febe77fe640 1 --2- 172.21.15.60:0/3875992969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febe80aecc0 0x7febe8144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febe7fff640 1 --2- 172.21.15.60:0/3875992969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febe815c050 0x7febe815a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febdffff640 1 -- 172.21.15.60:0/3875992969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febe815c050 msgr2=0x7febe815a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febdffff640 1 --2- 172.21.15.60:0/3875992969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febe815c050 0x7febe815a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febdffff640 1 -- 172.21.15.60:0/3875992969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febe80aecc0 msgr2=0x7febe8144f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febdffff640 1 --2- 172.21.15.60:0/3875992969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febe80aecc0 0x7febe8144f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febdffff640 1 -- 172.21.15.60:0/3875992969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe80a9a80 con 0x7febe815bbf0 2024-04-25T10:06:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febe7fff640 1 --2- 172.21.15.60:0/3875992969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febe815c050 0x7febe815a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febdffff640 1 --2- 172.21.15.60:0/3875992969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febe815bbf0 0x7febe80a8fe0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7febd00b3040 tx=0x7febd001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febe4ff9640 1 -- 172.21.15.60:0/3875992969 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd00bf070 con 0x7febe815bbf0 2024-04-25T10:06:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febe4ff9640 1 -- 172.21.15.60:0/3875992969 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd00b0740 con 0x7febe815bbf0 2024-04-25T10:06:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febecce5640 1 -- 172.21.15.60:0/3875992969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe80a9d60 con 0x7febe815bbf0 2024-04-25T10:06:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febe4ff9640 1 -- 172.21.15.60:0/3875992969 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd00b5040 con 0x7febe815bbf0 2024-04-25T10:06:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febecce5640 1 -- 172.21.15.60:0/3875992969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe80aa320 con 0x7febe815bbf0 2024-04-25T10:06:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febecce5640 1 -- 172.21.15.60:0/3875992969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe8074770 con 0x7febe815bbf0 2024-04-25T10:06:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febe4ff9640 1 -- 172.21.15.60:0/3875992969 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7febd00b0b30 con 0x7febe815bbf0 2024-04-25T10:06:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febe4ff9640 1 --2- 172.21.15.60:0/3875992969 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7febc8066f60 0x7febc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febe4ff9640 1 -- 172.21.15.60:0/3875992969 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5609+0+0 (secure 0 0 0) 0x7febd0121910 con 0x7febe815bbf0 2024-04-25T10:06:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.323+0000 7febe77fe640 1 --2- 172.21.15.60:0/3875992969 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7febc8066f60 0x7febc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.327+0000 7febe77fe640 1 --2- 172.21.15.60:0/3875992969 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7febc8066f60 0x7febc8069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7febd8009b60 tx=0x7febd8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:21.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.327+0000 7febe4ff9640 1 -- 172.21.15.60:0/3875992969 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7febd00187f0 con 0x7febe815bbf0 2024-04-25T10:06:21.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:21.527+0000 7febecce5640 1 -- 172.21.15.60:0/3875992969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7febe80aff30 con 0x7febe815bbf0 2024-04-25T10:06:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.195+0000 7febe4ff9640 1 -- 172.21.15.60:0/3875992969 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7febd00b0e20 con 0x7febe815bbf0 2024-04-25T10:06:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 4 and data pool 5 2024-04-25T10:06:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.199+0000 7febecce5640 1 -- 172.21.15.60:0/3875992969 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7febc8066f60 msgr2=0x7febc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.199+0000 7febecce5640 1 --2- 172.21.15.60:0/3875992969 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7febc8066f60 0x7febc8069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7febd8009b60 tx=0x7febd8009190 comp rx=0 tx=0).stop 2024-04-25T10:06:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.199+0000 7febecce5640 1 -- 172.21.15.60:0/3875992969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febe815bbf0 msgr2=0x7febe80a8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.199+0000 7febecce5640 1 --2- 172.21.15.60:0/3875992969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febe815bbf0 0x7febe80a8fe0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7febd00b3040 tx=0x7febd001ba30 comp rx=0 tx=0).stop 2024-04-25T10:06:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.199+0000 7febecce5640 1 -- 172.21.15.60:0/3875992969 shutdown_connections 2024-04-25T10:06:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.199+0000 7febecce5640 1 --2- 172.21.15.60:0/3875992969 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7febc8066f60 0x7febc8069420 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.199+0000 7febecce5640 1 --2- 172.21.15.60:0/3875992969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7febe815c050 0x7febe815a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.199+0000 7febecce5640 1 --2- 172.21.15.60:0/3875992969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febe815bbf0 0x7febe80a8fe0 secure :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7febd00b3040 tx=0x7febd001ba30 comp rx=0 tx=0).stop 2024-04-25T10:06:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.199+0000 7febecce5640 1 --2- 172.21.15.60:0/3875992969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febe80aecc0 0x7febe8144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.199+0000 7febecce5640 1 -- 172.21.15.60:0/3875992969 >> 172.21.15.60:0/3875992969 conn(0x7febe806afe0 msgr2=0x7febe8072220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.199+0000 7febecce5640 1 -- 172.21.15.60:0/3875992969 shutdown_connections 2024-04-25T10:06:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.203+0000 7febecce5640 1 -- 172.21.15.60:0/3875992969 wait complete. 2024-04-25T10:06:22.250 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:22.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.363+0000 7fcefc738640 1 Processor -- start 2024-04-25T10:06:22.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.363+0000 7fcefc738640 1 -- start start 2024-04-25T10:06:22.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.363+0000 7fcefc738640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcef40aed10 0x7fcef4159e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.363+0000 7fcefc738640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcef406bc20 0x7fcef406c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.363+0000 7fcefc738640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef40b2d90 con 0x7fcef406ff30 2024-04-25T10:06:22.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.363+0000 7fcefc738640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef40b4110 con 0x7fcef406bc20 2024-04-25T10:06:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.363+0000 7fcefc738640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef40670b0 con 0x7fcef40aed10 2024-04-25T10:06:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.363+0000 7fcefacd5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcef406bc20 0x7fcef406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.363+0000 7fcefacd5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcef406bc20 0x7fcef406c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37714/0 (socket says 172.21.15.60:37714) 2024-04-25T10:06:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.363+0000 7fcefacd5640 1 -- 172.21.15.60:0/2435027279 learned_addr learned my addr 172.21.15.60:0/2435027279 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.363+0000 7fcef9cd3640 1 --1- 172.21.15.60:0/2435027279 >> v1:172.21.15.60:6789/0 conn(0x7fcef406ff30 0x7fcef40669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45816/0 (socket says 172.21.15.60:45816) 2024-04-25T10:06:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.363+0000 7fcefa4d4640 1 --2- 172.21.15.60:0/2435027279 >> v2:172.21.15.100:3300/0 conn(0x7fcef40aed10 0x7fcef4159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.363+0000 7fcefacd5640 1 -- 172.21.15.60:0/2435027279 >> v2:172.21.15.100:3300/0 conn(0x7fcef40aed10 msgr2=0x7fcef4159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.363+0000 7fcefacd5640 1 --2- 172.21.15.60:0/2435027279 >> v2:172.21.15.100:3300/0 conn(0x7fcef40aed10 0x7fcef4159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.363+0000 7fcefacd5640 1 -- 172.21.15.60:0/2435027279 >> v1:172.21.15.60:6789/0 conn(0x7fcef406ff30 legacy=0x7fcef40669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.363+0000 7fcefacd5640 1 -- 172.21.15.60:0/2435027279 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef40b3070 con 0x7fcef406bc20 2024-04-25T10:06:22.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.367+0000 7fcefacd5640 1 --2- 172.21.15.60:0/2435027279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef406bc20 0x7fcef406c070 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcef4067360 tx=0x7fceec01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8d6220ee160400 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:22.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.367+0000 7fcef94d2640 1 -- 172.21.15.60:0/2435027279 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec00b2d0 con 0x7fcef406bc20 2024-04-25T10:06:22.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.367+0000 7fcef94d2640 1 -- 172.21.15.60:0/2435027279 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec00b470 con 0x7fcef406bc20 2024-04-25T10:06:22.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.367+0000 7fcef94d2640 1 -- 172.21.15.60:0/2435027279 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0a7ae0 con 0x7fcef406bc20 2024-04-25T10:06:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.367+0000 7fcefc738640 1 -- 172.21.15.60:0/2435027279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef406bc20 msgr2=0x7fcef406c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.367+0000 7fcefc738640 1 --2- 172.21.15.60:0/2435027279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef406bc20 0x7fcef406c070 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcef4067360 tx=0x7fceec01ebf0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.367+0000 7fcefc738640 1 -- 172.21.15.60:0/2435027279 shutdown_connections 2024-04-25T10:06:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.367+0000 7fcefc738640 1 --2- 172.21.15.60:0/2435027279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef406bc20 0x7fcef406c070 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.367+0000 7fcefc738640 1 --2- 172.21.15.60:0/2435027279 >> v2:172.21.15.100:3300/0 conn(0x7fcef40aed10 0x7fcef4159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.367+0000 7fcefc738640 1 -- 172.21.15.60:0/2435027279 >> 172.21.15.60:0/2435027279 conn(0x7fcef4071b90 msgr2=0x7fcef4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.367+0000 7fcefc738640 1 -- 172.21.15.60:0/2435027279 shutdown_connections 2024-04-25T10:06:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.367+0000 7fcefc738640 1 -- 172.21.15.60:0/2435027279 wait complete. 2024-04-25T10:06:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.367+0000 7fcefc738640 1 Processor -- start 2024-04-25T10:06:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.367+0000 7fcefc738640 1 -- start start 2024-04-25T10:06:22.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefc738640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef406bc20 0x7fcef4138780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefc738640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcef406ff30 0x7fcef40ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefc738640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef40aed10 0x7fcef4151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefc738640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef40b3c80 con 0x7fcef406bc20 2024-04-25T10:06:22.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefc738640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef415a3b0 con 0x7fcef40aed10 2024-04-25T10:06:22.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefc738640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef40ad580 con 0x7fcef406ff30 2024-04-25T10:06:22.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefa4d4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef406bc20 0x7fcef4138780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefa4d4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef406bc20 0x7fcef4138780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37718/0 (socket says 172.21.15.60:37718) 2024-04-25T10:06:22.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefa4d4640 1 -- 172.21.15.60:0/2510103741 learned_addr learned my addr 172.21.15.60:0/2510103741 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:22.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcef9cd3640 1 --2- 172.21.15.60:0/2510103741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcef406ff30 0x7fcef40ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefacd5640 1 --2- 172.21.15.60:0/2510103741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef40aed10 0x7fcef4151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefa4d4640 1 -- 172.21.15.60:0/2510103741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcef406ff30 msgr2=0x7fcef40ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefa4d4640 1 --2- 172.21.15.60:0/2510103741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcef406ff30 0x7fcef40ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefa4d4640 1 -- 172.21.15.60:0/2510103741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef40aed10 msgr2=0x7fcef4151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefa4d4640 1 --2- 172.21.15.60:0/2510103741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef40aed10 0x7fcef4151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefa4d4640 1 -- 172.21.15.60:0/2510103741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef40cc860 con 0x7fcef406bc20 2024-04-25T10:06:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefacd5640 1 --2- 172.21.15.60:0/2510103741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef40aed10 0x7fcef4151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefa4d4640 1 --2- 172.21.15.60:0/2510103741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef406bc20 0x7fcef4138780 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcef000d620 tx=0x7fcef000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fceeaffd640 1 -- 172.21.15.60:0/2510103741 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef000b620 con 0x7fcef406bc20 2024-04-25T10:06:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fceeaffd640 1 -- 172.21.15.60:0/2510103741 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef0019070 con 0x7fcef406bc20 2024-04-25T10:06:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefc738640 1 -- 172.21.15.60:0/2510103741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef40ccb40 con 0x7fcef406bc20 2024-04-25T10:06:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fceeaffd640 1 -- 172.21.15.60:0/2510103741 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef000b9e0 con 0x7fcef406bc20 2024-04-25T10:06:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefc738640 1 -- 172.21.15.60:0/2510103741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef40cd100 con 0x7fcef406bc20 2024-04-25T10:06:22.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcefc738640 1 -- 172.21.15.60:0/2510103741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef4074770 con 0x7fcef406bc20 2024-04-25T10:06:22.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fceeaffd640 1 -- 172.21.15.60:0/2510103741 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcef000bb80 con 0x7fcef406bc20 2024-04-25T10:06:22.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fceeaffd640 1 --2- 172.21.15.60:0/2510103741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fced8066f60 0x7fced8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fceeaffd640 1 -- 172.21.15.60:0/2510103741 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcef0085440 con 0x7fcef406bc20 2024-04-25T10:06:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.371+0000 7fcef9cd3640 1 --2- 172.21.15.60:0/2510103741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fced8066f60 0x7fced8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.375+0000 7fcef9cd3640 1 --2- 172.21.15.60:0/2510103741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fced8066f60 0x7fced8069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcee4000f70 tx=0x7fcee4007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:22.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.375+0000 7fceeaffd640 1 -- 172.21.15.60:0/2510103741 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcef0056450 con 0x7fcef406bc20 2024-04-25T10:06:22.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.543+0000 7fcefc738640 1 -- 172.21.15.60:0/2510103741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcef4153040 con 0x7fcef406bc20 2024-04-25T10:06:22.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.543+0000 7fceeaffd640 1 -- 172.21.15.60:0/2510103741 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16694 (secure 0 0 0) 0x7fcef0055d50 con 0x7fcef406bc20 2024-04-25T10:06:22.552 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:22.552 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":34,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:21.529112+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:20.088165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:20.507719+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.547+0000 7fcefc738640 1 -- 172.21.15.60:0/2510103741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fced8066f60 msgr2=0x7fced8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.547+0000 7fcefc738640 1 --2- 172.21.15.60:0/2510103741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fced8066f60 0x7fced8069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcee4000f70 tx=0x7fcee4007c90 comp rx=0 tx=0).stop 2024-04-25T10:06:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.547+0000 7fcefc738640 1 -- 172.21.15.60:0/2510103741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef406bc20 msgr2=0x7fcef4138780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.547+0000 7fcefc738640 1 --2- 172.21.15.60:0/2510103741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef406bc20 0x7fcef4138780 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcef000d620 tx=0x7fcef000dae0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.547+0000 7fcefc738640 1 -- 172.21.15.60:0/2510103741 shutdown_connections 2024-04-25T10:06:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.547+0000 7fcefc738640 1 --2- 172.21.15.60:0/2510103741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fced8066f60 0x7fced8069420 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.547+0000 7fcefc738640 1 --2- 172.21.15.60:0/2510103741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef40aed10 0x7fcef4151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.547+0000 7fcefc738640 1 --2- 172.21.15.60:0/2510103741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcef406ff30 0x7fcef40ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.547+0000 7fcefc738640 1 --2- 172.21.15.60:0/2510103741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef406bc20 0x7fcef4138780 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.547+0000 7fcefc738640 1 -- 172.21.15.60:0/2510103741 >> 172.21.15.60:0/2510103741 conn(0x7fcef4071b90 msgr2=0x7fcef406f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:22.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.547+0000 7fcefc738640 1 -- 172.21.15.60:0/2510103741 shutdown_connections 2024-04-25T10:06:22.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.547+0000 7fcefc738640 1 -- 172.21.15.60:0/2510103741 wait complete. 2024-04-25T10:06:22.571 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:22.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.683+0000 7f63e354c640 1 Processor -- start 2024-04-25T10:06:22.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.683+0000 7f63e354c640 1 -- start start 2024-04-25T10:06:22.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.683+0000 7f63e354c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63dc0ce7a0 0x7f63dc0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.683+0000 7f63e354c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63dc0b33f0 con 0x7f63dc15bc90 2024-04-25T10:06:22.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.683+0000 7f63e354c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63dc0cf0c0 con 0x7f63dc0b4660 2024-04-25T10:06:22.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.683+0000 7f63e354c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63dc0cc370 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.683+0000 7f63e12e8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63dc0ce7a0 0x7f63dc0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.683+0000 7f63e0ae7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f63dc15bc90 0x7f63dc0b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45832/0 (socket says 172.21.15.60:45832) 2024-04-25T10:06:22.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.683+0000 7f63e0ae7640 1 -- 172.21.15.60:0/3062501279 learned_addr learned my addr 172.21.15.60:0/3062501279 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:22.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.683+0000 7f63d3fff640 1 -- 172.21.15.60:0/3062501279 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841563923 0 0) 0x7f63dc0b33f0 con 0x7f63dc15bc90 2024-04-25T10:06:22.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.683+0000 7f63d3fff640 1 -- 172.21.15.60:0/3062501279 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63c8003530 con 0x7f63dc15bc90 2024-04-25T10:06:22.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.683+0000 7f63e12e8640 1 -- 172.21.15.60:0/3062501279 >> v1:172.21.15.100:6789/0 conn(0x7f63dc0b4660 legacy=0x7f63dc0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:22.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.683+0000 7f63e12e8640 1 -- 172.21.15.60:0/3062501279 >> v1:172.21.15.60:6789/0 conn(0x7f63dc15bc90 legacy=0x7f63dc0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.683+0000 7f63e12e8640 1 -- 172.21.15.60:0/3062501279 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63dc071ca0 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.687+0000 7f63e12e8640 1 --2- 172.21.15.60:0/3062501279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc0ce7a0 0x7f63dc0ceb80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f63d80097c0 tx=0x7f63d801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27def0209b7e0db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:22.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.687+0000 7f63d3fff640 1 -- 172.21.15.60:0/3062501279 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d80b0070 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.687+0000 7f63d3fff640 1 -- 172.21.15.60:0/3062501279 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d80ab600 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.687+0000 7f63d3fff640 1 -- 172.21.15.60:0/3062501279 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d80ab9d0 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.687+0000 7f63e354c640 1 -- 172.21.15.60:0/3062501279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc0ce7a0 msgr2=0x7f63dc0ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.687+0000 7f63e354c640 1 --2- 172.21.15.60:0/3062501279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc0ce7a0 0x7f63dc0ceb80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f63d80097c0 tx=0x7f63d801c460 comp rx=0 tx=0).stop 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.687+0000 7f63e354c640 1 -- 172.21.15.60:0/3062501279 shutdown_connections 2024-04-25T10:06:22.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.687+0000 7f63e354c640 1 --2- 172.21.15.60:0/3062501279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc0ce7a0 0x7f63dc0ceb80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.687+0000 7f63e354c640 1 -- 172.21.15.60:0/3062501279 >> 172.21.15.60:0/3062501279 conn(0x7f63dc0b12d0 msgr2=0x7f63dc0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:22.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.687+0000 7f63e354c640 1 -- 172.21.15.60:0/3062501279 shutdown_connections 2024-04-25T10:06:22.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.687+0000 7f63e354c640 1 -- 172.21.15.60:0/3062501279 wait complete. 2024-04-25T10:06:22.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.687+0000 7f63e354c640 1 Processor -- start 2024-04-25T10:06:22.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.687+0000 7f63e354c640 1 -- start start 2024-04-25T10:06:22.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e354c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc0ce7a0 0x7f63dc15c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e354c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc15bc90 0x7f63dc15c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e354c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63dc15aa60 0x7f63dc1559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e354c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc0b3d20 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e354c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc0b3ba0 con 0x7f63dc15bc90 2024-04-25T10:06:22.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e354c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc0b4280 con 0x7f63dc15aa60 2024-04-25T10:06:22.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e12e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc0ce7a0 0x7f63dc15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e12e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc0ce7a0 0x7f63dc15c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37732/0 (socket says 172.21.15.60:37732) 2024-04-25T10:06:22.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e12e8640 1 -- 172.21.15.60:0/1185005518 learned_addr learned my addr 172.21.15.60:0/1185005518 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:22.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e1ae9640 1 --2- 172.21.15.60:0/1185005518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63dc15aa60 0x7f63dc1559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e0ae7640 1 --2- 172.21.15.60:0/1185005518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc15bc90 0x7f63dc15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e12e8640 1 -- 172.21.15.60:0/1185005518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63dc15aa60 msgr2=0x7f63dc1559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e12e8640 1 --2- 172.21.15.60:0/1185005518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63dc15aa60 0x7f63dc1559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e12e8640 1 -- 172.21.15.60:0/1185005518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc15bc90 msgr2=0x7f63dc15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e12e8640 1 --2- 172.21.15.60:0/1185005518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc15bc90 0x7f63dc15c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e12e8640 1 -- 172.21.15.60:0/1185005518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63dc071ad0 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e12e8640 1 --2- 172.21.15.60:0/1185005518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc0ce7a0 0x7f63dc15c220 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f63d80ae040 tx=0x7f63d801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:22.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63d1ffb640 1 -- 172.21.15.60:0/1185005518 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d80b0050 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63d1ffb640 1 -- 172.21.15.60:0/1185005518 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d80b1040 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63d1ffb640 1 -- 172.21.15.60:0/1185005518 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d8006e20 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e354c640 1 -- 172.21.15.60:0/1185005518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63dc1599c0 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e354c640 1 -- 172.21.15.60:0/1185005518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63dc159f80 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e354c640 1 -- 172.21.15.60:0/1185005518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63dc074770 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63d1ffb640 1 -- 172.21.15.60:0/1185005518 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63d80be030 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63d1ffb640 1 --2- 172.21.15.60:0/1185005518 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63c0066f60 0x7f63c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63d1ffb640 1 -- 172.21.15.60:0/1185005518 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63d811ec50 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.691+0000 7f63e0ae7640 1 --2- 172.21.15.60:0/1185005518 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63c0066f60 0x7f63c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.695+0000 7f63e0ae7640 1 --2- 172.21.15.60:0/1185005518 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63c0066f60 0x7f63c0069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f63cc007fd0 tx=0x7f63cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:22.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.695+0000 7f63d1ffb640 1 -- 172.21.15.60:0/1185005518 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f63d8124020 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.859+0000 7f63e354c640 1 -- 172.21.15.60:0/1185005518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63dc156bb0 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.859+0000 7f63d1ffb640 1 -- 172.21.15.60:0/1185005518 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16694 (secure 0 0 0) 0x7f63d80c5060 con 0x7f63dc0ce7a0 2024-04-25T10:06:22.867 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:22.867 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":34,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:21.529112+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:20.088165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:20.507719+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:22.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.863+0000 7f63e354c640 1 -- 172.21.15.60:0/1185005518 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63c0066f60 msgr2=0x7f63c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.863+0000 7f63e354c640 1 --2- 172.21.15.60:0/1185005518 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63c0066f60 0x7f63c0069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f63cc007fd0 tx=0x7f63cc009040 comp rx=0 tx=0).stop 2024-04-25T10:06:22.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.863+0000 7f63e354c640 1 -- 172.21.15.60:0/1185005518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc0ce7a0 msgr2=0x7f63dc15c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.863+0000 7f63e354c640 1 --2- 172.21.15.60:0/1185005518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc0ce7a0 0x7f63dc15c220 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f63d80ae040 tx=0x7f63d801aca0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.863+0000 7f63e354c640 1 -- 172.21.15.60:0/1185005518 shutdown_connections 2024-04-25T10:06:22.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.863+0000 7f63e354c640 1 --2- 172.21.15.60:0/1185005518 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63c0066f60 0x7f63c0069420 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.863+0000 7f63e354c640 1 --2- 172.21.15.60:0/1185005518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63dc15aa60 0x7f63dc1559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.863+0000 7f63e354c640 1 --2- 172.21.15.60:0/1185005518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc15bc90 0x7f63dc15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.863+0000 7f63e354c640 1 --2- 172.21.15.60:0/1185005518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc0ce7a0 0x7f63dc15c220 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.863+0000 7f63e354c640 1 -- 172.21.15.60:0/1185005518 >> 172.21.15.60:0/1185005518 conn(0x7f63dc0b12d0 msgr2=0x7f63dc0cb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:22.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.863+0000 7f63e354c640 1 -- 172.21.15.60:0/1185005518 shutdown_connections 2024-04-25T10:06:22.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.863+0000 7f63e354c640 1 -- 172.21.15.60:0/1185005518 wait complete. 2024-04-25T10:06:22.885 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:06:23.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.995+0000 7f04ca2c0640 1 Processor -- start 2024-04-25T10:06:23.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.995+0000 7f04ca2c0640 1 -- start start 2024-04-25T10:06:23.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04ca2c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04c40cbcf0 0x7f04c4160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04ca2c0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c4144a40 con 0x7f04c4071c30 2024-04-25T10:06:23.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04ca2c0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c40cc9f0 con 0x7f04c415fbf0 2024-04-25T10:06:23.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04ca2c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40b2dc0 con 0x7f04c40cbcf0 2024-04-25T10:06:23.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f04c4071c30 0x7f04c415f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45834/0 (socket says 172.21.15.60:45834) 2024-04-25T10:06:23.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c37fe640 1 -- 172.21.15.60:0/1981994559 learned_addr learned my addr 172.21.15.60:0/1981994559 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:23.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c3fff640 1 --2- 172.21.15.60:0/1981994559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04c40cbcf0 0x7f04c4160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 -- 172.21.15.60:0/1981994559 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411271082 0 0) 0x7f04c4144a40 con 0x7f04c4071c30 2024-04-25T10:06:23.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 -- 172.21.15.60:0/1981994559 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04ac003530 con 0x7f04c4071c30 2024-04-25T10:06:23.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 -- 172.21.15.60:0/1981994559 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3919384101 0 0) 0x7f04ac003530 con 0x7f04c4071c30 2024-04-25T10:06:23.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 -- 172.21.15.60:0/1981994559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04c40cbcf0 msgr2=0x7f04c4160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 --2- 172.21.15.60:0/1981994559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04c40cbcf0 0x7f04c4160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 -- 172.21.15.60:0/1981994559 >> v1:172.21.15.100:6789/0 conn(0x7f04c415fbf0 legacy=0x7f04c40ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 -- 172.21.15.60:0/1981994559 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c40cccd0 con 0x7f04c4071c30 2024-04-25T10:06:23.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 -- 172.21.15.60:0/1981994559 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f04b8001680 con 0x7f04c4071c30 2024-04-25T10:06:23.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 -- 172.21.15.60:0/1981994559 >> v1:172.21.15.60:6789/0 conn(0x7f04c4071c30 legacy=0x7f04c415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 --2- 172.21.15.60:0/1981994559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04ac0039f0 0x7f04ac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 --2- 172.21.15.60:0/1981994559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04ac004810 0x7f04ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 --2- 172.21.15.60:0/1981994559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04c40cbcf0 0x7f04ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 -- 172.21.15.60:0/1981994559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac00ba80 con 0x7f04ac004810 2024-04-25T10:06:23.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 -- 172.21.15.60:0/1981994559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac00bc00 con 0x7f04ac0039f0 2024-04-25T10:06:23.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 -- 172.21.15.60:0/1981994559 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac00be00 con 0x7f04c40cbcf0 2024-04-25T10:06:23.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c37fe640 1 --2- 172.21.15.60:0/1981994559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04ac004810 0x7f04ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c3fff640 1 --2- 172.21.15.60:0/1981994559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04ac0039f0 0x7f04ac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c885d640 1 --2- 172.21.15.60:0/1981994559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04c40cbcf0 0x7f04ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c37fe640 1 -- 172.21.15.60:0/1981994559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04c40cbcf0 msgr2=0x7f04ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c37fe640 1 --2- 172.21.15.60:0/1981994559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04c40cbcf0 0x7f04ac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c37fe640 1 -- 172.21.15.60:0/1981994559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04ac0039f0 msgr2=0x7f04ac003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c37fe640 1 --2- 172.21.15.60:0/1981994559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04ac0039f0 0x7f04ac003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c37fe640 1 -- 172.21.15.60:0/1981994559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac003530 con 0x7f04ac004810 2024-04-25T10:06:23.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c3fff640 1 --2- 172.21.15.60:0/1981994559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04ac0039f0 0x7f04ac003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:23.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c885d640 1 --2- 172.21.15.60:0/1981994559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04c40cbcf0 0x7f04ac0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:23.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c37fe640 1 --2- 172.21.15.60:0/1981994559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04ac004810 0x7f04ac004c90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f04b800c210 tx=0x7f04b801f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e152478e3da851 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 -- 172.21.15.60:0/1981994559 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8007d20 con 0x7f04ac004810 2024-04-25T10:06:23.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 -- 172.21.15.60:0/1981994559 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b8005ae0 con 0x7f04ac004810 2024-04-25T10:06:23.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:22.999+0000 7f04c2ffd640 1 -- 172.21.15.60:0/1981994559 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8007070 con 0x7f04ac004810 2024-04-25T10:06:23.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 -- 172.21.15.60:0/1981994559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04ac004810 msgr2=0x7f04ac004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 --2- 172.21.15.60:0/1981994559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04ac004810 0x7f04ac004c90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f04b800c210 tx=0x7f04b801f390 comp rx=0 tx=0).stop 2024-04-25T10:06:23.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04c885d640 1 -- 172.21.15.60:0/1981994559 reap_dead start 2024-04-25T10:06:23.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 -- 172.21.15.60:0/1981994559 shutdown_connections 2024-04-25T10:06:23.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 -- 172.21.15.60:0/1981994559 >> 172.21.15.60:0/1981994559 conn(0x7f04c40670b0 msgr2=0x7f04c406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 -- 172.21.15.60:0/1981994559 shutdown_connections 2024-04-25T10:06:23.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 -- 172.21.15.60:0/1981994559 wait complete. 2024-04-25T10:06:23.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 Processor -- start 2024-04-25T10:06:23.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 -- start start 2024-04-25T10:06:23.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04ac004810 0x7f04c4139e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04c413dc60 0x7f04c413e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04c413fcc0 0x7f04c4140140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40cc0d0 con 0x7f04ac004810 2024-04-25T10:06:23.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40ce9a0 con 0x7f04c413fcc0 2024-04-25T10:06:23.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40720b0 con 0x7f04c413dc60 2024-04-25T10:06:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04c3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04ac004810 0x7f04c4139e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04c3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04ac004810 0x7f04c4139e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37740/0 (socket says 172.21.15.60:37740) 2024-04-25T10:06:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04c3fff640 1 -- 172.21.15.60:0/182722906 learned_addr learned my addr 172.21.15.60:0/182722906 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04c3fff640 1 -- 172.21.15.60:0/182722906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04c413dc60 msgr2=0x7f04c413e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04c37fe640 1 --2- 172.21.15.60:0/182722906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04c413dc60 0x7f04c413e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04c3fff640 1 --2- 172.21.15.60:0/182722906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04c413dc60 0x7f04c413e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04c3fff640 1 -- 172.21.15.60:0/182722906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04c413fcc0 msgr2=0x7f04c4140140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04c885d640 1 --2- 172.21.15.60:0/182722906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04c413fcc0 0x7f04c4140140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04c3fff640 1 --2- 172.21.15.60:0/182722906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04c413fcc0 0x7f04c4140140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04c3fff640 1 -- 172.21.15.60:0/182722906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c4140a60 con 0x7f04ac004810 2024-04-25T10:06:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04c3fff640 1 --2- 172.21.15.60:0/182722906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04ac004810 0x7f04c4139e40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f04b4010930 tx=0x7f04b4010df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04c0ff9640 1 -- 172.21.15.60:0/182722906 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b400f040 con 0x7f04ac004810 2024-04-25T10:06:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04c0ff9640 1 -- 172.21.15.60:0/182722906 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b400ec20 con 0x7f04ac004810 2024-04-25T10:06:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 -- 172.21.15.60:0/182722906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c4140d40 con 0x7f04ac004810 2024-04-25T10:06:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04c0ff9640 1 -- 172.21.15.60:0/182722906 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4004020 con 0x7f04ac004810 2024-04-25T10:06:23.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 -- 172.21.15.60:0/182722906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c406d6d0 con 0x7f04ac004810 2024-04-25T10:06:23.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.003+0000 7f04ca2c0640 1 -- 172.21.15.60:0/182722906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c406bba0 con 0x7f04ac004810 2024-04-25T10:06:23.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.007+0000 7f04c0ff9640 1 -- 172.21.15.60:0/182722906 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f04b40041c0 con 0x7f04ac004810 2024-04-25T10:06:23.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.007+0000 7f04c0ff9640 1 --2- 172.21.15.60:0/182722906 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f04b0066f60 0x7f04b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.007+0000 7f04c0ff9640 1 -- 172.21.15.60:0/182722906 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04b4085570 con 0x7f04ac004810 2024-04-25T10:06:23.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.007+0000 7f04c37fe640 1 --2- 172.21.15.60:0/182722906 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f04b0066f60 0x7f04b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.007+0000 7f04c37fe640 1 --2- 172.21.15.60:0/182722906 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f04b0066f60 0x7f04b0069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f04b80af000 tx=0x7f04b80b2040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.011+0000 7f04c0ff9640 1 -- 172.21.15.60:0/182722906 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f04b4018030 con 0x7f04ac004810 2024-04-25T10:06:23.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.199+0000 7f04ca2c0640 1 -- 172.21.15.60:0/182722906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f04c4074770 con 0x7f04ac004810 2024-04-25T10:06:23.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.215+0000 7f04c0ff9640 1 -- 172.21.15.60:0/182722906 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7f04b40564f0 con 0x7f04ac004810 2024-04-25T10:06:23.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.219+0000 7f04ca2c0640 1 -- 172.21.15.60:0/182722906 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f04b0066f60 msgr2=0x7f04b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.219+0000 7f04ca2c0640 1 --2- 172.21.15.60:0/182722906 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f04b0066f60 0x7f04b0069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f04b80af000 tx=0x7f04b80b2040 comp rx=0 tx=0).stop 2024-04-25T10:06:23.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.219+0000 7f04ca2c0640 1 -- 172.21.15.60:0/182722906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04ac004810 msgr2=0x7f04c4139e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.219+0000 7f04ca2c0640 1 --2- 172.21.15.60:0/182722906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04ac004810 0x7f04c4139e40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f04b4010930 tx=0x7f04b4010df0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.219+0000 7f04ca2c0640 1 -- 172.21.15.60:0/182722906 shutdown_connections 2024-04-25T10:06:23.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.219+0000 7f04ca2c0640 1 --2- 172.21.15.60:0/182722906 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f04b0066f60 0x7f04b0069420 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.219+0000 7f04ca2c0640 1 --2- 172.21.15.60:0/182722906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04c413fcc0 0x7f04c4140140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.219+0000 7f04ca2c0640 1 --2- 172.21.15.60:0/182722906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04c413dc60 0x7f04c413e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.219+0000 7f04ca2c0640 1 --2- 172.21.15.60:0/182722906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04ac004810 0x7f04c4139e40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.219+0000 7f04ca2c0640 1 -- 172.21.15.60:0/182722906 >> 172.21.15.60:0/182722906 conn(0x7f04c40670b0 msgr2=0x7f04c40b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.219+0000 7f04ca2c0640 1 -- 172.21.15.60:0/182722906 shutdown_connections 2024-04-25T10:06:23.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.219+0000 7f04ca2c0640 1 -- 172.21.15.60:0/182722906 wait complete. 2024-04-25T10:06:23.242 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:06:23.242 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:06:23.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1ac296a640 1 Processor -- start 2024-04-25T10:06:23.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1ac296a640 1 -- start start 2024-04-25T10:06:23.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1ac296a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1abc15b6f0 0x7f1abc1620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1ac296a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1abc15bad0 con 0x7f1abc0ceef0 2024-04-25T10:06:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1ac296a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1abc15ebe0 con 0x7f1abc15e0d0 2024-04-25T10:06:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1ac296a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc162610 con 0x7f1abc15b6f0 2024-04-25T10:06:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1ac0f07640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1abc0ceef0 0x7f1abc0d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45848/0 (socket says 172.21.15.60:45848) 2024-04-25T10:06:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1ac0f07640 1 -- 172.21.15.60:0/1751794271 learned_addr learned my addr 172.21.15.60:0/1751794271 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abbfff640 1 --2- 172.21.15.60:0/1751794271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1abc15b6f0 0x7f1abc1620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abaffd640 1 -- 172.21.15.60:0/1751794271 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176256874 0 0) 0x7f1abc15bad0 con 0x7f1abc0ceef0 2024-04-25T10:06:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abaffd640 1 -- 172.21.15.60:0/1751794271 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1aa4003530 con 0x7f1abc0ceef0 2024-04-25T10:06:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abaffd640 1 -- 172.21.15.60:0/1751794271 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3041168955 0 0) 0x7f1aa4003530 con 0x7f1abc0ceef0 2024-04-25T10:06:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abaffd640 1 -- 172.21.15.60:0/1751794271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1abc15b6f0 msgr2=0x7f1abc1620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abaffd640 1 --2- 172.21.15.60:0/1751794271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1abc15b6f0 0x7f1abc1620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abaffd640 1 -- 172.21.15.60:0/1751794271 >> v1:172.21.15.100:6789/0 conn(0x7f1abc15e0d0 legacy=0x7f1abc0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abaffd640 1 -- 172.21.15.60:0/1751794271 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1abc15eec0 con 0x7f1abc0ceef0 2024-04-25T10:06:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abaffd640 1 -- 172.21.15.60:0/1751794271 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f1ab4001a00 con 0x7f1abc0ceef0 2024-04-25T10:06:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abaffd640 1 -- 172.21.15.60:0/1751794271 >> v1:172.21.15.60:6789/0 conn(0x7f1abc0ceef0 legacy=0x7f1abc0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abaffd640 1 --2- 172.21.15.60:0/1751794271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1aa40039f0 0x7f1aa4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abaffd640 1 --2- 172.21.15.60:0/1751794271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1aa4004810 0x7f1aa4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abaffd640 1 --2- 172.21.15.60:0/1751794271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1abc15b6f0 0x7f1aa40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abaffd640 1 -- 172.21.15.60:0/1751794271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc162610 con 0x7f1aa4004810 2024-04-25T10:06:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abaffd640 1 -- 172.21.15.60:0/1751794271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa400ba80 con 0x7f1aa40039f0 2024-04-25T10:06:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abaffd640 1 -- 172.21.15.60:0/1751794271 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa400bc80 con 0x7f1abc15b6f0 2024-04-25T10:06:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abb7fe640 1 --2- 172.21.15.60:0/1751794271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1aa4004810 0x7f1aa4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abbfff640 1 --2- 172.21.15.60:0/1751794271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1aa40039f0 0x7f1aa4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1ac0f07640 1 --2- 172.21.15.60:0/1751794271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1abc15b6f0 0x7f1aa40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abb7fe640 1 -- 172.21.15.60:0/1751794271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1abc15b6f0 msgr2=0x7f1aa40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abb7fe640 1 --2- 172.21.15.60:0/1751794271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1abc15b6f0 0x7f1aa40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abb7fe640 1 -- 172.21.15.60:0/1751794271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1aa40039f0 msgr2=0x7f1aa4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abb7fe640 1 --2- 172.21.15.60:0/1751794271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1aa40039f0 0x7f1aa4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abb7fe640 1 -- 172.21.15.60:0/1751794271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa4003530 con 0x7f1aa4004810 2024-04-25T10:06:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.359+0000 7f1abbfff640 1 --2- 172.21.15.60:0/1751794271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1aa40039f0 0x7f1aa4003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1abb7fe640 1 --2- 172.21.15.60:0/1751794271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1aa4004810 0x7f1aa4004c90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0001750 tx=0x7f1ab001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc019597c8382e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1abaffd640 1 -- 172.21.15.60:0/1751794271 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00013a0 con 0x7f1aa4004810 2024-04-25T10:06:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1abaffd640 1 -- 172.21.15.60:0/1751794271 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab00052f0 con 0x7f1aa4004810 2024-04-25T10:06:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1abaffd640 1 -- 172.21.15.60:0/1751794271 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0003840 con 0x7f1aa4004810 2024-04-25T10:06:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1ac296a640 1 -- 172.21.15.60:0/1751794271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1aa4004810 msgr2=0x7f1aa4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1ac296a640 1 --2- 172.21.15.60:0/1751794271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1aa4004810 0x7f1aa4004c90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0001750 tx=0x7f1ab001c3c0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1ac0f07640 1 -- 172.21.15.60:0/1751794271 reap_dead start 2024-04-25T10:06:23.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1ac296a640 1 -- 172.21.15.60:0/1751794271 shutdown_connections 2024-04-25T10:06:23.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1ac296a640 1 -- 172.21.15.60:0/1751794271 >> 172.21.15.60:0/1751794271 conn(0x7f1abc072340 msgr2=0x7f1abc072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1ac296a640 1 -- 172.21.15.60:0/1751794271 shutdown_connections 2024-04-25T10:06:23.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1ac296a640 1 -- 172.21.15.60:0/1751794271 wait complete. 2024-04-25T10:06:23.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1ac296a640 1 Processor -- start 2024-04-25T10:06:23.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1ac296a640 1 -- start start 2024-04-25T10:06:23.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1ac296a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1aa4004810 0x7f1abc14b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1ac296a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1abc15c290 0x7f1abc15c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1ac296a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1abc0c77b0 0x7f1abc0c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1ac296a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc0b9550 con 0x7f1abc0c77b0 2024-04-25T10:06:23.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1ac296a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc161c40 con 0x7f1abc15c290 2024-04-25T10:06:23.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.363+0000 7f1ac296a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc0b96d0 con 0x7f1aa4004810 2024-04-25T10:06:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ac0f07640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1abc0c77b0 0x7f1abc0c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ac0f07640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1abc0c77b0 0x7f1abc0c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37752/0 (socket says 172.21.15.60:37752) 2024-04-25T10:06:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ac0f07640 1 -- 172.21.15.60:0/2773587637 learned_addr learned my addr 172.21.15.60:0/2773587637 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1abb7fe640 1 --2- 172.21.15.60:0/2773587637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1abc15c290 0x7f1abc15c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1abbfff640 1 --2- 172.21.15.60:0/2773587637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1aa4004810 0x7f1abc14b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ac0f07640 1 -- 172.21.15.60:0/2773587637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1aa4004810 msgr2=0x7f1abc14b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ac0f07640 1 --2- 172.21.15.60:0/2773587637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1aa4004810 0x7f1abc14b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ac0f07640 1 -- 172.21.15.60:0/2773587637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1abc15c290 msgr2=0x7f1abc15c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ac0f07640 1 --2- 172.21.15.60:0/2773587637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1abc15c290 0x7f1abc15c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ac0f07640 1 -- 172.21.15.60:0/2773587637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1abc0c84e0 con 0x7f1abc0c77b0 2024-04-25T10:06:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ac0f07640 1 --2- 172.21.15.60:0/2773587637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1abc0c77b0 0x7f1abc0c7c30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4005c30 tx=0x7f1ab40119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ab8ff9640 1 -- 172.21.15.60:0/2773587637 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4010040 con 0x7f1abc0c77b0 2024-04-25T10:06:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ab8ff9640 1 -- 172.21.15.60:0/2773587637 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab401a070 con 0x7f1abc0c77b0 2024-04-25T10:06:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ac296a640 1 -- 172.21.15.60:0/2773587637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1abc0c87c0 con 0x7f1abc0c77b0 2024-04-25T10:06:23.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ab8ff9640 1 -- 172.21.15.60:0/2773587637 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4013a00 con 0x7f1abc0c77b0 2024-04-25T10:06:23.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ac296a640 1 -- 172.21.15.60:0/2773587637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc0c8d80 con 0x7f1abc0c77b0 2024-04-25T10:06:23.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ac296a640 1 -- 172.21.15.60:0/2773587637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1abc074770 con 0x7f1abc0c77b0 2024-04-25T10:06:23.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ab8ff9640 1 -- 172.21.15.60:0/2773587637 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ab4013ba0 con 0x7f1abc0c77b0 2024-04-25T10:06:23.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ab8ff9640 1 --2- 172.21.15.60:0/2773587637 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1aa8066f60 0x7f1aa8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1ab8ff9640 1 -- 172.21.15.60:0/2773587637 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ab4087ea0 con 0x7f1abc0c77b0 2024-04-25T10:06:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.367+0000 7f1abbfff640 1 --2- 172.21.15.60:0/2773587637 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1aa8066f60 0x7f1aa8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.371+0000 7f1abbfff640 1 --2- 172.21.15.60:0/2773587637 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1aa8066f60 0x7f1aa8069420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009ac0 tx=0x7f1aac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.371+0000 7f1ab8ff9640 1 -- 172.21.15.60:0/2773587637 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ab4058eb0 con 0x7f1abc0c77b0 2024-04-25T10:06:23.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:23.571+0000 7f1ac296a640 1 -- 172.21.15.60:0/2773587637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1abc0d16e0 con 0x7f1abc0c77b0 2024-04-25T10:06:24.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.235+0000 7f1ab8ff9640 1 -- 172.21.15.60:0/2773587637 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1ab4013e90 con 0x7f1abc0c77b0 2024-04-25T10:06:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.239+0000 7f1ac296a640 1 -- 172.21.15.60:0/2773587637 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1aa8066f60 msgr2=0x7f1aa8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.239+0000 7f1ac296a640 1 --2- 172.21.15.60:0/2773587637 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1aa8066f60 0x7f1aa8069420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009ac0 tx=0x7f1aac008040 comp rx=0 tx=0).stop 2024-04-25T10:06:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.239+0000 7f1ac296a640 1 -- 172.21.15.60:0/2773587637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1abc0c77b0 msgr2=0x7f1abc0c7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.239+0000 7f1ac296a640 1 --2- 172.21.15.60:0/2773587637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1abc0c77b0 0x7f1abc0c7c30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4005c30 tx=0x7f1ab40119f0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.239+0000 7f1ac296a640 1 -- 172.21.15.60:0/2773587637 shutdown_connections 2024-04-25T10:06:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.239+0000 7f1ac296a640 1 --2- 172.21.15.60:0/2773587637 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1aa8066f60 0x7f1aa8069420 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.239+0000 7f1ac296a640 1 --2- 172.21.15.60:0/2773587637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1abc0c77b0 0x7f1abc0c7c30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.239+0000 7f1ac296a640 1 --2- 172.21.15.60:0/2773587637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1abc15c290 0x7f1abc15c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.239+0000 7f1ac296a640 1 --2- 172.21.15.60:0/2773587637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1aa4004810 0x7f1abc14b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.239+0000 7f1ac296a640 1 -- 172.21.15.60:0/2773587637 >> 172.21.15.60:0/2773587637 conn(0x7f1abc072340 msgr2=0x7f1abc15d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.239+0000 7f1ac296a640 1 -- 172.21.15.60:0/2773587637 shutdown_connections 2024-04-25T10:06:24.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.239+0000 7f1ac296a640 1 -- 172.21.15.60:0/2773587637 wait complete. 2024-04-25T10:06:24.268 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:06:24.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:24.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa6242ee640 1 Processor -- start 2024-04-25T10:06:24.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa6242ee640 1 -- start start 2024-04-25T10:06:24.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa6242ee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa61c15a7a0 0x7fa61c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa6242ee640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa61c06ec30 0x7fa61c071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa6242ee640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa61c0b37e0 0x7fa61c0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa6242ee640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa61c0cdaa0 con 0x7fa61c0b37e0 2024-04-25T10:06:24.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa6242ee640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa61c06d2f0 con 0x7fa61c06ec30 2024-04-25T10:06:24.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa6242ee640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c06d8b0 con 0x7fa61c15a7a0 2024-04-25T10:06:24.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa62288b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa61c0b37e0 0x7fa61c0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa62288b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa61c0b37e0 0x7fa61c0b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37754/0 (socket says 172.21.15.60:37754) 2024-04-25T10:06:24.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa62288b640 1 -- 172.21.15.60:0/358731278 learned_addr learned my addr 172.21.15.60:0/358731278 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:24.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa62208a640 1 --2- 172.21.15.60:0/358731278 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa61c15a7a0 0x7fa61c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa621889640 1 --2- 172.21.15.60:0/358731278 >> v2:172.21.15.100:3300/0 conn(0x7fa61c06ec30 0x7fa61c071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa62288b640 1 -- 172.21.15.60:0/358731278 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa61c15a7a0 msgr2=0x7fa61c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa62288b640 1 --2- 172.21.15.60:0/358731278 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa61c15a7a0 0x7fa61c0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa62288b640 1 -- 172.21.15.60:0/358731278 >> v2:172.21.15.100:3300/0 conn(0x7fa61c06ec30 msgr2=0x7fa61c071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa62288b640 1 --2- 172.21.15.60:0/358731278 >> v2:172.21.15.100:3300/0 conn(0x7fa61c06ec30 0x7fa61c071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.391+0000 7fa62288b640 1 -- 172.21.15.60:0/358731278 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c0721e0 con 0x7fa61c0b37e0 2024-04-25T10:06:24.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa62288b640 1 --2- 172.21.15.60:0/358731278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa61c0b37e0 0x7fa61c0b3c90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa61c072b20 tx=0x7fa61401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bea2b66e612fc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa621088640 1 -- 172.21.15.60:0/358731278 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6140a5780 con 0x7fa61c0b37e0 2024-04-25T10:06:24.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa621088640 1 -- 172.21.15.60:0/358731278 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6140a5920 con 0x7fa61c0b37e0 2024-04-25T10:06:24.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa621088640 1 -- 172.21.15.60:0/358731278 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6140a5cf0 con 0x7fa61c0b37e0 2024-04-25T10:06:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 -- 172.21.15.60:0/358731278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa61c0b37e0 msgr2=0x7fa61c0b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 --2- 172.21.15.60:0/358731278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa61c0b37e0 0x7fa61c0b3c90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa61c072b20 tx=0x7fa61401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 -- 172.21.15.60:0/358731278 shutdown_connections 2024-04-25T10:06:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 --2- 172.21.15.60:0/358731278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa61c0b37e0 0x7fa61c0b3c90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 --2- 172.21.15.60:0/358731278 >> v2:172.21.15.100:3300/0 conn(0x7fa61c06ec30 0x7fa61c071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 --2- 172.21.15.60:0/358731278 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa61c15a7a0 0x7fa61c0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 -- 172.21.15.60:0/358731278 >> 172.21.15.60:0/358731278 conn(0x7fa61c0b12d0 msgr2=0x7fa61c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 -- 172.21.15.60:0/358731278 shutdown_connections 2024-04-25T10:06:24.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 -- 172.21.15.60:0/358731278 wait complete. 2024-04-25T10:06:24.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 Processor -- start 2024-04-25T10:06:24.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 -- start start 2024-04-25T10:06:24.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa61c06ec30 0x7fa61c15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa61c0b37e0 0x7fa61c145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa61c15a7a0 0x7fa61c161dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c15ad00 con 0x7fa61c06ec30 2024-04-25T10:06:24.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c15ab80 con 0x7fa61c0b37e0 2024-04-25T10:06:24.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa6242ee640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c06e530 con 0x7fa61c15a7a0 2024-04-25T10:06:24.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa62208a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa61c06ec30 0x7fa61c15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa62208a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa61c06ec30 0x7fa61c15f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37770/0 (socket says 172.21.15.60:37770) 2024-04-25T10:06:24.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa62208a640 1 -- 172.21.15.60:0/2349729663 learned_addr learned my addr 172.21.15.60:0/2349729663 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:24.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa62288b640 1 --2- 172.21.15.60:0/2349729663 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa61c15a7a0 0x7fa61c161dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa621889640 1 --2- 172.21.15.60:0/2349729663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa61c0b37e0 0x7fa61c145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa62208a640 1 -- 172.21.15.60:0/2349729663 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa61c15a7a0 msgr2=0x7fa61c161dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa62208a640 1 --2- 172.21.15.60:0/2349729663 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa61c15a7a0 0x7fa61c161dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa62208a640 1 -- 172.21.15.60:0/2349729663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa61c0b37e0 msgr2=0x7fa61c145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa62208a640 1 --2- 172.21.15.60:0/2349729663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa61c0b37e0 0x7fa61c145ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa62208a640 1 -- 172.21.15.60:0/2349729663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c157f90 con 0x7fa61c06ec30 2024-04-25T10:06:24.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa62208a640 1 --2- 172.21.15.60:0/2349729663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa61c06ec30 0x7fa61c15f4b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa61800d5f0 tx=0x7fa61800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.395+0000 7fa612ffd640 1 -- 172.21.15.60:0/2349729663 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6180125f0 con 0x7fa61c06ec30 2024-04-25T10:06:24.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.399+0000 7fa612ffd640 1 -- 172.21.15.60:0/2349729663 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa618019070 con 0x7fa61c06ec30 2024-04-25T10:06:24.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.399+0000 7fa6242ee640 1 -- 172.21.15.60:0/2349729663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61c158270 con 0x7fa61c06ec30 2024-04-25T10:06:24.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.399+0000 7fa612ffd640 1 -- 172.21.15.60:0/2349729663 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6180129b0 con 0x7fa61c06ec30 2024-04-25T10:06:24.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.399+0000 7fa6242ee640 1 -- 172.21.15.60:0/2349729663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61c158b10 con 0x7fa61c06ec30 2024-04-25T10:06:24.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.399+0000 7fa6242ee640 1 -- 172.21.15.60:0/2349729663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61c074770 con 0x7fa61c06ec30 2024-04-25T10:06:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.399+0000 7fa612ffd640 1 -- 172.21.15.60:0/2349729663 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa618012b50 con 0x7fa61c06ec30 2024-04-25T10:06:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.399+0000 7fa612ffd640 1 --2- 172.21.15.60:0/2349729663 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa600066f60 0x7fa600069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.399+0000 7fa612ffd640 1 -- 172.21.15.60:0/2349729663 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa618085300 con 0x7fa61c06ec30 2024-04-25T10:06:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.399+0000 7fa621889640 1 --2- 172.21.15.60:0/2349729663 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa600066f60 0x7fa600069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.399+0000 7fa621889640 1 --2- 172.21.15.60:0/2349729663 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa600066f60 0x7fa600069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa61c157f20 tx=0x7fa60c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.403+0000 7fa612ffd640 1 -- 172.21.15.60:0/2349729663 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6180563a0 con 0x7fa61c06ec30 2024-04-25T10:06:24.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.599+0000 7fa6242ee640 1 -- 172.21.15.60:0/2349729663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa61c0b3e10 con 0x7fa61c06ec30 2024-04-25T10:06:24.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.599+0000 7fa612ffd640 1 -- 172.21.15.60:0/2349729663 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fa618055ca0 con 0x7fa61c06ec30 2024-04-25T10:06:24.606 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:24.606 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2422261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2422261630},{"type":"v1","addr":"172.21.15.100:6835","nonce":2422261630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2923340508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2923340508},{"type":"v1","addr":"172.21.15.100:6837","nonce":2923340508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1929386449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1929386449},{"type":"v1","addr":"172.21.15.60:6833","nonce":1929386449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:21.529455+0000","modified":"2024-04-25T10:06:23.575106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5283":{"gid":5283,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3681296171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3681296171},{"type":"v1","addr":"172.21.15.60:6835","nonce":3681296171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:24.607 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 15 2024-04-25T10:06:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.603+0000 7fa6242ee640 1 -- 172.21.15.60:0/2349729663 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa600066f60 msgr2=0x7fa600069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.603+0000 7fa6242ee640 1 --2- 172.21.15.60:0/2349729663 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa600066f60 0x7fa600069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa61c157f20 tx=0x7fa60c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.603+0000 7fa6242ee640 1 -- 172.21.15.60:0/2349729663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa61c06ec30 msgr2=0x7fa61c15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.603+0000 7fa6242ee640 1 --2- 172.21.15.60:0/2349729663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa61c06ec30 0x7fa61c15f4b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa61800d5f0 tx=0x7fa61800dab0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.603+0000 7fa6242ee640 1 -- 172.21.15.60:0/2349729663 shutdown_connections 2024-04-25T10:06:24.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.603+0000 7fa6242ee640 1 --2- 172.21.15.60:0/2349729663 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa600066f60 0x7fa600069420 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.603+0000 7fa6242ee640 1 --2- 172.21.15.60:0/2349729663 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa61c15a7a0 0x7fa61c161dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.603+0000 7fa6242ee640 1 --2- 172.21.15.60:0/2349729663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa61c0b37e0 0x7fa61c145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.603+0000 7fa6242ee640 1 --2- 172.21.15.60:0/2349729663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa61c06ec30 0x7fa61c15f4b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.603+0000 7fa6242ee640 1 -- 172.21.15.60:0/2349729663 >> 172.21.15.60:0/2349729663 conn(0x7fa61c0b12d0 msgr2=0x7fa61c0ce880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.603+0000 7fa6242ee640 1 -- 172.21.15.60:0/2349729663 shutdown_connections 2024-04-25T10:06:24.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.603+0000 7fa6242ee640 1 -- 172.21.15.60:0/2349729663 wait complete. 2024-04-25T10:06:24.626 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:24.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.735+0000 7ff5e9c4c640 1 Processor -- start 2024-04-25T10:06:24.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.735+0000 7ff5e9c4c640 1 -- start start 2024-04-25T10:06:24.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.739+0000 7ff5e9c4c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff5e40d0a50 0x7ff5e40b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.739+0000 7ff5e9c4c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e406d200 con 0x7ff5e415deb0 2024-04-25T10:06:24.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.739+0000 7ff5e9c4c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e40cefe0 con 0x7ff5e40ce2b0 2024-04-25T10:06:24.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.739+0000 7ff5e9c4c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e40cf2c0 con 0x7ff5e40d0a50 2024-04-25T10:06:24.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.739+0000 7ff5e37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff5e415deb0 0x7ff5e40d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45856/0 (socket says 172.21.15.60:45856) 2024-04-25T10:06:24.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.739+0000 7ff5e37fe640 1 -- 172.21.15.60:0/3581235483 learned_addr learned my addr 172.21.15.60:0/3581235483 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:24.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.739+0000 7ff5e2ffd640 1 --2- 172.21.15.60:0/3581235483 >> v2:172.21.15.60:3300/0 conn(0x7ff5e40d0a50 0x7ff5e40b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.739+0000 7ff5e27fc640 1 -- 172.21.15.60:0/3581235483 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3998028167 0 0) 0x7ff5e406d200 con 0x7ff5e415deb0 2024-04-25T10:06:24.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.739+0000 7ff5e27fc640 1 -- 172.21.15.60:0/3581235483 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5cc003530 con 0x7ff5e415deb0 2024-04-25T10:06:24.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.739+0000 7ff5e2ffd640 1 -- 172.21.15.60:0/3581235483 >> v1:172.21.15.100:6789/0 conn(0x7ff5e40ce2b0 legacy=0x7ff5e40ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.739+0000 7ff5e2ffd640 1 -- 172.21.15.60:0/3581235483 >> v1:172.21.15.60:6789/0 conn(0x7ff5e415deb0 legacy=0x7ff5e40d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.739+0000 7ff5e2ffd640 1 -- 172.21.15.60:0/3581235483 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e40d1d70 con 0x7ff5e40d0a50 2024-04-25T10:06:24.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.743+0000 7ff5e2ffd640 1 --2- 172.21.15.60:0/3581235483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e40d0a50 0x7ff5e40b6c10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80097c0 tx=0x7ff5d801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450369d3b2251f06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.743+0000 7ff5e27fc640 1 -- 172.21.15.60:0/3581235483 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d80b0070 con 0x7ff5e40d0a50 2024-04-25T10:06:24.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.743+0000 7ff5e27fc640 1 -- 172.21.15.60:0/3581235483 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d80ab600 con 0x7ff5e40d0a50 2024-04-25T10:06:24.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.743+0000 7ff5e27fc640 1 -- 172.21.15.60:0/3581235483 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d80ab9b0 con 0x7ff5e40d0a50 2024-04-25T10:06:24.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e9c4c640 1 -- 172.21.15.60:0/3581235483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e40d0a50 msgr2=0x7ff5e40b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e9c4c640 1 --2- 172.21.15.60:0/3581235483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e40d0a50 0x7ff5e40b6c10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80097c0 tx=0x7ff5d801c460 comp rx=0 tx=0).stop 2024-04-25T10:06:24.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e9c4c640 1 -- 172.21.15.60:0/3581235483 shutdown_connections 2024-04-25T10:06:24.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e9c4c640 1 --2- 172.21.15.60:0/3581235483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e40d0a50 0x7ff5e40b6c10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e9c4c640 1 -- 172.21.15.60:0/3581235483 >> 172.21.15.60:0/3581235483 conn(0x7ff5e406c090 msgr2=0x7ff5e406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e9c4c640 1 -- 172.21.15.60:0/3581235483 shutdown_connections 2024-04-25T10:06:24.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e9c4c640 1 -- 172.21.15.60:0/3581235483 wait complete. 2024-04-25T10:06:24.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e9c4c640 1 Processor -- start 2024-04-25T10:06:24.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e9c4c640 1 -- start start 2024-04-25T10:06:24.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e9c4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e40ce2b0 0x7ff5e40c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e9c4c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5e40d0a50 0x7ff5e4156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e9c4c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5e415deb0 0x7ff5e4157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e9c4c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e416b7d0 con 0x7ff5e40ce2b0 2024-04-25T10:06:24.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e9c4c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e416b650 con 0x7ff5e40d0a50 2024-04-25T10:06:24.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e9c4c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e416b950 con 0x7ff5e415deb0 2024-04-25T10:06:24.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e40ce2b0 0x7ff5e40c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e40ce2b0 0x7ff5e40c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37782/0 (socket says 172.21.15.60:37782) 2024-04-25T10:06:24.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e37fe640 1 -- 172.21.15.60:0/4173973482 learned_addr learned my addr 172.21.15.60:0/4173973482 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:24.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e2ffd640 1 --2- 172.21.15.60:0/4173973482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5e40d0a50 0x7ff5e4156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e3fff640 1 --2- 172.21.15.60:0/4173973482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5e415deb0 0x7ff5e4157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e37fe640 1 -- 172.21.15.60:0/4173973482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5e415deb0 msgr2=0x7ff5e4157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e37fe640 1 --2- 172.21.15.60:0/4173973482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5e415deb0 0x7ff5e4157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e37fe640 1 -- 172.21.15.60:0/4173973482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5e40d0a50 msgr2=0x7ff5e4156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e37fe640 1 --2- 172.21.15.60:0/4173973482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5e40d0a50 0x7ff5e4156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e37fe640 1 -- 172.21.15.60:0/4173973482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e40ca7b0 con 0x7ff5e40ce2b0 2024-04-25T10:06:24.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e37fe640 1 --2- 172.21.15.60:0/4173973482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e40ce2b0 0x7ff5e40c9630 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400c3c0 tx=0x7ff5d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5cbfff640 1 -- 172.21.15.60:0/4173973482 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4017070 con 0x7ff5e40ce2b0 2024-04-25T10:06:24.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5cbfff640 1 -- 172.21.15.60:0/4173973482 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d4005cf0 con 0x7ff5e40ce2b0 2024-04-25T10:06:24.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5cbfff640 1 -- 172.21.15.60:0/4173973482 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d40072f0 con 0x7ff5e40ce2b0 2024-04-25T10:06:24.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.747+0000 7ff5e9c4c640 1 -- 172.21.15.60:0/4173973482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e40b5f80 con 0x7ff5e40ce2b0 2024-04-25T10:06:24.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.751+0000 7ff5e9c4c640 1 -- 172.21.15.60:0/4173973482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e414b130 con 0x7ff5e40ce2b0 2024-04-25T10:06:24.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.751+0000 7ff5cbfff640 1 -- 172.21.15.60:0/4173973482 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5d401a070 con 0x7ff5e40ce2b0 2024-04-25T10:06:24.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.751+0000 7ff5e9c4c640 1 -- 172.21.15.60:0/4173973482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e4074770 con 0x7ff5e40ce2b0 2024-04-25T10:06:24.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.751+0000 7ff5cbfff640 1 --2- 172.21.15.60:0/4173973482 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff5d0066f60 0x7ff5d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.751+0000 7ff5cbfff640 1 -- 172.21.15.60:0/4173973482 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5d40843a0 con 0x7ff5e40ce2b0 2024-04-25T10:06:24.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.751+0000 7ff5e2ffd640 1 --2- 172.21.15.60:0/4173973482 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff5d0066f60 0x7ff5d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.751+0000 7ff5e2ffd640 1 --2- 172.21.15.60:0/4173973482 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff5d0066f60 0x7ff5d0069420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80ad000 tx=0x7ff5d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.755+0000 7ff5cbfff640 1 -- 172.21.15.60:0/4173973482 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5d4055440 con 0x7ff5e40ce2b0 2024-04-25T10:06:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.915+0000 7ff5e9c4c640 1 -- 172.21.15.60:0/4173973482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5e41511f0 con 0x7ff5e40ce2b0 2024-04-25T10:06:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.919+0000 7ff5cbfff640 1 -- 172.21.15.60:0/4173973482 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16681 (secure 0 0 0) 0x7ff5d4054d40 con 0x7ff5e40ce2b0 2024-04-25T10:06:24.926 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:24.926 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":35,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:23.196554+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:20.088165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:20.507719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:24.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.919+0000 7ff5e9c4c640 1 -- 172.21.15.60:0/4173973482 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff5d0066f60 msgr2=0x7ff5d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.919+0000 7ff5e9c4c640 1 --2- 172.21.15.60:0/4173973482 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff5d0066f60 0x7ff5d0069420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80ad000 tx=0x7ff5d8008040 comp rx=0 tx=0).stop 2024-04-25T10:06:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.923+0000 7ff5e9c4c640 1 -- 172.21.15.60:0/4173973482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e40ce2b0 msgr2=0x7ff5e40c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.923+0000 7ff5e9c4c640 1 --2- 172.21.15.60:0/4173973482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e40ce2b0 0x7ff5e40c9630 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400c3c0 tx=0x7ff5d400c880 comp rx=0 tx=0).stop 2024-04-25T10:06:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.923+0000 7ff5e9c4c640 1 -- 172.21.15.60:0/4173973482 shutdown_connections 2024-04-25T10:06:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.923+0000 7ff5e9c4c640 1 --2- 172.21.15.60:0/4173973482 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff5d0066f60 0x7ff5d0069420 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.923+0000 7ff5e9c4c640 1 --2- 172.21.15.60:0/4173973482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5e415deb0 0x7ff5e4157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.923+0000 7ff5e9c4c640 1 --2- 172.21.15.60:0/4173973482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5e40d0a50 0x7ff5e4156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.923+0000 7ff5e9c4c640 1 --2- 172.21.15.60:0/4173973482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5e40ce2b0 0x7ff5e40c9630 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.923+0000 7ff5e9c4c640 1 -- 172.21.15.60:0/4173973482 >> 172.21.15.60:0/4173973482 conn(0x7ff5e406c090 msgr2=0x7ff5e415b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.923+0000 7ff5e9c4c640 1 -- 172.21.15.60:0/4173973482 shutdown_connections 2024-04-25T10:06:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:24.923+0000 7ff5e9c4c640 1 -- 172.21.15.60:0/4173973482 wait complete. 2024-04-25T10:06:24.945 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:06:24.945 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:25.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.063+0000 7ff35aeca640 1 Processor -- start 2024-04-25T10:06:25.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.063+0000 7ff35aeca640 1 -- start start 2024-04-25T10:06:25.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.063+0000 7ff35aeca640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff354071560 0x7ff354165840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.063+0000 7ff35aeca640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3540d16c0 con 0x7ff35415ddf0 2024-04-25T10:06:25.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.063+0000 7ff35aeca640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3540b5c50 con 0x7ff35406d3f0 2024-04-25T10:06:25.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.063+0000 7ff35aeca640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3540b53b0 con 0x7ff354071560 2024-04-25T10:06:25.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.063+0000 7ff358c66640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff35415ddf0 0x7ff354070e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45872/0 (socket says 172.21.15.60:45872) 2024-04-25T10:06:25.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.063+0000 7ff358c66640 1 -- 172.21.15.60:0/3269102705 learned_addr learned my addr 172.21.15.60:0/3269102705 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:25.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.063+0000 7ff353fff640 1 --2- 172.21.15.60:0/3269102705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff354071560 0x7ff354165840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.063+0000 7ff3537fe640 1 -- 172.21.15.60:0/3269102705 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387086303 0 0) 0x7ff3540d16c0 con 0x7ff35415ddf0 2024-04-25T10:06:25.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.063+0000 7ff3537fe640 1 -- 172.21.15.60:0/3269102705 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff33c003530 con 0x7ff35415ddf0 2024-04-25T10:06:25.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.063+0000 7ff353fff640 1 -- 172.21.15.60:0/3269102705 >> v1:172.21.15.100:6789/0 conn(0x7ff35406d3f0 legacy=0x7ff354167d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.063+0000 7ff353fff640 1 -- 172.21.15.60:0/3269102705 >> v1:172.21.15.60:6789/0 conn(0x7ff35415ddf0 legacy=0x7ff354070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.063+0000 7ff353fff640 1 -- 172.21.15.60:0/3269102705 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3540b5690 con 0x7ff354071560 2024-04-25T10:06:25.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.067+0000 7ff353fff640 1 --2- 172.21.15.60:0/3269102705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff354071560 0x7ff354165840 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff3480097f0 tx=0x7ff34801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40806dd55d550754 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.067+0000 7ff3537fe640 1 -- 172.21.15.60:0/3269102705 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3480aa430 con 0x7ff354071560 2024-04-25T10:06:25.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.067+0000 7ff3537fe640 1 -- 172.21.15.60:0/3269102705 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3480af070 con 0x7ff354071560 2024-04-25T10:06:25.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.067+0000 7ff3537fe640 1 -- 172.21.15.60:0/3269102705 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3480aa7f0 con 0x7ff354071560 2024-04-25T10:06:25.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.067+0000 7ff35aeca640 1 -- 172.21.15.60:0/3269102705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff354071560 msgr2=0x7ff354165840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.067+0000 7ff35aeca640 1 --2- 172.21.15.60:0/3269102705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff354071560 0x7ff354165840 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff3480097f0 tx=0x7ff34801c4b0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.067+0000 7ff35aeca640 1 -- 172.21.15.60:0/3269102705 shutdown_connections 2024-04-25T10:06:25.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.067+0000 7ff35aeca640 1 --2- 172.21.15.60:0/3269102705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff354071560 0x7ff354165840 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.067+0000 7ff35aeca640 1 -- 172.21.15.60:0/3269102705 >> 172.21.15.60:0/3269102705 conn(0x7ff3540b3520 msgr2=0x7ff3540b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:25.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.067+0000 7ff35aeca640 1 -- 172.21.15.60:0/3269102705 shutdown_connections 2024-04-25T10:06:25.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.067+0000 7ff35aeca640 1 -- 172.21.15.60:0/3269102705 wait complete. 2024-04-25T10:06:25.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.067+0000 7ff35aeca640 1 Processor -- start 2024-04-25T10:06:25.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.067+0000 7ff35aeca640 1 -- start start 2024-04-25T10:06:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff35aeca640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff35406d3f0 0x7ff35415f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff35aeca640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354071560 0x7ff3541501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff35aeca640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35415ddf0 0x7ff3540cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff35aeca640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35416c060 con 0x7ff354071560 2024-04-25T10:06:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff35aeca640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35416bee0 con 0x7ff35415ddf0 2024-04-25T10:06:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff35aeca640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35416c1e0 con 0x7ff35406d3f0 2024-04-25T10:06:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff353fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354071560 0x7ff3541501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff353fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354071560 0x7ff3541501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37786/0 (socket says 172.21.15.60:37786) 2024-04-25T10:06:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff353fff640 1 -- 172.21.15.60:0/1311026970 learned_addr learned my addr 172.21.15.60:0/1311026970 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff358c66640 1 --2- 172.21.15.60:0/1311026970 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff35406d3f0 0x7ff35415f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff359467640 1 --2- 172.21.15.60:0/1311026970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35415ddf0 0x7ff3540cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff353fff640 1 -- 172.21.15.60:0/1311026970 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff35406d3f0 msgr2=0x7ff35415f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff353fff640 1 --2- 172.21.15.60:0/1311026970 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff35406d3f0 0x7ff35415f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff353fff640 1 -- 172.21.15.60:0/1311026970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35415ddf0 msgr2=0x7ff3540cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff353fff640 1 --2- 172.21.15.60:0/1311026970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35415ddf0 0x7ff3540cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff353fff640 1 -- 172.21.15.60:0/1311026970 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35415e480 con 0x7ff354071560 2024-04-25T10:06:25.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff359467640 1 --2- 172.21.15.60:0/1311026970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35415ddf0 0x7ff3540cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:25.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff353fff640 1 --2- 172.21.15.60:0/1311026970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354071560 0x7ff3541501f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff3480190c0 tx=0x7ff34801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff3517fa640 1 -- 172.21.15.60:0/1311026970 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3480be610 con 0x7ff354071560 2024-04-25T10:06:25.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff3517fa640 1 -- 172.21.15.60:0/1311026970 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3480be7d0 con 0x7ff354071560 2024-04-25T10:06:25.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff3517fa640 1 -- 172.21.15.60:0/1311026970 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3480bebc0 con 0x7ff354071560 2024-04-25T10:06:25.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff35aeca640 1 -- 172.21.15.60:0/1311026970 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff35415e760 con 0x7ff354071560 2024-04-25T10:06:25.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff35aeca640 1 -- 172.21.15.60:0/1311026970 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35415ed20 con 0x7ff354071560 2024-04-25T10:06:25.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff35aeca640 1 -- 172.21.15.60:0/1311026970 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff354074770 con 0x7ff354071560 2024-04-25T10:06:25.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff3517fa640 1 -- 172.21.15.60:0/1311026970 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3480bc020 con 0x7ff354071560 2024-04-25T10:06:25.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff3517fa640 1 --2- 172.21.15.60:0/1311026970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff340066f60 0x7ff340069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff3517fa640 1 -- 172.21.15.60:0/1311026970 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff34811f2d0 con 0x7ff354071560 2024-04-25T10:06:25.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.071+0000 7ff358c66640 1 --2- 172.21.15.60:0/1311026970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff340066f60 0x7ff340069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.075+0000 7ff358c66640 1 --2- 172.21.15.60:0/1311026970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff340066f60 0x7ff340069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff3440011f0 tx=0x7ff344009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.075+0000 7ff3517fa640 1 -- 172.21.15.60:0/1311026970 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3480160d0 con 0x7ff354071560 2024-04-25T10:06:25.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.227+0000 7ff35aeca640 1 -- 172.21.15.60:0/1311026970 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff3540cd9b0 con 0x7ff340066f60 2024-04-25T10:06:25.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.247+0000 7ff3517fa640 1 -- 172.21.15.60:0/1311026970 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251906 (secure 0 0 0) 0x7ff3540cd9b0 con 0x7ff340066f60 2024-04-25T10:06:25.257 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:25.259 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:06:25.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.251+0000 7ff35aeca640 1 -- 172.21.15.60:0/1311026970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff340066f60 msgr2=0x7ff340069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.251+0000 7ff35aeca640 1 --2- 172.21.15.60:0/1311026970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff340066f60 0x7ff340069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff3440011f0 tx=0x7ff344009040 comp rx=0 tx=0).stop 2024-04-25T10:06:25.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.251+0000 7ff35aeca640 1 -- 172.21.15.60:0/1311026970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354071560 msgr2=0x7ff3541501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.251+0000 7ff35aeca640 1 --2- 172.21.15.60:0/1311026970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354071560 0x7ff3541501f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff3480190c0 tx=0x7ff34801acf0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.251+0000 7ff35aeca640 1 -- 172.21.15.60:0/1311026970 shutdown_connections 2024-04-25T10:06:25.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.251+0000 7ff35aeca640 1 --2- 172.21.15.60:0/1311026970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff340066f60 0x7ff340069420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.251+0000 7ff35aeca640 1 --2- 172.21.15.60:0/1311026970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35415ddf0 0x7ff3540cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.251+0000 7ff35aeca640 1 --2- 172.21.15.60:0/1311026970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff354071560 0x7ff3541501f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.251+0000 7ff35aeca640 1 --2- 172.21.15.60:0/1311026970 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff35406d3f0 0x7ff35415f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.251+0000 7ff35aeca640 1 -- 172.21.15.60:0/1311026970 >> 172.21.15.60:0/1311026970 conn(0x7ff3540b3520 msgr2=0x7ff35406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:25.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.251+0000 7ff35aeca640 1 -- 172.21.15.60:0/1311026970 shutdown_connections 2024-04-25T10:06:25.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.251+0000 7ff35aeca640 1 -- 172.21.15.60:0/1311026970 wait complete. 2024-04-25T10:06:25.272 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-04-25T10:06:24.408593+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245220,"kb_used_data":4084,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754729500,"statfs":{"total":773094113280,"available":772843008000,"internally_reserved":0,"allocated":4182016,"data_stored":2337070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.261607"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.261413+0000","last_change":"2024-04-25T10:06:22.255457+0000","last_active":"2024-04-25T10:06:22.261413+0000","last_peered":"2024-04-25T10:06:22.261413+0000","last_clean":"2024-04-25T10:06:22.261413+0000","last_became_active":"2024-04-25T10:06:22.253854+0000","last_became_peered":"2024-04-25T10:06:22.253854+0000","last_unstale":"2024-04-25T10:06:22.261413+0000","last_undegraded":"2024-04-25T10:06:22.261413+0000","last_fullsized":"2024-04-25T10:06:22.261413+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:14.121327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.219127+0000","last_change":"2024-04-25T10:06:22.217046+0000","last_active":"2024-04-25T10:06:22.219127+0000","last_peered":"2024-04-25T10:06:22.219127+0000","last_clean":"2024-04-25T10:06:22.219127+0000","last_became_active":"2024-04-25T10:06:22.216754+0000","last_became_peered":"2024-04-25T10:06:22.216754+0000","last_unstale":"2024-04-25T10:06:22.219127+0000","last_undegraded":"2024-04-25T10:06:22.219127+0000","last_fullsized":"2024-04-25T10:06:22.219127+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:14.614718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.237958+0000","last_change":"2024-04-25T10:06:22.227055+0000","last_active":"2024-04-25T10:06:22.237958+0000","last_peered":"2024-04-25T10:06:22.237958+0000","last_clean":"2024-04-25T10:06:22.237958+0000","last_became_active":"2024-04-25T10:06:22.226372+0000","last_became_peered":"2024-04-25T10:06:22.226372+0000","last_unstale":"2024-04-25T10:06:22.237958+0000","last_undegraded":"2024-04-25T10:06:22.237958+0000","last_fullsized":"2024-04-25T10:06:22.237958+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:08.711769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.230197+0000","last_change":"2024-04-25T10:06:22.229697+0000","last_active":"2024-04-25T10:06:22.230197+0000","last_peered":"2024-04-25T10:06:22.230197+0000","last_clean":"2024-04-25T10:06:22.230197+0000","last_became_active":"2024-04-25T10:06:22.229417+0000","last_became_peered":"2024-04-25T10:06:22.229417+0000","last_unstale":"2024-04-25T10:06:22.230197+0000","last_undegraded":"2024-04-25T10:06:22.230197+0000","last_fullsized":"2024-04-25T10:06:22.230197+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:51.847161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.217697+0000","last_change":"2024-04-25T10:06:22.216363+0000","last_active":"2024-04-25T10:06:22.217697+0000","last_peered":"2024-04-25T10:06:22.217697+0000","last_clean":"2024-04-25T10:06:22.217697+0000","last_became_active":"2024-04-25T10:06:22.216118+0000","last_became_peered":"2024-04-25T10:06:22.216118+0000","last_unstale":"2024-04-25T10:06:22.217697+0000","last_undegraded":"2024-04-25T10:06:22.217697+0000","last_fullsized":"2024-04-25T10:06:22.217697+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:03.419847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.241785+0000","last_change":"2024-04-25T10:06:22.241246+0000","last_active":"2024-04-25T10:06:22.241785+0000","last_peered":"2024-04-25T10:06:22.241785+0000","last_clean":"2024-04-25T10:06:22.241785+0000","last_became_active":"2024-04-25T10:06:22.241010+0000","last_became_peered":"2024-04-25T10:06:22.241010+0000","last_unstale":"2024-04-25T10:06:22.241785+0000","last_undegraded":"2024-04-25T10:06:22.241785+0000","last_fullsized":"2024-04-25T10:06:22.241785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:40.277052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.206746+0000","last_change":"2024-04-25T10:06:22.263026+0000","last_active":"2024-04-25T10:06:23.206746+0000","last_peered":"2024-04-25T10:06:23.206746+0000","last_clean":"2024-04-25T10:06:23.206746+0000","last_became_active":"2024-04-25T10:06:22.262155+0000","last_became_peered":"2024-04-25T10:06:22.262155+0000","last_unstale":"2024-04-25T10:06:23.206746+0000","last_undegraded":"2024-04-25T10:06:23.206746+0000","last_fullsized":"2024-04-25T10:06:23.206746+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:12.354255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.205386+0000","last_change":"2024-04-25T10:06:22.254129+0000","last_active":"2024-04-25T10:06:23.205386+0000","last_peered":"2024-04-25T10:06:23.205386+0000","last_clean":"2024-04-25T10:06:23.205386+0000","last_became_active":"2024-04-25T10:06:22.253670+0000","last_became_peered":"2024-04-25T10:06:22.253670+0000","last_unstale":"2024-04-25T10:06:23.205386+0000","last_undegraded":"2024-04-25T10:06:23.205386+0000","last_fullsized":"2024-04-25T10:06:23.205386+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:19.349171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208856+0000","last_change":"2024-04-25T10:06:22.237973+0000","last_active":"2024-04-25T10:06:23.208856+0000","last_peered":"2024-04-25T10:06:23.208856+0000","last_clean":"2024-04-25T10:06:23.208856+0000","last_became_active":"2024-04-25T10:06:22.230343+0000","last_became_peered":"2024-04-25T10:06:22.230343+0000","last_unstale":"2024-04-25T10:06:23.208856+0000","last_undegraded":"2024-04-25T10:06:23.208856+0000","last_fullsized":"2024-04-25T10:06:23.208856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:29.570843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.226760+0000","last_change":"2024-04-25T10:06:22.222333+0000","last_active":"2024-04-25T10:06:22.226760+0000","last_peered":"2024-04-25T10:06:22.226760+0000","last_clean":"2024-04-25T10:06:22.226760+0000","last_became_active":"2024-04-25T10:06:22.222021+0000","last_became_peered":"2024-04-25T10:06:22.222021+0000","last_unstale":"2024-04-25T10:06:22.226760+0000","last_undegraded":"2024-04-25T10:06:22.226760+0000","last_fullsized":"2024-04-25T10:06:22.226760+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:11.865103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.206522+0000","last_change":"2024-04-25T10:06:22.251480+0000","last_active":"2024-04-25T10:06:23.206522+0000","last_peered":"2024-04-25T10:06:23.206522+0000","last_clean":"2024-04-25T10:06:23.206522+0000","last_became_active":"2024-04-25T10:06:22.250723+0000","last_became_peered":"2024-04-25T10:06:22.250723+0000","last_unstale":"2024-04-25T10:06:23.206522+0000","last_undegraded":"2024-04-25T10:06:23.206522+0000","last_fullsized":"2024-04-25T10:06:23.206522+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:49.891844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.237360+0000","last_change":"2024-04-25T10:06:22.223614+0000","last_active":"2024-04-25T10:06:22.237360+0000","last_peered":"2024-04-25T10:06:22.237360+0000","last_clean":"2024-04-25T10:06:22.237360+0000","last_became_active":"2024-04-25T10:06:22.220482+0000","last_became_peered":"2024-04-25T10:06:22.220482+0000","last_unstale":"2024-04-25T10:06:22.237360+0000","last_undegraded":"2024-04-25T10:06:22.237360+0000","last_fullsized":"2024-04-25T10:06:22.237360+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:35.778038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.221762+0000","last_change":"2024-04-25T10:06:22.220985+0000","last_active":"2024-04-25T10:06:22.221762+0000","last_peered":"2024-04-25T10:06:22.221762+0000","last_clean":"2024-04-25T10:06:22.221762+0000","last_became_active":"2024-04-25T10:06:22.220662+0000","last_became_peered":"2024-04-25T10:06:22.220662+0000","last_unstale":"2024-04-25T10:06:22.221762+0000","last_undegraded":"2024-04-25T10:06:22.221762+0000","last_fullsized":"2024-04-25T10:06:22.221762+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:02.182289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.216280+0000","last_change":"2024-04-25T10:06:22.212668+0000","last_active":"2024-04-25T10:06:22.216280+0000","last_peered":"2024-04-25T10:06:22.216280+0000","last_clean":"2024-04-25T10:06:22.216280+0000","last_became_active":"2024-04-25T10:06:22.212382+0000","last_became_peered":"2024-04-25T10:06:22.212382+0000","last_unstale":"2024-04-25T10:06:22.216280+0000","last_undegraded":"2024-04-25T10:06:22.216280+0000","last_fullsized":"2024-04-25T10:06:22.216280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:30.807883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.244094+0000","last_change":"2024-04-25T10:06:22.236243+0000","last_active":"2024-04-25T10:06:22.244094+0000","last_peered":"2024-04-25T10:06:22.244094+0000","last_clean":"2024-04-25T10:06:22.244094+0000","last_became_active":"2024-04-25T10:06:22.235488+0000","last_became_peered":"2024-04-25T10:06:22.235488+0000","last_unstale":"2024-04-25T10:06:22.244094+0000","last_undegraded":"2024-04-25T10:06:22.244094+0000","last_fullsized":"2024-04-25T10:06:22.244094+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:33.404890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.206240+0000","last_change":"2024-04-25T10:06:22.256282+0000","last_active":"2024-04-25T10:06:23.206240+0000","last_peered":"2024-04-25T10:06:23.206240+0000","last_clean":"2024-04-25T10:06:23.206240+0000","last_became_active":"2024-04-25T10:06:22.253832+0000","last_became_peered":"2024-04-25T10:06:22.253832+0000","last_unstale":"2024-04-25T10:06:23.206240+0000","last_undegraded":"2024-04-25T10:06:23.206240+0000","last_fullsized":"2024-04-25T10:06:23.206240+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:20.744775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.207417+0000","last_change":"2024-04-25T10:06:22.263384+0000","last_active":"2024-04-25T10:06:23.207417+0000","last_peered":"2024-04-25T10:06:23.207417+0000","last_clean":"2024-04-25T10:06:23.207417+0000","last_became_active":"2024-04-25T10:06:22.261831+0000","last_became_peered":"2024-04-25T10:06:22.261831+0000","last_unstale":"2024-04-25T10:06:23.207417+0000","last_undegraded":"2024-04-25T10:06:23.207417+0000","last_fullsized":"2024-04-25T10:06:23.207417+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:39.568884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.225189+0000","last_change":"2024-04-25T10:06:22.223700+0000","last_active":"2024-04-25T10:06:22.225189+0000","last_peered":"2024-04-25T10:06:22.225189+0000","last_clean":"2024-04-25T10:06:22.225189+0000","last_became_active":"2024-04-25T10:06:22.223222+0000","last_became_peered":"2024-04-25T10:06:22.223222+0000","last_unstale":"2024-04-25T10:06:22.225189+0000","last_undegraded":"2024-04-25T10:06:22.225189+0000","last_fullsized":"2024-04-25T10:06:22.225189+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.821436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.224561+0000","last_change":"2024-04-25T10:06:22.223672+0000","last_active":"2024-04-25T10:06:22.224561+0000","last_peered":"2024-04-25T10:06:22.224561+0000","last_clean":"2024-04-25T10:06:22.224561+0000","last_became_active":"2024-04-25T10:06:22.223238+0000","last_became_peered":"2024-04-25T10:06:22.223238+0000","last_unstale":"2024-04-25T10:06:22.224561+0000","last_undegraded":"2024-04-25T10:06:22.224561+0000","last_fullsized":"2024-04-25T10:06:22.224561+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:08.711769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.235383+0000","last_change":"2024-04-25T10:06:22.234861+0000","last_active":"2024-04-25T10:06:22.235383+0000","last_peered":"2024-04-25T10:06:22.235383+0000","last_clean":"2024-04-25T10:06:22.235383+0000","last_became_active":"2024-04-25T10:06:22.234541+0000","last_became_peered":"2024-04-25T10:06:22.234541+0000","last_unstale":"2024-04-25T10:06:22.235383+0000","last_undegraded":"2024-04-25T10:06:22.235383+0000","last_fullsized":"2024-04-25T10:06:22.235383+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:59.048260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.207877+0000","last_change":"2024-04-25T10:06:22.240249+0000","last_active":"2024-04-25T10:06:23.207877+0000","last_peered":"2024-04-25T10:06:23.207877+0000","last_clean":"2024-04-25T10:06:23.207877+0000","last_became_active":"2024-04-25T10:06:22.238840+0000","last_became_peered":"2024-04-25T10:06:22.238840+0000","last_unstale":"2024-04-25T10:06:23.207877+0000","last_undegraded":"2024-04-25T10:06:23.207877+0000","last_fullsized":"2024-04-25T10:06:23.207877+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:16.392430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.207784+0000","last_change":"2024-04-25T10:06:22.237603+0000","last_active":"2024-04-25T10:06:23.207784+0000","last_peered":"2024-04-25T10:06:23.207784+0000","last_clean":"2024-04-25T10:06:23.207784+0000","last_became_active":"2024-04-25T10:06:22.236765+0000","last_became_peered":"2024-04-25T10:06:22.236765+0000","last_unstale":"2024-04-25T10:06:23.207784+0000","last_undegraded":"2024-04-25T10:06:23.207784+0000","last_fullsized":"2024-04-25T10:06:23.207784+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:22.210589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.250560+0000","last_change":"2024-04-25T10:06:22.249840+0000","last_active":"2024-04-25T10:06:22.250560+0000","last_peered":"2024-04-25T10:06:22.250560+0000","last_clean":"2024-04-25T10:06:22.250560+0000","last_became_active":"2024-04-25T10:06:22.249489+0000","last_became_peered":"2024-04-25T10:06:22.249489+0000","last_unstale":"2024-04-25T10:06:22.250560+0000","last_undegraded":"2024-04-25T10:06:22.250560+0000","last_fullsized":"2024-04-25T10:06:22.250560+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:33.660776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.237232+0000","last_change":"2024-04-25T10:06:22.223941+0000","last_active":"2024-04-25T10:06:22.237232+0000","last_peered":"2024-04-25T10:06:22.237232+0000","last_clean":"2024-04-25T10:06:22.237232+0000","last_became_active":"2024-04-25T10:06:22.221241+0000","last_became_peered":"2024-04-25T10:06:22.221241+0000","last_unstale":"2024-04-25T10:06:22.237232+0000","last_undegraded":"2024-04-25T10:06:22.237232+0000","last_fullsized":"2024-04-25T10:06:22.237232+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:14.614718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.250393+0000","last_change":"2024-04-25T10:06:22.249671+0000","last_active":"2024-04-25T10:06:22.250393+0000","last_peered":"2024-04-25T10:06:22.250393+0000","last_clean":"2024-04-25T10:06:22.250393+0000","last_became_active":"2024-04-25T10:06:22.249384+0000","last_became_peered":"2024-04-25T10:06:22.249384+0000","last_unstale":"2024-04-25T10:06:22.250393+0000","last_undegraded":"2024-04-25T10:06:22.250393+0000","last_fullsized":"2024-04-25T10:06:22.250393+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:07.079403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.253503+0000","last_change":"2024-04-25T10:06:22.252972+0000","last_active":"2024-04-25T10:06:22.253503+0000","last_peered":"2024-04-25T10:06:22.253503+0000","last_clean":"2024-04-25T10:06:22.253503+0000","last_became_active":"2024-04-25T10:06:22.252728+0000","last_became_peered":"2024-04-25T10:06:22.252728+0000","last_unstale":"2024-04-25T10:06:22.253503+0000","last_undegraded":"2024-04-25T10:06:22.253503+0000","last_fullsized":"2024-04-25T10:06:22.253503+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:31.698949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.227227+0000","last_change":"2024-04-25T10:06:22.224194+0000","last_active":"2024-04-25T10:06:22.227227+0000","last_peered":"2024-04-25T10:06:22.227227+0000","last_clean":"2024-04-25T10:06:22.227227+0000","last_became_active":"2024-04-25T10:06:22.223686+0000","last_became_peered":"2024-04-25T10:06:22.223686+0000","last_unstale":"2024-04-25T10:06:22.227227+0000","last_undegraded":"2024-04-25T10:06:22.227227+0000","last_fullsized":"2024-04-25T10:06:22.227227+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:42.882629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.237312+0000","last_change":"2024-04-25T10:06:22.222953+0000","last_active":"2024-04-25T10:06:22.237312+0000","last_peered":"2024-04-25T10:06:22.237312+0000","last_clean":"2024-04-25T10:06:22.237312+0000","last_became_active":"2024-04-25T10:06:22.219544+0000","last_became_peered":"2024-04-25T10:06:22.219544+0000","last_unstale":"2024-04-25T10:06:22.237312+0000","last_undegraded":"2024-04-25T10:06:22.237312+0000","last_fullsized":"2024-04-25T10:06:22.237312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:30.807883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208214+0000","last_change":"2024-04-25T10:06:22.239514+0000","last_active":"2024-04-25T10:06:23.208214+0000","last_peered":"2024-04-25T10:06:23.208214+0000","last_clean":"2024-04-25T10:06:23.208214+0000","last_became_active":"2024-04-25T10:06:22.230693+0000","last_became_peered":"2024-04-25T10:06:22.230693+0000","last_unstale":"2024-04-25T10:06:23.208214+0000","last_undegraded":"2024-04-25T10:06:23.208214+0000","last_fullsized":"2024-04-25T10:06:23.208214+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:54.514623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.205988+0000","last_change":"2024-04-25T10:06:22.262417+0000","last_active":"2024-04-25T10:06:23.205988+0000","last_peered":"2024-04-25T10:06:23.205988+0000","last_clean":"2024-04-25T10:06:23.205988+0000","last_became_active":"2024-04-25T10:06:22.261977+0000","last_became_peered":"2024-04-25T10:06:22.261977+0000","last_unstale":"2024-04-25T10:06:23.205988+0000","last_undegraded":"2024-04-25T10:06:23.205988+0000","last_fullsized":"2024-04-25T10:06:23.205988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:36.452389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.261128+0000","last_change":"2024-04-25T10:06:22.233558+0000","last_active":"2024-04-25T10:06:22.261128+0000","last_peered":"2024-04-25T10:06:22.261128+0000","last_clean":"2024-04-25T10:06:22.261128+0000","last_became_active":"2024-04-25T10:06:22.232812+0000","last_became_peered":"2024-04-25T10:06:22.232812+0000","last_unstale":"2024-04-25T10:06:22.261128+0000","last_undegraded":"2024-04-25T10:06:22.261128+0000","last_fullsized":"2024-04-25T10:06:22.261128+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:49.656616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.205525+0000","last_change":"2024-04-25T10:06:22.258550+0000","last_active":"2024-04-25T10:06:23.205525+0000","last_peered":"2024-04-25T10:06:23.205525+0000","last_clean":"2024-04-25T10:06:23.205525+0000","last_became_active":"2024-04-25T10:06:22.257374+0000","last_became_peered":"2024-04-25T10:06:22.257374+0000","last_unstale":"2024-04-25T10:06:23.205525+0000","last_undegraded":"2024-04-25T10:06:23.205525+0000","last_fullsized":"2024-04-25T10:06:23.205525+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:06.870969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.252956+0000","last_change":"2024-04-25T10:06:22.252406+0000","last_active":"2024-04-25T10:06:22.252956+0000","last_peered":"2024-04-25T10:06:22.252956+0000","last_clean":"2024-04-25T10:06:22.252956+0000","last_became_active":"2024-04-25T10:06:22.252078+0000","last_became_peered":"2024-04-25T10:06:22.252078+0000","last_unstale":"2024-04-25T10:06:22.252956+0000","last_undegraded":"2024-04-25T10:06:22.252956+0000","last_fullsized":"2024-04-25T10:06:22.252956+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:57.040192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.259875+0000","last_change":"2024-04-25T10:06:21.196297+0000","last_active":"2024-04-25T10:06:22.259875+0000","last_peered":"2024-04-25T10:06:22.259875+0000","last_clean":"2024-04-25T10:06:22.259875+0000","last_became_active":"2024-04-25T10:06:21.195531+0000","last_became_peered":"2024-04-25T10:06:21.195531+0000","last_unstale":"2024-04-25T10:06:22.259875+0000","last_undegraded":"2024-04-25T10:06:22.259875+0000","last_fullsized":"2024-04-25T10:06:22.259875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:18.689528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.261442+0000","last_change":"2024-04-25T10:06:22.255434+0000","last_active":"2024-04-25T10:06:22.261442+0000","last_peered":"2024-04-25T10:06:22.261442+0000","last_clean":"2024-04-25T10:06:22.261442+0000","last_became_active":"2024-04-25T10:06:22.252918+0000","last_became_peered":"2024-04-25T10:06:22.252918+0000","last_unstale":"2024-04-25T10:06:22.261442+0000","last_undegraded":"2024-04-25T10:06:22.261442+0000","last_fullsized":"2024-04-25T10:06:22.261442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:46.770914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.245323+0000","last_change":"2024-04-25T10:06:21.180812+0000","last_active":"2024-04-25T10:06:22.245323+0000","last_peered":"2024-04-25T10:06:22.245323+0000","last_clean":"2024-04-25T10:06:22.245323+0000","last_became_active":"2024-04-25T10:06:21.180058+0000","last_became_peered":"2024-04-25T10:06:21.180058+0000","last_unstale":"2024-04-25T10:06:22.245323+0000","last_undegraded":"2024-04-25T10:06:22.245323+0000","last_fullsized":"2024-04-25T10:06:22.245323+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:48.653271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208680+0000","last_change":"2024-04-25T10:06:22.219265+0000","last_active":"2024-04-25T10:06:23.208680+0000","last_peered":"2024-04-25T10:06:23.208680+0000","last_clean":"2024-04-25T10:06:23.208680+0000","last_became_active":"2024-04-25T10:06:22.218449+0000","last_became_peered":"2024-04-25T10:06:22.218449+0000","last_unstale":"2024-04-25T10:06:23.208680+0000","last_undegraded":"2024-04-25T10:06:23.208680+0000","last_fullsized":"2024-04-25T10:06:23.208680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:06.870969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.226145+0000","last_change":"2024-04-25T10:06:21.168434+0000","last_active":"2024-04-25T10:06:22.226145+0000","last_peered":"2024-04-25T10:06:22.226145+0000","last_clean":"2024-04-25T10:06:22.226145+0000","last_became_active":"2024-04-25T10:06:21.167972+0000","last_became_peered":"2024-04-25T10:06:21.167972+0000","last_unstale":"2024-04-25T10:06:22.226145+0000","last_undegraded":"2024-04-25T10:06:22.226145+0000","last_fullsized":"2024-04-25T10:06:22.226145+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:52.499219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.227202+0000","last_change":"2024-04-25T10:06:22.223373+0000","last_active":"2024-04-25T10:06:22.227202+0000","last_peered":"2024-04-25T10:06:22.227202+0000","last_clean":"2024-04-25T10:06:22.227202+0000","last_became_active":"2024-04-25T10:06:22.223093+0000","last_became_peered":"2024-04-25T10:06:22.223093+0000","last_unstale":"2024-04-25T10:06:22.227202+0000","last_undegraded":"2024-04-25T10:06:22.227202+0000","last_fullsized":"2024-04-25T10:06:22.227202+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:08.399858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.220512+0000","last_change":"2024-04-25T10:06:21.174028+0000","last_active":"2024-04-25T10:06:22.220512+0000","last_peered":"2024-04-25T10:06:22.220512+0000","last_clean":"2024-04-25T10:06:22.220512+0000","last_became_active":"2024-04-25T10:06:21.173714+0000","last_became_peered":"2024-04-25T10:06:21.173714+0000","last_unstale":"2024-04-25T10:06:22.220512+0000","last_undegraded":"2024-04-25T10:06:22.220512+0000","last_fullsized":"2024-04-25T10:06:22.220512+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:09.215037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.206362+0000","last_change":"2024-04-25T10:06:22.258149+0000","last_active":"2024-04-25T10:06:23.206362+0000","last_peered":"2024-04-25T10:06:23.206362+0000","last_clean":"2024-04-25T10:06:23.206362+0000","last_became_active":"2024-04-25T10:06:22.257322+0000","last_became_peered":"2024-04-25T10:06:22.257322+0000","last_unstale":"2024-04-25T10:06:23.206362+0000","last_undegraded":"2024-04-25T10:06:23.206362+0000","last_fullsized":"2024-04-25T10:06:23.206362+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:54.457042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208190+0000","last_change":"2024-04-25T10:06:21.193111+0000","last_active":"2024-04-25T10:06:23.208190+0000","last_peered":"2024-04-25T10:06:23.208190+0000","last_clean":"2024-04-25T10:06:23.208190+0000","last_became_active":"2024-04-25T10:06:21.192445+0000","last_became_peered":"2024-04-25T10:06:21.192445+0000","last_unstale":"2024-04-25T10:06:23.208190+0000","last_undegraded":"2024-04-25T10:06:23.208190+0000","last_fullsized":"2024-04-25T10:06:23.208190+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:50.789277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.236940+0000","last_change":"2024-04-25T10:06:22.222486+0000","last_active":"2024-04-25T10:06:22.236940+0000","last_peered":"2024-04-25T10:06:22.236940+0000","last_clean":"2024-04-25T10:06:22.236940+0000","last_became_active":"2024-04-25T10:06:22.221406+0000","last_became_peered":"2024-04-25T10:06:22.221406+0000","last_unstale":"2024-04-25T10:06:22.236940+0000","last_undegraded":"2024-04-25T10:06:22.236940+0000","last_fullsized":"2024-04-25T10:06:22.236940+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:33.576802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.207693+0000","last_change":"2024-04-25T10:06:21.190130+0000","last_active":"2024-04-25T10:06:23.207693+0000","last_peered":"2024-04-25T10:06:23.207693+0000","last_clean":"2024-04-25T10:06:23.207693+0000","last_became_active":"2024-04-25T10:06:21.189305+0000","last_became_peered":"2024-04-25T10:06:21.189305+0000","last_unstale":"2024-04-25T10:06:23.207693+0000","last_undegraded":"2024-04-25T10:06:23.207693+0000","last_fullsized":"2024-04-25T10:06:23.207693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:05.714570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.205981+0000","last_change":"2024-04-25T10:06:22.238585+0000","last_active":"2024-04-25T10:06:23.205981+0000","last_peered":"2024-04-25T10:06:23.205981+0000","last_clean":"2024-04-25T10:06:23.205981+0000","last_became_active":"2024-04-25T10:06:22.237914+0000","last_became_peered":"2024-04-25T10:06:22.237914+0000","last_unstale":"2024-04-25T10:06:23.205981+0000","last_undegraded":"2024-04-25T10:06:23.205981+0000","last_fullsized":"2024-04-25T10:06:23.205981+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:07.079403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.205270+0000","last_change":"2024-04-25T10:06:21.165169+0000","last_active":"2024-04-25T10:06:22.205270+0000","last_peered":"2024-04-25T10:06:22.205270+0000","last_clean":"2024-04-25T10:06:22.205270+0000","last_became_active":"2024-04-25T10:06:21.164840+0000","last_became_peered":"2024-04-25T10:06:21.164840+0000","last_unstale":"2024-04-25T10:06:22.205270+0000","last_undegraded":"2024-04-25T10:06:22.205270+0000","last_fullsized":"2024-04-25T10:06:22.205270+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:56.036847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.221630+0000","last_change":"2024-04-25T10:06:22.220700+0000","last_active":"2024-04-25T10:06:22.221630+0000","last_peered":"2024-04-25T10:06:22.221630+0000","last_clean":"2024-04-25T10:06:22.221630+0000","last_became_active":"2024-04-25T10:06:22.220255+0000","last_became_peered":"2024-04-25T10:06:22.220255+0000","last_unstale":"2024-04-25T10:06:22.221630+0000","last_undegraded":"2024-04-25T10:06:22.221630+0000","last_fullsized":"2024-04-25T10:06:22.221630+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:08.377079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.206234+0000","last_change":"2024-04-25T10:06:21.198140+0000","last_active":"2024-04-25T10:06:23.206234+0000","last_peered":"2024-04-25T10:06:23.206234+0000","last_clean":"2024-04-25T10:06:23.206234+0000","last_became_active":"2024-04-25T10:06:21.197379+0000","last_became_peered":"2024-04-25T10:06:21.197379+0000","last_unstale":"2024-04-25T10:06:23.206234+0000","last_undegraded":"2024-04-25T10:06:23.206234+0000","last_fullsized":"2024-04-25T10:06:23.206234+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:55.726391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.244072+0000","last_change":"2024-04-25T10:06:22.236889+0000","last_active":"2024-04-25T10:06:22.244072+0000","last_peered":"2024-04-25T10:06:22.244072+0000","last_clean":"2024-04-25T10:06:22.244072+0000","last_became_active":"2024-04-25T10:06:22.235866+0000","last_became_peered":"2024-04-25T10:06:22.235866+0000","last_unstale":"2024-04-25T10:06:22.244072+0000","last_undegraded":"2024-04-25T10:06:22.244072+0000","last_fullsized":"2024-04-25T10:06:22.244072+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:02.550775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.205924+0000","last_change":"2024-04-25T10:06:21.180968+0000","last_active":"2024-04-25T10:06:22.205924+0000","last_peered":"2024-04-25T10:06:22.205924+0000","last_clean":"2024-04-25T10:06:22.205924+0000","last_became_active":"2024-04-25T10:06:21.180063+0000","last_became_peered":"2024-04-25T10:06:21.180063+0000","last_unstale":"2024-04-25T10:06:22.205924+0000","last_undegraded":"2024-04-25T10:06:22.205924+0000","last_fullsized":"2024-04-25T10:06:22.205924+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:08.126165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.227250+0000","last_change":"2024-04-25T10:06:22.226431+0000","last_active":"2024-04-25T10:06:22.227250+0000","last_peered":"2024-04-25T10:06:22.227250+0000","last_clean":"2024-04-25T10:06:22.227250+0000","last_became_active":"2024-04-25T10:06:22.226148+0000","last_became_peered":"2024-04-25T10:06:22.226148+0000","last_unstale":"2024-04-25T10:06:22.227250+0000","last_undegraded":"2024-04-25T10:06:22.227250+0000","last_fullsized":"2024-04-25T10:06:22.227250+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:02.550775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.215788+0000","last_change":"2024-04-25T10:06:21.172445+0000","last_active":"2024-04-25T10:06:22.215788+0000","last_peered":"2024-04-25T10:06:22.215788+0000","last_clean":"2024-04-25T10:06:22.215788+0000","last_became_active":"2024-04-25T10:06:21.171542+0000","last_became_peered":"2024-04-25T10:06:21.171542+0000","last_unstale":"2024-04-25T10:06:22.215788+0000","last_undegraded":"2024-04-25T10:06:22.215788+0000","last_fullsized":"2024-04-25T10:06:22.215788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:15.633989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.251006+0000","last_change":"2024-04-25T10:06:22.249640+0000","last_active":"2024-04-25T10:06:22.251006+0000","last_peered":"2024-04-25T10:06:22.251006+0000","last_clean":"2024-04-25T10:06:22.251006+0000","last_became_active":"2024-04-25T10:06:22.249349+0000","last_became_peered":"2024-04-25T10:06:22.249349+0000","last_unstale":"2024-04-25T10:06:22.251006+0000","last_undegraded":"2024-04-25T10:06:22.251006+0000","last_fullsized":"2024-04-25T10:06:22.251006+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:23.241367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.255499+0000","last_change":"2024-04-25T10:06:21.184541+0000","last_active":"2024-04-25T10:06:22.255499+0000","last_peered":"2024-04-25T10:06:22.255499+0000","last_clean":"2024-04-25T10:06:22.255499+0000","last_became_active":"2024-04-25T10:06:21.183778+0000","last_became_peered":"2024-04-25T10:06:21.183778+0000","last_unstale":"2024-04-25T10:06:22.255499+0000","last_undegraded":"2024-04-25T10:06:22.255499+0000","last_fullsized":"2024-04-25T10:06:22.255499+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:52.499219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.242999+0000","last_change":"2024-04-25T10:06:22.229582+0000","last_active":"2024-04-25T10:06:22.242999+0000","last_peered":"2024-04-25T10:06:22.242999+0000","last_clean":"2024-04-25T10:06:22.242999+0000","last_became_active":"2024-04-25T10:06:22.228605+0000","last_became_peered":"2024-04-25T10:06:22.228605+0000","last_unstale":"2024-04-25T10:06:22.242999+0000","last_undegraded":"2024-04-25T10:06:22.242999+0000","last_fullsized":"2024-04-25T10:06:22.242999+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.821436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.257118+0000","last_change":"2024-04-25T10:06:21.168138+0000","last_active":"2024-04-25T10:06:22.257118+0000","last_peered":"2024-04-25T10:06:22.257118+0000","last_clean":"2024-04-25T10:06:22.257118+0000","last_became_active":"2024-04-25T10:06:21.167848+0000","last_became_peered":"2024-04-25T10:06:21.167848+0000","last_unstale":"2024-04-25T10:06:22.257118+0000","last_undegraded":"2024-04-25T10:06:22.257118+0000","last_fullsized":"2024-04-25T10:06:22.257118+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:32.573457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.220333+0000","last_change":"2024-04-25T10:06:22.219395+0000","last_active":"2024-04-25T10:06:22.220333+0000","last_peered":"2024-04-25T10:06:22.220333+0000","last_clean":"2024-04-25T10:06:22.220333+0000","last_became_active":"2024-04-25T10:06:22.219080+0000","last_became_peered":"2024-04-25T10:06:22.219080+0000","last_unstale":"2024-04-25T10:06:22.220333+0000","last_undegraded":"2024-04-25T10:06:22.220333+0000","last_fullsized":"2024-04-25T10:06:22.220333+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:58.977828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208536+0000","last_change":"2024-04-25T10:06:21.199846+0000","last_active":"2024-04-25T10:06:23.208536+0000","last_peered":"2024-04-25T10:06:23.208536+0000","last_clean":"2024-04-25T10:06:23.208536+0000","last_became_active":"2024-04-25T10:06:21.199009+0000","last_became_peered":"2024-04-25T10:06:21.199009+0000","last_unstale":"2024-04-25T10:06:23.208536+0000","last_undegraded":"2024-04-25T10:06:23.208536+0000","last_fullsized":"2024-04-25T10:06:23.208536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:06.034487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.216514+0000","last_change":"2024-04-25T10:06:22.215346+0000","last_active":"2024-04-25T10:06:22.216514+0000","last_peered":"2024-04-25T10:06:22.216514+0000","last_clean":"2024-04-25T10:06:22.216514+0000","last_became_active":"2024-04-25T10:06:22.214423+0000","last_became_peered":"2024-04-25T10:06:22.214423+0000","last_unstale":"2024-04-25T10:06:22.216514+0000","last_undegraded":"2024-04-25T10:06:22.216514+0000","last_fullsized":"2024-04-25T10:06:22.216514+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:35.778038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.201320+0000","last_change":"2024-04-25T10:06:21.184643+0000","last_active":"2024-04-25T10:06:22.201320+0000","last_peered":"2024-04-25T10:06:22.201320+0000","last_clean":"2024-04-25T10:06:22.201320+0000","last_became_active":"2024-04-25T10:06:21.184119+0000","last_became_peered":"2024-04-25T10:06:21.184119+0000","last_unstale":"2024-04-25T10:06:22.201320+0000","last_undegraded":"2024-04-25T10:06:22.201320+0000","last_fullsized":"2024-04-25T10:06:22.201320+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:50.789277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.205856+0000","last_change":"2024-04-25T10:06:22.258959+0000","last_active":"2024-04-25T10:06:23.205856+0000","last_peered":"2024-04-25T10:06:23.205856+0000","last_clean":"2024-04-25T10:06:23.205856+0000","last_became_active":"2024-04-25T10:06:22.257939+0000","last_became_peered":"2024-04-25T10:06:22.257939+0000","last_unstale":"2024-04-25T10:06:23.205856+0000","last_undegraded":"2024-04-25T10:06:23.205856+0000","last_fullsized":"2024-04-25T10:06:23.205856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:29.570843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.205368+0000","last_change":"2024-04-25T10:06:21.180558+0000","last_active":"2024-04-25T10:06:22.205368+0000","last_peered":"2024-04-25T10:06:22.205368+0000","last_clean":"2024-04-25T10:06:22.205368+0000","last_became_active":"2024-04-25T10:06:21.180238+0000","last_became_peered":"2024-04-25T10:06:21.180238+0000","last_unstale":"2024-04-25T10:06:22.205368+0000","last_undegraded":"2024-04-25T10:06:22.205368+0000","last_fullsized":"2024-04-25T10:06:22.205368+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:26.115979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.261505+0000","last_change":"2024-04-25T10:06:22.245367+0000","last_active":"2024-04-25T10:06:22.261505+0000","last_peered":"2024-04-25T10:06:22.261505+0000","last_clean":"2024-04-25T10:06:22.261505+0000","last_became_active":"2024-04-25T10:06:22.244679+0000","last_became_peered":"2024-04-25T10:06:22.244679+0000","last_unstale":"2024-04-25T10:06:22.261505+0000","last_undegraded":"2024-04-25T10:06:22.261505+0000","last_fullsized":"2024-04-25T10:06:22.261505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:16.637334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.227843+0000","last_change":"2024-04-25T10:06:21.168016+0000","last_active":"2024-04-25T10:06:22.227843+0000","last_peered":"2024-04-25T10:06:22.227843+0000","last_clean":"2024-04-25T10:06:22.227843+0000","last_became_active":"2024-04-25T10:06:21.167676+0000","last_became_peered":"2024-04-25T10:06:21.167676+0000","last_unstale":"2024-04-25T10:06:22.227843+0000","last_undegraded":"2024-04-25T10:06:22.227843+0000","last_fullsized":"2024-04-25T10:06:22.227843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:03.425191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208559+0000","last_change":"2024-04-25T10:06:22.244819+0000","last_active":"2024-04-25T10:06:23.208559+0000","last_peered":"2024-04-25T10:06:23.208559+0000","last_clean":"2024-04-25T10:06:23.208559+0000","last_became_active":"2024-04-25T10:06:22.244100+0000","last_became_peered":"2024-04-25T10:06:22.244100+0000","last_unstale":"2024-04-25T10:06:23.208559+0000","last_undegraded":"2024-04-25T10:06:23.208559+0000","last_fullsized":"2024-04-25T10:06:23.208559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:19.349171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.225833+0000","last_change":"2024-04-25T10:06:21.176489+0000","last_active":"2024-04-25T10:06:22.225833+0000","last_peered":"2024-04-25T10:06:22.225833+0000","last_clean":"2024-04-25T10:06:22.225833+0000","last_became_active":"2024-04-25T10:06:21.174781+0000","last_became_peered":"2024-04-25T10:06:21.174781+0000","last_unstale":"2024-04-25T10:06:22.225833+0000","last_undegraded":"2024-04-25T10:06:22.225833+0000","last_fullsized":"2024-04-25T10:06:22.225833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:24.318301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.257967+0000","last_change":"2024-04-25T10:06:22.254281+0000","last_active":"2024-04-25T10:06:22.257967+0000","last_peered":"2024-04-25T10:06:22.257967+0000","last_clean":"2024-04-25T10:06:22.257967+0000","last_became_active":"2024-04-25T10:06:22.243902+0000","last_became_peered":"2024-04-25T10:06:22.243902+0000","last_unstale":"2024-04-25T10:06:22.257967+0000","last_undegraded":"2024-04-25T10:06:22.257967+0000","last_fullsized":"2024-04-25T10:06:22.257967+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:23.241367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208774+0000","last_change":"2024-04-25T10:06:21.193080+0000","last_active":"2024-04-25T10:06:23.208774+0000","last_peered":"2024-04-25T10:06:23.208774+0000","last_clean":"2024-04-25T10:06:23.208774+0000","last_became_active":"2024-04-25T10:06:21.192259+0000","last_became_peered":"2024-04-25T10:06:21.192259+0000","last_unstale":"2024-04-25T10:06:23.208774+0000","last_undegraded":"2024-04-25T10:06:23.208774+0000","last_fullsized":"2024-04-25T10:06:23.208774+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:38.038262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.230183+0000","last_change":"2024-04-25T10:06:22.229688+0000","last_active":"2024-04-25T10:06:22.230183+0000","last_peered":"2024-04-25T10:06:22.230183+0000","last_clean":"2024-04-25T10:06:22.230183+0000","last_became_active":"2024-04-25T10:06:22.229365+0000","last_became_peered":"2024-04-25T10:06:22.229365+0000","last_unstale":"2024-04-25T10:06:22.230183+0000","last_undegraded":"2024-04-25T10:06:22.230183+0000","last_fullsized":"2024-04-25T10:06:22.230183+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:49.701955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208542+0000","last_change":"2024-04-25T10:06:21.191734+0000","last_active":"2024-04-25T10:06:23.208542+0000","last_peered":"2024-04-25T10:06:23.208542+0000","last_clean":"2024-04-25T10:06:23.208542+0000","last_became_active":"2024-04-25T10:06:21.190643+0000","last_became_peered":"2024-04-25T10:06:21.190643+0000","last_unstale":"2024-04-25T10:06:23.208542+0000","last_undegraded":"2024-04-25T10:06:23.208542+0000","last_fullsized":"2024-04-25T10:06:23.208542+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:09.818048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.252544+0000","last_change":"2024-04-25T10:06:22.251943+0000","last_active":"2024-04-25T10:06:22.252544+0000","last_peered":"2024-04-25T10:06:22.252544+0000","last_clean":"2024-04-25T10:06:22.252544+0000","last_became_active":"2024-04-25T10:06:22.251649+0000","last_became_peered":"2024-04-25T10:06:22.251649+0000","last_unstale":"2024-04-25T10:06:22.252544+0000","last_undegraded":"2024-04-25T10:06:22.252544+0000","last_fullsized":"2024-04-25T10:06:22.252544+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:14.983380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208147+0000","last_change":"2024-04-25T10:06:21.190153+0000","last_active":"2024-04-25T10:06:23.208147+0000","last_peered":"2024-04-25T10:06:23.208147+0000","last_clean":"2024-04-25T10:06:23.208147+0000","last_became_active":"2024-04-25T10:06:21.189403+0000","last_became_peered":"2024-04-25T10:06:21.189403+0000","last_unstale":"2024-04-25T10:06:23.208147+0000","last_undegraded":"2024-04-25T10:06:23.208147+0000","last_fullsized":"2024-04-25T10:06:23.208147+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:42.024042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.243947+0000","last_change":"2024-04-25T10:06:22.243492+0000","last_active":"2024-04-25T10:06:22.243947+0000","last_peered":"2024-04-25T10:06:22.243947+0000","last_clean":"2024-04-25T10:06:22.243947+0000","last_became_active":"2024-04-25T10:06:22.243175+0000","last_became_peered":"2024-04-25T10:06:22.243175+0000","last_unstale":"2024-04-25T10:06:22.243947+0000","last_undegraded":"2024-04-25T10:06:22.243947+0000","last_fullsized":"2024-04-25T10:06:22.243947+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:39.934546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.227960+0000","last_change":"2024-04-25T10:06:21.177337+0000","last_active":"2024-04-25T10:06:22.227960+0000","last_peered":"2024-04-25T10:06:22.227960+0000","last_clean":"2024-04-25T10:06:22.227960+0000","last_became_active":"2024-04-25T10:06:21.176990+0000","last_became_peered":"2024-04-25T10:06:21.176990+0000","last_unstale":"2024-04-25T10:06:22.227960+0000","last_undegraded":"2024-04-25T10:06:22.227960+0000","last_fullsized":"2024-04-25T10:06:22.227960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:12.144339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.217062+0000","last_change":"2024-04-25T10:06:22.215358+0000","last_active":"2024-04-25T10:06:22.217062+0000","last_peered":"2024-04-25T10:06:22.217062+0000","last_clean":"2024-04-25T10:06:22.217062+0000","last_became_active":"2024-04-25T10:06:22.215016+0000","last_became_peered":"2024-04-25T10:06:22.215016+0000","last_unstale":"2024-04-25T10:06:22.217062+0000","last_undegraded":"2024-04-25T10:06:22.217062+0000","last_fullsized":"2024-04-25T10:06:22.217062+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:42.822768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.225117+0000","last_change":"2024-04-25T10:06:21.197358+0000","last_active":"2024-04-25T10:06:22.225117+0000","last_peered":"2024-04-25T10:06:22.225117+0000","last_clean":"2024-04-25T10:06:22.225117+0000","last_became_active":"2024-04-25T10:06:21.196637+0000","last_became_peered":"2024-04-25T10:06:21.196637+0000","last_unstale":"2024-04-25T10:06:22.225117+0000","last_undegraded":"2024-04-25T10:06:22.225117+0000","last_fullsized":"2024-04-25T10:06:22.225117+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:32.153259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.216523+0000","last_change":"2024-04-25T10:06:22.215062+0000","last_active":"2024-04-25T10:06:22.216523+0000","last_peered":"2024-04-25T10:06:22.216523+0000","last_clean":"2024-04-25T10:06:22.216523+0000","last_became_active":"2024-04-25T10:06:22.214759+0000","last_became_peered":"2024-04-25T10:06:22.214759+0000","last_unstale":"2024-04-25T10:06:22.216523+0000","last_undegraded":"2024-04-25T10:06:22.216523+0000","last_fullsized":"2024-04-25T10:06:22.216523+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:59.421390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.207040+0000","last_change":"2024-04-25T10:06:21.189994+0000","last_active":"2024-04-25T10:06:23.207040+0000","last_peered":"2024-04-25T10:06:23.207040+0000","last_clean":"2024-04-25T10:06:23.207040+0000","last_became_active":"2024-04-25T10:06:21.189247+0000","last_became_peered":"2024-04-25T10:06:21.189247+0000","last_unstale":"2024-04-25T10:06:23.207040+0000","last_undegraded":"2024-04-25T10:06:23.207040+0000","last_fullsized":"2024-04-25T10:06:23.207040+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:42.582923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.205937+0000","last_change":"2024-04-25T10:06:22.262596+0000","last_active":"2024-04-25T10:06:23.205937+0000","last_peered":"2024-04-25T10:06:23.205937+0000","last_clean":"2024-04-25T10:06:23.205937+0000","last_became_active":"2024-04-25T10:06:22.262004+0000","last_became_peered":"2024-04-25T10:06:22.262004+0000","last_unstale":"2024-04-25T10:06:23.205937+0000","last_undegraded":"2024-04-25T10:06:23.205937+0000","last_fullsized":"2024-04-25T10:06:23.205937+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:06.026925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.209835+0000","last_change":"2024-04-25T10:06:21.185975+0000","last_active":"2024-04-25T10:06:22.209835+0000","last_peered":"2024-04-25T10:06:22.209835+0000","last_clean":"2024-04-25T10:06:22.209835+0000","last_became_active":"2024-04-25T10:06:21.185003+0000","last_became_peered":"2024-04-25T10:06:21.185003+0000","last_unstale":"2024-04-25T10:06:22.209835+0000","last_undegraded":"2024-04-25T10:06:22.209835+0000","last_fullsized":"2024-04-25T10:06:22.209835+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:08.126165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.237097+0000","last_change":"2024-04-25T10:06:22.236587+0000","last_active":"2024-04-25T10:06:22.237097+0000","last_peered":"2024-04-25T10:06:22.237097+0000","last_clean":"2024-04-25T10:06:22.237097+0000","last_became_active":"2024-04-25T10:06:22.236249+0000","last_became_peered":"2024-04-25T10:06:22.236249+0000","last_unstale":"2024-04-25T10:06:22.237097+0000","last_undegraded":"2024-04-25T10:06:22.237097+0000","last_fullsized":"2024-04-25T10:06:22.237097+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:29.187792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.242140+0000","last_change":"2024-04-25T10:06:21.180674+0000","last_active":"2024-04-25T10:06:22.242140+0000","last_peered":"2024-04-25T10:06:22.242140+0000","last_clean":"2024-04-25T10:06:22.242140+0000","last_became_active":"2024-04-25T10:06:21.180369+0000","last_became_peered":"2024-04-25T10:06:21.180369+0000","last_unstale":"2024-04-25T10:06:22.242140+0000","last_undegraded":"2024-04-25T10:06:22.242140+0000","last_fullsized":"2024-04-25T10:06:22.242140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:54.064803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.254777+0000","last_change":"2024-04-25T10:06:22.245552+0000","last_active":"2024-04-25T10:06:22.254777+0000","last_peered":"2024-04-25T10:06:22.254777+0000","last_clean":"2024-04-25T10:06:22.254777+0000","last_became_active":"2024-04-25T10:06:22.243081+0000","last_became_peered":"2024-04-25T10:06:22.243081+0000","last_unstale":"2024-04-25T10:06:22.254777+0000","last_undegraded":"2024-04-25T10:06:22.254777+0000","last_fullsized":"2024-04-25T10:06:22.254777+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:40.277052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.215916+0000","last_change":"2024-04-25T10:06:21.179129+0000","last_active":"2024-04-25T10:06:22.215916+0000","last_peered":"2024-04-25T10:06:22.215916+0000","last_clean":"2024-04-25T10:06:22.215916+0000","last_became_active":"2024-04-25T10:06:21.178801+0000","last_became_peered":"2024-04-25T10:06:21.178801+0000","last_unstale":"2024-04-25T10:06:22.215916+0000","last_undegraded":"2024-04-25T10:06:22.215916+0000","last_fullsized":"2024-04-25T10:06:22.215916+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:13.117982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.228902+0000","last_change":"2024-04-25T10:06:22.228130+0000","last_active":"2024-04-25T10:06:22.228902+0000","last_peered":"2024-04-25T10:06:22.228902+0000","last_clean":"2024-04-25T10:06:22.228902+0000","last_became_active":"2024-04-25T10:06:22.227872+0000","last_became_peered":"2024-04-25T10:06:22.227872+0000","last_unstale":"2024-04-25T10:06:22.228902+0000","last_undegraded":"2024-04-25T10:06:22.228902+0000","last_fullsized":"2024-04-25T10:06:22.228902+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:46.770914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.201863+0000","last_change":"2024-04-25T10:06:21.173661+0000","last_active":"2024-04-25T10:06:22.201863+0000","last_peered":"2024-04-25T10:06:22.201863+0000","last_clean":"2024-04-25T10:06:22.201863+0000","last_became_active":"2024-04-25T10:06:21.172981+0000","last_became_peered":"2024-04-25T10:06:21.172981+0000","last_unstale":"2024-04-25T10:06:22.201863+0000","last_undegraded":"2024-04-25T10:06:22.201863+0000","last_fullsized":"2024-04-25T10:06:22.201863+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:06.034487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.241946+0000","last_change":"2024-04-25T10:06:22.231446+0000","last_active":"2024-04-25T10:06:22.241946+0000","last_peered":"2024-04-25T10:06:22.241946+0000","last_clean":"2024-04-25T10:06:22.241946+0000","last_became_active":"2024-04-25T10:06:22.230802+0000","last_became_peered":"2024-04-25T10:06:22.230802+0000","last_unstale":"2024-04-25T10:06:22.241946+0000","last_undegraded":"2024-04-25T10:06:22.241946+0000","last_fullsized":"2024-04-25T10:06:22.241946+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:33.567907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.213725+0000","last_change":"2024-04-25T10:06:21.175341+0000","last_active":"2024-04-25T10:06:22.213725+0000","last_peered":"2024-04-25T10:06:22.213725+0000","last_clean":"2024-04-25T10:06:22.213725+0000","last_became_active":"2024-04-25T10:06:21.175036+0000","last_became_peered":"2024-04-25T10:06:21.175036+0000","last_unstale":"2024-04-25T10:06:22.213725+0000","last_undegraded":"2024-04-25T10:06:22.213725+0000","last_fullsized":"2024-04-25T10:06:22.213725+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:32.564562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.232171+0000","last_change":"2024-04-25T10:06:22.231628+0000","last_active":"2024-04-25T10:06:22.232171+0000","last_peered":"2024-04-25T10:06:22.232171+0000","last_clean":"2024-04-25T10:06:22.232171+0000","last_became_active":"2024-04-25T10:06:22.231360+0000","last_became_peered":"2024-04-25T10:06:22.231360+0000","last_unstale":"2024-04-25T10:06:22.232171+0000","last_undegraded":"2024-04-25T10:06:22.232171+0000","last_fullsized":"2024-04-25T10:06:22.232171+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:44.331490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.201488+0000","last_change":"2024-04-25T10:06:21.180608+0000","last_active":"2024-04-25T10:06:22.201488+0000","last_peered":"2024-04-25T10:06:22.201488+0000","last_clean":"2024-04-25T10:06:22.201488+0000","last_became_active":"2024-04-25T10:06:21.180323+0000","last_became_peered":"2024-04-25T10:06:21.180323+0000","last_unstale":"2024-04-25T10:06:22.201488+0000","last_undegraded":"2024-04-25T10:06:22.201488+0000","last_fullsized":"2024-04-25T10:06:22.201488+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:42.582923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.224943+0000","last_change":"2024-04-25T10:06:22.223967+0000","last_active":"2024-04-25T10:06:22.224943+0000","last_peered":"2024-04-25T10:06:22.224943+0000","last_clean":"2024-04-25T10:06:22.224943+0000","last_became_active":"2024-04-25T10:06:22.223047+0000","last_became_peered":"2024-04-25T10:06:22.223047+0000","last_unstale":"2024-04-25T10:06:22.224943+0000","last_undegraded":"2024-04-25T10:06:22.224943+0000","last_fullsized":"2024-04-25T10:06:22.224943+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:33.404890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.207272+0000","last_change":"2024-04-25T10:06:21.175353+0000","last_active":"2024-04-25T10:06:23.207272+0000","last_peered":"2024-04-25T10:06:23.207272+0000","last_clean":"2024-04-25T10:06:23.207272+0000","last_became_active":"2024-04-25T10:06:21.174456+0000","last_became_peered":"2024-04-25T10:06:21.174456+0000","last_unstale":"2024-04-25T10:06:23.207272+0000","last_undegraded":"2024-04-25T10:06:23.207272+0000","last_fullsized":"2024-04-25T10:06:23.207272+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:54.064803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.206792+0000","last_change":"2024-04-25T10:06:22.241124+0000","last_active":"2024-04-25T10:06:23.206792+0000","last_peered":"2024-04-25T10:06:23.206792+0000","last_clean":"2024-04-25T10:06:23.206792+0000","last_became_active":"2024-04-25T10:06:22.230784+0000","last_became_peered":"2024-04-25T10:06:22.230784+0000","last_unstale":"2024-04-25T10:06:23.206792+0000","last_undegraded":"2024-04-25T10:06:23.206792+0000","last_fullsized":"2024-04-25T10:06:23.206792+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:36.091611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.205551+0000","last_change":"2024-04-25T10:06:21.187081+0000","last_active":"2024-04-25T10:06:22.205551+0000","last_peered":"2024-04-25T10:06:22.205551+0000","last_clean":"2024-04-25T10:06:22.205551+0000","last_became_active":"2024-04-25T10:06:21.186416+0000","last_became_peered":"2024-04-25T10:06:21.186416+0000","last_unstale":"2024-04-25T10:06:22.205551+0000","last_undegraded":"2024-04-25T10:06:22.205551+0000","last_fullsized":"2024-04-25T10:06:22.205551+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:54.296461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.206546+0000","last_change":"2024-04-25T10:06:22.251022+0000","last_active":"2024-04-25T10:06:23.206546+0000","last_peered":"2024-04-25T10:06:23.206546+0000","last_clean":"2024-04-25T10:06:23.206546+0000","last_became_active":"2024-04-25T10:06:22.250190+0000","last_became_peered":"2024-04-25T10:06:22.250190+0000","last_unstale":"2024-04-25T10:06:23.206546+0000","last_undegraded":"2024-04-25T10:06:23.206546+0000","last_fullsized":"2024-04-25T10:06:23.206546+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:09.093081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.229942+0000","last_change":"2024-04-25T10:06:21.196244+0000","last_active":"2024-04-25T10:06:22.229942+0000","last_peered":"2024-04-25T10:06:22.229942+0000","last_clean":"2024-04-25T10:06:22.229942+0000","last_became_active":"2024-04-25T10:06:21.195589+0000","last_became_peered":"2024-04-25T10:06:21.195589+0000","last_unstale":"2024-04-25T10:06:22.229942+0000","last_undegraded":"2024-04-25T10:06:22.229942+0000","last_fullsized":"2024-04-25T10:06:22.229942+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:12.144339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":195,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.206410+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:06:23.206410+0000","last_peered":"2024-04-25T10:06:23.206410+0000","last_clean":"2024-04-25T10:06:23.206410+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:06:23.206410+0000","last_undegraded":"2024-04-25T10:06:23.206410+0000","last_fullsized":"2024-04-25T10:06:23.206410+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":974848,"data_stored":738202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":147848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":148360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":148416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":364544,"data_stored":140969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":364544,"data_stored":140969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35700,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336140,"statfs":{"total":96636764160,"available":96600207360,"internally_reserved":0,"allocated":364544,"data_stored":140969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":958464,"data_stored":731337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:25.278 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:25.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.419+0000 7f8729335640 1 Processor -- start 2024-04-25T10:06:25.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.419+0000 7f8729335640 1 -- start start 2024-04-25T10:06:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.423+0000 7f8729335640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f87240ce7a0 0x7f87240ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.423+0000 7f8729335640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f87240b4660 0x7f87240b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.423+0000 7f8729335640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87240cf0c0 con 0x7f872415bc90 2024-04-25T10:06:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.423+0000 7f8729335640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87240cc370 con 0x7f87240b4660 2024-04-25T10:06:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.423+0000 7f8729335640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8724071ca0 con 0x7f87240ce7a0 2024-04-25T10:06:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.423+0000 7f87237fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f87240b4660 0x7f87240b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.423+0000 7f871bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f872415bc90 0x7f87240b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45876/0 (socket says 172.21.15.60:45876) 2024-04-25T10:06:25.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.423+0000 7f87237fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f87240b4660 0x7f87240b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37792/0 (socket says 172.21.15.60:37792) 2024-04-25T10:06:25.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.423+0000 7f871bfff640 1 -- 172.21.15.60:0/1765809741 learned_addr learned my addr 172.21.15.60:0/1765809741 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:25.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.423+0000 7f87237fe640 1 -- 172.21.15.60:0/1765809741 >> v2:172.21.15.100:3300/0 conn(0x7f87240ce7a0 msgr2=0x7f87240ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:25.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.423+0000 7f87237fe640 1 --2- 172.21.15.60:0/1765809741 >> v2:172.21.15.100:3300/0 conn(0x7f87240ce7a0 0x7f87240ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.423+0000 7f87237fe640 1 -- 172.21.15.60:0/1765809741 >> v1:172.21.15.60:6789/0 conn(0x7f872415bc90 legacy=0x7f87240b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.423+0000 7f87237fe640 1 -- 172.21.15.60:0/1765809741 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87240cca40 con 0x7f87240b4660 2024-04-25T10:06:25.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.423+0000 7f87227fc640 1 -- 172.21.15.60:0/1765809741 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3487509007 0 0) 0x7f87240cf0c0 con 0x7f872415bc90 2024-04-25T10:06:25.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f87237fe640 1 --2- 172.21.15.60:0/1765809741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240b4660 0x7f87240b2ce0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f872404f5d0 tx=0x7f871c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1958e863d3ff50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f87227fc640 1 -- 172.21.15.60:0/1765809741 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c0b2070 con 0x7f87240b4660 2024-04-25T10:06:25.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f87227fc640 1 -- 172.21.15.60:0/1765809741 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c0a57f0 con 0x7f87240b4660 2024-04-25T10:06:25.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f87227fc640 1 -- 172.21.15.60:0/1765809741 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c0a5bc0 con 0x7f87240b4660 2024-04-25T10:06:25.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8729335640 1 -- 172.21.15.60:0/1765809741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240b4660 msgr2=0x7f87240b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8729335640 1 --2- 172.21.15.60:0/1765809741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240b4660 0x7f87240b2ce0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f872404f5d0 tx=0x7f871c01c930 comp rx=0 tx=0).stop 2024-04-25T10:06:25.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8729335640 1 -- 172.21.15.60:0/1765809741 shutdown_connections 2024-04-25T10:06:25.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8729335640 1 --2- 172.21.15.60:0/1765809741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240b4660 0x7f87240b2ce0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8729335640 1 --2- 172.21.15.60:0/1765809741 >> v2:172.21.15.100:3300/0 conn(0x7f87240ce7a0 0x7f87240ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8729335640 1 -- 172.21.15.60:0/1765809741 >> 172.21.15.60:0/1765809741 conn(0x7f87240b12d0 msgr2=0x7f87240b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:25.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8729335640 1 -- 172.21.15.60:0/1765809741 shutdown_connections 2024-04-25T10:06:25.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8729335640 1 -- 172.21.15.60:0/1765809741 wait complete. 2024-04-25T10:06:25.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8729335640 1 Processor -- start 2024-04-25T10:06:25.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8729335640 1 -- start start 2024-04-25T10:06:25.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8729335640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87240ce7a0 0x7f8724156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8729335640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872415bc90 0x7f872414d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8729335640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8724153dd0 0x7f872414d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8729335640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87240b4400 con 0x7f872415bc90 2024-04-25T10:06:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8729335640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87240b4280 con 0x7f8724153dd0 2024-04-25T10:06:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8729335640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87240b3220 con 0x7f87240ce7a0 2024-04-25T10:06:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f871bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872415bc90 0x7f872414d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f871bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872415bc90 0x7f872414d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37796/0 (socket says 172.21.15.60:37796) 2024-04-25T10:06:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f871bfff640 1 -- 172.21.15.60:0/2183168839 learned_addr learned my addr 172.21.15.60:0/2183168839 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f8722ffd640 1 --2- 172.21.15.60:0/2183168839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87240ce7a0 0x7f8724156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f87237fe640 1 --2- 172.21.15.60:0/2183168839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8724153dd0 0x7f872414d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f871bfff640 1 -- 172.21.15.60:0/2183168839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87240ce7a0 msgr2=0x7f8724156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f871bfff640 1 --2- 172.21.15.60:0/2183168839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87240ce7a0 0x7f8724156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f871bfff640 1 -- 172.21.15.60:0/2183168839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8724153dd0 msgr2=0x7f872414d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f871bfff640 1 --2- 172.21.15.60:0/2183168839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8724153dd0 0x7f872414d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f871bfff640 1 -- 172.21.15.60:0/2183168839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872406e960 con 0x7f872415bc90 2024-04-25T10:06:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.427+0000 7f871bfff640 1 --2- 172.21.15.60:0/2183168839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872415bc90 0x7f872414d0b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f870c00c4b0 tx=0x7f870c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.431+0000 7f871b7fe640 1 -- 172.21.15.60:0/2183168839 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c018070 con 0x7f872415bc90 2024-04-25T10:06:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.431+0000 7f871b7fe640 1 -- 172.21.15.60:0/2183168839 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870c007500 con 0x7f872415bc90 2024-04-25T10:06:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.431+0000 7f8729335640 1 -- 172.21.15.60:0/2183168839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87241533c0 con 0x7f872415bc90 2024-04-25T10:06:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.431+0000 7f871b7fe640 1 -- 172.21.15.60:0/2183168839 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c004a30 con 0x7f872415bc90 2024-04-25T10:06:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.431+0000 7f8729335640 1 -- 172.21.15.60:0/2183168839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872406e680 con 0x7f872415bc90 2024-04-25T10:06:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.431+0000 7f8729335640 1 -- 172.21.15.60:0/2183168839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8724074770 con 0x7f872415bc90 2024-04-25T10:06:25.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.431+0000 7f871b7fe640 1 -- 172.21.15.60:0/2183168839 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f870c01b070 con 0x7f872415bc90 2024-04-25T10:06:25.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.431+0000 7f871b7fe640 1 --2- 172.21.15.60:0/2183168839 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8704066f60 0x7f8704069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.431+0000 7f871b7fe640 1 -- 172.21.15.60:0/2183168839 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f870c0842b0 con 0x7f872415bc90 2024-04-25T10:06:25.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.431+0000 7f8722ffd640 1 --2- 172.21.15.60:0/2183168839 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8704066f60 0x7f8704069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.431+0000 7f8722ffd640 1 --2- 172.21.15.60:0/2183168839 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8704066f60 0x7f8704069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8714006fd0 tx=0x7f8714008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.435+0000 7f871b7fe640 1 -- 172.21.15.60:0/2183168839 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f870c0552c0 con 0x7f872415bc90 2024-04-25T10:06:25.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.587+0000 7f8729335640 1 -- 172.21.15.60:0/2183168839 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f872415cf80 con 0x7f8704066f60 2024-04-25T10:06:25.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.603+0000 7f871b7fe640 1 -- 172.21.15.60:0/2183168839 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251906 (secure 0 0 0) 0x7f872415cf80 con 0x7f8704066f60 2024-04-25T10:06:25.615 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:25.616 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:06:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.611+0000 7f8729335640 1 -- 172.21.15.60:0/2183168839 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8704066f60 msgr2=0x7f8704069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.611+0000 7f8729335640 1 --2- 172.21.15.60:0/2183168839 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8704066f60 0x7f8704069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8714006fd0 tx=0x7f8714008040 comp rx=0 tx=0).stop 2024-04-25T10:06:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.611+0000 7f8729335640 1 -- 172.21.15.60:0/2183168839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872415bc90 msgr2=0x7f872414d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.611+0000 7f8729335640 1 --2- 172.21.15.60:0/2183168839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872415bc90 0x7f872414d0b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f870c00c4b0 tx=0x7f870c00c970 comp rx=0 tx=0).stop 2024-04-25T10:06:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.611+0000 7f8729335640 1 -- 172.21.15.60:0/2183168839 shutdown_connections 2024-04-25T10:06:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.611+0000 7f8729335640 1 --2- 172.21.15.60:0/2183168839 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8704066f60 0x7f8704069420 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.611+0000 7f8729335640 1 --2- 172.21.15.60:0/2183168839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8724153dd0 0x7f872414d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.611+0000 7f8729335640 1 --2- 172.21.15.60:0/2183168839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872415bc90 0x7f872414d0b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.611+0000 7f8729335640 1 --2- 172.21.15.60:0/2183168839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87240ce7a0 0x7f8724156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.611+0000 7f8729335640 1 -- 172.21.15.60:0/2183168839 >> 172.21.15.60:0/2183168839 conn(0x7f87240b12d0 msgr2=0x7f872415b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:25.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.611+0000 7f8729335640 1 -- 172.21.15.60:0/2183168839 shutdown_connections 2024-04-25T10:06:25.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.611+0000 7f8729335640 1 -- 172.21.15.60:0/2183168839 wait complete. 2024-04-25T10:06:25.632 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-04-25T10:06:24.408593+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245220,"kb_used_data":4084,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754729500,"statfs":{"total":773094113280,"available":772843008000,"internally_reserved":0,"allocated":4182016,"data_stored":2337070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.261607"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.261413+0000","last_change":"2024-04-25T10:06:22.255457+0000","last_active":"2024-04-25T10:06:22.261413+0000","last_peered":"2024-04-25T10:06:22.261413+0000","last_clean":"2024-04-25T10:06:22.261413+0000","last_became_active":"2024-04-25T10:06:22.253854+0000","last_became_peered":"2024-04-25T10:06:22.253854+0000","last_unstale":"2024-04-25T10:06:22.261413+0000","last_undegraded":"2024-04-25T10:06:22.261413+0000","last_fullsized":"2024-04-25T10:06:22.261413+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:14.121327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.219127+0000","last_change":"2024-04-25T10:06:22.217046+0000","last_active":"2024-04-25T10:06:22.219127+0000","last_peered":"2024-04-25T10:06:22.219127+0000","last_clean":"2024-04-25T10:06:22.219127+0000","last_became_active":"2024-04-25T10:06:22.216754+0000","last_became_peered":"2024-04-25T10:06:22.216754+0000","last_unstale":"2024-04-25T10:06:22.219127+0000","last_undegraded":"2024-04-25T10:06:22.219127+0000","last_fullsized":"2024-04-25T10:06:22.219127+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:14.614718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.237958+0000","last_change":"2024-04-25T10:06:22.227055+0000","last_active":"2024-04-25T10:06:22.237958+0000","last_peered":"2024-04-25T10:06:22.237958+0000","last_clean":"2024-04-25T10:06:22.237958+0000","last_became_active":"2024-04-25T10:06:22.226372+0000","last_became_peered":"2024-04-25T10:06:22.226372+0000","last_unstale":"2024-04-25T10:06:22.237958+0000","last_undegraded":"2024-04-25T10:06:22.237958+0000","last_fullsized":"2024-04-25T10:06:22.237958+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:08.711769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.230197+0000","last_change":"2024-04-25T10:06:22.229697+0000","last_active":"2024-04-25T10:06:22.230197+0000","last_peered":"2024-04-25T10:06:22.230197+0000","last_clean":"2024-04-25T10:06:22.230197+0000","last_became_active":"2024-04-25T10:06:22.229417+0000","last_became_peered":"2024-04-25T10:06:22.229417+0000","last_unstale":"2024-04-25T10:06:22.230197+0000","last_undegraded":"2024-04-25T10:06:22.230197+0000","last_fullsized":"2024-04-25T10:06:22.230197+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:51.847161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.217697+0000","last_change":"2024-04-25T10:06:22.216363+0000","last_active":"2024-04-25T10:06:22.217697+0000","last_peered":"2024-04-25T10:06:22.217697+0000","last_clean":"2024-04-25T10:06:22.217697+0000","last_became_active":"2024-04-25T10:06:22.216118+0000","last_became_peered":"2024-04-25T10:06:22.216118+0000","last_unstale":"2024-04-25T10:06:22.217697+0000","last_undegraded":"2024-04-25T10:06:22.217697+0000","last_fullsized":"2024-04-25T10:06:22.217697+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:03.419847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.241785+0000","last_change":"2024-04-25T10:06:22.241246+0000","last_active":"2024-04-25T10:06:22.241785+0000","last_peered":"2024-04-25T10:06:22.241785+0000","last_clean":"2024-04-25T10:06:22.241785+0000","last_became_active":"2024-04-25T10:06:22.241010+0000","last_became_peered":"2024-04-25T10:06:22.241010+0000","last_unstale":"2024-04-25T10:06:22.241785+0000","last_undegraded":"2024-04-25T10:06:22.241785+0000","last_fullsized":"2024-04-25T10:06:22.241785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:40.277052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.206746+0000","last_change":"2024-04-25T10:06:22.263026+0000","last_active":"2024-04-25T10:06:23.206746+0000","last_peered":"2024-04-25T10:06:23.206746+0000","last_clean":"2024-04-25T10:06:23.206746+0000","last_became_active":"2024-04-25T10:06:22.262155+0000","last_became_peered":"2024-04-25T10:06:22.262155+0000","last_unstale":"2024-04-25T10:06:23.206746+0000","last_undegraded":"2024-04-25T10:06:23.206746+0000","last_fullsized":"2024-04-25T10:06:23.206746+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:12.354255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.205386+0000","last_change":"2024-04-25T10:06:22.254129+0000","last_active":"2024-04-25T10:06:23.205386+0000","last_peered":"2024-04-25T10:06:23.205386+0000","last_clean":"2024-04-25T10:06:23.205386+0000","last_became_active":"2024-04-25T10:06:22.253670+0000","last_became_peered":"2024-04-25T10:06:22.253670+0000","last_unstale":"2024-04-25T10:06:23.205386+0000","last_undegraded":"2024-04-25T10:06:23.205386+0000","last_fullsized":"2024-04-25T10:06:23.205386+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:19.349171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208856+0000","last_change":"2024-04-25T10:06:22.237973+0000","last_active":"2024-04-25T10:06:23.208856+0000","last_peered":"2024-04-25T10:06:23.208856+0000","last_clean":"2024-04-25T10:06:23.208856+0000","last_became_active":"2024-04-25T10:06:22.230343+0000","last_became_peered":"2024-04-25T10:06:22.230343+0000","last_unstale":"2024-04-25T10:06:23.208856+0000","last_undegraded":"2024-04-25T10:06:23.208856+0000","last_fullsized":"2024-04-25T10:06:23.208856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:29.570843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.226760+0000","last_change":"2024-04-25T10:06:22.222333+0000","last_active":"2024-04-25T10:06:22.226760+0000","last_peered":"2024-04-25T10:06:22.226760+0000","last_clean":"2024-04-25T10:06:22.226760+0000","last_became_active":"2024-04-25T10:06:22.222021+0000","last_became_peered":"2024-04-25T10:06:22.222021+0000","last_unstale":"2024-04-25T10:06:22.226760+0000","last_undegraded":"2024-04-25T10:06:22.226760+0000","last_fullsized":"2024-04-25T10:06:22.226760+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:11.865103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.206522+0000","last_change":"2024-04-25T10:06:22.251480+0000","last_active":"2024-04-25T10:06:23.206522+0000","last_peered":"2024-04-25T10:06:23.206522+0000","last_clean":"2024-04-25T10:06:23.206522+0000","last_became_active":"2024-04-25T10:06:22.250723+0000","last_became_peered":"2024-04-25T10:06:22.250723+0000","last_unstale":"2024-04-25T10:06:23.206522+0000","last_undegraded":"2024-04-25T10:06:23.206522+0000","last_fullsized":"2024-04-25T10:06:23.206522+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:49.891844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.237360+0000","last_change":"2024-04-25T10:06:22.223614+0000","last_active":"2024-04-25T10:06:22.237360+0000","last_peered":"2024-04-25T10:06:22.237360+0000","last_clean":"2024-04-25T10:06:22.237360+0000","last_became_active":"2024-04-25T10:06:22.220482+0000","last_became_peered":"2024-04-25T10:06:22.220482+0000","last_unstale":"2024-04-25T10:06:22.237360+0000","last_undegraded":"2024-04-25T10:06:22.237360+0000","last_fullsized":"2024-04-25T10:06:22.237360+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:35.778038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.221762+0000","last_change":"2024-04-25T10:06:22.220985+0000","last_active":"2024-04-25T10:06:22.221762+0000","last_peered":"2024-04-25T10:06:22.221762+0000","last_clean":"2024-04-25T10:06:22.221762+0000","last_became_active":"2024-04-25T10:06:22.220662+0000","last_became_peered":"2024-04-25T10:06:22.220662+0000","last_unstale":"2024-04-25T10:06:22.221762+0000","last_undegraded":"2024-04-25T10:06:22.221762+0000","last_fullsized":"2024-04-25T10:06:22.221762+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:02.182289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.216280+0000","last_change":"2024-04-25T10:06:22.212668+0000","last_active":"2024-04-25T10:06:22.216280+0000","last_peered":"2024-04-25T10:06:22.216280+0000","last_clean":"2024-04-25T10:06:22.216280+0000","last_became_active":"2024-04-25T10:06:22.212382+0000","last_became_peered":"2024-04-25T10:06:22.212382+0000","last_unstale":"2024-04-25T10:06:22.216280+0000","last_undegraded":"2024-04-25T10:06:22.216280+0000","last_fullsized":"2024-04-25T10:06:22.216280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:30.807883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.244094+0000","last_change":"2024-04-25T10:06:22.236243+0000","last_active":"2024-04-25T10:06:22.244094+0000","last_peered":"2024-04-25T10:06:22.244094+0000","last_clean":"2024-04-25T10:06:22.244094+0000","last_became_active":"2024-04-25T10:06:22.235488+0000","last_became_peered":"2024-04-25T10:06:22.235488+0000","last_unstale":"2024-04-25T10:06:22.244094+0000","last_undegraded":"2024-04-25T10:06:22.244094+0000","last_fullsized":"2024-04-25T10:06:22.244094+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:33.404890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.206240+0000","last_change":"2024-04-25T10:06:22.256282+0000","last_active":"2024-04-25T10:06:23.206240+0000","last_peered":"2024-04-25T10:06:23.206240+0000","last_clean":"2024-04-25T10:06:23.206240+0000","last_became_active":"2024-04-25T10:06:22.253832+0000","last_became_peered":"2024-04-25T10:06:22.253832+0000","last_unstale":"2024-04-25T10:06:23.206240+0000","last_undegraded":"2024-04-25T10:06:23.206240+0000","last_fullsized":"2024-04-25T10:06:23.206240+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:20.744775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.207417+0000","last_change":"2024-04-25T10:06:22.263384+0000","last_active":"2024-04-25T10:06:23.207417+0000","last_peered":"2024-04-25T10:06:23.207417+0000","last_clean":"2024-04-25T10:06:23.207417+0000","last_became_active":"2024-04-25T10:06:22.261831+0000","last_became_peered":"2024-04-25T10:06:22.261831+0000","last_unstale":"2024-04-25T10:06:23.207417+0000","last_undegraded":"2024-04-25T10:06:23.207417+0000","last_fullsized":"2024-04-25T10:06:23.207417+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:39.568884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.225189+0000","last_change":"2024-04-25T10:06:22.223700+0000","last_active":"2024-04-25T10:06:22.225189+0000","last_peered":"2024-04-25T10:06:22.225189+0000","last_clean":"2024-04-25T10:06:22.225189+0000","last_became_active":"2024-04-25T10:06:22.223222+0000","last_became_peered":"2024-04-25T10:06:22.223222+0000","last_unstale":"2024-04-25T10:06:22.225189+0000","last_undegraded":"2024-04-25T10:06:22.225189+0000","last_fullsized":"2024-04-25T10:06:22.225189+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.821436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.224561+0000","last_change":"2024-04-25T10:06:22.223672+0000","last_active":"2024-04-25T10:06:22.224561+0000","last_peered":"2024-04-25T10:06:22.224561+0000","last_clean":"2024-04-25T10:06:22.224561+0000","last_became_active":"2024-04-25T10:06:22.223238+0000","last_became_peered":"2024-04-25T10:06:22.223238+0000","last_unstale":"2024-04-25T10:06:22.224561+0000","last_undegraded":"2024-04-25T10:06:22.224561+0000","last_fullsized":"2024-04-25T10:06:22.224561+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:08.711769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.235383+0000","last_change":"2024-04-25T10:06:22.234861+0000","last_active":"2024-04-25T10:06:22.235383+0000","last_peered":"2024-04-25T10:06:22.235383+0000","last_clean":"2024-04-25T10:06:22.235383+0000","last_became_active":"2024-04-25T10:06:22.234541+0000","last_became_peered":"2024-04-25T10:06:22.234541+0000","last_unstale":"2024-04-25T10:06:22.235383+0000","last_undegraded":"2024-04-25T10:06:22.235383+0000","last_fullsized":"2024-04-25T10:06:22.235383+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:59.048260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.207877+0000","last_change":"2024-04-25T10:06:22.240249+0000","last_active":"2024-04-25T10:06:23.207877+0000","last_peered":"2024-04-25T10:06:23.207877+0000","last_clean":"2024-04-25T10:06:23.207877+0000","last_became_active":"2024-04-25T10:06:22.238840+0000","last_became_peered":"2024-04-25T10:06:22.238840+0000","last_unstale":"2024-04-25T10:06:23.207877+0000","last_undegraded":"2024-04-25T10:06:23.207877+0000","last_fullsized":"2024-04-25T10:06:23.207877+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:16.392430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.207784+0000","last_change":"2024-04-25T10:06:22.237603+0000","last_active":"2024-04-25T10:06:23.207784+0000","last_peered":"2024-04-25T10:06:23.207784+0000","last_clean":"2024-04-25T10:06:23.207784+0000","last_became_active":"2024-04-25T10:06:22.236765+0000","last_became_peered":"2024-04-25T10:06:22.236765+0000","last_unstale":"2024-04-25T10:06:23.207784+0000","last_undegraded":"2024-04-25T10:06:23.207784+0000","last_fullsized":"2024-04-25T10:06:23.207784+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:22.210589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.250560+0000","last_change":"2024-04-25T10:06:22.249840+0000","last_active":"2024-04-25T10:06:22.250560+0000","last_peered":"2024-04-25T10:06:22.250560+0000","last_clean":"2024-04-25T10:06:22.250560+0000","last_became_active":"2024-04-25T10:06:22.249489+0000","last_became_peered":"2024-04-25T10:06:22.249489+0000","last_unstale":"2024-04-25T10:06:22.250560+0000","last_undegraded":"2024-04-25T10:06:22.250560+0000","last_fullsized":"2024-04-25T10:06:22.250560+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:33.660776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.237232+0000","last_change":"2024-04-25T10:06:22.223941+0000","last_active":"2024-04-25T10:06:22.237232+0000","last_peered":"2024-04-25T10:06:22.237232+0000","last_clean":"2024-04-25T10:06:22.237232+0000","last_became_active":"2024-04-25T10:06:22.221241+0000","last_became_peered":"2024-04-25T10:06:22.221241+0000","last_unstale":"2024-04-25T10:06:22.237232+0000","last_undegraded":"2024-04-25T10:06:22.237232+0000","last_fullsized":"2024-04-25T10:06:22.237232+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:14.614718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.250393+0000","last_change":"2024-04-25T10:06:22.249671+0000","last_active":"2024-04-25T10:06:22.250393+0000","last_peered":"2024-04-25T10:06:22.250393+0000","last_clean":"2024-04-25T10:06:22.250393+0000","last_became_active":"2024-04-25T10:06:22.249384+0000","last_became_peered":"2024-04-25T10:06:22.249384+0000","last_unstale":"2024-04-25T10:06:22.250393+0000","last_undegraded":"2024-04-25T10:06:22.250393+0000","last_fullsized":"2024-04-25T10:06:22.250393+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:07.079403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.253503+0000","last_change":"2024-04-25T10:06:22.252972+0000","last_active":"2024-04-25T10:06:22.253503+0000","last_peered":"2024-04-25T10:06:22.253503+0000","last_clean":"2024-04-25T10:06:22.253503+0000","last_became_active":"2024-04-25T10:06:22.252728+0000","last_became_peered":"2024-04-25T10:06:22.252728+0000","last_unstale":"2024-04-25T10:06:22.253503+0000","last_undegraded":"2024-04-25T10:06:22.253503+0000","last_fullsized":"2024-04-25T10:06:22.253503+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:31.698949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.227227+0000","last_change":"2024-04-25T10:06:22.224194+0000","last_active":"2024-04-25T10:06:22.227227+0000","last_peered":"2024-04-25T10:06:22.227227+0000","last_clean":"2024-04-25T10:06:22.227227+0000","last_became_active":"2024-04-25T10:06:22.223686+0000","last_became_peered":"2024-04-25T10:06:22.223686+0000","last_unstale":"2024-04-25T10:06:22.227227+0000","last_undegraded":"2024-04-25T10:06:22.227227+0000","last_fullsized":"2024-04-25T10:06:22.227227+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:42.882629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.237312+0000","last_change":"2024-04-25T10:06:22.222953+0000","last_active":"2024-04-25T10:06:22.237312+0000","last_peered":"2024-04-25T10:06:22.237312+0000","last_clean":"2024-04-25T10:06:22.237312+0000","last_became_active":"2024-04-25T10:06:22.219544+0000","last_became_peered":"2024-04-25T10:06:22.219544+0000","last_unstale":"2024-04-25T10:06:22.237312+0000","last_undegraded":"2024-04-25T10:06:22.237312+0000","last_fullsized":"2024-04-25T10:06:22.237312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:30.807883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208214+0000","last_change":"2024-04-25T10:06:22.239514+0000","last_active":"2024-04-25T10:06:23.208214+0000","last_peered":"2024-04-25T10:06:23.208214+0000","last_clean":"2024-04-25T10:06:23.208214+0000","last_became_active":"2024-04-25T10:06:22.230693+0000","last_became_peered":"2024-04-25T10:06:22.230693+0000","last_unstale":"2024-04-25T10:06:23.208214+0000","last_undegraded":"2024-04-25T10:06:23.208214+0000","last_fullsized":"2024-04-25T10:06:23.208214+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:54.514623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.205988+0000","last_change":"2024-04-25T10:06:22.262417+0000","last_active":"2024-04-25T10:06:23.205988+0000","last_peered":"2024-04-25T10:06:23.205988+0000","last_clean":"2024-04-25T10:06:23.205988+0000","last_became_active":"2024-04-25T10:06:22.261977+0000","last_became_peered":"2024-04-25T10:06:22.261977+0000","last_unstale":"2024-04-25T10:06:23.205988+0000","last_undegraded":"2024-04-25T10:06:23.205988+0000","last_fullsized":"2024-04-25T10:06:23.205988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:36.452389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.261128+0000","last_change":"2024-04-25T10:06:22.233558+0000","last_active":"2024-04-25T10:06:22.261128+0000","last_peered":"2024-04-25T10:06:22.261128+0000","last_clean":"2024-04-25T10:06:22.261128+0000","last_became_active":"2024-04-25T10:06:22.232812+0000","last_became_peered":"2024-04-25T10:06:22.232812+0000","last_unstale":"2024-04-25T10:06:22.261128+0000","last_undegraded":"2024-04-25T10:06:22.261128+0000","last_fullsized":"2024-04-25T10:06:22.261128+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:49.656616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.205525+0000","last_change":"2024-04-25T10:06:22.258550+0000","last_active":"2024-04-25T10:06:23.205525+0000","last_peered":"2024-04-25T10:06:23.205525+0000","last_clean":"2024-04-25T10:06:23.205525+0000","last_became_active":"2024-04-25T10:06:22.257374+0000","last_became_peered":"2024-04-25T10:06:22.257374+0000","last_unstale":"2024-04-25T10:06:23.205525+0000","last_undegraded":"2024-04-25T10:06:23.205525+0000","last_fullsized":"2024-04-25T10:06:23.205525+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:06.870969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.252956+0000","last_change":"2024-04-25T10:06:22.252406+0000","last_active":"2024-04-25T10:06:22.252956+0000","last_peered":"2024-04-25T10:06:22.252956+0000","last_clean":"2024-04-25T10:06:22.252956+0000","last_became_active":"2024-04-25T10:06:22.252078+0000","last_became_peered":"2024-04-25T10:06:22.252078+0000","last_unstale":"2024-04-25T10:06:22.252956+0000","last_undegraded":"2024-04-25T10:06:22.252956+0000","last_fullsized":"2024-04-25T10:06:22.252956+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:57.040192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.259875+0000","last_change":"2024-04-25T10:06:21.196297+0000","last_active":"2024-04-25T10:06:22.259875+0000","last_peered":"2024-04-25T10:06:22.259875+0000","last_clean":"2024-04-25T10:06:22.259875+0000","last_became_active":"2024-04-25T10:06:21.195531+0000","last_became_peered":"2024-04-25T10:06:21.195531+0000","last_unstale":"2024-04-25T10:06:22.259875+0000","last_undegraded":"2024-04-25T10:06:22.259875+0000","last_fullsized":"2024-04-25T10:06:22.259875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:18.689528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.261442+0000","last_change":"2024-04-25T10:06:22.255434+0000","last_active":"2024-04-25T10:06:22.261442+0000","last_peered":"2024-04-25T10:06:22.261442+0000","last_clean":"2024-04-25T10:06:22.261442+0000","last_became_active":"2024-04-25T10:06:22.252918+0000","last_became_peered":"2024-04-25T10:06:22.252918+0000","last_unstale":"2024-04-25T10:06:22.261442+0000","last_undegraded":"2024-04-25T10:06:22.261442+0000","last_fullsized":"2024-04-25T10:06:22.261442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:46.770914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.245323+0000","last_change":"2024-04-25T10:06:21.180812+0000","last_active":"2024-04-25T10:06:22.245323+0000","last_peered":"2024-04-25T10:06:22.245323+0000","last_clean":"2024-04-25T10:06:22.245323+0000","last_became_active":"2024-04-25T10:06:21.180058+0000","last_became_peered":"2024-04-25T10:06:21.180058+0000","last_unstale":"2024-04-25T10:06:22.245323+0000","last_undegraded":"2024-04-25T10:06:22.245323+0000","last_fullsized":"2024-04-25T10:06:22.245323+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:48.653271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208680+0000","last_change":"2024-04-25T10:06:22.219265+0000","last_active":"2024-04-25T10:06:23.208680+0000","last_peered":"2024-04-25T10:06:23.208680+0000","last_clean":"2024-04-25T10:06:23.208680+0000","last_became_active":"2024-04-25T10:06:22.218449+0000","last_became_peered":"2024-04-25T10:06:22.218449+0000","last_unstale":"2024-04-25T10:06:23.208680+0000","last_undegraded":"2024-04-25T10:06:23.208680+0000","last_fullsized":"2024-04-25T10:06:23.208680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:06.870969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.226145+0000","last_change":"2024-04-25T10:06:21.168434+0000","last_active":"2024-04-25T10:06:22.226145+0000","last_peered":"2024-04-25T10:06:22.226145+0000","last_clean":"2024-04-25T10:06:22.226145+0000","last_became_active":"2024-04-25T10:06:21.167972+0000","last_became_peered":"2024-04-25T10:06:21.167972+0000","last_unstale":"2024-04-25T10:06:22.226145+0000","last_undegraded":"2024-04-25T10:06:22.226145+0000","last_fullsized":"2024-04-25T10:06:22.226145+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:52.499219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.227202+0000","last_change":"2024-04-25T10:06:22.223373+0000","last_active":"2024-04-25T10:06:22.227202+0000","last_peered":"2024-04-25T10:06:22.227202+0000","last_clean":"2024-04-25T10:06:22.227202+0000","last_became_active":"2024-04-25T10:06:22.223093+0000","last_became_peered":"2024-04-25T10:06:22.223093+0000","last_unstale":"2024-04-25T10:06:22.227202+0000","last_undegraded":"2024-04-25T10:06:22.227202+0000","last_fullsized":"2024-04-25T10:06:22.227202+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:08.399858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.220512+0000","last_change":"2024-04-25T10:06:21.174028+0000","last_active":"2024-04-25T10:06:22.220512+0000","last_peered":"2024-04-25T10:06:22.220512+0000","last_clean":"2024-04-25T10:06:22.220512+0000","last_became_active":"2024-04-25T10:06:21.173714+0000","last_became_peered":"2024-04-25T10:06:21.173714+0000","last_unstale":"2024-04-25T10:06:22.220512+0000","last_undegraded":"2024-04-25T10:06:22.220512+0000","last_fullsized":"2024-04-25T10:06:22.220512+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:09.215037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.206362+0000","last_change":"2024-04-25T10:06:22.258149+0000","last_active":"2024-04-25T10:06:23.206362+0000","last_peered":"2024-04-25T10:06:23.206362+0000","last_clean":"2024-04-25T10:06:23.206362+0000","last_became_active":"2024-04-25T10:06:22.257322+0000","last_became_peered":"2024-04-25T10:06:22.257322+0000","last_unstale":"2024-04-25T10:06:23.206362+0000","last_undegraded":"2024-04-25T10:06:23.206362+0000","last_fullsized":"2024-04-25T10:06:23.206362+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:54.457042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208190+0000","last_change":"2024-04-25T10:06:21.193111+0000","last_active":"2024-04-25T10:06:23.208190+0000","last_peered":"2024-04-25T10:06:23.208190+0000","last_clean":"2024-04-25T10:06:23.208190+0000","last_became_active":"2024-04-25T10:06:21.192445+0000","last_became_peered":"2024-04-25T10:06:21.192445+0000","last_unstale":"2024-04-25T10:06:23.208190+0000","last_undegraded":"2024-04-25T10:06:23.208190+0000","last_fullsized":"2024-04-25T10:06:23.208190+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:50.789277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.236940+0000","last_change":"2024-04-25T10:06:22.222486+0000","last_active":"2024-04-25T10:06:22.236940+0000","last_peered":"2024-04-25T10:06:22.236940+0000","last_clean":"2024-04-25T10:06:22.236940+0000","last_became_active":"2024-04-25T10:06:22.221406+0000","last_became_peered":"2024-04-25T10:06:22.221406+0000","last_unstale":"2024-04-25T10:06:22.236940+0000","last_undegraded":"2024-04-25T10:06:22.236940+0000","last_fullsized":"2024-04-25T10:06:22.236940+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:33.576802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.207693+0000","last_change":"2024-04-25T10:06:21.190130+0000","last_active":"2024-04-25T10:06:23.207693+0000","last_peered":"2024-04-25T10:06:23.207693+0000","last_clean":"2024-04-25T10:06:23.207693+0000","last_became_active":"2024-04-25T10:06:21.189305+0000","last_became_peered":"2024-04-25T10:06:21.189305+0000","last_unstale":"2024-04-25T10:06:23.207693+0000","last_undegraded":"2024-04-25T10:06:23.207693+0000","last_fullsized":"2024-04-25T10:06:23.207693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:05.714570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.205981+0000","last_change":"2024-04-25T10:06:22.238585+0000","last_active":"2024-04-25T10:06:23.205981+0000","last_peered":"2024-04-25T10:06:23.205981+0000","last_clean":"2024-04-25T10:06:23.205981+0000","last_became_active":"2024-04-25T10:06:22.237914+0000","last_became_peered":"2024-04-25T10:06:22.237914+0000","last_unstale":"2024-04-25T10:06:23.205981+0000","last_undegraded":"2024-04-25T10:06:23.205981+0000","last_fullsized":"2024-04-25T10:06:23.205981+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:07.079403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.205270+0000","last_change":"2024-04-25T10:06:21.165169+0000","last_active":"2024-04-25T10:06:22.205270+0000","last_peered":"2024-04-25T10:06:22.205270+0000","last_clean":"2024-04-25T10:06:22.205270+0000","last_became_active":"2024-04-25T10:06:21.164840+0000","last_became_peered":"2024-04-25T10:06:21.164840+0000","last_unstale":"2024-04-25T10:06:22.205270+0000","last_undegraded":"2024-04-25T10:06:22.205270+0000","last_fullsized":"2024-04-25T10:06:22.205270+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:56.036847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.221630+0000","last_change":"2024-04-25T10:06:22.220700+0000","last_active":"2024-04-25T10:06:22.221630+0000","last_peered":"2024-04-25T10:06:22.221630+0000","last_clean":"2024-04-25T10:06:22.221630+0000","last_became_active":"2024-04-25T10:06:22.220255+0000","last_became_peered":"2024-04-25T10:06:22.220255+0000","last_unstale":"2024-04-25T10:06:22.221630+0000","last_undegraded":"2024-04-25T10:06:22.221630+0000","last_fullsized":"2024-04-25T10:06:22.221630+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:08.377079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.206234+0000","last_change":"2024-04-25T10:06:21.198140+0000","last_active":"2024-04-25T10:06:23.206234+0000","last_peered":"2024-04-25T10:06:23.206234+0000","last_clean":"2024-04-25T10:06:23.206234+0000","last_became_active":"2024-04-25T10:06:21.197379+0000","last_became_peered":"2024-04-25T10:06:21.197379+0000","last_unstale":"2024-04-25T10:06:23.206234+0000","last_undegraded":"2024-04-25T10:06:23.206234+0000","last_fullsized":"2024-04-25T10:06:23.206234+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:55.726391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.244072+0000","last_change":"2024-04-25T10:06:22.236889+0000","last_active":"2024-04-25T10:06:22.244072+0000","last_peered":"2024-04-25T10:06:22.244072+0000","last_clean":"2024-04-25T10:06:22.244072+0000","last_became_active":"2024-04-25T10:06:22.235866+0000","last_became_peered":"2024-04-25T10:06:22.235866+0000","last_unstale":"2024-04-25T10:06:22.244072+0000","last_undegraded":"2024-04-25T10:06:22.244072+0000","last_fullsized":"2024-04-25T10:06:22.244072+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:02.550775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.205924+0000","last_change":"2024-04-25T10:06:21.180968+0000","last_active":"2024-04-25T10:06:22.205924+0000","last_peered":"2024-04-25T10:06:22.205924+0000","last_clean":"2024-04-25T10:06:22.205924+0000","last_became_active":"2024-04-25T10:06:21.180063+0000","last_became_peered":"2024-04-25T10:06:21.180063+0000","last_unstale":"2024-04-25T10:06:22.205924+0000","last_undegraded":"2024-04-25T10:06:22.205924+0000","last_fullsized":"2024-04-25T10:06:22.205924+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:08.126165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.227250+0000","last_change":"2024-04-25T10:06:22.226431+0000","last_active":"2024-04-25T10:06:22.227250+0000","last_peered":"2024-04-25T10:06:22.227250+0000","last_clean":"2024-04-25T10:06:22.227250+0000","last_became_active":"2024-04-25T10:06:22.226148+0000","last_became_peered":"2024-04-25T10:06:22.226148+0000","last_unstale":"2024-04-25T10:06:22.227250+0000","last_undegraded":"2024-04-25T10:06:22.227250+0000","last_fullsized":"2024-04-25T10:06:22.227250+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:02.550775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.215788+0000","last_change":"2024-04-25T10:06:21.172445+0000","last_active":"2024-04-25T10:06:22.215788+0000","last_peered":"2024-04-25T10:06:22.215788+0000","last_clean":"2024-04-25T10:06:22.215788+0000","last_became_active":"2024-04-25T10:06:21.171542+0000","last_became_peered":"2024-04-25T10:06:21.171542+0000","last_unstale":"2024-04-25T10:06:22.215788+0000","last_undegraded":"2024-04-25T10:06:22.215788+0000","last_fullsized":"2024-04-25T10:06:22.215788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:15.633989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.251006+0000","last_change":"2024-04-25T10:06:22.249640+0000","last_active":"2024-04-25T10:06:22.251006+0000","last_peered":"2024-04-25T10:06:22.251006+0000","last_clean":"2024-04-25T10:06:22.251006+0000","last_became_active":"2024-04-25T10:06:22.249349+0000","last_became_peered":"2024-04-25T10:06:22.249349+0000","last_unstale":"2024-04-25T10:06:22.251006+0000","last_undegraded":"2024-04-25T10:06:22.251006+0000","last_fullsized":"2024-04-25T10:06:22.251006+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:23.241367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.255499+0000","last_change":"2024-04-25T10:06:21.184541+0000","last_active":"2024-04-25T10:06:22.255499+0000","last_peered":"2024-04-25T10:06:22.255499+0000","last_clean":"2024-04-25T10:06:22.255499+0000","last_became_active":"2024-04-25T10:06:21.183778+0000","last_became_peered":"2024-04-25T10:06:21.183778+0000","last_unstale":"2024-04-25T10:06:22.255499+0000","last_undegraded":"2024-04-25T10:06:22.255499+0000","last_fullsized":"2024-04-25T10:06:22.255499+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:52.499219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.242999+0000","last_change":"2024-04-25T10:06:22.229582+0000","last_active":"2024-04-25T10:06:22.242999+0000","last_peered":"2024-04-25T10:06:22.242999+0000","last_clean":"2024-04-25T10:06:22.242999+0000","last_became_active":"2024-04-25T10:06:22.228605+0000","last_became_peered":"2024-04-25T10:06:22.228605+0000","last_unstale":"2024-04-25T10:06:22.242999+0000","last_undegraded":"2024-04-25T10:06:22.242999+0000","last_fullsized":"2024-04-25T10:06:22.242999+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.821436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.257118+0000","last_change":"2024-04-25T10:06:21.168138+0000","last_active":"2024-04-25T10:06:22.257118+0000","last_peered":"2024-04-25T10:06:22.257118+0000","last_clean":"2024-04-25T10:06:22.257118+0000","last_became_active":"2024-04-25T10:06:21.167848+0000","last_became_peered":"2024-04-25T10:06:21.167848+0000","last_unstale":"2024-04-25T10:06:22.257118+0000","last_undegraded":"2024-04-25T10:06:22.257118+0000","last_fullsized":"2024-04-25T10:06:22.257118+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:32.573457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.220333+0000","last_change":"2024-04-25T10:06:22.219395+0000","last_active":"2024-04-25T10:06:22.220333+0000","last_peered":"2024-04-25T10:06:22.220333+0000","last_clean":"2024-04-25T10:06:22.220333+0000","last_became_active":"2024-04-25T10:06:22.219080+0000","last_became_peered":"2024-04-25T10:06:22.219080+0000","last_unstale":"2024-04-25T10:06:22.220333+0000","last_undegraded":"2024-04-25T10:06:22.220333+0000","last_fullsized":"2024-04-25T10:06:22.220333+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:58.977828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208536+0000","last_change":"2024-04-25T10:06:21.199846+0000","last_active":"2024-04-25T10:06:23.208536+0000","last_peered":"2024-04-25T10:06:23.208536+0000","last_clean":"2024-04-25T10:06:23.208536+0000","last_became_active":"2024-04-25T10:06:21.199009+0000","last_became_peered":"2024-04-25T10:06:21.199009+0000","last_unstale":"2024-04-25T10:06:23.208536+0000","last_undegraded":"2024-04-25T10:06:23.208536+0000","last_fullsized":"2024-04-25T10:06:23.208536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:06.034487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.216514+0000","last_change":"2024-04-25T10:06:22.215346+0000","last_active":"2024-04-25T10:06:22.216514+0000","last_peered":"2024-04-25T10:06:22.216514+0000","last_clean":"2024-04-25T10:06:22.216514+0000","last_became_active":"2024-04-25T10:06:22.214423+0000","last_became_peered":"2024-04-25T10:06:22.214423+0000","last_unstale":"2024-04-25T10:06:22.216514+0000","last_undegraded":"2024-04-25T10:06:22.216514+0000","last_fullsized":"2024-04-25T10:06:22.216514+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:35.778038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.201320+0000","last_change":"2024-04-25T10:06:21.184643+0000","last_active":"2024-04-25T10:06:22.201320+0000","last_peered":"2024-04-25T10:06:22.201320+0000","last_clean":"2024-04-25T10:06:22.201320+0000","last_became_active":"2024-04-25T10:06:21.184119+0000","last_became_peered":"2024-04-25T10:06:21.184119+0000","last_unstale":"2024-04-25T10:06:22.201320+0000","last_undegraded":"2024-04-25T10:06:22.201320+0000","last_fullsized":"2024-04-25T10:06:22.201320+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:50.789277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.205856+0000","last_change":"2024-04-25T10:06:22.258959+0000","last_active":"2024-04-25T10:06:23.205856+0000","last_peered":"2024-04-25T10:06:23.205856+0000","last_clean":"2024-04-25T10:06:23.205856+0000","last_became_active":"2024-04-25T10:06:22.257939+0000","last_became_peered":"2024-04-25T10:06:22.257939+0000","last_unstale":"2024-04-25T10:06:23.205856+0000","last_undegraded":"2024-04-25T10:06:23.205856+0000","last_fullsized":"2024-04-25T10:06:23.205856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:29.570843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.205368+0000","last_change":"2024-04-25T10:06:21.180558+0000","last_active":"2024-04-25T10:06:22.205368+0000","last_peered":"2024-04-25T10:06:22.205368+0000","last_clean":"2024-04-25T10:06:22.205368+0000","last_became_active":"2024-04-25T10:06:21.180238+0000","last_became_peered":"2024-04-25T10:06:21.180238+0000","last_unstale":"2024-04-25T10:06:22.205368+0000","last_undegraded":"2024-04-25T10:06:22.205368+0000","last_fullsized":"2024-04-25T10:06:22.205368+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:26.115979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.261505+0000","last_change":"2024-04-25T10:06:22.245367+0000","last_active":"2024-04-25T10:06:22.261505+0000","last_peered":"2024-04-25T10:06:22.261505+0000","last_clean":"2024-04-25T10:06:22.261505+0000","last_became_active":"2024-04-25T10:06:22.244679+0000","last_became_peered":"2024-04-25T10:06:22.244679+0000","last_unstale":"2024-04-25T10:06:22.261505+0000","last_undegraded":"2024-04-25T10:06:22.261505+0000","last_fullsized":"2024-04-25T10:06:22.261505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:16.637334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.227843+0000","last_change":"2024-04-25T10:06:21.168016+0000","last_active":"2024-04-25T10:06:22.227843+0000","last_peered":"2024-04-25T10:06:22.227843+0000","last_clean":"2024-04-25T10:06:22.227843+0000","last_became_active":"2024-04-25T10:06:21.167676+0000","last_became_peered":"2024-04-25T10:06:21.167676+0000","last_unstale":"2024-04-25T10:06:22.227843+0000","last_undegraded":"2024-04-25T10:06:22.227843+0000","last_fullsized":"2024-04-25T10:06:22.227843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:03.425191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208559+0000","last_change":"2024-04-25T10:06:22.244819+0000","last_active":"2024-04-25T10:06:23.208559+0000","last_peered":"2024-04-25T10:06:23.208559+0000","last_clean":"2024-04-25T10:06:23.208559+0000","last_became_active":"2024-04-25T10:06:22.244100+0000","last_became_peered":"2024-04-25T10:06:22.244100+0000","last_unstale":"2024-04-25T10:06:23.208559+0000","last_undegraded":"2024-04-25T10:06:23.208559+0000","last_fullsized":"2024-04-25T10:06:23.208559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:19.349171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.225833+0000","last_change":"2024-04-25T10:06:21.176489+0000","last_active":"2024-04-25T10:06:22.225833+0000","last_peered":"2024-04-25T10:06:22.225833+0000","last_clean":"2024-04-25T10:06:22.225833+0000","last_became_active":"2024-04-25T10:06:21.174781+0000","last_became_peered":"2024-04-25T10:06:21.174781+0000","last_unstale":"2024-04-25T10:06:22.225833+0000","last_undegraded":"2024-04-25T10:06:22.225833+0000","last_fullsized":"2024-04-25T10:06:22.225833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:24.318301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.257967+0000","last_change":"2024-04-25T10:06:22.254281+0000","last_active":"2024-04-25T10:06:22.257967+0000","last_peered":"2024-04-25T10:06:22.257967+0000","last_clean":"2024-04-25T10:06:22.257967+0000","last_became_active":"2024-04-25T10:06:22.243902+0000","last_became_peered":"2024-04-25T10:06:22.243902+0000","last_unstale":"2024-04-25T10:06:22.257967+0000","last_undegraded":"2024-04-25T10:06:22.257967+0000","last_fullsized":"2024-04-25T10:06:22.257967+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:23.241367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208774+0000","last_change":"2024-04-25T10:06:21.193080+0000","last_active":"2024-04-25T10:06:23.208774+0000","last_peered":"2024-04-25T10:06:23.208774+0000","last_clean":"2024-04-25T10:06:23.208774+0000","last_became_active":"2024-04-25T10:06:21.192259+0000","last_became_peered":"2024-04-25T10:06:21.192259+0000","last_unstale":"2024-04-25T10:06:23.208774+0000","last_undegraded":"2024-04-25T10:06:23.208774+0000","last_fullsized":"2024-04-25T10:06:23.208774+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:38.038262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.230183+0000","last_change":"2024-04-25T10:06:22.229688+0000","last_active":"2024-04-25T10:06:22.230183+0000","last_peered":"2024-04-25T10:06:22.230183+0000","last_clean":"2024-04-25T10:06:22.230183+0000","last_became_active":"2024-04-25T10:06:22.229365+0000","last_became_peered":"2024-04-25T10:06:22.229365+0000","last_unstale":"2024-04-25T10:06:22.230183+0000","last_undegraded":"2024-04-25T10:06:22.230183+0000","last_fullsized":"2024-04-25T10:06:22.230183+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:49.701955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208542+0000","last_change":"2024-04-25T10:06:21.191734+0000","last_active":"2024-04-25T10:06:23.208542+0000","last_peered":"2024-04-25T10:06:23.208542+0000","last_clean":"2024-04-25T10:06:23.208542+0000","last_became_active":"2024-04-25T10:06:21.190643+0000","last_became_peered":"2024-04-25T10:06:21.190643+0000","last_unstale":"2024-04-25T10:06:23.208542+0000","last_undegraded":"2024-04-25T10:06:23.208542+0000","last_fullsized":"2024-04-25T10:06:23.208542+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:09.818048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.252544+0000","last_change":"2024-04-25T10:06:22.251943+0000","last_active":"2024-04-25T10:06:22.252544+0000","last_peered":"2024-04-25T10:06:22.252544+0000","last_clean":"2024-04-25T10:06:22.252544+0000","last_became_active":"2024-04-25T10:06:22.251649+0000","last_became_peered":"2024-04-25T10:06:22.251649+0000","last_unstale":"2024-04-25T10:06:22.252544+0000","last_undegraded":"2024-04-25T10:06:22.252544+0000","last_fullsized":"2024-04-25T10:06:22.252544+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:14.983380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.208147+0000","last_change":"2024-04-25T10:06:21.190153+0000","last_active":"2024-04-25T10:06:23.208147+0000","last_peered":"2024-04-25T10:06:23.208147+0000","last_clean":"2024-04-25T10:06:23.208147+0000","last_became_active":"2024-04-25T10:06:21.189403+0000","last_became_peered":"2024-04-25T10:06:21.189403+0000","last_unstale":"2024-04-25T10:06:23.208147+0000","last_undegraded":"2024-04-25T10:06:23.208147+0000","last_fullsized":"2024-04-25T10:06:23.208147+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:42.024042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.243947+0000","last_change":"2024-04-25T10:06:22.243492+0000","last_active":"2024-04-25T10:06:22.243947+0000","last_peered":"2024-04-25T10:06:22.243947+0000","last_clean":"2024-04-25T10:06:22.243947+0000","last_became_active":"2024-04-25T10:06:22.243175+0000","last_became_peered":"2024-04-25T10:06:22.243175+0000","last_unstale":"2024-04-25T10:06:22.243947+0000","last_undegraded":"2024-04-25T10:06:22.243947+0000","last_fullsized":"2024-04-25T10:06:22.243947+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:39.934546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.227960+0000","last_change":"2024-04-25T10:06:21.177337+0000","last_active":"2024-04-25T10:06:22.227960+0000","last_peered":"2024-04-25T10:06:22.227960+0000","last_clean":"2024-04-25T10:06:22.227960+0000","last_became_active":"2024-04-25T10:06:21.176990+0000","last_became_peered":"2024-04-25T10:06:21.176990+0000","last_unstale":"2024-04-25T10:06:22.227960+0000","last_undegraded":"2024-04-25T10:06:22.227960+0000","last_fullsized":"2024-04-25T10:06:22.227960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:12.144339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.217062+0000","last_change":"2024-04-25T10:06:22.215358+0000","last_active":"2024-04-25T10:06:22.217062+0000","last_peered":"2024-04-25T10:06:22.217062+0000","last_clean":"2024-04-25T10:06:22.217062+0000","last_became_active":"2024-04-25T10:06:22.215016+0000","last_became_peered":"2024-04-25T10:06:22.215016+0000","last_unstale":"2024-04-25T10:06:22.217062+0000","last_undegraded":"2024-04-25T10:06:22.217062+0000","last_fullsized":"2024-04-25T10:06:22.217062+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:42.822768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.225117+0000","last_change":"2024-04-25T10:06:21.197358+0000","last_active":"2024-04-25T10:06:22.225117+0000","last_peered":"2024-04-25T10:06:22.225117+0000","last_clean":"2024-04-25T10:06:22.225117+0000","last_became_active":"2024-04-25T10:06:21.196637+0000","last_became_peered":"2024-04-25T10:06:21.196637+0000","last_unstale":"2024-04-25T10:06:22.225117+0000","last_undegraded":"2024-04-25T10:06:22.225117+0000","last_fullsized":"2024-04-25T10:06:22.225117+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:32.153259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.216523+0000","last_change":"2024-04-25T10:06:22.215062+0000","last_active":"2024-04-25T10:06:22.216523+0000","last_peered":"2024-04-25T10:06:22.216523+0000","last_clean":"2024-04-25T10:06:22.216523+0000","last_became_active":"2024-04-25T10:06:22.214759+0000","last_became_peered":"2024-04-25T10:06:22.214759+0000","last_unstale":"2024-04-25T10:06:22.216523+0000","last_undegraded":"2024-04-25T10:06:22.216523+0000","last_fullsized":"2024-04-25T10:06:22.216523+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:59.421390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.207040+0000","last_change":"2024-04-25T10:06:21.189994+0000","last_active":"2024-04-25T10:06:23.207040+0000","last_peered":"2024-04-25T10:06:23.207040+0000","last_clean":"2024-04-25T10:06:23.207040+0000","last_became_active":"2024-04-25T10:06:21.189247+0000","last_became_peered":"2024-04-25T10:06:21.189247+0000","last_unstale":"2024-04-25T10:06:23.207040+0000","last_undegraded":"2024-04-25T10:06:23.207040+0000","last_fullsized":"2024-04-25T10:06:23.207040+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:42.582923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.205937+0000","last_change":"2024-04-25T10:06:22.262596+0000","last_active":"2024-04-25T10:06:23.205937+0000","last_peered":"2024-04-25T10:06:23.205937+0000","last_clean":"2024-04-25T10:06:23.205937+0000","last_became_active":"2024-04-25T10:06:22.262004+0000","last_became_peered":"2024-04-25T10:06:22.262004+0000","last_unstale":"2024-04-25T10:06:23.205937+0000","last_undegraded":"2024-04-25T10:06:23.205937+0000","last_fullsized":"2024-04-25T10:06:23.205937+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:06.026925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.209835+0000","last_change":"2024-04-25T10:06:21.185975+0000","last_active":"2024-04-25T10:06:22.209835+0000","last_peered":"2024-04-25T10:06:22.209835+0000","last_clean":"2024-04-25T10:06:22.209835+0000","last_became_active":"2024-04-25T10:06:21.185003+0000","last_became_peered":"2024-04-25T10:06:21.185003+0000","last_unstale":"2024-04-25T10:06:22.209835+0000","last_undegraded":"2024-04-25T10:06:22.209835+0000","last_fullsized":"2024-04-25T10:06:22.209835+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:08.126165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.237097+0000","last_change":"2024-04-25T10:06:22.236587+0000","last_active":"2024-04-25T10:06:22.237097+0000","last_peered":"2024-04-25T10:06:22.237097+0000","last_clean":"2024-04-25T10:06:22.237097+0000","last_became_active":"2024-04-25T10:06:22.236249+0000","last_became_peered":"2024-04-25T10:06:22.236249+0000","last_unstale":"2024-04-25T10:06:22.237097+0000","last_undegraded":"2024-04-25T10:06:22.237097+0000","last_fullsized":"2024-04-25T10:06:22.237097+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:29.187792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.242140+0000","last_change":"2024-04-25T10:06:21.180674+0000","last_active":"2024-04-25T10:06:22.242140+0000","last_peered":"2024-04-25T10:06:22.242140+0000","last_clean":"2024-04-25T10:06:22.242140+0000","last_became_active":"2024-04-25T10:06:21.180369+0000","last_became_peered":"2024-04-25T10:06:21.180369+0000","last_unstale":"2024-04-25T10:06:22.242140+0000","last_undegraded":"2024-04-25T10:06:22.242140+0000","last_fullsized":"2024-04-25T10:06:22.242140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:54.064803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.254777+0000","last_change":"2024-04-25T10:06:22.245552+0000","last_active":"2024-04-25T10:06:22.254777+0000","last_peered":"2024-04-25T10:06:22.254777+0000","last_clean":"2024-04-25T10:06:22.254777+0000","last_became_active":"2024-04-25T10:06:22.243081+0000","last_became_peered":"2024-04-25T10:06:22.243081+0000","last_unstale":"2024-04-25T10:06:22.254777+0000","last_undegraded":"2024-04-25T10:06:22.254777+0000","last_fullsized":"2024-04-25T10:06:22.254777+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:40.277052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.215916+0000","last_change":"2024-04-25T10:06:21.179129+0000","last_active":"2024-04-25T10:06:22.215916+0000","last_peered":"2024-04-25T10:06:22.215916+0000","last_clean":"2024-04-25T10:06:22.215916+0000","last_became_active":"2024-04-25T10:06:21.178801+0000","last_became_peered":"2024-04-25T10:06:21.178801+0000","last_unstale":"2024-04-25T10:06:22.215916+0000","last_undegraded":"2024-04-25T10:06:22.215916+0000","last_fullsized":"2024-04-25T10:06:22.215916+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:13.117982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.228902+0000","last_change":"2024-04-25T10:06:22.228130+0000","last_active":"2024-04-25T10:06:22.228902+0000","last_peered":"2024-04-25T10:06:22.228902+0000","last_clean":"2024-04-25T10:06:22.228902+0000","last_became_active":"2024-04-25T10:06:22.227872+0000","last_became_peered":"2024-04-25T10:06:22.227872+0000","last_unstale":"2024-04-25T10:06:22.228902+0000","last_undegraded":"2024-04-25T10:06:22.228902+0000","last_fullsized":"2024-04-25T10:06:22.228902+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:46.770914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.201863+0000","last_change":"2024-04-25T10:06:21.173661+0000","last_active":"2024-04-25T10:06:22.201863+0000","last_peered":"2024-04-25T10:06:22.201863+0000","last_clean":"2024-04-25T10:06:22.201863+0000","last_became_active":"2024-04-25T10:06:21.172981+0000","last_became_peered":"2024-04-25T10:06:21.172981+0000","last_unstale":"2024-04-25T10:06:22.201863+0000","last_undegraded":"2024-04-25T10:06:22.201863+0000","last_fullsized":"2024-04-25T10:06:22.201863+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:06.034487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.241946+0000","last_change":"2024-04-25T10:06:22.231446+0000","last_active":"2024-04-25T10:06:22.241946+0000","last_peered":"2024-04-25T10:06:22.241946+0000","last_clean":"2024-04-25T10:06:22.241946+0000","last_became_active":"2024-04-25T10:06:22.230802+0000","last_became_peered":"2024-04-25T10:06:22.230802+0000","last_unstale":"2024-04-25T10:06:22.241946+0000","last_undegraded":"2024-04-25T10:06:22.241946+0000","last_fullsized":"2024-04-25T10:06:22.241946+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:33.567907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.213725+0000","last_change":"2024-04-25T10:06:21.175341+0000","last_active":"2024-04-25T10:06:22.213725+0000","last_peered":"2024-04-25T10:06:22.213725+0000","last_clean":"2024-04-25T10:06:22.213725+0000","last_became_active":"2024-04-25T10:06:21.175036+0000","last_became_peered":"2024-04-25T10:06:21.175036+0000","last_unstale":"2024-04-25T10:06:22.213725+0000","last_undegraded":"2024-04-25T10:06:22.213725+0000","last_fullsized":"2024-04-25T10:06:22.213725+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:32.564562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.232171+0000","last_change":"2024-04-25T10:06:22.231628+0000","last_active":"2024-04-25T10:06:22.232171+0000","last_peered":"2024-04-25T10:06:22.232171+0000","last_clean":"2024-04-25T10:06:22.232171+0000","last_became_active":"2024-04-25T10:06:22.231360+0000","last_became_peered":"2024-04-25T10:06:22.231360+0000","last_unstale":"2024-04-25T10:06:22.232171+0000","last_undegraded":"2024-04-25T10:06:22.232171+0000","last_fullsized":"2024-04-25T10:06:22.232171+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:44.331490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.201488+0000","last_change":"2024-04-25T10:06:21.180608+0000","last_active":"2024-04-25T10:06:22.201488+0000","last_peered":"2024-04-25T10:06:22.201488+0000","last_clean":"2024-04-25T10:06:22.201488+0000","last_became_active":"2024-04-25T10:06:21.180323+0000","last_became_peered":"2024-04-25T10:06:21.180323+0000","last_unstale":"2024-04-25T10:06:22.201488+0000","last_undegraded":"2024-04-25T10:06:22.201488+0000","last_fullsized":"2024-04-25T10:06:22.201488+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:42.582923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.224943+0000","last_change":"2024-04-25T10:06:22.223967+0000","last_active":"2024-04-25T10:06:22.224943+0000","last_peered":"2024-04-25T10:06:22.224943+0000","last_clean":"2024-04-25T10:06:22.224943+0000","last_became_active":"2024-04-25T10:06:22.223047+0000","last_became_peered":"2024-04-25T10:06:22.223047+0000","last_unstale":"2024-04-25T10:06:22.224943+0000","last_undegraded":"2024-04-25T10:06:22.224943+0000","last_fullsized":"2024-04-25T10:06:22.224943+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:33.404890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.207272+0000","last_change":"2024-04-25T10:06:21.175353+0000","last_active":"2024-04-25T10:06:23.207272+0000","last_peered":"2024-04-25T10:06:23.207272+0000","last_clean":"2024-04-25T10:06:23.207272+0000","last_became_active":"2024-04-25T10:06:21.174456+0000","last_became_peered":"2024-04-25T10:06:21.174456+0000","last_unstale":"2024-04-25T10:06:23.207272+0000","last_undegraded":"2024-04-25T10:06:23.207272+0000","last_fullsized":"2024-04-25T10:06:23.207272+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:54.064803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.206792+0000","last_change":"2024-04-25T10:06:22.241124+0000","last_active":"2024-04-25T10:06:23.206792+0000","last_peered":"2024-04-25T10:06:23.206792+0000","last_clean":"2024-04-25T10:06:23.206792+0000","last_became_active":"2024-04-25T10:06:22.230784+0000","last_became_peered":"2024-04-25T10:06:22.230784+0000","last_unstale":"2024-04-25T10:06:23.206792+0000","last_undegraded":"2024-04-25T10:06:23.206792+0000","last_fullsized":"2024-04-25T10:06:23.206792+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:36.091611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.205551+0000","last_change":"2024-04-25T10:06:21.187081+0000","last_active":"2024-04-25T10:06:22.205551+0000","last_peered":"2024-04-25T10:06:22.205551+0000","last_clean":"2024-04-25T10:06:22.205551+0000","last_became_active":"2024-04-25T10:06:21.186416+0000","last_became_peered":"2024-04-25T10:06:21.186416+0000","last_unstale":"2024-04-25T10:06:22.205551+0000","last_undegraded":"2024-04-25T10:06:22.205551+0000","last_fullsized":"2024-04-25T10:06:22.205551+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:54.296461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.206546+0000","last_change":"2024-04-25T10:06:22.251022+0000","last_active":"2024-04-25T10:06:23.206546+0000","last_peered":"2024-04-25T10:06:23.206546+0000","last_clean":"2024-04-25T10:06:23.206546+0000","last_became_active":"2024-04-25T10:06:22.250190+0000","last_became_peered":"2024-04-25T10:06:22.250190+0000","last_unstale":"2024-04-25T10:06:23.206546+0000","last_undegraded":"2024-04-25T10:06:23.206546+0000","last_fullsized":"2024-04-25T10:06:23.206546+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.144374+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.144374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:09.093081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:06:22.229942+0000","last_change":"2024-04-25T10:06:21.196244+0000","last_active":"2024-04-25T10:06:22.229942+0000","last_peered":"2024-04-25T10:06:22.229942+0000","last_clean":"2024-04-25T10:06:22.229942+0000","last_became_active":"2024-04-25T10:06:21.195589+0000","last_became_peered":"2024-04-25T10:06:21.195589+0000","last_unstale":"2024-04-25T10:06:22.229942+0000","last_undegraded":"2024-04-25T10:06:22.229942+0000","last_fullsized":"2024-04-25T10:06:22.229942+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:20.141029+0000","last_clean_scrub_stamp":"2024-04-25T10:06:20.141029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:12.144339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":195,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:06:23.206410+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:06:23.206410+0000","last_peered":"2024-04-25T10:06:23.206410+0000","last_clean":"2024-04-25T10:06:23.206410+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:06:23.206410+0000","last_undegraded":"2024-04-25T10:06:23.206410+0000","last_fullsized":"2024-04-25T10:06:23.206410+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":974848,"data_stored":738202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":147848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":148360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":148416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":364544,"data_stored":140969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":364544,"data_stored":140969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35700,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336140,"statfs":{"total":96636764160,"available":96600207360,"internally_reserved":0,"allocated":364544,"data_stored":140969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":958464,"data_stored":731337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:25.636 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:06:25.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:06:25.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.799+0000 7fdd7574d640 1 Processor -- start 2024-04-25T10:06:25.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.799+0000 7fdd7574d640 1 -- start start 2024-04-25T10:06:25.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.803+0000 7fdd7574d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdd70160e70 0x7fdd70161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.803+0000 7fdd7574d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd700b5b10 0x7fdd700d2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.803+0000 7fdd7574d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd70162200 con 0x7fdd70161250 2024-04-25T10:06:25.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.803+0000 7fdd7574d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd700d1a30 con 0x7fdd70160e70 2024-04-25T10:06:25.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.803+0000 7fdd7574d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd701624e0 con 0x7fdd700b5b10 2024-04-25T10:06:25.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.803+0000 7fdd6f7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd700b5b10 0x7fdd700d2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.803+0000 7fdd6effd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdd70160e70 0x7fdd70161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.803+0000 7fdd6f7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd700b5b10 0x7fdd700d2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:58698/0 (socket says 172.21.15.60:58698) 2024-04-25T10:06:25.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.803+0000 7fdd6f7fe640 1 -- 172.21.15.60:0/1274504296 learned_addr learned my addr 172.21.15.60:0/1274504296 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:25.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.803+0000 7fdd6effd640 1 -- 172.21.15.60:0/1274504296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd700b5b10 msgr2=0x7fdd700d2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:25.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.803+0000 7fdd6effd640 1 --2- 172.21.15.60:0/1274504296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd700b5b10 0x7fdd700d2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.803+0000 7fdd6effd640 1 -- 172.21.15.60:0/1274504296 >> v1:172.21.15.100:6789/0 conn(0x7fdd70161250 legacy=0x7fdd700b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.803+0000 7fdd6effd640 1 -- 172.21.15.60:0/1274504296 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd700d1c80 con 0x7fdd70160e70 2024-04-25T10:06:25.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.803+0000 7fdd6f7fe640 1 --2- 172.21.15.60:0/1274504296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd700b5b10 0x7fdd700d2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:25.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.803+0000 7fdd6effd640 1 --2- 172.21.15.60:0/1274504296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd70160e70 0x7fdd70161760 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdd600097c0 tx=0x7fdd6001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d3037c47c7b0213 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd6dffb640 1 -- 172.21.15.60:0/1274504296 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd600b0070 con 0x7fdd70160e70 2024-04-25T10:06:25.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd6dffb640 1 -- 172.21.15.60:0/1274504296 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd600ab600 con 0x7fdd70160e70 2024-04-25T10:06:25.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd6dffb640 1 -- 172.21.15.60:0/1274504296 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd600ab9d0 con 0x7fdd70160e70 2024-04-25T10:06:25.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 -- 172.21.15.60:0/1274504296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd70160e70 msgr2=0x7fdd70161760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 --2- 172.21.15.60:0/1274504296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd70160e70 0x7fdd70161760 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdd600097c0 tx=0x7fdd6001c3e0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 -- 172.21.15.60:0/1274504296 shutdown_connections 2024-04-25T10:06:25.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 --2- 172.21.15.60:0/1274504296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd700b5b10 0x7fdd700d2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 --2- 172.21.15.60:0/1274504296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd70160e70 0x7fdd70161760 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 -- 172.21.15.60:0/1274504296 >> 172.21.15.60:0/1274504296 conn(0x7fdd700d05d0 msgr2=0x7fdd700d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:25.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 -- 172.21.15.60:0/1274504296 shutdown_connections 2024-04-25T10:06:25.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 -- 172.21.15.60:0/1274504296 wait complete. 2024-04-25T10:06:25.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 Processor -- start 2024-04-25T10:06:25.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 -- start start 2024-04-25T10:06:25.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd700b5b10 0x7fdd70152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd70160e70 0x7fdd7015c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd70161250 0x7fdd70156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd700d3560 con 0x7fdd70160e70 2024-04-25T10:06:25.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd700d33e0 con 0x7fdd70161250 2024-04-25T10:06:25.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd700d36e0 con 0x7fdd700b5b10 2024-04-25T10:06:25.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd6e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd70160e70 0x7fdd7015c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd6e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd70160e70 0x7fdd7015c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37804/0 (socket says 172.21.15.60:37804) 2024-04-25T10:06:25.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd6e7fc640 1 -- 172.21.15.60:0/848609973 learned_addr learned my addr 172.21.15.60:0/848609973 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:25.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd6effd640 1 --2- 172.21.15.60:0/848609973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd700b5b10 0x7fdd70152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd6f7fe640 1 --2- 172.21.15.60:0/848609973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd70161250 0x7fdd70156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd6e7fc640 1 -- 172.21.15.60:0/848609973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd700b5b10 msgr2=0x7fdd70152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd6e7fc640 1 --2- 172.21.15.60:0/848609973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd700b5b10 0x7fdd70152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd6e7fc640 1 -- 172.21.15.60:0/848609973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd70161250 msgr2=0x7fdd70156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd6e7fc640 1 --2- 172.21.15.60:0/848609973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd70161250 0x7fdd70156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd6e7fc640 1 -- 172.21.15.60:0/848609973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7015ff00 con 0x7fdd70160e70 2024-04-25T10:06:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd6f7fe640 1 --2- 172.21.15.60:0/848609973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd70161250 0x7fdd70156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd6e7fc640 1 --2- 172.21.15.60:0/848609973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd70160e70 0x7fdd7015c1e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c340 tx=0x7fdd6400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd577fe640 1 -- 172.21.15.60:0/848609973 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd640052d0 con 0x7fdd70160e70 2024-04-25T10:06:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 -- 172.21.15.60:0/848609973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd701601e0 con 0x7fdd70160e70 2024-04-25T10:06:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd577fe640 1 -- 172.21.15.60:0/848609973 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd64005470 con 0x7fdd70160e70 2024-04-25T10:06:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd577fe640 1 -- 172.21.15.60:0/848609973 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64007260 con 0x7fdd70160e70 2024-04-25T10:06:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.807+0000 7fdd7574d640 1 -- 172.21.15.60:0/848609973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd701541e0 con 0x7fdd70160e70 2024-04-25T10:06:25.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.811+0000 7fdd7574d640 1 -- 172.21.15.60:0/848609973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd70074770 con 0x7fdd70160e70 2024-04-25T10:06:25.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.811+0000 7fdd577fe640 1 -- 172.21.15.60:0/848609973 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd64007400 con 0x7fdd70160e70 2024-04-25T10:06:25.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.811+0000 7fdd577fe640 1 --2- 172.21.15.60:0/848609973 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdd5c066f60 0x7fdd5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.811+0000 7fdd577fe640 1 -- 172.21.15.60:0/848609973 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd64084520 con 0x7fdd70160e70 2024-04-25T10:06:25.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.811+0000 7fdd6effd640 1 --2- 172.21.15.60:0/848609973 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdd5c066f60 0x7fdd5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.811+0000 7fdd6effd640 1 --2- 172.21.15.60:0/848609973 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdd5c066f60 0x7fdd5c069420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdd600ad000 tx=0x7fdd600a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:25.815+0000 7fdd577fe640 1 -- 172.21.15.60:0/848609973 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdd64054dc0 con 0x7fdd70160e70 2024-04-25T10:06:26.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.027+0000 7fdd7574d640 1 -- 172.21.15.60:0/848609973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdd701543b0 con 0x7fdd70160e70 2024-04-25T10:06:26.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.031+0000 7fdd577fe640 1 -- 172.21.15.60:0/848609973 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fdd64058e60 con 0x7fdd70160e70 2024-04-25T10:06:26.037 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T10:06:26.037 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCiKipmv65dLxAA6/QpPrqE9aT+CtZqkyutog== 2024-04-25T10:06:26.037 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-04-25T10:06:26.037 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-04-25T10:06:26.037 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:06:26.037 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-04-25T10:06:26.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.031+0000 7fdd7574d640 1 -- 172.21.15.60:0/848609973 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdd5c066f60 msgr2=0x7fdd5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.031+0000 7fdd7574d640 1 --2- 172.21.15.60:0/848609973 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdd5c066f60 0x7fdd5c069420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdd600ad000 tx=0x7fdd600a6000 comp rx=0 tx=0).stop 2024-04-25T10:06:26.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.031+0000 7fdd7574d640 1 -- 172.21.15.60:0/848609973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd70160e70 msgr2=0x7fdd7015c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.031+0000 7fdd7574d640 1 --2- 172.21.15.60:0/848609973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd70160e70 0x7fdd7015c1e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c340 tx=0x7fdd6400c800 comp rx=0 tx=0).stop 2024-04-25T10:06:26.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.031+0000 7fdd7574d640 1 -- 172.21.15.60:0/848609973 shutdown_connections 2024-04-25T10:06:26.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.031+0000 7fdd7574d640 1 --2- 172.21.15.60:0/848609973 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdd5c066f60 0x7fdd5c069420 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.031+0000 7fdd7574d640 1 --2- 172.21.15.60:0/848609973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd70161250 0x7fdd70156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.031+0000 7fdd7574d640 1 --2- 172.21.15.60:0/848609973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd70160e70 0x7fdd7015c1e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.031+0000 7fdd7574d640 1 --2- 172.21.15.60:0/848609973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd700b5b10 0x7fdd70152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.031+0000 7fdd7574d640 1 -- 172.21.15.60:0/848609973 >> 172.21.15.60:0/848609973 conn(0x7fdd700d05d0 msgr2=0x7fdd7015db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.031+0000 7fdd7574d640 1 -- 172.21.15.60:0/848609973 shutdown_connections 2024-04-25T10:06:26.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.031+0000 7fdd7574d640 1 -- 172.21.15.60:0/848609973 wait complete. 2024-04-25T10:06:26.054 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:26.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.171+0000 7f47af780640 1 Processor -- start 2024-04-25T10:06:26.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.171+0000 7f47af780640 1 -- start start 2024-04-25T10:06:26.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.171+0000 7f47af780640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f47a8071b20 0x7f47a8071f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.171+0000 7f47af780640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a8072440 con 0x7f47a806e480 2024-04-25T10:06:26.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.171+0000 7f47af780640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a815dc10 con 0x7f47a80cce50 2024-04-25T10:06:26.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.171+0000 7f47af780640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a815def0 con 0x7f47a8071b20 2024-04-25T10:06:26.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.171+0000 7f47ad51c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f47a8071b20 0x7f47a8071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.171+0000 7f47acd1b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f47a806e480 0x7f47a806e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45882/0 (socket says 172.21.15.60:45882) 2024-04-25T10:06:26.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.171+0000 7f47ad51c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f47a8071b20 0x7f47a8071f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37818/0 (socket says 172.21.15.60:37818) 2024-04-25T10:06:26.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.171+0000 7f47acd1b640 1 -- 172.21.15.60:0/2508571215 learned_addr learned my addr 172.21.15.60:0/2508571215 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:26.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.171+0000 7f47ad51c640 1 -- 172.21.15.60:0/2508571215 >> v1:172.21.15.100:6789/0 conn(0x7f47a80cce50 legacy=0x7f47a80cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.171+0000 7f47ad51c640 1 -- 172.21.15.60:0/2508571215 >> v1:172.21.15.60:6789/0 conn(0x7f47a806e480 legacy=0x7f47a806e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.171+0000 7f47ad51c640 1 -- 172.21.15.60:0/2508571215 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a815e1d0 con 0x7f47a8071b20 2024-04-25T10:06:26.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.171+0000 7f479ffff640 1 -- 172.21.15.60:0/2508571215 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949460225 0 0) 0x7f47a8072440 con 0x7f47a806e480 2024-04-25T10:06:26.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47ad51c640 1 --2- 172.21.15.60:0/2508571215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47a8071b20 0x7f47a8071f00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f47a400a960 tx=0x7f47a401d7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1debb29842badd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f479ffff640 1 -- 172.21.15.60:0/2508571215 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a400b040 con 0x7f47a8071b20 2024-04-25T10:06:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f479ffff640 1 -- 172.21.15.60:0/2508571215 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a40a9660 con 0x7f47a8071b20 2024-04-25T10:06:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f479ffff640 1 -- 172.21.15.60:0/2508571215 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a40a9a30 con 0x7f47a8071b20 2024-04-25T10:06:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47af780640 1 -- 172.21.15.60:0/2508571215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47a8071b20 msgr2=0x7f47a8071f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47af780640 1 --2- 172.21.15.60:0/2508571215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47a8071b20 0x7f47a8071f00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f47a400a960 tx=0x7f47a401d7a0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47af780640 1 -- 172.21.15.60:0/2508571215 shutdown_connections 2024-04-25T10:06:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47af780640 1 --2- 172.21.15.60:0/2508571215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47a8071b20 0x7f47a8071f00 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47af780640 1 -- 172.21.15.60:0/2508571215 >> 172.21.15.60:0/2508571215 conn(0x7f47a80b12d0 msgr2=0x7f47a80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47af780640 1 -- 172.21.15.60:0/2508571215 shutdown_connections 2024-04-25T10:06:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47af780640 1 -- 172.21.15.60:0/2508571215 wait complete. 2024-04-25T10:06:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47af780640 1 Processor -- start 2024-04-25T10:06:26.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47af780640 1 -- start start 2024-04-25T10:06:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47af780640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a806e480 0x7f47a8158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47af780640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47a8071b20 0x7f47a813eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47af780640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47a80cce50 0x7f47a806d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47af780640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a80cdaf0 con 0x7f47a8071b20 2024-04-25T10:06:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47af780640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a80cd970 con 0x7f47a806e480 2024-04-25T10:06:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47af780640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a80cdc70 con 0x7f47a80cce50 2024-04-25T10:06:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47acd1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47a8071b20 0x7f47a813eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47acd1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47a8071b20 0x7f47a813eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37820/0 (socket says 172.21.15.60:37820) 2024-04-25T10:06:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47acd1b640 1 -- 172.21.15.60:0/1247229425 learned_addr learned my addr 172.21.15.60:0/1247229425 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47ad51c640 1 --2- 172.21.15.60:0/1247229425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a806e480 0x7f47a8158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47add1d640 1 --2- 172.21.15.60:0/1247229425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47a80cce50 0x7f47a806d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47acd1b640 1 -- 172.21.15.60:0/1247229425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47a80cce50 msgr2=0x7f47a806d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.175+0000 7f47acd1b640 1 --2- 172.21.15.60:0/1247229425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47a80cce50 0x7f47a806d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.179+0000 7f47acd1b640 1 -- 172.21.15.60:0/1247229425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a806e480 msgr2=0x7f47a8158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.179+0000 7f47acd1b640 1 --2- 172.21.15.60:0/1247229425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a806e480 0x7f47a8158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.179+0000 7f47acd1b640 1 -- 172.21.15.60:0/1247229425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a80b3be0 con 0x7f47a8071b20 2024-04-25T10:06:26.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.179+0000 7f47acd1b640 1 --2- 172.21.15.60:0/1247229425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47a8071b20 0x7f47a813eff0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f47980011f0 tx=0x7f479800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.179+0000 7f479dffb640 1 -- 172.21.15.60:0/1247229425 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798018070 con 0x7f47a8071b20 2024-04-25T10:06:26.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.179+0000 7f479dffb640 1 -- 172.21.15.60:0/1247229425 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47980074e0 con 0x7f47a8071b20 2024-04-25T10:06:26.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.179+0000 7f47af780640 1 -- 172.21.15.60:0/1247229425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a80b3ec0 con 0x7f47a8071b20 2024-04-25T10:06:26.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.179+0000 7f479dffb640 1 -- 172.21.15.60:0/1247229425 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798004a10 con 0x7f47a8071b20 2024-04-25T10:06:26.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.179+0000 7f47af780640 1 -- 172.21.15.60:0/1247229425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a81568e0 con 0x7f47a8071b20 2024-04-25T10:06:26.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.179+0000 7f47af780640 1 -- 172.21.15.60:0/1247229425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a80b4090 con 0x7f47a8071b20 2024-04-25T10:06:26.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.179+0000 7f479dffb640 1 -- 172.21.15.60:0/1247229425 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f479801b070 con 0x7f47a8071b20 2024-04-25T10:06:26.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.179+0000 7f479dffb640 1 --2- 172.21.15.60:0/1247229425 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f478c066f60 0x7f478c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.179+0000 7f479dffb640 1 -- 172.21.15.60:0/1247229425 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47980843e0 con 0x7f47a8071b20 2024-04-25T10:06:26.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.179+0000 7f47ad51c640 1 --2- 172.21.15.60:0/1247229425 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f478c066f60 0x7f478c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.179+0000 7f47ad51c640 1 --2- 172.21.15.60:0/1247229425 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f478c066f60 0x7f478c069420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f47a40b0000 tx=0x7f47a40b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.183+0000 7f479dffb640 1 -- 172.21.15.60:0/1247229425 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f47980553f0 con 0x7f47a8071b20 2024-04-25T10:06:26.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.379+0000 7f47af780640 1 -- 172.21.15.60:0/1247229425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47a8156ae0 con 0x7f47a8071b20 2024-04-25T10:06:26.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.379+0000 7f479dffb640 1 -- 172.21.15.60:0/1247229425 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f47a8156ae0 con 0x7f47a8071b20 2024-04-25T10:06:26.386 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:26.386 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2422261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2422261630},{"type":"v1","addr":"172.21.15.100:6835","nonce":2422261630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2923340508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2923340508},{"type":"v1","addr":"172.21.15.100:6837","nonce":2923340508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1929386449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1929386449},{"type":"v1","addr":"172.21.15.60:6833","nonce":1929386449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:21.529455+0000","modified":"2024-04-25T10:06:23.575106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5283":{"gid":5283,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3681296171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3681296171},{"type":"v1","addr":"172.21.15.60:6835","nonce":3681296171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 15 2024-04-25T10:06:26.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.379+0000 7f47af780640 1 -- 172.21.15.60:0/1247229425 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f478c066f60 msgr2=0x7f478c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.379+0000 7f47af780640 1 --2- 172.21.15.60:0/1247229425 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f478c066f60 0x7f478c069420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f47a40b0000 tx=0x7f47a40b3040 comp rx=0 tx=0).stop 2024-04-25T10:06:26.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.379+0000 7f47af780640 1 -- 172.21.15.60:0/1247229425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47a8071b20 msgr2=0x7f47a813eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.379+0000 7f47af780640 1 --2- 172.21.15.60:0/1247229425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47a8071b20 0x7f47a813eff0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f47980011f0 tx=0x7f479800c9b0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.383+0000 7f47af780640 1 -- 172.21.15.60:0/1247229425 shutdown_connections 2024-04-25T10:06:26.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.383+0000 7f47af780640 1 --2- 172.21.15.60:0/1247229425 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f478c066f60 0x7f478c069420 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.383+0000 7f47af780640 1 --2- 172.21.15.60:0/1247229425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47a80cce50 0x7f47a806d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.383+0000 7f47af780640 1 --2- 172.21.15.60:0/1247229425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47a8071b20 0x7f47a813eff0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.383+0000 7f47af780640 1 --2- 172.21.15.60:0/1247229425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47a806e480 0x7f47a8158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.383+0000 7f47af780640 1 -- 172.21.15.60:0/1247229425 >> 172.21.15.60:0/1247229425 conn(0x7f47a80b12d0 msgr2=0x7f47a8071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.383+0000 7f47af780640 1 -- 172.21.15.60:0/1247229425 shutdown_connections 2024-04-25T10:06:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.383+0000 7f47af780640 1 -- 172.21.15.60:0/1247229425 wait complete. 2024-04-25T10:06:26.808 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:21.529455+0000', 'modified': '2024-04-25T10:06:23.575106+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5283}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5283': {'gid': 5283, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/3681296171', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 3681296171}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 3681296171}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:06:26.809 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:06:26.809 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:06:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.939+0000 7f6f43349640 1 Processor -- start 2024-04-25T10:06:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.939+0000 7f6f43349640 1 -- start start 2024-04-25T10:06:26.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.939+0000 7f6f43349640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f3c15bc90 0x7f6f3c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.939+0000 7f6f43349640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6f3c0b4660 0x7f6f3c15c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.939+0000 7f6f43349640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f3c071ca0 con 0x7f6f3c0ce7a0 2024-04-25T10:06:26.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.939+0000 7f6f43349640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f3c0cc370 con 0x7f6f3c0b4660 2024-04-25T10:06:26.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.939+0000 7f6f43349640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c0cca40 con 0x7f6f3c15bc90 2024-04-25T10:06:26.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.939+0000 7f6f410e5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6f3c0ce7a0 0x7f6f3c0ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45898/0 (socket says 172.21.15.60:45898) 2024-04-25T10:06:26.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.939+0000 7f6f410e5640 1 -- 172.21.15.60:0/3358800264 learned_addr learned my addr 172.21.15.60:0/3358800264 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:26.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.939+0000 7f6f408e4640 1 --2- 172.21.15.60:0/3358800264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f3c15bc90 0x7f6f3c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.939+0000 7f6f418e6640 1 --2- 172.21.15.60:0/3358800264 >> v2:172.21.15.100:3300/0 conn(0x7f6f3c0b4660 0x7f6f3c15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 -- 172.21.15.60:0/3358800264 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170156677 0 0) 0x7f6f3c071ca0 con 0x7f6f3c0ce7a0 2024-04-25T10:06:26.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 -- 172.21.15.60:0/3358800264 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f28003530 con 0x7f6f3c0ce7a0 2024-04-25T10:06:26.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 -- 172.21.15.60:0/3358800264 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 806994304 0 0) 0x7f6f28003530 con 0x7f6f3c0ce7a0 2024-04-25T10:06:26.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 -- 172.21.15.60:0/3358800264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f3c15bc90 msgr2=0x7f6f3c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 --2- 172.21.15.60:0/3358800264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f3c15bc90 0x7f6f3c15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 -- 172.21.15.60:0/3358800264 >> v2:172.21.15.100:3300/0 conn(0x7f6f3c0b4660 msgr2=0x7f6f3c15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 --2- 172.21.15.60:0/3358800264 >> v2:172.21.15.100:3300/0 conn(0x7f6f3c0b4660 0x7f6f3c15c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 -- 172.21.15.60:0/3358800264 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c0ccd20 con 0x7f6f3c0ce7a0 2024-04-25T10:06:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 -- 172.21.15.60:0/3358800264 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f6f38001680 con 0x7f6f3c0ce7a0 2024-04-25T10:06:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 -- 172.21.15.60:0/3358800264 >> v1:172.21.15.60:6789/0 conn(0x7f6f3c0ce7a0 legacy=0x7f6f3c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 --2- 172.21.15.60:0/3358800264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f280039c0 0x7f6f28003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 --2- 172.21.15.60:0/3358800264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f28004810 0x7f6f28004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 --2- 172.21.15.60:0/3358800264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f3c15bc90 0x7f6f280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 -- 172.21.15.60:0/3358800264 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c0cca40 con 0x7f6f28004810 2024-04-25T10:06:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 -- 172.21.15.60:0/3358800264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2800b9d0 con 0x7f6f280039c0 2024-04-25T10:06:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 -- 172.21.15.60:0/3358800264 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2800bb50 con 0x7f6f3c15bc90 2024-04-25T10:06:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f408e4640 1 --2- 172.21.15.60:0/3358800264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f28004810 0x7f6f28004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f410e5640 1 --2- 172.21.15.60:0/3358800264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f280039c0 0x7f6f28003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f418e6640 1 --2- 172.21.15.60:0/3358800264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f3c15bc90 0x7f6f280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f408e4640 1 -- 172.21.15.60:0/3358800264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f3c15bc90 msgr2=0x7f6f280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f408e4640 1 --2- 172.21.15.60:0/3358800264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f3c15bc90 0x7f6f280051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f408e4640 1 -- 172.21.15.60:0/3358800264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f280039c0 msgr2=0x7f6f28003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f408e4640 1 --2- 172.21.15.60:0/3358800264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f280039c0 0x7f6f28003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f408e4640 1 -- 172.21.15.60:0/3358800264 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f28003530 con 0x7f6f28004810 2024-04-25T10:06:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f410e5640 1 --2- 172.21.15.60:0/3358800264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f280039c0 0x7f6f28003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f408e4640 1 --2- 172.21.15.60:0/3358800264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f28004810 0x7f6f28004c90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009ae0 tx=0x7f6f2c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3c9e074263e556 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 -- 172.21.15.60:0/3358800264 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0b2070 con 0x7f6f28004810 2024-04-25T10:06:26.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 -- 172.21.15.60:0/3358800264 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c0a5580 con 0x7f6f28004810 2024-04-25T10:06:26.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f33fff640 1 -- 172.21.15.60:0/3358800264 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0096b0 con 0x7f6f28004810 2024-04-25T10:06:26.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f43349640 1 -- 172.21.15.60:0/3358800264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f28004810 msgr2=0x7f6f28004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f43349640 1 --2- 172.21.15.60:0/3358800264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f28004810 0x7f6f28004c90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009ae0 tx=0x7f6f2c01ca70 comp rx=0 tx=0).stop 2024-04-25T10:06:26.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f418e6640 1 -- 172.21.15.60:0/3358800264 reap_dead start 2024-04-25T10:06:26.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f43349640 1 -- 172.21.15.60:0/3358800264 shutdown_connections 2024-04-25T10:06:26.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.943+0000 7f6f43349640 1 -- 172.21.15.60:0/3358800264 >> 172.21.15.60:0/3358800264 conn(0x7f6f3c0b12d0 msgr2=0x7f6f3c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f43349640 1 -- 172.21.15.60:0/3358800264 shutdown_connections 2024-04-25T10:06:26.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f43349640 1 -- 172.21.15.60:0/3358800264 wait complete. 2024-04-25T10:06:26.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f43349640 1 Processor -- start 2024-04-25T10:06:26.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f43349640 1 -- start start 2024-04-25T10:06:26.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f43349640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f28004810 0x7f6f3c1f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f43349640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f3c14d060 0x7f6f3c1502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f43349640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f3c14d4a0 0x7f6f3c150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f43349640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c0b3d20 con 0x7f6f3c14d060 2024-04-25T10:06:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f43349640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c0b3ba0 con 0x7f6f3c14d4a0 2024-04-25T10:06:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f43349640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c0b4280 con 0x7f6f28004810 2024-04-25T10:06:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f408e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f3c14d060 0x7f6f3c1502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f408e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f3c14d060 0x7f6f3c1502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37840/0 (socket says 172.21.15.60:37840) 2024-04-25T10:06:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f408e4640 1 -- 172.21.15.60:0/2215521022 learned_addr learned my addr 172.21.15.60:0/2215521022 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f410e5640 1 --2- 172.21.15.60:0/2215521022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f28004810 0x7f6f3c1f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f418e6640 1 --2- 172.21.15.60:0/2215521022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f3c14d4a0 0x7f6f3c150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f408e4640 1 -- 172.21.15.60:0/2215521022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f28004810 msgr2=0x7f6f3c1f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f408e4640 1 --2- 172.21.15.60:0/2215521022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f28004810 0x7f6f3c1f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f408e4640 1 -- 172.21.15.60:0/2215521022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f3c14d4a0 msgr2=0x7f6f3c150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f408e4640 1 --2- 172.21.15.60:0/2215521022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f3c14d4a0 0x7f6f3c150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f408e4640 1 -- 172.21.15.60:0/2215521022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c06d460 con 0x7f6f3c14d060 2024-04-25T10:06:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f418e6640 1 --2- 172.21.15.60:0/2215521022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f3c14d4a0 0x7f6f3c150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f408e4640 1 --2- 172.21.15.60:0/2215521022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f3c14d060 0x7f6f3c1502d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0b0040 tx=0x7f6f28005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f31ffb640 1 -- 172.21.15.60:0/2215521022 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0bd020 con 0x7f6f3c14d060 2024-04-25T10:06:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f31ffb640 1 -- 172.21.15.60:0/2215521022 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c004020 con 0x7f6f3c14d060 2024-04-25T10:06:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f43349640 1 -- 172.21.15.60:0/2215521022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f3c06d740 con 0x7f6f3c14d060 2024-04-25T10:06:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f31ffb640 1 -- 172.21.15.60:0/2215521022 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0b2040 con 0x7f6f3c14d060 2024-04-25T10:06:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.947+0000 7f6f43349640 1 -- 172.21.15.60:0/2215521022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3c06dd00 con 0x7f6f3c14d060 2024-04-25T10:06:26.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f31ffb640 1 -- 172.21.15.60:0/2215521022 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6f2c003620 con 0x7f6f3c14d060 2024-04-25T10:06:26.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f31ffb640 1 --2- 172.21.15.60:0/2215521022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f20066f90 0x7f6f20069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f31ffb640 1 -- 172.21.15.60:0/2215521022 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f2c11ed20 con 0x7f6f3c14d060 2024-04-25T10:06:26.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f410e5640 1 --2- 172.21.15.60:0/2215521022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f20066f90 0x7f6f20069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f464b6000 1 Processor -- start 2024-04-25T10:06:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f464b6000 1 -- start start 2024-04-25T10:06:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f410e5640 1 --2- 172.21.15.60:0/2215521022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f20066f90 0x7f6f20069450 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6f38001ec0 tx=0x7f6f38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f464b6000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cac16920 0x5578cac16d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f464b6000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cac17240 0x5578cabcc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f464b6000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cabccf30 0x5578cabcd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f464b6000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578caab45f0 con 0x5578cabccf30 2024-04-25T10:06:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f464b6000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578caa1bda0 con 0x5578cac17240 2024-04-25T10:06:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f464b6000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5578ca9a4150 con 0x5578cac16920 2024-04-25T10:06:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f418e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cabccf30 0x5578cabcd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f418e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cabccf30 0x5578cabcd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37846/0 (socket says 172.21.15.60:37846) 2024-04-25T10:06:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f418e6640 1 -- 172.21.15.60:0/857966046 learned_addr learned my addr 172.21.15.60:0/857966046 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f408e4640 1 --2- 172.21.15.60:0/857966046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cac17240 0x5578cabcc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f410e5640 1 --2- 172.21.15.60:0/857966046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cac16920 0x5578cac16d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f418e6640 1 -- 172.21.15.60:0/857966046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cac16920 msgr2=0x5578cac16d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f418e6640 1 --2- 172.21.15.60:0/857966046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cac16920 0x5578cac16d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f418e6640 1 -- 172.21.15.60:0/857966046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cac17240 msgr2=0x5578cabcc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f418e6640 1 --2- 172.21.15.60:0/857966046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cac17240 0x5578cabcc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f418e6640 1 -- 172.21.15.60:0/857966046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578ca9bdca0 con 0x5578cabccf30 2024-04-25T10:06:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f418e6640 1 --2- 172.21.15.60:0/857966046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cabccf30 0x5578cabcd310 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c072c10 tx=0x7f6f34009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2bd7a3fe06f9a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f26ffd640 1 -- 172.21.15.60:0/857966046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3401a020 con 0x5578cabccf30 2024-04-25T10:06:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f26ffd640 1 -- 172.21.15.60:0/857966046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f34016870 con 0x5578cabccf30 2024-04-25T10:06:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.951+0000 7f6f26ffd640 1 -- 172.21.15.60:0/857966046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f34016c60 con 0x5578cabccf30 2024-04-25T10:06:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 -- 172.21.15.60:0/857966046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cabccf30 msgr2=0x5578cabcd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 --2- 172.21.15.60:0/857966046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cabccf30 0x5578cabcd310 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c072c10 tx=0x7f6f34009750 comp rx=0 tx=0).stop 2024-04-25T10:06:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 -- 172.21.15.60:0/857966046 shutdown_connections 2024-04-25T10:06:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 --2- 172.21.15.60:0/857966046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cabccf30 0x5578cabcd310 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 --2- 172.21.15.60:0/857966046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cac17240 0x5578cabcc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 --2- 172.21.15.60:0/857966046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cac16920 0x5578cac16d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 -- 172.21.15.60:0/857966046 >> 172.21.15.60:0/857966046 conn(0x5578cabd32a0 msgr2=0x5578cabcb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 -- 172.21.15.60:0/857966046 shutdown_connections 2024-04-25T10:06:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 -- 172.21.15.60:0/857966046 wait complete. 2024-04-25T10:06:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 -- start start 2024-04-25T10:06:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 Processor -- start 2024-04-25T10:06:26.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cabccf30 0x5578cac1dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cac16920 0x5578cac1e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac17240 0x5578cabd0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578ca9d6990 con 0x5578cac17240 2024-04-25T10:06:26.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578caa17e40 con 0x5578cabccf30 2024-04-25T10:06:26.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5578ca9a7900 con 0x5578cac16920 2024-04-25T10:06:26.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 10 client.0 _ll_register_callbacks cb 0x5578caacf1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:26.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f418e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac17240 0x5578cabd0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f418e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac17240 0x5578cabd0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37862/0 (socket says 172.21.15.60:37862) 2024-04-25T10:06:26.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f418e6640 1 -- 172.21.15.60:0/559253033 learned_addr learned my addr 172.21.15.60:0/559253033 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:26.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f410e5640 1 --2- 172.21.15.60:0/559253033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cabccf30 0x5578cac1dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f408e4640 1 --2- 172.21.15.60:0/559253033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cac16920 0x5578cac1e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f418e6640 1 -- 172.21.15.60:0/559253033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cac16920 msgr2=0x5578cac1e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f418e6640 1 --2- 172.21.15.60:0/559253033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cac16920 0x5578cac1e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f418e6640 1 -- 172.21.15.60:0/559253033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cabccf30 msgr2=0x5578cac1dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f418e6640 1 --2- 172.21.15.60:0/559253033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cabccf30 0x5578cac1dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f418e6640 1 -- 172.21.15.60:0/559253033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578ca9d4630 con 0x5578cac17240 2024-04-25T10:06:26.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f418e6640 1 --2- 172.21.15.60:0/559253033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac17240 0x5578cabd0aa0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400b010 tx=0x7f6f34004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb2b2e633346aaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:06:26.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f1bfff640 1 -- 172.21.15.60:0/559253033 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3402f060 con 0x5578cac17240 2024-04-25T10:06:26.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 -- 172.21.15.60:0/559253033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f34016870 con 0x5578cac17240 2024-04-25T10:06:26.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f1bfff640 1 -- 172.21.15.60:0/559253033 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f340207e0 con 0x5578cac17240 2024-04-25T10:06:26.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f1bfff640 1 -- 172.21.15.60:0/559253033 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f34035050 con 0x5578cac17240 2024-04-25T10:06:26.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 10 client.5478 fetch_fsmap learned FSMap version 15 2024-04-25T10:06:26.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 -- 172.21.15.60:0/559253033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x5578ca9a43a0 con 0x5578cac17240 2024-04-25T10:06:26.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f1bfff640 1 -- 172.21.15.60:0/559253033 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f34016870 con 0x5578cac17240 2024-04-25T10:06:26.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f1bfff640 1 -- 172.21.15.60:0/559253033 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6f34038020 con 0x5578cac17240 2024-04-25T10:06:26.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 10 client.5478 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T10:06:26.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 10 client.5478 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:06:26.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 20 client.5478 populate_metadata read hostname 'smithi060' 2024-04-25T10:06:26.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 --2- 172.21.15.60:0/559253033 >> [v2:172.21.15.60:6834/3681296171,v1:172.21.15.60:6835/3681296171] conn(0x5578cabd2160 0x5578cac27dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 4 client.5478 mds_command: new command op to 5283 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:26.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f464b6000 1 -- 172.21.15.60:0/559253033 --> [v2:172.21.15.60:6834/3681296171,v1:172.21.15.60:6835/3681296171] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578ca9b7a50 con 0x5578cabd2160 2024-04-25T10:06:26.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.955+0000 7f6f410e5640 1 --2- 172.21.15.60:0/559253033 >> [v2:172.21.15.60:6834/3681296171,v1:172.21.15.60:6835/3681296171] conn(0x5578cabd2160 0x5578cac27dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f1bfff640 10 client.5478 ms_handle_connect on v2:172.21.15.60:6834/3681296171 2024-04-25T10:06:26.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f410e5640 1 --2- 172.21.15.60:0/559253033 >> [v2:172.21.15.60:6834/3681296171,v1:172.21.15.60:6835/3681296171] conn(0x5578cabd2160 0x5578cac27dc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6222014bc4b1f26 server_cookie=6f33c30aa1c67ad8 in_seq=0 out_seq=0 2024-04-25T10:06:26.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f1bfff640 1 -- 172.21.15.60:0/559253033 <== mds.0 v2:172.21.15.60:6834/3681296171 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5578ca9b7a50 con 0x5578cabd2160 2024-04-25T10:06:26.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f1bfff640 10 client.5478 handle_command_reply: tid=0 2024-04-25T10:06:26.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f464b6000 1 client.5478 shutdown 2024-04-25T10:06:26.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f464b6000 1 -- 172.21.15.60:0/559253033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac17240 msgr2=0x5578cabd0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f464b6000 1 --2- 172.21.15.60:0/559253033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac17240 0x5578cabd0aa0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400b010 tx=0x7f6f34004a50 comp rx=0 tx=0).stop 2024-04-25T10:06:26.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f464b6000 1 -- 172.21.15.60:0/559253033 shutdown_connections 2024-04-25T10:06:26.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f464b6000 1 --2- 172.21.15.60:0/559253033 >> [v2:172.21.15.60:6834/3681296171,v1:172.21.15.60:6835/3681296171] conn(0x5578cabd2160 0x5578cac27dc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f464b6000 1 --2- 172.21.15.60:0/559253033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac17240 0x5578cabd0aa0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f464b6000 1 --2- 172.21.15.60:0/559253033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cac16920 0x5578cac1e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f464b6000 1 --2- 172.21.15.60:0/559253033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cabccf30 0x5578cac1dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f1bfff640 0 client.5478 ms_handle_reset on v2:172.21.15.60:6834/3681296171 2024-04-25T10:06:26.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f464b6000 1 -- 172.21.15.60:0/559253033 >> 172.21.15.60:0/559253033 conn(0x5578cabd32a0 msgr2=0x5578cabcb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f464b6000 1 -- 172.21.15.60:0/559253033 shutdown_connections 2024-04-25T10:06:26.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f464b6000 1 -- 172.21.15.60:0/559253033 wait complete. 2024-04-25T10:06:26.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.959+0000 7f6f464b6000 20 client.5478 trim_cache size 0 max 16384 2024-04-25T10:06:26.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 Processor -- start 2024-04-25T10:06:26.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 -- start start 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cabccf30 0x5578cac72270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac16920 0x5578cac727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cac17240 0x5578cac72cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c1ff4c0 con 0x5578cac16920 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c0ad1b0 con 0x5578cabccf30 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c1ff0c0 con 0x5578cac17240 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f408e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac16920 0x5578cac727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f408e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac16920 0x5578cac727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37866/0 (socket says 172.21.15.60:37866) 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f408e4640 1 -- 172.21.15.60:0/929665488 learned_addr learned my addr 172.21.15.60:0/929665488 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f408e4640 1 -- 172.21.15.60:0/929665488 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cac17240 msgr2=0x5578cac72cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f408e4640 1 --2- 172.21.15.60:0/929665488 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cac17240 0x5578cac72cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f408e4640 1 -- 172.21.15.60:0/929665488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cabccf30 msgr2=0x5578cac72270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f408e4640 1 --2- 172.21.15.60:0/929665488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cabccf30 0x5578cac72270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f408e4640 1 -- 172.21.15.60:0/929665488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578cac17070 con 0x5578cac16920 2024-04-25T10:06:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f408e4640 1 --2- 172.21.15.60:0/929665488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac16920 0x5578cac727b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c01d310 tx=0x7f6f2c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11030082f42181f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f267fc640 1 -- 172.21.15.60:0/929665488 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0f3780 con 0x5578cac16920 2024-04-25T10:06:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f267fc640 1 -- 172.21.15.60:0/929665488 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c0f7d40 con 0x5578cac16920 2024-04-25T10:06:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f267fc640 1 -- 172.21.15.60:0/929665488 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0ed930 con 0x5578cac16920 2024-04-25T10:06:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 -- 172.21.15.60:0/929665488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac16920 msgr2=0x5578cac727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 --2- 172.21.15.60:0/929665488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac16920 0x5578cac727b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c01d310 tx=0x7f6f2c009190 comp rx=0 tx=0).stop 2024-04-25T10:06:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 -- 172.21.15.60:0/929665488 shutdown_connections 2024-04-25T10:06:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 --2- 172.21.15.60:0/929665488 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cac17240 0x5578cac72cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 --2- 172.21.15.60:0/929665488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac16920 0x5578cac727b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 --2- 172.21.15.60:0/929665488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cabccf30 0x5578cac72270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 -- 172.21.15.60:0/929665488 >> 172.21.15.60:0/929665488 conn(0x5578cabd32a0 msgr2=0x5578cabd0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 -- 172.21.15.60:0/929665488 shutdown_connections 2024-04-25T10:06:26.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 -- 172.21.15.60:0/929665488 wait complete. 2024-04-25T10:06:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 -- start start 2024-04-25T10:06:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.975+0000 7f6f464b6000 1 Processor -- start 2024-04-25T10:06:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cabccf30 0x5578cac77500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac16920 0x5578cac77a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cac17240 0x5578cac73b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578caa203d0 con 0x5578cac16920 2024-04-25T10:06:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578caad7d50 con 0x5578cac17240 2024-04-25T10:06:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5578ca9b6980 con 0x5578cabccf30 2024-04-25T10:06:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 10 client.0 _ll_register_callbacks cb 0x7f6f34009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f408e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac16920 0x5578cac77a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f408e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac16920 0x5578cac77a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37878/0 (socket says 172.21.15.60:37878) 2024-04-25T10:06:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f408e4640 1 -- 172.21.15.60:0/3947584922 learned_addr learned my addr 172.21.15.60:0/3947584922 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f410e5640 1 --2- 172.21.15.60:0/3947584922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cabccf30 0x5578cac77500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f418e6640 1 --2- 172.21.15.60:0/3947584922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cac17240 0x5578cac73b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f408e4640 1 -- 172.21.15.60:0/3947584922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cabccf30 msgr2=0x5578cac77500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f408e4640 1 --2- 172.21.15.60:0/3947584922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cabccf30 0x5578cac77500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f408e4640 1 -- 172.21.15.60:0/3947584922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cac17240 msgr2=0x5578cac73b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f408e4640 1 --2- 172.21.15.60:0/3947584922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cac17240 0x5578cac73b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f408e4640 1 -- 172.21.15.60:0/3947584922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578caa9d600 con 0x5578cac16920 2024-04-25T10:06:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f408e4640 1 --2- 172.21.15.60:0/3947584922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac16920 0x5578cac77a40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0ef5c0 tx=0x7f6f2c11d120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4196f30e3403b5ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:06:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f1bfff640 1 -- 172.21.15.60:0/3947584922 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0043f0 con 0x5578cac16920 2024-04-25T10:06:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f1bfff640 1 -- 172.21.15.60:0/3947584922 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f2c0ed930 con 0x5578cac16920 2024-04-25T10:06:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 -- 172.21.15.60:0/3947584922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f2c0f7d40 con 0x5578cac16920 2024-04-25T10:06:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f1bfff640 1 -- 172.21.15.60:0/3947584922 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f2c0f78b0 con 0x5578cac16920 2024-04-25T10:06:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 10 client.5484 fetch_fsmap learned FSMap version 15 2024-04-25T10:06:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 -- 172.21.15.60:0/3947584922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x5578ca9b09b0 con 0x5578cac16920 2024-04-25T10:06:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f1bfff640 1 -- 172.21.15.60:0/3947584922 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c0f7d40 con 0x5578cac16920 2024-04-25T10:06:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f1bfff640 1 -- 172.21.15.60:0/3947584922 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6f2c0b5030 con 0x5578cac16920 2024-04-25T10:06:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 10 client.5484 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T10:06:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 10 client.5484 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:06:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 20 client.5484 populate_metadata read hostname 'smithi060' 2024-04-25T10:06:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 --2- 172.21.15.60:0/3947584922 >> [v2:172.21.15.60:6834/3681296171,v1:172.21.15.60:6835/3681296171] conn(0x5578cac74070 0x5578cac74450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 4 client.5484 mds_command: new command op to 5283 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:06:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 -- 172.21.15.60:0/3947584922 --> [v2:172.21.15.60:6834/3681296171,v1:172.21.15.60:6835/3681296171] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5578caa292e0 con 0x5578cac74070 2024-04-25T10:06:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f410e5640 1 --2- 172.21.15.60:0/3947584922 >> [v2:172.21.15.60:6834/3681296171,v1:172.21.15.60:6835/3681296171] conn(0x5578cac74070 0x5578cac74450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f1bfff640 10 client.5484 ms_handle_connect on v2:172.21.15.60:6834/3681296171 2024-04-25T10:06:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f410e5640 1 --2- 172.21.15.60:0/3947584922 >> [v2:172.21.15.60:6834/3681296171,v1:172.21.15.60:6835/3681296171] conn(0x5578cac74070 0x5578cac74450 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=386ec007fc48577f server_cookie=6c3e767740aedaac in_seq=0 out_seq=0 2024-04-25T10:06:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f1bfff640 1 -- 172.21.15.60:0/3947584922 <== mds.0 v2:172.21.15.60:6834/3681296171 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5578caa292e0 con 0x5578cac74070 2024-04-25T10:06:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f1bfff640 10 client.5484 handle_command_reply: tid=0 2024-04-25T10:06:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 client.5484 shutdown 2024-04-25T10:06:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 -- 172.21.15.60:0/3947584922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac16920 msgr2=0x5578cac77a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 --2- 172.21.15.60:0/3947584922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac16920 0x5578cac77a40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0ef5c0 tx=0x7f6f2c11d120 comp rx=0 tx=0).stop 2024-04-25T10:06:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 -- 172.21.15.60:0/3947584922 shutdown_connections 2024-04-25T10:06:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 --2- 172.21.15.60:0/3947584922 >> [v2:172.21.15.60:6834/3681296171,v1:172.21.15.60:6835/3681296171] conn(0x5578cac74070 0x5578cac74450 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 --2- 172.21.15.60:0/3947584922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5578cac17240 0x5578cac73b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 --2- 172.21.15.60:0/3947584922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5578cac16920 0x5578cac77a40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 --2- 172.21.15.60:0/3947584922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5578cabccf30 0x5578cac77500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f1bfff640 0 client.5484 ms_handle_reset on v2:172.21.15.60:6834/3681296171 2024-04-25T10:06:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 -- 172.21.15.60:0/3947584922 >> 172.21.15.60:0/3947584922 conn(0x5578cabd32a0 msgr2=0x5578cac1e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 -- 172.21.15.60:0/3947584922 shutdown_connections 2024-04-25T10:06:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 1 -- 172.21.15.60:0/3947584922 wait complete. 2024-04-25T10:06:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f464b6000 20 client.5484 trim_cache size 0 max 16384 2024-04-25T10:06:26.993 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:06:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "0ea93863-a079-45c4-abed-6e83540ea9fe", 2024-04-25T10:06:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T10:06:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5283, 2024-04-25T10:06:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T10:06:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T10:06:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T10:06:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 4.7844845720000002, 2024-04-25T10:06:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 15, 2024-04-25T10:06:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 35, 2024-04-25T10:06:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 35, 2024-04-25T10:06:26.994 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 11.720329157 2024-04-25T10:06:26.994 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:06:26.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f43349640 1 -- 172.21.15.60:0/2215521022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f20066f90 msgr2=0x7f6f20069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f43349640 1 --2- 172.21.15.60:0/2215521022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f20066f90 0x7f6f20069450 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6f38001ec0 tx=0x7f6f38009040 comp rx=0 tx=0).stop 2024-04-25T10:06:26.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f43349640 1 -- 172.21.15.60:0/2215521022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f3c14d060 msgr2=0x7f6f3c1502d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f43349640 1 --2- 172.21.15.60:0/2215521022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f3c14d060 0x7f6f3c1502d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0b0040 tx=0x7f6f28005840 comp rx=0 tx=0).stop 2024-04-25T10:06:26.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f43349640 1 -- 172.21.15.60:0/2215521022 shutdown_connections 2024-04-25T10:06:26.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f43349640 1 --2- 172.21.15.60:0/2215521022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f20066f90 0x7f6f20069450 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f43349640 1 --2- 172.21.15.60:0/2215521022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f3c14d4a0 0x7f6f3c150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f43349640 1 --2- 172.21.15.60:0/2215521022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f3c14d060 0x7f6f3c1502d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f43349640 1 --2- 172.21.15.60:0/2215521022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f28004810 0x7f6f3c1f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f43349640 1 -- 172.21.15.60:0/2215521022 >> 172.21.15.60:0/2215521022 conn(0x7f6f3c0b12d0 msgr2=0x7f6f3c0af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f43349640 1 -- 172.21.15.60:0/2215521022 shutdown_connections 2024-04-25T10:06:26.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:26.979+0000 7f6f43349640 1 -- 172.21.15.60:0/2215521022 wait complete. 2024-04-25T10:06:27.004 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:06:27.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81ad190640 1 Processor -- start 2024-04-25T10:06:27.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81ad190640 1 -- start start 2024-04-25T10:06:27.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81ad190640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f81a815a7a0 0x7f81a80cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81ad190640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f81a806ec30 0x7f81a8071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81ad190640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a80721e0 con 0x7f81a80b37e0 2024-04-25T10:06:27.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81ad190640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a80cdaa0 con 0x7f81a815a7a0 2024-04-25T10:06:27.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81ad190640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a806d2f0 con 0x7f81a806ec30 2024-04-25T10:06:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81a6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f81a815a7a0 0x7f81a80cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81a759e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f81a80b37e0 0x7f81a806dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45904/0 (socket says 172.21.15.60:45904) 2024-04-25T10:06:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81a6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f81a815a7a0 0x7f81a80cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37882/0 (socket says 172.21.15.60:37882) 2024-04-25T10:06:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81a759e640 1 -- 172.21.15.60:0/345380967 learned_addr learned my addr 172.21.15.60:0/345380967 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81a659c640 1 --2- 172.21.15.60:0/345380967 >> v2:172.21.15.100:3300/0 conn(0x7f81a806ec30 0x7f81a8071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81a6d9d640 1 -- 172.21.15.60:0/345380967 >> v2:172.21.15.100:3300/0 conn(0x7f81a806ec30 msgr2=0x7f81a8071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81a6d9d640 1 --2- 172.21.15.60:0/345380967 >> v2:172.21.15.100:3300/0 conn(0x7f81a806ec30 0x7f81a8071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81a6d9d640 1 -- 172.21.15.60:0/345380967 >> v1:172.21.15.60:6789/0 conn(0x7f81a80b37e0 legacy=0x7f81a806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81a5d9b640 1 -- 172.21.15.60:0/345380967 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 781167524 0 0) 0x7f81a80721e0 con 0x7f81a80b37e0 2024-04-25T10:06:27.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.111+0000 7f81a6d9d640 1 -- 172.21.15.60:0/345380967 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a806f130 con 0x7f81a815a7a0 2024-04-25T10:06:27.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81a6d9d640 1 --2- 172.21.15.60:0/345380967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81a815a7a0 0x7f81a80cd560 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f81a8072bf0 tx=0x7f819801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ab227b0e1f391b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:27.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81a5d9b640 1 -- 172.21.15.60:0/345380967 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81980b1070 con 0x7f81a815a7a0 2024-04-25T10:06:27.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81a5d9b640 1 -- 172.21.15.60:0/345380967 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81980ac5f0 con 0x7f81a815a7a0 2024-04-25T10:06:27.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81a5d9b640 1 -- 172.21.15.60:0/345380967 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81980ac9c0 con 0x7f81a815a7a0 2024-04-25T10:06:27.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81ad190640 1 -- 172.21.15.60:0/345380967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81a815a7a0 msgr2=0x7f81a80cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81ad190640 1 --2- 172.21.15.60:0/345380967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81a815a7a0 0x7f81a80cd560 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f81a8072bf0 tx=0x7f819801c480 comp rx=0 tx=0).stop 2024-04-25T10:06:27.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81ad190640 1 -- 172.21.15.60:0/345380967 shutdown_connections 2024-04-25T10:06:27.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81ad190640 1 --2- 172.21.15.60:0/345380967 >> v2:172.21.15.100:3300/0 conn(0x7f81a806ec30 0x7f81a8071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81ad190640 1 --2- 172.21.15.60:0/345380967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81a815a7a0 0x7f81a80cd560 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81ad190640 1 -- 172.21.15.60:0/345380967 >> 172.21.15.60:0/345380967 conn(0x7f81a80b12d0 msgr2=0x7f81a80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:27.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81ad190640 1 -- 172.21.15.60:0/345380967 shutdown_connections 2024-04-25T10:06:27.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81ad190640 1 -- 172.21.15.60:0/345380967 wait complete. 2024-04-25T10:06:27.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81ad190640 1 Processor -- start 2024-04-25T10:06:27.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81ad190640 1 -- start start 2024-04-25T10:06:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81ad190640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81a806ec30 0x7f81a8152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81ad190640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81a80b37e0 0x7f81a8152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81ad190640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81a815a7a0 0x7f81a81555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81ad190640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a815ad00 con 0x7f81a815a7a0 2024-04-25T10:06:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81ad190640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a815ab80 con 0x7f81a806ec30 2024-04-25T10:06:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81ad190640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a806e860 con 0x7f81a80b37e0 2024-04-25T10:06:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81a759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81a815a7a0 0x7f81a81555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81a759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81a815a7a0 0x7f81a81555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37892/0 (socket says 172.21.15.60:37892) 2024-04-25T10:06:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81a759e640 1 -- 172.21.15.60:0/3255557952 learned_addr learned my addr 172.21.15.60:0/3255557952 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81a659c640 1 --2- 172.21.15.60:0/3255557952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81a80b37e0 0x7f81a8152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81a6d9d640 1 --2- 172.21.15.60:0/3255557952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81a806ec30 0x7f81a8152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81a759e640 1 -- 172.21.15.60:0/3255557952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81a80b37e0 msgr2=0x7f81a8152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81a759e640 1 --2- 172.21.15.60:0/3255557952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81a80b37e0 0x7f81a8152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81a759e640 1 -- 172.21.15.60:0/3255557952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81a806ec30 msgr2=0x7f81a8152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81a759e640 1 --2- 172.21.15.60:0/3255557952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81a806ec30 0x7f81a8152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.115+0000 7f81a759e640 1 -- 172.21.15.60:0/3255557952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a813adc0 con 0x7f81a815a7a0 2024-04-25T10:06:27.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.119+0000 7f81a759e640 1 --2- 172.21.15.60:0/3255557952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81a815a7a0 0x7f81a81555d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f81a000c7a0 tx=0x7f81a000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:27.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.119+0000 7f818f7fe640 1 -- 172.21.15.60:0/3255557952 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0018070 con 0x7f81a815a7a0 2024-04-25T10:06:27.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.119+0000 7f818f7fe640 1 -- 172.21.15.60:0/3255557952 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a000a640 con 0x7f81a815a7a0 2024-04-25T10:06:27.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.119+0000 7f81ad190640 1 -- 172.21.15.60:0/3255557952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a813b0a0 con 0x7f81a815a7a0 2024-04-25T10:06:27.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.119+0000 7f818f7fe640 1 -- 172.21.15.60:0/3255557952 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a000aab0 con 0x7f81a815a7a0 2024-04-25T10:06:27.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.119+0000 7f81ad190640 1 -- 172.21.15.60:0/3255557952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a813b660 con 0x7f81a815a7a0 2024-04-25T10:06:27.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.119+0000 7f81ad190640 1 -- 172.21.15.60:0/3255557952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a8074770 con 0x7f81a815a7a0 2024-04-25T10:06:27.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.119+0000 7f818f7fe640 1 -- 172.21.15.60:0/3255557952 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f81a001b070 con 0x7f81a815a7a0 2024-04-25T10:06:27.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.119+0000 7f818f7fe640 1 --2- 172.21.15.60:0/3255557952 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8194066f90 0x7f8194069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.119+0000 7f818f7fe640 1 -- 172.21.15.60:0/3255557952 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81a0084a10 con 0x7f81a815a7a0 2024-04-25T10:06:27.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.119+0000 7f81a6d9d640 1 --2- 172.21.15.60:0/3255557952 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8194066f90 0x7f8194069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.123+0000 7f81a6d9d640 1 --2- 172.21.15.60:0/3255557952 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8194066f90 0x7f8194069450 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f81980ae000 tx=0x7f8198008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:27.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.123+0000 7f818f7fe640 1 -- 172.21.15.60:0/3255557952 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f81a0055a00 con 0x7f81a815a7a0 2024-04-25T10:06:27.143 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-04-25T10:06:27.135+0000 7fc9741c4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:06:27.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.311+0000 7f81ad190640 1 -- 172.21.15.60:0/3255557952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f81a80cd880 con 0x7f81a815a7a0 2024-04-25T10:06:27.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.311+0000 7f818f7fe640 1 -- 172.21.15.60:0/3255557952 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f81a0055300 con 0x7f81a815a7a0 2024-04-25T10:06:27.319 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:06:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.315+0000 7f81ad190640 1 -- 172.21.15.60:0/3255557952 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8194066f90 msgr2=0x7f8194069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.315+0000 7f81ad190640 1 --2- 172.21.15.60:0/3255557952 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8194066f90 0x7f8194069450 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f81980ae000 tx=0x7f8198008040 comp rx=0 tx=0).stop 2024-04-25T10:06:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.315+0000 7f81ad190640 1 -- 172.21.15.60:0/3255557952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81a815a7a0 msgr2=0x7f81a81555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.315+0000 7f81ad190640 1 --2- 172.21.15.60:0/3255557952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81a815a7a0 0x7f81a81555d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f81a000c7a0 tx=0x7f81a000cc60 comp rx=0 tx=0).stop 2024-04-25T10:06:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.315+0000 7f81ad190640 1 -- 172.21.15.60:0/3255557952 shutdown_connections 2024-04-25T10:06:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.315+0000 7f81ad190640 1 --2- 172.21.15.60:0/3255557952 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8194066f90 0x7f8194069450 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.315+0000 7f81ad190640 1 --2- 172.21.15.60:0/3255557952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81a815a7a0 0x7f81a81555d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.315+0000 7f81ad190640 1 --2- 172.21.15.60:0/3255557952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81a80b37e0 0x7f81a8152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.315+0000 7f81ad190640 1 --2- 172.21.15.60:0/3255557952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81a806ec30 0x7f81a8152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.315+0000 7f81ad190640 1 -- 172.21.15.60:0/3255557952 >> 172.21.15.60:0/3255557952 conn(0x7f81a80b12d0 msgr2=0x7f81a815d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.315+0000 7f81ad190640 1 -- 172.21.15.60:0/3255557952 shutdown_connections 2024-04-25T10:06:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:27.315+0000 7f81ad190640 1 -- 172.21.15.60:0/3255557952 wait complete. 2024-04-25T10:06:27.336 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:06:27.336 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:06:27.336 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:06:27.336 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:06:27.336 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:27.336 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:06:27.336 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:06:27.336 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:06:27.337 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:27.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:27.337 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:06:27.343 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:06:27.343 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:06:27.343 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:06:27.343 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:27.343 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:06:27.343 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:27.343 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:06:27.343 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:27.344 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:06:27.344 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:06:27.344 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:27.344 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:06:27.344 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:06:27.344 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:27.344 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:06:27.344 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:06:27.344 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42470sec preferred_lft 42470sec 2024-04-25T10:06:27.344 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:06:27.344 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:27.344 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:06:27.344 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:27.344 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:06:27.344 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:06:27.344 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:27.345 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:06:27.345 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:27.345 INFO:teuthology.orchestra.run.smithi060.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:06:27.345 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:27.345 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:06:27.345 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:27.345 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:06:27.345 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:27.345 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:06:27.345 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:27.345 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:06:27.345 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:27.345 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:06:27.345 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:27.346 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:06:27.346 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:27.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:27.346 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:06:27.392 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:06:27.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:27.393 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:06:27.441 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:06:27.441 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:06:27.441 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:06:27.441 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:06:27.441 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:06:27.441 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:06:27.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:27.442 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:06:27.442 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:06:27.442 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:06:27.442 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:06:27.442 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:06:27.442 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:06:27.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:27.516 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:06:27.569 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:06:27.569 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:06:27.570 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:27.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:27.570 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T10:06:27.625 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:06:27.625 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:06:27.625 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:27.673 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:27.674 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:06:27.674 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:27.721 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:06:27.780 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:06:27.831 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:06:27.832 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:06:27.832 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:06:27.832 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:06:27.832 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:06:27.832 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:06:27.832 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:06:27.832 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:06:27.832 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:06:27.832 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:06:27.832 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:06:27.832 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:06:27.832 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:06:27.832 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:06:27.833 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:06:27.833 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:06:27.833 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:06:27.833 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:06:27.833 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:06:27.833 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:06:27.833 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:06:27.833 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:06:27.833 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:06:27.833 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:06:27.833 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:06:27.833 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:06:27.833 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:06:27.833 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:06:27.833 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:06:27.833 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:27.834 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:27.834 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:27.834 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:27.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:27.835 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:06:27.881 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:06:27.881 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:06:27.926 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:06:27.942 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:06:27.990 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:06:27.990 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:06:27.990 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:06:27.990 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:06:27.990 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:06:27.990 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:06:27.990 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:06:27.990 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:06:27.990 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:06:27.990 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:06:27.990 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:06:27.991 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:06:27.991 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:06:27.991 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:06:27.991 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:06:27.991 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:06:27.991 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:06:27.991 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:06:27.991 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:06:27.991 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:06:27.991 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:06:27.991 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:06:27.991 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:06:27.991 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:06:27.991 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:06:27.991 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:06:27.992 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:06:27.992 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:06:27.992 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:06:27.992 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:27.992 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:27.992 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:27.992 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:27.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:27.992 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:06:28.013 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2024-04-25T10:06:28.007+0000 7f0f01e92580 -1 init, newargv = 0x558b48b02570 newargc=15 2024-04-25T10:06:28.013 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[36740]: starting ceph client 2024-04-25T10:06:28.018 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[36740]: starting fuse 2024-04-25T10:06:29.119 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:06:29.137 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:06:29.188 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:06:29.188 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:06:29.188 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:06:29.188 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:06:29.188 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:06:29.188 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:06:29.188 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:06:29.188 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:06:29.188 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:06:29.189 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:06:29.189 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:06:29.189 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:06:29.189 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:06:29.189 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:06:29.189 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:06:29.189 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:06:29.189 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:06:29.189 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:06:29.189 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:06:29.189 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:06:29.189 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:06:29.189 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:06:29.189 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:06:29.189 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:06:29.190 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:06:29.190 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:06:29.190 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:06:29.190 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:06:29.190 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:06:29.190 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:29.190 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:29.190 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:29.190 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:29.190 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:29.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:29.191 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:06:29.241 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:06:29.242 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:06:29.242 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:06:29.242 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:06:29.242 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:06:29.242 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:06:29.242 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:06:29.242 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:06:29.243 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:06:29.244 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:06:29.244 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2024-04-25T10:06:29.244 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:06:29.368 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.36740.asok 2024-04-25T10:06:29.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:29.380 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36740.asok 2024-04-25T10:06:29.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:29.381 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36740.asok status 2024-04-25T10:06:29.542 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:06:29.542 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:06:29.543 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:06:29.543 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:06:29.543 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2024-04-25T10:06:29.543 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:06:29.543 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:06:29.543 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "36740", 2024-04-25T10:06:29.543 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:06:29.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:29.543 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:06:29.543 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:06:29.543 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5499, 2024-04-25T10:06:29.543 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:06:29.543 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:06:29.543 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:06:29.544 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 5499 2024-04-25T10:06:29.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:29.544 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:06:29.544 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:06:29.544 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:06:29.544 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4162357461 2024-04-25T10:06:29.544 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:06:29.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:29.544 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:06:29.544 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:06:29.544 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:06:29.544 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4162357461 2024-04-25T10:06:29.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:06:29.544 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.5499 192.168.0.1:0/4162357461", 2024-04-25T10:06:29.544 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/4162357461", 2024-04-25T10:06:29.545 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:06:29.545 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 15, 2024-04-25T10:06:29.545 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 35, 2024-04-25T10:06:29.545 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:06:29.545 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:06:29.545 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:06:29.545 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:06:29.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:29.556 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:06:29.557 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:29.563 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:06:29.563 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:29.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:29.563 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:29.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:29.617 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-04-25T10:06:29.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:29.674 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-04-25T10:06:29.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:29.726 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-04-25T10:06:29.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:29.777 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-04-25T10:06:29.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:29.838 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-04-25T10:06:29.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:29.891 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:06:29.959 INFO:teuthology.orchestra.run.smithi060.stdout:./ 2024-04-25T10:06:29.959 INFO:teuthology.orchestra.run.smithi060.stdout:./rfile 2024-04-25T10:06:29.959 INFO:teuthology.orchestra.run.smithi060.stdout:./parentdir 2024-04-25T10:06:29.959 INFO:teuthology.orchestra.run.smithi060.stdout:./parentdir/pfile 2024-04-25T10:06:29.960 INFO:teuthology.orchestra.run.smithi060.stdout:./parentdir/childdir 2024-04-25T10:06:29.960 INFO:teuthology.orchestra.run.smithi060.stdout:./parentdir/childdir/cfile 2024-04-25T10:06:29.960 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:06:29.960 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:06:29.960 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:06:29.960 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:06:29.961 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:06:29.961 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:06:30.069 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:06:30.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:30.080 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:06:30.080 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:06:30.080 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:06:30.080 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:06:30.080 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-04-25T10:06:30.080 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:06:30.192 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627778 2024-04-25T10:06:30.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:30.202 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:06:30.202 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:06:30.203 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:06:30.203 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:06:30.203 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-04-25T10:06:30.203 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:06:30.316 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627776 2024-04-25T10:06:30.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:30.328 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:06:30.328 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:06:30.328 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:06:30.328 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:06:30.328 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-04-25T10:06:30.328 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:06:30.433 INFO:teuthology.orchestra.run.smithi060.stdout:1099511628279 2024-04-25T10:06:30.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:30.443 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:06:30.443 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:06:30.443 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:06:30.443 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:06:30.443 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-04-25T10:06:30.444 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:06:30.552 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627777 2024-04-25T10:06:30.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:30.562 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:06:30.563 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:06:30.563 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:06:30.563 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:06:30.563 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-04-25T10:06:30.563 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:06:30.668 INFO:teuthology.orchestra.run.smithi060.stdout:1099511628280 2024-04-25T10:06:30.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:30.679 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:06:30.679 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:30.726 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:30.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.895+0000 7f5d915eb640 1 Processor -- start 2024-04-25T10:06:30.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.895+0000 7f5d915eb640 1 -- start start 2024-04-25T10:06:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.895+0000 7f5d915eb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d8c0ce7a0 0x7f5d8c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.895+0000 7f5d915eb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8c0b33f0 con 0x7f5d8c15bc90 2024-04-25T10:06:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.895+0000 7f5d915eb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8c0cf0c0 con 0x7f5d8c0b4660 2024-04-25T10:06:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.895+0000 7f5d915eb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c0cc370 con 0x7f5d8c0ce7a0 2024-04-25T10:06:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d8a7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5d8c15bc90 0x7f5d8c0b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46066/0 (socket says 172.21.15.60:46066) 2024-04-25T10:06:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d8a7fc640 1 -- 172.21.15.60:0/213869821 learned_addr learned my addr 172.21.15.60:0/213869821 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d8affd640 1 --2- 172.21.15.60:0/213869821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d8c0ce7a0 0x7f5d8c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 -- 172.21.15.60:0/213869821 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2946803544 0 0) 0x7f5d8c0b33f0 con 0x7f5d8c15bc90 2024-04-25T10:06:30.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 -- 172.21.15.60:0/213869821 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d74003530 con 0x7f5d8c15bc90 2024-04-25T10:06:30.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 -- 172.21.15.60:0/213869821 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3613403465 0 0) 0x7f5d74003530 con 0x7f5d8c15bc90 2024-04-25T10:06:30.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 -- 172.21.15.60:0/213869821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d8c0ce7a0 msgr2=0x7f5d8c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 --2- 172.21.15.60:0/213869821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d8c0ce7a0 0x7f5d8c0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 -- 172.21.15.60:0/213869821 >> v1:172.21.15.100:6789/0 conn(0x7f5d8c0b4660 legacy=0x7f5d8c0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 -- 172.21.15.60:0/213869821 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c071ca0 con 0x7f5d8c15bc90 2024-04-25T10:06:30.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 -- 172.21.15.60:0/213869821 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f5d80001680 con 0x7f5d8c15bc90 2024-04-25T10:06:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 -- 172.21.15.60:0/213869821 >> v1:172.21.15.60:6789/0 conn(0x7f5d8c15bc90 legacy=0x7f5d8c0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 --2- 172.21.15.60:0/213869821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d740039f0 0x7f5d74003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 --2- 172.21.15.60:0/213869821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d8c0ce7a0 0x7f5d74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 --2- 172.21.15.60:0/213869821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d74004d50 0x7f5d740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 -- 172.21.15.60:0/213869821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7400ba80 con 0x7f5d74004d50 2024-04-25T10:06:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 -- 172.21.15.60:0/213869821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7400bc00 con 0x7f5d740039f0 2024-04-25T10:06:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 -- 172.21.15.60:0/213869821 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7400be00 con 0x7f5d8c0ce7a0 2024-04-25T10:06:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d8b7fe640 1 --2- 172.21.15.60:0/213869821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d74004d50 0x7f5d740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d8a7fc640 1 --2- 172.21.15.60:0/213869821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d8c0ce7a0 0x7f5d74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d8affd640 1 --2- 172.21.15.60:0/213869821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d740039f0 0x7f5d74003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d8b7fe640 1 -- 172.21.15.60:0/213869821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d8c0ce7a0 msgr2=0x7f5d74004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d8b7fe640 1 --2- 172.21.15.60:0/213869821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d8c0ce7a0 0x7f5d74004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d8b7fe640 1 -- 172.21.15.60:0/213869821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d740039f0 msgr2=0x7f5d74003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d8b7fe640 1 --2- 172.21.15.60:0/213869821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d740039f0 0x7f5d74003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d8b7fe640 1 -- 172.21.15.60:0/213869821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d74003530 con 0x7f5d74004d50 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d8b7fe640 1 --2- 172.21.15.60:0/213869821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d74004d50 0x7f5d740051d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c072c10 tx=0x7f5d8401c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2828f9ae974c12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 -- 172.21.15.60:0/213869821 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84001a90 con 0x7f5d74004d50 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 -- 172.21.15.60:0/213869821 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d84003b80 con 0x7f5d74004d50 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.899+0000 7f5d89ffb640 1 -- 172.21.15.60:0/213869821 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840050d0 con 0x7f5d74004d50 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d915eb640 1 -- 172.21.15.60:0/213869821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d74004d50 msgr2=0x7f5d740051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d915eb640 1 --2- 172.21.15.60:0/213869821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d74004d50 0x7f5d740051d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c072c10 tx=0x7f5d8401c700 comp rx=0 tx=0).stop 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d8b7fe640 1 -- 172.21.15.60:0/213869821 reap_dead start 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d915eb640 1 -- 172.21.15.60:0/213869821 shutdown_connections 2024-04-25T10:06:30.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d915eb640 1 -- 172.21.15.60:0/213869821 >> 172.21.15.60:0/213869821 conn(0x7f5d8c0b12d0 msgr2=0x7f5d8c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d915eb640 1 -- 172.21.15.60:0/213869821 shutdown_connections 2024-04-25T10:06:30.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d915eb640 1 -- 172.21.15.60:0/213869821 wait complete. 2024-04-25T10:06:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d915eb640 1 Processor -- start 2024-04-25T10:06:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d915eb640 1 -- start start 2024-04-25T10:06:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d915eb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d74004d50 0x7f5d8c1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d915eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d8c151bf0 0x7f5d8c1f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d915eb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d8c1f6ea0 0x7f5d8c0ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d915eb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c071f80 con 0x7f5d74004d50 2024-04-25T10:06:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d915eb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c0ccd20 con 0x7f5d8c151bf0 2024-04-25T10:06:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d915eb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c0b3ba0 con 0x7f5d8c1f6ea0 2024-04-25T10:06:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d8affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d74004d50 0x7f5d8c1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d8affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d74004d50 0x7f5d8c1546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50022/0 (socket says 172.21.15.60:50022) 2024-04-25T10:06:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d8affd640 1 -- 172.21.15.60:0/77672237 learned_addr learned my addr 172.21.15.60:0/77672237 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d8b7fe640 1 --2- 172.21.15.60:0/77672237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d8c1f6ea0 0x7f5d8c0ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d8a7fc640 1 --2- 172.21.15.60:0/77672237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d8c151bf0 0x7f5d8c1f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d8affd640 1 -- 172.21.15.60:0/77672237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d8c1f6ea0 msgr2=0x7f5d8c0ad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d8affd640 1 --2- 172.21.15.60:0/77672237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d8c1f6ea0 0x7f5d8c0ad750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d8affd640 1 -- 172.21.15.60:0/77672237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d8c151bf0 msgr2=0x7f5d8c1f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d8affd640 1 --2- 172.21.15.60:0/77672237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d8c151bf0 0x7f5d8c1f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d8affd640 1 -- 172.21.15.60:0/77672237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c153600 con 0x7f5d74004d50 2024-04-25T10:06:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d8a7fc640 1 --2- 172.21.15.60:0/77672237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d8c151bf0 0x7f5d8c1f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d8b7fe640 1 --2- 172.21.15.60:0/77672237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d8c1f6ea0 0x7f5d8c0ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d8affd640 1 --2- 172.21.15.60:0/77672237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d74004d50 0x7f5d8c1546a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c011640 tx=0x7f5d7c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d737fe640 1 -- 172.21.15.60:0/77672237 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c00f920 con 0x7f5d74004d50 2024-04-25T10:06:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d737fe640 1 -- 172.21.15.60:0/77672237 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c00fac0 con 0x7f5d74004d50 2024-04-25T10:06:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d737fe640 1 -- 172.21.15.60:0/77672237 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c00fe90 con 0x7f5d74004d50 2024-04-25T10:06:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d915eb640 1 -- 172.21.15.60:0/77672237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8c1538e0 con 0x7f5d74004d50 2024-04-25T10:06:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.903+0000 7f5d915eb640 1 -- 172.21.15.60:0/77672237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c153ea0 con 0x7f5d74004d50 2024-04-25T10:06:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.907+0000 7f5d915eb640 1 -- 172.21.15.60:0/77672237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d8c074770 con 0x7f5d74004d50 2024-04-25T10:06:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.907+0000 7f5d737fe640 1 -- 172.21.15.60:0/77672237 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d7c015460 con 0x7f5d74004d50 2024-04-25T10:06:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.907+0000 7f5d737fe640 1 --2- 172.21.15.60:0/77672237 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5d78067130 0x7f5d780695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.907+0000 7f5d737fe640 1 -- 172.21.15.60:0/77672237 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d7c085e90 con 0x7f5d74004d50 2024-04-25T10:06:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.907+0000 7f5d8a7fc640 1 --2- 172.21.15.60:0/77672237 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5d78067130 0x7f5d780695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.907+0000 7f5d8a7fc640 1 --2- 172.21.15.60:0/77672237 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5d78067130 0x7f5d780695f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0b38f0 tx=0x7f5d80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:30.911+0000 7f5d737fe640 1 -- 172.21.15.60:0/77672237 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5d7c019030 con 0x7f5d74004d50 2024-04-25T10:06:31.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.071+0000 7f5d915eb640 1 -- 172.21.15.60:0/77672237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d8c1f7280 con 0x7f5d74004d50 2024-04-25T10:06:31.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.075+0000 7f5d737fe640 1 -- 172.21.15.60:0/77672237 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16681 (secure 0 0 0) 0x7f5d7c056e80 con 0x7f5d74004d50 2024-04-25T10:06:31.080 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:31.080 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":35,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:23.196554+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:20.088165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:20.507719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:31.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.075+0000 7f5d915eb640 1 -- 172.21.15.60:0/77672237 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5d78067130 msgr2=0x7f5d780695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.075+0000 7f5d915eb640 1 --2- 172.21.15.60:0/77672237 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5d78067130 0x7f5d780695f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0b38f0 tx=0x7f5d80009040 comp rx=0 tx=0).stop 2024-04-25T10:06:31.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.075+0000 7f5d915eb640 1 -- 172.21.15.60:0/77672237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d74004d50 msgr2=0x7f5d8c1546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.075+0000 7f5d915eb640 1 --2- 172.21.15.60:0/77672237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d74004d50 0x7f5d8c1546a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c011640 tx=0x7f5d7c011b00 comp rx=0 tx=0).stop 2024-04-25T10:06:31.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.075+0000 7f5d915eb640 1 -- 172.21.15.60:0/77672237 shutdown_connections 2024-04-25T10:06:31.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.075+0000 7f5d915eb640 1 --2- 172.21.15.60:0/77672237 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5d78067130 0x7f5d780695f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.075+0000 7f5d915eb640 1 --2- 172.21.15.60:0/77672237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d8c1f6ea0 0x7f5d8c0ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.075+0000 7f5d915eb640 1 --2- 172.21.15.60:0/77672237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d8c151bf0 0x7f5d8c1f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.075+0000 7f5d915eb640 1 --2- 172.21.15.60:0/77672237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d74004d50 0x7f5d8c1546a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.075+0000 7f5d915eb640 1 -- 172.21.15.60:0/77672237 >> 172.21.15.60:0/77672237 conn(0x7f5d8c0b12d0 msgr2=0x7f5d8c15e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.075+0000 7f5d915eb640 1 -- 172.21.15.60:0/77672237 shutdown_connections 2024-04-25T10:06:31.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.075+0000 7f5d915eb640 1 -- 172.21.15.60:0/77672237 wait complete. 2024-04-25T10:06:31.100 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:20.088165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:20.507719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:31.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:31.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe932c86640 1 Processor -- start 2024-04-25T10:06:31.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe932c86640 1 -- start start 2024-04-25T10:06:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe932c86640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe92c15a5f0 0x7fe92c06eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe932c86640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92c15e270 con 0x7fe92c06c330 2024-04-25T10:06:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe932c86640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92c0b4410 con 0x7fe92c06f420 2024-04-25T10:06:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe932c86640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92c0b46f0 con 0x7fe92c15a5f0 2024-04-25T10:06:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe931223640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe92c06c330 0x7fe92c15e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46074/0 (socket says 172.21.15.60:46074) 2024-04-25T10:06:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe931223640 1 -- 172.21.15.60:0/451855443 learned_addr learned my addr 172.21.15.60:0/451855443 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe930a22640 1 --2- 172.21.15.60:0/451855443 >> v2:172.21.15.100:3300/0 conn(0x7fe92c15a5f0 0x7fe92c06eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92b7fe640 1 -- 172.21.15.60:0/451855443 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228664360 0 0) 0x7fe92c15e270 con 0x7fe92c06c330 2024-04-25T10:06:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92b7fe640 1 -- 172.21.15.60:0/451855443 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe914003530 con 0x7fe92c06c330 2024-04-25T10:06:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92b7fe640 1 -- 172.21.15.60:0/451855443 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1557201535 0 0) 0x7fe914003530 con 0x7fe92c06c330 2024-04-25T10:06:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92b7fe640 1 -- 172.21.15.60:0/451855443 >> v2:172.21.15.100:3300/0 conn(0x7fe92c15a5f0 msgr2=0x7fe92c06eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92b7fe640 1 --2- 172.21.15.60:0/451855443 >> v2:172.21.15.100:3300/0 conn(0x7fe92c15a5f0 0x7fe92c06eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92b7fe640 1 -- 172.21.15.60:0/451855443 >> v1:172.21.15.100:6789/0 conn(0x7fe92c06f420 legacy=0x7fe92c06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92b7fe640 1 -- 172.21.15.60:0/451855443 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c0b49d0 con 0x7fe92c06c330 2024-04-25T10:06:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92b7fe640 1 -- 172.21.15.60:0/451855443 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fe924001a00 con 0x7fe92c06c330 2024-04-25T10:06:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92b7fe640 1 -- 172.21.15.60:0/451855443 >> v1:172.21.15.60:6789/0 conn(0x7fe92c06c330 legacy=0x7fe92c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92b7fe640 1 --2- 172.21.15.60:0/451855443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9140039c0 0x7fe914003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92b7fe640 1 --2- 172.21.15.60:0/451855443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914004810 0x7fe914004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92b7fe640 1 --2- 172.21.15.60:0/451855443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9140051d0 0x7fe91400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92b7fe640 1 -- 172.21.15.60:0/451855443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0b46f0 con 0x7fe914004810 2024-04-25T10:06:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92b7fe640 1 -- 172.21.15.60:0/451855443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91400c070 con 0x7fe9140039c0 2024-04-25T10:06:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92b7fe640 1 -- 172.21.15.60:0/451855443 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91400c270 con 0x7fe9140051d0 2024-04-25T10:06:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92bfff640 1 --2- 172.21.15.60:0/451855443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914004810 0x7fe914004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe930a22640 1 --2- 172.21.15.60:0/451855443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9140039c0 0x7fe914003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe931223640 1 --2- 172.21.15.60:0/451855443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9140051d0 0x7fe91400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92bfff640 1 -- 172.21.15.60:0/451855443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9140051d0 msgr2=0x7fe91400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92bfff640 1 --2- 172.21.15.60:0/451855443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9140051d0 0x7fe91400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92bfff640 1 -- 172.21.15.60:0/451855443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9140039c0 msgr2=0x7fe914003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92bfff640 1 --2- 172.21.15.60:0/451855443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9140039c0 0x7fe914003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe92bfff640 1 -- 172.21.15.60:0/451855443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe914003530 con 0x7fe914004810 2024-04-25T10:06:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe931223640 1 --2- 172.21.15.60:0/451855443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9140051d0 0x7fe91400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.219+0000 7fe931223640 1 -- 172.21.15.60:0/451855443 reap_dead start 2024-04-25T10:06:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.223+0000 7fe92bfff640 1 --2- 172.21.15.60:0/451855443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914004810 0x7fe914004c90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe920001780 tx=0x7fe92001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5b7ef13869be1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.223+0000 7fe92b7fe640 1 -- 172.21.15.60:0/451855443 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9200013a0 con 0x7fe914004810 2024-04-25T10:06:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.223+0000 7fe92b7fe640 1 -- 172.21.15.60:0/451855443 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9200052f0 con 0x7fe914004810 2024-04-25T10:06:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.223+0000 7fe92b7fe640 1 -- 172.21.15.60:0/451855443 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920003840 con 0x7fe914004810 2024-04-25T10:06:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.223+0000 7fe932c86640 1 -- 172.21.15.60:0/451855443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914004810 msgr2=0x7fe914004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.223+0000 7fe932c86640 1 --2- 172.21.15.60:0/451855443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914004810 0x7fe914004c90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe920001780 tx=0x7fe92001c400 comp rx=0 tx=0).stop 2024-04-25T10:06:31.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.223+0000 7fe932c86640 1 -- 172.21.15.60:0/451855443 shutdown_connections 2024-04-25T10:06:31.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.223+0000 7fe932c86640 1 --2- 172.21.15.60:0/451855443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914004810 0x7fe914004c90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.223+0000 7fe932c86640 1 -- 172.21.15.60:0/451855443 >> 172.21.15.60:0/451855443 conn(0x7fe92c071b90 msgr2=0x7fe92c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.223+0000 7fe932c86640 1 -- 172.21.15.60:0/451855443 shutdown_connections 2024-04-25T10:06:31.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.223+0000 7fe932c86640 1 -- 172.21.15.60:0/451855443 wait complete. 2024-04-25T10:06:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.223+0000 7fe932c86640 1 Processor -- start 2024-04-25T10:06:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.223+0000 7fe932c86640 1 -- start start 2024-04-25T10:06:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe932c86640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914004810 0x7fe92c0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe932c86640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe92c0b11f0 0x7fe92c14cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe932c86640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe92c0b2c70 0x7fe92c0b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe932c86640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c15ab50 con 0x7fe914004810 2024-04-25T10:06:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe932c86640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c15a9d0 con 0x7fe92c0b11f0 2024-04-25T10:06:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe932c86640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0b3fa0 con 0x7fe92c0b2c70 2024-04-25T10:06:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe930a22640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914004810 0x7fe92c0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe930a22640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914004810 0x7fe92c0c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50042/0 (socket says 172.21.15.60:50042) 2024-04-25T10:06:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe930a22640 1 -- 172.21.15.60:0/80619220 learned_addr learned my addr 172.21.15.60:0/80619220 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe931223640 1 --2- 172.21.15.60:0/80619220 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe92c0b2c70 0x7fe92c0b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe92bfff640 1 --2- 172.21.15.60:0/80619220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe92c0b11f0 0x7fe92c14cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe930a22640 1 -- 172.21.15.60:0/80619220 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe92c0b2c70 msgr2=0x7fe92c0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe930a22640 1 --2- 172.21.15.60:0/80619220 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe92c0b2c70 0x7fe92c0b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe930a22640 1 -- 172.21.15.60:0/80619220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe92c0b11f0 msgr2=0x7fe92c14cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe930a22640 1 --2- 172.21.15.60:0/80619220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe92c0b11f0 0x7fe92c14cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe930a22640 1 -- 172.21.15.60:0/80619220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c0c9940 con 0x7fe914004810 2024-04-25T10:06:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe930a22640 1 --2- 172.21.15.60:0/80619220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914004810 0x7fe92c0c9110 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0116c0 tx=0x7fe91c011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe9297fa640 1 -- 172.21.15.60:0/80619220 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c00f960 con 0x7fe914004810 2024-04-25T10:06:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe9297fa640 1 -- 172.21.15.60:0/80619220 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91c00fb00 con 0x7fe914004810 2024-04-25T10:06:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe9297fa640 1 -- 172.21.15.60:0/80619220 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c015460 con 0x7fe914004810 2024-04-25T10:06:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe932c86640 1 -- 172.21.15.60:0/80619220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92c0c9c20 con 0x7fe914004810 2024-04-25T10:06:31.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe932c86640 1 -- 172.21.15.60:0/80619220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c0ca1e0 con 0x7fe914004810 2024-04-25T10:06:31.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe932c86640 1 -- 172.21.15.60:0/80619220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92c074770 con 0x7fe914004810 2024-04-25T10:06:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe9297fa640 1 -- 172.21.15.60:0/80619220 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe91c015600 con 0x7fe914004810 2024-04-25T10:06:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe9297fa640 1 --2- 172.21.15.60:0/80619220 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe918066f60 0x7fe918069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.227+0000 7fe9297fa640 1 -- 172.21.15.60:0/80619220 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe91c0858f0 con 0x7fe914004810 2024-04-25T10:06:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.231+0000 7fe92bfff640 1 --2- 172.21.15.60:0/80619220 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe918066f60 0x7fe918069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.231+0000 7fe92bfff640 1 --2- 172.21.15.60:0/80619220 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe918066f60 0x7fe918069420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe920004000 tx=0x7fe9200011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.231+0000 7fe9297fa640 1 -- 172.21.15.60:0/80619220 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe91c019030 con 0x7fe914004810 2024-04-25T10:06:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.395+0000 7fe932c86640 1 -- 172.21.15.60:0/80619220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe92c0b3630 con 0x7fe914004810 2024-04-25T10:06:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.395+0000 7fe9297fa640 1 -- 172.21.15.60:0/80619220 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe91c056990 con 0x7fe914004810 2024-04-25T10:06:31.402 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:06:31.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.399+0000 7fe932c86640 1 -- 172.21.15.60:0/80619220 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe918066f60 msgr2=0x7fe918069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.399+0000 7fe932c86640 1 --2- 172.21.15.60:0/80619220 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe918066f60 0x7fe918069420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe920004000 tx=0x7fe9200011a0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.399+0000 7fe932c86640 1 -- 172.21.15.60:0/80619220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914004810 msgr2=0x7fe92c0c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.399+0000 7fe932c86640 1 --2- 172.21.15.60:0/80619220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914004810 0x7fe92c0c9110 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0116c0 tx=0x7fe91c011b80 comp rx=0 tx=0).stop 2024-04-25T10:06:31.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.399+0000 7fe932c86640 1 -- 172.21.15.60:0/80619220 shutdown_connections 2024-04-25T10:06:31.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.399+0000 7fe932c86640 1 --2- 172.21.15.60:0/80619220 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe918066f60 0x7fe918069420 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.399+0000 7fe932c86640 1 --2- 172.21.15.60:0/80619220 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe92c0b2c70 0x7fe92c0b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.399+0000 7fe932c86640 1 --2- 172.21.15.60:0/80619220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe92c0b11f0 0x7fe92c14cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.399+0000 7fe932c86640 1 --2- 172.21.15.60:0/80619220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914004810 0x7fe92c0c9110 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.399+0000 7fe932c86640 1 -- 172.21.15.60:0/80619220 >> 172.21.15.60:0/80619220 conn(0x7fe92c071b90 msgr2=0x7fe92c1599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.399+0000 7fe932c86640 1 -- 172.21.15.60:0/80619220 shutdown_connections 2024-04-25T10:06:31.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.399+0000 7fe932c86640 1 -- 172.21.15.60:0/80619220 wait complete. 2024-04-25T10:06:31.422 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:31.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.547+0000 7ff5be06f640 1 Processor -- start 2024-04-25T10:06:31.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.547+0000 7ff5be06f640 1 -- start start 2024-04-25T10:06:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.547+0000 7ff5be06f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff5b815b6f0 0x7ff5b81620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.547+0000 7ff5be06f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff5b815e0d0 0x7ff5b815e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.547+0000 7ff5be06f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b815bad0 con 0x7ff5b80ce7e0 2024-04-25T10:06:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.547+0000 7ff5be06f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b8162610 con 0x7ff5b815e0d0 2024-04-25T10:06:31.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.547+0000 7ff5be06f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b8161c40 con 0x7ff5b815b6f0 2024-04-25T10:06:31.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.547+0000 7ff5b6ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff5b815e0d0 0x7ff5b815e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.547+0000 7ff5b7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff5b80ce7e0 0x7ff5b80d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46082/0 (socket says 172.21.15.60:46082) 2024-04-25T10:06:31.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.547+0000 7ff5b7fff640 1 -- 172.21.15.60:0/4204709739 learned_addr learned my addr 172.21.15.60:0/4204709739 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:31.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.547+0000 7ff5b77fe640 1 --2- 172.21.15.60:0/4204709739 >> v2:172.21.15.100:3300/0 conn(0x7ff5b815b6f0 0x7ff5b81620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.547+0000 7ff5b6ffd640 1 -- 172.21.15.60:0/4204709739 >> v2:172.21.15.100:3300/0 conn(0x7ff5b815b6f0 msgr2=0x7ff5b81620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.547+0000 7ff5b6ffd640 1 --2- 172.21.15.60:0/4204709739 >> v2:172.21.15.100:3300/0 conn(0x7ff5b815b6f0 0x7ff5b81620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.547+0000 7ff5b6ffd640 1 -- 172.21.15.60:0/4204709739 >> v1:172.21.15.60:6789/0 conn(0x7ff5b80ce7e0 legacy=0x7ff5b80d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.547+0000 7ff5b6ffd640 1 -- 172.21.15.60:0/4204709739 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b815ebe0 con 0x7ff5b815e0d0 2024-04-25T10:06:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5b6ffd640 1 --2- 172.21.15.60:0/4204709739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b815e0d0 0x7ff5b815e510 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8073060 tx=0x7ff5ac01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff1dd0c6008a3b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5b67fc640 1 -- 172.21.15.60:0/4204709739 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0b0070 con 0x7ff5b815e0d0 2024-04-25T10:06:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5b67fc640 1 -- 172.21.15.60:0/4204709739 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ac0ab600 con 0x7ff5b815e0d0 2024-04-25T10:06:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5b67fc640 1 -- 172.21.15.60:0/4204709739 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0ab9d0 con 0x7ff5b815e0d0 2024-04-25T10:06:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5be06f640 1 -- 172.21.15.60:0/4204709739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b815e0d0 msgr2=0x7ff5b815e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5be06f640 1 --2- 172.21.15.60:0/4204709739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b815e0d0 0x7ff5b815e510 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8073060 tx=0x7ff5ac01c480 comp rx=0 tx=0).stop 2024-04-25T10:06:31.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5be06f640 1 -- 172.21.15.60:0/4204709739 shutdown_connections 2024-04-25T10:06:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5be06f640 1 --2- 172.21.15.60:0/4204709739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b815e0d0 0x7ff5b815e510 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5be06f640 1 --2- 172.21.15.60:0/4204709739 >> v2:172.21.15.100:3300/0 conn(0x7ff5b815b6f0 0x7ff5b81620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5be06f640 1 -- 172.21.15.60:0/4204709739 >> 172.21.15.60:0/4204709739 conn(0x7ff5b8072340 msgr2=0x7ff5b8072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5be06f640 1 -- 172.21.15.60:0/4204709739 shutdown_connections 2024-04-25T10:06:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5be06f640 1 -- 172.21.15.60:0/4204709739 wait complete. 2024-04-25T10:06:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5be06f640 1 Processor -- start 2024-04-25T10:06:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5be06f640 1 -- start start 2024-04-25T10:06:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5be06f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5b80ce7e0 0x7ff5b80cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5be06f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b815b6f0 0x7ff5b80cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5be06f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5b815e0d0 0x7ff5b8150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5be06f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80b96d0 con 0x7ff5b815b6f0 2024-04-25T10:06:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5be06f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80b9550 con 0x7ff5b80ce7e0 2024-04-25T10:06:31.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.551+0000 7ff5be06f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80cf000 con 0x7ff5b815e0d0 2024-04-25T10:06:31.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff5b6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b815b6f0 0x7ff5b80cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff5b6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b815b6f0 0x7ff5b80cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50060/0 (socket says 172.21.15.60:50060) 2024-04-25T10:06:31.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff5b6ffd640 1 -- 172.21.15.60:0/2509917404 learned_addr learned my addr 172.21.15.60:0/2509917404 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:31.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff5b7fff640 1 --2- 172.21.15.60:0/2509917404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5b815e0d0 0x7ff5b8150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff5b77fe640 1 --2- 172.21.15.60:0/2509917404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5b80ce7e0 0x7ff5b80cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff5b6ffd640 1 -- 172.21.15.60:0/2509917404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5b815e0d0 msgr2=0x7ff5b8150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff5b6ffd640 1 --2- 172.21.15.60:0/2509917404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5b815e0d0 0x7ff5b8150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff5b6ffd640 1 -- 172.21.15.60:0/2509917404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5b80ce7e0 msgr2=0x7ff5b80cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff5b6ffd640 1 --2- 172.21.15.60:0/2509917404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5b80ce7e0 0x7ff5b80cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff5b6ffd640 1 -- 172.21.15.60:0/2509917404 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b80cab80 con 0x7ff5b815b6f0 2024-04-25T10:06:31.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff5b6ffd640 1 --2- 172.21.15.60:0/2509917404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b815b6f0 0x7ff5b80cbc20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac01a0b0 tx=0x7ff5ac019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff59ffff640 1 -- 172.21.15.60:0/2509917404 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0b0040 con 0x7ff5b815b6f0 2024-04-25T10:06:31.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff59ffff640 1 -- 172.21.15.60:0/2509917404 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ac0ba070 con 0x7ff5b815b6f0 2024-04-25T10:06:31.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff5be06f640 1 -- 172.21.15.60:0/2509917404 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b80cae60 con 0x7ff5b815b6f0 2024-04-25T10:06:31.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff59ffff640 1 -- 172.21.15.60:0/2509917404 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0b4870 con 0x7ff5b815b6f0 2024-04-25T10:06:31.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff5be06f640 1 -- 172.21.15.60:0/2509917404 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b80cb420 con 0x7ff5b815b6f0 2024-04-25T10:06:31.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff5be06f640 1 -- 172.21.15.60:0/2509917404 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b8074770 con 0x7ff5b815b6f0 2024-04-25T10:06:31.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff59ffff640 1 -- 172.21.15.60:0/2509917404 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5ac0ab4c0 con 0x7ff5b815b6f0 2024-04-25T10:06:31.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff59ffff640 1 --2- 172.21.15.60:0/2509917404 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff5a4066f90 0x7ff5a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff59ffff640 1 -- 172.21.15.60:0/2509917404 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5ac11ed50 con 0x7ff5b815b6f0 2024-04-25T10:06:31.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff5b77fe640 1 --2- 172.21.15.60:0/2509917404 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff5a4066f90 0x7ff5a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.555+0000 7ff5b77fe640 1 --2- 172.21.15.60:0/2509917404 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff5a4066f90 0x7ff5a4069450 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8151680 tx=0x7ff5a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.559+0000 7ff59ffff640 1 -- 172.21.15.60:0/2509917404 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5ac122030 con 0x7ff5b815b6f0 2024-04-25T10:06:31.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.719+0000 7ff5be06f640 1 -- 172.21.15.60:0/2509917404 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff5b81540e0 con 0x7ff5b815b6f0 2024-04-25T10:06:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.719+0000 7ff59ffff640 1 -- 172.21.15.60:0/2509917404 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5ac016070 con 0x7ff5b815b6f0 2024-04-25T10:06:31.727 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:06:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.723+0000 7ff5be06f640 1 -- 172.21.15.60:0/2509917404 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff5a4066f90 msgr2=0x7ff5a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.723+0000 7ff5be06f640 1 --2- 172.21.15.60:0/2509917404 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff5a4066f90 0x7ff5a4069450 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8151680 tx=0x7ff5a8008040 comp rx=0 tx=0).stop 2024-04-25T10:06:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.723+0000 7ff5be06f640 1 -- 172.21.15.60:0/2509917404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b815b6f0 msgr2=0x7ff5b80cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.723+0000 7ff5be06f640 1 --2- 172.21.15.60:0/2509917404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b815b6f0 0x7ff5b80cbc20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac01a0b0 tx=0x7ff5ac019220 comp rx=0 tx=0).stop 2024-04-25T10:06:31.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.723+0000 7ff5be06f640 1 -- 172.21.15.60:0/2509917404 shutdown_connections 2024-04-25T10:06:31.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.723+0000 7ff5be06f640 1 --2- 172.21.15.60:0/2509917404 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff5a4066f90 0x7ff5a4069450 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.723+0000 7ff5be06f640 1 --2- 172.21.15.60:0/2509917404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5b815e0d0 0x7ff5b8150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.723+0000 7ff5be06f640 1 --2- 172.21.15.60:0/2509917404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5b815b6f0 0x7ff5b80cbc20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.723+0000 7ff5be06f640 1 --2- 172.21.15.60:0/2509917404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5b80ce7e0 0x7ff5b80cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.723+0000 7ff5be06f640 1 -- 172.21.15.60:0/2509917404 >> 172.21.15.60:0/2509917404 conn(0x7ff5b8072340 msgr2=0x7ff5b815f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.723+0000 7ff5be06f640 1 -- 172.21.15.60:0/2509917404 shutdown_connections 2024-04-25T10:06:31.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.723+0000 7ff5be06f640 1 -- 172.21.15.60:0/2509917404 wait complete. 2024-04-25T10:06:31.746 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:31.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.871+0000 7f436ce08640 1 Processor -- start 2024-04-25T10:06:31.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.871+0000 7f436ce08640 1 -- start start 2024-04-25T10:06:31.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.875+0000 7f436ce08640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f436806f030 0x7f43680b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.875+0000 7f436ce08640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f43680cf4f0 0x7f4368071250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.875+0000 7f436ce08640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f436815ad30 con 0x7f43680b3f60 2024-04-25T10:06:31.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.875+0000 7f436ce08640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f436815aaf0 con 0x7f43680cf4f0 2024-04-25T10:06:31.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.875+0000 7f436ce08640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436815b010 con 0x7f436806f030 2024-04-25T10:06:31.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.875+0000 7f4365d9b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f436806f030 0x7f43680b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.875+0000 7f4366d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f43680cf4f0 0x7f4368071250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.875+0000 7f436659c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f43680b3f60 0x7f436815a3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:49806/0 (socket says 172.21.15.60:49806) 2024-04-25T10:06:31.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.875+0000 7f436659c640 1 -- 172.21.15.60:0/2692219438 learned_addr learned my addr 172.21.15.60:0/2692219438 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:31.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.875+0000 7f436559a640 1 -- 172.21.15.60:0/2692219438 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987174611 0 0) 0x7f436815ad30 con 0x7f43680b3f60 2024-04-25T10:06:31.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.875+0000 7f436559a640 1 -- 172.21.15.60:0/2692219438 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4350003530 con 0x7f43680b3f60 2024-04-25T10:06:31.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.875+0000 7f4365d9b640 1 -- 172.21.15.60:0/2692219438 >> v2:172.21.15.100:3300/0 conn(0x7f43680cf4f0 msgr2=0x7f4368071250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.875+0000 7f4365d9b640 1 --2- 172.21.15.60:0/2692219438 >> v2:172.21.15.100:3300/0 conn(0x7f43680cf4f0 0x7f4368071250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.875+0000 7f4365d9b640 1 -- 172.21.15.60:0/2692219438 >> v1:172.21.15.100:6789/0 conn(0x7f43680b3f60 legacy=0x7f436815a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.875+0000 7f4365d9b640 1 -- 172.21.15.60:0/2692219438 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436815b2f0 con 0x7f436806f030 2024-04-25T10:06:31.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.875+0000 7f4365d9b640 1 --2- 172.21.15.60:0/2692219438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f436806f030 0x7f43680b2e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f435c0097c0 tx=0x7f435c01ce70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6e33df82668dc1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436559a640 1 -- 172.21.15.60:0/2692219438 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c0b0070 con 0x7f436806f030 2024-04-25T10:06:31.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436559a640 1 -- 172.21.15.60:0/2692219438 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435c0ab600 con 0x7f436806f030 2024-04-25T10:06:31.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436559a640 1 -- 172.21.15.60:0/2692219438 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c0ab9d0 con 0x7f436806f030 2024-04-25T10:06:31.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436ce08640 1 -- 172.21.15.60:0/2692219438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f436806f030 msgr2=0x7f43680b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436ce08640 1 --2- 172.21.15.60:0/2692219438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f436806f030 0x7f43680b2e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f435c0097c0 tx=0x7f435c01ce70 comp rx=0 tx=0).stop 2024-04-25T10:06:31.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436ce08640 1 -- 172.21.15.60:0/2692219438 shutdown_connections 2024-04-25T10:06:31.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436ce08640 1 --2- 172.21.15.60:0/2692219438 >> v2:172.21.15.100:3300/0 conn(0x7f43680cf4f0 0x7f4368071250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436ce08640 1 --2- 172.21.15.60:0/2692219438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f436806f030 0x7f43680b2e00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436ce08640 1 -- 172.21.15.60:0/2692219438 >> 172.21.15.60:0/2692219438 conn(0x7f43680670b0 msgr2=0x7f436806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436ce08640 1 -- 172.21.15.60:0/2692219438 shutdown_connections 2024-04-25T10:06:31.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436ce08640 1 -- 172.21.15.60:0/2692219438 wait complete. 2024-04-25T10:06:31.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436ce08640 1 Processor -- start 2024-04-25T10:06:31.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436ce08640 1 -- start start 2024-04-25T10:06:31.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436ce08640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f436806f030 0x7f436814fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436ce08640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43680b3f60 0x7f4368158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436ce08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43680cf4f0 0x7f4368156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436ce08640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436815bdb0 con 0x7f43680cf4f0 2024-04-25T10:06:31.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436ce08640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4368071ba0 con 0x7f43680b3f60 2024-04-25T10:06:31.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436ce08640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436815bf30 con 0x7f436806f030 2024-04-25T10:06:31.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f4366d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43680cf4f0 0x7f4368156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f4366d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43680cf4f0 0x7f4368156020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50062/0 (socket says 172.21.15.60:50062) 2024-04-25T10:06:31.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f4366d9d640 1 -- 172.21.15.60:0/3345914261 learned_addr learned my addr 172.21.15.60:0/3345914261 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:31.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f436659c640 1 --2- 172.21.15.60:0/3345914261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f436806f030 0x7f436814fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f4365d9b640 1 --2- 172.21.15.60:0/3345914261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43680b3f60 0x7f4368158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f4366d9d640 1 -- 172.21.15.60:0/3345914261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f436806f030 msgr2=0x7f436814fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f4366d9d640 1 --2- 172.21.15.60:0/3345914261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f436806f030 0x7f436814fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f4366d9d640 1 -- 172.21.15.60:0/3345914261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43680b3f60 msgr2=0x7f4368158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f4366d9d640 1 --2- 172.21.15.60:0/3345914261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43680b3f60 0x7f4368158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f4366d9d640 1 -- 172.21.15.60:0/3345914261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4368159820 con 0x7f43680cf4f0 2024-04-25T10:06:31.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.879+0000 7f4366d9d640 1 --2- 172.21.15.60:0/3345914261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43680cf4f0 0x7f4368156020 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f436000efc0 tx=0x7f436000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.883+0000 7f4356ffd640 1 -- 172.21.15.60:0/3345914261 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360018020 con 0x7f43680cf4f0 2024-04-25T10:06:31.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.883+0000 7f4356ffd640 1 -- 172.21.15.60:0/3345914261 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4360014480 con 0x7f43680cf4f0 2024-04-25T10:06:31.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.883+0000 7f436ce08640 1 -- 172.21.15.60:0/3345914261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4368159540 con 0x7f43680cf4f0 2024-04-25T10:06:31.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.883+0000 7f4356ffd640 1 -- 172.21.15.60:0/3345914261 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360014870 con 0x7f43680cf4f0 2024-04-25T10:06:31.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.883+0000 7f436ce08640 1 -- 172.21.15.60:0/3345914261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43680acc20 con 0x7f43680cf4f0 2024-04-25T10:06:31.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.883+0000 7f436ce08640 1 -- 172.21.15.60:0/3345914261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4368074770 con 0x7f43680cf4f0 2024-04-25T10:06:31.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.883+0000 7f4356ffd640 1 -- 172.21.15.60:0/3345914261 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f436001b020 con 0x7f43680cf4f0 2024-04-25T10:06:31.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.883+0000 7f4356ffd640 1 --2- 172.21.15.60:0/3345914261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4348066f60 0x7f4348069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.883+0000 7f4356ffd640 1 -- 172.21.15.60:0/3345914261 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43600861d0 con 0x7f43680cf4f0 2024-04-25T10:06:31.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.883+0000 7f436659c640 1 --2- 172.21.15.60:0/3345914261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4348066f60 0x7f4348069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.883+0000 7f436659c640 1 --2- 172.21.15.60:0/3345914261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4348066f60 0x7f4348069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4358005890 tx=0x7f4358009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:31.887+0000 7f4356ffd640 1 -- 172.21.15.60:0/3345914261 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4360022070 con 0x7f43680cf4f0 2024-04-25T10:06:32.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.047+0000 7f436ce08640 1 -- 172.21.15.60:0/3345914261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f436806ac10 con 0x7f43680cf4f0 2024-04-25T10:06:32.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.047+0000 7f4356ffd640 1 -- 172.21.15.60:0/3345914261 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4360010040 con 0x7f43680cf4f0 2024-04-25T10:06:32.053 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:06:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.047+0000 7f436ce08640 1 -- 172.21.15.60:0/3345914261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4348066f60 msgr2=0x7f4348069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.047+0000 7f436ce08640 1 --2- 172.21.15.60:0/3345914261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4348066f60 0x7f4348069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4358005890 tx=0x7f4358009040 comp rx=0 tx=0).stop 2024-04-25T10:06:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.047+0000 7f436ce08640 1 -- 172.21.15.60:0/3345914261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43680cf4f0 msgr2=0x7f4368156020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.047+0000 7f436ce08640 1 --2- 172.21.15.60:0/3345914261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43680cf4f0 0x7f4368156020 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f436000efc0 tx=0x7f436000c490 comp rx=0 tx=0).stop 2024-04-25T10:06:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.047+0000 7f436ce08640 1 -- 172.21.15.60:0/3345914261 shutdown_connections 2024-04-25T10:06:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.047+0000 7f436ce08640 1 --2- 172.21.15.60:0/3345914261 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4348066f60 0x7f4348069420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.047+0000 7f436ce08640 1 --2- 172.21.15.60:0/3345914261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43680cf4f0 0x7f4368156020 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.047+0000 7f436ce08640 1 --2- 172.21.15.60:0/3345914261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43680b3f60 0x7f4368158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.047+0000 7f436ce08640 1 --2- 172.21.15.60:0/3345914261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f436806f030 0x7f436814fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.047+0000 7f436ce08640 1 -- 172.21.15.60:0/3345914261 >> 172.21.15.60:0/3345914261 conn(0x7f43680670b0 msgr2=0x7f4368152d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.047+0000 7f436ce08640 1 -- 172.21.15.60:0/3345914261 shutdown_connections 2024-04-25T10:06:32.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.047+0000 7f436ce08640 1 -- 172.21.15.60:0/3345914261 wait complete. 2024-04-25T10:06:32.073 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:06:32.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43ec9b6640 1 Processor -- start 2024-04-25T10:06:32.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43ec9b6640 1 -- start start 2024-04-25T10:06:32.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43ec9b6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43e815bfa0 0x7f43e80d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43ec9b6640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e80716c0 con 0x7f43e80cefc0 2024-04-25T10:06:32.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43ec9b6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e80719a0 con 0x7f43e80d0ec0 2024-04-25T10:06:32.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43ec9b6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e815d4e0 con 0x7f43e815bfa0 2024-04-25T10:06:32.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e659c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f43e80cefc0 0x7f43e8070e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46090/0 (socket says 172.21.15.60:46090) 2024-04-25T10:06:32.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e659c640 1 -- 172.21.15.60:0/2298476057 learned_addr learned my addr 172.21.15.60:0/2298476057 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:32.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e5d9b640 1 --2- 172.21.15.60:0/2298476057 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43e815bfa0 0x7f43e80d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e559a640 1 -- 172.21.15.60:0/2298476057 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439765815 0 0) 0x7f43e80716c0 con 0x7f43e80cefc0 2024-04-25T10:06:32.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e559a640 1 -- 172.21.15.60:0/2298476057 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43d0003530 con 0x7f43e80cefc0 2024-04-25T10:06:32.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e559a640 1 -- 172.21.15.60:0/2298476057 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2127675288 0 0) 0x7f43d0003530 con 0x7f43e80cefc0 2024-04-25T10:06:32.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e559a640 1 -- 172.21.15.60:0/2298476057 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43e815bfa0 msgr2=0x7f43e80d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e559a640 1 --2- 172.21.15.60:0/2298476057 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43e815bfa0 0x7f43e80d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e559a640 1 -- 172.21.15.60:0/2298476057 >> v1:172.21.15.100:6789/0 conn(0x7f43e80d0ec0 legacy=0x7f43e815d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e559a640 1 -- 172.21.15.60:0/2298476057 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e8071c80 con 0x7f43e80cefc0 2024-04-25T10:06:32.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e559a640 1 -- 172.21.15.60:0/2298476057 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f43d8001680 con 0x7f43e80cefc0 2024-04-25T10:06:32.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e559a640 1 -- 172.21.15.60:0/2298476057 >> v1:172.21.15.60:6789/0 conn(0x7f43e80cefc0 legacy=0x7f43e8070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e559a640 1 --2- 172.21.15.60:0/2298476057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43d00039f0 0x7f43d0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e559a640 1 --2- 172.21.15.60:0/2298476057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d0004810 0x7f43d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e559a640 1 --2- 172.21.15.60:0/2298476057 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43e815bfa0 0x7f43d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e559a640 1 -- 172.21.15.60:0/2298476057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e815d4e0 con 0x7f43d00039f0 2024-04-25T10:06:32.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e559a640 1 -- 172.21.15.60:0/2298476057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d000ba80 con 0x7f43d0004810 2024-04-25T10:06:32.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e559a640 1 -- 172.21.15.60:0/2298476057 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d000bc80 con 0x7f43e815bfa0 2024-04-25T10:06:32.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e659c640 1 --2- 172.21.15.60:0/2298476057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43d00039f0 0x7f43d0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e5d9b640 1 --2- 172.21.15.60:0/2298476057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d0004810 0x7f43d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e6d9d640 1 --2- 172.21.15.60:0/2298476057 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43e815bfa0 0x7f43d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e659c640 1 -- 172.21.15.60:0/2298476057 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43e815bfa0 msgr2=0x7f43d00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e659c640 1 --2- 172.21.15.60:0/2298476057 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43e815bfa0 0x7f43d00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e659c640 1 -- 172.21.15.60:0/2298476057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d0004810 msgr2=0x7f43d0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e659c640 1 --2- 172.21.15.60:0/2298476057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d0004810 0x7f43d0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e659c640 1 -- 172.21.15.60:0/2298476057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d0003530 con 0x7f43d00039f0 2024-04-25T10:06:32.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.195+0000 7f43e6d9d640 1 --2- 172.21.15.60:0/2298476057 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43e815bfa0 0x7f43d00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.199+0000 7f43e659c640 1 --2- 172.21.15.60:0/2298476057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43d00039f0 0x7f43d0003e10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f43d8001ec0 tx=0x7f43d801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb84a52e34e1aa8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.199+0000 7f43e559a640 1 -- 172.21.15.60:0/2298476057 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d8007d40 con 0x7f43d00039f0 2024-04-25T10:06:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.199+0000 7f43e559a640 1 -- 172.21.15.60:0/2298476057 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d80a9070 con 0x7f43d00039f0 2024-04-25T10:06:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.199+0000 7f43e559a640 1 -- 172.21.15.60:0/2298476057 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d8005d40 con 0x7f43d00039f0 2024-04-25T10:06:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.199+0000 7f43ec9b6640 1 -- 172.21.15.60:0/2298476057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43d00039f0 msgr2=0x7f43d0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.199+0000 7f43ec9b6640 1 --2- 172.21.15.60:0/2298476057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43d00039f0 0x7f43d0003e10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f43d8001ec0 tx=0x7f43d801cc10 comp rx=0 tx=0).stop 2024-04-25T10:06:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.199+0000 7f43e6d9d640 1 -- 172.21.15.60:0/2298476057 reap_dead start 2024-04-25T10:06:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.199+0000 7f43ec9b6640 1 -- 172.21.15.60:0/2298476057 shutdown_connections 2024-04-25T10:06:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.199+0000 7f43ec9b6640 1 -- 172.21.15.60:0/2298476057 >> 172.21.15.60:0/2298476057 conn(0x7f43e80b3520 msgr2=0x7f43e80b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.199+0000 7f43ec9b6640 1 -- 172.21.15.60:0/2298476057 shutdown_connections 2024-04-25T10:06:32.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.199+0000 7f43ec9b6640 1 -- 172.21.15.60:0/2298476057 wait complete. 2024-04-25T10:06:32.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43ec9b6640 1 Processor -- start 2024-04-25T10:06:32.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43ec9b6640 1 -- start start 2024-04-25T10:06:32.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43ec9b6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d00039f0 0x7f43e81433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43ec9b6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43e814c540 0x7f43e81fa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43ec9b6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43e80b2740 0x7f43e80b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43ec9b6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e815ea10 con 0x7f43e80b2740 2024-04-25T10:06:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43ec9b6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e815e890 con 0x7f43d00039f0 2024-04-25T10:06:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43ec9b6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e815eb90 con 0x7f43e814c540 2024-04-25T10:06:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43e6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43e80b2740 0x7f43e80b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43e6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43e80b2740 0x7f43e80b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50076/0 (socket says 172.21.15.60:50076) 2024-04-25T10:06:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43e6d9d640 1 -- 172.21.15.60:0/586208771 learned_addr learned my addr 172.21.15.60:0/586208771 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43e659c640 1 --2- 172.21.15.60:0/586208771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d00039f0 0x7f43e81433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43e5d9b640 1 --2- 172.21.15.60:0/586208771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43e814c540 0x7f43e81fa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43e6d9d640 1 -- 172.21.15.60:0/586208771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43e814c540 msgr2=0x7f43e81fa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43e6d9d640 1 --2- 172.21.15.60:0/586208771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43e814c540 0x7f43e81fa1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43e6d9d640 1 -- 172.21.15.60:0/586208771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d00039f0 msgr2=0x7f43e81433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43e6d9d640 1 --2- 172.21.15.60:0/586208771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d00039f0 0x7f43e81433e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43e6d9d640 1 -- 172.21.15.60:0/586208771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e81faab0 con 0x7f43e80b2740 2024-04-25T10:06:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43e5d9b640 1 --2- 172.21.15.60:0/586208771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43e814c540 0x7f43e81fa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43e6d9d640 1 --2- 172.21.15.60:0/586208771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43e80b2740 0x7f43e80b2bc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f43e0000f90 tx=0x7f43e000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43d6ffd640 1 -- 172.21.15.60:0/586208771 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0006b60 con 0x7f43e80b2740 2024-04-25T10:06:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43d6ffd640 1 -- 172.21.15.60:0/586208771 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e0006d20 con 0x7f43e80b2740 2024-04-25T10:06:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43ec9b6640 1 -- 172.21.15.60:0/586208771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e81fad90 con 0x7f43e80b2740 2024-04-25T10:06:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43d6ffd640 1 -- 172.21.15.60:0/586208771 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0003980 con 0x7f43e80b2740 2024-04-25T10:06:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43ec9b6640 1 -- 172.21.15.60:0/586208771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e815ee40 con 0x7f43e80b2740 2024-04-25T10:06:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.203+0000 7f43ec9b6640 1 -- 172.21.15.60:0/586208771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e8074770 con 0x7f43e80b2740 2024-04-25T10:06:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.207+0000 7f43d6ffd640 1 -- 172.21.15.60:0/586208771 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43e001a030 con 0x7f43e80b2740 2024-04-25T10:06:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.207+0000 7f43d6ffd640 1 --2- 172.21.15.60:0/586208771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f43c8066f90 0x7f43c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.207+0000 7f43d6ffd640 1 -- 172.21.15.60:0/586208771 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43e0084520 con 0x7f43e80b2740 2024-04-25T10:06:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.207+0000 7f43e659c640 1 --2- 172.21.15.60:0/586208771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f43c8066f90 0x7f43c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.207+0000 7f43e659c640 1 --2- 172.21.15.60:0/586208771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f43c8066f90 0x7f43c8069450 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f43d80ab000 tx=0x7f43d80b1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.211+0000 7f43d6ffd640 1 -- 172.21.15.60:0/586208771 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43e0021020 con 0x7f43e80b2740 2024-04-25T10:06:32.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.383+0000 7f43ec9b6640 1 -- 172.21.15.60:0/586208771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f43e815ce90 con 0x7f43e80b2740 2024-04-25T10:06:32.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.383+0000 7f43d6ffd640 1 -- 172.21.15.60:0/586208771 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7f43e0006f10 con 0x7f43e80b2740 2024-04-25T10:06:32.389 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:06:32.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.383+0000 7f43ec9b6640 1 -- 172.21.15.60:0/586208771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f43c8066f90 msgr2=0x7f43c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.383+0000 7f43ec9b6640 1 --2- 172.21.15.60:0/586208771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f43c8066f90 0x7f43c8069450 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f43d80ab000 tx=0x7f43d80b1040 comp rx=0 tx=0).stop 2024-04-25T10:06:32.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.383+0000 7f43ec9b6640 1 -- 172.21.15.60:0/586208771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43e80b2740 msgr2=0x7f43e80b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.383+0000 7f43ec9b6640 1 --2- 172.21.15.60:0/586208771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43e80b2740 0x7f43e80b2bc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f43e0000f90 tx=0x7f43e000eb90 comp rx=0 tx=0).stop 2024-04-25T10:06:32.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.383+0000 7f43ec9b6640 1 -- 172.21.15.60:0/586208771 shutdown_connections 2024-04-25T10:06:32.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.383+0000 7f43ec9b6640 1 --2- 172.21.15.60:0/586208771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f43c8066f90 0x7f43c8069450 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.383+0000 7f43ec9b6640 1 --2- 172.21.15.60:0/586208771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43e80b2740 0x7f43e80b2bc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.383+0000 7f43ec9b6640 1 --2- 172.21.15.60:0/586208771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43e814c540 0x7f43e81fa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.383+0000 7f43ec9b6640 1 --2- 172.21.15.60:0/586208771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d00039f0 0x7f43e81433e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.383+0000 7f43ec9b6640 1 -- 172.21.15.60:0/586208771 >> 172.21.15.60:0/586208771 conn(0x7f43e80b3520 msgr2=0x7f43e80b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.383+0000 7f43ec9b6640 1 -- 172.21.15.60:0/586208771 shutdown_connections 2024-04-25T10:06:32.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.383+0000 7f43ec9b6640 1 -- 172.21.15.60:0/586208771 wait complete. 2024-04-25T10:06:32.408 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:06:32.408 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:32.414 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:32.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4efdf8b640 1 Processor -- start 2024-04-25T10:06:32.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4efdf8b640 1 -- start start 2024-04-25T10:06:32.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4efdf8b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4ef815a7a0 0x7f4ef80cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4efdf8b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4ef806ec30 0x7f4ef8071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4efdf8b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef80721e0 con 0x7f4ef80b37e0 2024-04-25T10:06:32.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4efdf8b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef80cdaa0 con 0x7f4ef815a7a0 2024-04-25T10:06:32.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4efdf8b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef806d2f0 con 0x7f4ef806ec30 2024-04-25T10:06:32.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4ef77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4ef815a7a0 0x7f4ef80cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4ef7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4ef80b37e0 0x7f4ef806dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46104/0 (socket says 172.21.15.60:46104) 2024-04-25T10:06:32.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4ef77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4ef815a7a0 0x7f4ef80cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50088/0 (socket says 172.21.15.60:50088) 2024-04-25T10:06:32.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4ef7fff640 1 -- 172.21.15.60:0/1754568390 learned_addr learned my addr 172.21.15.60:0/1754568390 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:32.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4ef6ffd640 1 --2- 172.21.15.60:0/1754568390 >> v2:172.21.15.100:3300/0 conn(0x7f4ef806ec30 0x7f4ef8071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4ef77fe640 1 -- 172.21.15.60:0/1754568390 >> v2:172.21.15.100:3300/0 conn(0x7f4ef806ec30 msgr2=0x7f4ef8071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4ef77fe640 1 --2- 172.21.15.60:0/1754568390 >> v2:172.21.15.100:3300/0 conn(0x7f4ef806ec30 0x7f4ef8071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4ef77fe640 1 -- 172.21.15.60:0/1754568390 >> v1:172.21.15.60:6789/0 conn(0x7f4ef80b37e0 legacy=0x7f4ef806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4ef67fc640 1 -- 172.21.15.60:0/1754568390 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715999967 0 0) 0x7f4ef80721e0 con 0x7f4ef80b37e0 2024-04-25T10:06:32.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.571+0000 7f4ef77fe640 1 -- 172.21.15.60:0/1754568390 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef806f130 con 0x7f4ef815a7a0 2024-04-25T10:06:32.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4ef77fe640 1 --2- 172.21.15.60:0/1754568390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ef815a7a0 0x7f4ef80cd560 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8072bf0 tx=0x7f4ee801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f75869edbde0189 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4ef67fc640 1 -- 172.21.15.60:0/1754568390 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee80b1070 con 0x7f4ef815a7a0 2024-04-25T10:06:32.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4ef67fc640 1 -- 172.21.15.60:0/1754568390 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee80ac5f0 con 0x7f4ef815a7a0 2024-04-25T10:06:32.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4ef67fc640 1 -- 172.21.15.60:0/1754568390 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee80ac9c0 con 0x7f4ef815a7a0 2024-04-25T10:06:32.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4efdf8b640 1 -- 172.21.15.60:0/1754568390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ef815a7a0 msgr2=0x7f4ef80cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4efdf8b640 1 --2- 172.21.15.60:0/1754568390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ef815a7a0 0x7f4ef80cd560 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8072bf0 tx=0x7f4ee801c480 comp rx=0 tx=0).stop 2024-04-25T10:06:32.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4efdf8b640 1 -- 172.21.15.60:0/1754568390 shutdown_connections 2024-04-25T10:06:32.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4efdf8b640 1 --2- 172.21.15.60:0/1754568390 >> v2:172.21.15.100:3300/0 conn(0x7f4ef806ec30 0x7f4ef8071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4efdf8b640 1 --2- 172.21.15.60:0/1754568390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ef815a7a0 0x7f4ef80cd560 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4efdf8b640 1 -- 172.21.15.60:0/1754568390 >> 172.21.15.60:0/1754568390 conn(0x7f4ef80b12d0 msgr2=0x7f4ef80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4efdf8b640 1 -- 172.21.15.60:0/1754568390 shutdown_connections 2024-04-25T10:06:32.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4efdf8b640 1 -- 172.21.15.60:0/1754568390 wait complete. 2024-04-25T10:06:32.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4efdf8b640 1 Processor -- start 2024-04-25T10:06:32.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4efdf8b640 1 -- start start 2024-04-25T10:06:32.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4efdf8b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ef806ec30 0x7f4ef8071e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4efdf8b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ef80b37e0 0x7f4ef8146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4efdf8b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef815a7a0 0x7f4ef81535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4efdf8b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef815ad00 con 0x7f4ef806ec30 2024-04-25T10:06:32.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4efdf8b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef815ab80 con 0x7f4ef815a7a0 2024-04-25T10:06:32.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4efdf8b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef806e860 con 0x7f4ef80b37e0 2024-04-25T10:06:32.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4ef77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ef806ec30 0x7f4ef8071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4ef77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ef806ec30 0x7f4ef8071e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50090/0 (socket says 172.21.15.60:50090) 2024-04-25T10:06:32.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4ef77fe640 1 -- 172.21.15.60:0/2277827757 learned_addr learned my addr 172.21.15.60:0/2277827757 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:32.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4ef7fff640 1 --2- 172.21.15.60:0/2277827757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef815a7a0 0x7f4ef81535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4ef6ffd640 1 --2- 172.21.15.60:0/2277827757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ef80b37e0 0x7f4ef8146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4ef77fe640 1 -- 172.21.15.60:0/2277827757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ef80b37e0 msgr2=0x7f4ef8146330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4ef77fe640 1 --2- 172.21.15.60:0/2277827757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ef80b37e0 0x7f4ef8146330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4ef77fe640 1 -- 172.21.15.60:0/2277827757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef815a7a0 msgr2=0x7f4ef81535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4ef77fe640 1 --2- 172.21.15.60:0/2277827757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef815a7a0 0x7f4ef81535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4ef77fe640 1 -- 172.21.15.60:0/2277827757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef80cddd0 con 0x7f4ef806ec30 2024-04-25T10:06:32.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.575+0000 7f4ef6ffd640 1 --2- 172.21.15.60:0/2277827757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ef80b37e0 0x7f4ef8146330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:32.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.579+0000 7f4ef77fe640 1 --2- 172.21.15.60:0/2277827757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ef806ec30 0x7f4ef8071e00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80af040 tx=0x7f4ee8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.579+0000 7f4edffff640 1 -- 172.21.15.60:0/2277827757 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee80b9020 con 0x7f4ef806ec30 2024-04-25T10:06:32.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.579+0000 7f4edffff640 1 -- 172.21.15.60:0/2277827757 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee80b5410 con 0x7f4ef806ec30 2024-04-25T10:06:32.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.579+0000 7f4edffff640 1 -- 172.21.15.60:0/2277827757 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee80b1040 con 0x7f4ef806ec30 2024-04-25T10:06:32.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.579+0000 7f4efdf8b640 1 -- 172.21.15.60:0/2277827757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef80ce0b0 con 0x7f4ef806ec30 2024-04-25T10:06:32.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.579+0000 7f4efdf8b640 1 -- 172.21.15.60:0/2277827757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef8152da0 con 0x7f4ef806ec30 2024-04-25T10:06:32.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.579+0000 7f4efdf8b640 1 -- 172.21.15.60:0/2277827757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef8074770 con 0x7f4ef806ec30 2024-04-25T10:06:32.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.579+0000 7f4edffff640 1 -- 172.21.15.60:0/2277827757 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ee80bc020 con 0x7f4ef806ec30 2024-04-25T10:06:32.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.579+0000 7f4edffff640 1 --2- 172.21.15.60:0/2277827757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4ee4066f60 0x7f4ee4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.579+0000 7f4edffff640 1 -- 172.21.15.60:0/2277827757 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ee811ed50 con 0x7f4ef806ec30 2024-04-25T10:06:32.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.579+0000 7f4ef6ffd640 1 --2- 172.21.15.60:0/2277827757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4ee4066f60 0x7f4ee4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.579+0000 7f4ef6ffd640 1 --2- 172.21.15.60:0/2277827757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4ee4066f60 0x7f4ee4069420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4ef81645b0 tx=0x7f4eec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.583+0000 7f4edffff640 1 -- 172.21.15.60:0/2277827757 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ee80c3030 con 0x7f4ef806ec30 2024-04-25T10:06:32.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.747+0000 7f4efdf8b640 1 -- 172.21.15.60:0/2277827757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ef80cb9d0 con 0x7f4ef806ec30 2024-04-25T10:06:32.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.747+0000 7f4edffff640 1 -- 172.21.15.60:0/2277827757 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16681 (secure 0 0 0) 0x7f4ee80a9070 con 0x7f4ef806ec30 2024-04-25T10:06:32.754 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:32.754 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":35,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:23.196554+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:20.088165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:20.507719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:32.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.751+0000 7f4efdf8b640 1 -- 172.21.15.60:0/2277827757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4ee4066f60 msgr2=0x7f4ee4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.751+0000 7f4efdf8b640 1 --2- 172.21.15.60:0/2277827757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4ee4066f60 0x7f4ee4069420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4ef81645b0 tx=0x7f4eec008040 comp rx=0 tx=0).stop 2024-04-25T10:06:32.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.751+0000 7f4efdf8b640 1 -- 172.21.15.60:0/2277827757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ef806ec30 msgr2=0x7f4ef8071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.751+0000 7f4efdf8b640 1 --2- 172.21.15.60:0/2277827757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ef806ec30 0x7f4ef8071e00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80af040 tx=0x7f4ee8019220 comp rx=0 tx=0).stop 2024-04-25T10:06:32.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.751+0000 7f4efdf8b640 1 -- 172.21.15.60:0/2277827757 shutdown_connections 2024-04-25T10:06:32.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.751+0000 7f4efdf8b640 1 --2- 172.21.15.60:0/2277827757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4ee4066f60 0x7f4ee4069420 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.751+0000 7f4efdf8b640 1 --2- 172.21.15.60:0/2277827757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef815a7a0 0x7f4ef81535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.751+0000 7f4efdf8b640 1 --2- 172.21.15.60:0/2277827757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ef80b37e0 0x7f4ef8146330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.751+0000 7f4efdf8b640 1 --2- 172.21.15.60:0/2277827757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ef806ec30 0x7f4ef8071e00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.751+0000 7f4efdf8b640 1 -- 172.21.15.60:0/2277827757 >> 172.21.15.60:0/2277827757 conn(0x7f4ef80b12d0 msgr2=0x7f4ef814f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.751+0000 7f4efdf8b640 1 -- 172.21.15.60:0/2277827757 shutdown_connections 2024-04-25T10:06:32.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.751+0000 7f4efdf8b640 1 -- 172.21.15.60:0/2277827757 wait complete. 2024-04-25T10:06:32.772 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:20.088165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:20.507719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:32.773 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:32.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.883+0000 7f291cbd7640 1 Processor -- start 2024-04-25T10:06:32.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.883+0000 7f291cbd7640 1 -- start start 2024-04-25T10:06:32.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.883+0000 7f291cbd7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29180cbcf0 0x7f2918160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.883+0000 7f291cbd7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2918071c30 0x7f291815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.883+0000 7f291cbd7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2918144a40 con 0x7f291815fa20 2024-04-25T10:06:32.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.883+0000 7f291cbd7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29180b2dc0 con 0x7f2918071c30 2024-04-25T10:06:32.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.883+0000 7f291cbd7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291815fe80 con 0x7f29180cbcf0 2024-04-25T10:06:32.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.883+0000 7f2915d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2918071c30 0x7f291815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.883+0000 7f2915d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2918071c30 0x7f291815f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50106/0 (socket says 172.21.15.60:50106) 2024-04-25T10:06:32.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.883+0000 7f2916d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f291815fa20 0x7f291815bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46116/0 (socket says 172.21.15.60:46116) 2024-04-25T10:06:32.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.883+0000 7f2915d9b640 1 -- 172.21.15.60:0/4142104307 learned_addr learned my addr 172.21.15.60:0/4142104307 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:32.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.883+0000 7f291659c640 1 --2- 172.21.15.60:0/4142104307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29180cbcf0 0x7f2918160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.883+0000 7f2915d9b640 1 -- 172.21.15.60:0/4142104307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29180cbcf0 msgr2=0x7f2918160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.883+0000 7f2915d9b640 1 --2- 172.21.15.60:0/4142104307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29180cbcf0 0x7f2918160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.883+0000 7f2915d9b640 1 -- 172.21.15.60:0/4142104307 >> v1:172.21.15.60:6789/0 conn(0x7f291815fa20 legacy=0x7f291815bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.883+0000 7f2915d9b640 1 -- 172.21.15.60:0/4142104307 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29180cc9f0 con 0x7f2918071c30 2024-04-25T10:06:32.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f2915d9b640 1 --2- 172.21.15.60:0/4142104307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2918071c30 0x7f291815f4e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f290c009870 tx=0x7f290c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9443516ace21ad8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291559a640 1 -- 172.21.15.60:0/4142104307 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c0a95f0 con 0x7f2918071c30 2024-04-25T10:06:32.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291559a640 1 -- 172.21.15.60:0/4142104307 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c0a9790 con 0x7f2918071c30 2024-04-25T10:06:32.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291559a640 1 -- 172.21.15.60:0/4142104307 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c0a9b60 con 0x7f2918071c30 2024-04-25T10:06:32.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 -- 172.21.15.60:0/4142104307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2918071c30 msgr2=0x7f291815f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 --2- 172.21.15.60:0/4142104307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2918071c30 0x7f291815f4e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f290c009870 tx=0x7f290c01c530 comp rx=0 tx=0).stop 2024-04-25T10:06:32.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 -- 172.21.15.60:0/4142104307 shutdown_connections 2024-04-25T10:06:32.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 --2- 172.21.15.60:0/4142104307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2918071c30 0x7f291815f4e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 --2- 172.21.15.60:0/4142104307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29180cbcf0 0x7f2918160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 -- 172.21.15.60:0/4142104307 >> 172.21.15.60:0/4142104307 conn(0x7f29180670b0 msgr2=0x7f291806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 -- 172.21.15.60:0/4142104307 shutdown_connections 2024-04-25T10:06:32.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 -- 172.21.15.60:0/4142104307 wait complete. 2024-04-25T10:06:32.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 Processor -- start 2024-04-25T10:06:32.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 -- start start 2024-04-25T10:06:32.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2918071c30 0x7f29181592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29180cbcf0 0x7f2918157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f291815fa20 0x7f29181349d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29180720b0 con 0x7f29180cbcf0 2024-04-25T10:06:32.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29180cfed0 con 0x7f2918071c30 2024-04-25T10:06:32.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29181585c0 con 0x7f291815fa20 2024-04-25T10:06:32.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f2915d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29180cbcf0 0x7f2918157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f2915d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29180cbcf0 0x7f2918157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50116/0 (socket says 172.21.15.60:50116) 2024-04-25T10:06:32.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f2915d9b640 1 -- 172.21.15.60:0/262186262 learned_addr learned my addr 172.21.15.60:0/262186262 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:32.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f2915d9b640 1 -- 172.21.15.60:0/262186262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f291815fa20 msgr2=0x7f29181349d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f2916d9d640 1 --2- 172.21.15.60:0/262186262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f291815fa20 0x7f29181349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291659c640 1 --2- 172.21.15.60:0/262186262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2918071c30 0x7f29181592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f2915d9b640 1 --2- 172.21.15.60:0/262186262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f291815fa20 0x7f29181349d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f2915d9b640 1 -- 172.21.15.60:0/262186262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2918071c30 msgr2=0x7f29181592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f2915d9b640 1 --2- 172.21.15.60:0/262186262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2918071c30 0x7f29181592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f2915d9b640 1 -- 172.21.15.60:0/262186262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2918159830 con 0x7f29180cbcf0 2024-04-25T10:06:32.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f2915d9b640 1 --2- 172.21.15.60:0/262186262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29180cbcf0 0x7f2918157d90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f290c0ab040 tx=0x7f290c01ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f2906ffd640 1 -- 172.21.15.60:0/262186262 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c0b1800 con 0x7f29180cbcf0 2024-04-25T10:06:32.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f2906ffd640 1 -- 172.21.15.60:0/262186262 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c0b19c0 con 0x7f29180cbcf0 2024-04-25T10:06:32.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f291cbd7640 1 -- 172.21.15.60:0/262186262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2918135450 con 0x7f29180cbcf0 2024-04-25T10:06:32.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.887+0000 7f2906ffd640 1 -- 172.21.15.60:0/262186262 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c0b1db0 con 0x7f29180cbcf0 2024-04-25T10:06:32.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.891+0000 7f291cbd7640 1 -- 172.21.15.60:0/262186262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2918158af0 con 0x7f29180cbcf0 2024-04-25T10:06:32.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.891+0000 7f291cbd7640 1 -- 172.21.15.60:0/262186262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f291806bba0 con 0x7f29180cbcf0 2024-04-25T10:06:32.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.891+0000 7f2906ffd640 1 -- 172.21.15.60:0/262186262 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f290c0bc450 con 0x7f29180cbcf0 2024-04-25T10:06:32.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.891+0000 7f2906ffd640 1 --2- 172.21.15.60:0/262186262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f28f8066f60 0x7f28f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.891+0000 7f2906ffd640 1 -- 172.21.15.60:0/262186262 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f290c11faf0 con 0x7f29180cbcf0 2024-04-25T10:06:32.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.891+0000 7f291659c640 1 --2- 172.21.15.60:0/262186262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f28f8066f60 0x7f28f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.891+0000 7f291659c640 1 --2- 172.21.15.60:0/262186262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f28f8066f60 0x7f28f8069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2908009790 tx=0x7f2908008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:32.895+0000 7f2906ffd640 1 -- 172.21.15.60:0/262186262 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f290c0ad020 con 0x7f29180cbcf0 2024-04-25T10:06:33.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.051+0000 7f291cbd7640 1 -- 172.21.15.60:0/262186262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2918074770 con 0x7f29180cbcf0 2024-04-25T10:06:33.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.051+0000 7f2906ffd640 1 -- 172.21.15.60:0/262186262 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f290c016150 con 0x7f29180cbcf0 2024-04-25T10:06:33.057 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:06:33.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.051+0000 7f291cbd7640 1 -- 172.21.15.60:0/262186262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f28f8066f60 msgr2=0x7f28f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.051+0000 7f291cbd7640 1 --2- 172.21.15.60:0/262186262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f28f8066f60 0x7f28f8069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2908009790 tx=0x7f2908008040 comp rx=0 tx=0).stop 2024-04-25T10:06:33.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.051+0000 7f291cbd7640 1 -- 172.21.15.60:0/262186262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29180cbcf0 msgr2=0x7f2918157d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.051+0000 7f291cbd7640 1 --2- 172.21.15.60:0/262186262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29180cbcf0 0x7f2918157d90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f290c0ab040 tx=0x7f290c01ad70 comp rx=0 tx=0).stop 2024-04-25T10:06:33.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.051+0000 7f291cbd7640 1 -- 172.21.15.60:0/262186262 shutdown_connections 2024-04-25T10:06:33.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.051+0000 7f291cbd7640 1 --2- 172.21.15.60:0/262186262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f28f8066f60 0x7f28f8069420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.051+0000 7f291cbd7640 1 --2- 172.21.15.60:0/262186262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f291815fa20 0x7f29181349d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.051+0000 7f291cbd7640 1 --2- 172.21.15.60:0/262186262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29180cbcf0 0x7f2918157d90 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.051+0000 7f291cbd7640 1 --2- 172.21.15.60:0/262186262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2918071c30 0x7f29181592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.051+0000 7f291cbd7640 1 -- 172.21.15.60:0/262186262 >> 172.21.15.60:0/262186262 conn(0x7f29180670b0 msgr2=0x7f2918144eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.051+0000 7f291cbd7640 1 -- 172.21.15.60:0/262186262 shutdown_connections 2024-04-25T10:06:33.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.051+0000 7f291cbd7640 1 -- 172.21.15.60:0/262186262 wait complete. 2024-04-25T10:06:33.075 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:33.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.195+0000 7f92523fe640 1 Processor -- start 2024-04-25T10:06:33.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.195+0000 7f92523fe640 1 -- start start 2024-04-25T10:06:33.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.195+0000 7f92523fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f924c15b6f0 0x7f924c1620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.195+0000 7f92523fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f924c0ceef0 0x7f924c15e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.195+0000 7f92523fe640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f924c15bad0 con 0x7f924c15e0d0 2024-04-25T10:06:33.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.195+0000 7f92523fe640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f924c162610 con 0x7f924c15b6f0 2024-04-25T10:06:33.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.195+0000 7f92523fe640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924c161c40 con 0x7f924c0ceef0 2024-04-25T10:06:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.195+0000 7f924bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f924c15b6f0 0x7f924c1620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.195+0000 7f9243fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f924c15e0d0 0x7f924c0ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46120/0 (socket says 172.21.15.60:46120) 2024-04-25T10:06:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.195+0000 7f9243fff640 1 -- 172.21.15.60:0/4258380884 learned_addr learned my addr 172.21.15.60:0/4258380884 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.195+0000 7f925099b640 1 --2- 172.21.15.60:0/4258380884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f924c0ceef0 0x7f924c15e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.195+0000 7f924bfff640 1 -- 172.21.15.60:0/4258380884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f924c0ceef0 msgr2=0x7f924c15e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.195+0000 7f924bfff640 1 --2- 172.21.15.60:0/4258380884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f924c0ceef0 0x7f924c15e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.195+0000 7f924bfff640 1 -- 172.21.15.60:0/4258380884 >> v1:172.21.15.60:6789/0 conn(0x7f924c15e0d0 legacy=0x7f924c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.195+0000 7f924bfff640 1 -- 172.21.15.60:0/4258380884 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c15ebe0 con 0x7f924c15b6f0 2024-04-25T10:06:33.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f924bfff640 1 --2- 172.21.15.60:0/4258380884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f924c15b6f0 0x7f924c1620d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f923c0097c0 tx=0x7f923c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15c5a05a6c62ceb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f924b7fe640 1 -- 172.21.15.60:0/4258380884 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c0b2070 con 0x7f924c15b6f0 2024-04-25T10:06:33.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f924b7fe640 1 -- 172.21.15.60:0/4258380884 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923c0ad640 con 0x7f924c15b6f0 2024-04-25T10:06:33.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f924b7fe640 1 -- 172.21.15.60:0/4258380884 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c0ada10 con 0x7f924c15b6f0 2024-04-25T10:06:33.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f92523fe640 1 -- 172.21.15.60:0/4258380884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f924c15b6f0 msgr2=0x7f924c1620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f92523fe640 1 --2- 172.21.15.60:0/4258380884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f924c15b6f0 0x7f924c1620d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f923c0097c0 tx=0x7f923c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f92523fe640 1 -- 172.21.15.60:0/4258380884 shutdown_connections 2024-04-25T10:06:33.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f92523fe640 1 --2- 172.21.15.60:0/4258380884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f924c0ceef0 0x7f924c15e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f92523fe640 1 --2- 172.21.15.60:0/4258380884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f924c15b6f0 0x7f924c1620d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f92523fe640 1 -- 172.21.15.60:0/4258380884 >> 172.21.15.60:0/4258380884 conn(0x7f924c072340 msgr2=0x7f924c072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f92523fe640 1 -- 172.21.15.60:0/4258380884 shutdown_connections 2024-04-25T10:06:33.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f92523fe640 1 -- 172.21.15.60:0/4258380884 wait complete. 2024-04-25T10:06:33.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f92523fe640 1 Processor -- start 2024-04-25T10:06:33.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f92523fe640 1 -- start start 2024-04-25T10:06:33.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f92523fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f924c0ceef0 0x7f924c0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f92523fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f924c15b6f0 0x7f924c0cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f92523fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f924c15e0d0 0x7f924c0d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f92523fe640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0d24c0 con 0x7f924c15b6f0 2024-04-25T10:06:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f92523fe640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0d2340 con 0x7f924c15e0d0 2024-04-25T10:06:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f92523fe640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0d2640 con 0x7f924c0ceef0 2024-04-25T10:06:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f9243fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f924c15b6f0 0x7f924c0cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f9243fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f924c15b6f0 0x7f924c0cfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50136/0 (socket says 172.21.15.60:50136) 2024-04-25T10:06:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f9243fff640 1 -- 172.21.15.60:0/3949539910 learned_addr learned my addr 172.21.15.60:0/3949539910 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f924bfff640 1 --2- 172.21.15.60:0/3949539910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f924c0ceef0 0x7f924c0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f925099b640 1 --2- 172.21.15.60:0/3949539910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f924c15e0d0 0x7f924c0d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f9243fff640 1 -- 172.21.15.60:0/3949539910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f924c0ceef0 msgr2=0x7f924c0cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f9243fff640 1 --2- 172.21.15.60:0/3949539910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f924c0ceef0 0x7f924c0cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f9243fff640 1 -- 172.21.15.60:0/3949539910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f924c15e0d0 msgr2=0x7f924c0d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f9243fff640 1 --2- 172.21.15.60:0/3949539910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f924c15e0d0 0x7f924c0d01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.199+0000 7f9243fff640 1 -- 172.21.15.60:0/3949539910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c0cbc40 con 0x7f924c15b6f0 2024-04-25T10:06:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.203+0000 7f9243fff640 1 --2- 172.21.15.60:0/3949539910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f924c15b6f0 0x7f924c0cfc90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f923400c370 tx=0x7f923400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.203+0000 7f92497fa640 1 -- 172.21.15.60:0/3949539910 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234017070 con 0x7f924c15b6f0 2024-04-25T10:06:33.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.203+0000 7f92497fa640 1 -- 172.21.15.60:0/3949539910 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9234005cf0 con 0x7f924c15b6f0 2024-04-25T10:06:33.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.203+0000 7f92523fe640 1 -- 172.21.15.60:0/3949539910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924c0cbf20 con 0x7f924c15b6f0 2024-04-25T10:06:33.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.203+0000 7f92497fa640 1 -- 172.21.15.60:0/3949539910 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92340072f0 con 0x7f924c15b6f0 2024-04-25T10:06:33.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.203+0000 7f92523fe640 1 -- 172.21.15.60:0/3949539910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924c0b74d0 con 0x7f924c15b6f0 2024-04-25T10:06:33.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.203+0000 7f92523fe640 1 -- 172.21.15.60:0/3949539910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924c074770 con 0x7f924c15b6f0 2024-04-25T10:06:33.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.203+0000 7f92497fa640 1 -- 172.21.15.60:0/3949539910 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f923401a070 con 0x7f924c15b6f0 2024-04-25T10:06:33.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.203+0000 7f92497fa640 1 --2- 172.21.15.60:0/3949539910 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9230066f60 0x7f9230069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.203+0000 7f92497fa640 1 -- 172.21.15.60:0/3949539910 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92340843d0 con 0x7f924c15b6f0 2024-04-25T10:06:33.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.203+0000 7f924bfff640 1 --2- 172.21.15.60:0/3949539910 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9230066f60 0x7f9230069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.203+0000 7f924bfff640 1 --2- 172.21.15.60:0/3949539910 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9230066f60 0x7f9230069420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f923c0af000 tx=0x7f923c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.207+0000 7f92497fa640 1 -- 172.21.15.60:0/3949539910 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92340553e0 con 0x7f924c15b6f0 2024-04-25T10:06:33.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.359+0000 7f92523fe640 1 -- 172.21.15.60:0/3949539910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f924c0d16e0 con 0x7f924c15b6f0 2024-04-25T10:06:33.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.363+0000 7f92497fa640 1 -- 172.21.15.60:0/3949539910 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9234054ce0 con 0x7f924c15b6f0 2024-04-25T10:06:33.368 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:06:33.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.363+0000 7f92523fe640 1 -- 172.21.15.60:0/3949539910 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9230066f60 msgr2=0x7f9230069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.363+0000 7f92523fe640 1 --2- 172.21.15.60:0/3949539910 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9230066f60 0x7f9230069420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f923c0af000 tx=0x7f923c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:33.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.363+0000 7f92523fe640 1 -- 172.21.15.60:0/3949539910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f924c15b6f0 msgr2=0x7f924c0cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.363+0000 7f92523fe640 1 --2- 172.21.15.60:0/3949539910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f924c15b6f0 0x7f924c0cfc90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f923400c370 tx=0x7f923400c830 comp rx=0 tx=0).stop 2024-04-25T10:06:33.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.363+0000 7f92523fe640 1 -- 172.21.15.60:0/3949539910 shutdown_connections 2024-04-25T10:06:33.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.363+0000 7f92523fe640 1 --2- 172.21.15.60:0/3949539910 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9230066f60 0x7f9230069420 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.363+0000 7f92523fe640 1 --2- 172.21.15.60:0/3949539910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f924c15e0d0 0x7f924c0d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.363+0000 7f92523fe640 1 --2- 172.21.15.60:0/3949539910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f924c15b6f0 0x7f924c0cfc90 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.363+0000 7f92523fe640 1 --2- 172.21.15.60:0/3949539910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f924c0ceef0 0x7f924c0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.363+0000 7f92523fe640 1 -- 172.21.15.60:0/3949539910 >> 172.21.15.60:0/3949539910 conn(0x7f924c072340 msgr2=0x7f924c1545b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.363+0000 7f92523fe640 1 -- 172.21.15.60:0/3949539910 shutdown_connections 2024-04-25T10:06:33.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.363+0000 7f92523fe640 1 -- 172.21.15.60:0/3949539910 wait complete. 2024-04-25T10:06:33.386 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.499+0000 7f246cdae640 1 Processor -- start 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.499+0000 7f246cdae640 1 -- start start 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.499+0000 7f246cdae640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f246815bea0 0x7f24680b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.499+0000 7f246cdae640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24680b35e0 0x7f24680718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.499+0000 7f246cdae640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f246815ce20 con 0x7f24680cefe0 2024-04-25T10:06:33.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.499+0000 7f246cdae640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f246815c350 con 0x7f24680b35e0 2024-04-25T10:06:33.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.499+0000 7f246cdae640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2468071410 con 0x7f246815bea0 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.499+0000 7f2466d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24680b35e0 0x7f24680718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.499+0000 7f2466d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24680b35e0 0x7f24680718a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50152/0 (socket says 172.21.15.60:50152) 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.499+0000 7f2466d9d640 1 -- 172.21.15.60:0/362840582 learned_addr learned my addr 172.21.15.60:0/362840582 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.499+0000 7f2465d9b640 1 --2- 172.21.15.60:0/362840582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f246815bea0 0x7f24680b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.499+0000 7f2466d9d640 1 -- 172.21.15.60:0/362840582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f246815bea0 msgr2=0x7f24680b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.499+0000 7f2466d9d640 1 --2- 172.21.15.60:0/362840582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f246815bea0 0x7f24680b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.499+0000 7f2466d9d640 1 -- 172.21.15.60:0/362840582 >> v1:172.21.15.100:6789/0 conn(0x7f24680cefe0 legacy=0x7f24680cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.499+0000 7f2466d9d640 1 -- 172.21.15.60:0/362840582 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246815c580 con 0x7f24680b35e0 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f2466d9d640 1 --2- 172.21.15.60:0/362840582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24680b35e0 0x7f24680718a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2460009e50 tx=0x7f246001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af30b4f290af267d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246559a640 1 -- 172.21.15.60:0/362840582 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24600a5660 con 0x7f24680b35e0 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246559a640 1 -- 172.21.15.60:0/362840582 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24600a5800 con 0x7f24680b35e0 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246559a640 1 -- 172.21.15.60:0/362840582 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24600a5bd0 con 0x7f24680b35e0 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246cdae640 1 -- 172.21.15.60:0/362840582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24680b35e0 msgr2=0x7f24680718a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246cdae640 1 --2- 172.21.15.60:0/362840582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24680b35e0 0x7f24680718a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2460009e50 tx=0x7f246001c9a0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246cdae640 1 -- 172.21.15.60:0/362840582 shutdown_connections 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246cdae640 1 --2- 172.21.15.60:0/362840582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24680b35e0 0x7f24680718a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246cdae640 1 --2- 172.21.15.60:0/362840582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f246815bea0 0x7f24680b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246cdae640 1 -- 172.21.15.60:0/362840582 >> 172.21.15.60:0/362840582 conn(0x7f24680670b0 msgr2=0x7f246806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246cdae640 1 -- 172.21.15.60:0/362840582 shutdown_connections 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246cdae640 1 -- 172.21.15.60:0/362840582 wait complete. 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246cdae640 1 Processor -- start 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246cdae640 1 -- start start 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246cdae640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24680b35e0 0x7f246806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246cdae640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24680cefe0 0x7f246815a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246cdae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f246815bea0 0x7f246806d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246cdae640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2468168ba0 con 0x7f24680cefe0 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246cdae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2468168a20 con 0x7f246815bea0 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f246cdae640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2468168d20 con 0x7f24680b35e0 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f2465d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24680cefe0 0x7f246815a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.503+0000 7f2465d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24680cefe0 0x7f246815a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50160/0 (socket says 172.21.15.60:50160) 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f2465d9b640 1 -- 172.21.15.60:0/2339719433 learned_addr learned my addr 172.21.15.60:0/2339719433 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f246659c640 1 --2- 172.21.15.60:0/2339719433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24680b35e0 0x7f246806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f2466d9d640 1 --2- 172.21.15.60:0/2339719433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f246815bea0 0x7f246806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f2465d9b640 1 -- 172.21.15.60:0/2339719433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24680b35e0 msgr2=0x7f246806ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f2465d9b640 1 --2- 172.21.15.60:0/2339719433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24680b35e0 0x7f246806ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f2465d9b640 1 -- 172.21.15.60:0/2339719433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f246815bea0 msgr2=0x7f246806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f2465d9b640 1 --2- 172.21.15.60:0/2339719433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f246815bea0 0x7f246806d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f2465d9b640 1 -- 172.21.15.60:0/2339719433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246815b390 con 0x7f24680cefe0 2024-04-25T10:06:33.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f2465d9b640 1 --2- 172.21.15.60:0/2339719433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24680cefe0 0x7f246815a5a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f245c00d5f0 tx=0x7f245c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f2456ffd640 1 -- 172.21.15.60:0/2339719433 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c0125f0 con 0x7f24680cefe0 2024-04-25T10:06:33.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f2456ffd640 1 -- 172.21.15.60:0/2339719433 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245c019070 con 0x7f24680cefe0 2024-04-25T10:06:33.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f2456ffd640 1 -- 172.21.15.60:0/2339719433 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c0129b0 con 0x7f24680cefe0 2024-04-25T10:06:33.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f246cdae640 1 -- 172.21.15.60:0/2339719433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f246806ddc0 con 0x7f24680cefe0 2024-04-25T10:06:33.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f246cdae640 1 -- 172.21.15.60:0/2339719433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f246806e660 con 0x7f24680cefe0 2024-04-25T10:06:33.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f246cdae640 1 -- 172.21.15.60:0/2339719433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2468074770 con 0x7f24680cefe0 2024-04-25T10:06:33.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f2456ffd640 1 -- 172.21.15.60:0/2339719433 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f245c012b50 con 0x7f24680cefe0 2024-04-25T10:06:33.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f2456ffd640 1 --2- 172.21.15.60:0/2339719433 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2448066f60 0x7f2448069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f2456ffd640 1 -- 172.21.15.60:0/2339719433 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f245c0853e0 con 0x7f24680cefe0 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.507+0000 7f246659c640 1 --2- 172.21.15.60:0/2339719433 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2448066f60 0x7f2448069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.511+0000 7f246659c640 1 --2- 172.21.15.60:0/2339719433 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2448066f60 0x7f2448069420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2458001650 tx=0x7f2458009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.511+0000 7f2456ffd640 1 -- 172.21.15.60:0/2339719433 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f245c0563f0 con 0x7f24680cefe0 2024-04-25T10:06:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.671+0000 7f246cdae640 1 -- 172.21.15.60:0/2339719433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f24680cd510 con 0x7f24680cefe0 2024-04-25T10:06:33.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.671+0000 7f2456ffd640 1 -- 172.21.15.60:0/2339719433 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7f245c055cf0 con 0x7f24680cefe0 2024-04-25T10:06:33.679 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:06:33.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.675+0000 7f246cdae640 1 -- 172.21.15.60:0/2339719433 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2448066f60 msgr2=0x7f2448069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.675+0000 7f246cdae640 1 --2- 172.21.15.60:0/2339719433 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2448066f60 0x7f2448069420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2458001650 tx=0x7f2458009040 comp rx=0 tx=0).stop 2024-04-25T10:06:33.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.675+0000 7f246cdae640 1 -- 172.21.15.60:0/2339719433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24680cefe0 msgr2=0x7f246815a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.675+0000 7f246cdae640 1 --2- 172.21.15.60:0/2339719433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24680cefe0 0x7f246815a5a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f245c00d5f0 tx=0x7f245c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.675+0000 7f246cdae640 1 -- 172.21.15.60:0/2339719433 shutdown_connections 2024-04-25T10:06:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.675+0000 7f246cdae640 1 --2- 172.21.15.60:0/2339719433 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2448066f60 0x7f2448069420 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.675+0000 7f246cdae640 1 --2- 172.21.15.60:0/2339719433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f246815bea0 0x7f246806d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.675+0000 7f246cdae640 1 --2- 172.21.15.60:0/2339719433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24680cefe0 0x7f246815a5a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.675+0000 7f246cdae640 1 --2- 172.21.15.60:0/2339719433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24680b35e0 0x7f246806ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.675+0000 7f246cdae640 1 -- 172.21.15.60:0/2339719433 >> 172.21.15.60:0/2339719433 conn(0x7f24680670b0 msgr2=0x7f24680af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.675+0000 7f246cdae640 1 -- 172.21.15.60:0/2339719433 shutdown_connections 2024-04-25T10:06:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.675+0000 7f246cdae640 1 -- 172.21.15.60:0/2339719433 wait complete. 2024-04-25T10:06:33.698 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:06:33.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.811+0000 7f0a01186640 1 Processor -- start 2024-04-25T10:06:33.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.811+0000 7f0a01186640 1 -- start start 2024-04-25T10:06:33.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f0a01186640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f09fc0cf380 0x7f09fc0cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f0a01186640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09fc0cc5d0 con 0x7f09fc0cd390 2024-04-25T10:06:33.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f0a01186640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09fc0cc8b0 con 0x7f09fc15f480 2024-04-25T10:06:33.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f0a01186640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09fc0ccb90 con 0x7f09fc0cf380 2024-04-25T10:06:33.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09f259c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f09fc0cd390 0x7f09fc09a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46130/0 (socket says 172.21.15.60:46130) 2024-04-25T10:06:33.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09f259c640 1 -- 172.21.15.60:0/3882058550 learned_addr learned my addr 172.21.15.60:0/3882058550 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:33.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fb59e640 1 --2- 172.21.15.60:0/3882058550 >> v2:172.21.15.100:3300/0 conn(0x7f09fc0cf380 0x7f09fc0cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fa59c640 1 -- 172.21.15.60:0/3882058550 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495050797 0 0) 0x7f09fc0cc5d0 con 0x7f09fc0cd390 2024-04-25T10:06:33.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fa59c640 1 -- 172.21.15.60:0/3882058550 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09e8003530 con 0x7f09fc0cd390 2024-04-25T10:06:33.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fa59c640 1 -- 172.21.15.60:0/3882058550 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3165236664 0 0) 0x7f09e8003530 con 0x7f09fc0cd390 2024-04-25T10:06:33.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fa59c640 1 -- 172.21.15.60:0/3882058550 >> v2:172.21.15.100:3300/0 conn(0x7f09fc0cf380 msgr2=0x7f09fc0cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fa59c640 1 --2- 172.21.15.60:0/3882058550 >> v2:172.21.15.100:3300/0 conn(0x7f09fc0cf380 0x7f09fc0cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fa59c640 1 -- 172.21.15.60:0/3882058550 >> v1:172.21.15.100:6789/0 conn(0x7f09fc15f480 legacy=0x7f09fc15f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fa59c640 1 -- 172.21.15.60:0/3882058550 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc0cce70 con 0x7f09fc0cd390 2024-04-25T10:06:33.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fa59c640 1 -- 172.21.15.60:0/3882058550 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f09e4001680 con 0x7f09fc0cd390 2024-04-25T10:06:33.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fa59c640 1 -- 172.21.15.60:0/3882058550 >> v1:172.21.15.60:6789/0 conn(0x7f09fc0cd390 legacy=0x7f09fc09a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fa59c640 1 --2- 172.21.15.60:0/3882058550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09e80039c0 0x7f09e8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fa59c640 1 --2- 172.21.15.60:0/3882058550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e80043c0 0x7f09e8004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fa59c640 1 --2- 172.21.15.60:0/3882058550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09e8005090 0x7f09e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fa59c640 1 -- 172.21.15.60:0/3882058550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc0ccb90 con 0x7f09e80043c0 2024-04-25T10:06:33.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fa59c640 1 -- 172.21.15.60:0/3882058550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e800bff0 con 0x7f09e80039c0 2024-04-25T10:06:33.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fa59c640 1 -- 172.21.15.60:0/3882058550 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e800c1f0 con 0x7f09e8005090 2024-04-25T10:06:33.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09f259c640 1 --2- 172.21.15.60:0/3882058550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e80043c0 0x7f09e8004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fad9d640 1 --2- 172.21.15.60:0/3882058550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09e80039c0 0x7f09e8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fb59e640 1 --2- 172.21.15.60:0/3882058550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09e8005090 0x7f09e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09f259c640 1 -- 172.21.15.60:0/3882058550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09e8005090 msgr2=0x7f09e800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09f259c640 1 --2- 172.21.15.60:0/3882058550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09e8005090 0x7f09e800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09f259c640 1 -- 172.21.15.60:0/3882058550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09e80039c0 msgr2=0x7f09e8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09f259c640 1 --2- 172.21.15.60:0/3882058550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09e80039c0 0x7f09e8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09f259c640 1 -- 172.21.15.60:0/3882058550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e8003530 con 0x7f09e80043c0 2024-04-25T10:06:33.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09fb59e640 1 -- 172.21.15.60:0/3882058550 reap_dead start 2024-04-25T10:06:33.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.815+0000 7f09f259c640 1 --2- 172.21.15.60:0/3882058550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e80043c0 0x7f09e8004890 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f09e400c2c0 tx=0x7f09e401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc3db8cdff74df3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09fa59c640 1 -- 172.21.15.60:0/3882058550 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4007d60 con 0x7f09e80043c0 2024-04-25T10:06:33.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09fa59c640 1 -- 172.21.15.60:0/3882058550 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e40a9070 con 0x7f09e80043c0 2024-04-25T10:06:33.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09fa59c640 1 -- 172.21.15.60:0/3882058550 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e40a5db0 con 0x7f09e80043c0 2024-04-25T10:06:33.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f0a01186640 1 -- 172.21.15.60:0/3882058550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e80043c0 msgr2=0x7f09e8004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f0a01186640 1 --2- 172.21.15.60:0/3882058550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e80043c0 0x7f09e8004890 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f09e400c2c0 tx=0x7f09e401cca0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f0a01186640 1 -- 172.21.15.60:0/3882058550 shutdown_connections 2024-04-25T10:06:33.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f0a01186640 1 --2- 172.21.15.60:0/3882058550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e80043c0 0x7f09e8004890 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f0a01186640 1 -- 172.21.15.60:0/3882058550 >> 172.21.15.60:0/3882058550 conn(0x7f09fc0b32a0 msgr2=0x7f09fc0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f0a01186640 1 -- 172.21.15.60:0/3882058550 shutdown_connections 2024-04-25T10:06:33.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f0a01186640 1 -- 172.21.15.60:0/3882058550 wait complete. 2024-04-25T10:06:33.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f0a01186640 1 Processor -- start 2024-04-25T10:06:33.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f0a01186640 1 -- start start 2024-04-25T10:06:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f0a01186640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e80043c0 0x7f09fc06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f0a01186640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09fc1f4080 0x7f09fc1ed560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f0a01186640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09fc15e0c0 0x7f09fc15e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f0a01186640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc1602f0 con 0x7f09e80043c0 2024-04-25T10:06:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f0a01186640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc160170 con 0x7f09fc15e0c0 2024-04-25T10:06:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f0a01186640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc15f860 con 0x7f09fc1f4080 2024-04-25T10:06:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09fad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e80043c0 0x7f09fc06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09fad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e80043c0 0x7f09fc06d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50180/0 (socket says 172.21.15.60:50180) 2024-04-25T10:06:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09fad9d640 1 -- 172.21.15.60:0/779640410 learned_addr learned my addr 172.21.15.60:0/779640410 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09fb59e640 1 --2- 172.21.15.60:0/779640410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09fc15e0c0 0x7f09fc15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09f259c640 1 --2- 172.21.15.60:0/779640410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09fc1f4080 0x7f09fc1ed560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09fad9d640 1 -- 172.21.15.60:0/779640410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09fc1f4080 msgr2=0x7f09fc1ed560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09fad9d640 1 --2- 172.21.15.60:0/779640410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09fc1f4080 0x7f09fc1ed560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09fad9d640 1 -- 172.21.15.60:0/779640410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09fc15e0c0 msgr2=0x7f09fc15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09fad9d640 1 --2- 172.21.15.60:0/779640410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09fc15e0c0 0x7f09fc15e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09fad9d640 1 -- 172.21.15.60:0/779640410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc06db20 con 0x7f09e80043c0 2024-04-25T10:06:33.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09fad9d640 1 --2- 172.21.15.60:0/779640410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e80043c0 0x7f09fc06d500 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f09ec010f40 tx=0x7f09ec00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09f3fff640 1 -- 172.21.15.60:0/779640410 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec018020 con 0x7f09e80043c0 2024-04-25T10:06:33.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09f3fff640 1 -- 172.21.15.60:0/779640410 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec005290 con 0x7f09e80043c0 2024-04-25T10:06:33.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f09f3fff640 1 -- 172.21.15.60:0/779640410 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec003840 con 0x7f09e80043c0 2024-04-25T10:06:33.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.819+0000 7f0a01186640 1 -- 172.21.15.60:0/779640410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09fc06bf30 con 0x7f09e80043c0 2024-04-25T10:06:33.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.823+0000 7f0a01186640 1 -- 172.21.15.60:0/779640410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc06c4f0 con 0x7f09e80043c0 2024-04-25T10:06:33.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.823+0000 7f0a01186640 1 -- 172.21.15.60:0/779640410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09fc06c6f0 con 0x7f09e80043c0 2024-04-25T10:06:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.823+0000 7f09f3fff640 1 -- 172.21.15.60:0/779640410 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f09ec01b020 con 0x7f09e80043c0 2024-04-25T10:06:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.823+0000 7f09f3fff640 1 --2- 172.21.15.60:0/779640410 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f09dc066f90 0x7f09dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.823+0000 7f09f3fff640 1 -- 172.21.15.60:0/779640410 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09ec0889a0 con 0x7f09e80043c0 2024-04-25T10:06:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.823+0000 7f09f259c640 1 --2- 172.21.15.60:0/779640410 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f09dc066f90 0x7f09dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.823+0000 7f09f259c640 1 --2- 172.21.15.60:0/779640410 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f09dc066f90 0x7f09dc069450 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f09e40ab000 tx=0x7f09e40b0040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.827+0000 7f09f3fff640 1 -- 172.21.15.60:0/779640410 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f09fc06c6f0 con 0x7f09e80043c0 2024-04-25T10:06:34.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.995+0000 7f0a01186640 1 -- 172.21.15.60:0/779640410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f09fc06bd20 con 0x7f09e80043c0 2024-04-25T10:06:34.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.995+0000 7f09f3fff640 1 -- 172.21.15.60:0/779640410 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7f09ec022020 con 0x7f09e80043c0 2024-04-25T10:06:34.001 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:06:34.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.995+0000 7f0a01186640 1 -- 172.21.15.60:0/779640410 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f09dc066f90 msgr2=0x7f09dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.995+0000 7f0a01186640 1 --2- 172.21.15.60:0/779640410 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f09dc066f90 0x7f09dc069450 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f09e40ab000 tx=0x7f09e40b0040 comp rx=0 tx=0).stop 2024-04-25T10:06:34.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.995+0000 7f0a01186640 1 -- 172.21.15.60:0/779640410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e80043c0 msgr2=0x7f09fc06d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.995+0000 7f0a01186640 1 --2- 172.21.15.60:0/779640410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e80043c0 0x7f09fc06d500 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f09ec010f40 tx=0x7f09ec00e530 comp rx=0 tx=0).stop 2024-04-25T10:06:34.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.995+0000 7f0a01186640 1 -- 172.21.15.60:0/779640410 shutdown_connections 2024-04-25T10:06:34.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.995+0000 7f0a01186640 1 --2- 172.21.15.60:0/779640410 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f09dc066f90 0x7f09dc069450 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.995+0000 7f0a01186640 1 --2- 172.21.15.60:0/779640410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09fc15e0c0 0x7f09fc15e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.995+0000 7f0a01186640 1 --2- 172.21.15.60:0/779640410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09fc1f4080 0x7f09fc1ed560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.995+0000 7f0a01186640 1 --2- 172.21.15.60:0/779640410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e80043c0 0x7f09fc06d500 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.995+0000 7f0a01186640 1 -- 172.21.15.60:0/779640410 >> 172.21.15.60:0/779640410 conn(0x7f09fc0b32a0 msgr2=0x7f09fc15bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.995+0000 7f0a01186640 1 -- 172.21.15.60:0/779640410 shutdown_connections 2024-04-25T10:06:34.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:33.995+0000 7f0a01186640 1 -- 172.21.15.60:0/779640410 wait complete. 2024-04-25T10:06:34.019 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:06:34.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:34.019 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:34.029 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[36740]: fuse finished with error 0 and tester_r 0 2024-04-25T10:06:34.030 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:40.133 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:06:40.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:40.133 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:06:40.141 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:40.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.311+0000 7f03fe5f1640 1 Processor -- start 2024-04-25T10:06:40.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.311+0000 7f03fe5f1640 1 -- start start 2024-04-25T10:06:40.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.311+0000 7f03fe5f1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f03f80ce7a0 0x7f03f80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.311+0000 7f03fe5f1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03f80b4660 0x7f03f80b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.311+0000 7f03fe5f1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f80cf0c0 con 0x7f03f815bc90 2024-04-25T10:06:40.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.311+0000 7f03fe5f1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f80cc370 con 0x7f03f80ce7a0 2024-04-25T10:06:40.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.311+0000 7f03fe5f1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f8071ca0 con 0x7f03f80b4660 2024-04-25T10:06:40.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.311+0000 7f03f7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f03f80ce7a0 0x7f03f80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.311+0000 7f03f77fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f03f815bc90 0x7f03f80b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34368/0 (socket says 172.21.15.60:34368) 2024-04-25T10:06:40.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.311+0000 7f03f77fe640 1 -- 172.21.15.60:0/4000416773 learned_addr learned my addr 172.21.15.60:0/4000416773 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:40.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.311+0000 7f03f7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f03f80ce7a0 0x7f03f80ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36376/0 (socket says 172.21.15.60:36376) 2024-04-25T10:06:40.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.311+0000 7f03f7fff640 1 -- 172.21.15.60:0/4000416773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03f80b4660 msgr2=0x7f03f80b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.311+0000 7f03f7fff640 1 --2- 172.21.15.60:0/4000416773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03f80b4660 0x7f03f80b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.311+0000 7f03f7fff640 1 -- 172.21.15.60:0/4000416773 >> v1:172.21.15.60:6789/0 conn(0x7f03f815bc90 legacy=0x7f03f80b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.311+0000 7f03f6ffd640 1 -- 172.21.15.60:0/4000416773 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104337118 0 0) 0x7f03f80cf0c0 con 0x7f03f815bc90 2024-04-25T10:06:40.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.311+0000 7f03f7fff640 1 -- 172.21.15.60:0/4000416773 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f80cca40 con 0x7f03f80ce7a0 2024-04-25T10:06:40.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03f7fff640 1 --2- 172.21.15.60:0/4000416773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03f80ce7a0 0x7f03f80ceb80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f03e80097c0 tx=0x7f03e801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e0f6378505ecb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03f6ffd640 1 -- 172.21.15.60:0/4000416773 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e80b0070 con 0x7f03f80ce7a0 2024-04-25T10:06:40.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03f6ffd640 1 -- 172.21.15.60:0/4000416773 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e80ab600 con 0x7f03f80ce7a0 2024-04-25T10:06:40.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03f6ffd640 1 -- 172.21.15.60:0/4000416773 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e80ab9d0 con 0x7f03f80ce7a0 2024-04-25T10:06:40.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fe5f1640 1 -- 172.21.15.60:0/4000416773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03f80ce7a0 msgr2=0x7f03f80ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fe5f1640 1 --2- 172.21.15.60:0/4000416773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03f80ce7a0 0x7f03f80ceb80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f03e80097c0 tx=0x7f03e801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fe5f1640 1 -- 172.21.15.60:0/4000416773 shutdown_connections 2024-04-25T10:06:40.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fe5f1640 1 --2- 172.21.15.60:0/4000416773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03f80b4660 0x7f03f80b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fe5f1640 1 --2- 172.21.15.60:0/4000416773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03f80ce7a0 0x7f03f80ceb80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fe5f1640 1 -- 172.21.15.60:0/4000416773 >> 172.21.15.60:0/4000416773 conn(0x7f03f80b12d0 msgr2=0x7f03f80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fe5f1640 1 -- 172.21.15.60:0/4000416773 shutdown_connections 2024-04-25T10:06:40.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fe5f1640 1 -- 172.21.15.60:0/4000416773 wait complete. 2024-04-25T10:06:40.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fe5f1640 1 Processor -- start 2024-04-25T10:06:40.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fe5f1640 1 -- start start 2024-04-25T10:06:40.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fe5f1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03f80ce7a0 0x7f03f8156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fe5f1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03f815bc90 0x7f03f8152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fe5f1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03f8153ad0 0x7f03f8152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fe5f1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f80b4400 con 0x7f03f815bc90 2024-04-25T10:06:40.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fe5f1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f80b4280 con 0x7f03f8153ad0 2024-04-25T10:06:40.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fe5f1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f80b3220 con 0x7f03f80ce7a0 2024-04-25T10:06:40.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03f815bc90 0x7f03f8152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03f815bc90 0x7f03f8152a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36384/0 (socket says 172.21.15.60:36384) 2024-04-25T10:06:40.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03f77fe640 1 -- 172.21.15.60:0/1261421038 learned_addr learned my addr 172.21.15.60:0/1261421038 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:40.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03f7fff640 1 --2- 172.21.15.60:0/1261421038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03f80ce7a0 0x7f03f8156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03fcb8e640 1 --2- 172.21.15.60:0/1261421038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03f8153ad0 0x7f03f8152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03f77fe640 1 -- 172.21.15.60:0/1261421038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03f80ce7a0 msgr2=0x7f03f8156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03f77fe640 1 --2- 172.21.15.60:0/1261421038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03f80ce7a0 0x7f03f8156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03f77fe640 1 -- 172.21.15.60:0/1261421038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03f8153ad0 msgr2=0x7f03f8152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03f77fe640 1 --2- 172.21.15.60:0/1261421038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03f8153ad0 0x7f03f8152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.315+0000 7f03f77fe640 1 -- 172.21.15.60:0/1261421038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f8153f60 con 0x7f03f815bc90 2024-04-25T10:06:40.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.319+0000 7f03f7fff640 1 --2- 172.21.15.60:0/1261421038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03f80ce7a0 0x7f03f8156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:40.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.319+0000 7f03f77fe640 1 --2- 172.21.15.60:0/1261421038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03f815bc90 0x7f03f8152a90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00c480 tx=0x7f03ec00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.319+0000 7f03f4ff9640 1 -- 172.21.15.60:0/1261421038 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec018070 con 0x7f03f815bc90 2024-04-25T10:06:40.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.319+0000 7f03f4ff9640 1 -- 172.21.15.60:0/1261421038 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ec007500 con 0x7f03f815bc90 2024-04-25T10:06:40.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.319+0000 7f03f4ff9640 1 -- 172.21.15.60:0/1261421038 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec004a30 con 0x7f03f815bc90 2024-04-25T10:06:40.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.319+0000 7f03fe5f1640 1 -- 172.21.15.60:0/1261421038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f81545f0 con 0x7f03f815bc90 2024-04-25T10:06:40.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.319+0000 7f03fe5f1640 1 -- 172.21.15.60:0/1261421038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f8154bb0 con 0x7f03f815bc90 2024-04-25T10:06:40.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.319+0000 7f03fe5f1640 1 -- 172.21.15.60:0/1261421038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f8074770 con 0x7f03f815bc90 2024-04-25T10:06:40.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.319+0000 7f03f4ff9640 1 -- 172.21.15.60:0/1261421038 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03ec004bd0 con 0x7f03f815bc90 2024-04-25T10:06:40.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.319+0000 7f03f4ff9640 1 --2- 172.21.15.60:0/1261421038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f03e4066f60 0x7f03e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.319+0000 7f03f4ff9640 1 -- 172.21.15.60:0/1261421038 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03ec084280 con 0x7f03f815bc90 2024-04-25T10:06:40.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.319+0000 7f03f7fff640 1 --2- 172.21.15.60:0/1261421038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f03e4066f60 0x7f03e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.319+0000 7f03f7fff640 1 --2- 172.21.15.60:0/1261421038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f03e4066f60 0x7f03e4069420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f03e80ad000 tx=0x7f03e80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.323+0000 7f03f4ff9640 1 -- 172.21.15.60:0/1261421038 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f03ec055320 con 0x7f03f815bc90 2024-04-25T10:06:40.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.519+0000 7f03fe5f1640 1 -- 172.21.15.60:0/1261421038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03f80715a0 con 0x7f03f815bc90 2024-04-25T10:06:40.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.519+0000 7f03f4ff9640 1 -- 172.21.15.60:0/1261421038 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f03ec054c20 con 0x7f03f815bc90 2024-04-25T10:06:40.527 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:40.527 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2422261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2422261630},{"type":"v1","addr":"172.21.15.100:6835","nonce":2422261630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2923340508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2923340508},{"type":"v1","addr":"172.21.15.100:6837","nonce":2923340508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1929386449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1929386449},{"type":"v1","addr":"172.21.15.60:6833","nonce":1929386449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:21.529455+0000","modified":"2024-04-25T10:06:23.575106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5283":{"gid":5283,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3681296171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3681296171},{"type":"v1","addr":"172.21.15.60:6835","nonce":3681296171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:40.528 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 15 2024-04-25T10:06:40.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.519+0000 7f03fe5f1640 1 -- 172.21.15.60:0/1261421038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f03e4066f60 msgr2=0x7f03e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.519+0000 7f03fe5f1640 1 --2- 172.21.15.60:0/1261421038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f03e4066f60 0x7f03e4069420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f03e80ad000 tx=0x7f03e80a6000 comp rx=0 tx=0).stop 2024-04-25T10:06:40.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.519+0000 7f03fe5f1640 1 -- 172.21.15.60:0/1261421038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03f815bc90 msgr2=0x7f03f8152a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.519+0000 7f03fe5f1640 1 --2- 172.21.15.60:0/1261421038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03f815bc90 0x7f03f8152a90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00c480 tx=0x7f03ec00c940 comp rx=0 tx=0).stop 2024-04-25T10:06:40.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.519+0000 7f03fe5f1640 1 -- 172.21.15.60:0/1261421038 shutdown_connections 2024-04-25T10:06:40.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.519+0000 7f03fe5f1640 1 --2- 172.21.15.60:0/1261421038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f03e4066f60 0x7f03e4069420 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.519+0000 7f03fe5f1640 1 --2- 172.21.15.60:0/1261421038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03f8153ad0 0x7f03f8152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.519+0000 7f03fe5f1640 1 --2- 172.21.15.60:0/1261421038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03f815bc90 0x7f03f8152a90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.519+0000 7f03fe5f1640 1 --2- 172.21.15.60:0/1261421038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03f80ce7a0 0x7f03f8156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.519+0000 7f03fe5f1640 1 -- 172.21.15.60:0/1261421038 >> 172.21.15.60:0/1261421038 conn(0x7f03f80b12d0 msgr2=0x7f03f815d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.523+0000 7f03fe5f1640 1 -- 172.21.15.60:0/1261421038 shutdown_connections 2024-04-25T10:06:40.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.523+0000 7f03fe5f1640 1 -- 172.21.15.60:0/1261421038 wait complete. 2024-04-25T10:06:40.546 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:06:40.695 INFO:teuthology.orchestra.run.smithi060.stdout:{"message":"","return_code":0} 2024-04-25T10:06:40.696 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:06:40.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:40.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe217857640 1 Processor -- start 2024-04-25T10:06:40.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe217857640 1 -- start start 2024-04-25T10:06:40.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe217857640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2100719a0 0x7fe2100ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe217857640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe210071fd0 con 0x7fe2100ae7c0 2024-04-25T10:06:40.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe217857640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2100b2ce0 con 0x7fe21015dc90 2024-04-25T10:06:40.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe217857640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100accc0 con 0x7fe2100719a0 2024-04-25T10:06:40.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe214df2640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe2100ae7c0 0x7fe210071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34376/0 (socket says 172.21.15.60:34376) 2024-04-25T10:06:40.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe214df2640 1 -- 172.21.15.60:0/2802442850 learned_addr learned my addr 172.21.15.60:0/2802442850 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:40.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe215df4640 1 --2- 172.21.15.60:0/2802442850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2100719a0 0x7fe2100ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe207fff640 1 -- 172.21.15.60:0/2802442850 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347265963 0 0) 0x7fe210071fd0 con 0x7fe2100ae7c0 2024-04-25T10:06:40.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe207fff640 1 -- 172.21.15.60:0/2802442850 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1fc003530 con 0x7fe2100ae7c0 2024-04-25T10:06:40.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe207fff640 1 -- 172.21.15.60:0/2802442850 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 35525621 0 0) 0x7fe1fc003530 con 0x7fe2100ae7c0 2024-04-25T10:06:40.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe207fff640 1 -- 172.21.15.60:0/2802442850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2100719a0 msgr2=0x7fe2100ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe207fff640 1 --2- 172.21.15.60:0/2802442850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2100719a0 0x7fe2100ad6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe207fff640 1 -- 172.21.15.60:0/2802442850 >> v1:172.21.15.100:6789/0 conn(0x7fe21015dc90 legacy=0x7fe2100b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe207fff640 1 -- 172.21.15.60:0/2802442850 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe210072290 con 0x7fe2100ae7c0 2024-04-25T10:06:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe207fff640 1 -- 172.21.15.60:0/2802442850 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fe20c001680 con 0x7fe2100ae7c0 2024-04-25T10:06:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe207fff640 1 -- 172.21.15.60:0/2802442850 >> v1:172.21.15.60:6789/0 conn(0x7fe2100ae7c0 legacy=0x7fe210071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe207fff640 1 --2- 172.21.15.60:0/2802442850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2100719a0 0x7fe1fc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe207fff640 1 --2- 172.21.15.60:0/2802442850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1fc003fc0 0x7fe1fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe207fff640 1 --2- 172.21.15.60:0/2802442850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1fc004d50 0x7fe1fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe207fff640 1 -- 172.21.15.60:0/2802442850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc00bae0 con 0x7fe1fc003fc0 2024-04-25T10:06:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe207fff640 1 -- 172.21.15.60:0/2802442850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc00bc90 con 0x7fe1fc004d50 2024-04-25T10:06:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe207fff640 1 -- 172.21.15.60:0/2802442850 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc00be90 con 0x7fe2100719a0 2024-04-25T10:06:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe214df2640 1 --2- 172.21.15.60:0/2802442850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1fc003fc0 0x7fe1fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe2155f3640 1 --2- 172.21.15.60:0/2802442850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2100719a0 0x7fe1fc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe215df4640 1 --2- 172.21.15.60:0/2802442850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1fc004d50 0x7fe1fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe214df2640 1 -- 172.21.15.60:0/2802442850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2100719a0 msgr2=0x7fe1fc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe214df2640 1 --2- 172.21.15.60:0/2802442850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2100719a0 0x7fe1fc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe214df2640 1 -- 172.21.15.60:0/2802442850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1fc004d50 msgr2=0x7fe1fc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe214df2640 1 --2- 172.21.15.60:0/2802442850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1fc004d50 0x7fe1fc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.807+0000 7fe214df2640 1 -- 172.21.15.60:0/2802442850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc003530 con 0x7fe1fc003fc0 2024-04-25T10:06:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.811+0000 7fe214df2640 1 --2- 172.21.15.60:0/2802442850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1fc003fc0 0x7fe1fc004810 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00c190 tx=0x7fe20c01f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa59de9f3b0cdd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.811+0000 7fe207fff640 1 -- 172.21.15.60:0/2802442850 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c007c70 con 0x7fe1fc003fc0 2024-04-25T10:06:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.811+0000 7fe207fff640 1 -- 172.21.15.60:0/2802442850 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20c0ac070 con 0x7fe1fc003fc0 2024-04-25T10:06:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.811+0000 7fe207fff640 1 -- 172.21.15.60:0/2802442850 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c005b20 con 0x7fe1fc003fc0 2024-04-25T10:06:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.811+0000 7fe217857640 1 -- 172.21.15.60:0/2802442850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1fc003fc0 msgr2=0x7fe1fc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.811+0000 7fe217857640 1 --2- 172.21.15.60:0/2802442850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1fc003fc0 0x7fe1fc004810 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00c190 tx=0x7fe20c01f270 comp rx=0 tx=0).stop 2024-04-25T10:06:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.811+0000 7fe215df4640 1 -- 172.21.15.60:0/2802442850 reap_dead start 2024-04-25T10:06:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.811+0000 7fe217857640 1 -- 172.21.15.60:0/2802442850 shutdown_connections 2024-04-25T10:06:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.811+0000 7fe217857640 1 -- 172.21.15.60:0/2802442850 >> 172.21.15.60:0/2802442850 conn(0x7fe2100b12d0 msgr2=0x7fe2100b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.811+0000 7fe217857640 1 -- 172.21.15.60:0/2802442850 shutdown_connections 2024-04-25T10:06:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.811+0000 7fe217857640 1 -- 172.21.15.60:0/2802442850 wait complete. 2024-04-25T10:06:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.811+0000 7fe217857640 1 Processor -- start 2024-04-25T10:06:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.811+0000 7fe217857640 1 -- start start 2024-04-25T10:06:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe217857640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe21014a1c0 0x7fe210159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe217857640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe210160db0 0x7fe21015b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe217857640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe21015bdb0 0x7fe2101f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe217857640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100b2010 con 0x7fe21014a1c0 2024-04-25T10:06:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe217857640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100ad260 con 0x7fe210160db0 2024-04-25T10:06:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe217857640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100b2190 con 0x7fe21015bdb0 2024-04-25T10:06:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe2155f3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe21014a1c0 0x7fe210159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe2155f3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe21014a1c0 0x7fe210159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36392/0 (socket says 172.21.15.60:36392) 2024-04-25T10:06:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe2155f3640 1 -- 172.21.15.60:0/2782620469 learned_addr learned my addr 172.21.15.60:0/2782620469 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe215df4640 1 --2- 172.21.15.60:0/2782620469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe21015bdb0 0x7fe2101f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe2155f3640 1 -- 172.21.15.60:0/2782620469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe21015bdb0 msgr2=0x7fe2101f6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe2155f3640 1 --2- 172.21.15.60:0/2782620469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe21015bdb0 0x7fe2101f6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe2155f3640 1 -- 172.21.15.60:0/2782620469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe210160db0 msgr2=0x7fe21015b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe214df2640 1 --2- 172.21.15.60:0/2782620469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe210160db0 0x7fe21015b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe2155f3640 1 --2- 172.21.15.60:0/2782620469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe210160db0 0x7fe21015b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe2155f3640 1 -- 172.21.15.60:0/2782620469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21015a640 con 0x7fe21014a1c0 2024-04-25T10:06:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe2155f3640 1 --2- 172.21.15.60:0/2782620469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe21014a1c0 0x7fe210159e10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe20000f850 tx=0x7fe20000fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe205ffb640 1 -- 172.21.15.60:0/2782620469 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe200001cf0 con 0x7fe21014a1c0 2024-04-25T10:06:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe205ffb640 1 -- 172.21.15.60:0/2782620469 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe200018070 con 0x7fe21014a1c0 2024-04-25T10:06:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe217857640 1 -- 172.21.15.60:0/2782620469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21015a920 con 0x7fe21014a1c0 2024-04-25T10:06:40.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe205ffb640 1 -- 172.21.15.60:0/2782620469 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20000d730 con 0x7fe21014a1c0 2024-04-25T10:06:40.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe217857640 1 -- 172.21.15.60:0/2782620469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21015a350 con 0x7fe21014a1c0 2024-04-25T10:06:40.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe217857640 1 -- 172.21.15.60:0/2782620469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe210074770 con 0x7fe21014a1c0 2024-04-25T10:06:40.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe205ffb640 1 -- 172.21.15.60:0/2782620469 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe20000d8d0 con 0x7fe21014a1c0 2024-04-25T10:06:40.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe205ffb640 1 --2- 172.21.15.60:0/2782620469 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe1f4066f60 0x7fe1f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe205ffb640 1 -- 172.21.15.60:0/2782620469 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe200084db0 con 0x7fe21014a1c0 2024-04-25T10:06:40.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.815+0000 7fe214df2640 1 --2- 172.21.15.60:0/2782620469 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe1f4066f60 0x7fe1f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.819+0000 7fe214df2640 1 --2- 172.21.15.60:0/2782620469 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe1f4066f60 0x7fe1f4069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00c160 tx=0x7fe20c0aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:40.819+0000 7fe205ffb640 1 -- 172.21.15.60:0/2782620469 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe200055e50 con 0x7fe21014a1c0 2024-04-25T10:06:41.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:41.015+0000 7fe217857640 1 -- 172.21.15.60:0/2782620469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe210071fd0 con 0x7fe21014a1c0 2024-04-25T10:06:41.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:41.015+0000 7fe205ffb640 1 -- 172.21.15.60:0/2782620469 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fe200055750 con 0x7fe21014a1c0 2024-04-25T10:06:41.023 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:41.023 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2422261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2422261630},{"type":"v1","addr":"172.21.15.100:6835","nonce":2422261630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2923340508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2923340508},{"type":"v1","addr":"172.21.15.100:6837","nonce":2923340508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1929386449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1929386449},{"type":"v1","addr":"172.21.15.60:6833","nonce":1929386449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:21.529455+0000","modified":"2024-04-25T10:06:23.575106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5283":{"gid":5283,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3681296171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3681296171},{"type":"v1","addr":"172.21.15.60:6835","nonce":3681296171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:41.023 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 15 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:41.019+0000 7fe217857640 1 -- 172.21.15.60:0/2782620469 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe1f4066f60 msgr2=0x7fe1f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:41.019+0000 7fe217857640 1 --2- 172.21.15.60:0/2782620469 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe1f4066f60 0x7fe1f4069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00c160 tx=0x7fe20c0aa070 comp rx=0 tx=0).stop 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:41.019+0000 7fe217857640 1 -- 172.21.15.60:0/2782620469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe21014a1c0 msgr2=0x7fe210159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:41.019+0000 7fe217857640 1 --2- 172.21.15.60:0/2782620469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe21014a1c0 0x7fe210159e10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe20000f850 tx=0x7fe20000fd10 comp rx=0 tx=0).stop 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:41.019+0000 7fe217857640 1 -- 172.21.15.60:0/2782620469 shutdown_connections 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:41.019+0000 7fe217857640 1 --2- 172.21.15.60:0/2782620469 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe1f4066f60 0x7fe1f4069420 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:41.019+0000 7fe217857640 1 --2- 172.21.15.60:0/2782620469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe21015bdb0 0x7fe2101f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:41.019+0000 7fe217857640 1 --2- 172.21.15.60:0/2782620469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe210160db0 0x7fe21015b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:41.019+0000 7fe217857640 1 --2- 172.21.15.60:0/2782620469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe21014a1c0 0x7fe210159e10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:41.019+0000 7fe217857640 1 -- 172.21.15.60:0/2782620469 >> 172.21.15.60:0/2782620469 conn(0x7fe2100b12d0 msgr2=0x7fe21006dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:41.019+0000 7fe217857640 1 -- 172.21.15.60:0/2782620469 shutdown_connections 2024-04-25T10:06:41.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:41.019+0000 7fe217857640 1 -- 172.21.15.60:0/2782620469 wait complete. 2024-04-25T10:06:41.042 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parentdir mytag 2024-04-25T10:06:41.172 INFO:teuthology.orchestra.run.smithi060.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-04-25T10:06:41.172 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-04-25T10:06:46.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:46.313 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:51.174 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-04-25T10:06:51.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 Processor -- start 2024-04-25T10:06:51.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 -- start start 2024-04-25T10:06:51.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594472a61c0 0x55944729f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594472a99a0 0x55944728d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5594473332e0 0x559447330890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55944725c720 con 0x5594472a61c0 2024-04-25T10:06:51.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594472029f0 con 0x5594472a99a0 2024-04-25T10:06:51.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55944725c8a0 con 0x5594473332e0 2024-04-25T10:06:51.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a4a03640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594472a61c0 0x55944729f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a4a03640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594472a61c0 0x55944729f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39212/0 (socket says 172.21.15.60:39212) 2024-04-25T10:06:51.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a4a03640 1 -- 172.21.15.60:0/1735464461 learned_addr learned my addr 172.21.15.60:0/1735464461 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:51.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a4202640 1 --2- 172.21.15.60:0/1735464461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594472a99a0 0x55944728d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a5204640 1 --2- 172.21.15.60:0/1735464461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5594473332e0 0x559447330890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a4a03640 1 -- 172.21.15.60:0/1735464461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5594473332e0 msgr2=0x559447330890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a4a03640 1 --2- 172.21.15.60:0/1735464461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5594473332e0 0x559447330890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a4a03640 1 -- 172.21.15.60:0/1735464461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594472a99a0 msgr2=0x55944728d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a4a03640 1 --2- 172.21.15.60:0/1735464461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594472a99a0 0x55944728d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a4a03640 1 -- 172.21.15.60:0/1735464461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55944729ff90 con 0x5594472a61c0 2024-04-25T10:06:51.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a4a03640 1 --2- 172.21.15.60:0/1735464461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594472a61c0 0x55944729f880 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa99400b220 tx=0x7fa99400b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f90c1bb32fc5b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a3200640 1 -- 172.21.15.60:0/1735464461 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99400a040 con 0x5594472a61c0 2024-04-25T10:06:51.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a3200640 1 -- 172.21.15.60:0/1735464461 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99400f620 con 0x5594472a61c0 2024-04-25T10:06:51.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a3200640 1 -- 172.21.15.60:0/1735464461 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99400fa10 con 0x5594472a61c0 2024-04-25T10:06:51.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 -- 172.21.15.60:0/1735464461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594472a61c0 msgr2=0x55944729f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 --2- 172.21.15.60:0/1735464461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594472a61c0 0x55944729f880 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa99400b220 tx=0x7fa99400b6e0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 -- 172.21.15.60:0/1735464461 shutdown_connections 2024-04-25T10:06:51.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 --2- 172.21.15.60:0/1735464461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5594473332e0 0x559447330890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 --2- 172.21.15.60:0/1735464461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594472a99a0 0x55944728d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 --2- 172.21.15.60:0/1735464461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594472a61c0 0x55944729f880 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 -- 172.21.15.60:0/1735464461 >> 172.21.15.60:0/1735464461 conn(0x55944728a3d0 msgr2=0x5594473111d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 -- 172.21.15.60:0/1735464461 shutdown_connections 2024-04-25T10:06:51.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.211+0000 7fa9a6ee5080 1 -- 172.21.15.60:0/1735464461 wait complete. 2024-04-25T10:06:51.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a6ee5080 1 Processor -- start 2024-04-25T10:06:51.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a6ee5080 1 -- start start 2024-04-25T10:06:51.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a6ee5080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594472a61c0 0x55944732ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a6ee5080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5594472a99a0 0x559447288d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a6ee5080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594473332e0 0x5594472a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a6ee5080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594472a0230 con 0x5594472a61c0 2024-04-25T10:06:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a6ee5080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594472a04d0 con 0x5594473332e0 2024-04-25T10:06:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a6ee5080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55944728ddc0 con 0x5594472a99a0 2024-04-25T10:06:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a4a03640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594472a61c0 0x55944732ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a4a03640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594472a61c0 0x55944732ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39216/0 (socket says 172.21.15.60:39216) 2024-04-25T10:06:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a4a03640 1 -- 172.21.15.60:0/3079294124 learned_addr learned my addr 172.21.15.60:0/3079294124 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a4202640 1 --2- 172.21.15.60:0/3079294124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5594472a99a0 0x559447288d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a5204640 1 --2- 172.21.15.60:0/3079294124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594473332e0 0x5594472a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a4a03640 1 -- 172.21.15.60:0/3079294124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5594472a99a0 msgr2=0x559447288d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a4a03640 1 --2- 172.21.15.60:0/3079294124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5594472a99a0 0x559447288d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a4a03640 1 -- 172.21.15.60:0/3079294124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594473332e0 msgr2=0x5594472a5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a4a03640 1 --2- 172.21.15.60:0/3079294124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594473332e0 0x5594472a5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a4a03640 1 -- 172.21.15.60:0/3079294124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594473cd8f0 con 0x5594472a61c0 2024-04-25T10:06:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a5204640 1 --2- 172.21.15.60:0/3079294124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594473332e0 0x5594472a5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a4a03640 1 --2- 172.21.15.60:0/3079294124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594472a61c0 0x55944732ff30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa99400bbe0 tx=0x7fa994000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a21fe640 1 -- 172.21.15.60:0/3079294124 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99401ba40 con 0x5594472a61c0 2024-04-25T10:06:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a21fe640 1 -- 172.21.15.60:0/3079294124 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99401bbe0 con 0x5594472a61c0 2024-04-25T10:06:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a6ee5080 1 -- 172.21.15.60:0/3079294124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5594473cdb90 con 0x5594472a61c0 2024-04-25T10:06:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a21fe640 1 -- 172.21.15.60:0/3079294124 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa994019470 con 0x5594472a61c0 2024-04-25T10:06:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a6ee5080 1 -- 172.21.15.60:0/3079294124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594473ccf60 con 0x5594472a61c0 2024-04-25T10:06:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.215+0000 7fa9a21fe640 1 -- 172.21.15.60:0/3079294124 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa994017040 con 0x5594472a61c0 2024-04-25T10:06:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a21fe640 1 --2- 172.21.15.60:0/3079294124 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa990065970 0x7fa990067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a21fe640 1 -- 172.21.15.60:0/3079294124 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9940859a0 con 0x5594472a61c0 2024-04-25T10:06:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a6ee5080 1 --2- 172.21.15.60:0/3079294124 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x5594473cf2e0 0x5594473d78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a4202640 1 --2- 172.21.15.60:0/3079294124 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa990065970 0x7fa990067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a6ee5080 1 -- 172.21.15.60:0/3079294124 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v9 -- 0x5594473d7df0 con 0x5594473cf2e0 2024-04-25T10:06:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a5204640 1 --2- 172.21.15.60:0/3079294124 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x5594473cf2e0 0x5594473d78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a5204640 1 --2- 172.21.15.60:0/3079294124 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x5594473cf2e0 0x5594473d78b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a4202640 1 --2- 172.21.15.60:0/3079294124 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa990065970 0x7fa990067e30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa9980ab000 tx=0x7fa9980a5070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a5204640 1 -- 172.21.15.60:0/3079294124 <== osd.3 v2:172.21.15.60:6814/1867957156 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fa99c0097b0 con 0x5594473cf2e0 2024-04-25T10:06:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a6ee5080 1 -- 172.21.15.60:0/3079294124 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x5594473cf2e0 msgr2=0x5594473d78b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a6ee5080 1 --2- 172.21.15.60:0/3079294124 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x5594473cf2e0 0x5594473d78b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a6ee5080 1 -- 172.21.15.60:0/3079294124 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa990065970 msgr2=0x7fa990067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a6ee5080 1 --2- 172.21.15.60:0/3079294124 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa990065970 0x7fa990067e30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa9980ab000 tx=0x7fa9980a5070 comp rx=0 tx=0).stop 2024-04-25T10:06:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a6ee5080 1 -- 172.21.15.60:0/3079294124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594472a61c0 msgr2=0x55944732ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a6ee5080 1 --2- 172.21.15.60:0/3079294124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5594472a61c0 0x55944732ff30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa99400bbe0 tx=0x7fa994000bd0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a5204640 1 -- 172.21.15.60:0/3079294124 reap_dead start 2024-04-25T10:06:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a6ee5080 1 -- 172.21.15.60:0/3079294124 shutdown_connections 2024-04-25T10:06:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a6ee5080 1 -- 172.21.15.60:0/3079294124 >> 172.21.15.60:0/3079294124 conn(0x55944728a3d0 msgr2=0x55944732f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a6ee5080 1 -- 172.21.15.60:0/3079294124 shutdown_connections 2024-04-25T10:06:51.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.219+0000 7fa9a6ee5080 1 -- 172.21.15.60:0/3079294124 wait complete. 2024-04-25T10:06:51.231 INFO:teuthology.orchestra.run.smithi060.stdout:mytag 2024-04-25T10:06:51.232 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-04-25T10:06:51.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.307+0000 7f9fe70d8080 1 Processor -- start 2024-04-25T10:06:51.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.307+0000 7f9fe70d8080 1 -- start start 2024-04-25T10:06:51.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.307+0000 7f9fe70d8080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5570fd00faa0 0x5570fd0e3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.307+0000 7f9fe70d8080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5570fd00fe80 0x5570fd0e4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.307+0000 7f9fe70d8080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570fd0e7040 0x5570fd0e4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.307+0000 7f9fe70d8080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570fd0e83f0 con 0x5570fd00faa0 2024-04-25T10:06:51.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.307+0000 7f9fe70d8080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570fcfb69f0 con 0x5570fd0e7040 2024-04-25T10:06:51.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.307+0000 7f9fe70d8080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5570fd0e8570 con 0x5570fd00fe80 2024-04-25T10:06:51.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.307+0000 7f9fe4bf6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5570fd00faa0 0x5570fd0e3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.307+0000 7f9fe4bf6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5570fd00faa0 0x5570fd0e3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39242/0 (socket says 172.21.15.60:39242) 2024-04-25T10:06:51.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.307+0000 7f9fe4bf6640 1 -- 172.21.15.60:0/1130992201 learned_addr learned my addr 172.21.15.60:0/1130992201 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:51.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe43f5640 1 --2- 172.21.15.60:0/1130992201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5570fd00fe80 0x5570fd0e4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe53f7640 1 --2- 172.21.15.60:0/1130992201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570fd0e7040 0x5570fd0e4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe4bf6640 1 -- 172.21.15.60:0/1130992201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5570fd00fe80 msgr2=0x5570fd0e4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe4bf6640 1 --2- 172.21.15.60:0/1130992201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5570fd00fe80 0x5570fd0e4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe4bf6640 1 -- 172.21.15.60:0/1130992201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570fd0e7040 msgr2=0x5570fd0e4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe4bf6640 1 --2- 172.21.15.60:0/1130992201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570fd0e7040 0x5570fd0e4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe4bf6640 1 -- 172.21.15.60:0/1130992201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570fd0531a0 con 0x5570fd00faa0 2024-04-25T10:06:51.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe53f7640 1 --2- 172.21.15.60:0/1130992201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570fd0e7040 0x5570fd0e4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:51.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe4bf6640 1 --2- 172.21.15.60:0/1130992201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5570fd00faa0 0x5570fd0e3c30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400a4b0 tx=0x7f9fd40033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e427cf6e52c3dbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe33f3640 1 -- 172.21.15.60:0/1130992201 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40b9070 con 0x5570fd00faa0 2024-04-25T10:06:51.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe33f3640 1 -- 172.21.15.60:0/1130992201 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd40ab4d0 con 0x5570fd00faa0 2024-04-25T10:06:51.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe33f3640 1 -- 172.21.15.60:0/1130992201 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40ab8c0 con 0x5570fd00faa0 2024-04-25T10:06:51.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 -- 172.21.15.60:0/1130992201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5570fd00faa0 msgr2=0x5570fd0e3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 --2- 172.21.15.60:0/1130992201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5570fd00faa0 0x5570fd0e3c30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400a4b0 tx=0x7f9fd40033b0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 -- 172.21.15.60:0/1130992201 shutdown_connections 2024-04-25T10:06:51.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 --2- 172.21.15.60:0/1130992201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570fd0e7040 0x5570fd0e4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 --2- 172.21.15.60:0/1130992201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5570fd00fe80 0x5570fd0e4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 --2- 172.21.15.60:0/1130992201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5570fd00faa0 0x5570fd0e3c30 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 -- 172.21.15.60:0/1130992201 >> 172.21.15.60:0/1130992201 conn(0x5570fd010750 msgr2=0x5570fd03e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 -- 172.21.15.60:0/1130992201 shutdown_connections 2024-04-25T10:06:51.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 -- 172.21.15.60:0/1130992201 wait complete. 2024-04-25T10:06:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 Processor -- start 2024-04-25T10:06:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 -- start start 2024-04-25T10:06:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5570fd00faa0 0x5570fd17a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5570fd00fe80 0x5570fd17ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570fd0e7040 0x5570fd183bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570fd0536e0 con 0x5570fd00faa0 2024-04-25T10:06:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570fd053440 con 0x5570fd0e7040 2024-04-25T10:06:51.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5570fd044410 con 0x5570fd00fe80 2024-04-25T10:06:51.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe4bf6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5570fd00faa0 0x5570fd17a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe4bf6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5570fd00faa0 0x5570fd17a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39258/0 (socket says 172.21.15.60:39258) 2024-04-25T10:06:51.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe4bf6640 1 -- 172.21.15.60:0/2716299211 learned_addr learned my addr 172.21.15.60:0/2716299211 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:51.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe53f7640 1 --2- 172.21.15.60:0/2716299211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570fd0e7040 0x5570fd183bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe43f5640 1 --2- 172.21.15.60:0/2716299211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5570fd00fe80 0x5570fd17ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe4bf6640 1 -- 172.21.15.60:0/2716299211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5570fd00fe80 msgr2=0x5570fd17ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe4bf6640 1 --2- 172.21.15.60:0/2716299211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5570fd00fe80 0x5570fd17ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe4bf6640 1 -- 172.21.15.60:0/2716299211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570fd0e7040 msgr2=0x5570fd183bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe4bf6640 1 --2- 172.21.15.60:0/2716299211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5570fd0e7040 0x5570fd183bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe4bf6640 1 -- 172.21.15.60:0/2716299211 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570fd0c63b0 con 0x5570fd00faa0 2024-04-25T10:06:51.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe4bf6640 1 --2- 172.21.15.60:0/2716299211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5570fd00faa0 0x5570fd17a140 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4096a20 tx=0x7f9fd4017de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe23f1640 1 -- 172.21.15.60:0/2716299211 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40b9040 con 0x5570fd00faa0 2024-04-25T10:06:51.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe23f1640 1 -- 172.21.15.60:0/2716299211 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd40c6070 con 0x5570fd00faa0 2024-04-25T10:06:51.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 -- 172.21.15.60:0/2716299211 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5570fd0c6650 con 0x5570fd00faa0 2024-04-25T10:06:51.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe23f1640 1 -- 172.21.15.60:0/2716299211 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40bf6b0 con 0x5570fd00faa0 2024-04-25T10:06:51.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.311+0000 7f9fe70d8080 1 -- 172.21.15.60:0/2716299211 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570fd0c6b90 con 0x5570fd00faa0 2024-04-25T10:06:51.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe23f1640 1 -- 172.21.15.60:0/2716299211 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9fd40b3070 con 0x5570fd00faa0 2024-04-25T10:06:51.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe23f1640 1 --2- 172.21.15.60:0/2716299211 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9fc4065940 0x7f9fc4067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe23f1640 1 -- 172.21.15.60:0/2716299211 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fd40b0020 con 0x5570fd00faa0 2024-04-25T10:06:51.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe70d8080 1 --2- 172.21.15.60:0/2716299211 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5570fd17b340 0x5570fd0c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe43f5640 1 --2- 172.21.15.60:0/2716299211 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9fc4065940 0x7f9fc4067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe70d8080 1 -- 172.21.15.60:0/2716299211 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:1 4.7 4:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v9 -- 0x5570fd17a680 con 0x5570fd17b340 2024-04-25T10:06:51.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe53f7640 1 --2- 172.21.15.60:0/2716299211 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5570fd17b340 0x5570fd0c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe53f7640 1 --2- 172.21.15.60:0/2716299211 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5570fd17b340 0x5570fd0c5020 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe43f5640 1 --2- 172.21.15.60:0/2716299211 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9fc4065940 0x7f9fc4067e00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00f4e0 tx=0x7f9fcc003640 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe53f7640 1 -- 172.21.15.60:0/2716299211 <== osd.1 v2:172.21.15.60:6824/3313552458 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f9fdc009650 con 0x5570fd17b340 2024-04-25T10:06:51.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe70d8080 1 -- 172.21.15.60:0/2716299211 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5570fd17b340 msgr2=0x5570fd0c5020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe70d8080 1 --2- 172.21.15.60:0/2716299211 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5570fd17b340 0x5570fd0c5020 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe70d8080 1 -- 172.21.15.60:0/2716299211 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9fc4065940 msgr2=0x7f9fc4067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe70d8080 1 --2- 172.21.15.60:0/2716299211 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9fc4065940 0x7f9fc4067e00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00f4e0 tx=0x7f9fcc003640 comp rx=0 tx=0).stop 2024-04-25T10:06:51.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe70d8080 1 -- 172.21.15.60:0/2716299211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5570fd00faa0 msgr2=0x5570fd17a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe70d8080 1 --2- 172.21.15.60:0/2716299211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5570fd00faa0 0x5570fd17a140 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4096a20 tx=0x7f9fd4017de0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe53f7640 1 -- 172.21.15.60:0/2716299211 reap_dead start 2024-04-25T10:06:51.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe70d8080 1 -- 172.21.15.60:0/2716299211 shutdown_connections 2024-04-25T10:06:51.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe70d8080 1 -- 172.21.15.60:0/2716299211 >> 172.21.15.60:0/2716299211 conn(0x5570fd010750 msgr2=0x5570fd0e5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe70d8080 1 -- 172.21.15.60:0/2716299211 shutdown_connections 2024-04-25T10:06:51.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.315+0000 7f9fe70d8080 1 -- 172.21.15.60:0/2716299211 wait complete. 2024-04-25T10:06:51.329 INFO:teuthology.orchestra.run.smithi060.stdout:mytag 2024-04-25T10:06:51.329 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-25T10:06:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.399+0000 7fe041802080 1 Processor -- start 2024-04-25T10:06:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.399+0000 7fe041802080 1 -- start start 2024-04-25T10:06:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe028003100 0x564dae010220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564dae09b690 0x564dae142960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dae142ea0 0x564dae09bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dadf879f0 con 0x564dae09b690 2024-04-25T10:06:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dadf8b950 con 0x564dae142ea0 2024-04-25T10:06:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x564dae02b130 con 0x7fe028003100 2024-04-25T10:06:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03eb1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564dae09b690 0x564dae142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03eb1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564dae09b690 0x564dae142960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39272/0 (socket says 172.21.15.60:39272) 2024-04-25T10:06:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03eb1f640 1 -- 172.21.15.60:0/336892643 learned_addr learned my addr 172.21.15.60:0/336892643 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03fb21640 1 --2- 172.21.15.60:0/336892643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dae142ea0 0x564dae09bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03f320640 1 --2- 172.21.15.60:0/336892643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe028003100 0x564dae010220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03eb1f640 1 -- 172.21.15.60:0/336892643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe028003100 msgr2=0x564dae010220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03eb1f640 1 --2- 172.21.15.60:0/336892643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe028003100 0x564dae010220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03eb1f640 1 -- 172.21.15.60:0/336892643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dae142ea0 msgr2=0x564dae09bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03eb1f640 1 --2- 172.21.15.60:0/336892643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dae142ea0 0x564dae09bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03eb1f640 1 -- 172.21.15.60:0/336892643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dae011da0 con 0x564dae09b690 2024-04-25T10:06:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03f320640 1 --2- 172.21.15.60:0/336892643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe028003100 0x564dae010220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:51.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03eb1f640 1 --2- 172.21.15.60:0/336892643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564dae09b690 0x564dae142960 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe03401b3b0 tx=0x7fe034001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a62e3617b04999 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03db1d640 1 -- 172.21.15.60:0/336892643 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0340b0040 con 0x564dae09b690 2024-04-25T10:06:51.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03db1d640 1 -- 172.21.15.60:0/336892643 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0340b5640 con 0x564dae09b690 2024-04-25T10:06:51.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03db1d640 1 -- 172.21.15.60:0/336892643 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0340b5a10 con 0x564dae09b690 2024-04-25T10:06:51.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 -- 172.21.15.60:0/336892643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564dae09b690 msgr2=0x564dae142960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 --2- 172.21.15.60:0/336892643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564dae09b690 0x564dae142960 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe03401b3b0 tx=0x7fe034001130 comp rx=0 tx=0).stop 2024-04-25T10:06:51.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 -- 172.21.15.60:0/336892643 shutdown_connections 2024-04-25T10:06:51.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 --2- 172.21.15.60:0/336892643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dae142ea0 0x564dae09bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 --2- 172.21.15.60:0/336892643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564dae09b690 0x564dae142960 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 --2- 172.21.15.60:0/336892643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe028003100 0x564dae010220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 -- 172.21.15.60:0/336892643 >> 172.21.15.60:0/336892643 conn(0x564dae00be80 msgr2=0x564dae0273c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 -- 172.21.15.60:0/336892643 shutdown_connections 2024-04-25T10:06:51.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 -- 172.21.15.60:0/336892643 wait complete. 2024-04-25T10:06:51.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 Processor -- start 2024-04-25T10:06:51.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 -- start start 2024-04-25T10:06:51.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dae09b690 0x564dae14ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564dae142ea0 0x564dae1557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe028003100 0x564dae155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dae011b40 con 0x564dae142ea0 2024-04-25T10:06:51.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dae010760 con 0x564dae09b690 2024-04-25T10:06:51.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe041802080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x564dae012040 con 0x7fe028003100 2024-04-25T10:06:51.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03eb1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564dae142ea0 0x564dae1557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03eb1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564dae142ea0 0x564dae1557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39280/0 (socket says 172.21.15.60:39280) 2024-04-25T10:06:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03eb1f640 1 -- 172.21.15.60:0/3227878561 learned_addr learned my addr 172.21.15.60:0/3227878561 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03fb21640 1 --2- 172.21.15.60:0/3227878561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe028003100 0x564dae155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03eb1f640 1 -- 172.21.15.60:0/3227878561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe028003100 msgr2=0x564dae155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03eb1f640 1 --2- 172.21.15.60:0/3227878561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe028003100 0x564dae155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03f320640 1 --2- 172.21.15.60:0/3227878561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dae09b690 0x564dae14ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.403+0000 7fe03eb1f640 1 -- 172.21.15.60:0/3227878561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dae09b690 msgr2=0x564dae14ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03eb1f640 1 --2- 172.21.15.60:0/3227878561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dae09b690 0x564dae14ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03eb1f640 1 -- 172.21.15.60:0/3227878561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dae09eee0 con 0x564dae142ea0 2024-04-25T10:06:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03fb21640 1 --2- 172.21.15.60:0/3227878561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe028003100 0x564dae155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03f320640 1 --2- 172.21.15.60:0/3227878561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564dae09b690 0x564dae14ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03eb1f640 1 --2- 172.21.15.60:0/3227878561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564dae142ea0 0x564dae1557b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe034000c10 tx=0x7fe028004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03cb1b640 1 -- 172.21.15.60:0/3227878561 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0340ab600 con 0x564dae142ea0 2024-04-25T10:06:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03cb1b640 1 -- 172.21.15.60:0/3227878561 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0340bb070 con 0x564dae142ea0 2024-04-25T10:06:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03cb1b640 1 -- 172.21.15.60:0/3227878561 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0340ab9c0 con 0x564dae142ea0 2024-04-25T10:06:51.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe041802080 1 -- 172.21.15.60:0/3227878561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564dae09f180 con 0x564dae142ea0 2024-04-25T10:06:51.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe041802080 1 -- 172.21.15.60:0/3227878561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564dae145100 con 0x564dae142ea0 2024-04-25T10:06:51.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03cb1b640 1 -- 172.21.15.60:0/3227878561 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0340abb60 con 0x564dae142ea0 2024-04-25T10:06:51.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03cb1b640 1 --2- 172.21.15.60:0/3227878561 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe02c065970 0x7fe02c067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03cb1b640 1 -- 172.21.15.60:0/3227878561 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe03411eee0 con 0x564dae142ea0 2024-04-25T10:06:51.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe041802080 1 --2- 172.21.15.60:0/3227878561 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x564dae09ea40 0x564dae1455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe041802080 1 -- 172.21.15.60:0/3227878561 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:1 5.36 5:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v9 -- 0x564dae14cf00 con 0x564dae09ea40 2024-04-25T10:06:51.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03f320640 1 --2- 172.21.15.60:0/3227878561 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe02c065970 0x7fe02c067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03fb21640 1 --2- 172.21.15.60:0/3227878561 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x564dae09ea40 0x564dae1455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03fb21640 1 --2- 172.21.15.60:0/3227878561 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x564dae09ea40 0x564dae1455a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03f320640 1 --2- 172.21.15.60:0/3227878561 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe02c065970 0x7fe02c067e30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x564dadfcb870 tx=0x7fe0380021a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03fb21640 1 -- 172.21.15.60:0/3227878561 <== osd.1 v2:172.21.15.60:6824/3313552458 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fe030009760 con 0x564dae09ea40 2024-04-25T10:06:51.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe041802080 1 -- 172.21.15.60:0/3227878561 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x564dae09ea40 msgr2=0x564dae1455a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe041802080 1 --2- 172.21.15.60:0/3227878561 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x564dae09ea40 0x564dae1455a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe041802080 1 -- 172.21.15.60:0/3227878561 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe02c065970 msgr2=0x7fe02c067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe041802080 1 --2- 172.21.15.60:0/3227878561 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe02c065970 0x7fe02c067e30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x564dadfcb870 tx=0x7fe0380021a0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe041802080 1 -- 172.21.15.60:0/3227878561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564dae142ea0 msgr2=0x564dae1557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe041802080 1 --2- 172.21.15.60:0/3227878561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564dae142ea0 0x564dae1557b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe034000c10 tx=0x7fe028004130 comp rx=0 tx=0).stop 2024-04-25T10:06:51.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe03fb21640 1 -- 172.21.15.60:0/3227878561 reap_dead start 2024-04-25T10:06:51.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe041802080 1 -- 172.21.15.60:0/3227878561 shutdown_connections 2024-04-25T10:06:51.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.407+0000 7fe041802080 1 -- 172.21.15.60:0/3227878561 >> 172.21.15.60:0/3227878561 conn(0x564dae00be80 msgr2=0x564dae028800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.411+0000 7fe041802080 1 -- 172.21.15.60:0/3227878561 shutdown_connections 2024-04-25T10:06:51.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.411+0000 7fe041802080 1 -- 172.21.15.60:0/3227878561 wait complete. 2024-04-25T10:06:51.420 INFO:teuthology.orchestra.run.smithi060.stdout:mytag 2024-04-25T10:06:51.420 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-25T10:06:51.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.491+0000 7f8059e2e080 1 Processor -- start 2024-04-25T10:06:51.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.491+0000 7f8059e2e080 1 -- start start 2024-04-25T10:06:51.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8044002820 0x55d16610d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d16619e6f0 0x55d1660f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d16619d1e0 0x55d1660f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16606f9f0 con 0x7f8044002820 2024-04-25T10:06:51.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d166073950 con 0x55d16619e6f0 2024-04-25T10:06:51.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1660fb130 con 0x55d16619d1e0 2024-04-25T10:06:51.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805794c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8044002820 0x55d16610d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805794c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8044002820 0x55d16610d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39306/0 (socket says 172.21.15.60:39306) 2024-04-25T10:06:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805794c640 1 -- 172.21.15.60:0/765048083 learned_addr learned my addr 172.21.15.60:0/765048083 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805714b640 1 --2- 172.21.15.60:0/765048083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d16619e6f0 0x55d1660f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805814d640 1 --2- 172.21.15.60:0/765048083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d16619d1e0 0x55d1660f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805794c640 1 -- 172.21.15.60:0/765048083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d16619d1e0 msgr2=0x55d1660f82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805794c640 1 --2- 172.21.15.60:0/765048083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d16619d1e0 0x55d1660f82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805794c640 1 -- 172.21.15.60:0/765048083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d16619e6f0 msgr2=0x55d1660f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805794c640 1 --2- 172.21.15.60:0/765048083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d16619e6f0 0x55d1660f7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805794c640 1 -- 172.21.15.60:0/765048083 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d16619f8a0 con 0x7f8044002820 2024-04-25T10:06:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805814d640 1 --2- 172.21.15.60:0/765048083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d16619d1e0 0x55d1660f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805794c640 1 --2- 172.21.15.60:0/765048083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8044002820 0x55d16610d360 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f80500044a0 tx=0x7f8050004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac4c2526c53d43d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8056149640 1 -- 172.21.15.60:0/765048083 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8050018070 con 0x7f8044002820 2024-04-25T10:06:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8056149640 1 -- 172.21.15.60:0/765048083 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805000f510 con 0x7f8044002820 2024-04-25T10:06:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8056149640 1 -- 172.21.15.60:0/765048083 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805000f8e0 con 0x7f8044002820 2024-04-25T10:06:51.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 -- 172.21.15.60:0/765048083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8044002820 msgr2=0x55d16610d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 --2- 172.21.15.60:0/765048083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8044002820 0x55d16610d360 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f80500044a0 tx=0x7f8050004960 comp rx=0 tx=0).stop 2024-04-25T10:06:51.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 -- 172.21.15.60:0/765048083 shutdown_connections 2024-04-25T10:06:51.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 --2- 172.21.15.60:0/765048083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d16619d1e0 0x55d1660f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 --2- 172.21.15.60:0/765048083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d16619e6f0 0x55d1660f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 --2- 172.21.15.60:0/765048083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8044002820 0x55d16610d360 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 -- 172.21.15.60:0/765048083 >> 172.21.15.60:0/765048083 conn(0x55d1660c9750 msgr2=0x55d16619ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 -- 172.21.15.60:0/765048083 shutdown_connections 2024-04-25T10:06:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 -- 172.21.15.60:0/765048083 wait complete. 2024-04-25T10:06:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 Processor -- start 2024-04-25T10:06:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 -- start start 2024-04-25T10:06:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d16619d1e0 0x55d16618a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d16619e6f0 0x55d16618a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8044002820 0x55d1661883b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d166199ca0 con 0x55d16619e6f0 2024-04-25T10:06:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d166199a00 con 0x55d16619d1e0 2024-04-25T10:06:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f8059e2e080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1660f78f0 con 0x7f8044002820 2024-04-25T10:06:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805714b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d16619e6f0 0x55d16618a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805714b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d16619e6f0 0x55d16618a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39308/0 (socket says 172.21.15.60:39308) 2024-04-25T10:06:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805714b640 1 -- 172.21.15.60:0/1538190126 learned_addr learned my addr 172.21.15.60:0/1538190126 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805814d640 1 --2- 172.21.15.60:0/1538190126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8044002820 0x55d1661883b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805794c640 1 --2- 172.21.15.60:0/1538190126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d16619d1e0 0x55d16618a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805714b640 1 -- 172.21.15.60:0/1538190126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8044002820 msgr2=0x55d1661883b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805714b640 1 --2- 172.21.15.60:0/1538190126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8044002820 0x55d1661883b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805714b640 1 -- 172.21.15.60:0/1538190126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d16619d1e0 msgr2=0x55d16618a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805714b640 1 --2- 172.21.15.60:0/1538190126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d16619d1e0 0x55d16618a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.495+0000 7f805714b640 1 -- 172.21.15.60:0/1538190126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d16618af80 con 0x55d16619e6f0 2024-04-25T10:06:51.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f805714b640 1 --2- 172.21.15.60:0/1538190126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d16619e6f0 0x55d16618a770 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8040096910 tx=0x7f8040017db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8055147640 1 -- 172.21.15.60:0/1538190126 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80400b9070 con 0x55d16619e6f0 2024-04-25T10:06:51.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8055147640 1 -- 172.21.15.60:0/1538190126 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80400aba40 con 0x55d16619e6f0 2024-04-25T10:06:51.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8055147640 1 -- 172.21.15.60:0/1538190126 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80400b0040 con 0x55d16619e6f0 2024-04-25T10:06:51.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8059e2e080 1 -- 172.21.15.60:0/1538190126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d16618b220 con 0x55d16619e6f0 2024-04-25T10:06:51.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8059e2e080 1 -- 172.21.15.60:0/1538190126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d166239d20 con 0x55d16619e6f0 2024-04-25T10:06:51.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8055147640 1 -- 172.21.15.60:0/1538190126 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f80400be450 con 0x55d16619e6f0 2024-04-25T10:06:51.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8055147640 1 --2- 172.21.15.60:0/1538190126 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8038065940 0x7f8038067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8055147640 1 -- 172.21.15.60:0/1538190126 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f804011dc80 con 0x55d16619e6f0 2024-04-25T10:06:51.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8059e2e080 1 --2- 172.21.15.60:0/1538190126 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55d16623b1d0 0x55d1661865e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f805794c640 1 --2- 172.21.15.60:0/1538190126 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8038065940 0x7f8038067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8059e2e080 1 -- 172.21.15.60:0/1538190126 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:1 5.19 5:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v9 -- 0x55d166186b20 con 0x55d16623b1d0 2024-04-25T10:06:51.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f805814d640 1 --2- 172.21.15.60:0/1538190126 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55d16623b1d0 0x55d1661865e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f805814d640 1 --2- 172.21.15.60:0/1538190126 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55d16623b1d0 0x55d1661865e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f805794c640 1 --2- 172.21.15.60:0/1538190126 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8038065940 0x7f8038067e00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8050009040 tx=0x7f8050006070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f805814d640 1 -- 172.21.15.60:0/1538190126 <== osd.1 v2:172.21.15.60:6824/3313552458 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f8048009810 con 0x55d16623b1d0 2024-04-25T10:06:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8059e2e080 1 -- 172.21.15.60:0/1538190126 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55d16623b1d0 msgr2=0x55d1661865e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8059e2e080 1 --2- 172.21.15.60:0/1538190126 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55d16623b1d0 0x55d1661865e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8059e2e080 1 -- 172.21.15.60:0/1538190126 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8038065940 msgr2=0x7f8038067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8059e2e080 1 --2- 172.21.15.60:0/1538190126 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8038065940 0x7f8038067e00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8050009040 tx=0x7f8050006070 comp rx=0 tx=0).stop 2024-04-25T10:06:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8059e2e080 1 -- 172.21.15.60:0/1538190126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d16619e6f0 msgr2=0x55d16618a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8059e2e080 1 --2- 172.21.15.60:0/1538190126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d16619e6f0 0x55d16618a770 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8040096910 tx=0x7f8040017db0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f805814d640 1 -- 172.21.15.60:0/1538190126 reap_dead start 2024-04-25T10:06:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8059e2e080 1 -- 172.21.15.60:0/1538190126 shutdown_connections 2024-04-25T10:06:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8059e2e080 1 -- 172.21.15.60:0/1538190126 >> 172.21.15.60:0/1538190126 conn(0x55d1660c9750 msgr2=0x55d1660f7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.499+0000 7f8059e2e080 1 -- 172.21.15.60:0/1538190126 shutdown_connections 2024-04-25T10:06:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.503+0000 7f8059e2e080 1 -- 172.21.15.60:0/1538190126 wait complete. 2024-04-25T10:06:51.511 INFO:teuthology.orchestra.run.smithi060.stdout:mytag 2024-04-25T10:06:51.512 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-25T10:06:51.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f534271b080 1 Processor -- start 2024-04-25T10:06:51.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f534271b080 1 -- start start 2024-04-25T10:06:51.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f534271b080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5607b0344c20 0x5607b03b90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f534271b080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607b0345000 0x5607b03438d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f534271b080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b0345f10 0x5607b02fc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f534271b080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b0348460 con 0x5607b0345000 2024-04-25T10:06:51.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f534271b080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b02a49f0 con 0x5607b0345f10 2024-04-25T10:06:51.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f534271b080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607b03bc740 con 0x5607b0344c20 2024-04-25T10:06:51.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f533fa38640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607b0345000 0x5607b03438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f533fa38640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607b0345000 0x5607b03438d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39310/0 (socket says 172.21.15.60:39310) 2024-04-25T10:06:51.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f533fa38640 1 -- 172.21.15.60:0/4260522951 learned_addr learned my addr 172.21.15.60:0/4260522951 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:51.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f5340239640 1 --2- 172.21.15.60:0/4260522951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5607b0344c20 0x5607b03b90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f5340a3a640 1 --2- 172.21.15.60:0/4260522951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b0345f10 0x5607b02fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f533fa38640 1 -- 172.21.15.60:0/4260522951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5607b0344c20 msgr2=0x5607b03b90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f533fa38640 1 --2- 172.21.15.60:0/4260522951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5607b0344c20 0x5607b03b90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f533fa38640 1 -- 172.21.15.60:0/4260522951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b0345f10 msgr2=0x5607b02fc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f533fa38640 1 --2- 172.21.15.60:0/4260522951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b0345f10 0x5607b02fc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.583+0000 7f533fa38640 1 -- 172.21.15.60:0/4260522951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607b0343e10 con 0x5607b0345000 2024-04-25T10:06:51.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f533fa38640 1 --2- 172.21.15.60:0/4260522951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607b0345000 0x5607b03438d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f533400b330 tx=0x7f533400b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f812a30dbed71ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f533ea36640 1 -- 172.21.15.60:0/4260522951 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5334012020 con 0x5607b0345000 2024-04-25T10:06:51.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f533ea36640 1 -- 172.21.15.60:0/4260522951 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533400d6b0 con 0x5607b0345000 2024-04-25T10:06:51.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f533ea36640 1 -- 172.21.15.60:0/4260522951 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533400da80 con 0x5607b0345000 2024-04-25T10:06:51.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 -- 172.21.15.60:0/4260522951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607b0345000 msgr2=0x5607b03438d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 --2- 172.21.15.60:0/4260522951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607b0345000 0x5607b03438d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f533400b330 tx=0x7f533400b7f0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 -- 172.21.15.60:0/4260522951 shutdown_connections 2024-04-25T10:06:51.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 --2- 172.21.15.60:0/4260522951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b0345f10 0x5607b02fc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 --2- 172.21.15.60:0/4260522951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607b0345000 0x5607b03438d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 --2- 172.21.15.60:0/4260522951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5607b0344c20 0x5607b03b90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 -- 172.21.15.60:0/4260522951 >> 172.21.15.60:0/4260522951 conn(0x5607b0328e80 msgr2=0x5607b03bb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 -- 172.21.15.60:0/4260522951 shutdown_connections 2024-04-25T10:06:51.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 -- 172.21.15.60:0/4260522951 wait complete. 2024-04-25T10:06:51.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 Processor -- start 2024-04-25T10:06:51.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 -- start start 2024-04-25T10:06:51.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607b0344c20 0x5607b04751c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b0345000 0x5607b03d6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5607b0345f10 0x5607b03d7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b03d4860 con 0x5607b0344c20 2024-04-25T10:06:51.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b03beb20 con 0x5607b0345000 2024-04-25T10:06:51.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607b032e510 con 0x5607b0345f10 2024-04-25T10:06:51.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f5340239640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607b0344c20 0x5607b04751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f5340239640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607b0344c20 0x5607b04751c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39320/0 (socket says 172.21.15.60:39320) 2024-04-25T10:06:51.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f5340239640 1 -- 172.21.15.60:0/23916558 learned_addr learned my addr 172.21.15.60:0/23916558 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:51.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f533fa38640 1 --2- 172.21.15.60:0/23916558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b0345000 0x5607b03d6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f5340a3a640 1 --2- 172.21.15.60:0/23916558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5607b0345f10 0x5607b03d7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f5340239640 1 -- 172.21.15.60:0/23916558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5607b0345f10 msgr2=0x5607b03d7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f5340239640 1 --2- 172.21.15.60:0/23916558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5607b0345f10 0x5607b03d7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f5340239640 1 -- 172.21.15.60:0/23916558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b0345000 msgr2=0x5607b03d6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f5340239640 1 --2- 172.21.15.60:0/23916558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607b0345000 0x5607b03d6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f5340239640 1 -- 172.21.15.60:0/23916558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607b032e7b0 con 0x5607b0344c20 2024-04-25T10:06:51.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f5340239640 1 --2- 172.21.15.60:0/23916558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607b0344c20 0x5607b04751c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5330008880 tx=0x7f5330008d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f533da34640 1 -- 172.21.15.60:0/23916558 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330006d80 con 0x5607b0344c20 2024-04-25T10:06:51.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f533da34640 1 -- 172.21.15.60:0/23916558 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533001a070 con 0x5607b0344c20 2024-04-25T10:06:51.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 -- 172.21.15.60:0/23916558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5607b032ea50 con 0x5607b0344c20 2024-04-25T10:06:51.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f533da34640 1 -- 172.21.15.60:0/23916558 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330015680 con 0x5607b0344c20 2024-04-25T10:06:51.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.587+0000 7f534271b080 1 -- 172.21.15.60:0/23916558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607b032ef90 con 0x5607b0344c20 2024-04-25T10:06:51.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f533da34640 1 -- 172.21.15.60:0/23916558 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5330013030 con 0x5607b0344c20 2024-04-25T10:06:51.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f533da34640 1 --2- 172.21.15.60:0/23916558 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f532c065970 0x7f532c067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f533da34640 1 -- 172.21.15.60:0/23916558 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5330084530 con 0x5607b0344c20 2024-04-25T10:06:51.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f534271b080 1 --2- 172.21.15.60:0/23916558 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5607b0472710 0x5607b0472af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f534271b080 1 -- 172.21.15.60:0/23916558 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:1 5.12 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v9 -- 0x5607b0473030 con 0x5607b0472710 2024-04-25T10:06:51.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f533fa38640 1 --2- 172.21.15.60:0/23916558 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f532c065970 0x7f532c067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f5340a3a640 1 --2- 172.21.15.60:0/23916558 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5607b0472710 0x5607b0472af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f5340a3a640 1 --2- 172.21.15.60:0/23916558 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5607b0472710 0x5607b0472af0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f5340a3a640 1 -- 172.21.15.60:0/23916558 <== osd.1 v2:172.21.15.60:6824/3313552458 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f53380ab5f0 con 0x5607b0472710 2024-04-25T10:06:51.602 INFO:teuthology.orchestra.run.smithi060.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: 2024-04-25T10:06:51.591+0000 7f533fa38640 1 --2- 172.21.15.60:0/23916558 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f532c065970 0x7f532c067e30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f533400b330 tx=0x7f533401b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.602 INFO:teuthology.orchestra.run.smithi060.stderr:(61) No data available 2024-04-25T10:06:51.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f534271b080 1 -- 172.21.15.60:0/23916558 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5607b0472710 msgr2=0x5607b0472af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f534271b080 1 --2- 172.21.15.60:0/23916558 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5607b0472710 0x5607b0472af0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f534271b080 1 -- 172.21.15.60:0/23916558 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f532c065970 msgr2=0x7f532c067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f534271b080 1 --2- 172.21.15.60:0/23916558 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f532c065970 0x7f532c067e30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f533400b330 tx=0x7f533401b040 comp rx=0 tx=0).stop 2024-04-25T10:06:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f534271b080 1 -- 172.21.15.60:0/23916558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607b0344c20 msgr2=0x5607b04751c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f534271b080 1 --2- 172.21.15.60:0/23916558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5607b0344c20 0x5607b04751c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5330008880 tx=0x7f5330008d40 comp rx=0 tx=0).stop 2024-04-25T10:06:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f5340a3a640 1 -- 172.21.15.60:0/23916558 reap_dead start 2024-04-25T10:06:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f534271b080 1 -- 172.21.15.60:0/23916558 shutdown_connections 2024-04-25T10:06:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f534271b080 1 -- 172.21.15.60:0/23916558 >> 172.21.15.60:0/23916558 conn(0x5607b0328e80 msgr2=0x5607b032a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f534271b080 1 -- 172.21.15.60:0/23916558 shutdown_connections 2024-04-25T10:06:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:51.591+0000 7f534271b080 1 -- 172.21.15.60:0/23916558 wait complete. 2024-04-25T10:06:51.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:06:51.604 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:06:51.604 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:51.698 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:51.745 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:06:51.746 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:06:51.746 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:51.752 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:51.752 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:06:51.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:51.752 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:06:51.801 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:06:51.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:06:51.802 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:06:51.803 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:51.809 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:51.809 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:06:51.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:51.809 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:06:51.856 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:06:51.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:06:51.857 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:06:51.857 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:51.863 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:51.863 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:06:51.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:51.864 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:06:51.912 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:06:51.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:06:51.914 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:06:51.914 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:51.919 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:51.920 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:06:51.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:51.920 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:06:51.968 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:06:51.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:06:51.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:52.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.095+0000 7f3af0c05640 1 Processor -- start 2024-04-25T10:06:52.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.099+0000 7f3af0c05640 1 -- start start 2024-04-25T10:06:52.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.099+0000 7f3af0c05640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3aec0b4660 0x7f3aec15c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.099+0000 7f3af0c05640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aec071ca0 con 0x7f3aec15bc90 2024-04-25T10:06:52.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.099+0000 7f3af0c05640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aec0cca40 con 0x7f3aec0ce7a0 2024-04-25T10:06:52.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.099+0000 7f3af0c05640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aec0cc370 con 0x7f3aec0b4660 2024-04-25T10:06:52.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.099+0000 7f3aead9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3aec0b4660 0x7f3aec15c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.099+0000 7f3ae9d9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3aec15bc90 0x7f3aec15c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48496/0 (socket says 172.21.15.60:48496) 2024-04-25T10:06:52.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.099+0000 7f3ae9d9b640 1 -- 172.21.15.60:0/4025036530 learned_addr learned my addr 172.21.15.60:0/4025036530 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:52.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.099+0000 7f3aead9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3aec0b4660 0x7f3aec15c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39330/0 (socket says 172.21.15.60:39330) 2024-04-25T10:06:52.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.099+0000 7f3aead9d640 1 -- 172.21.15.60:0/4025036530 >> v1:172.21.15.100:6789/0 conn(0x7f3aec0ce7a0 legacy=0x7f3aec0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.099+0000 7f3aead9d640 1 -- 172.21.15.60:0/4025036530 >> v1:172.21.15.60:6789/0 conn(0x7f3aec15bc90 legacy=0x7f3aec15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.099+0000 7f3aead9d640 1 -- 172.21.15.60:0/4025036530 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec0ccd20 con 0x7f3aec0b4660 2024-04-25T10:06:52.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3aead9d640 1 --2- 172.21.15.60:0/4025036530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3aec0b4660 0x7f3aec15c930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40097c0 tx=0x7f3ae401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1613005c1aad86e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3ae959a640 1 -- 172.21.15.60:0/4025036530 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae40b0070 con 0x7f3aec0b4660 2024-04-25T10:06:52.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3ae959a640 1 -- 172.21.15.60:0/4025036530 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae40ab600 con 0x7f3aec0b4660 2024-04-25T10:06:52.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3ae959a640 1 -- 172.21.15.60:0/4025036530 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae40ab9d0 con 0x7f3aec0b4660 2024-04-25T10:06:52.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3af0c05640 1 -- 172.21.15.60:0/4025036530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3aec0b4660 msgr2=0x7f3aec15c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3af0c05640 1 --2- 172.21.15.60:0/4025036530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3aec0b4660 0x7f3aec15c930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40097c0 tx=0x7f3ae401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3af0c05640 1 -- 172.21.15.60:0/4025036530 shutdown_connections 2024-04-25T10:06:52.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3af0c05640 1 --2- 172.21.15.60:0/4025036530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3aec0b4660 0x7f3aec15c930 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3af0c05640 1 -- 172.21.15.60:0/4025036530 >> 172.21.15.60:0/4025036530 conn(0x7f3aec0b12d0 msgr2=0x7f3aec0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3af0c05640 1 -- 172.21.15.60:0/4025036530 shutdown_connections 2024-04-25T10:06:52.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3af0c05640 1 -- 172.21.15.60:0/4025036530 wait complete. 2024-04-25T10:06:52.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3af0c05640 1 Processor -- start 2024-04-25T10:06:52.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3af0c05640 1 -- start start 2024-04-25T10:06:52.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3af0c05640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3aec0ce7a0 0x7f3aec152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3af0c05640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3aec15bc90 0x7f3aec147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3af0c05640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3aec154a90 0x7f3aec14ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3af0c05640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec1695d0 con 0x7f3aec15bc90 2024-04-25T10:06:52.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3af0c05640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec0b3db0 con 0x7f3aec154a90 2024-04-25T10:06:52.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3af0c05640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec169750 con 0x7f3aec0ce7a0 2024-04-25T10:06:52.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3ae9d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3aec15bc90 0x7f3aec147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3ae9d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3aec15bc90 0x7f3aec147c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39340/0 (socket says 172.21.15.60:39340) 2024-04-25T10:06:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3ae9d9b640 1 -- 172.21.15.60:0/2308560460 learned_addr learned my addr 172.21.15.60:0/2308560460 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3aea59c640 1 --2- 172.21.15.60:0/2308560460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3aec0ce7a0 0x7f3aec152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3aead9d640 1 --2- 172.21.15.60:0/2308560460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3aec154a90 0x7f3aec14ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3ae9d9b640 1 -- 172.21.15.60:0/2308560460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3aec0ce7a0 msgr2=0x7f3aec152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3ae9d9b640 1 --2- 172.21.15.60:0/2308560460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3aec0ce7a0 0x7f3aec152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3ae9d9b640 1 -- 172.21.15.60:0/2308560460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3aec154a90 msgr2=0x7f3aec14ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3ae9d9b640 1 --2- 172.21.15.60:0/2308560460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3aec154a90 0x7f3aec14ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3ae9d9b640 1 -- 172.21.15.60:0/2308560460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec155030 con 0x7f3aec15bc90 2024-04-25T10:06:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3aead9d640 1 --2- 172.21.15.60:0/2308560460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3aec154a90 0x7f3aec14ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3aea59c640 1 --2- 172.21.15.60:0/2308560460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3aec0ce7a0 0x7f3aec152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.103+0000 7f3ae9d9b640 1 --2- 172.21.15.60:0/2308560460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3aec15bc90 0x7f3aec147c70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c500 tx=0x7f3ad400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.107+0000 7f3ae2ffd640 1 -- 172.21.15.60:0/2308560460 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4017070 con 0x7f3aec15bc90 2024-04-25T10:06:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.107+0000 7f3ae2ffd640 1 -- 172.21.15.60:0/2308560460 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad4005cf0 con 0x7f3aec15bc90 2024-04-25T10:06:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.107+0000 7f3af0c05640 1 -- 172.21.15.60:0/2308560460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aec155310 con 0x7f3aec15bc90 2024-04-25T10:06:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.107+0000 7f3ae2ffd640 1 -- 172.21.15.60:0/2308560460 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad40072f0 con 0x7f3aec15bc90 2024-04-25T10:06:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.107+0000 7f3af0c05640 1 -- 172.21.15.60:0/2308560460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aec1558d0 con 0x7f3aec15bc90 2024-04-25T10:06:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.107+0000 7f3af0c05640 1 -- 172.21.15.60:0/2308560460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aec074770 con 0x7f3aec15bc90 2024-04-25T10:06:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.107+0000 7f3ae2ffd640 1 -- 172.21.15.60:0/2308560460 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ad401a070 con 0x7f3aec15bc90 2024-04-25T10:06:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.107+0000 7f3ae2ffd640 1 --2- 172.21.15.60:0/2308560460 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3acc066f60 0x7f3acc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.107+0000 7f3ae2ffd640 1 -- 172.21.15.60:0/2308560460 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ad40842c0 con 0x7f3aec15bc90 2024-04-25T10:06:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.107+0000 7f3aea59c640 1 --2- 172.21.15.60:0/2308560460 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3acc066f60 0x7f3acc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.107+0000 7f3aea59c640 1 --2- 172.21.15.60:0/2308560460 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3acc066f60 0x7f3acc069420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3aec159470 tx=0x7f3adc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.111+0000 7f3ae2ffd640 1 -- 172.21.15.60:0/2308560460 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ad4054d10 con 0x7f3aec15bc90 2024-04-25T10:06:52.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.307+0000 7f3af0c05640 1 -- 172.21.15.60:0/2308560460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3aec0b3260 con 0x7f3aec15bc90 2024-04-25T10:06:52.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.307+0000 7f3ae2ffd640 1 -- 172.21.15.60:0/2308560460 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f3ad40840e0 con 0x7f3aec15bc90 2024-04-25T10:06:52.315 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:52.315 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2422261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2422261630},{"type":"v1","addr":"172.21.15.100:6835","nonce":2422261630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2923340508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2923340508},{"type":"v1","addr":"172.21.15.100:6837","nonce":2923340508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1929386449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1929386449},{"type":"v1","addr":"172.21.15.60:6833","nonce":1929386449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:21.529455+0000","modified":"2024-04-25T10:06:23.575106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5283":{"gid":5283,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3681296171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3681296171},{"type":"v1","addr":"172.21.15.60:6835","nonce":3681296171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:52.315 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 15 2024-04-25T10:06:52.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.311+0000 7f3af0c05640 1 -- 172.21.15.60:0/2308560460 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3acc066f60 msgr2=0x7f3acc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.311+0000 7f3af0c05640 1 --2- 172.21.15.60:0/2308560460 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3acc066f60 0x7f3acc069420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3aec159470 tx=0x7f3adc009040 comp rx=0 tx=0).stop 2024-04-25T10:06:52.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.311+0000 7f3af0c05640 1 -- 172.21.15.60:0/2308560460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3aec15bc90 msgr2=0x7f3aec147c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.311+0000 7f3af0c05640 1 --2- 172.21.15.60:0/2308560460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3aec15bc90 0x7f3aec147c70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c500 tx=0x7f3ad400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.311+0000 7f3af0c05640 1 -- 172.21.15.60:0/2308560460 shutdown_connections 2024-04-25T10:06:52.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.311+0000 7f3af0c05640 1 --2- 172.21.15.60:0/2308560460 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3acc066f60 0x7f3acc069420 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.311+0000 7f3af0c05640 1 --2- 172.21.15.60:0/2308560460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3aec154a90 0x7f3aec14ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.311+0000 7f3af0c05640 1 --2- 172.21.15.60:0/2308560460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3aec15bc90 0x7f3aec147c70 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.311+0000 7f3af0c05640 1 --2- 172.21.15.60:0/2308560460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3aec0ce7a0 0x7f3aec152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.311+0000 7f3af0c05640 1 -- 172.21.15.60:0/2308560460 >> 172.21.15.60:0/2308560460 conn(0x7f3aec0b12d0 msgr2=0x7f3aec15ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.311+0000 7f3af0c05640 1 -- 172.21.15.60:0/2308560460 shutdown_connections 2024-04-25T10:06:52.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.311+0000 7f3af0c05640 1 -- 172.21.15.60:0/2308560460 wait complete. 2024-04-25T10:06:52.335 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:52.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.455+0000 7fa5ba4b9640 1 Processor -- start 2024-04-25T10:06:52.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.455+0000 7fa5ba4b9640 1 -- start start 2024-04-25T10:06:52.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.459+0000 7fa5ba4b9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa5b4162250 0x7fa5b40b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.459+0000 7fa5ba4b9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b40b3aa0 con 0x7fa5b40b34f0 2024-04-25T10:06:52.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.459+0000 7fa5ba4b9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b40b3d80 con 0x7fa5b40719a0 2024-04-25T10:06:52.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.459+0000 7fa5ba4b9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b40af0c0 con 0x7fa5b4162250 2024-04-25T10:06:52.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.459+0000 7fa5b3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa5b4162250 0x7fa5b40b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.459+0000 7fa5b37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa5b40b34f0 0x7fa5b4071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48512/0 (socket says 172.21.15.60:48512) 2024-04-25T10:06:52.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.459+0000 7fa5b37fe640 1 -- 172.21.15.60:0/1873670727 learned_addr learned my addr 172.21.15.60:0/1873670727 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:52.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.459+0000 7fa5b3fff640 1 --2- 172.21.15.60:0/1873670727 >> v2:172.21.15.60:3300/0 conn(0x7fa5b4162250 0x7fa5b40b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39350/0 (socket says 172.21.15.60:39350) 2024-04-25T10:06:52.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.459+0000 7fa5b3fff640 1 -- 172.21.15.60:0/1873670727 >> v1:172.21.15.100:6789/0 conn(0x7fa5b40719a0 legacy=0x7fa5b40cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.459+0000 7fa5b3fff640 1 -- 172.21.15.60:0/1873670727 >> v1:172.21.15.60:6789/0 conn(0x7fa5b40b34f0 legacy=0x7fa5b4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.459+0000 7fa5b3fff640 1 -- 172.21.15.60:0/1873670727 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b40b4060 con 0x7fa5b4162250 2024-04-25T10:06:52.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.459+0000 7fa5b2ffd640 1 -- 172.21.15.60:0/1873670727 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541917569 0 0) 0x7fa5b40b3aa0 con 0x7fa5b40b34f0 2024-04-25T10:06:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.459+0000 7fa5b3fff640 1 --2- 172.21.15.60:0/1873670727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b4162250 0x7fa5b40b2fb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40097c0 tx=0x7fa5a401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746b0759ebb4edad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.459+0000 7fa5b2ffd640 1 -- 172.21.15.60:0/1873670727 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40b5070 con 0x7fa5b4162250 2024-04-25T10:06:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.459+0000 7fa5b2ffd640 1 -- 172.21.15.60:0/1873670727 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a40a8950 con 0x7fa5b4162250 2024-04-25T10:06:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.459+0000 7fa5b2ffd640 1 -- 172.21.15.60:0/1873670727 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40a8d20 con 0x7fa5b4162250 2024-04-25T10:06:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 -- 172.21.15.60:0/1873670727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b4162250 msgr2=0x7fa5b40b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 --2- 172.21.15.60:0/1873670727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b4162250 0x7fa5b40b2fb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40097c0 tx=0x7fa5a401ec50 comp rx=0 tx=0).stop 2024-04-25T10:06:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 -- 172.21.15.60:0/1873670727 shutdown_connections 2024-04-25T10:06:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 --2- 172.21.15.60:0/1873670727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b4162250 0x7fa5b40b2fb0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 -- 172.21.15.60:0/1873670727 >> 172.21.15.60:0/1873670727 conn(0x7fa5b40b12d0 msgr2=0x7fa5b40b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 -- 172.21.15.60:0/1873670727 shutdown_connections 2024-04-25T10:06:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 -- 172.21.15.60:0/1873670727 wait complete. 2024-04-25T10:06:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 Processor -- start 2024-04-25T10:06:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 -- start start 2024-04-25T10:06:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5b40719a0 0x7fa5b4150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5b40b34f0 0x7fa5b41f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b4162250 0x7fa5b40a8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b40cc440 con 0x7fa5b4162250 2024-04-25T10:06:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b406e5a0 con 0x7fa5b40719a0 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b40cd0a0 con 0x7fa5b40b34f0 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5b8a56640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b4162250 0x7fa5b40a8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5b8a56640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b4162250 0x7fa5b40a8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39354/0 (socket says 172.21.15.60:39354) 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5b8a56640 1 -- 172.21.15.60:0/1779416144 learned_addr learned my addr 172.21.15.60:0/1779416144 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5b3fff640 1 --2- 172.21.15.60:0/1779416144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5b40719a0 0x7fa5b4150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5b37fe640 1 --2- 172.21.15.60:0/1779416144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5b40b34f0 0x7fa5b41f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5b8a56640 1 -- 172.21.15.60:0/1779416144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5b40b34f0 msgr2=0x7fa5b41f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5b8a56640 1 --2- 172.21.15.60:0/1779416144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5b40b34f0 0x7fa5b41f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5b8a56640 1 -- 172.21.15.60:0/1779416144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5b40719a0 msgr2=0x7fa5b4150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5b8a56640 1 --2- 172.21.15.60:0/1779416144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5b40719a0 0x7fa5b4150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5b8a56640 1 -- 172.21.15.60:0/1779416144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b414d520 con 0x7fa5b4162250 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5b37fe640 1 --2- 172.21.15.60:0/1779416144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5b40b34f0 0x7fa5b41f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5b8a56640 1 --2- 172.21.15.60:0/1779416144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b4162250 0x7fa5b40a8500 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00c780 tx=0x7fa5ac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5b0ff9640 1 -- 172.21.15.60:0/1779416144 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac016020 con 0x7fa5b4162250 2024-04-25T10:06:52.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5b0ff9640 1 -- 172.21.15.60:0/1779416144 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac00a9f0 con 0x7fa5b4162250 2024-04-25T10:06:52.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 -- 172.21.15.60:0/1779416144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b414d7c0 con 0x7fa5b4162250 2024-04-25T10:06:52.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5b0ff9640 1 -- 172.21.15.60:0/1779416144 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00ade0 con 0x7fa5b4162250 2024-04-25T10:06:52.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.463+0000 7fa5ba4b9640 1 -- 172.21.15.60:0/1779416144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b414dd80 con 0x7fa5b4162250 2024-04-25T10:06:52.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.467+0000 7fa5ba4b9640 1 -- 172.21.15.60:0/1779416144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b4074770 con 0x7fa5b4162250 2024-04-25T10:06:52.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.467+0000 7fa5b0ff9640 1 -- 172.21.15.60:0/1779416144 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5ac019020 con 0x7fa5b4162250 2024-04-25T10:06:52.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.467+0000 7fa5b0ff9640 1 --2- 172.21.15.60:0/1779416144 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa5a0066f60 0x7fa5a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.467+0000 7fa5b0ff9640 1 -- 172.21.15.60:0/1779416144 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5ac084f50 con 0x7fa5b4162250 2024-04-25T10:06:52.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.467+0000 7fa5b3fff640 1 --2- 172.21.15.60:0/1779416144 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa5a0066f60 0x7fa5a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.467+0000 7fa5b3fff640 1 --2- 172.21.15.60:0/1779416144 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa5a0066f60 0x7fa5a0069420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40b2000 tx=0x7fa5a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.471+0000 7fa5b0ff9640 1 -- 172.21.15.60:0/1779416144 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5ac020030 con 0x7fa5b4162250 2024-04-25T10:06:52.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.631+0000 7fa5ba4b9640 1 -- 172.21.15.60:0/1779416144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5b40ccc30 con 0x7fa5b4162250 2024-04-25T10:06:52.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.635+0000 7fa5b0ff9640 1 -- 172.21.15.60:0/1779416144 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16681 (secure 0 0 0) 0x7fa5ac055f60 con 0x7fa5b4162250 2024-04-25T10:06:52.642 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:52.642 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":35,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:23.196554+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:20.088165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:20.507719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.635+0000 7fa5ba4b9640 1 -- 172.21.15.60:0/1779416144 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa5a0066f60 msgr2=0x7fa5a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.635+0000 7fa5ba4b9640 1 --2- 172.21.15.60:0/1779416144 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa5a0066f60 0x7fa5a0069420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40b2000 tx=0x7fa5a4008040 comp rx=0 tx=0).stop 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.635+0000 7fa5ba4b9640 1 -- 172.21.15.60:0/1779416144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b4162250 msgr2=0x7fa5b40a8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.635+0000 7fa5ba4b9640 1 --2- 172.21.15.60:0/1779416144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b4162250 0x7fa5b40a8500 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00c780 tx=0x7fa5ac00cc40 comp rx=0 tx=0).stop 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.635+0000 7fa5ba4b9640 1 -- 172.21.15.60:0/1779416144 shutdown_connections 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.635+0000 7fa5ba4b9640 1 --2- 172.21.15.60:0/1779416144 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa5a0066f60 0x7fa5a0069420 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.635+0000 7fa5ba4b9640 1 --2- 172.21.15.60:0/1779416144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5b4162250 0x7fa5b40a8500 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.635+0000 7fa5ba4b9640 1 --2- 172.21.15.60:0/1779416144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5b40b34f0 0x7fa5b41f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.635+0000 7fa5ba4b9640 1 --2- 172.21.15.60:0/1779416144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5b40719a0 0x7fa5b4150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.635+0000 7fa5ba4b9640 1 -- 172.21.15.60:0/1779416144 >> 172.21.15.60:0/1779416144 conn(0x7fa5b40b12d0 msgr2=0x7fa5b40b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.635+0000 7fa5ba4b9640 1 -- 172.21.15.60:0/1779416144 shutdown_connections 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.635+0000 7fa5ba4b9640 1 -- 172.21.15.60:0/1779416144 wait complete. 2024-04-25T10:06:52.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:20.088165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:20.507719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:52.661 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:52.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.775+0000 7f80dc92a640 1 Processor -- start 2024-04-25T10:06:52.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.775+0000 7f80dc92a640 1 -- start start 2024-04-25T10:06:52.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.775+0000 7f80dc92a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f80d80cbcf0 0x7f80d8160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.775+0000 7f80dc92a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f80d8071c30 0x7f80d815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.775+0000 7f80dc92a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80d815fa20 0x7f80d815bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.775+0000 7f80dc92a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d80b2dc0 con 0x7f80d80cbcf0 2024-04-25T10:06:52.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.775+0000 7f80dc92a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d8144a40 con 0x7f80d8071c30 2024-04-25T10:06:52.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.775+0000 7f80dc92a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d80cc9f0 con 0x7f80d815fa20 2024-04-25T10:06:52.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.775+0000 7f80d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f80d80cbcf0 0x7f80d8160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.775+0000 7f80d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f80d80cbcf0 0x7f80d8160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39362/0 (socket says 172.21.15.60:39362) 2024-04-25T10:06:52.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.775+0000 7f80d659c640 1 -- 172.21.15.60:0/3887929350 learned_addr learned my addr 172.21.15.60:0/3887929350 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:52.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80d6d9d640 1 --2- 172.21.15.60:0/3887929350 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80d815fa20 0x7f80d815bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80d5d9b640 1 --2- 172.21.15.60:0/3887929350 >> v2:172.21.15.100:3300/0 conn(0x7f80d8071c30 0x7f80d815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80d659c640 1 -- 172.21.15.60:0/3887929350 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80d815fa20 msgr2=0x7f80d815bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80d659c640 1 --2- 172.21.15.60:0/3887929350 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80d815fa20 0x7f80d815bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80d659c640 1 -- 172.21.15.60:0/3887929350 >> v2:172.21.15.100:3300/0 conn(0x7f80d8071c30 msgr2=0x7f80d815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80d659c640 1 --2- 172.21.15.60:0/3887929350 >> v2:172.21.15.100:3300/0 conn(0x7f80d8071c30 0x7f80d815f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80d659c640 1 -- 172.21.15.60:0/3887929350 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d80cccd0 con 0x7f80d80cbcf0 2024-04-25T10:06:52.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80d659c640 1 --2- 172.21.15.60:0/3887929350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d80cbcf0 0x7f80d8160080 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f80d80d0570 tx=0x7f80c801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68d4725904189c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80d559a640 1 -- 172.21.15.60:0/3887929350 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c80b1070 con 0x7f80d80cbcf0 2024-04-25T10:06:52.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80d559a640 1 -- 172.21.15.60:0/3887929350 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c80ac630 con 0x7f80d80cbcf0 2024-04-25T10:06:52.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80d559a640 1 -- 172.21.15.60:0/3887929350 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c80a8050 con 0x7f80d80cbcf0 2024-04-25T10:06:52.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80dc92a640 1 -- 172.21.15.60:0/3887929350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d80cbcf0 msgr2=0x7f80d8160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80dc92a640 1 --2- 172.21.15.60:0/3887929350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d80cbcf0 0x7f80d8160080 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f80d80d0570 tx=0x7f80c801c520 comp rx=0 tx=0).stop 2024-04-25T10:06:52.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80dc92a640 1 -- 172.21.15.60:0/3887929350 shutdown_connections 2024-04-25T10:06:52.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80dc92a640 1 --2- 172.21.15.60:0/3887929350 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80d815fa20 0x7f80d815bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80dc92a640 1 --2- 172.21.15.60:0/3887929350 >> v2:172.21.15.100:3300/0 conn(0x7f80d8071c30 0x7f80d815f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80dc92a640 1 --2- 172.21.15.60:0/3887929350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d80cbcf0 0x7f80d8160080 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80dc92a640 1 -- 172.21.15.60:0/3887929350 >> 172.21.15.60:0/3887929350 conn(0x7f80d80670b0 msgr2=0x7f80d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80dc92a640 1 -- 172.21.15.60:0/3887929350 shutdown_connections 2024-04-25T10:06:52.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.779+0000 7f80dc92a640 1 -- 172.21.15.60:0/3887929350 wait complete. 2024-04-25T10:06:52.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80dc92a640 1 Processor -- start 2024-04-25T10:06:52.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80dc92a640 1 -- start start 2024-04-25T10:06:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80dc92a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d8071c30 0x7f80d80af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80dc92a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80d80cbcf0 0x7f80d813ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80dc92a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80d815fa20 0x7f80d80ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80dc92a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d80ce740 con 0x7f80d8071c30 2024-04-25T10:06:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80dc92a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d80ce5c0 con 0x7f80d80cbcf0 2024-04-25T10:06:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80dc92a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d8072010 con 0x7f80d815fa20 2024-04-25T10:06:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80d659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d8071c30 0x7f80d80af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80d659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d8071c30 0x7f80d80af5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39372/0 (socket says 172.21.15.60:39372) 2024-04-25T10:06:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80d659c640 1 -- 172.21.15.60:0/4008530701 learned_addr learned my addr 172.21.15.60:0/4008530701 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80d5d9b640 1 --2- 172.21.15.60:0/4008530701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80d80cbcf0 0x7f80d813ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80d6d9d640 1 --2- 172.21.15.60:0/4008530701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80d815fa20 0x7f80d80ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80d659c640 1 -- 172.21.15.60:0/4008530701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80d815fa20 msgr2=0x7f80d80ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80d659c640 1 --2- 172.21.15.60:0/4008530701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80d815fa20 0x7f80d80ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80d659c640 1 -- 172.21.15.60:0/4008530701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80d80cbcf0 msgr2=0x7f80d813ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80d659c640 1 --2- 172.21.15.60:0/4008530701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80d80cbcf0 0x7f80d813ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80d659c640 1 -- 172.21.15.60:0/4008530701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d80afef0 con 0x7f80d8071c30 2024-04-25T10:06:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80d659c640 1 --2- 172.21.15.60:0/4008530701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d8071c30 0x7f80d80af5f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f80c80af040 tx=0x7f80c80192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80c6ffd640 1 -- 172.21.15.60:0/4008530701 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c80bc070 con 0x7f80d8071c30 2024-04-25T10:06:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80c6ffd640 1 -- 172.21.15.60:0/4008530701 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c80b7430 con 0x7f80d8071c30 2024-04-25T10:06:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80dc92a640 1 -- 172.21.15.60:0/4008530701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d80b01d0 con 0x7f80d8071c30 2024-04-25T10:06:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80c6ffd640 1 -- 172.21.15.60:0/4008530701 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c80b1040 con 0x7f80d8071c30 2024-04-25T10:06:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80dc92a640 1 -- 172.21.15.60:0/4008530701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d80b04b0 con 0x7f80d8071c30 2024-04-25T10:06:52.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.783+0000 7f80dc92a640 1 -- 172.21.15.60:0/4008530701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d806bba0 con 0x7f80d8071c30 2024-04-25T10:06:52.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.787+0000 7f80c6ffd640 1 -- 172.21.15.60:0/4008530701 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f80c80bf070 con 0x7f80d8071c30 2024-04-25T10:06:52.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.787+0000 7f80c6ffd640 1 --2- 172.21.15.60:0/4008530701 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f80b8066f60 0x7f80b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.787+0000 7f80c6ffd640 1 -- 172.21.15.60:0/4008530701 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80c811e7a0 con 0x7f80d8071c30 2024-04-25T10:06:52.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.787+0000 7f80d5d9b640 1 --2- 172.21.15.60:0/4008530701 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f80b8066f60 0x7f80b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.787+0000 7f80d5d9b640 1 --2- 172.21.15.60:0/4008530701 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f80b8066f60 0x7f80b8069420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f80d8144350 tx=0x7f80cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.791+0000 7f80c6ffd640 1 -- 172.21.15.60:0/4008530701 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f80c8016110 con 0x7f80d8071c30 2024-04-25T10:06:52.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.951+0000 7f80dc92a640 1 -- 172.21.15.60:0/4008530701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80d80b2970 con 0x7f80d8071c30 2024-04-25T10:06:52.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.951+0000 7f80c6ffd640 1 -- 172.21.15.60:0/4008530701 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f80c80ef680 con 0x7f80d8071c30 2024-04-25T10:06:52.959 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.955+0000 7f80dc92a640 1 -- 172.21.15.60:0/4008530701 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f80b8066f60 msgr2=0x7f80b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.955+0000 7f80dc92a640 1 --2- 172.21.15.60:0/4008530701 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f80b8066f60 0x7f80b8069420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f80d8144350 tx=0x7f80cc008040 comp rx=0 tx=0).stop 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.955+0000 7f80dc92a640 1 -- 172.21.15.60:0/4008530701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d8071c30 msgr2=0x7f80d80af5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.955+0000 7f80dc92a640 1 --2- 172.21.15.60:0/4008530701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d8071c30 0x7f80d80af5f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f80c80af040 tx=0x7f80c80192c0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.955+0000 7f80dc92a640 1 -- 172.21.15.60:0/4008530701 shutdown_connections 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.955+0000 7f80dc92a640 1 --2- 172.21.15.60:0/4008530701 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f80b8066f60 0x7f80b8069420 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.955+0000 7f80dc92a640 1 --2- 172.21.15.60:0/4008530701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80d815fa20 0x7f80d80ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.955+0000 7f80dc92a640 1 --2- 172.21.15.60:0/4008530701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80d80cbcf0 0x7f80d813ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.955+0000 7f80dc92a640 1 --2- 172.21.15.60:0/4008530701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80d8071c30 0x7f80d80af5f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.955+0000 7f80dc92a640 1 -- 172.21.15.60:0/4008530701 >> 172.21.15.60:0/4008530701 conn(0x7f80d80670b0 msgr2=0x7f80d80b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.955+0000 7f80dc92a640 1 -- 172.21.15.60:0/4008530701 shutdown_connections 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:52.955+0000 7f80dc92a640 1 -- 172.21.15.60:0/4008530701 wait complete. 2024-04-25T10:06:52.979 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:53.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a42b4640 1 Processor -- start 2024-04-25T10:06:53.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a42b4640 1 -- start start 2024-04-25T10:06:53.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a42b4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f369c15dd80 0x7f369c0b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a42b4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f369c0b2860 0x7f369c0b9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a42b4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f369c0b5470 0x7f369c0b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a42b4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f369c0b5e20 con 0x7f369c0b2860 2024-04-25T10:06:53.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a42b4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f369c15e160 con 0x7f369c15dd80 2024-04-25T10:06:53.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a42b4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0b7060 con 0x7f369c0b5470 2024-04-25T10:06:53.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a184f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f369c0b2860 0x7f369c0b9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a184f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f369c0b2860 0x7f369c0b9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39376/0 (socket says 172.21.15.60:39376) 2024-04-25T10:06:53.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a184f640 1 -- 172.21.15.60:0/214242330 learned_addr learned my addr 172.21.15.60:0/214242330 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:53.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a2851640 1 --2- 172.21.15.60:0/214242330 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f369c0b5470 0x7f369c0b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a2050640 1 --2- 172.21.15.60:0/214242330 >> v2:172.21.15.100:3300/0 conn(0x7f369c15dd80 0x7f369c0b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a184f640 1 -- 172.21.15.60:0/214242330 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f369c0b5470 msgr2=0x7f369c0b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a184f640 1 --2- 172.21.15.60:0/214242330 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f369c0b5470 0x7f369c0b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a184f640 1 -- 172.21.15.60:0/214242330 >> v2:172.21.15.100:3300/0 conn(0x7f369c15dd80 msgr2=0x7f369c0b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a184f640 1 --2- 172.21.15.60:0/214242330 >> v2:172.21.15.100:3300/0 conn(0x7f369c15dd80 0x7f369c0b6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.095+0000 7f36a184f640 1 -- 172.21.15.60:0/214242330 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c0d1b20 con 0x7f369c0b2860 2024-04-25T10:06:53.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a184f640 1 --2- 172.21.15.60:0/214242330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f369c0b2860 0x7f369c0b9310 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f369c0d1540 tx=0x7f369801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f571a9a81df53d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a104e640 1 -- 172.21.15.60:0/214242330 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36980aa610 con 0x7f369c0b2860 2024-04-25T10:06:53.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a104e640 1 -- 172.21.15.60:0/214242330 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36980af070 con 0x7f369c0b2860 2024-04-25T10:06:53.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a104e640 1 -- 172.21.15.60:0/214242330 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36980aa9d0 con 0x7f369c0b2860 2024-04-25T10:06:53.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 -- 172.21.15.60:0/214242330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f369c0b2860 msgr2=0x7f369c0b9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 --2- 172.21.15.60:0/214242330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f369c0b2860 0x7f369c0b9310 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f369c0d1540 tx=0x7f369801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 -- 172.21.15.60:0/214242330 shutdown_connections 2024-04-25T10:06:53.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 --2- 172.21.15.60:0/214242330 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f369c0b5470 0x7f369c0b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 --2- 172.21.15.60:0/214242330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f369c0b2860 0x7f369c0b9310 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 --2- 172.21.15.60:0/214242330 >> v2:172.21.15.100:3300/0 conn(0x7f369c15dd80 0x7f369c0b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 -- 172.21.15.60:0/214242330 >> 172.21.15.60:0/214242330 conn(0x7f369c0d0470 msgr2=0x7f369c0d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 -- 172.21.15.60:0/214242330 shutdown_connections 2024-04-25T10:06:53.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 -- 172.21.15.60:0/214242330 wait complete. 2024-04-25T10:06:53.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 Processor -- start 2024-04-25T10:06:53.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 -- start start 2024-04-25T10:06:53.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f369c0b2860 0x7f369c158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f369c0b5470 0x7f369c15cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f369c15dd80 0x7f369c15d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0b9850 con 0x7f369c0b5470 2024-04-25T10:06:53.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0d26a0 con 0x7f369c15dd80 2024-04-25T10:06:53.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.099+0000 7f36a42b4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0b99d0 con 0x7f369c0b2860 2024-04-25T10:06:53.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f36a184f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f369c0b5470 0x7f369c15cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f36a184f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f369c0b5470 0x7f369c15cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39380/0 (socket says 172.21.15.60:39380) 2024-04-25T10:06:53.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f36a184f640 1 -- 172.21.15.60:0/496952222 learned_addr learned my addr 172.21.15.60:0/496952222 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:53.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f36a2050640 1 --2- 172.21.15.60:0/496952222 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f369c0b2860 0x7f369c158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f36a2851640 1 --2- 172.21.15.60:0/496952222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f369c15dd80 0x7f369c15d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f36a184f640 1 -- 172.21.15.60:0/496952222 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f369c0b2860 msgr2=0x7f369c158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f36a184f640 1 --2- 172.21.15.60:0/496952222 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f369c0b2860 0x7f369c158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f36a184f640 1 -- 172.21.15.60:0/496952222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f369c15dd80 msgr2=0x7f369c15d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f36a184f640 1 --2- 172.21.15.60:0/496952222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f369c15dd80 0x7f369c15d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f36a184f640 1 -- 172.21.15.60:0/496952222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c1593a0 con 0x7f369c0b5470 2024-04-25T10:06:53.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f36a184f640 1 --2- 172.21.15.60:0/496952222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f369c0b5470 0x7f369c15cb80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f36980ac040 tx=0x7f369801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f368effd640 1 -- 172.21.15.60:0/496952222 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36980bd5f0 con 0x7f369c0b5470 2024-04-25T10:06:53.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f368effd640 1 -- 172.21.15.60:0/496952222 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36980bd790 con 0x7f369c0b5470 2024-04-25T10:06:53.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f36a42b4640 1 -- 172.21.15.60:0/496952222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f369c159680 con 0x7f369c0b5470 2024-04-25T10:06:53.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f368effd640 1 -- 172.21.15.60:0/496952222 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36980bdb80 con 0x7f369c0b5470 2024-04-25T10:06:53.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f36a42b4640 1 -- 172.21.15.60:0/496952222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369c159d30 con 0x7f369c0b5470 2024-04-25T10:06:53.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f36a42b4640 1 -- 172.21.15.60:0/496952222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f369c074770 con 0x7f369c0b5470 2024-04-25T10:06:53.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f368effd640 1 -- 172.21.15.60:0/496952222 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f36980b33f0 con 0x7f369c0b5470 2024-04-25T10:06:53.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f368effd640 1 --2- 172.21.15.60:0/496952222 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3680066f90 0x7f3680069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f368effd640 1 -- 172.21.15.60:0/496952222 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f369811fc80 con 0x7f369c0b5470 2024-04-25T10:06:53.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.103+0000 7f36a2050640 1 --2- 172.21.15.60:0/496952222 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3680066f90 0x7f3680069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.107+0000 7f36a2050640 1 --2- 172.21.15.60:0/496952222 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3680066f90 0x7f3680069450 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3690006fd0 tx=0x7f3690008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.107+0000 7f368effd640 1 -- 172.21.15.60:0/496952222 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f36980c3060 con 0x7f369c0b5470 2024-04-25T10:06:53.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.271+0000 7f36a42b4640 1 -- 172.21.15.60:0/496952222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f369c160760 con 0x7f369c0b5470 2024-04-25T10:06:53.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.271+0000 7f368effd640 1 -- 172.21.15.60:0/496952222 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f36980a7070 con 0x7f369c0b5470 2024-04-25T10:06:53.277 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:06:53.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.271+0000 7f36a42b4640 1 -- 172.21.15.60:0/496952222 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3680066f90 msgr2=0x7f3680069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.271+0000 7f36a42b4640 1 --2- 172.21.15.60:0/496952222 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3680066f90 0x7f3680069450 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3690006fd0 tx=0x7f3690008040 comp rx=0 tx=0).stop 2024-04-25T10:06:53.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.271+0000 7f36a42b4640 1 -- 172.21.15.60:0/496952222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f369c0b5470 msgr2=0x7f369c15cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.271+0000 7f36a42b4640 1 --2- 172.21.15.60:0/496952222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f369c0b5470 0x7f369c15cb80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f36980ac040 tx=0x7f369801ad30 comp rx=0 tx=0).stop 2024-04-25T10:06:53.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.271+0000 7f36a42b4640 1 -- 172.21.15.60:0/496952222 shutdown_connections 2024-04-25T10:06:53.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.271+0000 7f36a42b4640 1 --2- 172.21.15.60:0/496952222 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3680066f90 0x7f3680069450 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.271+0000 7f36a42b4640 1 --2- 172.21.15.60:0/496952222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f369c15dd80 0x7f369c15d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.271+0000 7f36a42b4640 1 --2- 172.21.15.60:0/496952222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f369c0b5470 0x7f369c15cb80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.271+0000 7f36a42b4640 1 --2- 172.21.15.60:0/496952222 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f369c0b2860 0x7f369c158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.271+0000 7f36a42b4640 1 -- 172.21.15.60:0/496952222 >> 172.21.15.60:0/496952222 conn(0x7f369c0d0470 msgr2=0x7f369c0b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.271+0000 7f36a42b4640 1 -- 172.21.15.60:0/496952222 shutdown_connections 2024-04-25T10:06:53.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.271+0000 7f36a42b4640 1 -- 172.21.15.60:0/496952222 wait complete. 2024-04-25T10:06:53.295 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:53.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.399+0000 7f296b6e4640 1 Processor -- start 2024-04-25T10:06:53.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.399+0000 7f296b6e4640 1 -- start start 2024-04-25T10:06:53.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.399+0000 7f296b6e4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f296415dd30 0x7f296415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.399+0000 7f296b6e4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f29640cd3d0 0x7f29640cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.399+0000 7f296b6e4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29640cd890 con 0x7f296406f310 2024-04-25T10:06:53.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.399+0000 7f296b6e4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29640ce4a0 con 0x7f296415dd30 2024-04-25T10:06:53.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.399+0000 7f296b6e4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29640cdb70 con 0x7f29640cd3d0 2024-04-25T10:06:53.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.399+0000 7f2969480640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f296415dd30 0x7f296415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.399+0000 7f2969480640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f296415dd30 0x7f296415ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39392/0 (socket says 172.21.15.60:39392) 2024-04-25T10:06:53.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.399+0000 7f2969480640 1 -- 172.21.15.60:0/519672056 learned_addr learned my addr 172.21.15.60:0/519672056 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.399+0000 7f2969480640 1 -- 172.21.15.60:0/519672056 >> v2:172.21.15.100:3300/0 conn(0x7f29640cd3d0 msgr2=0x7f29640cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.399+0000 7f2969480640 1 --2- 172.21.15.60:0/519672056 >> v2:172.21.15.100:3300/0 conn(0x7f29640cd3d0 0x7f29640cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.399+0000 7f2969480640 1 -- 172.21.15.60:0/519672056 >> v1:172.21.15.60:6789/0 conn(0x7f296406f310 legacy=0x7f296406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.399+0000 7f2969480640 1 -- 172.21.15.60:0/519672056 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29640cde50 con 0x7f296415dd30 2024-04-25T10:06:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.403+0000 7f2969480640 1 --2- 172.21.15.60:0/519672056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f296415dd30 0x7f296415ac30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2964072550 tx=0x7f296001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d08afe7d6c22d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.403+0000 7f295bfff640 1 -- 172.21.15.60:0/519672056 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29600a5810 con 0x7f296415dd30 2024-04-25T10:06:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.403+0000 7f295bfff640 1 -- 172.21.15.60:0/519672056 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29600a59b0 con 0x7f296415dd30 2024-04-25T10:06:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.403+0000 7f295bfff640 1 -- 172.21.15.60:0/519672056 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29600a5d80 con 0x7f296415dd30 2024-04-25T10:06:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.403+0000 7f296b6e4640 1 -- 172.21.15.60:0/519672056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f296415dd30 msgr2=0x7f296415ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.403+0000 7f296b6e4640 1 --2- 172.21.15.60:0/519672056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f296415dd30 0x7f296415ac30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2964072550 tx=0x7f296001c930 comp rx=0 tx=0).stop 2024-04-25T10:06:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.403+0000 7f296b6e4640 1 -- 172.21.15.60:0/519672056 shutdown_connections 2024-04-25T10:06:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.403+0000 7f296b6e4640 1 --2- 172.21.15.60:0/519672056 >> v2:172.21.15.100:3300/0 conn(0x7f29640cd3d0 0x7f29640cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.403+0000 7f296b6e4640 1 --2- 172.21.15.60:0/519672056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f296415dd30 0x7f296415ac30 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.403+0000 7f296b6e4640 1 -- 172.21.15.60:0/519672056 >> 172.21.15.60:0/519672056 conn(0x7f29640670b0 msgr2=0x7f296406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.403+0000 7f296b6e4640 1 -- 172.21.15.60:0/519672056 shutdown_connections 2024-04-25T10:06:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.403+0000 7f296b6e4640 1 -- 172.21.15.60:0/519672056 wait complete. 2024-04-25T10:06:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.403+0000 7f296b6e4640 1 Processor -- start 2024-04-25T10:06:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.403+0000 7f296b6e4640 1 -- start start 2024-04-25T10:06:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f296b6e4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296406f310 0x7f296413e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f296b6e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29640cd3d0 0x7f2964140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f296b6e4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f296415dd30 0x7f2964141340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f296b6e4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29641674b0 con 0x7f29640cd3d0 2024-04-25T10:06:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f296b6e4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29640cfae0 con 0x7f296406f310 2024-04-25T10:06:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f296b6e4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2964167630 con 0x7f296415dd30 2024-04-25T10:06:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2968c7f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29640cd3d0 0x7f2964140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2968c7f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29640cd3d0 0x7f2964140e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39408/0 (socket says 172.21.15.60:39408) 2024-04-25T10:06:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2968c7f640 1 -- 172.21.15.60:0/3671011654 learned_addr learned my addr 172.21.15.60:0/3671011654 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2968c7f640 1 -- 172.21.15.60:0/3671011654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f296415dd30 msgr2=0x7f2964141340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2968c7f640 1 --2- 172.21.15.60:0/3671011654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f296415dd30 0x7f2964141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2968c7f640 1 -- 172.21.15.60:0/3671011654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296406f310 msgr2=0x7f296413e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2969480640 1 --2- 172.21.15.60:0/3671011654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296406f310 0x7f296413e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2968c7f640 1 --2- 172.21.15.60:0/3671011654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296406f310 0x7f296413e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2968c7f640 1 -- 172.21.15.60:0/3671011654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2964141930 con 0x7f29640cd3d0 2024-04-25T10:06:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2968c7f640 1 --2- 172.21.15.60:0/3671011654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29640cd3d0 0x7f2964140e00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f295400b440 tx=0x7f295400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2959ffb640 1 -- 172.21.15.60:0/3671011654 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954015020 con 0x7f29640cd3d0 2024-04-25T10:06:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2959ffb640 1 -- 172.21.15.60:0/3671011654 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29540118b0 con 0x7f29640cd3d0 2024-04-25T10:06:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f296b6e4640 1 -- 172.21.15.60:0/3671011654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2964141c10 con 0x7f29640cd3d0 2024-04-25T10:06:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2959ffb640 1 -- 172.21.15.60:0/3671011654 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954011ca0 con 0x7f29640cd3d0 2024-04-25T10:06:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f296b6e4640 1 -- 172.21.15.60:0/3671011654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296413f2f0 con 0x7f29640cd3d0 2024-04-25T10:06:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f296b6e4640 1 -- 172.21.15.60:0/3671011654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296413f4f0 con 0x7f29640cd3d0 2024-04-25T10:06:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2959ffb640 1 -- 172.21.15.60:0/3671011654 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2954018020 con 0x7f29640cd3d0 2024-04-25T10:06:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2959ffb640 1 --2- 172.21.15.60:0/3671011654 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2948066f60 0x7f2948069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2959ffb640 1 -- 172.21.15.60:0/3671011654 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2954085170 con 0x7f29640cd3d0 2024-04-25T10:06:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.407+0000 7f2969480640 1 --2- 172.21.15.60:0/3671011654 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2948066f60 0x7f2948069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.411+0000 7f2969480640 1 --2- 172.21.15.60:0/3671011654 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2948066f60 0x7f2948069420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f29600ae000 tx=0x7f29600a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.411+0000 7f2959ffb640 1 -- 172.21.15.60:0/3671011654 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f296413f4f0 con 0x7f29640cd3d0 2024-04-25T10:06:53.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.563+0000 7f296b6e4640 1 -- 172.21.15.60:0/3671011654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f296413ea80 con 0x7f29640cd3d0 2024-04-25T10:06:53.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.567+0000 7f2959ffb640 1 -- 172.21.15.60:0/3671011654 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7f295401f020 con 0x7f29640cd3d0 2024-04-25T10:06:53.572 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:06:53.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.567+0000 7f296b6e4640 1 -- 172.21.15.60:0/3671011654 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2948066f60 msgr2=0x7f2948069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.567+0000 7f296b6e4640 1 --2- 172.21.15.60:0/3671011654 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2948066f60 0x7f2948069420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f29600ae000 tx=0x7f29600a7070 comp rx=0 tx=0).stop 2024-04-25T10:06:53.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.567+0000 7f296b6e4640 1 -- 172.21.15.60:0/3671011654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29640cd3d0 msgr2=0x7f2964140e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.567+0000 7f296b6e4640 1 --2- 172.21.15.60:0/3671011654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29640cd3d0 0x7f2964140e00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f295400b440 tx=0x7f295400b900 comp rx=0 tx=0).stop 2024-04-25T10:06:53.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.567+0000 7f296b6e4640 1 -- 172.21.15.60:0/3671011654 shutdown_connections 2024-04-25T10:06:53.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.567+0000 7f296b6e4640 1 --2- 172.21.15.60:0/3671011654 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2948066f60 0x7f2948069420 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.567+0000 7f296b6e4640 1 --2- 172.21.15.60:0/3671011654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f296415dd30 0x7f2964141340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.567+0000 7f296b6e4640 1 --2- 172.21.15.60:0/3671011654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29640cd3d0 0x7f2964140e00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.567+0000 7f296b6e4640 1 --2- 172.21.15.60:0/3671011654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f296406f310 0x7f296413e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.567+0000 7f296b6e4640 1 -- 172.21.15.60:0/3671011654 >> 172.21.15.60:0/3671011654 conn(0x7f29640670b0 msgr2=0x7f296415a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.567+0000 7f296b6e4640 1 -- 172.21.15.60:0/3671011654 shutdown_connections 2024-04-25T10:06:53.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.567+0000 7f296b6e4640 1 -- 172.21.15.60:0/3671011654 wait complete. 2024-04-25T10:06:53.590 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65ed4ce640 1 Processor -- start 2024-04-25T10:06:53.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65ed4ce640 1 -- start start 2024-04-25T10:06:53.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65ed4ce640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f65e80ce7a0 0x7f65e80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65ed4ce640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e80b33f0 con 0x7f65e815bc90 2024-04-25T10:06:53.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65ed4ce640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e80cf0c0 con 0x7f65e80b4660 2024-04-25T10:06:53.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65ed4ce640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e80cc370 con 0x7f65e80ce7a0 2024-04-25T10:06:53.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65de7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f65e815bc90 0x7f65e80b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48524/0 (socket says 172.21.15.60:48524) 2024-04-25T10:06:53.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65de7fc640 1 -- 172.21.15.60:0/2290419989 learned_addr learned my addr 172.21.15.60:0/2290419989 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:53.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e6ffd640 1 --2- 172.21.15.60:0/2290419989 >> v2:172.21.15.100:3300/0 conn(0x7f65e80ce7a0 0x7f65e80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e67fc640 1 -- 172.21.15.60:0/2290419989 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 53399264 0 0) 0x7f65e80b33f0 con 0x7f65e815bc90 2024-04-25T10:06:53.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e67fc640 1 -- 172.21.15.60:0/2290419989 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65d4003530 con 0x7f65e815bc90 2024-04-25T10:06:53.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e67fc640 1 -- 172.21.15.60:0/2290419989 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 443488900 0 0) 0x7f65d4003530 con 0x7f65e815bc90 2024-04-25T10:06:53.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e67fc640 1 -- 172.21.15.60:0/2290419989 >> v2:172.21.15.100:3300/0 conn(0x7f65e80ce7a0 msgr2=0x7f65e80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e67fc640 1 --2- 172.21.15.60:0/2290419989 >> v2:172.21.15.100:3300/0 conn(0x7f65e80ce7a0 0x7f65e80ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e67fc640 1 -- 172.21.15.60:0/2290419989 >> v1:172.21.15.100:6789/0 conn(0x7f65e80b4660 legacy=0x7f65e80b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e67fc640 1 -- 172.21.15.60:0/2290419989 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e8071ca0 con 0x7f65e815bc90 2024-04-25T10:06:53.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e67fc640 1 -- 172.21.15.60:0/2290419989 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f65d0001680 con 0x7f65e815bc90 2024-04-25T10:06:53.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e67fc640 1 -- 172.21.15.60:0/2290419989 >> v1:172.21.15.60:6789/0 conn(0x7f65e815bc90 legacy=0x7f65e80b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e67fc640 1 --2- 172.21.15.60:0/2290419989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65d40039c0 0x7f65d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e67fc640 1 --2- 172.21.15.60:0/2290419989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65d4004810 0x7f65d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e67fc640 1 --2- 172.21.15.60:0/2290419989 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65d40051d0 0x7f65d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e67fc640 1 -- 172.21.15.60:0/2290419989 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d400c040 con 0x7f65d40039c0 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e6ffd640 1 --2- 172.21.15.60:0/2290419989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65d40039c0 0x7f65d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e67fc640 1 -- 172.21.15.60:0/2290419989 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d400c1f0 con 0x7f65d4004810 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e67fc640 1 -- 172.21.15.60:0/2290419989 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d400c3f0 con 0x7f65d40051d0 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65de7fc640 1 --2- 172.21.15.60:0/2290419989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65d4004810 0x7f65d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e77fe640 1 --2- 172.21.15.60:0/2290419989 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65d40051d0 0x7f65d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e6ffd640 1 -- 172.21.15.60:0/2290419989 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65d40051d0 msgr2=0x7f65d400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e6ffd640 1 --2- 172.21.15.60:0/2290419989 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65d40051d0 0x7f65d400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e6ffd640 1 -- 172.21.15.60:0/2290419989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65d4004810 msgr2=0x7f65d4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e6ffd640 1 --2- 172.21.15.60:0/2290419989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65d4004810 0x7f65d4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e6ffd640 1 -- 172.21.15.60:0/2290419989 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d4003530 con 0x7f65d40039c0 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e77fe640 1 --2- 172.21.15.60:0/2290419989 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65d40051d0 0x7f65d400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.695+0000 7f65e77fe640 1 -- 172.21.15.60:0/2290419989 reap_dead start 2024-04-25T10:06:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65e6ffd640 1 --2- 172.21.15.60:0/2290419989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65d40039c0 0x7f65d4003de0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009ae0 tx=0x7f65d801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60668c445794c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65e67fc640 1 -- 172.21.15.60:0/2290419989 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d80096b0 con 0x7f65d40039c0 2024-04-25T10:06:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65e67fc640 1 -- 172.21.15.60:0/2290419989 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d80a6420 con 0x7f65d40039c0 2024-04-25T10:06:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65e67fc640 1 -- 172.21.15.60:0/2290419989 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d80a6810 con 0x7f65d40039c0 2024-04-25T10:06:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65ed4ce640 1 -- 172.21.15.60:0/2290419989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65d40039c0 msgr2=0x7f65d4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65ed4ce640 1 --2- 172.21.15.60:0/2290419989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65d40039c0 0x7f65d4003de0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009ae0 tx=0x7f65d801ca70 comp rx=0 tx=0).stop 2024-04-25T10:06:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65ed4ce640 1 -- 172.21.15.60:0/2290419989 shutdown_connections 2024-04-25T10:06:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65ed4ce640 1 --2- 172.21.15.60:0/2290419989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65d40039c0 0x7f65d4003de0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65ed4ce640 1 -- 172.21.15.60:0/2290419989 >> 172.21.15.60:0/2290419989 conn(0x7f65e80b12d0 msgr2=0x7f65e80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65ed4ce640 1 -- 172.21.15.60:0/2290419989 shutdown_connections 2024-04-25T10:06:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65ed4ce640 1 -- 172.21.15.60:0/2290419989 wait complete. 2024-04-25T10:06:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65ed4ce640 1 Processor -- start 2024-04-25T10:06:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65ed4ce640 1 -- start start 2024-04-25T10:06:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65ed4ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65d40039c0 0x7f65e81f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65ed4ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65e81567b0 0x7f65e8156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65ed4ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e8157170 0x7f65e81f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65ed4ce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80b3d20 con 0x7f65e8157170 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65ed4ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80b3ba0 con 0x7f65d40039c0 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.699+0000 7f65ed4ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80b4280 con 0x7f65e81567b0 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65e77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e8157170 0x7f65e81f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65e77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e8157170 0x7f65e81f78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39432/0 (socket says 172.21.15.60:39432) 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65e77fe640 1 -- 172.21.15.60:0/2564602799 learned_addr learned my addr 172.21.15.60:0/2564602799 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65de7fc640 1 --2- 172.21.15.60:0/2564602799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65e81567b0 0x7f65e8156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65e6ffd640 1 --2- 172.21.15.60:0/2564602799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65d40039c0 0x7f65e81f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65e77fe640 1 -- 172.21.15.60:0/2564602799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65e81567b0 msgr2=0x7f65e8156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65e77fe640 1 --2- 172.21.15.60:0/2564602799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65e81567b0 0x7f65e8156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65e77fe640 1 -- 172.21.15.60:0/2564602799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65d40039c0 msgr2=0x7f65e81f3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65e77fe640 1 --2- 172.21.15.60:0/2564602799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65d40039c0 0x7f65e81f3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65e77fe640 1 -- 172.21.15.60:0/2564602799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e81575f0 con 0x7f65e8157170 2024-04-25T10:06:53.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65e6ffd640 1 --2- 172.21.15.60:0/2564602799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65d40039c0 0x7f65e81f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:53.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65e77fe640 1 --2- 172.21.15.60:0/2564602799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e8157170 0x7f65e81f78d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f65e000d540 tx=0x7f65e000da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65dffff640 1 -- 172.21.15.60:0/2564602799 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e0018070 con 0x7f65e8157170 2024-04-25T10:06:53.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65dffff640 1 -- 172.21.15.60:0/2564602799 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e0003db0 con 0x7f65e8157170 2024-04-25T10:06:53.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65ed4ce640 1 -- 172.21.15.60:0/2564602799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e81f4160 con 0x7f65e8157170 2024-04-25T10:06:53.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65dffff640 1 -- 172.21.15.60:0/2564602799 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e00052b0 con 0x7f65e8157170 2024-04-25T10:06:53.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65ed4ce640 1 -- 172.21.15.60:0/2564602799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e8155480 con 0x7f65e8157170 2024-04-25T10:06:53.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65ed4ce640 1 -- 172.21.15.60:0/2564602799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e8074770 con 0x7f65e8157170 2024-04-25T10:06:53.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65dffff640 1 -- 172.21.15.60:0/2564602799 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65e0005450 con 0x7f65e8157170 2024-04-25T10:06:53.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65dffff640 1 --2- 172.21.15.60:0/2564602799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65c8066f90 0x7f65c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65dffff640 1 -- 172.21.15.60:0/2564602799 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65e0084480 con 0x7f65e8157170 2024-04-25T10:06:53.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65e6ffd640 1 --2- 172.21.15.60:0/2564602799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65c8066f90 0x7f65c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.703+0000 7f65e6ffd640 1 --2- 172.21.15.60:0/2564602799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65c8066f90 0x7f65c8069450 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f65d80ad000 tx=0x7f65d80a9040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.707+0000 7f65dffff640 1 -- 172.21.15.60:0/2564602799 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65e0054e40 con 0x7f65e8157170 2024-04-25T10:06:53.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.907+0000 7f65ed4ce640 1 -- 172.21.15.60:0/2564602799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65e806c250 con 0x7f65e8157170 2024-04-25T10:06:53.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.907+0000 7f65dffff640 1 -- 172.21.15.60:0/2564602799 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f65e0058ee0 con 0x7f65e8157170 2024-04-25T10:06:53.913 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:53.913 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2422261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2422261630},{"type":"v1","addr":"172.21.15.100:6835","nonce":2422261630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2923340508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2923340508},{"type":"v1","addr":"172.21.15.100:6837","nonce":2923340508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1929386449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1929386449},{"type":"v1","addr":"172.21.15.60:6833","nonce":1929386449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:21.529455+0000","modified":"2024-04-25T10:06:23.575106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5283":{"gid":5283,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3681296171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3681296171},{"type":"v1","addr":"172.21.15.60:6835","nonce":3681296171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:53.913 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 15 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.907+0000 7f65ed4ce640 1 -- 172.21.15.60:0/2564602799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65c8066f90 msgr2=0x7f65c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.907+0000 7f65ed4ce640 1 --2- 172.21.15.60:0/2564602799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65c8066f90 0x7f65c8069450 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f65d80ad000 tx=0x7f65d80a9040 comp rx=0 tx=0).stop 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.907+0000 7f65ed4ce640 1 -- 172.21.15.60:0/2564602799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e8157170 msgr2=0x7f65e81f78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.907+0000 7f65ed4ce640 1 --2- 172.21.15.60:0/2564602799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e8157170 0x7f65e81f78d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f65e000d540 tx=0x7f65e000da00 comp rx=0 tx=0).stop 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.907+0000 7f65ed4ce640 1 -- 172.21.15.60:0/2564602799 shutdown_connections 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.907+0000 7f65ed4ce640 1 --2- 172.21.15.60:0/2564602799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65c8066f90 0x7f65c8069450 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.907+0000 7f65ed4ce640 1 --2- 172.21.15.60:0/2564602799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e8157170 0x7f65e81f78d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.907+0000 7f65ed4ce640 1 --2- 172.21.15.60:0/2564602799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65e81567b0 0x7f65e8156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.907+0000 7f65ed4ce640 1 --2- 172.21.15.60:0/2564602799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65d40039c0 0x7f65e81f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.907+0000 7f65ed4ce640 1 -- 172.21.15.60:0/2564602799 >> 172.21.15.60:0/2564602799 conn(0x7f65e80b12d0 msgr2=0x7f65e815b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.907+0000 7f65ed4ce640 1 -- 172.21.15.60:0/2564602799 shutdown_connections 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:53.907+0000 7f65ed4ce640 1 -- 172.21.15.60:0/2564602799 wait complete. 2024-04-25T10:06:53.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:54.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.035+0000 7fc74c15a640 1 Processor -- start 2024-04-25T10:06:54.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.035+0000 7fc74c15a640 1 -- start start 2024-04-25T10:06:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.035+0000 7fc74c15a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc74406ec30 0x7fc7440cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.035+0000 7fc74c15a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7440b37e0 0x7fc7440b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.035+0000 7fc74c15a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc74406f130 con 0x7fc74415a7a0 2024-04-25T10:06:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.035+0000 7fc74c15a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7440cdaa0 con 0x7fc74406ec30 2024-04-25T10:06:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.035+0000 7fc74c15a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74406d2f0 con 0x7fc7440b37e0 2024-04-25T10:06:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.035+0000 7fc7496f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc74406ec30 0x7fc7440cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.035+0000 7fc7496f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc74406ec30 0x7fc7440cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39440/0 (socket says 172.21.15.60:39440) 2024-04-25T10:06:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.035+0000 7fc7496f5640 1 -- 172.21.15.60:0/2051661818 learned_addr learned my addr 172.21.15.60:0/2051661818 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.035+0000 7fc74a6f7640 1 --2- 172.21.15.60:0/2051661818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7440b37e0 0x7fc7440b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.035+0000 7fc7496f5640 1 -- 172.21.15.60:0/2051661818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7440b37e0 msgr2=0x7fc7440b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.035+0000 7fc7496f5640 1 --2- 172.21.15.60:0/2051661818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7440b37e0 0x7fc7440b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.035+0000 7fc7496f5640 1 -- 172.21.15.60:0/2051661818 >> v1:172.21.15.100:6789/0 conn(0x7fc74415a7a0 legacy=0x7fc744071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.035+0000 7fc7496f5640 1 -- 172.21.15.60:0/2051661818 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74406f410 con 0x7fc74406ec30 2024-04-25T10:06:54.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.039+0000 7fc7496f5640 1 --2- 172.21.15.60:0/2051661818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc74406ec30 0x7fc7440cd560 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc73400bb40 tx=0x7fc73401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c9f974310a7b45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.039+0000 7fc748ef4640 1 -- 172.21.15.60:0/2051661818 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7340b4070 con 0x7fc74406ec30 2024-04-25T10:06:54.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.039+0000 7fc748ef4640 1 -- 172.21.15.60:0/2051661818 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7340a7bf0 con 0x7fc74406ec30 2024-04-25T10:06:54.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.039+0000 7fc748ef4640 1 -- 172.21.15.60:0/2051661818 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7340af440 con 0x7fc74406ec30 2024-04-25T10:06:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.039+0000 7fc74c15a640 1 -- 172.21.15.60:0/2051661818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc74406ec30 msgr2=0x7fc7440cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.039+0000 7fc74c15a640 1 --2- 172.21.15.60:0/2051661818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc74406ec30 0x7fc7440cd560 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc73400bb40 tx=0x7fc73401ec10 comp rx=0 tx=0).stop 2024-04-25T10:06:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.039+0000 7fc74c15a640 1 -- 172.21.15.60:0/2051661818 shutdown_connections 2024-04-25T10:06:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.039+0000 7fc74c15a640 1 --2- 172.21.15.60:0/2051661818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc74406ec30 0x7fc7440cd560 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.039+0000 7fc74c15a640 1 --2- 172.21.15.60:0/2051661818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7440b37e0 0x7fc7440b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.039+0000 7fc74c15a640 1 -- 172.21.15.60:0/2051661818 >> 172.21.15.60:0/2051661818 conn(0x7fc7440b12d0 msgr2=0x7fc7440b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.039+0000 7fc74c15a640 1 -- 172.21.15.60:0/2051661818 shutdown_connections 2024-04-25T10:06:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.039+0000 7fc74c15a640 1 -- 172.21.15.60:0/2051661818 wait complete. 2024-04-25T10:06:54.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.039+0000 7fc74c15a640 1 Processor -- start 2024-04-25T10:06:54.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc74c15a640 1 -- start start 2024-04-25T10:06:54.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc74c15a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc74406ec30 0x7fc744147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc74c15a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7440b37e0 0x7fc7440c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc74c15a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc74415a7a0 0x7fc7441638a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc74c15a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74406e900 con 0x7fc74406ec30 2024-04-25T10:06:54.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc74c15a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74406e780 con 0x7fc74415a7a0 2024-04-25T10:06:54.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc74c15a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74406d8b0 con 0x7fc7440b37e0 2024-04-25T10:06:54.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc749ef6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc74406ec30 0x7fc744147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc749ef6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc74406ec30 0x7fc744147070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39442/0 (socket says 172.21.15.60:39442) 2024-04-25T10:06:54.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc749ef6640 1 -- 172.21.15.60:0/3727817664 learned_addr learned my addr 172.21.15.60:0/3727817664 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:54.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc7496f5640 1 --2- 172.21.15.60:0/3727817664 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7440b37e0 0x7fc7440c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc74a6f7640 1 --2- 172.21.15.60:0/3727817664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc74415a7a0 0x7fc7441638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc749ef6640 1 -- 172.21.15.60:0/3727817664 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7440b37e0 msgr2=0x7fc7440c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc749ef6640 1 --2- 172.21.15.60:0/3727817664 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7440b37e0 0x7fc7440c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc749ef6640 1 -- 172.21.15.60:0/3727817664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc74415a7a0 msgr2=0x7fc7441638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc749ef6640 1 --2- 172.21.15.60:0/3727817664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc74415a7a0 0x7fc7441638a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc749ef6640 1 -- 172.21.15.60:0/3727817664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc744155560 con 0x7fc74406ec30 2024-04-25T10:06:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc7496f5640 1 --2- 172.21.15.60:0/3727817664 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7440b37e0 0x7fc7440c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc749ef6640 1 --2- 172.21.15.60:0/3727817664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc74406ec30 0x7fc744147070 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc74000c370 tx=0x7fc74000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc73a7fc640 1 -- 172.21.15.60:0/3727817664 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7400052d0 con 0x7fc74406ec30 2024-04-25T10:06:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc73a7fc640 1 -- 172.21.15.60:0/3727817664 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc740005470 con 0x7fc74406ec30 2024-04-25T10:06:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc74c15a640 1 -- 172.21.15.60:0/3727817664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc744155840 con 0x7fc74406ec30 2024-04-25T10:06:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc73a7fc640 1 -- 172.21.15.60:0/3727817664 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc740007260 con 0x7fc74406ec30 2024-04-25T10:06:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc74c15a640 1 -- 172.21.15.60:0/3727817664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc744155e00 con 0x7fc74406ec30 2024-04-25T10:06:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc74c15a640 1 -- 172.21.15.60:0/3727817664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc744156000 con 0x7fc74406ec30 2024-04-25T10:06:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc73a7fc640 1 -- 172.21.15.60:0/3727817664 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc740007400 con 0x7fc74406ec30 2024-04-25T10:06:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc73a7fc640 1 --2- 172.21.15.60:0/3727817664 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc728066f60 0x7fc728069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc73a7fc640 1 -- 172.21.15.60:0/3727817664 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc740084400 con 0x7fc74406ec30 2024-04-25T10:06:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.043+0000 7fc7496f5640 1 --2- 172.21.15.60:0/3727817664 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc728066f60 0x7fc728069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.047+0000 7fc7496f5640 1 --2- 172.21.15.60:0/3727817664 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc728066f60 0x7fc728069420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc7340b1000 tx=0x7fc734008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.047+0000 7fc73a7fc640 1 -- 172.21.15.60:0/3727817664 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc744156000 con 0x7fc74406ec30 2024-04-25T10:06:54.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.203+0000 7fc74c15a640 1 -- 172.21.15.60:0/3727817664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc744074770 con 0x7fc74406ec30 2024-04-25T10:06:54.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.203+0000 7fc73a7fc640 1 -- 172.21.15.60:0/3727817664 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16681 (secure 0 0 0) 0x7fc7400554a0 con 0x7fc74406ec30 2024-04-25T10:06:54.211 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:54.211 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":35,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:23.196554+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:20.088165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:20.507719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:54.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.207+0000 7fc74c15a640 1 -- 172.21.15.60:0/3727817664 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc728066f60 msgr2=0x7fc728069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.207+0000 7fc74c15a640 1 --2- 172.21.15.60:0/3727817664 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc728066f60 0x7fc728069420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc7340b1000 tx=0x7fc734008040 comp rx=0 tx=0).stop 2024-04-25T10:06:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.207+0000 7fc74c15a640 1 -- 172.21.15.60:0/3727817664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc74406ec30 msgr2=0x7fc744147070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.207+0000 7fc74c15a640 1 --2- 172.21.15.60:0/3727817664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc74406ec30 0x7fc744147070 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc74000c370 tx=0x7fc74000c830 comp rx=0 tx=0).stop 2024-04-25T10:06:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.207+0000 7fc74c15a640 1 -- 172.21.15.60:0/3727817664 shutdown_connections 2024-04-25T10:06:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.207+0000 7fc74c15a640 1 --2- 172.21.15.60:0/3727817664 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc728066f60 0x7fc728069420 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.207+0000 7fc74c15a640 1 --2- 172.21.15.60:0/3727817664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc74415a7a0 0x7fc7441638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.207+0000 7fc74c15a640 1 --2- 172.21.15.60:0/3727817664 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7440b37e0 0x7fc7440c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.207+0000 7fc74c15a640 1 --2- 172.21.15.60:0/3727817664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc74406ec30 0x7fc744147070 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.207+0000 7fc74c15a640 1 -- 172.21.15.60:0/3727817664 >> 172.21.15.60:0/3727817664 conn(0x7fc7440b12d0 msgr2=0x7fc74415b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.207+0000 7fc74c15a640 1 -- 172.21.15.60:0/3727817664 shutdown_connections 2024-04-25T10:06:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.207+0000 7fc74c15a640 1 -- 172.21.15.60:0/3727817664 wait complete. 2024-04-25T10:06:54.230 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:06:54.231 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.335+0000 7fe3f1617640 1 Processor -- start 2024-04-25T10:06:54.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.335+0000 7fe3f1617640 1 -- start start 2024-04-25T10:06:54.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3f1617640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3ec15b950 0x7fe3ec0cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3f1617640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe3ec06ac10 0x7fe3ec06b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3f1617640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3ec15bd30 con 0x7fe3ec06ee90 2024-04-25T10:06:54.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3f1617640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3ec072710 con 0x7fe3ec06ac10 2024-04-25T10:06:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3f1617640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec0d0400 con 0x7fe3ec15b950 2024-04-25T10:06:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3eb7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe3ec06ac10 0x7fe3ec06b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3ea7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe3ec06ee90 0x7fe3ec072ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48536/0 (socket says 172.21.15.60:48536) 2024-04-25T10:06:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3ea7fc640 1 -- 172.21.15.60:0/4013049501 learned_addr learned my addr 172.21.15.60:0/4013049501 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3eaffd640 1 --2- 172.21.15.60:0/4013049501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3ec15b950 0x7fe3ec0cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3eb7fe640 1 -- 172.21.15.60:0/4013049501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3ec15b950 msgr2=0x7fe3ec0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3eb7fe640 1 --2- 172.21.15.60:0/4013049501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3ec15b950 0x7fe3ec0cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3eb7fe640 1 -- 172.21.15.60:0/4013049501 >> v1:172.21.15.60:6789/0 conn(0x7fe3ec06ee90 legacy=0x7fe3ec072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3e9ffb640 1 -- 172.21.15.60:0/4013049501 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495578579 0 0) 0x7fe3ec15bd30 con 0x7fe3ec06ee90 2024-04-25T10:06:54.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3eb7fe640 1 -- 172.21.15.60:0/4013049501 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec0cfe40 con 0x7fe3ec06ac10 2024-04-25T10:06:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3eb7fe640 1 --2- 172.21.15.60:0/4013049501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3ec06ac10 0x7fe3ec06b0a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009d90 tx=0x7fe3e401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f33b7b45bf6d61c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3e9ffb640 1 -- 172.21.15.60:0/4013049501 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e40b2070 con 0x7fe3ec06ac10 2024-04-25T10:06:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3e9ffb640 1 -- 172.21.15.60:0/4013049501 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e40a5890 con 0x7fe3ec06ac10 2024-04-25T10:06:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.339+0000 7fe3e9ffb640 1 -- 172.21.15.60:0/4013049501 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e40a5c60 con 0x7fe3ec06ac10 2024-04-25T10:06:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 -- 172.21.15.60:0/4013049501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3ec06ac10 msgr2=0x7fe3ec06b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 --2- 172.21.15.60:0/4013049501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3ec06ac10 0x7fe3ec06b0a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009d90 tx=0x7fe3e401c8e0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 -- 172.21.15.60:0/4013049501 shutdown_connections 2024-04-25T10:06:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 --2- 172.21.15.60:0/4013049501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3ec06ac10 0x7fe3ec06b0a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 --2- 172.21.15.60:0/4013049501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3ec15b950 0x7fe3ec0cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 -- 172.21.15.60:0/4013049501 >> 172.21.15.60:0/4013049501 conn(0x7fe3ec06e370 msgr2=0x7fe3ec06e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 -- 172.21.15.60:0/4013049501 shutdown_connections 2024-04-25T10:06:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 -- 172.21.15.60:0/4013049501 wait complete. 2024-04-25T10:06:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 Processor -- start 2024-04-25T10:06:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 -- start start 2024-04-25T10:06:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3ec06ac10 0x7fe3ec14bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3ec06ee90 0x7fe3ec156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3ec15b950 0x7fe3ec1565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec168840 con 0x7fe3ec06ac10 2024-04-25T10:06:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec070c10 con 0x7fe3ec06ee90 2024-04-25T10:06:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec1689c0 con 0x7fe3ec15b950 2024-04-25T10:06:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3eaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3ec06ac10 0x7fe3ec14bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3eaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3ec06ac10 0x7fe3ec14bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39456/0 (socket says 172.21.15.60:39456) 2024-04-25T10:06:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3eaffd640 1 -- 172.21.15.60:0/258682562 learned_addr learned my addr 172.21.15.60:0/258682562 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3eb7fe640 1 --2- 172.21.15.60:0/258682562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3ec15b950 0x7fe3ec1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3eaffd640 1 -- 172.21.15.60:0/258682562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3ec15b950 msgr2=0x7fe3ec1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3ea7fc640 1 --2- 172.21.15.60:0/258682562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3ec06ee90 0x7fe3ec156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3eaffd640 1 --2- 172.21.15.60:0/258682562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3ec15b950 0x7fe3ec1565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3eaffd640 1 -- 172.21.15.60:0/258682562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3ec06ee90 msgr2=0x7fe3ec156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3eaffd640 1 --2- 172.21.15.60:0/258682562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3ec06ee90 0x7fe3ec156080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3eaffd640 1 -- 172.21.15.60:0/258682562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec14c790 con 0x7fe3ec06ac10 2024-04-25T10:06:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3eaffd640 1 --2- 172.21.15.60:0/258682562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3ec06ac10 0x7fe3ec14bfc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00d5f0 tx=0x7fe3dc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3d37fe640 1 -- 172.21.15.60:0/258682562 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc0125f0 con 0x7fe3ec06ac10 2024-04-25T10:06:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3d37fe640 1 -- 172.21.15.60:0/258682562 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3dc019070 con 0x7fe3ec06ac10 2024-04-25T10:06:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 -- 172.21.15.60:0/258682562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3ec14cad0 con 0x7fe3ec06ac10 2024-04-25T10:06:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3d37fe640 1 -- 172.21.15.60:0/258682562 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc0129b0 con 0x7fe3ec06ac10 2024-04-25T10:06:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 -- 172.21.15.60:0/258682562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ec14d310 con 0x7fe3ec06ac10 2024-04-25T10:06:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.343+0000 7fe3f1617640 1 -- 172.21.15.60:0/258682562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3ec074770 con 0x7fe3ec06ac10 2024-04-25T10:06:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.347+0000 7fe3d37fe640 1 -- 172.21.15.60:0/258682562 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3dc012b50 con 0x7fe3ec06ac10 2024-04-25T10:06:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.347+0000 7fe3d37fe640 1 --2- 172.21.15.60:0/258682562 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe3d8066f60 0x7fe3d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.347+0000 7fe3d37fe640 1 -- 172.21.15.60:0/258682562 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3dc0853f0 con 0x7fe3ec06ac10 2024-04-25T10:06:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.347+0000 7fe3ea7fc640 1 --2- 172.21.15.60:0/258682562 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe3d8066f60 0x7fe3d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.347+0000 7fe3ea7fc640 1 --2- 172.21.15.60:0/258682562 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe3d8066f60 0x7fe3d8069420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec15ae70 tx=0x7fe3e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.351+0000 7fe3d37fe640 1 -- 172.21.15.60:0/258682562 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3ec14cad0 con 0x7fe3ec06ac10 2024-04-25T10:06:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.543+0000 7fe3f1617640 1 -- 172.21.15.60:0/258682562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3ec14ff40 con 0x7fe3ec06ac10 2024-04-25T10:06:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.543+0000 7fe3d37fe640 1 -- 172.21.15.60:0/258682562 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fe3dc056490 con 0x7fe3ec06ac10 2024-04-25T10:06:54.551 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:54.551 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2422261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2422261630},{"type":"v1","addr":"172.21.15.100:6835","nonce":2422261630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2923340508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2923340508},{"type":"v1","addr":"172.21.15.100:6837","nonce":2923340508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1929386449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1929386449},{"type":"v1","addr":"172.21.15.60:6833","nonce":1929386449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:21.529455+0000","modified":"2024-04-25T10:06:23.575106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5283":{"gid":5283,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3681296171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3681296171},{"type":"v1","addr":"172.21.15.60:6835","nonce":3681296171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 15 2024-04-25T10:06:54.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.547+0000 7fe3f1617640 1 -- 172.21.15.60:0/258682562 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe3d8066f60 msgr2=0x7fe3d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.547+0000 7fe3f1617640 1 --2- 172.21.15.60:0/258682562 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe3d8066f60 0x7fe3d8069420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec15ae70 tx=0x7fe3e0009040 comp rx=0 tx=0).stop 2024-04-25T10:06:54.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.547+0000 7fe3f1617640 1 -- 172.21.15.60:0/258682562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3ec06ac10 msgr2=0x7fe3ec14bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.547+0000 7fe3f1617640 1 --2- 172.21.15.60:0/258682562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3ec06ac10 0x7fe3ec14bfc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00d5f0 tx=0x7fe3dc00dab0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.547+0000 7fe3f1617640 1 -- 172.21.15.60:0/258682562 shutdown_connections 2024-04-25T10:06:54.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.547+0000 7fe3f1617640 1 --2- 172.21.15.60:0/258682562 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe3d8066f60 0x7fe3d8069420 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.547+0000 7fe3f1617640 1 --2- 172.21.15.60:0/258682562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3ec15b950 0x7fe3ec1565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.547+0000 7fe3f1617640 1 --2- 172.21.15.60:0/258682562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3ec06ee90 0x7fe3ec156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.547+0000 7fe3f1617640 1 --2- 172.21.15.60:0/258682562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3ec06ac10 0x7fe3ec14bfc0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.547+0000 7fe3f1617640 1 -- 172.21.15.60:0/258682562 >> 172.21.15.60:0/258682562 conn(0x7fe3ec06e370 msgr2=0x7fe3ec0b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.547+0000 7fe3f1617640 1 -- 172.21.15.60:0/258682562 shutdown_connections 2024-04-25T10:06:54.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.547+0000 7fe3f1617640 1 -- 172.21.15.60:0/258682562 wait complete. 2024-04-25T10:06:54.570 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:54.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.679+0000 7f736e8bb640 1 Processor -- start 2024-04-25T10:06:54.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.679+0000 7f736e8bb640 1 -- start start 2024-04-25T10:06:54.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.679+0000 7f736e8bb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f736806ec30 0x7f73680cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.679+0000 7f736e8bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73680b37e0 0x7f73680b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.679+0000 7f736e8bb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f736806f130 con 0x7f736815a7a0 2024-04-25T10:06:54.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.679+0000 7f736e8bb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73680cdaa0 con 0x7f736806ec30 2024-04-25T10:06:54.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.679+0000 7f736e8bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736806d2f0 con 0x7f73680b37e0 2024-04-25T10:06:54.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.679+0000 7f735ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f736806ec30 0x7f73680cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.679+0000 7f735ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f736806ec30 0x7f73680cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39462/0 (socket says 172.21.15.60:39462) 2024-04-25T10:06:54.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.679+0000 7f735ffff640 1 -- 172.21.15.60:0/3367132838 learned_addr learned my addr 172.21.15.60:0/3367132838 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:54.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.679+0000 7f736ce58640 1 --2- 172.21.15.60:0/3367132838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73680b37e0 0x7f73680b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.679+0000 7f735ffff640 1 -- 172.21.15.60:0/3367132838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73680b37e0 msgr2=0x7f73680b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.679+0000 7f735ffff640 1 --2- 172.21.15.60:0/3367132838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73680b37e0 0x7f73680b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.679+0000 7f735ffff640 1 -- 172.21.15.60:0/3367132838 >> v1:172.21.15.100:6789/0 conn(0x7f736815a7a0 legacy=0x7f7368071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.679+0000 7f735ffff640 1 -- 172.21.15.60:0/3367132838 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736806f410 con 0x7f736806ec30 2024-04-25T10:06:54.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f735ffff640 1 --2- 172.21.15.60:0/3367132838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736806ec30 0x7f73680cd560 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f73500098a0 tx=0x7f735001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6192f86669d0ce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f73677fe640 1 -- 172.21.15.60:0/3367132838 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73500ab650 con 0x7f736806ec30 2024-04-25T10:06:54.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f73677fe640 1 -- 172.21.15.60:0/3367132838 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73500b0070 con 0x7f736806ec30 2024-04-25T10:06:54.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f73677fe640 1 -- 172.21.15.60:0/3367132838 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73500aba10 con 0x7f736806ec30 2024-04-25T10:06:54.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736e8bb640 1 -- 172.21.15.60:0/3367132838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736806ec30 msgr2=0x7f73680cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736e8bb640 1 --2- 172.21.15.60:0/3367132838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736806ec30 0x7f73680cd560 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f73500098a0 tx=0x7f735001c540 comp rx=0 tx=0).stop 2024-04-25T10:06:54.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736e8bb640 1 -- 172.21.15.60:0/3367132838 shutdown_connections 2024-04-25T10:06:54.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736e8bb640 1 --2- 172.21.15.60:0/3367132838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736806ec30 0x7f73680cd560 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736e8bb640 1 --2- 172.21.15.60:0/3367132838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73680b37e0 0x7f73680b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736e8bb640 1 -- 172.21.15.60:0/3367132838 >> 172.21.15.60:0/3367132838 conn(0x7f73680b12d0 msgr2=0x7f73680b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736e8bb640 1 -- 172.21.15.60:0/3367132838 shutdown_connections 2024-04-25T10:06:54.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736e8bb640 1 -- 172.21.15.60:0/3367132838 wait complete. 2024-04-25T10:06:54.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736e8bb640 1 Processor -- start 2024-04-25T10:06:54.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736e8bb640 1 -- start start 2024-04-25T10:06:54.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736e8bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f736806ec30 0x7f7368159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736e8bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680b37e0 0x7f73681facf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736e8bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 0x7f736815bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736e8bb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736806e8c0 con 0x7f736815a7a0 2024-04-25T10:06:54.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736e8bb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736806e740 con 0x7f73680b37e0 2024-04-25T10:06:54.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736e8bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736806d8b0 con 0x7f736806ec30 2024-04-25T10:06:54.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736ce58640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 0x7f736815bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f7367fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f736806ec30 0x7f7368159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f7367fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f736806ec30 0x7f7368159040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:42608/0 (socket says 172.21.15.60:42608) 2024-04-25T10:06:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736ce58640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 0x7f736815bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39466/0 (socket says 172.21.15.60:39466) 2024-04-25T10:06:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f7367fff640 1 -- 172.21.15.60:0/3572662576 learned_addr learned my addr 172.21.15.60:0/3572662576 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f735ffff640 1 --2- 172.21.15.60:0/3572662576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680b37e0 0x7f73681facf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736ce58640 1 -- 172.21.15.60:0/3572662576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f736806ec30 msgr2=0x7f7368159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736ce58640 1 --2- 172.21.15.60:0/3572662576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f736806ec30 0x7f7368159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736ce58640 1 -- 172.21.15.60:0/3572662576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680b37e0 msgr2=0x7f73681facf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736ce58640 1 --2- 172.21.15.60:0/3572662576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680b37e0 0x7f73681facf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736ce58640 1 -- 172.21.15.60:0/3572662576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736815afe0 con 0x7f736815a7a0 2024-04-25T10:06:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.683+0000 7f736ce58640 1 --2- 172.21.15.60:0/3572662576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 0x7f736815bda0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f736000efc0 tx=0x7f736000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.687+0000 7f73657fa640 1 -- 172.21.15.60:0/3572662576 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360018020 con 0x7f736815a7a0 2024-04-25T10:06:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.687+0000 7f73657fa640 1 -- 172.21.15.60:0/3572662576 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7360014480 con 0x7f736815a7a0 2024-04-25T10:06:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.687+0000 7f736e8bb640 1 -- 172.21.15.60:0/3572662576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736815b2c0 con 0x7f736815a7a0 2024-04-25T10:06:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.687+0000 7f73657fa640 1 -- 172.21.15.60:0/3572662576 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360014870 con 0x7f736815a7a0 2024-04-25T10:06:54.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.687+0000 7f736e8bb640 1 -- 172.21.15.60:0/3572662576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f736815b880 con 0x7f736815a7a0 2024-04-25T10:06:54.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.687+0000 7f736e8bb640 1 -- 172.21.15.60:0/3572662576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7368074770 con 0x7f736815a7a0 2024-04-25T10:06:54.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.687+0000 7f73657fa640 1 -- 172.21.15.60:0/3572662576 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f736001b020 con 0x7f736815a7a0 2024-04-25T10:06:54.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.687+0000 7f73657fa640 1 --2- 172.21.15.60:0/3572662576 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7348066f60 0x7f7348069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.687+0000 7f73657fa640 1 -- 172.21.15.60:0/3572662576 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7360085e10 con 0x7f736815a7a0 2024-04-25T10:06:54.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.687+0000 7f7367fff640 1 --2- 172.21.15.60:0/3572662576 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7348066f60 0x7f7348069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.687+0000 7f7367fff640 1 --2- 172.21.15.60:0/3572662576 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7348066f60 0x7f7348069420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7358007fd0 tx=0x7f7358009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.691+0000 7f73657fa640 1 -- 172.21.15.60:0/3572662576 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7360022030 con 0x7f736815a7a0 2024-04-25T10:06:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.879+0000 7f736e8bb640 1 -- 172.21.15.60:0/3572662576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73680718d0 con 0x7f736815a7a0 2024-04-25T10:06:54.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.883+0000 7f73657fa640 1 -- 172.21.15.60:0/3572662576 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f7360010040 con 0x7f736815a7a0 2024-04-25T10:06:54.888 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:54.889 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2422261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2422261630},{"type":"v1","addr":"172.21.15.100:6835","nonce":2422261630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2923340508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2923340508},{"type":"v1","addr":"172.21.15.100:6837","nonce":2923340508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1929386449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1929386449},{"type":"v1","addr":"172.21.15.60:6833","nonce":1929386449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:21.529455+0000","modified":"2024-04-25T10:06:23.575106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5283":{"gid":5283,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3681296171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3681296171},{"type":"v1","addr":"172.21.15.60:6835","nonce":3681296171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:54.889 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 15 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.883+0000 7f736e8bb640 1 -- 172.21.15.60:0/3572662576 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7348066f60 msgr2=0x7f7348069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.883+0000 7f736e8bb640 1 --2- 172.21.15.60:0/3572662576 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7348066f60 0x7f7348069420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7358007fd0 tx=0x7f7358009040 comp rx=0 tx=0).stop 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.883+0000 7f736e8bb640 1 -- 172.21.15.60:0/3572662576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 msgr2=0x7f736815bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.883+0000 7f736e8bb640 1 --2- 172.21.15.60:0/3572662576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 0x7f736815bda0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f736000efc0 tx=0x7f736000c490 comp rx=0 tx=0).stop 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.883+0000 7f736e8bb640 1 -- 172.21.15.60:0/3572662576 shutdown_connections 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.883+0000 7f736e8bb640 1 --2- 172.21.15.60:0/3572662576 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7348066f60 0x7f7348069420 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.883+0000 7f736e8bb640 1 --2- 172.21.15.60:0/3572662576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 0x7f736815bda0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.883+0000 7f736e8bb640 1 --2- 172.21.15.60:0/3572662576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680b37e0 0x7f73681facf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.883+0000 7f736e8bb640 1 --2- 172.21.15.60:0/3572662576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f736806ec30 0x7f7368159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.883+0000 7f736e8bb640 1 -- 172.21.15.60:0/3572662576 >> 172.21.15.60:0/3572662576 conn(0x7f73680b12d0 msgr2=0x7f7368154570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.883+0000 7f736e8bb640 1 -- 172.21.15.60:0/3572662576 shutdown_connections 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:54.883+0000 7f736e8bb640 1 -- 172.21.15.60:0/3572662576 wait complete. 2024-04-25T10:06:54.906 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:55.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.011+0000 7f85cd39c640 1 Processor -- start 2024-04-25T10:06:55.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.011+0000 7f85cd39c640 1 -- start start 2024-04-25T10:06:55.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85cd39c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c8161290 0x7f85c80b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85cd39c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f85c80b6fb0 0x7f85c80b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85cd39c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c80b9900 con 0x7f85c8160eb0 2024-04-25T10:06:55.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85cd39c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c80ce9c0 con 0x7f85c80b6fb0 2024-04-25T10:06:55.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85cd39c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80ceca0 con 0x7f85c8161290 2024-04-25T10:06:55.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c6ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f85c8160eb0 0x7f85c80b91f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48542/0 (socket says 172.21.15.60:48542) 2024-04-25T10:06:55.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c6ffd640 1 -- 172.21.15.60:0/1044894976 learned_addr learned my addr 172.21.15.60:0/1044894976 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:55.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85bffff640 1 --2- 172.21.15.60:0/1044894976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c8161290 0x7f85c80b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c77fe640 1 --2- 172.21.15.60:0/1044894976 >> v2:172.21.15.100:3300/0 conn(0x7f85c80b6fb0 0x7f85c80b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c67fc640 1 -- 172.21.15.60:0/1044894976 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029759666 0 0) 0x7f85c80b9900 con 0x7f85c8160eb0 2024-04-25T10:06:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c67fc640 1 -- 172.21.15.60:0/1044894976 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b4003530 con 0x7f85c8160eb0 2024-04-25T10:06:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c67fc640 1 -- 172.21.15.60:0/1044894976 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 737322598 0 0) 0x7f85b4003530 con 0x7f85c8160eb0 2024-04-25T10:06:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c67fc640 1 -- 172.21.15.60:0/1044894976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c8161290 msgr2=0x7f85c80b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c67fc640 1 --2- 172.21.15.60:0/1044894976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c8161290 0x7f85c80b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c67fc640 1 -- 172.21.15.60:0/1044894976 >> v2:172.21.15.100:3300/0 conn(0x7f85c80b6fb0 msgr2=0x7f85c80b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c67fc640 1 --2- 172.21.15.60:0/1044894976 >> v2:172.21.15.100:3300/0 conn(0x7f85c80b6fb0 0x7f85c80b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c67fc640 1 -- 172.21.15.60:0/1044894976 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c80d2490 con 0x7f85c8160eb0 2024-04-25T10:06:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c67fc640 1 -- 172.21.15.60:0/1044894976 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f85b8001680 con 0x7f85c8160eb0 2024-04-25T10:06:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c67fc640 1 -- 172.21.15.60:0/1044894976 >> v1:172.21.15.60:6789/0 conn(0x7f85c8160eb0 legacy=0x7f85c80b91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c67fc640 1 --2- 172.21.15.60:0/1044894976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c8161290 0x7f85b40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c67fc640 1 --2- 172.21.15.60:0/1044894976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b4003f90 0x7f85b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c67fc640 1 --2- 172.21.15.60:0/1044894976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85b4004d50 0x7f85b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c67fc640 1 -- 172.21.15.60:0/1044894976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80ce9c0 con 0x7f85b4004d50 2024-04-25T10:06:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c67fc640 1 -- 172.21.15.60:0/1044894976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80ceca0 con 0x7f85b4003f90 2024-04-25T10:06:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c67fc640 1 -- 172.21.15.60:0/1044894976 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b400bb10 con 0x7f85c8161290 2024-04-25T10:06:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c77fe640 1 --2- 172.21.15.60:0/1044894976 >> v2:172.21.15.100:3300/0 conn(0x7f85c80b6fb0 0x7f85c80b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c77fe640 1 --2- 172.21.15.60:0/1044894976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85b4004d50 0x7f85b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c6ffd640 1 --2- 172.21.15.60:0/1044894976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c8161290 0x7f85b40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85bffff640 1 --2- 172.21.15.60:0/1044894976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b4003f90 0x7f85b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c77fe640 1 -- 172.21.15.60:0/1044894976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c8161290 msgr2=0x7f85b40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c77fe640 1 --2- 172.21.15.60:0/1044894976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c8161290 0x7f85b40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c77fe640 1 -- 172.21.15.60:0/1044894976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b4003f90 msgr2=0x7f85b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c77fe640 1 --2- 172.21.15.60:0/1044894976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85b4003f90 0x7f85b4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.015+0000 7f85c77fe640 1 -- 172.21.15.60:0/1044894976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b4003530 con 0x7f85b4004d50 2024-04-25T10:06:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85c77fe640 1 --2- 172.21.15.60:0/1044894976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85b4004d50 0x7f85b40051d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f85c00073b0 tx=0x7f85c001cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db70ef38391af7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85c67fc640 1 -- 172.21.15.60:0/1044894976 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c00b2070 con 0x7f85b4004d50 2024-04-25T10:06:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85c67fc640 1 -- 172.21.15.60:0/1044894976 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c00a5e20 con 0x7f85b4004d50 2024-04-25T10:06:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85c67fc640 1 -- 172.21.15.60:0/1044894976 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0009c70 con 0x7f85b4004d50 2024-04-25T10:06:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85cd39c640 1 -- 172.21.15.60:0/1044894976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85b4004d50 msgr2=0x7f85b40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85cd39c640 1 --2- 172.21.15.60:0/1044894976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85b4004d50 0x7f85b40051d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f85c00073b0 tx=0x7f85c001cf20 comp rx=0 tx=0).stop 2024-04-25T10:06:55.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85c77fe640 1 -- 172.21.15.60:0/1044894976 reap_dead start 2024-04-25T10:06:55.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85cd39c640 1 -- 172.21.15.60:0/1044894976 shutdown_connections 2024-04-25T10:06:55.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85cd39c640 1 -- 172.21.15.60:0/1044894976 >> 172.21.15.60:0/1044894976 conn(0x7f85c80d05c0 msgr2=0x7f85c80d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85cd39c640 1 -- 172.21.15.60:0/1044894976 shutdown_connections 2024-04-25T10:06:55.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85cd39c640 1 -- 172.21.15.60:0/1044894976 wait complete. 2024-04-25T10:06:55.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85cd39c640 1 Processor -- start 2024-04-25T10:06:55.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85cd39c640 1 -- start start 2024-04-25T10:06:55.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85cd39c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c81f2e80 0x7f85c81f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85cd39c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c81f6ea0 0x7f85c80cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85cd39c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c80cca50 0x7f85c814c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85cd39c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80cef80 con 0x7f85c81f2e80 2024-04-25T10:06:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85cd39c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80b2e70 con 0x7f85c81f6ea0 2024-04-25T10:06:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85cd39c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80b6a70 con 0x7f85c80cca50 2024-04-25T10:06:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85c6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c81f2e80 0x7f85c81f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85c6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c81f2e80 0x7f85c81f6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39496/0 (socket says 172.21.15.60:39496) 2024-04-25T10:06:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85c6ffd640 1 -- 172.21.15.60:0/2049708124 learned_addr learned my addr 172.21.15.60:0/2049708124 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85c77fe640 1 --2- 172.21.15.60:0/2049708124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c80cca50 0x7f85c814c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85bffff640 1 --2- 172.21.15.60:0/2049708124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c81f6ea0 0x7f85c80cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85c6ffd640 1 -- 172.21.15.60:0/2049708124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c80cca50 msgr2=0x7f85c814c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85c6ffd640 1 --2- 172.21.15.60:0/2049708124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c80cca50 0x7f85c814c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85c6ffd640 1 -- 172.21.15.60:0/2049708124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c81f6ea0 msgr2=0x7f85c80cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85c6ffd640 1 --2- 172.21.15.60:0/2049708124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c81f6ea0 0x7f85c80cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85c6ffd640 1 -- 172.21.15.60:0/2049708124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c8150900 con 0x7f85c81f2e80 2024-04-25T10:06:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85c77fe640 1 --2- 172.21.15.60:0/2049708124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c80cca50 0x7f85c814c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85c6ffd640 1 --2- 172.21.15.60:0/2049708124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c81f2e80 0x7f85c81f6960 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f85b8001ae0 tx=0x7f85b8004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85bf7fe640 1 -- 172.21.15.60:0/2049708124 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b800fa80 con 0x7f85c81f2e80 2024-04-25T10:06:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85bf7fe640 1 -- 172.21.15.60:0/2049708124 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b8018070 con 0x7f85c81f2e80 2024-04-25T10:06:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85cd39c640 1 -- 172.21.15.60:0/2049708124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c80cd280 con 0x7f85c81f2e80 2024-04-25T10:06:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85bf7fe640 1 -- 172.21.15.60:0/2049708124 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b800fea0 con 0x7f85c81f2e80 2024-04-25T10:06:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.019+0000 7f85cd39c640 1 -- 172.21.15.60:0/2049708124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c814d940 con 0x7f85c81f2e80 2024-04-25T10:06:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.023+0000 7f85cd39c640 1 -- 172.21.15.60:0/2049708124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c8074770 con 0x7f85c81f2e80 2024-04-25T10:06:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.023+0000 7f85bf7fe640 1 -- 172.21.15.60:0/2049708124 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85b801d460 con 0x7f85c81f2e80 2024-04-25T10:06:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.023+0000 7f85bf7fe640 1 --2- 172.21.15.60:0/2049708124 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f85a8066f90 0x7f85a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.023+0000 7f85bf7fe640 1 -- 172.21.15.60:0/2049708124 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85b8085180 con 0x7f85c81f2e80 2024-04-25T10:06:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.023+0000 7f85bffff640 1 --2- 172.21.15.60:0/2049708124 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f85a8066f90 0x7f85a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.023+0000 7f85bffff640 1 --2- 172.21.15.60:0/2049708124 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f85a8066f90 0x7f85a8069450 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f85c80cf600 tx=0x7f85b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.027+0000 7f85bf7fe640 1 -- 172.21.15.60:0/2049708124 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f85c8150900 con 0x7f85c81f2e80 2024-04-25T10:06:55.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.183+0000 7f85cd39c640 1 -- 172.21.15.60:0/2049708124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85c80d15b0 con 0x7f85c81f2e80 2024-04-25T10:06:55.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.183+0000 7f85bf7fe640 1 -- 172.21.15.60:0/2049708124 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16681 (secure 0 0 0) 0x7f85b8056190 con 0x7f85c81f2e80 2024-04-25T10:06:55.191 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:55.192 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":35,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:23.196554+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:20.088165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:20.507719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.187+0000 7f85cd39c640 1 -- 172.21.15.60:0/2049708124 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f85a8066f90 msgr2=0x7f85a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.187+0000 7f85cd39c640 1 --2- 172.21.15.60:0/2049708124 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f85a8066f90 0x7f85a8069450 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f85c80cf600 tx=0x7f85b0008040 comp rx=0 tx=0).stop 2024-04-25T10:06:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.187+0000 7f85cd39c640 1 -- 172.21.15.60:0/2049708124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c81f2e80 msgr2=0x7f85c81f6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.187+0000 7f85cd39c640 1 --2- 172.21.15.60:0/2049708124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c81f2e80 0x7f85c81f6960 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f85b8001ae0 tx=0x7f85b8004f90 comp rx=0 tx=0).stop 2024-04-25T10:06:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.187+0000 7f85cd39c640 1 -- 172.21.15.60:0/2049708124 shutdown_connections 2024-04-25T10:06:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.187+0000 7f85cd39c640 1 --2- 172.21.15.60:0/2049708124 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f85a8066f90 0x7f85a8069450 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.187+0000 7f85cd39c640 1 --2- 172.21.15.60:0/2049708124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c80cca50 0x7f85c814c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.187+0000 7f85cd39c640 1 --2- 172.21.15.60:0/2049708124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c81f6ea0 0x7f85c80cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.187+0000 7f85cd39c640 1 --2- 172.21.15.60:0/2049708124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c81f2e80 0x7f85c81f6960 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.187+0000 7f85cd39c640 1 -- 172.21.15.60:0/2049708124 >> 172.21.15.60:0/2049708124 conn(0x7f85c80d05c0 msgr2=0x7f85c8160110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.187+0000 7f85cd39c640 1 -- 172.21.15.60:0/2049708124 shutdown_connections 2024-04-25T10:06:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.187+0000 7f85cd39c640 1 -- 172.21.15.60:0/2049708124 wait complete. 2024-04-25T10:06:55.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:06:55.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fd0b06640 1 Processor -- start 2024-04-25T10:06:55.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fd0b06640 1 -- start start 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fd0b06640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc15bc90 0x7f1fcc15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fd0b06640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc071ca0 con 0x7f1fcc0ce7a0 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fd0b06640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc0cca40 con 0x7f1fcc0b4660 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fd0b06640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0cc370 con 0x7f1fcc15bc90 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fca59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1fcc0ce7a0 0x7f1fcc0ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48550/0 (socket says 172.21.15.60:48550) 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fca59c640 1 -- 172.21.15.60:0/1051210772 learned_addr learned my addr 172.21.15.60:0/1051210772 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fc9d9b640 1 --2- 172.21.15.60:0/1051210772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc15bc90 0x7f1fcc15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fc959a640 1 -- 172.21.15.60:0/1051210772 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080343475 0 0) 0x7f1fcc071ca0 con 0x7f1fcc0ce7a0 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fc959a640 1 -- 172.21.15.60:0/1051210772 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fb4003530 con 0x7f1fcc0ce7a0 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fc959a640 1 -- 172.21.15.60:0/1051210772 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3282012047 0 0) 0x7f1fb4003530 con 0x7f1fcc0ce7a0 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fc959a640 1 -- 172.21.15.60:0/1051210772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc15bc90 msgr2=0x7f1fcc15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fc959a640 1 --2- 172.21.15.60:0/1051210772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc15bc90 0x7f1fcc15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fc959a640 1 -- 172.21.15.60:0/1051210772 >> v1:172.21.15.100:6789/0 conn(0x7f1fcc0b4660 legacy=0x7f1fcc15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fc959a640 1 -- 172.21.15.60:0/1051210772 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc0ccd20 con 0x7f1fcc0ce7a0 2024-04-25T10:06:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fc959a640 1 -- 172.21.15.60:0/1051210772 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f1fbc001680 con 0x7f1fcc0ce7a0 2024-04-25T10:06:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fc959a640 1 -- 172.21.15.60:0/1051210772 >> v1:172.21.15.60:6789/0 conn(0x7f1fcc0ce7a0 legacy=0x7f1fcc0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fc959a640 1 --2- 172.21.15.60:0/1051210772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc15bc90 0x7f1fb40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fc959a640 1 --2- 172.21.15.60:0/1051210772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb4003fc0 0x7f1fb4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fc959a640 1 --2- 172.21.15.60:0/1051210772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fb4004d50 0x7f1fb40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fc959a640 1 -- 172.21.15.60:0/1051210772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb400bae0 con 0x7f1fb4003fc0 2024-04-25T10:06:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.315+0000 7f1fc959a640 1 -- 172.21.15.60:0/1051210772 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb400bc90 con 0x7f1fb4004d50 2024-04-25T10:06:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fc959a640 1 -- 172.21.15.60:0/1051210772 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb400be90 con 0x7f1fcc15bc90 2024-04-25T10:06:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fc9d9b640 1 --2- 172.21.15.60:0/1051210772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb4003fc0 0x7f1fb4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fcad9d640 1 --2- 172.21.15.60:0/1051210772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fb4004d50 0x7f1fb40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fca59c640 1 --2- 172.21.15.60:0/1051210772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc15bc90 0x7f1fb40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fc9d9b640 1 -- 172.21.15.60:0/1051210772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc15bc90 msgr2=0x7f1fb40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fc9d9b640 1 --2- 172.21.15.60:0/1051210772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc15bc90 0x7f1fb40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fc9d9b640 1 -- 172.21.15.60:0/1051210772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fb4004d50 msgr2=0x7f1fb40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fc9d9b640 1 --2- 172.21.15.60:0/1051210772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fb4004d50 0x7f1fb40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fc9d9b640 1 -- 172.21.15.60:0/1051210772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb4003530 con 0x7f1fb4003fc0 2024-04-25T10:06:55.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fc9d9b640 1 --2- 172.21.15.60:0/1051210772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb4003fc0 0x7f1fb4004810 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009ae0 tx=0x7f1fc001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e3c96143e83483 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fc959a640 1 -- 172.21.15.60:0/1051210772 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00b2070 con 0x7f1fb4003fc0 2024-04-25T10:06:55.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fc959a640 1 -- 172.21.15.60:0/1051210772 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc00a55a0 con 0x7f1fb4003fc0 2024-04-25T10:06:55.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fc959a640 1 -- 172.21.15.60:0/1051210772 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00096b0 con 0x7f1fb4003fc0 2024-04-25T10:06:55.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fd0b06640 1 -- 172.21.15.60:0/1051210772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb4003fc0 msgr2=0x7f1fb4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fd0b06640 1 --2- 172.21.15.60:0/1051210772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb4003fc0 0x7f1fb4004810 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009ae0 tx=0x7f1fc001c9b0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fcad9d640 1 -- 172.21.15.60:0/1051210772 reap_dead start 2024-04-25T10:06:55.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fd0b06640 1 -- 172.21.15.60:0/1051210772 shutdown_connections 2024-04-25T10:06:55.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fd0b06640 1 -- 172.21.15.60:0/1051210772 >> 172.21.15.60:0/1051210772 conn(0x7f1fcc0b12d0 msgr2=0x7f1fcc0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fd0b06640 1 -- 172.21.15.60:0/1051210772 shutdown_connections 2024-04-25T10:06:55.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.319+0000 7f1fd0b06640 1 -- 172.21.15.60:0/1051210772 wait complete. 2024-04-25T10:06:55.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fd0b06640 1 Processor -- start 2024-04-25T10:06:55.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fd0b06640 1 -- start start 2024-04-25T10:06:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fd0b06640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fb4003fc0 0x7f1fcc12e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fd0b06640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc15eac0 0x7f1fcc06e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fd0b06640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fcc06e8c0 0x7f1fcc0af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fd0b06640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0b3d20 con 0x7f1fcc06e8c0 2024-04-25T10:06:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fd0b06640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0b3ba0 con 0x7f1fb4003fc0 2024-04-25T10:06:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fd0b06640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0b4280 con 0x7f1fcc15eac0 2024-04-25T10:06:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fcad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fcc06e8c0 0x7f1fcc0af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fcad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fcc06e8c0 0x7f1fcc0af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39512/0 (socket says 172.21.15.60:39512) 2024-04-25T10:06:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fcad9d640 1 -- 172.21.15.60:0/3207213375 learned_addr learned my addr 172.21.15.60:0/3207213375 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fc9d9b640 1 --2- 172.21.15.60:0/3207213375 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc15eac0 0x7f1fcc06e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fca59c640 1 --2- 172.21.15.60:0/3207213375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fb4003fc0 0x7f1fcc12e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fcad9d640 1 -- 172.21.15.60:0/3207213375 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc15eac0 msgr2=0x7f1fcc06e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fcad9d640 1 --2- 172.21.15.60:0/3207213375 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc15eac0 0x7f1fcc06e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fcad9d640 1 -- 172.21.15.60:0/3207213375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fb4003fc0 msgr2=0x7f1fcc12e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fcad9d640 1 --2- 172.21.15.60:0/3207213375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fb4003fc0 0x7f1fcc12e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fcad9d640 1 -- 172.21.15.60:0/3207213375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc151560 con 0x7f1fcc06e8c0 2024-04-25T10:06:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fcad9d640 1 --2- 172.21.15.60:0/3207213375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fcc06e8c0 0x7f1fcc0af140 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc072c10 tx=0x7f1fc400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fbaffd640 1 -- 172.21.15.60:0/3207213375 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4003980 con 0x7f1fcc06e8c0 2024-04-25T10:06:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fbaffd640 1 -- 172.21.15.60:0/3207213375 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc4018070 con 0x7f1fcc06e8c0 2024-04-25T10:06:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fd0b06640 1 -- 172.21.15.60:0/3207213375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc151840 con 0x7f1fcc06e8c0 2024-04-25T10:06:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fbaffd640 1 -- 172.21.15.60:0/3207213375 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4003d60 con 0x7f1fcc06e8c0 2024-04-25T10:06:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fd0b06640 1 -- 172.21.15.60:0/3207213375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc151e00 con 0x7f1fcc06e8c0 2024-04-25T10:06:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fd0b06640 1 -- 172.21.15.60:0/3207213375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc074770 con 0x7f1fcc06e8c0 2024-04-25T10:06:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fbaffd640 1 -- 172.21.15.60:0/3207213375 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1fc40050d0 con 0x7f1fcc06e8c0 2024-04-25T10:06:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fbaffd640 1 --2- 172.21.15.60:0/3207213375 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1fac066f90 0x7f1fac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.323+0000 7f1fbaffd640 1 -- 172.21.15.60:0/3207213375 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fc4084d80 con 0x7f1fcc06e8c0 2024-04-25T10:06:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.327+0000 7f1fca59c640 1 --2- 172.21.15.60:0/3207213375 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1fac066f90 0x7f1fac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.327+0000 7f1fca59c640 1 --2- 172.21.15.60:0/3207213375 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1fac066f90 0x7f1fac069450 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc000fc0 tx=0x7f1fbc0047f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.327+0000 7f1fbaffd640 1 -- 172.21.15.60:0/3207213375 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1fc4055d90 con 0x7f1fcc06e8c0 2024-04-25T10:06:55.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.515+0000 7f1fd0b06640 1 -- 172.21.15.60:0/3207213375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1fcc12e9f0 con 0x7f1fcc06e8c0 2024-04-25T10:06:55.522 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:06:55.515+0000 7f3a792d6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:06:55.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.519+0000 7f1fbaffd640 1 -- 172.21.15.60:0/3207213375 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1fc4055690 con 0x7f1fcc06e8c0 2024-04-25T10:06:55.525 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:06:55.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.523+0000 7f1fd0b06640 1 -- 172.21.15.60:0/3207213375 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1fac066f90 msgr2=0x7f1fac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.523+0000 7f1fd0b06640 1 --2- 172.21.15.60:0/3207213375 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1fac066f90 0x7f1fac069450 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc000fc0 tx=0x7f1fbc0047f0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.523+0000 7f1fd0b06640 1 -- 172.21.15.60:0/3207213375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fcc06e8c0 msgr2=0x7f1fcc0af140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.523+0000 7f1fd0b06640 1 --2- 172.21.15.60:0/3207213375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fcc06e8c0 0x7f1fcc0af140 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc072c10 tx=0x7f1fc400ec10 comp rx=0 tx=0).stop 2024-04-25T10:06:55.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.523+0000 7f1fd0b06640 1 -- 172.21.15.60:0/3207213375 shutdown_connections 2024-04-25T10:06:55.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.523+0000 7f1fd0b06640 1 --2- 172.21.15.60:0/3207213375 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1fac066f90 0x7f1fac069450 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.523+0000 7f1fd0b06640 1 --2- 172.21.15.60:0/3207213375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fcc06e8c0 0x7f1fcc0af140 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.523+0000 7f1fd0b06640 1 --2- 172.21.15.60:0/3207213375 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc15eac0 0x7f1fcc06e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.523+0000 7f1fd0b06640 1 --2- 172.21.15.60:0/3207213375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fb4003fc0 0x7f1fcc12e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.523+0000 7f1fd0b06640 1 -- 172.21.15.60:0/3207213375 >> 172.21.15.60:0/3207213375 conn(0x7f1fcc0b12d0 msgr2=0x7f1fcc0b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.523+0000 7f1fd0b06640 1 -- 172.21.15.60:0/3207213375 shutdown_connections 2024-04-25T10:06:55.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.523+0000 7f1fd0b06640 1 -- 172.21.15.60:0/3207213375 wait complete. 2024-04-25T10:06:55.538 INFO:tasks.ceph.mds.c.smithi060.stderr: -5818> 2024-04-25T10:06:27.135+0000 7fc9741c4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:06:55.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:06:55.578 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:06:55.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.659+0000 7efc47fd7640 1 Processor -- start 2024-04-25T10:06:55.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.659+0000 7efc47fd7640 1 -- start start 2024-04-25T10:06:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.659+0000 7efc47fd7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc400b35f0 0x7efc400ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.659+0000 7efc47fd7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4015def0 con 0x7efc400aec10 2024-04-25T10:06:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.659+0000 7efc47fd7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc40071630 con 0x7efc4015db10 2024-04-25T10:06:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.659+0000 7efc47fd7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc400accc0 con 0x7efc400b35f0 2024-04-25T10:06:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.659+0000 7efc46574640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efc400aec10 0x7efc400ad670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48564/0 (socket says 172.21.15.60:48564) 2024-04-25T10:06:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.659+0000 7efc46574640 1 -- 172.21.15.60:0/1792224103 learned_addr learned my addr 172.21.15.60:0/1792224103 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.659+0000 7efc45572640 1 --2- 172.21.15.60:0/1792224103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc400b35f0 0x7efc400ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 -- 172.21.15.60:0/1792224103 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34489680 0 0) 0x7efc4015def0 con 0x7efc400aec10 2024-04-25T10:06:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 -- 172.21.15.60:0/1792224103 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc2c003530 con 0x7efc400aec10 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 -- 172.21.15.60:0/1792224103 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1620069632 0 0) 0x7efc2c003530 con 0x7efc400aec10 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 -- 172.21.15.60:0/1792224103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc400b35f0 msgr2=0x7efc400ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 --2- 172.21.15.60:0/1792224103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc400b35f0 0x7efc400ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 -- 172.21.15.60:0/1792224103 >> v1:172.21.15.100:6789/0 conn(0x7efc4015db10 legacy=0x7efc400b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 -- 172.21.15.60:0/1792224103 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc40071910 con 0x7efc400aec10 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 -- 172.21.15.60:0/1792224103 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7efc38001a00 con 0x7efc400aec10 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 -- 172.21.15.60:0/1792224103 >> v1:172.21.15.60:6789/0 conn(0x7efc400aec10 legacy=0x7efc400ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 --2- 172.21.15.60:0/1792224103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc400b35f0 0x7efc2c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 --2- 172.21.15.60:0/1792224103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc2c003f70 0x7efc2c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 --2- 172.21.15.60:0/1792224103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2c004d50 0x7efc2c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 -- 172.21.15.60:0/1792224103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c00ba80 con 0x7efc2c004d50 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 -- 172.21.15.60:0/1792224103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c00bc30 con 0x7efc2c003f70 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 -- 172.21.15.60:0/1792224103 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c00be30 con 0x7efc400b35f0 2024-04-25T10:06:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc46574640 1 --2- 172.21.15.60:0/1792224103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2c004d50 0x7efc2c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc45d73640 1 --2- 172.21.15.60:0/1792224103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc400b35f0 0x7efc2c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc45572640 1 --2- 172.21.15.60:0/1792224103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc2c003f70 0x7efc2c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc46574640 1 -- 172.21.15.60:0/1792224103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc400b35f0 msgr2=0x7efc2c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc46574640 1 --2- 172.21.15.60:0/1792224103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc400b35f0 0x7efc2c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc46574640 1 -- 172.21.15.60:0/1792224103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc2c003f70 msgr2=0x7efc2c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc46574640 1 --2- 172.21.15.60:0/1792224103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc2c003f70 0x7efc2c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc46574640 1 -- 172.21.15.60:0/1792224103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2c003530 con 0x7efc2c004d50 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc46574640 1 --2- 172.21.15.60:0/1792224103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2c004d50 0x7efc2c0051d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efc3800a860 tx=0x7efc3801d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faabb82cd4e61745 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 -- 172.21.15.60:0/1792224103 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3800a480 con 0x7efc2c004d50 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 -- 172.21.15.60:0/1792224103 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc380ad630 con 0x7efc2c004d50 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc44d71640 1 -- 172.21.15.60:0/1792224103 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc380ada80 con 0x7efc2c004d50 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc47fd7640 1 -- 172.21.15.60:0/1792224103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2c004d50 msgr2=0x7efc2c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc47fd7640 1 --2- 172.21.15.60:0/1792224103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2c004d50 0x7efc2c0051d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efc3800a860 tx=0x7efc3801d380 comp rx=0 tx=0).stop 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc46574640 1 -- 172.21.15.60:0/1792224103 reap_dead start 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc47fd7640 1 -- 172.21.15.60:0/1792224103 shutdown_connections 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc47fd7640 1 -- 172.21.15.60:0/1792224103 >> 172.21.15.60:0/1792224103 conn(0x7efc400670b0 msgr2=0x7efc4006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc47fd7640 1 -- 172.21.15.60:0/1792224103 shutdown_connections 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.663+0000 7efc47fd7640 1 -- 172.21.15.60:0/1792224103 wait complete. 2024-04-25T10:06:55.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc47fd7640 1 Processor -- start 2024-04-25T10:06:55.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc47fd7640 1 -- start start 2024-04-25T10:06:55.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc47fd7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc2c004d50 0x7efc40161580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc47fd7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc40160340 0x7efc4014aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc47fd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc40154e50 0x7efc4014af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc47fd7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4006db70 con 0x7efc40160340 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc47fd7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc40071ed0 con 0x7efc40154e50 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc47fd7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4006dcf0 con 0x7efc2c004d50 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc45572640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc40160340 0x7efc4014aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc45572640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc40160340 0x7efc4014aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39560/0 (socket says 172.21.15.60:39560) 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc45572640 1 -- 172.21.15.60:0/3793904265 learned_addr learned my addr 172.21.15.60:0/3793904265 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc45d73640 1 --2- 172.21.15.60:0/3793904265 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc2c004d50 0x7efc40161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc46574640 1 --2- 172.21.15.60:0/3793904265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc40154e50 0x7efc4014af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc45572640 1 -- 172.21.15.60:0/3793904265 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc2c004d50 msgr2=0x7efc40161580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc45572640 1 --2- 172.21.15.60:0/3793904265 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc2c004d50 0x7efc40161580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc45572640 1 -- 172.21.15.60:0/3793904265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc40154e50 msgr2=0x7efc4014af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc45572640 1 --2- 172.21.15.60:0/3793904265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc40154e50 0x7efc4014af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc45572640 1 -- 172.21.15.60:0/3793904265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4015bf40 con 0x7efc40160340 2024-04-25T10:06:55.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc45d73640 1 --2- 172.21.15.60:0/3793904265 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc2c004d50 0x7efc40161580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:55.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc45572640 1 --2- 172.21.15.60:0/3793904265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc40160340 0x7efc4014aa30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0115e0 tx=0x7efc3c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc367fc640 1 -- 172.21.15.60:0/3793904265 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c00f8f0 con 0x7efc40160340 2024-04-25T10:06:55.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc367fc640 1 -- 172.21.15.60:0/3793904265 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3c00fa90 con 0x7efc40160340 2024-04-25T10:06:55.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc47fd7640 1 -- 172.21.15.60:0/3793904265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc400af120 con 0x7efc40160340 2024-04-25T10:06:55.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc367fc640 1 -- 172.21.15.60:0/3793904265 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c00fe40 con 0x7efc40160340 2024-04-25T10:06:55.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc47fd7640 1 -- 172.21.15.60:0/3793904265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc400af6a0 con 0x7efc40160340 2024-04-25T10:06:55.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.667+0000 7efc47fd7640 1 -- 172.21.15.60:0/3793904265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4006bba0 con 0x7efc40160340 2024-04-25T10:06:55.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.671+0000 7efc367fc640 1 -- 172.21.15.60:0/3793904265 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efc3c015460 con 0x7efc40160340 2024-04-25T10:06:55.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.671+0000 7efc367fc640 1 --2- 172.21.15.60:0/3793904265 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc24066f60 0x7efc24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.671+0000 7efc367fc640 1 -- 172.21.15.60:0/3793904265 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5795+0+0 (secure 0 0 0) 0x7efc3c085a40 con 0x7efc40160340 2024-04-25T10:06:55.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.671+0000 7efc45d73640 1 --2- 172.21.15.60:0/3793904265 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc24066f60 0x7efc24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.671+0000 7efc45d73640 1 --2- 172.21.15.60:0/3793904265 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc24066f60 0x7efc24069420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efc30007fd0 tx=0x7efc30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.675+0000 7efc367fc640 1 -- 172.21.15.60:0/3793904265 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efc3c019030 con 0x7efc40160340 2024-04-25T10:06:55.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:55.859+0000 7efc47fd7640 1 -- 172.21.15.60:0/3793904265 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7efc40074770 con 0x7efc40160340 2024-04-25T10:06:56.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.543+0000 7efc367fc640 1 -- 172.21.15.60:0/3793904265 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7efc3c056a80 con 0x7efc40160340 2024-04-25T10:06:56.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.547+0000 7efc47fd7640 1 -- 172.21.15.60:0/3793904265 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc24066f60 msgr2=0x7efc24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.547+0000 7efc47fd7640 1 --2- 172.21.15.60:0/3793904265 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc24066f60 0x7efc24069420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efc30007fd0 tx=0x7efc30009040 comp rx=0 tx=0).stop 2024-04-25T10:06:56.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.547+0000 7efc47fd7640 1 -- 172.21.15.60:0/3793904265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc40160340 msgr2=0x7efc4014aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.547+0000 7efc47fd7640 1 --2- 172.21.15.60:0/3793904265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc40160340 0x7efc4014aa30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0115e0 tx=0x7efc3c011aa0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.547+0000 7efc47fd7640 1 -- 172.21.15.60:0/3793904265 shutdown_connections 2024-04-25T10:06:56.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.547+0000 7efc47fd7640 1 --2- 172.21.15.60:0/3793904265 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc24066f60 0x7efc24069420 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.547+0000 7efc47fd7640 1 --2- 172.21.15.60:0/3793904265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc40154e50 0x7efc4014af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.547+0000 7efc47fd7640 1 --2- 172.21.15.60:0/3793904265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc40160340 0x7efc4014aa30 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.547+0000 7efc47fd7640 1 --2- 172.21.15.60:0/3793904265 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc2c004d50 0x7efc40161580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.547+0000 7efc47fd7640 1 -- 172.21.15.60:0/3793904265 >> 172.21.15.60:0/3793904265 conn(0x7efc400670b0 msgr2=0x7efc400cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.547+0000 7efc47fd7640 1 -- 172.21.15.60:0/3793904265 shutdown_connections 2024-04-25T10:06:56.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.547+0000 7efc47fd7640 1 -- 172.21.15.60:0/3793904265 wait complete. 2024-04-25T10:06:56.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:06:56.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.751+0000 7f9c2c5cb640 1 Processor -- start 2024-04-25T10:06:56.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.751+0000 7f9c2c5cb640 1 -- start start 2024-04-25T10:06:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.751+0000 7f9c2c5cb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9c24071980 0x7f9c240ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.751+0000 7f9c2c5cb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c240cf0a0 0x7f9c240b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.751+0000 7f9c2c5cb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c240cf810 con 0x7f9c24144e40 2024-04-25T10:06:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.751+0000 7f9c2c5cb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c240b3060 con 0x7f9c240cf0a0 2024-04-25T10:06:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.751+0000 7f9c2c5cb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c240cfaf0 con 0x7f9c24071980 2024-04-25T10:06:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.751+0000 7f9c2ab68640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c240cf0a0 0x7f9c240b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.751+0000 7f9c2ab68640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c240cf0a0 0x7f9c240b34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39576/0 (socket says 172.21.15.60:39576) 2024-04-25T10:06:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.751+0000 7f9c2ab68640 1 -- 172.21.15.60:0/4186490463 learned_addr learned my addr 172.21.15.60:0/4186490463 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.751+0000 7f9c29b66640 1 --2- 172.21.15.60:0/4186490463 >> v2:172.21.15.100:3300/0 conn(0x7f9c24071980 0x7f9c240ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.751+0000 7f9c2ab68640 1 -- 172.21.15.60:0/4186490463 >> v2:172.21.15.100:3300/0 conn(0x7f9c24071980 msgr2=0x7f9c240ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.751+0000 7f9c2ab68640 1 --2- 172.21.15.60:0/4186490463 >> v2:172.21.15.100:3300/0 conn(0x7f9c24071980 0x7f9c240ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.751+0000 7f9c2ab68640 1 -- 172.21.15.60:0/4186490463 >> v1:172.21.15.100:6789/0 conn(0x7f9c24144e40 legacy=0x7f9c24071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.751+0000 7f9c2ab68640 1 -- 172.21.15.60:0/4186490463 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c240cfdd0 con 0x7f9c240cf0a0 2024-04-25T10:06:56.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2ab68640 1 --2- 172.21.15.60:0/4186490463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c240cf0a0 0x7f9c240b34f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9c2406b9d0 tx=0x7f9c1c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4038dfc92af5d393 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c29365640 1 -- 172.21.15.60:0/4186490463 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c0a58e0 con 0x7f9c240cf0a0 2024-04-25T10:06:56.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c29365640 1 -- 172.21.15.60:0/4186490463 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c24145430 con 0x7f9c240cf0a0 2024-04-25T10:06:56.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c29365640 1 -- 172.21.15.60:0/4186490463 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c0a5ca0 con 0x7f9c240cf0a0 2024-04-25T10:06:56.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2c5cb640 1 -- 172.21.15.60:0/4186490463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c240cf0a0 msgr2=0x7f9c240b34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2c5cb640 1 --2- 172.21.15.60:0/4186490463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c240cf0a0 0x7f9c240b34f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9c2406b9d0 tx=0x7f9c1c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2c5cb640 1 -- 172.21.15.60:0/4186490463 shutdown_connections 2024-04-25T10:06:56.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2c5cb640 1 --2- 172.21.15.60:0/4186490463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c240cf0a0 0x7f9c240b34f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2c5cb640 1 --2- 172.21.15.60:0/4186490463 >> v2:172.21.15.100:3300/0 conn(0x7f9c24071980 0x7f9c240ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2c5cb640 1 -- 172.21.15.60:0/4186490463 >> 172.21.15.60:0/4186490463 conn(0x7f9c24066fa0 msgr2=0x7f9c2406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2c5cb640 1 -- 172.21.15.60:0/4186490463 shutdown_connections 2024-04-25T10:06:56.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2c5cb640 1 -- 172.21.15.60:0/4186490463 wait complete. 2024-04-25T10:06:56.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2c5cb640 1 Processor -- start 2024-04-25T10:06:56.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2c5cb640 1 -- start start 2024-04-25T10:06:56.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2c5cb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c24071980 0x7f9c240ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2c5cb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c240cf0a0 0x7f9c2406e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2c5cb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c24144e40 0x7f9c2415a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2c5cb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c240b45d0 con 0x7f9c24144e40 2024-04-25T10:06:56.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2c5cb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c240b4450 con 0x7f9c240cf0a0 2024-04-25T10:06:56.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2c5cb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c240b4750 con 0x7f9c24071980 2024-04-25T10:06:56.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2ab68640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c24144e40 0x7f9c2415a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2ab68640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c24144e40 0x7f9c2415a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39582/0 (socket says 172.21.15.60:39582) 2024-04-25T10:06:56.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2ab68640 1 -- 172.21.15.60:0/1873274860 learned_addr learned my addr 172.21.15.60:0/1873274860 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:56.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c29b66640 1 --2- 172.21.15.60:0/1873274860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c240cf0a0 0x7f9c2406e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2a367640 1 --2- 172.21.15.60:0/1873274860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c24071980 0x7f9c240ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2ab68640 1 -- 172.21.15.60:0/1873274860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c24071980 msgr2=0x7f9c240ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2ab68640 1 --2- 172.21.15.60:0/1873274860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c24071980 0x7f9c240ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2ab68640 1 -- 172.21.15.60:0/1873274860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c240cf0a0 msgr2=0x7f9c2406e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2ab68640 1 --2- 172.21.15.60:0/1873274860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c240cf0a0 0x7f9c2406e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2ab68640 1 -- 172.21.15.60:0/1873274860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2415ace0 con 0x7f9c24144e40 2024-04-25T10:06:56.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2a367640 1 --2- 172.21.15.60:0/1873274860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c24071980 0x7f9c240ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:56.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c29b66640 1 --2- 172.21.15.60:0/1873274860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c240cf0a0 0x7f9c2406e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:56.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c2ab68640 1 --2- 172.21.15.60:0/1873274860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c24144e40 0x7f9c2415a7a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c019600 tx=0x7f9c240cf700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.755+0000 7f9c1affd640 1 -- 172.21.15.60:0/1873274860 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c0bc030 con 0x7f9c24144e40 2024-04-25T10:06:56.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.759+0000 7f9c1affd640 1 -- 172.21.15.60:0/1873274860 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c0b8470 con 0x7f9c24144e40 2024-04-25T10:06:56.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.759+0000 7f9c2c5cb640 1 -- 172.21.15.60:0/1873274860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2406d470 con 0x7f9c24144e40 2024-04-25T10:06:56.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.759+0000 7f9c1affd640 1 -- 172.21.15.60:0/1873274860 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c0b8840 con 0x7f9c24144e40 2024-04-25T10:06:56.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.759+0000 7f9c2c5cb640 1 -- 172.21.15.60:0/1873274860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2406d9f0 con 0x7f9c24144e40 2024-04-25T10:06:56.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.759+0000 7f9c2c5cb640 1 -- 172.21.15.60:0/1873274860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c24074770 con 0x7f9c24144e40 2024-04-25T10:06:56.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.759+0000 7f9c1affd640 1 -- 172.21.15.60:0/1873274860 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9c1c0bf070 con 0x7f9c24144e40 2024-04-25T10:06:56.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.759+0000 7f9c1affd640 1 --2- 172.21.15.60:0/1873274860 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9c08066f60 0x7f9c08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.759+0000 7f9c1affd640 1 -- 172.21.15.60:0/1873274860 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9c1c11ff90 con 0x7f9c24144e40 2024-04-25T10:06:56.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.759+0000 7f9c2a367640 1 --2- 172.21.15.60:0/1873274860 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9c08066f60 0x7f9c08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.759+0000 7f9c2a367640 1 --2- 172.21.15.60:0/1873274860 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9c08066f60 0x7f9c08069420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9c240cb360 tx=0x7f9c20009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.763+0000 7f9c1affd640 1 -- 172.21.15.60:0/1873274860 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9c1c0c6070 con 0x7f9c24144e40 2024-04-25T10:06:56.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:56.919+0000 7f9c2c5cb640 1 -- 172.21.15.60:0/1873274860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9c2406acb0 con 0x7f9c24144e40 2024-04-25T10:06:57.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.559+0000 7f9c1affd640 1 -- 172.21.15.60:0/1873274860 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9c1c0165e0 con 0x7f9c24144e40 2024-04-25T10:06:57.568 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:06:57.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.567+0000 7f9c2c5cb640 1 -- 172.21.15.60:0/1873274860 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9c08066f60 msgr2=0x7f9c08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.567+0000 7f9c2c5cb640 1 --2- 172.21.15.60:0/1873274860 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9c08066f60 0x7f9c08069420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9c240cb360 tx=0x7f9c20009040 comp rx=0 tx=0).stop 2024-04-25T10:06:57.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.567+0000 7f9c2c5cb640 1 -- 172.21.15.60:0/1873274860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c24144e40 msgr2=0x7f9c2415a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.567+0000 7f9c2c5cb640 1 --2- 172.21.15.60:0/1873274860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c24144e40 0x7f9c2415a7a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c019600 tx=0x7f9c240cf700 comp rx=0 tx=0).stop 2024-04-25T10:06:57.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.567+0000 7f9c2c5cb640 1 -- 172.21.15.60:0/1873274860 shutdown_connections 2024-04-25T10:06:57.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.567+0000 7f9c2c5cb640 1 --2- 172.21.15.60:0/1873274860 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9c08066f60 0x7f9c08069420 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.567+0000 7f9c2c5cb640 1 --2- 172.21.15.60:0/1873274860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c24144e40 0x7f9c2415a7a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.567+0000 7f9c2c5cb640 1 --2- 172.21.15.60:0/1873274860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c240cf0a0 0x7f9c2406e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.567+0000 7f9c2c5cb640 1 --2- 172.21.15.60:0/1873274860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c24071980 0x7f9c240ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.567+0000 7f9c2c5cb640 1 -- 172.21.15.60:0/1873274860 >> 172.21.15.60:0/1873274860 conn(0x7f9c24066fa0 msgr2=0x7f9c240cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:57.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.567+0000 7f9c2c5cb640 1 -- 172.21.15.60:0/1873274860 shutdown_connections 2024-04-25T10:06:57.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.567+0000 7f9c2c5cb640 1 -- 172.21.15.60:0/1873274860 wait complete. 2024-04-25T10:06:57.595 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:06:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.719+0000 7f3a3be25640 1 Processor -- start 2024-04-25T10:06:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.719+0000 7f3a3be25640 1 -- start start 2024-04-25T10:06:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a3be25640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3a3406f140 0x7f3a34071270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a3be25640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3a3406d2d0 0x7f3a340717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a3be25640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a340cd2f0 con 0x7f3a34159f90 2024-04-25T10:06:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a3be25640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a3415a370 con 0x7f3a3406f140 2024-04-25T10:06:57.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a3be25640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a3406d7d0 con 0x7f3a3406d2d0 2024-04-25T10:06:57.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a39bc1640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3a34159f90 0x7f3a340ccbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48576/0 (socket says 172.21.15.60:48576) 2024-04-25T10:06:57.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a393c0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3a3406f140 0x7f3a34071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a39bc1640 1 -- 172.21.15.60:0/449229374 learned_addr learned my addr 172.21.15.60:0/449229374 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:57.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a3a3c2640 1 --2- 172.21.15.60:0/449229374 >> v2:172.21.15.100:3300/0 conn(0x7f3a3406d2d0 0x7f3a340717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a393c0640 1 -- 172.21.15.60:0/449229374 >> v2:172.21.15.100:3300/0 conn(0x7f3a3406d2d0 msgr2=0x7f3a340717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:57.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a393c0640 1 --2- 172.21.15.60:0/449229374 >> v2:172.21.15.100:3300/0 conn(0x7f3a3406d2d0 0x7f3a340717b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a393c0640 1 -- 172.21.15.60:0/449229374 >> v1:172.21.15.60:6789/0 conn(0x7f3a34159f90 legacy=0x7f3a340ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a393c0640 1 -- 172.21.15.60:0/449229374 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3406dab0 con 0x7f3a3406f140 2024-04-25T10:06:57.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a393c0640 1 --2- 172.21.15.60:0/449229374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a3406f140 0x7f3a34071270 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3a3406f6b0 tx=0x7f3a2401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc21075f88557264 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a38bbf640 1 -- 172.21.15.60:0/449229374 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a240b0070 con 0x7f3a3406f140 2024-04-25T10:06:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a38bbf640 1 -- 172.21.15.60:0/449229374 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a240ab600 con 0x7f3a3406f140 2024-04-25T10:06:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.723+0000 7f3a38bbf640 1 -- 172.21.15.60:0/449229374 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a240ab9d0 con 0x7f3a3406f140 2024-04-25T10:06:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 -- 172.21.15.60:0/449229374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a3406f140 msgr2=0x7f3a34071270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 --2- 172.21.15.60:0/449229374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a3406f140 0x7f3a34071270 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3a3406f6b0 tx=0x7f3a2401c480 comp rx=0 tx=0).stop 2024-04-25T10:06:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 -- 172.21.15.60:0/449229374 shutdown_connections 2024-04-25T10:06:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 --2- 172.21.15.60:0/449229374 >> v2:172.21.15.100:3300/0 conn(0x7f3a3406d2d0 0x7f3a340717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 --2- 172.21.15.60:0/449229374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a3406f140 0x7f3a34071270 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 -- 172.21.15.60:0/449229374 >> 172.21.15.60:0/449229374 conn(0x7f3a34066fa0 msgr2=0x7f3a3406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 -- 172.21.15.60:0/449229374 shutdown_connections 2024-04-25T10:06:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 -- 172.21.15.60:0/449229374 wait complete. 2024-04-25T10:06:57.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 Processor -- start 2024-04-25T10:06:57.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 -- start start 2024-04-25T10:06:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a3406d2d0 0x7f3a34142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a3406f140 0x7f3a340a8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a34159f90 0x7f3a340a89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a340cf520 con 0x7f3a3406d2d0 2024-04-25T10:06:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a340b34c0 con 0x7f3a34159f90 2024-04-25T10:06:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a340cf6a0 con 0x7f3a3406f140 2024-04-25T10:06:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a39bc1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a3406d2d0 0x7f3a34142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a39bc1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a3406d2d0 0x7f3a34142670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39608/0 (socket says 172.21.15.60:39608) 2024-04-25T10:06:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a39bc1640 1 -- 172.21.15.60:0/220581945 learned_addr learned my addr 172.21.15.60:0/220581945 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3a3c2640 1 --2- 172.21.15.60:0/220581945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a34159f90 0x7f3a340a89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a393c0640 1 --2- 172.21.15.60:0/220581945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a3406f140 0x7f3a340a8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a39bc1640 1 -- 172.21.15.60:0/220581945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a3406f140 msgr2=0x7f3a340a8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a39bc1640 1 --2- 172.21.15.60:0/220581945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a3406f140 0x7f3a340a8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a39bc1640 1 -- 172.21.15.60:0/220581945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a34159f90 msgr2=0x7f3a340a89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a39bc1640 1 --2- 172.21.15.60:0/220581945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a34159f90 0x7f3a340a89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a39bc1640 1 -- 172.21.15.60:0/220581945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a34143240 con 0x7f3a3406d2d0 2024-04-25T10:06:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a39bc1640 1 --2- 172.21.15.60:0/220581945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a3406d2d0 0x7f3a34142670 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000c370 tx=0x7f3a3000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a2a7fc640 1 -- 172.21.15.60:0/220581945 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a30017070 con 0x7f3a3406d2d0 2024-04-25T10:06:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a2a7fc640 1 -- 172.21.15.60:0/220581945 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a30005cf0 con 0x7f3a3406d2d0 2024-04-25T10:06:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 -- 172.21.15.60:0/220581945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a34144a10 con 0x7f3a3406d2d0 2024-04-25T10:06:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a2a7fc640 1 -- 172.21.15.60:0/220581945 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300072f0 con 0x7f3a3406d2d0 2024-04-25T10:06:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.727+0000 7f3a3be25640 1 -- 172.21.15.60:0/220581945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a341464a0 con 0x7f3a3406d2d0 2024-04-25T10:06:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.731+0000 7f3a3be25640 1 -- 172.21.15.60:0/220581945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a34074770 con 0x7f3a3406d2d0 2024-04-25T10:06:57.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.731+0000 7f3a2a7fc640 1 -- 172.21.15.60:0/220581945 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a3001a070 con 0x7f3a3406d2d0 2024-04-25T10:06:57.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.731+0000 7f3a2a7fc640 1 --2- 172.21.15.60:0/220581945 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a18066f90 0x7f3a18069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.731+0000 7f3a2a7fc640 1 -- 172.21.15.60:0/220581945 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f3a30084190 con 0x7f3a3406d2d0 2024-04-25T10:06:57.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.731+0000 7f3a393c0640 1 --2- 172.21.15.60:0/220581945 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a18066f90 0x7f3a18069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.731+0000 7f3a393c0640 1 --2- 172.21.15.60:0/220581945 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a18066f90 0x7f3a18069450 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3a240ad000 tx=0x7f3a24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:57.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.735+0000 7f3a2a7fc640 1 -- 172.21.15.60:0/220581945 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a34143240 con 0x7f3a3406d2d0 2024-04-25T10:06:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:57.899+0000 7f3a3be25640 1 -- 172.21.15.60:0/220581945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3a3406c250 con 0x7f3a3406d2d0 2024-04-25T10:06:58.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.567+0000 7f3a2a7fc640 1 -- 172.21.15.60:0/220581945 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3a340cd2f0 con 0x7f3a3406d2d0 2024-04-25T10:06:58.576 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T10:06:58.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.571+0000 7f3a3be25640 1 -- 172.21.15.60:0/220581945 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a18066f90 msgr2=0x7f3a18069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.571+0000 7f3a3be25640 1 --2- 172.21.15.60:0/220581945 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a18066f90 0x7f3a18069450 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3a240ad000 tx=0x7f3a24008040 comp rx=0 tx=0).stop 2024-04-25T10:06:58.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.571+0000 7f3a3be25640 1 -- 172.21.15.60:0/220581945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a3406d2d0 msgr2=0x7f3a34142670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.571+0000 7f3a3be25640 1 --2- 172.21.15.60:0/220581945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a3406d2d0 0x7f3a34142670 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000c370 tx=0x7f3a3000c830 comp rx=0 tx=0).stop 2024-04-25T10:06:58.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.575+0000 7f3a3be25640 1 -- 172.21.15.60:0/220581945 shutdown_connections 2024-04-25T10:06:58.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.575+0000 7f3a3be25640 1 --2- 172.21.15.60:0/220581945 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a18066f90 0x7f3a18069450 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.575+0000 7f3a3be25640 1 --2- 172.21.15.60:0/220581945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a34159f90 0x7f3a340a89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.575+0000 7f3a3be25640 1 --2- 172.21.15.60:0/220581945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a3406f140 0x7f3a340a8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.575+0000 7f3a3be25640 1 --2- 172.21.15.60:0/220581945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a3406d2d0 0x7f3a34142670 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.575+0000 7f3a3be25640 1 -- 172.21.15.60:0/220581945 >> 172.21.15.60:0/220581945 conn(0x7f3a34066fa0 msgr2=0x7f3a34071170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.575+0000 7f3a3be25640 1 -- 172.21.15.60:0/220581945 shutdown_connections 2024-04-25T10:06:58.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.575+0000 7f3a3be25640 1 -- 172.21.15.60:0/220581945 wait complete. 2024-04-25T10:06:58.606 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:06:58.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd286d41640 1 Processor -- start 2024-04-25T10:06:58.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd286d41640 1 -- start start 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd286d41640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd2801620d0 0x7fd2800ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd286d41640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd28015bad0 con 0x7fd28015b6f0 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd286d41640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd28015ebe0 con 0x7fd2800ced20 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd286d41640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd280161c40 con 0x7fd2801620d0 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd284add640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd28015b6f0 0x7fd28015e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48590/0 (socket says 172.21.15.60:48590) 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd284add640 1 -- 172.21.15.60:0/1289770202 learned_addr learned my addr 172.21.15.60:0/1289770202 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:58.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd27ffff640 1 --2- 172.21.15.60:0/1289770202 >> v2:172.21.15.100:3300/0 conn(0x7fd2801620d0 0x7fd2800ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd27f7fe640 1 -- 172.21.15.60:0/1289770202 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685338180 0 0) 0x7fd28015bad0 con 0x7fd28015b6f0 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd27f7fe640 1 -- 172.21.15.60:0/1289770202 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd268003530 con 0x7fd28015b6f0 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd27f7fe640 1 -- 172.21.15.60:0/1289770202 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2155443069 0 0) 0x7fd268003530 con 0x7fd28015b6f0 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd27f7fe640 1 -- 172.21.15.60:0/1289770202 >> v2:172.21.15.100:3300/0 conn(0x7fd2801620d0 msgr2=0x7fd2800ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd27f7fe640 1 --2- 172.21.15.60:0/1289770202 >> v2:172.21.15.100:3300/0 conn(0x7fd2801620d0 0x7fd2800ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd27f7fe640 1 -- 172.21.15.60:0/1289770202 >> v1:172.21.15.100:6789/0 conn(0x7fd2800ced20 legacy=0x7fd2800d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd27f7fe640 1 -- 172.21.15.60:0/1289770202 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28015eec0 con 0x7fd28015b6f0 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd27f7fe640 1 -- 172.21.15.60:0/1289770202 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fd278001a00 con 0x7fd28015b6f0 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.731+0000 7fd27f7fe640 1 -- 172.21.15.60:0/1289770202 >> v1:172.21.15.60:6789/0 conn(0x7fd28015b6f0 legacy=0x7fd28015e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd27f7fe640 1 --2- 172.21.15.60:0/1289770202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2680039c0 0x7fd268003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd27f7fe640 1 --2- 172.21.15.60:0/1289770202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd268004810 0x7fd268004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd27f7fe640 1 --2- 172.21.15.60:0/1289770202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2680051d0 0x7fd26800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd27f7fe640 1 -- 172.21.15.60:0/1289770202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd280161c40 con 0x7fd2680051d0 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd27f7fe640 1 -- 172.21.15.60:0/1289770202 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26800c070 con 0x7fd2680039c0 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd27f7fe640 1 -- 172.21.15.60:0/1289770202 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26800c270 con 0x7fd268004810 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd2852de640 1 --2- 172.21.15.60:0/1289770202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2680051d0 0x7fd26800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd284add640 1 --2- 172.21.15.60:0/1289770202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2680039c0 0x7fd268003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd27ffff640 1 --2- 172.21.15.60:0/1289770202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd268004810 0x7fd268004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd2852de640 1 -- 172.21.15.60:0/1289770202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd268004810 msgr2=0x7fd268004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd2852de640 1 --2- 172.21.15.60:0/1289770202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd268004810 0x7fd268004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd2852de640 1 -- 172.21.15.60:0/1289770202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2680039c0 msgr2=0x7fd268003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd2852de640 1 --2- 172.21.15.60:0/1289770202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2680039c0 0x7fd268003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd2852de640 1 -- 172.21.15.60:0/1289770202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd268003530 con 0x7fd2680051d0 2024-04-25T10:06:58.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd2852de640 1 -- 172.21.15.60:0/1289770202 reap_dead start 2024-04-25T10:06:58.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd2852de640 1 --2- 172.21.15.60:0/1289770202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2680051d0 0x7fd26800b9d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd270001780 tx=0x7fd27001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e291865675d2d30b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd27f7fe640 1 -- 172.21.15.60:0/1289770202 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2700013a0 con 0x7fd2680051d0 2024-04-25T10:06:58.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd27f7fe640 1 -- 172.21.15.60:0/1289770202 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2700a9070 con 0x7fd2680051d0 2024-04-25T10:06:58.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd27f7fe640 1 -- 172.21.15.60:0/1289770202 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd270003600 con 0x7fd2680051d0 2024-04-25T10:06:58.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd286d41640 1 -- 172.21.15.60:0/1289770202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2680051d0 msgr2=0x7fd26800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd286d41640 1 --2- 172.21.15.60:0/1289770202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2680051d0 0x7fd26800b9d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd270001780 tx=0x7fd27001c420 comp rx=0 tx=0).stop 2024-04-25T10:06:58.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd286d41640 1 -- 172.21.15.60:0/1289770202 shutdown_connections 2024-04-25T10:06:58.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd286d41640 1 --2- 172.21.15.60:0/1289770202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2680051d0 0x7fd26800b9d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd286d41640 1 -- 172.21.15.60:0/1289770202 >> 172.21.15.60:0/1289770202 conn(0x7fd280072340 msgr2=0x7fd280072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd286d41640 1 -- 172.21.15.60:0/1289770202 shutdown_connections 2024-04-25T10:06:58.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.735+0000 7fd286d41640 1 -- 172.21.15.60:0/1289770202 wait complete. 2024-04-25T10:06:58.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd286d41640 1 Processor -- start 2024-04-25T10:06:58.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd286d41640 1 -- start start 2024-04-25T10:06:58.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd286d41640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2680051d0 0x7fd280154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd286d41640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2800d2f60 0x7fd2800d33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd286d41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd28014fe20 0x7fd2801502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd286d41640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd280162510 con 0x7fd2680051d0 2024-04-25T10:06:58.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd286d41640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2800cf150 con 0x7fd28014fe20 2024-04-25T10:06:58.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd286d41640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2800b9550 con 0x7fd2800d2f60 2024-04-25T10:06:58.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd284add640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2680051d0 0x7fd280154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd284add640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2680051d0 0x7fd280154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39626/0 (socket says 172.21.15.60:39626) 2024-04-25T10:06:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd284add640 1 -- 172.21.15.60:0/1038759035 learned_addr learned my addr 172.21.15.60:0/1038759035 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd27ffff640 1 --2- 172.21.15.60:0/1038759035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2800d2f60 0x7fd2800d33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd2852de640 1 --2- 172.21.15.60:0/1038759035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd28014fe20 0x7fd2801502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd284add640 1 -- 172.21.15.60:0/1038759035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2800d2f60 msgr2=0x7fd2800d33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd284add640 1 --2- 172.21.15.60:0/1038759035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2800d2f60 0x7fd2800d33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd284add640 1 -- 172.21.15.60:0/1038759035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd28014fe20 msgr2=0x7fd2801502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd284add640 1 --2- 172.21.15.60:0/1038759035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd28014fe20 0x7fd2801502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd284add640 1 -- 172.21.15.60:0/1038759035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2800d1c70 con 0x7fd2680051d0 2024-04-25T10:06:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd27ffff640 1 --2- 172.21.15.60:0/1038759035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2800d2f60 0x7fd2800d33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd284add640 1 --2- 172.21.15.60:0/1038759035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2680051d0 0x7fd280154990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd278007960 tx=0x7fd27800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd27d7fa640 1 -- 172.21.15.60:0/1038759035 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27800ee50 con 0x7fd2680051d0 2024-04-25T10:06:58.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd27d7fa640 1 -- 172.21.15.60:0/1038759035 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd278018070 con 0x7fd2680051d0 2024-04-25T10:06:58.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd286d41640 1 -- 172.21.15.60:0/1038759035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2800d1f50 con 0x7fd2680051d0 2024-04-25T10:06:58.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd27d7fa640 1 -- 172.21.15.60:0/1038759035 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278013660 con 0x7fd2680051d0 2024-04-25T10:06:58.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd286d41640 1 -- 172.21.15.60:0/1038759035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2800d2510 con 0x7fd2680051d0 2024-04-25T10:06:58.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.739+0000 7fd286d41640 1 -- 172.21.15.60:0/1038759035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd280074770 con 0x7fd2680051d0 2024-04-25T10:06:58.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.743+0000 7fd27d7fa640 1 -- 172.21.15.60:0/1038759035 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd278013800 con 0x7fd2680051d0 2024-04-25T10:06:58.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.743+0000 7fd27d7fa640 1 --2- 172.21.15.60:0/1038759035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd26c067130 0x7fd26c0695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.743+0000 7fd27d7fa640 1 -- 172.21.15.60:0/1038759035 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd278085280 con 0x7fd2680051d0 2024-04-25T10:06:58.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.743+0000 7fd27ffff640 1 --2- 172.21.15.60:0/1038759035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd26c067130 0x7fd26c0695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.743+0000 7fd27ffff640 1 --2- 172.21.15.60:0/1038759035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd26c067130 0x7fd26c0695f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd28015e6c0 tx=0x7fd274008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.747+0000 7fd27d7fa640 1 -- 172.21.15.60:0/1038759035 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd278056620 con 0x7fd2680051d0 2024-04-25T10:06:58.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.911+0000 7fd286d41640 1 -- 172.21.15.60:0/1038759035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd2800d2950 con 0x7fd2680051d0 2024-04-25T10:06:58.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.915+0000 7fd27d7fa640 1 -- 172.21.15.60:0/1038759035 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd278055f20 con 0x7fd2680051d0 2024-04-25T10:06:58.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.919+0000 7fd286d41640 1 -- 172.21.15.60:0/1038759035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd26c067130 msgr2=0x7fd26c0695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.919+0000 7fd286d41640 1 --2- 172.21.15.60:0/1038759035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd26c067130 0x7fd26c0695f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd28015e6c0 tx=0x7fd274008040 comp rx=0 tx=0).stop 2024-04-25T10:06:58.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.919+0000 7fd286d41640 1 -- 172.21.15.60:0/1038759035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2680051d0 msgr2=0x7fd280154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.919+0000 7fd286d41640 1 --2- 172.21.15.60:0/1038759035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2680051d0 0x7fd280154990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd278007960 tx=0x7fd27800e530 comp rx=0 tx=0).stop 2024-04-25T10:06:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.919+0000 7fd286d41640 1 -- 172.21.15.60:0/1038759035 shutdown_connections 2024-04-25T10:06:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.919+0000 7fd286d41640 1 --2- 172.21.15.60:0/1038759035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd26c067130 0x7fd26c0695f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.919+0000 7fd286d41640 1 --2- 172.21.15.60:0/1038759035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd28014fe20 0x7fd2801502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.919+0000 7fd286d41640 1 --2- 172.21.15.60:0/1038759035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2800d2f60 0x7fd2800d33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.919+0000 7fd286d41640 1 --2- 172.21.15.60:0/1038759035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2680051d0 0x7fd280154990 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.919+0000 7fd286d41640 1 -- 172.21.15.60:0/1038759035 >> 172.21.15.60:0/1038759035 conn(0x7fd280072340 msgr2=0x7fd28015fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.919+0000 7fd286d41640 1 -- 172.21.15.60:0/1038759035 shutdown_connections 2024-04-25T10:06:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:58.919+0000 7fd286d41640 1 -- 172.21.15.60:0/1038759035 wait complete. 2024-04-25T10:06:58.943 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-25T10:06:59.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.059+0000 7fabf31c8640 1 Processor -- start 2024-04-25T10:06:59.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.059+0000 7fabf31c8640 1 -- start start 2024-04-25T10:06:59.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.059+0000 7fabf31c8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabec15dc90 0x7fabec0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.059+0000 7fabf31c8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fabec0719a0 0x7fabec0ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.059+0000 7fabf31c8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabec071fd0 con 0x7fabec0b2b10 2024-04-25T10:06:59.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.059+0000 7fabf31c8640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabec0accc0 con 0x7fabec0719a0 2024-04-25T10:06:59.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.059+0000 7fabf31c8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0ad260 con 0x7fabec15dc90 2024-04-25T10:06:59.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.059+0000 7fabf1765640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fabec0719a0 0x7fabec0ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.059+0000 7fabf1765640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fabec0719a0 0x7fabec0ad6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55660/0 (socket says 172.21.15.60:55660) 2024-04-25T10:06:59.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.059+0000 7fabf1765640 1 -- 172.21.15.60:0/4239659495 learned_addr learned my addr 172.21.15.60:0/4239659495 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:59.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.059+0000 7fabf0f64640 1 --2- 172.21.15.60:0/4239659495 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabec15dc90 0x7fabec0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.059+0000 7fabf1765640 1 -- 172.21.15.60:0/4239659495 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabec15dc90 msgr2=0x7fabec0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.059+0000 7fabf1765640 1 --2- 172.21.15.60:0/4239659495 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabec15dc90 0x7fabec0b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.059+0000 7fabf1765640 1 -- 172.21.15.60:0/4239659495 >> v1:172.21.15.100:6789/0 conn(0x7fabec0b2b10 legacy=0x7fabec071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.059+0000 7fabf1765640 1 -- 172.21.15.60:0/4239659495 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec072290 con 0x7fabec0719a0 2024-04-25T10:06:59.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.063+0000 7fabf1765640 1 --2- 172.21.15.60:0/4239659495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabec0719a0 0x7fabec0ad6f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009870 tx=0x7fabdc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbeba59de80b9c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.063+0000 7fabeb7fe640 1 -- 172.21.15.60:0/4239659495 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc0a7070 con 0x7fabec0719a0 2024-04-25T10:06:59.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.063+0000 7fabeb7fe640 1 -- 172.21.15.60:0/4239659495 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabdc0b0070 con 0x7fabec0719a0 2024-04-25T10:06:59.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.063+0000 7fabeb7fe640 1 -- 172.21.15.60:0/4239659495 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc0ab890 con 0x7fabec0719a0 2024-04-25T10:06:59.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.063+0000 7fabf31c8640 1 -- 172.21.15.60:0/4239659495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabec0719a0 msgr2=0x7fabec0ad6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.063+0000 7fabf31c8640 1 --2- 172.21.15.60:0/4239659495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabec0719a0 0x7fabec0ad6f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009870 tx=0x7fabdc01c560 comp rx=0 tx=0).stop 2024-04-25T10:06:59.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.063+0000 7fabf31c8640 1 -- 172.21.15.60:0/4239659495 shutdown_connections 2024-04-25T10:06:59.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.063+0000 7fabf31c8640 1 --2- 172.21.15.60:0/4239659495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabec0719a0 0x7fabec0ad6f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.063+0000 7fabf31c8640 1 --2- 172.21.15.60:0/4239659495 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabec15dc90 0x7fabec0b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.063+0000 7fabf31c8640 1 -- 172.21.15.60:0/4239659495 >> 172.21.15.60:0/4239659495 conn(0x7fabec0b12d0 msgr2=0x7fabec0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.063+0000 7fabf31c8640 1 -- 172.21.15.60:0/4239659495 shutdown_connections 2024-04-25T10:06:59.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.063+0000 7fabf31c8640 1 -- 172.21.15.60:0/4239659495 wait complete. 2024-04-25T10:06:59.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.063+0000 7fabf31c8640 1 Processor -- start 2024-04-25T10:06:59.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.063+0000 7fabf31c8640 1 -- start start 2024-04-25T10:06:59.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf31c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabec0719a0 0x7fabec158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf31c8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabec15dc90 0x7fabec14d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf31c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabec155680 0x7fabec158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf31c8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0adfc0 con 0x7fabec0719a0 2024-04-25T10:06:59.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf31c8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0ade40 con 0x7fabec155680 2024-04-25T10:06:59.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf31c8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0b2220 con 0x7fabec15dc90 2024-04-25T10:06:59.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf0f64640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabec0719a0 0x7fabec158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf0f64640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabec0719a0 0x7fabec158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55666/0 (socket says 172.21.15.60:55666) 2024-04-25T10:06:59.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf0f64640 1 -- 172.21.15.60:0/821685293 learned_addr learned my addr 172.21.15.60:0/821685293 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:59.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabebfff640 1 --2- 172.21.15.60:0/821685293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabec15dc90 0x7fabec14d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf1765640 1 --2- 172.21.15.60:0/821685293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabec155680 0x7fabec158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf0f64640 1 -- 172.21.15.60:0/821685293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabec15dc90 msgr2=0x7fabec14d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf0f64640 1 --2- 172.21.15.60:0/821685293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabec15dc90 0x7fabec14d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf0f64640 1 -- 172.21.15.60:0/821685293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabec155680 msgr2=0x7fabec158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf0f64640 1 --2- 172.21.15.60:0/821685293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabec155680 0x7fabec158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf0f64640 1 -- 172.21.15.60:0/821685293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec1590d0 con 0x7fabec0719a0 2024-04-25T10:06:59.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabebfff640 1 --2- 172.21.15.60:0/821685293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabec15dc90 0x7fabec14d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:59.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf0f64640 1 --2- 172.21.15.60:0/821685293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabec0719a0 0x7fabec158320 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fabe400efc0 tx=0x7fabe400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabe97fa640 1 -- 172.21.15.60:0/821685293 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe4018020 con 0x7fabec0719a0 2024-04-25T10:06:59.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabe97fa640 1 -- 172.21.15.60:0/821685293 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe4014480 con 0x7fabec0719a0 2024-04-25T10:06:59.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf31c8640 1 -- 172.21.15.60:0/821685293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabec157840 con 0x7fabec0719a0 2024-04-25T10:06:59.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabe97fa640 1 -- 172.21.15.60:0/821685293 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe4014850 con 0x7fabec0719a0 2024-04-25T10:06:59.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf31c8640 1 -- 172.21.15.60:0/821685293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabec157e00 con 0x7fabec0719a0 2024-04-25T10:06:59.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabf31c8640 1 -- 172.21.15.60:0/821685293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabec074770 con 0x7fabec0719a0 2024-04-25T10:06:59.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabe97fa640 1 -- 172.21.15.60:0/821685293 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fabe401b020 con 0x7fabec0719a0 2024-04-25T10:06:59.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabe97fa640 1 --2- 172.21.15.60:0/821685293 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabcc066f60 0x7fabcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabe97fa640 1 -- 172.21.15.60:0/821685293 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fabe4085b90 con 0x7fabec0719a0 2024-04-25T10:06:59.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.067+0000 7fabebfff640 1 --2- 172.21.15.60:0/821685293 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabcc066f60 0x7fabcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.071+0000 7fabebfff640 1 --2- 172.21.15.60:0/821685293 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabcc066f60 0x7fabcc069420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fabec15c4f0 tx=0x7fabd4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.071+0000 7fabe97fa640 1 -- 172.21.15.60:0/821685293 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fabe4022020 con 0x7fabec0719a0 2024-04-25T10:06:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.295+0000 7fabf31c8640 1 -- 172.21.15.60:0/821685293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fabec14f590 con 0x7fabec0719a0 2024-04-25T10:06:59.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.571+0000 7fabe97fa640 1 -- 172.21.15.60:0/821685293 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v110) v1 ==== 142+0+0 (secure 0 0 0) 0x7fabe4010040 con 0x7fabec0719a0 2024-04-25T10:06:59.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.575+0000 7fabf31c8640 1 -- 172.21.15.60:0/821685293 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabcc066f60 msgr2=0x7fabcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.575+0000 7fabf31c8640 1 --2- 172.21.15.60:0/821685293 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabcc066f60 0x7fabcc069420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fabec15c4f0 tx=0x7fabd4009040 comp rx=0 tx=0).stop 2024-04-25T10:06:59.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.575+0000 7fabf31c8640 1 -- 172.21.15.60:0/821685293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabec0719a0 msgr2=0x7fabec158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.575+0000 7fabf31c8640 1 --2- 172.21.15.60:0/821685293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabec0719a0 0x7fabec158320 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fabe400efc0 tx=0x7fabe400c490 comp rx=0 tx=0).stop 2024-04-25T10:06:59.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.575+0000 7fabf31c8640 1 -- 172.21.15.60:0/821685293 shutdown_connections 2024-04-25T10:06:59.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.575+0000 7fabf31c8640 1 --2- 172.21.15.60:0/821685293 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabcc066f60 0x7fabcc069420 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.575+0000 7fabf31c8640 1 --2- 172.21.15.60:0/821685293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabec155680 0x7fabec158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.575+0000 7fabf31c8640 1 --2- 172.21.15.60:0/821685293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabec15dc90 0x7fabec14d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.575+0000 7fabf31c8640 1 --2- 172.21.15.60:0/821685293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabec0719a0 0x7fabec158320 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.575+0000 7fabf31c8640 1 -- 172.21.15.60:0/821685293 >> 172.21.15.60:0/821685293 conn(0x7fabec0b12d0 msgr2=0x7fabec0b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.575+0000 7fabf31c8640 1 -- 172.21.15.60:0/821685293 shutdown_connections 2024-04-25T10:06:59.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.575+0000 7fabf31c8640 1 -- 172.21.15.60:0/821685293 wait complete. 2024-04-25T10:06:59.603 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:06:59.603 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-04-25T10:06:59.603 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:06:59.604 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:59.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.727+0000 7fadfdc29640 1 Processor -- start 2024-04-25T10:06:59.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.727+0000 7fadfdc29640 1 -- start start 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.727+0000 7fadfdc29640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fadf80b6c10 0x7fadf80ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.727+0000 7fadfdc29640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf806d200 con 0x7fadf80d0a50 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.727+0000 7fadfdc29640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf80cefe0 con 0x7fadf815deb0 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.727+0000 7fadfdc29640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf80cf2c0 con 0x7fadf80b6c10 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.727+0000 7fadf7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fadf80b6c10 0x7fadf80ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.727+0000 7fadf6ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fadf80d0a50 0x7fadf80ce2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46080/0 (socket says 172.21.15.60:46080) 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.727+0000 7fadf6ffd640 1 -- 172.21.15.60:0/2002209080 learned_addr learned my addr 172.21.15.60:0/2002209080 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.727+0000 7fadf7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fadf80b6c10 0x7fadf80ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55672/0 (socket says 172.21.15.60:55672) 2024-04-25T10:06:59.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.727+0000 7fadf7fff640 1 -- 172.21.15.60:0/2002209080 >> v1:172.21.15.100:6789/0 conn(0x7fadf815deb0 legacy=0x7fadf80d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.727+0000 7fadf7fff640 1 -- 172.21.15.60:0/2002209080 >> v1:172.21.15.60:6789/0 conn(0x7fadf80d0a50 legacy=0x7fadf80ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.727+0000 7fadf67fc640 1 -- 172.21.15.60:0/2002209080 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009601454 0 0) 0x7fadf806d200 con 0x7fadf80d0a50 2024-04-25T10:06:59.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.727+0000 7fadf7fff640 1 -- 172.21.15.60:0/2002209080 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf80d1d70 con 0x7fadf80b6c10 2024-04-25T10:06:59.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf7fff640 1 --2- 172.21.15.60:0/2002209080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf80b6c10 0x7fadf80ce9c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009d90 tx=0x7fadf001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a5d416e97a2b2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf67fc640 1 -- 172.21.15.60:0/2002209080 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf00b2070 con 0x7fadf80b6c10 2024-04-25T10:06:59.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf67fc640 1 -- 172.21.15.60:0/2002209080 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf00a58e0 con 0x7fadf80b6c10 2024-04-25T10:06:59.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf67fc640 1 -- 172.21.15.60:0/2002209080 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf00a5cb0 con 0x7fadf80b6c10 2024-04-25T10:06:59.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadfdc29640 1 -- 172.21.15.60:0/2002209080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf80b6c10 msgr2=0x7fadf80ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadfdc29640 1 --2- 172.21.15.60:0/2002209080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf80b6c10 0x7fadf80ce9c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009d90 tx=0x7fadf001c930 comp rx=0 tx=0).stop 2024-04-25T10:06:59.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadfdc29640 1 -- 172.21.15.60:0/2002209080 shutdown_connections 2024-04-25T10:06:59.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadfdc29640 1 --2- 172.21.15.60:0/2002209080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf80b6c10 0x7fadf80ce9c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadfdc29640 1 -- 172.21.15.60:0/2002209080 >> 172.21.15.60:0/2002209080 conn(0x7fadf806c090 msgr2=0x7fadf806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadfdc29640 1 -- 172.21.15.60:0/2002209080 shutdown_connections 2024-04-25T10:06:59.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadfdc29640 1 -- 172.21.15.60:0/2002209080 wait complete. 2024-04-25T10:06:59.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadfdc29640 1 Processor -- start 2024-04-25T10:06:59.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadfdc29640 1 -- start start 2024-04-25T10:06:59.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadfdc29640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf80b6c10 0x7fadf80c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadfdc29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf80d0a50 0x7fadf8158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadfdc29640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf815deb0 0x7fadf81594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadfdc29640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf816b7d0 con 0x7fadf815deb0 2024-04-25T10:06:59.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadfdc29640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf816b650 con 0x7fadf80d0a50 2024-04-25T10:06:59.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadfdc29640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf816b950 con 0x7fadf80b6c10 2024-04-25T10:06:59.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf815deb0 0x7fadf81594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf815deb0 0x7fadf81594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55680/0 (socket says 172.21.15.60:55680) 2024-04-25T10:06:59.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf7fff640 1 -- 172.21.15.60:0/894956402 learned_addr learned my addr 172.21.15.60:0/894956402 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:06:59.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf77fe640 1 --2- 172.21.15.60:0/894956402 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf80b6c10 0x7fadf80c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf6ffd640 1 --2- 172.21.15.60:0/894956402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf80d0a50 0x7fadf8158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf7fff640 1 -- 172.21.15.60:0/894956402 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf80b6c10 msgr2=0x7fadf80c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf7fff640 1 --2- 172.21.15.60:0/894956402 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf80b6c10 0x7fadf80c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf7fff640 1 -- 172.21.15.60:0/894956402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf80d0a50 msgr2=0x7fadf8158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf7fff640 1 --2- 172.21.15.60:0/894956402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf80d0a50 0x7fadf8158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf7fff640 1 -- 172.21.15.60:0/894956402 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf8149640 con 0x7fadf815deb0 2024-04-25T10:06:59.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf6ffd640 1 --2- 172.21.15.60:0/894956402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf80d0a50 0x7fadf8158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:59.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf77fe640 1 --2- 172.21.15.60:0/894956402 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf80b6c10 0x7fadf80c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:59.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.731+0000 7fadf7fff640 1 --2- 172.21.15.60:0/894956402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf815deb0 0x7fadf81594c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fadf00b0040 tx=0x7fadf00196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.735+0000 7faddffff640 1 -- 172.21.15.60:0/894956402 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf00b2050 con 0x7fadf815deb0 2024-04-25T10:06:59.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.735+0000 7faddffff640 1 -- 172.21.15.60:0/894956402 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf00be070 con 0x7fadf815deb0 2024-04-25T10:06:59.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.735+0000 7fadfdc29640 1 -- 172.21.15.60:0/894956402 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf80c6dd0 con 0x7fadf815deb0 2024-04-25T10:06:59.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.735+0000 7faddffff640 1 -- 172.21.15.60:0/894956402 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf00adb10 con 0x7fadf815deb0 2024-04-25T10:06:59.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.735+0000 7fadfdc29640 1 -- 172.21.15.60:0/894956402 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf815a400 con 0x7fadf815deb0 2024-04-25T10:06:59.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.735+0000 7fadfdc29640 1 -- 172.21.15.60:0/894956402 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf8074770 con 0x7fadf815deb0 2024-04-25T10:06:59.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.735+0000 7faddffff640 1 -- 172.21.15.60:0/894956402 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fadf00adcb0 con 0x7fadf815deb0 2024-04-25T10:06:59.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.735+0000 7faddffff640 1 --2- 172.21.15.60:0/894956402 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fade4066f60 0x7fade4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.735+0000 7faddffff640 1 -- 172.21.15.60:0/894956402 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fadf0120020 con 0x7fadf815deb0 2024-04-25T10:06:59.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.735+0000 7fadf77fe640 1 --2- 172.21.15.60:0/894956402 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fade4066f60 0x7fade4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.735+0000 7fadf77fe640 1 --2- 172.21.15.60:0/894956402 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fade4066f60 0x7fade4069420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fade80098b0 tx=0x7fade8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.739+0000 7faddffff640 1 -- 172.21.15.60:0/894956402 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fadf0016520 con 0x7fadf815deb0 2024-04-25T10:06:59.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.899+0000 7fadfdc29640 1 -- 172.21.15.60:0/894956402 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadf8149c80 con 0x7fadf815deb0 2024-04-25T10:06:59.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.899+0000 7faddffff640 1 -- 172.21.15.60:0/894956402 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+13032 (secure 0 0 0) 0x7fadf00ab340 con 0x7fadf815deb0 2024-04-25T10:06:59.908 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:06:59.908 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":38,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:06:58.563570+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/3681296171":"2024-04-26T10:06:55.517715+0000","172.21.15.60:6834/3681296171":"2024-04-26T10:06:55.517715+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:59.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.903+0000 7fadfdc29640 1 -- 172.21.15.60:0/894956402 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fade4066f60 msgr2=0x7fade4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.903+0000 7fadfdc29640 1 --2- 172.21.15.60:0/894956402 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fade4066f60 0x7fade4069420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fade80098b0 tx=0x7fade8008040 comp rx=0 tx=0).stop 2024-04-25T10:06:59.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.903+0000 7fadfdc29640 1 -- 172.21.15.60:0/894956402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf815deb0 msgr2=0x7fadf81594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.903+0000 7fadfdc29640 1 --2- 172.21.15.60:0/894956402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf815deb0 0x7fadf81594c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fadf00b0040 tx=0x7fadf00196d0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.903+0000 7fadfdc29640 1 -- 172.21.15.60:0/894956402 shutdown_connections 2024-04-25T10:06:59.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.903+0000 7fadfdc29640 1 --2- 172.21.15.60:0/894956402 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fade4066f60 0x7fade4069420 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.903+0000 7fadfdc29640 1 --2- 172.21.15.60:0/894956402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf815deb0 0x7fadf81594c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.903+0000 7fadfdc29640 1 --2- 172.21.15.60:0/894956402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf80d0a50 0x7fadf8158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.903+0000 7fadfdc29640 1 --2- 172.21.15.60:0/894956402 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf80b6c10 0x7fadf80c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.903+0000 7fadfdc29640 1 -- 172.21.15.60:0/894956402 >> 172.21.15.60:0/894956402 conn(0x7fadf806c090 msgr2=0x7fadf80b7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.903+0000 7fadfdc29640 1 -- 172.21.15.60:0/894956402 shutdown_connections 2024-04-25T10:06:59.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:06:59.903+0000 7fadfdc29640 1 -- 172.21.15.60:0/894956402 wait complete. 2024-04-25T10:06:59.928 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:59.928 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:00.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e96373640 1 Processor -- start 2024-04-25T10:07:00.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e96373640 1 -- start start 2024-04-25T10:07:00.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e96373640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e900b4e40 0x7f9e9015c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e96373640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9e900af0d0 0x7f9e9006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e96373640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e90160280 con 0x7f9e900acde0 2024-04-25T10:07:00.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e96373640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e900af550 con 0x7f9e900af0d0 2024-04-25T10:07:00.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e96373640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9015fd50 con 0x7f9e900b4e40 2024-04-25T10:07:00.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9e900acde0 0x7f9e900ad1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46094/0 (socket says 172.21.15.60:46094) 2024-04-25T10:07:00.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8ffff640 1 -- 172.21.15.60:0/767472600 learned_addr learned my addr 172.21.15.60:0/767472600 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:00.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8f7fe640 1 --2- 172.21.15.60:0/767472600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e900b4e40 0x7f9e9015c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8effd640 1 -- 172.21.15.60:0/767472600 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018938864 0 0) 0x7f9e90160280 con 0x7f9e900acde0 2024-04-25T10:07:00.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e94910640 1 --2- 172.21.15.60:0/767472600 >> v2:172.21.15.100:3300/0 conn(0x7f9e900af0d0 0x7f9e9006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8effd640 1 -- 172.21.15.60:0/767472600 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e78003530 con 0x7f9e900acde0 2024-04-25T10:07:00.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8effd640 1 -- 172.21.15.60:0/767472600 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1731515817 0 0) 0x7f9e78003530 con 0x7f9e900acde0 2024-04-25T10:07:00.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8effd640 1 -- 172.21.15.60:0/767472600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e900b4e40 msgr2=0x7f9e9015c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8effd640 1 --2- 172.21.15.60:0/767472600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e900b4e40 0x7f9e9015c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8effd640 1 -- 172.21.15.60:0/767472600 >> v2:172.21.15.100:3300/0 conn(0x7f9e900af0d0 msgr2=0x7f9e9006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8effd640 1 --2- 172.21.15.60:0/767472600 >> v2:172.21.15.100:3300/0 conn(0x7f9e900af0d0 0x7f9e9006ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8effd640 1 -- 172.21.15.60:0/767472600 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9015bc20 con 0x7f9e900acde0 2024-04-25T10:07:00.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8effd640 1 -- 172.21.15.60:0/767472600 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f9e80001680 con 0x7f9e900acde0 2024-04-25T10:07:00.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8effd640 1 -- 172.21.15.60:0/767472600 >> v1:172.21.15.60:6789/0 conn(0x7f9e900acde0 legacy=0x7f9e900ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8effd640 1 --2- 172.21.15.60:0/767472600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e900b4e40 0x7f9e780039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8effd640 1 --2- 172.21.15.60:0/767472600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e78003f40 0x7f9e78004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8effd640 1 --2- 172.21.15.60:0/767472600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e78004d50 0x7f9e780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8effd640 1 -- 172.21.15.60:0/767472600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9015fd50 con 0x7f9e78004d50 2024-04-25T10:07:00.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8effd640 1 -- 172.21.15.60:0/767472600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7800b9d0 con 0x7f9e78003f40 2024-04-25T10:07:00.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8effd640 1 -- 172.21.15.60:0/767472600 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7800bb50 con 0x7f9e900b4e40 2024-04-25T10:07:00.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e94910640 1 --2- 172.21.15.60:0/767472600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e78004d50 0x7f9e780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8ffff640 1 --2- 172.21.15.60:0/767472600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e900b4e40 0x7f9e780039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8f7fe640 1 --2- 172.21.15.60:0/767472600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e78003f40 0x7f9e78004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e94910640 1 -- 172.21.15.60:0/767472600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e900b4e40 msgr2=0x7f9e780039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e94910640 1 --2- 172.21.15.60:0/767472600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e900b4e40 0x7f9e780039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e94910640 1 -- 172.21.15.60:0/767472600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e78003f40 msgr2=0x7f9e78004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e94910640 1 --2- 172.21.15.60:0/767472600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e78003f40 0x7f9e78004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e94910640 1 -- 172.21.15.60:0/767472600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e78003530 con 0x7f9e78004d50 2024-04-25T10:07:00.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8ffff640 1 --2- 172.21.15.60:0/767472600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e900b4e40 0x7f9e780039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:00.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.067+0000 7f9e8f7fe640 1 --2- 172.21.15.60:0/767472600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e78003f40 0x7f9e78004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:00.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e94910640 1 --2- 172.21.15.60:0/767472600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e78004d50 0x7f9e780051d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009f90 tx=0x7f9e8801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65dd7d74c93a91e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e8effd640 1 -- 172.21.15.60:0/767472600 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e880b2070 con 0x7f9e78004d50 2024-04-25T10:07:00.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e8effd640 1 -- 172.21.15.60:0/767472600 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e880a5c90 con 0x7f9e78004d50 2024-04-25T10:07:00.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e8effd640 1 -- 172.21.15.60:0/767472600 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e88009c10 con 0x7f9e78004d50 2024-04-25T10:07:00.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e96373640 1 -- 172.21.15.60:0/767472600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e78004d50 msgr2=0x7f9e780051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e96373640 1 --2- 172.21.15.60:0/767472600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e78004d50 0x7f9e780051d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009f90 tx=0x7f9e8801cc20 comp rx=0 tx=0).stop 2024-04-25T10:07:00.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e94910640 1 -- 172.21.15.60:0/767472600 reap_dead start 2024-04-25T10:07:00.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e96373640 1 -- 172.21.15.60:0/767472600 shutdown_connections 2024-04-25T10:07:00.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e96373640 1 -- 172.21.15.60:0/767472600 >> 172.21.15.60:0/767472600 conn(0x7f9e900cfdd0 msgr2=0x7f9e900d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e96373640 1 -- 172.21.15.60:0/767472600 shutdown_connections 2024-04-25T10:07:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e96373640 1 -- 172.21.15.60:0/767472600 wait complete. 2024-04-25T10:07:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e96373640 1 Processor -- start 2024-04-25T10:07:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e96373640 1 -- start start 2024-04-25T10:07:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e96373640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e90159c60 0x7f9e9006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e96373640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e9015e3f0 0x7f9e9015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e96373640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e9015a040 0x7f9e9015d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e96373640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9015c1d0 con 0x7f9e9015e3f0 2024-04-25T10:07:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e96373640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9015bef0 con 0x7f9e9015a040 2024-04-25T10:07:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.071+0000 7f9e96373640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9015c4b0 con 0x7f9e90159c60 2024-04-25T10:07:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e8f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e9015e3f0 0x7f9e9015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e8f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e9015e3f0 0x7f9e9015cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55708/0 (socket says 172.21.15.60:55708) 2024-04-25T10:07:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e8f7fe640 1 -- 172.21.15.60:0/3479751046 learned_addr learned my addr 172.21.15.60:0/3479751046 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e94910640 1 --2- 172.21.15.60:0/3479751046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e9015a040 0x7f9e9015d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e8f7fe640 1 -- 172.21.15.60:0/3479751046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e90159c60 msgr2=0x7f9e9006d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e8f7fe640 1 --2- 172.21.15.60:0/3479751046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e90159c60 0x7f9e9006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e8f7fe640 1 -- 172.21.15.60:0/3479751046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e9015a040 msgr2=0x7f9e9015d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e8f7fe640 1 --2- 172.21.15.60:0/3479751046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e9015a040 0x7f9e9015d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e8f7fe640 1 -- 172.21.15.60:0/3479751046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9006e180 con 0x7f9e9015e3f0 2024-04-25T10:07:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e8f7fe640 1 --2- 172.21.15.60:0/3479751046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e9015e3f0 0x7f9e9015cf90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009fd0 tx=0x7f9e84010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e8cff9640 1 -- 172.21.15.60:0/3479751046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8400f040 con 0x7f9e9015e3f0 2024-04-25T10:07:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e8cff9640 1 -- 172.21.15.60:0/3479751046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8400eb60 con 0x7f9e9015e3f0 2024-04-25T10:07:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e96373640 1 -- 172.21.15.60:0/3479751046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9006e460 con 0x7f9e9015e3f0 2024-04-25T10:07:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e8cff9640 1 -- 172.21.15.60:0/3479751046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8401c400 con 0x7f9e9015e3f0 2024-04-25T10:07:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e96373640 1 -- 172.21.15.60:0/3479751046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9015ca70 con 0x7f9e9015e3f0 2024-04-25T10:07:00.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e96373640 1 -- 172.21.15.60:0/3479751046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e90074770 con 0x7f9e9015e3f0 2024-04-25T10:07:00.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e8cff9640 1 -- 172.21.15.60:0/3479751046 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e840253f0 con 0x7f9e9015e3f0 2024-04-25T10:07:00.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e8cff9640 1 --2- 172.21.15.60:0/3479751046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e7c066f60 0x7f9e7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.075+0000 7f9e8cff9640 1 -- 172.21.15.60:0/3479751046 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9e840854b0 con 0x7f9e9015e3f0 2024-04-25T10:07:00.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.083+0000 7f9e8cff9640 1 -- 172.21.15.60:0/3479751046 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e84018030 con 0x7f9e9015e3f0 2024-04-25T10:07:00.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.083+0000 7f9e8ffff640 1 --2- 172.21.15.60:0/3479751046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e7c066f60 0x7f9e7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.087+0000 7f9e8ffff640 1 --2- 172.21.15.60:0/3479751046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e7c066f60 0x7f9e7c069420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9e80007b00 tx=0x7f9e80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.243+0000 7f9e96373640 1 -- 172.21.15.60:0/3479751046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9e90153e70 con 0x7f9e9015e3f0 2024-04-25T10:07:00.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.243+0000 7f9e8cff9640 1 -- 172.21.15.60:0/3479751046 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9e840567c0 con 0x7f9e9015e3f0 2024-04-25T10:07:00.251 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.247+0000 7f9e96373640 1 -- 172.21.15.60:0/3479751046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e7c066f60 msgr2=0x7f9e7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.247+0000 7f9e96373640 1 --2- 172.21.15.60:0/3479751046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e7c066f60 0x7f9e7c069420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9e80007b00 tx=0x7f9e80009040 comp rx=0 tx=0).stop 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.247+0000 7f9e96373640 1 -- 172.21.15.60:0/3479751046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e9015e3f0 msgr2=0x7f9e9015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.247+0000 7f9e96373640 1 --2- 172.21.15.60:0/3479751046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e9015e3f0 0x7f9e9015cf90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009fd0 tx=0x7f9e84010d60 comp rx=0 tx=0).stop 2024-04-25T10:07:00.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.247+0000 7f9e96373640 1 -- 172.21.15.60:0/3479751046 shutdown_connections 2024-04-25T10:07:00.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.247+0000 7f9e96373640 1 --2- 172.21.15.60:0/3479751046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e7c066f60 0x7f9e7c069420 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.247+0000 7f9e96373640 1 --2- 172.21.15.60:0/3479751046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e9015a040 0x7f9e9015d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.247+0000 7f9e96373640 1 --2- 172.21.15.60:0/3479751046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e9015e3f0 0x7f9e9015cf90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.247+0000 7f9e96373640 1 --2- 172.21.15.60:0/3479751046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e90159c60 0x7f9e9006d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.247+0000 7f9e96373640 1 -- 172.21.15.60:0/3479751046 >> 172.21.15.60:0/3479751046 conn(0x7f9e900cfdd0 msgr2=0x7f9e900afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.247+0000 7f9e96373640 1 -- 172.21.15.60:0/3479751046 shutdown_connections 2024-04-25T10:07:00.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.247+0000 7f9e96373640 1 -- 172.21.15.60:0/3479751046 wait complete. 2024-04-25T10:07:00.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-25T10:07:00.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.387+0000 7fcf097ec640 1 Processor -- start 2024-04-25T10:07:00.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.387+0000 7fcf097ec640 1 -- start start 2024-04-25T10:07:00.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.387+0000 7fcf097ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf040ce7a0 0x7fcf040ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.387+0000 7fcf097ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf040b33f0 con 0x7fcf040b4660 2024-04-25T10:07:00.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.387+0000 7fcf097ec640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf040cf0c0 con 0x7fcf0415bc90 2024-04-25T10:07:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.387+0000 7fcf097ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf040cc370 con 0x7fcf040ce7a0 2024-04-25T10:07:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.387+0000 7fcf02ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf040ce7a0 0x7fcf040ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.387+0000 7fcf02ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf040ce7a0 0x7fcf040ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55720/0 (socket says 172.21.15.60:55720) 2024-04-25T10:07:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.387+0000 7fcf037fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fcf040b4660 0x7fcf040b2ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46108/0 (socket says 172.21.15.60:46108) 2024-04-25T10:07:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.387+0000 7fcf02ffd640 1 -- 172.21.15.60:0/2813571906 learned_addr learned my addr 172.21.15.60:0/2813571906 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.387+0000 7fcf02ffd640 1 -- 172.21.15.60:0/2813571906 >> v1:172.21.15.100:6789/0 conn(0x7fcf0415bc90 legacy=0x7fcf040b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.387+0000 7fcf02ffd640 1 -- 172.21.15.60:0/2813571906 >> v1:172.21.15.60:6789/0 conn(0x7fcf040b4660 legacy=0x7fcf040b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.387+0000 7fcf02ffd640 1 -- 172.21.15.60:0/2813571906 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf04071ca0 con 0x7fcf040ce7a0 2024-04-25T10:07:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf02ffd640 1 --2- 172.21.15.60:0/2813571906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf040ce7a0 0x7fcf040ceb80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009e50 tx=0x7fcefc01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939c9253ca25ecf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf01ffb640 1 -- 172.21.15.60:0/2813571906 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0a5700 con 0x7fcf040ce7a0 2024-04-25T10:07:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf01ffb640 1 -- 172.21.15.60:0/2813571906 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcefc0a58a0 con 0x7fcf040ce7a0 2024-04-25T10:07:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf01ffb640 1 -- 172.21.15.60:0/2813571906 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0a5c70 con 0x7fcf040ce7a0 2024-04-25T10:07:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf097ec640 1 -- 172.21.15.60:0/2813571906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf040ce7a0 msgr2=0x7fcf040ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf097ec640 1 --2- 172.21.15.60:0/2813571906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf040ce7a0 0x7fcf040ceb80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009e50 tx=0x7fcefc01ca30 comp rx=0 tx=0).stop 2024-04-25T10:07:00.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf097ec640 1 -- 172.21.15.60:0/2813571906 shutdown_connections 2024-04-25T10:07:00.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf097ec640 1 --2- 172.21.15.60:0/2813571906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf040ce7a0 0x7fcf040ceb80 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf097ec640 1 -- 172.21.15.60:0/2813571906 >> 172.21.15.60:0/2813571906 conn(0x7fcf040b12d0 msgr2=0x7fcf040b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf097ec640 1 -- 172.21.15.60:0/2813571906 shutdown_connections 2024-04-25T10:07:00.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf097ec640 1 -- 172.21.15.60:0/2813571906 wait complete. 2024-04-25T10:07:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf097ec640 1 Processor -- start 2024-04-25T10:07:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf097ec640 1 -- start start 2024-04-25T10:07:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf097ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf040ce7a0 0x7fcf04149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf097ec640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf0415bc90 0x7fcf04151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf097ec640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0415dca0 0x7fcf0415e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf097ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf040b3d20 con 0x7fcf040ce7a0 2024-04-25T10:07:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf097ec640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf040b3ba0 con 0x7fcf0415dca0 2024-04-25T10:07:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf097ec640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf040b4280 con 0x7fcf0415bc90 2024-04-25T10:07:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf02ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf040ce7a0 0x7fcf04149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf02ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf040ce7a0 0x7fcf04149b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55724/0 (socket says 172.21.15.60:55724) 2024-04-25T10:07:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf02ffd640 1 -- 172.21.15.60:0/612479037 learned_addr learned my addr 172.21.15.60:0/612479037 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf037fe640 1 --2- 172.21.15.60:0/612479037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0415dca0 0x7fcf0415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf027fc640 1 --2- 172.21.15.60:0/612479037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf0415bc90 0x7fcf04151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf02ffd640 1 -- 172.21.15.60:0/612479037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf0415bc90 msgr2=0x7fcf04151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf02ffd640 1 --2- 172.21.15.60:0/612479037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf0415bc90 0x7fcf04151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf02ffd640 1 -- 172.21.15.60:0/612479037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0415dca0 msgr2=0x7fcf0415e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf02ffd640 1 --2- 172.21.15.60:0/612479037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0415dca0 0x7fcf0415e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf02ffd640 1 -- 172.21.15.60:0/612479037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf04152370 con 0x7fcf040ce7a0 2024-04-25T10:07:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf027fc640 1 --2- 172.21.15.60:0/612479037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf0415bc90 0x7fcf04151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.391+0000 7fcf037fe640 1 --2- 172.21.15.60:0/612479037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0415dca0 0x7fcf0415e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.395+0000 7fcf02ffd640 1 --2- 172.21.15.60:0/612479037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf040ce7a0 0x7fcf04149b50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0af040 tx=0x7fcefc0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.395+0000 7fceeb7fe640 1 -- 172.21.15.60:0/612479037 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0bc020 con 0x7fcf040ce7a0 2024-04-25T10:07:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.395+0000 7fceeb7fe640 1 -- 172.21.15.60:0/612479037 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcefc0b8670 con 0x7fcf040ce7a0 2024-04-25T10:07:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.395+0000 7fcf097ec640 1 -- 172.21.15.60:0/612479037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf04152650 con 0x7fcf040ce7a0 2024-04-25T10:07:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.395+0000 7fceeb7fe640 1 -- 172.21.15.60:0/612479037 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0b8a40 con 0x7fcf040ce7a0 2024-04-25T10:07:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.395+0000 7fcf097ec640 1 -- 172.21.15.60:0/612479037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf04152c10 con 0x7fcf040ce7a0 2024-04-25T10:07:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.395+0000 7fcf097ec640 1 -- 172.21.15.60:0/612479037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf04074770 con 0x7fcf040ce7a0 2024-04-25T10:07:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.395+0000 7fceeb7fe640 1 -- 172.21.15.60:0/612479037 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcefc0bf020 con 0x7fcf040ce7a0 2024-04-25T10:07:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.395+0000 7fceeb7fe640 1 --2- 172.21.15.60:0/612479037 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcef0066f90 0x7fcef0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.395+0000 7fceeb7fe640 1 -- 172.21.15.60:0/612479037 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcefc11f740 con 0x7fcf040ce7a0 2024-04-25T10:07:00.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.395+0000 7fcf027fc640 1 --2- 172.21.15.60:0/612479037 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcef0066f90 0x7fcef0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.395+0000 7fcf027fc640 1 --2- 172.21.15.60:0/612479037 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcef0066f90 0x7fcef0069450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcf041569f0 tx=0x7fcef8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.399+0000 7fceeb7fe640 1 -- 172.21.15.60:0/612479037 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcefc0c7020 con 0x7fcf040ce7a0 2024-04-25T10:07:00.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:00.623+0000 7fcf097ec640 1 -- 172.21.15.60:0/612479037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fcf041577b0 con 0x7fcf040ce7a0 2024-04-25T10:07:01.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.579+0000 7fceeb7fe640 1 -- 172.21.15.60:0/612479037 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v112) v1 ==== 152+0+0 (secure 0 0 0) 0x7fcefc016620 con 0x7fcf040ce7a0 2024-04-25T10:07:01.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.583+0000 7fcf097ec640 1 -- 172.21.15.60:0/612479037 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcef0066f90 msgr2=0x7fcef0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.583+0000 7fcf097ec640 1 --2- 172.21.15.60:0/612479037 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcef0066f90 0x7fcef0069450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcf041569f0 tx=0x7fcef8009040 comp rx=0 tx=0).stop 2024-04-25T10:07:01.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.583+0000 7fcf097ec640 1 -- 172.21.15.60:0/612479037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf040ce7a0 msgr2=0x7fcf04149b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.583+0000 7fcf097ec640 1 --2- 172.21.15.60:0/612479037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf040ce7a0 0x7fcf04149b50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0af040 tx=0x7fcefc0197d0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.583+0000 7fcf097ec640 1 -- 172.21.15.60:0/612479037 shutdown_connections 2024-04-25T10:07:01.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.583+0000 7fcf097ec640 1 --2- 172.21.15.60:0/612479037 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcef0066f90 0x7fcef0069450 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.583+0000 7fcf097ec640 1 --2- 172.21.15.60:0/612479037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf0415dca0 0x7fcf0415e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.583+0000 7fcf097ec640 1 --2- 172.21.15.60:0/612479037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf0415bc90 0x7fcf04151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.583+0000 7fcf097ec640 1 --2- 172.21.15.60:0/612479037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf040ce7a0 0x7fcf04149b50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.583+0000 7fcf097ec640 1 -- 172.21.15.60:0/612479037 >> 172.21.15.60:0/612479037 conn(0x7fcf040b12d0 msgr2=0x7fcf040af290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.583+0000 7fcf097ec640 1 -- 172.21.15.60:0/612479037 shutdown_connections 2024-04-25T10:07:01.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.583+0000 7fcf097ec640 1 -- 172.21.15.60:0/612479037 wait complete. 2024-04-25T10:07:01.612 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:07:01.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb981a31640 1 Processor -- start 2024-04-25T10:07:01.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb981a31640 1 -- start start 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb981a31640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb97c06f030 0x7fb97c0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb981a31640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c15ad30 con 0x7fb97c071250 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb981a31640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c15b010 con 0x7fb97c0b3f60 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb981a31640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb97c15aaf0 con 0x7fb97c06f030 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb97b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb97c071250 0x7fb97c15b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46124/0 (socket says 172.21.15.60:46124) 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb97b7fe640 1 -- 172.21.15.60:0/3845505315 learned_addr learned my addr 172.21.15.60:0/3845505315 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb97a7fc640 1 --2- 172.21.15.60:0/3845505315 >> v2:172.21.15.100:3300/0 conn(0x7fb97c06f030 0x7fb97c0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb979ffb640 1 -- 172.21.15.60:0/3845505315 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42788783 0 0) 0x7fb97c15ad30 con 0x7fb97c071250 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb979ffb640 1 -- 172.21.15.60:0/3845505315 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb964004640 con 0x7fb97c071250 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb979ffb640 1 -- 172.21.15.60:0/3845505315 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3980001449 0 0) 0x7fb964004640 con 0x7fb97c071250 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb979ffb640 1 -- 172.21.15.60:0/3845505315 >> v2:172.21.15.100:3300/0 conn(0x7fb97c06f030 msgr2=0x7fb97c0b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb979ffb640 1 --2- 172.21.15.60:0/3845505315 >> v2:172.21.15.100:3300/0 conn(0x7fb97c06f030 0x7fb97c0b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb979ffb640 1 -- 172.21.15.60:0/3845505315 >> v1:172.21.15.100:6789/0 conn(0x7fb97c0b3f60 legacy=0x7fb97c15a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb979ffb640 1 -- 172.21.15.60:0/3845505315 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c15b2f0 con 0x7fb97c071250 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb979ffb640 1 -- 172.21.15.60:0/3845505315 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fb974001680 con 0x7fb97c071250 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb979ffb640 1 -- 172.21.15.60:0/3845505315 >> v1:172.21.15.60:6789/0 conn(0x7fb97c071250 legacy=0x7fb97c15b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb979ffb640 1 --2- 172.21.15.60:0/3845505315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb964004b20 0x7fb964004f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb979ffb640 1 --2- 172.21.15.60:0/3845505315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb964005920 0x7fb964005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb979ffb640 1 --2- 172.21.15.60:0/3845505315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb964006330 0x7fb96400cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb979ffb640 1 -- 172.21.15.60:0/3845505315 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c15aaf0 con 0x7fb964006330 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb979ffb640 1 -- 172.21.15.60:0/3845505315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96400d280 con 0x7fb964005920 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb979ffb640 1 -- 172.21.15.60:0/3845505315 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96400d480 con 0x7fb964004b20 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb97b7fe640 1 --2- 172.21.15.60:0/3845505315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb964006330 0x7fb96400cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb97a7fc640 1 --2- 172.21.15.60:0/3845505315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb964005920 0x7fb964005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb97affd640 1 --2- 172.21.15.60:0/3845505315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb964004b20 0x7fb964004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb97b7fe640 1 -- 172.21.15.60:0/3845505315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb964004b20 msgr2=0x7fb964004f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb97b7fe640 1 --2- 172.21.15.60:0/3845505315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb964004b20 0x7fb964004f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb97b7fe640 1 -- 172.21.15.60:0/3845505315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb964005920 msgr2=0x7fb964005da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb97b7fe640 1 --2- 172.21.15.60:0/3845505315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb964005920 0x7fb964005da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb97b7fe640 1 -- 172.21.15.60:0/3845505315 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb964004640 con 0x7fb964006330 2024-04-25T10:07:01.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.739+0000 7fb97a7fc640 1 --2- 172.21.15.60:0/3845505315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb964005920 0x7fb964005da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:01.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.743+0000 7fb97b7fe640 1 -- 172.21.15.60:0/3845505315 reap_dead start 2024-04-25T10:07:01.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.743+0000 7fb97b7fe640 1 --2- 172.21.15.60:0/3845505315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb964006330 0x7fb96400cae0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb97400c0e0 tx=0x7fb97401f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e89abfb86ce6e4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.743+0000 7fb979ffb640 1 -- 172.21.15.60:0/3845505315 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974007c30 con 0x7fb964006330 2024-04-25T10:07:01.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.743+0000 7fb979ffb640 1 -- 172.21.15.60:0/3845505315 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9740ac070 con 0x7fb964006330 2024-04-25T10:07:01.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.743+0000 7fb979ffb640 1 -- 172.21.15.60:0/3845505315 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974005b20 con 0x7fb964006330 2024-04-25T10:07:01.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.743+0000 7fb981a31640 1 -- 172.21.15.60:0/3845505315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb964006330 msgr2=0x7fb96400cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.743+0000 7fb981a31640 1 --2- 172.21.15.60:0/3845505315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb964006330 0x7fb96400cae0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb97400c0e0 tx=0x7fb97401f2d0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.743+0000 7fb981a31640 1 -- 172.21.15.60:0/3845505315 shutdown_connections 2024-04-25T10:07:01.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.743+0000 7fb981a31640 1 --2- 172.21.15.60:0/3845505315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb964006330 0x7fb96400cae0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.743+0000 7fb981a31640 1 -- 172.21.15.60:0/3845505315 >> 172.21.15.60:0/3845505315 conn(0x7fb97c0670b0 msgr2=0x7fb97c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.743+0000 7fb981a31640 1 -- 172.21.15.60:0/3845505315 shutdown_connections 2024-04-25T10:07:01.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.743+0000 7fb981a31640 1 -- 172.21.15.60:0/3845505315 wait complete. 2024-04-25T10:07:01.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.743+0000 7fb981a31640 1 Processor -- start 2024-04-25T10:07:01.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.743+0000 7fb981a31640 1 -- start start 2024-04-25T10:07:01.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb981a31640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb964006330 0x7fb97c144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb981a31640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb97c143f50 0x7fb97c1553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb981a31640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb97c1518c0 0x7fb97c151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb981a31640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c0716e0 con 0x7fb964006330 2024-04-25T10:07:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb981a31640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c0b4340 con 0x7fb97c1518c0 2024-04-25T10:07:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb981a31640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c071860 con 0x7fb97c143f50 2024-04-25T10:07:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb97affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb964006330 0x7fb97c144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb97affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb964006330 0x7fb97c144890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55742/0 (socket says 172.21.15.60:55742) 2024-04-25T10:07:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb97affd640 1 -- 172.21.15.60:0/3266035852 learned_addr learned my addr 172.21.15.60:0/3266035852 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb97a7fc640 1 --2- 172.21.15.60:0/3266035852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb97c143f50 0x7fb97c1553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb97b7fe640 1 --2- 172.21.15.60:0/3266035852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb97c1518c0 0x7fb97c151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb97affd640 1 -- 172.21.15.60:0/3266035852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb97c143f50 msgr2=0x7fb97c1553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb97affd640 1 --2- 172.21.15.60:0/3266035852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb97c143f50 0x7fb97c1553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb97affd640 1 -- 172.21.15.60:0/3266035852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb97c1518c0 msgr2=0x7fb97c151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb97affd640 1 --2- 172.21.15.60:0/3266035852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb97c1518c0 0x7fb97c151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb97affd640 1 -- 172.21.15.60:0/3266035852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c156950 con 0x7fb964006330 2024-04-25T10:07:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb97a7fc640 1 --2- 172.21.15.60:0/3266035852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb97c143f50 0x7fb97c1553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb97affd640 1 --2- 172.21.15.60:0/3266035852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb964006330 0x7fb97c144890 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb96c010d30 tx=0x7fb96c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb963fff640 1 -- 172.21.15.60:0/3266035852 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c003980 con 0x7fb964006330 2024-04-25T10:07:01.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb981a31640 1 -- 172.21.15.60:0/3266035852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97c156c30 con 0x7fb964006330 2024-04-25T10:07:01.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb963fff640 1 -- 172.21.15.60:0/3266035852 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96c003b20 con 0x7fb964006330 2024-04-25T10:07:01.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb963fff640 1 -- 172.21.15.60:0/3266035852 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0050d0 con 0x7fb964006330 2024-04-25T10:07:01.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb981a31640 1 -- 172.21.15.60:0/3266035852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97c1571f0 con 0x7fb964006330 2024-04-25T10:07:01.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb981a31640 1 -- 172.21.15.60:0/3266035852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97c074770 con 0x7fb964006330 2024-04-25T10:07:01.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb963fff640 1 -- 172.21.15.60:0/3266035852 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb96c00c050 con 0x7fb964006330 2024-04-25T10:07:01.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb963fff640 1 --2- 172.21.15.60:0/3266035852 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb968066f60 0x7fb968069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb963fff640 1 -- 172.21.15.60:0/3266035852 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb96c0878d0 con 0x7fb964006330 2024-04-25T10:07:01.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.747+0000 7fb97a7fc640 1 --2- 172.21.15.60:0/3266035852 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb968066f60 0x7fb968069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.751+0000 7fb97a7fc640 1 --2- 172.21.15.60:0/3266035852 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb968066f60 0x7fb968069420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb97c147a60 tx=0x7fb970008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.755+0000 7fb963fff640 1 -- 172.21.15.60:0/3266035852 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb96c016030 con 0x7fb964006330 2024-04-25T10:07:01.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.915+0000 7fb981a31640 1 -- 172.21.15.60:0/3266035852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb97c067d70 con 0x7fb964006330 2024-04-25T10:07:01.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.919+0000 7fb963fff640 1 -- 172.21.15.60:0/3266035852 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb96c058c70 con 0x7fb964006330 2024-04-25T10:07:01.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.923+0000 7fb981a31640 1 -- 172.21.15.60:0/3266035852 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb968066f60 msgr2=0x7fb968069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.923+0000 7fb981a31640 1 --2- 172.21.15.60:0/3266035852 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb968066f60 0x7fb968069420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb97c147a60 tx=0x7fb970008040 comp rx=0 tx=0).stop 2024-04-25T10:07:01.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.923+0000 7fb981a31640 1 -- 172.21.15.60:0/3266035852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb964006330 msgr2=0x7fb97c144890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.923+0000 7fb981a31640 1 --2- 172.21.15.60:0/3266035852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb964006330 0x7fb97c144890 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb96c010d30 tx=0x7fb96c00e6c0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.923+0000 7fb981a31640 1 -- 172.21.15.60:0/3266035852 shutdown_connections 2024-04-25T10:07:01.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.923+0000 7fb981a31640 1 --2- 172.21.15.60:0/3266035852 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb968066f60 0x7fb968069420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.923+0000 7fb981a31640 1 --2- 172.21.15.60:0/3266035852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb97c1518c0 0x7fb97c151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.923+0000 7fb981a31640 1 --2- 172.21.15.60:0/3266035852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb97c143f50 0x7fb97c1553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.923+0000 7fb981a31640 1 --2- 172.21.15.60:0/3266035852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb964006330 0x7fb97c144890 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.923+0000 7fb981a31640 1 -- 172.21.15.60:0/3266035852 >> 172.21.15.60:0/3266035852 conn(0x7fb97c0670b0 msgr2=0x7fb97c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.923+0000 7fb981a31640 1 -- 172.21.15.60:0/3266035852 shutdown_connections 2024-04-25T10:07:01.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:01.923+0000 7fb981a31640 1 -- 172.21.15.60:0/3266035852 wait complete. 2024-04-25T10:07:01.948 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:07:01.948 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:01.998 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:02.053 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:07:02.053 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:07:02.054 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:02.059 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:07:02.059 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:07:02.059 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:02.064 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:02.064 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:07:02.064 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:02.104 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:07:02.105 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:07:02.105 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:02.111 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:02.111 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:07:02.111 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:02.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.275+0000 7efd56c93640 1 Processor -- start 2024-04-25T10:07:02.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.275+0000 7efd56c93640 1 -- start start 2024-04-25T10:07:02.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.275+0000 7efd56c93640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efd5015dea0 0x7efd5006c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.275+0000 7efd56c93640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efd500ce4f0 0x7efd500d2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.275+0000 7efd56c93640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd5006c680 con 0x7efd500d3110 2024-04-25T10:07:02.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.275+0000 7efd56c93640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5015e280 con 0x7efd5015dea0 2024-04-25T10:07:02.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.275+0000 7efd56c93640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5006c930 con 0x7efd500ce4f0 2024-04-25T10:07:02.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.275+0000 7efd54a2f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efd5015dea0 0x7efd5006c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.275+0000 7efd54a2f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efd5015dea0 0x7efd5006c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55744/0 (socket says 172.21.15.60:55744) 2024-04-25T10:07:02.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.275+0000 7efd54a2f640 1 -- 172.21.15.60:0/3278792000 learned_addr learned my addr 172.21.15.60:0/3278792000 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:02.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.275+0000 7efd4ffff640 1 --2- 172.21.15.60:0/3278792000 >> v2:172.21.15.100:3300/0 conn(0x7efd500ce4f0 0x7efd500d2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.275+0000 7efd54a2f640 1 -- 172.21.15.60:0/3278792000 >> v2:172.21.15.100:3300/0 conn(0x7efd500ce4f0 msgr2=0x7efd500d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.275+0000 7efd54a2f640 1 --2- 172.21.15.60:0/3278792000 >> v2:172.21.15.100:3300/0 conn(0x7efd500ce4f0 0x7efd500d2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.275+0000 7efd54a2f640 1 -- 172.21.15.60:0/3278792000 >> v1:172.21.15.100:6789/0 conn(0x7efd500d3110 legacy=0x7efd500ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.275+0000 7efd54a2f640 1 -- 172.21.15.60:0/3278792000 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd500d20f0 con 0x7efd5015dea0 2024-04-25T10:07:02.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.275+0000 7efd4ffff640 1 --2- 172.21.15.60:0/3278792000 >> v2:172.21.15.100:3300/0 conn(0x7efd500ce4f0 0x7efd500d2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:02.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd54a2f640 1 --2- 172.21.15.60:0/3278792000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd5015dea0 0x7efd5006c140 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efd500b9680 tx=0x7efd4001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a523dac516779e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd4f7fe640 1 -- 172.21.15.60:0/3278792000 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd400b1070 con 0x7efd5015dea0 2024-04-25T10:07:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd4f7fe640 1 -- 172.21.15.60:0/3278792000 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd400ac630 con 0x7efd5015dea0 2024-04-25T10:07:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd4f7fe640 1 -- 172.21.15.60:0/3278792000 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd400a8050 con 0x7efd5015dea0 2024-04-25T10:07:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd56c93640 1 -- 172.21.15.60:0/3278792000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd5015dea0 msgr2=0x7efd5006c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd56c93640 1 --2- 172.21.15.60:0/3278792000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd5015dea0 0x7efd5006c140 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efd500b9680 tx=0x7efd4001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd56c93640 1 -- 172.21.15.60:0/3278792000 shutdown_connections 2024-04-25T10:07:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd56c93640 1 --2- 172.21.15.60:0/3278792000 >> v2:172.21.15.100:3300/0 conn(0x7efd500ce4f0 0x7efd500d2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd56c93640 1 --2- 172.21.15.60:0/3278792000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd5015dea0 0x7efd5006c140 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd56c93640 1 -- 172.21.15.60:0/3278792000 >> 172.21.15.60:0/3278792000 conn(0x7efd500d0480 msgr2=0x7efd500d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd56c93640 1 -- 172.21.15.60:0/3278792000 shutdown_connections 2024-04-25T10:07:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd56c93640 1 -- 172.21.15.60:0/3278792000 wait complete. 2024-04-25T10:07:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd56c93640 1 Processor -- start 2024-04-25T10:07:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd56c93640 1 -- start start 2024-04-25T10:07:02.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd56c93640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd500ce4f0 0x7efd501541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd56c93640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd500d3110 0x7efd50154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd56c93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5015dea0 0x7efd5015c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd56c93640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd500b7620 con 0x7efd500ce4f0 2024-04-25T10:07:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd56c93640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd500cf2f0 con 0x7efd5015dea0 2024-04-25T10:07:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd56c93640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd500b77a0 con 0x7efd500d3110 2024-04-25T10:07:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd54a2f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd500ce4f0 0x7efd501541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd54a2f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd500ce4f0 0x7efd501541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55754/0 (socket says 172.21.15.60:55754) 2024-04-25T10:07:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd54a2f640 1 -- 172.21.15.60:0/2751485192 learned_addr learned my addr 172.21.15.60:0/2751485192 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd55230640 1 --2- 172.21.15.60:0/2751485192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5015dea0 0x7efd5015c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd4ffff640 1 --2- 172.21.15.60:0/2751485192 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd500d3110 0x7efd50154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd54a2f640 1 -- 172.21.15.60:0/2751485192 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd500d3110 msgr2=0x7efd50154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd54a2f640 1 --2- 172.21.15.60:0/2751485192 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd500d3110 0x7efd50154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd54a2f640 1 -- 172.21.15.60:0/2751485192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5015dea0 msgr2=0x7efd5015c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd54a2f640 1 --2- 172.21.15.60:0/2751485192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5015dea0 0x7efd5015c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd54a2f640 1 -- 172.21.15.60:0/2751485192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd501552c0 con 0x7efd500ce4f0 2024-04-25T10:07:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.279+0000 7efd4ffff640 1 --2- 172.21.15.60:0/2751485192 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd500d3110 0x7efd50154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.283+0000 7efd54a2f640 1 --2- 172.21.15.60:0/2751485192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd500ce4f0 0x7efd501541b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efd400af040 tx=0x7efd40019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.283+0000 7efd4d7fa640 1 -- 172.21.15.60:0/2751485192 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd400bc070 con 0x7efd500ce4f0 2024-04-25T10:07:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.283+0000 7efd4d7fa640 1 -- 172.21.15.60:0/2751485192 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd400b7430 con 0x7efd500ce4f0 2024-04-25T10:07:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.283+0000 7efd56c93640 1 -- 172.21.15.60:0/2751485192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd501555a0 con 0x7efd500ce4f0 2024-04-25T10:07:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.283+0000 7efd4d7fa640 1 -- 172.21.15.60:0/2751485192 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd400b1040 con 0x7efd500ce4f0 2024-04-25T10:07:02.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.283+0000 7efd56c93640 1 -- 172.21.15.60:0/2751485192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd50155b60 con 0x7efd500ce4f0 2024-04-25T10:07:02.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.283+0000 7efd56c93640 1 -- 172.21.15.60:0/2751485192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd50074770 con 0x7efd500ce4f0 2024-04-25T10:07:02.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.283+0000 7efd4d7fa640 1 -- 172.21.15.60:0/2751485192 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd400bf070 con 0x7efd500ce4f0 2024-04-25T10:07:02.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.283+0000 7efd4d7fa640 1 --2- 172.21.15.60:0/2751485192 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efd3c066f90 0x7efd3c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.283+0000 7efd4d7fa640 1 -- 172.21.15.60:0/2751485192 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efd4011e480 con 0x7efd500ce4f0 2024-04-25T10:07:02.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.283+0000 7efd4ffff640 1 --2- 172.21.15.60:0/2751485192 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efd3c066f90 0x7efd3c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.283+0000 7efd4ffff640 1 --2- 172.21.15.60:0/2751485192 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efd3c066f90 0x7efd3c069450 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efd500b9790 tx=0x7efd44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.287+0000 7efd4d7fa640 1 -- 172.21.15.60:0/2751485192 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efd400160d0 con 0x7efd500ce4f0 2024-04-25T10:07:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.483+0000 7efd56c93640 1 -- 172.21.15.60:0/2751485192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd50157570 con 0x7efd500ce4f0 2024-04-25T10:07:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.483+0000 7efd4d7fa640 1 -- 172.21.15.60:0/2751485192 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+3705 (secure 0 0 0) 0x7efd400ef790 con 0x7efd500ce4f0 2024-04-25T10:07:02.491 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:02.491 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":17,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2422261630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2422261630},{"type":"v1","addr":"172.21.15.100:6835","nonce":2422261630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2923340508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2923340508},{"type":"v1","addr":"172.21.15.100:6837","nonce":2923340508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1929386449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1929386449},{"type":"v1","addr":"172.21.15.60:6833","nonce":1929386449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1465930305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1465930305},{"type":"v1","addr":"172.21.15.60:6835","nonce":1465930305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[]} 2024-04-25T10:07:02.491 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 17 2024-04-25T10:07:02.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.487+0000 7efd56c93640 1 -- 172.21.15.60:0/2751485192 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efd3c066f90 msgr2=0x7efd3c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.487+0000 7efd56c93640 1 --2- 172.21.15.60:0/2751485192 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efd3c066f90 0x7efd3c069450 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efd500b9790 tx=0x7efd44008040 comp rx=0 tx=0).stop 2024-04-25T10:07:02.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.487+0000 7efd56c93640 1 -- 172.21.15.60:0/2751485192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd500ce4f0 msgr2=0x7efd501541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.487+0000 7efd56c93640 1 --2- 172.21.15.60:0/2751485192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd500ce4f0 0x7efd501541b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efd400af040 tx=0x7efd40019280 comp rx=0 tx=0).stop 2024-04-25T10:07:02.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.487+0000 7efd56c93640 1 -- 172.21.15.60:0/2751485192 shutdown_connections 2024-04-25T10:07:02.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.487+0000 7efd56c93640 1 --2- 172.21.15.60:0/2751485192 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efd3c066f90 0x7efd3c069450 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.487+0000 7efd56c93640 1 --2- 172.21.15.60:0/2751485192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5015dea0 0x7efd5015c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.487+0000 7efd56c93640 1 --2- 172.21.15.60:0/2751485192 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd500d3110 0x7efd50154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.487+0000 7efd56c93640 1 --2- 172.21.15.60:0/2751485192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd500ce4f0 0x7efd501541b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.487+0000 7efd56c93640 1 -- 172.21.15.60:0/2751485192 >> 172.21.15.60:0/2751485192 conn(0x7efd500d0480 msgr2=0x7efd501607e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.487+0000 7efd56c93640 1 -- 172.21.15.60:0/2751485192 shutdown_connections 2024-04-25T10:07:02.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.487+0000 7efd56c93640 1 -- 172.21.15.60:0/2751485192 wait complete. 2024-04-25T10:07:02.511 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:07:02.511 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:07:02.511 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:07:02.511 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:02.512 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:07:02.512 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:07:02.512 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:07:02.512 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:02.512 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:07:02.512 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:07:02.512 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:07:02.513 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:02.513 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:07:02.513 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:07:02.513 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:07:02.513 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:02.523 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:07:02.523 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:07:02.526 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:07:02.526 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:07:02.527 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:07:02.527 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:07:02.530 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:07:02.530 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:07:02.569 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:07:02.570 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:02.571 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:07:02.571 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:02.576 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:07:02.577 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:02.578 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:07:02.578 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:02.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:07:02.636 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T10:07:02.636 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:07:02.647 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T10:07:02.649 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T10:07:02.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.695+0000 7f2b23df3640 1 Processor -- start 2024-04-25T10:07:02.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.699+0000 7f2b23df3640 1 -- start start 2024-04-25T10:07:02.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.699+0000 7f2b23df3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2b1c15a7a0 0x7f2b1c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.699+0000 7f2b23df3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b1c0b37e0 0x7f2b1c0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.699+0000 7f2b23df3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b1c06f130 con 0x7f2b1c06ec30 2024-04-25T10:07:02.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.699+0000 7f2b23df3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b1c0cdaa0 con 0x7f2b1c15a7a0 2024-04-25T10:07:02.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.699+0000 7f2b23df3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c06d2f0 con 0x7f2b1c0b37e0 2024-04-25T10:07:02.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.699+0000 7f2b22390640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b1c0b37e0 0x7f2b1c0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.699+0000 7f2b21b8f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2b1c15a7a0 0x7f2b1c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.699+0000 7f2b22390640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b1c0b37e0 0x7f2b1c0b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:45090/0 (socket says 172.21.15.60:45090) 2024-04-25T10:07:02.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.699+0000 7f2b22390640 1 -- 172.21.15.60:0/3418916376 learned_addr learned my addr 172.21.15.60:0/3418916376 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:02.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.699+0000 7f2b2138e640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2b1c06ec30 0x7f2b1c071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:42852/0 (socket says 172.21.15.60:42852) 2024-04-25T10:07:02.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.699+0000 7f2b21b8f640 1 -- 172.21.15.60:0/3418916376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b1c0b37e0 msgr2=0x7f2b1c0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.699+0000 7f2b21b8f640 1 --2- 172.21.15.60:0/3418916376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b1c0b37e0 0x7f2b1c0b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.699+0000 7f2b21b8f640 1 -- 172.21.15.60:0/3418916376 >> v1:172.21.15.100:6789/0 conn(0x7f2b1c06ec30 legacy=0x7f2b1c071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.699+0000 7f2b21b8f640 1 -- 172.21.15.60:0/3418916376 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c06f410 con 0x7f2b1c15a7a0 2024-04-25T10:07:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.699+0000 7f2b20b8d640 1 -- 172.21.15.60:0/3418916376 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145503685 0 0) 0x7f2b1c06f130 con 0x7f2b1c06ec30 2024-04-25T10:07:02.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b21b8f640 1 --2- 172.21.15.60:0/3418916376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c15a7a0 0x7f2b1c0cd560 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2b180097f0 tx=0x7f2b1801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84c7f1e532966ef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b20b8d640 1 -- 172.21.15.60:0/3418916376 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b180b1070 con 0x7f2b1c15a7a0 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b20b8d640 1 -- 172.21.15.60:0/3418916376 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b180ac410 con 0x7f2b1c15a7a0 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b20b8d640 1 -- 172.21.15.60:0/3418916376 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b180ac7e0 con 0x7f2b1c15a7a0 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b23df3640 1 -- 172.21.15.60:0/3418916376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c15a7a0 msgr2=0x7f2b1c0cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b23df3640 1 --2- 172.21.15.60:0/3418916376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c15a7a0 0x7f2b1c0cd560 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2b180097f0 tx=0x7f2b1801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b23df3640 1 -- 172.21.15.60:0/3418916376 shutdown_connections 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b23df3640 1 --2- 172.21.15.60:0/3418916376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b1c0b37e0 0x7f2b1c0b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b23df3640 1 --2- 172.21.15.60:0/3418916376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c15a7a0 0x7f2b1c0cd560 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b23df3640 1 -- 172.21.15.60:0/3418916376 >> 172.21.15.60:0/3418916376 conn(0x7f2b1c0b12d0 msgr2=0x7f2b1c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b23df3640 1 -- 172.21.15.60:0/3418916376 shutdown_connections 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b23df3640 1 -- 172.21.15.60:0/3418916376 wait complete. 2024-04-25T10:07:02.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b23df3640 1 Processor -- start 2024-04-25T10:07:02.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b23df3640 1 -- start start 2024-04-25T10:07:02.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b23df3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b1c06ec30 0x7f2b1c1f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b23df3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c0b37e0 0x7f2b1c1638a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b23df3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b1c15a7a0 0x7f2b1c163de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b23df3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c06e8e0 con 0x7f2b1c06ec30 2024-04-25T10:07:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b23df3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c06e760 con 0x7f2b1c0b37e0 2024-04-25T10:07:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b23df3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c06d8b0 con 0x7f2b1c15a7a0 2024-04-25T10:07:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b21b8f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b1c06ec30 0x7f2b1c1f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b21b8f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b1c06ec30 0x7f2b1c1f9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55804/0 (socket says 172.21.15.60:55804) 2024-04-25T10:07:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b21b8f640 1 -- 172.21.15.60:0/1097955861 learned_addr learned my addr 172.21.15.60:0/1097955861 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b2138e640 1 --2- 172.21.15.60:0/1097955861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c0b37e0 0x7f2b1c1638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b22390640 1 --2- 172.21.15.60:0/1097955861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b1c15a7a0 0x7f2b1c163de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b21b8f640 1 -- 172.21.15.60:0/1097955861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b1c15a7a0 msgr2=0x7f2b1c163de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b21b8f640 1 --2- 172.21.15.60:0/1097955861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b1c15a7a0 0x7f2b1c163de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b21b8f640 1 -- 172.21.15.60:0/1097955861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c0b37e0 msgr2=0x7f2b1c1638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b21b8f640 1 --2- 172.21.15.60:0/1097955861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c0b37e0 0x7f2b1c1638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b21b8f640 1 -- 172.21.15.60:0/1097955861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c067d00 con 0x7f2b1c06ec30 2024-04-25T10:07:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.703+0000 7f2b21b8f640 1 --2- 172.21.15.60:0/1097955861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b1c06ec30 0x7f2b1c1f9570 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2b180af040 tx=0x7f2b18019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.707+0000 7f2b127fc640 1 -- 172.21.15.60:0/1097955861 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b180b9070 con 0x7f2b1c06ec30 2024-04-25T10:07:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.707+0000 7f2b127fc640 1 -- 172.21.15.60:0/1097955861 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b180acd90 con 0x7f2b1c06ec30 2024-04-25T10:07:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.707+0000 7f2b23df3640 1 -- 172.21.15.60:0/1097955861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b1c152040 con 0x7f2b1c06ec30 2024-04-25T10:07:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.707+0000 7f2b127fc640 1 -- 172.21.15.60:0/1097955861 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b180b1040 con 0x7f2b1c06ec30 2024-04-25T10:07:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.707+0000 7f2b23df3640 1 -- 172.21.15.60:0/1097955861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1c152600 con 0x7f2b1c06ec30 2024-04-25T10:07:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.707+0000 7f2b23df3640 1 -- 172.21.15.60:0/1097955861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b1c074770 con 0x7f2b1c06ec30 2024-04-25T10:07:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.707+0000 7f2b127fc640 1 -- 172.21.15.60:0/1097955861 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2b180bc070 con 0x7f2b1c06ec30 2024-04-25T10:07:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.707+0000 7f2b127fc640 1 --2- 172.21.15.60:0/1097955861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b00066f60 0x7f2b00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.707+0000 7f2b127fc640 1 -- 172.21.15.60:0/1097955861 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2b1811e5c0 con 0x7f2b1c06ec30 2024-04-25T10:07:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.707+0000 7f2b2138e640 1 --2- 172.21.15.60:0/1097955861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b00066f60 0x7f2b00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.707+0000 7f2b2138e640 1 --2- 172.21.15.60:0/1097955861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b00066f60 0x7f2b00069420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c159090 tx=0x7f2b0c007240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.711+0000 7f2b127fc640 1 -- 172.21.15.60:0/1097955861 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2b18016090 con 0x7f2b1c06ec30 2024-04-25T10:07:02.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:02.883+0000 7f2b23df3640 1 -- 172.21.15.60:0/1097955861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2b1c0cd520 con 0x7f2b1c06ec30 2024-04-25T10:07:03.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:03.947+0000 7f2b127fc640 1 -- 172.21.15.60:0/1097955861 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v43) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2b180ef310 con 0x7f2b1c06ec30 2024-04-25T10:07:03.954 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-04-25T10:07:03.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:03.951+0000 7f2b23df3640 1 -- 172.21.15.60:0/1097955861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b00066f60 msgr2=0x7f2b00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:03.951+0000 7f2b23df3640 1 --2- 172.21.15.60:0/1097955861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b00066f60 0x7f2b00069420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c159090 tx=0x7f2b0c007240 comp rx=0 tx=0).stop 2024-04-25T10:07:03.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:03.951+0000 7f2b23df3640 1 -- 172.21.15.60:0/1097955861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b1c06ec30 msgr2=0x7f2b1c1f9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:03.951+0000 7f2b23df3640 1 --2- 172.21.15.60:0/1097955861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b1c06ec30 0x7f2b1c1f9570 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2b180af040 tx=0x7f2b18019240 comp rx=0 tx=0).stop 2024-04-25T10:07:03.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:03.951+0000 7f2b23df3640 1 -- 172.21.15.60:0/1097955861 shutdown_connections 2024-04-25T10:07:03.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:03.951+0000 7f2b23df3640 1 --2- 172.21.15.60:0/1097955861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b00066f60 0x7f2b00069420 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:03.951+0000 7f2b23df3640 1 --2- 172.21.15.60:0/1097955861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b1c15a7a0 0x7f2b1c163de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:03.951+0000 7f2b23df3640 1 --2- 172.21.15.60:0/1097955861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b1c0b37e0 0x7f2b1c1638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:03.951+0000 7f2b23df3640 1 --2- 172.21.15.60:0/1097955861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b1c06ec30 0x7f2b1c1f9570 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:03.951+0000 7f2b23df3640 1 -- 172.21.15.60:0/1097955861 >> 172.21.15.60:0/1097955861 conn(0x7f2b1c0b12d0 msgr2=0x7f2b1c0cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:03.951+0000 7f2b23df3640 1 -- 172.21.15.60:0/1097955861 shutdown_connections 2024-04-25T10:07:03.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:03.951+0000 7f2b23df3640 1 -- 172.21.15.60:0/1097955861 wait complete. 2024-04-25T10:07:03.977 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:07:04.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.095+0000 7f435aae9640 1 Processor -- start 2024-04-25T10:07:04.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.095+0000 7f435aae9640 1 -- start start 2024-04-25T10:07:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.099+0000 7f435aae9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f435415bc90 0x7f435415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.099+0000 7f435aae9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43540b4660 0x7f435415c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.099+0000 7f435aae9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4354071ca0 con 0x7f43540ce7a0 2024-04-25T10:07:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.099+0000 7f435aae9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43540cc370 con 0x7f43540b4660 2024-04-25T10:07:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.099+0000 7f435aae9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43540cca40 con 0x7f435415bc90 2024-04-25T10:07:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.099+0000 7f4359086640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43540b4660 0x7f435415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.099+0000 7f4359086640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43540b4660 0x7f435415c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55818/0 (socket says 172.21.15.60:55818) 2024-04-25T10:07:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.099+0000 7f4359086640 1 -- 172.21.15.60:0/1346404429 learned_addr learned my addr 172.21.15.60:0/1346404429 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.099+0000 7f4353fff640 1 --2- 172.21.15.60:0/1346404429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f435415bc90 0x7f435415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.099+0000 7f4359086640 1 -- 172.21.15.60:0/1346404429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f435415bc90 msgr2=0x7f435415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.099+0000 7f4359086640 1 --2- 172.21.15.60:0/1346404429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f435415bc90 0x7f435415c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.099+0000 7f4359086640 1 -- 172.21.15.60:0/1346404429 >> v1:172.21.15.100:6789/0 conn(0x7f43540ce7a0 legacy=0x7f43540ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.099+0000 7f4359086640 1 -- 172.21.15.60:0/1346404429 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43540ccd20 con 0x7f43540b4660 2024-04-25T10:07:04.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.099+0000 7f4359086640 1 --2- 172.21.15.60:0/1346404429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43540b4660 0x7f435415c760 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f434c009e50 tx=0x7f434c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ba4fe940bfeadc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.099+0000 7f43537fe640 1 -- 172.21.15.60:0/1346404429 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c0a5700 con 0x7f43540b4660 2024-04-25T10:07:04.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f43537fe640 1 -- 172.21.15.60:0/1346404429 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434c0a58a0 con 0x7f43540b4660 2024-04-25T10:07:04.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f43537fe640 1 -- 172.21.15.60:0/1346404429 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c0a5c70 con 0x7f43540b4660 2024-04-25T10:07:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 -- 172.21.15.60:0/1346404429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43540b4660 msgr2=0x7f435415c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 --2- 172.21.15.60:0/1346404429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43540b4660 0x7f435415c760 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f434c009e50 tx=0x7f434c01ca30 comp rx=0 tx=0).stop 2024-04-25T10:07:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 -- 172.21.15.60:0/1346404429 shutdown_connections 2024-04-25T10:07:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 --2- 172.21.15.60:0/1346404429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43540b4660 0x7f435415c760 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 --2- 172.21.15.60:0/1346404429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f435415bc90 0x7f435415c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 -- 172.21.15.60:0/1346404429 >> 172.21.15.60:0/1346404429 conn(0x7f43540b12d0 msgr2=0x7f43540b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 -- 172.21.15.60:0/1346404429 shutdown_connections 2024-04-25T10:07:04.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 -- 172.21.15.60:0/1346404429 wait complete. 2024-04-25T10:07:04.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 Processor -- start 2024-04-25T10:07:04.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 -- start start 2024-04-25T10:07:04.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43540ce7a0 0x7f4354148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f435415bc90 0x7f43540b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43540b0690 0x7f4354155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43540b3db0 con 0x7f43540b0690 2024-04-25T10:07:04.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43540b4490 con 0x7f43540ce7a0 2024-04-25T10:07:04.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43541685c0 con 0x7f435415bc90 2024-04-25T10:07:04.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f4359086640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43540b0690 0x7f4354155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f4359086640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43540b0690 0x7f4354155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55824/0 (socket says 172.21.15.60:55824) 2024-04-25T10:07:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f4359086640 1 -- 172.21.15.60:0/1719898674 learned_addr learned my addr 172.21.15.60:0/1719898674 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f4358885640 1 --2- 172.21.15.60:0/1719898674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43540ce7a0 0x7f4354148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f4353fff640 1 --2- 172.21.15.60:0/1719898674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f435415bc90 0x7f43540b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f4359086640 1 -- 172.21.15.60:0/1719898674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f435415bc90 msgr2=0x7f43540b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f4359086640 1 --2- 172.21.15.60:0/1719898674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f435415bc90 0x7f43540b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f4359086640 1 -- 172.21.15.60:0/1719898674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43540ce7a0 msgr2=0x7f4354148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f4359086640 1 --2- 172.21.15.60:0/1719898674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43540ce7a0 0x7f4354148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f4359086640 1 -- 172.21.15.60:0/1719898674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43541493e0 con 0x7f43540b0690 2024-04-25T10:07:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f4358885640 1 --2- 172.21.15.60:0/1719898674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43540ce7a0 0x7f4354148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f4353fff640 1 --2- 172.21.15.60:0/1719898674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f435415bc90 0x7f43540b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f4359086640 1 --2- 172.21.15.60:0/1719898674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43540b0690 0x7f4354155e10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f434c0af040 tx=0x7f434c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f43517fa640 1 -- 172.21.15.60:0/1719898674 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c0b8670 con 0x7f43540b0690 2024-04-25T10:07:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f43517fa640 1 -- 172.21.15.60:0/1719898674 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434c0be070 con 0x7f43540b0690 2024-04-25T10:07:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 -- 172.21.15.60:0/1719898674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43541569f0 con 0x7f43540b0690 2024-04-25T10:07:04.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f43517fa640 1 -- 172.21.15.60:0/1719898674 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c0b8a30 con 0x7f43540b0690 2024-04-25T10:07:04.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.103+0000 7f435aae9640 1 -- 172.21.15.60:0/1719898674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4354156fb0 con 0x7f43540b0690 2024-04-25T10:07:04.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.107+0000 7f435aae9640 1 -- 172.21.15.60:0/1719898674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4354157a70 con 0x7f43540b0690 2024-04-25T10:07:04.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.107+0000 7f43517fa640 1 -- 172.21.15.60:0/1719898674 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f434c0b8bd0 con 0x7f43540b0690 2024-04-25T10:07:04.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.107+0000 7f43517fa640 1 --2- 172.21.15.60:0/1719898674 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4340066f60 0x7f4340069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.107+0000 7f43517fa640 1 -- 172.21.15.60:0/1719898674 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f434c11fed0 con 0x7f43540b0690 2024-04-25T10:07:04.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.107+0000 7f4358885640 1 --2- 172.21.15.60:0/1719898674 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4340066f60 0x7f4340069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.107+0000 7f4358885640 1 --2- 172.21.15.60:0/1719898674 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4340066f60 0x7f4340069420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f43540b3610 tx=0x7f4344009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.111+0000 7f43517fa640 1 -- 172.21.15.60:0/1719898674 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f434c016620 con 0x7f43540b0690 2024-04-25T10:07:04.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.327+0000 7f435aae9640 1 -- 172.21.15.60:0/1719898674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4354074770 con 0x7f43540b0690 2024-04-25T10:07:04.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.327+0000 7f43517fa640 1 -- 172.21.15.60:0/1719898674 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f434c0f12c0 con 0x7f43540b0690 2024-04-25T10:07:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:04.336 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:07:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-04-25T10:07:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-04-25T10:07:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmKr12EhAAnHHZ5VjG0cywMrIv6VElVg==", 2024-04-25T10:07:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:07:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:07:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:07:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:07:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-04-25T10:07:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmU39rIRAAk6Ca0tKG4LpmR4pNxjzv0w==", 2024-04-25T10:07:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:07:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:07:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:07:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:07:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-04-25T10:07:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmVsshGxAAttXv0jYEGKgo0sPhIRTuqQ==", 2024-04-25T10:07:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:07:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:07:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:07:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:07:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-04-25T10:07:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmQtXUKRAAI3Iq02h3KH/PZbWKvJD9Jg==", 2024-04-25T10:07:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:07:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:07:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:07:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:07:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-04-25T10:07:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCkKipmE+UIBRAAc8ctpHoL3PkkbkMRCAJUGw==", 2024-04-25T10:07:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:07:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:07:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-04-25T10:07:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCnKipmbAnYCBAALLsPH2FjUpjoIZwTw+8aOA==", 2024-04-25T10:07:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:07:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:07:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-04-25T10:07:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCqKipmqrNSDxAAZdgFrcyeVwRQMKJJYIM98g==", 2024-04-25T10:07:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:07:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:07:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-04-25T10:07:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCtKipm1L+hFxAAPnPWlCm1Fmi0mN3tyReeQw==", 2024-04-25T10:07:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:07:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:07:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-04-25T10:07:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCzKipm2WyDFRAAmyfg6qu5drE9yMaR2FKd/A==", 2024-04-25T10:07:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:07:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:07:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-04-25T10:07:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC2KipmZLDhGRAAcgaz8YaPGUMIfxOLtWObyA==", 2024-04-25T10:07:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:07:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:07:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-04-25T10:07:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC5Kipm/3VMIBAA+rhGCyA/7d+s8m7ZWTzX/Q==", 2024-04-25T10:07:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:07:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:07:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-04-25T10:07:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8KipmFNCCJxAAv0OSTpAG+xIHhLqYXAZazQ==", 2024-04-25T10:07:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:07:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:07:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-04-25T10:07:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmv65dLxAA6/QpPrqE9aT+CtZqkyutog==", 2024-04-25T10:07:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:07:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-04-25T10:07:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:07:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-04-25T10:07:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipm2NiONxAAd3X4zx1JnBXKHJvdZ/zMdQ==", 2024-04-25T10:07:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:07:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:07:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:07:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:07:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-04-25T10:07:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmZRONNBAAa+XTv5YIBJWeUvf5dfoWxw==", 2024-04-25T10:07:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:07:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:07:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:07:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:07:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.352 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-04-25T10:07:04.352 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCjKipmgvfiABAApLvTC9U6HzeAefCvX4tiSQ==", 2024-04-25T10:07:04.352 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.352 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:07:04.352 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:07:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:07:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:07:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-04-25T10:07:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQChKipmumzYNhAAlCRYMpSNaTo7bz8P6x0W9g==", 2024-04-25T10:07:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:07:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:07:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:07:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:07:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-04-25T10:07:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmhxlLDxAAegBrJWdMNz6ZC3oXT2Qo0A==", 2024-04-25T10:07:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.356 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:07:04.356 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T10:07:04.356 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:07:04.356 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:04.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:04.357 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-04-25T10:07:04.357 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmpl1cDBAAGqx2kIak9NoWpkcdHczcFA==", 2024-04-25T10:07:04.357 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:07:04.357 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:07:04.357 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T10:07:04.358 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:07:04.358 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.358 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:04.358 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T10:07:04.358 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:07:04.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.331+0000 7f435aae9640 1 -- 172.21.15.60:0/1719898674 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4340066f60 msgr2=0x7f4340069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.331+0000 7f435aae9640 1 --2- 172.21.15.60:0/1719898674 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4340066f60 0x7f4340069420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f43540b3610 tx=0x7f4344009040 comp rx=0 tx=0).stop 2024-04-25T10:07:04.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.331+0000 7f435aae9640 1 -- 172.21.15.60:0/1719898674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43540b0690 msgr2=0x7f4354155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.331+0000 7f435aae9640 1 --2- 172.21.15.60:0/1719898674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43540b0690 0x7f4354155e10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f434c0af040 tx=0x7f434c0197d0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.331+0000 7f435aae9640 1 -- 172.21.15.60:0/1719898674 shutdown_connections 2024-04-25T10:07:04.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.331+0000 7f435aae9640 1 --2- 172.21.15.60:0/1719898674 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4340066f60 0x7f4340069420 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.331+0000 7f435aae9640 1 --2- 172.21.15.60:0/1719898674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43540b0690 0x7f4354155e10 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.331+0000 7f435aae9640 1 --2- 172.21.15.60:0/1719898674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f435415bc90 0x7f43540b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.331+0000 7f435aae9640 1 --2- 172.21.15.60:0/1719898674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43540ce7a0 0x7f4354148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.331+0000 7f435aae9640 1 -- 172.21.15.60:0/1719898674 >> 172.21.15.60:0/1719898674 conn(0x7f43540b12d0 msgr2=0x7f435415e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.331+0000 7f435aae9640 1 -- 172.21.15.60:0/1719898674 shutdown_connections 2024-04-25T10:07:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.331+0000 7f435aae9640 1 -- 172.21.15.60:0/1719898674 wait complete. 2024-04-25T10:07:04.361 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.483+0000 7f7604f09640 1 Processor -- start 2024-04-25T10:07:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.483+0000 7f7604f09640 1 -- start start 2024-04-25T10:07:04.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.483+0000 7f7604f09640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f760015dea0 0x7f760006c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.483+0000 7f7604f09640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f76000ce4f0 0x7f76000d2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.483+0000 7f7604f09640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f760006c680 con 0x7f76000d3110 2024-04-25T10:07:04.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.483+0000 7f7604f09640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f760015e280 con 0x7f760015dea0 2024-04-25T10:07:04.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.483+0000 7f7604f09640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f760006c930 con 0x7f76000ce4f0 2024-04-25T10:07:04.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.483+0000 7f75fe59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f760015dea0 0x7f760006c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.483+0000 7f75fe59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f760015dea0 0x7f760006c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55826/0 (socket says 172.21.15.60:55826) 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.483+0000 7f75fe59c640 1 -- 172.21.15.60:0/331772432 learned_addr learned my addr 172.21.15.60:0/331772432 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.483+0000 7f75f7fff640 1 --2- 172.21.15.60:0/331772432 >> v2:172.21.15.100:3300/0 conn(0x7f76000ce4f0 0x7f76000d2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.483+0000 7f75fe59c640 1 -- 172.21.15.60:0/331772432 >> v2:172.21.15.100:3300/0 conn(0x7f76000ce4f0 msgr2=0x7f76000d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.483+0000 7f75fe59c640 1 --2- 172.21.15.60:0/331772432 >> v2:172.21.15.100:3300/0 conn(0x7f76000ce4f0 0x7f76000d2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.483+0000 7f75fe59c640 1 -- 172.21.15.60:0/331772432 >> v1:172.21.15.60:6789/0 conn(0x7f76000d3110 legacy=0x7f76000ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.483+0000 7f75fe59c640 1 -- 172.21.15.60:0/331772432 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76000d20f0 con 0x7f760015dea0 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.483+0000 7f75f7fff640 1 --2- 172.21.15.60:0/331772432 >> v2:172.21.15.100:3300/0 conn(0x7f76000ce4f0 0x7f76000d2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:04.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f75fe59c640 1 --2- 172.21.15.60:0/331772432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f760015dea0 0x7f760006c140 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f76000b9680 tx=0x7f75e801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e189f702cb6c3ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f75fdd9b640 1 -- 172.21.15.60:0/331772432 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e80ab610 con 0x7f760015dea0 2024-04-25T10:07:04.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f75fdd9b640 1 -- 172.21.15.60:0/331772432 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e80b0070 con 0x7f760015dea0 2024-04-25T10:07:04.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f75fdd9b640 1 -- 172.21.15.60:0/331772432 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e80ab9d0 con 0x7f760015dea0 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f7604f09640 1 -- 172.21.15.60:0/331772432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f760015dea0 msgr2=0x7f760006c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f7604f09640 1 --2- 172.21.15.60:0/331772432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f760015dea0 0x7f760006c140 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f76000b9680 tx=0x7f75e801c520 comp rx=0 tx=0).stop 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f7604f09640 1 -- 172.21.15.60:0/331772432 shutdown_connections 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f7604f09640 1 --2- 172.21.15.60:0/331772432 >> v2:172.21.15.100:3300/0 conn(0x7f76000ce4f0 0x7f76000d2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f7604f09640 1 --2- 172.21.15.60:0/331772432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f760015dea0 0x7f760006c140 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f7604f09640 1 -- 172.21.15.60:0/331772432 >> 172.21.15.60:0/331772432 conn(0x7f76000d0480 msgr2=0x7f76000d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f7604f09640 1 -- 172.21.15.60:0/331772432 shutdown_connections 2024-04-25T10:07:04.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f7604f09640 1 -- 172.21.15.60:0/331772432 wait complete. 2024-04-25T10:07:04.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f7604f09640 1 Processor -- start 2024-04-25T10:07:04.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f7604f09640 1 -- start start 2024-04-25T10:07:04.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f7604f09640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76000ce4f0 0x7f7600168000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f7604f09640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76000d3110 0x7f7600155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f7604f09640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f760015dea0 0x7f7600153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f7604f09640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76000b7620 con 0x7f76000d3110 2024-04-25T10:07:04.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f7604f09640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76000cf330 con 0x7f760015dea0 2024-04-25T10:07:04.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f7604f09640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76000b77a0 con 0x7f76000ce4f0 2024-04-25T10:07:04.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f75f7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76000d3110 0x7f7600155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f75f7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76000d3110 0x7f7600155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55834/0 (socket says 172.21.15.60:55834) 2024-04-25T10:07:04.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f75f7fff640 1 -- 172.21.15.60:0/1237684709 learned_addr learned my addr 172.21.15.60:0/1237684709 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:04.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f75fed9d640 1 --2- 172.21.15.60:0/1237684709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f760015dea0 0x7f7600153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f75f7fff640 1 -- 172.21.15.60:0/1237684709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76000ce4f0 msgr2=0x7f7600168000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f75fe59c640 1 --2- 172.21.15.60:0/1237684709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76000ce4f0 0x7f7600168000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f75f7fff640 1 --2- 172.21.15.60:0/1237684709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76000ce4f0 0x7f7600168000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f75f7fff640 1 -- 172.21.15.60:0/1237684709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f760015dea0 msgr2=0x7f7600153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f75f7fff640 1 --2- 172.21.15.60:0/1237684709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f760015dea0 0x7f7600153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f75f7fff640 1 -- 172.21.15.60:0/1237684709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760015e7a0 con 0x7f76000d3110 2024-04-25T10:07:04.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f75fed9d640 1 --2- 172.21.15.60:0/1237684709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f760015dea0 0x7f7600153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:04.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.487+0000 7f75f7fff640 1 --2- 172.21.15.60:0/1237684709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76000d3110 0x7f7600155f60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f75f000c7f0 tx=0x7f75f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.491+0000 7f75f6ffd640 1 -- 172.21.15.60:0/1237684709 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0015070 con 0x7f76000d3110 2024-04-25T10:07:04.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.491+0000 7f75f6ffd640 1 -- 172.21.15.60:0/1237684709 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f0007820 con 0x7f76000d3110 2024-04-25T10:07:04.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.491+0000 7f7604f09640 1 -- 172.21.15.60:0/1237684709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760015ea80 con 0x7f76000d3110 2024-04-25T10:07:04.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.491+0000 7f75f6ffd640 1 -- 172.21.15.60:0/1237684709 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0007c10 con 0x7f76000d3110 2024-04-25T10:07:04.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.491+0000 7f7604f09640 1 -- 172.21.15.60:0/1237684709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f760015f040 con 0x7f76000d3110 2024-04-25T10:07:04.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.491+0000 7f7604f09640 1 -- 172.21.15.60:0/1237684709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7600074770 con 0x7f76000d3110 2024-04-25T10:07:04.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.491+0000 7f75f6ffd640 1 -- 172.21.15.60:0/1237684709 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f75f0018070 con 0x7f76000d3110 2024-04-25T10:07:04.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.491+0000 7f75f6ffd640 1 --2- 172.21.15.60:0/1237684709 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f75e0066f60 0x7f75e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.491+0000 7f75f6ffd640 1 -- 172.21.15.60:0/1237684709 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f75f0084bc0 con 0x7f76000d3110 2024-04-25T10:07:04.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.491+0000 7f75fe59c640 1 --2- 172.21.15.60:0/1237684709 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f75e0066f60 0x7f75e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.491+0000 7f75fe59c640 1 --2- 172.21.15.60:0/1237684709 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f75e0066f60 0x7f75e0069420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f75e80af000 tx=0x7f75e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.495+0000 7f75f6ffd640 1 -- 172.21.15.60:0/1237684709 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75f0055f20 con 0x7f76000d3110 2024-04-25T10:07:04.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.655+0000 7f7604f09640 1 -- 172.21.15.60:0/1237684709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f760015f3e0 con 0x7f76000d3110 2024-04-25T10:07:04.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.655+0000 7f75f6ffd640 1 -- 172.21.15.60:0/1237684709 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+12903 (secure 0 0 0) 0x7f75f0055820 con 0x7f76000d3110 2024-04-25T10:07:04.664 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:04.664 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":43,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:07:03.946469+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:04.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.659+0000 7f7604f09640 1 -- 172.21.15.60:0/1237684709 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f75e0066f60 msgr2=0x7f75e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.659+0000 7f7604f09640 1 --2- 172.21.15.60:0/1237684709 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f75e0066f60 0x7f75e0069420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f75e80af000 tx=0x7f75e8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:04.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.659+0000 7f7604f09640 1 -- 172.21.15.60:0/1237684709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76000d3110 msgr2=0x7f7600155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.659+0000 7f7604f09640 1 --2- 172.21.15.60:0/1237684709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76000d3110 0x7f7600155f60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f75f000c7f0 tx=0x7f75f000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.659+0000 7f7604f09640 1 -- 172.21.15.60:0/1237684709 shutdown_connections 2024-04-25T10:07:04.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.659+0000 7f7604f09640 1 --2- 172.21.15.60:0/1237684709 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f75e0066f60 0x7f75e0069420 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.659+0000 7f7604f09640 1 --2- 172.21.15.60:0/1237684709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f760015dea0 0x7f7600153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.659+0000 7f7604f09640 1 --2- 172.21.15.60:0/1237684709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76000d3110 0x7f7600155f60 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.659+0000 7f7604f09640 1 --2- 172.21.15.60:0/1237684709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76000ce4f0 0x7f7600168000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.659+0000 7f7604f09640 1 -- 172.21.15.60:0/1237684709 >> 172.21.15.60:0/1237684709 conn(0x7f76000d0480 msgr2=0x7f76000cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.659+0000 7f7604f09640 1 -- 172.21.15.60:0/1237684709 shutdown_connections 2024-04-25T10:07:04.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.659+0000 7f7604f09640 1 -- 172.21.15.60:0/1237684709 wait complete. 2024-04-25T10:07:04.684 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:04.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:04.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff76059e640 1 Processor -- start 2024-04-25T10:07:04.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff76059e640 1 -- start start 2024-04-25T10:07:04.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff76059e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75815ab30 0x7ff7580ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff76059e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff7580d0b90 0x7ff7580b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff76059e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff758072340 con 0x7ff7580afc30 2024-04-25T10:07:04.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff76059e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7580725b0 con 0x7ff7580d0b90 2024-04-25T10:07:04.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff76059e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff758072890 con 0x7ff75815ab30 2024-04-25T10:07:04.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff75db39640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff7580afc30 0x7ff7580d0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46142/0 (socket says 172.21.15.60:46142) 2024-04-25T10:07:04.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff75db39640 1 -- 172.21.15.60:0/3493950485 learned_addr learned my addr 172.21.15.60:0/3493950485 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:04.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff75eb3b640 1 --2- 172.21.15.60:0/3493950485 >> v2:172.21.15.60:3300/0 conn(0x7ff7580d0b90 0x7ff7580b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff75e33a640 1 --2- 172.21.15.60:0/3493950485 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75815ab30 0x7ff7580ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff75d338640 1 -- 172.21.15.60:0/3493950485 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954775170 0 0) 0x7ff758072340 con 0x7ff7580afc30 2024-04-25T10:07:04.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff75d338640 1 -- 172.21.15.60:0/3493950485 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff744003530 con 0x7ff7580afc30 2024-04-25T10:07:04.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff75eb3b640 1 -- 172.21.15.60:0/3493950485 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75815ab30 msgr2=0x7ff7580ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff75eb3b640 1 --2- 172.21.15.60:0/3493950485 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75815ab30 0x7ff7580ce1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff75eb3b640 1 -- 172.21.15.60:0/3493950485 >> v1:172.21.15.60:6789/0 conn(0x7ff7580afc30 legacy=0x7ff7580d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.803+0000 7ff75eb3b640 1 -- 172.21.15.60:0/3493950485 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff758072b70 con 0x7ff7580d0b90 2024-04-25T10:07:04.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff75eb3b640 1 --2- 172.21.15.60:0/3493950485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7580d0b90 0x7ff7580b3b40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff750009d90 tx=0x7ff75001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f526f4d9cc92645 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff75d338640 1 -- 172.21.15.60:0/3493950485 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7500b2070 con 0x7ff7580d0b90 2024-04-25T10:07:04.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff75d338640 1 -- 172.21.15.60:0/3493950485 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7500a5990 con 0x7ff7580d0b90 2024-04-25T10:07:04.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff75d338640 1 -- 172.21.15.60:0/3493950485 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7500a5d40 con 0x7ff7580d0b90 2024-04-25T10:07:04.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff76059e640 1 -- 172.21.15.60:0/3493950485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7580d0b90 msgr2=0x7ff7580b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff76059e640 1 --2- 172.21.15.60:0/3493950485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7580d0b90 0x7ff7580b3b40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff750009d90 tx=0x7ff75001c8b0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff76059e640 1 -- 172.21.15.60:0/3493950485 shutdown_connections 2024-04-25T10:07:04.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff76059e640 1 --2- 172.21.15.60:0/3493950485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7580d0b90 0x7ff7580b3b40 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff76059e640 1 --2- 172.21.15.60:0/3493950485 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75815ab30 0x7ff7580ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff76059e640 1 -- 172.21.15.60:0/3493950485 >> 172.21.15.60:0/3493950485 conn(0x7ff7580b2360 msgr2=0x7ff7580b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff76059e640 1 -- 172.21.15.60:0/3493950485 shutdown_connections 2024-04-25T10:07:04.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff76059e640 1 -- 172.21.15.60:0/3493950485 wait complete. 2024-04-25T10:07:04.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff76059e640 1 Processor -- start 2024-04-25T10:07:04.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff76059e640 1 -- start start 2024-04-25T10:07:04.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff76059e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7580afc30 0x7ff758164780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff76059e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7580d0b90 0x7ff75806ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff76059e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75815ab30 0x7ff758156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff76059e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7580b4c60 con 0x7ff75815ab30 2024-04-25T10:07:04.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff76059e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7580b4ae0 con 0x7ff7580afc30 2024-04-25T10:07:04.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff76059e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7580b4de0 con 0x7ff7580d0b90 2024-04-25T10:07:04.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.807+0000 7ff75eb3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75815ab30 0x7ff758156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff75eb3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75815ab30 0x7ff758156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55846/0 (socket says 172.21.15.60:55846) 2024-04-25T10:07:04.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff75eb3b640 1 -- 172.21.15.60:0/1374945878 learned_addr learned my addr 172.21.15.60:0/1374945878 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:04.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff75e33a640 1 --2- 172.21.15.60:0/1374945878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7580afc30 0x7ff758164780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff75db39640 1 --2- 172.21.15.60:0/1374945878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7580d0b90 0x7ff75806ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff75eb3b640 1 -- 172.21.15.60:0/1374945878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7580d0b90 msgr2=0x7ff75806ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff75eb3b640 1 --2- 172.21.15.60:0/1374945878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7580d0b90 0x7ff75806ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff75eb3b640 1 -- 172.21.15.60:0/1374945878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7580afc30 msgr2=0x7ff758164780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff75eb3b640 1 --2- 172.21.15.60:0/1374945878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7580afc30 0x7ff758164780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff75eb3b640 1 -- 172.21.15.60:0/1374945878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7580b00d0 con 0x7ff75815ab30 2024-04-25T10:07:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff75eb3b640 1 --2- 172.21.15.60:0/1374945878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75815ab30 0x7ff758156c10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff75001a4e0 tx=0x7ff750019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff74effd640 1 -- 172.21.15.60:0/1374945878 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7500b2050 con 0x7ff75815ab30 2024-04-25T10:07:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff74effd640 1 -- 172.21.15.60:0/1374945878 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7500be070 con 0x7ff75815ab30 2024-04-25T10:07:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff76059e640 1 -- 172.21.15.60:0/1374945878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff758164fe0 con 0x7ff75815ab30 2024-04-25T10:07:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff74effd640 1 -- 172.21.15.60:0/1374945878 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7500adab0 con 0x7ff75815ab30 2024-04-25T10:07:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff76059e640 1 -- 172.21.15.60:0/1374945878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7580b0670 con 0x7ff75815ab30 2024-04-25T10:07:04.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff76059e640 1 -- 172.21.15.60:0/1374945878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff758074770 con 0x7ff75815ab30 2024-04-25T10:07:04.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff74effd640 1 -- 172.21.15.60:0/1374945878 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7500adc50 con 0x7ff75815ab30 2024-04-25T10:07:04.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff74effd640 1 --2- 172.21.15.60:0/1374945878 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff73c066f60 0x7ff73c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff74effd640 1 -- 172.21.15.60:0/1374945878 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff75011fea0 con 0x7ff75815ab30 2024-04-25T10:07:04.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.811+0000 7ff75e33a640 1 --2- 172.21.15.60:0/1374945878 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff73c066f60 0x7ff73c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.815+0000 7ff75e33a640 1 --2- 172.21.15.60:0/1374945878 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff73c066f60 0x7ff73c069420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff75400b9d0 tx=0x7ff754008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.815+0000 7ff74effd640 1 -- 172.21.15.60:0/1374945878 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff750016560 con 0x7ff75815ab30 2024-04-25T10:07:04.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.979+0000 7ff76059e640 1 -- 172.21.15.60:0/1374945878 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7581580b0 con 0x7ff75815ab30 2024-04-25T10:07:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.979+0000 7ff74effd640 1 -- 172.21.15.60:0/1374945878 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff7500ab340 con 0x7ff75815ab30 2024-04-25T10:07:04.985 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:07:04.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.979+0000 7ff76059e640 1 -- 172.21.15.60:0/1374945878 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff73c066f60 msgr2=0x7ff73c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.979+0000 7ff76059e640 1 --2- 172.21.15.60:0/1374945878 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff73c066f60 0x7ff73c069420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff75400b9d0 tx=0x7ff754008040 comp rx=0 tx=0).stop 2024-04-25T10:07:04.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.979+0000 7ff76059e640 1 -- 172.21.15.60:0/1374945878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75815ab30 msgr2=0x7ff758156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.979+0000 7ff76059e640 1 --2- 172.21.15.60:0/1374945878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75815ab30 0x7ff758156c10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff75001a4e0 tx=0x7ff750019650 comp rx=0 tx=0).stop 2024-04-25T10:07:04.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.979+0000 7ff76059e640 1 -- 172.21.15.60:0/1374945878 shutdown_connections 2024-04-25T10:07:04.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.979+0000 7ff76059e640 1 --2- 172.21.15.60:0/1374945878 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff73c066f60 0x7ff73c069420 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.979+0000 7ff76059e640 1 --2- 172.21.15.60:0/1374945878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75815ab30 0x7ff758156c10 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.979+0000 7ff76059e640 1 --2- 172.21.15.60:0/1374945878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7580d0b90 0x7ff75806ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.979+0000 7ff76059e640 1 --2- 172.21.15.60:0/1374945878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7580afc30 0x7ff758164780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.979+0000 7ff76059e640 1 -- 172.21.15.60:0/1374945878 >> 172.21.15.60:0/1374945878 conn(0x7ff7580b2360 msgr2=0x7ff758070360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.979+0000 7ff76059e640 1 -- 172.21.15.60:0/1374945878 shutdown_connections 2024-04-25T10:07:04.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:04.979+0000 7ff76059e640 1 -- 172.21.15.60:0/1374945878 wait complete. 2024-04-25T10:07:05.007 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:07:05.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.119+0000 7f295691b640 1 Processor -- start 2024-04-25T10:07:05.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.123+0000 7f295691b640 1 -- start start 2024-04-25T10:07:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.123+0000 7f295691b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2950145300 0x7f295015bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.123+0000 7f295691b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f295006b320 0x7f295006ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.123+0000 7f295691b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29500b3880 con 0x7f2950072d80 2024-04-25T10:07:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.123+0000 7f295691b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f295006faf0 con 0x7f2950145300 2024-04-25T10:07:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.123+0000 7f295691b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29500b4cd0 con 0x7f295006b320 2024-04-25T10:07:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.123+0000 7f294ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2950145300 0x7f295015bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.123+0000 7f294ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2950145300 0x7f295015bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55860/0 (socket says 172.21.15.60:55860) 2024-04-25T10:07:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.123+0000 7f294ffff640 1 -- 172.21.15.60:0/762436315 learned_addr learned my addr 172.21.15.60:0/762436315 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.123+0000 7f2954eb8640 1 --2- 172.21.15.60:0/762436315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f295006b320 0x7f295006ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.123+0000 7f294ffff640 1 -- 172.21.15.60:0/762436315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f295006b320 msgr2=0x7f295006ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.123+0000 7f294ffff640 1 --2- 172.21.15.60:0/762436315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f295006b320 0x7f295006ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.123+0000 7f294ffff640 1 -- 172.21.15.60:0/762436315 >> v1:172.21.15.100:6789/0 conn(0x7f2950072d80 legacy=0x7f295006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.123+0000 7f294ffff640 1 -- 172.21.15.60:0/762436315 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29500b4f70 con 0x7f2950145300 2024-04-25T10:07:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.123+0000 7f294ffff640 1 --2- 172.21.15.60:0/762436315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2950145300 0x7f295015bed0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2940009870 tx=0x7f294001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33128cbd41625ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f294effd640 1 -- 172.21.15.60:0/762436315 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29400aa410 con 0x7f2950145300 2024-04-25T10:07:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f294effd640 1 -- 172.21.15.60:0/762436315 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29400af070 con 0x7f2950145300 2024-04-25T10:07:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f294effd640 1 -- 172.21.15.60:0/762436315 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29400aa7d0 con 0x7f2950145300 2024-04-25T10:07:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f295691b640 1 -- 172.21.15.60:0/762436315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2950145300 msgr2=0x7f295015bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f295691b640 1 --2- 172.21.15.60:0/762436315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2950145300 0x7f295015bed0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2940009870 tx=0x7f294001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f295691b640 1 -- 172.21.15.60:0/762436315 shutdown_connections 2024-04-25T10:07:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f295691b640 1 --2- 172.21.15.60:0/762436315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f295006b320 0x7f295006ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f295691b640 1 --2- 172.21.15.60:0/762436315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2950145300 0x7f295015bed0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f295691b640 1 -- 172.21.15.60:0/762436315 >> 172.21.15.60:0/762436315 conn(0x7f29500b2360 msgr2=0x7f29500b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f295691b640 1 -- 172.21.15.60:0/762436315 shutdown_connections 2024-04-25T10:07:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f295691b640 1 -- 172.21.15.60:0/762436315 wait complete. 2024-04-25T10:07:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f295691b640 1 Processor -- start 2024-04-25T10:07:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f295691b640 1 -- start start 2024-04-25T10:07:05.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f295691b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295006b320 0x7f29500cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f295691b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2950072d80 0x7f295015a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f295691b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2950145300 0x7f29500c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f295691b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29500731e0 con 0x7f2950145300 2024-04-25T10:07:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f295691b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295006b950 con 0x7f295006b320 2024-04-25T10:07:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f295691b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29500ccb20 con 0x7f2950072d80 2024-04-25T10:07:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f2954eb8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2950145300 0x7f29500c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f2954eb8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2950145300 0x7f29500c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55866/0 (socket says 172.21.15.60:55866) 2024-04-25T10:07:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f2954eb8640 1 -- 172.21.15.60:0/1965241567 learned_addr learned my addr 172.21.15.60:0/1965241567 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f294f7fe640 1 --2- 172.21.15.60:0/1965241567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2950072d80 0x7f295015a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f294ffff640 1 --2- 172.21.15.60:0/1965241567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295006b320 0x7f29500cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f2954eb8640 1 -- 172.21.15.60:0/1965241567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2950072d80 msgr2=0x7f295015a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f2954eb8640 1 --2- 172.21.15.60:0/1965241567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2950072d80 0x7f295015a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f2954eb8640 1 -- 172.21.15.60:0/1965241567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295006b320 msgr2=0x7f29500cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f2954eb8640 1 --2- 172.21.15.60:0/1965241567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295006b320 0x7f29500cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f2954eb8640 1 -- 172.21.15.60:0/1965241567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29500cf8b0 con 0x7f2950145300 2024-04-25T10:07:05.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.127+0000 7f2954eb8640 1 --2- 172.21.15.60:0/1965241567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2950145300 0x7f29500c9fd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2948007e40 tx=0x7f294800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.131+0000 7f294cff9640 1 -- 172.21.15.60:0/1965241567 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948014620 con 0x7f2950145300 2024-04-25T10:07:05.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.131+0000 7f294cff9640 1 -- 172.21.15.60:0/1965241567 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2948019070 con 0x7f2950145300 2024-04-25T10:07:05.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.131+0000 7f295691b640 1 -- 172.21.15.60:0/1965241567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29500cc840 con 0x7f2950145300 2024-04-25T10:07:05.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.131+0000 7f294cff9640 1 -- 172.21.15.60:0/1965241567 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948014a20 con 0x7f2950145300 2024-04-25T10:07:05.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.131+0000 7f295691b640 1 -- 172.21.15.60:0/1965241567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29500b0650 con 0x7f2950145300 2024-04-25T10:07:05.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.131+0000 7f295691b640 1 -- 172.21.15.60:0/1965241567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295006ccb0 con 0x7f2950145300 2024-04-25T10:07:05.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.131+0000 7f294cff9640 1 -- 172.21.15.60:0/1965241567 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2948014bc0 con 0x7f2950145300 2024-04-25T10:07:05.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.131+0000 7f294cff9640 1 --2- 172.21.15.60:0/1965241567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f293c066f60 0x7f293c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.131+0000 7f294cff9640 1 -- 172.21.15.60:0/1965241567 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f29480855b0 con 0x7f2950145300 2024-04-25T10:07:05.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.131+0000 7f294ffff640 1 --2- 172.21.15.60:0/1965241567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f293c066f60 0x7f293c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.131+0000 7f294ffff640 1 --2- 172.21.15.60:0/1965241567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f293c066f60 0x7f293c069420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f29400ae000 tx=0x7f29400a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.135+0000 7f294cff9640 1 -- 172.21.15.60:0/1965241567 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2948056920 con 0x7f2950145300 2024-04-25T10:07:05.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.295+0000 7f295691b640 1 -- 172.21.15.60:0/1965241567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f295006ad90 con 0x7f2950145300 2024-04-25T10:07:05.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.295+0000 7f294cff9640 1 -- 172.21.15.60:0/1965241567 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2948056220 con 0x7f2950145300 2024-04-25T10:07:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:05.303 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T10:07:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T10:07:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T10:07:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:05.304 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T10:07:05.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.299+0000 7f295691b640 1 -- 172.21.15.60:0/1965241567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f293c066f60 msgr2=0x7f293c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.299+0000 7f295691b640 1 --2- 172.21.15.60:0/1965241567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f293c066f60 0x7f293c069420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f29400ae000 tx=0x7f29400a7000 comp rx=0 tx=0).stop 2024-04-25T10:07:05.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.299+0000 7f295691b640 1 -- 172.21.15.60:0/1965241567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2950145300 msgr2=0x7f29500c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.299+0000 7f295691b640 1 --2- 172.21.15.60:0/1965241567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2950145300 0x7f29500c9fd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2948007e40 tx=0x7f294800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.299+0000 7f295691b640 1 -- 172.21.15.60:0/1965241567 shutdown_connections 2024-04-25T10:07:05.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.299+0000 7f295691b640 1 --2- 172.21.15.60:0/1965241567 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f293c066f60 0x7f293c069420 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.299+0000 7f295691b640 1 --2- 172.21.15.60:0/1965241567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2950145300 0x7f29500c9fd0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.299+0000 7f295691b640 1 --2- 172.21.15.60:0/1965241567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2950072d80 0x7f295015a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.299+0000 7f295691b640 1 --2- 172.21.15.60:0/1965241567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f295006b320 0x7f29500cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.299+0000 7f295691b640 1 -- 172.21.15.60:0/1965241567 >> 172.21.15.60:0/1965241567 conn(0x7f29500b2360 msgr2=0x7f29500cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.299+0000 7f295691b640 1 -- 172.21.15.60:0/1965241567 shutdown_connections 2024-04-25T10:07:05.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.299+0000 7f295691b640 1 -- 172.21.15.60:0/1965241567 wait complete. 2024-04-25T10:07:05.322 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:07:05.323 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:07:05.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e4a912640 1 Processor -- start 2024-04-25T10:07:05.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e4a912640 1 -- start start 2024-04-25T10:07:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e4a912640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8e440cbc10 0x7f8e440cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e4a912640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e440cfda0 con 0x7f8e4415a680 2024-04-25T10:07:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e4a912640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e440b2670 con 0x7f8e440cf9c0 2024-04-25T10:07:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e4a912640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e440b2950 con 0x7f8e440cbc10 2024-04-25T10:07:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e43fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8e4415a680 0x7f8e440b3770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46150/0 (socket says 172.21.15.60:46150) 2024-04-25T10:07:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e43fff640 1 -- 172.21.15.60:0/4078964290 learned_addr learned my addr 172.21.15.60:0/4078964290 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:05.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e437fe640 1 --2- 172.21.15.60:0/4078964290 >> v2:172.21.15.100:3300/0 conn(0x7f8e440cbc10 0x7f8e440cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e42ffd640 1 -- 172.21.15.60:0/4078964290 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890284598 0 0) 0x7f8e440cfda0 con 0x7f8e4415a680 2024-04-25T10:07:05.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e42ffd640 1 -- 172.21.15.60:0/4078964290 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e2c005740 con 0x7f8e4415a680 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e42ffd640 1 -- 172.21.15.60:0/4078964290 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1953253530 0 0) 0x7f8e2c005740 con 0x7f8e4415a680 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e42ffd640 1 -- 172.21.15.60:0/4078964290 >> v2:172.21.15.100:3300/0 conn(0x7f8e440cbc10 msgr2=0x7f8e440cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e42ffd640 1 --2- 172.21.15.60:0/4078964290 >> v2:172.21.15.100:3300/0 conn(0x7f8e440cbc10 0x7f8e440cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e42ffd640 1 -- 172.21.15.60:0/4078964290 >> v1:172.21.15.100:6789/0 conn(0x7f8e440cf9c0 legacy=0x7f8e4406d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e42ffd640 1 -- 172.21.15.60:0/4078964290 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e440b2c30 con 0x7f8e4415a680 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e42ffd640 1 -- 172.21.15.60:0/4078964290 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f8e34001680 con 0x7f8e4415a680 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e42ffd640 1 -- 172.21.15.60:0/4078964290 >> v1:172.21.15.60:6789/0 conn(0x7f8e4415a680 legacy=0x7f8e440b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.439+0000 7f8e42ffd640 1 --2- 172.21.15.60:0/4078964290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e2c005bd0 0x7f8e2c005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e42ffd640 1 --2- 172.21.15.60:0/4078964290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e2c006a20 0x7f8e2c006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e42ffd640 1 --2- 172.21.15.60:0/4078964290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c0073e0 0x7f8e2c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e42ffd640 1 -- 172.21.15.60:0/4078964290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440b2950 con 0x7f8e2c0073e0 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e42ffd640 1 -- 172.21.15.60:0/4078964290 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c00e280 con 0x7f8e2c006a20 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e42ffd640 1 -- 172.21.15.60:0/4078964290 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c00e480 con 0x7f8e2c005bd0 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e48eaf640 1 --2- 172.21.15.60:0/4078964290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c0073e0 0x7f8e2c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e437fe640 1 --2- 172.21.15.60:0/4078964290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e2c006a20 0x7f8e2c006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e43fff640 1 --2- 172.21.15.60:0/4078964290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e2c005bd0 0x7f8e2c005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e48eaf640 1 -- 172.21.15.60:0/4078964290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e2c005bd0 msgr2=0x7f8e2c005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e48eaf640 1 --2- 172.21.15.60:0/4078964290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e2c005bd0 0x7f8e2c005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e48eaf640 1 -- 172.21.15.60:0/4078964290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e2c006a20 msgr2=0x7f8e2c006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e48eaf640 1 --2- 172.21.15.60:0/4078964290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e2c006a20 0x7f8e2c006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e48eaf640 1 -- 172.21.15.60:0/4078964290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c005740 con 0x7f8e2c0073e0 2024-04-25T10:07:05.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e48eaf640 1 -- 172.21.15.60:0/4078964290 reap_dead start 2024-04-25T10:07:05.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e48eaf640 1 --2- 172.21.15.60:0/4078964290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c0073e0 0x7f8e2c00dbe0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c001e70 tx=0x7f8e3c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8de3e1daa24000 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e42ffd640 1 -- 172.21.15.60:0/4078964290 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c001a90 con 0x7f8e2c0073e0 2024-04-25T10:07:05.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e42ffd640 1 -- 172.21.15.60:0/4078964290 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c003bc0 con 0x7f8e2c0073e0 2024-04-25T10:07:05.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e42ffd640 1 -- 172.21.15.60:0/4078964290 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c005020 con 0x7f8e2c0073e0 2024-04-25T10:07:05.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e4a912640 1 -- 172.21.15.60:0/4078964290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c0073e0 msgr2=0x7f8e2c00dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e4a912640 1 --2- 172.21.15.60:0/4078964290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c0073e0 0x7f8e2c00dbe0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c001e70 tx=0x7f8e3c01c780 comp rx=0 tx=0).stop 2024-04-25T10:07:05.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e4a912640 1 -- 172.21.15.60:0/4078964290 shutdown_connections 2024-04-25T10:07:05.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e4a912640 1 --2- 172.21.15.60:0/4078964290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c0073e0 0x7f8e2c00dbe0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e4a912640 1 -- 172.21.15.60:0/4078964290 >> 172.21.15.60:0/4078964290 conn(0x7f8e44066fa0 msgr2=0x7f8e4406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e4a912640 1 -- 172.21.15.60:0/4078964290 shutdown_connections 2024-04-25T10:07:05.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.443+0000 7f8e4a912640 1 -- 172.21.15.60:0/4078964290 wait complete. 2024-04-25T10:07:05.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e4a912640 1 Processor -- start 2024-04-25T10:07:05.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e4a912640 1 -- start start 2024-04-25T10:07:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e4a912640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c0073e0 0x7f8e441f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e4a912640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e4414fb20 0x7f8e4414c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e4a912640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e441526d0 0x7f8e44158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e4a912640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4415aa60 con 0x7f8e2c0073e0 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e4a912640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440b31f0 con 0x7f8e4414fb20 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e4a912640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440cd4f0 con 0x7f8e441526d0 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e43fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c0073e0 0x7f8e441f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e43fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c0073e0 0x7f8e441f4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55876/0 (socket says 172.21.15.60:55876) 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e43fff640 1 -- 172.21.15.60:0/2575449147 learned_addr learned my addr 172.21.15.60:0/2575449147 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e437fe640 1 --2- 172.21.15.60:0/2575449147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e4414fb20 0x7f8e4414c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e48eaf640 1 --2- 172.21.15.60:0/2575449147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e441526d0 0x7f8e44158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e43fff640 1 -- 172.21.15.60:0/2575449147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e441526d0 msgr2=0x7f8e44158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e43fff640 1 --2- 172.21.15.60:0/2575449147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e441526d0 0x7f8e44158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e43fff640 1 -- 172.21.15.60:0/2575449147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e4414fb20 msgr2=0x7f8e4414c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e43fff640 1 --2- 172.21.15.60:0/2575449147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e4414fb20 0x7f8e4414c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e43fff640 1 -- 172.21.15.60:0/2575449147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e44153000 con 0x7f8e2c0073e0 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e437fe640 1 --2- 172.21.15.60:0/2575449147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e4414fb20 0x7f8e4414c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:05.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e48eaf640 1 --2- 172.21.15.60:0/2575449147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e441526d0 0x7f8e44158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:05.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e43fff640 1 --2- 172.21.15.60:0/2575449147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c0073e0 0x7f8e441f4430 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400e700 tx=0x7f8e3400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e40ff9640 1 -- 172.21.15.60:0/2575449147 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400c6c0 con 0x7f8e2c0073e0 2024-04-25T10:07:05.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e40ff9640 1 -- 172.21.15.60:0/2575449147 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e34018070 con 0x7f8e2c0073e0 2024-04-25T10:07:05.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e4a912640 1 -- 172.21.15.60:0/2575449147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e441f3950 con 0x7f8e2c0073e0 2024-04-25T10:07:05.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e40ff9640 1 -- 172.21.15.60:0/2575449147 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400caa0 con 0x7f8e2c0073e0 2024-04-25T10:07:05.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e4a912640 1 -- 172.21.15.60:0/2575449147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e441f3f10 con 0x7f8e2c0073e0 2024-04-25T10:07:05.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e4a912640 1 -- 172.21.15.60:0/2575449147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e44074770 con 0x7f8e2c0073e0 2024-04-25T10:07:05.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.447+0000 7f8e40ff9640 1 -- 172.21.15.60:0/2575449147 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8e3400cc40 con 0x7f8e2c0073e0 2024-04-25T10:07:05.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.451+0000 7f8e40ff9640 1 --2- 172.21.15.60:0/2575449147 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8e30066f90 0x7f8e30069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.451+0000 7f8e40ff9640 1 -- 172.21.15.60:0/2575449147 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8e340847b0 con 0x7f8e2c0073e0 2024-04-25T10:07:05.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.451+0000 7f8e437fe640 1 --2- 172.21.15.60:0/2575449147 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8e30066f90 0x7f8e30069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.451+0000 7f8e437fe640 1 --2- 172.21.15.60:0/2575449147 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8e30066f90 0x7f8e30069450 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8e4414f1b0 tx=0x7f8e38008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.455+0000 7f8e40ff9640 1 -- 172.21.15.60:0/2575449147 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8e34055b10 con 0x7f8e2c0073e0 2024-04-25T10:07:05.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:05.611+0000 7f8e4a912640 1 -- 172.21.15.60:0/2575449147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8e4406e9a0 con 0x7f8e2c0073e0 2024-04-25T10:07:06.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.603+0000 7f8e40ff9640 1 -- 172.21.15.60:0/2575449147 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8e34055410 con 0x7f8e2c0073e0 2024-04-25T10:07:06.612 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T10:07:06.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.607+0000 7f8e4a912640 1 -- 172.21.15.60:0/2575449147 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8e30066f90 msgr2=0x7f8e30069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.607+0000 7f8e4a912640 1 --2- 172.21.15.60:0/2575449147 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8e30066f90 0x7f8e30069450 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8e4414f1b0 tx=0x7f8e38008040 comp rx=0 tx=0).stop 2024-04-25T10:07:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.607+0000 7f8e4a912640 1 -- 172.21.15.60:0/2575449147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c0073e0 msgr2=0x7f8e441f4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.607+0000 7f8e4a912640 1 --2- 172.21.15.60:0/2575449147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c0073e0 0x7f8e441f4430 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400e700 tx=0x7f8e3400ebc0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.607+0000 7f8e4a912640 1 -- 172.21.15.60:0/2575449147 shutdown_connections 2024-04-25T10:07:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.607+0000 7f8e4a912640 1 --2- 172.21.15.60:0/2575449147 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8e30066f90 0x7f8e30069450 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.607+0000 7f8e4a912640 1 --2- 172.21.15.60:0/2575449147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e441526d0 0x7f8e44158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.607+0000 7f8e4a912640 1 --2- 172.21.15.60:0/2575449147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e4414fb20 0x7f8e4414c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.607+0000 7f8e4a912640 1 --2- 172.21.15.60:0/2575449147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e2c0073e0 0x7f8e441f4430 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.607+0000 7f8e4a912640 1 -- 172.21.15.60:0/2575449147 >> 172.21.15.60:0/2575449147 conn(0x7f8e44066fa0 msgr2=0x7f8e440ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:06.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.611+0000 7f8e4a912640 1 -- 172.21.15.60:0/2575449147 shutdown_connections 2024-04-25T10:07:06.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.611+0000 7f8e4a912640 1 -- 172.21.15.60:0/2575449147 wait complete. 2024-04-25T10:07:06.641 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:07:06.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347f8a9640 1 Processor -- start 2024-04-25T10:07:06.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347f8a9640 1 -- start start 2024-04-25T10:07:06.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347f8a9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34780aee30 0x7f34780b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347f8a9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34780ceb90 con 0x7f34780b3020 2024-04-25T10:07:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347f8a9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34780cee70 con 0x7f347815f070 2024-04-25T10:07:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347f8a9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34780af210 con 0x7f34780aee30 2024-04-25T10:07:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347ce44640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f34780b3020 0x7f347815e960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46152/0 (socket says 172.21.15.60:46152) 2024-04-25T10:07:06.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347ce44640 1 -- 172.21.15.60:0/1443923945 learned_addr learned my addr 172.21.15.60:0/1443923945 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347d645640 1 --2- 172.21.15.60:0/1443923945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34780aee30 0x7f34780b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f346ffff640 1 -- 172.21.15.60:0/1443923945 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1748855046 0 0) 0x7f34780ceb90 con 0x7f34780b3020 2024-04-25T10:07:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f346ffff640 1 -- 172.21.15.60:0/1443923945 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3464003530 con 0x7f34780b3020 2024-04-25T10:07:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f346ffff640 1 -- 172.21.15.60:0/1443923945 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 434828220 0 0) 0x7f3464003530 con 0x7f34780b3020 2024-04-25T10:07:06.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f346ffff640 1 -- 172.21.15.60:0/1443923945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34780aee30 msgr2=0x7f34780b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f346ffff640 1 --2- 172.21.15.60:0/1443923945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34780aee30 0x7f34780b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f346ffff640 1 -- 172.21.15.60:0/1443923945 >> v1:172.21.15.100:6789/0 conn(0x7f347815f070 legacy=0x7f347806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f346ffff640 1 -- 172.21.15.60:0/1443923945 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34780cf150 con 0x7f34780b3020 2024-04-25T10:07:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f346ffff640 1 -- 172.21.15.60:0/1443923945 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f3468001680 con 0x7f34780b3020 2024-04-25T10:07:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f346ffff640 1 -- 172.21.15.60:0/1443923945 >> v1:172.21.15.60:6789/0 conn(0x7f34780b3020 legacy=0x7f347815e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f346ffff640 1 --2- 172.21.15.60:0/1443923945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34780aee30 0x7f34640039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f346ffff640 1 --2- 172.21.15.60:0/1443923945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3464003f30 0x7f3464004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f346ffff640 1 --2- 172.21.15.60:0/1443923945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3464004d50 0x7f34640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f346ffff640 1 -- 172.21.15.60:0/1443923945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34780af210 con 0x7f3464004d50 2024-04-25T10:07:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f346ffff640 1 -- 172.21.15.60:0/1443923945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346400ba30 con 0x7f3464003f30 2024-04-25T10:07:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f346ffff640 1 -- 172.21.15.60:0/1443923945 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346400bc30 con 0x7f34780aee30 2024-04-25T10:07:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347de46640 1 --2- 172.21.15.60:0/1443923945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3464004d50 0x7f34640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347ce44640 1 --2- 172.21.15.60:0/1443923945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3464003f30 0x7f3464004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347d645640 1 --2- 172.21.15.60:0/1443923945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34780aee30 0x7f34640039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347de46640 1 -- 172.21.15.60:0/1443923945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34780aee30 msgr2=0x7f34640039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347de46640 1 --2- 172.21.15.60:0/1443923945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34780aee30 0x7f34640039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347de46640 1 -- 172.21.15.60:0/1443923945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3464003f30 msgr2=0x7f3464004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:06.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347de46640 1 --2- 172.21.15.60:0/1443923945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3464003f30 0x7f3464004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347de46640 1 -- 172.21.15.60:0/1443923945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3464003530 con 0x7f3464004d50 2024-04-25T10:07:06.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.771+0000 7f347ce44640 1 --2- 172.21.15.60:0/1443923945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3464003f30 0x7f3464004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:06.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.775+0000 7f347de46640 1 --2- 172.21.15.60:0/1443923945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3464004d50 0x7f34640051d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f34780cc5c0 tx=0x7f347001c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45617814eea1463 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.775+0000 7f346ffff640 1 -- 172.21.15.60:0/1443923945 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34700b2070 con 0x7f3464004d50 2024-04-25T10:07:06.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.775+0000 7f346ffff640 1 -- 172.21.15.60:0/1443923945 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34700a5710 con 0x7f3464004d50 2024-04-25T10:07:06.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.775+0000 7f346ffff640 1 -- 172.21.15.60:0/1443923945 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470001b60 con 0x7f3464004d50 2024-04-25T10:07:06.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.775+0000 7f347f8a9640 1 -- 172.21.15.60:0/1443923945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3464004d50 msgr2=0x7f34640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.775+0000 7f347f8a9640 1 --2- 172.21.15.60:0/1443923945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3464004d50 0x7f34640051d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f34780cc5c0 tx=0x7f347001c820 comp rx=0 tx=0).stop 2024-04-25T10:07:06.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.775+0000 7f347de46640 1 -- 172.21.15.60:0/1443923945 reap_dead start 2024-04-25T10:07:06.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.775+0000 7f347f8a9640 1 -- 172.21.15.60:0/1443923945 shutdown_connections 2024-04-25T10:07:06.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.775+0000 7f347f8a9640 1 -- 172.21.15.60:0/1443923945 >> 172.21.15.60:0/1443923945 conn(0x7f3478067060 msgr2=0x7f347806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:06.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.775+0000 7f347f8a9640 1 -- 172.21.15.60:0/1443923945 shutdown_connections 2024-04-25T10:07:06.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.775+0000 7f347f8a9640 1 -- 172.21.15.60:0/1443923945 wait complete. 2024-04-25T10:07:06.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.775+0000 7f347f8a9640 1 Processor -- start 2024-04-25T10:07:06.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.775+0000 7f347f8a9640 1 -- start start 2024-04-25T10:07:06.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347f8a9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f347815d2f0 0x7f34781f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347f8a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34780adde0 0x7f34780afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347f8a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34780b02a0 0x7f34781f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347f8a9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34780ce7b0 con 0x7f34780adde0 2024-04-25T10:07:06.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347f8a9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34781442f0 con 0x7f34780b02a0 2024-04-25T10:07:06.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347f8a9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34780ce930 con 0x7f347815d2f0 2024-04-25T10:07:06.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347ce44640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34780adde0 0x7f34780afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347ce44640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34780adde0 0x7f34780afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55908/0 (socket says 172.21.15.60:55908) 2024-04-25T10:07:06.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347ce44640 1 -- 172.21.15.60:0/1903530669 learned_addr learned my addr 172.21.15.60:0/1903530669 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:06.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347de46640 1 --2- 172.21.15.60:0/1903530669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34780b02a0 0x7f34781f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347d645640 1 --2- 172.21.15.60:0/1903530669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f347815d2f0 0x7f34781f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347ce44640 1 -- 172.21.15.60:0/1903530669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f347815d2f0 msgr2=0x7f34781f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347ce44640 1 --2- 172.21.15.60:0/1903530669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f347815d2f0 0x7f34781f1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347ce44640 1 -- 172.21.15.60:0/1903530669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34780b02a0 msgr2=0x7f34781f4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347ce44640 1 --2- 172.21.15.60:0/1903530669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34780b02a0 0x7f34781f4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347ce44640 1 -- 172.21.15.60:0/1903530669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34780ae510 con 0x7f34780adde0 2024-04-25T10:07:06.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347ce44640 1 --2- 172.21.15.60:0/1903530669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34780adde0 0x7f34780afd60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f34680118a0 tx=0x7f3468011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f346dffb640 1 -- 172.21.15.60:0/1903530669 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468010040 con 0x7f34780adde0 2024-04-25T10:07:06.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f346dffb640 1 -- 172.21.15.60:0/1903530669 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346801a070 con 0x7f34780adde0 2024-04-25T10:07:06.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f346dffb640 1 -- 172.21.15.60:0/1903530669 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346800fd00 con 0x7f34780adde0 2024-04-25T10:07:06.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347f8a9640 1 -- 172.21.15.60:0/1903530669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34780ae260 con 0x7f34780adde0 2024-04-25T10:07:06.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347f8a9640 1 -- 172.21.15.60:0/1903530669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34781f2900 con 0x7f34780adde0 2024-04-25T10:07:06.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347f8a9640 1 -- 172.21.15.60:0/1903530669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3478074770 con 0x7f34780adde0 2024-04-25T10:07:06.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f346dffb640 1 -- 172.21.15.60:0/1903530669 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f346801f460 con 0x7f34780adde0 2024-04-25T10:07:06.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f346dffb640 1 --2- 172.21.15.60:0/1903530669 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f345c066f60 0x7f345c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f346dffb640 1 -- 172.21.15.60:0/1903530669 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f3468088070 con 0x7f34780adde0 2024-04-25T10:07:06.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.779+0000 7f347d645640 1 --2- 172.21.15.60:0/1903530669 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f345c066f60 0x7f345c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.783+0000 7f347d645640 1 --2- 172.21.15.60:0/1903530669 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f345c066f60 0x7f345c069420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3474009a30 tx=0x7f3474008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.783+0000 7f346dffb640 1 -- 172.21.15.60:0/1903530669 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3468059260 con 0x7f34780adde0 2024-04-25T10:07:06.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:06.939+0000 7f347f8a9640 1 -- 172.21.15.60:0/1903530669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f34780b0680 con 0x7f34780adde0 2024-04-25T10:07:07.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.607+0000 7f346dffb640 1 -- 172.21.15.60:0/1903530669 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3468058b60 con 0x7f34780adde0 2024-04-25T10:07:07.616 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T10:07:07.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.611+0000 7f347f8a9640 1 -- 172.21.15.60:0/1903530669 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f345c066f60 msgr2=0x7f345c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.611+0000 7f347f8a9640 1 --2- 172.21.15.60:0/1903530669 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f345c066f60 0x7f345c069420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3474009a30 tx=0x7f3474008040 comp rx=0 tx=0).stop 2024-04-25T10:07:07.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.611+0000 7f347f8a9640 1 -- 172.21.15.60:0/1903530669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34780adde0 msgr2=0x7f34780afd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.611+0000 7f347f8a9640 1 --2- 172.21.15.60:0/1903530669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34780adde0 0x7f34780afd60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f34680118a0 tx=0x7f3468011d60 comp rx=0 tx=0).stop 2024-04-25T10:07:07.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.611+0000 7f347f8a9640 1 -- 172.21.15.60:0/1903530669 shutdown_connections 2024-04-25T10:07:07.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.611+0000 7f347f8a9640 1 --2- 172.21.15.60:0/1903530669 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f345c066f60 0x7f345c069420 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.611+0000 7f347f8a9640 1 --2- 172.21.15.60:0/1903530669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34780b02a0 0x7f34781f4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.611+0000 7f347f8a9640 1 --2- 172.21.15.60:0/1903530669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34780adde0 0x7f34780afd60 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.611+0000 7f347f8a9640 1 --2- 172.21.15.60:0/1903530669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f347815d2f0 0x7f34781f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.611+0000 7f347f8a9640 1 -- 172.21.15.60:0/1903530669 >> 172.21.15.60:0/1903530669 conn(0x7f3478067060 msgr2=0x7f347806e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.611+0000 7f347f8a9640 1 -- 172.21.15.60:0/1903530669 shutdown_connections 2024-04-25T10:07:07.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.611+0000 7f347f8a9640 1 -- 172.21.15.60:0/1903530669 wait complete. 2024-04-25T10:07:07.656 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:07:07.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.771+0000 7f935f760640 1 Processor -- start 2024-04-25T10:07:07.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.771+0000 7f935f760640 1 -- start start 2024-04-25T10:07:07.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.771+0000 7f935f760640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f935815dd30 0x7f935815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.771+0000 7f935f760640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93580cd3d0 0x7f93580cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.771+0000 7f935f760640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93580cd890 con 0x7f935806f310 2024-04-25T10:07:07.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.771+0000 7f935f760640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93580ce4a0 con 0x7f935815dd30 2024-04-25T10:07:07.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.771+0000 7f935f760640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93580cdb70 con 0x7f93580cd3d0 2024-04-25T10:07:07.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.771+0000 7f935dcfd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f935806f310 0x7f935806d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46156/0 (socket says 172.21.15.60:46156) 2024-04-25T10:07:07.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f935dcfd640 1 -- 172.21.15.60:0/3435494451 learned_addr learned my addr 172.21.15.60:0/3435494451 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:07.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f935d4fc640 1 --2- 172.21.15.60:0/3435494451 >> v2:172.21.15.100:3300/0 conn(0x7f935815dd30 0x7f935815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f935ccfb640 1 --2- 172.21.15.60:0/3435494451 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93580cd3d0 0x7f93580cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f934ffff640 1 -- 172.21.15.60:0/3435494451 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732112258 0 0) 0x7f93580cd890 con 0x7f935806f310 2024-04-25T10:07:07.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f934ffff640 1 -- 172.21.15.60:0/3435494451 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9344003530 con 0x7f935806f310 2024-04-25T10:07:07.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f934ffff640 1 -- 172.21.15.60:0/3435494451 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2562802225 0 0) 0x7f9344003530 con 0x7f935806f310 2024-04-25T10:07:07.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f935d4fc640 1 -- 172.21.15.60:0/3435494451 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93580cd3d0 msgr2=0x7f93580cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f935d4fc640 1 --2- 172.21.15.60:0/3435494451 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93580cd3d0 0x7f93580cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f935d4fc640 1 -- 172.21.15.60:0/3435494451 >> v1:172.21.15.60:6789/0 conn(0x7f935806f310 legacy=0x7f935806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f935d4fc640 1 -- 172.21.15.60:0/3435494451 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93580cde50 con 0x7f935815dd30 2024-04-25T10:07:07.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f935d4fc640 1 --2- 172.21.15.60:0/3435494451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935815dd30 0x7f935815ac30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f93580ce620 tx=0x7f935401c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6886e5453e524868 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f934ffff640 1 -- 172.21.15.60:0/3435494451 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93540b0070 con 0x7f935815dd30 2024-04-25T10:07:07.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f934ffff640 1 -- 172.21.15.60:0/3435494451 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93540ab400 con 0x7f935815dd30 2024-04-25T10:07:07.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f934ffff640 1 -- 172.21.15.60:0/3435494451 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350001a00 con 0x7f935815dd30 2024-04-25T10:07:07.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f935f760640 1 -- 172.21.15.60:0/3435494451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935815dd30 msgr2=0x7f935815ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f935f760640 1 --2- 172.21.15.60:0/3435494451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935815dd30 0x7f935815ac30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f93580ce620 tx=0x7f935401c300 comp rx=0 tx=0).stop 2024-04-25T10:07:07.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f935f760640 1 -- 172.21.15.60:0/3435494451 shutdown_connections 2024-04-25T10:07:07.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f935f760640 1 --2- 172.21.15.60:0/3435494451 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93580cd3d0 0x7f93580cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f935f760640 1 --2- 172.21.15.60:0/3435494451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935815dd30 0x7f935815ac30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.775+0000 7f935f760640 1 -- 172.21.15.60:0/3435494451 >> 172.21.15.60:0/3435494451 conn(0x7f93580670b0 msgr2=0x7f935806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935f760640 1 -- 172.21.15.60:0/3435494451 shutdown_connections 2024-04-25T10:07:07.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935f760640 1 -- 172.21.15.60:0/3435494451 wait complete. 2024-04-25T10:07:07.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935f760640 1 Processor -- start 2024-04-25T10:07:07.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935f760640 1 -- start start 2024-04-25T10:07:07.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935f760640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935806f310 0x7f9358153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935f760640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93580cd3d0 0x7f93580c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935f760640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f935815dd30 0x7f9358163a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935f760640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93581674b0 con 0x7f93580cd3d0 2024-04-25T10:07:07.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935f760640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93580cfae0 con 0x7f935806f310 2024-04-25T10:07:07.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935f760640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9358167630 con 0x7f935815dd30 2024-04-25T10:07:07.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935ccfb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93580cd3d0 0x7f93580c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935ccfb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93580cd3d0 0x7f93580c4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55920/0 (socket says 172.21.15.60:55920) 2024-04-25T10:07:07.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935ccfb640 1 -- 172.21.15.60:0/3274230744 learned_addr learned my addr 172.21.15.60:0/3274230744 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935d4fc640 1 --2- 172.21.15.60:0/3274230744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935806f310 0x7f9358153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935dcfd640 1 --2- 172.21.15.60:0/3274230744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f935815dd30 0x7f9358163a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935ccfb640 1 -- 172.21.15.60:0/3274230744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f935815dd30 msgr2=0x7f9358163a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935ccfb640 1 --2- 172.21.15.60:0/3274230744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f935815dd30 0x7f9358163a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935ccfb640 1 -- 172.21.15.60:0/3274230744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935806f310 msgr2=0x7f9358153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935ccfb640 1 --2- 172.21.15.60:0/3274230744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935806f310 0x7f9358153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935ccfb640 1 -- 172.21.15.60:0/3274230744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935815d3d0 con 0x7f93580cd3d0 2024-04-25T10:07:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935ccfb640 1 --2- 172.21.15.60:0/3274230744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93580cd3d0 0x7f93580c4eb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f934800e930 tx=0x7f934800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f934dffb640 1 -- 172.21.15.60:0/3274230744 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348017070 con 0x7f93580cd3d0 2024-04-25T10:07:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f934dffb640 1 -- 172.21.15.60:0/3274230744 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934800ca00 con 0x7f93580cd3d0 2024-04-25T10:07:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935f760640 1 -- 172.21.15.60:0/3274230744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93580cca00 con 0x7f93580cd3d0 2024-04-25T10:07:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f934dffb640 1 -- 172.21.15.60:0/3274230744 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934800cdb0 con 0x7f93580cd3d0 2024-04-25T10:07:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.779+0000 7f935f760640 1 -- 172.21.15.60:0/3274230744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93580b2480 con 0x7f93580cd3d0 2024-04-25T10:07:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.783+0000 7f935f760640 1 -- 172.21.15.60:0/3274230744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9358074770 con 0x7f93580cd3d0 2024-04-25T10:07:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.783+0000 7f934dffb640 1 -- 172.21.15.60:0/3274230744 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f934801a070 con 0x7f93580cd3d0 2024-04-25T10:07:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.783+0000 7f934dffb640 1 --2- 172.21.15.60:0/3274230744 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f933c066f60 0x7f933c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.783+0000 7f934dffb640 1 -- 172.21.15.60:0/3274230744 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f9348084fd0 con 0x7f93580cd3d0 2024-04-25T10:07:07.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.783+0000 7f935d4fc640 1 --2- 172.21.15.60:0/3274230744 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f933c066f60 0x7f933c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.783+0000 7f935d4fc640 1 --2- 172.21.15.60:0/3274230744 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f933c066f60 0x7f933c069420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f93540ad000 tx=0x7f93540a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.787+0000 7f934dffb640 1 -- 172.21.15.60:0/3274230744 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9348056190 con 0x7f93580cd3d0 2024-04-25T10:07:07.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:07.975+0000 7f935f760640 1 -- 172.21.15.60:0/3274230744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f93580cbe70 con 0x7f93580cd3d0 2024-04-25T10:07:08.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.643+0000 7f934dffb640 1 -- 172.21.15.60:0/3274230744 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v19) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9348055a90 con 0x7f93580cd3d0 2024-04-25T10:07:08.668 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 6 and data pool 7 2024-04-25T10:07:08.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.671+0000 7f93437fe640 1 -- 172.21.15.60:0/3274230744 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f933c066f60 msgr2=0x7f933c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.671+0000 7f93437fe640 1 --2- 172.21.15.60:0/3274230744 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f933c066f60 0x7f933c069420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f93540ad000 tx=0x7f93540a6000 comp rx=0 tx=0).stop 2024-04-25T10:07:08.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.671+0000 7f93437fe640 1 -- 172.21.15.60:0/3274230744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93580cd3d0 msgr2=0x7f93580c4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.671+0000 7f93437fe640 1 --2- 172.21.15.60:0/3274230744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93580cd3d0 0x7f93580c4eb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f934800e930 tx=0x7f934800edf0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.675+0000 7f93437fe640 1 -- 172.21.15.60:0/3274230744 shutdown_connections 2024-04-25T10:07:08.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.675+0000 7f93437fe640 1 --2- 172.21.15.60:0/3274230744 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f933c066f60 0x7f933c069420 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.675+0000 7f93437fe640 1 --2- 172.21.15.60:0/3274230744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f935815dd30 0x7f9358163a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.675+0000 7f93437fe640 1 --2- 172.21.15.60:0/3274230744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93580cd3d0 0x7f93580c4eb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.675+0000 7f93437fe640 1 --2- 172.21.15.60:0/3274230744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935806f310 0x7f9358153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.675+0000 7f93437fe640 1 -- 172.21.15.60:0/3274230744 >> 172.21.15.60:0/3274230744 conn(0x7f93580670b0 msgr2=0x7f9358071b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.675+0000 7f93437fe640 1 -- 172.21.15.60:0/3274230744 shutdown_connections 2024-04-25T10:07:08.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.675+0000 7f93437fe640 1 -- 172.21.15.60:0/3274230744 wait complete. 2024-04-25T10:07:08.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:08.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.807+0000 7fc9cf75b640 1 Processor -- start 2024-04-25T10:07:08.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cf75b640 1 -- start start 2024-04-25T10:07:08.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cf75b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc9c80ce7a0 0x7fc9c80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cf75b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9c815bc90 0x7fc9c80b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cf75b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c80cf0c0 con 0x7fc9c80b4660 2024-04-25T10:07:08.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cf75b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c80cc370 con 0x7fc9c80ce7a0 2024-04-25T10:07:08.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cf75b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c8071ca0 con 0x7fc9c815bc90 2024-04-25T10:07:08.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cdcf8640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc9c80b4660 0x7fc9c80b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46166/0 (socket says 172.21.15.60:46166) 2024-04-25T10:07:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cdcf8640 1 -- 172.21.15.60:0/1640214263 learned_addr learned my addr 172.21.15.60:0/1640214263 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cd4f7640 1 --2- 172.21.15.60:0/1640214263 >> v2:172.21.15.100:3300/0 conn(0x7fc9c80ce7a0 0x7fc9c80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9bffff640 1 -- 172.21.15.60:0/1640214263 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473830230 0 0) 0x7fc9c80cf0c0 con 0x7fc9c80b4660 2024-04-25T10:07:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cccf6640 1 --2- 172.21.15.60:0/1640214263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9c815bc90 0x7fc9c80b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9bffff640 1 -- 172.21.15.60:0/1640214263 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9b4003530 con 0x7fc9c80b4660 2024-04-25T10:07:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9bffff640 1 -- 172.21.15.60:0/1640214263 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2191803833 0 0) 0x7fc9b4003530 con 0x7fc9c80b4660 2024-04-25T10:07:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9bffff640 1 -- 172.21.15.60:0/1640214263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9c815bc90 msgr2=0x7fc9c80b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9bffff640 1 --2- 172.21.15.60:0/1640214263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9c815bc90 0x7fc9c80b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9bffff640 1 -- 172.21.15.60:0/1640214263 >> v2:172.21.15.100:3300/0 conn(0x7fc9c80ce7a0 msgr2=0x7fc9c80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9bffff640 1 --2- 172.21.15.60:0/1640214263 >> v2:172.21.15.100:3300/0 conn(0x7fc9c80ce7a0 0x7fc9c80ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9bffff640 1 -- 172.21.15.60:0/1640214263 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c80cca40 con 0x7fc9c80b4660 2024-04-25T10:07:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9bffff640 1 -- 172.21.15.60:0/1640214263 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fc9c0001a00 con 0x7fc9c80b4660 2024-04-25T10:07:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9bffff640 1 -- 172.21.15.60:0/1640214263 >> v1:172.21.15.60:6789/0 conn(0x7fc9c80b4660 legacy=0x7fc9c80b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9bffff640 1 --2- 172.21.15.60:0/1640214263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b40039c0 0x7fc9b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9bffff640 1 --2- 172.21.15.60:0/1640214263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b4004810 0x7fc9b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9bffff640 1 --2- 172.21.15.60:0/1640214263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9c815bc90 0x7fc9b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9bffff640 1 -- 172.21.15.60:0/1640214263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c8071ca0 con 0x7fc9b40039c0 2024-04-25T10:07:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9bffff640 1 -- 172.21.15.60:0/1640214263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b400ba80 con 0x7fc9b4004810 2024-04-25T10:07:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9bffff640 1 -- 172.21.15.60:0/1640214263 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b400bc30 con 0x7fc9c815bc90 2024-04-25T10:07:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cd4f7640 1 --2- 172.21.15.60:0/1640214263 >> v2:172.21.15.100:3300/0 conn(0x7fc9c80ce7a0 0x7fc9c80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cd4f7640 1 --2- 172.21.15.60:0/1640214263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b40039c0 0x7fc9b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cdcf8640 1 --2- 172.21.15.60:0/1640214263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9c815bc90 0x7fc9b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cccf6640 1 --2- 172.21.15.60:0/1640214263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b4004810 0x7fc9b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cd4f7640 1 -- 172.21.15.60:0/1640214263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9c815bc90 msgr2=0x7fc9b40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cd4f7640 1 --2- 172.21.15.60:0/1640214263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9c815bc90 0x7fc9b40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cd4f7640 1 -- 172.21.15.60:0/1640214263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b4004810 msgr2=0x7fc9b4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cd4f7640 1 --2- 172.21.15.60:0/1640214263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b4004810 0x7fc9b4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cd4f7640 1 -- 172.21.15.60:0/1640214263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b4003530 con 0x7fc9b40039c0 2024-04-25T10:07:08.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cdcf8640 1 --2- 172.21.15.60:0/1640214263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9c815bc90 0x7fc9b40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:08.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.811+0000 7fc9cccf6640 1 --2- 172.21.15.60:0/1640214263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b4004810 0x7fc9b4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:08.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cd4f7640 1 --2- 172.21.15.60:0/1640214263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b40039c0 0x7fc9b4003de0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009880 tx=0x7fc9c401c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92091724ddabfbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9bffff640 1 -- 172.21.15.60:0/1640214263 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c400f040 con 0x7fc9b40039c0 2024-04-25T10:07:08.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9bffff640 1 -- 172.21.15.60:0/1640214263 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c4004020 con 0x7fc9b40039c0 2024-04-25T10:07:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9bffff640 1 -- 172.21.15.60:0/1640214263 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c40043f0 con 0x7fc9b40039c0 2024-04-25T10:07:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cf75b640 1 -- 172.21.15.60:0/1640214263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b40039c0 msgr2=0x7fc9b4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cf75b640 1 --2- 172.21.15.60:0/1640214263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b40039c0 0x7fc9b4003de0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009880 tx=0x7fc9c401c5d0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cdcf8640 1 -- 172.21.15.60:0/1640214263 reap_dead start 2024-04-25T10:07:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cf75b640 1 -- 172.21.15.60:0/1640214263 shutdown_connections 2024-04-25T10:07:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cf75b640 1 -- 172.21.15.60:0/1640214263 >> 172.21.15.60:0/1640214263 conn(0x7fc9c80b12d0 msgr2=0x7fc9c80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cf75b640 1 -- 172.21.15.60:0/1640214263 shutdown_connections 2024-04-25T10:07:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cf75b640 1 -- 172.21.15.60:0/1640214263 wait complete. 2024-04-25T10:07:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cf75b640 1 Processor -- start 2024-04-25T10:07:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cf75b640 1 -- start start 2024-04-25T10:07:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cf75b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b40039c0 0x7fc9c8161f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cf75b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9c8150e10 0x7fc9c8154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cf75b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9c8148b60 0x7fc9c81568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cf75b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c80b4280 con 0x7fc9c8150e10 2024-04-25T10:07:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cf75b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c80b3db0 con 0x7fc9b40039c0 2024-04-25T10:07:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cf75b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c80b4400 con 0x7fc9c8148b60 2024-04-25T10:07:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cccf6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9c8150e10 0x7fc9c8154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cccf6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9c8150e10 0x7fc9c8154b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55946/0 (socket says 172.21.15.60:55946) 2024-04-25T10:07:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cccf6640 1 -- 172.21.15.60:0/4047958062 learned_addr learned my addr 172.21.15.60:0/4047958062 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cdcf8640 1 --2- 172.21.15.60:0/4047958062 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9c8148b60 0x7fc9c81568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cd4f7640 1 --2- 172.21.15.60:0/4047958062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b40039c0 0x7fc9c8161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cccf6640 1 -- 172.21.15.60:0/4047958062 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9c8148b60 msgr2=0x7fc9c81568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cccf6640 1 --2- 172.21.15.60:0/4047958062 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9c8148b60 0x7fc9c81568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cccf6640 1 -- 172.21.15.60:0/4047958062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b40039c0 msgr2=0x7fc9c8161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cccf6640 1 --2- 172.21.15.60:0/4047958062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b40039c0 0x7fc9c8161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cccf6640 1 -- 172.21.15.60:0/4047958062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c8151290 con 0x7fc9c8150e10 2024-04-25T10:07:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.815+0000 7fc9cccf6640 1 --2- 172.21.15.60:0/4047958062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9c8150e10 0x7fc9c8154b80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc9b800e810 tx=0x7fc9b800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.819+0000 7fc9bdffb640 1 -- 172.21.15.60:0/4047958062 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8017070 con 0x7fc9c8150e10 2024-04-25T10:07:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.819+0000 7fc9bdffb640 1 -- 172.21.15.60:0/4047958062 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b800c670 con 0x7fc9c8150e10 2024-04-25T10:07:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.819+0000 7fc9bdffb640 1 -- 172.21.15.60:0/4047958062 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b800f040 con 0x7fc9c8150e10 2024-04-25T10:07:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.819+0000 7fc9cf75b640 1 -- 172.21.15.60:0/4047958062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c8160150 con 0x7fc9c8150e10 2024-04-25T10:07:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.819+0000 7fc9cf75b640 1 -- 172.21.15.60:0/4047958062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c8160710 con 0x7fc9c8150e10 2024-04-25T10:07:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.819+0000 7fc9cf75b640 1 -- 172.21.15.60:0/4047958062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c8074770 con 0x7fc9c8150e10 2024-04-25T10:07:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.819+0000 7fc9bdffb640 1 -- 172.21.15.60:0/4047958062 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc9b801a070 con 0x7fc9c8150e10 2024-04-25T10:07:08.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.819+0000 7fc9bdffb640 1 --2- 172.21.15.60:0/4047958062 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc9ac066f60 0x7fc9ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.819+0000 7fc9bdffb640 1 -- 172.21.15.60:0/4047958062 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9b8084be0 con 0x7fc9c8150e10 2024-04-25T10:07:08.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.819+0000 7fc9cd4f7640 1 --2- 172.21.15.60:0/4047958062 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc9ac066f60 0x7fc9ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.819+0000 7fc9cd4f7640 1 --2- 172.21.15.60:0/4047958062 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc9ac066f60 0x7fc9ac069420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40ae000 tx=0x7fc9c40046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.823+0000 7fc9bdffb640 1 -- 172.21.15.60:0/4047958062 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc9c8151290 con 0x7fc9c8150e10 2024-04-25T10:07:08.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.979+0000 7fc9cf75b640 1 -- 172.21.15.60:0/4047958062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9c80cee30 con 0x7fc9c8150e10 2024-04-25T10:07:08.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.979+0000 7fc9bdffb640 1 -- 172.21.15.60:0/4047958062 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16638 (secure 0 0 0) 0x7fc9b8055bf0 con 0x7fc9c8150e10 2024-04-25T10:07:08.986 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:08.986 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":46,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:07:07.979328+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.652298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:06.983225+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:08.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.979+0000 7fc9cf75b640 1 -- 172.21.15.60:0/4047958062 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc9ac066f60 msgr2=0x7fc9ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.979+0000 7fc9cf75b640 1 --2- 172.21.15.60:0/4047958062 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc9ac066f60 0x7fc9ac069420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40ae000 tx=0x7fc9c40046d0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.979+0000 7fc9cf75b640 1 -- 172.21.15.60:0/4047958062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9c8150e10 msgr2=0x7fc9c8154b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.979+0000 7fc9cf75b640 1 --2- 172.21.15.60:0/4047958062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9c8150e10 0x7fc9c8154b80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc9b800e810 tx=0x7fc9b800ecd0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.979+0000 7fc9cf75b640 1 -- 172.21.15.60:0/4047958062 shutdown_connections 2024-04-25T10:07:08.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.979+0000 7fc9cf75b640 1 --2- 172.21.15.60:0/4047958062 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc9ac066f60 0x7fc9ac069420 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.979+0000 7fc9cf75b640 1 --2- 172.21.15.60:0/4047958062 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9c8148b60 0x7fc9c81568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.979+0000 7fc9cf75b640 1 --2- 172.21.15.60:0/4047958062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9c8150e10 0x7fc9c8154b80 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.979+0000 7fc9cf75b640 1 --2- 172.21.15.60:0/4047958062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b40039c0 0x7fc9c8161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.979+0000 7fc9cf75b640 1 -- 172.21.15.60:0/4047958062 >> 172.21.15.60:0/4047958062 conn(0x7fc9c80b12d0 msgr2=0x7fc9c814d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.979+0000 7fc9cf75b640 1 -- 172.21.15.60:0/4047958062 shutdown_connections 2024-04-25T10:07:08.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:08.979+0000 7fc9cf75b640 1 -- 172.21.15.60:0/4047958062 wait complete. 2024-04-25T10:07:09.004 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:09.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.115+0000 7feacc360640 1 Processor -- start 2024-04-25T10:07:09.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.115+0000 7feacc360640 1 -- start start 2024-04-25T10:07:09.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feacc360640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feac40ce7a0 0x7feac40ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feacc360640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7feac40b4660 0x7feac40b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feacc360640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac40cf0c0 con 0x7feac415bc90 2024-04-25T10:07:09.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feacc360640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac40cc370 con 0x7feac40ce7a0 2024-04-25T10:07:09.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feacc360640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac4071ca0 con 0x7feac40b4660 2024-04-25T10:07:09.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feaca0fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feac40ce7a0 0x7feac40ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feaca0fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feac40ce7a0 0x7feac40ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38940/0 (socket says 172.21.15.60:38940) 2024-04-25T10:07:09.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feaca0fc640 1 -- 172.21.15.60:0/1428766594 learned_addr learned my addr 172.21.15.60:0/1428766594 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:09.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feaca8fd640 1 --2- 172.21.15.60:0/1428766594 >> v2:172.21.15.100:3300/0 conn(0x7feac40b4660 0x7feac40b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feaca0fc640 1 -- 172.21.15.60:0/1428766594 >> v2:172.21.15.100:3300/0 conn(0x7feac40b4660 msgr2=0x7feac40b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feaca0fc640 1 --2- 172.21.15.60:0/1428766594 >> v2:172.21.15.100:3300/0 conn(0x7feac40b4660 0x7feac40b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feaca0fc640 1 -- 172.21.15.60:0/1428766594 >> v1:172.21.15.60:6789/0 conn(0x7feac415bc90 legacy=0x7feac40b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feaca0fc640 1 -- 172.21.15.60:0/1428766594 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac40cca40 con 0x7feac40ce7a0 2024-04-25T10:07:09.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feaca8fd640 1 --2- 172.21.15.60:0/1428766594 >> v2:172.21.15.100:3300/0 conn(0x7feac40b4660 0x7feac40b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:09.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feaca0fc640 1 --2- 172.21.15.60:0/1428766594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feac40ce7a0 0x7feac40ceb80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7feac406cf50 tx=0x7feac001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7455ec6898d18f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feac90fa640 1 -- 172.21.15.60:0/1428766594 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac00b0070 con 0x7feac40ce7a0 2024-04-25T10:07:09.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feac90fa640 1 -- 172.21.15.60:0/1428766594 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac00ab600 con 0x7feac40ce7a0 2024-04-25T10:07:09.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.119+0000 7feac90fa640 1 -- 172.21.15.60:0/1428766594 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac00ab9d0 con 0x7feac40ce7a0 2024-04-25T10:07:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 -- 172.21.15.60:0/1428766594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feac40ce7a0 msgr2=0x7feac40ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 --2- 172.21.15.60:0/1428766594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feac40ce7a0 0x7feac40ceb80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7feac406cf50 tx=0x7feac001c520 comp rx=0 tx=0).stop 2024-04-25T10:07:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 -- 172.21.15.60:0/1428766594 shutdown_connections 2024-04-25T10:07:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 --2- 172.21.15.60:0/1428766594 >> v2:172.21.15.100:3300/0 conn(0x7feac40b4660 0x7feac40b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 --2- 172.21.15.60:0/1428766594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feac40ce7a0 0x7feac40ceb80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 -- 172.21.15.60:0/1428766594 >> 172.21.15.60:0/1428766594 conn(0x7feac40b12d0 msgr2=0x7feac40b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 -- 172.21.15.60:0/1428766594 shutdown_connections 2024-04-25T10:07:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 -- 172.21.15.60:0/1428766594 wait complete. 2024-04-25T10:07:09.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 Processor -- start 2024-04-25T10:07:09.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 -- start start 2024-04-25T10:07:09.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feac40ce7a0 0x7feac40ac510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feac415bc90 0x7feac4152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feac4154c10 0x7feac4153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac40b4400 con 0x7feac40ce7a0 2024-04-25T10:07:09.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac40b4280 con 0x7feac415bc90 2024-04-25T10:07:09.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac40b3220 con 0x7feac4154c10 2024-04-25T10:07:09.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feaca0fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feac40ce7a0 0x7feac40ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feaca0fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feac40ce7a0 0x7feac40ac510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38956/0 (socket says 172.21.15.60:38956) 2024-04-25T10:07:09.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feaca0fc640 1 -- 172.21.15.60:0/2291876799 learned_addr learned my addr 172.21.15.60:0/2291876799 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feaca0fc640 1 -- 172.21.15.60:0/2291876799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feac4154c10 msgr2=0x7feac4153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feaca0fc640 1 --2- 172.21.15.60:0/2291876799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feac4154c10 0x7feac4153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feaca0fc640 1 -- 172.21.15.60:0/2291876799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feac415bc90 msgr2=0x7feac4152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feaca0fc640 1 --2- 172.21.15.60:0/2291876799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feac415bc90 0x7feac4152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feaca0fc640 1 -- 172.21.15.60:0/2291876799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac4153360 con 0x7feac40ce7a0 2024-04-25T10:07:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feaca0fc640 1 --2- 172.21.15.60:0/2291876799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feac40ce7a0 0x7feac40ac510 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7feac001ca00 tx=0x7feac00192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feabaffd640 1 -- 172.21.15.60:0/2291876799 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac00b8020 con 0x7feac40ce7a0 2024-04-25T10:07:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feabaffd640 1 -- 172.21.15.60:0/2291876799 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac00b4630 con 0x7feac40ce7a0 2024-04-25T10:07:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 -- 172.21.15.60:0/2291876799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac40ab750 con 0x7feac40ce7a0 2024-04-25T10:07:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feabaffd640 1 -- 172.21.15.60:0/2291876799 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac00b0040 con 0x7feac40ce7a0 2024-04-25T10:07:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.123+0000 7feacc360640 1 -- 172.21.15.60:0/2291876799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac40abd10 con 0x7feac40ce7a0 2024-04-25T10:07:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.127+0000 7feacc360640 1 -- 172.21.15.60:0/2291876799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac4074770 con 0x7feac40ce7a0 2024-04-25T10:07:09.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.127+0000 7feabaffd640 1 -- 172.21.15.60:0/2291876799 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feac00bb020 con 0x7feac40ce7a0 2024-04-25T10:07:09.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.127+0000 7feabaffd640 1 --2- 172.21.15.60:0/2291876799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feaa8066f60 0x7feaa8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.127+0000 7feabaffd640 1 -- 172.21.15.60:0/2291876799 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feac011e8f0 con 0x7feac40ce7a0 2024-04-25T10:07:09.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.127+0000 7feac98fb640 1 --2- 172.21.15.60:0/2291876799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feaa8066f60 0x7feaa8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.127+0000 7feac98fb640 1 --2- 172.21.15.60:0/2291876799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feaa8066f60 0x7feaa8069420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feab4000f70 tx=0x7feab40079e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.131+0000 7feabaffd640 1 -- 172.21.15.60:0/2291876799 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feac00c2070 con 0x7feac40ce7a0 2024-04-25T10:07:09.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.283+0000 7feacc360640 1 -- 172.21.15.60:0/2291876799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feac40b3800 con 0x7feac40ce7a0 2024-04-25T10:07:09.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.287+0000 7feabaffd640 1 -- 172.21.15.60:0/2291876799 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16638 (secure 0 0 0) 0x7feac4153360 con 0x7feac40ce7a0 2024-04-25T10:07:09.294 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:09.294 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":46,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:07:07.979328+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.652298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:06.983225+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.287+0000 7feacc360640 1 -- 172.21.15.60:0/2291876799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feaa8066f60 msgr2=0x7feaa8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.287+0000 7feacc360640 1 --2- 172.21.15.60:0/2291876799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feaa8066f60 0x7feaa8069420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feab4000f70 tx=0x7feab40079e0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.287+0000 7feacc360640 1 -- 172.21.15.60:0/2291876799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feac40ce7a0 msgr2=0x7feac40ac510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.287+0000 7feacc360640 1 --2- 172.21.15.60:0/2291876799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feac40ce7a0 0x7feac40ac510 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7feac001ca00 tx=0x7feac00192c0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.287+0000 7feacc360640 1 -- 172.21.15.60:0/2291876799 shutdown_connections 2024-04-25T10:07:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.287+0000 7feacc360640 1 --2- 172.21.15.60:0/2291876799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feaa8066f60 0x7feaa8069420 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.287+0000 7feacc360640 1 --2- 172.21.15.60:0/2291876799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feac4154c10 0x7feac4153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.287+0000 7feacc360640 1 --2- 172.21.15.60:0/2291876799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feac415bc90 0x7feac4152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.287+0000 7feacc360640 1 --2- 172.21.15.60:0/2291876799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feac40ce7a0 0x7feac40ac510 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.287+0000 7feacc360640 1 -- 172.21.15.60:0/2291876799 >> 172.21.15.60:0/2291876799 conn(0x7feac40b12d0 msgr2=0x7feac415b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.287+0000 7feacc360640 1 -- 172.21.15.60:0/2291876799 shutdown_connections 2024-04-25T10:07:09.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.287+0000 7feacc360640 1 -- 172.21.15.60:0/2291876799 wait complete. 2024-04-25T10:07:09.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:07:09.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe91c150640 1 Processor -- start 2024-04-25T10:07:09.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe91c150640 1 -- start start 2024-04-25T10:07:09.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe91c150640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9140cbcf0 0x7fe914160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe91c150640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe914071c30 0x7fe91415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe91c150640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe914144a40 con 0x7fe91415fa20 2024-04-25T10:07:09.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe91c150640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9140b2dc0 con 0x7fe9140cbcf0 2024-04-25T10:07:09.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe91c150640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9140cc9f0 con 0x7fe914071c30 2024-04-25T10:07:09.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe919eec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9140cbcf0 0x7fe914160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe919eec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9140cbcf0 0x7fe914160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38970/0 (socket says 172.21.15.60:38970) 2024-04-25T10:07:09.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe919eec640 1 -- 172.21.15.60:0/1572672295 learned_addr learned my addr 172.21.15.60:0/1572672295 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe91a6ed640 1 --1- 172.21.15.60:0/1572672295 >> v1:172.21.15.60:6789/0 conn(0x7fe91415fa20 0x7fe91415bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60038/0 (socket says 172.21.15.60:60038) 2024-04-25T10:07:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe9196eb640 1 --2- 172.21.15.60:0/1572672295 >> v2:172.21.15.100:3300/0 conn(0x7fe914071c30 0x7fe91415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe919eec640 1 -- 172.21.15.60:0/1572672295 >> v2:172.21.15.100:3300/0 conn(0x7fe914071c30 msgr2=0x7fe91415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe919eec640 1 --2- 172.21.15.60:0/1572672295 >> v2:172.21.15.100:3300/0 conn(0x7fe914071c30 0x7fe91415f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe919eec640 1 -- 172.21.15.60:0/1572672295 >> v1:172.21.15.60:6789/0 conn(0x7fe91415fa20 legacy=0x7fe91415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe919eec640 1 -- 172.21.15.60:0/1572672295 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9140cccd0 con 0x7fe9140cbcf0 2024-04-25T10:07:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe918eea640 1 -- 172.21.15.60:0/1572672295 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3002446922 0 0) 0x7fe914144a40 con 0x7fe91415fa20 2024-04-25T10:07:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.427+0000 7fe919eec640 1 --2- 172.21.15.60:0/1572672295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9140cbcf0 0x7fe914160080 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe9140d0570 tx=0x7fe91001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3247d073c86e5468 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe918eea640 1 -- 172.21.15.60:0/1572672295 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9100ab610 con 0x7fe9140cbcf0 2024-04-25T10:07:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe918eea640 1 -- 172.21.15.60:0/1572672295 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9100b0070 con 0x7fe9140cbcf0 2024-04-25T10:07:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe918eea640 1 -- 172.21.15.60:0/1572672295 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9100ab9d0 con 0x7fe9140cbcf0 2024-04-25T10:07:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 -- 172.21.15.60:0/1572672295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9140cbcf0 msgr2=0x7fe914160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 --2- 172.21.15.60:0/1572672295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9140cbcf0 0x7fe914160080 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe9140d0570 tx=0x7fe91001c520 comp rx=0 tx=0).stop 2024-04-25T10:07:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 -- 172.21.15.60:0/1572672295 shutdown_connections 2024-04-25T10:07:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 --2- 172.21.15.60:0/1572672295 >> v2:172.21.15.100:3300/0 conn(0x7fe914071c30 0x7fe91415f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 --2- 172.21.15.60:0/1572672295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9140cbcf0 0x7fe914160080 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 -- 172.21.15.60:0/1572672295 >> 172.21.15.60:0/1572672295 conn(0x7fe9140670b0 msgr2=0x7fe91406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 -- 172.21.15.60:0/1572672295 shutdown_connections 2024-04-25T10:07:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 -- 172.21.15.60:0/1572672295 wait complete. 2024-04-25T10:07:09.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 Processor -- start 2024-04-25T10:07:09.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 -- start start 2024-04-25T10:07:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914071c30 0x7fe9140a9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9140cbcf0 0x7fe91406dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe91415fa20 0x7fe91415b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9140cfed0 con 0x7fe914071c30 2024-04-25T10:07:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9140ce5c0 con 0x7fe91415fa20 2024-04-25T10:07:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91406e4a0 con 0x7fe9140cbcf0 2024-04-25T10:07:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe919eec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914071c30 0x7fe9140a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe919eec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914071c30 0x7fe9140a9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38986/0 (socket says 172.21.15.60:38986) 2024-04-25T10:07:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe919eec640 1 -- 172.21.15.60:0/1941027539 learned_addr learned my addr 172.21.15.60:0/1941027539 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe9196eb640 1 --2- 172.21.15.60:0/1941027539 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9140cbcf0 0x7fe91406dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91a6ed640 1 --2- 172.21.15.60:0/1941027539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe91415fa20 0x7fe91415b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe919eec640 1 -- 172.21.15.60:0/1941027539 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9140cbcf0 msgr2=0x7fe91406dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe919eec640 1 --2- 172.21.15.60:0/1941027539 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9140cbcf0 0x7fe91406dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe919eec640 1 -- 172.21.15.60:0/1941027539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe91415fa20 msgr2=0x7fe91415b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe919eec640 1 --2- 172.21.15.60:0/1941027539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe91415fa20 0x7fe91415b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe919eec640 1 -- 172.21.15.60:0/1941027539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91406e780 con 0x7fe914071c30 2024-04-25T10:07:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe9196eb640 1 --2- 172.21.15.60:0/1941027539 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9140cbcf0 0x7fe91406dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe919eec640 1 --2- 172.21.15.60:0/1941027539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914071c30 0x7fe9140a9140 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe9100ad040 tx=0x7fe91001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe90a7fc640 1 -- 172.21.15.60:0/1941027539 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9100be5f0 con 0x7fe914071c30 2024-04-25T10:07:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe90a7fc640 1 -- 172.21.15.60:0/1941027539 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9100be7b0 con 0x7fe914071c30 2024-04-25T10:07:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 -- 172.21.15.60:0/1941027539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91406ea60 con 0x7fe914071c30 2024-04-25T10:07:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe90a7fc640 1 -- 172.21.15.60:0/1941027539 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9100beba0 con 0x7fe914071c30 2024-04-25T10:07:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.431+0000 7fe91c150640 1 -- 172.21.15.60:0/1941027539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91415b870 con 0x7fe914071c30 2024-04-25T10:07:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.435+0000 7fe91c150640 1 -- 172.21.15.60:0/1941027539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91406bba0 con 0x7fe914071c30 2024-04-25T10:07:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.435+0000 7fe90a7fc640 1 -- 172.21.15.60:0/1941027539 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9100b43f0 con 0x7fe914071c30 2024-04-25T10:07:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.435+0000 7fe90a7fc640 1 --2- 172.21.15.60:0/1941027539 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8f8066f60 0x7fe8f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.435+0000 7fe90a7fc640 1 -- 172.21.15.60:0/1941027539 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe91011f5d0 con 0x7fe914071c30 2024-04-25T10:07:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.435+0000 7fe9196eb640 1 --2- 172.21.15.60:0/1941027539 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8f8066f60 0x7fe8f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.435+0000 7fe9196eb640 1 --2- 172.21.15.60:0/1941027539 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8f8066f60 0x7fe8f8069420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe9140b2200 tx=0x7fe904008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.439+0000 7fe90a7fc640 1 -- 172.21.15.60:0/1941027539 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9100c4060 con 0x7fe914071c30 2024-04-25T10:07:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.635+0000 7fe91c150640 1 -- 172.21.15.60:0/1941027539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe914074770 con 0x7fe914071c30 2024-04-25T10:07:09.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.647+0000 7fe90a7fc640 1 -- 172.21.15.60:0/1941027539 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v20) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe9100a8050 con 0x7fe914071c30 2024-04-25T10:07:09.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.647+0000 7fe91c150640 1 -- 172.21.15.60:0/1941027539 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8f8066f60 msgr2=0x7fe8f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.647+0000 7fe91c150640 1 --2- 172.21.15.60:0/1941027539 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8f8066f60 0x7fe8f8069420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe9140b2200 tx=0x7fe904008040 comp rx=0 tx=0).stop 2024-04-25T10:07:09.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.647+0000 7fe91c150640 1 -- 172.21.15.60:0/1941027539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914071c30 msgr2=0x7fe9140a9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.647+0000 7fe91c150640 1 --2- 172.21.15.60:0/1941027539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914071c30 0x7fe9140a9140 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe9100ad040 tx=0x7fe91001ad60 comp rx=0 tx=0).stop 2024-04-25T10:07:09.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.647+0000 7fe91c150640 1 -- 172.21.15.60:0/1941027539 shutdown_connections 2024-04-25T10:07:09.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.647+0000 7fe91c150640 1 --2- 172.21.15.60:0/1941027539 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8f8066f60 0x7fe8f8069420 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.647+0000 7fe91c150640 1 --2- 172.21.15.60:0/1941027539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe91415fa20 0x7fe91415b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.647+0000 7fe91c150640 1 --2- 172.21.15.60:0/1941027539 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9140cbcf0 0x7fe91406dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.647+0000 7fe91c150640 1 --2- 172.21.15.60:0/1941027539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe914071c30 0x7fe9140a9140 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.647+0000 7fe91c150640 1 -- 172.21.15.60:0/1941027539 >> 172.21.15.60:0/1941027539 conn(0x7fe9140670b0 msgr2=0x7fe9140b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.651+0000 7fe91c150640 1 -- 172.21.15.60:0/1941027539 shutdown_connections 2024-04-25T10:07:09.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.651+0000 7fe91c150640 1 -- 172.21.15.60:0/1941027539 wait complete. 2024-04-25T10:07:09.676 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:07:09.676 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:07:09.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb460ff640 1 Processor -- start 2024-04-25T10:07:09.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb460ff640 1 -- start start 2024-04-25T10:07:09.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb460ff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb4006f030 0x7fbb400b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb460ff640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4015ad30 con 0x7fbb40071250 2024-04-25T10:07:09.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb460ff640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4015b010 con 0x7fbb400b3f60 2024-04-25T10:07:09.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb460ff640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4015aaf0 con 0x7fbb4006f030 2024-04-25T10:07:09.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbb40071250 0x7fbb4015b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60042/0 (socket says 172.21.15.60:60042) 2024-04-25T10:07:09.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3ffff640 1 -- 172.21.15.60:0/3502784688 learned_addr learned my addr 172.21.15.60:0/3502784688 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:09.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3effd640 1 --2- 172.21.15.60:0/3502784688 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb4006f030 0x7fbb400b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3e7fc640 1 -- 172.21.15.60:0/3502784688 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64882841 0 0) 0x7fbb4015ad30 con 0x7fbb40071250 2024-04-25T10:07:09.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3e7fc640 1 -- 172.21.15.60:0/3502784688 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb28003530 con 0x7fbb40071250 2024-04-25T10:07:09.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3e7fc640 1 -- 172.21.15.60:0/3502784688 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1833841820 0 0) 0x7fbb28003530 con 0x7fbb40071250 2024-04-25T10:07:09.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3e7fc640 1 -- 172.21.15.60:0/3502784688 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb4006f030 msgr2=0x7fbb400b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3e7fc640 1 --2- 172.21.15.60:0/3502784688 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb4006f030 0x7fbb400b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3e7fc640 1 -- 172.21.15.60:0/3502784688 >> v1:172.21.15.100:6789/0 conn(0x7fbb400b3f60 legacy=0x7fbb4015a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3e7fc640 1 -- 172.21.15.60:0/3502784688 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4015b2f0 con 0x7fbb40071250 2024-04-25T10:07:09.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3e7fc640 1 -- 172.21.15.60:0/3502784688 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fbb38001a00 con 0x7fbb40071250 2024-04-25T10:07:09.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3e7fc640 1 -- 172.21.15.60:0/3502784688 >> v1:172.21.15.60:6789/0 conn(0x7fbb40071250 legacy=0x7fbb4015b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3e7fc640 1 --2- 172.21.15.60:0/3502784688 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb4006f030 0x7fbb280039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3e7fc640 1 --2- 172.21.15.60:0/3502784688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb28003f70 0x7fbb28004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3e7fc640 1 --2- 172.21.15.60:0/3502784688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28004d50 0x7fbb280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3e7fc640 1 -- 172.21.15.60:0/3502784688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4015aaf0 con 0x7fbb28004d50 2024-04-25T10:07:09.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3e7fc640 1 -- 172.21.15.60:0/3502784688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2800dd70 con 0x7fbb28003f70 2024-04-25T10:07:09.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3e7fc640 1 -- 172.21.15.60:0/3502784688 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2800df70 con 0x7fbb4006f030 2024-04-25T10:07:09.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3ffff640 1 --2- 172.21.15.60:0/3502784688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28004d50 0x7fbb280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3effd640 1 --2- 172.21.15.60:0/3502784688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb28003f70 0x7fbb28004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3f7fe640 1 --2- 172.21.15.60:0/3502784688 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb4006f030 0x7fbb280039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3ffff640 1 -- 172.21.15.60:0/3502784688 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb4006f030 msgr2=0x7fbb280039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3ffff640 1 --2- 172.21.15.60:0/3502784688 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb4006f030 0x7fbb280039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3ffff640 1 -- 172.21.15.60:0/3502784688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb28003f70 msgr2=0x7fbb28004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3ffff640 1 --2- 172.21.15.60:0/3502784688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb28003f70 0x7fbb28004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.795+0000 7fbb3ffff640 1 -- 172.21.15.60:0/3502784688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb28003530 con 0x7fbb28004d50 2024-04-25T10:07:09.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.799+0000 7fbb3ffff640 1 --2- 172.21.15.60:0/3502784688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28004d50 0x7fbb280051d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800a840 tx=0x7fbb3801fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c729bd136dd4feeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.799+0000 7fbb3e7fc640 1 -- 172.21.15.60:0/3502784688 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3800a460 con 0x7fbb28004d50 2024-04-25T10:07:09.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.799+0000 7fbb3e7fc640 1 -- 172.21.15.60:0/3502784688 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb380aa6f0 con 0x7fbb28004d50 2024-04-25T10:07:09.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.799+0000 7fbb3e7fc640 1 -- 172.21.15.60:0/3502784688 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb380aaac0 con 0x7fbb28004d50 2024-04-25T10:07:09.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.799+0000 7fbb460ff640 1 -- 172.21.15.60:0/3502784688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28004d50 msgr2=0x7fbb280051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.799+0000 7fbb460ff640 1 --2- 172.21.15.60:0/3502784688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb28004d50 0x7fbb280051d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800a840 tx=0x7fbb3801fac0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.799+0000 7fbb3ffff640 1 -- 172.21.15.60:0/3502784688 reap_dead start 2024-04-25T10:07:09.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.799+0000 7fbb460ff640 1 -- 172.21.15.60:0/3502784688 shutdown_connections 2024-04-25T10:07:09.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.799+0000 7fbb460ff640 1 -- 172.21.15.60:0/3502784688 >> 172.21.15.60:0/3502784688 conn(0x7fbb400670b0 msgr2=0x7fbb4006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.799+0000 7fbb460ff640 1 -- 172.21.15.60:0/3502784688 shutdown_connections 2024-04-25T10:07:09.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.799+0000 7fbb460ff640 1 -- 172.21.15.60:0/3502784688 wait complete. 2024-04-25T10:07:09.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.799+0000 7fbb460ff640 1 Processor -- start 2024-04-25T10:07:09.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.799+0000 7fbb460ff640 1 -- start start 2024-04-25T10:07:09.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb460ff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb40143db0 0x7fbb40158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb460ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb40158950 0x7fbb400ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb460ff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb400acf00 0x7fbb400cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb460ff640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4015bcb0 con 0x7fbb400acf00 2024-04-25T10:07:09.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb460ff640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400b4340 con 0x7fbb40158950 2024-04-25T10:07:09.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb460ff640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4015be30 con 0x7fbb40143db0 2024-04-25T10:07:09.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb3ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb400acf00 0x7fbb400cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb3ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb400acf00 0x7fbb400cdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39008/0 (socket says 172.21.15.60:39008) 2024-04-25T10:07:09.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb3ffff640 1 -- 172.21.15.60:0/3749584690 learned_addr learned my addr 172.21.15.60:0/3749584690 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:09.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb3effd640 1 --2- 172.21.15.60:0/3749584690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb40158950 0x7fbb400ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb3ffff640 1 -- 172.21.15.60:0/3749584690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb40143db0 msgr2=0x7fbb40158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb3ffff640 1 --2- 172.21.15.60:0/3749584690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb40143db0 0x7fbb40158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb3ffff640 1 -- 172.21.15.60:0/3749584690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb40158950 msgr2=0x7fbb400ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb3ffff640 1 --2- 172.21.15.60:0/3749584690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb40158950 0x7fbb400ac9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb3ffff640 1 -- 172.21.15.60:0/3749584690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb40144940 con 0x7fbb400acf00 2024-04-25T10:07:09.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb3ffff640 1 --2- 172.21.15.60:0/3749584690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb400acf00 0x7fbb400cdcb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800a930 tx=0x7fbb3801c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb27fff640 1 -- 172.21.15.60:0/3749584690 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3800a460 con 0x7fbb400acf00 2024-04-25T10:07:09.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb27fff640 1 -- 172.21.15.60:0/3749584690 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb380aada0 con 0x7fbb400acf00 2024-04-25T10:07:09.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb460ff640 1 -- 172.21.15.60:0/3749584690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb40144c20 con 0x7fbb400acf00 2024-04-25T10:07:09.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb27fff640 1 -- 172.21.15.60:0/3749584690 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38001fc0 con 0x7fbb400acf00 2024-04-25T10:07:09.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb460ff640 1 -- 172.21.15.60:0/3749584690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb401451e0 con 0x7fbb400acf00 2024-04-25T10:07:09.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb460ff640 1 -- 172.21.15.60:0/3749584690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb40074770 con 0x7fbb400acf00 2024-04-25T10:07:09.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb27fff640 1 -- 172.21.15.60:0/3749584690 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb380b2400 con 0x7fbb400acf00 2024-04-25T10:07:09.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb27fff640 1 --2- 172.21.15.60:0/3749584690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbb2c066f90 0x7fbb2c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb27fff640 1 -- 172.21.15.60:0/3749584690 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb381225c0 con 0x7fbb400acf00 2024-04-25T10:07:09.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.803+0000 7fbb3f7fe640 1 --2- 172.21.15.60:0/3749584690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbb2c066f90 0x7fbb2c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.807+0000 7fbb3f7fe640 1 --2- 172.21.15.60:0/3749584690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbb2c066f90 0x7fbb2c069450 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbb30001580 tx=0x7fbb30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.807+0000 7fbb27fff640 1 -- 172.21.15.60:0/3749584690 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb380ae030 con 0x7fbb400acf00 2024-04-25T10:07:10.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:09.999+0000 7fbb460ff640 1 -- 172.21.15.60:0/3749584690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbb40154420 con 0x7fbb400acf00 2024-04-25T10:07:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.659+0000 7fbb27fff640 1 -- 172.21.15.60:0/3749584690 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v21) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbb380197d0 con 0x7fbb400acf00 2024-04-25T10:07:10.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.663+0000 7fbb460ff640 1 -- 172.21.15.60:0/3749584690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbb2c066f90 msgr2=0x7fbb2c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.663+0000 7fbb460ff640 1 --2- 172.21.15.60:0/3749584690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbb2c066f90 0x7fbb2c069450 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbb30001580 tx=0x7fbb30009040 comp rx=0 tx=0).stop 2024-04-25T10:07:10.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.663+0000 7fbb460ff640 1 -- 172.21.15.60:0/3749584690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb400acf00 msgr2=0x7fbb400cdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.663+0000 7fbb460ff640 1 --2- 172.21.15.60:0/3749584690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb400acf00 0x7fbb400cdcb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800a930 tx=0x7fbb3801c860 comp rx=0 tx=0).stop 2024-04-25T10:07:10.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.663+0000 7fbb460ff640 1 -- 172.21.15.60:0/3749584690 shutdown_connections 2024-04-25T10:07:10.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.663+0000 7fbb460ff640 1 --2- 172.21.15.60:0/3749584690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbb2c066f90 0x7fbb2c069450 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.663+0000 7fbb460ff640 1 --2- 172.21.15.60:0/3749584690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb400acf00 0x7fbb400cdcb0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.663+0000 7fbb460ff640 1 --2- 172.21.15.60:0/3749584690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb40158950 0x7fbb400ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.663+0000 7fbb460ff640 1 --2- 172.21.15.60:0/3749584690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb40143db0 0x7fbb40158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.663+0000 7fbb460ff640 1 -- 172.21.15.60:0/3749584690 >> 172.21.15.60:0/3749584690 conn(0x7fbb400670b0 msgr2=0x7fbb400b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.663+0000 7fbb460ff640 1 -- 172.21.15.60:0/3749584690 shutdown_connections 2024-04-25T10:07:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.663+0000 7fbb460ff640 1 -- 172.21.15.60:0/3749584690 wait complete. 2024-04-25T10:07:10.688 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:07:10.688 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:10.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.803+0000 7f891f47d640 1 Processor -- start 2024-04-25T10:07:10.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.803+0000 7f891f47d640 1 -- start start 2024-04-25T10:07:10.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.803+0000 7f891f47d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f891815a7a0 0x7f89180cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.803+0000 7f891f47d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f891806ec30 0x7f8918071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.803+0000 7f891f47d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89180721e0 con 0x7f89180b37e0 2024-04-25T10:07:10.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.803+0000 7f891f47d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89180cdaa0 con 0x7f891815a7a0 2024-04-25T10:07:10.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.803+0000 7f891f47d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891806d2f0 con 0x7f891806ec30 2024-04-25T10:07:10.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.803+0000 7f891d219640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f891815a7a0 0x7f89180cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.803+0000 7f891da1a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f89180b37e0 0x7f891806dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60050/0 (socket says 172.21.15.60:60050) 2024-04-25T10:07:10.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.803+0000 7f891da1a640 1 -- 172.21.15.60:0/4035519994 learned_addr learned my addr 172.21.15.60:0/4035519994 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:10.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.803+0000 7f891ca18640 1 --2- 172.21.15.60:0/4035519994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f891806ec30 0x7f8918071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.803+0000 7f891d219640 1 -- 172.21.15.60:0/4035519994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f891806ec30 msgr2=0x7f8918071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.803+0000 7f891d219640 1 --2- 172.21.15.60:0/4035519994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f891806ec30 0x7f8918071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.803+0000 7f891d219640 1 -- 172.21.15.60:0/4035519994 >> v1:172.21.15.60:6789/0 conn(0x7f89180b37e0 legacy=0x7f891806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.803+0000 7f891d219640 1 -- 172.21.15.60:0/4035519994 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891806f130 con 0x7f891815a7a0 2024-04-25T10:07:10.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.803+0000 7f891d219640 1 --2- 172.21.15.60:0/4035519994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f891815a7a0 0x7f89180cd560 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f89140097c0 tx=0x7f891401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46f80b4f71b3db7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f890ffff640 1 -- 172.21.15.60:0/4035519994 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89140b1070 con 0x7f891815a7a0 2024-04-25T10:07:10.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f890ffff640 1 -- 172.21.15.60:0/4035519994 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89140ac410 con 0x7f891815a7a0 2024-04-25T10:07:10.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f890ffff640 1 -- 172.21.15.60:0/4035519994 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89140ac7e0 con 0x7f891815a7a0 2024-04-25T10:07:10.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 -- 172.21.15.60:0/4035519994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f891815a7a0 msgr2=0x7f89180cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 --2- 172.21.15.60:0/4035519994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f891815a7a0 0x7f89180cd560 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f89140097c0 tx=0x7f891401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 -- 172.21.15.60:0/4035519994 shutdown_connections 2024-04-25T10:07:10.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 --2- 172.21.15.60:0/4035519994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f891806ec30 0x7f8918071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 --2- 172.21.15.60:0/4035519994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f891815a7a0 0x7f89180cd560 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 -- 172.21.15.60:0/4035519994 >> 172.21.15.60:0/4035519994 conn(0x7f89180b12d0 msgr2=0x7f89180b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 -- 172.21.15.60:0/4035519994 shutdown_connections 2024-04-25T10:07:10.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 -- 172.21.15.60:0/4035519994 wait complete. 2024-04-25T10:07:10.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 Processor -- start 2024-04-25T10:07:10.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 -- start start 2024-04-25T10:07:10.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f891806ec30 0x7f89180c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89180b37e0 0x7f8918154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f891815a7a0 0x7f89180c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891815ad00 con 0x7f89180b37e0 2024-04-25T10:07:10.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891815ab80 con 0x7f891806ec30 2024-04-25T10:07:10.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891806e860 con 0x7f891815a7a0 2024-04-25T10:07:10.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891ca18640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89180b37e0 0x7f8918154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891ca18640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89180b37e0 0x7f8918154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39034/0 (socket says 172.21.15.60:39034) 2024-04-25T10:07:10.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891ca18640 1 -- 172.21.15.60:0/2419466740 learned_addr learned my addr 172.21.15.60:0/2419466740 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:10.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891d219640 1 --2- 172.21.15.60:0/2419466740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f891806ec30 0x7f89180c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891da1a640 1 --2- 172.21.15.60:0/2419466740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f891815a7a0 0x7f89180c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891ca18640 1 -- 172.21.15.60:0/2419466740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f891815a7a0 msgr2=0x7f89180c4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891ca18640 1 --2- 172.21.15.60:0/2419466740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f891815a7a0 0x7f89180c4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891ca18640 1 -- 172.21.15.60:0/2419466740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f891806ec30 msgr2=0x7f89180c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891ca18640 1 --2- 172.21.15.60:0/2419466740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f891806ec30 0x7f89180c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891ca18640 1 -- 172.21.15.60:0/2419466740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8918161e50 con 0x7f89180b37e0 2024-04-25T10:07:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891d219640 1 --2- 172.21.15.60:0/2419466740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f891806ec30 0x7f89180c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891ca18640 1 --2- 172.21.15.60:0/2419466740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89180b37e0 0x7f8918154160 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f890800d830 tx=0x7f890800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f890dffb640 1 -- 172.21.15.60:0/2419466740 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890800b6c0 con 0x7f89180b37e0 2024-04-25T10:07:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f890dffb640 1 -- 172.21.15.60:0/2419466740 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8908019070 con 0x7f89180b37e0 2024-04-25T10:07:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 -- 172.21.15.60:0/2419466740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89180c4a80 con 0x7f89180b37e0 2024-04-25T10:07:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f890dffb640 1 -- 172.21.15.60:0/2419466740 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890800ba60 con 0x7f89180b37e0 2024-04-25T10:07:10.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.807+0000 7f891f47d640 1 -- 172.21.15.60:0/2419466740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8918161370 con 0x7f89180b37e0 2024-04-25T10:07:10.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.811+0000 7f891f47d640 1 -- 172.21.15.60:0/2419466740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8918074770 con 0x7f89180b37e0 2024-04-25T10:07:10.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.811+0000 7f890dffb640 1 -- 172.21.15.60:0/2419466740 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f890801e460 con 0x7f89180b37e0 2024-04-25T10:07:10.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.811+0000 7f890dffb640 1 --2- 172.21.15.60:0/2419466740 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f88fc066f60 0x7f88fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.811+0000 7f890dffb640 1 -- 172.21.15.60:0/2419466740 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8908085680 con 0x7f89180b37e0 2024-04-25T10:07:10.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.811+0000 7f891d219640 1 --2- 172.21.15.60:0/2419466740 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f88fc066f60 0x7f88fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.811+0000 7f891d219640 1 --2- 172.21.15.60:0/2419466740 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f88fc066f60 0x7f88fc069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f89140ae000 tx=0x7f8914008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:10.815+0000 7f890dffb640 1 -- 172.21.15.60:0/2419466740 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8908056720 con 0x7f89180b37e0 2024-04-25T10:07:11.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.007+0000 7f891f47d640 1 -- 172.21.15.60:0/2419466740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8918071e80 con 0x7f89180b37e0 2024-04-25T10:07:11.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.007+0000 7f890dffb640 1 -- 172.21.15.60:0/2419466740 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f8908056020 con 0x7f89180b37e0 2024-04-25T10:07:11.014 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:11.014 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1138162580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1138162580},{"type":"v1","addr":"172.21.15.100:6837","nonce":1138162580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2985559168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2985559168},{"type":"v1","addr":"172.21.15.100:6835","nonce":2985559168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/50795370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":50795370},{"type":"v1","addr":"172.21.15.60:6834","nonce":50795370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:07.980095+0000","modified":"2024-04-25T10:07:10.002670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5796":{"gid":5796,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1747757196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1747757196},{"type":"v1","addr":"172.21.15.60:6835","nonce":1747757196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:11.014 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 21 2024-04-25T10:07:11.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.007+0000 7f891f47d640 1 -- 172.21.15.60:0/2419466740 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f88fc066f60 msgr2=0x7f88fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.007+0000 7f891f47d640 1 --2- 172.21.15.60:0/2419466740 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f88fc066f60 0x7f88fc069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f89140ae000 tx=0x7f8914008040 comp rx=0 tx=0).stop 2024-04-25T10:07:11.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.007+0000 7f891f47d640 1 -- 172.21.15.60:0/2419466740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89180b37e0 msgr2=0x7f8918154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.007+0000 7f891f47d640 1 --2- 172.21.15.60:0/2419466740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89180b37e0 0x7f8918154160 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f890800d830 tx=0x7f890800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.011+0000 7f891f47d640 1 -- 172.21.15.60:0/2419466740 shutdown_connections 2024-04-25T10:07:11.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.011+0000 7f891f47d640 1 --2- 172.21.15.60:0/2419466740 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f88fc066f60 0x7f88fc069420 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.011+0000 7f891f47d640 1 --2- 172.21.15.60:0/2419466740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f891815a7a0 0x7f89180c4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.011+0000 7f891f47d640 1 --2- 172.21.15.60:0/2419466740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89180b37e0 0x7f8918154160 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.011+0000 7f891f47d640 1 --2- 172.21.15.60:0/2419466740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f891806ec30 0x7f89180c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.011+0000 7f891f47d640 1 -- 172.21.15.60:0/2419466740 >> 172.21.15.60:0/2419466740 conn(0x7f89180b12d0 msgr2=0x7f891815af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.011+0000 7f891f47d640 1 -- 172.21.15.60:0/2419466740 shutdown_connections 2024-04-25T10:07:11.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.011+0000 7f891f47d640 1 -- 172.21.15.60:0/2419466740 wait complete. 2024-04-25T10:07:11.032 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:11.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd0a6253640 1 Processor -- start 2024-04-25T10:07:11.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd0a6253640 1 -- start start 2024-04-25T10:07:11.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd0a6253640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd0a0071b20 0x7fd0a0071f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd0a6253640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a006e480 0x7fd0a006e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd0a6253640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0a006d2f0 0x7fd0a006d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd0a6253640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a0072440 con 0x7fd0a006d2f0 2024-04-25T10:07:11.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd0a6253640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a015dc10 con 0x7fd0a0071b20 2024-04-25T10:07:11.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd0a6253640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a015def0 con 0x7fd0a006e480 2024-04-25T10:07:11.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd09ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0a006d2f0 0x7fd0a006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd09ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0a006d2f0 0x7fd0a006d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39038/0 (socket says 172.21.15.60:39038) 2024-04-25T10:07:11.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd09ffff640 1 -- 172.21.15.60:0/75739147 learned_addr learned my addr 172.21.15.60:0/75739147 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:11.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd09f7fe640 1 --2- 172.21.15.60:0/75739147 >> v2:172.21.15.100:3300/0 conn(0x7fd0a0071b20 0x7fd0a0071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd097fff640 1 --2- 172.21.15.60:0/75739147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a006e480 0x7fd0a006e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd09ffff640 1 -- 172.21.15.60:0/75739147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a006e480 msgr2=0x7fd0a006e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd09ffff640 1 --2- 172.21.15.60:0/75739147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a006e480 0x7fd0a006e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd09ffff640 1 -- 172.21.15.60:0/75739147 >> v2:172.21.15.100:3300/0 conn(0x7fd0a0071b20 msgr2=0x7fd0a0071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd09ffff640 1 --2- 172.21.15.60:0/75739147 >> v2:172.21.15.100:3300/0 conn(0x7fd0a0071b20 0x7fd0a0071f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd09ffff640 1 -- 172.21.15.60:0/75739147 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a015e1d0 con 0x7fd0a006d2f0 2024-04-25T10:07:11.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.143+0000 7fd09f7fe640 1 --2- 172.21.15.60:0/75739147 >> v2:172.21.15.100:3300/0 conn(0x7fd0a0071b20 0x7fd0a0071f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:11.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd09ffff640 1 --2- 172.21.15.60:0/75739147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a006d2f0 0x7fd0a006d720 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00d05d0 tx=0x7fd09801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97859135261d7a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd09effd640 1 -- 172.21.15.60:0/75739147 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0980a5650 con 0x7fd0a006d2f0 2024-04-25T10:07:11.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd09effd640 1 -- 172.21.15.60:0/75739147 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0980a57f0 con 0x7fd0a006d2f0 2024-04-25T10:07:11.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd09effd640 1 -- 172.21.15.60:0/75739147 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0980a5bc0 con 0x7fd0a006d2f0 2024-04-25T10:07:11.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 -- 172.21.15.60:0/75739147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a006d2f0 msgr2=0x7fd0a006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 --2- 172.21.15.60:0/75739147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a006d2f0 0x7fd0a006d720 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00d05d0 tx=0x7fd09801c960 comp rx=0 tx=0).stop 2024-04-25T10:07:11.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 -- 172.21.15.60:0/75739147 shutdown_connections 2024-04-25T10:07:11.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 --2- 172.21.15.60:0/75739147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a006d2f0 0x7fd0a006d720 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 --2- 172.21.15.60:0/75739147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a006e480 0x7fd0a006e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 --2- 172.21.15.60:0/75739147 >> v2:172.21.15.100:3300/0 conn(0x7fd0a0071b20 0x7fd0a0071f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 -- 172.21.15.60:0/75739147 >> 172.21.15.60:0/75739147 conn(0x7fd0a00b12d0 msgr2=0x7fd0a00b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 -- 172.21.15.60:0/75739147 shutdown_connections 2024-04-25T10:07:11.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 -- 172.21.15.60:0/75739147 wait complete. 2024-04-25T10:07:11.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 Processor -- start 2024-04-25T10:07:11.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 -- start start 2024-04-25T10:07:11.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a006d2f0 0x7fd0a0164630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a006e480 0x7fd0a0141970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0a0071b20 0x7fd0a0141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a00cf070 con 0x7fd0a006e480 2024-04-25T10:07:11.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a00ceef0 con 0x7fd0a0071b20 2024-04-25T10:07:11.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd0a6253640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a006ee20 con 0x7fd0a006d2f0 2024-04-25T10:07:11.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd097fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a006e480 0x7fd0a0141970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd097fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a006e480 0x7fd0a0141970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39042/0 (socket says 172.21.15.60:39042) 2024-04-25T10:07:11.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd097fff640 1 -- 172.21.15.60:0/1608127673 learned_addr learned my addr 172.21.15.60:0/1608127673 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:11.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd09ffff640 1 --2- 172.21.15.60:0/1608127673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0a0071b20 0x7fd0a0141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd09f7fe640 1 --2- 172.21.15.60:0/1608127673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a006d2f0 0x7fd0a0164630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd097fff640 1 -- 172.21.15.60:0/1608127673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a006d2f0 msgr2=0x7fd0a0164630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd097fff640 1 --2- 172.21.15.60:0/1608127673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a006d2f0 0x7fd0a0164630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd097fff640 1 -- 172.21.15.60:0/1608127673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0a0071b20 msgr2=0x7fd0a0141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd097fff640 1 --2- 172.21.15.60:0/1608127673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0a0071b20 0x7fd0a0141eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd097fff640 1 -- 172.21.15.60:0/1608127673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a014cc00 con 0x7fd0a006e480 2024-04-25T10:07:11.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd097fff640 1 --2- 172.21.15.60:0/1608127673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a006e480 0x7fd0a0141970 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd08800d800 tx=0x7fd08800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd09cff9640 1 -- 172.21.15.60:0/1608127673 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08800b6c0 con 0x7fd0a006e480 2024-04-25T10:07:11.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd09cff9640 1 -- 172.21.15.60:0/1608127673 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd088019070 con 0x7fd0a006e480 2024-04-25T10:07:11.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.147+0000 7fd09cff9640 1 -- 172.21.15.60:0/1608127673 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08800ba60 con 0x7fd0a006e480 2024-04-25T10:07:11.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.151+0000 7fd0a6253640 1 -- 172.21.15.60:0/1608127673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a014cee0 con 0x7fd0a006e480 2024-04-25T10:07:11.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.151+0000 7fd0a6253640 1 -- 172.21.15.60:0/1608127673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a015afa0 con 0x7fd0a006e480 2024-04-25T10:07:11.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.151+0000 7fd0a6253640 1 -- 172.21.15.60:0/1608127673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a0074770 con 0x7fd0a006e480 2024-04-25T10:07:11.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.151+0000 7fd09cff9640 1 -- 172.21.15.60:0/1608127673 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd08800bc00 con 0x7fd0a006e480 2024-04-25T10:07:11.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.151+0000 7fd09cff9640 1 --2- 172.21.15.60:0/1608127673 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd080066f60 0x7fd080069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.151+0000 7fd09cff9640 1 -- 172.21.15.60:0/1608127673 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd088085570 con 0x7fd0a006e480 2024-04-25T10:07:11.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.151+0000 7fd09f7fe640 1 --2- 172.21.15.60:0/1608127673 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd080066f60 0x7fd080069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.155+0000 7fd09f7fe640 1 --2- 172.21.15.60:0/1608127673 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd080066f60 0x7fd080069420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd09000d8d0 tx=0x7fd090008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.155+0000 7fd09cff9640 1 -- 172.21.15.60:0/1608127673 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd0880564e0 con 0x7fd0a006e480 2024-04-25T10:07:11.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.315+0000 7fd0a6253640 1 -- 172.21.15.60:0/1608127673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0a01404a0 con 0x7fd0a006e480 2024-04-25T10:07:11.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.319+0000 7fd09cff9640 1 -- 172.21.15.60:0/1608127673 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16625 (secure 0 0 0) 0x7fd088083e20 con 0x7fd0a006e480 2024-04-25T10:07:11.326 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:11.326 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":47,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:07:09.646687+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.652298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:06.983225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:11.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.323+0000 7fd0a6253640 1 -- 172.21.15.60:0/1608127673 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd080066f60 msgr2=0x7fd080069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.323+0000 7fd0a6253640 1 --2- 172.21.15.60:0/1608127673 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd080066f60 0x7fd080069420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd09000d8d0 tx=0x7fd090008040 comp rx=0 tx=0).stop 2024-04-25T10:07:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.323+0000 7fd0a6253640 1 -- 172.21.15.60:0/1608127673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a006e480 msgr2=0x7fd0a0141970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.323+0000 7fd0a6253640 1 --2- 172.21.15.60:0/1608127673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a006e480 0x7fd0a0141970 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd08800d800 tx=0x7fd08800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.323+0000 7fd0a6253640 1 -- 172.21.15.60:0/1608127673 shutdown_connections 2024-04-25T10:07:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.323+0000 7fd0a6253640 1 --2- 172.21.15.60:0/1608127673 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd080066f60 0x7fd080069420 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.323+0000 7fd0a6253640 1 --2- 172.21.15.60:0/1608127673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0a0071b20 0x7fd0a0141eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.323+0000 7fd0a6253640 1 --2- 172.21.15.60:0/1608127673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a006e480 0x7fd0a0141970 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.323+0000 7fd0a6253640 1 --2- 172.21.15.60:0/1608127673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a006d2f0 0x7fd0a0164630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.323+0000 7fd0a6253640 1 -- 172.21.15.60:0/1608127673 >> 172.21.15.60:0/1608127673 conn(0x7fd0a00b12d0 msgr2=0x7fd0a00cb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.323+0000 7fd0a6253640 1 -- 172.21.15.60:0/1608127673 shutdown_connections 2024-04-25T10:07:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.323+0000 7fd0a6253640 1 -- 172.21.15.60:0/1608127673 wait complete. 2024-04-25T10:07:11.345 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:07:11.346 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:11.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9ab5e2a640 1 Processor -- start 2024-04-25T10:07:11.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9ab5e2a640 1 -- start start 2024-04-25T10:07:11.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9ab5e2a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9ab006ec30 0x7f9ab00cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9ab5e2a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ab00b37e0 0x7f9ab00b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9ab5e2a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab006f130 con 0x7f9ab015a7a0 2024-04-25T10:07:11.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9ab5e2a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab00b4190 con 0x7f9ab00b37e0 2024-04-25T10:07:11.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9ab5e2a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab00cdaa0 con 0x7f9ab006ec30 2024-04-25T10:07:11.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9aaf7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9ab015a7a0 0x7f9ab0071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60058/0 (socket says 172.21.15.60:60058) 2024-04-25T10:07:11.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9aaf7fe640 1 -- 172.21.15.60:0/517200150 learned_addr learned my addr 172.21.15.60:0/517200150 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:11.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9aaffff640 1 --2- 172.21.15.60:0/517200150 >> v2:172.21.15.60:3300/0 conn(0x7f9ab00b37e0 0x7f9ab00b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9aa6ffd640 1 --2- 172.21.15.60:0/517200150 >> v2:172.21.15.100:3300/0 conn(0x7f9ab006ec30 0x7f9ab00cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9aaeffd640 1 -- 172.21.15.60:0/517200150 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334705289 0 0) 0x7f9ab006f130 con 0x7f9ab015a7a0 2024-04-25T10:07:11.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9aaeffd640 1 -- 172.21.15.60:0/517200150 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a9c003530 con 0x7f9ab015a7a0 2024-04-25T10:07:11.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9aaffff640 1 -- 172.21.15.60:0/517200150 >> v2:172.21.15.100:3300/0 conn(0x7f9ab006ec30 msgr2=0x7f9ab00cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9aaffff640 1 --2- 172.21.15.60:0/517200150 >> v2:172.21.15.100:3300/0 conn(0x7f9ab006ec30 0x7f9ab00cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9aaffff640 1 -- 172.21.15.60:0/517200150 >> v1:172.21.15.60:6789/0 conn(0x7f9ab015a7a0 legacy=0x7f9ab0071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.463+0000 7f9aaffff640 1 -- 172.21.15.60:0/517200150 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab006f410 con 0x7f9ab00b37e0 2024-04-25T10:07:11.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9aaffff640 1 --2- 172.21.15.60:0/517200150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ab00b37e0 0x7f9ab00b3c50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009d90 tx=0x7f9aa801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3eb9ecd746662f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9aaeffd640 1 -- 172.21.15.60:0/517200150 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa80b2070 con 0x7f9ab00b37e0 2024-04-25T10:07:11.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9aaeffd640 1 -- 172.21.15.60:0/517200150 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa80a5900 con 0x7f9ab00b37e0 2024-04-25T10:07:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9aaeffd640 1 -- 172.21.15.60:0/517200150 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa80a5cb0 con 0x7f9ab00b37e0 2024-04-25T10:07:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9ab5e2a640 1 -- 172.21.15.60:0/517200150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ab00b37e0 msgr2=0x7f9ab00b3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9ab5e2a640 1 --2- 172.21.15.60:0/517200150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ab00b37e0 0x7f9ab00b3c50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009d90 tx=0x7f9aa801c930 comp rx=0 tx=0).stop 2024-04-25T10:07:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9ab5e2a640 1 -- 172.21.15.60:0/517200150 shutdown_connections 2024-04-25T10:07:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9ab5e2a640 1 --2- 172.21.15.60:0/517200150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ab00b37e0 0x7f9ab00b3c50 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9ab5e2a640 1 --2- 172.21.15.60:0/517200150 >> v2:172.21.15.100:3300/0 conn(0x7f9ab006ec30 0x7f9ab00cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9ab5e2a640 1 -- 172.21.15.60:0/517200150 >> 172.21.15.60:0/517200150 conn(0x7f9ab00b12d0 msgr2=0x7f9ab00b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9ab5e2a640 1 -- 172.21.15.60:0/517200150 shutdown_connections 2024-04-25T10:07:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9ab5e2a640 1 -- 172.21.15.60:0/517200150 wait complete. 2024-04-25T10:07:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9ab5e2a640 1 Processor -- start 2024-04-25T10:07:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9ab5e2a640 1 -- start start 2024-04-25T10:07:11.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9ab5e2a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ab006ec30 0x7f9ab00b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9ab5e2a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ab00b37e0 0x7f9ab0156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9ab5e2a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ab015a7a0 0x7f9ab0156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9ab5e2a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab006e8c0 con 0x7f9ab006ec30 2024-04-25T10:07:11.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9ab5e2a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab006e740 con 0x7f9ab015a7a0 2024-04-25T10:07:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9ab5e2a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab006d8b0 con 0x7f9ab00b37e0 2024-04-25T10:07:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9aaf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ab006ec30 0x7f9ab00b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9aaf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ab006ec30 0x7f9ab00b2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39052/0 (socket says 172.21.15.60:39052) 2024-04-25T10:07:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9aaf7fe640 1 -- 172.21.15.60:0/3988210887 learned_addr learned my addr 172.21.15.60:0/3988210887 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9aaf7fe640 1 -- 172.21.15.60:0/3988210887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ab00b37e0 msgr2=0x7f9ab0156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9aaf7fe640 1 --2- 172.21.15.60:0/3988210887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ab00b37e0 0x7f9ab0156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9aaf7fe640 1 -- 172.21.15.60:0/3988210887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ab015a7a0 msgr2=0x7f9ab0156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9aaffff640 1 --2- 172.21.15.60:0/3988210887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ab015a7a0 0x7f9ab0156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9aaf7fe640 1 --2- 172.21.15.60:0/3988210887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ab015a7a0 0x7f9ab0156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9aaf7fe640 1 -- 172.21.15.60:0/3988210887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab0157550 con 0x7f9ab006ec30 2024-04-25T10:07:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.467+0000 7f9aaf7fe640 1 --2- 172.21.15.60:0/3988210887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ab006ec30 0x7f9ab00b2b80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800c3f0 tx=0x7f9a9800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.471+0000 7f9aacff9640 1 -- 172.21.15.60:0/3988210887 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a98017070 con 0x7f9ab006ec30 2024-04-25T10:07:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.471+0000 7f9aacff9640 1 -- 172.21.15.60:0/3988210887 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a98005cf0 con 0x7f9ab006ec30 2024-04-25T10:07:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.471+0000 7f9ab5e2a640 1 -- 172.21.15.60:0/3988210887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab0157830 con 0x7f9ab006ec30 2024-04-25T10:07:11.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.471+0000 7f9aacff9640 1 -- 172.21.15.60:0/3988210887 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a980072f0 con 0x7f9ab006ec30 2024-04-25T10:07:11.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.471+0000 7f9ab5e2a640 1 -- 172.21.15.60:0/3988210887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab0157df0 con 0x7f9ab006ec30 2024-04-25T10:07:11.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.471+0000 7f9ab5e2a640 1 -- 172.21.15.60:0/3988210887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab0074770 con 0x7f9ab006ec30 2024-04-25T10:07:11.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.471+0000 7f9aacff9640 1 -- 172.21.15.60:0/3988210887 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a9801a070 con 0x7f9ab006ec30 2024-04-25T10:07:11.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.471+0000 7f9aacff9640 1 --2- 172.21.15.60:0/3988210887 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a90066f60 0x7f9a90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.471+0000 7f9aacff9640 1 -- 172.21.15.60:0/3988210887 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a98084460 con 0x7f9ab006ec30 2024-04-25T10:07:11.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.471+0000 7f9aa6ffd640 1 --2- 172.21.15.60:0/3988210887 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a90066f60 0x7f9a90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.471+0000 7f9aa6ffd640 1 --2- 172.21.15.60:0/3988210887 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a90066f60 0x7f9a90069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9ab015cc10 tx=0x7f9aa0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.475+0000 7f9aacff9640 1 -- 172.21.15.60:0/3988210887 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a98055470 con 0x7f9ab006ec30 2024-04-25T10:07:11.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.627+0000 7f9ab5e2a640 1 -- 172.21.15.60:0/3988210887 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ab00ce420 con 0x7f9a90066f60 2024-04-25T10:07:11.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.643+0000 7f9aacff9640 1 -- 172.21.15.60:0/3988210887 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+241671 (secure 0 0 0) 0x7f9ab00ce420 con 0x7f9a90066f60 2024-04-25T10:07:11.655 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:11.657 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:07:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.647+0000 7f9ab5e2a640 1 -- 172.21.15.60:0/3988210887 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a90066f60 msgr2=0x7f9a90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.647+0000 7f9ab5e2a640 1 --2- 172.21.15.60:0/3988210887 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a90066f60 0x7f9a90069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9ab015cc10 tx=0x7f9aa0008040 comp rx=0 tx=0).stop 2024-04-25T10:07:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.647+0000 7f9ab5e2a640 1 -- 172.21.15.60:0/3988210887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ab006ec30 msgr2=0x7f9ab00b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.647+0000 7f9ab5e2a640 1 --2- 172.21.15.60:0/3988210887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ab006ec30 0x7f9ab00b2b80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800c3f0 tx=0x7f9a9800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.647+0000 7f9ab5e2a640 1 -- 172.21.15.60:0/3988210887 shutdown_connections 2024-04-25T10:07:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.647+0000 7f9ab5e2a640 1 --2- 172.21.15.60:0/3988210887 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a90066f60 0x7f9a90069420 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.647+0000 7f9ab5e2a640 1 --2- 172.21.15.60:0/3988210887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ab015a7a0 0x7f9ab0156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.647+0000 7f9ab5e2a640 1 --2- 172.21.15.60:0/3988210887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ab00b37e0 0x7f9ab0156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.647+0000 7f9ab5e2a640 1 --2- 172.21.15.60:0/3988210887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ab006ec30 0x7f9ab00b2b80 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.647+0000 7f9ab5e2a640 1 -- 172.21.15.60:0/3988210887 >> 172.21.15.60:0/3988210887 conn(0x7f9ab00b12d0 msgr2=0x7f9ab0164c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.647+0000 7f9ab5e2a640 1 -- 172.21.15.60:0/3988210887 shutdown_connections 2024-04-25T10:07:11.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.647+0000 7f9ab5e2a640 1 -- 172.21.15.60:0/3988210887 wait complete. 2024-04-25T10:07:11.672 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-04-25T10:07:10.425464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":94,"acting":94,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":246264,"kb_used_data":5140,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754728456,"statfs":{"total":773094113280,"available":772841938944,"internally_reserved":0,"allocated":5263360,"data_stored":2813125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001273"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.628546+0000","last_change":"2024-04-25T10:07:07.628546+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.628546+0000","last_undegraded":"2024-04-25T10:07:07.628546+0000","last_fullsized":"2024-04-25T10:07:07.628546+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.623001+0000","last_change":"2024-04-25T10:07:07.623001+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.623001+0000","last_undegraded":"2024-04-25T10:07:07.623001+0000","last_fullsized":"2024-04-25T10:07:07.623001+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.628165+0000","last_change":"2024-04-25T10:07:07.628165+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.628165+0000","last_undegraded":"2024-04-25T10:07:07.628165+0000","last_fullsized":"2024-04-25T10:07:07.628165+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.624555+0000","last_change":"2024-04-25T10:07:07.624555+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.624555+0000","last_undegraded":"2024-04-25T10:07:07.624555+0000","last_fullsized":"2024-04-25T10:07:07.624555+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.623659+0000","last_change":"2024-04-25T10:07:07.623659+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.623659+0000","last_undegraded":"2024-04-25T10:07:07.623659+0000","last_fullsized":"2024-04-25T10:07:07.623659+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.619319+0000","last_change":"2024-04-25T10:07:07.619319+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.619319+0000","last_undegraded":"2024-04-25T10:07:07.619319+0000","last_fullsized":"2024-04-25T10:07:07.619319+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.620746+0000","last_change":"2024-04-25T10:07:07.620746+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.620746+0000","last_undegraded":"2024-04-25T10:07:07.620746+0000","last_fullsized":"2024-04-25T10:07:07.620746+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614524+0000","last_change":"2024-04-25T10:07:06.614524+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614524+0000","last_undegraded":"2024-04-25T10:07:06.614524+0000","last_fullsized":"2024-04-25T10:07:06.614524+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663264+0000","last_change":"2024-04-25T10:07:07.655701+0000","last_active":"2024-04-25T10:07:07.663264+0000","last_peered":"2024-04-25T10:07:07.663264+0000","last_clean":"2024-04-25T10:07:07.663264+0000","last_became_active":"2024-04-25T10:07:07.653511+0000","last_became_peered":"2024-04-25T10:07:07.653511+0000","last_unstale":"2024-04-25T10:07:07.663264+0000","last_undegraded":"2024-04-25T10:07:07.663264+0000","last_fullsized":"2024-04-25T10:07:07.663264+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:56.489633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.615716+0000","last_change":"2024-04-25T10:07:06.615716+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.615716+0000","last_undegraded":"2024-04-25T10:07:06.615716+0000","last_fullsized":"2024-04-25T10:07:06.615716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.668582+0000","last_change":"2024-04-25T10:07:07.661239+0000","last_active":"2024-04-25T10:07:07.668582+0000","last_peered":"2024-04-25T10:07:07.668582+0000","last_clean":"2024-04-25T10:07:07.668582+0000","last_became_active":"2024-04-25T10:07:07.660654+0000","last_became_peered":"2024-04-25T10:07:07.660654+0000","last_unstale":"2024-04-25T10:07:07.668582+0000","last_undegraded":"2024-04-25T10:07:07.668582+0000","last_fullsized":"2024-04-25T10:07:07.668582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:59.863704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.619660+0000","last_change":"2024-04-25T10:07:07.619660+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.619660+0000","last_undegraded":"2024-04-25T10:07:07.619660+0000","last_fullsized":"2024-04-25T10:07:07.619660+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.615803+0000","last_change":"2024-04-25T10:07:06.615803+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.615803+0000","last_undegraded":"2024-04-25T10:07:06.615803+0000","last_fullsized":"2024-04-25T10:07:06.615803+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663157+0000","last_change":"2024-04-25T10:07:07.657392+0000","last_active":"2024-04-25T10:07:07.663157+0000","last_peered":"2024-04-25T10:07:07.663157+0000","last_clean":"2024-04-25T10:07:07.663157+0000","last_became_active":"2024-04-25T10:07:07.656810+0000","last_became_peered":"2024-04-25T10:07:07.656810+0000","last_unstale":"2024-04-25T10:07:07.663157+0000","last_undegraded":"2024-04-25T10:07:07.663157+0000","last_fullsized":"2024-04-25T10:07:07.663157+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:16.056727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.613378+0000","last_change":"2024-04-25T10:07:06.613378+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.613378+0000","last_undegraded":"2024-04-25T10:07:06.613378+0000","last_fullsized":"2024-04-25T10:07:06.613378+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.619506+0000","last_change":"2024-04-25T10:07:06.619506+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.619506+0000","last_undegraded":"2024-04-25T10:07:06.619506+0000","last_fullsized":"2024-04-25T10:07:06.619506+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.622656+0000","last_change":"2024-04-25T10:07:07.622656+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.622656+0000","last_undegraded":"2024-04-25T10:07:07.622656+0000","last_fullsized":"2024-04-25T10:07:07.622656+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.658561+0000","last_change":"2024-04-25T10:07:07.639391+0000","last_active":"2024-04-25T10:07:07.658561+0000","last_peered":"2024-04-25T10:07:07.658561+0000","last_clean":"2024-04-25T10:07:07.658561+0000","last_became_active":"2024-04-25T10:07:07.638645+0000","last_became_peered":"2024-04-25T10:07:07.638645+0000","last_unstale":"2024-04-25T10:07:07.658561+0000","last_undegraded":"2024-04-25T10:07:07.658561+0000","last_fullsized":"2024-04-25T10:07:07.658561+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:42.132953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.627466+0000","last_change":"2024-04-25T10:07:07.627466+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.627466+0000","last_undegraded":"2024-04-25T10:07:07.627466+0000","last_fullsized":"2024-04-25T10:07:07.627466+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.618710+0000","last_change":"2024-04-25T10:07:06.618710+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.618710+0000","last_undegraded":"2024-04-25T10:07:06.618710+0000","last_fullsized":"2024-04-25T10:07:06.618710+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.671738+0000","last_change":"2024-04-25T10:07:07.664212+0000","last_active":"2024-04-25T10:07:07.671738+0000","last_peered":"2024-04-25T10:07:07.671738+0000","last_clean":"2024-04-25T10:07:07.671738+0000","last_became_active":"2024-04-25T10:07:07.663582+0000","last_became_peered":"2024-04-25T10:07:07.663582+0000","last_unstale":"2024-04-25T10:07:07.671738+0000","last_undegraded":"2024-04-25T10:07:07.671738+0000","last_fullsized":"2024-04-25T10:07:07.671738+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:53.347594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.632969+0000","last_change":"2024-04-25T10:07:07.632969+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.632969+0000","last_undegraded":"2024-04-25T10:07:07.632969+0000","last_fullsized":"2024-04-25T10:07:07.632969+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.611929+0000","last_change":"2024-04-25T10:07:06.611929+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.611929+0000","last_undegraded":"2024-04-25T10:07:06.611929+0000","last_fullsized":"2024-04-25T10:07:06.611929+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.615190+0000","last_change":"2024-04-25T10:07:06.615190+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.615190+0000","last_undegraded":"2024-04-25T10:07:06.615190+0000","last_fullsized":"2024-04-25T10:07:06.615190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663496+0000","last_change":"2024-04-25T10:07:07.657378+0000","last_active":"2024-04-25T10:07:07.663496+0000","last_peered":"2024-04-25T10:07:07.663496+0000","last_clean":"2024-04-25T10:07:07.663496+0000","last_became_active":"2024-04-25T10:07:07.656779+0000","last_became_peered":"2024-04-25T10:07:07.656779+0000","last_unstale":"2024-04-25T10:07:07.663496+0000","last_undegraded":"2024-04-25T10:07:07.663496+0000","last_fullsized":"2024-04-25T10:07:07.663496+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:52.281359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614255+0000","last_change":"2024-04-25T10:07:06.614255+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614255+0000","last_undegraded":"2024-04-25T10:07:06.614255+0000","last_fullsized":"2024-04-25T10:07:06.614255+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.627083+0000","last_change":"2024-04-25T10:07:07.627083+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.627083+0000","last_undegraded":"2024-04-25T10:07:07.627083+0000","last_fullsized":"2024-04-25T10:07:07.627083+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.618847+0000","last_change":"2024-04-25T10:07:06.618847+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.618847+0000","last_undegraded":"2024-04-25T10:07:06.618847+0000","last_fullsized":"2024-04-25T10:07:06.618847+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.611910+0000","last_change":"2024-04-25T10:07:06.611910+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.611910+0000","last_undegraded":"2024-04-25T10:07:06.611910+0000","last_fullsized":"2024-04-25T10:07:06.611910+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663314+0000","last_change":"2024-04-25T10:07:07.655902+0000","last_active":"2024-04-25T10:07:07.663314+0000","last_peered":"2024-04-25T10:07:07.663314+0000","last_clean":"2024-04-25T10:07:07.663314+0000","last_became_active":"2024-04-25T10:07:07.654431+0000","last_became_peered":"2024-04-25T10:07:07.654431+0000","last_unstale":"2024-04-25T10:07:07.663314+0000","last_undegraded":"2024-04-25T10:07:07.663314+0000","last_fullsized":"2024-04-25T10:07:07.663314+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:04.957714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614950+0000","last_change":"2024-04-25T10:07:06.614950+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614950+0000","last_undegraded":"2024-04-25T10:07:06.614950+0000","last_fullsized":"2024-04-25T10:07:06.614950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614239+0000","last_change":"2024-04-25T10:07:06.614239+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614239+0000","last_undegraded":"2024-04-25T10:07:06.614239+0000","last_fullsized":"2024-04-25T10:07:06.614239+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614217+0000","last_change":"2024-04-25T10:07:06.614217+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614217+0000","last_undegraded":"2024-04-25T10:07:06.614217+0000","last_fullsized":"2024-04-25T10:07:06.614217+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614114+0000","last_change":"2024-04-25T10:07:06.614114+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614114+0000","last_undegraded":"2024-04-25T10:07:06.614114+0000","last_fullsized":"2024-04-25T10:07:06.614114+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.618157+0000","last_change":"2024-04-25T10:07:06.618157+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.618157+0000","last_undegraded":"2024-04-25T10:07:06.618157+0000","last_fullsized":"2024-04-25T10:07:06.618157+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663454+0000","last_change":"2024-04-25T10:07:07.651442+0000","last_active":"2024-04-25T10:07:07.663454+0000","last_peered":"2024-04-25T10:07:07.663454+0000","last_clean":"2024-04-25T10:07:07.663454+0000","last_became_active":"2024-04-25T10:07:07.649340+0000","last_became_peered":"2024-04-25T10:07:07.649340+0000","last_unstale":"2024-04-25T10:07:07.663454+0000","last_undegraded":"2024-04-25T10:07:07.663454+0000","last_fullsized":"2024-04-25T10:07:07.663454+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:35.334997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614607+0000","last_change":"2024-04-25T10:07:06.614607+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614607+0000","last_undegraded":"2024-04-25T10:07:06.614607+0000","last_fullsized":"2024-04-25T10:07:06.614607+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.671762+0000","last_change":"2024-04-25T10:07:07.665347+0000","last_active":"2024-04-25T10:07:07.671762+0000","last_peered":"2024-04-25T10:07:07.671762+0000","last_clean":"2024-04-25T10:07:07.671762+0000","last_became_active":"2024-04-25T10:07:07.664822+0000","last_became_peered":"2024-04-25T10:07:07.664822+0000","last_unstale":"2024-04-25T10:07:07.671762+0000","last_undegraded":"2024-04-25T10:07:07.671762+0000","last_fullsized":"2024-04-25T10:07:07.671762+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:55.111760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.631949+0000","last_change":"2024-04-25T10:07:07.631949+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.631949+0000","last_undegraded":"2024-04-25T10:07:07.631949+0000","last_fullsized":"2024-04-25T10:07:07.631949+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.615594+0000","last_change":"2024-04-25T10:07:06.615594+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.615594+0000","last_undegraded":"2024-04-25T10:07:06.615594+0000","last_fullsized":"2024-04-25T10:07:06.615594+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.668623+0000","last_change":"2024-04-25T10:07:07.661444+0000","last_active":"2024-04-25T10:07:07.668623+0000","last_peered":"2024-04-25T10:07:07.668623+0000","last_clean":"2024-04-25T10:07:07.668623+0000","last_became_active":"2024-04-25T10:07:07.660752+0000","last_became_peered":"2024-04-25T10:07:07.660752+0000","last_unstale":"2024-04-25T10:07:07.668623+0000","last_undegraded":"2024-04-25T10:07:07.668623+0000","last_fullsized":"2024-04-25T10:07:07.668623+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:50.282234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.631316+0000","last_change":"2024-04-25T10:07:07.628649+0000","last_active":"2024-04-25T10:07:07.631316+0000","last_peered":"2024-04-25T10:07:07.631316+0000","last_clean":"2024-04-25T10:07:07.631316+0000","last_became_active":"2024-04-25T10:07:07.627827+0000","last_became_peered":"2024-04-25T10:07:07.627827+0000","last_unstale":"2024-04-25T10:07:07.631316+0000","last_undegraded":"2024-04-25T10:07:07.631316+0000","last_fullsized":"2024-04-25T10:07:07.631316+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:00.780462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.616512+0000","last_change":"2024-04-25T10:07:06.616512+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.616512+0000","last_undegraded":"2024-04-25T10:07:06.616512+0000","last_fullsized":"2024-04-25T10:07:06.616512+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.618043+0000","last_change":"2024-04-25T10:07:07.618043+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.618043+0000","last_undegraded":"2024-04-25T10:07:07.618043+0000","last_fullsized":"2024-04-25T10:07:07.618043+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.618127+0000","last_change":"2024-04-25T10:07:06.618127+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.618127+0000","last_undegraded":"2024-04-25T10:07:06.618127+0000","last_fullsized":"2024-04-25T10:07:06.618127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614925+0000","last_change":"2024-04-25T10:07:06.614925+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614925+0000","last_undegraded":"2024-04-25T10:07:06.614925+0000","last_fullsized":"2024-04-25T10:07:06.614925+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":212,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.635994+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:07:07.635994+0000","last_peered":"2024-04-25T10:07:07.635994+0000","last_clean":"2024-04-25T10:07:07.635994+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:07:07.635994+0000","last_undegraded":"2024-04-25T10:07:07.635994+0000","last_fullsized":"2024-04-25T10:07:07.635994+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":28,"acting":28,"num_store_stats":2},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":799844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509467,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":209476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":203209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":203209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542171,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":203209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":203209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35824,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336016,"statfs":{"total":96636764160,"available":96600080384,"internally_reserved":0,"allocated":495616,"data_stored":197392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1101824,"data_stored":793577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:11.677 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.831+0000 7fe9ff2b0640 1 Processor -- start 2024-04-25T10:07:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.831+0000 7fe9ff2b0640 1 -- start start 2024-04-25T10:07:11.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.831+0000 7fe9ff2b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9f80b4660 0x7fe9f815c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9ff2b0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f8071ca0 con 0x7fe9f80ce7a0 2024-04-25T10:07:11.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9ff2b0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f80cca40 con 0x7fe9f815bc90 2024-04-25T10:07:11.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9ff2b0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f80cc370 con 0x7fe9f80b4660 2024-04-25T10:07:11.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9fd84d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9f80b4660 0x7fe9f815c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9fd84d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9f80b4660 0x7fe9f815c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39064/0 (socket says 172.21.15.60:39064) 2024-04-25T10:07:11.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9fd84d640 1 -- 172.21.15.60:0/2000835137 learned_addr learned my addr 172.21.15.60:0/2000835137 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:11.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9fd04c640 1 --1- 172.21.15.60:0/2000835137 >> v1:172.21.15.60:6789/0 conn(0x7fe9f80ce7a0 0x7fe9f80ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60060/0 (socket says 172.21.15.60:60060) 2024-04-25T10:07:11.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9fd84d640 1 -- 172.21.15.60:0/2000835137 >> v1:172.21.15.100:6789/0 conn(0x7fe9f815bc90 legacy=0x7fe9f815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9fd84d640 1 -- 172.21.15.60:0/2000835137 >> v1:172.21.15.60:6789/0 conn(0x7fe9f80ce7a0 legacy=0x7fe9f80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9fd84d640 1 -- 172.21.15.60:0/2000835137 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f80ccd20 con 0x7fe9f80b4660 2024-04-25T10:07:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9fd84d640 1 --2- 172.21.15.60:0/2000835137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f80b4660 0x7fe9f815c930 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009870 tx=0x7fe9f001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c6156d56399d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9effff640 1 -- 172.21.15.60:0/2000835137 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00b0070 con 0x7fe9f80b4660 2024-04-25T10:07:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9effff640 1 -- 172.21.15.60:0/2000835137 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f00ab600 con 0x7fe9f80b4660 2024-04-25T10:07:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9effff640 1 -- 172.21.15.60:0/2000835137 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00ab9d0 con 0x7fe9f80b4660 2024-04-25T10:07:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9ff2b0640 1 -- 172.21.15.60:0/2000835137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f80b4660 msgr2=0x7fe9f815c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9ff2b0640 1 --2- 172.21.15.60:0/2000835137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f80b4660 0x7fe9f815c930 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009870 tx=0x7fe9f001c520 comp rx=0 tx=0).stop 2024-04-25T10:07:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9ff2b0640 1 -- 172.21.15.60:0/2000835137 shutdown_connections 2024-04-25T10:07:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9ff2b0640 1 --2- 172.21.15.60:0/2000835137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f80b4660 0x7fe9f815c930 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9ff2b0640 1 -- 172.21.15.60:0/2000835137 >> 172.21.15.60:0/2000835137 conn(0x7fe9f80b12d0 msgr2=0x7fe9f80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9ff2b0640 1 -- 172.21.15.60:0/2000835137 shutdown_connections 2024-04-25T10:07:11.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.835+0000 7fe9ff2b0640 1 -- 172.21.15.60:0/2000835137 wait complete. 2024-04-25T10:07:11.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9ff2b0640 1 Processor -- start 2024-04-25T10:07:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9ff2b0640 1 -- start start 2024-04-25T10:07:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9ff2b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80ce7a0 0x7fe9f8153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9ff2b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f815bc90 0x7fe9f81575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9ff2b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f81fd6d0 0x7fe9f81fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9ff2b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f81695d0 con 0x7fe9f81fd6d0 2024-04-25T10:07:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9ff2b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80b3db0 con 0x7fe9f815bc90 2024-04-25T10:07:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9ff2b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f8169750 con 0x7fe9f80ce7a0 2024-04-25T10:07:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9fd84d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f81fd6d0 0x7fe9f81fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9fd84d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f81fd6d0 0x7fe9f81fdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39076/0 (socket says 172.21.15.60:39076) 2024-04-25T10:07:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9fd84d640 1 -- 172.21.15.60:0/2330476059 learned_addr learned my addr 172.21.15.60:0/2330476059 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9fd04c640 1 --2- 172.21.15.60:0/2330476059 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80ce7a0 0x7fe9f8153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9fc84b640 1 --2- 172.21.15.60:0/2330476059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f815bc90 0x7fe9f81575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9fd84d640 1 -- 172.21.15.60:0/2330476059 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80ce7a0 msgr2=0x7fe9f8153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9fd84d640 1 --2- 172.21.15.60:0/2330476059 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80ce7a0 0x7fe9f8153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9fd84d640 1 -- 172.21.15.60:0/2330476059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f815bc90 msgr2=0x7fe9f81575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9fd84d640 1 --2- 172.21.15.60:0/2330476059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f815bc90 0x7fe9f81575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9fd84d640 1 -- 172.21.15.60:0/2330476059 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f8153540 con 0x7fe9f81fd6d0 2024-04-25T10:07:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9fc84b640 1 --2- 172.21.15.60:0/2330476059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f815bc90 0x7fe9f81575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9fd84d640 1 --2- 172.21.15.60:0/2330476059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f81fd6d0 0x7fe9f81fdb80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe9f001abd0 tx=0x7fe9f815d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9edffb640 1 -- 172.21.15.60:0/2330476059 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00b0040 con 0x7fe9f81fd6d0 2024-04-25T10:07:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9edffb640 1 -- 172.21.15.60:0/2330476059 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f00ba070 con 0x7fe9f81fd6d0 2024-04-25T10:07:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9edffb640 1 -- 172.21.15.60:0/2330476059 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00b4870 con 0x7fe9f81fd6d0 2024-04-25T10:07:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9ff2b0640 1 -- 172.21.15.60:0/2330476059 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f8151ab0 con 0x7fe9f81fd6d0 2024-04-25T10:07:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9ff2b0640 1 -- 172.21.15.60:0/2330476059 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f8152070 con 0x7fe9f81fd6d0 2024-04-25T10:07:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9ff2b0640 1 -- 172.21.15.60:0/2330476059 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f8074770 con 0x7fe9f81fd6d0 2024-04-25T10:07:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.839+0000 7fe9edffb640 1 -- 172.21.15.60:0/2330476059 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9f00ab4c0 con 0x7fe9f81fd6d0 2024-04-25T10:07:11.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.843+0000 7fe9edffb640 1 --2- 172.21.15.60:0/2330476059 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f60 0x7fe9dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.843+0000 7fe9edffb640 1 -- 172.21.15.60:0/2330476059 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9f011e590 con 0x7fe9f81fd6d0 2024-04-25T10:07:11.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.843+0000 7fe9fd04c640 1 --2- 172.21.15.60:0/2330476059 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f60 0x7fe9dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.843+0000 7fe9fd04c640 1 --2- 172.21.15.60:0/2330476059 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f60 0x7fe9dc069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4007fd0 tx=0x7fe9f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.847+0000 7fe9edffb640 1 -- 172.21.15.60:0/2330476059 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9f00c3020 con 0x7fe9f81fd6d0 2024-04-25T10:07:12.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:11.995+0000 7fe9ff2b0640 1 -- 172.21.15.60:0/2330476059 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe9f81543a0 con 0x7fe9dc066f60 2024-04-25T10:07:12.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.011+0000 7fe9edffb640 1 -- 172.21.15.60:0/2330476059 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+241671 (secure 0 0 0) 0x7fe9f81543a0 con 0x7fe9dc066f60 2024-04-25T10:07:12.020 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:12.021 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:07:12.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.015+0000 7fe9ff2b0640 1 -- 172.21.15.60:0/2330476059 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f60 msgr2=0x7fe9dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.015+0000 7fe9ff2b0640 1 --2- 172.21.15.60:0/2330476059 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f60 0x7fe9dc069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4007fd0 tx=0x7fe9f4009040 comp rx=0 tx=0).stop 2024-04-25T10:07:12.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.015+0000 7fe9ff2b0640 1 -- 172.21.15.60:0/2330476059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f81fd6d0 msgr2=0x7fe9f81fdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.015+0000 7fe9ff2b0640 1 --2- 172.21.15.60:0/2330476059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f81fd6d0 0x7fe9f81fdb80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe9f001abd0 tx=0x7fe9f815d680 comp rx=0 tx=0).stop 2024-04-25T10:07:12.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.015+0000 7fe9ff2b0640 1 -- 172.21.15.60:0/2330476059 shutdown_connections 2024-04-25T10:07:12.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.015+0000 7fe9ff2b0640 1 --2- 172.21.15.60:0/2330476059 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f60 0x7fe9dc069420 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.015+0000 7fe9ff2b0640 1 --2- 172.21.15.60:0/2330476059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f81fd6d0 0x7fe9f81fdb80 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.015+0000 7fe9ff2b0640 1 --2- 172.21.15.60:0/2330476059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f815bc90 0x7fe9f81575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.015+0000 7fe9ff2b0640 1 --2- 172.21.15.60:0/2330476059 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80ce7a0 0x7fe9f8153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.015+0000 7fe9ff2b0640 1 -- 172.21.15.60:0/2330476059 >> 172.21.15.60:0/2330476059 conn(0x7fe9f80b12d0 msgr2=0x7fe9f815c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.015+0000 7fe9ff2b0640 1 -- 172.21.15.60:0/2330476059 shutdown_connections 2024-04-25T10:07:12.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.015+0000 7fe9ff2b0640 1 -- 172.21.15.60:0/2330476059 wait complete. 2024-04-25T10:07:12.037 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-04-25T10:07:10.425464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":94,"acting":94,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":246264,"kb_used_data":5140,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754728456,"statfs":{"total":773094113280,"available":772841938944,"internally_reserved":0,"allocated":5263360,"data_stored":2813125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001273"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.628546+0000","last_change":"2024-04-25T10:07:07.628546+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.628546+0000","last_undegraded":"2024-04-25T10:07:07.628546+0000","last_fullsized":"2024-04-25T10:07:07.628546+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.623001+0000","last_change":"2024-04-25T10:07:07.623001+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.623001+0000","last_undegraded":"2024-04-25T10:07:07.623001+0000","last_fullsized":"2024-04-25T10:07:07.623001+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.628165+0000","last_change":"2024-04-25T10:07:07.628165+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.628165+0000","last_undegraded":"2024-04-25T10:07:07.628165+0000","last_fullsized":"2024-04-25T10:07:07.628165+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.624555+0000","last_change":"2024-04-25T10:07:07.624555+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.624555+0000","last_undegraded":"2024-04-25T10:07:07.624555+0000","last_fullsized":"2024-04-25T10:07:07.624555+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.623659+0000","last_change":"2024-04-25T10:07:07.623659+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.623659+0000","last_undegraded":"2024-04-25T10:07:07.623659+0000","last_fullsized":"2024-04-25T10:07:07.623659+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.619319+0000","last_change":"2024-04-25T10:07:07.619319+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.619319+0000","last_undegraded":"2024-04-25T10:07:07.619319+0000","last_fullsized":"2024-04-25T10:07:07.619319+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.620746+0000","last_change":"2024-04-25T10:07:07.620746+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.620746+0000","last_undegraded":"2024-04-25T10:07:07.620746+0000","last_fullsized":"2024-04-25T10:07:07.620746+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614524+0000","last_change":"2024-04-25T10:07:06.614524+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614524+0000","last_undegraded":"2024-04-25T10:07:06.614524+0000","last_fullsized":"2024-04-25T10:07:06.614524+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663264+0000","last_change":"2024-04-25T10:07:07.655701+0000","last_active":"2024-04-25T10:07:07.663264+0000","last_peered":"2024-04-25T10:07:07.663264+0000","last_clean":"2024-04-25T10:07:07.663264+0000","last_became_active":"2024-04-25T10:07:07.653511+0000","last_became_peered":"2024-04-25T10:07:07.653511+0000","last_unstale":"2024-04-25T10:07:07.663264+0000","last_undegraded":"2024-04-25T10:07:07.663264+0000","last_fullsized":"2024-04-25T10:07:07.663264+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:56.489633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.615716+0000","last_change":"2024-04-25T10:07:06.615716+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.615716+0000","last_undegraded":"2024-04-25T10:07:06.615716+0000","last_fullsized":"2024-04-25T10:07:06.615716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.668582+0000","last_change":"2024-04-25T10:07:07.661239+0000","last_active":"2024-04-25T10:07:07.668582+0000","last_peered":"2024-04-25T10:07:07.668582+0000","last_clean":"2024-04-25T10:07:07.668582+0000","last_became_active":"2024-04-25T10:07:07.660654+0000","last_became_peered":"2024-04-25T10:07:07.660654+0000","last_unstale":"2024-04-25T10:07:07.668582+0000","last_undegraded":"2024-04-25T10:07:07.668582+0000","last_fullsized":"2024-04-25T10:07:07.668582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:59.863704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.619660+0000","last_change":"2024-04-25T10:07:07.619660+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.619660+0000","last_undegraded":"2024-04-25T10:07:07.619660+0000","last_fullsized":"2024-04-25T10:07:07.619660+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.615803+0000","last_change":"2024-04-25T10:07:06.615803+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.615803+0000","last_undegraded":"2024-04-25T10:07:06.615803+0000","last_fullsized":"2024-04-25T10:07:06.615803+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663157+0000","last_change":"2024-04-25T10:07:07.657392+0000","last_active":"2024-04-25T10:07:07.663157+0000","last_peered":"2024-04-25T10:07:07.663157+0000","last_clean":"2024-04-25T10:07:07.663157+0000","last_became_active":"2024-04-25T10:07:07.656810+0000","last_became_peered":"2024-04-25T10:07:07.656810+0000","last_unstale":"2024-04-25T10:07:07.663157+0000","last_undegraded":"2024-04-25T10:07:07.663157+0000","last_fullsized":"2024-04-25T10:07:07.663157+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:16.056727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.613378+0000","last_change":"2024-04-25T10:07:06.613378+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.613378+0000","last_undegraded":"2024-04-25T10:07:06.613378+0000","last_fullsized":"2024-04-25T10:07:06.613378+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.619506+0000","last_change":"2024-04-25T10:07:06.619506+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.619506+0000","last_undegraded":"2024-04-25T10:07:06.619506+0000","last_fullsized":"2024-04-25T10:07:06.619506+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.622656+0000","last_change":"2024-04-25T10:07:07.622656+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.622656+0000","last_undegraded":"2024-04-25T10:07:07.622656+0000","last_fullsized":"2024-04-25T10:07:07.622656+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.658561+0000","last_change":"2024-04-25T10:07:07.639391+0000","last_active":"2024-04-25T10:07:07.658561+0000","last_peered":"2024-04-25T10:07:07.658561+0000","last_clean":"2024-04-25T10:07:07.658561+0000","last_became_active":"2024-04-25T10:07:07.638645+0000","last_became_peered":"2024-04-25T10:07:07.638645+0000","last_unstale":"2024-04-25T10:07:07.658561+0000","last_undegraded":"2024-04-25T10:07:07.658561+0000","last_fullsized":"2024-04-25T10:07:07.658561+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:42.132953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.627466+0000","last_change":"2024-04-25T10:07:07.627466+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.627466+0000","last_undegraded":"2024-04-25T10:07:07.627466+0000","last_fullsized":"2024-04-25T10:07:07.627466+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.618710+0000","last_change":"2024-04-25T10:07:06.618710+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.618710+0000","last_undegraded":"2024-04-25T10:07:06.618710+0000","last_fullsized":"2024-04-25T10:07:06.618710+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.671738+0000","last_change":"2024-04-25T10:07:07.664212+0000","last_active":"2024-04-25T10:07:07.671738+0000","last_peered":"2024-04-25T10:07:07.671738+0000","last_clean":"2024-04-25T10:07:07.671738+0000","last_became_active":"2024-04-25T10:07:07.663582+0000","last_became_peered":"2024-04-25T10:07:07.663582+0000","last_unstale":"2024-04-25T10:07:07.671738+0000","last_undegraded":"2024-04-25T10:07:07.671738+0000","last_fullsized":"2024-04-25T10:07:07.671738+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:53.347594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.632969+0000","last_change":"2024-04-25T10:07:07.632969+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.632969+0000","last_undegraded":"2024-04-25T10:07:07.632969+0000","last_fullsized":"2024-04-25T10:07:07.632969+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.611929+0000","last_change":"2024-04-25T10:07:06.611929+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.611929+0000","last_undegraded":"2024-04-25T10:07:06.611929+0000","last_fullsized":"2024-04-25T10:07:06.611929+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.615190+0000","last_change":"2024-04-25T10:07:06.615190+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.615190+0000","last_undegraded":"2024-04-25T10:07:06.615190+0000","last_fullsized":"2024-04-25T10:07:06.615190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663496+0000","last_change":"2024-04-25T10:07:07.657378+0000","last_active":"2024-04-25T10:07:07.663496+0000","last_peered":"2024-04-25T10:07:07.663496+0000","last_clean":"2024-04-25T10:07:07.663496+0000","last_became_active":"2024-04-25T10:07:07.656779+0000","last_became_peered":"2024-04-25T10:07:07.656779+0000","last_unstale":"2024-04-25T10:07:07.663496+0000","last_undegraded":"2024-04-25T10:07:07.663496+0000","last_fullsized":"2024-04-25T10:07:07.663496+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:52.281359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614255+0000","last_change":"2024-04-25T10:07:06.614255+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614255+0000","last_undegraded":"2024-04-25T10:07:06.614255+0000","last_fullsized":"2024-04-25T10:07:06.614255+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.627083+0000","last_change":"2024-04-25T10:07:07.627083+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.627083+0000","last_undegraded":"2024-04-25T10:07:07.627083+0000","last_fullsized":"2024-04-25T10:07:07.627083+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.618847+0000","last_change":"2024-04-25T10:07:06.618847+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.618847+0000","last_undegraded":"2024-04-25T10:07:06.618847+0000","last_fullsized":"2024-04-25T10:07:06.618847+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.611910+0000","last_change":"2024-04-25T10:07:06.611910+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.611910+0000","last_undegraded":"2024-04-25T10:07:06.611910+0000","last_fullsized":"2024-04-25T10:07:06.611910+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663314+0000","last_change":"2024-04-25T10:07:07.655902+0000","last_active":"2024-04-25T10:07:07.663314+0000","last_peered":"2024-04-25T10:07:07.663314+0000","last_clean":"2024-04-25T10:07:07.663314+0000","last_became_active":"2024-04-25T10:07:07.654431+0000","last_became_peered":"2024-04-25T10:07:07.654431+0000","last_unstale":"2024-04-25T10:07:07.663314+0000","last_undegraded":"2024-04-25T10:07:07.663314+0000","last_fullsized":"2024-04-25T10:07:07.663314+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:04.957714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614950+0000","last_change":"2024-04-25T10:07:06.614950+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614950+0000","last_undegraded":"2024-04-25T10:07:06.614950+0000","last_fullsized":"2024-04-25T10:07:06.614950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614239+0000","last_change":"2024-04-25T10:07:06.614239+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614239+0000","last_undegraded":"2024-04-25T10:07:06.614239+0000","last_fullsized":"2024-04-25T10:07:06.614239+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614217+0000","last_change":"2024-04-25T10:07:06.614217+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614217+0000","last_undegraded":"2024-04-25T10:07:06.614217+0000","last_fullsized":"2024-04-25T10:07:06.614217+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614114+0000","last_change":"2024-04-25T10:07:06.614114+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614114+0000","last_undegraded":"2024-04-25T10:07:06.614114+0000","last_fullsized":"2024-04-25T10:07:06.614114+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.618157+0000","last_change":"2024-04-25T10:07:06.618157+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.618157+0000","last_undegraded":"2024-04-25T10:07:06.618157+0000","last_fullsized":"2024-04-25T10:07:06.618157+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663454+0000","last_change":"2024-04-25T10:07:07.651442+0000","last_active":"2024-04-25T10:07:07.663454+0000","last_peered":"2024-04-25T10:07:07.663454+0000","last_clean":"2024-04-25T10:07:07.663454+0000","last_became_active":"2024-04-25T10:07:07.649340+0000","last_became_peered":"2024-04-25T10:07:07.649340+0000","last_unstale":"2024-04-25T10:07:07.663454+0000","last_undegraded":"2024-04-25T10:07:07.663454+0000","last_fullsized":"2024-04-25T10:07:07.663454+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:35.334997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614607+0000","last_change":"2024-04-25T10:07:06.614607+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614607+0000","last_undegraded":"2024-04-25T10:07:06.614607+0000","last_fullsized":"2024-04-25T10:07:06.614607+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.671762+0000","last_change":"2024-04-25T10:07:07.665347+0000","last_active":"2024-04-25T10:07:07.671762+0000","last_peered":"2024-04-25T10:07:07.671762+0000","last_clean":"2024-04-25T10:07:07.671762+0000","last_became_active":"2024-04-25T10:07:07.664822+0000","last_became_peered":"2024-04-25T10:07:07.664822+0000","last_unstale":"2024-04-25T10:07:07.671762+0000","last_undegraded":"2024-04-25T10:07:07.671762+0000","last_fullsized":"2024-04-25T10:07:07.671762+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:55.111760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.631949+0000","last_change":"2024-04-25T10:07:07.631949+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.631949+0000","last_undegraded":"2024-04-25T10:07:07.631949+0000","last_fullsized":"2024-04-25T10:07:07.631949+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.615594+0000","last_change":"2024-04-25T10:07:06.615594+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.615594+0000","last_undegraded":"2024-04-25T10:07:06.615594+0000","last_fullsized":"2024-04-25T10:07:06.615594+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.668623+0000","last_change":"2024-04-25T10:07:07.661444+0000","last_active":"2024-04-25T10:07:07.668623+0000","last_peered":"2024-04-25T10:07:07.668623+0000","last_clean":"2024-04-25T10:07:07.668623+0000","last_became_active":"2024-04-25T10:07:07.660752+0000","last_became_peered":"2024-04-25T10:07:07.660752+0000","last_unstale":"2024-04-25T10:07:07.668623+0000","last_undegraded":"2024-04-25T10:07:07.668623+0000","last_fullsized":"2024-04-25T10:07:07.668623+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:50.282234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.631316+0000","last_change":"2024-04-25T10:07:07.628649+0000","last_active":"2024-04-25T10:07:07.631316+0000","last_peered":"2024-04-25T10:07:07.631316+0000","last_clean":"2024-04-25T10:07:07.631316+0000","last_became_active":"2024-04-25T10:07:07.627827+0000","last_became_peered":"2024-04-25T10:07:07.627827+0000","last_unstale":"2024-04-25T10:07:07.631316+0000","last_undegraded":"2024-04-25T10:07:07.631316+0000","last_fullsized":"2024-04-25T10:07:07.631316+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:00.780462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.616512+0000","last_change":"2024-04-25T10:07:06.616512+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.616512+0000","last_undegraded":"2024-04-25T10:07:06.616512+0000","last_fullsized":"2024-04-25T10:07:06.616512+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.618043+0000","last_change":"2024-04-25T10:07:07.618043+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.618043+0000","last_undegraded":"2024-04-25T10:07:07.618043+0000","last_fullsized":"2024-04-25T10:07:07.618043+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.618127+0000","last_change":"2024-04-25T10:07:06.618127+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.618127+0000","last_undegraded":"2024-04-25T10:07:06.618127+0000","last_fullsized":"2024-04-25T10:07:06.618127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614925+0000","last_change":"2024-04-25T10:07:06.614925+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614925+0000","last_undegraded":"2024-04-25T10:07:06.614925+0000","last_fullsized":"2024-04-25T10:07:06.614925+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":212,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.635994+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:07:07.635994+0000","last_peered":"2024-04-25T10:07:07.635994+0000","last_clean":"2024-04-25T10:07:07.635994+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:07:07.635994+0000","last_undegraded":"2024-04-25T10:07:07.635994+0000","last_fullsized":"2024-04-25T10:07:07.635994+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":28,"acting":28,"num_store_stats":2},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":799844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509467,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":209476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":203209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":203209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542171,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":203209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":203209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35824,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336016,"statfs":{"total":96636764160,"available":96600080384,"internally_reserved":0,"allocated":495616,"data_stored":197392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1101824,"data_stored":793577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:12.041 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:12.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec3fc4640 1 Processor -- start 2024-04-25T10:07:12.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec3fc4640 1 -- start start 2024-04-25T10:07:12.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec3fc4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcebc15deb0 0x7fcebc0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec3fc4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcebc0d0340 0x7fcebc0d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec3fc4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcebc0ce2b0 0x7fcebc0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec3fc4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcebc0cf2c0 con 0x7fcebc0ce2b0 2024-04-25T10:07:12.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec3fc4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcebc0cefe0 con 0x7fcebc15deb0 2024-04-25T10:07:12.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec3fc4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc0d1d70 con 0x7fcebc0d0340 2024-04-25T10:07:12.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec2561640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcebc0ce2b0 0x7fcebc0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec2561640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcebc0ce2b0 0x7fcebc0ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39082/0 (socket says 172.21.15.60:39082) 2024-04-25T10:07:12.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec2561640 1 -- 172.21.15.60:0/2901978920 learned_addr learned my addr 172.21.15.60:0/2901978920 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:12.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec1d60640 1 --2- 172.21.15.60:0/2901978920 >> v2:172.21.15.100:3300/0 conn(0x7fcebc15deb0 0x7fcebc0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec155f640 1 --2- 172.21.15.60:0/2901978920 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcebc0d0340 0x7fcebc0d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec2561640 1 -- 172.21.15.60:0/2901978920 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcebc0d0340 msgr2=0x7fcebc0d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:12.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec2561640 1 --2- 172.21.15.60:0/2901978920 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcebc0d0340 0x7fcebc0d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec2561640 1 -- 172.21.15.60:0/2901978920 >> v2:172.21.15.100:3300/0 conn(0x7fcebc15deb0 msgr2=0x7fcebc0b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:12.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec2561640 1 --2- 172.21.15.60:0/2901978920 >> v2:172.21.15.100:3300/0 conn(0x7fcebc15deb0 0x7fcebc0b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.195+0000 7fcec2561640 1 -- 172.21.15.60:0/2901978920 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc06d200 con 0x7fcebc0ce2b0 2024-04-25T10:07:12.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec2561640 1 --2- 172.21.15.60:0/2901978920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcebc0ce2b0 0x7fcebc0ce740 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcebc08bd20 tx=0x7fceb401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb563494c3d63f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec0d5e640 1 -- 172.21.15.60:0/2901978920 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb40a5630 con 0x7fcebc0ce2b0 2024-04-25T10:07:12.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec0d5e640 1 -- 172.21.15.60:0/2901978920 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb40a57d0 con 0x7fcebc0ce2b0 2024-04-25T10:07:12.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec0d5e640 1 -- 172.21.15.60:0/2901978920 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb40a5ba0 con 0x7fcebc0ce2b0 2024-04-25T10:07:12.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 -- 172.21.15.60:0/2901978920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcebc0ce2b0 msgr2=0x7fcebc0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 --2- 172.21.15.60:0/2901978920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcebc0ce2b0 0x7fcebc0ce740 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcebc08bd20 tx=0x7fceb401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 -- 172.21.15.60:0/2901978920 shutdown_connections 2024-04-25T10:07:12.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 --2- 172.21.15.60:0/2901978920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcebc0ce2b0 0x7fcebc0ce740 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 --2- 172.21.15.60:0/2901978920 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcebc0d0340 0x7fcebc0d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 --2- 172.21.15.60:0/2901978920 >> v2:172.21.15.100:3300/0 conn(0x7fcebc15deb0 0x7fcebc0b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 -- 172.21.15.60:0/2901978920 >> 172.21.15.60:0/2901978920 conn(0x7fcebc06c090 msgr2=0x7fcebc06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 -- 172.21.15.60:0/2901978920 shutdown_connections 2024-04-25T10:07:12.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 -- 172.21.15.60:0/2901978920 wait complete. 2024-04-25T10:07:12.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 Processor -- start 2024-04-25T10:07:12.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 -- start start 2024-04-25T10:07:12.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcebc0ce2b0 0x7fcebc149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcebc0d0340 0x7fcebc0d3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcebc15deb0 0x7fcebc0d37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc0b5d40 con 0x7fcebc15deb0 2024-04-25T10:07:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc0b5bc0 con 0x7fcebc0ce2b0 2024-04-25T10:07:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec3fc4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc0b5ec0 con 0x7fcebc0d0340 2024-04-25T10:07:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec2561640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcebc15deb0 0x7fcebc0d37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec2561640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcebc15deb0 0x7fcebc0d37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39098/0 (socket says 172.21.15.60:39098) 2024-04-25T10:07:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec2561640 1 -- 172.21.15.60:0/1728621216 learned_addr learned my addr 172.21.15.60:0/1728621216 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec1d60640 1 --2- 172.21.15.60:0/1728621216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcebc0ce2b0 0x7fcebc149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.199+0000 7fcec155f640 1 --2- 172.21.15.60:0/1728621216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcebc0d0340 0x7fcebc0d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fcec2561640 1 -- 172.21.15.60:0/1728621216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcebc0d0340 msgr2=0x7fcebc0d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fcec2561640 1 --2- 172.21.15.60:0/1728621216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcebc0d0340 0x7fcebc0d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fcec2561640 1 -- 172.21.15.60:0/1728621216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcebc0ce2b0 msgr2=0x7fcebc149960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fcec2561640 1 --2- 172.21.15.60:0/1728621216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcebc0ce2b0 0x7fcebc149960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fcec2561640 1 -- 172.21.15.60:0/1728621216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc0cb580 con 0x7fcebc15deb0 2024-04-25T10:07:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fcec1d60640 1 --2- 172.21.15.60:0/1728621216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcebc0ce2b0 0x7fcebc149960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:12.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fcec2561640 1 --2- 172.21.15.60:0/1728621216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcebc15deb0 0x7fcebc0d37a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fceb401d290 tx=0x7fceb4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fceb27fc640 1 -- 172.21.15.60:0/1728621216 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4009510 con 0x7fcebc15deb0 2024-04-25T10:07:12.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fceb27fc640 1 -- 172.21.15.60:0/1728621216 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb40be070 con 0x7fcebc15deb0 2024-04-25T10:07:12.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fceb27fc640 1 -- 172.21.15.60:0/1728621216 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb40b8890 con 0x7fcebc15deb0 2024-04-25T10:07:12.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fcec3fc4640 1 -- 172.21.15.60:0/1728621216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcebc0d3dd0 con 0x7fcebc15deb0 2024-04-25T10:07:12.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fcec3fc4640 1 -- 172.21.15.60:0/1728621216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcebc0d4390 con 0x7fcebc15deb0 2024-04-25T10:07:12.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fcec3fc4640 1 -- 172.21.15.60:0/1728621216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcebc0cb750 con 0x7fcebc15deb0 2024-04-25T10:07:12.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fceb27fc640 1 -- 172.21.15.60:0/1728621216 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fceb40b8a30 con 0x7fcebc15deb0 2024-04-25T10:07:12.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fceb27fc640 1 --2- 172.21.15.60:0/1728621216 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcea0066f60 0x7fcea0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fceb27fc640 1 -- 172.21.15.60:0/1728621216 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fceb41203d0 con 0x7fcebc15deb0 2024-04-25T10:07:12.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.203+0000 7fcec1d60640 1 --2- 172.21.15.60:0/1728621216 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcea0066f60 0x7fcea0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.207+0000 7fcec1d60640 1 --2- 172.21.15.60:0/1728621216 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcea0066f60 0x7fcea0069420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fceac00b9f0 tx=0x7fceac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.207+0000 7fceb27fc640 1 -- 172.21.15.60:0/1728621216 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fceb40165e0 con 0x7fcebc15deb0 2024-04-25T10:07:12.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.363+0000 7fcec3fc4640 1 -- 172.21.15.60:0/1728621216 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcebc074770 con 0x7fcea0066f60 2024-04-25T10:07:12.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.379+0000 7fceb27fc640 1 -- 172.21.15.60:0/1728621216 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+241671 (secure 0 0 0) 0x7fcebc074770 con 0x7fcea0066f60 2024-04-25T10:07:12.390 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:12.391 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:07:12.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.383+0000 7fcec3fc4640 1 -- 172.21.15.60:0/1728621216 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcea0066f60 msgr2=0x7fcea0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.383+0000 7fcec3fc4640 1 --2- 172.21.15.60:0/1728621216 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcea0066f60 0x7fcea0069420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fceac00b9f0 tx=0x7fceac008040 comp rx=0 tx=0).stop 2024-04-25T10:07:12.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.383+0000 7fcec3fc4640 1 -- 172.21.15.60:0/1728621216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcebc15deb0 msgr2=0x7fcebc0d37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.383+0000 7fcec3fc4640 1 --2- 172.21.15.60:0/1728621216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcebc15deb0 0x7fcebc0d37a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fceb401d290 tx=0x7fceb4019790 comp rx=0 tx=0).stop 2024-04-25T10:07:12.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.383+0000 7fcec3fc4640 1 -- 172.21.15.60:0/1728621216 shutdown_connections 2024-04-25T10:07:12.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.383+0000 7fcec3fc4640 1 --2- 172.21.15.60:0/1728621216 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcea0066f60 0x7fcea0069420 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.383+0000 7fcec3fc4640 1 --2- 172.21.15.60:0/1728621216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcebc15deb0 0x7fcebc0d37a0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.383+0000 7fcec3fc4640 1 --2- 172.21.15.60:0/1728621216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcebc0d0340 0x7fcebc0d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.383+0000 7fcec3fc4640 1 --2- 172.21.15.60:0/1728621216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcebc0ce2b0 0x7fcebc149960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.383+0000 7fcec3fc4640 1 -- 172.21.15.60:0/1728621216 >> 172.21.15.60:0/1728621216 conn(0x7fcebc06c090 msgr2=0x7fcebc153050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.383+0000 7fcec3fc4640 1 -- 172.21.15.60:0/1728621216 shutdown_connections 2024-04-25T10:07:12.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.383+0000 7fcec3fc4640 1 -- 172.21.15.60:0/1728621216 wait complete. 2024-04-25T10:07:12.409 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-04-25T10:07:10.425464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":94,"acting":94,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":246264,"kb_used_data":5140,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754728456,"statfs":{"total":773094113280,"available":772841938944,"internally_reserved":0,"allocated":5263360,"data_stored":2813125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001273"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.628546+0000","last_change":"2024-04-25T10:07:07.628546+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.628546+0000","last_undegraded":"2024-04-25T10:07:07.628546+0000","last_fullsized":"2024-04-25T10:07:07.628546+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.623001+0000","last_change":"2024-04-25T10:07:07.623001+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.623001+0000","last_undegraded":"2024-04-25T10:07:07.623001+0000","last_fullsized":"2024-04-25T10:07:07.623001+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.628165+0000","last_change":"2024-04-25T10:07:07.628165+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.628165+0000","last_undegraded":"2024-04-25T10:07:07.628165+0000","last_fullsized":"2024-04-25T10:07:07.628165+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.624555+0000","last_change":"2024-04-25T10:07:07.624555+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.624555+0000","last_undegraded":"2024-04-25T10:07:07.624555+0000","last_fullsized":"2024-04-25T10:07:07.624555+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.623659+0000","last_change":"2024-04-25T10:07:07.623659+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.623659+0000","last_undegraded":"2024-04-25T10:07:07.623659+0000","last_fullsized":"2024-04-25T10:07:07.623659+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.619319+0000","last_change":"2024-04-25T10:07:07.619319+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.619319+0000","last_undegraded":"2024-04-25T10:07:07.619319+0000","last_fullsized":"2024-04-25T10:07:07.619319+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.620746+0000","last_change":"2024-04-25T10:07:07.620746+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.620746+0000","last_undegraded":"2024-04-25T10:07:07.620746+0000","last_fullsized":"2024-04-25T10:07:07.620746+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614524+0000","last_change":"2024-04-25T10:07:06.614524+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614524+0000","last_undegraded":"2024-04-25T10:07:06.614524+0000","last_fullsized":"2024-04-25T10:07:06.614524+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663264+0000","last_change":"2024-04-25T10:07:07.655701+0000","last_active":"2024-04-25T10:07:07.663264+0000","last_peered":"2024-04-25T10:07:07.663264+0000","last_clean":"2024-04-25T10:07:07.663264+0000","last_became_active":"2024-04-25T10:07:07.653511+0000","last_became_peered":"2024-04-25T10:07:07.653511+0000","last_unstale":"2024-04-25T10:07:07.663264+0000","last_undegraded":"2024-04-25T10:07:07.663264+0000","last_fullsized":"2024-04-25T10:07:07.663264+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:56.489633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.615716+0000","last_change":"2024-04-25T10:07:06.615716+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.615716+0000","last_undegraded":"2024-04-25T10:07:06.615716+0000","last_fullsized":"2024-04-25T10:07:06.615716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.668582+0000","last_change":"2024-04-25T10:07:07.661239+0000","last_active":"2024-04-25T10:07:07.668582+0000","last_peered":"2024-04-25T10:07:07.668582+0000","last_clean":"2024-04-25T10:07:07.668582+0000","last_became_active":"2024-04-25T10:07:07.660654+0000","last_became_peered":"2024-04-25T10:07:07.660654+0000","last_unstale":"2024-04-25T10:07:07.668582+0000","last_undegraded":"2024-04-25T10:07:07.668582+0000","last_fullsized":"2024-04-25T10:07:07.668582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:59.863704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.619660+0000","last_change":"2024-04-25T10:07:07.619660+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.619660+0000","last_undegraded":"2024-04-25T10:07:07.619660+0000","last_fullsized":"2024-04-25T10:07:07.619660+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.615803+0000","last_change":"2024-04-25T10:07:06.615803+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.615803+0000","last_undegraded":"2024-04-25T10:07:06.615803+0000","last_fullsized":"2024-04-25T10:07:06.615803+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663157+0000","last_change":"2024-04-25T10:07:07.657392+0000","last_active":"2024-04-25T10:07:07.663157+0000","last_peered":"2024-04-25T10:07:07.663157+0000","last_clean":"2024-04-25T10:07:07.663157+0000","last_became_active":"2024-04-25T10:07:07.656810+0000","last_became_peered":"2024-04-25T10:07:07.656810+0000","last_unstale":"2024-04-25T10:07:07.663157+0000","last_undegraded":"2024-04-25T10:07:07.663157+0000","last_fullsized":"2024-04-25T10:07:07.663157+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:16.056727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.613378+0000","last_change":"2024-04-25T10:07:06.613378+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.613378+0000","last_undegraded":"2024-04-25T10:07:06.613378+0000","last_fullsized":"2024-04-25T10:07:06.613378+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.619506+0000","last_change":"2024-04-25T10:07:06.619506+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.619506+0000","last_undegraded":"2024-04-25T10:07:06.619506+0000","last_fullsized":"2024-04-25T10:07:06.619506+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.622656+0000","last_change":"2024-04-25T10:07:07.622656+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.622656+0000","last_undegraded":"2024-04-25T10:07:07.622656+0000","last_fullsized":"2024-04-25T10:07:07.622656+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.658561+0000","last_change":"2024-04-25T10:07:07.639391+0000","last_active":"2024-04-25T10:07:07.658561+0000","last_peered":"2024-04-25T10:07:07.658561+0000","last_clean":"2024-04-25T10:07:07.658561+0000","last_became_active":"2024-04-25T10:07:07.638645+0000","last_became_peered":"2024-04-25T10:07:07.638645+0000","last_unstale":"2024-04-25T10:07:07.658561+0000","last_undegraded":"2024-04-25T10:07:07.658561+0000","last_fullsized":"2024-04-25T10:07:07.658561+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:42.132953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.627466+0000","last_change":"2024-04-25T10:07:07.627466+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.627466+0000","last_undegraded":"2024-04-25T10:07:07.627466+0000","last_fullsized":"2024-04-25T10:07:07.627466+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.618710+0000","last_change":"2024-04-25T10:07:06.618710+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.618710+0000","last_undegraded":"2024-04-25T10:07:06.618710+0000","last_fullsized":"2024-04-25T10:07:06.618710+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.671738+0000","last_change":"2024-04-25T10:07:07.664212+0000","last_active":"2024-04-25T10:07:07.671738+0000","last_peered":"2024-04-25T10:07:07.671738+0000","last_clean":"2024-04-25T10:07:07.671738+0000","last_became_active":"2024-04-25T10:07:07.663582+0000","last_became_peered":"2024-04-25T10:07:07.663582+0000","last_unstale":"2024-04-25T10:07:07.671738+0000","last_undegraded":"2024-04-25T10:07:07.671738+0000","last_fullsized":"2024-04-25T10:07:07.671738+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:53.347594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.632969+0000","last_change":"2024-04-25T10:07:07.632969+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.632969+0000","last_undegraded":"2024-04-25T10:07:07.632969+0000","last_fullsized":"2024-04-25T10:07:07.632969+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.611929+0000","last_change":"2024-04-25T10:07:06.611929+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.611929+0000","last_undegraded":"2024-04-25T10:07:06.611929+0000","last_fullsized":"2024-04-25T10:07:06.611929+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.615190+0000","last_change":"2024-04-25T10:07:06.615190+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.615190+0000","last_undegraded":"2024-04-25T10:07:06.615190+0000","last_fullsized":"2024-04-25T10:07:06.615190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663496+0000","last_change":"2024-04-25T10:07:07.657378+0000","last_active":"2024-04-25T10:07:07.663496+0000","last_peered":"2024-04-25T10:07:07.663496+0000","last_clean":"2024-04-25T10:07:07.663496+0000","last_became_active":"2024-04-25T10:07:07.656779+0000","last_became_peered":"2024-04-25T10:07:07.656779+0000","last_unstale":"2024-04-25T10:07:07.663496+0000","last_undegraded":"2024-04-25T10:07:07.663496+0000","last_fullsized":"2024-04-25T10:07:07.663496+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:52.281359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614255+0000","last_change":"2024-04-25T10:07:06.614255+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614255+0000","last_undegraded":"2024-04-25T10:07:06.614255+0000","last_fullsized":"2024-04-25T10:07:06.614255+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.627083+0000","last_change":"2024-04-25T10:07:07.627083+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.627083+0000","last_undegraded":"2024-04-25T10:07:07.627083+0000","last_fullsized":"2024-04-25T10:07:07.627083+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.618847+0000","last_change":"2024-04-25T10:07:06.618847+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.618847+0000","last_undegraded":"2024-04-25T10:07:06.618847+0000","last_fullsized":"2024-04-25T10:07:06.618847+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.611910+0000","last_change":"2024-04-25T10:07:06.611910+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.611910+0000","last_undegraded":"2024-04-25T10:07:06.611910+0000","last_fullsized":"2024-04-25T10:07:06.611910+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663314+0000","last_change":"2024-04-25T10:07:07.655902+0000","last_active":"2024-04-25T10:07:07.663314+0000","last_peered":"2024-04-25T10:07:07.663314+0000","last_clean":"2024-04-25T10:07:07.663314+0000","last_became_active":"2024-04-25T10:07:07.654431+0000","last_became_peered":"2024-04-25T10:07:07.654431+0000","last_unstale":"2024-04-25T10:07:07.663314+0000","last_undegraded":"2024-04-25T10:07:07.663314+0000","last_fullsized":"2024-04-25T10:07:07.663314+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:04.957714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614950+0000","last_change":"2024-04-25T10:07:06.614950+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614950+0000","last_undegraded":"2024-04-25T10:07:06.614950+0000","last_fullsized":"2024-04-25T10:07:06.614950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614239+0000","last_change":"2024-04-25T10:07:06.614239+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614239+0000","last_undegraded":"2024-04-25T10:07:06.614239+0000","last_fullsized":"2024-04-25T10:07:06.614239+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614217+0000","last_change":"2024-04-25T10:07:06.614217+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614217+0000","last_undegraded":"2024-04-25T10:07:06.614217+0000","last_fullsized":"2024-04-25T10:07:06.614217+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614114+0000","last_change":"2024-04-25T10:07:06.614114+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614114+0000","last_undegraded":"2024-04-25T10:07:06.614114+0000","last_fullsized":"2024-04-25T10:07:06.614114+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.618157+0000","last_change":"2024-04-25T10:07:06.618157+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.618157+0000","last_undegraded":"2024-04-25T10:07:06.618157+0000","last_fullsized":"2024-04-25T10:07:06.618157+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663454+0000","last_change":"2024-04-25T10:07:07.651442+0000","last_active":"2024-04-25T10:07:07.663454+0000","last_peered":"2024-04-25T10:07:07.663454+0000","last_clean":"2024-04-25T10:07:07.663454+0000","last_became_active":"2024-04-25T10:07:07.649340+0000","last_became_peered":"2024-04-25T10:07:07.649340+0000","last_unstale":"2024-04-25T10:07:07.663454+0000","last_undegraded":"2024-04-25T10:07:07.663454+0000","last_fullsized":"2024-04-25T10:07:07.663454+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:35.334997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614607+0000","last_change":"2024-04-25T10:07:06.614607+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614607+0000","last_undegraded":"2024-04-25T10:07:06.614607+0000","last_fullsized":"2024-04-25T10:07:06.614607+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.671762+0000","last_change":"2024-04-25T10:07:07.665347+0000","last_active":"2024-04-25T10:07:07.671762+0000","last_peered":"2024-04-25T10:07:07.671762+0000","last_clean":"2024-04-25T10:07:07.671762+0000","last_became_active":"2024-04-25T10:07:07.664822+0000","last_became_peered":"2024-04-25T10:07:07.664822+0000","last_unstale":"2024-04-25T10:07:07.671762+0000","last_undegraded":"2024-04-25T10:07:07.671762+0000","last_fullsized":"2024-04-25T10:07:07.671762+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:55.111760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.631949+0000","last_change":"2024-04-25T10:07:07.631949+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.631949+0000","last_undegraded":"2024-04-25T10:07:07.631949+0000","last_fullsized":"2024-04-25T10:07:07.631949+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.615594+0000","last_change":"2024-04-25T10:07:06.615594+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.615594+0000","last_undegraded":"2024-04-25T10:07:06.615594+0000","last_fullsized":"2024-04-25T10:07:06.615594+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.668623+0000","last_change":"2024-04-25T10:07:07.661444+0000","last_active":"2024-04-25T10:07:07.668623+0000","last_peered":"2024-04-25T10:07:07.668623+0000","last_clean":"2024-04-25T10:07:07.668623+0000","last_became_active":"2024-04-25T10:07:07.660752+0000","last_became_peered":"2024-04-25T10:07:07.660752+0000","last_unstale":"2024-04-25T10:07:07.668623+0000","last_undegraded":"2024-04-25T10:07:07.668623+0000","last_fullsized":"2024-04-25T10:07:07.668623+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:50.282234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.631316+0000","last_change":"2024-04-25T10:07:07.628649+0000","last_active":"2024-04-25T10:07:07.631316+0000","last_peered":"2024-04-25T10:07:07.631316+0000","last_clean":"2024-04-25T10:07:07.631316+0000","last_became_active":"2024-04-25T10:07:07.627827+0000","last_became_peered":"2024-04-25T10:07:07.627827+0000","last_unstale":"2024-04-25T10:07:07.631316+0000","last_undegraded":"2024-04-25T10:07:07.631316+0000","last_fullsized":"2024-04-25T10:07:07.631316+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:00.780462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.616512+0000","last_change":"2024-04-25T10:07:06.616512+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.616512+0000","last_undegraded":"2024-04-25T10:07:06.616512+0000","last_fullsized":"2024-04-25T10:07:06.616512+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.618043+0000","last_change":"2024-04-25T10:07:07.618043+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.618043+0000","last_undegraded":"2024-04-25T10:07:07.618043+0000","last_fullsized":"2024-04-25T10:07:07.618043+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.618127+0000","last_change":"2024-04-25T10:07:06.618127+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.618127+0000","last_undegraded":"2024-04-25T10:07:06.618127+0000","last_fullsized":"2024-04-25T10:07:06.618127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:07:07.606865+0000","last_change":"2024-04-25T10:07:07.606865+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.606865+0000","last_undegraded":"2024-04-25T10:07:07.606865+0000","last_fullsized":"2024-04-25T10:07:07.606865+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-04-25T10:07:06.614925+0000","last_change":"2024-04-25T10:07:06.614925+0000","last_active":"2024-04-25T10:07:06.598164+0000","last_peered":"2024-04-25T10:07:06.598164+0000","last_clean":"2024-04-25T10:07:06.598164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:06.614925+0000","last_undegraded":"2024-04-25T10:07:06.614925+0000","last_fullsized":"2024-04-25T10:07:06.614925+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":212,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.635994+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:07:07.635994+0000","last_peered":"2024-04-25T10:07:07.635994+0000","last_clean":"2024-04-25T10:07:07.635994+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:07:07.635994+0000","last_undegraded":"2024-04-25T10:07:07.635994+0000","last_fullsized":"2024-04-25T10:07:07.635994+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":28,"acting":28,"num_store_stats":2},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":799844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509467,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":209476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":203209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":203209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542171,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":203209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":203209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35824,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336016,"statfs":{"total":96636764160,"available":96600080384,"internally_reserved":0,"allocated":495616,"data_stored":197392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1101824,"data_stored":793577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:12.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2024-04-25T10:07:12.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:07.628546+0000', 'last_change': '2024-04-25T10:07:07.628546+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.628546+0000', 'last_undegraded': '2024-04-25T10:07:07.628546+0000', 'last_fullsized': '2024-04-25T10:07:07.628546+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:07:12.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2024-04-25T10:07:12.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2024-04-25T10:07:12.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:07.623001+0000', 'last_change': '2024-04-25T10:07:07.623001+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.623001+0000', 'last_undegraded': '2024-04-25T10:07:07.623001+0000', 'last_fullsized': '2024-04-25T10:07:07.623001+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:07:12.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2024-04-25T10:07:12.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2024-04-25T10:07:12.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2024-04-25T10:07:12.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2024-04-25T10:07:12.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:07.628165+0000', 'last_change': '2024-04-25T10:07:07.628165+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.628165+0000', 'last_undegraded': '2024-04-25T10:07:07.628165+0000', 'last_fullsized': '2024-04-25T10:07:07.628165+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:07:12.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2024-04-25T10:07:12.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2024-04-25T10:07:12.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2024-04-25T10:07:12.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2024-04-25T10:07:12.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2024-04-25T10:07:12.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2024-04-25T10:07:12.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2024-04-25T10:07:12.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2024-04-25T10:07:12.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2024-04-25T10:07:12.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2024-04-25T10:07:12.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:07.624555+0000', 'last_change': '2024-04-25T10:07:07.624555+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.624555+0000', 'last_undegraded': '2024-04-25T10:07:07.624555+0000', 'last_fullsized': '2024-04-25T10:07:07.624555+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:07:12.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2024-04-25T10:07:12.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:07.623659+0000', 'last_change': '2024-04-25T10:07:07.623659+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.623659+0000', 'last_undegraded': '2024-04-25T10:07:07.623659+0000', 'last_fullsized': '2024-04-25T10:07:07.623659+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:07:12.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2024-04-25T10:07:12.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2024-04-25T10:07:12.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2024-04-25T10:07:12.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2024-04-25T10:07:12.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2024-04-25T10:07:12.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2024-04-25T10:07:12.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2024-04-25T10:07:12.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2024-04-25T10:07:12.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2024-04-25T10:07:12.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:07.619319+0000', 'last_change': '2024-04-25T10:07:07.619319+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.619319+0000', 'last_undegraded': '2024-04-25T10:07:07.619319+0000', 'last_fullsized': '2024-04-25T10:07:07.619319+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:07:12.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2024-04-25T10:07:12.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2024-04-25T10:07:12.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2024-04-25T10:07:12.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:07.620746+0000', 'last_change': '2024-04-25T10:07:07.620746+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.620746+0000', 'last_undegraded': '2024-04-25T10:07:07.620746+0000', 'last_fullsized': '2024-04-25T10:07:07.620746+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:07:12.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2024-04-25T10:07:12.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2024-04-25T10:07:12.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-04-25T10:07:12.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-04-25T10:07:12.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.614524+0000', 'last_change': '2024-04-25T10:07:06.614524+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.614524+0000', 'last_undegraded': '2024-04-25T10:07:06.614524+0000', 'last_fullsized': '2024-04-25T10:07:06.614524+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:07:12.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-04-25T10:07:12.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-04-25T10:07:12.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-04-25T10:07:12.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.615716+0000', 'last_change': '2024-04-25T10:07:06.615716+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.615716+0000', 'last_undegraded': '2024-04-25T10:07:06.615716+0000', 'last_fullsized': '2024-04-25T10:07:06.615716+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:07:12.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-04-25T10:07:12.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-04-25T10:07:12.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:07.619660+0000', 'last_change': '2024-04-25T10:07:07.619660+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.619660+0000', 'last_undegraded': '2024-04-25T10:07:07.619660+0000', 'last_fullsized': '2024-04-25T10:07:07.619660+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:07:12.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-04-25T10:07:12.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.615803+0000', 'last_change': '2024-04-25T10:07:06.615803+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.615803+0000', 'last_undegraded': '2024-04-25T10:07:06.615803+0000', 'last_fullsized': '2024-04-25T10:07:06.615803+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:07:12.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2024-04-25T10:07:12.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2024-04-25T10:07:12.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-04-25T10:07:12.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.613378+0000', 'last_change': '2024-04-25T10:07:06.613378+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.613378+0000', 'last_undegraded': '2024-04-25T10:07:06.613378+0000', 'last_fullsized': '2024-04-25T10:07:06.613378+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:07:12.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-04-25T10:07:12.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-04-25T10:07:12.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.619506+0000', 'last_change': '2024-04-25T10:07:06.619506+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.619506+0000', 'last_undegraded': '2024-04-25T10:07:06.619506+0000', 'last_fullsized': '2024-04-25T10:07:06.619506+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:07:12.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-04-25T10:07:12.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:07.622656+0000', 'last_change': '2024-04-25T10:07:07.622656+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.622656+0000', 'last_undegraded': '2024-04-25T10:07:07.622656+0000', 'last_fullsized': '2024-04-25T10:07:07.622656+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:07:12.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-04-25T10:07:12.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:07.627466+0000', 'last_change': '2024-04-25T10:07:07.627466+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.627466+0000', 'last_undegraded': '2024-04-25T10:07:07.627466+0000', 'last_fullsized': '2024-04-25T10:07:07.627466+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:07:12.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-04-25T10:07:12.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.618710+0000', 'last_change': '2024-04-25T10:07:06.618710+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.618710+0000', 'last_undegraded': '2024-04-25T10:07:06.618710+0000', 'last_fullsized': '2024-04-25T10:07:06.618710+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:07:12.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-04-25T10:07:12.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-04-25T10:07:12.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:07.632969+0000', 'last_change': '2024-04-25T10:07:07.632969+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.632969+0000', 'last_undegraded': '2024-04-25T10:07:07.632969+0000', 'last_fullsized': '2024-04-25T10:07:07.632969+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:07:12.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-04-25T10:07:12.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.611929+0000', 'last_change': '2024-04-25T10:07:06.611929+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.611929+0000', 'last_undegraded': '2024-04-25T10:07:06.611929+0000', 'last_fullsized': '2024-04-25T10:07:06.611929+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:07:12.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-04-25T10:07:12.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-04-25T10:07:12.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.615190+0000', 'last_change': '2024-04-25T10:07:06.615190+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.615190+0000', 'last_undegraded': '2024-04-25T10:07:06.615190+0000', 'last_fullsized': '2024-04-25T10:07:06.615190+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:07:12.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2024-04-25T10:07:12.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-04-25T10:07:12.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-04-25T10:07:12.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.614255+0000', 'last_change': '2024-04-25T10:07:06.614255+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.614255+0000', 'last_undegraded': '2024-04-25T10:07:06.614255+0000', 'last_fullsized': '2024-04-25T10:07:06.614255+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:07:12.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-04-25T10:07:12.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:07.627083+0000', 'last_change': '2024-04-25T10:07:07.627083+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.627083+0000', 'last_undegraded': '2024-04-25T10:07:07.627083+0000', 'last_fullsized': '2024-04-25T10:07:07.627083+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:07:12.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-04-25T10:07:12.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.618847+0000', 'last_change': '2024-04-25T10:07:06.618847+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.618847+0000', 'last_undegraded': '2024-04-25T10:07:06.618847+0000', 'last_fullsized': '2024-04-25T10:07:06.618847+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:07:12.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-04-25T10:07:12.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-04-25T10:07:12.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.611910+0000', 'last_change': '2024-04-25T10:07:06.611910+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.611910+0000', 'last_undegraded': '2024-04-25T10:07:06.611910+0000', 'last_fullsized': '2024-04-25T10:07:06.611910+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:07:12.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2024-04-25T10:07:12.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-04-25T10:07:12.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-04-25T10:07:12.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.614950+0000', 'last_change': '2024-04-25T10:07:06.614950+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.614950+0000', 'last_undegraded': '2024-04-25T10:07:06.614950+0000', 'last_fullsized': '2024-04-25T10:07:06.614950+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:07:12.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-04-25T10:07:12.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-04-25T10:07:12.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.614239+0000', 'last_change': '2024-04-25T10:07:06.614239+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.614239+0000', 'last_undegraded': '2024-04-25T10:07:06.614239+0000', 'last_fullsized': '2024-04-25T10:07:06.614239+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:07:12.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-04-25T10:07:12.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-04-25T10:07:12.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.614217+0000', 'last_change': '2024-04-25T10:07:06.614217+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.614217+0000', 'last_undegraded': '2024-04-25T10:07:06.614217+0000', 'last_fullsized': '2024-04-25T10:07:06.614217+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:07:12.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-04-25T10:07:12.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-04-25T10:07:12.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.614114+0000', 'last_change': '2024-04-25T10:07:06.614114+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.614114+0000', 'last_undegraded': '2024-04-25T10:07:06.614114+0000', 'last_fullsized': '2024-04-25T10:07:06.614114+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:07:12.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-04-25T10:07:12.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-04-25T10:07:12.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.618157+0000', 'last_change': '2024-04-25T10:07:06.618157+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.618157+0000', 'last_undegraded': '2024-04-25T10:07:06.618157+0000', 'last_fullsized': '2024-04-25T10:07:06.618157+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:07:12.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-04-25T10:07:12.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-04-25T10:07:12.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-04-25T10:07:12.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.614607+0000', 'last_change': '2024-04-25T10:07:06.614607+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.614607+0000', 'last_undegraded': '2024-04-25T10:07:06.614607+0000', 'last_fullsized': '2024-04-25T10:07:06.614607+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:07:12.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2024-04-25T10:07:12.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-04-25T10:07:12.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:07.631949+0000', 'last_change': '2024-04-25T10:07:07.631949+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.631949+0000', 'last_undegraded': '2024-04-25T10:07:07.631949+0000', 'last_fullsized': '2024-04-25T10:07:07.631949+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:07:12.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-04-25T10:07:12.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.615594+0000', 'last_change': '2024-04-25T10:07:06.615594+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.615594+0000', 'last_undegraded': '2024-04-25T10:07:06.615594+0000', 'last_fullsized': '2024-04-25T10:07:06.615594+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:07:12.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-04-25T10:07:12.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-04-25T10:07:12.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-04-25T10:07:12.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-04-25T10:07:12.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.616512+0000', 'last_change': '2024-04-25T10:07:06.616512+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.616512+0000', 'last_undegraded': '2024-04-25T10:07:06.616512+0000', 'last_fullsized': '2024-04-25T10:07:06.616512+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:07:12.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-04-25T10:07:12.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:07.618043+0000', 'last_change': '2024-04-25T10:07:07.618043+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.618043+0000', 'last_undegraded': '2024-04-25T10:07:07.618043+0000', 'last_fullsized': '2024-04-25T10:07:07.618043+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:07:12.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-04-25T10:07:12.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.618127+0000', 'last_change': '2024-04-25T10:07:06.618127+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.618127+0000', 'last_undegraded': '2024-04-25T10:07:06.618127+0000', 'last_fullsized': '2024-04-25T10:07:06.618127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:07:12.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-04-25T10:07:12.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:07:07.606865+0000', 'last_change': '2024-04-25T10:07:07.606865+0000', 'last_active': '2024-04-25T10:07:07.606865+0000', 'last_peered': '2024-04-25T10:07:07.606865+0000', 'last_clean': '2024-04-25T10:07:07.606865+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:07.606865+0000', 'last_undegraded': '2024-04-25T10:07:07.606865+0000', 'last_fullsized': '2024-04-25T10:07:07.606865+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:07.606865+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:07:12.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-04-25T10:07:12.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:07:06.614925+0000', 'last_change': '2024-04-25T10:07:06.614925+0000', 'last_active': '2024-04-25T10:07:06.598164+0000', 'last_peered': '2024-04-25T10:07:06.598164+0000', 'last_clean': '2024-04-25T10:07:06.598164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:07:06.614925+0000', 'last_undegraded': '2024-04-25T10:07:06.614925+0000', 'last_fullsized': '2024-04-25T10:07:06.614925+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:07:06.598164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:07:12.429 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:07:12.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.571+0000 7f76b754c640 1 Processor -- start 2024-04-25T10:07:12.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.571+0000 7f76b754c640 1 -- start start 2024-04-25T10:07:12.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.575+0000 7f76b754c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76b015dd20 0x7f76b00d32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.575+0000 7f76b754c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b00b5e00 0x7f76b00b9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.575+0000 7f76b754c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b00d1fb0 con 0x7f76b00d1b50 2024-04-25T10:07:12.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.575+0000 7f76b754c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b00d2250 con 0x7f76b015dd20 2024-04-25T10:07:12.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.575+0000 7f76b754c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b00d2530 con 0x7f76b00b5e00 2024-04-25T10:07:12.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.575+0000 7f76b52e8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76b015dd20 0x7f76b00d32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.575+0000 7f76b52e8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76b015dd20 0x7f76b00d32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39114/0 (socket says 172.21.15.60:39114) 2024-04-25T10:07:12.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.575+0000 7f76b52e8640 1 -- 172.21.15.60:0/1806742611 learned_addr learned my addr 172.21.15.60:0/1806742611 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:12.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.575+0000 7f76b5ae9640 1 --2- 172.21.15.60:0/1806742611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b00b5e00 0x7f76b00b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.575+0000 7f76b52e8640 1 -- 172.21.15.60:0/1806742611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b00b5e00 msgr2=0x7f76b00b9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:12.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.575+0000 7f76b52e8640 1 --2- 172.21.15.60:0/1806742611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b00b5e00 0x7f76b00b9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.575+0000 7f76b52e8640 1 -- 172.21.15.60:0/1806742611 >> v1:172.21.15.100:6789/0 conn(0x7f76b00d1b50 legacy=0x7f76b00b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:12.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.575+0000 7f76b52e8640 1 -- 172.21.15.60:0/1806742611 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b00d2810 con 0x7f76b015dd20 2024-04-25T10:07:12.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.579+0000 7f76b52e8640 1 --2- 172.21.15.60:0/1806742611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b015dd20 0x7f76b00d32f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009e50 tx=0x7f76a801f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b284c00b88f55b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.579+0000 7f76affff640 1 -- 172.21.15.60:0/1806742611 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a80b4070 con 0x7f76b015dd20 2024-04-25T10:07:12.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.579+0000 7f76affff640 1 -- 172.21.15.60:0/1806742611 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a80af400 con 0x7f76b015dd20 2024-04-25T10:07:12.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.579+0000 7f76affff640 1 -- 172.21.15.60:0/1806742611 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a80af7d0 con 0x7f76b015dd20 2024-04-25T10:07:12.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.579+0000 7f76b754c640 1 -- 172.21.15.60:0/1806742611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b015dd20 msgr2=0x7f76b00d32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.579+0000 7f76b754c640 1 --2- 172.21.15.60:0/1806742611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b015dd20 0x7f76b00d32f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009e50 tx=0x7f76a801f210 comp rx=0 tx=0).stop 2024-04-25T10:07:12.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.579+0000 7f76b754c640 1 -- 172.21.15.60:0/1806742611 shutdown_connections 2024-04-25T10:07:12.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.579+0000 7f76b754c640 1 --2- 172.21.15.60:0/1806742611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b00b5e00 0x7f76b00b9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.579+0000 7f76b754c640 1 --2- 172.21.15.60:0/1806742611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b015dd20 0x7f76b00d32f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.579+0000 7f76b754c640 1 -- 172.21.15.60:0/1806742611 >> 172.21.15.60:0/1806742611 conn(0x7f76b00d07a0 msgr2=0x7f76b00d0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.579+0000 7f76b754c640 1 -- 172.21.15.60:0/1806742611 shutdown_connections 2024-04-25T10:07:12.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.579+0000 7f76b754c640 1 -- 172.21.15.60:0/1806742611 wait complete. 2024-04-25T10:07:12.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.579+0000 7f76b754c640 1 Processor -- start 2024-04-25T10:07:12.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.579+0000 7f76b754c640 1 -- start start 2024-04-25T10:07:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b754c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b00b5e00 0x7f76b01521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b754c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b00d1b50 0x7f76b00c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b754c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b015dd20 0x7f76b00c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b754c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b00b6a50 con 0x7f76b00b5e00 2024-04-25T10:07:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b754c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b00d2dd0 con 0x7f76b015dd20 2024-04-25T10:07:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b754c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b00b6bd0 con 0x7f76b00d1b50 2024-04-25T10:07:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b52e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b00b5e00 0x7f76b01521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b52e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b00b5e00 0x7f76b01521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39130/0 (socket says 172.21.15.60:39130) 2024-04-25T10:07:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b52e8640 1 -- 172.21.15.60:0/326626600 learned_addr learned my addr 172.21.15.60:0/326626600 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b5ae9640 1 --2- 172.21.15.60:0/326626600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b015dd20 0x7f76b00c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b4ae7640 1 --2- 172.21.15.60:0/326626600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b00d1b50 0x7f76b00c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b52e8640 1 -- 172.21.15.60:0/326626600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b00d1b50 msgr2=0x7f76b00c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b52e8640 1 --2- 172.21.15.60:0/326626600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b00d1b50 0x7f76b00c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b52e8640 1 -- 172.21.15.60:0/326626600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b015dd20 msgr2=0x7f76b00c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b52e8640 1 --2- 172.21.15.60:0/326626600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b015dd20 0x7f76b00c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b52e8640 1 -- 172.21.15.60:0/326626600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b0157070 con 0x7f76b00b5e00 2024-04-25T10:07:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b4ae7640 1 --2- 172.21.15.60:0/326626600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b00d1b50 0x7f76b00c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b52e8640 1 --2- 172.21.15.60:0/326626600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b00b5e00 0x7f76b01521b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f76a80b2040 tx=0x7f76a801bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76adffb640 1 -- 172.21.15.60:0/326626600 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a80b4040 con 0x7f76b00b5e00 2024-04-25T10:07:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76adffb640 1 -- 172.21.15.60:0/326626600 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a80c0070 con 0x7f76b00b5e00 2024-04-25T10:07:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b754c640 1 -- 172.21.15.60:0/326626600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b0157350 con 0x7f76b00b5e00 2024-04-25T10:07:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76adffb640 1 -- 172.21.15.60:0/326626600 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8009510 con 0x7f76b00b5e00 2024-04-25T10:07:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b754c640 1 -- 172.21.15.60:0/326626600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b015e5b0 con 0x7f76b00b5e00 2024-04-25T10:07:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76b754c640 1 -- 172.21.15.60:0/326626600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b0074770 con 0x7f76b00b5e00 2024-04-25T10:07:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76adffb640 1 -- 172.21.15.60:0/326626600 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76a80a8d10 con 0x7f76b00b5e00 2024-04-25T10:07:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76adffb640 1 --2- 172.21.15.60:0/326626600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f769c066f60 0x7f769c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.583+0000 7f76adffb640 1 -- 172.21.15.60:0/326626600 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76a8123020 con 0x7f76b00b5e00 2024-04-25T10:07:12.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.587+0000 7f76b4ae7640 1 --2- 172.21.15.60:0/326626600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f769c066f60 0x7f769c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.587+0000 7f76b4ae7640 1 --2- 172.21.15.60:0/326626600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f769c066f60 0x7f769c069420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f76b006cfa0 tx=0x7f76a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.587+0000 7f76adffb640 1 -- 172.21.15.60:0/326626600 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76a8018d70 con 0x7f76b00b5e00 2024-04-25T10:07:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.811+0000 7f76b754c640 1 -- 172.21.15.60:0/326626600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f76b015cc10 con 0x7f76b00b5e00 2024-04-25T10:07:12.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.811+0000 7f76adffb640 1 -- 172.21.15.60:0/326626600 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1518 (secure 0 0 0) 0x7f76a80c3360 con 0x7f76b00b5e00 2024-04-25T10:07:12.818 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:12.818 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":137,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039507,"num_in_osds":8,"osd_in_since":1714039498,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":83},{"state_name":"creating+peering","count":14}],"num_pgs":97,"num_pools":3,"num_objects":20,"data_bytes":592688,"bytes_used":256724992,"bytes_avail":772837388288,"bytes_total":773094113280,"inactive_pgs_ratio":0.14432989060878754,"write_bytes_sec":2288,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":21,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5796}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-04-25T10:06:56.421552+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:07:12.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.815+0000 7f76b754c640 1 -- 172.21.15.60:0/326626600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f769c066f60 msgr2=0x7f769c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.815+0000 7f76b754c640 1 --2- 172.21.15.60:0/326626600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f769c066f60 0x7f769c069420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f76b006cfa0 tx=0x7f76a4009040 comp rx=0 tx=0).stop 2024-04-25T10:07:12.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.815+0000 7f76b754c640 1 -- 172.21.15.60:0/326626600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b00b5e00 msgr2=0x7f76b01521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.815+0000 7f76b754c640 1 --2- 172.21.15.60:0/326626600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b00b5e00 0x7f76b01521b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f76a80b2040 tx=0x7f76a801bfb0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.815+0000 7f76b754c640 1 -- 172.21.15.60:0/326626600 shutdown_connections 2024-04-25T10:07:12.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.815+0000 7f76b754c640 1 --2- 172.21.15.60:0/326626600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f769c066f60 0x7f769c069420 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.815+0000 7f76b754c640 1 --2- 172.21.15.60:0/326626600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b015dd20 0x7f76b00c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.815+0000 7f76b754c640 1 --2- 172.21.15.60:0/326626600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b00d1b50 0x7f76b00c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.815+0000 7f76b754c640 1 --2- 172.21.15.60:0/326626600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b00b5e00 0x7f76b01521b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.815+0000 7f76b754c640 1 -- 172.21.15.60:0/326626600 >> 172.21.15.60:0/326626600 conn(0x7f76b00d07a0 msgr2=0x7f76b00d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.815+0000 7f76b754c640 1 -- 172.21.15.60:0/326626600 shutdown_connections 2024-04-25T10:07:12.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.815+0000 7f76b754c640 1 -- 172.21.15.60:0/326626600 wait complete. 2024-04-25T10:07:12.838 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:07:12.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:12.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.947+0000 7f7d7f200640 1 Processor -- start 2024-04-25T10:07:12.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.947+0000 7f7d7f200640 1 -- start start 2024-04-25T10:07:12.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d7f200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d7815a7a0 0x7f7d780cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d7f200640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7806f130 con 0x7f7d780b37e0 2024-04-25T10:07:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d7f200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7806f410 con 0x7f7d7806ec30 2024-04-25T10:07:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d7f200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d780cdaa0 con 0x7f7d7815a7a0 2024-04-25T10:07:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d7d79d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7d780b37e0 0x7f7d7806dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60062/0 (socket says 172.21.15.60:60062) 2024-04-25T10:07:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d7d79d640 1 -- 172.21.15.60:0/3709669009 learned_addr learned my addr 172.21.15.60:0/3709669009 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d7cf9c640 1 --2- 172.21.15.60:0/3709669009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d7815a7a0 0x7f7d780cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d777fe640 1 -- 172.21.15.60:0/3709669009 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743574877 0 0) 0x7f7d7806f130 con 0x7f7d780b37e0 2024-04-25T10:07:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d777fe640 1 -- 172.21.15.60:0/3709669009 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d60003530 con 0x7f7d780b37e0 2024-04-25T10:07:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d777fe640 1 -- 172.21.15.60:0/3709669009 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3782158246 0 0) 0x7f7d60003530 con 0x7f7d780b37e0 2024-04-25T10:07:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d777fe640 1 -- 172.21.15.60:0/3709669009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d7815a7a0 msgr2=0x7f7d780cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d777fe640 1 --2- 172.21.15.60:0/3709669009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d7815a7a0 0x7f7d780cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d777fe640 1 -- 172.21.15.60:0/3709669009 >> v1:172.21.15.100:6789/0 conn(0x7f7d7806ec30 legacy=0x7f7d78071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d777fe640 1 -- 172.21.15.60:0/3709669009 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7806d540 con 0x7f7d780b37e0 2024-04-25T10:07:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d777fe640 1 -- 172.21.15.60:0/3709669009 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f7d70001a00 con 0x7f7d780b37e0 2024-04-25T10:07:12.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d777fe640 1 -- 172.21.15.60:0/3709669009 >> v1:172.21.15.60:6789/0 conn(0x7f7d780b37e0 legacy=0x7f7d7806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d777fe640 1 --2- 172.21.15.60:0/3709669009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d600039f0 0x7f7d60003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d777fe640 1 --2- 172.21.15.60:0/3709669009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d60004810 0x7f7d60004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d777fe640 1 --2- 172.21.15.60:0/3709669009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d7815a7a0 0x7f7d600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d777fe640 1 -- 172.21.15.60:0/3709669009 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6000ba80 con 0x7f7d60004810 2024-04-25T10:07:12.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d777fe640 1 -- 172.21.15.60:0/3709669009 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6000bc00 con 0x7f7d600039f0 2024-04-25T10:07:12.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d777fe640 1 -- 172.21.15.60:0/3709669009 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6000e010 con 0x7f7d7815a7a0 2024-04-25T10:07:12.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d77fff640 1 --2- 172.21.15.60:0/3709669009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d60004810 0x7f7d60004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d7cf9c640 1 --2- 172.21.15.60:0/3709669009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d600039f0 0x7f7d60003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d7d79d640 1 --2- 172.21.15.60:0/3709669009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d7815a7a0 0x7f7d600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d77fff640 1 -- 172.21.15.60:0/3709669009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d7815a7a0 msgr2=0x7f7d600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:12.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d77fff640 1 --2- 172.21.15.60:0/3709669009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d7815a7a0 0x7f7d600051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d77fff640 1 -- 172.21.15.60:0/3709669009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d600039f0 msgr2=0x7f7d60003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:12.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d77fff640 1 --2- 172.21.15.60:0/3709669009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d600039f0 0x7f7d60003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.951+0000 7f7d77fff640 1 -- 172.21.15.60:0/3709669009 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d60003530 con 0x7f7d60004810 2024-04-25T10:07:12.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d77fff640 1 --2- 172.21.15.60:0/3709669009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d60004810 0x7f7d60004c90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c001750 tx=0x7f7d6c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebed1b3e8791167 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d777fe640 1 -- 172.21.15.60:0/3709669009 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c0013a0 con 0x7f7d60004810 2024-04-25T10:07:12.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d777fe640 1 -- 172.21.15.60:0/3709669009 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6c0052f0 con 0x7f7d60004810 2024-04-25T10:07:12.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d777fe640 1 -- 172.21.15.60:0/3709669009 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c003840 con 0x7f7d60004810 2024-04-25T10:07:12.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7f200640 1 -- 172.21.15.60:0/3709669009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d60004810 msgr2=0x7f7d60004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7f200640 1 --2- 172.21.15.60:0/3709669009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d60004810 0x7f7d60004c90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c001750 tx=0x7f7d6c01c3c0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7d79d640 1 -- 172.21.15.60:0/3709669009 reap_dead start 2024-04-25T10:07:12.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7f200640 1 -- 172.21.15.60:0/3709669009 shutdown_connections 2024-04-25T10:07:12.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7f200640 1 -- 172.21.15.60:0/3709669009 >> 172.21.15.60:0/3709669009 conn(0x7f7d780b12d0 msgr2=0x7f7d780b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7f200640 1 -- 172.21.15.60:0/3709669009 shutdown_connections 2024-04-25T10:07:12.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7f200640 1 -- 172.21.15.60:0/3709669009 wait complete. 2024-04-25T10:07:12.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7f200640 1 Processor -- start 2024-04-25T10:07:12.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7f200640 1 -- start start 2024-04-25T10:07:12.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d60004810 0x7f7d781f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d780ce030 0x7f7d780ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7f200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d780c7f50 0x7f7d780c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7f200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7806d8b0 con 0x7f7d60004810 2024-04-25T10:07:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7806d2f0 con 0x7f7d780ce030 2024-04-25T10:07:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7f200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d780cd560 con 0x7f7d780c7f50 2024-04-25T10:07:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7cf9c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d60004810 0x7f7d781f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7cf9c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d60004810 0x7f7d781f2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39144/0 (socket says 172.21.15.60:39144) 2024-04-25T10:07:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.955+0000 7f7d7cf9c640 1 -- 172.21.15.60:0/3206488069 learned_addr learned my addr 172.21.15.60:0/3206488069 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d77fff640 1 --2- 172.21.15.60:0/3206488069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d780ce030 0x7f7d780ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d7d79d640 1 --2- 172.21.15.60:0/3206488069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d780c7f50 0x7f7d780c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d7cf9c640 1 -- 172.21.15.60:0/3206488069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d780c7f50 msgr2=0x7f7d780c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d7cf9c640 1 --2- 172.21.15.60:0/3206488069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d780c7f50 0x7f7d780c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d7cf9c640 1 -- 172.21.15.60:0/3206488069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d780ce030 msgr2=0x7f7d780ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d7cf9c640 1 --2- 172.21.15.60:0/3206488069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d780ce030 0x7f7d780ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d7cf9c640 1 -- 172.21.15.60:0/3206488069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d781ef130 con 0x7f7d60004810 2024-04-25T10:07:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d7cf9c640 1 --2- 172.21.15.60:0/3206488069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d60004810 0x7f7d781f2ef0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7d680119d0 tx=0x7f7d68011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d757fa640 1 -- 172.21.15.60:0/3206488069 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6800faf0 con 0x7f7d60004810 2024-04-25T10:07:12.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d757fa640 1 -- 172.21.15.60:0/3206488069 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6800fc90 con 0x7f7d60004810 2024-04-25T10:07:12.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d7f200640 1 -- 172.21.15.60:0/3206488069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d781ef410 con 0x7f7d60004810 2024-04-25T10:07:12.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d757fa640 1 -- 172.21.15.60:0/3206488069 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d68014440 con 0x7f7d60004810 2024-04-25T10:07:12.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d7f200640 1 -- 172.21.15.60:0/3206488069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d781ef9d0 con 0x7f7d60004810 2024-04-25T10:07:12.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d7f200640 1 -- 172.21.15.60:0/3206488069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d78074770 con 0x7f7d60004810 2024-04-25T10:07:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d757fa640 1 -- 172.21.15.60:0/3206488069 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d68012040 con 0x7f7d60004810 2024-04-25T10:07:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d757fa640 1 --2- 172.21.15.60:0/3206488069 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7d64066f60 0x7f7d64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d757fa640 1 -- 172.21.15.60:0/3206488069 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d68085650 con 0x7f7d60004810 2024-04-25T10:07:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d77fff640 1 --2- 172.21.15.60:0/3206488069 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7d64066f60 0x7f7d64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.959+0000 7f7d77fff640 1 --2- 172.21.15.60:0/3206488069 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7d64066f60 0x7f7d64069420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7d600057f0 tx=0x7f7d6c0011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:12.963+0000 7f7d757fa640 1 -- 172.21.15.60:0/3206488069 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7d68019030 con 0x7f7d60004810 2024-04-25T10:07:13.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:13.119+0000 7f7d7f200640 1 -- 172.21.15.60:0/3206488069 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d78072100 con 0x7f7d64066f60 2024-04-25T10:07:13.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:13.139+0000 7f7d757fa640 1 -- 172.21.15.60:0/3206488069 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250796 (secure 0 0 0) 0x7f7d78072100 con 0x7f7d64066f60 2024-04-25T10:07:13.149 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:13.150 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:07:13.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:13.143+0000 7f7d7f200640 1 -- 172.21.15.60:0/3206488069 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7d64066f60 msgr2=0x7f7d64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:13.143+0000 7f7d7f200640 1 --2- 172.21.15.60:0/3206488069 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7d64066f60 0x7f7d64069420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7d600057f0 tx=0x7f7d6c0011a0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:13.143+0000 7f7d7f200640 1 -- 172.21.15.60:0/3206488069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d60004810 msgr2=0x7f7d781f2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:13.143+0000 7f7d7f200640 1 --2- 172.21.15.60:0/3206488069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d60004810 0x7f7d781f2ef0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7d680119d0 tx=0x7f7d68011e90 comp rx=0 tx=0).stop 2024-04-25T10:07:13.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:13.143+0000 7f7d7f200640 1 -- 172.21.15.60:0/3206488069 shutdown_connections 2024-04-25T10:07:13.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:13.143+0000 7f7d7f200640 1 --2- 172.21.15.60:0/3206488069 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7d64066f60 0x7f7d64069420 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:13.143+0000 7f7d7f200640 1 --2- 172.21.15.60:0/3206488069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d780c7f50 0x7f7d780c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:13.143+0000 7f7d7f200640 1 --2- 172.21.15.60:0/3206488069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d780ce030 0x7f7d780ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:13.143+0000 7f7d7f200640 1 --2- 172.21.15.60:0/3206488069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d60004810 0x7f7d781f2ef0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:13.143+0000 7f7d7f200640 1 -- 172.21.15.60:0/3206488069 >> 172.21.15.60:0/3206488069 conn(0x7f7d780b12d0 msgr2=0x7f7d78071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:13.143+0000 7f7d7f200640 1 -- 172.21.15.60:0/3206488069 shutdown_connections 2024-04-25T10:07:13.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:13.143+0000 7f7d7f200640 1 -- 172.21.15.60:0/3206488069 wait complete. 2024-04-25T10:07:13.168 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-04-25T10:07:12.426096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592688,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":194,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250708,"kb_used_data":5416,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754724012,"statfs":{"total":773094113280,"available":772837388288,"internally_reserved":0,"allocated":5545984,"data_stored":2938112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2320,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.816654"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.628546+0000","last_change":"2024-04-25T10:07:07.628546+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.628546+0000","last_undegraded":"2024-04-25T10:07:07.628546+0000","last_fullsized":"2024-04-25T10:07:07.628546+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661452+0000","last_change":"2024-04-25T10:07:08.668541+0000","last_active":"2024-04-25T10:07:09.661452+0000","last_peered":"2024-04-25T10:07:09.661452+0000","last_clean":"2024-04-25T10:07:09.661452+0000","last_became_active":"2024-04-25T10:07:08.667612+0000","last_became_peered":"2024-04-25T10:07:08.667612+0000","last_unstale":"2024-04-25T10:07:09.661452+0000","last_undegraded":"2024-04-25T10:07:09.661452+0000","last_fullsized":"2024-04-25T10:07:09.661452+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:03.943867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.623001+0000","last_change":"2024-04-25T10:07:07.623001+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.623001+0000","last_undegraded":"2024-04-25T10:07:07.623001+0000","last_fullsized":"2024-04-25T10:07:07.623001+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950895+0000","last_change":"2024-04-25T10:07:08.698327+0000","last_active":"2024-04-25T10:07:09.950895+0000","last_peered":"2024-04-25T10:07:09.950895+0000","last_clean":"2024-04-25T10:07:09.950895+0000","last_became_active":"2024-04-25T10:07:08.697951+0000","last_became_peered":"2024-04-25T10:07:08.697951+0000","last_unstale":"2024-04-25T10:07:09.950895+0000","last_undegraded":"2024-04-25T10:07:09.950895+0000","last_fullsized":"2024-04-25T10:07:09.950895+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:03.943867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.659823+0000","last_change":"2024-04-25T10:07:08.679048+0000","last_active":"2024-04-25T10:07:09.659823+0000","last_peered":"2024-04-25T10:07:09.659823+0000","last_clean":"2024-04-25T10:07:09.659823+0000","last_became_active":"2024-04-25T10:07:08.678118+0000","last_became_peered":"2024-04-25T10:07:08.678118+0000","last_unstale":"2024-04-25T10:07:09.659823+0000","last_undegraded":"2024-04-25T10:07:09.659823+0000","last_fullsized":"2024-04-25T10:07:09.659823+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:36.511758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.660928+0000","last_change":"2024-04-25T10:07:08.668584+0000","last_active":"2024-04-25T10:07:09.660928+0000","last_peered":"2024-04-25T10:07:09.660928+0000","last_clean":"2024-04-25T10:07:09.660928+0000","last_became_active":"2024-04-25T10:07:08.667972+0000","last_became_peered":"2024-04-25T10:07:08.667972+0000","last_unstale":"2024-04-25T10:07:09.660928+0000","last_undegraded":"2024-04-25T10:07:09.660928+0000","last_fullsized":"2024-04-25T10:07:09.660928+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:47.115160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.628165+0000","last_change":"2024-04-25T10:07:07.628165+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.628165+0000","last_undegraded":"2024-04-25T10:07:07.628165+0000","last_fullsized":"2024-04-25T10:07:07.628165+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949295+0000","last_change":"2024-04-25T10:07:08.695951+0000","last_active":"2024-04-25T10:07:09.949295+0000","last_peered":"2024-04-25T10:07:09.949295+0000","last_clean":"2024-04-25T10:07:09.949295+0000","last_became_active":"2024-04-25T10:07:08.695759+0000","last_became_peered":"2024-04-25T10:07:08.695759+0000","last_unstale":"2024-04-25T10:07:09.949295+0000","last_undegraded":"2024-04-25T10:07:09.949295+0000","last_fullsized":"2024-04-25T10:07:09.949295+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:13.347150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950667+0000","last_change":"2024-04-25T10:07:08.670041+0000","last_active":"2024-04-25T10:07:09.950667+0000","last_peered":"2024-04-25T10:07:09.950667+0000","last_clean":"2024-04-25T10:07:09.950667+0000","last_became_active":"2024-04-25T10:07:08.665509+0000","last_became_peered":"2024-04-25T10:07:08.665509+0000","last_unstale":"2024-04-25T10:07:09.950667+0000","last_undegraded":"2024-04-25T10:07:09.950667+0000","last_fullsized":"2024-04-25T10:07:09.950667+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:36.511758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950347+0000","last_change":"2024-04-25T10:07:08.695982+0000","last_active":"2024-04-25T10:07:09.950347+0000","last_peered":"2024-04-25T10:07:09.950347+0000","last_clean":"2024-04-25T10:07:09.950347+0000","last_became_active":"2024-04-25T10:07:08.695686+0000","last_became_peered":"2024-04-25T10:07:08.695686+0000","last_unstale":"2024-04-25T10:07:09.950347+0000","last_undegraded":"2024-04-25T10:07:09.950347+0000","last_fullsized":"2024-04-25T10:07:09.950347+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:55.221728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950004+0000","last_change":"2024-04-25T10:07:08.669751+0000","last_active":"2024-04-25T10:07:09.950004+0000","last_peered":"2024-04-25T10:07:09.950004+0000","last_clean":"2024-04-25T10:07:09.950004+0000","last_became_active":"2024-04-25T10:07:08.669405+0000","last_became_peered":"2024-04-25T10:07:08.669405+0000","last_unstale":"2024-04-25T10:07:09.950004+0000","last_undegraded":"2024-04-25T10:07:09.950004+0000","last_fullsized":"2024-04-25T10:07:09.950004+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:14.113763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.659863+0000","last_change":"2024-04-25T10:07:08.653737+0000","last_active":"2024-04-25T10:07:09.659863+0000","last_peered":"2024-04-25T10:07:09.659863+0000","last_clean":"2024-04-25T10:07:09.659863+0000","last_became_active":"2024-04-25T10:07:08.652205+0000","last_became_peered":"2024-04-25T10:07:08.652205+0000","last_unstale":"2024-04-25T10:07:09.659863+0000","last_undegraded":"2024-04-25T10:07:09.659863+0000","last_fullsized":"2024-04-25T10:07:09.659863+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:51.945868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948399+0000","last_change":"2024-04-25T10:07:08.674061+0000","last_active":"2024-04-25T10:07:09.948399+0000","last_peered":"2024-04-25T10:07:09.948399+0000","last_clean":"2024-04-25T10:07:09.948399+0000","last_became_active":"2024-04-25T10:07:08.673472+0000","last_became_peered":"2024-04-25T10:07:08.673472+0000","last_unstale":"2024-04-25T10:07:09.948399+0000","last_undegraded":"2024-04-25T10:07:09.948399+0000","last_fullsized":"2024-04-25T10:07:09.948399+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:18.161440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948878+0000","last_change":"2024-04-25T10:07:08.663911+0000","last_active":"2024-04-25T10:07:09.948878+0000","last_peered":"2024-04-25T10:07:09.948878+0000","last_clean":"2024-04-25T10:07:09.948878+0000","last_became_active":"2024-04-25T10:07:08.661107+0000","last_became_peered":"2024-04-25T10:07:08.661107+0000","last_unstale":"2024-04-25T10:07:09.948878+0000","last_undegraded":"2024-04-25T10:07:09.948878+0000","last_fullsized":"2024-04-25T10:07:09.948878+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:36.924623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661214+0000","last_change":"2024-04-25T10:07:08.656468+0000","last_active":"2024-04-25T10:07:09.661214+0000","last_peered":"2024-04-25T10:07:09.661214+0000","last_clean":"2024-04-25T10:07:09.661214+0000","last_became_active":"2024-04-25T10:07:08.656157+0000","last_became_peered":"2024-04-25T10:07:08.656157+0000","last_unstale":"2024-04-25T10:07:09.661214+0000","last_undegraded":"2024-04-25T10:07:09.661214+0000","last_fullsized":"2024-04-25T10:07:09.661214+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:49.434227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949586+0000","last_change":"2024-04-25T10:07:08.687524+0000","last_active":"2024-04-25T10:07:09.949586+0000","last_peered":"2024-04-25T10:07:09.949586+0000","last_clean":"2024-04-25T10:07:09.949586+0000","last_became_active":"2024-04-25T10:07:08.686853+0000","last_became_peered":"2024-04-25T10:07:08.686853+0000","last_unstale":"2024-04-25T10:07:09.949586+0000","last_undegraded":"2024-04-25T10:07:09.949586+0000","last_fullsized":"2024-04-25T10:07:09.949586+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:42.792000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.624555+0000","last_change":"2024-04-25T10:07:07.624555+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.624555+0000","last_undegraded":"2024-04-25T10:07:07.624555+0000","last_fullsized":"2024-04-25T10:07:07.624555+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.623659+0000","last_change":"2024-04-25T10:07:07.623659+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.623659+0000","last_undegraded":"2024-04-25T10:07:07.623659+0000","last_fullsized":"2024-04-25T10:07:07.623659+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950860+0000","last_change":"2024-04-25T10:07:08.692631+0000","last_active":"2024-04-25T10:07:09.950860+0000","last_peered":"2024-04-25T10:07:09.950860+0000","last_clean":"2024-04-25T10:07:09.950860+0000","last_became_active":"2024-04-25T10:07:08.691998+0000","last_became_peered":"2024-04-25T10:07:08.691998+0000","last_unstale":"2024-04-25T10:07:09.950860+0000","last_undegraded":"2024-04-25T10:07:09.950860+0000","last_fullsized":"2024-04-25T10:07:09.950860+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:46.415556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948984+0000","last_change":"2024-04-25T10:07:08.662832+0000","last_active":"2024-04-25T10:07:09.948984+0000","last_peered":"2024-04-25T10:07:09.948984+0000","last_clean":"2024-04-25T10:07:09.948984+0000","last_became_active":"2024-04-25T10:07:08.662481+0000","last_became_peered":"2024-04-25T10:07:08.662481+0000","last_unstale":"2024-04-25T10:07:09.948984+0000","last_undegraded":"2024-04-25T10:07:09.948984+0000","last_fullsized":"2024-04-25T10:07:09.948984+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:36.884665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661354+0000","last_change":"2024-04-25T10:07:08.674228+0000","last_active":"2024-04-25T10:07:09.661354+0000","last_peered":"2024-04-25T10:07:09.661354+0000","last_clean":"2024-04-25T10:07:09.661354+0000","last_became_active":"2024-04-25T10:07:08.673895+0000","last_became_peered":"2024-04-25T10:07:08.673895+0000","last_unstale":"2024-04-25T10:07:09.661354+0000","last_undegraded":"2024-04-25T10:07:09.661354+0000","last_fullsized":"2024-04-25T10:07:09.661354+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:20.505697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949709+0000","last_change":"2024-04-25T10:07:08.697904+0000","last_active":"2024-04-25T10:07:09.949709+0000","last_peered":"2024-04-25T10:07:09.949709+0000","last_clean":"2024-04-25T10:07:09.949709+0000","last_became_active":"2024-04-25T10:07:08.697630+0000","last_became_peered":"2024-04-25T10:07:08.697630+0000","last_unstale":"2024-04-25T10:07:09.949709+0000","last_undegraded":"2024-04-25T10:07:09.949709+0000","last_fullsized":"2024-04-25T10:07:09.949709+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:10.939081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.951293+0000","last_change":"2024-04-25T10:07:08.673762+0000","last_active":"2024-04-25T10:07:09.951293+0000","last_peered":"2024-04-25T10:07:09.951293+0000","last_clean":"2024-04-25T10:07:09.951293+0000","last_became_active":"2024-04-25T10:07:08.673008+0000","last_became_peered":"2024-04-25T10:07:08.673008+0000","last_unstale":"2024-04-25T10:07:09.951293+0000","last_undegraded":"2024-04-25T10:07:09.951293+0000","last_fullsized":"2024-04-25T10:07:09.951293+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:58.989660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949682+0000","last_change":"2024-04-25T10:07:08.683621+0000","last_active":"2024-04-25T10:07:09.949682+0000","last_peered":"2024-04-25T10:07:09.949682+0000","last_clean":"2024-04-25T10:07:09.949682+0000","last_became_active":"2024-04-25T10:07:08.682216+0000","last_became_peered":"2024-04-25T10:07:08.682216+0000","last_unstale":"2024-04-25T10:07:09.949682+0000","last_undegraded":"2024-04-25T10:07:09.949682+0000","last_fullsized":"2024-04-25T10:07:09.949682+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:24.838866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950456+0000","last_change":"2024-04-25T10:07:08.661024+0000","last_active":"2024-04-25T10:07:09.950456+0000","last_peered":"2024-04-25T10:07:09.950456+0000","last_clean":"2024-04-25T10:07:09.950456+0000","last_became_active":"2024-04-25T10:07:08.660244+0000","last_became_peered":"2024-04-25T10:07:08.660244+0000","last_unstale":"2024-04-25T10:07:09.950456+0000","last_undegraded":"2024-04-25T10:07:09.950456+0000","last_fullsized":"2024-04-25T10:07:09.950456+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:36.752046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949446+0000","last_change":"2024-04-25T10:07:08.654991+0000","last_active":"2024-04-25T10:07:09.949446+0000","last_peered":"2024-04-25T10:07:09.949446+0000","last_clean":"2024-04-25T10:07:09.949446+0000","last_became_active":"2024-04-25T10:07:08.654393+0000","last_became_peered":"2024-04-25T10:07:08.654393+0000","last_unstale":"2024-04-25T10:07:09.949446+0000","last_undegraded":"2024-04-25T10:07:09.949446+0000","last_fullsized":"2024-04-25T10:07:09.949446+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:06.921997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.619319+0000","last_change":"2024-04-25T10:07:07.619319+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.619319+0000","last_undegraded":"2024-04-25T10:07:07.619319+0000","last_fullsized":"2024-04-25T10:07:07.619319+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.659967+0000","last_change":"2024-04-25T10:07:08.654204+0000","last_active":"2024-04-25T10:07:09.659967+0000","last_peered":"2024-04-25T10:07:09.659967+0000","last_clean":"2024-04-25T10:07:09.659967+0000","last_became_active":"2024-04-25T10:07:08.653136+0000","last_became_peered":"2024-04-25T10:07:08.653136+0000","last_unstale":"2024-04-25T10:07:09.659967+0000","last_undegraded":"2024-04-25T10:07:09.659967+0000","last_fullsized":"2024-04-25T10:07:09.659967+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:25.167540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950303+0000","last_change":"2024-04-25T10:07:08.671968+0000","last_active":"2024-04-25T10:07:09.950303+0000","last_peered":"2024-04-25T10:07:09.950303+0000","last_clean":"2024-04-25T10:07:09.950303+0000","last_became_active":"2024-04-25T10:07:08.662880+0000","last_became_peered":"2024-04-25T10:07:08.662880+0000","last_unstale":"2024-04-25T10:07:09.950303+0000","last_undegraded":"2024-04-25T10:07:09.950303+0000","last_fullsized":"2024-04-25T10:07:09.950303+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:53.086888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.620746+0000","last_change":"2024-04-25T10:07:07.620746+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.620746+0000","last_undegraded":"2024-04-25T10:07:07.620746+0000","last_fullsized":"2024-04-25T10:07:07.620746+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661064+0000","last_change":"2024-04-25T10:07:08.660010+0000","last_active":"2024-04-25T10:07:09.661064+0000","last_peered":"2024-04-25T10:07:09.661064+0000","last_clean":"2024-04-25T10:07:09.661064+0000","last_became_active":"2024-04-25T10:07:08.659234+0000","last_became_peered":"2024-04-25T10:07:08.659234+0000","last_unstale":"2024-04-25T10:07:09.661064+0000","last_undegraded":"2024-04-25T10:07:09.661064+0000","last_fullsized":"2024-04-25T10:07:09.661064+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:33.067413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948434+0000","last_change":"2024-04-25T10:07:08.648491+0000","last_active":"2024-04-25T10:07:09.948434+0000","last_peered":"2024-04-25T10:07:09.948434+0000","last_clean":"2024-04-25T10:07:09.948434+0000","last_became_active":"2024-04-25T10:07:08.647710+0000","last_became_peered":"2024-04-25T10:07:08.647710+0000","last_unstale":"2024-04-25T10:07:09.948434+0000","last_undegraded":"2024-04-25T10:07:09.948434+0000","last_fullsized":"2024-04-25T10:07:09.948434+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:41.621343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.657032+0000","last_change":"2024-04-25T10:07:08.672817+0000","last_active":"2024-04-25T10:07:09.657032+0000","last_peered":"2024-04-25T10:07:09.657032+0000","last_clean":"2024-04-25T10:07:09.657032+0000","last_became_active":"2024-04-25T10:07:08.672477+0000","last_became_peered":"2024-04-25T10:07:08.672477+0000","last_unstale":"2024-04-25T10:07:09.657032+0000","last_undegraded":"2024-04-25T10:07:09.657032+0000","last_fullsized":"2024-04-25T10:07:09.657032+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:51.290935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661578+0000","last_change":"2024-04-25T10:07:07.645402+0000","last_active":"2024-04-25T10:07:09.661578+0000","last_peered":"2024-04-25T10:07:09.661578+0000","last_clean":"2024-04-25T10:07:09.661578+0000","last_became_active":"2024-04-25T10:07:07.644964+0000","last_became_peered":"2024-04-25T10:07:07.644964+0000","last_unstale":"2024-04-25T10:07:09.661578+0000","last_undegraded":"2024-04-25T10:07:09.661578+0000","last_fullsized":"2024-04-25T10:07:09.661578+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:35.915922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950676+0000","last_change":"2024-04-25T10:07:08.661409+0000","last_active":"2024-04-25T10:07:09.950676+0000","last_peered":"2024-04-25T10:07:09.950676+0000","last_clean":"2024-04-25T10:07:09.950676+0000","last_became_active":"2024-04-25T10:07:08.660766+0000","last_became_peered":"2024-04-25T10:07:08.660766+0000","last_unstale":"2024-04-25T10:07:09.950676+0000","last_undegraded":"2024-04-25T10:07:09.950676+0000","last_fullsized":"2024-04-25T10:07:09.950676+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:05.519316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663264+0000","last_change":"2024-04-25T10:07:07.655701+0000","last_active":"2024-04-25T10:07:07.663264+0000","last_peered":"2024-04-25T10:07:07.663264+0000","last_clean":"2024-04-25T10:07:07.663264+0000","last_became_active":"2024-04-25T10:07:07.653511+0000","last_became_peered":"2024-04-25T10:07:07.653511+0000","last_unstale":"2024-04-25T10:07:07.663264+0000","last_undegraded":"2024-04-25T10:07:07.663264+0000","last_fullsized":"2024-04-25T10:07:07.663264+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:56.489633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.660867+0000","last_change":"2024-04-25T10:07:08.675806+0000","last_active":"2024-04-25T10:07:09.660867+0000","last_peered":"2024-04-25T10:07:09.660867+0000","last_clean":"2024-04-25T10:07:09.660867+0000","last_became_active":"2024-04-25T10:07:08.675208+0000","last_became_peered":"2024-04-25T10:07:08.675208+0000","last_unstale":"2024-04-25T10:07:09.660867+0000","last_undegraded":"2024-04-25T10:07:09.660867+0000","last_fullsized":"2024-04-25T10:07:09.660867+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:46.314031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.657793+0000","last_change":"2024-04-25T10:07:07.648517+0000","last_active":"2024-04-25T10:07:09.657793+0000","last_peered":"2024-04-25T10:07:09.657793+0000","last_clean":"2024-04-25T10:07:09.657793+0000","last_became_active":"2024-04-25T10:07:07.648183+0000","last_became_peered":"2024-04-25T10:07:07.648183+0000","last_unstale":"2024-04-25T10:07:09.657793+0000","last_undegraded":"2024-04-25T10:07:09.657793+0000","last_fullsized":"2024-04-25T10:07:09.657793+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.578592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.657722+0000","last_change":"2024-04-25T10:07:08.663431+0000","last_active":"2024-04-25T10:07:09.657722+0000","last_peered":"2024-04-25T10:07:09.657722+0000","last_clean":"2024-04-25T10:07:09.657722+0000","last_became_active":"2024-04-25T10:07:08.663153+0000","last_became_peered":"2024-04-25T10:07:08.663153+0000","last_unstale":"2024-04-25T10:07:09.657722+0000","last_undegraded":"2024-04-25T10:07:09.657722+0000","last_fullsized":"2024-04-25T10:07:09.657722+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:54.356296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.668582+0000","last_change":"2024-04-25T10:07:07.661239+0000","last_active":"2024-04-25T10:07:07.668582+0000","last_peered":"2024-04-25T10:07:07.668582+0000","last_clean":"2024-04-25T10:07:07.668582+0000","last_became_active":"2024-04-25T10:07:07.660654+0000","last_became_peered":"2024-04-25T10:07:07.660654+0000","last_unstale":"2024-04-25T10:07:07.668582+0000","last_undegraded":"2024-04-25T10:07:07.668582+0000","last_fullsized":"2024-04-25T10:07:07.668582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:59.863704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.619660+0000","last_change":"2024-04-25T10:07:07.619660+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.619660+0000","last_undegraded":"2024-04-25T10:07:07.619660+0000","last_fullsized":"2024-04-25T10:07:07.619660+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661190+0000","last_change":"2024-04-25T10:07:07.637715+0000","last_active":"2024-04-25T10:07:09.661190+0000","last_peered":"2024-04-25T10:07:09.661190+0000","last_clean":"2024-04-25T10:07:09.661190+0000","last_became_active":"2024-04-25T10:07:07.637192+0000","last_became_peered":"2024-04-25T10:07:07.637192+0000","last_unstale":"2024-04-25T10:07:09.661190+0000","last_undegraded":"2024-04-25T10:07:09.661190+0000","last_fullsized":"2024-04-25T10:07:09.661190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:34.432109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950543+0000","last_change":"2024-04-25T10:07:08.665669+0000","last_active":"2024-04-25T10:07:09.950543+0000","last_peered":"2024-04-25T10:07:09.950543+0000","last_clean":"2024-04-25T10:07:09.950543+0000","last_became_active":"2024-04-25T10:07:08.664399+0000","last_became_peered":"2024-04-25T10:07:08.664399+0000","last_unstale":"2024-04-25T10:07:09.950543+0000","last_undegraded":"2024-04-25T10:07:09.950543+0000","last_fullsized":"2024-04-25T10:07:09.950543+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:13.428895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663157+0000","last_change":"2024-04-25T10:07:07.657392+0000","last_active":"2024-04-25T10:07:07.663157+0000","last_peered":"2024-04-25T10:07:07.663157+0000","last_clean":"2024-04-25T10:07:07.663157+0000","last_became_active":"2024-04-25T10:07:07.656810+0000","last_became_peered":"2024-04-25T10:07:07.656810+0000","last_unstale":"2024-04-25T10:07:07.663157+0000","last_undegraded":"2024-04-25T10:07:07.663157+0000","last_fullsized":"2024-04-25T10:07:07.663157+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:16.056727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949662+0000","last_change":"2024-04-25T10:07:08.696119+0000","last_active":"2024-04-25T10:07:09.949662+0000","last_peered":"2024-04-25T10:07:09.949662+0000","last_clean":"2024-04-25T10:07:09.949662+0000","last_became_active":"2024-04-25T10:07:08.695408+0000","last_became_peered":"2024-04-25T10:07:08.695408+0000","last_unstale":"2024-04-25T10:07:09.949662+0000","last_undegraded":"2024-04-25T10:07:09.949662+0000","last_fullsized":"2024-04-25T10:07:09.949662+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:25.943908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.659176+0000","last_change":"2024-04-25T10:07:07.647409+0000","last_active":"2024-04-25T10:07:09.659176+0000","last_peered":"2024-04-25T10:07:09.659176+0000","last_clean":"2024-04-25T10:07:09.659176+0000","last_became_active":"2024-04-25T10:07:07.646680+0000","last_became_peered":"2024-04-25T10:07:07.646680+0000","last_unstale":"2024-04-25T10:07:09.659176+0000","last_undegraded":"2024-04-25T10:07:09.659176+0000","last_fullsized":"2024-04-25T10:07:09.659176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:32.974250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949802+0000","last_change":"2024-04-25T10:07:08.682581+0000","last_active":"2024-04-25T10:07:09.949802+0000","last_peered":"2024-04-25T10:07:09.949802+0000","last_clean":"2024-04-25T10:07:09.949802+0000","last_became_active":"2024-04-25T10:07:08.673438+0000","last_became_peered":"2024-04-25T10:07:08.673438+0000","last_unstale":"2024-04-25T10:07:09.949802+0000","last_undegraded":"2024-04-25T10:07:09.949802+0000","last_fullsized":"2024-04-25T10:07:09.949802+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:18.931476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949439+0000","last_change":"2024-04-25T10:07:07.635219+0000","last_active":"2024-04-25T10:07:09.949439+0000","last_peered":"2024-04-25T10:07:09.949439+0000","last_clean":"2024-04-25T10:07:09.949439+0000","last_became_active":"2024-04-25T10:07:07.633856+0000","last_became_peered":"2024-04-25T10:07:07.633856+0000","last_unstale":"2024-04-25T10:07:09.949439+0000","last_undegraded":"2024-04-25T10:07:09.949439+0000","last_fullsized":"2024-04-25T10:07:09.949439+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:18.596183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.622656+0000","last_change":"2024-04-25T10:07:07.622656+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.622656+0000","last_undegraded":"2024-04-25T10:07:07.622656+0000","last_fullsized":"2024-04-25T10:07:07.622656+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.658561+0000","last_change":"2024-04-25T10:07:07.639391+0000","last_active":"2024-04-25T10:07:07.658561+0000","last_peered":"2024-04-25T10:07:07.658561+0000","last_clean":"2024-04-25T10:07:07.658561+0000","last_became_active":"2024-04-25T10:07:07.638645+0000","last_became_peered":"2024-04-25T10:07:07.638645+0000","last_unstale":"2024-04-25T10:07:07.658561+0000","last_undegraded":"2024-04-25T10:07:07.658561+0000","last_fullsized":"2024-04-25T10:07:07.658561+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:42.132953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.627466+0000","last_change":"2024-04-25T10:07:07.627466+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.627466+0000","last_undegraded":"2024-04-25T10:07:07.627466+0000","last_fullsized":"2024-04-25T10:07:07.627466+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950434+0000","last_change":"2024-04-25T10:07:07.639070+0000","last_active":"2024-04-25T10:07:09.950434+0000","last_peered":"2024-04-25T10:07:09.950434+0000","last_clean":"2024-04-25T10:07:09.950434+0000","last_became_active":"2024-04-25T10:07:07.637998+0000","last_became_peered":"2024-04-25T10:07:07.637998+0000","last_unstale":"2024-04-25T10:07:09.950434+0000","last_undegraded":"2024-04-25T10:07:09.950434+0000","last_fullsized":"2024-04-25T10:07:09.950434+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:41.783298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661511+0000","last_change":"2024-04-25T10:07:08.656781+0000","last_active":"2024-04-25T10:07:09.661511+0000","last_peered":"2024-04-25T10:07:09.661511+0000","last_clean":"2024-04-25T10:07:09.661511+0000","last_became_active":"2024-04-25T10:07:08.656024+0000","last_became_peered":"2024-04-25T10:07:08.656024+0000","last_unstale":"2024-04-25T10:07:09.661511+0000","last_undegraded":"2024-04-25T10:07:09.661511+0000","last_fullsized":"2024-04-25T10:07:09.661511+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:16.459887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.671738+0000","last_change":"2024-04-25T10:07:07.664212+0000","last_active":"2024-04-25T10:07:07.671738+0000","last_peered":"2024-04-25T10:07:07.671738+0000","last_clean":"2024-04-25T10:07:07.671738+0000","last_became_active":"2024-04-25T10:07:07.663582+0000","last_became_peered":"2024-04-25T10:07:07.663582+0000","last_unstale":"2024-04-25T10:07:07.671738+0000","last_undegraded":"2024-04-25T10:07:07.671738+0000","last_fullsized":"2024-04-25T10:07:07.671738+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:53.347594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.632969+0000","last_change":"2024-04-25T10:07:07.632969+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.632969+0000","last_undegraded":"2024-04-25T10:07:07.632969+0000","last_fullsized":"2024-04-25T10:07:07.632969+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948922+0000","last_change":"2024-04-25T10:07:07.641278+0000","last_active":"2024-04-25T10:07:09.948922+0000","last_peered":"2024-04-25T10:07:09.948922+0000","last_clean":"2024-04-25T10:07:09.948922+0000","last_became_active":"2024-04-25T10:07:07.640843+0000","last_became_peered":"2024-04-25T10:07:07.640843+0000","last_unstale":"2024-04-25T10:07:09.948922+0000","last_undegraded":"2024-04-25T10:07:09.948922+0000","last_fullsized":"2024-04-25T10:07:09.948922+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:21.231828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950245+0000","last_change":"2024-04-25T10:07:08.687193+0000","last_active":"2024-04-25T10:07:09.950245+0000","last_peered":"2024-04-25T10:07:09.950245+0000","last_clean":"2024-04-25T10:07:09.950245+0000","last_became_active":"2024-04-25T10:07:08.686381+0000","last_became_peered":"2024-04-25T10:07:08.686381+0000","last_unstale":"2024-04-25T10:07:09.950245+0000","last_undegraded":"2024-04-25T10:07:09.950245+0000","last_fullsized":"2024-04-25T10:07:09.950245+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:08.123048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661299+0000","last_change":"2024-04-25T10:07:07.636262+0000","last_active":"2024-04-25T10:07:09.661299+0000","last_peered":"2024-04-25T10:07:09.661299+0000","last_clean":"2024-04-25T10:07:09.661299+0000","last_became_active":"2024-04-25T10:07:07.635974+0000","last_became_peered":"2024-04-25T10:07:07.635974+0000","last_unstale":"2024-04-25T10:07:09.661299+0000","last_undegraded":"2024-04-25T10:07:09.661299+0000","last_fullsized":"2024-04-25T10:07:09.661299+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:12.420194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950414+0000","last_change":"2024-04-25T10:07:08.679079+0000","last_active":"2024-04-25T10:07:09.950414+0000","last_peered":"2024-04-25T10:07:09.950414+0000","last_clean":"2024-04-25T10:07:09.950414+0000","last_became_active":"2024-04-25T10:07:08.676480+0000","last_became_peered":"2024-04-25T10:07:08.676480+0000","last_unstale":"2024-04-25T10:07:09.950414+0000","last_undegraded":"2024-04-25T10:07:09.950414+0000","last_fullsized":"2024-04-25T10:07:09.950414+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:35.440810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663496+0000","last_change":"2024-04-25T10:07:07.657378+0000","last_active":"2024-04-25T10:07:07.663496+0000","last_peered":"2024-04-25T10:07:07.663496+0000","last_clean":"2024-04-25T10:07:07.663496+0000","last_became_active":"2024-04-25T10:07:07.656779+0000","last_became_peered":"2024-04-25T10:07:07.656779+0000","last_unstale":"2024-04-25T10:07:07.663496+0000","last_undegraded":"2024-04-25T10:07:07.663496+0000","last_fullsized":"2024-04-25T10:07:07.663496+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:52.281359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661648+0000","last_change":"2024-04-25T10:07:08.647206+0000","last_active":"2024-04-25T10:07:09.661648+0000","last_peered":"2024-04-25T10:07:09.661648+0000","last_clean":"2024-04-25T10:07:09.661648+0000","last_became_active":"2024-04-25T10:07:08.646487+0000","last_became_peered":"2024-04-25T10:07:08.646487+0000","last_unstale":"2024-04-25T10:07:09.661648+0000","last_undegraded":"2024-04-25T10:07:09.661648+0000","last_fullsized":"2024-04-25T10:07:09.661648+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:28.242898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661754+0000","last_change":"2024-04-25T10:07:07.633387+0000","last_active":"2024-04-25T10:07:09.661754+0000","last_peered":"2024-04-25T10:07:09.661754+0000","last_clean":"2024-04-25T10:07:09.661754+0000","last_became_active":"2024-04-25T10:07:07.632167+0000","last_became_peered":"2024-04-25T10:07:07.632167+0000","last_unstale":"2024-04-25T10:07:09.661754+0000","last_undegraded":"2024-04-25T10:07:09.661754+0000","last_fullsized":"2024-04-25T10:07:09.661754+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:12.338449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.627083+0000","last_change":"2024-04-25T10:07:07.627083+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.627083+0000","last_undegraded":"2024-04-25T10:07:07.627083+0000","last_fullsized":"2024-04-25T10:07:07.627083+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":22,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949731+0000","last_change":"2024-04-25T10:07:07.649029+0000","last_active":"2024-04-25T10:07:09.949731+0000","last_peered":"2024-04-25T10:07:09.949731+0000","last_clean":"2024-04-25T10:07:09.949731+0000","last_became_active":"2024-04-25T10:07:07.648616+0000","last_became_peered":"2024-04-25T10:07:07.648616+0000","last_unstale":"2024-04-25T10:07:09.949731+0000","last_undegraded":"2024-04-25T10:07:09.949731+0000","last_fullsized":"2024-04-25T10:07:09.949731+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:11.313783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949188+0000","last_change":"2024-04-25T10:07:08.649502+0000","last_active":"2024-04-25T10:07:09.949188+0000","last_peered":"2024-04-25T10:07:09.949188+0000","last_clean":"2024-04-25T10:07:09.949188+0000","last_became_active":"2024-04-25T10:07:08.648547+0000","last_became_peered":"2024-04-25T10:07:08.648547+0000","last_unstale":"2024-04-25T10:07:09.949188+0000","last_undegraded":"2024-04-25T10:07:09.949188+0000","last_fullsized":"2024-04-25T10:07:09.949188+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:32.108613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948265+0000","last_change":"2024-04-25T10:07:07.627080+0000","last_active":"2024-04-25T10:07:09.948265+0000","last_peered":"2024-04-25T10:07:09.948265+0000","last_clean":"2024-04-25T10:07:09.948265+0000","last_became_active":"2024-04-25T10:07:07.626664+0000","last_became_peered":"2024-04-25T10:07:07.626664+0000","last_unstale":"2024-04-25T10:07:09.948265+0000","last_undegraded":"2024-04-25T10:07:09.948265+0000","last_fullsized":"2024-04-25T10:07:09.948265+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:00.068508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950706+0000","last_change":"2024-04-25T10:07:08.676048+0000","last_active":"2024-04-25T10:07:09.950706+0000","last_peered":"2024-04-25T10:07:09.950706+0000","last_clean":"2024-04-25T10:07:09.950706+0000","last_became_active":"2024-04-25T10:07:08.675743+0000","last_became_peered":"2024-04-25T10:07:08.675743+0000","last_unstale":"2024-04-25T10:07:09.950706+0000","last_undegraded":"2024-04-25T10:07:09.950706+0000","last_fullsized":"2024-04-25T10:07:09.950706+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:51.945868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663314+0000","last_change":"2024-04-25T10:07:07.655902+0000","last_active":"2024-04-25T10:07:07.663314+0000","last_peered":"2024-04-25T10:07:07.663314+0000","last_clean":"2024-04-25T10:07:07.663314+0000","last_became_active":"2024-04-25T10:07:07.654431+0000","last_became_peered":"2024-04-25T10:07:07.654431+0000","last_unstale":"2024-04-25T10:07:07.663314+0000","last_undegraded":"2024-04-25T10:07:07.663314+0000","last_fullsized":"2024-04-25T10:07:07.663314+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:04.957714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950075+0000","last_change":"2024-04-25T10:07:08.695389+0000","last_active":"2024-04-25T10:07:09.950075+0000","last_peered":"2024-04-25T10:07:09.950075+0000","last_clean":"2024-04-25T10:07:09.950075+0000","last_became_active":"2024-04-25T10:07:08.693944+0000","last_became_peered":"2024-04-25T10:07:08.693944+0000","last_unstale":"2024-04-25T10:07:09.950075+0000","last_undegraded":"2024-04-25T10:07:09.950075+0000","last_fullsized":"2024-04-25T10:07:09.950075+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:48.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661113+0000","last_change":"2024-04-25T10:07:07.647747+0000","last_active":"2024-04-25T10:07:09.661113+0000","last_peered":"2024-04-25T10:07:09.661113+0000","last_clean":"2024-04-25T10:07:09.661113+0000","last_became_active":"2024-04-25T10:07:07.647009+0000","last_became_peered":"2024-04-25T10:07:07.647009+0000","last_unstale":"2024-04-25T10:07:09.661113+0000","last_undegraded":"2024-04-25T10:07:09.661113+0000","last_fullsized":"2024-04-25T10:07:09.661113+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:16.783244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950504+0000","last_change":"2024-04-25T10:07:08.672969+0000","last_active":"2024-04-25T10:07:09.950504+0000","last_peered":"2024-04-25T10:07:09.950504+0000","last_clean":"2024-04-25T10:07:09.950504+0000","last_became_active":"2024-04-25T10:07:08.663043+0000","last_became_peered":"2024-04-25T10:07:08.663043+0000","last_unstale":"2024-04-25T10:07:09.950504+0000","last_undegraded":"2024-04-25T10:07:09.950504+0000","last_fullsized":"2024-04-25T10:07:09.950504+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:40.481239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661568+0000","last_change":"2024-04-25T10:07:07.638916+0000","last_active":"2024-04-25T10:07:09.661568+0000","last_peered":"2024-04-25T10:07:09.661568+0000","last_clean":"2024-04-25T10:07:09.661568+0000","last_became_active":"2024-04-25T10:07:07.638374+0000","last_became_peered":"2024-04-25T10:07:07.638374+0000","last_unstale":"2024-04-25T10:07:09.661568+0000","last_undegraded":"2024-04-25T10:07:09.661568+0000","last_fullsized":"2024-04-25T10:07:09.661568+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:54.213027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661174+0000","last_change":"2024-04-25T10:07:08.662023+0000","last_active":"2024-04-25T10:07:09.661174+0000","last_peered":"2024-04-25T10:07:09.661174+0000","last_clean":"2024-04-25T10:07:09.661174+0000","last_became_active":"2024-04-25T10:07:08.661348+0000","last_became_peered":"2024-04-25T10:07:08.661348+0000","last_unstale":"2024-04-25T10:07:09.661174+0000","last_undegraded":"2024-04-25T10:07:09.661174+0000","last_fullsized":"2024-04-25T10:07:09.661174+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:18.547456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661521+0000","last_change":"2024-04-25T10:07:07.637481+0000","last_active":"2024-04-25T10:07:09.661521+0000","last_peered":"2024-04-25T10:07:09.661521+0000","last_clean":"2024-04-25T10:07:09.661521+0000","last_became_active":"2024-04-25T10:07:07.636896+0000","last_became_peered":"2024-04-25T10:07:07.636896+0000","last_unstale":"2024-04-25T10:07:09.661521+0000","last_undegraded":"2024-04-25T10:07:09.661521+0000","last_fullsized":"2024-04-25T10:07:09.661521+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:35.503057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.656958+0000","last_change":"2024-04-25T10:07:08.645533+0000","last_active":"2024-04-25T10:07:09.656958+0000","last_peered":"2024-04-25T10:07:09.656958+0000","last_clean":"2024-04-25T10:07:09.656958+0000","last_became_active":"2024-04-25T10:07:08.644698+0000","last_became_peered":"2024-04-25T10:07:08.644698+0000","last_unstale":"2024-04-25T10:07:09.656958+0000","last_undegraded":"2024-04-25T10:07:09.656958+0000","last_fullsized":"2024-04-25T10:07:09.656958+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:00.872405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.659296+0000","last_change":"2024-04-25T10:07:07.647335+0000","last_active":"2024-04-25T10:07:09.659296+0000","last_peered":"2024-04-25T10:07:09.659296+0000","last_clean":"2024-04-25T10:07:09.659296+0000","last_became_active":"2024-04-25T10:07:07.646551+0000","last_became_peered":"2024-04-25T10:07:07.646551+0000","last_unstale":"2024-04-25T10:07:09.659296+0000","last_undegraded":"2024-04-25T10:07:09.659296+0000","last_fullsized":"2024-04-25T10:07:09.659296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:10.241456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948691+0000","last_change":"2024-04-25T10:07:08.668558+0000","last_active":"2024-04-25T10:07:09.948691+0000","last_peered":"2024-04-25T10:07:09.948691+0000","last_clean":"2024-04-25T10:07:09.948691+0000","last_became_active":"2024-04-25T10:07:08.668272+0000","last_became_peered":"2024-04-25T10:07:08.668272+0000","last_unstale":"2024-04-25T10:07:09.948691+0000","last_undegraded":"2024-04-25T10:07:09.948691+0000","last_fullsized":"2024-04-25T10:07:09.948691+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:08.373478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948600+0000","last_change":"2024-04-25T10:07:07.651220+0000","last_active":"2024-04-25T10:07:09.948600+0000","last_peered":"2024-04-25T10:07:09.948600+0000","last_clean":"2024-04-25T10:07:09.948600+0000","last_became_active":"2024-04-25T10:07:07.650901+0000","last_became_peered":"2024-04-25T10:07:07.650901+0000","last_unstale":"2024-04-25T10:07:09.948600+0000","last_undegraded":"2024-04-25T10:07:09.948600+0000","last_fullsized":"2024-04-25T10:07:09.948600+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:16.400436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948483+0000","last_change":"2024-04-25T10:07:08.668430+0000","last_active":"2024-04-25T10:07:09.948483+0000","last_peered":"2024-04-25T10:07:09.948483+0000","last_clean":"2024-04-25T10:07:09.948483+0000","last_became_active":"2024-04-25T10:07:08.665538+0000","last_became_peered":"2024-04-25T10:07:08.665538+0000","last_unstale":"2024-04-25T10:07:09.948483+0000","last_undegraded":"2024-04-25T10:07:09.948483+0000","last_fullsized":"2024-04-25T10:07:09.948483+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:15.389333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.663454+0000","last_change":"2024-04-25T10:07:07.651442+0000","last_active":"2024-04-25T10:07:07.663454+0000","last_peered":"2024-04-25T10:07:07.663454+0000","last_clean":"2024-04-25T10:07:07.663454+0000","last_became_active":"2024-04-25T10:07:07.649340+0000","last_became_peered":"2024-04-25T10:07:07.649340+0000","last_unstale":"2024-04-25T10:07:07.663454+0000","last_undegraded":"2024-04-25T10:07:07.663454+0000","last_fullsized":"2024-04-25T10:07:07.663454+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:35.334997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950929+0000","last_change":"2024-04-25T10:07:08.692556+0000","last_active":"2024-04-25T10:07:09.950929+0000","last_peered":"2024-04-25T10:07:09.950929+0000","last_clean":"2024-04-25T10:07:09.950929+0000","last_became_active":"2024-04-25T10:07:08.691811+0000","last_became_peered":"2024-04-25T10:07:08.691811+0000","last_unstale":"2024-04-25T10:07:09.950929+0000","last_undegraded":"2024-04-25T10:07:09.950929+0000","last_fullsized":"2024-04-25T10:07:09.950929+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:55.114883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"47'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661696+0000","last_change":"2024-04-25T10:07:07.663123+0000","last_active":"2024-04-25T10:07:09.661696+0000","last_peered":"2024-04-25T10:07:09.661696+0000","last_clean":"2024-04-25T10:07:09.661696+0000","last_became_active":"2024-04-25T10:07:07.662759+0000","last_became_peered":"2024-04-25T10:07:07.662759+0000","last_unstale":"2024-04-25T10:07:09.661696+0000","last_undegraded":"2024-04-25T10:07:09.661696+0000","last_fullsized":"2024-04-25T10:07:09.661696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:24.935207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950853+0000","last_change":"2024-04-25T10:07:08.674835+0000","last_active":"2024-04-25T10:07:09.950853+0000","last_peered":"2024-04-25T10:07:09.950853+0000","last_clean":"2024-04-25T10:07:09.950853+0000","last_became_active":"2024-04-25T10:07:08.674302+0000","last_became_peered":"2024-04-25T10:07:08.674302+0000","last_unstale":"2024-04-25T10:07:09.950853+0000","last_undegraded":"2024-04-25T10:07:09.950853+0000","last_fullsized":"2024-04-25T10:07:09.950853+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:46.314031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.671762+0000","last_change":"2024-04-25T10:07:07.665347+0000","last_active":"2024-04-25T10:07:07.671762+0000","last_peered":"2024-04-25T10:07:07.671762+0000","last_clean":"2024-04-25T10:07:07.671762+0000","last_became_active":"2024-04-25T10:07:07.664822+0000","last_became_peered":"2024-04-25T10:07:07.664822+0000","last_unstale":"2024-04-25T10:07:07.671762+0000","last_undegraded":"2024-04-25T10:07:07.671762+0000","last_fullsized":"2024-04-25T10:07:07.671762+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:55.111760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.631949+0000","last_change":"2024-04-25T10:07:07.631949+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.631949+0000","last_undegraded":"2024-04-25T10:07:07.631949+0000","last_fullsized":"2024-04-25T10:07:07.631949+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.657015+0000","last_change":"2024-04-25T10:07:07.648943+0000","last_active":"2024-04-25T10:07:09.657015+0000","last_peered":"2024-04-25T10:07:09.657015+0000","last_clean":"2024-04-25T10:07:09.657015+0000","last_became_active":"2024-04-25T10:07:07.648706+0000","last_became_peered":"2024-04-25T10:07:07.648706+0000","last_unstale":"2024-04-25T10:07:09.657015+0000","last_undegraded":"2024-04-25T10:07:09.657015+0000","last_fullsized":"2024-04-25T10:07:09.657015+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:52.190399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661113+0000","last_change":"2024-04-25T10:07:08.651605+0000","last_active":"2024-04-25T10:07:09.661113+0000","last_peered":"2024-04-25T10:07:09.661113+0000","last_clean":"2024-04-25T10:07:09.661113+0000","last_became_active":"2024-04-25T10:07:08.651056+0000","last_became_peered":"2024-04-25T10:07:08.651056+0000","last_unstale":"2024-04-25T10:07:09.661113+0000","last_undegraded":"2024-04-25T10:07:09.661113+0000","last_fullsized":"2024-04-25T10:07:09.661113+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:20.289026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.668623+0000","last_change":"2024-04-25T10:07:07.661444+0000","last_active":"2024-04-25T10:07:07.668623+0000","last_peered":"2024-04-25T10:07:07.668623+0000","last_clean":"2024-04-25T10:07:07.668623+0000","last_became_active":"2024-04-25T10:07:07.660752+0000","last_became_peered":"2024-04-25T10:07:07.660752+0000","last_unstale":"2024-04-25T10:07:07.668623+0000","last_undegraded":"2024-04-25T10:07:07.668623+0000","last_fullsized":"2024-04-25T10:07:07.668623+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:50.282234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661139+0000","last_change":"2024-04-25T10:07:08.673869+0000","last_active":"2024-04-25T10:07:09.661139+0000","last_peered":"2024-04-25T10:07:09.661139+0000","last_clean":"2024-04-25T10:07:09.661139+0000","last_became_active":"2024-04-25T10:07:08.673558+0000","last_became_peered":"2024-04-25T10:07:08.673558+0000","last_unstale":"2024-04-25T10:07:09.661139+0000","last_undegraded":"2024-04-25T10:07:09.661139+0000","last_fullsized":"2024-04-25T10:07:09.661139+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:47.631323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.631316+0000","last_change":"2024-04-25T10:07:07.628649+0000","last_active":"2024-04-25T10:07:07.631316+0000","last_peered":"2024-04-25T10:07:07.631316+0000","last_clean":"2024-04-25T10:07:07.631316+0000","last_became_active":"2024-04-25T10:07:07.627827+0000","last_became_peered":"2024-04-25T10:07:07.627827+0000","last_unstale":"2024-04-25T10:07:07.631316+0000","last_undegraded":"2024-04-25T10:07:07.631316+0000","last_fullsized":"2024-04-25T10:07:07.631316+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:00.780462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.659188+0000","last_change":"2024-04-25T10:07:08.645430+0000","last_active":"2024-04-25T10:07:09.659188+0000","last_peered":"2024-04-25T10:07:09.659188+0000","last_clean":"2024-04-25T10:07:09.659188+0000","last_became_active":"2024-04-25T10:07:08.644595+0000","last_became_peered":"2024-04-25T10:07:08.644595+0000","last_unstale":"2024-04-25T10:07:09.659188+0000","last_undegraded":"2024-04-25T10:07:09.659188+0000","last_fullsized":"2024-04-25T10:07:09.659188+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:13.428895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.656737+0000","last_change":"2024-04-25T10:07:07.644691+0000","last_active":"2024-04-25T10:07:09.656737+0000","last_peered":"2024-04-25T10:07:09.656737+0000","last_clean":"2024-04-25T10:07:09.656737+0000","last_became_active":"2024-04-25T10:07:07.644108+0000","last_became_peered":"2024-04-25T10:07:07.644108+0000","last_unstale":"2024-04-25T10:07:09.656737+0000","last_undegraded":"2024-04-25T10:07:09.656737+0000","last_fullsized":"2024-04-25T10:07:09.656737+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:06.777436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-04-25T10:07:07.618043+0000","last_change":"2024-04-25T10:07:07.618043+0000","last_active":"2024-04-25T10:07:07.606865+0000","last_peered":"2024-04-25T10:07:07.606865+0000","last_clean":"2024-04-25T10:07:07.606865+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:07.618043+0000","last_undegraded":"2024-04-25T10:07:07.618043+0000","last_fullsized":"2024-04-25T10:07:07.618043+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949797+0000","last_change":"2024-04-25T10:07:07.634215+0000","last_active":"2024-04-25T10:07:09.949797+0000","last_peered":"2024-04-25T10:07:09.949797+0000","last_clean":"2024-04-25T10:07:09.949797+0000","last_became_active":"2024-04-25T10:07:07.633430+0000","last_became_peered":"2024-04-25T10:07:07.633430+0000","last_unstale":"2024-04-25T10:07:09.949797+0000","last_undegraded":"2024-04-25T10:07:09.949797+0000","last_fullsized":"2024-04-25T10:07:09.949797+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:42.026976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950722+0000","last_change":"2024-04-25T10:07:08.672738+0000","last_active":"2024-04-25T10:07:09.950722+0000","last_peered":"2024-04-25T10:07:09.950722+0000","last_clean":"2024-04-25T10:07:09.950722+0000","last_became_active":"2024-04-25T10:07:08.672207+0000","last_became_peered":"2024-04-25T10:07:08.672207+0000","last_unstale":"2024-04-25T10:07:09.950722+0000","last_undegraded":"2024-04-25T10:07:09.950722+0000","last_fullsized":"2024-04-25T10:07:09.950722+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:25.167540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.662031+0000","last_change":"2024-04-25T10:07:07.647454+0000","last_active":"2024-04-25T10:07:09.662031+0000","last_peered":"2024-04-25T10:07:09.662031+0000","last_clean":"2024-04-25T10:07:09.662031+0000","last_became_active":"2024-04-25T10:07:07.646575+0000","last_became_peered":"2024-04-25T10:07:07.646575+0000","last_unstale":"2024-04-25T10:07:09.662031+0000","last_undegraded":"2024-04-25T10:07:09.662031+0000","last_fullsized":"2024-04-25T10:07:09.662031+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:14.380632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":212,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T10:07:07.635994+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:07:07.635994+0000","last_peered":"2024-04-25T10:07:07.635994+0000","last_clean":"2024-04-25T10:07:07.635994+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:07:07.635994+0000","last_undegraded":"2024-04-25T10:07:07.635994+0000","last_fullsized":"2024-04-25T10:07:07.635994+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2320,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":799844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509467,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":209476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":552960,"data_stored":223223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542172,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":548864,"data_stored":222621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542172,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":557056,"data_stored":223189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542172,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":561152,"data_stored":224185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35884,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335956,"statfs":{"total":96636764160,"available":96600018944,"internally_reserved":0,"allocated":548864,"data_stored":222621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36456,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335384,"statfs":{"total":96636764160,"available":96599433216,"internally_reserved":0,"allocated":1142784,"data_stored":812953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:13.611 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T10:07:13.603+0000 7f9710f5b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:16.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:16.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f5987231640 1 Processor -- start 2024-04-25T10:07:16.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f5987231640 1 -- start start 2024-04-25T10:07:16.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f5987231640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59800ce7a0 0x7f59800ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f5987231640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f598015bc90 0x7f59800b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f5987231640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59800cf0c0 con 0x7f59800b4660 2024-04-25T10:07:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f5987231640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59800cc370 con 0x7f598015bc90 2024-04-25T10:07:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f5987231640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5980071ca0 con 0x7f59800ce7a0 2024-04-25T10:07:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f59857ce640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f59800b4660 0x7f59800b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60064/0 (socket says 172.21.15.60:60064) 2024-04-25T10:07:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f59857ce640 1 -- 172.21.15.60:0/3070486900 learned_addr learned my addr 172.21.15.60:0/3070486900 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597ffff640 1 --2- 172.21.15.60:0/3070486900 >> v2:172.21.15.100:3300/0 conn(0x7f598015bc90 0x7f59800b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f5984fcd640 1 --2- 172.21.15.60:0/3070486900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59800ce7a0 0x7f59800ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597f7fe640 1 -- 172.21.15.60:0/3070486900 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165616705 0 0) 0x7f59800cf0c0 con 0x7f59800b4660 2024-04-25T10:07:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597f7fe640 1 -- 172.21.15.60:0/3070486900 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f596c003530 con 0x7f59800b4660 2024-04-25T10:07:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597f7fe640 1 -- 172.21.15.60:0/3070486900 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2112614152 0 0) 0x7f596c003530 con 0x7f59800b4660 2024-04-25T10:07:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597f7fe640 1 -- 172.21.15.60:0/3070486900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59800ce7a0 msgr2=0x7f59800ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597f7fe640 1 --2- 172.21.15.60:0/3070486900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59800ce7a0 0x7f59800ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597f7fe640 1 -- 172.21.15.60:0/3070486900 >> v2:172.21.15.100:3300/0 conn(0x7f598015bc90 msgr2=0x7f59800b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597f7fe640 1 --2- 172.21.15.60:0/3070486900 >> v2:172.21.15.100:3300/0 conn(0x7f598015bc90 0x7f59800b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597f7fe640 1 -- 172.21.15.60:0/3070486900 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59800cca40 con 0x7f59800b4660 2024-04-25T10:07:16.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597f7fe640 1 -- 172.21.15.60:0/3070486900 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f5978001680 con 0x7f59800b4660 2024-04-25T10:07:16.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597f7fe640 1 -- 172.21.15.60:0/3070486900 >> v1:172.21.15.60:6789/0 conn(0x7f59800b4660 legacy=0x7f59800b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597f7fe640 1 --2- 172.21.15.60:0/3070486900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f596c0039c0 0x7f596c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597f7fe640 1 --2- 172.21.15.60:0/3070486900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59800ce7a0 0x7f596c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597f7fe640 1 --2- 172.21.15.60:0/3070486900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f596c004d50 0x7f596c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597f7fe640 1 -- 172.21.15.60:0/3070486900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5980071ca0 con 0x7f596c004d50 2024-04-25T10:07:16.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597f7fe640 1 -- 172.21.15.60:0/3070486900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c00b9d0 con 0x7f596c0039c0 2024-04-25T10:07:16.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597f7fe640 1 -- 172.21.15.60:0/3070486900 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c00bb50 con 0x7f59800ce7a0 2024-04-25T10:07:16.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f59857ce640 1 --2- 172.21.15.60:0/3070486900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f596c004d50 0x7f596c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f5984fcd640 1 --2- 172.21.15.60:0/3070486900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f596c0039c0 0x7f596c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597ffff640 1 --2- 172.21.15.60:0/3070486900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59800ce7a0 0x7f596c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f59857ce640 1 -- 172.21.15.60:0/3070486900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59800ce7a0 msgr2=0x7f596c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f59857ce640 1 --2- 172.21.15.60:0/3070486900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59800ce7a0 0x7f596c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f59857ce640 1 -- 172.21.15.60:0/3070486900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f596c0039c0 msgr2=0x7f596c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f59857ce640 1 --2- 172.21.15.60:0/3070486900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f596c0039c0 0x7f596c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f59857ce640 1 -- 172.21.15.60:0/3070486900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c003530 con 0x7f596c004d50 2024-04-25T10:07:16.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.307+0000 7f597ffff640 1 --2- 172.21.15.60:0/3070486900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59800ce7a0 0x7f596c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:16.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.311+0000 7f59857ce640 1 --2- 172.21.15.60:0/3070486900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f596c004d50 0x7f596c0051d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f598006cf70 tx=0x7f597801cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539e4afeac23456f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.311+0000 7f597f7fe640 1 -- 172.21.15.60:0/3070486900 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5978007c50 con 0x7f596c004d50 2024-04-25T10:07:16.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.311+0000 7f597f7fe640 1 -- 172.21.15.60:0/3070486900 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59780a9070 con 0x7f596c004d50 2024-04-25T10:07:16.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.311+0000 7f597f7fe640 1 -- 172.21.15.60:0/3070486900 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5978005d40 con 0x7f596c004d50 2024-04-25T10:07:16.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.311+0000 7f5987231640 1 -- 172.21.15.60:0/3070486900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f596c004d50 msgr2=0x7f596c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.311+0000 7f5987231640 1 --2- 172.21.15.60:0/3070486900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f596c004d50 0x7f596c0051d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f598006cf70 tx=0x7f597801cb60 comp rx=0 tx=0).stop 2024-04-25T10:07:16.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.311+0000 7f59857ce640 1 -- 172.21.15.60:0/3070486900 reap_dead start 2024-04-25T10:07:16.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.311+0000 7f5987231640 1 -- 172.21.15.60:0/3070486900 shutdown_connections 2024-04-25T10:07:16.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.311+0000 7f5987231640 1 -- 172.21.15.60:0/3070486900 >> 172.21.15.60:0/3070486900 conn(0x7f59800b12d0 msgr2=0x7f59800b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.311+0000 7f5987231640 1 -- 172.21.15.60:0/3070486900 shutdown_connections 2024-04-25T10:07:16.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.311+0000 7f5987231640 1 -- 172.21.15.60:0/3070486900 wait complete. 2024-04-25T10:07:16.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.311+0000 7f5987231640 1 Processor -- start 2024-04-25T10:07:16.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.311+0000 7f5987231640 1 -- start start 2024-04-25T10:07:16.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f5987231640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f596c004d50 0x7f59800b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f5987231640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59801e4dd0 0x7f59801e31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f5987231640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59801e3720 0x7f5980146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f5987231640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59800b3ba0 con 0x7f59801e3720 2024-04-25T10:07:16.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f5987231640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5980071f80 con 0x7f596c004d50 2024-04-25T10:07:16.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f5987231640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59800b3d20 con 0x7f59801e4dd0 2024-04-25T10:07:16.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f59857ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59801e3720 0x7f5980146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f59857ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59801e3720 0x7f5980146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39170/0 (socket says 172.21.15.60:39170) 2024-04-25T10:07:16.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f59857ce640 1 -- 172.21.15.60:0/2926624822 learned_addr learned my addr 172.21.15.60:0/2926624822 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:16.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f5984fcd640 1 --2- 172.21.15.60:0/2926624822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f596c004d50 0x7f59800b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f597ffff640 1 --2- 172.21.15.60:0/2926624822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59801e4dd0 0x7f59801e31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f59857ce640 1 -- 172.21.15.60:0/2926624822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59801e4dd0 msgr2=0x7f59801e31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f59857ce640 1 --2- 172.21.15.60:0/2926624822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59801e4dd0 0x7f59801e31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f59857ce640 1 -- 172.21.15.60:0/2926624822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f596c004d50 msgr2=0x7f59800b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f59857ce640 1 --2- 172.21.15.60:0/2926624822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f596c004d50 0x7f59800b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f59857ce640 1 -- 172.21.15.60:0/2926624822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59801e3d30 con 0x7f59801e3720 2024-04-25T10:07:16.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f597ffff640 1 --2- 172.21.15.60:0/2926624822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59801e4dd0 0x7f59801e31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:16.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f59857ce640 1 --2- 172.21.15.60:0/2926624822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59801e3720 0x7f5980146ae0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5978000eb0 tx=0x7f596c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f597d7fa640 1 -- 172.21.15.60:0/2926624822 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5978007c50 con 0x7f59801e3720 2024-04-25T10:07:16.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f597d7fa640 1 -- 172.21.15.60:0/2926624822 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59780a6c90 con 0x7f59801e3720 2024-04-25T10:07:16.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f597d7fa640 1 -- 172.21.15.60:0/2926624822 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59780049f0 con 0x7f59801e3720 2024-04-25T10:07:16.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f5987231640 1 -- 172.21.15.60:0/2926624822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59801e4010 con 0x7f59801e3720 2024-04-25T10:07:16.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f5987231640 1 -- 172.21.15.60:0/2926624822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59801e45d0 con 0x7f59801e3720 2024-04-25T10:07:16.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f5987231640 1 -- 172.21.15.60:0/2926624822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5980074770 con 0x7f59801e3720 2024-04-25T10:07:16.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f597d7fa640 1 -- 172.21.15.60:0/2926624822 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5978004b90 con 0x7f59801e3720 2024-04-25T10:07:16.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f597d7fa640 1 --2- 172.21.15.60:0/2926624822 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5960066f60 0x7f5960069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f597d7fa640 1 -- 172.21.15.60:0/2926624822 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f597811e850 con 0x7f59801e3720 2024-04-25T10:07:16.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.315+0000 7f5984fcd640 1 --2- 172.21.15.60:0/2926624822 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5960066f60 0x7f5960069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.319+0000 7f5984fcd640 1 --2- 172.21.15.60:0/2926624822 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5960066f60 0x7f5960069420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5970009aa0 tx=0x7f5970008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.319+0000 7f597d7fa640 1 -- 172.21.15.60:0/2926624822 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5978016870 con 0x7f59801e3720 2024-04-25T10:07:16.366 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:16.369 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:16.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.475+0000 7f5987231640 1 -- 172.21.15.60:0/2926624822 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f598006ef50 con 0x7f5960066f60 2024-04-25T10:07:16.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.487+0000 7f597d7fa640 1 -- 172.21.15.60:0/2926624822 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251959 (secure 0 0 0) 0x7f598006ef50 con 0x7f5960066f60 2024-04-25T10:07:16.495 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:16.497 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.491+0000 7f5987231640 1 -- 172.21.15.60:0/2926624822 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5960066f60 msgr2=0x7f5960069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.491+0000 7f5987231640 1 --2- 172.21.15.60:0/2926624822 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5960066f60 0x7f5960069420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5970009aa0 tx=0x7f5970008040 comp rx=0 tx=0).stop 2024-04-25T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.491+0000 7f5987231640 1 -- 172.21.15.60:0/2926624822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59801e3720 msgr2=0x7f5980146ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.491+0000 7f5987231640 1 --2- 172.21.15.60:0/2926624822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59801e3720 0x7f5980146ae0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5978000eb0 tx=0x7f596c005840 comp rx=0 tx=0).stop 2024-04-25T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.491+0000 7f5987231640 1 -- 172.21.15.60:0/2926624822 shutdown_connections 2024-04-25T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.491+0000 7f5987231640 1 --2- 172.21.15.60:0/2926624822 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5960066f60 0x7f5960069420 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.491+0000 7f5987231640 1 --2- 172.21.15.60:0/2926624822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59801e3720 0x7f5980146ae0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.491+0000 7f5987231640 1 --2- 172.21.15.60:0/2926624822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59801e4dd0 0x7f59801e31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.491+0000 7f5987231640 1 --2- 172.21.15.60:0/2926624822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f596c004d50 0x7f59800b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.491+0000 7f5987231640 1 -- 172.21.15.60:0/2926624822 >> 172.21.15.60:0/2926624822 conn(0x7f59800b12d0 msgr2=0x7f59800cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.491+0000 7f5987231640 1 -- 172.21.15.60:0/2926624822 shutdown_connections 2024-04-25T10:07:16.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.491+0000 7f5987231640 1 -- 172.21.15.60:0/2926624822 wait complete. 2024-04-25T10:07:16.513 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-04-25T10:07:16.427769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254940,"kb_used_data":5480,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719780,"statfs":{"total":773094113280,"available":772833054720,"internally_reserved":0,"allocated":5611520,"data_stored":2965964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002956"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.951579+0000","last_change":"2024-04-25T10:07:08.679956+0000","last_active":"2024-04-25T10:07:09.951579+0000","last_peered":"2024-04-25T10:07:09.951579+0000","last_clean":"2024-04-25T10:07:09.951579+0000","last_became_active":"2024-04-25T10:07:08.677179+0000","last_became_peered":"2024-04-25T10:07:08.677179+0000","last_unstale":"2024-04-25T10:07:09.951579+0000","last_undegraded":"2024-04-25T10:07:09.951579+0000","last_fullsized":"2024-04-25T10:07:09.951579+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:38.353090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661452+0000","last_change":"2024-04-25T10:07:08.668541+0000","last_active":"2024-04-25T10:07:09.661452+0000","last_peered":"2024-04-25T10:07:09.661452+0000","last_clean":"2024-04-25T10:07:09.661452+0000","last_became_active":"2024-04-25T10:07:08.667612+0000","last_became_peered":"2024-04-25T10:07:08.667612+0000","last_unstale":"2024-04-25T10:07:09.661452+0000","last_undegraded":"2024-04-25T10:07:09.661452+0000","last_fullsized":"2024-04-25T10:07:09.661452+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:03.943867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.660622+0000","last_change":"2024-04-25T10:07:08.656487+0000","last_active":"2024-04-25T10:07:09.660622+0000","last_peered":"2024-04-25T10:07:09.660622+0000","last_clean":"2024-04-25T10:07:09.660622+0000","last_became_active":"2024-04-25T10:07:08.651630+0000","last_became_peered":"2024-04-25T10:07:08.651630+0000","last_unstale":"2024-04-25T10:07:09.660622+0000","last_undegraded":"2024-04-25T10:07:09.660622+0000","last_fullsized":"2024-04-25T10:07:09.660622+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:10.756293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950895+0000","last_change":"2024-04-25T10:07:08.698327+0000","last_active":"2024-04-25T10:07:09.950895+0000","last_peered":"2024-04-25T10:07:09.950895+0000","last_clean":"2024-04-25T10:07:09.950895+0000","last_became_active":"2024-04-25T10:07:08.697951+0000","last_became_peered":"2024-04-25T10:07:08.697951+0000","last_unstale":"2024-04-25T10:07:09.950895+0000","last_undegraded":"2024-04-25T10:07:09.950895+0000","last_fullsized":"2024-04-25T10:07:09.950895+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:03.943867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.659823+0000","last_change":"2024-04-25T10:07:08.679048+0000","last_active":"2024-04-25T10:07:09.659823+0000","last_peered":"2024-04-25T10:07:09.659823+0000","last_clean":"2024-04-25T10:07:09.659823+0000","last_became_active":"2024-04-25T10:07:08.678118+0000","last_became_peered":"2024-04-25T10:07:08.678118+0000","last_unstale":"2024-04-25T10:07:09.659823+0000","last_undegraded":"2024-04-25T10:07:09.659823+0000","last_fullsized":"2024-04-25T10:07:09.659823+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:36.511758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.660928+0000","last_change":"2024-04-25T10:07:08.668584+0000","last_active":"2024-04-25T10:07:09.660928+0000","last_peered":"2024-04-25T10:07:09.660928+0000","last_clean":"2024-04-25T10:07:09.660928+0000","last_became_active":"2024-04-25T10:07:08.667972+0000","last_became_peered":"2024-04-25T10:07:08.667972+0000","last_unstale":"2024-04-25T10:07:09.660928+0000","last_undegraded":"2024-04-25T10:07:09.660928+0000","last_fullsized":"2024-04-25T10:07:09.660928+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:47.115160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950737+0000","last_change":"2024-04-25T10:07:08.689054+0000","last_active":"2024-04-25T10:07:09.950737+0000","last_peered":"2024-04-25T10:07:09.950737+0000","last_clean":"2024-04-25T10:07:09.950737+0000","last_became_active":"2024-04-25T10:07:08.688113+0000","last_became_peered":"2024-04-25T10:07:08.688113+0000","last_unstale":"2024-04-25T10:07:09.950737+0000","last_undegraded":"2024-04-25T10:07:09.950737+0000","last_fullsized":"2024-04-25T10:07:09.950737+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:15.059089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949295+0000","last_change":"2024-04-25T10:07:08.695951+0000","last_active":"2024-04-25T10:07:09.949295+0000","last_peered":"2024-04-25T10:07:09.949295+0000","last_clean":"2024-04-25T10:07:09.949295+0000","last_became_active":"2024-04-25T10:07:08.695759+0000","last_became_peered":"2024-04-25T10:07:08.695759+0000","last_unstale":"2024-04-25T10:07:09.949295+0000","last_undegraded":"2024-04-25T10:07:09.949295+0000","last_fullsized":"2024-04-25T10:07:09.949295+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:13.347150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950667+0000","last_change":"2024-04-25T10:07:08.670041+0000","last_active":"2024-04-25T10:07:09.950667+0000","last_peered":"2024-04-25T10:07:09.950667+0000","last_clean":"2024-04-25T10:07:09.950667+0000","last_became_active":"2024-04-25T10:07:08.665509+0000","last_became_peered":"2024-04-25T10:07:08.665509+0000","last_unstale":"2024-04-25T10:07:09.950667+0000","last_undegraded":"2024-04-25T10:07:09.950667+0000","last_fullsized":"2024-04-25T10:07:09.950667+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:36.511758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950347+0000","last_change":"2024-04-25T10:07:08.695982+0000","last_active":"2024-04-25T10:07:09.950347+0000","last_peered":"2024-04-25T10:07:09.950347+0000","last_clean":"2024-04-25T10:07:09.950347+0000","last_became_active":"2024-04-25T10:07:08.695686+0000","last_became_peered":"2024-04-25T10:07:08.695686+0000","last_unstale":"2024-04-25T10:07:09.950347+0000","last_undegraded":"2024-04-25T10:07:09.950347+0000","last_fullsized":"2024-04-25T10:07:09.950347+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:55.221728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950004+0000","last_change":"2024-04-25T10:07:08.669751+0000","last_active":"2024-04-25T10:07:09.950004+0000","last_peered":"2024-04-25T10:07:09.950004+0000","last_clean":"2024-04-25T10:07:09.950004+0000","last_became_active":"2024-04-25T10:07:08.669405+0000","last_became_peered":"2024-04-25T10:07:08.669405+0000","last_unstale":"2024-04-25T10:07:09.950004+0000","last_undegraded":"2024-04-25T10:07:09.950004+0000","last_fullsized":"2024-04-25T10:07:09.950004+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:14.113763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.659863+0000","last_change":"2024-04-25T10:07:08.653737+0000","last_active":"2024-04-25T10:07:09.659863+0000","last_peered":"2024-04-25T10:07:09.659863+0000","last_clean":"2024-04-25T10:07:09.659863+0000","last_became_active":"2024-04-25T10:07:08.652205+0000","last_became_peered":"2024-04-25T10:07:08.652205+0000","last_unstale":"2024-04-25T10:07:09.659863+0000","last_undegraded":"2024-04-25T10:07:09.659863+0000","last_fullsized":"2024-04-25T10:07:09.659863+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:51.945868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948399+0000","last_change":"2024-04-25T10:07:08.674061+0000","last_active":"2024-04-25T10:07:09.948399+0000","last_peered":"2024-04-25T10:07:09.948399+0000","last_clean":"2024-04-25T10:07:09.948399+0000","last_became_active":"2024-04-25T10:07:08.673472+0000","last_became_peered":"2024-04-25T10:07:08.673472+0000","last_unstale":"2024-04-25T10:07:09.948399+0000","last_undegraded":"2024-04-25T10:07:09.948399+0000","last_fullsized":"2024-04-25T10:07:09.948399+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:18.161440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948878+0000","last_change":"2024-04-25T10:07:08.663911+0000","last_active":"2024-04-25T10:07:09.948878+0000","last_peered":"2024-04-25T10:07:09.948878+0000","last_clean":"2024-04-25T10:07:09.948878+0000","last_became_active":"2024-04-25T10:07:08.661107+0000","last_became_peered":"2024-04-25T10:07:08.661107+0000","last_unstale":"2024-04-25T10:07:09.948878+0000","last_undegraded":"2024-04-25T10:07:09.948878+0000","last_fullsized":"2024-04-25T10:07:09.948878+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:36.924623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661214+0000","last_change":"2024-04-25T10:07:08.656468+0000","last_active":"2024-04-25T10:07:09.661214+0000","last_peered":"2024-04-25T10:07:09.661214+0000","last_clean":"2024-04-25T10:07:09.661214+0000","last_became_active":"2024-04-25T10:07:08.656157+0000","last_became_peered":"2024-04-25T10:07:08.656157+0000","last_unstale":"2024-04-25T10:07:09.661214+0000","last_undegraded":"2024-04-25T10:07:09.661214+0000","last_fullsized":"2024-04-25T10:07:09.661214+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:49.434227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949586+0000","last_change":"2024-04-25T10:07:08.687524+0000","last_active":"2024-04-25T10:07:09.949586+0000","last_peered":"2024-04-25T10:07:09.949586+0000","last_clean":"2024-04-25T10:07:09.949586+0000","last_became_active":"2024-04-25T10:07:08.686853+0000","last_became_peered":"2024-04-25T10:07:08.686853+0000","last_unstale":"2024-04-25T10:07:09.949586+0000","last_undegraded":"2024-04-25T10:07:09.949586+0000","last_fullsized":"2024-04-25T10:07:09.949586+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:42.792000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661414+0000","last_change":"2024-04-25T10:07:08.659254+0000","last_active":"2024-04-25T10:07:09.661414+0000","last_peered":"2024-04-25T10:07:09.661414+0000","last_clean":"2024-04-25T10:07:09.661414+0000","last_became_active":"2024-04-25T10:07:08.655011+0000","last_became_peered":"2024-04-25T10:07:08.655011+0000","last_unstale":"2024-04-25T10:07:09.661414+0000","last_undegraded":"2024-04-25T10:07:09.661414+0000","last_fullsized":"2024-04-25T10:07:09.661414+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:01.966194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661109+0000","last_change":"2024-04-25T10:07:08.677136+0000","last_active":"2024-04-25T10:07:09.661109+0000","last_peered":"2024-04-25T10:07:09.661109+0000","last_clean":"2024-04-25T10:07:09.661109+0000","last_became_active":"2024-04-25T10:07:08.676306+0000","last_became_peered":"2024-04-25T10:07:08.676306+0000","last_unstale":"2024-04-25T10:07:09.661109+0000","last_undegraded":"2024-04-25T10:07:09.661109+0000","last_fullsized":"2024-04-25T10:07:09.661109+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:20.636742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950860+0000","last_change":"2024-04-25T10:07:08.692631+0000","last_active":"2024-04-25T10:07:09.950860+0000","last_peered":"2024-04-25T10:07:09.950860+0000","last_clean":"2024-04-25T10:07:09.950860+0000","last_became_active":"2024-04-25T10:07:08.691998+0000","last_became_peered":"2024-04-25T10:07:08.691998+0000","last_unstale":"2024-04-25T10:07:09.950860+0000","last_undegraded":"2024-04-25T10:07:09.950860+0000","last_fullsized":"2024-04-25T10:07:09.950860+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:46.415556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948984+0000","last_change":"2024-04-25T10:07:08.662832+0000","last_active":"2024-04-25T10:07:09.948984+0000","last_peered":"2024-04-25T10:07:09.948984+0000","last_clean":"2024-04-25T10:07:09.948984+0000","last_became_active":"2024-04-25T10:07:08.662481+0000","last_became_peered":"2024-04-25T10:07:08.662481+0000","last_unstale":"2024-04-25T10:07:09.948984+0000","last_undegraded":"2024-04-25T10:07:09.948984+0000","last_fullsized":"2024-04-25T10:07:09.948984+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:36.884665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661354+0000","last_change":"2024-04-25T10:07:08.674228+0000","last_active":"2024-04-25T10:07:09.661354+0000","last_peered":"2024-04-25T10:07:09.661354+0000","last_clean":"2024-04-25T10:07:09.661354+0000","last_became_active":"2024-04-25T10:07:08.673895+0000","last_became_peered":"2024-04-25T10:07:08.673895+0000","last_unstale":"2024-04-25T10:07:09.661354+0000","last_undegraded":"2024-04-25T10:07:09.661354+0000","last_fullsized":"2024-04-25T10:07:09.661354+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:20.505697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949709+0000","last_change":"2024-04-25T10:07:08.697904+0000","last_active":"2024-04-25T10:07:09.949709+0000","last_peered":"2024-04-25T10:07:09.949709+0000","last_clean":"2024-04-25T10:07:09.949709+0000","last_became_active":"2024-04-25T10:07:08.697630+0000","last_became_peered":"2024-04-25T10:07:08.697630+0000","last_unstale":"2024-04-25T10:07:09.949709+0000","last_undegraded":"2024-04-25T10:07:09.949709+0000","last_fullsized":"2024-04-25T10:07:09.949709+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:10.939081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.951293+0000","last_change":"2024-04-25T10:07:08.673762+0000","last_active":"2024-04-25T10:07:09.951293+0000","last_peered":"2024-04-25T10:07:09.951293+0000","last_clean":"2024-04-25T10:07:09.951293+0000","last_became_active":"2024-04-25T10:07:08.673008+0000","last_became_peered":"2024-04-25T10:07:08.673008+0000","last_unstale":"2024-04-25T10:07:09.951293+0000","last_undegraded":"2024-04-25T10:07:09.951293+0000","last_fullsized":"2024-04-25T10:07:09.951293+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:58.989660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949682+0000","last_change":"2024-04-25T10:07:08.683621+0000","last_active":"2024-04-25T10:07:09.949682+0000","last_peered":"2024-04-25T10:07:09.949682+0000","last_clean":"2024-04-25T10:07:09.949682+0000","last_became_active":"2024-04-25T10:07:08.682216+0000","last_became_peered":"2024-04-25T10:07:08.682216+0000","last_unstale":"2024-04-25T10:07:09.949682+0000","last_undegraded":"2024-04-25T10:07:09.949682+0000","last_fullsized":"2024-04-25T10:07:09.949682+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:24.838866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950456+0000","last_change":"2024-04-25T10:07:08.661024+0000","last_active":"2024-04-25T10:07:09.950456+0000","last_peered":"2024-04-25T10:07:09.950456+0000","last_clean":"2024-04-25T10:07:09.950456+0000","last_became_active":"2024-04-25T10:07:08.660244+0000","last_became_peered":"2024-04-25T10:07:08.660244+0000","last_unstale":"2024-04-25T10:07:09.950456+0000","last_undegraded":"2024-04-25T10:07:09.950456+0000","last_fullsized":"2024-04-25T10:07:09.950456+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:36.752046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949446+0000","last_change":"2024-04-25T10:07:08.654991+0000","last_active":"2024-04-25T10:07:09.949446+0000","last_peered":"2024-04-25T10:07:09.949446+0000","last_clean":"2024-04-25T10:07:09.949446+0000","last_became_active":"2024-04-25T10:07:08.654393+0000","last_became_peered":"2024-04-25T10:07:08.654393+0000","last_unstale":"2024-04-25T10:07:09.949446+0000","last_undegraded":"2024-04-25T10:07:09.949446+0000","last_fullsized":"2024-04-25T10:07:09.949446+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:06.921997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661015+0000","last_change":"2024-04-25T10:07:08.660629+0000","last_active":"2024-04-25T10:07:09.661015+0000","last_peered":"2024-04-25T10:07:09.661015+0000","last_clean":"2024-04-25T10:07:09.661015+0000","last_became_active":"2024-04-25T10:07:08.657279+0000","last_became_peered":"2024-04-25T10:07:08.657279+0000","last_unstale":"2024-04-25T10:07:09.661015+0000","last_undegraded":"2024-04-25T10:07:09.661015+0000","last_fullsized":"2024-04-25T10:07:09.661015+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:25.902522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.659967+0000","last_change":"2024-04-25T10:07:08.654204+0000","last_active":"2024-04-25T10:07:09.659967+0000","last_peered":"2024-04-25T10:07:09.659967+0000","last_clean":"2024-04-25T10:07:09.659967+0000","last_became_active":"2024-04-25T10:07:08.653136+0000","last_became_peered":"2024-04-25T10:07:08.653136+0000","last_unstale":"2024-04-25T10:07:09.659967+0000","last_undegraded":"2024-04-25T10:07:09.659967+0000","last_fullsized":"2024-04-25T10:07:09.659967+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:25.167540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950303+0000","last_change":"2024-04-25T10:07:08.671968+0000","last_active":"2024-04-25T10:07:09.950303+0000","last_peered":"2024-04-25T10:07:09.950303+0000","last_clean":"2024-04-25T10:07:09.950303+0000","last_became_active":"2024-04-25T10:07:08.662880+0000","last_became_peered":"2024-04-25T10:07:08.662880+0000","last_unstale":"2024-04-25T10:07:09.950303+0000","last_undegraded":"2024-04-25T10:07:09.950303+0000","last_fullsized":"2024-04-25T10:07:09.950303+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:53.086888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.660171+0000","last_change":"2024-04-25T10:07:08.667581+0000","last_active":"2024-04-25T10:07:09.660171+0000","last_peered":"2024-04-25T10:07:09.660171+0000","last_clean":"2024-04-25T10:07:09.660171+0000","last_became_active":"2024-04-25T10:07:08.666815+0000","last_became_peered":"2024-04-25T10:07:08.666815+0000","last_unstale":"2024-04-25T10:07:09.660171+0000","last_undegraded":"2024-04-25T10:07:09.660171+0000","last_fullsized":"2024-04-25T10:07:09.660171+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:32.688035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661064+0000","last_change":"2024-04-25T10:07:08.660010+0000","last_active":"2024-04-25T10:07:09.661064+0000","last_peered":"2024-04-25T10:07:09.661064+0000","last_clean":"2024-04-25T10:07:09.661064+0000","last_became_active":"2024-04-25T10:07:08.659234+0000","last_became_peered":"2024-04-25T10:07:08.659234+0000","last_unstale":"2024-04-25T10:07:09.661064+0000","last_undegraded":"2024-04-25T10:07:09.661064+0000","last_fullsized":"2024-04-25T10:07:09.661064+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:33.067413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948434+0000","last_change":"2024-04-25T10:07:08.648491+0000","last_active":"2024-04-25T10:07:09.948434+0000","last_peered":"2024-04-25T10:07:09.948434+0000","last_clean":"2024-04-25T10:07:09.948434+0000","last_became_active":"2024-04-25T10:07:08.647710+0000","last_became_peered":"2024-04-25T10:07:08.647710+0000","last_unstale":"2024-04-25T10:07:09.948434+0000","last_undegraded":"2024-04-25T10:07:09.948434+0000","last_fullsized":"2024-04-25T10:07:09.948434+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:41.621343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.657032+0000","last_change":"2024-04-25T10:07:08.672817+0000","last_active":"2024-04-25T10:07:09.657032+0000","last_peered":"2024-04-25T10:07:09.657032+0000","last_clean":"2024-04-25T10:07:09.657032+0000","last_became_active":"2024-04-25T10:07:08.672477+0000","last_became_peered":"2024-04-25T10:07:08.672477+0000","last_unstale":"2024-04-25T10:07:09.657032+0000","last_undegraded":"2024-04-25T10:07:09.657032+0000","last_fullsized":"2024-04-25T10:07:09.657032+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:51.290935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661578+0000","last_change":"2024-04-25T10:07:07.645402+0000","last_active":"2024-04-25T10:07:09.661578+0000","last_peered":"2024-04-25T10:07:09.661578+0000","last_clean":"2024-04-25T10:07:09.661578+0000","last_became_active":"2024-04-25T10:07:07.644964+0000","last_became_peered":"2024-04-25T10:07:07.644964+0000","last_unstale":"2024-04-25T10:07:09.661578+0000","last_undegraded":"2024-04-25T10:07:09.661578+0000","last_fullsized":"2024-04-25T10:07:09.661578+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:35.915922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950676+0000","last_change":"2024-04-25T10:07:08.661409+0000","last_active":"2024-04-25T10:07:09.950676+0000","last_peered":"2024-04-25T10:07:09.950676+0000","last_clean":"2024-04-25T10:07:09.950676+0000","last_became_active":"2024-04-25T10:07:08.660766+0000","last_became_peered":"2024-04-25T10:07:08.660766+0000","last_unstale":"2024-04-25T10:07:09.950676+0000","last_undegraded":"2024-04-25T10:07:09.950676+0000","last_fullsized":"2024-04-25T10:07:09.950676+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:05.519316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661437+0000","last_change":"2024-04-25T10:07:07.655701+0000","last_active":"2024-04-25T10:07:09.661437+0000","last_peered":"2024-04-25T10:07:09.661437+0000","last_clean":"2024-04-25T10:07:09.661437+0000","last_became_active":"2024-04-25T10:07:07.653511+0000","last_became_peered":"2024-04-25T10:07:07.653511+0000","last_unstale":"2024-04-25T10:07:09.661437+0000","last_undegraded":"2024-04-25T10:07:09.661437+0000","last_fullsized":"2024-04-25T10:07:09.661437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:13.928674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.660867+0000","last_change":"2024-04-25T10:07:08.675806+0000","last_active":"2024-04-25T10:07:09.660867+0000","last_peered":"2024-04-25T10:07:09.660867+0000","last_clean":"2024-04-25T10:07:09.660867+0000","last_became_active":"2024-04-25T10:07:08.675208+0000","last_became_peered":"2024-04-25T10:07:08.675208+0000","last_unstale":"2024-04-25T10:07:09.660867+0000","last_undegraded":"2024-04-25T10:07:09.660867+0000","last_fullsized":"2024-04-25T10:07:09.660867+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:46.314031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.657793+0000","last_change":"2024-04-25T10:07:07.648517+0000","last_active":"2024-04-25T10:07:09.657793+0000","last_peered":"2024-04-25T10:07:09.657793+0000","last_clean":"2024-04-25T10:07:09.657793+0000","last_became_active":"2024-04-25T10:07:07.648183+0000","last_became_peered":"2024-04-25T10:07:07.648183+0000","last_unstale":"2024-04-25T10:07:09.657793+0000","last_undegraded":"2024-04-25T10:07:09.657793+0000","last_fullsized":"2024-04-25T10:07:09.657793+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.578592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.657722+0000","last_change":"2024-04-25T10:07:08.663431+0000","last_active":"2024-04-25T10:07:09.657722+0000","last_peered":"2024-04-25T10:07:09.657722+0000","last_clean":"2024-04-25T10:07:09.657722+0000","last_became_active":"2024-04-25T10:07:08.663153+0000","last_became_peered":"2024-04-25T10:07:08.663153+0000","last_unstale":"2024-04-25T10:07:09.657722+0000","last_undegraded":"2024-04-25T10:07:09.657722+0000","last_fullsized":"2024-04-25T10:07:09.657722+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:54.356296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.951521+0000","last_change":"2024-04-25T10:07:07.661239+0000","last_active":"2024-04-25T10:07:09.951521+0000","last_peered":"2024-04-25T10:07:09.951521+0000","last_clean":"2024-04-25T10:07:09.951521+0000","last_became_active":"2024-04-25T10:07:07.660654+0000","last_became_peered":"2024-04-25T10:07:07.660654+0000","last_unstale":"2024-04-25T10:07:09.951521+0000","last_undegraded":"2024-04-25T10:07:09.951521+0000","last_fullsized":"2024-04-25T10:07:09.951521+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:23.652463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.659907+0000","last_change":"2024-04-25T10:07:08.661981+0000","last_active":"2024-04-25T10:07:09.659907+0000","last_peered":"2024-04-25T10:07:09.659907+0000","last_clean":"2024-04-25T10:07:09.659907+0000","last_became_active":"2024-04-25T10:07:08.661269+0000","last_became_peered":"2024-04-25T10:07:08.661269+0000","last_unstale":"2024-04-25T10:07:09.659907+0000","last_undegraded":"2024-04-25T10:07:09.659907+0000","last_fullsized":"2024-04-25T10:07:09.659907+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:37.443625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661190+0000","last_change":"2024-04-25T10:07:07.637715+0000","last_active":"2024-04-25T10:07:09.661190+0000","last_peered":"2024-04-25T10:07:09.661190+0000","last_clean":"2024-04-25T10:07:09.661190+0000","last_became_active":"2024-04-25T10:07:07.637192+0000","last_became_peered":"2024-04-25T10:07:07.637192+0000","last_unstale":"2024-04-25T10:07:09.661190+0000","last_undegraded":"2024-04-25T10:07:09.661190+0000","last_fullsized":"2024-04-25T10:07:09.661190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:34.432109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950543+0000","last_change":"2024-04-25T10:07:08.665669+0000","last_active":"2024-04-25T10:07:09.950543+0000","last_peered":"2024-04-25T10:07:09.950543+0000","last_clean":"2024-04-25T10:07:09.950543+0000","last_became_active":"2024-04-25T10:07:08.664399+0000","last_became_peered":"2024-04-25T10:07:08.664399+0000","last_unstale":"2024-04-25T10:07:09.950543+0000","last_undegraded":"2024-04-25T10:07:09.950543+0000","last_fullsized":"2024-04-25T10:07:09.950543+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:13.428895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.660237+0000","last_change":"2024-04-25T10:07:07.657392+0000","last_active":"2024-04-25T10:07:09.660237+0000","last_peered":"2024-04-25T10:07:09.660237+0000","last_clean":"2024-04-25T10:07:09.660237+0000","last_became_active":"2024-04-25T10:07:07.656810+0000","last_became_peered":"2024-04-25T10:07:07.656810+0000","last_unstale":"2024-04-25T10:07:09.660237+0000","last_undegraded":"2024-04-25T10:07:09.660237+0000","last_fullsized":"2024-04-25T10:07:09.660237+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:52.190399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949662+0000","last_change":"2024-04-25T10:07:08.696119+0000","last_active":"2024-04-25T10:07:09.949662+0000","last_peered":"2024-04-25T10:07:09.949662+0000","last_clean":"2024-04-25T10:07:09.949662+0000","last_became_active":"2024-04-25T10:07:08.695408+0000","last_became_peered":"2024-04-25T10:07:08.695408+0000","last_unstale":"2024-04-25T10:07:09.949662+0000","last_undegraded":"2024-04-25T10:07:09.949662+0000","last_fullsized":"2024-04-25T10:07:09.949662+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:25.943908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.659176+0000","last_change":"2024-04-25T10:07:07.647409+0000","last_active":"2024-04-25T10:07:09.659176+0000","last_peered":"2024-04-25T10:07:09.659176+0000","last_clean":"2024-04-25T10:07:09.659176+0000","last_became_active":"2024-04-25T10:07:07.646680+0000","last_became_peered":"2024-04-25T10:07:07.646680+0000","last_unstale":"2024-04-25T10:07:09.659176+0000","last_undegraded":"2024-04-25T10:07:09.659176+0000","last_fullsized":"2024-04-25T10:07:09.659176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:32.974250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949802+0000","last_change":"2024-04-25T10:07:08.682581+0000","last_active":"2024-04-25T10:07:09.949802+0000","last_peered":"2024-04-25T10:07:09.949802+0000","last_clean":"2024-04-25T10:07:09.949802+0000","last_became_active":"2024-04-25T10:07:08.673438+0000","last_became_peered":"2024-04-25T10:07:08.673438+0000","last_unstale":"2024-04-25T10:07:09.949802+0000","last_undegraded":"2024-04-25T10:07:09.949802+0000","last_fullsized":"2024-04-25T10:07:09.949802+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:18.931476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949439+0000","last_change":"2024-04-25T10:07:07.635219+0000","last_active":"2024-04-25T10:07:09.949439+0000","last_peered":"2024-04-25T10:07:09.949439+0000","last_clean":"2024-04-25T10:07:09.949439+0000","last_became_active":"2024-04-25T10:07:07.633856+0000","last_became_peered":"2024-04-25T10:07:07.633856+0000","last_unstale":"2024-04-25T10:07:09.949439+0000","last_undegraded":"2024-04-25T10:07:09.949439+0000","last_fullsized":"2024-04-25T10:07:09.949439+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:18.596183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661619+0000","last_change":"2024-04-25T10:07:08.670275+0000","last_active":"2024-04-25T10:07:09.661619+0000","last_peered":"2024-04-25T10:07:09.661619+0000","last_clean":"2024-04-25T10:07:09.661619+0000","last_became_active":"2024-04-25T10:07:08.669423+0000","last_became_peered":"2024-04-25T10:07:08.669423+0000","last_unstale":"2024-04-25T10:07:09.661619+0000","last_undegraded":"2024-04-25T10:07:09.661619+0000","last_fullsized":"2024-04-25T10:07:09.661619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:44.327012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661624+0000","last_change":"2024-04-25T10:07:07.639391+0000","last_active":"2024-04-25T10:07:09.661624+0000","last_peered":"2024-04-25T10:07:09.661624+0000","last_clean":"2024-04-25T10:07:09.661624+0000","last_became_active":"2024-04-25T10:07:07.638645+0000","last_became_peered":"2024-04-25T10:07:07.638645+0000","last_unstale":"2024-04-25T10:07:09.661624+0000","last_undegraded":"2024-04-25T10:07:09.661624+0000","last_fullsized":"2024-04-25T10:07:09.661624+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.578592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.951869+0000","last_change":"2024-04-25T10:07:08.676443+0000","last_active":"2024-04-25T10:07:09.951869+0000","last_peered":"2024-04-25T10:07:09.951869+0000","last_clean":"2024-04-25T10:07:09.951869+0000","last_became_active":"2024-04-25T10:07:08.675684+0000","last_became_peered":"2024-04-25T10:07:08.675684+0000","last_unstale":"2024-04-25T10:07:09.951869+0000","last_undegraded":"2024-04-25T10:07:09.951869+0000","last_fullsized":"2024-04-25T10:07:09.951869+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:38.216511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950434+0000","last_change":"2024-04-25T10:07:07.639070+0000","last_active":"2024-04-25T10:07:09.950434+0000","last_peered":"2024-04-25T10:07:09.950434+0000","last_clean":"2024-04-25T10:07:09.950434+0000","last_became_active":"2024-04-25T10:07:07.637998+0000","last_became_peered":"2024-04-25T10:07:07.637998+0000","last_unstale":"2024-04-25T10:07:09.950434+0000","last_undegraded":"2024-04-25T10:07:09.950434+0000","last_fullsized":"2024-04-25T10:07:09.950434+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:41.783298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661511+0000","last_change":"2024-04-25T10:07:08.656781+0000","last_active":"2024-04-25T10:07:09.661511+0000","last_peered":"2024-04-25T10:07:09.661511+0000","last_clean":"2024-04-25T10:07:09.661511+0000","last_became_active":"2024-04-25T10:07:08.656024+0000","last_became_peered":"2024-04-25T10:07:08.656024+0000","last_unstale":"2024-04-25T10:07:09.661511+0000","last_undegraded":"2024-04-25T10:07:09.661511+0000","last_fullsized":"2024-04-25T10:07:09.661511+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:16.459887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.951988+0000","last_change":"2024-04-25T10:07:07.664212+0000","last_active":"2024-04-25T10:07:09.951988+0000","last_peered":"2024-04-25T10:07:09.951988+0000","last_clean":"2024-04-25T10:07:09.951988+0000","last_became_active":"2024-04-25T10:07:07.663582+0000","last_became_peered":"2024-04-25T10:07:07.663582+0000","last_unstale":"2024-04-25T10:07:09.951988+0000","last_undegraded":"2024-04-25T10:07:09.951988+0000","last_fullsized":"2024-04-25T10:07:09.951988+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:32.078953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.951975+0000","last_change":"2024-04-25T10:07:08.667658+0000","last_active":"2024-04-25T10:07:09.951975+0000","last_peered":"2024-04-25T10:07:09.951975+0000","last_clean":"2024-04-25T10:07:09.951975+0000","last_became_active":"2024-04-25T10:07:08.666183+0000","last_became_peered":"2024-04-25T10:07:08.666183+0000","last_unstale":"2024-04-25T10:07:09.951975+0000","last_undegraded":"2024-04-25T10:07:09.951975+0000","last_fullsized":"2024-04-25T10:07:09.951975+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:36.712660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948922+0000","last_change":"2024-04-25T10:07:07.641278+0000","last_active":"2024-04-25T10:07:09.948922+0000","last_peered":"2024-04-25T10:07:09.948922+0000","last_clean":"2024-04-25T10:07:09.948922+0000","last_became_active":"2024-04-25T10:07:07.640843+0000","last_became_peered":"2024-04-25T10:07:07.640843+0000","last_unstale":"2024-04-25T10:07:09.948922+0000","last_undegraded":"2024-04-25T10:07:09.948922+0000","last_fullsized":"2024-04-25T10:07:09.948922+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:21.231828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950245+0000","last_change":"2024-04-25T10:07:08.687193+0000","last_active":"2024-04-25T10:07:09.950245+0000","last_peered":"2024-04-25T10:07:09.950245+0000","last_clean":"2024-04-25T10:07:09.950245+0000","last_became_active":"2024-04-25T10:07:08.686381+0000","last_became_peered":"2024-04-25T10:07:08.686381+0000","last_unstale":"2024-04-25T10:07:09.950245+0000","last_undegraded":"2024-04-25T10:07:09.950245+0000","last_fullsized":"2024-04-25T10:07:09.950245+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:08.123048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661299+0000","last_change":"2024-04-25T10:07:07.636262+0000","last_active":"2024-04-25T10:07:09.661299+0000","last_peered":"2024-04-25T10:07:09.661299+0000","last_clean":"2024-04-25T10:07:09.661299+0000","last_became_active":"2024-04-25T10:07:07.635974+0000","last_became_peered":"2024-04-25T10:07:07.635974+0000","last_unstale":"2024-04-25T10:07:09.661299+0000","last_undegraded":"2024-04-25T10:07:09.661299+0000","last_fullsized":"2024-04-25T10:07:09.661299+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:12.420194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950414+0000","last_change":"2024-04-25T10:07:08.679079+0000","last_active":"2024-04-25T10:07:09.950414+0000","last_peered":"2024-04-25T10:07:09.950414+0000","last_clean":"2024-04-25T10:07:09.950414+0000","last_became_active":"2024-04-25T10:07:08.676480+0000","last_became_peered":"2024-04-25T10:07:08.676480+0000","last_unstale":"2024-04-25T10:07:09.950414+0000","last_undegraded":"2024-04-25T10:07:09.950414+0000","last_fullsized":"2024-04-25T10:07:09.950414+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:35.440810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.951286+0000","last_change":"2024-04-25T10:07:07.657378+0000","last_active":"2024-04-25T10:07:09.951286+0000","last_peered":"2024-04-25T10:07:09.951286+0000","last_clean":"2024-04-25T10:07:09.951286+0000","last_became_active":"2024-04-25T10:07:07.656779+0000","last_became_peered":"2024-04-25T10:07:07.656779+0000","last_unstale":"2024-04-25T10:07:09.951286+0000","last_undegraded":"2024-04-25T10:07:09.951286+0000","last_fullsized":"2024-04-25T10:07:09.951286+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:40.838356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661648+0000","last_change":"2024-04-25T10:07:08.647206+0000","last_active":"2024-04-25T10:07:09.661648+0000","last_peered":"2024-04-25T10:07:09.661648+0000","last_clean":"2024-04-25T10:07:09.661648+0000","last_became_active":"2024-04-25T10:07:08.646487+0000","last_became_peered":"2024-04-25T10:07:08.646487+0000","last_unstale":"2024-04-25T10:07:09.661648+0000","last_undegraded":"2024-04-25T10:07:09.661648+0000","last_fullsized":"2024-04-25T10:07:09.661648+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:28.242898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661754+0000","last_change":"2024-04-25T10:07:07.633387+0000","last_active":"2024-04-25T10:07:09.661754+0000","last_peered":"2024-04-25T10:07:09.661754+0000","last_clean":"2024-04-25T10:07:09.661754+0000","last_became_active":"2024-04-25T10:07:07.632167+0000","last_became_peered":"2024-04-25T10:07:07.632167+0000","last_unstale":"2024-04-25T10:07:09.661754+0000","last_undegraded":"2024-04-25T10:07:09.661754+0000","last_fullsized":"2024-04-25T10:07:09.661754+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:12.338449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.951154+0000","last_change":"2024-04-25T10:07:08.697319+0000","last_active":"2024-04-25T10:07:09.951154+0000","last_peered":"2024-04-25T10:07:09.951154+0000","last_clean":"2024-04-25T10:07:09.951154+0000","last_became_active":"2024-04-25T10:07:08.696669+0000","last_became_peered":"2024-04-25T10:07:08.696669+0000","last_unstale":"2024-04-25T10:07:09.951154+0000","last_undegraded":"2024-04-25T10:07:09.951154+0000","last_fullsized":"2024-04-25T10:07:09.951154+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:48.325042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":22,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949731+0000","last_change":"2024-04-25T10:07:07.649029+0000","last_active":"2024-04-25T10:07:09.949731+0000","last_peered":"2024-04-25T10:07:09.949731+0000","last_clean":"2024-04-25T10:07:09.949731+0000","last_became_active":"2024-04-25T10:07:07.648616+0000","last_became_peered":"2024-04-25T10:07:07.648616+0000","last_unstale":"2024-04-25T10:07:09.949731+0000","last_undegraded":"2024-04-25T10:07:09.949731+0000","last_fullsized":"2024-04-25T10:07:09.949731+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:11.313783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949188+0000","last_change":"2024-04-25T10:07:08.649502+0000","last_active":"2024-04-25T10:07:09.949188+0000","last_peered":"2024-04-25T10:07:09.949188+0000","last_clean":"2024-04-25T10:07:09.949188+0000","last_became_active":"2024-04-25T10:07:08.648547+0000","last_became_peered":"2024-04-25T10:07:08.648547+0000","last_unstale":"2024-04-25T10:07:09.949188+0000","last_undegraded":"2024-04-25T10:07:09.949188+0000","last_fullsized":"2024-04-25T10:07:09.949188+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:32.108613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948265+0000","last_change":"2024-04-25T10:07:07.627080+0000","last_active":"2024-04-25T10:07:09.948265+0000","last_peered":"2024-04-25T10:07:09.948265+0000","last_clean":"2024-04-25T10:07:09.948265+0000","last_became_active":"2024-04-25T10:07:07.626664+0000","last_became_peered":"2024-04-25T10:07:07.626664+0000","last_unstale":"2024-04-25T10:07:09.948265+0000","last_undegraded":"2024-04-25T10:07:09.948265+0000","last_fullsized":"2024-04-25T10:07:09.948265+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:00.068508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950706+0000","last_change":"2024-04-25T10:07:08.676048+0000","last_active":"2024-04-25T10:07:09.950706+0000","last_peered":"2024-04-25T10:07:09.950706+0000","last_clean":"2024-04-25T10:07:09.950706+0000","last_became_active":"2024-04-25T10:07:08.675743+0000","last_became_peered":"2024-04-25T10:07:08.675743+0000","last_unstale":"2024-04-25T10:07:09.950706+0000","last_undegraded":"2024-04-25T10:07:09.950706+0000","last_fullsized":"2024-04-25T10:07:09.950706+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:51.945868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661756+0000","last_change":"2024-04-25T10:07:07.655902+0000","last_active":"2024-04-25T10:07:09.661756+0000","last_peered":"2024-04-25T10:07:09.661756+0000","last_clean":"2024-04-25T10:07:09.661756+0000","last_became_active":"2024-04-25T10:07:07.654431+0000","last_became_peered":"2024-04-25T10:07:07.654431+0000","last_unstale":"2024-04-25T10:07:09.661756+0000","last_undegraded":"2024-04-25T10:07:09.661756+0000","last_fullsized":"2024-04-25T10:07:09.661756+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:35.334997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950075+0000","last_change":"2024-04-25T10:07:08.695389+0000","last_active":"2024-04-25T10:07:09.950075+0000","last_peered":"2024-04-25T10:07:09.950075+0000","last_clean":"2024-04-25T10:07:09.950075+0000","last_became_active":"2024-04-25T10:07:08.693944+0000","last_became_peered":"2024-04-25T10:07:08.693944+0000","last_unstale":"2024-04-25T10:07:09.950075+0000","last_undegraded":"2024-04-25T10:07:09.950075+0000","last_fullsized":"2024-04-25T10:07:09.950075+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:48.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661113+0000","last_change":"2024-04-25T10:07:07.647747+0000","last_active":"2024-04-25T10:07:09.661113+0000","last_peered":"2024-04-25T10:07:09.661113+0000","last_clean":"2024-04-25T10:07:09.661113+0000","last_became_active":"2024-04-25T10:07:07.647009+0000","last_became_peered":"2024-04-25T10:07:07.647009+0000","last_unstale":"2024-04-25T10:07:09.661113+0000","last_undegraded":"2024-04-25T10:07:09.661113+0000","last_fullsized":"2024-04-25T10:07:09.661113+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:16.783244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950504+0000","last_change":"2024-04-25T10:07:08.672969+0000","last_active":"2024-04-25T10:07:09.950504+0000","last_peered":"2024-04-25T10:07:09.950504+0000","last_clean":"2024-04-25T10:07:09.950504+0000","last_became_active":"2024-04-25T10:07:08.663043+0000","last_became_peered":"2024-04-25T10:07:08.663043+0000","last_unstale":"2024-04-25T10:07:09.950504+0000","last_undegraded":"2024-04-25T10:07:09.950504+0000","last_fullsized":"2024-04-25T10:07:09.950504+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:40.481239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661568+0000","last_change":"2024-04-25T10:07:07.638916+0000","last_active":"2024-04-25T10:07:09.661568+0000","last_peered":"2024-04-25T10:07:09.661568+0000","last_clean":"2024-04-25T10:07:09.661568+0000","last_became_active":"2024-04-25T10:07:07.638374+0000","last_became_peered":"2024-04-25T10:07:07.638374+0000","last_unstale":"2024-04-25T10:07:09.661568+0000","last_undegraded":"2024-04-25T10:07:09.661568+0000","last_fullsized":"2024-04-25T10:07:09.661568+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:54.213027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661174+0000","last_change":"2024-04-25T10:07:08.662023+0000","last_active":"2024-04-25T10:07:09.661174+0000","last_peered":"2024-04-25T10:07:09.661174+0000","last_clean":"2024-04-25T10:07:09.661174+0000","last_became_active":"2024-04-25T10:07:08.661348+0000","last_became_peered":"2024-04-25T10:07:08.661348+0000","last_unstale":"2024-04-25T10:07:09.661174+0000","last_undegraded":"2024-04-25T10:07:09.661174+0000","last_fullsized":"2024-04-25T10:07:09.661174+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:18.547456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661521+0000","last_change":"2024-04-25T10:07:07.637481+0000","last_active":"2024-04-25T10:07:09.661521+0000","last_peered":"2024-04-25T10:07:09.661521+0000","last_clean":"2024-04-25T10:07:09.661521+0000","last_became_active":"2024-04-25T10:07:07.636896+0000","last_became_peered":"2024-04-25T10:07:07.636896+0000","last_unstale":"2024-04-25T10:07:09.661521+0000","last_undegraded":"2024-04-25T10:07:09.661521+0000","last_fullsized":"2024-04-25T10:07:09.661521+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:35.503057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.656958+0000","last_change":"2024-04-25T10:07:08.645533+0000","last_active":"2024-04-25T10:07:09.656958+0000","last_peered":"2024-04-25T10:07:09.656958+0000","last_clean":"2024-04-25T10:07:09.656958+0000","last_became_active":"2024-04-25T10:07:08.644698+0000","last_became_peered":"2024-04-25T10:07:08.644698+0000","last_unstale":"2024-04-25T10:07:09.656958+0000","last_undegraded":"2024-04-25T10:07:09.656958+0000","last_fullsized":"2024-04-25T10:07:09.656958+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:00.872405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.659296+0000","last_change":"2024-04-25T10:07:07.647335+0000","last_active":"2024-04-25T10:07:09.659296+0000","last_peered":"2024-04-25T10:07:09.659296+0000","last_clean":"2024-04-25T10:07:09.659296+0000","last_became_active":"2024-04-25T10:07:07.646551+0000","last_became_peered":"2024-04-25T10:07:07.646551+0000","last_unstale":"2024-04-25T10:07:09.659296+0000","last_undegraded":"2024-04-25T10:07:09.659296+0000","last_fullsized":"2024-04-25T10:07:09.659296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:10.241456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948691+0000","last_change":"2024-04-25T10:07:08.668558+0000","last_active":"2024-04-25T10:07:09.948691+0000","last_peered":"2024-04-25T10:07:09.948691+0000","last_clean":"2024-04-25T10:07:09.948691+0000","last_became_active":"2024-04-25T10:07:08.668272+0000","last_became_peered":"2024-04-25T10:07:08.668272+0000","last_unstale":"2024-04-25T10:07:09.948691+0000","last_undegraded":"2024-04-25T10:07:09.948691+0000","last_fullsized":"2024-04-25T10:07:09.948691+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:08.373478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948600+0000","last_change":"2024-04-25T10:07:07.651220+0000","last_active":"2024-04-25T10:07:09.948600+0000","last_peered":"2024-04-25T10:07:09.948600+0000","last_clean":"2024-04-25T10:07:09.948600+0000","last_became_active":"2024-04-25T10:07:07.650901+0000","last_became_peered":"2024-04-25T10:07:07.650901+0000","last_unstale":"2024-04-25T10:07:09.948600+0000","last_undegraded":"2024-04-25T10:07:09.948600+0000","last_fullsized":"2024-04-25T10:07:09.948600+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:16.400436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.948483+0000","last_change":"2024-04-25T10:07:08.668430+0000","last_active":"2024-04-25T10:07:09.948483+0000","last_peered":"2024-04-25T10:07:09.948483+0000","last_clean":"2024-04-25T10:07:09.948483+0000","last_became_active":"2024-04-25T10:07:08.665538+0000","last_became_peered":"2024-04-25T10:07:08.665538+0000","last_unstale":"2024-04-25T10:07:09.948483+0000","last_undegraded":"2024-04-25T10:07:09.948483+0000","last_fullsized":"2024-04-25T10:07:09.948483+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:15.389333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950495+0000","last_change":"2024-04-25T10:07:07.651442+0000","last_active":"2024-04-25T10:07:09.950495+0000","last_peered":"2024-04-25T10:07:09.950495+0000","last_clean":"2024-04-25T10:07:09.950495+0000","last_became_active":"2024-04-25T10:07:07.649340+0000","last_became_peered":"2024-04-25T10:07:07.649340+0000","last_unstale":"2024-04-25T10:07:09.950495+0000","last_undegraded":"2024-04-25T10:07:09.950495+0000","last_fullsized":"2024-04-25T10:07:09.950495+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:56.146350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950929+0000","last_change":"2024-04-25T10:07:08.692556+0000","last_active":"2024-04-25T10:07:09.950929+0000","last_peered":"2024-04-25T10:07:09.950929+0000","last_clean":"2024-04-25T10:07:09.950929+0000","last_became_active":"2024-04-25T10:07:08.691811+0000","last_became_peered":"2024-04-25T10:07:08.691811+0000","last_unstale":"2024-04-25T10:07:09.950929+0000","last_undegraded":"2024-04-25T10:07:09.950929+0000","last_fullsized":"2024-04-25T10:07:09.950929+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:55.114883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"47'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661696+0000","last_change":"2024-04-25T10:07:07.663123+0000","last_active":"2024-04-25T10:07:09.661696+0000","last_peered":"2024-04-25T10:07:09.661696+0000","last_clean":"2024-04-25T10:07:09.661696+0000","last_became_active":"2024-04-25T10:07:07.662759+0000","last_became_peered":"2024-04-25T10:07:07.662759+0000","last_unstale":"2024-04-25T10:07:09.661696+0000","last_undegraded":"2024-04-25T10:07:09.661696+0000","last_fullsized":"2024-04-25T10:07:09.661696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:24.935207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950853+0000","last_change":"2024-04-25T10:07:08.674835+0000","last_active":"2024-04-25T10:07:09.950853+0000","last_peered":"2024-04-25T10:07:09.950853+0000","last_clean":"2024-04-25T10:07:09.950853+0000","last_became_active":"2024-04-25T10:07:08.674302+0000","last_became_peered":"2024-04-25T10:07:08.674302+0000","last_unstale":"2024-04-25T10:07:09.950853+0000","last_undegraded":"2024-04-25T10:07:09.950853+0000","last_fullsized":"2024-04-25T10:07:09.950853+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:46.314031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.952158+0000","last_change":"2024-04-25T10:07:07.665347+0000","last_active":"2024-04-25T10:07:09.952158+0000","last_peered":"2024-04-25T10:07:09.952158+0000","last_clean":"2024-04-25T10:07:09.952158+0000","last_became_active":"2024-04-25T10:07:07.664822+0000","last_became_peered":"2024-04-25T10:07:07.664822+0000","last_unstale":"2024-04-25T10:07:09.952158+0000","last_undegraded":"2024-04-25T10:07:09.952158+0000","last_fullsized":"2024-04-25T10:07:09.952158+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:36.434923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.951682+0000","last_change":"2024-04-25T10:07:08.668258+0000","last_active":"2024-04-25T10:07:09.951682+0000","last_peered":"2024-04-25T10:07:09.951682+0000","last_clean":"2024-04-25T10:07:09.951682+0000","last_became_active":"2024-04-25T10:07:08.666916+0000","last_became_peered":"2024-04-25T10:07:08.666916+0000","last_unstale":"2024-04-25T10:07:09.951682+0000","last_undegraded":"2024-04-25T10:07:09.951682+0000","last_fullsized":"2024-04-25T10:07:09.951682+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:50.593878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.657015+0000","last_change":"2024-04-25T10:07:07.648943+0000","last_active":"2024-04-25T10:07:09.657015+0000","last_peered":"2024-04-25T10:07:09.657015+0000","last_clean":"2024-04-25T10:07:09.657015+0000","last_became_active":"2024-04-25T10:07:07.648706+0000","last_became_peered":"2024-04-25T10:07:07.648706+0000","last_unstale":"2024-04-25T10:07:09.657015+0000","last_undegraded":"2024-04-25T10:07:09.657015+0000","last_fullsized":"2024-04-25T10:07:09.657015+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:52.190399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661113+0000","last_change":"2024-04-25T10:07:08.651605+0000","last_active":"2024-04-25T10:07:09.661113+0000","last_peered":"2024-04-25T10:07:09.661113+0000","last_clean":"2024-04-25T10:07:09.661113+0000","last_became_active":"2024-04-25T10:07:08.651056+0000","last_became_peered":"2024-04-25T10:07:08.651056+0000","last_unstale":"2024-04-25T10:07:09.661113+0000","last_undegraded":"2024-04-25T10:07:09.661113+0000","last_fullsized":"2024-04-25T10:07:09.661113+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:20.289026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.951611+0000","last_change":"2024-04-25T10:07:07.661444+0000","last_active":"2024-04-25T10:07:09.951611+0000","last_peered":"2024-04-25T10:07:09.951611+0000","last_clean":"2024-04-25T10:07:09.951611+0000","last_became_active":"2024-04-25T10:07:07.660752+0000","last_became_peered":"2024-04-25T10:07:07.660752+0000","last_unstale":"2024-04-25T10:07:09.951611+0000","last_undegraded":"2024-04-25T10:07:09.951611+0000","last_fullsized":"2024-04-25T10:07:09.951611+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:51.552392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.661139+0000","last_change":"2024-04-25T10:07:08.673869+0000","last_active":"2024-04-25T10:07:09.661139+0000","last_peered":"2024-04-25T10:07:09.661139+0000","last_clean":"2024-04-25T10:07:09.661139+0000","last_became_active":"2024-04-25T10:07:08.673558+0000","last_became_peered":"2024-04-25T10:07:08.673558+0000","last_unstale":"2024-04-25T10:07:09.661139+0000","last_undegraded":"2024-04-25T10:07:09.661139+0000","last_fullsized":"2024-04-25T10:07:09.661139+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:47.631323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.660445+0000","last_change":"2024-04-25T10:07:07.628649+0000","last_active":"2024-04-25T10:07:09.660445+0000","last_peered":"2024-04-25T10:07:09.660445+0000","last_clean":"2024-04-25T10:07:09.660445+0000","last_became_active":"2024-04-25T10:07:07.627827+0000","last_became_peered":"2024-04-25T10:07:07.627827+0000","last_unstale":"2024-04-25T10:07:09.660445+0000","last_undegraded":"2024-04-25T10:07:09.660445+0000","last_fullsized":"2024-04-25T10:07:09.660445+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:06.777436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.659188+0000","last_change":"2024-04-25T10:07:08.645430+0000","last_active":"2024-04-25T10:07:09.659188+0000","last_peered":"2024-04-25T10:07:09.659188+0000","last_clean":"2024-04-25T10:07:09.659188+0000","last_became_active":"2024-04-25T10:07:08.644595+0000","last_became_peered":"2024-04-25T10:07:08.644595+0000","last_unstale":"2024-04-25T10:07:09.659188+0000","last_undegraded":"2024-04-25T10:07:09.659188+0000","last_fullsized":"2024-04-25T10:07:09.659188+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:13.428895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.656737+0000","last_change":"2024-04-25T10:07:07.644691+0000","last_active":"2024-04-25T10:07:09.656737+0000","last_peered":"2024-04-25T10:07:09.656737+0000","last_clean":"2024-04-25T10:07:09.656737+0000","last_became_active":"2024-04-25T10:07:07.644108+0000","last_became_peered":"2024-04-25T10:07:07.644108+0000","last_unstale":"2024-04-25T10:07:09.656737+0000","last_undegraded":"2024-04-25T10:07:09.656737+0000","last_fullsized":"2024-04-25T10:07:09.656737+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:06.777436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.659933+0000","last_change":"2024-04-25T10:07:08.675846+0000","last_active":"2024-04-25T10:07:09.659933+0000","last_peered":"2024-04-25T10:07:09.659933+0000","last_clean":"2024-04-25T10:07:09.659933+0000","last_became_active":"2024-04-25T10:07:08.675038+0000","last_became_peered":"2024-04-25T10:07:08.675038+0000","last_unstale":"2024-04-25T10:07:09.659933+0000","last_undegraded":"2024-04-25T10:07:09.659933+0000","last_fullsized":"2024-04-25T10:07:09.659933+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:57.155052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.949797+0000","last_change":"2024-04-25T10:07:07.634215+0000","last_active":"2024-04-25T10:07:09.949797+0000","last_peered":"2024-04-25T10:07:09.949797+0000","last_clean":"2024-04-25T10:07:09.949797+0000","last_became_active":"2024-04-25T10:07:07.633430+0000","last_became_peered":"2024-04-25T10:07:07.633430+0000","last_unstale":"2024-04-25T10:07:09.949797+0000","last_undegraded":"2024-04-25T10:07:09.949797+0000","last_fullsized":"2024-04-25T10:07:09.949797+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:42.026976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950722+0000","last_change":"2024-04-25T10:07:08.672738+0000","last_active":"2024-04-25T10:07:09.950722+0000","last_peered":"2024-04-25T10:07:09.950722+0000","last_clean":"2024-04-25T10:07:09.950722+0000","last_became_active":"2024-04-25T10:07:08.672207+0000","last_became_peered":"2024-04-25T10:07:08.672207+0000","last_unstale":"2024-04-25T10:07:09.950722+0000","last_undegraded":"2024-04-25T10:07:09.950722+0000","last_fullsized":"2024-04-25T10:07:09.950722+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.606865+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.606865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:25.167540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.662031+0000","last_change":"2024-04-25T10:07:07.647454+0000","last_active":"2024-04-25T10:07:09.662031+0000","last_peered":"2024-04-25T10:07:09.662031+0000","last_clean":"2024-04-25T10:07:09.662031+0000","last_became_active":"2024-04-25T10:07:07.646575+0000","last_became_peered":"2024-04-25T10:07:07.646575+0000","last_unstale":"2024-04-25T10:07:09.662031+0000","last_undegraded":"2024-04-25T10:07:09.662031+0000","last_fullsized":"2024-04-25T10:07:09.662031+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.598164+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.598164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:14.380632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":217,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.950894+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:07:09.950894+0000","last_peered":"2024-04-25T10:07:09.950894+0000","last_clean":"2024-04-25T10:07:09.950894+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:07:09.950894+0000","last_undegraded":"2024-04-25T10:07:09.950894+0000","last_fullsized":"2024-04-25T10:07:09.950894+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":813009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":557056,"data_stored":224163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":552960,"data_stored":223223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542172,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":548864,"data_stored":222621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542172,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":557056,"data_stored":223189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542172,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":561152,"data_stored":224185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35884,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335956,"statfs":{"total":96636764160,"available":96600018944,"internally_reserved":0,"allocated":548864,"data_stored":222621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36456,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335384,"statfs":{"total":96636764160,"available":96599433216,"internally_reserved":0,"allocated":1142784,"data_stored":812953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:16.517 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:07:16.517 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:07:16.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe89208f640 1 Processor -- start 2024-04-25T10:07:16.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe89208f640 1 -- start start 2024-04-25T10:07:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe89208f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe88c15bb50 0x7fe88c06d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe89208f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe88c06c330 0x7fe88c0b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe89208f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88c06e3c0 con 0x7fe88c06ffe0 2024-04-25T10:07:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe89208f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe88c06e660 con 0x7fe88c15bb50 2024-04-25T10:07:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe89208f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c06e940 con 0x7fe88c06c330 2024-04-25T10:07:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe88b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe88c15bb50 0x7fe88c06d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe88b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe88c15bb50 0x7fe88c06d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39180/0 (socket says 172.21.15.60:39180) 2024-04-25T10:07:16.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe88b7fe640 1 -- 172.21.15.60:0/3527717906 learned_addr learned my addr 172.21.15.60:0/3527717906 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:16.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe88affd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe88c06ffe0 0x7fe88c06bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60066/0 (socket says 172.21.15.60:60066) 2024-04-25T10:07:16.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe88bfff640 1 --2- 172.21.15.60:0/3527717906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe88c06c330 0x7fe88c0b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe88b7fe640 1 -- 172.21.15.60:0/3527717906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe88c06c330 msgr2=0x7fe88c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe88b7fe640 1 --2- 172.21.15.60:0/3527717906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe88c06c330 0x7fe88c0b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe88b7fe640 1 -- 172.21.15.60:0/3527717906 >> v1:172.21.15.60:6789/0 conn(0x7fe88c06ffe0 legacy=0x7fe88c06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe88b7fe640 1 -- 172.21.15.60:0/3527717906 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c0704f0 con 0x7fe88c15bb50 2024-04-25T10:07:16.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.687+0000 7fe88bfff640 1 --2- 172.21.15.60:0/3527717906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe88c06c330 0x7fe88c0b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:16.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe88b7fe640 1 --2- 172.21.15.60:0/3527717906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe88c15bb50 0x7fe88c06d910 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009870 tx=0x7fe87c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f25d76efe2fcd57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe88a7fc640 1 -- 172.21.15.60:0/3527717906 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c0a9640 con 0x7fe88c15bb50 2024-04-25T10:07:16.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe88a7fc640 1 -- 172.21.15.60:0/3527717906 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87c0a97e0 con 0x7fe88c15bb50 2024-04-25T10:07:16.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe88a7fc640 1 -- 172.21.15.60:0/3527717906 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c0a9bb0 con 0x7fe88c15bb50 2024-04-25T10:07:16.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 -- 172.21.15.60:0/3527717906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe88c15bb50 msgr2=0x7fe88c06d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 --2- 172.21.15.60:0/3527717906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe88c15bb50 0x7fe88c06d910 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009870 tx=0x7fe87c01c560 comp rx=0 tx=0).stop 2024-04-25T10:07:16.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 -- 172.21.15.60:0/3527717906 shutdown_connections 2024-04-25T10:07:16.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 --2- 172.21.15.60:0/3527717906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe88c06c330 0x7fe88c0b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 --2- 172.21.15.60:0/3527717906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe88c15bb50 0x7fe88c06d910 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 -- 172.21.15.60:0/3527717906 >> 172.21.15.60:0/3527717906 conn(0x7fe88c071b90 msgr2=0x7fe88c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 -- 172.21.15.60:0/3527717906 shutdown_connections 2024-04-25T10:07:16.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 -- 172.21.15.60:0/3527717906 wait complete. 2024-04-25T10:07:16.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 Processor -- start 2024-04-25T10:07:16.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 -- start start 2024-04-25T10:07:16.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe88c06c330 0x7fe88c157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe88c06ffe0 0x7fe88c152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe88c15bb50 0x7fe88c1510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c0b4b90 con 0x7fe88c06ffe0 2024-04-25T10:07:16.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c0cc600 con 0x7fe88c06c330 2024-04-25T10:07:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c0b4d10 con 0x7fe88c15bb50 2024-04-25T10:07:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe88affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe88c06ffe0 0x7fe88c152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe88affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe88c06ffe0 0x7fe88c152200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39186/0 (socket says 172.21.15.60:39186) 2024-04-25T10:07:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe88affd640 1 -- 172.21.15.60:0/2000328118 learned_addr learned my addr 172.21.15.60:0/2000328118 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe88bfff640 1 --2- 172.21.15.60:0/2000328118 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe88c15bb50 0x7fe88c1510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe88b7fe640 1 --2- 172.21.15.60:0/2000328118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe88c06c330 0x7fe88c157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe88affd640 1 -- 172.21.15.60:0/2000328118 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe88c15bb50 msgr2=0x7fe88c1510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe88affd640 1 --2- 172.21.15.60:0/2000328118 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe88c15bb50 0x7fe88c1510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe88affd640 1 -- 172.21.15.60:0/2000328118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe88c06c330 msgr2=0x7fe88c157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe88affd640 1 --2- 172.21.15.60:0/2000328118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe88c06c330 0x7fe88c157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe88affd640 1 -- 172.21.15.60:0/2000328118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c152b00 con 0x7fe88c06ffe0 2024-04-25T10:07:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe88affd640 1 --2- 172.21.15.60:0/2000328118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe88c06ffe0 0x7fe88c152200 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe88000c430 tx=0x7fe88000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe873fff640 1 -- 172.21.15.60:0/2000328118 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8800074e0 con 0x7fe88c06ffe0 2024-04-25T10:07:16.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe873fff640 1 -- 172.21.15.60:0/2000328118 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe880007680 con 0x7fe88c06ffe0 2024-04-25T10:07:16.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe89208f640 1 -- 172.21.15.60:0/2000328118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88c152de0 con 0x7fe88c06ffe0 2024-04-25T10:07:16.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.691+0000 7fe873fff640 1 -- 172.21.15.60:0/2000328118 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880004a30 con 0x7fe88c06ffe0 2024-04-25T10:07:16.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.695+0000 7fe89208f640 1 -- 172.21.15.60:0/2000328118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88c1533a0 con 0x7fe88c06ffe0 2024-04-25T10:07:16.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.695+0000 7fe89208f640 1 -- 172.21.15.60:0/2000328118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88c074770 con 0x7fe88c06ffe0 2024-04-25T10:07:16.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.695+0000 7fe873fff640 1 -- 172.21.15.60:0/2000328118 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe88001a020 con 0x7fe88c06ffe0 2024-04-25T10:07:16.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.695+0000 7fe873fff640 1 --2- 172.21.15.60:0/2000328118 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe878066f60 0x7fe878069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.695+0000 7fe873fff640 1 -- 172.21.15.60:0/2000328118 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8800847d0 con 0x7fe88c06ffe0 2024-04-25T10:07:16.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.695+0000 7fe88b7fe640 1 --2- 172.21.15.60:0/2000328118 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe878066f60 0x7fe878069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.695+0000 7fe88b7fe640 1 --2- 172.21.15.60:0/2000328118 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe878066f60 0x7fe878069420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe87c0ae000 tx=0x7fe87c0b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.699+0000 7fe873fff640 1 -- 172.21.15.60:0/2000328118 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8800557e0 con 0x7fe88c06ffe0 2024-04-25T10:07:16.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.919+0000 7fe89208f640 1 -- 172.21.15.60:0/2000328118 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe88c0af540 con 0x7fe88c06ffe0 2024-04-25T10:07:16.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.923+0000 7fe873fff640 1 -- 172.21.15.60:0/2000328118 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe8800550e0 con 0x7fe88c06ffe0 2024-04-25T10:07:16.929 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T10:07:16.929 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCiKipmv65dLxAA6/QpPrqE9aT+CtZqkyutog== 2024-04-25T10:07:16.929 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-04-25T10:07:16.930 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-04-25T10:07:16.930 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:07:16.930 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-04-25T10:07:16.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.923+0000 7fe89208f640 1 -- 172.21.15.60:0/2000328118 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe878066f60 msgr2=0x7fe878069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.923+0000 7fe89208f640 1 --2- 172.21.15.60:0/2000328118 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe878066f60 0x7fe878069420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe87c0ae000 tx=0x7fe87c0b3040 comp rx=0 tx=0).stop 2024-04-25T10:07:16.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.923+0000 7fe89208f640 1 -- 172.21.15.60:0/2000328118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe88c06ffe0 msgr2=0x7fe88c152200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.923+0000 7fe89208f640 1 --2- 172.21.15.60:0/2000328118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe88c06ffe0 0x7fe88c152200 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe88000c430 tx=0x7fe88000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.923+0000 7fe89208f640 1 -- 172.21.15.60:0/2000328118 shutdown_connections 2024-04-25T10:07:16.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.923+0000 7fe89208f640 1 --2- 172.21.15.60:0/2000328118 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe878066f60 0x7fe878069420 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.923+0000 7fe89208f640 1 --2- 172.21.15.60:0/2000328118 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe88c15bb50 0x7fe88c1510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.923+0000 7fe89208f640 1 --2- 172.21.15.60:0/2000328118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe88c06ffe0 0x7fe88c152200 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.923+0000 7fe89208f640 1 --2- 172.21.15.60:0/2000328118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe88c06c330 0x7fe88c157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.923+0000 7fe89208f640 1 -- 172.21.15.60:0/2000328118 >> 172.21.15.60:0/2000328118 conn(0x7fe88c071b90 msgr2=0x7fe88c071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.923+0000 7fe89208f640 1 -- 172.21.15.60:0/2000328118 shutdown_connections 2024-04-25T10:07:16.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:16.923+0000 7fe89208f640 1 -- 172.21.15.60:0/2000328118 wait complete. 2024-04-25T10:07:16.948 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:17.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.059+0000 7f9737ae0640 1 Processor -- start 2024-04-25T10:07:17.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.059+0000 7f9737ae0640 1 -- start start 2024-04-25T10:07:17.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.063+0000 7f9737ae0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97300ce7a0 0x7f97300ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.063+0000 7f9737ae0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f97300b4660 0x7f97300b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.063+0000 7f9737ae0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97300cf0c0 con 0x7f973015bc90 2024-04-25T10:07:17.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.063+0000 7f9737ae0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97300cc370 con 0x7f97300b4660 2024-04-25T10:07:17.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.063+0000 7f9737ae0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9730071ca0 con 0x7f97300ce7a0 2024-04-25T10:07:17.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.063+0000 7f973587c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97300ce7a0 0x7f97300ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.063+0000 7f973607d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f97300b4660 0x7f97300b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.063+0000 7f973507b640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f973015bc90 0x7f97300b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:35324/0 (socket says 172.21.15.60:35324) 2024-04-25T10:07:17.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.063+0000 7f973607d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f97300b4660 0x7f97300b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:41852/0 (socket says 172.21.15.60:41852) 2024-04-25T10:07:17.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.063+0000 7f973507b640 1 -- 172.21.15.60:0/2388992242 learned_addr learned my addr 172.21.15.60:0/2388992242 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:17.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.063+0000 7f973607d640 1 -- 172.21.15.60:0/2388992242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97300ce7a0 msgr2=0x7f97300ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.063+0000 7f973607d640 1 --2- 172.21.15.60:0/2388992242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97300ce7a0 0x7f97300ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.063+0000 7f973607d640 1 -- 172.21.15.60:0/2388992242 >> v1:172.21.15.100:6789/0 conn(0x7f973015bc90 legacy=0x7f97300b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.063+0000 7f973607d640 1 -- 172.21.15.60:0/2388992242 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97300cca40 con 0x7f97300b4660 2024-04-25T10:07:17.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.067+0000 7f973607d640 1 --2- 172.21.15.60:0/2388992242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97300b4660 0x7f97300b2ce0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9728009dc0 tx=0x7f972801c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44d1fe0ad9b41bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.067+0000 7f973487a640 1 -- 172.21.15.60:0/2388992242 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97280b2070 con 0x7f97300b4660 2024-04-25T10:07:17.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.067+0000 7f973487a640 1 -- 172.21.15.60:0/2388992242 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97280a5810 con 0x7f97300b4660 2024-04-25T10:07:17.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.067+0000 7f973487a640 1 -- 172.21.15.60:0/2388992242 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97280a5be0 con 0x7f97300b4660 2024-04-25T10:07:17.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.067+0000 7f9737ae0640 1 -- 172.21.15.60:0/2388992242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97300b4660 msgr2=0x7f97300b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.067+0000 7f9737ae0640 1 --2- 172.21.15.60:0/2388992242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97300b4660 0x7f97300b2ce0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9728009dc0 tx=0x7f972801c950 comp rx=0 tx=0).stop 2024-04-25T10:07:17.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.067+0000 7f9737ae0640 1 -- 172.21.15.60:0/2388992242 shutdown_connections 2024-04-25T10:07:17.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.067+0000 7f9737ae0640 1 --2- 172.21.15.60:0/2388992242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97300b4660 0x7f97300b2ce0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.067+0000 7f9737ae0640 1 --2- 172.21.15.60:0/2388992242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97300ce7a0 0x7f97300ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.067+0000 7f9737ae0640 1 -- 172.21.15.60:0/2388992242 >> 172.21.15.60:0/2388992242 conn(0x7f97300b12d0 msgr2=0x7f97300b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.067+0000 7f9737ae0640 1 -- 172.21.15.60:0/2388992242 shutdown_connections 2024-04-25T10:07:17.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.067+0000 7f9737ae0640 1 -- 172.21.15.60:0/2388992242 wait complete. 2024-04-25T10:07:17.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f9737ae0640 1 Processor -- start 2024-04-25T10:07:17.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f9737ae0640 1 -- start start 2024-04-25T10:07:17.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f9737ae0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97300ce7a0 0x7f973015d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f9737ae0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973015bc90 0x7f9730156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f9737ae0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9730156b40 0x7f97301f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f9737ae0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97300b4280 con 0x7f97300ce7a0 2024-04-25T10:07:17.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f9737ae0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97300b3db0 con 0x7f973015bc90 2024-04-25T10:07:17.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f9737ae0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97300b4400 con 0x7f9730156b40 2024-04-25T10:07:17.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f973587c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97300ce7a0 0x7f973015d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f973587c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97300ce7a0 0x7f973015d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39202/0 (socket says 172.21.15.60:39202) 2024-04-25T10:07:17.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f973587c640 1 -- 172.21.15.60:0/486693137 learned_addr learned my addr 172.21.15.60:0/486693137 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:17.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f973507b640 1 --2- 172.21.15.60:0/486693137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973015bc90 0x7f9730156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f973587c640 1 -- 172.21.15.60:0/486693137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9730156b40 msgr2=0x7f97301f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f973607d640 1 --2- 172.21.15.60:0/486693137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9730156b40 0x7f97301f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f973587c640 1 --2- 172.21.15.60:0/486693137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9730156b40 0x7f97301f4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f973587c640 1 -- 172.21.15.60:0/486693137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973015bc90 msgr2=0x7f9730156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f973587c640 1 --2- 172.21.15.60:0/486693137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973015bc90 0x7f9730156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f973587c640 1 -- 172.21.15.60:0/486693137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9730157000 con 0x7f97300ce7a0 2024-04-25T10:07:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f973507b640 1 --2- 172.21.15.60:0/486693137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973015bc90 0x7f9730156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f973587c640 1 --2- 172.21.15.60:0/486693137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97300ce7a0 0x7f973015d930 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f972c00e870 tx=0x7f972c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f97267fc640 1 -- 172.21.15.60:0/486693137 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c017070 con 0x7f97300ce7a0 2024-04-25T10:07:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f97267fc640 1 -- 172.21.15.60:0/486693137 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972c00c8e0 con 0x7f97300ce7a0 2024-04-25T10:07:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f9737ae0640 1 -- 172.21.15.60:0/486693137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97301572e0 con 0x7f97300ce7a0 2024-04-25T10:07:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f97267fc640 1 -- 172.21.15.60:0/486693137 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c00cc90 con 0x7f97300ce7a0 2024-04-25T10:07:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f9737ae0640 1 -- 172.21.15.60:0/486693137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97301596c0 con 0x7f97300ce7a0 2024-04-25T10:07:17.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.071+0000 7f9737ae0640 1 -- 172.21.15.60:0/486693137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97301591f0 con 0x7f97300ce7a0 2024-04-25T10:07:17.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.075+0000 7f97267fc640 1 -- 172.21.15.60:0/486693137 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f972c01a070 con 0x7f97300ce7a0 2024-04-25T10:07:17.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.075+0000 7f97267fc640 1 --2- 172.21.15.60:0/486693137 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9714066f60 0x7f9714069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.075+0000 7f97267fc640 1 -- 172.21.15.60:0/486693137 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f972c085680 con 0x7f97300ce7a0 2024-04-25T10:07:17.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.075+0000 7f973507b640 1 --2- 172.21.15.60:0/486693137 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9714066f60 0x7f9714069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.075+0000 7f973507b640 1 --2- 172.21.15.60:0/486693137 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9714066f60 0x7f9714069420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9720007fd0 tx=0x7f9720009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.079+0000 7f97267fc640 1 -- 172.21.15.60:0/486693137 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f972c056700 con 0x7f97300ce7a0 2024-04-25T10:07:17.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.263+0000 7f9737ae0640 1 -- 172.21.15.60:0/486693137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9730074770 con 0x7f97300ce7a0 2024-04-25T10:07:17.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.263+0000 7f97267fc640 1 -- 172.21.15.60:0/486693137 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f972c056000 con 0x7f97300ce7a0 2024-04-25T10:07:17.272 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:17.272 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1138162580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1138162580},{"type":"v1","addr":"172.21.15.100:6837","nonce":1138162580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2985559168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2985559168},{"type":"v1","addr":"172.21.15.100:6835","nonce":2985559168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/50795370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":50795370},{"type":"v1","addr":"172.21.15.60:6834","nonce":50795370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:07.980095+0000","modified":"2024-04-25T10:07:10.002670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5796":{"gid":5796,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1747757196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1747757196},{"type":"v1","addr":"172.21.15.60:6835","nonce":1747757196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:17.273 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 21 2024-04-25T10:07:17.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.267+0000 7f9737ae0640 1 -- 172.21.15.60:0/486693137 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9714066f60 msgr2=0x7f9714069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.267+0000 7f9737ae0640 1 --2- 172.21.15.60:0/486693137 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9714066f60 0x7f9714069420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9720007fd0 tx=0x7f9720009040 comp rx=0 tx=0).stop 2024-04-25T10:07:17.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.267+0000 7f9737ae0640 1 -- 172.21.15.60:0/486693137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97300ce7a0 msgr2=0x7f973015d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.267+0000 7f9737ae0640 1 --2- 172.21.15.60:0/486693137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97300ce7a0 0x7f973015d930 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f972c00e870 tx=0x7f972c00ec70 comp rx=0 tx=0).stop 2024-04-25T10:07:17.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.267+0000 7f9737ae0640 1 -- 172.21.15.60:0/486693137 shutdown_connections 2024-04-25T10:07:17.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.267+0000 7f9737ae0640 1 --2- 172.21.15.60:0/486693137 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9714066f60 0x7f9714069420 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.267+0000 7f9737ae0640 1 --2- 172.21.15.60:0/486693137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9730156b40 0x7f97301f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.267+0000 7f9737ae0640 1 --2- 172.21.15.60:0/486693137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973015bc90 0x7f9730156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.267+0000 7f9737ae0640 1 --2- 172.21.15.60:0/486693137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97300ce7a0 0x7f973015d930 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.267+0000 7f9737ae0640 1 -- 172.21.15.60:0/486693137 >> 172.21.15.60:0/486693137 conn(0x7f97300b12d0 msgr2=0x7f973015c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.267+0000 7f9737ae0640 1 -- 172.21.15.60:0/486693137 shutdown_connections 2024-04-25T10:07:17.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.267+0000 7f9737ae0640 1 -- 172.21.15.60:0/486693137 wait complete. 2024-04-25T10:07:17.693 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:07:07.980095+0000', 'modified': '2024-04-25T10:07:10.002670+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5796}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5796': {'gid': 5796, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/1747757196', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 1747757196}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 1747757196}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:17.693 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:07:17.693 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:07:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f7cbf640 1 Processor -- start 2024-04-25T10:07:17.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f7cbf640 1 -- start start 2024-04-25T10:07:17.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f7cbf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f00ce7a0 0x7f63f00ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f7cbf640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f00b33f0 con 0x7f63f015bc90 2024-04-25T10:07:17.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f7cbf640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f00cf0c0 con 0x7f63f00b4660 2024-04-25T10:07:17.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f7cbf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00cc370 con 0x7f63f00ce7a0 2024-04-25T10:07:17.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f525a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f63f015bc90 0x7f63f00b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60074/0 (socket says 172.21.15.60:60074) 2024-04-25T10:07:17.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f525a640 1 -- 172.21.15.60:0/1226404919 learned_addr learned my addr 172.21.15.60:0/1226404919 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:17.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f5a5b640 1 --2- 172.21.15.60:0/1226404919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f00ce7a0 0x7f63f00ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f4a59640 1 -- 172.21.15.60:0/1226404919 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026402794 0 0) 0x7f63f00b33f0 con 0x7f63f015bc90 2024-04-25T10:07:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f4a59640 1 -- 172.21.15.60:0/1226404919 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63dc003530 con 0x7f63f015bc90 2024-04-25T10:07:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f4a59640 1 -- 172.21.15.60:0/1226404919 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1818916964 0 0) 0x7f63dc003530 con 0x7f63f015bc90 2024-04-25T10:07:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f4a59640 1 -- 172.21.15.60:0/1226404919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f00ce7a0 msgr2=0x7f63f00ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f4a59640 1 --2- 172.21.15.60:0/1226404919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f00ce7a0 0x7f63f00ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f4a59640 1 -- 172.21.15.60:0/1226404919 >> v1:172.21.15.100:6789/0 conn(0x7f63f00b4660 legacy=0x7f63f00b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f4a59640 1 -- 172.21.15.60:0/1226404919 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f0071ca0 con 0x7f63f015bc90 2024-04-25T10:07:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f4a59640 1 -- 172.21.15.60:0/1226404919 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f63ec001680 con 0x7f63f015bc90 2024-04-25T10:07:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f4a59640 1 -- 172.21.15.60:0/1226404919 >> v1:172.21.15.60:6789/0 conn(0x7f63f015bc90 legacy=0x7f63f00b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f4a59640 1 --2- 172.21.15.60:0/1226404919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f00ce7a0 0x7f63dc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f4a59640 1 --2- 172.21.15.60:0/1226404919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc003f70 0x7f63dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f4a59640 1 --2- 172.21.15.60:0/1226404919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc004d50 0x7f63dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f4a59640 1 -- 172.21.15.60:0/1226404919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc00ba80 con 0x7f63dc003f70 2024-04-25T10:07:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f4a59640 1 -- 172.21.15.60:0/1226404919 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc00bc30 con 0x7f63dc004d50 2024-04-25T10:07:17.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f4a59640 1 -- 172.21.15.60:0/1226404919 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc00be30 con 0x7f63f00ce7a0 2024-04-25T10:07:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f525a640 1 --2- 172.21.15.60:0/1226404919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc003f70 0x7f63dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f625c640 1 --2- 172.21.15.60:0/1226404919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc004d50 0x7f63dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f5a5b640 1 --2- 172.21.15.60:0/1226404919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f00ce7a0 0x7f63dc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f525a640 1 -- 172.21.15.60:0/1226404919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f00ce7a0 msgr2=0x7f63dc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f525a640 1 --2- 172.21.15.60:0/1226404919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f00ce7a0 0x7f63dc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f525a640 1 -- 172.21.15.60:0/1226404919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc004d50 msgr2=0x7f63dc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f525a640 1 --2- 172.21.15.60:0/1226404919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc004d50 0x7f63dc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f525a640 1 -- 172.21.15.60:0/1226404919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63dc003530 con 0x7f63dc003f70 2024-04-25T10:07:17.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.822+0000 7f63f625c640 1 --2- 172.21.15.60:0/1226404919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc004d50 0x7f63dc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:17.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.826+0000 7f63f525a640 1 --2- 172.21.15.60:0/1226404919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc003f70 0x7f63dc004810 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00c240 tx=0x7f63ec01f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03ec2288f819b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.826+0000 7f63f4a59640 1 -- 172.21.15.60:0/1226404919 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec007d20 con 0x7f63dc003f70 2024-04-25T10:07:17.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.826+0000 7f63f4a59640 1 -- 172.21.15.60:0/1226404919 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ec005ae0 con 0x7f63dc003f70 2024-04-25T10:07:17.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.826+0000 7f63f4a59640 1 -- 172.21.15.60:0/1226404919 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec007070 con 0x7f63dc003f70 2024-04-25T10:07:17.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.826+0000 7f63f7cbf640 1 -- 172.21.15.60:0/1226404919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc003f70 msgr2=0x7f63dc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.826+0000 7f63f7cbf640 1 --2- 172.21.15.60:0/1226404919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc003f70 0x7f63dc004810 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00c240 tx=0x7f63ec01f3f0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.826+0000 7f63f625c640 1 -- 172.21.15.60:0/1226404919 reap_dead start 2024-04-25T10:07:17.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.826+0000 7f63f7cbf640 1 -- 172.21.15.60:0/1226404919 shutdown_connections 2024-04-25T10:07:17.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.826+0000 7f63f7cbf640 1 -- 172.21.15.60:0/1226404919 >> 172.21.15.60:0/1226404919 conn(0x7f63f00b12d0 msgr2=0x7f63f00b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.826+0000 7f63f7cbf640 1 -- 172.21.15.60:0/1226404919 shutdown_connections 2024-04-25T10:07:17.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.826+0000 7f63f7cbf640 1 -- 172.21.15.60:0/1226404919 wait complete. 2024-04-25T10:07:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.826+0000 7f63f7cbf640 1 Processor -- start 2024-04-25T10:07:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.826+0000 7f63f7cbf640 1 -- start start 2024-04-25T10:07:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f7cbf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63dc003f70 0x7f63f0143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f7cbf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63f0154080 0x7f63f01544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f7cbf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63f0156840 0x7f63f0156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f7cbf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f0071f80 con 0x7f63f0156840 2024-04-25T10:07:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f7cbf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00ccd20 con 0x7f63f0154080 2024-04-25T10:07:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f7cbf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00b3ba0 con 0x7f63dc003f70 2024-04-25T10:07:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f625c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63f0156840 0x7f63f0156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f625c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63f0156840 0x7f63f0156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39228/0 (socket says 172.21.15.60:39228) 2024-04-25T10:07:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f625c640 1 -- 172.21.15.60:0/969283888 learned_addr learned my addr 172.21.15.60:0/969283888 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f5a5b640 1 --2- 172.21.15.60:0/969283888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63dc003f70 0x7f63f0143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f525a640 1 --2- 172.21.15.60:0/969283888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63f0154080 0x7f63f01544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f625c640 1 -- 172.21.15.60:0/969283888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63dc003f70 msgr2=0x7f63f0143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f625c640 1 --2- 172.21.15.60:0/969283888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63dc003f70 0x7f63f0143e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f625c640 1 -- 172.21.15.60:0/969283888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63f0154080 msgr2=0x7f63f01544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f625c640 1 --2- 172.21.15.60:0/969283888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63f0154080 0x7f63f01544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f625c640 1 -- 172.21.15.60:0/969283888 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f0152530 con 0x7f63f0156840 2024-04-25T10:07:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f625c640 1 --2- 172.21.15.60:0/969283888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63f0156840 0x7f63f0156ca0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f63f0072c10 tx=0x7f63e800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63e27fc640 1 -- 172.21.15.60:0/969283888 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8006b60 con 0x7f63f0156840 2024-04-25T10:07:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63e27fc640 1 -- 172.21.15.60:0/969283888 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e8006d20 con 0x7f63f0156840 2024-04-25T10:07:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f7cbf640 1 -- 172.21.15.60:0/969283888 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f0152810 con 0x7f63f0156840 2024-04-25T10:07:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63e27fc640 1 -- 172.21.15.60:0/969283888 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8003980 con 0x7f63f0156840 2024-04-25T10:07:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f7cbf640 1 -- 172.21.15.60:0/969283888 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f0152dd0 con 0x7f63f0156840 2024-04-25T10:07:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63e27fc640 1 -- 172.21.15.60:0/969283888 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63e801a020 con 0x7f63f0156840 2024-04-25T10:07:17.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63e27fc640 1 --2- 172.21.15.60:0/969283888 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63d4066f60 0x7f63d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63e27fc640 1 -- 172.21.15.60:0/969283888 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63e8084660 con 0x7f63f0156840 2024-04-25T10:07:17.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.830+0000 7f63f5a5b640 1 --2- 172.21.15.60:0/969283888 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63d4066f60 0x7f63d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 Processor -- start 2024-04-25T10:07:17.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 -- start start 2024-04-25T10:07:17.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63f5a5b640 1 --2- 172.21.15.60:0/969283888 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63d4066f60 0x7f63d4069420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009a30 tx=0x7f63e4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89ba920 0x55b5a89bad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a89bb240 0x55b5a89709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a8970f30 0x55b5a8971310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a88585f0 con 0x55b5a8970f30 2024-04-25T10:07:17.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a87bfda0 con 0x55b5a89ba920 2024-04-25T10:07:17.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5a8748150 con 0x55b5a89bb240 2024-04-25T10:07:17.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63f625c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a8970f30 0x55b5a8971310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63f625c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a8970f30 0x55b5a8971310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39236/0 (socket says 172.21.15.60:39236) 2024-04-25T10:07:17.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63f625c640 1 -- 172.21.15.60:0/3050644674 learned_addr learned my addr 172.21.15.60:0/3050644674 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:17.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63f525a640 1 --2- 172.21.15.60:0/3050644674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a89bb240 0x55b5a89709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63f5a5b640 1 --2- 172.21.15.60:0/3050644674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89ba920 0x55b5a89bad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63f625c640 1 -- 172.21.15.60:0/3050644674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a89bb240 msgr2=0x55b5a89709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63f625c640 1 --2- 172.21.15.60:0/3050644674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a89bb240 0x55b5a89709f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63f625c640 1 -- 172.21.15.60:0/3050644674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89ba920 msgr2=0x55b5a89bad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63f625c640 1 --2- 172.21.15.60:0/3050644674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89ba920 0x55b5a89bad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63f625c640 1 -- 172.21.15.60:0/3050644674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5a8761ca0 con 0x55b5a8970f30 2024-04-25T10:07:17.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63f625c640 1 --2- 172.21.15.60:0/3050644674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a8970f30 0x55b5a8971310 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f63e8082bb0 tx=0x7f63e8054be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13c65358342548b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63d37fe640 1 -- 172.21.15.60:0/3050644674 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e805d680 con 0x55b5a8970f30 2024-04-25T10:07:17.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63d37fe640 1 -- 172.21.15.60:0/3050644674 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e8053270 con 0x55b5a8970f30 2024-04-25T10:07:17.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63d37fe640 1 -- 172.21.15.60:0/3050644674 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e805d1f0 con 0x55b5a8970f30 2024-04-25T10:07:17.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 -- 172.21.15.60:0/3050644674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a8970f30 msgr2=0x55b5a8971310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 --2- 172.21.15.60:0/3050644674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a8970f30 0x55b5a8971310 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f63e8082bb0 tx=0x7f63e8054be0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 -- 172.21.15.60:0/3050644674 shutdown_connections 2024-04-25T10:07:17.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 --2- 172.21.15.60:0/3050644674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a8970f30 0x55b5a8971310 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 --2- 172.21.15.60:0/3050644674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a89bb240 0x55b5a89709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 --2- 172.21.15.60:0/3050644674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89ba920 0x55b5a89bad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 -- 172.21.15.60:0/3050644674 >> 172.21.15.60:0/3050644674 conn(0x55b5a89772a0 msgr2=0x55b5a896f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 -- 172.21.15.60:0/3050644674 shutdown_connections 2024-04-25T10:07:17.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 -- 172.21.15.60:0/3050644674 wait complete. 2024-04-25T10:07:17.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 -- start start 2024-04-25T10:07:17.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.834+0000 7f63fae2c000 1 Processor -- start 2024-04-25T10:07:17.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a8970f30 0x55b5a89c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a89ba920 0x55b5a89c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89bb240 0x55b5a8974aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a877a990 con 0x55b5a89ba920 2024-04-25T10:07:17.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a87bbe40 con 0x55b5a89bb240 2024-04-25T10:07:17.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5a874b900 con 0x55b5a8970f30 2024-04-25T10:07:17.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 10 client.0 _ll_register_callbacks cb 0x55b5a88731c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:17.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63f525a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a89ba920 0x55b5a89c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63f525a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a89ba920 0x55b5a89c21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39244/0 (socket says 172.21.15.60:39244) 2024-04-25T10:07:17.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63f525a640 1 -- 172.21.15.60:0/3293200843 learned_addr learned my addr 172.21.15.60:0/3293200843 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:17.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63f625c640 1 --2- 172.21.15.60:0/3293200843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89bb240 0x55b5a8974aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63f5a5b640 1 --2- 172.21.15.60:0/3293200843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a8970f30 0x55b5a89c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63f525a640 1 -- 172.21.15.60:0/3293200843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a8970f30 msgr2=0x55b5a89c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63f525a640 1 --2- 172.21.15.60:0/3293200843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a8970f30 0x55b5a89c1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63f525a640 1 -- 172.21.15.60:0/3293200843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89bb240 msgr2=0x55b5a8974aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63f525a640 1 --2- 172.21.15.60:0/3293200843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89bb240 0x55b5a8974aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63f525a640 1 -- 172.21.15.60:0/3293200843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5a8778630 con 0x55b5a89ba920 2024-04-25T10:07:17.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63f5a5b640 1 --2- 172.21.15.60:0/3293200843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a8970f30 0x55b5a89c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:17.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63f525a640 1 --2- 172.21.15.60:0/3293200843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a89ba920 0x55b5a89c21c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00c240 tx=0x7f63ec01c190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8c412203e1fb16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:07:17.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63d0ff9640 1 -- 172.21.15.60:0/3293200843 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec001480 con 0x55b5a89ba920 2024-04-25T10:07:17.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63d0ff9640 1 -- 172.21.15.60:0/3293200843 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63ec001620 con 0x55b5a89ba920 2024-04-25T10:07:17.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 -- 172.21.15.60:0/3293200843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63e8053270 con 0x55b5a89ba920 2024-04-25T10:07:17.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63d0ff9640 1 -- 172.21.15.60:0/3293200843 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63ec0047f0 con 0x55b5a89ba920 2024-04-25T10:07:17.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 10 client.5976 fetch_fsmap learned FSMap version 21 2024-04-25T10:07:17.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 -- 172.21.15.60:0/3293200843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55b5a87483a0 con 0x55b5a89ba920 2024-04-25T10:07:17.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63d0ff9640 1 -- 172.21.15.60:0/3293200843 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e8053270 con 0x55b5a89ba920 2024-04-25T10:07:17.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63d0ff9640 1 -- 172.21.15.60:0/3293200843 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f63ec0c5050 con 0x55b5a89ba920 2024-04-25T10:07:17.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 10 client.5976 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T10:07:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 10 client.5976 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:07:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 20 client.5976 populate_metadata read hostname 'smithi060' 2024-04-25T10:07:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 --2- 172.21.15.60:0/3293200843 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x55b5a8976160 0x55b5a89cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 4 client.5976 mds_command: new command op to 5796 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 -- 172.21.15.60:0/3293200843 --> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5a875ba50 con 0x55b5a8976160 2024-04-25T10:07:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63f5a5b640 1 --2- 172.21.15.60:0/3293200843 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x55b5a8976160 0x55b5a89cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63d0ff9640 10 client.5976 ms_handle_connect on v2:172.21.15.60:6833/1747757196 2024-04-25T10:07:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63f5a5b640 1 --2- 172.21.15.60:0/3293200843 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x55b5a8976160 0x55b5a89cbc70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=671b283e33730ded server_cookie=8a7840aa8fdc4a5d in_seq=0 out_seq=0 2024-04-25T10:07:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63d0ff9640 1 -- 172.21.15.60:0/3293200843 <== mds.0 v2:172.21.15.60:6833/1747757196 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b5a875ba50 con 0x55b5a8976160 2024-04-25T10:07:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63d0ff9640 10 client.5976 handle_command_reply: tid=0 2024-04-25T10:07:17.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 client.5976 shutdown 2024-04-25T10:07:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 -- 172.21.15.60:0/3293200843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a89ba920 msgr2=0x55b5a89c21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 --2- 172.21.15.60:0/3293200843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a89ba920 0x55b5a89c21c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00c240 tx=0x7f63ec01c190 comp rx=0 tx=0).stop 2024-04-25T10:07:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 -- 172.21.15.60:0/3293200843 shutdown_connections 2024-04-25T10:07:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 --2- 172.21.15.60:0/3293200843 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x55b5a8976160 0x55b5a89cbc70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 --2- 172.21.15.60:0/3293200843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89bb240 0x55b5a8974aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 --2- 172.21.15.60:0/3293200843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a89ba920 0x55b5a89c21c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 --2- 172.21.15.60:0/3293200843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a8970f30 0x55b5a89c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63d0ff9640 0 client.5976 ms_handle_reset on v2:172.21.15.60:6833/1747757196 2024-04-25T10:07:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 -- 172.21.15.60:0/3293200843 >> 172.21.15.60:0/3293200843 conn(0x55b5a89772a0 msgr2=0x55b5a896f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 -- 172.21.15.60:0/3293200843 shutdown_connections 2024-04-25T10:07:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 1 -- 172.21.15.60:0/3293200843 wait complete. 2024-04-25T10:07:17.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.838+0000 7f63fae2c000 20 client.5976 trim_cache size 0 max 16384 2024-04-25T10:07:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63fae2c000 1 Processor -- start 2024-04-25T10:07:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63fae2c000 1 -- start start 2024-04-25T10:07:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63fae2c000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a8970f30 0x55b5a8974330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63fae2c000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a89ba920 0x55b5a8974870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63fae2c000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89bb240 0x55b5a89c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63fae2c000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f01fd1e0 con 0x55b5a8970f30 2024-04-25T10:07:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63fae2c000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f01458f0 con 0x55b5a89bb240 2024-04-25T10:07:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63fae2c000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f01fcbe0 con 0x55b5a89ba920 2024-04-25T10:07:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63f5a5b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a8970f30 0x55b5a8974330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63f5a5b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a8970f30 0x55b5a8974330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39256/0 (socket says 172.21.15.60:39256) 2024-04-25T10:07:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63f5a5b640 1 -- 172.21.15.60:0/2014951437 learned_addr learned my addr 172.21.15.60:0/2014951437 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63f625c640 1 --2- 172.21.15.60:0/2014951437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89bb240 0x55b5a89c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63f5a5b640 1 -- 172.21.15.60:0/2014951437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a89ba920 msgr2=0x55b5a8974870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63f5a5b640 1 --2- 172.21.15.60:0/2014951437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a89ba920 0x55b5a8974870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63f5a5b640 1 -- 172.21.15.60:0/2014951437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89bb240 msgr2=0x55b5a89c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63f5a5b640 1 --2- 172.21.15.60:0/2014951437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89bb240 0x55b5a89c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63f5a5b640 1 -- 172.21.15.60:0/2014951437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5a89bb070 con 0x55b5a8970f30 2024-04-25T10:07:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63f625c640 1 --2- 172.21.15.60:0/2014951437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89bb240 0x55b5a89c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.854+0000 7f63f5a5b640 1 --2- 172.21.15.60:0/2014951437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a8970f30 0x55b5a8974330 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f63e4013c90 tx=0x7f63e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4394d5b13260a415 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63d2ffd640 1 -- 172.21.15.60:0/2014951437 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4016c70 con 0x55b5a8970f30 2024-04-25T10:07:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63d2ffd640 1 -- 172.21.15.60:0/2014951437 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e401c040 con 0x55b5a8970f30 2024-04-25T10:07:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63d2ffd640 1 -- 172.21.15.60:0/2014951437 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4004220 con 0x55b5a8970f30 2024-04-25T10:07:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 -- 172.21.15.60:0/2014951437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a8970f30 msgr2=0x55b5a8974330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 --2- 172.21.15.60:0/2014951437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a8970f30 0x55b5a8974330 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f63e4013c90 tx=0x7f63e4016000 comp rx=0 tx=0).stop 2024-04-25T10:07:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 -- 172.21.15.60:0/2014951437 shutdown_connections 2024-04-25T10:07:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 --2- 172.21.15.60:0/2014951437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89bb240 0x55b5a89c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 --2- 172.21.15.60:0/2014951437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a89ba920 0x55b5a8974870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 --2- 172.21.15.60:0/2014951437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a8970f30 0x55b5a8974330 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 -- 172.21.15.60:0/2014951437 >> 172.21.15.60:0/2014951437 conn(0x55b5a89772a0 msgr2=0x55b5a89a7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 -- 172.21.15.60:0/2014951437 shutdown_connections 2024-04-25T10:07:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 -- 172.21.15.60:0/2014951437 wait complete. 2024-04-25T10:07:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 -- start start 2024-04-25T10:07:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 Processor -- start 2024-04-25T10:07:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a8970f30 0x55b5a89c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89ba920 0x55b5a89ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a89bb240 0x55b5a89ff700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a87c43d0 con 0x55b5a89bb240 2024-04-25T10:07:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5a887bd50 con 0x55b5a89ba920 2024-04-25T10:07:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5a875a980 con 0x55b5a8970f30 2024-04-25T10:07:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 10 client.0 _ll_register_callbacks cb 0x7f63e8051210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63f625c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a89bb240 0x55b5a89ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63f625c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a89bb240 0x55b5a89ff700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39270/0 (socket says 172.21.15.60:39270) 2024-04-25T10:07:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63f625c640 1 -- 172.21.15.60:0/1939947559 learned_addr learned my addr 172.21.15.60:0/1939947559 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63f5a5b640 1 --2- 172.21.15.60:0/1939947559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a8970f30 0x55b5a89c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63f525a640 1 --2- 172.21.15.60:0/1939947559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89ba920 0x55b5a89ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63f625c640 1 -- 172.21.15.60:0/1939947559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a8970f30 msgr2=0x55b5a89c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63f625c640 1 --2- 172.21.15.60:0/1939947559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a8970f30 0x55b5a89c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63f625c640 1 -- 172.21.15.60:0/1939947559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89ba920 msgr2=0x55b5a89ff1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63f625c640 1 --2- 172.21.15.60:0/1939947559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89ba920 0x55b5a89ff1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63f625c640 1 -- 172.21.15.60:0/1939947559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5a8841600 con 0x55b5a89bb240 2024-04-25T10:07:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63f525a640 1 --2- 172.21.15.60:0/1939947559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89ba920 0x55b5a89ff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63f625c640 1 --2- 172.21.15.60:0/1939947559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a89bb240 0x55b5a89ff700 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f63e805d150 tx=0x7f63e8001370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3208e47b037db32d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:07:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63d0ff9640 1 -- 172.21.15.60:0/1939947559 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e805d1f0 con 0x55b5a89bb240 2024-04-25T10:07:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63d0ff9640 1 -- 172.21.15.60:0/1939947559 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63e805d390 con 0x55b5a89bb240 2024-04-25T10:07:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 -- 172.21.15.60:0/1939947559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63e401c040 con 0x55b5a89bb240 2024-04-25T10:07:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63d0ff9640 1 -- 172.21.15.60:0/1939947559 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63e80543c0 con 0x55b5a89bb240 2024-04-25T10:07:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 10 client.5982 fetch_fsmap learned FSMap version 21 2024-04-25T10:07:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 -- 172.21.15.60:0/1939947559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55b5a87549b0 con 0x55b5a89bb240 2024-04-25T10:07:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63d0ff9640 1 -- 172.21.15.60:0/1939947559 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e401c040 con 0x55b5a89bb240 2024-04-25T10:07:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63d0ff9640 1 -- 172.21.15.60:0/1939947559 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f63e8054560 con 0x55b5a89bb240 2024-04-25T10:07:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 10 client.5982 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T10:07:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 10 client.5982 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:07:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 20 client.5982 populate_metadata read hostname 'smithi060' 2024-04-25T10:07:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 --2- 172.21.15.60:0/1939947559 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x55b5a8976e80 0x55b5a89ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 4 client.5982 mds_command: new command op to 5796 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:07:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63fae2c000 1 -- 172.21.15.60:0/1939947559 --> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b5a87cd2e0 con 0x55b5a8976e80 2024-04-25T10:07:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63f5a5b640 1 --2- 172.21.15.60:0/1939947559 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x55b5a8976e80 0x55b5a89ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63f5a5b640 1 --2- 172.21.15.60:0/1939947559 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x55b5a8976e80 0x55b5a89ffc40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9bcb48280ff104 server_cookie=90669ee540fbbe81 in_seq=0 out_seq=0 2024-04-25T10:07:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.858+0000 7f63d0ff9640 10 client.5982 ms_handle_connect on v2:172.21.15.60:6833/1747757196 2024-04-25T10:07:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63d0ff9640 1 -- 172.21.15.60:0/1939947559 <== mds.0 v2:172.21.15.60:6833/1747757196 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55b5a87cd2e0 con 0x55b5a8976e80 2024-04-25T10:07:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63d0ff9640 10 client.5982 handle_command_reply: tid=0 2024-04-25T10:07:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63fae2c000 1 client.5982 shutdown 2024-04-25T10:07:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63fae2c000 1 -- 172.21.15.60:0/1939947559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a89bb240 msgr2=0x55b5a89ff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63fae2c000 1 --2- 172.21.15.60:0/1939947559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a89bb240 0x55b5a89ff700 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f63e805d150 tx=0x7f63e8001370 comp rx=0 tx=0).stop 2024-04-25T10:07:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63fae2c000 1 -- 172.21.15.60:0/1939947559 shutdown_connections 2024-04-25T10:07:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63fae2c000 1 --2- 172.21.15.60:0/1939947559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b5a89bb240 0x55b5a89ff700 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63fae2c000 1 --2- 172.21.15.60:0/1939947559 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x55b5a8976e80 0x55b5a89ffc40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63fae2c000 1 --2- 172.21.15.60:0/1939947559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5a89ba920 0x55b5a89ff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63fae2c000 1 --2- 172.21.15.60:0/1939947559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b5a8970f30 0x55b5a89c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63d0ff9640 0 client.5982 ms_handle_reset on v2:172.21.15.60:6833/1747757196 2024-04-25T10:07:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63fae2c000 1 -- 172.21.15.60:0/1939947559 >> 172.21.15.60:0/1939947559 conn(0x55b5a89772a0 msgr2=0x55b5a8973320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63fae2c000 1 -- 172.21.15.60:0/1939947559 shutdown_connections 2024-04-25T10:07:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63fae2c000 1 -- 172.21.15.60:0/1939947559 wait complete. 2024-04-25T10:07:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63fae2c000 20 client.5982 trim_cache size 0 max 16384 2024-04-25T10:07:17.875 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:07:17.875 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "0ea93863-a079-45c4-abed-6e83540ea9fe", 2024-04-25T10:07:17.875 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T10:07:17.875 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5796, 2024-04-25T10:07:17.875 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T10:07:17.875 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T10:07:17.875 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T10:07:17.875 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 9.2124125289999999, 2024-04-25T10:07:17.875 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 21, 2024-04-25T10:07:17.875 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 47, 2024-04-25T10:07:17.876 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 47, 2024-04-25T10:07:17.876 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 15.230145651999999 2024-04-25T10:07:17.876 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:07:17.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63f7cbf640 1 -- 172.21.15.60:0/969283888 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63d4066f60 msgr2=0x7f63d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63f7cbf640 1 --2- 172.21.15.60:0/969283888 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63d4066f60 0x7f63d4069420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009a30 tx=0x7f63e4008040 comp rx=0 tx=0).stop 2024-04-25T10:07:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63f7cbf640 1 -- 172.21.15.60:0/969283888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63f0156840 msgr2=0x7f63f0156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63f7cbf640 1 --2- 172.21.15.60:0/969283888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63f0156840 0x7f63f0156ca0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f63f0072c10 tx=0x7f63e800eb90 comp rx=0 tx=0).stop 2024-04-25T10:07:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63f7cbf640 1 -- 172.21.15.60:0/969283888 shutdown_connections 2024-04-25T10:07:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63f7cbf640 1 --2- 172.21.15.60:0/969283888 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f63d4066f60 0x7f63d4069420 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63f7cbf640 1 --2- 172.21.15.60:0/969283888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63f0156840 0x7f63f0156ca0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63f7cbf640 1 --2- 172.21.15.60:0/969283888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63f0154080 0x7f63f01544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63f7cbf640 1 --2- 172.21.15.60:0/969283888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63dc003f70 0x7f63f0143e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63f7cbf640 1 -- 172.21.15.60:0/969283888 >> 172.21.15.60:0/969283888 conn(0x7f63f00b12d0 msgr2=0x7f63f0071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63f7cbf640 1 -- 172.21.15.60:0/969283888 shutdown_connections 2024-04-25T10:07:17.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.862+0000 7f63f7cbf640 1 -- 172.21.15.60:0/969283888 wait complete. 2024-04-25T10:07:17.885 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:07:17.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.986+0000 7fabd78ce640 1 Processor -- start 2024-04-25T10:07:17.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.986+0000 7fabd78ce640 1 -- start start 2024-04-25T10:07:17.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.986+0000 7fabd78ce640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fabd015f480 0x7fabd015f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.986+0000 7fabd78ce640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd015fda0 con 0x7fabd00cd390 2024-04-25T10:07:17.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.986+0000 7fabd78ce640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd00cc5d0 con 0x7fabd00cf380 2024-04-25T10:07:17.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.986+0000 7fabd78ce640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd00cc8b0 con 0x7fabd015f480 2024-04-25T10:07:17.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.986+0000 7fabd4e69640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fabd00cd390 0x7fabd009a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60084/0 (socket says 172.21.15.60:60084) 2024-04-25T10:07:17.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.986+0000 7fabd4e69640 1 -- 172.21.15.60:0/1865690861 learned_addr learned my addr 172.21.15.60:0/1865690861 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:17.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.986+0000 7fabd566a640 1 --2- 172.21.15.60:0/1865690861 >> v2:172.21.15.60:3300/0 conn(0x7fabd015f480 0x7fabd015f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.986+0000 7fabc7fff640 1 -- 172.21.15.60:0/1865690861 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398426488 0 0) 0x7fabd015fda0 con 0x7fabd00cd390 2024-04-25T10:07:17.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.986+0000 7fabc7fff640 1 -- 172.21.15.60:0/1865690861 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabbc003530 con 0x7fabd00cd390 2024-04-25T10:07:17.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.986+0000 7fabd566a640 1 -- 172.21.15.60:0/1865690861 >> v1:172.21.15.100:6789/0 conn(0x7fabd00cf380 legacy=0x7fabd00cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.986+0000 7fabd566a640 1 -- 172.21.15.60:0/1865690861 >> v1:172.21.15.60:6789/0 conn(0x7fabd00cd390 legacy=0x7fabd009a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.986+0000 7fabd566a640 1 -- 172.21.15.60:0/1865690861 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd00ccb90 con 0x7fabd015f480 2024-04-25T10:07:17.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd566a640 1 --2- 172.21.15.60:0/1865690861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd015f480 0x7fabd015f860 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fabcc009d90 tx=0x7fabcc01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf806e3f53c0dab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabc7fff640 1 -- 172.21.15.60:0/1865690861 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc0b2070 con 0x7fabd015f480 2024-04-25T10:07:17.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabc7fff640 1 -- 172.21.15.60:0/1865690861 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc0a5a20 con 0x7fabd015f480 2024-04-25T10:07:17.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabc7fff640 1 -- 172.21.15.60:0/1865690861 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc0a5dd0 con 0x7fabd015f480 2024-04-25T10:07:17.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd78ce640 1 -- 172.21.15.60:0/1865690861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd015f480 msgr2=0x7fabd015f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd78ce640 1 --2- 172.21.15.60:0/1865690861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd015f480 0x7fabd015f860 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fabcc009d90 tx=0x7fabcc01c940 comp rx=0 tx=0).stop 2024-04-25T10:07:17.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd78ce640 1 -- 172.21.15.60:0/1865690861 shutdown_connections 2024-04-25T10:07:17.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd78ce640 1 --2- 172.21.15.60:0/1865690861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd015f480 0x7fabd015f860 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd78ce640 1 -- 172.21.15.60:0/1865690861 >> 172.21.15.60:0/1865690861 conn(0x7fabd00b32a0 msgr2=0x7fabd00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd78ce640 1 -- 172.21.15.60:0/1865690861 shutdown_connections 2024-04-25T10:07:17.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd78ce640 1 -- 172.21.15.60:0/1865690861 wait complete. 2024-04-25T10:07:17.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd78ce640 1 Processor -- start 2024-04-25T10:07:17.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd78ce640 1 -- start start 2024-04-25T10:07:18.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd78ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabd00cd390 0x7fabd00af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd78ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd00cf380 0x7fabd00c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd78ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd015f480 0x7fabd0154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd78ce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd01602f0 con 0x7fabd015f480 2024-04-25T10:07:18.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd78ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd0160170 con 0x7fabd00cf380 2024-04-25T10:07:18.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd78ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd015cbc0 con 0x7fabd00cd390 2024-04-25T10:07:18.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd5e6b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd015f480 0x7fabd0154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd5e6b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd015f480 0x7fabd0154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39292/0 (socket says 172.21.15.60:39292) 2024-04-25T10:07:18.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd5e6b640 1 -- 172.21.15.60:0/1783647111 learned_addr learned my addr 172.21.15.60:0/1783647111 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:18.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd566a640 1 --2- 172.21.15.60:0/1783647111 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabd00cd390 0x7fabd00af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd4e69640 1 --2- 172.21.15.60:0/1783647111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd00cf380 0x7fabd00c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.990+0000 7fabd5e6b640 1 -- 172.21.15.60:0/1783647111 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabd00cd390 msgr2=0x7fabd00af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabd5e6b640 1 --2- 172.21.15.60:0/1783647111 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabd00cd390 0x7fabd00af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabd5e6b640 1 -- 172.21.15.60:0/1783647111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd00cf380 msgr2=0x7fabd00c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabd5e6b640 1 --2- 172.21.15.60:0/1783647111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd00cf380 0x7fabd00c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabd5e6b640 1 -- 172.21.15.60:0/1783647111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd00c9fc0 con 0x7fabd015f480 2024-04-25T10:07:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabd4e69640 1 --2- 172.21.15.60:0/1783647111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd00cf380 0x7fabd00c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabd5e6b640 1 --2- 172.21.15.60:0/1783647111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd015f480 0x7fabd0154010 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fabc800c430 tx=0x7fabc800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabc5ffb640 1 -- 172.21.15.60:0/1783647111 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8005220 con 0x7fabd015f480 2024-04-25T10:07:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabc5ffb640 1 -- 172.21.15.60:0/1783647111 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc80053c0 con 0x7fabd015f480 2024-04-25T10:07:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabd78ce640 1 -- 172.21.15.60:0/1783647111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd00ca260 con 0x7fabd015f480 2024-04-25T10:07:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabc5ffb640 1 -- 172.21.15.60:0/1783647111 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8007270 con 0x7fabd015f480 2024-04-25T10:07:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabd78ce640 1 -- 172.21.15.60:0/1783647111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd00afe30 con 0x7fabd015f480 2024-04-25T10:07:18.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabd78ce640 1 -- 172.21.15.60:0/1783647111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd0074770 con 0x7fabd015f480 2024-04-25T10:07:18.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabc5ffb640 1 -- 172.21.15.60:0/1783647111 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fabc8019020 con 0x7fabd015f480 2024-04-25T10:07:18.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabc5ffb640 1 --2- 172.21.15.60:0/1783647111 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabb4066f60 0x7fabb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabc5ffb640 1 -- 172.21.15.60:0/1783647111 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabc80843d0 con 0x7fabd015f480 2024-04-25T10:07:18.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabd566a640 1 --2- 172.21.15.60:0/1783647111 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabb4066f60 0x7fabb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.994+0000 7fabd566a640 1 --2- 172.21.15.60:0/1783647111 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabb4066f60 0x7fabb4069420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0af000 tx=0x7fabcc0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:17.998+0000 7fabc5ffb640 1 -- 172.21.15.60:0/1783647111 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fabc80553e0 con 0x7fabd015f480 2024-04-25T10:07:18.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:18.190+0000 7fabd78ce640 1 -- 172.21.15.60:0/1783647111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fabd00b7730 con 0x7fabd015f480 2024-04-25T10:07:18.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:18.190+0000 7fabc5ffb640 1 -- 172.21.15.60:0/1783647111 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+73 (secure 0 0 0) 0x7fabc8054ce0 con 0x7fabd015f480 2024-04-25T10:07:18.198 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:07:18.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:18.190+0000 7fabd78ce640 1 -- 172.21.15.60:0/1783647111 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabb4066f60 msgr2=0x7fabb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:18.190+0000 7fabd78ce640 1 --2- 172.21.15.60:0/1783647111 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabb4066f60 0x7fabb4069420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0af000 tx=0x7fabcc0a7070 comp rx=0 tx=0).stop 2024-04-25T10:07:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:18.190+0000 7fabd78ce640 1 -- 172.21.15.60:0/1783647111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd015f480 msgr2=0x7fabd0154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:18.190+0000 7fabd78ce640 1 --2- 172.21.15.60:0/1783647111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd015f480 0x7fabd0154010 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fabc800c430 tx=0x7fabc800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:18.194+0000 7fabd78ce640 1 -- 172.21.15.60:0/1783647111 shutdown_connections 2024-04-25T10:07:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:18.194+0000 7fabd78ce640 1 --2- 172.21.15.60:0/1783647111 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabb4066f60 0x7fabb4069420 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:18.194+0000 7fabd78ce640 1 --2- 172.21.15.60:0/1783647111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd015f480 0x7fabd0154010 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:18.194+0000 7fabd78ce640 1 --2- 172.21.15.60:0/1783647111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd00cf380 0x7fabd00c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:18.194+0000 7fabd78ce640 1 --2- 172.21.15.60:0/1783647111 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabd00cd390 0x7fabd00af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:18.194+0000 7fabd78ce640 1 -- 172.21.15.60:0/1783647111 >> 172.21.15.60:0/1783647111 conn(0x7fabd00b32a0 msgr2=0x7fabd00b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:18.194+0000 7fabd78ce640 1 -- 172.21.15.60:0/1783647111 shutdown_connections 2024-04-25T10:07:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:18.194+0000 7fabd78ce640 1 -- 172.21.15.60:0/1783647111 wait complete. 2024-04-25T10:07:18.215 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:07:18.215 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:07:18.215 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:07:18.215 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:07:18.215 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:18.215 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:07:18.215 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:07:18.216 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:07:18.216 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:18.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:18.216 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:07:18.222 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:18.222 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:18.222 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:18.222 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:18.222 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:18.222 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:18.222 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:07:18.222 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:18.222 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:07:18.222 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:07:18.223 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:18.223 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:07:18.223 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:07:18.223 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:18.223 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:07:18.223 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:07:18.223 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42419sec preferred_lft 42419sec 2024-04-25T10:07:18.223 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:07:18.223 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:18.223 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:07:18.223 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:18.223 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:07:18.223 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:07:18.223 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:18.224 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:07:18.224 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:18.224 INFO:teuthology.orchestra.run.smithi060.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:07:18.224 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:18.224 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:07:18.224 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:18.224 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:07:18.224 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:18.224 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:18.224 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:18.224 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:07:18.224 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:18.224 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:18.224 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:18.224 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:07:18.225 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:18.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:18.225 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:07:18.279 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:07:18.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:18.280 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:07:18.330 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:07:18.330 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:07:18.330 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:07:18.330 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:07:18.330 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:07:18.330 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:07:18.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:18.331 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:07:18.331 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:07:18.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:07:18.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:07:18.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:07:18.331 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:07:18.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:18.404 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:07:18.453 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:07:18.453 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:07:18.453 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:18.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:18.454 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T10:07:18.509 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:07:18.510 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:18.510 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:18.557 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:18.557 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:18.558 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:18.605 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:07:18.663 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:07:18.711 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:07:18.711 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:07:18.711 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:07:18.711 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:07:18.711 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:07:18.711 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:07:18.711 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:07:18.712 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:07:18.712 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:07:18.712 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:07:18.712 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:07:18.712 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:07:18.712 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:07:18.712 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:07:18.712 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:07:18.712 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:07:18.712 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:07:18.712 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:07:18.712 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:07:18.712 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:07:18.712 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:07:18.713 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:07:18.713 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:07:18.713 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:07:18.713 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:07:18.713 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:07:18.713 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:07:18.713 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:07:18.713 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:07:18.713 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:18.713 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:18.713 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:18.713 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:18.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:18.714 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:07:18.761 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:07:18.761 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:07:18.806 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:07:18.820 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:07:18.872 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:07:18.872 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:07:18.872 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:07:18.872 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:07:18.872 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:07:18.873 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:07:18.873 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:07:18.873 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:07:18.873 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:07:18.873 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:07:18.873 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:07:18.873 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:07:18.873 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:07:18.873 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:07:18.873 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:07:18.873 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:07:18.874 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:07:18.874 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:07:18.874 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:07:18.874 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:07:18.874 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:07:18.874 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:07:18.874 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:07:18.874 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:07:18.874 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:07:18.874 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:07:18.875 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:07:18.875 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:07:18.875 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:07:18.875 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:18.875 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:18.875 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:18.875 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:18.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:18.877 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:07:18.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2024-04-25T10:07:18.878+0000 7f1016e46580 -1 init, newargv = 0x55c0e1558cb0 newargc=15 2024-04-25T10:07:18.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[38583]: starting ceph client 2024-04-25T10:07:18.894 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[38583]: starting fuse 2024-04-25T10:07:19.994 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:07:20.012 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:20.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:20.070 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:07:20.117 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:07:20.117 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:07:20.117 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:07:20.117 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:07:20.118 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:07:20.118 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:07:20.118 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:07:20.118 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:07:20.118 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:07:20.118 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:07:20.118 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:07:20.118 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:07:20.118 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:07:20.118 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:07:20.118 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:07:20.119 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:07:20.119 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:07:20.119 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:07:20.119 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:07:20.119 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:07:20.119 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:07:20.119 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:07:20.119 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:07:20.119 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:07:20.119 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:07:20.119 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:07:20.119 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:07:20.119 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2024-04-25T10:07:20.119 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:07:20.245 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.38583.asok 2024-04-25T10:07:20.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:07:20.258 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38583.asok 2024-04-25T10:07:20.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:20.258 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38583.asok status 2024-04-25T10:07:20.416 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:07:20.713 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:07:20.713 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:07:20.713 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:07:20.713 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2024-04-25T10:07:20.713 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:07:20.713 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:07:20.713 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "38583", 2024-04-25T10:07:20.714 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:07:20.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:20.714 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:07:20.714 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:07:20.714 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6000, 2024-04-25T10:07:20.714 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:07:20.714 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:07:20.714 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:07:20.714 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 6000 2024-04-25T10:07:20.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:20.714 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:07:20.714 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:07:20.714 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:07:20.714 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2377985170 2024-04-25T10:07:20.714 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:20.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:20.715 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:07:20.715 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:07:20.715 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:07:20.715 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2377985170 2024-04-25T10:07:20.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:20.715 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.6000 192.168.0.1:0/2377985170", 2024-04-25T10:07:20.715 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/2377985170", 2024-04-25T10:07:20.715 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:07:20.715 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 21, 2024-04-25T10:07:20.715 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 47, 2024-04-25T10:07:20.715 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:07:20.715 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:07:20.715 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:07:20.715 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:07:20.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:20.717 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:07:20.717 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:20.724 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:07:20.725 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:20.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:20.725 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:20.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:20.784 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-04-25T10:07:20.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:20.843 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-04-25T10:07:20.894 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:07:20.894 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:07:20.894 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:07:20.894 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:07:20.895 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-04-25T10:07:20.895 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:07:21.006 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627777 2024-04-25T10:07:21.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:07:21.017 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:21.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.182+0000 7f326efca640 1 Processor -- start 2024-04-25T10:07:21.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.182+0000 7f326efca640 1 -- start start 2024-04-25T10:07:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.182+0000 7f326efca640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f326815bc90 0x7f326815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.182+0000 7f326efca640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3268071ca0 con 0x7f32680b4660 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.182+0000 7f326efca640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32680cca40 con 0x7f32680ce7a0 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.182+0000 7f326efca640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32680cc370 con 0x7f326815bc90 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f3267fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f326815bc90 0x7f326815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f326d567640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f32680b4660 0x7f326815c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34726/0 (socket says 172.21.15.60:34726) 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f326d567640 1 -- 172.21.15.60:0/3907463737 learned_addr learned my addr 172.21.15.60:0/3907463737 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f3267fff640 1 -- 172.21.15.60:0/3907463737 >> v1:172.21.15.100:6789/0 conn(0x7f32680ce7a0 legacy=0x7f32680ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f3267fff640 1 -- 172.21.15.60:0/3907463737 >> v1:172.21.15.60:6789/0 conn(0x7f32680b4660 legacy=0x7f326815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f3267fff640 1 -- 172.21.15.60:0/3907463737 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32680ccd20 con 0x7f326815bc90 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f3267fff640 1 --2- 172.21.15.60:0/3907463737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f326815bc90 0x7f326815c220 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f325c0097c0 tx=0x7f325c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5abd57ab1f85ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f32677fe640 1 -- 172.21.15.60:0/3907463737 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c0b0070 con 0x7f326815bc90 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f32677fe640 1 -- 172.21.15.60:0/3907463737 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325c0ab600 con 0x7f326815bc90 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f32677fe640 1 -- 172.21.15.60:0/3907463737 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c0ab9d0 con 0x7f326815bc90 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f326efca640 1 -- 172.21.15.60:0/3907463737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f326815bc90 msgr2=0x7f326815c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f326efca640 1 --2- 172.21.15.60:0/3907463737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f326815bc90 0x7f326815c220 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f325c0097c0 tx=0x7f325c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f326efca640 1 -- 172.21.15.60:0/3907463737 shutdown_connections 2024-04-25T10:07:21.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f326efca640 1 --2- 172.21.15.60:0/3907463737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f326815bc90 0x7f326815c220 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f326efca640 1 -- 172.21.15.60:0/3907463737 >> 172.21.15.60:0/3907463737 conn(0x7f32680b12d0 msgr2=0x7f32680b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f326efca640 1 -- 172.21.15.60:0/3907463737 shutdown_connections 2024-04-25T10:07:21.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.186+0000 7f326efca640 1 -- 172.21.15.60:0/3907463737 wait complete. 2024-04-25T10:07:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f326efca640 1 Processor -- start 2024-04-25T10:07:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f326efca640 1 -- start start 2024-04-25T10:07:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f326efca640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32680ce7a0 0x7f3268151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f326efca640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f326815bc90 0x7f3268150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f326efca640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3268149cf0 0x7f326814e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f326efca640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32681695d0 con 0x7f326815bc90 2024-04-25T10:07:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f326efca640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32680b3db0 con 0x7f3268149cf0 2024-04-25T10:07:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f326efca640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3268169750 con 0x7f32680ce7a0 2024-04-25T10:07:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f3267fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f326815bc90 0x7f3268150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f3267fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f326815bc90 0x7f3268150ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41846/0 (socket says 172.21.15.60:41846) 2024-04-25T10:07:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f3267fff640 1 -- 172.21.15.60:0/2016212891 learned_addr learned my addr 172.21.15.60:0/2016212891 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f326cd66640 1 --2- 172.21.15.60:0/2016212891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32680ce7a0 0x7f3268151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f326d567640 1 --2- 172.21.15.60:0/2016212891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3268149cf0 0x7f326814e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f3267fff640 1 -- 172.21.15.60:0/2016212891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32680ce7a0 msgr2=0x7f3268151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f3267fff640 1 --2- 172.21.15.60:0/2016212891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32680ce7a0 0x7f3268151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f3267fff640 1 -- 172.21.15.60:0/2016212891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3268149cf0 msgr2=0x7f326814e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f3267fff640 1 --2- 172.21.15.60:0/2016212891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3268149cf0 0x7f326814e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f3267fff640 1 -- 172.21.15.60:0/2016212891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32681510e0 con 0x7f326815bc90 2024-04-25T10:07:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f326cd66640 1 --2- 172.21.15.60:0/2016212891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32680ce7a0 0x7f3268151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f326d567640 1 --2- 172.21.15.60:0/2016212891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3268149cf0 0x7f326814e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f3267fff640 1 --2- 172.21.15.60:0/2016212891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f326815bc90 0x7f3268150ba0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f325c0190b0 tx=0x7f326815d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f32657fa640 1 -- 172.21.15.60:0/2016212891 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c0b0040 con 0x7f326815bc90 2024-04-25T10:07:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f32657fa640 1 -- 172.21.15.60:0/2016212891 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325c0b5460 con 0x7f326815bc90 2024-04-25T10:07:21.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f326efca640 1 -- 172.21.15.60:0/2016212891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3268152c80 con 0x7f326815bc90 2024-04-25T10:07:21.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f32657fa640 1 -- 172.21.15.60:0/2016212891 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c0b5830 con 0x7f326815bc90 2024-04-25T10:07:21.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f326efca640 1 -- 172.21.15.60:0/2016212891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32681526c0 con 0x7f326815bc90 2024-04-25T10:07:21.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.190+0000 7f326efca640 1 -- 172.21.15.60:0/2016212891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3268074770 con 0x7f326815bc90 2024-04-25T10:07:21.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.194+0000 7f32657fa640 1 -- 172.21.15.60:0/2016212891 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f325c0bf020 con 0x7f326815bc90 2024-04-25T10:07:21.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.194+0000 7f32657fa640 1 --2- 172.21.15.60:0/2016212891 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3254066f90 0x7f3254069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.194+0000 7f32657fa640 1 -- 172.21.15.60:0/2016212891 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f325c11ec80 con 0x7f326815bc90 2024-04-25T10:07:21.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.194+0000 7f326cd66640 1 --2- 172.21.15.60:0/2016212891 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3254066f90 0x7f3254069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.194+0000 7f326cd66640 1 --2- 172.21.15.60:0/2016212891 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3254066f90 0x7f3254069450 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3258007fd0 tx=0x7f3258009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.198+0000 7f32657fa640 1 -- 172.21.15.60:0/2016212891 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f325c0c6020 con 0x7f326815bc90 2024-04-25T10:07:21.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.390+0000 7f326efca640 1 -- 172.21.15.60:0/2016212891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3268154530 con 0x7f326815bc90 2024-04-25T10:07:21.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.394+0000 7f32657fa640 1 -- 172.21.15.60:0/2016212891 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f325c016090 con 0x7f326815bc90 2024-04-25T10:07:21.401 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:21.401 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1138162580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1138162580},{"type":"v1","addr":"172.21.15.100:6837","nonce":1138162580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2985559168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2985559168},{"type":"v1","addr":"172.21.15.100:6835","nonce":2985559168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/50795370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":50795370},{"type":"v1","addr":"172.21.15.60:6834","nonce":50795370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:07.980095+0000","modified":"2024-04-25T10:07:10.002670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5796":{"gid":5796,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1747757196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1747757196},{"type":"v1","addr":"172.21.15.60:6835","nonce":1747757196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:21.401 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 21 2024-04-25T10:07:21.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.394+0000 7f326efca640 1 -- 172.21.15.60:0/2016212891 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3254066f90 msgr2=0x7f3254069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.394+0000 7f326efca640 1 --2- 172.21.15.60:0/2016212891 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3254066f90 0x7f3254069450 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3258007fd0 tx=0x7f3258009040 comp rx=0 tx=0).stop 2024-04-25T10:07:21.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.394+0000 7f326efca640 1 -- 172.21.15.60:0/2016212891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f326815bc90 msgr2=0x7f3268150ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.394+0000 7f326efca640 1 --2- 172.21.15.60:0/2016212891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f326815bc90 0x7f3268150ba0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f325c0190b0 tx=0x7f326815d680 comp rx=0 tx=0).stop 2024-04-25T10:07:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.394+0000 7f326efca640 1 -- 172.21.15.60:0/2016212891 shutdown_connections 2024-04-25T10:07:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.394+0000 7f326efca640 1 --2- 172.21.15.60:0/2016212891 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3254066f90 0x7f3254069450 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.394+0000 7f326efca640 1 --2- 172.21.15.60:0/2016212891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3268149cf0 0x7f326814e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.394+0000 7f326efca640 1 --2- 172.21.15.60:0/2016212891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f326815bc90 0x7f3268150ba0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.394+0000 7f326efca640 1 --2- 172.21.15.60:0/2016212891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32680ce7a0 0x7f3268151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.394+0000 7f326efca640 1 -- 172.21.15.60:0/2016212891 >> 172.21.15.60:0/2016212891 conn(0x7f32680b12d0 msgr2=0x7f32680b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.394+0000 7f326efca640 1 -- 172.21.15.60:0/2016212891 shutdown_connections 2024-04-25T10:07:21.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.394+0000 7f326efca640 1 -- 172.21.15.60:0/2016212891 wait complete. 2024-04-25T10:07:21.420 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:07:21.575 INFO:teuthology.orchestra.run.smithi060.stdout:{"message":"","return_code":0} 2024-04-25T10:07:21.575 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:07:21.575 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:07:21.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde0b925080 1 Processor -- start 2024-04-25T10:07:21.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde0b925080 1 -- start start 2024-04-25T10:07:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde0b925080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b89765960 0x563b89761d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde0b925080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b8977e320 0x563b898022c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde0b925080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b8980a1e0 0x563b89802800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde0b925080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b8980bb20 con 0x563b8977e320 2024-04-25T10:07:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde0b925080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b896dc9f0 con 0x563b89765960 2024-04-25T10:07:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde0b925080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b8980bca0 con 0x563b8980a1e0 2024-04-25T10:07:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde08c42640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b8977e320 0x563b898022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde08c42640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b8977e320 0x563b898022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41854/0 (socket says 172.21.15.60:41854) 2024-04-25T10:07:21.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde08c42640 1 -- 172.21.15.60:0/781939517 learned_addr learned my addr 172.21.15.60:0/781939517 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:21.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde09443640 1 --2- 172.21.15.60:0/781939517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b89765960 0x563b89761d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde08c42640 1 -- 172.21.15.60:0/781939517 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b8980a1e0 msgr2=0x563b89802800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:21.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde08c42640 1 --2- 172.21.15.60:0/781939517 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b8980a1e0 0x563b89802800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde08c42640 1 -- 172.21.15.60:0/781939517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b89765960 msgr2=0x563b89761d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:21.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde08c42640 1 --2- 172.21.15.60:0/781939517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b89765960 0x563b89761d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde08c42640 1 -- 172.21.15.60:0/781939517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b8980e900 con 0x563b8977e320 2024-04-25T10:07:21.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde09443640 1 --2- 172.21.15.60:0/781939517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b89765960 0x563b89761d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:21.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fde08c42640 1 --2- 172.21.15.60:0/781939517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b8977e320 0x563b898022c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fde00001ea0 tx=0x7fde0000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b122cdcc063e08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fddfb7fe640 1 -- 172.21.15.60:0/781939517 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00012020 con 0x563b8977e320 2024-04-25T10:07:21.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fddfb7fe640 1 -- 172.21.15.60:0/781939517 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0000e690 con 0x563b8977e320 2024-04-25T10:07:21.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.606+0000 7fddfb7fe640 1 -- 172.21.15.60:0/781939517 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0000ea80 con 0x563b8977e320 2024-04-25T10:07:21.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 -- 172.21.15.60:0/781939517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b8977e320 msgr2=0x563b898022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 --2- 172.21.15.60:0/781939517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b8977e320 0x563b898022c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fde00001ea0 tx=0x7fde0000b790 comp rx=0 tx=0).stop 2024-04-25T10:07:21.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 -- 172.21.15.60:0/781939517 shutdown_connections 2024-04-25T10:07:21.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 --2- 172.21.15.60:0/781939517 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b8980a1e0 0x563b89802800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 --2- 172.21.15.60:0/781939517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b8977e320 0x563b898022c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 --2- 172.21.15.60:0/781939517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b89765960 0x563b89761d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 -- 172.21.15.60:0/781939517 >> 172.21.15.60:0/781939517 conn(0x563b89760e80 msgr2=0x563b89765200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 -- 172.21.15.60:0/781939517 shutdown_connections 2024-04-25T10:07:21.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 -- 172.21.15.60:0/781939517 wait complete. 2024-04-25T10:07:21.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 Processor -- start 2024-04-25T10:07:21.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 -- start start 2024-04-25T10:07:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b89765960 0x563b89764000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b8977e320 0x563b89805140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b8980a1e0 0x563b89801310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b89738750 con 0x563b8977e320 2024-04-25T10:07:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b897384b0 con 0x563b8980a1e0 2024-04-25T10:07:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b897389f0 con 0x563b89765960 2024-04-25T10:07:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde08c42640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b8977e320 0x563b89805140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde08c42640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b8977e320 0x563b89805140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41860/0 (socket says 172.21.15.60:41860) 2024-04-25T10:07:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde08c42640 1 -- 172.21.15.60:0/265773522 learned_addr learned my addr 172.21.15.60:0/265773522 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde09c44640 1 --2- 172.21.15.60:0/265773522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b8980a1e0 0x563b89801310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde09443640 1 --2- 172.21.15.60:0/265773522 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b89765960 0x563b89764000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde08c42640 1 -- 172.21.15.60:0/265773522 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b89765960 msgr2=0x563b89764000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde08c42640 1 --2- 172.21.15.60:0/265773522 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b89765960 0x563b89764000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde08c42640 1 -- 172.21.15.60:0/265773522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b8980a1e0 msgr2=0x563b89801310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde08c42640 1 --2- 172.21.15.60:0/265773522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b8980a1e0 0x563b89801310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde08c42640 1 -- 172.21.15.60:0/265773522 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b89809320 con 0x563b8977e320 2024-04-25T10:07:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde08c42640 1 --2- 172.21.15.60:0/265773522 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b8977e320 0x563b89805140 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fde00010000 tx=0x7fde00000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fddfa7fc640 1 -- 172.21.15.60:0/265773522 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00027070 con 0x563b8977e320 2024-04-25T10:07:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fddfa7fc640 1 -- 172.21.15.60:0/265773522 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde000197d0 con 0x563b8977e320 2024-04-25T10:07:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 -- 172.21.15.60:0/265773522 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563b89805c50 con 0x563b8977e320 2024-04-25T10:07:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fddfa7fc640 1 -- 172.21.15.60:0/265773522 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00019bc0 con 0x563b8977e320 2024-04-25T10:07:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.610+0000 7fde0b925080 1 -- 172.21.15.60:0/265773522 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b898135d0 con 0x563b8977e320 2024-04-25T10:07:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fddfa7fc640 1 -- 172.21.15.60:0/265773522 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde00015020 con 0x563b8977e320 2024-04-25T10:07:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fddfa7fc640 1 --2- 172.21.15.60:0/265773522 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fddec065970 0x7fddec067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fddfa7fc640 1 -- 172.21.15.60:0/265773522 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde00085660 con 0x563b8977e320 2024-04-25T10:07:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde0b925080 1 --2- 172.21.15.60:0/265773522 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x563b89764540 0x563b89813d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde0b925080 1 -- 172.21.15.60:0/265773522 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v9 -- 0x563b89814250 con 0x563b89764540 2024-04-25T10:07:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde09443640 1 --2- 172.21.15.60:0/265773522 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fddec065970 0x7fddec067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde09c44640 1 --2- 172.21.15.60:0/265773522 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x563b89764540 0x563b89813d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde09c44640 1 --2- 172.21.15.60:0/265773522 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x563b89764540 0x563b89813d10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde09443640 1 --2- 172.21.15.60:0/265773522 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fddec065970 0x7fddec067e30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009610 tx=0x7fddfc005830 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde09c44640 1 -- 172.21.15.60:0/265773522 <== osd.6 v2:172.21.15.100:6824/4086390550 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7fde040b3040 con 0x563b89764540 2024-04-25T10:07:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde0b925080 1 -- 172.21.15.60:0/265773522 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x563b89764540 msgr2=0x563b89813d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde0b925080 1 --2- 172.21.15.60:0/265773522 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x563b89764540 0x563b89813d10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde0b925080 1 -- 172.21.15.60:0/265773522 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fddec065970 msgr2=0x7fddec067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde0b925080 1 --2- 172.21.15.60:0/265773522 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fddec065970 0x7fddec067e30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009610 tx=0x7fddfc005830 comp rx=0 tx=0).stop 2024-04-25T10:07:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde0b925080 1 -- 172.21.15.60:0/265773522 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b8977e320 msgr2=0x563b89805140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde0b925080 1 --2- 172.21.15.60:0/265773522 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b8977e320 0x563b89805140 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fde00010000 tx=0x7fde00000bd0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde09c44640 1 -- 172.21.15.60:0/265773522 reap_dead start 2024-04-25T10:07:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde0b925080 1 -- 172.21.15.60:0/265773522 shutdown_connections 2024-04-25T10:07:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde0b925080 1 -- 172.21.15.60:0/265773522 >> 172.21.15.60:0/265773522 conn(0x563b89760e80 msgr2=0x563b89765200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde0b925080 1 -- 172.21.15.60:0/265773522 shutdown_connections 2024-04-25T10:07:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.614+0000 7fde0b925080 1 -- 172.21.15.60:0/265773522 wait complete. 2024-04-25T10:07:21.627 INFO:teuthology.orchestra.run.smithi060.stdout:Yalphaparent_a 2024-04-25T10:07:21.627 DEBUG:teuthology.orchestra.run.smithi060:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:07:21.712 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:07:21.712 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511627777, 2024-04-25T10:07:21.713 INFO:teuthology.orchestra.run.smithi060.stdout: "ancestors": [ 2024-04-25T10:07:21.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:21.713 INFO:teuthology.orchestra.run.smithi060.stdout: "dirino": 1099511627776, 2024-04-25T10:07:21.713 INFO:teuthology.orchestra.run.smithi060.stdout: "dname": "alpha", 2024-04-25T10:07:21.713 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 2 2024-04-25T10:07:21.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:21.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:21.713 INFO:teuthology.orchestra.run.smithi060.stdout: "dirino": 1, 2024-04-25T10:07:21.713 INFO:teuthology.orchestra.run.smithi060.stdout: "dname": "parent_a", 2024-04-25T10:07:21.713 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 4 2024-04-25T10:07:21.713 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:21.713 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-04-25T10:07:21.713 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 7, 2024-04-25T10:07:21.713 INFO:teuthology.orchestra.run.smithi060.stdout: "old_pools": [] 2024-04-25T10:07:21.713 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:07:21.714 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:21.715 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-04-25T10:07:21.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5674ee7080 1 Processor -- start 2024-04-25T10:07:21.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5674ee7080 1 -- start start 2024-04-25T10:07:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5674ee7080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564fcd1fbaa0 0x564fcd2d0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5674ee7080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564fcd1fbe80 0x564fcd2d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5674ee7080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fcd2d3040 0x564fcd3658c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5674ee7080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcd2d4380 con 0x564fcd1fbaa0 2024-04-25T10:07:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5674ee7080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcd1a29f0 con 0x564fcd2d3040 2024-04-25T10:07:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5674ee7080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x564fcd2d4500 con 0x564fcd1fbe80 2024-04-25T10:07:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5672a05640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564fcd1fbaa0 0x564fcd2d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5672a05640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564fcd1fbaa0 0x564fcd2d0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41872/0 (socket says 172.21.15.60:41872) 2024-04-25T10:07:21.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5672a05640 1 -- 172.21.15.60:0/510786768 learned_addr learned my addr 172.21.15.60:0/510786768 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5672204640 1 --2- 172.21.15.60:0/510786768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564fcd1fbe80 0x564fcd2d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5673206640 1 --2- 172.21.15.60:0/510786768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fcd2d3040 0x564fcd3658c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5672a05640 1 -- 172.21.15.60:0/510786768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564fcd1fbe80 msgr2=0x564fcd2d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5672a05640 1 --2- 172.21.15.60:0/510786768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564fcd1fbe80 0x564fcd2d0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5672a05640 1 -- 172.21.15.60:0/510786768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fcd2d3040 msgr2=0x564fcd3658c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5672a05640 1 --2- 172.21.15.60:0/510786768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fcd2d3040 0x564fcd3658c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5672a05640 1 -- 172.21.15.60:0/510786768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fcd367f20 con 0x564fcd1fbaa0 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.790+0000 7f5672204640 1 --2- 172.21.15.60:0/510786768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564fcd1fbe80 0x564fcd2d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5672a05640 1 --2- 172.21.15.60:0/510786768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564fcd1fbaa0 0x564fcd2d0860 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f566400a590 tx=0x7f5664003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbd332ddfe31d7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5671202640 1 -- 172.21.15.60:0/510786768 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56640af040 con 0x564fcd1fbaa0 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5671202640 1 -- 172.21.15.60:0/510786768 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56640b9070 con 0x564fcd1fbaa0 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5671202640 1 -- 172.21.15.60:0/510786768 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56640aad10 con 0x564fcd1fbaa0 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 -- 172.21.15.60:0/510786768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564fcd1fbaa0 msgr2=0x564fcd2d0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 --2- 172.21.15.60:0/510786768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564fcd1fbaa0 0x564fcd2d0860 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f566400a590 tx=0x7f5664003030 comp rx=0 tx=0).stop 2024-04-25T10:07:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 -- 172.21.15.60:0/510786768 shutdown_connections 2024-04-25T10:07:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 --2- 172.21.15.60:0/510786768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fcd2d3040 0x564fcd3658c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 --2- 172.21.15.60:0/510786768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564fcd1fbe80 0x564fcd2d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 --2- 172.21.15.60:0/510786768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564fcd1fbaa0 0x564fcd2d0860 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 -- 172.21.15.60:0/510786768 >> 172.21.15.60:0/510786768 conn(0x564fcd1fc750 msgr2=0x564fcd22e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 -- 172.21.15.60:0/510786768 shutdown_connections 2024-04-25T10:07:21.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 -- 172.21.15.60:0/510786768 wait complete. 2024-04-25T10:07:21.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 Processor -- start 2024-04-25T10:07:21.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 -- start start 2024-04-25T10:07:21.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564fcd1fbaa0 0x564fcd2cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fcd1fbe80 0x564fcd241800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564fcd2d3040 0x564fcd2d0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcd2d1c70 con 0x564fcd2d3040 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fcd240890 con 0x564fcd1fbe80 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x564fcd22ba50 con 0x564fcd1fbaa0 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5673206640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564fcd2d3040 0x564fcd2d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5673206640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564fcd2d3040 0x564fcd2d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41884/0 (socket says 172.21.15.60:41884) 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5673206640 1 -- 172.21.15.60:0/2401665038 learned_addr learned my addr 172.21.15.60:0/2401665038 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5672a05640 1 --2- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564fcd1fbaa0 0x564fcd2cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5672204640 1 --2- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fcd1fbe80 0x564fcd241800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5673206640 1 -- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564fcd1fbaa0 msgr2=0x564fcd2cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5673206640 1 --2- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564fcd1fbaa0 0x564fcd2cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5673206640 1 -- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fcd1fbe80 msgr2=0x564fcd241800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5673206640 1 --2- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fcd1fbe80 0x564fcd241800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5673206640 1 -- 172.21.15.60:0/2401665038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fcd2422b0 con 0x564fcd2d3040 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5672204640 1 --2- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564fcd1fbe80 0x564fcd241800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5673206640 1 --2- 172.21.15.60:0/2401665038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564fcd2d3040 0x564fcd2d0380 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f566c00c160 tx=0x7f566c00c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f566bfff640 1 -- 172.21.15.60:0/2401665038 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c017020 con 0x564fcd2d3040 2024-04-25T10:07:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f566bfff640 1 -- 172.21.15.60:0/2401665038 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c00ce50 con 0x564fcd2d3040 2024-04-25T10:07:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f566bfff640 1 -- 172.21.15.60:0/2401665038 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c0138b0 con 0x564fcd2d3040 2024-04-25T10:07:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.794+0000 7f5674ee7080 1 -- 172.21.15.60:0/2401665038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564fcd242550 con 0x564fcd2d3040 2024-04-25T10:07:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.798+0000 7f5674ee7080 1 -- 172.21.15.60:0/2401665038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564fcd243fc0 con 0x564fcd2d3040 2024-04-25T10:07:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.798+0000 7f566bfff640 1 -- 172.21.15.60:0/2401665038 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f566c01a020 con 0x564fcd2d3040 2024-04-25T10:07:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.798+0000 7f566bfff640 1 --2- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5654065940 0x7f5654067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.798+0000 7f566bfff640 1 -- 172.21.15.60:0/2401665038 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f566c083fc0 con 0x564fcd2d3040 2024-04-25T10:07:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.798+0000 7f5674ee7080 1 --2- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x564fcd245470 0x564fcd3786f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.798+0000 7f5674ee7080 1 -- 172.21.15.60:0/2401665038 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) v9 -- 0x564fcd378c30 con 0x564fcd245470 2024-04-25T10:07:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.798+0000 7f5672a05640 1 --2- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5654065940 0x7f5654067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.798+0000 7f5672204640 1 --2- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x564fcd245470 0x564fcd3786f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.798+0000 7f5672a05640 1 --2- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5654065940 0x7f5654067e00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5664018f80 tx=0x7f56640ac040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.798+0000 7f5672204640 1 --2- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x564fcd245470 0x564fcd3786f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.802+0000 7f5672204640 1 -- 172.21.15.60:0/2401665038 <== osd.6 v2:172.21.15.100:6824/4086390550 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v47'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f565c00d040 con 0x564fcd245470 2024-04-25T10:07:21.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.802+0000 7f5674ee7080 1 -- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x564fcd245470 msgr2=0x564fcd3786f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.802+0000 7f5674ee7080 1 --2- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x564fcd245470 0x564fcd3786f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.802+0000 7f5674ee7080 1 -- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5654065940 msgr2=0x7f5654067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.802+0000 7f5674ee7080 1 --2- 172.21.15.60:0/2401665038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5654065940 0x7f5654067e00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5664018f80 tx=0x7f56640ac040 comp rx=0 tx=0).stop 2024-04-25T10:07:21.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.802+0000 7f5674ee7080 1 -- 172.21.15.60:0/2401665038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564fcd2d3040 msgr2=0x564fcd2d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.802+0000 7f5674ee7080 1 --2- 172.21.15.60:0/2401665038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564fcd2d3040 0x564fcd2d0380 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f566c00c160 tx=0x7f566c00c620 comp rx=0 tx=0).stop 2024-04-25T10:07:21.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.802+0000 7f5673206640 1 -- 172.21.15.60:0/2401665038 reap_dead start 2024-04-25T10:07:21.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.802+0000 7f5674ee7080 1 -- 172.21.15.60:0/2401665038 shutdown_connections 2024-04-25T10:07:21.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.802+0000 7f5674ee7080 1 -- 172.21.15.60:0/2401665038 >> 172.21.15.60:0/2401665038 conn(0x564fcd1fc750 msgr2=0x564fcd23f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.802+0000 7f5674ee7080 1 -- 172.21.15.60:0/2401665038 shutdown_connections 2024-04-25T10:07:21.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.802+0000 7f5674ee7080 1 -- 172.21.15.60:0/2401665038 wait complete. 2024-04-25T10:07:21.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:07:21.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:21.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-04-25T10:07:21.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.978+0000 7f4fe9631640 1 Processor -- start 2024-04-25T10:07:21.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.978+0000 7f4fe9631640 1 -- start start 2024-04-25T10:07:21.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.978+0000 7f4fe9631640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fe40b95b0 0x7f4fe40ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.978+0000 7f4fe9631640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4fe40cec00 0x7f4fe40d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.978+0000 7f4fe9631640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe40d1a70 con 0x7f4fe415da40 2024-04-25T10:07:21.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.978+0000 7f4fe9631640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe415e530 con 0x7f4fe40cec00 2024-04-25T10:07:21.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.978+0000 7f4fe9631640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe415e7f0 con 0x7f4fe40b95b0 2024-04-25T10:07:21.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.978+0000 7f4fe37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4fe40cec00 0x7f4fe40d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.978+0000 7f4fe37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4fe40cec00 0x7f4fe40d2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41896/0 (socket says 172.21.15.60:41896) 2024-04-25T10:07:21.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.978+0000 7f4fe37fe640 1 -- 172.21.15.60:0/885538661 learned_addr learned my addr 172.21.15.60:0/885538661 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:21.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.978+0000 7f4fdbfff640 1 --2- 172.21.15.60:0/885538661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fe40b95b0 0x7f4fe40ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.982+0000 7f4fe37fe640 1 -- 172.21.15.60:0/885538661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fe40b95b0 msgr2=0x7f4fe40ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:21.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.982+0000 7f4fe37fe640 1 --2- 172.21.15.60:0/885538661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fe40b95b0 0x7f4fe40ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.982+0000 7f4fe37fe640 1 -- 172.21.15.60:0/885538661 >> v1:172.21.15.100:6789/0 conn(0x7f4fe415da40 legacy=0x7f4fe415de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.982+0000 7f4fe37fe640 1 -- 172.21.15.60:0/885538661 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe415ead0 con 0x7f4fe40cec00 2024-04-25T10:07:21.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.982+0000 7f4fe37fe640 1 --2- 172.21.15.60:0/885538661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe40cec00 0x7f4fe40d2230 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009e50 tx=0x7f4fdc01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cdb5c17179dca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.982+0000 7f4fe27fc640 1 -- 172.21.15.60:0/885538661 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0a5700 con 0x7f4fe40cec00 2024-04-25T10:07:21.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.982+0000 7f4fe27fc640 1 -- 172.21.15.60:0/885538661 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc0a58a0 con 0x7f4fe40cec00 2024-04-25T10:07:21.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.982+0000 7f4fe27fc640 1 -- 172.21.15.60:0/885538661 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0a5c70 con 0x7f4fe40cec00 2024-04-25T10:07:21.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 -- 172.21.15.60:0/885538661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe40cec00 msgr2=0x7f4fe40d2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 --2- 172.21.15.60:0/885538661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe40cec00 0x7f4fe40d2230 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009e50 tx=0x7f4fdc01ca30 comp rx=0 tx=0).stop 2024-04-25T10:07:21.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 -- 172.21.15.60:0/885538661 shutdown_connections 2024-04-25T10:07:21.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 --2- 172.21.15.60:0/885538661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe40cec00 0x7f4fe40d2230 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 --2- 172.21.15.60:0/885538661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fe40b95b0 0x7f4fe40ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 -- 172.21.15.60:0/885538661 >> 172.21.15.60:0/885538661 conn(0x7f4fe40d0470 msgr2=0x7f4fe40d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 -- 172.21.15.60:0/885538661 shutdown_connections 2024-04-25T10:07:21.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 -- 172.21.15.60:0/885538661 wait complete. 2024-04-25T10:07:21.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 Processor -- start 2024-04-25T10:07:21.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 -- start start 2024-04-25T10:07:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe40b95b0 0x7f4fe41f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fe40cec00 0x7f4fe40cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fe415da40 0x7f4fe415b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40d3310 con 0x7f4fe40b95b0 2024-04-25T10:07:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40d3190 con 0x7f4fe40cec00 2024-04-25T10:07:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40d3490 con 0x7f4fe415da40 2024-04-25T10:07:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe40b95b0 0x7f4fe41f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe40b95b0 0x7f4fe41f6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41906/0 (socket says 172.21.15.60:41906) 2024-04-25T10:07:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe2ffd640 1 -- 172.21.15.60:0/3661481282 learned_addr learned my addr 172.21.15.60:0/3661481282 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fdbfff640 1 --2- 172.21.15.60:0/3661481282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fe40cec00 0x7f4fe40cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe37fe640 1 --2- 172.21.15.60:0/3661481282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fe415da40 0x7f4fe415b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe2ffd640 1 -- 172.21.15.60:0/3661481282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fe415da40 msgr2=0x7f4fe415b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe2ffd640 1 --2- 172.21.15.60:0/3661481282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fe415da40 0x7f4fe415b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe2ffd640 1 -- 172.21.15.60:0/3661481282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fe40cec00 msgr2=0x7f4fe40cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe2ffd640 1 --2- 172.21.15.60:0/3661481282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fe40cec00 0x7f4fe40cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe2ffd640 1 -- 172.21.15.60:0/3661481282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe41f6950 con 0x7f4fe40b95b0 2024-04-25T10:07:21.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe2ffd640 1 --2- 172.21.15.60:0/3661481282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe40b95b0 0x7f4fe41f6410 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400c390 tx=0x7f4fd400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fdb7fe640 1 -- 172.21.15.60:0/3661481282 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4017070 con 0x7f4fe40b95b0 2024-04-25T10:07:21.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fdb7fe640 1 -- 172.21.15.60:0/3661481282 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd40052f0 con 0x7f4fe40b95b0 2024-04-25T10:07:21.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 -- 172.21.15.60:0/3661481282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe40c8560 con 0x7f4fe40b95b0 2024-04-25T10:07:21.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fdb7fe640 1 -- 172.21.15.60:0/3661481282 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4003840 con 0x7f4fe40b95b0 2024-04-25T10:07:21.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.986+0000 7f4fe9631640 1 -- 172.21.15.60:0/3661481282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe40c8b20 con 0x7f4fe40b95b0 2024-04-25T10:07:21.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fdb7fe640 1 -- 172.21.15.60:0/3661481282 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4fd401a070 con 0x7f4fe40b95b0 2024-04-25T10:07:21.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fdb7fe640 1 --2- 172.21.15.60:0/3661481282 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fc4066f60 0x7f4fc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fdb7fe640 1 -- 172.21.15.60:0/3661481282 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fd4084360 con 0x7f4fe40b95b0 2024-04-25T10:07:21.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fdbfff640 1 --2- 172.21.15.60:0/3661481282 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fc4066f60 0x7f4fc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fec79e000 1 Processor -- start 2024-04-25T10:07:22.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fec79e000 1 -- start start 2024-04-25T10:07:22.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fdbfff640 1 --2- 172.21.15.60:0/3661481282 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fc4066f60 0x7f4fc4069420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4fe41f9930 tx=0x7f4fcc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fec79e000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8dc90 0x5580e8a8e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fec79e000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a8e5b0 0x5580e8a43d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fec79e000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a44290 0x5580e8a44670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fec79e000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e892af40 con 0x5580e8a8dc90 2024-04-25T10:07:22.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fec79e000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e8892070 con 0x5580e8a44290 2024-04-25T10:07:22.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fec79e000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5580e881a1a0 con 0x5580e8a8e5b0 2024-04-25T10:07:22.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fe2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8dc90 0x5580e8a8e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fe2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8dc90 0x5580e8a8e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41916/0 (socket says 172.21.15.60:41916) 2024-04-25T10:07:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fe2ffd640 1 -- 172.21.15.60:0/2629848644 learned_addr learned my addr 172.21.15.60:0/2629848644 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fdbfff640 1 --2- 172.21.15.60:0/2629848644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a8e5b0 0x5580e8a43d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fe37fe640 1 --2- 172.21.15.60:0/2629848644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a44290 0x5580e8a44670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fe2ffd640 1 -- 172.21.15.60:0/2629848644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a8e5b0 msgr2=0x5580e8a43d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fe2ffd640 1 --2- 172.21.15.60:0/2629848644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a8e5b0 0x5580e8a43d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fe2ffd640 1 -- 172.21.15.60:0/2629848644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a44290 msgr2=0x5580e8a44670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fe2ffd640 1 --2- 172.21.15.60:0/2629848644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a44290 0x5580e8a44670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fe2ffd640 1 -- 172.21.15.60:0/2629848644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580e8818510 con 0x5580e8a8dc90 2024-04-25T10:07:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.990+0000 7f4fe37fe640 1 --2- 172.21.15.60:0/2629848644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a44290 0x5580e8a44670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fe2ffd640 1 --2- 172.21.15.60:0/2629848644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8dc90 0x5580e8a8e070 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40828b0 tx=0x7f4fd40055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4be19ce064387 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fd8ff9640 1 -- 172.21.15.60:0/2629848644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd405b070 con 0x5580e8a8dc90 2024-04-25T10:07:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fd8ff9640 1 -- 172.21.15.60:0/2629848644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd405d380 con 0x5580e8a8dc90 2024-04-25T10:07:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fd8ff9640 1 -- 172.21.15.60:0/2629848644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40840d0 con 0x5580e8a8dc90 2024-04-25T10:07:22.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 -- 172.21.15.60:0/2629848644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8dc90 msgr2=0x5580e8a8e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 --2- 172.21.15.60:0/2629848644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8dc90 0x5580e8a8e070 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40828b0 tx=0x7f4fd40055d0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 -- 172.21.15.60:0/2629848644 shutdown_connections 2024-04-25T10:07:22.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 --2- 172.21.15.60:0/2629848644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a44290 0x5580e8a44670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 --2- 172.21.15.60:0/2629848644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a8e5b0 0x5580e8a43d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 --2- 172.21.15.60:0/2629848644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8dc90 0x5580e8a8e070 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 -- 172.21.15.60:0/2629848644 >> 172.21.15.60:0/2629848644 conn(0x5580e8a40240 msgr2=0x5580e8a42630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 -- 172.21.15.60:0/2629848644 shutdown_connections 2024-04-25T10:07:22.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 -- 172.21.15.60:0/2629848644 wait complete. 2024-04-25T10:07:22.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 -- start start 2024-04-25T10:07:22.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 Processor -- start 2024-04-25T10:07:22.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a44290 0x5580e8a4ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a8dc90 0x5580e8a4f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8e5b0 0x5580e8a4f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e884c6c0 con 0x5580e8a8e5b0 2024-04-25T10:07:22.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e888e110 con 0x5580e8a8dc90 2024-04-25T10:07:22.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5580e8948f20 con 0x5580e8a44290 2024-04-25T10:07:22.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 10 client.0 _ll_register_callbacks cb 0x5580e8a2f570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:22.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fe37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8e5b0 0x5580e8a4f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fe37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8e5b0 0x5580e8a4f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41926/0 (socket says 172.21.15.60:41926) 2024-04-25T10:07:22.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fe37fe640 1 -- 172.21.15.60:0/3569402781 learned_addr learned my addr 172.21.15.60:0/3569402781 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:22.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fdbfff640 1 --2- 172.21.15.60:0/3569402781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a8dc90 0x5580e8a4f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fe2ffd640 1 --2- 172.21.15.60:0/3569402781 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a44290 0x5580e8a4ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fe37fe640 1 -- 172.21.15.60:0/3569402781 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a44290 msgr2=0x5580e8a4ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fe37fe640 1 --2- 172.21.15.60:0/3569402781 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a44290 0x5580e8a4ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fe37fe640 1 -- 172.21.15.60:0/3569402781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a8dc90 msgr2=0x5580e8a4f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fe37fe640 1 --2- 172.21.15.60:0/3569402781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a8dc90 0x5580e8a4f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fe37fe640 1 -- 172.21.15.60:0/3569402781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580e881a3b0 con 0x5580e8a8e5b0 2024-04-25T10:07:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fe37fe640 1 --2- 172.21.15.60:0/3569402781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8e5b0 0x5580e8a4f8b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc01b0e0 tx=0x7f4fdc01b270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95c386da58b1310 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:07:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fc1ffb640 1 -- 172.21.15.60:0/3569402781 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0a5ac0 con 0x5580e8a8e5b0 2024-04-25T10:07:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fc1ffb640 1 -- 172.21.15.60:0/3569402781 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fdc0a5c60 con 0x5580e8a8e5b0 2024-04-25T10:07:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 -- 172.21.15.60:0/3569402781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fd405d380 con 0x5580e8a8e5b0 2024-04-25T10:07:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fc1ffb640 1 -- 172.21.15.60:0/3569402781 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fdc0c8050 con 0x5580e8a8e5b0 2024-04-25T10:07:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 10 client.6042 fetch_fsmap learned FSMap version 21 2024-04-25T10:07:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.994+0000 7f4fec79e000 1 -- 172.21.15.60:0/3569402781 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x5580e8913e00 con 0x5580e8a8e5b0 2024-04-25T10:07:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fc1ffb640 1 -- 172.21.15.60:0/3569402781 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd405d380 con 0x5580e8a8e5b0 2024-04-25T10:07:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fc1ffb640 1 -- 172.21.15.60:0/3569402781 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4fdc0cb020 con 0x5580e8a8e5b0 2024-04-25T10:07:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 10 client.6042 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T10:07:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 10 client.6042 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:07:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 20 client.6042 populate_metadata read hostname 'smithi060' 2024-04-25T10:07:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 1 --2- 172.21.15.60:0/3569402781 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x5580e8a50180 0x5580e8a9f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 4 client.6042 mds_command: new command op to 5796 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 1 -- 172.21.15.60:0/3569402781 --> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580e882dd60 con 0x5580e8a50180 2024-04-25T10:07:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fe2ffd640 1 --2- 172.21.15.60:0/3569402781 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x5580e8a50180 0x5580e8a9f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fe2ffd640 1 --2- 172.21.15.60:0/3569402781 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x5580e8a50180 0x5580e8a9f410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a11a02dee123077 server_cookie=ba53e4dd4a604cd8 in_seq=0 out_seq=0 2024-04-25T10:07:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fc1ffb640 10 client.6042 ms_handle_connect on v2:172.21.15.60:6833/1747757196 2024-04-25T10:07:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fc1ffb640 1 -- 172.21.15.60:0/3569402781 <== mds.0 v2:172.21.15.60:6833/1747757196 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5580e882dd60 con 0x5580e8a50180 2024-04-25T10:07:22.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fc1ffb640 10 client.6042 handle_command_reply: tid=0 2024-04-25T10:07:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 1 client.6042 shutdown 2024-04-25T10:07:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 1 -- 172.21.15.60:0/3569402781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8e5b0 msgr2=0x5580e8a4f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 1 --2- 172.21.15.60:0/3569402781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8e5b0 0x5580e8a4f8b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc01b0e0 tx=0x7f4fdc01b270 comp rx=0 tx=0).stop 2024-04-25T10:07:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 1 -- 172.21.15.60:0/3569402781 shutdown_connections 2024-04-25T10:07:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 1 --2- 172.21.15.60:0/3569402781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8e5b0 0x5580e8a4f8b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 1 --2- 172.21.15.60:0/3569402781 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x5580e8a50180 0x5580e8a9f410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 1 --2- 172.21.15.60:0/3569402781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a8dc90 0x5580e8a4f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fc1ffb640 0 client.6042 ms_handle_reset on v2:172.21.15.60:6833/1747757196 2024-04-25T10:07:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 1 --2- 172.21.15.60:0/3569402781 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a44290 0x5580e8a4ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 1 -- 172.21.15.60:0/3569402781 >> 172.21.15.60:0/3569402781 conn(0x5580e8a40240 msgr2=0x5580e8a42ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 1 -- 172.21.15.60:0/3569402781 shutdown_connections 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 1 -- 172.21.15.60:0/3569402781 wait complete. 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f4fec79e000 20 client.6042 trim_cache size 0 max 16384 2024-04-25T10:07:22.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f153e250640 1 Processor -- start 2024-04-25T10:07:22.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f153e250640 1 -- start start 2024-04-25T10:07:22.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f153e250640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f153815dd60 0x7f1538161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f153e250640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f15380ce710 0x7f153806c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f153e250640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f153806c680 0x7f153815e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f153e250640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15380d3840 con 0x7f153815dd60 2024-04-25T10:07:22.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f153e250640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1538162920 con 0x7f15380ce710 2024-04-25T10:07:22.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f153e250640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1538162080 con 0x7f153806c680 2024-04-25T10:07:22.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f15377fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f153815dd60 0x7f1538161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f15377fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f153815dd60 0x7f1538161a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41940/0 (socket says 172.21.15.60:41940) 2024-04-25T10:07:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f15377fe640 1 -- 172.21.15.60:0/1732003711 learned_addr learned my addr 172.21.15.60:0/1732003711 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f1537fff640 1 --2- 172.21.15.60:0/1732003711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f153806c680 0x7f153815e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f1536ffd640 1 --2- 172.21.15.60:0/1732003711 >> v2:172.21.15.100:3300/0 conn(0x7f15380ce710 0x7f153806c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f15377fe640 1 -- 172.21.15.60:0/1732003711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f153806c680 msgr2=0x7f153815e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f15377fe640 1 --2- 172.21.15.60:0/1732003711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f153806c680 0x7f153815e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f15377fe640 1 -- 172.21.15.60:0/1732003711 >> v2:172.21.15.100:3300/0 conn(0x7f15380ce710 msgr2=0x7f153806c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f15377fe640 1 --2- 172.21.15.60:0/1732003711 >> v2:172.21.15.100:3300/0 conn(0x7f15380ce710 0x7f153806c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f15377fe640 1 -- 172.21.15.60:0/1732003711 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1538162360 con 0x7f153815dd60 2024-04-25T10:07:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:21.998+0000 7f1537fff640 1 --2- 172.21.15.60:0/1732003711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f153806c680 0x7f153815e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.002+0000 7f15377fe640 1 --2- 172.21.15.60:0/1732003711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153815dd60 0x7f1538161a10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1528009870 tx=0x7f152801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3709483128a0ee90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.002+0000 7f15367fc640 1 -- 172.21.15.60:0/1732003711 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15280b0070 con 0x7f153815dd60 2024-04-25T10:07:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.002+0000 7f15367fc640 1 -- 172.21.15.60:0/1732003711 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15280ab600 con 0x7f153815dd60 2024-04-25T10:07:22.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.002+0000 7f15367fc640 1 -- 172.21.15.60:0/1732003711 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15280ab9d0 con 0x7f153815dd60 2024-04-25T10:07:22.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.002+0000 7f153e250640 1 -- 172.21.15.60:0/1732003711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153815dd60 msgr2=0x7f1538161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.002+0000 7f153e250640 1 --2- 172.21.15.60:0/1732003711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153815dd60 0x7f1538161a10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1528009870 tx=0x7f152801c490 comp rx=0 tx=0).stop 2024-04-25T10:07:22.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.002+0000 7f153e250640 1 -- 172.21.15.60:0/1732003711 shutdown_connections 2024-04-25T10:07:22.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.002+0000 7f153e250640 1 --2- 172.21.15.60:0/1732003711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f153806c680 0x7f153815e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.002+0000 7f153e250640 1 --2- 172.21.15.60:0/1732003711 >> v2:172.21.15.100:3300/0 conn(0x7f15380ce710 0x7f153806c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.002+0000 7f153e250640 1 --2- 172.21.15.60:0/1732003711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153815dd60 0x7f1538161a10 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.002+0000 7f153e250640 1 -- 172.21.15.60:0/1732003711 >> 172.21.15.60:0/1732003711 conn(0x7f15380d04e0 msgr2=0x7f15380d08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.002+0000 7f153e250640 1 -- 172.21.15.60:0/1732003711 shutdown_connections 2024-04-25T10:07:22.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.002+0000 7f153e250640 1 -- 172.21.15.60:0/1732003711 wait complete. 2024-04-25T10:07:22.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.002+0000 7f153e250640 1 Processor -- start 2024-04-25T10:07:22.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f153e250640 1 -- start start 2024-04-25T10:07:22.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f153e250640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153806c680 0x7f1538150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f153e250640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15380ce710 0x7f1538152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f153e250640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f153815dd60 0x7f1538158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f153e250640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153815f010 con 0x7f15380ce710 2024-04-25T10:07:22.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f153e250640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153815ee90 con 0x7f153806c680 2024-04-25T10:07:22.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f153e250640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153815f190 con 0x7f153815dd60 2024-04-25T10:07:22.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f1536ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15380ce710 0x7f1538152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f1536ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15380ce710 0x7f1538152b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41948/0 (socket says 172.21.15.60:41948) 2024-04-25T10:07:22.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f1536ffd640 1 -- 172.21.15.60:0/3366662654 learned_addr learned my addr 172.21.15.60:0/3366662654 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:22.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f15377fe640 1 --2- 172.21.15.60:0/3366662654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153806c680 0x7f1538150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f1537fff640 1 --2- 172.21.15.60:0/3366662654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f153815dd60 0x7f1538158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f1536ffd640 1 -- 172.21.15.60:0/3366662654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f153815dd60 msgr2=0x7f1538158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f1536ffd640 1 --2- 172.21.15.60:0/3366662654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f153815dd60 0x7f1538158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f1536ffd640 1 -- 172.21.15.60:0/3366662654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153806c680 msgr2=0x7f1538150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f1536ffd640 1 --2- 172.21.15.60:0/3366662654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153806c680 0x7f1538150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f1536ffd640 1 -- 172.21.15.60:0/3366662654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15381585d0 con 0x7f15380ce710 2024-04-25T10:07:22.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f15377fe640 1 --2- 172.21.15.60:0/3366662654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153806c680 0x7f1538150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:22.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f1536ffd640 1 --2- 172.21.15.60:0/3366662654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15380ce710 0x7f1538152b30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f152c00d5f0 tx=0x7f152c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f151ffff640 1 -- 172.21.15.60:0/3366662654 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c0125f0 con 0x7f15380ce710 2024-04-25T10:07:22.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f151ffff640 1 -- 172.21.15.60:0/3366662654 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152c019070 con 0x7f15380ce710 2024-04-25T10:07:22.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f153e250640 1 -- 172.21.15.60:0/3366662654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15381588b0 con 0x7f15380ce710 2024-04-25T10:07:22.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f151ffff640 1 -- 172.21.15.60:0/3366662654 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c0129b0 con 0x7f15380ce710 2024-04-25T10:07:22.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f153e250640 1 -- 172.21.15.60:0/3366662654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1538150920 con 0x7f15380ce710 2024-04-25T10:07:22.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f153e250640 1 -- 172.21.15.60:0/3366662654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f1538074770 con 0x7f15380ce710 2024-04-25T10:07:22.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f151ffff640 1 -- 172.21.15.60:0/3366662654 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f152c012b50 con 0x7f15380ce710 2024-04-25T10:07:22.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f151ffff640 1 --2- 172.21.15.60:0/3366662654 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1524066f60 0x7f1524069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f151ffff640 1 -- 172.21.15.60:0/3366662654 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f152c085ea0 con 0x7f15380ce710 2024-04-25T10:07:22.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.006+0000 7f151ffff640 1 -- 172.21.15.60:0/3366662654 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+501 (secure 0 0 0) 0x7f152c086350 con 0x7f15380ce710 2024-04-25T10:07:22.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.010+0000 7f15377fe640 1 --2- 172.21.15.60:0/3366662654 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1524066f60 0x7f1524069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.010+0000 7f153e250640 1 -- 172.21.15.60:0/3366662654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f153806cc70 con 0x7f15380ce710 2024-04-25T10:07:22.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.010+0000 7f15377fe640 1 --2- 172.21.15.60:0/3366662654 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1524066f60 0x7f1524069420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f15380b65d0 tx=0x7f1528009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.014+0000 7f4fec79e000 1 Processor -- start 2024-04-25T10:07:22.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.014+0000 7f4fec79e000 1 -- start start 2024-04-25T10:07:22.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.014+0000 7f4fec79e000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a44290 0x5580e8a43640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.014+0000 7f4fec79e000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a8dc90 0x5580e8a43b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.014+0000 7f4fec79e000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8e5b0 0x5580e8a9f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.014+0000 7f4fec79e000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4202b40 con 0x5580e8a8e5b0 2024-04-25T10:07:22.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.014+0000 7f4fec79e000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4201d40 con 0x5580e8a8dc90 2024-04-25T10:07:22.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.014+0000 7f4fec79e000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40ca090 con 0x5580e8a44290 2024-04-25T10:07:22.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.014+0000 7f4fe37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8e5b0 0x5580e8a9f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.014+0000 7f4fe37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8e5b0 0x5580e8a9f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41958/0 (socket says 172.21.15.60:41958) 2024-04-25T10:07:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.014+0000 7f4fe37fe640 1 -- 172.21.15.60:0/1391742575 learned_addr learned my addr 172.21.15.60:0/1391742575 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.014+0000 7f4fdbfff640 1 --2- 172.21.15.60:0/1391742575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a8dc90 0x5580e8a43b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.014+0000 7f4fe2ffd640 1 --2- 172.21.15.60:0/1391742575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a44290 0x5580e8a43640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.014+0000 7f4fe37fe640 1 -- 172.21.15.60:0/1391742575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a44290 msgr2=0x5580e8a43640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.014+0000 7f4fe37fe640 1 --2- 172.21.15.60:0/1391742575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a44290 0x5580e8a43640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fe37fe640 1 -- 172.21.15.60:0/1391742575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a8dc90 msgr2=0x5580e8a43b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fe37fe640 1 --2- 172.21.15.60:0/1391742575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a8dc90 0x5580e8a43b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fe37fe640 1 -- 172.21.15.60:0/1391742575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580e88e9c50 con 0x5580e8a8e5b0 2024-04-25T10:07:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fe2ffd640 1 --2- 172.21.15.60:0/1391742575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a44290 0x5580e8a43640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fe37fe640 1 --2- 172.21.15.60:0/1391742575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8e5b0 0x5580e8a9f410 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc002240 tx=0x7f4fdc0b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58fc803cee9daa82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fc3fff640 1 -- 172.21.15.60:0/1391742575 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc009510 con 0x5580e8a8e5b0 2024-04-25T10:07:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fc3fff640 1 -- 172.21.15.60:0/1391742575 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc0d0070 con 0x5580e8a8e5b0 2024-04-25T10:07:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fc3fff640 1 -- 172.21.15.60:0/1391742575 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0039a0 con 0x5580e8a8e5b0 2024-04-25T10:07:22.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 -- 172.21.15.60:0/1391742575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8e5b0 msgr2=0x5580e8a9f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 --2- 172.21.15.60:0/1391742575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8e5b0 0x5580e8a9f410 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc002240 tx=0x7f4fdc0b7000 comp rx=0 tx=0).stop 2024-04-25T10:07:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 -- 172.21.15.60:0/1391742575 shutdown_connections 2024-04-25T10:07:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 --2- 172.21.15.60:0/1391742575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8e5b0 0x5580e8a9f410 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 --2- 172.21.15.60:0/1391742575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a8dc90 0x5580e8a43b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 --2- 172.21.15.60:0/1391742575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a44290 0x5580e8a43640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 -- 172.21.15.60:0/1391742575 >> 172.21.15.60:0/1391742575 conn(0x5580e8a40240 msgr2=0x5580e8a48d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 -- 172.21.15.60:0/1391742575 shutdown_connections 2024-04-25T10:07:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 -- 172.21.15.60:0/1391742575 wait complete. 2024-04-25T10:07:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 -- start start 2024-04-25T10:07:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 Processor -- start 2024-04-25T10:07:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a44290 0x5580e8aa1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8dc90 0x5580e8aa9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a8e5b0 0x5580e8aaa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e8891380 con 0x5580e8a8dc90 2024-04-25T10:07:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40c7e50 con 0x5580e8a44290 2024-04-25T10:07:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5580e888e110 con 0x5580e8a8e5b0 2024-04-25T10:07:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 10 client.0 _ll_register_callbacks cb 0x7f4fd4054ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fdbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8dc90 0x5580e8aa9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fdbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8dc90 0x5580e8aa9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41968/0 (socket says 172.21.15.60:41968) 2024-04-25T10:07:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fdbfff640 1 -- 172.21.15.60:0/3315663973 learned_addr learned my addr 172.21.15.60:0/3315663973 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fe2ffd640 1 --2- 172.21.15.60:0/3315663973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a44290 0x5580e8aa1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fe37fe640 1 --2- 172.21.15.60:0/3315663973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a8e5b0 0x5580e8aaa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fdbfff640 1 -- 172.21.15.60:0/3315663973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a8e5b0 msgr2=0x5580e8aaa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fdbfff640 1 --2- 172.21.15.60:0/3315663973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a8e5b0 0x5580e8aaa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fdbfff640 1 -- 172.21.15.60:0/3315663973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a44290 msgr2=0x5580e8aa1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fdbfff640 1 --2- 172.21.15.60:0/3315663973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a44290 0x5580e8aa1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fdbfff640 1 -- 172.21.15.60:0/3315663973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580e885a120 con 0x5580e8a8dc90 2024-04-25T10:07:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fdbfff640 1 --2- 172.21.15.60:0/3315663973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8dc90 0x5580e8aa9ee0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc014710 tx=0x7f4fcc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35733c966b514121 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:07:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fc1ffb640 1 -- 172.21.15.60:0/3315663973 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc01b040 con 0x5580e8a8dc90 2024-04-25T10:07:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fc1ffb640 1 -- 172.21.15.60:0/3315663973 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fcc015b20 con 0x5580e8a8dc90 2024-04-25T10:07:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 -- 172.21.15.60:0/3315663973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fdc0d0070 con 0x5580e8a8dc90 2024-04-25T10:07:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fc1ffb640 1 -- 172.21.15.60:0/3315663973 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fcc034050 con 0x5580e8a8dc90 2024-04-25T10:07:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 10 client.6054 fetch_fsmap learned FSMap version 21 2024-04-25T10:07:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.018+0000 7f4fec79e000 1 -- 172.21.15.60:0/3315663973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x5580e8839170 con 0x5580e8a8dc90 2024-04-25T10:07:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fc1ffb640 1 -- 172.21.15.60:0/3315663973 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc0d0070 con 0x5580e8a8dc90 2024-04-25T10:07:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fc1ffb640 1 -- 172.21.15.60:0/3315663973 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4fcc021040 con 0x5580e8a8dc90 2024-04-25T10:07:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 10 client.6054 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T10:07:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 10 client.6054 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:07:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 20 client.6054 populate_metadata read hostname 'smithi060' 2024-04-25T10:07:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 1 --2- 172.21.15.60:0/3315663973 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x5580e8aedf70 0x5580e8aee380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 4 client.6054 mds_command: new command op to 5796 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:07:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 1 -- 172.21.15.60:0/3315663973 --> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x5580e889f5b0 con 0x5580e8aedf70 2024-04-25T10:07:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fe2ffd640 1 --2- 172.21.15.60:0/3315663973 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x5580e8aedf70 0x5580e8aee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fc1ffb640 10 client.6054 ms_handle_connect on v2:172.21.15.60:6833/1747757196 2024-04-25T10:07:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fe2ffd640 1 --2- 172.21.15.60:0/3315663973 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x5580e8aedf70 0x5580e8aee380 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69938d850613abfc server_cookie=c8e15056dd411cb1 in_seq=0 out_seq=0 2024-04-25T10:07:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fc1ffb640 1 -- 172.21.15.60:0/3315663973 <== mds.0 v2:172.21.15.60:6833/1747757196 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5580e889f5b0 con 0x5580e8aedf70 2024-04-25T10:07:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fc1ffb640 10 client.6054 handle_command_reply: tid=0 2024-04-25T10:07:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 1 client.6054 shutdown 2024-04-25T10:07:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 1 -- 172.21.15.60:0/3315663973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8dc90 msgr2=0x5580e8aa9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 1 --2- 172.21.15.60:0/3315663973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8dc90 0x5580e8aa9ee0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc014710 tx=0x7f4fcc004740 comp rx=0 tx=0).stop 2024-04-25T10:07:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 1 -- 172.21.15.60:0/3315663973 shutdown_connections 2024-04-25T10:07:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 1 --2- 172.21.15.60:0/3315663973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5580e8a8e5b0 0x5580e8aaa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 1 --2- 172.21.15.60:0/3315663973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5580e8a8dc90 0x5580e8aa9ee0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 1 --2- 172.21.15.60:0/3315663973 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x5580e8aedf70 0x5580e8aee380 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 1 --2- 172.21.15.60:0/3315663973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580e8a44290 0x5580e8aa1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 1 -- 172.21.15.60:0/3315663973 >> 172.21.15.60:0/3315663973 conn(0x5580e8a40240 msgr2=0x5580e8a49450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fc1ffb640 0 client.6054 ms_handle_reset on v2:172.21.15.60:6833/1747757196 2024-04-25T10:07:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 1 -- 172.21.15.60:0/3315663973 shutdown_connections 2024-04-25T10:07:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 1 -- 172.21.15.60:0/3315663973 wait complete. 2024-04-25T10:07:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fec79e000 20 client.6054 trim_cache size 0 max 16384 2024-04-25T10:07:22.038 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:07:22.039 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T10:07:22.039 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "9c4fd353-7131-4529-b64f-3d51bfb895d1", 2024-04-25T10:07:22.039 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T10:07:22.039 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:07:22.039 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f970cf53640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6000=0-4194304@1} caps={6000=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6000 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x5573ebd8c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:07:22.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fe9631640 1 -- 172.21.15.60:0/3661481282 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fc4066f60 msgr2=0x7f4fc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fe9631640 1 --2- 172.21.15.60:0/3661481282 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fc4066f60 0x7f4fc4069420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4fe41f9930 tx=0x7f4fcc008040 comp rx=0 tx=0).stop 2024-04-25T10:07:22.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fe9631640 1 -- 172.21.15.60:0/3661481282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe40b95b0 msgr2=0x7f4fe41f6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fe9631640 1 --2- 172.21.15.60:0/3661481282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe40b95b0 0x7f4fe41f6410 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400c390 tx=0x7f4fd400c850 comp rx=0 tx=0).stop 2024-04-25T10:07:22.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fe9631640 1 -- 172.21.15.60:0/3661481282 shutdown_connections 2024-04-25T10:07:22.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fe9631640 1 --2- 172.21.15.60:0/3661481282 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fc4066f60 0x7f4fc4069420 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fe9631640 1 --2- 172.21.15.60:0/3661481282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fe415da40 0x7f4fe415b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fe9631640 1 --2- 172.21.15.60:0/3661481282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fe40cec00 0x7f4fe40cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fe9631640 1 --2- 172.21.15.60:0/3661481282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe40b95b0 0x7f4fe41f6410 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fe9631640 1 -- 172.21.15.60:0/3661481282 >> 172.21.15.60:0/3661481282 conn(0x7f4fe40d0470 msgr2=0x7f4fe415c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fe9631640 1 -- 172.21.15.60:0/3661481282 shutdown_connections 2024-04-25T10:07:22.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.022+0000 7f4fe9631640 1 -- 172.21.15.60:0/3661481282 wait complete. 2024-04-25T10:07:22.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:22.048 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:07:22.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.158+0000 7fc214d67640 1 Processor -- start 2024-04-25T10:07:22.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.158+0000 7fc214d67640 1 -- start start 2024-04-25T10:07:22.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.158+0000 7fc214d67640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc21015dd30 0x7fc21015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.158+0000 7fc214d67640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc2100cd3d0 0x7fc2100cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.158+0000 7fc214d67640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2100cd890 con 0x7fc21006f310 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.158+0000 7fc214d67640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2100ce4a0 con 0x7fc21015dd30 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.158+0000 7fc214d67640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2100cdb70 con 0x7fc2100cd3d0 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.158+0000 7fc20e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc21015dd30 0x7fc21015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.158+0000 7fc20e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc21015dd30 0x7fc21015ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41970/0 (socket says 172.21.15.60:41970) 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.158+0000 7fc20e59c640 1 -- 172.21.15.60:0/3235805622 learned_addr learned my addr 172.21.15.60:0/3235805622 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.158+0000 7fc20dd9b640 1 --2- 172.21.15.60:0/3235805622 >> v2:172.21.15.100:3300/0 conn(0x7fc2100cd3d0 0x7fc2100cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.158+0000 7fc20e59c640 1 -- 172.21.15.60:0/3235805622 >> v2:172.21.15.100:3300/0 conn(0x7fc2100cd3d0 msgr2=0x7fc2100cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.158+0000 7fc20e59c640 1 --2- 172.21.15.60:0/3235805622 >> v2:172.21.15.100:3300/0 conn(0x7fc2100cd3d0 0x7fc2100cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.158+0000 7fc20e59c640 1 -- 172.21.15.60:0/3235805622 >> v1:172.21.15.100:6789/0 conn(0x7fc21006f310 legacy=0x7fc21006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.158+0000 7fc20e59c640 1 -- 172.21.15.60:0/3235805622 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2100cde50 con 0x7fc21015dd30 2024-04-25T10:07:22.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc20e59c640 1 --2- 172.21.15.60:0/3235805622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc21015dd30 0x7fc21015ac30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc210072550 tx=0x7fc20001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2cb1fd6402871b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc20d59a640 1 -- 172.21.15.60:0/3235805622 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000ab640 con 0x7fc21015dd30 2024-04-25T10:07:22.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc20d59a640 1 -- 172.21.15.60:0/3235805622 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2000b0070 con 0x7fc21015dd30 2024-04-25T10:07:22.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc20d59a640 1 -- 172.21.15.60:0/3235805622 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000aba00 con 0x7fc21015dd30 2024-04-25T10:07:22.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc214d67640 1 -- 172.21.15.60:0/3235805622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc21015dd30 msgr2=0x7fc21015ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc214d67640 1 --2- 172.21.15.60:0/3235805622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc21015dd30 0x7fc21015ac30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc210072550 tx=0x7fc20001c420 comp rx=0 tx=0).stop 2024-04-25T10:07:22.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc214d67640 1 -- 172.21.15.60:0/3235805622 shutdown_connections 2024-04-25T10:07:22.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc214d67640 1 --2- 172.21.15.60:0/3235805622 >> v2:172.21.15.100:3300/0 conn(0x7fc2100cd3d0 0x7fc2100cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc214d67640 1 --2- 172.21.15.60:0/3235805622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc21015dd30 0x7fc21015ac30 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc214d67640 1 -- 172.21.15.60:0/3235805622 >> 172.21.15.60:0/3235805622 conn(0x7fc2100670b0 msgr2=0x7fc21006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc214d67640 1 -- 172.21.15.60:0/3235805622 shutdown_connections 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc214d67640 1 -- 172.21.15.60:0/3235805622 wait complete. 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc214d67640 1 Processor -- start 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc214d67640 1 -- start start 2024-04-25T10:07:22.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc214d67640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc21006f310 0x7fc210141cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc214d67640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2100cd3d0 0x7fc2100cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc214d67640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc21015dd30 0x7fc210160b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc214d67640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2101674b0 con 0x7fc21015dd30 2024-04-25T10:07:22.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc214d67640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100cfae0 con 0x7fc2100cd3d0 2024-04-25T10:07:22.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.162+0000 7fc214d67640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc210167630 con 0x7fc21006f310 2024-04-25T10:07:22.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc20ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc21015dd30 0x7fc210160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc20ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc21015dd30 0x7fc210160b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41972/0 (socket says 172.21.15.60:41972) 2024-04-25T10:07:22.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc20ed9d640 1 -- 172.21.15.60:0/2802453132 learned_addr learned my addr 172.21.15.60:0/2802453132 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:22.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc20dd9b640 1 --2- 172.21.15.60:0/2802453132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2100cd3d0 0x7fc2100cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc20e59c640 1 --2- 172.21.15.60:0/2802453132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc21006f310 0x7fc210141cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc20ed9d640 1 -- 172.21.15.60:0/2802453132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc21006f310 msgr2=0x7fc210141cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc20ed9d640 1 --2- 172.21.15.60:0/2802453132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc21006f310 0x7fc210141cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc20ed9d640 1 -- 172.21.15.60:0/2802453132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2100cd3d0 msgr2=0x7fc2100cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc20ed9d640 1 --2- 172.21.15.60:0/2802453132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2100cd3d0 0x7fc2100cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc20ed9d640 1 -- 172.21.15.60:0/2802453132 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2101614f0 con 0x7fc21015dd30 2024-04-25T10:07:22.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc20e59c640 1 --2- 172.21.15.60:0/2802453132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc21006f310 0x7fc210141cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:22.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc20ed9d640 1 --2- 172.21.15.60:0/2802453132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc21015dd30 0x7fc210160b80 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc20800ba10 tx=0x7fc20800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc1feffd640 1 -- 172.21.15.60:0/2802453132 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208019070 con 0x7fc21015dd30 2024-04-25T10:07:22.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc1feffd640 1 -- 172.21.15.60:0/2802453132 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20800cb40 con 0x7fc21015dd30 2024-04-25T10:07:22.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc214d67640 1 -- 172.21.15.60:0/2802453132 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2101617d0 con 0x7fc21015dd30 2024-04-25T10:07:22.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc1feffd640 1 -- 172.21.15.60:0/2802453132 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208014440 con 0x7fc21015dd30 2024-04-25T10:07:22.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc214d67640 1 -- 172.21.15.60:0/2802453132 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc210161d90 con 0x7fc21015dd30 2024-04-25T10:07:22.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc1feffd640 1 -- 172.21.15.60:0/2802453132 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc20801c070 con 0x7fc21015dd30 2024-04-25T10:07:22.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc1feffd640 1 --2- 172.21.15.60:0/2802453132 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc1f0066f60 0x7fc1f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc1feffd640 1 -- 172.21.15.60:0/2802453132 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc208084d30 con 0x7fc21015dd30 2024-04-25T10:07:22.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc20e59c640 1 --2- 172.21.15.60:0/2802453132 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc1f0066f60 0x7fc1f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc217ed4000 1 Processor -- start 2024-04-25T10:07:22.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.166+0000 7fc217ed4000 1 -- start start 2024-04-25T10:07:22.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20e59c640 1 --2- 172.21.15.60:0/2802453132 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc1f0066f60 0x7fc1f0069420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc2000af000 tx=0x7fc2000a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533dbbe30 0x558533dbc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533dbc750 0x558533d71f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533d724b0 0x558533d72890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558533bbaff0 con 0x558533d724b0 2024-04-25T10:07:22.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558533b43150 con 0x558533dbc750 2024-04-25T10:07:22.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558533bb7090 con 0x558533dbbe30 2024-04-25T10:07:22.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533d724b0 0x558533d72890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533d724b0 0x558533d72890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41986/0 (socket says 172.21.15.60:41986) 2024-04-25T10:07:22.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20ed9d640 1 -- 172.21.15.60:0/1443791195 learned_addr learned my addr 172.21.15.60:0/1443791195 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:22.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20e59c640 1 --2- 172.21.15.60:0/1443791195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533dbbe30 0x558533dbc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20dd9b640 1 --2- 172.21.15.60:0/1443791195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533dbc750 0x558533d71f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20ed9d640 1 -- 172.21.15.60:0/1443791195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533dbbe30 msgr2=0x558533dbc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20ed9d640 1 --2- 172.21.15.60:0/1443791195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533dbbe30 0x558533dbc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20ed9d640 1 -- 172.21.15.60:0/1443791195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533dbc750 msgr2=0x558533d71f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20ed9d640 1 --2- 172.21.15.60:0/1443791195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533dbc750 0x558533d71f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20ed9d640 1 -- 172.21.15.60:0/1443791195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558533d510d0 con 0x558533d724b0 2024-04-25T10:07:22.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20dd9b640 1 --2- 172.21.15.60:0/1443791195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533dbc750 0x558533d71f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:22.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20ed9d640 1 --2- 172.21.15.60:0/1443791195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533d724b0 0x558533d72890 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc20805d780 tx=0x7fc208055d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de384b698ae07f13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc1effff640 1 -- 172.21.15.60:0/1443791195 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20805dcb0 con 0x558533d724b0 2024-04-25T10:07:22.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc1effff640 1 -- 172.21.15.60:0/1443791195 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2080538a0 con 0x558533d724b0 2024-04-25T10:07:22.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc1effff640 1 -- 172.21.15.60:0/1443791195 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20805d820 con 0x558533d724b0 2024-04-25T10:07:22.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 -- 172.21.15.60:0/1443791195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533d724b0 msgr2=0x558533d72890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 --2- 172.21.15.60:0/1443791195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533d724b0 0x558533d72890 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc20805d780 tx=0x7fc208055d60 comp rx=0 tx=0).stop 2024-04-25T10:07:22.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 -- 172.21.15.60:0/1443791195 shutdown_connections 2024-04-25T10:07:22.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 --2- 172.21.15.60:0/1443791195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533d724b0 0x558533d72890 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 --2- 172.21.15.60:0/1443791195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533dbc750 0x558533d71f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 --2- 172.21.15.60:0/1443791195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533dbbe30 0x558533dbc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 -- 172.21.15.60:0/1443791195 >> 172.21.15.60:0/1443791195 conn(0x558533d6a020 msgr2=0x558533d6c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 -- 172.21.15.60:0/1443791195 shutdown_connections 2024-04-25T10:07:22.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 -- 172.21.15.60:0/1443791195 wait complete. 2024-04-25T10:07:22.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 -- start start 2024-04-25T10:07:22.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 Processor -- start 2024-04-25T10:07:22.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533d724b0 0x558533dbb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533dbbe30 0x558533d78360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533dbc750 0x558533d788a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558533bbf620 con 0x558533dbbe30 2024-04-25T10:07:22.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558533b75690 con 0x558533d724b0 2024-04-25T10:07:22.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558533b43a40 con 0x558533dbc750 2024-04-25T10:07:22.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc217ed4000 10 client.0 _ll_register_callbacks cb 0x558533d58e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:22.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533dbbe30 0x558533d78360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533dbbe30 0x558533d78360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41992/0 (socket says 172.21.15.60:41992) 2024-04-25T10:07:22.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20dd9b640 1 -- 172.21.15.60:0/2748519547 learned_addr learned my addr 172.21.15.60:0/2748519547 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:22.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.170+0000 7fc20ed9d640 1 --2- 172.21.15.60:0/2748519547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533dbc750 0x558533d788a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc20e59c640 1 --2- 172.21.15.60:0/2748519547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533d724b0 0x558533dbb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc20dd9b640 1 -- 172.21.15.60:0/2748519547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533dbc750 msgr2=0x558533d788a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc20dd9b640 1 --2- 172.21.15.60:0/2748519547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533dbc750 0x558533d788a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc20dd9b640 1 -- 172.21.15.60:0/2748519547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533d724b0 msgr2=0x558533dbb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc20dd9b640 1 --2- 172.21.15.60:0/2748519547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533d724b0 0x558533dbb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc20dd9b640 1 -- 172.21.15.60:0/2748519547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558533c681c0 con 0x558533dbbe30 2024-04-25T10:07:22.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc20dd9b640 1 --2- 172.21.15.60:0/2748519547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533dbbe30 0x558533d78360 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc204010270 tx=0x7fc204010730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2469a30e94fa3657 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc1ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:07:22.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/2748519547 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20400f040 con 0x558533dbbe30 2024-04-25T10:07:22.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/2748519547 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc204014800 con 0x558533dbbe30 2024-04-25T10:07:22.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 1 -- 172.21.15.60:0/2748519547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2080538a0 con 0x558533dbbe30 2024-04-25T10:07:22.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/2748519547 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc204024050 con 0x558533dbbe30 2024-04-25T10:07:22.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 10 client.6066 fetch_fsmap learned FSMap version 21 2024-04-25T10:07:22.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 1 -- 172.21.15.60:0/2748519547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x558533c64170 con 0x558533dbbe30 2024-04-25T10:07:22.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/2748519547 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2080538a0 con 0x558533dbbe30 2024-04-25T10:07:22.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/2748519547 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc204027020 con 0x558533dbbe30 2024-04-25T10:07:22.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 10 client.6066 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T10:07:22.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 10 client.6066 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:07:22.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 20 client.6066 populate_metadata read hostname 'smithi060' 2024-04-25T10:07:22.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 1 --2- 172.21.15.60:0/2748519547 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x558533d79640 0x558533dce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 4 client.6066 mds_command: new command op to 5796 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:22.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 1 -- 172.21.15.60:0/2748519547 --> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558533b56c60 con 0x558533d79640 2024-04-25T10:07:22.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc20e59c640 1 --2- 172.21.15.60:0/2748519547 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x558533d79640 0x558533dce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc20e59c640 1 --2- 172.21.15.60:0/2748519547 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x558533d79640 0x558533dce0f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=221d3989d0e4a489 server_cookie=f2882b9ec9b2c3e0 in_seq=0 out_seq=0 2024-04-25T10:07:22.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc1ed7fa640 10 client.6066 ms_handle_connect on v2:172.21.15.60:6833/1747757196 2024-04-25T10:07:22.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/2748519547 <== mds.0 v2:172.21.15.60:6833/1747757196 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558533b56c60 con 0x558533d79640 2024-04-25T10:07:22.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc1ed7fa640 10 client.6066 handle_command_reply: tid=0 2024-04-25T10:07:22.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 1 client.6066 shutdown 2024-04-25T10:07:22.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 1 -- 172.21.15.60:0/2748519547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533dbbe30 msgr2=0x558533d78360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 1 --2- 172.21.15.60:0/2748519547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533dbbe30 0x558533d78360 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc204010270 tx=0x7fc204010730 comp rx=0 tx=0).stop 2024-04-25T10:07:22.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 1 -- 172.21.15.60:0/2748519547 shutdown_connections 2024-04-25T10:07:22.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 1 --2- 172.21.15.60:0/2748519547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533dbc750 0x558533d788a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 1 --2- 172.21.15.60:0/2748519547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533dbbe30 0x558533d78360 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 1 --2- 172.21.15.60:0/2748519547 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x558533d79640 0x558533dce0f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 1 --2- 172.21.15.60:0/2748519547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533d724b0 0x558533dbb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc1ed7fa640 0 client.6066 ms_handle_reset on v2:172.21.15.60:6833/1747757196 2024-04-25T10:07:22.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 1 -- 172.21.15.60:0/2748519547 >> 172.21.15.60:0/2748519547 conn(0x558533d6a020 msgr2=0x558533d73120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 1 -- 172.21.15.60:0/2748519547 shutdown_connections 2024-04-25T10:07:22.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 1 -- 172.21.15.60:0/2748519547 wait complete. 2024-04-25T10:07:22.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.174+0000 7fc217ed4000 20 client.6066 trim_cache size 0 max 16384 2024-04-25T10:07:22.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc217ed4000 1 Processor -- start 2024-04-25T10:07:22.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc217ed4000 1 -- start start 2024-04-25T10:07:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc217ed4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533d6a020 0x558533dd0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc217ed4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533d724b0 0x558533dd0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc217ed4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533dbbe30 0x558533dc92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc217ed4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2101feba0 con 0x558533d6a020 2024-04-25T10:07:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc217ed4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100b38e0 con 0x558533d724b0 2024-04-25T10:07:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc217ed4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc210163340 con 0x558533dbbe30 2024-04-25T10:07:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc20e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533d6a020 0x558533dd0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc20e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533d6a020 0x558533dd0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42006/0 (socket says 172.21.15.60:42006) 2024-04-25T10:07:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc20e59c640 1 -- 172.21.15.60:0/674800685 learned_addr learned my addr 172.21.15.60:0/674800685 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc20e59c640 1 -- 172.21.15.60:0/674800685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533dbbe30 msgr2=0x558533dc92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc20e59c640 1 --2- 172.21.15.60:0/674800685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533dbbe30 0x558533dc92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc20e59c640 1 -- 172.21.15.60:0/674800685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533d724b0 msgr2=0x558533dd0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc20dd9b640 1 --2- 172.21.15.60:0/674800685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533d724b0 0x558533dd0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc20e59c640 1 --2- 172.21.15.60:0/674800685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533d724b0 0x558533dd0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc20e59c640 1 -- 172.21.15.60:0/674800685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558533c01de0 con 0x558533d6a020 2024-04-25T10:07:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc20e59c640 1 --2- 172.21.15.60:0/674800685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533d6a020 0x558533dd0110 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc20001c460 tx=0x7fc200009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508c1d757b94fdaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc1ef7fe640 1 -- 172.21.15.60:0/674800685 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000ba070 con 0x558533d6a020 2024-04-25T10:07:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc1ef7fe640 1 -- 172.21.15.60:0/674800685 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2000ce070 con 0x558533d6a020 2024-04-25T10:07:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc1ef7fe640 1 -- 172.21.15.60:0/674800685 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000be400 con 0x558533d6a020 2024-04-25T10:07:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc217ed4000 1 -- 172.21.15.60:0/674800685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533d6a020 msgr2=0x558533dd0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc217ed4000 1 --2- 172.21.15.60:0/674800685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533d6a020 0x558533dd0110 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc20001c460 tx=0x7fc200009240 comp rx=0 tx=0).stop 2024-04-25T10:07:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc217ed4000 1 -- 172.21.15.60:0/674800685 shutdown_connections 2024-04-25T10:07:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc217ed4000 1 --2- 172.21.15.60:0/674800685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533dbbe30 0x558533dc92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc217ed4000 1 --2- 172.21.15.60:0/674800685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533d724b0 0x558533dd0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.190+0000 7fc217ed4000 1 --2- 172.21.15.60:0/674800685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533d6a020 0x558533dd0110 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 -- 172.21.15.60:0/674800685 >> 172.21.15.60:0/674800685 conn(0x558533d69b00 msgr2=0x558533dc8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 -- 172.21.15.60:0/674800685 shutdown_connections 2024-04-25T10:07:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 -- 172.21.15.60:0/674800685 wait complete. 2024-04-25T10:07:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 -- start start 2024-04-25T10:07:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 Processor -- start 2024-04-25T10:07:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533d6a020 0x558533dce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533d724b0 0x558533dce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533dbbe30 0x558533dced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558533b55e00 con 0x558533dbbe30 2024-04-25T10:07:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2101606f0 con 0x558533d724b0 2024-04-25T10:07:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558533bba300 con 0x558533d6a020 2024-04-25T10:07:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 10 client.0 _ll_register_callbacks cb 0x558533d39040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc20ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533dbbe30 0x558533dced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc20ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533dbbe30 0x558533dced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42010/0 (socket says 172.21.15.60:42010) 2024-04-25T10:07:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc20ed9d640 1 -- 172.21.15.60:0/3067421868 learned_addr learned my addr 172.21.15.60:0/3067421868 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc20e59c640 1 --2- 172.21.15.60:0/3067421868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533d6a020 0x558533dce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc20dd9b640 1 --2- 172.21.15.60:0/3067421868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533d724b0 0x558533dce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc20ed9d640 1 -- 172.21.15.60:0/3067421868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533d6a020 msgr2=0x558533dce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc20ed9d640 1 --2- 172.21.15.60:0/3067421868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533d6a020 0x558533dce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc20ed9d640 1 -- 172.21.15.60:0/3067421868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533d724b0 msgr2=0x558533dce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc20ed9d640 1 --2- 172.21.15.60:0/3067421868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533d724b0 0x558533dce810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc20ed9d640 1 -- 172.21.15.60:0/3067421868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558533b5c8e0 con 0x558533dbbe30 2024-04-25T10:07:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc20ed9d640 1 --2- 172.21.15.60:0/3067421868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533dbbe30 0x558533dced50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc208051a00 tx=0x7fc208083090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a833df2d2b75d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc1ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:07:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/3067421868 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20805da20 con 0x558533dbbe30 2024-04-25T10:07:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/3067421868 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2080549f0 con 0x558533dbbe30 2024-04-25T10:07:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 -- 172.21.15.60:0/3067421868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2000ce070 con 0x558533dbbe30 2024-04-25T10:07:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/3067421868 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc20805dcb0 con 0x558533dbbe30 2024-04-25T10:07:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 10 client.6072 fetch_fsmap learned FSMap version 21 2024-04-25T10:07:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 -- 172.21.15.60:0/3067421868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x558533c148f0 con 0x558533dbbe30 2024-04-25T10:07:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/3067421868 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2000ce070 con 0x558533dbbe30 2024-04-25T10:07:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/3067421868 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc20807cb20 con 0x558533dbbe30 2024-04-25T10:07:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 10 client.6072 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T10:07:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 10 client.6072 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:07:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 20 client.6072 populate_metadata read hostname 'smithi060' 2024-04-25T10:07:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 --2- 172.21.15.60:0/3067421868 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x558533dbc750 0x558533e23fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 4 client.6072 mds_command: new command op to 5796 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:07:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 -- 172.21.15.60:0/3067421868 --> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558533bc8530 con 0x558533dbc750 2024-04-25T10:07:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc20e59c640 1 --2- 172.21.15.60:0/3067421868 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x558533dbc750 0x558533e23fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc20e59c640 1 --2- 172.21.15.60:0/3067421868 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x558533dbc750 0x558533e23fe0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d009bf131fb0e4e server_cookie=6c992b5ac5778f4c in_seq=0 out_seq=0 2024-04-25T10:07:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc1ed7fa640 10 client.6072 ms_handle_connect on v2:172.21.15.60:6833/1747757196 2024-04-25T10:07:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/3067421868 <== mds.0 v2:172.21.15.60:6833/1747757196 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x558533bc8530 con 0x558533dbc750 2024-04-25T10:07:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc1ed7fa640 10 client.6072 handle_command_reply: tid=0 2024-04-25T10:07:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 client.6072 shutdown 2024-04-25T10:07:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 -- 172.21.15.60:0/3067421868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533dbbe30 msgr2=0x558533dced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 --2- 172.21.15.60:0/3067421868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533dbbe30 0x558533dced50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc208051a00 tx=0x7fc208083090 comp rx=0 tx=0).stop 2024-04-25T10:07:22.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 -- 172.21.15.60:0/3067421868 shutdown_connections 2024-04-25T10:07:22.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 --2- 172.21.15.60:0/3067421868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558533dbbe30 0x558533dced50 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 --2- 172.21.15.60:0/3067421868 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x558533dbc750 0x558533e23fe0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 --2- 172.21.15.60:0/3067421868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558533d724b0 0x558533dce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 --2- 172.21.15.60:0/3067421868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558533d6a020 0x558533dce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc1ed7fa640 0 client.6072 ms_handle_reset on v2:172.21.15.60:6833/1747757196 2024-04-25T10:07:22.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 -- 172.21.15.60:0/3067421868 >> 172.21.15.60:0/3067421868 conn(0x558533d69b00 msgr2=0x558533dc9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 -- 172.21.15.60:0/3067421868 shutdown_connections 2024-04-25T10:07:22.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 1 -- 172.21.15.60:0/3067421868 wait complete. 2024-04-25T10:07:22.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.194+0000 7fc217ed4000 20 client.6072 trim_cache size 0 max 16384 2024-04-25T10:07:22.209 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:07:22.209 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:07:22.210 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T10:07:22.210 INFO:teuthology.orchestra.run.smithi060.stdout: "9c4fd353-7131-4529-b64f-3d51bfb895d1": { 2024-04-25T10:07:22.210 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-04-25T10:07:22.210 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "9c4fd353-7131-4529-b64f-3d51bfb895d1", 2024-04-25T10:07:22.210 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,repair" 2024-04-25T10:07:22.210 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:22.210 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:22.210 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:07:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.198+0000 7fc214d67640 1 -- 172.21.15.60:0/2802453132 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc1f0066f60 msgr2=0x7fc1f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.198+0000 7fc214d67640 1 --2- 172.21.15.60:0/2802453132 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc1f0066f60 0x7fc1f0069420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc2000af000 tx=0x7fc2000a7000 comp rx=0 tx=0).stop 2024-04-25T10:07:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.198+0000 7fc214d67640 1 -- 172.21.15.60:0/2802453132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc21015dd30 msgr2=0x7fc210160b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.198+0000 7fc214d67640 1 --2- 172.21.15.60:0/2802453132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc21015dd30 0x7fc210160b80 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc20800ba10 tx=0x7fc20800bed0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.198+0000 7fc214d67640 1 -- 172.21.15.60:0/2802453132 shutdown_connections 2024-04-25T10:07:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.198+0000 7fc214d67640 1 --2- 172.21.15.60:0/2802453132 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc1f0066f60 0x7fc1f0069420 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.198+0000 7fc214d67640 1 --2- 172.21.15.60:0/2802453132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc21015dd30 0x7fc210160b80 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.198+0000 7fc214d67640 1 --2- 172.21.15.60:0/2802453132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2100cd3d0 0x7fc2100cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.198+0000 7fc214d67640 1 --2- 172.21.15.60:0/2802453132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc21006f310 0x7fc210141cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.198+0000 7fc214d67640 1 -- 172.21.15.60:0/2802453132 >> 172.21.15.60:0/2802453132 conn(0x7fc2100670b0 msgr2=0x7fc21015ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.198+0000 7fc214d67640 1 -- 172.21.15.60:0/2802453132 shutdown_connections 2024-04-25T10:07:22.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.198+0000 7fc214d67640 1 -- 172.21.15.60:0/2802453132 wait complete. 2024-04-25T10:07:22.220 INFO:tasks.cephfs.filesystem:scrub status for tag:9c4fd353-7131-4529-b64f-3d51bfb895d1 - {'path': '/', 'tag': '9c4fd353-7131-4529-b64f-3d51bfb895d1', 'options': 'recursive,repair'} 2024-04-25T10:07:22.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:22.670+0000 7f151ffff640 1 -- 172.21.15.60:0/3366662654 <== mon.0 v2:172.21.15.60:3300/0 7 ==== log(6 entries from seq 1 at 2024-04-25T10:07:22.024813+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f152c0844e0 con 0x7f15380ce710 2024-04-25T10:07:22.678 INFO:teuthology.orchestra.run.smithi060.stdout: cluster: 2024-04-25T10:07:22.678 INFO:teuthology.orchestra.run.smithi060.stdout: id: 0ea93863-a079-45c4-abed-6e83540ea9fe 2024-04-25T10:07:22.678 INFO:teuthology.orchestra.run.smithi060.stdout: health: HEALTH_OK 2024-04-25T10:07:22.678 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:22.678 INFO:teuthology.orchestra.run.smithi060.stdout: services: 2024-04-25T10:07:22.678 INFO:teuthology.orchestra.run.smithi060.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-04-25T10:07:22.678 INFO:teuthology.orchestra.run.smithi060.stdout: mgr: x(active, since 2m), standbys: y 2024-04-25T10:07:22.678 INFO:teuthology.orchestra.run.smithi060.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:07:22.678 INFO:teuthology.orchestra.run.smithi060.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-04-25T10:07:22.678 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:22.678 INFO:teuthology.orchestra.run.smithi060.stdout: data: 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi060.stdout: volumes: 1/1 healthy 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi060.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi060.stdout: objects: 24 objects, 579 KiB 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi060.stdout: usage: 249 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi060.stdout: pgs: 97 active+clean 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi060.stdout: io: 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi060.stdout: client: 1.3 KiB/s wr, 0 op/s rd, 4 op/s wr 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:07:22.024813+0000 mds.a [INF] scrub queued for path: / 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:07:22.024817+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:07:22.024821+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:07:22.026300+0000 mds.a [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:07:22.026312+0000 mds.a [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-04-25T10:07:22.680 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:07:22.026351+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:07:23.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:23.678+0000 7f151ffff640 1 -- 172.21.15.60:0/3366662654 <== mon.0 v2:172.21.15.60:3300/0 8 ==== log(1 entries from seq 7 at 2024-04-25T10:07:22.637257+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f152c012e40 con 0x7f15380ce710 2024-04-25T10:07:23.686 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:07:22.637257+0000 mds.a [INF] scrub summary: idle 2024-04-25T10:07:46.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:46.483 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:52.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:52.221 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:07:52.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.350+0000 7ff63648d640 1 Processor -- start 2024-04-25T10:07:52.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.350+0000 7ff63648d640 1 -- start start 2024-04-25T10:07:52.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff63648d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff630160080 0x7ff63015f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff63648d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff63015fa20 0x7ff6300ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff63648d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff630144a40 con 0x7ff6300cbcf0 2024-04-25T10:07:52.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff63648d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63015fe90 con 0x7ff630160080 2024-04-25T10:07:52.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff63648d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6300b2dc0 con 0x7ff63015fa20 2024-04-25T10:07:52.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff62f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff630160080 0x7ff63015f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff62f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff630160080 0x7ff63015f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50918/0 (socket says 172.21.15.60:50918) 2024-04-25T10:07:52.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff62f7fe640 1 -- 172.21.15.60:0/2497810402 learned_addr learned my addr 172.21.15.60:0/2497810402 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:52.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff634a2a640 1 --2- 172.21.15.60:0/2497810402 >> v2:172.21.15.100:3300/0 conn(0x7ff63015fa20 0x7ff6300ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff62f7fe640 1 -- 172.21.15.60:0/2497810402 >> v2:172.21.15.100:3300/0 conn(0x7ff63015fa20 msgr2=0x7ff6300ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff62f7fe640 1 --2- 172.21.15.60:0/2497810402 >> v2:172.21.15.100:3300/0 conn(0x7ff63015fa20 0x7ff6300ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff62f7fe640 1 -- 172.21.15.60:0/2497810402 >> v1:172.21.15.100:6789/0 conn(0x7ff6300cbcf0 legacy=0x7ff630071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff62f7fe640 1 -- 172.21.15.60:0/2497810402 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6300cc9f0 con 0x7ff630160080 2024-04-25T10:07:52.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff634a2a640 1 --2- 172.21.15.60:0/2497810402 >> v2:172.21.15.100:3300/0 conn(0x7ff63015fa20 0x7ff6300ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:52.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff62f7fe640 1 --2- 172.21.15.60:0/2497810402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff630160080 0x7ff63015f4e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff6300d03a0 tx=0x7ff62001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c9d12ddc6260eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff62effd640 1 -- 172.21.15.60:0/2497810402 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6200b1070 con 0x7ff630160080 2024-04-25T10:07:52.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff62effd640 1 -- 172.21.15.60:0/2497810402 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6200ac630 con 0x7ff630160080 2024-04-25T10:07:52.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.354+0000 7ff62effd640 1 -- 172.21.15.60:0/2497810402 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6200a8050 con 0x7ff630160080 2024-04-25T10:07:52.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 -- 172.21.15.60:0/2497810402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff630160080 msgr2=0x7ff63015f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 --2- 172.21.15.60:0/2497810402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff630160080 0x7ff63015f4e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff6300d03a0 tx=0x7ff62001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 -- 172.21.15.60:0/2497810402 shutdown_connections 2024-04-25T10:07:52.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 --2- 172.21.15.60:0/2497810402 >> v2:172.21.15.100:3300/0 conn(0x7ff63015fa20 0x7ff6300ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 --2- 172.21.15.60:0/2497810402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff630160080 0x7ff63015f4e0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 -- 172.21.15.60:0/2497810402 >> 172.21.15.60:0/2497810402 conn(0x7ff6300670b0 msgr2=0x7ff63006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:52.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 -- 172.21.15.60:0/2497810402 shutdown_connections 2024-04-25T10:07:52.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 -- 172.21.15.60:0/2497810402 wait complete. 2024-04-25T10:07:52.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 Processor -- start 2024-04-25T10:07:52.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 -- start start 2024-04-25T10:07:52.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6300cbcf0 0x7ff63015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff63015fa20 0x7ff6300afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff630160080 0x7ff63013d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6300ce5c0 con 0x7ff63015fa20 2024-04-25T10:07:52.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6300cc110 con 0x7ff630160080 2024-04-25T10:07:52.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6300ce740 con 0x7ff6300cbcf0 2024-04-25T10:07:52.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff62f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff63015fa20 0x7ff6300afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff62f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff63015fa20 0x7ff6300afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50930/0 (socket says 172.21.15.60:50930) 2024-04-25T10:07:52.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff62f7fe640 1 -- 172.21.15.60:0/3701293679 learned_addr learned my addr 172.21.15.60:0/3701293679 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:52.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff634a2a640 1 --2- 172.21.15.60:0/3701293679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff630160080 0x7ff63013d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff62f7fe640 1 -- 172.21.15.60:0/3701293679 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6300cbcf0 msgr2=0x7ff63015ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff62f7fe640 1 --2- 172.21.15.60:0/3701293679 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6300cbcf0 0x7ff63015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff62f7fe640 1 -- 172.21.15.60:0/3701293679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff630160080 msgr2=0x7ff63013d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff62f7fe640 1 --2- 172.21.15.60:0/3701293679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff630160080 0x7ff63013d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff62f7fe640 1 -- 172.21.15.60:0/3701293679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6300b0000 con 0x7ff63015fa20 2024-04-25T10:07:52.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff634a2a640 1 --2- 172.21.15.60:0/3701293679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff630160080 0x7ff63013d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:52.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff62f7fe640 1 --2- 172.21.15.60:0/3701293679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff63015fa20 0x7ff6300afac0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff6200af040 tx=0x7ff620019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff62cff9640 1 -- 172.21.15.60:0/3701293679 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6200a8050 con 0x7ff63015fa20 2024-04-25T10:07:52.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff62cff9640 1 -- 172.21.15.60:0/3701293679 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6200bc070 con 0x7ff63015fa20 2024-04-25T10:07:52.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 -- 172.21.15.60:0/3701293679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63013ba00 con 0x7ff63015fa20 2024-04-25T10:07:52.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff62cff9640 1 -- 172.21.15.60:0/3701293679 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6200b1040 con 0x7ff63015fa20 2024-04-25T10:07:52.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.358+0000 7ff63648d640 1 -- 172.21.15.60:0/3701293679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63013bfc0 con 0x7ff63015fa20 2024-04-25T10:07:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff62cff9640 1 -- 172.21.15.60:0/3701293679 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff6200ac4f0 con 0x7ff63015fa20 2024-04-25T10:07:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff62cff9640 1 --2- 172.21.15.60:0/3701293679 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff610066f60 0x7ff610069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff62cff9640 1 -- 172.21.15.60:0/3701293679 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff62011f550 con 0x7ff63015fa20 2024-04-25T10:07:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff62ffff640 1 --2- 172.21.15.60:0/3701293679 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff610066f60 0x7ff610069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff6395fa000 1 Processor -- start 2024-04-25T10:07:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff6395fa000 1 -- start start 2024-04-25T10:07:52.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff62ffff640 1 --2- 172.21.15.60:0/3701293679 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff610066f60 0x7ff610069420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff63015bb20 tx=0x7ff618006bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff6395fa000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cf9d5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff6395fa000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf9d62f0 0x5584cf98bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff6395fa000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 0x5584cf98c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff6395fa000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584cf7d8ff0 con 0x5584cf9d59d0 2024-04-25T10:07:52.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff6395fa000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584cf761150 con 0x5584cf98c0a0 2024-04-25T10:07:52.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff6395fa000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584cf7d5090 con 0x5584cf9d62f0 2024-04-25T10:07:52.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff62ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cf9d5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff62ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cf9d5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50944/0 (socket says 172.21.15.60:50944) 2024-04-25T10:07:52.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff62ffff640 1 -- 172.21.15.60:0/1857631238 learned_addr learned my addr 172.21.15.60:0/1857631238 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:52.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff62f7fe640 1 --2- 172.21.15.60:0/1857631238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf9d62f0 0x5584cf98bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff634a2a640 1 --2- 172.21.15.60:0/1857631238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 0x5584cf98c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff62ffff640 1 -- 172.21.15.60:0/1857631238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf9d62f0 msgr2=0x5584cf98bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff62ffff640 1 --2- 172.21.15.60:0/1857631238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf9d62f0 0x5584cf98bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff62ffff640 1 -- 172.21.15.60:0/1857631238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 msgr2=0x5584cf98c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff62ffff640 1 --2- 172.21.15.60:0/1857631238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 0x5584cf98c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff62ffff640 1 -- 172.21.15.60:0/1857631238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584cf96f0d0 con 0x5584cf9d59d0 2024-04-25T10:07:52.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff634a2a640 1 --2- 172.21.15.60:0/1857631238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 0x5584cf98c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:52.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff62ffff640 1 --2- 172.21.15.60:0/1857631238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cf9d5db0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff6180138c0 tx=0x7ff61800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8ed50c82505144 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff625ffb640 1 -- 172.21.15.60:0/1857631238 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618012ca0 con 0x5584cf9d59d0 2024-04-25T10:07:52.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff625ffb640 1 -- 172.21.15.60:0/1857631238 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff618022070 con 0x5584cf9d59d0 2024-04-25T10:07:52.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.362+0000 7ff625ffb640 1 -- 172.21.15.60:0/1857631238 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618012e60 con 0x5584cf9d59d0 2024-04-25T10:07:52.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 -- 172.21.15.60:0/1857631238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 msgr2=0x5584cf9d5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 --2- 172.21.15.60:0/1857631238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cf9d5db0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff6180138c0 tx=0x7ff61800e000 comp rx=0 tx=0).stop 2024-04-25T10:07:52.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 -- 172.21.15.60:0/1857631238 shutdown_connections 2024-04-25T10:07:52.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 --2- 172.21.15.60:0/1857631238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 0x5584cf98c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 --2- 172.21.15.60:0/1857631238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf9d62f0 0x5584cf98bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 --2- 172.21.15.60:0/1857631238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cf9d5db0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 -- 172.21.15.60:0/1857631238 >> 172.21.15.60:0/1857631238 conn(0x5584cf988020 msgr2=0x5584cf98a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:52.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 -- 172.21.15.60:0/1857631238 shutdown_connections 2024-04-25T10:07:52.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 -- 172.21.15.60:0/1857631238 wait complete. 2024-04-25T10:07:52.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 -- start start 2024-04-25T10:07:52.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 Processor -- start 2024-04-25T10:07:52.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 0x5584cf9ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf9d59d0 0x5584cf9bad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d62f0 0x5584cf9bb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584cf79d250 con 0x5584cf9d62f0 2024-04-25T10:07:52.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584cf793690 con 0x5584cf98c0a0 2024-04-25T10:07:52.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584cf761a40 con 0x5584cf9d59d0 2024-04-25T10:07:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 10 client.0 _ll_register_callbacks cb 0x5584cf976e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff634a2a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d62f0 0x5584cf9bb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff634a2a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d62f0 0x5584cf9bb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50946/0 (socket says 172.21.15.60:50946) 2024-04-25T10:07:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff634a2a640 1 -- 172.21.15.60:0/3430256821 learned_addr learned my addr 172.21.15.60:0/3430256821 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff62f7fe640 1 --2- 172.21.15.60:0/3430256821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf9d59d0 0x5584cf9bad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff62ffff640 1 --2- 172.21.15.60:0/3430256821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 0x5584cf9ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff634a2a640 1 -- 172.21.15.60:0/3430256821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf9d59d0 msgr2=0x5584cf9bad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff634a2a640 1 --2- 172.21.15.60:0/3430256821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf9d59d0 0x5584cf9bad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff634a2a640 1 -- 172.21.15.60:0/3430256821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 msgr2=0x5584cf9ba7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff634a2a640 1 --2- 172.21.15.60:0/3430256821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 0x5584cf9ba7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff634a2a640 1 -- 172.21.15.60:0/3430256821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584cf8861c0 con 0x5584cf9d62f0 2024-04-25T10:07:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff634a2a640 1 --2- 172.21.15.60:0/3430256821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d62f0 0x5584cf9bb270 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff62800cb30 tx=0x7ff628009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a876c135f735359a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff60effd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:07:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff60effd640 1 -- 172.21.15.60:0/3430256821 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62801a060 con 0x5584cf9d62f0 2024-04-25T10:07:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff60effd640 1 -- 172.21.15.60:0/3430256821 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff628009bc0 con 0x5584cf9d62f0 2024-04-25T10:07:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 -- 172.21.15.60:0/3430256821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff618022070 con 0x5584cf9d62f0 2024-04-25T10:07:52.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff60effd640 1 -- 172.21.15.60:0/3430256821 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff628020050 con 0x5584cf9d62f0 2024-04-25T10:07:52.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 10 client.6084 fetch_fsmap learned FSMap version 21 2024-04-25T10:07:52.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 -- 172.21.15.60:0/3430256821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x5584cf882170 con 0x5584cf9d62f0 2024-04-25T10:07:52.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff60effd640 1 -- 172.21.15.60:0/3430256821 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff618022070 con 0x5584cf9d62f0 2024-04-25T10:07:52.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff60effd640 1 -- 172.21.15.60:0/3430256821 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff628023020 con 0x5584cf9d62f0 2024-04-25T10:07:52.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 10 client.6084 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T10:07:52.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 10 client.6084 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:07:52.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 20 client.6084 populate_metadata read hostname 'smithi060' 2024-04-25T10:07:52.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 --2- 172.21.15.60:0/3430256821 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x5584cf9e5c80 0x5584cf9e8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 4 client.6084 mds_command: new command op to 5796 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:52.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff6395fa000 1 -- 172.21.15.60:0/3430256821 --> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584cf774c60 con 0x5584cf9e5c80 2024-04-25T10:07:52.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.366+0000 7ff62ffff640 1 --2- 172.21.15.60:0/3430256821 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x5584cf9e5c80 0x5584cf9e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff60effd640 10 client.6084 ms_handle_connect on v2:172.21.15.60:6833/1747757196 2024-04-25T10:07:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff62ffff640 1 --2- 172.21.15.60:0/3430256821 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x5584cf9e5c80 0x5584cf9e8070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed3041fec1bc147f server_cookie=6881f3fa356b27b5 in_seq=0 out_seq=0 2024-04-25T10:07:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff60effd640 1 -- 172.21.15.60:0/3430256821 <== mds.0 v2:172.21.15.60:6833/1747757196 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5584cf774c60 con 0x5584cf9e5c80 2024-04-25T10:07:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff60effd640 10 client.6084 handle_command_reply: tid=0 2024-04-25T10:07:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff6395fa000 1 client.6084 shutdown 2024-04-25T10:07:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff6395fa000 1 -- 172.21.15.60:0/3430256821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d62f0 msgr2=0x5584cf9bb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff6395fa000 1 --2- 172.21.15.60:0/3430256821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d62f0 0x5584cf9bb270 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff62800cb30 tx=0x7ff628009390 comp rx=0 tx=0).stop 2024-04-25T10:07:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff6395fa000 1 -- 172.21.15.60:0/3430256821 shutdown_connections 2024-04-25T10:07:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff6395fa000 1 --2- 172.21.15.60:0/3430256821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d62f0 0x5584cf9bb270 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff6395fa000 1 --2- 172.21.15.60:0/3430256821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf9d59d0 0x5584cf9bad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff6395fa000 1 --2- 172.21.15.60:0/3430256821 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x5584cf9e5c80 0x5584cf9e8070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff6395fa000 1 --2- 172.21.15.60:0/3430256821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 0x5584cf9ba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff60effd640 0 client.6084 ms_handle_reset on v2:172.21.15.60:6833/1747757196 2024-04-25T10:07:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff6395fa000 1 -- 172.21.15.60:0/3430256821 >> 172.21.15.60:0/3430256821 conn(0x5584cf988020 msgr2=0x5584cf98aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:52.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff6395fa000 1 -- 172.21.15.60:0/3430256821 shutdown_connections 2024-04-25T10:07:52.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff6395fa000 1 -- 172.21.15.60:0/3430256821 wait complete. 2024-04-25T10:07:52.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.370+0000 7ff6395fa000 20 client.6084 trim_cache size 0 max 16384 2024-04-25T10:07:52.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 Processor -- start 2024-04-25T10:07:52.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 -- start start 2024-04-25T10:07:52.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 0x5584cfa31ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cfa32420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf9d62f0 0x5584cfa32960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff630206da0 con 0x5584cf9d59d0 2024-04-25T10:07:52.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6300b1c00 con 0x5584cf98c0a0 2024-04-25T10:07:52.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6300b1000 con 0x5584cf9d62f0 2024-04-25T10:07:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff62f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cfa32420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff62f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cfa32420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50948/0 (socket says 172.21.15.60:50948) 2024-04-25T10:07:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff62f7fe640 1 -- 172.21.15.60:0/2001894513 learned_addr learned my addr 172.21.15.60:0/2001894513 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff62ffff640 1 --2- 172.21.15.60:0/2001894513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 0x5584cfa31ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff62f7fe640 1 -- 172.21.15.60:0/2001894513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf9d62f0 msgr2=0x5584cfa32960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff634a2a640 1 --2- 172.21.15.60:0/2001894513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf9d62f0 0x5584cfa32960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff62f7fe640 1 --2- 172.21.15.60:0/2001894513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf9d62f0 0x5584cfa32960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff62f7fe640 1 -- 172.21.15.60:0/2001894513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 msgr2=0x5584cfa31ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff62f7fe640 1 --2- 172.21.15.60:0/2001894513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 0x5584cfa31ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff62f7fe640 1 -- 172.21.15.60:0/2001894513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584cf81fde0 con 0x5584cf9d59d0 2024-04-25T10:07:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff62ffff640 1 --2- 172.21.15.60:0/2001894513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 0x5584cfa31ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff62f7fe640 1 --2- 172.21.15.60:0/2001894513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cfa32420 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff6200efb50 tx=0x7ff6200f6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbe0b440cb6d5b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6257fa640 1 -- 172.21.15.60:0/2001894513 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6200efe50 con 0x5584cf9d59d0 2024-04-25T10:07:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6257fa640 1 -- 172.21.15.60:0/2001894513 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6200f2040 con 0x5584cf9d59d0 2024-04-25T10:07:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6257fa640 1 -- 172.21.15.60:0/2001894513 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6200f3fa0 con 0x5584cf9d59d0 2024-04-25T10:07:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 -- 172.21.15.60:0/2001894513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 msgr2=0x5584cfa32420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 --2- 172.21.15.60:0/2001894513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cfa32420 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff6200efb50 tx=0x7ff6200f6040 comp rx=0 tx=0).stop 2024-04-25T10:07:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 -- 172.21.15.60:0/2001894513 shutdown_connections 2024-04-25T10:07:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 --2- 172.21.15.60:0/2001894513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf9d62f0 0x5584cfa32960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 --2- 172.21.15.60:0/2001894513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cfa32420 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 --2- 172.21.15.60:0/2001894513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf98c0a0 0x5584cfa31ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 -- 172.21.15.60:0/2001894513 >> 172.21.15.60:0/2001894513 conn(0x5584cf988020 msgr2=0x5584cf9e2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 -- 172.21.15.60:0/2001894513 shutdown_connections 2024-04-25T10:07:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 -- 172.21.15.60:0/2001894513 wait complete. 2024-04-25T10:07:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 -- start start 2024-04-25T10:07:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 Processor -- start 2024-04-25T10:07:52.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf98c0a0 0x5584cfa35b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.386+0000 7ff6395fa000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cfa36080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf9d62f0 0x5584cf9e7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584cf773e00 con 0x5584cf9d59d0 2024-04-25T10:07:52.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6300b1600 con 0x5584cf9d62f0 2024-04-25T10:07:52.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584cf7d8300 con 0x5584cf98c0a0 2024-04-25T10:07:52.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 10 client.0 _ll_register_callbacks cb 0x5584cf957040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:52.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff62f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cfa36080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff62f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cfa36080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50964/0 (socket says 172.21.15.60:50964) 2024-04-25T10:07:52.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff62f7fe640 1 -- 172.21.15.60:0/4261946504 learned_addr learned my addr 172.21.15.60:0/4261946504 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:52.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff634a2a640 1 --2- 172.21.15.60:0/4261946504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf9d62f0 0x5584cf9e7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff62ffff640 1 --2- 172.21.15.60:0/4261946504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf98c0a0 0x5584cfa35b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff62f7fe640 1 -- 172.21.15.60:0/4261946504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf98c0a0 msgr2=0x5584cfa35b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff62f7fe640 1 --2- 172.21.15.60:0/4261946504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf98c0a0 0x5584cfa35b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff62f7fe640 1 -- 172.21.15.60:0/4261946504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf9d62f0 msgr2=0x5584cf9e7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff62f7fe640 1 --2- 172.21.15.60:0/4261946504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf9d62f0 0x5584cf9e7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff62f7fe640 1 -- 172.21.15.60:0/4261946504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584cf77a8e0 con 0x5584cf9d59d0 2024-04-25T10:07:52.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff62f7fe640 1 --2- 172.21.15.60:0/4261946504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cfa36080 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff62001c9c0 tx=0x7ff620104000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244cd1924d72f2bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff60effd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:07:52.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff60effd640 1 -- 172.21.15.60:0/4261946504 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6200f3fa0 con 0x5584cf9d59d0 2024-04-25T10:07:52.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff60effd640 1 -- 172.21.15.60:0/4261946504 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6200f60b0 con 0x5584cf9d59d0 2024-04-25T10:07:52.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 -- 172.21.15.60:0/4261946504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6200f2040 con 0x5584cf9d59d0 2024-04-25T10:07:52.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff60effd640 1 -- 172.21.15.60:0/4261946504 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6200efe50 con 0x5584cf9d59d0 2024-04-25T10:07:52.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 10 client.6090 fetch_fsmap learned FSMap version 21 2024-04-25T10:07:52.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 -- 172.21.15.60:0/4261946504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x5584cf8328f0 con 0x5584cf9d59d0 2024-04-25T10:07:52.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff60effd640 1 -- 172.21.15.60:0/4261946504 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6200f2040 con 0x5584cf9d59d0 2024-04-25T10:07:52.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff60effd640 1 -- 172.21.15.60:0/4261946504 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff6200b4030 con 0x5584cf9d59d0 2024-04-25T10:07:52.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 10 client.6090 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T10:07:52.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 10 client.6090 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:07:52.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 20 client.6090 populate_metadata read hostname 'smithi060' 2024-04-25T10:07:52.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 --2- 172.21.15.60:0/4261946504 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x5584cf9e89b0 0x5584cf9e8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 4 client.6090 mds_command: new command op to 5796 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:07:52.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 -- 172.21.15.60:0/4261946504 --> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5584cf7e6530 con 0x5584cf9e89b0 2024-04-25T10:07:52.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff62ffff640 1 --2- 172.21.15.60:0/4261946504 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x5584cf9e89b0 0x5584cf9e8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff60effd640 10 client.6090 ms_handle_connect on v2:172.21.15.60:6833/1747757196 2024-04-25T10:07:52.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff62ffff640 1 --2- 172.21.15.60:0/4261946504 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x5584cf9e89b0 0x5584cf9e8dc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e573230e3e3f5f0 server_cookie=8e8c64ec8c3c8fa0 in_seq=0 out_seq=0 2024-04-25T10:07:52.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff60effd640 1 -- 172.21.15.60:0/4261946504 <== mds.0 v2:172.21.15.60:6833/1747757196 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5584cf7e6530 con 0x5584cf9e89b0 2024-04-25T10:07:52.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff60effd640 10 client.6090 handle_command_reply: tid=0 2024-04-25T10:07:52.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 client.6090 shutdown 2024-04-25T10:07:52.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 -- 172.21.15.60:0/4261946504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 msgr2=0x5584cfa36080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 --2- 172.21.15.60:0/4261946504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cfa36080 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff62001c9c0 tx=0x7ff620104000 comp rx=0 tx=0).stop 2024-04-25T10:07:52.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 -- 172.21.15.60:0/4261946504 shutdown_connections 2024-04-25T10:07:52.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 --2- 172.21.15.60:0/4261946504 >> [v2:172.21.15.60:6833/1747757196,v1:172.21.15.60:6835/1747757196] conn(0x5584cf9e89b0 0x5584cf9e8dc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 --2- 172.21.15.60:0/4261946504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584cf9d62f0 0x5584cf9e7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 --2- 172.21.15.60:0/4261946504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5584cf9d59d0 0x5584cfa36080 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 --2- 172.21.15.60:0/4261946504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5584cf98c0a0 0x5584cfa35b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff60effd640 0 client.6090 ms_handle_reset on v2:172.21.15.60:6833/1747757196 2024-04-25T10:07:52.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 -- 172.21.15.60:0/4261946504 >> 172.21.15.60:0/4261946504 conn(0x5584cf988020 msgr2=0x5584cf9d40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:52.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 -- 172.21.15.60:0/4261946504 shutdown_connections 2024-04-25T10:07:52.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 1 -- 172.21.15.60:0/4261946504 wait complete. 2024-04-25T10:07:52.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff6395fa000 20 client.6090 trim_cache size 0 max 16384 2024-04-25T10:07:52.405 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:07:52.405 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T10:07:52.405 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T10:07:52.405 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:07:52.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff63648d640 1 -- 172.21.15.60:0/3701293679 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff610066f60 msgr2=0x7ff610069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff63648d640 1 --2- 172.21.15.60:0/3701293679 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff610066f60 0x7ff610069420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff63015bb20 tx=0x7ff618006bd0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff63648d640 1 -- 172.21.15.60:0/3701293679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff63015fa20 msgr2=0x7ff6300afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff63648d640 1 --2- 172.21.15.60:0/3701293679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff63015fa20 0x7ff6300afac0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff6200af040 tx=0x7ff620019280 comp rx=0 tx=0).stop 2024-04-25T10:07:52.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff63648d640 1 -- 172.21.15.60:0/3701293679 shutdown_connections 2024-04-25T10:07:52.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff63648d640 1 --2- 172.21.15.60:0/3701293679 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff610066f60 0x7ff610069420 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff63648d640 1 --2- 172.21.15.60:0/3701293679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff630160080 0x7ff63013d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff63648d640 1 --2- 172.21.15.60:0/3701293679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff63015fa20 0x7ff6300afac0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff63648d640 1 --2- 172.21.15.60:0/3701293679 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6300cbcf0 0x7ff63015ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff63648d640 1 -- 172.21.15.60:0/3701293679 >> 172.21.15.60:0/3701293679 conn(0x7ff6300670b0 msgr2=0x7ff63013e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:52.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff63648d640 1 -- 172.21.15.60:0/3701293679 shutdown_connections 2024-04-25T10:07:52.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.390+0000 7ff63648d640 1 -- 172.21.15.60:0/3701293679 wait complete. 2024-04-25T10:07:52.416 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:07:52.517 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:52.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.630+0000 7fbb67a43640 1 Processor -- start 2024-04-25T10:07:52.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.630+0000 7fbb67a43640 1 -- start start 2024-04-25T10:07:52.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.630+0000 7fbb67a43640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbb600b4660 0x7fbb6015c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.630+0000 7fbb67a43640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb60071ca0 con 0x7fbb6015bc90 2024-04-25T10:07:52.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.630+0000 7fbb67a43640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb600cca40 con 0x7fbb600ce7a0 2024-04-25T10:07:52.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.630+0000 7fbb67a43640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb600cc370 con 0x7fbb600b4660 2024-04-25T10:07:52.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.630+0000 7fbb65fe0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbb600b4660 0x7fbb6015c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.630+0000 7fbb64fde640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbb6015bc90 0x7fbb6015c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42888/0 (socket says 172.21.15.60:42888) 2024-04-25T10:07:52.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.630+0000 7fbb64fde640 1 -- 172.21.15.60:0/2793605306 learned_addr learned my addr 172.21.15.60:0/2793605306 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:52.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.630+0000 7fbb53fff640 1 -- 172.21.15.60:0/2793605306 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3959937926 0 0) 0x7fbb60071ca0 con 0x7fbb6015bc90 2024-04-25T10:07:52.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.630+0000 7fbb53fff640 1 -- 172.21.15.60:0/2793605306 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb4c003530 con 0x7fbb6015bc90 2024-04-25T10:07:52.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.630+0000 7fbb65fe0640 1 -- 172.21.15.60:0/2793605306 >> v1:172.21.15.100:6789/0 conn(0x7fbb600ce7a0 legacy=0x7fbb600ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.630+0000 7fbb65fe0640 1 -- 172.21.15.60:0/2793605306 >> v1:172.21.15.60:6789/0 conn(0x7fbb6015bc90 legacy=0x7fbb6015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.630+0000 7fbb65fe0640 1 -- 172.21.15.60:0/2793605306 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb600ccd20 con 0x7fbb600b4660 2024-04-25T10:07:52.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.634+0000 7fbb65fe0640 1 --2- 172.21.15.60:0/2793605306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb600b4660 0x7fbb6015c930 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbb580097c0 tx=0x7fbb580208b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7688966e28b2c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.634+0000 7fbb53fff640 1 -- 172.21.15.60:0/2793605306 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb580aa480 con 0x7fbb600b4660 2024-04-25T10:07:52.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.634+0000 7fbb53fff640 1 -- 172.21.15.60:0/2793605306 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb580aa620 con 0x7fbb600b4660 2024-04-25T10:07:52.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.634+0000 7fbb53fff640 1 -- 172.21.15.60:0/2793605306 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb580aa9f0 con 0x7fbb600b4660 2024-04-25T10:07:52.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.634+0000 7fbb67a43640 1 -- 172.21.15.60:0/2793605306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb600b4660 msgr2=0x7fbb6015c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.634+0000 7fbb67a43640 1 --2- 172.21.15.60:0/2793605306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb600b4660 0x7fbb6015c930 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbb580097c0 tx=0x7fbb580208b0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.634+0000 7fbb67a43640 1 -- 172.21.15.60:0/2793605306 shutdown_connections 2024-04-25T10:07:52.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.634+0000 7fbb67a43640 1 --2- 172.21.15.60:0/2793605306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb600b4660 0x7fbb6015c930 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.634+0000 7fbb67a43640 1 -- 172.21.15.60:0/2793605306 >> 172.21.15.60:0/2793605306 conn(0x7fbb600b12d0 msgr2=0x7fbb600b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:52.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.634+0000 7fbb67a43640 1 -- 172.21.15.60:0/2793605306 shutdown_connections 2024-04-25T10:07:52.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.634+0000 7fbb67a43640 1 -- 172.21.15.60:0/2793605306 wait complete. 2024-04-25T10:07:52.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb67a43640 1 Processor -- start 2024-04-25T10:07:52.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb67a43640 1 -- start start 2024-04-25T10:07:52.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb67a43640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb600ce7a0 0x7fbb60151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb67a43640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb6015bc90 0x7fbb6014d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb67a43640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb60157cd0 0x7fbb6015a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb67a43640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb601675c0 con 0x7fbb600ce7a0 2024-04-25T10:07:52.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb67a43640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb600b3db0 con 0x7fbb6015bc90 2024-04-25T10:07:52.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb67a43640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb60167740 con 0x7fbb60157cd0 2024-04-25T10:07:52.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb657df640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb600ce7a0 0x7fbb60151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb657df640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb600ce7a0 0x7fbb60151be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50992/0 (socket says 172.21.15.60:50992) 2024-04-25T10:07:52.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb657df640 1 -- 172.21.15.60:0/1220258162 learned_addr learned my addr 172.21.15.60:0/1220258162 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:52.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb64fde640 1 --2- 172.21.15.60:0/1220258162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb6015bc90 0x7fbb6014d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb657df640 1 -- 172.21.15.60:0/1220258162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb60157cd0 msgr2=0x7fbb6015a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb657df640 1 --2- 172.21.15.60:0/1220258162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb60157cd0 0x7fbb6015a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb657df640 1 -- 172.21.15.60:0/1220258162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb6015bc90 msgr2=0x7fbb6014d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb657df640 1 --2- 172.21.15.60:0/1220258162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb6015bc90 0x7fbb6014d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb657df640 1 -- 172.21.15.60:0/1220258162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb601522e0 con 0x7fbb600ce7a0 2024-04-25T10:07:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb64fde640 1 --2- 172.21.15.60:0/1220258162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb6015bc90 0x7fbb6014d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb657df640 1 --2- 172.21.15.60:0/1220258162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb600ce7a0 0x7fbb60151be0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00c760 tx=0x7fbb5c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb51ffb640 1 -- 172.21.15.60:0/1220258162 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c016020 con 0x7fbb600ce7a0 2024-04-25T10:07:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb51ffb640 1 -- 172.21.15.60:0/1220258162 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c006d60 con 0x7fbb600ce7a0 2024-04-25T10:07:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb67a43640 1 -- 172.21.15.60:0/1220258162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb601525c0 con 0x7fbb600ce7a0 2024-04-25T10:07:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb51ffb640 1 -- 172.21.15.60:0/1220258162 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c003bc0 con 0x7fbb600ce7a0 2024-04-25T10:07:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb67a43640 1 -- 172.21.15.60:0/1220258162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6014f3a0 con 0x7fbb600ce7a0 2024-04-25T10:07:52.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb67a43640 1 -- 172.21.15.60:0/1220258162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb60152ae0 con 0x7fbb600ce7a0 2024-04-25T10:07:52.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb51ffb640 1 -- 172.21.15.60:0/1220258162 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb5c00a7f0 con 0x7fbb600ce7a0 2024-04-25T10:07:52.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb51ffb640 1 --2- 172.21.15.60:0/1220258162 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbb44066f60 0x7fbb44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.638+0000 7fbb51ffb640 1 -- 172.21.15.60:0/1220258162 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb5c084c40 con 0x7fbb600ce7a0 2024-04-25T10:07:52.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.642+0000 7fbb64fde640 1 --2- 172.21.15.60:0/1220258162 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbb44066f60 0x7fbb44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.642+0000 7fbb64fde640 1 --2- 172.21.15.60:0/1220258162 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbb44066f60 0x7fbb44069420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbb600b06c0 tx=0x7fbb54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.646+0000 7fbb51ffb640 1 -- 172.21.15.60:0/1220258162 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb60152ae0 con 0x7fbb600ce7a0 2024-04-25T10:07:52.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.834+0000 7fbb67a43640 1 -- 172.21.15.60:0/1220258162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb60074770 con 0x7fbb600ce7a0 2024-04-25T10:07:52.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.838+0000 7fbb51ffb640 1 -- 172.21.15.60:0/1220258162 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fbb5c020020 con 0x7fbb600ce7a0 2024-04-25T10:07:52.847 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:52.847 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1138162580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1138162580},{"type":"v1","addr":"172.21.15.100:6837","nonce":1138162580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2985559168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2985559168},{"type":"v1","addr":"172.21.15.100:6835","nonce":2985559168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/50795370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":50795370},{"type":"v1","addr":"172.21.15.60:6834","nonce":50795370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:07.980095+0000","modified":"2024-04-25T10:07:10.002670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5796":{"gid":5796,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1747757196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1747757196},{"type":"v1","addr":"172.21.15.60:6835","nonce":1747757196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:52.848 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 21 2024-04-25T10:07:52.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.838+0000 7fbb67a43640 1 -- 172.21.15.60:0/1220258162 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbb44066f60 msgr2=0x7fbb44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.838+0000 7fbb67a43640 1 --2- 172.21.15.60:0/1220258162 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbb44066f60 0x7fbb44069420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbb600b06c0 tx=0x7fbb54009040 comp rx=0 tx=0).stop 2024-04-25T10:07:52.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.838+0000 7fbb67a43640 1 -- 172.21.15.60:0/1220258162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb600ce7a0 msgr2=0x7fbb60151be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.838+0000 7fbb67a43640 1 --2- 172.21.15.60:0/1220258162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb600ce7a0 0x7fbb60151be0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00c760 tx=0x7fbb5c00cc20 comp rx=0 tx=0).stop 2024-04-25T10:07:52.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.838+0000 7fbb67a43640 1 -- 172.21.15.60:0/1220258162 shutdown_connections 2024-04-25T10:07:52.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.838+0000 7fbb67a43640 1 --2- 172.21.15.60:0/1220258162 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbb44066f60 0x7fbb44069420 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.838+0000 7fbb67a43640 1 --2- 172.21.15.60:0/1220258162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb60157cd0 0x7fbb6015a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.838+0000 7fbb67a43640 1 --2- 172.21.15.60:0/1220258162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb6015bc90 0x7fbb6014d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.838+0000 7fbb67a43640 1 --2- 172.21.15.60:0/1220258162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb600ce7a0 0x7fbb60151be0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.838+0000 7fbb67a43640 1 -- 172.21.15.60:0/1220258162 >> 172.21.15.60:0/1220258162 conn(0x7fbb600b12d0 msgr2=0x7fbb6015c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:52.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.838+0000 7fbb67a43640 1 -- 172.21.15.60:0/1220258162 shutdown_connections 2024-04-25T10:07:52.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:52.838+0000 7fbb67a43640 1 -- 172.21.15.60:0/1220258162 wait complete. 2024-04-25T10:07:52.863 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:07:53.010 INFO:teuthology.orchestra.run.smithi060.stdout:{"message":"","return_code":0} 2024-04-25T10:07:53.010 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:07:53.011 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:07:53.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 Processor -- start 2024-04-25T10:07:53.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 -- start start 2024-04-25T10:07:53.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f339c0022b0 0x558cc394d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cc3947700 0x558cc3949760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558cc388f960 0x558cc38bae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc394b740 con 0x7f339c0022b0 2024-04-25T10:07:53.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc38339f0 con 0x558cc3947700 2024-04-25T10:07:53.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cc394b9e0 con 0x558cc388f960 2024-04-25T10:07:53.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b398f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f339c0022b0 0x558cc394d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b398f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f339c0022b0 0x558cc394d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51016/0 (socket says 172.21.15.60:51016) 2024-04-25T10:07:53.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b398f640 1 -- 172.21.15.60:0/2619795724 learned_addr learned my addr 172.21.15.60:0/2619795724 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:53.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b318e640 1 --2- 172.21.15.60:0/2619795724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cc3947700 0x558cc3949760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b398f640 1 -- 172.21.15.60:0/2619795724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558cc388f960 msgr2=0x558cc38bae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:53.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b4190640 1 --2- 172.21.15.60:0/2619795724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558cc388f960 0x558cc38bae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b398f640 1 --2- 172.21.15.60:0/2619795724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558cc388f960 0x558cc38bae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b398f640 1 -- 172.21.15.60:0/2619795724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cc3947700 msgr2=0x558cc3949760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:53.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b398f640 1 --2- 172.21.15.60:0/2619795724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cc3947700 0x558cc3949760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b398f640 1 -- 172.21.15.60:0/2619795724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cc394db70 con 0x7f339c0022b0 2024-04-25T10:07:53.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b398f640 1 --2- 172.21.15.60:0/2619795724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f339c0022b0 0x558cc394d460 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f33ac01b4a0 tx=0x7f33ac019eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35eab958b2b2d955 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b218c640 1 -- 172.21.15.60:0/2619795724 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0ba070 con 0x7f339c0022b0 2024-04-25T10:07:53.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b218c640 1 -- 172.21.15.60:0/2619795724 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ac005240 con 0x7f339c0022b0 2024-04-25T10:07:53.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b218c640 1 -- 172.21.15.60:0/2619795724 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac004610 con 0x7f339c0022b0 2024-04-25T10:07:53.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 -- 172.21.15.60:0/2619795724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f339c0022b0 msgr2=0x558cc394d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 --2- 172.21.15.60:0/2619795724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f339c0022b0 0x558cc394d460 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f33ac01b4a0 tx=0x7f33ac019eb0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 -- 172.21.15.60:0/2619795724 shutdown_connections 2024-04-25T10:07:53.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 --2- 172.21.15.60:0/2619795724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558cc388f960 0x558cc38bae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 --2- 172.21.15.60:0/2619795724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cc3947700 0x558cc3949760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 --2- 172.21.15.60:0/2619795724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f339c0022b0 0x558cc394d460 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 -- 172.21.15.60:0/2619795724 >> 172.21.15.60:0/2619795724 conn(0x558cc38b7e80 msgr2=0x558cc3967220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:53.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 -- 172.21.15.60:0/2619795724 shutdown_connections 2024-04-25T10:07:53.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 -- 172.21.15.60:0/2619795724 wait complete. 2024-04-25T10:07:53.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 Processor -- start 2024-04-25T10:07:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.038+0000 7f33b5e71080 1 -- start start 2024-04-25T10:07:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b5e71080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558cc388f960 0x558cc396bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b5e71080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cc3947700 0x558cc39fe080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b5e71080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cc38bb7e0 0x558cc3a00dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b5e71080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc3948d80 con 0x558cc3947700 2024-04-25T10:07:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b5e71080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cc394de10 con 0x558cc38bb7e0 2024-04-25T10:07:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b5e71080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cc3949020 con 0x558cc388f960 2024-04-25T10:07:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b318e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cc3947700 0x558cc39fe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b318e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cc3947700 0x558cc39fe080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51022/0 (socket says 172.21.15.60:51022) 2024-04-25T10:07:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b318e640 1 -- 172.21.15.60:0/157513135 learned_addr learned my addr 172.21.15.60:0/157513135 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b4190640 1 --2- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cc38bb7e0 0x558cc3a00dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b398f640 1 --2- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558cc388f960 0x558cc396bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b318e640 1 -- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558cc388f960 msgr2=0x558cc396bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b318e640 1 --2- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558cc388f960 0x558cc396bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b318e640 1 -- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cc38bb7e0 msgr2=0x558cc3a00dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b318e640 1 --2- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558cc38bb7e0 0x558cc3a00dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b318e640 1 -- 172.21.15.60:0/157513135 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cc38bc020 con 0x558cc3947700 2024-04-25T10:07:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b398f640 1 --2- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558cc388f960 0x558cc396bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b318e640 1 --2- 172.21.15.60:0/157513135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cc3947700 0x558cc39fe080 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f33a800c6b0 tx=0x7f33a800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b118a640 1 -- 172.21.15.60:0/157513135 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a800aad0 con 0x558cc3947700 2024-04-25T10:07:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b118a640 1 -- 172.21.15.60:0/157513135 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a800ac70 con 0x558cc3947700 2024-04-25T10:07:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b5e71080 1 -- 172.21.15.60:0/157513135 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558cc38bc2c0 con 0x558cc3947700 2024-04-25T10:07:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b118a640 1 -- 172.21.15.60:0/157513135 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8017640 con 0x558cc3947700 2024-04-25T10:07:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b5e71080 1 -- 172.21.15.60:0/157513135 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cc38bc800 con 0x558cc3947700 2024-04-25T10:07:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b118a640 1 -- 172.21.15.60:0/157513135 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33a8020440 con 0x558cc3947700 2024-04-25T10:07:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b118a640 1 --2- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f33a0065970 0x7f33a0067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b118a640 1 -- 172.21.15.60:0/157513135 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33a8084380 con 0x558cc3947700 2024-04-25T10:07:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b5e71080 1 --2- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x558cc38bcf40 0x558cc38bd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b398f640 1 --2- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f33a0065970 0x7f33a0067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b5e71080 1 -- 172.21.15.60:0/157513135 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) v9 -- 0x558cc38bbbc0 con 0x558cc38bcf40 2024-04-25T10:07:53.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.042+0000 7f33b4190640 1 --2- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x558cc38bcf40 0x558cc38bd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.046+0000 7f33b398f640 1 --2- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f33a0065970 0x7f33a0067e30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x558cc38b7c70 tx=0x7f33ac003340 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.046+0000 7f33b4190640 1 --2- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x558cc38bcf40 0x558cc38bd320 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.046+0000 7f33b4190640 1 -- 172.21.15.60:0/157513135 <== osd.6 v2:172.21.15.100:6824/4086390550 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f33a4009720 con 0x558cc38bcf40 2024-04-25T10:07:53.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.046+0000 7f33b5e71080 1 -- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x558cc38bcf40 msgr2=0x558cc38bd320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.046+0000 7f33b5e71080 1 --2- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x558cc38bcf40 0x558cc38bd320 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.046+0000 7f33b5e71080 1 -- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f33a0065970 msgr2=0x7f33a0067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.046+0000 7f33b5e71080 1 --2- 172.21.15.60:0/157513135 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f33a0065970 0x7f33a0067e30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x558cc38b7c70 tx=0x7f33ac003340 comp rx=0 tx=0).stop 2024-04-25T10:07:53.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.046+0000 7f33b5e71080 1 -- 172.21.15.60:0/157513135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cc3947700 msgr2=0x558cc39fe080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.046+0000 7f33b5e71080 1 --2- 172.21.15.60:0/157513135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558cc3947700 0x558cc39fe080 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f33a800c6b0 tx=0x7f33a800cb70 comp rx=0 tx=0).stop 2024-04-25T10:07:53.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.046+0000 7f33b4190640 1 -- 172.21.15.60:0/157513135 reap_dead start 2024-04-25T10:07:53.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.046+0000 7f33b5e71080 1 -- 172.21.15.60:0/157513135 shutdown_connections 2024-04-25T10:07:53.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.046+0000 7f33b5e71080 1 -- 172.21.15.60:0/157513135 >> 172.21.15.60:0/157513135 conn(0x558cc38b7e80 msgr2=0x558cc39496f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:53.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.046+0000 7f33b5e71080 1 -- 172.21.15.60:0/157513135 shutdown_connections 2024-04-25T10:07:53.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.046+0000 7f33b5e71080 1 -- 172.21.15.60:0/157513135 wait complete. 2024-04-25T10:07:53.057 INFO:teuthology.orchestra.run.smithi060.stdout:Yalphaparent_a 2024-04-25T10:07:53.057 DEBUG:teuthology.orchestra.run.smithi060:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:07:53.144 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:07:53.145 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511627777, 2024-04-25T10:07:53.145 INFO:teuthology.orchestra.run.smithi060.stdout: "ancestors": [ 2024-04-25T10:07:53.145 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:53.145 INFO:teuthology.orchestra.run.smithi060.stdout: "dirino": 1099511627776, 2024-04-25T10:07:53.145 INFO:teuthology.orchestra.run.smithi060.stdout: "dname": "alpha", 2024-04-25T10:07:53.145 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 4 2024-04-25T10:07:53.145 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:07:53.145 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:07:53.146 INFO:teuthology.orchestra.run.smithi060.stdout: "dirino": 1, 2024-04-25T10:07:53.146 INFO:teuthology.orchestra.run.smithi060.stdout: "dname": "parent_a", 2024-04-25T10:07:53.146 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 7 2024-04-25T10:07:53.146 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:07:53.146 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-04-25T10:07:53.146 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 7, 2024-04-25T10:07:53.146 INFO:teuthology.orchestra.run.smithi060.stdout: "old_pools": [] 2024-04-25T10:07:53.147 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:07:53.147 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:53.148 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:07:53.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:53.245 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:53.294 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:07:53.294 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:07:53.294 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:53.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:53.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9103edf640 1 Processor -- start 2024-04-25T10:07:53.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9103edf640 1 -- start start 2024-04-25T10:07:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9103edf640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f90fc15bc90 0x7f90fc15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9103edf640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90fc071ca0 con 0x7f90fc0ce7a0 2024-04-25T10:07:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9103edf640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90fc0cca40 con 0x7f90fc0b4660 2024-04-25T10:07:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9103edf640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90fc0cc370 con 0x7f90fc15bc90 2024-04-25T10:07:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9101c7b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f90fc0ce7a0 0x7f90fc0ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42904/0 (socket says 172.21.15.60:42904) 2024-04-25T10:07:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9101c7b640 1 -- 172.21.15.60:0/2073055395 learned_addr learned my addr 172.21.15.60:0/2073055395 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:53.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f910147a640 1 --2- 172.21.15.60:0/2073055395 >> v2:172.21.15.100:3300/0 conn(0x7f90fc15bc90 0x7f90fc15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9100c79640 1 -- 172.21.15.60:0/2073055395 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819543437 0 0) 0x7f90fc071ca0 con 0x7f90fc0ce7a0 2024-04-25T10:07:53.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9100c79640 1 -- 172.21.15.60:0/2073055395 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90e8003530 con 0x7f90fc0ce7a0 2024-04-25T10:07:53.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9100c79640 1 -- 172.21.15.60:0/2073055395 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2113722378 0 0) 0x7f90e8003530 con 0x7f90fc0ce7a0 2024-04-25T10:07:53.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9100c79640 1 -- 172.21.15.60:0/2073055395 >> v2:172.21.15.100:3300/0 conn(0x7f90fc15bc90 msgr2=0x7f90fc15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:53.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9100c79640 1 --2- 172.21.15.60:0/2073055395 >> v2:172.21.15.100:3300/0 conn(0x7f90fc15bc90 0x7f90fc15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9100c79640 1 -- 172.21.15.60:0/2073055395 >> v1:172.21.15.100:6789/0 conn(0x7f90fc0b4660 legacy=0x7f90fc15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9100c79640 1 -- 172.21.15.60:0/2073055395 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90fc0ccd20 con 0x7f90fc0ce7a0 2024-04-25T10:07:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9100c79640 1 -- 172.21.15.60:0/2073055395 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f90f8001680 con 0x7f90fc0ce7a0 2024-04-25T10:07:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9100c79640 1 -- 172.21.15.60:0/2073055395 >> v1:172.21.15.60:6789/0 conn(0x7f90fc0ce7a0 legacy=0x7f90fc0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9100c79640 1 --2- 172.21.15.60:0/2073055395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90e80039c0 0x7f90e8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9100c79640 1 --2- 172.21.15.60:0/2073055395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90e8004810 0x7f90e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9100c79640 1 --2- 172.21.15.60:0/2073055395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90e80051d0 0x7f90e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9100c79640 1 -- 172.21.15.60:0/2073055395 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e800c040 con 0x7f90e80051d0 2024-04-25T10:07:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9100c79640 1 -- 172.21.15.60:0/2073055395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e800c1f0 con 0x7f90e8004810 2024-04-25T10:07:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9100c79640 1 -- 172.21.15.60:0/2073055395 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e800c3f0 con 0x7f90e80039c0 2024-04-25T10:07:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f910247c640 1 --2- 172.21.15.60:0/2073055395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90e80051d0 0x7f90e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f910147a640 1 --2- 172.21.15.60:0/2073055395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90e8004810 0x7f90e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9101c7b640 1 --2- 172.21.15.60:0/2073055395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90e80039c0 0x7f90e8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f910247c640 1 -- 172.21.15.60:0/2073055395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90e80039c0 msgr2=0x7f90e8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f910247c640 1 --2- 172.21.15.60:0/2073055395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90e80039c0 0x7f90e8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f910247c640 1 -- 172.21.15.60:0/2073055395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90e8004810 msgr2=0x7f90e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f910247c640 1 --2- 172.21.15.60:0/2073055395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90e8004810 0x7f90e8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f910247c640 1 -- 172.21.15.60:0/2073055395 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e8003530 con 0x7f90e80051d0 2024-04-25T10:07:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.466+0000 7f9101c7b640 1 --2- 172.21.15.60:0/2073055395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90e80039c0 0x7f90e8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:53.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.470+0000 7f910247c640 1 -- 172.21.15.60:0/2073055395 reap_dead start 2024-04-25T10:07:53.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.470+0000 7f910247c640 1 --2- 172.21.15.60:0/2073055395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90e80051d0 0x7f90e800b9d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f90f4001e40 tx=0x7f90f401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5913aeea5233f141 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.470+0000 7f9100c79640 1 -- 172.21.15.60:0/2073055395 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f4001a90 con 0x7f90e80051d0 2024-04-25T10:07:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.470+0000 7f9100c79640 1 -- 172.21.15.60:0/2073055395 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f4003b80 con 0x7f90e80051d0 2024-04-25T10:07:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.470+0000 7f9100c79640 1 -- 172.21.15.60:0/2073055395 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f4005020 con 0x7f90e80051d0 2024-04-25T10:07:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.470+0000 7f9103edf640 1 -- 172.21.15.60:0/2073055395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90e80051d0 msgr2=0x7f90e800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.470+0000 7f9103edf640 1 --2- 172.21.15.60:0/2073055395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90e80051d0 0x7f90e800b9d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f90f4001e40 tx=0x7f90f401c760 comp rx=0 tx=0).stop 2024-04-25T10:07:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.470+0000 7f9103edf640 1 -- 172.21.15.60:0/2073055395 shutdown_connections 2024-04-25T10:07:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.470+0000 7f9103edf640 1 --2- 172.21.15.60:0/2073055395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90e80051d0 0x7f90e800b9d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.470+0000 7f9103edf640 1 -- 172.21.15.60:0/2073055395 >> 172.21.15.60:0/2073055395 conn(0x7f90fc0b12d0 msgr2=0x7f90fc0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.470+0000 7f9103edf640 1 -- 172.21.15.60:0/2073055395 shutdown_connections 2024-04-25T10:07:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.470+0000 7f9103edf640 1 -- 172.21.15.60:0/2073055395 wait complete. 2024-04-25T10:07:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.470+0000 7f9103edf640 1 Processor -- start 2024-04-25T10:07:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.470+0000 7f9103edf640 1 -- start start 2024-04-25T10:07:53.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9103edf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90e80051d0 0x7f90fc06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9103edf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90fc1f3190 0x7f90fc0b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9103edf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc15e240 0x7f90fc15e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9103edf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc0b4400 con 0x7f90e80051d0 2024-04-25T10:07:53.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9103edf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc0b4280 con 0x7f90fc15e240 2024-04-25T10:07:53.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9103edf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc0b3ba0 con 0x7f90fc1f3190 2024-04-25T10:07:53.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9101c7b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90e80051d0 0x7f90fc06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9101c7b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90e80051d0 0x7f90fc06d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51034/0 (socket says 172.21.15.60:51034) 2024-04-25T10:07:53.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9101c7b640 1 -- 172.21.15.60:0/2337863158 learned_addr learned my addr 172.21.15.60:0/2337863158 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:53.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f910147a640 1 --2- 172.21.15.60:0/2337863158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90fc1f3190 0x7f90fc0b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f910247c640 1 --2- 172.21.15.60:0/2337863158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc15e240 0x7f90fc15e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9101c7b640 1 -- 172.21.15.60:0/2337863158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90fc1f3190 msgr2=0x7f90fc0b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9101c7b640 1 --2- 172.21.15.60:0/2337863158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90fc1f3190 0x7f90fc0b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9101c7b640 1 -- 172.21.15.60:0/2337863158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc15e240 msgr2=0x7f90fc15e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9101c7b640 1 --2- 172.21.15.60:0/2337863158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc15e240 0x7f90fc15e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9101c7b640 1 -- 172.21.15.60:0/2337863158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90fc1f3970 con 0x7f90e80051d0 2024-04-25T10:07:53.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f910247c640 1 --2- 172.21.15.60:0/2337863158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc15e240 0x7f90fc15e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:53.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f910147a640 1 --2- 172.21.15.60:0/2337863158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90fc1f3190 0x7f90fc0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:53.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9101c7b640 1 --2- 172.21.15.60:0/2337863158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90e80051d0 0x7f90fc06d870 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f90f8000fc0 tx=0x7f90f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f90f27fc640 1 -- 172.21.15.60:0/2337863158 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8019070 con 0x7f90e80051d0 2024-04-25T10:07:53.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f90f27fc640 1 -- 172.21.15.60:0/2337863158 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f80074e0 con 0x7f90e80051d0 2024-04-25T10:07:53.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9103edf640 1 -- 172.21.15.60:0/2337863158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90fc1f3c50 con 0x7f90e80051d0 2024-04-25T10:07:53.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f90f27fc640 1 -- 172.21.15.60:0/2337863158 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8004a10 con 0x7f90e80051d0 2024-04-25T10:07:53.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9103edf640 1 -- 172.21.15.60:0/2337863158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90fc1f3610 con 0x7f90e80051d0 2024-04-25T10:07:53.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f9103edf640 1 -- 172.21.15.60:0/2337863158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90fc074770 con 0x7f90e80051d0 2024-04-25T10:07:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f90f27fc640 1 -- 172.21.15.60:0/2337863158 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f90f801c070 con 0x7f90e80051d0 2024-04-25T10:07:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f90f27fc640 1 --2- 172.21.15.60:0/2337863158 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f90e0066f60 0x7f90e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f90f27fc640 1 -- 172.21.15.60:0/2337863158 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90f8084a20 con 0x7f90e80051d0 2024-04-25T10:07:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.474+0000 7f910147a640 1 --2- 172.21.15.60:0/2337863158 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f90e0066f60 0x7f90e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.478+0000 7f910147a640 1 --2- 172.21.15.60:0/2337863158 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f90e0066f60 0x7f90e0069420 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009a70 tx=0x7f90ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.478+0000 7f90f27fc640 1 -- 172.21.15.60:0/2337863158 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f90f8055ac0 con 0x7f90e80051d0 2024-04-25T10:07:53.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.634+0000 7f9103edf640 1 -- 172.21.15.60:0/2337863158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90fc0b2d10 con 0x7f90e80051d0 2024-04-25T10:07:53.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.634+0000 7f90f27fc640 1 -- 172.21.15.60:0/2337863158 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16625 (secure 0 0 0) 0x7f90f80553c0 con 0x7f90e80051d0 2024-04-25T10:07:53.644 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:53.644 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":47,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:07:09.646687+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.652298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:06.983225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:53.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.638+0000 7f9103edf640 1 -- 172.21.15.60:0/2337863158 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f90e0066f60 msgr2=0x7f90e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.638+0000 7f9103edf640 1 --2- 172.21.15.60:0/2337863158 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f90e0066f60 0x7f90e0069420 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009a70 tx=0x7f90ec008040 comp rx=0 tx=0).stop 2024-04-25T10:07:53.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.638+0000 7f9103edf640 1 -- 172.21.15.60:0/2337863158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90e80051d0 msgr2=0x7f90fc06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.638+0000 7f9103edf640 1 --2- 172.21.15.60:0/2337863158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90e80051d0 0x7f90fc06d870 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f90f8000fc0 tx=0x7f90f800dc90 comp rx=0 tx=0).stop 2024-04-25T10:07:53.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.638+0000 7f9103edf640 1 -- 172.21.15.60:0/2337863158 shutdown_connections 2024-04-25T10:07:53.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.638+0000 7f9103edf640 1 --2- 172.21.15.60:0/2337863158 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f90e0066f60 0x7f90e0069420 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.638+0000 7f9103edf640 1 --2- 172.21.15.60:0/2337863158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc15e240 0x7f90fc15e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.638+0000 7f9103edf640 1 --2- 172.21.15.60:0/2337863158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90fc1f3190 0x7f90fc0b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.638+0000 7f9103edf640 1 --2- 172.21.15.60:0/2337863158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90e80051d0 0x7f90fc06d870 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.638+0000 7f9103edf640 1 -- 172.21.15.60:0/2337863158 >> 172.21.15.60:0/2337863158 conn(0x7f90fc0b12d0 msgr2=0x7f90fc150fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:53.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.638+0000 7f9103edf640 1 -- 172.21.15.60:0/2337863158 shutdown_connections 2024-04-25T10:07:53.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.638+0000 7f9103edf640 1 -- 172.21.15.60:0/2337863158 wait complete. 2024-04-25T10:07:53.664 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:05.652298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:06.983225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:53.665 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:53.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.770+0000 7fb464239640 1 Processor -- start 2024-04-25T10:07:53.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.770+0000 7fb464239640 1 -- start start 2024-04-25T10:07:53.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.774+0000 7fb464239640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb45c0cf8a0 0x7fb45c15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.774+0000 7fb464239640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb45c0cd3d0 0x7fb45c15be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.774+0000 7fb464239640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45c0cd890 con 0x7fb45c15dd30 2024-04-25T10:07:53.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.774+0000 7fb464239640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb45c0ce4a0 con 0x7fb45c0cd3d0 2024-04-25T10:07:53.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.774+0000 7fb464239640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c0cdb70 con 0x7fb45c0cf8a0 2024-04-25T10:07:53.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.774+0000 7fb4627d6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb45c0cd3d0 0x7fb45c15be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.774+0000 7fb4627d6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb45c0cd3d0 0x7fb45c15be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51036/0 (socket says 172.21.15.60:51036) 2024-04-25T10:07:53.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.774+0000 7fb4627d6640 1 -- 172.21.15.60:0/3812916251 learned_addr learned my addr 172.21.15.60:0/3812916251 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:53.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.774+0000 7fb4627d6640 1 -- 172.21.15.60:0/3812916251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb45c0cf8a0 msgr2=0x7fb45c15ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:53.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.774+0000 7fb4627d6640 1 --2- 172.21.15.60:0/3812916251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb45c0cf8a0 0x7fb45c15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.774+0000 7fb4627d6640 1 -- 172.21.15.60:0/3812916251 >> v1:172.21.15.100:6789/0 conn(0x7fb45c15dd30 legacy=0x7fb45c0cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:53.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.774+0000 7fb4627d6640 1 -- 172.21.15.60:0/3812916251 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c0cde50 con 0x7fb45c0cd3d0 2024-04-25T10:07:53.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.774+0000 7fb4627d6640 1 --2- 172.21.15.60:0/3812916251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb45c0cd3d0 0x7fb45c15be20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb454009e50 tx=0x7fb45401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0df2af8d3829998 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.774+0000 7fb460fd3640 1 -- 172.21.15.60:0/3812916251 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4540a57d0 con 0x7fb45c0cd3d0 2024-04-25T10:07:53.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.774+0000 7fb460fd3640 1 -- 172.21.15.60:0/3812916251 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4540a5970 con 0x7fb45c0cd3d0 2024-04-25T10:07:53.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.774+0000 7fb460fd3640 1 -- 172.21.15.60:0/3812916251 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4540a5d40 con 0x7fb45c0cd3d0 2024-04-25T10:07:53.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 -- 172.21.15.60:0/3812916251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb45c0cd3d0 msgr2=0x7fb45c15be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 --2- 172.21.15.60:0/3812916251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb45c0cd3d0 0x7fb45c15be20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb454009e50 tx=0x7fb45401c930 comp rx=0 tx=0).stop 2024-04-25T10:07:53.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 -- 172.21.15.60:0/3812916251 shutdown_connections 2024-04-25T10:07:53.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 --2- 172.21.15.60:0/3812916251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb45c0cd3d0 0x7fb45c15be20 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 --2- 172.21.15.60:0/3812916251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb45c0cf8a0 0x7fb45c15ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 -- 172.21.15.60:0/3812916251 >> 172.21.15.60:0/3812916251 conn(0x7fb45c0670b0 msgr2=0x7fb45c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:53.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 -- 172.21.15.60:0/3812916251 shutdown_connections 2024-04-25T10:07:53.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 -- 172.21.15.60:0/3812916251 wait complete. 2024-04-25T10:07:53.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 Processor -- start 2024-04-25T10:07:53.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 -- start start 2024-04-25T10:07:53.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb45c0cd3d0 0x7fb45c13f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb45c0cf8a0 0x7fb45c153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb45c15dd30 0x7fb45c152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c15cf00 con 0x7fb45c15dd30 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c15cd80 con 0x7fb45c0cf8a0 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c15d080 con 0x7fb45c0cd3d0 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb4627d6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb45c15dd30 0x7fb45c152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb4627d6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb45c15dd30 0x7fb45c152fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51052/0 (socket says 172.21.15.60:51052) 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb4627d6640 1 -- 172.21.15.60:0/59860337 learned_addr learned my addr 172.21.15.60:0/59860337 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb4617d4640 1 --2- 172.21.15.60:0/59860337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb45c0cf8a0 0x7fb45c153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb461fd5640 1 --2- 172.21.15.60:0/59860337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb45c0cd3d0 0x7fb45c13f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb4627d6640 1 -- 172.21.15.60:0/59860337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb45c0cd3d0 msgr2=0x7fb45c13f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb4627d6640 1 --2- 172.21.15.60:0/59860337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb45c0cd3d0 0x7fb45c13f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb4627d6640 1 -- 172.21.15.60:0/59860337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb45c0cf8a0 msgr2=0x7fb45c153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb4627d6640 1 --2- 172.21.15.60:0/59860337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb45c0cf8a0 0x7fb45c153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb4627d6640 1 -- 172.21.15.60:0/59860337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c152800 con 0x7fb45c15dd30 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb4617d4640 1 --2- 172.21.15.60:0/59860337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb45c0cf8a0 0x7fb45c153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb4627d6640 1 --2- 172.21.15.60:0/59860337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb45c15dd30 0x7fb45c152fc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb4540af040 tx=0x7fb4540196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb4527fc640 1 -- 172.21.15.60:0/59860337 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4540bc020 con 0x7fb45c15dd30 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb4527fc640 1 -- 172.21.15.60:0/59860337 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4540b8880 con 0x7fb45c15dd30 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 -- 172.21.15.60:0/59860337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c0b2450 con 0x7fb45c15dd30 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb4527fc640 1 -- 172.21.15.60:0/59860337 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4540b8c50 con 0x7fb45c15dd30 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.778+0000 7fb464239640 1 -- 172.21.15.60:0/59860337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45c152aa0 con 0x7fb45c15dd30 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.782+0000 7fb464239640 1 -- 172.21.15.60:0/59860337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb45c074770 con 0x7fb45c15dd30 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.782+0000 7fb4527fc640 1 -- 172.21.15.60:0/59860337 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4540bf020 con 0x7fb45c15dd30 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.782+0000 7fb4527fc640 1 --2- 172.21.15.60:0/59860337 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb440066f60 0x7fb440069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.782+0000 7fb4527fc640 1 -- 172.21.15.60:0/59860337 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb454120630 con 0x7fb45c15dd30 2024-04-25T10:07:53.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.782+0000 7fb461fd5640 1 --2- 172.21.15.60:0/59860337 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb440066f60 0x7fb440069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.782+0000 7fb461fd5640 1 --2- 172.21.15.60:0/59860337 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb440066f60 0x7fb440069420 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb44c007fd0 tx=0x7fb44c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.786+0000 7fb4527fc640 1 -- 172.21.15.60:0/59860337 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4540c6030 con 0x7fb45c15dd30 2024-04-25T10:07:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.938+0000 7fb464239640 1 -- 172.21.15.60:0/59860337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb45c158d00 con 0x7fb45c15dd30 2024-04-25T10:07:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.938+0000 7fb4527fc640 1 -- 172.21.15.60:0/59860337 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4540165e0 con 0x7fb45c15dd30 2024-04-25T10:07:53.947 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:07:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.942+0000 7fb464239640 1 -- 172.21.15.60:0/59860337 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb440066f60 msgr2=0x7fb440069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.942+0000 7fb464239640 1 --2- 172.21.15.60:0/59860337 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb440066f60 0x7fb440069420 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb44c007fd0 tx=0x7fb44c009040 comp rx=0 tx=0).stop 2024-04-25T10:07:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.942+0000 7fb464239640 1 -- 172.21.15.60:0/59860337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb45c15dd30 msgr2=0x7fb45c152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.942+0000 7fb464239640 1 --2- 172.21.15.60:0/59860337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb45c15dd30 0x7fb45c152fc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb4540af040 tx=0x7fb4540196d0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.942+0000 7fb464239640 1 -- 172.21.15.60:0/59860337 shutdown_connections 2024-04-25T10:07:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.942+0000 7fb464239640 1 --2- 172.21.15.60:0/59860337 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb440066f60 0x7fb440069420 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.942+0000 7fb464239640 1 --2- 172.21.15.60:0/59860337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb45c15dd30 0x7fb45c152fc0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.942+0000 7fb464239640 1 --2- 172.21.15.60:0/59860337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb45c0cf8a0 0x7fb45c153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.942+0000 7fb464239640 1 --2- 172.21.15.60:0/59860337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb45c0cd3d0 0x7fb45c13f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.942+0000 7fb464239640 1 -- 172.21.15.60:0/59860337 >> 172.21.15.60:0/59860337 conn(0x7fb45c0670b0 msgr2=0x7fb45c0af370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.942+0000 7fb464239640 1 -- 172.21.15.60:0/59860337 shutdown_connections 2024-04-25T10:07:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:53.942+0000 7fb464239640 1 -- 172.21.15.60:0/59860337 wait complete. 2024-04-25T10:07:53.965 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:54.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.082+0000 7f91447e7640 1 Processor -- start 2024-04-25T10:07:54.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.082+0000 7f91447e7640 1 -- start start 2024-04-25T10:07:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.082+0000 7f91447e7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f913c15a810 0x7f913c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.082+0000 7f91447e7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f913c0b43d0 0x7f913c0b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.082+0000 7f91447e7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913c15b7f0 con 0x7f913c15abf0 2024-04-25T10:07:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.082+0000 7f91447e7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913c0b4d70 con 0x7f913c15a810 2024-04-25T10:07:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.082+0000 7f91447e7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913c0ae530 con 0x7f913c0b43d0 2024-04-25T10:07:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.082+0000 7f9142d84640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f913c15abf0 0x7f913c15b020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42920/0 (socket says 172.21.15.60:42920) 2024-04-25T10:07:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.082+0000 7f9142583640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f913c15a810 0x7f913c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.082+0000 7f9142d84640 1 -- 172.21.15.60:0/3875665690 learned_addr learned my addr 172.21.15.60:0/3875665690 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.082+0000 7f9141d82640 1 --2- 172.21.15.60:0/3875665690 >> v2:172.21.15.100:3300/0 conn(0x7f913c0b43d0 0x7f913c0b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.082+0000 7f9142583640 1 -- 172.21.15.60:0/3875665690 >> v2:172.21.15.100:3300/0 conn(0x7f913c0b43d0 msgr2=0x7f913c0b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.082+0000 7f9142583640 1 --2- 172.21.15.60:0/3875665690 >> v2:172.21.15.100:3300/0 conn(0x7f913c0b43d0 0x7f913c0b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.082+0000 7f9142583640 1 -- 172.21.15.60:0/3875665690 >> v1:172.21.15.60:6789/0 conn(0x7f913c15abf0 legacy=0x7f913c15b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.082+0000 7f9142583640 1 -- 172.21.15.60:0/3875665690 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c15bad0 con 0x7f913c15a810 2024-04-25T10:07:54.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.082+0000 7f9141581640 1 -- 172.21.15.60:0/3875665690 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217041278 0 0) 0x7f913c15b7f0 con 0x7f913c15abf0 2024-04-25T10:07:54.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.086+0000 7f9142583640 1 --2- 172.21.15.60:0/3875665690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f913c15a810 0x7f913c06d8a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f913c066a50 tx=0x7f913401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494d160922904b63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.086+0000 7f9141581640 1 -- 172.21.15.60:0/3875665690 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91340ab650 con 0x7f913c15a810 2024-04-25T10:07:54.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.086+0000 7f9141581640 1 -- 172.21.15.60:0/3875665690 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91340b0070 con 0x7f913c15a810 2024-04-25T10:07:54.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.086+0000 7f9141581640 1 -- 172.21.15.60:0/3875665690 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91340aba10 con 0x7f913c15a810 2024-04-25T10:07:54.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.086+0000 7f91447e7640 1 -- 172.21.15.60:0/3875665690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f913c15a810 msgr2=0x7f913c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.086+0000 7f91447e7640 1 --2- 172.21.15.60:0/3875665690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f913c15a810 0x7f913c06d8a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f913c066a50 tx=0x7f913401c450 comp rx=0 tx=0).stop 2024-04-25T10:07:54.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.086+0000 7f91447e7640 1 -- 172.21.15.60:0/3875665690 shutdown_connections 2024-04-25T10:07:54.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.086+0000 7f91447e7640 1 --2- 172.21.15.60:0/3875665690 >> v2:172.21.15.100:3300/0 conn(0x7f913c0b43d0 0x7f913c0b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.086+0000 7f91447e7640 1 --2- 172.21.15.60:0/3875665690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f913c15a810 0x7f913c06d8a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.086+0000 7f91447e7640 1 -- 172.21.15.60:0/3875665690 >> 172.21.15.60:0/3875665690 conn(0x7f913c0cfdd0 msgr2=0x7f913c0d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:54.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.086+0000 7f91447e7640 1 -- 172.21.15.60:0/3875665690 shutdown_connections 2024-04-25T10:07:54.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.086+0000 7f91447e7640 1 -- 172.21.15.60:0/3875665690 wait complete. 2024-04-25T10:07:54.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.086+0000 7f91447e7640 1 Processor -- start 2024-04-25T10:07:54.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.086+0000 7f91447e7640 1 -- start start 2024-04-25T10:07:54.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f91447e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f913c0b43d0 0x7f913c14d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f91447e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f913c15a810 0x7f913c14b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f91447e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f913c15abf0 0x7f913c14bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f91447e7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0b5330 con 0x7f913c15abf0 2024-04-25T10:07:54.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f91447e7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0b51b0 con 0x7f913c15a810 2024-04-25T10:07:54.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f91447e7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0af1d0 con 0x7f913c0b43d0 2024-04-25T10:07:54.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f9142d84640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f913c15abf0 0x7f913c14bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f9142d84640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f913c15abf0 0x7f913c14bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51072/0 (socket says 172.21.15.60:51072) 2024-04-25T10:07:54.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f9142d84640 1 -- 172.21.15.60:0/216954315 learned_addr learned my addr 172.21.15.60:0/216954315 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:54.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f9141d82640 1 --2- 172.21.15.60:0/216954315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f913c15a810 0x7f913c14b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f9142583640 1 --2- 172.21.15.60:0/216954315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f913c0b43d0 0x7f913c14d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f9142d84640 1 -- 172.21.15.60:0/216954315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f913c0b43d0 msgr2=0x7f913c14d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f9142d84640 1 --2- 172.21.15.60:0/216954315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f913c0b43d0 0x7f913c14d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f9142d84640 1 -- 172.21.15.60:0/216954315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f913c15a810 msgr2=0x7f913c14b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f9142d84640 1 --2- 172.21.15.60:0/216954315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f913c15a810 0x7f913c14b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f9142d84640 1 -- 172.21.15.60:0/216954315 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c06f1a0 con 0x7f913c15abf0 2024-04-25T10:07:54.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f9142d84640 1 --2- 172.21.15.60:0/216954315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f913c15abf0 0x7f913c14bd70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f912c00c730 tx=0x7f912c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f913affd640 1 -- 172.21.15.60:0/216954315 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c018070 con 0x7f913c15abf0 2024-04-25T10:07:54.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f913affd640 1 -- 172.21.15.60:0/216954315 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c00ce40 con 0x7f913c15abf0 2024-04-25T10:07:54.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f91447e7640 1 -- 172.21.15.60:0/216954315 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913c1520b0 con 0x7f913c15abf0 2024-04-25T10:07:54.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f913affd640 1 -- 172.21.15.60:0/216954315 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c00a860 con 0x7f913c15abf0 2024-04-25T10:07:54.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f91447e7640 1 -- 172.21.15.60:0/216954315 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913c152950 con 0x7f913c15abf0 2024-04-25T10:07:54.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f91447e7640 1 -- 172.21.15.60:0/216954315 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913c074770 con 0x7f913c15abf0 2024-04-25T10:07:54.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f913affd640 1 -- 172.21.15.60:0/216954315 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f912c01b070 con 0x7f913c15abf0 2024-04-25T10:07:54.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f913affd640 1 --2- 172.21.15.60:0/216954315 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9128066f60 0x7f9128069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f913affd640 1 -- 172.21.15.60:0/216954315 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f912c084670 con 0x7f913c15abf0 2024-04-25T10:07:54.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.090+0000 7f9142583640 1 --2- 172.21.15.60:0/216954315 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9128066f60 0x7f9128069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.094+0000 7f9142583640 1 --2- 172.21.15.60:0/216954315 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9128066f60 0x7f9128069420 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f91340af000 tx=0x7f91340a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.094+0000 7f913affd640 1 -- 172.21.15.60:0/216954315 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f912c0555e0 con 0x7f913c15abf0 2024-04-25T10:07:54.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.250+0000 7f91447e7640 1 -- 172.21.15.60:0/216954315 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f913c0b3bb0 con 0x7f913c15abf0 2024-04-25T10:07:54.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.250+0000 7f913affd640 1 -- 172.21.15.60:0/216954315 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f912c054ee0 con 0x7f913c15abf0 2024-04-25T10:07:54.258 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:07:54.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.254+0000 7f91447e7640 1 -- 172.21.15.60:0/216954315 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9128066f60 msgr2=0x7f9128069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.254+0000 7f91447e7640 1 --2- 172.21.15.60:0/216954315 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9128066f60 0x7f9128069420 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f91340af000 tx=0x7f91340a7000 comp rx=0 tx=0).stop 2024-04-25T10:07:54.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.254+0000 7f91447e7640 1 -- 172.21.15.60:0/216954315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f913c15abf0 msgr2=0x7f913c14bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.254+0000 7f91447e7640 1 --2- 172.21.15.60:0/216954315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f913c15abf0 0x7f913c14bd70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f912c00c730 tx=0x7f912c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.254+0000 7f91447e7640 1 -- 172.21.15.60:0/216954315 shutdown_connections 2024-04-25T10:07:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.254+0000 7f91447e7640 1 --2- 172.21.15.60:0/216954315 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9128066f60 0x7f9128069420 secure :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f91340af000 tx=0x7f91340a7000 comp rx=0 tx=0).stop 2024-04-25T10:07:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.254+0000 7f91447e7640 1 --2- 172.21.15.60:0/216954315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f913c15abf0 0x7f913c14bd70 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.254+0000 7f91447e7640 1 --2- 172.21.15.60:0/216954315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f913c15a810 0x7f913c14b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.254+0000 7f91447e7640 1 --2- 172.21.15.60:0/216954315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f913c0b43d0 0x7f913c14d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.254+0000 7f91447e7640 1 -- 172.21.15.60:0/216954315 >> 172.21.15.60:0/216954315 conn(0x7f913c0cfdd0 msgr2=0x7f913c06b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.254+0000 7f91447e7640 1 -- 172.21.15.60:0/216954315 shutdown_connections 2024-04-25T10:07:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.254+0000 7f91447e7640 1 -- 172.21.15.60:0/216954315 wait complete. 2024-04-25T10:07:54.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:54.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d9146c640 1 Processor -- start 2024-04-25T10:07:54.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d9146c640 1 -- start start 2024-04-25T10:07:54.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d9146c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d8c0b37c0 0x7f9d8c0af510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d9146c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9d8c06ac10 0x7f9d8c06b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d9146c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d8c06c550 con 0x7f9d8c0b33e0 2024-04-25T10:07:54.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d9146c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8c06b940 con 0x7f9d8c06ac10 2024-04-25T10:07:54.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d9146c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c06bc20 con 0x7f9d8c0b37c0 2024-04-25T10:07:54.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d8affd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9d8c0b33e0 0x7f9d8c0aee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:40616/0 (socket says 172.21.15.60:40616) 2024-04-25T10:07:54.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d8affd640 1 -- 172.21.15.60:0/109656139 learned_addr learned my addr 172.21.15.60:0/109656139 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:54.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d8a7fc640 1 --2- 172.21.15.60:0/109656139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d8c0b37c0 0x7f9d8c0af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d8b7fe640 1 --2- 172.21.15.60:0/109656139 >> v2:172.21.15.100:3300/0 conn(0x7f9d8c06ac10 0x7f9d8c06b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d89ffb640 1 -- 172.21.15.60:0/109656139 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343485993 0 0) 0x7f9d8c06c550 con 0x7f9d8c0b33e0 2024-04-25T10:07:54.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d89ffb640 1 -- 172.21.15.60:0/109656139 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d74003530 con 0x7f9d8c0b33e0 2024-04-25T10:07:54.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d8b7fe640 1 -- 172.21.15.60:0/109656139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d8c0b37c0 msgr2=0x7f9d8c0af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:54.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d8b7fe640 1 --2- 172.21.15.60:0/109656139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d8c0b37c0 0x7f9d8c0af510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d8b7fe640 1 -- 172.21.15.60:0/109656139 >> v1:172.21.15.100:6789/0 conn(0x7f9d8c0b33e0 legacy=0x7f9d8c0aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d8b7fe640 1 -- 172.21.15.60:0/109656139 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c06bf00 con 0x7f9d8c06ac10 2024-04-25T10:07:54.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.394+0000 7f9d8a7fc640 1 --2- 172.21.15.60:0/109656139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d8c0b37c0 0x7f9d8c0af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:54.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.398+0000 7f9d8b7fe640 1 --2- 172.21.15.60:0/109656139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d8c06ac10 0x7f9d8c06b0c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0097f0 tx=0x7f9d7c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6e9b20055a62c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.398+0000 7f9d89ffb640 1 -- 172.21.15.60:0/109656139 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0b2070 con 0x7f9d8c06ac10 2024-04-25T10:07:54.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.398+0000 7f9d89ffb640 1 -- 172.21.15.60:0/109656139 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c0ad680 con 0x7f9d8c06ac10 2024-04-25T10:07:54.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.398+0000 7f9d89ffb640 1 -- 172.21.15.60:0/109656139 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0ada50 con 0x7f9d8c06ac10 2024-04-25T10:07:54.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.398+0000 7f9d9146c640 1 -- 172.21.15.60:0/109656139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d8c06ac10 msgr2=0x7f9d8c06b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.398+0000 7f9d9146c640 1 --2- 172.21.15.60:0/109656139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d8c06ac10 0x7f9d8c06b0c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0097f0 tx=0x7f9d7c01c470 comp rx=0 tx=0).stop 2024-04-25T10:07:54.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.398+0000 7f9d9146c640 1 -- 172.21.15.60:0/109656139 shutdown_connections 2024-04-25T10:07:54.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.398+0000 7f9d9146c640 1 --2- 172.21.15.60:0/109656139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d8c06ac10 0x7f9d8c06b0c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.398+0000 7f9d9146c640 1 --2- 172.21.15.60:0/109656139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d8c0b37c0 0x7f9d8c0af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.398+0000 7f9d9146c640 1 -- 172.21.15.60:0/109656139 >> 172.21.15.60:0/109656139 conn(0x7f9d8c0cfdd0 msgr2=0x7f9d8c0d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:54.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.398+0000 7f9d9146c640 1 -- 172.21.15.60:0/109656139 shutdown_connections 2024-04-25T10:07:54.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.398+0000 7f9d9146c640 1 -- 172.21.15.60:0/109656139 wait complete. 2024-04-25T10:07:54.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.398+0000 7f9d9146c640 1 Processor -- start 2024-04-25T10:07:54.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d9146c640 1 -- start start 2024-04-25T10:07:54.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d9146c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d8c06ac10 0x7f9d8c0acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d9146c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d8c0b33e0 0x7f9d8c0cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d9146c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d8c0b37c0 0x7f9d8c0cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d9146c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c06db60 con 0x7f9d8c0b37c0 2024-04-25T10:07:54.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d9146c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c15ee80 con 0x7f9d8c0b33e0 2024-04-25T10:07:54.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d9146c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c06dce0 con 0x7f9d8c06ac10 2024-04-25T10:07:54.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d8b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d8c0b37c0 0x7f9d8c0cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d8b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d8c0b37c0 0x7f9d8c0cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51078/0 (socket says 172.21.15.60:51078) 2024-04-25T10:07:54.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d8b7fe640 1 -- 172.21.15.60:0/1516394945 learned_addr learned my addr 172.21.15.60:0/1516394945 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:54.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d8a7fc640 1 --2- 172.21.15.60:0/1516394945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d8c0b33e0 0x7f9d8c0cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d8affd640 1 --2- 172.21.15.60:0/1516394945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d8c06ac10 0x7f9d8c0acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d8b7fe640 1 -- 172.21.15.60:0/1516394945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d8c06ac10 msgr2=0x7f9d8c0acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d8b7fe640 1 --2- 172.21.15.60:0/1516394945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d8c06ac10 0x7f9d8c0acf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d8b7fe640 1 -- 172.21.15.60:0/1516394945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d8c0b33e0 msgr2=0x7f9d8c0cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d8b7fe640 1 --2- 172.21.15.60:0/1516394945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d8c0b33e0 0x7f9d8c0cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d8b7fe640 1 -- 172.21.15.60:0/1516394945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c0ceb10 con 0x7f9d8c0b37c0 2024-04-25T10:07:54.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d8b7fe640 1 --2- 172.21.15.60:0/1516394945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d8c0b37c0 0x7f9d8c0cce80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c01ab20 tx=0x7f9d7c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d737fe640 1 -- 172.21.15.60:0/1516394945 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0b2040 con 0x7f9d8c0b37c0 2024-04-25T10:07:54.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d737fe640 1 -- 172.21.15.60:0/1516394945 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c0be070 con 0x7f9d8c0b37c0 2024-04-25T10:07:54.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d9146c640 1 -- 172.21.15.60:0/1516394945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8c0cedf0 con 0x7f9d8c0b37c0 2024-04-25T10:07:54.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d737fe640 1 -- 172.21.15.60:0/1516394945 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0add90 con 0x7f9d8c0b37c0 2024-04-25T10:07:54.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d9146c640 1 -- 172.21.15.60:0/1516394945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8c0cf3b0 con 0x7f9d8c0b37c0 2024-04-25T10:07:54.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d9146c640 1 -- 172.21.15.60:0/1516394945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8c074770 con 0x7f9d8c0b37c0 2024-04-25T10:07:54.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d737fe640 1 -- 172.21.15.60:0/1516394945 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d7c0c3460 con 0x7f9d8c0b37c0 2024-04-25T10:07:54.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d737fe640 1 --2- 172.21.15.60:0/1516394945 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d78066f60 0x7f9d78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.402+0000 7f9d737fe640 1 -- 172.21.15.60:0/1516394945 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d7c11f2f0 con 0x7f9d8c0b37c0 2024-04-25T10:07:54.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.406+0000 7f9d8affd640 1 --2- 172.21.15.60:0/1516394945 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d78066f60 0x7f9d78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.406+0000 7f9d8affd640 1 --2- 172.21.15.60:0/1516394945 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d78066f60 0x7f9d78069420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9d84005890 tx=0x7f9d84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.410+0000 7f9d737fe640 1 -- 172.21.15.60:0/1516394945 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d7c016090 con 0x7f9d8c0b37c0 2024-04-25T10:07:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.566+0000 7f9d9146c640 1 -- 172.21.15.60:0/1516394945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9d8c160c20 con 0x7f9d8c0b37c0 2024-04-25T10:07:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.566+0000 7f9d737fe640 1 -- 172.21.15.60:0/1516394945 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9d7c0f0420 con 0x7f9d8c0b37c0 2024-04-25T10:07:54.576 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:07:54.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.570+0000 7f9d9146c640 1 -- 172.21.15.60:0/1516394945 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d78066f60 msgr2=0x7f9d78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.570+0000 7f9d9146c640 1 --2- 172.21.15.60:0/1516394945 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d78066f60 0x7f9d78069420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9d84005890 tx=0x7f9d84009040 comp rx=0 tx=0).stop 2024-04-25T10:07:54.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.570+0000 7f9d9146c640 1 -- 172.21.15.60:0/1516394945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d8c0b37c0 msgr2=0x7f9d8c0cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.570+0000 7f9d9146c640 1 --2- 172.21.15.60:0/1516394945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d8c0b37c0 0x7f9d8c0cce80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c01ab20 tx=0x7f9d7c01acb0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.570+0000 7f9d9146c640 1 -- 172.21.15.60:0/1516394945 shutdown_connections 2024-04-25T10:07:54.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.570+0000 7f9d9146c640 1 --2- 172.21.15.60:0/1516394945 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d78066f60 0x7f9d78069420 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.570+0000 7f9d9146c640 1 --2- 172.21.15.60:0/1516394945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d8c0b37c0 0x7f9d8c0cce80 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.570+0000 7f9d9146c640 1 --2- 172.21.15.60:0/1516394945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d8c0b33e0 0x7f9d8c0cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.570+0000 7f9d9146c640 1 --2- 172.21.15.60:0/1516394945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d8c06ac10 0x7f9d8c0acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.570+0000 7f9d9146c640 1 -- 172.21.15.60:0/1516394945 >> 172.21.15.60:0/1516394945 conn(0x7f9d8c0cfdd0 msgr2=0x7f9d8c15f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:54.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.570+0000 7f9d9146c640 1 -- 172.21.15.60:0/1516394945 shutdown_connections 2024-04-25T10:07:54.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.570+0000 7f9d9146c640 1 -- 172.21.15.60:0/1516394945 wait complete. 2024-04-25T10:07:54.595 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:07:54.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.706+0000 7f3e651df640 1 Processor -- start 2024-04-25T10:07:54.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.706+0000 7f3e651df640 1 -- start start 2024-04-25T10:07:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.706+0000 7f3e651df640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3e600d0a50 0x7f3e600b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.706+0000 7f3e651df640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3e600ce2b0 0x7f3e600ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.706+0000 7f3e651df640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e6006d200 con 0x7f3e6015deb0 2024-04-25T10:07:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.706+0000 7f3e651df640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e600cf2c0 con 0x7f3e600d0a50 2024-04-25T10:07:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.706+0000 7f3e651df640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e600cefe0 con 0x7f3e600ce2b0 2024-04-25T10:07:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.706+0000 7f3e5e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3e600d0a50 0x7f3e600b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.706+0000 7f3e5ed9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3e6015deb0 0x7f3e600d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42924/0 (socket says 172.21.15.60:42924) 2024-04-25T10:07:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.706+0000 7f3e5ed9d640 1 -- 172.21.15.60:0/1703570261 learned_addr learned my addr 172.21.15.60:0/1703570261 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.706+0000 7f3e5f59e640 1 --2- 172.21.15.60:0/1703570261 >> v2:172.21.15.100:3300/0 conn(0x7f3e600ce2b0 0x7f3e600ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.706+0000 7f3e5dd9b640 1 -- 172.21.15.60:0/1703570261 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897712348 0 0) 0x7f3e6006d200 con 0x7f3e6015deb0 2024-04-25T10:07:54.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.706+0000 7f3e5dd9b640 1 -- 172.21.15.60:0/1703570261 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e48003530 con 0x7f3e6015deb0 2024-04-25T10:07:54.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.710+0000 7f3e5e59c640 1 -- 172.21.15.60:0/1703570261 >> v2:172.21.15.100:3300/0 conn(0x7f3e600ce2b0 msgr2=0x7f3e600ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:54.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.710+0000 7f3e5e59c640 1 --2- 172.21.15.60:0/1703570261 >> v2:172.21.15.100:3300/0 conn(0x7f3e600ce2b0 0x7f3e600ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.710+0000 7f3e5e59c640 1 -- 172.21.15.60:0/1703570261 >> v1:172.21.15.60:6789/0 conn(0x7f3e6015deb0 legacy=0x7f3e600d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.710+0000 7f3e5e59c640 1 -- 172.21.15.60:0/1703570261 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e600d1d70 con 0x7f3e600d0a50 2024-04-25T10:07:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.710+0000 7f3e5e59c640 1 --2- 172.21.15.60:0/1703570261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e600d0a50 0x7f3e600b6c10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3e600d1590 tx=0x7f3e5401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ce91272000fd45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.710+0000 7f3e5dd9b640 1 -- 172.21.15.60:0/1703570261 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e540b0070 con 0x7f3e600d0a50 2024-04-25T10:07:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.710+0000 7f3e5dd9b640 1 -- 172.21.15.60:0/1703570261 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e540ab600 con 0x7f3e600d0a50 2024-04-25T10:07:54.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.710+0000 7f3e5dd9b640 1 -- 172.21.15.60:0/1703570261 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e540ab9b0 con 0x7f3e600d0a50 2024-04-25T10:07:54.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.710+0000 7f3e651df640 1 -- 172.21.15.60:0/1703570261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e600d0a50 msgr2=0x7f3e600b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.710+0000 7f3e651df640 1 --2- 172.21.15.60:0/1703570261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e600d0a50 0x7f3e600b6c10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3e600d1590 tx=0x7f3e5401c480 comp rx=0 tx=0).stop 2024-04-25T10:07:54.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.710+0000 7f3e651df640 1 -- 172.21.15.60:0/1703570261 shutdown_connections 2024-04-25T10:07:54.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.710+0000 7f3e651df640 1 --2- 172.21.15.60:0/1703570261 >> v2:172.21.15.100:3300/0 conn(0x7f3e600ce2b0 0x7f3e600ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.710+0000 7f3e651df640 1 --2- 172.21.15.60:0/1703570261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e600d0a50 0x7f3e600b6c10 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.710+0000 7f3e651df640 1 -- 172.21.15.60:0/1703570261 >> 172.21.15.60:0/1703570261 conn(0x7f3e6006c090 msgr2=0x7f3e6006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:54.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.710+0000 7f3e651df640 1 -- 172.21.15.60:0/1703570261 shutdown_connections 2024-04-25T10:07:54.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.710+0000 7f3e651df640 1 -- 172.21.15.60:0/1703570261 wait complete. 2024-04-25T10:07:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e651df640 1 Processor -- start 2024-04-25T10:07:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e651df640 1 -- start start 2024-04-25T10:07:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e651df640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e600ce2b0 0x7f3e601591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e651df640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e600d0a50 0x7f3e600c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e651df640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e6015deb0 0x7f3e6014e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e651df640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e600d31e0 con 0x7f3e600ce2b0 2024-04-25T10:07:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e651df640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e600d3060 con 0x7f3e600d0a50 2024-04-25T10:07:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e651df640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e600d3360 con 0x7f3e6015deb0 2024-04-25T10:07:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e5ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e600ce2b0 0x7f3e601591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e5ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e600ce2b0 0x7f3e601591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51094/0 (socket says 172.21.15.60:51094) 2024-04-25T10:07:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e5ed9d640 1 -- 172.21.15.60:0/3507105592 learned_addr learned my addr 172.21.15.60:0/3507105592 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e5ed9d640 1 -- 172.21.15.60:0/3507105592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e6015deb0 msgr2=0x7f3e6014e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e5ed9d640 1 --2- 172.21.15.60:0/3507105592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e6015deb0 0x7f3e6014e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e5ed9d640 1 -- 172.21.15.60:0/3507105592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e600d0a50 msgr2=0x7f3e600c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e5e59c640 1 --2- 172.21.15.60:0/3507105592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e600d0a50 0x7f3e600c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e5ed9d640 1 --2- 172.21.15.60:0/3507105592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e600d0a50 0x7f3e600c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e5ed9d640 1 -- 172.21.15.60:0/3507105592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e60159cf0 con 0x7f3e600ce2b0 2024-04-25T10:07:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e5ed9d640 1 --2- 172.21.15.60:0/3507105592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e600ce2b0 0x7f3e601591c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000c3f0 tx=0x7f3e5000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e477fe640 1 -- 172.21.15.60:0/3507105592 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e50017070 con 0x7f3e600ce2b0 2024-04-25T10:07:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e477fe640 1 -- 172.21.15.60:0/3507105592 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e50005cf0 con 0x7f3e600ce2b0 2024-04-25T10:07:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e651df640 1 -- 172.21.15.60:0/3507105592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e60159fd0 con 0x7f3e600ce2b0 2024-04-25T10:07:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e477fe640 1 -- 172.21.15.60:0/3507105592 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e500072f0 con 0x7f3e600ce2b0 2024-04-25T10:07:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e651df640 1 -- 172.21.15.60:0/3507105592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6015a590 con 0x7f3e600ce2b0 2024-04-25T10:07:54.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e651df640 1 -- 172.21.15.60:0/3507105592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e60074770 con 0x7f3e600ce2b0 2024-04-25T10:07:54.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e477fe640 1 -- 172.21.15.60:0/3507105592 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e5001a070 con 0x7f3e600ce2b0 2024-04-25T10:07:54.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.714+0000 7f3e477fe640 1 --2- 172.21.15.60:0/3507105592 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3e4c066f90 0x7f3e4c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.718+0000 7f3e477fe640 1 -- 172.21.15.60:0/3507105592 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e500843b0 con 0x7f3e600ce2b0 2024-04-25T10:07:54.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.718+0000 7f3e5e59c640 1 --2- 172.21.15.60:0/3507105592 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3e4c066f90 0x7f3e4c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.718+0000 7f3e5e59c640 1 --2- 172.21.15.60:0/3507105592 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3e4c066f90 0x7f3e4c069450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3e540ad000 tx=0x7f3e54008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.722+0000 7f3e477fe640 1 -- 172.21.15.60:0/3507105592 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e50055450 con 0x7f3e600ce2b0 2024-04-25T10:07:54.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.890+0000 7f3e651df640 1 -- 172.21.15.60:0/3507105592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3e600cfc20 con 0x7f3e600ce2b0 2024-04-25T10:07:54.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.894+0000 7f3e477fe640 1 -- 172.21.15.60:0/3507105592 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3e50054d50 con 0x7f3e600ce2b0 2024-04-25T10:07:54.901 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:07:54.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.894+0000 7f3e651df640 1 -- 172.21.15.60:0/3507105592 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3e4c066f90 msgr2=0x7f3e4c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.894+0000 7f3e651df640 1 --2- 172.21.15.60:0/3507105592 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3e4c066f90 0x7f3e4c069450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3e540ad000 tx=0x7f3e54008040 comp rx=0 tx=0).stop 2024-04-25T10:07:54.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.894+0000 7f3e651df640 1 -- 172.21.15.60:0/3507105592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e600ce2b0 msgr2=0x7f3e601591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.894+0000 7f3e651df640 1 --2- 172.21.15.60:0/3507105592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e600ce2b0 0x7f3e601591c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000c3f0 tx=0x7f3e5000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.894+0000 7f3e651df640 1 -- 172.21.15.60:0/3507105592 shutdown_connections 2024-04-25T10:07:54.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.894+0000 7f3e651df640 1 --2- 172.21.15.60:0/3507105592 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3e4c066f90 0x7f3e4c069450 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.894+0000 7f3e651df640 1 --2- 172.21.15.60:0/3507105592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e6015deb0 0x7f3e6014e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.894+0000 7f3e651df640 1 --2- 172.21.15.60:0/3507105592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e600d0a50 0x7f3e600c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.894+0000 7f3e651df640 1 --2- 172.21.15.60:0/3507105592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e600ce2b0 0x7f3e601591c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.894+0000 7f3e651df640 1 -- 172.21.15.60:0/3507105592 >> 172.21.15.60:0/3507105592 conn(0x7f3e6006c090 msgr2=0x7f3e600d0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:54.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.894+0000 7f3e651df640 1 -- 172.21.15.60:0/3507105592 shutdown_connections 2024-04-25T10:07:54.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:54.894+0000 7f3e651df640 1 -- 172.21.15.60:0/3507105592 wait complete. 2024-04-25T10:07:54.919 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:07:54.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:54.920 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:54.928 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[38583]: fuse finished with error 0 and tester_r 0 2024-04-25T10:07:54.929 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:07:54.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:54.930 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:07:55.100 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:07:55.100 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:55.107 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:55.107 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:07:55.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:55.107 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:07:55.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:07:55.155 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:07:55.156 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:07:55.156 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:55.162 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:07:55.162 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:07:55.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:55.163 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:07:55.208 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:07:55.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:07:55.210 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:07:55.210 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:55.215 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:55.215 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:07:55.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:55.216 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:07:55.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:07:55.261 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:07:55.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:55.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.382+0000 7f4e5e6ef640 1 Processor -- start 2024-04-25T10:07:55.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.382+0000 7f4e5e6ef640 1 -- start start 2024-04-25T10:07:55.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e5e6ef640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e580ce7a0 0x7f4e580ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e5e6ef640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e5815bc90 0x7f4e580b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e5e6ef640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4e580b4660 0x7f4e580b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e5e6ef640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e580cc370 con 0x7f4e580ce7a0 2024-04-25T10:07:55.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e5e6ef640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e580cf0c0 con 0x7f4e580b4660 2024-04-25T10:07:55.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e5e6ef640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e58071ca0 con 0x7f4e5815bc90 2024-04-25T10:07:55.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e57fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e580ce7a0 0x7f4e580ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e57fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4e580ce7a0 0x7f4e580ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51110/0 (socket says 172.21.15.60:51110) 2024-04-25T10:07:55.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e57fff640 1 -- 172.21.15.60:0/20872066 learned_addr learned my addr 172.21.15.60:0/20872066 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:55.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e5cc8c640 1 --2- 172.21.15.60:0/20872066 >> v2:172.21.15.100:3300/0 conn(0x7f4e580b4660 0x7f4e580b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e57fff640 1 -- 172.21.15.60:0/20872066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e5815bc90 msgr2=0x7f4e580b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:55.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e57fff640 1 --2- 172.21.15.60:0/20872066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e5815bc90 0x7f4e580b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e57fff640 1 -- 172.21.15.60:0/20872066 >> v2:172.21.15.100:3300/0 conn(0x7f4e580b4660 msgr2=0x7f4e580b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:55.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e57fff640 1 --2- 172.21.15.60:0/20872066 >> v2:172.21.15.100:3300/0 conn(0x7f4e580b4660 0x7f4e580b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e57fff640 1 -- 172.21.15.60:0/20872066 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e580cca40 con 0x7f4e580ce7a0 2024-04-25T10:07:55.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e5cc8c640 1 --2- 172.21.15.60:0/20872066 >> v2:172.21.15.100:3300/0 conn(0x7f4e580b4660 0x7f4e580b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e57fff640 1 --2- 172.21.15.60:0/20872066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580ce7a0 0x7f4e580ceb80 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009e50 tx=0x7f4e4801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191c7e7656f38d2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e56ffd640 1 -- 172.21.15.60:0/20872066 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480a5670 con 0x7f4e580ce7a0 2024-04-25T10:07:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e56ffd640 1 -- 172.21.15.60:0/20872066 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e480a5810 con 0x7f4e580ce7a0 2024-04-25T10:07:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.386+0000 7f4e56ffd640 1 -- 172.21.15.60:0/20872066 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480a5be0 con 0x7f4e580ce7a0 2024-04-25T10:07:55.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 -- 172.21.15.60:0/20872066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580ce7a0 msgr2=0x7f4e580ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 --2- 172.21.15.60:0/20872066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580ce7a0 0x7f4e580ceb80 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009e50 tx=0x7f4e4801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 -- 172.21.15.60:0/20872066 shutdown_connections 2024-04-25T10:07:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 --2- 172.21.15.60:0/20872066 >> v2:172.21.15.100:3300/0 conn(0x7f4e580b4660 0x7f4e580b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 --2- 172.21.15.60:0/20872066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e5815bc90 0x7f4e580b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 --2- 172.21.15.60:0/20872066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580ce7a0 0x7f4e580ceb80 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 -- 172.21.15.60:0/20872066 >> 172.21.15.60:0/20872066 conn(0x7f4e580b12d0 msgr2=0x7f4e580b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 -- 172.21.15.60:0/20872066 shutdown_connections 2024-04-25T10:07:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 -- 172.21.15.60:0/20872066 wait complete. 2024-04-25T10:07:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 Processor -- start 2024-04-25T10:07:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 -- start start 2024-04-25T10:07:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580ce7a0 0x7f4e5815ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e5815bc90 0x7f4e5815d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e581513a0 0x7f4e58159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580b3db0 con 0x7f4e580ce7a0 2024-04-25T10:07:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580b3460 con 0x7f4e581513a0 2024-04-25T10:07:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580b4280 con 0x7f4e5815bc90 2024-04-25T10:07:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e57fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580ce7a0 0x7f4e5815ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e57fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580ce7a0 0x7f4e5815ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51122/0 (socket says 172.21.15.60:51122) 2024-04-25T10:07:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e57fff640 1 -- 172.21.15.60:0/3004760582 learned_addr learned my addr 172.21.15.60:0/3004760582 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5cc8c640 1 --2- 172.21.15.60:0/3004760582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e581513a0 0x7f4e58159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e577fe640 1 --2- 172.21.15.60:0/3004760582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e5815bc90 0x7f4e5815d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e57fff640 1 -- 172.21.15.60:0/3004760582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e5815bc90 msgr2=0x7f4e5815d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e57fff640 1 --2- 172.21.15.60:0/3004760582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e5815bc90 0x7f4e5815d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e57fff640 1 -- 172.21.15.60:0/3004760582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e581513a0 msgr2=0x7f4e58159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e57fff640 1 --2- 172.21.15.60:0/3004760582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e581513a0 0x7f4e58159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e57fff640 1 -- 172.21.15.60:0/3004760582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5815acf0 con 0x7f4e580ce7a0 2024-04-25T10:07:55.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e577fe640 1 --2- 172.21.15.60:0/3004760582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e5815bc90 0x7f4e5815d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:55.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e57fff640 1 --2- 172.21.15.60:0/3004760582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580ce7a0 0x7f4e5815ce10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4e480af040 tx=0x7f4e48019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e54ff9640 1 -- 172.21.15.60:0/3004760582 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480bc020 con 0x7f4e580ce7a0 2024-04-25T10:07:55.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e54ff9640 1 -- 172.21.15.60:0/3004760582 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e480b8690 con 0x7f4e580ce7a0 2024-04-25T10:07:55.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 -- 172.21.15.60:0/3004760582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5815afd0 con 0x7f4e580ce7a0 2024-04-25T10:07:55.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e54ff9640 1 -- 172.21.15.60:0/3004760582 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480b8a80 con 0x7f4e580ce7a0 2024-04-25T10:07:55.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 -- 172.21.15.60:0/3004760582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5815b590 con 0x7f4e580ce7a0 2024-04-25T10:07:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.390+0000 7f4e5e6ef640 1 -- 172.21.15.60:0/3004760582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e58074770 con 0x7f4e580ce7a0 2024-04-25T10:07:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.394+0000 7f4e54ff9640 1 -- 172.21.15.60:0/3004760582 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4e480bf020 con 0x7f4e580ce7a0 2024-04-25T10:07:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.394+0000 7f4e54ff9640 1 --2- 172.21.15.60:0/3004760582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e38066f60 0x7f4e38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.394+0000 7f4e54ff9640 1 -- 172.21.15.60:0/3004760582 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e481200e0 con 0x7f4e580ce7a0 2024-04-25T10:07:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.394+0000 7f4e577fe640 1 --2- 172.21.15.60:0/3004760582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e38066f60 0x7f4e38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.394+0000 7f4e577fe640 1 --2- 172.21.15.60:0/3004760582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e38066f60 0x7f4e38069420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4e580af3d0 tx=0x7f4e40008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.398+0000 7f4e54ff9640 1 -- 172.21.15.60:0/3004760582 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4e480c6030 con 0x7f4e580ce7a0 2024-04-25T10:07:55.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.590+0000 7f4e5e6ef640 1 -- 172.21.15.60:0/3004760582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e58159d10 con 0x7f4e580ce7a0 2024-04-25T10:07:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.590+0000 7f4e54ff9640 1 -- 172.21.15.60:0/3004760582 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f4e480165e0 con 0x7f4e580ce7a0 2024-04-25T10:07:55.599 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:55.599 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1138162580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1138162580},{"type":"v1","addr":"172.21.15.100:6837","nonce":1138162580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2985559168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2985559168},{"type":"v1","addr":"172.21.15.100:6835","nonce":2985559168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/50795370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":50795370},{"type":"v1","addr":"172.21.15.60:6834","nonce":50795370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:07.980095+0000","modified":"2024-04-25T10:07:10.002670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5796":{"gid":5796,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1747757196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1747757196},{"type":"v1","addr":"172.21.15.60:6835","nonce":1747757196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:55.600 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 21 2024-04-25T10:07:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.594+0000 7f4e5e6ef640 1 -- 172.21.15.60:0/3004760582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e38066f60 msgr2=0x7f4e38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.594+0000 7f4e5e6ef640 1 --2- 172.21.15.60:0/3004760582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e38066f60 0x7f4e38069420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4e580af3d0 tx=0x7f4e40008040 comp rx=0 tx=0).stop 2024-04-25T10:07:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.594+0000 7f4e5e6ef640 1 -- 172.21.15.60:0/3004760582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580ce7a0 msgr2=0x7f4e5815ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.594+0000 7f4e5e6ef640 1 --2- 172.21.15.60:0/3004760582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580ce7a0 0x7f4e5815ce10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4e480af040 tx=0x7f4e48019790 comp rx=0 tx=0).stop 2024-04-25T10:07:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.594+0000 7f4e5e6ef640 1 -- 172.21.15.60:0/3004760582 shutdown_connections 2024-04-25T10:07:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.594+0000 7f4e5e6ef640 1 --2- 172.21.15.60:0/3004760582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4e38066f60 0x7f4e38069420 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.594+0000 7f4e5e6ef640 1 --2- 172.21.15.60:0/3004760582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e581513a0 0x7f4e58159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.594+0000 7f4e5e6ef640 1 --2- 172.21.15.60:0/3004760582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e5815bc90 0x7f4e5815d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.594+0000 7f4e5e6ef640 1 --2- 172.21.15.60:0/3004760582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e580ce7a0 0x7f4e5815ce10 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.594+0000 7f4e5e6ef640 1 -- 172.21.15.60:0/3004760582 >> 172.21.15.60:0/3004760582 conn(0x7f4e580b12d0 msgr2=0x7f4e580b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.594+0000 7f4e5e6ef640 1 -- 172.21.15.60:0/3004760582 shutdown_connections 2024-04-25T10:07:55.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.594+0000 7f4e5e6ef640 1 -- 172.21.15.60:0/3004760582 wait complete. 2024-04-25T10:07:55.619 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:55.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.734+0000 7f768f63b640 1 Processor -- start 2024-04-25T10:07:55.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.734+0000 7f768f63b640 1 -- start start 2024-04-25T10:07:55.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f768f63b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7688162250 0x7f76880b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f768f63b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76880b3aa0 con 0x7f76880b34f0 2024-04-25T10:07:55.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f768f63b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76880b3d80 con 0x7f76880719a0 2024-04-25T10:07:55.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f768f63b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76880af0c0 con 0x7f7688162250 2024-04-25T10:07:55.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f768d3d7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7688162250 0x7f76880b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f768d3d7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7688162250 0x7f76880b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51136/0 (socket says 172.21.15.60:51136) 2024-04-25T10:07:55.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f768d3d7640 1 -- 172.21.15.60:0/205620536 learned_addr learned my addr 172.21.15.60:0/205620536 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:55.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f768d3d7640 1 -- 172.21.15.60:0/205620536 >> v1:172.21.15.100:6789/0 conn(0x7f76880719a0 legacy=0x7f76880cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:55.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f768d3d7640 1 -- 172.21.15.60:0/205620536 >> v1:172.21.15.60:6789/0 conn(0x7f76880b34f0 legacy=0x7f7688071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f7687fff640 1 -- 172.21.15.60:0/205620536 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 451661631 0 0) 0x7f76880b3aa0 con 0x7f76880b34f0 2024-04-25T10:07:55.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f768d3d7640 1 -- 172.21.15.60:0/205620536 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76880b4060 con 0x7f7688162250 2024-04-25T10:07:55.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f768d3d7640 1 --2- 172.21.15.60:0/205620536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7688162250 0x7f76880b2fb0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f767c009e50 tx=0x7f767c01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4240859f2b084784 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f7687fff640 1 -- 172.21.15.60:0/205620536 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c0b4070 con 0x7f7688162250 2024-04-25T10:07:55.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f7687fff640 1 -- 172.21.15.60:0/205620536 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767c0af400 con 0x7f7688162250 2024-04-25T10:07:55.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f7687fff640 1 -- 172.21.15.60:0/205620536 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c0af7d0 con 0x7f7688162250 2024-04-25T10:07:55.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f768f63b640 1 -- 172.21.15.60:0/205620536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7688162250 msgr2=0x7f76880b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f768f63b640 1 --2- 172.21.15.60:0/205620536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7688162250 0x7f76880b2fb0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f767c009e50 tx=0x7f767c01f1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f768f63b640 1 -- 172.21.15.60:0/205620536 shutdown_connections 2024-04-25T10:07:55.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f768f63b640 1 --2- 172.21.15.60:0/205620536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7688162250 0x7f76880b2fb0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.738+0000 7f768f63b640 1 -- 172.21.15.60:0/205620536 >> 172.21.15.60:0/205620536 conn(0x7f76880b12d0 msgr2=0x7f76880b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:55.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768f63b640 1 -- 172.21.15.60:0/205620536 shutdown_connections 2024-04-25T10:07:55.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768f63b640 1 -- 172.21.15.60:0/205620536 wait complete. 2024-04-25T10:07:55.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768f63b640 1 Processor -- start 2024-04-25T10:07:55.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768f63b640 1 -- start start 2024-04-25T10:07:55.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768f63b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76880719a0 0x7f7688150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768f63b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76880b34f0 0x7f76881f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768f63b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7688162250 0x7f76880a8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768f63b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76880cc440 con 0x7f76880719a0 2024-04-25T10:07:55.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768f63b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768806e5a0 con 0x7f76880b34f0 2024-04-25T10:07:55.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768f63b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76880cd0a0 con 0x7f7688162250 2024-04-25T10:07:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768d3d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76880719a0 0x7f7688150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768d3d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76880719a0 0x7f7688150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51140/0 (socket says 172.21.15.60:51140) 2024-04-25T10:07:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768d3d7640 1 -- 172.21.15.60:0/3384796775 learned_addr learned my addr 172.21.15.60:0/3384796775 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768cbd6640 1 --2- 172.21.15.60:0/3384796775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76880b34f0 0x7f76881f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768dbd8640 1 --2- 172.21.15.60:0/3384796775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7688162250 0x7f76880a8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768d3d7640 1 -- 172.21.15.60:0/3384796775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7688162250 msgr2=0x7f76880a8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768d3d7640 1 --2- 172.21.15.60:0/3384796775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7688162250 0x7f76880a8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768d3d7640 1 -- 172.21.15.60:0/3384796775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76880b34f0 msgr2=0x7f76881f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768d3d7640 1 --2- 172.21.15.60:0/3384796775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76880b34f0 0x7f76881f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768d3d7640 1 -- 172.21.15.60:0/3384796775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768814d520 con 0x7f76880719a0 2024-04-25T10:07:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768cbd6640 1 --2- 172.21.15.60:0/3384796775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76880b34f0 0x7f76881f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768d3d7640 1 --2- 172.21.15.60:0/3384796775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76880719a0 0x7f7688150970 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f767c0b2040 tx=0x7f767c01bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f7685ffb640 1 -- 172.21.15.60:0/3384796775 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c0b4040 con 0x7f76880719a0 2024-04-25T10:07:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f7685ffb640 1 -- 172.21.15.60:0/3384796775 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767c0c1070 con 0x7f76880719a0 2024-04-25T10:07:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f7685ffb640 1 -- 172.21.15.60:0/3384796775 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c0b96a0 con 0x7f76880719a0 2024-04-25T10:07:55.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768f63b640 1 -- 172.21.15.60:0/3384796775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f768814d7c0 con 0x7f76880719a0 2024-04-25T10:07:55.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.742+0000 7f768f63b640 1 -- 172.21.15.60:0/3384796775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768814dd80 con 0x7f76880719a0 2024-04-25T10:07:55.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.746+0000 7f768f63b640 1 -- 172.21.15.60:0/3384796775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7688074770 con 0x7f76880719a0 2024-04-25T10:07:55.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.746+0000 7f7685ffb640 1 -- 172.21.15.60:0/3384796775 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f767c0b7070 con 0x7f76880719a0 2024-04-25T10:07:55.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.746+0000 7f7685ffb640 1 --2- 172.21.15.60:0/3384796775 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7674066f60 0x7f7674069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.746+0000 7f7685ffb640 1 -- 172.21.15.60:0/3384796775 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f767c123250 con 0x7f76880719a0 2024-04-25T10:07:55.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.746+0000 7f768cbd6640 1 --2- 172.21.15.60:0/3384796775 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7674066f60 0x7f7674069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.746+0000 7f768cbd6640 1 --2- 172.21.15.60:0/3384796775 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7674066f60 0x7f7674069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f76881faef0 tx=0x7f7680009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.750+0000 7f7685ffb640 1 -- 172.21.15.60:0/3384796775 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f767c018d40 con 0x7f76880719a0 2024-04-25T10:07:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.902+0000 7f768f63b640 1 -- 172.21.15.60:0/3384796775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f768814e160 con 0x7f76880719a0 2024-04-25T10:07:55.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.906+0000 7f7685ffb640 1 -- 172.21.15.60:0/3384796775 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16625 (secure 0 0 0) 0x7f767c0f4440 con 0x7f76880719a0 2024-04-25T10:07:55.913 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:55.913 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":47,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:07:09.646687+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.652298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:06.983225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:55.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.906+0000 7f768f63b640 1 -- 172.21.15.60:0/3384796775 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7674066f60 msgr2=0x7f7674069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.906+0000 7f768f63b640 1 --2- 172.21.15.60:0/3384796775 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7674066f60 0x7f7674069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f76881faef0 tx=0x7f7680009040 comp rx=0 tx=0).stop 2024-04-25T10:07:55.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.906+0000 7f768f63b640 1 -- 172.21.15.60:0/3384796775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76880719a0 msgr2=0x7f7688150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.906+0000 7f768f63b640 1 --2- 172.21.15.60:0/3384796775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76880719a0 0x7f7688150970 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f767c0b2040 tx=0x7f767c01bf80 comp rx=0 tx=0).stop 2024-04-25T10:07:55.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.906+0000 7f768f63b640 1 -- 172.21.15.60:0/3384796775 shutdown_connections 2024-04-25T10:07:55.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.906+0000 7f768f63b640 1 --2- 172.21.15.60:0/3384796775 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7674066f60 0x7f7674069420 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.906+0000 7f768f63b640 1 --2- 172.21.15.60:0/3384796775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7688162250 0x7f76880a8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.906+0000 7f768f63b640 1 --2- 172.21.15.60:0/3384796775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76880b34f0 0x7f76881f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.906+0000 7f768f63b640 1 --2- 172.21.15.60:0/3384796775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76880719a0 0x7f7688150970 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.906+0000 7f768f63b640 1 -- 172.21.15.60:0/3384796775 >> 172.21.15.60:0/3384796775 conn(0x7f76880b12d0 msgr2=0x7f76880b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:55.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.906+0000 7f768f63b640 1 -- 172.21.15.60:0/3384796775 shutdown_connections 2024-04-25T10:07:55.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:55.906+0000 7f768f63b640 1 -- 172.21.15.60:0/3384796775 wait complete. 2024-04-25T10:07:55.931 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:05.652298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:06.983225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:55.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:56.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe896fe640 1 Processor -- start 2024-04-25T10:07:56.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe896fe640 1 -- start start 2024-04-25T10:07:56.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe896fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbe840b2860 0x7fbe840d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe896fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbe840d0fe0 0x7fbe840cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe896fe640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8406d220 con 0x7fbe8415dcd0 2024-04-25T10:07:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe896fe640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe840cfa40 con 0x7fbe840b2860 2024-04-25T10:07:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe896fe640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe840cfce0 con 0x7fbe840d0fe0 2024-04-25T10:07:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe827fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbe840b2860 0x7fbe840d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe827fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbe840b2860 0x7fbe840d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51142/0 (socket says 172.21.15.60:51142) 2024-04-25T10:07:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe827fc640 1 -- 172.21.15.60:0/3985740080 learned_addr learned my addr 172.21.15.60:0/3985740080 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe82ffd640 1 --1- 172.21.15.60:0/3985740080 >> v1:172.21.15.60:6789/0 conn(0x7fbe8415dcd0 0x7fbe8406c940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42936/0 (socket says 172.21.15.60:42936) 2024-04-25T10:07:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe837fe640 1 --2- 172.21.15.60:0/3985740080 >> v2:172.21.15.100:3300/0 conn(0x7fbe840d0fe0 0x7fbe840cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe827fc640 1 -- 172.21.15.60:0/3985740080 >> v2:172.21.15.100:3300/0 conn(0x7fbe840d0fe0 msgr2=0x7fbe840cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe827fc640 1 --2- 172.21.15.60:0/3985740080 >> v2:172.21.15.100:3300/0 conn(0x7fbe840d0fe0 0x7fbe840cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe827fc640 1 -- 172.21.15.60:0/3985740080 >> v1:172.21.15.60:6789/0 conn(0x7fbe8415dcd0 legacy=0x7fbe8406c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe81ffb640 1 -- 172.21.15.60:0/3985740080 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212984926 0 0) 0x7fbe8406d220 con 0x7fbe8415dcd0 2024-04-25T10:07:56.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.042+0000 7fbe827fc640 1 -- 172.21.15.60:0/3985740080 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe840cffc0 con 0x7fbe840b2860 2024-04-25T10:07:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe827fc640 1 --2- 172.21.15.60:0/3985740080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe840b2860 0x7fbe840d0aa0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fbe8406f250 tx=0x7fbe7801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb37275f571b13ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe81ffb640 1 -- 172.21.15.60:0/3985740080 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe780b0070 con 0x7fbe840b2860 2024-04-25T10:07:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe81ffb640 1 -- 172.21.15.60:0/3985740080 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe780ab600 con 0x7fbe840b2860 2024-04-25T10:07:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe81ffb640 1 -- 172.21.15.60:0/3985740080 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe780ab9d0 con 0x7fbe840b2860 2024-04-25T10:07:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe896fe640 1 -- 172.21.15.60:0/3985740080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe840b2860 msgr2=0x7fbe840d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe896fe640 1 --2- 172.21.15.60:0/3985740080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe840b2860 0x7fbe840d0aa0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fbe8406f250 tx=0x7fbe7801c490 comp rx=0 tx=0).stop 2024-04-25T10:07:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe896fe640 1 -- 172.21.15.60:0/3985740080 shutdown_connections 2024-04-25T10:07:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe896fe640 1 --2- 172.21.15.60:0/3985740080 >> v2:172.21.15.100:3300/0 conn(0x7fbe840d0fe0 0x7fbe840cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe896fe640 1 --2- 172.21.15.60:0/3985740080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe840b2860 0x7fbe840d0aa0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe896fe640 1 -- 172.21.15.60:0/3985740080 >> 172.21.15.60:0/3985740080 conn(0x7fbe840b3200 msgr2=0x7fbe840b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe896fe640 1 -- 172.21.15.60:0/3985740080 shutdown_connections 2024-04-25T10:07:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe896fe640 1 -- 172.21.15.60:0/3985740080 wait complete. 2024-04-25T10:07:56.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe896fe640 1 Processor -- start 2024-04-25T10:07:56.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe896fe640 1 -- start start 2024-04-25T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe896fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe840b2860 0x7fbe840c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe896fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe840d0fe0 0x7fbe84158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe896fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe8415dcd0 0x7fbe8414b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe896fe640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8416adc0 con 0x7fbe840d0fe0 2024-04-25T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe896fe640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8416ac40 con 0x7fbe840b2860 2024-04-25T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe896fe640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8416af40 con 0x7fbe8415dcd0 2024-04-25T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe827fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe840d0fe0 0x7fbe84158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe827fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe840d0fe0 0x7fbe84158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51154/0 (socket says 172.21.15.60:51154) 2024-04-25T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.046+0000 7fbe827fc640 1 -- 172.21.15.60:0/1297335097 learned_addr learned my addr 172.21.15.60:0/1297335097 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe82ffd640 1 --2- 172.21.15.60:0/1297335097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe840b2860 0x7fbe840c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe837fe640 1 --2- 172.21.15.60:0/1297335097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe8415dcd0 0x7fbe8414b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe827fc640 1 -- 172.21.15.60:0/1297335097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe8415dcd0 msgr2=0x7fbe8414b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe827fc640 1 --2- 172.21.15.60:0/1297335097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe8415dcd0 0x7fbe8414b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe827fc640 1 -- 172.21.15.60:0/1297335097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe840b2860 msgr2=0x7fbe840c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe827fc640 1 --2- 172.21.15.60:0/1297335097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe840b2860 0x7fbe840c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe827fc640 1 -- 172.21.15.60:0/1297335097 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe840b6d50 con 0x7fbe840d0fe0 2024-04-25T10:07:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe827fc640 1 --2- 172.21.15.60:0/1297335097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe840d0fe0 0x7fbe84158d90 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbe780190a0 tx=0x7fbe78019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe6b7fe640 1 -- 172.21.15.60:0/1297335097 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe780b0040 con 0x7fbe840d0fe0 2024-04-25T10:07:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe6b7fe640 1 -- 172.21.15.60:0/1297335097 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe780b4620 con 0x7fbe840d0fe0 2024-04-25T10:07:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe896fe640 1 -- 172.21.15.60:0/1297335097 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe840b51c0 con 0x7fbe840d0fe0 2024-04-25T10:07:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe6b7fe640 1 -- 172.21.15.60:0/1297335097 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe780b4a10 con 0x7fbe840d0fe0 2024-04-25T10:07:56.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe896fe640 1 -- 172.21.15.60:0/1297335097 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe840c8db0 con 0x7fbe840d0fe0 2024-04-25T10:07:56.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe896fe640 1 -- 172.21.15.60:0/1297335097 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe840b4fb0 con 0x7fbe840d0fe0 2024-04-25T10:07:56.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe6b7fe640 1 -- 172.21.15.60:0/1297335097 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe780ab4c0 con 0x7fbe840d0fe0 2024-04-25T10:07:56.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe6b7fe640 1 --2- 172.21.15.60:0/1297335097 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe70066f60 0x7fbe70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe6b7fe640 1 -- 172.21.15.60:0/1297335097 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe7811ec80 con 0x7fbe840d0fe0 2024-04-25T10:07:56.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe82ffd640 1 --2- 172.21.15.60:0/1297335097 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe70066f60 0x7fbe70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.050+0000 7fbe82ffd640 1 --2- 172.21.15.60:0/1297335097 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe70066f60 0x7fbe70069420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbe74000f70 tx=0x7fbe74007c10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.054+0000 7fbe6b7fe640 1 -- 172.21.15.60:0/1297335097 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbe78016110 con 0x7fbe840d0fe0 2024-04-25T10:07:56.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.206+0000 7fbe896fe640 1 -- 172.21.15.60:0/1297335097 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe84074770 con 0x7fbe840d0fe0 2024-04-25T10:07:56.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.210+0000 7fbe6b7fe640 1 -- 172.21.15.60:0/1297335097 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe780ba2b0 con 0x7fbe840d0fe0 2024-04-25T10:07:56.216 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:07:56.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.210+0000 7fbe896fe640 1 -- 172.21.15.60:0/1297335097 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe70066f60 msgr2=0x7fbe70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.210+0000 7fbe896fe640 1 --2- 172.21.15.60:0/1297335097 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe70066f60 0x7fbe70069420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbe74000f70 tx=0x7fbe74007c10 comp rx=0 tx=0).stop 2024-04-25T10:07:56.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.210+0000 7fbe896fe640 1 -- 172.21.15.60:0/1297335097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe840d0fe0 msgr2=0x7fbe84158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.210+0000 7fbe896fe640 1 --2- 172.21.15.60:0/1297335097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe840d0fe0 0x7fbe84158d90 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbe780190a0 tx=0x7fbe78019230 comp rx=0 tx=0).stop 2024-04-25T10:07:56.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.210+0000 7fbe896fe640 1 -- 172.21.15.60:0/1297335097 shutdown_connections 2024-04-25T10:07:56.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.210+0000 7fbe896fe640 1 --2- 172.21.15.60:0/1297335097 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe70066f60 0x7fbe70069420 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.210+0000 7fbe896fe640 1 --2- 172.21.15.60:0/1297335097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe8415dcd0 0x7fbe8414b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.210+0000 7fbe896fe640 1 --2- 172.21.15.60:0/1297335097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe840d0fe0 0x7fbe84158d90 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.210+0000 7fbe896fe640 1 --2- 172.21.15.60:0/1297335097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe840b2860 0x7fbe840c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.210+0000 7fbe896fe640 1 -- 172.21.15.60:0/1297335097 >> 172.21.15.60:0/1297335097 conn(0x7fbe840b3200 msgr2=0x7fbe840b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:56.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.210+0000 7fbe896fe640 1 -- 172.21.15.60:0/1297335097 shutdown_connections 2024-04-25T10:07:56.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.210+0000 7fbe896fe640 1 -- 172.21.15.60:0/1297335097 wait complete. 2024-04-25T10:07:56.234 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:56.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f0a06d00640 1 Processor -- start 2024-04-25T10:07:56.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f0a06d00640 1 -- start start 2024-04-25T10:07:56.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f0a06d00640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0a000cf8a0 0x7f0a0015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f0a06d00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a000cd3d0 0x7f0a0015be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f0a06d00640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a000cd890 con 0x7f0a0015dd30 2024-04-25T10:07:56.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f0a06d00640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a000ce4a0 con 0x7f0a000cf8a0 2024-04-25T10:07:56.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f0a06d00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a000cdb70 con 0x7f0a000cd3d0 2024-04-25T10:07:56.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f0a04a9c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0a0015dd30 0x7f0a000cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42942/0 (socket says 172.21.15.60:42942) 2024-04-25T10:07:56.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f0a04a9c640 1 -- 172.21.15.60:0/578248239 learned_addr learned my addr 172.21.15.60:0/578248239 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:56.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09fffff640 1 --2- 172.21.15.60:0/578248239 >> v2:172.21.15.100:3300/0 conn(0x7f0a000cf8a0 0x7f0a0015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f0a0529d640 1 --2- 172.21.15.60:0/578248239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a000cd3d0 0x7f0a0015be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09ff7fe640 1 -- 172.21.15.60:0/578248239 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36763446 0 0) 0x7f0a000cd890 con 0x7f0a0015dd30 2024-04-25T10:07:56.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09ff7fe640 1 -- 172.21.15.60:0/578248239 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09e8005740 con 0x7f0a0015dd30 2024-04-25T10:07:56.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09ff7fe640 1 -- 172.21.15.60:0/578248239 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 122532690 0 0) 0x7f09e8005740 con 0x7f0a0015dd30 2024-04-25T10:07:56.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09ff7fe640 1 -- 172.21.15.60:0/578248239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a000cd3d0 msgr2=0x7f0a0015be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:56.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09ff7fe640 1 --2- 172.21.15.60:0/578248239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a000cd3d0 0x7f0a0015be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09ff7fe640 1 -- 172.21.15.60:0/578248239 >> v2:172.21.15.100:3300/0 conn(0x7f0a000cf8a0 msgr2=0x7f0a0015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:56.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09ff7fe640 1 --2- 172.21.15.60:0/578248239 >> v2:172.21.15.100:3300/0 conn(0x7f0a000cf8a0 0x7f0a0015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09ff7fe640 1 -- 172.21.15.60:0/578248239 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a000cde50 con 0x7f0a0015dd30 2024-04-25T10:07:56.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09ff7fe640 1 -- 172.21.15.60:0/578248239 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f09f0001a00 con 0x7f0a0015dd30 2024-04-25T10:07:56.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09ff7fe640 1 -- 172.21.15.60:0/578248239 >> v1:172.21.15.60:6789/0 conn(0x7f0a0015dd30 legacy=0x7f0a000cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09ff7fe640 1 --2- 172.21.15.60:0/578248239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e8005bd0 0x7f09e8005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09ff7fe640 1 --2- 172.21.15.60:0/578248239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09e8006a20 0x7f09e8006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09ff7fe640 1 --2- 172.21.15.60:0/578248239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a000cd3d0 0x7f09e80073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09ff7fe640 1 -- 172.21.15.60:0/578248239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a000ce4a0 con 0x7f09e8005bd0 2024-04-25T10:07:56.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09ff7fe640 1 -- 172.21.15.60:0/578248239 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a000cdb70 con 0x7f09e8006a20 2024-04-25T10:07:56.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09ff7fe640 1 -- 172.21.15.60:0/578248239 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e800dc30 con 0x7f0a000cd3d0 2024-04-25T10:07:56.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f0a04a9c640 1 --2- 172.21.15.60:0/578248239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e8005bd0 0x7f09e8005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f09fffff640 1 --2- 172.21.15.60:0/578248239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09e8006a20 0x7f09e8006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f0a04a9c640 1 -- 172.21.15.60:0/578248239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a000cd3d0 msgr2=0x7f09e80073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:56.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f0a04a9c640 1 --2- 172.21.15.60:0/578248239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a000cd3d0 0x7f09e80073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f0a04a9c640 1 -- 172.21.15.60:0/578248239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09e8006a20 msgr2=0x7f09e8006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:56.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f0a04a9c640 1 --2- 172.21.15.60:0/578248239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09e8006a20 0x7f09e8006ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.342+0000 7f0a04a9c640 1 -- 172.21.15.60:0/578248239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e8005740 con 0x7f09e8005bd0 2024-04-25T10:07:56.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.346+0000 7f0a04a9c640 1 --2- 172.21.15.60:0/578248239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e8005bd0 0x7f09e8005ff0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f09f000a8b0 tx=0x7f09f001d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a53a7dd0b478ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.346+0000 7f09ff7fe640 1 -- 172.21.15.60:0/578248239 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f000a4a0 con 0x7f09e8005bd0 2024-04-25T10:07:56.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.346+0000 7f09ff7fe640 1 -- 172.21.15.60:0/578248239 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f00a5e50 con 0x7f09e8005bd0 2024-04-25T10:07:56.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.346+0000 7f09ff7fe640 1 -- 172.21.15.60:0/578248239 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f00ad890 con 0x7f09e8005bd0 2024-04-25T10:07:56.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.346+0000 7f0a06d00640 1 -- 172.21.15.60:0/578248239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e8005bd0 msgr2=0x7f09e8005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.346+0000 7f0a06d00640 1 --2- 172.21.15.60:0/578248239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09e8005bd0 0x7f09e8005ff0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f09f000a8b0 tx=0x7f09f001d370 comp rx=0 tx=0).stop 2024-04-25T10:07:56.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.346+0000 7f0a0529d640 1 -- 172.21.15.60:0/578248239 reap_dead start 2024-04-25T10:07:56.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.346+0000 7f0a06d00640 1 -- 172.21.15.60:0/578248239 shutdown_connections 2024-04-25T10:07:56.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.346+0000 7f0a06d00640 1 -- 172.21.15.60:0/578248239 >> 172.21.15.60:0/578248239 conn(0x7f0a000670b0 msgr2=0x7f0a0006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:56.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.346+0000 7f0a06d00640 1 -- 172.21.15.60:0/578248239 shutdown_connections 2024-04-25T10:07:56.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.346+0000 7f0a06d00640 1 -- 172.21.15.60:0/578248239 wait complete. 2024-04-25T10:07:56.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.346+0000 7f0a06d00640 1 Processor -- start 2024-04-25T10:07:56.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.346+0000 7f0a06d00640 1 -- start start 2024-04-25T10:07:56.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a06d00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09e8005bd0 0x7f0a0014ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a06d00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a00158830 0x7f0a0014a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a06d00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a0014d490 0x7f0a0014d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a06d00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a00071df0 con 0x7f0a0014d490 2024-04-25T10:07:56.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a06d00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a00071c70 con 0x7f0a00158830 2024-04-25T10:07:56.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a06d00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a00071f70 con 0x7f09e8005bd0 2024-04-25T10:07:56.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a0529d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a0014d490 0x7f0a0014d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a0529d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a0014d490 0x7f0a0014d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51162/0 (socket says 172.21.15.60:51162) 2024-04-25T10:07:56.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a0529d640 1 -- 172.21.15.60:0/772731343 learned_addr learned my addr 172.21.15.60:0/772731343 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f09fffff640 1 --2- 172.21.15.60:0/772731343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a00158830 0x7f0a0014a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a04a9c640 1 --2- 172.21.15.60:0/772731343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09e8005bd0 0x7f0a0014ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a0529d640 1 -- 172.21.15.60:0/772731343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09e8005bd0 msgr2=0x7f0a0014ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a0529d640 1 --2- 172.21.15.60:0/772731343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09e8005bd0 0x7f0a0014ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a0529d640 1 -- 172.21.15.60:0/772731343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a00158830 msgr2=0x7f0a0014a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a0529d640 1 --2- 172.21.15.60:0/772731343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a00158830 0x7f0a0014a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a0529d640 1 -- 172.21.15.60:0/772731343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a00158cb0 con 0x7f0a0014d490 2024-04-25T10:07:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a0529d640 1 --2- 172.21.15.60:0/772731343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a0014d490 0x7f0a0014d940 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0a00072500 tx=0x7f09f800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f09fd7fa640 1 -- 172.21.15.60:0/772731343 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f800c680 con 0x7f0a0014d490 2024-04-25T10:07:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f09fd7fa640 1 -- 172.21.15.60:0/772731343 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f801a070 con 0x7f0a0014d490 2024-04-25T10:07:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a06d00640 1 -- 172.21.15.60:0/772731343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0014e260 con 0x7f0a0014d490 2024-04-25T10:07:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f09fd7fa640 1 -- 172.21.15.60:0/772731343 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f800ca80 con 0x7f0a0014d490 2024-04-25T10:07:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a06d00640 1 -- 172.21.15.60:0/772731343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0014e500 con 0x7f0a0014d490 2024-04-25T10:07:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a06d00640 1 -- 172.21.15.60:0/772731343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a00074770 con 0x7f0a0014d490 2024-04-25T10:07:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f09fd7fa640 1 -- 172.21.15.60:0/772731343 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f09f800cc20 con 0x7f0a0014d490 2024-04-25T10:07:56.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f09fd7fa640 1 --2- 172.21.15.60:0/772731343 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f09ec066f90 0x7f09ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f09fd7fa640 1 -- 172.21.15.60:0/772731343 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09f8084840 con 0x7f0a0014d490 2024-04-25T10:07:56.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a04a9c640 1 --2- 172.21.15.60:0/772731343 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f09ec066f90 0x7f09ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.350+0000 7f0a04a9c640 1 --2- 172.21.15.60:0/772731343 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f09ec066f90 0x7f09ec069450 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f09f00ad000 tx=0x7f0a000ce410 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.354+0000 7f09fd7fa640 1 -- 172.21.15.60:0/772731343 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f09f8023020 con 0x7f0a0014d490 2024-04-25T10:07:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.506+0000 7f0a06d00640 1 -- 172.21.15.60:0/772731343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a000aee90 con 0x7f0a0014d490 2024-04-25T10:07:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.506+0000 7f09fd7fa640 1 -- 172.21.15.60:0/772731343 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f09f8055850 con 0x7f0a0014d490 2024-04-25T10:07:56.516 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:07:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.510+0000 7f0a06d00640 1 -- 172.21.15.60:0/772731343 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f09ec066f90 msgr2=0x7f09ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.510+0000 7f0a06d00640 1 --2- 172.21.15.60:0/772731343 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f09ec066f90 0x7f09ec069450 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f09f00ad000 tx=0x7f0a000ce410 comp rx=0 tx=0).stop 2024-04-25T10:07:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.510+0000 7f0a06d00640 1 -- 172.21.15.60:0/772731343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a0014d490 msgr2=0x7f0a0014d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.510+0000 7f0a06d00640 1 --2- 172.21.15.60:0/772731343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a0014d490 0x7f0a0014d940 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0a00072500 tx=0x7f09f800ec00 comp rx=0 tx=0).stop 2024-04-25T10:07:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.510+0000 7f0a06d00640 1 -- 172.21.15.60:0/772731343 shutdown_connections 2024-04-25T10:07:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.510+0000 7f0a06d00640 1 --2- 172.21.15.60:0/772731343 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f09ec066f90 0x7f09ec069450 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.510+0000 7f0a06d00640 1 --2- 172.21.15.60:0/772731343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a0014d490 0x7f0a0014d940 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.510+0000 7f0a06d00640 1 --2- 172.21.15.60:0/772731343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a00158830 0x7f0a0014a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.510+0000 7f0a06d00640 1 --2- 172.21.15.60:0/772731343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09e8005bd0 0x7f0a0014ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.510+0000 7f0a06d00640 1 -- 172.21.15.60:0/772731343 >> 172.21.15.60:0/772731343 conn(0x7f0a000670b0 msgr2=0x7f0a000af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.510+0000 7f0a06d00640 1 -- 172.21.15.60:0/772731343 shutdown_connections 2024-04-25T10:07:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.510+0000 7f0a06d00640 1 -- 172.21.15.60:0/772731343 wait complete. 2024-04-25T10:07:56.534 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:56.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9fd6a640 1 Processor -- start 2024-04-25T10:07:56.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9fd6a640 1 -- start start 2024-04-25T10:07:56.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9fd6a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbe9815db10 0x7fbe980b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9fd6a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe980aec10 0x7fbe980b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9fd6a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe980ae140 con 0x7fbe980cfcf0 2024-04-25T10:07:56.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9fd6a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe980accc0 con 0x7fbe9815db10 2024-04-25T10:07:56.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9fd6a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9815def0 con 0x7fbe980aec10 2024-04-25T10:07:56.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9e307640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbe980cfcf0 0x7fbe98161660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42944/0 (socket says 172.21.15.60:42944) 2024-04-25T10:07:56.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9e307640 1 -- 172.21.15.60:0/204591411 learned_addr learned my addr 172.21.15.60:0/204591411 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:56.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9db06640 1 --2- 172.21.15.60:0/204591411 >> v2:172.21.15.100:3300/0 conn(0x7fbe9815db10 0x7fbe980b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9d305640 1 --2- 172.21.15.60:0/204591411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe980aec10 0x7fbe980b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9cb04640 1 -- 172.21.15.60:0/204591411 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695253907 0 0) 0x7fbe88001120 con 0x7fbe980cfcf0 2024-04-25T10:07:56.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9cb04640 1 -- 172.21.15.60:0/204591411 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe80003530 con 0x7fbe980cfcf0 2024-04-25T10:07:56.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9cb04640 1 -- 172.21.15.60:0/204591411 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3402524240 0 0) 0x7fbe80003530 con 0x7fbe980cfcf0 2024-04-25T10:07:56.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9cb04640 1 -- 172.21.15.60:0/204591411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe980aec10 msgr2=0x7fbe980b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:56.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9cb04640 1 --2- 172.21.15.60:0/204591411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe980aec10 0x7fbe980b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9cb04640 1 -- 172.21.15.60:0/204591411 >> v2:172.21.15.100:3300/0 conn(0x7fbe9815db10 msgr2=0x7fbe980b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:56.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9cb04640 1 --2- 172.21.15.60:0/204591411 >> v2:172.21.15.100:3300/0 conn(0x7fbe9815db10 0x7fbe980b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9cb04640 1 -- 172.21.15.60:0/204591411 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe98071630 con 0x7fbe980cfcf0 2024-04-25T10:07:56.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9cb04640 1 -- 172.21.15.60:0/204591411 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fbe88001850 con 0x7fbe980cfcf0 2024-04-25T10:07:56.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9cb04640 1 -- 172.21.15.60:0/204591411 >> v1:172.21.15.60:6789/0 conn(0x7fbe980cfcf0 legacy=0x7fbe98161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9cb04640 1 --2- 172.21.15.60:0/204591411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe800039c0 0x7fbe80003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9cb04640 1 --2- 172.21.15.60:0/204591411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe80004810 0x7fbe80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9cb04640 1 --2- 172.21.15.60:0/204591411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe980aec10 0x7fbe800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9cb04640 1 -- 172.21.15.60:0/204591411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9815def0 con 0x7fbe80004810 2024-04-25T10:07:56.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9cb04640 1 -- 172.21.15.60:0/204591411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8000b9d0 con 0x7fbe800039c0 2024-04-25T10:07:56.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9cb04640 1 -- 172.21.15.60:0/204591411 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8000bb50 con 0x7fbe980aec10 2024-04-25T10:07:56.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9d305640 1 --2- 172.21.15.60:0/204591411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe80004810 0x7fbe80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9db06640 1 --2- 172.21.15.60:0/204591411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe800039c0 0x7fbe80003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9d305640 1 -- 172.21.15.60:0/204591411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe980aec10 msgr2=0x7fbe800051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:56.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9e307640 1 --2- 172.21.15.60:0/204591411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe980aec10 0x7fbe800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9d305640 1 --2- 172.21.15.60:0/204591411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe980aec10 0x7fbe800051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9d305640 1 -- 172.21.15.60:0/204591411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe800039c0 msgr2=0x7fbe80003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:56.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9d305640 1 --2- 172.21.15.60:0/204591411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe800039c0 0x7fbe80003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9d305640 1 -- 172.21.15.60:0/204591411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe80003530 con 0x7fbe80004810 2024-04-25T10:07:56.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9db06640 1 --2- 172.21.15.60:0/204591411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe800039c0 0x7fbe80003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:56.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.642+0000 7fbe9e307640 1 --2- 172.21.15.60:0/204591411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe980aec10 0x7fbe800051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:56.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.646+0000 7fbe9d305640 1 --2- 172.21.15.60:0/204591411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe80004810 0x7fbe80004c90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009ae0 tx=0x7fbe8c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35f43b67932ab06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.646+0000 7fbe9cb04640 1 -- 172.21.15.60:0/204591411 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c0096b0 con 0x7fbe80004810 2024-04-25T10:07:56.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.646+0000 7fbe9cb04640 1 -- 172.21.15.60:0/204591411 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8c0a6480 con 0x7fbe80004810 2024-04-25T10:07:56.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.646+0000 7fbe9cb04640 1 -- 172.21.15.60:0/204591411 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c0a6870 con 0x7fbe80004810 2024-04-25T10:07:56.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.646+0000 7fbe9fd6a640 1 -- 172.21.15.60:0/204591411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe80004810 msgr2=0x7fbe80004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.646+0000 7fbe9fd6a640 1 --2- 172.21.15.60:0/204591411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe80004810 0x7fbe80004c90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009ae0 tx=0x7fbe8c01ca70 comp rx=0 tx=0).stop 2024-04-25T10:07:56.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.646+0000 7fbe9e307640 1 -- 172.21.15.60:0/204591411 reap_dead start 2024-04-25T10:07:56.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.646+0000 7fbe9fd6a640 1 -- 172.21.15.60:0/204591411 shutdown_connections 2024-04-25T10:07:56.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.646+0000 7fbe9fd6a640 1 -- 172.21.15.60:0/204591411 >> 172.21.15.60:0/204591411 conn(0x7fbe980670b0 msgr2=0x7fbe9806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:56.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.646+0000 7fbe9fd6a640 1 -- 172.21.15.60:0/204591411 shutdown_connections 2024-04-25T10:07:56.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.646+0000 7fbe9fd6a640 1 -- 172.21.15.60:0/204591411 wait complete. 2024-04-25T10:07:56.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.646+0000 7fbe9fd6a640 1 Processor -- start 2024-04-25T10:07:56.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.646+0000 7fbe9fd6a640 1 -- start start 2024-04-25T10:07:56.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9fd6a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe80004810 0x7fbe98159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9fd6a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe9814e990 0x7fbe98159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9fd6a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe981f7220 0x7fbe981f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9fd6a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe98071ed0 con 0x7fbe9814e990 2024-04-25T10:07:56.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9fd6a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe98071bf0 con 0x7fbe981f7220 2024-04-25T10:07:56.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9fd6a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9806db70 con 0x7fbe80004810 2024-04-25T10:07:56.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9d305640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe9814e990 0x7fbe98159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9d305640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe9814e990 0x7fbe98159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51180/0 (socket says 172.21.15.60:51180) 2024-04-25T10:07:56.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9d305640 1 -- 172.21.15.60:0/2682131607 learned_addr learned my addr 172.21.15.60:0/2682131607 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:56.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9e307640 1 --2- 172.21.15.60:0/2682131607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe981f7220 0x7fbe981f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9db06640 1 --2- 172.21.15.60:0/2682131607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe80004810 0x7fbe98159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9d305640 1 -- 172.21.15.60:0/2682131607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe80004810 msgr2=0x7fbe98159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9d305640 1 --2- 172.21.15.60:0/2682131607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe80004810 0x7fbe98159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9d305640 1 -- 172.21.15.60:0/2682131607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe981f7220 msgr2=0x7fbe981f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9d305640 1 --2- 172.21.15.60:0/2682131607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe981f7220 0x7fbe981f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9d305640 1 -- 172.21.15.60:0/2682131607 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe981f3b00 con 0x7fbe9814e990 2024-04-25T10:07:56.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9e307640 1 --2- 172.21.15.60:0/2682131607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe981f7220 0x7fbe981f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:56.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9d305640 1 --2- 172.21.15.60:0/2682131607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe9814e990 0x7fbe98159dd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c01cf50 tx=0x7fbe80005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe967fc640 1 -- 172.21.15.60:0/2682131607 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c00f000 con 0x7fbe9814e990 2024-04-25T10:07:56.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe967fc640 1 -- 172.21.15.60:0/2682131607 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8c0a6dc0 con 0x7fbe9814e990 2024-04-25T10:07:56.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9fd6a640 1 -- 172.21.15.60:0/2682131607 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe981f3de0 con 0x7fbe9814e990 2024-04-25T10:07:56.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe967fc640 1 -- 172.21.15.60:0/2682131607 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c004240 con 0x7fbe9814e990 2024-04-25T10:07:56.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9fd6a640 1 -- 172.21.15.60:0/2682131607 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe981f43a0 con 0x7fbe9814e990 2024-04-25T10:07:56.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9fd6a640 1 -- 172.21.15.60:0/2682131607 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe98074770 con 0x7fbe9814e990 2024-04-25T10:07:56.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe967fc640 1 -- 172.21.15.60:0/2682131607 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe8c0c2060 con 0x7fbe9814e990 2024-04-25T10:07:56.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe967fc640 1 --2- 172.21.15.60:0/2682131607 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe84066f60 0x7fbe84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe967fc640 1 -- 172.21.15.60:0/2682131607 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe8c11ef50 con 0x7fbe9814e990 2024-04-25T10:07:56.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.650+0000 7fbe9db06640 1 --2- 172.21.15.60:0/2682131607 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe84066f60 0x7fbe84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.654+0000 7fbe9db06640 1 --2- 172.21.15.60:0/2682131607 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe84066f60 0x7fbe84069420 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbe90007440 tx=0x7fbe9000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.654+0000 7fbe967fc640 1 -- 172.21.15.60:0/2682131607 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbe8c0ac020 con 0x7fbe9814e990 2024-04-25T10:07:56.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.818+0000 7fbe9fd6a640 1 -- 172.21.15.60:0/2682131607 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbe98157dd0 con 0x7fbe9814e990 2024-04-25T10:07:56.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.818+0000 7fbe967fc640 1 -- 172.21.15.60:0/2682131607 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbe8c016780 con 0x7fbe9814e990 2024-04-25T10:07:56.825 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:07:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.818+0000 7fbe9fd6a640 1 -- 172.21.15.60:0/2682131607 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe84066f60 msgr2=0x7fbe84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.818+0000 7fbe9fd6a640 1 --2- 172.21.15.60:0/2682131607 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe84066f60 0x7fbe84069420 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbe90007440 tx=0x7fbe9000b040 comp rx=0 tx=0).stop 2024-04-25T10:07:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.818+0000 7fbe9fd6a640 1 -- 172.21.15.60:0/2682131607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe9814e990 msgr2=0x7fbe98159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.818+0000 7fbe9fd6a640 1 --2- 172.21.15.60:0/2682131607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe9814e990 0x7fbe98159dd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c01cf50 tx=0x7fbe80005840 comp rx=0 tx=0).stop 2024-04-25T10:07:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.818+0000 7fbe9fd6a640 1 -- 172.21.15.60:0/2682131607 shutdown_connections 2024-04-25T10:07:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.818+0000 7fbe9fd6a640 1 --2- 172.21.15.60:0/2682131607 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe84066f60 0x7fbe84069420 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.818+0000 7fbe9fd6a640 1 --2- 172.21.15.60:0/2682131607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe981f7220 0x7fbe981f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.818+0000 7fbe9fd6a640 1 --2- 172.21.15.60:0/2682131607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe9814e990 0x7fbe98159dd0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.818+0000 7fbe9fd6a640 1 --2- 172.21.15.60:0/2682131607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe80004810 0x7fbe98159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.818+0000 7fbe9fd6a640 1 -- 172.21.15.60:0/2682131607 >> 172.21.15.60:0/2682131607 conn(0x7fbe980670b0 msgr2=0x7fbe9815fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.818+0000 7fbe9fd6a640 1 -- 172.21.15.60:0/2682131607 shutdown_connections 2024-04-25T10:07:56.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.818+0000 7fbe9fd6a640 1 -- 172.21.15.60:0/2682131607 wait complete. 2024-04-25T10:07:56.843 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:56.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.954+0000 7fda53d65640 1 Processor -- start 2024-04-25T10:07:56.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.954+0000 7fda53d65640 1 -- start start 2024-04-25T10:07:56.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.958+0000 7fda53d65640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda4c0ce7a0 0x7fda4c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.958+0000 7fda53d65640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fda4c15bc90 0x7fda4c0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.958+0000 7fda53d65640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4c0cf0c0 con 0x7fda4c0b4660 2024-04-25T10:07:56.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.958+0000 7fda53d65640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4c0cc370 con 0x7fda4c15bc90 2024-04-25T10:07:56.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.958+0000 7fda53d65640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c071ca0 con 0x7fda4c0ce7a0 2024-04-25T10:07:56.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.958+0000 7fda51b01640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda4c0ce7a0 0x7fda4c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.958+0000 7fda51300640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fda4c15bc90 0x7fda4c0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.958+0000 7fda51b01640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda4c0ce7a0 0x7fda4c0ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:60942/0 (socket says 172.21.15.60:60942) 2024-04-25T10:07:56.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.958+0000 7fda51b01640 1 -- 172.21.15.60:0/2399689910 learned_addr learned my addr 172.21.15.60:0/2399689910 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:56.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.958+0000 7fda51300640 1 -- 172.21.15.60:0/2399689910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda4c0ce7a0 msgr2=0x7fda4c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:56.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.958+0000 7fda51300640 1 --2- 172.21.15.60:0/2399689910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda4c0ce7a0 0x7fda4c0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.958+0000 7fda51300640 1 -- 172.21.15.60:0/2399689910 >> v1:172.21.15.100:6789/0 conn(0x7fda4c0b4660 legacy=0x7fda4c0b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.958+0000 7fda51300640 1 -- 172.21.15.60:0/2399689910 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c0cca40 con 0x7fda4c15bc90 2024-04-25T10:07:56.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.958+0000 7fda51300640 1 --2- 172.21.15.60:0/2399689910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda4c15bc90 0x7fda4c0b25d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fda3c0097f0 tx=0x7fda3c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a2f389edbb635b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda50aff640 1 -- 172.21.15.60:0/2399689910 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0b0070 con 0x7fda4c15bc90 2024-04-25T10:07:56.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda50aff640 1 -- 172.21.15.60:0/2399689910 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c0ab600 con 0x7fda4c15bc90 2024-04-25T10:07:56.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda50aff640 1 -- 172.21.15.60:0/2399689910 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0ab9d0 con 0x7fda4c15bc90 2024-04-25T10:07:56.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 -- 172.21.15.60:0/2399689910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda4c15bc90 msgr2=0x7fda4c0b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 --2- 172.21.15.60:0/2399689910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda4c15bc90 0x7fda4c0b25d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fda3c0097f0 tx=0x7fda3c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 -- 172.21.15.60:0/2399689910 shutdown_connections 2024-04-25T10:07:56.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 --2- 172.21.15.60:0/2399689910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda4c15bc90 0x7fda4c0b25d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 --2- 172.21.15.60:0/2399689910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda4c0ce7a0 0x7fda4c0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 -- 172.21.15.60:0/2399689910 >> 172.21.15.60:0/2399689910 conn(0x7fda4c0b12d0 msgr2=0x7fda4c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:56.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 -- 172.21.15.60:0/2399689910 shutdown_connections 2024-04-25T10:07:56.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 -- 172.21.15.60:0/2399689910 wait complete. 2024-04-25T10:07:56.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 Processor -- start 2024-04-25T10:07:56.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 -- start start 2024-04-25T10:07:56.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda4c0ce7a0 0x7fda4c14aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda4c15bc90 0x7fda4c14e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda4c14e9b0 0x7fda4c15a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0b4280 con 0x7fda4c14e9b0 2024-04-25T10:07:56.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0b3db0 con 0x7fda4c15bc90 2024-04-25T10:07:56.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0b4400 con 0x7fda4c0ce7a0 2024-04-25T10:07:56.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda52302640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda4c14e9b0 0x7fda4c15a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda52302640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda4c14e9b0 0x7fda4c15a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51202/0 (socket says 172.21.15.60:51202) 2024-04-25T10:07:56.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda52302640 1 -- 172.21.15.60:0/696822019 learned_addr learned my addr 172.21.15.60:0/696822019 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:56.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda51b01640 1 --2- 172.21.15.60:0/696822019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda4c0ce7a0 0x7fda4c14aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda51300640 1 --2- 172.21.15.60:0/696822019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda4c15bc90 0x7fda4c14e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda52302640 1 -- 172.21.15.60:0/696822019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda4c0ce7a0 msgr2=0x7fda4c14aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda52302640 1 --2- 172.21.15.60:0/696822019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda4c0ce7a0 0x7fda4c14aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda52302640 1 -- 172.21.15.60:0/696822019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda4c15bc90 msgr2=0x7fda4c14e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda52302640 1 --2- 172.21.15.60:0/696822019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda4c15bc90 0x7fda4c14e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda52302640 1 -- 172.21.15.60:0/696822019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c159740 con 0x7fda4c14e9b0 2024-04-25T10:07:56.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda51b01640 1 --2- 172.21.15.60:0/696822019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda4c0ce7a0 0x7fda4c14aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:56.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda51300640 1 --2- 172.21.15.60:0/696822019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda4c15bc90 0x7fda4c14e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:56.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda52302640 1 --2- 172.21.15.60:0/696822019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda4c14e9b0 0x7fda4c15a7e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fda4400c730 tx=0x7fda4400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda427fc640 1 -- 172.21.15.60:0/696822019 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44016020 con 0x7fda4c14e9b0 2024-04-25T10:07:56.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda427fc640 1 -- 172.21.15.60:0/696822019 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda44006d60 con 0x7fda4c14e9b0 2024-04-25T10:07:56.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 -- 172.21.15.60:0/696822019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4c159a20 con 0x7fda4c14e9b0 2024-04-25T10:07:56.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda427fc640 1 -- 172.21.15.60:0/696822019 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44003bc0 con 0x7fda4c14e9b0 2024-04-25T10:07:56.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.962+0000 7fda53d65640 1 -- 172.21.15.60:0/696822019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4c159fe0 con 0x7fda4c14e9b0 2024-04-25T10:07:56.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.966+0000 7fda53d65640 1 -- 172.21.15.60:0/696822019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4c074770 con 0x7fda4c14e9b0 2024-04-25T10:07:56.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.966+0000 7fda427fc640 1 -- 172.21.15.60:0/696822019 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda440050d0 con 0x7fda4c14e9b0 2024-04-25T10:07:56.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.966+0000 7fda427fc640 1 --2- 172.21.15.60:0/696822019 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fda30066f90 0x7fda30069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.966+0000 7fda427fc640 1 -- 172.21.15.60:0/696822019 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda44084b50 con 0x7fda4c14e9b0 2024-04-25T10:07:56.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.966+0000 7fda51b01640 1 --2- 172.21.15.60:0/696822019 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fda30066f90 0x7fda30069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.966+0000 7fda51b01640 1 --2- 172.21.15.60:0/696822019 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fda30066f90 0x7fda30069450 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fda4c14bc40 tx=0x7fda48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:56.970+0000 7fda427fc640 1 -- 172.21.15.60:0/696822019 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fda44020020 con 0x7fda4c14e9b0 2024-04-25T10:07:57.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.158+0000 7fda53d65640 1 -- 172.21.15.60:0/696822019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda4c158690 con 0x7fda4c14e9b0 2024-04-25T10:07:57.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.158+0000 7fda427fc640 1 -- 172.21.15.60:0/696822019 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fda44055bf0 con 0x7fda4c14e9b0 2024-04-25T10:07:57.167 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:57.167 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1138162580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1138162580},{"type":"v1","addr":"172.21.15.100:6837","nonce":1138162580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2985559168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2985559168},{"type":"v1","addr":"172.21.15.100:6835","nonce":2985559168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/50795370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":50795370},{"type":"v1","addr":"172.21.15.60:6834","nonce":50795370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:07.980095+0000","modified":"2024-04-25T10:07:10.002670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5796":{"gid":5796,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1747757196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1747757196},{"type":"v1","addr":"172.21.15.60:6835","nonce":1747757196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:57.167 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 21 2024-04-25T10:07:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.162+0000 7fda53d65640 1 -- 172.21.15.60:0/696822019 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fda30066f90 msgr2=0x7fda30069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.162+0000 7fda53d65640 1 --2- 172.21.15.60:0/696822019 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fda30066f90 0x7fda30069450 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fda4c14bc40 tx=0x7fda48008040 comp rx=0 tx=0).stop 2024-04-25T10:07:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.162+0000 7fda53d65640 1 -- 172.21.15.60:0/696822019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda4c14e9b0 msgr2=0x7fda4c15a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.162+0000 7fda53d65640 1 --2- 172.21.15.60:0/696822019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda4c14e9b0 0x7fda4c15a7e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fda4400c730 tx=0x7fda4400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.162+0000 7fda53d65640 1 -- 172.21.15.60:0/696822019 shutdown_connections 2024-04-25T10:07:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.162+0000 7fda53d65640 1 --2- 172.21.15.60:0/696822019 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fda30066f90 0x7fda30069450 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.162+0000 7fda53d65640 1 --2- 172.21.15.60:0/696822019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda4c14e9b0 0x7fda4c15a7e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.162+0000 7fda53d65640 1 --2- 172.21.15.60:0/696822019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda4c15bc90 0x7fda4c14e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.162+0000 7fda53d65640 1 --2- 172.21.15.60:0/696822019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda4c0ce7a0 0x7fda4c14aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.162+0000 7fda53d65640 1 -- 172.21.15.60:0/696822019 >> 172.21.15.60:0/696822019 conn(0x7fda4c0b12d0 msgr2=0x7fda4c0cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:57.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.162+0000 7fda53d65640 1 -- 172.21.15.60:0/696822019 shutdown_connections 2024-04-25T10:07:57.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.162+0000 7fda53d65640 1 -- 172.21.15.60:0/696822019 wait complete. 2024-04-25T10:07:57.185 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:57.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.298+0000 7ff334e4c640 1 Processor -- start 2024-04-25T10:07:57.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.298+0000 7ff334e4c640 1 -- start start 2024-04-25T10:07:57.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.302+0000 7ff334e4c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33015deb0 0x7ff3300b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.302+0000 7ff334e4c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff3300d0340 0x7ff3300d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.302+0000 7ff334e4c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33006d200 con 0x7ff3300ce2b0 2024-04-25T10:07:57.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.302+0000 7ff334e4c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3300cf2c0 con 0x7ff3300d0340 2024-04-25T10:07:57.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.302+0000 7ff334e4c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3300cefe0 con 0x7ff33015deb0 2024-04-25T10:07:57.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.302+0000 7ff32e59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33015deb0 0x7ff3300b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.302+0000 7ff32dd9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff3300d0340 0x7ff3300d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.302+0000 7ff32ed9d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff3300ce2b0 0x7ff3300ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:40638/0 (socket says 172.21.15.60:40638) 2024-04-25T10:07:57.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.302+0000 7ff32ed9d640 1 -- 172.21.15.60:0/1493181016 learned_addr learned my addr 172.21.15.60:0/1493181016 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:57.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.302+0000 7ff32dd9b640 1 -- 172.21.15.60:0/1493181016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33015deb0 msgr2=0x7ff3300b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:57.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.302+0000 7ff32dd9b640 1 --2- 172.21.15.60:0/1493181016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33015deb0 0x7ff3300b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.302+0000 7ff32dd9b640 1 -- 172.21.15.60:0/1493181016 >> v1:172.21.15.100:6789/0 conn(0x7ff3300ce2b0 legacy=0x7ff3300ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.302+0000 7ff32dd9b640 1 -- 172.21.15.60:0/1493181016 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3300d1d70 con 0x7ff3300d0340 2024-04-25T10:07:57.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.302+0000 7ff32dd9b640 1 --2- 172.21.15.60:0/1493181016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3300d0340 0x7ff3300d07c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff3240097c0 tx=0x7ff32401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=571f70da45d7cea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:57.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff32d59a640 1 -- 172.21.15.60:0/1493181016 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3240b0070 con 0x7ff3300d0340 2024-04-25T10:07:57.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff32d59a640 1 -- 172.21.15.60:0/1493181016 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3240ab600 con 0x7ff3300d0340 2024-04-25T10:07:57.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff32d59a640 1 -- 172.21.15.60:0/1493181016 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3240ab9d0 con 0x7ff3300d0340 2024-04-25T10:07:57.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 -- 172.21.15.60:0/1493181016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3300d0340 msgr2=0x7ff3300d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 --2- 172.21.15.60:0/1493181016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3300d0340 0x7ff3300d07c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff3240097c0 tx=0x7ff32401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 -- 172.21.15.60:0/1493181016 shutdown_connections 2024-04-25T10:07:57.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 --2- 172.21.15.60:0/1493181016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3300d0340 0x7ff3300d07c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 --2- 172.21.15.60:0/1493181016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33015deb0 0x7ff3300b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 -- 172.21.15.60:0/1493181016 >> 172.21.15.60:0/1493181016 conn(0x7ff33006c090 msgr2=0x7ff33006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:57.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 -- 172.21.15.60:0/1493181016 shutdown_connections 2024-04-25T10:07:57.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 -- 172.21.15.60:0/1493181016 wait complete. 2024-04-25T10:07:57.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 Processor -- start 2024-04-25T10:07:57.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 -- start start 2024-04-25T10:07:57.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3300ce2b0 0x7ff3300d3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3300d0340 0x7ff330157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33015deb0 0x7ff33014d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3300d3430 con 0x7ff3300d0340 2024-04-25T10:07:57.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3300d32b0 con 0x7ff3300ce2b0 2024-04-25T10:07:57.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3300d35b0 con 0x7ff33015deb0 2024-04-25T10:07:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff32e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3300ce2b0 0x7ff3300d3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff32dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3300d0340 0x7ff330157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff32dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3300d0340 0x7ff330157510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51208/0 (socket says 172.21.15.60:51208) 2024-04-25T10:07:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff32dd9b640 1 -- 172.21.15.60:0/4154482658 learned_addr learned my addr 172.21.15.60:0/4154482658 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff32ed9d640 1 --2- 172.21.15.60:0/4154482658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33015deb0 0x7ff33014d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff32dd9b640 1 -- 172.21.15.60:0/4154482658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33015deb0 msgr2=0x7ff33014d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff32dd9b640 1 --2- 172.21.15.60:0/4154482658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33015deb0 0x7ff33014d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff32dd9b640 1 -- 172.21.15.60:0/4154482658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3300ce2b0 msgr2=0x7ff3300d3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff32dd9b640 1 --2- 172.21.15.60:0/4154482658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3300ce2b0 0x7ff3300d3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff32dd9b640 1 -- 172.21.15.60:0/4154482658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33014d640 con 0x7ff3300d0340 2024-04-25T10:07:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff32dd9b640 1 --2- 172.21.15.60:0/4154482658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3300d0340 0x7ff330157510 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff3240ae040 tx=0x7ff324019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff31effd640 1 -- 172.21.15.60:0/4154482658 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3240ba070 con 0x7ff3300d0340 2024-04-25T10:07:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff31effd640 1 -- 172.21.15.60:0/4154482658 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3240b43f0 con 0x7ff3300d0340 2024-04-25T10:07:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 -- 172.21.15.60:0/4154482658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33014f670 con 0x7ff3300d0340 2024-04-25T10:07:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff31effd640 1 -- 172.21.15.60:0/4154482658 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3240b0040 con 0x7ff3300d0340 2024-04-25T10:07:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.306+0000 7ff334e4c640 1 -- 172.21.15.60:0/4154482658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33014fc30 con 0x7ff3300d0340 2024-04-25T10:07:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.310+0000 7ff334e4c640 1 -- 172.21.15.60:0/4154482658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff330074770 con 0x7ff3300d0340 2024-04-25T10:07:57.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.310+0000 7ff31effd640 1 -- 172.21.15.60:0/4154482658 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3240bd070 con 0x7ff3300d0340 2024-04-25T10:07:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.310+0000 7ff31effd640 1 --2- 172.21.15.60:0/4154482658 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff310066f90 0x7ff310069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.310+0000 7ff31effd640 1 -- 172.21.15.60:0/4154482658 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff32411df50 con 0x7ff3300d0340 2024-04-25T10:07:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.310+0000 7ff32e59c640 1 --2- 172.21.15.60:0/4154482658 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff310066f90 0x7ff310069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.310+0000 7ff32e59c640 1 --2- 172.21.15.60:0/4154482658 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff310066f90 0x7ff310069450 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff320009ac0 tx=0x7ff320009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:57.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.314+0000 7ff31effd640 1 -- 172.21.15.60:0/4154482658 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff324016090 con 0x7ff3300d0340 2024-04-25T10:07:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.466+0000 7ff334e4c640 1 -- 172.21.15.60:0/4154482658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff33015d110 con 0x7ff3300d0340 2024-04-25T10:07:57.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.470+0000 7ff31effd640 1 -- 172.21.15.60:0/4154482658 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16625 (secure 0 0 0) 0x7ff3240ee910 con 0x7ff3300d0340 2024-04-25T10:07:57.479 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:57.479 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":47,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:07:09.646687+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.652298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:06.983225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:57.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.474+0000 7ff334e4c640 1 -- 172.21.15.60:0/4154482658 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff310066f90 msgr2=0x7ff310069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.474+0000 7ff334e4c640 1 --2- 172.21.15.60:0/4154482658 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff310066f90 0x7ff310069450 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff320009ac0 tx=0x7ff320009190 comp rx=0 tx=0).stop 2024-04-25T10:07:57.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.474+0000 7ff334e4c640 1 -- 172.21.15.60:0/4154482658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3300d0340 msgr2=0x7ff330157510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.474+0000 7ff334e4c640 1 --2- 172.21.15.60:0/4154482658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3300d0340 0x7ff330157510 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff3240ae040 tx=0x7ff324019240 comp rx=0 tx=0).stop 2024-04-25T10:07:57.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.474+0000 7ff334e4c640 1 -- 172.21.15.60:0/4154482658 shutdown_connections 2024-04-25T10:07:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.474+0000 7ff334e4c640 1 --2- 172.21.15.60:0/4154482658 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff310066f90 0x7ff310069450 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.474+0000 7ff334e4c640 1 --2- 172.21.15.60:0/4154482658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33015deb0 0x7ff33014d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.474+0000 7ff334e4c640 1 --2- 172.21.15.60:0/4154482658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3300d0340 0x7ff330157510 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.474+0000 7ff334e4c640 1 --2- 172.21.15.60:0/4154482658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3300ce2b0 0x7ff3300d3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.474+0000 7ff334e4c640 1 -- 172.21.15.60:0/4154482658 >> 172.21.15.60:0/4154482658 conn(0x7ff33006c090 msgr2=0x7ff3300c6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.474+0000 7ff334e4c640 1 -- 172.21.15.60:0/4154482658 shutdown_connections 2024-04-25T10:07:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.474+0000 7ff334e4c640 1 -- 172.21.15.60:0/4154482658 wait complete. 2024-04-25T10:07:57.496 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:07:57.496 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:57.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb23b604640 1 Processor -- start 2024-04-25T10:07:57.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb23b604640 1 -- start start 2024-04-25T10:07:57.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb23b604640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb234071b20 0x7fb234071f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb23b604640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb2340cce50 0x7fb2340cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb23b604640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb234072440 con 0x7fb23406e480 2024-04-25T10:07:57.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb23b604640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23406eff0 con 0x7fb2340cce50 2024-04-25T10:07:57.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb23b604640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23415dc10 con 0x7fb234071b20 2024-04-25T10:07:57.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb238b9f640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb23406e480 0x7fb23406e8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42960/0 (socket says 172.21.15.60:42960) 2024-04-25T10:07:57.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb238b9f640 1 -- 172.21.15.60:0/393675851 learned_addr learned my addr 172.21.15.60:0/393675851 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:57.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb2393a0640 1 --2- 172.21.15.60:0/393675851 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb234071b20 0x7fb234071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb239ba1640 1 --2- 172.21.15.60:0/393675851 >> v2:172.21.15.100:3300/0 conn(0x7fb2340cce50 0x7fb2340cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb22bfff640 1 -- 172.21.15.60:0/393675851 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771973372 0 0) 0x7fb234072440 con 0x7fb23406e480 2024-04-25T10:07:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb22bfff640 1 -- 172.21.15.60:0/393675851 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb220003530 con 0x7fb23406e480 2024-04-25T10:07:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb22bfff640 1 -- 172.21.15.60:0/393675851 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2183318656 0 0) 0x7fb220003530 con 0x7fb23406e480 2024-04-25T10:07:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb22bfff640 1 -- 172.21.15.60:0/393675851 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb234071b20 msgr2=0x7fb234071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb22bfff640 1 --2- 172.21.15.60:0/393675851 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb234071b20 0x7fb234071f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb22bfff640 1 -- 172.21.15.60:0/393675851 >> v2:172.21.15.100:3300/0 conn(0x7fb2340cce50 msgr2=0x7fb2340cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb22bfff640 1 --2- 172.21.15.60:0/393675851 >> v2:172.21.15.100:3300/0 conn(0x7fb2340cce50 0x7fb2340cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb22bfff640 1 -- 172.21.15.60:0/393675851 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23415def0 con 0x7fb23406e480 2024-04-25T10:07:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb22bfff640 1 -- 172.21.15.60:0/393675851 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fb224001680 con 0x7fb23406e480 2024-04-25T10:07:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb22bfff640 1 -- 172.21.15.60:0/393675851 >> v1:172.21.15.60:6789/0 conn(0x7fb23406e480 legacy=0x7fb23406e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb22bfff640 1 --2- 172.21.15.60:0/393675851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2200039f0 0x7fb220003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb22bfff640 1 --2- 172.21.15.60:0/393675851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb220004810 0x7fb220004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb22bfff640 1 --2- 172.21.15.60:0/393675851 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb234071b20 0x7fb2200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb22bfff640 1 -- 172.21.15.60:0/393675851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23415dc10 con 0x7fb220004810 2024-04-25T10:07:57.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb22bfff640 1 -- 172.21.15.60:0/393675851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22000b9d0 con 0x7fb2200039f0 2024-04-25T10:07:57.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb22bfff640 1 -- 172.21.15.60:0/393675851 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22000dd90 con 0x7fb234071b20 2024-04-25T10:07:57.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb238b9f640 1 --2- 172.21.15.60:0/393675851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb220004810 0x7fb220004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb2393a0640 1 --2- 172.21.15.60:0/393675851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2200039f0 0x7fb220003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb239ba1640 1 --2- 172.21.15.60:0/393675851 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb234071b20 0x7fb2200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb238b9f640 1 -- 172.21.15.60:0/393675851 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb234071b20 msgr2=0x7fb2200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:57.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb238b9f640 1 --2- 172.21.15.60:0/393675851 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb234071b20 0x7fb2200051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb238b9f640 1 -- 172.21.15.60:0/393675851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2200039f0 msgr2=0x7fb220003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:57.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb238b9f640 1 --2- 172.21.15.60:0/393675851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2200039f0 0x7fb220003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.602+0000 7fb238b9f640 1 -- 172.21.15.60:0/393675851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb220003530 con 0x7fb220004810 2024-04-25T10:07:57.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb238b9f640 1 --2- 172.21.15.60:0/393675851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb220004810 0x7fb220004c90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb22400c210 tx=0x7fb22401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63514babcf3a5031 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:57.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb22bfff640 1 -- 172.21.15.60:0/393675851 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb224007d40 con 0x7fb220004810 2024-04-25T10:07:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb22bfff640 1 -- 172.21.15.60:0/393675851 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2240a9070 con 0x7fb220004810 2024-04-25T10:07:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb22bfff640 1 -- 172.21.15.60:0/393675851 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb224005d40 con 0x7fb220004810 2024-04-25T10:07:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb23b604640 1 -- 172.21.15.60:0/393675851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb220004810 msgr2=0x7fb220004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb23b604640 1 --2- 172.21.15.60:0/393675851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb220004810 0x7fb220004c90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb22400c210 tx=0x7fb22401cc40 comp rx=0 tx=0).stop 2024-04-25T10:07:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb239ba1640 1 -- 172.21.15.60:0/393675851 reap_dead start 2024-04-25T10:07:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb23b604640 1 -- 172.21.15.60:0/393675851 shutdown_connections 2024-04-25T10:07:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb23b604640 1 -- 172.21.15.60:0/393675851 >> 172.21.15.60:0/393675851 conn(0x7fb2340b12d0 msgr2=0x7fb2340b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb23b604640 1 -- 172.21.15.60:0/393675851 shutdown_connections 2024-04-25T10:07:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb23b604640 1 -- 172.21.15.60:0/393675851 wait complete. 2024-04-25T10:07:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb23b604640 1 Processor -- start 2024-04-25T10:07:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb23b604640 1 -- start start 2024-04-25T10:07:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb23b604640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb220004810 0x7fb234156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb23b604640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2341f2810 0x7fb234153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb23b604640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb234141a70 0x7fb234141f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb23b604640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2340cf070 con 0x7fb220004810 2024-04-25T10:07:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb23b604640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2340ceef0 con 0x7fb2341f2810 2024-04-25T10:07:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb23b604640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2340cd7e0 con 0x7fb234141a70 2024-04-25T10:07:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb2393a0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb220004810 0x7fb234156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb2393a0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb220004810 0x7fb234156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51230/0 (socket says 172.21.15.60:51230) 2024-04-25T10:07:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb2393a0640 1 -- 172.21.15.60:0/1030048216 learned_addr learned my addr 172.21.15.60:0/1030048216 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb238b9f640 1 --2- 172.21.15.60:0/1030048216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2341f2810 0x7fb234153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb239ba1640 1 --2- 172.21.15.60:0/1030048216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb234141a70 0x7fb234141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb2393a0640 1 -- 172.21.15.60:0/1030048216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb234141a70 msgr2=0x7fb234141f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb2393a0640 1 --2- 172.21.15.60:0/1030048216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb234141a70 0x7fb234141f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb2393a0640 1 -- 172.21.15.60:0/1030048216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2341f2810 msgr2=0x7fb234153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb2393a0640 1 --2- 172.21.15.60:0/1030048216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2341f2810 0x7fb234153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.606+0000 7fb2393a0640 1 -- 172.21.15.60:0/1030048216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb234156f00 con 0x7fb220004810 2024-04-25T10:07:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.610+0000 7fb2393a0640 1 --2- 172.21.15.60:0/1030048216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb220004810 0x7fb234156840 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb230011560 tx=0x7fb230011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.610+0000 7fb229ffb640 1 -- 172.21.15.60:0/1030048216 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb230015930 con 0x7fb220004810 2024-04-25T10:07:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.610+0000 7fb229ffb640 1 -- 172.21.15.60:0/1030048216 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb230015ad0 con 0x7fb220004810 2024-04-25T10:07:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.610+0000 7fb229ffb640 1 -- 172.21.15.60:0/1030048216 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23001d400 con 0x7fb220004810 2024-04-25T10:07:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.610+0000 7fb23b604640 1 -- 172.21.15.60:0/1030048216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2341571e0 con 0x7fb220004810 2024-04-25T10:07:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.610+0000 7fb23b604640 1 -- 172.21.15.60:0/1030048216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2341577a0 con 0x7fb220004810 2024-04-25T10:07:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.610+0000 7fb23b604640 1 -- 172.21.15.60:0/1030048216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb234074770 con 0x7fb220004810 2024-04-25T10:07:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.610+0000 7fb229ffb640 1 -- 172.21.15.60:0/1030048216 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb23001d5a0 con 0x7fb220004810 2024-04-25T10:07:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.610+0000 7fb229ffb640 1 --2- 172.21.15.60:0/1030048216 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb218066f90 0x7fb218069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.610+0000 7fb229ffb640 1 -- 172.21.15.60:0/1030048216 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2300857b0 con 0x7fb220004810 2024-04-25T10:07:57.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.610+0000 7fb238b9f640 1 --2- 172.21.15.60:0/1030048216 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb218066f90 0x7fb218069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.610+0000 7fb238b9f640 1 --2- 172.21.15.60:0/1030048216 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb218066f90 0x7fb218069450 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb2240ab000 tx=0x7fb2240b1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:57.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.614+0000 7fb229ffb640 1 -- 172.21.15.60:0/1030048216 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb230019030 con 0x7fb220004810 2024-04-25T10:07:57.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.802+0000 7fb23b604640 1 -- 172.21.15.60:0/1030048216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb234143030 con 0x7fb220004810 2024-04-25T10:07:57.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.802+0000 7fb229ffb640 1 -- 172.21.15.60:0/1030048216 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fb230056170 con 0x7fb220004810 2024-04-25T10:07:57.810 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:57.810 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1138162580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1138162580},{"type":"v1","addr":"172.21.15.100:6837","nonce":1138162580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2985559168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2985559168},{"type":"v1","addr":"172.21.15.100:6835","nonce":2985559168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/50795370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":50795370},{"type":"v1","addr":"172.21.15.60:6834","nonce":50795370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:07.980095+0000","modified":"2024-04-25T10:07:10.002670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5796":{"gid":5796,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1747757196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1747757196},{"type":"v1","addr":"172.21.15.60:6835","nonce":1747757196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:57.810 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 21 2024-04-25T10:07:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.802+0000 7fb23b604640 1 -- 172.21.15.60:0/1030048216 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb218066f90 msgr2=0x7fb218069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.802+0000 7fb23b604640 1 --2- 172.21.15.60:0/1030048216 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb218066f90 0x7fb218069450 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb2240ab000 tx=0x7fb2240b1040 comp rx=0 tx=0).stop 2024-04-25T10:07:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.802+0000 7fb23b604640 1 -- 172.21.15.60:0/1030048216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb220004810 msgr2=0x7fb234156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.802+0000 7fb23b604640 1 --2- 172.21.15.60:0/1030048216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb220004810 0x7fb234156840 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb230011560 tx=0x7fb230011a20 comp rx=0 tx=0).stop 2024-04-25T10:07:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.802+0000 7fb23b604640 1 -- 172.21.15.60:0/1030048216 shutdown_connections 2024-04-25T10:07:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.802+0000 7fb23b604640 1 --2- 172.21.15.60:0/1030048216 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb218066f90 0x7fb218069450 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.802+0000 7fb23b604640 1 --2- 172.21.15.60:0/1030048216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb234141a70 0x7fb234141f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.802+0000 7fb23b604640 1 --2- 172.21.15.60:0/1030048216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2341f2810 0x7fb234153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.802+0000 7fb23b604640 1 --2- 172.21.15.60:0/1030048216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb220004810 0x7fb234156840 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.802+0000 7fb23b604640 1 -- 172.21.15.60:0/1030048216 >> 172.21.15.60:0/1030048216 conn(0x7fb2340b12d0 msgr2=0x7fb2340cb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.802+0000 7fb23b604640 1 -- 172.21.15.60:0/1030048216 shutdown_connections 2024-04-25T10:07:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.802+0000 7fb23b604640 1 -- 172.21.15.60:0/1030048216 wait complete. 2024-04-25T10:07:57.827 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:57.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.930+0000 7fc768d33640 1 Processor -- start 2024-04-25T10:07:57.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.930+0000 7fc768d33640 1 -- start start 2024-04-25T10:07:57.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc768d33640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc76406ec30 0x7fc7640cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc768d33640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc7640b37e0 0x7fc7640b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc768d33640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76406f130 con 0x7fc76415a7a0 2024-04-25T10:07:57.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc768d33640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7640b4190 con 0x7fc76406ec30 2024-04-25T10:07:57.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc768d33640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7640cdaa0 con 0x7fc7640b37e0 2024-04-25T10:07:57.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc75bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc76406ec30 0x7fc7640cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc75bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc76406ec30 0x7fc7640cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51242/0 (socket says 172.21.15.60:51242) 2024-04-25T10:07:57.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc75bfff640 1 -- 172.21.15.60:0/3249944341 learned_addr learned my addr 172.21.15.60:0/3249944341 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:57.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc762d9d640 1 --2- 172.21.15.60:0/3249944341 >> v2:172.21.15.100:3300/0 conn(0x7fc7640b37e0 0x7fc7640b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc75bfff640 1 -- 172.21.15.60:0/3249944341 >> v2:172.21.15.100:3300/0 conn(0x7fc7640b37e0 msgr2=0x7fc7640b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:57.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc75bfff640 1 --2- 172.21.15.60:0/3249944341 >> v2:172.21.15.100:3300/0 conn(0x7fc7640b37e0 0x7fc7640b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc75bfff640 1 -- 172.21.15.60:0/3249944341 >> v1:172.21.15.100:6789/0 conn(0x7fc76415a7a0 legacy=0x7fc764071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:57.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc75bfff640 1 -- 172.21.15.60:0/3249944341 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76406f410 con 0x7fc76406ec30 2024-04-25T10:07:57.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc762d9d640 1 --2- 172.21.15.60:0/3249944341 >> v2:172.21.15.100:3300/0 conn(0x7fc7640b37e0 0x7fc7640b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:57.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc75bfff640 1 --2- 172.21.15.60:0/3249944341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76406ec30 0x7fc7640cd560 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009870 tx=0x7fc74c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b24c0e8275bf99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:57.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc761d9b640 1 -- 172.21.15.60:0/3249944341 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0ab430 con 0x7fc76406ec30 2024-04-25T10:07:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc761d9b640 1 -- 172.21.15.60:0/3249944341 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c0b0070 con 0x7fc76406ec30 2024-04-25T10:07:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.934+0000 7fc761d9b640 1 -- 172.21.15.60:0/3249944341 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0ab7f0 con 0x7fc76406ec30 2024-04-25T10:07:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 -- 172.21.15.60:0/3249944341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76406ec30 msgr2=0x7fc7640cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 --2- 172.21.15.60:0/3249944341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76406ec30 0x7fc7640cd560 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009870 tx=0x7fc74c01c520 comp rx=0 tx=0).stop 2024-04-25T10:07:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 -- 172.21.15.60:0/3249944341 shutdown_connections 2024-04-25T10:07:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 --2- 172.21.15.60:0/3249944341 >> v2:172.21.15.100:3300/0 conn(0x7fc7640b37e0 0x7fc7640b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 --2- 172.21.15.60:0/3249944341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76406ec30 0x7fc7640cd560 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 -- 172.21.15.60:0/3249944341 >> 172.21.15.60:0/3249944341 conn(0x7fc7640b12d0 msgr2=0x7fc7640b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 -- 172.21.15.60:0/3249944341 shutdown_connections 2024-04-25T10:07:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 -- 172.21.15.60:0/3249944341 wait complete. 2024-04-25T10:07:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 Processor -- start 2024-04-25T10:07:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 -- start start 2024-04-25T10:07:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc76406ec30 0x7fc764150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7640b37e0 0x7fc76413be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76415a7a0 0x7fc76413c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76406e8c0 con 0x7fc76415a7a0 2024-04-25T10:07:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76406e740 con 0x7fc7640b37e0 2024-04-25T10:07:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76406d8b0 con 0x7fc76406ec30 2024-04-25T10:07:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc762d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76415a7a0 0x7fc76413c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc762d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76415a7a0 0x7fc76413c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51258/0 (socket says 172.21.15.60:51258) 2024-04-25T10:07:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc762d9d640 1 -- 172.21.15.60:0/2928304421 learned_addr learned my addr 172.21.15.60:0/2928304421 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc75bfff640 1 --2- 172.21.15.60:0/2928304421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7640b37e0 0x7fc76413be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc76259c640 1 --2- 172.21.15.60:0/2928304421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc76406ec30 0x7fc764150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc762d9d640 1 -- 172.21.15.60:0/2928304421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc76406ec30 msgr2=0x7fc764150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc762d9d640 1 --2- 172.21.15.60:0/2928304421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc76406ec30 0x7fc764150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc762d9d640 1 -- 172.21.15.60:0/2928304421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7640b37e0 msgr2=0x7fc76413be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc762d9d640 1 --2- 172.21.15.60:0/2928304421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7640b37e0 0x7fc76413be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc762d9d640 1 -- 172.21.15.60:0/2928304421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76413c900 con 0x7fc76415a7a0 2024-04-25T10:07:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc75bfff640 1 --2- 172.21.15.60:0/2928304421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7640b37e0 0x7fc76413be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc76259c640 1 --2- 172.21.15.60:0/2928304421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc76406ec30 0x7fc764150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc762d9d640 1 --2- 172.21.15.60:0/2928304421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76415a7a0 0x7fc76413c3c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009fd0 tx=0x7fc75c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc75affd640 1 -- 172.21.15.60:0/2928304421 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c01a070 con 0x7fc76415a7a0 2024-04-25T10:07:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc75affd640 1 -- 172.21.15.60:0/2928304421 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75c003bc0 con 0x7fc76415a7a0 2024-04-25T10:07:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 -- 172.21.15.60:0/2928304421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc764139cf0 con 0x7fc76415a7a0 2024-04-25T10:07:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc75affd640 1 -- 172.21.15.60:0/2928304421 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c004030 con 0x7fc76415a7a0 2024-04-25T10:07:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 -- 172.21.15.60:0/2928304421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76413a2b0 con 0x7fc76415a7a0 2024-04-25T10:07:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.938+0000 7fc768d33640 1 -- 172.21.15.60:0/2928304421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc764074770 con 0x7fc76415a7a0 2024-04-25T10:07:57.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.942+0000 7fc75affd640 1 -- 172.21.15.60:0/2928304421 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc75c01d070 con 0x7fc76415a7a0 2024-04-25T10:07:57.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.942+0000 7fc75affd640 1 --2- 172.21.15.60:0/2928304421 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc744066f60 0x7fc744069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.942+0000 7fc75affd640 1 -- 172.21.15.60:0/2928304421 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc75c085380 con 0x7fc76415a7a0 2024-04-25T10:07:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.942+0000 7fc76259c640 1 --2- 172.21.15.60:0/2928304421 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc744066f60 0x7fc744069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.942+0000 7fc76259c640 1 --2- 172.21.15.60:0/2928304421 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc744066f60 0x7fc744069420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc7641f7ad0 tx=0x7fc754009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:57.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:57.946+0000 7fc75affd640 1 -- 172.21.15.60:0/2928304421 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc75c010070 con 0x7fc76415a7a0 2024-04-25T10:07:58.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.138+0000 7fc768d33640 1 -- 172.21.15.60:0/2928304421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc764154f10 con 0x7fc76415a7a0 2024-04-25T10:07:58.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.138+0000 7fc75affd640 1 -- 172.21.15.60:0/2928304421 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fc75c056380 con 0x7fc76415a7a0 2024-04-25T10:07:58.146 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:58.146 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1138162580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1138162580},{"type":"v1","addr":"172.21.15.100:6837","nonce":1138162580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2985559168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2985559168},{"type":"v1","addr":"172.21.15.100:6835","nonce":2985559168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/50795370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":50795370},{"type":"v1","addr":"172.21.15.60:6834","nonce":50795370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:07.980095+0000","modified":"2024-04-25T10:07:10.002670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5796":{"gid":5796,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1747757196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1747757196},{"type":"v1","addr":"172.21.15.60:6835","nonce":1747757196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:58.147 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 21 2024-04-25T10:07:58.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.138+0000 7fc768d33640 1 -- 172.21.15.60:0/2928304421 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc744066f60 msgr2=0x7fc744069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.138+0000 7fc768d33640 1 --2- 172.21.15.60:0/2928304421 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc744066f60 0x7fc744069420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc7641f7ad0 tx=0x7fc754009040 comp rx=0 tx=0).stop 2024-04-25T10:07:58.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.138+0000 7fc768d33640 1 -- 172.21.15.60:0/2928304421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76415a7a0 msgr2=0x7fc76413c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.138+0000 7fc768d33640 1 --2- 172.21.15.60:0/2928304421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76415a7a0 0x7fc76413c3c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009fd0 tx=0x7fc75c00d560 comp rx=0 tx=0).stop 2024-04-25T10:07:58.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.142+0000 7fc768d33640 1 -- 172.21.15.60:0/2928304421 shutdown_connections 2024-04-25T10:07:58.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.142+0000 7fc768d33640 1 --2- 172.21.15.60:0/2928304421 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc744066f60 0x7fc744069420 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.142+0000 7fc768d33640 1 --2- 172.21.15.60:0/2928304421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc76415a7a0 0x7fc76413c3c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.142+0000 7fc768d33640 1 --2- 172.21.15.60:0/2928304421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7640b37e0 0x7fc76413be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.142+0000 7fc768d33640 1 --2- 172.21.15.60:0/2928304421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc76406ec30 0x7fc764150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.142+0000 7fc768d33640 1 -- 172.21.15.60:0/2928304421 >> 172.21.15.60:0/2928304421 conn(0x7fc7640b12d0 msgr2=0x7fc7640cdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.142+0000 7fc768d33640 1 -- 172.21.15.60:0/2928304421 shutdown_connections 2024-04-25T10:07:58.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.142+0000 7fc768d33640 1 -- 172.21.15.60:0/2928304421 wait complete. 2024-04-25T10:07:58.165 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:58.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0173ed8640 1 Processor -- start 2024-04-25T10:07:58.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0173ed8640 1 -- start start 2024-04-25T10:07:58.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0173ed8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016c0b6c10 0x7f016c0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0173ed8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016c06d200 con 0x7f016c15deb0 2024-04-25T10:07:58.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0173ed8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016c0cefe0 con 0x7f016c0d0a50 2024-04-25T10:07:58.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0173ed8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c0cf2c0 con 0x7f016c0b6c10 2024-04-25T10:07:58.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0171c74640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f016c15deb0 0x7f016c0d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42964/0 (socket says 172.21.15.60:42964) 2024-04-25T10:07:58.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0171c74640 1 -- 172.21.15.60:0/2159540894 learned_addr learned my addr 172.21.15.60:0/2159540894 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:58.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0172475640 1 --2- 172.21.15.60:0/2159540894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016c0b6c10 0x7f016c0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0170c72640 1 -- 172.21.15.60:0/2159540894 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759605131 0 0) 0x7f016c06d200 con 0x7f016c15deb0 2024-04-25T10:07:58.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0170c72640 1 -- 172.21.15.60:0/2159540894 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0158003530 con 0x7f016c15deb0 2024-04-25T10:07:58.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0170c72640 1 -- 172.21.15.60:0/2159540894 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3707059842 0 0) 0x7f0158003530 con 0x7f016c15deb0 2024-04-25T10:07:58.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0170c72640 1 -- 172.21.15.60:0/2159540894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016c0b6c10 msgr2=0x7f016c0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0170c72640 1 --2- 172.21.15.60:0/2159540894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016c0b6c10 0x7f016c0ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0170c72640 1 -- 172.21.15.60:0/2159540894 >> v1:172.21.15.100:6789/0 conn(0x7f016c0d0a50 legacy=0x7f016c0ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0170c72640 1 -- 172.21.15.60:0/2159540894 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c0d1d70 con 0x7f016c15deb0 2024-04-25T10:07:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0170c72640 1 -- 172.21.15.60:0/2159540894 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f0168001680 con 0x7f016c15deb0 2024-04-25T10:07:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0170c72640 1 -- 172.21.15.60:0/2159540894 >> v1:172.21.15.60:6789/0 conn(0x7f016c15deb0 legacy=0x7f016c0d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0170c72640 1 --2- 172.21.15.60:0/2159540894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580039f0 0x7f0158003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0170c72640 1 --2- 172.21.15.60:0/2159540894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0158004810 0x7f0158004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0170c72640 1 --2- 172.21.15.60:0/2159540894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016c0b6c10 0x7f01580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0170c72640 1 -- 172.21.15.60:0/2159540894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c0cf2c0 con 0x7f01580039f0 2024-04-25T10:07:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0170c72640 1 -- 172.21.15.60:0/2159540894 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015800bab0 con 0x7f0158004810 2024-04-25T10:07:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0170c72640 1 -- 172.21.15.60:0/2159540894 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015800bcb0 con 0x7f016c0b6c10 2024-04-25T10:07:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0171c74640 1 --2- 172.21.15.60:0/2159540894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580039f0 0x7f0158003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0171473640 1 --2- 172.21.15.60:0/2159540894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0158004810 0x7f0158004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0171c74640 1 -- 172.21.15.60:0/2159540894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016c0b6c10 msgr2=0x7f01580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0172475640 1 --2- 172.21.15.60:0/2159540894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016c0b6c10 0x7f01580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0171c74640 1 --2- 172.21.15.60:0/2159540894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016c0b6c10 0x7f01580051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0171c74640 1 -- 172.21.15.60:0/2159540894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0158004810 msgr2=0x7f0158004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0171c74640 1 --2- 172.21.15.60:0/2159540894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0158004810 0x7f0158004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0171c74640 1 -- 172.21.15.60:0/2159540894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0158003530 con 0x7f01580039f0 2024-04-25T10:07:58.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.270+0000 7f0171473640 1 --2- 172.21.15.60:0/2159540894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0158004810 0x7f0158004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:58.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0171c74640 1 --2- 172.21.15.60:0/2159540894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580039f0 0x7f0158003e10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f016800c160 tx=0x7f016801cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31f0316658ed10d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0170c72640 1 -- 172.21.15.60:0/2159540894 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0168007c70 con 0x7f01580039f0 2024-04-25T10:07:58.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0170c72640 1 -- 172.21.15.60:0/2159540894 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01680a9070 con 0x7f01580039f0 2024-04-25T10:07:58.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0170c72640 1 -- 172.21.15.60:0/2159540894 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0168005d60 con 0x7f01580039f0 2024-04-25T10:07:58.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0173ed8640 1 -- 172.21.15.60:0/2159540894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580039f0 msgr2=0x7f0158003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0173ed8640 1 --2- 172.21.15.60:0/2159540894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580039f0 0x7f0158003e10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f016800c160 tx=0x7f016801cb80 comp rx=0 tx=0).stop 2024-04-25T10:07:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0172475640 1 -- 172.21.15.60:0/2159540894 reap_dead start 2024-04-25T10:07:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0173ed8640 1 -- 172.21.15.60:0/2159540894 shutdown_connections 2024-04-25T10:07:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0173ed8640 1 -- 172.21.15.60:0/2159540894 >> 172.21.15.60:0/2159540894 conn(0x7f016c06c090 msgr2=0x7f016c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0173ed8640 1 -- 172.21.15.60:0/2159540894 shutdown_connections 2024-04-25T10:07:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0173ed8640 1 -- 172.21.15.60:0/2159540894 wait complete. 2024-04-25T10:07:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0173ed8640 1 Processor -- start 2024-04-25T10:07:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0173ed8640 1 -- start start 2024-04-25T10:07:58.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0173ed8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580039f0 0x7f016c150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0173ed8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016c14bf40 0x7f016c0cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0173ed8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f016c15d380 0x7f016c0b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0173ed8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c1697c0 con 0x7f01580039f0 2024-04-25T10:07:58.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0173ed8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c169640 con 0x7f016c15d380 2024-04-25T10:07:58.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0173ed8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c169940 con 0x7f016c14bf40 2024-04-25T10:07:58.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0171c74640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580039f0 0x7f016c150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0171c74640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580039f0 0x7f016c150c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51278/0 (socket says 172.21.15.60:51278) 2024-04-25T10:07:58.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.274+0000 7f0171c74640 1 -- 172.21.15.60:0/2101060950 learned_addr learned my addr 172.21.15.60:0/2101060950 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:58.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f0172475640 1 --2- 172.21.15.60:0/2101060950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f016c15d380 0x7f016c0b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f0171473640 1 --2- 172.21.15.60:0/2101060950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016c14bf40 0x7f016c0cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f0171c74640 1 -- 172.21.15.60:0/2101060950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016c14bf40 msgr2=0x7f016c0cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f0171c74640 1 --2- 172.21.15.60:0/2101060950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016c14bf40 0x7f016c0cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f0171c74640 1 -- 172.21.15.60:0/2101060950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f016c15d380 msgr2=0x7f016c0b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f0171c74640 1 --2- 172.21.15.60:0/2101060950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f016c15d380 0x7f016c0b6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f0171c74640 1 -- 172.21.15.60:0/2101060950 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c151180 con 0x7f01580039f0 2024-04-25T10:07:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f0172475640 1 --2- 172.21.15.60:0/2101060950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f016c15d380 0x7f016c0b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f0171c74640 1 --2- 172.21.15.60:0/2101060950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580039f0 0x7f016c150c40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f016800c160 tx=0x7f0158005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f01627fc640 1 -- 172.21.15.60:0/2101060950 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0168005d40 con 0x7f01580039f0 2024-04-25T10:07:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f01627fc640 1 -- 172.21.15.60:0/2101060950 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01680013c0 con 0x7f01580039f0 2024-04-25T10:07:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f0173ed8640 1 -- 172.21.15.60:0/2101060950 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016c0cd080 con 0x7f01580039f0 2024-04-25T10:07:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f0173ed8640 1 -- 172.21.15.60:0/2101060950 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016c0cae80 con 0x7f01580039f0 2024-04-25T10:07:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f01627fc640 1 -- 172.21.15.60:0/2101060950 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01680070a0 con 0x7f01580039f0 2024-04-25T10:07:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f0173ed8640 1 -- 172.21.15.60:0/2101060950 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f016c074770 con 0x7f01580039f0 2024-04-25T10:07:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f01627fc640 1 -- 172.21.15.60:0/2101060950 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0168007240 con 0x7f01580039f0 2024-04-25T10:07:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f01627fc640 1 --2- 172.21.15.60:0/2101060950 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0150066f90 0x7f0150069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f01627fc640 1 -- 172.21.15.60:0/2101060950 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f016811ea40 con 0x7f01580039f0 2024-04-25T10:07:58.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.278+0000 7f0171473640 1 --2- 172.21.15.60:0/2101060950 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0150066f90 0x7f0150069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.282+0000 7f0171473640 1 --2- 172.21.15.60:0/2101060950 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0150066f90 0x7f0150069450 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f016c14c320 tx=0x7f015c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.282+0000 7f01627fc640 1 -- 172.21.15.60:0/2101060950 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0168016890 con 0x7f01580039f0 2024-04-25T10:07:58.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.438+0000 7f0173ed8640 1 -- 172.21.15.60:0/2101060950 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f016c14c350 con 0x7f01580039f0 2024-04-25T10:07:58.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.438+0000 7f01627fc640 1 -- 172.21.15.60:0/2101060950 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16625 (secure 0 0 0) 0x7f01680efb30 con 0x7f01580039f0 2024-04-25T10:07:58.447 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:07:58.447 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":47,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:07:09.646687+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.652298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:06.983225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:58.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.442+0000 7f0173ed8640 1 -- 172.21.15.60:0/2101060950 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0150066f90 msgr2=0x7f0150069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.442+0000 7f0173ed8640 1 --2- 172.21.15.60:0/2101060950 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0150066f90 0x7f0150069450 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f016c14c320 tx=0x7f015c009040 comp rx=0 tx=0).stop 2024-04-25T10:07:58.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.442+0000 7f0173ed8640 1 -- 172.21.15.60:0/2101060950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580039f0 msgr2=0x7f016c150c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.442+0000 7f0173ed8640 1 --2- 172.21.15.60:0/2101060950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580039f0 0x7f016c150c40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f016800c160 tx=0x7f0158005840 comp rx=0 tx=0).stop 2024-04-25T10:07:58.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.442+0000 7f0173ed8640 1 -- 172.21.15.60:0/2101060950 shutdown_connections 2024-04-25T10:07:58.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.442+0000 7f0173ed8640 1 --2- 172.21.15.60:0/2101060950 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0150066f90 0x7f0150069450 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.442+0000 7f0173ed8640 1 --2- 172.21.15.60:0/2101060950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f016c15d380 0x7f016c0b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.442+0000 7f0173ed8640 1 --2- 172.21.15.60:0/2101060950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016c14bf40 0x7f016c0cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.442+0000 7f0173ed8640 1 --2- 172.21.15.60:0/2101060950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580039f0 0x7f016c150c40 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.442+0000 7f0173ed8640 1 -- 172.21.15.60:0/2101060950 >> 172.21.15.60:0/2101060950 conn(0x7f016c06c090 msgr2=0x7f016c0b85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.442+0000 7f0173ed8640 1 -- 172.21.15.60:0/2101060950 shutdown_connections 2024-04-25T10:07:58.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.442+0000 7f0173ed8640 1 -- 172.21.15.60:0/2101060950 wait complete. 2024-04-25T10:07:58.465 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:07:58.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.578+0000 7f5a6a415640 1 Processor -- start 2024-04-25T10:07:58.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.578+0000 7f5a6a415640 1 -- start start 2024-04-25T10:07:58.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a6a415640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a6415f480 0x7f5a6415f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a6a415640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5a640cf380 0x7f5a640cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a6a415640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6415fda0 con 0x7f5a640cd390 2024-04-25T10:07:58.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a6a415640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a640cc5d0 con 0x7f5a6415f480 2024-04-25T10:07:58.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a6a415640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a640cc8b0 con 0x7f5a640cf380 2024-04-25T10:07:58.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a63fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a6415f480 0x7f5a6415f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a637fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5a640cd390 0x7f5a6409a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42974/0 (socket says 172.21.15.60:42974) 2024-04-25T10:07:58.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a637fe640 1 -- 172.21.15.60:0/1390074524 learned_addr learned my addr 172.21.15.60:0/1390074524 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:58.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a689b2640 1 --2- 172.21.15.60:0/1390074524 >> v2:172.21.15.100:3300/0 conn(0x7f5a640cf380 0x7f5a640cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a63fff640 1 -- 172.21.15.60:0/1390074524 >> v2:172.21.15.100:3300/0 conn(0x7f5a640cf380 msgr2=0x7f5a640cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a63fff640 1 --2- 172.21.15.60:0/1390074524 >> v2:172.21.15.100:3300/0 conn(0x7f5a640cf380 0x7f5a640cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a63fff640 1 -- 172.21.15.60:0/1390074524 >> v1:172.21.15.60:6789/0 conn(0x7f5a640cd390 legacy=0x7f5a6409a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a63fff640 1 -- 172.21.15.60:0/1390074524 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a640ccb90 con 0x7f5a6415f480 2024-04-25T10:07:58.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a63fff640 1 --2- 172.21.15.60:0/1390074524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6415f480 0x7f5a6415f860 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5a64071e80 tx=0x7f5a5401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcedce98cd78d236 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a62ffd640 1 -- 172.21.15.60:0/1390074524 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a540a7070 con 0x7f5a6415f480 2024-04-25T10:07:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a62ffd640 1 -- 172.21.15.60:0/1390074524 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a540b0070 con 0x7f5a6415f480 2024-04-25T10:07:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a62ffd640 1 -- 172.21.15.60:0/1390074524 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a540ab890 con 0x7f5a6415f480 2024-04-25T10:07:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a6a415640 1 -- 172.21.15.60:0/1390074524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6415f480 msgr2=0x7f5a6415f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a6a415640 1 --2- 172.21.15.60:0/1390074524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6415f480 0x7f5a6415f860 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5a64071e80 tx=0x7f5a5401c450 comp rx=0 tx=0).stop 2024-04-25T10:07:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a6a415640 1 -- 172.21.15.60:0/1390074524 shutdown_connections 2024-04-25T10:07:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a6a415640 1 --2- 172.21.15.60:0/1390074524 >> v2:172.21.15.100:3300/0 conn(0x7f5a640cf380 0x7f5a640cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a6a415640 1 --2- 172.21.15.60:0/1390074524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6415f480 0x7f5a6415f860 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a6a415640 1 -- 172.21.15.60:0/1390074524 >> 172.21.15.60:0/1390074524 conn(0x7f5a640b32a0 msgr2=0x7f5a640b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a6a415640 1 -- 172.21.15.60:0/1390074524 shutdown_connections 2024-04-25T10:07:58.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.582+0000 7f5a6a415640 1 -- 172.21.15.60:0/1390074524 wait complete. 2024-04-25T10:07:58.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a6a415640 1 Processor -- start 2024-04-25T10:07:58.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a6a415640 1 -- start start 2024-04-25T10:07:58.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a6a415640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a640cd390 0x7f5a64153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a6a415640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a640cf380 0x7f5a641525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a6a415640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6415f480 0x7f5a640b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a6a415640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a64160170 con 0x7f5a6415f480 2024-04-25T10:07:58.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a6a415640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a641602f0 con 0x7f5a640cf380 2024-04-25T10:07:58.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a6a415640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6415cbc0 con 0x7f5a640cd390 2024-04-25T10:07:58.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a689b2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6415f480 0x7f5a640b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a689b2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6415f480 0x7f5a640b6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51298/0 (socket says 172.21.15.60:51298) 2024-04-25T10:07:58.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a689b2640 1 -- 172.21.15.60:0/104143336 learned_addr learned my addr 172.21.15.60:0/104143336 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:58.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a637fe640 1 --2- 172.21.15.60:0/104143336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a640cf380 0x7f5a641525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a63fff640 1 --2- 172.21.15.60:0/104143336 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a640cd390 0x7f5a64153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a689b2640 1 -- 172.21.15.60:0/104143336 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a640cd390 msgr2=0x7f5a64153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a689b2640 1 --2- 172.21.15.60:0/104143336 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a640cd390 0x7f5a64153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a689b2640 1 -- 172.21.15.60:0/104143336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a640cf380 msgr2=0x7f5a641525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a689b2640 1 --2- 172.21.15.60:0/104143336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a640cf380 0x7f5a641525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a689b2640 1 -- 172.21.15.60:0/104143336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a64152df0 con 0x7f5a6415f480 2024-04-25T10:07:58.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a63fff640 1 --2- 172.21.15.60:0/104143336 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a640cd390 0x7f5a64153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:58.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a689b2640 1 --2- 172.21.15.60:0/104143336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6415f480 0x7f5a640b6cc0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c007e90 tx=0x7f5a5c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a60ff9640 1 -- 172.21.15.60:0/104143336 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c014420 con 0x7f5a6415f480 2024-04-25T10:07:58.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a60ff9640 1 -- 172.21.15.60:0/104143336 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c019070 con 0x7f5a6415f480 2024-04-25T10:07:58.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a6a415640 1 -- 172.21.15.60:0/104143336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a641530d0 con 0x7f5a6415f480 2024-04-25T10:07:58.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a60ff9640 1 -- 172.21.15.60:0/104143336 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c014800 con 0x7f5a6415f480 2024-04-25T10:07:58.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a6a415640 1 -- 172.21.15.60:0/104143336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a64153690 con 0x7f5a6415f480 2024-04-25T10:07:58.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a6a415640 1 -- 172.21.15.60:0/104143336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a64074770 con 0x7f5a6415f480 2024-04-25T10:07:58.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.586+0000 7f5a60ff9640 1 -- 172.21.15.60:0/104143336 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a5c0149a0 con 0x7f5a6415f480 2024-04-25T10:07:58.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.590+0000 7f5a60ff9640 1 --2- 172.21.15.60:0/104143336 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5a50066f90 0x7f5a50069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.590+0000 7f5a60ff9640 1 -- 172.21.15.60:0/104143336 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a5c085810 con 0x7f5a6415f480 2024-04-25T10:07:58.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.590+0000 7f5a63fff640 1 --2- 172.21.15.60:0/104143336 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5a50066f90 0x7f5a50069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.590+0000 7f5a63fff640 1 --2- 172.21.15.60:0/104143336 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5a50066f90 0x7f5a50069450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5a540af000 tx=0x7f5a54008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.594+0000 7f5a60ff9640 1 -- 172.21.15.60:0/104143336 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a5c0568b0 con 0x7f5a6415f480 2024-04-25T10:07:58.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.778+0000 7f5a6a415640 1 -- 172.21.15.60:0/104143336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5a64153860 con 0x7f5a6415f480 2024-04-25T10:07:58.789 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:07:58.782+0000 7f3a792d6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:07:58.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.782+0000 7f5a60ff9640 1 -- 172.21.15.60:0/104143336 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5a5c0561b0 con 0x7f5a6415f480 2024-04-25T10:07:58.791 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:07:58.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.786+0000 7f5a6a415640 1 -- 172.21.15.60:0/104143336 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5a50066f90 msgr2=0x7f5a50069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.786+0000 7f5a6a415640 1 --2- 172.21.15.60:0/104143336 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5a50066f90 0x7f5a50069450 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5a540af000 tx=0x7f5a54008040 comp rx=0 tx=0).stop 2024-04-25T10:07:58.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.786+0000 7f5a6a415640 1 -- 172.21.15.60:0/104143336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6415f480 msgr2=0x7f5a640b6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.786+0000 7f5a6a415640 1 --2- 172.21.15.60:0/104143336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6415f480 0x7f5a640b6cc0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c007e90 tx=0x7f5a5c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.786+0000 7f5a6a415640 1 -- 172.21.15.60:0/104143336 shutdown_connections 2024-04-25T10:07:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.786+0000 7f5a6a415640 1 --2- 172.21.15.60:0/104143336 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5a50066f90 0x7f5a50069450 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.786+0000 7f5a6a415640 1 --2- 172.21.15.60:0/104143336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6415f480 0x7f5a640b6cc0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.786+0000 7f5a6a415640 1 --2- 172.21.15.60:0/104143336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a640cf380 0x7f5a641525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.786+0000 7f5a6a415640 1 --2- 172.21.15.60:0/104143336 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a640cd390 0x7f5a64153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.786+0000 7f5a6a415640 1 -- 172.21.15.60:0/104143336 >> 172.21.15.60:0/104143336 conn(0x7f5a640b32a0 msgr2=0x7f5a640b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.786+0000 7f5a6a415640 1 -- 172.21.15.60:0/104143336 shutdown_connections 2024-04-25T10:07:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.786+0000 7f5a6a415640 1 -- 172.21.15.60:0/104143336 wait complete. 2024-04-25T10:07:58.798 INFO:tasks.ceph.mds.a.smithi060.stderr: -4948> 2024-04-25T10:07:13.603+0000 7f9710f5b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:58.802 INFO:tasks.ceph.mds.a.smithi060.stderr: -3134> 2024-04-25T10:07:22.022+0000 7f970cf53640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6000=0-4194304@1} caps={6000=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6000 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x5573ebd8c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:07:58.817 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:07:58.840 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T10:07:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.922+0000 7f86a97ff640 1 Processor -- start 2024-04-25T10:07:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.922+0000 7f86a97ff640 1 -- start start 2024-04-25T10:07:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.922+0000 7f86a97ff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86a40cbcf0 0x7f86a4160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.922+0000 7f86a97ff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f86a4071c30 0x7f86a415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.922+0000 7f86a97ff640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a4144a40 con 0x7f86a415fa20 2024-04-25T10:07:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.922+0000 7f86a97ff640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a40b2dc0 con 0x7f86a4071c30 2024-04-25T10:07:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.922+0000 7f86a97ff640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a415fe80 con 0x7f86a40cbcf0 2024-04-25T10:07:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.922+0000 7f86a27fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f86a4071c30 0x7f86a415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.922+0000 7f86a27fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f86a4071c30 0x7f86a415f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33836/0 (socket says 172.21.15.60:33836) 2024-04-25T10:07:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.922+0000 7f86a27fc640 1 -- 172.21.15.60:0/1987352647 learned_addr learned my addr 172.21.15.60:0/1987352647 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.922+0000 7f86a2ffd640 1 --2- 172.21.15.60:0/1987352647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86a40cbcf0 0x7f86a4160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.922+0000 7f86a27fc640 1 -- 172.21.15.60:0/1987352647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86a40cbcf0 msgr2=0x7f86a4160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.922+0000 7f86a27fc640 1 --2- 172.21.15.60:0/1987352647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86a40cbcf0 0x7f86a4160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.922+0000 7f86a27fc640 1 -- 172.21.15.60:0/1987352647 >> v1:172.21.15.100:6789/0 conn(0x7f86a415fa20 legacy=0x7f86a415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.922+0000 7f86a27fc640 1 -- 172.21.15.60:0/1987352647 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a40cc9f0 con 0x7f86a4071c30 2024-04-25T10:07:58.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.926+0000 7f86a27fc640 1 --2- 172.21.15.60:0/1987352647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a4071c30 0x7f86a415f4e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8694009e50 tx=0x7f869401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27e62a9650857d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.926+0000 7f86a1ffb640 1 -- 172.21.15.60:0/1987352647 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86940a58d0 con 0x7f86a4071c30 2024-04-25T10:07:58.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.926+0000 7f86a1ffb640 1 -- 172.21.15.60:0/1987352647 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86940a5a70 con 0x7f86a4071c30 2024-04-25T10:07:58.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.926+0000 7f86a1ffb640 1 -- 172.21.15.60:0/1987352647 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86940a5e40 con 0x7f86a4071c30 2024-04-25T10:07:58.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.926+0000 7f86a97ff640 1 -- 172.21.15.60:0/1987352647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a4071c30 msgr2=0x7f86a415f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.926+0000 7f86a97ff640 1 --2- 172.21.15.60:0/1987352647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a4071c30 0x7f86a415f4e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8694009e50 tx=0x7f869401ca00 comp rx=0 tx=0).stop 2024-04-25T10:07:58.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.926+0000 7f86a97ff640 1 -- 172.21.15.60:0/1987352647 shutdown_connections 2024-04-25T10:07:58.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.926+0000 7f86a97ff640 1 --2- 172.21.15.60:0/1987352647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a4071c30 0x7f86a415f4e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.926+0000 7f86a97ff640 1 --2- 172.21.15.60:0/1987352647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86a40cbcf0 0x7f86a4160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.926+0000 7f86a97ff640 1 -- 172.21.15.60:0/1987352647 >> 172.21.15.60:0/1987352647 conn(0x7f86a40670b0 msgr2=0x7f86a406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.926+0000 7f86a97ff640 1 -- 172.21.15.60:0/1987352647 shutdown_connections 2024-04-25T10:07:58.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.926+0000 7f86a97ff640 1 -- 172.21.15.60:0/1987352647 wait complete. 2024-04-25T10:07:58.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.926+0000 7f86a97ff640 1 Processor -- start 2024-04-25T10:07:58.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.926+0000 7f86a97ff640 1 -- start start 2024-04-25T10:07:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a97ff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86a4071c30 0x7f86a415b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a97ff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a40cbcf0 0x7f86a40b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a97ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a415fa20 0x7f86a40b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a97ff640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a40720b0 con 0x7f86a40cbcf0 2024-04-25T10:07:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a97ff640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a40cc0d0 con 0x7f86a415fa20 2024-04-25T10:07:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a97ff640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a40cfed0 con 0x7f86a4071c30 2024-04-25T10:07:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a40cbcf0 0x7f86a40b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a40cbcf0 0x7f86a40b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33850/0 (socket says 172.21.15.60:33850) 2024-04-25T10:07:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a27fc640 1 -- 172.21.15.60:0/101384007 learned_addr learned my addr 172.21.15.60:0/101384007 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:07:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a2ffd640 1 --2- 172.21.15.60:0/101384007 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86a4071c30 0x7f86a415b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a37fe640 1 --2- 172.21.15.60:0/101384007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a415fa20 0x7f86a40b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a27fc640 1 -- 172.21.15.60:0/101384007 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86a4071c30 msgr2=0x7f86a415b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a27fc640 1 --2- 172.21.15.60:0/101384007 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86a4071c30 0x7f86a415b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a27fc640 1 -- 172.21.15.60:0/101384007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a415fa20 msgr2=0x7f86a40b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a27fc640 1 --2- 172.21.15.60:0/101384007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a415fa20 0x7f86a40b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a27fc640 1 -- 172.21.15.60:0/101384007 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a415b960 con 0x7f86a40cbcf0 2024-04-25T10:07:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a27fc640 1 --2- 172.21.15.60:0/101384007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a40cbcf0 0x7f86a40b1860 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f86940af040 tx=0x7f86940197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f868b7fe640 1 -- 172.21.15.60:0/101384007 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86940bc030 con 0x7f86a40cbcf0 2024-04-25T10:07:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f868b7fe640 1 -- 172.21.15.60:0/101384007 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86940090d0 con 0x7f86a40cbcf0 2024-04-25T10:07:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a97ff640 1 -- 172.21.15.60:0/101384007 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a4142570 con 0x7f86a40cbcf0 2024-04-25T10:07:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f868b7fe640 1 -- 172.21.15.60:0/101384007 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86940a58d0 con 0x7f86a40cbcf0 2024-04-25T10:07:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a97ff640 1 -- 172.21.15.60:0/101384007 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a4142b30 con 0x7f86a40cbcf0 2024-04-25T10:07:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a97ff640 1 -- 172.21.15.60:0/101384007 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a4074770 con 0x7f86a40cbcf0 2024-04-25T10:07:58.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f868b7fe640 1 -- 172.21.15.60:0/101384007 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86940bf070 con 0x7f86a40cbcf0 2024-04-25T10:07:58.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f868b7fe640 1 --2- 172.21.15.60:0/101384007 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8690066f90 0x7f8690069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f868b7fe640 1 -- 172.21.15.60:0/101384007 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f869411fb70 con 0x7f86a40cbcf0 2024-04-25T10:07:58.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.930+0000 7f86a2ffd640 1 --2- 172.21.15.60:0/101384007 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8690066f90 0x7f8690069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.934+0000 7f86a2ffd640 1 --2- 172.21.15.60:0/101384007 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8690066f90 0x7f8690069450 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f86a40cc410 tx=0x7f8698008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:58.934+0000 7f868b7fe640 1 -- 172.21.15.60:0/101384007 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8694016620 con 0x7f86a40cbcf0 2024-04-25T10:07:59.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.130+0000 7f86a97ff640 1 -- 172.21.15.60:0/101384007 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f86a40cdbe0 con 0x7f86a40cbcf0 2024-04-25T10:07:59.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.786+0000 7f868b7fe640 1 -- 172.21.15.60:0/101384007 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f86940b53a0 con 0x7f86a40cbcf0 2024-04-25T10:07:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.790+0000 7f86a97ff640 1 -- 172.21.15.60:0/101384007 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8690066f90 msgr2=0x7f8690069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.790+0000 7f86a97ff640 1 --2- 172.21.15.60:0/101384007 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8690066f90 0x7f8690069450 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f86a40cc410 tx=0x7f8698008040 comp rx=0 tx=0).stop 2024-04-25T10:07:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.790+0000 7f86a97ff640 1 -- 172.21.15.60:0/101384007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a40cbcf0 msgr2=0x7f86a40b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.790+0000 7f86a97ff640 1 --2- 172.21.15.60:0/101384007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a40cbcf0 0x7f86a40b1860 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f86940af040 tx=0x7f86940197a0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.790+0000 7f86a97ff640 1 -- 172.21.15.60:0/101384007 shutdown_connections 2024-04-25T10:07:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.790+0000 7f86a97ff640 1 --2- 172.21.15.60:0/101384007 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8690066f90 0x7f8690069450 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.790+0000 7f86a97ff640 1 --2- 172.21.15.60:0/101384007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a415fa20 0x7f86a40b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.790+0000 7f86a97ff640 1 --2- 172.21.15.60:0/101384007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86a40cbcf0 0x7f86a40b1860 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.790+0000 7f86a97ff640 1 --2- 172.21.15.60:0/101384007 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86a4071c30 0x7f86a415b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.790+0000 7f86a97ff640 1 -- 172.21.15.60:0/101384007 >> 172.21.15.60:0/101384007 conn(0x7f86a40670b0 msgr2=0x7f86a413e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.790+0000 7f86a97ff640 1 -- 172.21.15.60:0/101384007 shutdown_connections 2024-04-25T10:07:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.790+0000 7f86a97ff640 1 -- 172.21.15.60:0/101384007 wait complete. 2024-04-25T10:07:59.818 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:07:59.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.986+0000 7f425b321640 1 Processor -- start 2024-04-25T10:07:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.986+0000 7f425b321640 1 -- start start 2024-04-25T10:07:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.986+0000 7f425b321640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f425415b450 0x7f42540727e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.986+0000 7f425b321640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425415bd20 con 0x7f42541583f0 2024-04-25T10:07:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.986+0000 7f425b321640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42540cd170 con 0x7f4254072370 2024-04-25T10:07:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.986+0000 7f425b321640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4254072d20 con 0x7f425415b450 2024-04-25T10:07:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.986+0000 7f42590bd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f42541583f0 0x7f42540cdc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49776/0 (socket says 172.21.15.60:49776) 2024-04-25T10:07:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.986+0000 7f42590bd640 1 -- 172.21.15.60:0/3695464786 learned_addr learned my addr 172.21.15.60:0/3695464786 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:07:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.986+0000 7f42598be640 1 --2- 172.21.15.60:0/3695464786 >> v2:172.21.15.60:3300/0 conn(0x7f425415b450 0x7f42540727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.986+0000 7f424bfff640 1 -- 172.21.15.60:0/3695464786 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3968998294 0 0) 0x7f425415bd20 con 0x7f42541583f0 2024-04-25T10:07:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.986+0000 7f424bfff640 1 -- 172.21.15.60:0/3695464786 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4240005740 con 0x7f42541583f0 2024-04-25T10:07:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.986+0000 7f42598be640 1 -- 172.21.15.60:0/3695464786 >> v1:172.21.15.100:6789/0 conn(0x7f4254072370 legacy=0x7f425415ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.986+0000 7f42598be640 1 -- 172.21.15.60:0/3695464786 >> v1:172.21.15.60:6789/0 conn(0x7f42541583f0 legacy=0x7f42540cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.986+0000 7f42598be640 1 -- 172.21.15.60:0/3695464786 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42540ce320 con 0x7f425415b450 2024-04-25T10:07:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f42598be640 1 --2- 172.21.15.60:0/3695464786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f425415b450 0x7f42540727e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f42540ce4f0 tx=0x7f424401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9fab3e2ec8ed0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f424bfff640 1 -- 172.21.15.60:0/3695464786 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42440b0070 con 0x7f425415b450 2024-04-25T10:07:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f424bfff640 1 -- 172.21.15.60:0/3695464786 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42440ab600 con 0x7f425415b450 2024-04-25T10:07:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f424bfff640 1 -- 172.21.15.60:0/3695464786 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42440ab9d0 con 0x7f425415b450 2024-04-25T10:07:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f425b321640 1 -- 172.21.15.60:0/3695464786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f425415b450 msgr2=0x7f42540727e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f425b321640 1 --2- 172.21.15.60:0/3695464786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f425415b450 0x7f42540727e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f42540ce4f0 tx=0x7f424401c410 comp rx=0 tx=0).stop 2024-04-25T10:07:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f425b321640 1 -- 172.21.15.60:0/3695464786 shutdown_connections 2024-04-25T10:07:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f425b321640 1 --2- 172.21.15.60:0/3695464786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f425415b450 0x7f42540727e0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f425b321640 1 -- 172.21.15.60:0/3695464786 >> 172.21.15.60:0/3695464786 conn(0x7f425406e400 msgr2=0x7f425406e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f425b321640 1 -- 172.21.15.60:0/3695464786 shutdown_connections 2024-04-25T10:07:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f425b321640 1 -- 172.21.15.60:0/3695464786 wait complete. 2024-04-25T10:07:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f425b321640 1 Processor -- start 2024-04-25T10:07:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f425b321640 1 -- start start 2024-04-25T10:08:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f425b321640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4254072370 0x7f4254151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f425b321640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42541583f0 0x7f425413aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f425b321640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f425415b450 0x7f42541f2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f425b321640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42540b4a50 con 0x7f425415b450 2024-04-25T10:08:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f425b321640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42540b48d0 con 0x7f4254072370 2024-04-25T10:08:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f425b321640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425406ac10 con 0x7f42541583f0 2024-04-25T10:08:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f42598be640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f425415b450 0x7f42541f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f42598be640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f425415b450 0x7f42541f2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33864/0 (socket says 172.21.15.60:33864) 2024-04-25T10:08:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.990+0000 7f42598be640 1 -- 172.21.15.60:0/3150138030 learned_addr learned my addr 172.21.15.60:0/3150138030 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f42590bd640 1 --2- 172.21.15.60:0/3150138030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4254072370 0x7f4254151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f42598be640 1 -- 172.21.15.60:0/3150138030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42541583f0 msgr2=0x7f425413aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f42588bc640 1 --2- 172.21.15.60:0/3150138030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42541583f0 0x7f425413aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f42598be640 1 --2- 172.21.15.60:0/3150138030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42541583f0 0x7f425413aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f42598be640 1 -- 172.21.15.60:0/3150138030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4254072370 msgr2=0x7f4254151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f42598be640 1 --2- 172.21.15.60:0/3150138030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4254072370 0x7f4254151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f42598be640 1 -- 172.21.15.60:0/3150138030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4254150860 con 0x7f425415b450 2024-04-25T10:08:00.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f42590bd640 1 --2- 172.21.15.60:0/3150138030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4254072370 0x7f4254151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:00.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f42598be640 1 --2- 172.21.15.60:0/3150138030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f425415b450 0x7f42541f2550 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f42440ae040 tx=0x7f424401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f4249ffb640 1 -- 172.21.15.60:0/3150138030 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42440b0050 con 0x7f425415b450 2024-04-25T10:08:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f4249ffb640 1 -- 172.21.15.60:0/3150138030 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42440bc070 con 0x7f425415b450 2024-04-25T10:08:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f425b321640 1 -- 172.21.15.60:0/3150138030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42541520c0 con 0x7f425415b450 2024-04-25T10:08:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f4249ffb640 1 -- 172.21.15.60:0/3150138030 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244006e20 con 0x7f425415b450 2024-04-25T10:08:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f425b321640 1 -- 172.21.15.60:0/3150138030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4254150dc0 con 0x7f425415b450 2024-04-25T10:08:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f425b321640 1 -- 172.21.15.60:0/3150138030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4254074770 con 0x7f425415b450 2024-04-25T10:08:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f4249ffb640 1 -- 172.21.15.60:0/3150138030 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f42440b5830 con 0x7f425415b450 2024-04-25T10:08:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f4249ffb640 1 --2- 172.21.15.60:0/3150138030 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4238066f60 0x7f4238069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f4249ffb640 1 -- 172.21.15.60:0/3150138030 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f424411f5c0 con 0x7f425415b450 2024-04-25T10:08:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f42590bd640 1 --2- 172.21.15.60:0/3150138030 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4238066f60 0x7f4238069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:07:59.994+0000 7f42590bd640 1 --2- 172.21.15.60:0/3150138030 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4238066f60 0x7f4238069420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f424c001570 tx=0x7f424c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.002+0000 7f4249ffb640 1 -- 172.21.15.60:0/3150138030 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4254150860 con 0x7f425415b450 2024-04-25T10:08:00.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.154+0000 7f425b321640 1 -- 172.21.15.60:0/3150138030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4254166190 con 0x7f425415b450 2024-04-25T10:08:00.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.794+0000 7f4249ffb640 1 -- 172.21.15.60:0/3150138030 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4244016090 con 0x7f425415b450 2024-04-25T10:08:00.801 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:08:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.794+0000 7f425b321640 1 -- 172.21.15.60:0/3150138030 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4238066f60 msgr2=0x7f4238069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.794+0000 7f425b321640 1 --2- 172.21.15.60:0/3150138030 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4238066f60 0x7f4238069420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f424c001570 tx=0x7f424c009040 comp rx=0 tx=0).stop 2024-04-25T10:08:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.794+0000 7f425b321640 1 -- 172.21.15.60:0/3150138030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f425415b450 msgr2=0x7f42541f2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.794+0000 7f425b321640 1 --2- 172.21.15.60:0/3150138030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f425415b450 0x7f42541f2550 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f42440ae040 tx=0x7f424401ac50 comp rx=0 tx=0).stop 2024-04-25T10:08:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.798+0000 7f425b321640 1 -- 172.21.15.60:0/3150138030 shutdown_connections 2024-04-25T10:08:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.798+0000 7f425b321640 1 --2- 172.21.15.60:0/3150138030 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4238066f60 0x7f4238069420 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.798+0000 7f425b321640 1 --2- 172.21.15.60:0/3150138030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f425415b450 0x7f42541f2550 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.798+0000 7f425b321640 1 --2- 172.21.15.60:0/3150138030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42541583f0 0x7f425413aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.798+0000 7f425b321640 1 --2- 172.21.15.60:0/3150138030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4254072370 0x7f4254151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.798+0000 7f425b321640 1 -- 172.21.15.60:0/3150138030 >> 172.21.15.60:0/3150138030 conn(0x7f425406e400 msgr2=0x7f42540b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.798+0000 7f425b321640 1 -- 172.21.15.60:0/3150138030 shutdown_connections 2024-04-25T10:08:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.798+0000 7f425b321640 1 -- 172.21.15.60:0/3150138030 wait complete. 2024-04-25T10:08:00.833 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:08:00.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.958+0000 7f77e88f1640 1 Processor -- start 2024-04-25T10:08:00.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.958+0000 7f77e88f1640 1 -- start start 2024-04-25T10:08:00.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.958+0000 7f77e88f1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f77e4144e40 0x7f77e40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.958+0000 7f77e88f1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77e4071980 0x7f77e40b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.958+0000 7f77e88f1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e40cf810 con 0x7f77e40cf0a0 2024-04-25T10:08:00.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.958+0000 7f77e88f1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e40b3060 con 0x7f77e4144e40 2024-04-25T10:08:00.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.958+0000 7f77e88f1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e40cfaf0 con 0x7f77e4071980 2024-04-25T10:08:00.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.958+0000 7f77e259c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f77e4144e40 0x7f77e40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.958+0000 7f77e2d9d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77e4071980 0x7f77e40b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.958+0000 7f77e1d9b640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f77e40cf0a0 0x7f77e4071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:33122/0 (socket says 172.21.15.60:33122) 2024-04-25T10:08:00.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.958+0000 7f77e259c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f77e4144e40 0x7f77e40ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:48042/0 (socket says 172.21.15.60:48042) 2024-04-25T10:08:00.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.958+0000 7f77e1d9b640 1 -- 172.21.15.60:0/727327106 learned_addr learned my addr 172.21.15.60:0/727327106 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:00.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.962+0000 7f77e2d9d640 1 -- 172.21.15.60:0/727327106 >> v2:172.21.15.100:3300/0 conn(0x7f77e4144e40 msgr2=0x7f77e40ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.962+0000 7f77e2d9d640 1 --2- 172.21.15.60:0/727327106 >> v2:172.21.15.100:3300/0 conn(0x7f77e4144e40 0x7f77e40ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.962+0000 7f77e2d9d640 1 -- 172.21.15.60:0/727327106 >> v1:172.21.15.100:6789/0 conn(0x7f77e40cf0a0 legacy=0x7f77e4071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.962+0000 7f77e2d9d640 1 -- 172.21.15.60:0/727327106 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e40cfdd0 con 0x7f77e4071980 2024-04-25T10:08:00.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.962+0000 7f77e2d9d640 1 --2- 172.21.15.60:0/727327106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77e4071980 0x7f77e40b34f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009d90 tx=0x7f77dc01c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efe55315b1d1ccec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.962+0000 7f77e159a640 1 -- 172.21.15.60:0/727327106 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0b2070 con 0x7f77e4071980 2024-04-25T10:08:00.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.962+0000 7f77e159a640 1 -- 172.21.15.60:0/727327106 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e4145430 con 0x7f77e4071980 2024-04-25T10:08:00.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.962+0000 7f77e159a640 1 -- 172.21.15.60:0/727327106 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0a5c10 con 0x7f77e4071980 2024-04-25T10:08:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.962+0000 7f77e88f1640 1 -- 172.21.15.60:0/727327106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77e4071980 msgr2=0x7f77e40b34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.962+0000 7f77e88f1640 1 --2- 172.21.15.60:0/727327106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77e4071980 0x7f77e40b34f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009d90 tx=0x7f77dc01c910 comp rx=0 tx=0).stop 2024-04-25T10:08:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.962+0000 7f77e88f1640 1 -- 172.21.15.60:0/727327106 shutdown_connections 2024-04-25T10:08:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.962+0000 7f77e88f1640 1 --2- 172.21.15.60:0/727327106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77e4071980 0x7f77e40b34f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.962+0000 7f77e88f1640 1 --2- 172.21.15.60:0/727327106 >> v2:172.21.15.100:3300/0 conn(0x7f77e4144e40 0x7f77e40ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.962+0000 7f77e88f1640 1 -- 172.21.15.60:0/727327106 >> 172.21.15.60:0/727327106 conn(0x7f77e4066fa0 msgr2=0x7f77e406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.962+0000 7f77e88f1640 1 -- 172.21.15.60:0/727327106 shutdown_connections 2024-04-25T10:08:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.962+0000 7f77e88f1640 1 -- 172.21.15.60:0/727327106 wait complete. 2024-04-25T10:08:00.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e88f1640 1 Processor -- start 2024-04-25T10:08:00.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e88f1640 1 -- start start 2024-04-25T10:08:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e88f1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77e4071980 0x7f77e415b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e88f1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77e40cf0a0 0x7f77e406eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e88f1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77e4144e40 0x7f77e4143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e88f1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e40b4610 con 0x7f77e4144e40 2024-04-25T10:08:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e88f1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e40b4490 con 0x7f77e4071980 2024-04-25T10:08:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e88f1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e40b4790 con 0x7f77e40cf0a0 2024-04-25T10:08:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77e4144e40 0x7f77e4143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77e4144e40 0x7f77e4143c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33868/0 (socket says 172.21.15.60:33868) 2024-04-25T10:08:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e2d9d640 1 -- 172.21.15.60:0/1917860725 learned_addr learned my addr 172.21.15.60:0/1917860725 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e259c640 1 --2- 172.21.15.60:0/1917860725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77e4071980 0x7f77e415b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e1d9b640 1 --2- 172.21.15.60:0/1917860725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77e40cf0a0 0x7f77e406eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e2d9d640 1 -- 172.21.15.60:0/1917860725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77e40cf0a0 msgr2=0x7f77e406eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e2d9d640 1 --2- 172.21.15.60:0/1917860725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77e40cf0a0 0x7f77e406eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e2d9d640 1 -- 172.21.15.60:0/1917860725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77e4071980 msgr2=0x7f77e415b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e2d9d640 1 --2- 172.21.15.60:0/1917860725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77e4071980 0x7f77e415b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e2d9d640 1 -- 172.21.15.60:0/1917860725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e406d520 con 0x7f77e4144e40 2024-04-25T10:08:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e2d9d640 1 --2- 172.21.15.60:0/1917860725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77e4144e40 0x7f77e4143c10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0b0040 tx=0x7f77dc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77daffd640 1 -- 172.21.15.60:0/1917860725 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc0b2050 con 0x7f77e4144e40 2024-04-25T10:08:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77daffd640 1 -- 172.21.15.60:0/1917860725 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77dc0be070 con 0x7f77e4144e40 2024-04-25T10:08:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e88f1640 1 -- 172.21.15.60:0/1917860725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e406d800 con 0x7f77e4144e40 2024-04-25T10:08:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77daffd640 1 -- 172.21.15.60:0/1917860725 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc009450 con 0x7f77e4144e40 2024-04-25T10:08:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e88f1640 1 -- 172.21.15.60:0/1917860725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e406ddc0 con 0x7f77e4144e40 2024-04-25T10:08:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.966+0000 7f77e88f1640 1 -- 172.21.15.60:0/1917860725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e40b29e0 con 0x7f77e4144e40 2024-04-25T10:08:00.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.970+0000 7f77daffd640 1 -- 172.21.15.60:0/1917860725 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77dc0ada90 con 0x7f77e4144e40 2024-04-25T10:08:00.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.970+0000 7f77daffd640 1 --2- 172.21.15.60:0/1917860725 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f77c4066f60 0x7f77c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.970+0000 7f77daffd640 1 -- 172.21.15.60:0/1917860725 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f77dc120af0 con 0x7f77e4144e40 2024-04-25T10:08:00.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.970+0000 7f77e259c640 1 --2- 172.21.15.60:0/1917860725 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f77c4066f60 0x7f77c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.970+0000 7f77e259c640 1 --2- 172.21.15.60:0/1917860725 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f77c4066f60 0x7f77c4069420 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009af0 tx=0x7f77cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:00.974+0000 7f77daffd640 1 -- 172.21.15.60:0/1917860725 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f77dc016560 con 0x7f77e4144e40 2024-04-25T10:08:01.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.134+0000 7f77e88f1640 1 -- 172.21.15.60:0/1917860725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f77e4166230 con 0x7f77e4144e40 2024-04-25T10:08:01.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.794+0000 7f77daffd640 1 -- 172.21.15.60:0/1917860725 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) v1 ==== 169+0+0 (secure 0 0 0) 0x7f77dc0f1d90 con 0x7f77e4144e40 2024-04-25T10:08:01.804 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T10:08:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.802+0000 7f77e88f1640 1 -- 172.21.15.60:0/1917860725 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f77c4066f60 msgr2=0x7f77c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.802+0000 7f77e88f1640 1 --2- 172.21.15.60:0/1917860725 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f77c4066f60 0x7f77c4069420 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009af0 tx=0x7f77cc008040 comp rx=0 tx=0).stop 2024-04-25T10:08:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.802+0000 7f77e88f1640 1 -- 172.21.15.60:0/1917860725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77e4144e40 msgr2=0x7f77e4143c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.802+0000 7f77e88f1640 1 --2- 172.21.15.60:0/1917860725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77e4144e40 0x7f77e4143c10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0b0040 tx=0x7f77dc019710 comp rx=0 tx=0).stop 2024-04-25T10:08:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.802+0000 7f77e88f1640 1 -- 172.21.15.60:0/1917860725 shutdown_connections 2024-04-25T10:08:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.802+0000 7f77e88f1640 1 --2- 172.21.15.60:0/1917860725 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f77c4066f60 0x7f77c4069420 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.802+0000 7f77e88f1640 1 --2- 172.21.15.60:0/1917860725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77e4144e40 0x7f77e4143c10 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.802+0000 7f77e88f1640 1 --2- 172.21.15.60:0/1917860725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77e40cf0a0 0x7f77e406eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.802+0000 7f77e88f1640 1 --2- 172.21.15.60:0/1917860725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77e4071980 0x7f77e415b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.802+0000 7f77e88f1640 1 -- 172.21.15.60:0/1917860725 >> 172.21.15.60:0/1917860725 conn(0x7f77e4066fa0 msgr2=0x7f77e4073ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.802+0000 7f77e88f1640 1 -- 172.21.15.60:0/1917860725 shutdown_connections 2024-04-25T10:08:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.802+0000 7f77e88f1640 1 -- 172.21.15.60:0/1917860725 wait complete. 2024-04-25T10:08:01.835 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.958+0000 7f6d14730640 1 Processor -- start 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.958+0000 7f6d14730640 1 -- start start 2024-04-25T10:08:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.958+0000 7f6d14730640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6d0c0b4e40 0x7f6d0c15c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.958+0000 7f6d14730640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d0c0af0d0 0x7f6d0c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.958+0000 7f6d14730640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d0c160280 con 0x7f6d0c0acde0 2024-04-25T10:08:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.958+0000 7f6d14730640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d0c0af550 con 0x7f6d0c0b4e40 2024-04-25T10:08:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.958+0000 7f6d14730640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c15fd50 con 0x7f6d0c0af0d0 2024-04-25T10:08:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.958+0000 7f6d11ccb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6d0c0b4e40 0x7f6d0c15c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.958+0000 7f6d11ccb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6d0c0b4e40 0x7f6d0c15c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33880/0 (socket says 172.21.15.60:33880) 2024-04-25T10:08:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.958+0000 7f6d11ccb640 1 -- 172.21.15.60:0/2868113777 learned_addr learned my addr 172.21.15.60:0/2868113777 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.962+0000 7f6d11ccb640 1 -- 172.21.15.60:0/2868113777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d0c0af0d0 msgr2=0x7f6d0c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.962+0000 7f6d11ccb640 1 --2- 172.21.15.60:0/2868113777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d0c0af0d0 0x7f6d0c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.962+0000 7f6d11ccb640 1 -- 172.21.15.60:0/2868113777 >> v1:172.21.15.60:6789/0 conn(0x7f6d0c0acde0 legacy=0x7f6d0c0ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.962+0000 7f6d11ccb640 1 -- 172.21.15.60:0/2868113777 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0c15bc20 con 0x7f6d0c0b4e40 2024-04-25T10:08:01.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.962+0000 7f6d11ccb640 1 --2- 172.21.15.60:0/2868113777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d0c0b4e40 0x7f6d0c15c9d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009870 tx=0x7f6cfc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2402dd65e0dc14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.962+0000 7f6d114ca640 1 -- 172.21.15.60:0/2868113777 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0ab650 con 0x7f6d0c0b4e40 2024-04-25T10:08:01.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.962+0000 7f6d114ca640 1 -- 172.21.15.60:0/2868113777 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cfc0b0070 con 0x7f6d0c0b4e40 2024-04-25T10:08:01.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.962+0000 7f6d114ca640 1 -- 172.21.15.60:0/2868113777 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0aba10 con 0x7f6d0c0b4e40 2024-04-25T10:08:01.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.962+0000 7f6d14730640 1 -- 172.21.15.60:0/2868113777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d0c0b4e40 msgr2=0x7f6d0c15c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.962+0000 7f6d14730640 1 --2- 172.21.15.60:0/2868113777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d0c0b4e40 0x7f6d0c15c9d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009870 tx=0x7f6cfc01c560 comp rx=0 tx=0).stop 2024-04-25T10:08:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.962+0000 7f6d14730640 1 -- 172.21.15.60:0/2868113777 shutdown_connections 2024-04-25T10:08:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.962+0000 7f6d14730640 1 --2- 172.21.15.60:0/2868113777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d0c0af0d0 0x7f6d0c06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.962+0000 7f6d14730640 1 --2- 172.21.15.60:0/2868113777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d0c0b4e40 0x7f6d0c15c9d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.962+0000 7f6d14730640 1 -- 172.21.15.60:0/2868113777 >> 172.21.15.60:0/2868113777 conn(0x7f6d0c0cfdd0 msgr2=0x7f6d0c0d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.962+0000 7f6d14730640 1 -- 172.21.15.60:0/2868113777 shutdown_connections 2024-04-25T10:08:01.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.962+0000 7f6d14730640 1 -- 172.21.15.60:0/2868113777 wait complete. 2024-04-25T10:08:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d14730640 1 Processor -- start 2024-04-25T10:08:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d14730640 1 -- start start 2024-04-25T10:08:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d14730640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d0c0acde0 0x7f6d0c164980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d14730640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d0c0af0d0 0x7f6d0c1435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d14730640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d0c0b4e40 0x7f6d0c0b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d14730640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c0adbc0 con 0x7f6d0c0acde0 2024-04-25T10:08:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d14730640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c15c4b0 con 0x7f6d0c0b4e40 2024-04-25T10:08:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d14730640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c1686e0 con 0x7f6d0c0af0d0 2024-04-25T10:08:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d124cc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d0c0acde0 0x7f6d0c164980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d124cc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d0c0acde0 0x7f6d0c164980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33884/0 (socket says 172.21.15.60:33884) 2024-04-25T10:08:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d124cc640 1 -- 172.21.15.60:0/634114808 learned_addr learned my addr 172.21.15.60:0/634114808 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d12ccd640 1 --2- 172.21.15.60:0/634114808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d0c0b4e40 0x7f6d0c0b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d124cc640 1 -- 172.21.15.60:0/634114808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d0c0af0d0 msgr2=0x7f6d0c1435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d11ccb640 1 --2- 172.21.15.60:0/634114808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d0c0af0d0 0x7f6d0c1435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d124cc640 1 --2- 172.21.15.60:0/634114808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d0c0af0d0 0x7f6d0c1435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d124cc640 1 -- 172.21.15.60:0/634114808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d0c0b4e40 msgr2=0x7f6d0c0b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d124cc640 1 --2- 172.21.15.60:0/634114808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d0c0b4e40 0x7f6d0c0b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d124cc640 1 -- 172.21.15.60:0/634114808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0c165280 con 0x7f6d0c0acde0 2024-04-25T10:08:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d12ccd640 1 --2- 172.21.15.60:0/634114808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d0c0b4e40 0x7f6d0c0b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d124cc640 1 --2- 172.21.15.60:0/634114808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d0c0acde0 0x7f6d0c164980 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800c450 tx=0x7f6d0800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d02ffd640 1 -- 172.21.15.60:0/634114808 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d08016020 con 0x7f6d0c0acde0 2024-04-25T10:08:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d02ffd640 1 -- 172.21.15.60:0/634114808 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d080052d0 con 0x7f6d0c0acde0 2024-04-25T10:08:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d14730640 1 -- 172.21.15.60:0/634114808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0c165560 con 0x7f6d0c0acde0 2024-04-25T10:08:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d02ffd640 1 -- 172.21.15.60:0/634114808 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d08007260 con 0x7f6d0c0acde0 2024-04-25T10:08:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d14730640 1 -- 172.21.15.60:0/634114808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0c165b20 con 0x7f6d0c0acde0 2024-04-25T10:08:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d14730640 1 -- 172.21.15.60:0/634114808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0c074770 con 0x7f6d0c0acde0 2024-04-25T10:08:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.966+0000 7f6d02ffd640 1 -- 172.21.15.60:0/634114808 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d08007400 con 0x7f6d0c0acde0 2024-04-25T10:08:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.970+0000 7f6d02ffd640 1 --2- 172.21.15.60:0/634114808 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6cf0066f90 0x7f6cf0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.970+0000 7f6d02ffd640 1 -- 172.21.15.60:0/634114808 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6d080847a0 con 0x7f6d0c0acde0 2024-04-25T10:08:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.970+0000 7f6d11ccb640 1 --2- 172.21.15.60:0/634114808 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6cf0066f90 0x7f6cf0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.970+0000 7f6d11ccb640 1 --2- 172.21.15.60:0/634114808 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6cf0066f90 0x7f6cf0069450 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0af000 tx=0x7f6cfc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:01.974+0000 7f6d02ffd640 1 -- 172.21.15.60:0/634114808 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6d08020020 con 0x7f6d0c0acde0 2024-04-25T10:08:02.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.134+0000 7f6d14730640 1 -- 172.21.15.60:0/634114808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6d0c0b0520 con 0x7f6d0c0acde0 2024-04-25T10:08:02.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.142+0000 7f6d02ffd640 1 -- 172.21.15.60:0/634114808 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6d08005480 con 0x7f6d0c0acde0 2024-04-25T10:08:02.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.142+0000 7f6d14730640 1 -- 172.21.15.60:0/634114808 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6cf0066f90 msgr2=0x7f6cf0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.142+0000 7f6d14730640 1 --2- 172.21.15.60:0/634114808 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6cf0066f90 0x7f6cf0069450 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0af000 tx=0x7f6cfc008040 comp rx=0 tx=0).stop 2024-04-25T10:08:02.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.142+0000 7f6d14730640 1 -- 172.21.15.60:0/634114808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d0c0acde0 msgr2=0x7f6d0c164980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.142+0000 7f6d14730640 1 --2- 172.21.15.60:0/634114808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d0c0acde0 0x7f6d0c164980 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800c450 tx=0x7f6d0800c910 comp rx=0 tx=0).stop 2024-04-25T10:08:02.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.142+0000 7f6d14730640 1 -- 172.21.15.60:0/634114808 shutdown_connections 2024-04-25T10:08:02.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.142+0000 7f6d14730640 1 --2- 172.21.15.60:0/634114808 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6cf0066f90 0x7f6cf0069450 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.142+0000 7f6d14730640 1 --2- 172.21.15.60:0/634114808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d0c0b4e40 0x7f6d0c0b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.146+0000 7f6d14730640 1 --2- 172.21.15.60:0/634114808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d0c0af0d0 0x7f6d0c1435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.146+0000 7f6d14730640 1 --2- 172.21.15.60:0/634114808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d0c0acde0 0x7f6d0c164980 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.146+0000 7f6d14730640 1 -- 172.21.15.60:0/634114808 >> 172.21.15.60:0/634114808 conn(0x7f6d0c0cfdd0 msgr2=0x7f6d0c0ad1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.146+0000 7f6d14730640 1 -- 172.21.15.60:0/634114808 shutdown_connections 2024-04-25T10:08:02.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.146+0000 7f6d14730640 1 -- 172.21.15.60:0/634114808 wait complete. 2024-04-25T10:08:02.169 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-25T10:08:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.286+0000 7fb8c36b4640 1 Processor -- start 2024-04-25T10:08:02.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.286+0000 7fb8c36b4640 1 -- start start 2024-04-25T10:08:02.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.286+0000 7fb8c36b4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8bc06bc20 0x7fb8bc06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.286+0000 7fb8c36b4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb8bc06f0e0 0x7fb8bc06f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.286+0000 7fb8c36b4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8bc06c540 con 0x7fb8bc159f90 2024-04-25T10:08:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.286+0000 7fb8c36b4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8bc070030 con 0x7fb8bc06bc20 2024-04-25T10:08:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.286+0000 7fb8c36b4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8bc0b2d90 con 0x7fb8bc06f0e0 2024-04-25T10:08:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.286+0000 7fb8c1450640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8bc06bc20 0x7fb8bc06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.286+0000 7fb8c1450640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8bc06bc20 0x7fb8bc06c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33896/0 (socket says 172.21.15.60:33896) 2024-04-25T10:08:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.286+0000 7fb8c1450640 1 -- 172.21.15.60:0/3781735740 learned_addr learned my addr 172.21.15.60:0/3781735740 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.286+0000 7fb8c1c51640 1 --2- 172.21.15.60:0/3781735740 >> v2:172.21.15.100:3300/0 conn(0x7fb8bc06f0e0 0x7fb8bc06f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.286+0000 7fb8c1450640 1 -- 172.21.15.60:0/3781735740 >> v2:172.21.15.100:3300/0 conn(0x7fb8bc06f0e0 msgr2=0x7fb8bc06f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.286+0000 7fb8c1450640 1 --2- 172.21.15.60:0/3781735740 >> v2:172.21.15.100:3300/0 conn(0x7fb8bc06f0e0 0x7fb8bc06f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.286+0000 7fb8c1450640 1 -- 172.21.15.60:0/3781735740 >> v1:172.21.15.100:6789/0 conn(0x7fb8bc159f90 legacy=0x7fb8bc0b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.286+0000 7fb8c1450640 1 -- 172.21.15.60:0/3781735740 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8bc0b44b0 con 0x7fb8bc06bc20 2024-04-25T10:08:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.290+0000 7fb8c1450640 1 --2- 172.21.15.60:0/3781735740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8bc06bc20 0x7fb8bc06c000 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0717e0 tx=0x7fb8b401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f025613a8d116f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.290+0000 7fb8b3fff640 1 -- 172.21.15.60:0/3781735740 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b40a5540 con 0x7fb8bc06bc20 2024-04-25T10:08:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.290+0000 7fb8b3fff640 1 -- 172.21.15.60:0/3781735740 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b40a56e0 con 0x7fb8bc06bc20 2024-04-25T10:08:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.290+0000 7fb8b3fff640 1 -- 172.21.15.60:0/3781735740 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b40a5ab0 con 0x7fb8bc06bc20 2024-04-25T10:08:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.290+0000 7fb8c36b4640 1 -- 172.21.15.60:0/3781735740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8bc06bc20 msgr2=0x7fb8bc06c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.290+0000 7fb8c36b4640 1 --2- 172.21.15.60:0/3781735740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8bc06bc20 0x7fb8bc06c000 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0717e0 tx=0x7fb8b401c920 comp rx=0 tx=0).stop 2024-04-25T10:08:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.290+0000 7fb8c36b4640 1 -- 172.21.15.60:0/3781735740 shutdown_connections 2024-04-25T10:08:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.290+0000 7fb8c36b4640 1 --2- 172.21.15.60:0/3781735740 >> v2:172.21.15.100:3300/0 conn(0x7fb8bc06f0e0 0x7fb8bc06f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.290+0000 7fb8c36b4640 1 --2- 172.21.15.60:0/3781735740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8bc06bc20 0x7fb8bc06c000 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.290+0000 7fb8c36b4640 1 -- 172.21.15.60:0/3781735740 >> 172.21.15.60:0/3781735740 conn(0x7fb8bc071b90 msgr2=0x7fb8bc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.290+0000 7fb8c36b4640 1 -- 172.21.15.60:0/3781735740 shutdown_connections 2024-04-25T10:08:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.290+0000 7fb8c36b4640 1 -- 172.21.15.60:0/3781735740 wait complete. 2024-04-25T10:08:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.290+0000 7fb8c36b4640 1 Processor -- start 2024-04-25T10:08:02.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.290+0000 7fb8c36b4640 1 -- start start 2024-04-25T10:08:02.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c36b4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8bc06bc20 0x7fb8bc167670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c36b4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8bc06f0e0 0x7fb8bc149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c36b4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8bc159f90 0x7fb8bc14bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c36b4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc0acd70 con 0x7fb8bc06f0e0 2024-04-25T10:08:02.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c36b4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc0b3880 con 0x7fb8bc06bc20 2024-04-25T10:08:02.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c36b4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc0acef0 con 0x7fb8bc159f90 2024-04-25T10:08:02.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c0c4f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8bc06f0e0 0x7fb8bc149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c0c4f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8bc06f0e0 0x7fb8bc149a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33910/0 (socket says 172.21.15.60:33910) 2024-04-25T10:08:02.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c0c4f640 1 -- 172.21.15.60:0/3773410043 learned_addr learned my addr 172.21.15.60:0/3773410043 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:02.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c1450640 1 --2- 172.21.15.60:0/3773410043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8bc06bc20 0x7fb8bc167670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c1c51640 1 --2- 172.21.15.60:0/3773410043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8bc159f90 0x7fb8bc14bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c0c4f640 1 -- 172.21.15.60:0/3773410043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8bc159f90 msgr2=0x7fb8bc14bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c0c4f640 1 --2- 172.21.15.60:0/3773410043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8bc159f90 0x7fb8bc14bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c0c4f640 1 -- 172.21.15.60:0/3773410043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8bc06bc20 msgr2=0x7fb8bc167670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c0c4f640 1 --2- 172.21.15.60:0/3773410043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8bc06bc20 0x7fb8bc167670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c0c4f640 1 -- 172.21.15.60:0/3773410043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8bc167f70 con 0x7fb8bc06f0e0 2024-04-25T10:08:02.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c0c4f640 1 --2- 172.21.15.60:0/3773410043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8bc06f0e0 0x7fb8bc149a40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00b440 tx=0x7fb8ac00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8b1ffb640 1 -- 172.21.15.60:0/3773410043 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac015020 con 0x7fb8bc06f0e0 2024-04-25T10:08:02.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8b1ffb640 1 -- 172.21.15.60:0/3773410043 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac011890 con 0x7fb8bc06f0e0 2024-04-25T10:08:02.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c36b4640 1 -- 172.21.15.60:0/3773410043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8bc168250 con 0x7fb8bc06f0e0 2024-04-25T10:08:02.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8b1ffb640 1 -- 172.21.15.60:0/3773410043 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac011c80 con 0x7fb8bc06f0e0 2024-04-25T10:08:02.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c36b4640 1 -- 172.21.15.60:0/3773410043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8bc168810 con 0x7fb8bc06f0e0 2024-04-25T10:08:02.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c36b4640 1 -- 172.21.15.60:0/3773410043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8bc074770 con 0x7fb8bc06f0e0 2024-04-25T10:08:02.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8b1ffb640 1 -- 172.21.15.60:0/3773410043 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8ac018020 con 0x7fb8bc06f0e0 2024-04-25T10:08:02.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8b1ffb640 1 --2- 172.21.15.60:0/3773410043 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8a0066f90 0x7fb8a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8b1ffb640 1 -- 172.21.15.60:0/3773410043 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb8ac084ec0 con 0x7fb8bc06f0e0 2024-04-25T10:08:02.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.294+0000 7fb8c1450640 1 --2- 172.21.15.60:0/3773410043 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8a0066f90 0x7fb8a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.298+0000 7fb8c1450640 1 --2- 172.21.15.60:0/3773410043 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8a0066f90 0x7fb8a0069450 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb8b40ae000 tx=0x7fb8b40a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.298+0000 7fb8b1ffb640 1 -- 172.21.15.60:0/3773410043 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb8ac01f020 con 0x7fb8bc06f0e0 2024-04-25T10:08:02.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.518+0000 7fb8c36b4640 1 -- 172.21.15.60:0/3773410043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fb8bc0b4080 con 0x7fb8bc06f0e0 2024-04-25T10:08:02.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.810+0000 7fb8b1ffb640 1 -- 172.21.15.60:0/3773410043 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v156) v1 ==== 149+0+0 (secure 0 0 0) 0x7fb8ac00d040 con 0x7fb8bc06f0e0 2024-04-25T10:08:02.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.814+0000 7fb8c36b4640 1 -- 172.21.15.60:0/3773410043 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8a0066f90 msgr2=0x7fb8a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.814+0000 7fb8c36b4640 1 --2- 172.21.15.60:0/3773410043 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8a0066f90 0x7fb8a0069450 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb8b40ae000 tx=0x7fb8b40a7070 comp rx=0 tx=0).stop 2024-04-25T10:08:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.814+0000 7fb8c36b4640 1 -- 172.21.15.60:0/3773410043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8bc06f0e0 msgr2=0x7fb8bc149a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.814+0000 7fb8c36b4640 1 --2- 172.21.15.60:0/3773410043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8bc06f0e0 0x7fb8bc149a40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00b440 tx=0x7fb8ac00b900 comp rx=0 tx=0).stop 2024-04-25T10:08:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.814+0000 7fb8c36b4640 1 -- 172.21.15.60:0/3773410043 shutdown_connections 2024-04-25T10:08:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.814+0000 7fb8c36b4640 1 --2- 172.21.15.60:0/3773410043 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8a0066f90 0x7fb8a0069450 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.814+0000 7fb8c36b4640 1 --2- 172.21.15.60:0/3773410043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8bc159f90 0x7fb8bc14bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.814+0000 7fb8c36b4640 1 --2- 172.21.15.60:0/3773410043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8bc06f0e0 0x7fb8bc149a40 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.814+0000 7fb8c36b4640 1 --2- 172.21.15.60:0/3773410043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8bc06bc20 0x7fb8bc167670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.814+0000 7fb8c36b4640 1 -- 172.21.15.60:0/3773410043 >> 172.21.15.60:0/3773410043 conn(0x7fb8bc071b90 msgr2=0x7fb8bc0b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.814+0000 7fb8c36b4640 1 -- 172.21.15.60:0/3773410043 shutdown_connections 2024-04-25T10:08:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.814+0000 7fb8c36b4640 1 -- 172.21.15.60:0/3773410043 wait complete. 2024-04-25T10:08:02.843 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-04-25T10:08:02.843 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T10:08:02.843 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:08:02.844 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:02.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.970+0000 7f50fde47640 1 Processor -- start 2024-04-25T10:08:02.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50fde47640 1 -- start start 2024-04-25T10:08:02.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50fde47640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f80d0a50 0x7f50f80b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50fde47640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50f80ce2b0 0x7f50f80ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50fde47640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f806d200 con 0x7f50f815deb0 2024-04-25T10:08:02.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50fde47640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f80cf2c0 con 0x7f50f80ce2b0 2024-04-25T10:08:02.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50fde47640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f80cefe0 con 0x7f50f80d0a50 2024-04-25T10:08:02.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50f7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50f80ce2b0 0x7f50f80ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50f7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50f80ce2b0 0x7f50f80ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33920/0 (socket says 172.21.15.60:33920) 2024-04-25T10:08:02.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50f7fff640 1 -- 172.21.15.60:0/1073415905 learned_addr learned my addr 172.21.15.60:0/1073415905 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:02.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50f7fff640 1 -- 172.21.15.60:0/1073415905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f80d0a50 msgr2=0x7f50f80b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50f6ffd640 1 --2- 172.21.15.60:0/1073415905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f80d0a50 0x7f50f80b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50f7fff640 1 --2- 172.21.15.60:0/1073415905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f80d0a50 0x7f50f80b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50f7fff640 1 -- 172.21.15.60:0/1073415905 >> v1:172.21.15.100:6789/0 conn(0x7f50f815deb0 legacy=0x7f50f80d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50f7fff640 1 -- 172.21.15.60:0/1073415905 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f80d1d70 con 0x7f50f80ce2b0 2024-04-25T10:08:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50f7fff640 1 --2- 172.21.15.60:0/1073415905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f80ce2b0 0x7f50f80ce760 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009870 tx=0x7f50e801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54828f83bd2f6571 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50f67fc640 1 -- 172.21.15.60:0/1073415905 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e80b1070 con 0x7f50f80ce2b0 2024-04-25T10:08:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50f67fc640 1 -- 172.21.15.60:0/1073415905 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e80ac630 con 0x7f50f80ce2b0 2024-04-25T10:08:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.974+0000 7f50f67fc640 1 -- 172.21.15.60:0/1073415905 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e80a8050 con 0x7f50f80ce2b0 2024-04-25T10:08:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50fde47640 1 -- 172.21.15.60:0/1073415905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f80ce2b0 msgr2=0x7f50f80ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50fde47640 1 --2- 172.21.15.60:0/1073415905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f80ce2b0 0x7f50f80ce760 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009870 tx=0x7f50e801c520 comp rx=0 tx=0).stop 2024-04-25T10:08:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50fde47640 1 -- 172.21.15.60:0/1073415905 shutdown_connections 2024-04-25T10:08:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50fde47640 1 --2- 172.21.15.60:0/1073415905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f80ce2b0 0x7f50f80ce760 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50fde47640 1 --2- 172.21.15.60:0/1073415905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f80d0a50 0x7f50f80b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50fde47640 1 -- 172.21.15.60:0/1073415905 >> 172.21.15.60:0/1073415905 conn(0x7f50f806c090 msgr2=0x7f50f806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50fde47640 1 -- 172.21.15.60:0/1073415905 shutdown_connections 2024-04-25T10:08:02.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50fde47640 1 -- 172.21.15.60:0/1073415905 wait complete. 2024-04-25T10:08:02.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50fde47640 1 Processor -- start 2024-04-25T10:08:02.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50fde47640 1 -- start start 2024-04-25T10:08:02.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50fde47640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f80ce2b0 0x7f50f80b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50fde47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f80d0a50 0x7f50f80b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50fde47640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f815deb0 0x7f50f81483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50fde47640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f80d31e0 con 0x7f50f80d0a50 2024-04-25T10:08:02.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50fde47640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f80d3060 con 0x7f50f815deb0 2024-04-25T10:08:02.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50fde47640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f80d3360 con 0x7f50f80ce2b0 2024-04-25T10:08:02.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f80d0a50 0x7f50f80b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50f7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f815deb0 0x7f50f81483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f80d0a50 0x7f50f80b7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33922/0 (socket says 172.21.15.60:33922) 2024-04-25T10:08:02.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50f6ffd640 1 -- 172.21.15.60:0/4131606054 learned_addr learned my addr 172.21.15.60:0/4131606054 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:02.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50f77fe640 1 --2- 172.21.15.60:0/4131606054 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f80ce2b0 0x7f50f80b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50f6ffd640 1 -- 172.21.15.60:0/4131606054 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f80ce2b0 msgr2=0x7f50f80b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50f6ffd640 1 --2- 172.21.15.60:0/4131606054 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f80ce2b0 0x7f50f80b79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50f6ffd640 1 -- 172.21.15.60:0/4131606054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f815deb0 msgr2=0x7f50f81483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50f6ffd640 1 --2- 172.21.15.60:0/4131606054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f815deb0 0x7f50f81483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.978+0000 7f50f6ffd640 1 -- 172.21.15.60:0/4131606054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f8159f30 con 0x7f50f80d0a50 2024-04-25T10:08:02.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.982+0000 7f50f6ffd640 1 --2- 172.21.15.60:0/4131606054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f80d0a50 0x7f50f80b7040 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f50e000c7f0 tx=0x7f50e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.982+0000 7f50effff640 1 -- 172.21.15.60:0/4131606054 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0015070 con 0x7f50f80d0a50 2024-04-25T10:08:02.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.982+0000 7f50effff640 1 -- 172.21.15.60:0/4131606054 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e0004240 con 0x7f50f80d0a50 2024-04-25T10:08:02.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.982+0000 7f50fde47640 1 -- 172.21.15.60:0/4131606054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f815a210 con 0x7f50f80d0a50 2024-04-25T10:08:02.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.982+0000 7f50effff640 1 -- 172.21.15.60:0/4131606054 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0003620 con 0x7f50f80d0a50 2024-04-25T10:08:02.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.982+0000 7f50fde47640 1 -- 172.21.15.60:0/4131606054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f815a7d0 con 0x7f50f80d0a50 2024-04-25T10:08:02.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.982+0000 7f50fde47640 1 -- 172.21.15.60:0/4131606054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f8074770 con 0x7f50f80d0a50 2024-04-25T10:08:02.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.982+0000 7f50effff640 1 -- 172.21.15.60:0/4131606054 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f50e00039d0 con 0x7f50f80d0a50 2024-04-25T10:08:02.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.982+0000 7f50effff640 1 --2- 172.21.15.60:0/4131606054 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f50d8066f60 0x7f50d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.982+0000 7f50effff640 1 -- 172.21.15.60:0/4131606054 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f50e00850b0 con 0x7f50f80d0a50 2024-04-25T10:08:02.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.982+0000 7f50f77fe640 1 --2- 172.21.15.60:0/4131606054 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f50d8066f60 0x7f50d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.986+0000 7f50f77fe640 1 --2- 172.21.15.60:0/4131606054 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f50d8066f60 0x7f50d8069420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f50f000b450 tx=0x7f50f000d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:02.986+0000 7f50effff640 1 -- 172.21.15.60:0/4131606054 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50e0055ee0 con 0x7f50f80d0a50 2024-04-25T10:08:03.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.150+0000 7f50fde47640 1 -- 172.21.15.60:0/4131606054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50f80b9180 con 0x7f50f80d0a50 2024-04-25T10:08:03.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.150+0000 7f50effff640 1 -- 172.21.15.60:0/4131606054 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+13032 (secure 0 0 0) 0x7f50e0059f80 con 0x7f50f80d0a50 2024-04-25T10:08:03.160 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:03.160 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":50,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:08:01.796661+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/1747757196":"2024-04-26T10:07:58.784523+0000","172.21.15.60:6833/1747757196":"2024-04-26T10:07:58.784523+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:03.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.154+0000 7f50fde47640 1 -- 172.21.15.60:0/4131606054 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f50d8066f60 msgr2=0x7f50d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.154+0000 7f50fde47640 1 --2- 172.21.15.60:0/4131606054 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f50d8066f60 0x7f50d8069420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f50f000b450 tx=0x7f50f000d040 comp rx=0 tx=0).stop 2024-04-25T10:08:03.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.154+0000 7f50fde47640 1 -- 172.21.15.60:0/4131606054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f80d0a50 msgr2=0x7f50f80b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.154+0000 7f50fde47640 1 --2- 172.21.15.60:0/4131606054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f80d0a50 0x7f50f80b7040 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f50e000c7f0 tx=0x7f50e000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.154+0000 7f50fde47640 1 -- 172.21.15.60:0/4131606054 shutdown_connections 2024-04-25T10:08:03.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.154+0000 7f50fde47640 1 --2- 172.21.15.60:0/4131606054 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f50d8066f60 0x7f50d8069420 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.154+0000 7f50fde47640 1 --2- 172.21.15.60:0/4131606054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f815deb0 0x7f50f81483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.154+0000 7f50fde47640 1 --2- 172.21.15.60:0/4131606054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f80d0a50 0x7f50f80b7040 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.154+0000 7f50fde47640 1 --2- 172.21.15.60:0/4131606054 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f80ce2b0 0x7f50f80b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.154+0000 7f50fde47640 1 -- 172.21.15.60:0/4131606054 >> 172.21.15.60:0/4131606054 conn(0x7f50f806c090 msgr2=0x7f50f8157f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.154+0000 7f50fde47640 1 -- 172.21.15.60:0/4131606054 shutdown_connections 2024-04-25T10:08:03.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.154+0000 7f50fde47640 1 -- 172.21.15.60:0/4131606054 wait complete. 2024-04-25T10:08:03.180 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:03.180 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:03.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f792cc0b640 1 Processor -- start 2024-04-25T10:08:03.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f792cc0b640 1 -- start start 2024-04-25T10:08:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f792cc0b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f792815db10 0x7f79280b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f792cc0b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79280aec10 0x7f79280b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f792cc0b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f79280cfcf0 0x7f7928161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f792cc0b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79280accc0 con 0x7f792815db10 2024-04-25T10:08:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f792cc0b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f792815def0 con 0x7f79280cfcf0 2024-04-25T10:08:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f792cc0b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7928071630 con 0x7f79280aec10 2024-04-25T10:08:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f792659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f792815db10 0x7f79280b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f792659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f792815db10 0x7f79280b2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33936/0 (socket says 172.21.15.60:33936) 2024-04-25T10:08:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f792659c640 1 -- 172.21.15.60:0/3726053062 learned_addr learned my addr 172.21.15.60:0/3726053062 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f7926d9d640 1 --2- 172.21.15.60:0/3726053062 >> v2:172.21.15.100:3300/0 conn(0x7f79280cfcf0 0x7f7928161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f7925d9b640 1 --2- 172.21.15.60:0/3726053062 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79280aec10 0x7f79280b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f792659c640 1 -- 172.21.15.60:0/3726053062 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79280aec10 msgr2=0x7f79280b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f792659c640 1 --2- 172.21.15.60:0/3726053062 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79280aec10 0x7f79280b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f792659c640 1 -- 172.21.15.60:0/3726053062 >> v2:172.21.15.100:3300/0 conn(0x7f79280cfcf0 msgr2=0x7f7928161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f792659c640 1 --2- 172.21.15.60:0/3726053062 >> v2:172.21.15.100:3300/0 conn(0x7f79280cfcf0 0x7f7928161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f792659c640 1 -- 172.21.15.60:0/3726053062 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7928071910 con 0x7f792815db10 2024-04-25T10:08:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.302+0000 7f7925d9b640 1 --2- 172.21.15.60:0/3726053062 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79280aec10 0x7f79280b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:03.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.306+0000 7f792659c640 1 --2- 172.21.15.60:0/3726053062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f792815db10 0x7f79280b2ee0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f79280b4ad0 tx=0x7f791c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf3a13709ed24d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.306+0000 7f792559a640 1 -- 172.21.15.60:0/3726053062 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c0b0070 con 0x7f792815db10 2024-04-25T10:08:03.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.306+0000 7f792559a640 1 -- 172.21.15.60:0/3726053062 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c0ab600 con 0x7f792815db10 2024-04-25T10:08:03.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.306+0000 7f792559a640 1 -- 172.21.15.60:0/3726053062 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c0ab9d0 con 0x7f792815db10 2024-04-25T10:08:03.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.306+0000 7f792cc0b640 1 -- 172.21.15.60:0/3726053062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f792815db10 msgr2=0x7f79280b2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.306+0000 7f792cc0b640 1 --2- 172.21.15.60:0/3726053062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f792815db10 0x7f79280b2ee0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f79280b4ad0 tx=0x7f791c01c260 comp rx=0 tx=0).stop 2024-04-25T10:08:03.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.306+0000 7f792cc0b640 1 -- 172.21.15.60:0/3726053062 shutdown_connections 2024-04-25T10:08:03.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.306+0000 7f792cc0b640 1 --2- 172.21.15.60:0/3726053062 >> v2:172.21.15.100:3300/0 conn(0x7f79280cfcf0 0x7f7928161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.306+0000 7f792cc0b640 1 --2- 172.21.15.60:0/3726053062 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79280aec10 0x7f79280b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.306+0000 7f792cc0b640 1 --2- 172.21.15.60:0/3726053062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f792815db10 0x7f79280b2ee0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.306+0000 7f792cc0b640 1 -- 172.21.15.60:0/3726053062 >> 172.21.15.60:0/3726053062 conn(0x7f79280670b0 msgr2=0x7f792806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.306+0000 7f792cc0b640 1 -- 172.21.15.60:0/3726053062 shutdown_connections 2024-04-25T10:08:03.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.306+0000 7f792cc0b640 1 -- 172.21.15.60:0/3726053062 wait complete. 2024-04-25T10:08:03.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f792cc0b640 1 Processor -- start 2024-04-25T10:08:03.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f792cc0b640 1 -- start start 2024-04-25T10:08:03.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f792cc0b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79280aec10 0x7f792814f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f792cc0b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79280cfcf0 0x7f792813de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f792cc0b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f792815db10 0x7f79280b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f792cc0b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79280713d0 con 0x7f792815db10 2024-04-25T10:08:03.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f792cc0b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7928071250 con 0x7f79280aec10 2024-04-25T10:08:03.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f792cc0b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79280ad130 con 0x7f79280cfcf0 2024-04-25T10:08:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f7926d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f792815db10 0x7f79280b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f7926d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f792815db10 0x7f79280b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33946/0 (socket says 172.21.15.60:33946) 2024-04-25T10:08:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f7926d9d640 1 -- 172.21.15.60:0/2397969521 learned_addr learned my addr 172.21.15.60:0/2397969521 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f792659c640 1 --2- 172.21.15.60:0/2397969521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79280aec10 0x7f792814f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f7925d9b640 1 --2- 172.21.15.60:0/2397969521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79280cfcf0 0x7f792813de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f7926d9d640 1 -- 172.21.15.60:0/2397969521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79280cfcf0 msgr2=0x7f792813de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f7926d9d640 1 --2- 172.21.15.60:0/2397969521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79280cfcf0 0x7f792813de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f7926d9d640 1 -- 172.21.15.60:0/2397969521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79280aec10 msgr2=0x7f792814f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f7926d9d640 1 --2- 172.21.15.60:0/2397969521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79280aec10 0x7f792814f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f7926d9d640 1 -- 172.21.15.60:0/2397969521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792814fb90 con 0x7f792815db10 2024-04-25T10:08:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f7926d9d640 1 --2- 172.21.15.60:0/2397969521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f792815db10 0x7f79280b2b70 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f792000efc0 tx=0x7f792000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f7916ffd640 1 -- 172.21.15.60:0/2397969521 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7920018020 con 0x7f792815db10 2024-04-25T10:08:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f7916ffd640 1 -- 172.21.15.60:0/2397969521 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7920014480 con 0x7f792815db10 2024-04-25T10:08:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f792cc0b640 1 -- 172.21.15.60:0/2397969521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792814fe70 con 0x7f792815db10 2024-04-25T10:08:03.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f7916ffd640 1 -- 172.21.15.60:0/2397969521 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7920014870 con 0x7f792815db10 2024-04-25T10:08:03.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f792cc0b640 1 -- 172.21.15.60:0/2397969521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7928150430 con 0x7f792815db10 2024-04-25T10:08:03.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f792cc0b640 1 -- 172.21.15.60:0/2397969521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7928074770 con 0x7f792815db10 2024-04-25T10:08:03.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.310+0000 7f7916ffd640 1 -- 172.21.15.60:0/2397969521 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f792001b020 con 0x7f792815db10 2024-04-25T10:08:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.314+0000 7f7916ffd640 1 --2- 172.21.15.60:0/2397969521 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7908066f90 0x7f7908069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.314+0000 7f7916ffd640 1 -- 172.21.15.60:0/2397969521 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f79200856d0 con 0x7f792815db10 2024-04-25T10:08:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.314+0000 7f792659c640 1 --2- 172.21.15.60:0/2397969521 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7908066f90 0x7f7908069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.314+0000 7f792659c640 1 --2- 172.21.15.60:0/2397969521 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7908066f90 0x7f7908069450 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f791c0ad000 tx=0x7f791c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.318+0000 7f7916ffd640 1 -- 172.21.15.60:0/2397969521 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7920022070 con 0x7f792815db10 2024-04-25T10:08:03.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.478+0000 7f792cc0b640 1 -- 172.21.15.60:0/2397969521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79281606e0 con 0x7f792815db10 2024-04-25T10:08:03.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.478+0000 7f7916ffd640 1 -- 172.21.15.60:0/2397969521 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7920010040 con 0x7f792815db10 2024-04-25T10:08:03.485 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:08:03.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.478+0000 7f792cc0b640 1 -- 172.21.15.60:0/2397969521 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7908066f90 msgr2=0x7f7908069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.478+0000 7f792cc0b640 1 --2- 172.21.15.60:0/2397969521 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7908066f90 0x7f7908069450 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f791c0ad000 tx=0x7f791c008040 comp rx=0 tx=0).stop 2024-04-25T10:08:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.478+0000 7f792cc0b640 1 -- 172.21.15.60:0/2397969521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f792815db10 msgr2=0x7f79280b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.478+0000 7f792cc0b640 1 --2- 172.21.15.60:0/2397969521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f792815db10 0x7f79280b2b70 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f792000efc0 tx=0x7f792000c490 comp rx=0 tx=0).stop 2024-04-25T10:08:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.478+0000 7f792cc0b640 1 -- 172.21.15.60:0/2397969521 shutdown_connections 2024-04-25T10:08:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.478+0000 7f792cc0b640 1 --2- 172.21.15.60:0/2397969521 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7908066f90 0x7f7908069450 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.478+0000 7f792cc0b640 1 --2- 172.21.15.60:0/2397969521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f792815db10 0x7f79280b2b70 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.478+0000 7f792cc0b640 1 --2- 172.21.15.60:0/2397969521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79280cfcf0 0x7f792813de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.478+0000 7f792cc0b640 1 --2- 172.21.15.60:0/2397969521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79280aec10 0x7f792814f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.478+0000 7f792cc0b640 1 -- 172.21.15.60:0/2397969521 >> 172.21.15.60:0/2397969521 conn(0x7f79280670b0 msgr2=0x7f79280b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.478+0000 7f792cc0b640 1 -- 172.21.15.60:0/2397969521 shutdown_connections 2024-04-25T10:08:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.478+0000 7f792cc0b640 1 -- 172.21.15.60:0/2397969521 wait complete. 2024-04-25T10:08:03.505 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-25T10:08:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.614+0000 7f9bca5e3640 1 Processor -- start 2024-04-25T10:08:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.614+0000 7f9bca5e3640 1 -- start start 2024-04-25T10:08:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.614+0000 7f9bca5e3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9bc415bc90 0x7f9bc415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.614+0000 7f9bca5e3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bc40b4660 0x7f9bc415c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.614+0000 7f9bca5e3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc4071ca0 con 0x7f9bc40ce7a0 2024-04-25T10:08:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.614+0000 7f9bca5e3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bc40cc370 con 0x7f9bc415bc90 2024-04-25T10:08:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.614+0000 7f9bca5e3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc40cca40 con 0x7f9bc40b4660 2024-04-25T10:08:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.614+0000 7f9bc37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9bc415bc90 0x7f9bc415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.614+0000 7f9bc37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9bc415bc90 0x7f9bc415c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33952/0 (socket says 172.21.15.60:33952) 2024-04-25T10:08:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.614+0000 7f9bc37fe640 1 -- 172.21.15.60:0/2762361290 learned_addr learned my addr 172.21.15.60:0/2762361290 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.614+0000 7f9bc8b80640 1 --2- 172.21.15.60:0/2762361290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bc40b4660 0x7f9bc415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.614+0000 7f9bc37fe640 1 -- 172.21.15.60:0/2762361290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bc40b4660 msgr2=0x7f9bc415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.614+0000 7f9bc37fe640 1 --2- 172.21.15.60:0/2762361290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bc40b4660 0x7f9bc415c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.614+0000 7f9bc37fe640 1 -- 172.21.15.60:0/2762361290 >> v1:172.21.15.100:6789/0 conn(0x7f9bc40ce7a0 legacy=0x7f9bc40ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.614+0000 7f9bc37fe640 1 -- 172.21.15.60:0/2762361290 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc40ccd20 con 0x7f9bc415bc90 2024-04-25T10:08:03.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc37fe640 1 --2- 172.21.15.60:0/2762361290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bc415bc90 0x7f9bc415c220 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009870 tx=0x7f9bb801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be90568372011d2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc2ffd640 1 -- 172.21.15.60:0/2762361290 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb80b2070 con 0x7f9bc415bc90 2024-04-25T10:08:03.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc2ffd640 1 -- 172.21.15.60:0/2762361290 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb80a9070 con 0x7f9bc415bc90 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc2ffd640 1 -- 172.21.15.60:0/2762361290 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb80ad880 con 0x7f9bc415bc90 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 -- 172.21.15.60:0/2762361290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bc415bc90 msgr2=0x7f9bc415c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 --2- 172.21.15.60:0/2762361290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bc415bc90 0x7f9bc415c220 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009870 tx=0x7f9bb801c560 comp rx=0 tx=0).stop 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 -- 172.21.15.60:0/2762361290 shutdown_connections 2024-04-25T10:08:03.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 --2- 172.21.15.60:0/2762361290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bc415bc90 0x7f9bc415c220 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 --2- 172.21.15.60:0/2762361290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bc40b4660 0x7f9bc415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 -- 172.21.15.60:0/2762361290 >> 172.21.15.60:0/2762361290 conn(0x7f9bc40b12d0 msgr2=0x7f9bc40b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 -- 172.21.15.60:0/2762361290 shutdown_connections 2024-04-25T10:08:03.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 -- 172.21.15.60:0/2762361290 wait complete. 2024-04-25T10:08:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 Processor -- start 2024-04-25T10:08:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 -- start start 2024-04-25T10:08:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bc40ce7a0 0x7f9bc414a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bc415bc90 0x7f9bc406d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bc406d830 0x7f9bc41529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc41685c0 con 0x7f9bc40ce7a0 2024-04-25T10:08:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc40b3db0 con 0x7f9bc406d830 2024-04-25T10:08:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc4168740 con 0x7f9bc415bc90 2024-04-25T10:08:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bc40ce7a0 0x7f9bc414a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bc40ce7a0 0x7f9bc414a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33968/0 (socket says 172.21.15.60:33968) 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc3fff640 1 -- 172.21.15.60:0/3231308001 learned_addr learned my addr 172.21.15.60:0/3231308001 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc37fe640 1 --2- 172.21.15.60:0/3231308001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bc415bc90 0x7f9bc406d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc3fff640 1 -- 172.21.15.60:0/3231308001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bc415bc90 msgr2=0x7f9bc406d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc3fff640 1 --2- 172.21.15.60:0/3231308001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bc415bc90 0x7f9bc406d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc3fff640 1 -- 172.21.15.60:0/3231308001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bc406d830 msgr2=0x7f9bc41529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc8b80640 1 --2- 172.21.15.60:0/3231308001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bc406d830 0x7f9bc41529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc3fff640 1 --2- 172.21.15.60:0/3231308001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bc406d830 0x7f9bc41529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc3fff640 1 -- 172.21.15.60:0/3231308001 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc414a600 con 0x7f9bc40ce7a0 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc37fe640 1 --2- 172.21.15.60:0/3231308001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bc415bc90 0x7f9bc406d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc3fff640 1 --2- 172.21.15.60:0/3231308001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bc40ce7a0 0x7f9bc414a0c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400c370 tx=0x7f9bb400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc0ff9640 1 -- 172.21.15.60:0/3231308001 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb40052d0 con 0x7f9bc40ce7a0 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc0ff9640 1 -- 172.21.15.60:0/3231308001 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb4005470 con 0x7f9bc40ce7a0 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 -- 172.21.15.60:0/3231308001 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc40ad2b0 con 0x7f9bc40ce7a0 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bc0ff9640 1 -- 172.21.15.60:0/3231308001 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4007260 con 0x7f9bc40ce7a0 2024-04-25T10:08:03.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.618+0000 7f9bca5e3640 1 -- 172.21.15.60:0/3231308001 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc40ad870 con 0x7f9bc40ce7a0 2024-04-25T10:08:03.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.622+0000 7f9bca5e3640 1 -- 172.21.15.60:0/3231308001 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc406dc40 con 0x7f9bc40ce7a0 2024-04-25T10:08:03.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.622+0000 7f9bc0ff9640 1 -- 172.21.15.60:0/3231308001 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9bb4007400 con 0x7f9bc40ce7a0 2024-04-25T10:08:03.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.622+0000 7f9bc0ff9640 1 --2- 172.21.15.60:0/3231308001 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9bb0066f60 0x7f9bb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.622+0000 7f9bc0ff9640 1 -- 172.21.15.60:0/3231308001 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9bb4084170 con 0x7f9bc40ce7a0 2024-04-25T10:08:03.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.622+0000 7f9bc37fe640 1 --2- 172.21.15.60:0/3231308001 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9bb0066f60 0x7f9bb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.622+0000 7f9bc37fe640 1 --2- 172.21.15.60:0/3231308001 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9bb0066f60 0x7f9bb0069420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9bb80af000 tx=0x7f9bb80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.626+0000 7f9bc0ff9640 1 -- 172.21.15.60:0/3231308001 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9bc406dc40 con 0x7f9bc40ce7a0 2024-04-25T10:08:03.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:03.842+0000 7f9bca5e3640 1 -- 172.21.15.60:0/3231308001 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f9bc4074770 con 0x7f9bc40ce7a0 2024-04-25T10:08:04.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.822+0000 7f9bc0ff9640 1 -- 172.21.15.60:0/3231308001 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v158) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9bb4055480 con 0x7f9bc40ce7a0 2024-04-25T10:08:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.826+0000 7f9bca5e3640 1 -- 172.21.15.60:0/3231308001 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9bb0066f60 msgr2=0x7f9bb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.826+0000 7f9bca5e3640 1 --2- 172.21.15.60:0/3231308001 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9bb0066f60 0x7f9bb0069420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9bb80af000 tx=0x7f9bb80a7000 comp rx=0 tx=0).stop 2024-04-25T10:08:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.826+0000 7f9bca5e3640 1 -- 172.21.15.60:0/3231308001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bc40ce7a0 msgr2=0x7f9bc414a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.826+0000 7f9bca5e3640 1 --2- 172.21.15.60:0/3231308001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bc40ce7a0 0x7f9bc414a0c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400c370 tx=0x7f9bb400c830 comp rx=0 tx=0).stop 2024-04-25T10:08:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.826+0000 7f9bca5e3640 1 -- 172.21.15.60:0/3231308001 shutdown_connections 2024-04-25T10:08:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.826+0000 7f9bca5e3640 1 --2- 172.21.15.60:0/3231308001 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9bb0066f60 0x7f9bb0069420 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.826+0000 7f9bca5e3640 1 --2- 172.21.15.60:0/3231308001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bc406d830 0x7f9bc41529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.826+0000 7f9bca5e3640 1 --2- 172.21.15.60:0/3231308001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bc415bc90 0x7f9bc406d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.826+0000 7f9bca5e3640 1 --2- 172.21.15.60:0/3231308001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bc40ce7a0 0x7f9bc414a0c0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.826+0000 7f9bca5e3640 1 -- 172.21.15.60:0/3231308001 >> 172.21.15.60:0/3231308001 conn(0x7f9bc40b12d0 msgr2=0x7f9bc415c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:04.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.826+0000 7f9bca5e3640 1 -- 172.21.15.60:0/3231308001 shutdown_connections 2024-04-25T10:08:04.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.826+0000 7f9bca5e3640 1 -- 172.21.15.60:0/3231308001 wait complete. 2024-04-25T10:08:04.856 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:08:04.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e8444b640 1 Processor -- start 2024-04-25T10:08:04.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e8444b640 1 -- start start 2024-04-25T10:08:04.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e8444b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7c0d0540 0x7f5e7c0b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e8444b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5e7c0b3c10 0x7f5e7c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e8444b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7c0d09d0 con 0x7f5e7c15d930 2024-04-25T10:08:04.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e8444b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7c0d0cb0 con 0x7f5e7c0b3c10 2024-04-25T10:08:04.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e8444b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0d0f90 con 0x7f5e7c0d0540 2024-04-25T10:08:04.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e821e7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5e7c15d930 0x7f5e7c0cd440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49782/0 (socket says 172.21.15.60:49782) 2024-04-25T10:08:04.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e821e7640 1 -- 172.21.15.60:0/2199493130 learned_addr learned my addr 172.21.15.60:0/2199493130 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:04.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e829e8640 1 --2- 172.21.15.60:0/2199493130 >> v2:172.21.15.100:3300/0 conn(0x7f5e7c0b3c10 0x7f5e7c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e819e6640 1 --2- 172.21.15.60:0/2199493130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7c0d0540 0x7f5e7c0b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e811e5640 1 -- 172.21.15.60:0/2199493130 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3530664080 0 0) 0x7f5e7c0d09d0 con 0x7f5e7c15d930 2024-04-25T10:08:04.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e811e5640 1 -- 172.21.15.60:0/2199493130 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e68003530 con 0x7f5e7c15d930 2024-04-25T10:08:04.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e811e5640 1 -- 172.21.15.60:0/2199493130 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 913276677 0 0) 0x7f5e68003530 con 0x7f5e7c15d930 2024-04-25T10:08:04.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e811e5640 1 -- 172.21.15.60:0/2199493130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7c0d0540 msgr2=0x7f5e7c0b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e811e5640 1 --2- 172.21.15.60:0/2199493130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7c0d0540 0x7f5e7c0b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e811e5640 1 -- 172.21.15.60:0/2199493130 >> v2:172.21.15.100:3300/0 conn(0x7f5e7c0b3c10 msgr2=0x7f5e7c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e811e5640 1 --2- 172.21.15.60:0/2199493130 >> v2:172.21.15.100:3300/0 conn(0x7f5e7c0b3c10 0x7f5e7c06ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e811e5640 1 -- 172.21.15.60:0/2199493130 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7c0d1270 con 0x7f5e7c15d930 2024-04-25T10:08:04.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e811e5640 1 -- 172.21.15.60:0/2199493130 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f5e78001680 con 0x7f5e7c15d930 2024-04-25T10:08:04.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e811e5640 1 -- 172.21.15.60:0/2199493130 >> v1:172.21.15.60:6789/0 conn(0x7f5e7c15d930 legacy=0x7f5e7c0cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e811e5640 1 --2- 172.21.15.60:0/2199493130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e680039c0 0x7f5e68003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e811e5640 1 --2- 172.21.15.60:0/2199493130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e68004810 0x7f5e68004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e811e5640 1 --2- 172.21.15.60:0/2199493130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e68005190 0x7f5e6800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e811e5640 1 -- 172.21.15.60:0/2199493130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0d0cb0 con 0x7f5e68005190 2024-04-25T10:08:04.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e811e5640 1 -- 172.21.15.60:0/2199493130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0d0f90 con 0x7f5e680039c0 2024-04-25T10:08:04.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e811e5640 1 -- 172.21.15.60:0/2199493130 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e68004610 con 0x7f5e68004810 2024-04-25T10:08:04.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e829e8640 1 --2- 172.21.15.60:0/2199493130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e68005190 0x7f5e6800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e819e6640 1 --2- 172.21.15.60:0/2199493130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e68004810 0x7f5e68004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e821e7640 1 --2- 172.21.15.60:0/2199493130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e680039c0 0x7f5e68003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e829e8640 1 -- 172.21.15.60:0/2199493130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e68004810 msgr2=0x7f5e68004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e829e8640 1 --2- 172.21.15.60:0/2199493130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e68004810 0x7f5e68004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e829e8640 1 -- 172.21.15.60:0/2199493130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e680039c0 msgr2=0x7f5e68003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e829e8640 1 --2- 172.21.15.60:0/2199493130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e680039c0 0x7f5e68003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.982+0000 7f5e829e8640 1 -- 172.21.15.60:0/2199493130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e68003530 con 0x7f5e68005190 2024-04-25T10:08:04.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.986+0000 7f5e829e8640 1 --2- 172.21.15.60:0/2199493130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e68005190 0x7f5e6800b9d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009fc0 tx=0x7f5e7401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea881ba4f1812f67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.986+0000 7f5e811e5640 1 -- 172.21.15.60:0/2199493130 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e740b2070 con 0x7f5e68005190 2024-04-25T10:08:04.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.986+0000 7f5e811e5640 1 -- 172.21.15.60:0/2199493130 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e740a5cd0 con 0x7f5e68005190 2024-04-25T10:08:04.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.986+0000 7f5e811e5640 1 -- 172.21.15.60:0/2199493130 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74009c10 con 0x7f5e68005190 2024-04-25T10:08:04.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.986+0000 7f5e8444b640 1 -- 172.21.15.60:0/2199493130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e68005190 msgr2=0x7f5e6800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.986+0000 7f5e8444b640 1 --2- 172.21.15.60:0/2199493130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e68005190 0x7f5e6800b9d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009fc0 tx=0x7f5e7401ccc0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.986+0000 7f5e829e8640 1 -- 172.21.15.60:0/2199493130 reap_dead start 2024-04-25T10:08:04.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.986+0000 7f5e8444b640 1 -- 172.21.15.60:0/2199493130 shutdown_connections 2024-04-25T10:08:04.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.986+0000 7f5e8444b640 1 -- 172.21.15.60:0/2199493130 >> 172.21.15.60:0/2199493130 conn(0x7f5e7c0b2360 msgr2=0x7f5e7c0b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:04.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.986+0000 7f5e8444b640 1 -- 172.21.15.60:0/2199493130 shutdown_connections 2024-04-25T10:08:04.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.986+0000 7f5e8444b640 1 -- 172.21.15.60:0/2199493130 wait complete. 2024-04-25T10:08:04.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.986+0000 7f5e8444b640 1 Processor -- start 2024-04-25T10:08:04.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.986+0000 7f5e8444b640 1 -- start start 2024-04-25T10:08:04.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e8444b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7c0cfe30 0x7f5e7c0c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e8444b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e7c0d0210 0x7f5e7c141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e8444b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7c1421c0 0x7f5e7c142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e8444b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0b3850 con 0x7f5e7c0cfe30 2024-04-25T10:08:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e8444b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0b36d0 con 0x7f5e7c0d0210 2024-04-25T10:08:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e8444b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0b39d0 con 0x7f5e7c1421c0 2024-04-25T10:08:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e821e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7c0cfe30 0x7f5e7c0c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e821e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7c0cfe30 0x7f5e7c0c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33988/0 (socket says 172.21.15.60:33988) 2024-04-25T10:08:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e821e7640 1 -- 172.21.15.60:0/1098675628 learned_addr learned my addr 172.21.15.60:0/1098675628 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e829e8640 1 --2- 172.21.15.60:0/1098675628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7c1421c0 0x7f5e7c142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e819e6640 1 --2- 172.21.15.60:0/1098675628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e7c0d0210 0x7f5e7c141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e821e7640 1 -- 172.21.15.60:0/1098675628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7c1421c0 msgr2=0x7f5e7c142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e821e7640 1 --2- 172.21.15.60:0/1098675628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7c1421c0 0x7f5e7c142670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e821e7640 1 -- 172.21.15.60:0/1098675628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e7c0d0210 msgr2=0x7f5e7c141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e821e7640 1 --2- 172.21.15.60:0/1098675628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e7c0d0210 0x7f5e7c141c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e821e7640 1 -- 172.21.15.60:0/1098675628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7c0cc230 con 0x7f5e7c0cfe30 2024-04-25T10:08:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e829e8640 1 --2- 172.21.15.60:0/1098675628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7c1421c0 0x7f5e7c142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e821e7640 1 --2- 172.21.15.60:0/1098675628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7c0cfe30 0x7f5e7c0c99e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800eaf0 tx=0x7f5e78001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e72ffd640 1 -- 172.21.15.60:0/1098675628 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800c8d0 con 0x7f5e7c0cfe30 2024-04-25T10:08:05.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e72ffd640 1 -- 172.21.15.60:0/1098675628 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e78018070 con 0x7f5e7c0cfe30 2024-04-25T10:08:05.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e8444b640 1 -- 172.21.15.60:0/1098675628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7c0cc510 con 0x7f5e7c0cfe30 2024-04-25T10:08:05.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e72ffd640 1 -- 172.21.15.60:0/1098675628 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800ccb0 con 0x7f5e7c0cfe30 2024-04-25T10:08:05.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e8444b640 1 -- 172.21.15.60:0/1098675628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7c1f6cf0 con 0x7f5e7c0cfe30 2024-04-25T10:08:05.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e8444b640 1 -- 172.21.15.60:0/1098675628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7c074770 con 0x7f5e7c0cfe30 2024-04-25T10:08:05.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e72ffd640 1 -- 172.21.15.60:0/1098675628 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e78013420 con 0x7f5e7c0cfe30 2024-04-25T10:08:05.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e72ffd640 1 --2- 172.21.15.60:0/1098675628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e60066f90 0x7f5e60069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e72ffd640 1 -- 172.21.15.60:0/1098675628 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5e780849c0 con 0x7f5e7c0cfe30 2024-04-25T10:08:05.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.990+0000 7f5e819e6640 1 --2- 172.21.15.60:0/1098675628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e60066f90 0x7f5e60069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.994+0000 7f5e819e6640 1 --2- 172.21.15.60:0/1098675628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e60066f90 0x7f5e60069450 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0b1b70 tx=0x7f5e6c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:04.998+0000 7f5e72ffd640 1 -- 172.21.15.60:0/1098675628 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5e78086e20 con 0x7f5e7c0cfe30 2024-04-25T10:08:05.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.154+0000 7f5e8444b640 1 -- 172.21.15.60:0/1098675628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5e7c0732a0 con 0x7f5e7c0cfe30 2024-04-25T10:08:05.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.158+0000 7f5e72ffd640 1 -- 172.21.15.60:0/1098675628 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5e78055cd0 con 0x7f5e7c0cfe30 2024-04-25T10:08:05.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.162+0000 7f5e8444b640 1 -- 172.21.15.60:0/1098675628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e60066f90 msgr2=0x7f5e60069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.162+0000 7f5e8444b640 1 --2- 172.21.15.60:0/1098675628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e60066f90 0x7f5e60069450 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0b1b70 tx=0x7f5e6c008040 comp rx=0 tx=0).stop 2024-04-25T10:08:05.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.162+0000 7f5e8444b640 1 -- 172.21.15.60:0/1098675628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7c0cfe30 msgr2=0x7f5e7c0c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.162+0000 7f5e8444b640 1 --2- 172.21.15.60:0/1098675628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7c0cfe30 0x7f5e7c0c99e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800eaf0 tx=0x7f5e78001720 comp rx=0 tx=0).stop 2024-04-25T10:08:05.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.162+0000 7f5e8444b640 1 -- 172.21.15.60:0/1098675628 shutdown_connections 2024-04-25T10:08:05.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.162+0000 7f5e8444b640 1 --2- 172.21.15.60:0/1098675628 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e60066f90 0x7f5e60069450 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.162+0000 7f5e8444b640 1 --2- 172.21.15.60:0/1098675628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7c1421c0 0x7f5e7c142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.162+0000 7f5e8444b640 1 --2- 172.21.15.60:0/1098675628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e7c0d0210 0x7f5e7c141c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.162+0000 7f5e8444b640 1 --2- 172.21.15.60:0/1098675628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7c0cfe30 0x7f5e7c0c99e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.162+0000 7f5e8444b640 1 -- 172.21.15.60:0/1098675628 >> 172.21.15.60:0/1098675628 conn(0x7f5e7c0b2360 msgr2=0x7f5e7c15a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.162+0000 7f5e8444b640 1 -- 172.21.15.60:0/1098675628 shutdown_connections 2024-04-25T10:08:05.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.162+0000 7f5e8444b640 1 -- 172.21.15.60:0/1098675628 wait complete. 2024-04-25T10:08:05.186 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:08:05.186 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:05.238 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:05.294 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:08:05.294 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:08:05.294 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:05.299 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:05.299 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:08:05.299 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:05.305 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:05.305 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:08:05.305 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:05.345 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:05.345 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:08:05.345 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:05.353 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:05.353 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:08:05.353 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:05.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.506+0000 7fd990199640 1 Processor -- start 2024-04-25T10:08:05.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.510+0000 7fd990199640 1 -- start start 2024-04-25T10:08:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.510+0000 7fd990199640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd988071b20 0x7fd988071f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.510+0000 7fd990199640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd98806e480 0x7fd98806e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.510+0000 7fd990199640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd988072440 con 0x7fd98806d2f0 2024-04-25T10:08:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.510+0000 7fd990199640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd98815dc10 con 0x7fd988071b20 2024-04-25T10:08:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.510+0000 7fd990199640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd98815def0 con 0x7fd98806e480 2024-04-25T10:08:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.510+0000 7fd98df35640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd988071b20 0x7fd988071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.510+0000 7fd98df35640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd988071b20 0x7fd988071f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34004/0 (socket says 172.21.15.60:34004) 2024-04-25T10:08:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.510+0000 7fd98df35640 1 -- 172.21.15.60:0/1629586158 learned_addr learned my addr 172.21.15.60:0/1629586158 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.510+0000 7fd98d734640 1 --2- 172.21.15.60:0/1629586158 >> v2:172.21.15.100:3300/0 conn(0x7fd98806e480 0x7fd98806e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.510+0000 7fd98df35640 1 -- 172.21.15.60:0/1629586158 >> v2:172.21.15.100:3300/0 conn(0x7fd98806e480 msgr2=0x7fd98806e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.510+0000 7fd98df35640 1 --2- 172.21.15.60:0/1629586158 >> v2:172.21.15.100:3300/0 conn(0x7fd98806e480 0x7fd98806e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.510+0000 7fd98df35640 1 -- 172.21.15.60:0/1629586158 >> v1:172.21.15.100:6789/0 conn(0x7fd98806d2f0 legacy=0x7fd98806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.510+0000 7fd98df35640 1 -- 172.21.15.60:0/1629586158 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98815e1d0 con 0x7fd988071b20 2024-04-25T10:08:05.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.510+0000 7fd98d734640 1 --2- 172.21.15.60:0/1629586158 >> v2:172.21.15.100:3300/0 conn(0x7fd98806e480 0x7fd98806e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.514+0000 7fd98df35640 1 --2- 172.21.15.60:0/1629586158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd988071b20 0x7fd988071f00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd988071220 tx=0x7fd98401d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cb8417f96ecfd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.514+0000 7fd98cf33640 1 -- 172.21.15.60:0/1629586158 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9840b3070 con 0x7fd988071b20 2024-04-25T10:08:05.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.514+0000 7fd98cf33640 1 -- 172.21.15.60:0/1629586158 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9840a67c0 con 0x7fd988071b20 2024-04-25T10:08:05.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.514+0000 7fd98cf33640 1 -- 172.21.15.60:0/1629586158 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98400b040 con 0x7fd988071b20 2024-04-25T10:08:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.514+0000 7fd990199640 1 -- 172.21.15.60:0/1629586158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd988071b20 msgr2=0x7fd988071f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.514+0000 7fd990199640 1 --2- 172.21.15.60:0/1629586158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd988071b20 0x7fd988071f00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd988071220 tx=0x7fd98401d820 comp rx=0 tx=0).stop 2024-04-25T10:08:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.514+0000 7fd990199640 1 -- 172.21.15.60:0/1629586158 shutdown_connections 2024-04-25T10:08:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.514+0000 7fd990199640 1 --2- 172.21.15.60:0/1629586158 >> v2:172.21.15.100:3300/0 conn(0x7fd98806e480 0x7fd98806e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.514+0000 7fd990199640 1 --2- 172.21.15.60:0/1629586158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd988071b20 0x7fd988071f00 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.514+0000 7fd990199640 1 -- 172.21.15.60:0/1629586158 >> 172.21.15.60:0/1629586158 conn(0x7fd9880b12d0 msgr2=0x7fd9880b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.514+0000 7fd990199640 1 -- 172.21.15.60:0/1629586158 shutdown_connections 2024-04-25T10:08:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.514+0000 7fd990199640 1 -- 172.21.15.60:0/1629586158 wait complete. 2024-04-25T10:08:05.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd990199640 1 Processor -- start 2024-04-25T10:08:05.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd990199640 1 -- start start 2024-04-25T10:08:05.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd990199640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd98806d2f0 0x7fd988152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd990199640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd98806e480 0x7fd98813fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd990199640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd988071b20 0x7fd9881400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd990199640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880cf070 con 0x7fd988071b20 2024-04-25T10:08:05.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd990199640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880ceef0 con 0x7fd98806d2f0 2024-04-25T10:08:05.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd990199640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd988166e60 con 0x7fd98806e480 2024-04-25T10:08:05.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd98e736640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd988071b20 0x7fd9881400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd98e736640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd988071b20 0x7fd9881400b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34012/0 (socket says 172.21.15.60:34012) 2024-04-25T10:08:05.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd98e736640 1 -- 172.21.15.60:0/915954155 learned_addr learned my addr 172.21.15.60:0/915954155 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:05.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd98df35640 1 --2- 172.21.15.60:0/915954155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd98806d2f0 0x7fd988152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd98d734640 1 --2- 172.21.15.60:0/915954155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd98806e480 0x7fd98813fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd98e736640 1 -- 172.21.15.60:0/915954155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd98806e480 msgr2=0x7fd98813fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd98e736640 1 --2- 172.21.15.60:0/915954155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd98806e480 0x7fd98813fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd98e736640 1 -- 172.21.15.60:0/915954155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd98806d2f0 msgr2=0x7fd988152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd98e736640 1 --2- 172.21.15.60:0/915954155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd98806d2f0 0x7fd988152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd98e736640 1 -- 172.21.15.60:0/915954155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd988158be0 con 0x7fd988071b20 2024-04-25T10:08:05.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd98df35640 1 --2- 172.21.15.60:0/915954155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd98806d2f0 0x7fd988152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:05.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd98e736640 1 --2- 172.21.15.60:0/915954155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd988071b20 0x7fd9881400b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd98000c7b0 tx=0x7fd98000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd97e7fc640 1 -- 172.21.15.60:0/915954155 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd980004b70 con 0x7fd988071b20 2024-04-25T10:08:05.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd97e7fc640 1 -- 172.21.15.60:0/915954155 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd980004d10 con 0x7fd988071b20 2024-04-25T10:08:05.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd990199640 1 -- 172.21.15.60:0/915954155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd988158ec0 con 0x7fd988071b20 2024-04-25T10:08:05.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd97e7fc640 1 -- 172.21.15.60:0/915954155 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98000aa90 con 0x7fd988071b20 2024-04-25T10:08:05.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd990199640 1 -- 172.21.15.60:0/915954155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd988159480 con 0x7fd988071b20 2024-04-25T10:08:05.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.518+0000 7fd990199640 1 -- 172.21.15.60:0/915954155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd988074770 con 0x7fd988071b20 2024-04-25T10:08:05.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.522+0000 7fd97e7fc640 1 -- 172.21.15.60:0/915954155 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd98000ac30 con 0x7fd988071b20 2024-04-25T10:08:05.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.522+0000 7fd97e7fc640 1 --2- 172.21.15.60:0/915954155 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd96c066f90 0x7fd96c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.522+0000 7fd97e7fc640 1 -- 172.21.15.60:0/915954155 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd9800855d0 con 0x7fd988071b20 2024-04-25T10:08:05.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.522+0000 7fd98df35640 1 --2- 172.21.15.60:0/915954155 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd96c066f90 0x7fd96c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.522+0000 7fd98df35640 1 --2- 172.21.15.60:0/915954155 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd96c066f90 0x7fd96c069450 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd9840b0000 tx=0x7fd9840a8070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.526+0000 7fd97e7fc640 1 -- 172.21.15.60:0/915954155 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd980020070 con 0x7fd988071b20 2024-04-25T10:08:05.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.718+0000 7fd990199640 1 -- 172.21.15.60:0/915954155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9880b33a0 con 0x7fd988071b20 2024-04-25T10:08:05.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.722+0000 7fd97e7fc640 1 -- 172.21.15.60:0/915954155 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+3693 (secure 0 0 0) 0x7fd980011380 con 0x7fd988071b20 2024-04-25T10:08:05.728 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:05.729 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1138162580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1138162580},{"type":"v1","addr":"172.21.15.100:6837","nonce":1138162580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5299,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2985559168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2985559168},{"type":"v1","addr":"172.21.15.100:6835","nonce":2985559168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/50795370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":50795370},{"type":"v1","addr":"172.21.15.60:6834","nonce":50795370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":6216,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/25759586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":25759586},{"type":"v1","addr":"172.21.15.60:6835","nonce":25759586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[]} 2024-04-25T10:08:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-04-25T10:08:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.722+0000 7fd990199640 1 -- 172.21.15.60:0/915954155 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd96c066f90 msgr2=0x7fd96c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.722+0000 7fd990199640 1 --2- 172.21.15.60:0/915954155 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd96c066f90 0x7fd96c069450 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd9840b0000 tx=0x7fd9840a8070 comp rx=0 tx=0).stop 2024-04-25T10:08:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.722+0000 7fd990199640 1 -- 172.21.15.60:0/915954155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd988071b20 msgr2=0x7fd9881400b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.722+0000 7fd990199640 1 --2- 172.21.15.60:0/915954155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd988071b20 0x7fd9881400b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd98000c7b0 tx=0x7fd98000cc70 comp rx=0 tx=0).stop 2024-04-25T10:08:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.722+0000 7fd990199640 1 -- 172.21.15.60:0/915954155 shutdown_connections 2024-04-25T10:08:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.722+0000 7fd990199640 1 --2- 172.21.15.60:0/915954155 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd96c066f90 0x7fd96c069450 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.722+0000 7fd990199640 1 --2- 172.21.15.60:0/915954155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd988071b20 0x7fd9881400b0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.722+0000 7fd990199640 1 --2- 172.21.15.60:0/915954155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd98806e480 0x7fd98813fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.722+0000 7fd990199640 1 --2- 172.21.15.60:0/915954155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd98806d2f0 0x7fd988152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.722+0000 7fd990199640 1 -- 172.21.15.60:0/915954155 >> 172.21.15.60:0/915954155 conn(0x7fd9880b12d0 msgr2=0x7fd988156d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.722+0000 7fd990199640 1 -- 172.21.15.60:0/915954155 shutdown_connections 2024-04-25T10:08:05.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.722+0000 7fd990199640 1 -- 172.21.15.60:0/915954155 wait complete. 2024-04-25T10:08:05.747 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:08:05.747 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:08:05.748 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:08:05.748 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:05.748 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:08:05.748 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:08:05.748 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:08:05.748 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:05.748 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:08:05.749 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:08:05.749 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:08:05.749 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:05.749 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:08:05.749 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:08:05.749 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:08:05.749 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:05.761 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:08:05.762 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:08:05.762 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:08:05.762 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:08:05.763 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:08:05.764 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:08:05.764 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:08:05.765 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:08:05.806 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:08:05.806 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:05.807 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:08:05.807 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:05.809 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:08:05.810 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:05.811 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:08:05.811 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:05.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:08:05.865 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:08:05.866 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T10:08:05.884 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T10:08:05.884 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T10:08:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.930+0000 7f5cced6c640 1 Processor -- start 2024-04-25T10:08:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.930+0000 7f5cced6c640 1 -- start start 2024-04-25T10:08:05.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cced6c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5cc80ce4f0 0x7f5cc80d2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cced6c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc80d3110 0x7f5cc80ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cced6c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc806c930 con 0x7f5cc815dea0 2024-04-25T10:08:05.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cced6c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc815e280 con 0x7f5cc80ce4f0 2024-04-25T10:08:05.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cced6c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc80d20f0 con 0x7f5cc80d3110 2024-04-25T10:08:05.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cc7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5cc80ce4f0 0x7f5cc80d2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cccb08640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5cc815dea0 0x7f5cc806c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49796/0 (socket says 172.21.15.60:49796) 2024-04-25T10:08:05.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cccb08640 1 -- 172.21.15.60:0/505711067 learned_addr learned my addr 172.21.15.60:0/505711067 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:05.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5ccd309640 1 --2- 172.21.15.60:0/505711067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc80d3110 0x7f5cc80ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cc7fff640 1 -- 172.21.15.60:0/505711067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc80d3110 msgr2=0x7f5cc80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cc7fff640 1 --2- 172.21.15.60:0/505711067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc80d3110 0x7f5cc80ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cc7fff640 1 -- 172.21.15.60:0/505711067 >> v1:172.21.15.60:6789/0 conn(0x7f5cc815dea0 legacy=0x7f5cc806c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cc7fff640 1 -- 172.21.15.60:0/505711067 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc80d23d0 con 0x7f5cc80ce4f0 2024-04-25T10:08:05.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cc77fe640 1 -- 172.21.15.60:0/505711067 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2846673570 0 0) 0x7f5cc806c930 con 0x7f5cc815dea0 2024-04-25T10:08:05.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5ccd309640 1 --2- 172.21.15.60:0/505711067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc80d3110 0x7f5cc80ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:05.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cc7fff640 1 --2- 172.21.15.60:0/505711067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc80ce4f0 0x7f5cc80d2bd0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40097c0 tx=0x7f5cb401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5ef1aafd1d0f4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cc77fe640 1 -- 172.21.15.60:0/505711067 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40b0070 con 0x7f5cc80ce4f0 2024-04-25T10:08:05.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cc77fe640 1 -- 172.21.15.60:0/505711067 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb40ab600 con 0x7f5cc80ce4f0 2024-04-25T10:08:05.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.934+0000 7f5cc77fe640 1 -- 172.21.15.60:0/505711067 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40ab9d0 con 0x7f5cc80ce4f0 2024-04-25T10:08:05.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 -- 172.21.15.60:0/505711067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc80ce4f0 msgr2=0x7f5cc80d2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 --2- 172.21.15.60:0/505711067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc80ce4f0 0x7f5cc80d2bd0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40097c0 tx=0x7f5cb401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 -- 172.21.15.60:0/505711067 shutdown_connections 2024-04-25T10:08:05.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 --2- 172.21.15.60:0/505711067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc80d3110 0x7f5cc80ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 --2- 172.21.15.60:0/505711067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc80ce4f0 0x7f5cc80d2bd0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 -- 172.21.15.60:0/505711067 >> 172.21.15.60:0/505711067 conn(0x7f5cc80d0480 msgr2=0x7f5cc80d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 -- 172.21.15.60:0/505711067 shutdown_connections 2024-04-25T10:08:05.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 -- 172.21.15.60:0/505711067 wait complete. 2024-04-25T10:08:05.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 Processor -- start 2024-04-25T10:08:05.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 -- start start 2024-04-25T10:08:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc80ce4f0 0x7f5cc80c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc80d3110 0x7f5cc80caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc815dea0 0x7f5cc80b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc816aa90 con 0x7f5cc80d3110 2024-04-25T10:08:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc80b7830 con 0x7f5cc80ce4f0 2024-04-25T10:08:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc816ac10 con 0x7f5cc815dea0 2024-04-25T10:08:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cc7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc80d3110 0x7f5cc80caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cc7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc80d3110 0x7f5cc80caa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34072/0 (socket says 172.21.15.60:34072) 2024-04-25T10:08:05.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cc7fff640 1 -- 172.21.15.60:0/2972223036 learned_addr learned my addr 172.21.15.60:0/2972223036 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:05.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cccb08640 1 --2- 172.21.15.60:0/2972223036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc80ce4f0 0x7f5cc80c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5ccd309640 1 --2- 172.21.15.60:0/2972223036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc815dea0 0x7f5cc80b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cc7fff640 1 -- 172.21.15.60:0/2972223036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc815dea0 msgr2=0x7f5cc80b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cc7fff640 1 --2- 172.21.15.60:0/2972223036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc815dea0 0x7f5cc80b7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cc7fff640 1 -- 172.21.15.60:0/2972223036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc80ce4f0 msgr2=0x7f5cc80c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cc7fff640 1 --2- 172.21.15.60:0/2972223036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc80ce4f0 0x7f5cc80c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cc7fff640 1 -- 172.21.15.60:0/2972223036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc8150880 con 0x7f5cc80d3110 2024-04-25T10:08:05.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cccb08640 1 --2- 172.21.15.60:0/2972223036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc80ce4f0 0x7f5cc80c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:05.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cc7fff640 1 --2- 172.21.15.60:0/2972223036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc80d3110 0x7f5cc80caa40 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5cb401a0d0 tx=0x7f5cb401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cc57fa640 1 -- 172.21.15.60:0/2972223036 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40b0050 con 0x7f5cc80d3110 2024-04-25T10:08:05.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cc57fa640 1 -- 172.21.15.60:0/2972223036 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb40bc070 con 0x7f5cc80d3110 2024-04-25T10:08:05.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cc57fa640 1 -- 172.21.15.60:0/2972223036 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40aac00 con 0x7f5cc80d3110 2024-04-25T10:08:05.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 -- 172.21.15.60:0/2972223036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc814eff0 con 0x7f5cc80d3110 2024-04-25T10:08:05.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.938+0000 7f5cced6c640 1 -- 172.21.15.60:0/2972223036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc814f5b0 con 0x7f5cc80d3110 2024-04-25T10:08:05.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.942+0000 7f5cced6c640 1 -- 172.21.15.60:0/2972223036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc8150350 con 0x7f5cc80d3110 2024-04-25T10:08:05.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.942+0000 7f5cc57fa640 1 -- 172.21.15.60:0/2972223036 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5cb40c1460 con 0x7f5cc80d3110 2024-04-25T10:08:05.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.942+0000 7f5cc57fa640 1 --2- 172.21.15.60:0/2972223036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5cac066f90 0x7f5cac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.942+0000 7f5cc57fa640 1 -- 172.21.15.60:0/2972223036 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5cb411f770 con 0x7f5cc80d3110 2024-04-25T10:08:05.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.942+0000 7f5cccb08640 1 --2- 172.21.15.60:0/2972223036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5cac066f90 0x7f5cac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.942+0000 7f5cccb08640 1 --2- 172.21.15.60:0/2972223036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5cac066f90 0x7f5cac069450 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc000f70 tx=0x7f5cbc007cb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:05.946+0000 7f5cc57fa640 1 -- 172.21.15.60:0/2972223036 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5cc8150350 con 0x7f5cc80d3110 2024-04-25T10:08:06.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.118+0000 7f5cced6c640 1 -- 172.21.15.60:0/2972223036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5cc815aad0 con 0x7f5cc80d3110 2024-04-25T10:08:06.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.842+0000 7f5cc57fa640 1 -- 172.21.15.60:0/2972223036 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v52) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5cc8150880 con 0x7f5cc80d3110 2024-04-25T10:08:06.850 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-04-25T10:08:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.846+0000 7f5cced6c640 1 -- 172.21.15.60:0/2972223036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5cac066f90 msgr2=0x7f5cac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.846+0000 7f5cced6c640 1 --2- 172.21.15.60:0/2972223036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5cac066f90 0x7f5cac069450 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc000f70 tx=0x7f5cbc007cb0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.846+0000 7f5cced6c640 1 -- 172.21.15.60:0/2972223036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc80d3110 msgr2=0x7f5cc80caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.846+0000 7f5cced6c640 1 --2- 172.21.15.60:0/2972223036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc80d3110 0x7f5cc80caa40 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5cb401a0d0 tx=0x7f5cb401ace0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.846+0000 7f5cced6c640 1 -- 172.21.15.60:0/2972223036 shutdown_connections 2024-04-25T10:08:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.846+0000 7f5cced6c640 1 --2- 172.21.15.60:0/2972223036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5cac066f90 0x7f5cac069450 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.846+0000 7f5cced6c640 1 --2- 172.21.15.60:0/2972223036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc815dea0 0x7f5cc80b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.846+0000 7f5cced6c640 1 --2- 172.21.15.60:0/2972223036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc80d3110 0x7f5cc80caa40 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.846+0000 7f5cced6c640 1 --2- 172.21.15.60:0/2972223036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc80ce4f0 0x7f5cc80c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.846+0000 7f5cced6c640 1 -- 172.21.15.60:0/2972223036 >> 172.21.15.60:0/2972223036 conn(0x7f5cc80d0480 msgr2=0x7f5cc806c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.846+0000 7f5cced6c640 1 -- 172.21.15.60:0/2972223036 shutdown_connections 2024-04-25T10:08:06.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.846+0000 7f5cced6c640 1 -- 172.21.15.60:0/2972223036 wait complete. 2024-04-25T10:08:06.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:08:07.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.994+0000 7f89337db640 1 Processor -- start 2024-04-25T10:08:07.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.994+0000 7f89337db640 1 -- start start 2024-04-25T10:08:07.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.998+0000 7f89337db640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f892c0b35f0 0x7f892c0b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.998+0000 7f89337db640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892c15ac70 con 0x7f892c15a890 2024-04-25T10:08:07.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.998+0000 7f89337db640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892c0ae590 con 0x7f892c0b45d0 2024-04-25T10:08:07.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.998+0000 7f89337db640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f892c0b4a20 con 0x7f892c0b35f0 2024-04-25T10:08:07.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.998+0000 7f8930d76640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f892c0b35f0 0x7f892c0b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.998+0000 7f8930d76640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f892c0b35f0 0x7f892c0b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34086/0 (socket says 172.21.15.60:34086) 2024-04-25T10:08:07.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.998+0000 7f8930d76640 1 -- 172.21.15.60:0/2848374244 learned_addr learned my addr 172.21.15.60:0/2848374244 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:07.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.998+0000 7f8930d76640 1 -- 172.21.15.60:0/2848374244 >> v1:172.21.15.100:6789/0 conn(0x7f892c0b45d0 legacy=0x7f892c06e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:07.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.998+0000 7f8930d76640 1 -- 172.21.15.60:0/2848374244 >> v1:172.21.15.60:6789/0 conn(0x7f892c15a890 legacy=0x7f892c0adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.998+0000 7f8930d76640 1 -- 172.21.15.60:0/2848374244 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c0ae870 con 0x7f892c0b35f0 2024-04-25T10:08:07.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.998+0000 7f8930d76640 1 --2- 172.21.15.60:0/2848374244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f892c0b35f0 0x7f892c0b4090 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f891c009870 tx=0x7f891c01e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e97eed79729562 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:07.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.998+0000 7f8923fff640 1 -- 172.21.15.60:0/2848374244 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c0a8640 con 0x7f892c0b35f0 2024-04-25T10:08:07.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.998+0000 7f8923fff640 1 -- 172.21.15.60:0/2848374244 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891c0a87e0 con 0x7f892c0b35f0 2024-04-25T10:08:07.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:06.998+0000 7f8923fff640 1 -- 172.21.15.60:0/2848374244 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c0a8bb0 con 0x7f892c0b35f0 2024-04-25T10:08:07.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 -- 172.21.15.60:0/2848374244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f892c0b35f0 msgr2=0x7f892c0b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 --2- 172.21.15.60:0/2848374244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f892c0b35f0 0x7f892c0b4090 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f891c009870 tx=0x7f891c01e670 comp rx=0 tx=0).stop 2024-04-25T10:08:07.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 -- 172.21.15.60:0/2848374244 shutdown_connections 2024-04-25T10:08:07.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 --2- 172.21.15.60:0/2848374244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f892c0b35f0 0x7f892c0b4090 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 -- 172.21.15.60:0/2848374244 >> 172.21.15.60:0/2848374244 conn(0x7f892c0b5aa0 msgr2=0x7f892c0b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:07.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 -- 172.21.15.60:0/2848374244 shutdown_connections 2024-04-25T10:08:07.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 -- 172.21.15.60:0/2848374244 wait complete. 2024-04-25T10:08:07.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 Processor -- start 2024-04-25T10:08:07.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 -- start start 2024-04-25T10:08:07.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f892c0b35f0 0x7f892c14b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f892c0b45d0 0x7f892c15d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892c15a890 0x7f892c15d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c06f910 con 0x7f892c0b45d0 2024-04-25T10:08:07.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c0ae2a0 con 0x7f892c15a890 2024-04-25T10:08:07.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892c06fa90 con 0x7f892c0b35f0 2024-04-25T10:08:07.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f8930d76640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f892c0b45d0 0x7f892c15d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f8930d76640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f892c0b45d0 0x7f892c15d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34090/0 (socket says 172.21.15.60:34090) 2024-04-25T10:08:07.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f8930d76640 1 -- 172.21.15.60:0/2223879006 learned_addr learned my addr 172.21.15.60:0/2223879006 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:07.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f8931577640 1 --2- 172.21.15.60:0/2223879006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f892c0b35f0 0x7f892c14b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f8931d78640 1 --2- 172.21.15.60:0/2223879006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892c15a890 0x7f892c15d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f8930d76640 1 -- 172.21.15.60:0/2223879006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f892c0b35f0 msgr2=0x7f892c14b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f8930d76640 1 --2- 172.21.15.60:0/2223879006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f892c0b35f0 0x7f892c14b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f8930d76640 1 -- 172.21.15.60:0/2223879006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892c15a890 msgr2=0x7f892c15d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f8930d76640 1 --2- 172.21.15.60:0/2223879006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892c15a890 0x7f892c15d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f8930d76640 1 -- 172.21.15.60:0/2223879006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c0b0a40 con 0x7f892c0b45d0 2024-04-25T10:08:07.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f8931d78640 1 --2- 172.21.15.60:0/2223879006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892c15a890 0x7f892c15d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:07.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f8930d76640 1 --2- 172.21.15.60:0/2223879006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f892c0b45d0 0x7f892c15d390 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f891c01c2a0 tx=0x7f891c01b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:07.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f8921ffb640 1 -- 172.21.15.60:0/2223879006 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c0bc640 con 0x7f892c0b45d0 2024-04-25T10:08:07.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f8921ffb640 1 -- 172.21.15.60:0/2223879006 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891c0bc7e0 con 0x7f892c0b45d0 2024-04-25T10:08:07.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 -- 172.21.15.60:0/2223879006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892c0b0d20 con 0x7f892c0b45d0 2024-04-25T10:08:07.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f8921ffb640 1 -- 172.21.15.60:0/2223879006 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c0bcbd0 con 0x7f892c0b45d0 2024-04-25T10:08:07.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.002+0000 7f89337db640 1 -- 172.21.15.60:0/2223879006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892c0b12e0 con 0x7f892c0b45d0 2024-04-25T10:08:07.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.006+0000 7f89337db640 1 -- 172.21.15.60:0/2223879006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f892c074770 con 0x7f892c0b45d0 2024-04-25T10:08:07.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.006+0000 7f8921ffb640 1 -- 172.21.15.60:0/2223879006 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f891c0c5430 con 0x7f892c0b45d0 2024-04-25T10:08:07.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.006+0000 7f8921ffb640 1 --2- 172.21.15.60:0/2223879006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8910066f60 0x7f8910069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.006+0000 7f8921ffb640 1 -- 172.21.15.60:0/2223879006 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f891c121490 con 0x7f892c0b45d0 2024-04-25T10:08:07.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.006+0000 7f8931577640 1 --2- 172.21.15.60:0/2223879006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8910066f60 0x7f8910069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.006+0000 7f8931577640 1 --2- 172.21.15.60:0/2223879006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8910066f60 0x7f8910069420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f892c14f690 tx=0x7f8928009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:07.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.010+0000 7f8921ffb640 1 -- 172.21.15.60:0/2223879006 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f891c0ae030 con 0x7f892c0b45d0 2024-04-25T10:08:07.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.226+0000 7f89337db640 1 -- 172.21.15.60:0/2223879006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f892c15a0c0 con 0x7f892c0b45d0 2024-04-25T10:08:07.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.226+0000 7f8921ffb640 1 -- 172.21.15.60:0/2223879006 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f891c018320 con 0x7f892c0b45d0 2024-04-25T10:08:07.234 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:07.234 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:08:07.234 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-04-25T10:08:07.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.235 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-04-25T10:08:07.235 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmKr12EhAAnHHZ5VjG0cywMrIv6VElVg==", 2024-04-25T10:08:07.235 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.235 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:08:07.235 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:08:07.235 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:08:07.235 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:08:07.235 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.235 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-04-25T10:08:07.235 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmU39rIRAAk6Ca0tKG4LpmR4pNxjzv0w==", 2024-04-25T10:08:07.235 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.236 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:08:07.236 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:08:07.236 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:08:07.236 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:08:07.236 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.236 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-04-25T10:08:07.236 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmVsshGxAAttXv0jYEGKgo0sPhIRTuqQ==", 2024-04-25T10:08:07.236 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.236 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:08:07.236 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:08:07.236 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:08:07.236 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:08:07.236 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.237 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-04-25T10:08:07.237 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmQtXUKRAAI3Iq02h3KH/PZbWKvJD9Jg==", 2024-04-25T10:08:07.237 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.237 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:08:07.237 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:08:07.237 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:08:07.237 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:08:07.237 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.237 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-04-25T10:08:07.237 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCkKipmE+UIBRAAc8ctpHoL3PkkbkMRCAJUGw==", 2024-04-25T10:08:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:08:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:08:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-04-25T10:08:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCnKipmbAnYCBAALLsPH2FjUpjoIZwTw+8aOA==", 2024-04-25T10:08:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:08:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:08:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.239 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-04-25T10:08:07.239 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCqKipmqrNSDxAAZdgFrcyeVwRQMKJJYIM98g==", 2024-04-25T10:08:07.239 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.239 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:07.239 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:08:07.239 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:08:07.239 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.239 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-04-25T10:08:07.239 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCtKipm1L+hFxAAPnPWlCm1Fmi0mN3tyReeQw==", 2024-04-25T10:08:07.239 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.239 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:07.240 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:08:07.240 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:08:07.240 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.240 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-04-25T10:08:07.240 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCzKipm2WyDFRAAmyfg6qu5drE9yMaR2FKd/A==", 2024-04-25T10:08:07.240 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.240 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:07.240 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:08:07.240 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:08:07.240 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.240 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-04-25T10:08:07.241 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC2KipmZLDhGRAAcgaz8YaPGUMIfxOLtWObyA==", 2024-04-25T10:08:07.241 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.241 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:07.241 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:08:07.241 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:08:07.241 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.241 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-04-25T10:08:07.241 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC5Kipm/3VMIBAA+rhGCyA/7d+s8m7ZWTzX/Q==", 2024-04-25T10:08:07.241 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.241 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:07.241 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:08:07.241 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:08:07.241 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.242 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-04-25T10:08:07.242 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8KipmFNCCJxAAv0OSTpAG+xIHhLqYXAZazQ==", 2024-04-25T10:08:07.242 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.242 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:07.242 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:08:07.242 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:08:07.242 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.242 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-04-25T10:08:07.242 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmv65dLxAA6/QpPrqE9aT+CtZqkyutog==", 2024-04-25T10:08:07.242 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.243 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:08:07.243 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-04-25T10:08:07.243 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:08:07.243 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.243 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-04-25T10:08:07.243 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipm2NiONxAAd3X4zx1JnBXKHJvdZ/zMdQ==", 2024-04-25T10:08:07.243 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.243 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:08:07.243 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:08:07.243 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:08:07.243 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:08:07.243 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.244 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-04-25T10:08:07.244 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmZRONNBAAa+XTv5YIBJWeUvf5dfoWxw==", 2024-04-25T10:08:07.244 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.244 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:08:07.244 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:08:07.244 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:08:07.244 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:08:07.244 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.244 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-04-25T10:08:07.244 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCjKipmgvfiABAApLvTC9U6HzeAefCvX4tiSQ==", 2024-04-25T10:08:07.244 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.245 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:08:07.245 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:08:07.245 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:08:07.245 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:08:07.245 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.245 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-04-25T10:08:07.245 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQChKipmumzYNhAAlCRYMpSNaTo7bz8P6x0W9g==", 2024-04-25T10:08:07.245 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.245 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:08:07.245 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:08:07.245 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:08:07.245 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:08:07.245 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.246 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-04-25T10:08:07.246 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmhxlLDxAAegBrJWdMNz6ZC3oXT2Qo0A==", 2024-04-25T10:08:07.246 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.246 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:08:07.246 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T10:08:07.246 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:08:07.246 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:07.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:07.246 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-04-25T10:08:07.246 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmpl1cDBAAGqx2kIak9NoWpkcdHczcFA==", 2024-04-25T10:08:07.246 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:08:07.247 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:08:07.247 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T10:08:07.247 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:08:07.247 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.247 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:07.247 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T10:08:07.247 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:08:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.230+0000 7f89337db640 1 -- 172.21.15.60:0/2223879006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8910066f60 msgr2=0x7f8910069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.230+0000 7f89337db640 1 --2- 172.21.15.60:0/2223879006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8910066f60 0x7f8910069420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f892c14f690 tx=0x7f8928009040 comp rx=0 tx=0).stop 2024-04-25T10:08:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.230+0000 7f89337db640 1 -- 172.21.15.60:0/2223879006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f892c0b45d0 msgr2=0x7f892c15d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.230+0000 7f89337db640 1 --2- 172.21.15.60:0/2223879006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f892c0b45d0 0x7f892c15d390 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f891c01c2a0 tx=0x7f891c01b410 comp rx=0 tx=0).stop 2024-04-25T10:08:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.230+0000 7f89337db640 1 -- 172.21.15.60:0/2223879006 shutdown_connections 2024-04-25T10:08:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.230+0000 7f89337db640 1 --2- 172.21.15.60:0/2223879006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8910066f60 0x7f8910069420 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.230+0000 7f89337db640 1 --2- 172.21.15.60:0/2223879006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f892c15a890 0x7f892c15d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.230+0000 7f89337db640 1 --2- 172.21.15.60:0/2223879006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f892c0b45d0 0x7f892c15d390 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.230+0000 7f89337db640 1 --2- 172.21.15.60:0/2223879006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f892c0b35f0 0x7f892c14b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.230+0000 7f89337db640 1 -- 172.21.15.60:0/2223879006 >> 172.21.15.60:0/2223879006 conn(0x7f892c0b5aa0 msgr2=0x7f892c06b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.230+0000 7f89337db640 1 -- 172.21.15.60:0/2223879006 shutdown_connections 2024-04-25T10:08:07.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.230+0000 7f89337db640 1 -- 172.21.15.60:0/2223879006 wait complete. 2024-04-25T10:08:07.253 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.358+0000 7f24f328e640 1 Processor -- start 2024-04-25T10:08:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.358+0000 7f24f328e640 1 -- start start 2024-04-25T10:08:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.358+0000 7f24f328e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f24ec0ce7a0 0x7f24ec0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.358+0000 7f24f328e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24ec0b4660 0x7f24ec0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.358+0000 7f24f328e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ec0cf0c0 con 0x7f24ec15bc90 2024-04-25T10:08:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.358+0000 7f24f328e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ec0cc370 con 0x7f24ec0b4660 2024-04-25T10:08:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.358+0000 7f24f328e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ec071ca0 con 0x7f24ec0ce7a0 2024-04-25T10:08:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.358+0000 7f24f182b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24ec0b4660 0x7f24ec0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.358+0000 7f24f182b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24ec0b4660 0x7f24ec0b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34100/0 (socket says 172.21.15.60:34100) 2024-04-25T10:08:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.358+0000 7f24f182b640 1 -- 172.21.15.60:0/1881850004 learned_addr learned my addr 172.21.15.60:0/1881850004 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:07.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.358+0000 7f24f182b640 1 -- 172.21.15.60:0/1881850004 >> v2:172.21.15.100:3300/0 conn(0x7f24ec0ce7a0 msgr2=0x7f24ec0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:07.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.358+0000 7f24f182b640 1 --2- 172.21.15.60:0/1881850004 >> v2:172.21.15.100:3300/0 conn(0x7f24ec0ce7a0 0x7f24ec0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.358+0000 7f24f182b640 1 -- 172.21.15.60:0/1881850004 >> v1:172.21.15.100:6789/0 conn(0x7f24ec15bc90 legacy=0x7f24ec0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:07.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.358+0000 7f24f182b640 1 -- 172.21.15.60:0/1881850004 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec0cca40 con 0x7f24ec0b4660 2024-04-25T10:08:07.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f182b640 1 --2- 172.21.15.60:0/1881850004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0b4660 0x7f24ec0b2ce0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f24ec04f5d0 tx=0x7f24e401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7e52e6d4f39013 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24e3fff640 1 -- 172.21.15.60:0/1881850004 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e40a5670 con 0x7f24ec0b4660 2024-04-25T10:08:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24e3fff640 1 -- 172.21.15.60:0/1881850004 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e40a5810 con 0x7f24ec0b4660 2024-04-25T10:08:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24e3fff640 1 -- 172.21.15.60:0/1881850004 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e40a5be0 con 0x7f24ec0b4660 2024-04-25T10:08:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f328e640 1 -- 172.21.15.60:0/1881850004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0b4660 msgr2=0x7f24ec0b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f328e640 1 --2- 172.21.15.60:0/1881850004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0b4660 0x7f24ec0b2ce0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f24ec04f5d0 tx=0x7f24e401c9b0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f328e640 1 -- 172.21.15.60:0/1881850004 shutdown_connections 2024-04-25T10:08:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f328e640 1 --2- 172.21.15.60:0/1881850004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0b4660 0x7f24ec0b2ce0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f328e640 1 --2- 172.21.15.60:0/1881850004 >> v2:172.21.15.100:3300/0 conn(0x7f24ec0ce7a0 0x7f24ec0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f328e640 1 -- 172.21.15.60:0/1881850004 >> 172.21.15.60:0/1881850004 conn(0x7f24ec0b12d0 msgr2=0x7f24ec0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f328e640 1 -- 172.21.15.60:0/1881850004 shutdown_connections 2024-04-25T10:08:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f328e640 1 -- 172.21.15.60:0/1881850004 wait complete. 2024-04-25T10:08:07.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f328e640 1 Processor -- start 2024-04-25T10:08:07.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f328e640 1 -- start start 2024-04-25T10:08:07.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f328e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec0ce7a0 0x7f24ec1f9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f328e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24ec15bc90 0x7f24ec159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f328e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec153e20 0x7f24ec1f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f328e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec0b4400 con 0x7f24ec153e20 2024-04-25T10:08:07.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f328e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec0b4280 con 0x7f24ec0ce7a0 2024-04-25T10:08:07.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f328e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec0b3220 con 0x7f24ec15bc90 2024-04-25T10:08:07.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f182b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec153e20 0x7f24ec1f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f182b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec153e20 0x7f24ec1f7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34104/0 (socket says 172.21.15.60:34104) 2024-04-25T10:08:07.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f182b640 1 -- 172.21.15.60:0/3843190697 learned_addr learned my addr 172.21.15.60:0/3843190697 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:07.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f102a640 1 --2- 172.21.15.60:0/3843190697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec0ce7a0 0x7f24ec1f9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f0829640 1 --2- 172.21.15.60:0/3843190697 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24ec15bc90 0x7f24ec159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f182b640 1 -- 172.21.15.60:0/3843190697 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24ec15bc90 msgr2=0x7f24ec159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f182b640 1 --2- 172.21.15.60:0/3843190697 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24ec15bc90 0x7f24ec159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f182b640 1 -- 172.21.15.60:0/3843190697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec0ce7a0 msgr2=0x7f24ec1f9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f182b640 1 --2- 172.21.15.60:0/3843190697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec0ce7a0 0x7f24ec1f9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.362+0000 7f24f182b640 1 -- 172.21.15.60:0/3843190697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec1fa780 con 0x7f24ec153e20 2024-04-25T10:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.366+0000 7f24f182b640 1 --2- 172.21.15.60:0/3843190697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec153e20 0x7f24ec1f7170 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f24e401a5e0 tx=0x7f24e4019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.366+0000 7f24e1ffb640 1 -- 172.21.15.60:0/3843190697 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e40bc020 con 0x7f24ec153e20 2024-04-25T10:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.366+0000 7f24e1ffb640 1 -- 172.21.15.60:0/3843190697 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e40b8690 con 0x7f24ec153e20 2024-04-25T10:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.366+0000 7f24f328e640 1 -- 172.21.15.60:0/3843190697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24ec1faa60 con 0x7f24ec153e20 2024-04-25T10:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.366+0000 7f24e1ffb640 1 -- 172.21.15.60:0/3843190697 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e40b8a80 con 0x7f24ec153e20 2024-04-25T10:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.366+0000 7f24f328e640 1 -- 172.21.15.60:0/3843190697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ec1f6c50 con 0x7f24ec153e20 2024-04-25T10:08:07.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.366+0000 7f24f328e640 1 -- 172.21.15.60:0/3843190697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24ec074770 con 0x7f24ec153e20 2024-04-25T10:08:07.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.366+0000 7f24e1ffb640 1 -- 172.21.15.60:0/3843190697 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24e40bf020 con 0x7f24ec153e20 2024-04-25T10:08:07.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.366+0000 7f24e1ffb640 1 --2- 172.21.15.60:0/3843190697 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f24d0066f90 0x7f24d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.366+0000 7f24e1ffb640 1 -- 172.21.15.60:0/3843190697 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f24e411f6e0 con 0x7f24ec153e20 2024-04-25T10:08:07.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.366+0000 7f24f102a640 1 --2- 172.21.15.60:0/3843190697 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f24d0066f90 0x7f24d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.366+0000 7f24f102a640 1 --2- 172.21.15.60:0/3843190697 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f24d0066f90 0x7f24d0069450 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f24dc006fd0 tx=0x7f24dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:07.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.370+0000 7f24e1ffb640 1 -- 172.21.15.60:0/3843190697 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f24e40c7020 con 0x7f24ec153e20 2024-04-25T10:08:07.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.526+0000 7f24f328e640 1 -- 172.21.15.60:0/3843190697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24ec158830 con 0x7f24ec153e20 2024-04-25T10:08:07.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.530+0000 7f24e1ffb640 1 -- 172.21.15.60:0/3843190697 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+12903 (secure 0 0 0) 0x7f24e40165a0 con 0x7f24ec153e20 2024-04-25T10:08:07.538 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:07.538 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":54,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:08:06.858110+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.530+0000 7f24f328e640 1 -- 172.21.15.60:0/3843190697 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f24d0066f90 msgr2=0x7f24d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.530+0000 7f24f328e640 1 --2- 172.21.15.60:0/3843190697 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f24d0066f90 0x7f24d0069450 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f24dc006fd0 tx=0x7f24dc008040 comp rx=0 tx=0).stop 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.530+0000 7f24f328e640 1 -- 172.21.15.60:0/3843190697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec153e20 msgr2=0x7f24ec1f7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.530+0000 7f24f328e640 1 --2- 172.21.15.60:0/3843190697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec153e20 0x7f24ec1f7170 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f24e401a5e0 tx=0x7f24e4019750 comp rx=0 tx=0).stop 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.530+0000 7f24f328e640 1 -- 172.21.15.60:0/3843190697 shutdown_connections 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.530+0000 7f24f328e640 1 --2- 172.21.15.60:0/3843190697 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f24d0066f90 0x7f24d0069450 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.530+0000 7f24f328e640 1 --2- 172.21.15.60:0/3843190697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec153e20 0x7f24ec1f7170 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.530+0000 7f24f328e640 1 --2- 172.21.15.60:0/3843190697 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24ec15bc90 0x7f24ec159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.530+0000 7f24f328e640 1 --2- 172.21.15.60:0/3843190697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec0ce7a0 0x7f24ec1f9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.530+0000 7f24f328e640 1 -- 172.21.15.60:0/3843190697 >> 172.21.15.60:0/3843190697 conn(0x7f24ec0b12d0 msgr2=0x7f24ec15c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.530+0000 7f24f328e640 1 -- 172.21.15.60:0/3843190697 shutdown_connections 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.530+0000 7f24f328e640 1 -- 172.21.15.60:0/3843190697 wait complete. 2024-04-25T10:08:07.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:07.556 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:07.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.678+0000 7fa2b342a640 1 Processor -- start 2024-04-25T10:08:07.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.678+0000 7fa2b342a640 1 -- start start 2024-04-25T10:08:07.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.678+0000 7fa2b342a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa2ac1620d0 0x7fa2ac0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.678+0000 7fa2b342a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa2ac0ced20 0x7fa2ac0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.678+0000 7fa2b342a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ac15bad0 con 0x7fa2ac15b6f0 2024-04-25T10:08:07.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.678+0000 7fa2b342a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ac161c40 con 0x7fa2ac1620d0 2024-04-25T10:08:07.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.678+0000 7fa2b342a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ac0cf150 con 0x7fa2ac0ced20 2024-04-25T10:08:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.678+0000 7fa2b09c5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa2ac1620d0 0x7fa2ac0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.678+0000 7fa2b09c5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa2ac1620d0 0x7fa2ac0ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34116/0 (socket says 172.21.15.60:34116) 2024-04-25T10:08:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.678+0000 7fa2b09c5640 1 -- 172.21.15.60:0/3651668160 learned_addr learned my addr 172.21.15.60:0/3651668160 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.678+0000 7fa2b19c7640 1 --2- 172.21.15.60:0/3651668160 >> v2:172.21.15.100:3300/0 conn(0x7fa2ac0ced20 0x7fa2ac0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.678+0000 7fa2b09c5640 1 -- 172.21.15.60:0/3651668160 >> v2:172.21.15.100:3300/0 conn(0x7fa2ac0ced20 msgr2=0x7fa2ac0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.678+0000 7fa2b09c5640 1 --2- 172.21.15.60:0/3651668160 >> v2:172.21.15.100:3300/0 conn(0x7fa2ac0ced20 0x7fa2ac0d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.678+0000 7fa2b09c5640 1 -- 172.21.15.60:0/3651668160 >> v1:172.21.15.100:6789/0 conn(0x7fa2ac15b6f0 legacy=0x7fa2ac15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.678+0000 7fa2b09c5640 1 -- 172.21.15.60:0/3651668160 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac15ebe0 con 0x7fa2ac1620d0 2024-04-25T10:08:07.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.682+0000 7fa2b09c5640 1 --2- 172.21.15.60:0/3651668160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac1620d0 0x7fa2ac0ce7e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac073010 tx=0x7fa29c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0e0f546bec83af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:07.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.682+0000 7fa2a3fff640 1 -- 172.21.15.60:0/3651668160 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c0ab610 con 0x7fa2ac1620d0 2024-04-25T10:08:07.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.682+0000 7fa2a3fff640 1 -- 172.21.15.60:0/3651668160 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29c0b0070 con 0x7fa2ac1620d0 2024-04-25T10:08:07.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.682+0000 7fa2a3fff640 1 -- 172.21.15.60:0/3651668160 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c0ab9d0 con 0x7fa2ac1620d0 2024-04-25T10:08:07.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.682+0000 7fa2b342a640 1 -- 172.21.15.60:0/3651668160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac1620d0 msgr2=0x7fa2ac0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.682+0000 7fa2b342a640 1 --2- 172.21.15.60:0/3651668160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac1620d0 0x7fa2ac0ce7e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac073010 tx=0x7fa29c01c520 comp rx=0 tx=0).stop 2024-04-25T10:08:07.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.682+0000 7fa2b342a640 1 -- 172.21.15.60:0/3651668160 shutdown_connections 2024-04-25T10:08:07.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.682+0000 7fa2b342a640 1 --2- 172.21.15.60:0/3651668160 >> v2:172.21.15.100:3300/0 conn(0x7fa2ac0ced20 0x7fa2ac0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.682+0000 7fa2b342a640 1 --2- 172.21.15.60:0/3651668160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac1620d0 0x7fa2ac0ce7e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.682+0000 7fa2b342a640 1 -- 172.21.15.60:0/3651668160 >> 172.21.15.60:0/3651668160 conn(0x7fa2ac072340 msgr2=0x7fa2ac072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:07.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.682+0000 7fa2b342a640 1 -- 172.21.15.60:0/3651668160 shutdown_connections 2024-04-25T10:08:07.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.682+0000 7fa2b342a640 1 -- 172.21.15.60:0/3651668160 wait complete. 2024-04-25T10:08:07.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.682+0000 7fa2b342a640 1 Processor -- start 2024-04-25T10:08:07.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.682+0000 7fa2b342a640 1 -- start start 2024-04-25T10:08:07.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b342a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2ac0ced20 0x7fa2ac0c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b342a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2ac15b6f0 0x7fa2ac0c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b342a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac1620d0 0x7fa2ac14f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b342a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac0b96d0 con 0x7fa2ac1620d0 2024-04-25T10:08:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b342a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac0b9550 con 0x7fa2ac15b6f0 2024-04-25T10:08:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b342a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac15e7e0 con 0x7fa2ac0ced20 2024-04-25T10:08:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b19c7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac1620d0 0x7fa2ac14f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b19c7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac1620d0 0x7fa2ac14f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34124/0 (socket says 172.21.15.60:34124) 2024-04-25T10:08:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b19c7640 1 -- 172.21.15.60:0/3309919915 learned_addr learned my addr 172.21.15.60:0/3309919915 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b09c5640 1 --2- 172.21.15.60:0/3309919915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2ac15b6f0 0x7fa2ac0c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b11c6640 1 --2- 172.21.15.60:0/3309919915 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2ac0ced20 0x7fa2ac0c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b19c7640 1 -- 172.21.15.60:0/3309919915 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2ac0ced20 msgr2=0x7fa2ac0c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b19c7640 1 --2- 172.21.15.60:0/3309919915 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2ac0ced20 0x7fa2ac0c63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b19c7640 1 -- 172.21.15.60:0/3309919915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2ac15b6f0 msgr2=0x7fa2ac0c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b19c7640 1 --2- 172.21.15.60:0/3309919915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2ac15b6f0 0x7fa2ac0c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b19c7640 1 -- 172.21.15.60:0/3309919915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac0c8490 con 0x7fa2ac1620d0 2024-04-25T10:08:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b19c7640 1 --2- 172.21.15.60:0/3309919915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac1620d0 0x7fa2ac14f890 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4007e40 tx=0x7fa2a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2a1ffb640 1 -- 172.21.15.60:0/3309919915 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4014620 con 0x7fa2ac1620d0 2024-04-25T10:08:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2a1ffb640 1 -- 172.21.15.60:0/3309919915 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a4019070 con 0x7fa2ac1620d0 2024-04-25T10:08:07.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2a1ffb640 1 -- 172.21.15.60:0/3309919915 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4014a20 con 0x7fa2ac1620d0 2024-04-25T10:08:07.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b342a640 1 -- 172.21.15.60:0/3309919915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ac0c8710 con 0x7fa2ac1620d0 2024-04-25T10:08:07.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b342a640 1 -- 172.21.15.60:0/3309919915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac0c8cd0 con 0x7fa2ac1620d0 2024-04-25T10:08:07.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2b342a640 1 -- 172.21.15.60:0/3309919915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ac074770 con 0x7fa2ac1620d0 2024-04-25T10:08:07.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.686+0000 7fa2a1ffb640 1 -- 172.21.15.60:0/3309919915 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa2a4014bc0 con 0x7fa2ac1620d0 2024-04-25T10:08:07.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.690+0000 7fa2a1ffb640 1 --2- 172.21.15.60:0/3309919915 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa290066f60 0x7fa290069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.690+0000 7fa2a1ffb640 1 -- 172.21.15.60:0/3309919915 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa2a40854e0 con 0x7fa2ac1620d0 2024-04-25T10:08:07.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.690+0000 7fa2b11c6640 1 --2- 172.21.15.60:0/3309919915 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa290066f60 0x7fa290069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.690+0000 7fa2b11c6640 1 --2- 172.21.15.60:0/3309919915 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa290066f60 0x7fa290069420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0b9120 tx=0x7fa2a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:07.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.694+0000 7fa2a1ffb640 1 -- 172.21.15.60:0/3309919915 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa2ac0c8490 con 0x7fa2ac1620d0 2024-04-25T10:08:07.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.858+0000 7fa2b342a640 1 -- 172.21.15.60:0/3309919915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2ac0d16e0 con 0x7fa2ac1620d0 2024-04-25T10:08:07.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.858+0000 7fa2a1ffb640 1 -- 172.21.15.60:0/3309919915 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2a40568d0 con 0x7fa2ac1620d0 2024-04-25T10:08:07.867 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:08:07.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.862+0000 7fa2b342a640 1 -- 172.21.15.60:0/3309919915 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa290066f60 msgr2=0x7fa290069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.862+0000 7fa2b342a640 1 --2- 172.21.15.60:0/3309919915 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa290066f60 0x7fa290069420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0b9120 tx=0x7fa2a8009040 comp rx=0 tx=0).stop 2024-04-25T10:08:07.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.862+0000 7fa2b342a640 1 -- 172.21.15.60:0/3309919915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac1620d0 msgr2=0x7fa2ac14f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.862+0000 7fa2b342a640 1 --2- 172.21.15.60:0/3309919915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac1620d0 0x7fa2ac14f890 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4007e40 tx=0x7fa2a400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.862+0000 7fa2b342a640 1 -- 172.21.15.60:0/3309919915 shutdown_connections 2024-04-25T10:08:07.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.862+0000 7fa2b342a640 1 --2- 172.21.15.60:0/3309919915 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa290066f60 0x7fa290069420 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.862+0000 7fa2b342a640 1 --2- 172.21.15.60:0/3309919915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac1620d0 0x7fa2ac14f890 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.862+0000 7fa2b342a640 1 --2- 172.21.15.60:0/3309919915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2ac15b6f0 0x7fa2ac0c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.862+0000 7fa2b342a640 1 --2- 172.21.15.60:0/3309919915 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2ac0ced20 0x7fa2ac0c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.862+0000 7fa2b342a640 1 -- 172.21.15.60:0/3309919915 >> 172.21.15.60:0/3309919915 conn(0x7fa2ac072340 msgr2=0x7fa2ac154400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:07.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.862+0000 7fa2b342a640 1 -- 172.21.15.60:0/3309919915 shutdown_connections 2024-04-25T10:08:07.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.862+0000 7fa2b342a640 1 -- 172.21.15.60:0/3309919915 wait complete. 2024-04-25T10:08:07.886 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:08:08.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.998+0000 7f62f2c51640 1 Processor -- start 2024-04-25T10:08:08.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.998+0000 7f62f2c51640 1 -- start start 2024-04-25T10:08:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.998+0000 7f62f2c51640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f62ec0ce7a0 0x7f62ec0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.998+0000 7f62f2c51640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f62ec0b4660 0x7f62ec0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.998+0000 7f62f2c51640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ec0cf0c0 con 0x7f62ec15bc90 2024-04-25T10:08:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.998+0000 7f62f2c51640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec0cc370 con 0x7f62ec0b4660 2024-04-25T10:08:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.998+0000 7f62f2c51640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec071ca0 con 0x7f62ec0ce7a0 2024-04-25T10:08:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.998+0000 7f62f11ee640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f62ec0b4660 0x7f62ec0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.998+0000 7f62f11ee640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f62ec0b4660 0x7f62ec0b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34128/0 (socket says 172.21.15.60:34128) 2024-04-25T10:08:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.998+0000 7f62f11ee640 1 -- 172.21.15.60:0/169169480 learned_addr learned my addr 172.21.15.60:0/169169480 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.998+0000 7f62f09ed640 1 --2- 172.21.15.60:0/169169480 >> v2:172.21.15.100:3300/0 conn(0x7f62ec0ce7a0 0x7f62ec0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.998+0000 7f62f11ee640 1 -- 172.21.15.60:0/169169480 >> v2:172.21.15.100:3300/0 conn(0x7f62ec0ce7a0 msgr2=0x7f62ec0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.998+0000 7f62f11ee640 1 --2- 172.21.15.60:0/169169480 >> v2:172.21.15.100:3300/0 conn(0x7f62ec0ce7a0 0x7f62ec0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.998+0000 7f62f11ee640 1 -- 172.21.15.60:0/169169480 >> v1:172.21.15.100:6789/0 conn(0x7f62ec15bc90 legacy=0x7f62ec0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.998+0000 7f62f11ee640 1 -- 172.21.15.60:0/169169480 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec0cca40 con 0x7f62ec0b4660 2024-04-25T10:08:08.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:07.998+0000 7f62f09ed640 1 --2- 172.21.15.60:0/169169480 >> v2:172.21.15.100:3300/0 conn(0x7f62ec0ce7a0 0x7f62ec0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:08.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.002+0000 7f62f11ee640 1 --2- 172.21.15.60:0/169169480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec0b4660 0x7f62ec0b2ce0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f62ec04f5d0 tx=0x7f62e401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e179adb3f112c9cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.002+0000 7f62e37fe640 1 -- 172.21.15.60:0/169169480 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e40a5670 con 0x7f62ec0b4660 2024-04-25T10:08:08.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.002+0000 7f62e37fe640 1 -- 172.21.15.60:0/169169480 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e40a5810 con 0x7f62ec0b4660 2024-04-25T10:08:08.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.002+0000 7f62e37fe640 1 -- 172.21.15.60:0/169169480 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e40a5be0 con 0x7f62ec0b4660 2024-04-25T10:08:08.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.002+0000 7f62f2c51640 1 -- 172.21.15.60:0/169169480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec0b4660 msgr2=0x7f62ec0b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.002+0000 7f62f2c51640 1 --2- 172.21.15.60:0/169169480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec0b4660 0x7f62ec0b2ce0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f62ec04f5d0 tx=0x7f62e401c9b0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.002+0000 7f62f2c51640 1 -- 172.21.15.60:0/169169480 shutdown_connections 2024-04-25T10:08:08.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.002+0000 7f62f2c51640 1 --2- 172.21.15.60:0/169169480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec0b4660 0x7f62ec0b2ce0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.002+0000 7f62f2c51640 1 --2- 172.21.15.60:0/169169480 >> v2:172.21.15.100:3300/0 conn(0x7f62ec0ce7a0 0x7f62ec0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.002+0000 7f62f2c51640 1 -- 172.21.15.60:0/169169480 >> 172.21.15.60:0/169169480 conn(0x7f62ec0b12d0 msgr2=0x7f62ec0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:08.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.002+0000 7f62f2c51640 1 -- 172.21.15.60:0/169169480 shutdown_connections 2024-04-25T10:08:08.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.002+0000 7f62f2c51640 1 -- 172.21.15.60:0/169169480 wait complete. 2024-04-25T10:08:08.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f2c51640 1 Processor -- start 2024-04-25T10:08:08.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f2c51640 1 -- start start 2024-04-25T10:08:08.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f2c51640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec0ce7a0 0x7f62ec13bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f2c51640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62ec15bc90 0x7f62ec1f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f2c51640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62ec1f7dd0 0x7f62ec156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f2c51640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0b4400 con 0x7f62ec0ce7a0 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f2c51640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0b4280 con 0x7f62ec1f7dd0 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f2c51640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0b3220 con 0x7f62ec15bc90 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f09ed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec0ce7a0 0x7f62ec13bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f09ed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec0ce7a0 0x7f62ec13bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34132/0 (socket says 172.21.15.60:34132) 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f09ed640 1 -- 172.21.15.60:0/1825827161 learned_addr learned my addr 172.21.15.60:0/1825827161 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f11ee640 1 --2- 172.21.15.60:0/1825827161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62ec1f7dd0 0x7f62ec156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62e3fff640 1 --2- 172.21.15.60:0/1825827161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62ec15bc90 0x7f62ec1f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f09ed640 1 -- 172.21.15.60:0/1825827161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62ec15bc90 msgr2=0x7f62ec1f7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f09ed640 1 --2- 172.21.15.60:0/1825827161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62ec15bc90 0x7f62ec1f7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f09ed640 1 -- 172.21.15.60:0/1825827161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62ec1f7dd0 msgr2=0x7f62ec156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f09ed640 1 --2- 172.21.15.60:0/1825827161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62ec1f7dd0 0x7f62ec156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f09ed640 1 -- 172.21.15.60:0/1825827161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec13c420 con 0x7f62ec0ce7a0 2024-04-25T10:08:08.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62e3fff640 1 --2- 172.21.15.60:0/1825827161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62ec15bc90 0x7f62ec1f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:08.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f09ed640 1 --2- 172.21.15.60:0/1825827161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec0ce7a0 0x7f62ec13bee0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f62e800c7c0 tx=0x7f62e800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62e17fa640 1 -- 172.21.15.60:0/1825827161 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e8007a30 con 0x7f62ec0ce7a0 2024-04-25T10:08:08.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62e17fa640 1 -- 172.21.15.60:0/1825827161 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e8007bd0 con 0x7f62ec0ce7a0 2024-04-25T10:08:08.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f2c51640 1 -- 172.21.15.60:0/1825827161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec1f8270 con 0x7f62ec0ce7a0 2024-04-25T10:08:08.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62e17fa640 1 -- 172.21.15.60:0/1825827161 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e80113f0 con 0x7f62ec0ce7a0 2024-04-25T10:08:08.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f2c51640 1 -- 172.21.15.60:0/1825827161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec14fbb0 con 0x7f62ec0ce7a0 2024-04-25T10:08:08.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62f2c51640 1 -- 172.21.15.60:0/1825827161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec074770 con 0x7f62ec0ce7a0 2024-04-25T10:08:08.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62e17fa640 1 -- 172.21.15.60:0/1825827161 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62e8011590 con 0x7f62ec0ce7a0 2024-04-25T10:08:08.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62e17fa640 1 --2- 172.21.15.60:0/1825827161 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62d8066f60 0x7f62d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.006+0000 7f62e17fa640 1 -- 172.21.15.60:0/1825827161 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f62e8084ef0 con 0x7f62ec0ce7a0 2024-04-25T10:08:08.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.010+0000 7f62e3fff640 1 --2- 172.21.15.60:0/1825827161 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62d8066f60 0x7f62d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.010+0000 7f62e3fff640 1 --2- 172.21.15.60:0/1825827161 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62d8066f60 0x7f62d8069420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f62ec15b640 tx=0x7f62dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.014+0000 7f62e17fa640 1 -- 172.21.15.60:0/1825827161 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62e8015020 con 0x7f62ec0ce7a0 2024-04-25T10:08:08.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.170+0000 7f62f2c51640 1 -- 172.21.15.60:0/1825827161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f62ec14f960 con 0x7f62ec0ce7a0 2024-04-25T10:08:08.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.174+0000 7f62e17fa640 1 -- 172.21.15.60:0/1825827161 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) v1 ==== 84+0+65 (secure 0 0 0) 0x7f62e8056240 con 0x7f62ec0ce7a0 2024-04-25T10:08:08.180 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:08.181 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T10:08:08.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:08.181 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T10:08:08.181 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T10:08:08.181 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:08.181 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T10:08:08.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.174+0000 7f62f2c51640 1 -- 172.21.15.60:0/1825827161 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62d8066f60 msgr2=0x7f62d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.174+0000 7f62f2c51640 1 --2- 172.21.15.60:0/1825827161 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62d8066f60 0x7f62d8069420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f62ec15b640 tx=0x7f62dc008040 comp rx=0 tx=0).stop 2024-04-25T10:08:08.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.174+0000 7f62f2c51640 1 -- 172.21.15.60:0/1825827161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec0ce7a0 msgr2=0x7f62ec13bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.174+0000 7f62f2c51640 1 --2- 172.21.15.60:0/1825827161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec0ce7a0 0x7f62ec13bee0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f62e800c7c0 tx=0x7f62e800cc80 comp rx=0 tx=0).stop 2024-04-25T10:08:08.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.174+0000 7f62f2c51640 1 -- 172.21.15.60:0/1825827161 shutdown_connections 2024-04-25T10:08:08.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.174+0000 7f62f2c51640 1 --2- 172.21.15.60:0/1825827161 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62d8066f60 0x7f62d8069420 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.174+0000 7f62f2c51640 1 --2- 172.21.15.60:0/1825827161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62ec1f7dd0 0x7f62ec156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.174+0000 7f62f2c51640 1 --2- 172.21.15.60:0/1825827161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62ec15bc90 0x7f62ec1f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.174+0000 7f62f2c51640 1 --2- 172.21.15.60:0/1825827161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec0ce7a0 0x7f62ec13bee0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.174+0000 7f62f2c51640 1 -- 172.21.15.60:0/1825827161 >> 172.21.15.60:0/1825827161 conn(0x7f62ec0b12d0 msgr2=0x7f62ec157f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:08.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.174+0000 7f62f2c51640 1 -- 172.21.15.60:0/1825827161 shutdown_connections 2024-04-25T10:08:08.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.174+0000 7f62f2c51640 1 -- 172.21.15.60:0/1825827161 wait complete. 2024-04-25T10:08:08.200 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:08:08.201 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:08:08.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.322+0000 7ff90549c640 1 Processor -- start 2024-04-25T10:08:08.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.322+0000 7ff90549c640 1 -- start start 2024-04-25T10:08:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff90549c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff90015b980 0x7ff9000ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff90549c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff9000ce540 0x7ff9000cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff90549c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9000cccd0 con 0x7ff9000cc450 2024-04-25T10:08:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff90549c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9000ce9a0 con 0x7ff90015b980 2024-04-25T10:08:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff90549c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff90015bd60 con 0x7ff9000ce540 2024-04-25T10:08:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff8feffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff90015b980 0x7ff9000ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff8ff7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff9000cc450 0x7ff900070d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49818/0 (socket says 172.21.15.60:49818) 2024-04-25T10:08:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff8ff7fe640 1 -- 172.21.15.60:0/2308189831 learned_addr learned my addr 172.21.15.60:0/2308189831 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff8fe7fc640 1 --2- 172.21.15.60:0/2308189831 >> v2:172.21.15.100:3300/0 conn(0x7ff9000ce540 0x7ff9000cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff8fdffb640 1 -- 172.21.15.60:0/2308189831 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706565700 0 0) 0x7ff9000cccd0 con 0x7ff9000cc450 2024-04-25T10:08:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff8feffd640 1 -- 172.21.15.60:0/2308189831 >> v2:172.21.15.100:3300/0 conn(0x7ff9000ce540 msgr2=0x7ff9000cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff8feffd640 1 --2- 172.21.15.60:0/2308189831 >> v2:172.21.15.100:3300/0 conn(0x7ff9000ce540 0x7ff9000cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff8feffd640 1 -- 172.21.15.60:0/2308189831 >> v1:172.21.15.60:6789/0 conn(0x7ff9000cc450 legacy=0x7ff900070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff8feffd640 1 -- 172.21.15.60:0/2308189831 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9000ccfb0 con 0x7ff90015b980 2024-04-25T10:08:08.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff8feffd640 1 --2- 172.21.15.60:0/2308189831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff90015b980 0x7ff9000ce000 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff9000d0fb0 tx=0x7ff8f001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f215350ed5f01c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff8fdffb640 1 -- 172.21.15.60:0/2308189831 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f00b0070 con 0x7ff90015b980 2024-04-25T10:08:08.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff8fdffb640 1 -- 172.21.15.60:0/2308189831 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f00ab600 con 0x7ff90015b980 2024-04-25T10:08:08.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.326+0000 7ff8fdffb640 1 -- 172.21.15.60:0/2308189831 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f00a4e60 con 0x7ff90015b980 2024-04-25T10:08:08.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 -- 172.21.15.60:0/2308189831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff90015b980 msgr2=0x7ff9000ce000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 --2- 172.21.15.60:0/2308189831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff90015b980 0x7ff9000ce000 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff9000d0fb0 tx=0x7ff8f001c3d0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 -- 172.21.15.60:0/2308189831 shutdown_connections 2024-04-25T10:08:08.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 --2- 172.21.15.60:0/2308189831 >> v2:172.21.15.100:3300/0 conn(0x7ff9000ce540 0x7ff9000cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 --2- 172.21.15.60:0/2308189831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff90015b980 0x7ff9000ce000 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 -- 172.21.15.60:0/2308189831 >> 172.21.15.60:0/2308189831 conn(0x7ff9000b33d0 msgr2=0x7ff9000b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:08.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 -- 172.21.15.60:0/2308189831 shutdown_connections 2024-04-25T10:08:08.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 -- 172.21.15.60:0/2308189831 wait complete. 2024-04-25T10:08:08.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 Processor -- start 2024-04-25T10:08:08.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 -- start start 2024-04-25T10:08:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9000cc450 0x7ff9000b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9000ce540 0x7ff9000b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff90015b980 0x7ff9001562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9001686a0 con 0x7ff9000ce540 2024-04-25T10:08:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9000cca70 con 0x7ff90015b980 2024-04-25T10:08:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff900168820 con 0x7ff9000cc450 2024-04-25T10:08:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff8fe7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9000ce540 0x7ff9000b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff8fe7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9000ce540 0x7ff9000b1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34150/0 (socket says 172.21.15.60:34150) 2024-04-25T10:08:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff8fe7fc640 1 -- 172.21.15.60:0/766779178 learned_addr learned my addr 172.21.15.60:0/766779178 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff8feffd640 1 --2- 172.21.15.60:0/766779178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9000cc450 0x7ff9000b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff8fe7fc640 1 -- 172.21.15.60:0/766779178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9000cc450 msgr2=0x7ff9000b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff8ff7fe640 1 --2- 172.21.15.60:0/766779178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff90015b980 0x7ff9001562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff8fe7fc640 1 --2- 172.21.15.60:0/766779178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9000cc450 0x7ff9000b2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff8fe7fc640 1 -- 172.21.15.60:0/766779178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff90015b980 msgr2=0x7ff9001562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff8fe7fc640 1 --2- 172.21.15.60:0/766779178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff90015b980 0x7ff9001562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff8fe7fc640 1 -- 172.21.15.60:0/766779178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9000b1940 con 0x7ff9000ce540 2024-04-25T10:08:08.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff8fe7fc640 1 --2- 172.21.15.60:0/766779178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9000ce540 0x7ff9000b1240 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400e9b0 tx=0x7ff8f400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff8e77fe640 1 -- 172.21.15.60:0/766779178 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4017070 con 0x7ff9000ce540 2024-04-25T10:08:08.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 -- 172.21.15.60:0/766779178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9000b1c20 con 0x7ff9000ce540 2024-04-25T10:08:08.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff8e77fe640 1 -- 172.21.15.60:0/766779178 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f400c9a0 con 0x7ff9000ce540 2024-04-25T10:08:08.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff8e77fe640 1 -- 172.21.15.60:0/766779178 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f400cd70 con 0x7ff9000ce540 2024-04-25T10:08:08.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 -- 172.21.15.60:0/766779178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff900146f40 con 0x7ff9000ce540 2024-04-25T10:08:08.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.330+0000 7ff90549c640 1 -- 172.21.15.60:0/766779178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff900147140 con 0x7ff9000ce540 2024-04-25T10:08:08.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.334+0000 7ff8e77fe640 1 -- 172.21.15.60:0/766779178 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8f401a070 con 0x7ff9000ce540 2024-04-25T10:08:08.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.334+0000 7ff8e77fe640 1 --2- 172.21.15.60:0/766779178 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff8ec066f60 0x7ff8ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.334+0000 7ff8e77fe640 1 -- 172.21.15.60:0/766779178 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff8f40850e0 con 0x7ff9000ce540 2024-04-25T10:08:08.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.334+0000 7ff8feffd640 1 --2- 172.21.15.60:0/766779178 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff8ec066f60 0x7ff8ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.334+0000 7ff8feffd640 1 --2- 172.21.15.60:0/766779178 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff8ec066f60 0x7ff8ec069420 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00ad000 tx=0x7ff8f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.338+0000 7ff8e77fe640 1 -- 172.21.15.60:0/766779178 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff900147140 con 0x7ff9000ce540 2024-04-25T10:08:08.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.494+0000 7ff90549c640 1 -- 172.21.15.60:0/766779178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff9000b2140 con 0x7ff9000ce540 2024-04-25T10:08:08.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.846+0000 7ff8e77fe640 1 -- 172.21.15.60:0/766779178 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff8f4056450 con 0x7ff9000ce540 2024-04-25T10:08:08.854 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T10:08:08.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.850+0000 7ff90549c640 1 -- 172.21.15.60:0/766779178 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff8ec066f60 msgr2=0x7ff8ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.850+0000 7ff90549c640 1 --2- 172.21.15.60:0/766779178 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff8ec066f60 0x7ff8ec069420 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00ad000 tx=0x7ff8f0008040 comp rx=0 tx=0).stop 2024-04-25T10:08:08.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.850+0000 7ff90549c640 1 -- 172.21.15.60:0/766779178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9000ce540 msgr2=0x7ff9000b1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.850+0000 7ff90549c640 1 --2- 172.21.15.60:0/766779178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9000ce540 0x7ff9000b1240 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400e9b0 tx=0x7ff8f400ee70 comp rx=0 tx=0).stop 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.850+0000 7ff90549c640 1 -- 172.21.15.60:0/766779178 shutdown_connections 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.850+0000 7ff90549c640 1 --2- 172.21.15.60:0/766779178 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff8ec066f60 0x7ff8ec069420 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.850+0000 7ff90549c640 1 --2- 172.21.15.60:0/766779178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff90015b980 0x7ff9001562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.850+0000 7ff90549c640 1 --2- 172.21.15.60:0/766779178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9000ce540 0x7ff9000b1240 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.850+0000 7ff90549c640 1 --2- 172.21.15.60:0/766779178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9000cc450 0x7ff9000b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.850+0000 7ff90549c640 1 -- 172.21.15.60:0/766779178 >> 172.21.15.60:0/766779178 conn(0x7ff9000b33d0 msgr2=0x7ff900158860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.850+0000 7ff90549c640 1 -- 172.21.15.60:0/766779178 shutdown_connections 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:08.850+0000 7ff90549c640 1 -- 172.21.15.60:0/766779178 wait complete. 2024-04-25T10:08:08.885 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:08:09.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.014+0000 7f9e9c725640 1 Processor -- start 2024-04-25T10:08:09.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.014+0000 7f9e9c725640 1 -- start start 2024-04-25T10:08:09.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.014+0000 7f9e9c725640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9e940b3d80 0x7f9e94160000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.014+0000 7f9e9c725640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e940cffd0 con 0x7f9e9415cb50 2024-04-25T10:08:09.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.014+0000 7f9e9c725640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e9415f520 con 0x7f9e9415ba30 2024-04-25T10:08:09.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.014+0000 7f9e9c725640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e940cfd90 con 0x7f9e940b3d80 2024-04-25T10:08:09.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.014+0000 7f9e99cc0640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9e9415cb50 0x7f9e940b3670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41474/0 (socket says 172.21.15.60:41474) 2024-04-25T10:08:09.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.014+0000 7f9e99cc0640 1 -- 172.21.15.60:0/3820061010 learned_addr learned my addr 172.21.15.60:0/3820061010 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:09.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.014+0000 7f9e9acc2640 1 --2- 172.21.15.60:0/3820061010 >> v2:172.21.15.60:3300/0 conn(0x7f9e940b3d80 0x7f9e94160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.014+0000 7f9e9acc2640 1 -- 172.21.15.60:0/3820061010 >> v1:172.21.15.100:6789/0 conn(0x7f9e9415ba30 legacy=0x7f9e940cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:09.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.014+0000 7f9e9acc2640 1 -- 172.21.15.60:0/3820061010 >> v1:172.21.15.60:6789/0 conn(0x7f9e9415cb50 legacy=0x7f9e940b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.014+0000 7f9e9acc2640 1 -- 172.21.15.60:0/3820061010 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9415f800 con 0x7f9e940b3d80 2024-04-25T10:08:09.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.018+0000 7f9e9acc2640 1 --2- 172.21.15.60:0/3820061010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e940b3d80 0x7f9e94160000 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9e900097c0 tx=0x7f9e9001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2c7008222f8a2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.018+0000 7f9e994bf640 1 -- 172.21.15.60:0/3820061010 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e900b0070 con 0x7f9e940b3d80 2024-04-25T10:08:09.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.018+0000 7f9e994bf640 1 -- 172.21.15.60:0/3820061010 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e900ab600 con 0x7f9e940b3d80 2024-04-25T10:08:09.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.018+0000 7f9e994bf640 1 -- 172.21.15.60:0/3820061010 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e900ab9d0 con 0x7f9e940b3d80 2024-04-25T10:08:09.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.018+0000 7f9e9c725640 1 -- 172.21.15.60:0/3820061010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e940b3d80 msgr2=0x7f9e94160000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.018+0000 7f9e9c725640 1 --2- 172.21.15.60:0/3820061010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e940b3d80 0x7f9e94160000 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9e900097c0 tx=0x7f9e9001c460 comp rx=0 tx=0).stop 2024-04-25T10:08:09.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.018+0000 7f9e9c725640 1 -- 172.21.15.60:0/3820061010 shutdown_connections 2024-04-25T10:08:09.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.018+0000 7f9e9c725640 1 --2- 172.21.15.60:0/3820061010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e940b3d80 0x7f9e94160000 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.018+0000 7f9e9c725640 1 -- 172.21.15.60:0/3820061010 >> 172.21.15.60:0/3820061010 conn(0x7f9e94066b70 msgr2=0x7f9e94066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:09.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.018+0000 7f9e9c725640 1 -- 172.21.15.60:0/3820061010 shutdown_connections 2024-04-25T10:08:09.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.018+0000 7f9e9c725640 1 -- 172.21.15.60:0/3820061010 wait complete. 2024-04-25T10:08:09.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.018+0000 7f9e9c725640 1 Processor -- start 2024-04-25T10:08:09.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.018+0000 7f9e9c725640 1 -- start start 2024-04-25T10:08:09.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e9c725640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e940b3d80 0x7f9e940b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e9c725640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e9415cb50 0x7f9e94164410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e9c725640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e94164950 0x7f9e940ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e9c725640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e940c76f0 con 0x7f9e9415cb50 2024-04-25T10:08:09.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e9c725640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e940c7570 con 0x7f9e94164950 2024-04-25T10:08:09.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e9c725640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e940c7870 con 0x7f9e940b3d80 2024-04-25T10:08:09.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e99cc0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e9415cb50 0x7f9e94164410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e99cc0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e9415cb50 0x7f9e94164410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58648/0 (socket says 172.21.15.60:58648) 2024-04-25T10:08:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e99cc0640 1 -- 172.21.15.60:0/4148195846 learned_addr learned my addr 172.21.15.60:0/4148195846 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e9acc2640 1 --2- 172.21.15.60:0/4148195846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e94164950 0x7f9e940ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e99cc0640 1 -- 172.21.15.60:0/4148195846 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e940b3d80 msgr2=0x7f9e940b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e9a4c1640 1 --2- 172.21.15.60:0/4148195846 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e940b3d80 0x7f9e940b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e99cc0640 1 --2- 172.21.15.60:0/4148195846 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e940b3d80 0x7f9e940b6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e99cc0640 1 -- 172.21.15.60:0/4148195846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e94164950 msgr2=0x7f9e940ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e99cc0640 1 --2- 172.21.15.60:0/4148195846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e94164950 0x7f9e940ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e99cc0640 1 -- 172.21.15.60:0/4148195846 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e94155c30 con 0x7f9e9415cb50 2024-04-25T10:08:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e9acc2640 1 --2- 172.21.15.60:0/4148195846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e94164950 0x7f9e940ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e99cc0640 1 --2- 172.21.15.60:0/4148195846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e9415cb50 0x7f9e94164410 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400c430 tx=0x7f9e8400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e8affd640 1 -- 172.21.15.60:0/4148195846 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e84017070 con 0x7f9e9415cb50 2024-04-25T10:08:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e8affd640 1 -- 172.21.15.60:0/4148195846 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e84005cf0 con 0x7f9e9415cb50 2024-04-25T10:08:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e8affd640 1 -- 172.21.15.60:0/4148195846 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e840072f0 con 0x7f9e9415cb50 2024-04-25T10:08:09.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e9c725640 1 -- 172.21.15.60:0/4148195846 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e94164eb0 con 0x7f9e9415cb50 2024-04-25T10:08:09.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e9c725640 1 -- 172.21.15.60:0/4148195846 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e941561d0 con 0x7f9e9415cb50 2024-04-25T10:08:09.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e9c725640 1 -- 172.21.15.60:0/4148195846 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e941563d0 con 0x7f9e9415cb50 2024-04-25T10:08:09.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e8affd640 1 -- 172.21.15.60:0/4148195846 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e8401a070 con 0x7f9e9415cb50 2024-04-25T10:08:09.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e8affd640 1 --2- 172.21.15.60:0/4148195846 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e78066f60 0x7f9e78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e8affd640 1 -- 172.21.15.60:0/4148195846 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9e840841b0 con 0x7f9e9415cb50 2024-04-25T10:08:09.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.022+0000 7f9e9a4c1640 1 --2- 172.21.15.60:0/4148195846 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e78066f60 0x7f9e78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.026+0000 7f9e9a4c1640 1 --2- 172.21.15.60:0/4148195846 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e78066f60 0x7f9e78069420 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c004450 tx=0x7f9e8c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.030+0000 7f9e8affd640 1 -- 172.21.15.60:0/4148195846 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e941563d0 con 0x7f9e9415cb50 2024-04-25T10:08:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.190+0000 7f9e9c725640 1 -- 172.21.15.60:0/4148195846 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9e94074770 con 0x7f9e9415cb50 2024-04-25T10:08:09.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.858+0000 7f9e8affd640 1 -- 172.21.15.60:0/4148195846 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9e84055430 con 0x7f9e9415cb50 2024-04-25T10:08:09.866 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T10:08:09.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.862+0000 7f9e9c725640 1 -- 172.21.15.60:0/4148195846 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e78066f60 msgr2=0x7f9e78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.862+0000 7f9e9c725640 1 --2- 172.21.15.60:0/4148195846 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e78066f60 0x7f9e78069420 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c004450 tx=0x7f9e8c009040 comp rx=0 tx=0).stop 2024-04-25T10:08:09.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.862+0000 7f9e9c725640 1 -- 172.21.15.60:0/4148195846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e9415cb50 msgr2=0x7f9e94164410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.862+0000 7f9e9c725640 1 --2- 172.21.15.60:0/4148195846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e9415cb50 0x7f9e94164410 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400c430 tx=0x7f9e8400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.862+0000 7f9e9c725640 1 -- 172.21.15.60:0/4148195846 shutdown_connections 2024-04-25T10:08:09.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.862+0000 7f9e9c725640 1 --2- 172.21.15.60:0/4148195846 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e78066f60 0x7f9e78069420 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.862+0000 7f9e9c725640 1 --2- 172.21.15.60:0/4148195846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e94164950 0x7f9e940ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.862+0000 7f9e9c725640 1 --2- 172.21.15.60:0/4148195846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e9415cb50 0x7f9e94164410 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.862+0000 7f9e9c725640 1 --2- 172.21.15.60:0/4148195846 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e940b3d80 0x7f9e940b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.862+0000 7f9e9c725640 1 -- 172.21.15.60:0/4148195846 >> 172.21.15.60:0/4148195846 conn(0x7f9e94066b70 msgr2=0x7f9e940cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:09.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.862+0000 7f9e9c725640 1 -- 172.21.15.60:0/4148195846 shutdown_connections 2024-04-25T10:08:09.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:09.862+0000 7f9e9c725640 1 -- 172.21.15.60:0/4148195846 wait complete. 2024-04-25T10:08:09.904 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:08:10.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.022+0000 7f40c2be3640 1 Processor -- start 2024-04-25T10:08:10.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.022+0000 7f40c2be3640 1 -- start start 2024-04-25T10:08:10.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.022+0000 7f40c2be3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f40bc157e60 0x7f40bc15eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.022+0000 7f40c2be3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40bc0b3410 0x7f40bc0b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.022+0000 7f40c2be3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40bc0d2560 con 0x7f40bc0d20e0 2024-04-25T10:08:10.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.022+0000 7f40c2be3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40bc0c7430 con 0x7f40bc157e60 2024-04-25T10:08:10.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.022+0000 7f40c2be3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0c6ef0 con 0x7f40bc0b3410 2024-04-25T10:08:10.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.022+0000 7f40c097f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f40bc157e60 0x7f40bc15eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.022+0000 7f40c097f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f40bc157e60 0x7f40bc15eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58652/0 (socket says 172.21.15.60:58652) 2024-04-25T10:08:10.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.022+0000 7f40c097f640 1 -- 172.21.15.60:0/4204848187 learned_addr learned my addr 172.21.15.60:0/4204848187 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:10.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.022+0000 7f40c1180640 1 --2- 172.21.15.60:0/4204848187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40bc0b3410 0x7f40bc0b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.022+0000 7f40c097f640 1 -- 172.21.15.60:0/4204848187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40bc0b3410 msgr2=0x7f40bc0b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.022+0000 7f40c097f640 1 --2- 172.21.15.60:0/4204848187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40bc0b3410 0x7f40bc0b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.022+0000 7f40c097f640 1 -- 172.21.15.60:0/4204848187 >> v1:172.21.15.100:6789/0 conn(0x7f40bc0d20e0 legacy=0x7f40bc0c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.022+0000 7f40c097f640 1 -- 172.21.15.60:0/4204848187 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc0c7150 con 0x7f40bc157e60 2024-04-25T10:08:10.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.022+0000 7f40c1180640 1 --2- 172.21.15.60:0/4204848187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40bc0b3410 0x7f40bc0b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:10.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c097f640 1 --2- 172.21.15.60:0/4204848187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40bc157e60 0x7f40bc15eb50 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009e50 tx=0x7f40b401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d211527316361635 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40bb7fe640 1 -- 172.21.15.60:0/4204848187 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b40a6500 con 0x7f40bc157e60 2024-04-25T10:08:10.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40bb7fe640 1 -- 172.21.15.60:0/4204848187 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b40a66a0 con 0x7f40bc157e60 2024-04-25T10:08:10.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40bb7fe640 1 -- 172.21.15.60:0/4204848187 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b40a6a70 con 0x7f40bc157e60 2024-04-25T10:08:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c2be3640 1 -- 172.21.15.60:0/4204848187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40bc157e60 msgr2=0x7f40bc15eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c2be3640 1 --2- 172.21.15.60:0/4204848187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40bc157e60 0x7f40bc15eb50 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009e50 tx=0x7f40b401c780 comp rx=0 tx=0).stop 2024-04-25T10:08:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c2be3640 1 -- 172.21.15.60:0/4204848187 shutdown_connections 2024-04-25T10:08:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c2be3640 1 --2- 172.21.15.60:0/4204848187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40bc0b3410 0x7f40bc0b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c2be3640 1 --2- 172.21.15.60:0/4204848187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40bc157e60 0x7f40bc15eb50 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c2be3640 1 -- 172.21.15.60:0/4204848187 >> 172.21.15.60:0/4204848187 conn(0x7f40bc0669a0 msgr2=0x7f40bc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c2be3640 1 -- 172.21.15.60:0/4204848187 shutdown_connections 2024-04-25T10:08:10.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c2be3640 1 -- 172.21.15.60:0/4204848187 wait complete. 2024-04-25T10:08:10.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c2be3640 1 Processor -- start 2024-04-25T10:08:10.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c2be3640 1 -- start start 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c2be3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40bc0b3410 0x7f40bc14e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c2be3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40bc0d20e0 0x7f40bc151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c2be3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40bc157e60 0x7f40bc0cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c2be3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc1583c0 con 0x7f40bc157e60 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c2be3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc158240 con 0x7f40bc0d20e0 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c2be3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc06d9d0 con 0x7f40bc0b3410 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c1180640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40bc157e60 0x7f40bc0cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c1180640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40bc157e60 0x7f40bc0cb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58654/0 (socket says 172.21.15.60:58654) 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c1180640 1 -- 172.21.15.60:0/2307457755 learned_addr learned my addr 172.21.15.60:0/2307457755 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40bbfff640 1 --2- 172.21.15.60:0/2307457755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40bc0d20e0 0x7f40bc151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.026+0000 7f40c097f640 1 --2- 172.21.15.60:0/2307457755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40bc0b3410 0x7f40bc14e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.030+0000 7f40c1180640 1 -- 172.21.15.60:0/2307457755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40bc0b3410 msgr2=0x7f40bc14e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.030+0000 7f40c1180640 1 --2- 172.21.15.60:0/2307457755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40bc0b3410 0x7f40bc14e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.030+0000 7f40c1180640 1 -- 172.21.15.60:0/2307457755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40bc0d20e0 msgr2=0x7f40bc151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.030+0000 7f40c1180640 1 --2- 172.21.15.60:0/2307457755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40bc0d20e0 0x7f40bc151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.030+0000 7f40c1180640 1 -- 172.21.15.60:0/2307457755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc151c90 con 0x7f40bc157e60 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.030+0000 7f40c1180640 1 --2- 172.21.15.60:0/2307457755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40bc157e60 0x7f40bc0cb480 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00c820 tx=0x7f40ac00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.030+0000 7f40b97fa640 1 -- 172.21.15.60:0/2307457755 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac015070 con 0x7f40bc157e60 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.030+0000 7f40b97fa640 1 -- 172.21.15.60:0/2307457755 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ac007a70 con 0x7f40bc157e60 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.030+0000 7f40b97fa640 1 -- 172.21.15.60:0/2307457755 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac007e60 con 0x7f40bc157e60 2024-04-25T10:08:10.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.030+0000 7f40c2be3640 1 -- 172.21.15.60:0/2307457755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40bc0c8e30 con 0x7f40bc157e60 2024-04-25T10:08:10.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.030+0000 7f40c2be3640 1 -- 172.21.15.60:0/2307457755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40bc0c93f0 con 0x7f40bc157e60 2024-04-25T10:08:10.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.030+0000 7f40c2be3640 1 -- 172.21.15.60:0/2307457755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40bc074770 con 0x7f40bc157e60 2024-04-25T10:08:10.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.030+0000 7f40b97fa640 1 -- 172.21.15.60:0/2307457755 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40ac018070 con 0x7f40bc157e60 2024-04-25T10:08:10.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.030+0000 7f40b97fa640 1 --2- 172.21.15.60:0/2307457755 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f40a8066f60 0x7f40a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.030+0000 7f40b97fa640 1 -- 172.21.15.60:0/2307457755 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f40ac084f30 con 0x7f40bc157e60 2024-04-25T10:08:10.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.030+0000 7f40c097f640 1 --2- 172.21.15.60:0/2307457755 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f40a8066f60 0x7f40a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.034+0000 7f40c097f640 1 --2- 172.21.15.60:0/2307457755 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f40a8066f60 0x7f40a8069420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f40b40ae000 tx=0x7f40b40b1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.038+0000 7f40b97fa640 1 -- 172.21.15.60:0/2307457755 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f40ac055fa0 con 0x7f40bc157e60 2024-04-25T10:08:10.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.230+0000 7f40c2be3640 1 -- 172.21.15.60:0/2307457755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f40bc0cef70 con 0x7f40bc157e60 2024-04-25T10:08:10.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.862+0000 7f40b97fa640 1 -- 172.21.15.60:0/2307457755 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v26) v1 ==== 172+0+0 (secure 0 0 0) 0x7f40ac0558a0 con 0x7f40bc157e60 2024-04-25T10:08:10.869 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 8 and data pool 9 2024-04-25T10:08:10.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.866+0000 7f40c2be3640 1 -- 172.21.15.60:0/2307457755 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f40a8066f60 msgr2=0x7f40a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.866+0000 7f40c2be3640 1 --2- 172.21.15.60:0/2307457755 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f40a8066f60 0x7f40a8069420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f40b40ae000 tx=0x7f40b40b1040 comp rx=0 tx=0).stop 2024-04-25T10:08:10.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.866+0000 7f40c2be3640 1 -- 172.21.15.60:0/2307457755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40bc157e60 msgr2=0x7f40bc0cb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.866+0000 7f40c2be3640 1 --2- 172.21.15.60:0/2307457755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40bc157e60 0x7f40bc0cb480 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00c820 tx=0x7f40ac00cce0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.866+0000 7f40c2be3640 1 -- 172.21.15.60:0/2307457755 shutdown_connections 2024-04-25T10:08:10.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.866+0000 7f40c2be3640 1 --2- 172.21.15.60:0/2307457755 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f40a8066f60 0x7f40a8069420 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.866+0000 7f40c2be3640 1 --2- 172.21.15.60:0/2307457755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40bc157e60 0x7f40bc0cb480 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.866+0000 7f40c2be3640 1 --2- 172.21.15.60:0/2307457755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40bc0d20e0 0x7f40bc151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.866+0000 7f40c2be3640 1 --2- 172.21.15.60:0/2307457755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40bc0b3410 0x7f40bc14e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.866+0000 7f40c2be3640 1 -- 172.21.15.60:0/2307457755 >> 172.21.15.60:0/2307457755 conn(0x7f40bc0669a0 msgr2=0x7f40bc15f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.866+0000 7f40c2be3640 1 -- 172.21.15.60:0/2307457755 shutdown_connections 2024-04-25T10:08:10.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:10.866+0000 7f40c2be3640 1 -- 172.21.15.60:0/2307457755 wait complete. 2024-04-25T10:08:10.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.018+0000 7fb0ca69a640 1 Processor -- start 2024-04-25T10:08:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.018+0000 7fb0ca69a640 1 -- start start 2024-04-25T10:08:11.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.022+0000 7fb0ca69a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb0c415a7a0 0x7fb0c40cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.022+0000 7fb0ca69a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0c406ec30 0x7fb0c4071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.022+0000 7fb0ca69a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb0c40b37e0 0x7fb0c40b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.022+0000 7fb0ca69a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c40cdaa0 con 0x7fb0c415a7a0 2024-04-25T10:08:11.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.022+0000 7fb0ca69a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c406d2f0 con 0x7fb0c40b37e0 2024-04-25T10:08:11.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.022+0000 7fb0ca69a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c406d8b0 con 0x7fb0c406ec30 2024-04-25T10:08:11.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.022+0000 7fb0c3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb0c415a7a0 0x7fb0c40cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.022+0000 7fb0c3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb0c415a7a0 0x7fb0c40cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58670/0 (socket says 172.21.15.60:58670) 2024-04-25T10:08:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.022+0000 7fb0c3fff640 1 -- 172.21.15.60:0/3250830879 learned_addr learned my addr 172.21.15.60:0/3250830879 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.022+0000 7fb0c8c37640 1 --2- 172.21.15.60:0/3250830879 >> v2:172.21.15.100:3300/0 conn(0x7fb0c40b37e0 0x7fb0c40b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.022+0000 7fb0c37fe640 1 --2- 172.21.15.60:0/3250830879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0c406ec30 0x7fb0c4071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.022+0000 7fb0c3fff640 1 -- 172.21.15.60:0/3250830879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0c406ec30 msgr2=0x7fb0c4071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.022+0000 7fb0c3fff640 1 --2- 172.21.15.60:0/3250830879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0c406ec30 0x7fb0c4071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.022+0000 7fb0c3fff640 1 -- 172.21.15.60:0/3250830879 >> v2:172.21.15.100:3300/0 conn(0x7fb0c40b37e0 msgr2=0x7fb0c40b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.022+0000 7fb0c3fff640 1 --2- 172.21.15.60:0/3250830879 >> v2:172.21.15.100:3300/0 conn(0x7fb0c40b37e0 0x7fb0c40b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.022+0000 7fb0c3fff640 1 -- 172.21.15.60:0/3250830879 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c40721e0 con 0x7fb0c415a7a0 2024-04-25T10:08:11.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.026+0000 7fb0c3fff640 1 --2- 172.21.15.60:0/3250830879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c415a7a0 0x7fb0c40cd560 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40b2660 tx=0x7fb0b401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8154f2fc5ae7810c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.026+0000 7fb0c2ffd640 1 -- 172.21.15.60:0/3250830879 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40ab5f0 con 0x7fb0c415a7a0 2024-04-25T10:08:11.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.026+0000 7fb0c2ffd640 1 -- 172.21.15.60:0/3250830879 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b40b0070 con 0x7fb0c415a7a0 2024-04-25T10:08:11.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.026+0000 7fb0c2ffd640 1 -- 172.21.15.60:0/3250830879 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40ab9b0 con 0x7fb0c415a7a0 2024-04-25T10:08:11.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.026+0000 7fb0ca69a640 1 -- 172.21.15.60:0/3250830879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c415a7a0 msgr2=0x7fb0c40cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.026+0000 7fb0ca69a640 1 --2- 172.21.15.60:0/3250830879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c415a7a0 0x7fb0c40cd560 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40b2660 tx=0x7fb0b401c520 comp rx=0 tx=0).stop 2024-04-25T10:08:11.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.026+0000 7fb0ca69a640 1 -- 172.21.15.60:0/3250830879 shutdown_connections 2024-04-25T10:08:11.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.026+0000 7fb0ca69a640 1 --2- 172.21.15.60:0/3250830879 >> v2:172.21.15.100:3300/0 conn(0x7fb0c40b37e0 0x7fb0c40b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.026+0000 7fb0ca69a640 1 --2- 172.21.15.60:0/3250830879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0c406ec30 0x7fb0c4071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.026+0000 7fb0ca69a640 1 --2- 172.21.15.60:0/3250830879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c415a7a0 0x7fb0c40cd560 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.026+0000 7fb0ca69a640 1 -- 172.21.15.60:0/3250830879 >> 172.21.15.60:0/3250830879 conn(0x7fb0c40b12d0 msgr2=0x7fb0c40b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.026+0000 7fb0ca69a640 1 -- 172.21.15.60:0/3250830879 shutdown_connections 2024-04-25T10:08:11.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.026+0000 7fb0ca69a640 1 -- 172.21.15.60:0/3250830879 wait complete. 2024-04-25T10:08:11.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.026+0000 7fb0ca69a640 1 Processor -- start 2024-04-25T10:08:11.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.026+0000 7fb0ca69a640 1 -- start start 2024-04-25T10:08:11.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0ca69a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0c406ec30 0x7fb0c4152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0ca69a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0c40b37e0 0x7fb0c41539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0ca69a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c4153f10 0x7fb0c41638a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0ca69a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c415ad00 con 0x7fb0c4153f10 2024-04-25T10:08:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0ca69a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c415ab80 con 0x7fb0c406ec30 2024-04-25T10:08:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0ca69a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c406e550 con 0x7fb0c40b37e0 2024-04-25T10:08:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c8c37640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c4153f10 0x7fb0c41638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c8c37640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c4153f10 0x7fb0c41638a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58678/0 (socket says 172.21.15.60:58678) 2024-04-25T10:08:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c8c37640 1 -- 172.21.15.60:0/2017822320 learned_addr learned my addr 172.21.15.60:0/2017822320 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c37fe640 1 --2- 172.21.15.60:0/2017822320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0c40b37e0 0x7fb0c41539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c8c37640 1 -- 172.21.15.60:0/2017822320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0c40b37e0 msgr2=0x7fb0c41539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c3fff640 1 --2- 172.21.15.60:0/2017822320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0c406ec30 0x7fb0c4152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c8c37640 1 --2- 172.21.15.60:0/2017822320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0c40b37e0 0x7fb0c41539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c8c37640 1 -- 172.21.15.60:0/2017822320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0c406ec30 msgr2=0x7fb0c4152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c8c37640 1 --2- 172.21.15.60:0/2017822320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0c406ec30 0x7fb0c4152840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c8c37640 1 -- 172.21.15.60:0/2017822320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c4155560 con 0x7fb0c4153f10 2024-04-25T10:08:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c37fe640 1 --2- 172.21.15.60:0/2017822320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0c40b37e0 0x7fb0c41539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:11.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c8c37640 1 --2- 172.21.15.60:0/2017822320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c4153f10 0x7fb0c41638a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc007e60 tx=0x7fb0bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c0ff9640 1 -- 172.21.15.60:0/2017822320 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc014420 con 0x7fb0c4153f10 2024-04-25T10:08:11.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c0ff9640 1 -- 172.21.15.60:0/2017822320 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0bc019070 con 0x7fb0c4153f10 2024-04-25T10:08:11.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0ca69a640 1 -- 172.21.15.60:0/2017822320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c41544e0 con 0x7fb0c4153f10 2024-04-25T10:08:11.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c0ff9640 1 -- 172.21.15.60:0/2017822320 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc014820 con 0x7fb0c4153f10 2024-04-25T10:08:11.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0ca69a640 1 -- 172.21.15.60:0/2017822320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c41547a0 con 0x7fb0c4153f10 2024-04-25T10:08:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0ca69a640 1 -- 172.21.15.60:0/2017822320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c4074770 con 0x7fb0c4153f10 2024-04-25T10:08:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c0ff9640 1 -- 172.21.15.60:0/2017822320 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0bc0149c0 con 0x7fb0c4153f10 2024-04-25T10:08:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c0ff9640 1 --2- 172.21.15.60:0/2017822320 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb0b0066f60 0x7fb0b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c0ff9640 1 -- 172.21.15.60:0/2017822320 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0bc085790 con 0x7fb0c4153f10 2024-04-25T10:08:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.030+0000 7fb0c3fff640 1 --2- 172.21.15.60:0/2017822320 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb0b0066f60 0x7fb0b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.034+0000 7fb0c3fff640 1 --2- 172.21.15.60:0/2017822320 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb0b0066f60 0x7fb0b0069420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40af000 tx=0x7fb0b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.038+0000 7fb0c0ff9640 1 -- 172.21.15.60:0/2017822320 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0bc056830 con 0x7fb0c4153f10 2024-04-25T10:08:11.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.190+0000 7fb0ca69a640 1 -- 172.21.15.60:0/2017822320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0c4152d80 con 0x7fb0c4153f10 2024-04-25T10:08:11.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.194+0000 7fb0c0ff9640 1 -- 172.21.15.60:0/2017822320 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16690 (secure 0 0 0) 0x7fb0bc056130 con 0x7fb0c4153f10 2024-04-25T10:08:11.202 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:11.203 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":57,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:08:10.235933+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:08.538247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:09.233842+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:11.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.198+0000 7fb0ca69a640 1 -- 172.21.15.60:0/2017822320 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb0b0066f60 msgr2=0x7fb0b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.198+0000 7fb0ca69a640 1 --2- 172.21.15.60:0/2017822320 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb0b0066f60 0x7fb0b0069420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40af000 tx=0x7fb0b4008040 comp rx=0 tx=0).stop 2024-04-25T10:08:11.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.198+0000 7fb0ca69a640 1 -- 172.21.15.60:0/2017822320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c4153f10 msgr2=0x7fb0c41638a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.198+0000 7fb0ca69a640 1 --2- 172.21.15.60:0/2017822320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c4153f10 0x7fb0c41638a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc007e60 tx=0x7fb0bc00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.198+0000 7fb0ca69a640 1 -- 172.21.15.60:0/2017822320 shutdown_connections 2024-04-25T10:08:11.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.198+0000 7fb0ca69a640 1 --2- 172.21.15.60:0/2017822320 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb0b0066f60 0x7fb0b0069420 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.198+0000 7fb0ca69a640 1 --2- 172.21.15.60:0/2017822320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0c4153f10 0x7fb0c41638a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.198+0000 7fb0ca69a640 1 --2- 172.21.15.60:0/2017822320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0c40b37e0 0x7fb0c41539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.198+0000 7fb0ca69a640 1 --2- 172.21.15.60:0/2017822320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0c406ec30 0x7fb0c4152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.198+0000 7fb0ca69a640 1 -- 172.21.15.60:0/2017822320 >> 172.21.15.60:0/2017822320 conn(0x7fb0c40b12d0 msgr2=0x7fb0c40b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.198+0000 7fb0ca69a640 1 -- 172.21.15.60:0/2017822320 shutdown_connections 2024-04-25T10:08:11.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.198+0000 7fb0ca69a640 1 -- 172.21.15.60:0/2017822320 wait complete. 2024-04-25T10:08:11.220 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:11.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.322+0000 7f3c881fd640 1 Processor -- start 2024-04-25T10:08:11.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.322+0000 7f3c881fd640 1 -- start start 2024-04-25T10:08:11.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c881fd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3c8006e480 0x7f3c8006e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c881fd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c800cce50 0x7f3c800cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c881fd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8015dc10 con 0x7f3c80071b20 2024-04-25T10:08:11.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c881fd640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c8015def0 con 0x7f3c8006e480 2024-04-25T10:08:11.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c881fd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8015e1d0 con 0x7f3c800cce50 2024-04-25T10:08:11.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c85798640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3c8006e480 0x7f3c8006e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c85798640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3c8006e480 0x7f3c8006e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58680/0 (socket says 172.21.15.60:58680) 2024-04-25T10:08:11.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c85798640 1 -- 172.21.15.60:0/913221460 learned_addr learned my addr 172.21.15.60:0/913221460 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:11.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c8679a640 1 --2- 172.21.15.60:0/913221460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c800cce50 0x7f3c800cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c85798640 1 -- 172.21.15.60:0/913221460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c800cce50 msgr2=0x7f3c800cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c85798640 1 --2- 172.21.15.60:0/913221460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c800cce50 0x7f3c800cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c85798640 1 -- 172.21.15.60:0/913221460 >> v1:172.21.15.100:6789/0 conn(0x7f3c80071b20 legacy=0x7f3c80071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c85798640 1 -- 172.21.15.60:0/913221460 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8015e4b0 con 0x7f3c8006e480 2024-04-25T10:08:11.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c8679a640 1 --2- 172.21.15.60:0/913221460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c800cce50 0x7f3c800cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:11.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c85798640 1 --2- 172.21.15.60:0/913221460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c8006e480 0x7f3c8006e910 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3c70006fd0 tx=0x7f3c7001e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c8960330c0e893 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c84f97640 1 -- 172.21.15.60:0/913221460 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c700ad070 con 0x7f3c8006e480 2024-04-25T10:08:11.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c84f97640 1 -- 172.21.15.60:0/913221460 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c70004230 con 0x7f3c8006e480 2024-04-25T10:08:11.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.326+0000 7f3c84f97640 1 -- 172.21.15.60:0/913221460 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c700a7640 con 0x7f3c8006e480 2024-04-25T10:08:11.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 -- 172.21.15.60:0/913221460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c8006e480 msgr2=0x7f3c8006e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 --2- 172.21.15.60:0/913221460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c8006e480 0x7f3c8006e910 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3c70006fd0 tx=0x7f3c7001e920 comp rx=0 tx=0).stop 2024-04-25T10:08:11.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 -- 172.21.15.60:0/913221460 shutdown_connections 2024-04-25T10:08:11.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 --2- 172.21.15.60:0/913221460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c8006e480 0x7f3c8006e910 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 --2- 172.21.15.60:0/913221460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c800cce50 0x7f3c800cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 -- 172.21.15.60:0/913221460 >> 172.21.15.60:0/913221460 conn(0x7f3c800b12d0 msgr2=0x7f3c800b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 -- 172.21.15.60:0/913221460 shutdown_connections 2024-04-25T10:08:11.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 -- 172.21.15.60:0/913221460 wait complete. 2024-04-25T10:08:11.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 Processor -- start 2024-04-25T10:08:11.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 -- start start 2024-04-25T10:08:11.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c8006e480 0x7f3c80156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c80071b20 0x7f3c800b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c800cce50 0x7f3c80157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c800cd980 con 0x7f3c80071b20 2024-04-25T10:08:11.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c800cd800 con 0x7f3c8006e480 2024-04-25T10:08:11.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c800cdb00 con 0x7f3c800cce50 2024-04-25T10:08:11.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c85798640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c80071b20 0x7f3c800b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c85798640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c80071b20 0x7f3c800b0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58682/0 (socket says 172.21.15.60:58682) 2024-04-25T10:08:11.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c85798640 1 -- 172.21.15.60:0/4262941270 learned_addr learned my addr 172.21.15.60:0/4262941270 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:11.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c85f99640 1 --2- 172.21.15.60:0/4262941270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c8006e480 0x7f3c80156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c8679a640 1 --2- 172.21.15.60:0/4262941270 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c800cce50 0x7f3c80157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c85798640 1 -- 172.21.15.60:0/4262941270 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c800cce50 msgr2=0x7f3c80157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c85798640 1 --2- 172.21.15.60:0/4262941270 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c800cce50 0x7f3c80157c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c85798640 1 -- 172.21.15.60:0/4262941270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c8006e480 msgr2=0x7f3c80156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c85798640 1 --2- 172.21.15.60:0/4262941270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c8006e480 0x7f3c80156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c85798640 1 -- 172.21.15.60:0/4262941270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8012e080 con 0x7f3c80071b20 2024-04-25T10:08:11.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c85f99640 1 --2- 172.21.15.60:0/4262941270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c8006e480 0x7f3c80156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:11.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c85798640 1 --2- 172.21.15.60:0/4262941270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c80071b20 0x7f3c800b0490 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3c70006fd0 tx=0x7f3c7001b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c767fc640 1 -- 172.21.15.60:0/4262941270 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c700c0070 con 0x7f3c80071b20 2024-04-25T10:08:11.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c767fc640 1 -- 172.21.15.60:0/4262941270 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c70004030 con 0x7f3c80071b20 2024-04-25T10:08:11.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 -- 172.21.15.60:0/4262941270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8012e360 con 0x7f3c80071b20 2024-04-25T10:08:11.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c767fc640 1 -- 172.21.15.60:0/4262941270 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c700ad040 con 0x7f3c80071b20 2024-04-25T10:08:11.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.330+0000 7f3c881fd640 1 -- 172.21.15.60:0/4262941270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8012e920 con 0x7f3c80071b20 2024-04-25T10:08:11.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.334+0000 7f3c881fd640 1 -- 172.21.15.60:0/4262941270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c800aef40 con 0x7f3c80071b20 2024-04-25T10:08:11.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.334+0000 7f3c767fc640 1 -- 172.21.15.60:0/4262941270 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c700c3070 con 0x7f3c80071b20 2024-04-25T10:08:11.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.334+0000 7f3c767fc640 1 --2- 172.21.15.60:0/4262941270 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c64066f90 0x7f3c64069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.334+0000 7f3c767fc640 1 -- 172.21.15.60:0/4262941270 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c70120f30 con 0x7f3c80071b20 2024-04-25T10:08:11.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.334+0000 7f3c85f99640 1 --2- 172.21.15.60:0/4262941270 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c64066f90 0x7f3c64069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.334+0000 7f3c85f99640 1 --2- 172.21.15.60:0/4262941270 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c64066f90 0x7f3c64069450 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c003e40 tx=0x7f3c7c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.338+0000 7f3c767fc640 1 -- 172.21.15.60:0/4262941270 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c700185a0 con 0x7f3c80071b20 2024-04-25T10:08:11.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.494+0000 7f3c881fd640 1 -- 172.21.15.60:0/4262941270 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c80143030 con 0x7f3c80071b20 2024-04-25T10:08:11.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.498+0000 7f3c767fc640 1 -- 172.21.15.60:0/4262941270 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16690 (secure 0 0 0) 0x7f3c700f20f0 con 0x7f3c80071b20 2024-04-25T10:08:11.505 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:11.506 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":57,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:08:10.235933+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:08.538247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:09.233842+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:11.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.502+0000 7f3c881fd640 1 -- 172.21.15.60:0/4262941270 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c64066f90 msgr2=0x7f3c64069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.502+0000 7f3c881fd640 1 --2- 172.21.15.60:0/4262941270 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c64066f90 0x7f3c64069450 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c003e40 tx=0x7f3c7c009040 comp rx=0 tx=0).stop 2024-04-25T10:08:11.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.502+0000 7f3c881fd640 1 -- 172.21.15.60:0/4262941270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c80071b20 msgr2=0x7f3c800b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.502+0000 7f3c881fd640 1 --2- 172.21.15.60:0/4262941270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c80071b20 0x7f3c800b0490 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3c70006fd0 tx=0x7f3c7001b6c0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.502+0000 7f3c881fd640 1 -- 172.21.15.60:0/4262941270 shutdown_connections 2024-04-25T10:08:11.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.502+0000 7f3c881fd640 1 --2- 172.21.15.60:0/4262941270 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c64066f90 0x7f3c64069450 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.502+0000 7f3c881fd640 1 --2- 172.21.15.60:0/4262941270 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c800cce50 0x7f3c80157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.502+0000 7f3c881fd640 1 --2- 172.21.15.60:0/4262941270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c80071b20 0x7f3c800b0490 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.502+0000 7f3c881fd640 1 --2- 172.21.15.60:0/4262941270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c8006e480 0x7f3c80156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.502+0000 7f3c881fd640 1 -- 172.21.15.60:0/4262941270 >> 172.21.15.60:0/4262941270 conn(0x7f3c800b12d0 msgr2=0x7f3c800af200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.502+0000 7f3c881fd640 1 -- 172.21.15.60:0/4262941270 shutdown_connections 2024-04-25T10:08:11.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.502+0000 7f3c881fd640 1 -- 172.21.15.60:0/4262941270 wait complete. 2024-04-25T10:08:11.525 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:08:11.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.638+0000 7f81e03d5640 1 Processor -- start 2024-04-25T10:08:11.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.638+0000 7f81e03d5640 1 -- start start 2024-04-25T10:08:11.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.638+0000 7f81e03d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f81d80cf8a0 0x7f81d815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.638+0000 7f81e03d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d80cd3d0 0x7f81d815be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.638+0000 7f81e03d5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d80cd890 con 0x7f81d815dd30 2024-04-25T10:08:11.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.638+0000 7f81e03d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d80ce4a0 con 0x7f81d80cf8a0 2024-04-25T10:08:11.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.638+0000 7f81e03d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80cdb70 con 0x7f81d80cd3d0 2024-04-25T10:08:11.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81de171640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f81d815dd30 0x7f81d80cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41480/0 (socket says 172.21.15.60:41480) 2024-04-25T10:08:11.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81de171640 1 -- 172.21.15.60:0/1694853940 learned_addr learned my addr 172.21.15.60:0/1694853940 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:11.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81de972640 1 --2- 172.21.15.60:0/1694853940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d80cd3d0 0x7f81d815be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd970640 1 --2- 172.21.15.60:0/1694853940 >> v2:172.21.15.100:3300/0 conn(0x7f81d80cf8a0 0x7f81d815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 -- 172.21.15.60:0/1694853940 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999583729 0 0) 0x7f81d80cd890 con 0x7f81d815dd30 2024-04-25T10:08:11.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 -- 172.21.15.60:0/1694853940 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81c4003530 con 0x7f81d815dd30 2024-04-25T10:08:11.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 -- 172.21.15.60:0/1694853940 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1917971521 0 0) 0x7f81c4003530 con 0x7f81d815dd30 2024-04-25T10:08:11.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 -- 172.21.15.60:0/1694853940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d80cd3d0 msgr2=0x7f81d815be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 --2- 172.21.15.60:0/1694853940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d80cd3d0 0x7f81d815be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 -- 172.21.15.60:0/1694853940 >> v2:172.21.15.100:3300/0 conn(0x7f81d80cf8a0 msgr2=0x7f81d815ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 --2- 172.21.15.60:0/1694853940 >> v2:172.21.15.100:3300/0 conn(0x7f81d80cf8a0 0x7f81d815ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 -- 172.21.15.60:0/1694853940 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d80cde50 con 0x7f81d815dd30 2024-04-25T10:08:11.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 -- 172.21.15.60:0/1694853940 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f81d4001680 con 0x7f81d815dd30 2024-04-25T10:08:11.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 -- 172.21.15.60:0/1694853940 >> v1:172.21.15.60:6789/0 conn(0x7f81d815dd30 legacy=0x7f81d80cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 --2- 172.21.15.60:0/1694853940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c40039c0 0x7f81c4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 --2- 172.21.15.60:0/1694853940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d80cd3d0 0x7f81c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 --2- 172.21.15.60:0/1694853940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81c4004d50 0x7f81c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 -- 172.21.15.60:0/1694853940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80ce4a0 con 0x7f81c40039c0 2024-04-25T10:08:11.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 -- 172.21.15.60:0/1694853940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80cdb70 con 0x7f81c4004d50 2024-04-25T10:08:11.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 -- 172.21.15.60:0/1694853940 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c400ba20 con 0x7f81d80cd3d0 2024-04-25T10:08:11.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81de171640 1 --2- 172.21.15.60:0/1694853940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c40039c0 0x7f81c4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd970640 1 --2- 172.21.15.60:0/1694853940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d80cd3d0 0x7f81c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81de171640 1 -- 172.21.15.60:0/1694853940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d80cd3d0 msgr2=0x7f81c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81de171640 1 --2- 172.21.15.60:0/1694853940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d80cd3d0 0x7f81c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81de171640 1 -- 172.21.15.60:0/1694853940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81c4004d50 msgr2=0x7f81c40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81de972640 1 --2- 172.21.15.60:0/1694853940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81c4004d50 0x7f81c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81de171640 1 --2- 172.21.15.60:0/1694853940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81c4004d50 0x7f81c40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81de171640 1 -- 172.21.15.60:0/1694853940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c4003530 con 0x7f81c40039c0 2024-04-25T10:08:11.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd970640 1 --2- 172.21.15.60:0/1694853940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d80cd3d0 0x7f81c4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:11.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81de171640 1 --2- 172.21.15.60:0/1694853940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c40039c0 0x7f81c4003de0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f81d400c270 tx=0x7f81d401cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bf791b1855a311 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 -- 172.21.15.60:0/1694853940 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d4007d40 con 0x7f81c40039c0 2024-04-25T10:08:11.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 -- 172.21.15.60:0/1694853940 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d40a9020 con 0x7f81c40039c0 2024-04-25T10:08:11.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.642+0000 7f81dd16f640 1 -- 172.21.15.60:0/1694853940 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d40a5b90 con 0x7f81c40039c0 2024-04-25T10:08:11.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81e03d5640 1 -- 172.21.15.60:0/1694853940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c40039c0 msgr2=0x7f81c4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81e03d5640 1 --2- 172.21.15.60:0/1694853940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c40039c0 0x7f81c4003de0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f81d400c270 tx=0x7f81d401cc70 comp rx=0 tx=0).stop 2024-04-25T10:08:11.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81de972640 1 -- 172.21.15.60:0/1694853940 reap_dead start 2024-04-25T10:08:11.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81e03d5640 1 -- 172.21.15.60:0/1694853940 shutdown_connections 2024-04-25T10:08:11.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81e03d5640 1 -- 172.21.15.60:0/1694853940 >> 172.21.15.60:0/1694853940 conn(0x7f81d80670b0 msgr2=0x7f81d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81e03d5640 1 -- 172.21.15.60:0/1694853940 shutdown_connections 2024-04-25T10:08:11.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81e03d5640 1 -- 172.21.15.60:0/1694853940 wait complete. 2024-04-25T10:08:11.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81e03d5640 1 Processor -- start 2024-04-25T10:08:11.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81e03d5640 1 -- start start 2024-04-25T10:08:11.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81e03d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c40039c0 0x7f81d81eb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81e03d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d80a8490 0x7f81d8157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81e03d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81d80ac560 0x7f81d80ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81e03d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d815c9c0 con 0x7f81c40039c0 2024-04-25T10:08:11.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81e03d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d815c840 con 0x7f81d80ac560 2024-04-25T10:08:11.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81e03d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d815cb40 con 0x7f81d80a8490 2024-04-25T10:08:11.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81de171640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c40039c0 0x7f81d81eb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81de171640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c40039c0 0x7f81d81eb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58700/0 (socket says 172.21.15.60:58700) 2024-04-25T10:08:11.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81de171640 1 -- 172.21.15.60:0/1399335122 learned_addr learned my addr 172.21.15.60:0/1399335122 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:11.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81dd970640 1 --2- 172.21.15.60:0/1399335122 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d80a8490 0x7f81d8157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81de972640 1 --2- 172.21.15.60:0/1399335122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81d80ac560 0x7f81d80ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81de171640 1 -- 172.21.15.60:0/1399335122 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d80a8490 msgr2=0x7f81d8157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81de171640 1 --2- 172.21.15.60:0/1399335122 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d80a8490 0x7f81d8157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81de171640 1 -- 172.21.15.60:0/1399335122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81d80ac560 msgr2=0x7f81d80ac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81de171640 1 --2- 172.21.15.60:0/1399335122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81d80ac560 0x7f81d80ac9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81de171640 1 -- 172.21.15.60:0/1399335122 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d80a8a30 con 0x7f81c40039c0 2024-04-25T10:08:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81de972640 1 --2- 172.21.15.60:0/1399335122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81d80ac560 0x7f81d80ac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81de171640 1 --2- 172.21.15.60:0/1399335122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c40039c0 0x7f81d81eb810 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f81d40ab040 tx=0x7f81c4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81ceffd640 1 -- 172.21.15.60:0/1399335122 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d4001c80 con 0x7f81c40039c0 2024-04-25T10:08:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81ceffd640 1 -- 172.21.15.60:0/1399335122 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d40b3030 con 0x7f81c40039c0 2024-04-25T10:08:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81e03d5640 1 -- 172.21.15.60:0/1399335122 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d80a8d10 con 0x7f81c40039c0 2024-04-25T10:08:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81ceffd640 1 -- 172.21.15.60:0/1399335122 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d40047f0 con 0x7f81c40039c0 2024-04-25T10:08:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.646+0000 7f81e03d5640 1 -- 172.21.15.60:0/1399335122 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d80a92d0 con 0x7f81c40039c0 2024-04-25T10:08:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.650+0000 7f81e03d5640 1 -- 172.21.15.60:0/1399335122 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d8074770 con 0x7f81c40039c0 2024-04-25T10:08:11.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.650+0000 7f81ceffd640 1 -- 172.21.15.60:0/1399335122 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f81d4004990 con 0x7f81c40039c0 2024-04-25T10:08:11.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.650+0000 7f81ceffd640 1 --2- 172.21.15.60:0/1399335122 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81bc066f60 0x7f81bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.650+0000 7f81ceffd640 1 -- 172.21.15.60:0/1399335122 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81d411e7f0 con 0x7f81c40039c0 2024-04-25T10:08:11.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.650+0000 7f81dd970640 1 --2- 172.21.15.60:0/1399335122 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81bc066f60 0x7f81bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.650+0000 7f81dd970640 1 --2- 172.21.15.60:0/1399335122 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81bc066f60 0x7f81bc069420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f81d81f9850 tx=0x7f81c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.654+0000 7f81ceffd640 1 -- 172.21.15.60:0/1399335122 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f81d40169b0 con 0x7f81c40039c0 2024-04-25T10:08:11.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.846+0000 7f81e03d5640 1 -- 172.21.15.60:0/1399335122 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f81d81591c0 con 0x7f81c40039c0 2024-04-25T10:08:11.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.882+0000 7f81ceffd640 1 -- 172.21.15.60:0/1399335122 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v27) v1 ==== 118+0+0 (secure 0 0 0) 0x7f81d40ef800 con 0x7f81c40039c0 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.886+0000 7f81e03d5640 1 -- 172.21.15.60:0/1399335122 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81bc066f60 msgr2=0x7f81bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.886+0000 7f81e03d5640 1 --2- 172.21.15.60:0/1399335122 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81bc066f60 0x7f81bc069420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f81d81f9850 tx=0x7f81c8008040 comp rx=0 tx=0).stop 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.886+0000 7f81e03d5640 1 -- 172.21.15.60:0/1399335122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c40039c0 msgr2=0x7f81d81eb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.886+0000 7f81e03d5640 1 --2- 172.21.15.60:0/1399335122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c40039c0 0x7f81d81eb810 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f81d40ab040 tx=0x7f81c4005840 comp rx=0 tx=0).stop 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.886+0000 7f81e03d5640 1 -- 172.21.15.60:0/1399335122 shutdown_connections 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.886+0000 7f81e03d5640 1 --2- 172.21.15.60:0/1399335122 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81bc066f60 0x7f81bc069420 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.886+0000 7f81e03d5640 1 --2- 172.21.15.60:0/1399335122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81d80ac560 0x7f81d80ac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.886+0000 7f81e03d5640 1 --2- 172.21.15.60:0/1399335122 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d80a8490 0x7f81d8157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.886+0000 7f81e03d5640 1 --2- 172.21.15.60:0/1399335122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c40039c0 0x7f81d81eb810 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.886+0000 7f81e03d5640 1 -- 172.21.15.60:0/1399335122 >> 172.21.15.60:0/1399335122 conn(0x7f81d80670b0 msgr2=0x7f81d806db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.886+0000 7f81e03d5640 1 -- 172.21.15.60:0/1399335122 shutdown_connections 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:11.886+0000 7f81e03d5640 1 -- 172.21.15.60:0/1399335122 wait complete. 2024-04-25T10:08:11.911 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:08:11.911 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:08:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f19137d7640 1 Processor -- start 2024-04-25T10:08:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f19137d7640 1 -- start start 2024-04-25T10:08:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f19137d7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f190c15bb00 0x7f190c09aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f19137d7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f190c0b1060 0x7f190c099a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f19137d7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190c0cc210 con 0x7f190c0b14f0 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f19137d7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f190c15bee0 con 0x7f190c15bb00 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f19137d7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f190c0d0710 con 0x7f190c0b1060 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f1911573640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f190c15bb00 0x7f190c09aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f1911d74640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f190c0b14f0 0x7f190c0ccc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41486/0 (socket says 172.21.15.60:41486) 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f1911d74640 1 -- 172.21.15.60:0/4263502304 learned_addr learned my addr 172.21.15.60:0/4263502304 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f1903fff640 1 -- 172.21.15.60:0/4263502304 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589861943 0 0) 0x7f19040014a0 con 0x7f190c0b14f0 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f1903fff640 1 -- 172.21.15.60:0/4263502304 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18f8003530 con 0x7f190c0b14f0 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f1910d72640 1 --2- 172.21.15.60:0/4263502304 >> v2:172.21.15.100:3300/0 conn(0x7f190c0b1060 0x7f190c099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f1911573640 1 -- 172.21.15.60:0/4263502304 >> v2:172.21.15.100:3300/0 conn(0x7f190c0b1060 msgr2=0x7f190c099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f1911573640 1 --2- 172.21.15.60:0/4263502304 >> v2:172.21.15.100:3300/0 conn(0x7f190c0b1060 0x7f190c099a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f1911573640 1 -- 172.21.15.60:0/4263502304 >> v1:172.21.15.60:6789/0 conn(0x7f190c0b14f0 legacy=0x7f190c0ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.022+0000 7f1911573640 1 -- 172.21.15.60:0/4263502304 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c0cc490 con 0x7f190c15bb00 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f1911573640 1 --2- 172.21.15.60:0/4263502304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f190c15bb00 0x7f190c09aa70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f190c09afb0 tx=0x7f190801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5c22ff048c6077 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f1903fff640 1 -- 172.21.15.60:0/4263502304 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19080a7070 con 0x7f190c15bb00 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f1903fff640 1 -- 172.21.15.60:0/4263502304 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19080b0070 con 0x7f190c15bb00 2024-04-25T10:08:12.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f1903fff640 1 -- 172.21.15.60:0/4263502304 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19080ab870 con 0x7f190c15bb00 2024-04-25T10:08:12.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f19137d7640 1 -- 172.21.15.60:0/4263502304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f190c15bb00 msgr2=0x7f190c09aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f19137d7640 1 --2- 172.21.15.60:0/4263502304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f190c15bb00 0x7f190c09aa70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f190c09afb0 tx=0x7f190801c450 comp rx=0 tx=0).stop 2024-04-25T10:08:12.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f19137d7640 1 -- 172.21.15.60:0/4263502304 shutdown_connections 2024-04-25T10:08:12.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f19137d7640 1 --2- 172.21.15.60:0/4263502304 >> v2:172.21.15.100:3300/0 conn(0x7f190c0b1060 0x7f190c099a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f19137d7640 1 --2- 172.21.15.60:0/4263502304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f190c15bb00 0x7f190c09aa70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f19137d7640 1 -- 172.21.15.60:0/4263502304 >> 172.21.15.60:0/4263502304 conn(0x7f190c0b31f0 msgr2=0x7f190c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:12.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f19137d7640 1 -- 172.21.15.60:0/4263502304 shutdown_connections 2024-04-25T10:08:12.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f19137d7640 1 -- 172.21.15.60:0/4263502304 wait complete. 2024-04-25T10:08:12.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f19137d7640 1 Processor -- start 2024-04-25T10:08:12.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f19137d7640 1 -- start start 2024-04-25T10:08:12.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f19137d7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f190c0b1060 0x7f190c06d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f19137d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f190c0b14f0 0x7f190c164aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f19137d7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c15bb00 0x7f190c13a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f19137d7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c0b0be0 con 0x7f190c0b14f0 2024-04-25T10:08:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f19137d7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c09a380 con 0x7f190c15bb00 2024-04-25T10:08:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f19137d7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190c0b0d60 con 0x7f190c0b1060 2024-04-25T10:08:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f1910d72640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f190c0b14f0 0x7f190c164aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f1910d72640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f190c0b14f0 0x7f190c164aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58710/0 (socket says 172.21.15.60:58710) 2024-04-25T10:08:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f1910d72640 1 -- 172.21.15.60:0/3873951516 learned_addr learned my addr 172.21.15.60:0/3873951516 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f1911573640 1 --2- 172.21.15.60:0/3873951516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f190c0b1060 0x7f190c06d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f1910d72640 1 -- 172.21.15.60:0/3873951516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f190c0b1060 msgr2=0x7f190c06d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f1911d74640 1 --2- 172.21.15.60:0/3873951516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c15bb00 0x7f190c13a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f1910d72640 1 --2- 172.21.15.60:0/3873951516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f190c0b1060 0x7f190c06d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f1910d72640 1 -- 172.21.15.60:0/3873951516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c15bb00 msgr2=0x7f190c13a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f1910d72640 1 --2- 172.21.15.60:0/3873951516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c15bb00 0x7f190c13a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f1910d72640 1 -- 172.21.15.60:0/3873951516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c165210 con 0x7f190c0b14f0 2024-04-25T10:08:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.026+0000 7f1910d72640 1 --2- 172.21.15.60:0/3873951516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f190c0b14f0 0x7f190c164aa0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c7f0 tx=0x7f18fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.030+0000 7f1901ffb640 1 -- 172.21.15.60:0/3873951516 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc015070 con 0x7f190c0b14f0 2024-04-25T10:08:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.030+0000 7f1901ffb640 1 -- 172.21.15.60:0/3873951516 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc007820 con 0x7f190c0b14f0 2024-04-25T10:08:12.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.030+0000 7f19137d7640 1 -- 172.21.15.60:0/3873951516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190c1654f0 con 0x7f190c0b14f0 2024-04-25T10:08:12.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.030+0000 7f1901ffb640 1 -- 172.21.15.60:0/3873951516 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc007c10 con 0x7f190c0b14f0 2024-04-25T10:08:12.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.030+0000 7f19137d7640 1 -- 172.21.15.60:0/3873951516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190c165ab0 con 0x7f190c0b14f0 2024-04-25T10:08:12.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.030+0000 7f19137d7640 1 -- 172.21.15.60:0/3873951516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190c074770 con 0x7f190c0b14f0 2024-04-25T10:08:12.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.030+0000 7f1901ffb640 1 -- 172.21.15.60:0/3873951516 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18fc018070 con 0x7f190c0b14f0 2024-04-25T10:08:12.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.030+0000 7f1901ffb640 1 --2- 172.21.15.60:0/3873951516 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f18f0066f60 0x7f18f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.030+0000 7f1901ffb640 1 -- 172.21.15.60:0/3873951516 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18fc085530 con 0x7f190c0b14f0 2024-04-25T10:08:12.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.030+0000 7f1911573640 1 --2- 172.21.15.60:0/3873951516 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f18f0066f60 0x7f18f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.030+0000 7f1911573640 1 --2- 172.21.15.60:0/3873951516 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f18f0066f60 0x7f18f0069420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f19080af000 tx=0x7f1908008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:12.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.034+0000 7f1901ffb640 1 -- 172.21.15.60:0/3873951516 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18fc056590 con 0x7f190c0b14f0 2024-04-25T10:08:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.226+0000 7f19137d7640 1 -- 172.21.15.60:0/3873951516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f190c0b3f70 con 0x7f190c0b14f0 2024-04-25T10:08:12.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.894+0000 7f1901ffb640 1 -- 172.21.15.60:0/3873951516 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f18fc055e90 con 0x7f190c0b14f0 2024-04-25T10:08:12.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.898+0000 7f19137d7640 1 -- 172.21.15.60:0/3873951516 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f18f0066f60 msgr2=0x7f18f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.898+0000 7f19137d7640 1 --2- 172.21.15.60:0/3873951516 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f18f0066f60 0x7f18f0069420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f19080af000 tx=0x7f1908008040 comp rx=0 tx=0).stop 2024-04-25T10:08:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.898+0000 7f19137d7640 1 -- 172.21.15.60:0/3873951516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f190c0b14f0 msgr2=0x7f190c164aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.898+0000 7f19137d7640 1 --2- 172.21.15.60:0/3873951516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f190c0b14f0 0x7f190c164aa0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c7f0 tx=0x7f18fc00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.898+0000 7f19137d7640 1 -- 172.21.15.60:0/3873951516 shutdown_connections 2024-04-25T10:08:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.898+0000 7f19137d7640 1 --2- 172.21.15.60:0/3873951516 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f18f0066f60 0x7f18f0069420 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.898+0000 7f19137d7640 1 --2- 172.21.15.60:0/3873951516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f190c15bb00 0x7f190c13a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.898+0000 7f19137d7640 1 --2- 172.21.15.60:0/3873951516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f190c0b14f0 0x7f190c164aa0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.898+0000 7f19137d7640 1 --2- 172.21.15.60:0/3873951516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f190c0b1060 0x7f190c06d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.898+0000 7f19137d7640 1 -- 172.21.15.60:0/3873951516 >> 172.21.15.60:0/3873951516 conn(0x7f190c0b31f0 msgr2=0x7f190c0ce260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.898+0000 7f19137d7640 1 -- 172.21.15.60:0/3873951516 shutdown_connections 2024-04-25T10:08:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:12.898+0000 7f19137d7640 1 -- 172.21.15.60:0/3873951516 wait complete. 2024-04-25T10:08:12.926 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:08:12.926 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:13.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.046+0000 7f97431e0640 1 Processor -- start 2024-04-25T10:08:13.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.046+0000 7f97431e0640 1 -- start start 2024-04-25T10:08:13.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.046+0000 7f97431e0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f973c06ec30 0x7f973c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.046+0000 7f97431e0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c06f130 con 0x7f973c0b37e0 2024-04-25T10:08:13.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.046+0000 7f97431e0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c06f410 con 0x7f973c15a7a0 2024-04-25T10:08:13.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.046+0000 7f97431e0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f973c0cdaa0 con 0x7f973c06ec30 2024-04-25T10:08:13.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f974177d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f973c0b37e0 0x7f973c06dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41498/0 (socket says 172.21.15.60:41498) 2024-04-25T10:08:13.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f974177d640 1 -- 172.21.15.60:0/4230333573 learned_addr learned my addr 172.21.15.60:0/4230333573 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:13.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973bfff640 1 --2- 172.21.15.60:0/4230333573 >> v2:172.21.15.100:3300/0 conn(0x7f973c06ec30 0x7f973c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 -- 172.21.15.60:0/4230333573 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272440413 0 0) 0x7f973c06f130 con 0x7f973c0b37e0 2024-04-25T10:08:13.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 -- 172.21.15.60:0/4230333573 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f972c003530 con 0x7f973c0b37e0 2024-04-25T10:08:13.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 -- 172.21.15.60:0/4230333573 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1234689239 0 0) 0x7f972c003530 con 0x7f973c0b37e0 2024-04-25T10:08:13.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 -- 172.21.15.60:0/4230333573 >> v2:172.21.15.100:3300/0 conn(0x7f973c06ec30 msgr2=0x7f973c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 --2- 172.21.15.60:0/4230333573 >> v2:172.21.15.100:3300/0 conn(0x7f973c06ec30 0x7f973c0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 -- 172.21.15.60:0/4230333573 >> v1:172.21.15.100:6789/0 conn(0x7f973c15a7a0 legacy=0x7f973c071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 -- 172.21.15.60:0/4230333573 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c06d540 con 0x7f973c0b37e0 2024-04-25T10:08:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 -- 172.21.15.60:0/4230333573 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f9734001a00 con 0x7f973c0b37e0 2024-04-25T10:08:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 -- 172.21.15.60:0/4230333573 >> v1:172.21.15.60:6789/0 conn(0x7f973c0b37e0 legacy=0x7f973c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 --2- 172.21.15.60:0/4230333573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f972c0039f0 0x7f972c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 --2- 172.21.15.60:0/4230333573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972c004810 0x7f972c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 --2- 172.21.15.60:0/4230333573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f972c0051d0 0x7f972c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 -- 172.21.15.60:0/4230333573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c00c0a0 con 0x7f972c004810 2024-04-25T10:08:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 -- 172.21.15.60:0/4230333573 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c00c220 con 0x7f972c0051d0 2024-04-25T10:08:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 -- 172.21.15.60:0/4230333573 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972c00e630 con 0x7f972c0039f0 2024-04-25T10:08:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973bfff640 1 --2- 172.21.15.60:0/4230333573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972c004810 0x7f972c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f974177d640 1 --2- 172.21.15.60:0/4230333573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f972c0051d0 0x7f972c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f9740f7c640 1 --2- 172.21.15.60:0/4230333573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f972c0039f0 0x7f972c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973bfff640 1 -- 172.21.15.60:0/4230333573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f972c0039f0 msgr2=0x7f972c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973bfff640 1 --2- 172.21.15.60:0/4230333573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f972c0039f0 0x7f972c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973bfff640 1 -- 172.21.15.60:0/4230333573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f972c0051d0 msgr2=0x7f972c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973bfff640 1 --2- 172.21.15.60:0/4230333573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f972c0051d0 0x7f972c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f974177d640 1 -- 172.21.15.60:0/4230333573 reap_dead start 2024-04-25T10:08:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973bfff640 1 -- 172.21.15.60:0/4230333573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c003530 con 0x7f972c004810 2024-04-25T10:08:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f9740f7c640 1 --2- 172.21.15.60:0/4230333573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f972c0039f0 0x7f972c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:13.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973bfff640 1 --2- 172.21.15.60:0/4230333573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972c004810 0x7f972c004c90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9728009ab0 tx=0x7f972801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f77718a97ca8e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 -- 172.21.15.60:0/4230333573 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97280b2070 con 0x7f972c004810 2024-04-25T10:08:13.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 -- 172.21.15.60:0/4230333573 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97280a5530 con 0x7f972c004810 2024-04-25T10:08:13.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.050+0000 7f973b7fe640 1 -- 172.21.15.60:0/4230333573 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97280096b0 con 0x7f972c004810 2024-04-25T10:08:13.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 -- 172.21.15.60:0/4230333573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972c004810 msgr2=0x7f972c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 --2- 172.21.15.60:0/4230333573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972c004810 0x7f972c004c90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9728009ab0 tx=0x7f972801ca50 comp rx=0 tx=0).stop 2024-04-25T10:08:13.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 -- 172.21.15.60:0/4230333573 shutdown_connections 2024-04-25T10:08:13.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 --2- 172.21.15.60:0/4230333573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972c004810 0x7f972c004c90 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 -- 172.21.15.60:0/4230333573 >> 172.21.15.60:0/4230333573 conn(0x7f973c0b12d0 msgr2=0x7f973c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:13.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 -- 172.21.15.60:0/4230333573 shutdown_connections 2024-04-25T10:08:13.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 -- 172.21.15.60:0/4230333573 wait complete. 2024-04-25T10:08:13.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 Processor -- start 2024-04-25T10:08:13.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 -- start start 2024-04-25T10:08:13.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972c004810 0x7f973c153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f973c159c00 0x7f973c1f6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973c153a80 0x7f973c1f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c06d8b0 con 0x7f972c004810 2024-04-25T10:08:13.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c06d2f0 con 0x7f973c153a80 2024-04-25T10:08:13.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0cd560 con 0x7f973c159c00 2024-04-25T10:08:13.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f9740f7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972c004810 0x7f973c153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f9740f7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972c004810 0x7f973c153540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58724/0 (socket says 172.21.15.60:58724) 2024-04-25T10:08:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f9740f7c640 1 -- 172.21.15.60:0/2426428588 learned_addr learned my addr 172.21.15.60:0/2426428588 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f973bfff640 1 --2- 172.21.15.60:0/2426428588 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f973c159c00 0x7f973c1f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f974177d640 1 --2- 172.21.15.60:0/2426428588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973c153a80 0x7f973c1f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f9740f7c640 1 -- 172.21.15.60:0/2426428588 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f973c159c00 msgr2=0x7f973c1f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f9740f7c640 1 --2- 172.21.15.60:0/2426428588 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f973c159c00 0x7f973c1f6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f9740f7c640 1 -- 172.21.15.60:0/2426428588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973c153a80 msgr2=0x7f973c1f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f9740f7c640 1 --2- 172.21.15.60:0/2426428588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973c153a80 0x7f973c1f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f9740f7c640 1 -- 172.21.15.60:0/2426428588 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c154000 con 0x7f972c004810 2024-04-25T10:08:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f9740f7c640 1 --2- 172.21.15.60:0/2426428588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972c004810 0x7f973c153540 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f973000d4a0 tx=0x7f973000d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97397fa640 1 -- 172.21.15.60:0/2426428588 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730019070 con 0x7f972c004810 2024-04-25T10:08:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97397fa640 1 -- 172.21.15.60:0/2426428588 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97300054a0 con 0x7f972c004810 2024-04-25T10:08:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 -- 172.21.15.60:0/2426428588 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c1542e0 con 0x7f972c004810 2024-04-25T10:08:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97397fa640 1 -- 172.21.15.60:0/2426428588 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730007270 con 0x7f972c004810 2024-04-25T10:08:13.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.054+0000 7f97431e0640 1 -- 172.21.15.60:0/2426428588 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c15b1f0 con 0x7f972c004810 2024-04-25T10:08:13.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.058+0000 7f97431e0640 1 -- 172.21.15.60:0/2426428588 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c074770 con 0x7f972c004810 2024-04-25T10:08:13.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.058+0000 7f97397fa640 1 -- 172.21.15.60:0/2426428588 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f973001c070 con 0x7f972c004810 2024-04-25T10:08:13.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.058+0000 7f97397fa640 1 --2- 172.21.15.60:0/2426428588 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9720066f90 0x7f9720069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.058+0000 7f97397fa640 1 -- 172.21.15.60:0/2426428588 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97300847a0 con 0x7f972c004810 2024-04-25T10:08:13.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.058+0000 7f973bfff640 1 --2- 172.21.15.60:0/2426428588 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9720066f90 0x7f9720069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.058+0000 7f973bfff640 1 --2- 172.21.15.60:0/2426428588 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9720066f90 0x7f9720069450 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f97280af000 tx=0x7f9728008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.062+0000 7f97397fa640 1 -- 172.21.15.60:0/2426428588 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9730055870 con 0x7f972c004810 2024-04-25T10:08:13.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.254+0000 7f97431e0640 1 -- 172.21.15.60:0/2426428588 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f973c0b3db0 con 0x7f972c004810 2024-04-25T10:08:13.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.258+0000 7f97397fa640 1 -- 172.21.15.60:0/2426428588 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f9730055170 con 0x7f972c004810 2024-04-25T10:08:13.265 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:13.265 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":28,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/380273391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":380273391},{"type":"v1","addr":"172.21.15.100:6836","nonce":380273391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5635,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2965977166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2965977166},{"type":"v1","addr":"172.21.15.100:6837","nonce":2965977166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1105028629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1105028629},{"type":"v1","addr":"172.21.15.60:6834","nonce":1105028629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:10.236590+0000","modified":"2024-04-25T10:08:12.229638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2718963870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2718963870},{"type":"v1","addr":"172.21.15.60:6835","nonce":2718963870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:08:13.266 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 28 2024-04-25T10:08:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.258+0000 7f97431e0640 1 -- 172.21.15.60:0/2426428588 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9720066f90 msgr2=0x7f9720069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.258+0000 7f97431e0640 1 --2- 172.21.15.60:0/2426428588 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9720066f90 0x7f9720069450 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f97280af000 tx=0x7f9728008040 comp rx=0 tx=0).stop 2024-04-25T10:08:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.258+0000 7f97431e0640 1 -- 172.21.15.60:0/2426428588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972c004810 msgr2=0x7f973c153540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.258+0000 7f97431e0640 1 --2- 172.21.15.60:0/2426428588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972c004810 0x7f973c153540 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f973000d4a0 tx=0x7f973000d960 comp rx=0 tx=0).stop 2024-04-25T10:08:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.258+0000 7f97431e0640 1 -- 172.21.15.60:0/2426428588 shutdown_connections 2024-04-25T10:08:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.258+0000 7f97431e0640 1 --2- 172.21.15.60:0/2426428588 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9720066f90 0x7f9720069450 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.258+0000 7f97431e0640 1 --2- 172.21.15.60:0/2426428588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f973c153a80 0x7f973c1f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.258+0000 7f97431e0640 1 --2- 172.21.15.60:0/2426428588 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f973c159c00 0x7f973c1f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.258+0000 7f97431e0640 1 --2- 172.21.15.60:0/2426428588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972c004810 0x7f973c153540 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.258+0000 7f97431e0640 1 -- 172.21.15.60:0/2426428588 >> 172.21.15.60:0/2426428588 conn(0x7f973c0b12d0 msgr2=0x7f973c15e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.258+0000 7f97431e0640 1 -- 172.21.15.60:0/2426428588 shutdown_connections 2024-04-25T10:08:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.258+0000 7f97431e0640 1 -- 172.21.15.60:0/2426428588 wait complete. 2024-04-25T10:08:13.284 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:13.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f39a0925640 1 Processor -- start 2024-04-25T10:08:13.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f39a0925640 1 -- start start 2024-04-25T10:08:13.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f39a0925640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c0b4660 0x7f399c15c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f39a0925640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399c071ca0 con 0x7f399c0ce7a0 2024-04-25T10:08:13.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f39a0925640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399c0cca40 con 0x7f399c15bc90 2024-04-25T10:08:13.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f39a0925640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0cc370 con 0x7f399c0b4660 2024-04-25T10:08:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399a59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f399c0ce7a0 0x7f399c0ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41504/0 (socket says 172.21.15.60:41504) 2024-04-25T10:08:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399a59c640 1 -- 172.21.15.60:0/437948040 learned_addr learned my addr 172.21.15.60:0/437948040 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399ad9d640 1 --2- 172.21.15.60:0/437948040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c0b4660 0x7f399c15c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399959a640 1 -- 172.21.15.60:0/437948040 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816379527 0 0) 0x7f399c071ca0 con 0x7f399c0ce7a0 2024-04-25T10:08:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399959a640 1 -- 172.21.15.60:0/437948040 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3988003530 con 0x7f399c0ce7a0 2024-04-25T10:08:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399959a640 1 -- 172.21.15.60:0/437948040 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 24485836 0 0) 0x7f3988003530 con 0x7f399c0ce7a0 2024-04-25T10:08:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399959a640 1 -- 172.21.15.60:0/437948040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c0b4660 msgr2=0x7f399c15c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399959a640 1 --2- 172.21.15.60:0/437948040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c0b4660 0x7f399c15c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399959a640 1 -- 172.21.15.60:0/437948040 >> v1:172.21.15.100:6789/0 conn(0x7f399c15bc90 legacy=0x7f399c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399959a640 1 -- 172.21.15.60:0/437948040 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c0ccd20 con 0x7f399c0ce7a0 2024-04-25T10:08:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399959a640 1 -- 172.21.15.60:0/437948040 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f3984001a00 con 0x7f399c0ce7a0 2024-04-25T10:08:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399959a640 1 -- 172.21.15.60:0/437948040 >> v1:172.21.15.60:6789/0 conn(0x7f399c0ce7a0 legacy=0x7f399c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399959a640 1 --2- 172.21.15.60:0/437948040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39880039f0 0x7f3988003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399959a640 1 --2- 172.21.15.60:0/437948040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3988004810 0x7f3988004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399959a640 1 --2- 172.21.15.60:0/437948040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39880051d0 0x7f398800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399959a640 1 -- 172.21.15.60:0/437948040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398800bf10 con 0x7f3988004810 2024-04-25T10:08:13.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399959a640 1 -- 172.21.15.60:0/437948040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398800c090 con 0x7f39880039f0 2024-04-25T10:08:13.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399959a640 1 -- 172.21.15.60:0/437948040 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398800c210 con 0x7f39880051d0 2024-04-25T10:08:13.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f3999d9b640 1 --2- 172.21.15.60:0/437948040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3988004810 0x7f3988004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399a59c640 1 --2- 172.21.15.60:0/437948040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39880039f0 0x7f3988003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f399ad9d640 1 --2- 172.21.15.60:0/437948040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39880051d0 0x7f398800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f3999d9b640 1 -- 172.21.15.60:0/437948040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39880051d0 msgr2=0x7f398800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f3999d9b640 1 --2- 172.21.15.60:0/437948040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39880051d0 0x7f398800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f3999d9b640 1 -- 172.21.15.60:0/437948040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39880039f0 msgr2=0x7f3988003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f3999d9b640 1 --2- 172.21.15.60:0/437948040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39880039f0 0x7f3988003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.402+0000 7f3999d9b640 1 -- 172.21.15.60:0/437948040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3988003530 con 0x7f3988004810 2024-04-25T10:08:13.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.406+0000 7f3999d9b640 1 --2- 172.21.15.60:0/437948040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3988004810 0x7f3988004c90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f398c001780 tx=0x7f398c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d138f2de9ec176ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.406+0000 7f399959a640 1 -- 172.21.15.60:0/437948040 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c0013a0 con 0x7f3988004810 2024-04-25T10:08:13.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.406+0000 7f399959a640 1 -- 172.21.15.60:0/437948040 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c0052f0 con 0x7f3988004810 2024-04-25T10:08:13.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.406+0000 7f399959a640 1 -- 172.21.15.60:0/437948040 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c003840 con 0x7f3988004810 2024-04-25T10:08:13.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.406+0000 7f39a0925640 1 -- 172.21.15.60:0/437948040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3988004810 msgr2=0x7f3988004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.406+0000 7f39a0925640 1 --2- 172.21.15.60:0/437948040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3988004810 0x7f3988004c90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f398c001780 tx=0x7f398c01c3c0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.406+0000 7f399ad9d640 1 -- 172.21.15.60:0/437948040 reap_dead start 2024-04-25T10:08:13.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.406+0000 7f39a0925640 1 -- 172.21.15.60:0/437948040 shutdown_connections 2024-04-25T10:08:13.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.406+0000 7f39a0925640 1 -- 172.21.15.60:0/437948040 >> 172.21.15.60:0/437948040 conn(0x7f399c0b12d0 msgr2=0x7f399c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:13.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.406+0000 7f39a0925640 1 -- 172.21.15.60:0/437948040 shutdown_connections 2024-04-25T10:08:13.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.406+0000 7f39a0925640 1 -- 172.21.15.60:0/437948040 wait complete. 2024-04-25T10:08:13.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.406+0000 7f39a0925640 1 Processor -- start 2024-04-25T10:08:13.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f39a0925640 1 -- start start 2024-04-25T10:08:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f39a0925640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3988004810 0x7f399c15d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f39a0925640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399c15d7c0 0x7f399c14e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f39a0925640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c14ec70 0x7f399c0afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f39a0925640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0b3d20 con 0x7f399c15d7c0 2024-04-25T10:08:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f39a0925640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0b3ba0 con 0x7f3988004810 2024-04-25T10:08:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f39a0925640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0b4280 con 0x7f399c14ec70 2024-04-25T10:08:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f3999d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399c15d7c0 0x7f399c14e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f3999d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399c15d7c0 0x7f399c14e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58752/0 (socket says 172.21.15.60:58752) 2024-04-25T10:08:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f3999d9b640 1 -- 172.21.15.60:0/1864245474 learned_addr learned my addr 172.21.15.60:0/1864245474 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f3999d9b640 1 -- 172.21.15.60:0/1864245474 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c14ec70 msgr2=0x7f399c0afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f399ad9d640 1 --2- 172.21.15.60:0/1864245474 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c14ec70 0x7f399c0afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f399a59c640 1 --2- 172.21.15.60:0/1864245474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3988004810 0x7f399c15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f3999d9b640 1 --2- 172.21.15.60:0/1864245474 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c14ec70 0x7f399c0afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f3999d9b640 1 -- 172.21.15.60:0/1864245474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3988004810 msgr2=0x7f399c15d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f3999d9b640 1 --2- 172.21.15.60:0/1864245474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3988004810 0x7f399c15d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f3999d9b640 1 -- 172.21.15.60:0/1864245474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c1f6ed0 con 0x7f399c15d7c0 2024-04-25T10:08:13.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f399ad9d640 1 --2- 172.21.15.60:0/1864245474 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c14ec70 0x7f399c0afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:13.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f3999d9b640 1 --2- 172.21.15.60:0/1864245474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399c15d7c0 0x7f399c14e730 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f398c001780 tx=0x7f3988005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f3992ffd640 1 -- 172.21.15.60:0/1864245474 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c0ba020 con 0x7f399c15d7c0 2024-04-25T10:08:13.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f3992ffd640 1 -- 172.21.15.60:0/1864245474 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c000ee0 con 0x7f399c15d7c0 2024-04-25T10:08:13.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f39a0925640 1 -- 172.21.15.60:0/1864245474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399c1f71b0 con 0x7f399c15d7c0 2024-04-25T10:08:13.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f3992ffd640 1 -- 172.21.15.60:0/1864245474 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c0069e0 con 0x7f399c15d7c0 2024-04-25T10:08:13.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f39a0925640 1 -- 172.21.15.60:0/1864245474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399c1f7770 con 0x7f399c15d7c0 2024-04-25T10:08:13.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f3990ff9640 1 -- 172.21.15.60:0/1864245474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399c074770 con 0x7f399c15d7c0 2024-04-25T10:08:13.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f3992ffd640 1 -- 172.21.15.60:0/1864245474 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f398c0bd070 con 0x7f399c15d7c0 2024-04-25T10:08:13.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f3992ffd640 1 --2- 172.21.15.60:0/1864245474 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f397c066f90 0x7f397c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.410+0000 7f3992ffd640 1 -- 172.21.15.60:0/1864245474 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f398c11ead0 con 0x7f399c15d7c0 2024-04-25T10:08:13.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.414+0000 7f399a59c640 1 --2- 172.21.15.60:0/1864245474 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f397c066f90 0x7f397c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.414+0000 7f399a59c640 1 --2- 172.21.15.60:0/1864245474 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f397c066f90 0x7f397c069450 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3984002050 tx=0x7f39840017c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.418+0000 7f3992ffd640 1 -- 172.21.15.60:0/1864245474 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f398c016130 con 0x7f399c15d7c0 2024-04-25T10:08:13.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.578+0000 7f3990ff9640 1 -- 172.21.15.60:0/1864245474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f399c0b2fb0 con 0x7f399c15d7c0 2024-04-25T10:08:13.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.582+0000 7f3992ffd640 1 -- 172.21.15.60:0/1864245474 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16677 (secure 0 0 0) 0x7f398c0b4330 con 0x7f399c15d7c0 2024-04-25T10:08:13.590 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:13.591 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":58,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:08:11.878055+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:08.538247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:09.233842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.586+0000 7f3990ff9640 1 -- 172.21.15.60:0/1864245474 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f397c066f90 msgr2=0x7f397c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.586+0000 7f3990ff9640 1 --2- 172.21.15.60:0/1864245474 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f397c066f90 0x7f397c069450 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3984002050 tx=0x7f39840017c0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.586+0000 7f3990ff9640 1 -- 172.21.15.60:0/1864245474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399c15d7c0 msgr2=0x7f399c14e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.586+0000 7f3990ff9640 1 --2- 172.21.15.60:0/1864245474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399c15d7c0 0x7f399c14e730 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f398c001780 tx=0x7f3988005840 comp rx=0 tx=0).stop 2024-04-25T10:08:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.586+0000 7f3990ff9640 1 -- 172.21.15.60:0/1864245474 shutdown_connections 2024-04-25T10:08:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.586+0000 7f3990ff9640 1 --2- 172.21.15.60:0/1864245474 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f397c066f90 0x7f397c069450 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.586+0000 7f3990ff9640 1 --2- 172.21.15.60:0/1864245474 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c14ec70 0x7f399c0afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.586+0000 7f3990ff9640 1 --2- 172.21.15.60:0/1864245474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399c15d7c0 0x7f399c14e730 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.586+0000 7f3990ff9640 1 --2- 172.21.15.60:0/1864245474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3988004810 0x7f399c15d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.586+0000 7f3990ff9640 1 -- 172.21.15.60:0/1864245474 >> 172.21.15.60:0/1864245474 conn(0x7f399c0b12d0 msgr2=0x7f399c150fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:13.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.586+0000 7f3990ff9640 1 -- 172.21.15.60:0/1864245474 shutdown_connections 2024-04-25T10:08:13.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.586+0000 7f3990ff9640 1 -- 172.21.15.60:0/1864245474 wait complete. 2024-04-25T10:08:13.610 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:08:13.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:13.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f599cb40640 1 Processor -- start 2024-04-25T10:08:13.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f599cb40640 1 -- start start 2024-04-25T10:08:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f599cb40640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59980ce7a0 0x7f59980ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f599cb40640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f599815bc90 0x7f59980b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f599cb40640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f59980b4660 0x7f59980b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f599cb40640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59980cc370 con 0x7f59980b4660 2024-04-25T10:08:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f599cb40640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59980cf0c0 con 0x7f599815bc90 2024-04-25T10:08:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f599cb40640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5998071ca0 con 0x7f59980ce7a0 2024-04-25T10:08:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f5996d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f59980b4660 0x7f59980b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f5996d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f59980b4660 0x7f59980b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58754/0 (socket says 172.21.15.60:58754) 2024-04-25T10:08:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f5996d9d640 1 -- 172.21.15.60:0/1386462867 learned_addr learned my addr 172.21.15.60:0/1386462867 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f5995d9b640 1 --2- 172.21.15.60:0/1386462867 >> v2:172.21.15.100:3300/0 conn(0x7f599815bc90 0x7f59980b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f599659c640 1 --2- 172.21.15.60:0/1386462867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59980ce7a0 0x7f59980ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f5996d9d640 1 -- 172.21.15.60:0/1386462867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59980ce7a0 msgr2=0x7f59980ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f5996d9d640 1 --2- 172.21.15.60:0/1386462867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59980ce7a0 0x7f59980ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f5996d9d640 1 -- 172.21.15.60:0/1386462867 >> v2:172.21.15.100:3300/0 conn(0x7f599815bc90 msgr2=0x7f59980b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f5996d9d640 1 --2- 172.21.15.60:0/1386462867 >> v2:172.21.15.100:3300/0 conn(0x7f599815bc90 0x7f59980b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f5996d9d640 1 -- 172.21.15.60:0/1386462867 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59980cca40 con 0x7f59980b4660 2024-04-25T10:08:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.718+0000 7f5995d9b640 1 --2- 172.21.15.60:0/1386462867 >> v2:172.21.15.100:3300/0 conn(0x7f599815bc90 0x7f59980b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f5996d9d640 1 --2- 172.21.15.60:0/1386462867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59980b4660 0x7f59980b2b10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5990009870 tx=0x7f599001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c20a4f955dcc1b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599559a640 1 -- 172.21.15.60:0/1386462867 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59900b0070 con 0x7f59980b4660 2024-04-25T10:08:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599559a640 1 -- 172.21.15.60:0/1386462867 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59900ab600 con 0x7f59980b4660 2024-04-25T10:08:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599559a640 1 -- 172.21.15.60:0/1386462867 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59900ab9d0 con 0x7f59980b4660 2024-04-25T10:08:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 -- 172.21.15.60:0/1386462867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59980b4660 msgr2=0x7f59980b2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 --2- 172.21.15.60:0/1386462867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59980b4660 0x7f59980b2b10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5990009870 tx=0x7f599001c520 comp rx=0 tx=0).stop 2024-04-25T10:08:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 -- 172.21.15.60:0/1386462867 shutdown_connections 2024-04-25T10:08:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 --2- 172.21.15.60:0/1386462867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59980b4660 0x7f59980b2b10 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 --2- 172.21.15.60:0/1386462867 >> v2:172.21.15.100:3300/0 conn(0x7f599815bc90 0x7f59980b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 --2- 172.21.15.60:0/1386462867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59980ce7a0 0x7f59980ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 -- 172.21.15.60:0/1386462867 >> 172.21.15.60:0/1386462867 conn(0x7f59980b12d0 msgr2=0x7f59980b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:13.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 -- 172.21.15.60:0/1386462867 shutdown_connections 2024-04-25T10:08:13.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 -- 172.21.15.60:0/1386462867 wait complete. 2024-04-25T10:08:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 Processor -- start 2024-04-25T10:08:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 -- start start 2024-04-25T10:08:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59980ce7a0 0x7f5998157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599815bc90 0x7f5998155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59981591c0 0x7f5998146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59980b3db0 con 0x7f599815bc90 2024-04-25T10:08:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59980b3460 con 0x7f59981591c0 2024-04-25T10:08:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599cb40640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59980b4280 con 0x7f59980ce7a0 2024-04-25T10:08:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f5995d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599815bc90 0x7f5998155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f5995d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599815bc90 0x7f5998155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58770/0 (socket says 172.21.15.60:58770) 2024-04-25T10:08:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f5995d9b640 1 -- 172.21.15.60:0/3811635901 learned_addr learned my addr 172.21.15.60:0/3811635901 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599659c640 1 --2- 172.21.15.60:0/3811635901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59980ce7a0 0x7f5998157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f5995d9b640 1 -- 172.21.15.60:0/3811635901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59980ce7a0 msgr2=0x7f5998157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f5995d9b640 1 --2- 172.21.15.60:0/3811635901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59980ce7a0 0x7f5998157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f5995d9b640 1 -- 172.21.15.60:0/3811635901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59981591c0 msgr2=0x7f5998146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f5995d9b640 1 --2- 172.21.15.60:0/3811635901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59981591c0 0x7f5998146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f5995d9b640 1 -- 172.21.15.60:0/3811635901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5998158120 con 0x7f599815bc90 2024-04-25T10:08:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.722+0000 7f599659c640 1 --2- 172.21.15.60:0/3811635901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59980ce7a0 0x7f5998157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.726+0000 7f5995d9b640 1 --2- 172.21.15.60:0/3811635901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599815bc90 0x7f5998155830 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f598000c7f0 tx=0x7f598000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.726+0000 7f598effd640 1 -- 172.21.15.60:0/3811635901 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980015070 con 0x7f599815bc90 2024-04-25T10:08:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.726+0000 7f598effd640 1 -- 172.21.15.60:0/3811635901 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5980007820 con 0x7f599815bc90 2024-04-25T10:08:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.726+0000 7f599cb40640 1 -- 172.21.15.60:0/3811635901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5998158400 con 0x7f599815bc90 2024-04-25T10:08:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.726+0000 7f598effd640 1 -- 172.21.15.60:0/3811635901 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980007c10 con 0x7f599815bc90 2024-04-25T10:08:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.726+0000 7f599cb40640 1 -- 172.21.15.60:0/3811635901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59981589c0 con 0x7f599815bc90 2024-04-25T10:08:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.726+0000 7f599cb40640 1 -- 172.21.15.60:0/3811635901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5998074770 con 0x7f599815bc90 2024-04-25T10:08:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.726+0000 7f598effd640 1 -- 172.21.15.60:0/3811635901 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5980018070 con 0x7f599815bc90 2024-04-25T10:08:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.726+0000 7f598effd640 1 --2- 172.21.15.60:0/3811635901 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5978066f90 0x7f5978069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.726+0000 7f598effd640 1 -- 172.21.15.60:0/3811635901 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5980085130 con 0x7f599815bc90 2024-04-25T10:08:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.726+0000 7f599659c640 1 --2- 172.21.15.60:0/3811635901 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5978066f90 0x7f5978069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.726+0000 7f599659c640 1 --2- 172.21.15.60:0/3811635901 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5978066f90 0x7f5978069450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5988007da0 tx=0x7f59880072f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.730+0000 7f598effd640 1 -- 172.21.15.60:0/3811635901 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5980056140 con 0x7f599815bc90 2024-04-25T10:08:13.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.878+0000 7f599cb40640 1 -- 172.21.15.60:0/3811635901 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5998155d70 con 0x7f5978066f90 2024-04-25T10:08:13.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.898+0000 7f598effd640 1 -- 172.21.15.60:0/3811635901 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248142 (secure 0 0 0) 0x7f5998155d70 con 0x7f5978066f90 2024-04-25T10:08:13.910 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:13.912 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:08:13.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.902+0000 7f599cb40640 1 -- 172.21.15.60:0/3811635901 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5978066f90 msgr2=0x7f5978069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.902+0000 7f599cb40640 1 --2- 172.21.15.60:0/3811635901 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5978066f90 0x7f5978069450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5988007da0 tx=0x7f59880072f0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.902+0000 7f599cb40640 1 -- 172.21.15.60:0/3811635901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599815bc90 msgr2=0x7f5998155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.902+0000 7f599cb40640 1 --2- 172.21.15.60:0/3811635901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599815bc90 0x7f5998155830 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f598000c7f0 tx=0x7f598000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.902+0000 7f599cb40640 1 -- 172.21.15.60:0/3811635901 shutdown_connections 2024-04-25T10:08:13.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.902+0000 7f599cb40640 1 --2- 172.21.15.60:0/3811635901 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5978066f90 0x7f5978069450 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.902+0000 7f599cb40640 1 --2- 172.21.15.60:0/3811635901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59981591c0 0x7f5998146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.902+0000 7f599cb40640 1 --2- 172.21.15.60:0/3811635901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f599815bc90 0x7f5998155830 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.902+0000 7f599cb40640 1 --2- 172.21.15.60:0/3811635901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59980ce7a0 0x7f5998157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.902+0000 7f599cb40640 1 -- 172.21.15.60:0/3811635901 >> 172.21.15.60:0/3811635901 conn(0x7f59980b12d0 msgr2=0x7f59980af610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:13.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.902+0000 7f599cb40640 1 -- 172.21.15.60:0/3811635901 shutdown_connections 2024-04-25T10:08:13.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:13.902+0000 7f599cb40640 1 -- 172.21.15.60:0/3811635901 wait complete. 2024-04-25T10:08:13.924 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-04-25T10:08:12.446968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591076,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":181,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":144,"acting":144,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":260064,"kb_used_data":6436,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754714656,"statfs":{"total":773094113280,"available":772827807744,"internally_reserved":0,"allocated":6590464,"data_stored":3395474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":708,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001223"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.896431+0000","last_change":"2024-04-25T10:08:10.894975+0000","last_active":"2024-04-25T10:08:10.896431+0000","last_peered":"2024-04-25T10:08:10.896431+0000","last_clean":"2024-04-25T10:08:10.896431+0000","last_became_active":"2024-04-25T10:08:10.894724+0000","last_became_peered":"2024-04-25T10:08:10.894724+0000","last_unstale":"2024-04-25T10:08:10.896431+0000","last_undegraded":"2024-04-25T10:08:10.896431+0000","last_fullsized":"2024-04-25T10:08:10.896431+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:57.357697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901577+0000","last_change":"2024-04-25T10:08:10.893083+0000","last_active":"2024-04-25T10:08:10.901577+0000","last_peered":"2024-04-25T10:08:10.901577+0000","last_clean":"2024-04-25T10:08:10.901577+0000","last_became_active":"2024-04-25T10:08:10.892785+0000","last_became_peered":"2024-04-25T10:08:10.892785+0000","last_unstale":"2024-04-25T10:08:10.901577+0000","last_undegraded":"2024-04-25T10:08:10.901577+0000","last_fullsized":"2024-04-25T10:08:10.901577+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:31.332615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893692+0000","last_change":"2024-04-25T10:08:10.901432+0000","last_active":"2024-04-25T10:08:11.893692+0000","last_peered":"2024-04-25T10:08:11.893692+0000","last_clean":"2024-04-25T10:08:11.893692+0000","last_became_active":"2024-04-25T10:08:10.900367+0000","last_became_peered":"2024-04-25T10:08:10.900367+0000","last_unstale":"2024-04-25T10:08:11.893692+0000","last_undegraded":"2024-04-25T10:08:11.893692+0000","last_fullsized":"2024-04-25T10:08:11.893692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:27.633134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.910107+0000","last_change":"2024-04-25T10:08:10.909538+0000","last_active":"2024-04-25T10:08:10.910107+0000","last_peered":"2024-04-25T10:08:10.910107+0000","last_clean":"2024-04-25T10:08:10.910107+0000","last_became_active":"2024-04-25T10:08:10.909284+0000","last_became_peered":"2024-04-25T10:08:10.909284+0000","last_unstale":"2024-04-25T10:08:10.910107+0000","last_undegraded":"2024-04-25T10:08:10.910107+0000","last_fullsized":"2024-04-25T10:08:10.910107+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:36.225766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.889260+0000","last_change":"2024-04-25T10:08:10.887190+0000","last_active":"2024-04-25T10:08:10.889260+0000","last_peered":"2024-04-25T10:08:10.889260+0000","last_clean":"2024-04-25T10:08:10.889260+0000","last_became_active":"2024-04-25T10:08:10.886910+0000","last_became_peered":"2024-04-25T10:08:10.886910+0000","last_unstale":"2024-04-25T10:08:10.889260+0000","last_undegraded":"2024-04-25T10:08:10.889260+0000","last_fullsized":"2024-04-25T10:08:10.889260+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:33.123610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.906628+0000","last_change":"2024-04-25T10:08:10.905892+0000","last_active":"2024-04-25T10:08:10.906628+0000","last_peered":"2024-04-25T10:08:10.906628+0000","last_clean":"2024-04-25T10:08:10.906628+0000","last_became_active":"2024-04-25T10:08:10.905443+0000","last_became_peered":"2024-04-25T10:08:10.905443+0000","last_unstale":"2024-04-25T10:08:10.906628+0000","last_undegraded":"2024-04-25T10:08:10.906628+0000","last_fullsized":"2024-04-25T10:08:10.906628+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:09.407230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889904+0000","last_change":"2024-04-25T10:08:10.894120+0000","last_active":"2024-04-25T10:08:11.889904+0000","last_peered":"2024-04-25T10:08:11.889904+0000","last_clean":"2024-04-25T10:08:11.889904+0000","last_became_active":"2024-04-25T10:08:10.893807+0000","last_became_peered":"2024-04-25T10:08:10.893807+0000","last_unstale":"2024-04-25T10:08:11.889904+0000","last_undegraded":"2024-04-25T10:08:11.889904+0000","last_fullsized":"2024-04-25T10:08:11.889904+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:10.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.890117+0000","last_change":"2024-04-25T10:08:10.893934+0000","last_active":"2024-04-25T10:08:11.890117+0000","last_peered":"2024-04-25T10:08:11.890117+0000","last_clean":"2024-04-25T10:08:11.890117+0000","last_became_active":"2024-04-25T10:08:10.893270+0000","last_became_peered":"2024-04-25T10:08:10.893270+0000","last_unstale":"2024-04-25T10:08:11.890117+0000","last_undegraded":"2024-04-25T10:08:11.890117+0000","last_fullsized":"2024-04-25T10:08:11.890117+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:13.537677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894126+0000","last_change":"2024-04-25T10:08:10.923912+0000","last_active":"2024-04-25T10:08:11.894126+0000","last_peered":"2024-04-25T10:08:11.894126+0000","last_clean":"2024-04-25T10:08:11.894126+0000","last_became_active":"2024-04-25T10:08:10.923319+0000","last_became_peered":"2024-04-25T10:08:10.923319+0000","last_unstale":"2024-04-25T10:08:11.894126+0000","last_undegraded":"2024-04-25T10:08:11.894126+0000","last_fullsized":"2024-04-25T10:08:11.894126+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:44.473365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.968119+0000","last_change":"2024-04-25T10:08:10.886846+0000","last_active":"2024-04-25T10:08:11.968119+0000","last_peered":"2024-04-25T10:08:11.968119+0000","last_clean":"2024-04-25T10:08:11.968119+0000","last_became_active":"2024-04-25T10:08:10.886497+0000","last_became_peered":"2024-04-25T10:08:10.886497+0000","last_unstale":"2024-04-25T10:08:11.968119+0000","last_undegraded":"2024-04-25T10:08:11.968119+0000","last_fullsized":"2024-04-25T10:08:11.968119+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:56.861799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.897978+0000","last_change":"2024-04-25T10:08:10.896865+0000","last_active":"2024-04-25T10:08:10.897978+0000","last_peered":"2024-04-25T10:08:10.897978+0000","last_clean":"2024-04-25T10:08:10.897978+0000","last_became_active":"2024-04-25T10:08:10.896575+0000","last_became_peered":"2024-04-25T10:08:10.896575+0000","last_unstale":"2024-04-25T10:08:10.897978+0000","last_undegraded":"2024-04-25T10:08:10.897978+0000","last_fullsized":"2024-04-25T10:08:10.897978+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:31.167704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901912+0000","last_change":"2024-04-25T10:08:10.898151+0000","last_active":"2024-04-25T10:08:10.901912+0000","last_peered":"2024-04-25T10:08:10.901912+0000","last_clean":"2024-04-25T10:08:10.901912+0000","last_became_active":"2024-04-25T10:08:10.897849+0000","last_became_peered":"2024-04-25T10:08:10.897849+0000","last_unstale":"2024-04-25T10:08:10.901912+0000","last_undegraded":"2024-04-25T10:08:10.901912+0000","last_fullsized":"2024-04-25T10:08:10.901912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:47.458907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.895610+0000","last_change":"2024-04-25T10:08:10.891553+0000","last_active":"2024-04-25T10:08:10.895610+0000","last_peered":"2024-04-25T10:08:10.895610+0000","last_clean":"2024-04-25T10:08:10.895610+0000","last_became_active":"2024-04-25T10:08:10.891348+0000","last_became_peered":"2024-04-25T10:08:10.891348+0000","last_unstale":"2024-04-25T10:08:10.895610+0000","last_undegraded":"2024-04-25T10:08:10.895610+0000","last_fullsized":"2024-04-25T10:08:10.895610+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:48.658370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.897906+0000","last_change":"2024-04-25T10:08:10.890188+0000","last_active":"2024-04-25T10:08:10.897906+0000","last_peered":"2024-04-25T10:08:10.897906+0000","last_clean":"2024-04-25T10:08:10.897906+0000","last_became_active":"2024-04-25T10:08:10.889827+0000","last_became_peered":"2024-04-25T10:08:10.889827+0000","last_unstale":"2024-04-25T10:08:10.897906+0000","last_undegraded":"2024-04-25T10:08:10.897906+0000","last_fullsized":"2024-04-25T10:08:10.897906+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.834110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.900281+0000","last_change":"2024-04-25T10:08:10.892469+0000","last_active":"2024-04-25T10:08:10.900281+0000","last_peered":"2024-04-25T10:08:10.900281+0000","last_clean":"2024-04-25T10:08:10.900281+0000","last_became_active":"2024-04-25T10:08:10.890763+0000","last_became_peered":"2024-04-25T10:08:10.890763+0000","last_unstale":"2024-04-25T10:08:10.900281+0000","last_undegraded":"2024-04-25T10:08:10.900281+0000","last_fullsized":"2024-04-25T10:08:10.900281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:30.075855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967689+0000","last_change":"2024-04-25T10:08:10.897718+0000","last_active":"2024-04-25T10:08:11.967689+0000","last_peered":"2024-04-25T10:08:11.967689+0000","last_clean":"2024-04-25T10:08:11.967689+0000","last_became_active":"2024-04-25T10:08:10.897021+0000","last_became_peered":"2024-04-25T10:08:10.897021+0000","last_unstale":"2024-04-25T10:08:11.967689+0000","last_undegraded":"2024-04-25T10:08:11.967689+0000","last_fullsized":"2024-04-25T10:08:11.967689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:10.256509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.889404+0000","last_change":"2024-04-25T10:08:10.885585+0000","last_active":"2024-04-25T10:08:10.889404+0000","last_peered":"2024-04-25T10:08:10.889404+0000","last_clean":"2024-04-25T10:08:10.889404+0000","last_became_active":"2024-04-25T10:08:10.885318+0000","last_became_peered":"2024-04-25T10:08:10.885318+0000","last_unstale":"2024-04-25T10:08:10.889404+0000","last_undegraded":"2024-04-25T10:08:10.889404+0000","last_fullsized":"2024-04-25T10:08:10.889404+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:07.762131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893986+0000","last_change":"2024-04-25T10:08:10.911391+0000","last_active":"2024-04-25T10:08:11.893986+0000","last_peered":"2024-04-25T10:08:11.893986+0000","last_clean":"2024-04-25T10:08:11.893986+0000","last_became_active":"2024-04-25T10:08:10.910461+0000","last_became_peered":"2024-04-25T10:08:10.910461+0000","last_unstale":"2024-04-25T10:08:11.893986+0000","last_undegraded":"2024-04-25T10:08:11.893986+0000","last_fullsized":"2024-04-25T10:08:11.893986+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:48.269167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.896261+0000","last_change":"2024-04-25T10:08:10.894475+0000","last_active":"2024-04-25T10:08:10.896261+0000","last_peered":"2024-04-25T10:08:10.896261+0000","last_clean":"2024-04-25T10:08:10.896261+0000","last_became_active":"2024-04-25T10:08:10.894176+0000","last_became_peered":"2024-04-25T10:08:10.894176+0000","last_unstale":"2024-04-25T10:08:10.896261+0000","last_undegraded":"2024-04-25T10:08:10.896261+0000","last_fullsized":"2024-04-25T10:08:10.896261+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:57.804679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.902435+0000","last_change":"2024-04-25T10:08:10.901642+0000","last_active":"2024-04-25T10:08:10.902435+0000","last_peered":"2024-04-25T10:08:10.902435+0000","last_clean":"2024-04-25T10:08:10.902435+0000","last_became_active":"2024-04-25T10:08:10.901389+0000","last_became_peered":"2024-04-25T10:08:10.901389+0000","last_unstale":"2024-04-25T10:08:10.902435+0000","last_undegraded":"2024-04-25T10:08:10.902435+0000","last_fullsized":"2024-04-25T10:08:10.902435+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:01.232475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.915810+0000","last_change":"2024-04-25T10:08:10.915357+0000","last_active":"2024-04-25T10:08:10.915810+0000","last_peered":"2024-04-25T10:08:10.915810+0000","last_clean":"2024-04-25T10:08:10.915810+0000","last_became_active":"2024-04-25T10:08:10.915052+0000","last_became_peered":"2024-04-25T10:08:10.915052+0000","last_unstale":"2024-04-25T10:08:10.915810+0000","last_undegraded":"2024-04-25T10:08:10.915810+0000","last_fullsized":"2024-04-25T10:08:10.915810+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:13.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.889411+0000","last_change":"2024-04-25T10:08:10.884959+0000","last_active":"2024-04-25T10:08:10.889411+0000","last_peered":"2024-04-25T10:08:10.889411+0000","last_clean":"2024-04-25T10:08:10.889411+0000","last_became_active":"2024-04-25T10:08:10.884659+0000","last_became_peered":"2024-04-25T10:08:10.884659+0000","last_unstale":"2024-04-25T10:08:10.889411+0000","last_undegraded":"2024-04-25T10:08:10.889411+0000","last_fullsized":"2024-04-25T10:08:10.889411+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:42.724053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.896757+0000","last_change":"2024-04-25T10:08:10.895828+0000","last_active":"2024-04-25T10:08:10.896757+0000","last_peered":"2024-04-25T10:08:10.896757+0000","last_clean":"2024-04-25T10:08:10.896757+0000","last_became_active":"2024-04-25T10:08:10.895589+0000","last_became_peered":"2024-04-25T10:08:10.895589+0000","last_unstale":"2024-04-25T10:08:10.896757+0000","last_undegraded":"2024-04-25T10:08:10.896757+0000","last_fullsized":"2024-04-25T10:08:10.896757+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:45.805791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.912538+0000","last_change":"2024-04-25T10:08:10.911873+0000","last_active":"2024-04-25T10:08:10.912538+0000","last_peered":"2024-04-25T10:08:10.912538+0000","last_clean":"2024-04-25T10:08:10.912538+0000","last_became_active":"2024-04-25T10:08:10.911603+0000","last_became_peered":"2024-04-25T10:08:10.911603+0000","last_unstale":"2024-04-25T10:08:10.912538+0000","last_undegraded":"2024-04-25T10:08:10.912538+0000","last_fullsized":"2024-04-25T10:08:10.912538+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:27.633134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.926180+0000","last_change":"2024-04-25T10:08:09.906122+0000","last_active":"2024-04-25T10:08:10.926180+0000","last_peered":"2024-04-25T10:08:10.926180+0000","last_clean":"2024-04-25T10:08:10.926180+0000","last_became_active":"2024-04-25T10:08:09.905879+0000","last_became_peered":"2024-04-25T10:08:09.905879+0000","last_unstale":"2024-04-25T10:08:10.926180+0000","last_undegraded":"2024-04-25T10:08:10.926180+0000","last_fullsized":"2024-04-25T10:08:10.926180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:48.349703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.890056+0000","last_change":"2024-04-25T10:08:10.911299+0000","last_active":"2024-04-25T10:08:11.890056+0000","last_peered":"2024-04-25T10:08:11.890056+0000","last_clean":"2024-04-25T10:08:11.890056+0000","last_became_active":"2024-04-25T10:08:10.910974+0000","last_became_peered":"2024-04-25T10:08:10.910974+0000","last_unstale":"2024-04-25T10:08:11.890056+0000","last_undegraded":"2024-04-25T10:08:11.890056+0000","last_fullsized":"2024-04-25T10:08:11.890056+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:22.511670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.921727+0000","last_change":"2024-04-25T10:08:09.898224+0000","last_active":"2024-04-25T10:08:10.921727+0000","last_peered":"2024-04-25T10:08:10.921727+0000","last_clean":"2024-04-25T10:08:10.921727+0000","last_became_active":"2024-04-25T10:08:09.897935+0000","last_became_peered":"2024-04-25T10:08:09.897935+0000","last_unstale":"2024-04-25T10:08:10.921727+0000","last_undegraded":"2024-04-25T10:08:10.921727+0000","last_fullsized":"2024-04-25T10:08:10.921727+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:16.589298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.884524+0000","last_change":"2024-04-25T10:08:10.882470+0000","last_active":"2024-04-25T10:08:10.884524+0000","last_peered":"2024-04-25T10:08:10.884524+0000","last_clean":"2024-04-25T10:08:10.884524+0000","last_became_active":"2024-04-25T10:08:10.882185+0000","last_became_peered":"2024-04-25T10:08:10.882185+0000","last_unstale":"2024-04-25T10:08:10.884524+0000","last_undegraded":"2024-04-25T10:08:10.884524+0000","last_fullsized":"2024-04-25T10:08:10.884524+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.344111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.889456+0000","last_change":"2024-04-25T10:08:09.888457+0000","last_active":"2024-04-25T10:08:10.889456+0000","last_peered":"2024-04-25T10:08:10.889456+0000","last_clean":"2024-04-25T10:08:10.889456+0000","last_became_active":"2024-04-25T10:08:09.888103+0000","last_became_peered":"2024-04-25T10:08:09.888103+0000","last_unstale":"2024-04-25T10:08:10.889456+0000","last_undegraded":"2024-04-25T10:08:10.889456+0000","last_fullsized":"2024-04-25T10:08:10.889456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.255517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.927157+0000","last_change":"2024-04-25T10:08:10.926635+0000","last_active":"2024-04-25T10:08:10.927157+0000","last_peered":"2024-04-25T10:08:10.927157+0000","last_clean":"2024-04-25T10:08:10.927157+0000","last_became_active":"2024-04-25T10:08:10.926349+0000","last_became_peered":"2024-04-25T10:08:10.926349+0000","last_unstale":"2024-04-25T10:08:10.927157+0000","last_undegraded":"2024-04-25T10:08:10.927157+0000","last_fullsized":"2024-04-25T10:08:10.927157+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:17.598272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894215+0000","last_change":"2024-04-25T10:08:09.907608+0000","last_active":"2024-04-25T10:08:11.894215+0000","last_peered":"2024-04-25T10:08:11.894215+0000","last_clean":"2024-04-25T10:08:11.894215+0000","last_became_active":"2024-04-25T10:08:09.906963+0000","last_became_peered":"2024-04-25T10:08:09.906963+0000","last_unstale":"2024-04-25T10:08:11.894215+0000","last_undegraded":"2024-04-25T10:08:11.894215+0000","last_fullsized":"2024-04-25T10:08:11.894215+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.255517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.900251+0000","last_change":"2024-04-25T10:08:10.892319+0000","last_active":"2024-04-25T10:08:10.900251+0000","last_peered":"2024-04-25T10:08:10.900251+0000","last_clean":"2024-04-25T10:08:10.900251+0000","last_became_active":"2024-04-25T10:08:10.890374+0000","last_became_peered":"2024-04-25T10:08:10.890374+0000","last_unstale":"2024-04-25T10:08:10.900251+0000","last_undegraded":"2024-04-25T10:08:10.900251+0000","last_fullsized":"2024-04-25T10:08:10.900251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:21.713220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:08.840704+0000","last_change":"2024-04-25T10:08:08.840704+0000","last_active":"2024-04-25T10:08:08.840704+0000","last_peered":"2024-04-25T10:08:08.840704+0000","last_clean":"2024-04-25T10:08:08.840704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:08.840704+0000","last_undegraded":"2024-04-25T10:08:08.840704+0000","last_fullsized":"2024-04-25T10:08:08.840704+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.900555+0000","last_change":"2024-04-25T10:08:10.890541+0000","last_active":"2024-04-25T10:08:10.900555+0000","last_peered":"2024-04-25T10:08:10.900555+0000","last_clean":"2024-04-25T10:08:10.900555+0000","last_became_active":"2024-04-25T10:08:10.890280+0000","last_became_peered":"2024-04-25T10:08:10.890280+0000","last_unstale":"2024-04-25T10:08:10.900555+0000","last_undegraded":"2024-04-25T10:08:10.900555+0000","last_fullsized":"2024-04-25T10:08:10.900555+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:35.281267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.895707+0000","last_change":"2024-04-25T10:08:09.894473+0000","last_active":"2024-04-25T10:08:10.895707+0000","last_peered":"2024-04-25T10:08:10.895707+0000","last_clean":"2024-04-25T10:08:10.895707+0000","last_became_active":"2024-04-25T10:08:09.894181+0000","last_became_peered":"2024-04-25T10:08:09.894181+0000","last_unstale":"2024-04-25T10:08:10.895707+0000","last_undegraded":"2024-04-25T10:08:10.895707+0000","last_fullsized":"2024-04-25T10:08:10.895707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:10.625997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967711+0000","last_change":"2024-04-25T10:08:10.923620+0000","last_active":"2024-04-25T10:08:11.967711+0000","last_peered":"2024-04-25T10:08:11.967711+0000","last_clean":"2024-04-25T10:08:11.967711+0000","last_became_active":"2024-04-25T10:08:10.923032+0000","last_became_peered":"2024-04-25T10:08:10.923032+0000","last_unstale":"2024-04-25T10:08:11.967711+0000","last_undegraded":"2024-04-25T10:08:11.967711+0000","last_fullsized":"2024-04-25T10:08:11.967711+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:10.085497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893167+0000","last_change":"2024-04-25T10:08:09.907088+0000","last_active":"2024-04-25T10:08:11.893167+0000","last_peered":"2024-04-25T10:08:11.893167+0000","last_clean":"2024-04-25T10:08:11.893167+0000","last_became_active":"2024-04-25T10:08:09.906310+0000","last_became_peered":"2024-04-25T10:08:09.906310+0000","last_unstale":"2024-04-25T10:08:11.893167+0000","last_undegraded":"2024-04-25T10:08:11.893167+0000","last_fullsized":"2024-04-25T10:08:11.893167+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.592539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901609+0000","last_change":"2024-04-25T10:08:10.892966+0000","last_active":"2024-04-25T10:08:10.901609+0000","last_peered":"2024-04-25T10:08:10.901609+0000","last_clean":"2024-04-25T10:08:10.901609+0000","last_became_active":"2024-04-25T10:08:10.892667+0000","last_became_peered":"2024-04-25T10:08:10.892667+0000","last_unstale":"2024-04-25T10:08:10.901609+0000","last_undegraded":"2024-04-25T10:08:10.901609+0000","last_fullsized":"2024-04-25T10:08:10.901609+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:34.974943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:08.840704+0000","last_change":"2024-04-25T10:08:08.840704+0000","last_active":"2024-04-25T10:08:08.840704+0000","last_peered":"2024-04-25T10:08:08.840704+0000","last_clean":"2024-04-25T10:08:08.840704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:08.840704+0000","last_undegraded":"2024-04-25T10:08:08.840704+0000","last_fullsized":"2024-04-25T10:08:08.840704+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:08.840704+0000","last_change":"2024-04-25T10:08:08.840704+0000","last_active":"2024-04-25T10:08:08.840704+0000","last_peered":"2024-04-25T10:08:08.840704+0000","last_clean":"2024-04-25T10:08:08.840704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:08.840704+0000","last_undegraded":"2024-04-25T10:08:08.840704+0000","last_fullsized":"2024-04-25T10:08:08.840704+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.884018+0000","last_change":"2024-04-25T10:08:09.880232+0000","last_active":"2024-04-25T10:08:10.884018+0000","last_peered":"2024-04-25T10:08:10.884018+0000","last_clean":"2024-04-25T10:08:10.884018+0000","last_became_active":"2024-04-25T10:08:09.879917+0000","last_became_peered":"2024-04-25T10:08:09.879917+0000","last_unstale":"2024-04-25T10:08:10.884018+0000","last_undegraded":"2024-04-25T10:08:10.884018+0000","last_fullsized":"2024-04-25T10:08:10.884018+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:43.464391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889651+0000","last_change":"2024-04-25T10:08:10.893598+0000","last_active":"2024-04-25T10:08:11.889651+0000","last_peered":"2024-04-25T10:08:11.889651+0000","last_clean":"2024-04-25T10:08:11.889651+0000","last_became_active":"2024-04-25T10:08:10.893008+0000","last_became_peered":"2024-04-25T10:08:10.893008+0000","last_unstale":"2024-04-25T10:08:11.889651+0000","last_undegraded":"2024-04-25T10:08:11.889651+0000","last_fullsized":"2024-04-25T10:08:11.889651+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:57.430500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:08.840704+0000","last_change":"2024-04-25T10:08:08.840704+0000","last_active":"2024-04-25T10:08:08.840704+0000","last_peered":"2024-04-25T10:08:08.840704+0000","last_clean":"2024-04-25T10:08:08.840704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:08.840704+0000","last_undegraded":"2024-04-25T10:08:08.840704+0000","last_fullsized":"2024-04-25T10:08:08.840704+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:08.840704+0000","last_change":"2024-04-25T10:08:08.840704+0000","last_active":"2024-04-25T10:08:08.840704+0000","last_peered":"2024-04-25T10:08:08.840704+0000","last_clean":"2024-04-25T10:08:08.840704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:08.840704+0000","last_undegraded":"2024-04-25T10:08:08.840704+0000","last_fullsized":"2024-04-25T10:08:08.840704+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894725+0000","last_change":"2024-04-25T10:08:10.917026+0000","last_active":"2024-04-25T10:08:11.894725+0000","last_peered":"2024-04-25T10:08:11.894725+0000","last_clean":"2024-04-25T10:08:11.894725+0000","last_became_active":"2024-04-25T10:08:10.916406+0000","last_became_peered":"2024-04-25T10:08:10.916406+0000","last_unstale":"2024-04-25T10:08:11.894725+0000","last_undegraded":"2024-04-25T10:08:11.894725+0000","last_fullsized":"2024-04-25T10:08:11.894725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.834110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901694+0000","last_change":"2024-04-25T10:08:09.881070+0000","last_active":"2024-04-25T10:08:10.901694+0000","last_peered":"2024-04-25T10:08:10.901694+0000","last_clean":"2024-04-25T10:08:10.901694+0000","last_became_active":"2024-04-25T10:08:09.880757+0000","last_became_peered":"2024-04-25T10:08:09.880757+0000","last_unstale":"2024-04-25T10:08:10.901694+0000","last_undegraded":"2024-04-25T10:08:10.901694+0000","last_fullsized":"2024-04-25T10:08:10.901694+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:26.624159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967486+0000","last_change":"2024-04-25T10:08:09.899426+0000","last_active":"2024-04-25T10:08:11.967486+0000","last_peered":"2024-04-25T10:08:11.967486+0000","last_clean":"2024-04-25T10:08:11.967486+0000","last_became_active":"2024-04-25T10:08:09.898830+0000","last_became_peered":"2024-04-25T10:08:09.898830+0000","last_unstale":"2024-04-25T10:08:11.967486+0000","last_undegraded":"2024-04-25T10:08:11.967486+0000","last_fullsized":"2024-04-25T10:08:11.967486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:19.781296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893451+0000","last_change":"2024-04-25T10:08:09.902684+0000","last_active":"2024-04-25T10:08:11.893451+0000","last_peered":"2024-04-25T10:08:11.893451+0000","last_clean":"2024-04-25T10:08:11.893451+0000","last_became_active":"2024-04-25T10:08:09.901806+0000","last_became_peered":"2024-04-25T10:08:09.901806+0000","last_unstale":"2024-04-25T10:08:11.893451+0000","last_undegraded":"2024-04-25T10:08:11.893451+0000","last_fullsized":"2024-04-25T10:08:11.893451+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:17.199198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:08.840704+0000","last_change":"2024-04-25T10:08:08.840704+0000","last_active":"2024-04-25T10:08:08.840704+0000","last_peered":"2024-04-25T10:08:08.840704+0000","last_clean":"2024-04-25T10:08:08.840704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:08.840704+0000","last_undegraded":"2024-04-25T10:08:08.840704+0000","last_fullsized":"2024-04-25T10:08:08.840704+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.899177+0000","last_change":"2024-04-25T10:08:09.884420+0000","last_active":"2024-04-25T10:08:10.899177+0000","last_peered":"2024-04-25T10:08:10.899177+0000","last_clean":"2024-04-25T10:08:10.899177+0000","last_became_active":"2024-04-25T10:08:09.883427+0000","last_became_peered":"2024-04-25T10:08:09.883427+0000","last_unstale":"2024-04-25T10:08:10.899177+0000","last_undegraded":"2024-04-25T10:08:10.899177+0000","last_fullsized":"2024-04-25T10:08:10.899177+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:47.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894725+0000","last_change":"2024-04-25T10:08:10.918083+0000","last_active":"2024-04-25T10:08:11.894725+0000","last_peered":"2024-04-25T10:08:11.894725+0000","last_clean":"2024-04-25T10:08:11.894725+0000","last_became_active":"2024-04-25T10:08:10.917512+0000","last_became_peered":"2024-04-25T10:08:10.917512+0000","last_unstale":"2024-04-25T10:08:11.894725+0000","last_undegraded":"2024-04-25T10:08:11.894725+0000","last_fullsized":"2024-04-25T10:08:11.894725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:35.281267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.891599+0000","last_change":"2024-04-25T10:08:09.888804+0000","last_active":"2024-04-25T10:08:10.891599+0000","last_peered":"2024-04-25T10:08:10.891599+0000","last_clean":"2024-04-25T10:08:10.891599+0000","last_became_active":"2024-04-25T10:08:09.888488+0000","last_became_peered":"2024-04-25T10:08:09.888488+0000","last_unstale":"2024-04-25T10:08:10.891599+0000","last_undegraded":"2024-04-25T10:08:10.891599+0000","last_fullsized":"2024-04-25T10:08:10.891599+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:17.199198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.906640+0000","last_change":"2024-04-25T10:08:10.905832+0000","last_active":"2024-04-25T10:08:10.906640+0000","last_peered":"2024-04-25T10:08:10.906640+0000","last_clean":"2024-04-25T10:08:10.906640+0000","last_became_active":"2024-04-25T10:08:10.905524+0000","last_became_peered":"2024-04-25T10:08:10.905524+0000","last_unstale":"2024-04-25T10:08:10.906640+0000","last_undegraded":"2024-04-25T10:08:10.906640+0000","last_fullsized":"2024-04-25T10:08:10.906640+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:29.292616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893944+0000","last_change":"2024-04-25T10:08:09.902367+0000","last_active":"2024-04-25T10:08:11.893944+0000","last_peered":"2024-04-25T10:08:11.893944+0000","last_clean":"2024-04-25T10:08:11.893944+0000","last_became_active":"2024-04-25T10:08:09.900592+0000","last_became_peered":"2024-04-25T10:08:09.900592+0000","last_unstale":"2024-04-25T10:08:11.893944+0000","last_undegraded":"2024-04-25T10:08:11.893944+0000","last_fullsized":"2024-04-25T10:08:11.893944+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:38.492052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.878704+0000","last_change":"2024-04-25T10:08:09.890143+0000","last_active":"2024-04-25T10:08:10.878704+0000","last_peered":"2024-04-25T10:08:10.878704+0000","last_clean":"2024-04-25T10:08:10.878704+0000","last_became_active":"2024-04-25T10:08:09.889126+0000","last_became_peered":"2024-04-25T10:08:09.889126+0000","last_unstale":"2024-04-25T10:08:10.878704+0000","last_undegraded":"2024-04-25T10:08:10.878704+0000","last_fullsized":"2024-04-25T10:08:10.878704+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:38.492052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893893+0000","last_change":"2024-04-25T10:08:10.917214+0000","last_active":"2024-04-25T10:08:11.893893+0000","last_peered":"2024-04-25T10:08:11.893893+0000","last_clean":"2024-04-25T10:08:11.893893+0000","last_became_active":"2024-04-25T10:08:10.915770+0000","last_became_peered":"2024-04-25T10:08:10.915770+0000","last_unstale":"2024-04-25T10:08:11.893893+0000","last_undegraded":"2024-04-25T10:08:11.893893+0000","last_fullsized":"2024-04-25T10:08:11.893893+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:44.066536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.907624+0000","last_change":"2024-04-25T10:08:09.889071+0000","last_active":"2024-04-25T10:08:10.907624+0000","last_peered":"2024-04-25T10:08:10.907624+0000","last_clean":"2024-04-25T10:08:10.907624+0000","last_became_active":"2024-04-25T10:08:09.888746+0000","last_became_peered":"2024-04-25T10:08:09.888746+0000","last_unstale":"2024-04-25T10:08:10.907624+0000","last_undegraded":"2024-04-25T10:08:10.907624+0000","last_fullsized":"2024-04-25T10:08:10.907624+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:43.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.895295+0000","last_change":"2024-04-25T10:08:10.893599+0000","last_active":"2024-04-25T10:08:10.895295+0000","last_peered":"2024-04-25T10:08:10.895295+0000","last_clean":"2024-04-25T10:08:10.895295+0000","last_became_active":"2024-04-25T10:08:10.893206+0000","last_became_peered":"2024-04-25T10:08:10.893206+0000","last_unstale":"2024-04-25T10:08:10.895295+0000","last_undegraded":"2024-04-25T10:08:10.895295+0000","last_fullsized":"2024-04-25T10:08:10.895295+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:51.143246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889836+0000","last_change":"2024-04-25T10:08:09.880919+0000","last_active":"2024-04-25T10:08:11.889836+0000","last_peered":"2024-04-25T10:08:11.889836+0000","last_clean":"2024-04-25T10:08:11.889836+0000","last_became_active":"2024-04-25T10:08:09.880611+0000","last_became_peered":"2024-04-25T10:08:09.880611+0000","last_unstale":"2024-04-25T10:08:11.889836+0000","last_undegraded":"2024-04-25T10:08:11.889836+0000","last_fullsized":"2024-04-25T10:08:11.889836+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:22.155681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.902841+0000","last_change":"2024-04-25T10:08:10.901150+0000","last_active":"2024-04-25T10:08:10.902841+0000","last_peered":"2024-04-25T10:08:10.902841+0000","last_clean":"2024-04-25T10:08:10.902841+0000","last_became_active":"2024-04-25T10:08:10.900624+0000","last_became_peered":"2024-04-25T10:08:10.900624+0000","last_unstale":"2024-04-25T10:08:10.902841+0000","last_undegraded":"2024-04-25T10:08:10.902841+0000","last_fullsized":"2024-04-25T10:08:10.902841+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:08.745250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.892866+0000","last_change":"2024-04-25T10:08:09.901094+0000","last_active":"2024-04-25T10:08:11.892866+0000","last_peered":"2024-04-25T10:08:11.892866+0000","last_clean":"2024-04-25T10:08:11.892866+0000","last_became_active":"2024-04-25T10:08:09.900405+0000","last_became_peered":"2024-04-25T10:08:09.900405+0000","last_unstale":"2024-04-25T10:08:11.892866+0000","last_undegraded":"2024-04-25T10:08:11.892866+0000","last_fullsized":"2024-04-25T10:08:11.892866+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:30.789048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893987+0000","last_change":"2024-04-25T10:08:10.909724+0000","last_active":"2024-04-25T10:08:11.893987+0000","last_peered":"2024-04-25T10:08:11.893987+0000","last_clean":"2024-04-25T10:08:11.893987+0000","last_became_active":"2024-04-25T10:08:10.904979+0000","last_became_peered":"2024-04-25T10:08:10.904979+0000","last_unstale":"2024-04-25T10:08:11.893987+0000","last_undegraded":"2024-04-25T10:08:11.893987+0000","last_fullsized":"2024-04-25T10:08:11.893987+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:17.598272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889987+0000","last_change":"2024-04-25T10:08:09.885875+0000","last_active":"2024-04-25T10:08:11.889987+0000","last_peered":"2024-04-25T10:08:11.889987+0000","last_clean":"2024-04-25T10:08:11.889987+0000","last_became_active":"2024-04-25T10:08:09.885529+0000","last_became_peered":"2024-04-25T10:08:09.885529+0000","last_unstale":"2024-04-25T10:08:11.889987+0000","last_undegraded":"2024-04-25T10:08:11.889987+0000","last_fullsized":"2024-04-25T10:08:11.889987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:30.869245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.895037+0000","last_change":"2024-04-25T10:08:10.923948+0000","last_active":"2024-04-25T10:08:11.895037+0000","last_peered":"2024-04-25T10:08:11.895037+0000","last_clean":"2024-04-25T10:08:11.895037+0000","last_became_active":"2024-04-25T10:08:10.923383+0000","last_became_peered":"2024-04-25T10:08:10.923383+0000","last_unstale":"2024-04-25T10:08:11.895037+0000","last_undegraded":"2024-04-25T10:08:11.895037+0000","last_fullsized":"2024-04-25T10:08:11.895037+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:55.065275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.879776+0000","last_change":"2024-04-25T10:08:09.894894+0000","last_active":"2024-04-25T10:08:10.879776+0000","last_peered":"2024-04-25T10:08:10.879776+0000","last_clean":"2024-04-25T10:08:10.879776+0000","last_became_active":"2024-04-25T10:08:09.894350+0000","last_became_peered":"2024-04-25T10:08:09.894350+0000","last_unstale":"2024-04-25T10:08:10.879776+0000","last_undegraded":"2024-04-25T10:08:10.879776+0000","last_fullsized":"2024-04-25T10:08:10.879776+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.592539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889925+0000","last_change":"2024-04-25T10:08:10.893322+0000","last_active":"2024-04-25T10:08:11.889925+0000","last_peered":"2024-04-25T10:08:11.889925+0000","last_clean":"2024-04-25T10:08:11.889925+0000","last_became_active":"2024-04-25T10:08:10.892639+0000","last_became_peered":"2024-04-25T10:08:10.892639+0000","last_unstale":"2024-04-25T10:08:11.889925+0000","last_undegraded":"2024-04-25T10:08:11.889925+0000","last_fullsized":"2024-04-25T10:08:11.889925+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:37.445694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967957+0000","last_change":"2024-04-25T10:08:09.893305+0000","last_active":"2024-04-25T10:08:11.967957+0000","last_peered":"2024-04-25T10:08:11.967957+0000","last_clean":"2024-04-25T10:08:11.967957+0000","last_became_active":"2024-04-25T10:08:09.892501+0000","last_became_peered":"2024-04-25T10:08:09.892501+0000","last_unstale":"2024-04-25T10:08:11.967957+0000","last_undegraded":"2024-04-25T10:08:11.967957+0000","last_fullsized":"2024-04-25T10:08:11.967957+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:21.522866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.926945+0000","last_change":"2024-04-25T10:08:10.926260+0000","last_active":"2024-04-25T10:08:10.926945+0000","last_peered":"2024-04-25T10:08:10.926945+0000","last_clean":"2024-04-25T10:08:10.926945+0000","last_became_active":"2024-04-25T10:08:10.926038+0000","last_became_peered":"2024-04-25T10:08:10.926038+0000","last_unstale":"2024-04-25T10:08:10.926945+0000","last_undegraded":"2024-04-25T10:08:10.926945+0000","last_fullsized":"2024-04-25T10:08:10.926945+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:44.066536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894376+0000","last_change":"2024-04-25T10:08:09.906942+0000","last_active":"2024-04-25T10:08:11.894376+0000","last_peered":"2024-04-25T10:08:11.894376+0000","last_clean":"2024-04-25T10:08:11.894376+0000","last_became_active":"2024-04-25T10:08:09.906192+0000","last_became_peered":"2024-04-25T10:08:09.906192+0000","last_unstale":"2024-04-25T10:08:11.894376+0000","last_undegraded":"2024-04-25T10:08:11.894376+0000","last_fullsized":"2024-04-25T10:08:11.894376+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:48.349703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.899168+0000","last_change":"2024-04-25T10:08:10.890698+0000","last_active":"2024-04-25T10:08:10.899168+0000","last_peered":"2024-04-25T10:08:10.899168+0000","last_clean":"2024-04-25T10:08:10.899168+0000","last_became_active":"2024-04-25T10:08:10.889982+0000","last_became_peered":"2024-04-25T10:08:10.889982+0000","last_unstale":"2024-04-25T10:08:10.899168+0000","last_undegraded":"2024-04-25T10:08:10.899168+0000","last_fullsized":"2024-04-25T10:08:10.899168+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:55.065275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889624+0000","last_change":"2024-04-25T10:08:09.902559+0000","last_active":"2024-04-25T10:08:11.889624+0000","last_peered":"2024-04-25T10:08:11.889624+0000","last_clean":"2024-04-25T10:08:11.889624+0000","last_became_active":"2024-04-25T10:08:09.902240+0000","last_became_peered":"2024-04-25T10:08:09.902240+0000","last_unstale":"2024-04-25T10:08:11.889624+0000","last_undegraded":"2024-04-25T10:08:11.889624+0000","last_fullsized":"2024-04-25T10:08:11.889624+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:30.202367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:08.840704+0000","last_change":"2024-04-25T10:08:08.840704+0000","last_active":"2024-04-25T10:08:08.840704+0000","last_peered":"2024-04-25T10:08:08.840704+0000","last_clean":"2024-04-25T10:08:08.840704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:08.840704+0000","last_undegraded":"2024-04-25T10:08:08.840704+0000","last_fullsized":"2024-04-25T10:08:08.840704+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889431+0000","last_change":"2024-04-25T10:08:10.894492+0000","last_active":"2024-04-25T10:08:11.889431+0000","last_peered":"2024-04-25T10:08:11.889431+0000","last_clean":"2024-04-25T10:08:11.889431+0000","last_became_active":"2024-04-25T10:08:10.894221+0000","last_became_peered":"2024-04-25T10:08:10.894221+0000","last_unstale":"2024-04-25T10:08:11.889431+0000","last_undegraded":"2024-04-25T10:08:11.889431+0000","last_fullsized":"2024-04-25T10:08:11.889431+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:47.897507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889533+0000","last_change":"2024-04-25T10:08:09.890601+0000","last_active":"2024-04-25T10:08:11.889533+0000","last_peered":"2024-04-25T10:08:11.889533+0000","last_clean":"2024-04-25T10:08:11.889533+0000","last_became_active":"2024-04-25T10:08:09.890208+0000","last_became_peered":"2024-04-25T10:08:09.890208+0000","last_unstale":"2024-04-25T10:08:11.889533+0000","last_undegraded":"2024-04-25T10:08:11.889533+0000","last_fullsized":"2024-04-25T10:08:11.889533+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:01.465554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901843+0000","last_change":"2024-04-25T10:08:10.898030+0000","last_active":"2024-04-25T10:08:10.901843+0000","last_peered":"2024-04-25T10:08:10.901843+0000","last_clean":"2024-04-25T10:08:10.901843+0000","last_became_active":"2024-04-25T10:08:10.897719+0000","last_became_peered":"2024-04-25T10:08:10.897719+0000","last_unstale":"2024-04-25T10:08:10.901843+0000","last_undegraded":"2024-04-25T10:08:10.901843+0000","last_fullsized":"2024-04-25T10:08:10.901843+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:18.216787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.879881+0000","last_change":"2024-04-25T10:08:09.900914+0000","last_active":"2024-04-25T10:08:10.879881+0000","last_peered":"2024-04-25T10:08:10.879881+0000","last_clean":"2024-04-25T10:08:10.879881+0000","last_became_active":"2024-04-25T10:08:09.900669+0000","last_became_peered":"2024-04-25T10:08:09.900669+0000","last_unstale":"2024-04-25T10:08:10.879881+0000","last_undegraded":"2024-04-25T10:08:10.879881+0000","last_fullsized":"2024-04-25T10:08:10.879881+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.736275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901873+0000","last_change":"2024-04-25T10:08:10.894360+0000","last_active":"2024-04-25T10:08:10.901873+0000","last_peered":"2024-04-25T10:08:10.901873+0000","last_clean":"2024-04-25T10:08:10.901873+0000","last_became_active":"2024-04-25T10:08:10.894017+0000","last_became_peered":"2024-04-25T10:08:10.894017+0000","last_unstale":"2024-04-25T10:08:10.901873+0000","last_undegraded":"2024-04-25T10:08:10.901873+0000","last_fullsized":"2024-04-25T10:08:10.901873+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:43.207466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967562+0000","last_change":"2024-04-25T10:08:09.909488+0000","last_active":"2024-04-25T10:08:11.967562+0000","last_peered":"2024-04-25T10:08:11.967562+0000","last_clean":"2024-04-25T10:08:11.967562+0000","last_became_active":"2024-04-25T10:08:09.908863+0000","last_became_peered":"2024-04-25T10:08:09.908863+0000","last_unstale":"2024-04-25T10:08:11.967562+0000","last_undegraded":"2024-04-25T10:08:11.967562+0000","last_fullsized":"2024-04-25T10:08:11.967562+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:48.865163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":229,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:08:07.503843+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:08:07.503843+0000","last_peered":"2024-04-25T10:08:07.503843+0000","last_clean":"2024-04-25T10:08:07.503843+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:08:07.503843+0000","last_undegraded":"2024-04-25T10:08:07.503843+0000","last_fullsized":"2024-04-25T10:08:07.503843+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":708,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":50,"acting":50,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1232896,"data_stored":851110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31816,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":638976,"data_stored":260742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542183,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":688128,"data_stored":285901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":286553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542184,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36012,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335828,"statfs":{"total":96636764160,"available":96599887872,"internally_reserved":0,"allocated":684032,"data_stored":279657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":684032,"data_stored":279657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36004,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335836,"statfs":{"total":96636764160,"available":96599896064,"internally_reserved":0,"allocated":679936,"data_stored":279691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36596,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335244,"statfs":{"total":96636764160,"available":96599289856,"internally_reserved":0,"allocated":1286144,"data_stored":872163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:13.929 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:14.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.070+0000 7f90d4ff8640 1 Processor -- start 2024-04-25T10:08:14.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.070+0000 7f90d4ff8640 1 -- start start 2024-04-25T10:08:14.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.074+0000 7f90d4ff8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f90d0071290 0x7f90d0071720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.074+0000 7f90d4ff8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90d00cc990 0x7f90d00cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.074+0000 7f90d4ff8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d00b3aa0 con 0x7f90d0162250 2024-04-25T10:08:14.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.074+0000 7f90d4ff8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d00af0c0 con 0x7f90d0071290 2024-04-25T10:08:14.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.074+0000 7f90d4ff8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d00b3d80 con 0x7f90d00cc990 2024-04-25T10:08:14.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.074+0000 7f90ced9d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90d00cc990 0x7f90d00cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.074+0000 7f90cdd9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f90d0071290 0x7f90d0071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.074+0000 7f90cdd9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f90d0071290 0x7f90d0071720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:39548/0 (socket says 172.21.15.60:39548) 2024-04-25T10:08:14.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.074+0000 7f90ced9d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90d00cc990 0x7f90d00cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:50102/0 (socket says 172.21.15.60:50102) 2024-04-25T10:08:14.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.074+0000 7f90cdd9b640 1 -- 172.21.15.60:0/3285843847 learned_addr learned my addr 172.21.15.60:0/3285843847 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:14.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.074+0000 7f90ce59c640 1 --1- 172.21.15.60:0/3285843847 >> v1:172.21.15.100:6789/0 conn(0x7f90d0162250 0x7f90d00b2fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:51292/0 (socket says 172.21.15.60:51292) 2024-04-25T10:08:14.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.074+0000 7f90cdd9b640 1 -- 172.21.15.60:0/3285843847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90d00cc990 msgr2=0x7f90d00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:14.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.074+0000 7f90cdd9b640 1 --2- 172.21.15.60:0/3285843847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90d00cc990 0x7f90d00cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.074+0000 7f90cdd9b640 1 -- 172.21.15.60:0/3285843847 >> v1:172.21.15.100:6789/0 conn(0x7f90d0162250 legacy=0x7f90d00b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.074+0000 7f90cdd9b640 1 -- 172.21.15.60:0/3285843847 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d00b4060 con 0x7f90d0071290 2024-04-25T10:08:14.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90cdd9b640 1 --2- 172.21.15.60:0/3285843847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d0071290 0x7f90d0071720 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009870 tx=0x7f90b801ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f09eff45ea9a6530 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90cd59a640 1 -- 172.21.15.60:0/3285843847 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b80b5070 con 0x7f90d0071290 2024-04-25T10:08:14.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90cd59a640 1 -- 172.21.15.60:0/3285843847 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b80a8a50 con 0x7f90d0071290 2024-04-25T10:08:14.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90cd59a640 1 -- 172.21.15.60:0/3285843847 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b80a8e20 con 0x7f90d0071290 2024-04-25T10:08:14.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90d4ff8640 1 -- 172.21.15.60:0/3285843847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d0071290 msgr2=0x7f90d0071720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90d4ff8640 1 --2- 172.21.15.60:0/3285843847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d0071290 0x7f90d0071720 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009870 tx=0x7f90b801ed50 comp rx=0 tx=0).stop 2024-04-25T10:08:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90d4ff8640 1 -- 172.21.15.60:0/3285843847 shutdown_connections 2024-04-25T10:08:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90d4ff8640 1 --2- 172.21.15.60:0/3285843847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d0071290 0x7f90d0071720 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90d4ff8640 1 --2- 172.21.15.60:0/3285843847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90d00cc990 0x7f90d00cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90d4ff8640 1 -- 172.21.15.60:0/3285843847 >> 172.21.15.60:0/3285843847 conn(0x7f90d00b12d0 msgr2=0x7f90d00b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90d4ff8640 1 -- 172.21.15.60:0/3285843847 shutdown_connections 2024-04-25T10:08:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90d4ff8640 1 -- 172.21.15.60:0/3285843847 wait complete. 2024-04-25T10:08:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90d4ff8640 1 Processor -- start 2024-04-25T10:08:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90d4ff8640 1 -- start start 2024-04-25T10:08:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90d4ff8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90d0071290 0x7f90d01573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90d4ff8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d00cc990 0x7f90d00af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90d4ff8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d0162250 0x7f90d00af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90d4ff8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d00cc440 con 0x7f90d00cc990 2024-04-25T10:08:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90d4ff8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d006e5a0 con 0x7f90d0162250 2024-04-25T10:08:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90d4ff8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d00b36c0 con 0x7f90d0071290 2024-04-25T10:08:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90cdd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d00cc990 0x7f90d00af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90cdd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d00cc990 0x7f90d00af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58774/0 (socket says 172.21.15.60:58774) 2024-04-25T10:08:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90cdd9b640 1 -- 172.21.15.60:0/823224800 learned_addr learned my addr 172.21.15.60:0/823224800 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90ce59c640 1 --2- 172.21.15.60:0/823224800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90d0071290 0x7f90d01573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90ced9d640 1 --2- 172.21.15.60:0/823224800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d0162250 0x7f90d00af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90cdd9b640 1 -- 172.21.15.60:0/823224800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90d0071290 msgr2=0x7f90d01573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90cdd9b640 1 --2- 172.21.15.60:0/823224800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90d0071290 0x7f90d01573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90cdd9b640 1 -- 172.21.15.60:0/823224800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d0162250 msgr2=0x7f90d00af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90cdd9b640 1 --2- 172.21.15.60:0/823224800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d0162250 0x7f90d00af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90cdd9b640 1 -- 172.21.15.60:0/823224800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d015ddf0 con 0x7f90d00cc990 2024-04-25T10:08:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90cdd9b640 1 --2- 172.21.15.60:0/823224800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d00cc990 0x7f90d00af240 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f90b80b3040 tx=0x7f90b801baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.078+0000 7f90c6ffd640 1 -- 172.21.15.60:0/823224800 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b80b5070 con 0x7f90d00cc990 2024-04-25T10:08:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.082+0000 7f90c6ffd640 1 -- 172.21.15.60:0/823224800 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b80b0de0 con 0x7f90d00cc990 2024-04-25T10:08:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.082+0000 7f90c6ffd640 1 -- 172.21.15.60:0/823224800 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b80bb670 con 0x7f90d00cc990 2024-04-25T10:08:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.082+0000 7f90d4ff8640 1 -- 172.21.15.60:0/823224800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d015e0d0 con 0x7f90d00cc990 2024-04-25T10:08:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.082+0000 7f90d4ff8640 1 -- 172.21.15.60:0/823224800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d015dae0 con 0x7f90d00cc990 2024-04-25T10:08:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.082+0000 7f90d4ff8640 1 -- 172.21.15.60:0/823224800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d0074770 con 0x7f90d00cc990 2024-04-25T10:08:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.082+0000 7f90c6ffd640 1 -- 172.21.15.60:0/823224800 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f90b80c2060 con 0x7f90d00cc990 2024-04-25T10:08:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.082+0000 7f90c6ffd640 1 --2- 172.21.15.60:0/823224800 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f90b0066f60 0x7f90b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.082+0000 7f90c6ffd640 1 -- 172.21.15.60:0/823224800 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90b8122540 con 0x7f90d00cc990 2024-04-25T10:08:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.082+0000 7f90ce59c640 1 --2- 172.21.15.60:0/823224800 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f90b0066f60 0x7f90b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.082+0000 7f90ce59c640 1 --2- 172.21.15.60:0/823224800 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f90b0066f60 0x7f90b0069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f90d00ccf90 tx=0x7f90c0007cf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.086+0000 7f90c6ffd640 1 -- 172.21.15.60:0/823224800 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f90b80188b0 con 0x7f90d00cc990 2024-04-25T10:08:14.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.238+0000 7f90d4ff8640 1 -- 172.21.15.60:0/823224800 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90d00cedb0 con 0x7f90b0066f60 2024-04-25T10:08:14.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.254+0000 7f90c6ffd640 1 -- 172.21.15.60:0/823224800 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248142 (secure 0 0 0) 0x7f90d00cedb0 con 0x7f90b0066f60 2024-04-25T10:08:14.265 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:14.267 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:08:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.258+0000 7f90d4ff8640 1 -- 172.21.15.60:0/823224800 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f90b0066f60 msgr2=0x7f90b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.258+0000 7f90d4ff8640 1 --2- 172.21.15.60:0/823224800 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f90b0066f60 0x7f90b0069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f90d00ccf90 tx=0x7f90c0007cf0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.258+0000 7f90d4ff8640 1 -- 172.21.15.60:0/823224800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d00cc990 msgr2=0x7f90d00af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.258+0000 7f90d4ff8640 1 --2- 172.21.15.60:0/823224800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d00cc990 0x7f90d00af240 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f90b80b3040 tx=0x7f90b801baf0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.258+0000 7f90d4ff8640 1 -- 172.21.15.60:0/823224800 shutdown_connections 2024-04-25T10:08:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.258+0000 7f90d4ff8640 1 --2- 172.21.15.60:0/823224800 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f90b0066f60 0x7f90b0069420 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.258+0000 7f90d4ff8640 1 --2- 172.21.15.60:0/823224800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d0162250 0x7f90d00af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.258+0000 7f90d4ff8640 1 --2- 172.21.15.60:0/823224800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d00cc990 0x7f90d00af240 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.258+0000 7f90d4ff8640 1 --2- 172.21.15.60:0/823224800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90d0071290 0x7f90d01573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.258+0000 7f90d4ff8640 1 -- 172.21.15.60:0/823224800 >> 172.21.15.60:0/823224800 conn(0x7f90d00b12d0 msgr2=0x7f90d00cdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.258+0000 7f90d4ff8640 1 -- 172.21.15.60:0/823224800 shutdown_connections 2024-04-25T10:08:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.258+0000 7f90d4ff8640 1 -- 172.21.15.60:0/823224800 wait complete. 2024-04-25T10:08:14.281 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-04-25T10:08:12.446968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591076,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":181,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":144,"acting":144,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":260064,"kb_used_data":6436,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754714656,"statfs":{"total":773094113280,"available":772827807744,"internally_reserved":0,"allocated":6590464,"data_stored":3395474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":708,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001223"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.896431+0000","last_change":"2024-04-25T10:08:10.894975+0000","last_active":"2024-04-25T10:08:10.896431+0000","last_peered":"2024-04-25T10:08:10.896431+0000","last_clean":"2024-04-25T10:08:10.896431+0000","last_became_active":"2024-04-25T10:08:10.894724+0000","last_became_peered":"2024-04-25T10:08:10.894724+0000","last_unstale":"2024-04-25T10:08:10.896431+0000","last_undegraded":"2024-04-25T10:08:10.896431+0000","last_fullsized":"2024-04-25T10:08:10.896431+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:57.357697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901577+0000","last_change":"2024-04-25T10:08:10.893083+0000","last_active":"2024-04-25T10:08:10.901577+0000","last_peered":"2024-04-25T10:08:10.901577+0000","last_clean":"2024-04-25T10:08:10.901577+0000","last_became_active":"2024-04-25T10:08:10.892785+0000","last_became_peered":"2024-04-25T10:08:10.892785+0000","last_unstale":"2024-04-25T10:08:10.901577+0000","last_undegraded":"2024-04-25T10:08:10.901577+0000","last_fullsized":"2024-04-25T10:08:10.901577+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:31.332615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893692+0000","last_change":"2024-04-25T10:08:10.901432+0000","last_active":"2024-04-25T10:08:11.893692+0000","last_peered":"2024-04-25T10:08:11.893692+0000","last_clean":"2024-04-25T10:08:11.893692+0000","last_became_active":"2024-04-25T10:08:10.900367+0000","last_became_peered":"2024-04-25T10:08:10.900367+0000","last_unstale":"2024-04-25T10:08:11.893692+0000","last_undegraded":"2024-04-25T10:08:11.893692+0000","last_fullsized":"2024-04-25T10:08:11.893692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:27.633134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.910107+0000","last_change":"2024-04-25T10:08:10.909538+0000","last_active":"2024-04-25T10:08:10.910107+0000","last_peered":"2024-04-25T10:08:10.910107+0000","last_clean":"2024-04-25T10:08:10.910107+0000","last_became_active":"2024-04-25T10:08:10.909284+0000","last_became_peered":"2024-04-25T10:08:10.909284+0000","last_unstale":"2024-04-25T10:08:10.910107+0000","last_undegraded":"2024-04-25T10:08:10.910107+0000","last_fullsized":"2024-04-25T10:08:10.910107+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:36.225766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.889260+0000","last_change":"2024-04-25T10:08:10.887190+0000","last_active":"2024-04-25T10:08:10.889260+0000","last_peered":"2024-04-25T10:08:10.889260+0000","last_clean":"2024-04-25T10:08:10.889260+0000","last_became_active":"2024-04-25T10:08:10.886910+0000","last_became_peered":"2024-04-25T10:08:10.886910+0000","last_unstale":"2024-04-25T10:08:10.889260+0000","last_undegraded":"2024-04-25T10:08:10.889260+0000","last_fullsized":"2024-04-25T10:08:10.889260+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:33.123610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.906628+0000","last_change":"2024-04-25T10:08:10.905892+0000","last_active":"2024-04-25T10:08:10.906628+0000","last_peered":"2024-04-25T10:08:10.906628+0000","last_clean":"2024-04-25T10:08:10.906628+0000","last_became_active":"2024-04-25T10:08:10.905443+0000","last_became_peered":"2024-04-25T10:08:10.905443+0000","last_unstale":"2024-04-25T10:08:10.906628+0000","last_undegraded":"2024-04-25T10:08:10.906628+0000","last_fullsized":"2024-04-25T10:08:10.906628+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:09.407230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889904+0000","last_change":"2024-04-25T10:08:10.894120+0000","last_active":"2024-04-25T10:08:11.889904+0000","last_peered":"2024-04-25T10:08:11.889904+0000","last_clean":"2024-04-25T10:08:11.889904+0000","last_became_active":"2024-04-25T10:08:10.893807+0000","last_became_peered":"2024-04-25T10:08:10.893807+0000","last_unstale":"2024-04-25T10:08:11.889904+0000","last_undegraded":"2024-04-25T10:08:11.889904+0000","last_fullsized":"2024-04-25T10:08:11.889904+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:10.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.890117+0000","last_change":"2024-04-25T10:08:10.893934+0000","last_active":"2024-04-25T10:08:11.890117+0000","last_peered":"2024-04-25T10:08:11.890117+0000","last_clean":"2024-04-25T10:08:11.890117+0000","last_became_active":"2024-04-25T10:08:10.893270+0000","last_became_peered":"2024-04-25T10:08:10.893270+0000","last_unstale":"2024-04-25T10:08:11.890117+0000","last_undegraded":"2024-04-25T10:08:11.890117+0000","last_fullsized":"2024-04-25T10:08:11.890117+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:13.537677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894126+0000","last_change":"2024-04-25T10:08:10.923912+0000","last_active":"2024-04-25T10:08:11.894126+0000","last_peered":"2024-04-25T10:08:11.894126+0000","last_clean":"2024-04-25T10:08:11.894126+0000","last_became_active":"2024-04-25T10:08:10.923319+0000","last_became_peered":"2024-04-25T10:08:10.923319+0000","last_unstale":"2024-04-25T10:08:11.894126+0000","last_undegraded":"2024-04-25T10:08:11.894126+0000","last_fullsized":"2024-04-25T10:08:11.894126+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:44.473365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.968119+0000","last_change":"2024-04-25T10:08:10.886846+0000","last_active":"2024-04-25T10:08:11.968119+0000","last_peered":"2024-04-25T10:08:11.968119+0000","last_clean":"2024-04-25T10:08:11.968119+0000","last_became_active":"2024-04-25T10:08:10.886497+0000","last_became_peered":"2024-04-25T10:08:10.886497+0000","last_unstale":"2024-04-25T10:08:11.968119+0000","last_undegraded":"2024-04-25T10:08:11.968119+0000","last_fullsized":"2024-04-25T10:08:11.968119+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:56.861799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.897978+0000","last_change":"2024-04-25T10:08:10.896865+0000","last_active":"2024-04-25T10:08:10.897978+0000","last_peered":"2024-04-25T10:08:10.897978+0000","last_clean":"2024-04-25T10:08:10.897978+0000","last_became_active":"2024-04-25T10:08:10.896575+0000","last_became_peered":"2024-04-25T10:08:10.896575+0000","last_unstale":"2024-04-25T10:08:10.897978+0000","last_undegraded":"2024-04-25T10:08:10.897978+0000","last_fullsized":"2024-04-25T10:08:10.897978+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:31.167704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901912+0000","last_change":"2024-04-25T10:08:10.898151+0000","last_active":"2024-04-25T10:08:10.901912+0000","last_peered":"2024-04-25T10:08:10.901912+0000","last_clean":"2024-04-25T10:08:10.901912+0000","last_became_active":"2024-04-25T10:08:10.897849+0000","last_became_peered":"2024-04-25T10:08:10.897849+0000","last_unstale":"2024-04-25T10:08:10.901912+0000","last_undegraded":"2024-04-25T10:08:10.901912+0000","last_fullsized":"2024-04-25T10:08:10.901912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:47.458907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.895610+0000","last_change":"2024-04-25T10:08:10.891553+0000","last_active":"2024-04-25T10:08:10.895610+0000","last_peered":"2024-04-25T10:08:10.895610+0000","last_clean":"2024-04-25T10:08:10.895610+0000","last_became_active":"2024-04-25T10:08:10.891348+0000","last_became_peered":"2024-04-25T10:08:10.891348+0000","last_unstale":"2024-04-25T10:08:10.895610+0000","last_undegraded":"2024-04-25T10:08:10.895610+0000","last_fullsized":"2024-04-25T10:08:10.895610+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:48.658370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.897906+0000","last_change":"2024-04-25T10:08:10.890188+0000","last_active":"2024-04-25T10:08:10.897906+0000","last_peered":"2024-04-25T10:08:10.897906+0000","last_clean":"2024-04-25T10:08:10.897906+0000","last_became_active":"2024-04-25T10:08:10.889827+0000","last_became_peered":"2024-04-25T10:08:10.889827+0000","last_unstale":"2024-04-25T10:08:10.897906+0000","last_undegraded":"2024-04-25T10:08:10.897906+0000","last_fullsized":"2024-04-25T10:08:10.897906+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.834110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.900281+0000","last_change":"2024-04-25T10:08:10.892469+0000","last_active":"2024-04-25T10:08:10.900281+0000","last_peered":"2024-04-25T10:08:10.900281+0000","last_clean":"2024-04-25T10:08:10.900281+0000","last_became_active":"2024-04-25T10:08:10.890763+0000","last_became_peered":"2024-04-25T10:08:10.890763+0000","last_unstale":"2024-04-25T10:08:10.900281+0000","last_undegraded":"2024-04-25T10:08:10.900281+0000","last_fullsized":"2024-04-25T10:08:10.900281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:30.075855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967689+0000","last_change":"2024-04-25T10:08:10.897718+0000","last_active":"2024-04-25T10:08:11.967689+0000","last_peered":"2024-04-25T10:08:11.967689+0000","last_clean":"2024-04-25T10:08:11.967689+0000","last_became_active":"2024-04-25T10:08:10.897021+0000","last_became_peered":"2024-04-25T10:08:10.897021+0000","last_unstale":"2024-04-25T10:08:11.967689+0000","last_undegraded":"2024-04-25T10:08:11.967689+0000","last_fullsized":"2024-04-25T10:08:11.967689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:10.256509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.889404+0000","last_change":"2024-04-25T10:08:10.885585+0000","last_active":"2024-04-25T10:08:10.889404+0000","last_peered":"2024-04-25T10:08:10.889404+0000","last_clean":"2024-04-25T10:08:10.889404+0000","last_became_active":"2024-04-25T10:08:10.885318+0000","last_became_peered":"2024-04-25T10:08:10.885318+0000","last_unstale":"2024-04-25T10:08:10.889404+0000","last_undegraded":"2024-04-25T10:08:10.889404+0000","last_fullsized":"2024-04-25T10:08:10.889404+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:07.762131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893986+0000","last_change":"2024-04-25T10:08:10.911391+0000","last_active":"2024-04-25T10:08:11.893986+0000","last_peered":"2024-04-25T10:08:11.893986+0000","last_clean":"2024-04-25T10:08:11.893986+0000","last_became_active":"2024-04-25T10:08:10.910461+0000","last_became_peered":"2024-04-25T10:08:10.910461+0000","last_unstale":"2024-04-25T10:08:11.893986+0000","last_undegraded":"2024-04-25T10:08:11.893986+0000","last_fullsized":"2024-04-25T10:08:11.893986+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:48.269167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.896261+0000","last_change":"2024-04-25T10:08:10.894475+0000","last_active":"2024-04-25T10:08:10.896261+0000","last_peered":"2024-04-25T10:08:10.896261+0000","last_clean":"2024-04-25T10:08:10.896261+0000","last_became_active":"2024-04-25T10:08:10.894176+0000","last_became_peered":"2024-04-25T10:08:10.894176+0000","last_unstale":"2024-04-25T10:08:10.896261+0000","last_undegraded":"2024-04-25T10:08:10.896261+0000","last_fullsized":"2024-04-25T10:08:10.896261+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:57.804679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.902435+0000","last_change":"2024-04-25T10:08:10.901642+0000","last_active":"2024-04-25T10:08:10.902435+0000","last_peered":"2024-04-25T10:08:10.902435+0000","last_clean":"2024-04-25T10:08:10.902435+0000","last_became_active":"2024-04-25T10:08:10.901389+0000","last_became_peered":"2024-04-25T10:08:10.901389+0000","last_unstale":"2024-04-25T10:08:10.902435+0000","last_undegraded":"2024-04-25T10:08:10.902435+0000","last_fullsized":"2024-04-25T10:08:10.902435+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:01.232475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.915810+0000","last_change":"2024-04-25T10:08:10.915357+0000","last_active":"2024-04-25T10:08:10.915810+0000","last_peered":"2024-04-25T10:08:10.915810+0000","last_clean":"2024-04-25T10:08:10.915810+0000","last_became_active":"2024-04-25T10:08:10.915052+0000","last_became_peered":"2024-04-25T10:08:10.915052+0000","last_unstale":"2024-04-25T10:08:10.915810+0000","last_undegraded":"2024-04-25T10:08:10.915810+0000","last_fullsized":"2024-04-25T10:08:10.915810+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:13.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.889411+0000","last_change":"2024-04-25T10:08:10.884959+0000","last_active":"2024-04-25T10:08:10.889411+0000","last_peered":"2024-04-25T10:08:10.889411+0000","last_clean":"2024-04-25T10:08:10.889411+0000","last_became_active":"2024-04-25T10:08:10.884659+0000","last_became_peered":"2024-04-25T10:08:10.884659+0000","last_unstale":"2024-04-25T10:08:10.889411+0000","last_undegraded":"2024-04-25T10:08:10.889411+0000","last_fullsized":"2024-04-25T10:08:10.889411+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:42.724053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.896757+0000","last_change":"2024-04-25T10:08:10.895828+0000","last_active":"2024-04-25T10:08:10.896757+0000","last_peered":"2024-04-25T10:08:10.896757+0000","last_clean":"2024-04-25T10:08:10.896757+0000","last_became_active":"2024-04-25T10:08:10.895589+0000","last_became_peered":"2024-04-25T10:08:10.895589+0000","last_unstale":"2024-04-25T10:08:10.896757+0000","last_undegraded":"2024-04-25T10:08:10.896757+0000","last_fullsized":"2024-04-25T10:08:10.896757+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:45.805791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.912538+0000","last_change":"2024-04-25T10:08:10.911873+0000","last_active":"2024-04-25T10:08:10.912538+0000","last_peered":"2024-04-25T10:08:10.912538+0000","last_clean":"2024-04-25T10:08:10.912538+0000","last_became_active":"2024-04-25T10:08:10.911603+0000","last_became_peered":"2024-04-25T10:08:10.911603+0000","last_unstale":"2024-04-25T10:08:10.912538+0000","last_undegraded":"2024-04-25T10:08:10.912538+0000","last_fullsized":"2024-04-25T10:08:10.912538+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:27.633134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.926180+0000","last_change":"2024-04-25T10:08:09.906122+0000","last_active":"2024-04-25T10:08:10.926180+0000","last_peered":"2024-04-25T10:08:10.926180+0000","last_clean":"2024-04-25T10:08:10.926180+0000","last_became_active":"2024-04-25T10:08:09.905879+0000","last_became_peered":"2024-04-25T10:08:09.905879+0000","last_unstale":"2024-04-25T10:08:10.926180+0000","last_undegraded":"2024-04-25T10:08:10.926180+0000","last_fullsized":"2024-04-25T10:08:10.926180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:48.349703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.890056+0000","last_change":"2024-04-25T10:08:10.911299+0000","last_active":"2024-04-25T10:08:11.890056+0000","last_peered":"2024-04-25T10:08:11.890056+0000","last_clean":"2024-04-25T10:08:11.890056+0000","last_became_active":"2024-04-25T10:08:10.910974+0000","last_became_peered":"2024-04-25T10:08:10.910974+0000","last_unstale":"2024-04-25T10:08:11.890056+0000","last_undegraded":"2024-04-25T10:08:11.890056+0000","last_fullsized":"2024-04-25T10:08:11.890056+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:22.511670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.921727+0000","last_change":"2024-04-25T10:08:09.898224+0000","last_active":"2024-04-25T10:08:10.921727+0000","last_peered":"2024-04-25T10:08:10.921727+0000","last_clean":"2024-04-25T10:08:10.921727+0000","last_became_active":"2024-04-25T10:08:09.897935+0000","last_became_peered":"2024-04-25T10:08:09.897935+0000","last_unstale":"2024-04-25T10:08:10.921727+0000","last_undegraded":"2024-04-25T10:08:10.921727+0000","last_fullsized":"2024-04-25T10:08:10.921727+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:16.589298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.884524+0000","last_change":"2024-04-25T10:08:10.882470+0000","last_active":"2024-04-25T10:08:10.884524+0000","last_peered":"2024-04-25T10:08:10.884524+0000","last_clean":"2024-04-25T10:08:10.884524+0000","last_became_active":"2024-04-25T10:08:10.882185+0000","last_became_peered":"2024-04-25T10:08:10.882185+0000","last_unstale":"2024-04-25T10:08:10.884524+0000","last_undegraded":"2024-04-25T10:08:10.884524+0000","last_fullsized":"2024-04-25T10:08:10.884524+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.344111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.889456+0000","last_change":"2024-04-25T10:08:09.888457+0000","last_active":"2024-04-25T10:08:10.889456+0000","last_peered":"2024-04-25T10:08:10.889456+0000","last_clean":"2024-04-25T10:08:10.889456+0000","last_became_active":"2024-04-25T10:08:09.888103+0000","last_became_peered":"2024-04-25T10:08:09.888103+0000","last_unstale":"2024-04-25T10:08:10.889456+0000","last_undegraded":"2024-04-25T10:08:10.889456+0000","last_fullsized":"2024-04-25T10:08:10.889456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.255517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.927157+0000","last_change":"2024-04-25T10:08:10.926635+0000","last_active":"2024-04-25T10:08:10.927157+0000","last_peered":"2024-04-25T10:08:10.927157+0000","last_clean":"2024-04-25T10:08:10.927157+0000","last_became_active":"2024-04-25T10:08:10.926349+0000","last_became_peered":"2024-04-25T10:08:10.926349+0000","last_unstale":"2024-04-25T10:08:10.927157+0000","last_undegraded":"2024-04-25T10:08:10.927157+0000","last_fullsized":"2024-04-25T10:08:10.927157+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:17.598272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894215+0000","last_change":"2024-04-25T10:08:09.907608+0000","last_active":"2024-04-25T10:08:11.894215+0000","last_peered":"2024-04-25T10:08:11.894215+0000","last_clean":"2024-04-25T10:08:11.894215+0000","last_became_active":"2024-04-25T10:08:09.906963+0000","last_became_peered":"2024-04-25T10:08:09.906963+0000","last_unstale":"2024-04-25T10:08:11.894215+0000","last_undegraded":"2024-04-25T10:08:11.894215+0000","last_fullsized":"2024-04-25T10:08:11.894215+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.255517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.900251+0000","last_change":"2024-04-25T10:08:10.892319+0000","last_active":"2024-04-25T10:08:10.900251+0000","last_peered":"2024-04-25T10:08:10.900251+0000","last_clean":"2024-04-25T10:08:10.900251+0000","last_became_active":"2024-04-25T10:08:10.890374+0000","last_became_peered":"2024-04-25T10:08:10.890374+0000","last_unstale":"2024-04-25T10:08:10.900251+0000","last_undegraded":"2024-04-25T10:08:10.900251+0000","last_fullsized":"2024-04-25T10:08:10.900251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:21.713220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:08.840704+0000","last_change":"2024-04-25T10:08:08.840704+0000","last_active":"2024-04-25T10:08:08.840704+0000","last_peered":"2024-04-25T10:08:08.840704+0000","last_clean":"2024-04-25T10:08:08.840704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:08.840704+0000","last_undegraded":"2024-04-25T10:08:08.840704+0000","last_fullsized":"2024-04-25T10:08:08.840704+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.900555+0000","last_change":"2024-04-25T10:08:10.890541+0000","last_active":"2024-04-25T10:08:10.900555+0000","last_peered":"2024-04-25T10:08:10.900555+0000","last_clean":"2024-04-25T10:08:10.900555+0000","last_became_active":"2024-04-25T10:08:10.890280+0000","last_became_peered":"2024-04-25T10:08:10.890280+0000","last_unstale":"2024-04-25T10:08:10.900555+0000","last_undegraded":"2024-04-25T10:08:10.900555+0000","last_fullsized":"2024-04-25T10:08:10.900555+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:35.281267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.895707+0000","last_change":"2024-04-25T10:08:09.894473+0000","last_active":"2024-04-25T10:08:10.895707+0000","last_peered":"2024-04-25T10:08:10.895707+0000","last_clean":"2024-04-25T10:08:10.895707+0000","last_became_active":"2024-04-25T10:08:09.894181+0000","last_became_peered":"2024-04-25T10:08:09.894181+0000","last_unstale":"2024-04-25T10:08:10.895707+0000","last_undegraded":"2024-04-25T10:08:10.895707+0000","last_fullsized":"2024-04-25T10:08:10.895707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:10.625997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967711+0000","last_change":"2024-04-25T10:08:10.923620+0000","last_active":"2024-04-25T10:08:11.967711+0000","last_peered":"2024-04-25T10:08:11.967711+0000","last_clean":"2024-04-25T10:08:11.967711+0000","last_became_active":"2024-04-25T10:08:10.923032+0000","last_became_peered":"2024-04-25T10:08:10.923032+0000","last_unstale":"2024-04-25T10:08:11.967711+0000","last_undegraded":"2024-04-25T10:08:11.967711+0000","last_fullsized":"2024-04-25T10:08:11.967711+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:10.085497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893167+0000","last_change":"2024-04-25T10:08:09.907088+0000","last_active":"2024-04-25T10:08:11.893167+0000","last_peered":"2024-04-25T10:08:11.893167+0000","last_clean":"2024-04-25T10:08:11.893167+0000","last_became_active":"2024-04-25T10:08:09.906310+0000","last_became_peered":"2024-04-25T10:08:09.906310+0000","last_unstale":"2024-04-25T10:08:11.893167+0000","last_undegraded":"2024-04-25T10:08:11.893167+0000","last_fullsized":"2024-04-25T10:08:11.893167+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.592539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901609+0000","last_change":"2024-04-25T10:08:10.892966+0000","last_active":"2024-04-25T10:08:10.901609+0000","last_peered":"2024-04-25T10:08:10.901609+0000","last_clean":"2024-04-25T10:08:10.901609+0000","last_became_active":"2024-04-25T10:08:10.892667+0000","last_became_peered":"2024-04-25T10:08:10.892667+0000","last_unstale":"2024-04-25T10:08:10.901609+0000","last_undegraded":"2024-04-25T10:08:10.901609+0000","last_fullsized":"2024-04-25T10:08:10.901609+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:34.974943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:08.840704+0000","last_change":"2024-04-25T10:08:08.840704+0000","last_active":"2024-04-25T10:08:08.840704+0000","last_peered":"2024-04-25T10:08:08.840704+0000","last_clean":"2024-04-25T10:08:08.840704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:08.840704+0000","last_undegraded":"2024-04-25T10:08:08.840704+0000","last_fullsized":"2024-04-25T10:08:08.840704+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:08.840704+0000","last_change":"2024-04-25T10:08:08.840704+0000","last_active":"2024-04-25T10:08:08.840704+0000","last_peered":"2024-04-25T10:08:08.840704+0000","last_clean":"2024-04-25T10:08:08.840704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:08.840704+0000","last_undegraded":"2024-04-25T10:08:08.840704+0000","last_fullsized":"2024-04-25T10:08:08.840704+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.884018+0000","last_change":"2024-04-25T10:08:09.880232+0000","last_active":"2024-04-25T10:08:10.884018+0000","last_peered":"2024-04-25T10:08:10.884018+0000","last_clean":"2024-04-25T10:08:10.884018+0000","last_became_active":"2024-04-25T10:08:09.879917+0000","last_became_peered":"2024-04-25T10:08:09.879917+0000","last_unstale":"2024-04-25T10:08:10.884018+0000","last_undegraded":"2024-04-25T10:08:10.884018+0000","last_fullsized":"2024-04-25T10:08:10.884018+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:43.464391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889651+0000","last_change":"2024-04-25T10:08:10.893598+0000","last_active":"2024-04-25T10:08:11.889651+0000","last_peered":"2024-04-25T10:08:11.889651+0000","last_clean":"2024-04-25T10:08:11.889651+0000","last_became_active":"2024-04-25T10:08:10.893008+0000","last_became_peered":"2024-04-25T10:08:10.893008+0000","last_unstale":"2024-04-25T10:08:11.889651+0000","last_undegraded":"2024-04-25T10:08:11.889651+0000","last_fullsized":"2024-04-25T10:08:11.889651+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:57.430500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:08.840704+0000","last_change":"2024-04-25T10:08:08.840704+0000","last_active":"2024-04-25T10:08:08.840704+0000","last_peered":"2024-04-25T10:08:08.840704+0000","last_clean":"2024-04-25T10:08:08.840704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:08.840704+0000","last_undegraded":"2024-04-25T10:08:08.840704+0000","last_fullsized":"2024-04-25T10:08:08.840704+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:08.840704+0000","last_change":"2024-04-25T10:08:08.840704+0000","last_active":"2024-04-25T10:08:08.840704+0000","last_peered":"2024-04-25T10:08:08.840704+0000","last_clean":"2024-04-25T10:08:08.840704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:08.840704+0000","last_undegraded":"2024-04-25T10:08:08.840704+0000","last_fullsized":"2024-04-25T10:08:08.840704+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894725+0000","last_change":"2024-04-25T10:08:10.917026+0000","last_active":"2024-04-25T10:08:11.894725+0000","last_peered":"2024-04-25T10:08:11.894725+0000","last_clean":"2024-04-25T10:08:11.894725+0000","last_became_active":"2024-04-25T10:08:10.916406+0000","last_became_peered":"2024-04-25T10:08:10.916406+0000","last_unstale":"2024-04-25T10:08:11.894725+0000","last_undegraded":"2024-04-25T10:08:11.894725+0000","last_fullsized":"2024-04-25T10:08:11.894725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.834110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901694+0000","last_change":"2024-04-25T10:08:09.881070+0000","last_active":"2024-04-25T10:08:10.901694+0000","last_peered":"2024-04-25T10:08:10.901694+0000","last_clean":"2024-04-25T10:08:10.901694+0000","last_became_active":"2024-04-25T10:08:09.880757+0000","last_became_peered":"2024-04-25T10:08:09.880757+0000","last_unstale":"2024-04-25T10:08:10.901694+0000","last_undegraded":"2024-04-25T10:08:10.901694+0000","last_fullsized":"2024-04-25T10:08:10.901694+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:26.624159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967486+0000","last_change":"2024-04-25T10:08:09.899426+0000","last_active":"2024-04-25T10:08:11.967486+0000","last_peered":"2024-04-25T10:08:11.967486+0000","last_clean":"2024-04-25T10:08:11.967486+0000","last_became_active":"2024-04-25T10:08:09.898830+0000","last_became_peered":"2024-04-25T10:08:09.898830+0000","last_unstale":"2024-04-25T10:08:11.967486+0000","last_undegraded":"2024-04-25T10:08:11.967486+0000","last_fullsized":"2024-04-25T10:08:11.967486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:19.781296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893451+0000","last_change":"2024-04-25T10:08:09.902684+0000","last_active":"2024-04-25T10:08:11.893451+0000","last_peered":"2024-04-25T10:08:11.893451+0000","last_clean":"2024-04-25T10:08:11.893451+0000","last_became_active":"2024-04-25T10:08:09.901806+0000","last_became_peered":"2024-04-25T10:08:09.901806+0000","last_unstale":"2024-04-25T10:08:11.893451+0000","last_undegraded":"2024-04-25T10:08:11.893451+0000","last_fullsized":"2024-04-25T10:08:11.893451+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:17.199198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:08.840704+0000","last_change":"2024-04-25T10:08:08.840704+0000","last_active":"2024-04-25T10:08:08.840704+0000","last_peered":"2024-04-25T10:08:08.840704+0000","last_clean":"2024-04-25T10:08:08.840704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:08.840704+0000","last_undegraded":"2024-04-25T10:08:08.840704+0000","last_fullsized":"2024-04-25T10:08:08.840704+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.899177+0000","last_change":"2024-04-25T10:08:09.884420+0000","last_active":"2024-04-25T10:08:10.899177+0000","last_peered":"2024-04-25T10:08:10.899177+0000","last_clean":"2024-04-25T10:08:10.899177+0000","last_became_active":"2024-04-25T10:08:09.883427+0000","last_became_peered":"2024-04-25T10:08:09.883427+0000","last_unstale":"2024-04-25T10:08:10.899177+0000","last_undegraded":"2024-04-25T10:08:10.899177+0000","last_fullsized":"2024-04-25T10:08:10.899177+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:47.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894725+0000","last_change":"2024-04-25T10:08:10.918083+0000","last_active":"2024-04-25T10:08:11.894725+0000","last_peered":"2024-04-25T10:08:11.894725+0000","last_clean":"2024-04-25T10:08:11.894725+0000","last_became_active":"2024-04-25T10:08:10.917512+0000","last_became_peered":"2024-04-25T10:08:10.917512+0000","last_unstale":"2024-04-25T10:08:11.894725+0000","last_undegraded":"2024-04-25T10:08:11.894725+0000","last_fullsized":"2024-04-25T10:08:11.894725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:35.281267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.891599+0000","last_change":"2024-04-25T10:08:09.888804+0000","last_active":"2024-04-25T10:08:10.891599+0000","last_peered":"2024-04-25T10:08:10.891599+0000","last_clean":"2024-04-25T10:08:10.891599+0000","last_became_active":"2024-04-25T10:08:09.888488+0000","last_became_peered":"2024-04-25T10:08:09.888488+0000","last_unstale":"2024-04-25T10:08:10.891599+0000","last_undegraded":"2024-04-25T10:08:10.891599+0000","last_fullsized":"2024-04-25T10:08:10.891599+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:17.199198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.906640+0000","last_change":"2024-04-25T10:08:10.905832+0000","last_active":"2024-04-25T10:08:10.906640+0000","last_peered":"2024-04-25T10:08:10.906640+0000","last_clean":"2024-04-25T10:08:10.906640+0000","last_became_active":"2024-04-25T10:08:10.905524+0000","last_became_peered":"2024-04-25T10:08:10.905524+0000","last_unstale":"2024-04-25T10:08:10.906640+0000","last_undegraded":"2024-04-25T10:08:10.906640+0000","last_fullsized":"2024-04-25T10:08:10.906640+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:29.292616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893944+0000","last_change":"2024-04-25T10:08:09.902367+0000","last_active":"2024-04-25T10:08:11.893944+0000","last_peered":"2024-04-25T10:08:11.893944+0000","last_clean":"2024-04-25T10:08:11.893944+0000","last_became_active":"2024-04-25T10:08:09.900592+0000","last_became_peered":"2024-04-25T10:08:09.900592+0000","last_unstale":"2024-04-25T10:08:11.893944+0000","last_undegraded":"2024-04-25T10:08:11.893944+0000","last_fullsized":"2024-04-25T10:08:11.893944+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:38.492052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.878704+0000","last_change":"2024-04-25T10:08:09.890143+0000","last_active":"2024-04-25T10:08:10.878704+0000","last_peered":"2024-04-25T10:08:10.878704+0000","last_clean":"2024-04-25T10:08:10.878704+0000","last_became_active":"2024-04-25T10:08:09.889126+0000","last_became_peered":"2024-04-25T10:08:09.889126+0000","last_unstale":"2024-04-25T10:08:10.878704+0000","last_undegraded":"2024-04-25T10:08:10.878704+0000","last_fullsized":"2024-04-25T10:08:10.878704+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:38.492052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893893+0000","last_change":"2024-04-25T10:08:10.917214+0000","last_active":"2024-04-25T10:08:11.893893+0000","last_peered":"2024-04-25T10:08:11.893893+0000","last_clean":"2024-04-25T10:08:11.893893+0000","last_became_active":"2024-04-25T10:08:10.915770+0000","last_became_peered":"2024-04-25T10:08:10.915770+0000","last_unstale":"2024-04-25T10:08:11.893893+0000","last_undegraded":"2024-04-25T10:08:11.893893+0000","last_fullsized":"2024-04-25T10:08:11.893893+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:44.066536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.907624+0000","last_change":"2024-04-25T10:08:09.889071+0000","last_active":"2024-04-25T10:08:10.907624+0000","last_peered":"2024-04-25T10:08:10.907624+0000","last_clean":"2024-04-25T10:08:10.907624+0000","last_became_active":"2024-04-25T10:08:09.888746+0000","last_became_peered":"2024-04-25T10:08:09.888746+0000","last_unstale":"2024-04-25T10:08:10.907624+0000","last_undegraded":"2024-04-25T10:08:10.907624+0000","last_fullsized":"2024-04-25T10:08:10.907624+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:43.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.895295+0000","last_change":"2024-04-25T10:08:10.893599+0000","last_active":"2024-04-25T10:08:10.895295+0000","last_peered":"2024-04-25T10:08:10.895295+0000","last_clean":"2024-04-25T10:08:10.895295+0000","last_became_active":"2024-04-25T10:08:10.893206+0000","last_became_peered":"2024-04-25T10:08:10.893206+0000","last_unstale":"2024-04-25T10:08:10.895295+0000","last_undegraded":"2024-04-25T10:08:10.895295+0000","last_fullsized":"2024-04-25T10:08:10.895295+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:51.143246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889836+0000","last_change":"2024-04-25T10:08:09.880919+0000","last_active":"2024-04-25T10:08:11.889836+0000","last_peered":"2024-04-25T10:08:11.889836+0000","last_clean":"2024-04-25T10:08:11.889836+0000","last_became_active":"2024-04-25T10:08:09.880611+0000","last_became_peered":"2024-04-25T10:08:09.880611+0000","last_unstale":"2024-04-25T10:08:11.889836+0000","last_undegraded":"2024-04-25T10:08:11.889836+0000","last_fullsized":"2024-04-25T10:08:11.889836+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:22.155681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.902841+0000","last_change":"2024-04-25T10:08:10.901150+0000","last_active":"2024-04-25T10:08:10.902841+0000","last_peered":"2024-04-25T10:08:10.902841+0000","last_clean":"2024-04-25T10:08:10.902841+0000","last_became_active":"2024-04-25T10:08:10.900624+0000","last_became_peered":"2024-04-25T10:08:10.900624+0000","last_unstale":"2024-04-25T10:08:10.902841+0000","last_undegraded":"2024-04-25T10:08:10.902841+0000","last_fullsized":"2024-04-25T10:08:10.902841+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:08.745250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.892866+0000","last_change":"2024-04-25T10:08:09.901094+0000","last_active":"2024-04-25T10:08:11.892866+0000","last_peered":"2024-04-25T10:08:11.892866+0000","last_clean":"2024-04-25T10:08:11.892866+0000","last_became_active":"2024-04-25T10:08:09.900405+0000","last_became_peered":"2024-04-25T10:08:09.900405+0000","last_unstale":"2024-04-25T10:08:11.892866+0000","last_undegraded":"2024-04-25T10:08:11.892866+0000","last_fullsized":"2024-04-25T10:08:11.892866+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:30.789048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893987+0000","last_change":"2024-04-25T10:08:10.909724+0000","last_active":"2024-04-25T10:08:11.893987+0000","last_peered":"2024-04-25T10:08:11.893987+0000","last_clean":"2024-04-25T10:08:11.893987+0000","last_became_active":"2024-04-25T10:08:10.904979+0000","last_became_peered":"2024-04-25T10:08:10.904979+0000","last_unstale":"2024-04-25T10:08:11.893987+0000","last_undegraded":"2024-04-25T10:08:11.893987+0000","last_fullsized":"2024-04-25T10:08:11.893987+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:17.598272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889987+0000","last_change":"2024-04-25T10:08:09.885875+0000","last_active":"2024-04-25T10:08:11.889987+0000","last_peered":"2024-04-25T10:08:11.889987+0000","last_clean":"2024-04-25T10:08:11.889987+0000","last_became_active":"2024-04-25T10:08:09.885529+0000","last_became_peered":"2024-04-25T10:08:09.885529+0000","last_unstale":"2024-04-25T10:08:11.889987+0000","last_undegraded":"2024-04-25T10:08:11.889987+0000","last_fullsized":"2024-04-25T10:08:11.889987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:30.869245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.895037+0000","last_change":"2024-04-25T10:08:10.923948+0000","last_active":"2024-04-25T10:08:11.895037+0000","last_peered":"2024-04-25T10:08:11.895037+0000","last_clean":"2024-04-25T10:08:11.895037+0000","last_became_active":"2024-04-25T10:08:10.923383+0000","last_became_peered":"2024-04-25T10:08:10.923383+0000","last_unstale":"2024-04-25T10:08:11.895037+0000","last_undegraded":"2024-04-25T10:08:11.895037+0000","last_fullsized":"2024-04-25T10:08:11.895037+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:55.065275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.879776+0000","last_change":"2024-04-25T10:08:09.894894+0000","last_active":"2024-04-25T10:08:10.879776+0000","last_peered":"2024-04-25T10:08:10.879776+0000","last_clean":"2024-04-25T10:08:10.879776+0000","last_became_active":"2024-04-25T10:08:09.894350+0000","last_became_peered":"2024-04-25T10:08:09.894350+0000","last_unstale":"2024-04-25T10:08:10.879776+0000","last_undegraded":"2024-04-25T10:08:10.879776+0000","last_fullsized":"2024-04-25T10:08:10.879776+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.592539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889925+0000","last_change":"2024-04-25T10:08:10.893322+0000","last_active":"2024-04-25T10:08:11.889925+0000","last_peered":"2024-04-25T10:08:11.889925+0000","last_clean":"2024-04-25T10:08:11.889925+0000","last_became_active":"2024-04-25T10:08:10.892639+0000","last_became_peered":"2024-04-25T10:08:10.892639+0000","last_unstale":"2024-04-25T10:08:11.889925+0000","last_undegraded":"2024-04-25T10:08:11.889925+0000","last_fullsized":"2024-04-25T10:08:11.889925+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:37.445694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967957+0000","last_change":"2024-04-25T10:08:09.893305+0000","last_active":"2024-04-25T10:08:11.967957+0000","last_peered":"2024-04-25T10:08:11.967957+0000","last_clean":"2024-04-25T10:08:11.967957+0000","last_became_active":"2024-04-25T10:08:09.892501+0000","last_became_peered":"2024-04-25T10:08:09.892501+0000","last_unstale":"2024-04-25T10:08:11.967957+0000","last_undegraded":"2024-04-25T10:08:11.967957+0000","last_fullsized":"2024-04-25T10:08:11.967957+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:21.522866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.926945+0000","last_change":"2024-04-25T10:08:10.926260+0000","last_active":"2024-04-25T10:08:10.926945+0000","last_peered":"2024-04-25T10:08:10.926945+0000","last_clean":"2024-04-25T10:08:10.926945+0000","last_became_active":"2024-04-25T10:08:10.926038+0000","last_became_peered":"2024-04-25T10:08:10.926038+0000","last_unstale":"2024-04-25T10:08:10.926945+0000","last_undegraded":"2024-04-25T10:08:10.926945+0000","last_fullsized":"2024-04-25T10:08:10.926945+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:44.066536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894376+0000","last_change":"2024-04-25T10:08:09.906942+0000","last_active":"2024-04-25T10:08:11.894376+0000","last_peered":"2024-04-25T10:08:11.894376+0000","last_clean":"2024-04-25T10:08:11.894376+0000","last_became_active":"2024-04-25T10:08:09.906192+0000","last_became_peered":"2024-04-25T10:08:09.906192+0000","last_unstale":"2024-04-25T10:08:11.894376+0000","last_undegraded":"2024-04-25T10:08:11.894376+0000","last_fullsized":"2024-04-25T10:08:11.894376+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:48.349703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.899168+0000","last_change":"2024-04-25T10:08:10.890698+0000","last_active":"2024-04-25T10:08:10.899168+0000","last_peered":"2024-04-25T10:08:10.899168+0000","last_clean":"2024-04-25T10:08:10.899168+0000","last_became_active":"2024-04-25T10:08:10.889982+0000","last_became_peered":"2024-04-25T10:08:10.889982+0000","last_unstale":"2024-04-25T10:08:10.899168+0000","last_undegraded":"2024-04-25T10:08:10.899168+0000","last_fullsized":"2024-04-25T10:08:10.899168+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:55.065275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889624+0000","last_change":"2024-04-25T10:08:09.902559+0000","last_active":"2024-04-25T10:08:11.889624+0000","last_peered":"2024-04-25T10:08:11.889624+0000","last_clean":"2024-04-25T10:08:11.889624+0000","last_became_active":"2024-04-25T10:08:09.902240+0000","last_became_peered":"2024-04-25T10:08:09.902240+0000","last_unstale":"2024-04-25T10:08:11.889624+0000","last_undegraded":"2024-04-25T10:08:11.889624+0000","last_fullsized":"2024-04-25T10:08:11.889624+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:30.202367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:09.849679+0000","last_change":"2024-04-25T10:08:09.849679+0000","last_active":"2024-04-25T10:08:09.849679+0000","last_peered":"2024-04-25T10:08:09.849679+0000","last_clean":"2024-04-25T10:08:09.849679+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:09.849679+0000","last_undegraded":"2024-04-25T10:08:09.849679+0000","last_fullsized":"2024-04-25T10:08:09.849679+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:08:08.840704+0000","last_change":"2024-04-25T10:08:08.840704+0000","last_active":"2024-04-25T10:08:08.840704+0000","last_peered":"2024-04-25T10:08:08.840704+0000","last_clean":"2024-04-25T10:08:08.840704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:08.840704+0000","last_undegraded":"2024-04-25T10:08:08.840704+0000","last_fullsized":"2024-04-25T10:08:08.840704+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889431+0000","last_change":"2024-04-25T10:08:10.894492+0000","last_active":"2024-04-25T10:08:11.889431+0000","last_peered":"2024-04-25T10:08:11.889431+0000","last_clean":"2024-04-25T10:08:11.889431+0000","last_became_active":"2024-04-25T10:08:10.894221+0000","last_became_peered":"2024-04-25T10:08:10.894221+0000","last_unstale":"2024-04-25T10:08:11.889431+0000","last_undegraded":"2024-04-25T10:08:11.889431+0000","last_fullsized":"2024-04-25T10:08:11.889431+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:47.897507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889533+0000","last_change":"2024-04-25T10:08:09.890601+0000","last_active":"2024-04-25T10:08:11.889533+0000","last_peered":"2024-04-25T10:08:11.889533+0000","last_clean":"2024-04-25T10:08:11.889533+0000","last_became_active":"2024-04-25T10:08:09.890208+0000","last_became_peered":"2024-04-25T10:08:09.890208+0000","last_unstale":"2024-04-25T10:08:11.889533+0000","last_undegraded":"2024-04-25T10:08:11.889533+0000","last_fullsized":"2024-04-25T10:08:11.889533+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:01.465554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901843+0000","last_change":"2024-04-25T10:08:10.898030+0000","last_active":"2024-04-25T10:08:10.901843+0000","last_peered":"2024-04-25T10:08:10.901843+0000","last_clean":"2024-04-25T10:08:10.901843+0000","last_became_active":"2024-04-25T10:08:10.897719+0000","last_became_peered":"2024-04-25T10:08:10.897719+0000","last_unstale":"2024-04-25T10:08:10.901843+0000","last_undegraded":"2024-04-25T10:08:10.901843+0000","last_fullsized":"2024-04-25T10:08:10.901843+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:18.216787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.879881+0000","last_change":"2024-04-25T10:08:09.900914+0000","last_active":"2024-04-25T10:08:10.879881+0000","last_peered":"2024-04-25T10:08:10.879881+0000","last_clean":"2024-04-25T10:08:10.879881+0000","last_became_active":"2024-04-25T10:08:09.900669+0000","last_became_peered":"2024-04-25T10:08:09.900669+0000","last_unstale":"2024-04-25T10:08:10.879881+0000","last_undegraded":"2024-04-25T10:08:10.879881+0000","last_fullsized":"2024-04-25T10:08:10.879881+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.736275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901873+0000","last_change":"2024-04-25T10:08:10.894360+0000","last_active":"2024-04-25T10:08:10.901873+0000","last_peered":"2024-04-25T10:08:10.901873+0000","last_clean":"2024-04-25T10:08:10.901873+0000","last_became_active":"2024-04-25T10:08:10.894017+0000","last_became_peered":"2024-04-25T10:08:10.894017+0000","last_unstale":"2024-04-25T10:08:10.901873+0000","last_undegraded":"2024-04-25T10:08:10.901873+0000","last_fullsized":"2024-04-25T10:08:10.901873+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:43.207466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967562+0000","last_change":"2024-04-25T10:08:09.909488+0000","last_active":"2024-04-25T10:08:11.967562+0000","last_peered":"2024-04-25T10:08:11.967562+0000","last_clean":"2024-04-25T10:08:11.967562+0000","last_became_active":"2024-04-25T10:08:09.908863+0000","last_became_peered":"2024-04-25T10:08:09.908863+0000","last_unstale":"2024-04-25T10:08:11.967562+0000","last_undegraded":"2024-04-25T10:08:11.967562+0000","last_fullsized":"2024-04-25T10:08:11.967562+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:48.865163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":229,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:08:07.503843+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:08:07.503843+0000","last_peered":"2024-04-25T10:08:07.503843+0000","last_clean":"2024-04-25T10:08:07.503843+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:08:07.503843+0000","last_undegraded":"2024-04-25T10:08:07.503843+0000","last_fullsized":"2024-04-25T10:08:07.503843+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":708,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":50,"acting":50,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1232896,"data_stored":851110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31816,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":638976,"data_stored":260742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542183,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":688128,"data_stored":285901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":286553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542184,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36012,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335828,"statfs":{"total":96636764160,"available":96599887872,"internally_reserved":0,"allocated":684032,"data_stored":279657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":684032,"data_stored":279657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36004,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335836,"statfs":{"total":96636764160,"available":96599896064,"internally_reserved":0,"allocated":679936,"data_stored":279691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36596,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335244,"statfs":{"total":96636764160,"available":96599289856,"internally_reserved":0,"allocated":1286144,"data_stored":872163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:14.285 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:14.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.442+0000 7fd1d0b8d640 1 Processor -- start 2024-04-25T10:08:14.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.442+0000 7fd1d0b8d640 1 -- start start 2024-04-25T10:08:14.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.442+0000 7fd1d0b8d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1cc15dcc0 0x7fd1cc0b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.442+0000 7fd1d0b8d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1cc0b1cc0 0x7fd1cc0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.442+0000 7fd1d0b8d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1cc0b5310 con 0x7fd1cc06ac10 2024-04-25T10:08:14.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.442+0000 7fd1d0b8d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1cc0b5b70 con 0x7fd1cc0b1cc0 2024-04-25T10:08:14.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.442+0000 7fd1d0b8d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0aefa0 con 0x7fd1cc15dcc0 2024-04-25T10:08:14.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.442+0000 7fd1ca59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd1cc06ac10 0x7fd1cc06aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41512/0 (socket says 172.21.15.60:41512) 2024-04-25T10:08:14.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.442+0000 7fd1cad9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1cc0b1cc0 0x7fd1cc0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.442+0000 7fd1ca59c640 1 -- 172.21.15.60:0/446940315 learned_addr learned my addr 172.21.15.60:0/446940315 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:14.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.442+0000 7fd1c9d9b640 1 --2- 172.21.15.60:0/446940315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1cc15dcc0 0x7fd1cc0b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.442+0000 7fd1cad9d640 1 -- 172.21.15.60:0/446940315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1cc15dcc0 msgr2=0x7fd1cc0b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:14.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.442+0000 7fd1cad9d640 1 --2- 172.21.15.60:0/446940315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1cc15dcc0 0x7fd1cc0b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.442+0000 7fd1cad9d640 1 -- 172.21.15.60:0/446940315 >> v1:172.21.15.60:6789/0 conn(0x7fd1cc06ac10 legacy=0x7fd1cc06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.442+0000 7fd1c959a640 1 -- 172.21.15.60:0/446940315 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715647044 0 0) 0x7fd1cc0b5310 con 0x7fd1cc06ac10 2024-04-25T10:08:14.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.442+0000 7fd1cad9d640 1 -- 172.21.15.60:0/446940315 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc0b55b0 con 0x7fd1cc0b1cc0 2024-04-25T10:08:14.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.446+0000 7fd1cad9d640 1 --2- 172.21.15.60:0/446940315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1cc0b1cc0 0x7fd1cc0b06c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40097c0 tx=0x7fd1c401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fc5dbad2ec92f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.446+0000 7fd1c959a640 1 -- 172.21.15.60:0/446940315 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40b2070 con 0x7fd1cc0b1cc0 2024-04-25T10:08:14.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.446+0000 7fd1c959a640 1 -- 172.21.15.60:0/446940315 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c40ad640 con 0x7fd1cc0b1cc0 2024-04-25T10:08:14.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.446+0000 7fd1c959a640 1 -- 172.21.15.60:0/446940315 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40ada10 con 0x7fd1cc0b1cc0 2024-04-25T10:08:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.446+0000 7fd1d0b8d640 1 -- 172.21.15.60:0/446940315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1cc0b1cc0 msgr2=0x7fd1cc0b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.446+0000 7fd1d0b8d640 1 --2- 172.21.15.60:0/446940315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1cc0b1cc0 0x7fd1cc0b06c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40097c0 tx=0x7fd1c401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.446+0000 7fd1d0b8d640 1 -- 172.21.15.60:0/446940315 shutdown_connections 2024-04-25T10:08:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.446+0000 7fd1d0b8d640 1 --2- 172.21.15.60:0/446940315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1cc0b1cc0 0x7fd1cc0b06c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.446+0000 7fd1d0b8d640 1 --2- 172.21.15.60:0/446940315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1cc15dcc0 0x7fd1cc0b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.446+0000 7fd1d0b8d640 1 -- 172.21.15.60:0/446940315 >> 172.21.15.60:0/446940315 conn(0x7fd1cc072920 msgr2=0x7fd1cc099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.446+0000 7fd1d0b8d640 1 -- 172.21.15.60:0/446940315 shutdown_connections 2024-04-25T10:08:14.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.446+0000 7fd1d0b8d640 1 -- 172.21.15.60:0/446940315 wait complete. 2024-04-25T10:08:14.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1d0b8d640 1 Processor -- start 2024-04-25T10:08:14.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1d0b8d640 1 -- start start 2024-04-25T10:08:14.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1d0b8d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1cc06ac10 0x7fd1cc147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1d0b8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1cc0b1cc0 0x7fd1cc14fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1d0b8d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1cc15dcc0 0x7fd1cc1575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1d0b8d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0b3e50 con 0x7fd1cc06ac10 2024-04-25T10:08:14.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1d0b8d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0b3cd0 con 0x7fd1cc0b1cc0 2024-04-25T10:08:14.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1d0b8d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc16a630 con 0x7fd1cc15dcc0 2024-04-25T10:08:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1ca59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1cc06ac10 0x7fd1cc147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1ca59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1cc06ac10 0x7fd1cc147a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58788/0 (socket says 172.21.15.60:58788) 2024-04-25T10:08:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1ca59c640 1 -- 172.21.15.60:0/1969407317 learned_addr learned my addr 172.21.15.60:0/1969407317 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1c9d9b640 1 --2- 172.21.15.60:0/1969407317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1cc0b1cc0 0x7fd1cc14fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1cad9d640 1 --2- 172.21.15.60:0/1969407317 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1cc15dcc0 0x7fd1cc1575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1ca59c640 1 -- 172.21.15.60:0/1969407317 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1cc15dcc0 msgr2=0x7fd1cc1575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1ca59c640 1 --2- 172.21.15.60:0/1969407317 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1cc15dcc0 0x7fd1cc1575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1ca59c640 1 -- 172.21.15.60:0/1969407317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1cc0b1cc0 msgr2=0x7fd1cc14fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1ca59c640 1 --2- 172.21.15.60:0/1969407317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1cc0b1cc0 0x7fd1cc14fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1ca59c640 1 -- 172.21.15.60:0/1969407317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc15d890 con 0x7fd1cc06ac10 2024-04-25T10:08:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1c9d9b640 1 --2- 172.21.15.60:0/1969407317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1cc0b1cc0 0x7fd1cc14fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1ca59c640 1 --2- 172.21.15.60:0/1969407317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1cc06ac10 0x7fd1cc147a90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc001570 tx=0x7fd1bc00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1baffd640 1 -- 172.21.15.60:0/1969407317 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc018070 con 0x7fd1cc06ac10 2024-04-25T10:08:14.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1baffd640 1 -- 172.21.15.60:0/1969407317 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1bc00a640 con 0x7fd1cc06ac10 2024-04-25T10:08:14.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1baffd640 1 -- 172.21.15.60:0/1969407317 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc00aa30 con 0x7fd1cc06ac10 2024-04-25T10:08:14.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1d0b8d640 1 -- 172.21.15.60:0/1969407317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1cc14a8a0 con 0x7fd1cc06ac10 2024-04-25T10:08:14.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1d0b8d640 1 -- 172.21.15.60:0/1969407317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc1507f0 con 0x7fd1cc06ac10 2024-04-25T10:08:14.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1d0b8d640 1 -- 172.21.15.60:0/1969407317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1cc074770 con 0x7fd1cc06ac10 2024-04-25T10:08:14.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.450+0000 7fd1baffd640 1 -- 172.21.15.60:0/1969407317 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd1bc00abd0 con 0x7fd1cc06ac10 2024-04-25T10:08:14.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.454+0000 7fd1baffd640 1 --2- 172.21.15.60:0/1969407317 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd1ac066f60 0x7fd1ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.454+0000 7fd1baffd640 1 -- 172.21.15.60:0/1969407317 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1bc084760 con 0x7fd1cc06ac10 2024-04-25T10:08:14.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.454+0000 7fd1c9d9b640 1 --2- 172.21.15.60:0/1969407317 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd1ac066f60 0x7fd1ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.454+0000 7fd1c9d9b640 1 --2- 172.21.15.60:0/1969407317 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd1ac066f60 0x7fd1ac069420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc1f9e60 tx=0x7fd1c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.458+0000 7fd1baffd640 1 -- 172.21.15.60:0/1969407317 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd1bc055770 con 0x7fd1cc06ac10 2024-04-25T10:08:14.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.618+0000 7fd1d0b8d640 1 -- 172.21.15.60:0/1969407317 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1cc072670 con 0x7fd1ac066f60 2024-04-25T10:08:14.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.634+0000 7fd1baffd640 1 -- 172.21.15.60:0/1969407317 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251948 (secure 0 0 0) 0x7fd1cc072670 con 0x7fd1ac066f60 2024-04-25T10:08:14.644 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:14.646 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:08:14.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.638+0000 7fd1d0b8d640 1 -- 172.21.15.60:0/1969407317 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd1ac066f60 msgr2=0x7fd1ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.638+0000 7fd1d0b8d640 1 --2- 172.21.15.60:0/1969407317 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd1ac066f60 0x7fd1ac069420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc1f9e60 tx=0x7fd1c0008040 comp rx=0 tx=0).stop 2024-04-25T10:08:14.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.638+0000 7fd1d0b8d640 1 -- 172.21.15.60:0/1969407317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1cc06ac10 msgr2=0x7fd1cc147a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.638+0000 7fd1d0b8d640 1 --2- 172.21.15.60:0/1969407317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1cc06ac10 0x7fd1cc147a90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc001570 tx=0x7fd1bc00cd20 comp rx=0 tx=0).stop 2024-04-25T10:08:14.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.638+0000 7fd1d0b8d640 1 -- 172.21.15.60:0/1969407317 shutdown_connections 2024-04-25T10:08:14.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.638+0000 7fd1d0b8d640 1 --2- 172.21.15.60:0/1969407317 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd1ac066f60 0x7fd1ac069420 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.638+0000 7fd1d0b8d640 1 --2- 172.21.15.60:0/1969407317 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1cc15dcc0 0x7fd1cc1575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.638+0000 7fd1d0b8d640 1 --2- 172.21.15.60:0/1969407317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1cc0b1cc0 0x7fd1cc14fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.638+0000 7fd1d0b8d640 1 --2- 172.21.15.60:0/1969407317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1cc06ac10 0x7fd1cc147a90 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.638+0000 7fd1d0b8d640 1 -- 172.21.15.60:0/1969407317 >> 172.21.15.60:0/1969407317 conn(0x7fd1cc072920 msgr2=0x7fd1cc152540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:14.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.638+0000 7fd1d0b8d640 1 -- 172.21.15.60:0/1969407317 shutdown_connections 2024-04-25T10:08:14.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.638+0000 7fd1d0b8d640 1 -- 172.21.15.60:0/1969407317 wait complete. 2024-04-25T10:08:14.665 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-04-25T10:08:14.447680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260156,"kb_used_data":6540,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754714564,"statfs":{"total":773094113280,"available":772827713536,"internally_reserved":0,"allocated":6696960,"data_stored":3441264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.596128"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.896431+0000","last_change":"2024-04-25T10:08:10.894975+0000","last_active":"2024-04-25T10:08:10.896431+0000","last_peered":"2024-04-25T10:08:10.896431+0000","last_clean":"2024-04-25T10:08:10.896431+0000","last_became_active":"2024-04-25T10:08:10.894724+0000","last_became_peered":"2024-04-25T10:08:10.894724+0000","last_unstale":"2024-04-25T10:08:10.896431+0000","last_undegraded":"2024-04-25T10:08:10.896431+0000","last_fullsized":"2024-04-25T10:08:10.896431+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:57.357697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901577+0000","last_change":"2024-04-25T10:08:10.893083+0000","last_active":"2024-04-25T10:08:10.901577+0000","last_peered":"2024-04-25T10:08:10.901577+0000","last_clean":"2024-04-25T10:08:10.901577+0000","last_became_active":"2024-04-25T10:08:10.892785+0000","last_became_peered":"2024-04-25T10:08:10.892785+0000","last_unstale":"2024-04-25T10:08:10.901577+0000","last_undegraded":"2024-04-25T10:08:10.901577+0000","last_fullsized":"2024-04-25T10:08:10.901577+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:31.332615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396629+0000","last_change":"2024-04-25T10:08:10.916953+0000","last_active":"2024-04-25T10:08:13.396629+0000","last_peered":"2024-04-25T10:08:13.396629+0000","last_clean":"2024-04-25T10:08:13.396629+0000","last_became_active":"2024-04-25T10:08:10.910705+0000","last_became_peered":"2024-04-25T10:08:10.910705+0000","last_unstale":"2024-04-25T10:08:13.396629+0000","last_undegraded":"2024-04-25T10:08:13.396629+0000","last_fullsized":"2024-04-25T10:08:13.396629+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:04.768312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893692+0000","last_change":"2024-04-25T10:08:10.901432+0000","last_active":"2024-04-25T10:08:11.893692+0000","last_peered":"2024-04-25T10:08:11.893692+0000","last_clean":"2024-04-25T10:08:11.893692+0000","last_became_active":"2024-04-25T10:08:10.900367+0000","last_became_peered":"2024-04-25T10:08:10.900367+0000","last_unstale":"2024-04-25T10:08:11.893692+0000","last_undegraded":"2024-04-25T10:08:11.893692+0000","last_fullsized":"2024-04-25T10:08:11.893692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:27.633134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.910107+0000","last_change":"2024-04-25T10:08:10.909538+0000","last_active":"2024-04-25T10:08:10.910107+0000","last_peered":"2024-04-25T10:08:10.910107+0000","last_clean":"2024-04-25T10:08:10.910107+0000","last_became_active":"2024-04-25T10:08:10.909284+0000","last_became_peered":"2024-04-25T10:08:10.909284+0000","last_unstale":"2024-04-25T10:08:10.910107+0000","last_undegraded":"2024-04-25T10:08:10.910107+0000","last_fullsized":"2024-04-25T10:08:10.910107+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:36.225766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.889260+0000","last_change":"2024-04-25T10:08:10.887190+0000","last_active":"2024-04-25T10:08:10.889260+0000","last_peered":"2024-04-25T10:08:10.889260+0000","last_clean":"2024-04-25T10:08:10.889260+0000","last_became_active":"2024-04-25T10:08:10.886910+0000","last_became_peered":"2024-04-25T10:08:10.886910+0000","last_unstale":"2024-04-25T10:08:10.889260+0000","last_undegraded":"2024-04-25T10:08:10.889260+0000","last_fullsized":"2024-04-25T10:08:10.889260+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:33.123610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893402+0000","last_change":"2024-04-25T10:08:10.917873+0000","last_active":"2024-04-25T10:08:11.893402+0000","last_peered":"2024-04-25T10:08:11.893402+0000","last_clean":"2024-04-25T10:08:11.893402+0000","last_became_active":"2024-04-25T10:08:10.916140+0000","last_became_peered":"2024-04-25T10:08:10.916140+0000","last_unstale":"2024-04-25T10:08:11.893402+0000","last_undegraded":"2024-04-25T10:08:11.893402+0000","last_fullsized":"2024-04-25T10:08:11.893402+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:12.105474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.906628+0000","last_change":"2024-04-25T10:08:10.905892+0000","last_active":"2024-04-25T10:08:10.906628+0000","last_peered":"2024-04-25T10:08:10.906628+0000","last_clean":"2024-04-25T10:08:10.906628+0000","last_became_active":"2024-04-25T10:08:10.905443+0000","last_became_peered":"2024-04-25T10:08:10.905443+0000","last_unstale":"2024-04-25T10:08:10.906628+0000","last_undegraded":"2024-04-25T10:08:10.906628+0000","last_fullsized":"2024-04-25T10:08:10.906628+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:09.407230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894567+0000","last_change":"2024-04-25T10:08:10.884528+0000","last_active":"2024-04-25T10:08:11.894567+0000","last_peered":"2024-04-25T10:08:11.894567+0000","last_clean":"2024-04-25T10:08:11.894567+0000","last_became_active":"2024-04-25T10:08:10.884183+0000","last_became_peered":"2024-04-25T10:08:10.884183+0000","last_unstale":"2024-04-25T10:08:11.894567+0000","last_undegraded":"2024-04-25T10:08:11.894567+0000","last_fullsized":"2024-04-25T10:08:11.894567+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:42.851719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889904+0000","last_change":"2024-04-25T10:08:10.894120+0000","last_active":"2024-04-25T10:08:11.889904+0000","last_peered":"2024-04-25T10:08:11.889904+0000","last_clean":"2024-04-25T10:08:11.889904+0000","last_became_active":"2024-04-25T10:08:10.893807+0000","last_became_peered":"2024-04-25T10:08:10.893807+0000","last_unstale":"2024-04-25T10:08:11.889904+0000","last_undegraded":"2024-04-25T10:08:11.889904+0000","last_fullsized":"2024-04-25T10:08:11.889904+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:10.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.890117+0000","last_change":"2024-04-25T10:08:10.893934+0000","last_active":"2024-04-25T10:08:11.890117+0000","last_peered":"2024-04-25T10:08:11.890117+0000","last_clean":"2024-04-25T10:08:11.890117+0000","last_became_active":"2024-04-25T10:08:10.893270+0000","last_became_peered":"2024-04-25T10:08:10.893270+0000","last_unstale":"2024-04-25T10:08:11.890117+0000","last_undegraded":"2024-04-25T10:08:11.890117+0000","last_fullsized":"2024-04-25T10:08:11.890117+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:13.537677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894126+0000","last_change":"2024-04-25T10:08:10.923912+0000","last_active":"2024-04-25T10:08:11.894126+0000","last_peered":"2024-04-25T10:08:11.894126+0000","last_clean":"2024-04-25T10:08:11.894126+0000","last_became_active":"2024-04-25T10:08:10.923319+0000","last_became_peered":"2024-04-25T10:08:10.923319+0000","last_unstale":"2024-04-25T10:08:11.894126+0000","last_undegraded":"2024-04-25T10:08:11.894126+0000","last_fullsized":"2024-04-25T10:08:11.894126+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:44.473365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.968119+0000","last_change":"2024-04-25T10:08:10.886846+0000","last_active":"2024-04-25T10:08:11.968119+0000","last_peered":"2024-04-25T10:08:11.968119+0000","last_clean":"2024-04-25T10:08:11.968119+0000","last_became_active":"2024-04-25T10:08:10.886497+0000","last_became_peered":"2024-04-25T10:08:10.886497+0000","last_unstale":"2024-04-25T10:08:11.968119+0000","last_undegraded":"2024-04-25T10:08:11.968119+0000","last_fullsized":"2024-04-25T10:08:11.968119+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:56.861799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893740+0000","last_change":"2024-04-25T10:08:10.907514+0000","last_active":"2024-04-25T10:08:11.893740+0000","last_peered":"2024-04-25T10:08:11.893740+0000","last_clean":"2024-04-25T10:08:11.893740+0000","last_became_active":"2024-04-25T10:08:10.905752+0000","last_became_peered":"2024-04-25T10:08:10.905752+0000","last_unstale":"2024-04-25T10:08:11.893740+0000","last_undegraded":"2024-04-25T10:08:11.893740+0000","last_fullsized":"2024-04-25T10:08:11.893740+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:15.349210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893812+0000","last_change":"2024-04-25T10:08:10.882723+0000","last_active":"2024-04-25T10:08:11.893812+0000","last_peered":"2024-04-25T10:08:11.893812+0000","last_clean":"2024-04-25T10:08:11.893812+0000","last_became_active":"2024-04-25T10:08:10.882339+0000","last_became_peered":"2024-04-25T10:08:10.882339+0000","last_unstale":"2024-04-25T10:08:11.893812+0000","last_undegraded":"2024-04-25T10:08:11.893812+0000","last_fullsized":"2024-04-25T10:08:11.893812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:13.179674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.897978+0000","last_change":"2024-04-25T10:08:10.896865+0000","last_active":"2024-04-25T10:08:10.897978+0000","last_peered":"2024-04-25T10:08:10.897978+0000","last_clean":"2024-04-25T10:08:10.897978+0000","last_became_active":"2024-04-25T10:08:10.896575+0000","last_became_peered":"2024-04-25T10:08:10.896575+0000","last_unstale":"2024-04-25T10:08:10.897978+0000","last_undegraded":"2024-04-25T10:08:10.897978+0000","last_fullsized":"2024-04-25T10:08:10.897978+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:31.167704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901912+0000","last_change":"2024-04-25T10:08:10.898151+0000","last_active":"2024-04-25T10:08:10.901912+0000","last_peered":"2024-04-25T10:08:10.901912+0000","last_clean":"2024-04-25T10:08:10.901912+0000","last_became_active":"2024-04-25T10:08:10.897849+0000","last_became_peered":"2024-04-25T10:08:10.897849+0000","last_unstale":"2024-04-25T10:08:10.901912+0000","last_undegraded":"2024-04-25T10:08:10.901912+0000","last_fullsized":"2024-04-25T10:08:10.901912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:47.458907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396698+0000","last_change":"2024-04-25T10:08:10.898156+0000","last_active":"2024-04-25T10:08:13.396698+0000","last_peered":"2024-04-25T10:08:13.396698+0000","last_clean":"2024-04-25T10:08:13.396698+0000","last_became_active":"2024-04-25T10:08:10.897326+0000","last_became_peered":"2024-04-25T10:08:10.897326+0000","last_unstale":"2024-04-25T10:08:13.396698+0000","last_undegraded":"2024-04-25T10:08:13.396698+0000","last_fullsized":"2024-04-25T10:08:13.396698+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:03.978144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.895610+0000","last_change":"2024-04-25T10:08:10.891553+0000","last_active":"2024-04-25T10:08:10.895610+0000","last_peered":"2024-04-25T10:08:10.895610+0000","last_clean":"2024-04-25T10:08:10.895610+0000","last_became_active":"2024-04-25T10:08:10.891348+0000","last_became_peered":"2024-04-25T10:08:10.891348+0000","last_unstale":"2024-04-25T10:08:10.895610+0000","last_undegraded":"2024-04-25T10:08:10.895610+0000","last_fullsized":"2024-04-25T10:08:10.895610+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:48.658370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.897906+0000","last_change":"2024-04-25T10:08:10.890188+0000","last_active":"2024-04-25T10:08:10.897906+0000","last_peered":"2024-04-25T10:08:10.897906+0000","last_clean":"2024-04-25T10:08:10.897906+0000","last_became_active":"2024-04-25T10:08:10.889827+0000","last_became_peered":"2024-04-25T10:08:10.889827+0000","last_unstale":"2024-04-25T10:08:10.897906+0000","last_undegraded":"2024-04-25T10:08:10.897906+0000","last_fullsized":"2024-04-25T10:08:10.897906+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.834110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396312+0000","last_change":"2024-04-25T10:08:10.930150+0000","last_active":"2024-04-25T10:08:13.396312+0000","last_peered":"2024-04-25T10:08:13.396312+0000","last_clean":"2024-04-25T10:08:13.396312+0000","last_became_active":"2024-04-25T10:08:10.929406+0000","last_became_peered":"2024-04-25T10:08:10.929406+0000","last_unstale":"2024-04-25T10:08:13.396312+0000","last_undegraded":"2024-04-25T10:08:13.396312+0000","last_fullsized":"2024-04-25T10:08:13.396312+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:43.406629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.900281+0000","last_change":"2024-04-25T10:08:10.892469+0000","last_active":"2024-04-25T10:08:10.900281+0000","last_peered":"2024-04-25T10:08:10.900281+0000","last_clean":"2024-04-25T10:08:10.900281+0000","last_became_active":"2024-04-25T10:08:10.890763+0000","last_became_peered":"2024-04-25T10:08:10.890763+0000","last_unstale":"2024-04-25T10:08:10.900281+0000","last_undegraded":"2024-04-25T10:08:10.900281+0000","last_fullsized":"2024-04-25T10:08:10.900281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:30.075855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396550+0000","last_change":"2024-04-25T10:08:10.923545+0000","last_active":"2024-04-25T10:08:13.396550+0000","last_peered":"2024-04-25T10:08:13.396550+0000","last_clean":"2024-04-25T10:08:13.396550+0000","last_became_active":"2024-04-25T10:08:10.919185+0000","last_became_peered":"2024-04-25T10:08:10.919185+0000","last_unstale":"2024-04-25T10:08:13.396550+0000","last_undegraded":"2024-04-25T10:08:13.396550+0000","last_fullsized":"2024-04-25T10:08:13.396550+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:48.684084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967689+0000","last_change":"2024-04-25T10:08:10.897718+0000","last_active":"2024-04-25T10:08:11.967689+0000","last_peered":"2024-04-25T10:08:11.967689+0000","last_clean":"2024-04-25T10:08:11.967689+0000","last_became_active":"2024-04-25T10:08:10.897021+0000","last_became_peered":"2024-04-25T10:08:10.897021+0000","last_unstale":"2024-04-25T10:08:11.967689+0000","last_undegraded":"2024-04-25T10:08:11.967689+0000","last_fullsized":"2024-04-25T10:08:11.967689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:10.256509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.889404+0000","last_change":"2024-04-25T10:08:10.885585+0000","last_active":"2024-04-25T10:08:10.889404+0000","last_peered":"2024-04-25T10:08:10.889404+0000","last_clean":"2024-04-25T10:08:10.889404+0000","last_became_active":"2024-04-25T10:08:10.885318+0000","last_became_peered":"2024-04-25T10:08:10.885318+0000","last_unstale":"2024-04-25T10:08:10.889404+0000","last_undegraded":"2024-04-25T10:08:10.889404+0000","last_fullsized":"2024-04-25T10:08:10.889404+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:07.762131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893986+0000","last_change":"2024-04-25T10:08:10.911391+0000","last_active":"2024-04-25T10:08:11.893986+0000","last_peered":"2024-04-25T10:08:11.893986+0000","last_clean":"2024-04-25T10:08:11.893986+0000","last_became_active":"2024-04-25T10:08:10.910461+0000","last_became_peered":"2024-04-25T10:08:10.910461+0000","last_unstale":"2024-04-25T10:08:11.893986+0000","last_undegraded":"2024-04-25T10:08:11.893986+0000","last_fullsized":"2024-04-25T10:08:11.893986+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:48.269167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.896261+0000","last_change":"2024-04-25T10:08:10.894475+0000","last_active":"2024-04-25T10:08:10.896261+0000","last_peered":"2024-04-25T10:08:10.896261+0000","last_clean":"2024-04-25T10:08:10.896261+0000","last_became_active":"2024-04-25T10:08:10.894176+0000","last_became_peered":"2024-04-25T10:08:10.894176+0000","last_unstale":"2024-04-25T10:08:10.896261+0000","last_undegraded":"2024-04-25T10:08:10.896261+0000","last_fullsized":"2024-04-25T10:08:10.896261+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:57.804679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.902435+0000","last_change":"2024-04-25T10:08:10.901642+0000","last_active":"2024-04-25T10:08:10.902435+0000","last_peered":"2024-04-25T10:08:10.902435+0000","last_clean":"2024-04-25T10:08:10.902435+0000","last_became_active":"2024-04-25T10:08:10.901389+0000","last_became_peered":"2024-04-25T10:08:10.901389+0000","last_unstale":"2024-04-25T10:08:10.902435+0000","last_undegraded":"2024-04-25T10:08:10.902435+0000","last_fullsized":"2024-04-25T10:08:10.902435+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:01.232475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.915810+0000","last_change":"2024-04-25T10:08:10.915357+0000","last_active":"2024-04-25T10:08:10.915810+0000","last_peered":"2024-04-25T10:08:10.915810+0000","last_clean":"2024-04-25T10:08:10.915810+0000","last_became_active":"2024-04-25T10:08:10.915052+0000","last_became_peered":"2024-04-25T10:08:10.915052+0000","last_unstale":"2024-04-25T10:08:10.915810+0000","last_undegraded":"2024-04-25T10:08:10.915810+0000","last_fullsized":"2024-04-25T10:08:10.915810+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:13.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396716+0000","last_change":"2024-04-25T10:08:10.922837+0000","last_active":"2024-04-25T10:08:13.396716+0000","last_peered":"2024-04-25T10:08:13.396716+0000","last_clean":"2024-04-25T10:08:13.396716+0000","last_became_active":"2024-04-25T10:08:10.921976+0000","last_became_peered":"2024-04-25T10:08:10.921976+0000","last_unstale":"2024-04-25T10:08:13.396716+0000","last_undegraded":"2024-04-25T10:08:13.396716+0000","last_fullsized":"2024-04-25T10:08:13.396716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:09.814025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.889411+0000","last_change":"2024-04-25T10:08:10.884959+0000","last_active":"2024-04-25T10:08:10.889411+0000","last_peered":"2024-04-25T10:08:10.889411+0000","last_clean":"2024-04-25T10:08:10.889411+0000","last_became_active":"2024-04-25T10:08:10.884659+0000","last_became_peered":"2024-04-25T10:08:10.884659+0000","last_unstale":"2024-04-25T10:08:10.889411+0000","last_undegraded":"2024-04-25T10:08:10.889411+0000","last_fullsized":"2024-04-25T10:08:10.889411+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:42.724053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.896757+0000","last_change":"2024-04-25T10:08:10.895828+0000","last_active":"2024-04-25T10:08:10.896757+0000","last_peered":"2024-04-25T10:08:10.896757+0000","last_clean":"2024-04-25T10:08:10.896757+0000","last_became_active":"2024-04-25T10:08:10.895589+0000","last_became_peered":"2024-04-25T10:08:10.895589+0000","last_unstale":"2024-04-25T10:08:10.896757+0000","last_undegraded":"2024-04-25T10:08:10.896757+0000","last_fullsized":"2024-04-25T10:08:10.896757+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:45.805791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.912538+0000","last_change":"2024-04-25T10:08:10.911873+0000","last_active":"2024-04-25T10:08:10.912538+0000","last_peered":"2024-04-25T10:08:10.912538+0000","last_clean":"2024-04-25T10:08:10.912538+0000","last_became_active":"2024-04-25T10:08:10.911603+0000","last_became_peered":"2024-04-25T10:08:10.911603+0000","last_unstale":"2024-04-25T10:08:10.912538+0000","last_undegraded":"2024-04-25T10:08:10.912538+0000","last_fullsized":"2024-04-25T10:08:10.912538+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:27.633134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.926180+0000","last_change":"2024-04-25T10:08:09.906122+0000","last_active":"2024-04-25T10:08:10.926180+0000","last_peered":"2024-04-25T10:08:10.926180+0000","last_clean":"2024-04-25T10:08:10.926180+0000","last_became_active":"2024-04-25T10:08:09.905879+0000","last_became_peered":"2024-04-25T10:08:09.905879+0000","last_unstale":"2024-04-25T10:08:10.926180+0000","last_undegraded":"2024-04-25T10:08:10.926180+0000","last_fullsized":"2024-04-25T10:08:10.926180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:48.349703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.890056+0000","last_change":"2024-04-25T10:08:10.911299+0000","last_active":"2024-04-25T10:08:11.890056+0000","last_peered":"2024-04-25T10:08:11.890056+0000","last_clean":"2024-04-25T10:08:11.890056+0000","last_became_active":"2024-04-25T10:08:10.910974+0000","last_became_peered":"2024-04-25T10:08:10.910974+0000","last_unstale":"2024-04-25T10:08:11.890056+0000","last_undegraded":"2024-04-25T10:08:11.890056+0000","last_fullsized":"2024-04-25T10:08:11.890056+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:22.511670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.921727+0000","last_change":"2024-04-25T10:08:09.898224+0000","last_active":"2024-04-25T10:08:10.921727+0000","last_peered":"2024-04-25T10:08:10.921727+0000","last_clean":"2024-04-25T10:08:10.921727+0000","last_became_active":"2024-04-25T10:08:09.897935+0000","last_became_peered":"2024-04-25T10:08:09.897935+0000","last_unstale":"2024-04-25T10:08:10.921727+0000","last_undegraded":"2024-04-25T10:08:10.921727+0000","last_fullsized":"2024-04-25T10:08:10.921727+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:16.589298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.884524+0000","last_change":"2024-04-25T10:08:10.882470+0000","last_active":"2024-04-25T10:08:10.884524+0000","last_peered":"2024-04-25T10:08:10.884524+0000","last_clean":"2024-04-25T10:08:10.884524+0000","last_became_active":"2024-04-25T10:08:10.882185+0000","last_became_peered":"2024-04-25T10:08:10.882185+0000","last_unstale":"2024-04-25T10:08:10.884524+0000","last_undegraded":"2024-04-25T10:08:10.884524+0000","last_fullsized":"2024-04-25T10:08:10.884524+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.344111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.889456+0000","last_change":"2024-04-25T10:08:09.888457+0000","last_active":"2024-04-25T10:08:10.889456+0000","last_peered":"2024-04-25T10:08:10.889456+0000","last_clean":"2024-04-25T10:08:10.889456+0000","last_became_active":"2024-04-25T10:08:09.888103+0000","last_became_peered":"2024-04-25T10:08:09.888103+0000","last_unstale":"2024-04-25T10:08:10.889456+0000","last_undegraded":"2024-04-25T10:08:10.889456+0000","last_fullsized":"2024-04-25T10:08:10.889456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.255517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.927157+0000","last_change":"2024-04-25T10:08:10.926635+0000","last_active":"2024-04-25T10:08:10.927157+0000","last_peered":"2024-04-25T10:08:10.927157+0000","last_clean":"2024-04-25T10:08:10.927157+0000","last_became_active":"2024-04-25T10:08:10.926349+0000","last_became_peered":"2024-04-25T10:08:10.926349+0000","last_unstale":"2024-04-25T10:08:10.927157+0000","last_undegraded":"2024-04-25T10:08:10.927157+0000","last_fullsized":"2024-04-25T10:08:10.927157+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:17.598272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894215+0000","last_change":"2024-04-25T10:08:09.907608+0000","last_active":"2024-04-25T10:08:11.894215+0000","last_peered":"2024-04-25T10:08:11.894215+0000","last_clean":"2024-04-25T10:08:11.894215+0000","last_became_active":"2024-04-25T10:08:09.906963+0000","last_became_peered":"2024-04-25T10:08:09.906963+0000","last_unstale":"2024-04-25T10:08:11.894215+0000","last_undegraded":"2024-04-25T10:08:11.894215+0000","last_fullsized":"2024-04-25T10:08:11.894215+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.255517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.900251+0000","last_change":"2024-04-25T10:08:10.892319+0000","last_active":"2024-04-25T10:08:10.900251+0000","last_peered":"2024-04-25T10:08:10.900251+0000","last_clean":"2024-04-25T10:08:10.900251+0000","last_became_active":"2024-04-25T10:08:10.890374+0000","last_became_peered":"2024-04-25T10:08:10.890374+0000","last_unstale":"2024-04-25T10:08:10.900251+0000","last_undegraded":"2024-04-25T10:08:10.900251+0000","last_fullsized":"2024-04-25T10:08:10.900251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:21.713220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893927+0000","last_change":"2024-04-25T10:08:09.886156+0000","last_active":"2024-04-25T10:08:11.893927+0000","last_peered":"2024-04-25T10:08:11.893927+0000","last_clean":"2024-04-25T10:08:11.893927+0000","last_became_active":"2024-04-25T10:08:09.885288+0000","last_became_peered":"2024-04-25T10:08:09.885288+0000","last_unstale":"2024-04-25T10:08:11.893927+0000","last_undegraded":"2024-04-25T10:08:11.893927+0000","last_fullsized":"2024-04-25T10:08:11.893927+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:39.542515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.900555+0000","last_change":"2024-04-25T10:08:10.890541+0000","last_active":"2024-04-25T10:08:10.900555+0000","last_peered":"2024-04-25T10:08:10.900555+0000","last_clean":"2024-04-25T10:08:10.900555+0000","last_became_active":"2024-04-25T10:08:10.890280+0000","last_became_peered":"2024-04-25T10:08:10.890280+0000","last_unstale":"2024-04-25T10:08:10.900555+0000","last_undegraded":"2024-04-25T10:08:10.900555+0000","last_fullsized":"2024-04-25T10:08:10.900555+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:35.281267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.895707+0000","last_change":"2024-04-25T10:08:09.894473+0000","last_active":"2024-04-25T10:08:10.895707+0000","last_peered":"2024-04-25T10:08:10.895707+0000","last_clean":"2024-04-25T10:08:10.895707+0000","last_became_active":"2024-04-25T10:08:09.894181+0000","last_became_peered":"2024-04-25T10:08:09.894181+0000","last_unstale":"2024-04-25T10:08:10.895707+0000","last_undegraded":"2024-04-25T10:08:10.895707+0000","last_fullsized":"2024-04-25T10:08:10.895707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:10.625997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967711+0000","last_change":"2024-04-25T10:08:10.923620+0000","last_active":"2024-04-25T10:08:11.967711+0000","last_peered":"2024-04-25T10:08:11.967711+0000","last_clean":"2024-04-25T10:08:11.967711+0000","last_became_active":"2024-04-25T10:08:10.923032+0000","last_became_peered":"2024-04-25T10:08:10.923032+0000","last_unstale":"2024-04-25T10:08:11.967711+0000","last_undegraded":"2024-04-25T10:08:11.967711+0000","last_fullsized":"2024-04-25T10:08:11.967711+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:10.085497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893167+0000","last_change":"2024-04-25T10:08:09.907088+0000","last_active":"2024-04-25T10:08:11.893167+0000","last_peered":"2024-04-25T10:08:11.893167+0000","last_clean":"2024-04-25T10:08:11.893167+0000","last_became_active":"2024-04-25T10:08:09.906310+0000","last_became_peered":"2024-04-25T10:08:09.906310+0000","last_unstale":"2024-04-25T10:08:11.893167+0000","last_undegraded":"2024-04-25T10:08:11.893167+0000","last_fullsized":"2024-04-25T10:08:11.893167+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.592539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901609+0000","last_change":"2024-04-25T10:08:10.892966+0000","last_active":"2024-04-25T10:08:10.901609+0000","last_peered":"2024-04-25T10:08:10.901609+0000","last_clean":"2024-04-25T10:08:10.901609+0000","last_became_active":"2024-04-25T10:08:10.892667+0000","last_became_peered":"2024-04-25T10:08:10.892667+0000","last_unstale":"2024-04-25T10:08:10.901609+0000","last_undegraded":"2024-04-25T10:08:10.901609+0000","last_fullsized":"2024-04-25T10:08:10.901609+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:34.974943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893151+0000","last_change":"2024-04-25T10:08:09.889905+0000","last_active":"2024-04-25T10:08:11.893151+0000","last_peered":"2024-04-25T10:08:11.893151+0000","last_clean":"2024-04-25T10:08:11.893151+0000","last_became_active":"2024-04-25T10:08:09.889171+0000","last_became_peered":"2024-04-25T10:08:09.889171+0000","last_unstale":"2024-04-25T10:08:11.893151+0000","last_undegraded":"2024-04-25T10:08:11.893151+0000","last_fullsized":"2024-04-25T10:08:11.893151+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:46.888532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894707+0000","last_change":"2024-04-25T10:08:10.886122+0000","last_active":"2024-04-25T10:08:11.894707+0000","last_peered":"2024-04-25T10:08:11.894707+0000","last_clean":"2024-04-25T10:08:11.894707+0000","last_became_active":"2024-04-25T10:08:10.882247+0000","last_became_peered":"2024-04-25T10:08:10.882247+0000","last_unstale":"2024-04-25T10:08:11.894707+0000","last_undegraded":"2024-04-25T10:08:11.894707+0000","last_fullsized":"2024-04-25T10:08:11.894707+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:58.336224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.397277+0000","last_change":"2024-04-25T10:08:09.899045+0000","last_active":"2024-04-25T10:08:13.397277+0000","last_peered":"2024-04-25T10:08:13.397277+0000","last_clean":"2024-04-25T10:08:13.397277+0000","last_became_active":"2024-04-25T10:08:09.898282+0000","last_became_peered":"2024-04-25T10:08:09.898282+0000","last_unstale":"2024-04-25T10:08:13.397277+0000","last_undegraded":"2024-04-25T10:08:13.397277+0000","last_fullsized":"2024-04-25T10:08:13.397277+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:34.532573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.397322+0000","last_change":"2024-04-25T10:08:10.919031+0000","last_active":"2024-04-25T10:08:13.397322+0000","last_peered":"2024-04-25T10:08:13.397322+0000","last_clean":"2024-04-25T10:08:13.397322+0000","last_became_active":"2024-04-25T10:08:10.911880+0000","last_became_peered":"2024-04-25T10:08:10.911880+0000","last_unstale":"2024-04-25T10:08:13.397322+0000","last_undegraded":"2024-04-25T10:08:13.397322+0000","last_fullsized":"2024-04-25T10:08:13.397322+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:45.833608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.884018+0000","last_change":"2024-04-25T10:08:09.880232+0000","last_active":"2024-04-25T10:08:10.884018+0000","last_peered":"2024-04-25T10:08:10.884018+0000","last_clean":"2024-04-25T10:08:10.884018+0000","last_became_active":"2024-04-25T10:08:09.879917+0000","last_became_peered":"2024-04-25T10:08:09.879917+0000","last_unstale":"2024-04-25T10:08:10.884018+0000","last_undegraded":"2024-04-25T10:08:10.884018+0000","last_fullsized":"2024-04-25T10:08:10.884018+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:43.464391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889651+0000","last_change":"2024-04-25T10:08:10.893598+0000","last_active":"2024-04-25T10:08:11.889651+0000","last_peered":"2024-04-25T10:08:11.889651+0000","last_clean":"2024-04-25T10:08:11.889651+0000","last_became_active":"2024-04-25T10:08:10.893008+0000","last_became_peered":"2024-04-25T10:08:10.893008+0000","last_unstale":"2024-04-25T10:08:11.889651+0000","last_undegraded":"2024-04-25T10:08:11.889651+0000","last_fullsized":"2024-04-25T10:08:11.889651+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:57.430500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396978+0000","last_change":"2024-04-25T10:08:09.899600+0000","last_active":"2024-04-25T10:08:13.396978+0000","last_peered":"2024-04-25T10:08:13.396978+0000","last_clean":"2024-04-25T10:08:13.396978+0000","last_became_active":"2024-04-25T10:08:09.898985+0000","last_became_peered":"2024-04-25T10:08:09.898985+0000","last_unstale":"2024-04-25T10:08:13.396978+0000","last_undegraded":"2024-04-25T10:08:13.396978+0000","last_fullsized":"2024-04-25T10:08:13.396978+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:54.732376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396711+0000","last_change":"2024-04-25T10:08:10.918300+0000","last_active":"2024-04-25T10:08:13.396711+0000","last_peered":"2024-04-25T10:08:13.396711+0000","last_clean":"2024-04-25T10:08:13.396711+0000","last_became_active":"2024-04-25T10:08:10.911496+0000","last_became_peered":"2024-04-25T10:08:10.911496+0000","last_unstale":"2024-04-25T10:08:13.396711+0000","last_undegraded":"2024-04-25T10:08:13.396711+0000","last_fullsized":"2024-04-25T10:08:13.396711+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:40.184599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894808+0000","last_change":"2024-04-25T10:08:09.891850+0000","last_active":"2024-04-25T10:08:11.894808+0000","last_peered":"2024-04-25T10:08:11.894808+0000","last_clean":"2024-04-25T10:08:11.894808+0000","last_became_active":"2024-04-25T10:08:09.891235+0000","last_became_peered":"2024-04-25T10:08:09.891235+0000","last_unstale":"2024-04-25T10:08:11.894808+0000","last_undegraded":"2024-04-25T10:08:11.894808+0000","last_fullsized":"2024-04-25T10:08:11.894808+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:56.421526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894725+0000","last_change":"2024-04-25T10:08:10.917026+0000","last_active":"2024-04-25T10:08:11.894725+0000","last_peered":"2024-04-25T10:08:11.894725+0000","last_clean":"2024-04-25T10:08:11.894725+0000","last_became_active":"2024-04-25T10:08:10.916406+0000","last_became_peered":"2024-04-25T10:08:10.916406+0000","last_unstale":"2024-04-25T10:08:11.894725+0000","last_undegraded":"2024-04-25T10:08:11.894725+0000","last_fullsized":"2024-04-25T10:08:11.894725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.834110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901694+0000","last_change":"2024-04-25T10:08:09.881070+0000","last_active":"2024-04-25T10:08:10.901694+0000","last_peered":"2024-04-25T10:08:10.901694+0000","last_clean":"2024-04-25T10:08:10.901694+0000","last_became_active":"2024-04-25T10:08:09.880757+0000","last_became_peered":"2024-04-25T10:08:09.880757+0000","last_unstale":"2024-04-25T10:08:10.901694+0000","last_undegraded":"2024-04-25T10:08:10.901694+0000","last_fullsized":"2024-04-25T10:08:10.901694+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:26.624159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396665+0000","last_change":"2024-04-25T10:08:10.923429+0000","last_active":"2024-04-25T10:08:13.396665+0000","last_peered":"2024-04-25T10:08:13.396665+0000","last_clean":"2024-04-25T10:08:13.396665+0000","last_became_active":"2024-04-25T10:08:10.922036+0000","last_became_peered":"2024-04-25T10:08:10.922036+0000","last_unstale":"2024-04-25T10:08:13.396665+0000","last_undegraded":"2024-04-25T10:08:13.396665+0000","last_fullsized":"2024-04-25T10:08:13.396665+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:54.437016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967486+0000","last_change":"2024-04-25T10:08:09.899426+0000","last_active":"2024-04-25T10:08:11.967486+0000","last_peered":"2024-04-25T10:08:11.967486+0000","last_clean":"2024-04-25T10:08:11.967486+0000","last_became_active":"2024-04-25T10:08:09.898830+0000","last_became_peered":"2024-04-25T10:08:09.898830+0000","last_unstale":"2024-04-25T10:08:11.967486+0000","last_undegraded":"2024-04-25T10:08:11.967486+0000","last_fullsized":"2024-04-25T10:08:11.967486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:19.781296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.892879+0000","last_change":"2024-04-25T10:08:10.901543+0000","last_active":"2024-04-25T10:08:11.892879+0000","last_peered":"2024-04-25T10:08:11.892879+0000","last_clean":"2024-04-25T10:08:11.892879+0000","last_became_active":"2024-04-25T10:08:10.900826+0000","last_became_peered":"2024-04-25T10:08:10.900826+0000","last_unstale":"2024-04-25T10:08:11.892879+0000","last_undegraded":"2024-04-25T10:08:11.892879+0000","last_fullsized":"2024-04-25T10:08:11.892879+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:32.461497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893451+0000","last_change":"2024-04-25T10:08:09.902684+0000","last_active":"2024-04-25T10:08:11.893451+0000","last_peered":"2024-04-25T10:08:11.893451+0000","last_clean":"2024-04-25T10:08:11.893451+0000","last_became_active":"2024-04-25T10:08:09.901806+0000","last_became_peered":"2024-04-25T10:08:09.901806+0000","last_unstale":"2024-04-25T10:08:11.893451+0000","last_undegraded":"2024-04-25T10:08:11.893451+0000","last_fullsized":"2024-04-25T10:08:11.893451+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:17.199198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396317+0000","last_change":"2024-04-25T10:08:10.925089+0000","last_active":"2024-04-25T10:08:13.396317+0000","last_peered":"2024-04-25T10:08:13.396317+0000","last_clean":"2024-04-25T10:08:13.396317+0000","last_became_active":"2024-04-25T10:08:10.922906+0000","last_became_peered":"2024-04-25T10:08:10.922906+0000","last_unstale":"2024-04-25T10:08:13.396317+0000","last_undegraded":"2024-04-25T10:08:13.396317+0000","last_fullsized":"2024-04-25T10:08:13.396317+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:55.438825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":22,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.395853+0000","last_change":"2024-04-25T10:08:09.908904+0000","last_active":"2024-04-25T10:08:13.395853+0000","last_peered":"2024-04-25T10:08:13.395853+0000","last_clean":"2024-04-25T10:08:13.395853+0000","last_became_active":"2024-04-25T10:08:09.908076+0000","last_became_peered":"2024-04-25T10:08:09.908076+0000","last_unstale":"2024-04-25T10:08:13.395853+0000","last_undegraded":"2024-04-25T10:08:13.395853+0000","last_fullsized":"2024-04-25T10:08:13.395853+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:12.528702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.397170+0000","last_change":"2024-04-25T10:08:10.916890+0000","last_active":"2024-04-25T10:08:13.397170+0000","last_peered":"2024-04-25T10:08:13.397170+0000","last_clean":"2024-04-25T10:08:13.397170+0000","last_became_active":"2024-04-25T10:08:10.910435+0000","last_became_peered":"2024-04-25T10:08:10.910435+0000","last_unstale":"2024-04-25T10:08:13.397170+0000","last_undegraded":"2024-04-25T10:08:13.397170+0000","last_fullsized":"2024-04-25T10:08:13.397170+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:10.886429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.899177+0000","last_change":"2024-04-25T10:08:09.884420+0000","last_active":"2024-04-25T10:08:10.899177+0000","last_peered":"2024-04-25T10:08:10.899177+0000","last_clean":"2024-04-25T10:08:10.899177+0000","last_became_active":"2024-04-25T10:08:09.883427+0000","last_became_peered":"2024-04-25T10:08:09.883427+0000","last_unstale":"2024-04-25T10:08:10.899177+0000","last_undegraded":"2024-04-25T10:08:10.899177+0000","last_fullsized":"2024-04-25T10:08:10.899177+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:47.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894725+0000","last_change":"2024-04-25T10:08:10.918083+0000","last_active":"2024-04-25T10:08:11.894725+0000","last_peered":"2024-04-25T10:08:11.894725+0000","last_clean":"2024-04-25T10:08:11.894725+0000","last_became_active":"2024-04-25T10:08:10.917512+0000","last_became_peered":"2024-04-25T10:08:10.917512+0000","last_unstale":"2024-04-25T10:08:11.894725+0000","last_undegraded":"2024-04-25T10:08:11.894725+0000","last_fullsized":"2024-04-25T10:08:11.894725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:35.281267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.891599+0000","last_change":"2024-04-25T10:08:09.888804+0000","last_active":"2024-04-25T10:08:10.891599+0000","last_peered":"2024-04-25T10:08:10.891599+0000","last_clean":"2024-04-25T10:08:10.891599+0000","last_became_active":"2024-04-25T10:08:09.888488+0000","last_became_peered":"2024-04-25T10:08:09.888488+0000","last_unstale":"2024-04-25T10:08:10.891599+0000","last_undegraded":"2024-04-25T10:08:10.891599+0000","last_fullsized":"2024-04-25T10:08:10.891599+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:17.199198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.906640+0000","last_change":"2024-04-25T10:08:10.905832+0000","last_active":"2024-04-25T10:08:10.906640+0000","last_peered":"2024-04-25T10:08:10.906640+0000","last_clean":"2024-04-25T10:08:10.906640+0000","last_became_active":"2024-04-25T10:08:10.905524+0000","last_became_peered":"2024-04-25T10:08:10.905524+0000","last_unstale":"2024-04-25T10:08:10.906640+0000","last_undegraded":"2024-04-25T10:08:10.906640+0000","last_fullsized":"2024-04-25T10:08:10.906640+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:29.292616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893944+0000","last_change":"2024-04-25T10:08:09.902367+0000","last_active":"2024-04-25T10:08:11.893944+0000","last_peered":"2024-04-25T10:08:11.893944+0000","last_clean":"2024-04-25T10:08:11.893944+0000","last_became_active":"2024-04-25T10:08:09.900592+0000","last_became_peered":"2024-04-25T10:08:09.900592+0000","last_unstale":"2024-04-25T10:08:11.893944+0000","last_undegraded":"2024-04-25T10:08:11.893944+0000","last_fullsized":"2024-04-25T10:08:11.893944+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:38.492052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.397108+0000","last_change":"2024-04-25T10:08:10.928860+0000","last_active":"2024-04-25T10:08:13.397108+0000","last_peered":"2024-04-25T10:08:13.397108+0000","last_clean":"2024-04-25T10:08:13.397108+0000","last_became_active":"2024-04-25T10:08:10.921892+0000","last_became_peered":"2024-04-25T10:08:10.921892+0000","last_unstale":"2024-04-25T10:08:13.397108+0000","last_undegraded":"2024-04-25T10:08:13.397108+0000","last_fullsized":"2024-04-25T10:08:13.397108+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:31.914077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.878704+0000","last_change":"2024-04-25T10:08:09.890143+0000","last_active":"2024-04-25T10:08:10.878704+0000","last_peered":"2024-04-25T10:08:10.878704+0000","last_clean":"2024-04-25T10:08:10.878704+0000","last_became_active":"2024-04-25T10:08:09.889126+0000","last_became_peered":"2024-04-25T10:08:09.889126+0000","last_unstale":"2024-04-25T10:08:10.878704+0000","last_undegraded":"2024-04-25T10:08:10.878704+0000","last_fullsized":"2024-04-25T10:08:10.878704+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:38.492052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893893+0000","last_change":"2024-04-25T10:08:10.917214+0000","last_active":"2024-04-25T10:08:11.893893+0000","last_peered":"2024-04-25T10:08:11.893893+0000","last_clean":"2024-04-25T10:08:11.893893+0000","last_became_active":"2024-04-25T10:08:10.915770+0000","last_became_peered":"2024-04-25T10:08:10.915770+0000","last_unstale":"2024-04-25T10:08:11.893893+0000","last_undegraded":"2024-04-25T10:08:11.893893+0000","last_fullsized":"2024-04-25T10:08:11.893893+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:44.066536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.907624+0000","last_change":"2024-04-25T10:08:09.889071+0000","last_active":"2024-04-25T10:08:10.907624+0000","last_peered":"2024-04-25T10:08:10.907624+0000","last_clean":"2024-04-25T10:08:10.907624+0000","last_became_active":"2024-04-25T10:08:09.888746+0000","last_became_peered":"2024-04-25T10:08:09.888746+0000","last_unstale":"2024-04-25T10:08:10.907624+0000","last_undegraded":"2024-04-25T10:08:10.907624+0000","last_fullsized":"2024-04-25T10:08:10.907624+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:43.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.895295+0000","last_change":"2024-04-25T10:08:10.893599+0000","last_active":"2024-04-25T10:08:10.895295+0000","last_peered":"2024-04-25T10:08:10.895295+0000","last_clean":"2024-04-25T10:08:10.895295+0000","last_became_active":"2024-04-25T10:08:10.893206+0000","last_became_peered":"2024-04-25T10:08:10.893206+0000","last_unstale":"2024-04-25T10:08:10.895295+0000","last_undegraded":"2024-04-25T10:08:10.895295+0000","last_fullsized":"2024-04-25T10:08:10.895295+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:51.143246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889836+0000","last_change":"2024-04-25T10:08:09.880919+0000","last_active":"2024-04-25T10:08:11.889836+0000","last_peered":"2024-04-25T10:08:11.889836+0000","last_clean":"2024-04-25T10:08:11.889836+0000","last_became_active":"2024-04-25T10:08:09.880611+0000","last_became_peered":"2024-04-25T10:08:09.880611+0000","last_unstale":"2024-04-25T10:08:11.889836+0000","last_undegraded":"2024-04-25T10:08:11.889836+0000","last_fullsized":"2024-04-25T10:08:11.889836+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:22.155681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.902841+0000","last_change":"2024-04-25T10:08:10.901150+0000","last_active":"2024-04-25T10:08:10.902841+0000","last_peered":"2024-04-25T10:08:10.902841+0000","last_clean":"2024-04-25T10:08:10.902841+0000","last_became_active":"2024-04-25T10:08:10.900624+0000","last_became_peered":"2024-04-25T10:08:10.900624+0000","last_unstale":"2024-04-25T10:08:10.902841+0000","last_undegraded":"2024-04-25T10:08:10.902841+0000","last_fullsized":"2024-04-25T10:08:10.902841+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:08.745250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.892866+0000","last_change":"2024-04-25T10:08:09.901094+0000","last_active":"2024-04-25T10:08:11.892866+0000","last_peered":"2024-04-25T10:08:11.892866+0000","last_clean":"2024-04-25T10:08:11.892866+0000","last_became_active":"2024-04-25T10:08:09.900405+0000","last_became_peered":"2024-04-25T10:08:09.900405+0000","last_unstale":"2024-04-25T10:08:11.892866+0000","last_undegraded":"2024-04-25T10:08:11.892866+0000","last_fullsized":"2024-04-25T10:08:11.892866+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:30.789048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893987+0000","last_change":"2024-04-25T10:08:10.909724+0000","last_active":"2024-04-25T10:08:11.893987+0000","last_peered":"2024-04-25T10:08:11.893987+0000","last_clean":"2024-04-25T10:08:11.893987+0000","last_became_active":"2024-04-25T10:08:10.904979+0000","last_became_peered":"2024-04-25T10:08:10.904979+0000","last_unstale":"2024-04-25T10:08:11.893987+0000","last_undegraded":"2024-04-25T10:08:11.893987+0000","last_fullsized":"2024-04-25T10:08:11.893987+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:17.598272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889987+0000","last_change":"2024-04-25T10:08:09.885875+0000","last_active":"2024-04-25T10:08:11.889987+0000","last_peered":"2024-04-25T10:08:11.889987+0000","last_clean":"2024-04-25T10:08:11.889987+0000","last_became_active":"2024-04-25T10:08:09.885529+0000","last_became_peered":"2024-04-25T10:08:09.885529+0000","last_unstale":"2024-04-25T10:08:11.889987+0000","last_undegraded":"2024-04-25T10:08:11.889987+0000","last_fullsized":"2024-04-25T10:08:11.889987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:30.869245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.895037+0000","last_change":"2024-04-25T10:08:10.923948+0000","last_active":"2024-04-25T10:08:11.895037+0000","last_peered":"2024-04-25T10:08:11.895037+0000","last_clean":"2024-04-25T10:08:11.895037+0000","last_became_active":"2024-04-25T10:08:10.923383+0000","last_became_peered":"2024-04-25T10:08:10.923383+0000","last_unstale":"2024-04-25T10:08:11.895037+0000","last_undegraded":"2024-04-25T10:08:11.895037+0000","last_fullsized":"2024-04-25T10:08:11.895037+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:55.065275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.879776+0000","last_change":"2024-04-25T10:08:09.894894+0000","last_active":"2024-04-25T10:08:10.879776+0000","last_peered":"2024-04-25T10:08:10.879776+0000","last_clean":"2024-04-25T10:08:10.879776+0000","last_became_active":"2024-04-25T10:08:09.894350+0000","last_became_peered":"2024-04-25T10:08:09.894350+0000","last_unstale":"2024-04-25T10:08:10.879776+0000","last_undegraded":"2024-04-25T10:08:10.879776+0000","last_fullsized":"2024-04-25T10:08:10.879776+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.592539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889925+0000","last_change":"2024-04-25T10:08:10.893322+0000","last_active":"2024-04-25T10:08:11.889925+0000","last_peered":"2024-04-25T10:08:11.889925+0000","last_clean":"2024-04-25T10:08:11.889925+0000","last_became_active":"2024-04-25T10:08:10.892639+0000","last_became_peered":"2024-04-25T10:08:10.892639+0000","last_unstale":"2024-04-25T10:08:11.889925+0000","last_undegraded":"2024-04-25T10:08:11.889925+0000","last_fullsized":"2024-04-25T10:08:11.889925+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:37.445694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967957+0000","last_change":"2024-04-25T10:08:09.893305+0000","last_active":"2024-04-25T10:08:11.967957+0000","last_peered":"2024-04-25T10:08:11.967957+0000","last_clean":"2024-04-25T10:08:11.967957+0000","last_became_active":"2024-04-25T10:08:09.892501+0000","last_became_peered":"2024-04-25T10:08:09.892501+0000","last_unstale":"2024-04-25T10:08:11.967957+0000","last_undegraded":"2024-04-25T10:08:11.967957+0000","last_fullsized":"2024-04-25T10:08:11.967957+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:21.522866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.926945+0000","last_change":"2024-04-25T10:08:10.926260+0000","last_active":"2024-04-25T10:08:10.926945+0000","last_peered":"2024-04-25T10:08:10.926945+0000","last_clean":"2024-04-25T10:08:10.926945+0000","last_became_active":"2024-04-25T10:08:10.926038+0000","last_became_peered":"2024-04-25T10:08:10.926038+0000","last_unstale":"2024-04-25T10:08:10.926945+0000","last_undegraded":"2024-04-25T10:08:10.926945+0000","last_fullsized":"2024-04-25T10:08:10.926945+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:44.066536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894376+0000","last_change":"2024-04-25T10:08:09.906942+0000","last_active":"2024-04-25T10:08:11.894376+0000","last_peered":"2024-04-25T10:08:11.894376+0000","last_clean":"2024-04-25T10:08:11.894376+0000","last_became_active":"2024-04-25T10:08:09.906192+0000","last_became_peered":"2024-04-25T10:08:09.906192+0000","last_unstale":"2024-04-25T10:08:11.894376+0000","last_undegraded":"2024-04-25T10:08:11.894376+0000","last_fullsized":"2024-04-25T10:08:11.894376+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:48.349703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.899168+0000","last_change":"2024-04-25T10:08:10.890698+0000","last_active":"2024-04-25T10:08:10.899168+0000","last_peered":"2024-04-25T10:08:10.899168+0000","last_clean":"2024-04-25T10:08:10.899168+0000","last_became_active":"2024-04-25T10:08:10.889982+0000","last_became_peered":"2024-04-25T10:08:10.889982+0000","last_unstale":"2024-04-25T10:08:10.899168+0000","last_undegraded":"2024-04-25T10:08:10.899168+0000","last_fullsized":"2024-04-25T10:08:10.899168+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:55.065275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889624+0000","last_change":"2024-04-25T10:08:09.902559+0000","last_active":"2024-04-25T10:08:11.889624+0000","last_peered":"2024-04-25T10:08:11.889624+0000","last_clean":"2024-04-25T10:08:11.889624+0000","last_became_active":"2024-04-25T10:08:09.902240+0000","last_became_peered":"2024-04-25T10:08:09.902240+0000","last_unstale":"2024-04-25T10:08:11.889624+0000","last_undegraded":"2024-04-25T10:08:11.889624+0000","last_fullsized":"2024-04-25T10:08:11.889624+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:30.202367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396825+0000","last_change":"2024-04-25T10:08:10.924116+0000","last_active":"2024-04-25T10:08:13.396825+0000","last_peered":"2024-04-25T10:08:13.396825+0000","last_clean":"2024-04-25T10:08:13.396825+0000","last_became_active":"2024-04-25T10:08:10.922605+0000","last_became_peered":"2024-04-25T10:08:10.922605+0000","last_unstale":"2024-04-25T10:08:13.396825+0000","last_undegraded":"2024-04-25T10:08:13.396825+0000","last_fullsized":"2024-04-25T10:08:13.396825+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:07.785780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894016+0000","last_change":"2024-04-25T10:08:09.887885+0000","last_active":"2024-04-25T10:08:11.894016+0000","last_peered":"2024-04-25T10:08:11.894016+0000","last_clean":"2024-04-25T10:08:11.894016+0000","last_became_active":"2024-04-25T10:08:09.887165+0000","last_became_peered":"2024-04-25T10:08:09.887165+0000","last_unstale":"2024-04-25T10:08:11.894016+0000","last_undegraded":"2024-04-25T10:08:11.894016+0000","last_fullsized":"2024-04-25T10:08:11.894016+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:09.379274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889431+0000","last_change":"2024-04-25T10:08:10.894492+0000","last_active":"2024-04-25T10:08:11.889431+0000","last_peered":"2024-04-25T10:08:11.889431+0000","last_clean":"2024-04-25T10:08:11.889431+0000","last_became_active":"2024-04-25T10:08:10.894221+0000","last_became_peered":"2024-04-25T10:08:10.894221+0000","last_unstale":"2024-04-25T10:08:11.889431+0000","last_undegraded":"2024-04-25T10:08:11.889431+0000","last_fullsized":"2024-04-25T10:08:11.889431+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:47.897507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889533+0000","last_change":"2024-04-25T10:08:09.890601+0000","last_active":"2024-04-25T10:08:11.889533+0000","last_peered":"2024-04-25T10:08:11.889533+0000","last_clean":"2024-04-25T10:08:11.889533+0000","last_became_active":"2024-04-25T10:08:09.890208+0000","last_became_peered":"2024-04-25T10:08:09.890208+0000","last_unstale":"2024-04-25T10:08:11.889533+0000","last_undegraded":"2024-04-25T10:08:11.889533+0000","last_fullsized":"2024-04-25T10:08:11.889533+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:01.465554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901843+0000","last_change":"2024-04-25T10:08:10.898030+0000","last_active":"2024-04-25T10:08:10.901843+0000","last_peered":"2024-04-25T10:08:10.901843+0000","last_clean":"2024-04-25T10:08:10.901843+0000","last_became_active":"2024-04-25T10:08:10.897719+0000","last_became_peered":"2024-04-25T10:08:10.897719+0000","last_unstale":"2024-04-25T10:08:10.901843+0000","last_undegraded":"2024-04-25T10:08:10.901843+0000","last_fullsized":"2024-04-25T10:08:10.901843+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:18.216787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.879881+0000","last_change":"2024-04-25T10:08:09.900914+0000","last_active":"2024-04-25T10:08:10.879881+0000","last_peered":"2024-04-25T10:08:10.879881+0000","last_clean":"2024-04-25T10:08:10.879881+0000","last_became_active":"2024-04-25T10:08:09.900669+0000","last_became_peered":"2024-04-25T10:08:09.900669+0000","last_unstale":"2024-04-25T10:08:10.879881+0000","last_undegraded":"2024-04-25T10:08:10.879881+0000","last_fullsized":"2024-04-25T10:08:10.879881+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.736275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901873+0000","last_change":"2024-04-25T10:08:10.894360+0000","last_active":"2024-04-25T10:08:10.901873+0000","last_peered":"2024-04-25T10:08:10.901873+0000","last_clean":"2024-04-25T10:08:10.901873+0000","last_became_active":"2024-04-25T10:08:10.894017+0000","last_became_peered":"2024-04-25T10:08:10.894017+0000","last_unstale":"2024-04-25T10:08:10.901873+0000","last_undegraded":"2024-04-25T10:08:10.901873+0000","last_fullsized":"2024-04-25T10:08:10.901873+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:43.207466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967562+0000","last_change":"2024-04-25T10:08:09.909488+0000","last_active":"2024-04-25T10:08:11.967562+0000","last_peered":"2024-04-25T10:08:11.967562+0000","last_clean":"2024-04-25T10:08:11.967562+0000","last_became_active":"2024-04-25T10:08:09.908863+0000","last_became_peered":"2024-04-25T10:08:09.908863+0000","last_unstale":"2024-04-25T10:08:11.967562+0000","last_undegraded":"2024-04-25T10:08:11.967562+0000","last_fullsized":"2024-04-25T10:08:11.967562+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:48.865163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":237,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.395879+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:08:13.395879+0000","last_peered":"2024-04-25T10:08:13.395879+0000","last_clean":"2024-04-25T10:08:13.395879+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:08:13.395879+0000","last_undegraded":"2024-04-25T10:08:13.395879+0000","last_fullsized":"2024-04-25T10:08:13.395879+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":870129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":700416,"data_stored":287513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542183,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":688128,"data_stored":285901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":286553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542184,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36012,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335828,"statfs":{"total":96636764160,"available":96599887872,"internally_reserved":0,"allocated":684032,"data_stored":279657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":684032,"data_stored":279657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36004,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335836,"statfs":{"total":96636764160,"available":96599896064,"internally_reserved":0,"allocated":679936,"data_stored":279691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36596,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335244,"statfs":{"total":96636764160,"available":96599289856,"internally_reserved":0,"allocated":1286144,"data_stored":872163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:14.669 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:08:14.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.818+0000 7f7d20f31640 1 Processor -- start 2024-04-25T10:08:14.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.818+0000 7f7d20f31640 1 -- start start 2024-04-25T10:08:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.818+0000 7f7d20f31640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1c09a520 0x7f7d1c0cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.818+0000 7f7d20f31640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1c0cc2e0 con 0x7f7d1c099a30 2024-04-25T10:08:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.818+0000 7f7d20f31640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1c0cc580 con 0x7f7d1c0cdd50 2024-04-25T10:08:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.818+0000 7f7d20f31640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c0cc860 con 0x7f7d1c09a520 2024-04-25T10:08:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1a59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7d1c099a30 0x7f7d1c099e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41524/0 (socket says 172.21.15.60:41524) 2024-04-25T10:08:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1a59c640 1 -- 172.21.15.60:0/2638855433 learned_addr learned my addr 172.21.15.60:0/2638855433 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1ad9d640 1 --2- 172.21.15.60:0/2638855433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1c09a520 0x7f7d1c0cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 -- 172.21.15.60:0/2638855433 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331613689 0 0) 0x7f7d1c0cc2e0 con 0x7f7d1c099a30 2024-04-25T10:08:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 -- 172.21.15.60:0/2638855433 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d04003530 con 0x7f7d1c099a30 2024-04-25T10:08:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 -- 172.21.15.60:0/2638855433 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780747929 0 0) 0x7f7d1c0cc580 con 0x7f7d1c0cdd50 2024-04-25T10:08:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 -- 172.21.15.60:0/2638855433 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d1c0cc2e0 con 0x7f7d1c0cdd50 2024-04-25T10:08:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 -- 172.21.15.60:0/2638855433 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 344834012 0 0) 0x7f7d04003530 con 0x7f7d1c099a30 2024-04-25T10:08:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 -- 172.21.15.60:0/2638855433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1c09a520 msgr2=0x7f7d1c0cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 --2- 172.21.15.60:0/2638855433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1c09a520 0x7f7d1c0cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 -- 172.21.15.60:0/2638855433 >> v1:172.21.15.100:6789/0 conn(0x7f7d1c0cdd50 legacy=0x7f7d1c0cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 -- 172.21.15.60:0/2638855433 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c0ccb40 con 0x7f7d1c099a30 2024-04-25T10:08:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 -- 172.21.15.60:0/2638855433 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f7d0c001680 con 0x7f7d1c099a30 2024-04-25T10:08:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 -- 172.21.15.60:0/2638855433 >> v1:172.21.15.60:6789/0 conn(0x7f7d1c099a30 legacy=0x7f7d1c099e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 --2- 172.21.15.60:0/2638855433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1c09a520 0x7f7d04003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 --2- 172.21.15.60:0/2638855433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d04004130 0x7f7d04004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 --2- 172.21.15.60:0/2638855433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d04004d50 0x7f7d040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 -- 172.21.15.60:0/2638855433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c0cc860 con 0x7f7d04004130 2024-04-25T10:08:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 -- 172.21.15.60:0/2638855433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0400bb90 con 0x7f7d04004d50 2024-04-25T10:08:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 -- 172.21.15.60:0/2638855433 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0400bd90 con 0x7f7d1c09a520 2024-04-25T10:08:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d19d9b640 1 --2- 172.21.15.60:0/2638855433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d04004130 0x7f7d04004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1a59c640 1 --2- 172.21.15.60:0/2638855433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1c09a520 0x7f7d04003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1ad9d640 1 --2- 172.21.15.60:0/2638855433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d04004d50 0x7f7d040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d19d9b640 1 -- 172.21.15.60:0/2638855433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1c09a520 msgr2=0x7f7d04003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d19d9b640 1 --2- 172.21.15.60:0/2638855433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1c09a520 0x7f7d04003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d19d9b640 1 -- 172.21.15.60:0/2638855433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d04004d50 msgr2=0x7f7d040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d19d9b640 1 --2- 172.21.15.60:0/2638855433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d04004d50 0x7f7d040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d19d9b640 1 -- 172.21.15.60:0/2638855433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d04003530 con 0x7f7d04004130 2024-04-25T10:08:14.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d19d9b640 1 --2- 172.21.15.60:0/2638855433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d04004130 0x7f7d04004810 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0ce150 tx=0x7f7d1001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b750d5bb5abfc5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 -- 172.21.15.60:0/2638855433 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10001520 con 0x7f7d04004130 2024-04-25T10:08:14.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 -- 172.21.15.60:0/2638855433 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d100a9070 con 0x7f7d04004130 2024-04-25T10:08:14.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.822+0000 7f7d1959a640 1 -- 172.21.15.60:0/2638855433 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10003600 con 0x7f7d04004130 2024-04-25T10:08:14.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d20f31640 1 -- 172.21.15.60:0/2638855433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d04004130 msgr2=0x7f7d04004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d20f31640 1 --2- 172.21.15.60:0/2638855433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d04004130 0x7f7d04004810 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0ce150 tx=0x7f7d1001c530 comp rx=0 tx=0).stop 2024-04-25T10:08:14.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d1ad9d640 1 -- 172.21.15.60:0/2638855433 reap_dead start 2024-04-25T10:08:14.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d20f31640 1 -- 172.21.15.60:0/2638855433 shutdown_connections 2024-04-25T10:08:14.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d20f31640 1 -- 172.21.15.60:0/2638855433 >> 172.21.15.60:0/2638855433 conn(0x7f7d1c0d17d0 msgr2=0x7f7d1c0d1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:14.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d20f31640 1 -- 172.21.15.60:0/2638855433 shutdown_connections 2024-04-25T10:08:14.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d20f31640 1 -- 172.21.15.60:0/2638855433 wait complete. 2024-04-25T10:08:14.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d20f31640 1 Processor -- start 2024-04-25T10:08:14.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d20f31640 1 -- start start 2024-04-25T10:08:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d20f31640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d1c14aa50 0x7f7d1c0ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d20f31640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d1c0ca890 0x7f7d1c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d20f31640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1c15b150 0x7f7d1c0b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d20f31640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c167b00 con 0x7f7d1c14aa50 2024-04-25T10:08:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d20f31640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c167980 con 0x7f7d1c0ca890 2024-04-25T10:08:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d20f31640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c167c80 con 0x7f7d1c15b150 2024-04-25T10:08:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d1a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d1c14aa50 0x7f7d1c0ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d1a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d1c14aa50 0x7f7d1c0ca350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58798/0 (socket says 172.21.15.60:58798) 2024-04-25T10:08:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d1a59c640 1 -- 172.21.15.60:0/2934365412 learned_addr learned my addr 172.21.15.60:0/2934365412 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d19d9b640 1 --2- 172.21.15.60:0/2934365412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d1c0ca890 0x7f7d1c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d1ad9d640 1 --2- 172.21.15.60:0/2934365412 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1c15b150 0x7f7d1c0b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d1a59c640 1 -- 172.21.15.60:0/2934365412 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1c15b150 msgr2=0x7f7d1c0b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d1a59c640 1 --2- 172.21.15.60:0/2934365412 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1c15b150 0x7f7d1c0b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d1a59c640 1 -- 172.21.15.60:0/2934365412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d1c0ca890 msgr2=0x7f7d1c0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d1a59c640 1 --2- 172.21.15.60:0/2934365412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d1c0ca890 0x7f7d1c0b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d1a59c640 1 -- 172.21.15.60:0/2934365412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c0cad30 con 0x7f7d1c14aa50 2024-04-25T10:08:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d1a59c640 1 --2- 172.21.15.60:0/2934365412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d1c14aa50 0x7f7d1c0ca350 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00e7b0 tx=0x7f7d0c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d0affd640 1 -- 172.21.15.60:0/2934365412 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c00ee60 con 0x7f7d1c14aa50 2024-04-25T10:08:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d0affd640 1 -- 172.21.15.60:0/2934365412 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c018070 con 0x7f7d1c14aa50 2024-04-25T10:08:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d20f31640 1 -- 172.21.15.60:0/2934365412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1c0cb010 con 0x7f7d1c14aa50 2024-04-25T10:08:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d0affd640 1 -- 172.21.15.60:0/2934365412 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c00cb10 con 0x7f7d1c14aa50 2024-04-25T10:08:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.826+0000 7f7d20f31640 1 -- 172.21.15.60:0/2934365412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1c0cb5d0 con 0x7f7d1c14aa50 2024-04-25T10:08:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.830+0000 7f7d20f31640 1 -- 172.21.15.60:0/2934365412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1c074770 con 0x7f7d1c14aa50 2024-04-25T10:08:14.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.830+0000 7f7d0affd640 1 -- 172.21.15.60:0/2934365412 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d0c00ccb0 con 0x7f7d1c14aa50 2024-04-25T10:08:14.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.830+0000 7f7d0affd640 1 --2- 172.21.15.60:0/2934365412 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7cfc066f90 0x7f7cfc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.830+0000 7f7d0affd640 1 -- 172.21.15.60:0/2934365412 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d0c084e10 con 0x7f7d1c14aa50 2024-04-25T10:08:14.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.830+0000 7f7d19d9b640 1 --2- 172.21.15.60:0/2934365412 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7cfc066f90 0x7f7cfc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.830+0000 7f7d19d9b640 1 --2- 172.21.15.60:0/2934365412 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7cfc066f90 0x7f7cfc069450 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7d100ad000 tx=0x7f7d100a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:14.834+0000 7f7d0affd640 1 -- 172.21.15.60:0/2934365412 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7d0c055eb0 con 0x7f7d1c14aa50 2024-04-25T10:08:15.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.050+0000 7f7d20f31640 1 -- 172.21.15.60:0/2934365412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f7d1c0d2600 con 0x7f7d1c14aa50 2024-04-25T10:08:15.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.050+0000 7f7d0affd640 1 -- 172.21.15.60:0/2934365412 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1174 (secure 0 0 0) 0x7f7d0c0557b0 con 0x7f7d1c14aa50 2024-04-25T10:08:15.057 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:15.057 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":199,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039507,"num_in_osds":8,"osd_in_since":1714039498,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":266399744,"bytes_avail":772827713536,"bytes_total":773094113280,"write_bytes_sec":2561,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":28,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":6291}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-04-25T10:08:04.444772+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:08:15.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.050+0000 7f7d20f31640 1 -- 172.21.15.60:0/2934365412 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7cfc066f90 msgr2=0x7f7cfc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.050+0000 7f7d20f31640 1 --2- 172.21.15.60:0/2934365412 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7cfc066f90 0x7f7cfc069450 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7d100ad000 tx=0x7f7d100a7000 comp rx=0 tx=0).stop 2024-04-25T10:08:15.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.050+0000 7f7d20f31640 1 -- 172.21.15.60:0/2934365412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d1c14aa50 msgr2=0x7f7d1c0ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.050+0000 7f7d20f31640 1 --2- 172.21.15.60:0/2934365412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d1c14aa50 0x7f7d1c0ca350 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00e7b0 tx=0x7f7d0c00ec70 comp rx=0 tx=0).stop 2024-04-25T10:08:15.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.050+0000 7f7d20f31640 1 -- 172.21.15.60:0/2934365412 shutdown_connections 2024-04-25T10:08:15.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.050+0000 7f7d20f31640 1 --2- 172.21.15.60:0/2934365412 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7cfc066f90 0x7f7cfc069450 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.050+0000 7f7d20f31640 1 --2- 172.21.15.60:0/2934365412 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1c15b150 0x7f7d1c0b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.050+0000 7f7d20f31640 1 --2- 172.21.15.60:0/2934365412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d1c0ca890 0x7f7d1c0b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.050+0000 7f7d20f31640 1 --2- 172.21.15.60:0/2934365412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d1c14aa50 0x7f7d1c0ca350 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.050+0000 7f7d20f31640 1 -- 172.21.15.60:0/2934365412 >> 172.21.15.60:0/2934365412 conn(0x7f7d1c0d17d0 msgr2=0x7f7d1c0b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:15.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.050+0000 7f7d20f31640 1 -- 172.21.15.60:0/2934365412 shutdown_connections 2024-04-25T10:08:15.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.050+0000 7f7d20f31640 1 -- 172.21.15.60:0/2934365412 wait complete. 2024-04-25T10:08:15.075 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:08:15.075 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:15.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.178+0000 7f9926523640 1 Processor -- start 2024-04-25T10:08:15.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.178+0000 7f9926523640 1 -- start start 2024-04-25T10:08:15.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f9926523640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9920161290 0x7f99200b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f9926523640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99200b9900 con 0x7f99200b6fb0 2024-04-25T10:08:15.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f9926523640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99200ce9c0 con 0x7f9920160eb0 2024-04-25T10:08:15.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f9926523640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99200ceca0 con 0x7f9920161290 2024-04-25T10:08:15.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f9924ac0640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f99200b6fb0 0x7f99200b53f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41528/0 (socket says 172.21.15.60:41528) 2024-04-25T10:08:15.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f9924ac0640 1 -- 172.21.15.60:0/2871653883 learned_addr learned my addr 172.21.15.60:0/2871653883 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:15.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991f7fe640 1 --2- 172.21.15.60:0/2871653883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9920161290 0x7f99200b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 -- 172.21.15.60:0/2871653883 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190836916 0 0) 0x7f99200b9900 con 0x7f99200b6fb0 2024-04-25T10:08:15.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 -- 172.21.15.60:0/2871653883 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9908003530 con 0x7f99200b6fb0 2024-04-25T10:08:15.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 -- 172.21.15.60:0/2871653883 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2169167574 0 0) 0x7f9908003530 con 0x7f99200b6fb0 2024-04-25T10:08:15.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 -- 172.21.15.60:0/2871653883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9920161290 msgr2=0x7f99200b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:15.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 --2- 172.21.15.60:0/2871653883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9920161290 0x7f99200b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 -- 172.21.15.60:0/2871653883 >> v1:172.21.15.100:6789/0 conn(0x7f9920160eb0 legacy=0x7f99200b91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 -- 172.21.15.60:0/2871653883 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99200d2490 con 0x7f99200b6fb0 2024-04-25T10:08:15.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 -- 172.21.15.60:0/2871653883 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f9918001680 con 0x7f99200b6fb0 2024-04-25T10:08:15.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 -- 172.21.15.60:0/2871653883 >> v1:172.21.15.60:6789/0 conn(0x7f99200b6fb0 legacy=0x7f99200b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 --2- 172.21.15.60:0/2871653883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9920161290 0x7f99080039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 --2- 172.21.15.60:0/2871653883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9908003f70 0x7f9908004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 --2- 172.21.15.60:0/2871653883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9908004d50 0x7f99080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 -- 172.21.15.60:0/2871653883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99200ceca0 con 0x7f9908003f70 2024-04-25T10:08:15.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 -- 172.21.15.60:0/2871653883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990800bb40 con 0x7f9908004d50 2024-04-25T10:08:15.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 -- 172.21.15.60:0/2871653883 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990800bd40 con 0x7f9920161290 2024-04-25T10:08:15.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991f7fe640 1 --2- 172.21.15.60:0/2871653883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9908003f70 0x7f9908004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991ffff640 1 --2- 172.21.15.60:0/2871653883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9920161290 0x7f99080039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991f7fe640 1 -- 172.21.15.60:0/2871653883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9920161290 msgr2=0x7f99080039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:15.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991f7fe640 1 --2- 172.21.15.60:0/2871653883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9920161290 0x7f99080039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991f7fe640 1 -- 172.21.15.60:0/2871653883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9908004d50 msgr2=0x7f99080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:15.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991f7fe640 1 --2- 172.21.15.60:0/2871653883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9908004d50 0x7f99080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991f7fe640 1 -- 172.21.15.60:0/2871653883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9908003530 con 0x7f9908003f70 2024-04-25T10:08:15.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991ffff640 1 --2- 172.21.15.60:0/2871653883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9920161290 0x7f99080039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:15.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991f7fe640 1 --2- 172.21.15.60:0/2871653883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9908003f70 0x7f9908004810 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9914009ae0 tx=0x7f991401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d727587157089c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 -- 172.21.15.60:0/2871653883 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99140b2070 con 0x7f9908003f70 2024-04-25T10:08:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 -- 172.21.15.60:0/2871653883 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99140a5660 con 0x7f9908003f70 2024-04-25T10:08:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.182+0000 7f991effd640 1 -- 172.21.15.60:0/2871653883 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99140096b0 con 0x7f9908003f70 2024-04-25T10:08:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9926523640 1 -- 172.21.15.60:0/2871653883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9908003f70 msgr2=0x7f9908004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9926523640 1 --2- 172.21.15.60:0/2871653883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9908003f70 0x7f9908004810 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9914009ae0 tx=0x7f991401ca70 comp rx=0 tx=0).stop 2024-04-25T10:08:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9924ac0640 1 -- 172.21.15.60:0/2871653883 reap_dead start 2024-04-25T10:08:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9926523640 1 -- 172.21.15.60:0/2871653883 shutdown_connections 2024-04-25T10:08:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9926523640 1 -- 172.21.15.60:0/2871653883 >> 172.21.15.60:0/2871653883 conn(0x7f99200d05c0 msgr2=0x7f99200d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9926523640 1 -- 172.21.15.60:0/2871653883 shutdown_connections 2024-04-25T10:08:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9926523640 1 -- 172.21.15.60:0/2871653883 wait complete. 2024-04-25T10:08:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9926523640 1 Processor -- start 2024-04-25T10:08:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9926523640 1 -- start start 2024-04-25T10:08:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9926523640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99200b4960 0x7f99200ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9926523640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99200b4d40 0x7f992014e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9926523640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f992014eeb0 0x7f9920158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9926523640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99200cef80 con 0x7f99200b4960 2024-04-25T10:08:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9926523640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99200b2e70 con 0x7f992014eeb0 2024-04-25T10:08:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9926523640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99200b5b00 con 0x7f99200b4d40 2024-04-25T10:08:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f991ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99200b4960 0x7f99200ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f991ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99200b4960 0x7f99200ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58816/0 (socket says 172.21.15.60:58816) 2024-04-25T10:08:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f991ffff640 1 -- 172.21.15.60:0/4215942735 learned_addr learned my addr 172.21.15.60:0/4215942735 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9924ac0640 1 --2- 172.21.15.60:0/4215942735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f992014eeb0 0x7f9920158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f991f7fe640 1 --2- 172.21.15.60:0/4215942735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99200b4d40 0x7f992014e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f991ffff640 1 -- 172.21.15.60:0/4215942735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99200b4d40 msgr2=0x7f992014e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f991ffff640 1 --2- 172.21.15.60:0/4215942735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99200b4d40 0x7f992014e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f991ffff640 1 -- 172.21.15.60:0/4215942735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f992014eeb0 msgr2=0x7f9920158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f991ffff640 1 --2- 172.21.15.60:0/4215942735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f992014eeb0 0x7f9920158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f991ffff640 1 -- 172.21.15.60:0/4215942735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992014f310 con 0x7f99200b4960 2024-04-25T10:08:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f991f7fe640 1 --2- 172.21.15.60:0/4215942735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99200b4d40 0x7f992014e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9924ac0640 1 --2- 172.21.15.60:0/4215942735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f992014eeb0 0x7f9920158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f991ffff640 1 --2- 172.21.15.60:0/4215942735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99200b4960 0x7f99200ccc50 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f991000db40 tx=0x7f99100020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f991cff9640 1 -- 172.21.15.60:0/4215942735 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910003bb0 con 0x7f99200b4960 2024-04-25T10:08:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f991cff9640 1 -- 172.21.15.60:0/4215942735 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9910003d50 con 0x7f99200b4960 2024-04-25T10:08:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9926523640 1 -- 172.21.15.60:0/4215942735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9920159df0 con 0x7f99200b4960 2024-04-25T10:08:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f991cff9640 1 -- 172.21.15.60:0/4215942735 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910006d80 con 0x7f99200b4960 2024-04-25T10:08:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.186+0000 7f9926523640 1 -- 172.21.15.60:0/4215942735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992015a690 con 0x7f99200b4960 2024-04-25T10:08:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.190+0000 7f9926523640 1 -- 172.21.15.60:0/4215942735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9920074770 con 0x7f99200b4960 2024-04-25T10:08:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.190+0000 7f991cff9640 1 -- 172.21.15.60:0/4215942735 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f991001a020 con 0x7f99200b4960 2024-04-25T10:08:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.190+0000 7f991cff9640 1 --2- 172.21.15.60:0/4215942735 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f990c066f60 0x7f990c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.190+0000 7f991cff9640 1 -- 172.21.15.60:0/4215942735 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9910084c10 con 0x7f99200b4960 2024-04-25T10:08:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.190+0000 7f991f7fe640 1 --2- 172.21.15.60:0/4215942735 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f990c066f60 0x7f990c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.190+0000 7f991f7fe640 1 --2- 172.21.15.60:0/4215942735 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f990c066f60 0x7f990c069420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f99140af000 tx=0x7f9914008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.194+0000 7f991cff9640 1 -- 172.21.15.60:0/4215942735 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9910055c20 con 0x7f99200b4960 2024-04-25T10:08:15.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.346+0000 7f9926523640 1 -- 172.21.15.60:0/4215942735 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f99200d15b0 con 0x7f990c066f60 2024-04-25T10:08:15.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.362+0000 7f991cff9640 1 -- 172.21.15.60:0/4215942735 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251948 (secure 0 0 0) 0x7f99200d15b0 con 0x7f990c066f60 2024-04-25T10:08:15.374 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:15.376 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:08:15.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.366+0000 7f9926523640 1 -- 172.21.15.60:0/4215942735 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f990c066f60 msgr2=0x7f990c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.366+0000 7f9926523640 1 --2- 172.21.15.60:0/4215942735 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f990c066f60 0x7f990c069420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f99140af000 tx=0x7f9914008040 comp rx=0 tx=0).stop 2024-04-25T10:08:15.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.366+0000 7f9926523640 1 -- 172.21.15.60:0/4215942735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99200b4960 msgr2=0x7f99200ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.366+0000 7f9926523640 1 --2- 172.21.15.60:0/4215942735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99200b4960 0x7f99200ccc50 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f991000db40 tx=0x7f99100020f0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.370+0000 7f9926523640 1 -- 172.21.15.60:0/4215942735 shutdown_connections 2024-04-25T10:08:15.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.370+0000 7f9926523640 1 --2- 172.21.15.60:0/4215942735 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f990c066f60 0x7f990c069420 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.370+0000 7f9926523640 1 --2- 172.21.15.60:0/4215942735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f992014eeb0 0x7f9920158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.370+0000 7f9926523640 1 --2- 172.21.15.60:0/4215942735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99200b4d40 0x7f992014e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.370+0000 7f9926523640 1 --2- 172.21.15.60:0/4215942735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99200b4960 0x7f99200ccc50 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.370+0000 7f9926523640 1 -- 172.21.15.60:0/4215942735 >> 172.21.15.60:0/4215942735 conn(0x7f99200d05c0 msgr2=0x7f99200d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:15.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.370+0000 7f9926523640 1 -- 172.21.15.60:0/4215942735 shutdown_connections 2024-04-25T10:08:15.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:15.370+0000 7f9926523640 1 -- 172.21.15.60:0/4215942735 wait complete. 2024-04-25T10:08:15.392 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-04-25T10:08:14.447680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260156,"kb_used_data":6540,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754714564,"statfs":{"total":773094113280,"available":772827713536,"internally_reserved":0,"allocated":6696960,"data_stored":3441264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.596128"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.896431+0000","last_change":"2024-04-25T10:08:10.894975+0000","last_active":"2024-04-25T10:08:10.896431+0000","last_peered":"2024-04-25T10:08:10.896431+0000","last_clean":"2024-04-25T10:08:10.896431+0000","last_became_active":"2024-04-25T10:08:10.894724+0000","last_became_peered":"2024-04-25T10:08:10.894724+0000","last_unstale":"2024-04-25T10:08:10.896431+0000","last_undegraded":"2024-04-25T10:08:10.896431+0000","last_fullsized":"2024-04-25T10:08:10.896431+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:57.357697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901577+0000","last_change":"2024-04-25T10:08:10.893083+0000","last_active":"2024-04-25T10:08:10.901577+0000","last_peered":"2024-04-25T10:08:10.901577+0000","last_clean":"2024-04-25T10:08:10.901577+0000","last_became_active":"2024-04-25T10:08:10.892785+0000","last_became_peered":"2024-04-25T10:08:10.892785+0000","last_unstale":"2024-04-25T10:08:10.901577+0000","last_undegraded":"2024-04-25T10:08:10.901577+0000","last_fullsized":"2024-04-25T10:08:10.901577+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:31.332615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396629+0000","last_change":"2024-04-25T10:08:10.916953+0000","last_active":"2024-04-25T10:08:13.396629+0000","last_peered":"2024-04-25T10:08:13.396629+0000","last_clean":"2024-04-25T10:08:13.396629+0000","last_became_active":"2024-04-25T10:08:10.910705+0000","last_became_peered":"2024-04-25T10:08:10.910705+0000","last_unstale":"2024-04-25T10:08:13.396629+0000","last_undegraded":"2024-04-25T10:08:13.396629+0000","last_fullsized":"2024-04-25T10:08:13.396629+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:04.768312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893692+0000","last_change":"2024-04-25T10:08:10.901432+0000","last_active":"2024-04-25T10:08:11.893692+0000","last_peered":"2024-04-25T10:08:11.893692+0000","last_clean":"2024-04-25T10:08:11.893692+0000","last_became_active":"2024-04-25T10:08:10.900367+0000","last_became_peered":"2024-04-25T10:08:10.900367+0000","last_unstale":"2024-04-25T10:08:11.893692+0000","last_undegraded":"2024-04-25T10:08:11.893692+0000","last_fullsized":"2024-04-25T10:08:11.893692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:27.633134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.910107+0000","last_change":"2024-04-25T10:08:10.909538+0000","last_active":"2024-04-25T10:08:10.910107+0000","last_peered":"2024-04-25T10:08:10.910107+0000","last_clean":"2024-04-25T10:08:10.910107+0000","last_became_active":"2024-04-25T10:08:10.909284+0000","last_became_peered":"2024-04-25T10:08:10.909284+0000","last_unstale":"2024-04-25T10:08:10.910107+0000","last_undegraded":"2024-04-25T10:08:10.910107+0000","last_fullsized":"2024-04-25T10:08:10.910107+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:36.225766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.889260+0000","last_change":"2024-04-25T10:08:10.887190+0000","last_active":"2024-04-25T10:08:10.889260+0000","last_peered":"2024-04-25T10:08:10.889260+0000","last_clean":"2024-04-25T10:08:10.889260+0000","last_became_active":"2024-04-25T10:08:10.886910+0000","last_became_peered":"2024-04-25T10:08:10.886910+0000","last_unstale":"2024-04-25T10:08:10.889260+0000","last_undegraded":"2024-04-25T10:08:10.889260+0000","last_fullsized":"2024-04-25T10:08:10.889260+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:33.123610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893402+0000","last_change":"2024-04-25T10:08:10.917873+0000","last_active":"2024-04-25T10:08:11.893402+0000","last_peered":"2024-04-25T10:08:11.893402+0000","last_clean":"2024-04-25T10:08:11.893402+0000","last_became_active":"2024-04-25T10:08:10.916140+0000","last_became_peered":"2024-04-25T10:08:10.916140+0000","last_unstale":"2024-04-25T10:08:11.893402+0000","last_undegraded":"2024-04-25T10:08:11.893402+0000","last_fullsized":"2024-04-25T10:08:11.893402+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:12.105474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.906628+0000","last_change":"2024-04-25T10:08:10.905892+0000","last_active":"2024-04-25T10:08:10.906628+0000","last_peered":"2024-04-25T10:08:10.906628+0000","last_clean":"2024-04-25T10:08:10.906628+0000","last_became_active":"2024-04-25T10:08:10.905443+0000","last_became_peered":"2024-04-25T10:08:10.905443+0000","last_unstale":"2024-04-25T10:08:10.906628+0000","last_undegraded":"2024-04-25T10:08:10.906628+0000","last_fullsized":"2024-04-25T10:08:10.906628+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:09.407230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894567+0000","last_change":"2024-04-25T10:08:10.884528+0000","last_active":"2024-04-25T10:08:11.894567+0000","last_peered":"2024-04-25T10:08:11.894567+0000","last_clean":"2024-04-25T10:08:11.894567+0000","last_became_active":"2024-04-25T10:08:10.884183+0000","last_became_peered":"2024-04-25T10:08:10.884183+0000","last_unstale":"2024-04-25T10:08:11.894567+0000","last_undegraded":"2024-04-25T10:08:11.894567+0000","last_fullsized":"2024-04-25T10:08:11.894567+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:42.851719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889904+0000","last_change":"2024-04-25T10:08:10.894120+0000","last_active":"2024-04-25T10:08:11.889904+0000","last_peered":"2024-04-25T10:08:11.889904+0000","last_clean":"2024-04-25T10:08:11.889904+0000","last_became_active":"2024-04-25T10:08:10.893807+0000","last_became_peered":"2024-04-25T10:08:10.893807+0000","last_unstale":"2024-04-25T10:08:11.889904+0000","last_undegraded":"2024-04-25T10:08:11.889904+0000","last_fullsized":"2024-04-25T10:08:11.889904+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:10.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.890117+0000","last_change":"2024-04-25T10:08:10.893934+0000","last_active":"2024-04-25T10:08:11.890117+0000","last_peered":"2024-04-25T10:08:11.890117+0000","last_clean":"2024-04-25T10:08:11.890117+0000","last_became_active":"2024-04-25T10:08:10.893270+0000","last_became_peered":"2024-04-25T10:08:10.893270+0000","last_unstale":"2024-04-25T10:08:11.890117+0000","last_undegraded":"2024-04-25T10:08:11.890117+0000","last_fullsized":"2024-04-25T10:08:11.890117+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:13.537677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894126+0000","last_change":"2024-04-25T10:08:10.923912+0000","last_active":"2024-04-25T10:08:11.894126+0000","last_peered":"2024-04-25T10:08:11.894126+0000","last_clean":"2024-04-25T10:08:11.894126+0000","last_became_active":"2024-04-25T10:08:10.923319+0000","last_became_peered":"2024-04-25T10:08:10.923319+0000","last_unstale":"2024-04-25T10:08:11.894126+0000","last_undegraded":"2024-04-25T10:08:11.894126+0000","last_fullsized":"2024-04-25T10:08:11.894126+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:44.473365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.968119+0000","last_change":"2024-04-25T10:08:10.886846+0000","last_active":"2024-04-25T10:08:11.968119+0000","last_peered":"2024-04-25T10:08:11.968119+0000","last_clean":"2024-04-25T10:08:11.968119+0000","last_became_active":"2024-04-25T10:08:10.886497+0000","last_became_peered":"2024-04-25T10:08:10.886497+0000","last_unstale":"2024-04-25T10:08:11.968119+0000","last_undegraded":"2024-04-25T10:08:11.968119+0000","last_fullsized":"2024-04-25T10:08:11.968119+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:56.861799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893740+0000","last_change":"2024-04-25T10:08:10.907514+0000","last_active":"2024-04-25T10:08:11.893740+0000","last_peered":"2024-04-25T10:08:11.893740+0000","last_clean":"2024-04-25T10:08:11.893740+0000","last_became_active":"2024-04-25T10:08:10.905752+0000","last_became_peered":"2024-04-25T10:08:10.905752+0000","last_unstale":"2024-04-25T10:08:11.893740+0000","last_undegraded":"2024-04-25T10:08:11.893740+0000","last_fullsized":"2024-04-25T10:08:11.893740+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:15.349210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893812+0000","last_change":"2024-04-25T10:08:10.882723+0000","last_active":"2024-04-25T10:08:11.893812+0000","last_peered":"2024-04-25T10:08:11.893812+0000","last_clean":"2024-04-25T10:08:11.893812+0000","last_became_active":"2024-04-25T10:08:10.882339+0000","last_became_peered":"2024-04-25T10:08:10.882339+0000","last_unstale":"2024-04-25T10:08:11.893812+0000","last_undegraded":"2024-04-25T10:08:11.893812+0000","last_fullsized":"2024-04-25T10:08:11.893812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:13.179674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.897978+0000","last_change":"2024-04-25T10:08:10.896865+0000","last_active":"2024-04-25T10:08:10.897978+0000","last_peered":"2024-04-25T10:08:10.897978+0000","last_clean":"2024-04-25T10:08:10.897978+0000","last_became_active":"2024-04-25T10:08:10.896575+0000","last_became_peered":"2024-04-25T10:08:10.896575+0000","last_unstale":"2024-04-25T10:08:10.897978+0000","last_undegraded":"2024-04-25T10:08:10.897978+0000","last_fullsized":"2024-04-25T10:08:10.897978+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:31.167704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901912+0000","last_change":"2024-04-25T10:08:10.898151+0000","last_active":"2024-04-25T10:08:10.901912+0000","last_peered":"2024-04-25T10:08:10.901912+0000","last_clean":"2024-04-25T10:08:10.901912+0000","last_became_active":"2024-04-25T10:08:10.897849+0000","last_became_peered":"2024-04-25T10:08:10.897849+0000","last_unstale":"2024-04-25T10:08:10.901912+0000","last_undegraded":"2024-04-25T10:08:10.901912+0000","last_fullsized":"2024-04-25T10:08:10.901912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:47.458907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396698+0000","last_change":"2024-04-25T10:08:10.898156+0000","last_active":"2024-04-25T10:08:13.396698+0000","last_peered":"2024-04-25T10:08:13.396698+0000","last_clean":"2024-04-25T10:08:13.396698+0000","last_became_active":"2024-04-25T10:08:10.897326+0000","last_became_peered":"2024-04-25T10:08:10.897326+0000","last_unstale":"2024-04-25T10:08:13.396698+0000","last_undegraded":"2024-04-25T10:08:13.396698+0000","last_fullsized":"2024-04-25T10:08:13.396698+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:03.978144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.895610+0000","last_change":"2024-04-25T10:08:10.891553+0000","last_active":"2024-04-25T10:08:10.895610+0000","last_peered":"2024-04-25T10:08:10.895610+0000","last_clean":"2024-04-25T10:08:10.895610+0000","last_became_active":"2024-04-25T10:08:10.891348+0000","last_became_peered":"2024-04-25T10:08:10.891348+0000","last_unstale":"2024-04-25T10:08:10.895610+0000","last_undegraded":"2024-04-25T10:08:10.895610+0000","last_fullsized":"2024-04-25T10:08:10.895610+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:48.658370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.897906+0000","last_change":"2024-04-25T10:08:10.890188+0000","last_active":"2024-04-25T10:08:10.897906+0000","last_peered":"2024-04-25T10:08:10.897906+0000","last_clean":"2024-04-25T10:08:10.897906+0000","last_became_active":"2024-04-25T10:08:10.889827+0000","last_became_peered":"2024-04-25T10:08:10.889827+0000","last_unstale":"2024-04-25T10:08:10.897906+0000","last_undegraded":"2024-04-25T10:08:10.897906+0000","last_fullsized":"2024-04-25T10:08:10.897906+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.834110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396312+0000","last_change":"2024-04-25T10:08:10.930150+0000","last_active":"2024-04-25T10:08:13.396312+0000","last_peered":"2024-04-25T10:08:13.396312+0000","last_clean":"2024-04-25T10:08:13.396312+0000","last_became_active":"2024-04-25T10:08:10.929406+0000","last_became_peered":"2024-04-25T10:08:10.929406+0000","last_unstale":"2024-04-25T10:08:13.396312+0000","last_undegraded":"2024-04-25T10:08:13.396312+0000","last_fullsized":"2024-04-25T10:08:13.396312+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:43.406629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.900281+0000","last_change":"2024-04-25T10:08:10.892469+0000","last_active":"2024-04-25T10:08:10.900281+0000","last_peered":"2024-04-25T10:08:10.900281+0000","last_clean":"2024-04-25T10:08:10.900281+0000","last_became_active":"2024-04-25T10:08:10.890763+0000","last_became_peered":"2024-04-25T10:08:10.890763+0000","last_unstale":"2024-04-25T10:08:10.900281+0000","last_undegraded":"2024-04-25T10:08:10.900281+0000","last_fullsized":"2024-04-25T10:08:10.900281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:30.075855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396550+0000","last_change":"2024-04-25T10:08:10.923545+0000","last_active":"2024-04-25T10:08:13.396550+0000","last_peered":"2024-04-25T10:08:13.396550+0000","last_clean":"2024-04-25T10:08:13.396550+0000","last_became_active":"2024-04-25T10:08:10.919185+0000","last_became_peered":"2024-04-25T10:08:10.919185+0000","last_unstale":"2024-04-25T10:08:13.396550+0000","last_undegraded":"2024-04-25T10:08:13.396550+0000","last_fullsized":"2024-04-25T10:08:13.396550+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:48.684084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967689+0000","last_change":"2024-04-25T10:08:10.897718+0000","last_active":"2024-04-25T10:08:11.967689+0000","last_peered":"2024-04-25T10:08:11.967689+0000","last_clean":"2024-04-25T10:08:11.967689+0000","last_became_active":"2024-04-25T10:08:10.897021+0000","last_became_peered":"2024-04-25T10:08:10.897021+0000","last_unstale":"2024-04-25T10:08:11.967689+0000","last_undegraded":"2024-04-25T10:08:11.967689+0000","last_fullsized":"2024-04-25T10:08:11.967689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:10.256509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.889404+0000","last_change":"2024-04-25T10:08:10.885585+0000","last_active":"2024-04-25T10:08:10.889404+0000","last_peered":"2024-04-25T10:08:10.889404+0000","last_clean":"2024-04-25T10:08:10.889404+0000","last_became_active":"2024-04-25T10:08:10.885318+0000","last_became_peered":"2024-04-25T10:08:10.885318+0000","last_unstale":"2024-04-25T10:08:10.889404+0000","last_undegraded":"2024-04-25T10:08:10.889404+0000","last_fullsized":"2024-04-25T10:08:10.889404+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:07.762131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893986+0000","last_change":"2024-04-25T10:08:10.911391+0000","last_active":"2024-04-25T10:08:11.893986+0000","last_peered":"2024-04-25T10:08:11.893986+0000","last_clean":"2024-04-25T10:08:11.893986+0000","last_became_active":"2024-04-25T10:08:10.910461+0000","last_became_peered":"2024-04-25T10:08:10.910461+0000","last_unstale":"2024-04-25T10:08:11.893986+0000","last_undegraded":"2024-04-25T10:08:11.893986+0000","last_fullsized":"2024-04-25T10:08:11.893986+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:48.269167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.896261+0000","last_change":"2024-04-25T10:08:10.894475+0000","last_active":"2024-04-25T10:08:10.896261+0000","last_peered":"2024-04-25T10:08:10.896261+0000","last_clean":"2024-04-25T10:08:10.896261+0000","last_became_active":"2024-04-25T10:08:10.894176+0000","last_became_peered":"2024-04-25T10:08:10.894176+0000","last_unstale":"2024-04-25T10:08:10.896261+0000","last_undegraded":"2024-04-25T10:08:10.896261+0000","last_fullsized":"2024-04-25T10:08:10.896261+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:57.804679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.902435+0000","last_change":"2024-04-25T10:08:10.901642+0000","last_active":"2024-04-25T10:08:10.902435+0000","last_peered":"2024-04-25T10:08:10.902435+0000","last_clean":"2024-04-25T10:08:10.902435+0000","last_became_active":"2024-04-25T10:08:10.901389+0000","last_became_peered":"2024-04-25T10:08:10.901389+0000","last_unstale":"2024-04-25T10:08:10.902435+0000","last_undegraded":"2024-04-25T10:08:10.902435+0000","last_fullsized":"2024-04-25T10:08:10.902435+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:01.232475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.915810+0000","last_change":"2024-04-25T10:08:10.915357+0000","last_active":"2024-04-25T10:08:10.915810+0000","last_peered":"2024-04-25T10:08:10.915810+0000","last_clean":"2024-04-25T10:08:10.915810+0000","last_became_active":"2024-04-25T10:08:10.915052+0000","last_became_peered":"2024-04-25T10:08:10.915052+0000","last_unstale":"2024-04-25T10:08:10.915810+0000","last_undegraded":"2024-04-25T10:08:10.915810+0000","last_fullsized":"2024-04-25T10:08:10.915810+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:13.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396716+0000","last_change":"2024-04-25T10:08:10.922837+0000","last_active":"2024-04-25T10:08:13.396716+0000","last_peered":"2024-04-25T10:08:13.396716+0000","last_clean":"2024-04-25T10:08:13.396716+0000","last_became_active":"2024-04-25T10:08:10.921976+0000","last_became_peered":"2024-04-25T10:08:10.921976+0000","last_unstale":"2024-04-25T10:08:13.396716+0000","last_undegraded":"2024-04-25T10:08:13.396716+0000","last_fullsized":"2024-04-25T10:08:13.396716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:09.814025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.889411+0000","last_change":"2024-04-25T10:08:10.884959+0000","last_active":"2024-04-25T10:08:10.889411+0000","last_peered":"2024-04-25T10:08:10.889411+0000","last_clean":"2024-04-25T10:08:10.889411+0000","last_became_active":"2024-04-25T10:08:10.884659+0000","last_became_peered":"2024-04-25T10:08:10.884659+0000","last_unstale":"2024-04-25T10:08:10.889411+0000","last_undegraded":"2024-04-25T10:08:10.889411+0000","last_fullsized":"2024-04-25T10:08:10.889411+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:42.724053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.896757+0000","last_change":"2024-04-25T10:08:10.895828+0000","last_active":"2024-04-25T10:08:10.896757+0000","last_peered":"2024-04-25T10:08:10.896757+0000","last_clean":"2024-04-25T10:08:10.896757+0000","last_became_active":"2024-04-25T10:08:10.895589+0000","last_became_peered":"2024-04-25T10:08:10.895589+0000","last_unstale":"2024-04-25T10:08:10.896757+0000","last_undegraded":"2024-04-25T10:08:10.896757+0000","last_fullsized":"2024-04-25T10:08:10.896757+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:45.805791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.912538+0000","last_change":"2024-04-25T10:08:10.911873+0000","last_active":"2024-04-25T10:08:10.912538+0000","last_peered":"2024-04-25T10:08:10.912538+0000","last_clean":"2024-04-25T10:08:10.912538+0000","last_became_active":"2024-04-25T10:08:10.911603+0000","last_became_peered":"2024-04-25T10:08:10.911603+0000","last_unstale":"2024-04-25T10:08:10.912538+0000","last_undegraded":"2024-04-25T10:08:10.912538+0000","last_fullsized":"2024-04-25T10:08:10.912538+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:27.633134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.926180+0000","last_change":"2024-04-25T10:08:09.906122+0000","last_active":"2024-04-25T10:08:10.926180+0000","last_peered":"2024-04-25T10:08:10.926180+0000","last_clean":"2024-04-25T10:08:10.926180+0000","last_became_active":"2024-04-25T10:08:09.905879+0000","last_became_peered":"2024-04-25T10:08:09.905879+0000","last_unstale":"2024-04-25T10:08:10.926180+0000","last_undegraded":"2024-04-25T10:08:10.926180+0000","last_fullsized":"2024-04-25T10:08:10.926180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:48.349703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.890056+0000","last_change":"2024-04-25T10:08:10.911299+0000","last_active":"2024-04-25T10:08:11.890056+0000","last_peered":"2024-04-25T10:08:11.890056+0000","last_clean":"2024-04-25T10:08:11.890056+0000","last_became_active":"2024-04-25T10:08:10.910974+0000","last_became_peered":"2024-04-25T10:08:10.910974+0000","last_unstale":"2024-04-25T10:08:11.890056+0000","last_undegraded":"2024-04-25T10:08:11.890056+0000","last_fullsized":"2024-04-25T10:08:11.890056+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:22.511670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.921727+0000","last_change":"2024-04-25T10:08:09.898224+0000","last_active":"2024-04-25T10:08:10.921727+0000","last_peered":"2024-04-25T10:08:10.921727+0000","last_clean":"2024-04-25T10:08:10.921727+0000","last_became_active":"2024-04-25T10:08:09.897935+0000","last_became_peered":"2024-04-25T10:08:09.897935+0000","last_unstale":"2024-04-25T10:08:10.921727+0000","last_undegraded":"2024-04-25T10:08:10.921727+0000","last_fullsized":"2024-04-25T10:08:10.921727+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:16.589298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.884524+0000","last_change":"2024-04-25T10:08:10.882470+0000","last_active":"2024-04-25T10:08:10.884524+0000","last_peered":"2024-04-25T10:08:10.884524+0000","last_clean":"2024-04-25T10:08:10.884524+0000","last_became_active":"2024-04-25T10:08:10.882185+0000","last_became_peered":"2024-04-25T10:08:10.882185+0000","last_unstale":"2024-04-25T10:08:10.884524+0000","last_undegraded":"2024-04-25T10:08:10.884524+0000","last_fullsized":"2024-04-25T10:08:10.884524+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.344111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.889456+0000","last_change":"2024-04-25T10:08:09.888457+0000","last_active":"2024-04-25T10:08:10.889456+0000","last_peered":"2024-04-25T10:08:10.889456+0000","last_clean":"2024-04-25T10:08:10.889456+0000","last_became_active":"2024-04-25T10:08:09.888103+0000","last_became_peered":"2024-04-25T10:08:09.888103+0000","last_unstale":"2024-04-25T10:08:10.889456+0000","last_undegraded":"2024-04-25T10:08:10.889456+0000","last_fullsized":"2024-04-25T10:08:10.889456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.255517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.927157+0000","last_change":"2024-04-25T10:08:10.926635+0000","last_active":"2024-04-25T10:08:10.927157+0000","last_peered":"2024-04-25T10:08:10.927157+0000","last_clean":"2024-04-25T10:08:10.927157+0000","last_became_active":"2024-04-25T10:08:10.926349+0000","last_became_peered":"2024-04-25T10:08:10.926349+0000","last_unstale":"2024-04-25T10:08:10.927157+0000","last_undegraded":"2024-04-25T10:08:10.927157+0000","last_fullsized":"2024-04-25T10:08:10.927157+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:17.598272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894215+0000","last_change":"2024-04-25T10:08:09.907608+0000","last_active":"2024-04-25T10:08:11.894215+0000","last_peered":"2024-04-25T10:08:11.894215+0000","last_clean":"2024-04-25T10:08:11.894215+0000","last_became_active":"2024-04-25T10:08:09.906963+0000","last_became_peered":"2024-04-25T10:08:09.906963+0000","last_unstale":"2024-04-25T10:08:11.894215+0000","last_undegraded":"2024-04-25T10:08:11.894215+0000","last_fullsized":"2024-04-25T10:08:11.894215+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.255517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.900251+0000","last_change":"2024-04-25T10:08:10.892319+0000","last_active":"2024-04-25T10:08:10.900251+0000","last_peered":"2024-04-25T10:08:10.900251+0000","last_clean":"2024-04-25T10:08:10.900251+0000","last_became_active":"2024-04-25T10:08:10.890374+0000","last_became_peered":"2024-04-25T10:08:10.890374+0000","last_unstale":"2024-04-25T10:08:10.900251+0000","last_undegraded":"2024-04-25T10:08:10.900251+0000","last_fullsized":"2024-04-25T10:08:10.900251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:21.713220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893927+0000","last_change":"2024-04-25T10:08:09.886156+0000","last_active":"2024-04-25T10:08:11.893927+0000","last_peered":"2024-04-25T10:08:11.893927+0000","last_clean":"2024-04-25T10:08:11.893927+0000","last_became_active":"2024-04-25T10:08:09.885288+0000","last_became_peered":"2024-04-25T10:08:09.885288+0000","last_unstale":"2024-04-25T10:08:11.893927+0000","last_undegraded":"2024-04-25T10:08:11.893927+0000","last_fullsized":"2024-04-25T10:08:11.893927+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:39.542515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.900555+0000","last_change":"2024-04-25T10:08:10.890541+0000","last_active":"2024-04-25T10:08:10.900555+0000","last_peered":"2024-04-25T10:08:10.900555+0000","last_clean":"2024-04-25T10:08:10.900555+0000","last_became_active":"2024-04-25T10:08:10.890280+0000","last_became_peered":"2024-04-25T10:08:10.890280+0000","last_unstale":"2024-04-25T10:08:10.900555+0000","last_undegraded":"2024-04-25T10:08:10.900555+0000","last_fullsized":"2024-04-25T10:08:10.900555+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:35.281267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.895707+0000","last_change":"2024-04-25T10:08:09.894473+0000","last_active":"2024-04-25T10:08:10.895707+0000","last_peered":"2024-04-25T10:08:10.895707+0000","last_clean":"2024-04-25T10:08:10.895707+0000","last_became_active":"2024-04-25T10:08:09.894181+0000","last_became_peered":"2024-04-25T10:08:09.894181+0000","last_unstale":"2024-04-25T10:08:10.895707+0000","last_undegraded":"2024-04-25T10:08:10.895707+0000","last_fullsized":"2024-04-25T10:08:10.895707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:10.625997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967711+0000","last_change":"2024-04-25T10:08:10.923620+0000","last_active":"2024-04-25T10:08:11.967711+0000","last_peered":"2024-04-25T10:08:11.967711+0000","last_clean":"2024-04-25T10:08:11.967711+0000","last_became_active":"2024-04-25T10:08:10.923032+0000","last_became_peered":"2024-04-25T10:08:10.923032+0000","last_unstale":"2024-04-25T10:08:11.967711+0000","last_undegraded":"2024-04-25T10:08:11.967711+0000","last_fullsized":"2024-04-25T10:08:11.967711+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:10.085497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893167+0000","last_change":"2024-04-25T10:08:09.907088+0000","last_active":"2024-04-25T10:08:11.893167+0000","last_peered":"2024-04-25T10:08:11.893167+0000","last_clean":"2024-04-25T10:08:11.893167+0000","last_became_active":"2024-04-25T10:08:09.906310+0000","last_became_peered":"2024-04-25T10:08:09.906310+0000","last_unstale":"2024-04-25T10:08:11.893167+0000","last_undegraded":"2024-04-25T10:08:11.893167+0000","last_fullsized":"2024-04-25T10:08:11.893167+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.592539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901609+0000","last_change":"2024-04-25T10:08:10.892966+0000","last_active":"2024-04-25T10:08:10.901609+0000","last_peered":"2024-04-25T10:08:10.901609+0000","last_clean":"2024-04-25T10:08:10.901609+0000","last_became_active":"2024-04-25T10:08:10.892667+0000","last_became_peered":"2024-04-25T10:08:10.892667+0000","last_unstale":"2024-04-25T10:08:10.901609+0000","last_undegraded":"2024-04-25T10:08:10.901609+0000","last_fullsized":"2024-04-25T10:08:10.901609+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:34.974943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893151+0000","last_change":"2024-04-25T10:08:09.889905+0000","last_active":"2024-04-25T10:08:11.893151+0000","last_peered":"2024-04-25T10:08:11.893151+0000","last_clean":"2024-04-25T10:08:11.893151+0000","last_became_active":"2024-04-25T10:08:09.889171+0000","last_became_peered":"2024-04-25T10:08:09.889171+0000","last_unstale":"2024-04-25T10:08:11.893151+0000","last_undegraded":"2024-04-25T10:08:11.893151+0000","last_fullsized":"2024-04-25T10:08:11.893151+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:46.888532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894707+0000","last_change":"2024-04-25T10:08:10.886122+0000","last_active":"2024-04-25T10:08:11.894707+0000","last_peered":"2024-04-25T10:08:11.894707+0000","last_clean":"2024-04-25T10:08:11.894707+0000","last_became_active":"2024-04-25T10:08:10.882247+0000","last_became_peered":"2024-04-25T10:08:10.882247+0000","last_unstale":"2024-04-25T10:08:11.894707+0000","last_undegraded":"2024-04-25T10:08:11.894707+0000","last_fullsized":"2024-04-25T10:08:11.894707+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:58.336224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.397277+0000","last_change":"2024-04-25T10:08:09.899045+0000","last_active":"2024-04-25T10:08:13.397277+0000","last_peered":"2024-04-25T10:08:13.397277+0000","last_clean":"2024-04-25T10:08:13.397277+0000","last_became_active":"2024-04-25T10:08:09.898282+0000","last_became_peered":"2024-04-25T10:08:09.898282+0000","last_unstale":"2024-04-25T10:08:13.397277+0000","last_undegraded":"2024-04-25T10:08:13.397277+0000","last_fullsized":"2024-04-25T10:08:13.397277+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:34.532573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.397322+0000","last_change":"2024-04-25T10:08:10.919031+0000","last_active":"2024-04-25T10:08:13.397322+0000","last_peered":"2024-04-25T10:08:13.397322+0000","last_clean":"2024-04-25T10:08:13.397322+0000","last_became_active":"2024-04-25T10:08:10.911880+0000","last_became_peered":"2024-04-25T10:08:10.911880+0000","last_unstale":"2024-04-25T10:08:13.397322+0000","last_undegraded":"2024-04-25T10:08:13.397322+0000","last_fullsized":"2024-04-25T10:08:13.397322+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:45.833608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.884018+0000","last_change":"2024-04-25T10:08:09.880232+0000","last_active":"2024-04-25T10:08:10.884018+0000","last_peered":"2024-04-25T10:08:10.884018+0000","last_clean":"2024-04-25T10:08:10.884018+0000","last_became_active":"2024-04-25T10:08:09.879917+0000","last_became_peered":"2024-04-25T10:08:09.879917+0000","last_unstale":"2024-04-25T10:08:10.884018+0000","last_undegraded":"2024-04-25T10:08:10.884018+0000","last_fullsized":"2024-04-25T10:08:10.884018+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:43.464391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889651+0000","last_change":"2024-04-25T10:08:10.893598+0000","last_active":"2024-04-25T10:08:11.889651+0000","last_peered":"2024-04-25T10:08:11.889651+0000","last_clean":"2024-04-25T10:08:11.889651+0000","last_became_active":"2024-04-25T10:08:10.893008+0000","last_became_peered":"2024-04-25T10:08:10.893008+0000","last_unstale":"2024-04-25T10:08:11.889651+0000","last_undegraded":"2024-04-25T10:08:11.889651+0000","last_fullsized":"2024-04-25T10:08:11.889651+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:57.430500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396978+0000","last_change":"2024-04-25T10:08:09.899600+0000","last_active":"2024-04-25T10:08:13.396978+0000","last_peered":"2024-04-25T10:08:13.396978+0000","last_clean":"2024-04-25T10:08:13.396978+0000","last_became_active":"2024-04-25T10:08:09.898985+0000","last_became_peered":"2024-04-25T10:08:09.898985+0000","last_unstale":"2024-04-25T10:08:13.396978+0000","last_undegraded":"2024-04-25T10:08:13.396978+0000","last_fullsized":"2024-04-25T10:08:13.396978+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:54.732376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396711+0000","last_change":"2024-04-25T10:08:10.918300+0000","last_active":"2024-04-25T10:08:13.396711+0000","last_peered":"2024-04-25T10:08:13.396711+0000","last_clean":"2024-04-25T10:08:13.396711+0000","last_became_active":"2024-04-25T10:08:10.911496+0000","last_became_peered":"2024-04-25T10:08:10.911496+0000","last_unstale":"2024-04-25T10:08:13.396711+0000","last_undegraded":"2024-04-25T10:08:13.396711+0000","last_fullsized":"2024-04-25T10:08:13.396711+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:40.184599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894808+0000","last_change":"2024-04-25T10:08:09.891850+0000","last_active":"2024-04-25T10:08:11.894808+0000","last_peered":"2024-04-25T10:08:11.894808+0000","last_clean":"2024-04-25T10:08:11.894808+0000","last_became_active":"2024-04-25T10:08:09.891235+0000","last_became_peered":"2024-04-25T10:08:09.891235+0000","last_unstale":"2024-04-25T10:08:11.894808+0000","last_undegraded":"2024-04-25T10:08:11.894808+0000","last_fullsized":"2024-04-25T10:08:11.894808+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:56.421526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894725+0000","last_change":"2024-04-25T10:08:10.917026+0000","last_active":"2024-04-25T10:08:11.894725+0000","last_peered":"2024-04-25T10:08:11.894725+0000","last_clean":"2024-04-25T10:08:11.894725+0000","last_became_active":"2024-04-25T10:08:10.916406+0000","last_became_peered":"2024-04-25T10:08:10.916406+0000","last_unstale":"2024-04-25T10:08:11.894725+0000","last_undegraded":"2024-04-25T10:08:11.894725+0000","last_fullsized":"2024-04-25T10:08:11.894725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.834110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901694+0000","last_change":"2024-04-25T10:08:09.881070+0000","last_active":"2024-04-25T10:08:10.901694+0000","last_peered":"2024-04-25T10:08:10.901694+0000","last_clean":"2024-04-25T10:08:10.901694+0000","last_became_active":"2024-04-25T10:08:09.880757+0000","last_became_peered":"2024-04-25T10:08:09.880757+0000","last_unstale":"2024-04-25T10:08:10.901694+0000","last_undegraded":"2024-04-25T10:08:10.901694+0000","last_fullsized":"2024-04-25T10:08:10.901694+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:26.624159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396665+0000","last_change":"2024-04-25T10:08:10.923429+0000","last_active":"2024-04-25T10:08:13.396665+0000","last_peered":"2024-04-25T10:08:13.396665+0000","last_clean":"2024-04-25T10:08:13.396665+0000","last_became_active":"2024-04-25T10:08:10.922036+0000","last_became_peered":"2024-04-25T10:08:10.922036+0000","last_unstale":"2024-04-25T10:08:13.396665+0000","last_undegraded":"2024-04-25T10:08:13.396665+0000","last_fullsized":"2024-04-25T10:08:13.396665+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:54.437016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967486+0000","last_change":"2024-04-25T10:08:09.899426+0000","last_active":"2024-04-25T10:08:11.967486+0000","last_peered":"2024-04-25T10:08:11.967486+0000","last_clean":"2024-04-25T10:08:11.967486+0000","last_became_active":"2024-04-25T10:08:09.898830+0000","last_became_peered":"2024-04-25T10:08:09.898830+0000","last_unstale":"2024-04-25T10:08:11.967486+0000","last_undegraded":"2024-04-25T10:08:11.967486+0000","last_fullsized":"2024-04-25T10:08:11.967486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:19.781296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.892879+0000","last_change":"2024-04-25T10:08:10.901543+0000","last_active":"2024-04-25T10:08:11.892879+0000","last_peered":"2024-04-25T10:08:11.892879+0000","last_clean":"2024-04-25T10:08:11.892879+0000","last_became_active":"2024-04-25T10:08:10.900826+0000","last_became_peered":"2024-04-25T10:08:10.900826+0000","last_unstale":"2024-04-25T10:08:11.892879+0000","last_undegraded":"2024-04-25T10:08:11.892879+0000","last_fullsized":"2024-04-25T10:08:11.892879+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:32.461497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893451+0000","last_change":"2024-04-25T10:08:09.902684+0000","last_active":"2024-04-25T10:08:11.893451+0000","last_peered":"2024-04-25T10:08:11.893451+0000","last_clean":"2024-04-25T10:08:11.893451+0000","last_became_active":"2024-04-25T10:08:09.901806+0000","last_became_peered":"2024-04-25T10:08:09.901806+0000","last_unstale":"2024-04-25T10:08:11.893451+0000","last_undegraded":"2024-04-25T10:08:11.893451+0000","last_fullsized":"2024-04-25T10:08:11.893451+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:17.199198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396317+0000","last_change":"2024-04-25T10:08:10.925089+0000","last_active":"2024-04-25T10:08:13.396317+0000","last_peered":"2024-04-25T10:08:13.396317+0000","last_clean":"2024-04-25T10:08:13.396317+0000","last_became_active":"2024-04-25T10:08:10.922906+0000","last_became_peered":"2024-04-25T10:08:10.922906+0000","last_unstale":"2024-04-25T10:08:13.396317+0000","last_undegraded":"2024-04-25T10:08:13.396317+0000","last_fullsized":"2024-04-25T10:08:13.396317+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:55.438825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":22,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.395853+0000","last_change":"2024-04-25T10:08:09.908904+0000","last_active":"2024-04-25T10:08:13.395853+0000","last_peered":"2024-04-25T10:08:13.395853+0000","last_clean":"2024-04-25T10:08:13.395853+0000","last_became_active":"2024-04-25T10:08:09.908076+0000","last_became_peered":"2024-04-25T10:08:09.908076+0000","last_unstale":"2024-04-25T10:08:13.395853+0000","last_undegraded":"2024-04-25T10:08:13.395853+0000","last_fullsized":"2024-04-25T10:08:13.395853+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:12.528702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.397170+0000","last_change":"2024-04-25T10:08:10.916890+0000","last_active":"2024-04-25T10:08:13.397170+0000","last_peered":"2024-04-25T10:08:13.397170+0000","last_clean":"2024-04-25T10:08:13.397170+0000","last_became_active":"2024-04-25T10:08:10.910435+0000","last_became_peered":"2024-04-25T10:08:10.910435+0000","last_unstale":"2024-04-25T10:08:13.397170+0000","last_undegraded":"2024-04-25T10:08:13.397170+0000","last_fullsized":"2024-04-25T10:08:13.397170+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:10.886429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.899177+0000","last_change":"2024-04-25T10:08:09.884420+0000","last_active":"2024-04-25T10:08:10.899177+0000","last_peered":"2024-04-25T10:08:10.899177+0000","last_clean":"2024-04-25T10:08:10.899177+0000","last_became_active":"2024-04-25T10:08:09.883427+0000","last_became_peered":"2024-04-25T10:08:09.883427+0000","last_unstale":"2024-04-25T10:08:10.899177+0000","last_undegraded":"2024-04-25T10:08:10.899177+0000","last_fullsized":"2024-04-25T10:08:10.899177+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:47.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894725+0000","last_change":"2024-04-25T10:08:10.918083+0000","last_active":"2024-04-25T10:08:11.894725+0000","last_peered":"2024-04-25T10:08:11.894725+0000","last_clean":"2024-04-25T10:08:11.894725+0000","last_became_active":"2024-04-25T10:08:10.917512+0000","last_became_peered":"2024-04-25T10:08:10.917512+0000","last_unstale":"2024-04-25T10:08:11.894725+0000","last_undegraded":"2024-04-25T10:08:11.894725+0000","last_fullsized":"2024-04-25T10:08:11.894725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:35.281267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.891599+0000","last_change":"2024-04-25T10:08:09.888804+0000","last_active":"2024-04-25T10:08:10.891599+0000","last_peered":"2024-04-25T10:08:10.891599+0000","last_clean":"2024-04-25T10:08:10.891599+0000","last_became_active":"2024-04-25T10:08:09.888488+0000","last_became_peered":"2024-04-25T10:08:09.888488+0000","last_unstale":"2024-04-25T10:08:10.891599+0000","last_undegraded":"2024-04-25T10:08:10.891599+0000","last_fullsized":"2024-04-25T10:08:10.891599+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:17.199198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.906640+0000","last_change":"2024-04-25T10:08:10.905832+0000","last_active":"2024-04-25T10:08:10.906640+0000","last_peered":"2024-04-25T10:08:10.906640+0000","last_clean":"2024-04-25T10:08:10.906640+0000","last_became_active":"2024-04-25T10:08:10.905524+0000","last_became_peered":"2024-04-25T10:08:10.905524+0000","last_unstale":"2024-04-25T10:08:10.906640+0000","last_undegraded":"2024-04-25T10:08:10.906640+0000","last_fullsized":"2024-04-25T10:08:10.906640+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:29.292616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893944+0000","last_change":"2024-04-25T10:08:09.902367+0000","last_active":"2024-04-25T10:08:11.893944+0000","last_peered":"2024-04-25T10:08:11.893944+0000","last_clean":"2024-04-25T10:08:11.893944+0000","last_became_active":"2024-04-25T10:08:09.900592+0000","last_became_peered":"2024-04-25T10:08:09.900592+0000","last_unstale":"2024-04-25T10:08:11.893944+0000","last_undegraded":"2024-04-25T10:08:11.893944+0000","last_fullsized":"2024-04-25T10:08:11.893944+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:38.492052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.397108+0000","last_change":"2024-04-25T10:08:10.928860+0000","last_active":"2024-04-25T10:08:13.397108+0000","last_peered":"2024-04-25T10:08:13.397108+0000","last_clean":"2024-04-25T10:08:13.397108+0000","last_became_active":"2024-04-25T10:08:10.921892+0000","last_became_peered":"2024-04-25T10:08:10.921892+0000","last_unstale":"2024-04-25T10:08:13.397108+0000","last_undegraded":"2024-04-25T10:08:13.397108+0000","last_fullsized":"2024-04-25T10:08:13.397108+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:31.914077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.878704+0000","last_change":"2024-04-25T10:08:09.890143+0000","last_active":"2024-04-25T10:08:10.878704+0000","last_peered":"2024-04-25T10:08:10.878704+0000","last_clean":"2024-04-25T10:08:10.878704+0000","last_became_active":"2024-04-25T10:08:09.889126+0000","last_became_peered":"2024-04-25T10:08:09.889126+0000","last_unstale":"2024-04-25T10:08:10.878704+0000","last_undegraded":"2024-04-25T10:08:10.878704+0000","last_fullsized":"2024-04-25T10:08:10.878704+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:38.492052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893893+0000","last_change":"2024-04-25T10:08:10.917214+0000","last_active":"2024-04-25T10:08:11.893893+0000","last_peered":"2024-04-25T10:08:11.893893+0000","last_clean":"2024-04-25T10:08:11.893893+0000","last_became_active":"2024-04-25T10:08:10.915770+0000","last_became_peered":"2024-04-25T10:08:10.915770+0000","last_unstale":"2024-04-25T10:08:11.893893+0000","last_undegraded":"2024-04-25T10:08:11.893893+0000","last_fullsized":"2024-04-25T10:08:11.893893+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:44.066536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.907624+0000","last_change":"2024-04-25T10:08:09.889071+0000","last_active":"2024-04-25T10:08:10.907624+0000","last_peered":"2024-04-25T10:08:10.907624+0000","last_clean":"2024-04-25T10:08:10.907624+0000","last_became_active":"2024-04-25T10:08:09.888746+0000","last_became_peered":"2024-04-25T10:08:09.888746+0000","last_unstale":"2024-04-25T10:08:10.907624+0000","last_undegraded":"2024-04-25T10:08:10.907624+0000","last_fullsized":"2024-04-25T10:08:10.907624+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:43.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.895295+0000","last_change":"2024-04-25T10:08:10.893599+0000","last_active":"2024-04-25T10:08:10.895295+0000","last_peered":"2024-04-25T10:08:10.895295+0000","last_clean":"2024-04-25T10:08:10.895295+0000","last_became_active":"2024-04-25T10:08:10.893206+0000","last_became_peered":"2024-04-25T10:08:10.893206+0000","last_unstale":"2024-04-25T10:08:10.895295+0000","last_undegraded":"2024-04-25T10:08:10.895295+0000","last_fullsized":"2024-04-25T10:08:10.895295+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:51.143246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889836+0000","last_change":"2024-04-25T10:08:09.880919+0000","last_active":"2024-04-25T10:08:11.889836+0000","last_peered":"2024-04-25T10:08:11.889836+0000","last_clean":"2024-04-25T10:08:11.889836+0000","last_became_active":"2024-04-25T10:08:09.880611+0000","last_became_peered":"2024-04-25T10:08:09.880611+0000","last_unstale":"2024-04-25T10:08:11.889836+0000","last_undegraded":"2024-04-25T10:08:11.889836+0000","last_fullsized":"2024-04-25T10:08:11.889836+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:22.155681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.902841+0000","last_change":"2024-04-25T10:08:10.901150+0000","last_active":"2024-04-25T10:08:10.902841+0000","last_peered":"2024-04-25T10:08:10.902841+0000","last_clean":"2024-04-25T10:08:10.902841+0000","last_became_active":"2024-04-25T10:08:10.900624+0000","last_became_peered":"2024-04-25T10:08:10.900624+0000","last_unstale":"2024-04-25T10:08:10.902841+0000","last_undegraded":"2024-04-25T10:08:10.902841+0000","last_fullsized":"2024-04-25T10:08:10.902841+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:08.745250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.892866+0000","last_change":"2024-04-25T10:08:09.901094+0000","last_active":"2024-04-25T10:08:11.892866+0000","last_peered":"2024-04-25T10:08:11.892866+0000","last_clean":"2024-04-25T10:08:11.892866+0000","last_became_active":"2024-04-25T10:08:09.900405+0000","last_became_peered":"2024-04-25T10:08:09.900405+0000","last_unstale":"2024-04-25T10:08:11.892866+0000","last_undegraded":"2024-04-25T10:08:11.892866+0000","last_fullsized":"2024-04-25T10:08:11.892866+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:30.789048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893987+0000","last_change":"2024-04-25T10:08:10.909724+0000","last_active":"2024-04-25T10:08:11.893987+0000","last_peered":"2024-04-25T10:08:11.893987+0000","last_clean":"2024-04-25T10:08:11.893987+0000","last_became_active":"2024-04-25T10:08:10.904979+0000","last_became_peered":"2024-04-25T10:08:10.904979+0000","last_unstale":"2024-04-25T10:08:11.893987+0000","last_undegraded":"2024-04-25T10:08:11.893987+0000","last_fullsized":"2024-04-25T10:08:11.893987+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:17.598272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889987+0000","last_change":"2024-04-25T10:08:09.885875+0000","last_active":"2024-04-25T10:08:11.889987+0000","last_peered":"2024-04-25T10:08:11.889987+0000","last_clean":"2024-04-25T10:08:11.889987+0000","last_became_active":"2024-04-25T10:08:09.885529+0000","last_became_peered":"2024-04-25T10:08:09.885529+0000","last_unstale":"2024-04-25T10:08:11.889987+0000","last_undegraded":"2024-04-25T10:08:11.889987+0000","last_fullsized":"2024-04-25T10:08:11.889987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:30.869245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.895037+0000","last_change":"2024-04-25T10:08:10.923948+0000","last_active":"2024-04-25T10:08:11.895037+0000","last_peered":"2024-04-25T10:08:11.895037+0000","last_clean":"2024-04-25T10:08:11.895037+0000","last_became_active":"2024-04-25T10:08:10.923383+0000","last_became_peered":"2024-04-25T10:08:10.923383+0000","last_unstale":"2024-04-25T10:08:11.895037+0000","last_undegraded":"2024-04-25T10:08:11.895037+0000","last_fullsized":"2024-04-25T10:08:11.895037+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:55.065275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.879776+0000","last_change":"2024-04-25T10:08:09.894894+0000","last_active":"2024-04-25T10:08:10.879776+0000","last_peered":"2024-04-25T10:08:10.879776+0000","last_clean":"2024-04-25T10:08:10.879776+0000","last_became_active":"2024-04-25T10:08:09.894350+0000","last_became_peered":"2024-04-25T10:08:09.894350+0000","last_unstale":"2024-04-25T10:08:10.879776+0000","last_undegraded":"2024-04-25T10:08:10.879776+0000","last_fullsized":"2024-04-25T10:08:10.879776+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.592539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889925+0000","last_change":"2024-04-25T10:08:10.893322+0000","last_active":"2024-04-25T10:08:11.889925+0000","last_peered":"2024-04-25T10:08:11.889925+0000","last_clean":"2024-04-25T10:08:11.889925+0000","last_became_active":"2024-04-25T10:08:10.892639+0000","last_became_peered":"2024-04-25T10:08:10.892639+0000","last_unstale":"2024-04-25T10:08:11.889925+0000","last_undegraded":"2024-04-25T10:08:11.889925+0000","last_fullsized":"2024-04-25T10:08:11.889925+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:37.445694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967957+0000","last_change":"2024-04-25T10:08:09.893305+0000","last_active":"2024-04-25T10:08:11.967957+0000","last_peered":"2024-04-25T10:08:11.967957+0000","last_clean":"2024-04-25T10:08:11.967957+0000","last_became_active":"2024-04-25T10:08:09.892501+0000","last_became_peered":"2024-04-25T10:08:09.892501+0000","last_unstale":"2024-04-25T10:08:11.967957+0000","last_undegraded":"2024-04-25T10:08:11.967957+0000","last_fullsized":"2024-04-25T10:08:11.967957+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:21.522866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.926945+0000","last_change":"2024-04-25T10:08:10.926260+0000","last_active":"2024-04-25T10:08:10.926945+0000","last_peered":"2024-04-25T10:08:10.926945+0000","last_clean":"2024-04-25T10:08:10.926945+0000","last_became_active":"2024-04-25T10:08:10.926038+0000","last_became_peered":"2024-04-25T10:08:10.926038+0000","last_unstale":"2024-04-25T10:08:10.926945+0000","last_undegraded":"2024-04-25T10:08:10.926945+0000","last_fullsized":"2024-04-25T10:08:10.926945+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:44.066536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894376+0000","last_change":"2024-04-25T10:08:09.906942+0000","last_active":"2024-04-25T10:08:11.894376+0000","last_peered":"2024-04-25T10:08:11.894376+0000","last_clean":"2024-04-25T10:08:11.894376+0000","last_became_active":"2024-04-25T10:08:09.906192+0000","last_became_peered":"2024-04-25T10:08:09.906192+0000","last_unstale":"2024-04-25T10:08:11.894376+0000","last_undegraded":"2024-04-25T10:08:11.894376+0000","last_fullsized":"2024-04-25T10:08:11.894376+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:48.349703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.899168+0000","last_change":"2024-04-25T10:08:10.890698+0000","last_active":"2024-04-25T10:08:10.899168+0000","last_peered":"2024-04-25T10:08:10.899168+0000","last_clean":"2024-04-25T10:08:10.899168+0000","last_became_active":"2024-04-25T10:08:10.889982+0000","last_became_peered":"2024-04-25T10:08:10.889982+0000","last_unstale":"2024-04-25T10:08:10.899168+0000","last_undegraded":"2024-04-25T10:08:10.899168+0000","last_fullsized":"2024-04-25T10:08:10.899168+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:55.065275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889624+0000","last_change":"2024-04-25T10:08:09.902559+0000","last_active":"2024-04-25T10:08:11.889624+0000","last_peered":"2024-04-25T10:08:11.889624+0000","last_clean":"2024-04-25T10:08:11.889624+0000","last_became_active":"2024-04-25T10:08:09.902240+0000","last_became_peered":"2024-04-25T10:08:09.902240+0000","last_unstale":"2024-04-25T10:08:11.889624+0000","last_undegraded":"2024-04-25T10:08:11.889624+0000","last_fullsized":"2024-04-25T10:08:11.889624+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:30.202367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396825+0000","last_change":"2024-04-25T10:08:10.924116+0000","last_active":"2024-04-25T10:08:13.396825+0000","last_peered":"2024-04-25T10:08:13.396825+0000","last_clean":"2024-04-25T10:08:13.396825+0000","last_became_active":"2024-04-25T10:08:10.922605+0000","last_became_peered":"2024-04-25T10:08:10.922605+0000","last_unstale":"2024-04-25T10:08:13.396825+0000","last_undegraded":"2024-04-25T10:08:13.396825+0000","last_fullsized":"2024-04-25T10:08:13.396825+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:07.785780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894016+0000","last_change":"2024-04-25T10:08:09.887885+0000","last_active":"2024-04-25T10:08:11.894016+0000","last_peered":"2024-04-25T10:08:11.894016+0000","last_clean":"2024-04-25T10:08:11.894016+0000","last_became_active":"2024-04-25T10:08:09.887165+0000","last_became_peered":"2024-04-25T10:08:09.887165+0000","last_unstale":"2024-04-25T10:08:11.894016+0000","last_undegraded":"2024-04-25T10:08:11.894016+0000","last_fullsized":"2024-04-25T10:08:11.894016+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:09.379274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889431+0000","last_change":"2024-04-25T10:08:10.894492+0000","last_active":"2024-04-25T10:08:11.889431+0000","last_peered":"2024-04-25T10:08:11.889431+0000","last_clean":"2024-04-25T10:08:11.889431+0000","last_became_active":"2024-04-25T10:08:10.894221+0000","last_became_peered":"2024-04-25T10:08:10.894221+0000","last_unstale":"2024-04-25T10:08:11.889431+0000","last_undegraded":"2024-04-25T10:08:11.889431+0000","last_fullsized":"2024-04-25T10:08:11.889431+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:47.897507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889533+0000","last_change":"2024-04-25T10:08:09.890601+0000","last_active":"2024-04-25T10:08:11.889533+0000","last_peered":"2024-04-25T10:08:11.889533+0000","last_clean":"2024-04-25T10:08:11.889533+0000","last_became_active":"2024-04-25T10:08:09.890208+0000","last_became_peered":"2024-04-25T10:08:09.890208+0000","last_unstale":"2024-04-25T10:08:11.889533+0000","last_undegraded":"2024-04-25T10:08:11.889533+0000","last_fullsized":"2024-04-25T10:08:11.889533+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:01.465554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901843+0000","last_change":"2024-04-25T10:08:10.898030+0000","last_active":"2024-04-25T10:08:10.901843+0000","last_peered":"2024-04-25T10:08:10.901843+0000","last_clean":"2024-04-25T10:08:10.901843+0000","last_became_active":"2024-04-25T10:08:10.897719+0000","last_became_peered":"2024-04-25T10:08:10.897719+0000","last_unstale":"2024-04-25T10:08:10.901843+0000","last_undegraded":"2024-04-25T10:08:10.901843+0000","last_fullsized":"2024-04-25T10:08:10.901843+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:18.216787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.879881+0000","last_change":"2024-04-25T10:08:09.900914+0000","last_active":"2024-04-25T10:08:10.879881+0000","last_peered":"2024-04-25T10:08:10.879881+0000","last_clean":"2024-04-25T10:08:10.879881+0000","last_became_active":"2024-04-25T10:08:09.900669+0000","last_became_peered":"2024-04-25T10:08:09.900669+0000","last_unstale":"2024-04-25T10:08:10.879881+0000","last_undegraded":"2024-04-25T10:08:10.879881+0000","last_fullsized":"2024-04-25T10:08:10.879881+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.736275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-25T10:08:10.901873+0000","last_change":"2024-04-25T10:08:10.894360+0000","last_active":"2024-04-25T10:08:10.901873+0000","last_peered":"2024-04-25T10:08:10.901873+0000","last_clean":"2024-04-25T10:08:10.901873+0000","last_became_active":"2024-04-25T10:08:10.894017+0000","last_became_peered":"2024-04-25T10:08:10.894017+0000","last_unstale":"2024-04-25T10:08:10.901873+0000","last_undegraded":"2024-04-25T10:08:10.901873+0000","last_fullsized":"2024-04-25T10:08:10.901873+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:43.207466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967562+0000","last_change":"2024-04-25T10:08:09.909488+0000","last_active":"2024-04-25T10:08:11.967562+0000","last_peered":"2024-04-25T10:08:11.967562+0000","last_clean":"2024-04-25T10:08:11.967562+0000","last_became_active":"2024-04-25T10:08:09.908863+0000","last_became_peered":"2024-04-25T10:08:09.908863+0000","last_unstale":"2024-04-25T10:08:11.967562+0000","last_undegraded":"2024-04-25T10:08:11.967562+0000","last_fullsized":"2024-04-25T10:08:11.967562+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:48.865163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":237,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.395879+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:08:13.395879+0000","last_peered":"2024-04-25T10:08:13.395879+0000","last_clean":"2024-04-25T10:08:13.395879+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:08:13.395879+0000","last_undegraded":"2024-04-25T10:08:13.395879+0000","last_fullsized":"2024-04-25T10:08:13.395879+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":870129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":700416,"data_stored":287513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542183,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":688128,"data_stored":285901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":286553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542184,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36012,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335828,"statfs":{"total":96636764160,"available":96599887872,"internally_reserved":0,"allocated":684032,"data_stored":279657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":684032,"data_stored":279657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36004,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335836,"statfs":{"total":96636764160,"available":96599896064,"internally_reserved":0,"allocated":679936,"data_stored":279691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36596,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335244,"statfs":{"total":96636764160,"available":96599289856,"internally_reserved":0,"allocated":1286144,"data_stored":872163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:16.543 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:16.545 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:16.854 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T10:08:16.846+0000 7f273a33f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:08:18.398 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:18.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f732e73c640 1 Processor -- start 2024-04-25T10:08:18.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f732e73c640 1 -- start start 2024-04-25T10:08:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f732e73c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f732815bc90 0x7f732815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f732e73c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73280b4660 0x7f732815c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f732e73c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7328071ca0 con 0x7f73280ce7a0 2024-04-25T10:08:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f732e73c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73280cc370 con 0x7f732815bc90 2024-04-25T10:08:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f732e73c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73280cca40 con 0x7f73280b4660 2024-04-25T10:08:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7327fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f73280ce7a0 0x7f73280ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41536/0 (socket says 172.21.15.60:41536) 2024-04-25T10:08:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7327fff640 1 -- 172.21.15.60:0/997920349 learned_addr learned my addr 172.21.15.60:0/997920349 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7326ffd640 1 -- 172.21.15.60:0/997920349 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316665131 0 0) 0x7f7328071ca0 con 0x7f73280ce7a0 2024-04-25T10:08:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f732ccd9640 1 --2- 172.21.15.60:0/997920349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73280b4660 0x7f732815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7326ffd640 1 -- 172.21.15.60:0/997920349 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7310003530 con 0x7f73280ce7a0 2024-04-25T10:08:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f73277fe640 1 --2- 172.21.15.60:0/997920349 >> v2:172.21.15.100:3300/0 conn(0x7f732815bc90 0x7f732815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7326ffd640 1 -- 172.21.15.60:0/997920349 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3597356471 0 0) 0x7f7310003530 con 0x7f73280ce7a0 2024-04-25T10:08:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7326ffd640 1 -- 172.21.15.60:0/997920349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73280b4660 msgr2=0x7f732815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7326ffd640 1 --2- 172.21.15.60:0/997920349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73280b4660 0x7f732815c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7326ffd640 1 -- 172.21.15.60:0/997920349 >> v2:172.21.15.100:3300/0 conn(0x7f732815bc90 msgr2=0x7f732815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7326ffd640 1 --2- 172.21.15.60:0/997920349 >> v2:172.21.15.100:3300/0 conn(0x7f732815bc90 0x7f732815c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7326ffd640 1 -- 172.21.15.60:0/997920349 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73280ccd20 con 0x7f73280ce7a0 2024-04-25T10:08:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7326ffd640 1 -- 172.21.15.60:0/997920349 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f7318001680 con 0x7f73280ce7a0 2024-04-25T10:08:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7326ffd640 1 -- 172.21.15.60:0/997920349 >> v1:172.21.15.60:6789/0 conn(0x7f73280ce7a0 legacy=0x7f73280ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7326ffd640 1 --2- 172.21.15.60:0/997920349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73100039c0 0x7f7310003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7326ffd640 1 --2- 172.21.15.60:0/997920349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7310004810 0x7f7310004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7326ffd640 1 --2- 172.21.15.60:0/997920349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7310005190 0x7f731000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7326ffd640 1 -- 172.21.15.60:0/997920349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73280cca40 con 0x7f73100039c0 2024-04-25T10:08:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7326ffd640 1 -- 172.21.15.60:0/997920349 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73100045e0 con 0x7f7310005190 2024-04-25T10:08:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7326ffd640 1 -- 172.21.15.60:0/997920349 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731000bf10 con 0x7f7310004810 2024-04-25T10:08:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7327fff640 1 --2- 172.21.15.60:0/997920349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73100039c0 0x7f7310003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f732ccd9640 1 --2- 172.21.15.60:0/997920349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7310005190 0x7f731000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f73277fe640 1 --2- 172.21.15.60:0/997920349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7310004810 0x7f7310004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7327fff640 1 -- 172.21.15.60:0/997920349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7310004810 msgr2=0x7f7310004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:18.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7327fff640 1 --2- 172.21.15.60:0/997920349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7310004810 0x7f7310004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7327fff640 1 -- 172.21.15.60:0/997920349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7310005190 msgr2=0x7f731000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:18.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7327fff640 1 --2- 172.21.15.60:0/997920349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7310005190 0x7f731000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f7327fff640 1 -- 172.21.15.60:0/997920349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7310003530 con 0x7f73100039c0 2024-04-25T10:08:18.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.526+0000 7f732ccd9640 1 --2- 172.21.15.60:0/997920349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7310005190 0x7f731000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.530+0000 7f7327fff640 1 --2- 172.21.15.60:0/997920349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73100039c0 0x7f7310003de0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f731800c190 tx=0x7f731801cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b538d2eef6600d2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.530+0000 7f7326ffd640 1 -- 172.21.15.60:0/997920349 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318007c90 con 0x7f73100039c0 2024-04-25T10:08:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.530+0000 7f7326ffd640 1 -- 172.21.15.60:0/997920349 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73180a9020 con 0x7f73100039c0 2024-04-25T10:08:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.530+0000 7f7326ffd640 1 -- 172.21.15.60:0/997920349 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318005b00 con 0x7f73100039c0 2024-04-25T10:08:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.530+0000 7f732e73c640 1 -- 172.21.15.60:0/997920349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73100039c0 msgr2=0x7f7310003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.530+0000 7f732e73c640 1 --2- 172.21.15.60:0/997920349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73100039c0 0x7f7310003de0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f731800c190 tx=0x7f731801cb60 comp rx=0 tx=0).stop 2024-04-25T10:08:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.530+0000 7f732ccd9640 1 -- 172.21.15.60:0/997920349 reap_dead start 2024-04-25T10:08:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.530+0000 7f732e73c640 1 -- 172.21.15.60:0/997920349 shutdown_connections 2024-04-25T10:08:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.530+0000 7f732e73c640 1 -- 172.21.15.60:0/997920349 >> 172.21.15.60:0/997920349 conn(0x7f73280b12d0 msgr2=0x7f73280b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.530+0000 7f732e73c640 1 -- 172.21.15.60:0/997920349 shutdown_connections 2024-04-25T10:08:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.530+0000 7f732e73c640 1 -- 172.21.15.60:0/997920349 wait complete. 2024-04-25T10:08:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f732e73c640 1 Processor -- start 2024-04-25T10:08:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f732e73c640 1 -- start start 2024-04-25T10:08:18.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f732e73c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73100039c0 0x7f7328163c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f732e73c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73281522e0 0x7f7328156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f732e73c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73281570f0 0x7f7328071270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f732e73c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73280b3db0 con 0x7f73281522e0 2024-04-25T10:08:18.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f732e73c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73280b4490 con 0x7f73100039c0 2024-04-25T10:08:18.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f732e73c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732815cf70 con 0x7f73281570f0 2024-04-25T10:08:18.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f73277fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73281522e0 0x7f7328156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f73277fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73281522e0 0x7f7328156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58830/0 (socket says 172.21.15.60:58830) 2024-04-25T10:08:18.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f73277fe640 1 -- 172.21.15.60:0/687010865 learned_addr learned my addr 172.21.15.60:0/687010865 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:18.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f7327fff640 1 --2- 172.21.15.60:0/687010865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73100039c0 0x7f7328163c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f732ccd9640 1 --2- 172.21.15.60:0/687010865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73281570f0 0x7f7328071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f73277fe640 1 -- 172.21.15.60:0/687010865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73281570f0 msgr2=0x7f7328071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f73277fe640 1 --2- 172.21.15.60:0/687010865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73281570f0 0x7f7328071270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f73277fe640 1 -- 172.21.15.60:0/687010865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73100039c0 msgr2=0x7f7328163c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f73277fe640 1 --2- 172.21.15.60:0/687010865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73100039c0 0x7f7328163c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f73277fe640 1 -- 172.21.15.60:0/687010865 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7328164140 con 0x7f73281522e0 2024-04-25T10:08:18.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f7327fff640 1 --2- 172.21.15.60:0/687010865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73100039c0 0x7f7328163c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:18.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f732ccd9640 1 --2- 172.21.15.60:0/687010865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73281570f0 0x7f7328071270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:18.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f73277fe640 1 --2- 172.21.15.60:0/687010865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73281522e0 0x7f7328156bb0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f731c0115e0 tx=0x7f731c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:18.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f7324ff9640 1 -- 172.21.15.60:0/687010865 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c00f8f0 con 0x7f73281522e0 2024-04-25T10:08:18.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f7324ff9640 1 -- 172.21.15.60:0/687010865 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c00fa90 con 0x7f73281522e0 2024-04-25T10:08:18.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f732e73c640 1 -- 172.21.15.60:0/687010865 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73281550e0 con 0x7f73281522e0 2024-04-25T10:08:18.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f7324ff9640 1 -- 172.21.15.60:0/687010865 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c00fe40 con 0x7f73281522e0 2024-04-25T10:08:18.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f732e73c640 1 -- 172.21.15.60:0/687010865 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73281556a0 con 0x7f73281522e0 2024-04-25T10:08:18.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f732e73c640 1 -- 172.21.15.60:0/687010865 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7328155dc0 con 0x7f73281522e0 2024-04-25T10:08:18.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.534+0000 7f7324ff9640 1 -- 172.21.15.60:0/687010865 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f731c015460 con 0x7f73281522e0 2024-04-25T10:08:18.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.538+0000 7f7324ff9640 1 --2- 172.21.15.60:0/687010865 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7314066f60 0x7f7314069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.538+0000 7f7324ff9640 1 -- 172.21.15.60:0/687010865 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f731c085890 con 0x7f73281522e0 2024-04-25T10:08:18.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.538+0000 7f7327fff640 1 --2- 172.21.15.60:0/687010865 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7314066f60 0x7f7314069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.538+0000 7f7327fff640 1 --2- 172.21.15.60:0/687010865 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7314066f60 0x7f7314069420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f73180aa000 tx=0x7f7318000ff0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:18.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.542+0000 7f7324ff9640 1 -- 172.21.15.60:0/687010865 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f731c019030 con 0x7f73281522e0 2024-04-25T10:08:18.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.698+0000 7f732e73c640 1 -- 172.21.15.60:0/687010865 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f732815d6d0 con 0x7f7314066f60 2024-04-25T10:08:18.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.718+0000 7f7324ff9640 1 -- 172.21.15.60:0/687010865 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251903 (secure 0 0 0) 0x7f732815d6d0 con 0x7f7314066f60 2024-04-25T10:08:18.727 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:18.729 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:08:18.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.722+0000 7f732e73c640 1 -- 172.21.15.60:0/687010865 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7314066f60 msgr2=0x7f7314069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.722+0000 7f732e73c640 1 --2- 172.21.15.60:0/687010865 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7314066f60 0x7f7314069420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f73180aa000 tx=0x7f7318000ff0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.722+0000 7f732e73c640 1 -- 172.21.15.60:0/687010865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73281522e0 msgr2=0x7f7328156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.722+0000 7f732e73c640 1 --2- 172.21.15.60:0/687010865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73281522e0 0x7f7328156bb0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f731c0115e0 tx=0x7f731c011aa0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.722+0000 7f732e73c640 1 -- 172.21.15.60:0/687010865 shutdown_connections 2024-04-25T10:08:18.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.722+0000 7f732e73c640 1 --2- 172.21.15.60:0/687010865 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7314066f60 0x7f7314069420 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.722+0000 7f732e73c640 1 --2- 172.21.15.60:0/687010865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73281570f0 0x7f7328071270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.722+0000 7f732e73c640 1 --2- 172.21.15.60:0/687010865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73281522e0 0x7f7328156bb0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.722+0000 7f732e73c640 1 --2- 172.21.15.60:0/687010865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73100039c0 0x7f7328163c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.722+0000 7f732e73c640 1 -- 172.21.15.60:0/687010865 >> 172.21.15.60:0/687010865 conn(0x7f73280b12d0 msgr2=0x7f732815b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:18.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.722+0000 7f732e73c640 1 -- 172.21.15.60:0/687010865 shutdown_connections 2024-04-25T10:08:18.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.722+0000 7f732e73c640 1 -- 172.21.15.60:0/687010865 wait complete. 2024-04-25T10:08:18.745 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-04-25T10:08:18.448940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260212,"kb_used_data":6600,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754714508,"statfs":{"total":773094113280,"available":772827656192,"internally_reserved":0,"allocated":6758400,"data_stored":3472764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002578"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.395900+0000","last_change":"2024-04-25T10:08:10.894975+0000","last_active":"2024-04-25T10:08:13.395900+0000","last_peered":"2024-04-25T10:08:13.395900+0000","last_clean":"2024-04-25T10:08:13.395900+0000","last_became_active":"2024-04-25T10:08:10.894724+0000","last_became_peered":"2024-04-25T10:08:10.894724+0000","last_unstale":"2024-04-25T10:08:13.395900+0000","last_undegraded":"2024-04-25T10:08:13.395900+0000","last_fullsized":"2024-04-25T10:08:13.395900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:15.589965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.394370+0000","last_change":"2024-04-25T10:08:10.893083+0000","last_active":"2024-04-25T10:08:13.394370+0000","last_peered":"2024-04-25T10:08:13.394370+0000","last_clean":"2024-04-25T10:08:13.394370+0000","last_became_active":"2024-04-25T10:08:10.892785+0000","last_became_peered":"2024-04-25T10:08:10.892785+0000","last_unstale":"2024-04-25T10:08:13.394370+0000","last_undegraded":"2024-04-25T10:08:13.394370+0000","last_fullsized":"2024-04-25T10:08:13.394370+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:24.707429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396629+0000","last_change":"2024-04-25T10:08:10.916953+0000","last_active":"2024-04-25T10:08:13.396629+0000","last_peered":"2024-04-25T10:08:13.396629+0000","last_clean":"2024-04-25T10:08:13.396629+0000","last_became_active":"2024-04-25T10:08:10.910705+0000","last_became_peered":"2024-04-25T10:08:10.910705+0000","last_unstale":"2024-04-25T10:08:13.396629+0000","last_undegraded":"2024-04-25T10:08:13.396629+0000","last_fullsized":"2024-04-25T10:08:13.396629+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:04.768312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893692+0000","last_change":"2024-04-25T10:08:10.901432+0000","last_active":"2024-04-25T10:08:11.893692+0000","last_peered":"2024-04-25T10:08:11.893692+0000","last_clean":"2024-04-25T10:08:11.893692+0000","last_became_active":"2024-04-25T10:08:10.900367+0000","last_became_peered":"2024-04-25T10:08:10.900367+0000","last_unstale":"2024-04-25T10:08:11.893692+0000","last_undegraded":"2024-04-25T10:08:11.893692+0000","last_fullsized":"2024-04-25T10:08:11.893692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:27.633134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.394623+0000","last_change":"2024-04-25T10:08:10.909538+0000","last_active":"2024-04-25T10:08:13.394623+0000","last_peered":"2024-04-25T10:08:13.394623+0000","last_clean":"2024-04-25T10:08:13.394623+0000","last_became_active":"2024-04-25T10:08:10.909284+0000","last_became_peered":"2024-04-25T10:08:10.909284+0000","last_unstale":"2024-04-25T10:08:13.394623+0000","last_undegraded":"2024-04-25T10:08:13.394623+0000","last_fullsized":"2024-04-25T10:08:13.394623+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:13.492971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:12.874252+0000","last_change":"2024-04-25T10:08:10.887190+0000","last_active":"2024-04-25T10:08:12.874252+0000","last_peered":"2024-04-25T10:08:12.874252+0000","last_clean":"2024-04-25T10:08:12.874252+0000","last_became_active":"2024-04-25T10:08:10.886910+0000","last_became_peered":"2024-04-25T10:08:10.886910+0000","last_unstale":"2024-04-25T10:08:12.874252+0000","last_undegraded":"2024-04-25T10:08:12.874252+0000","last_fullsized":"2024-04-25T10:08:12.874252+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.834110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893402+0000","last_change":"2024-04-25T10:08:10.917873+0000","last_active":"2024-04-25T10:08:11.893402+0000","last_peered":"2024-04-25T10:08:11.893402+0000","last_clean":"2024-04-25T10:08:11.893402+0000","last_became_active":"2024-04-25T10:08:10.916140+0000","last_became_peered":"2024-04-25T10:08:10.916140+0000","last_unstale":"2024-04-25T10:08:11.893402+0000","last_undegraded":"2024-04-25T10:08:11.893402+0000","last_fullsized":"2024-04-25T10:08:11.893402+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:12.105474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.395454+0000","last_change":"2024-04-25T10:08:10.905892+0000","last_active":"2024-04-25T10:08:13.395454+0000","last_peered":"2024-04-25T10:08:13.395454+0000","last_clean":"2024-04-25T10:08:13.395454+0000","last_became_active":"2024-04-25T10:08:10.905443+0000","last_became_peered":"2024-04-25T10:08:10.905443+0000","last_unstale":"2024-04-25T10:08:13.395454+0000","last_undegraded":"2024-04-25T10:08:13.395454+0000","last_fullsized":"2024-04-25T10:08:13.395454+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:31.308784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894567+0000","last_change":"2024-04-25T10:08:10.884528+0000","last_active":"2024-04-25T10:08:11.894567+0000","last_peered":"2024-04-25T10:08:11.894567+0000","last_clean":"2024-04-25T10:08:11.894567+0000","last_became_active":"2024-04-25T10:08:10.884183+0000","last_became_peered":"2024-04-25T10:08:10.884183+0000","last_unstale":"2024-04-25T10:08:11.894567+0000","last_undegraded":"2024-04-25T10:08:11.894567+0000","last_fullsized":"2024-04-25T10:08:11.894567+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:42.851719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889904+0000","last_change":"2024-04-25T10:08:10.894120+0000","last_active":"2024-04-25T10:08:11.889904+0000","last_peered":"2024-04-25T10:08:11.889904+0000","last_clean":"2024-04-25T10:08:11.889904+0000","last_became_active":"2024-04-25T10:08:10.893807+0000","last_became_peered":"2024-04-25T10:08:10.893807+0000","last_unstale":"2024-04-25T10:08:11.889904+0000","last_undegraded":"2024-04-25T10:08:11.889904+0000","last_fullsized":"2024-04-25T10:08:11.889904+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:10.388249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.890117+0000","last_change":"2024-04-25T10:08:10.893934+0000","last_active":"2024-04-25T10:08:11.890117+0000","last_peered":"2024-04-25T10:08:11.890117+0000","last_clean":"2024-04-25T10:08:11.890117+0000","last_became_active":"2024-04-25T10:08:10.893270+0000","last_became_peered":"2024-04-25T10:08:10.893270+0000","last_unstale":"2024-04-25T10:08:11.890117+0000","last_undegraded":"2024-04-25T10:08:11.890117+0000","last_fullsized":"2024-04-25T10:08:11.890117+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:13.537677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894126+0000","last_change":"2024-04-25T10:08:10.923912+0000","last_active":"2024-04-25T10:08:11.894126+0000","last_peered":"2024-04-25T10:08:11.894126+0000","last_clean":"2024-04-25T10:08:11.894126+0000","last_became_active":"2024-04-25T10:08:10.923319+0000","last_became_peered":"2024-04-25T10:08:10.923319+0000","last_unstale":"2024-04-25T10:08:11.894126+0000","last_undegraded":"2024-04-25T10:08:11.894126+0000","last_fullsized":"2024-04-25T10:08:11.894126+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:44.473365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.968119+0000","last_change":"2024-04-25T10:08:10.886846+0000","last_active":"2024-04-25T10:08:11.968119+0000","last_peered":"2024-04-25T10:08:11.968119+0000","last_clean":"2024-04-25T10:08:11.968119+0000","last_became_active":"2024-04-25T10:08:10.886497+0000","last_became_peered":"2024-04-25T10:08:10.886497+0000","last_unstale":"2024-04-25T10:08:11.968119+0000","last_undegraded":"2024-04-25T10:08:11.968119+0000","last_fullsized":"2024-04-25T10:08:11.968119+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:56.861799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893740+0000","last_change":"2024-04-25T10:08:10.907514+0000","last_active":"2024-04-25T10:08:11.893740+0000","last_peered":"2024-04-25T10:08:11.893740+0000","last_clean":"2024-04-25T10:08:11.893740+0000","last_became_active":"2024-04-25T10:08:10.905752+0000","last_became_peered":"2024-04-25T10:08:10.905752+0000","last_unstale":"2024-04-25T10:08:11.893740+0000","last_undegraded":"2024-04-25T10:08:11.893740+0000","last_fullsized":"2024-04-25T10:08:11.893740+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:15.349210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893812+0000","last_change":"2024-04-25T10:08:10.882723+0000","last_active":"2024-04-25T10:08:11.893812+0000","last_peered":"2024-04-25T10:08:11.893812+0000","last_clean":"2024-04-25T10:08:11.893812+0000","last_became_active":"2024-04-25T10:08:10.882339+0000","last_became_peered":"2024-04-25T10:08:10.882339+0000","last_unstale":"2024-04-25T10:08:11.893812+0000","last_undegraded":"2024-04-25T10:08:11.893812+0000","last_fullsized":"2024-04-25T10:08:11.893812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:13.179674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:12.874193+0000","last_change":"2024-04-25T10:08:10.896865+0000","last_active":"2024-04-25T10:08:12.874193+0000","last_peered":"2024-04-25T10:08:12.874193+0000","last_clean":"2024-04-25T10:08:12.874193+0000","last_became_active":"2024-04-25T10:08:10.896575+0000","last_became_peered":"2024-04-25T10:08:10.896575+0000","last_unstale":"2024-04-25T10:08:12.874193+0000","last_undegraded":"2024-04-25T10:08:12.874193+0000","last_fullsized":"2024-04-25T10:08:12.874193+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:48.269167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396130+0000","last_change":"2024-04-25T10:08:10.898151+0000","last_active":"2024-04-25T10:08:13.396130+0000","last_peered":"2024-04-25T10:08:13.396130+0000","last_clean":"2024-04-25T10:08:13.396130+0000","last_became_active":"2024-04-25T10:08:10.897849+0000","last_became_peered":"2024-04-25T10:08:10.897849+0000","last_unstale":"2024-04-25T10:08:13.396130+0000","last_undegraded":"2024-04-25T10:08:13.396130+0000","last_fullsized":"2024-04-25T10:08:13.396130+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:33.472720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396698+0000","last_change":"2024-04-25T10:08:10.898156+0000","last_active":"2024-04-25T10:08:13.396698+0000","last_peered":"2024-04-25T10:08:13.396698+0000","last_clean":"2024-04-25T10:08:13.396698+0000","last_became_active":"2024-04-25T10:08:10.897326+0000","last_became_peered":"2024-04-25T10:08:10.897326+0000","last_unstale":"2024-04-25T10:08:13.396698+0000","last_undegraded":"2024-04-25T10:08:13.396698+0000","last_fullsized":"2024-04-25T10:08:13.396698+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:03.978144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.395823+0000","last_change":"2024-04-25T10:08:10.891553+0000","last_active":"2024-04-25T10:08:13.395823+0000","last_peered":"2024-04-25T10:08:13.395823+0000","last_clean":"2024-04-25T10:08:13.395823+0000","last_became_active":"2024-04-25T10:08:10.891348+0000","last_became_peered":"2024-04-25T10:08:10.891348+0000","last_unstale":"2024-04-25T10:08:13.395823+0000","last_undegraded":"2024-04-25T10:08:13.395823+0000","last_fullsized":"2024-04-25T10:08:13.395823+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:39.167438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.394701+0000","last_change":"2024-04-25T10:08:10.890188+0000","last_active":"2024-04-25T10:08:13.394701+0000","last_peered":"2024-04-25T10:08:13.394701+0000","last_clean":"2024-04-25T10:08:13.394701+0000","last_became_active":"2024-04-25T10:08:10.889827+0000","last_became_peered":"2024-04-25T10:08:10.889827+0000","last_unstale":"2024-04-25T10:08:13.394701+0000","last_undegraded":"2024-04-25T10:08:13.394701+0000","last_fullsized":"2024-04-25T10:08:13.394701+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:44.458213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396312+0000","last_change":"2024-04-25T10:08:10.930150+0000","last_active":"2024-04-25T10:08:13.396312+0000","last_peered":"2024-04-25T10:08:13.396312+0000","last_clean":"2024-04-25T10:08:13.396312+0000","last_became_active":"2024-04-25T10:08:10.929406+0000","last_became_peered":"2024-04-25T10:08:10.929406+0000","last_unstale":"2024-04-25T10:08:13.396312+0000","last_undegraded":"2024-04-25T10:08:13.396312+0000","last_fullsized":"2024-04-25T10:08:13.396312+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:43.406629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.394544+0000","last_change":"2024-04-25T10:08:10.892469+0000","last_active":"2024-04-25T10:08:13.394544+0000","last_peered":"2024-04-25T10:08:13.394544+0000","last_clean":"2024-04-25T10:08:13.394544+0000","last_became_active":"2024-04-25T10:08:10.890763+0000","last_became_peered":"2024-04-25T10:08:10.890763+0000","last_unstale":"2024-04-25T10:08:13.394544+0000","last_undegraded":"2024-04-25T10:08:13.394544+0000","last_fullsized":"2024-04-25T10:08:13.394544+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:16.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396550+0000","last_change":"2024-04-25T10:08:10.923545+0000","last_active":"2024-04-25T10:08:13.396550+0000","last_peered":"2024-04-25T10:08:13.396550+0000","last_clean":"2024-04-25T10:08:13.396550+0000","last_became_active":"2024-04-25T10:08:10.919185+0000","last_became_peered":"2024-04-25T10:08:10.919185+0000","last_unstale":"2024-04-25T10:08:13.396550+0000","last_undegraded":"2024-04-25T10:08:13.396550+0000","last_fullsized":"2024-04-25T10:08:13.396550+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:48.684084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967689+0000","last_change":"2024-04-25T10:08:10.897718+0000","last_active":"2024-04-25T10:08:11.967689+0000","last_peered":"2024-04-25T10:08:11.967689+0000","last_clean":"2024-04-25T10:08:11.967689+0000","last_became_active":"2024-04-25T10:08:10.897021+0000","last_became_peered":"2024-04-25T10:08:10.897021+0000","last_unstale":"2024-04-25T10:08:11.967689+0000","last_undegraded":"2024-04-25T10:08:11.967689+0000","last_fullsized":"2024-04-25T10:08:11.967689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:10.256509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396347+0000","last_change":"2024-04-25T10:08:10.885585+0000","last_active":"2024-04-25T10:08:13.396347+0000","last_peered":"2024-04-25T10:08:13.396347+0000","last_clean":"2024-04-25T10:08:13.396347+0000","last_became_active":"2024-04-25T10:08:10.885318+0000","last_became_peered":"2024-04-25T10:08:10.885318+0000","last_unstale":"2024-04-25T10:08:13.396347+0000","last_undegraded":"2024-04-25T10:08:13.396347+0000","last_fullsized":"2024-04-25T10:08:13.396347+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:13.181896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893986+0000","last_change":"2024-04-25T10:08:10.911391+0000","last_active":"2024-04-25T10:08:11.893986+0000","last_peered":"2024-04-25T10:08:11.893986+0000","last_clean":"2024-04-25T10:08:11.893986+0000","last_became_active":"2024-04-25T10:08:10.910461+0000","last_became_peered":"2024-04-25T10:08:10.910461+0000","last_unstale":"2024-04-25T10:08:11.893986+0000","last_undegraded":"2024-04-25T10:08:11.893986+0000","last_fullsized":"2024-04-25T10:08:11.893986+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:48.269167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:12.874492+0000","last_change":"2024-04-25T10:08:10.894475+0000","last_active":"2024-04-25T10:08:12.874492+0000","last_peered":"2024-04-25T10:08:12.874492+0000","last_clean":"2024-04-25T10:08:12.874492+0000","last_became_active":"2024-04-25T10:08:10.894176+0000","last_became_peered":"2024-04-25T10:08:10.894176+0000","last_unstale":"2024-04-25T10:08:12.874492+0000","last_undegraded":"2024-04-25T10:08:12.874492+0000","last_fullsized":"2024-04-25T10:08:12.874492+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:55.065275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396082+0000","last_change":"2024-04-25T10:08:10.901642+0000","last_active":"2024-04-25T10:08:13.396082+0000","last_peered":"2024-04-25T10:08:13.396082+0000","last_clean":"2024-04-25T10:08:13.396082+0000","last_became_active":"2024-04-25T10:08:10.901389+0000","last_became_peered":"2024-04-25T10:08:10.901389+0000","last_unstale":"2024-04-25T10:08:13.396082+0000","last_undegraded":"2024-04-25T10:08:13.396082+0000","last_fullsized":"2024-04-25T10:08:13.396082+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:28.186723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.394860+0000","last_change":"2024-04-25T10:08:10.915357+0000","last_active":"2024-04-25T10:08:13.394860+0000","last_peered":"2024-04-25T10:08:13.394860+0000","last_clean":"2024-04-25T10:08:13.394860+0000","last_became_active":"2024-04-25T10:08:10.915052+0000","last_became_peered":"2024-04-25T10:08:10.915052+0000","last_unstale":"2024-04-25T10:08:13.394860+0000","last_undegraded":"2024-04-25T10:08:13.394860+0000","last_fullsized":"2024-04-25T10:08:13.394860+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:31.995068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396716+0000","last_change":"2024-04-25T10:08:10.922837+0000","last_active":"2024-04-25T10:08:13.396716+0000","last_peered":"2024-04-25T10:08:13.396716+0000","last_clean":"2024-04-25T10:08:13.396716+0000","last_became_active":"2024-04-25T10:08:10.921976+0000","last_became_peered":"2024-04-25T10:08:10.921976+0000","last_unstale":"2024-04-25T10:08:13.396716+0000","last_undegraded":"2024-04-25T10:08:13.396716+0000","last_fullsized":"2024-04-25T10:08:13.396716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:09.814025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.395452+0000","last_change":"2024-04-25T10:08:10.884959+0000","last_active":"2024-04-25T10:08:13.395452+0000","last_peered":"2024-04-25T10:08:13.395452+0000","last_clean":"2024-04-25T10:08:13.395452+0000","last_became_active":"2024-04-25T10:08:10.884659+0000","last_became_peered":"2024-04-25T10:08:10.884659+0000","last_unstale":"2024-04-25T10:08:13.395452+0000","last_undegraded":"2024-04-25T10:08:13.395452+0000","last_fullsized":"2024-04-25T10:08:13.395452+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:17.632147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:12.874446+0000","last_change":"2024-04-25T10:08:10.895828+0000","last_active":"2024-04-25T10:08:12.874446+0000","last_peered":"2024-04-25T10:08:12.874446+0000","last_clean":"2024-04-25T10:08:12.874446+0000","last_became_active":"2024-04-25T10:08:10.895589+0000","last_became_peered":"2024-04-25T10:08:10.895589+0000","last_unstale":"2024-04-25T10:08:12.874446+0000","last_undegraded":"2024-04-25T10:08:12.874446+0000","last_fullsized":"2024-04-25T10:08:12.874446+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:35.281267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:12.875203+0000","last_change":"2024-04-25T10:08:10.911873+0000","last_active":"2024-04-25T10:08:12.875203+0000","last_peered":"2024-04-25T10:08:12.875203+0000","last_clean":"2024-04-25T10:08:12.875203+0000","last_became_active":"2024-04-25T10:08:10.911603+0000","last_became_peered":"2024-04-25T10:08:10.911603+0000","last_unstale":"2024-04-25T10:08:12.875203+0000","last_undegraded":"2024-04-25T10:08:12.875203+0000","last_fullsized":"2024-04-25T10:08:12.875203+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:43.207466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:12.874971+0000","last_change":"2024-04-25T10:08:09.906122+0000","last_active":"2024-04-25T10:08:12.874971+0000","last_peered":"2024-04-25T10:08:12.874971+0000","last_clean":"2024-04-25T10:08:12.874971+0000","last_became_active":"2024-04-25T10:08:09.905879+0000","last_became_peered":"2024-04-25T10:08:09.905879+0000","last_unstale":"2024-04-25T10:08:12.874971+0000","last_undegraded":"2024-04-25T10:08:12.874971+0000","last_fullsized":"2024-04-25T10:08:12.874971+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:48.349703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.890056+0000","last_change":"2024-04-25T10:08:10.911299+0000","last_active":"2024-04-25T10:08:11.890056+0000","last_peered":"2024-04-25T10:08:11.890056+0000","last_clean":"2024-04-25T10:08:11.890056+0000","last_became_active":"2024-04-25T10:08:10.910974+0000","last_became_peered":"2024-04-25T10:08:10.910974+0000","last_unstale":"2024-04-25T10:08:11.890056+0000","last_undegraded":"2024-04-25T10:08:11.890056+0000","last_fullsized":"2024-04-25T10:08:11.890056+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:22.511670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396248+0000","last_change":"2024-04-25T10:08:09.898224+0000","last_active":"2024-04-25T10:08:13.396248+0000","last_peered":"2024-04-25T10:08:13.396248+0000","last_clean":"2024-04-25T10:08:13.396248+0000","last_became_active":"2024-04-25T10:08:09.897935+0000","last_became_peered":"2024-04-25T10:08:09.897935+0000","last_unstale":"2024-04-25T10:08:13.396248+0000","last_undegraded":"2024-04-25T10:08:13.396248+0000","last_fullsized":"2024-04-25T10:08:13.396248+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:16.589298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:12.875159+0000","last_change":"2024-04-25T10:08:10.882470+0000","last_active":"2024-04-25T10:08:12.875159+0000","last_peered":"2024-04-25T10:08:12.875159+0000","last_clean":"2024-04-25T10:08:12.875159+0000","last_became_active":"2024-04-25T10:08:10.882185+0000","last_became_peered":"2024-04-25T10:08:10.882185+0000","last_unstale":"2024-04-25T10:08:12.875159+0000","last_undegraded":"2024-04-25T10:08:12.875159+0000","last_fullsized":"2024-04-25T10:08:12.875159+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:31.332615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:12.875287+0000","last_change":"2024-04-25T10:08:09.888457+0000","last_active":"2024-04-25T10:08:12.875287+0000","last_peered":"2024-04-25T10:08:12.875287+0000","last_clean":"2024-04-25T10:08:12.875287+0000","last_became_active":"2024-04-25T10:08:09.888103+0000","last_became_peered":"2024-04-25T10:08:09.888103+0000","last_unstale":"2024-04-25T10:08:12.875287+0000","last_undegraded":"2024-04-25T10:08:12.875287+0000","last_fullsized":"2024-04-25T10:08:12.875287+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.255517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:12.875010+0000","last_change":"2024-04-25T10:08:10.926635+0000","last_active":"2024-04-25T10:08:12.875010+0000","last_peered":"2024-04-25T10:08:12.875010+0000","last_clean":"2024-04-25T10:08:12.875010+0000","last_became_active":"2024-04-25T10:08:10.926349+0000","last_became_peered":"2024-04-25T10:08:10.926349+0000","last_unstale":"2024-04-25T10:08:12.875010+0000","last_undegraded":"2024-04-25T10:08:12.875010+0000","last_fullsized":"2024-04-25T10:08:12.875010+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:21.713220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894215+0000","last_change":"2024-04-25T10:08:09.907608+0000","last_active":"2024-04-25T10:08:11.894215+0000","last_peered":"2024-04-25T10:08:11.894215+0000","last_clean":"2024-04-25T10:08:11.894215+0000","last_became_active":"2024-04-25T10:08:09.906963+0000","last_became_peered":"2024-04-25T10:08:09.906963+0000","last_unstale":"2024-04-25T10:08:11.894215+0000","last_undegraded":"2024-04-25T10:08:11.894215+0000","last_fullsized":"2024-04-25T10:08:11.894215+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:04.255517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.394787+0000","last_change":"2024-04-25T10:08:10.892319+0000","last_active":"2024-04-25T10:08:13.394787+0000","last_peered":"2024-04-25T10:08:13.394787+0000","last_clean":"2024-04-25T10:08:13.394787+0000","last_became_active":"2024-04-25T10:08:10.890374+0000","last_became_peered":"2024-04-25T10:08:10.890374+0000","last_unstale":"2024-04-25T10:08:13.394787+0000","last_undegraded":"2024-04-25T10:08:13.394787+0000","last_fullsized":"2024-04-25T10:08:13.394787+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:39.733174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893927+0000","last_change":"2024-04-25T10:08:09.886156+0000","last_active":"2024-04-25T10:08:11.893927+0000","last_peered":"2024-04-25T10:08:11.893927+0000","last_clean":"2024-04-25T10:08:11.893927+0000","last_became_active":"2024-04-25T10:08:09.885288+0000","last_became_peered":"2024-04-25T10:08:09.885288+0000","last_unstale":"2024-04-25T10:08:11.893927+0000","last_undegraded":"2024-04-25T10:08:11.893927+0000","last_fullsized":"2024-04-25T10:08:11.893927+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:39.542515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.394694+0000","last_change":"2024-04-25T10:08:10.890541+0000","last_active":"2024-04-25T10:08:13.394694+0000","last_peered":"2024-04-25T10:08:13.394694+0000","last_clean":"2024-04-25T10:08:13.394694+0000","last_became_active":"2024-04-25T10:08:10.890280+0000","last_became_peered":"2024-04-25T10:08:10.890280+0000","last_unstale":"2024-04-25T10:08:13.394694+0000","last_undegraded":"2024-04-25T10:08:13.394694+0000","last_fullsized":"2024-04-25T10:08:13.394694+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:50.701501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:12.874244+0000","last_change":"2024-04-25T10:08:09.894473+0000","last_active":"2024-04-25T10:08:12.874244+0000","last_peered":"2024-04-25T10:08:12.874244+0000","last_clean":"2024-04-25T10:08:12.874244+0000","last_became_active":"2024-04-25T10:08:09.894181+0000","last_became_peered":"2024-04-25T10:08:09.894181+0000","last_unstale":"2024-04-25T10:08:12.874244+0000","last_undegraded":"2024-04-25T10:08:12.874244+0000","last_fullsized":"2024-04-25T10:08:12.874244+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:10.625997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967711+0000","last_change":"2024-04-25T10:08:10.923620+0000","last_active":"2024-04-25T10:08:11.967711+0000","last_peered":"2024-04-25T10:08:11.967711+0000","last_clean":"2024-04-25T10:08:11.967711+0000","last_became_active":"2024-04-25T10:08:10.923032+0000","last_became_peered":"2024-04-25T10:08:10.923032+0000","last_unstale":"2024-04-25T10:08:11.967711+0000","last_undegraded":"2024-04-25T10:08:11.967711+0000","last_fullsized":"2024-04-25T10:08:11.967711+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:10.085497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893167+0000","last_change":"2024-04-25T10:08:09.907088+0000","last_active":"2024-04-25T10:08:11.893167+0000","last_peered":"2024-04-25T10:08:11.893167+0000","last_clean":"2024-04-25T10:08:11.893167+0000","last_became_active":"2024-04-25T10:08:09.906310+0000","last_became_peered":"2024-04-25T10:08:09.906310+0000","last_unstale":"2024-04-25T10:08:11.893167+0000","last_undegraded":"2024-04-25T10:08:11.893167+0000","last_fullsized":"2024-04-25T10:08:11.893167+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.592539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.395275+0000","last_change":"2024-04-25T10:08:10.892966+0000","last_active":"2024-04-25T10:08:13.395275+0000","last_peered":"2024-04-25T10:08:13.395275+0000","last_clean":"2024-04-25T10:08:13.395275+0000","last_became_active":"2024-04-25T10:08:10.892667+0000","last_became_peered":"2024-04-25T10:08:10.892667+0000","last_unstale":"2024-04-25T10:08:13.395275+0000","last_undegraded":"2024-04-25T10:08:13.395275+0000","last_fullsized":"2024-04-25T10:08:13.395275+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:53.313093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893151+0000","last_change":"2024-04-25T10:08:09.889905+0000","last_active":"2024-04-25T10:08:11.893151+0000","last_peered":"2024-04-25T10:08:11.893151+0000","last_clean":"2024-04-25T10:08:11.893151+0000","last_became_active":"2024-04-25T10:08:09.889171+0000","last_became_peered":"2024-04-25T10:08:09.889171+0000","last_unstale":"2024-04-25T10:08:11.893151+0000","last_undegraded":"2024-04-25T10:08:11.893151+0000","last_fullsized":"2024-04-25T10:08:11.893151+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:46.888532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894707+0000","last_change":"2024-04-25T10:08:10.886122+0000","last_active":"2024-04-25T10:08:11.894707+0000","last_peered":"2024-04-25T10:08:11.894707+0000","last_clean":"2024-04-25T10:08:11.894707+0000","last_became_active":"2024-04-25T10:08:10.882247+0000","last_became_peered":"2024-04-25T10:08:10.882247+0000","last_unstale":"2024-04-25T10:08:11.894707+0000","last_undegraded":"2024-04-25T10:08:11.894707+0000","last_fullsized":"2024-04-25T10:08:11.894707+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:58.336224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.397277+0000","last_change":"2024-04-25T10:08:09.899045+0000","last_active":"2024-04-25T10:08:13.397277+0000","last_peered":"2024-04-25T10:08:13.397277+0000","last_clean":"2024-04-25T10:08:13.397277+0000","last_became_active":"2024-04-25T10:08:09.898282+0000","last_became_peered":"2024-04-25T10:08:09.898282+0000","last_unstale":"2024-04-25T10:08:13.397277+0000","last_undegraded":"2024-04-25T10:08:13.397277+0000","last_fullsized":"2024-04-25T10:08:13.397277+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:34.532573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.397322+0000","last_change":"2024-04-25T10:08:10.919031+0000","last_active":"2024-04-25T10:08:13.397322+0000","last_peered":"2024-04-25T10:08:13.397322+0000","last_clean":"2024-04-25T10:08:13.397322+0000","last_became_active":"2024-04-25T10:08:10.911880+0000","last_became_peered":"2024-04-25T10:08:10.911880+0000","last_unstale":"2024-04-25T10:08:13.397322+0000","last_undegraded":"2024-04-25T10:08:13.397322+0000","last_fullsized":"2024-04-25T10:08:13.397322+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:45.833608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396347+0000","last_change":"2024-04-25T10:08:09.880232+0000","last_active":"2024-04-25T10:08:13.396347+0000","last_peered":"2024-04-25T10:08:13.396347+0000","last_clean":"2024-04-25T10:08:13.396347+0000","last_became_active":"2024-04-25T10:08:09.879917+0000","last_became_peered":"2024-04-25T10:08:09.879917+0000","last_unstale":"2024-04-25T10:08:13.396347+0000","last_undegraded":"2024-04-25T10:08:13.396347+0000","last_fullsized":"2024-04-25T10:08:13.396347+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:43.464391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889651+0000","last_change":"2024-04-25T10:08:10.893598+0000","last_active":"2024-04-25T10:08:11.889651+0000","last_peered":"2024-04-25T10:08:11.889651+0000","last_clean":"2024-04-25T10:08:11.889651+0000","last_became_active":"2024-04-25T10:08:10.893008+0000","last_became_peered":"2024-04-25T10:08:10.893008+0000","last_unstale":"2024-04-25T10:08:11.889651+0000","last_undegraded":"2024-04-25T10:08:11.889651+0000","last_fullsized":"2024-04-25T10:08:11.889651+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:57.430500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396978+0000","last_change":"2024-04-25T10:08:09.899600+0000","last_active":"2024-04-25T10:08:13.396978+0000","last_peered":"2024-04-25T10:08:13.396978+0000","last_clean":"2024-04-25T10:08:13.396978+0000","last_became_active":"2024-04-25T10:08:09.898985+0000","last_became_peered":"2024-04-25T10:08:09.898985+0000","last_unstale":"2024-04-25T10:08:13.396978+0000","last_undegraded":"2024-04-25T10:08:13.396978+0000","last_fullsized":"2024-04-25T10:08:13.396978+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:54.732376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396711+0000","last_change":"2024-04-25T10:08:10.918300+0000","last_active":"2024-04-25T10:08:13.396711+0000","last_peered":"2024-04-25T10:08:13.396711+0000","last_clean":"2024-04-25T10:08:13.396711+0000","last_became_active":"2024-04-25T10:08:10.911496+0000","last_became_peered":"2024-04-25T10:08:10.911496+0000","last_unstale":"2024-04-25T10:08:13.396711+0000","last_undegraded":"2024-04-25T10:08:13.396711+0000","last_fullsized":"2024-04-25T10:08:13.396711+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:40.184599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894808+0000","last_change":"2024-04-25T10:08:09.891850+0000","last_active":"2024-04-25T10:08:11.894808+0000","last_peered":"2024-04-25T10:08:11.894808+0000","last_clean":"2024-04-25T10:08:11.894808+0000","last_became_active":"2024-04-25T10:08:09.891235+0000","last_became_peered":"2024-04-25T10:08:09.891235+0000","last_unstale":"2024-04-25T10:08:11.894808+0000","last_undegraded":"2024-04-25T10:08:11.894808+0000","last_fullsized":"2024-04-25T10:08:11.894808+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:56.421526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894725+0000","last_change":"2024-04-25T10:08:10.917026+0000","last_active":"2024-04-25T10:08:11.894725+0000","last_peered":"2024-04-25T10:08:11.894725+0000","last_clean":"2024-04-25T10:08:11.894725+0000","last_became_active":"2024-04-25T10:08:10.916406+0000","last_became_peered":"2024-04-25T10:08:10.916406+0000","last_unstale":"2024-04-25T10:08:11.894725+0000","last_undegraded":"2024-04-25T10:08:11.894725+0000","last_fullsized":"2024-04-25T10:08:11.894725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:17.834110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.395190+0000","last_change":"2024-04-25T10:08:09.881070+0000","last_active":"2024-04-25T10:08:13.395190+0000","last_peered":"2024-04-25T10:08:13.395190+0000","last_clean":"2024-04-25T10:08:13.395190+0000","last_became_active":"2024-04-25T10:08:09.880757+0000","last_became_peered":"2024-04-25T10:08:09.880757+0000","last_unstale":"2024-04-25T10:08:13.395190+0000","last_undegraded":"2024-04-25T10:08:13.395190+0000","last_fullsized":"2024-04-25T10:08:13.395190+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:26.624159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396665+0000","last_change":"2024-04-25T10:08:10.923429+0000","last_active":"2024-04-25T10:08:13.396665+0000","last_peered":"2024-04-25T10:08:13.396665+0000","last_clean":"2024-04-25T10:08:13.396665+0000","last_became_active":"2024-04-25T10:08:10.922036+0000","last_became_peered":"2024-04-25T10:08:10.922036+0000","last_unstale":"2024-04-25T10:08:13.396665+0000","last_undegraded":"2024-04-25T10:08:13.396665+0000","last_fullsized":"2024-04-25T10:08:13.396665+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:54.437016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967486+0000","last_change":"2024-04-25T10:08:09.899426+0000","last_active":"2024-04-25T10:08:11.967486+0000","last_peered":"2024-04-25T10:08:11.967486+0000","last_clean":"2024-04-25T10:08:11.967486+0000","last_became_active":"2024-04-25T10:08:09.898830+0000","last_became_peered":"2024-04-25T10:08:09.898830+0000","last_unstale":"2024-04-25T10:08:11.967486+0000","last_undegraded":"2024-04-25T10:08:11.967486+0000","last_fullsized":"2024-04-25T10:08:11.967486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:19.781296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.892879+0000","last_change":"2024-04-25T10:08:10.901543+0000","last_active":"2024-04-25T10:08:11.892879+0000","last_peered":"2024-04-25T10:08:11.892879+0000","last_clean":"2024-04-25T10:08:11.892879+0000","last_became_active":"2024-04-25T10:08:10.900826+0000","last_became_peered":"2024-04-25T10:08:10.900826+0000","last_unstale":"2024-04-25T10:08:11.892879+0000","last_undegraded":"2024-04-25T10:08:11.892879+0000","last_fullsized":"2024-04-25T10:08:11.892879+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:32.461497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893451+0000","last_change":"2024-04-25T10:08:09.902684+0000","last_active":"2024-04-25T10:08:11.893451+0000","last_peered":"2024-04-25T10:08:11.893451+0000","last_clean":"2024-04-25T10:08:11.893451+0000","last_became_active":"2024-04-25T10:08:09.901806+0000","last_became_peered":"2024-04-25T10:08:09.901806+0000","last_unstale":"2024-04-25T10:08:11.893451+0000","last_undegraded":"2024-04-25T10:08:11.893451+0000","last_fullsized":"2024-04-25T10:08:11.893451+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:17.199198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396317+0000","last_change":"2024-04-25T10:08:10.925089+0000","last_active":"2024-04-25T10:08:13.396317+0000","last_peered":"2024-04-25T10:08:13.396317+0000","last_clean":"2024-04-25T10:08:13.396317+0000","last_became_active":"2024-04-25T10:08:10.922906+0000","last_became_peered":"2024-04-25T10:08:10.922906+0000","last_unstale":"2024-04-25T10:08:13.396317+0000","last_undegraded":"2024-04-25T10:08:13.396317+0000","last_fullsized":"2024-04-25T10:08:13.396317+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:55.438825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":22,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.395853+0000","last_change":"2024-04-25T10:08:09.908904+0000","last_active":"2024-04-25T10:08:13.395853+0000","last_peered":"2024-04-25T10:08:13.395853+0000","last_clean":"2024-04-25T10:08:13.395853+0000","last_became_active":"2024-04-25T10:08:09.908076+0000","last_became_peered":"2024-04-25T10:08:09.908076+0000","last_unstale":"2024-04-25T10:08:13.395853+0000","last_undegraded":"2024-04-25T10:08:13.395853+0000","last_fullsized":"2024-04-25T10:08:13.395853+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:12.528702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.397170+0000","last_change":"2024-04-25T10:08:10.916890+0000","last_active":"2024-04-25T10:08:13.397170+0000","last_peered":"2024-04-25T10:08:13.397170+0000","last_clean":"2024-04-25T10:08:13.397170+0000","last_became_active":"2024-04-25T10:08:10.910435+0000","last_became_peered":"2024-04-25T10:08:10.910435+0000","last_unstale":"2024-04-25T10:08:13.397170+0000","last_undegraded":"2024-04-25T10:08:13.397170+0000","last_fullsized":"2024-04-25T10:08:13.397170+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:10.886429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396184+0000","last_change":"2024-04-25T10:08:09.884420+0000","last_active":"2024-04-25T10:08:13.396184+0000","last_peered":"2024-04-25T10:08:13.396184+0000","last_clean":"2024-04-25T10:08:13.396184+0000","last_became_active":"2024-04-25T10:08:09.883427+0000","last_became_peered":"2024-04-25T10:08:09.883427+0000","last_unstale":"2024-04-25T10:08:13.396184+0000","last_undegraded":"2024-04-25T10:08:13.396184+0000","last_fullsized":"2024-04-25T10:08:13.396184+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:47.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894725+0000","last_change":"2024-04-25T10:08:10.918083+0000","last_active":"2024-04-25T10:08:11.894725+0000","last_peered":"2024-04-25T10:08:11.894725+0000","last_clean":"2024-04-25T10:08:11.894725+0000","last_became_active":"2024-04-25T10:08:10.917512+0000","last_became_peered":"2024-04-25T10:08:10.917512+0000","last_unstale":"2024-04-25T10:08:11.894725+0000","last_undegraded":"2024-04-25T10:08:11.894725+0000","last_fullsized":"2024-04-25T10:08:11.894725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:35.281267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:12.875117+0000","last_change":"2024-04-25T10:08:09.888804+0000","last_active":"2024-04-25T10:08:12.875117+0000","last_peered":"2024-04-25T10:08:12.875117+0000","last_clean":"2024-04-25T10:08:12.875117+0000","last_became_active":"2024-04-25T10:08:09.888488+0000","last_became_peered":"2024-04-25T10:08:09.888488+0000","last_unstale":"2024-04-25T10:08:12.875117+0000","last_undegraded":"2024-04-25T10:08:12.875117+0000","last_fullsized":"2024-04-25T10:08:12.875117+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:17.199198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.394995+0000","last_change":"2024-04-25T10:08:10.905832+0000","last_active":"2024-04-25T10:08:13.394995+0000","last_peered":"2024-04-25T10:08:13.394995+0000","last_clean":"2024-04-25T10:08:13.394995+0000","last_became_active":"2024-04-25T10:08:10.905524+0000","last_became_peered":"2024-04-25T10:08:10.905524+0000","last_unstale":"2024-04-25T10:08:13.394995+0000","last_undegraded":"2024-04-25T10:08:13.394995+0000","last_fullsized":"2024-04-25T10:08:13.394995+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:15.689286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893944+0000","last_change":"2024-04-25T10:08:09.902367+0000","last_active":"2024-04-25T10:08:11.893944+0000","last_peered":"2024-04-25T10:08:11.893944+0000","last_clean":"2024-04-25T10:08:11.893944+0000","last_became_active":"2024-04-25T10:08:09.900592+0000","last_became_peered":"2024-04-25T10:08:09.900592+0000","last_unstale":"2024-04-25T10:08:11.893944+0000","last_undegraded":"2024-04-25T10:08:11.893944+0000","last_fullsized":"2024-04-25T10:08:11.893944+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:38.492052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.397108+0000","last_change":"2024-04-25T10:08:10.928860+0000","last_active":"2024-04-25T10:08:13.397108+0000","last_peered":"2024-04-25T10:08:13.397108+0000","last_clean":"2024-04-25T10:08:13.397108+0000","last_became_active":"2024-04-25T10:08:10.921892+0000","last_became_peered":"2024-04-25T10:08:10.921892+0000","last_unstale":"2024-04-25T10:08:13.397108+0000","last_undegraded":"2024-04-25T10:08:13.397108+0000","last_fullsized":"2024-04-25T10:08:13.397108+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:31.914077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:12.875281+0000","last_change":"2024-04-25T10:08:09.890143+0000","last_active":"2024-04-25T10:08:12.875281+0000","last_peered":"2024-04-25T10:08:12.875281+0000","last_clean":"2024-04-25T10:08:12.875281+0000","last_became_active":"2024-04-25T10:08:09.889126+0000","last_became_peered":"2024-04-25T10:08:09.889126+0000","last_unstale":"2024-04-25T10:08:12.875281+0000","last_undegraded":"2024-04-25T10:08:12.875281+0000","last_fullsized":"2024-04-25T10:08:12.875281+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:38.492052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893893+0000","last_change":"2024-04-25T10:08:10.917214+0000","last_active":"2024-04-25T10:08:11.893893+0000","last_peered":"2024-04-25T10:08:11.893893+0000","last_clean":"2024-04-25T10:08:11.893893+0000","last_became_active":"2024-04-25T10:08:10.915770+0000","last_became_peered":"2024-04-25T10:08:10.915770+0000","last_unstale":"2024-04-25T10:08:11.893893+0000","last_undegraded":"2024-04-25T10:08:11.893893+0000","last_fullsized":"2024-04-25T10:08:11.893893+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:44.066536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.394969+0000","last_change":"2024-04-25T10:08:09.889071+0000","last_active":"2024-04-25T10:08:13.394969+0000","last_peered":"2024-04-25T10:08:13.394969+0000","last_clean":"2024-04-25T10:08:13.394969+0000","last_became_active":"2024-04-25T10:08:09.888746+0000","last_became_peered":"2024-04-25T10:08:09.888746+0000","last_unstale":"2024-04-25T10:08:13.394969+0000","last_undegraded":"2024-04-25T10:08:13.394969+0000","last_fullsized":"2024-04-25T10:08:13.394969+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:43.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:12.875152+0000","last_change":"2024-04-25T10:08:10.893599+0000","last_active":"2024-04-25T10:08:12.875152+0000","last_peered":"2024-04-25T10:08:12.875152+0000","last_clean":"2024-04-25T10:08:12.875152+0000","last_became_active":"2024-04-25T10:08:10.893206+0000","last_became_peered":"2024-04-25T10:08:10.893206+0000","last_unstale":"2024-04-25T10:08:12.875152+0000","last_undegraded":"2024-04-25T10:08:12.875152+0000","last_fullsized":"2024-04-25T10:08:12.875152+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:34.974943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889836+0000","last_change":"2024-04-25T10:08:09.880919+0000","last_active":"2024-04-25T10:08:11.889836+0000","last_peered":"2024-04-25T10:08:11.889836+0000","last_clean":"2024-04-25T10:08:11.889836+0000","last_became_active":"2024-04-25T10:08:09.880611+0000","last_became_peered":"2024-04-25T10:08:09.880611+0000","last_unstale":"2024-04-25T10:08:11.889836+0000","last_undegraded":"2024-04-25T10:08:11.889836+0000","last_fullsized":"2024-04-25T10:08:11.889836+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:22.155681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:12.874231+0000","last_change":"2024-04-25T10:08:10.901150+0000","last_active":"2024-04-25T10:08:12.874231+0000","last_peered":"2024-04-25T10:08:12.874231+0000","last_clean":"2024-04-25T10:08:12.874231+0000","last_became_active":"2024-04-25T10:08:10.900624+0000","last_became_peered":"2024-04-25T10:08:10.900624+0000","last_unstale":"2024-04-25T10:08:12.874231+0000","last_undegraded":"2024-04-25T10:08:12.874231+0000","last_fullsized":"2024-04-25T10:08:12.874231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:44.473365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.892866+0000","last_change":"2024-04-25T10:08:09.901094+0000","last_active":"2024-04-25T10:08:11.892866+0000","last_peered":"2024-04-25T10:08:11.892866+0000","last_clean":"2024-04-25T10:08:11.892866+0000","last_became_active":"2024-04-25T10:08:09.900405+0000","last_became_peered":"2024-04-25T10:08:09.900405+0000","last_unstale":"2024-04-25T10:08:11.892866+0000","last_undegraded":"2024-04-25T10:08:11.892866+0000","last_fullsized":"2024-04-25T10:08:11.892866+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:30.789048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.893987+0000","last_change":"2024-04-25T10:08:10.909724+0000","last_active":"2024-04-25T10:08:11.893987+0000","last_peered":"2024-04-25T10:08:11.893987+0000","last_clean":"2024-04-25T10:08:11.893987+0000","last_became_active":"2024-04-25T10:08:10.904979+0000","last_became_peered":"2024-04-25T10:08:10.904979+0000","last_unstale":"2024-04-25T10:08:11.893987+0000","last_undegraded":"2024-04-25T10:08:11.893987+0000","last_fullsized":"2024-04-25T10:08:11.893987+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:17.598272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889987+0000","last_change":"2024-04-25T10:08:09.885875+0000","last_active":"2024-04-25T10:08:11.889987+0000","last_peered":"2024-04-25T10:08:11.889987+0000","last_clean":"2024-04-25T10:08:11.889987+0000","last_became_active":"2024-04-25T10:08:09.885529+0000","last_became_peered":"2024-04-25T10:08:09.885529+0000","last_unstale":"2024-04-25T10:08:11.889987+0000","last_undegraded":"2024-04-25T10:08:11.889987+0000","last_fullsized":"2024-04-25T10:08:11.889987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:30.869245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.895037+0000","last_change":"2024-04-25T10:08:10.923948+0000","last_active":"2024-04-25T10:08:11.895037+0000","last_peered":"2024-04-25T10:08:11.895037+0000","last_clean":"2024-04-25T10:08:11.895037+0000","last_became_active":"2024-04-25T10:08:10.923383+0000","last_became_peered":"2024-04-25T10:08:10.923383+0000","last_unstale":"2024-04-25T10:08:11.895037+0000","last_undegraded":"2024-04-25T10:08:11.895037+0000","last_fullsized":"2024-04-25T10:08:11.895037+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:55.065275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:12.875520+0000","last_change":"2024-04-25T10:08:09.894894+0000","last_active":"2024-04-25T10:08:12.875520+0000","last_peered":"2024-04-25T10:08:12.875520+0000","last_clean":"2024-04-25T10:08:12.875520+0000","last_became_active":"2024-04-25T10:08:09.894350+0000","last_became_peered":"2024-04-25T10:08:09.894350+0000","last_unstale":"2024-04-25T10:08:12.875520+0000","last_undegraded":"2024-04-25T10:08:12.875520+0000","last_fullsized":"2024-04-25T10:08:12.875520+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.592539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889925+0000","last_change":"2024-04-25T10:08:10.893322+0000","last_active":"2024-04-25T10:08:11.889925+0000","last_peered":"2024-04-25T10:08:11.889925+0000","last_clean":"2024-04-25T10:08:11.889925+0000","last_became_active":"2024-04-25T10:08:10.892639+0000","last_became_peered":"2024-04-25T10:08:10.892639+0000","last_unstale":"2024-04-25T10:08:11.889925+0000","last_undegraded":"2024-04-25T10:08:11.889925+0000","last_fullsized":"2024-04-25T10:08:11.889925+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:37.445694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967957+0000","last_change":"2024-04-25T10:08:09.893305+0000","last_active":"2024-04-25T10:08:11.967957+0000","last_peered":"2024-04-25T10:08:11.967957+0000","last_clean":"2024-04-25T10:08:11.967957+0000","last_became_active":"2024-04-25T10:08:09.892501+0000","last_became_peered":"2024-04-25T10:08:09.892501+0000","last_unstale":"2024-04-25T10:08:11.967957+0000","last_undegraded":"2024-04-25T10:08:11.967957+0000","last_fullsized":"2024-04-25T10:08:11.967957+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:21.522866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:12.875095+0000","last_change":"2024-04-25T10:08:10.926260+0000","last_active":"2024-04-25T10:08:12.875095+0000","last_peered":"2024-04-25T10:08:12.875095+0000","last_clean":"2024-04-25T10:08:12.875095+0000","last_became_active":"2024-04-25T10:08:10.926038+0000","last_became_peered":"2024-04-25T10:08:10.926038+0000","last_unstale":"2024-04-25T10:08:12.875095+0000","last_undegraded":"2024-04-25T10:08:12.875095+0000","last_fullsized":"2024-04-25T10:08:12.875095+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:30.075855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894376+0000","last_change":"2024-04-25T10:08:09.906942+0000","last_active":"2024-04-25T10:08:11.894376+0000","last_peered":"2024-04-25T10:08:11.894376+0000","last_clean":"2024-04-25T10:08:11.894376+0000","last_became_active":"2024-04-25T10:08:09.906192+0000","last_became_peered":"2024-04-25T10:08:09.906192+0000","last_unstale":"2024-04-25T10:08:11.894376+0000","last_undegraded":"2024-04-25T10:08:11.894376+0000","last_fullsized":"2024-04-25T10:08:11.894376+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:48.349703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.395031+0000","last_change":"2024-04-25T10:08:10.890698+0000","last_active":"2024-04-25T10:08:13.395031+0000","last_peered":"2024-04-25T10:08:13.395031+0000","last_clean":"2024-04-25T10:08:13.395031+0000","last_became_active":"2024-04-25T10:08:10.889982+0000","last_became_peered":"2024-04-25T10:08:10.889982+0000","last_unstale":"2024-04-25T10:08:13.395031+0000","last_undegraded":"2024-04-25T10:08:13.395031+0000","last_fullsized":"2024-04-25T10:08:13.395031+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:32.413213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889624+0000","last_change":"2024-04-25T10:08:09.902559+0000","last_active":"2024-04-25T10:08:11.889624+0000","last_peered":"2024-04-25T10:08:11.889624+0000","last_clean":"2024-04-25T10:08:11.889624+0000","last_became_active":"2024-04-25T10:08:09.902240+0000","last_became_peered":"2024-04-25T10:08:09.902240+0000","last_unstale":"2024-04-25T10:08:11.889624+0000","last_undegraded":"2024-04-25T10:08:11.889624+0000","last_fullsized":"2024-04-25T10:08:11.889624+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:30.202367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.396825+0000","last_change":"2024-04-25T10:08:10.924116+0000","last_active":"2024-04-25T10:08:13.396825+0000","last_peered":"2024-04-25T10:08:13.396825+0000","last_clean":"2024-04-25T10:08:13.396825+0000","last_became_active":"2024-04-25T10:08:10.922605+0000","last_became_peered":"2024-04-25T10:08:10.922605+0000","last_unstale":"2024-04-25T10:08:13.396825+0000","last_undegraded":"2024-04-25T10:08:13.396825+0000","last_fullsized":"2024-04-25T10:08:13.396825+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:07.785780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.894016+0000","last_change":"2024-04-25T10:08:09.887885+0000","last_active":"2024-04-25T10:08:11.894016+0000","last_peered":"2024-04-25T10:08:11.894016+0000","last_clean":"2024-04-25T10:08:11.894016+0000","last_became_active":"2024-04-25T10:08:09.887165+0000","last_became_peered":"2024-04-25T10:08:09.887165+0000","last_unstale":"2024-04-25T10:08:11.894016+0000","last_undegraded":"2024-04-25T10:08:11.894016+0000","last_fullsized":"2024-04-25T10:08:11.894016+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:09.379274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889431+0000","last_change":"2024-04-25T10:08:10.894492+0000","last_active":"2024-04-25T10:08:11.889431+0000","last_peered":"2024-04-25T10:08:11.889431+0000","last_clean":"2024-04-25T10:08:11.889431+0000","last_became_active":"2024-04-25T10:08:10.894221+0000","last_became_peered":"2024-04-25T10:08:10.894221+0000","last_unstale":"2024-04-25T10:08:11.889431+0000","last_undegraded":"2024-04-25T10:08:11.889431+0000","last_fullsized":"2024-04-25T10:08:11.889431+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:47.897507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.889533+0000","last_change":"2024-04-25T10:08:09.890601+0000","last_active":"2024-04-25T10:08:11.889533+0000","last_peered":"2024-04-25T10:08:11.889533+0000","last_clean":"2024-04-25T10:08:11.889533+0000","last_became_active":"2024-04-25T10:08:09.890208+0000","last_became_peered":"2024-04-25T10:08:09.890208+0000","last_unstale":"2024-04-25T10:08:11.889533+0000","last_undegraded":"2024-04-25T10:08:11.889533+0000","last_fullsized":"2024-04-25T10:08:11.889533+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:01.465554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.394525+0000","last_change":"2024-04-25T10:08:10.898030+0000","last_active":"2024-04-25T10:08:13.394525+0000","last_peered":"2024-04-25T10:08:13.394525+0000","last_clean":"2024-04-25T10:08:13.394525+0000","last_became_active":"2024-04-25T10:08:10.897719+0000","last_became_peered":"2024-04-25T10:08:10.897719+0000","last_unstale":"2024-04-25T10:08:13.394525+0000","last_undegraded":"2024-04-25T10:08:13.394525+0000","last_fullsized":"2024-04-25T10:08:13.394525+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:48.916229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.394773+0000","last_change":"2024-04-25T10:08:09.900914+0000","last_active":"2024-04-25T10:08:13.394773+0000","last_peered":"2024-04-25T10:08:13.394773+0000","last_clean":"2024-04-25T10:08:13.394773+0000","last_became_active":"2024-04-25T10:08:09.900669+0000","last_became_peered":"2024-04-25T10:08:09.900669+0000","last_unstale":"2024-04-25T10:08:13.394773+0000","last_undegraded":"2024-04-25T10:08:13.394773+0000","last_fullsized":"2024-04-25T10:08:13.394773+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.736275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.394502+0000","last_change":"2024-04-25T10:08:10.894360+0000","last_active":"2024-04-25T10:08:13.394502+0000","last_peered":"2024-04-25T10:08:13.394502+0000","last_clean":"2024-04-25T10:08:13.394502+0000","last_became_active":"2024-04-25T10:08:10.894017+0000","last_became_peered":"2024-04-25T10:08:10.894017+0000","last_unstale":"2024-04-25T10:08:13.394502+0000","last_undegraded":"2024-04-25T10:08:13.394502+0000","last_fullsized":"2024-04-25T10:08:13.394502+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:09.849679+0000","last_clean_scrub_stamp":"2024-04-25T10:08:09.849679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:21.963782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:11.967562+0000","last_change":"2024-04-25T10:08:09.909488+0000","last_active":"2024-04-25T10:08:11.967562+0000","last_peered":"2024-04-25T10:08:11.967562+0000","last_clean":"2024-04-25T10:08:11.967562+0000","last_became_active":"2024-04-25T10:08:09.908863+0000","last_became_peered":"2024-04-25T10:08:09.908863+0000","last_unstale":"2024-04-25T10:08:11.967562+0000","last_undegraded":"2024-04-25T10:08:11.967562+0000","last_fullsized":"2024-04-25T10:08:11.967562+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:08.840704+0000","last_clean_scrub_stamp":"2024-04-25T10:08:08.840704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:48.865163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":237,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.395879+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:08:13.395879+0000","last_peered":"2024-04-25T10:08:13.395879+0000","last_clean":"2024-04-25T10:08:13.395879+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:08:13.395879+0000","last_undegraded":"2024-04-25T10:08:13.395879+0000","last_fullsized":"2024-04-25T10:08:13.395879+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":876429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":700416,"data_stored":287513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542184,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":688128,"data_stored":285901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":286553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542185,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":696320,"data_stored":285957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":696320,"data_stored":285957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":692224,"data_stored":285991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1298432,"data_stored":878463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:18.749 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:08:18.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:08:18.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.922+0000 7f6611096640 1 Processor -- start 2024-04-25T10:08:18.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.922+0000 7f6611096640 1 -- start start 2024-04-25T10:08:18.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.922+0000 7f6611096640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f660c0ce950 0x7f660c15fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.922+0000 7f6611096640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660c0b0c60 con 0x7f660c15ec40 2024-04-25T10:08:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.922+0000 7f6611096640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660c0d02a0 con 0x7f660c0cfe10 2024-04-25T10:08:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.922+0000 7f6611096640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660c160400 con 0x7f660c0ce950 2024-04-25T10:08:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.922+0000 7f660ad9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f660c15ec40 0x7f660c0ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52018/0 (socket says 172.21.15.60:52018) 2024-04-25T10:08:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.922+0000 7f660ad9d640 1 -- 172.21.15.60:0/2268603868 learned_addr learned my addr 172.21.15.60:0/2268603868 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660b59e640 1 --2- 172.21.15.60:0/2268603868 >> v2:172.21.15.100:3300/0 conn(0x7f660c0ce950 0x7f660c15fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 -- 172.21.15.60:0/2268603868 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792911581 0 0) 0x7f660c0b0c60 con 0x7f660c15ec40 2024-04-25T10:08:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 -- 172.21.15.60:0/2268603868 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65f8003530 con 0x7f660c15ec40 2024-04-25T10:08:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 -- 172.21.15.60:0/2268603868 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2660236761 0 0) 0x7f65f8003530 con 0x7f660c15ec40 2024-04-25T10:08:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 -- 172.21.15.60:0/2268603868 >> v2:172.21.15.100:3300/0 conn(0x7f660c0ce950 msgr2=0x7f660c15fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 --2- 172.21.15.60:0/2268603868 >> v2:172.21.15.100:3300/0 conn(0x7f660c0ce950 0x7f660c15fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 -- 172.21.15.60:0/2268603868 >> v1:172.21.15.100:6789/0 conn(0x7f660c0cfe10 legacy=0x7f660c0cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 -- 172.21.15.60:0/2268603868 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c0cc340 con 0x7f660c15ec40 2024-04-25T10:08:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 -- 172.21.15.60:0/2268603868 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f65fc0016a0 con 0x7f660c15ec40 2024-04-25T10:08:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 -- 172.21.15.60:0/2268603868 >> v1:172.21.15.60:6789/0 conn(0x7f660c15ec40 legacy=0x7f660c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 --2- 172.21.15.60:0/2268603868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f80039c0 0x7f65f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 --2- 172.21.15.60:0/2268603868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f8004810 0x7f65f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 --2- 172.21.15.60:0/2268603868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f80051d0 0x7f65f800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 -- 172.21.15.60:0/2268603868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c160400 con 0x7f65f80051d0 2024-04-25T10:08:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 -- 172.21.15.60:0/2268603868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f800c170 con 0x7f65f8004810 2024-04-25T10:08:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 -- 172.21.15.60:0/2268603868 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f800c370 con 0x7f65f80039c0 2024-04-25T10:08:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660b59e640 1 --2- 172.21.15.60:0/2268603868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f80051d0 0x7f65f800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660259c640 1 --2- 172.21.15.60:0/2268603868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f8004810 0x7f65f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660ad9d640 1 --2- 172.21.15.60:0/2268603868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f80039c0 0x7f65f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660b59e640 1 -- 172.21.15.60:0/2268603868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f80039c0 msgr2=0x7f65f8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660b59e640 1 --2- 172.21.15.60:0/2268603868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f80039c0 0x7f65f8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660b59e640 1 -- 172.21.15.60:0/2268603868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f8004810 msgr2=0x7f65f8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:18.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660b59e640 1 --2- 172.21.15.60:0/2268603868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f8004810 0x7f65f8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660b59e640 1 -- 172.21.15.60:0/2268603868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f8003530 con 0x7f65f80051d0 2024-04-25T10:08:18.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660259c640 1 --2- 172.21.15.60:0/2268603868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f8004810 0x7f65f8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:18.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660b59e640 1 -- 172.21.15.60:0/2268603868 reap_dead start 2024-04-25T10:08:18.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660b59e640 1 --2- 172.21.15.60:0/2268603868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f80051d0 0x7f65f800b9d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6604009fc0 tx=0x7f660401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d222c9488b11b07e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:18.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 -- 172.21.15.60:0/2268603868 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604009c10 con 0x7f65f80051d0 2024-04-25T10:08:18.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 -- 172.21.15.60:0/2268603868 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66040a5c00 con 0x7f65f80051d0 2024-04-25T10:08:18.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f660a59c640 1 -- 172.21.15.60:0/2268603868 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66040039a0 con 0x7f65f80051d0 2024-04-25T10:08:18.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f6611096640 1 -- 172.21.15.60:0/2268603868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f80051d0 msgr2=0x7f65f800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f6611096640 1 --2- 172.21.15.60:0/2268603868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f80051d0 0x7f65f800b9d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6604009fc0 tx=0x7f660401ccc0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f6611096640 1 -- 172.21.15.60:0/2268603868 shutdown_connections 2024-04-25T10:08:18.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f6611096640 1 --2- 172.21.15.60:0/2268603868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f80051d0 0x7f65f800b9d0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f6611096640 1 -- 172.21.15.60:0/2268603868 >> 172.21.15.60:0/2268603868 conn(0x7f660c0d1830 msgr2=0x7f660c0d1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:18.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.926+0000 7f6611096640 1 -- 172.21.15.60:0/2268603868 shutdown_connections 2024-04-25T10:08:18.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f6611096640 1 -- 172.21.15.60:0/2268603868 wait complete. 2024-04-25T10:08:18.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f6611096640 1 Processor -- start 2024-04-25T10:08:18.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f6611096640 1 -- start start 2024-04-25T10:08:18.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f6611096640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f80051d0 0x7f660c150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f6611096640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f660c0ca870 0x7f660c15d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f6611096640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f660c15dd70 0x7f660c0c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f6611096640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c0ae5c0 con 0x7f65f80051d0 2024-04-25T10:08:18.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f6611096640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c0cc900 con 0x7f660c0ca870 2024-04-25T10:08:18.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f6611096640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660c0d0580 con 0x7f660c15dd70 2024-04-25T10:08:18.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f660ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f80051d0 0x7f660c150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f660ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f80051d0 0x7f660c150860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33320/0 (socket says 172.21.15.60:33320) 2024-04-25T10:08:18.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f660ad9d640 1 -- 172.21.15.60:0/2890991160 learned_addr learned my addr 172.21.15.60:0/2890991160 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:18.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f660259c640 1 --2- 172.21.15.60:0/2890991160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f660c0ca870 0x7f660c15d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f660b59e640 1 --2- 172.21.15.60:0/2890991160 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f660c15dd70 0x7f660c0c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f660ad9d640 1 -- 172.21.15.60:0/2890991160 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f660c15dd70 msgr2=0x7f660c0c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f660ad9d640 1 --2- 172.21.15.60:0/2890991160 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f660c15dd70 0x7f660c0c6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f660ad9d640 1 -- 172.21.15.60:0/2890991160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f660c0ca870 msgr2=0x7f660c15d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f660ad9d640 1 --2- 172.21.15.60:0/2890991160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f660c0ca870 0x7f660c15d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f660ad9d640 1 -- 172.21.15.60:0/2890991160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c1f0f80 con 0x7f65f80051d0 2024-04-25T10:08:18.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f660259c640 1 --2- 172.21.15.60:0/2890991160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f660c0ca870 0x7f660c15d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:18.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f660ad9d640 1 --2- 172.21.15.60:0/2890991160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f80051d0 0x7f660c150860 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f65fc011810 tx=0x7f65fc011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:18.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f6603fff640 1 -- 172.21.15.60:0/2890991160 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc010040 con 0x7f65f80051d0 2024-04-25T10:08:18.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f6603fff640 1 -- 172.21.15.60:0/2890991160 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc01a070 con 0x7f65f80051d0 2024-04-25T10:08:18.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f6611096640 1 -- 172.21.15.60:0/2890991160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660c1f1260 con 0x7f65f80051d0 2024-04-25T10:08:18.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f6603fff640 1 -- 172.21.15.60:0/2890991160 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc00fba0 con 0x7f65f80051d0 2024-04-25T10:08:18.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f6611096640 1 -- 172.21.15.60:0/2890991160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660c1f1820 con 0x7f65f80051d0 2024-04-25T10:08:18.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.930+0000 7f6611096640 1 -- 172.21.15.60:0/2890991160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660c074770 con 0x7f65f80051d0 2024-04-25T10:08:18.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.934+0000 7f6603fff640 1 -- 172.21.15.60:0/2890991160 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65fc01f460 con 0x7f65f80051d0 2024-04-25T10:08:18.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.934+0000 7f6603fff640 1 --2- 172.21.15.60:0/2890991160 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65ec066f60 0x7f65ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.934+0000 7f6603fff640 1 -- 172.21.15.60:0/2890991160 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65fc088160 con 0x7f65f80051d0 2024-04-25T10:08:18.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.934+0000 7f660259c640 1 --2- 172.21.15.60:0/2890991160 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65ec066f60 0x7f65ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.934+0000 7f660259c640 1 --2- 172.21.15.60:0/2890991160 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65ec066f60 0x7f65ec069420 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f65f4000c10 tx=0x7f65f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:18.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:18.938+0000 7f6603fff640 1 -- 172.21.15.60:0/2890991160 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65fc059170 con 0x7f65f80051d0 2024-04-25T10:08:19.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.158+0000 7f6611096640 1 -- 172.21.15.60:0/2890991160 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f660c15e150 con 0x7f65f80051d0 2024-04-25T10:08:19.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.162+0000 7f6603fff640 1 -- 172.21.15.60:0/2890991160 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f65fc058a70 con 0x7f65f80051d0 2024-04-25T10:08:19.168 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T10:08:19.168 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCiKipmv65dLxAA6/QpPrqE9aT+CtZqkyutog== 2024-04-25T10:08:19.169 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-04-25T10:08:19.169 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-04-25T10:08:19.169 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:08:19.169 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-04-25T10:08:19.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.162+0000 7f6611096640 1 -- 172.21.15.60:0/2890991160 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65ec066f60 msgr2=0x7f65ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.162+0000 7f6611096640 1 --2- 172.21.15.60:0/2890991160 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65ec066f60 0x7f65ec069420 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f65f4000c10 tx=0x7f65f4009040 comp rx=0 tx=0).stop 2024-04-25T10:08:19.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.162+0000 7f6611096640 1 -- 172.21.15.60:0/2890991160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f80051d0 msgr2=0x7f660c150860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.162+0000 7f6611096640 1 --2- 172.21.15.60:0/2890991160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f80051d0 0x7f660c150860 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f65fc011810 tx=0x7f65fc011cd0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.162+0000 7f6611096640 1 -- 172.21.15.60:0/2890991160 shutdown_connections 2024-04-25T10:08:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.162+0000 7f6611096640 1 --2- 172.21.15.60:0/2890991160 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65ec066f60 0x7f65ec069420 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.162+0000 7f6611096640 1 --2- 172.21.15.60:0/2890991160 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f660c15dd70 0x7f660c0c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.162+0000 7f6611096640 1 --2- 172.21.15.60:0/2890991160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f660c0ca870 0x7f660c15d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.162+0000 7f6611096640 1 --2- 172.21.15.60:0/2890991160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f80051d0 0x7f660c150860 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.162+0000 7f6611096640 1 -- 172.21.15.60:0/2890991160 >> 172.21.15.60:0/2890991160 conn(0x7f660c0d1830 msgr2=0x7f660c0703d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.162+0000 7f6611096640 1 -- 172.21.15.60:0/2890991160 shutdown_connections 2024-04-25T10:08:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.162+0000 7f6611096640 1 -- 172.21.15.60:0/2890991160 wait complete. 2024-04-25T10:08:19.186 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:19.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.298+0000 7f8f1c44d640 1 Processor -- start 2024-04-25T10:08:19.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.298+0000 7f8f1c44d640 1 -- start start 2024-04-25T10:08:19.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f1c44d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1415f480 0x7f8f1415f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f1c44d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8f140cd390 0x7f8f1409a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f1c44d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f1415fda0 con 0x7f8f1409a930 2024-04-25T10:08:19.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f1c44d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f140cc5d0 con 0x7f8f140cd390 2024-04-25T10:08:19.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f1c44d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f140cc8b0 con 0x7f8f1415f480 2024-04-25T10:08:19.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f199e8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8f140cd390 0x7f8f1409a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f1a9ea640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8f1409a930 0x7f8f140cf380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52024/0 (socket says 172.21.15.60:52024) 2024-04-25T10:08:19.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f1a9ea640 1 -- 172.21.15.60:0/3579680064 learned_addr learned my addr 172.21.15.60:0/3579680064 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:19.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f1a1e9640 1 --2- 172.21.15.60:0/3579680064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1415f480 0x7f8f1415f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f199e8640 1 -- 172.21.15.60:0/3579680064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1415f480 msgr2=0x7f8f1415f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:19.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f199e8640 1 --2- 172.21.15.60:0/3579680064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1415f480 0x7f8f1415f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f199e8640 1 -- 172.21.15.60:0/3579680064 >> v1:172.21.15.60:6789/0 conn(0x7f8f1409a930 legacy=0x7f8f140cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f199e8640 1 -- 172.21.15.60:0/3579680064 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f140ccb90 con 0x7f8f140cd390 2024-04-25T10:08:19.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f191e7640 1 -- 172.21.15.60:0/3579680064 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826631023 0 0) 0x7f8f1415fda0 con 0x7f8f1409a930 2024-04-25T10:08:19.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f199e8640 1 --2- 172.21.15.60:0/3579680064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f140cd390 0x7f8f1409a3f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8f040097c0 tx=0x7f8f0401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efda6e4c1b9cea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f191e7640 1 -- 172.21.15.60:0/3579680064 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f040ab5f0 con 0x7f8f140cd390 2024-04-25T10:08:19.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f191e7640 1 -- 172.21.15.60:0/3579680064 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f040b0070 con 0x7f8f140cd390 2024-04-25T10:08:19.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.302+0000 7f8f191e7640 1 -- 172.21.15.60:0/3579680064 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f040ab9b0 con 0x7f8f140cd390 2024-04-25T10:08:19.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 -- 172.21.15.60:0/3579680064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f140cd390 msgr2=0x7f8f1409a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 --2- 172.21.15.60:0/3579680064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f140cd390 0x7f8f1409a3f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8f040097c0 tx=0x7f8f0401c470 comp rx=0 tx=0).stop 2024-04-25T10:08:19.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 -- 172.21.15.60:0/3579680064 shutdown_connections 2024-04-25T10:08:19.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 --2- 172.21.15.60:0/3579680064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f140cd390 0x7f8f1409a3f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 --2- 172.21.15.60:0/3579680064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1415f480 0x7f8f1415f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 -- 172.21.15.60:0/3579680064 >> 172.21.15.60:0/3579680064 conn(0x7f8f140b32a0 msgr2=0x7f8f140b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:19.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 -- 172.21.15.60:0/3579680064 shutdown_connections 2024-04-25T10:08:19.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 -- 172.21.15.60:0/3579680064 wait complete. 2024-04-25T10:08:19.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 Processor -- start 2024-04-25T10:08:19.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 -- start start 2024-04-25T10:08:19.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1409a930 0x7f8f14152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f140cd390 0x7f8f141f9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f1415f480 0x7f8f141fa120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1415cd40 con 0x7f8f140cd390 2024-04-25T10:08:19.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1415cbc0 con 0x7f8f1415f480 2024-04-25T10:08:19.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f141687c0 con 0x7f8f1409a930 2024-04-25T10:08:19.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f199e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f140cd390 0x7f8f141f9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f199e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f140cd390 0x7f8f141f9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33334/0 (socket says 172.21.15.60:33334) 2024-04-25T10:08:19.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f199e8640 1 -- 172.21.15.60:0/1770857769 learned_addr learned my addr 172.21.15.60:0/1770857769 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:19.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1a9ea640 1 --2- 172.21.15.60:0/1770857769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f1415f480 0x7f8f141fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f199e8640 1 -- 172.21.15.60:0/1770857769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1409a930 msgr2=0x7f8f14152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1a1e9640 1 --2- 172.21.15.60:0/1770857769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1409a930 0x7f8f14152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f199e8640 1 --2- 172.21.15.60:0/1770857769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1409a930 0x7f8f14152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f199e8640 1 -- 172.21.15.60:0/1770857769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f1415f480 msgr2=0x7f8f141fa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f199e8640 1 --2- 172.21.15.60:0/1770857769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f1415f480 0x7f8f141fa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f199e8640 1 -- 172.21.15.60:0/1770857769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1415a250 con 0x7f8f140cd390 2024-04-25T10:08:19.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1a9ea640 1 --2- 172.21.15.60:0/1770857769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f1415f480 0x7f8f141fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:19.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f199e8640 1 --2- 172.21.15.60:0/1770857769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f140cd390 0x7f8f141f9be0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8f040ad040 tx=0x7f8f0401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f0affd640 1 -- 172.21.15.60:0/1770857769 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f040bf610 con 0x7f8f140cd390 2024-04-25T10:08:19.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f0affd640 1 -- 172.21.15.60:0/1770857769 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f040bf7d0 con 0x7f8f140cd390 2024-04-25T10:08:19.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 -- 172.21.15.60:0/1770857769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f1415a530 con 0x7f8f140cd390 2024-04-25T10:08:19.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f0affd640 1 -- 172.21.15.60:0/1770857769 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f040bfbc0 con 0x7f8f140cd390 2024-04-25T10:08:19.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.306+0000 7f8f1c44d640 1 -- 172.21.15.60:0/1770857769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1415aaf0 con 0x7f8f140cd390 2024-04-25T10:08:19.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.310+0000 7f8f1c44d640 1 -- 172.21.15.60:0/1770857769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f14074770 con 0x7f8f140cd390 2024-04-25T10:08:19.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.310+0000 7f8f0affd640 1 -- 172.21.15.60:0/1770857769 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f040bd020 con 0x7f8f140cd390 2024-04-25T10:08:19.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.310+0000 7f8f0affd640 1 --2- 172.21.15.60:0/1770857769 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ef8066f90 0x7f8ef8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.310+0000 7f8f0affd640 1 -- 172.21.15.60:0/1770857769 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f0411fa70 con 0x7f8f140cd390 2024-04-25T10:08:19.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.310+0000 7f8f1a1e9640 1 --2- 172.21.15.60:0/1770857769 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ef8066f90 0x7f8ef8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.310+0000 7f8f1a1e9640 1 --2- 172.21.15.60:0/1770857769 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ef8066f90 0x7f8ef8069450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009790 tx=0x7f8f10008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.314+0000 7f8f0affd640 1 -- 172.21.15.60:0/1770857769 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f04016090 con 0x7f8f140cd390 2024-04-25T10:08:19.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.506+0000 7f8f1c44d640 1 -- 172.21.15.60:0/1770857769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f140b40a0 con 0x7f8f140cd390 2024-04-25T10:08:19.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.510+0000 7f8f0affd640 1 -- 172.21.15.60:0/1770857769 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f8f040f0af0 con 0x7f8f140cd390 2024-04-25T10:08:19.517 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:19.517 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":28,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/380273391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":380273391},{"type":"v1","addr":"172.21.15.100:6836","nonce":380273391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5635,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2965977166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2965977166},{"type":"v1","addr":"172.21.15.100:6837","nonce":2965977166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1105028629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1105028629},{"type":"v1","addr":"172.21.15.60:6834","nonce":1105028629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:10.236590+0000","modified":"2024-04-25T10:08:12.229638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2718963870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2718963870},{"type":"v1","addr":"172.21.15.60:6835","nonce":2718963870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:08:19.518 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 28 2024-04-25T10:08:19.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.510+0000 7f8f1c44d640 1 -- 172.21.15.60:0/1770857769 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ef8066f90 msgr2=0x7f8ef8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.510+0000 7f8f1c44d640 1 --2- 172.21.15.60:0/1770857769 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ef8066f90 0x7f8ef8069450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009790 tx=0x7f8f10008040 comp rx=0 tx=0).stop 2024-04-25T10:08:19.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.510+0000 7f8f1c44d640 1 -- 172.21.15.60:0/1770857769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f140cd390 msgr2=0x7f8f141f9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.510+0000 7f8f1c44d640 1 --2- 172.21.15.60:0/1770857769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f140cd390 0x7f8f141f9be0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8f040ad040 tx=0x7f8f0401acb0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.510+0000 7f8f1c44d640 1 -- 172.21.15.60:0/1770857769 shutdown_connections 2024-04-25T10:08:19.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.510+0000 7f8f1c44d640 1 --2- 172.21.15.60:0/1770857769 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ef8066f90 0x7f8ef8069450 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.510+0000 7f8f1c44d640 1 --2- 172.21.15.60:0/1770857769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f1415f480 0x7f8f141fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.510+0000 7f8f1c44d640 1 --2- 172.21.15.60:0/1770857769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f140cd390 0x7f8f141f9be0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.510+0000 7f8f1c44d640 1 --2- 172.21.15.60:0/1770857769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1409a930 0x7f8f14152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.510+0000 7f8f1c44d640 1 -- 172.21.15.60:0/1770857769 >> 172.21.15.60:0/1770857769 conn(0x7f8f140b32a0 msgr2=0x7f8f1415f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:19.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.510+0000 7f8f1c44d640 1 -- 172.21.15.60:0/1770857769 shutdown_connections 2024-04-25T10:08:19.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:19.510+0000 7f8f1c44d640 1 -- 172.21.15.60:0/1770857769 wait complete. 2024-04-25T10:08:19.939 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:10.236590+0000', 'modified': '2024-04-25T10:08:12.229638+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6291': {'gid': 6291, 'name': 'a', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/2718963870', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 2718963870}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 2718963870}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:08:19.939 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:08:19.939 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:08:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.066+0000 7f396cd28640 1 Processor -- start 2024-04-25T10:08:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.066+0000 7f396cd28640 1 -- start start 2024-04-25T10:08:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.066+0000 7f396cd28640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39680ce7a0 0x7f39680ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.066+0000 7f396cd28640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f396815bc90 0x7f39680b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.066+0000 7f396cd28640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39680cf0c0 con 0x7f39680b4660 2024-04-25T10:08:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.066+0000 7f396cd28640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39680cc370 con 0x7f39680ce7a0 2024-04-25T10:08:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.066+0000 7f396cd28640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3968071ca0 con 0x7f396815bc90 2024-04-25T10:08:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.066+0000 7f396659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39680ce7a0 0x7f39680ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.066+0000 7f3966d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f39680b4660 0x7f39680b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52036/0 (socket says 172.21.15.60:52036) 2024-04-25T10:08:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.066+0000 7f3966d9d640 1 -- 172.21.15.60:0/3161325752 learned_addr learned my addr 172.21.15.60:0/3161325752 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f395dd9b640 1 --2- 172.21.15.60:0/3161325752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f396815bc90 0x7f39680b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f3965d9b640 1 -- 172.21.15.60:0/3161325752 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959078238 0 0) 0x7f39680cf0c0 con 0x7f39680b4660 2024-04-25T10:08:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f3965d9b640 1 -- 172.21.15.60:0/3161325752 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3954003530 con 0x7f39680b4660 2024-04-25T10:08:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f396659c640 1 -- 172.21.15.60:0/3161325752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f396815bc90 msgr2=0x7f39680b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f396659c640 1 --2- 172.21.15.60:0/3161325752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f396815bc90 0x7f39680b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f396659c640 1 -- 172.21.15.60:0/3161325752 >> v1:172.21.15.60:6789/0 conn(0x7f39680b4660 legacy=0x7f39680b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f396659c640 1 -- 172.21.15.60:0/3161325752 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39680cca40 con 0x7f39680ce7a0 2024-04-25T10:08:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f396659c640 1 --2- 172.21.15.60:0/3161325752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39680ce7a0 0x7f39680ceb80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f39580097f0 tx=0x7f395801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c778c0f27404d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f3965d9b640 1 -- 172.21.15.60:0/3161325752 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39580b0070 con 0x7f39680ce7a0 2024-04-25T10:08:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f3965d9b640 1 -- 172.21.15.60:0/3161325752 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39580ab600 con 0x7f39680ce7a0 2024-04-25T10:08:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f3965d9b640 1 -- 172.21.15.60:0/3161325752 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39580ab9b0 con 0x7f39680ce7a0 2024-04-25T10:08:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f396cd28640 1 -- 172.21.15.60:0/3161325752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39680ce7a0 msgr2=0x7f39680ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f396cd28640 1 --2- 172.21.15.60:0/3161325752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39680ce7a0 0x7f39680ceb80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f39580097f0 tx=0x7f395801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f396cd28640 1 -- 172.21.15.60:0/3161325752 shutdown_connections 2024-04-25T10:08:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f396cd28640 1 --2- 172.21.15.60:0/3161325752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f396815bc90 0x7f39680b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f396cd28640 1 --2- 172.21.15.60:0/3161325752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39680ce7a0 0x7f39680ceb80 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f396cd28640 1 -- 172.21.15.60:0/3161325752 >> 172.21.15.60:0/3161325752 conn(0x7f39680b12d0 msgr2=0x7f39680b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f396cd28640 1 -- 172.21.15.60:0/3161325752 shutdown_connections 2024-04-25T10:08:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.070+0000 7f396cd28640 1 -- 172.21.15.60:0/3161325752 wait complete. 2024-04-25T10:08:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f396cd28640 1 Processor -- start 2024-04-25T10:08:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f396cd28640 1 -- start start 2024-04-25T10:08:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f396cd28640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39680ce7a0 0x7f396814e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f396cd28640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f396815bc90 0x7f39680b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f396cd28640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39681496b0 0x7f3968149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f396cd28640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39680b4280 con 0x7f39681496b0 2024-04-25T10:08:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f396cd28640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39680b3db0 con 0x7f396815bc90 2024-04-25T10:08:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f396cd28640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39680b4400 con 0x7f39680ce7a0 2024-04-25T10:08:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f3966d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39681496b0 0x7f3968149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f3966d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39681496b0 0x7f3968149b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33356/0 (socket says 172.21.15.60:33356) 2024-04-25T10:08:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f3966d9d640 1 -- 172.21.15.60:0/770029939 learned_addr learned my addr 172.21.15.60:0/770029939 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f3966d9d640 1 -- 172.21.15.60:0/770029939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39680ce7a0 msgr2=0x7f396814e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f3966d9d640 1 --2- 172.21.15.60:0/770029939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39680ce7a0 0x7f396814e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f3966d9d640 1 -- 172.21.15.60:0/770029939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f396815bc90 msgr2=0x7f39680b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f395dd9b640 1 --2- 172.21.15.60:0/770029939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f396815bc90 0x7f39680b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f3966d9d640 1 --2- 172.21.15.60:0/770029939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f396815bc90 0x7f39680b0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f3966d9d640 1 -- 172.21.15.60:0/770029939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39681380f0 con 0x7f39681496b0 2024-04-25T10:08:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f3966d9d640 1 --2- 172.21.15.60:0/770029939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39681496b0 0x7f3968149b40 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f396000cad0 tx=0x7f396000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f395f7fe640 1 -- 172.21.15.60:0/770029939 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960017070 con 0x7f39681496b0 2024-04-25T10:08:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f395f7fe640 1 -- 172.21.15.60:0/770029939 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396000abd0 con 0x7f39681496b0 2024-04-25T10:08:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f396cd28640 1 -- 172.21.15.60:0/770029939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39681383d0 con 0x7f39681496b0 2024-04-25T10:08:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f395f7fe640 1 -- 172.21.15.60:0/770029939 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960012400 con 0x7f39681496b0 2024-04-25T10:08:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f396cd28640 1 -- 172.21.15.60:0/770029939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396814e010 con 0x7f39681496b0 2024-04-25T10:08:20.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.074+0000 7f395f7fe640 1 -- 172.21.15.60:0/770029939 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f396001a070 con 0x7f39681496b0 2024-04-25T10:08:20.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f395f7fe640 1 --2- 172.21.15.60:0/770029939 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3948066f90 0x7f3948069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f395f7fe640 1 -- 172.21.15.60:0/770029939 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3960084cc0 con 0x7f39681496b0 2024-04-25T10:08:20.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396659c640 1 --2- 172.21.15.60:0/770029939 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3948066f90 0x7f3948069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 Processor -- start 2024-04-25T10:08:20.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 -- start start 2024-04-25T10:08:20.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396659c640 1 --2- 172.21.15.60:0/770029939 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3948066f90 0x7f3948069450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f39580ad000 tx=0x7f3958004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c67b70 0x558927c67f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c1de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c1e340 0x558927c1e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558927b015f0 con 0x558927c68490 2024-04-25T10:08:20.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558927a68da0 con 0x558927c67b70 2024-04-25T10:08:20.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589279f1150 con 0x558927c1e340 2024-04-25T10:08:20.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f395dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c1de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f395dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c1de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33372/0 (socket says 172.21.15.60:33372) 2024-04-25T10:08:20.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f395dd9b640 1 -- 172.21.15.60:0/2078181681 learned_addr learned my addr 172.21.15.60:0/2078181681 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:20.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396659c640 1 --2- 172.21.15.60:0/2078181681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c67b70 0x558927c67f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f3966d9d640 1 --2- 172.21.15.60:0/2078181681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c1e340 0x558927c1e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f395dd9b640 1 -- 172.21.15.60:0/2078181681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c1e340 msgr2=0x558927c1e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f395dd9b640 1 --2- 172.21.15.60:0/2078181681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c1e340 0x558927c1e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f395dd9b640 1 -- 172.21.15.60:0/2078181681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c67b70 msgr2=0x558927c67f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f395dd9b640 1 --2- 172.21.15.60:0/2078181681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c67b70 0x558927c67f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f395dd9b640 1 -- 172.21.15.60:0/2078181681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558927a0aca0 con 0x558927c68490 2024-04-25T10:08:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f395dd9b640 1 --2- 172.21.15.60:0/2078181681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c1de00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3950009890 tx=0x7f3950009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f965db0d37e324 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f3947fff640 1 -- 172.21.15.60:0/2078181681 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950017020 con 0x558927c68490 2024-04-25T10:08:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f3947fff640 1 -- 172.21.15.60:0/2078181681 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395000e950 con 0x558927c68490 2024-04-25T10:08:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f3947fff640 1 -- 172.21.15.60:0/2078181681 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395000f040 con 0x558927c68490 2024-04-25T10:08:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 -- 172.21.15.60:0/2078181681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 msgr2=0x558927c1de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 --2- 172.21.15.60:0/2078181681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c1de00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3950009890 tx=0x7f3950009d50 comp rx=0 tx=0).stop 2024-04-25T10:08:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 -- 172.21.15.60:0/2078181681 shutdown_connections 2024-04-25T10:08:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 --2- 172.21.15.60:0/2078181681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c1e340 0x558927c1e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 --2- 172.21.15.60:0/2078181681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c1de00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 --2- 172.21.15.60:0/2078181681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c67b70 0x558927c67f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 -- 172.21.15.60:0/2078181681 >> 172.21.15.60:0/2078181681 conn(0x558927c202a0 msgr2=0x558927c182d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 -- 172.21.15.60:0/2078181681 shutdown_connections 2024-04-25T10:08:20.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.078+0000 7f396fe95000 1 -- 172.21.15.60:0/2078181681 wait complete. 2024-04-25T10:08:20.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 -- start start 2024-04-25T10:08:20.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 Processor -- start 2024-04-25T10:08:20.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c1e340 0x558927c6f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c67b70 0x558927c6f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c73fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558927a23990 con 0x558927c68490 2024-04-25T10:08:20.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558927a64e40 con 0x558927c1e340 2024-04-25T10:08:20.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589279f4900 con 0x558927c67b70 2024-04-25T10:08:20.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 10 client.0 _ll_register_callbacks cb 0x558927b1c1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:20.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f3966d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c73fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f3966d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c73fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33380/0 (socket says 172.21.15.60:33380) 2024-04-25T10:08:20.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f3966d9d640 1 -- 172.21.15.60:0/357573885 learned_addr learned my addr 172.21.15.60:0/357573885 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:20.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396659c640 1 --2- 172.21.15.60:0/357573885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c1e340 0x558927c6f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f395dd9b640 1 --2- 172.21.15.60:0/357573885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c67b70 0x558927c6f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f3966d9d640 1 -- 172.21.15.60:0/357573885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c67b70 msgr2=0x558927c6f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f3966d9d640 1 --2- 172.21.15.60:0/357573885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c67b70 0x558927c6f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f3966d9d640 1 -- 172.21.15.60:0/357573885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c1e340 msgr2=0x558927c6f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f3966d9d640 1 --2- 172.21.15.60:0/357573885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c1e340 0x558927c6f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f3966d9d640 1 -- 172.21.15.60:0/357573885 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558927a21630 con 0x558927c68490 2024-04-25T10:08:20.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f3966d9d640 1 --2- 172.21.15.60:0/357573885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c73fe0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f396000a4f0 tx=0x7f3960059860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac45971d687484ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f39457fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:08:20.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f39457fa640 1 -- 172.21.15.60:0/357573885 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396006d020 con 0x558927c68490 2024-04-25T10:08:20.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f39457fa640 1 -- 172.21.15.60:0/357573885 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3960004b70 con 0x558927c68490 2024-04-25T10:08:20.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 -- 172.21.15.60:0/357573885 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f395000e950 con 0x558927c68490 2024-04-25T10:08:20.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f39457fa640 1 -- 172.21.15.60:0/357573885 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3960004d70 con 0x558927c68490 2024-04-25T10:08:20.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 10 client.6468 fetch_fsmap learned FSMap version 28 2024-04-25T10:08:20.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 -- 172.21.15.60:0/357573885 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5589279f13a0 con 0x558927c68490 2024-04-25T10:08:20.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f39457fa640 1 -- 172.21.15.60:0/357573885 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395000e950 con 0x558927c68490 2024-04-25T10:08:20.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f39457fa640 1 -- 172.21.15.60:0/357573885 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f396007cb50 con 0x558927c68490 2024-04-25T10:08:20.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 10 client.6468 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:08:20.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 10 client.6468 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:08:20.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 20 client.6468 populate_metadata read hostname 'smithi060' 2024-04-25T10:08:20.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 --2- 172.21.15.60:0/357573885 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x558927c76cf0 0x558927c790e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 4 client.6468 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:20.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 -- 172.21.15.60:0/357573885 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558927a04a50 con 0x558927c76cf0 2024-04-25T10:08:20.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396659c640 1 --2- 172.21.15.60:0/357573885 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x558927c76cf0 0x558927c790e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396659c640 1 --2- 172.21.15.60:0/357573885 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x558927c76cf0 0x558927c790e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a18197dc802e6e18 server_cookie=208a4c9cad2edf6a in_seq=0 out_seq=0 2024-04-25T10:08:20.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f39457fa640 10 client.6468 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:20.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f39457fa640 1 -- 172.21.15.60:0/357573885 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558927a04a50 con 0x558927c76cf0 2024-04-25T10:08:20.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f39457fa640 10 client.6468 handle_command_reply: tid=0 2024-04-25T10:08:20.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 client.6468 shutdown 2024-04-25T10:08:20.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 -- 172.21.15.60:0/357573885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 msgr2=0x558927c73fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 --2- 172.21.15.60:0/357573885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c73fe0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f396000a4f0 tx=0x7f3960059860 comp rx=0 tx=0).stop 2024-04-25T10:08:20.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 -- 172.21.15.60:0/357573885 shutdown_connections 2024-04-25T10:08:20.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 --2- 172.21.15.60:0/357573885 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x558927c76cf0 0x558927c790e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 --2- 172.21.15.60:0/357573885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c73fe0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 --2- 172.21.15.60:0/357573885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c67b70 0x558927c6f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f39457fa640 0 client.6468 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:20.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 --2- 172.21.15.60:0/357573885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c1e340 0x558927c6f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 -- 172.21.15.60:0/357573885 >> 172.21.15.60:0/357573885 conn(0x558927c202a0 msgr2=0x558927c1f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 -- 172.21.15.60:0/357573885 shutdown_connections 2024-04-25T10:08:20.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 1 -- 172.21.15.60:0/357573885 wait complete. 2024-04-25T10:08:20.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.082+0000 7f396fe95000 20 client.6468 trim_cache size 0 max 16384 2024-04-25T10:08:20.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.098+0000 7f396fe95000 1 Processor -- start 2024-04-25T10:08:20.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.098+0000 7f396fe95000 1 -- start start 2024-04-25T10:08:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.098+0000 7f396fe95000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c1e340 0x558927c66d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.098+0000 7f396fe95000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c67b70 0x558927c67240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.098+0000 7f396fe95000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c780e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.098+0000 7f396fe95000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3968137cf0 con 0x558927c68490 2024-04-25T10:08:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.098+0000 7f396fe95000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396815ab10 con 0x558927c67b70 2024-04-25T10:08:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.098+0000 7f396fe95000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39681376f0 con 0x558927c1e340 2024-04-25T10:08:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f3966d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c780e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f3966d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c780e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33396/0 (socket says 172.21.15.60:33396) 2024-04-25T10:08:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f3966d9d640 1 -- 172.21.15.60:0/11768744 learned_addr learned my addr 172.21.15.60:0/11768744 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396659c640 1 --2- 172.21.15.60:0/11768744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c1e340 0x558927c66d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f395dd9b640 1 --2- 172.21.15.60:0/11768744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c67b70 0x558927c67240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f3966d9d640 1 -- 172.21.15.60:0/11768744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c1e340 msgr2=0x558927c66d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f3966d9d640 1 --2- 172.21.15.60:0/11768744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c1e340 0x558927c66d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f3966d9d640 1 -- 172.21.15.60:0/11768744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c67b70 msgr2=0x558927c67240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f3966d9d640 1 --2- 172.21.15.60:0/11768744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c67b70 0x558927c67240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f3966d9d640 1 -- 172.21.15.60:0/11768744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558927c682c0 con 0x558927c68490 2024-04-25T10:08:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f3966d9d640 1 --2- 172.21.15.60:0/11768744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c780e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3960084fe0 tx=0x7f3960058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c466c87d5e5a807 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f39477fe640 1 -- 172.21.15.60:0/11768744 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960072070 con 0x558927c68490 2024-04-25T10:08:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f39477fe640 1 -- 172.21.15.60:0/11768744 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3960083130 con 0x558927c68490 2024-04-25T10:08:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f39477fe640 1 -- 172.21.15.60:0/11768744 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396006d050 con 0x558927c68490 2024-04-25T10:08:20.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 -- 172.21.15.60:0/11768744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 msgr2=0x558927c780e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 --2- 172.21.15.60:0/11768744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c780e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3960084fe0 tx=0x7f3960058000 comp rx=0 tx=0).stop 2024-04-25T10:08:20.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 -- 172.21.15.60:0/11768744 shutdown_connections 2024-04-25T10:08:20.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 --2- 172.21.15.60:0/11768744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c68490 0x558927c780e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 --2- 172.21.15.60:0/11768744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c67b70 0x558927c67240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 --2- 172.21.15.60:0/11768744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c1e340 0x558927c66d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 -- 172.21.15.60:0/11768744 >> 172.21.15.60:0/11768744 conn(0x558927c202a0 msgr2=0x558927c182d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 -- 172.21.15.60:0/11768744 shutdown_connections 2024-04-25T10:08:20.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 -- 172.21.15.60:0/11768744 wait complete. 2024-04-25T10:08:20.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 -- start start 2024-04-25T10:08:20.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 Processor -- start 2024-04-25T10:08:20.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c1e340 0x558927c7ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c67b70 0x558927caeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c68490 0x558927caf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558927a6d3d0 con 0x558927c1e340 2024-04-25T10:08:20.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558927b24d50 con 0x558927c68490 2024-04-25T10:08:20.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558927a03980 con 0x558927c67b70 2024-04-25T10:08:20.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 10 client.0 _ll_register_callbacks cb 0x7f3950009120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:20.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c1e340 0x558927c7ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c1e340 0x558927c7ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33400/0 (socket says 172.21.15.60:33400) 2024-04-25T10:08:20.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396659c640 1 -- 172.21.15.60:0/3964996874 learned_addr learned my addr 172.21.15.60:0/3964996874 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:20.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396659c640 1 -- 172.21.15.60:0/3964996874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c67b70 msgr2=0x558927caeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396659c640 1 --2- 172.21.15.60:0/3964996874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c67b70 0x558927caeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396659c640 1 -- 172.21.15.60:0/3964996874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c68490 msgr2=0x558927caf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396659c640 1 --2- 172.21.15.60:0/3964996874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c68490 0x558927caf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396659c640 1 -- 172.21.15.60:0/3964996874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558927aea600 con 0x558927c1e340 2024-04-25T10:08:20.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396659c640 1 --2- 172.21.15.60:0/3964996874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c1e340 0x558927c7ad00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f395801c4e0 tx=0x7f395801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16225b99fb02af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f39457fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:08:20.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f39457fa640 1 -- 172.21.15.60:0/3964996874 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39580b0040 con 0x558927c1e340 2024-04-25T10:08:20.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f39457fa640 1 -- 172.21.15.60:0/3964996874 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39580c70c0 con 0x558927c1e340 2024-04-25T10:08:20.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 -- 172.21.15.60:0/3964996874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3960083130 con 0x558927c1e340 2024-04-25T10:08:20.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f39457fa640 1 -- 172.21.15.60:0/3964996874 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39580d7050 con 0x558927c1e340 2024-04-25T10:08:20.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 10 client.6474 fetch_fsmap learned FSMap version 28 2024-04-25T10:08:20.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 1 -- 172.21.15.60:0/3964996874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5589279fd9b0 con 0x558927c1e340 2024-04-25T10:08:20.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f39457fa640 1 -- 172.21.15.60:0/3964996874 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3960083130 con 0x558927c1e340 2024-04-25T10:08:20.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f39457fa640 1 -- 172.21.15.60:0/3964996874 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f39580da020 con 0x558927c1e340 2024-04-25T10:08:20.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 10 client.6474 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:08:20.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 10 client.6474 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:08:20.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.102+0000 7f396fe95000 20 client.6474 populate_metadata read hostname 'smithi060' 2024-04-25T10:08:20.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396fe95000 1 --2- 172.21.15.60:0/3964996874 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x558927c82b60 0x558927c82f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396fe95000 4 client.6474 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:08:20.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396fe95000 1 -- 172.21.15.60:0/3964996874 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558927a762e0 con 0x558927c82b60 2024-04-25T10:08:20.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f395dd9b640 1 --2- 172.21.15.60:0/3964996874 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x558927c82b60 0x558927c82f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f395dd9b640 1 --2- 172.21.15.60:0/3964996874 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x558927c82b60 0x558927c82f40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=564c1e1465668e8b server_cookie=ade1ae9d3f998fee in_seq=0 out_seq=0 2024-04-25T10:08:20.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f39457fa640 10 client.6474 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:20.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f39457fa640 1 -- 172.21.15.60:0/3964996874 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x558927a762e0 con 0x558927c82b60 2024-04-25T10:08:20.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f39457fa640 10 client.6474 handle_command_reply: tid=0 2024-04-25T10:08:20.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396fe95000 1 client.6474 shutdown 2024-04-25T10:08:20.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396fe95000 1 -- 172.21.15.60:0/3964996874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c1e340 msgr2=0x558927c7ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396fe95000 1 --2- 172.21.15.60:0/3964996874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c1e340 0x558927c7ad00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f395801c4e0 tx=0x7f395801ace0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396fe95000 1 -- 172.21.15.60:0/3964996874 shutdown_connections 2024-04-25T10:08:20.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396fe95000 1 --2- 172.21.15.60:0/3964996874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558927c68490 0x558927caf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396fe95000 1 --2- 172.21.15.60:0/3964996874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558927c67b70 0x558927caeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396fe95000 1 --2- 172.21.15.60:0/3964996874 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x558927c82b60 0x558927c82f40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396fe95000 1 --2- 172.21.15.60:0/3964996874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558927c1e340 0x558927c7ad00 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f39457fa640 0 client.6474 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:20.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396fe95000 1 -- 172.21.15.60:0/3964996874 >> 172.21.15.60:0/3964996874 conn(0x558927c202a0 msgr2=0x558927c189e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396fe95000 1 -- 172.21.15.60:0/3964996874 shutdown_connections 2024-04-25T10:08:20.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396fe95000 1 -- 172.21.15.60:0/3964996874 wait complete. 2024-04-25T10:08:20.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396fe95000 20 client.6474 trim_cache size 0 max 16384 2024-04-25T10:08:20.119 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:08:20.119 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "0ea93863-a079-45c4-abed-6e83540ea9fe", 2024-04-25T10:08:20.119 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T10:08:20.119 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6291, 2024-04-25T10:08:20.119 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T10:08:20.119 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T10:08:20.119 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T10:08:20.120 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 9.2433084989999994, 2024-04-25T10:08:20.120 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 28, 2024-04-25T10:08:20.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 58, 2024-04-25T10:08:20.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 58, 2024-04-25T10:08:20.120 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 14.245594468 2024-04-25T10:08:20.120 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:08:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396cd28640 1 -- 172.21.15.60:0/770029939 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3948066f90 msgr2=0x7f3948069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396cd28640 1 --2- 172.21.15.60:0/770029939 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3948066f90 0x7f3948069450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f39580ad000 tx=0x7f3958004740 comp rx=0 tx=0).stop 2024-04-25T10:08:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396cd28640 1 -- 172.21.15.60:0/770029939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39681496b0 msgr2=0x7f3968149b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396cd28640 1 --2- 172.21.15.60:0/770029939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39681496b0 0x7f3968149b40 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f396000cad0 tx=0x7f396000cf90 comp rx=0 tx=0).stop 2024-04-25T10:08:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396cd28640 1 -- 172.21.15.60:0/770029939 shutdown_connections 2024-04-25T10:08:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396cd28640 1 --2- 172.21.15.60:0/770029939 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3948066f90 0x7f3948069450 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396cd28640 1 --2- 172.21.15.60:0/770029939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39681496b0 0x7f3968149b40 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396cd28640 1 --2- 172.21.15.60:0/770029939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f396815bc90 0x7f39680b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396cd28640 1 --2- 172.21.15.60:0/770029939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39680ce7a0 0x7f396814e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396cd28640 1 -- 172.21.15.60:0/770029939 >> 172.21.15.60:0/770029939 conn(0x7f39680b12d0 msgr2=0x7f39680af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396cd28640 1 -- 172.21.15.60:0/770029939 shutdown_connections 2024-04-25T10:08:20.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.106+0000 7f396cd28640 1 -- 172.21.15.60:0/770029939 wait complete. 2024-04-25T10:08:20.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:08:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.230+0000 7fc09fb48640 1 Processor -- start 2024-04-25T10:08:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.230+0000 7fc09fb48640 1 -- start start 2024-04-25T10:08:20.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09fb48640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc09806ec30 0x7fc0980cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09fb48640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc0980b37e0 0x7fc0980b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09fb48640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc09806f130 con 0x7fc09815a7a0 2024-04-25T10:08:20.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09fb48640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0980b4190 con 0x7fc0980b37e0 2024-04-25T10:08:20.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09fb48640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0980cdaa0 con 0x7fc09806ec30 2024-04-25T10:08:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09e0e5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc0980b37e0 0x7fc0980b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09d8e4640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc09815a7a0 0x7fc098071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52044/0 (socket says 172.21.15.60:52044) 2024-04-25T10:08:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09e0e5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc0980b37e0 0x7fc0980b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33406/0 (socket says 172.21.15.60:33406) 2024-04-25T10:08:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09d8e4640 1 -- 172.21.15.60:0/914585184 learned_addr learned my addr 172.21.15.60:0/914585184 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09d0e3640 1 --2- 172.21.15.60:0/914585184 >> v2:172.21.15.100:3300/0 conn(0x7fc09806ec30 0x7fc0980cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09e0e5640 1 -- 172.21.15.60:0/914585184 >> v2:172.21.15.100:3300/0 conn(0x7fc09806ec30 msgr2=0x7fc0980cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09e0e5640 1 --2- 172.21.15.60:0/914585184 >> v2:172.21.15.100:3300/0 conn(0x7fc09806ec30 0x7fc0980cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09e0e5640 1 -- 172.21.15.60:0/914585184 >> v1:172.21.15.60:6789/0 conn(0x7fc09815a7a0 legacy=0x7fc098071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09e0e5640 1 -- 172.21.15.60:0/914585184 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09806f410 con 0x7fc0980b37e0 2024-04-25T10:08:20.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09c8e2640 1 -- 172.21.15.60:0/914585184 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820594029 0 0) 0x7fc09806f130 con 0x7fc09815a7a0 2024-04-25T10:08:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09e0e5640 1 --2- 172.21.15.60:0/914585184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0980b37e0 0x7fc0980b3c50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc090009d90 tx=0x7fc09001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb523967b9279d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.234+0000 7fc09c8e2640 1 -- 172.21.15.60:0/914585184 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0900b2070 con 0x7fc0980b37e0 2024-04-25T10:08:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09c8e2640 1 -- 172.21.15.60:0/914585184 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0900a5900 con 0x7fc0980b37e0 2024-04-25T10:08:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09c8e2640 1 -- 172.21.15.60:0/914585184 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0900a5cd0 con 0x7fc0980b37e0 2024-04-25T10:08:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 -- 172.21.15.60:0/914585184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0980b37e0 msgr2=0x7fc0980b3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 --2- 172.21.15.60:0/914585184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0980b37e0 0x7fc0980b3c50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc090009d90 tx=0x7fc09001c930 comp rx=0 tx=0).stop 2024-04-25T10:08:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 -- 172.21.15.60:0/914585184 shutdown_connections 2024-04-25T10:08:20.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 --2- 172.21.15.60:0/914585184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0980b37e0 0x7fc0980b3c50 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 --2- 172.21.15.60:0/914585184 >> v2:172.21.15.100:3300/0 conn(0x7fc09806ec30 0x7fc0980cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 -- 172.21.15.60:0/914585184 >> 172.21.15.60:0/914585184 conn(0x7fc0980b12d0 msgr2=0x7fc0980b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 -- 172.21.15.60:0/914585184 shutdown_connections 2024-04-25T10:08:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 -- 172.21.15.60:0/914585184 wait complete. 2024-04-25T10:08:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 Processor -- start 2024-04-25T10:08:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 -- start start 2024-04-25T10:08:20.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc09806ec30 0x7fc098147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0980b37e0 0x7fc098147300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc09815a7a0 0x7fc09815f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09806e8e0 con 0x7fc09806ec30 2024-04-25T10:08:20.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09806e760 con 0x7fc0980b37e0 2024-04-25T10:08:20.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09806d8b0 con 0x7fc09815a7a0 2024-04-25T10:08:20.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09d8e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc09806ec30 0x7fc098147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09d8e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc09806ec30 0x7fc098147da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33422/0 (socket says 172.21.15.60:33422) 2024-04-25T10:08:20.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09d8e4640 1 -- 172.21.15.60:0/161212738 learned_addr learned my addr 172.21.15.60:0/161212738 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:20.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09d0e3640 1 --2- 172.21.15.60:0/161212738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0980b37e0 0x7fc098147300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09d8e4640 1 -- 172.21.15.60:0/161212738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc09815a7a0 msgr2=0x7fc09815f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09e0e5640 1 --2- 172.21.15.60:0/161212738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc09815a7a0 0x7fc09815f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09d8e4640 1 --2- 172.21.15.60:0/161212738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc09815a7a0 0x7fc09815f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09d8e4640 1 -- 172.21.15.60:0/161212738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0980b37e0 msgr2=0x7fc098147300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09d8e4640 1 --2- 172.21.15.60:0/161212738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0980b37e0 0x7fc098147300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09d8e4640 1 -- 172.21.15.60:0/161212738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc098149190 con 0x7fc09806ec30 2024-04-25T10:08:20.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09d8e4640 1 --2- 172.21.15.60:0/161212738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc09806ec30 0x7fc098147da0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc09400c4b0 tx=0x7fc09400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc08e7fc640 1 -- 172.21.15.60:0/161212738 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094018070 con 0x7fc09806ec30 2024-04-25T10:08:20.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc08e7fc640 1 -- 172.21.15.60:0/161212738 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc094007500 con 0x7fc09806ec30 2024-04-25T10:08:20.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 -- 172.21.15.60:0/161212738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc098149470 con 0x7fc09806ec30 2024-04-25T10:08:20.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc08e7fc640 1 -- 172.21.15.60:0/161212738 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094004a30 con 0x7fc09806ec30 2024-04-25T10:08:20.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.238+0000 7fc09fb48640 1 -- 172.21.15.60:0/161212738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc098149a30 con 0x7fc09806ec30 2024-04-25T10:08:20.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.242+0000 7fc09fb48640 1 -- 172.21.15.60:0/161212738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc098074770 con 0x7fc09806ec30 2024-04-25T10:08:20.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.242+0000 7fc08e7fc640 1 -- 172.21.15.60:0/161212738 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc09401b070 con 0x7fc09806ec30 2024-04-25T10:08:20.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.242+0000 7fc08e7fc640 1 --2- 172.21.15.60:0/161212738 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc07c066f90 0x7fc07c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.242+0000 7fc08e7fc640 1 -- 172.21.15.60:0/161212738 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc094084220 con 0x7fc09806ec30 2024-04-25T10:08:20.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.242+0000 7fc09d0e3640 1 --2- 172.21.15.60:0/161212738 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc07c066f90 0x7fc07c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.242+0000 7fc09d0e3640 1 --2- 172.21.15.60:0/161212738 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc07c066f90 0x7fc07c069450 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc09814f380 tx=0x7fc088008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.246+0000 7fc08e7fc640 1 -- 172.21.15.60:0/161212738 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc094055230 con 0x7fc09806ec30 2024-04-25T10:08:20.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.434+0000 7fc09fb48640 1 -- 172.21.15.60:0/161212738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc098067c90 con 0x7fc09806ec30 2024-04-25T10:08:20.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.434+0000 7fc08e7fc640 1 -- 172.21.15.60:0/161212738 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc094054b30 con 0x7fc09806ec30 2024-04-25T10:08:20.443 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:08:20.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.438+0000 7fc09fb48640 1 -- 172.21.15.60:0/161212738 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc07c066f90 msgr2=0x7fc07c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.438+0000 7fc09fb48640 1 --2- 172.21.15.60:0/161212738 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc07c066f90 0x7fc07c069450 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc09814f380 tx=0x7fc088008040 comp rx=0 tx=0).stop 2024-04-25T10:08:20.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.438+0000 7fc09fb48640 1 -- 172.21.15.60:0/161212738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc09806ec30 msgr2=0x7fc098147da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.438+0000 7fc09fb48640 1 --2- 172.21.15.60:0/161212738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc09806ec30 0x7fc098147da0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc09400c4b0 tx=0x7fc09400c970 comp rx=0 tx=0).stop 2024-04-25T10:08:20.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.438+0000 7fc09fb48640 1 -- 172.21.15.60:0/161212738 shutdown_connections 2024-04-25T10:08:20.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.438+0000 7fc09fb48640 1 --2- 172.21.15.60:0/161212738 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc07c066f90 0x7fc07c069450 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.438+0000 7fc09fb48640 1 --2- 172.21.15.60:0/161212738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc09815a7a0 0x7fc09815f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.438+0000 7fc09fb48640 1 --2- 172.21.15.60:0/161212738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0980b37e0 0x7fc098147300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.438+0000 7fc09fb48640 1 --2- 172.21.15.60:0/161212738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc09806ec30 0x7fc098147da0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.438+0000 7fc09fb48640 1 -- 172.21.15.60:0/161212738 >> 172.21.15.60:0/161212738 conn(0x7fc0980b12d0 msgr2=0x7fc0980b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.438+0000 7fc09fb48640 1 -- 172.21.15.60:0/161212738 shutdown_connections 2024-04-25T10:08:20.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:20.438+0000 7fc09fb48640 1 -- 172.21.15.60:0/161212738 wait complete. 2024-04-25T10:08:20.461 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:08:20.461 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:08:20.461 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:08:20.461 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:08:20.462 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:20.462 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:08:20.462 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:08:20.462 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:08:20.462 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:20.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:20.463 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:08:20.470 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:08:20.470 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:08:20.470 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:08:20.471 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:20.471 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:08:20.471 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:20.471 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:08:20.471 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:20.471 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:08:20.472 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:08:20.472 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:20.472 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:08:20.472 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:08:20.472 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:20.472 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:08:20.473 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:08:20.473 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42357sec preferred_lft 42357sec 2024-04-25T10:08:20.473 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:08:20.473 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:20.473 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:08:20.473 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:20.474 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:08:20.474 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:08:20.474 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:20.474 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:08:20.474 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:20.475 INFO:teuthology.orchestra.run.smithi060.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:08:20.475 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:20.475 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:08:20.475 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:20.475 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:08:20.475 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:20.476 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:08:20.476 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:20.476 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:08:20.476 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:20.477 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:08:20.477 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:20.477 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:08:20.477 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:20.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:20.478 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:08:20.528 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:08:20.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:20.530 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:08:20.578 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:08:20.578 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:08:20.578 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:08:20.578 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:08:20.578 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:08:20.579 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:08:20.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:20.579 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:08:20.579 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:08:20.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:08:20.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:08:20.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:08:20.579 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:08:20.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:20.651 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:08:20.701 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:08:20.701 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:08:20.701 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:20.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:20.701 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T10:08:20.757 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:08:20.757 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:08:20.757 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:20.805 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:20.805 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:08:20.805 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:20.853 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:08:20.910 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:08:20.961 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:08:20.961 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:08:20.961 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:08:20.962 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:08:20.962 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:08:20.962 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:08:20.962 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:08:20.962 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:08:20.962 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:08:20.962 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:08:20.962 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:08:20.962 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:08:20.962 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:08:20.962 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:08:20.962 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:08:20.962 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:08:20.963 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:08:20.963 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:08:20.963 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:08:20.963 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:08:20.963 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:08:20.963 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:08:20.963 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:08:20.963 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:08:20.963 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:08:20.963 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:08:20.963 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:08:20.963 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:08:20.963 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:08:20.963 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:20.964 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:20.964 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:20.964 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:20.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:20.965 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:08:21.013 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:08:21.013 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:08:21.058 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:08:21.075 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:08:21.121 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:08:21.121 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:08:21.121 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:08:21.121 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:08:21.121 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:08:21.121 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:08:21.121 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:08:21.121 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:08:21.122 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:08:21.122 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:08:21.122 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:08:21.122 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:08:21.122 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:08:21.122 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:08:21.122 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:08:21.122 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:08:21.122 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:08:21.122 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:08:21.122 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:08:21.122 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:08:21.122 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:08:21.123 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:08:21.123 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:08:21.123 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:08:21.123 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:08:21.123 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:08:21.123 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:08:21.123 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:08:21.123 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:08:21.123 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:21.123 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:21.123 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:21.123 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:21.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:21.124 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:08:21.144 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2024-04-25T10:08:21.134+0000 7fe5a34a3580 -1 init, newargv = 0x562ec5ee2dd0 newargc=15 2024-04-25T10:08:21.144 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[40405]: starting ceph client 2024-04-25T10:08:21.150 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[40405]: starting fuse 2024-04-25T10:08:22.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:08:22.267 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:08:22.314 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:08:22.314 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:22.317 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:22.317 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:22.317 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:22.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:22.317 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:08:22.365 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:08:22.365 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:08:22.365 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:08:22.365 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:08:22.365 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:08:22.366 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:08:22.366 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:08:22.366 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:08:22.366 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:08:22.366 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:08:22.366 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:08:22.366 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:08:22.366 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:08:22.366 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:08:22.366 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:08:22.366 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:08:22.366 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:08:22.366 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:08:22.366 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:08:22.366 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:08:22.366 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:08:22.367 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:08:22.367 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:08:22.367 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:08:22.367 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:08:22.367 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:08:22.367 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:08:22.367 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2024-04-25T10:08:22.367 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:08:22.493 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.40405.asok 2024-04-25T10:08:22.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:08:22.507 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40405.asok 2024-04-25T10:08:22.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:22.507 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.40405.asok status 2024-04-25T10:08:22.660 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:08:22.660 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:08:22.660 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:08:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:08:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2024-04-25T10:08:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:08:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:08:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "40405", 2024-04-25T10:08:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:08:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:08:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:08:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6489, 2024-04-25T10:08:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:08:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:08:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:08:22.662 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 6489 2024-04-25T10:08:22.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:22.662 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:08:22.662 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:08:22.662 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:08:22.662 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1630893037 2024-04-25T10:08:22.662 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:22.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:22.662 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:08:22.662 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:08:22.662 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:08:22.662 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1630893037 2024-04-25T10:08:22.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:22.663 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.6489 192.168.0.1:0/1630893037", 2024-04-25T10:08:22.663 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/1630893037", 2024-04-25T10:08:22.663 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:08:22.663 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 28, 2024-04-25T10:08:22.663 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 58, 2024-04-25T10:08:22.663 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:08:22.663 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:08:22.663 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:08:22.663 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:08:22.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:22.676 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:08:22.676 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:22.682 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:08:22.682 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:22.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:22.682 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:22.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:22.742 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-04-25T10:08:22.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:22.801 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-04-25T10:08:22.856 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:08:22.856 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:08:22.856 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:08:22.856 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:08:22.856 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-04-25T10:08:22.856 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:08:22.970 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627777 2024-04-25T10:08:22.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:08:22.980 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:23.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.150+0000 7f169d83e640 1 Processor -- start 2024-04-25T10:08:23.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.150+0000 7f169d83e640 1 -- start start 2024-04-25T10:08:23.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f169d83e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f169806c0e0 0x7f16980d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f169d83e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16980d0930 0x7f16980d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f169d83e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f169815f5a0 con 0x7f1698099b10 2024-04-25T10:08:23.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f169d83e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f169806c570 con 0x7f169806c0e0 2024-04-25T10:08:23.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f169d83e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169815ea60 con 0x7f16980d0930 2024-04-25T10:08:23.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f168ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f169806c0e0 0x7f16980d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f168ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f169806c0e0 0x7f16980d03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33428/0 (socket says 172.21.15.60:33428) 2024-04-25T10:08:23.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f168ffff640 1 -- 172.21.15.60:0/4176128835 learned_addr learned my addr 172.21.15.60:0/4176128835 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:23.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f16977fe640 1 --2- 172.21.15.60:0/4176128835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16980d0930 0x7f16980d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f168ffff640 1 -- 172.21.15.60:0/4176128835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16980d0930 msgr2=0x7f16980d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f168ffff640 1 --2- 172.21.15.60:0/4176128835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16980d0930 0x7f16980d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f168ffff640 1 -- 172.21.15.60:0/4176128835 >> v1:172.21.15.100:6789/0 conn(0x7f1698099b10 legacy=0x7f1698099ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f168ffff640 1 -- 172.21.15.60:0/4176128835 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169815ed40 con 0x7f169806c0e0 2024-04-25T10:08:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f168ffff640 1 --2- 172.21.15.60:0/4176128835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f169806c0e0 0x7f16980d03f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1688009870 tx=0x7f168801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86258f78ff48b91f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f16967fc640 1 -- 172.21.15.60:0/4176128835 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16880aa600 con 0x7f169806c0e0 2024-04-25T10:08:23.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f16967fc640 1 -- 172.21.15.60:0/4176128835 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16880af070 con 0x7f169806c0e0 2024-04-25T10:08:23.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.154+0000 7f16967fc640 1 -- 172.21.15.60:0/4176128835 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16880aa9c0 con 0x7f169806c0e0 2024-04-25T10:08:23.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 -- 172.21.15.60:0/4176128835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f169806c0e0 msgr2=0x7f16980d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 --2- 172.21.15.60:0/4176128835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f169806c0e0 0x7f16980d03f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1688009870 tx=0x7f168801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 -- 172.21.15.60:0/4176128835 shutdown_connections 2024-04-25T10:08:23.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 --2- 172.21.15.60:0/4176128835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f169806c0e0 0x7f16980d03f0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 --2- 172.21.15.60:0/4176128835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16980d0930 0x7f16980d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 -- 172.21.15.60:0/4176128835 >> 172.21.15.60:0/4176128835 conn(0x7f16980b3320 msgr2=0x7f16980b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 -- 172.21.15.60:0/4176128835 shutdown_connections 2024-04-25T10:08:23.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 -- 172.21.15.60:0/4176128835 wait complete. 2024-04-25T10:08:23.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 Processor -- start 2024-04-25T10:08:23.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 -- start start 2024-04-25T10:08:23.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f169806c0e0 0x7f16980b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698099b10 0x7f169814f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16980d0930 0x7f1698152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1698161510 con 0x7f1698099b10 2024-04-25T10:08:23.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1698161390 con 0x7f169806c0e0 2024-04-25T10:08:23.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16980d1300 con 0x7f16980d0930 2024-04-25T10:08:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f168ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698099b10 0x7f169814f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f1696ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f169806c0e0 0x7f16980b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f168ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698099b10 0x7f169814f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33444/0 (socket says 172.21.15.60:33444) 2024-04-25T10:08:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f168ffff640 1 -- 172.21.15.60:0/1355766105 learned_addr learned my addr 172.21.15.60:0/1355766105 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f16977fe640 1 --2- 172.21.15.60:0/1355766105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16980d0930 0x7f1698152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f168ffff640 1 -- 172.21.15.60:0/1355766105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16980d0930 msgr2=0x7f1698152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f168ffff640 1 --2- 172.21.15.60:0/1355766105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16980d0930 0x7f1698152470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f168ffff640 1 -- 172.21.15.60:0/1355766105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f169806c0e0 msgr2=0x7f16980b7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f168ffff640 1 --2- 172.21.15.60:0/1355766105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f169806c0e0 0x7f16980b7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f168ffff640 1 -- 172.21.15.60:0/1355766105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16980b4af0 con 0x7f1698099b10 2024-04-25T10:08:23.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f168ffff640 1 --2- 172.21.15.60:0/1355766105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698099b10 0x7f169814f850 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f16880ac040 tx=0x7f168801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f1694ff9640 1 -- 172.21.15.60:0/1355766105 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16880bd5f0 con 0x7f1698099b10 2024-04-25T10:08:23.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f1694ff9640 1 -- 172.21.15.60:0/1355766105 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16880bd7b0 con 0x7f1698099b10 2024-04-25T10:08:23.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 -- 172.21.15.60:0/1355766105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16980b4dd0 con 0x7f1698099b10 2024-04-25T10:08:23.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f1694ff9640 1 -- 172.21.15.60:0/1355766105 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16880bdba0 con 0x7f1698099b10 2024-04-25T10:08:23.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.158+0000 7f169d83e640 1 -- 172.21.15.60:0/1355766105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16980b5390 con 0x7f1698099b10 2024-04-25T10:08:23.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.162+0000 7f169d83e640 1 -- 172.21.15.60:0/1355766105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1698074770 con 0x7f1698099b10 2024-04-25T10:08:23.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.162+0000 7f1694ff9640 1 -- 172.21.15.60:0/1355766105 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f16880a9070 con 0x7f1698099b10 2024-04-25T10:08:23.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.162+0000 7f1694ff9640 1 --2- 172.21.15.60:0/1355766105 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1678066f60 0x7f1678069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.162+0000 7f1694ff9640 1 -- 172.21.15.60:0/1355766105 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f168811f990 con 0x7f1698099b10 2024-04-25T10:08:23.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.162+0000 7f1696ffd640 1 --2- 172.21.15.60:0/1355766105 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1678066f60 0x7f1678069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.162+0000 7f1696ffd640 1 --2- 172.21.15.60:0/1355766105 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1678066f60 0x7f1678069420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1698153370 tx=0x7f1680009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.166+0000 7f1694ff9640 1 -- 172.21.15.60:0/1355766105 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16880c3060 con 0x7f1698099b10 2024-04-25T10:08:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.358+0000 7f169d83e640 1 -- 172.21.15.60:0/1355766105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1698074770 con 0x7f1698099b10 2024-04-25T10:08:23.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.358+0000 7f1694ff9640 1 -- 172.21.15.60:0/1355766105 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f1688016150 con 0x7f1698099b10 2024-04-25T10:08:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:23.367 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":28,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/380273391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":380273391},{"type":"v1","addr":"172.21.15.100:6836","nonce":380273391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5635,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2965977166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2965977166},{"type":"v1","addr":"172.21.15.100:6837","nonce":2965977166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1105028629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1105028629},{"type":"v1","addr":"172.21.15.60:6834","nonce":1105028629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:10.236590+0000","modified":"2024-04-25T10:08:12.229638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2718963870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2718963870},{"type":"v1","addr":"172.21.15.60:6835","nonce":2718963870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:08:23.367 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 28 2024-04-25T10:08:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.362+0000 7f169d83e640 1 -- 172.21.15.60:0/1355766105 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1678066f60 msgr2=0x7f1678069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.362+0000 7f169d83e640 1 --2- 172.21.15.60:0/1355766105 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1678066f60 0x7f1678069420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1698153370 tx=0x7f1680009040 comp rx=0 tx=0).stop 2024-04-25T10:08:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.362+0000 7f169d83e640 1 -- 172.21.15.60:0/1355766105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698099b10 msgr2=0x7f169814f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.362+0000 7f169d83e640 1 --2- 172.21.15.60:0/1355766105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698099b10 0x7f169814f850 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f16880ac040 tx=0x7f168801ace0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.362+0000 7f169d83e640 1 -- 172.21.15.60:0/1355766105 shutdown_connections 2024-04-25T10:08:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.362+0000 7f169d83e640 1 --2- 172.21.15.60:0/1355766105 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1678066f60 0x7f1678069420 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.362+0000 7f169d83e640 1 --2- 172.21.15.60:0/1355766105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16980d0930 0x7f1698152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.362+0000 7f169d83e640 1 --2- 172.21.15.60:0/1355766105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698099b10 0x7f169814f850 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.362+0000 7f169d83e640 1 --2- 172.21.15.60:0/1355766105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f169806c0e0 0x7f16980b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.362+0000 7f169d83e640 1 -- 172.21.15.60:0/1355766105 >> 172.21.15.60:0/1355766105 conn(0x7f16980b3320 msgr2=0x7f1698099ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.362+0000 7f169d83e640 1 -- 172.21.15.60:0/1355766105 shutdown_connections 2024-04-25T10:08:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.362+0000 7f169d83e640 1 -- 172.21.15.60:0/1355766105 wait complete. 2024-04-25T10:08:23.385 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:08:23.536 INFO:teuthology.orchestra.run.smithi060.stdout:{"message":"","return_code":0} 2024-04-25T10:08:23.537 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:08:23.537 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:08:23.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 Processor -- start 2024-04-25T10:08:23.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 -- start start 2024-04-25T10:08:23.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9dc003ac0 0x55c996cf0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c996d92140 0x55c996d90e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c996d913d0 0x55c996d016c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c996c649f0 con 0x55c996d913d0 2024-04-25T10:08:23.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c996c68950 con 0x55c996d92140 2024-04-25T10:08:23.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c996d958d0 con 0x7ff9dc003ac0 2024-04-25T10:08:23.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f001d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c996d913d0 0x55c996d016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f001d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c996d913d0 0x55c996d016c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33466/0 (socket says 172.21.15.60:33466) 2024-04-25T10:08:23.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f001d640 1 -- 172.21.15.60:0/2915979469 learned_addr learned my addr 172.21.15.60:0/2915979469 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:23.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9ef81c640 1 --2- 172.21.15.60:0/2915979469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9dc003ac0 0x55c996cf0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9ef01b640 1 --2- 172.21.15.60:0/2915979469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c996d92140 0x55c996d90e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f001d640 1 -- 172.21.15.60:0/2915979469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9dc003ac0 msgr2=0x55c996cf0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f001d640 1 --2- 172.21.15.60:0/2915979469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9dc003ac0 0x55c996cf0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f001d640 1 -- 172.21.15.60:0/2915979469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c996d92140 msgr2=0x55c996d90e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f001d640 1 --2- 172.21.15.60:0/2915979469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c996d92140 0x55c996d90e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f001d640 1 -- 172.21.15.60:0/2915979469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c996d97380 con 0x55c996d913d0 2024-04-25T10:08:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f001d640 1 --2- 172.21.15.60:0/2915979469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c996d913d0 0x55c996d016c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff9e801ec70 tx=0x55c996d0b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48250cbf503cbccf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9ee019640 1 -- 172.21.15.60:0/2915979469 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80bd070 con 0x55c996d913d0 2024-04-25T10:08:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9ee019640 1 -- 172.21.15.60:0/2915979469 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e80018a0 con 0x55c996d913d0 2024-04-25T10:08:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9ee019640 1 -- 172.21.15.60:0/2915979469 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80b3040 con 0x55c996d913d0 2024-04-25T10:08:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 -- 172.21.15.60:0/2915979469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c996d913d0 msgr2=0x55c996d016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 --2- 172.21.15.60:0/2915979469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c996d913d0 0x55c996d016c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff9e801ec70 tx=0x55c996d0b1e0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 -- 172.21.15.60:0/2915979469 shutdown_connections 2024-04-25T10:08:23.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 --2- 172.21.15.60:0/2915979469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c996d913d0 0x55c996d016c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 --2- 172.21.15.60:0/2915979469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c996d92140 0x55c996d90e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 --2- 172.21.15.60:0/2915979469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9dc003ac0 0x55c996cf0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 -- 172.21.15.60:0/2915979469 >> 172.21.15.60:0/2915979469 conn(0x55c996cec3d0 msgr2=0x55c996cbf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 -- 172.21.15.60:0/2915979469 shutdown_connections 2024-04-25T10:08:23.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.570+0000 7ff9f1cfe080 1 -- 172.21.15.60:0/2915979469 wait complete. 2024-04-25T10:08:23.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f1cfe080 1 Processor -- start 2024-04-25T10:08:23.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f1cfe080 1 -- start start 2024-04-25T10:08:23.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f1cfe080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c996d913d0 0x55c996d90460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f1cfe080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c996d92140 0x55c996d8d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f1cfe080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9dc003ac0 0x55c996d8e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f1cfe080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c996d97620 con 0x7ff9dc003ac0 2024-04-25T10:08:23.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f1cfe080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c996cbe8e0 con 0x55c996d913d0 2024-04-25T10:08:23.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f1cfe080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c996d978c0 con 0x55c996d92140 2024-04-25T10:08:23.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f001d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9dc003ac0 0x55c996d8e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f001d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9dc003ac0 0x55c996d8e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33468/0 (socket says 172.21.15.60:33468) 2024-04-25T10:08:23.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f001d640 1 -- 172.21.15.60:0/1404915958 learned_addr learned my addr 172.21.15.60:0/1404915958 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:23.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9ef81c640 1 --2- 172.21.15.60:0/1404915958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c996d913d0 0x55c996d90460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9ef01b640 1 --2- 172.21.15.60:0/1404915958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c996d92140 0x55c996d8d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f001d640 1 -- 172.21.15.60:0/1404915958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c996d92140 msgr2=0x55c996d8d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f001d640 1 --2- 172.21.15.60:0/1404915958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c996d92140 0x55c996d8d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f001d640 1 -- 172.21.15.60:0/1404915958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c996d913d0 msgr2=0x55c996d90460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f001d640 1 --2- 172.21.15.60:0/1404915958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c996d913d0 0x55c996d90460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f001d640 1 -- 172.21.15.60:0/1404915958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c996d78730 con 0x7ff9dc003ac0 2024-04-25T10:08:23.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f001d640 1 --2- 172.21.15.60:0/1404915958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9dc003ac0 0x55c996d8e6b0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff9e809ac30 tx=0x7ff9dc004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9ed017640 1 -- 172.21.15.60:0/1404915958 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80c9070 con 0x7ff9dc003ac0 2024-04-25T10:08:23.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9ed017640 1 -- 172.21.15.60:0/1404915958 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e80ae5d0 con 0x7ff9dc003ac0 2024-04-25T10:08:23.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f1cfe080 1 -- 172.21.15.60:0/1404915958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c996d789d0 con 0x7ff9dc003ac0 2024-04-25T10:08:23.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9ed017640 1 -- 172.21.15.60:0/1404915958 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80bd040 con 0x7ff9dc003ac0 2024-04-25T10:08:23.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9f1cfe080 1 -- 172.21.15.60:0/1404915958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c996d78f10 con 0x7ff9dc003ac0 2024-04-25T10:08:23.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.574+0000 7ff9ed017640 1 -- 172.21.15.60:0/1404915958 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9e80cc070 con 0x7ff9dc003ac0 2024-04-25T10:08:23.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9ed017640 1 --2- 172.21.15.60:0/1404915958 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff9d0065970 0x7ff9d0067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9ed017640 1 -- 172.21.15.60:0/1404915958 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9e8122270 con 0x7ff9dc003ac0 2024-04-25T10:08:23.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9f1cfe080 1 --2- 172.21.15.60:0/1404915958 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55c996d8d660 0x55c996d8f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9ef81c640 1 --2- 172.21.15.60:0/1404915958 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff9d0065970 0x7ff9d0067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9f1cfe080 1 -- 172.21.15.60:0/1404915958 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58) v9 -- 0x55c996d79650 con 0x55c996d8d660 2024-04-25T10:08:23.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9ef01b640 1 --2- 172.21.15.60:0/1404915958 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55c996d8d660 0x55c996d8f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9ef81c640 1 --2- 172.21.15.60:0/1404915958 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff9d0065970 0x7ff9d0067e30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0000f90 tx=0x7ff9e0003040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9ef01b640 1 --2- 172.21.15.60:0/1404915958 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55c996d8d660 0x55c996d8f100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9ef01b640 1 -- 172.21.15.60:0/1404915958 <== osd.5 v2:172.21.15.100:6800/3542120824 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7ff9d8009890 con 0x55c996d8d660 2024-04-25T10:08:23.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9f1cfe080 1 -- 172.21.15.60:0/1404915958 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55c996d8d660 msgr2=0x55c996d8f100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9f1cfe080 1 --2- 172.21.15.60:0/1404915958 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55c996d8d660 0x55c996d8f100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9f1cfe080 1 -- 172.21.15.60:0/1404915958 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff9d0065970 msgr2=0x7ff9d0067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9f1cfe080 1 --2- 172.21.15.60:0/1404915958 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff9d0065970 0x7ff9d0067e30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0000f90 tx=0x7ff9e0003040 comp rx=0 tx=0).stop 2024-04-25T10:08:23.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9f1cfe080 1 -- 172.21.15.60:0/1404915958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9dc003ac0 msgr2=0x55c996d8e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9f1cfe080 1 --2- 172.21.15.60:0/1404915958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9dc003ac0 0x55c996d8e6b0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff9e809ac30 tx=0x7ff9dc004130 comp rx=0 tx=0).stop 2024-04-25T10:08:23.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9f001d640 1 -- 172.21.15.60:0/1404915958 reap_dead start 2024-04-25T10:08:23.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9f1cfe080 1 -- 172.21.15.60:0/1404915958 shutdown_connections 2024-04-25T10:08:23.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9f1cfe080 1 -- 172.21.15.60:0/1404915958 >> 172.21.15.60:0/1404915958 conn(0x55c996cec3d0 msgr2=0x55c996cbdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9f1cfe080 1 -- 172.21.15.60:0/1404915958 shutdown_connections 2024-04-25T10:08:23.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.578+0000 7ff9f1cfe080 1 -- 172.21.15.60:0/1404915958 wait complete. 2024-04-25T10:08:23.591 INFO:teuthology.orchestra.run.smithi060.stdout:Xfiledir_test 2024-04-25T10:08:23.591 DEBUG:teuthology.orchestra.run.smithi060:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:08:23.680 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:08:23.681 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511627777, 2024-04-25T10:08:23.681 INFO:teuthology.orchestra.run.smithi060.stdout: "ancestors": [ 2024-04-25T10:08:23.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:23.681 INFO:teuthology.orchestra.run.smithi060.stdout: "dirino": 1099511627776, 2024-04-25T10:08:23.681 INFO:teuthology.orchestra.run.smithi060.stdout: "dname": "file", 2024-04-25T10:08:23.681 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 2 2024-04-25T10:08:23.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:08:23.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:23.681 INFO:teuthology.orchestra.run.smithi060.stdout: "dirino": 1, 2024-04-25T10:08:23.681 INFO:teuthology.orchestra.run.smithi060.stdout: "dname": "dir_test", 2024-04-25T10:08:23.681 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 4 2024-04-25T10:08:23.681 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:23.681 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-04-25T10:08:23.682 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 9, 2024-04-25T10:08:23.682 INFO:teuthology.orchestra.run.smithi060.stdout: "old_pools": [] 2024-04-25T10:08:23.682 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:08:23.682 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:23.683 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-04-25T10:08:23.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801cef8080 1 Processor -- start 2024-04-25T10:08:23.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801cef8080 1 -- start start 2024-04-25T10:08:23.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801cef8080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8008003640 0x55f607a4abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801cef8080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f607a64f80 0x55f607a4b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801cef8080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f607a605a0 0x55f607aeed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801cef8080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6079c29f0 con 0x7f8008003640 2024-04-25T10:08:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801cef8080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6079c6950 con 0x55f607a64f80 2024-04-25T10:08:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801cef8080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f607a4c4c0 con 0x55f607a605a0 2024-04-25T10:08:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801aa16640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8008003640 0x55f607a4abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801aa16640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8008003640 0x55f607a4abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33482/0 (socket says 172.21.15.60:33482) 2024-04-25T10:08:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801aa16640 1 -- 172.21.15.60:0/2120150491 learned_addr learned my addr 172.21.15.60:0/2120150491 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801a215640 1 --2- 172.21.15.60:0/2120150491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f607a64f80 0x55f607a4b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801aa16640 1 -- 172.21.15.60:0/2120150491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f607a605a0 msgr2=0x55f607aeed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801aa16640 1 --2- 172.21.15.60:0/2120150491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f607a605a0 0x55f607aeed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801aa16640 1 -- 172.21.15.60:0/2120150491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f607a64f80 msgr2=0x55f607a4b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801aa16640 1 --2- 172.21.15.60:0/2120150491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f607a64f80 0x55f607a4b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801aa16640 1 -- 172.21.15.60:0/2120150491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f607a1f2a0 con 0x7f8008003640 2024-04-25T10:08:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f801aa16640 1 --2- 172.21.15.60:0/2120150491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8008003640 0x55f607a4abf0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f801400a810 tx=0x7f8014005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cec90fccaab886 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f8019213640 1 -- 172.21.15.60:0/2120150491 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014017030 con 0x7f8008003640 2024-04-25T10:08:23.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f8019213640 1 -- 172.21.15.60:0/2120150491 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8014010a50 con 0x7f8008003640 2024-04-25T10:08:23.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.754+0000 7f8019213640 1 -- 172.21.15.60:0/2120150491 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014010c00 con 0x7f8008003640 2024-04-25T10:08:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 -- 172.21.15.60:0/2120150491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8008003640 msgr2=0x55f607a4abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 --2- 172.21.15.60:0/2120150491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8008003640 0x55f607a4abf0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f801400a810 tx=0x7f8014005d90 comp rx=0 tx=0).stop 2024-04-25T10:08:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 -- 172.21.15.60:0/2120150491 shutdown_connections 2024-04-25T10:08:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 --2- 172.21.15.60:0/2120150491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f607a605a0 0x55f607aeed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 --2- 172.21.15.60:0/2120150491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f607a64f80 0x55f607a4b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 --2- 172.21.15.60:0/2120150491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8008003640 0x55f607a4abf0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 -- 172.21.15.60:0/2120150491 >> 172.21.15.60:0/2120150491 conn(0x55f607a46e80 msgr2=0x55f607af2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 -- 172.21.15.60:0/2120150491 shutdown_connections 2024-04-25T10:08:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 -- 172.21.15.60:0/2120150491 wait complete. 2024-04-25T10:08:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 Processor -- start 2024-04-25T10:08:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 -- start start 2024-04-25T10:08:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f607a605a0 0x55f607a4b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f607a64f80 0x55f607a3ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8008003640 0x55f607ae4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f607a5fe60 con 0x55f607a605a0 2024-04-25T10:08:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f607a1f540 con 0x7f8008003640 2024-04-25T10:08:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f607a60100 con 0x55f607a64f80 2024-04-25T10:08:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801aa16640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f607a605a0 0x55f607a4b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801aa16640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f607a605a0 0x55f607a4b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33484/0 (socket says 172.21.15.60:33484) 2024-04-25T10:08:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801aa16640 1 -- 172.21.15.60:0/2883645887 learned_addr learned my addr 172.21.15.60:0/2883645887 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801a215640 1 --2- 172.21.15.60:0/2883645887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f607a64f80 0x55f607a3ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801aa16640 1 -- 172.21.15.60:0/2883645887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f607a64f80 msgr2=0x55f607a3ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801b217640 1 --2- 172.21.15.60:0/2883645887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8008003640 0x55f607ae4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801aa16640 1 --2- 172.21.15.60:0/2883645887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f607a64f80 0x55f607a3ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801aa16640 1 -- 172.21.15.60:0/2883645887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8008003640 msgr2=0x55f607ae4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801aa16640 1 --2- 172.21.15.60:0/2883645887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8008003640 0x55f607ae4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801aa16640 1 -- 172.21.15.60:0/2883645887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f607a3f2d0 con 0x55f607a605a0 2024-04-25T10:08:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801aa16640 1 --2- 172.21.15.60:0/2883645887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f607a605a0 0x55f607a4b100 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8014007010 tx=0x7f8014003340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f8013fff640 1 -- 172.21.15.60:0/2883645887 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014013d00 con 0x55f607a605a0 2024-04-25T10:08:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f8013fff640 1 -- 172.21.15.60:0/2883645887 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801401c410 con 0x55f607a605a0 2024-04-25T10:08:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 -- 172.21.15.60:0/2883645887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f607a3f570 con 0x55f607a605a0 2024-04-25T10:08:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f8013fff640 1 -- 172.21.15.60:0/2883645887 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801401c800 con 0x55f607a605a0 2024-04-25T10:08:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.758+0000 7f801cef8080 1 -- 172.21.15.60:0/2883645887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f607a3fab0 con 0x55f607a605a0 2024-04-25T10:08:23.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.762+0000 7f8013fff640 1 -- 172.21.15.60:0/2883645887 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f801401a030 con 0x55f607a605a0 2024-04-25T10:08:23.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.762+0000 7f8013fff640 1 --2- 172.21.15.60:0/2883645887 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ffc065970 0x7f7ffc067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.762+0000 7f8013fff640 1 -- 172.21.15.60:0/2883645887 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80140858f0 con 0x55f607a605a0 2024-04-25T10:08:23.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.762+0000 7f801cef8080 1 --2- 172.21.15.60:0/2883645887 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55f607b88e10 0x55f607b94270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.762+0000 7f801cef8080 1 -- 172.21.15.60:0/2883645887 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58) v9 -- 0x55f607ae4a70 con 0x55f607b88e10 2024-04-25T10:08:23.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.762+0000 7f801a215640 1 --2- 172.21.15.60:0/2883645887 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ffc065970 0x7f7ffc067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.762+0000 7f801b217640 1 --2- 172.21.15.60:0/2883645887 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55f607b88e10 0x55f607b94270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.762+0000 7f801b217640 1 --2- 172.21.15.60:0/2883645887 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55f607b88e10 0x55f607b94270 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.762+0000 7f801a215640 1 --2- 172.21.15.60:0/2883645887 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ffc065970 0x7f7ffc067e30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8004001bb0 tx=0x7f8004001b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.766+0000 7f801b217640 1 -- 172.21.15.60:0/2883645887 <== osd.5 v2:172.21.15.100:6800/3542120824 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v58'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f800c0b9420 con 0x55f607b88e10 2024-04-25T10:08:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.766+0000 7f801cef8080 1 -- 172.21.15.60:0/2883645887 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55f607b88e10 msgr2=0x55f607b94270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.766+0000 7f801cef8080 1 --2- 172.21.15.60:0/2883645887 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55f607b88e10 0x55f607b94270 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.766+0000 7f801cef8080 1 -- 172.21.15.60:0/2883645887 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ffc065970 msgr2=0x7f7ffc067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.766+0000 7f801cef8080 1 --2- 172.21.15.60:0/2883645887 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ffc065970 0x7f7ffc067e30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8004001bb0 tx=0x7f8004001b20 comp rx=0 tx=0).stop 2024-04-25T10:08:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.766+0000 7f801cef8080 1 -- 172.21.15.60:0/2883645887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f607a605a0 msgr2=0x55f607a4b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.766+0000 7f801cef8080 1 --2- 172.21.15.60:0/2883645887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f607a605a0 0x55f607a4b100 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8014007010 tx=0x7f8014003340 comp rx=0 tx=0).stop 2024-04-25T10:08:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.766+0000 7f801b217640 1 -- 172.21.15.60:0/2883645887 reap_dead start 2024-04-25T10:08:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.766+0000 7f801cef8080 1 -- 172.21.15.60:0/2883645887 shutdown_connections 2024-04-25T10:08:23.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.766+0000 7f801cef8080 1 -- 172.21.15.60:0/2883645887 >> 172.21.15.60:0/2883645887 conn(0x55f607a46e80 msgr2=0x55f607a5bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.766+0000 7f801cef8080 1 -- 172.21.15.60:0/2883645887 shutdown_connections 2024-04-25T10:08:23.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.766+0000 7f801cef8080 1 -- 172.21.15.60:0/2883645887 wait complete. 2024-04-25T10:08:23.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:23.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-04-25T10:08:23.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.942+0000 7f82131b1640 1 Processor -- start 2024-04-25T10:08:23.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.942+0000 7f82131b1640 1 -- start start 2024-04-25T10:08:23.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.942+0000 7f82131b1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f820c144e40 0x7f820c0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.942+0000 7f82131b1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f820c071980 0x7f820c0b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.942+0000 7f82131b1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820c0cf810 con 0x7f820c0cf0a0 2024-04-25T10:08:23.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.942+0000 7f82131b1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820c0b3060 con 0x7f820c144e40 2024-04-25T10:08:23.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.942+0000 7f82131b1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820c0cfaf0 con 0x7f820c071980 2024-04-25T10:08:23.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.942+0000 7f8210f4d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f820c144e40 0x7f820c0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.942+0000 7f8210f4d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f820c144e40 0x7f820c0ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33494/0 (socket says 172.21.15.60:33494) 2024-04-25T10:08:23.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.942+0000 7f8210f4d640 1 -- 172.21.15.60:0/3082959176 learned_addr learned my addr 172.21.15.60:0/3082959176 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:23.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.942+0000 7f821174e640 1 --2- 172.21.15.60:0/3082959176 >> v2:172.21.15.100:3300/0 conn(0x7f820c071980 0x7f820c0b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.942+0000 7f8210f4d640 1 -- 172.21.15.60:0/3082959176 >> v2:172.21.15.100:3300/0 conn(0x7f820c071980 msgr2=0x7f820c0b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.942+0000 7f8210f4d640 1 --2- 172.21.15.60:0/3082959176 >> v2:172.21.15.100:3300/0 conn(0x7f820c071980 0x7f820c0b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.942+0000 7f8210f4d640 1 -- 172.21.15.60:0/3082959176 >> v1:172.21.15.100:6789/0 conn(0x7f820c0cf0a0 legacy=0x7f820c071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.942+0000 7f8210f4d640 1 -- 172.21.15.60:0/3082959176 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c0cfdd0 con 0x7f820c144e40 2024-04-25T10:08:23.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.946+0000 7f8210f4d640 1 --2- 172.21.15.60:0/3082959176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f820c144e40 0x7f820c0ccc60 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f820c0710c0 tx=0x7f820401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4994e6d8772cd628 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.946+0000 7f820b7fe640 1 -- 172.21.15.60:0/3082959176 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040a6040 con 0x7f820c144e40 2024-04-25T10:08:23.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.946+0000 7f820b7fe640 1 -- 172.21.15.60:0/3082959176 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82040af070 con 0x7f820c144e40 2024-04-25T10:08:23.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.946+0000 7f820b7fe640 1 -- 172.21.15.60:0/3082959176 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040aa860 con 0x7f820c144e40 2024-04-25T10:08:23.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.946+0000 7f82131b1640 1 -- 172.21.15.60:0/3082959176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f820c144e40 msgr2=0x7f820c0ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.946+0000 7f82131b1640 1 --2- 172.21.15.60:0/3082959176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f820c144e40 0x7f820c0ccc60 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f820c0710c0 tx=0x7f820401c4c0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.946+0000 7f82131b1640 1 -- 172.21.15.60:0/3082959176 shutdown_connections 2024-04-25T10:08:23.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.946+0000 7f82131b1640 1 --2- 172.21.15.60:0/3082959176 >> v2:172.21.15.100:3300/0 conn(0x7f820c071980 0x7f820c0b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.946+0000 7f82131b1640 1 --2- 172.21.15.60:0/3082959176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f820c144e40 0x7f820c0ccc60 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.946+0000 7f82131b1640 1 -- 172.21.15.60:0/3082959176 >> 172.21.15.60:0/3082959176 conn(0x7f820c066fa0 msgr2=0x7f820c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.946+0000 7f82131b1640 1 -- 172.21.15.60:0/3082959176 shutdown_connections 2024-04-25T10:08:23.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.946+0000 7f82131b1640 1 -- 172.21.15.60:0/3082959176 wait complete. 2024-04-25T10:08:23.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f82131b1640 1 Processor -- start 2024-04-25T10:08:23.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f82131b1640 1 -- start start 2024-04-25T10:08:23.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f82131b1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f820c071980 0x7f820c0ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f82131b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f820c0cf0a0 0x7f820c06e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f82131b1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f820c144e40 0x7f820c15a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f82131b1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0b45d0 con 0x7f820c144e40 2024-04-25T10:08:23.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f82131b1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0b4450 con 0x7f820c0cf0a0 2024-04-25T10:08:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f82131b1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0b4750 con 0x7f820c071980 2024-04-25T10:08:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f821174e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f820c144e40 0x7f820c15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f821174e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f820c144e40 0x7f820c15a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33508/0 (socket says 172.21.15.60:33508) 2024-04-25T10:08:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f821174e640 1 -- 172.21.15.60:0/2266484669 learned_addr learned my addr 172.21.15.60:0/2266484669 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f820bfff640 1 --2- 172.21.15.60:0/2266484669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f820c0cf0a0 0x7f820c06e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f8210f4d640 1 --2- 172.21.15.60:0/2266484669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f820c071980 0x7f820c0ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f821174e640 1 -- 172.21.15.60:0/2266484669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f820c071980 msgr2=0x7f820c0ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f821174e640 1 --2- 172.21.15.60:0/2266484669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f820c071980 0x7f820c0ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f821174e640 1 -- 172.21.15.60:0/2266484669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f820c0cf0a0 msgr2=0x7f820c06e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f821174e640 1 --2- 172.21.15.60:0/2266484669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f820c0cf0a0 0x7f820c06e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f821174e640 1 -- 172.21.15.60:0/2266484669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c0c9400 con 0x7f820c144e40 2024-04-25T10:08:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f821174e640 1 --2- 172.21.15.60:0/2266484669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f820c144e40 0x7f820c15a680 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00ef90 tx=0x7f81fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f82097fa640 1 -- 172.21.15.60:0/2266484669 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc018020 con 0x7f820c144e40 2024-04-25T10:08:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f82097fa640 1 -- 172.21.15.60:0/2266484669 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc014480 con 0x7f820c144e40 2024-04-25T10:08:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f82131b1640 1 -- 172.21.15.60:0/2266484669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820c0c96e0 con 0x7f820c144e40 2024-04-25T10:08:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f82097fa640 1 -- 172.21.15.60:0/2266484669 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc014870 con 0x7f820c144e40 2024-04-25T10:08:23.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f82131b1640 1 -- 172.21.15.60:0/2266484669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c0c9ca0 con 0x7f820c144e40 2024-04-25T10:08:23.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.950+0000 7f82097fa640 1 -- 172.21.15.60:0/2266484669 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f81fc01b020 con 0x7f820c144e40 2024-04-25T10:08:23.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f82097fa640 1 --2- 172.21.15.60:0/2266484669 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81f8066f60 0x7f81f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f82097fa640 1 -- 172.21.15.60:0/2266484669 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81fc085db0 con 0x7f820c144e40 2024-04-25T10:08:23.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f8210f4d640 1 --2- 172.21.15.60:0/2266484669 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81f8066f60 0x7f81f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 Processor -- start 2024-04-25T10:08:23.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 -- start start 2024-04-25T10:08:23.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f8210f4d640 1 --2- 172.21.15.60:0/2266484669 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81f8066f60 0x7f81f8069420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f82040ae000 tx=0x7f82040a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e247590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e247ad0 0x56093e22b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e22bf10 0x56093e22c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093e04afd0 con 0x56093e2471b0 2024-04-25T10:08:23.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093dfd3170 con 0x56093e247ad0 2024-04-25T10:08:23.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56093e047070 con 0x56093e22bf10 2024-04-25T10:08:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f8210f4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e247590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f8210f4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e247590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33512/0 (socket says 172.21.15.60:33512) 2024-04-25T10:08:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f8210f4d640 1 -- 172.21.15.60:0/539286644 learned_addr learned my addr 172.21.15.60:0/539286644 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821174e640 1 --2- 172.21.15.60:0/539286644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e22bf10 0x56093e22c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f8210f4d640 1 -- 172.21.15.60:0/539286644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e22bf10 msgr2=0x56093e22c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f8210f4d640 1 --2- 172.21.15.60:0/539286644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e22bf10 0x56093e22c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f820bfff640 1 --2- 172.21.15.60:0/539286644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e247ad0 0x56093e22b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f8210f4d640 1 -- 172.21.15.60:0/539286644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e247ad0 msgr2=0x56093e22b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f8210f4d640 1 --2- 172.21.15.60:0/539286644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e247ad0 0x56093e22b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f8210f4d640 1 -- 172.21.15.60:0/539286644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56093e091d20 con 0x56093e2471b0 2024-04-25T10:08:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f8210f4d640 1 --2- 172.21.15.60:0/539286644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e247590 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f82040ac040 tx=0x7f8204004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bddfa19453a4a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f81f27fc640 1 -- 172.21.15.60:0/539286644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040c9680 con 0x56093e2471b0 2024-04-25T10:08:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f81f27fc640 1 -- 172.21.15.60:0/539286644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82040af040 con 0x56093e2471b0 2024-04-25T10:08:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f81f27fc640 1 -- 172.21.15.60:0/539286644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040c9a20 con 0x56093e2471b0 2024-04-25T10:08:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 -- 172.21.15.60:0/539286644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 msgr2=0x56093e247590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 --2- 172.21.15.60:0/539286644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e247590 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f82040ac040 tx=0x7f8204004740 comp rx=0 tx=0).stop 2024-04-25T10:08:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 -- 172.21.15.60:0/539286644 shutdown_connections 2024-04-25T10:08:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 --2- 172.21.15.60:0/539286644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e22bf10 0x56093e22c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 --2- 172.21.15.60:0/539286644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e247ad0 0x56093e22b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 --2- 172.21.15.60:0/539286644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e247590 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 -- 172.21.15.60:0/539286644 >> 172.21.15.60:0/539286644 conn(0x56093e203710 msgr2=0x56093e203af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 -- 172.21.15.60:0/539286644 shutdown_connections 2024-04-25T10:08:23.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 -- 172.21.15.60:0/539286644 wait complete. 2024-04-25T10:08:23.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.954+0000 7f821631e000 1 -- start start 2024-04-25T10:08:23.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 Processor -- start 2024-04-25T10:08:23.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e22bf10 0x56093e24e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e24ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e247ad0 0x56093e22c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093e108c10 con 0x56093e2471b0 2024-04-25T10:08:23.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093e005680 con 0x56093e247ad0 2024-04-25T10:08:23.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56093e04f600 con 0x56093e22bf10 2024-04-25T10:08:23.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 10 client.0 _ll_register_callbacks cb 0x56093e100080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:23.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f820bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e24ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f820bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e24ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33520/0 (socket says 172.21.15.60:33520) 2024-04-25T10:08:23.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f820bfff640 1 -- 172.21.15.60:0/1753685983 learned_addr learned my addr 172.21.15.60:0/1753685983 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:23.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821174e640 1 --2- 172.21.15.60:0/1753685983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e247ad0 0x56093e22c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f8210f4d640 1 --2- 172.21.15.60:0/1753685983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e22bf10 0x56093e24e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f820bfff640 1 -- 172.21.15.60:0/1753685983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e22bf10 msgr2=0x56093e24e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f820bfff640 1 --2- 172.21.15.60:0/1753685983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e22bf10 0x56093e24e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f820bfff640 1 -- 172.21.15.60:0/1753685983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e247ad0 msgr2=0x56093e22c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f820bfff640 1 --2- 172.21.15.60:0/1753685983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e247ad0 0x56093e22c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f820bfff640 1 -- 172.21.15.60:0/1753685983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56093e022d30 con 0x56093e2471b0 2024-04-25T10:08:23.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f820bfff640 1 --2- 172.21.15.60:0/1753685983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e24ee70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8200003af0 tx=0x7f8200001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63679794be847ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f81eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:08:23.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f81eb7fe640 1 -- 172.21.15.60:0/1753685983 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820001b060 con 0x56093e2471b0 2024-04-25T10:08:23.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f81eb7fe640 1 -- 172.21.15.60:0/1753685983 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8200016b10 con 0x56093e2471b0 2024-04-25T10:08:23.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 -- 172.21.15.60:0/1753685983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82040af040 con 0x56093e2471b0 2024-04-25T10:08:23.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f81eb7fe640 1 -- 172.21.15.60:0/1753685983 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8200016d10 con 0x56093e2471b0 2024-04-25T10:08:23.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 10 client.6525 fetch_fsmap learned FSMap version 28 2024-04-25T10:08:23.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 -- 172.21.15.60:0/1753685983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x56093dfd33c0 con 0x56093e2471b0 2024-04-25T10:08:23.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f81eb7fe640 1 -- 172.21.15.60:0/1753685983 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82040af040 con 0x56093e2471b0 2024-04-25T10:08:23.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f81eb7fe640 1 -- 172.21.15.60:0/1753685983 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f82000067e0 con 0x56093e2471b0 2024-04-25T10:08:23.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 10 client.6525 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:08:23.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 10 client.6525 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:23.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 20 client.6525 populate_metadata read hostname 'smithi060' 2024-04-25T10:08:23.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 --2- 172.21.15.60:0/1753685983 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x56093e22d060 0x56093e22d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 4 client.6525 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:23.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 -- 172.21.15.60:0/1753685983 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56093dfe6e00 con 0x56093e22d060 2024-04-25T10:08:23.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f8210f4d640 1 --2- 172.21.15.60:0/1753685983 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x56093e22d060 0x56093e22d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f8210f4d640 1 --2- 172.21.15.60:0/1753685983 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x56093e22d060 0x56093e22d470 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a19a0e7e8bf59f05 server_cookie=5a817941ea7b3683 in_seq=0 out_seq=0 2024-04-25T10:08:23.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f81eb7fe640 10 client.6525 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:23.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f81eb7fe640 1 -- 172.21.15.60:0/1753685983 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56093dfe6e00 con 0x56093e22d060 2024-04-25T10:08:23.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f81eb7fe640 10 client.6525 handle_command_reply: tid=0 2024-04-25T10:08:23.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 client.6525 shutdown 2024-04-25T10:08:23.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 -- 172.21.15.60:0/1753685983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 msgr2=0x56093e24ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 --2- 172.21.15.60:0/1753685983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e24ee70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8200003af0 tx=0x7f8200001250 comp rx=0 tx=0).stop 2024-04-25T10:08:23.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 -- 172.21.15.60:0/1753685983 shutdown_connections 2024-04-25T10:08:23.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 --2- 172.21.15.60:0/1753685983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e247ad0 0x56093e22c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 --2- 172.21.15.60:0/1753685983 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x56093e22d060 0x56093e22d470 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 --2- 172.21.15.60:0/1753685983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e24ee70 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 --2- 172.21.15.60:0/1753685983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e22bf10 0x56093e24e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 -- 172.21.15.60:0/1753685983 >> 172.21.15.60:0/1753685983 conn(0x56093e203710 msgr2=0x56093e1fb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f81eb7fe640 0 client.6525 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:23.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 -- 172.21.15.60:0/1753685983 shutdown_connections 2024-04-25T10:08:23.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 1 -- 172.21.15.60:0/1753685983 wait complete. 2024-04-25T10:08:23.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.958+0000 7f821631e000 20 client.6525 trim_cache size 0 max 16384 2024-04-25T10:08:23.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.974+0000 7f821631e000 1 Processor -- start 2024-04-25T10:08:23.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.974+0000 7f821631e000 1 -- start start 2024-04-25T10:08:23.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.974+0000 7f821631e000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e22bf10 0x56093e1fdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.974+0000 7f821631e000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e2471b0 0x56093e1fe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.974+0000 7f821631e000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e247ad0 0x56093e1fb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.974+0000 7f821631e000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c202ac0 con 0x56093e247ad0 2024-04-25T10:08:23.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.974+0000 7f821631e000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c201cb0 con 0x56093e22bf10 2024-04-25T10:08:23.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.974+0000 7f821631e000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c140c60 con 0x56093e2471b0 2024-04-25T10:08:23.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.974+0000 7f821174e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e247ad0 0x56093e1fb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.974+0000 7f821174e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e247ad0 0x56093e1fb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33524/0 (socket says 172.21.15.60:33524) 2024-04-25T10:08:23.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.974+0000 7f821174e640 1 -- 172.21.15.60:0/2976025540 learned_addr learned my addr 172.21.15.60:0/2976025540 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:23.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f820bfff640 1 --2- 172.21.15.60:0/2976025540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e2471b0 0x56093e1fe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821174e640 1 -- 172.21.15.60:0/2976025540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e2471b0 msgr2=0x56093e1fe150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821174e640 1 --2- 172.21.15.60:0/2976025540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e2471b0 0x56093e1fe150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821174e640 1 -- 172.21.15.60:0/2976025540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e22bf10 msgr2=0x56093e1fdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f8210f4d640 1 --2- 172.21.15.60:0/2976025540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e22bf10 0x56093e1fdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821174e640 1 --2- 172.21.15.60:0/2976025540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e22bf10 0x56093e1fdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821174e640 1 -- 172.21.15.60:0/2976025540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56093e22c660 con 0x56093e247ad0 2024-04-25T10:08:23.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821174e640 1 --2- 172.21.15.60:0/2976025540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e247ad0 0x56093e1fb0c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f81fc05e8a0 tx=0x7f81fc056770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9d6d72de875286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f81f1ffb640 1 -- 172.21.15.60:0/2976025540 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc082070 con 0x56093e247ad0 2024-04-25T10:08:23.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f81f1ffb640 1 -- 172.21.15.60:0/2976025540 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc05edd0 con 0x56093e247ad0 2024-04-25T10:08:23.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f81f1ffb640 1 -- 172.21.15.60:0/2976025540 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc052de0 con 0x56093e247ad0 2024-04-25T10:08:23.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 -- 172.21.15.60:0/2976025540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e247ad0 msgr2=0x56093e1fb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 --2- 172.21.15.60:0/2976025540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e247ad0 0x56093e1fb0c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f81fc05e8a0 tx=0x7f81fc056770 comp rx=0 tx=0).stop 2024-04-25T10:08:23.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 -- 172.21.15.60:0/2976025540 shutdown_connections 2024-04-25T10:08:23.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 --2- 172.21.15.60:0/2976025540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e247ad0 0x56093e1fb0c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 --2- 172.21.15.60:0/2976025540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e2471b0 0x56093e1fe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 --2- 172.21.15.60:0/2976025540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e22bf10 0x56093e1fdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 -- 172.21.15.60:0/2976025540 >> 172.21.15.60:0/2976025540 conn(0x56093e203710 msgr2=0x56093e22af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 -- 172.21.15.60:0/2976025540 shutdown_connections 2024-04-25T10:08:23.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 -- 172.21.15.60:0/2976025540 wait complete. 2024-04-25T10:08:23.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 -- start start 2024-04-25T10:08:23.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 Processor -- start 2024-04-25T10:08:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e22bf10 0x56093e2a7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e2a7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e247ad0 0x56093e262fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093e001e70 con 0x56093e2471b0 2024-04-25T10:08:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c141260 con 0x56093e22bf10 2024-04-25T10:08:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56093dff1d40 con 0x56093e247ad0 2024-04-25T10:08:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 10 client.0 _ll_register_callbacks cb 0x56093dfe8b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f820bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e2a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f820bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e2a7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33534/0 (socket says 172.21.15.60:33534) 2024-04-25T10:08:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f820bfff640 1 -- 172.21.15.60:0/2077701532 learned_addr learned my addr 172.21.15.60:0/2077701532 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f820bfff640 1 -- 172.21.15.60:0/2077701532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e247ad0 msgr2=0x56093e262fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f820bfff640 1 --2- 172.21.15.60:0/2077701532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e247ad0 0x56093e262fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f820bfff640 1 -- 172.21.15.60:0/2077701532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e22bf10 msgr2=0x56093e2a7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f8210f4d640 1 --2- 172.21.15.60:0/2077701532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e22bf10 0x56093e2a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f820bfff640 1 --2- 172.21.15.60:0/2077701532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e22bf10 0x56093e2a7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f820bfff640 1 -- 172.21.15.60:0/2077701532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56093e0cd0f0 con 0x56093e2471b0 2024-04-25T10:08:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f820bfff640 1 --2- 172.21.15.60:0/2077701532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e2a7870 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8200000c10 tx=0x7f8200007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8752ffdf96efab75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f81eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:08:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f81eb7fe640 1 -- 172.21.15.60:0/2077701532 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8200007b20 con 0x56093e2471b0 2024-04-25T10:08:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f81eb7fe640 1 -- 172.21.15.60:0/2077701532 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8200010020 con 0x56093e2471b0 2024-04-25T10:08:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 -- 172.21.15.60:0/2077701532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81fc05edd0 con 0x56093e2471b0 2024-04-25T10:08:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f81eb7fe640 1 -- 172.21.15.60:0/2077701532 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8200036050 con 0x56093e2471b0 2024-04-25T10:08:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 10 client.6531 fetch_fsmap learned FSMap version 28 2024-04-25T10:08:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 -- 172.21.15.60:0/2077701532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x56093dfdfd20 con 0x56093e2471b0 2024-04-25T10:08:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f81eb7fe640 1 -- 172.21.15.60:0/2077701532 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc05edd0 con 0x56093e2471b0 2024-04-25T10:08:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f81eb7fe640 1 -- 172.21.15.60:0/2077701532 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8200039020 con 0x56093e2471b0 2024-04-25T10:08:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 10 client.6531 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:08:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 10 client.6531 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 20 client.6531 populate_metadata read hostname 'smithi060' 2024-04-25T10:08:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 --2- 172.21.15.60:0/2077701532 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x56093e263970 0x56093e263d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 4 client.6531 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-04-25T10:08:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.978+0000 7f821631e000 1 -- 172.21.15.60:0/2077701532 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x56093e058510 con 0x56093e263970 2024-04-25T10:08:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f8210f4d640 1 --2- 172.21.15.60:0/2077701532 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x56093e263970 0x56093e263d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f8210f4d640 1 --2- 172.21.15.60:0/2077701532 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x56093e263970 0x56093e263d50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6bae9f497c31a83 server_cookie=6918091b2d9d26d9 in_seq=0 out_seq=0 2024-04-25T10:08:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f81eb7fe640 10 client.6531 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f81eb7fe640 1 -- 172.21.15.60:0/2077701532 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56093e058510 con 0x56093e263970 2024-04-25T10:08:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f81eb7fe640 10 client.6531 handle_command_reply: tid=0 2024-04-25T10:08:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f821631e000 1 client.6531 shutdown 2024-04-25T10:08:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f821631e000 1 -- 172.21.15.60:0/2077701532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 msgr2=0x56093e2a7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f821631e000 1 --2- 172.21.15.60:0/2077701532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e2a7870 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8200000c10 tx=0x7f8200007f90 comp rx=0 tx=0).stop 2024-04-25T10:08:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f821631e000 1 -- 172.21.15.60:0/2077701532 shutdown_connections 2024-04-25T10:08:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f821631e000 1 --2- 172.21.15.60:0/2077701532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56093e247ad0 0x56093e262fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f821631e000 1 --2- 172.21.15.60:0/2077701532 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x56093e263970 0x56093e263d50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f821631e000 1 --2- 172.21.15.60:0/2077701532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56093e2471b0 0x56093e2a7870 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f821631e000 1 --2- 172.21.15.60:0/2077701532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56093e22bf10 0x56093e2a7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f821631e000 1 -- 172.21.15.60:0/2077701532 >> 172.21.15.60:0/2077701532 conn(0x56093e203710 msgr2=0x56093e22b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f81eb7fe640 0 client.6531 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:23.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f821631e000 1 -- 172.21.15.60:0/2077701532 shutdown_connections 2024-04-25T10:08:23.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f821631e000 1 -- 172.21.15.60:0/2077701532 wait complete. 2024-04-25T10:08:23.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f821631e000 20 client.6531 trim_cache size 0 max 16384 2024-04-25T10:08:23.996 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:08:23.996 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T10:08:23.996 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "62b397f4-dd9c-4014-8a11-2bb5e16a891a", 2024-04-25T10:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T10:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:08:23.997 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f2736337640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6489=0-4194304@1} caps={6489=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6489 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x56343c443b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:08:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f82131b1640 1 -- 172.21.15.60:0/2266484669 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81f8066f60 msgr2=0x7f81f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f82131b1640 1 --2- 172.21.15.60:0/2266484669 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81f8066f60 0x7f81f8069420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f82040ae000 tx=0x7f82040a7000 comp rx=0 tx=0).stop 2024-04-25T10:08:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f82131b1640 1 -- 172.21.15.60:0/2266484669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f820c144e40 msgr2=0x7f820c15a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f82131b1640 1 --2- 172.21.15.60:0/2266484669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f820c144e40 0x7f820c15a680 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00ef90 tx=0x7f81fc00c490 comp rx=0 tx=0).stop 2024-04-25T10:08:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f82131b1640 1 -- 172.21.15.60:0/2266484669 shutdown_connections 2024-04-25T10:08:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f82131b1640 1 --2- 172.21.15.60:0/2266484669 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81f8066f60 0x7f81f8069420 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f82131b1640 1 --2- 172.21.15.60:0/2266484669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f820c144e40 0x7f820c15a680 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f82131b1640 1 --2- 172.21.15.60:0/2266484669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f820c0cf0a0 0x7f820c06e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f82131b1640 1 --2- 172.21.15.60:0/2266484669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f820c071980 0x7f820c0ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f82131b1640 1 -- 172.21.15.60:0/2266484669 >> 172.21.15.60:0/2266484669 conn(0x7f820c066fa0 msgr2=0x7f820c073ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f82131b1640 1 -- 172.21.15.60:0/2266484669 shutdown_connections 2024-04-25T10:08:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:23.982+0000 7f82131b1640 1 -- 172.21.15.60:0/2266484669 wait complete. 2024-04-25T10:08:24.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:24.006 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:08:24.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.114+0000 7f5c2306f640 1 Processor -- start 2024-04-25T10:08:24.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.114+0000 7f5c2306f640 1 -- start start 2024-04-25T10:08:24.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c2306f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5c1c161140 0x7f5c1c0d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c2306f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5c1c09a140 0x7f5c1c0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c2306f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1c0cf370 con 0x7f5c1c0cce90 2024-04-25T10:08:24.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c2306f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1c0cf650 con 0x7f5c1c161140 2024-04-25T10:08:24.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c2306f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1c0cf930 con 0x7f5c1c09a140 2024-04-25T10:08:24.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c20e0b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5c1c161140 0x7f5c1c0d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c1bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5c1c0cce90 0x7f5c1c099a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52068/0 (socket says 172.21.15.60:52068) 2024-04-25T10:08:24.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c20e0b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5c1c161140 0x7f5c1c0d0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33548/0 (socket says 172.21.15.60:33548) 2024-04-25T10:08:24.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c1bfff640 1 -- 172.21.15.60:0/959524924 learned_addr learned my addr 172.21.15.60:0/959524924 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:24.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c2160c640 1 --2- 172.21.15.60:0/959524924 >> v2:172.21.15.100:3300/0 conn(0x7f5c1c09a140 0x7f5c1c0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c20e0b640 1 -- 172.21.15.60:0/959524924 >> v2:172.21.15.100:3300/0 conn(0x7f5c1c09a140 msgr2=0x7f5c1c0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c20e0b640 1 --2- 172.21.15.60:0/959524924 >> v2:172.21.15.100:3300/0 conn(0x7f5c1c09a140 0x7f5c1c0cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c20e0b640 1 -- 172.21.15.60:0/959524924 >> v1:172.21.15.60:6789/0 conn(0x7f5c1c0cce90 legacy=0x7f5c1c099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c20e0b640 1 -- 172.21.15.60:0/959524924 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c0cfc10 con 0x7f5c1c161140 2024-04-25T10:08:24.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c1b7fe640 1 -- 172.21.15.60:0/959524924 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 963636128 0 0) 0x7f5c1c0cf370 con 0x7f5c1c0cce90 2024-04-25T10:08:24.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c20e0b640 1 --2- 172.21.15.60:0/959524924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c1c161140 0x7f5c1c0d0410 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c06dce0 tx=0x7f5c1001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa3d13ca76f2d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c1b7fe640 1 -- 172.21.15.60:0/959524924 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c100a7070 con 0x7f5c1c161140 2024-04-25T10:08:24.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c1b7fe640 1 -- 172.21.15.60:0/959524924 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c100b0070 con 0x7f5c1c161140 2024-04-25T10:08:24.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.118+0000 7f5c1b7fe640 1 -- 172.21.15.60:0/959524924 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c100ab890 con 0x7f5c1c161140 2024-04-25T10:08:24.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 -- 172.21.15.60:0/959524924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c1c161140 msgr2=0x7f5c1c0d0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 --2- 172.21.15.60:0/959524924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c1c161140 0x7f5c1c0d0410 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c06dce0 tx=0x7f5c1001c450 comp rx=0 tx=0).stop 2024-04-25T10:08:24.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 -- 172.21.15.60:0/959524924 shutdown_connections 2024-04-25T10:08:24.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 --2- 172.21.15.60:0/959524924 >> v2:172.21.15.100:3300/0 conn(0x7f5c1c09a140 0x7f5c1c0cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 --2- 172.21.15.60:0/959524924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c1c161140 0x7f5c1c0d0410 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 -- 172.21.15.60:0/959524924 >> 172.21.15.60:0/959524924 conn(0x7f5c1c0b3290 msgr2=0x7f5c1c0b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 -- 172.21.15.60:0/959524924 shutdown_connections 2024-04-25T10:08:24.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 -- 172.21.15.60:0/959524924 wait complete. 2024-04-25T10:08:24.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 Processor -- start 2024-04-25T10:08:24.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 -- start start 2024-04-25T10:08:24.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c1c09a140 0x7f5c1c0ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c1c0cce90 0x7f5c1c155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c1c161140 0x7f5c1c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c06c6b0 con 0x7f5c1c161140 2024-04-25T10:08:24.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c06c530 con 0x7f5c1c09a140 2024-04-25T10:08:24.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c06c830 con 0x7f5c1c0cce90 2024-04-25T10:08:24.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2160c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c1c161140 0x7f5c1c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2160c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c1c161140 0x7f5c1c0c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33558/0 (socket says 172.21.15.60:33558) 2024-04-25T10:08:24.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2160c640 1 -- 172.21.15.60:0/3859310619 learned_addr learned my addr 172.21.15.60:0/3859310619 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:24.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c20e0b640 1 --2- 172.21.15.60:0/3859310619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c1c09a140 0x7f5c1c0ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c1bfff640 1 --2- 172.21.15.60:0/3859310619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c1c0cce90 0x7f5c1c155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2160c640 1 -- 172.21.15.60:0/3859310619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c1c0cce90 msgr2=0x7f5c1c155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2160c640 1 --2- 172.21.15.60:0/3859310619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c1c0cce90 0x7f5c1c155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2160c640 1 -- 172.21.15.60:0/3859310619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c1c09a140 msgr2=0x7f5c1c0ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2160c640 1 --2- 172.21.15.60:0/3859310619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c1c09a140 0x7f5c1c0ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2160c640 1 -- 172.21.15.60:0/3859310619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c155eb0 con 0x7f5c1c161140 2024-04-25T10:08:24.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2160c640 1 --2- 172.21.15.60:0/3859310619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c1c161140 0x7f5c1c0c7cb0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5c14007e70 tx=0x7f5c1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c197fa640 1 -- 172.21.15.60:0/3859310619 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c14014620 con 0x7f5c1c161140 2024-04-25T10:08:24.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c197fa640 1 -- 172.21.15.60:0/3859310619 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c14019070 con 0x7f5c1c161140 2024-04-25T10:08:24.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 -- 172.21.15.60:0/3859310619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1c0ca7c0 con 0x7f5c1c161140 2024-04-25T10:08:24.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c197fa640 1 -- 172.21.15.60:0/3859310619 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c14014a20 con 0x7f5c1c161140 2024-04-25T10:08:24.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.122+0000 7f5c2306f640 1 -- 172.21.15.60:0/3859310619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c0b4560 con 0x7f5c1c161140 2024-04-25T10:08:24.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c197fa640 1 -- 172.21.15.60:0/3859310619 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c14014bc0 con 0x7f5c1c161140 2024-04-25T10:08:24.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c197fa640 1 --2- 172.21.15.60:0/3859310619 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5c00066f90 0x7f5c00069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c197fa640 1 -- 172.21.15.60:0/3859310619 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c140858d0 con 0x7f5c1c161140 2024-04-25T10:08:24.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c20e0b640 1 --2- 172.21.15.60:0/3859310619 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5c00066f90 0x7f5c00069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c261dc000 1 Processor -- start 2024-04-25T10:08:24.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c261dc000 1 -- start start 2024-04-25T10:08:24.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c20e0b640 1 --2- 172.21.15.60:0/3859310619 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5c00066f90 0x7f5c00069450 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5c100af000 tx=0x7f5c100a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c261dc000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56287956f9d0 0x56287956fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c261dc000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628795702f0 0x562879525b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c261dc000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x562879526480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c261dc000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562879372ff0 con 0x5628795260a0 2024-04-25T10:08:24.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c261dc000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628792fb150 con 0x5628795702f0 2024-04-25T10:08:24.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c261dc000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56287936f090 con 0x56287956f9d0 2024-04-25T10:08:24.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c2160c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x562879526480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c2160c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x562879526480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33562/0 (socket says 172.21.15.60:33562) 2024-04-25T10:08:24.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c2160c640 1 -- 172.21.15.60:0/3392765703 learned_addr learned my addr 172.21.15.60:0/3392765703 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:24.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c20e0b640 1 --2- 172.21.15.60:0/3392765703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56287956f9d0 0x56287956fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c1bfff640 1 --2- 172.21.15.60:0/3392765703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628795702f0 0x562879525b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c2160c640 1 -- 172.21.15.60:0/3392765703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56287956f9d0 msgr2=0x56287956fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c2160c640 1 --2- 172.21.15.60:0/3392765703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56287956f9d0 0x56287956fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c2160c640 1 -- 172.21.15.60:0/3392765703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628795702f0 msgr2=0x562879525b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c2160c640 1 --2- 172.21.15.60:0/3392765703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628795702f0 0x562879525b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c2160c640 1 -- 172.21.15.60:0/3392765703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628795090d0 con 0x5628795260a0 2024-04-25T10:08:24.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c1bfff640 1 --2- 172.21.15.60:0/3392765703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628795702f0 0x562879525b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:24.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c2160c640 1 --2- 172.21.15.60:0/3392765703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x562879526480 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5c14056170 tx=0x7f5c140568e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ef3a2663e8d7b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c067fc640 1 -- 172.21.15.60:0/3392765703 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1405e8f0 con 0x5628795260a0 2024-04-25T10:08:24.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c067fc640 1 -- 172.21.15.60:0/3392765703 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c140544e0 con 0x5628795260a0 2024-04-25T10:08:24.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.126+0000 7f5c067fc640 1 -- 172.21.15.60:0/3392765703 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1405e460 con 0x5628795260a0 2024-04-25T10:08:24.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 -- 172.21.15.60:0/3392765703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 msgr2=0x562879526480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 --2- 172.21.15.60:0/3392765703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x562879526480 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5c14056170 tx=0x7f5c140568e0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 -- 172.21.15.60:0/3392765703 shutdown_connections 2024-04-25T10:08:24.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 --2- 172.21.15.60:0/3392765703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x562879526480 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 --2- 172.21.15.60:0/3392765703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628795702f0 0x562879525b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 --2- 172.21.15.60:0/3392765703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56287956f9d0 0x56287956fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 -- 172.21.15.60:0/3392765703 >> 172.21.15.60:0/3392765703 conn(0x562879522020 msgr2=0x562879524440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 -- 172.21.15.60:0/3392765703 shutdown_connections 2024-04-25T10:08:24.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 -- 172.21.15.60:0/3392765703 wait complete. 2024-04-25T10:08:24.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 -- start start 2024-04-25T10:08:24.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 Processor -- start 2024-04-25T10:08:24.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628795260a0 0x5628795547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56287956f9d0 0x562879554d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5628795702f0 0x562879555270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562879337250 con 0x56287956f9d0 2024-04-25T10:08:24.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56287932d690 con 0x5628795260a0 2024-04-25T10:08:24.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628792fba40 con 0x5628795702f0 2024-04-25T10:08:24.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 10 client.0 _ll_register_callbacks cb 0x562879510e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:24.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c1bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56287956f9d0 0x562879554d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c1bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56287956f9d0 0x562879554d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33570/0 (socket says 172.21.15.60:33570) 2024-04-25T10:08:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c1bfff640 1 -- 172.21.15.60:0/4150785381 learned_addr learned my addr 172.21.15.60:0/4150785381 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c2160c640 1 --2- 172.21.15.60:0/4150785381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5628795702f0 0x562879555270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c20e0b640 1 --2- 172.21.15.60:0/4150785381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628795260a0 0x5628795547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c1bfff640 1 -- 172.21.15.60:0/4150785381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5628795702f0 msgr2=0x562879555270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c1bfff640 1 --2- 172.21.15.60:0/4150785381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5628795702f0 0x562879555270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c1bfff640 1 -- 172.21.15.60:0/4150785381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628795260a0 msgr2=0x5628795547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c1bfff640 1 --2- 172.21.15.60:0/4150785381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628795260a0 0x5628795547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c1bfff640 1 -- 172.21.15.60:0/4150785381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628794201c0 con 0x56287956f9d0 2024-04-25T10:08:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c20e0b640 1 --2- 172.21.15.60:0/4150785381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628795260a0 0x5628795547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c1bfff640 1 --2- 172.21.15.60:0/4150785381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56287956f9d0 0x562879554d30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c09a0c0 tx=0x7f5c08001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e190cc424f975b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5bfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:08:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5bfb7fe640 1 -- 172.21.15.60:0/4150785381 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0801a060 con 0x56287956f9d0 2024-04-25T10:08:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5bfb7fe640 1 -- 172.21.15.60:0/4150785381 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c08015aa0 con 0x56287956f9d0 2024-04-25T10:08:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 -- 172.21.15.60:0/4150785381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c140544e0 con 0x56287956f9d0 2024-04-25T10:08:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5bfb7fe640 1 -- 172.21.15.60:0/4150785381 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c0800d070 con 0x56287956f9d0 2024-04-25T10:08:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 10 client.6546 fetch_fsmap learned FSMap version 28 2024-04-25T10:08:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 -- 172.21.15.60:0/4150785381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x56287941c170 con 0x56287956f9d0 2024-04-25T10:08:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5bfb7fe640 1 -- 172.21.15.60:0/4150785381 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c140544e0 con 0x56287956f9d0 2024-04-25T10:08:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5bfb7fe640 1 -- 172.21.15.60:0/4150785381 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5c08013030 con 0x56287956f9d0 2024-04-25T10:08:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 10 client.6546 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:08:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 10 client.6546 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 20 client.6546 populate_metadata read hostname 'smithi060' 2024-04-25T10:08:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 --2- 172.21.15.60:0/4150785381 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x56287957fc80 0x562879582070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 4 client.6546 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c261dc000 1 -- 172.21.15.60:0/4150785381 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56287930ec60 con 0x56287957fc80 2024-04-25T10:08:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c20e0b640 1 --2- 172.21.15.60:0/4150785381 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x56287957fc80 0x562879582070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5bfb7fe640 10 client.6546 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.130+0000 7f5c20e0b640 1 --2- 172.21.15.60:0/4150785381 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x56287957fc80 0x562879582070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b7eef3ea71f258a server_cookie=9efee11ae2e05170 in_seq=0 out_seq=0 2024-04-25T10:08:24.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.134+0000 7f5bfb7fe640 1 -- 172.21.15.60:0/4150785381 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56287930ec60 con 0x56287957fc80 2024-04-25T10:08:24.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.134+0000 7f5bfb7fe640 10 client.6546 handle_command_reply: tid=0 2024-04-25T10:08:24.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.134+0000 7f5c261dc000 1 client.6546 shutdown 2024-04-25T10:08:24.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.134+0000 7f5c261dc000 1 -- 172.21.15.60:0/4150785381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56287956f9d0 msgr2=0x562879554d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.134+0000 7f5c261dc000 1 --2- 172.21.15.60:0/4150785381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56287956f9d0 0x562879554d30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c09a0c0 tx=0x7f5c08001690 comp rx=0 tx=0).stop 2024-04-25T10:08:24.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.134+0000 7f5c261dc000 1 -- 172.21.15.60:0/4150785381 shutdown_connections 2024-04-25T10:08:24.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.134+0000 7f5c261dc000 1 --2- 172.21.15.60:0/4150785381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5628795702f0 0x562879555270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.134+0000 7f5c261dc000 1 --2- 172.21.15.60:0/4150785381 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x56287957fc80 0x562879582070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.134+0000 7f5c261dc000 1 --2- 172.21.15.60:0/4150785381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56287956f9d0 0x562879554d30 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.134+0000 7f5c261dc000 1 --2- 172.21.15.60:0/4150785381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628795260a0 0x5628795547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.134+0000 7f5bfb7fe640 0 client.6546 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:24.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.134+0000 7f5c261dc000 1 -- 172.21.15.60:0/4150785381 >> 172.21.15.60:0/4150785381 conn(0x562879522020 msgr2=0x562879524ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.134+0000 7f5c261dc000 1 -- 172.21.15.60:0/4150785381 shutdown_connections 2024-04-25T10:08:24.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.134+0000 7f5c261dc000 1 -- 172.21.15.60:0/4150785381 wait complete. 2024-04-25T10:08:24.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.134+0000 7f5c261dc000 20 client.6546 trim_cache size 0 max 16384 2024-04-25T10:08:24.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.146+0000 7f5c261dc000 1 Processor -- start 2024-04-25T10:08:24.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.146+0000 7f5c261dc000 1 -- start start 2024-04-25T10:08:24.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x5628795cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56287956f9d0 0x5628795cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5628795702f0 0x5628795ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c202840 con 0x5628795260a0 2024-04-25T10:08:24.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c154fa0 con 0x56287956f9d0 2024-04-25T10:08:24.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c1549a0 con 0x5628795702f0 2024-04-25T10:08:24.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x5628795cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x5628795cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33582/0 (socket says 172.21.15.60:33582) 2024-04-25T10:08:24.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 -- 172.21.15.60:0/3295699599 learned_addr learned my addr 172.21.15.60:0/3295699599 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:24.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 -- 172.21.15.60:0/3295699599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5628795702f0 msgr2=0x5628795ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 --2- 172.21.15.60:0/3295699599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5628795702f0 0x5628795ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 -- 172.21.15.60:0/3295699599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56287956f9d0 msgr2=0x5628795cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 --2- 172.21.15.60:0/3295699599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56287956f9d0 0x5628795cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 -- 172.21.15.60:0/3295699599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628793b9de0 con 0x5628795260a0 2024-04-25T10:08:24.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 --2- 172.21.15.60:0/3295699599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x5628795cc300 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5c1001c490 tx=0x7f5c10009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6e5df4a7fe5545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c05ffb640 1 -- 172.21.15.60:0/3295699599 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c100bb070 con 0x5628795260a0 2024-04-25T10:08:24.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c05ffb640 1 -- 172.21.15.60:0/3295699599 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c100d1070 con 0x5628795260a0 2024-04-25T10:08:24.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c05ffb640 1 -- 172.21.15.60:0/3295699599 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c100cc890 con 0x5628795260a0 2024-04-25T10:08:24.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 -- 172.21.15.60:0/3295699599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 msgr2=0x5628795cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 --2- 172.21.15.60:0/3295699599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x5628795cc300 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5c1001c490 tx=0x7f5c10009190 comp rx=0 tx=0).stop 2024-04-25T10:08:24.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 -- 172.21.15.60:0/3295699599 shutdown_connections 2024-04-25T10:08:24.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 --2- 172.21.15.60:0/3295699599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5628795702f0 0x5628795ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 --2- 172.21.15.60:0/3295699599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56287956f9d0 0x5628795cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 --2- 172.21.15.60:0/3295699599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x5628795cc300 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 -- 172.21.15.60:0/3295699599 >> 172.21.15.60:0/3295699599 conn(0x562879522020 msgr2=0x56287957cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 -- 172.21.15.60:0/3295699599 shutdown_connections 2024-04-25T10:08:24.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 -- 172.21.15.60:0/3295699599 wait complete. 2024-04-25T10:08:24.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 -- start start 2024-04-25T10:08:24.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 Processor -- start 2024-04-25T10:08:24.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x562879582380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56287956f9d0 0x5628795828c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5628795702f0 0x562879582e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562879372300 con 0x5628795260a0 2024-04-25T10:08:24.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56287930de00 con 0x56287956f9d0 2024-04-25T10:08:24.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x562879377620 con 0x5628795702f0 2024-04-25T10:08:24.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c261dc000 10 client.0 _ll_register_callbacks cb 0x5628794f1040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:24.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x562879582380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x562879582380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33590/0 (socket says 172.21.15.60:33590) 2024-04-25T10:08:24.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 -- 172.21.15.60:0/581434749 learned_addr learned my addr 172.21.15.60:0/581434749 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:24.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c1bfff640 1 --2- 172.21.15.60:0/581434749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56287956f9d0 0x5628795828c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 -- 172.21.15.60:0/581434749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5628795702f0 msgr2=0x562879582e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c2160c640 1 --2- 172.21.15.60:0/581434749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5628795702f0 0x562879582e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 --2- 172.21.15.60:0/581434749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5628795702f0 0x562879582e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 -- 172.21.15.60:0/581434749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56287956f9d0 msgr2=0x5628795828c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 --2- 172.21.15.60:0/581434749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56287956f9d0 0x5628795828c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c20e0b640 1 -- 172.21.15.60:0/581434749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628793148e0 con 0x5628795260a0 2024-04-25T10:08:24.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.150+0000 7f5c1bfff640 1 --2- 172.21.15.60:0/581434749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56287956f9d0 0x5628795828c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:24.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c20e0b640 1 --2- 172.21.15.60:0/581434749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x562879582380 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5c1001c490 tx=0x7f5c1001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431fa29d975ce57a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5bfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:08:24.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5bfb7fe640 1 -- 172.21.15.60:0/581434749 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c100d9030 con 0x5628795260a0 2024-04-25T10:08:24.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5bfb7fe640 1 -- 172.21.15.60:0/581434749 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c100cc550 con 0x5628795260a0 2024-04-25T10:08:24.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 1 -- 172.21.15.60:0/581434749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c100d1070 con 0x5628795260a0 2024-04-25T10:08:24.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5bfb7fe640 1 -- 172.21.15.60:0/581434749 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c100bb040 con 0x5628795260a0 2024-04-25T10:08:24.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 10 client.6552 fetch_fsmap learned FSMap version 28 2024-04-25T10:08:24.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 1 -- 172.21.15.60:0/581434749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5628793cc8f0 con 0x5628795260a0 2024-04-25T10:08:24.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5bfb7fe640 1 -- 172.21.15.60:0/581434749 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c100d1070 con 0x5628795260a0 2024-04-25T10:08:24.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5bfb7fe640 1 -- 172.21.15.60:0/581434749 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5c100ccce0 con 0x5628795260a0 2024-04-25T10:08:24.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 10 client.6552 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:08:24.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 10 client.6552 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:24.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 20 client.6552 populate_metadata read hostname 'smithi060' 2024-04-25T10:08:24.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 1 --2- 172.21.15.60:0/581434749 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x562879583760 0x5628795d7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 4 client.6552 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:08:24.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 1 -- 172.21.15.60:0/581434749 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562879380530 con 0x562879583760 2024-04-25T10:08:24.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c1bfff640 1 --2- 172.21.15.60:0/581434749 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x562879583760 0x5628795d7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c1bfff640 1 --2- 172.21.15.60:0/581434749 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x562879583760 0x5628795d7f60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3727ff6be944d518 server_cookie=5a3e20c960bdf9e9 in_seq=0 out_seq=0 2024-04-25T10:08:24.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5bfb7fe640 10 client.6552 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:24.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5bfb7fe640 1 -- 172.21.15.60:0/581434749 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x562879380530 con 0x562879583760 2024-04-25T10:08:24.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5bfb7fe640 10 client.6552 handle_command_reply: tid=0 2024-04-25T10:08:24.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 1 client.6552 shutdown 2024-04-25T10:08:24.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 1 -- 172.21.15.60:0/581434749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 msgr2=0x562879582380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 1 --2- 172.21.15.60:0/581434749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x562879582380 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5c1001c490 tx=0x7f5c1001ac90 comp rx=0 tx=0).stop 2024-04-25T10:08:24.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 1 -- 172.21.15.60:0/581434749 shutdown_connections 2024-04-25T10:08:24.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 1 --2- 172.21.15.60:0/581434749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5628795702f0 0x562879582e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 1 --2- 172.21.15.60:0/581434749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56287956f9d0 0x5628795828c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 1 --2- 172.21.15.60:0/581434749 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x562879583760 0x5628795d7f60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 1 --2- 172.21.15.60:0/581434749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5628795260a0 0x562879582380 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5bfb7fe640 0 client.6552 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:24.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 1 -- 172.21.15.60:0/581434749 >> 172.21.15.60:0/581434749 conn(0x562879522020 msgr2=0x56287956e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 1 -- 172.21.15.60:0/581434749 shutdown_connections 2024-04-25T10:08:24.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 1 -- 172.21.15.60:0/581434749 wait complete. 2024-04-25T10:08:24.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c261dc000 20 client.6552 trim_cache size 0 max 16384 2024-04-25T10:08:24.167 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:08:24.168 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:08:24.168 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T10:08:24.168 INFO:teuthology.orchestra.run.smithi060.stdout: "62b397f4-dd9c-4014-8a11-2bb5e16a891a": { 2024-04-25T10:08:24.168 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-04-25T10:08:24.168 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "62b397f4-dd9c-4014-8a11-2bb5e16a891a", 2024-04-25T10:08:24.168 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive" 2024-04-25T10:08:24.168 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:24.168 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:24.168 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:08:24.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c2306f640 1 -- 172.21.15.60:0/3859310619 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5c00066f90 msgr2=0x7f5c00069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c2306f640 1 --2- 172.21.15.60:0/3859310619 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5c00066f90 0x7f5c00069450 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5c100af000 tx=0x7f5c100a7000 comp rx=0 tx=0).stop 2024-04-25T10:08:24.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c2306f640 1 -- 172.21.15.60:0/3859310619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c1c161140 msgr2=0x7f5c1c0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c2306f640 1 --2- 172.21.15.60:0/3859310619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c1c161140 0x7f5c1c0c7cb0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5c14007e70 tx=0x7f5c1400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c2306f640 1 -- 172.21.15.60:0/3859310619 shutdown_connections 2024-04-25T10:08:24.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c2306f640 1 --2- 172.21.15.60:0/3859310619 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5c00066f90 0x7f5c00069450 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c2306f640 1 --2- 172.21.15.60:0/3859310619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c1c161140 0x7f5c1c0c7cb0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c2306f640 1 --2- 172.21.15.60:0/3859310619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c1c0cce90 0x7f5c1c155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c2306f640 1 --2- 172.21.15.60:0/3859310619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c1c09a140 0x7f5c1c0ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c2306f640 1 -- 172.21.15.60:0/3859310619 >> 172.21.15.60:0/3859310619 conn(0x7f5c1c0b3290 msgr2=0x7f5c1c0cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c2306f640 1 -- 172.21.15.60:0/3859310619 shutdown_connections 2024-04-25T10:08:24.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:24.154+0000 7f5c2306f640 1 -- 172.21.15.60:0/3859310619 wait complete. 2024-04-25T10:08:24.179 INFO:tasks.cephfs.filesystem:scrub status for tag:62b397f4-dd9c-4014-8a11-2bb5e16a891a - {'path': '/', 'tag': '62b397f4-dd9c-4014-8a11-2bb5e16a891a', 'options': 'recursive'} 2024-04-25T10:08:27.529 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:08:27.522+0000 7f3a7badb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:08:46.590 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:46.593 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:54.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:54.180 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:08:54.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.310+0000 7f57a5969640 1 Processor -- start 2024-04-25T10:08:54.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.310+0000 7f57a5969640 1 -- start start 2024-04-25T10:08:54.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f57a5969640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a015b6f0 0x7f57a01620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f57a5969640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57a00ceef0 0x7f57a015e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f57a5969640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a015bad0 con 0x7f57a015e0d0 2024-04-25T10:08:54.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f57a5969640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a0162610 con 0x7f57a00ceef0 2024-04-25T10:08:54.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f57a5969640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a0161c40 con 0x7f57a015b6f0 2024-04-25T10:08:54.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f579f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57a00ceef0 0x7f57a015e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f579f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57a00ceef0 0x7f57a015e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55752/0 (socket says 172.21.15.60:55752) 2024-04-25T10:08:54.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f579f7fe640 1 -- 172.21.15.60:0/949619689 learned_addr learned my addr 172.21.15.60:0/949619689 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:54.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f579effd640 1 --2- 172.21.15.60:0/949619689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a015b6f0 0x7f57a01620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f579f7fe640 1 -- 172.21.15.60:0/949619689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a015b6f0 msgr2=0x7f57a01620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f579f7fe640 1 --2- 172.21.15.60:0/949619689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a015b6f0 0x7f57a01620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f579f7fe640 1 -- 172.21.15.60:0/949619689 >> v1:172.21.15.100:6789/0 conn(0x7f57a015e0d0 legacy=0x7f57a00ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f579f7fe640 1 -- 172.21.15.60:0/949619689 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a015ebe0 con 0x7f57a00ceef0 2024-04-25T10:08:54.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f579effd640 1 --2- 172.21.15.60:0/949619689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a015b6f0 0x7f57a01620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:54.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f579f7fe640 1 --2- 172.21.15.60:0/949619689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00ceef0 0x7f57a015e510 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5798009e50 tx=0x7f579801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59349d37b5dab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f579e7fc640 1 -- 172.21.15.60:0/949619689 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980a57f0 con 0x7f57a00ceef0 2024-04-25T10:08:54.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f579e7fc640 1 -- 172.21.15.60:0/949619689 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57980a5990 con 0x7f57a00ceef0 2024-04-25T10:08:54.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.314+0000 7f579e7fc640 1 -- 172.21.15.60:0/949619689 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980a5d60 con 0x7f57a00ceef0 2024-04-25T10:08:54.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f57a5969640 1 -- 172.21.15.60:0/949619689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00ceef0 msgr2=0x7f57a015e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f57a5969640 1 --2- 172.21.15.60:0/949619689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00ceef0 0x7f57a015e510 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5798009e50 tx=0x7f579801ca30 comp rx=0 tx=0).stop 2024-04-25T10:08:54.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f57a5969640 1 -- 172.21.15.60:0/949619689 shutdown_connections 2024-04-25T10:08:54.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f57a5969640 1 --2- 172.21.15.60:0/949619689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00ceef0 0x7f57a015e510 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f57a5969640 1 --2- 172.21.15.60:0/949619689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a015b6f0 0x7f57a01620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f57a5969640 1 -- 172.21.15.60:0/949619689 >> 172.21.15.60:0/949619689 conn(0x7f57a0072340 msgr2=0x7f57a0072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f57a5969640 1 -- 172.21.15.60:0/949619689 shutdown_connections 2024-04-25T10:08:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f57a5969640 1 -- 172.21.15.60:0/949619689 wait complete. 2024-04-25T10:08:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f57a5969640 1 Processor -- start 2024-04-25T10:08:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f57a5969640 1 -- start start 2024-04-25T10:08:54.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f57a5969640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00ceef0 0x7f57a015a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f57a5969640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a015b6f0 0x7f57a00c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f57a5969640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a015e0d0 0x7f57a00c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f57a5969640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00d24c0 con 0x7f57a00ceef0 2024-04-25T10:08:54.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f57a5969640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00d2340 con 0x7f57a015b6f0 2024-04-25T10:08:54.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f57a5969640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00d2640 con 0x7f57a015e0d0 2024-04-25T10:08:54.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f579effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00ceef0 0x7f57a015a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f579effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00ceef0 0x7f57a015a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55758/0 (socket says 172.21.15.60:55758) 2024-04-25T10:08:54.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f579effd640 1 -- 172.21.15.60:0/2089632433 learned_addr learned my addr 172.21.15.60:0/2089632433 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f5797fff640 1 --2- 172.21.15.60:0/2089632433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a015b6f0 0x7f57a00c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.318+0000 7f579f7fe640 1 --2- 172.21.15.60:0/2089632433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a015e0d0 0x7f57a00c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f579effd640 1 -- 172.21.15.60:0/2089632433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a015e0d0 msgr2=0x7f57a00c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f579effd640 1 --2- 172.21.15.60:0/2089632433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a015e0d0 0x7f57a00c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f579effd640 1 -- 172.21.15.60:0/2089632433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a015b6f0 msgr2=0x7f57a00c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f579effd640 1 --2- 172.21.15.60:0/2089632433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a015b6f0 0x7f57a00c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f579effd640 1 -- 172.21.15.60:0/2089632433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a00b8b20 con 0x7f57a00ceef0 2024-04-25T10:08:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f5797fff640 1 --2- 172.21.15.60:0/2089632433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a015b6f0 0x7f57a00c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f579effd640 1 --2- 172.21.15.60:0/2089632433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00ceef0 0x7f57a015a960 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f579000e810 tx=0x7f579000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f579cff9640 1 -- 172.21.15.60:0/2089632433 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790017070 con 0x7f57a00ceef0 2024-04-25T10:08:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f579cff9640 1 -- 172.21.15.60:0/2089632433 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579000c8d0 con 0x7f57a00ceef0 2024-04-25T10:08:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f57a5969640 1 -- 172.21.15.60:0/2089632433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a00b8e00 con 0x7f57a00ceef0 2024-04-25T10:08:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f579cff9640 1 -- 172.21.15.60:0/2089632433 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579000f040 con 0x7f57a00ceef0 2024-04-25T10:08:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f57a5969640 1 -- 172.21.15.60:0/2089632433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a00c6a90 con 0x7f57a00ceef0 2024-04-25T10:08:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f579cff9640 1 -- 172.21.15.60:0/2089632433 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f579001a070 con 0x7f57a00ceef0 2024-04-25T10:08:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f579cff9640 1 --2- 172.21.15.60:0/2089632433 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5780066f60 0x7f5780069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f579cff9640 1 -- 172.21.15.60:0/2089632433 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5790084ce0 con 0x7f57a00ceef0 2024-04-25T10:08:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f5797fff640 1 --2- 172.21.15.60:0/2089632433 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5780066f60 0x7f5780069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f57a8ad6000 1 Processor -- start 2024-04-25T10:08:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.322+0000 7f57a8ad6000 1 -- start start 2024-04-25T10:08:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f5797fff640 1 --2- 172.21.15.60:0/2089632433 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5780066f60 0x7f5780069420 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f57a015aea0 tx=0x7f5788009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a67e9d0 0x562b6a67edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a67f2f0 0x562b6a634b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a6350a0 0x562b6a635480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b6a481ff0 con 0x562b6a6350a0 2024-04-25T10:08:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b6a40a150 con 0x562b6a67f2f0 2024-04-25T10:08:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b6a47e090 con 0x562b6a67e9d0 2024-04-25T10:08:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a6350a0 0x562b6a635480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a6350a0 0x562b6a635480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55762/0 (socket says 172.21.15.60:55762) 2024-04-25T10:08:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f579f7fe640 1 -- 172.21.15.60:0/1141416514 learned_addr learned my addr 172.21.15.60:0/1141416514 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f579effd640 1 --2- 172.21.15.60:0/1141416514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a67e9d0 0x562b6a67edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f5797fff640 1 --2- 172.21.15.60:0/1141416514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a67f2f0 0x562b6a634b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f579f7fe640 1 -- 172.21.15.60:0/1141416514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a67e9d0 msgr2=0x562b6a67edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f579f7fe640 1 --2- 172.21.15.60:0/1141416514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a67e9d0 0x562b6a67edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f579f7fe640 1 -- 172.21.15.60:0/1141416514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a67f2f0 msgr2=0x562b6a634b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f579f7fe640 1 --2- 172.21.15.60:0/1141416514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a67f2f0 0x562b6a634b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f579f7fe640 1 -- 172.21.15.60:0/1141416514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b6a6180d0 con 0x562b6a6350a0 2024-04-25T10:08:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f579f7fe640 1 --2- 172.21.15.60:0/1141416514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a6350a0 0x562b6a635480 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f579801b0e0 tx=0x7f5798004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a145a8e746134a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57957fa640 1 -- 172.21.15.60:0/1141416514 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980a57f0 con 0x562b6a6350a0 2024-04-25T10:08:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57957fa640 1 -- 172.21.15.60:0/1141416514 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57980a5990 con 0x562b6a6350a0 2024-04-25T10:08:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57957fa640 1 -- 172.21.15.60:0/1141416514 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980a5d60 con 0x562b6a6350a0 2024-04-25T10:08:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 -- 172.21.15.60:0/1141416514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a6350a0 msgr2=0x562b6a635480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/1141416514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a6350a0 0x562b6a635480 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f579801b0e0 tx=0x7f5798004ac0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 -- 172.21.15.60:0/1141416514 shutdown_connections 2024-04-25T10:08:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/1141416514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a6350a0 0x562b6a635480 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/1141416514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a67f2f0 0x562b6a634b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/1141416514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a67e9d0 0x562b6a67edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 -- 172.21.15.60:0/1141416514 >> 172.21.15.60:0/1141416514 conn(0x562b6a631020 msgr2=0x562b6a633440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 -- 172.21.15.60:0/1141416514 shutdown_connections 2024-04-25T10:08:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 -- 172.21.15.60:0/1141416514 wait complete. 2024-04-25T10:08:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 -- start start 2024-04-25T10:08:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 Processor -- start 2024-04-25T10:08:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a6350a0 0x562b6a6637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a67e9d0 0x562b6a663d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a67f2f0 0x562b6a664270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b6a446250 con 0x562b6a67e9d0 2024-04-25T10:08:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b6a43c690 con 0x562b6a6350a0 2024-04-25T10:08:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b6a40aa40 con 0x562b6a67f2f0 2024-04-25T10:08:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f57a8ad6000 10 client.0 _ll_register_callbacks cb 0x562b6a61fe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f5797fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a67e9d0 0x562b6a663d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f5797fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a67e9d0 0x562b6a663d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55772/0 (socket says 172.21.15.60:55772) 2024-04-25T10:08:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f5797fff640 1 -- 172.21.15.60:0/2713664359 learned_addr learned my addr 172.21.15.60:0/2713664359 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f579effd640 1 --2- 172.21.15.60:0/2713664359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a6350a0 0x562b6a6637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.326+0000 7f579f7fe640 1 --2- 172.21.15.60:0/2713664359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a67f2f0 0x562b6a664270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f5797fff640 1 -- 172.21.15.60:0/2713664359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a67f2f0 msgr2=0x562b6a664270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f5797fff640 1 --2- 172.21.15.60:0/2713664359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a67f2f0 0x562b6a664270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f5797fff640 1 -- 172.21.15.60:0/2713664359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a6350a0 msgr2=0x562b6a6637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f5797fff640 1 --2- 172.21.15.60:0/2713664359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a6350a0 0x562b6a6637f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f5797fff640 1 -- 172.21.15.60:0/2713664359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b6a52f1c0 con 0x562b6a67e9d0 2024-04-25T10:08:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f5797fff640 1 --2- 172.21.15.60:0/2713664359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a67e9d0 0x562b6a663d30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f5788007e90 tx=0x7f5788000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46bb4f97cbe0d771 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f577e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:08:54.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f577e7fc640 1 -- 172.21.15.60:0/2713664359 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788027060 con 0x562b6a67e9d0 2024-04-25T10:08:54.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f577e7fc640 1 -- 172.21.15.60:0/2713664359 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5788010040 con 0x562b6a67e9d0 2024-04-25T10:08:54.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 1 -- 172.21.15.60:0/2713664359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57980a5990 con 0x562b6a67e9d0 2024-04-25T10:08:54.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f577e7fc640 1 -- 172.21.15.60:0/2713664359 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f578800ec90 con 0x562b6a67e9d0 2024-04-25T10:08:54.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 10 client.6564 fetch_fsmap learned FSMap version 29 2024-04-25T10:08:54.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 1 -- 172.21.15.60:0/2713664359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x562b6a52b170 con 0x562b6a67e9d0 2024-04-25T10:08:54.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f577e7fc640 1 -- 172.21.15.60:0/2713664359 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57980a5990 con 0x562b6a67e9d0 2024-04-25T10:08:54.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f577e7fc640 1 -- 172.21.15.60:0/2713664359 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5788013420 con 0x562b6a67e9d0 2024-04-25T10:08:54.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 10 client.6564 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:08:54.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 10 client.6564 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:54.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 20 client.6564 populate_metadata read hostname 'smithi060' 2024-04-25T10:08:54.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/2713664359 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x562b6a68ec80 0x562b6a691070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 4 client.6564 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 1 -- 172.21.15.60:0/2713664359 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b6a41dc60 con 0x562b6a68ec80 2024-04-25T10:08:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f579effd640 1 --2- 172.21.15.60:0/2713664359 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x562b6a68ec80 0x562b6a691070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f577e7fc640 10 client.6564 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f579effd640 1 --2- 172.21.15.60:0/2713664359 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x562b6a68ec80 0x562b6a691070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc99bda1b0bb52c server_cookie=c205ae65fde5f2fe in_seq=0 out_seq=0 2024-04-25T10:08:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f577e7fc640 1 -- 172.21.15.60:0/2713664359 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562b6a41dc60 con 0x562b6a68ec80 2024-04-25T10:08:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f577e7fc640 10 client.6564 handle_command_reply: tid=0 2024-04-25T10:08:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 1 client.6564 shutdown 2024-04-25T10:08:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 1 -- 172.21.15.60:0/2713664359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a67e9d0 msgr2=0x562b6a663d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/2713664359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a67e9d0 0x562b6a663d30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f5788007e90 tx=0x7f5788000bd0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 1 -- 172.21.15.60:0/2713664359 shutdown_connections 2024-04-25T10:08:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/2713664359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a67f2f0 0x562b6a664270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/2713664359 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x562b6a68ec80 0x562b6a691070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/2713664359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a67e9d0 0x562b6a663d30 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/2713664359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a6350a0 0x562b6a6637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 1 -- 172.21.15.60:0/2713664359 >> 172.21.15.60:0/2713664359 conn(0x562b6a631020 msgr2=0x562b6a633ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f577e7fc640 0 client.6564 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:54.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 1 -- 172.21.15.60:0/2713664359 shutdown_connections 2024-04-25T10:08:54.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 1 -- 172.21.15.60:0/2713664359 wait complete. 2024-04-25T10:08:54.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.330+0000 7f57a8ad6000 20 client.6564 trim_cache size 0 max 16384 2024-04-25T10:08:54.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f57a8ad6000 1 Processor -- start 2024-04-25T10:08:54.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f57a8ad6000 1 -- start start 2024-04-25T10:08:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f57a8ad6000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a6350a0 0x562b6a6db0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f57a8ad6000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a67e9d0 0x562b6a6db630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f57a8ad6000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a67f2f0 0x562b6a6dbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f57a8ad6000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a0202ae0 con 0x562b6a67e9d0 2024-04-25T10:08:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f57a8ad6000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00b4070 con 0x562b6a67f2f0 2024-04-25T10:08:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f57a8ad6000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00b3470 con 0x562b6a6350a0 2024-04-25T10:08:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f5797fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a67e9d0 0x562b6a6db630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f5797fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a67e9d0 0x562b6a6db630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55786/0 (socket says 172.21.15.60:55786) 2024-04-25T10:08:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f5797fff640 1 -- 172.21.15.60:0/2098721775 learned_addr learned my addr 172.21.15.60:0/2098721775 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f5797fff640 1 -- 172.21.15.60:0/2098721775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a6350a0 msgr2=0x562b6a6db0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f5797fff640 1 --2- 172.21.15.60:0/2098721775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a6350a0 0x562b6a6db0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f5797fff640 1 -- 172.21.15.60:0/2098721775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a67f2f0 msgr2=0x562b6a6dbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f5797fff640 1 --2- 172.21.15.60:0/2098721775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a67f2f0 0x562b6a6dbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f5797fff640 1 -- 172.21.15.60:0/2098721775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b6a4c8de0 con 0x562b6a67e9d0 2024-04-25T10:08:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f5797fff640 1 --2- 172.21.15.60:0/2098721775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a67e9d0 0x562b6a6db630 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5788009010 tx=0x7f578801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f0a3cd79622f32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f5794ff9640 1 -- 172.21.15.60:0/2098721775 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578800e8d0 con 0x562b6a67e9d0 2024-04-25T10:08:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f5794ff9640 1 -- 172.21.15.60:0/2098721775 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578800ee30 con 0x562b6a67e9d0 2024-04-25T10:08:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.346+0000 7f5794ff9640 1 -- 172.21.15.60:0/2098721775 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788013420 con 0x562b6a67e9d0 2024-04-25T10:08:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 -- 172.21.15.60:0/2098721775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a67e9d0 msgr2=0x562b6a6db630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/2098721775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a67e9d0 0x562b6a6db630 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5788009010 tx=0x7f578801f000 comp rx=0 tx=0).stop 2024-04-25T10:08:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 -- 172.21.15.60:0/2098721775 shutdown_connections 2024-04-25T10:08:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/2098721775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a67f2f0 0x562b6a6dbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/2098721775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a67e9d0 0x562b6a6db630 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/2098721775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a6350a0 0x562b6a6db0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 -- 172.21.15.60:0/2098721775 >> 172.21.15.60:0/2098721775 conn(0x562b6a631020 msgr2=0x562b6a68bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 -- 172.21.15.60:0/2098721775 shutdown_connections 2024-04-25T10:08:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 -- 172.21.15.60:0/2098721775 wait complete. 2024-04-25T10:08:54.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 -- start start 2024-04-25T10:08:54.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 Processor -- start 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a6350a0 0x562b6a690fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a67e9d0 0x562b6a6914e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a67f2f0 0x562b6a691a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b6a41ce00 con 0x562b6a6350a0 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00b3a70 con 0x562b6a67e9d0 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b6a481300 con 0x562b6a67f2f0 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 10 client.0 _ll_register_callbacks cb 0x562b6a600040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f579effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a6350a0 0x562b6a690fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f579effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a6350a0 0x562b6a690fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55794/0 (socket says 172.21.15.60:55794) 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f579effd640 1 -- 172.21.15.60:0/3714698038 learned_addr learned my addr 172.21.15.60:0/3714698038 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f5797fff640 1 --2- 172.21.15.60:0/3714698038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a67e9d0 0x562b6a6914e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f579f7fe640 1 --2- 172.21.15.60:0/3714698038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a67f2f0 0x562b6a691a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f579effd640 1 -- 172.21.15.60:0/3714698038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a67f2f0 msgr2=0x562b6a691a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f579effd640 1 --2- 172.21.15.60:0/3714698038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a67f2f0 0x562b6a691a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f579effd640 1 -- 172.21.15.60:0/3714698038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a67e9d0 msgr2=0x562b6a6914e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f579effd640 1 --2- 172.21.15.60:0/3714698038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a67e9d0 0x562b6a6914e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f579effd640 1 -- 172.21.15.60:0/3714698038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b6a4238e0 con 0x562b6a6350a0 2024-04-25T10:08:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f579effd640 1 --2- 172.21.15.60:0/3714698038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a6350a0 0x562b6a690fa0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f57900554f0 tx=0x7f5790006d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38053baedf3584f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f577e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:08:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f577e7fc640 1 -- 172.21.15.60:0/3714698038 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790021040 con 0x562b6a6350a0 2024-04-25T10:08:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f577e7fc640 1 -- 172.21.15.60:0/3714698038 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5790051bd0 con 0x562b6a6350a0 2024-04-25T10:08:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 -- 172.21.15.60:0/3714698038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f578800ee30 con 0x562b6a6350a0 2024-04-25T10:08:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f577e7fc640 1 -- 172.21.15.60:0/3714698038 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f579005dd00 con 0x562b6a6350a0 2024-04-25T10:08:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 10 client.6570 fetch_fsmap learned FSMap version 29 2024-04-25T10:08:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 -- 172.21.15.60:0/3714698038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x562b6a4db8f0 con 0x562b6a6350a0 2024-04-25T10:08:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f577e7fc640 1 -- 172.21.15.60:0/3714698038 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578800ee30 con 0x562b6a6350a0 2024-04-25T10:08:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f577e7fc640 1 -- 172.21.15.60:0/3714698038 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5790012050 con 0x562b6a6350a0 2024-04-25T10:08:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 10 client.6570 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:08:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 10 client.6570 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 20 client.6570 populate_metadata read hostname 'smithi060' 2024-04-25T10:08:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/3714698038 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x562b6a692680 0x562b6a6e6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 4 client.6570 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:08:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 -- 172.21.15.60:0/3714698038 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562b6a48f530 con 0x562b6a692680 2024-04-25T10:08:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f5797fff640 1 --2- 172.21.15.60:0/3714698038 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x562b6a692680 0x562b6a6e6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f577e7fc640 10 client.6570 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f5797fff640 1 --2- 172.21.15.60:0/3714698038 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x562b6a692680 0x562b6a6e6f60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c99652323186b7b8 server_cookie=291869a2df1d9f92 in_seq=0 out_seq=0 2024-04-25T10:08:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f577e7fc640 1 -- 172.21.15.60:0/3714698038 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562b6a48f530 con 0x562b6a692680 2024-04-25T10:08:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f577e7fc640 10 client.6570 handle_command_reply: tid=0 2024-04-25T10:08:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 client.6570 shutdown 2024-04-25T10:08:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 -- 172.21.15.60:0/3714698038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a6350a0 msgr2=0x562b6a690fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/3714698038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a6350a0 0x562b6a690fa0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f57900554f0 tx=0x7f5790006d80 comp rx=0 tx=0).stop 2024-04-25T10:08:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 -- 172.21.15.60:0/3714698038 shutdown_connections 2024-04-25T10:08:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/3714698038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562b6a67f2f0 0x562b6a691a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/3714698038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562b6a67e9d0 0x562b6a6914e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/3714698038 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x562b6a692680 0x562b6a6e6f60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 --2- 172.21.15.60:0/3714698038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562b6a6350a0 0x562b6a690fa0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 -- 172.21.15.60:0/3714698038 >> 172.21.15.60:0/3714698038 conn(0x562b6a631020 msgr2=0x562b6a67d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f577e7fc640 0 client.6570 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.350+0000 7f57a8ad6000 1 -- 172.21.15.60:0/3714698038 shutdown_connections 2024-04-25T10:08:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.354+0000 7f57a8ad6000 1 -- 172.21.15.60:0/3714698038 wait complete. 2024-04-25T10:08:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.354+0000 7f57a8ad6000 20 client.6570 trim_cache size 0 max 16384 2024-04-25T10:08:54.367 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:08:54.367 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T10:08:54.368 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T10:08:54.368 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:08:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.354+0000 7f57a5969640 1 -- 172.21.15.60:0/2089632433 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5780066f60 msgr2=0x7f5780069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.354+0000 7f57a5969640 1 --2- 172.21.15.60:0/2089632433 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5780066f60 0x7f5780069420 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f57a015aea0 tx=0x7f5788009040 comp rx=0 tx=0).stop 2024-04-25T10:08:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.354+0000 7f57a5969640 1 -- 172.21.15.60:0/2089632433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00ceef0 msgr2=0x7f57a015a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.354+0000 7f57a5969640 1 --2- 172.21.15.60:0/2089632433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00ceef0 0x7f57a015a960 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f579000e810 tx=0x7f579000ecd0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.354+0000 7f57a5969640 1 -- 172.21.15.60:0/2089632433 shutdown_connections 2024-04-25T10:08:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.354+0000 7f57a5969640 1 --2- 172.21.15.60:0/2089632433 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5780066f60 0x7f5780069420 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.354+0000 7f57a5969640 1 --2- 172.21.15.60:0/2089632433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a015e0d0 0x7f57a00c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.354+0000 7f57a5969640 1 --2- 172.21.15.60:0/2089632433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a015b6f0 0x7f57a00c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.354+0000 7f57a5969640 1 --2- 172.21.15.60:0/2089632433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00ceef0 0x7f57a015a960 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.354+0000 7f57a5969640 1 -- 172.21.15.60:0/2089632433 >> 172.21.15.60:0/2089632433 conn(0x7f57a0072340 msgr2=0x7f57a00d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.354+0000 7f57a5969640 1 -- 172.21.15.60:0/2089632433 shutdown_connections 2024-04-25T10:08:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.354+0000 7f57a5969640 1 -- 172.21.15.60:0/2089632433 wait complete. 2024-04-25T10:08:54.377 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:08:54.378 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.482+0000 7fb24df9c640 1 Processor -- start 2024-04-25T10:08:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.482+0000 7fb24df9c640 1 -- start start 2024-04-25T10:08:54.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.482+0000 7fb24df9c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb248162250 0x7fb2480b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.482+0000 7fb24df9c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2480b34f0 0x7fb248071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.482+0000 7fb24df9c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb2480717d0 0x7fb24806d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.482+0000 7fb24df9c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2480af0c0 con 0x7fb248162250 2024-04-25T10:08:54.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.482+0000 7fb24df9c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb248071c40 con 0x7fb2480717d0 2024-04-25T10:08:54.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.482+0000 7fb24df9c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480b3aa0 con 0x7fb2480b34f0 2024-04-25T10:08:54.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb2477fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb248162250 0x7fb2480b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb2477fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb248162250 0x7fb2480b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55810/0 (socket says 172.21.15.60:55810) 2024-04-25T10:08:54.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb2477fe640 1 -- 172.21.15.60:0/3975797314 learned_addr learned my addr 172.21.15.60:0/3975797314 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:54.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb246ffd640 1 --2- 172.21.15.60:0/3975797314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2480b34f0 0x7fb248071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb247fff640 1 --2- 172.21.15.60:0/3975797314 >> v2:172.21.15.100:3300/0 conn(0x7fb2480717d0 0x7fb24806d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb2477fe640 1 -- 172.21.15.60:0/3975797314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2480b34f0 msgr2=0x7fb248071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb2477fe640 1 --2- 172.21.15.60:0/3975797314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2480b34f0 0x7fb248071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb2477fe640 1 -- 172.21.15.60:0/3975797314 >> v2:172.21.15.100:3300/0 conn(0x7fb2480717d0 msgr2=0x7fb24806d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb2477fe640 1 --2- 172.21.15.60:0/3975797314 >> v2:172.21.15.100:3300/0 conn(0x7fb2480717d0 0x7fb24806d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb2477fe640 1 -- 172.21.15.60:0/3975797314 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2480b3d80 con 0x7fb248162250 2024-04-25T10:08:54.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb2477fe640 1 --2- 172.21.15.60:0/3975797314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb248162250 0x7fb2480b2fb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb2480cf4b0 tx=0x7fb24001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d447185dbe2bf2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb2467fc640 1 -- 172.21.15.60:0/3975797314 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2400b4070 con 0x7fb248162250 2024-04-25T10:08:54.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb2467fc640 1 -- 172.21.15.60:0/3975797314 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2400af400 con 0x7fb248162250 2024-04-25T10:08:54.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb2467fc640 1 -- 172.21.15.60:0/3975797314 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2400af7d0 con 0x7fb248162250 2024-04-25T10:08:54.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb24df9c640 1 -- 172.21.15.60:0/3975797314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb248162250 msgr2=0x7fb2480b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb24df9c640 1 --2- 172.21.15.60:0/3975797314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb248162250 0x7fb2480b2fb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb2480cf4b0 tx=0x7fb24001f1e0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb24df9c640 1 -- 172.21.15.60:0/3975797314 shutdown_connections 2024-04-25T10:08:54.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb24df9c640 1 --2- 172.21.15.60:0/3975797314 >> v2:172.21.15.100:3300/0 conn(0x7fb2480717d0 0x7fb24806d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb24df9c640 1 --2- 172.21.15.60:0/3975797314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2480b34f0 0x7fb248071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb24df9c640 1 --2- 172.21.15.60:0/3975797314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb248162250 0x7fb2480b2fb0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.486+0000 7fb24df9c640 1 -- 172.21.15.60:0/3975797314 >> 172.21.15.60:0/3975797314 conn(0x7fb2480b12d0 msgr2=0x7fb2480b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb24df9c640 1 -- 172.21.15.60:0/3975797314 shutdown_connections 2024-04-25T10:08:54.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb24df9c640 1 -- 172.21.15.60:0/3975797314 wait complete. 2024-04-25T10:08:54.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb24df9c640 1 Processor -- start 2024-04-25T10:08:54.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb24df9c640 1 -- start start 2024-04-25T10:08:54.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb24df9c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2480717d0 0x7fb2481608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb24df9c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2480b34f0 0x7fb24815e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb24df9c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb248162250 0x7fb24815e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb24df9c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24806e5a0 con 0x7fb2480b34f0 2024-04-25T10:08:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb24df9c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480b4340 con 0x7fb2480717d0 2024-04-25T10:08:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb24df9c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2480cc440 con 0x7fb248162250 2024-04-25T10:08:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb246ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2480b34f0 0x7fb24815e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb246ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2480b34f0 0x7fb24815e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55826/0 (socket says 172.21.15.60:55826) 2024-04-25T10:08:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb246ffd640 1 -- 172.21.15.60:0/2872114509 learned_addr learned my addr 172.21.15.60:0/2872114509 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb247fff640 1 --2- 172.21.15.60:0/2872114509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb248162250 0x7fb24815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb2477fe640 1 --2- 172.21.15.60:0/2872114509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2480717d0 0x7fb2481608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb246ffd640 1 -- 172.21.15.60:0/2872114509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb248162250 msgr2=0x7fb24815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb246ffd640 1 --2- 172.21.15.60:0/2872114509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb248162250 0x7fb24815e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb246ffd640 1 -- 172.21.15.60:0/2872114509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2480717d0 msgr2=0x7fb2481608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb246ffd640 1 --2- 172.21.15.60:0/2872114509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2480717d0 0x7fb2481608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb246ffd640 1 -- 172.21.15.60:0/2872114509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24815ec40 con 0x7fb2480b34f0 2024-04-25T10:08:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb2477fe640 1 --2- 172.21.15.60:0/2872114509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2480717d0 0x7fb2481608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb246ffd640 1 --2- 172.21.15.60:0/2872114509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2480b34f0 0x7fb24815e1c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00d800 tx=0x7fb23c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb22ffff640 1 -- 172.21.15.60:0/2872114509 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c00b6c0 con 0x7fb2480b34f0 2024-04-25T10:08:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb22ffff640 1 -- 172.21.15.60:0/2872114509 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23c019070 con 0x7fb2480b34f0 2024-04-25T10:08:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb22ffff640 1 -- 172.21.15.60:0/2872114509 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c00ba60 con 0x7fb2480b34f0 2024-04-25T10:08:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb24df9c640 1 -- 172.21.15.60:0/2872114509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2481fb010 con 0x7fb2480b34f0 2024-04-25T10:08:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.490+0000 7fb24df9c640 1 -- 172.21.15.60:0/2872114509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2481fb5d0 con 0x7fb2480b34f0 2024-04-25T10:08:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.494+0000 7fb24df9c640 1 -- 172.21.15.60:0/2872114509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2480afcf0 con 0x7fb2480b34f0 2024-04-25T10:08:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.494+0000 7fb22ffff640 1 -- 172.21.15.60:0/2872114509 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb23c00bc00 con 0x7fb2480b34f0 2024-04-25T10:08:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.494+0000 7fb22ffff640 1 --2- 172.21.15.60:0/2872114509 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb234066f60 0x7fb234069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.494+0000 7fb22ffff640 1 -- 172.21.15.60:0/2872114509 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb23c0853c0 con 0x7fb2480b34f0 2024-04-25T10:08:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.494+0000 7fb2477fe640 1 --2- 172.21.15.60:0/2872114509 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb234066f60 0x7fb234069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.494+0000 7fb2477fe640 1 --2- 172.21.15.60:0/2872114509 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb234066f60 0x7fb234069420 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb2400b1000 tx=0x7fb2400aa000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.498+0000 7fb22ffff640 1 -- 172.21.15.60:0/2872114509 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb23c056460 con 0x7fb2480b34f0 2024-04-25T10:08:54.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.690+0000 7fb24df9c640 1 -- 172.21.15.60:0/2872114509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb24806c250 con 0x7fb2480b34f0 2024-04-25T10:08:54.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.690+0000 7fb22ffff640 1 -- 172.21.15.60:0/2872114509 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fb23c055d60 con 0x7fb2480b34f0 2024-04-25T10:08:54.697 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:54.697 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/380273391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":380273391},{"type":"v1","addr":"172.21.15.100:6836","nonce":380273391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5635,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2965977166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2965977166},{"type":"v1","addr":"172.21.15.100:6837","nonce":2965977166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1105028629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1105028629},{"type":"v1","addr":"172.21.15.60:6834","nonce":1105028629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:10.236590+0000","modified":"2024-04-25T10:08:26.930556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6835/2718963870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2718963870},{"type":"v1","addr":"172.21.15.60:6835","nonce":2718963870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:08:54.697 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 29 2024-04-25T10:08:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.690+0000 7fb24df9c640 1 -- 172.21.15.60:0/2872114509 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb234066f60 msgr2=0x7fb234069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.690+0000 7fb24df9c640 1 --2- 172.21.15.60:0/2872114509 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb234066f60 0x7fb234069420 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb2400b1000 tx=0x7fb2400aa000 comp rx=0 tx=0).stop 2024-04-25T10:08:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.690+0000 7fb24df9c640 1 -- 172.21.15.60:0/2872114509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2480b34f0 msgr2=0x7fb24815e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.690+0000 7fb24df9c640 1 --2- 172.21.15.60:0/2872114509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2480b34f0 0x7fb24815e1c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00d800 tx=0x7fb23c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.690+0000 7fb24df9c640 1 -- 172.21.15.60:0/2872114509 shutdown_connections 2024-04-25T10:08:54.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.690+0000 7fb24df9c640 1 --2- 172.21.15.60:0/2872114509 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb234066f60 0x7fb234069420 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.690+0000 7fb24df9c640 1 --2- 172.21.15.60:0/2872114509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb248162250 0x7fb24815e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.690+0000 7fb24df9c640 1 --2- 172.21.15.60:0/2872114509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2480b34f0 0x7fb24815e1c0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.690+0000 7fb24df9c640 1 --2- 172.21.15.60:0/2872114509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2480717d0 0x7fb2481608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.690+0000 7fb24df9c640 1 -- 172.21.15.60:0/2872114509 >> 172.21.15.60:0/2872114509 conn(0x7fb2480b12d0 msgr2=0x7fb248164640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.690+0000 7fb24df9c640 1 -- 172.21.15.60:0/2872114509 shutdown_connections 2024-04-25T10:08:54.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.690+0000 7fb24df9c640 1 -- 172.21.15.60:0/2872114509 wait complete. 2024-04-25T10:08:54.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:08:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.834+0000 7fd652b18640 1 Processor -- start 2024-04-25T10:08:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.834+0000 7fd652b18640 1 -- start start 2024-04-25T10:08:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.834+0000 7fd652b18640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd64c0cbcf0 0x7fd64c160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.834+0000 7fd652b18640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64c144a40 con 0x7fd64c15fbf0 2024-04-25T10:08:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.834+0000 7fd652b18640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64c0cc9f0 con 0x7fd64c071c30 2024-04-25T10:08:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.834+0000 7fd652b18640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64c0b2dc0 con 0x7fd64c0cbcf0 2024-04-25T10:08:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.834+0000 7fd6510b5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd64c15fbf0 0x7fd64c0ceec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54440/0 (socket says 172.21.15.60:54440) 2024-04-25T10:08:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.834+0000 7fd6510b5640 1 -- 172.21.15.60:0/4145945063 learned_addr learned my addr 172.21.15.60:0/4145945063 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.834+0000 7fd6508b4640 1 --2- 172.21.15.60:0/4145945063 >> v2:172.21.15.60:3300/0 conn(0x7fd64c0cbcf0 0x7fd64c160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.834+0000 7fd6508b4640 1 -- 172.21.15.60:0/4145945063 >> v1:172.21.15.100:6789/0 conn(0x7fd64c071c30 legacy=0x7fd64c15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.834+0000 7fd6508b4640 1 -- 172.21.15.60:0/4145945063 >> v1:172.21.15.60:6789/0 conn(0x7fd64c15fbf0 legacy=0x7fd64c0ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.834+0000 7fd6508b4640 1 -- 172.21.15.60:0/4145945063 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c0cccd0 con 0x7fd64c0cbcf0 2024-04-25T10:08:54.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd6508b4640 1 --2- 172.21.15.60:0/4145945063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64c0cbcf0 0x7fd64c160080 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009d90 tx=0x7fd63c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70526c51a542bf84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd64b7fe640 1 -- 172.21.15.60:0/4145945063 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0b2070 con 0x7fd64c0cbcf0 2024-04-25T10:08:54.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd64b7fe640 1 -- 172.21.15.60:0/4145945063 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63c0a59c0 con 0x7fd64c0cbcf0 2024-04-25T10:08:54.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd64b7fe640 1 -- 172.21.15.60:0/4145945063 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0a5d90 con 0x7fd64c0cbcf0 2024-04-25T10:08:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd652b18640 1 -- 172.21.15.60:0/4145945063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64c0cbcf0 msgr2=0x7fd64c160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd652b18640 1 --2- 172.21.15.60:0/4145945063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64c0cbcf0 0x7fd64c160080 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009d90 tx=0x7fd63c01c900 comp rx=0 tx=0).stop 2024-04-25T10:08:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd652b18640 1 -- 172.21.15.60:0/4145945063 shutdown_connections 2024-04-25T10:08:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd652b18640 1 --2- 172.21.15.60:0/4145945063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64c0cbcf0 0x7fd64c160080 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd652b18640 1 -- 172.21.15.60:0/4145945063 >> 172.21.15.60:0/4145945063 conn(0x7fd64c0670b0 msgr2=0x7fd64c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd652b18640 1 -- 172.21.15.60:0/4145945063 shutdown_connections 2024-04-25T10:08:54.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd652b18640 1 -- 172.21.15.60:0/4145945063 wait complete. 2024-04-25T10:08:54.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd652b18640 1 Processor -- start 2024-04-25T10:08:54.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd652b18640 1 -- start start 2024-04-25T10:08:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd652b18640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd64c071c30 0x7fd64c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd652b18640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd64c0cbcf0 0x7fd64c140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd652b18640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64c15fbf0 0x7fd64c1423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd652b18640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0ce5c0 con 0x7fd64c15fbf0 2024-04-25T10:08:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd652b18640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0720b0 con 0x7fd64c071c30 2024-04-25T10:08:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd652b18640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0ce740 con 0x7fd64c0cbcf0 2024-04-25T10:08:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.838+0000 7fd6510b5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64c15fbf0 0x7fd64c1423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd6510b5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64c15fbf0 0x7fd64c1423f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55836/0 (socket says 172.21.15.60:55836) 2024-04-25T10:08:54.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd6510b5640 1 -- 172.21.15.60:0/2332553262 learned_addr learned my addr 172.21.15.60:0/2332553262 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:54.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd64bfff640 1 --2- 172.21.15.60:0/2332553262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd64c0cbcf0 0x7fd64c140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd6508b4640 1 --2- 172.21.15.60:0/2332553262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd64c071c30 0x7fd64c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd6510b5640 1 -- 172.21.15.60:0/2332553262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd64c0cbcf0 msgr2=0x7fd64c140e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd6510b5640 1 --2- 172.21.15.60:0/2332553262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd64c0cbcf0 0x7fd64c140e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd6510b5640 1 -- 172.21.15.60:0/2332553262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd64c071c30 msgr2=0x7fd64c15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd6510b5640 1 --2- 172.21.15.60:0/2332553262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd64c071c30 0x7fd64c15b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd6510b5640 1 -- 172.21.15.60:0/2332553262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c141870 con 0x7fd64c15fbf0 2024-04-25T10:08:54.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd6510b5640 1 --2- 172.21.15.60:0/2332553262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64c15fbf0 0x7fd64c1423f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd64400c3c0 tx=0x7fd64400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd6497fa640 1 -- 172.21.15.60:0/2332553262 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd644017070 con 0x7fd64c15fbf0 2024-04-25T10:08:54.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd6497fa640 1 -- 172.21.15.60:0/2332553262 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd644005cf0 con 0x7fd64c15fbf0 2024-04-25T10:08:54.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd6497fa640 1 -- 172.21.15.60:0/2332553262 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd644007310 con 0x7fd64c15fbf0 2024-04-25T10:08:54.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd652b18640 1 -- 172.21.15.60:0/2332553262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64c141b50 con 0x7fd64c15fbf0 2024-04-25T10:08:54.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd652b18640 1 -- 172.21.15.60:0/2332553262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c15b7c0 con 0x7fd64c15fbf0 2024-04-25T10:08:54.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd6497fa640 1 -- 172.21.15.60:0/2332553262 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd64401a070 con 0x7fd64c15fbf0 2024-04-25T10:08:54.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd6497fa640 1 --2- 172.21.15.60:0/2332553262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd62c066f60 0x7fd62c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd6497fa640 1 -- 172.21.15.60:0/2332553262 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6440842d0 con 0x7fd64c15fbf0 2024-04-25T10:08:54.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd6508b4640 1 --2- 172.21.15.60:0/2332553262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd62c066f60 0x7fd62c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd655c85000 1 Processor -- start 2024-04-25T10:08:54.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.842+0000 7fd655c85000 1 -- start start 2024-04-25T10:08:54.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd6508b4640 1 --2- 172.21.15.60:0/2332553262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd62c066f60 0x7fd62c069420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0af000 tx=0x7fd63c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704da32e0 0x557704da36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da3c00 0x557704d593a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704d598e0 0x557704d59cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557704c40ba0 con 0x557704da3c00 2024-04-25T10:08:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557704ba6090 con 0x557704d598e0 2024-04-25T10:08:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x557704b2e160 con 0x557704da32e0 2024-04-25T10:08:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da3c00 0x557704d593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da3c00 0x557704d593a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55844/0 (socket says 172.21.15.60:55844) 2024-04-25T10:08:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 -- 172.21.15.60:0/1659839640 learned_addr learned my addr 172.21.15.60:0/1659839640 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 -- 172.21.15.60:0/1659839640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704da32e0 msgr2=0x557704da36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 --2- 172.21.15.60:0/1659839640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704da32e0 0x557704da36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 -- 172.21.15.60:0/1659839640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704d598e0 msgr2=0x557704d59cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 --2- 172.21.15.60:0/1659839640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704d598e0 0x557704d59cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 -- 172.21.15.60:0/1659839640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557704c4fc20 con 0x557704da3c00 2024-04-25T10:08:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 --2- 172.21.15.60:0/1659839640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da3c00 0x557704d593a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd634004db0 tx=0x7fd634005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d3bfee746ff34a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd6427fc640 1 -- 172.21.15.60:0/1659839640 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340054c0 con 0x557704da3c00 2024-04-25T10:08:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd6427fc640 1 -- 172.21.15.60:0/1659839640 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd634018070 con 0x557704da3c00 2024-04-25T10:08:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd6427fc640 1 -- 172.21.15.60:0/1659839640 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634007480 con 0x557704da3c00 2024-04-25T10:08:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 -- 172.21.15.60:0/1659839640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da3c00 msgr2=0x557704d593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 --2- 172.21.15.60:0/1659839640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da3c00 0x557704d593a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd634004db0 tx=0x7fd634005270 comp rx=0 tx=0).stop 2024-04-25T10:08:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 -- 172.21.15.60:0/1659839640 shutdown_connections 2024-04-25T10:08:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 --2- 172.21.15.60:0/1659839640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704d598e0 0x557704d59cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 --2- 172.21.15.60:0/1659839640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da3c00 0x557704d593a0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 --2- 172.21.15.60:0/1659839640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704da32e0 0x557704da36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 -- 172.21.15.60:0/1659839640 >> 172.21.15.60:0/1659839640 conn(0x557704d55890 msgr2=0x557704d57c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 -- 172.21.15.60:0/1659839640 shutdown_connections 2024-04-25T10:08:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 -- 172.21.15.60:0/1659839640 wait complete. 2024-04-25T10:08:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 -- start start 2024-04-25T10:08:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 Processor -- start 2024-04-25T10:08:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704d598e0 0x557704daa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da32e0 0x557704daab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704da3c00 0x557704d91090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557704ba2130 con 0x557704da32e0 2024-04-25T10:08:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557704b6a130 con 0x557704da3c00 2024-04-25T10:08:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x557704c64300 con 0x557704d598e0 2024-04-25T10:08:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd655c85000 10 client.0 _ll_register_callbacks cb 0x557704d44bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da32e0 0x557704daab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da32e0 0x557704daab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55846/0 (socket says 172.21.15.60:55846) 2024-04-25T10:08:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 -- 172.21.15.60:0/2906417405 learned_addr learned my addr 172.21.15.60:0/2906417405 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd6508b4640 1 --2- 172.21.15.60:0/2906417405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704d598e0 0x557704daa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 -- 172.21.15.60:0/2906417405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704d598e0 msgr2=0x557704daa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 --2- 172.21.15.60:0/2906417405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704d598e0 0x557704daa640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 -- 172.21.15.60:0/2906417405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704da3c00 msgr2=0x557704d91090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 --2- 172.21.15.60:0/2906417405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704da3c00 0x557704d91090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.846+0000 7fd64bfff640 1 -- 172.21.15.60:0/2906417405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557704b83060 con 0x557704da32e0 2024-04-25T10:08:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd64bfff640 1 --2- 172.21.15.60:0/2906417405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da32e0 0x557704daab80 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd634015040 tx=0x7fd634003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a224538d76fd98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd62b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:08:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd62b7fe640 1 -- 172.21.15.60:0/2906417405 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634021070 con 0x557704da32e0 2024-04-25T10:08:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd62b7fe640 1 -- 172.21.15.60:0/2906417405 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6340067e0 con 0x557704da32e0 2024-04-25T10:08:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 1 -- 172.21.15.60:0/2906417405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd634018070 con 0x557704da32e0 2024-04-25T10:08:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd62b7fe640 1 -- 172.21.15.60:0/2906417405 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd63402e050 con 0x557704da32e0 2024-04-25T10:08:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 10 client.6591 fetch_fsmap learned FSMap version 29 2024-04-25T10:08:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 1 -- 172.21.15.60:0/2906417405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x557704b477c0 con 0x557704da32e0 2024-04-25T10:08:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd62b7fe640 1 -- 172.21.15.60:0/2906417405 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd634018070 con 0x557704da32e0 2024-04-25T10:08:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd62b7fe640 1 -- 172.21.15.60:0/2906417405 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd634031020 con 0x557704da32e0 2024-04-25T10:08:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 10 client.6591 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:08:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 10 client.6591 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:08:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 20 client.6591 populate_metadata read hostname 'smithi060' 2024-04-25T10:08:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 1 --2- 172.21.15.60:0/2906417405 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x557704d91e30 0x557704d92210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 4 client.6591 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 1 -- 172.21.15.60:0/2906417405 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557704b41cd0 con 0x557704d91e30 2024-04-25T10:08:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd6508b4640 1 --2- 172.21.15.60:0/2906417405 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x557704d91e30 0x557704d92210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd6508b4640 1 --2- 172.21.15.60:0/2906417405 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x557704d91e30 0x557704d92210 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b9141f6ccbed454 server_cookie=1ff503a39db05bd7 in_seq=0 out_seq=0 2024-04-25T10:08:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd62b7fe640 10 client.6591 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd62b7fe640 1 -- 172.21.15.60:0/2906417405 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557704b41cd0 con 0x557704d91e30 2024-04-25T10:08:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd62b7fe640 10 client.6591 handle_command_reply: tid=0 2024-04-25T10:08:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 1 client.6591 shutdown 2024-04-25T10:08:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 1 -- 172.21.15.60:0/2906417405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da32e0 msgr2=0x557704daab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 1 --2- 172.21.15.60:0/2906417405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da32e0 0x557704daab80 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd634015040 tx=0x7fd634003700 comp rx=0 tx=0).stop 2024-04-25T10:08:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 1 -- 172.21.15.60:0/2906417405 shutdown_connections 2024-04-25T10:08:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 1 --2- 172.21.15.60:0/2906417405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704da3c00 0x557704d91090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 1 --2- 172.21.15.60:0/2906417405 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x557704d91e30 0x557704d92210 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 1 --2- 172.21.15.60:0/2906417405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da32e0 0x557704daab80 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 1 --2- 172.21.15.60:0/2906417405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704d598e0 0x557704daa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd62b7fe640 0 client.6591 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 1 -- 172.21.15.60:0/2906417405 >> 172.21.15.60:0/2906417405 conn(0x557704d55890 msgr2=0x557704d58510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 1 -- 172.21.15.60:0/2906417405 shutdown_connections 2024-04-25T10:08:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 1 -- 172.21.15.60:0/2906417405 wait complete. 2024-04-25T10:08:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.850+0000 7fd655c85000 20 client.6591 trim_cache size 0 max 16384 2024-04-25T10:08:54.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd655c85000 1 Processor -- start 2024-04-25T10:08:54.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd655c85000 1 -- start start 2024-04-25T10:08:54.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd655c85000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704d598e0 0x557704dff710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd655c85000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704da32e0 0x557704dffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd655c85000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704da3c00 0x557704e00190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd655c85000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c200000 con 0x557704d598e0 2024-04-25T10:08:54.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd655c85000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c1ff540 con 0x557704da3c00 2024-04-25T10:08:54.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd655c85000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c1ffa30 con 0x557704da32e0 2024-04-25T10:08:54.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd6508b4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704d598e0 0x557704dff710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd6508b4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704d598e0 0x557704dff710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55862/0 (socket says 172.21.15.60:55862) 2024-04-25T10:08:54.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd6508b4640 1 -- 172.21.15.60:0/2793497011 learned_addr learned my addr 172.21.15.60:0/2793497011 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:54.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd6508b4640 1 -- 172.21.15.60:0/2793497011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704da32e0 msgr2=0x557704dffc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd6508b4640 1 --2- 172.21.15.60:0/2793497011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704da32e0 0x557704dffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd6508b4640 1 -- 172.21.15.60:0/2793497011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704da3c00 msgr2=0x557704e00190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd6508b4640 1 --2- 172.21.15.60:0/2793497011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704da3c00 0x557704e00190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd6508b4640 1 -- 172.21.15.60:0/2793497011 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557704b61fc0 con 0x557704d598e0 2024-04-25T10:08:54.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.862+0000 7fd6508b4640 1 --2- 172.21.15.60:0/2793497011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704d598e0 0x557704dff710 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009bd0 tx=0x7fd63c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8d9363315c0b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd641ffb640 1 -- 172.21.15.60:0/2793497011 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0b2040 con 0x557704d598e0 2024-04-25T10:08:54.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd641ffb640 1 -- 172.21.15.60:0/2793497011 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63c0cd070 con 0x557704d598e0 2024-04-25T10:08:54.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd641ffb640 1 -- 172.21.15.60:0/2793497011 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0c9630 con 0x557704d598e0 2024-04-25T10:08:54.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 -- 172.21.15.60:0/2793497011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704d598e0 msgr2=0x557704dff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 --2- 172.21.15.60:0/2793497011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704d598e0 0x557704dff710 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009bd0 tx=0x7fd63c004ac0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 -- 172.21.15.60:0/2793497011 shutdown_connections 2024-04-25T10:08:54.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 --2- 172.21.15.60:0/2793497011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704da3c00 0x557704e00190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 --2- 172.21.15.60:0/2793497011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704da32e0 0x557704dffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 --2- 172.21.15.60:0/2793497011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704d598e0 0x557704dff710 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 -- 172.21.15.60:0/2793497011 >> 172.21.15.60:0/2793497011 conn(0x557704d55890 msgr2=0x557704da2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 -- 172.21.15.60:0/2793497011 shutdown_connections 2024-04-25T10:08:54.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 -- 172.21.15.60:0/2793497011 wait complete. 2024-04-25T10:08:54.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 -- start start 2024-04-25T10:08:54.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 Processor -- start 2024-04-25T10:08:54.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704d598e0 0x557704e04170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704da32e0 0x557704e046b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da3c00 0x557704da84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557704b40e70 con 0x557704da3c00 2024-04-25T10:08:54.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557704b2ea50 con 0x557704da32e0 2024-04-25T10:08:54.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x557704ba53a0 con 0x557704d598e0 2024-04-25T10:08:54.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 10 client.0 _ll_register_callbacks cb 0x557704b52240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:54.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd6510b5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da3c00 0x557704da84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd6510b5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da3c00 0x557704da84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55876/0 (socket says 172.21.15.60:55876) 2024-04-25T10:08:54.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd6510b5640 1 -- 172.21.15.60:0/2056928834 learned_addr learned my addr 172.21.15.60:0/2056928834 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:54.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd6508b4640 1 --2- 172.21.15.60:0/2056928834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704d598e0 0x557704e04170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd64bfff640 1 --2- 172.21.15.60:0/2056928834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704da32e0 0x557704e046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd6510b5640 1 -- 172.21.15.60:0/2056928834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704d598e0 msgr2=0x557704e04170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd6510b5640 1 --2- 172.21.15.60:0/2056928834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704d598e0 0x557704e04170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd6510b5640 1 -- 172.21.15.60:0/2056928834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704da32e0 msgr2=0x557704e046b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd6510b5640 1 --2- 172.21.15.60:0/2056928834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704da32e0 0x557704e046b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd6510b5640 1 -- 172.21.15.60:0/2056928834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557704bff9e0 con 0x557704da3c00 2024-04-25T10:08:54.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd64bfff640 1 --2- 172.21.15.60:0/2056928834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704da32e0 0x557704e046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd6510b5640 1 --2- 172.21.15.60:0/2056928834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da3c00 0x557704da84c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd6440548d0 tx=0x7fd6440547d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad0bc9ef0dd364d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd62b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:08:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd62b7fe640 1 -- 172.21.15.60:0/2056928834 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd644082dc0 con 0x557704da3c00 2024-04-25T10:08:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd62b7fe640 1 -- 172.21.15.60:0/2056928834 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6440074b0 con 0x557704da3c00 2024-04-25T10:08:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 -- 172.21.15.60:0/2056928834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd63c0cd070 con 0x557704da3c00 2024-04-25T10:08:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd62b7fe640 1 -- 172.21.15.60:0/2056928834 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd644054bd0 con 0x557704da3c00 2024-04-25T10:08:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 10 client.6597 fetch_fsmap learned FSMap version 29 2024-04-25T10:08:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 -- 172.21.15.60:0/2056928834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x557704b45630 con 0x557704da3c00 2024-04-25T10:08:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd62b7fe640 1 -- 172.21.15.60:0/2056928834 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63c0cd070 con 0x557704da3c00 2024-04-25T10:08:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd62b7fe640 1 -- 172.21.15.60:0/2056928834 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd644068b90 con 0x557704da3c00 2024-04-25T10:08:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 10 client.6597 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:08:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 10 client.6597 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:08:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 20 client.6597 populate_metadata read hostname 'smithi060' 2024-04-25T10:08:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 --2- 172.21.15.60:0/2056928834 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x557704e0afd0 0x557704da8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 4 client.6597 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:08:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd655c85000 1 -- 172.21.15.60:0/2056928834 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x557704bb35d0 con 0x557704e0afd0 2024-04-25T10:08:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.866+0000 7fd6508b4640 1 --2- 172.21.15.60:0/2056928834 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x557704e0afd0 0x557704da8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd6508b4640 1 --2- 172.21.15.60:0/2056928834 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x557704e0afd0 0x557704da8fa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12e55082ce7ca3e7 server_cookie=d7dfba97e32b25dc in_seq=0 out_seq=0 2024-04-25T10:08:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd62b7fe640 10 client.6597 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd62b7fe640 1 -- 172.21.15.60:0/2056928834 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x557704bb35d0 con 0x557704e0afd0 2024-04-25T10:08:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd62b7fe640 10 client.6597 handle_command_reply: tid=0 2024-04-25T10:08:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd655c85000 1 client.6597 shutdown 2024-04-25T10:08:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd655c85000 1 -- 172.21.15.60:0/2056928834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da3c00 msgr2=0x557704da84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd655c85000 1 --2- 172.21.15.60:0/2056928834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da3c00 0x557704da84c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd6440548d0 tx=0x7fd6440547d0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd655c85000 1 -- 172.21.15.60:0/2056928834 shutdown_connections 2024-04-25T10:08:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd655c85000 1 --2- 172.21.15.60:0/2056928834 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x557704e0afd0 0x557704da8fa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd655c85000 1 --2- 172.21.15.60:0/2056928834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557704da3c00 0x557704da84c0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd655c85000 1 --2- 172.21.15.60:0/2056928834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557704da32e0 0x557704e046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd655c85000 1 --2- 172.21.15.60:0/2056928834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557704d598e0 0x557704e04170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd62b7fe640 0 client.6597 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:54.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd655c85000 1 -- 172.21.15.60:0/2056928834 >> 172.21.15.60:0/2056928834 conn(0x557704d55890 msgr2=0x557704da2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd655c85000 1 -- 172.21.15.60:0/2056928834 shutdown_connections 2024-04-25T10:08:54.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd655c85000 1 -- 172.21.15.60:0/2056928834 wait complete. 2024-04-25T10:08:54.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd655c85000 20 client.6597 trim_cache size 0 max 16384 2024-04-25T10:08:54.882 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:54.882 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T10:08:54.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:08:54.882 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-04-25T10:08:54.882 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 1067748375, 2024-04-25T10:08:54.882 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511627777, 2024-04-25T10:08:54.883 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/dir_test/file" 2024-04-25T10:08:54.883 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:54.883 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T10:08:54.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd652b18640 1 -- 172.21.15.60:0/2332553262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd62c066f60 msgr2=0x7fd62c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd652b18640 1 --2- 172.21.15.60:0/2332553262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd62c066f60 0x7fd62c069420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0af000 tx=0x7fd63c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:08:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd652b18640 1 -- 172.21.15.60:0/2332553262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64c15fbf0 msgr2=0x7fd64c1423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd652b18640 1 --2- 172.21.15.60:0/2332553262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64c15fbf0 0x7fd64c1423f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd64400c3c0 tx=0x7fd64400c880 comp rx=0 tx=0).stop 2024-04-25T10:08:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd652b18640 1 -- 172.21.15.60:0/2332553262 shutdown_connections 2024-04-25T10:08:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd652b18640 1 --2- 172.21.15.60:0/2332553262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd62c066f60 0x7fd62c069420 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd652b18640 1 --2- 172.21.15.60:0/2332553262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64c15fbf0 0x7fd64c1423f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd652b18640 1 --2- 172.21.15.60:0/2332553262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd64c0cbcf0 0x7fd64c140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd652b18640 1 --2- 172.21.15.60:0/2332553262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd64c071c30 0x7fd64c15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd652b18640 1 -- 172.21.15.60:0/2332553262 >> 172.21.15.60:0/2332553262 conn(0x7fd64c0670b0 msgr2=0x7fd64c13c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd652b18640 1 -- 172.21.15.60:0/2332553262 shutdown_connections 2024-04-25T10:08:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.870+0000 7fd652b18640 1 -- 172.21.15.60:0/2332553262 wait complete. 2024-04-25T10:08:54.893 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:08:55.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.998+0000 7f43307fb640 1 Processor -- start 2024-04-25T10:08:55.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:54.998+0000 7f43307fb640 1 -- start start 2024-04-25T10:08:55.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.002+0000 7f43307fb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43281583f0 0x7f43280cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.002+0000 7f43307fb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432815bd20 con 0x7f432815b450 2024-04-25T10:08:55.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.002+0000 7f43307fb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43280cd170 con 0x7f4328072370 2024-04-25T10:08:55.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.002+0000 7f43307fb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43281587d0 con 0x7f43281583f0 2024-04-25T10:08:55.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.002+0000 7f432ed98640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f432815b450 0x7f43280b40c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54446/0 (socket says 172.21.15.60:54446) 2024-04-25T10:08:55.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.002+0000 7f432ed98640 1 -- 172.21.15.60:0/4267294334 learned_addr learned my addr 172.21.15.60:0/4267294334 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:55.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.002+0000 7f432e597640 1 --2- 172.21.15.60:0/4267294334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43281583f0 0x7f43280cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.002+0000 7f432d595640 1 -- 172.21.15.60:0/4267294334 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481746920 0 0) 0x7f432815bd20 con 0x7f432815b450 2024-04-25T10:08:55.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.002+0000 7f432d595640 1 -- 172.21.15.60:0/4267294334 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4314003530 con 0x7f432815b450 2024-04-25T10:08:55.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.002+0000 7f432e597640 1 -- 172.21.15.60:0/4267294334 >> v1:172.21.15.100:6789/0 conn(0x7f4328072370 legacy=0x7f432815ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.002+0000 7f432d595640 1 -- 172.21.15.60:0/4267294334 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903791863 0 0) 0x7f43280cd170 con 0x7f4328072370 2024-04-25T10:08:55.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.002+0000 7f432e597640 1 -- 172.21.15.60:0/4267294334 >> v1:172.21.15.60:6789/0 conn(0x7f432815b450 legacy=0x7f43280b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.002+0000 7f432e597640 1 -- 172.21.15.60:0/4267294334 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432815ba40 con 0x7f43281583f0 2024-04-25T10:08:55.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.002+0000 7f432e597640 1 --2- 172.21.15.60:0/4267294334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43281583f0 0x7f43280cdc10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f43240097f0 tx=0x7f432401c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e1141056544f95c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f432d595640 1 -- 172.21.15.60:0/4267294334 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43240b0070 con 0x7f43281583f0 2024-04-25T10:08:55.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f432d595640 1 -- 172.21.15.60:0/4267294334 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43240ab600 con 0x7f43281583f0 2024-04-25T10:08:55.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f432d595640 1 -- 172.21.15.60:0/4267294334 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43240ab9b0 con 0x7f43281583f0 2024-04-25T10:08:55.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f43307fb640 1 -- 172.21.15.60:0/4267294334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43281583f0 msgr2=0x7f43280cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f43307fb640 1 --2- 172.21.15.60:0/4267294334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43281583f0 0x7f43280cdc10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f43240097f0 tx=0x7f432401c420 comp rx=0 tx=0).stop 2024-04-25T10:08:55.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f43307fb640 1 -- 172.21.15.60:0/4267294334 shutdown_connections 2024-04-25T10:08:55.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f43307fb640 1 --2- 172.21.15.60:0/4267294334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43281583f0 0x7f43280cdc10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f43307fb640 1 -- 172.21.15.60:0/4267294334 >> 172.21.15.60:0/4267294334 conn(0x7f432806e400 msgr2=0x7f432806e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f43307fb640 1 -- 172.21.15.60:0/4267294334 shutdown_connections 2024-04-25T10:08:55.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f43307fb640 1 -- 172.21.15.60:0/4267294334 wait complete. 2024-04-25T10:08:55.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f43307fb640 1 Processor -- start 2024-04-25T10:08:55.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f43307fb640 1 -- start start 2024-04-25T10:08:55.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f43307fb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4328072370 0x7f4328158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f43307fb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43281583f0 0x7f432814a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f43307fb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432815b450 0x7f43280ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f43307fb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43280ce2d0 con 0x7f432815b450 2024-04-25T10:08:55.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f43307fb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43280ce150 con 0x7f43281583f0 2024-04-25T10:08:55.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f43307fb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4328168bd0 con 0x7f4328072370 2024-04-25T10:08:55.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f432ed98640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432815b450 0x7f43280ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f432ed98640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432815b450 0x7f43280ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55882/0 (socket says 172.21.15.60:55882) 2024-04-25T10:08:55.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f432ed98640 1 -- 172.21.15.60:0/1737378491 learned_addr learned my addr 172.21.15.60:0/1737378491 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:55.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f432e597640 1 --2- 172.21.15.60:0/1737378491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4328072370 0x7f4328158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f432dd96640 1 --2- 172.21.15.60:0/1737378491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43281583f0 0x7f432814a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f432ed98640 1 -- 172.21.15.60:0/1737378491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4328072370 msgr2=0x7f4328158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f432ed98640 1 --2- 172.21.15.60:0/1737378491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4328072370 0x7f4328158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f432ed98640 1 -- 172.21.15.60:0/1737378491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43281583f0 msgr2=0x7f432814a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f432ed98640 1 --2- 172.21.15.60:0/1737378491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43281583f0 0x7f432814a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f432ed98640 1 -- 172.21.15.60:0/1737378491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4328159910 con 0x7f432815b450 2024-04-25T10:08:55.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f432ed98640 1 --2- 172.21.15.60:0/1737378491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432815b450 0x7f43280ac8a0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f432000cb40 tx=0x7f432000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f431effd640 1 -- 172.21.15.60:0/1737378491 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4320017070 con 0x7f432815b450 2024-04-25T10:08:55.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f431effd640 1 -- 172.21.15.60:0/1737378491 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432000aca0 con 0x7f432815b450 2024-04-25T10:08:55.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f431effd640 1 -- 172.21.15.60:0/1737378491 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4320012620 con 0x7f432815b450 2024-04-25T10:08:55.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.006+0000 7f43307fb640 1 -- 172.21.15.60:0/1737378491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4328159070 con 0x7f432815b450 2024-04-25T10:08:55.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.010+0000 7f43307fb640 1 -- 172.21.15.60:0/1737378491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4328159630 con 0x7f432815b450 2024-04-25T10:08:55.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.010+0000 7f43307fb640 1 -- 172.21.15.60:0/1737378491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4328074770 con 0x7f432815b450 2024-04-25T10:08:55.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.010+0000 7f431effd640 1 -- 172.21.15.60:0/1737378491 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f432001a070 con 0x7f432815b450 2024-04-25T10:08:55.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.010+0000 7f431effd640 1 --2- 172.21.15.60:0/1737378491 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f430c066f60 0x7f430c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.010+0000 7f431effd640 1 -- 172.21.15.60:0/1737378491 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4320084d90 con 0x7f432815b450 2024-04-25T10:08:55.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.010+0000 7f432e597640 1 --2- 172.21.15.60:0/1737378491 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f430c066f60 0x7f430c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.010+0000 7f432e597640 1 --2- 172.21.15.60:0/1737378491 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f430c066f60 0x7f430c069420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f43240ad000 tx=0x7f43240a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.014+0000 7f431effd640 1 -- 172.21.15.60:0/1737378491 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4320055760 con 0x7f432815b450 2024-04-25T10:08:55.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.206+0000 7f43307fb640 1 -- 172.21.15.60:0/1737378491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f43280cce20 con 0x7f432815b450 2024-04-25T10:08:55.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.206+0000 7f431effd640 1 -- 172.21.15.60:0/1737378491 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f4320059800 con 0x7f432815b450 2024-04-25T10:08:55.216 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:08:55.216 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:08:55.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.210+0000 7f43307fb640 1 -- 172.21.15.60:0/1737378491 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f430c066f60 msgr2=0x7f430c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.210+0000 7f43307fb640 1 --2- 172.21.15.60:0/1737378491 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f430c066f60 0x7f430c069420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f43240ad000 tx=0x7f43240a6000 comp rx=0 tx=0).stop 2024-04-25T10:08:55.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.210+0000 7f43307fb640 1 -- 172.21.15.60:0/1737378491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432815b450 msgr2=0x7f43280ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.210+0000 7f43307fb640 1 --2- 172.21.15.60:0/1737378491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432815b450 0x7f43280ac8a0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f432000cb40 tx=0x7f432000a060 comp rx=0 tx=0).stop 2024-04-25T10:08:55.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.210+0000 7f43307fb640 1 -- 172.21.15.60:0/1737378491 shutdown_connections 2024-04-25T10:08:55.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.210+0000 7f43307fb640 1 --2- 172.21.15.60:0/1737378491 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f430c066f60 0x7f430c069420 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.210+0000 7f43307fb640 1 --2- 172.21.15.60:0/1737378491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432815b450 0x7f43280ac8a0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.210+0000 7f43307fb640 1 --2- 172.21.15.60:0/1737378491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43281583f0 0x7f432814a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.210+0000 7f43307fb640 1 --2- 172.21.15.60:0/1737378491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4328072370 0x7f4328158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.210+0000 7f43307fb640 1 -- 172.21.15.60:0/1737378491 >> 172.21.15.60:0/1737378491 conn(0x7f432806e400 msgr2=0x7f43280b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.210+0000 7f43307fb640 1 -- 172.21.15.60:0/1737378491 shutdown_connections 2024-04-25T10:08:55.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.210+0000 7f43307fb640 1 -- 172.21.15.60:0/1737378491 wait complete. 2024-04-25T10:08:55.233 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:08:55.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:55.233 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-04-25T10:08:55.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d3747640 1 Processor -- start 2024-04-25T10:08:55.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d3747640 1 -- start start 2024-04-25T10:08:55.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d3747640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27cc144e40 0x7f27cc0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d3747640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f27cc0cf0a0 0x7f27cc071270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d3747640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f27cc0717b0 0x7f27cc15f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d3747640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27cc0b3060 con 0x7f27cc0cf0a0 2024-04-25T10:08:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d3747640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27cc0cf810 con 0x7f27cc0717b0 2024-04-25T10:08:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d3747640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc0cfaf0 con 0x7f27cc144e40 2024-04-25T10:08:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d0ce2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f27cc0cf0a0 0x7f27cc071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d0ce2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f27cc0cf0a0 0x7f27cc071270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55886/0 (socket says 172.21.15.60:55886) 2024-04-25T10:08:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d0ce2640 1 -- 172.21.15.60:0/2741804994 learned_addr learned my addr 172.21.15.60:0/2741804994 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d1ce4640 1 --2- 172.21.15.60:0/2741804994 >> v2:172.21.15.100:3300/0 conn(0x7f27cc0717b0 0x7f27cc15f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d14e3640 1 --2- 172.21.15.60:0/2741804994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27cc144e40 0x7f27cc0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d0ce2640 1 -- 172.21.15.60:0/2741804994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27cc144e40 msgr2=0x7f27cc0ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d0ce2640 1 --2- 172.21.15.60:0/2741804994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27cc144e40 0x7f27cc0ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d0ce2640 1 -- 172.21.15.60:0/2741804994 >> v2:172.21.15.100:3300/0 conn(0x7f27cc0717b0 msgr2=0x7f27cc15f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d0ce2640 1 --2- 172.21.15.60:0/2741804994 >> v2:172.21.15.100:3300/0 conn(0x7f27cc0717b0 0x7f27cc15f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.354+0000 7f27d0ce2640 1 -- 172.21.15.60:0/2741804994 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc0cfdd0 con 0x7f27cc0cf0a0 2024-04-25T10:08:55.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d0ce2640 1 --2- 172.21.15.60:0/2741804994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27cc0cf0a0 0x7f27cc071270 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0b26f0 tx=0x7f27bc01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c945bda4de40db0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27c3fff640 1 -- 172.21.15.60:0/2741804994 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc0b0070 con 0x7f27cc0cf0a0 2024-04-25T10:08:55.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27c3fff640 1 -- 172.21.15.60:0/2741804994 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27cc145430 con 0x7f27cc0cf0a0 2024-04-25T10:08:55.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27c3fff640 1 -- 172.21.15.60:0/2741804994 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc0ab820 con 0x7f27cc0cf0a0 2024-04-25T10:08:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 -- 172.21.15.60:0/2741804994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27cc0cf0a0 msgr2=0x7f27cc071270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 --2- 172.21.15.60:0/2741804994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27cc0cf0a0 0x7f27cc071270 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0b26f0 tx=0x7f27bc01c260 comp rx=0 tx=0).stop 2024-04-25T10:08:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 -- 172.21.15.60:0/2741804994 shutdown_connections 2024-04-25T10:08:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 --2- 172.21.15.60:0/2741804994 >> v2:172.21.15.100:3300/0 conn(0x7f27cc0717b0 0x7f27cc15f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 --2- 172.21.15.60:0/2741804994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27cc0cf0a0 0x7f27cc071270 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 --2- 172.21.15.60:0/2741804994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27cc144e40 0x7f27cc0ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 -- 172.21.15.60:0/2741804994 >> 172.21.15.60:0/2741804994 conn(0x7f27cc066fa0 msgr2=0x7f27cc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 -- 172.21.15.60:0/2741804994 shutdown_connections 2024-04-25T10:08:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 -- 172.21.15.60:0/2741804994 wait complete. 2024-04-25T10:08:55.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 Processor -- start 2024-04-25T10:08:55.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 -- start start 2024-04-25T10:08:55.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27cc0717b0 0x7f27cc13de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27cc0cf0a0 0x7f27cc159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27cc144e40 0x7f27cc15a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc0b3860 con 0x7f27cc0cf0a0 2024-04-25T10:08:55.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc0b36e0 con 0x7f27cc0717b0 2024-04-25T10:08:55.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d3747640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc0b39e0 con 0x7f27cc144e40 2024-04-25T10:08:55.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d0ce2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27cc0cf0a0 0x7f27cc159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d0ce2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27cc0cf0a0 0x7f27cc159ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55888/0 (socket says 172.21.15.60:55888) 2024-04-25T10:08:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.358+0000 7f27d0ce2640 1 -- 172.21.15.60:0/3431859032 learned_addr learned my addr 172.21.15.60:0/3431859032 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27d14e3640 1 --2- 172.21.15.60:0/3431859032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27cc0717b0 0x7f27cc13de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27d1ce4640 1 --2- 172.21.15.60:0/3431859032 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27cc144e40 0x7f27cc15a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27d0ce2640 1 -- 172.21.15.60:0/3431859032 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27cc144e40 msgr2=0x7f27cc15a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27d0ce2640 1 --2- 172.21.15.60:0/3431859032 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27cc144e40 0x7f27cc15a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27d0ce2640 1 -- 172.21.15.60:0/3431859032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27cc0717b0 msgr2=0x7f27cc13de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27d0ce2640 1 --2- 172.21.15.60:0/3431859032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27cc0717b0 0x7f27cc13de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27d0ce2640 1 -- 172.21.15.60:0/3431859032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc13e830 con 0x7f27cc0cf0a0 2024-04-25T10:08:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27d1ce4640 1 --2- 172.21.15.60:0/3431859032 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27cc144e40 0x7f27cc15a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27d0ce2640 1 --2- 172.21.15.60:0/3431859032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27cc0cf0a0 0x7f27cc159ac0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0ae040 tx=0x7f27bc01aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27c1ffb640 1 -- 172.21.15.60:0/3431859032 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc0b0040 con 0x7f27cc0cf0a0 2024-04-25T10:08:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27c1ffb640 1 -- 172.21.15.60:0/3431859032 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc0bc070 con 0x7f27cc0cf0a0 2024-04-25T10:08:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27d3747640 1 -- 172.21.15.60:0/3431859032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27cc13eb10 con 0x7f27cc0cf0a0 2024-04-25T10:08:55.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27c1ffb640 1 -- 172.21.15.60:0/3431859032 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc0b6660 con 0x7f27cc0cf0a0 2024-04-25T10:08:55.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27d3747640 1 -- 172.21.15.60:0/3431859032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27cc13f0d0 con 0x7f27cc0cf0a0 2024-04-25T10:08:55.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27c1ffb640 1 -- 172.21.15.60:0/3431859032 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27bc0b6800 con 0x7f27cc0cf0a0 2024-04-25T10:08:55.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27c1ffb640 1 --2- 172.21.15.60:0/3431859032 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f27b0066f60 0x7f27b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27c1ffb640 1 -- 172.21.15.60:0/3431859032 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27bc11f430 con 0x7f27cc0cf0a0 2024-04-25T10:08:55.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27d14e3640 1 --2- 172.21.15.60:0/3431859032 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f27b0066f60 0x7f27b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27d68b4000 1 Processor -- start 2024-04-25T10:08:55.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.362+0000 7f27d68b4000 1 -- start start 2024-04-25T10:08:55.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d14e3640 1 --2- 172.21.15.60:0/3431859032 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f27b0066f60 0x7f27b0069420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f27cc13fc30 tx=0x7f27c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561859864520 0x561859864900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561859864e40 0x56185981b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56185981b970 0x56185981bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561859668d00 con 0x56185981b970 2024-04-25T10:08:55.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618595f0df0 con 0x561859864520 2024-04-25T10:08:55.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x561859664da0 con 0x561859864e40 2024-04-25T10:08:55.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d1ce4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56185981b970 0x56185981bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d1ce4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56185981b970 0x56185981bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55892/0 (socket says 172.21.15.60:55892) 2024-04-25T10:08:55.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d1ce4640 1 -- 172.21.15.60:0/2895794145 learned_addr learned my addr 172.21.15.60:0/2895794145 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:55.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d14e3640 1 --2- 172.21.15.60:0/2895794145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561859864520 0x561859864900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d0ce2640 1 --2- 172.21.15.60:0/2895794145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561859864e40 0x56185981b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d1ce4640 1 -- 172.21.15.60:0/2895794145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561859864e40 msgr2=0x56185981b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d1ce4640 1 --2- 172.21.15.60:0/2895794145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561859864e40 0x56185981b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d1ce4640 1 -- 172.21.15.60:0/2895794145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561859864520 msgr2=0x561859864900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d1ce4640 1 --2- 172.21.15.60:0/2895794145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561859864520 0x561859864900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d1ce4640 1 -- 172.21.15.60:0/2895794145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561859623580 con 0x56185981b970 2024-04-25T10:08:55.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d1ce4640 1 --2- 172.21.15.60:0/2895794145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56185981b970 0x56185981bd50 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f27c40073e0 tx=0x7f27c40078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af815afb6cf5c06a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27b6ffd640 1 -- 172.21.15.60:0/2895794145 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c401a070 con 0x56185981b970 2024-04-25T10:08:55.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27b6ffd640 1 -- 172.21.15.60:0/2895794145 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c4003db0 con 0x56185981b970 2024-04-25T10:08:55.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27b6ffd640 1 -- 172.21.15.60:0/2895794145 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4004230 con 0x56185981b970 2024-04-25T10:08:55.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 -- 172.21.15.60:0/2895794145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56185981b970 msgr2=0x56185981bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 --2- 172.21.15.60:0/2895794145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56185981b970 0x56185981bd50 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f27c40073e0 tx=0x7f27c40078a0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 -- 172.21.15.60:0/2895794145 shutdown_connections 2024-04-25T10:08:55.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 --2- 172.21.15.60:0/2895794145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56185981b970 0x56185981bd50 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 --2- 172.21.15.60:0/2895794145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561859864e40 0x56185981b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 --2- 172.21.15.60:0/2895794145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561859864520 0x561859864900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 -- 172.21.15.60:0/2895794145 >> 172.21.15.60:0/2895794145 conn(0x561859702c10 msgr2=0x561859819d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 -- 172.21.15.60:0/2895794145 shutdown_connections 2024-04-25T10:08:55.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 -- 172.21.15.60:0/2895794145 wait complete. 2024-04-25T10:08:55.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 -- start start 2024-04-25T10:08:55.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 Processor -- start 2024-04-25T10:08:55.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56185981b970 0x56185986aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561859864520 0x56185986b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561859864e40 0x56185981fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618596016d0 con 0x56185981b970 2024-04-25T10:08:55.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561859623950 con 0x561859864520 2024-04-25T10:08:55.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x561859668010 con 0x561859864e40 2024-04-25T10:08:55.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d68b4000 10 client.0 _ll_register_callbacks cb 0x561859807030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:55.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d14e3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56185981b970 0x56185986aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d14e3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56185981b970 0x56185986aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55904/0 (socket says 172.21.15.60:55904) 2024-04-25T10:08:55.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d14e3640 1 -- 172.21.15.60:0/1452557526 learned_addr learned my addr 172.21.15.60:0/1452557526 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:55.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d0ce2640 1 --2- 172.21.15.60:0/1452557526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561859864520 0x56185986b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d1ce4640 1 --2- 172.21.15.60:0/1452557526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561859864e40 0x56185981fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d14e3640 1 -- 172.21.15.60:0/1452557526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561859864e40 msgr2=0x56185981fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d14e3640 1 --2- 172.21.15.60:0/1452557526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561859864e40 0x56185981fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d14e3640 1 -- 172.21.15.60:0/1452557526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561859864520 msgr2=0x56185986b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d14e3640 1 --2- 172.21.15.60:0/1452557526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561859864520 0x56185986b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.366+0000 7f27d14e3640 1 -- 172.21.15.60:0/1452557526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561859605d40 con 0x56185981b970 2024-04-25T10:08:55.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d14e3640 1 --2- 172.21.15.60:0/1452557526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56185981b970 0x56185986aaf0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f27c800e010 tx=0x7f27c8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1faf0363bb23cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27abfff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:08:55.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27abfff640 1 -- 172.21.15.60:0/1452557526 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8025020 con 0x56185981b970 2024-04-25T10:08:55.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27abfff640 1 -- 172.21.15.60:0/1452557526 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27c80158e0 con 0x56185981b970 2024-04-25T10:08:55.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 1 -- 172.21.15.60:0/1452557526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27c4003db0 con 0x56185981b970 2024-04-25T10:08:55.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27abfff640 1 -- 172.21.15.60:0/1452557526 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c802f050 con 0x56185981b970 2024-04-25T10:08:55.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 10 client.6615 fetch_fsmap learned FSMap version 29 2024-04-25T10:08:55.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 1 -- 172.21.15.60:0/1452557526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x561859663cf0 con 0x56185981b970 2024-04-25T10:08:55.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27abfff640 1 -- 172.21.15.60:0/1452557526 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c4003db0 con 0x56185981b970 2024-04-25T10:08:55.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27abfff640 1 -- 172.21.15.60:0/1452557526 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f27c8032020 con 0x56185981b970 2024-04-25T10:08:55.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 10 client.6615 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:08:55.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 10 client.6615 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:55.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 20 client.6615 populate_metadata read hostname 'smithi060' 2024-04-25T10:08:55.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 1 --2- 172.21.15.60:0/1452557526 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x561859820ba0 0x561859820fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 4 client.6615 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:55.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 1 -- 172.21.15.60:0/1452557526 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618596049d0 con 0x561859820ba0 2024-04-25T10:08:55.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d0ce2640 1 --2- 172.21.15.60:0/1452557526 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x561859820ba0 0x561859820fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27abfff640 10 client.6615 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:55.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d0ce2640 1 --2- 172.21.15.60:0/1452557526 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x561859820ba0 0x561859820fb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0885b3c0e5353f server_cookie=775f3c67bd7270c1 in_seq=0 out_seq=0 2024-04-25T10:08:55.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27abfff640 1 -- 172.21.15.60:0/1452557526 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5618596049d0 con 0x561859820ba0 2024-04-25T10:08:55.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27abfff640 10 client.6615 handle_command_reply: tid=0 2024-04-25T10:08:55.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 1 client.6615 shutdown 2024-04-25T10:08:55.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 1 -- 172.21.15.60:0/1452557526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56185981b970 msgr2=0x56185986aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 1 --2- 172.21.15.60:0/1452557526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56185981b970 0x56185986aaf0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f27c800e010 tx=0x7f27c8004740 comp rx=0 tx=0).stop 2024-04-25T10:08:55.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 1 -- 172.21.15.60:0/1452557526 shutdown_connections 2024-04-25T10:08:55.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 1 --2- 172.21.15.60:0/1452557526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561859864e40 0x56185981fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 1 --2- 172.21.15.60:0/1452557526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561859864520 0x56185986b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 1 --2- 172.21.15.60:0/1452557526 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x561859820ba0 0x561859820fb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 1 --2- 172.21.15.60:0/1452557526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56185981b970 0x56185986aaf0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27abfff640 0 client.6615 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:55.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 1 -- 172.21.15.60:0/1452557526 >> 172.21.15.60:0/1452557526 conn(0x561859702c10 msgr2=0x56185981a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 1 -- 172.21.15.60:0/1452557526 shutdown_connections 2024-04-25T10:08:55.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 1 -- 172.21.15.60:0/1452557526 wait complete. 2024-04-25T10:08:55.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.370+0000 7f27d68b4000 20 client.6615 trim_cache size 0 max 16384 2024-04-25T10:08:55.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 Processor -- start 2024-04-25T10:08:55.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 -- start start 2024-04-25T10:08:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56185981b970 0x5618598c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561859864520 0x5618598c4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561859864e40 0x5618598c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc2009a0 con 0x561859864e40 2024-04-25T10:08:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc1ffb60 con 0x561859864520 2024-04-25T10:08:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc1fef90 con 0x56185981b970 2024-04-25T10:08:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d1ce4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561859864e40 0x5618598c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d1ce4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561859864e40 0x5618598c4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55906/0 (socket says 172.21.15.60:55906) 2024-04-25T10:08:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d1ce4640 1 -- 172.21.15.60:0/227569962 learned_addr learned my addr 172.21.15.60:0/227569962 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d14e3640 1 --2- 172.21.15.60:0/227569962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56185981b970 0x5618598c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d0ce2640 1 --2- 172.21.15.60:0/227569962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561859864520 0x5618598c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d1ce4640 1 -- 172.21.15.60:0/227569962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56185981b970 msgr2=0x5618598c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d1ce4640 1 --2- 172.21.15.60:0/227569962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56185981b970 0x5618598c41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d1ce4640 1 -- 172.21.15.60:0/227569962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561859864520 msgr2=0x5618598c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d1ce4640 1 --2- 172.21.15.60:0/227569962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561859864520 0x5618598c4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d1ce4640 1 -- 172.21.15.60:0/227569962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561859864c70 con 0x561859864e40 2024-04-25T10:08:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d1ce4640 1 --2- 172.21.15.60:0/227569962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561859864e40 0x5618598c4c70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009f70 tx=0x7f27c4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13db940b1a640ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27b67fc640 1 -- 172.21.15.60:0/227569962 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4026070 con 0x561859864e40 2024-04-25T10:08:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27b67fc640 1 -- 172.21.15.60:0/227569962 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c4015940 con 0x561859864e40 2024-04-25T10:08:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27b67fc640 1 -- 172.21.15.60:0/227569962 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c401a040 con 0x561859864e40 2024-04-25T10:08:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 -- 172.21.15.60:0/227569962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561859864e40 msgr2=0x5618598c4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 --2- 172.21.15.60:0/227569962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561859864e40 0x5618598c4c70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009f70 tx=0x7f27c4013000 comp rx=0 tx=0).stop 2024-04-25T10:08:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 -- 172.21.15.60:0/227569962 shutdown_connections 2024-04-25T10:08:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 --2- 172.21.15.60:0/227569962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561859864e40 0x5618598c4c70 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 --2- 172.21.15.60:0/227569962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561859864520 0x5618598c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 --2- 172.21.15.60:0/227569962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56185981b970 0x5618598c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 -- 172.21.15.60:0/227569962 >> 172.21.15.60:0/227569962 conn(0x561859702c10 msgr2=0x5618598635f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 -- 172.21.15.60:0/227569962 shutdown_connections 2024-04-25T10:08:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.386+0000 7f27d68b4000 1 -- 172.21.15.60:0/227569962 wait complete. 2024-04-25T10:08:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 1 -- start start 2024-04-25T10:08:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 1 Processor -- start 2024-04-25T10:08:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56185981b970 0x5618598aca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561859864520 0x5618598ad290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561859864e40 0x5618598ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185966d330 con 0x561859864e40 2024-04-25T10:08:55.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc1ff560 con 0x56185981b970 2024-04-25T10:08:55.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56185962d210 con 0x561859864520 2024-04-25T10:08:55.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 10 client.0 _ll_register_callbacks cb 0x7f27c4009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:55.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d1ce4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561859864e40 0x5618598ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d1ce4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561859864e40 0x5618598ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55908/0 (socket says 172.21.15.60:55908) 2024-04-25T10:08:55.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d1ce4640 1 -- 172.21.15.60:0/3657843580 learned_addr learned my addr 172.21.15.60:0/3657843580 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:55.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d14e3640 1 --2- 172.21.15.60:0/3657843580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56185981b970 0x5618598aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d0ce2640 1 --2- 172.21.15.60:0/3657843580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561859864520 0x5618598ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d1ce4640 1 -- 172.21.15.60:0/3657843580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561859864520 msgr2=0x5618598ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d1ce4640 1 --2- 172.21.15.60:0/3657843580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561859864520 0x5618598ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d1ce4640 1 -- 172.21.15.60:0/3657843580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56185981b970 msgr2=0x5618598aca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d1ce4640 1 --2- 172.21.15.60:0/3657843580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56185981b970 0x5618598aca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d1ce4640 1 -- 172.21.15.60:0/3657843580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618596ebd30 con 0x561859864e40 2024-04-25T10:08:55.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d1ce4640 1 --2- 172.21.15.60:0/3657843580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561859864e40 0x5618598ad7d0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009f70 tx=0x7f27c4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117db7f4e379df6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27abfff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:08:55.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27abfff640 1 -- 172.21.15.60:0/3657843580 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4026070 con 0x561859864e40 2024-04-25T10:08:55.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 1 -- 172.21.15.60:0/3657843580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27c4015940 con 0x561859864e40 2024-04-25T10:08:55.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27abfff640 1 -- 172.21.15.60:0/3657843580 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27c4015d00 con 0x561859864e40 2024-04-25T10:08:55.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27abfff640 1 -- 172.21.15.60:0/3657843580 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c403a050 con 0x561859864e40 2024-04-25T10:08:55.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 10 client.6621 fetch_fsmap learned FSMap version 29 2024-04-25T10:08:55.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 1 -- 172.21.15.60:0/3657843580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5618596355a0 con 0x561859864e40 2024-04-25T10:08:55.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27abfff640 1 -- 172.21.15.60:0/3657843580 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c4015940 con 0x561859864e40 2024-04-25T10:08:55.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27abfff640 1 -- 172.21.15.60:0/3657843580 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f27c403d020 con 0x561859864e40 2024-04-25T10:08:55.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 10 client.6621 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:08:55.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 10 client.6621 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:55.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 20 client.6621 populate_metadata read hostname 'smithi060' 2024-04-25T10:08:55.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 1 --2- 172.21.15.60:0/3657843580 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x5618598add10 0x5618598ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 4 client.6621 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-04-25T10:08:55.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 1 -- 172.21.15.60:0/3657843580 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) v1 -- 0x561859645610 con 0x5618598add10 2024-04-25T10:08:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d14e3640 1 --2- 172.21.15.60:0/3657843580 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x5618598add10 0x5618598ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d14e3640 1 --2- 172.21.15.60:0/3657843580 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x5618598add10 0x5618598ae0f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ecc23597d1abe44 server_cookie=e6a5fcea9c6ada02 in_seq=0 out_seq=0 2024-04-25T10:08:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27abfff640 10 client.6621 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27abfff640 1 -- 172.21.15.60:0/3657843580 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561859645610 con 0x5618598add10 2024-04-25T10:08:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27abfff640 10 client.6621 handle_command_reply: tid=0 2024-04-25T10:08:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.390+0000 7f27d68b4000 1 client.6621 shutdown 2024-04-25T10:08:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d68b4000 1 -- 172.21.15.60:0/3657843580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561859864e40 msgr2=0x5618598ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d68b4000 1 --2- 172.21.15.60:0/3657843580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561859864e40 0x5618598ad7d0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009f70 tx=0x7f27c4004ac0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d68b4000 1 -- 172.21.15.60:0/3657843580 shutdown_connections 2024-04-25T10:08:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d68b4000 1 --2- 172.21.15.60:0/3657843580 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x5618598add10 0x5618598ae0f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d68b4000 1 --2- 172.21.15.60:0/3657843580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561859864e40 0x5618598ad7d0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d68b4000 1 --2- 172.21.15.60:0/3657843580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561859864520 0x5618598ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d68b4000 1 --2- 172.21.15.60:0/3657843580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56185981b970 0x5618598aca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27abfff640 0 client.6621 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d68b4000 1 -- 172.21.15.60:0/3657843580 >> 172.21.15.60:0/3657843580 conn(0x561859702c10 msgr2=0x561859862d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d68b4000 1 -- 172.21.15.60:0/3657843580 shutdown_connections 2024-04-25T10:08:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d68b4000 1 -- 172.21.15.60:0/3657843580 wait complete. 2024-04-25T10:08:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d68b4000 20 client.6621 trim_cache size 0 max 16384 2024-04-25T10:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T10:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "8ffc5464-8236-4520-82f8-780abc0c451d", 2024-04-25T10:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T10:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:08:55.407 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f2736337640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-25T10:08:22.850813+0000 1=1+0) (iversion lock) caps={6489=pAsLsXsFs/-@2},l=6489(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x56343c443b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:08:55.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d3747640 1 -- 172.21.15.60:0/3431859032 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f27b0066f60 msgr2=0x7f27b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d3747640 1 --2- 172.21.15.60:0/3431859032 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f27b0066f60 0x7f27b0069420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f27cc13fc30 tx=0x7f27c8008040 comp rx=0 tx=0).stop 2024-04-25T10:08:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d3747640 1 -- 172.21.15.60:0/3431859032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27cc0cf0a0 msgr2=0x7f27cc159ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d3747640 1 --2- 172.21.15.60:0/3431859032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27cc0cf0a0 0x7f27cc159ac0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0ae040 tx=0x7f27bc01aaa0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d3747640 1 -- 172.21.15.60:0/3431859032 shutdown_connections 2024-04-25T10:08:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d3747640 1 --2- 172.21.15.60:0/3431859032 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f27b0066f60 0x7f27b0069420 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d3747640 1 --2- 172.21.15.60:0/3431859032 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27cc144e40 0x7f27cc15a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d3747640 1 --2- 172.21.15.60:0/3431859032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27cc0cf0a0 0x7f27cc159ac0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d3747640 1 --2- 172.21.15.60:0/3431859032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27cc0717b0 0x7f27cc13de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d3747640 1 -- 172.21.15.60:0/3431859032 >> 172.21.15.60:0/3431859032 conn(0x7f27cc066fa0 msgr2=0x7f27cc0afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d3747640 1 -- 172.21.15.60:0/3431859032 shutdown_connections 2024-04-25T10:08:55.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.394+0000 7f27d3747640 1 -- 172.21.15.60:0/3431859032 wait complete. 2024-04-25T10:08:55.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:55.417 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:08:55.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.522+0000 7f9cfe311640 1 Processor -- start 2024-04-25T10:08:55.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.522+0000 7f9cfe311640 1 -- start start 2024-04-25T10:08:55.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.522+0000 7f9cfe311640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9cf81450b0 0x7f9cf815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.522+0000 7f9cfe311640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9cf8071960 0x7f9cf80cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.522+0000 7f9cfe311640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf8145490 con 0x7f9cf80b2b90 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.522+0000 7f9cfe311640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf80b3c50 con 0x7f9cf8071960 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.522+0000 7f9cfe311640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf80b3110 con 0x7f9cf81450b0 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cfc8ae640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9cf8071960 0x7f9cf80cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cf77fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9cf80b2b90 0x7f9cf8071250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54454/0 (socket says 172.21.15.60:54454) 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cf77fe640 1 -- 172.21.15.60:0/3405843718 learned_addr learned my addr 172.21.15.60:0/3405843718 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cf7fff640 1 --2- 172.21.15.60:0/3405843718 >> v2:172.21.15.100:3300/0 conn(0x7f9cf81450b0 0x7f9cf815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cfc8ae640 1 -- 172.21.15.60:0/3405843718 >> v2:172.21.15.100:3300/0 conn(0x7f9cf81450b0 msgr2=0x7f9cf815c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cfc8ae640 1 --2- 172.21.15.60:0/3405843718 >> v2:172.21.15.100:3300/0 conn(0x7f9cf81450b0 0x7f9cf815c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cfc8ae640 1 -- 172.21.15.60:0/3405843718 >> v1:172.21.15.60:6789/0 conn(0x7f9cf80b2b90 legacy=0x7f9cf8071250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cfc8ae640 1 -- 172.21.15.60:0/3405843718 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80b33b0 con 0x7f9cf8071960 2024-04-25T10:08:55.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cfc8ae640 1 --2- 172.21.15.60:0/3405843718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cf8071960 0x7f9cf80cba10 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80b4c50 tx=0x7f9cf001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43488cee9318140e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cf6ffd640 1 -- 172.21.15.60:0/3405843718 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00b0070 con 0x7f9cf8071960 2024-04-25T10:08:55.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cf6ffd640 1 -- 172.21.15.60:0/3405843718 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf00ab600 con 0x7f9cf8071960 2024-04-25T10:08:55.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cf6ffd640 1 -- 172.21.15.60:0/3405843718 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00ab9d0 con 0x7f9cf8071960 2024-04-25T10:08:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cfe311640 1 -- 172.21.15.60:0/3405843718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cf8071960 msgr2=0x7f9cf80cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cfe311640 1 --2- 172.21.15.60:0/3405843718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cf8071960 0x7f9cf80cba10 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80b4c50 tx=0x7f9cf001c480 comp rx=0 tx=0).stop 2024-04-25T10:08:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cfe311640 1 -- 172.21.15.60:0/3405843718 shutdown_connections 2024-04-25T10:08:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cfe311640 1 --2- 172.21.15.60:0/3405843718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cf8071960 0x7f9cf80cba10 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cfe311640 1 --2- 172.21.15.60:0/3405843718 >> v2:172.21.15.100:3300/0 conn(0x7f9cf81450b0 0x7f9cf815c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cfe311640 1 -- 172.21.15.60:0/3405843718 >> 172.21.15.60:0/3405843718 conn(0x7f9cf80670b0 msgr2=0x7f9cf806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cfe311640 1 -- 172.21.15.60:0/3405843718 shutdown_connections 2024-04-25T10:08:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.526+0000 7f9cfe311640 1 -- 172.21.15.60:0/3405843718 wait complete. 2024-04-25T10:08:55.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cfe311640 1 Processor -- start 2024-04-25T10:08:55.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cfe311640 1 -- start start 2024-04-25T10:08:55.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cfe311640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cf8071960 0x7f9cf8156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cfe311640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cf80b2b90 0x7f9cf813ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cfe311640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9cf81450b0 0x7f9cf813b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cfe311640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80ccb70 con 0x7f9cf8071960 2024-04-25T10:08:55.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cfe311640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80cc9f0 con 0x7f9cf80b2b90 2024-04-25T10:08:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cfe311640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80cccf0 con 0x7f9cf81450b0 2024-04-25T10:08:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cf7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cf8071960 0x7f9cf8156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cf7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cf8071960 0x7f9cf8156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55930/0 (socket says 172.21.15.60:55930) 2024-04-25T10:08:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cf7fff640 1 -- 172.21.15.60:0/1804911281 learned_addr learned my addr 172.21.15.60:0/1804911281 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cfc8ae640 1 --2- 172.21.15.60:0/1804911281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9cf81450b0 0x7f9cf813b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cf77fe640 1 --2- 172.21.15.60:0/1804911281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cf80b2b90 0x7f9cf813ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cf7fff640 1 -- 172.21.15.60:0/1804911281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9cf81450b0 msgr2=0x7f9cf813b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cf7fff640 1 --2- 172.21.15.60:0/1804911281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9cf81450b0 0x7f9cf813b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cf7fff640 1 -- 172.21.15.60:0/1804911281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cf80b2b90 msgr2=0x7f9cf813ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cf7fff640 1 --2- 172.21.15.60:0/1804911281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cf80b2b90 0x7f9cf813ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cf7fff640 1 -- 172.21.15.60:0/1804911281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf8165720 con 0x7f9cf8071960 2024-04-25T10:08:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cf77fe640 1 --2- 172.21.15.60:0/1804911281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cf80b2b90 0x7f9cf813ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cf7fff640 1 --2- 172.21.15.60:0/1804911281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cf8071960 0x7f9cf8156d80 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8007e40 tx=0x7f9ce800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cf4ff9640 1 -- 172.21.15.60:0/1804911281 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8014620 con 0x7f9cf8071960 2024-04-25T10:08:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cf4ff9640 1 -- 172.21.15.60:0/1804911281 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce8019070 con 0x7f9cf8071960 2024-04-25T10:08:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cfe311640 1 -- 172.21.15.60:0/1804911281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf806d3f0 con 0x7f9cf8071960 2024-04-25T10:08:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cf4ff9640 1 -- 172.21.15.60:0/1804911281 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8014a00 con 0x7f9cf8071960 2024-04-25T10:08:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cfe311640 1 -- 172.21.15.60:0/1804911281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf806d9b0 con 0x7f9cf8071960 2024-04-25T10:08:55.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.530+0000 7f9cf4ff9640 1 -- 172.21.15.60:0/1804911281 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ce8014ba0 con 0x7f9cf8071960 2024-04-25T10:08:55.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9cf4ff9640 1 --2- 172.21.15.60:0/1804911281 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9cd8066f60 0x7f9cd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9cf4ff9640 1 -- 172.21.15.60:0/1804911281 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ce8085830 con 0x7f9cf8071960 2024-04-25T10:08:55.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9cf77fe640 1 --2- 172.21.15.60:0/1804911281 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9cd8066f60 0x7f9cd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 Processor -- start 2024-04-25T10:08:55.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 -- start start 2024-04-25T10:08:55.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9cf77fe640 1 --2- 172.21.15.60:0/1804911281 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9cd8066f60 0x7f9cd8069420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9cf81577d0 tx=0x7f9ce0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637563839d0 0x563756383db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563842f0 0x563756339b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56375633a0a0 0x56375633a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563756186ff0 con 0x56375633a0a0 2024-04-25T10:08:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56375610f150 con 0x5637563839d0 2024-04-25T10:08:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x563756183090 con 0x5637563842f0 2024-04-25T10:08:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9cfc8ae640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56375633a0a0 0x56375633a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9cfc8ae640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56375633a0a0 0x56375633a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55934/0 (socket says 172.21.15.60:55934) 2024-04-25T10:08:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9cfc8ae640 1 -- 172.21.15.60:0/154974059 learned_addr learned my addr 172.21.15.60:0/154974059 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9cfc8ae640 1 -- 172.21.15.60:0/154974059 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563842f0 msgr2=0x563756339b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9cfc8ae640 1 --2- 172.21.15.60:0/154974059 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563842f0 0x563756339b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9cfc8ae640 1 -- 172.21.15.60:0/154974059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637563839d0 msgr2=0x563756383db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9cfc8ae640 1 --2- 172.21.15.60:0/154974059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637563839d0 0x563756383db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9cfc8ae640 1 -- 172.21.15.60:0/154974059 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56375631d0d0 con 0x56375633a0a0 2024-04-25T10:08:55.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9cfc8ae640 1 --2- 172.21.15.60:0/154974059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56375633a0a0 0x56375633a480 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0019090 tx=0x7f9cf00a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311621fc83cc067 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9cedffb640 1 -- 172.21.15.60:0/154974059 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00b0050 con 0x56375633a0a0 2024-04-25T10:08:55.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9cedffb640 1 -- 172.21.15.60:0/154974059 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf00bc070 con 0x56375633a0a0 2024-04-25T10:08:55.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9cedffb640 1 -- 172.21.15.60:0/154974059 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00ab9e0 con 0x56375633a0a0 2024-04-25T10:08:55.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 -- 172.21.15.60:0/154974059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56375633a0a0 msgr2=0x56375633a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 --2- 172.21.15.60:0/154974059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56375633a0a0 0x56375633a480 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0019090 tx=0x7f9cf00a6000 comp rx=0 tx=0).stop 2024-04-25T10:08:55.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 -- 172.21.15.60:0/154974059 shutdown_connections 2024-04-25T10:08:55.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 --2- 172.21.15.60:0/154974059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56375633a0a0 0x56375633a480 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 --2- 172.21.15.60:0/154974059 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563842f0 0x563756339b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 --2- 172.21.15.60:0/154974059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637563839d0 0x563756383db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 -- 172.21.15.60:0/154974059 >> 172.21.15.60:0/154974059 conn(0x563756336020 msgr2=0x563756338440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 -- 172.21.15.60:0/154974059 shutdown_connections 2024-04-25T10:08:55.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 -- 172.21.15.60:0/154974059 wait complete. 2024-04-25T10:08:55.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 -- start start 2024-04-25T10:08:55.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.534+0000 7f9d0147e000 1 Processor -- start 2024-04-25T10:08:55.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56375633a0a0 0x5637563687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563839d0 0x563756368d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637563842f0 0x563756369270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56375614b250 con 0x56375633a0a0 2024-04-25T10:08:55.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563756141690 con 0x5637563842f0 2024-04-25T10:08:55.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56375610fa40 con 0x5637563839d0 2024-04-25T10:08:55.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 10 client.0 _ll_register_callbacks cb 0x563756324e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:55.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cf7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56375633a0a0 0x5637563687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cf7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56375633a0a0 0x5637563687f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55948/0 (socket says 172.21.15.60:55948) 2024-04-25T10:08:55.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cf7fff640 1 -- 172.21.15.60:0/9750608 learned_addr learned my addr 172.21.15.60:0/9750608 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:55.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cf77fe640 1 --2- 172.21.15.60:0/9750608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563839d0 0x563756368d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cf7fff640 1 -- 172.21.15.60:0/9750608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563839d0 msgr2=0x563756368d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cf7fff640 1 --2- 172.21.15.60:0/9750608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563839d0 0x563756368d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cfc8ae640 1 --2- 172.21.15.60:0/9750608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637563842f0 0x563756369270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cf7fff640 1 -- 172.21.15.60:0/9750608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637563842f0 msgr2=0x563756369270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cf7fff640 1 --2- 172.21.15.60:0/9750608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637563842f0 0x563756369270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cf7fff640 1 -- 172.21.15.60:0/9750608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637562341c0 con 0x56375633a0a0 2024-04-25T10:08:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cf7fff640 1 --2- 172.21.15.60:0/9750608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56375633a0a0 0x5637563687f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8052630 tx=0x7f9ce8056a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec611a0f846b3b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:08:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cd6ffd640 1 -- 172.21.15.60:0/9750608 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8056430 con 0x56375633a0a0 2024-04-25T10:08:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cd6ffd640 1 -- 172.21.15.60:0/9750608 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ce80565d0 con 0x56375633a0a0 2024-04-25T10:08:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 -- 172.21.15.60:0/9750608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cf00bc070 con 0x56375633a0a0 2024-04-25T10:08:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cd6ffd640 1 -- 172.21.15.60:0/9750608 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ce8074120 con 0x56375633a0a0 2024-04-25T10:08:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 10 client.6636 fetch_fsmap learned FSMap version 29 2024-04-25T10:08:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 -- 172.21.15.60:0/9750608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x563756230170 con 0x56375633a0a0 2024-04-25T10:08:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cd6ffd640 1 -- 172.21.15.60:0/9750608 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf00bc070 con 0x56375633a0a0 2024-04-25T10:08:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cd6ffd640 1 -- 172.21.15.60:0/9750608 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9ce805e450 con 0x56375633a0a0 2024-04-25T10:08:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 10 client.6636 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:08:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 10 client.6636 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 20 client.6636 populate_metadata read hostname 'smithi060' 2024-04-25T10:08:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 --2- 172.21.15.60:0/9750608 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x563756393c80 0x563756396070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 4 client.6636 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 -- 172.21.15.60:0/9750608 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563756122c60 con 0x563756393c80 2024-04-25T10:08:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cf77fe640 1 --2- 172.21.15.60:0/9750608 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x563756393c80 0x563756396070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cf77fe640 1 --2- 172.21.15.60:0/9750608 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x563756393c80 0x563756396070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abd54ab7d0e67ea3 server_cookie=67c1deae32fed96 in_seq=0 out_seq=0 2024-04-25T10:08:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cd6ffd640 10 client.6636 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cd6ffd640 1 -- 172.21.15.60:0/9750608 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563756122c60 con 0x563756393c80 2024-04-25T10:08:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cd6ffd640 10 client.6636 handle_command_reply: tid=0 2024-04-25T10:08:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 client.6636 shutdown 2024-04-25T10:08:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 -- 172.21.15.60:0/9750608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56375633a0a0 msgr2=0x5637563687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 --2- 172.21.15.60:0/9750608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56375633a0a0 0x5637563687f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8052630 tx=0x7f9ce8056a10 comp rx=0 tx=0).stop 2024-04-25T10:08:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 -- 172.21.15.60:0/9750608 shutdown_connections 2024-04-25T10:08:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 --2- 172.21.15.60:0/9750608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637563842f0 0x563756369270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 --2- 172.21.15.60:0/9750608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563839d0 0x563756368d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 --2- 172.21.15.60:0/9750608 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x563756393c80 0x563756396070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 --2- 172.21.15.60:0/9750608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56375633a0a0 0x5637563687f0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9cd6ffd640 0 client.6636 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:55.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 -- 172.21.15.60:0/9750608 >> 172.21.15.60:0/9750608 conn(0x563756336020 msgr2=0x563756338ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 -- 172.21.15.60:0/9750608 shutdown_connections 2024-04-25T10:08:55.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 1 -- 172.21.15.60:0/9750608 wait complete. 2024-04-25T10:08:55.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.538+0000 7f9d0147e000 20 client.6636 trim_cache size 0 max 16384 2024-04-25T10:08:55.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9d0147e000 1 Processor -- start 2024-04-25T10:08:55.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9d0147e000 1 -- start start 2024-04-25T10:08:55.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9d0147e000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56375633a0a0 0x5637563e0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9d0147e000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563839d0 0x5637563e0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9d0147e000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637563842f0 0x5637563e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9d0147e000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf8202bc0 con 0x5637563842f0 2024-04-25T10:08:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9d0147e000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf806f110 con 0x56375633a0a0 2024-04-25T10:08:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9d0147e000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf815adf0 con 0x5637563839d0 2024-04-25T10:08:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9cfc8ae640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637563842f0 0x5637563e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9cfc8ae640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637563842f0 0x5637563e0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55952/0 (socket says 172.21.15.60:55952) 2024-04-25T10:08:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9cfc8ae640 1 -- 172.21.15.60:0/3792038515 learned_addr learned my addr 172.21.15.60:0/3792038515 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9cfc8ae640 1 -- 172.21.15.60:0/3792038515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563839d0 msgr2=0x5637563e0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9cf77fe640 1 --2- 172.21.15.60:0/3792038515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563839d0 0x5637563e0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9cf7fff640 1 --2- 172.21.15.60:0/3792038515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56375633a0a0 0x5637563e0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9cfc8ae640 1 --2- 172.21.15.60:0/3792038515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563839d0 0x5637563e0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9cfc8ae640 1 -- 172.21.15.60:0/3792038515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56375633a0a0 msgr2=0x5637563e0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9cfc8ae640 1 --2- 172.21.15.60:0/3792038515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56375633a0a0 0x5637563e0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9cfc8ae640 1 -- 172.21.15.60:0/3792038515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637561cdde0 con 0x5637563842f0 2024-04-25T10:08:55.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9cfc8ae640 1 --2- 172.21.15.60:0/3792038515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637563842f0 0x5637563e0d80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0019090 tx=0x7f9cf0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe085c97b7951 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9ced7fa640 1 -- 172.21.15.60:0/3792038515 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00c5070 con 0x5637563842f0 2024-04-25T10:08:55.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9ced7fa640 1 -- 172.21.15.60:0/3792038515 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf00bc040 con 0x5637563842f0 2024-04-25T10:08:55.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.554+0000 7f9ced7fa640 1 -- 172.21.15.60:0/3792038515 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00b0040 con 0x5637563842f0 2024-04-25T10:08:55.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- 172.21.15.60:0/3792038515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637563842f0 msgr2=0x5637563e0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 --2- 172.21.15.60:0/3792038515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637563842f0 0x5637563e0d80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0019090 tx=0x7f9cf0004740 comp rx=0 tx=0).stop 2024-04-25T10:08:55.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- 172.21.15.60:0/3792038515 shutdown_connections 2024-04-25T10:08:55.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 --2- 172.21.15.60:0/3792038515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637563842f0 0x5637563e0d80 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 --2- 172.21.15.60:0/3792038515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563839d0 0x5637563e0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 --2- 172.21.15.60:0/3792038515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56375633a0a0 0x5637563e0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- 172.21.15.60:0/3792038515 >> 172.21.15.60:0/3792038515 conn(0x563756336020 msgr2=0x563756390fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- 172.21.15.60:0/3792038515 shutdown_connections 2024-04-25T10:08:55.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- 172.21.15.60:0/3792038515 wait complete. 2024-04-25T10:08:55.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- start start 2024-04-25T10:08:55.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 Processor -- start 2024-04-25T10:08:55.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56375633a0a0 0x563756396380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563839d0 0x5637563968c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637563842f0 0x563756396e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563756121e00 con 0x5637563842f0 2024-04-25T10:08:55.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf806eb10 con 0x56375633a0a0 2024-04-25T10:08:55.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x563756186300 con 0x5637563839d0 2024-04-25T10:08:55.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 10 client.0 _ll_register_callbacks cb 0x563756305040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:55.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cfc8ae640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637563842f0 0x563756396e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cfc8ae640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637563842f0 0x563756396e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55962/0 (socket says 172.21.15.60:55962) 2024-04-25T10:08:55.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cfc8ae640 1 -- 172.21.15.60:0/1890062753 learned_addr learned my addr 172.21.15.60:0/1890062753 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:08:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cf7fff640 1 --2- 172.21.15.60:0/1890062753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56375633a0a0 0x563756396380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cfc8ae640 1 -- 172.21.15.60:0/1890062753 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563839d0 msgr2=0x5637563968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cfc8ae640 1 --2- 172.21.15.60:0/1890062753 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563839d0 0x5637563968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cfc8ae640 1 -- 172.21.15.60:0/1890062753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56375633a0a0 msgr2=0x563756396380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cfc8ae640 1 --2- 172.21.15.60:0/1890062753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56375633a0a0 0x563756396380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cfc8ae640 1 -- 172.21.15.60:0/1890062753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637561288e0 con 0x5637563842f0 2024-04-25T10:08:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cfc8ae640 1 --2- 172.21.15.60:0/1890062753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637563842f0 0x563756396e00 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f9cf001c4c0 tx=0x7f9cf001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e5273d7f53958e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:08:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cd6ffd640 1 -- 172.21.15.60:0/1890062753 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00c5040 con 0x5637563842f0 2024-04-25T10:08:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cd6ffd640 1 -- 172.21.15.60:0/1890062753 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9cf00caae0 con 0x5637563842f0 2024-04-25T10:08:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- 172.21.15.60:0/1890062753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cf00bc040 con 0x5637563842f0 2024-04-25T10:08:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cd6ffd640 1 -- 172.21.15.60:0/1890062753 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cf00b0040 con 0x5637563842f0 2024-04-25T10:08:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 10 client.6642 fetch_fsmap learned FSMap version 29 2024-04-25T10:08:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- 172.21.15.60:0/1890062753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5637561e08f0 con 0x5637563842f0 2024-04-25T10:08:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cd6ffd640 1 -- 172.21.15.60:0/1890062753 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf00bc040 con 0x5637563842f0 2024-04-25T10:08:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cd6ffd640 1 -- 172.21.15.60:0/1890062753 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9cf00c8070 con 0x5637563842f0 2024-04-25T10:08:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 10 client.6642 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:08:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 10 client.6642 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:08:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 20 client.6642 populate_metadata read hostname 'smithi060' 2024-04-25T10:08:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 --2- 172.21.15.60:0/1890062753 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x563756397790 0x5637563ebf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 4 client.6642 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:08:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- 172.21.15.60:0/1890062753 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563756194530 con 0x563756397790 2024-04-25T10:08:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cf7fff640 1 --2- 172.21.15.60:0/1890062753 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x563756397790 0x5637563ebf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cd6ffd640 10 client.6642 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cf7fff640 1 --2- 172.21.15.60:0/1890062753 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x563756397790 0x5637563ebf60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13743d641b044083 server_cookie=eb32a4d066fe1929 in_seq=0 out_seq=0 2024-04-25T10:08:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cd6ffd640 1 -- 172.21.15.60:0/1890062753 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x563756194530 con 0x563756397790 2024-04-25T10:08:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cd6ffd640 10 client.6642 handle_command_reply: tid=0 2024-04-25T10:08:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 client.6642 shutdown 2024-04-25T10:08:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- 172.21.15.60:0/1890062753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637563842f0 msgr2=0x563756396e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 --2- 172.21.15.60:0/1890062753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637563842f0 0x563756396e00 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f9cf001c4c0 tx=0x7f9cf001acc0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- 172.21.15.60:0/1890062753 shutdown_connections 2024-04-25T10:08:55.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 --2- 172.21.15.60:0/1890062753 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x563756397790 0x5637563ebf60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 --2- 172.21.15.60:0/1890062753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637563842f0 0x563756396e00 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9cd6ffd640 0 client.6642 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:08:55.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 --2- 172.21.15.60:0/1890062753 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637563839d0 0x5637563968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 --2- 172.21.15.60:0/1890062753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56375633a0a0 0x563756396380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- 172.21.15.60:0/1890062753 >> 172.21.15.60:0/1890062753 conn(0x563756336020 msgr2=0x5637563820d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- 172.21.15.60:0/1890062753 shutdown_connections 2024-04-25T10:08:55.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 1 -- 172.21.15.60:0/1890062753 wait complete. 2024-04-25T10:08:55.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.558+0000 7f9d0147e000 20 client.6642 trim_cache size 0 max 16384 2024-04-25T10:08:55.573 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:08:55.574 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:08:55.574 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T10:08:55.574 INFO:teuthology.orchestra.run.smithi060.stdout: "8ffc5464-8236-4520-82f8-780abc0c451d": { 2024-04-25T10:08:55.574 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-04-25T10:08:55.574 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "8ffc5464-8236-4520-82f8-780abc0c451d", 2024-04-25T10:08:55.574 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,repair,force" 2024-04-25T10:08:55.574 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:55.574 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:08:55.574 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:08:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.562+0000 7f9cfe311640 1 -- 172.21.15.60:0/1804911281 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9cd8066f60 msgr2=0x7f9cd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.562+0000 7f9cfe311640 1 --2- 172.21.15.60:0/1804911281 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9cd8066f60 0x7f9cd8069420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9cf81577d0 tx=0x7f9ce0009040 comp rx=0 tx=0).stop 2024-04-25T10:08:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.562+0000 7f9cfe311640 1 -- 172.21.15.60:0/1804911281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cf8071960 msgr2=0x7f9cf8156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.562+0000 7f9cfe311640 1 --2- 172.21.15.60:0/1804911281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cf8071960 0x7f9cf8156d80 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8007e40 tx=0x7f9ce800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.562+0000 7f9cfe311640 1 -- 172.21.15.60:0/1804911281 shutdown_connections 2024-04-25T10:08:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.562+0000 7f9cfe311640 1 --2- 172.21.15.60:0/1804911281 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9cd8066f60 0x7f9cd8069420 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.562+0000 7f9cfe311640 1 --2- 172.21.15.60:0/1804911281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9cf81450b0 0x7f9cf813b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.562+0000 7f9cfe311640 1 --2- 172.21.15.60:0/1804911281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cf80b2b90 0x7f9cf813ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.562+0000 7f9cfe311640 1 --2- 172.21.15.60:0/1804911281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cf8071960 0x7f9cf8156d80 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.562+0000 7f9cfe311640 1 -- 172.21.15.60:0/1804911281 >> 172.21.15.60:0/1804911281 conn(0x7f9cf80670b0 msgr2=0x7f9cf813fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.562+0000 7f9cfe311640 1 -- 172.21.15.60:0/1804911281 shutdown_connections 2024-04-25T10:08:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:08:55.562+0000 7f9cfe311640 1 -- 172.21.15.60:0/1804911281 wait complete. 2024-04-25T10:08:55.583 INFO:tasks.cephfs.filesystem:scrub status for tag:8ffc5464-8236-4520-82f8-780abc0c451d - {'path': '/', 'tag': '8ffc5464-8236-4520-82f8-780abc0c451d', 'options': 'recursive,repair,force'} 2024-04-25T10:09:16.647 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:16.651 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:25.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:25.585 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:09:25.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.714+0000 7f62d7107640 1 Processor -- start 2024-04-25T10:09:25.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.714+0000 7f62d7107640 1 -- start start 2024-04-25T10:09:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62d7107640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62d0071250 0x7f62d00716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62d7107640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d00b3510 con 0x7f62d006f030 2024-04-25T10:09:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62d7107640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d015ad30 con 0x7f62d00b3f60 2024-04-25T10:09:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62d7107640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d015aaf0 con 0x7f62d0071250 2024-04-25T10:09:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f62d006f030 0x7f62d00b2e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40456/0 (socket says 172.21.15.60:40456) 2024-04-25T10:09:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cffff640 1 -- 172.21.15.60:0/180399638 learned_addr learned my addr 172.21.15.60:0/180399638 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62d56a4640 1 --2- 172.21.15.60:0/180399638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62d0071250 0x7f62d00716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cf7fe640 1 -- 172.21.15.60:0/180399638 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592181507 0 0) 0x7f62d00b3510 con 0x7f62d006f030 2024-04-25T10:09:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cf7fe640 1 -- 172.21.15.60:0/180399638 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62bc003530 con 0x7f62d006f030 2024-04-25T10:09:25.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cf7fe640 1 -- 172.21.15.60:0/180399638 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3989546474 0 0) 0x7f62bc003530 con 0x7f62d006f030 2024-04-25T10:09:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cf7fe640 1 -- 172.21.15.60:0/180399638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62d0071250 msgr2=0x7f62d00716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cf7fe640 1 --2- 172.21.15.60:0/180399638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62d0071250 0x7f62d00716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cf7fe640 1 -- 172.21.15.60:0/180399638 >> v1:172.21.15.100:6789/0 conn(0x7f62d00b3f60 legacy=0x7f62d015a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cf7fe640 1 -- 172.21.15.60:0/180399638 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d015b010 con 0x7f62d006f030 2024-04-25T10:09:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cf7fe640 1 -- 172.21.15.60:0/180399638 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f62b8001680 con 0x7f62d006f030 2024-04-25T10:09:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cf7fe640 1 -- 172.21.15.60:0/180399638 >> v1:172.21.15.60:6789/0 conn(0x7f62d006f030 legacy=0x7f62d00b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cf7fe640 1 --2- 172.21.15.60:0/180399638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62d0071250 0x7f62bc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cf7fe640 1 --2- 172.21.15.60:0/180399638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62bc003f70 0x7f62bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cf7fe640 1 --2- 172.21.15.60:0/180399638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62bc004d50 0x7f62bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cf7fe640 1 -- 172.21.15.60:0/180399638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d015aaf0 con 0x7f62bc004d50 2024-04-25T10:09:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cf7fe640 1 -- 172.21.15.60:0/180399638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc00bb60 con 0x7f62bc003f70 2024-04-25T10:09:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cf7fe640 1 -- 172.21.15.60:0/180399638 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc00bd60 con 0x7f62d0071250 2024-04-25T10:09:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62d56a4640 1 --2- 172.21.15.60:0/180399638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62bc004d50 0x7f62bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62d4ea3640 1 --2- 172.21.15.60:0/180399638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62d0071250 0x7f62bc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62cffff640 1 --2- 172.21.15.60:0/180399638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62bc003f70 0x7f62bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62d56a4640 1 -- 172.21.15.60:0/180399638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62d0071250 msgr2=0x7f62bc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62d56a4640 1 --2- 172.21.15.60:0/180399638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62d0071250 0x7f62bc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62d56a4640 1 -- 172.21.15.60:0/180399638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62bc003f70 msgr2=0x7f62bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62d56a4640 1 --2- 172.21.15.60:0/180399638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62bc003f70 0x7f62bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.718+0000 7f62d56a4640 1 -- 172.21.15.60:0/180399638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc003530 con 0x7f62bc004d50 2024-04-25T10:09:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62d56a4640 1 --2- 172.21.15.60:0/180399638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62bc004d50 0x7f62bc0051d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f62c8009ae0 tx=0x7f62c801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc23a7dc21383002 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62cf7fe640 1 -- 172.21.15.60:0/180399638 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c80b2070 con 0x7f62bc004d50 2024-04-25T10:09:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62cf7fe640 1 -- 172.21.15.60:0/180399638 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c80a5550 con 0x7f62bc004d50 2024-04-25T10:09:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62cf7fe640 1 -- 172.21.15.60:0/180399638 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c80096b0 con 0x7f62bc004d50 2024-04-25T10:09:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62d7107640 1 -- 172.21.15.60:0/180399638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62bc004d50 msgr2=0x7f62bc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62d7107640 1 --2- 172.21.15.60:0/180399638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62bc004d50 0x7f62bc0051d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f62c8009ae0 tx=0x7f62c801ca70 comp rx=0 tx=0).stop 2024-04-25T10:09:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62d56a4640 1 -- 172.21.15.60:0/180399638 reap_dead start 2024-04-25T10:09:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62d7107640 1 -- 172.21.15.60:0/180399638 shutdown_connections 2024-04-25T10:09:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62d7107640 1 -- 172.21.15.60:0/180399638 >> 172.21.15.60:0/180399638 conn(0x7f62d00670b0 msgr2=0x7f62d006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62d7107640 1 -- 172.21.15.60:0/180399638 shutdown_connections 2024-04-25T10:09:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62d7107640 1 -- 172.21.15.60:0/180399638 wait complete. 2024-04-25T10:09:25.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62d7107640 1 Processor -- start 2024-04-25T10:09:25.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62d7107640 1 -- start start 2024-04-25T10:09:25.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62d7107640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62bc004d50 0x7f62d014cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62d7107640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d0149a50 0x7f62d0153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62d7107640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62d0154200 0x7f62d01549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62d7107640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00b4340 con 0x7f62d0149a50 2024-04-25T10:09:25.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62d7107640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00b2600 con 0x7f62bc004d50 2024-04-25T10:09:25.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.722+0000 7f62d7107640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00716e0 con 0x7f62d0154200 2024-04-25T10:09:25.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62cffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d0149a50 0x7f62d0153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62cffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d0149a50 0x7f62d0153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42574/0 (socket says 172.21.15.60:42574) 2024-04-25T10:09:25.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62cffff640 1 -- 172.21.15.60:0/3853464974 learned_addr learned my addr 172.21.15.60:0/3853464974 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:25.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62d56a4640 1 --2- 172.21.15.60:0/3853464974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62d0154200 0x7f62d01549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62d4ea3640 1 --2- 172.21.15.60:0/3853464974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62bc004d50 0x7f62d014cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62cffff640 1 -- 172.21.15.60:0/3853464974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62d0154200 msgr2=0x7f62d01549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62cffff640 1 --2- 172.21.15.60:0/3853464974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62d0154200 0x7f62d01549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62cffff640 1 -- 172.21.15.60:0/3853464974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62bc004d50 msgr2=0x7f62d014cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62cffff640 1 --2- 172.21.15.60:0/3853464974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62bc004d50 0x7f62d014cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62cffff640 1 -- 172.21.15.60:0/3853464974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d014a350 con 0x7f62d0149a50 2024-04-25T10:09:25.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62cffff640 1 --2- 172.21.15.60:0/3853464974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d0149a50 0x7f62d0153cc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f62b80119b0 tx=0x7f62b8011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62cd7fa640 1 -- 172.21.15.60:0/3853464974 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b8010040 con 0x7f62d0149a50 2024-04-25T10:09:25.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62cd7fa640 1 -- 172.21.15.60:0/3853464974 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b801a070 con 0x7f62d0149a50 2024-04-25T10:09:25.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62d7107640 1 -- 172.21.15.60:0/3853464974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d014d290 con 0x7f62d0149a50 2024-04-25T10:09:25.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62cd7fa640 1 -- 172.21.15.60:0/3853464974 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b800fde0 con 0x7f62d0149a50 2024-04-25T10:09:25.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62d7107640 1 -- 172.21.15.60:0/3853464974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d014a5c0 con 0x7f62d0149a50 2024-04-25T10:09:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62cd7fa640 1 -- 172.21.15.60:0/3853464974 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62b801f460 con 0x7f62d0149a50 2024-04-25T10:09:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62cd7fa640 1 --2- 172.21.15.60:0/3853464974 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62b0066f60 0x7f62b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62cd7fa640 1 -- 172.21.15.60:0/3853464974 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62b8088470 con 0x7f62d0149a50 2024-04-25T10:09:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62d4ea3640 1 --2- 172.21.15.60:0/3853464974 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62b0066f60 0x7f62b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62da274000 1 Processor -- start 2024-04-25T10:09:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.726+0000 7f62da274000 1 -- start start 2024-04-25T10:09:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62d4ea3640 1 --2- 172.21.15.60:0/3853464974 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62b0066f60 0x7f62b0069420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f62c0005210 tx=0x7f62c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e10a9d0 0x55687e10adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e10b2f0 0x55687e0c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e0c10a0 0x55687e0c1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687df0dff0 con 0x55687e10a9d0 2024-04-25T10:09:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687de96150 con 0x55687e0c10a0 2024-04-25T10:09:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55687df0a090 con 0x55687e10b2f0 2024-04-25T10:09:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62d4ea3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e10a9d0 0x55687e10adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62d4ea3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e10a9d0 0x55687e10adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42582/0 (socket says 172.21.15.60:42582) 2024-04-25T10:09:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62d4ea3640 1 -- 172.21.15.60:0/1161947675 learned_addr learned my addr 172.21.15.60:0/1161947675 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62cffff640 1 --2- 172.21.15.60:0/1161947675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e10b2f0 0x55687e0c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62d56a4640 1 --2- 172.21.15.60:0/1161947675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e0c10a0 0x55687e0c1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62d4ea3640 1 -- 172.21.15.60:0/1161947675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e10b2f0 msgr2=0x55687e0c0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62d4ea3640 1 --2- 172.21.15.60:0/1161947675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e10b2f0 0x55687e0c0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62d4ea3640 1 -- 172.21.15.60:0/1161947675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e0c10a0 msgr2=0x55687e0c1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62d4ea3640 1 --2- 172.21.15.60:0/1161947675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e0c10a0 0x55687e0c1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62d4ea3640 1 -- 172.21.15.60:0/1161947675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55687e0a40d0 con 0x55687e10a9d0 2024-04-25T10:09:25.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62cffff640 1 --2- 172.21.15.60:0/1161947675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e10b2f0 0x55687e0c0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:25.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62d4ea3640 1 --2- 172.21.15.60:0/1161947675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e10a9d0 0x55687e10adb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f62c0001d20 tx=0x7f62c0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0ceec537ec8781 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62c67fc640 1 -- 172.21.15.60:0/1161947675 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0027070 con 0x55687e10a9d0 2024-04-25T10:09:25.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62c67fc640 1 -- 172.21.15.60:0/1161947675 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c0014b80 con 0x55687e10a9d0 2024-04-25T10:09:25.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62c67fc640 1 -- 172.21.15.60:0/1161947675 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0013360 con 0x55687e10a9d0 2024-04-25T10:09:25.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 -- 172.21.15.60:0/1161947675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e10a9d0 msgr2=0x55687e10adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 --2- 172.21.15.60:0/1161947675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e10a9d0 0x55687e10adb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f62c0001d20 tx=0x7f62c0013000 comp rx=0 tx=0).stop 2024-04-25T10:09:25.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 -- 172.21.15.60:0/1161947675 shutdown_connections 2024-04-25T10:09:25.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 --2- 172.21.15.60:0/1161947675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e0c10a0 0x55687e0c1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 --2- 172.21.15.60:0/1161947675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e10b2f0 0x55687e0c0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 --2- 172.21.15.60:0/1161947675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e10a9d0 0x55687e10adb0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 -- 172.21.15.60:0/1161947675 >> 172.21.15.60:0/1161947675 conn(0x55687e0bd020 msgr2=0x55687e0bf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:25.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 -- 172.21.15.60:0/1161947675 shutdown_connections 2024-04-25T10:09:25.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 -- 172.21.15.60:0/1161947675 wait complete. 2024-04-25T10:09:25.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 -- start start 2024-04-25T10:09:25.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 Processor -- start 2024-04-25T10:09:25.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e0c10a0 0x55687e0ef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e10a9d0 0x55687e0efd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e10b2f0 0x55687e0f0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687ded2250 con 0x55687e0c10a0 2024-04-25T10:09:25.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687dec8690 con 0x55687e10b2f0 2024-04-25T10:09:25.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55687de96a40 con 0x55687e10a9d0 2024-04-25T10:09:25.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62da274000 10 client.0 _ll_register_callbacks cb 0x55687e0abe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:25.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.730+0000 7f62d4ea3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e0c10a0 0x55687e0ef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62d4ea3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e0c10a0 0x55687e0ef7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42594/0 (socket says 172.21.15.60:42594) 2024-04-25T10:09:25.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62d4ea3640 1 -- 172.21.15.60:0/3787453016 learned_addr learned my addr 172.21.15.60:0/3787453016 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:25.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62d56a4640 1 --2- 172.21.15.60:0/3787453016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e10b2f0 0x55687e0f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62cffff640 1 --2- 172.21.15.60:0/3787453016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e10a9d0 0x55687e0efd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62d4ea3640 1 -- 172.21.15.60:0/3787453016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e10a9d0 msgr2=0x55687e0efd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62d4ea3640 1 --2- 172.21.15.60:0/3787453016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e10a9d0 0x55687e0efd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62d4ea3640 1 -- 172.21.15.60:0/3787453016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e10b2f0 msgr2=0x55687e0f0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62d4ea3640 1 --2- 172.21.15.60:0/3787453016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e10b2f0 0x55687e0f0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62d4ea3640 1 -- 172.21.15.60:0/3787453016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55687dfbb1c0 con 0x55687e0c10a0 2024-04-25T10:09:25.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62d56a4640 1 --2- 172.21.15.60:0/3787453016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e10b2f0 0x55687e0f0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:25.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62d4ea3640 1 --2- 172.21.15.60:0/3787453016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e0c10a0 0x55687e0ef7f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009010 tx=0x7f62c0001520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de48bfa9fb1ed9a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:09:25.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62af7fe640 1 -- 172.21.15.60:0/3787453016 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0033060 con 0x55687e0c10a0 2024-04-25T10:09:25.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62af7fe640 1 -- 172.21.15.60:0/3787453016 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62c0021c90 con 0x55687e0c10a0 2024-04-25T10:09:25.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 1 -- 172.21.15.60:0/3787453016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62c0014b80 con 0x55687e0c10a0 2024-04-25T10:09:25.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62af7fe640 1 -- 172.21.15.60:0/3787453016 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62c0039050 con 0x55687e0c10a0 2024-04-25T10:09:25.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 10 client.6654 fetch_fsmap learned FSMap version 30 2024-04-25T10:09:25.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 1 -- 172.21.15.60:0/3787453016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55687dfb7170 con 0x55687e0c10a0 2024-04-25T10:09:25.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62af7fe640 1 -- 172.21.15.60:0/3787453016 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c0014b80 con 0x55687e0c10a0 2024-04-25T10:09:25.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62af7fe640 1 -- 172.21.15.60:0/3787453016 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f62c003c020 con 0x55687e0c10a0 2024-04-25T10:09:25.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 10 client.6654 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T10:09:25.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 10 client.6654 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:09:25.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 20 client.6654 populate_metadata read hostname 'smithi060' 2024-04-25T10:09:25.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 1 --2- 172.21.15.60:0/3787453016 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x55687e11ac80 0x55687e11d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 4 client.6654 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:25.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 1 -- 172.21.15.60:0/3787453016 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55687dea9c60 con 0x55687e11ac80 2024-04-25T10:09:25.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62cffff640 1 --2- 172.21.15.60:0/3787453016 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x55687e11ac80 0x55687e11d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62cffff640 1 --2- 172.21.15.60:0/3787453016 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x55687e11ac80 0x55687e11d070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f1bb669cc5bac43 server_cookie=eb5efd0324f16f6f in_seq=0 out_seq=0 2024-04-25T10:09:25.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62af7fe640 10 client.6654 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:09:25.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62af7fe640 1 -- 172.21.15.60:0/3787453016 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55687dea9c60 con 0x55687e11ac80 2024-04-25T10:09:25.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62af7fe640 10 client.6654 handle_command_reply: tid=0 2024-04-25T10:09:25.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 1 client.6654 shutdown 2024-04-25T10:09:25.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 1 -- 172.21.15.60:0/3787453016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e0c10a0 msgr2=0x55687e0ef7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 1 --2- 172.21.15.60:0/3787453016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e0c10a0 0x55687e0ef7f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009010 tx=0x7f62c0001520 comp rx=0 tx=0).stop 2024-04-25T10:09:25.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 1 -- 172.21.15.60:0/3787453016 shutdown_connections 2024-04-25T10:09:25.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 1 --2- 172.21.15.60:0/3787453016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e10b2f0 0x55687e0f0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 1 --2- 172.21.15.60:0/3787453016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e10a9d0 0x55687e0efd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 1 --2- 172.21.15.60:0/3787453016 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x55687e11ac80 0x55687e11d070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 1 --2- 172.21.15.60:0/3787453016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e0c10a0 0x55687e0ef7f0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62af7fe640 0 client.6654 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:09:25.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 1 -- 172.21.15.60:0/3787453016 >> 172.21.15.60:0/3787453016 conn(0x55687e0bd020 msgr2=0x55687e0bfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:25.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 1 -- 172.21.15.60:0/3787453016 shutdown_connections 2024-04-25T10:09:25.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 1 -- 172.21.15.60:0/3787453016 wait complete. 2024-04-25T10:09:25.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.734+0000 7f62da274000 20 client.6654 trim_cache size 0 max 16384 2024-04-25T10:09:25.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 Processor -- start 2024-04-25T10:09:25.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 -- start start 2024-04-25T10:09:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e0c10a0 0x55687e167300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e10a9d0 0x55687e167840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e10b2f0 0x55687e167d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d02044c0 con 0x55687e0c10a0 2024-04-25T10:09:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d0203640 con 0x55687e10b2f0 2024-04-25T10:09:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d014bab0 con 0x55687e10a9d0 2024-04-25T10:09:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62d4ea3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e0c10a0 0x55687e167300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62d4ea3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e0c10a0 0x55687e167300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42602/0 (socket says 172.21.15.60:42602) 2024-04-25T10:09:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62d4ea3640 1 -- 172.21.15.60:0/877006041 learned_addr learned my addr 172.21.15.60:0/877006041 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62d4ea3640 1 -- 172.21.15.60:0/877006041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e10a9d0 msgr2=0x55687e167840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62d4ea3640 1 --2- 172.21.15.60:0/877006041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e10a9d0 0x55687e167840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62d4ea3640 1 -- 172.21.15.60:0/877006041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e10b2f0 msgr2=0x55687e167d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62d4ea3640 1 --2- 172.21.15.60:0/877006041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e10b2f0 0x55687e167d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62d4ea3640 1 -- 172.21.15.60:0/877006041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55687df54de0 con 0x55687e0c10a0 2024-04-25T10:09:25.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62d4ea3640 1 --2- 172.21.15.60:0/877006041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e0c10a0 0x55687e167300 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f62c0003a70 tx=0x7f62c002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed697fd59cf5b1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62c5ffb640 1 -- 172.21.15.60:0/877006041 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0027070 con 0x55687e0c10a0 2024-04-25T10:09:25.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62c5ffb640 1 -- 172.21.15.60:0/877006041 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c0021d10 con 0x55687e0c10a0 2024-04-25T10:09:25.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62c5ffb640 1 -- 172.21.15.60:0/877006041 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c00149d0 con 0x55687e0c10a0 2024-04-25T10:09:25.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 -- 172.21.15.60:0/877006041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e0c10a0 msgr2=0x55687e167300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 --2- 172.21.15.60:0/877006041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e0c10a0 0x55687e167300 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f62c0003a70 tx=0x7f62c002d000 comp rx=0 tx=0).stop 2024-04-25T10:09:25.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 -- 172.21.15.60:0/877006041 shutdown_connections 2024-04-25T10:09:25.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 --2- 172.21.15.60:0/877006041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e10b2f0 0x55687e167d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 --2- 172.21.15.60:0/877006041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e10a9d0 0x55687e167840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 --2- 172.21.15.60:0/877006041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e0c10a0 0x55687e167300 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 -- 172.21.15.60:0/877006041 >> 172.21.15.60:0/877006041 conn(0x55687e0bd020 msgr2=0x55687e117fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:25.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 -- 172.21.15.60:0/877006041 shutdown_connections 2024-04-25T10:09:25.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.750+0000 7f62da274000 1 -- 172.21.15.60:0/877006041 wait complete. 2024-04-25T10:09:25.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 -- start start 2024-04-25T10:09:25.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 Processor -- start 2024-04-25T10:09:25.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e0c10a0 0x55687e11d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e10a9d0 0x55687e11d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e10b2f0 0x55687e11de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687dea8e00 con 0x55687e10b2f0 2024-04-25T10:09:25.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d014c0b0 con 0x55687e10a9d0 2024-04-25T10:09:25.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55687df0d300 con 0x55687e0c10a0 2024-04-25T10:09:25.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 10 client.0 _ll_register_callbacks cb 0x55687e08c040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:25.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62d56a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e10b2f0 0x55687e11de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62d56a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e10b2f0 0x55687e11de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42612/0 (socket says 172.21.15.60:42612) 2024-04-25T10:09:25.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62d56a4640 1 -- 172.21.15.60:0/4140311863 learned_addr learned my addr 172.21.15.60:0/4140311863 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:25.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62d56a4640 1 -- 172.21.15.60:0/4140311863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e0c10a0 msgr2=0x55687e11d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62d56a4640 1 --2- 172.21.15.60:0/4140311863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e0c10a0 0x55687e11d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62d56a4640 1 -- 172.21.15.60:0/4140311863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e10a9d0 msgr2=0x55687e11d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62d56a4640 1 --2- 172.21.15.60:0/4140311863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e10a9d0 0x55687e11d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62d56a4640 1 -- 172.21.15.60:0/4140311863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55687deaf8e0 con 0x55687e10b2f0 2024-04-25T10:09:25.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62d56a4640 1 --2- 172.21.15.60:0/4140311863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e10b2f0 0x55687e11de00 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f62c80b0040 tx=0x7f62bc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09ca47d3576457b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:09:25.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62af7fe640 1 -- 172.21.15.60:0/4140311863 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c80be060 con 0x55687e10b2f0 2024-04-25T10:09:25.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62af7fe640 1 -- 172.21.15.60:0/4140311863 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62c8004440 con 0x55687e10b2f0 2024-04-25T10:09:25.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 -- 172.21.15.60:0/4140311863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62c0021d10 con 0x55687e10b2f0 2024-04-25T10:09:25.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62af7fe640 1 -- 172.21.15.60:0/4140311863 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62c80c4050 con 0x55687e10b2f0 2024-04-25T10:09:25.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 10 client.6660 fetch_fsmap learned FSMap version 30 2024-04-25T10:09:25.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 -- 172.21.15.60:0/4140311863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55687df678f0 con 0x55687e10b2f0 2024-04-25T10:09:25.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62af7fe640 1 -- 172.21.15.60:0/4140311863 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c0021d10 con 0x55687e10b2f0 2024-04-25T10:09:25.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62af7fe640 1 -- 172.21.15.60:0/4140311863 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f62c8003620 con 0x55687e10b2f0 2024-04-25T10:09:25.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 10 client.6660 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T10:09:25.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 10 client.6660 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:09:25.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 20 client.6660 populate_metadata read hostname 'smithi060' 2024-04-25T10:09:25.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 --2- 172.21.15.60:0/4140311863 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x55687e11ea40 0x55687e172f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 4 client.6660 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:09:25.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 -- 172.21.15.60:0/4140311863 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55687df1b530 con 0x55687e11ea40 2024-04-25T10:09:25.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62d4ea3640 1 --2- 172.21.15.60:0/4140311863 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x55687e11ea40 0x55687e172f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62af7fe640 10 client.6660 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:09:25.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62d4ea3640 1 --2- 172.21.15.60:0/4140311863 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x55687e11ea40 0x55687e172f60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e8c0181393e7764 server_cookie=1872af9fc105fefb in_seq=0 out_seq=0 2024-04-25T10:09:25.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62af7fe640 1 -- 172.21.15.60:0/4140311863 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55687df1b530 con 0x55687e11ea40 2024-04-25T10:09:25.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62af7fe640 10 client.6660 handle_command_reply: tid=0 2024-04-25T10:09:25.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 client.6660 shutdown 2024-04-25T10:09:25.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 -- 172.21.15.60:0/4140311863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e10b2f0 msgr2=0x55687e11de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 --2- 172.21.15.60:0/4140311863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e10b2f0 0x55687e11de00 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f62c80b0040 tx=0x7f62bc005840 comp rx=0 tx=0).stop 2024-04-25T10:09:25.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 -- 172.21.15.60:0/4140311863 shutdown_connections 2024-04-25T10:09:25.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 --2- 172.21.15.60:0/4140311863 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x55687e11ea40 0x55687e172f60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 --2- 172.21.15.60:0/4140311863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55687e10b2f0 0x55687e11de00 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 --2- 172.21.15.60:0/4140311863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55687e10a9d0 0x55687e11d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 --2- 172.21.15.60:0/4140311863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55687e0c10a0 0x55687e11d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62af7fe640 0 client.6660 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:09:25.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 -- 172.21.15.60:0/4140311863 >> 172.21.15.60:0/4140311863 conn(0x55687e0bd020 msgr2=0x55687e1090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:25.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 -- 172.21.15.60:0/4140311863 shutdown_connections 2024-04-25T10:09:25.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 1 -- 172.21.15.60:0/4140311863 wait complete. 2024-04-25T10:09:25.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.754+0000 7f62da274000 20 client.6660 trim_cache size 0 max 16384 2024-04-25T10:09:25.771 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:09:25.771 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T10:09:25.771 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T10:09:25.771 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:09:25.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.758+0000 7f62d7107640 1 -- 172.21.15.60:0/3853464974 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62b0066f60 msgr2=0x7f62b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.758+0000 7f62d7107640 1 --2- 172.21.15.60:0/3853464974 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62b0066f60 0x7f62b0069420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f62c0005210 tx=0x7f62c0009040 comp rx=0 tx=0).stop 2024-04-25T10:09:25.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.758+0000 7f62d7107640 1 -- 172.21.15.60:0/3853464974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d0149a50 msgr2=0x7f62d0153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.758+0000 7f62d7107640 1 --2- 172.21.15.60:0/3853464974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d0149a50 0x7f62d0153cc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f62b80119b0 tx=0x7f62b8011e70 comp rx=0 tx=0).stop 2024-04-25T10:09:25.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.758+0000 7f62d7107640 1 -- 172.21.15.60:0/3853464974 shutdown_connections 2024-04-25T10:09:25.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.758+0000 7f62d7107640 1 --2- 172.21.15.60:0/3853464974 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f62b0066f60 0x7f62b0069420 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.758+0000 7f62d7107640 1 --2- 172.21.15.60:0/3853464974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62d0154200 0x7f62d01549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.758+0000 7f62d7107640 1 --2- 172.21.15.60:0/3853464974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62d0149a50 0x7f62d0153cc0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.758+0000 7f62d7107640 1 --2- 172.21.15.60:0/3853464974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62bc004d50 0x7f62d014cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.758+0000 7f62d7107640 1 -- 172.21.15.60:0/3853464974 >> 172.21.15.60:0/3853464974 conn(0x7f62d00670b0 msgr2=0x7f62d006dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:25.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.758+0000 7f62d7107640 1 -- 172.21.15.60:0/3853464974 shutdown_connections 2024-04-25T10:09:25.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.758+0000 7f62d7107640 1 -- 172.21.15.60:0/3853464974 wait complete. 2024-04-25T10:09:25.782 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:09:25.782 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:25.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.890+0000 7f775d113640 1 Processor -- start 2024-04-25T10:09:25.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.890+0000 7f775d113640 1 -- start start 2024-04-25T10:09:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.890+0000 7f775d113640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f775815b8d0 0x7f77580b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.890+0000 7f775d113640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f775806ff50 0x7f775815bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.890+0000 7f775d113640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77580b3370 con 0x7f775806bc20 2024-04-25T10:09:25.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.890+0000 7f775d113640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7758070390 con 0x7f775806ff50 2024-04-25T10:09:25.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.890+0000 7f775d113640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77580b3650 con 0x7f775815b8d0 2024-04-25T10:09:25.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.890+0000 7f775659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f775806ff50 0x7f775815bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.890+0000 7f775659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f775806ff50 0x7f775815bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42614/0 (socket says 172.21.15.60:42614) 2024-04-25T10:09:25.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.890+0000 7f775659c640 1 -- 172.21.15.60:0/4282269686 learned_addr learned my addr 172.21.15.60:0/4282269686 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:25.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.890+0000 7f7756d9d640 1 --2- 172.21.15.60:0/4282269686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f775815b8d0 0x7f77580b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.890+0000 7f775659c640 1 -- 172.21.15.60:0/4282269686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f775815b8d0 msgr2=0x7f77580b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.890+0000 7f775659c640 1 --2- 172.21.15.60:0/4282269686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f775815b8d0 0x7f77580b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.890+0000 7f775659c640 1 -- 172.21.15.60:0/4282269686 >> v1:172.21.15.100:6789/0 conn(0x7f775806bc20 legacy=0x7f77580b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:25.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.890+0000 7f775659c640 1 -- 172.21.15.60:0/4282269686 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77580b3930 con 0x7f775806ff50 2024-04-25T10:09:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.894+0000 7f775659c640 1 --2- 172.21.15.60:0/4282269686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775806ff50 0x7f775815bcb0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f7740009e50 tx=0x7f774001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4cdd0aa1554a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.894+0000 7f7755d9b640 1 -- 172.21.15.60:0/4282269686 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77400a5840 con 0x7f775806ff50 2024-04-25T10:09:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.894+0000 7f7755d9b640 1 -- 172.21.15.60:0/4282269686 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77400a59e0 con 0x7f775806ff50 2024-04-25T10:09:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.894+0000 7f7755d9b640 1 -- 172.21.15.60:0/4282269686 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77400a5db0 con 0x7f775806ff50 2024-04-25T10:09:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.894+0000 7f775d113640 1 -- 172.21.15.60:0/4282269686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775806ff50 msgr2=0x7f775815bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.894+0000 7f775d113640 1 --2- 172.21.15.60:0/4282269686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775806ff50 0x7f775815bcb0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f7740009e50 tx=0x7f774001c9c0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.894+0000 7f775d113640 1 -- 172.21.15.60:0/4282269686 shutdown_connections 2024-04-25T10:09:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.894+0000 7f775d113640 1 --2- 172.21.15.60:0/4282269686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775806ff50 0x7f775815bcb0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.894+0000 7f775d113640 1 --2- 172.21.15.60:0/4282269686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f775815b8d0 0x7f77580b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.894+0000 7f775d113640 1 -- 172.21.15.60:0/4282269686 >> 172.21.15.60:0/4282269686 conn(0x7f7758071b70 msgr2=0x7f7758071f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.894+0000 7f775d113640 1 -- 172.21.15.60:0/4282269686 shutdown_connections 2024-04-25T10:09:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.894+0000 7f775d113640 1 -- 172.21.15.60:0/4282269686 wait complete. 2024-04-25T10:09:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.894+0000 7f775d113640 1 Processor -- start 2024-04-25T10:09:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.894+0000 7f775d113640 1 -- start start 2024-04-25T10:09:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775d113640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f775806bc20 0x7f775814fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775d113640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775806ff50 0x7f775815a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775d113640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f775815b8d0 0x7f775813c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775d113640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77580af2e0 con 0x7f775806ff50 2024-04-25T10:09:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775d113640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775815c1f0 con 0x7f775806bc20 2024-04-25T10:09:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775d113640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77581519b0 con 0x7f775815b8d0 2024-04-25T10:09:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775806ff50 0x7f775815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775806ff50 0x7f775815a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42618/0 (socket says 172.21.15.60:42618) 2024-04-25T10:09:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775659c640 1 -- 172.21.15.60:0/3278813498 learned_addr learned my addr 172.21.15.60:0/3278813498 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775759e640 1 --2- 172.21.15.60:0/3278813498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f775815b8d0 0x7f775813c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f7756d9d640 1 --2- 172.21.15.60:0/3278813498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f775806bc20 0x7f775814fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775659c640 1 -- 172.21.15.60:0/3278813498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f775815b8d0 msgr2=0x7f775813c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775659c640 1 --2- 172.21.15.60:0/3278813498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f775815b8d0 0x7f775813c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775659c640 1 -- 172.21.15.60:0/3278813498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f775806bc20 msgr2=0x7f775814fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:25.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775659c640 1 --2- 172.21.15.60:0/3278813498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f775806bc20 0x7f775814fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:25.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775659c640 1 -- 172.21.15.60:0/3278813498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7758153bb0 con 0x7f775806ff50 2024-04-25T10:09:25.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775659c640 1 --2- 172.21.15.60:0/3278813498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775806ff50 0x7f775815a280 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f77400af040 tx=0x7f7740019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f774f7fe640 1 -- 172.21.15.60:0/3278813498 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77400b8880 con 0x7f775806ff50 2024-04-25T10:09:25.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f774f7fe640 1 -- 172.21.15.60:0/3278813498 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77400be070 con 0x7f775806ff50 2024-04-25T10:09:25.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775d113640 1 -- 172.21.15.60:0/3278813498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7758153e90 con 0x7f775806ff50 2024-04-25T10:09:25.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f774f7fe640 1 -- 172.21.15.60:0/3278813498 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77400b8ca0 con 0x7f775806ff50 2024-04-25T10:09:25.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775d113640 1 -- 172.21.15.60:0/3278813498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7758154450 con 0x7f775806ff50 2024-04-25T10:09:25.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f775d113640 1 -- 172.21.15.60:0/3278813498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7758074770 con 0x7f775806ff50 2024-04-25T10:09:25.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f774f7fe640 1 -- 172.21.15.60:0/3278813498 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77400c3460 con 0x7f775806ff50 2024-04-25T10:09:25.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f774f7fe640 1 --2- 172.21.15.60:0/3278813498 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7738066f60 0x7f7738069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:25.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f774f7fe640 1 -- 172.21.15.60:0/3278813498 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77401204c0 con 0x7f775806ff50 2024-04-25T10:09:25.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.898+0000 7f7756d9d640 1 --2- 172.21.15.60:0/3278813498 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7738066f60 0x7f7738069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:25.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.902+0000 7f7756d9d640 1 --2- 172.21.15.60:0/3278813498 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7738066f60 0x7f7738069420 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f774800a8d0 tx=0x7f7748008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:25.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:25.902+0000 7f774f7fe640 1 -- 172.21.15.60:0/3278813498 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f77400165e0 con 0x7f775806ff50 2024-04-25T10:09:26.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.090+0000 7f775d113640 1 -- 172.21.15.60:0/3278813498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77580cf180 con 0x7f775806ff50 2024-04-25T10:09:26.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.094+0000 7f774f7fe640 1 -- 172.21.15.60:0/3278813498 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f77400f1430 con 0x7f775806ff50 2024-04-25T10:09:26.101 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:26.101 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/380273391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":380273391},{"type":"v1","addr":"172.21.15.100:6836","nonce":380273391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5635,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2965977166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2965977166},{"type":"v1","addr":"172.21.15.100:6837","nonce":2965977166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1105028629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1105028629},{"type":"v1","addr":"172.21.15.60:6834","nonce":1105028629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:10.236590+0000","modified":"2024-04-25T10:08:58.929165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.60:6835/2718963870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2718963870},{"type":"v1","addr":"172.21.15.60:6835","nonce":2718963870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:09:26.101 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 30 2024-04-25T10:09:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.094+0000 7f775d113640 1 -- 172.21.15.60:0/3278813498 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7738066f60 msgr2=0x7f7738069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.094+0000 7f775d113640 1 --2- 172.21.15.60:0/3278813498 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7738066f60 0x7f7738069420 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f774800a8d0 tx=0x7f7748008040 comp rx=0 tx=0).stop 2024-04-25T10:09:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.094+0000 7f775d113640 1 -- 172.21.15.60:0/3278813498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775806ff50 msgr2=0x7f775815a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.094+0000 7f775d113640 1 --2- 172.21.15.60:0/3278813498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775806ff50 0x7f775815a280 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f77400af040 tx=0x7f7740019760 comp rx=0 tx=0).stop 2024-04-25T10:09:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.094+0000 7f775d113640 1 -- 172.21.15.60:0/3278813498 shutdown_connections 2024-04-25T10:09:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.094+0000 7f775d113640 1 --2- 172.21.15.60:0/3278813498 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7738066f60 0x7f7738069420 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.094+0000 7f775d113640 1 --2- 172.21.15.60:0/3278813498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f775815b8d0 0x7f775813c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.094+0000 7f775d113640 1 --2- 172.21.15.60:0/3278813498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775806ff50 0x7f775815a280 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.094+0000 7f775d113640 1 --2- 172.21.15.60:0/3278813498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f775806bc20 0x7f775814fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.094+0000 7f775d113640 1 -- 172.21.15.60:0/3278813498 >> 172.21.15.60:0/3278813498 conn(0x7f7758071b70 msgr2=0x7f7758165ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:26.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.094+0000 7f775d113640 1 -- 172.21.15.60:0/3278813498 shutdown_connections 2024-04-25T10:09:26.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.094+0000 7f775d113640 1 -- 172.21.15.60:0/3278813498 wait complete. 2024-04-25T10:09:26.118 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:09:26.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.218+0000 7fc2eceb0640 1 Processor -- start 2024-04-25T10:09:26.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.218+0000 7fc2eceb0640 1 -- start start 2024-04-25T10:09:26.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2eceb0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc2e815dbe0 0x7fc2e80d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2eceb0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc2e80d1ff0 0x7fc2e80d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2eceb0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e80d2b90 con 0x7fc2e80d0e40 2024-04-25T10:09:26.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2eceb0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e80d3450 con 0x7fc2e815dbe0 2024-04-25T10:09:26.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2eceb0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e80d1280 con 0x7fc2e80d1ff0 2024-04-25T10:09:26.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2e659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc2e815dbe0 0x7fc2e80d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2e659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc2e815dbe0 0x7fc2e80d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42634/0 (socket says 172.21.15.60:42634) 2024-04-25T10:09:26.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2e659c640 1 -- 172.21.15.60:0/3260502574 learned_addr learned my addr 172.21.15.60:0/3260502574 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:26.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2e6d9d640 1 --2- 172.21.15.60:0/3260502574 >> v2:172.21.15.100:3300/0 conn(0x7fc2e80d1ff0 0x7fc2e80d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2e659c640 1 -- 172.21.15.60:0/3260502574 >> v2:172.21.15.100:3300/0 conn(0x7fc2e80d1ff0 msgr2=0x7fc2e80d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2e659c640 1 --2- 172.21.15.60:0/3260502574 >> v2:172.21.15.100:3300/0 conn(0x7fc2e80d1ff0 0x7fc2e80d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2e659c640 1 -- 172.21.15.60:0/3260502574 >> v1:172.21.15.100:6789/0 conn(0x7fc2e80d0e40 legacy=0x7fc2e80d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2e659c640 1 -- 172.21.15.60:0/3260502574 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e80d25d0 con 0x7fc2e815dbe0 2024-04-25T10:09:26.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2e659c640 1 --2- 172.21.15.60:0/3260502574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2e815dbe0 0x7fc2e80d0900 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80d35d0 tx=0x7fc2e001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbdbff6681813ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2e559a640 1 -- 172.21.15.60:0/3260502574 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00a5750 con 0x7fc2e815dbe0 2024-04-25T10:09:26.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2e559a640 1 -- 172.21.15.60:0/3260502574 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e00a58f0 con 0x7fc2e815dbe0 2024-04-25T10:09:26.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.222+0000 7fc2e559a640 1 -- 172.21.15.60:0/3260502574 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00a5cc0 con 0x7fc2e815dbe0 2024-04-25T10:09:26.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 -- 172.21.15.60:0/3260502574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2e815dbe0 msgr2=0x7fc2e80d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 --2- 172.21.15.60:0/3260502574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2e815dbe0 0x7fc2e80d0900 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80d35d0 tx=0x7fc2e001c980 comp rx=0 tx=0).stop 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 -- 172.21.15.60:0/3260502574 shutdown_connections 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 --2- 172.21.15.60:0/3260502574 >> v2:172.21.15.100:3300/0 conn(0x7fc2e80d1ff0 0x7fc2e80d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 --2- 172.21.15.60:0/3260502574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2e815dbe0 0x7fc2e80d0900 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 -- 172.21.15.60:0/3260502574 >> 172.21.15.60:0/3260502574 conn(0x7fc2e8072510 msgr2=0x7fc2e8072920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 -- 172.21.15.60:0/3260502574 shutdown_connections 2024-04-25T10:09:26.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 -- 172.21.15.60:0/3260502574 wait complete. 2024-04-25T10:09:26.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 Processor -- start 2024-04-25T10:09:26.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 -- start start 2024-04-25T10:09:26.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2e80d0e40 0x7fc2e80cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2e80d1ff0 0x7fc2e815b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2e815dbe0 0x7fc2e8154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e80b6bd0 con 0x7fc2e80d1ff0 2024-04-25T10:09:26.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e80b6a50 con 0x7fc2e80d0e40 2024-04-25T10:09:26.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e80d4630 con 0x7fc2e815dbe0 2024-04-25T10:09:26.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2e5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2e80d1ff0 0x7fc2e815b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2e5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2e80d1ff0 0x7fc2e815b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42648/0 (socket says 172.21.15.60:42648) 2024-04-25T10:09:26.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2e5d9b640 1 -- 172.21.15.60:0/3459435824 learned_addr learned my addr 172.21.15.60:0/3459435824 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:26.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2e659c640 1 --2- 172.21.15.60:0/3459435824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2e80d0e40 0x7fc2e80cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2e5d9b640 1 -- 172.21.15.60:0/3459435824 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2e815dbe0 msgr2=0x7fc2e8154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2e5d9b640 1 --2- 172.21.15.60:0/3459435824 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2e815dbe0 0x7fc2e8154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2e5d9b640 1 -- 172.21.15.60:0/3459435824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2e80d0e40 msgr2=0x7fc2e80cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2e5d9b640 1 --2- 172.21.15.60:0/3459435824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2e80d0e40 0x7fc2e80cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2e5d9b640 1 -- 172.21.15.60:0/3459435824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e815bab0 con 0x7fc2e80d1ff0 2024-04-25T10:09:26.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2e5d9b640 1 --2- 172.21.15.60:0/3459435824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2e80d1ff0 0x7fc2e815b0c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000c490 tx=0x7fc2d0001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2deffd640 1 -- 172.21.15.60:0/3459435824 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0017070 con 0x7fc2e80d1ff0 2024-04-25T10:09:26.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2deffd640 1 -- 172.21.15.60:0/3459435824 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d0003600 con 0x7fc2e80d1ff0 2024-04-25T10:09:26.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 -- 172.21.15.60:0/3459435824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e80caf90 con 0x7fc2e80d1ff0 2024-04-25T10:09:26.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2deffd640 1 -- 172.21.15.60:0/3459435824 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00039f0 con 0x7fc2e80d1ff0 2024-04-25T10:09:26.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.226+0000 7fc2eceb0640 1 -- 172.21.15.60:0/3459435824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e80cb550 con 0x7fc2e80d1ff0 2024-04-25T10:09:26.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2deffd640 1 -- 172.21.15.60:0/3459435824 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc2d001a070 con 0x7fc2e80d1ff0 2024-04-25T10:09:26.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2deffd640 1 --2- 172.21.15.60:0/3459435824 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2c8066f60 0x7fc2c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2deffd640 1 -- 172.21.15.60:0/3459435824 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2d0084430 con 0x7fc2e80d1ff0 2024-04-25T10:09:26.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2f001d000 1 Processor -- start 2024-04-25T10:09:26.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2e659c640 1 --2- 172.21.15.60:0/3459435824 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2c8066f60 0x7fc2c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2f001d000 1 -- start start 2024-04-25T10:09:26.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2f001d000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf5102e0 0x564cbf5106c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2f001d000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf510c00 0x564cbf4c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2f001d000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf4c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2e659c640 1 --2- 172.21.15.60:0/3459435824 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2c8066f60 0x7fc2c8069420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00ae000 tx=0x7fc2e00a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2f001d000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cbf3adba0 con 0x564cbf4c68e0 2024-04-25T10:09:26.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2f001d000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cbf313090 con 0x564cbf510c00 2024-04-25T10:09:26.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2f001d000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x564cbf29b160 con 0x564cbf5102e0 2024-04-25T10:09:26.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2e6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf4c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2e659c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf5102e0 0x564cbf5106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2e6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf4c6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42660/0 (socket says 172.21.15.60:42660) 2024-04-25T10:09:26.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2e6d9d640 1 -- 172.21.15.60:0/3993495480 learned_addr learned my addr 172.21.15.60:0/3993495480 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:26.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.230+0000 7fc2e5d9b640 1 --2- 172.21.15.60:0/3993495480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf510c00 0x564cbf4c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e6d9d640 1 -- 172.21.15.60:0/3993495480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf5102e0 msgr2=0x564cbf5106c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e6d9d640 1 --2- 172.21.15.60:0/3993495480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf5102e0 0x564cbf5106c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e6d9d640 1 -- 172.21.15.60:0/3993495480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf510c00 msgr2=0x564cbf4c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e6d9d640 1 --2- 172.21.15.60:0/3993495480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf510c00 0x564cbf4c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e6d9d640 1 -- 172.21.15.60:0/3993495480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cbf3bcc20 con 0x564cbf4c68e0 2024-04-25T10:09:26.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e6d9d640 1 --2- 172.21.15.60:0/3993495480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf4c6cc0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800a910 tx=0x7fc2d800add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509ae15e0e199873 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2c7fff640 1 -- 172.21.15.60:0/3993495480 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8015070 con 0x564cbf4c68e0 2024-04-25T10:09:26.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2c7fff640 1 -- 172.21.15.60:0/3993495480 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d800cc00 con 0x564cbf4c68e0 2024-04-25T10:09:26.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2c7fff640 1 -- 172.21.15.60:0/3993495480 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8004020 con 0x564cbf4c68e0 2024-04-25T10:09:26.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 -- 172.21.15.60:0/3993495480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 msgr2=0x564cbf4c6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 --2- 172.21.15.60:0/3993495480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf4c6cc0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800a910 tx=0x7fc2d800add0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 -- 172.21.15.60:0/3993495480 shutdown_connections 2024-04-25T10:09:26.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 --2- 172.21.15.60:0/3993495480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf4c6cc0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 --2- 172.21.15.60:0/3993495480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf510c00 0x564cbf4c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 --2- 172.21.15.60:0/3993495480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf5102e0 0x564cbf5106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 -- 172.21.15.60:0/3993495480 >> 172.21.15.60:0/3993495480 conn(0x564cbf4c2890 msgr2=0x564cbf4c4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:26.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 -- 172.21.15.60:0/3993495480 shutdown_connections 2024-04-25T10:09:26.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 -- 172.21.15.60:0/3993495480 wait complete. 2024-04-25T10:09:26.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 -- start start 2024-04-25T10:09:26.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 Processor -- start 2024-04-25T10:09:26.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf4c68e0 0x564cbf517430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf5102e0 0x564cbf517970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf510c00 0x564cbf4d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cbf30f130 con 0x564cbf510c00 2024-04-25T10:09:26.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cbf2d7130 con 0x564cbf4c68e0 2024-04-25T10:09:26.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x564cbf3d1300 con 0x564cbf5102e0 2024-04-25T10:09:26.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 10 client.0 _ll_register_callbacks cb 0x564cbf4b1bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:26.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf510c00 0x564cbf4d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf510c00 0x564cbf4d1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42676/0 (socket says 172.21.15.60:42676) 2024-04-25T10:09:26.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e6d9d640 1 -- 172.21.15.60:0/1155669033 learned_addr learned my addr 172.21.15.60:0/1155669033 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:26.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e659c640 1 --2- 172.21.15.60:0/1155669033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf4c68e0 0x564cbf517430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e5d9b640 1 --2- 172.21.15.60:0/1155669033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf5102e0 0x564cbf517970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e6d9d640 1 -- 172.21.15.60:0/1155669033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf5102e0 msgr2=0x564cbf517970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e6d9d640 1 --2- 172.21.15.60:0/1155669033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf5102e0 0x564cbf517970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e6d9d640 1 -- 172.21.15.60:0/1155669033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf4c68e0 msgr2=0x564cbf517430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e6d9d640 1 --2- 172.21.15.60:0/1155669033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf4c68e0 0x564cbf517430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e6d9d640 1 -- 172.21.15.60:0/1155669033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cbf2f0060 con 0x564cbf510c00 2024-04-25T10:09:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2e6d9d640 1 --2- 172.21.15.60:0/1155669033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf510c00 0x564cbf4d1090 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x564cbf316150 tx=0x7fc2d800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d30480b64d47e7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:09:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2c57fa640 1 -- 172.21.15.60:0/1155669033 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8028060 con 0x564cbf510c00 2024-04-25T10:09:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2c57fa640 1 -- 172.21.15.60:0/1155669033 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2d800ca00 con 0x564cbf510c00 2024-04-25T10:09:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 -- 172.21.15.60:0/1155669033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2d800cc00 con 0x564cbf510c00 2024-04-25T10:09:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2c57fa640 1 -- 172.21.15.60:0/1155669033 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2d802e050 con 0x564cbf510c00 2024-04-25T10:09:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 10 client.6678 fetch_fsmap learned FSMap version 30 2024-04-25T10:09:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2f001d000 1 -- 172.21.15.60:0/1155669033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x564cbf2b47c0 con 0x564cbf510c00 2024-04-25T10:09:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2c57fa640 1 -- 172.21.15.60:0/1155669033 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d800cc00 con 0x564cbf510c00 2024-04-25T10:09:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.234+0000 7fc2c57fa640 1 -- 172.21.15.60:0/1155669033 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc2d8031020 con 0x564cbf510c00 2024-04-25T10:09:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 10 client.6678 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T10:09:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 10 client.6678 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:09:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 20 client.6678 populate_metadata read hostname 'smithi060' 2024-04-25T10:09:26.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 1 --2- 172.21.15.60:0/1155669033 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x564cbf4d1e30 0x564cbf4d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 4 client.6678 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:26.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 1 -- 172.21.15.60:0/1155669033 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564cbf2aecd0 con 0x564cbf4d1e30 2024-04-25T10:09:26.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2e659c640 1 --2- 172.21.15.60:0/1155669033 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x564cbf4d1e30 0x564cbf4d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2e659c640 1 --2- 172.21.15.60:0/1155669033 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x564cbf4d1e30 0x564cbf4d2210 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fed84cb459ca6138 server_cookie=956b4dbb19cd8ddd in_seq=0 out_seq=0 2024-04-25T10:09:26.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2c57fa640 10 client.6678 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:09:26.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2c57fa640 1 -- 172.21.15.60:0/1155669033 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564cbf2aecd0 con 0x564cbf4d1e30 2024-04-25T10:09:26.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2c57fa640 10 client.6678 handle_command_reply: tid=0 2024-04-25T10:09:26.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 1 client.6678 shutdown 2024-04-25T10:09:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 1 -- 172.21.15.60:0/1155669033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf510c00 msgr2=0x564cbf4d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 1 --2- 172.21.15.60:0/1155669033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf510c00 0x564cbf4d1090 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x564cbf316150 tx=0x7fc2d800f000 comp rx=0 tx=0).stop 2024-04-25T10:09:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 1 -- 172.21.15.60:0/1155669033 shutdown_connections 2024-04-25T10:09:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 1 --2- 172.21.15.60:0/1155669033 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x564cbf4d1e30 0x564cbf4d2210 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 1 --2- 172.21.15.60:0/1155669033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf510c00 0x564cbf4d1090 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2c57fa640 0 client.6678 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:09:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 1 --2- 172.21.15.60:0/1155669033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf5102e0 0x564cbf517970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 1 --2- 172.21.15.60:0/1155669033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf4c68e0 0x564cbf517430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 1 -- 172.21.15.60:0/1155669033 >> 172.21.15.60:0/1155669033 conn(0x564cbf4c2890 msgr2=0x564cbf4c5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 1 -- 172.21.15.60:0/1155669033 shutdown_connections 2024-04-25T10:09:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 1 -- 172.21.15.60:0/1155669033 wait complete. 2024-04-25T10:09:26.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.238+0000 7fc2f001d000 20 client.6678 trim_cache size 0 max 16384 2024-04-25T10:09:26.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 Processor -- start 2024-04-25T10:09:26.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 -- start start 2024-04-25T10:09:26.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf4c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf5102e0 0x564cbf4c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf510c00 0x564cbf50e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e8201cd0 con 0x564cbf4c68e0 2024-04-25T10:09:26.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e8155500 con 0x564cbf5102e0 2024-04-25T10:09:26.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e80ca0d0 con 0x564cbf510c00 2024-04-25T10:09:26.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2e659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf4c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2e659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf4c5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42688/0 (socket says 172.21.15.60:42688) 2024-04-25T10:09:26.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2e659c640 1 -- 172.21.15.60:0/3215117080 learned_addr learned my addr 172.21.15.60:0/3215117080 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:26.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2e659c640 1 -- 172.21.15.60:0/3215117080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf510c00 msgr2=0x564cbf50e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2e659c640 1 --2- 172.21.15.60:0/3215117080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf510c00 0x564cbf50e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2e659c640 1 -- 172.21.15.60:0/3215117080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf5102e0 msgr2=0x564cbf4c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2e659c640 1 --2- 172.21.15.60:0/3215117080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf5102e0 0x564cbf4c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2e659c640 1 -- 172.21.15.60:0/3215117080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cbf2cefc0 con 0x564cbf4c68e0 2024-04-25T10:09:26.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2e659c640 1 --2- 172.21.15.60:0/3215117080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf4c5c90 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009c90 tx=0x7fc2e00a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67b4bcc37710b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2c77fe640 1 -- 172.21.15.60:0/3215117080 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00cd070 con 0x564cbf4c68e0 2024-04-25T10:09:26.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2c77fe640 1 -- 172.21.15.60:0/3215117080 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e00c7230 con 0x564cbf4c68e0 2024-04-25T10:09:26.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2c77fe640 1 -- 172.21.15.60:0/3215117080 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00c75e0 con 0x564cbf4c68e0 2024-04-25T10:09:26.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 -- 172.21.15.60:0/3215117080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 msgr2=0x564cbf4c5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 --2- 172.21.15.60:0/3215117080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf4c5c90 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009c90 tx=0x7fc2e00a7000 comp rx=0 tx=0).stop 2024-04-25T10:09:26.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 -- 172.21.15.60:0/3215117080 shutdown_connections 2024-04-25T10:09:26.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 --2- 172.21.15.60:0/3215117080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf510c00 0x564cbf50e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 --2- 172.21.15.60:0/3215117080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf5102e0 0x564cbf4c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 --2- 172.21.15.60:0/3215117080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf4c5c90 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 -- 172.21.15.60:0/3215117080 >> 172.21.15.60:0/3215117080 conn(0x564cbf4c2890 msgr2=0x564cbf51db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:26.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 -- 172.21.15.60:0/3215117080 shutdown_connections 2024-04-25T10:09:26.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 -- 172.21.15.60:0/3215117080 wait complete. 2024-04-25T10:09:26.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 -- start start 2024-04-25T10:09:26.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.250+0000 7fc2f001d000 1 Processor -- start 2024-04-25T10:09:26.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf570720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf5102e0 0x564cbf570c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf510c00 0x564cbf5149b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cbf2ade70 con 0x564cbf4c68e0 2024-04-25T10:09:26.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cbf29ba50 con 0x564cbf510c00 2024-04-25T10:09:26.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x564cbf3123a0 con 0x564cbf5102e0 2024-04-25T10:09:26.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 10 client.0 _ll_register_callbacks cb 0x564cbf2bf240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:26.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2e659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf570720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2e659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf570720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42702/0 (socket says 172.21.15.60:42702) 2024-04-25T10:09:26.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2e659c640 1 -- 172.21.15.60:0/1753411548 learned_addr learned my addr 172.21.15.60:0/1753411548 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:26.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2e659c640 1 -- 172.21.15.60:0/1753411548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf5102e0 msgr2=0x564cbf570c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2e659c640 1 --2- 172.21.15.60:0/1753411548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf5102e0 0x564cbf570c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2e659c640 1 -- 172.21.15.60:0/1753411548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf510c00 msgr2=0x564cbf5149b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2e659c640 1 --2- 172.21.15.60:0/1753411548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf510c00 0x564cbf5149b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2e659c640 1 -- 172.21.15.60:0/1753411548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cbf36c9e0 con 0x564cbf4c68e0 2024-04-25T10:09:26.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2e659c640 1 --2- 172.21.15.60:0/1753411548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf570720 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc2e001ce60 tx=0x7fc2e001b1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7243f2f84cfb22c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:09:26.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2c57fa640 1 -- 172.21.15.60:0/1753411548 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00cd040 con 0x564cbf4c68e0 2024-04-25T10:09:26.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2c57fa640 1 -- 172.21.15.60:0/1753411548 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2e0003ba0 con 0x564cbf4c68e0 2024-04-25T10:09:26.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 -- 172.21.15.60:0/1753411548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2e00c7230 con 0x564cbf4c68e0 2024-04-25T10:09:26.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2c57fa640 1 -- 172.21.15.60:0/1753411548 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2e0003da0 con 0x564cbf4c68e0 2024-04-25T10:09:26.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 10 client.6684 fetch_fsmap learned FSMap version 30 2024-04-25T10:09:26.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 -- 172.21.15.60:0/1753411548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x564cbf2b2630 con 0x564cbf4c68e0 2024-04-25T10:09:26.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2c57fa640 1 -- 172.21.15.60:0/1753411548 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e00c7230 con 0x564cbf4c68e0 2024-04-25T10:09:26.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2c57fa640 1 -- 172.21.15.60:0/1753411548 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc2e00e0080 con 0x564cbf4c68e0 2024-04-25T10:09:26.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 10 client.6684 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T10:09:26.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 10 client.6684 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:09:26.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 20 client.6684 populate_metadata read hostname 'smithi060' 2024-04-25T10:09:26.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 --2- 172.21.15.60:0/1753411548 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x564cbf515370 0x564cbf515750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 4 client.6684 mds_command: new command op to 6291 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:09:26.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 -- 172.21.15.60:0/1753411548 --> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x564cbf3205d0 con 0x564cbf515370 2024-04-25T10:09:26.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2e5d9b640 1 --2- 172.21.15.60:0/1753411548 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x564cbf515370 0x564cbf515750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2c57fa640 10 client.6684 ms_handle_connect on v2:172.21.15.60:6833/2718963870 2024-04-25T10:09:26.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2e5d9b640 1 --2- 172.21.15.60:0/1753411548 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x564cbf515370 0x564cbf515750 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd20afbaa562f05e server_cookie=f44547f18fdb2d01 in_seq=0 out_seq=0 2024-04-25T10:09:26.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2c57fa640 1 -- 172.21.15.60:0/1753411548 <== mds.0 v2:172.21.15.60:6833/2718963870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564cbf3205d0 con 0x564cbf515370 2024-04-25T10:09:26.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2c57fa640 10 client.6684 handle_command_reply: tid=0 2024-04-25T10:09:26.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 client.6684 shutdown 2024-04-25T10:09:26.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 -- 172.21.15.60:0/1753411548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 msgr2=0x564cbf570720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 --2- 172.21.15.60:0/1753411548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf570720 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc2e001ce60 tx=0x7fc2e001b1c0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 -- 172.21.15.60:0/1753411548 shutdown_connections 2024-04-25T10:09:26.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 --2- 172.21.15.60:0/1753411548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564cbf510c00 0x564cbf5149b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 --2- 172.21.15.60:0/1753411548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564cbf5102e0 0x564cbf570c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 --2- 172.21.15.60:0/1753411548 >> [v2:172.21.15.60:6833/2718963870,v1:172.21.15.60:6835/2718963870] conn(0x564cbf515370 0x564cbf515750 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 --2- 172.21.15.60:0/1753411548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564cbf4c68e0 0x564cbf570720 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2c57fa640 0 client.6684 ms_handle_reset on v2:172.21.15.60:6833/2718963870 2024-04-25T10:09:26.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 -- 172.21.15.60:0/1753411548 >> 172.21.15.60:0/1753411548 conn(0x564cbf4c2890 msgr2=0x564cbf4c4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:26.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 -- 172.21.15.60:0/1753411548 shutdown_connections 2024-04-25T10:09:26.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 1 -- 172.21.15.60:0/1753411548 wait complete. 2024-04-25T10:09:26.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2f001d000 20 client.6684 trim_cache size 0 max 16384 2024-04-25T10:09:26.274 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:26.274 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-04-25T10:09:26.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2eceb0640 1 -- 172.21.15.60:0/3459435824 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2c8066f60 msgr2=0x7fc2c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2eceb0640 1 --2- 172.21.15.60:0/3459435824 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2c8066f60 0x7fc2c8069420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00ae000 tx=0x7fc2e00a7070 comp rx=0 tx=0).stop 2024-04-25T10:09:26.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2eceb0640 1 -- 172.21.15.60:0/3459435824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2e80d1ff0 msgr2=0x7fc2e815b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2eceb0640 1 --2- 172.21.15.60:0/3459435824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2e80d1ff0 0x7fc2e815b0c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000c490 tx=0x7fc2d0001e80 comp rx=0 tx=0).stop 2024-04-25T10:09:26.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2eceb0640 1 -- 172.21.15.60:0/3459435824 shutdown_connections 2024-04-25T10:09:26.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2eceb0640 1 --2- 172.21.15.60:0/3459435824 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2c8066f60 0x7fc2c8069420 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2eceb0640 1 --2- 172.21.15.60:0/3459435824 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2e815dbe0 0x7fc2e8154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2eceb0640 1 --2- 172.21.15.60:0/3459435824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2e80d1ff0 0x7fc2e815b0c0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2eceb0640 1 --2- 172.21.15.60:0/3459435824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2e80d0e40 0x7fc2e80cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2eceb0640 1 -- 172.21.15.60:0/3459435824 >> 172.21.15.60:0/3459435824 conn(0x7fc2e8072510 msgr2=0x7fc2e80cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:26.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2eceb0640 1 -- 172.21.15.60:0/3459435824 shutdown_connections 2024-04-25T10:09:26.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.254+0000 7fc2eceb0640 1 -- 172.21.15.60:0/3459435824 wait complete. 2024-04-25T10:09:26.279 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:09:26.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.378+0000 7fc2390c5640 1 Processor -- start 2024-04-25T10:09:26.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.378+0000 7fc2390c5640 1 -- start start 2024-04-25T10:09:26.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.378+0000 7fc2390c5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2340b3610 0x7fc23406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.378+0000 7fc2390c5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc23406b150 0x7fc2340b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.378+0000 7fc2390c5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2340b3bd0 con 0x7fc2340d11b0 2024-04-25T10:09:26.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.378+0000 7fc2390c5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2340b5b70 con 0x7fc23406b150 2024-04-25T10:09:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.378+0000 7fc2390c5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2340b55b0 con 0x7fc2340b3610 2024-04-25T10:09:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.378+0000 7fc23359e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc23406b150 0x7fc2340b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.378+0000 7fc23259c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2340b3610 0x7fc23406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.378+0000 7fc23359e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc23406b150 0x7fc2340b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:40650/0 (socket says 172.21.15.60:40650) 2024-04-25T10:09:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.378+0000 7fc232d9d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc2340d11b0 0x7fc23415ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:47740/0 (socket says 172.21.15.60:47740) 2024-04-25T10:09:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.378+0000 7fc23359e640 1 -- 172.21.15.60:0/1446618309 learned_addr learned my addr 172.21.15.60:0/1446618309 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.378+0000 7fc23359e640 1 -- 172.21.15.60:0/1446618309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2340b3610 msgr2=0x7fc23406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.378+0000 7fc23359e640 1 --2- 172.21.15.60:0/1446618309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2340b3610 0x7fc23406ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.378+0000 7fc23359e640 1 -- 172.21.15.60:0/1446618309 >> v1:172.21.15.100:6789/0 conn(0x7fc2340d11b0 legacy=0x7fc23415ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.378+0000 7fc23359e640 1 -- 172.21.15.60:0/1446618309 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2340b5890 con 0x7fc23406b150 2024-04-25T10:09:26.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.382+0000 7fc23359e640 1 --2- 172.21.15.60:0/1446618309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc23406b150 0x7fc2340b05b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009e50 tx=0x7fc22c01c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e7507e4f0df3a8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.382+0000 7fc231d9b640 1 -- 172.21.15.60:0/1446618309 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c0a5660 con 0x7fc23406b150 2024-04-25T10:09:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.382+0000 7fc231d9b640 1 -- 172.21.15.60:0/1446618309 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22c0a5800 con 0x7fc23406b150 2024-04-25T10:09:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.382+0000 7fc231d9b640 1 -- 172.21.15.60:0/1446618309 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c0a5bd0 con 0x7fc23406b150 2024-04-25T10:09:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.382+0000 7fc2390c5640 1 -- 172.21.15.60:0/1446618309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc23406b150 msgr2=0x7fc2340b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.382+0000 7fc2390c5640 1 --2- 172.21.15.60:0/1446618309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc23406b150 0x7fc2340b05b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009e50 tx=0x7fc22c01c9a0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.382+0000 7fc2390c5640 1 -- 172.21.15.60:0/1446618309 shutdown_connections 2024-04-25T10:09:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.382+0000 7fc2390c5640 1 --2- 172.21.15.60:0/1446618309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc23406b150 0x7fc2340b05b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.382+0000 7fc2390c5640 1 --2- 172.21.15.60:0/1446618309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2340b3610 0x7fc23406ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.382+0000 7fc2390c5640 1 -- 172.21.15.60:0/1446618309 >> 172.21.15.60:0/1446618309 conn(0x7fc234072900 msgr2=0x7fc234099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.382+0000 7fc2390c5640 1 -- 172.21.15.60:0/1446618309 shutdown_connections 2024-04-25T10:09:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.382+0000 7fc2390c5640 1 -- 172.21.15.60:0/1446618309 wait complete. 2024-04-25T10:09:26.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.382+0000 7fc2390c5640 1 Processor -- start 2024-04-25T10:09:26.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc2390c5640 1 -- start start 2024-04-25T10:09:26.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc2390c5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc23406b150 0x7fc234141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc2390c5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2340b3610 0x7fc23415d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc2390c5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2340d11b0 0x7fc23415d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc2390c5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc234160120 con 0x7fc2340b3610 2024-04-25T10:09:26.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc2390c5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23415ffa0 con 0x7fc2340d11b0 2024-04-25T10:09:26.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc2390c5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2341602a0 con 0x7fc23406b150 2024-04-25T10:09:26.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc23259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2340b3610 0x7fc23415d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc23259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2340b3610 0x7fc23415d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42708/0 (socket says 172.21.15.60:42708) 2024-04-25T10:09:26.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc23259c640 1 -- 172.21.15.60:0/2048631737 learned_addr learned my addr 172.21.15.60:0/2048631737 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:26.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc23359e640 1 --2- 172.21.15.60:0/2048631737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2340d11b0 0x7fc23415d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc232d9d640 1 --2- 172.21.15.60:0/2048631737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc23406b150 0x7fc234141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc23259c640 1 -- 172.21.15.60:0/2048631737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc23406b150 msgr2=0x7fc234141e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc23259c640 1 --2- 172.21.15.60:0/2048631737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc23406b150 0x7fc234141e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc23259c640 1 -- 172.21.15.60:0/2048631737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2340d11b0 msgr2=0x7fc23415d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc23259c640 1 --2- 172.21.15.60:0/2048631737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2340d11b0 0x7fc23415d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc23259c640 1 -- 172.21.15.60:0/2048631737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2340ae400 con 0x7fc2340b3610 2024-04-25T10:09:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc23259c640 1 --2- 172.21.15.60:0/2048631737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2340b3610 0x7fc23415d490 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc22800e960 tx=0x7fc22800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc21b7fe640 1 -- 172.21.15.60:0/2048631737 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc228017070 con 0x7fc2340b3610 2024-04-25T10:09:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc21b7fe640 1 -- 172.21.15.60:0/2048631737 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22800ca00 con 0x7fc2340b3610 2024-04-25T10:09:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc2390c5640 1 -- 172.21.15.60:0/2048631737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2340ae6e0 con 0x7fc2340b3610 2024-04-25T10:09:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc21b7fe640 1 -- 172.21.15.60:0/2048631737 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22800cdb0 con 0x7fc2340b3610 2024-04-25T10:09:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc2390c5640 1 -- 172.21.15.60:0/2048631737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2340aeca0 con 0x7fc2340b3610 2024-04-25T10:09:26.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc2390c5640 1 -- 172.21.15.60:0/2048631737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23406ccb0 con 0x7fc2340b3610 2024-04-25T10:09:26.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc21b7fe640 1 -- 172.21.15.60:0/2048631737 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc22801a070 con 0x7fc2340b3610 2024-04-25T10:09:26.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc21b7fe640 1 --2- 172.21.15.60:0/2048631737 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc220066f90 0x7fc220069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc21b7fe640 1 -- 172.21.15.60:0/2048631737 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc228085420 con 0x7fc2340b3610 2024-04-25T10:09:26.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.386+0000 7fc232d9d640 1 --2- 172.21.15.60:0/2048631737 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc220066f90 0x7fc220069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.390+0000 7fc232d9d640 1 --2- 172.21.15.60:0/2048631737 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc220066f90 0x7fc220069450 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc224000f70 tx=0x7fc224009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.390+0000 7fc21b7fe640 1 -- 172.21.15.60:0/2048631737 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc228056420 con 0x7fc2340b3610 2024-04-25T10:09:26.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.578+0000 7fc2390c5640 1 -- 172.21.15.60:0/2048631737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc234074770 con 0x7fc2340b3610 2024-04-25T10:09:26.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.578+0000 7fc21b7fe640 1 -- 172.21.15.60:0/2048631737 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc228055d20 con 0x7fc2340b3610 2024-04-25T10:09:26.587 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:26.587 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:09:26.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.582+0000 7fc2390c5640 1 -- 172.21.15.60:0/2048631737 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc220066f90 msgr2=0x7fc220069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.582+0000 7fc2390c5640 1 --2- 172.21.15.60:0/2048631737 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc220066f90 0x7fc220069450 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc224000f70 tx=0x7fc224009040 comp rx=0 tx=0).stop 2024-04-25T10:09:26.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.582+0000 7fc2390c5640 1 -- 172.21.15.60:0/2048631737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2340b3610 msgr2=0x7fc23415d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.582+0000 7fc2390c5640 1 --2- 172.21.15.60:0/2048631737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2340b3610 0x7fc23415d490 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc22800e960 tx=0x7fc22800ee20 comp rx=0 tx=0).stop 2024-04-25T10:09:26.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.582+0000 7fc2390c5640 1 -- 172.21.15.60:0/2048631737 shutdown_connections 2024-04-25T10:09:26.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.582+0000 7fc2390c5640 1 --2- 172.21.15.60:0/2048631737 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc220066f90 0x7fc220069450 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.582+0000 7fc2390c5640 1 --2- 172.21.15.60:0/2048631737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2340d11b0 0x7fc23415d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.582+0000 7fc2390c5640 1 --2- 172.21.15.60:0/2048631737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2340b3610 0x7fc23415d490 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.582+0000 7fc2390c5640 1 --2- 172.21.15.60:0/2048631737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc23406b150 0x7fc234141e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.582+0000 7fc2390c5640 1 -- 172.21.15.60:0/2048631737 >> 172.21.15.60:0/2048631737 conn(0x7fc234072900 msgr2=0x7fc23415c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:26.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.582+0000 7fc2390c5640 1 -- 172.21.15.60:0/2048631737 shutdown_connections 2024-04-25T10:09:26.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.582+0000 7fc2390c5640 1 -- 172.21.15.60:0/2048631737 wait complete. 2024-04-25T10:09:26.604 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:09:26.605 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:09:26.605 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:09:26.658 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:09:26.706 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:09:26.706 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:09:26.706 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:26.712 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:26.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd070e93640 1 Processor -- start 2024-04-25T10:09:26.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd070e93640 1 -- start start 2024-04-25T10:09:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd070e93640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06c15bc90 0x7fd06c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd070e93640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd06c0b4660 0x7fd06c15c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd070e93640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06c071ca0 con 0x7fd06c0ce7a0 2024-04-25T10:09:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd070e93640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06c0cc370 con 0x7fd06c0b4660 2024-04-25T10:09:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd070e93640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0cca40 con 0x7fd06c15bc90 2024-04-25T10:09:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06a59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd06c0ce7a0 0x7fd06c0ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40466/0 (socket says 172.21.15.60:40466) 2024-04-25T10:09:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06a59c640 1 -- 172.21.15.60:0/607397282 learned_addr learned my addr 172.21.15.60:0/607397282 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd069d9b640 1 --2- 172.21.15.60:0/607397282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06c15bc90 0x7fd06c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06ad9d640 1 --2- 172.21.15.60:0/607397282 >> v2:172.21.15.100:3300/0 conn(0x7fd06c0b4660 0x7fd06c15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06959a640 1 -- 172.21.15.60:0/607397282 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569457259 0 0) 0x7fd06c071ca0 con 0x7fd06c0ce7a0 2024-04-25T10:09:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06959a640 1 -- 172.21.15.60:0/607397282 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd054003530 con 0x7fd06c0ce7a0 2024-04-25T10:09:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06959a640 1 -- 172.21.15.60:0/607397282 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3685731322 0 0) 0x7fd054003530 con 0x7fd06c0ce7a0 2024-04-25T10:09:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06959a640 1 -- 172.21.15.60:0/607397282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06c15bc90 msgr2=0x7fd06c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06959a640 1 --2- 172.21.15.60:0/607397282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06c15bc90 0x7fd06c15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06959a640 1 -- 172.21.15.60:0/607397282 >> v2:172.21.15.100:3300/0 conn(0x7fd06c0b4660 msgr2=0x7fd06c15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06959a640 1 --2- 172.21.15.60:0/607397282 >> v2:172.21.15.100:3300/0 conn(0x7fd06c0b4660 0x7fd06c15c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06959a640 1 -- 172.21.15.60:0/607397282 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c0ccd20 con 0x7fd06c0ce7a0 2024-04-25T10:09:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06959a640 1 -- 172.21.15.60:0/607397282 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fd05c001680 con 0x7fd06c0ce7a0 2024-04-25T10:09:26.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06959a640 1 -- 172.21.15.60:0/607397282 >> v1:172.21.15.60:6789/0 conn(0x7fd06c0ce7a0 legacy=0x7fd06c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06959a640 1 --2- 172.21.15.60:0/607397282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0540039c0 0x7fd054003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06959a640 1 --2- 172.21.15.60:0/607397282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd054004810 0x7fd054004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06959a640 1 --2- 172.21.15.60:0/607397282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06c15bc90 0x7fd0540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06959a640 1 -- 172.21.15.60:0/607397282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0cca40 con 0x7fd054004810 2024-04-25T10:09:26.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06959a640 1 -- 172.21.15.60:0/607397282 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05400b9d0 con 0x7fd0540039c0 2024-04-25T10:09:26.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06959a640 1 -- 172.21.15.60:0/607397282 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05400bb50 con 0x7fd06c15bc90 2024-04-25T10:09:26.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd069d9b640 1 --2- 172.21.15.60:0/607397282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd054004810 0x7fd054004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06a59c640 1 --2- 172.21.15.60:0/607397282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0540039c0 0x7fd054003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06ad9d640 1 --2- 172.21.15.60:0/607397282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06c15bc90 0x7fd0540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd069d9b640 1 -- 172.21.15.60:0/607397282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06c15bc90 msgr2=0x7fd0540051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd069d9b640 1 --2- 172.21.15.60:0/607397282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06c15bc90 0x7fd0540051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd069d9b640 1 -- 172.21.15.60:0/607397282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0540039c0 msgr2=0x7fd054003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:26.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd069d9b640 1 --2- 172.21.15.60:0/607397282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0540039c0 0x7fd054003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd069d9b640 1 -- 172.21.15.60:0/607397282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd054003530 con 0x7fd054004810 2024-04-25T10:09:26.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.862+0000 7fd06ad9d640 1 --2- 172.21.15.60:0/607397282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06c15bc90 0x7fd0540051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:26.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.866+0000 7fd069d9b640 1 --2- 172.21.15.60:0/607397282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd054004810 0x7fd054004c90 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fd060009ae0 tx=0x7fd06001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fbc2f1478ab211 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.866+0000 7fd06959a640 1 -- 172.21.15.60:0/607397282 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0600096b0 con 0x7fd054004810 2024-04-25T10:09:26.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.866+0000 7fd06959a640 1 -- 172.21.15.60:0/607397282 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0600a6480 con 0x7fd054004810 2024-04-25T10:09:26.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.866+0000 7fd06959a640 1 -- 172.21.15.60:0/607397282 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0600a6870 con 0x7fd054004810 2024-04-25T10:09:26.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.866+0000 7fd070e93640 1 -- 172.21.15.60:0/607397282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd054004810 msgr2=0x7fd054004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.866+0000 7fd070e93640 1 --2- 172.21.15.60:0/607397282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd054004810 0x7fd054004c90 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fd060009ae0 tx=0x7fd06001ca70 comp rx=0 tx=0).stop 2024-04-25T10:09:26.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.866+0000 7fd06ad9d640 1 -- 172.21.15.60:0/607397282 reap_dead start 2024-04-25T10:09:26.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.866+0000 7fd070e93640 1 -- 172.21.15.60:0/607397282 shutdown_connections 2024-04-25T10:09:26.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.866+0000 7fd070e93640 1 -- 172.21.15.60:0/607397282 >> 172.21.15.60:0/607397282 conn(0x7fd06c0b12d0 msgr2=0x7fd06c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:26.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.866+0000 7fd070e93640 1 -- 172.21.15.60:0/607397282 shutdown_connections 2024-04-25T10:09:26.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.866+0000 7fd070e93640 1 -- 172.21.15.60:0/607397282 wait complete. 2024-04-25T10:09:26.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.866+0000 7fd070e93640 1 Processor -- start 2024-04-25T10:09:26.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.866+0000 7fd070e93640 1 -- start start 2024-04-25T10:09:26.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd070e93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd054004810 0x7fd06c150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd070e93640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06c1f2730 0x7fd06c1e49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd070e93640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd06c1e4f10 0x7fd06c1f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd070e93640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0b3d20 con 0x7fd06c1e4f10 2024-04-25T10:09:26.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd070e93640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0b3ba0 con 0x7fd054004810 2024-04-25T10:09:26.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd070e93640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0b4280 con 0x7fd06c1f2730 2024-04-25T10:09:26.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd06ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd06c1e4f10 0x7fd06c1f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd06ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd06c1e4f10 0x7fd06c1f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42718/0 (socket says 172.21.15.60:42718) 2024-04-25T10:09:26.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd06ad9d640 1 -- 172.21.15.60:0/2502998626 learned_addr learned my addr 172.21.15.60:0/2502998626 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:26.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd06a59c640 1 --2- 172.21.15.60:0/2502998626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd054004810 0x7fd06c150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd069d9b640 1 --2- 172.21.15.60:0/2502998626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06c1f2730 0x7fd06c1e49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd06ad9d640 1 -- 172.21.15.60:0/2502998626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06c1f2730 msgr2=0x7fd06c1e49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd06ad9d640 1 --2- 172.21.15.60:0/2502998626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06c1f2730 0x7fd06c1e49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd06ad9d640 1 -- 172.21.15.60:0/2502998626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd054004810 msgr2=0x7fd06c150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:26.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd06ad9d640 1 --2- 172.21.15.60:0/2502998626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd054004810 0x7fd06c150e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:26.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd06ad9d640 1 -- 172.21.15.60:0/2502998626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c1f19a0 con 0x7fd06c1e4f10 2024-04-25T10:09:26.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd069d9b640 1 --2- 172.21.15.60:0/2502998626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06c1f2730 0x7fd06c1e49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:26.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd06ad9d640 1 --2- 172.21.15.60:0/2502998626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd06c1e4f10 0x7fd06c1f4fc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd06c072c10 tx=0x7fd06400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd05affd640 1 -- 172.21.15.60:0/2502998626 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06401c070 con 0x7fd06c1e4f10 2024-04-25T10:09:26.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd05affd640 1 -- 172.21.15.60:0/2502998626 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd064017400 con 0x7fd06c1e4f10 2024-04-25T10:09:26.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd070e93640 1 -- 172.21.15.60:0/2502998626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06c1f5cd0 con 0x7fd06c1e4f10 2024-04-25T10:09:26.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd05affd640 1 -- 172.21.15.60:0/2502998626 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0640177d0 con 0x7fd06c1e4f10 2024-04-25T10:09:26.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd070e93640 1 -- 172.21.15.60:0/2502998626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06c1f1f30 con 0x7fd06c1e4f10 2024-04-25T10:09:26.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd070e93640 1 -- 172.21.15.60:0/2502998626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06c074770 con 0x7fd06c1e4f10 2024-04-25T10:09:26.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd05affd640 1 -- 172.21.15.60:0/2502998626 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd06401f070 con 0x7fd06c1e4f10 2024-04-25T10:09:26.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd05affd640 1 --2- 172.21.15.60:0/2502998626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd04c066f60 0x7fd04c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:26.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd05affd640 1 -- 172.21.15.60:0/2502998626 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd064085140 con 0x7fd06c1e4f10 2024-04-25T10:09:26.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.870+0000 7fd06a59c640 1 --2- 172.21.15.60:0/2502998626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd04c066f60 0x7fd04c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:26.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.874+0000 7fd06a59c640 1 --2- 172.21.15.60:0/2502998626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd04c066f60 0x7fd04c069420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd05c000fc0 tx=0x7fd05c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:26.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:26.874+0000 7fd05affd640 1 -- 172.21.15.60:0/2502998626 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd064012070 con 0x7fd06c1e4f10 2024-04-25T10:09:27.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.030+0000 7fd070e93640 1 -- 172.21.15.60:0/2502998626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd06c0cd420 con 0x7fd06c1e4f10 2024-04-25T10:09:27.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.034+0000 7fd05affd640 1 -- 172.21.15.60:0/2502998626 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16677 (secure 0 0 0) 0x7fd0640561e0 con 0x7fd06c1e4f10 2024-04-25T10:09:27.042 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:27.043 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":58,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:08:11.878055+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:08.538247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:09.233842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:27.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.038+0000 7fd070e93640 1 -- 172.21.15.60:0/2502998626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd04c066f60 msgr2=0x7fd04c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.038+0000 7fd070e93640 1 --2- 172.21.15.60:0/2502998626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd04c066f60 0x7fd04c069420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd05c000fc0 tx=0x7fd05c009040 comp rx=0 tx=0).stop 2024-04-25T10:09:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.038+0000 7fd070e93640 1 -- 172.21.15.60:0/2502998626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd06c1e4f10 msgr2=0x7fd06c1f4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.038+0000 7fd070e93640 1 --2- 172.21.15.60:0/2502998626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd06c1e4f10 0x7fd06c1f4fc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd06c072c10 tx=0x7fd06400f4a0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.038+0000 7fd070e93640 1 -- 172.21.15.60:0/2502998626 shutdown_connections 2024-04-25T10:09:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.038+0000 7fd070e93640 1 --2- 172.21.15.60:0/2502998626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd04c066f60 0x7fd04c069420 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.038+0000 7fd070e93640 1 --2- 172.21.15.60:0/2502998626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd06c1e4f10 0x7fd06c1f4fc0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.038+0000 7fd070e93640 1 --2- 172.21.15.60:0/2502998626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06c1f2730 0x7fd06c1e49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.038+0000 7fd070e93640 1 --2- 172.21.15.60:0/2502998626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd054004810 0x7fd06c150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.038+0000 7fd070e93640 1 -- 172.21.15.60:0/2502998626 >> 172.21.15.60:0/2502998626 conn(0x7fd06c0b12d0 msgr2=0x7fd06c071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.038+0000 7fd070e93640 1 -- 172.21.15.60:0/2502998626 shutdown_connections 2024-04-25T10:09:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.038+0000 7fd070e93640 1 -- 172.21.15.60:0/2502998626 wait complete. 2024-04-25T10:09:27.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:08.538247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:09.233842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:27.061 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:27.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.182+0000 7fe7fade6640 1 Processor -- start 2024-04-25T10:09:27.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.182+0000 7fe7fade6640 1 -- start start 2024-04-25T10:09:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7fade6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe7f40aeaa0 0x7fe7f4166450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7fade6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe7f40b2f70 0x7fe7f40b3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7fade6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f4166990 con 0x7fe7f4162450 2024-04-25T10:09:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7fade6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f4162910 con 0x7fe7f40b2f70 2024-04-25T10:09:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7fade6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f4162bf0 con 0x7fe7f40aeaa0 2024-04-25T10:09:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7f9383640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe7f40b2f70 0x7fe7f40b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7f9383640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe7f40b2f70 0x7fe7f40b3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42728/0 (socket says 172.21.15.60:42728) 2024-04-25T10:09:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7f9383640 1 -- 172.21.15.60:0/371131795 learned_addr learned my addr 172.21.15.60:0/371131795 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:27.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7f8b82640 1 --2- 172.21.15.60:0/371131795 >> v2:172.21.15.100:3300/0 conn(0x7fe7f40aeaa0 0x7fe7f4166450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7f9383640 1 -- 172.21.15.60:0/371131795 >> v2:172.21.15.100:3300/0 conn(0x7fe7f40aeaa0 msgr2=0x7fe7f4166450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7f9383640 1 --2- 172.21.15.60:0/371131795 >> v2:172.21.15.100:3300/0 conn(0x7fe7f40aeaa0 0x7fe7f4166450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7f9383640 1 -- 172.21.15.60:0/371131795 >> v1:172.21.15.100:6789/0 conn(0x7fe7f4162450 legacy=0x7fe7f406db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7f9383640 1 -- 172.21.15.60:0/371131795 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f4162ed0 con 0x7fe7f40b2f70 2024-04-25T10:09:27.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7f9383640 1 --2- 172.21.15.60:0/371131795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40b2f70 0x7fe7f40b3400 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40726f0 tx=0x7fe7ec01c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88773112f0934004 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7f37fe640 1 -- 172.21.15.60:0/371131795 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec0ac020 con 0x7fe7f40b2f70 2024-04-25T10:09:27.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7f37fe640 1 -- 172.21.15.60:0/371131795 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ec0a64a0 con 0x7fe7f40b2f70 2024-04-25T10:09:27.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7f37fe640 1 -- 172.21.15.60:0/371131795 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec0a6870 con 0x7fe7f40b2f70 2024-04-25T10:09:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7fade6640 1 -- 172.21.15.60:0/371131795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40b2f70 msgr2=0x7fe7f40b3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7fade6640 1 --2- 172.21.15.60:0/371131795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40b2f70 0x7fe7f40b3400 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40726f0 tx=0x7fe7ec01c7d0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7fade6640 1 -- 172.21.15.60:0/371131795 shutdown_connections 2024-04-25T10:09:27.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7fade6640 1 --2- 172.21.15.60:0/371131795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40b2f70 0x7fe7f40b3400 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.186+0000 7fe7fade6640 1 --2- 172.21.15.60:0/371131795 >> v2:172.21.15.100:3300/0 conn(0x7fe7f40aeaa0 0x7fe7f4166450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7fade6640 1 -- 172.21.15.60:0/371131795 >> 172.21.15.60:0/371131795 conn(0x7fe7f40670b0 msgr2=0x7fe7f406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:27.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7fade6640 1 -- 172.21.15.60:0/371131795 shutdown_connections 2024-04-25T10:09:27.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7fade6640 1 -- 172.21.15.60:0/371131795 wait complete. 2024-04-25T10:09:27.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7fade6640 1 Processor -- start 2024-04-25T10:09:27.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7fade6640 1 -- start start 2024-04-25T10:09:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7fade6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40aeaa0 0x7fe7f4151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7fade6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f40b2f70 0x7fe7f41503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7fade6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7f4162450 0x7fe7f40b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7fade6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f4071fa0 con 0x7fe7f40aeaa0 2024-04-25T10:09:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7fade6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f4071e20 con 0x7fe7f40b2f70 2024-04-25T10:09:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7fade6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f4072120 con 0x7fe7f4162450 2024-04-25T10:09:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7f8b82640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40aeaa0 0x7fe7f4151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7f8b82640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40aeaa0 0x7fe7f4151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42734/0 (socket says 172.21.15.60:42734) 2024-04-25T10:09:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7f8b82640 1 -- 172.21.15.60:0/1299662002 learned_addr learned my addr 172.21.15.60:0/1299662002 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7f9383640 1 --2- 172.21.15.60:0/1299662002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7f4162450 0x7fe7f40b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7f3fff640 1 --2- 172.21.15.60:0/1299662002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f40b2f70 0x7fe7f41503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7f8b82640 1 -- 172.21.15.60:0/1299662002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7f4162450 msgr2=0x7fe7f40b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7f8b82640 1 --2- 172.21.15.60:0/1299662002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7f4162450 0x7fe7f40b0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7f8b82640 1 -- 172.21.15.60:0/1299662002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f40b2f70 msgr2=0x7fe7f41503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7f8b82640 1 --2- 172.21.15.60:0/1299662002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f40b2f70 0x7fe7f41503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7f8b82640 1 -- 172.21.15.60:0/1299662002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f40b02c0 con 0x7fe7f40aeaa0 2024-04-25T10:09:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7f8b82640 1 --2- 172.21.15.60:0/1299662002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40aeaa0 0x7fe7f4151da0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400d830 tx=0x7fe7e400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7f17fa640 1 -- 172.21.15.60:0/1299662002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e400b6c0 con 0x7fe7f40aeaa0 2024-04-25T10:09:27.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7f17fa640 1 -- 172.21.15.60:0/1299662002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e4019070 con 0x7fe7f40aeaa0 2024-04-25T10:09:27.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7fade6640 1 -- 172.21.15.60:0/1299662002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f4152460 con 0x7fe7f40aeaa0 2024-04-25T10:09:27.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7f17fa640 1 -- 172.21.15.60:0/1299662002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e400ba60 con 0x7fe7f40aeaa0 2024-04-25T10:09:27.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.190+0000 7fe7fade6640 1 -- 172.21.15.60:0/1299662002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f4152a20 con 0x7fe7f40aeaa0 2024-04-25T10:09:27.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.194+0000 7fe7fade6640 1 -- 172.21.15.60:0/1299662002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f406bba0 con 0x7fe7f40aeaa0 2024-04-25T10:09:27.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.194+0000 7fe7f17fa640 1 -- 172.21.15.60:0/1299662002 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe7e401e460 con 0x7fe7f40aeaa0 2024-04-25T10:09:27.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.194+0000 7fe7f17fa640 1 --2- 172.21.15.60:0/1299662002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe7e0066f60 0x7fe7e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.194+0000 7fe7f17fa640 1 -- 172.21.15.60:0/1299662002 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7e4085710 con 0x7fe7f40aeaa0 2024-04-25T10:09:27.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.194+0000 7fe7f3fff640 1 --2- 172.21.15.60:0/1299662002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe7e0066f60 0x7fe7e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.194+0000 7fe7f3fff640 1 --2- 172.21.15.60:0/1299662002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe7e0066f60 0x7fe7e0069420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4150fe0 tx=0x7fe7e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.198+0000 7fe7f17fa640 1 -- 172.21.15.60:0/1299662002 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe7f40b02c0 con 0x7fe7f40aeaa0 2024-04-25T10:09:27.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.349+0000 7fe7fade6640 1 -- 172.21.15.60:0/1299662002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7f4074770 con 0x7fe7f40aeaa0 2024-04-25T10:09:27.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.353+0000 7fe7f17fa640 1 -- 172.21.15.60:0/1299662002 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe7e4056720 con 0x7fe7f40aeaa0 2024-04-25T10:09:27.361 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:09:27.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.353+0000 7fe7fade6640 1 -- 172.21.15.60:0/1299662002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe7e0066f60 msgr2=0x7fe7e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.353+0000 7fe7fade6640 1 --2- 172.21.15.60:0/1299662002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe7e0066f60 0x7fe7e0069420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4150fe0 tx=0x7fe7e8008040 comp rx=0 tx=0).stop 2024-04-25T10:09:27.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.353+0000 7fe7fade6640 1 -- 172.21.15.60:0/1299662002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40aeaa0 msgr2=0x7fe7f4151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.353+0000 7fe7fade6640 1 --2- 172.21.15.60:0/1299662002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40aeaa0 0x7fe7f4151da0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400d830 tx=0x7fe7e400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.353+0000 7fe7fade6640 1 -- 172.21.15.60:0/1299662002 shutdown_connections 2024-04-25T10:09:27.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.353+0000 7fe7fade6640 1 --2- 172.21.15.60:0/1299662002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe7e0066f60 0x7fe7e0069420 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.353+0000 7fe7fade6640 1 --2- 172.21.15.60:0/1299662002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7f4162450 0x7fe7f40b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.353+0000 7fe7fade6640 1 --2- 172.21.15.60:0/1299662002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f40b2f70 0x7fe7f41503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.353+0000 7fe7fade6640 1 --2- 172.21.15.60:0/1299662002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40aeaa0 0x7fe7f4151da0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.353+0000 7fe7fade6640 1 -- 172.21.15.60:0/1299662002 >> 172.21.15.60:0/1299662002 conn(0x7fe7f40670b0 msgr2=0x7fe7f41633f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:27.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.353+0000 7fe7fade6640 1 -- 172.21.15.60:0/1299662002 shutdown_connections 2024-04-25T10:09:27.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.353+0000 7fe7fade6640 1 -- 172.21.15.60:0/1299662002 wait complete. 2024-04-25T10:09:27.378 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:09:27.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.493+0000 7f431c145640 1 Processor -- start 2024-04-25T10:09:27.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.493+0000 7f431c145640 1 -- start start 2024-04-25T10:09:27.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f431c145640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f431415dd30 0x7f431415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f431c145640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43140cf8a0 0x7f431415be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f431c145640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43140cd890 con 0x7f43140cd3d0 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f431c145640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43140ce4a0 con 0x7f431415dd30 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f431c145640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43140cdb70 con 0x7f43140cf8a0 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f4319ee1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f431415dd30 0x7f431415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f43196e0640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f43140cd3d0 0x7f43140cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40470/0 (socket says 172.21.15.60:40470) 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f43196e0640 1 -- 172.21.15.60:0/2958944762 learned_addr learned my addr 172.21.15.60:0/2958944762 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f431a6e2640 1 --2- 172.21.15.60:0/2958944762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43140cf8a0 0x7f431415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f4319ee1640 1 -- 172.21.15.60:0/2958944762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43140cf8a0 msgr2=0x7f431415be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f4319ee1640 1 --2- 172.21.15.60:0/2958944762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43140cf8a0 0x7f431415be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f4319ee1640 1 -- 172.21.15.60:0/2958944762 >> v1:172.21.15.60:6789/0 conn(0x7f43140cd3d0 legacy=0x7f43140cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f4319ee1640 1 -- 172.21.15.60:0/2958944762 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43140cde50 con 0x7f431415dd30 2024-04-25T10:09:27.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f4318edf640 1 -- 172.21.15.60:0/2958944762 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893574024 0 0) 0x7f43140cd890 con 0x7f43140cd3d0 2024-04-25T10:09:27.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f431a6e2640 1 --2- 172.21.15.60:0/2958944762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43140cf8a0 0x7f431415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f4319ee1640 1 --2- 172.21.15.60:0/2958944762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f431415dd30 0x7f431415ac30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f43100097c0 tx=0x7f431001e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb268099ecf35b63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.497+0000 7f4318edf640 1 -- 172.21.15.60:0/2958944762 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43100aa640 con 0x7f431415dd30 2024-04-25T10:09:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f4318edf640 1 -- 172.21.15.60:0/2958944762 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43100aa7e0 con 0x7f431415dd30 2024-04-25T10:09:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f4318edf640 1 -- 172.21.15.60:0/2958944762 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43100aabb0 con 0x7f431415dd30 2024-04-25T10:09:27.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 -- 172.21.15.60:0/2958944762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f431415dd30 msgr2=0x7f431415ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 --2- 172.21.15.60:0/2958944762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f431415dd30 0x7f431415ac30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f43100097c0 tx=0x7f431001e5f0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 -- 172.21.15.60:0/2958944762 shutdown_connections 2024-04-25T10:09:27.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 --2- 172.21.15.60:0/2958944762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43140cf8a0 0x7f431415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 --2- 172.21.15.60:0/2958944762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f431415dd30 0x7f431415ac30 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 -- 172.21.15.60:0/2958944762 >> 172.21.15.60:0/2958944762 conn(0x7f43140670b0 msgr2=0x7f431406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:27.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 -- 172.21.15.60:0/2958944762 shutdown_connections 2024-04-25T10:09:27.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 -- 172.21.15.60:0/2958944762 wait complete. 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 Processor -- start 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 -- start start 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43140cd3d0 0x7f4314148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43140cf8a0 0x7f4314155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f431415dd30 0x7f4314146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431415cf00 con 0x7f43140cd3d0 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431415cd80 con 0x7f43140cf8a0 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431415d080 con 0x7f431415dd30 2024-04-25T10:09:27.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f4319ee1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43140cd3d0 0x7f4314148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f4319ee1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43140cd3d0 0x7f4314148c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42752/0 (socket says 172.21.15.60:42752) 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f4319ee1640 1 -- 172.21.15.60:0/2011456390 learned_addr learned my addr 172.21.15.60:0/2011456390 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431a6e2640 1 --2- 172.21.15.60:0/2011456390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f431415dd30 0x7f4314146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f4319ee1640 1 -- 172.21.15.60:0/2011456390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f431415dd30 msgr2=0x7f4314146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f4319ee1640 1 --2- 172.21.15.60:0/2011456390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f431415dd30 0x7f4314146070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f43196e0640 1 --2- 172.21.15.60:0/2011456390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43140cf8a0 0x7f4314155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f4319ee1640 1 -- 172.21.15.60:0/2011456390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43140cf8a0 msgr2=0x7f4314155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f4319ee1640 1 --2- 172.21.15.60:0/2011456390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43140cf8a0 0x7f4314155fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f4319ee1640 1 -- 172.21.15.60:0/2011456390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43141550a0 con 0x7f43140cd3d0 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f4319ee1640 1 --2- 172.21.15.60:0/2011456390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43140cd3d0 0x7f4314148c60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f431001cca0 tx=0x7f431001ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f430a7fc640 1 -- 172.21.15.60:0/2011456390 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43100b2a10 con 0x7f43140cd3d0 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f430a7fc640 1 -- 172.21.15.60:0/2011456390 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43100bf070 con 0x7f43140cd3d0 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f430a7fc640 1 -- 172.21.15.60:0/2011456390 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43100b2e10 con 0x7f43140cd3d0 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 -- 172.21.15.60:0/2011456390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4314155380 con 0x7f43140cd3d0 2024-04-25T10:09:27.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.501+0000 7f431c145640 1 -- 172.21.15.60:0/2011456390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4314149800 con 0x7f43140cd3d0 2024-04-25T10:09:27.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.505+0000 7f431c145640 1 -- 172.21.15.60:0/2011456390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4314074770 con 0x7f43140cd3d0 2024-04-25T10:09:27.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.505+0000 7f430a7fc640 1 -- 172.21.15.60:0/2011456390 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43100c4460 con 0x7f43140cd3d0 2024-04-25T10:09:27.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.505+0000 7f430a7fc640 1 --2- 172.21.15.60:0/2011456390 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f42f8066f60 0x7f42f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.505+0000 7f430a7fc640 1 -- 172.21.15.60:0/2011456390 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4310122210 con 0x7f43140cd3d0 2024-04-25T10:09:27.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.505+0000 7f43196e0640 1 --2- 172.21.15.60:0/2011456390 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f42f8066f60 0x7f42f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.505+0000 7f43196e0640 1 --2- 172.21.15.60:0/2011456390 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f42f8066f60 0x7f42f8069420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f43140b40a0 tx=0x7f4304009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.509+0000 7f430a7fc640 1 -- 172.21.15.60:0/2011456390 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43100ae030 con 0x7f43140cd3d0 2024-04-25T10:09:27.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.665+0000 7f431c145640 1 -- 172.21.15.60:0/2011456390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43140af320 con 0x7f43140cd3d0 2024-04-25T10:09:27.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.665+0000 7f430a7fc640 1 -- 172.21.15.60:0/2011456390 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43100182a0 con 0x7f43140cd3d0 2024-04-25T10:09:27.676 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:09:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.669+0000 7f431c145640 1 -- 172.21.15.60:0/2011456390 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f42f8066f60 msgr2=0x7f42f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.669+0000 7f431c145640 1 --2- 172.21.15.60:0/2011456390 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f42f8066f60 0x7f42f8069420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f43140b40a0 tx=0x7f4304009040 comp rx=0 tx=0).stop 2024-04-25T10:09:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.669+0000 7f431c145640 1 -- 172.21.15.60:0/2011456390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43140cd3d0 msgr2=0x7f4314148c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.669+0000 7f431c145640 1 --2- 172.21.15.60:0/2011456390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43140cd3d0 0x7f4314148c60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f431001cca0 tx=0x7f431001ce30 comp rx=0 tx=0).stop 2024-04-25T10:09:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.669+0000 7f431c145640 1 -- 172.21.15.60:0/2011456390 shutdown_connections 2024-04-25T10:09:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.669+0000 7f431c145640 1 --2- 172.21.15.60:0/2011456390 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f42f8066f60 0x7f42f8069420 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.669+0000 7f431c145640 1 --2- 172.21.15.60:0/2011456390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f431415dd30 0x7f4314146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.669+0000 7f431c145640 1 --2- 172.21.15.60:0/2011456390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43140cf8a0 0x7f4314155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.669+0000 7f431c145640 1 --2- 172.21.15.60:0/2011456390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43140cd3d0 0x7f4314148c60 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.669+0000 7f431c145640 1 -- 172.21.15.60:0/2011456390 >> 172.21.15.60:0/2011456390 conn(0x7f43140670b0 msgr2=0x7f431415b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.669+0000 7f431c145640 1 -- 172.21.15.60:0/2011456390 shutdown_connections 2024-04-25T10:09:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.669+0000 7f431c145640 1 -- 172.21.15.60:0/2011456390 wait complete. 2024-04-25T10:09:27.695 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:09:27.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f9871610640 1 Processor -- start 2024-04-25T10:09:27.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f9871610640 1 -- start start 2024-04-25T10:09:27.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f9871610640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f986c15fbf0 0x7f986c166870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f9871610640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986c144a40 con 0x7f986c160080 2024-04-25T10:09:27.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f9871610640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986c0cc9f0 con 0x7f986c0cbcf0 2024-04-25T10:09:27.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f9871610640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f986c0b2dc0 con 0x7f986c15fbf0 2024-04-25T10:09:27.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f98627fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f986c160080 0x7f986c15f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40478/0 (socket says 172.21.15.60:40478) 2024-04-25T10:09:27.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f98627fc640 1 -- 172.21.15.60:0/1764630832 learned_addr learned my addr 172.21.15.60:0/1764630832 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:27.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986b7fe640 1 --2- 172.21.15.60:0/1764630832 >> v2:172.21.15.100:3300/0 conn(0x7f986c15fbf0 0x7f986c166870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986a7fc640 1 -- 172.21.15.60:0/1764630832 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720512996 0 0) 0x7f986c144a40 con 0x7f986c160080 2024-04-25T10:09:27.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986a7fc640 1 -- 172.21.15.60:0/1764630832 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9858003530 con 0x7f986c160080 2024-04-25T10:09:27.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986a7fc640 1 -- 172.21.15.60:0/1764630832 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1947226203 0 0) 0x7f9858003530 con 0x7f986c160080 2024-04-25T10:09:27.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986a7fc640 1 -- 172.21.15.60:0/1764630832 >> v2:172.21.15.100:3300/0 conn(0x7f986c15fbf0 msgr2=0x7f986c166870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986a7fc640 1 --2- 172.21.15.60:0/1764630832 >> v2:172.21.15.100:3300/0 conn(0x7f986c15fbf0 0x7f986c166870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986a7fc640 1 -- 172.21.15.60:0/1764630832 >> v1:172.21.15.100:6789/0 conn(0x7f986c0cbcf0 legacy=0x7f986c071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986a7fc640 1 -- 172.21.15.60:0/1764630832 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c0cccd0 con 0x7f986c160080 2024-04-25T10:09:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986a7fc640 1 -- 172.21.15.60:0/1764630832 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f9854001680 con 0x7f986c160080 2024-04-25T10:09:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986a7fc640 1 -- 172.21.15.60:0/1764630832 >> v1:172.21.15.60:6789/0 conn(0x7f986c160080 legacy=0x7f986c15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986a7fc640 1 --2- 172.21.15.60:0/1764630832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98580039c0 0x7f9858003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986a7fc640 1 --2- 172.21.15.60:0/1764630832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9858004810 0x7f9858004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986a7fc640 1 --2- 172.21.15.60:0/1764630832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98580051d0 0x7f985800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986a7fc640 1 -- 172.21.15.60:0/1764630832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985800c040 con 0x7f98580051d0 2024-04-25T10:09:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986a7fc640 1 -- 172.21.15.60:0/1764630832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985800c1f0 con 0x7f98580039c0 2024-04-25T10:09:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986a7fc640 1 -- 172.21.15.60:0/1764630832 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985800c3f0 con 0x7f9858004810 2024-04-25T10:09:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986b7fe640 1 --2- 172.21.15.60:0/1764630832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98580051d0 0x7f985800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986affd640 1 --2- 172.21.15.60:0/1764630832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98580039c0 0x7f9858003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f98627fc640 1 --2- 172.21.15.60:0/1764630832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9858004810 0x7f9858004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986b7fe640 1 -- 172.21.15.60:0/1764630832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9858004810 msgr2=0x7f9858004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986b7fe640 1 --2- 172.21.15.60:0/1764630832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9858004810 0x7f9858004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986b7fe640 1 -- 172.21.15.60:0/1764630832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98580039c0 msgr2=0x7f9858003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986b7fe640 1 --2- 172.21.15.60:0/1764630832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98580039c0 0x7f9858003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986b7fe640 1 -- 172.21.15.60:0/1764630832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9858003530 con 0x7f98580051d0 2024-04-25T10:09:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.817+0000 7f986affd640 1 --2- 172.21.15.60:0/1764630832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98580039c0 0x7f9858003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:27.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f986b7fe640 1 -- 172.21.15.60:0/1764630832 reap_dead start 2024-04-25T10:09:27.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f986b7fe640 1 --2- 172.21.15.60:0/1764630832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98580051d0 0x7f985800b9d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9864009fc0 tx=0x7f986401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94339d52406f14e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f986a7fc640 1 -- 172.21.15.60:0/1764630832 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864009c10 con 0x7f98580051d0 2024-04-25T10:09:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f986a7fc640 1 -- 172.21.15.60:0/1764630832 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98640a5d10 con 0x7f98580051d0 2024-04-25T10:09:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f986a7fc640 1 -- 172.21.15.60:0/1764630832 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864003be0 con 0x7f98580051d0 2024-04-25T10:09:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f9871610640 1 -- 172.21.15.60:0/1764630832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98580051d0 msgr2=0x7f985800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f9871610640 1 --2- 172.21.15.60:0/1764630832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98580051d0 0x7f985800b9d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9864009fc0 tx=0x7f986401ccc0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f9871610640 1 -- 172.21.15.60:0/1764630832 shutdown_connections 2024-04-25T10:09:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f9871610640 1 --2- 172.21.15.60:0/1764630832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98580051d0 0x7f985800b9d0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f9871610640 1 -- 172.21.15.60:0/1764630832 >> 172.21.15.60:0/1764630832 conn(0x7f986c0670b0 msgr2=0x7f986c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f9871610640 1 -- 172.21.15.60:0/1764630832 shutdown_connections 2024-04-25T10:09:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f9871610640 1 -- 172.21.15.60:0/1764630832 wait complete. 2024-04-25T10:09:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f9871610640 1 Processor -- start 2024-04-25T10:09:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f9871610640 1 -- start start 2024-04-25T10:09:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f9871610640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98580051d0 0x7f986c0accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f9871610640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f986c1f6ed0 0x7f986c133b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f9871610640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f986c1f7330 0x7f986c1340b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f9871610640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c0cc0d0 con 0x7f98580051d0 2024-04-25T10:09:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f9871610640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c0ce9a0 con 0x7f986c1f7330 2024-04-25T10:09:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f9871610640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986c0ce5c0 con 0x7f986c1f6ed0 2024-04-25T10:09:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f986affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98580051d0 0x7f986c0accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f986affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98580051d0 0x7f986c0accd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42760/0 (socket says 172.21.15.60:42760) 2024-04-25T10:09:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f986affd640 1 -- 172.21.15.60:0/1745037340 learned_addr learned my addr 172.21.15.60:0/1745037340 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f986b7fe640 1 --2- 172.21.15.60:0/1745037340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f986c1f7330 0x7f986c1340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f98627fc640 1 --2- 172.21.15.60:0/1745037340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f986c1f6ed0 0x7f986c133b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f986affd640 1 -- 172.21.15.60:0/1745037340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f986c1f6ed0 msgr2=0x7f986c133b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f986affd640 1 --2- 172.21.15.60:0/1745037340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f986c1f6ed0 0x7f986c133b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f986affd640 1 -- 172.21.15.60:0/1745037340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f986c1f7330 msgr2=0x7f986c1340b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f986affd640 1 --2- 172.21.15.60:0/1745037340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f986c1f7330 0x7f986c1340b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:27.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.821+0000 7f986affd640 1 -- 172.21.15.60:0/1745037340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c0ad210 con 0x7f98580051d0 2024-04-25T10:09:27.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.825+0000 7f986affd640 1 --2- 172.21.15.60:0/1745037340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98580051d0 0x7f986c0accd0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f985c00e4e0 tx=0x7f985c00e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.825+0000 7f9863fff640 1 -- 172.21.15.60:0/1745037340 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c0052e0 con 0x7f98580051d0 2024-04-25T10:09:27.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.825+0000 7f9863fff640 1 -- 172.21.15.60:0/1745037340 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985c018070 con 0x7f98580051d0 2024-04-25T10:09:27.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.825+0000 7f9871610640 1 -- 172.21.15.60:0/1745037340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986c0ad490 con 0x7f98580051d0 2024-04-25T10:09:27.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.825+0000 7f9863fff640 1 -- 172.21.15.60:0/1745037340 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c005480 con 0x7f98580051d0 2024-04-25T10:09:27.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.825+0000 7f9871610640 1 -- 172.21.15.60:0/1745037340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986c0ada50 con 0x7f98580051d0 2024-04-25T10:09:27.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.825+0000 7f9871610640 1 -- 172.21.15.60:0/1745037340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986c074770 con 0x7f98580051d0 2024-04-25T10:09:27.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.825+0000 7f9863fff640 1 -- 172.21.15.60:0/1745037340 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f985c003820 con 0x7f98580051d0 2024-04-25T10:09:27.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.825+0000 7f9863fff640 1 --2- 172.21.15.60:0/1745037340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f984c066f90 0x7f984c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:27.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.825+0000 7f9863fff640 1 -- 172.21.15.60:0/1745037340 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f985c084980 con 0x7f98580051d0 2024-04-25T10:09:27.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.825+0000 7f98627fc640 1 --2- 172.21.15.60:0/1745037340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f984c066f90 0x7f984c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:27.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.825+0000 7f98627fc640 1 --2- 172.21.15.60:0/1745037340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f984c066f90 0x7f984c069450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9854000c10 tx=0x7f9854001e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:27.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.829+0000 7f9863fff640 1 -- 172.21.15.60:0/1745037340 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f986c0ad210 con 0x7f98580051d0 2024-04-25T10:09:27.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.989+0000 7f9871610640 1 -- 172.21.15.60:0/1745037340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f986c06aef0 con 0x7f98580051d0 2024-04-25T10:09:27.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.989+0000 7f9863fff640 1 -- 172.21.15.60:0/1745037340 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7f985c055990 con 0x7f98580051d0 2024-04-25T10:09:27.997 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:09:27.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.989+0000 7f9871610640 1 -- 172.21.15.60:0/1745037340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f984c066f90 msgr2=0x7f984c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.989+0000 7f9871610640 1 --2- 172.21.15.60:0/1745037340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f984c066f90 0x7f984c069450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9854000c10 tx=0x7f9854001e00 comp rx=0 tx=0).stop 2024-04-25T10:09:27.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.989+0000 7f9871610640 1 -- 172.21.15.60:0/1745037340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98580051d0 msgr2=0x7f986c0accd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:27.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.989+0000 7f9871610640 1 --2- 172.21.15.60:0/1745037340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98580051d0 0x7f986c0accd0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f985c00e4e0 tx=0x7f985c00e9a0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.989+0000 7f9871610640 1 -- 172.21.15.60:0/1745037340 shutdown_connections 2024-04-25T10:09:28.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.989+0000 7f9871610640 1 --2- 172.21.15.60:0/1745037340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f984c066f90 0x7f984c069450 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.989+0000 7f9871610640 1 --2- 172.21.15.60:0/1745037340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f986c1f7330 0x7f986c1340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.989+0000 7f9871610640 1 --2- 172.21.15.60:0/1745037340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f986c1f6ed0 0x7f986c133b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.989+0000 7f9871610640 1 --2- 172.21.15.60:0/1745037340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98580051d0 0x7f986c0accd0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.989+0000 7f9871610640 1 -- 172.21.15.60:0/1745037340 >> 172.21.15.60:0/1745037340 conn(0x7f986c0670b0 msgr2=0x7f986c15d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:28.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.989+0000 7f9871610640 1 -- 172.21.15.60:0/1745037340 shutdown_connections 2024-04-25T10:09:28.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:27.989+0000 7f9871610640 1 -- 172.21.15.60:0/1745037340 wait complete. 2024-04-25T10:09:28.015 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:09:28.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.129+0000 7f8f7b9c4640 1 Processor -- start 2024-04-25T10:09:28.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.129+0000 7f8f7b9c4640 1 -- start start 2024-04-25T10:09:28.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.129+0000 7f8f7b9c4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8f7415a7a0 0x7f8f740cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.129+0000 7f8f7b9c4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8f740b37e0 0x7f8f740b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.129+0000 7f8f7b9c4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f7406f130 con 0x7f8f7406ec30 2024-04-25T10:09:28.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.129+0000 7f8f7b9c4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f740b4190 con 0x7f8f740b37e0 2024-04-25T10:09:28.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.129+0000 7f8f7b9c4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f740cdaa0 con 0x7f8f7415a7a0 2024-04-25T10:09:28.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.129+0000 7f8f79f61640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8f740b37e0 0x7f8f740b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.129+0000 7f8f79f61640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8f740b37e0 0x7f8f740b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42762/0 (socket says 172.21.15.60:42762) 2024-04-25T10:09:28.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.129+0000 7f8f79f61640 1 -- 172.21.15.60:0/3239186254 learned_addr learned my addr 172.21.15.60:0/3239186254 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:28.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.129+0000 7f8f79760640 1 --2- 172.21.15.60:0/3239186254 >> v2:172.21.15.100:3300/0 conn(0x7f8f7415a7a0 0x7f8f740cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.133+0000 7f8f79f61640 1 -- 172.21.15.60:0/3239186254 >> v2:172.21.15.100:3300/0 conn(0x7f8f7415a7a0 msgr2=0x7f8f740cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:28.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.133+0000 7f8f79f61640 1 --2- 172.21.15.60:0/3239186254 >> v2:172.21.15.100:3300/0 conn(0x7f8f7415a7a0 0x7f8f740cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.133+0000 7f8f79f61640 1 -- 172.21.15.60:0/3239186254 >> v1:172.21.15.100:6789/0 conn(0x7f8f7406ec30 legacy=0x7f8f74071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:28.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.133+0000 7f8f79f61640 1 -- 172.21.15.60:0/3239186254 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7406f410 con 0x7f8f740b37e0 2024-04-25T10:09:28.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.133+0000 7f8f79f61640 1 --2- 172.21.15.60:0/3239186254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f740b37e0 0x7f8f740b3c50 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009e50 tx=0x7f8f6c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deabfed66122e266 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:28.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.133+0000 7f8f6bfff640 1 -- 172.21.15.60:0/3239186254 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0a57d0 con 0x7f8f740b37e0 2024-04-25T10:09:28.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.133+0000 7f8f6bfff640 1 -- 172.21.15.60:0/3239186254 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c0a5970 con 0x7f8f740b37e0 2024-04-25T10:09:28.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.133+0000 7f8f6bfff640 1 -- 172.21.15.60:0/3239186254 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0a5d40 con 0x7f8f740b37e0 2024-04-25T10:09:28.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.133+0000 7f8f7b9c4640 1 -- 172.21.15.60:0/3239186254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f740b37e0 msgr2=0x7f8f740b3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:28.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.133+0000 7f8f7b9c4640 1 --2- 172.21.15.60:0/3239186254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f740b37e0 0x7f8f740b3c50 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009e50 tx=0x7f8f6c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.133+0000 7f8f7b9c4640 1 -- 172.21.15.60:0/3239186254 shutdown_connections 2024-04-25T10:09:28.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.133+0000 7f8f7b9c4640 1 --2- 172.21.15.60:0/3239186254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f740b37e0 0x7f8f740b3c50 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.133+0000 7f8f7b9c4640 1 --2- 172.21.15.60:0/3239186254 >> v2:172.21.15.100:3300/0 conn(0x7f8f7415a7a0 0x7f8f740cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.133+0000 7f8f7b9c4640 1 -- 172.21.15.60:0/3239186254 >> 172.21.15.60:0/3239186254 conn(0x7f8f740b12d0 msgr2=0x7f8f740b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:28.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.133+0000 7f8f7b9c4640 1 -- 172.21.15.60:0/3239186254 shutdown_connections 2024-04-25T10:09:28.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.133+0000 7f8f7b9c4640 1 -- 172.21.15.60:0/3239186254 wait complete. 2024-04-25T10:09:28.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f7b9c4640 1 Processor -- start 2024-04-25T10:09:28.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f7b9c4640 1 -- start start 2024-04-25T10:09:28.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f7b9c4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f7406ec30 0x7f8f741596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f7b9c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f740b37e0 0x7f8f7414fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f7b9c4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f7415a7a0 0x7f8f7415cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f7b9c4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7406e8c0 con 0x7f8f7415a7a0 2024-04-25T10:09:28.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f7b9c4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7406e740 con 0x7f8f7406ec30 2024-04-25T10:09:28.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f7b9c4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7406d8b0 con 0x7f8f740b37e0 2024-04-25T10:09:28.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f79f61640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f7415a7a0 0x7f8f7415cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f79f61640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f7415a7a0 0x7f8f7415cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42766/0 (socket says 172.21.15.60:42766) 2024-04-25T10:09:28.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f79f61640 1 -- 172.21.15.60:0/888961842 learned_addr learned my addr 172.21.15.60:0/888961842 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:28.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f78f5f640 1 --2- 172.21.15.60:0/888961842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f740b37e0 0x7f8f7414fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f79760640 1 --2- 172.21.15.60:0/888961842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f7406ec30 0x7f8f741596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f79f61640 1 -- 172.21.15.60:0/888961842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f740b37e0 msgr2=0x7f8f7414fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:28.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f79f61640 1 --2- 172.21.15.60:0/888961842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f740b37e0 0x7f8f7414fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f79f61640 1 -- 172.21.15.60:0/888961842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f7406ec30 msgr2=0x7f8f741596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:28.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f79f61640 1 --2- 172.21.15.60:0/888961842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f7406ec30 0x7f8f741596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f79f61640 1 -- 172.21.15.60:0/888961842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f741583c0 con 0x7f8f7415a7a0 2024-04-25T10:09:28.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f79f61640 1 --2- 172.21.15.60:0/888961842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f7415a7a0 0x7f8f7415cad0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c019600 tx=0x7f8f6c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:28.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f69ffb640 1 -- 172.21.15.60:0/888961842 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0b8670 con 0x7f8f7415a7a0 2024-04-25T10:09:28.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f69ffb640 1 -- 172.21.15.60:0/888961842 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c0be070 con 0x7f8f7415a7a0 2024-04-25T10:09:28.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f7b9c4640 1 -- 172.21.15.60:0/888961842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f74158660 con 0x7f8f7415a7a0 2024-04-25T10:09:28.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f69ffb640 1 -- 172.21.15.60:0/888961842 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0b8a50 con 0x7f8f7415a7a0 2024-04-25T10:09:28.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f7b9c4640 1 -- 172.21.15.60:0/888961842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f74158be0 con 0x7f8f7415a7a0 2024-04-25T10:09:28.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.137+0000 7f8f7b9c4640 1 -- 172.21.15.60:0/888961842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f74074770 con 0x7f8f7415a7a0 2024-04-25T10:09:28.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.141+0000 7f8f69ffb640 1 -- 172.21.15.60:0/888961842 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f6c0b8bf0 con 0x7f8f7415a7a0 2024-04-25T10:09:28.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.141+0000 7f8f69ffb640 1 --2- 172.21.15.60:0/888961842 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8f58066f60 0x7f8f58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.141+0000 7f8f69ffb640 1 -- 172.21.15.60:0/888961842 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f6c11fff0 con 0x7f8f7415a7a0 2024-04-25T10:09:28.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.141+0000 7f8f79760640 1 --2- 172.21.15.60:0/888961842 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8f58066f60 0x7f8f58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.141+0000 7f8f79760640 1 --2- 172.21.15.60:0/888961842 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8f58066f60 0x7f8f58069420 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8f7415da30 tx=0x7f8f70008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:28.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.145+0000 7f8f69ffb640 1 -- 172.21.15.60:0/888961842 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f6c0165e0 con 0x7f8f7415a7a0 2024-04-25T10:09:28.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.313+0000 7f8f7b9c4640 1 -- 172.21.15.60:0/888961842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8f74150110 con 0x7f8f7415a7a0 2024-04-25T10:09:28.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.313+0000 7f8f69ffb640 1 -- 172.21.15.60:0/888961842 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8f6c0f09f0 con 0x7f8f7415a7a0 2024-04-25T10:09:28.323 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:09:28.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.317+0000 7f8f7b9c4640 1 -- 172.21.15.60:0/888961842 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8f58066f60 msgr2=0x7f8f58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:28.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.317+0000 7f8f7b9c4640 1 --2- 172.21.15.60:0/888961842 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8f58066f60 0x7f8f58069420 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8f7415da30 tx=0x7f8f70008040 comp rx=0 tx=0).stop 2024-04-25T10:09:28.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.317+0000 7f8f7b9c4640 1 -- 172.21.15.60:0/888961842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f7415a7a0 msgr2=0x7f8f7415cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:28.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.317+0000 7f8f7b9c4640 1 --2- 172.21.15.60:0/888961842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f7415a7a0 0x7f8f7415cad0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c019600 tx=0x7f8f6c019790 comp rx=0 tx=0).stop 2024-04-25T10:09:28.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.317+0000 7f8f7b9c4640 1 -- 172.21.15.60:0/888961842 shutdown_connections 2024-04-25T10:09:28.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.317+0000 7f8f7b9c4640 1 --2- 172.21.15.60:0/888961842 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8f58066f60 0x7f8f58069420 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.317+0000 7f8f7b9c4640 1 --2- 172.21.15.60:0/888961842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f7415a7a0 0x7f8f7415cad0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.317+0000 7f8f7b9c4640 1 --2- 172.21.15.60:0/888961842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f740b37e0 0x7f8f7414fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.317+0000 7f8f7b9c4640 1 --2- 172.21.15.60:0/888961842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f7406ec30 0x7f8f741596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.317+0000 7f8f7b9c4640 1 -- 172.21.15.60:0/888961842 >> 172.21.15.60:0/888961842 conn(0x7f8f740b12d0 msgr2=0x7f8f74164b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:28.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.317+0000 7f8f7b9c4640 1 -- 172.21.15.60:0/888961842 shutdown_connections 2024-04-25T10:09:28.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.317+0000 7f8f7b9c4640 1 -- 172.21.15.60:0/888961842 wait complete. 2024-04-25T10:09:28.340 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:09:28.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:28.341 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:28.352 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[40405]: fuse finished with error 0 and tester_r 0 2024-04-25T10:09:28.353 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:09:28.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:28.353 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:09:28.424 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:09:28.424 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:28.433 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:09:28.433 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:09:28.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:28.433 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:09:28.481 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:09:28.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:09:28.482 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:09:28.482 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:28.487 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:09:28.488 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:09:28.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:28.488 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:09:28.536 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:09:28.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:09:28.537 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:09:28.537 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:28.543 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:09:28.543 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:09:28.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:28.543 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:09:28.588 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:09:28.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:09:28.589 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:28.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.705+0000 7f57a6866640 1 Processor -- start 2024-04-25T10:09:28.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.705+0000 7f57a6866640 1 -- start start 2024-04-25T10:09:28.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.705+0000 7f57a6866640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f57a015bfa0 0x7f57a00d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.705+0000 7f57a6866640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a00d0ec0 0x7f57a015d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.705+0000 7f57a6866640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a00716c0 con 0x7f57a00cefc0 2024-04-25T10:09:28.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.705+0000 7f57a6866640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a015d4e0 con 0x7f57a015bfa0 2024-04-25T10:09:28.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.705+0000 7f57a6866640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00719a0 con 0x7f57a00d0ec0 2024-04-25T10:09:28.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.705+0000 7f579f7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f57a015bfa0 0x7f57a00d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.705+0000 7f579f7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f57a015bfa0 0x7f57a00d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:40752/0 (socket says 172.21.15.60:40752) 2024-04-25T10:09:28.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.705+0000 7f579f7fe640 1 -- 172.21.15.60:0/3760338777 learned_addr learned my addr 172.21.15.60:0/3760338777 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:28.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.705+0000 7f57a4e03640 1 --2- 172.21.15.60:0/3760338777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a00d0ec0 0x7f57a015d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.705+0000 7f579f7fe640 1 -- 172.21.15.60:0/3760338777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a00d0ec0 msgr2=0x7f57a015d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:28.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.705+0000 7f579f7fe640 1 --2- 172.21.15.60:0/3760338777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a00d0ec0 0x7f57a015d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.705+0000 7f579f7fe640 1 -- 172.21.15.60:0/3760338777 >> v1:172.21.15.100:6789/0 conn(0x7f57a00cefc0 legacy=0x7f57a0070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:28.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.705+0000 7f579f7fe640 1 -- 172.21.15.60:0/3760338777 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a0071c80 con 0x7f57a015bfa0 2024-04-25T10:09:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f579f7fe640 1 --2- 172.21.15.60:0/3760338777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a015bfa0 0x7f57a00d0980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5794009870 tx=0x7f579401c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ab8c4fe6637dc30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f579effd640 1 -- 172.21.15.60:0/3760338777 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57940a9610 con 0x7f57a015bfa0 2024-04-25T10:09:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f579effd640 1 -- 172.21.15.60:0/3760338777 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57940a97b0 con 0x7f57a015bfa0 2024-04-25T10:09:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f579effd640 1 -- 172.21.15.60:0/3760338777 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57940a9b80 con 0x7f57a015bfa0 2024-04-25T10:09:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a6866640 1 -- 172.21.15.60:0/3760338777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a015bfa0 msgr2=0x7f57a00d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a6866640 1 --2- 172.21.15.60:0/3760338777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a015bfa0 0x7f57a00d0980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5794009870 tx=0x7f579401c530 comp rx=0 tx=0).stop 2024-04-25T10:09:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a6866640 1 -- 172.21.15.60:0/3760338777 shutdown_connections 2024-04-25T10:09:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a6866640 1 --2- 172.21.15.60:0/3760338777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a015bfa0 0x7f57a00d0980 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a6866640 1 --2- 172.21.15.60:0/3760338777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a00d0ec0 0x7f57a015d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a6866640 1 -- 172.21.15.60:0/3760338777 >> 172.21.15.60:0/3760338777 conn(0x7f57a00b3520 msgr2=0x7f57a00b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a6866640 1 -- 172.21.15.60:0/3760338777 shutdown_connections 2024-04-25T10:09:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a6866640 1 -- 172.21.15.60:0/3760338777 wait complete. 2024-04-25T10:09:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a6866640 1 Processor -- start 2024-04-25T10:09:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a6866640 1 -- start start 2024-04-25T10:09:28.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a6866640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00cefc0 0x7f57a0145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a6866640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a00d0ec0 0x7f57a0149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a6866640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a015bfa0 0x7f57a01439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a6866640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a015e840 con 0x7f57a00cefc0 2024-04-25T10:09:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a6866640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a015e6c0 con 0x7f57a00d0ec0 2024-04-25T10:09:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a6866640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a015e9c0 con 0x7f57a015bfa0 2024-04-25T10:09:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f579ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00cefc0 0x7f57a0145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f579ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00cefc0 0x7f57a0145460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42782/0 (socket says 172.21.15.60:42782) 2024-04-25T10:09:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f579ffff640 1 -- 172.21.15.60:0/1912862983 learned_addr learned my addr 172.21.15.60:0/1912862983 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f57a4e03640 1 --2- 172.21.15.60:0/1912862983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a015bfa0 0x7f57a01439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.709+0000 7f579f7fe640 1 --2- 172.21.15.60:0/1912862983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a00d0ec0 0x7f57a0149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f579ffff640 1 -- 172.21.15.60:0/1912862983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a015bfa0 msgr2=0x7f57a01439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f579ffff640 1 --2- 172.21.15.60:0/1912862983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a015bfa0 0x7f57a01439e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f579ffff640 1 -- 172.21.15.60:0/1912862983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a00d0ec0 msgr2=0x7f57a0149260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f579ffff640 1 --2- 172.21.15.60:0/1912862983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a00d0ec0 0x7f57a0149260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f579ffff640 1 -- 172.21.15.60:0/1912862983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a0145bb0 con 0x7f57a00cefc0 2024-04-25T10:09:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f579ffff640 1 --2- 172.21.15.60:0/1912862983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00cefc0 0x7f57a0145460 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f579000c3f0 tx=0x7f579000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f579cff9640 1 -- 172.21.15.60:0/1912862983 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790017070 con 0x7f57a00cefc0 2024-04-25T10:09:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f579cff9640 1 -- 172.21.15.60:0/1912862983 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57900052d0 con 0x7f57a00cefc0 2024-04-25T10:09:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f57a6866640 1 -- 172.21.15.60:0/1912862983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a0145e90 con 0x7f57a00cefc0 2024-04-25T10:09:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f579cff9640 1 -- 172.21.15.60:0/1912862983 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790003840 con 0x7f57a00cefc0 2024-04-25T10:09:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f57a6866640 1 -- 172.21.15.60:0/1912862983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a006bdc0 con 0x7f57a00cefc0 2024-04-25T10:09:28.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f57a6866640 1 -- 172.21.15.60:0/1912862983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a0074770 con 0x7f57a00cefc0 2024-04-25T10:09:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f579cff9640 1 -- 172.21.15.60:0/1912862983 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f579001a070 con 0x7f57a00cefc0 2024-04-25T10:09:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f579cff9640 1 --2- 172.21.15.60:0/1912862983 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f578c066f60 0x7f578c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f579cff9640 1 -- 172.21.15.60:0/1912862983 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5790084330 con 0x7f57a00cefc0 2024-04-25T10:09:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f579f7fe640 1 --2- 172.21.15.60:0/1912862983 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f578c066f60 0x7f578c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.713+0000 7f579f7fe640 1 --2- 172.21.15.60:0/1912862983 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f578c066f60 0x7f578c069420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f57940ae000 tx=0x7f57940b4040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.717+0000 7f579cff9640 1 -- 172.21.15.60:0/1912862983 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5790005480 con 0x7f57a00cefc0 2024-04-25T10:09:28.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.905+0000 7f57a6866640 1 -- 172.21.15.60:0/1912862983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57a014c890 con 0x7f57a00cefc0 2024-04-25T10:09:28.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.909+0000 7f579cff9640 1 -- 172.21.15.60:0/1912862983 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f5790054c50 con 0x7f57a00cefc0 2024-04-25T10:09:28.916 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:28.917 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/380273391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":380273391},{"type":"v1","addr":"172.21.15.100:6836","nonce":380273391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5635,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2965977166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2965977166},{"type":"v1","addr":"172.21.15.100:6837","nonce":2965977166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1105028629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1105028629},{"type":"v1","addr":"172.21.15.60:6834","nonce":1105028629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:10.236590+0000","modified":"2024-04-25T10:08:58.929165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.60:6835/2718963870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2718963870},{"type":"v1","addr":"172.21.15.60:6835","nonce":2718963870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:09:28.917 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 30 2024-04-25T10:09:28.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.909+0000 7f57a6866640 1 -- 172.21.15.60:0/1912862983 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f578c066f60 msgr2=0x7f578c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.909+0000 7f57a6866640 1 --2- 172.21.15.60:0/1912862983 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f578c066f60 0x7f578c069420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f57940ae000 tx=0x7f57940b4040 comp rx=0 tx=0).stop 2024-04-25T10:09:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.909+0000 7f57a6866640 1 -- 172.21.15.60:0/1912862983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00cefc0 msgr2=0x7f57a0145460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.909+0000 7f57a6866640 1 --2- 172.21.15.60:0/1912862983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00cefc0 0x7f57a0145460 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f579000c3f0 tx=0x7f579000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.909+0000 7f57a6866640 1 -- 172.21.15.60:0/1912862983 shutdown_connections 2024-04-25T10:09:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.909+0000 7f57a6866640 1 --2- 172.21.15.60:0/1912862983 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f578c066f60 0x7f578c069420 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.909+0000 7f57a6866640 1 --2- 172.21.15.60:0/1912862983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a015bfa0 0x7f57a01439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.909+0000 7f57a6866640 1 --2- 172.21.15.60:0/1912862983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a00d0ec0 0x7f57a0149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.909+0000 7f57a6866640 1 --2- 172.21.15.60:0/1912862983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a00cefc0 0x7f57a0145460 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.909+0000 7f57a6866640 1 -- 172.21.15.60:0/1912862983 >> 172.21.15.60:0/1912862983 conn(0x7f57a00b3520 msgr2=0x7f57a015c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.909+0000 7f57a6866640 1 -- 172.21.15.60:0/1912862983 shutdown_connections 2024-04-25T10:09:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:28.909+0000 7f57a6866640 1 -- 172.21.15.60:0/1912862983 wait complete. 2024-04-25T10:09:28.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:29.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b295ed640 1 Processor -- start 2024-04-25T10:09:29.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b295ed640 1 -- start start 2024-04-25T10:09:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b295ed640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9b2415bc90 0x7f9b2415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b295ed640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b24071ca0 con 0x7f9b240ce7a0 2024-04-25T10:09:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b295ed640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b240cca40 con 0x7f9b240b4660 2024-04-25T10:09:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b295ed640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b240cc370 con 0x7f9b2415bc90 2024-04-25T10:09:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b22ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9b240ce7a0 0x7f9b240ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44530/0 (socket says 172.21.15.60:44530) 2024-04-25T10:09:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b22ffd640 1 -- 172.21.15.60:0/1049812457 learned_addr learned my addr 172.21.15.60:0/1049812457 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b227fc640 1 --2- 172.21.15.60:0/1049812457 >> v2:172.21.15.100:3300/0 conn(0x7f9b2415bc90 0x7f9b2415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b21ffb640 1 -- 172.21.15.60:0/1049812457 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685129577 0 0) 0x7f9b24071ca0 con 0x7f9b240ce7a0 2024-04-25T10:09:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b21ffb640 1 -- 172.21.15.60:0/1049812457 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b0c003530 con 0x7f9b240ce7a0 2024-04-25T10:09:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b21ffb640 1 -- 172.21.15.60:0/1049812457 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 724188863 0 0) 0x7f9b0c003530 con 0x7f9b240ce7a0 2024-04-25T10:09:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b21ffb640 1 -- 172.21.15.60:0/1049812457 >> v2:172.21.15.100:3300/0 conn(0x7f9b2415bc90 msgr2=0x7f9b2415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b21ffb640 1 --2- 172.21.15.60:0/1049812457 >> v2:172.21.15.100:3300/0 conn(0x7f9b2415bc90 0x7f9b2415c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b21ffb640 1 -- 172.21.15.60:0/1049812457 >> v1:172.21.15.100:6789/0 conn(0x7f9b240b4660 legacy=0x7f9b2415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b21ffb640 1 -- 172.21.15.60:0/1049812457 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b240ccd20 con 0x7f9b240ce7a0 2024-04-25T10:09:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b21ffb640 1 -- 172.21.15.60:0/1049812457 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f9b18001680 con 0x7f9b240ce7a0 2024-04-25T10:09:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b21ffb640 1 -- 172.21.15.60:0/1049812457 >> v1:172.21.15.60:6789/0 conn(0x7f9b240ce7a0 legacy=0x7f9b240ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b21ffb640 1 --2- 172.21.15.60:0/1049812457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b0c0039c0 0x7f9b0c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b21ffb640 1 --2- 172.21.15.60:0/1049812457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b0c004810 0x7f9b0c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b21ffb640 1 --2- 172.21.15.60:0/1049812457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b0c0051d0 0x7f9b0c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b21ffb640 1 -- 172.21.15.60:0/1049812457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c00c040 con 0x7f9b0c0051d0 2024-04-25T10:09:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b21ffb640 1 -- 172.21.15.60:0/1049812457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c00c1f0 con 0x7f9b0c0039c0 2024-04-25T10:09:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b21ffb640 1 -- 172.21.15.60:0/1049812457 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c00c3f0 con 0x7f9b0c004810 2024-04-25T10:09:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b237fe640 1 --2- 172.21.15.60:0/1049812457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b0c0051d0 0x7f9b0c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b22ffd640 1 --2- 172.21.15.60:0/1049812457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b0c0039c0 0x7f9b0c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b227fc640 1 --2- 172.21.15.60:0/1049812457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b0c004810 0x7f9b0c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b237fe640 1 -- 172.21.15.60:0/1049812457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b0c004810 msgr2=0x7f9b0c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b237fe640 1 --2- 172.21.15.60:0/1049812457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b0c004810 0x7f9b0c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b237fe640 1 -- 172.21.15.60:0/1049812457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b0c0039c0 msgr2=0x7f9b0c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b237fe640 1 --2- 172.21.15.60:0/1049812457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b0c0039c0 0x7f9b0c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b237fe640 1 -- 172.21.15.60:0/1049812457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c003530 con 0x7f9b0c0051d0 2024-04-25T10:09:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.049+0000 7f9b22ffd640 1 --2- 172.21.15.60:0/1049812457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b0c0039c0 0x7f9b0c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:29.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b237fe640 1 -- 172.21.15.60:0/1049812457 reap_dead start 2024-04-25T10:09:29.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b237fe640 1 --2- 172.21.15.60:0/1049812457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b0c0051d0 0x7f9b0c00b9d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c001e40 tx=0x7f9b1c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8672b8f411d0a7fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b21ffb640 1 -- 172.21.15.60:0/1049812457 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c001a90 con 0x7f9b0c0051d0 2024-04-25T10:09:29.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b21ffb640 1 -- 172.21.15.60:0/1049812457 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c003b80 con 0x7f9b0c0051d0 2024-04-25T10:09:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b21ffb640 1 -- 172.21.15.60:0/1049812457 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c005020 con 0x7f9b0c0051d0 2024-04-25T10:09:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b295ed640 1 -- 172.21.15.60:0/1049812457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b0c0051d0 msgr2=0x7f9b0c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b295ed640 1 --2- 172.21.15.60:0/1049812457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b0c0051d0 0x7f9b0c00b9d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c001e40 tx=0x7f9b1c01c760 comp rx=0 tx=0).stop 2024-04-25T10:09:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b295ed640 1 -- 172.21.15.60:0/1049812457 shutdown_connections 2024-04-25T10:09:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b295ed640 1 --2- 172.21.15.60:0/1049812457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b0c0051d0 0x7f9b0c00b9d0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b295ed640 1 -- 172.21.15.60:0/1049812457 >> 172.21.15.60:0/1049812457 conn(0x7f9b240b12d0 msgr2=0x7f9b240b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b295ed640 1 -- 172.21.15.60:0/1049812457 shutdown_connections 2024-04-25T10:09:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b295ed640 1 -- 172.21.15.60:0/1049812457 wait complete. 2024-04-25T10:09:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b295ed640 1 Processor -- start 2024-04-25T10:09:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b295ed640 1 -- start start 2024-04-25T10:09:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b295ed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b0c0051d0 0x7f9b24150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b295ed640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b241f4bc0 0x7f9b241eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b295ed640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b241f1d00 0x7f9b241f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b295ed640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b240b4400 con 0x7f9b0c0051d0 2024-04-25T10:09:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b295ed640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b240b4280 con 0x7f9b241f4bc0 2024-04-25T10:09:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.053+0000 7f9b295ed640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b240b3ba0 con 0x7f9b241f1d00 2024-04-25T10:09:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b22ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b0c0051d0 0x7f9b24150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b22ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b0c0051d0 0x7f9b24150610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37520/0 (socket says 172.21.15.60:37520) 2024-04-25T10:09:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b22ffd640 1 -- 172.21.15.60:0/3278364022 learned_addr learned my addr 172.21.15.60:0/3278364022 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b227fc640 1 --2- 172.21.15.60:0/3278364022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b241f4bc0 0x7f9b241eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b22ffd640 1 -- 172.21.15.60:0/3278364022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b241f1d00 msgr2=0x7f9b241f2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b237fe640 1 --2- 172.21.15.60:0/3278364022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b241f1d00 0x7f9b241f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b22ffd640 1 --2- 172.21.15.60:0/3278364022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b241f1d00 0x7f9b241f2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b22ffd640 1 -- 172.21.15.60:0/3278364022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b241f4bc0 msgr2=0x7f9b241eeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b22ffd640 1 --2- 172.21.15.60:0/3278364022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b241f4bc0 0x7f9b241eeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b22ffd640 1 -- 172.21.15.60:0/3278364022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b241f5040 con 0x7f9b0c0051d0 2024-04-25T10:09:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b227fc640 1 --2- 172.21.15.60:0/3278364022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b241f4bc0 0x7f9b241eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b22ffd640 1 --2- 172.21.15.60:0/3278364022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b0c0051d0 0x7f9b24150610 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9b18000fc0 tx=0x7f9b1800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b0b7fe640 1 -- 172.21.15.60:0/3278364022 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1800ee00 con 0x7f9b0c0051d0 2024-04-25T10:09:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b0b7fe640 1 -- 172.21.15.60:0/3278364022 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b18018070 con 0x7f9b0c0051d0 2024-04-25T10:09:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b295ed640 1 -- 172.21.15.60:0/3278364022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b241f52c0 con 0x7f9b0c0051d0 2024-04-25T10:09:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b0b7fe640 1 -- 172.21.15.60:0/3278364022 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1800ca40 con 0x7f9b0c0051d0 2024-04-25T10:09:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b295ed640 1 -- 172.21.15.60:0/3278364022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b241f5880 con 0x7f9b0c0051d0 2024-04-25T10:09:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b295ed640 1 -- 172.21.15.60:0/3278364022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b24074770 con 0x7f9b0c0051d0 2024-04-25T10:09:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b0b7fe640 1 -- 172.21.15.60:0/3278364022 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b1800cbe0 con 0x7f9b0c0051d0 2024-04-25T10:09:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b0b7fe640 1 --2- 172.21.15.60:0/3278364022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9b10066f90 0x7f9b10069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b0b7fe640 1 -- 172.21.15.60:0/3278364022 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b18084ce0 con 0x7f9b0c0051d0 2024-04-25T10:09:29.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b227fc640 1 --2- 172.21.15.60:0/3278364022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9b10066f90 0x7f9b10069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.057+0000 7f9b227fc640 1 --2- 172.21.15.60:0/3278364022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9b10066f90 0x7f9b10069450 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9b2414fcb0 tx=0x7f9b14008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.061+0000 7f9b0b7fe640 1 -- 172.21.15.60:0/3278364022 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b241f5040 con 0x7f9b0c0051d0 2024-04-25T10:09:29.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.221+0000 7f9b295ed640 1 -- 172.21.15.60:0/3278364022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b2415ea10 con 0x7f9b0c0051d0 2024-04-25T10:09:29.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.225+0000 7f9b0b7fe640 1 -- 172.21.15.60:0/3278364022 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16677 (secure 0 0 0) 0x7f9b18055d80 con 0x7f9b0c0051d0 2024-04-25T10:09:29.234 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:29.234 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":58,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:08:11.878055+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:08.538247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:09.233842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:29.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.225+0000 7f9b295ed640 1 -- 172.21.15.60:0/3278364022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9b10066f90 msgr2=0x7f9b10069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.225+0000 7f9b295ed640 1 --2- 172.21.15.60:0/3278364022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9b10066f90 0x7f9b10069450 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9b2414fcb0 tx=0x7f9b14008040 comp rx=0 tx=0).stop 2024-04-25T10:09:29.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.225+0000 7f9b295ed640 1 -- 172.21.15.60:0/3278364022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b0c0051d0 msgr2=0x7f9b24150610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.225+0000 7f9b295ed640 1 --2- 172.21.15.60:0/3278364022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b0c0051d0 0x7f9b24150610 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9b18000fc0 tx=0x7f9b1800ec10 comp rx=0 tx=0).stop 2024-04-25T10:09:29.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.229+0000 7f9b295ed640 1 -- 172.21.15.60:0/3278364022 shutdown_connections 2024-04-25T10:09:29.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.229+0000 7f9b295ed640 1 --2- 172.21.15.60:0/3278364022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9b10066f90 0x7f9b10069450 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.229+0000 7f9b295ed640 1 --2- 172.21.15.60:0/3278364022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b241f1d00 0x7f9b241f2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.229+0000 7f9b295ed640 1 --2- 172.21.15.60:0/3278364022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b241f4bc0 0x7f9b241eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.229+0000 7f9b295ed640 1 --2- 172.21.15.60:0/3278364022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b0c0051d0 0x7f9b24150610 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.229+0000 7f9b295ed640 1 -- 172.21.15.60:0/3278364022 >> 172.21.15.60:0/3278364022 conn(0x7f9b240b12d0 msgr2=0x7f9b240ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:29.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.229+0000 7f9b295ed640 1 -- 172.21.15.60:0/3278364022 shutdown_connections 2024-04-25T10:09:29.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.229+0000 7f9b295ed640 1 -- 172.21.15.60:0/3278364022 wait complete. 2024-04-25T10:09:29.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:08.538247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:09.233842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:29.254 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:29.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.373+0000 7fda05464640 1 Processor -- start 2024-04-25T10:09:29.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.373+0000 7fda05464640 1 -- start start 2024-04-25T10:09:29.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fda05464640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda000cbcf0 0x7fda00160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fda05464640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda00144a40 con 0x7fda0015fbf0 2024-04-25T10:09:29.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fda05464640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda000cc9f0 con 0x7fda00071c30 2024-04-25T10:09:29.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fda05464640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda000b2dc0 con 0x7fda000cbcf0 2024-04-25T10:09:29.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9ff7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fda0015fbf0 0x7fda000ceec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44538/0 (socket says 172.21.15.60:44538) 2024-04-25T10:09:29.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9ff7fe640 1 -- 172.21.15.60:0/531588382 learned_addr learned my addr 172.21.15.60:0/531588382 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:29.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9feffd640 1 --2- 172.21.15.60:0/531588382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda000cbcf0 0x7fda00160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fdffb640 1 -- 172.21.15.60:0/531588382 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525786368 0 0) 0x7fda00144a40 con 0x7fda0015fbf0 2024-04-25T10:09:29.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fdffb640 1 -- 172.21.15.60:0/531588382 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9e8003530 con 0x7fda0015fbf0 2024-04-25T10:09:29.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fdffb640 1 -- 172.21.15.60:0/531588382 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2887603800 0 0) 0x7fd9e8003530 con 0x7fda0015fbf0 2024-04-25T10:09:29.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fdffb640 1 -- 172.21.15.60:0/531588382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda000cbcf0 msgr2=0x7fda00160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:29.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fdffb640 1 --2- 172.21.15.60:0/531588382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda000cbcf0 0x7fda00160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fdffb640 1 -- 172.21.15.60:0/531588382 >> v1:172.21.15.100:6789/0 conn(0x7fda00071c30 legacy=0x7fda0015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fdffb640 1 -- 172.21.15.60:0/531588382 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda000cccd0 con 0x7fda0015fbf0 2024-04-25T10:09:29.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fdffb640 1 -- 172.21.15.60:0/531588382 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fd9f8001a00 con 0x7fda0015fbf0 2024-04-25T10:09:29.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fdffb640 1 -- 172.21.15.60:0/531588382 >> v1:172.21.15.60:6789/0 conn(0x7fda0015fbf0 legacy=0x7fda000ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fdffb640 1 --2- 172.21.15.60:0/531588382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda000cbcf0 0x7fd9e80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fdffb640 1 --2- 172.21.15.60:0/531588382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9e8003f70 0x7fd9e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fdffb640 1 --2- 172.21.15.60:0/531588382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9e8004d50 0x7fd9e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fdffb640 1 -- 172.21.15.60:0/531588382 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e800ba80 con 0x7fd9e8004d50 2024-04-25T10:09:29.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fdffb640 1 -- 172.21.15.60:0/531588382 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e800bc30 con 0x7fd9e8003f70 2024-04-25T10:09:29.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fdffb640 1 -- 172.21.15.60:0/531588382 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e800be30 con 0x7fda000cbcf0 2024-04-25T10:09:29.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9ff7fe640 1 --2- 172.21.15.60:0/531588382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9e8004d50 0x7fd9e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fe7fc640 1 --2- 172.21.15.60:0/531588382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9e8003f70 0x7fd9e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9feffd640 1 --2- 172.21.15.60:0/531588382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda000cbcf0 0x7fd9e80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9ff7fe640 1 -- 172.21.15.60:0/531588382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda000cbcf0 msgr2=0x7fd9e80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:29.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9ff7fe640 1 --2- 172.21.15.60:0/531588382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda000cbcf0 0x7fd9e80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9ff7fe640 1 -- 172.21.15.60:0/531588382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9e8003f70 msgr2=0x7fd9e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:29.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9ff7fe640 1 --2- 172.21.15.60:0/531588382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9e8003f70 0x7fd9e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9ff7fe640 1 -- 172.21.15.60:0/531588382 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e8003530 con 0x7fd9e8004d50 2024-04-25T10:09:29.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fe7fc640 1 --2- 172.21.15.60:0/531588382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9e8003f70 0x7fd9e8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:29.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9feffd640 1 --2- 172.21.15.60:0/531588382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda000cbcf0 0x7fd9e80039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:29.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9ff7fe640 1 --2- 172.21.15.60:0/531588382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9e8004d50 0x7fd9e80051d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800a860 tx=0x7fd9f801d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74925eb098f93d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.377+0000 7fd9fdffb640 1 -- 172.21.15.60:0/531588382 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f800a480 con 0x7fd9e8004d50 2024-04-25T10:09:29.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9fdffb640 1 -- 172.21.15.60:0/531588382 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f80ad630 con 0x7fd9e8004d50 2024-04-25T10:09:29.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9fdffb640 1 -- 172.21.15.60:0/531588382 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80adaa0 con 0x7fd9e8004d50 2024-04-25T10:09:29.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fda05464640 1 -- 172.21.15.60:0/531588382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9e8004d50 msgr2=0x7fd9e80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fda05464640 1 --2- 172.21.15.60:0/531588382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9e8004d50 0x7fd9e80051d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800a860 tx=0x7fd9f801d380 comp rx=0 tx=0).stop 2024-04-25T10:09:29.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9ff7fe640 1 -- 172.21.15.60:0/531588382 reap_dead start 2024-04-25T10:09:29.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fda05464640 1 -- 172.21.15.60:0/531588382 shutdown_connections 2024-04-25T10:09:29.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fda05464640 1 -- 172.21.15.60:0/531588382 >> 172.21.15.60:0/531588382 conn(0x7fda000670b0 msgr2=0x7fda0006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:29.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fda05464640 1 -- 172.21.15.60:0/531588382 shutdown_connections 2024-04-25T10:09:29.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fda05464640 1 -- 172.21.15.60:0/531588382 wait complete. 2024-04-25T10:09:29.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fda05464640 1 Processor -- start 2024-04-25T10:09:29.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fda05464640 1 -- start start 2024-04-25T10:09:29.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fda05464640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9e8004d50 0x7fda0013e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fda05464640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda0015b050 0x7fda0013be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fda05464640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0015b4d0 0x7fda0013c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fda05464640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda000cc0d0 con 0x7fda0015b050 2024-04-25T10:09:29.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fda05464640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda000ce9a0 con 0x7fd9e8004d50 2024-04-25T10:09:29.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fda05464640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda000720b0 con 0x7fda0015b4d0 2024-04-25T10:09:29.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9fe7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda0015b050 0x7fda0013be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9fe7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda0015b050 0x7fda0013be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37542/0 (socket says 172.21.15.60:37542) 2024-04-25T10:09:29.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9fe7fc640 1 -- 172.21.15.60:0/3937147600 learned_addr learned my addr 172.21.15.60:0/3937147600 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:29.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9feffd640 1 --2- 172.21.15.60:0/3937147600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9e8004d50 0x7fda0013e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9ff7fe640 1 --2- 172.21.15.60:0/3937147600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0015b4d0 0x7fda0013c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9fe7fc640 1 -- 172.21.15.60:0/3937147600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0015b4d0 msgr2=0x7fda0013c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9fe7fc640 1 --2- 172.21.15.60:0/3937147600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0015b4d0 0x7fda0013c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9fe7fc640 1 -- 172.21.15.60:0/3937147600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9e8004d50 msgr2=0x7fda0013e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9fe7fc640 1 --2- 172.21.15.60:0/3937147600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9e8004d50 0x7fda0013e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9fe7fc640 1 -- 172.21.15.60:0/3937147600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0013b3a0 con 0x7fda0015b050 2024-04-25T10:09:29.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9fe7fc640 1 --2- 172.21.15.60:0/3937147600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda0015b050 0x7fda0013be30 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400d770 tx=0x7fd9f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9e77fe640 1 -- 172.21.15.60:0/3937147600 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4005070 con 0x7fda0015b050 2024-04-25T10:09:29.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9e77fe640 1 -- 172.21.15.60:0/3937147600 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f4005230 con 0x7fda0015b050 2024-04-25T10:09:29.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fda05464640 1 -- 172.21.15.60:0/3937147600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda000af2a0 con 0x7fda0015b050 2024-04-25T10:09:29.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fd9e77fe640 1 -- 172.21.15.60:0/3937147600 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4007030 con 0x7fda0015b050 2024-04-25T10:09:29.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.381+0000 7fda05464640 1 -- 172.21.15.60:0/3937147600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda000af860 con 0x7fda0015b050 2024-04-25T10:09:29.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.385+0000 7fda05464640 1 -- 172.21.15.60:0/3937147600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda00074770 con 0x7fda0015b050 2024-04-25T10:09:29.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.385+0000 7fd9e77fe640 1 -- 172.21.15.60:0/3937147600 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9f401a020 con 0x7fda0015b050 2024-04-25T10:09:29.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.385+0000 7fd9e77fe640 1 --2- 172.21.15.60:0/3937147600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd9ec066f90 0x7fd9ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.385+0000 7fd9e77fe640 1 -- 172.21.15.60:0/3937147600 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9f4084870 con 0x7fda0015b050 2024-04-25T10:09:29.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.385+0000 7fd9feffd640 1 --2- 172.21.15.60:0/3937147600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd9ec066f90 0x7fd9ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.385+0000 7fd9feffd640 1 --2- 172.21.15.60:0/3937147600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd9ec066f90 0x7fd9ec069450 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fda000b0ee0 tx=0x7fd9f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.389+0000 7fd9e77fe640 1 -- 172.21.15.60:0/3937147600 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9f4055880 con 0x7fda0015b050 2024-04-25T10:09:29.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.549+0000 7fda05464640 1 -- 172.21.15.60:0/3937147600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda000717a0 con 0x7fda0015b050 2024-04-25T10:09:29.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.549+0000 7fd9e77fe640 1 -- 172.21.15.60:0/3937147600 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9f4055180 con 0x7fda0015b050 2024-04-25T10:09:29.558 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:09:29.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.553+0000 7fda05464640 1 -- 172.21.15.60:0/3937147600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd9ec066f90 msgr2=0x7fd9ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.553+0000 7fda05464640 1 --2- 172.21.15.60:0/3937147600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd9ec066f90 0x7fd9ec069450 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fda000b0ee0 tx=0x7fd9f0008040 comp rx=0 tx=0).stop 2024-04-25T10:09:29.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.553+0000 7fda05464640 1 -- 172.21.15.60:0/3937147600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda0015b050 msgr2=0x7fda0013be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.553+0000 7fda05464640 1 --2- 172.21.15.60:0/3937147600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda0015b050 0x7fda0013be30 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400d770 tx=0x7fd9f400dc30 comp rx=0 tx=0).stop 2024-04-25T10:09:29.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.553+0000 7fda05464640 1 -- 172.21.15.60:0/3937147600 shutdown_connections 2024-04-25T10:09:29.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.553+0000 7fda05464640 1 --2- 172.21.15.60:0/3937147600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd9ec066f90 0x7fd9ec069450 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.553+0000 7fda05464640 1 --2- 172.21.15.60:0/3937147600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda0015b4d0 0x7fda0013c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.553+0000 7fda05464640 1 --2- 172.21.15.60:0/3937147600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda0015b050 0x7fda0013be30 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.553+0000 7fda05464640 1 --2- 172.21.15.60:0/3937147600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9e8004d50 0x7fda0013e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.553+0000 7fda05464640 1 -- 172.21.15.60:0/3937147600 >> 172.21.15.60:0/3937147600 conn(0x7fda000670b0 msgr2=0x7fda000cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:29.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.553+0000 7fda05464640 1 -- 172.21.15.60:0/3937147600 shutdown_connections 2024-04-25T10:09:29.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.553+0000 7fda05464640 1 -- 172.21.15.60:0/3937147600 wait complete. 2024-04-25T10:09:29.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:09:29.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79cc30a640 1 Processor -- start 2024-04-25T10:09:29.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79cc30a640 1 -- start start 2024-04-25T10:09:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79cc30a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c40aeaa0 0x7f79c4166450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79cc30a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c4166990 con 0x7f79c40b2f70 2024-04-25T10:09:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79cc30a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c4162910 con 0x7f79c4162450 2024-04-25T10:09:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79cc30a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4162bf0 con 0x7f79c40aeaa0 2024-04-25T10:09:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79ca8a7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f79c40b2f70 0x7f79c40b3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44542/0 (socket says 172.21.15.60:44542) 2024-04-25T10:09:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79ca8a7640 1 -- 172.21.15.60:0/1867416964 learned_addr learned my addr 172.21.15.60:0/1867416964 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79ca0a6640 1 --2- 172.21.15.60:0/1867416964 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c40aeaa0 0x7f79c4166450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 -- 172.21.15.60:0/1867416964 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719453992 0 0) 0x7f79bc0014a0 con 0x7f79c40b2f70 2024-04-25T10:09:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 -- 172.21.15.60:0/1867416964 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79b0003530 con 0x7f79c40b2f70 2024-04-25T10:09:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 -- 172.21.15.60:0/1867416964 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260310028 0 0) 0x7f79c4162910 con 0x7f79c4162450 2024-04-25T10:09:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 -- 172.21.15.60:0/1867416964 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79bc0014a0 con 0x7f79c4162450 2024-04-25T10:09:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 -- 172.21.15.60:0/1867416964 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3805398096 0 0) 0x7f79b0003530 con 0x7f79c40b2f70 2024-04-25T10:09:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 -- 172.21.15.60:0/1867416964 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c40aeaa0 msgr2=0x7f79c4166450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 --2- 172.21.15.60:0/1867416964 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c40aeaa0 0x7f79c4166450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 -- 172.21.15.60:0/1867416964 >> v1:172.21.15.100:6789/0 conn(0x7f79c4162450 legacy=0x7f79c406db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 -- 172.21.15.60:0/1867416964 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c4162ed0 con 0x7f79c40b2f70 2024-04-25T10:09:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 -- 172.21.15.60:0/1867416964 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f79bc001bd0 con 0x7f79c40b2f70 2024-04-25T10:09:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 -- 172.21.15.60:0/1867416964 >> v1:172.21.15.60:6789/0 conn(0x7f79c40b2f70 legacy=0x7f79c40b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 --2- 172.21.15.60:0/1867416964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79b0003b80 0x7f79b0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 --2- 172.21.15.60:0/1867416964 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c40aeaa0 0x7f79b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 --2- 172.21.15.60:0/1867416964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b0004d50 0x7f79b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 -- 172.21.15.60:0/1867416964 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4162bf0 con 0x7f79b0003b80 2024-04-25T10:09:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 -- 172.21.15.60:0/1867416964 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b000ba80 con 0x7f79b0004d50 2024-04-25T10:09:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c90a4640 1 -- 172.21.15.60:0/1867416964 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b000bc80 con 0x7f79c40aeaa0 2024-04-25T10:09:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79ca0a6640 1 --2- 172.21.15.60:0/1867416964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79b0003b80 0x7f79b0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79c98a5640 1 --2- 172.21.15.60:0/1867416964 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c40aeaa0 0x7f79b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79ca8a7640 1 --2- 172.21.15.60:0/1867416964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b0004d50 0x7f79b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79ca0a6640 1 -- 172.21.15.60:0/1867416964 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c40aeaa0 msgr2=0x7f79b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79ca0a6640 1 --2- 172.21.15.60:0/1867416964 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c40aeaa0 0x7f79b0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79ca0a6640 1 -- 172.21.15.60:0/1867416964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b0004d50 msgr2=0x7f79b00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79ca0a6640 1 --2- 172.21.15.60:0/1867416964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b0004d50 0x7f79b00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79ca0a6640 1 -- 172.21.15.60:0/1867416964 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b0003530 con 0x7f79b0003b80 2024-04-25T10:09:29.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.701+0000 7f79ca8a7640 1 --2- 172.21.15.60:0/1867416964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b0004d50 0x7f79b00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.705+0000 7f79ca0a6640 1 --2- 172.21.15.60:0/1867416964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79b0003b80 0x7f79b0003f80 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009ac0 tx=0x7f79c001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dff7618c548032f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.705+0000 7f79c90a4640 1 -- 172.21.15.60:0/1867416964 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c0009690 con 0x7f79b0003b80 2024-04-25T10:09:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.705+0000 7f79c90a4640 1 -- 172.21.15.60:0/1867416964 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c00a6430 con 0x7f79b0003b80 2024-04-25T10:09:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.705+0000 7f79c90a4640 1 -- 172.21.15.60:0/1867416964 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c00a6800 con 0x7f79b0003b80 2024-04-25T10:09:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.705+0000 7f79cc30a640 1 -- 172.21.15.60:0/1867416964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79b0003b80 msgr2=0x7f79b0003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.705+0000 7f79cc30a640 1 --2- 172.21.15.60:0/1867416964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79b0003b80 0x7f79b0003f80 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009ac0 tx=0x7f79c001ca50 comp rx=0 tx=0).stop 2024-04-25T10:09:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.705+0000 7f79ca8a7640 1 -- 172.21.15.60:0/1867416964 reap_dead start 2024-04-25T10:09:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.705+0000 7f79cc30a640 1 -- 172.21.15.60:0/1867416964 shutdown_connections 2024-04-25T10:09:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.705+0000 7f79cc30a640 1 -- 172.21.15.60:0/1867416964 >> 172.21.15.60:0/1867416964 conn(0x7f79c40670b0 msgr2=0x7f79c406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.705+0000 7f79cc30a640 1 -- 172.21.15.60:0/1867416964 shutdown_connections 2024-04-25T10:09:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.705+0000 7f79cc30a640 1 -- 172.21.15.60:0/1867416964 wait complete. 2024-04-25T10:09:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.705+0000 7f79cc30a640 1 Processor -- start 2024-04-25T10:09:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.705+0000 7f79cc30a640 1 -- start start 2024-04-25T10:09:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79cc30a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b0003b80 0x7f79c40cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79cc30a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c41fb2e0 0x7f79c40b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79cc30a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c40b0a50 0x7f79c40b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79cc30a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4071fa0 con 0x7f79c41fb2e0 2024-04-25T10:09:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79cc30a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4071e20 con 0x7f79b0003b80 2024-04-25T10:09:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79cc30a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4072120 con 0x7f79c40b0a50 2024-04-25T10:09:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79c98a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c41fb2e0 0x7f79c40b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79c98a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c41fb2e0 0x7f79c40b0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37552/0 (socket says 172.21.15.60:37552) 2024-04-25T10:09:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79c98a5640 1 -- 172.21.15.60:0/3165676573 learned_addr learned my addr 172.21.15.60:0/3165676573 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79ca0a6640 1 --2- 172.21.15.60:0/3165676573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b0003b80 0x7f79c40cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79ca8a7640 1 --2- 172.21.15.60:0/3165676573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c40b0a50 0x7f79c40b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79c98a5640 1 -- 172.21.15.60:0/3165676573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c40b0a50 msgr2=0x7f79c40b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79c98a5640 1 --2- 172.21.15.60:0/3165676573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c40b0a50 0x7f79c40b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79c98a5640 1 -- 172.21.15.60:0/3165676573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b0003b80 msgr2=0x7f79c40cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79c98a5640 1 --2- 172.21.15.60:0/3165676573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b0003b80 0x7f79c40cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79c98a5640 1 -- 172.21.15.60:0/3165676573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c415a130 con 0x7f79c41fb2e0 2024-04-25T10:09:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79ca8a7640 1 --2- 172.21.15.60:0/3165676573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c40b0a50 0x7f79c40b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79c98a5640 1 --2- 172.21.15.60:0/3165676573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c41fb2e0 0x7f79c40b0510 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f79b400e7b0 tx=0x7f79b400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79baffd640 1 -- 172.21.15.60:0/3165676573 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b40050d0 con 0x7f79c41fb2e0 2024-04-25T10:09:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79baffd640 1 -- 172.21.15.60:0/3165676573 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b4005290 con 0x7f79c41fb2e0 2024-04-25T10:09:29.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79cc30a640 1 -- 172.21.15.60:0/3165676573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c415b7b0 con 0x7f79c41fb2e0 2024-04-25T10:09:29.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79baffd640 1 -- 172.21.15.60:0/3165676573 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4005480 con 0x7f79c41fb2e0 2024-04-25T10:09:29.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79cc30a640 1 -- 172.21.15.60:0/3165676573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c415bd70 con 0x7f79c41fb2e0 2024-04-25T10:09:29.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79cc30a640 1 -- 172.21.15.60:0/3165676573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c4074770 con 0x7f79c41fb2e0 2024-04-25T10:09:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79baffd640 1 -- 172.21.15.60:0/3165676573 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79b401a020 con 0x7f79c41fb2e0 2024-04-25T10:09:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79baffd640 1 --2- 172.21.15.60:0/3165676573 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79a8066f60 0x7f79a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79baffd640 1 -- 172.21.15.60:0/3165676573 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79b4084680 con 0x7f79c41fb2e0 2024-04-25T10:09:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.709+0000 7f79ca0a6640 1 --2- 172.21.15.60:0/3165676573 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79a8066f60 0x7f79a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.713+0000 7f79ca0a6640 1 --2- 172.21.15.60:0/3165676573 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79a8066f60 0x7f79a8069420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f79c00ad000 tx=0x7f79c00b2040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:29.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.713+0000 7f79baffd640 1 -- 172.21.15.60:0/3165676573 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79b4021020 con 0x7f79c41fb2e0 2024-04-25T10:09:29.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.873+0000 7f79cc30a640 1 -- 172.21.15.60:0/3165676573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79c4164400 con 0x7f79c41fb2e0 2024-04-25T10:09:29.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.873+0000 7f79baffd640 1 -- 172.21.15.60:0/3165676573 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7f79b4055690 con 0x7f79c41fb2e0 2024-04-25T10:09:29.884 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:09:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.877+0000 7f79cc30a640 1 -- 172.21.15.60:0/3165676573 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79a8066f60 msgr2=0x7f79a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.877+0000 7f79cc30a640 1 --2- 172.21.15.60:0/3165676573 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79a8066f60 0x7f79a8069420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f79c00ad000 tx=0x7f79c00b2040 comp rx=0 tx=0).stop 2024-04-25T10:09:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.877+0000 7f79cc30a640 1 -- 172.21.15.60:0/3165676573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c41fb2e0 msgr2=0x7f79c40b0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.877+0000 7f79cc30a640 1 --2- 172.21.15.60:0/3165676573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c41fb2e0 0x7f79c40b0510 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f79b400e7b0 tx=0x7f79b400ec70 comp rx=0 tx=0).stop 2024-04-25T10:09:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.877+0000 7f79cc30a640 1 -- 172.21.15.60:0/3165676573 shutdown_connections 2024-04-25T10:09:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.877+0000 7f79cc30a640 1 --2- 172.21.15.60:0/3165676573 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79a8066f60 0x7f79a8069420 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.877+0000 7f79cc30a640 1 --2- 172.21.15.60:0/3165676573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c40b0a50 0x7f79c40b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.877+0000 7f79cc30a640 1 --2- 172.21.15.60:0/3165676573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c41fb2e0 0x7f79c40b0510 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.877+0000 7f79cc30a640 1 --2- 172.21.15.60:0/3165676573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79b0003b80 0x7f79c40cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.877+0000 7f79cc30a640 1 -- 172.21.15.60:0/3165676573 >> 172.21.15.60:0/3165676573 conn(0x7f79c40670b0 msgr2=0x7f79c40cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:29.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.877+0000 7f79cc30a640 1 -- 172.21.15.60:0/3165676573 shutdown_connections 2024-04-25T10:09:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:29.877+0000 7f79cc30a640 1 -- 172.21.15.60:0/3165676573 wait complete. 2024-04-25T10:09:29.905 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:09:30.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca85899640 1 Processor -- start 2024-04-25T10:09:30.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca85899640 1 -- start start 2024-04-25T10:09:30.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca85899640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca8015b6f0 0x7fca801620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca85899640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8015bad0 con 0x7fca800ceef0 2024-04-25T10:09:30.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca85899640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8015ebe0 con 0x7fca8015e0d0 2024-04-25T10:09:30.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca85899640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca80162610 con 0x7fca8015b6f0 2024-04-25T10:09:30.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca7f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fca800ceef0 0x7fca800d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44550/0 (socket says 172.21.15.60:44550) 2024-04-25T10:09:30.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca7f7fe640 1 -- 172.21.15.60:0/2412062968 learned_addr learned my addr 172.21.15.60:0/2412062968 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:30.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca7effd640 1 --2- 172.21.15.60:0/2412062968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca8015b6f0 0x7fca801620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca7dffb640 1 -- 172.21.15.60:0/2412062968 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203767273 0 0) 0x7fca8015bad0 con 0x7fca800ceef0 2024-04-25T10:09:30.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca7dffb640 1 -- 172.21.15.60:0/2412062968 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca6c003530 con 0x7fca800ceef0 2024-04-25T10:09:30.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca7dffb640 1 -- 172.21.15.60:0/2412062968 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 608852212 0 0) 0x7fca6c003530 con 0x7fca800ceef0 2024-04-25T10:09:30.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca7dffb640 1 -- 172.21.15.60:0/2412062968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca8015b6f0 msgr2=0x7fca801620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:30.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca7dffb640 1 --2- 172.21.15.60:0/2412062968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca8015b6f0 0x7fca801620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca7dffb640 1 -- 172.21.15.60:0/2412062968 >> v1:172.21.15.100:6789/0 conn(0x7fca8015e0d0 legacy=0x7fca800ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca7dffb640 1 -- 172.21.15.60:0/2412062968 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8015eec0 con 0x7fca800ceef0 2024-04-25T10:09:30.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca7dffb640 1 -- 172.21.15.60:0/2412062968 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fca78001680 con 0x7fca800ceef0 2024-04-25T10:09:30.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca7dffb640 1 -- 172.21.15.60:0/2412062968 >> v1:172.21.15.60:6789/0 conn(0x7fca800ceef0 legacy=0x7fca800d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca7dffb640 1 --2- 172.21.15.60:0/2412062968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca6c0039f0 0x7fca6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.017+0000 7fca7dffb640 1 --2- 172.21.15.60:0/2412062968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca8015b6f0 0x7fca6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7dffb640 1 --2- 172.21.15.60:0/2412062968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca6c004d50 0x7fca6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7dffb640 1 -- 172.21.15.60:0/2412062968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca80162610 con 0x7fca6c0039f0 2024-04-25T10:09:30.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7dffb640 1 -- 172.21.15.60:0/2412062968 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c00ba80 con 0x7fca6c004d50 2024-04-25T10:09:30.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7dffb640 1 -- 172.21.15.60:0/2412062968 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c00bc30 con 0x7fca8015b6f0 2024-04-25T10:09:30.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7effd640 1 --2- 172.21.15.60:0/2412062968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca6c0039f0 0x7fca6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7e7fc640 1 --2- 172.21.15.60:0/2412062968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca8015b6f0 0x7fca6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7effd640 1 -- 172.21.15.60:0/2412062968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca8015b6f0 msgr2=0x7fca6c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:30.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7f7fe640 1 --2- 172.21.15.60:0/2412062968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca6c004d50 0x7fca6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7effd640 1 --2- 172.21.15.60:0/2412062968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca8015b6f0 0x7fca6c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7effd640 1 -- 172.21.15.60:0/2412062968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca6c004d50 msgr2=0x7fca6c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:30.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7effd640 1 --2- 172.21.15.60:0/2412062968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca6c004d50 0x7fca6c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7effd640 1 -- 172.21.15.60:0/2412062968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c003530 con 0x7fca6c0039f0 2024-04-25T10:09:30.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7effd640 1 --2- 172.21.15.60:0/2412062968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca6c0039f0 0x7fca6c003dd0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fca68009ae0 tx=0x7fca6801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e78070afa4e7426 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:30.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7dffb640 1 -- 172.21.15.60:0/2412062968 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca680b2070 con 0x7fca6c0039f0 2024-04-25T10:09:30.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7dffb640 1 -- 172.21.15.60:0/2412062968 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca680a5660 con 0x7fca6c0039f0 2024-04-25T10:09:30.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7dffb640 1 -- 172.21.15.60:0/2412062968 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca680096b0 con 0x7fca6c0039f0 2024-04-25T10:09:30.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca85899640 1 -- 172.21.15.60:0/2412062968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca6c0039f0 msgr2=0x7fca6c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca85899640 1 --2- 172.21.15.60:0/2412062968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca6c0039f0 0x7fca6c003dd0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fca68009ae0 tx=0x7fca6801ca70 comp rx=0 tx=0).stop 2024-04-25T10:09:30.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca7f7fe640 1 -- 172.21.15.60:0/2412062968 reap_dead start 2024-04-25T10:09:30.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca85899640 1 -- 172.21.15.60:0/2412062968 shutdown_connections 2024-04-25T10:09:30.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca85899640 1 -- 172.21.15.60:0/2412062968 >> 172.21.15.60:0/2412062968 conn(0x7fca80072340 msgr2=0x7fca80072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:30.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca85899640 1 -- 172.21.15.60:0/2412062968 shutdown_connections 2024-04-25T10:09:30.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.021+0000 7fca85899640 1 -- 172.21.15.60:0/2412062968 wait complete. 2024-04-25T10:09:30.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca85899640 1 Processor -- start 2024-04-25T10:09:30.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca85899640 1 -- start start 2024-04-25T10:09:30.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca85899640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca6c0039f0 0x7fca80152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca85899640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca800c7690 0x7fca800c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca85899640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca800cd150 0x7fca800cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca85899640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca800b9550 con 0x7fca6c0039f0 2024-04-25T10:09:30.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca85899640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca80161c40 con 0x7fca800cd150 2024-04-25T10:09:30.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca85899640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca800b96d0 con 0x7fca800c7690 2024-04-25T10:09:30.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca7effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca6c0039f0 0x7fca80152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca7effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca6c0039f0 0x7fca80152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37566/0 (socket says 172.21.15.60:37566) 2024-04-25T10:09:30.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca7f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca800cd150 0x7fca800cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca7effd640 1 -- 172.21.15.60:0/3246723856 learned_addr learned my addr 172.21.15.60:0/3246723856 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:30.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca7e7fc640 1 --2- 172.21.15.60:0/3246723856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca800c7690 0x7fca800c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca7effd640 1 -- 172.21.15.60:0/3246723856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca800c7690 msgr2=0x7fca800c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca7effd640 1 --2- 172.21.15.60:0/3246723856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca800c7690 0x7fca800c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca7effd640 1 -- 172.21.15.60:0/3246723856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca800cd150 msgr2=0x7fca800cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca7effd640 1 --2- 172.21.15.60:0/3246723856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca800cd150 0x7fca800cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca7effd640 1 -- 172.21.15.60:0/3246723856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca800cc3d0 con 0x7fca6c0039f0 2024-04-25T10:09:30.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca7effd640 1 --2- 172.21.15.60:0/3246723856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca6c0039f0 0x7fca80152f20 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fca680b0040 tx=0x7fca6c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:30.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca77fff640 1 -- 172.21.15.60:0/3246723856 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca680bc030 con 0x7fca6c0039f0 2024-04-25T10:09:30.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca77fff640 1 -- 172.21.15.60:0/3246723856 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca680a5e10 con 0x7fca6c0039f0 2024-04-25T10:09:30.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca85899640 1 -- 172.21.15.60:0/3246723856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca801f40f0 con 0x7fca6c0039f0 2024-04-25T10:09:30.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca77fff640 1 -- 172.21.15.60:0/3246723856 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca680b2040 con 0x7fca6c0039f0 2024-04-25T10:09:30.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.025+0000 7fca85899640 1 -- 172.21.15.60:0/3246723856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca800cc950 con 0x7fca6c0039f0 2024-04-25T10:09:30.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.029+0000 7fca85899640 1 -- 172.21.15.60:0/3246723856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca80074770 con 0x7fca6c0039f0 2024-04-25T10:09:30.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.029+0000 7fca77fff640 1 -- 172.21.15.60:0/3246723856 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca680bf070 con 0x7fca6c0039f0 2024-04-25T10:09:30.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.029+0000 7fca77fff640 1 --2- 172.21.15.60:0/3246723856 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fca60066f60 0x7fca60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.029+0000 7fca77fff640 1 -- 172.21.15.60:0/3246723856 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca6811ec70 con 0x7fca6c0039f0 2024-04-25T10:09:30.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.029+0000 7fca7e7fc640 1 --2- 172.21.15.60:0/3246723856 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fca60066f60 0x7fca60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.029+0000 7fca7e7fc640 1 --2- 172.21.15.60:0/3246723856 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fca60066f60 0x7fca60069420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fca70005700 tx=0x7fca70005690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:30.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.033+0000 7fca77fff640 1 -- 172.21.15.60:0/3246723856 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca680c7020 con 0x7fca6c0039f0 2024-04-25T10:09:30.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.193+0000 7fca85899640 1 -- 172.21.15.60:0/3246723856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca800d16e0 con 0x7fca6c0039f0 2024-04-25T10:09:30.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.193+0000 7fca77fff640 1 -- 172.21.15.60:0/3246723856 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7fca68016780 con 0x7fca6c0039f0 2024-04-25T10:09:30.202 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:09:30.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.193+0000 7fca85899640 1 -- 172.21.15.60:0/3246723856 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fca60066f60 msgr2=0x7fca60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.193+0000 7fca85899640 1 --2- 172.21.15.60:0/3246723856 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fca60066f60 0x7fca60069420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fca70005700 tx=0x7fca70005690 comp rx=0 tx=0).stop 2024-04-25T10:09:30.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.193+0000 7fca85899640 1 -- 172.21.15.60:0/3246723856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca6c0039f0 msgr2=0x7fca80152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.193+0000 7fca85899640 1 --2- 172.21.15.60:0/3246723856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca6c0039f0 0x7fca80152f20 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fca680b0040 tx=0x7fca6c005840 comp rx=0 tx=0).stop 2024-04-25T10:09:30.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.193+0000 7fca85899640 1 -- 172.21.15.60:0/3246723856 shutdown_connections 2024-04-25T10:09:30.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.193+0000 7fca85899640 1 --2- 172.21.15.60:0/3246723856 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fca60066f60 0x7fca60069420 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.193+0000 7fca85899640 1 --2- 172.21.15.60:0/3246723856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca800cd150 0x7fca800cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.193+0000 7fca85899640 1 --2- 172.21.15.60:0/3246723856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca800c7690 0x7fca800c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.193+0000 7fca85899640 1 --2- 172.21.15.60:0/3246723856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca6c0039f0 0x7fca80152f20 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.193+0000 7fca85899640 1 -- 172.21.15.60:0/3246723856 >> 172.21.15.60:0/3246723856 conn(0x7fca80072340 msgr2=0x7fca8015f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:30.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.193+0000 7fca85899640 1 -- 172.21.15.60:0/3246723856 shutdown_connections 2024-04-25T10:09:30.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.193+0000 7fca85899640 1 -- 172.21.15.60:0/3246723856 wait complete. 2024-04-25T10:09:30.222 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b797af640 1 Processor -- start 2024-04-25T10:09:30.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b797af640 1 -- start start 2024-04-25T10:09:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b797af640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7415daa0 0x7f3b7406f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b797af640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3b7406d720 0x7f3b740716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b797af640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3b7406dba0 0x7f3b74071c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b797af640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7406ed80 con 0x7f3b7406dba0 2024-04-25T10:09:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b797af640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7406e170 con 0x7f3b7406d720 2024-04-25T10:09:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b797af640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7406e450 con 0x7f3b7415daa0 2024-04-25T10:09:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b737fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3b7406dba0 0x7f3b74071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b737fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3b7406dba0 0x7f3b74071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37578/0 (socket says 172.21.15.60:37578) 2024-04-25T10:09:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b737fe640 1 -- 172.21.15.60:0/1645053002 learned_addr learned my addr 172.21.15.60:0/1645053002 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b727fc640 1 --2- 172.21.15.60:0/1645053002 >> v2:172.21.15.100:3300/0 conn(0x7f3b7406d720 0x7f3b740716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b72ffd640 1 --2- 172.21.15.60:0/1645053002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7415daa0 0x7f3b7406f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b737fe640 1 -- 172.21.15.60:0/1645053002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7415daa0 msgr2=0x7f3b7406f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:30.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b737fe640 1 --2- 172.21.15.60:0/1645053002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7415daa0 0x7f3b7406f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b737fe640 1 -- 172.21.15.60:0/1645053002 >> v2:172.21.15.100:3300/0 conn(0x7f3b7406d720 msgr2=0x7f3b740716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b737fe640 1 --2- 172.21.15.60:0/1645053002 >> v2:172.21.15.100:3300/0 conn(0x7f3b7406d720 0x7f3b740716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.333+0000 7f3b737fe640 1 -- 172.21.15.60:0/1645053002 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7406e730 con 0x7f3b7406dba0 2024-04-25T10:09:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.337+0000 7f3b737fe640 1 --2- 172.21.15.60:0/1645053002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7406dba0 0x7f3b74071c00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f3b74070950 tx=0x7f3b6c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bb84e7353a5974 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.337+0000 7f3b71ffb640 1 -- 172.21.15.60:0/1645053002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0a5820 con 0x7f3b7406dba0 2024-04-25T10:09:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.337+0000 7f3b71ffb640 1 -- 172.21.15.60:0/1645053002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c0a59c0 con 0x7f3b7406dba0 2024-04-25T10:09:30.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.337+0000 7f3b71ffb640 1 -- 172.21.15.60:0/1645053002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0a5d90 con 0x7f3b7406dba0 2024-04-25T10:09:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.337+0000 7f3b797af640 1 -- 172.21.15.60:0/1645053002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7406dba0 msgr2=0x7f3b74071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.337+0000 7f3b797af640 1 --2- 172.21.15.60:0/1645053002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7406dba0 0x7f3b74071c00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f3b74070950 tx=0x7f3b6c01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.337+0000 7f3b797af640 1 -- 172.21.15.60:0/1645053002 shutdown_connections 2024-04-25T10:09:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.337+0000 7f3b797af640 1 --2- 172.21.15.60:0/1645053002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7406dba0 0x7f3b74071c00 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.337+0000 7f3b797af640 1 --2- 172.21.15.60:0/1645053002 >> v2:172.21.15.100:3300/0 conn(0x7f3b7406d720 0x7f3b740716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.337+0000 7f3b797af640 1 --2- 172.21.15.60:0/1645053002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7415daa0 0x7f3b7406f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.337+0000 7f3b797af640 1 -- 172.21.15.60:0/1645053002 >> 172.21.15.60:0/1645053002 conn(0x7f3b740cf9d0 msgr2=0x7f3b740cfdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.337+0000 7f3b797af640 1 -- 172.21.15.60:0/1645053002 shutdown_connections 2024-04-25T10:09:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.337+0000 7f3b797af640 1 -- 172.21.15.60:0/1645053002 wait complete. 2024-04-25T10:09:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.337+0000 7f3b797af640 1 Processor -- start 2024-04-25T10:09:30.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.337+0000 7f3b797af640 1 -- start start 2024-04-25T10:09:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b797af640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7406d720 0x7f3b74156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b797af640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7406dba0 0x7f3b741607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b797af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7415daa0 0x7f3b7415e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b797af640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b74161fe0 con 0x7f3b7406dba0 2024-04-25T10:09:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b797af640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b74161e60 con 0x7f3b7415daa0 2024-04-25T10:09:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b797af640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b74162160 con 0x7f3b7406d720 2024-04-25T10:09:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b727fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7406dba0 0x7f3b741607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b727fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7406dba0 0x7f3b741607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37586/0 (socket says 172.21.15.60:37586) 2024-04-25T10:09:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b727fc640 1 -- 172.21.15.60:0/2727383289 learned_addr learned my addr 172.21.15.60:0/2727383289 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b72ffd640 1 --2- 172.21.15.60:0/2727383289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7406d720 0x7f3b74156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b737fe640 1 --2- 172.21.15.60:0/2727383289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7415daa0 0x7f3b7415e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b727fc640 1 -- 172.21.15.60:0/2727383289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7406d720 msgr2=0x7f3b74156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b727fc640 1 --2- 172.21.15.60:0/2727383289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7406d720 0x7f3b74156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b727fc640 1 -- 172.21.15.60:0/2727383289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7415daa0 msgr2=0x7f3b7415e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b727fc640 1 --2- 172.21.15.60:0/2727383289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7415daa0 0x7f3b7415e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b727fc640 1 -- 172.21.15.60:0/2727383289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7414a0e0 con 0x7f3b7406dba0 2024-04-25T10:09:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b727fc640 1 --2- 172.21.15.60:0/2727383289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7406dba0 0x7f3b741607e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800d800 tx=0x7f3b6800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b5b7fe640 1 -- 172.21.15.60:0/2727383289 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68018020 con 0x7f3b7406dba0 2024-04-25T10:09:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b5b7fe640 1 -- 172.21.15.60:0/2727383289 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b680096a0 con 0x7f3b7406dba0 2024-04-25T10:09:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b5b7fe640 1 -- 172.21.15.60:0/2727383289 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68009a50 con 0x7f3b7406dba0 2024-04-25T10:09:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b797af640 1 -- 172.21.15.60:0/2727383289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7414a3c0 con 0x7f3b7406dba0 2024-04-25T10:09:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b797af640 1 -- 172.21.15.60:0/2727383289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b741500b0 con 0x7f3b7406dba0 2024-04-25T10:09:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b797af640 1 -- 172.21.15.60:0/2727383289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b74074770 con 0x7f3b7406dba0 2024-04-25T10:09:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b5b7fe640 1 -- 172.21.15.60:0/2727383289 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b6801b020 con 0x7f3b7406dba0 2024-04-25T10:09:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b5b7fe640 1 --2- 172.21.15.60:0/2727383289 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3b60066f60 0x7f3b60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b5b7fe640 1 -- 172.21.15.60:0/2727383289 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b68085610 con 0x7f3b7406dba0 2024-04-25T10:09:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.341+0000 7f3b72ffd640 1 --2- 172.21.15.60:0/2727383289 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3b60066f60 0x7f3b60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.345+0000 7f3b72ffd640 1 --2- 172.21.15.60:0/2727383289 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3b60066f60 0x7f3b60069420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400b930 tx=0x7f3b64008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:30.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.345+0000 7f3b5b7fe640 1 -- 172.21.15.60:0/2727383289 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3b68022020 con 0x7f3b7406dba0 2024-04-25T10:09:30.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.541+0000 7f3b797af640 1 -- 172.21.15.60:0/2727383289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b740b30d0 con 0x7f3b7406dba0 2024-04-25T10:09:30.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.545+0000 7f3b5b7fe640 1 -- 172.21.15.60:0/2727383289 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f3b68056620 con 0x7f3b7406dba0 2024-04-25T10:09:30.553 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:30.553 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/380273391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":380273391},{"type":"v1","addr":"172.21.15.100:6836","nonce":380273391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5635,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2965977166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2965977166},{"type":"v1","addr":"172.21.15.100:6837","nonce":2965977166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1105028629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1105028629},{"type":"v1","addr":"172.21.15.60:6834","nonce":1105028629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:10.236590+0000","modified":"2024-04-25T10:08:58.929165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.60:6835/2718963870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2718963870},{"type":"v1","addr":"172.21.15.60:6835","nonce":2718963870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:09:30.553 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 30 2024-04-25T10:09:30.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.545+0000 7f3b797af640 1 -- 172.21.15.60:0/2727383289 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3b60066f60 msgr2=0x7f3b60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.545+0000 7f3b797af640 1 --2- 172.21.15.60:0/2727383289 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3b60066f60 0x7f3b60069420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400b930 tx=0x7f3b64008040 comp rx=0 tx=0).stop 2024-04-25T10:09:30.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.545+0000 7f3b797af640 1 -- 172.21.15.60:0/2727383289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7406dba0 msgr2=0x7f3b741607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.545+0000 7f3b797af640 1 --2- 172.21.15.60:0/2727383289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7406dba0 0x7f3b741607e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800d800 tx=0x7f3b6800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.545+0000 7f3b797af640 1 -- 172.21.15.60:0/2727383289 shutdown_connections 2024-04-25T10:09:30.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.545+0000 7f3b797af640 1 --2- 172.21.15.60:0/2727383289 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3b60066f60 0x7f3b60069420 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.545+0000 7f3b797af640 1 --2- 172.21.15.60:0/2727383289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7415daa0 0x7f3b7415e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.545+0000 7f3b797af640 1 --2- 172.21.15.60:0/2727383289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7406dba0 0x7f3b741607e0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.545+0000 7f3b797af640 1 --2- 172.21.15.60:0/2727383289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7406d720 0x7f3b74156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.545+0000 7f3b797af640 1 -- 172.21.15.60:0/2727383289 >> 172.21.15.60:0/2727383289 conn(0x7f3b740cf9d0 msgr2=0x7f3b740716c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:30.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.545+0000 7f3b797af640 1 -- 172.21.15.60:0/2727383289 shutdown_connections 2024-04-25T10:09:30.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.545+0000 7f3b797af640 1 -- 172.21.15.60:0/2727383289 wait complete. 2024-04-25T10:09:30.571 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:30.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.685+0000 7faf2f96e640 1 Processor -- start 2024-04-25T10:09:30.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.689+0000 7faf2f96e640 1 -- start start 2024-04-25T10:09:30.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.689+0000 7faf2f96e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faf280ce7a0 0x7faf280ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.689+0000 7faf2f96e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faf2815bc90 0x7faf280b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.689+0000 7faf2f96e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf280cf0c0 con 0x7faf280b4660 2024-04-25T10:09:30.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.689+0000 7faf2f96e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf280cc370 con 0x7faf280ce7a0 2024-04-25T10:09:30.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.689+0000 7faf2f96e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf28071ca0 con 0x7faf2815bc90 2024-04-25T10:09:30.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.689+0000 7faf2d70a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faf280ce7a0 0x7faf280ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.689+0000 7faf2d70a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faf280ce7a0 0x7faf280ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37602/0 (socket says 172.21.15.60:37602) 2024-04-25T10:09:30.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.689+0000 7faf2d70a640 1 -- 172.21.15.60:0/1630109004 learned_addr learned my addr 172.21.15.60:0/1630109004 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:30.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.689+0000 7faf2df0b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7faf280b4660 0x7faf280b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44562/0 (socket says 172.21.15.60:44562) 2024-04-25T10:09:30.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.689+0000 7faf2cf09640 1 --2- 172.21.15.60:0/1630109004 >> v2:172.21.15.100:3300/0 conn(0x7faf2815bc90 0x7faf280b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.689+0000 7faf2d70a640 1 -- 172.21.15.60:0/1630109004 >> v2:172.21.15.100:3300/0 conn(0x7faf2815bc90 msgr2=0x7faf280b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:30.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.689+0000 7faf2d70a640 1 --2- 172.21.15.60:0/1630109004 >> v2:172.21.15.100:3300/0 conn(0x7faf2815bc90 0x7faf280b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.689+0000 7faf2d70a640 1 -- 172.21.15.60:0/1630109004 >> v1:172.21.15.60:6789/0 conn(0x7faf280b4660 legacy=0x7faf280b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.689+0000 7faf1ffff640 1 -- 172.21.15.60:0/1630109004 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388350689 0 0) 0x7faf280cf0c0 con 0x7faf280b4660 2024-04-25T10:09:30.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.689+0000 7faf2d70a640 1 -- 172.21.15.60:0/1630109004 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf280cca40 con 0x7faf280ce7a0 2024-04-25T10:09:30.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2d70a640 1 --2- 172.21.15.60:0/1630109004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf280ce7a0 0x7faf280ceb80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7faf280d0530 tx=0x7faf2401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8856d28f56da61ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:30.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf1ffff640 1 -- 172.21.15.60:0/1630109004 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf240b0070 con 0x7faf280ce7a0 2024-04-25T10:09:30.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf1ffff640 1 -- 172.21.15.60:0/1630109004 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf240ab600 con 0x7faf280ce7a0 2024-04-25T10:09:30.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf1ffff640 1 -- 172.21.15.60:0/1630109004 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf240ab9d0 con 0x7faf280ce7a0 2024-04-25T10:09:30.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2f96e640 1 -- 172.21.15.60:0/1630109004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf280ce7a0 msgr2=0x7faf280ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2f96e640 1 --2- 172.21.15.60:0/1630109004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf280ce7a0 0x7faf280ceb80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7faf280d0530 tx=0x7faf2401c520 comp rx=0 tx=0).stop 2024-04-25T10:09:30.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2f96e640 1 -- 172.21.15.60:0/1630109004 shutdown_connections 2024-04-25T10:09:30.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2f96e640 1 --2- 172.21.15.60:0/1630109004 >> v2:172.21.15.100:3300/0 conn(0x7faf2815bc90 0x7faf280b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2f96e640 1 --2- 172.21.15.60:0/1630109004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf280ce7a0 0x7faf280ceb80 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2f96e640 1 -- 172.21.15.60:0/1630109004 >> 172.21.15.60:0/1630109004 conn(0x7faf280b12d0 msgr2=0x7faf280b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:30.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2f96e640 1 -- 172.21.15.60:0/1630109004 shutdown_connections 2024-04-25T10:09:30.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2f96e640 1 -- 172.21.15.60:0/1630109004 wait complete. 2024-04-25T10:09:30.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2f96e640 1 Processor -- start 2024-04-25T10:09:30.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2f96e640 1 -- start start 2024-04-25T10:09:30.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2f96e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf280ce7a0 0x7faf2814de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2f96e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf2815bc90 0x7faf2814c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2f96e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf28153210 0x7faf28159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2f96e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf280b4400 con 0x7faf28153210 2024-04-25T10:09:30.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2f96e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf280b4280 con 0x7faf280ce7a0 2024-04-25T10:09:30.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2f96e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf280b3220 con 0x7faf2815bc90 2024-04-25T10:09:30.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2df0b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf28153210 0x7faf28159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2df0b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf28153210 0x7faf28159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37606/0 (socket says 172.21.15.60:37606) 2024-04-25T10:09:30.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2df0b640 1 -- 172.21.15.60:0/1962002784 learned_addr learned my addr 172.21.15.60:0/1962002784 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:30.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2d70a640 1 --2- 172.21.15.60:0/1962002784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf280ce7a0 0x7faf2814de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2cf09640 1 --2- 172.21.15.60:0/1962002784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf2815bc90 0x7faf2814c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2df0b640 1 -- 172.21.15.60:0/1962002784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf2815bc90 msgr2=0x7faf2814c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2df0b640 1 --2- 172.21.15.60:0/1962002784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf2815bc90 0x7faf2814c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2df0b640 1 -- 172.21.15.60:0/1962002784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf280ce7a0 msgr2=0x7faf2814de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2df0b640 1 --2- 172.21.15.60:0/1962002784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf280ce7a0 0x7faf2814de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2df0b640 1 -- 172.21.15.60:0/1962002784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf28152a80 con 0x7faf28153210 2024-04-25T10:09:30.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2cf09640 1 --2- 172.21.15.60:0/1962002784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf2815bc90 0x7faf2814c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:30.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.693+0000 7faf2df0b640 1 --2- 172.21.15.60:0/1962002784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf28153210 0x7faf28159010 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7faf2000c810 tx=0x7faf2000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:30.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.697+0000 7faf1dffb640 1 -- 172.21.15.60:0/1962002784 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20018070 con 0x7faf28153210 2024-04-25T10:09:30.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.697+0000 7faf1dffb640 1 -- 172.21.15.60:0/1962002784 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2000a640 con 0x7faf28153210 2024-04-25T10:09:30.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.697+0000 7faf1dffb640 1 -- 172.21.15.60:0/1962002784 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2000aa30 con 0x7faf28153210 2024-04-25T10:09:30.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.697+0000 7faf2f96e640 1 -- 172.21.15.60:0/1962002784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf28151740 con 0x7faf28153210 2024-04-25T10:09:30.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.697+0000 7faf2f96e640 1 -- 172.21.15.60:0/1962002784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf28152cf0 con 0x7faf28153210 2024-04-25T10:09:30.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.697+0000 7faf2f96e640 1 -- 172.21.15.60:0/1962002784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf28074770 con 0x7faf28153210 2024-04-25T10:09:30.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.697+0000 7faf1dffb640 1 -- 172.21.15.60:0/1962002784 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf2001b070 con 0x7faf28153210 2024-04-25T10:09:30.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.697+0000 7faf1dffb640 1 --2- 172.21.15.60:0/1962002784 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faf0c066f90 0x7faf0c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:30.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.697+0000 7faf1dffb640 1 -- 172.21.15.60:0/1962002784 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf20084590 con 0x7faf28153210 2024-04-25T10:09:30.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.697+0000 7faf2d70a640 1 --2- 172.21.15.60:0/1962002784 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faf0c066f90 0x7faf0c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:30.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.697+0000 7faf2d70a640 1 --2- 172.21.15.60:0/1962002784 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faf0c066f90 0x7faf0c069450 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7faf240ad000 tx=0x7faf24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:30.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.705+0000 7faf1dffb640 1 -- 172.21.15.60:0/1962002784 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf20055630 con 0x7faf28153210 2024-04-25T10:09:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.865+0000 7faf2f96e640 1 -- 172.21.15.60:0/1962002784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf280cbfc0 con 0x7faf28153210 2024-04-25T10:09:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.869+0000 7faf1dffb640 1 -- 172.21.15.60:0/1962002784 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16677 (secure 0 0 0) 0x7faf20054f30 con 0x7faf28153210 2024-04-25T10:09:30.877 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:30.877 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":58,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:08:11.878055+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:08.538247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:09.233842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.869+0000 7faf2f96e640 1 -- 172.21.15.60:0/1962002784 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faf0c066f90 msgr2=0x7faf0c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.869+0000 7faf2f96e640 1 --2- 172.21.15.60:0/1962002784 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faf0c066f90 0x7faf0c069450 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7faf240ad000 tx=0x7faf24008040 comp rx=0 tx=0).stop 2024-04-25T10:09:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.869+0000 7faf2f96e640 1 -- 172.21.15.60:0/1962002784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf28153210 msgr2=0x7faf28159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.869+0000 7faf2f96e640 1 --2- 172.21.15.60:0/1962002784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf28153210 0x7faf28159010 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7faf2000c810 tx=0x7faf2000ccd0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.869+0000 7faf2f96e640 1 -- 172.21.15.60:0/1962002784 shutdown_connections 2024-04-25T10:09:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.869+0000 7faf2f96e640 1 --2- 172.21.15.60:0/1962002784 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faf0c066f90 0x7faf0c069450 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.869+0000 7faf2f96e640 1 --2- 172.21.15.60:0/1962002784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf28153210 0x7faf28159010 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.869+0000 7faf2f96e640 1 --2- 172.21.15.60:0/1962002784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf2815bc90 0x7faf2814c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.869+0000 7faf2f96e640 1 --2- 172.21.15.60:0/1962002784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf280ce7a0 0x7faf2814de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.869+0000 7faf2f96e640 1 -- 172.21.15.60:0/1962002784 >> 172.21.15.60:0/1962002784 conn(0x7faf280b12d0 msgr2=0x7faf2815e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.869+0000 7faf2f96e640 1 -- 172.21.15.60:0/1962002784 shutdown_connections 2024-04-25T10:09:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:30.869+0000 7faf2f96e640 1 -- 172.21.15.60:0/1962002784 wait complete. 2024-04-25T10:09:30.896 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:09:30.897 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:31.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.013+0000 7f00967c4640 1 Processor -- start 2024-04-25T10:09:31.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.013+0000 7f00967c4640 1 -- start start 2024-04-25T10:09:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f00967c4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f009006ec30 0x7f00900cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f00967c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00900b37e0 0x7f00900b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f00967c4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f009006f130 con 0x7f009015a7a0 2024-04-25T10:09:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f00967c4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00900cdaa0 con 0x7f009006ec30 2024-04-25T10:09:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f00967c4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009006d2f0 con 0x7f00900b37e0 2024-04-25T10:09:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f008f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f009006ec30 0x7f00900cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f008f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f009006ec30 0x7f00900cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37618/0 (socket says 172.21.15.60:37618) 2024-04-25T10:09:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f008f7fe640 1 -- 172.21.15.60:0/1106130595 learned_addr learned my addr 172.21.15.60:0/1106130595 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f008ffff640 1 --1- 172.21.15.60:0/1106130595 >> v1:172.21.15.60:6789/0 conn(0x7f009015a7a0 0x7f0090071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44570/0 (socket says 172.21.15.60:44570) 2024-04-25T10:09:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f008f7fe640 1 -- 172.21.15.60:0/1106130595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00900b37e0 msgr2=0x7f00900b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f008f7fe640 1 --2- 172.21.15.60:0/1106130595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00900b37e0 0x7f00900b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f008f7fe640 1 -- 172.21.15.60:0/1106130595 >> v1:172.21.15.60:6789/0 conn(0x7f009015a7a0 legacy=0x7f0090071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f008f7fe640 1 -- 172.21.15.60:0/1106130595 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009006f410 con 0x7f009006ec30 2024-04-25T10:09:31.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f008f7fe640 1 --2- 172.21.15.60:0/1106130595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f009006ec30 0x7f00900cd560 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f00840098a0 tx=0x7f008401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa248625df716c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f008effd640 1 -- 172.21.15.60:0/1106130595 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00840ab650 con 0x7f009006ec30 2024-04-25T10:09:31.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f008effd640 1 -- 172.21.15.60:0/1106130595 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00840b0070 con 0x7f009006ec30 2024-04-25T10:09:31.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f008effd640 1 -- 172.21.15.60:0/1106130595 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00840aba10 con 0x7f009006ec30 2024-04-25T10:09:31.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f00967c4640 1 -- 172.21.15.60:0/1106130595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f009006ec30 msgr2=0x7f00900cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f00967c4640 1 --2- 172.21.15.60:0/1106130595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f009006ec30 0x7f00900cd560 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f00840098a0 tx=0x7f008401c540 comp rx=0 tx=0).stop 2024-04-25T10:09:31.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f00967c4640 1 -- 172.21.15.60:0/1106130595 shutdown_connections 2024-04-25T10:09:31.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f00967c4640 1 --2- 172.21.15.60:0/1106130595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00900b37e0 0x7f00900b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f00967c4640 1 --2- 172.21.15.60:0/1106130595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f009006ec30 0x7f00900cd560 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f00967c4640 1 -- 172.21.15.60:0/1106130595 >> 172.21.15.60:0/1106130595 conn(0x7f00900b12d0 msgr2=0x7f00900b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:31.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.017+0000 7f00967c4640 1 -- 172.21.15.60:0/1106130595 shutdown_connections 2024-04-25T10:09:31.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f00967c4640 1 -- 172.21.15.60:0/1106130595 wait complete. 2024-04-25T10:09:31.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f00967c4640 1 Processor -- start 2024-04-25T10:09:31.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f00967c4640 1 -- start start 2024-04-25T10:09:31.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f00967c4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f009006ec30 0x7f0090152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f00967c4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00900b37e0 0x7f00900c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f00967c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f009015a7a0 0x7f00901638a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f00967c4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009006e8c0 con 0x7f00900b37e0 2024-04-25T10:09:31.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f00967c4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009006e740 con 0x7f009006ec30 2024-04-25T10:09:31.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f00967c4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009006d8b0 con 0x7f009015a7a0 2024-04-25T10:09:31.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f008f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00900b37e0 0x7f00900c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f008f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00900b37e0 0x7f00900c4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37620/0 (socket says 172.21.15.60:37620) 2024-04-25T10:09:31.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f008f7fe640 1 -- 172.21.15.60:0/2868010709 learned_addr learned my addr 172.21.15.60:0/2868010709 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:31.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f008ffff640 1 --2- 172.21.15.60:0/2868010709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f009006ec30 0x7f0090152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f0094d61640 1 --2- 172.21.15.60:0/2868010709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f009015a7a0 0x7f00901638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f008f7fe640 1 -- 172.21.15.60:0/2868010709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f009015a7a0 msgr2=0x7f00901638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f008f7fe640 1 --2- 172.21.15.60:0/2868010709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f009015a7a0 0x7f00901638a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f008f7fe640 1 -- 172.21.15.60:0/2868010709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f009006ec30 msgr2=0x7f0090152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f008f7fe640 1 --2- 172.21.15.60:0/2868010709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f009006ec30 0x7f0090152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f008f7fe640 1 -- 172.21.15.60:0/2868010709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0090152fb0 con 0x7f00900b37e0 2024-04-25T10:09:31.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f008ffff640 1 --2- 172.21.15.60:0/2868010709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f009006ec30 0x7f0090152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:31.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f008f7fe640 1 --2- 172.21.15.60:0/2868010709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00900b37e0 0x7f00900c4dc0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f00840ad040 tx=0x7f008401ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f008cff9640 1 -- 172.21.15.60:0/2868010709 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00840b4800 con 0x7f00900b37e0 2024-04-25T10:09:31.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f008cff9640 1 -- 172.21.15.60:0/2868010709 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00840b49a0 con 0x7f00900b37e0 2024-04-25T10:09:31.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f00967c4640 1 -- 172.21.15.60:0/2868010709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0090155b90 con 0x7f00900b37e0 2024-04-25T10:09:31.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f008cff9640 1 -- 172.21.15.60:0/2868010709 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00840b4d90 con 0x7f00900b37e0 2024-04-25T10:09:31.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.021+0000 7f00967c4640 1 -- 172.21.15.60:0/2868010709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0090156150 con 0x7f00900b37e0 2024-04-25T10:09:31.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.025+0000 7f00967c4640 1 -- 172.21.15.60:0/2868010709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0090074770 con 0x7f00900b37e0 2024-04-25T10:09:31.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.025+0000 7f008cff9640 1 -- 172.21.15.60:0/2868010709 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00840c7390 con 0x7f00900b37e0 2024-04-25T10:09:31.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.025+0000 7f008cff9640 1 --2- 172.21.15.60:0/2868010709 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f007c066f90 0x7f007c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.025+0000 7f008cff9640 1 -- 172.21.15.60:0/2868010709 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f008411f870 con 0x7f00900b37e0 2024-04-25T10:09:31.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.025+0000 7f008ffff640 1 --2- 172.21.15.60:0/2868010709 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f007c066f90 0x7f007c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.025+0000 7f008ffff640 1 --2- 172.21.15.60:0/2868010709 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f007c066f90 0x7f007c069450 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0080007fd0 tx=0x7f0080009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.029+0000 7f008cff9640 1 -- 172.21.15.60:0/2868010709 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f00840c4060 con 0x7f00900b37e0 2024-04-25T10:09:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.225+0000 7f00967c4640 1 -- 172.21.15.60:0/2868010709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00900b3c90 con 0x7f00900b37e0 2024-04-25T10:09:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.225+0000 7f008cff9640 1 -- 172.21.15.60:0/2868010709 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f0090152fb0 con 0x7f00900b37e0 2024-04-25T10:09:31.235 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:31.235 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/380273391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":380273391},{"type":"v1","addr":"172.21.15.100:6836","nonce":380273391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5635,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2965977166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2965977166},{"type":"v1","addr":"172.21.15.100:6837","nonce":2965977166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1105028629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1105028629},{"type":"v1","addr":"172.21.15.60:6834","nonce":1105028629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:10.236590+0000","modified":"2024-04-25T10:08:58.929165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.60:6835/2718963870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2718963870},{"type":"v1","addr":"172.21.15.60:6835","nonce":2718963870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:09:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 30 2024-04-25T10:09:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.229+0000 7f00967c4640 1 -- 172.21.15.60:0/2868010709 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f007c066f90 msgr2=0x7f007c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.229+0000 7f00967c4640 1 --2- 172.21.15.60:0/2868010709 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f007c066f90 0x7f007c069450 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0080007fd0 tx=0x7f0080009040 comp rx=0 tx=0).stop 2024-04-25T10:09:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.229+0000 7f00967c4640 1 -- 172.21.15.60:0/2868010709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00900b37e0 msgr2=0x7f00900c4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.229+0000 7f00967c4640 1 --2- 172.21.15.60:0/2868010709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00900b37e0 0x7f00900c4dc0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f00840ad040 tx=0x7f008401ad80 comp rx=0 tx=0).stop 2024-04-25T10:09:31.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.229+0000 7f00967c4640 1 -- 172.21.15.60:0/2868010709 shutdown_connections 2024-04-25T10:09:31.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.229+0000 7f00967c4640 1 --2- 172.21.15.60:0/2868010709 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f007c066f90 0x7f007c069450 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.229+0000 7f00967c4640 1 --2- 172.21.15.60:0/2868010709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f009015a7a0 0x7f00901638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.229+0000 7f00967c4640 1 --2- 172.21.15.60:0/2868010709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00900b37e0 0x7f00900c4dc0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.229+0000 7f00967c4640 1 --2- 172.21.15.60:0/2868010709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f009006ec30 0x7f0090152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.229+0000 7f00967c4640 1 -- 172.21.15.60:0/2868010709 >> 172.21.15.60:0/2868010709 conn(0x7f00900b12d0 msgr2=0x7f0090071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:31.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.229+0000 7f00967c4640 1 -- 172.21.15.60:0/2868010709 shutdown_connections 2024-04-25T10:09:31.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.229+0000 7f00967c4640 1 -- 172.21.15.60:0/2868010709 wait complete. 2024-04-25T10:09:31.255 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:31.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.373+0000 7f1a71f27640 1 Processor -- start 2024-04-25T10:09:31.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.373+0000 7f1a71f27640 1 -- start start 2024-04-25T10:09:31.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.373+0000 7f1a71f27640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1a6c0ce7a0 0x7f1a6c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.373+0000 7f1a71f27640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a6c0b4660 0x7f1a6c0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.373+0000 7f1a71f27640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6c0cf0c0 con 0x7f1a6c15bc90 2024-04-25T10:09:31.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.373+0000 7f1a71f27640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6c0cc370 con 0x7f1a6c0ce7a0 2024-04-25T10:09:31.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.373+0000 7f1a71f27640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c071ca0 con 0x7f1a6c0b4660 2024-04-25T10:09:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.373+0000 7f1a6affd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1a6c15bc90 0x7f1a6c0b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44586/0 (socket says 172.21.15.60:44586) 2024-04-25T10:09:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.373+0000 7f1a6affd640 1 -- 172.21.15.60:0/9294083 learned_addr learned my addr 172.21.15.60:0/9294083 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.373+0000 7f1a6bfff640 1 --2- 172.21.15.60:0/9294083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a6c0b4660 0x7f1a6c0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.373+0000 7f1a6b7fe640 1 --2- 172.21.15.60:0/9294083 >> v2:172.21.15.100:3300/0 conn(0x7f1a6c0ce7a0 0x7f1a6c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.373+0000 7f1a6a7fc640 1 -- 172.21.15.60:0/9294083 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099958011 0 0) 0x7f1a6c0cf0c0 con 0x7f1a6c15bc90 2024-04-25T10:09:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 -- 172.21.15.60:0/9294083 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a54003530 con 0x7f1a6c15bc90 2024-04-25T10:09:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 -- 172.21.15.60:0/9294083 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2123299543 0 0) 0x7f1a54003530 con 0x7f1a6c15bc90 2024-04-25T10:09:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 -- 172.21.15.60:0/9294083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a6c0b4660 msgr2=0x7f1a6c0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 --2- 172.21.15.60:0/9294083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a6c0b4660 0x7f1a6c0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 -- 172.21.15.60:0/9294083 >> v2:172.21.15.100:3300/0 conn(0x7f1a6c0ce7a0 msgr2=0x7f1a6c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 --2- 172.21.15.60:0/9294083 >> v2:172.21.15.100:3300/0 conn(0x7f1a6c0ce7a0 0x7f1a6c0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 -- 172.21.15.60:0/9294083 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c0cca40 con 0x7f1a6c15bc90 2024-04-25T10:09:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 -- 172.21.15.60:0/9294083 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f1a60001680 con 0x7f1a6c15bc90 2024-04-25T10:09:31.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 -- 172.21.15.60:0/9294083 >> v1:172.21.15.60:6789/0 conn(0x7f1a6c15bc90 legacy=0x7f1a6c0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 --2- 172.21.15.60:0/9294083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a540039c0 0x7f1a54003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 --2- 172.21.15.60:0/9294083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a54004810 0x7f1a54004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 --2- 172.21.15.60:0/9294083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a54005190 0x7f1a5400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 -- 172.21.15.60:0/9294083 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c071ca0 con 0x7f1a540039c0 2024-04-25T10:09:31.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 -- 172.21.15.60:0/9294083 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a54004590 con 0x7f1a54005190 2024-04-25T10:09:31.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 -- 172.21.15.60:0/9294083 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5400bf10 con 0x7f1a54004810 2024-04-25T10:09:31.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6b7fe640 1 --2- 172.21.15.60:0/9294083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a540039c0 0x7f1a54003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6affd640 1 --2- 172.21.15.60:0/9294083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a54004810 0x7f1a54004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6bfff640 1 --2- 172.21.15.60:0/9294083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a54005190 0x7f1a5400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6b7fe640 1 -- 172.21.15.60:0/9294083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a54004810 msgr2=0x7f1a54004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:31.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6b7fe640 1 --2- 172.21.15.60:0/9294083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a54004810 0x7f1a54004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6b7fe640 1 -- 172.21.15.60:0/9294083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a54005190 msgr2=0x7f1a5400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:31.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6b7fe640 1 --2- 172.21.15.60:0/9294083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a54005190 0x7f1a5400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6b7fe640 1 -- 172.21.15.60:0/9294083 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a54003530 con 0x7f1a540039c0 2024-04-25T10:09:31.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6bfff640 1 --2- 172.21.15.60:0/9294083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a54005190 0x7f1a5400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:31.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6b7fe640 1 --2- 172.21.15.60:0/9294083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a540039c0 0x7f1a54003de0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c009ae0 tx=0x7f1a5c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882b892123c3ddc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 -- 172.21.15.60:0/9294083 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c0096b0 con 0x7f1a540039c0 2024-04-25T10:09:31.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 -- 172.21.15.60:0/9294083 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5c0a6490 con 0x7f1a540039c0 2024-04-25T10:09:31.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6a7fc640 1 -- 172.21.15.60:0/9294083 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c0a6880 con 0x7f1a540039c0 2024-04-25T10:09:31.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a71f27640 1 -- 172.21.15.60:0/9294083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a540039c0 msgr2=0x7f1a54003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a71f27640 1 --2- 172.21.15.60:0/9294083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a540039c0 0x7f1a54003de0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c009ae0 tx=0x7f1a5c01cab0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a6bfff640 1 -- 172.21.15.60:0/9294083 reap_dead start 2024-04-25T10:09:31.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a71f27640 1 -- 172.21.15.60:0/9294083 shutdown_connections 2024-04-25T10:09:31.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a71f27640 1 -- 172.21.15.60:0/9294083 >> 172.21.15.60:0/9294083 conn(0x7f1a6c0b12d0 msgr2=0x7f1a6c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:31.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a71f27640 1 -- 172.21.15.60:0/9294083 shutdown_connections 2024-04-25T10:09:31.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.377+0000 7f1a71f27640 1 -- 172.21.15.60:0/9294083 wait complete. 2024-04-25T10:09:31.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a71f27640 1 Processor -- start 2024-04-25T10:09:31.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a71f27640 1 -- start start 2024-04-25T10:09:31.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a71f27640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a540039c0 0x7f1a6c153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a71f27640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a6c163cd0 0x7f1a6c1549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a71f27640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a6c164150 0x7f1a6c154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a71f27640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0b4280 con 0x7f1a6c163cd0 2024-04-25T10:09:31.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a71f27640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0b3db0 con 0x7f1a540039c0 2024-04-25T10:09:31.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a71f27640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0b4400 con 0x7f1a6c164150 2024-04-25T10:09:31.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a6affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a6c163cd0 0x7f1a6c1549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a6affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a6c163cd0 0x7f1a6c1549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37636/0 (socket says 172.21.15.60:37636) 2024-04-25T10:09:31.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a6affd640 1 -- 172.21.15.60:0/2246462133 learned_addr learned my addr 172.21.15.60:0/2246462133 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:31.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a6b7fe640 1 --2- 172.21.15.60:0/2246462133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a540039c0 0x7f1a6c153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a6bfff640 1 --2- 172.21.15.60:0/2246462133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a6c164150 0x7f1a6c154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a6affd640 1 -- 172.21.15.60:0/2246462133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a6c164150 msgr2=0x7f1a6c154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a6affd640 1 --2- 172.21.15.60:0/2246462133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a6c164150 0x7f1a6c154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a6affd640 1 -- 172.21.15.60:0/2246462133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a540039c0 msgr2=0x7f1a6c153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a6affd640 1 --2- 172.21.15.60:0/2246462133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a540039c0 0x7f1a6c153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a6affd640 1 -- 172.21.15.60:0/2246462133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c1540a0 con 0x7f1a6c163cd0 2024-04-25T10:09:31.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a6affd640 1 --2- 172.21.15.60:0/2246462133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a6c163cd0 0x7f1a6c1549b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000e730 tx=0x7f1a6000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a53fff640 1 -- 172.21.15.60:0/2246462133 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6000c6c0 con 0x7f1a6c163cd0 2024-04-25T10:09:31.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a53fff640 1 -- 172.21.15.60:0/2246462133 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a60018070 con 0x7f1a6c163cd0 2024-04-25T10:09:31.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a71f27640 1 -- 172.21.15.60:0/2246462133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c155530 con 0x7f1a6c163cd0 2024-04-25T10:09:31.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a53fff640 1 -- 172.21.15.60:0/2246462133 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6000caa0 con 0x7f1a6c163cd0 2024-04-25T10:09:31.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.381+0000 7f1a71f27640 1 -- 172.21.15.60:0/2246462133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c153080 con 0x7f1a6c163cd0 2024-04-25T10:09:31.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.385+0000 7f1a71f27640 1 -- 172.21.15.60:0/2246462133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c074770 con 0x7f1a6c163cd0 2024-04-25T10:09:31.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.385+0000 7f1a53fff640 1 -- 172.21.15.60:0/2246462133 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a6000cc40 con 0x7f1a6c163cd0 2024-04-25T10:09:31.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.385+0000 7f1a53fff640 1 --2- 172.21.15.60:0/2246462133 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1a58066f60 0x7f1a58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.385+0000 7f1a53fff640 1 -- 172.21.15.60:0/2246462133 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a60084cd0 con 0x7f1a6c163cd0 2024-04-25T10:09:31.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.385+0000 7f1a6b7fe640 1 --2- 172.21.15.60:0/2246462133 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1a58066f60 0x7f1a58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.385+0000 7f1a6b7fe640 1 --2- 172.21.15.60:0/2246462133 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1a58066f60 0x7f1a58069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0ad000 tx=0x7f1a5c0b2040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.389+0000 7f1a53fff640 1 -- 172.21.15.60:0/2246462133 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a60055d70 con 0x7f1a6c163cd0 2024-04-25T10:09:31.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.585+0000 7f1a71f27640 1 -- 172.21.15.60:0/2246462133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a6c0ced20 con 0x7f1a6c163cd0 2024-04-25T10:09:31.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.585+0000 7f1a53fff640 1 -- 172.21.15.60:0/2246462133 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f1a60055670 con 0x7f1a6c163cd0 2024-04-25T10:09:31.596 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:31.597 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/380273391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":380273391},{"type":"v1","addr":"172.21.15.100:6836","nonce":380273391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5635,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2965977166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2965977166},{"type":"v1","addr":"172.21.15.100:6837","nonce":2965977166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1105028629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1105028629},{"type":"v1","addr":"172.21.15.60:6834","nonce":1105028629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:10.236590+0000","modified":"2024-04-25T10:08:58.929165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6291":{"gid":6291,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.60:6835/2718963870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2718963870},{"type":"v1","addr":"172.21.15.60:6835","nonce":2718963870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:09:31.598 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 30 2024-04-25T10:09:31.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.589+0000 7f1a71f27640 1 -- 172.21.15.60:0/2246462133 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1a58066f60 msgr2=0x7f1a58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.589+0000 7f1a71f27640 1 --2- 172.21.15.60:0/2246462133 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1a58066f60 0x7f1a58069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0ad000 tx=0x7f1a5c0b2040 comp rx=0 tx=0).stop 2024-04-25T10:09:31.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.589+0000 7f1a71f27640 1 -- 172.21.15.60:0/2246462133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a6c163cd0 msgr2=0x7f1a6c1549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.589+0000 7f1a71f27640 1 --2- 172.21.15.60:0/2246462133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a6c163cd0 0x7f1a6c1549b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000e730 tx=0x7f1a6000ebf0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.589+0000 7f1a71f27640 1 -- 172.21.15.60:0/2246462133 shutdown_connections 2024-04-25T10:09:31.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.589+0000 7f1a71f27640 1 --2- 172.21.15.60:0/2246462133 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1a58066f60 0x7f1a58069420 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.589+0000 7f1a71f27640 1 --2- 172.21.15.60:0/2246462133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a6c164150 0x7f1a6c154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.589+0000 7f1a71f27640 1 --2- 172.21.15.60:0/2246462133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a6c163cd0 0x7f1a6c1549b0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.589+0000 7f1a71f27640 1 --2- 172.21.15.60:0/2246462133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a540039c0 0x7f1a6c153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.589+0000 7f1a71f27640 1 -- 172.21.15.60:0/2246462133 >> 172.21.15.60:0/2246462133 conn(0x7f1a6c0b12d0 msgr2=0x7f1a6c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:31.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.589+0000 7f1a71f27640 1 -- 172.21.15.60:0/2246462133 shutdown_connections 2024-04-25T10:09:31.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.589+0000 7f1a71f27640 1 -- 172.21.15.60:0/2246462133 wait complete. 2024-04-25T10:09:31.615 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:31.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.725+0000 7f1972641640 1 Processor -- start 2024-04-25T10:09:31.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.725+0000 7f1972641640 1 -- start start 2024-04-25T10:09:31.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f1972641640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f196c06ec30 0x7f196c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f1972641640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c06f130 con 0x7f196c15a7a0 2024-04-25T10:09:31.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f1972641640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c06f410 con 0x7f196c0b37e0 2024-04-25T10:09:31.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f1972641640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196c0cdaa0 con 0x7f196c06ec30 2024-04-25T10:09:31.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f196c15a7a0 0x7f196c071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44602/0 (socket says 172.21.15.60:44602) 2024-04-25T10:09:31.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196bfff640 1 -- 172.21.15.60:0/512983331 learned_addr learned my addr 172.21.15.60:0/512983331 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:31.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196affd640 1 -- 172.21.15.60:0/512983331 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2250063812 0 0) 0x7f196c06f130 con 0x7f196c15a7a0 2024-04-25T10:09:31.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196affd640 1 -- 172.21.15.60:0/512983331 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1954003530 con 0x7f196c15a7a0 2024-04-25T10:09:31.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196b7fe640 1 --2- 172.21.15.60:0/512983331 >> v2:172.21.15.100:3300/0 conn(0x7f196c06ec30 0x7f196c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196affd640 1 -- 172.21.15.60:0/512983331 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 805913496 0 0) 0x7f1954003530 con 0x7f196c15a7a0 2024-04-25T10:09:31.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196affd640 1 -- 172.21.15.60:0/512983331 >> v2:172.21.15.100:3300/0 conn(0x7f196c06ec30 msgr2=0x7f196c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:31.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196affd640 1 --2- 172.21.15.60:0/512983331 >> v2:172.21.15.100:3300/0 conn(0x7f196c06ec30 0x7f196c0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196affd640 1 -- 172.21.15.60:0/512983331 >> v1:172.21.15.100:6789/0 conn(0x7f196c0b37e0 legacy=0x7f196c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196affd640 1 -- 172.21.15.60:0/512983331 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c06d540 con 0x7f196c15a7a0 2024-04-25T10:09:31.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196affd640 1 -- 172.21.15.60:0/512983331 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f195c001680 con 0x7f196c15a7a0 2024-04-25T10:09:31.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196affd640 1 -- 172.21.15.60:0/512983331 >> v1:172.21.15.60:6789/0 conn(0x7f196c15a7a0 legacy=0x7f196c071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196affd640 1 --2- 172.21.15.60:0/512983331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19540039f0 0x7f1954003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196affd640 1 --2- 172.21.15.60:0/512983331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1954004810 0x7f1954004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196affd640 1 --2- 172.21.15.60:0/512983331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19540051d0 0x7f195400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196affd640 1 -- 172.21.15.60:0/512983331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195400c0a0 con 0x7f19540051d0 2024-04-25T10:09:31.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196affd640 1 -- 172.21.15.60:0/512983331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195400c220 con 0x7f19540039f0 2024-04-25T10:09:31.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196affd640 1 -- 172.21.15.60:0/512983331 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195400c420 con 0x7f1954004810 2024-04-25T10:09:31.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f1970bde640 1 --2- 172.21.15.60:0/512983331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19540051d0 0x7f195400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196bfff640 1 --2- 172.21.15.60:0/512983331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19540039f0 0x7f1954003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196b7fe640 1 --2- 172.21.15.60:0/512983331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1954004810 0x7f1954004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f1970bde640 1 -- 172.21.15.60:0/512983331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1954004810 msgr2=0x7f1954004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:31.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f1970bde640 1 --2- 172.21.15.60:0/512983331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1954004810 0x7f1954004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f1970bde640 1 -- 172.21.15.60:0/512983331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19540039f0 msgr2=0x7f1954003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:31.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f1970bde640 1 --2- 172.21.15.60:0/512983331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19540039f0 0x7f1954003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f1970bde640 1 -- 172.21.15.60:0/512983331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1954003530 con 0x7f19540051d0 2024-04-25T10:09:31.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f1970bde640 1 -- 172.21.15.60:0/512983331 reap_dead start 2024-04-25T10:09:31.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f1970bde640 1 --2- 172.21.15.60:0/512983331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19540051d0 0x7f195400b9d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1964001e70 tx=0x7f196401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6198ff7980f6b1fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.729+0000 7f196affd640 1 -- 172.21.15.60:0/512983331 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1964001a90 con 0x7f19540051d0 2024-04-25T10:09:31.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f196affd640 1 -- 172.21.15.60:0/512983331 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1964003bc0 con 0x7f19540051d0 2024-04-25T10:09:31.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f196affd640 1 -- 172.21.15.60:0/512983331 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1964005020 con 0x7f19540051d0 2024-04-25T10:09:31.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1972641640 1 -- 172.21.15.60:0/512983331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19540051d0 msgr2=0x7f195400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1972641640 1 --2- 172.21.15.60:0/512983331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19540051d0 0x7f195400b9d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1964001e70 tx=0x7f196401c780 comp rx=0 tx=0).stop 2024-04-25T10:09:31.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1972641640 1 -- 172.21.15.60:0/512983331 shutdown_connections 2024-04-25T10:09:31.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1972641640 1 --2- 172.21.15.60:0/512983331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19540051d0 0x7f195400b9d0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1972641640 1 -- 172.21.15.60:0/512983331 >> 172.21.15.60:0/512983331 conn(0x7f196c0b12d0 msgr2=0x7f196c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:31.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1972641640 1 -- 172.21.15.60:0/512983331 shutdown_connections 2024-04-25T10:09:31.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1972641640 1 -- 172.21.15.60:0/512983331 wait complete. 2024-04-25T10:09:31.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1972641640 1 Processor -- start 2024-04-25T10:09:31.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1972641640 1 -- start start 2024-04-25T10:09:31.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1972641640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19540051d0 0x7f196c0a8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1972641640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f196c1f0cb0 0x7f196c158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1972641640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f196c146c00 0x7f196c147080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1972641640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c06d8b0 con 0x7f19540051d0 2024-04-25T10:09:31.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1972641640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c06d2f0 con 0x7f196c146c00 2024-04-25T10:09:31.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1972641640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0cd560 con 0x7f196c1f0cb0 2024-04-25T10:09:31.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f196bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19540051d0 0x7f196c0a8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f196bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19540051d0 0x7f196c0a8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37648/0 (socket says 172.21.15.60:37648) 2024-04-25T10:09:31.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f196bfff640 1 -- 172.21.15.60:0/1121978322 learned_addr learned my addr 172.21.15.60:0/1121978322 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:31.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f196b7fe640 1 --2- 172.21.15.60:0/1121978322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f196c1f0cb0 0x7f196c158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1970bde640 1 --2- 172.21.15.60:0/1121978322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f196c146c00 0x7f196c147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f196bfff640 1 -- 172.21.15.60:0/1121978322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f196c1f0cb0 msgr2=0x7f196c158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f196bfff640 1 --2- 172.21.15.60:0/1121978322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f196c1f0cb0 0x7f196c158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f196bfff640 1 -- 172.21.15.60:0/1121978322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f196c146c00 msgr2=0x7f196c147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f196bfff640 1 --2- 172.21.15.60:0/1121978322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f196c146c00 0x7f196c147080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f196bfff640 1 -- 172.21.15.60:0/1121978322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c15f270 con 0x7f19540051d0 2024-04-25T10:09:31.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f196bfff640 1 --2- 172.21.15.60:0/1121978322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19540051d0 0x7f196c0a8f80 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f195c0072f0 tx=0x7f195c0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1968ff9640 1 -- 172.21.15.60:0/1121978322 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c01a070 con 0x7f19540051d0 2024-04-25T10:09:31.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1968ff9640 1 -- 172.21.15.60:0/1121978322 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c00f8e0 con 0x7f19540051d0 2024-04-25T10:09:31.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1972641640 1 -- 172.21.15.60:0/1121978322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c1f1110 con 0x7f19540051d0 2024-04-25T10:09:31.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.733+0000 7f1968ff9640 1 -- 172.21.15.60:0/1121978322 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c00fcd0 con 0x7f19540051d0 2024-04-25T10:09:31.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.737+0000 7f1972641640 1 -- 172.21.15.60:0/1121978322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c0a9650 con 0x7f19540051d0 2024-04-25T10:09:31.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.737+0000 7f1972641640 1 -- 172.21.15.60:0/1121978322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c074770 con 0x7f19540051d0 2024-04-25T10:09:31.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.737+0000 7f1968ff9640 1 -- 172.21.15.60:0/1121978322 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f195c01d070 con 0x7f19540051d0 2024-04-25T10:09:31.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.737+0000 7f1968ff9640 1 --2- 172.21.15.60:0/1121978322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1958066f60 0x7f1958069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:31.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.737+0000 7f1968ff9640 1 -- 172.21.15.60:0/1121978322 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f195c084cf0 con 0x7f19540051d0 2024-04-25T10:09:31.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.737+0000 7f196b7fe640 1 --2- 172.21.15.60:0/1121978322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1958066f60 0x7f1958069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:31.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.737+0000 7f196b7fe640 1 --2- 172.21.15.60:0/1121978322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1958066f60 0x7f1958069420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f196c149e20 tx=0x7f1960008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:31.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.741+0000 7f1968ff9640 1 -- 172.21.15.60:0/1121978322 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f195c055d50 con 0x7f19540051d0 2024-04-25T10:09:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.901+0000 7f1972641640 1 -- 172.21.15.60:0/1121978322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f196c15ef90 con 0x7f19540051d0 2024-04-25T10:09:31.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.905+0000 7f1968ff9640 1 -- 172.21.15.60:0/1121978322 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16677 (secure 0 0 0) 0x7f195c055650 con 0x7f19540051d0 2024-04-25T10:09:31.914 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:31.914 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":58,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:08:11.878055+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:08.538247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:09.233842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.905+0000 7f1972641640 1 -- 172.21.15.60:0/1121978322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1958066f60 msgr2=0x7f1958069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.905+0000 7f1972641640 1 --2- 172.21.15.60:0/1121978322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1958066f60 0x7f1958069420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f196c149e20 tx=0x7f1960008040 comp rx=0 tx=0).stop 2024-04-25T10:09:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.905+0000 7f1972641640 1 -- 172.21.15.60:0/1121978322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19540051d0 msgr2=0x7f196c0a8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.905+0000 7f1972641640 1 --2- 172.21.15.60:0/1121978322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19540051d0 0x7f196c0a8f80 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f195c0072f0 tx=0x7f195c0077b0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.905+0000 7f1972641640 1 -- 172.21.15.60:0/1121978322 shutdown_connections 2024-04-25T10:09:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.905+0000 7f1972641640 1 --2- 172.21.15.60:0/1121978322 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1958066f60 0x7f1958069420 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.905+0000 7f1972641640 1 --2- 172.21.15.60:0/1121978322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f196c146c00 0x7f196c147080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.905+0000 7f1972641640 1 --2- 172.21.15.60:0/1121978322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f196c1f0cb0 0x7f196c158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.905+0000 7f1972641640 1 --2- 172.21.15.60:0/1121978322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19540051d0 0x7f196c0a8f80 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.905+0000 7f1972641640 1 -- 172.21.15.60:0/1121978322 >> 172.21.15.60:0/1121978322 conn(0x7f196c0b12d0 msgr2=0x7f196c0cecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.905+0000 7f1972641640 1 -- 172.21.15.60:0/1121978322 shutdown_connections 2024-04-25T10:09:31.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:31.909+0000 7f1972641640 1 -- 172.21.15.60:0/1121978322 wait complete. 2024-04-25T10:09:31.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:09:32.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.037+0000 7fac59d4f640 1 Processor -- start 2024-04-25T10:09:32.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.037+0000 7fac59d4f640 1 -- start start 2024-04-25T10:09:32.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.037+0000 7fac59d4f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fac540b3ef0 0x7fac540cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.037+0000 7fac59d4f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5406f130 con 0x7fac5415a7a0 2024-04-25T10:09:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.037+0000 7fac59d4f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5406f410 con 0x7fac5406ec30 2024-04-25T10:09:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.037+0000 7fac59d4f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac540cdaa0 con 0x7fac540b3ef0 2024-04-25T10:09:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.037+0000 7fac537fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fac5415a7a0 0x7fac54071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44610/0 (socket says 172.21.15.60:44610) 2024-04-25T10:09:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.037+0000 7fac537fe640 1 -- 172.21.15.60:0/3785682653 learned_addr learned my addr 172.21.15.60:0/3785682653 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.037+0000 7fac52ffd640 1 -- 172.21.15.60:0/3785682653 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2845014146 0 0) 0x7fac5406f130 con 0x7fac5415a7a0 2024-04-25T10:09:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.037+0000 7fac53fff640 1 --2- 172.21.15.60:0/3785682653 >> v2:172.21.15.100:3300/0 conn(0x7fac540b3ef0 0x7fac540cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.037+0000 7fac52ffd640 1 -- 172.21.15.60:0/3785682653 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac40003530 con 0x7fac5415a7a0 2024-04-25T10:09:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac52ffd640 1 -- 172.21.15.60:0/3785682653 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3315941962 0 0) 0x7fac40003530 con 0x7fac5415a7a0 2024-04-25T10:09:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac52ffd640 1 -- 172.21.15.60:0/3785682653 >> v2:172.21.15.100:3300/0 conn(0x7fac540b3ef0 msgr2=0x7fac540cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac52ffd640 1 --2- 172.21.15.60:0/3785682653 >> v2:172.21.15.100:3300/0 conn(0x7fac540b3ef0 0x7fac540cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac52ffd640 1 -- 172.21.15.60:0/3785682653 >> v1:172.21.15.100:6789/0 conn(0x7fac5406ec30 legacy=0x7fac540b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac52ffd640 1 -- 172.21.15.60:0/3785682653 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5406d540 con 0x7fac5415a7a0 2024-04-25T10:09:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac52ffd640 1 -- 172.21.15.60:0/3785682653 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fac44001680 con 0x7fac5415a7a0 2024-04-25T10:09:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac52ffd640 1 -- 172.21.15.60:0/3785682653 >> v1:172.21.15.60:6789/0 conn(0x7fac5415a7a0 legacy=0x7fac54071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac52ffd640 1 --2- 172.21.15.60:0/3785682653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac400039c0 0x7fac40003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac52ffd640 1 --2- 172.21.15.60:0/3785682653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac40004810 0x7fac40004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac52ffd640 1 --2- 172.21.15.60:0/3785682653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac400051d0 0x7fac4000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac52ffd640 1 -- 172.21.15.60:0/3785682653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4000c040 con 0x7fac400051d0 2024-04-25T10:09:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac52ffd640 1 -- 172.21.15.60:0/3785682653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4000c1f0 con 0x7fac400039c0 2024-04-25T10:09:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac52ffd640 1 -- 172.21.15.60:0/3785682653 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4000c3f0 con 0x7fac40004810 2024-04-25T10:09:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac53fff640 1 --2- 172.21.15.60:0/3785682653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac400051d0 0x7fac4000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac537fe640 1 --2- 172.21.15.60:0/3785682653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac400039c0 0x7fac40003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac4affd640 1 --2- 172.21.15.60:0/3785682653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac40004810 0x7fac40004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac53fff640 1 -- 172.21.15.60:0/3785682653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac40004810 msgr2=0x7fac40004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:32.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac53fff640 1 --2- 172.21.15.60:0/3785682653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac40004810 0x7fac40004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac53fff640 1 -- 172.21.15.60:0/3785682653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac400039c0 msgr2=0x7fac40003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:32.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac53fff640 1 --2- 172.21.15.60:0/3785682653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac400039c0 0x7fac40003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac53fff640 1 -- 172.21.15.60:0/3785682653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac40003530 con 0x7fac400051d0 2024-04-25T10:09:32.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac53fff640 1 -- 172.21.15.60:0/3785682653 reap_dead start 2024-04-25T10:09:32.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac53fff640 1 --2- 172.21.15.60:0/3785682653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac400051d0 0x7fac4000b9d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fac540b2680 tx=0x7fac4c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624cf6384c3f8143 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:32.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac52ffd640 1 -- 172.21.15.60:0/3785682653 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c0b2070 con 0x7fac400051d0 2024-04-25T10:09:32.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac52ffd640 1 -- 172.21.15.60:0/3785682653 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4c0a5c70 con 0x7fac400051d0 2024-04-25T10:09:32.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac52ffd640 1 -- 172.21.15.60:0/3785682653 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c009c10 con 0x7fac400051d0 2024-04-25T10:09:32.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac59d4f640 1 -- 172.21.15.60:0/3785682653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac400051d0 msgr2=0x7fac4000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac59d4f640 1 --2- 172.21.15.60:0/3785682653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac400051d0 0x7fac4000b9d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fac540b2680 tx=0x7fac4c01cc60 comp rx=0 tx=0).stop 2024-04-25T10:09:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac59d4f640 1 -- 172.21.15.60:0/3785682653 shutdown_connections 2024-04-25T10:09:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac59d4f640 1 --2- 172.21.15.60:0/3785682653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac400051d0 0x7fac4000b9d0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac59d4f640 1 -- 172.21.15.60:0/3785682653 >> 172.21.15.60:0/3785682653 conn(0x7fac540b12d0 msgr2=0x7fac540b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac59d4f640 1 -- 172.21.15.60:0/3785682653 shutdown_connections 2024-04-25T10:09:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.041+0000 7fac59d4f640 1 -- 172.21.15.60:0/3785682653 wait complete. 2024-04-25T10:09:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac59d4f640 1 Processor -- start 2024-04-25T10:09:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac59d4f640 1 -- start start 2024-04-25T10:09:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac59d4f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac400051d0 0x7fac5414cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac59d4f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac5414fe70 0x7fac5414e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac59d4f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac54151b80 0x7fac541f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac59d4f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5406d8b0 con 0x7fac54151b80 2024-04-25T10:09:32.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac59d4f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5406d2f0 con 0x7fac5414fe70 2024-04-25T10:09:32.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac59d4f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac540cd560 con 0x7fac400051d0 2024-04-25T10:09:32.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac53fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac54151b80 0x7fac541f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac53fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac54151b80 0x7fac541f16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37676/0 (socket says 172.21.15.60:37676) 2024-04-25T10:09:32.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac53fff640 1 -- 172.21.15.60:0/342910012 learned_addr learned my addr 172.21.15.60:0/342910012 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:32.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac4affd640 1 --2- 172.21.15.60:0/342910012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac5414fe70 0x7fac5414e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac537fe640 1 --2- 172.21.15.60:0/342910012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac400051d0 0x7fac5414cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac53fff640 1 -- 172.21.15.60:0/342910012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac400051d0 msgr2=0x7fac5414cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac53fff640 1 --2- 172.21.15.60:0/342910012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac400051d0 0x7fac5414cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac53fff640 1 -- 172.21.15.60:0/342910012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac5414fe70 msgr2=0x7fac5414e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac53fff640 1 --2- 172.21.15.60:0/342910012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac5414fe70 0x7fac5414e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac53fff640 1 -- 172.21.15.60:0/342910012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5414d2d0 con 0x7fac54151b80 2024-04-25T10:09:32.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac537fe640 1 --2- 172.21.15.60:0/342910012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac400051d0 0x7fac5414cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:32.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac53fff640 1 --2- 172.21.15.60:0/342910012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac54151b80 0x7fac541f16a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fac4c01d500 tx=0x7fac4c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:32.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac50ff9640 1 -- 172.21.15.60:0/342910012 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c0b2040 con 0x7fac54151b80 2024-04-25T10:09:32.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac50ff9640 1 -- 172.21.15.60:0/342910012 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4c0a5a70 con 0x7fac54151b80 2024-04-25T10:09:32.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac59d4f640 1 -- 172.21.15.60:0/342910012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5414d5b0 con 0x7fac54151b80 2024-04-25T10:09:32.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac50ff9640 1 -- 172.21.15.60:0/342910012 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c0ade60 con 0x7fac54151b80 2024-04-25T10:09:32.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac59d4f640 1 -- 172.21.15.60:0/342910012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac5414db70 con 0x7fac54151b80 2024-04-25T10:09:32.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.045+0000 7fac59d4f640 1 -- 172.21.15.60:0/342910012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac54074770 con 0x7fac54151b80 2024-04-25T10:09:32.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.049+0000 7fac50ff9640 1 -- 172.21.15.60:0/342910012 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac4c0c0030 con 0x7fac54151b80 2024-04-25T10:09:32.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.049+0000 7fac50ff9640 1 --2- 172.21.15.60:0/342910012 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fac34066f60 0x7fac34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.049+0000 7fac50ff9640 1 -- 172.21.15.60:0/342910012 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac4c11fb50 con 0x7fac54151b80 2024-04-25T10:09:32.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.049+0000 7fac537fe640 1 --2- 172.21.15.60:0/342910012 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fac34066f60 0x7fac34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.049+0000 7fac537fe640 1 --2- 172.21.15.60:0/342910012 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fac34066f60 0x7fac34069420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fac44001060 tx=0x7fac44009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:32.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.053+0000 7fac50ff9640 1 -- 172.21.15.60:0/342910012 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fac4c0c7060 con 0x7fac54151b80 2024-04-25T10:09:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.241+0000 7fac59d4f640 1 -- 172.21.15.60:0/342910012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fac540cba00 con 0x7fac54151b80 2024-04-25T10:09:32.250 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:09:32.241+0000 7f3a792d6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:09:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.241+0000 7fac50ff9640 1 -- 172.21.15.60:0/342910012 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7fac4c016970 con 0x7fac54151b80 2024-04-25T10:09:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:09:32.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.245+0000 7fac59d4f640 1 -- 172.21.15.60:0/342910012 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fac34066f60 msgr2=0x7fac34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.245+0000 7fac59d4f640 1 --2- 172.21.15.60:0/342910012 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fac34066f60 0x7fac34069420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fac44001060 tx=0x7fac44009040 comp rx=0 tx=0).stop 2024-04-25T10:09:32.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.245+0000 7fac59d4f640 1 -- 172.21.15.60:0/342910012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac54151b80 msgr2=0x7fac541f16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.245+0000 7fac59d4f640 1 --2- 172.21.15.60:0/342910012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac54151b80 0x7fac541f16a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fac4c01d500 tx=0x7fac4c009750 comp rx=0 tx=0).stop 2024-04-25T10:09:32.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.245+0000 7fac59d4f640 1 -- 172.21.15.60:0/342910012 shutdown_connections 2024-04-25T10:09:32.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.245+0000 7fac59d4f640 1 --2- 172.21.15.60:0/342910012 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fac34066f60 0x7fac34069420 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.245+0000 7fac59d4f640 1 --2- 172.21.15.60:0/342910012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac54151b80 0x7fac541f16a0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.245+0000 7fac59d4f640 1 --2- 172.21.15.60:0/342910012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac5414fe70 0x7fac5414e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.245+0000 7fac59d4f640 1 --2- 172.21.15.60:0/342910012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac400051d0 0x7fac5414cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.245+0000 7fac59d4f640 1 -- 172.21.15.60:0/342910012 >> 172.21.15.60:0/342910012 conn(0x7fac540b12d0 msgr2=0x7fac540b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:32.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.245+0000 7fac59d4f640 1 -- 172.21.15.60:0/342910012 shutdown_connections 2024-04-25T10:09:32.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.245+0000 7fac59d4f640 1 -- 172.21.15.60:0/342910012 wait complete. 2024-04-25T10:09:32.262 INFO:tasks.ceph.mds.a.smithi060.stderr: -6550> 2024-04-25T10:08:16.846+0000 7f273a33f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:09:32.265 INFO:tasks.ceph.mds.a.smithi060.stderr: -4818> 2024-04-25T10:08:23.982+0000 7f2736337640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6489=0-4194304@1} caps={6489=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6489 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x56343c443b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:09:32.267 INFO:tasks.ceph.mds.a.smithi060.stderr: -2893> 2024-04-25T10:08:55.394+0000 7f2736337640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-25T10:08:22.850813+0000 1=1+0) (iversion lock) caps={6489=pAsLsXsFs/-@2},l=6489(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x56343c443b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:09:32.273 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:09:32.304 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T10:09:32.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.393+0000 7f253b0b9640 1 Processor -- start 2024-04-25T10:09:32.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.393+0000 7f253b0b9640 1 -- start start 2024-04-25T10:09:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.393+0000 7f253b0b9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f25340b52a0 0x7f25340b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.393+0000 7f253b0b9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f25340b3b50 0x7f253415e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.393+0000 7f253b0b9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25341463d0 con 0x7f253406ac10 2024-04-25T10:09:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.393+0000 7f253b0b9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25340b5780 con 0x7f25340b52a0 2024-04-25T10:09:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.393+0000 7f253b0b9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2534146640 con 0x7f25340b3b50 2024-04-25T10:09:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.393+0000 7f2533fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f25340b52a0 0x7f25340b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.393+0000 7f2533fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f25340b52a0 0x7f25340b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37708/0 (socket says 172.21.15.60:37708) 2024-04-25T10:09:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.393+0000 7f2533fff640 1 -- 172.21.15.60:0/3362146839 learned_addr learned my addr 172.21.15.60:0/3362146839 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.393+0000 7f2539656640 1 --2- 172.21.15.60:0/3362146839 >> v2:172.21.15.100:3300/0 conn(0x7f25340b3b50 0x7f253415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.393+0000 7f2533fff640 1 -- 172.21.15.60:0/3362146839 >> v2:172.21.15.100:3300/0 conn(0x7f25340b3b50 msgr2=0x7f253415e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:32.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.393+0000 7f2533fff640 1 --2- 172.21.15.60:0/3362146839 >> v2:172.21.15.100:3300/0 conn(0x7f25340b3b50 0x7f253415e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.393+0000 7f2533fff640 1 -- 172.21.15.60:0/3362146839 >> v1:172.21.15.100:6789/0 conn(0x7f253406ac10 legacy=0x7f253406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:32.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.393+0000 7f2533fff640 1 -- 172.21.15.60:0/3362146839 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2534146920 con 0x7f25340b52a0 2024-04-25T10:09:32.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.397+0000 7f2533fff640 1 --2- 172.21.15.60:0/3362146839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25340b52a0 0x7f25340b3610 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f25340b3320 tx=0x7f252801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7643dacae894e13c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:32.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.397+0000 7f25337fe640 1 -- 172.21.15.60:0/3362146839 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25280a7070 con 0x7f25340b52a0 2024-04-25T10:09:32.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.397+0000 7f25337fe640 1 -- 172.21.15.60:0/3362146839 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25280b0070 con 0x7f25340b52a0 2024-04-25T10:09:32.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.397+0000 7f25337fe640 1 -- 172.21.15.60:0/3362146839 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25280ab890 con 0x7f25340b52a0 2024-04-25T10:09:32.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.397+0000 7f253b0b9640 1 -- 172.21.15.60:0/3362146839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25340b52a0 msgr2=0x7f25340b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.397+0000 7f253b0b9640 1 --2- 172.21.15.60:0/3362146839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25340b52a0 0x7f25340b3610 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f25340b3320 tx=0x7f252801c450 comp rx=0 tx=0).stop 2024-04-25T10:09:32.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.397+0000 7f253b0b9640 1 -- 172.21.15.60:0/3362146839 shutdown_connections 2024-04-25T10:09:32.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.397+0000 7f253b0b9640 1 --2- 172.21.15.60:0/3362146839 >> v2:172.21.15.100:3300/0 conn(0x7f25340b3b50 0x7f253415e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.397+0000 7f253b0b9640 1 --2- 172.21.15.60:0/3362146839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25340b52a0 0x7f25340b3610 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.397+0000 7f253b0b9640 1 -- 172.21.15.60:0/3362146839 >> 172.21.15.60:0/3362146839 conn(0x7f25340728a0 msgr2=0x7f2534099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:32.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.397+0000 7f253b0b9640 1 -- 172.21.15.60:0/3362146839 shutdown_connections 2024-04-25T10:09:32.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.397+0000 7f253b0b9640 1 -- 172.21.15.60:0/3362146839 wait complete. 2024-04-25T10:09:32.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f253b0b9640 1 Processor -- start 2024-04-25T10:09:32.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f253b0b9640 1 -- start start 2024-04-25T10:09:32.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f253b0b9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253406ac10 0x7f2534139d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f253b0b9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25340b3b50 0x7f253415b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f253b0b9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25340b52a0 0x7f253415cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f253b0b9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253416aad0 con 0x7f25340b52a0 2024-04-25T10:09:32.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f253b0b9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253416a950 con 0x7f25340b3b50 2024-04-25T10:09:32.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f253b0b9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253416ac50 con 0x7f253406ac10 2024-04-25T10:09:32.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f2539656640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25340b52a0 0x7f253415cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f2539656640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25340b52a0 0x7f253415cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37710/0 (socket says 172.21.15.60:37710) 2024-04-25T10:09:32.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f2539656640 1 -- 172.21.15.60:0/2181790943 learned_addr learned my addr 172.21.15.60:0/2181790943 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:32.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f2538e55640 1 --2- 172.21.15.60:0/2181790943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253406ac10 0x7f2534139d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f2533fff640 1 --2- 172.21.15.60:0/2181790943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25340b3b50 0x7f253415b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f2539656640 1 -- 172.21.15.60:0/2181790943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253406ac10 msgr2=0x7f2534139d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f2539656640 1 --2- 172.21.15.60:0/2181790943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253406ac10 0x7f2534139d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f2539656640 1 -- 172.21.15.60:0/2181790943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25340b3b50 msgr2=0x7f253415b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:32.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f2539656640 1 --2- 172.21.15.60:0/2181790943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25340b3b50 0x7f253415b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:32.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f2539656640 1 -- 172.21.15.60:0/2181790943 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253415c170 con 0x7f25340b52a0 2024-04-25T10:09:32.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f2539656640 1 --2- 172.21.15.60:0/2181790943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25340b52a0 0x7f253415cee0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f252c00cd70 tx=0x7f252c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:32.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f25317fa640 1 -- 172.21.15.60:0/2181790943 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c003ba0 con 0x7f25340b52a0 2024-04-25T10:09:32.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f25317fa640 1 -- 172.21.15.60:0/2181790943 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c003d40 con 0x7f25340b52a0 2024-04-25T10:09:32.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f253b0b9640 1 -- 172.21.15.60:0/2181790943 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253415c450 con 0x7f25340b52a0 2024-04-25T10:09:32.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f25317fa640 1 -- 172.21.15.60:0/2181790943 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c004240 con 0x7f25340b52a0 2024-04-25T10:09:32.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f253b0b9640 1 -- 172.21.15.60:0/2181790943 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25340cb930 con 0x7f25340b52a0 2024-04-25T10:09:32.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f253b0b9640 1 -- 172.21.15.60:0/2181790943 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2534074770 con 0x7f25340b52a0 2024-04-25T10:09:32.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f25317fa640 1 -- 172.21.15.60:0/2181790943 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f252c0043e0 con 0x7f25340b52a0 2024-04-25T10:09:32.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.401+0000 7f25317fa640 1 --2- 172.21.15.60:0/2181790943 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2520066f60 0x7f2520069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:32.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.405+0000 7f25317fa640 1 -- 172.21.15.60:0/2181790943 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f252c085b60 con 0x7f25340b52a0 2024-04-25T10:09:32.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.405+0000 7f2538e55640 1 --2- 172.21.15.60:0/2181790943 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2520066f60 0x7f2520069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:32.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.405+0000 7f2538e55640 1 --2- 172.21.15.60:0/2181790943 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2520066f60 0x7f2520069420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2524004e30 tx=0x7f2524009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:32.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.405+0000 7f25317fa640 1 -- 172.21.15.60:0/2181790943 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f252c056ba0 con 0x7f25340b52a0 2024-04-25T10:09:32.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:32.605+0000 7f253b0b9640 1 -- 172.21.15.60:0/2181790943 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f25341610a0 con 0x7f25340b52a0 2024-04-25T10:09:33.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.245+0000 7f25317fa640 1 -- 172.21.15.60:0/2181790943 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7f252c0564a0 con 0x7f25340b52a0 2024-04-25T10:09:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.249+0000 7f253b0b9640 1 -- 172.21.15.60:0/2181790943 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2520066f60 msgr2=0x7f2520069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.249+0000 7f253b0b9640 1 --2- 172.21.15.60:0/2181790943 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2520066f60 0x7f2520069420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2524004e30 tx=0x7f2524009040 comp rx=0 tx=0).stop 2024-04-25T10:09:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.249+0000 7f253b0b9640 1 -- 172.21.15.60:0/2181790943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25340b52a0 msgr2=0x7f253415cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.249+0000 7f253b0b9640 1 --2- 172.21.15.60:0/2181790943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25340b52a0 0x7f253415cee0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f252c00cd70 tx=0x7f252c00a660 comp rx=0 tx=0).stop 2024-04-25T10:09:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.249+0000 7f253b0b9640 1 -- 172.21.15.60:0/2181790943 shutdown_connections 2024-04-25T10:09:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.249+0000 7f253b0b9640 1 --2- 172.21.15.60:0/2181790943 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2520066f60 0x7f2520069420 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.249+0000 7f253b0b9640 1 --2- 172.21.15.60:0/2181790943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25340b52a0 0x7f253415cee0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.249+0000 7f253b0b9640 1 --2- 172.21.15.60:0/2181790943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25340b3b50 0x7f253415b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.249+0000 7f253b0b9640 1 --2- 172.21.15.60:0/2181790943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253406ac10 0x7f2534139d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.249+0000 7f253b0b9640 1 -- 172.21.15.60:0/2181790943 >> 172.21.15.60:0/2181790943 conn(0x7f25340728a0 msgr2=0x7f253413e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.249+0000 7f253b0b9640 1 -- 172.21.15.60:0/2181790943 shutdown_connections 2024-04-25T10:09:33.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.249+0000 7f253b0b9640 1 -- 172.21.15.60:0/2181790943 wait complete. 2024-04-25T10:09:33.283 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:09:33.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.457+0000 7f973a387640 1 Processor -- start 2024-04-25T10:09:33.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.457+0000 7f973a387640 1 -- start start 2024-04-25T10:09:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.457+0000 7f973a387640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97340b0b30 0x7f9734099e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.457+0000 7f973a387640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97340cfa10 0x7f97340cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.457+0000 7f973a387640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973409a350 con 0x7f9734099a30 2024-04-25T10:09:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.457+0000 7f973a387640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f973415ef50 con 0x7f97340b0b30 2024-04-25T10:09:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.457+0000 7f973a387640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973406bec0 con 0x7f97340cfa10 2024-04-25T10:09:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.457+0000 7f972bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97340b0b30 0x7f9734099e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.457+0000 7f9733fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9734099a30 0x7f9734161fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44642/0 (socket says 172.21.15.60:44642) 2024-04-25T10:09:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.457+0000 7f9733fff640 1 -- 172.21.15.60:0/4099435943 learned_addr learned my addr 172.21.15.60:0/4099435943 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.457+0000 7f9738924640 1 --2- 172.21.15.60:0/4099435943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97340cfa10 0x7f97340cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.457+0000 7f972bfff640 1 -- 172.21.15.60:0/4099435943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97340cfa10 msgr2=0x7f97340cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.457+0000 7f972bfff640 1 --2- 172.21.15.60:0/4099435943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97340cfa10 0x7f97340cfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.457+0000 7f97337fe640 1 -- 172.21.15.60:0/4099435943 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2070056142 0 0) 0x7f973409a350 con 0x7f9734099a30 2024-04-25T10:09:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.457+0000 7f972bfff640 1 -- 172.21.15.60:0/4099435943 >> v1:172.21.15.60:6789/0 conn(0x7f9734099a30 legacy=0x7f9734161fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.457+0000 7f972bfff640 1 -- 172.21.15.60:0/4099435943 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973406c1a0 con 0x7f97340b0b30 2024-04-25T10:09:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f972bfff640 1 --2- 172.21.15.60:0/4099435943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97340b0b30 0x7f9734099e10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f971c0097c0 tx=0x7f971c01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83ea8155225b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:33.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f97337fe640 1 -- 172.21.15.60:0/4099435943 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c0b1070 con 0x7f97340b0b30 2024-04-25T10:09:33.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f97337fe640 1 -- 172.21.15.60:0/4099435943 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971c0ac410 con 0x7f97340b0b30 2024-04-25T10:09:33.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f97337fe640 1 -- 172.21.15.60:0/4099435943 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c0ac7e0 con 0x7f97340b0b30 2024-04-25T10:09:33.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f973a387640 1 -- 172.21.15.60:0/4099435943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97340b0b30 msgr2=0x7f9734099e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:33.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f973a387640 1 --2- 172.21.15.60:0/4099435943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97340b0b30 0x7f9734099e10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f971c0097c0 tx=0x7f971c01c3b0 comp rx=0 tx=0).stop 2024-04-25T10:09:33.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f973a387640 1 -- 172.21.15.60:0/4099435943 shutdown_connections 2024-04-25T10:09:33.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f973a387640 1 --2- 172.21.15.60:0/4099435943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97340cfa10 0x7f97340cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:33.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f973a387640 1 --2- 172.21.15.60:0/4099435943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97340b0b30 0x7f9734099e10 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:33.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f973a387640 1 -- 172.21.15.60:0/4099435943 >> 172.21.15.60:0/4099435943 conn(0x7f97340d17d0 msgr2=0x7f97340d1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:33.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f973a387640 1 -- 172.21.15.60:0/4099435943 shutdown_connections 2024-04-25T10:09:33.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f973a387640 1 -- 172.21.15.60:0/4099435943 wait complete. 2024-04-25T10:09:33.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f973a387640 1 Processor -- start 2024-04-25T10:09:33.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f973a387640 1 -- start start 2024-04-25T10:09:33.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f973a387640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9734099a30 0x7f97341567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:33.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f973a387640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97340b0b30 0x7f9734158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:33.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f973a387640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97340cfa10 0x7f973415c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:33.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f973a387640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973416a970 con 0x7f9734099a30 2024-04-25T10:09:33.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f973a387640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973406cf60 con 0x7f97340b0b30 2024-04-25T10:09:33.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f973a387640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973416aaf0 con 0x7f97340cfa10 2024-04-25T10:09:33.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f9733fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9734099a30 0x7f97341567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:33.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f9733fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9734099a30 0x7f97341567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37726/0 (socket says 172.21.15.60:37726) 2024-04-25T10:09:33.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f9733fff640 1 -- 172.21.15.60:0/513630025 learned_addr learned my addr 172.21.15.60:0/513630025 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:33.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f972bfff640 1 --2- 172.21.15.60:0/513630025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97340b0b30 0x7f9734158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:33.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f9738924640 1 --2- 172.21.15.60:0/513630025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97340cfa10 0x7f973415c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:33.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f9733fff640 1 -- 172.21.15.60:0/513630025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97340cfa10 msgr2=0x7f973415c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:33.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f9733fff640 1 --2- 172.21.15.60:0/513630025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97340cfa10 0x7f973415c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f9733fff640 1 -- 172.21.15.60:0/513630025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97340b0b30 msgr2=0x7f9734158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f9733fff640 1 --2- 172.21.15.60:0/513630025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97340b0b30 0x7f9734158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.461+0000 7f9733fff640 1 -- 172.21.15.60:0/513630025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9734153030 con 0x7f9734099a30 2024-04-25T10:09:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.465+0000 7f9733fff640 1 --2- 172.21.15.60:0/513630025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9734099a30 0x7f97341567e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f972400c520 tx=0x7f972400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.465+0000 7f97317fa640 1 -- 172.21.15.60:0/513630025 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9724018070 con 0x7f9734099a30 2024-04-25T10:09:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.465+0000 7f97317fa640 1 -- 172.21.15.60:0/513630025 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97240074e0 con 0x7f9734099a30 2024-04-25T10:09:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.465+0000 7f97317fa640 1 -- 172.21.15.60:0/513630025 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97240049f0 con 0x7f9734099a30 2024-04-25T10:09:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.465+0000 7f973a387640 1 -- 172.21.15.60:0/513630025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9734152d50 con 0x7f9734099a30 2024-04-25T10:09:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.465+0000 7f973a387640 1 -- 172.21.15.60:0/513630025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973415b9a0 con 0x7f9734099a30 2024-04-25T10:09:33.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.465+0000 7f973a387640 1 -- 172.21.15.60:0/513630025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9734074770 con 0x7f9734099a30 2024-04-25T10:09:33.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.465+0000 7f97317fa640 1 -- 172.21.15.60:0/513630025 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f972401b070 con 0x7f9734099a30 2024-04-25T10:09:33.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.465+0000 7f97317fa640 1 --2- 172.21.15.60:0/513630025 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9714066f60 0x7f9714069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:33.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.465+0000 7f97317fa640 1 -- 172.21.15.60:0/513630025 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9724084540 con 0x7f9734099a30 2024-04-25T10:09:33.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.465+0000 7f972bfff640 1 --2- 172.21.15.60:0/513630025 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9714066f60 0x7f9714069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:33.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.465+0000 7f972bfff640 1 --2- 172.21.15.60:0/513630025 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9714066f60 0x7f9714069420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f971c0ae000 tx=0x7f971c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:33.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.469+0000 7f97317fa640 1 -- 172.21.15.60:0/513630025 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f97240554e0 con 0x7f9734099a30 2024-04-25T10:09:33.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:33.625+0000 7f973a387640 1 -- 172.21.15.60:0/513630025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f97340d2600 con 0x7f9734099a30 2024-04-25T10:09:34.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.253+0000 7f97317fa640 1 -- 172.21.15.60:0/513630025 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9724054de0 con 0x7f9734099a30 2024-04-25T10:09:34.264 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:09:34.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.257+0000 7f973a387640 1 -- 172.21.15.60:0/513630025 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9714066f60 msgr2=0x7f9714069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:34.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.257+0000 7f973a387640 1 --2- 172.21.15.60:0/513630025 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9714066f60 0x7f9714069420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f971c0ae000 tx=0x7f971c008040 comp rx=0 tx=0).stop 2024-04-25T10:09:34.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.257+0000 7f973a387640 1 -- 172.21.15.60:0/513630025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9734099a30 msgr2=0x7f97341567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:34.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.257+0000 7f973a387640 1 --2- 172.21.15.60:0/513630025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9734099a30 0x7f97341567e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f972400c520 tx=0x7f972400c9e0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.261+0000 7f973a387640 1 -- 172.21.15.60:0/513630025 shutdown_connections 2024-04-25T10:09:34.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.261+0000 7f973a387640 1 --2- 172.21.15.60:0/513630025 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9714066f60 0x7f9714069420 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.261+0000 7f973a387640 1 --2- 172.21.15.60:0/513630025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97340cfa10 0x7f973415c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.261+0000 7f973a387640 1 --2- 172.21.15.60:0/513630025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97340b0b30 0x7f9734158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.261+0000 7f973a387640 1 --2- 172.21.15.60:0/513630025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9734099a30 0x7f97341567e0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.261+0000 7f973a387640 1 -- 172.21.15.60:0/513630025 >> 172.21.15.60:0/513630025 conn(0x7f97340d17d0 msgr2=0x7f97340b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:34.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.261+0000 7f973a387640 1 -- 172.21.15.60:0/513630025 shutdown_connections 2024-04-25T10:09:34.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.261+0000 7f973a387640 1 -- 172.21.15.60:0/513630025 wait complete. 2024-04-25T10:09:34.290 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:09:34.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.413+0000 7fa58af04640 1 Processor -- start 2024-04-25T10:09:34.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.413+0000 7fa58af04640 1 -- start start 2024-04-25T10:09:34.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa58af04640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa5840b3720 0x7fa5840cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:34.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa58af04640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa584162490 0x7fa58415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:34.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa58af04640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa584162d50 con 0x7fa5840b3340 2024-04-25T10:09:34.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa58af04640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa584163030 con 0x7fa5840b3720 2024-04-25T10:09:34.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa58af04640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa584165b20 con 0x7fa584162490 2024-04-25T10:09:34.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa583fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa5840b3720 0x7fa5840cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:34.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa5894a1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa584162490 0x7fa58415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:34.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa583fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa5840b3720 0x7fa5840cfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:54426/0 (socket says 172.21.15.60:54426) 2024-04-25T10:09:34.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa5894a1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa584162490 0x7fa58415bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:40246/0 (socket says 172.21.15.60:40246) 2024-04-25T10:09:34.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa583fff640 1 -- 172.21.15.60:0/1678441308 learned_addr learned my addr 172.21.15.60:0/1678441308 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:34.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa588ca0640 1 --1- 172.21.15.60:0/1678441308 >> v1:172.21.15.100:6789/0 conn(0x7fa5840b3340 0x7fa584166320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:52628/0 (socket says 172.21.15.60:52628) 2024-04-25T10:09:34.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa5894a1640 1 -- 172.21.15.60:0/1678441308 >> v2:172.21.15.100:3300/0 conn(0x7fa5840b3720 msgr2=0x7fa5840cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:34.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa5894a1640 1 --2- 172.21.15.60:0/1678441308 >> v2:172.21.15.100:3300/0 conn(0x7fa5840b3720 0x7fa5840cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa5894a1640 1 -- 172.21.15.60:0/1678441308 >> v1:172.21.15.100:6789/0 conn(0x7fa5840b3340 legacy=0x7fa584166320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:34.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa5894a1640 1 -- 172.21.15.60:0/1678441308 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa584165e00 con 0x7fa584162490 2024-04-25T10:09:34.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa583fff640 1 --2- 172.21.15.60:0/1678441308 >> v2:172.21.15.100:3300/0 conn(0x7fa5840b3720 0x7fa5840cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:34.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.417+0000 7fa5894a1640 1 --2- 172.21.15.60:0/1678441308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa584162490 0x7fa58415bf90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009d90 tx=0x7fa57c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62efffaa693592de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:34.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa5837fe640 1 -- 172.21.15.60:0/1678441308 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0b2070 con 0x7fa584162490 2024-04-25T10:09:34.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa5837fe640 1 -- 172.21.15.60:0/1678441308 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57c0a58e0 con 0x7fa584162490 2024-04-25T10:09:34.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa5837fe640 1 -- 172.21.15.60:0/1678441308 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0a5cb0 con 0x7fa584162490 2024-04-25T10:09:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa58af04640 1 -- 172.21.15.60:0/1678441308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa584162490 msgr2=0x7fa58415bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa58af04640 1 --2- 172.21.15.60:0/1678441308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa584162490 0x7fa58415bf90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009d90 tx=0x7fa57c01c970 comp rx=0 tx=0).stop 2024-04-25T10:09:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa58af04640 1 -- 172.21.15.60:0/1678441308 shutdown_connections 2024-04-25T10:09:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa58af04640 1 --2- 172.21.15.60:0/1678441308 >> v2:172.21.15.100:3300/0 conn(0x7fa5840b3720 0x7fa5840cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa58af04640 1 --2- 172.21.15.60:0/1678441308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa584162490 0x7fa58415bf90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa58af04640 1 -- 172.21.15.60:0/1678441308 >> 172.21.15.60:0/1678441308 conn(0x7fa584066fa0 msgr2=0x7fa58406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa58af04640 1 -- 172.21.15.60:0/1678441308 shutdown_connections 2024-04-25T10:09:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa58af04640 1 -- 172.21.15.60:0/1678441308 wait complete. 2024-04-25T10:09:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa58af04640 1 Processor -- start 2024-04-25T10:09:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa58af04640 1 -- start start 2024-04-25T10:09:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa58af04640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5840b3340 0x7fa58414e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa58af04640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5840b3720 0x7fa58415e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa58af04640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa584162490 0x7fa58415e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa58af04640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58406e520 con 0x7fa5840b3340 2024-04-25T10:09:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa58af04640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa584162b30 con 0x7fa5840b3720 2024-04-25T10:09:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa58af04640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58406e6a0 con 0x7fa584162490 2024-04-25T10:09:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa588ca0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5840b3340 0x7fa58414e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa588ca0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5840b3340 0x7fa58414e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37732/0 (socket says 172.21.15.60:37732) 2024-04-25T10:09:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa588ca0640 1 -- 172.21.15.60:0/2158509820 learned_addr learned my addr 172.21.15.60:0/2158509820 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa583fff640 1 --2- 172.21.15.60:0/2158509820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5840b3720 0x7fa58415e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.421+0000 7fa5894a1640 1 --2- 172.21.15.60:0/2158509820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa584162490 0x7fa58415e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa588ca0640 1 -- 172.21.15.60:0/2158509820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa584162490 msgr2=0x7fa58415e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa588ca0640 1 --2- 172.21.15.60:0/2158509820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa584162490 0x7fa58415e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa588ca0640 1 -- 172.21.15.60:0/2158509820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5840b3720 msgr2=0x7fa58415e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa588ca0640 1 --2- 172.21.15.60:0/2158509820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5840b3720 0x7fa58415e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa588ca0640 1 -- 172.21.15.60:0/2158509820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5840cc4a0 con 0x7fa5840b3340 2024-04-25T10:09:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa583fff640 1 --2- 172.21.15.60:0/2158509820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5840b3720 0x7fa58415e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa588ca0640 1 --2- 172.21.15.60:0/2158509820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5840b3340 0x7fa58414e790 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa57400c370 tx=0x7fa57400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa5817fa640 1 -- 172.21.15.60:0/2158509820 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5740074e0 con 0x7fa5840b3340 2024-04-25T10:09:34.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa5817fa640 1 -- 172.21.15.60:0/2158509820 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa574007680 con 0x7fa5840b3340 2024-04-25T10:09:34.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa58af04640 1 -- 172.21.15.60:0/2158509820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5840cc780 con 0x7fa5840b3340 2024-04-25T10:09:34.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa5817fa640 1 -- 172.21.15.60:0/2158509820 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa574004a30 con 0x7fa5840b3340 2024-04-25T10:09:34.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa58af04640 1 -- 172.21.15.60:0/2158509820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5840cd260 con 0x7fa5840b3340 2024-04-25T10:09:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa58af04640 1 -- 172.21.15.60:0/2158509820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa584074770 con 0x7fa5840b3340 2024-04-25T10:09:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa5817fa640 1 -- 172.21.15.60:0/2158509820 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa57401a020 con 0x7fa5840b3340 2024-04-25T10:09:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa5817fa640 1 --2- 172.21.15.60:0/2158509820 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 0x7fa564069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa5817fa640 1 -- 172.21.15.60:0/2158509820 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fa574084710 con 0x7fa5840b3340 2024-04-25T10:09:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa583fff640 1 --2- 172.21.15.60:0/2158509820 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 0x7fa564069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.425+0000 7fa583fff640 1 --2- 172.21.15.60:0/2158509820 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 0x7fa564069420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa5840d0110 tx=0x7fa56c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:34.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.429+0000 7fa5817fa640 1 -- 172.21.15.60:0/2158509820 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5740557e0 con 0x7fa5840b3340 2024-04-25T10:09:34.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:34.585+0000 7fa58af04640 1 -- 172.21.15.60:0/2158509820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa58406d820 con 0x7fa5840b3340 2024-04-25T10:09:35.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.265+0000 7fa5817fa640 1 -- 172.21.15.60:0/2158509820 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa5740550e0 con 0x7fa5840b3340 2024-04-25T10:09:35.274 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T10:09:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.269+0000 7fa58af04640 1 -- 172.21.15.60:0/2158509820 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 msgr2=0x7fa564069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.269+0000 7fa58af04640 1 --2- 172.21.15.60:0/2158509820 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 0x7fa564069420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa5840d0110 tx=0x7fa56c008040 comp rx=0 tx=0).stop 2024-04-25T10:09:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.269+0000 7fa58af04640 1 -- 172.21.15.60:0/2158509820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5840b3340 msgr2=0x7fa58414e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.269+0000 7fa58af04640 1 --2- 172.21.15.60:0/2158509820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5840b3340 0x7fa58414e790 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa57400c370 tx=0x7fa57400c830 comp rx=0 tx=0).stop 2024-04-25T10:09:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.269+0000 7fa58af04640 1 -- 172.21.15.60:0/2158509820 shutdown_connections 2024-04-25T10:09:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.269+0000 7fa58af04640 1 --2- 172.21.15.60:0/2158509820 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 0x7fa564069420 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.269+0000 7fa58af04640 1 --2- 172.21.15.60:0/2158509820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa584162490 0x7fa58415e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.269+0000 7fa58af04640 1 --2- 172.21.15.60:0/2158509820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5840b3720 0x7fa58415e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.269+0000 7fa58af04640 1 --2- 172.21.15.60:0/2158509820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5840b3340 0x7fa58414e790 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.269+0000 7fa58af04640 1 -- 172.21.15.60:0/2158509820 >> 172.21.15.60:0/2158509820 conn(0x7fa584066fa0 msgr2=0x7fa5840b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.269+0000 7fa58af04640 1 -- 172.21.15.60:0/2158509820 shutdown_connections 2024-04-25T10:09:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.269+0000 7fa58af04640 1 -- 172.21.15.60:0/2158509820 wait complete. 2024-04-25T10:09:35.315 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:09:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.437+0000 7fbde3f81640 1 Processor -- start 2024-04-25T10:09:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.437+0000 7fbde3f81640 1 -- start start 2024-04-25T10:09:35.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde3f81640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbddc15b6f0 0x7fbddc1620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:35.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde3f81640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbddc15e0d0 0x7fbddc15e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:35.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde3f81640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbddc15bad0 con 0x7fbddc0ce7e0 2024-04-25T10:09:35.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde3f81640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbddc162610 con 0x7fbddc15b6f0 2024-04-25T10:09:35.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde3f81640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbddc161c40 con 0x7fbddc15e0d0 2024-04-25T10:09:35.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde1d1d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbddc15b6f0 0x7fbddc1620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:35.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde1d1d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbddc15b6f0 0x7fbddc1620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37736/0 (socket says 172.21.15.60:37736) 2024-04-25T10:09:35.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde1d1d640 1 -- 172.21.15.60:0/435559717 learned_addr learned my addr 172.21.15.60:0/435559717 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:35.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde151c640 1 --2- 172.21.15.60:0/435559717 >> v2:172.21.15.100:3300/0 conn(0x7fbddc15e0d0 0x7fbddc15e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:35.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde1d1d640 1 -- 172.21.15.60:0/435559717 >> v2:172.21.15.100:3300/0 conn(0x7fbddc15e0d0 msgr2=0x7fbddc15e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:35.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde1d1d640 1 --2- 172.21.15.60:0/435559717 >> v2:172.21.15.100:3300/0 conn(0x7fbddc15e0d0 0x7fbddc15e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde1d1d640 1 -- 172.21.15.60:0/435559717 >> v1:172.21.15.60:6789/0 conn(0x7fbddc0ce7e0 legacy=0x7fbddc0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:35.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde1d1d640 1 -- 172.21.15.60:0/435559717 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc15ebe0 con 0x7fbddc15b6f0 2024-04-25T10:09:35.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde151c640 1 --2- 172.21.15.60:0/435559717 >> v2:172.21.15.100:3300/0 conn(0x7fbddc15e0d0 0x7fbddc15e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde1d1d640 1 --2- 172.21.15.60:0/435559717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc15b6f0 0x7fbddc1620d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0b6600 tx=0x7fbdd801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fa7fb773c5836b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde0d1b640 1 -- 172.21.15.60:0/435559717 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80ab610 con 0x7fbddc15b6f0 2024-04-25T10:09:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde0d1b640 1 -- 172.21.15.60:0/435559717 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd80b0070 con 0x7fbddc15b6f0 2024-04-25T10:09:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.441+0000 7fbde0d1b640 1 -- 172.21.15.60:0/435559717 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80ab9d0 con 0x7fbddc15b6f0 2024-04-25T10:09:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 -- 172.21.15.60:0/435559717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc15b6f0 msgr2=0x7fbddc1620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 --2- 172.21.15.60:0/435559717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc15b6f0 0x7fbddc1620d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0b6600 tx=0x7fbdd801c520 comp rx=0 tx=0).stop 2024-04-25T10:09:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 -- 172.21.15.60:0/435559717 shutdown_connections 2024-04-25T10:09:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 --2- 172.21.15.60:0/435559717 >> v2:172.21.15.100:3300/0 conn(0x7fbddc15e0d0 0x7fbddc15e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 --2- 172.21.15.60:0/435559717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc15b6f0 0x7fbddc1620d0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 -- 172.21.15.60:0/435559717 >> 172.21.15.60:0/435559717 conn(0x7fbddc072340 msgr2=0x7fbddc072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 -- 172.21.15.60:0/435559717 shutdown_connections 2024-04-25T10:09:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 -- 172.21.15.60:0/435559717 wait complete. 2024-04-25T10:09:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 Processor -- start 2024-04-25T10:09:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 -- start start 2024-04-25T10:09:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc0ce7e0 0x7fbddc0cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbddc15b6f0 0x7fbddc0b8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbddc15e0d0 0x7fbddc0cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc0b96d0 con 0x7fbddc0ce7e0 2024-04-25T10:09:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc0b9550 con 0x7fbddc15b6f0 2024-04-25T10:09:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc0cf000 con 0x7fbddc15e0d0 2024-04-25T10:09:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde1d1d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc0ce7e0 0x7fbddc0cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde1d1d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc0ce7e0 0x7fbddc0cb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37738/0 (socket says 172.21.15.60:37738) 2024-04-25T10:09:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde1d1d640 1 -- 172.21.15.60:0/3169886237 learned_addr learned my addr 172.21.15.60:0/3169886237 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde151c640 1 --2- 172.21.15.60:0/3169886237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbddc15b6f0 0x7fbddc0b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde1d1d640 1 -- 172.21.15.60:0/3169886237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbddc15e0d0 msgr2=0x7fbddc0cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde1d1d640 1 --2- 172.21.15.60:0/3169886237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbddc15e0d0 0x7fbddc0cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde1d1d640 1 -- 172.21.15.60:0/3169886237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbddc15b6f0 msgr2=0x7fbddc0b8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde1d1d640 1 --2- 172.21.15.60:0/3169886237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbddc15b6f0 0x7fbddc0b8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde1d1d640 1 -- 172.21.15.60:0/3169886237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc0cd890 con 0x7fbddc0ce7e0 2024-04-25T10:09:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde1d1d640 1 --2- 172.21.15.60:0/3169886237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc0ce7e0 0x7fbddc0cb070 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80ad040 tx=0x7fbdd801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbdd27fc640 1 -- 172.21.15.60:0/3169886237 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80abe60 con 0x7fbddc0ce7e0 2024-04-25T10:09:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbdd27fc640 1 -- 172.21.15.60:0/3169886237 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd80b4800 con 0x7fbddc0ce7e0 2024-04-25T10:09:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbdd27fc640 1 -- 172.21.15.60:0/3169886237 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80b4bf0 con 0x7fbddc0ce7e0 2024-04-25T10:09:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 -- 172.21.15.60:0/3169886237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbddc0cb5b0 con 0x7fbddc0ce7e0 2024-04-25T10:09:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.445+0000 7fbde3f81640 1 -- 172.21.15.60:0/3169886237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbddc0ccdb0 con 0x7fbddc0ce7e0 2024-04-25T10:09:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.449+0000 7fbde3f81640 1 -- 172.21.15.60:0/3169886237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbddc074770 con 0x7fbddc0ce7e0 2024-04-25T10:09:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.449+0000 7fbdd27fc640 1 -- 172.21.15.60:0/3169886237 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbdd80c7390 con 0x7fbddc0ce7e0 2024-04-25T10:09:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.449+0000 7fbdd27fc640 1 --2- 172.21.15.60:0/3169886237 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbdc0066f90 0x7fbdc0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.449+0000 7fbdd27fc640 1 -- 172.21.15.60:0/3169886237 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbdd811f450 con 0x7fbddc0ce7e0 2024-04-25T10:09:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.449+0000 7fbde151c640 1 --2- 172.21.15.60:0/3169886237 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbdc0066f90 0x7fbdc0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.449+0000 7fbde151c640 1 --2- 172.21.15.60:0/3169886237 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbdc0066f90 0x7fbdc0069450 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc006fd0 tx=0x7fbdcc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.453+0000 7fbdd27fc640 1 -- 172.21.15.60:0/3169886237 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbdd80c4060 con 0x7fbddc0ce7e0 2024-04-25T10:09:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.625+0000 7fbde3f81640 1 -- 172.21.15.60:0/3169886237 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbddc0d16e0 con 0x7fbddc0ce7e0 2024-04-25T10:09:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.629+0000 7fbdd27fc640 1 -- 172.21.15.60:0/3169886237 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbdd80a8050 con 0x7fbddc0ce7e0 2024-04-25T10:09:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.633+0000 7fbde3f81640 1 -- 172.21.15.60:0/3169886237 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbdc0066f90 msgr2=0x7fbdc0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.633+0000 7fbde3f81640 1 --2- 172.21.15.60:0/3169886237 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbdc0066f90 0x7fbdc0069450 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc006fd0 tx=0x7fbdcc008040 comp rx=0 tx=0).stop 2024-04-25T10:09:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.633+0000 7fbde3f81640 1 -- 172.21.15.60:0/3169886237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc0ce7e0 msgr2=0x7fbddc0cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.633+0000 7fbde3f81640 1 --2- 172.21.15.60:0/3169886237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc0ce7e0 0x7fbddc0cb070 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80ad040 tx=0x7fbdd801ad60 comp rx=0 tx=0).stop 2024-04-25T10:09:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.633+0000 7fbde3f81640 1 -- 172.21.15.60:0/3169886237 shutdown_connections 2024-04-25T10:09:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.633+0000 7fbde3f81640 1 --2- 172.21.15.60:0/3169886237 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbdc0066f90 0x7fbdc0069450 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.633+0000 7fbde3f81640 1 --2- 172.21.15.60:0/3169886237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbddc15e0d0 0x7fbddc0cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.633+0000 7fbde3f81640 1 --2- 172.21.15.60:0/3169886237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbddc15b6f0 0x7fbddc0b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.633+0000 7fbde3f81640 1 --2- 172.21.15.60:0/3169886237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc0ce7e0 0x7fbddc0cb070 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.633+0000 7fbde3f81640 1 -- 172.21.15.60:0/3169886237 >> 172.21.15.60:0/3169886237 conn(0x7fbddc072340 msgr2=0x7fbddc0c6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:35.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.633+0000 7fbde3f81640 1 -- 172.21.15.60:0/3169886237 shutdown_connections 2024-04-25T10:09:35.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.633+0000 7fbde3f81640 1 -- 172.21.15.60:0/3169886237 wait complete. 2024-04-25T10:09:35.659 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-25T10:09:35.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea60865640 1 Processor -- start 2024-04-25T10:09:35.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea60865640 1 -- start start 2024-04-25T10:09:35.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea60865640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fea5c06ac10 0x7fea5c0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:35.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea60865640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5c0ce080 con 0x7fea5c0cf120 2024-04-25T10:09:35.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea60865640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5c0ce360 con 0x7fea5c0ad580 2024-04-25T10:09:35.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea60865640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea5c0ce640 con 0x7fea5c06ac10 2024-04-25T10:09:35.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5ad9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fea5c0cf120 0x7fea5c0cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44668/0 (socket says 172.21.15.60:44668) 2024-04-25T10:09:35.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5ad9d640 1 -- 172.21.15.60:0/896096425 learned_addr learned my addr 172.21.15.60:0/896096425 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:35.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5a59c640 1 --2- 172.21.15.60:0/896096425 >> v2:172.21.15.100:3300/0 conn(0x7fea5c06ac10 0x7fea5c0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:35.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5959a640 1 -- 172.21.15.60:0/896096425 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993143930 0 0) 0x7fea5c0ce080 con 0x7fea5c0cf120 2024-04-25T10:09:35.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5959a640 1 -- 172.21.15.60:0/896096425 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea44007940 con 0x7fea5c0cf120 2024-04-25T10:09:35.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5959a640 1 -- 172.21.15.60:0/896096425 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 84899934 0 0) 0x7fea44007940 con 0x7fea5c0cf120 2024-04-25T10:09:35.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5959a640 1 -- 172.21.15.60:0/896096425 >> v2:172.21.15.100:3300/0 conn(0x7fea5c06ac10 msgr2=0x7fea5c0cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:35.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5959a640 1 --2- 172.21.15.60:0/896096425 >> v2:172.21.15.100:3300/0 conn(0x7fea5c06ac10 0x7fea5c0cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5959a640 1 -- 172.21.15.60:0/896096425 >> v1:172.21.15.100:6789/0 conn(0x7fea5c0ad580 legacy=0x7fea5c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:35.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5959a640 1 -- 172.21.15.60:0/896096425 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5c0ce920 con 0x7fea5c0cf120 2024-04-25T10:09:35.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5959a640 1 -- 172.21.15.60:0/896096425 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fea54001680 con 0x7fea5c0cf120 2024-04-25T10:09:35.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5959a640 1 -- 172.21.15.60:0/896096425 >> v1:172.21.15.60:6789/0 conn(0x7fea5c0cf120 legacy=0x7fea5c0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:35.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5959a640 1 --2- 172.21.15.60:0/896096425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea44007dd0 0x7fea440081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:35.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5959a640 1 --2- 172.21.15.60:0/896096425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea44008c20 0x7fea440090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:35.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5959a640 1 --2- 172.21.15.60:0/896096425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea440095e0 0x7fea4400fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:35.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5959a640 1 -- 172.21.15.60:0/896096425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c0ce640 con 0x7fea44008c20 2024-04-25T10:09:35.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5959a640 1 -- 172.21.15.60:0/896096425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea44010480 con 0x7fea440095e0 2024-04-25T10:09:35.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5959a640 1 -- 172.21.15.60:0/896096425 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea44010680 con 0x7fea44007dd0 2024-04-25T10:09:35.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea59d9b640 1 --2- 172.21.15.60:0/896096425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea44008c20 0x7fea440090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:35.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5a59c640 1 --2- 172.21.15.60:0/896096425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea44007dd0 0x7fea440081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:35.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.777+0000 7fea5ad9d640 1 --2- 172.21.15.60:0/896096425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea440095e0 0x7fea4400fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:35.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea59d9b640 1 -- 172.21.15.60:0/896096425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea44007dd0 msgr2=0x7fea440081f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:35.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea59d9b640 1 --2- 172.21.15.60:0/896096425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea44007dd0 0x7fea440081f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea59d9b640 1 -- 172.21.15.60:0/896096425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea440095e0 msgr2=0x7fea4400fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:35.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea59d9b640 1 --2- 172.21.15.60:0/896096425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea440095e0 0x7fea4400fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea59d9b640 1 -- 172.21.15.60:0/896096425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea44007940 con 0x7fea44008c20 2024-04-25T10:09:35.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea5ad9d640 1 -- 172.21.15.60:0/896096425 reap_dead start 2024-04-25T10:09:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea59d9b640 1 --2- 172.21.15.60:0/896096425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea44008c20 0x7fea440090a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fea50001e40 tx=0x7fea5001c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5b035eef0af300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea5959a640 1 -- 172.21.15.60:0/896096425 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea50001a90 con 0x7fea44008c20 2024-04-25T10:09:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea5959a640 1 -- 172.21.15.60:0/896096425 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea50003b80 con 0x7fea44008c20 2024-04-25T10:09:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea5959a640 1 -- 172.21.15.60:0/896096425 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea50005080 con 0x7fea44008c20 2024-04-25T10:09:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea60865640 1 -- 172.21.15.60:0/896096425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea44008c20 msgr2=0x7fea440090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea60865640 1 --2- 172.21.15.60:0/896096425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea44008c20 0x7fea440090a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fea50001e40 tx=0x7fea5001c740 comp rx=0 tx=0).stop 2024-04-25T10:09:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea60865640 1 -- 172.21.15.60:0/896096425 shutdown_connections 2024-04-25T10:09:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea60865640 1 --2- 172.21.15.60:0/896096425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea44008c20 0x7fea440090a0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea60865640 1 -- 172.21.15.60:0/896096425 >> 172.21.15.60:0/896096425 conn(0x7fea5c072920 msgr2=0x7fea5c099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea60865640 1 -- 172.21.15.60:0/896096425 shutdown_connections 2024-04-25T10:09:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.781+0000 7fea60865640 1 -- 172.21.15.60:0/896096425 wait complete. 2024-04-25T10:09:35.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea60865640 1 Processor -- start 2024-04-25T10:09:35.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea60865640 1 -- start start 2024-04-25T10:09:35.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea60865640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea44008c20 0x7fea5c159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:35.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea60865640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea5c09a7e0 0x7fea5c1f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:35.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea60865640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea5c09de40 0x7fea5c09e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:35.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea60865640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c0aedd0 con 0x7fea5c09de40 2024-04-25T10:09:35.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea60865640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c06b960 con 0x7fea44008c20 2024-04-25T10:09:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea60865640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c0aef50 con 0x7fea5c09a7e0 2024-04-25T10:09:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea5ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea5c09de40 0x7fea5c09e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea5ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea5c09de40 0x7fea5c09e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37758/0 (socket says 172.21.15.60:37758) 2024-04-25T10:09:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea5ad9d640 1 -- 172.21.15.60:0/2473720386 learned_addr learned my addr 172.21.15.60:0/2473720386 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea59d9b640 1 --2- 172.21.15.60:0/2473720386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea5c09a7e0 0x7fea5c1f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea5a59c640 1 --2- 172.21.15.60:0/2473720386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea44008c20 0x7fea5c159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea5ad9d640 1 -- 172.21.15.60:0/2473720386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea5c09a7e0 msgr2=0x7fea5c1f2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea5ad9d640 1 --2- 172.21.15.60:0/2473720386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea5c09a7e0 0x7fea5c1f2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea5ad9d640 1 -- 172.21.15.60:0/2473720386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea44008c20 msgr2=0x7fea5c159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea5ad9d640 1 --2- 172.21.15.60:0/2473720386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea44008c20 0x7fea5c159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea5ad9d640 1 -- 172.21.15.60:0/2473720386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5c1f3920 con 0x7fea5c09de40 2024-04-25T10:09:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea5a59c640 1 --2- 172.21.15.60:0/2473720386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea44008c20 0x7fea5c159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea59d9b640 1 --2- 172.21.15.60:0/2473720386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea5c09a7e0 0x7fea5c1f2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea5ad9d640 1 --2- 172.21.15.60:0/2473720386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea5c09de40 0x7fea5c09e2d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fea54001ec0 tx=0x7fea54011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:35.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea4affd640 1 -- 172.21.15.60:0/2473720386 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54010040 con 0x7fea5c09de40 2024-04-25T10:09:35.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea4affd640 1 -- 172.21.15.60:0/2473720386 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5401a070 con 0x7fea5c09de40 2024-04-25T10:09:35.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea4affd640 1 -- 172.21.15.60:0/2473720386 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5400fae0 con 0x7fea5c09de40 2024-04-25T10:09:35.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea60865640 1 -- 172.21.15.60:0/2473720386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5c1f3360 con 0x7fea5c09de40 2024-04-25T10:09:35.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea60865640 1 -- 172.21.15.60:0/2473720386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5c09d640 con 0x7fea5c09de40 2024-04-25T10:09:35.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.785+0000 7fea60865640 1 -- 172.21.15.60:0/2473720386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea5c074770 con 0x7fea5c09de40 2024-04-25T10:09:35.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.789+0000 7fea4affd640 1 -- 172.21.15.60:0/2473720386 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fea5400fc80 con 0x7fea5c09de40 2024-04-25T10:09:35.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.789+0000 7fea4affd640 1 --2- 172.21.15.60:0/2473720386 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fea3c066f60 0x7fea3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:35.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.789+0000 7fea4affd640 1 -- 172.21.15.60:0/2473720386 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fea54087be0 con 0x7fea5c09de40 2024-04-25T10:09:35.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.789+0000 7fea5a59c640 1 --2- 172.21.15.60:0/2473720386 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fea3c066f60 0x7fea3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:35.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.789+0000 7fea5a59c640 1 --2- 172.21.15.60:0/2473720386 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fea3c066f60 0x7fea3c069420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009aa0 tx=0x7fea4c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:35.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:35.793+0000 7fea4affd640 1 -- 172.21.15.60:0/2473720386 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fea54058ef0 con 0x7fea5c09de40 2024-04-25T10:09:36.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.009+0000 7fea60865640 1 -- 172.21.15.60:0/2473720386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fea5c09d410 con 0x7fea5c09de40 2024-04-25T10:09:36.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.269+0000 7fea4affd640 1 -- 172.21.15.60:0/2473720386 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v218) v1 ==== 169+0+0 (secure 0 0 0) 0x7fea540587f0 con 0x7fea5c09de40 2024-04-25T10:09:36.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.269+0000 7fea60865640 1 -- 172.21.15.60:0/2473720386 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fea3c066f60 msgr2=0x7fea3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.269+0000 7fea60865640 1 --2- 172.21.15.60:0/2473720386 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fea3c066f60 0x7fea3c069420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009aa0 tx=0x7fea4c008040 comp rx=0 tx=0).stop 2024-04-25T10:09:36.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.269+0000 7fea60865640 1 -- 172.21.15.60:0/2473720386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea5c09de40 msgr2=0x7fea5c09e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.269+0000 7fea60865640 1 --2- 172.21.15.60:0/2473720386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea5c09de40 0x7fea5c09e2d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fea54001ec0 tx=0x7fea54011c20 comp rx=0 tx=0).stop 2024-04-25T10:09:36.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.269+0000 7fea60865640 1 -- 172.21.15.60:0/2473720386 shutdown_connections 2024-04-25T10:09:36.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.269+0000 7fea60865640 1 --2- 172.21.15.60:0/2473720386 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fea3c066f60 0x7fea3c069420 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.269+0000 7fea60865640 1 --2- 172.21.15.60:0/2473720386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea5c09de40 0x7fea5c09e2d0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.269+0000 7fea60865640 1 --2- 172.21.15.60:0/2473720386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea5c09a7e0 0x7fea5c1f2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.269+0000 7fea60865640 1 --2- 172.21.15.60:0/2473720386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea44008c20 0x7fea5c159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.269+0000 7fea60865640 1 -- 172.21.15.60:0/2473720386 >> 172.21.15.60:0/2473720386 conn(0x7fea5c072920 msgr2=0x7fea5c143f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:36.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.269+0000 7fea60865640 1 -- 172.21.15.60:0/2473720386 shutdown_connections 2024-04-25T10:09:36.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.269+0000 7fea60865640 1 -- 172.21.15.60:0/2473720386 wait complete. 2024-04-25T10:09:36.303 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T10:09:36.303 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T10:09:36.303 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:09:36.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f22308fc640 1 Processor -- start 2024-04-25T10:09:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f22308fc640 1 -- start start 2024-04-25T10:09:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f22308fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f222c0ce7a0 0x7f222c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f22308fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f222c15bc90 0x7f222c0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f22308fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f222c0b4660 0x7f222c0b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f22308fc640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f222c0cc370 con 0x7f222c15bc90 2024-04-25T10:09:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f22308fc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f222c0cf0c0 con 0x7f222c0ce7a0 2024-04-25T10:09:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f22308fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222c071ca0 con 0x7f222c0b4660 2024-04-25T10:09:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f2221d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f222c15bc90 0x7f222c0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f2221d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f222c15bc90 0x7f222c0b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37774/0 (socket says 172.21.15.60:37774) 2024-04-25T10:09:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f2221d9b640 1 -- 172.21.15.60:0/3849542181 learned_addr learned my addr 172.21.15.60:0/3849542181 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f222ad9d640 1 --2- 172.21.15.60:0/3849542181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f222c0b4660 0x7f222c0b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f222a59c640 1 --2- 172.21.15.60:0/3849542181 >> v2:172.21.15.100:3300/0 conn(0x7f222c0ce7a0 0x7f222c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f2221d9b640 1 -- 172.21.15.60:0/3849542181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f222c0b4660 msgr2=0x7f222c0b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f2221d9b640 1 --2- 172.21.15.60:0/3849542181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f222c0b4660 0x7f222c0b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f2221d9b640 1 -- 172.21.15.60:0/3849542181 >> v2:172.21.15.100:3300/0 conn(0x7f222c0ce7a0 msgr2=0x7f222c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f2221d9b640 1 --2- 172.21.15.60:0/3849542181 >> v2:172.21.15.100:3300/0 conn(0x7f222c0ce7a0 0x7f222c0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.425+0000 7f2221d9b640 1 -- 172.21.15.60:0/3849542181 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c0cca40 con 0x7f222c15bc90 2024-04-25T10:09:36.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f2221d9b640 1 --2- 172.21.15.60:0/3849542181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f222c15bc90 0x7f222c0b25d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f222c04f5d0 tx=0x7f221401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605e608726aea71e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f2229d9b640 1 -- 172.21.15.60:0/3849542181 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22140b0070 con 0x7f222c15bc90 2024-04-25T10:09:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f2229d9b640 1 -- 172.21.15.60:0/3849542181 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22140ab600 con 0x7f222c15bc90 2024-04-25T10:09:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f2229d9b640 1 -- 172.21.15.60:0/3849542181 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22140ab9d0 con 0x7f222c15bc90 2024-04-25T10:09:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 -- 172.21.15.60:0/3849542181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f222c15bc90 msgr2=0x7f222c0b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 --2- 172.21.15.60:0/3849542181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f222c15bc90 0x7f222c0b25d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f222c04f5d0 tx=0x7f221401c520 comp rx=0 tx=0).stop 2024-04-25T10:09:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 -- 172.21.15.60:0/3849542181 shutdown_connections 2024-04-25T10:09:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 --2- 172.21.15.60:0/3849542181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f222c0b4660 0x7f222c0b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 --2- 172.21.15.60:0/3849542181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f222c15bc90 0x7f222c0b25d0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 --2- 172.21.15.60:0/3849542181 >> v2:172.21.15.100:3300/0 conn(0x7f222c0ce7a0 0x7f222c0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 -- 172.21.15.60:0/3849542181 >> 172.21.15.60:0/3849542181 conn(0x7f222c0b12d0 msgr2=0x7f222c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 -- 172.21.15.60:0/3849542181 shutdown_connections 2024-04-25T10:09:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 -- 172.21.15.60:0/3849542181 wait complete. 2024-04-25T10:09:36.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 Processor -- start 2024-04-25T10:09:36.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 -- start start 2024-04-25T10:09:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f222c0ce7a0 0x7f222c1526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f222c15bc90 0x7f222c155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f222c155fe0 0x7f222c1493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c0b4280 con 0x7f222c0ce7a0 2024-04-25T10:09:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c0b3460 con 0x7f222c155fe0 2024-04-25T10:09:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f22308fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222c0b4400 con 0x7f222c15bc90 2024-04-25T10:09:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f222a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f222c0ce7a0 0x7f222c1526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f222a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f222c0ce7a0 0x7f222c1526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37790/0 (socket says 172.21.15.60:37790) 2024-04-25T10:09:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f222a59c640 1 -- 172.21.15.60:0/1955524494 learned_addr learned my addr 172.21.15.60:0/1955524494 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f222ad9d640 1 --2- 172.21.15.60:0/1955524494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f222c155fe0 0x7f222c1493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f2221d9b640 1 --2- 172.21.15.60:0/1955524494 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f222c15bc90 0x7f222c155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f222a59c640 1 -- 172.21.15.60:0/1955524494 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f222c15bc90 msgr2=0x7f222c155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f222a59c640 1 --2- 172.21.15.60:0/1955524494 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f222c15bc90 0x7f222c155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f222a59c640 1 -- 172.21.15.60:0/1955524494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f222c155fe0 msgr2=0x7f222c1493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f222a59c640 1 --2- 172.21.15.60:0/1955524494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f222c155fe0 0x7f222c1493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f222a59c640 1 -- 172.21.15.60:0/1955524494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c0b4080 con 0x7f222c0ce7a0 2024-04-25T10:09:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.429+0000 7f222ad9d640 1 --2- 172.21.15.60:0/1955524494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f222c155fe0 0x7f222c1493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.433+0000 7f222a59c640 1 --2- 172.21.15.60:0/1955524494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f222c0ce7a0 0x7f222c1526e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f221c00d5f0 tx=0x7f221c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.433+0000 7f22237fe640 1 -- 172.21.15.60:0/1955524494 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c0125f0 con 0x7f222c0ce7a0 2024-04-25T10:09:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.433+0000 7f22237fe640 1 -- 172.21.15.60:0/1955524494 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221c019070 con 0x7f222c0ce7a0 2024-04-25T10:09:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.433+0000 7f22308fc640 1 -- 172.21.15.60:0/1955524494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222c1543c0 con 0x7f222c0ce7a0 2024-04-25T10:09:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.433+0000 7f22237fe640 1 -- 172.21.15.60:0/1955524494 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c0129b0 con 0x7f222c0ce7a0 2024-04-25T10:09:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.433+0000 7f22308fc640 1 -- 172.21.15.60:0/1955524494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c154980 con 0x7f222c0ce7a0 2024-04-25T10:09:36.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.433+0000 7f22308fc640 1 -- 172.21.15.60:0/1955524494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f222c074770 con 0x7f222c0ce7a0 2024-04-25T10:09:36.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.433+0000 7f22237fe640 1 -- 172.21.15.60:0/1955524494 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f221c012b50 con 0x7f222c0ce7a0 2024-04-25T10:09:36.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.433+0000 7f22237fe640 1 --2- 172.21.15.60:0/1955524494 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f220c066f90 0x7f220c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.433+0000 7f22237fe640 1 -- 172.21.15.60:0/1955524494 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f221c0851e0 con 0x7f222c0ce7a0 2024-04-25T10:09:36.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.433+0000 7f2221d9b640 1 --2- 172.21.15.60:0/1955524494 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f220c066f90 0x7f220c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.433+0000 7f2221d9b640 1 --2- 172.21.15.60:0/1955524494 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f220c066f90 0x7f220c069450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f22140ad000 tx=0x7f22140a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.437+0000 7f22237fe640 1 -- 172.21.15.60:0/1955524494 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f221c056450 con 0x7f222c0ce7a0 2024-04-25T10:09:36.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.601+0000 7f22308fc640 1 -- 172.21.15.60:0/1955524494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f222c15e940 con 0x7f222c0ce7a0 2024-04-25T10:09:36.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.601+0000 7f22237fe640 1 -- 172.21.15.60:0/1955524494 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+13032 (secure 0 0 0) 0x7f221c055d50 con 0x7f222c0ce7a0 2024-04-25T10:09:36.611 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:36.611 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":61,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:09:35.259922+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/2718963870":"2024-04-26T10:09:32.245504+0000","172.21.15.60:6833/2718963870":"2024-04-26T10:09:32.245504+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:36.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.605+0000 7f22308fc640 1 -- 172.21.15.60:0/1955524494 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f220c066f90 msgr2=0x7f220c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.605+0000 7f22308fc640 1 --2- 172.21.15.60:0/1955524494 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f220c066f90 0x7f220c069450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f22140ad000 tx=0x7f22140a6000 comp rx=0 tx=0).stop 2024-04-25T10:09:36.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.605+0000 7f22308fc640 1 -- 172.21.15.60:0/1955524494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f222c0ce7a0 msgr2=0x7f222c1526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.605+0000 7f22308fc640 1 --2- 172.21.15.60:0/1955524494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f222c0ce7a0 0x7f222c1526e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f221c00d5f0 tx=0x7f221c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.605+0000 7f22308fc640 1 -- 172.21.15.60:0/1955524494 shutdown_connections 2024-04-25T10:09:36.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.605+0000 7f22308fc640 1 --2- 172.21.15.60:0/1955524494 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f220c066f90 0x7f220c069450 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.605+0000 7f22308fc640 1 --2- 172.21.15.60:0/1955524494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f222c155fe0 0x7f222c1493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.605+0000 7f22308fc640 1 --2- 172.21.15.60:0/1955524494 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f222c15bc90 0x7f222c155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.605+0000 7f22308fc640 1 --2- 172.21.15.60:0/1955524494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f222c0ce7a0 0x7f222c1526e0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.605+0000 7f22308fc640 1 -- 172.21.15.60:0/1955524494 >> 172.21.15.60:0/1955524494 conn(0x7f222c0b12d0 msgr2=0x7f222c15ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:36.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.605+0000 7f22308fc640 1 -- 172.21.15.60:0/1955524494 shutdown_connections 2024-04-25T10:09:36.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.605+0000 7f22308fc640 1 -- 172.21.15.60:0/1955524494 wait complete. 2024-04-25T10:09:36.631 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:36.631 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:36.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.753+0000 7fbe366ec640 1 Processor -- start 2024-04-25T10:09:36.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.753+0000 7fbe366ec640 1 -- start start 2024-04-25T10:09:36.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.753+0000 7fbe366ec640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbe300cf8a0 0x7fbe3015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.753+0000 7fbe366ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbe300cd3d0 0x7fbe3015be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.753+0000 7fbe366ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe300cd890 con 0x7fbe3015dd30 2024-04-25T10:09:36.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.753+0000 7fbe366ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe300ce4a0 con 0x7fbe300cd3d0 2024-04-25T10:09:36.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.753+0000 7fbe366ec640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe300cdb70 con 0x7fbe300cf8a0 2024-04-25T10:09:36.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.753+0000 7fbe34c89640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbe300cd3d0 0x7fbe3015be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.753+0000 7fbe34c89640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbe300cd3d0 0x7fbe3015be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37800/0 (socket says 172.21.15.60:37800) 2024-04-25T10:09:36.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.753+0000 7fbe34c89640 1 -- 172.21.15.60:0/930114837 learned_addr learned my addr 172.21.15.60:0/930114837 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:36.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.753+0000 7fbe2f7fe640 1 --2- 172.21.15.60:0/930114837 >> v2:172.21.15.100:3300/0 conn(0x7fbe300cf8a0 0x7fbe3015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.753+0000 7fbe34c89640 1 -- 172.21.15.60:0/930114837 >> v2:172.21.15.100:3300/0 conn(0x7fbe300cf8a0 msgr2=0x7fbe3015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:36.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.753+0000 7fbe34c89640 1 --2- 172.21.15.60:0/930114837 >> v2:172.21.15.100:3300/0 conn(0x7fbe300cf8a0 0x7fbe3015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.753+0000 7fbe2effd640 1 -- 172.21.15.60:0/930114837 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263670568 0 0) 0x7fbe300cd890 con 0x7fbe3015dd30 2024-04-25T10:09:36.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.753+0000 7fbe34c89640 1 -- 172.21.15.60:0/930114837 >> v1:172.21.15.60:6789/0 conn(0x7fbe3015dd30 legacy=0x7fbe300cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.753+0000 7fbe34c89640 1 -- 172.21.15.60:0/930114837 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe300cde50 con 0x7fbe300cd3d0 2024-04-25T10:09:36.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.757+0000 7fbe34c89640 1 --2- 172.21.15.60:0/930114837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe300cd3d0 0x7fbe3015be20 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbe30072550 tx=0x7fbe2801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73aba68045fe333 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.757+0000 7fbe2effd640 1 -- 172.21.15.60:0/930114837 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280a5810 con 0x7fbe300cd3d0 2024-04-25T10:09:36.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.757+0000 7fbe2effd640 1 -- 172.21.15.60:0/930114837 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe280a59b0 con 0x7fbe300cd3d0 2024-04-25T10:09:36.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.757+0000 7fbe2effd640 1 -- 172.21.15.60:0/930114837 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280a5d80 con 0x7fbe300cd3d0 2024-04-25T10:09:36.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.757+0000 7fbe366ec640 1 -- 172.21.15.60:0/930114837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe300cd3d0 msgr2=0x7fbe3015be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.757+0000 7fbe366ec640 1 --2- 172.21.15.60:0/930114837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe300cd3d0 0x7fbe3015be20 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbe30072550 tx=0x7fbe2801c930 comp rx=0 tx=0).stop 2024-04-25T10:09:36.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.757+0000 7fbe366ec640 1 -- 172.21.15.60:0/930114837 shutdown_connections 2024-04-25T10:09:36.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.757+0000 7fbe366ec640 1 --2- 172.21.15.60:0/930114837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe300cd3d0 0x7fbe3015be20 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.757+0000 7fbe366ec640 1 --2- 172.21.15.60:0/930114837 >> v2:172.21.15.100:3300/0 conn(0x7fbe300cf8a0 0x7fbe3015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.757+0000 7fbe366ec640 1 -- 172.21.15.60:0/930114837 >> 172.21.15.60:0/930114837 conn(0x7fbe300670b0 msgr2=0x7fbe3006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:36.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.757+0000 7fbe366ec640 1 -- 172.21.15.60:0/930114837 shutdown_connections 2024-04-25T10:09:36.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.757+0000 7fbe366ec640 1 -- 172.21.15.60:0/930114837 wait complete. 2024-04-25T10:09:36.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.757+0000 7fbe366ec640 1 Processor -- start 2024-04-25T10:09:36.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.757+0000 7fbe366ec640 1 -- start start 2024-04-25T10:09:36.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe366ec640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe300cd3d0 0x7fbe300b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe366ec640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe300cf8a0 0x7fbe30145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe366ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe3015dd30 0x7fbe3015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe366ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3015cf00 con 0x7fbe3015dd30 2024-04-25T10:09:36.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe366ec640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3015cd80 con 0x7fbe300cd3d0 2024-04-25T10:09:36.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe366ec640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3015d080 con 0x7fbe300cf8a0 2024-04-25T10:09:36.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe34c89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe3015dd30 0x7fbe3015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe34c89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe3015dd30 0x7fbe3015ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37802/0 (socket says 172.21.15.60:37802) 2024-04-25T10:09:36.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe34c89640 1 -- 172.21.15.60:0/2720639789 learned_addr learned my addr 172.21.15.60:0/2720639789 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:36.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe2ffff640 1 --2- 172.21.15.60:0/2720639789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe300cd3d0 0x7fbe300b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe2f7fe640 1 --2- 172.21.15.60:0/2720639789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe300cf8a0 0x7fbe30145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe34c89640 1 -- 172.21.15.60:0/2720639789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe300cf8a0 msgr2=0x7fbe30145ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe34c89640 1 --2- 172.21.15.60:0/2720639789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe300cf8a0 0x7fbe30145ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe34c89640 1 -- 172.21.15.60:0/2720639789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe300cd3d0 msgr2=0x7fbe300b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe34c89640 1 --2- 172.21.15.60:0/2720639789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe300cd3d0 0x7fbe300b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe34c89640 1 -- 172.21.15.60:0/2720639789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe300b13e0 con 0x7fbe3015dd30 2024-04-25T10:09:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe2f7fe640 1 --2- 172.21.15.60:0/2720639789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe300cf8a0 0x7fbe30145ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe34c89640 1 --2- 172.21.15.60:0/2720639789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe3015dd30 0x7fbe3015ac60 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fbe28019540 tx=0x7fbe280196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe2cff9640 1 -- 172.21.15.60:0/2720639789 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280bc020 con 0x7fbe3015dd30 2024-04-25T10:09:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe2cff9640 1 -- 172.21.15.60:0/2720639789 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe280b88c0 con 0x7fbe3015dd30 2024-04-25T10:09:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe366ec640 1 -- 172.21.15.60:0/2720639789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe300b16c0 con 0x7fbe3015dd30 2024-04-25T10:09:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe2cff9640 1 -- 172.21.15.60:0/2720639789 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280b8cb0 con 0x7fbe3015dd30 2024-04-25T10:09:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe366ec640 1 -- 172.21.15.60:0/2720639789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe300b1c80 con 0x7fbe3015dd30 2024-04-25T10:09:36.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe366ec640 1 -- 172.21.15.60:0/2720639789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe30074770 con 0x7fbe3015dd30 2024-04-25T10:09:36.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe2cff9640 1 -- 172.21.15.60:0/2720639789 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe280bf020 con 0x7fbe3015dd30 2024-04-25T10:09:36.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe2cff9640 1 --2- 172.21.15.60:0/2720639789 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe1c066f60 0x7fbe1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:36.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe2cff9640 1 -- 172.21.15.60:0/2720639789 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbe28120060 con 0x7fbe3015dd30 2024-04-25T10:09:36.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.761+0000 7fbe2ffff640 1 --2- 172.21.15.60:0/2720639789 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe1c066f60 0x7fbe1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:36.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.765+0000 7fbe2ffff640 1 --2- 172.21.15.60:0/2720639789 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe1c066f60 0x7fbe1c069420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbe20000f70 tx=0x7fbe20009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:36.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.765+0000 7fbe2cff9640 1 -- 172.21.15.60:0/2720639789 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbe280c6030 con 0x7fbe3015dd30 2024-04-25T10:09:36.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.925+0000 7fbe366ec640 1 -- 172.21.15.60:0/2720639789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe3006c250 con 0x7fbe3015dd30 2024-04-25T10:09:36.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.925+0000 7fbe2cff9640 1 -- 172.21.15.60:0/2720639789 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe280165e0 con 0x7fbe3015dd30 2024-04-25T10:09:36.936 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:09:36.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.929+0000 7fbe366ec640 1 -- 172.21.15.60:0/2720639789 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe1c066f60 msgr2=0x7fbe1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.929+0000 7fbe366ec640 1 --2- 172.21.15.60:0/2720639789 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe1c066f60 0x7fbe1c069420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbe20000f70 tx=0x7fbe20009040 comp rx=0 tx=0).stop 2024-04-25T10:09:36.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.929+0000 7fbe366ec640 1 -- 172.21.15.60:0/2720639789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe3015dd30 msgr2=0x7fbe3015ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:36.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.929+0000 7fbe366ec640 1 --2- 172.21.15.60:0/2720639789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe3015dd30 0x7fbe3015ac60 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fbe28019540 tx=0x7fbe280196d0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.929+0000 7fbe366ec640 1 -- 172.21.15.60:0/2720639789 shutdown_connections 2024-04-25T10:09:36.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.929+0000 7fbe366ec640 1 --2- 172.21.15.60:0/2720639789 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbe1c066f60 0x7fbe1c069420 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.929+0000 7fbe366ec640 1 --2- 172.21.15.60:0/2720639789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe3015dd30 0x7fbe3015ac60 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.929+0000 7fbe366ec640 1 --2- 172.21.15.60:0/2720639789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe300cf8a0 0x7fbe30145ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.929+0000 7fbe366ec640 1 --2- 172.21.15.60:0/2720639789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe300cd3d0 0x7fbe300b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:36.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.929+0000 7fbe366ec640 1 -- 172.21.15.60:0/2720639789 >> 172.21.15.60:0/2720639789 conn(0x7fbe300670b0 msgr2=0x7fbe3015d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:36.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.929+0000 7fbe366ec640 1 -- 172.21.15.60:0/2720639789 shutdown_connections 2024-04-25T10:09:36.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:36.929+0000 7fbe366ec640 1 -- 172.21.15.60:0/2720639789 wait complete. 2024-04-25T10:09:36.956 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-25T10:09:37.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.069+0000 7f86569ba640 1 Processor -- start 2024-04-25T10:09:37.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.069+0000 7f86569ba640 1 -- start start 2024-04-25T10:09:37.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.069+0000 7f86569ba640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f865015deb0 0x7f86500b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.069+0000 7f86569ba640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86500d0340 0x7f86500d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.069+0000 7f86569ba640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865006d200 con 0x7f86500ce2b0 2024-04-25T10:09:37.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.069+0000 7f86569ba640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86500cf2c0 con 0x7f865015deb0 2024-04-25T10:09:37.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.069+0000 7f86569ba640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86500cefe0 con 0x7f86500d0340 2024-04-25T10:09:37.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.069+0000 7f864ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f865015deb0 0x7f86500b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.069+0000 7f864ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f865015deb0 0x7f86500b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37818/0 (socket says 172.21.15.60:37818) 2024-04-25T10:09:37.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.069+0000 7f864ffff640 1 -- 172.21.15.60:0/3483244401 learned_addr learned my addr 172.21.15.60:0/3483244401 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:37.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.069+0000 7f864f7fe640 1 --2- 172.21.15.60:0/3483244401 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86500d0340 0x7f86500d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.069+0000 7f864ffff640 1 -- 172.21.15.60:0/3483244401 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86500d0340 msgr2=0x7f86500d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:37.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.069+0000 7f864ffff640 1 --2- 172.21.15.60:0/3483244401 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86500d0340 0x7f86500d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.069+0000 7f864ffff640 1 -- 172.21.15.60:0/3483244401 >> v1:172.21.15.100:6789/0 conn(0x7f86500ce2b0 legacy=0x7f86500ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:37.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.069+0000 7f864ffff640 1 -- 172.21.15.60:0/3483244401 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86500d1d70 con 0x7f865015deb0 2024-04-25T10:09:37.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.069+0000 7f864ffff640 1 --2- 172.21.15.60:0/3483244401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f865015deb0 0x7f86500b6c10 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8640009e50 tx=0x7f864001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52de66dec2ef3510 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:37.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f864effd640 1 -- 172.21.15.60:0/3483244401 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86400a5670 con 0x7f865015deb0 2024-04-25T10:09:37.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f864effd640 1 -- 172.21.15.60:0/3483244401 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86400a5810 con 0x7f865015deb0 2024-04-25T10:09:37.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f864effd640 1 -- 172.21.15.60:0/3483244401 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86400a5be0 con 0x7f865015deb0 2024-04-25T10:09:37.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f86569ba640 1 -- 172.21.15.60:0/3483244401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f865015deb0 msgr2=0x7f86500b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:37.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f86569ba640 1 --2- 172.21.15.60:0/3483244401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f865015deb0 0x7f86500b6c10 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8640009e50 tx=0x7f864001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f86569ba640 1 -- 172.21.15.60:0/3483244401 shutdown_connections 2024-04-25T10:09:37.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f86569ba640 1 --2- 172.21.15.60:0/3483244401 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86500d0340 0x7f86500d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f86569ba640 1 --2- 172.21.15.60:0/3483244401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f865015deb0 0x7f86500b6c10 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f86569ba640 1 -- 172.21.15.60:0/3483244401 >> 172.21.15.60:0/3483244401 conn(0x7f865006c090 msgr2=0x7f865006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:37.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f86569ba640 1 -- 172.21.15.60:0/3483244401 shutdown_connections 2024-04-25T10:09:37.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f86569ba640 1 -- 172.21.15.60:0/3483244401 wait complete. 2024-04-25T10:09:37.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f86569ba640 1 Processor -- start 2024-04-25T10:09:37.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f86569ba640 1 -- start start 2024-04-25T10:09:37.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f86569ba640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86500ce2b0 0x7f8650151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f86569ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86500d0340 0x7f865015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f86569ba640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f865015deb0 0x7f865015a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f86569ba640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86500d3430 con 0x7f86500d0340 2024-04-25T10:09:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f86569ba640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86500d32b0 con 0x7f86500ce2b0 2024-04-25T10:09:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f86569ba640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86500d35b0 con 0x7f865015deb0 2024-04-25T10:09:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f864f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86500d0340 0x7f865015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f864f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86500d0340 0x7f865015ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37832/0 (socket says 172.21.15.60:37832) 2024-04-25T10:09:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f864f7fe640 1 -- 172.21.15.60:0/3781004894 learned_addr learned my addr 172.21.15.60:0/3781004894 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f864ffff640 1 --2- 172.21.15.60:0/3781004894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86500ce2b0 0x7f8650151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f8654f57640 1 --2- 172.21.15.60:0/3781004894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f865015deb0 0x7f865015a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f864f7fe640 1 -- 172.21.15.60:0/3781004894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f865015deb0 msgr2=0x7f865015a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f864f7fe640 1 --2- 172.21.15.60:0/3781004894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f865015deb0 0x7f865015a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f864f7fe640 1 -- 172.21.15.60:0/3781004894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86500ce2b0 msgr2=0x7f8650151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f864f7fe640 1 --2- 172.21.15.60:0/3781004894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86500ce2b0 0x7f8650151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f864f7fe640 1 -- 172.21.15.60:0/3781004894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86500b6bd0 con 0x7f86500d0340 2024-04-25T10:09:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f864f7fe640 1 --2- 172.21.15.60:0/3781004894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86500d0340 0x7f865015ca70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f863800d800 tx=0x7f863800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.073+0000 7f864cff9640 1 -- 172.21.15.60:0/3781004894 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863800b6c0 con 0x7f86500d0340 2024-04-25T10:09:37.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.077+0000 7f864cff9640 1 -- 172.21.15.60:0/3781004894 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8638019070 con 0x7f86500d0340 2024-04-25T10:09:37.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.077+0000 7f86569ba640 1 -- 172.21.15.60:0/3781004894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86500b6eb0 con 0x7f86500d0340 2024-04-25T10:09:37.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.077+0000 7f864cff9640 1 -- 172.21.15.60:0/3781004894 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863800bae0 con 0x7f86500d0340 2024-04-25T10:09:37.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.077+0000 7f86569ba640 1 -- 172.21.15.60:0/3781004894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86500b7470 con 0x7f86500d0340 2024-04-25T10:09:37.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.077+0000 7f86569ba640 1 -- 172.21.15.60:0/3781004894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8650074770 con 0x7f86500d0340 2024-04-25T10:09:37.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.077+0000 7f864cff9640 1 -- 172.21.15.60:0/3781004894 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f863800bc80 con 0x7f86500d0340 2024-04-25T10:09:37.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.077+0000 7f864cff9640 1 --2- 172.21.15.60:0/3781004894 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8630066f90 0x7f8630069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.077+0000 7f864cff9640 1 -- 172.21.15.60:0/3781004894 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f86380852a0 con 0x7f86500d0340 2024-04-25T10:09:37.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.077+0000 7f864ffff640 1 --2- 172.21.15.60:0/3781004894 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8630066f90 0x7f8630069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.077+0000 7f864ffff640 1 --2- 172.21.15.60:0/3781004894 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8630066f90 0x7f8630069450 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f86400ae000 tx=0x7f86400a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:37.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.081+0000 7f864cff9640 1 -- 172.21.15.60:0/3781004894 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8638056520 con 0x7f86500d0340 2024-04-25T10:09:37.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:37.297+0000 7f86569ba640 1 -- 172.21.15.60:0/3781004894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7f86500cf8d0 con 0x7f86500d0340 2024-04-25T10:09:38.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.277+0000 7f864cff9640 1 -- 172.21.15.60:0/3781004894 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v220) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8638055e20 con 0x7f86500d0340 2024-04-25T10:09:38.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.281+0000 7f86569ba640 1 -- 172.21.15.60:0/3781004894 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8630066f90 msgr2=0x7f8630069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.281+0000 7f86569ba640 1 --2- 172.21.15.60:0/3781004894 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8630066f90 0x7f8630069450 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f86400ae000 tx=0x7f86400a7070 comp rx=0 tx=0).stop 2024-04-25T10:09:38.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.281+0000 7f86569ba640 1 -- 172.21.15.60:0/3781004894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86500d0340 msgr2=0x7f865015ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.281+0000 7f86569ba640 1 --2- 172.21.15.60:0/3781004894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86500d0340 0x7f865015ca70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f863800d800 tx=0x7f863800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.281+0000 7f86569ba640 1 -- 172.21.15.60:0/3781004894 shutdown_connections 2024-04-25T10:09:38.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.281+0000 7f86569ba640 1 --2- 172.21.15.60:0/3781004894 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8630066f90 0x7f8630069450 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.281+0000 7f86569ba640 1 --2- 172.21.15.60:0/3781004894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f865015deb0 0x7f865015a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.281+0000 7f86569ba640 1 --2- 172.21.15.60:0/3781004894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86500d0340 0x7f865015ca70 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.281+0000 7f86569ba640 1 --2- 172.21.15.60:0/3781004894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86500ce2b0 0x7f8650151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.281+0000 7f86569ba640 1 -- 172.21.15.60:0/3781004894 >> 172.21.15.60:0/3781004894 conn(0x7f865006c090 msgr2=0x7f86500d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.281+0000 7f86569ba640 1 -- 172.21.15.60:0/3781004894 shutdown_connections 2024-04-25T10:09:38.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.281+0000 7f86569ba640 1 -- 172.21.15.60:0/3781004894 wait complete. 2024-04-25T10:09:38.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:09:38.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16b7162640 1 Processor -- start 2024-04-25T10:09:38.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16b7162640 1 -- start start 2024-04-25T10:09:38.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16b7162640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16b00cbcf0 0x7f16b0160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16b7162640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f16b015fbf0 0x7f16b00ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16b7162640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b0144a40 con 0x7f16b0071c30 2024-04-25T10:09:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16b7162640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b00b2dc0 con 0x7f16b015fbf0 2024-04-25T10:09:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16b7162640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b00cc9f0 con 0x7f16b00cbcf0 2024-04-25T10:09:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f16b0071c30 0x7f16b015f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44686/0 (socket says 172.21.15.60:44686) 2024-04-25T10:09:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a7fff640 1 -- 172.21.15.60:0/1050521640 learned_addr learned my addr 172.21.15.60:0/1050521640 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16b56ff640 1 --2- 172.21.15.60:0/1050521640 >> v2:172.21.15.100:3300/0 conn(0x7f16b015fbf0 0x7f16b00ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16b4efe640 1 --2- 172.21.15.60:0/1050521640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16b00cbcf0 0x7f16b0160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a77fe640 1 -- 172.21.15.60:0/1050521640 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727053604 0 0) 0x7f16b0144a40 con 0x7f16b0071c30 2024-04-25T10:09:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a77fe640 1 -- 172.21.15.60:0/1050521640 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1698003530 con 0x7f16b0071c30 2024-04-25T10:09:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a77fe640 1 -- 172.21.15.60:0/1050521640 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 561352177 0 0) 0x7f1698003530 con 0x7f16b0071c30 2024-04-25T10:09:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a77fe640 1 -- 172.21.15.60:0/1050521640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16b00cbcf0 msgr2=0x7f16b0160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a77fe640 1 --2- 172.21.15.60:0/1050521640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16b00cbcf0 0x7f16b0160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a77fe640 1 -- 172.21.15.60:0/1050521640 >> v2:172.21.15.100:3300/0 conn(0x7f16b015fbf0 msgr2=0x7f16b00ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a77fe640 1 --2- 172.21.15.60:0/1050521640 >> v2:172.21.15.100:3300/0 conn(0x7f16b015fbf0 0x7f16b00ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a77fe640 1 -- 172.21.15.60:0/1050521640 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b00cccd0 con 0x7f16b0071c30 2024-04-25T10:09:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a77fe640 1 -- 172.21.15.60:0/1050521640 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f16a0001680 con 0x7f16b0071c30 2024-04-25T10:09:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a77fe640 1 -- 172.21.15.60:0/1050521640 >> v1:172.21.15.60:6789/0 conn(0x7f16b0071c30 legacy=0x7f16b015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a77fe640 1 --2- 172.21.15.60:0/1050521640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16980039c0 0x7f1698003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a77fe640 1 --2- 172.21.15.60:0/1050521640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16b00cbcf0 0x7f1698004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a77fe640 1 --2- 172.21.15.60:0/1050521640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698004d50 0x7f16980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a77fe640 1 -- 172.21.15.60:0/1050521640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b00cc9f0 con 0x7f1698004d50 2024-04-25T10:09:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a77fe640 1 -- 172.21.15.60:0/1050521640 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169800b9d0 con 0x7f16980039c0 2024-04-25T10:09:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a77fe640 1 -- 172.21.15.60:0/1050521640 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169800bb50 con 0x7f16b00cbcf0 2024-04-25T10:09:38.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16b56ff640 1 --2- 172.21.15.60:0/1050521640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698004d50 0x7f16980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16b4efe640 1 --2- 172.21.15.60:0/1050521640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16980039c0 0x7f1698003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16a7fff640 1 --2- 172.21.15.60:0/1050521640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16b00cbcf0 0x7f1698004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16b56ff640 1 -- 172.21.15.60:0/1050521640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16b00cbcf0 msgr2=0x7f1698004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:38.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16b56ff640 1 --2- 172.21.15.60:0/1050521640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16b00cbcf0 0x7f1698004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16b56ff640 1 -- 172.21.15.60:0/1050521640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16980039c0 msgr2=0x7f1698003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:38.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16b56ff640 1 --2- 172.21.15.60:0/1050521640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16980039c0 0x7f1698003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.429+0000 7f16b56ff640 1 -- 172.21.15.60:0/1050521640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1698003530 con 0x7f1698004d50 2024-04-25T10:09:38.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b56ff640 1 --2- 172.21.15.60:0/1050521640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698004d50 0x7f16980051d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009fc0 tx=0x7f16a801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6cb2da4cf4501a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16a77fe640 1 -- 172.21.15.60:0/1050521640 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a80b2070 con 0x7f1698004d50 2024-04-25T10:09:38.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16a77fe640 1 -- 172.21.15.60:0/1050521640 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a80a5cd0 con 0x7f1698004d50 2024-04-25T10:09:38.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16a77fe640 1 -- 172.21.15.60:0/1050521640 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8009c10 con 0x7f1698004d50 2024-04-25T10:09:38.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b7162640 1 -- 172.21.15.60:0/1050521640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698004d50 msgr2=0x7f16980051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b7162640 1 --2- 172.21.15.60:0/1050521640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698004d50 0x7f16980051d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009fc0 tx=0x7f16a801ccc0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b56ff640 1 -- 172.21.15.60:0/1050521640 reap_dead start 2024-04-25T10:09:38.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b7162640 1 -- 172.21.15.60:0/1050521640 shutdown_connections 2024-04-25T10:09:38.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b7162640 1 -- 172.21.15.60:0/1050521640 >> 172.21.15.60:0/1050521640 conn(0x7f16b00670b0 msgr2=0x7f16b006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:38.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b7162640 1 -- 172.21.15.60:0/1050521640 shutdown_connections 2024-04-25T10:09:38.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b7162640 1 -- 172.21.15.60:0/1050521640 wait complete. 2024-04-25T10:09:38.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b7162640 1 Processor -- start 2024-04-25T10:09:38.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b7162640 1 -- start start 2024-04-25T10:09:38.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b7162640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698004d50 0x7f16b013f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b7162640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16b01f49a0 0x7f16b00cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b7162640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16b01f4e20 0x7f16b00cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b7162640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b00cc0d0 con 0x7f1698004d50 2024-04-25T10:09:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b7162640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b00ce9a0 con 0x7f16b01f49a0 2024-04-25T10:09:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b7162640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b00720b0 con 0x7f16b01f4e20 2024-04-25T10:09:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b4efe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698004d50 0x7f16b013f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b4efe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698004d50 0x7f16b013f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37848/0 (socket says 172.21.15.60:37848) 2024-04-25T10:09:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16b4efe640 1 -- 172.21.15.60:0/3718202098 learned_addr learned my addr 172.21.15.60:0/3718202098 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.433+0000 7f16a7fff640 1 --2- 172.21.15.60:0/3718202098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16b01f49a0 0x7f16b00cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16b56ff640 1 --2- 172.21.15.60:0/3718202098 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16b01f4e20 0x7f16b00cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16b4efe640 1 -- 172.21.15.60:0/3718202098 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16b01f4e20 msgr2=0x7f16b00cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16b4efe640 1 --2- 172.21.15.60:0/3718202098 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16b01f4e20 0x7f16b00cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16b4efe640 1 -- 172.21.15.60:0/3718202098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16b01f49a0 msgr2=0x7f16b00cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16b4efe640 1 --2- 172.21.15.60:0/3718202098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16b01f49a0 0x7f16b00cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16b4efe640 1 -- 172.21.15.60:0/3718202098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b013ff20 con 0x7f1698004d50 2024-04-25T10:09:38.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16a7fff640 1 --2- 172.21.15.60:0/3718202098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16b01f49a0 0x7f16b00cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:38.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16b56ff640 1 --2- 172.21.15.60:0/3718202098 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16b01f4e20 0x7f16b00cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:38.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16b4efe640 1 --2- 172.21.15.60:0/3718202098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698004d50 0x7f16b013f9e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0115c0 tx=0x7f16ac011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16a57fa640 1 -- 172.21.15.60:0/3718202098 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac015930 con 0x7f1698004d50 2024-04-25T10:09:38.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16a57fa640 1 -- 172.21.15.60:0/3718202098 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ac015ad0 con 0x7f1698004d50 2024-04-25T10:09:38.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16b7162640 1 -- 172.21.15.60:0/3718202098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b01f5380 con 0x7f1698004d50 2024-04-25T10:09:38.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16a57fa640 1 -- 172.21.15.60:0/3718202098 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac01d400 con 0x7f1698004d50 2024-04-25T10:09:38.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16b7162640 1 -- 172.21.15.60:0/3718202098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b014de50 con 0x7f1698004d50 2024-04-25T10:09:38.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16b7162640 1 -- 172.21.15.60:0/3718202098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b006bba0 con 0x7f1698004d50 2024-04-25T10:09:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16a57fa640 1 -- 172.21.15.60:0/3718202098 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f16ac01d5a0 con 0x7f1698004d50 2024-04-25T10:09:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16a57fa640 1 --2- 172.21.15.60:0/3718202098 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f169c066f90 0x7f169c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16a57fa640 1 -- 172.21.15.60:0/3718202098 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f16ac085550 con 0x7f1698004d50 2024-04-25T10:09:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16a7fff640 1 --2- 172.21.15.60:0/3718202098 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f169c066f90 0x7f169c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.437+0000 7f16a7fff640 1 --2- 172.21.15.60:0/3718202098 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f169c066f90 0x7f169c069450 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f16a0001d30 tx=0x7f16a0001cc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.441+0000 7f16a57fa640 1 -- 172.21.15.60:0/3718202098 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16ac019030 con 0x7f1698004d50 2024-04-25T10:09:38.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.605+0000 7f16b7162640 1 -- 172.21.15.60:0/3718202098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f16b01f58a0 con 0x7f1698004d50 2024-04-25T10:09:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.609+0000 7f16a57fa640 1 -- 172.21.15.60:0/3718202098 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f16b01f58a0 con 0x7f1698004d50 2024-04-25T10:09:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.613+0000 7f16b7162640 1 -- 172.21.15.60:0/3718202098 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f169c066f90 msgr2=0x7f169c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.613+0000 7f16b7162640 1 --2- 172.21.15.60:0/3718202098 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f169c066f90 0x7f169c069450 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f16a0001d30 tx=0x7f16a0001cc0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.613+0000 7f16b7162640 1 -- 172.21.15.60:0/3718202098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698004d50 msgr2=0x7f16b013f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.613+0000 7f16b7162640 1 --2- 172.21.15.60:0/3718202098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698004d50 0x7f16b013f9e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0115c0 tx=0x7f16ac011a80 comp rx=0 tx=0).stop 2024-04-25T10:09:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.613+0000 7f16b7162640 1 -- 172.21.15.60:0/3718202098 shutdown_connections 2024-04-25T10:09:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.613+0000 7f16b7162640 1 --2- 172.21.15.60:0/3718202098 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f169c066f90 0x7f169c069450 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.613+0000 7f16b7162640 1 --2- 172.21.15.60:0/3718202098 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16b01f4e20 0x7f16b00cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.613+0000 7f16b7162640 1 --2- 172.21.15.60:0/3718202098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16b01f49a0 0x7f16b00cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.613+0000 7f16b7162640 1 --2- 172.21.15.60:0/3718202098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1698004d50 0x7f16b013f9e0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.613+0000 7f16b7162640 1 -- 172.21.15.60:0/3718202098 >> 172.21.15.60:0/3718202098 conn(0x7f16b00670b0 msgr2=0x7f16b0143c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.613+0000 7f16b7162640 1 -- 172.21.15.60:0/3718202098 shutdown_connections 2024-04-25T10:09:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.613+0000 7f16b7162640 1 -- 172.21.15.60:0/3718202098 wait complete. 2024-04-25T10:09:38.670 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:09:38.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:09:38.718 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:09:38.770 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:09:38.771 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:09:38.771 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:38.778 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:09:38.778 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:09:38.778 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:38.785 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:09:38.785 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:09:38.785 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:38.827 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:09:38.827 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:09:38.827 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:38.833 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:09:38.834 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:09:38.834 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:39.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.989+0000 7f99496d0640 1 Processor -- start 2024-04-25T10:09:39.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.989+0000 7f99496d0640 1 -- start start 2024-04-25T10:09:39.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99496d0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9944160eb0 0x7f99440b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99496d0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9944161290 0x7f99440b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99496d0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f99440b9730 0x7f99440b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99496d0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99440ce9c0 con 0x7f9944161290 2024-04-25T10:09:39.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99496d0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99440ceca0 con 0x7f99440b9730 2024-04-25T10:09:39.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99496d0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99440d2490 con 0x7f9944160eb0 2024-04-25T10:09:39.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99427fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9944161290 0x7f99440b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99427fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9944161290 0x7f99440b6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44826/0 (socket says 172.21.15.60:44826) 2024-04-25T10:09:39.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99427fc640 1 -- 172.21.15.60:0/1320009468 learned_addr learned my addr 172.21.15.60:0/1320009468 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:39.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99437fe640 1 --2- 172.21.15.60:0/1320009468 >> v2:172.21.15.100:3300/0 conn(0x7f99440b9730 0x7f99440b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f9942ffd640 1 --2- 172.21.15.60:0/1320009468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9944160eb0 0x7f99440b91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99427fc640 1 -- 172.21.15.60:0/1320009468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9944160eb0 msgr2=0x7f99440b91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:39.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99427fc640 1 --2- 172.21.15.60:0/1320009468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9944160eb0 0x7f99440b91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99427fc640 1 -- 172.21.15.60:0/1320009468 >> v2:172.21.15.100:3300/0 conn(0x7f99440b9730 msgr2=0x7f99440b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:39.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99427fc640 1 --2- 172.21.15.60:0/1320009468 >> v2:172.21.15.100:3300/0 conn(0x7f99440b9730 0x7f99440b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99427fc640 1 -- 172.21.15.60:0/1320009468 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99440d2770 con 0x7f9944161290 2024-04-25T10:09:39.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99437fe640 1 --2- 172.21.15.60:0/1320009468 >> v2:172.21.15.100:3300/0 conn(0x7f99440b9730 0x7f99440b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:39.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.993+0000 7f99427fc640 1 --2- 172.21.15.60:0/1320009468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9944161290 0x7f99440b6a70 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f994408c580 tx=0x7f993801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579319a116a42bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f9941ffb640 1 -- 172.21.15.60:0/1320009468 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99380b5070 con 0x7f9944161290 2024-04-25T10:09:39.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f9941ffb640 1 -- 172.21.15.60:0/1320009468 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99380a89d0 con 0x7f9944161290 2024-04-25T10:09:39.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f9941ffb640 1 -- 172.21.15.60:0/1320009468 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99380a8da0 con 0x7f9944161290 2024-04-25T10:09:39.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 -- 172.21.15.60:0/1320009468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9944161290 msgr2=0x7f99440b6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 --2- 172.21.15.60:0/1320009468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9944161290 0x7f99440b6a70 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f994408c580 tx=0x7f993801ed10 comp rx=0 tx=0).stop 2024-04-25T10:09:39.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 -- 172.21.15.60:0/1320009468 shutdown_connections 2024-04-25T10:09:39.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 --2- 172.21.15.60:0/1320009468 >> v2:172.21.15.100:3300/0 conn(0x7f99440b9730 0x7f99440b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 --2- 172.21.15.60:0/1320009468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9944161290 0x7f99440b6a70 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 --2- 172.21.15.60:0/1320009468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9944160eb0 0x7f99440b91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 -- 172.21.15.60:0/1320009468 >> 172.21.15.60:0/1320009468 conn(0x7f99440d05c0 msgr2=0x7f99440d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:39.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 -- 172.21.15.60:0/1320009468 shutdown_connections 2024-04-25T10:09:39.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 -- 172.21.15.60:0/1320009468 wait complete. 2024-04-25T10:09:39.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 Processor -- start 2024-04-25T10:09:39.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 -- start start 2024-04-25T10:09:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99440b9730 0x7f99441f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9944160eb0 0x7f99441f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9944161290 0x7f994415a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99440b6fb0 con 0x7f9944160eb0 2024-04-25T10:09:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99440cef80 con 0x7f9944161290 2024-04-25T10:09:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99496d0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99440b7130 con 0x7f99440b9730 2024-04-25T10:09:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99427fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9944160eb0 0x7f99441f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99427fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9944160eb0 0x7f99441f2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44842/0 (socket says 172.21.15.60:44842) 2024-04-25T10:09:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99427fc640 1 -- 172.21.15.60:0/1841303502 learned_addr learned my addr 172.21.15.60:0/1841303502 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f9942ffd640 1 --2- 172.21.15.60:0/1841303502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99440b9730 0x7f99441f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99437fe640 1 --2- 172.21.15.60:0/1841303502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9944161290 0x7f994415a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99427fc640 1 -- 172.21.15.60:0/1841303502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99440b9730 msgr2=0x7f99441f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99427fc640 1 --2- 172.21.15.60:0/1841303502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99440b9730 0x7f99441f3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99427fc640 1 -- 172.21.15.60:0/1841303502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9944161290 msgr2=0x7f994415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99427fc640 1 --2- 172.21.15.60:0/1841303502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9944161290 0x7f994415a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99427fc640 1 -- 172.21.15.60:0/1841303502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99441f4390 con 0x7f9944160eb0 2024-04-25T10:09:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99437fe640 1 --2- 172.21.15.60:0/1841303502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9944161290 0x7f994415a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:39.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f99427fc640 1 --2- 172.21.15.60:0/1841303502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9944160eb0 0x7f99441f2cb0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f99380b3040 tx=0x7f993801bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f992b7fe640 1 -- 172.21.15.60:0/1841303502 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99380c0070 con 0x7f9944160eb0 2024-04-25T10:09:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:38.997+0000 7f992b7fe640 1 -- 172.21.15.60:0/1841303502 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99380b0860 con 0x7f9944160eb0 2024-04-25T10:09:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.001+0000 7f99496d0640 1 -- 172.21.15.60:0/1841303502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9944159e00 con 0x7f9944160eb0 2024-04-25T10:09:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.001+0000 7f992b7fe640 1 -- 172.21.15.60:0/1841303502 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99380b5040 con 0x7f9944160eb0 2024-04-25T10:09:39.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.001+0000 7f99496d0640 1 -- 172.21.15.60:0/1841303502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9944159040 con 0x7f9944160eb0 2024-04-25T10:09:39.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.001+0000 7f99496d0640 1 -- 172.21.15.60:0/1841303502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9944074770 con 0x7f9944160eb0 2024-04-25T10:09:39.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.001+0000 7f992b7fe640 1 -- 172.21.15.60:0/1841303502 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99380c3070 con 0x7f9944160eb0 2024-04-25T10:09:39.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.001+0000 7f992b7fe640 1 --2- 172.21.15.60:0/1841303502 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9930066f60 0x7f9930069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.001+0000 7f992b7fe640 1 -- 172.21.15.60:0/1841303502 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9938121cc0 con 0x7f9944160eb0 2024-04-25T10:09:39.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.001+0000 7f9942ffd640 1 --2- 172.21.15.60:0/1841303502 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9930066f60 0x7f9930069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.001+0000 7f9942ffd640 1 --2- 172.21.15.60:0/1841303502 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9930066f60 0x7f9930069420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f993400b9d0 tx=0x7f9934008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.005+0000 7f992b7fe640 1 -- 172.21.15.60:0/1841303502 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99441f4390 con 0x7f9944160eb0 2024-04-25T10:09:39.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.201+0000 7f99496d0640 1 -- 172.21.15.60:0/1841303502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99440d15b0 con 0x7f9944160eb0 2024-04-25T10:09:39.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.201+0000 7f992b7fe640 1 -- 172.21.15.60:0/1841303502 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f9938018870 con 0x7f9944160eb0 2024-04-25T10:09:39.210 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:39.211 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":32,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/380273391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":380273391},{"type":"v1","addr":"172.21.15.100:6836","nonce":380273391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":5635,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2965977166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2965977166},{"type":"v1","addr":"172.21.15.100:6837","nonce":2965977166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1105028629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1105028629},{"type":"v1","addr":"172.21.15.60:6834","nonce":1105028629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":6795,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/871322571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":871322571},{"type":"v1","addr":"172.21.15.60:6835","nonce":871322571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[]} 2024-04-25T10:09:39.211 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 32 2024-04-25T10:09:39.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.205+0000 7f99496d0640 1 -- 172.21.15.60:0/1841303502 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9930066f60 msgr2=0x7f9930069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.205+0000 7f99496d0640 1 --2- 172.21.15.60:0/1841303502 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9930066f60 0x7f9930069420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f993400b9d0 tx=0x7f9934008040 comp rx=0 tx=0).stop 2024-04-25T10:09:39.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.205+0000 7f99496d0640 1 -- 172.21.15.60:0/1841303502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9944160eb0 msgr2=0x7f99441f2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.205+0000 7f99496d0640 1 --2- 172.21.15.60:0/1841303502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9944160eb0 0x7f99441f2cb0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f99380b3040 tx=0x7f993801bab0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.205+0000 7f99496d0640 1 -- 172.21.15.60:0/1841303502 shutdown_connections 2024-04-25T10:09:39.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.205+0000 7f99496d0640 1 --2- 172.21.15.60:0/1841303502 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9930066f60 0x7f9930069420 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.205+0000 7f99496d0640 1 --2- 172.21.15.60:0/1841303502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9944161290 0x7f994415a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.205+0000 7f99496d0640 1 --2- 172.21.15.60:0/1841303502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9944160eb0 0x7f99441f2cb0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.205+0000 7f99496d0640 1 --2- 172.21.15.60:0/1841303502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99440b9730 0x7f99441f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.205+0000 7f99496d0640 1 -- 172.21.15.60:0/1841303502 >> 172.21.15.60:0/1841303502 conn(0x7f99440d05c0 msgr2=0x7f99440b8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:39.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.205+0000 7f99496d0640 1 -- 172.21.15.60:0/1841303502 shutdown_connections 2024-04-25T10:09:39.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.205+0000 7f99496d0640 1 -- 172.21.15.60:0/1841303502 wait complete. 2024-04-25T10:09:39.229 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:09:39.230 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:09:39.230 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:09:39.230 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:09:39.230 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:09:39.230 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:09:39.230 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:09:39.230 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:09:39.231 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:09:39.231 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:09:39.231 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:09:39.231 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:09:39.231 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:09:39.231 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:09:39.231 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:09:39.231 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:09:39.241 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:09:39.241 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:09:39.243 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:09:39.243 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:09:39.245 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:09:39.245 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:09:39.246 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:09:39.246 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:09:39.285 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:09:39.285 DEBUG:teuthology.parallel:result is None 2024-04-25T10:09:39.286 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:09:39.286 DEBUG:teuthology.parallel:result is None 2024-04-25T10:09:39.289 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:09:39.289 DEBUG:teuthology.parallel:result is None 2024-04-25T10:09:39.290 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:09:39.290 DEBUG:teuthology.parallel:result is None 2024-04-25T10:09:39.291 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:09:39.346 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:09:39.347 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T10:09:39.356 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T10:09:39.356 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T10:09:39.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.397+0000 7fe9117fd640 1 Processor -- start 2024-04-25T10:09:39.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.397+0000 7fe9117fd640 1 -- start start 2024-04-25T10:09:39.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.397+0000 7fe9117fd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe90c15b9c0 0x7fe90c0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.397+0000 7fe9117fd640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90c067e50 con 0x7fe90c0cf120 2024-04-25T10:09:39.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.397+0000 7fe9117fd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90c071780 con 0x7fe90c0cc770 2024-04-25T10:09:39.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.397+0000 7fe9117fd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c0ccbd0 con 0x7fe90c15b9c0 2024-04-25T10:09:39.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe90b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe90c0cf120 0x7fe90c06d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45106/0 (socket says 172.21.15.60:45106) 2024-04-25T10:09:39.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe90b7fe640 1 -- 172.21.15.60:0/3818826674 learned_addr learned my addr 172.21.15.60:0/3818826674 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:39.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe90affd640 1 --2- 172.21.15.60:0/3818826674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe90c15b9c0 0x7fe90c0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe909ffb640 1 -- 172.21.15.60:0/3818826674 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026148649 0 0) 0x7fe90c067e50 con 0x7fe90c0cf120 2024-04-25T10:09:39.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe909ffb640 1 -- 172.21.15.60:0/3818826674 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f4003530 con 0x7fe90c0cf120 2024-04-25T10:09:39.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe90affd640 1 -- 172.21.15.60:0/3818826674 >> v1:172.21.15.100:6789/0 conn(0x7fe90c0cc770 legacy=0x7fe90c0cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe90affd640 1 -- 172.21.15.60:0/3818826674 >> v1:172.21.15.60:6789/0 conn(0x7fe90c0cf120 legacy=0x7fe90c06d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe90affd640 1 -- 172.21.15.60:0/3818826674 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c071a60 con 0x7fe90c15b9c0 2024-04-25T10:09:39.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe90affd640 1 --2- 172.21.15.60:0/3818826674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe90c15b9c0 0x7fe90c0ccee0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe904009dc0 tx=0x7fe90401c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd154c3e4e2f78d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe909ffb640 1 -- 172.21.15.60:0/3818826674 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9040b2070 con 0x7fe90c15b9c0 2024-04-25T10:09:39.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe909ffb640 1 -- 172.21.15.60:0/3818826674 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9040a58b0 con 0x7fe90c15b9c0 2024-04-25T10:09:39.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe909ffb640 1 -- 172.21.15.60:0/3818826674 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9040a5c60 con 0x7fe90c15b9c0 2024-04-25T10:09:39.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe9117fd640 1 -- 172.21.15.60:0/3818826674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe90c15b9c0 msgr2=0x7fe90c0ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe9117fd640 1 --2- 172.21.15.60:0/3818826674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe90c15b9c0 0x7fe90c0ccee0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe904009dc0 tx=0x7fe90401c900 comp rx=0 tx=0).stop 2024-04-25T10:09:39.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe9117fd640 1 -- 172.21.15.60:0/3818826674 shutdown_connections 2024-04-25T10:09:39.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe9117fd640 1 --2- 172.21.15.60:0/3818826674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe90c15b9c0 0x7fe90c0ccee0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe9117fd640 1 -- 172.21.15.60:0/3818826674 >> 172.21.15.60:0/3818826674 conn(0x7fe90c0cf9d0 msgr2=0x7fe90c0cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:39.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe9117fd640 1 -- 172.21.15.60:0/3818826674 shutdown_connections 2024-04-25T10:09:39.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.401+0000 7fe9117fd640 1 -- 172.21.15.60:0/3818826674 wait complete. 2024-04-25T10:09:39.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe9117fd640 1 Processor -- start 2024-04-25T10:09:39.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe9117fd640 1 -- start start 2024-04-25T10:09:39.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe9117fd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe90c0cc770 0x7fe90c0b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe9117fd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe90c0cf120 0x7fe90c1502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe9117fd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe90c15b9c0 0x7fe90c0ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe9117fd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c06e7e0 con 0x7fe90c15b9c0 2024-04-25T10:09:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe9117fd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c06e660 con 0x7fe90c0cf120 2024-04-25T10:09:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe9117fd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c06e960 con 0x7fe90c0cc770 2024-04-25T10:09:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe90b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe90c15b9c0 0x7fe90c0ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe90b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe90c15b9c0 0x7fe90c0ad520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44874/0 (socket says 172.21.15.60:44874) 2024-04-25T10:09:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe90b7fe640 1 -- 172.21.15.60:0/2320736812 learned_addr learned my addr 172.21.15.60:0/2320736812 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe90affd640 1 --2- 172.21.15.60:0/2320736812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe90c0cc770 0x7fe90c0b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe90a7fc640 1 --2- 172.21.15.60:0/2320736812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe90c0cf120 0x7fe90c1502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe90b7fe640 1 -- 172.21.15.60:0/2320736812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe90c0cc770 msgr2=0x7fe90c0b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe90b7fe640 1 --2- 172.21.15.60:0/2320736812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe90c0cc770 0x7fe90c0b2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe90b7fe640 1 -- 172.21.15.60:0/2320736812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe90c0cf120 msgr2=0x7fe90c1502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe90b7fe640 1 --2- 172.21.15.60:0/2320736812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe90c0cf120 0x7fe90c1502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe90b7fe640 1 -- 172.21.15.60:0/2320736812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c0cc220 con 0x7fe90c15b9c0 2024-04-25T10:09:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe90a7fc640 1 --2- 172.21.15.60:0/2320736812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe90c0cf120 0x7fe90c1502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:39.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe90b7fe640 1 --2- 172.21.15.60:0/2320736812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe90c15b9c0 0x7fe90c0ad520 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00c3f0 tx=0x7fe8fc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe8f37fe640 1 -- 172.21.15.60:0/2320736812 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc017070 con 0x7fe90c15b9c0 2024-04-25T10:09:39.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe8f37fe640 1 -- 172.21.15.60:0/2320736812 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc005cf0 con 0x7fe90c15b9c0 2024-04-25T10:09:39.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe8f37fe640 1 -- 172.21.15.60:0/2320736812 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc007310 con 0x7fe90c15b9c0 2024-04-25T10:09:39.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe9117fd640 1 -- 172.21.15.60:0/2320736812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90c0aca40 con 0x7fe90c15b9c0 2024-04-25T10:09:39.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe9117fd640 1 -- 172.21.15.60:0/2320736812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90c0ad000 con 0x7fe90c15b9c0 2024-04-25T10:09:39.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.405+0000 7fe9117fd640 1 -- 172.21.15.60:0/2320736812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90c074770 con 0x7fe90c15b9c0 2024-04-25T10:09:39.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.409+0000 7fe8f37fe640 1 -- 172.21.15.60:0/2320736812 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8fc01a070 con 0x7fe90c15b9c0 2024-04-25T10:09:39.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.409+0000 7fe8f37fe640 1 --2- 172.21.15.60:0/2320736812 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8f8066f60 0x7fe8f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.409+0000 7fe8f37fe640 1 -- 172.21.15.60:0/2320736812 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe8fc083ef0 con 0x7fe90c15b9c0 2024-04-25T10:09:39.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.409+0000 7fe90affd640 1 --2- 172.21.15.60:0/2320736812 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8f8066f60 0x7fe8f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.409+0000 7fe90affd640 1 --2- 172.21.15.60:0/2320736812 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8f8066f60 0x7fe8f8069420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe9040af000 tx=0x7fe9040a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.413+0000 7fe8f37fe640 1 -- 172.21.15.60:0/2320736812 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8fc054c40 con 0x7fe90c15b9c0 2024-04-25T10:09:39.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:39.581+0000 7fe9117fd640 1 -- 172.21.15.60:0/2320736812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe90c0ccff0 con 0x7fe90c15b9c0 2024-04-25T10:09:40.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.301+0000 7fe8f37fe640 1 -- 172.21.15.60:0/2320736812 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v63) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe8fc058ce0 con 0x7fe90c15b9c0 2024-04-25T10:09:40.309 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-04-25T10:09:40.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.301+0000 7fe9117fd640 1 -- 172.21.15.60:0/2320736812 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8f8066f60 msgr2=0x7fe8f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.301+0000 7fe9117fd640 1 --2- 172.21.15.60:0/2320736812 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8f8066f60 0x7fe8f8069420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe9040af000 tx=0x7fe9040a7070 comp rx=0 tx=0).stop 2024-04-25T10:09:40.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.301+0000 7fe9117fd640 1 -- 172.21.15.60:0/2320736812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe90c15b9c0 msgr2=0x7fe90c0ad520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.301+0000 7fe9117fd640 1 --2- 172.21.15.60:0/2320736812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe90c15b9c0 0x7fe90c0ad520 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00c3f0 tx=0x7fe8fc00c8b0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.301+0000 7fe9117fd640 1 -- 172.21.15.60:0/2320736812 shutdown_connections 2024-04-25T10:09:40.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.301+0000 7fe9117fd640 1 --2- 172.21.15.60:0/2320736812 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8f8066f60 0x7fe8f8069420 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.301+0000 7fe9117fd640 1 --2- 172.21.15.60:0/2320736812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe90c15b9c0 0x7fe90c0ad520 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.301+0000 7fe9117fd640 1 --2- 172.21.15.60:0/2320736812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe90c0cf120 0x7fe90c1502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.301+0000 7fe9117fd640 1 --2- 172.21.15.60:0/2320736812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe90c0cc770 0x7fe90c0b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.301+0000 7fe9117fd640 1 -- 172.21.15.60:0/2320736812 >> 172.21.15.60:0/2320736812 conn(0x7fe90c0cf9d0 msgr2=0x7fe90c166d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.305+0000 7fe9117fd640 1 -- 172.21.15.60:0/2320736812 shutdown_connections 2024-04-25T10:09:40.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.305+0000 7fe9117fd640 1 -- 172.21.15.60:0/2320736812 wait complete. 2024-04-25T10:09:40.333 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:09:40.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.445+0000 7f302a6a7640 1 Processor -- start 2024-04-25T10:09:40.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.445+0000 7f302a6a7640 1 -- start start 2024-04-25T10:09:40.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f302a6a7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f30240ae7c0 0x7f3024071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f302a6a7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f302406d2f0 0x7f30240717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f302a6a7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3024071fd0 con 0x7f302415dc90 2024-04-25T10:09:40.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f302a6a7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30240accc0 con 0x7f30240ae7c0 2024-04-25T10:09:40.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f302a6a7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30240ad260 con 0x7f302406d2f0 2024-04-25T10:09:40.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3023fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f302415dc90 0x7f30240b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45120/0 (socket says 172.21.15.60:45120) 2024-04-25T10:09:40.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3023fff640 1 -- 172.21.15.60:0/783010444 learned_addr learned my addr 172.21.15.60:0/783010444 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:40.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3022ffd640 1 -- 172.21.15.60:0/783010444 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404501503 0 0) 0x7f3024071fd0 con 0x7f302415dc90 2024-04-25T10:09:40.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3022ffd640 1 -- 172.21.15.60:0/783010444 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f300c003530 con 0x7f302415dc90 2024-04-25T10:09:40.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f30237fe640 1 --2- 172.21.15.60:0/783010444 >> v2:172.21.15.100:3300/0 conn(0x7f30240ae7c0 0x7f3024071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3028c44640 1 --2- 172.21.15.60:0/783010444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f302406d2f0 0x7f30240717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3022ffd640 1 -- 172.21.15.60:0/783010444 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1908151531 0 0) 0x7f300c003530 con 0x7f302415dc90 2024-04-25T10:09:40.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3022ffd640 1 -- 172.21.15.60:0/783010444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f302406d2f0 msgr2=0x7f30240717d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3022ffd640 1 --2- 172.21.15.60:0/783010444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f302406d2f0 0x7f30240717d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3022ffd640 1 -- 172.21.15.60:0/783010444 >> v2:172.21.15.100:3300/0 conn(0x7f30240ae7c0 msgr2=0x7f3024071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3022ffd640 1 --2- 172.21.15.60:0/783010444 >> v2:172.21.15.100:3300/0 conn(0x7f30240ae7c0 0x7f3024071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3022ffd640 1 -- 172.21.15.60:0/783010444 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30240b2ce0 con 0x7f302415dc90 2024-04-25T10:09:40.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3022ffd640 1 -- 172.21.15.60:0/783010444 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f3018001680 con 0x7f302415dc90 2024-04-25T10:09:40.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3022ffd640 1 -- 172.21.15.60:0/783010444 >> v1:172.21.15.60:6789/0 conn(0x7f302415dc90 legacy=0x7f30240b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3022ffd640 1 --2- 172.21.15.60:0/783010444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f302406d2f0 0x7f300c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3022ffd640 1 --2- 172.21.15.60:0/783010444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f300c003f40 0x7f300c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3022ffd640 1 --2- 172.21.15.60:0/783010444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f300c004d50 0x7f300c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3022ffd640 1 -- 172.21.15.60:0/783010444 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30240ad260 con 0x7f300c003f40 2024-04-25T10:09:40.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3022ffd640 1 -- 172.21.15.60:0/783010444 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c00b9d0 con 0x7f300c004d50 2024-04-25T10:09:40.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3022ffd640 1 -- 172.21.15.60:0/783010444 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300c00bba0 con 0x7f302406d2f0 2024-04-25T10:09:40.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f30237fe640 1 --2- 172.21.15.60:0/783010444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f300c003f40 0x7f300c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3023fff640 1 --2- 172.21.15.60:0/783010444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f302406d2f0 0x7f300c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f3028c44640 1 --2- 172.21.15.60:0/783010444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f300c004d50 0x7f300c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f30237fe640 1 -- 172.21.15.60:0/783010444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f302406d2f0 msgr2=0x7f300c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f30237fe640 1 --2- 172.21.15.60:0/783010444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f302406d2f0 0x7f300c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f30237fe640 1 -- 172.21.15.60:0/783010444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f300c004d50 msgr2=0x7f300c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f30237fe640 1 --2- 172.21.15.60:0/783010444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f300c004d50 0x7f300c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.449+0000 7f30237fe640 1 -- 172.21.15.60:0/783010444 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300c003530 con 0x7f300c003f40 2024-04-25T10:09:40.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f30237fe640 1 --2- 172.21.15.60:0/783010444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f300c003f40 0x7f300c004810 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3014007360 tx=0x7f301401ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1881d25387a137 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f3022ffd640 1 -- 172.21.15.60:0/783010444 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30140b2070 con 0x7f300c003f40 2024-04-25T10:09:40.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f3022ffd640 1 -- 172.21.15.60:0/783010444 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30140a5d80 con 0x7f300c003f40 2024-04-25T10:09:40.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f3022ffd640 1 -- 172.21.15.60:0/783010444 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3014009be0 con 0x7f300c003f40 2024-04-25T10:09:40.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f302a6a7640 1 -- 172.21.15.60:0/783010444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f300c003f40 msgr2=0x7f300c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f302a6a7640 1 --2- 172.21.15.60:0/783010444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f300c003f40 0x7f300c004810 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3014007360 tx=0x7f301401ce80 comp rx=0 tx=0).stop 2024-04-25T10:09:40.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f3028c44640 1 -- 172.21.15.60:0/783010444 reap_dead start 2024-04-25T10:09:40.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f302a6a7640 1 -- 172.21.15.60:0/783010444 shutdown_connections 2024-04-25T10:09:40.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f302a6a7640 1 -- 172.21.15.60:0/783010444 >> 172.21.15.60:0/783010444 conn(0x7f30240b12d0 msgr2=0x7f30240b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f302a6a7640 1 -- 172.21.15.60:0/783010444 shutdown_connections 2024-04-25T10:09:40.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f302a6a7640 1 -- 172.21.15.60:0/783010444 wait complete. 2024-04-25T10:09:40.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f302a6a7640 1 Processor -- start 2024-04-25T10:09:40.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f302a6a7640 1 -- start start 2024-04-25T10:09:40.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f302a6a7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f300c003f40 0x7f3024159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f302a6a7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f302414fdf0 0x7f3024152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f302a6a7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f302415f1f0 0x7f302415f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f302a6a7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30240b2010 con 0x7f300c003f40 2024-04-25T10:09:40.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f302a6a7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30240acef0 con 0x7f302414fdf0 2024-04-25T10:09:40.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f302a6a7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30240b2190 con 0x7f302415f1f0 2024-04-25T10:09:40.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f3023fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f300c003f40 0x7f3024159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f3023fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f300c003f40 0x7f3024159ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44886/0 (socket says 172.21.15.60:44886) 2024-04-25T10:09:40.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f3023fff640 1 -- 172.21.15.60:0/2287264874 learned_addr learned my addr 172.21.15.60:0/2287264874 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:40.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f3023fff640 1 -- 172.21.15.60:0/2287264874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f302415f1f0 msgr2=0x7f302415f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f3023fff640 1 --2- 172.21.15.60:0/2287264874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f302415f1f0 0x7f302415f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f3023fff640 1 -- 172.21.15.60:0/2287264874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f302414fdf0 msgr2=0x7f3024152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f3023fff640 1 --2- 172.21.15.60:0/2287264874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f302414fdf0 0x7f3024152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f3023fff640 1 -- 172.21.15.60:0/2287264874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3024153480 con 0x7f300c003f40 2024-04-25T10:09:40.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.453+0000 7f3023fff640 1 --2- 172.21.15.60:0/2287264874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f300c003f40 0x7f3024159ac0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f30180117d0 tx=0x7f3018011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.457+0000 7f3020ff9640 1 -- 172.21.15.60:0/2287264874 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3018010040 con 0x7f300c003f40 2024-04-25T10:09:40.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.457+0000 7f3020ff9640 1 -- 172.21.15.60:0/2287264874 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301801a070 con 0x7f300c003f40 2024-04-25T10:09:40.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.457+0000 7f302a6a7640 1 -- 172.21.15.60:0/2287264874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302415a1c0 con 0x7f300c003f40 2024-04-25T10:09:40.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.457+0000 7f3020ff9640 1 -- 172.21.15.60:0/2287264874 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301800fd80 con 0x7f300c003f40 2024-04-25T10:09:40.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.457+0000 7f302a6a7640 1 -- 172.21.15.60:0/2287264874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302415a780 con 0x7f300c003f40 2024-04-25T10:09:40.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.457+0000 7f302a6a7640 1 -- 172.21.15.60:0/2287264874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3024074770 con 0x7f300c003f40 2024-04-25T10:09:40.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.457+0000 7f3020ff9640 1 -- 172.21.15.60:0/2287264874 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f301801f460 con 0x7f300c003f40 2024-04-25T10:09:40.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.457+0000 7f3020ff9640 1 --2- 172.21.15.60:0/2287264874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3010066f60 0x7f3010069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.457+0000 7f3020ff9640 1 -- 172.21.15.60:0/2287264874 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3018087e20 con 0x7f300c003f40 2024-04-25T10:09:40.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.457+0000 7f30237fe640 1 --2- 172.21.15.60:0/2287264874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3010066f60 0x7f3010069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.457+0000 7f30237fe640 1 --2- 172.21.15.60:0/2287264874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3010066f60 0x7f3010069420 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f30140af000 tx=0x7f3014004ac0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.461+0000 7f3020ff9640 1 -- 172.21.15.60:0/2287264874 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3018059210 con 0x7f300c003f40 2024-04-25T10:09:40.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.669+0000 7f302a6a7640 1 -- 172.21.15.60:0/2287264874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f30240b27a0 con 0x7f300c003f40 2024-04-25T10:09:40.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.669+0000 7f3020ff9640 1 -- 172.21.15.60:0/2287264874 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3018058b10 con 0x7f300c003f40 2024-04-25T10:09:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:40.679 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:09:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-04-25T10:09:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-04-25T10:09:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmKr12EhAAnHHZ5VjG0cywMrIv6VElVg==", 2024-04-25T10:09:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:09:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:09:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:09:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:09:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-04-25T10:09:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmU39rIRAAk6Ca0tKG4LpmR4pNxjzv0w==", 2024-04-25T10:09:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:09:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:09:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:09:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:09:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.680 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-04-25T10:09:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmVsshGxAAttXv0jYEGKgo0sPhIRTuqQ==", 2024-04-25T10:09:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:09:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:09:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:09:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:09:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-04-25T10:09:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmQtXUKRAAI3Iq02h3KH/PZbWKvJD9Jg==", 2024-04-25T10:09:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:09:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:09:40.681 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:09:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:09:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-04-25T10:09:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCkKipmE+UIBRAAc8ctpHoL3PkkbkMRCAJUGw==", 2024-04-25T10:09:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:09:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:09:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.682 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-04-25T10:09:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCnKipmbAnYCBAALLsPH2FjUpjoIZwTw+8aOA==", 2024-04-25T10:09:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:09:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:09:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-04-25T10:09:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCqKipmqrNSDxAAZdgFrcyeVwRQMKJJYIM98g==", 2024-04-25T10:09:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:09:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:09:40.683 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-04-25T10:09:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCtKipm1L+hFxAAPnPWlCm1Fmi0mN3tyReeQw==", 2024-04-25T10:09:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:09:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:09:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-04-25T10:09:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCzKipm2WyDFRAAmyfg6qu5drE9yMaR2FKd/A==", 2024-04-25T10:09:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.684 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:09:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:09:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-04-25T10:09:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC2KipmZLDhGRAAcgaz8YaPGUMIfxOLtWObyA==", 2024-04-25T10:09:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:09:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:09:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.685 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-04-25T10:09:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC5Kipm/3VMIBAA+rhGCyA/7d+s8m7ZWTzX/Q==", 2024-04-25T10:09:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:09:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:09:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-04-25T10:09:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8KipmFNCCJxAAv0OSTpAG+xIHhLqYXAZazQ==", 2024-04-25T10:09:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:09:40.686 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:09:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-04-25T10:09:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmv65dLxAA6/QpPrqE9aT+CtZqkyutog==", 2024-04-25T10:09:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:09:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-04-25T10:09:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:09:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-04-25T10:09:40.687 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipm2NiONxAAd3X4zx1JnBXKHJvdZ/zMdQ==", 2024-04-25T10:09:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:09:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:09:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:09:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:09:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-04-25T10:09:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmZRONNBAAa+XTv5YIBJWeUvf5dfoWxw==", 2024-04-25T10:09:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:09:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:09:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:09:40.688 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:09:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-04-25T10:09:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCjKipmgvfiABAApLvTC9U6HzeAefCvX4tiSQ==", 2024-04-25T10:09:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:09:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:09:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:09:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:09:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.689 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-04-25T10:09:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQChKipmumzYNhAAlCRYMpSNaTo7bz8P6x0W9g==", 2024-04-25T10:09:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:09:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:09:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:09:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:09:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-04-25T10:09:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmhxlLDxAAegBrJWdMNz6ZC3oXT2Qo0A==", 2024-04-25T10:09:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:09:40.690 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T10:09:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:09:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-04-25T10:09:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmpl1cDBAAGqx2kIak9NoWpkcdHczcFA==", 2024-04-25T10:09:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:09:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:09:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T10:09:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:09:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:40.691 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T10:09:40.691 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:09:40.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.673+0000 7f302a6a7640 1 -- 172.21.15.60:0/2287264874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3010066f60 msgr2=0x7f3010069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.673+0000 7f302a6a7640 1 --2- 172.21.15.60:0/2287264874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3010066f60 0x7f3010069420 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f30140af000 tx=0x7f3014004ac0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.673+0000 7f302a6a7640 1 -- 172.21.15.60:0/2287264874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f300c003f40 msgr2=0x7f3024159ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.673+0000 7f302a6a7640 1 --2- 172.21.15.60:0/2287264874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f300c003f40 0x7f3024159ac0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f30180117d0 tx=0x7f3018011c90 comp rx=0 tx=0).stop 2024-04-25T10:09:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.673+0000 7f302a6a7640 1 -- 172.21.15.60:0/2287264874 shutdown_connections 2024-04-25T10:09:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.673+0000 7f302a6a7640 1 --2- 172.21.15.60:0/2287264874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3010066f60 0x7f3010069420 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.673+0000 7f302a6a7640 1 --2- 172.21.15.60:0/2287264874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f302415f1f0 0x7f302415f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.673+0000 7f302a6a7640 1 --2- 172.21.15.60:0/2287264874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f302414fdf0 0x7f3024152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.673+0000 7f302a6a7640 1 --2- 172.21.15.60:0/2287264874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f300c003f40 0x7f3024159ac0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.673+0000 7f302a6a7640 1 -- 172.21.15.60:0/2287264874 >> 172.21.15.60:0/2287264874 conn(0x7f30240b12d0 msgr2=0x7f30240b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.673+0000 7f302a6a7640 1 -- 172.21.15.60:0/2287264874 shutdown_connections 2024-04-25T10:09:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.673+0000 7f302a6a7640 1 -- 172.21.15.60:0/2287264874 wait complete. 2024-04-25T10:09:40.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd0900aa640 1 Processor -- start 2024-04-25T10:09:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd0900aa640 1 -- start start 2024-04-25T10:09:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd0900aa640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0880d0360 0x7fd0880b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd0900aa640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd0880b19e0 0x7fd0880cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd0900aa640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0880d0860 con 0x7fd08815a700 2024-04-25T10:09:40.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd0900aa640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0880d0b40 con 0x7fd0880d0360 2024-04-25T10:09:40.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd0900aa640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0880d0e20 con 0x7fd0880b19e0 2024-04-25T10:09:40.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd08d645640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0880d0360 0x7fd0880b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd08d645640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0880d0360 0x7fd0880b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44896/0 (socket says 172.21.15.60:44896) 2024-04-25T10:09:40.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd08de46640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd08815a700 0x7fd0880b37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45134/0 (socket says 172.21.15.60:45134) 2024-04-25T10:09:40.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd08d645640 1 -- 172.21.15.60:0/143019460 learned_addr learned my addr 172.21.15.60:0/143019460 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:40.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd08e647640 1 --2- 172.21.15.60:0/143019460 >> v2:172.21.15.100:3300/0 conn(0x7fd0880b19e0 0x7fd0880cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd08d645640 1 -- 172.21.15.60:0/143019460 >> v2:172.21.15.100:3300/0 conn(0x7fd0880b19e0 msgr2=0x7fd0880cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd08d645640 1 --2- 172.21.15.60:0/143019460 >> v2:172.21.15.100:3300/0 conn(0x7fd0880b19e0 0x7fd0880cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd08d645640 1 -- 172.21.15.60:0/143019460 >> v1:172.21.15.60:6789/0 conn(0x7fd08815a700 legacy=0x7fd0880b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd08d645640 1 -- 172.21.15.60:0/143019460 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0880d1100 con 0x7fd0880d0360 2024-04-25T10:09:40.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.813+0000 7fd08ce44640 1 -- 172.21.15.60:0/143019460 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511767986 0 0) 0x7fd0880d0860 con 0x7fd08815a700 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.817+0000 7fd08d645640 1 --2- 172.21.15.60:0/143019460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0880d0360 0x7fd0880b14a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd088071df0 tx=0x7fd07801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bb933bb292514a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.817+0000 7fd08ce44640 1 -- 172.21.15.60:0/143019460 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0780b0070 con 0x7fd0880d0360 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.817+0000 7fd08ce44640 1 -- 172.21.15.60:0/143019460 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0780ab600 con 0x7fd0880d0360 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.817+0000 7fd08ce44640 1 -- 172.21.15.60:0/143019460 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0780ab9d0 con 0x7fd0880d0360 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.817+0000 7fd0900aa640 1 -- 172.21.15.60:0/143019460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0880d0360 msgr2=0x7fd0880b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.817+0000 7fd0900aa640 1 --2- 172.21.15.60:0/143019460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0880d0360 0x7fd0880b14a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd088071df0 tx=0x7fd07801c520 comp rx=0 tx=0).stop 2024-04-25T10:09:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.817+0000 7fd0900aa640 1 -- 172.21.15.60:0/143019460 shutdown_connections 2024-04-25T10:09:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.817+0000 7fd0900aa640 1 --2- 172.21.15.60:0/143019460 >> v2:172.21.15.100:3300/0 conn(0x7fd0880b19e0 0x7fd0880cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.817+0000 7fd0900aa640 1 --2- 172.21.15.60:0/143019460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0880d0360 0x7fd0880b14a0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.817+0000 7fd0900aa640 1 -- 172.21.15.60:0/143019460 >> 172.21.15.60:0/143019460 conn(0x7fd088072920 msgr2=0x7fd088099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.817+0000 7fd0900aa640 1 -- 172.21.15.60:0/143019460 shutdown_connections 2024-04-25T10:09:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.817+0000 7fd0900aa640 1 -- 172.21.15.60:0/143019460 wait complete. 2024-04-25T10:09:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd0900aa640 1 Processor -- start 2024-04-25T10:09:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd0900aa640 1 -- start start 2024-04-25T10:09:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd0900aa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0880b19e0 0x7fd088154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd0900aa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0880d0360 0x7fd088147a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd0900aa640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd08815a700 0x7fd088147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd0900aa640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0880ce340 con 0x7fd0880b19e0 2024-04-25T10:09:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd0900aa640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0880ce1c0 con 0x7fd0880d0360 2024-04-25T10:09:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd0900aa640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0880ce4c0 con 0x7fd08815a700 2024-04-25T10:09:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd08de46640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0880b19e0 0x7fd088154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd08de46640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0880b19e0 0x7fd088154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44904/0 (socket says 172.21.15.60:44904) 2024-04-25T10:09:40.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd08de46640 1 -- 172.21.15.60:0/2945660308 learned_addr learned my addr 172.21.15.60:0/2945660308 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:40.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd08d645640 1 --2- 172.21.15.60:0/2945660308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0880d0360 0x7fd088147a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd08e647640 1 --2- 172.21.15.60:0/2945660308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd08815a700 0x7fd088147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd08de46640 1 -- 172.21.15.60:0/2945660308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd08815a700 msgr2=0x7fd088147fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd08de46640 1 --2- 172.21.15.60:0/2945660308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd08815a700 0x7fd088147fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd08de46640 1 -- 172.21.15.60:0/2945660308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0880d0360 msgr2=0x7fd088147a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd08de46640 1 --2- 172.21.15.60:0/2945660308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0880d0360 0x7fd088147a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd08de46640 1 -- 172.21.15.60:0/2945660308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0881527d0 con 0x7fd0880b19e0 2024-04-25T10:09:40.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd08de46640 1 --2- 172.21.15.60:0/2945660308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0880b19e0 0x7fd088154240 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd084001570 tx=0x7fd08400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd07e7fc640 1 -- 172.21.15.60:0/2945660308 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084018070 con 0x7fd0880b19e0 2024-04-25T10:09:40.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd07e7fc640 1 -- 172.21.15.60:0/2945660308 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08400a640 con 0x7fd0880b19e0 2024-04-25T10:09:40.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd07e7fc640 1 -- 172.21.15.60:0/2945660308 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08400aa30 con 0x7fd0880b19e0 2024-04-25T10:09:40.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd0900aa640 1 -- 172.21.15.60:0/2945660308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd088152ab0 con 0x7fd0880b19e0 2024-04-25T10:09:40.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd0900aa640 1 -- 172.21.15.60:0/2945660308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd088153070 con 0x7fd0880b19e0 2024-04-25T10:09:40.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.821+0000 7fd0900aa640 1 -- 172.21.15.60:0/2945660308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd088152520 con 0x7fd0880b19e0 2024-04-25T10:09:40.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.825+0000 7fd07e7fc640 1 -- 172.21.15.60:0/2945660308 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd08400abd0 con 0x7fd0880b19e0 2024-04-25T10:09:40.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.825+0000 7fd07e7fc640 1 --2- 172.21.15.60:0/2945660308 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd06c066f60 0x7fd06c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.825+0000 7fd07e7fc640 1 -- 172.21.15.60:0/2945660308 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd084084310 con 0x7fd0880b19e0 2024-04-25T10:09:40.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.825+0000 7fd08d645640 1 --2- 172.21.15.60:0/2945660308 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd06c066f60 0x7fd06c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.825+0000 7fd08d645640 1 --2- 172.21.15.60:0/2945660308 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd06c066f60 0x7fd06c069420 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd0780ad000 tx=0x7fd0780a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.829+0000 7fd07e7fc640 1 -- 172.21.15.60:0/2945660308 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd084055670 con 0x7fd0880b19e0 2024-04-25T10:09:40.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.989+0000 7fd0900aa640 1 -- 172.21.15.60:0/2945660308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd088074770 con 0x7fd0880b19e0 2024-04-25T10:09:40.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.989+0000 7fd07e7fc640 1 -- 172.21.15.60:0/2945660308 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+12903 (secure 0 0 0) 0x7fd084054f70 con 0x7fd0880b19e0 2024-04-25T10:09:41.000 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:41.000 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":65,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:09:40.314861+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:41.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.993+0000 7fd0900aa640 1 -- 172.21.15.60:0/2945660308 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd06c066f60 msgr2=0x7fd06c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.993+0000 7fd0900aa640 1 --2- 172.21.15.60:0/2945660308 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd06c066f60 0x7fd06c069420 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd0780ad000 tx=0x7fd0780a6000 comp rx=0 tx=0).stop 2024-04-25T10:09:41.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.993+0000 7fd0900aa640 1 -- 172.21.15.60:0/2945660308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0880b19e0 msgr2=0x7fd088154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.993+0000 7fd0900aa640 1 --2- 172.21.15.60:0/2945660308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0880b19e0 0x7fd088154240 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd084001570 tx=0x7fd08400cd20 comp rx=0 tx=0).stop 2024-04-25T10:09:41.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.993+0000 7fd0900aa640 1 -- 172.21.15.60:0/2945660308 shutdown_connections 2024-04-25T10:09:41.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.993+0000 7fd0900aa640 1 --2- 172.21.15.60:0/2945660308 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd06c066f60 0x7fd06c069420 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.993+0000 7fd0900aa640 1 --2- 172.21.15.60:0/2945660308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd08815a700 0x7fd088147fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.993+0000 7fd0900aa640 1 --2- 172.21.15.60:0/2945660308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0880d0360 0x7fd088147a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.993+0000 7fd0900aa640 1 --2- 172.21.15.60:0/2945660308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0880b19e0 0x7fd088154240 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.993+0000 7fd0900aa640 1 -- 172.21.15.60:0/2945660308 >> 172.21.15.60:0/2945660308 conn(0x7fd088072920 msgr2=0x7fd088157280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:41.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.993+0000 7fd0900aa640 1 -- 172.21.15.60:0/2945660308 shutdown_connections 2024-04-25T10:09:41.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:40.993+0000 7fd0900aa640 1 -- 172.21.15.60:0/2945660308 wait complete. 2024-04-25T10:09:41.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:41.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:41.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe7a6162640 1 Processor -- start 2024-04-25T10:09:41.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe7a6162640 1 -- start start 2024-04-25T10:09:41.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe7a6162640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7a015ef20 0x7fe7a00b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe7a6162640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe7a00cfcf0 0x7fe7a00b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe7a6162640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe7a0099a30 0x7fe7a0099ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe7a6162640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a00cc320 con 0x7fe7a0099a30 2024-04-25T10:09:41.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe7a6162640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a00cc5c0 con 0x7fe7a00cfcf0 2024-04-25T10:09:41.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe7a6162640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00d07b0 con 0x7fe7a015ef20 2024-04-25T10:09:41.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe79ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe7a0099a30 0x7fe7a0099ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe79ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe7a0099a30 0x7fe7a0099ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44908/0 (socket says 172.21.15.60:44908) 2024-04-25T10:09:41.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe79ffff640 1 -- 172.21.15.60:0/3503079790 learned_addr learned my addr 172.21.15.60:0/3503079790 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:41.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe79f7fe640 1 --2- 172.21.15.60:0/3503079790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7a015ef20 0x7fe7a00b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe79effd640 1 --2- 172.21.15.60:0/3503079790 >> v2:172.21.15.100:3300/0 conn(0x7fe7a00cfcf0 0x7fe7a00b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe79ffff640 1 -- 172.21.15.60:0/3503079790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7a015ef20 msgr2=0x7fe7a00b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:41.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe79ffff640 1 --2- 172.21.15.60:0/3503079790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7a015ef20 0x7fe7a00b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe79ffff640 1 -- 172.21.15.60:0/3503079790 >> v2:172.21.15.100:3300/0 conn(0x7fe7a00cfcf0 msgr2=0x7fe7a00b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:41.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe79ffff640 1 --2- 172.21.15.60:0/3503079790 >> v2:172.21.15.100:3300/0 conn(0x7fe7a00cfcf0 0x7fe7a00b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.137+0000 7fe79ffff640 1 -- 172.21.15.60:0/3503079790 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00d04d0 con 0x7fe7a0099a30 2024-04-25T10:09:41.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe79ffff640 1 --2- 172.21.15.60:0/3503079790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7a0099a30 0x7fe7a0099ec0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0072b10 tx=0x7fe79801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ac0e51c511d9a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:41.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe79e7fc640 1 -- 172.21.15.60:0/3503079790 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7980a5670 con 0x7fe7a0099a30 2024-04-25T10:09:41.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe79e7fc640 1 -- 172.21.15.60:0/3503079790 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7980a5810 con 0x7fe7a0099a30 2024-04-25T10:09:41.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe79e7fc640 1 -- 172.21.15.60:0/3503079790 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7980a5be0 con 0x7fe7a0099a30 2024-04-25T10:09:41.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 -- 172.21.15.60:0/3503079790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7a0099a30 msgr2=0x7fe7a0099ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 --2- 172.21.15.60:0/3503079790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7a0099a30 0x7fe7a0099ec0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0072b10 tx=0x7fe79801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 -- 172.21.15.60:0/3503079790 shutdown_connections 2024-04-25T10:09:41.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 --2- 172.21.15.60:0/3503079790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7a0099a30 0x7fe7a0099ec0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 --2- 172.21.15.60:0/3503079790 >> v2:172.21.15.100:3300/0 conn(0x7fe7a00cfcf0 0x7fe7a00b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 --2- 172.21.15.60:0/3503079790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7a015ef20 0x7fe7a00b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 -- 172.21.15.60:0/3503079790 >> 172.21.15.60:0/3503079790 conn(0x7fe7a00b3200 msgr2=0x7fe7a00b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:41.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 -- 172.21.15.60:0/3503079790 shutdown_connections 2024-04-25T10:09:41.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 -- 172.21.15.60:0/3503079790 wait complete. 2024-04-25T10:09:41.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 Processor -- start 2024-04-25T10:09:41.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 -- start start 2024-04-25T10:09:41.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7a0099a30 0x7fe7a00b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7a00cfcf0 0x7fe7a00c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7a015ef20 0x7fe7a00c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00d0250 con 0x7fe7a0099a30 2024-04-25T10:09:41.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00d00d0 con 0x7fe7a015ef20 2024-04-25T10:09:41.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe7a6162640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a015f300 con 0x7fe7a00cfcf0 2024-04-25T10:09:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe79f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7a0099a30 0x7fe7a00b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe79f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7a0099a30 0x7fe7a00b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44914/0 (socket says 172.21.15.60:44914) 2024-04-25T10:09:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.141+0000 7fe79f7fe640 1 -- 172.21.15.60:0/1904063581 learned_addr learned my addr 172.21.15.60:0/1904063581 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe79effd640 1 --2- 172.21.15.60:0/1904063581 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7a00cfcf0 0x7fe7a00c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe79ffff640 1 --2- 172.21.15.60:0/1904063581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7a015ef20 0x7fe7a00c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe79f7fe640 1 -- 172.21.15.60:0/1904063581 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7a00cfcf0 msgr2=0x7fe7a00c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe79f7fe640 1 --2- 172.21.15.60:0/1904063581 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7a00cfcf0 0x7fe7a00c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe79f7fe640 1 -- 172.21.15.60:0/1904063581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7a015ef20 msgr2=0x7fe7a00c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe79f7fe640 1 --2- 172.21.15.60:0/1904063581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7a015ef20 0x7fe7a00c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe79f7fe640 1 -- 172.21.15.60:0/1904063581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00b5f00 con 0x7fe7a0099a30 2024-04-25T10:09:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe79f7fe640 1 --2- 172.21.15.60:0/1904063581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7a0099a30 0x7fe7a00b5520 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fe79000d5f0 tx=0x7fe79000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe787fff640 1 -- 172.21.15.60:0/1904063581 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7900125f0 con 0x7fe7a0099a30 2024-04-25T10:09:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe787fff640 1 -- 172.21.15.60:0/1904063581 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe790019070 con 0x7fe7a0099a30 2024-04-25T10:09:41.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe7a6162640 1 -- 172.21.15.60:0/1904063581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a00b61e0 con 0x7fe7a0099a30 2024-04-25T10:09:41.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe787fff640 1 -- 172.21.15.60:0/1904063581 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7900129b0 con 0x7fe7a0099a30 2024-04-25T10:09:41.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe7a6162640 1 -- 172.21.15.60:0/1904063581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a00b67a0 con 0x7fe7a0099a30 2024-04-25T10:09:41.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe7a6162640 1 -- 172.21.15.60:0/1904063581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a0074770 con 0x7fe7a0099a30 2024-04-25T10:09:41.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe787fff640 1 -- 172.21.15.60:0/1904063581 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe790012b50 con 0x7fe7a0099a30 2024-04-25T10:09:41.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe787fff640 1 --2- 172.21.15.60:0/1904063581 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe78c066f60 0x7fe78c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe787fff640 1 -- 172.21.15.60:0/1904063581 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe790085210 con 0x7fe7a0099a30 2024-04-25T10:09:41.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe79effd640 1 --2- 172.21.15.60:0/1904063581 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe78c066f60 0x7fe78c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.145+0000 7fe79effd640 1 --2- 172.21.15.60:0/1904063581 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe78c066f60 0x7fe78c069420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0157590 tx=0x7fe794008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:41.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.149+0000 7fe787fff640 1 -- 172.21.15.60:0/1904063581 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe790056450 con 0x7fe7a0099a30 2024-04-25T10:09:41.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.313+0000 7fe7a6162640 1 -- 172.21.15.60:0/1904063581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7a00b25d0 con 0x7fe7a0099a30 2024-04-25T10:09:41.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.313+0000 7fe787fff640 1 -- 172.21.15.60:0/1904063581 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe790055d50 con 0x7fe7a0099a30 2024-04-25T10:09:41.323 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:09:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.317+0000 7fe7a6162640 1 -- 172.21.15.60:0/1904063581 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe78c066f60 msgr2=0x7fe78c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.317+0000 7fe7a6162640 1 --2- 172.21.15.60:0/1904063581 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe78c066f60 0x7fe78c069420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0157590 tx=0x7fe794008040 comp rx=0 tx=0).stop 2024-04-25T10:09:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.317+0000 7fe7a6162640 1 -- 172.21.15.60:0/1904063581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7a0099a30 msgr2=0x7fe7a00b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.317+0000 7fe7a6162640 1 --2- 172.21.15.60:0/1904063581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7a0099a30 0x7fe7a00b5520 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fe79000d5f0 tx=0x7fe79000dab0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.317+0000 7fe7a6162640 1 -- 172.21.15.60:0/1904063581 shutdown_connections 2024-04-25T10:09:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.317+0000 7fe7a6162640 1 --2- 172.21.15.60:0/1904063581 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe78c066f60 0x7fe78c069420 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.317+0000 7fe7a6162640 1 --2- 172.21.15.60:0/1904063581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7a015ef20 0x7fe7a00c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.317+0000 7fe7a6162640 1 --2- 172.21.15.60:0/1904063581 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7a00cfcf0 0x7fe7a00c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.317+0000 7fe7a6162640 1 --2- 172.21.15.60:0/1904063581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7a0099a30 0x7fe7a00b5520 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.317+0000 7fe7a6162640 1 -- 172.21.15.60:0/1904063581 >> 172.21.15.60:0/1904063581 conn(0x7fe7a00b3200 msgr2=0x7fe7a00ce810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:41.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.317+0000 7fe7a6162640 1 -- 172.21.15.60:0/1904063581 shutdown_connections 2024-04-25T10:09:41.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.317+0000 7fe7a6162640 1 -- 172.21.15.60:0/1904063581 wait complete. 2024-04-25T10:09:41.343 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:09:41.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd28e1ed640 1 Processor -- start 2024-04-25T10:09:41.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd28e1ed640 1 -- start start 2024-04-25T10:09:41.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd28e1ed640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd28815a7a0 0x7fd2880cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd28e1ed640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd28806ec30 0x7fd288071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd28e1ed640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2880721e0 con 0x7fd2880b37e0 2024-04-25T10:09:41.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd28e1ed640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2880cdaa0 con 0x7fd28806ec30 2024-04-25T10:09:41.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd28e1ed640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28806d2f0 con 0x7fd28815a7a0 2024-04-25T10:09:41.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd27effd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd28806ec30 0x7fd288071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd2877fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd28815a7a0 0x7fd2880cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd2877fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd28815a7a0 0x7fd2880cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:44158/0 (socket says 172.21.15.60:44158) 2024-04-25T10:09:41.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd2877fe640 1 -- 172.21.15.60:0/4001193129 learned_addr learned my addr 172.21.15.60:0/4001193129 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:41.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd27effd640 1 --2- 172.21.15.60:0/4001193129 >> v2:172.21.15.100:3300/0 conn(0x7fd28806ec30 0x7fd288071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:57886/0 (socket says 172.21.15.60:57886) 2024-04-25T10:09:41.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd287fff640 1 --1- 172.21.15.60:0/4001193129 >> v1:172.21.15.100:6789/0 conn(0x7fd2880b37e0 0x7fd28806dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:54072/0 (socket says 172.21.15.60:54072) 2024-04-25T10:09:41.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd27effd640 1 -- 172.21.15.60:0/4001193129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd28815a7a0 msgr2=0x7fd2880cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:41.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd27effd640 1 --2- 172.21.15.60:0/4001193129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd28815a7a0 0x7fd2880cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd27effd640 1 -- 172.21.15.60:0/4001193129 >> v1:172.21.15.100:6789/0 conn(0x7fd2880b37e0 legacy=0x7fd28806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd27effd640 1 -- 172.21.15.60:0/4001193129 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28806f130 con 0x7fd28806ec30 2024-04-25T10:09:41.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.457+0000 7fd2877fe640 1 --2- 172.21.15.60:0/4001193129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd28815a7a0 0x7fd2880cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:41.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.461+0000 7fd27effd640 1 --2- 172.21.15.60:0/4001193129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd28806ec30 0x7fd288071ca0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd2700097c0 tx=0x7fd27001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca01d382d3c64aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:41.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.461+0000 7fd286ffd640 1 -- 172.21.15.60:0/4001193129 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2700b0070 con 0x7fd28806ec30 2024-04-25T10:09:41.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.461+0000 7fd286ffd640 1 -- 172.21.15.60:0/4001193129 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2700ab600 con 0x7fd28806ec30 2024-04-25T10:09:41.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.461+0000 7fd286ffd640 1 -- 172.21.15.60:0/4001193129 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2700ab9d0 con 0x7fd28806ec30 2024-04-25T10:09:41.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.461+0000 7fd28e1ed640 1 -- 172.21.15.60:0/4001193129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd28806ec30 msgr2=0x7fd288071ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.461+0000 7fd28e1ed640 1 --2- 172.21.15.60:0/4001193129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd28806ec30 0x7fd288071ca0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd2700097c0 tx=0x7fd27001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.461+0000 7fd28e1ed640 1 -- 172.21.15.60:0/4001193129 shutdown_connections 2024-04-25T10:09:41.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.461+0000 7fd28e1ed640 1 --2- 172.21.15.60:0/4001193129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd28806ec30 0x7fd288071ca0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.461+0000 7fd28e1ed640 1 --2- 172.21.15.60:0/4001193129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd28815a7a0 0x7fd2880cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.461+0000 7fd28e1ed640 1 -- 172.21.15.60:0/4001193129 >> 172.21.15.60:0/4001193129 conn(0x7fd2880b12d0 msgr2=0x7fd2880b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:41.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.461+0000 7fd28e1ed640 1 -- 172.21.15.60:0/4001193129 shutdown_connections 2024-04-25T10:09:41.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.461+0000 7fd28e1ed640 1 -- 172.21.15.60:0/4001193129 wait complete. 2024-04-25T10:09:41.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.461+0000 7fd28e1ed640 1 Processor -- start 2024-04-25T10:09:41.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.461+0000 7fd28e1ed640 1 -- start start 2024-04-25T10:09:41.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd28e1ed640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd28806ec30 0x7fd288153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd28e1ed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2880b37e0 0x7fd288150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd28e1ed640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd28815a7a0 0x7fd288146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd28e1ed640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28815ad00 con 0x7fd2880b37e0 2024-04-25T10:09:41.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd28e1ed640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28815ab80 con 0x7fd28815a7a0 2024-04-25T10:09:41.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd28e1ed640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28806e860 con 0x7fd28806ec30 2024-04-25T10:09:41.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd27effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2880b37e0 0x7fd288150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd27effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2880b37e0 0x7fd288150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44928/0 (socket says 172.21.15.60:44928) 2024-04-25T10:09:41.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd27effd640 1 -- 172.21.15.60:0/2867608536 learned_addr learned my addr 172.21.15.60:0/2867608536 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:41.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd2877fe640 1 --2- 172.21.15.60:0/2867608536 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd28806ec30 0x7fd288153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd287fff640 1 --2- 172.21.15.60:0/2867608536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd28815a7a0 0x7fd288146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd27effd640 1 -- 172.21.15.60:0/2867608536 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd28806ec30 msgr2=0x7fd288153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd27effd640 1 --2- 172.21.15.60:0/2867608536 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd28806ec30 0x7fd288153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd27effd640 1 -- 172.21.15.60:0/2867608536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd28815a7a0 msgr2=0x7fd288146600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd27effd640 1 --2- 172.21.15.60:0/2867608536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd28815a7a0 0x7fd288146600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd27effd640 1 -- 172.21.15.60:0/2867608536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28815c5f0 con 0x7fd2880b37e0 2024-04-25T10:09:41.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd27effd640 1 --2- 172.21.15.60:0/2867608536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2880b37e0 0x7fd288150f90 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd27001a0d0 tx=0x7fd27001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:41.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd284ff9640 1 -- 172.21.15.60:0/2867608536 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2700b0050 con 0x7fd2880b37e0 2024-04-25T10:09:41.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd284ff9640 1 -- 172.21.15.60:0/2867608536 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2700bc070 con 0x7fd2880b37e0 2024-04-25T10:09:41.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd28e1ed640 1 -- 172.21.15.60:0/2867608536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28815c890 con 0x7fd2880b37e0 2024-04-25T10:09:41.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd284ff9640 1 -- 172.21.15.60:0/2867608536 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2700b5830 con 0x7fd2880b37e0 2024-04-25T10:09:41.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd28e1ed640 1 -- 172.21.15.60:0/2867608536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd28815ce50 con 0x7fd2880b37e0 2024-04-25T10:09:41.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd28e1ed640 1 -- 172.21.15.60:0/2867608536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd288074770 con 0x7fd2880b37e0 2024-04-25T10:09:41.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd284ff9640 1 -- 172.21.15.60:0/2867608536 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2700b59d0 con 0x7fd2880b37e0 2024-04-25T10:09:41.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd284ff9640 1 --2- 172.21.15.60:0/2867608536 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd268066f60 0x7fd268069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd284ff9640 1 -- 172.21.15.60:0/2867608536 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd27011f220 con 0x7fd2880b37e0 2024-04-25T10:09:41.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.465+0000 7fd2877fe640 1 --2- 172.21.15.60:0/2867608536 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd268066f60 0x7fd268069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.469+0000 7fd2877fe640 1 --2- 172.21.15.60:0/2867608536 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd268066f60 0x7fd268069420 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd27800b240 tx=0x7fd27800b450 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:41.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.469+0000 7fd284ff9640 1 -- 172.21.15.60:0/2867608536 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd270016090 con 0x7fd2880b37e0 2024-04-25T10:09:41.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.633+0000 7fd28e1ed640 1 -- 172.21.15.60:0/2867608536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd28806c250 con 0x7fd2880b37e0 2024-04-25T10:09:41.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.633+0000 7fd284ff9640 1 -- 172.21.15.60:0/2867608536 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v65) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd2700f0580 con 0x7fd2880b37e0 2024-04-25T10:09:41.643 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:41.643 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T10:09:41.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:09:41.643 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T10:09:41.643 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T10:09:41.643 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:41.643 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T10:09:41.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.637+0000 7fd28e1ed640 1 -- 172.21.15.60:0/2867608536 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd268066f60 msgr2=0x7fd268069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.637+0000 7fd28e1ed640 1 --2- 172.21.15.60:0/2867608536 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd268066f60 0x7fd268069420 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd27800b240 tx=0x7fd27800b450 comp rx=0 tx=0).stop 2024-04-25T10:09:41.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.637+0000 7fd28e1ed640 1 -- 172.21.15.60:0/2867608536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2880b37e0 msgr2=0x7fd288150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.637+0000 7fd28e1ed640 1 --2- 172.21.15.60:0/2867608536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2880b37e0 0x7fd288150f90 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd27001a0d0 tx=0x7fd27001ace0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.637+0000 7fd28e1ed640 1 -- 172.21.15.60:0/2867608536 shutdown_connections 2024-04-25T10:09:41.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.637+0000 7fd28e1ed640 1 --2- 172.21.15.60:0/2867608536 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd268066f60 0x7fd268069420 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.637+0000 7fd28e1ed640 1 --2- 172.21.15.60:0/2867608536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd28815a7a0 0x7fd288146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.637+0000 7fd28e1ed640 1 --2- 172.21.15.60:0/2867608536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2880b37e0 0x7fd288150f90 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.637+0000 7fd28e1ed640 1 --2- 172.21.15.60:0/2867608536 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd28806ec30 0x7fd288153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.637+0000 7fd28e1ed640 1 -- 172.21.15.60:0/2867608536 >> 172.21.15.60:0/2867608536 conn(0x7fd2880b12d0 msgr2=0x7fd288071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:41.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.637+0000 7fd28e1ed640 1 -- 172.21.15.60:0/2867608536 shutdown_connections 2024-04-25T10:09:41.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.637+0000 7fd28e1ed640 1 -- 172.21.15.60:0/2867608536 wait complete. 2024-04-25T10:09:41.663 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:09:41.663 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:09:41.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.785+0000 7fb53c533640 1 Processor -- start 2024-04-25T10:09:41.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.785+0000 7fb53c533640 1 -- start start 2024-04-25T10:09:41.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.785+0000 7fb53c533640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb534070990 0x7fb53415c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.785+0000 7fb53c533640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb53415cb70 0x7fb53406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.785+0000 7fb53c533640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb53415be50 con 0x7fb5340afc10 2024-04-25T10:09:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.785+0000 7fb53c533640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5340c57c0 con 0x7fb53415cb70 2024-04-25T10:09:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.785+0000 7fb53c533640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53415c1a0 con 0x7fb534070990 2024-04-25T10:09:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.785+0000 7fb53aad0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb53415cb70 0x7fb53406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.785+0000 7fb53aad0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb53415cb70 0x7fb53406ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44934/0 (socket says 172.21.15.60:44934) 2024-04-25T10:09:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.785+0000 7fb53aad0640 1 -- 172.21.15.60:0/1716833958 learned_addr learned my addr 172.21.15.60:0/1716833958 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.785+0000 7fb539ace640 1 --2- 172.21.15.60:0/1716833958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb534070990 0x7fb53415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.785+0000 7fb53aad0640 1 -- 172.21.15.60:0/1716833958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb534070990 msgr2=0x7fb53415c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.785+0000 7fb53aad0640 1 --2- 172.21.15.60:0/1716833958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb534070990 0x7fb53415c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.785+0000 7fb53aad0640 1 -- 172.21.15.60:0/1716833958 >> v1:172.21.15.100:6789/0 conn(0x7fb5340afc10 legacy=0x7fb534140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.785+0000 7fb53aad0640 1 -- 172.21.15.60:0/1716833958 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb534141730 con 0x7fb53415cb70 2024-04-25T10:09:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53aad0640 1 --2- 172.21.15.60:0/1716833958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb53415cb70 0x7fb53406ac10 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb530009870 tx=0x7fb53001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdbe2b43a955bca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb5392cd640 1 -- 172.21.15.60:0/1716833958 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300b0070 con 0x7fb53415cb70 2024-04-25T10:09:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb5392cd640 1 -- 172.21.15.60:0/1716833958 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5300ab600 con 0x7fb53415cb70 2024-04-25T10:09:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb5392cd640 1 -- 172.21.15.60:0/1716833958 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300ab9d0 con 0x7fb53415cb70 2024-04-25T10:09:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53c533640 1 -- 172.21.15.60:0/1716833958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb53415cb70 msgr2=0x7fb53406ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53c533640 1 --2- 172.21.15.60:0/1716833958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb53415cb70 0x7fb53406ac10 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb530009870 tx=0x7fb53001c520 comp rx=0 tx=0).stop 2024-04-25T10:09:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53c533640 1 -- 172.21.15.60:0/1716833958 shutdown_connections 2024-04-25T10:09:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53c533640 1 --2- 172.21.15.60:0/1716833958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb53415cb70 0x7fb53406ac10 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53c533640 1 --2- 172.21.15.60:0/1716833958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb534070990 0x7fb53415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53c533640 1 -- 172.21.15.60:0/1716833958 >> 172.21.15.60:0/1716833958 conn(0x7fb53406e400 msgr2=0x7fb53406e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53c533640 1 -- 172.21.15.60:0/1716833958 shutdown_connections 2024-04-25T10:09:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53c533640 1 -- 172.21.15.60:0/1716833958 wait complete. 2024-04-25T10:09:41.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53c533640 1 Processor -- start 2024-04-25T10:09:41.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53c533640 1 -- start start 2024-04-25T10:09:41.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53c533640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb534070990 0x7fb53413a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53c533640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5340afc10 0x7fb534155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53c533640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5341316c0 0x7fb53413b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53c533640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5340ce5f0 con 0x7fb5341316c0 2024-04-25T10:09:41.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53c533640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5340b0200 con 0x7fb5340afc10 2024-04-25T10:09:41.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53c533640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5340ce770 con 0x7fb534070990 2024-04-25T10:09:41.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53aad0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5341316c0 0x7fb53413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53aad0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5341316c0 0x7fb53413b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44940/0 (socket says 172.21.15.60:44940) 2024-04-25T10:09:41.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53aad0640 1 -- 172.21.15.60:0/2506317341 learned_addr learned my addr 172.21.15.60:0/2506317341 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:41.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53a2cf640 1 --2- 172.21.15.60:0/2506317341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb534070990 0x7fb53413a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb539ace640 1 --2- 172.21.15.60:0/2506317341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5340afc10 0x7fb534155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53aad0640 1 -- 172.21.15.60:0/2506317341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb534070990 msgr2=0x7fb53413a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53aad0640 1 --2- 172.21.15.60:0/2506317341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb534070990 0x7fb53413a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53aad0640 1 -- 172.21.15.60:0/2506317341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5340afc10 msgr2=0x7fb534155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53aad0640 1 --2- 172.21.15.60:0/2506317341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5340afc10 0x7fb534155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.789+0000 7fb53aad0640 1 -- 172.21.15.60:0/2506317341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53413ee60 con 0x7fb5341316c0 2024-04-25T10:09:41.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.793+0000 7fb53aad0640 1 --2- 172.21.15.60:0/2506317341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5341316c0 0x7fb53413b740 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb53001cdc0 tx=0x7fb5300192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:41.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.793+0000 7fb526ffd640 1 -- 172.21.15.60:0/2506317341 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300b0050 con 0x7fb5341316c0 2024-04-25T10:09:41.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.793+0000 7fb526ffd640 1 -- 172.21.15.60:0/2506317341 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5300bc070 con 0x7fb5341316c0 2024-04-25T10:09:41.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.793+0000 7fb53c533640 1 -- 172.21.15.60:0/2506317341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53413f0d0 con 0x7fb5341316c0 2024-04-25T10:09:41.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.793+0000 7fb526ffd640 1 -- 172.21.15.60:0/2506317341 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300b5850 con 0x7fb5341316c0 2024-04-25T10:09:41.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.793+0000 7fb53c533640 1 -- 172.21.15.60:0/2506317341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53413f690 con 0x7fb5341316c0 2024-04-25T10:09:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.793+0000 7fb53c533640 1 -- 172.21.15.60:0/2506317341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb534074770 con 0x7fb5341316c0 2024-04-25T10:09:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.793+0000 7fb526ffd640 1 -- 172.21.15.60:0/2506317341 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5300b59f0 con 0x7fb5341316c0 2024-04-25T10:09:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.793+0000 7fb526ffd640 1 --2- 172.21.15.60:0/2506317341 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb518066f60 0x7fb518069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.793+0000 7fb526ffd640 1 -- 172.21.15.60:0/2506317341 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb53011f4f0 con 0x7fb5341316c0 2024-04-25T10:09:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.793+0000 7fb53a2cf640 1 --2- 172.21.15.60:0/2506317341 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb518066f60 0x7fb518069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.793+0000 7fb53a2cf640 1 --2- 172.21.15.60:0/2506317341 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb518066f60 0x7fb518069420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb52c005530 tx=0x7fb52c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:41.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.797+0000 7fb526ffd640 1 -- 172.21.15.60:0/2506317341 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb530016110 con 0x7fb5341316c0 2024-04-25T10:09:41.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:41.961+0000 7fb53c533640 1 -- 172.21.15.60:0/2506317341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb53406fcf0 con 0x7fb5341316c0 2024-04-25T10:09:42.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.317+0000 7fb526ffd640 1 -- 172.21.15.60:0/2506317341 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v66) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb5300f0850 con 0x7fb5341316c0 2024-04-25T10:09:42.328 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T10:09:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.325+0000 7fb53c533640 1 -- 172.21.15.60:0/2506317341 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb518066f60 msgr2=0x7fb518069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.325+0000 7fb53c533640 1 --2- 172.21.15.60:0/2506317341 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb518066f60 0x7fb518069420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb52c005530 tx=0x7fb52c009040 comp rx=0 tx=0).stop 2024-04-25T10:09:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.325+0000 7fb53c533640 1 -- 172.21.15.60:0/2506317341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5341316c0 msgr2=0x7fb53413b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.325+0000 7fb53c533640 1 --2- 172.21.15.60:0/2506317341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5341316c0 0x7fb53413b740 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb53001cdc0 tx=0x7fb5300192c0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.325+0000 7fb53c533640 1 -- 172.21.15.60:0/2506317341 shutdown_connections 2024-04-25T10:09:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.325+0000 7fb53c533640 1 --2- 172.21.15.60:0/2506317341 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb518066f60 0x7fb518069420 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.325+0000 7fb53c533640 1 --2- 172.21.15.60:0/2506317341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5341316c0 0x7fb53413b740 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.325+0000 7fb53c533640 1 --2- 172.21.15.60:0/2506317341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5340afc10 0x7fb534155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.325+0000 7fb53c533640 1 --2- 172.21.15.60:0/2506317341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb534070990 0x7fb53413a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.325+0000 7fb53c533640 1 -- 172.21.15.60:0/2506317341 >> 172.21.15.60:0/2506317341 conn(0x7fb53406e400 msgr2=0x7fb5340ceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:42.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.325+0000 7fb53c533640 1 -- 172.21.15.60:0/2506317341 shutdown_connections 2024-04-25T10:09:42.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.325+0000 7fb53c533640 1 -- 172.21.15.60:0/2506317341 wait complete. 2024-04-25T10:09:42.360 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:09:42.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.489+0000 7f688a582640 1 Processor -- start 2024-04-25T10:09:42.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.489+0000 7f688a582640 1 -- start start 2024-04-25T10:09:42.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.489+0000 7f688a582640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f688415ec40 0x7f688415fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.489+0000 7f688a582640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68840ce950 0x7f68840cce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.489+0000 7f688a582640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68840b0c60 con 0x7f68840cfe10 2024-04-25T10:09:42.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.489+0000 7f688a582640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6884160400 con 0x7f688415ec40 2024-04-25T10:09:42.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.489+0000 7f688a582640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68840d02a0 con 0x7f68840ce950 2024-04-25T10:09:42.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.489+0000 7f6883fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f688415ec40 0x7f688415fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.489+0000 7f6883fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f688415ec40 0x7f688415fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:57910/0 (socket says 172.21.15.60:57910) 2024-04-25T10:09:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.489+0000 7f6883fff640 1 -- 172.21.15.60:0/2768471444 learned_addr learned my addr 172.21.15.60:0/2768471444 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.489+0000 7f6888b1f640 1 --2- 172.21.15.60:0/2768471444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68840ce950 0x7f68840cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.489+0000 7f6883fff640 1 -- 172.21.15.60:0/2768471444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68840ce950 msgr2=0x7f68840cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.489+0000 7f6883fff640 1 --2- 172.21.15.60:0/2768471444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68840ce950 0x7f68840cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.489+0000 7f6883fff640 1 -- 172.21.15.60:0/2768471444 >> v1:172.21.15.100:6789/0 conn(0x7f68840cfe10 legacy=0x7f68840ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.489+0000 7f6883fff640 1 -- 172.21.15.60:0/2768471444 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68840cc340 con 0x7f688415ec40 2024-04-25T10:09:42.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.489+0000 7f6888b1f640 1 --2- 172.21.15.60:0/2768471444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68840ce950 0x7f68840cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:42.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.493+0000 7f6883fff640 1 --2- 172.21.15.60:0/2768471444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f688415ec40 0x7f688415fec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6874009870 tx=0x7f687401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdac5fd308e08255 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:42.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.493+0000 7f6882ffd640 1 -- 172.21.15.60:0/2768471444 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68740b1070 con 0x7f688415ec40 2024-04-25T10:09:42.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.493+0000 7f6882ffd640 1 -- 172.21.15.60:0/2768471444 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68740ac620 con 0x7f688415ec40 2024-04-25T10:09:42.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.493+0000 7f6882ffd640 1 -- 172.21.15.60:0/2768471444 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68740ac9f0 con 0x7f688415ec40 2024-04-25T10:09:42.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.493+0000 7f688a582640 1 -- 172.21.15.60:0/2768471444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f688415ec40 msgr2=0x7f688415fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.493+0000 7f688a582640 1 --2- 172.21.15.60:0/2768471444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f688415ec40 0x7f688415fec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6874009870 tx=0x7f687401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.493+0000 7f688a582640 1 -- 172.21.15.60:0/2768471444 shutdown_connections 2024-04-25T10:09:42.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.493+0000 7f688a582640 1 --2- 172.21.15.60:0/2768471444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68840ce950 0x7f68840cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.493+0000 7f688a582640 1 --2- 172.21.15.60:0/2768471444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f688415ec40 0x7f688415fec0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.493+0000 7f688a582640 1 -- 172.21.15.60:0/2768471444 >> 172.21.15.60:0/2768471444 conn(0x7f68840d1830 msgr2=0x7f68840d1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.493+0000 7f688a582640 1 -- 172.21.15.60:0/2768471444 shutdown_connections 2024-04-25T10:09:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.493+0000 7f688a582640 1 -- 172.21.15.60:0/2768471444 wait complete. 2024-04-25T10:09:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f688a582640 1 Processor -- start 2024-04-25T10:09:42.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f688a582640 1 -- start start 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f688a582640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68840ce950 0x7f68840cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f688a582640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f688415ec40 0x7f688415cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f688a582640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68840cb540 0x7f688415d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f688a582640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688415f020 con 0x7f68840cb540 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f688a582640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68840d0580 con 0x7f68840ce950 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f688a582640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688415f1a0 con 0x7f688415ec40 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f6888b1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68840cb540 0x7f688415d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f6888b1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68840cb540 0x7f688415d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44946/0 (socket says 172.21.15.60:44946) 2024-04-25T10:09:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f6888b1f640 1 -- 172.21.15.60:0/1879726090 learned_addr learned my addr 172.21.15.60:0/1879726090 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f6883fff640 1 --2- 172.21.15.60:0/1879726090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68840ce950 0x7f68840cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f68837fe640 1 --2- 172.21.15.60:0/1879726090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f688415ec40 0x7f688415cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f6888b1f640 1 -- 172.21.15.60:0/1879726090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f688415ec40 msgr2=0x7f688415cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f6888b1f640 1 --2- 172.21.15.60:0/1879726090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f688415ec40 0x7f688415cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f6888b1f640 1 -- 172.21.15.60:0/1879726090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68840ce950 msgr2=0x7f68840cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f6888b1f640 1 --2- 172.21.15.60:0/1879726090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68840ce950 0x7f68840cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f6888b1f640 1 -- 172.21.15.60:0/1879726090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688415da60 con 0x7f68840cb540 2024-04-25T10:09:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f6888b1f640 1 --2- 172.21.15.60:0/1879726090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68840cb540 0x7f688415d2f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f687c009eb0 tx=0x7f687c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f6880ff9640 1 -- 172.21.15.60:0/1879726090 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c018020 con 0x7f68840cb540 2024-04-25T10:09:42.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f6880ff9640 1 -- 172.21.15.60:0/1879726090 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c014480 con 0x7f68840cb540 2024-04-25T10:09:42.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f6880ff9640 1 -- 172.21.15.60:0/1879726090 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c014850 con 0x7f68840cb540 2024-04-25T10:09:42.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f688a582640 1 -- 172.21.15.60:0/1879726090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f688415dd40 con 0x7f68840cb540 2024-04-25T10:09:42.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.497+0000 7f688a582640 1 -- 172.21.15.60:0/1879726090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f688414f9f0 con 0x7f68840cb540 2024-04-25T10:09:42.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.501+0000 7f688a582640 1 -- 172.21.15.60:0/1879726090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f688414fbf0 con 0x7f68840cb540 2024-04-25T10:09:42.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.501+0000 7f6880ff9640 1 -- 172.21.15.60:0/1879726090 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f687c01b020 con 0x7f68840cb540 2024-04-25T10:09:42.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.501+0000 7f6880ff9640 1 --2- 172.21.15.60:0/1879726090 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6870066f60 0x7f6870069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.501+0000 7f6880ff9640 1 -- 172.21.15.60:0/1879726090 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f687c085f20 con 0x7f68840cb540 2024-04-25T10:09:42.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.501+0000 7f6883fff640 1 --2- 172.21.15.60:0/1879726090 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6870066f60 0x7f6870069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.501+0000 7f6883fff640 1 --2- 172.21.15.60:0/1879726090 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6870066f60 0x7f6870069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f68740ae000 tx=0x7f68740a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:42.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.505+0000 7f6880ff9640 1 -- 172.21.15.60:0/1879726090 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f688414fbf0 con 0x7f68840cb540 2024-04-25T10:09:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:42.665+0000 7f688a582640 1 -- 172.21.15.60:0/1879726090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f68840b77b0 con 0x7f68840cb540 2024-04-25T10:09:43.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.329+0000 7f6880ff9640 1 -- 172.21.15.60:0/1879726090 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v67) v1 ==== 170+0+0 (secure 0 0 0) 0x7f687c022030 con 0x7f68840cb540 2024-04-25T10:09:43.339 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T10:09:43.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.333+0000 7f688a582640 1 -- 172.21.15.60:0/1879726090 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6870066f60 msgr2=0x7f6870069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.333+0000 7f688a582640 1 --2- 172.21.15.60:0/1879726090 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6870066f60 0x7f6870069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f68740ae000 tx=0x7f68740a7000 comp rx=0 tx=0).stop 2024-04-25T10:09:43.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.333+0000 7f688a582640 1 -- 172.21.15.60:0/1879726090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68840cb540 msgr2=0x7f688415d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.333+0000 7f688a582640 1 --2- 172.21.15.60:0/1879726090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68840cb540 0x7f688415d2f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f687c009eb0 tx=0x7f687c00c490 comp rx=0 tx=0).stop 2024-04-25T10:09:43.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.333+0000 7f688a582640 1 -- 172.21.15.60:0/1879726090 shutdown_connections 2024-04-25T10:09:43.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.333+0000 7f688a582640 1 --2- 172.21.15.60:0/1879726090 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6870066f60 0x7f6870069420 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.333+0000 7f688a582640 1 --2- 172.21.15.60:0/1879726090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68840cb540 0x7f688415d2f0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.333+0000 7f688a582640 1 --2- 172.21.15.60:0/1879726090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f688415ec40 0x7f688415cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.333+0000 7f688a582640 1 --2- 172.21.15.60:0/1879726090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68840ce950 0x7f68840cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.333+0000 7f688a582640 1 -- 172.21.15.60:0/1879726090 >> 172.21.15.60:0/1879726090 conn(0x7f68840d1830 msgr2=0x7f68840af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:43.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.333+0000 7f688a582640 1 -- 172.21.15.60:0/1879726090 shutdown_connections 2024-04-25T10:09:43.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.333+0000 7f688a582640 1 -- 172.21.15.60:0/1879726090 wait complete. 2024-04-25T10:09:43.375 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:09:43.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.493+0000 7f9aff94b640 1 Processor -- start 2024-04-25T10:09:43.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.493+0000 7f9aff94b640 1 -- start start 2024-04-25T10:09:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.493+0000 7f9aff94b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9af8152ad0 0x7f9af815c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.493+0000 7f9aff94b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af815beb0 con 0x7f9af80cd1d0 2024-04-25T10:09:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.493+0000 7f9aff94b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af80cf360 con 0x7f9af80716a0 2024-04-25T10:09:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.493+0000 7f9aff94b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af815c150 con 0x7f9af8152ad0 2024-04-25T10:09:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.493+0000 7f9afd6e7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9af8152ad0 0x7f9af815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.493+0000 7f9afd6e7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9af8152ad0 0x7f9af815c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44948/0 (socket says 172.21.15.60:44948) 2024-04-25T10:09:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.493+0000 7f9afd6e7640 1 -- 172.21.15.60:0/3928257782 learned_addr learned my addr 172.21.15.60:0/3928257782 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:43.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.493+0000 7f9afd6e7640 1 -- 172.21.15.60:0/3928257782 >> v1:172.21.15.100:6789/0 conn(0x7f9af80716a0 legacy=0x7f9af8071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:43.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.493+0000 7f9afd6e7640 1 -- 172.21.15.60:0/3928257782 >> v1:172.21.15.60:6789/0 conn(0x7f9af80cd1d0 legacy=0x7f9af80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.493+0000 7f9aeffff640 1 -- 172.21.15.60:0/3928257782 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403212964 0 0) 0x7f9af815beb0 con 0x7f9af80cd1d0 2024-04-25T10:09:43.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.493+0000 7f9afd6e7640 1 -- 172.21.15.60:0/3928257782 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af815c430 con 0x7f9af8152ad0 2024-04-25T10:09:43.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9afd6e7640 1 --2- 172.21.15.60:0/3928257782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af8152ad0 0x7f9af815c950 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009870 tx=0x7f9af401cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c96c5008a99b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:43.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aeffff640 1 -- 172.21.15.60:0/3928257782 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af40a9400 con 0x7f9af8152ad0 2024-04-25T10:09:43.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aeffff640 1 -- 172.21.15.60:0/3928257782 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af40a95a0 con 0x7f9af8152ad0 2024-04-25T10:09:43.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aeffff640 1 -- 172.21.15.60:0/3928257782 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af40a9970 con 0x7f9af8152ad0 2024-04-25T10:09:43.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aff94b640 1 -- 172.21.15.60:0/3928257782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af8152ad0 msgr2=0x7f9af815c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aff94b640 1 --2- 172.21.15.60:0/3928257782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af8152ad0 0x7f9af815c950 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009870 tx=0x7f9af401cf80 comp rx=0 tx=0).stop 2024-04-25T10:09:43.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aff94b640 1 -- 172.21.15.60:0/3928257782 shutdown_connections 2024-04-25T10:09:43.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aff94b640 1 --2- 172.21.15.60:0/3928257782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af8152ad0 0x7f9af815c950 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aff94b640 1 -- 172.21.15.60:0/3928257782 >> 172.21.15.60:0/3928257782 conn(0x7f9af80cf940 msgr2=0x7f9af80cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:43.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aff94b640 1 -- 172.21.15.60:0/3928257782 shutdown_connections 2024-04-25T10:09:43.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aff94b640 1 -- 172.21.15.60:0/3928257782 wait complete. 2024-04-25T10:09:43.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aff94b640 1 Processor -- start 2024-04-25T10:09:43.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aff94b640 1 -- start start 2024-04-25T10:09:43.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aff94b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9af80716a0 0x7f9af80b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aff94b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af80cd1d0 0x7f9af814e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aff94b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9af80c52c0 0x7f9af815e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aff94b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af815db80 con 0x7f9af80cd1d0 2024-04-25T10:09:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aff94b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af815da00 con 0x7f9af80c52c0 2024-04-25T10:09:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9aff94b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af8169740 con 0x7f9af80716a0 2024-04-25T10:09:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9afcee6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af80cd1d0 0x7f9af814e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9afcee6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af80cd1d0 0x7f9af814e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44956/0 (socket says 172.21.15.60:44956) 2024-04-25T10:09:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9afcee6640 1 -- 172.21.15.60:0/1070640161 learned_addr learned my addr 172.21.15.60:0/1070640161 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9afd6e7640 1 --2- 172.21.15.60:0/1070640161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9af80716a0 0x7f9af80b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.497+0000 7f9afdee8640 1 --2- 172.21.15.60:0/1070640161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9af80c52c0 0x7f9af815e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9afcee6640 1 -- 172.21.15.60:0/1070640161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9af80716a0 msgr2=0x7f9af80b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9afcee6640 1 --2- 172.21.15.60:0/1070640161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9af80716a0 0x7f9af80b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9afcee6640 1 -- 172.21.15.60:0/1070640161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9af80c52c0 msgr2=0x7f9af815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9afcee6640 1 --2- 172.21.15.60:0/1070640161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9af80c52c0 0x7f9af815e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9afcee6640 1 -- 172.21.15.60:0/1070640161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af80c57c0 con 0x7f9af80cd1d0 2024-04-25T10:09:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9afcee6640 1 --2- 172.21.15.60:0/1070640161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af80cd1d0 0x7f9af814e5b0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800c4d0 tx=0x7f9ae800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9aedffb640 1 -- 172.21.15.60:0/1070640161 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8017070 con 0x7f9af80cd1d0 2024-04-25T10:09:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9aedffb640 1 -- 172.21.15.60:0/1070640161 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae8005480 con 0x7f9af80cd1d0 2024-04-25T10:09:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9aedffb640 1 -- 172.21.15.60:0/1070640161 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8003840 con 0x7f9af80cd1d0 2024-04-25T10:09:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9aff94b640 1 -- 172.21.15.60:0/1070640161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af80c5aa0 con 0x7f9af80cd1d0 2024-04-25T10:09:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9aff94b640 1 -- 172.21.15.60:0/1070640161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af80c6060 con 0x7f9af80cd1d0 2024-04-25T10:09:43.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9aff94b640 1 -- 172.21.15.60:0/1070640161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af8074770 con 0x7f9af80cd1d0 2024-04-25T10:09:43.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9aedffb640 1 -- 172.21.15.60:0/1070640161 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ae801a070 con 0x7f9af80cd1d0 2024-04-25T10:09:43.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9aedffb640 1 --2- 172.21.15.60:0/1070640161 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9adc066f90 0x7f9adc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:43.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9aedffb640 1 -- 172.21.15.60:0/1070640161 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f9ae8084360 con 0x7f9af80cd1d0 2024-04-25T10:09:43.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9afd6e7640 1 --2- 172.21.15.60:0/1070640161 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9adc066f90 0x7f9adc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:43.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.501+0000 7f9afd6e7640 1 --2- 172.21.15.60:0/1070640161 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9adc066f90 0x7f9adc069450 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9af40ae000 tx=0x7f9af40b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:43.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.505+0000 7f9aedffb640 1 -- 172.21.15.60:0/1070640161 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ae8055520 con 0x7f9af80cd1d0 2024-04-25T10:09:43.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:43.701+0000 7f9aff94b640 1 -- 172.21.15.60:0/1070640161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9af806fb60 con 0x7f9af80cd1d0 2024-04-25T10:09:44.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.349+0000 7f9aedffb640 1 -- 172.21.15.60:0/1070640161 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v35) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9ae8054e20 con 0x7f9af80cd1d0 2024-04-25T10:09:44.362 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 10 and data pool 11 2024-04-25T10:09:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.357+0000 7f9aff94b640 1 -- 172.21.15.60:0/1070640161 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9adc066f90 msgr2=0x7f9adc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.357+0000 7f9aff94b640 1 --2- 172.21.15.60:0/1070640161 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9adc066f90 0x7f9adc069450 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9af40ae000 tx=0x7f9af40b3040 comp rx=0 tx=0).stop 2024-04-25T10:09:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.357+0000 7f9aff94b640 1 -- 172.21.15.60:0/1070640161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af80cd1d0 msgr2=0x7f9af814e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.357+0000 7f9aff94b640 1 --2- 172.21.15.60:0/1070640161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af80cd1d0 0x7f9af814e5b0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800c4d0 tx=0x7f9ae800c990 comp rx=0 tx=0).stop 2024-04-25T10:09:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.357+0000 7f9aff94b640 1 -- 172.21.15.60:0/1070640161 shutdown_connections 2024-04-25T10:09:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.357+0000 7f9aff94b640 1 --2- 172.21.15.60:0/1070640161 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9adc066f90 0x7f9adc069450 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.357+0000 7f9aff94b640 1 --2- 172.21.15.60:0/1070640161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9af80c52c0 0x7f9af815e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.357+0000 7f9aff94b640 1 --2- 172.21.15.60:0/1070640161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9af80cd1d0 0x7f9af814e5b0 secure :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800c4d0 tx=0x7f9ae800c990 comp rx=0 tx=0).stop 2024-04-25T10:09:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.357+0000 7f9aff94b640 1 --2- 172.21.15.60:0/1070640161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9af80716a0 0x7f9af80b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.357+0000 7f9aff94b640 1 -- 172.21.15.60:0/1070640161 >> 172.21.15.60:0/1070640161 conn(0x7f9af80cf940 msgr2=0x7f9af80cbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.357+0000 7f9aff94b640 1 -- 172.21.15.60:0/1070640161 shutdown_connections 2024-04-25T10:09:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.357+0000 7f9aff94b640 1 -- 172.21.15.60:0/1070640161 wait complete. 2024-04-25T10:09:44.400 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:44.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.509+0000 7f669d42d640 1 Processor -- start 2024-04-25T10:09:44.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.509+0000 7f669d42d640 1 -- start start 2024-04-25T10:09:44.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.509+0000 7f669d42d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669815bc90 0x7f669815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.509+0000 7f669d42d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f66980b4660 0x7f669815c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.509+0000 7f669d42d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6698071ca0 con 0x7f66980ce7a0 2024-04-25T10:09:44.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.509+0000 7f669d42d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66980cc370 con 0x7f66980b4660 2024-04-25T10:09:44.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.509+0000 7f669d42d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66980cca40 con 0x7f669815bc90 2024-04-25T10:09:44.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.509+0000 7f66977fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f66980b4660 0x7f669815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.509+0000 7f66977fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f66980b4660 0x7f669815c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44972/0 (socket says 172.21.15.60:44972) 2024-04-25T10:09:44.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.509+0000 7f66977fe640 1 -- 172.21.15.60:0/779715763 learned_addr learned my addr 172.21.15.60:0/779715763 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:44.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.509+0000 7f668ffff640 1 --2- 172.21.15.60:0/779715763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669815bc90 0x7f669815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.509+0000 7f66977fe640 1 -- 172.21.15.60:0/779715763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669815bc90 msgr2=0x7f669815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:44.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.509+0000 7f66977fe640 1 --2- 172.21.15.60:0/779715763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669815bc90 0x7f669815c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.509+0000 7f66977fe640 1 -- 172.21.15.60:0/779715763 >> v1:172.21.15.100:6789/0 conn(0x7f66980ce7a0 legacy=0x7f66980ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.509+0000 7f66977fe640 1 -- 172.21.15.60:0/779715763 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66980ccd20 con 0x7f66980b4660 2024-04-25T10:09:44.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.513+0000 7f66977fe640 1 --2- 172.21.15.60:0/779715763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66980b4660 0x7f669815c760 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f6690009e50 tx=0x7f669001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d384770ded73f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:44.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.513+0000 7f66967fc640 1 -- 172.21.15.60:0/779715763 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66900a5700 con 0x7f66980b4660 2024-04-25T10:09:44.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.513+0000 7f66967fc640 1 -- 172.21.15.60:0/779715763 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66900a58a0 con 0x7f66980b4660 2024-04-25T10:09:44.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.513+0000 7f66967fc640 1 -- 172.21.15.60:0/779715763 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66900a5c70 con 0x7f66980b4660 2024-04-25T10:09:44.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.513+0000 7f669d42d640 1 -- 172.21.15.60:0/779715763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66980b4660 msgr2=0x7f669815c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.513+0000 7f669d42d640 1 --2- 172.21.15.60:0/779715763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66980b4660 0x7f669815c760 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f6690009e50 tx=0x7f669001ca30 comp rx=0 tx=0).stop 2024-04-25T10:09:44.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.513+0000 7f669d42d640 1 -- 172.21.15.60:0/779715763 shutdown_connections 2024-04-25T10:09:44.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.513+0000 7f669d42d640 1 --2- 172.21.15.60:0/779715763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66980b4660 0x7f669815c760 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.513+0000 7f669d42d640 1 --2- 172.21.15.60:0/779715763 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669815bc90 0x7f669815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.513+0000 7f669d42d640 1 -- 172.21.15.60:0/779715763 >> 172.21.15.60:0/779715763 conn(0x7f66980b12d0 msgr2=0x7f66980b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:44.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.513+0000 7f669d42d640 1 -- 172.21.15.60:0/779715763 shutdown_connections 2024-04-25T10:09:44.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.513+0000 7f669d42d640 1 -- 172.21.15.60:0/779715763 wait complete. 2024-04-25T10:09:44.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.513+0000 7f669d42d640 1 Processor -- start 2024-04-25T10:09:44.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.513+0000 7f669d42d640 1 -- start start 2024-04-25T10:09:44.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f669d42d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66980ce7a0 0x7f669814eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f669d42d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669815bc90 0x7f669814b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f669d42d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669815a4c0 0x7f6698153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f669d42d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66980b3db0 con 0x7f669815bc90 2024-04-25T10:09:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f669d42d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66980b4490 con 0x7f66980ce7a0 2024-04-25T10:09:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f669d42d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66981685c0 con 0x7f669815a4c0 2024-04-25T10:09:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f668ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669815bc90 0x7f669814b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f668ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669815bc90 0x7f669814b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44976/0 (socket says 172.21.15.60:44976) 2024-04-25T10:09:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f668ffff640 1 -- 172.21.15.60:0/1645204287 learned_addr learned my addr 172.21.15.60:0/1645204287 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f6696ffd640 1 --2- 172.21.15.60:0/1645204287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66980ce7a0 0x7f669814eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f66977fe640 1 --2- 172.21.15.60:0/1645204287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669815a4c0 0x7f6698153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f668ffff640 1 -- 172.21.15.60:0/1645204287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669815a4c0 msgr2=0x7f6698153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f668ffff640 1 --2- 172.21.15.60:0/1645204287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669815a4c0 0x7f6698153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f668ffff640 1 -- 172.21.15.60:0/1645204287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66980ce7a0 msgr2=0x7f669814eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f668ffff640 1 --2- 172.21.15.60:0/1645204287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66980ce7a0 0x7f669814eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f668ffff640 1 -- 172.21.15.60:0/1645204287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669814b8a0 con 0x7f669815bc90 2024-04-25T10:09:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f668ffff640 1 --2- 172.21.15.60:0/1645204287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669815bc90 0x7f669814b360 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f668000d800 tx=0x7f668000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f668f7fe640 1 -- 172.21.15.60:0/1645204287 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668000b6c0 con 0x7f669815bc90 2024-04-25T10:09:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f668f7fe640 1 -- 172.21.15.60:0/1645204287 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6680019070 con 0x7f669815bc90 2024-04-25T10:09:44.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f669d42d640 1 -- 172.21.15.60:0/1645204287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669815adf0 con 0x7f669815bc90 2024-04-25T10:09:44.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f668f7fe640 1 -- 172.21.15.60:0/1645204287 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668000bae0 con 0x7f669815bc90 2024-04-25T10:09:44.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f669d42d640 1 -- 172.21.15.60:0/1645204287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669806d540 con 0x7f669815bc90 2024-04-25T10:09:44.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f669d42d640 1 -- 172.21.15.60:0/1645204287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6698074770 con 0x7f669815bc90 2024-04-25T10:09:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f668f7fe640 1 -- 172.21.15.60:0/1645204287 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f668000bc80 con 0x7f669815bc90 2024-04-25T10:09:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f668f7fe640 1 --2- 172.21.15.60:0/1645204287 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6678066f60 0x7f6678069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f668f7fe640 1 -- 172.21.15.60:0/1645204287 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6680085460 con 0x7f669815bc90 2024-04-25T10:09:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.517+0000 7f6696ffd640 1 --2- 172.21.15.60:0/1645204287 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6678066f60 0x7f6678069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.521+0000 7f6696ffd640 1 --2- 172.21.15.60:0/1645204287 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6678066f60 0x7f6678069420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f66880015b0 tx=0x7f6688009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:44.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.521+0000 7f668f7fe640 1 -- 172.21.15.60:0/1645204287 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6680056500 con 0x7f669815bc90 2024-04-25T10:09:44.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.693+0000 7f669d42d640 1 -- 172.21.15.60:0/1645204287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66980af300 con 0x7f669815bc90 2024-04-25T10:09:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.693+0000 7f668f7fe640 1 -- 172.21.15.60:0/1645204287 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16693 (secure 0 0 0) 0x7f6680055e00 con 0x7f669815bc90 2024-04-25T10:09:44.703 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:44.703 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":68,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:09:43.707107+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:42.005281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:42.710028+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:44.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.697+0000 7f669d42d640 1 -- 172.21.15.60:0/1645204287 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6678066f60 msgr2=0x7f6678069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.697+0000 7f669d42d640 1 --2- 172.21.15.60:0/1645204287 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6678066f60 0x7f6678069420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f66880015b0 tx=0x7f6688009040 comp rx=0 tx=0).stop 2024-04-25T10:09:44.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.697+0000 7f669d42d640 1 -- 172.21.15.60:0/1645204287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669815bc90 msgr2=0x7f669814b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.697+0000 7f669d42d640 1 --2- 172.21.15.60:0/1645204287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669815bc90 0x7f669814b360 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f668000d800 tx=0x7f668000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.697+0000 7f669d42d640 1 -- 172.21.15.60:0/1645204287 shutdown_connections 2024-04-25T10:09:44.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.697+0000 7f669d42d640 1 --2- 172.21.15.60:0/1645204287 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6678066f60 0x7f6678069420 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.697+0000 7f669d42d640 1 --2- 172.21.15.60:0/1645204287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669815a4c0 0x7f6698153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.697+0000 7f669d42d640 1 --2- 172.21.15.60:0/1645204287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669815bc90 0x7f669814b360 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.697+0000 7f669d42d640 1 --2- 172.21.15.60:0/1645204287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66980ce7a0 0x7f669814eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.697+0000 7f669d42d640 1 -- 172.21.15.60:0/1645204287 >> 172.21.15.60:0/1645204287 conn(0x7f66980b12d0 msgr2=0x7f669815d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:44.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.697+0000 7f669d42d640 1 -- 172.21.15.60:0/1645204287 shutdown_connections 2024-04-25T10:09:44.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.697+0000 7f669d42d640 1 -- 172.21.15.60:0/1645204287 wait complete. 2024-04-25T10:09:44.723 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:44.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.841+0000 7f8e351f4640 1 Processor -- start 2024-04-25T10:09:44.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.841+0000 7f8e351f4640 1 -- start start 2024-04-25T10:09:44.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.841+0000 7f8e351f4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8e3006ec30 0x7f8e300cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.841+0000 7f8e351f4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e300b37e0 0x7f8e300b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.841+0000 7f8e351f4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3006f130 con 0x7f8e3015a7a0 2024-04-25T10:09:44.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.841+0000 7f8e351f4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e300cdaa0 con 0x7f8e3006ec30 2024-04-25T10:09:44.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.841+0000 7f8e351f4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3006d2f0 con 0x7f8e300b37e0 2024-04-25T10:09:44.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.841+0000 7f8e2e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8e3006ec30 0x7f8e300cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.841+0000 7f8e2e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8e3006ec30 0x7f8e300cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44982/0 (socket says 172.21.15.60:44982) 2024-04-25T10:09:44.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.841+0000 7f8e2e59c640 1 -- 172.21.15.60:0/2016692811 learned_addr learned my addr 172.21.15.60:0/2016692811 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:44.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.841+0000 7f8e2f59e640 1 --2- 172.21.15.60:0/2016692811 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e300b37e0 0x7f8e300b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.841+0000 7f8e2e59c640 1 -- 172.21.15.60:0/2016692811 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e300b37e0 msgr2=0x7f8e300b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:44.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.841+0000 7f8e2e59c640 1 --2- 172.21.15.60:0/2016692811 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e300b37e0 0x7f8e300b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.841+0000 7f8e2e59c640 1 -- 172.21.15.60:0/2016692811 >> v1:172.21.15.100:6789/0 conn(0x7f8e3015a7a0 legacy=0x7f8e30071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:44.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.841+0000 7f8e2e59c640 1 -- 172.21.15.60:0/2016692811 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3006f410 con 0x7f8e3006ec30 2024-04-25T10:09:44.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e2e59c640 1 --2- 172.21.15.60:0/2016692811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e3006ec30 0x7f8e300cd560 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f8e240098a0 tx=0x7f8e2401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66577c8f5a08df8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:44.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e2dd9b640 1 -- 172.21.15.60:0/2016692811 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e240ab650 con 0x7f8e3006ec30 2024-04-25T10:09:44.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e2dd9b640 1 -- 172.21.15.60:0/2016692811 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e240b0070 con 0x7f8e3006ec30 2024-04-25T10:09:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e2dd9b640 1 -- 172.21.15.60:0/2016692811 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e240aba10 con 0x7f8e3006ec30 2024-04-25T10:09:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e351f4640 1 -- 172.21.15.60:0/2016692811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e3006ec30 msgr2=0x7f8e300cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e351f4640 1 --2- 172.21.15.60:0/2016692811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e3006ec30 0x7f8e300cd560 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f8e240098a0 tx=0x7f8e2401c540 comp rx=0 tx=0).stop 2024-04-25T10:09:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e351f4640 1 -- 172.21.15.60:0/2016692811 shutdown_connections 2024-04-25T10:09:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e351f4640 1 --2- 172.21.15.60:0/2016692811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e3006ec30 0x7f8e300cd560 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e351f4640 1 --2- 172.21.15.60:0/2016692811 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e300b37e0 0x7f8e300b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e351f4640 1 -- 172.21.15.60:0/2016692811 >> 172.21.15.60:0/2016692811 conn(0x7f8e300b12d0 msgr2=0x7f8e300b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e351f4640 1 -- 172.21.15.60:0/2016692811 shutdown_connections 2024-04-25T10:09:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e351f4640 1 -- 172.21.15.60:0/2016692811 wait complete. 2024-04-25T10:09:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e351f4640 1 Processor -- start 2024-04-25T10:09:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e351f4640 1 -- start start 2024-04-25T10:09:44.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e351f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e3006ec30 0x7f8e3015f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e351f4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e300b37e0 0x7f8e301564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e351f4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e3015a7a0 0x7f8e30141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e351f4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3006e8c0 con 0x7f8e300b37e0 2024-04-25T10:09:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e351f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3006e740 con 0x7f8e3006ec30 2024-04-25T10:09:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e351f4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3006d8b0 con 0x7f8e3015a7a0 2024-04-25T10:09:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e2e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e300b37e0 0x7f8e301564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e2e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e300b37e0 0x7f8e301564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44986/0 (socket says 172.21.15.60:44986) 2024-04-25T10:09:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e2e59c640 1 -- 172.21.15.60:0/1482687819 learned_addr learned my addr 172.21.15.60:0/1482687819 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e2ed9d640 1 --2- 172.21.15.60:0/1482687819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e3006ec30 0x7f8e3015f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e2f59e640 1 --2- 172.21.15.60:0/1482687819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e3015a7a0 0x7f8e30141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e2e59c640 1 -- 172.21.15.60:0/1482687819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e3015a7a0 msgr2=0x7f8e30141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e2e59c640 1 --2- 172.21.15.60:0/1482687819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e3015a7a0 0x7f8e30141e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e2e59c640 1 -- 172.21.15.60:0/1482687819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e3006ec30 msgr2=0x7f8e3015f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e2e59c640 1 --2- 172.21.15.60:0/1482687819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e3006ec30 0x7f8e3015f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e2e59c640 1 -- 172.21.15.60:0/1482687819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e30156c40 con 0x7f8e300b37e0 2024-04-25T10:09:44.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.845+0000 7f8e2ed9d640 1 --2- 172.21.15.60:0/1482687819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e3006ec30 0x7f8e3015f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:44.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.849+0000 7f8e2e59c640 1 --2- 172.21.15.60:0/1482687819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e300b37e0 0x7f8e301564a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8e240ad040 tx=0x7f8e2401ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:44.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.849+0000 7f8e177fe640 1 -- 172.21.15.60:0/1482687819 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e240b4600 con 0x7f8e300b37e0 2024-04-25T10:09:44.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.849+0000 7f8e177fe640 1 -- 172.21.15.60:0/1482687819 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e240b47a0 con 0x7f8e300b37e0 2024-04-25T10:09:44.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.849+0000 7f8e351f4640 1 -- 172.21.15.60:0/1482687819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e30156f20 con 0x7f8e300b37e0 2024-04-25T10:09:44.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.849+0000 7f8e177fe640 1 -- 172.21.15.60:0/1482687819 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e240b4b90 con 0x7f8e300b37e0 2024-04-25T10:09:44.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.849+0000 7f8e351f4640 1 -- 172.21.15.60:0/1482687819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e301574e0 con 0x7f8e300b37e0 2024-04-25T10:09:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.849+0000 7f8e351f4640 1 -- 172.21.15.60:0/1482687819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e30074770 con 0x7f8e300b37e0 2024-04-25T10:09:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.849+0000 7f8e177fe640 1 -- 172.21.15.60:0/1482687819 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8e240b3380 con 0x7f8e300b37e0 2024-04-25T10:09:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.849+0000 7f8e177fe640 1 --2- 172.21.15.60:0/1482687819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8e1c066f60 0x7f8e1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.849+0000 7f8e177fe640 1 -- 172.21.15.60:0/1482687819 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e2411f520 con 0x7f8e300b37e0 2024-04-25T10:09:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.849+0000 7f8e2ed9d640 1 --2- 172.21.15.60:0/1482687819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8e1c066f60 0x7f8e1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.849+0000 7f8e2ed9d640 1 --2- 172.21.15.60:0/1482687819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8e1c066f60 0x7f8e1c069420 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8e20001670 tx=0x7f8e20009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:44.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:44.853+0000 7f8e177fe640 1 -- 172.21.15.60:0/1482687819 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8e240c4060 con 0x7f8e300b37e0 2024-04-25T10:09:45.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.013+0000 7f8e351f4640 1 -- 172.21.15.60:0/1482687819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e30142350 con 0x7f8e300b37e0 2024-04-25T10:09:45.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.017+0000 7f8e177fe640 1 -- 172.21.15.60:0/1482687819 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16693 (secure 0 0 0) 0x7f8e24016130 con 0x7f8e300b37e0 2024-04-25T10:09:45.026 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:45.027 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":68,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:09:43.707107+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:42.005281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:42.710028+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:45.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.021+0000 7f8e351f4640 1 -- 172.21.15.60:0/1482687819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8e1c066f60 msgr2=0x7f8e1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.021+0000 7f8e351f4640 1 --2- 172.21.15.60:0/1482687819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8e1c066f60 0x7f8e1c069420 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8e20001670 tx=0x7f8e20009040 comp rx=0 tx=0).stop 2024-04-25T10:09:45.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.021+0000 7f8e351f4640 1 -- 172.21.15.60:0/1482687819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e300b37e0 msgr2=0x7f8e301564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.021+0000 7f8e351f4640 1 --2- 172.21.15.60:0/1482687819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e300b37e0 0x7f8e301564a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8e240ad040 tx=0x7f8e2401ad80 comp rx=0 tx=0).stop 2024-04-25T10:09:45.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.021+0000 7f8e351f4640 1 -- 172.21.15.60:0/1482687819 shutdown_connections 2024-04-25T10:09:45.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.021+0000 7f8e351f4640 1 --2- 172.21.15.60:0/1482687819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8e1c066f60 0x7f8e1c069420 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.021+0000 7f8e351f4640 1 --2- 172.21.15.60:0/1482687819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e3015a7a0 0x7f8e30141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.021+0000 7f8e351f4640 1 --2- 172.21.15.60:0/1482687819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e300b37e0 0x7f8e301564a0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.021+0000 7f8e351f4640 1 --2- 172.21.15.60:0/1482687819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e3006ec30 0x7f8e3015f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.021+0000 7f8e351f4640 1 -- 172.21.15.60:0/1482687819 >> 172.21.15.60:0/1482687819 conn(0x7f8e300b12d0 msgr2=0x7f8e30154570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:45.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.021+0000 7f8e351f4640 1 -- 172.21.15.60:0/1482687819 shutdown_connections 2024-04-25T10:09:45.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.021+0000 7f8e351f4640 1 -- 172.21.15.60:0/1482687819 wait complete. 2024-04-25T10:09:45.047 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:09:45.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc9646c640 1 Processor -- start 2024-04-25T10:09:45.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc9646c640 1 -- start start 2024-04-25T10:09:45.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc9646c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efc900b33e0 0x7efc900aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc9646c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc9006c550 con 0x7efc9006ac10 2024-04-25T10:09:45.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc9646c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc9006b940 con 0x7efc900b37c0 2024-04-25T10:09:45.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc9646c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc9006bc20 con 0x7efc900b33e0 2024-04-25T10:09:45.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc94a09640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efc9006ac10 0x7efc9006b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45156/0 (socket says 172.21.15.60:45156) 2024-04-25T10:09:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc94a09640 1 -- 172.21.15.60:0/317613018 learned_addr learned my addr 172.21.15.60:0/317613018 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8ffff640 1 --2- 172.21.15.60:0/317613018 >> v2:172.21.15.100:3300/0 conn(0x7efc900b33e0 0x7efc900aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8effd640 1 -- 172.21.15.60:0/317613018 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38355802 0 0) 0x7efc9006c550 con 0x7efc9006ac10 2024-04-25T10:09:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8effd640 1 -- 172.21.15.60:0/317613018 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc78003530 con 0x7efc9006ac10 2024-04-25T10:09:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8effd640 1 -- 172.21.15.60:0/317613018 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 721657432 0 0) 0x7efc78003530 con 0x7efc9006ac10 2024-04-25T10:09:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8effd640 1 -- 172.21.15.60:0/317613018 >> v2:172.21.15.100:3300/0 conn(0x7efc900b33e0 msgr2=0x7efc900aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8effd640 1 --2- 172.21.15.60:0/317613018 >> v2:172.21.15.100:3300/0 conn(0x7efc900b33e0 0x7efc900aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8effd640 1 -- 172.21.15.60:0/317613018 >> v1:172.21.15.100:6789/0 conn(0x7efc900b37c0 legacy=0x7efc900af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8effd640 1 -- 172.21.15.60:0/317613018 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9006bf00 con 0x7efc9006ac10 2024-04-25T10:09:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8effd640 1 -- 172.21.15.60:0/317613018 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7efc88001a00 con 0x7efc9006ac10 2024-04-25T10:09:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8effd640 1 -- 172.21.15.60:0/317613018 >> v1:172.21.15.60:6789/0 conn(0x7efc9006ac10 legacy=0x7efc9006b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8effd640 1 --2- 172.21.15.60:0/317613018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc780039c0 0x7efc78003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8effd640 1 --2- 172.21.15.60:0/317613018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc78004810 0x7efc78004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8effd640 1 --2- 172.21.15.60:0/317613018 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc780051d0 0x7efc7800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8effd640 1 -- 172.21.15.60:0/317613018 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9006bc20 con 0x7efc780039c0 2024-04-25T10:09:45.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8effd640 1 -- 172.21.15.60:0/317613018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7800c070 con 0x7efc78004810 2024-04-25T10:09:45.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8effd640 1 -- 172.21.15.60:0/317613018 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc7800c270 con 0x7efc780051d0 2024-04-25T10:09:45.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8ffff640 1 --2- 172.21.15.60:0/317613018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc780039c0 0x7efc78003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8f7fe640 1 --2- 172.21.15.60:0/317613018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc78004810 0x7efc78004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8ffff640 1 -- 172.21.15.60:0/317613018 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc780051d0 msgr2=0x7efc7800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:45.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc94a09640 1 --2- 172.21.15.60:0/317613018 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc780051d0 0x7efc7800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8ffff640 1 --2- 172.21.15.60:0/317613018 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc780051d0 0x7efc7800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8ffff640 1 -- 172.21.15.60:0/317613018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc78004810 msgr2=0x7efc78004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:45.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8ffff640 1 --2- 172.21.15.60:0/317613018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc78004810 0x7efc78004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.169+0000 7efc8ffff640 1 -- 172.21.15.60:0/317613018 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc78003530 con 0x7efc780039c0 2024-04-25T10:09:45.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.173+0000 7efc94a09640 1 -- 172.21.15.60:0/317613018 reap_dead start 2024-04-25T10:09:45.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.173+0000 7efc8ffff640 1 --2- 172.21.15.60:0/317613018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc780039c0 0x7efc78003de0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7efc80009ae0 tx=0x7efc8001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b5e1b8711665b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:45.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.173+0000 7efc8effd640 1 -- 172.21.15.60:0/317613018 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc800096b0 con 0x7efc780039c0 2024-04-25T10:09:45.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.173+0000 7efc8effd640 1 -- 172.21.15.60:0/317613018 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc800a6440 con 0x7efc780039c0 2024-04-25T10:09:45.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.173+0000 7efc8effd640 1 -- 172.21.15.60:0/317613018 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc800a6830 con 0x7efc780039c0 2024-04-25T10:09:45.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.173+0000 7efc9646c640 1 -- 172.21.15.60:0/317613018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc780039c0 msgr2=0x7efc78003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.173+0000 7efc9646c640 1 --2- 172.21.15.60:0/317613018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc780039c0 0x7efc78003de0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7efc80009ae0 tx=0x7efc8001ca10 comp rx=0 tx=0).stop 2024-04-25T10:09:45.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.173+0000 7efc9646c640 1 -- 172.21.15.60:0/317613018 shutdown_connections 2024-04-25T10:09:45.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.173+0000 7efc9646c640 1 --2- 172.21.15.60:0/317613018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc780039c0 0x7efc78003de0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.173+0000 7efc9646c640 1 -- 172.21.15.60:0/317613018 >> 172.21.15.60:0/317613018 conn(0x7efc900cfdd0 msgr2=0x7efc900d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:45.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.173+0000 7efc9646c640 1 -- 172.21.15.60:0/317613018 shutdown_connections 2024-04-25T10:09:45.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.173+0000 7efc9646c640 1 -- 172.21.15.60:0/317613018 wait complete. 2024-04-25T10:09:45.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc9646c640 1 Processor -- start 2024-04-25T10:09:45.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc9646c640 1 -- start start 2024-04-25T10:09:45.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc9646c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc780039c0 0x7efc901355b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc9646c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc90156f40 0x7efc901339d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc9646c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc901573c0 0x7efc90133f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc9646c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9015e420 con 0x7efc901573c0 2024-04-25T10:09:45.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc9646c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc900af820 con 0x7efc780039c0 2024-04-25T10:09:45.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc9646c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc9015e5a0 con 0x7efc90156f40 2024-04-25T10:09:45.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc94a09640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc901573c0 0x7efc90133f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc94a09640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc901573c0 0x7efc90133f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45006/0 (socket says 172.21.15.60:45006) 2024-04-25T10:09:45.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc94a09640 1 -- 172.21.15.60:0/2096712203 learned_addr learned my addr 172.21.15.60:0/2096712203 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:45.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc8f7fe640 1 --2- 172.21.15.60:0/2096712203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc90156f40 0x7efc901339d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc8ffff640 1 --2- 172.21.15.60:0/2096712203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc780039c0 0x7efc901355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc94a09640 1 -- 172.21.15.60:0/2096712203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc90156f40 msgr2=0x7efc901339d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc94a09640 1 --2- 172.21.15.60:0/2096712203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc90156f40 0x7efc901339d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc94a09640 1 -- 172.21.15.60:0/2096712203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc780039c0 msgr2=0x7efc901355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc94a09640 1 --2- 172.21.15.60:0/2096712203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc780039c0 0x7efc901355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc94a09640 1 -- 172.21.15.60:0/2096712203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc90157990 con 0x7efc901573c0 2024-04-25T10:09:45.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc94a09640 1 --2- 172.21.15.60:0/2096712203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc901573c0 0x7efc90133f10 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7efc9006c7a0 tx=0x7efc8800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:45.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc8cff9640 1 -- 172.21.15.60:0/2096712203 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88004d70 con 0x7efc901573c0 2024-04-25T10:09:45.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc8cff9640 1 -- 172.21.15.60:0/2096712203 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8800bb20 con 0x7efc901573c0 2024-04-25T10:09:45.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc9646c640 1 -- 172.21.15.60:0/2096712203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc90157c70 con 0x7efc901573c0 2024-04-25T10:09:45.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc8cff9640 1 -- 172.21.15.60:0/2096712203 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88007070 con 0x7efc901573c0 2024-04-25T10:09:45.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc9646c640 1 -- 172.21.15.60:0/2096712203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc901331d0 con 0x7efc901573c0 2024-04-25T10:09:45.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc9646c640 1 -- 172.21.15.60:0/2096712203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc90074770 con 0x7efc901573c0 2024-04-25T10:09:45.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc8cff9640 1 -- 172.21.15.60:0/2096712203 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efc8801b020 con 0x7efc901573c0 2024-04-25T10:09:45.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.177+0000 7efc8cff9640 1 --2- 172.21.15.60:0/2096712203 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc7c066f60 0x7efc7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.181+0000 7efc8cff9640 1 -- 172.21.15.60:0/2096712203 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc88084fb0 con 0x7efc901573c0 2024-04-25T10:09:45.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.181+0000 7efc8ffff640 1 --2- 172.21.15.60:0/2096712203 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc7c066f60 0x7efc7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.181+0000 7efc8ffff640 1 --2- 172.21.15.60:0/2096712203 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc7c066f60 0x7efc7c069420 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efc800ad000 tx=0x7efc800046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:45.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.181+0000 7efc8cff9640 1 -- 172.21.15.60:0/2096712203 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efc88055fc0 con 0x7efc901573c0 2024-04-25T10:09:45.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:45.377+0000 7efc9646c640 1 -- 172.21.15.60:0/2096712203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efc90072550 con 0x7efc901573c0 2024-04-25T10:09:46.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.361+0000 7efc8cff9640 1 -- 172.21.15.60:0/2096712203 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v38) v1 ==== 118+0+0 (secure 0 0 0) 0x7efc880558c0 con 0x7efc901573c0 2024-04-25T10:09:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.365+0000 7efc9646c640 1 -- 172.21.15.60:0/2096712203 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc7c066f60 msgr2=0x7efc7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.365+0000 7efc9646c640 1 --2- 172.21.15.60:0/2096712203 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc7c066f60 0x7efc7c069420 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efc800ad000 tx=0x7efc800046d0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.365+0000 7efc9646c640 1 -- 172.21.15.60:0/2096712203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc901573c0 msgr2=0x7efc90133f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.365+0000 7efc9646c640 1 --2- 172.21.15.60:0/2096712203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc901573c0 0x7efc90133f10 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7efc9006c7a0 tx=0x7efc8800b4a0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.365+0000 7efc9646c640 1 -- 172.21.15.60:0/2096712203 shutdown_connections 2024-04-25T10:09:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.365+0000 7efc9646c640 1 --2- 172.21.15.60:0/2096712203 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efc7c066f60 0x7efc7c069420 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.365+0000 7efc9646c640 1 --2- 172.21.15.60:0/2096712203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc901573c0 0x7efc90133f10 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.365+0000 7efc9646c640 1 --2- 172.21.15.60:0/2096712203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc90156f40 0x7efc901339d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.365+0000 7efc9646c640 1 --2- 172.21.15.60:0/2096712203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc780039c0 0x7efc901355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.365+0000 7efc9646c640 1 -- 172.21.15.60:0/2096712203 >> 172.21.15.60:0/2096712203 conn(0x7efc900cfdd0 msgr2=0x7efc900b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.365+0000 7efc9646c640 1 -- 172.21.15.60:0/2096712203 shutdown_connections 2024-04-25T10:09:46.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.365+0000 7efc9646c640 1 -- 172.21.15.60:0/2096712203 wait complete. 2024-04-25T10:09:46.395 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:09:46.395 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:09:46.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.517+0000 7f7672639640 1 Processor -- start 2024-04-25T10:09:46.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.517+0000 7f7672639640 1 -- start start 2024-04-25T10:09:46.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.517+0000 7f7672639640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f766c0aeaa0 0x7f766c166450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.517+0000 7f7672639640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f766c162450 0x7f766c06db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.517+0000 7f7672639640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766c166990 con 0x7f766c0b2f70 2024-04-25T10:09:46.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.517+0000 7f7672639640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766c162910 con 0x7f766c0aeaa0 2024-04-25T10:09:46.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.517+0000 7f7672639640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766c162bf0 con 0x7f766c162450 2024-04-25T10:09:46.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.517+0000 7f766bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f766c0aeaa0 0x7f766c166450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:46.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.517+0000 7f766bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f766c0aeaa0 0x7f766c166450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45014/0 (socket says 172.21.15.60:45014) 2024-04-25T10:09:46.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.517+0000 7f7670bd6640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f766c0b2f70 0x7f766c071610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45168/0 (socket says 172.21.15.60:45168) 2024-04-25T10:09:46.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.517+0000 7f766bfff640 1 -- 172.21.15.60:0/3276157891 learned_addr learned my addr 172.21.15.60:0/3276157891 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:46.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.517+0000 7f766b7fe640 1 --2- 172.21.15.60:0/3276157891 >> v2:172.21.15.100:3300/0 conn(0x7f766c162450 0x7f766c06db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:46.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.517+0000 7f766bfff640 1 -- 172.21.15.60:0/3276157891 >> v2:172.21.15.100:3300/0 conn(0x7f766c162450 msgr2=0x7f766c06db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:46.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.517+0000 7f766bfff640 1 --2- 172.21.15.60:0/3276157891 >> v2:172.21.15.100:3300/0 conn(0x7f766c162450 0x7f766c06db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.517+0000 7f766bfff640 1 -- 172.21.15.60:0/3276157891 >> v1:172.21.15.60:6789/0 conn(0x7f766c0b2f70 legacy=0x7f766c071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.517+0000 7f766bfff640 1 -- 172.21.15.60:0/3276157891 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c162ed0 con 0x7f766c0aeaa0 2024-04-25T10:09:46.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f766bfff640 1 --2- 172.21.15.60:0/3276157891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c0aeaa0 0x7f766c166450 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f766c06b8d0 tx=0x7f765c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb0f6f9578acb6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:46.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f766affd640 1 -- 172.21.15.60:0/3276157891 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c0ab610 con 0x7f766c0aeaa0 2024-04-25T10:09:46.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f766affd640 1 -- 172.21.15.60:0/3276157891 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765c0b0070 con 0x7f766c0aeaa0 2024-04-25T10:09:46.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f766affd640 1 -- 172.21.15.60:0/3276157891 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c0ab9d0 con 0x7f766c0aeaa0 2024-04-25T10:09:46.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f7672639640 1 -- 172.21.15.60:0/3276157891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c0aeaa0 msgr2=0x7f766c166450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f7672639640 1 --2- 172.21.15.60:0/3276157891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c0aeaa0 0x7f766c166450 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f766c06b8d0 tx=0x7f765c01c520 comp rx=0 tx=0).stop 2024-04-25T10:09:46.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f7672639640 1 -- 172.21.15.60:0/3276157891 shutdown_connections 2024-04-25T10:09:46.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f7672639640 1 --2- 172.21.15.60:0/3276157891 >> v2:172.21.15.100:3300/0 conn(0x7f766c162450 0x7f766c06db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f7672639640 1 --2- 172.21.15.60:0/3276157891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c0aeaa0 0x7f766c166450 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f7672639640 1 -- 172.21.15.60:0/3276157891 >> 172.21.15.60:0/3276157891 conn(0x7f766c0670b0 msgr2=0x7f766c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:46.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f7672639640 1 -- 172.21.15.60:0/3276157891 shutdown_connections 2024-04-25T10:09:46.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f7672639640 1 -- 172.21.15.60:0/3276157891 wait complete. 2024-04-25T10:09:46.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f7672639640 1 Processor -- start 2024-04-25T10:09:46.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f7672639640 1 -- start start 2024-04-25T10:09:46.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f7672639640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c0aeaa0 0x7f766c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f7672639640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f766c0b2f70 0x7f766c153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f7672639640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f766c162450 0x7f766c151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f7672639640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0b3790 con 0x7f766c0aeaa0 2024-04-25T10:09:46.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f7672639640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c06f2c0 con 0x7f766c162450 2024-04-25T10:09:46.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f7672639640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c15b2c0 con 0x7f766c0b2f70 2024-04-25T10:09:46.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f766bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c0aeaa0 0x7f766c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:46.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f766bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c0aeaa0 0x7f766c0af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45022/0 (socket says 172.21.15.60:45022) 2024-04-25T10:09:46.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.521+0000 7f766bfff640 1 -- 172.21.15.60:0/2748016508 learned_addr learned my addr 172.21.15.60:0/2748016508 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:46.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.525+0000 7f766bfff640 1 -- 172.21.15.60:0/2748016508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f766c0b2f70 msgr2=0x7f766c153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.525+0000 7f766bfff640 1 --2- 172.21.15.60:0/2748016508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f766c0b2f70 0x7f766c153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.525+0000 7f766bfff640 1 -- 172.21.15.60:0/2748016508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f766c162450 msgr2=0x7f766c151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.525+0000 7f766bfff640 1 --2- 172.21.15.60:0/2748016508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f766c162450 0x7f766c151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.525+0000 7f766bfff640 1 -- 172.21.15.60:0/2748016508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c15b510 con 0x7f766c0aeaa0 2024-04-25T10:09:46.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.525+0000 7f766bfff640 1 --2- 172.21.15.60:0/2748016508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c0aeaa0 0x7f766c0af400 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f765c0ad040 tx=0x7f765c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:46.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.525+0000 7f7668ff9640 1 -- 172.21.15.60:0/2748016508 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c0be5f0 con 0x7f766c0aeaa0 2024-04-25T10:09:46.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.525+0000 7f7668ff9640 1 -- 172.21.15.60:0/2748016508 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765c0be7b0 con 0x7f766c0aeaa0 2024-04-25T10:09:46.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.525+0000 7f7672639640 1 -- 172.21.15.60:0/2748016508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766c15b7f0 con 0x7f766c0aeaa0 2024-04-25T10:09:46.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.525+0000 7f7668ff9640 1 -- 172.21.15.60:0/2748016508 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c0beba0 con 0x7f766c0aeaa0 2024-04-25T10:09:46.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.525+0000 7f7672639640 1 -- 172.21.15.60:0/2748016508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766c15bdb0 con 0x7f766c0aeaa0 2024-04-25T10:09:46.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.525+0000 7f7672639640 1 -- 172.21.15.60:0/2748016508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766c06bba0 con 0x7f766c0aeaa0 2024-04-25T10:09:46.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.525+0000 7f7668ff9640 1 -- 172.21.15.60:0/2748016508 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f765c0b43f0 con 0x7f766c0aeaa0 2024-04-25T10:09:46.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.525+0000 7f7668ff9640 1 --2- 172.21.15.60:0/2748016508 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7658066f60 0x7f7658069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.525+0000 7f7668ff9640 1 -- 172.21.15.60:0/2748016508 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f765c11fc40 con 0x7f766c0aeaa0 2024-04-25T10:09:46.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.525+0000 7f766b7fe640 1 --2- 172.21.15.60:0/2748016508 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7658066f60 0x7f7658069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:46.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.529+0000 7f766b7fe640 1 --2- 172.21.15.60:0/2748016508 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7658066f60 0x7f7658069420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f766c15d470 tx=0x7f7660008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:46.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.529+0000 7f7668ff9640 1 -- 172.21.15.60:0/2748016508 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f765c0c4060 con 0x7f766c0aeaa0 2024-04-25T10:09:46.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:46.729+0000 7f7672639640 1 -- 172.21.15.60:0/2748016508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f766c074770 con 0x7f766c0aeaa0 2024-04-25T10:09:46.771 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:46.773 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:47.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.369+0000 7f7668ff9640 1 -- 172.21.15.60:0/2748016508 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v39) v1 ==== 122+0+0 (secure 0 0 0) 0x7f765c0a8050 con 0x7f766c0aeaa0 2024-04-25T10:09:47.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.377+0000 7f7672639640 1 -- 172.21.15.60:0/2748016508 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7658066f60 msgr2=0x7f7658069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.377+0000 7f7672639640 1 --2- 172.21.15.60:0/2748016508 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7658066f60 0x7f7658069420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f766c15d470 tx=0x7f7660008040 comp rx=0 tx=0).stop 2024-04-25T10:09:47.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.377+0000 7f7672639640 1 -- 172.21.15.60:0/2748016508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c0aeaa0 msgr2=0x7f766c0af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.377+0000 7f7672639640 1 --2- 172.21.15.60:0/2748016508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c0aeaa0 0x7f766c0af400 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f765c0ad040 tx=0x7f765c01ad60 comp rx=0 tx=0).stop 2024-04-25T10:09:47.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.377+0000 7f7672639640 1 -- 172.21.15.60:0/2748016508 shutdown_connections 2024-04-25T10:09:47.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.377+0000 7f7672639640 1 --2- 172.21.15.60:0/2748016508 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7658066f60 0x7f7658069420 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.377+0000 7f7672639640 1 --2- 172.21.15.60:0/2748016508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f766c162450 0x7f766c151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.377+0000 7f7672639640 1 --2- 172.21.15.60:0/2748016508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f766c0b2f70 0x7f766c153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.377+0000 7f7672639640 1 --2- 172.21.15.60:0/2748016508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f766c0aeaa0 0x7f766c0af400 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.377+0000 7f7672639640 1 -- 172.21.15.60:0/2748016508 >> 172.21.15.60:0/2748016508 conn(0x7f766c0670b0 msgr2=0x7f766c0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:47.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.377+0000 7f7672639640 1 -- 172.21.15.60:0/2748016508 shutdown_connections 2024-04-25T10:09:47.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.377+0000 7f7672639640 1 -- 172.21.15.60:0/2748016508 wait complete. 2024-04-25T10:09:47.409 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:09:47.409 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:47.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f033b998640 1 Processor -- start 2024-04-25T10:09:47.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f033b998640 1 -- start start 2024-04-25T10:09:47.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f033b998640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f033415deb0 0x7f03340b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f033b998640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f03340d0340 0x7f03340d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f033b998640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03340ce2b0 0x7f03340ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f033b998640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03340cf2c0 con 0x7f033415deb0 2024-04-25T10:09:47.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f033b998640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03340cefe0 con 0x7f03340d0340 2024-04-25T10:09:47.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f033b998640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03340d1d70 con 0x7f03340ce2b0 2024-04-25T10:09:47.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f0339734640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f033415deb0 0x7f03340b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f0339734640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f033415deb0 0x7f03340b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45026/0 (socket says 172.21.15.60:45026) 2024-04-25T10:09:47.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f0339734640 1 -- 172.21.15.60:0/786775816 learned_addr learned my addr 172.21.15.60:0/786775816 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:47.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f0338f33640 1 --2- 172.21.15.60:0/786775816 >> v2:172.21.15.100:3300/0 conn(0x7f03340d0340 0x7f03340d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f0339f35640 1 --2- 172.21.15.60:0/786775816 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03340ce2b0 0x7f03340ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f0339734640 1 -- 172.21.15.60:0/786775816 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03340ce2b0 msgr2=0x7f03340ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:47.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f0339734640 1 --2- 172.21.15.60:0/786775816 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03340ce2b0 0x7f03340ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f0339734640 1 -- 172.21.15.60:0/786775816 >> v2:172.21.15.100:3300/0 conn(0x7f03340d0340 msgr2=0x7f03340d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:47.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f0339734640 1 --2- 172.21.15.60:0/786775816 >> v2:172.21.15.100:3300/0 conn(0x7f03340d0340 0x7f03340d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.533+0000 7f0339734640 1 -- 172.21.15.60:0/786775816 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033406d200 con 0x7f033415deb0 2024-04-25T10:09:47.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.537+0000 7f0339734640 1 --2- 172.21.15.60:0/786775816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f033415deb0 0x7f03340b6c10 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f03340b6490 tx=0x7f033001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e7607dd94422af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:47.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.537+0000 7f032bfff640 1 -- 172.21.15.60:0/786775816 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03300a5630 con 0x7f033415deb0 2024-04-25T10:09:47.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.537+0000 7f032bfff640 1 -- 172.21.15.60:0/786775816 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03300a57d0 con 0x7f033415deb0 2024-04-25T10:09:47.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.537+0000 7f032bfff640 1 -- 172.21.15.60:0/786775816 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03300a5ba0 con 0x7f033415deb0 2024-04-25T10:09:47.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.537+0000 7f033b998640 1 -- 172.21.15.60:0/786775816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f033415deb0 msgr2=0x7f03340b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.537+0000 7f033b998640 1 --2- 172.21.15.60:0/786775816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f033415deb0 0x7f03340b6c10 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f03340b6490 tx=0x7f033001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.537+0000 7f033b998640 1 -- 172.21.15.60:0/786775816 shutdown_connections 2024-04-25T10:09:47.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.537+0000 7f033b998640 1 --2- 172.21.15.60:0/786775816 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03340ce2b0 0x7f03340ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.537+0000 7f033b998640 1 --2- 172.21.15.60:0/786775816 >> v2:172.21.15.100:3300/0 conn(0x7f03340d0340 0x7f03340d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.537+0000 7f033b998640 1 --2- 172.21.15.60:0/786775816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f033415deb0 0x7f03340b6c10 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.537+0000 7f033b998640 1 -- 172.21.15.60:0/786775816 >> 172.21.15.60:0/786775816 conn(0x7f033406c090 msgr2=0x7f033406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:47.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.537+0000 7f033b998640 1 -- 172.21.15.60:0/786775816 shutdown_connections 2024-04-25T10:09:47.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.537+0000 7f033b998640 1 -- 172.21.15.60:0/786775816 wait complete. 2024-04-25T10:09:47.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f033b998640 1 Processor -- start 2024-04-25T10:09:47.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f033b998640 1 -- start start 2024-04-25T10:09:47.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f033b998640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03340ce2b0 0x7f0334164520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f033b998640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03340d0340 0x7f033414c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f033b998640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f033415deb0 0x7f03341629e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f033b998640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03340b5d40 con 0x7f03340d0340 2024-04-25T10:09:47.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f033b998640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03340b5bc0 con 0x7f033415deb0 2024-04-25T10:09:47.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f033b998640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03340b5ec0 con 0x7f03340ce2b0 2024-04-25T10:09:47.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f0338f33640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03340d0340 0x7f033414c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f0338f33640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03340d0340 0x7f033414c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45040/0 (socket says 172.21.15.60:45040) 2024-04-25T10:09:47.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f0338f33640 1 -- 172.21.15.60:0/3028613906 learned_addr learned my addr 172.21.15.60:0/3028613906 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:47.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f0339f35640 1 --2- 172.21.15.60:0/3028613906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f033415deb0 0x7f03341629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f0339734640 1 --2- 172.21.15.60:0/3028613906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03340ce2b0 0x7f0334164520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f0338f33640 1 -- 172.21.15.60:0/3028613906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03340ce2b0 msgr2=0x7f0334164520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f0338f33640 1 --2- 172.21.15.60:0/3028613906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03340ce2b0 0x7f0334164520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f0338f33640 1 -- 172.21.15.60:0/3028613906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f033415deb0 msgr2=0x7f03341629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f0338f33640 1 --2- 172.21.15.60:0/3028613906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f033415deb0 0x7f03341629e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f0338f33640 1 -- 172.21.15.60:0/3028613906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033414db90 con 0x7f03340d0340 2024-04-25T10:09:47.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f0338f33640 1 --2- 172.21.15.60:0/3028613906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03340d0340 0x7f033414c0d0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f032400d830 tx=0x7f032400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:47.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f0329ffb640 1 -- 172.21.15.60:0/3028613906 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032400b6c0 con 0x7f03340d0340 2024-04-25T10:09:47.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f0329ffb640 1 -- 172.21.15.60:0/3028613906 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0324019070 con 0x7f03340d0340 2024-04-25T10:09:47.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f033b998640 1 -- 172.21.15.60:0/3028613906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033414de70 con 0x7f03340d0340 2024-04-25T10:09:47.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f0329ffb640 1 -- 172.21.15.60:0/3028613906 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032400ba60 con 0x7f03340d0340 2024-04-25T10:09:47.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f033b998640 1 -- 172.21.15.60:0/3028613906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033414e710 con 0x7f03340d0340 2024-04-25T10:09:47.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.541+0000 7f033b998640 1 -- 172.21.15.60:0/3028613906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033414e040 con 0x7f03340d0340 2024-04-25T10:09:47.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.545+0000 7f0329ffb640 1 -- 172.21.15.60:0/3028613906 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f032401e460 con 0x7f03340d0340 2024-04-25T10:09:47.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.545+0000 7f0329ffb640 1 --2- 172.21.15.60:0/3028613906 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0318066f60 0x7f0318069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.545+0000 7f0329ffb640 1 -- 172.21.15.60:0/3028613906 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0324085810 con 0x7f03340d0340 2024-04-25T10:09:47.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.545+0000 7f0339734640 1 --2- 172.21.15.60:0/3028613906 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0318066f60 0x7f0318069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.545+0000 7f0339734640 1 --2- 172.21.15.60:0/3028613906 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0318066f60 0x7f0318069420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f03300ae000 tx=0x7f03300a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:47.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.549+0000 7f0329ffb640 1 -- 172.21.15.60:0/3028613906 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0324056780 con 0x7f03340d0340 2024-04-25T10:09:47.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.745+0000 7f033b998640 1 -- 172.21.15.60:0/3028613906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0334074770 con 0x7f03340d0340 2024-04-25T10:09:47.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.749+0000 7f0329ffb640 1 -- 172.21.15.60:0/3028613906 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f0324056080 con 0x7f03340d0340 2024-04-25T10:09:47.757 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:47.757 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3185787428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3185787428},{"type":"v1","addr":"172.21.15.100:6836","nonce":3185787428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2667415244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2667415244},{"type":"v1","addr":"172.21.15.100:6837","nonce":2667415244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/56747111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":56747111},{"type":"v1","addr":"172.21.15.60:6834","nonce":56747111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:43.707831+0000","modified":"2024-04-25T10:09:46.733783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6873":{"gid":6873,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1289503429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1289503429},{"type":"v1","addr":"172.21.15.60:6835","nonce":1289503429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:09:47.758 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 39 2024-04-25T10:09:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.749+0000 7f033b998640 1 -- 172.21.15.60:0/3028613906 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0318066f60 msgr2=0x7f0318069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.749+0000 7f033b998640 1 --2- 172.21.15.60:0/3028613906 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0318066f60 0x7f0318069420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f03300ae000 tx=0x7f03300a7070 comp rx=0 tx=0).stop 2024-04-25T10:09:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.749+0000 7f033b998640 1 -- 172.21.15.60:0/3028613906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03340d0340 msgr2=0x7f033414c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.749+0000 7f033b998640 1 --2- 172.21.15.60:0/3028613906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03340d0340 0x7f033414c0d0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f032400d830 tx=0x7f032400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.749+0000 7f033b998640 1 -- 172.21.15.60:0/3028613906 shutdown_connections 2024-04-25T10:09:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.749+0000 7f033b998640 1 --2- 172.21.15.60:0/3028613906 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0318066f60 0x7f0318069420 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.749+0000 7f033b998640 1 --2- 172.21.15.60:0/3028613906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f033415deb0 0x7f03341629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.749+0000 7f033b998640 1 --2- 172.21.15.60:0/3028613906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03340d0340 0x7f033414c0d0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.749+0000 7f033b998640 1 --2- 172.21.15.60:0/3028613906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03340ce2b0 0x7f0334164520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.749+0000 7f033b998640 1 -- 172.21.15.60:0/3028613906 >> 172.21.15.60:0/3028613906 conn(0x7f033406c090 msgr2=0x7f0334151510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.749+0000 7f033b998640 1 -- 172.21.15.60:0/3028613906 shutdown_connections 2024-04-25T10:09:47.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.749+0000 7f033b998640 1 -- 172.21.15.60:0/3028613906 wait complete. 2024-04-25T10:09:47.777 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:47.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.889+0000 7f989039b640 1 Processor -- start 2024-04-25T10:09:47.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.889+0000 7f989039b640 1 -- start start 2024-04-25T10:09:47.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.889+0000 7f989039b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98880ad6e0 0x7f988806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.889+0000 7f989039b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f98880acd70 0x7f98880adac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.889+0000 7f989039b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98880b4fd0 con 0x7f98880b44e0 2024-04-25T10:09:47.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.889+0000 7f989039b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98880ae000 con 0x7f98880ad6e0 2024-04-25T10:09:47.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.889+0000 7f989039b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98880b4cf0 con 0x7f98880acd70 2024-04-25T10:09:47.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.889+0000 7f988e137640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98880ad6e0 0x7f988806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.889+0000 7f988e137640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98880ad6e0 0x7f988806db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45050/0 (socket says 172.21.15.60:45050) 2024-04-25T10:09:47.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.889+0000 7f988e137640 1 -- 172.21.15.60:0/2716905686 learned_addr learned my addr 172.21.15.60:0/2716905686 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:47.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.889+0000 7f988d936640 1 --2- 172.21.15.60:0/2716905686 >> v2:172.21.15.100:3300/0 conn(0x7f98880acd70 0x7f98880adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.889+0000 7f988e137640 1 -- 172.21.15.60:0/2716905686 >> v2:172.21.15.100:3300/0 conn(0x7f98880acd70 msgr2=0x7f98880adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:47.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.889+0000 7f988e137640 1 --2- 172.21.15.60:0/2716905686 >> v2:172.21.15.100:3300/0 conn(0x7f98880acd70 0x7f98880adac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.889+0000 7f988e137640 1 -- 172.21.15.60:0/2716905686 >> v1:172.21.15.60:6789/0 conn(0x7f98880b44e0 legacy=0x7f988806b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.889+0000 7f988e137640 1 -- 172.21.15.60:0/2716905686 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988806ad20 con 0x7f98880ad6e0 2024-04-25T10:09:47.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.889+0000 7f988d936640 1 --2- 172.21.15.60:0/2716905686 >> v2:172.21.15.100:3300/0 conn(0x7f98880acd70 0x7f98880adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:47.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.893+0000 7f988e137640 1 --2- 172.21.15.60:0/2716905686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98880ad6e0 0x7f988806db20 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f988809dfb0 tx=0x7f988401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91849f50ee11e95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:47.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.893+0000 7f988d135640 1 -- 172.21.15.60:0/2716905686 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98840b0070 con 0x7f98880ad6e0 2024-04-25T10:09:47.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.893+0000 7f988d135640 1 -- 172.21.15.60:0/2716905686 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98840ab600 con 0x7f98880ad6e0 2024-04-25T10:09:47.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.893+0000 7f988d135640 1 -- 172.21.15.60:0/2716905686 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98840ab9d0 con 0x7f98880ad6e0 2024-04-25T10:09:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.893+0000 7f989039b640 1 -- 172.21.15.60:0/2716905686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98880ad6e0 msgr2=0x7f988806db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.893+0000 7f989039b640 1 --2- 172.21.15.60:0/2716905686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98880ad6e0 0x7f988806db20 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f988809dfb0 tx=0x7f988401c460 comp rx=0 tx=0).stop 2024-04-25T10:09:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.893+0000 7f989039b640 1 -- 172.21.15.60:0/2716905686 shutdown_connections 2024-04-25T10:09:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.893+0000 7f989039b640 1 --2- 172.21.15.60:0/2716905686 >> v2:172.21.15.100:3300/0 conn(0x7f98880acd70 0x7f98880adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.893+0000 7f989039b640 1 --2- 172.21.15.60:0/2716905686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98880ad6e0 0x7f988806db20 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.893+0000 7f989039b640 1 -- 172.21.15.60:0/2716905686 >> 172.21.15.60:0/2716905686 conn(0x7f98880d0380 msgr2=0x7f98880d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.893+0000 7f989039b640 1 -- 172.21.15.60:0/2716905686 shutdown_connections 2024-04-25T10:09:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.893+0000 7f989039b640 1 -- 172.21.15.60:0/2716905686 wait complete. 2024-04-25T10:09:47.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.893+0000 7f989039b640 1 Processor -- start 2024-04-25T10:09:47.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.893+0000 7f989039b640 1 -- start start 2024-04-25T10:09:47.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f989039b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98880acd70 0x7f9888168220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f989039b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98880ad6e0 0x7f988813e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f989039b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98880b44e0 0x7f9888153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f989039b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988816ab80 con 0x7f98880acd70 2024-04-25T10:09:47.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f989039b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988816aa00 con 0x7f98880ad6e0 2024-04-25T10:09:47.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f989039b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988816ad00 con 0x7f98880b44e0 2024-04-25T10:09:47.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f988e137640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98880acd70 0x7f9888168220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f988e137640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98880acd70 0x7f9888168220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45052/0 (socket says 172.21.15.60:45052) 2024-04-25T10:09:47.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f988e137640 1 -- 172.21.15.60:0/1953067545 learned_addr learned my addr 172.21.15.60:0/1953067545 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:47.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f988d936640 1 --2- 172.21.15.60:0/1953067545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98880ad6e0 0x7f988813e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f988e938640 1 --2- 172.21.15.60:0/1953067545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98880b44e0 0x7f9888153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f988e137640 1 -- 172.21.15.60:0/1953067545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98880b44e0 msgr2=0x7f9888153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f988e137640 1 --2- 172.21.15.60:0/1953067545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98880b44e0 0x7f9888153020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f988e137640 1 -- 172.21.15.60:0/1953067545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98880ad6e0 msgr2=0x7f988813e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f988e137640 1 --2- 172.21.15.60:0/1953067545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98880ad6e0 0x7f988813e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f988e137640 1 -- 172.21.15.60:0/1953067545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9888168760 con 0x7f98880acd70 2024-04-25T10:09:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f988d936640 1 --2- 172.21.15.60:0/1953067545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98880ad6e0 0x7f988813e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f988e137640 1 --2- 172.21.15.60:0/1953067545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98880acd70 0x7f9888168220 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f98840ae040 tx=0x7f988401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f987effd640 1 -- 172.21.15.60:0/1953067545 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98840ba020 con 0x7f98880acd70 2024-04-25T10:09:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f987effd640 1 -- 172.21.15.60:0/1953067545 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98840b6640 con 0x7f98880acd70 2024-04-25T10:09:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f989039b640 1 -- 172.21.15.60:0/1953067545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9888156750 con 0x7f98880acd70 2024-04-25T10:09:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f987effd640 1 -- 172.21.15.60:0/1953067545 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98840b0040 con 0x7f98880acd70 2024-04-25T10:09:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f989039b640 1 -- 172.21.15.60:0/1953067545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9888156d10 con 0x7f98880acd70 2024-04-25T10:09:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f989039b640 1 -- 172.21.15.60:0/1953067545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9888074770 con 0x7f98880acd70 2024-04-25T10:09:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f987effd640 1 -- 172.21.15.60:0/1953067545 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98840bd020 con 0x7f98880acd70 2024-04-25T10:09:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f987effd640 1 --2- 172.21.15.60:0/1953067545 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f986c066f60 0x7f986c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f987effd640 1 -- 172.21.15.60:0/1953067545 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f988411f310 con 0x7f98880acd70 2024-04-25T10:09:47.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.897+0000 7f988d936640 1 --2- 172.21.15.60:0/1953067545 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f986c066f60 0x7f986c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.901+0000 7f988d936640 1 --2- 172.21.15.60:0/1953067545 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f986c066f60 0x7f986c069420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f98881582a0 tx=0x7f9878008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:47.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:47.905+0000 7f987effd640 1 -- 172.21.15.60:0/1953067545 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f98840c4030 con 0x7f98880acd70 2024-04-25T10:09:48.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.065+0000 7f989039b640 1 -- 172.21.15.60:0/1953067545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f988806b800 con 0x7f98880acd70 2024-04-25T10:09:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.069+0000 7f987effd640 1 -- 172.21.15.60:0/1953067545 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16680 (secure 0 0 0) 0x7f9884016110 con 0x7f98880acd70 2024-04-25T10:09:48.077 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:48.077 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":69,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:09:45.353654+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:42.005281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:42.710028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.069+0000 7f989039b640 1 -- 172.21.15.60:0/1953067545 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f986c066f60 msgr2=0x7f986c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.069+0000 7f989039b640 1 --2- 172.21.15.60:0/1953067545 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f986c066f60 0x7f986c069420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f98881582a0 tx=0x7f9878008040 comp rx=0 tx=0).stop 2024-04-25T10:09:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.069+0000 7f989039b640 1 -- 172.21.15.60:0/1953067545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98880acd70 msgr2=0x7f9888168220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.069+0000 7f989039b640 1 --2- 172.21.15.60:0/1953067545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98880acd70 0x7f9888168220 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f98840ae040 tx=0x7f988401aca0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.069+0000 7f989039b640 1 -- 172.21.15.60:0/1953067545 shutdown_connections 2024-04-25T10:09:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.069+0000 7f989039b640 1 --2- 172.21.15.60:0/1953067545 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f986c066f60 0x7f986c069420 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.069+0000 7f989039b640 1 --2- 172.21.15.60:0/1953067545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98880b44e0 0x7f9888153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.069+0000 7f989039b640 1 --2- 172.21.15.60:0/1953067545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98880ad6e0 0x7f988813e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.069+0000 7f989039b640 1 --2- 172.21.15.60:0/1953067545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98880acd70 0x7f9888168220 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.069+0000 7f989039b640 1 -- 172.21.15.60:0/1953067545 >> 172.21.15.60:0/1953067545 conn(0x7f98880d0380 msgr2=0x7f98880afd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.069+0000 7f989039b640 1 -- 172.21.15.60:0/1953067545 shutdown_connections 2024-04-25T10:09:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.069+0000 7f989039b640 1 -- 172.21.15.60:0/1953067545 wait complete. 2024-04-25T10:09:48.095 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:09:48.096 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:09:48.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.209+0000 7f858d39f640 1 Processor -- start 2024-04-25T10:09:48.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.209+0000 7f858d39f640 1 -- start start 2024-04-25T10:09:48.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f858d39f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85880ce7a0 0x7f85880ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f858d39f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f858815bc90 0x7f85880b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f858d39f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f85880b4660 0x7f85880b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f858d39f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85880cc370 con 0x7f85880b4660 2024-04-25T10:09:48.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f858d39f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85880cf0c0 con 0x7f858815bc90 2024-04-25T10:09:48.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f858d39f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8588071ca0 con 0x7f85880ce7a0 2024-04-25T10:09:48.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f85877fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f85880b4660 0x7f85880b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f85877fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f85880b4660 0x7f85880b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45068/0 (socket says 172.21.15.60:45068) 2024-04-25T10:09:48.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f85877fe640 1 -- 172.21.15.60:0/3503374005 learned_addr learned my addr 172.21.15.60:0/3503374005 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:48.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f85867fc640 1 --2- 172.21.15.60:0/3503374005 >> v2:172.21.15.100:3300/0 conn(0x7f858815bc90 0x7f85880b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f8586ffd640 1 --2- 172.21.15.60:0/3503374005 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85880ce7a0 0x7f85880ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f85877fe640 1 -- 172.21.15.60:0/3503374005 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85880ce7a0 msgr2=0x7f85880ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:48.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f85877fe640 1 --2- 172.21.15.60:0/3503374005 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85880ce7a0 0x7f85880ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f85877fe640 1 -- 172.21.15.60:0/3503374005 >> v2:172.21.15.100:3300/0 conn(0x7f858815bc90 msgr2=0x7f85880b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:48.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f85877fe640 1 --2- 172.21.15.60:0/3503374005 >> v2:172.21.15.100:3300/0 conn(0x7f858815bc90 0x7f85880b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f85877fe640 1 -- 172.21.15.60:0/3503374005 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85880cca40 con 0x7f85880b4660 2024-04-25T10:09:48.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f85877fe640 1 --2- 172.21.15.60:0/3503374005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85880b4660 0x7f85880b2b10 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f8580009e50 tx=0x7f858001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e5e9b3f110775f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:48.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f8585ffb640 1 -- 172.21.15.60:0/3503374005 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85800a5670 con 0x7f85880b4660 2024-04-25T10:09:48.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.213+0000 7f8585ffb640 1 -- 172.21.15.60:0/3503374005 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85800a5810 con 0x7f85880b4660 2024-04-25T10:09:48.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f8585ffb640 1 -- 172.21.15.60:0/3503374005 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85800a5be0 con 0x7f85880b4660 2024-04-25T10:09:48.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 -- 172.21.15.60:0/3503374005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85880b4660 msgr2=0x7f85880b2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 --2- 172.21.15.60:0/3503374005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85880b4660 0x7f85880b2b10 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f8580009e50 tx=0x7f858001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 -- 172.21.15.60:0/3503374005 shutdown_connections 2024-04-25T10:09:48.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 --2- 172.21.15.60:0/3503374005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85880b4660 0x7f85880b2b10 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 --2- 172.21.15.60:0/3503374005 >> v2:172.21.15.100:3300/0 conn(0x7f858815bc90 0x7f85880b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 --2- 172.21.15.60:0/3503374005 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85880ce7a0 0x7f85880ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 -- 172.21.15.60:0/3503374005 >> 172.21.15.60:0/3503374005 conn(0x7f85880b12d0 msgr2=0x7f85880b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:48.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 -- 172.21.15.60:0/3503374005 shutdown_connections 2024-04-25T10:09:48.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 -- 172.21.15.60:0/3503374005 wait complete. 2024-04-25T10:09:48.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 Processor -- start 2024-04-25T10:09:48.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 -- start start 2024-04-25T10:09:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85880ce7a0 0x7f8588139f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f858815bc90 0x7f858815b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8588149460 0x7f85880afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85880b3db0 con 0x7f8588149460 2024-04-25T10:09:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85880b3460 con 0x7f85880ce7a0 2024-04-25T10:09:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85880b4280 con 0x7f858815bc90 2024-04-25T10:09:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f85877fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8588149460 0x7f85880afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f85877fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8588149460 0x7f85880afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45070/0 (socket says 172.21.15.60:45070) 2024-04-25T10:09:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f85877fe640 1 -- 172.21.15.60:0/1151236913 learned_addr learned my addr 172.21.15.60:0/1151236913 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f8586ffd640 1 --2- 172.21.15.60:0/1151236913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85880ce7a0 0x7f8588139f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f85867fc640 1 --2- 172.21.15.60:0/1151236913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f858815bc90 0x7f858815b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f85877fe640 1 -- 172.21.15.60:0/1151236913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f858815bc90 msgr2=0x7f858815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f85877fe640 1 --2- 172.21.15.60:0/1151236913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f858815bc90 0x7f858815b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f85877fe640 1 -- 172.21.15.60:0/1151236913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85880ce7a0 msgr2=0x7f8588139f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f85877fe640 1 --2- 172.21.15.60:0/1151236913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85880ce7a0 0x7f8588139f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f85877fe640 1 -- 172.21.15.60:0/1151236913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858815b7b0 con 0x7f8588149460 2024-04-25T10:09:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f85877fe640 1 --2- 172.21.15.60:0/1151236913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8588149460 0x7f85880afb70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f85800af040 tx=0x7f8580019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f856f7fe640 1 -- 172.21.15.60:0/1151236913 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8580009510 con 0x7f8588149460 2024-04-25T10:09:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f856f7fe640 1 -- 172.21.15.60:0/1151236913 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85800be070 con 0x7f8588149460 2024-04-25T10:09:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f856f7fe640 1 -- 172.21.15.60:0/1151236913 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85800b88b0 con 0x7f8588149460 2024-04-25T10:09:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 -- 172.21.15.60:0/1151236913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8588148050 con 0x7f8588149460 2024-04-25T10:09:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.217+0000 7f858d39f640 1 -- 172.21.15.60:0/1151236913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8588148610 con 0x7f8588149460 2024-04-25T10:09:48.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.221+0000 7f858d39f640 1 -- 172.21.15.60:0/1151236913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8588074770 con 0x7f8588149460 2024-04-25T10:09:48.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.221+0000 7f856f7fe640 1 -- 172.21.15.60:0/1151236913 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85800b8a50 con 0x7f8588149460 2024-04-25T10:09:48.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.221+0000 7f856f7fe640 1 --2- 172.21.15.60:0/1151236913 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8574066f60 0x7f8574069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.221+0000 7f856f7fe640 1 -- 172.21.15.60:0/1151236913 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8580120220 con 0x7f8588149460 2024-04-25T10:09:48.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.221+0000 7f8586ffd640 1 --2- 172.21.15.60:0/1151236913 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8574066f60 0x7f8574069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.221+0000 7f8586ffd640 1 --2- 172.21.15.60:0/1151236913 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8574066f60 0x7f8574069420 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f858815ebb0 tx=0x7f857c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:48.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.225+0000 7f856f7fe640 1 -- 172.21.15.60:0/1151236913 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f85800165e0 con 0x7f8588149460 2024-04-25T10:09:48.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.381+0000 7f858d39f640 1 -- 172.21.15.60:0/1151236913 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8588147e00 con 0x7f8574066f60 2024-04-25T10:09:48.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.401+0000 7f856f7fe640 1 -- 172.21.15.60:0/1151236913 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+241841 (secure 0 0 0) 0x7f8588147e00 con 0x7f8574066f60 2024-04-25T10:09:48.412 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:48.413 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:09:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.405+0000 7f858d39f640 1 -- 172.21.15.60:0/1151236913 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8574066f60 msgr2=0x7f8574069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.405+0000 7f858d39f640 1 --2- 172.21.15.60:0/1151236913 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8574066f60 0x7f8574069420 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f858815ebb0 tx=0x7f857c008040 comp rx=0 tx=0).stop 2024-04-25T10:09:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.405+0000 7f858d39f640 1 -- 172.21.15.60:0/1151236913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8588149460 msgr2=0x7f85880afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.405+0000 7f858d39f640 1 --2- 172.21.15.60:0/1151236913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8588149460 0x7f85880afb70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f85800af040 tx=0x7f8580019790 comp rx=0 tx=0).stop 2024-04-25T10:09:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.405+0000 7f858d39f640 1 -- 172.21.15.60:0/1151236913 shutdown_connections 2024-04-25T10:09:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.405+0000 7f858d39f640 1 --2- 172.21.15.60:0/1151236913 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8574066f60 0x7f8574069420 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.405+0000 7f858d39f640 1 --2- 172.21.15.60:0/1151236913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8588149460 0x7f85880afb70 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.405+0000 7f858d39f640 1 --2- 172.21.15.60:0/1151236913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f858815bc90 0x7f858815b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.405+0000 7f858d39f640 1 --2- 172.21.15.60:0/1151236913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85880ce7a0 0x7f8588139f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.405+0000 7f858d39f640 1 -- 172.21.15.60:0/1151236913 >> 172.21.15.60:0/1151236913 conn(0x7f85880b12d0 msgr2=0x7f8588154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:48.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.405+0000 7f858d39f640 1 -- 172.21.15.60:0/1151236913 shutdown_connections 2024-04-25T10:09:48.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.405+0000 7f858d39f640 1 -- 172.21.15.60:0/1151236913 wait complete. 2024-04-25T10:09:48.430 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-04-25T10:09:46.479733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":104,"acting":104,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260940,"kb_used_data":7308,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754713780,"statfs":{"total":773094113280,"available":772826910720,"internally_reserved":0,"allocated":7483392,"data_stored":3790117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.154298"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.362772+0000","last_change":"2024-04-25T10:09:43.362772+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.362772+0000","last_undegraded":"2024-04-25T10:09:43.362772+0000","last_fullsized":"2024-04-25T10:09:43.362772+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.345340+0000","last_change":"2024-04-25T10:09:43.345340+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.345340+0000","last_undegraded":"2024-04-25T10:09:43.345340+0000","last_fullsized":"2024-04-25T10:09:43.345340+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.360649+0000","last_change":"2024-04-25T10:09:43.360649+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.360649+0000","last_undegraded":"2024-04-25T10:09:43.360649+0000","last_fullsized":"2024-04-25T10:09:43.360649+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.355147+0000","last_change":"2024-04-25T10:09:43.355147+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.355147+0000","last_undegraded":"2024-04-25T10:09:43.355147+0000","last_fullsized":"2024-04-25T10:09:43.355147+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.347809+0000","last_change":"2024-04-25T10:09:43.347809+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.347809+0000","last_undegraded":"2024-04-25T10:09:43.347809+0000","last_fullsized":"2024-04-25T10:09:43.347809+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.345489+0000","last_change":"2024-04-25T10:09:43.345489+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.345489+0000","last_undegraded":"2024-04-25T10:09:43.345489+0000","last_fullsized":"2024-04-25T10:09:43.345489+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.359668+0000","last_change":"2024-04-25T10:09:43.359668+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.359668+0000","last_undegraded":"2024-04-25T10:09:43.359668+0000","last_fullsized":"2024-04-25T10:09:43.359668+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.350981+0000","last_change":"2024-04-25T10:09:43.350981+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.350981+0000","last_undegraded":"2024-04-25T10:09:43.350981+0000","last_fullsized":"2024-04-25T10:09:43.350981+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.345976+0000","last_change":"2024-04-25T10:09:43.345976+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.345976+0000","last_undegraded":"2024-04-25T10:09:43.345976+0000","last_fullsized":"2024-04-25T10:09:43.345976+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.343084+0000","last_change":"2024-04-25T10:09:43.343084+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.343084+0000","last_undegraded":"2024-04-25T10:09:43.343084+0000","last_fullsized":"2024-04-25T10:09:43.343084+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.359499+0000","last_change":"2024-04-25T10:09:43.359499+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.359499+0000","last_undegraded":"2024-04-25T10:09:43.359499+0000","last_fullsized":"2024-04-25T10:09:43.359499+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T10:09:43.382563+0000","last_change":"2024-04-25T10:09:43.379590+0000","last_active":"2024-04-25T10:09:43.382563+0000","last_peered":"2024-04-25T10:09:43.382563+0000","last_clean":"2024-04-25T10:09:43.382563+0000","last_became_active":"2024-04-25T10:09:43.378739+0000","last_became_peered":"2024-04-25T10:09:43.378739+0000","last_unstale":"2024-04-25T10:09:43.382563+0000","last_undegraded":"2024-04-25T10:09:43.382563+0000","last_fullsized":"2024-04-25T10:09:43.382563+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:20.698509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.328329+0000","last_change":"2024-04-25T10:09:42.328329+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.328329+0000","last_undegraded":"2024-04-25T10:09:42.328329+0000","last_fullsized":"2024-04-25T10:09:42.328329+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T10:09:43.379437+0000","last_change":"2024-04-25T10:09:43.372515+0000","last_active":"2024-04-25T10:09:43.379437+0000","last_peered":"2024-04-25T10:09:43.379437+0000","last_clean":"2024-04-25T10:09:43.379437+0000","last_became_active":"2024-04-25T10:09:43.371360+0000","last_became_peered":"2024-04-25T10:09:43.371360+0000","last_unstale":"2024-04-25T10:09:43.379437+0000","last_undegraded":"2024-04-25T10:09:43.379437+0000","last_fullsized":"2024-04-25T10:09:43.379437+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:57.733608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.354086+0000","last_change":"2024-04-25T10:09:43.354086+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.354086+0000","last_undegraded":"2024-04-25T10:09:43.354086+0000","last_fullsized":"2024-04-25T10:09:43.354086+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.337294+0000","last_change":"2024-04-25T10:09:42.337294+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.337294+0000","last_undegraded":"2024-04-25T10:09:42.337294+0000","last_fullsized":"2024-04-25T10:09:42.337294+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.335707+0000","last_change":"2024-04-25T10:09:42.335707+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.335707+0000","last_undegraded":"2024-04-25T10:09:42.335707+0000","last_fullsized":"2024-04-25T10:09:42.335707+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.346638+0000","last_change":"2024-04-25T10:09:43.346638+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.346638+0000","last_undegraded":"2024-04-25T10:09:43.346638+0000","last_fullsized":"2024-04-25T10:09:43.346638+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.342205+0000","last_change":"2024-04-25T10:09:42.342205+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.342205+0000","last_undegraded":"2024-04-25T10:09:42.342205+0000","last_fullsized":"2024-04-25T10:09:42.342205+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.329506+0000","last_change":"2024-04-25T10:09:42.329506+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.329506+0000","last_undegraded":"2024-04-25T10:09:42.329506+0000","last_fullsized":"2024-04-25T10:09:42.329506+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.331060+0000","last_change":"2024-04-25T10:09:42.331060+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.331060+0000","last_undegraded":"2024-04-25T10:09:42.331060+0000","last_fullsized":"2024-04-25T10:09:42.331060+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T10:09:43.367541+0000","last_change":"2024-04-25T10:09:43.365252+0000","last_active":"2024-04-25T10:09:43.367541+0000","last_peered":"2024-04-25T10:09:43.367541+0000","last_clean":"2024-04-25T10:09:43.367541+0000","last_became_active":"2024-04-25T10:09:43.363864+0000","last_became_peered":"2024-04-25T10:09:43.363864+0000","last_unstale":"2024-04-25T10:09:43.367541+0000","last_undegraded":"2024-04-25T10:09:43.367541+0000","last_fullsized":"2024-04-25T10:09:43.367541+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:51.130358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.361925+0000","last_change":"2024-04-25T10:09:43.361925+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.361925+0000","last_undegraded":"2024-04-25T10:09:43.361925+0000","last_fullsized":"2024-04-25T10:09:43.361925+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.338258+0000","last_change":"2024-04-25T10:09:42.338258+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.338258+0000","last_undegraded":"2024-04-25T10:09:42.338258+0000","last_fullsized":"2024-04-25T10:09:42.338258+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.336004+0000","last_change":"2024-04-25T10:09:42.336004+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.336004+0000","last_undegraded":"2024-04-25T10:09:42.336004+0000","last_fullsized":"2024-04-25T10:09:42.336004+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.329557+0000","last_change":"2024-04-25T10:09:42.329557+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.329557+0000","last_undegraded":"2024-04-25T10:09:42.329557+0000","last_fullsized":"2024-04-25T10:09:42.329557+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.353555+0000","last_change":"2024-04-25T10:09:43.353555+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.353555+0000","last_undegraded":"2024-04-25T10:09:43.353555+0000","last_fullsized":"2024-04-25T10:09:43.353555+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.329516+0000","last_change":"2024-04-25T10:09:42.329516+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.329516+0000","last_undegraded":"2024-04-25T10:09:42.329516+0000","last_fullsized":"2024-04-25T10:09:42.329516+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.340602+0000","last_change":"2024-04-25T10:09:42.340602+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.340602+0000","last_undegraded":"2024-04-25T10:09:42.340602+0000","last_fullsized":"2024-04-25T10:09:42.340602+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.331026+0000","last_change":"2024-04-25T10:09:42.331026+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.331026+0000","last_undegraded":"2024-04-25T10:09:42.331026+0000","last_fullsized":"2024-04-25T10:09:42.331026+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T10:09:43.385505+0000","last_change":"2024-04-25T10:09:43.380141+0000","last_active":"2024-04-25T10:09:43.385505+0000","last_peered":"2024-04-25T10:09:43.385505+0000","last_clean":"2024-04-25T10:09:43.385505+0000","last_became_active":"2024-04-25T10:09:43.379538+0000","last_became_peered":"2024-04-25T10:09:43.379538+0000","last_unstale":"2024-04-25T10:09:43.385505+0000","last_undegraded":"2024-04-25T10:09:43.385505+0000","last_fullsized":"2024-04-25T10:09:43.385505+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:39.616882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.344078+0000","last_change":"2024-04-25T10:09:43.344078+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.344078+0000","last_undegraded":"2024-04-25T10:09:43.344078+0000","last_fullsized":"2024-04-25T10:09:43.344078+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.340094+0000","last_change":"2024-04-25T10:09:42.340094+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.340094+0000","last_undegraded":"2024-04-25T10:09:42.340094+0000","last_fullsized":"2024-04-25T10:09:42.340094+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.359180+0000","last_change":"2024-04-25T10:09:43.359180+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.359180+0000","last_undegraded":"2024-04-25T10:09:43.359180+0000","last_fullsized":"2024-04-25T10:09:43.359180+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T10:09:43.380869+0000","last_change":"2024-04-25T10:09:43.373239+0000","last_active":"2024-04-25T10:09:43.380869+0000","last_peered":"2024-04-25T10:09:43.380869+0000","last_clean":"2024-04-25T10:09:43.380869+0000","last_became_active":"2024-04-25T10:09:43.370508+0000","last_became_peered":"2024-04-25T10:09:43.370508+0000","last_unstale":"2024-04-25T10:09:43.380869+0000","last_undegraded":"2024-04-25T10:09:43.380869+0000","last_fullsized":"2024-04-25T10:09:43.380869+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:05.577095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.337374+0000","last_change":"2024-04-25T10:09:42.337374+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.337374+0000","last_undegraded":"2024-04-25T10:09:42.337374+0000","last_fullsized":"2024-04-25T10:09:42.337374+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T10:09:43.381551+0000","last_change":"2024-04-25T10:09:43.377693+0000","last_active":"2024-04-25T10:09:43.381551+0000","last_peered":"2024-04-25T10:09:43.381551+0000","last_clean":"2024-04-25T10:09:43.381551+0000","last_became_active":"2024-04-25T10:09:43.374049+0000","last_became_peered":"2024-04-25T10:09:43.374049+0000","last_unstale":"2024-04-25T10:09:43.381551+0000","last_undegraded":"2024-04-25T10:09:43.381551+0000","last_fullsized":"2024-04-25T10:09:43.381551+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:31.373692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.339447+0000","last_change":"2024-04-25T10:09:42.339447+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.339447+0000","last_undegraded":"2024-04-25T10:09:42.339447+0000","last_fullsized":"2024-04-25T10:09:42.339447+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.328666+0000","last_change":"2024-04-25T10:09:42.328666+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.328666+0000","last_undegraded":"2024-04-25T10:09:42.328666+0000","last_fullsized":"2024-04-25T10:09:42.328666+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.328061+0000","last_change":"2024-04-25T10:09:42.328061+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.328061+0000","last_undegraded":"2024-04-25T10:09:42.328061+0000","last_fullsized":"2024-04-25T10:09:42.328061+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.339139+0000","last_change":"2024-04-25T10:09:42.339139+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.339139+0000","last_undegraded":"2024-04-25T10:09:42.339139+0000","last_fullsized":"2024-04-25T10:09:42.339139+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.328906+0000","last_change":"2024-04-25T10:09:42.328906+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.328906+0000","last_undegraded":"2024-04-25T10:09:42.328906+0000","last_fullsized":"2024-04-25T10:09:42.328906+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.353903+0000","last_change":"2024-04-25T10:09:43.353903+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.353903+0000","last_undegraded":"2024-04-25T10:09:43.353903+0000","last_fullsized":"2024-04-25T10:09:43.353903+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.332298+0000","last_change":"2024-04-25T10:09:42.332298+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.332298+0000","last_undegraded":"2024-04-25T10:09:42.332298+0000","last_fullsized":"2024-04-25T10:09:42.332298+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-04-25T10:09:43.353015+0000","last_change":"2024-04-25T10:09:43.353015+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.353015+0000","last_undegraded":"2024-04-25T10:09:43.353015+0000","last_fullsized":"2024-04-25T10:09:43.353015+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.328261+0000","last_change":"2024-04-25T10:09:42.328261+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.328261+0000","last_undegraded":"2024-04-25T10:09:42.328261+0000","last_fullsized":"2024-04-25T10:09:42.328261+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.331088+0000","last_change":"2024-04-25T10:09:42.331088+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.331088+0000","last_undegraded":"2024-04-25T10:09:42.331088+0000","last_fullsized":"2024-04-25T10:09:42.331088+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.330088+0000","last_change":"2024-04-25T10:09:42.330088+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.330088+0000","last_undegraded":"2024-04-25T10:09:42.330088+0000","last_fullsized":"2024-04-25T10:09:42.330088+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T10:09:43.386212+0000","last_change":"2024-04-25T10:09:43.381091+0000","last_active":"2024-04-25T10:09:43.386212+0000","last_peered":"2024-04-25T10:09:43.386212+0000","last_clean":"2024-04-25T10:09:43.386212+0000","last_became_active":"2024-04-25T10:09:43.379688+0000","last_became_peered":"2024-04-25T10:09:43.379688+0000","last_unstale":"2024-04-25T10:09:43.386212+0000","last_undegraded":"2024-04-25T10:09:43.386212+0000","last_fullsized":"2024-04-25T10:09:43.386212+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:31.673229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.337181+0000","last_change":"2024-04-25T10:09:42.337181+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.337181+0000","last_undegraded":"2024-04-25T10:09:42.337181+0000","last_fullsized":"2024-04-25T10:09:42.337181+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:09:43.324048+0000","last_change":"2024-04-25T10:09:43.324048+0000","last_active":"2024-04-25T10:09:43.324048+0000","last_peered":"2024-04-25T10:09:43.324048+0000","last_clean":"2024-04-25T10:09:43.324048+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:43.324048+0000","last_undegraded":"2024-04-25T10:09:43.324048+0000","last_fullsized":"2024-04-25T10:09:43.324048+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T10:09:42.337232+0000","last_change":"2024-04-25T10:09:42.337232+0000","last_active":"2024-04-25T10:09:42.320222+0000","last_peered":"2024-04-25T10:09:42.320222+0000","last_clean":"2024-04-25T10:09:42.320222+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:09:42.337232+0000","last_undegraded":"2024-04-25T10:09:42.337232+0000","last_fullsized":"2024-04-25T10:09:42.337232+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":252,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T10:09:43.351645+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:09:43.351645+0000","last_peered":"2024-04-25T10:09:43.351645+0000","last_clean":"2024-04-25T10:09:43.351645+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:09:43.351645+0000","last_undegraded":"2024-04-25T10:09:43.351645+0000","last_fullsized":"2024-04-25T10:09:43.351645+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":38,"acting":38,"num_store_stats":2},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509499,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1388544,"data_stored":920119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509498,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":806912,"data_stored":336026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542201,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":782336,"data_stored":323934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542202,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":782336,"data_stored":323934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542202,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36104,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335736,"statfs":{"total":96636764160,"available":96599793664,"internally_reserved":0,"allocated":782336,"data_stored":323934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542202,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":782336,"data_stored":323934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542202,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36108,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335732,"statfs":{"total":96636764160,"available":96599789568,"internally_reserved":0,"allocated":782336,"data_stored":323934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542203,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36688,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335152,"statfs":{"total":96636764160,"available":96599195648,"internally_reserved":0,"allocated":1376256,"data_stored":914302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:09:48.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:09:48.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.581+0000 7ff83c873640 1 Processor -- start 2024-04-25T10:09:48.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83c873640 1 -- start start 2024-04-25T10:09:48.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83c873640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff83815bc90 0x7ff83815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83c873640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8380b4660 0x7ff83815c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83c873640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff838071ca0 con 0x7ff8380ce7a0 2024-04-25T10:09:48.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83c873640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8380cc370 con 0x7ff83815bc90 2024-04-25T10:09:48.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83c873640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8380cca40 con 0x7ff8380b4660 2024-04-25T10:09:48.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83659c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff8380ce7a0 0x7ff8380ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45182/0 (socket says 172.21.15.60:45182) 2024-04-25T10:09:48.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83659c640 1 -- 172.21.15.60:0/328456045 learned_addr learned my addr 172.21.15.60:0/328456045 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:48.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83559a640 1 -- 172.21.15.60:0/328456045 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694269790 0 0) 0x7ff838071ca0 con 0x7ff8380ce7a0 2024-04-25T10:09:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff835d9b640 1 --2- 172.21.15.60:0/328456045 >> v2:172.21.15.100:3300/0 conn(0x7ff83815bc90 0x7ff83815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff836d9d640 1 --2- 172.21.15.60:0/328456045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8380b4660 0x7ff83815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83559a640 1 -- 172.21.15.60:0/328456045 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff820003530 con 0x7ff8380ce7a0 2024-04-25T10:09:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83559a640 1 -- 172.21.15.60:0/328456045 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 432493484 0 0) 0x7ff820003530 con 0x7ff8380ce7a0 2024-04-25T10:09:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83559a640 1 -- 172.21.15.60:0/328456045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8380b4660 msgr2=0x7ff83815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83559a640 1 --2- 172.21.15.60:0/328456045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8380b4660 0x7ff83815c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83559a640 1 -- 172.21.15.60:0/328456045 >> v2:172.21.15.100:3300/0 conn(0x7ff83815bc90 msgr2=0x7ff83815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83559a640 1 --2- 172.21.15.60:0/328456045 >> v2:172.21.15.100:3300/0 conn(0x7ff83815bc90 0x7ff83815c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83559a640 1 -- 172.21.15.60:0/328456045 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8380ccd20 con 0x7ff8380ce7a0 2024-04-25T10:09:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83559a640 1 -- 172.21.15.60:0/328456045 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7ff828001680 con 0x7ff8380ce7a0 2024-04-25T10:09:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83559a640 1 -- 172.21.15.60:0/328456045 >> v1:172.21.15.60:6789/0 conn(0x7ff8380ce7a0 legacy=0x7ff8380ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83559a640 1 --2- 172.21.15.60:0/328456045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8200039c0 0x7ff820003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83559a640 1 --2- 172.21.15.60:0/328456045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8200043b0 0x7ff820004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83559a640 1 --2- 172.21.15.60:0/328456045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff820004fb0 0x7ff82000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83559a640 1 -- 172.21.15.60:0/328456045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8380cca40 con 0x7ff820004fb0 2024-04-25T10:09:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83559a640 1 -- 172.21.15.60:0/328456045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82000bf10 con 0x7ff8200043b0 2024-04-25T10:09:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83559a640 1 -- 172.21.15.60:0/328456045 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82000c0c0 con 0x7ff8200039c0 2024-04-25T10:09:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff836d9d640 1 --2- 172.21.15.60:0/328456045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8380b4660 0x7ff83815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff836d9d640 1 --2- 172.21.15.60:0/328456045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff820004fb0 0x7ff82000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff835d9b640 1 --2- 172.21.15.60:0/328456045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8200043b0 0x7ff820004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff83659c640 1 --2- 172.21.15.60:0/328456045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8200039c0 0x7ff820003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff836d9d640 1 -- 172.21.15.60:0/328456045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8200039c0 msgr2=0x7ff820003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff836d9d640 1 --2- 172.21.15.60:0/328456045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8200039c0 0x7ff820003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff836d9d640 1 -- 172.21.15.60:0/328456045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8200043b0 msgr2=0x7ff820004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff836d9d640 1 --2- 172.21.15.60:0/328456045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8200043b0 0x7ff820004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff836d9d640 1 -- 172.21.15.60:0/328456045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff820003530 con 0x7ff820004fb0 2024-04-25T10:09:48.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.585+0000 7ff835d9b640 1 --2- 172.21.15.60:0/328456045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8200043b0 0x7ff820004890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:48.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.589+0000 7ff836d9d640 1 --2- 172.21.15.60:0/328456045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff820004fb0 0x7ff82000b9d0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ff8380b3ff0 tx=0x7ff83001ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc3980f915b3958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:48.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.589+0000 7ff83559a640 1 -- 172.21.15.60:0/328456045 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff830009b60 con 0x7ff820004fb0 2024-04-25T10:09:48.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.589+0000 7ff83559a640 1 -- 172.21.15.60:0/328456045 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8300a5bc0 con 0x7ff820004fb0 2024-04-25T10:09:48.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.589+0000 7ff83559a640 1 -- 172.21.15.60:0/328456045 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8300039a0 con 0x7ff820004fb0 2024-04-25T10:09:48.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.589+0000 7ff83c873640 1 -- 172.21.15.60:0/328456045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff820004fb0 msgr2=0x7ff82000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.589+0000 7ff83c873640 1 --2- 172.21.15.60:0/328456045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff820004fb0 0x7ff82000b9d0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ff8380b3ff0 tx=0x7ff83001ccb0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.589+0000 7ff836d9d640 1 -- 172.21.15.60:0/328456045 reap_dead start 2024-04-25T10:09:48.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.589+0000 7ff83c873640 1 -- 172.21.15.60:0/328456045 shutdown_connections 2024-04-25T10:09:48.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.589+0000 7ff83c873640 1 -- 172.21.15.60:0/328456045 >> 172.21.15.60:0/328456045 conn(0x7ff8380b12d0 msgr2=0x7ff8380b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:48.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.589+0000 7ff83c873640 1 -- 172.21.15.60:0/328456045 shutdown_connections 2024-04-25T10:09:48.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.589+0000 7ff83c873640 1 -- 172.21.15.60:0/328456045 wait complete. 2024-04-25T10:09:48.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.589+0000 7ff83c873640 1 Processor -- start 2024-04-25T10:09:48.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.589+0000 7ff83c873640 1 -- start start 2024-04-25T10:09:48.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83c873640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff820004fb0 0x7ff838157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83c873640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff838155000 0x7ff838155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83c873640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8380b1ea0 0x7ff8380b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83c873640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8380b3d20 con 0x7ff820004fb0 2024-04-25T10:09:48.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83c873640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8380b3ba0 con 0x7ff8380b1ea0 2024-04-25T10:09:48.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83c873640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8380b4280 con 0x7ff838155000 2024-04-25T10:09:48.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff820004fb0 0x7ff838157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff820004fb0 0x7ff838157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45096/0 (socket says 172.21.15.60:45096) 2024-04-25T10:09:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83659c640 1 -- 172.21.15.60:0/280583393 learned_addr learned my addr 172.21.15.60:0/280583393 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff836d9d640 1 --2- 172.21.15.60:0/280583393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8380b1ea0 0x7ff8380b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff835d9b640 1 --2- 172.21.15.60:0/280583393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff838155000 0x7ff838155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83659c640 1 -- 172.21.15.60:0/280583393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff838155000 msgr2=0x7ff838155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83659c640 1 --2- 172.21.15.60:0/280583393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff838155000 0x7ff838155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83659c640 1 -- 172.21.15.60:0/280583393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8380b1ea0 msgr2=0x7ff8380b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83659c640 1 --2- 172.21.15.60:0/280583393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8380b1ea0 0x7ff8380b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83659c640 1 -- 172.21.15.60:0/280583393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83815a1d0 con 0x7ff820004fb0 2024-04-25T10:09:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff836d9d640 1 --2- 172.21.15.60:0/280583393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8380b1ea0 0x7ff8380b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83659c640 1 --2- 172.21.15.60:0/280583393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff820004fb0 0x7ff838157de0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff82800e7e0 tx=0x7ff82800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff826ffd640 1 -- 172.21.15.60:0/280583393 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82800ee90 con 0x7ff820004fb0 2024-04-25T10:09:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff826ffd640 1 -- 172.21.15.60:0/280583393 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff828018070 con 0x7ff820004fb0 2024-04-25T10:09:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83c873640 1 -- 172.21.15.60:0/280583393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff838155500 con 0x7ff820004fb0 2024-04-25T10:09:48.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff826ffd640 1 -- 172.21.15.60:0/280583393 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82800cb10 con 0x7ff820004fb0 2024-04-25T10:09:48.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83c873640 1 -- 172.21.15.60:0/280583393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8381f1630 con 0x7ff820004fb0 2024-04-25T10:09:48.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff83c873640 1 -- 172.21.15.60:0/280583393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff838074770 con 0x7ff820004fb0 2024-04-25T10:09:48.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff826ffd640 1 -- 172.21.15.60:0/280583393 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff82800ccb0 con 0x7ff820004fb0 2024-04-25T10:09:48.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff826ffd640 1 --2- 172.21.15.60:0/280583393 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff818067130 0x7ff8180695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff826ffd640 1 -- 172.21.15.60:0/280583393 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff828084af0 con 0x7ff820004fb0 2024-04-25T10:09:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.593+0000 7ff835d9b640 1 --2- 172.21.15.60:0/280583393 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff818067130 0x7ff8180695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.597+0000 7ff835d9b640 1 --2- 172.21.15.60:0/280583393 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff818067130 0x7ff8180695f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff8380b11c0 tx=0x7ff82c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:48.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.597+0000 7ff826ffd640 1 -- 172.21.15.60:0/280583393 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff83815a1d0 con 0x7ff820004fb0 2024-04-25T10:09:48.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.757+0000 7ff83c873640 1 -- 172.21.15.60:0/280583393 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff838158320 con 0x7ff818067130 2024-04-25T10:09:48.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.773+0000 7ff826ffd640 1 -- 172.21.15.60:0/280583393 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252077 (secure 0 0 0) 0x7ff838158320 con 0x7ff818067130 2024-04-25T10:09:48.785 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:48.787 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:09:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.777+0000 7ff83c873640 1 -- 172.21.15.60:0/280583393 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff818067130 msgr2=0x7ff8180695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.777+0000 7ff83c873640 1 --2- 172.21.15.60:0/280583393 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff818067130 0x7ff8180695f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff8380b11c0 tx=0x7ff82c008040 comp rx=0 tx=0).stop 2024-04-25T10:09:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.777+0000 7ff83c873640 1 -- 172.21.15.60:0/280583393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff820004fb0 msgr2=0x7ff838157de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.777+0000 7ff83c873640 1 --2- 172.21.15.60:0/280583393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff820004fb0 0x7ff838157de0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff82800e7e0 tx=0x7ff82800eca0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.777+0000 7ff83c873640 1 -- 172.21.15.60:0/280583393 shutdown_connections 2024-04-25T10:09:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.777+0000 7ff83c873640 1 --2- 172.21.15.60:0/280583393 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff818067130 0x7ff8180695f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.777+0000 7ff83c873640 1 --2- 172.21.15.60:0/280583393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8380b1ea0 0x7ff8380b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.777+0000 7ff83c873640 1 --2- 172.21.15.60:0/280583393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff838155000 0x7ff838155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.777+0000 7ff83c873640 1 --2- 172.21.15.60:0/280583393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff820004fb0 0x7ff838157de0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.777+0000 7ff83c873640 1 -- 172.21.15.60:0/280583393 >> 172.21.15.60:0/280583393 conn(0x7ff8380b12d0 msgr2=0x7ff83815eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:48.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.777+0000 7ff83c873640 1 -- 172.21.15.60:0/280583393 shutdown_connections 2024-04-25T10:09:48.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.777+0000 7ff83c873640 1 -- 172.21.15.60:0/280583393 wait complete. 2024-04-25T10:09:48.804 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-04-25T10:09:48.480304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265512,"kb_used_data":7720,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754709208,"statfs":{"total":773094113280,"available":772822228992,"internally_reserved":0,"allocated":7905280,"data_stored":3978364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001824"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.443343+0000","last_change":"2024-04-25T10:09:44.407952+0000","last_active":"2024-04-25T10:09:45.443343+0000","last_peered":"2024-04-25T10:09:45.443343+0000","last_clean":"2024-04-25T10:09:45.443343+0000","last_became_active":"2024-04-25T10:09:44.407206+0000","last_became_peered":"2024-04-25T10:09:44.407206+0000","last_unstale":"2024-04-25T10:09:45.443343+0000","last_undegraded":"2024-04-25T10:09:45.443343+0000","last_fullsized":"2024-04-25T10:09:45.443343+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:14.060504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.374158+0000","last_change":"2024-04-25T10:09:44.373797+0000","last_active":"2024-04-25T10:09:45.374158+0000","last_peered":"2024-04-25T10:09:45.374158+0000","last_clean":"2024-04-25T10:09:45.374158+0000","last_became_active":"2024-04-25T10:09:44.372821+0000","last_became_peered":"2024-04-25T10:09:44.372821+0000","last_unstale":"2024-04-25T10:09:45.374158+0000","last_undegraded":"2024-04-25T10:09:45.374158+0000","last_fullsized":"2024-04-25T10:09:45.374158+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:45.968823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.442581+0000","last_change":"2024-04-25T10:09:44.369592+0000","last_active":"2024-04-25T10:09:45.442581+0000","last_peered":"2024-04-25T10:09:45.442581+0000","last_clean":"2024-04-25T10:09:45.442581+0000","last_became_active":"2024-04-25T10:09:44.369279+0000","last_became_peered":"2024-04-25T10:09:44.369279+0000","last_unstale":"2024-04-25T10:09:45.442581+0000","last_undegraded":"2024-04-25T10:09:45.442581+0000","last_fullsized":"2024-04-25T10:09:45.442581+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:08.029612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.442527+0000","last_change":"2024-04-25T10:09:44.361868+0000","last_active":"2024-04-25T10:09:45.442527+0000","last_peered":"2024-04-25T10:09:45.442527+0000","last_clean":"2024-04-25T10:09:45.442527+0000","last_became_active":"2024-04-25T10:09:44.361567+0000","last_became_peered":"2024-04-25T10:09:44.361567+0000","last_unstale":"2024-04-25T10:09:45.442527+0000","last_undegraded":"2024-04-25T10:09:45.442527+0000","last_fullsized":"2024-04-25T10:09:45.442527+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:50.540389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.365816+0000","last_change":"2024-04-25T10:09:44.394751+0000","last_active":"2024-04-25T10:09:45.365816+0000","last_peered":"2024-04-25T10:09:45.365816+0000","last_clean":"2024-04-25T10:09:45.365816+0000","last_became_active":"2024-04-25T10:09:44.394422+0000","last_became_peered":"2024-04-25T10:09:44.394422+0000","last_unstale":"2024-04-25T10:09:45.365816+0000","last_undegraded":"2024-04-25T10:09:45.365816+0000","last_fullsized":"2024-04-25T10:09:45.365816+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:04.642073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.370938+0000","last_change":"2024-04-25T10:09:44.375240+0000","last_active":"2024-04-25T10:09:45.370938+0000","last_peered":"2024-04-25T10:09:45.370938+0000","last_clean":"2024-04-25T10:09:45.370938+0000","last_became_active":"2024-04-25T10:09:44.373824+0000","last_became_peered":"2024-04-25T10:09:44.373824+0000","last_unstale":"2024-04-25T10:09:45.370938+0000","last_undegraded":"2024-04-25T10:09:45.370938+0000","last_fullsized":"2024-04-25T10:09:45.370938+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:35.627984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.441538+0000","last_change":"2024-04-25T10:09:44.368971+0000","last_active":"2024-04-25T10:09:45.441538+0000","last_peered":"2024-04-25T10:09:45.441538+0000","last_clean":"2024-04-25T10:09:45.441538+0000","last_became_active":"2024-04-25T10:09:44.368186+0000","last_became_peered":"2024-04-25T10:09:44.368186+0000","last_unstale":"2024-04-25T10:09:45.441538+0000","last_undegraded":"2024-04-25T10:09:45.441538+0000","last_fullsized":"2024-04-25T10:09:45.441538+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:54.356078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.370863+0000","last_change":"2024-04-25T10:09:44.401855+0000","last_active":"2024-04-25T10:09:45.370863+0000","last_peered":"2024-04-25T10:09:45.370863+0000","last_clean":"2024-04-25T10:09:45.370863+0000","last_became_active":"2024-04-25T10:09:44.397958+0000","last_became_peered":"2024-04-25T10:09:44.397958+0000","last_unstale":"2024-04-25T10:09:45.370863+0000","last_undegraded":"2024-04-25T10:09:45.370863+0000","last_fullsized":"2024-04-25T10:09:45.370863+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:45.366576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.366801+0000","last_change":"2024-04-25T10:09:44.375095+0000","last_active":"2024-04-25T10:09:45.366801+0000","last_peered":"2024-04-25T10:09:45.366801+0000","last_clean":"2024-04-25T10:09:45.366801+0000","last_became_active":"2024-04-25T10:09:44.374636+0000","last_became_peered":"2024-04-25T10:09:44.374636+0000","last_unstale":"2024-04-25T10:09:45.366801+0000","last_undegraded":"2024-04-25T10:09:45.366801+0000","last_fullsized":"2024-04-25T10:09:45.366801+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:08.755635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.372400+0000","last_change":"2024-04-25T10:09:44.361360+0000","last_active":"2024-04-25T10:09:45.372400+0000","last_peered":"2024-04-25T10:09:45.372400+0000","last_clean":"2024-04-25T10:09:45.372400+0000","last_became_active":"2024-04-25T10:09:44.360666+0000","last_became_peered":"2024-04-25T10:09:44.360666+0000","last_unstale":"2024-04-25T10:09:45.372400+0000","last_undegraded":"2024-04-25T10:09:45.372400+0000","last_fullsized":"2024-04-25T10:09:45.372400+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:44.009504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.442558+0000","last_change":"2024-04-25T10:09:44.390788+0000","last_active":"2024-04-25T10:09:45.442558+0000","last_peered":"2024-04-25T10:09:45.442558+0000","last_clean":"2024-04-25T10:09:45.442558+0000","last_became_active":"2024-04-25T10:09:44.389946+0000","last_became_peered":"2024-04-25T10:09:44.389946+0000","last_unstale":"2024-04-25T10:09:45.442558+0000","last_undegraded":"2024-04-25T10:09:45.442558+0000","last_fullsized":"2024-04-25T10:09:45.442558+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:03.906035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.442720+0000","last_change":"2024-04-25T10:09:44.373065+0000","last_active":"2024-04-25T10:09:45.442720+0000","last_peered":"2024-04-25T10:09:45.442720+0000","last_clean":"2024-04-25T10:09:45.442720+0000","last_became_active":"2024-04-25T10:09:44.372679+0000","last_became_peered":"2024-04-25T10:09:44.372679+0000","last_unstale":"2024-04-25T10:09:45.442720+0000","last_undegraded":"2024-04-25T10:09:45.442720+0000","last_fullsized":"2024-04-25T10:09:45.442720+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:04.329024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.424919+0000","last_change":"2024-04-25T10:09:44.375264+0000","last_active":"2024-04-25T10:09:45.424919+0000","last_peered":"2024-04-25T10:09:45.424919+0000","last_clean":"2024-04-25T10:09:45.424919+0000","last_became_active":"2024-04-25T10:09:44.374969+0000","last_became_peered":"2024-04-25T10:09:44.374969+0000","last_unstale":"2024-04-25T10:09:45.424919+0000","last_undegraded":"2024-04-25T10:09:45.424919+0000","last_fullsized":"2024-04-25T10:09:45.424919+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:10.969262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.366016+0000","last_change":"2024-04-25T10:09:44.374875+0000","last_active":"2024-04-25T10:09:45.366016+0000","last_peered":"2024-04-25T10:09:45.366016+0000","last_clean":"2024-04-25T10:09:45.366016+0000","last_became_active":"2024-04-25T10:09:44.374305+0000","last_became_peered":"2024-04-25T10:09:44.374305+0000","last_unstale":"2024-04-25T10:09:45.366016+0000","last_undegraded":"2024-04-25T10:09:45.366016+0000","last_fullsized":"2024-04-25T10:09:45.366016+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:01.107502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.424756+0000","last_change":"2024-04-25T10:09:44.374948+0000","last_active":"2024-04-25T10:09:45.424756+0000","last_peered":"2024-04-25T10:09:45.424756+0000","last_clean":"2024-04-25T10:09:45.424756+0000","last_became_active":"2024-04-25T10:09:44.374652+0000","last_became_peered":"2024-04-25T10:09:44.374652+0000","last_unstale":"2024-04-25T10:09:45.424756+0000","last_undegraded":"2024-04-25T10:09:45.424756+0000","last_fullsized":"2024-04-25T10:09:45.424756+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:13.093130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.371379+0000","last_change":"2024-04-25T10:09:44.403441+0000","last_active":"2024-04-25T10:09:45.371379+0000","last_peered":"2024-04-25T10:09:45.371379+0000","last_clean":"2024-04-25T10:09:45.371379+0000","last_became_active":"2024-04-25T10:09:44.402805+0000","last_became_peered":"2024-04-25T10:09:44.402805+0000","last_unstale":"2024-04-25T10:09:45.371379+0000","last_undegraded":"2024-04-25T10:09:45.371379+0000","last_fullsized":"2024-04-25T10:09:45.371379+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:06.499376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.366595+0000","last_change":"2024-04-25T10:09:44.374901+0000","last_active":"2024-04-25T10:09:45.366595+0000","last_peered":"2024-04-25T10:09:45.366595+0000","last_clean":"2024-04-25T10:09:45.366595+0000","last_became_active":"2024-04-25T10:09:44.374363+0000","last_became_peered":"2024-04-25T10:09:44.374363+0000","last_unstale":"2024-04-25T10:09:45.366595+0000","last_undegraded":"2024-04-25T10:09:45.366595+0000","last_fullsized":"2024-04-25T10:09:45.366595+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:17.947734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.442220+0000","last_change":"2024-04-25T10:09:44.360037+0000","last_active":"2024-04-25T10:09:45.442220+0000","last_peered":"2024-04-25T10:09:45.442220+0000","last_clean":"2024-04-25T10:09:45.442220+0000","last_became_active":"2024-04-25T10:09:44.359711+0000","last_became_peered":"2024-04-25T10:09:44.359711+0000","last_unstale":"2024-04-25T10:09:45.442220+0000","last_undegraded":"2024-04-25T10:09:45.442220+0000","last_fullsized":"2024-04-25T10:09:45.442220+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:55.109759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.373294+0000","last_change":"2024-04-25T10:09:44.376011+0000","last_active":"2024-04-25T10:09:45.373294+0000","last_peered":"2024-04-25T10:09:45.373294+0000","last_clean":"2024-04-25T10:09:45.373294+0000","last_became_active":"2024-04-25T10:09:44.375301+0000","last_became_peered":"2024-04-25T10:09:44.375301+0000","last_unstale":"2024-04-25T10:09:45.373294+0000","last_undegraded":"2024-04-25T10:09:45.373294+0000","last_fullsized":"2024-04-25T10:09:45.373294+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:20.843580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.424981+0000","last_change":"2024-04-25T10:09:44.369414+0000","last_active":"2024-04-25T10:09:45.424981+0000","last_peered":"2024-04-25T10:09:45.424981+0000","last_clean":"2024-04-25T10:09:45.424981+0000","last_became_active":"2024-04-25T10:09:44.369119+0000","last_became_peered":"2024-04-25T10:09:44.369119+0000","last_unstale":"2024-04-25T10:09:45.424981+0000","last_undegraded":"2024-04-25T10:09:45.424981+0000","last_fullsized":"2024-04-25T10:09:45.424981+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:56.556523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.372326+0000","last_change":"2024-04-25T10:09:44.389541+0000","last_active":"2024-04-25T10:09:45.372326+0000","last_peered":"2024-04-25T10:09:45.372326+0000","last_clean":"2024-04-25T10:09:45.372326+0000","last_became_active":"2024-04-25T10:09:44.383177+0000","last_became_peered":"2024-04-25T10:09:44.383177+0000","last_unstale":"2024-04-25T10:09:45.372326+0000","last_undegraded":"2024-04-25T10:09:45.372326+0000","last_fullsized":"2024-04-25T10:09:45.372326+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:08.074655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.371434+0000","last_change":"2024-04-25T10:09:44.390035+0000","last_active":"2024-04-25T10:09:45.371434+0000","last_peered":"2024-04-25T10:09:45.371434+0000","last_clean":"2024-04-25T10:09:45.371434+0000","last_became_active":"2024-04-25T10:09:44.383381+0000","last_became_peered":"2024-04-25T10:09:44.383381+0000","last_unstale":"2024-04-25T10:09:45.371434+0000","last_undegraded":"2024-04-25T10:09:45.371434+0000","last_fullsized":"2024-04-25T10:09:45.371434+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:32.377518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.442324+0000","last_change":"2024-04-25T10:09:44.395935+0000","last_active":"2024-04-25T10:09:45.442324+0000","last_peered":"2024-04-25T10:09:45.442324+0000","last_clean":"2024-04-25T10:09:45.442324+0000","last_became_active":"2024-04-25T10:09:44.395197+0000","last_became_peered":"2024-04-25T10:09:44.395197+0000","last_unstale":"2024-04-25T10:09:45.442324+0000","last_undegraded":"2024-04-25T10:09:45.442324+0000","last_fullsized":"2024-04-25T10:09:45.442324+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:10.884976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.441623+0000","last_change":"2024-04-25T10:09:44.411774+0000","last_active":"2024-04-25T10:09:45.441623+0000","last_peered":"2024-04-25T10:09:45.441623+0000","last_clean":"2024-04-25T10:09:45.441623+0000","last_became_active":"2024-04-25T10:09:44.411248+0000","last_became_peered":"2024-04-25T10:09:44.411248+0000","last_unstale":"2024-04-25T10:09:45.441623+0000","last_undegraded":"2024-04-25T10:09:45.441623+0000","last_fullsized":"2024-04-25T10:09:45.441623+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:42.278860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.372386+0000","last_change":"2024-04-25T10:09:44.408470+0000","last_active":"2024-04-25T10:09:45.372386+0000","last_peered":"2024-04-25T10:09:45.372386+0000","last_clean":"2024-04-25T10:09:45.372386+0000","last_became_active":"2024-04-25T10:09:44.407766+0000","last_became_peered":"2024-04-25T10:09:44.407766+0000","last_unstale":"2024-04-25T10:09:45.372386+0000","last_undegraded":"2024-04-25T10:09:45.372386+0000","last_fullsized":"2024-04-25T10:09:45.372386+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:35.957461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.443671+0000","last_change":"2024-04-25T10:09:44.375741+0000","last_active":"2024-04-25T10:09:45.443671+0000","last_peered":"2024-04-25T10:09:45.443671+0000","last_clean":"2024-04-25T10:09:45.443671+0000","last_became_active":"2024-04-25T10:09:44.372093+0000","last_became_peered":"2024-04-25T10:09:44.372093+0000","last_unstale":"2024-04-25T10:09:45.443671+0000","last_undegraded":"2024-04-25T10:09:45.443671+0000","last_fullsized":"2024-04-25T10:09:45.443671+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:51.528145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.441922+0000","last_change":"2024-04-25T10:09:44.387485+0000","last_active":"2024-04-25T10:09:45.441922+0000","last_peered":"2024-04-25T10:09:45.441922+0000","last_clean":"2024-04-25T10:09:45.441922+0000","last_became_active":"2024-04-25T10:09:44.387132+0000","last_became_peered":"2024-04-25T10:09:44.387132+0000","last_unstale":"2024-04-25T10:09:45.441922+0000","last_undegraded":"2024-04-25T10:09:45.441922+0000","last_fullsized":"2024-04-25T10:09:45.441922+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:12.867293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.372016+0000","last_change":"2024-04-25T10:09:44.390545+0000","last_active":"2024-04-25T10:09:45.372016+0000","last_peered":"2024-04-25T10:09:45.372016+0000","last_clean":"2024-04-25T10:09:45.372016+0000","last_became_active":"2024-04-25T10:09:44.389820+0000","last_became_peered":"2024-04-25T10:09:44.389820+0000","last_unstale":"2024-04-25T10:09:45.372016+0000","last_undegraded":"2024-04-25T10:09:45.372016+0000","last_fullsized":"2024-04-25T10:09:45.372016+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:21.702334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.441645+0000","last_change":"2024-04-25T10:09:44.365203+0000","last_active":"2024-04-25T10:09:45.441645+0000","last_peered":"2024-04-25T10:09:45.441645+0000","last_clean":"2024-04-25T10:09:45.441645+0000","last_became_active":"2024-04-25T10:09:44.364935+0000","last_became_peered":"2024-04-25T10:09:44.364935+0000","last_unstale":"2024-04-25T10:09:45.441645+0000","last_undegraded":"2024-04-25T10:09:45.441645+0000","last_fullsized":"2024-04-25T10:09:45.441645+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:18.981269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.441661+0000","last_change":"2024-04-25T10:09:44.372880+0000","last_active":"2024-04-25T10:09:45.441661+0000","last_peered":"2024-04-25T10:09:45.441661+0000","last_clean":"2024-04-25T10:09:45.441661+0000","last_became_active":"2024-04-25T10:09:44.372125+0000","last_became_peered":"2024-04-25T10:09:44.372125+0000","last_unstale":"2024-04-25T10:09:45.441661+0000","last_undegraded":"2024-04-25T10:09:45.441661+0000","last_fullsized":"2024-04-25T10:09:45.441661+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:35.246161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.441954+0000","last_change":"2024-04-25T10:09:44.377571+0000","last_active":"2024-04-25T10:09:45.441954+0000","last_peered":"2024-04-25T10:09:45.441954+0000","last_clean":"2024-04-25T10:09:45.441954+0000","last_became_active":"2024-04-25T10:09:44.376800+0000","last_became_peered":"2024-04-25T10:09:44.376800+0000","last_unstale":"2024-04-25T10:09:45.441954+0000","last_undegraded":"2024-04-25T10:09:45.441954+0000","last_fullsized":"2024-04-25T10:09:45.441954+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:28.964611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.371223+0000","last_change":"2024-04-25T10:09:44.364006+0000","last_active":"2024-04-25T10:09:45.371223+0000","last_peered":"2024-04-25T10:09:45.371223+0000","last_clean":"2024-04-25T10:09:45.371223+0000","last_became_active":"2024-04-25T10:09:44.363218+0000","last_became_peered":"2024-04-25T10:09:44.363218+0000","last_unstale":"2024-04-25T10:09:45.371223+0000","last_undegraded":"2024-04-25T10:09:45.371223+0000","last_fullsized":"2024-04-25T10:09:45.371223+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:20.516177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.443300+0000","last_change":"2024-04-25T10:09:44.411671+0000","last_active":"2024-04-25T10:09:45.443300+0000","last_peered":"2024-04-25T10:09:45.443300+0000","last_clean":"2024-04-25T10:09:45.443300+0000","last_became_active":"2024-04-25T10:09:44.411114+0000","last_became_peered":"2024-04-25T10:09:44.411114+0000","last_unstale":"2024-04-25T10:09:45.443300+0000","last_undegraded":"2024-04-25T10:09:45.443300+0000","last_fullsized":"2024-04-25T10:09:45.443300+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:11.122377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.443984+0000","last_change":"2024-04-25T10:09:43.379590+0000","last_active":"2024-04-25T10:09:45.443984+0000","last_peered":"2024-04-25T10:09:45.443984+0000","last_clean":"2024-04-25T10:09:45.443984+0000","last_became_active":"2024-04-25T10:09:43.378739+0000","last_became_peered":"2024-04-25T10:09:43.378739+0000","last_unstale":"2024-04-25T10:09:45.443984+0000","last_undegraded":"2024-04-25T10:09:45.443984+0000","last_fullsized":"2024-04-25T10:09:45.443984+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:34.953635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.366687+0000","last_change":"2024-04-25T10:09:44.376422+0000","last_active":"2024-04-25T10:09:45.366687+0000","last_peered":"2024-04-25T10:09:45.366687+0000","last_clean":"2024-04-25T10:09:45.366687+0000","last_became_active":"2024-04-25T10:09:44.374192+0000","last_became_peered":"2024-04-25T10:09:44.374192+0000","last_unstale":"2024-04-25T10:09:45.366687+0000","last_undegraded":"2024-04-25T10:09:45.366687+0000","last_fullsized":"2024-04-25T10:09:45.366687+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:51.308479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.425215+0000","last_change":"2024-04-25T10:09:43.355374+0000","last_active":"2024-04-25T10:09:45.425215+0000","last_peered":"2024-04-25T10:09:45.425215+0000","last_clean":"2024-04-25T10:09:45.425215+0000","last_became_active":"2024-04-25T10:09:43.355099+0000","last_became_peered":"2024-04-25T10:09:43.355099+0000","last_unstale":"2024-04-25T10:09:45.425215+0000","last_undegraded":"2024-04-25T10:09:45.425215+0000","last_fullsized":"2024-04-25T10:09:45.425215+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:37.482540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.373719+0000","last_change":"2024-04-25T10:09:44.403692+0000","last_active":"2024-04-25T10:09:45.373719+0000","last_peered":"2024-04-25T10:09:45.373719+0000","last_clean":"2024-04-25T10:09:45.373719+0000","last_became_active":"2024-04-25T10:09:44.402862+0000","last_became_peered":"2024-04-25T10:09:44.402862+0000","last_unstale":"2024-04-25T10:09:45.373719+0000","last_undegraded":"2024-04-25T10:09:45.373719+0000","last_fullsized":"2024-04-25T10:09:45.373719+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:32.379697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.372180+0000","last_change":"2024-04-25T10:09:43.372515+0000","last_active":"2024-04-25T10:09:45.372180+0000","last_peered":"2024-04-25T10:09:45.372180+0000","last_clean":"2024-04-25T10:09:45.372180+0000","last_became_active":"2024-04-25T10:09:43.371360+0000","last_became_peered":"2024-04-25T10:09:43.371360+0000","last_unstale":"2024-04-25T10:09:45.372180+0000","last_undegraded":"2024-04-25T10:09:45.372180+0000","last_fullsized":"2024-04-25T10:09:45.372180+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:49.536564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.443144+0000","last_change":"2024-04-25T10:09:44.395561+0000","last_active":"2024-04-25T10:09:45.443144+0000","last_peered":"2024-04-25T10:09:45.443144+0000","last_clean":"2024-04-25T10:09:45.443144+0000","last_became_active":"2024-04-25T10:09:44.394888+0000","last_became_peered":"2024-04-25T10:09:44.394888+0000","last_unstale":"2024-04-25T10:09:45.443144+0000","last_undegraded":"2024-04-25T10:09:45.443144+0000","last_fullsized":"2024-04-25T10:09:45.443144+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:12.769846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.373859+0000","last_change":"2024-04-25T10:09:43.372300+0000","last_active":"2024-04-25T10:09:45.373859+0000","last_peered":"2024-04-25T10:09:45.373859+0000","last_clean":"2024-04-25T10:09:45.373859+0000","last_became_active":"2024-04-25T10:09:43.371600+0000","last_became_peered":"2024-04-25T10:09:43.371600+0000","last_unstale":"2024-04-25T10:09:45.373859+0000","last_undegraded":"2024-04-25T10:09:45.373859+0000","last_fullsized":"2024-04-25T10:09:45.373859+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:54.434325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.441956+0000","last_change":"2024-04-25T10:09:44.365910+0000","last_active":"2024-04-25T10:09:45.441956+0000","last_peered":"2024-04-25T10:09:45.441956+0000","last_clean":"2024-04-25T10:09:45.441956+0000","last_became_active":"2024-04-25T10:09:44.364932+0000","last_became_peered":"2024-04-25T10:09:44.364932+0000","last_unstale":"2024-04-25T10:09:45.441956+0000","last_undegraded":"2024-04-25T10:09:45.441956+0000","last_fullsized":"2024-04-25T10:09:45.441956+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:50.947742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.442183+0000","last_change":"2024-04-25T10:09:43.366172+0000","last_active":"2024-04-25T10:09:45.442183+0000","last_peered":"2024-04-25T10:09:45.442183+0000","last_clean":"2024-04-25T10:09:45.442183+0000","last_became_active":"2024-04-25T10:09:43.365760+0000","last_became_peered":"2024-04-25T10:09:43.365760+0000","last_unstale":"2024-04-25T10:09:45.442183+0000","last_undegraded":"2024-04-25T10:09:45.442183+0000","last_fullsized":"2024-04-25T10:09:45.442183+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:33.588477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.372744+0000","last_change":"2024-04-25T10:09:44.395771+0000","last_active":"2024-04-25T10:09:45.372744+0000","last_peered":"2024-04-25T10:09:45.372744+0000","last_clean":"2024-04-25T10:09:45.372744+0000","last_became_active":"2024-04-25T10:09:44.395124+0000","last_became_peered":"2024-04-25T10:09:44.395124+0000","last_unstale":"2024-04-25T10:09:45.372744+0000","last_undegraded":"2024-04-25T10:09:45.372744+0000","last_fullsized":"2024-04-25T10:09:45.372744+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:25.429055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.371068+0000","last_change":"2024-04-25T10:09:43.386826+0000","last_active":"2024-04-25T10:09:45.371068+0000","last_peered":"2024-04-25T10:09:45.371068+0000","last_clean":"2024-04-25T10:09:45.371068+0000","last_became_active":"2024-04-25T10:09:43.386180+0000","last_became_peered":"2024-04-25T10:09:43.386180+0000","last_unstale":"2024-04-25T10:09:45.371068+0000","last_undegraded":"2024-04-25T10:09:45.371068+0000","last_fullsized":"2024-04-25T10:09:45.371068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:46.075831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.365737+0000","last_change":"2024-04-25T10:09:44.362464+0000","last_active":"2024-04-25T10:09:45.365737+0000","last_peered":"2024-04-25T10:09:45.365737+0000","last_clean":"2024-04-25T10:09:45.365737+0000","last_became_active":"2024-04-25T10:09:44.362056+0000","last_became_peered":"2024-04-25T10:09:44.362056+0000","last_unstale":"2024-04-25T10:09:45.365737+0000","last_undegraded":"2024-04-25T10:09:45.365737+0000","last_fullsized":"2024-04-25T10:09:45.365737+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:19.280160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.440536+0000","last_change":"2024-04-25T10:09:43.359301+0000","last_active":"2024-04-25T10:09:45.440536+0000","last_peered":"2024-04-25T10:09:45.440536+0000","last_clean":"2024-04-25T10:09:45.440536+0000","last_became_active":"2024-04-25T10:09:43.358892+0000","last_became_peered":"2024-04-25T10:09:43.358892+0000","last_unstale":"2024-04-25T10:09:45.440536+0000","last_undegraded":"2024-04-25T10:09:45.440536+0000","last_fullsized":"2024-04-25T10:09:45.440536+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:32.575055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.424766+0000","last_change":"2024-04-25T10:09:44.388707+0000","last_active":"2024-04-25T10:09:45.424766+0000","last_peered":"2024-04-25T10:09:45.424766+0000","last_clean":"2024-04-25T10:09:45.424766+0000","last_became_active":"2024-04-25T10:09:44.388398+0000","last_became_peered":"2024-04-25T10:09:44.388398+0000","last_unstale":"2024-04-25T10:09:45.424766+0000","last_undegraded":"2024-04-25T10:09:45.424766+0000","last_fullsized":"2024-04-25T10:09:45.424766+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:51.528145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.365772+0000","last_change":"2024-04-25T10:09:43.348894+0000","last_active":"2024-04-25T10:09:45.365772+0000","last_peered":"2024-04-25T10:09:45.365772+0000","last_clean":"2024-04-25T10:09:45.365772+0000","last_became_active":"2024-04-25T10:09:43.348486+0000","last_became_peered":"2024-04-25T10:09:43.348486+0000","last_unstale":"2024-04-25T10:09:45.365772+0000","last_undegraded":"2024-04-25T10:09:45.365772+0000","last_fullsized":"2024-04-25T10:09:45.365772+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:53.396051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.443867+0000","last_change":"2024-04-25T10:09:44.393074+0000","last_active":"2024-04-25T10:09:45.443867+0000","last_peered":"2024-04-25T10:09:45.443867+0000","last_clean":"2024-04-25T10:09:45.443867+0000","last_became_active":"2024-04-25T10:09:44.392146+0000","last_became_peered":"2024-04-25T10:09:44.392146+0000","last_unstale":"2024-04-25T10:09:45.443867+0000","last_undegraded":"2024-04-25T10:09:45.443867+0000","last_fullsized":"2024-04-25T10:09:45.443867+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:10.969262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.372493+0000","last_change":"2024-04-25T10:09:43.365252+0000","last_active":"2024-04-25T10:09:45.372493+0000","last_peered":"2024-04-25T10:09:45.372493+0000","last_clean":"2024-04-25T10:09:45.372493+0000","last_became_active":"2024-04-25T10:09:43.363864+0000","last_became_peered":"2024-04-25T10:09:43.363864+0000","last_unstale":"2024-04-25T10:09:45.372493+0000","last_undegraded":"2024-04-25T10:09:45.372493+0000","last_fullsized":"2024-04-25T10:09:45.372493+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:54.105934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.443747+0000","last_change":"2024-04-25T10:09:44.411925+0000","last_active":"2024-04-25T10:09:45.443747+0000","last_peered":"2024-04-25T10:09:45.443747+0000","last_clean":"2024-04-25T10:09:45.443747+0000","last_became_active":"2024-04-25T10:09:44.411223+0000","last_became_peered":"2024-04-25T10:09:44.411223+0000","last_unstale":"2024-04-25T10:09:45.443747+0000","last_undegraded":"2024-04-25T10:09:45.443747+0000","last_fullsized":"2024-04-25T10:09:45.443747+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:02.926036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.374207+0000","last_change":"2024-04-25T10:09:43.380083+0000","last_active":"2024-04-25T10:09:45.374207+0000","last_peered":"2024-04-25T10:09:45.374207+0000","last_clean":"2024-04-25T10:09:45.374207+0000","last_became_active":"2024-04-25T10:09:43.379393+0000","last_became_peered":"2024-04-25T10:09:43.379393+0000","last_unstale":"2024-04-25T10:09:45.374207+0000","last_undegraded":"2024-04-25T10:09:45.374207+0000","last_fullsized":"2024-04-25T10:09:45.374207+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:41.877773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.373811+0000","last_change":"2024-04-25T10:09:44.373998+0000","last_active":"2024-04-25T10:09:45.373811+0000","last_peered":"2024-04-25T10:09:45.373811+0000","last_clean":"2024-04-25T10:09:45.373811+0000","last_became_active":"2024-04-25T10:09:44.373110+0000","last_became_peered":"2024-04-25T10:09:44.373110+0000","last_unstale":"2024-04-25T10:09:45.373811+0000","last_undegraded":"2024-04-25T10:09:45.373811+0000","last_fullsized":"2024-04-25T10:09:45.373811+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:31.024318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.442072+0000","last_change":"2024-04-25T10:09:43.353813+0000","last_active":"2024-04-25T10:09:45.442072+0000","last_peered":"2024-04-25T10:09:45.442072+0000","last_clean":"2024-04-25T10:09:45.442072+0000","last_became_active":"2024-04-25T10:09:43.353299+0000","last_became_peered":"2024-04-25T10:09:43.353299+0000","last_unstale":"2024-04-25T10:09:45.442072+0000","last_undegraded":"2024-04-25T10:09:45.442072+0000","last_fullsized":"2024-04-25T10:09:45.442072+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:11.983424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.366324+0000","last_change":"2024-04-25T10:09:44.374238+0000","last_active":"2024-04-25T10:09:45.366324+0000","last_peered":"2024-04-25T10:09:45.366324+0000","last_clean":"2024-04-25T10:09:45.366324+0000","last_became_active":"2024-04-25T10:09:44.373925+0000","last_became_peered":"2024-04-25T10:09:44.373925+0000","last_unstale":"2024-04-25T10:09:45.366324+0000","last_undegraded":"2024-04-25T10:09:45.366324+0000","last_fullsized":"2024-04-25T10:09:45.366324+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:51.072641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.442805+0000","last_change":"2024-04-25T10:09:43.355710+0000","last_active":"2024-04-25T10:09:45.442805+0000","last_peered":"2024-04-25T10:09:45.442805+0000","last_clean":"2024-04-25T10:09:45.442805+0000","last_became_active":"2024-04-25T10:09:43.355327+0000","last_became_peered":"2024-04-25T10:09:43.355327+0000","last_unstale":"2024-04-25T10:09:45.442805+0000","last_undegraded":"2024-04-25T10:09:45.442805+0000","last_fullsized":"2024-04-25T10:09:45.442805+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:05.577095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.442381+0000","last_change":"2024-04-25T10:09:44.389893+0000","last_active":"2024-04-25T10:09:45.442381+0000","last_peered":"2024-04-25T10:09:45.442381+0000","last_clean":"2024-04-25T10:09:45.442381+0000","last_became_active":"2024-04-25T10:09:44.388914+0000","last_became_peered":"2024-04-25T10:09:44.388914+0000","last_unstale":"2024-04-25T10:09:45.442381+0000","last_undegraded":"2024-04-25T10:09:45.442381+0000","last_fullsized":"2024-04-25T10:09:45.442381+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:00.984115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.440975+0000","last_change":"2024-04-25T10:09:43.373841+0000","last_active":"2024-04-25T10:09:45.440975+0000","last_peered":"2024-04-25T10:09:45.440975+0000","last_clean":"2024-04-25T10:09:45.440975+0000","last_became_active":"2024-04-25T10:09:43.373513+0000","last_became_peered":"2024-04-25T10:09:43.373513+0000","last_unstale":"2024-04-25T10:09:45.440975+0000","last_undegraded":"2024-04-25T10:09:45.440975+0000","last_fullsized":"2024-04-25T10:09:45.440975+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:57.733608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.443114+0000","last_change":"2024-04-25T10:09:44.367344+0000","last_active":"2024-04-25T10:09:45.443114+0000","last_peered":"2024-04-25T10:09:45.443114+0000","last_clean":"2024-04-25T10:09:45.443114+0000","last_became_active":"2024-04-25T10:09:44.365295+0000","last_became_peered":"2024-04-25T10:09:44.365295+0000","last_unstale":"2024-04-25T10:09:45.443114+0000","last_undegraded":"2024-04-25T10:09:45.443114+0000","last_fullsized":"2024-04-25T10:09:45.443114+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:05.129656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.371322+0000","last_change":"2024-04-25T10:09:43.381879+0000","last_active":"2024-04-25T10:09:45.371322+0000","last_peered":"2024-04-25T10:09:45.371322+0000","last_clean":"2024-04-25T10:09:45.371322+0000","last_became_active":"2024-04-25T10:09:43.380515+0000","last_became_peered":"2024-04-25T10:09:43.380515+0000","last_unstale":"2024-04-25T10:09:45.371322+0000","last_undegraded":"2024-04-25T10:09:45.371322+0000","last_fullsized":"2024-04-25T10:09:45.371322+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:57.177972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.424532+0000","last_change":"2024-04-25T10:09:44.400148+0000","last_active":"2024-04-25T10:09:45.424532+0000","last_peered":"2024-04-25T10:09:45.424532+0000","last_clean":"2024-04-25T10:09:45.424532+0000","last_became_active":"2024-04-25T10:09:44.399889+0000","last_became_peered":"2024-04-25T10:09:44.399889+0000","last_unstale":"2024-04-25T10:09:45.424532+0000","last_undegraded":"2024-04-25T10:09:45.424532+0000","last_fullsized":"2024-04-25T10:09:45.424532+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:05.129656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.441429+0000","last_change":"2024-04-25T10:09:43.358364+0000","last_active":"2024-04-25T10:09:45.441429+0000","last_peered":"2024-04-25T10:09:45.441429+0000","last_clean":"2024-04-25T10:09:45.441429+0000","last_became_active":"2024-04-25T10:09:43.357019+0000","last_became_peered":"2024-04-25T10:09:43.357019+0000","last_unstale":"2024-04-25T10:09:45.441429+0000","last_undegraded":"2024-04-25T10:09:45.441429+0000","last_fullsized":"2024-04-25T10:09:45.441429+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:44.964997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.365876+0000","last_change":"2024-04-25T10:09:44.372827+0000","last_active":"2024-04-25T10:09:45.365876+0000","last_peered":"2024-04-25T10:09:45.365876+0000","last_clean":"2024-04-25T10:09:45.365876+0000","last_became_active":"2024-04-25T10:09:44.372375+0000","last_became_peered":"2024-04-25T10:09:44.372375+0000","last_unstale":"2024-04-25T10:09:45.365876+0000","last_undegraded":"2024-04-25T10:09:45.365876+0000","last_fullsized":"2024-04-25T10:09:45.365876+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:42.219619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.371791+0000","last_change":"2024-04-25T10:09:43.380141+0000","last_active":"2024-04-25T10:09:45.371791+0000","last_peered":"2024-04-25T10:09:45.371791+0000","last_clean":"2024-04-25T10:09:45.371791+0000","last_became_active":"2024-04-25T10:09:43.379538+0000","last_became_peered":"2024-04-25T10:09:43.379538+0000","last_unstale":"2024-04-25T10:09:45.371791+0000","last_undegraded":"2024-04-25T10:09:45.371791+0000","last_fullsized":"2024-04-25T10:09:45.371791+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:20.549003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.372638+0000","last_change":"2024-04-25T10:09:44.368231+0000","last_active":"2024-04-25T10:09:45.372638+0000","last_peered":"2024-04-25T10:09:45.372638+0000","last_clean":"2024-04-25T10:09:45.372638+0000","last_became_active":"2024-04-25T10:09:44.367404+0000","last_became_peered":"2024-04-25T10:09:44.367404+0000","last_unstale":"2024-04-25T10:09:45.372638+0000","last_undegraded":"2024-04-25T10:09:45.372638+0000","last_fullsized":"2024-04-25T10:09:45.372638+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:13.337953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.373523+0000","last_change":"2024-04-25T10:09:43.375256+0000","last_active":"2024-04-25T10:09:45.373523+0000","last_peered":"2024-04-25T10:09:45.373523+0000","last_clean":"2024-04-25T10:09:45.373523+0000","last_became_active":"2024-04-25T10:09:43.374546+0000","last_became_peered":"2024-04-25T10:09:43.374546+0000","last_unstale":"2024-04-25T10:09:45.373523+0000","last_undegraded":"2024-04-25T10:09:45.373523+0000","last_fullsized":"2024-04-25T10:09:45.373523+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:19.929450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.443766+0000","last_change":"2024-04-25T10:09:44.408074+0000","last_active":"2024-04-25T10:09:45.443766+0000","last_peered":"2024-04-25T10:09:45.443766+0000","last_clean":"2024-04-25T10:09:45.443766+0000","last_became_active":"2024-04-25T10:09:44.407369+0000","last_became_peered":"2024-04-25T10:09:44.407369+0000","last_unstale":"2024-04-25T10:09:45.443766+0000","last_undegraded":"2024-04-25T10:09:45.443766+0000","last_fullsized":"2024-04-25T10:09:45.443766+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:29.193905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.372646+0000","last_change":"2024-04-25T10:09:43.373239+0000","last_active":"2024-04-25T10:09:45.372646+0000","last_peered":"2024-04-25T10:09:45.372646+0000","last_clean":"2024-04-25T10:09:45.372646+0000","last_became_active":"2024-04-25T10:09:43.370508+0000","last_became_peered":"2024-04-25T10:09:43.370508+0000","last_unstale":"2024-04-25T10:09:45.372646+0000","last_undegraded":"2024-04-25T10:09:45.372646+0000","last_fullsized":"2024-04-25T10:09:45.372646+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:24.490794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.442771+0000","last_change":"2024-04-25T10:09:44.363980+0000","last_active":"2024-04-25T10:09:45.442771+0000","last_peered":"2024-04-25T10:09:45.442771+0000","last_clean":"2024-04-25T10:09:45.442771+0000","last_became_active":"2024-04-25T10:09:44.363681+0000","last_became_peered":"2024-04-25T10:09:44.363681+0000","last_unstale":"2024-04-25T10:09:45.442771+0000","last_undegraded":"2024-04-25T10:09:45.442771+0000","last_fullsized":"2024-04-25T10:09:45.442771+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:50.028392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.373405+0000","last_change":"2024-04-25T10:09:43.370279+0000","last_active":"2024-04-25T10:09:45.373405+0000","last_peered":"2024-04-25T10:09:45.373405+0000","last_clean":"2024-04-25T10:09:45.373405+0000","last_became_active":"2024-04-25T10:09:43.369559+0000","last_became_peered":"2024-04-25T10:09:43.369559+0000","last_unstale":"2024-04-25T10:09:45.373405+0000","last_undegraded":"2024-04-25T10:09:45.373405+0000","last_fullsized":"2024-04-25T10:09:45.373405+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:48.929786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.366369+0000","last_change":"2024-04-25T10:09:44.382873+0000","last_active":"2024-04-25T10:09:45.366369+0000","last_peered":"2024-04-25T10:09:45.366369+0000","last_clean":"2024-04-25T10:09:45.366369+0000","last_became_active":"2024-04-25T10:09:44.382527+0000","last_became_peered":"2024-04-25T10:09:44.382527+0000","last_unstale":"2024-04-25T10:09:45.366369+0000","last_undegraded":"2024-04-25T10:09:45.366369+0000","last_fullsized":"2024-04-25T10:09:45.366369+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:21.743535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.442853+0000","last_change":"2024-04-25T10:09:43.377693+0000","last_active":"2024-04-25T10:09:45.442853+0000","last_peered":"2024-04-25T10:09:45.442853+0000","last_clean":"2024-04-25T10:09:45.442853+0000","last_became_active":"2024-04-25T10:09:43.374049+0000","last_became_peered":"2024-04-25T10:09:43.374049+0000","last_unstale":"2024-04-25T10:09:45.442853+0000","last_undegraded":"2024-04-25T10:09:45.442853+0000","last_fullsized":"2024-04-25T10:09:45.442853+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:07.070829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.442042+0000","last_change":"2024-04-25T10:09:44.366280+0000","last_active":"2024-04-25T10:09:45.442042+0000","last_peered":"2024-04-25T10:09:45.442042+0000","last_clean":"2024-04-25T10:09:45.442042+0000","last_became_active":"2024-04-25T10:09:44.365957+0000","last_became_peered":"2024-04-25T10:09:44.365957+0000","last_unstale":"2024-04-25T10:09:45.442042+0000","last_undegraded":"2024-04-25T10:09:45.442042+0000","last_fullsized":"2024-04-25T10:09:45.442042+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:57.716839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.371545+0000","last_change":"2024-04-25T10:09:43.386806+0000","last_active":"2024-04-25T10:09:45.371545+0000","last_peered":"2024-04-25T10:09:45.371545+0000","last_clean":"2024-04-25T10:09:45.371545+0000","last_became_active":"2024-04-25T10:09:43.385915+0000","last_became_peered":"2024-04-25T10:09:43.385915+0000","last_unstale":"2024-04-25T10:09:45.371545+0000","last_undegraded":"2024-04-25T10:09:45.371545+0000","last_fullsized":"2024-04-25T10:09:45.371545+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:21.386772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.441419+0000","last_change":"2024-04-25T10:09:44.371745+0000","last_active":"2024-04-25T10:09:45.441419+0000","last_peered":"2024-04-25T10:09:45.441419+0000","last_clean":"2024-04-25T10:09:45.441419+0000","last_became_active":"2024-04-25T10:09:44.371328+0000","last_became_peered":"2024-04-25T10:09:44.371328+0000","last_unstale":"2024-04-25T10:09:45.441419+0000","last_undegraded":"2024-04-25T10:09:45.441419+0000","last_fullsized":"2024-04-25T10:09:45.441419+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:51.809216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.441731+0000","last_change":"2024-04-25T10:09:43.357734+0000","last_active":"2024-04-25T10:09:45.441731+0000","last_peered":"2024-04-25T10:09:45.441731+0000","last_clean":"2024-04-25T10:09:45.441731+0000","last_became_active":"2024-04-25T10:09:43.356058+0000","last_became_peered":"2024-04-25T10:09:43.356058+0000","last_unstale":"2024-04-25T10:09:45.441731+0000","last_undegraded":"2024-04-25T10:09:45.441731+0000","last_fullsized":"2024-04-25T10:09:45.441731+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:52.858806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.441499+0000","last_change":"2024-04-25T10:09:44.399564+0000","last_active":"2024-04-25T10:09:45.441499+0000","last_peered":"2024-04-25T10:09:45.441499+0000","last_clean":"2024-04-25T10:09:45.441499+0000","last_became_active":"2024-04-25T10:09:44.399212+0000","last_became_peered":"2024-04-25T10:09:44.399212+0000","last_unstale":"2024-04-25T10:09:45.441499+0000","last_undegraded":"2024-04-25T10:09:45.441499+0000","last_fullsized":"2024-04-25T10:09:45.441499+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:46.704976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.424247+0000","last_change":"2024-04-25T10:09:43.355582+0000","last_active":"2024-04-25T10:09:45.424247+0000","last_peered":"2024-04-25T10:09:45.424247+0000","last_clean":"2024-04-25T10:09:45.424247+0000","last_became_active":"2024-04-25T10:09:43.355240+0000","last_became_peered":"2024-04-25T10:09:43.355240+0000","last_unstale":"2024-04-25T10:09:45.424247+0000","last_undegraded":"2024-04-25T10:09:45.424247+0000","last_fullsized":"2024-04-25T10:09:45.424247+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:30.468014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.425175+0000","last_change":"2024-04-25T10:09:44.375361+0000","last_active":"2024-04-25T10:09:45.425175+0000","last_peered":"2024-04-25T10:09:45.425175+0000","last_clean":"2024-04-25T10:09:45.425175+0000","last_became_active":"2024-04-25T10:09:44.374935+0000","last_became_peered":"2024-04-25T10:09:44.374935+0000","last_unstale":"2024-04-25T10:09:45.425175+0000","last_undegraded":"2024-04-25T10:09:45.425175+0000","last_fullsized":"2024-04-25T10:09:45.425175+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:12.987249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.373576+0000","last_change":"2024-04-25T10:09:43.374605+0000","last_active":"2024-04-25T10:09:45.373576+0000","last_peered":"2024-04-25T10:09:45.373576+0000","last_clean":"2024-04-25T10:09:45.373576+0000","last_became_active":"2024-04-25T10:09:43.373960+0000","last_became_peered":"2024-04-25T10:09:43.373960+0000","last_unstale":"2024-04-25T10:09:45.373576+0000","last_undegraded":"2024-04-25T10:09:45.373576+0000","last_fullsized":"2024-04-25T10:09:45.373576+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:16.424483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.443229+0000","last_change":"2024-04-25T10:09:44.369314+0000","last_active":"2024-04-25T10:09:45.443229+0000","last_peered":"2024-04-25T10:09:45.443229+0000","last_clean":"2024-04-25T10:09:45.443229+0000","last_became_active":"2024-04-25T10:09:44.368200+0000","last_became_peered":"2024-04-25T10:09:44.368200+0000","last_unstale":"2024-04-25T10:09:45.443229+0000","last_undegraded":"2024-04-25T10:09:45.443229+0000","last_fullsized":"2024-04-25T10:09:45.443229+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:13.093130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.443939+0000","last_change":"2024-04-25T10:09:43.357438+0000","last_active":"2024-04-25T10:09:45.443939+0000","last_peered":"2024-04-25T10:09:45.443939+0000","last_clean":"2024-04-25T10:09:45.443939+0000","last_became_active":"2024-04-25T10:09:43.356729+0000","last_became_peered":"2024-04-25T10:09:43.356729+0000","last_unstale":"2024-04-25T10:09:45.443939+0000","last_undegraded":"2024-04-25T10:09:45.443939+0000","last_fullsized":"2024-04-25T10:09:45.443939+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:03.378299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.442695+0000","last_change":"2024-04-25T10:09:44.411931+0000","last_active":"2024-04-25T10:09:45.442695+0000","last_peered":"2024-04-25T10:09:45.442695+0000","last_clean":"2024-04-25T10:09:45.442695+0000","last_became_active":"2024-04-25T10:09:44.411290+0000","last_became_peered":"2024-04-25T10:09:44.411290+0000","last_unstale":"2024-04-25T10:09:45.442695+0000","last_undegraded":"2024-04-25T10:09:45.442695+0000","last_fullsized":"2024-04-25T10:09:45.442695+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:53.055528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.366368+0000","last_change":"2024-04-25T10:09:43.355496+0000","last_active":"2024-04-25T10:09:45.366368+0000","last_peered":"2024-04-25T10:09:45.366368+0000","last_clean":"2024-04-25T10:09:45.366368+0000","last_became_active":"2024-04-25T10:09:43.355094+0000","last_became_peered":"2024-04-25T10:09:43.355094+0000","last_unstale":"2024-04-25T10:09:45.366368+0000","last_undegraded":"2024-04-25T10:09:45.366368+0000","last_fullsized":"2024-04-25T10:09:45.366368+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:11.971570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.443519+0000","last_change":"2024-04-25T10:09:44.395959+0000","last_active":"2024-04-25T10:09:45.443519+0000","last_peered":"2024-04-25T10:09:45.443519+0000","last_clean":"2024-04-25T10:09:45.443519+0000","last_became_active":"2024-04-25T10:09:44.395304+0000","last_became_peered":"2024-04-25T10:09:44.395304+0000","last_unstale":"2024-04-25T10:09:45.443519+0000","last_undegraded":"2024-04-25T10:09:45.443519+0000","last_fullsized":"2024-04-25T10:09:45.443519+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:19.474170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.424713+0000","last_change":"2024-04-25T10:09:43.366158+0000","last_active":"2024-04-25T10:09:45.424713+0000","last_peered":"2024-04-25T10:09:45.424713+0000","last_clean":"2024-04-25T10:09:45.424713+0000","last_became_active":"2024-04-25T10:09:43.365706+0000","last_became_peered":"2024-04-25T10:09:43.365706+0000","last_unstale":"2024-04-25T10:09:45.424713+0000","last_undegraded":"2024-04-25T10:09:45.424713+0000","last_fullsized":"2024-04-25T10:09:45.424713+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:12.775042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.366083+0000","last_change":"2024-04-25T10:09:44.374622+0000","last_active":"2024-04-25T10:09:45.366083+0000","last_peered":"2024-04-25T10:09:45.366083+0000","last_clean":"2024-04-25T10:09:45.366083+0000","last_became_active":"2024-04-25T10:09:44.373986+0000","last_became_peered":"2024-04-25T10:09:44.373986+0000","last_unstale":"2024-04-25T10:09:45.366083+0000","last_undegraded":"2024-04-25T10:09:45.366083+0000","last_fullsized":"2024-04-25T10:09:45.366083+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:17.540905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.365933+0000","last_change":"2024-04-25T10:09:43.355900+0000","last_active":"2024-04-25T10:09:45.365933+0000","last_peered":"2024-04-25T10:09:45.365933+0000","last_clean":"2024-04-25T10:09:45.365933+0000","last_became_active":"2024-04-25T10:09:43.355303+0000","last_became_peered":"2024-04-25T10:09:43.355303+0000","last_unstale":"2024-04-25T10:09:45.365933+0000","last_undegraded":"2024-04-25T10:09:45.365933+0000","last_fullsized":"2024-04-25T10:09:45.365933+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:04.268565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.373156+0000","last_change":"2024-04-25T10:09:44.405848+0000","last_active":"2024-04-25T10:09:45.373156+0000","last_peered":"2024-04-25T10:09:45.373156+0000","last_clean":"2024-04-25T10:09:45.373156+0000","last_became_active":"2024-04-25T10:09:44.405201+0000","last_became_peered":"2024-04-25T10:09:44.405201+0000","last_unstale":"2024-04-25T10:09:45.373156+0000","last_undegraded":"2024-04-25T10:09:45.373156+0000","last_fullsized":"2024-04-25T10:09:45.373156+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:10.798163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.440591+0000","last_change":"2024-04-25T10:09:43.365732+0000","last_active":"2024-04-25T10:09:45.440591+0000","last_peered":"2024-04-25T10:09:45.440591+0000","last_clean":"2024-04-25T10:09:45.440591+0000","last_became_active":"2024-04-25T10:09:43.365335+0000","last_became_peered":"2024-04-25T10:09:43.365335+0000","last_unstale":"2024-04-25T10:09:45.440591+0000","last_undegraded":"2024-04-25T10:09:45.440591+0000","last_fullsized":"2024-04-25T10:09:45.440591+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:51.130358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.440881+0000","last_change":"2024-04-25T10:09:44.357121+0000","last_active":"2024-04-25T10:09:45.440881+0000","last_peered":"2024-04-25T10:09:45.440881+0000","last_clean":"2024-04-25T10:09:45.440881+0000","last_became_active":"2024-04-25T10:09:44.356821+0000","last_became_peered":"2024-04-25T10:09:44.356821+0000","last_unstale":"2024-04-25T10:09:45.440881+0000","last_undegraded":"2024-04-25T10:09:45.440881+0000","last_fullsized":"2024-04-25T10:09:45.440881+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:25.494620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.370799+0000","last_change":"2024-04-25T10:09:43.381091+0000","last_active":"2024-04-25T10:09:45.370799+0000","last_peered":"2024-04-25T10:09:45.370799+0000","last_clean":"2024-04-25T10:09:45.370799+0000","last_became_active":"2024-04-25T10:09:43.379688+0000","last_became_peered":"2024-04-25T10:09:43.379688+0000","last_unstale":"2024-04-25T10:09:45.370799+0000","last_undegraded":"2024-04-25T10:09:45.370799+0000","last_fullsized":"2024-04-25T10:09:45.370799+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:03.947418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.370803+0000","last_change":"2024-04-25T10:09:44.379653+0000","last_active":"2024-04-25T10:09:45.370803+0000","last_peered":"2024-04-25T10:09:45.370803+0000","last_clean":"2024-04-25T10:09:45.370803+0000","last_became_active":"2024-04-25T10:09:44.378916+0000","last_became_peered":"2024-04-25T10:09:44.378916+0000","last_unstale":"2024-04-25T10:09:45.370803+0000","last_undegraded":"2024-04-25T10:09:45.370803+0000","last_fullsized":"2024-04-25T10:09:45.370803+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:12.531766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.371333+0000","last_change":"2024-04-25T10:09:43.374952+0000","last_active":"2024-04-25T10:09:45.371333+0000","last_peered":"2024-04-25T10:09:45.371333+0000","last_clean":"2024-04-25T10:09:45.371333+0000","last_became_active":"2024-04-25T10:09:43.374058+0000","last_became_peered":"2024-04-25T10:09:43.374058+0000","last_unstale":"2024-04-25T10:09:45.371333+0000","last_undegraded":"2024-04-25T10:09:45.371333+0000","last_fullsized":"2024-04-25T10:09:45.371333+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:23.172044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.442236+0000","last_change":"2024-04-25T10:09:44.371686+0000","last_active":"2024-04-25T10:09:45.442236+0000","last_peered":"2024-04-25T10:09:45.442236+0000","last_clean":"2024-04-25T10:09:45.442236+0000","last_became_active":"2024-04-25T10:09:44.371411+0000","last_became_peered":"2024-04-25T10:09:44.371411+0000","last_unstale":"2024-04-25T10:09:45.442236+0000","last_undegraded":"2024-04-25T10:09:45.442236+0000","last_fullsized":"2024-04-25T10:09:45.442236+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:43.324048+0000","last_clean_scrub_stamp":"2024-04-25T10:09:43.324048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:07.129061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.372891+0000","last_change":"2024-04-25T10:09:43.368449+0000","last_active":"2024-04-25T10:09:45.372891+0000","last_peered":"2024-04-25T10:09:45.372891+0000","last_clean":"2024-04-25T10:09:45.372891+0000","last_became_active":"2024-04-25T10:09:43.367552+0000","last_became_peered":"2024-04-25T10:09:43.367552+0000","last_unstale":"2024-04-25T10:09:45.372891+0000","last_undegraded":"2024-04-25T10:09:45.372891+0000","last_fullsized":"2024-04-25T10:09:45.372891+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:42.320222+0000","last_clean_scrub_stamp":"2024-04-25T10:09:42.320222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:01.763159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":256,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T10:09:45.441942+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:09:45.441942+0000","last_peered":"2024-04-25T10:09:45.441942+0000","last_clean":"2024-04-25T10:09:45.441942+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:09:45.441942+0000","last_undegraded":"2024-04-25T10:09:45.441942+0000","last_fullsized":"2024-04-25T10:09:45.441942+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509500,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":940085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509499,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":835584,"data_stored":349171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542202,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":839680,"data_stored":349193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542203,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":835584,"data_stored":349191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36160,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335680,"statfs":{"total":96636764160,"available":96599736320,"internally_reserved":0,"allocated":839680,"data_stored":349213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542203,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36164,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335676,"statfs":{"total":96636764160,"available":96599732224,"internally_reserved":0,"allocated":843776,"data_stored":350713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542203,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36180,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335660,"statfs":{"total":96636764160,"available":96599715840,"internally_reserved":0,"allocated":851968,"data_stored":351329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36740,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335100,"statfs":{"total":96636764160,"available":96599142400,"internally_reserved":0,"allocated":1425408,"data_stored":939469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:09:48.808 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:09:48.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:09:48.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.977+0000 7fcd731ed640 1 Processor -- start 2024-04-25T10:09:48.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.977+0000 7fcd731ed640 1 -- start start 2024-04-25T10:09:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.977+0000 7fcd731ed640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcd6c0ced20 0x7fcd6c0d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.977+0000 7fcd731ed640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcd6c0d0970 0x7fcd6c0b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.977+0000 7fcd731ed640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd6c15e000 con 0x7fcd6c15dc20 2024-04-25T10:09:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.977+0000 7fcd731ed640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6c0d0da0 con 0x7fcd6c0d0970 2024-04-25T10:09:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.977+0000 7fcd731ed640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6c0d1cc0 con 0x7fcd6c0ced20 2024-04-25T10:09:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.977+0000 7fcd7178a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcd6c0d0970 0x7fcd6c0b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.977+0000 7fcd7178a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcd6c0d0970 0x7fcd6c0b6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36702/0 (socket says 172.21.15.60:36702) 2024-04-25T10:09:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.977+0000 7fcd7178a640 1 -- 172.21.15.60:0/3354023113 learned_addr learned my addr 172.21.15.60:0/3354023113 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:48.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.977+0000 7fcd6bfff640 1 --2- 172.21.15.60:0/3354023113 >> v2:172.21.15.100:3300/0 conn(0x7fcd6c0ced20 0x7fcd6c0d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.977+0000 7fcd7178a640 1 -- 172.21.15.60:0/3354023113 >> v2:172.21.15.100:3300/0 conn(0x7fcd6c0ced20 msgr2=0x7fcd6c0d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:48.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.977+0000 7fcd7178a640 1 --2- 172.21.15.60:0/3354023113 >> v2:172.21.15.100:3300/0 conn(0x7fcd6c0ced20 0x7fcd6c0d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.977+0000 7fcd7178a640 1 -- 172.21.15.60:0/3354023113 >> v1:172.21.15.100:6789/0 conn(0x7fcd6c15dc20 legacy=0x7fcd6c0d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:48.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.977+0000 7fcd7178a640 1 -- 172.21.15.60:0/3354023113 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c0d1fa0 con 0x7fcd6c0d0970 2024-04-25T10:09:48.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.981+0000 7fcd7178a640 1 --2- 172.21.15.60:0/3354023113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6c0d0970 0x7fcd6c0b6ab0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0d4ac0 tx=0x7fcd6401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f960161741183b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:48.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.981+0000 7fcd6b7fe640 1 -- 172.21.15.60:0/3354023113 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd640a5670 con 0x7fcd6c0d0970 2024-04-25T10:09:48.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.981+0000 7fcd6b7fe640 1 -- 172.21.15.60:0/3354023113 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd640a5810 con 0x7fcd6c0d0970 2024-04-25T10:09:48.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.981+0000 7fcd6b7fe640 1 -- 172.21.15.60:0/3354023113 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd640a5be0 con 0x7fcd6c0d0970 2024-04-25T10:09:48.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.981+0000 7fcd731ed640 1 -- 172.21.15.60:0/3354023113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6c0d0970 msgr2=0x7fcd6c0b6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.981+0000 7fcd731ed640 1 --2- 172.21.15.60:0/3354023113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6c0d0970 0x7fcd6c0b6ab0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0d4ac0 tx=0x7fcd6401c9b0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.981+0000 7fcd731ed640 1 -- 172.21.15.60:0/3354023113 shutdown_connections 2024-04-25T10:09:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.981+0000 7fcd731ed640 1 --2- 172.21.15.60:0/3354023113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6c0d0970 0x7fcd6c0b6ab0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.981+0000 7fcd731ed640 1 --2- 172.21.15.60:0/3354023113 >> v2:172.21.15.100:3300/0 conn(0x7fcd6c0ced20 0x7fcd6c0d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.981+0000 7fcd731ed640 1 -- 172.21.15.60:0/3354023113 >> 172.21.15.60:0/3354023113 conn(0x7fcd6c072940 msgr2=0x7fcd6c0b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.981+0000 7fcd731ed640 1 -- 172.21.15.60:0/3354023113 shutdown_connections 2024-04-25T10:09:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.981+0000 7fcd731ed640 1 -- 172.21.15.60:0/3354023113 wait complete. 2024-04-25T10:09:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.981+0000 7fcd731ed640 1 Processor -- start 2024-04-25T10:09:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.981+0000 7fcd731ed640 1 -- start start 2024-04-25T10:09:48.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd731ed640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd6c0ced20 0x7fcd6c0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd731ed640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd6c0d0970 0x7fcd6c155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd731ed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6c15dc20 0x7fcd6c155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd731ed640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c0b7980 con 0x7fcd6c15dc20 2024-04-25T10:09:48.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd731ed640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c0b7800 con 0x7fcd6c0d0970 2024-04-25T10:09:48.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd731ed640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c0b7b00 con 0x7fcd6c0ced20 2024-04-25T10:09:48.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd7178a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6c15dc20 0x7fcd6c155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd7178a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6c15dc20 0x7fcd6c155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36708/0 (socket says 172.21.15.60:36708) 2024-04-25T10:09:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd7178a640 1 -- 172.21.15.60:0/1414582026 learned_addr learned my addr 172.21.15.60:0/1414582026 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd6bfff640 1 --2- 172.21.15.60:0/1414582026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd6c0d0970 0x7fcd6c155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd70f89640 1 --2- 172.21.15.60:0/1414582026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd6c0ced20 0x7fcd6c0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd7178a640 1 -- 172.21.15.60:0/1414582026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd6c0ced20 msgr2=0x7fcd6c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd7178a640 1 --2- 172.21.15.60:0/1414582026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd6c0ced20 0x7fcd6c0ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd7178a640 1 -- 172.21.15.60:0/1414582026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd6c0d0970 msgr2=0x7fcd6c155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd7178a640 1 --2- 172.21.15.60:0/1414582026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd6c0d0970 0x7fcd6c155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd7178a640 1 -- 172.21.15.60:0/1414582026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c0cd550 con 0x7fcd6c15dc20 2024-04-25T10:09:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd7178a640 1 --2- 172.21.15.60:0/1414582026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6c15dc20 0x7fcd6c155660 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fcd6401a5e0 tx=0x7fcd64019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd697fa640 1 -- 172.21.15.60:0/1414582026 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd64009510 con 0x7fcd6c15dc20 2024-04-25T10:09:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd697fa640 1 -- 172.21.15.60:0/1414582026 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd640be070 con 0x7fcd6c15dc20 2024-04-25T10:09:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd731ed640 1 -- 172.21.15.60:0/1414582026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd6c0cd830 con 0x7fcd6c15dc20 2024-04-25T10:09:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd697fa640 1 -- 172.21.15.60:0/1414582026 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd640b88d0 con 0x7fcd6c15dc20 2024-04-25T10:09:48.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd731ed640 1 -- 172.21.15.60:0/1414582026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6c154920 con 0x7fcd6c15dc20 2024-04-25T10:09:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd731ed640 1 -- 172.21.15.60:0/1414582026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6c074770 con 0x7fcd6c15dc20 2024-04-25T10:09:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd697fa640 1 -- 172.21.15.60:0/1414582026 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd640b8a70 con 0x7fcd6c15dc20 2024-04-25T10:09:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.985+0000 7fcd697fa640 1 --2- 172.21.15.60:0/1414582026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcd50066f60 0x7fcd50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.989+0000 7fcd697fa640 1 -- 172.21.15.60:0/1414582026 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd641201c0 con 0x7fcd6c15dc20 2024-04-25T10:09:48.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.989+0000 7fcd70f89640 1 --2- 172.21.15.60:0/1414582026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcd50066f60 0x7fcd50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:49.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.989+0000 7fcd70f89640 1 --2- 172.21.15.60:0/1414582026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcd50066f60 0x7fcd50069420 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcd60004e30 tx=0x7fcd60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:49.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:48.993+0000 7fcd697fa640 1 -- 172.21.15.60:0/1414582026 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd640165a0 con 0x7fcd6c15dc20 2024-04-25T10:09:49.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.205+0000 7fcd731ed640 1 -- 172.21.15.60:0/1414582026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fcd6c15ab20 con 0x7fcd6c15dc20 2024-04-25T10:09:49.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.209+0000 7fcd697fa640 1 -- 172.21.15.60:0/1414582026 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7fcd640f1260 con 0x7fcd6c15dc20 2024-04-25T10:09:49.219 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T10:09:49.219 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCiKipmv65dLxAA6/QpPrqE9aT+CtZqkyutog== 2024-04-25T10:09:49.219 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-04-25T10:09:49.219 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-04-25T10:09:49.219 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:09:49.219 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-04-25T10:09:49.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.213+0000 7fcd731ed640 1 -- 172.21.15.60:0/1414582026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcd50066f60 msgr2=0x7fcd50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:49.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.213+0000 7fcd731ed640 1 --2- 172.21.15.60:0/1414582026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcd50066f60 0x7fcd50069420 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcd60004e30 tx=0x7fcd60009040 comp rx=0 tx=0).stop 2024-04-25T10:09:49.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.213+0000 7fcd731ed640 1 -- 172.21.15.60:0/1414582026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6c15dc20 msgr2=0x7fcd6c155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:49.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.213+0000 7fcd731ed640 1 --2- 172.21.15.60:0/1414582026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6c15dc20 0x7fcd6c155660 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fcd6401a5e0 tx=0x7fcd64019750 comp rx=0 tx=0).stop 2024-04-25T10:09:49.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.213+0000 7fcd731ed640 1 -- 172.21.15.60:0/1414582026 shutdown_connections 2024-04-25T10:09:49.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.213+0000 7fcd731ed640 1 --2- 172.21.15.60:0/1414582026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcd50066f60 0x7fcd50069420 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.213+0000 7fcd731ed640 1 --2- 172.21.15.60:0/1414582026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6c15dc20 0x7fcd6c155660 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.213+0000 7fcd731ed640 1 --2- 172.21.15.60:0/1414582026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd6c0d0970 0x7fcd6c155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.213+0000 7fcd731ed640 1 --2- 172.21.15.60:0/1414582026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd6c0ced20 0x7fcd6c0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.213+0000 7fcd731ed640 1 -- 172.21.15.60:0/1414582026 >> 172.21.15.60:0/1414582026 conn(0x7fcd6c072940 msgr2=0x7fcd6c0b5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.213+0000 7fcd731ed640 1 -- 172.21.15.60:0/1414582026 shutdown_connections 2024-04-25T10:09:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.213+0000 7fcd731ed640 1 -- 172.21.15.60:0/1414582026 wait complete. 2024-04-25T10:09:49.237 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:49.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.353+0000 7f9f0674a640 1 Processor -- start 2024-04-25T10:09:49.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.353+0000 7f9f0674a640 1 -- start start 2024-04-25T10:09:49.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.353+0000 7f9f0674a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9f000ce7a0 0x7f9f000ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:49.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.353+0000 7f9f0674a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9f000b4660 0x7f9f000b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:49.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.353+0000 7f9f0674a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f000cf0c0 con 0x7f9f0015bc90 2024-04-25T10:09:49.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.353+0000 7f9f0674a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f000cc370 con 0x7f9f000ce7a0 2024-04-25T10:09:49.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.353+0000 7f9f0674a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f00071ca0 con 0x7f9f000b4660 2024-04-25T10:09:49.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.353+0000 7f9efffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9f000ce7a0 0x7f9f000ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:49.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.353+0000 7f9efffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9f000ce7a0 0x7f9f000ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36720/0 (socket says 172.21.15.60:36720) 2024-04-25T10:09:49.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.353+0000 7f9efffff640 1 -- 172.21.15.60:0/2534482004 learned_addr learned my addr 172.21.15.60:0/2534482004 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:49.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.353+0000 7f9f04ce7640 1 --2- 172.21.15.60:0/2534482004 >> v2:172.21.15.100:3300/0 conn(0x7f9f000b4660 0x7f9f000b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:49.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.353+0000 7f9efffff640 1 -- 172.21.15.60:0/2534482004 >> v2:172.21.15.100:3300/0 conn(0x7f9f000b4660 msgr2=0x7f9f000b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:49.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.353+0000 7f9efffff640 1 --2- 172.21.15.60:0/2534482004 >> v2:172.21.15.100:3300/0 conn(0x7f9f000b4660 0x7f9f000b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.353+0000 7f9efffff640 1 -- 172.21.15.60:0/2534482004 >> v1:172.21.15.100:6789/0 conn(0x7f9f0015bc90 legacy=0x7f9f000b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:49.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.353+0000 7f9efffff640 1 -- 172.21.15.60:0/2534482004 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f000cca40 con 0x7f9f000ce7a0 2024-04-25T10:09:49.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9efffff640 1 --2- 172.21.15.60:0/2534482004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f000ce7a0 0x7f9f000ceb80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9f0006cf50 tx=0x7f9ef001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ec9daaf53fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:49.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9efeffd640 1 -- 172.21.15.60:0/2534482004 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00b0070 con 0x7f9f000ce7a0 2024-04-25T10:09:49.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9efeffd640 1 -- 172.21.15.60:0/2534482004 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef00ab600 con 0x7f9f000ce7a0 2024-04-25T10:09:49.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9efeffd640 1 -- 172.21.15.60:0/2534482004 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00ab9d0 con 0x7f9f000ce7a0 2024-04-25T10:09:49.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f0674a640 1 -- 172.21.15.60:0/2534482004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f000ce7a0 msgr2=0x7f9f000ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:49.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f0674a640 1 --2- 172.21.15.60:0/2534482004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f000ce7a0 0x7f9f000ceb80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9f0006cf50 tx=0x7f9ef001c520 comp rx=0 tx=0).stop 2024-04-25T10:09:49.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f0674a640 1 -- 172.21.15.60:0/2534482004 shutdown_connections 2024-04-25T10:09:49.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f0674a640 1 --2- 172.21.15.60:0/2534482004 >> v2:172.21.15.100:3300/0 conn(0x7f9f000b4660 0x7f9f000b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f0674a640 1 --2- 172.21.15.60:0/2534482004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f000ce7a0 0x7f9f000ceb80 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f0674a640 1 -- 172.21.15.60:0/2534482004 >> 172.21.15.60:0/2534482004 conn(0x7f9f000b12d0 msgr2=0x7f9f000b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:49.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f0674a640 1 -- 172.21.15.60:0/2534482004 shutdown_connections 2024-04-25T10:09:49.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f0674a640 1 -- 172.21.15.60:0/2534482004 wait complete. 2024-04-25T10:09:49.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f0674a640 1 Processor -- start 2024-04-25T10:09:49.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f0674a640 1 -- start start 2024-04-25T10:09:49.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f0674a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f000ce7a0 0x7f9f0015fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:49.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f0674a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f0015bc90 0x7f9f00155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:49.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f0674a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0015f5b0 0x7f9f0015df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:49.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f0674a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f000b4400 con 0x7f9f0015f5b0 2024-04-25T10:09:49.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f0674a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f000b4280 con 0x7f9f0015bc90 2024-04-25T10:09:49.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f0674a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f000b3220 con 0x7f9f000ce7a0 2024-04-25T10:09:49.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f04ce7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0015f5b0 0x7f9f0015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:49.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f04ce7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0015f5b0 0x7f9f0015df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36730/0 (socket says 172.21.15.60:36730) 2024-04-25T10:09:49.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9f04ce7640 1 -- 172.21.15.60:0/3329522782 learned_addr learned my addr 172.21.15.60:0/3329522782 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:49.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.357+0000 7f9eff7fe640 1 --2- 172.21.15.60:0/3329522782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f0015bc90 0x7f9f00155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:49.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9efffff640 1 --2- 172.21.15.60:0/3329522782 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f000ce7a0 0x7f9f0015fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:49.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9f04ce7640 1 -- 172.21.15.60:0/3329522782 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f000ce7a0 msgr2=0x7f9f0015fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:49.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9f04ce7640 1 --2- 172.21.15.60:0/3329522782 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f000ce7a0 0x7f9f0015fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9f04ce7640 1 -- 172.21.15.60:0/3329522782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f0015bc90 msgr2=0x7f9f00155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:49.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9f04ce7640 1 --2- 172.21.15.60:0/3329522782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f0015bc90 0x7f9f00155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9f04ce7640 1 -- 172.21.15.60:0/3329522782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f001f57d0 con 0x7f9f0015f5b0 2024-04-25T10:09:49.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9f04ce7640 1 --2- 172.21.15.60:0/3329522782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0015f5b0 0x7f9f0015df30 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800efc0 tx=0x7f9ef800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:49.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9efcff9640 1 -- 172.21.15.60:0/3329522782 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8018020 con 0x7f9f0015f5b0 2024-04-25T10:09:49.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9efcff9640 1 -- 172.21.15.60:0/3329522782 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef8014480 con 0x7f9f0015f5b0 2024-04-25T10:09:49.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9efcff9640 1 -- 172.21.15.60:0/3329522782 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8014870 con 0x7f9f0015f5b0 2024-04-25T10:09:49.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9f0674a640 1 -- 172.21.15.60:0/3329522782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f001f5ab0 con 0x7f9f0015f5b0 2024-04-25T10:09:49.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9f0674a640 1 -- 172.21.15.60:0/3329522782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f001f6070 con 0x7f9f0015f5b0 2024-04-25T10:09:49.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9f0674a640 1 -- 172.21.15.60:0/3329522782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f00074770 con 0x7f9f0015f5b0 2024-04-25T10:09:49.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9efcff9640 1 -- 172.21.15.60:0/3329522782 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ef801b020 con 0x7f9f0015f5b0 2024-04-25T10:09:49.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9efcff9640 1 --2- 172.21.15.60:0/3329522782 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9eec066f60 0x7f9eec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:49.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9efcff9640 1 -- 172.21.15.60:0/3329522782 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ef8085eb0 con 0x7f9f0015f5b0 2024-04-25T10:09:49.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9efffff640 1 --2- 172.21.15.60:0/3329522782 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9eec066f60 0x7f9eec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:49.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.361+0000 7f9efffff640 1 --2- 172.21.15.60:0/3329522782 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9eec066f60 0x7f9eec069420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00ad000 tx=0x7f9ef0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:49.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.365+0000 7f9efcff9640 1 -- 172.21.15.60:0/3329522782 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ef8022030 con 0x7f9f0015f5b0 2024-04-25T10:09:49.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.553+0000 7f9f0674a640 1 -- 172.21.15.60:0/3329522782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f0006c250 con 0x7f9f0015f5b0 2024-04-25T10:09:49.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.553+0000 7f9efcff9640 1 -- 172.21.15.60:0/3329522782 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f9ef8010040 con 0x7f9f0015f5b0 2024-04-25T10:09:49.563 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:49.563 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3185787428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3185787428},{"type":"v1","addr":"172.21.15.100:6836","nonce":3185787428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2667415244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2667415244},{"type":"v1","addr":"172.21.15.100:6837","nonce":2667415244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/56747111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":56747111},{"type":"v1","addr":"172.21.15.60:6834","nonce":56747111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:43.707831+0000","modified":"2024-04-25T10:09:46.733783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6873":{"gid":6873,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1289503429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1289503429},{"type":"v1","addr":"172.21.15.60:6835","nonce":1289503429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:09:49.563 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 39 2024-04-25T10:09:49.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.557+0000 7f9f0674a640 1 -- 172.21.15.60:0/3329522782 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9eec066f60 msgr2=0x7f9eec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:49.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.557+0000 7f9f0674a640 1 --2- 172.21.15.60:0/3329522782 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9eec066f60 0x7f9eec069420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00ad000 tx=0x7f9ef0008040 comp rx=0 tx=0).stop 2024-04-25T10:09:49.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.557+0000 7f9f0674a640 1 -- 172.21.15.60:0/3329522782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0015f5b0 msgr2=0x7f9f0015df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:49.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.557+0000 7f9f0674a640 1 --2- 172.21.15.60:0/3329522782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0015f5b0 0x7f9f0015df30 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800efc0 tx=0x7f9ef800c490 comp rx=0 tx=0).stop 2024-04-25T10:09:49.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.557+0000 7f9f0674a640 1 -- 172.21.15.60:0/3329522782 shutdown_connections 2024-04-25T10:09:49.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.557+0000 7f9f0674a640 1 --2- 172.21.15.60:0/3329522782 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9eec066f60 0x7f9eec069420 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.557+0000 7f9f0674a640 1 --2- 172.21.15.60:0/3329522782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f0015f5b0 0x7f9f0015df30 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.557+0000 7f9f0674a640 1 --2- 172.21.15.60:0/3329522782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f0015bc90 0x7f9f00155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.557+0000 7f9f0674a640 1 --2- 172.21.15.60:0/3329522782 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f000ce7a0 0x7f9f0015fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.557+0000 7f9f0674a640 1 -- 172.21.15.60:0/3329522782 >> 172.21.15.60:0/3329522782 conn(0x7f9f000b12d0 msgr2=0x7f9f000af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:49.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.557+0000 7f9f0674a640 1 -- 172.21.15.60:0/3329522782 shutdown_connections 2024-04-25T10:09:49.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:49.557+0000 7f9f0674a640 1 -- 172.21.15.60:0/3329522782 wait complete. 2024-04-25T10:09:49.983 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:09:43.707831+0000', 'modified': '2024-04-25T10:09:46.733783+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6873}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6873': {'gid': 6873, 'name': 'c', 'rank': 0, 'incarnation': 35, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/1289503429', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 1289503429}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 1289503429}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:09:49.984 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:09:49.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:09:50.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f2372e90640 1 Processor -- start 2024-04-25T10:09:50.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f2372e90640 1 -- start start 2024-04-25T10:09:50.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f2372e90640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f236c15bc90 0x7f236c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f2372e90640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f236c0b4660 0x7f236c15c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f2372e90640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c071ca0 con 0x7f236c0ce7a0 2024-04-25T10:09:50.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f2372e90640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c0cc370 con 0x7f236c0b4660 2024-04-25T10:09:50.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f2372e90640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c0cca40 con 0x7f236c15bc90 2024-04-25T10:09:50.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f2370c2c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f236c0ce7a0 0x7f236c0ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44576/0 (socket says 172.21.15.60:44576) 2024-04-25T10:09:50.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f2370c2c640 1 -- 172.21.15.60:0/998725899 learned_addr learned my addr 172.21.15.60:0/998725899 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:50.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f237142d640 1 --2- 172.21.15.60:0/998725899 >> v2:172.21.15.60:3300/0 conn(0x7f236c0b4660 0x7f236c15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f236bfff640 1 --2- 172.21.15.60:0/998725899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f236c15bc90 0x7f236c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f236b7fe640 1 -- 172.21.15.60:0/998725899 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1484450437 0 0) 0x7f236c071ca0 con 0x7f236c0ce7a0 2024-04-25T10:09:50.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f236b7fe640 1 -- 172.21.15.60:0/998725899 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f235c003530 con 0x7f236c0ce7a0 2024-04-25T10:09:50.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f237142d640 1 -- 172.21.15.60:0/998725899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f236c15bc90 msgr2=0x7f236c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f237142d640 1 --2- 172.21.15.60:0/998725899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f236c15bc90 0x7f236c15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f237142d640 1 -- 172.21.15.60:0/998725899 >> v1:172.21.15.60:6789/0 conn(0x7f236c0ce7a0 legacy=0x7f236c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.109+0000 7f237142d640 1 -- 172.21.15.60:0/998725899 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c0ccd20 con 0x7f236c0b4660 2024-04-25T10:09:50.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.113+0000 7f237142d640 1 --2- 172.21.15.60:0/998725899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b4660 0x7f236c15c760 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2364009d90 tx=0x7f236401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127fb76f33c927b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.113+0000 7f236b7fe640 1 -- 172.21.15.60:0/998725899 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23640b2070 con 0x7f236c0b4660 2024-04-25T10:09:50.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.113+0000 7f236b7fe640 1 -- 172.21.15.60:0/998725899 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23640a5830 con 0x7f236c0b4660 2024-04-25T10:09:50.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.113+0000 7f236b7fe640 1 -- 172.21.15.60:0/998725899 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23640a5be0 con 0x7f236c0b4660 2024-04-25T10:09:50.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.113+0000 7f2372e90640 1 -- 172.21.15.60:0/998725899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b4660 msgr2=0x7f236c15c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.113+0000 7f2372e90640 1 --2- 172.21.15.60:0/998725899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b4660 0x7f236c15c760 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2364009d90 tx=0x7f236401c970 comp rx=0 tx=0).stop 2024-04-25T10:09:50.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.113+0000 7f2372e90640 1 -- 172.21.15.60:0/998725899 shutdown_connections 2024-04-25T10:09:50.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.113+0000 7f2372e90640 1 --2- 172.21.15.60:0/998725899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c0b4660 0x7f236c15c760 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.113+0000 7f2372e90640 1 --2- 172.21.15.60:0/998725899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f236c15bc90 0x7f236c15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.113+0000 7f2372e90640 1 -- 172.21.15.60:0/998725899 >> 172.21.15.60:0/998725899 conn(0x7f236c0b12d0 msgr2=0x7f236c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.113+0000 7f2372e90640 1 -- 172.21.15.60:0/998725899 shutdown_connections 2024-04-25T10:09:50.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.113+0000 7f2372e90640 1 -- 172.21.15.60:0/998725899 wait complete. 2024-04-25T10:09:50.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.113+0000 7f2372e90640 1 Processor -- start 2024-04-25T10:09:50.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.113+0000 7f2372e90640 1 -- start start 2024-04-25T10:09:50.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f2372e90640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f236c0ce7a0 0x7f236c1565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f2372e90640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c15bc90 0x7f236c157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f2372e90640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c153890 0x7f236c0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f2372e90640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c0b3db0 con 0x7f236c153890 2024-04-25T10:09:50.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f2372e90640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c0b4490 con 0x7f236c15bc90 2024-04-25T10:09:50.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f2372e90640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c1685c0 con 0x7f236c0ce7a0 2024-04-25T10:09:50.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f237142d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c153890 0x7f236c0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f237142d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c153890 0x7f236c0ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36736/0 (socket says 172.21.15.60:36736) 2024-04-25T10:09:50.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f237142d640 1 -- 172.21.15.60:0/2978045874 learned_addr learned my addr 172.21.15.60:0/2978045874 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:50.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f236bfff640 1 --2- 172.21.15.60:0/2978045874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c15bc90 0x7f236c157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f2370c2c640 1 --2- 172.21.15.60:0/2978045874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f236c0ce7a0 0x7f236c1565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f237142d640 1 -- 172.21.15.60:0/2978045874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f236c0ce7a0 msgr2=0x7f236c1565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f237142d640 1 --2- 172.21.15.60:0/2978045874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f236c0ce7a0 0x7f236c1565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f237142d640 1 -- 172.21.15.60:0/2978045874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c15bc90 msgr2=0x7f236c157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f237142d640 1 --2- 172.21.15.60:0/2978045874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c15bc90 0x7f236c157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f237142d640 1 -- 172.21.15.60:0/2978045874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c06d390 con 0x7f236c153890 2024-04-25T10:09:50.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f237142d640 1 --2- 172.21.15.60:0/2978045874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c153890 0x7f236c0ac8a0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f23640b0040 tx=0x7f2364019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f23697fa640 1 -- 172.21.15.60:0/2978045874 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23640b2050 con 0x7f236c153890 2024-04-25T10:09:50.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f23697fa640 1 -- 172.21.15.60:0/2978045874 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23640be070 con 0x7f236c153890 2024-04-25T10:09:50.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f2372e90640 1 -- 172.21.15.60:0/2978045874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c1f69b0 con 0x7f236c153890 2024-04-25T10:09:50.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f23697fa640 1 -- 172.21.15.60:0/2978045874 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364003db0 con 0x7f236c153890 2024-04-25T10:09:50.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f2372e90640 1 -- 172.21.15.60:0/2978045874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c158a00 con 0x7f236c153890 2024-04-25T10:09:50.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f23697fa640 1 -- 172.21.15.60:0/2978045874 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2364004040 con 0x7f236c153890 2024-04-25T10:09:50.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f23697fa640 1 --2- 172.21.15.60:0/2978045874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2350066f60 0x7f2350069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f23697fa640 1 -- 172.21.15.60:0/2978045874 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f236411ff70 con 0x7f236c153890 2024-04-25T10:09:50.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.117+0000 7f2370c2c640 1 --2- 172.21.15.60:0/2978045874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2350066f60 0x7f2350069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 Processor -- start 2024-04-25T10:09:50.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 -- start start 2024-04-25T10:09:50.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2370c2c640 1 --2- 172.21.15.60:0/2978045874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2350066f60 0x7f2350069420 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2360007fd0 tx=0x7f2360009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883334920 0x55f883334d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f883335240 0x55f8832ea9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8832eaf30 0x55f8832eb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8831d25f0 con 0x55f883334920 2024-04-25T10:09:50.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f883139da0 con 0x55f8832eaf30 2024-04-25T10:09:50.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8830c2150 con 0x55f883335240 2024-04-25T10:09:50.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2370c2c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883334920 0x55f883334d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2370c2c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883334920 0x55f883334d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36750/0 (socket says 172.21.15.60:36750) 2024-04-25T10:09:50.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2370c2c640 1 -- 172.21.15.60:0/3337680906 learned_addr learned my addr 172.21.15.60:0/3337680906 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:50.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f236bfff640 1 --2- 172.21.15.60:0/3337680906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f883335240 0x55f8832ea9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2370c2c640 1 -- 172.21.15.60:0/3337680906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f883335240 msgr2=0x55f8832ea9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2370c2c640 1 --2- 172.21.15.60:0/3337680906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f883335240 0x55f8832ea9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f237142d640 1 --2- 172.21.15.60:0/3337680906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8832eaf30 0x55f8832eb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2370c2c640 1 -- 172.21.15.60:0/3337680906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8832eaf30 msgr2=0x55f8832eb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2370c2c640 1 --2- 172.21.15.60:0/3337680906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8832eaf30 0x55f8832eb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2370c2c640 1 -- 172.21.15.60:0/3337680906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8830dbca0 con 0x55f883334920 2024-04-25T10:09:50.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f236bfff640 1 --2- 172.21.15.60:0/3337680906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f883335240 0x55f8832ea9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:50.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f237142d640 1 --2- 172.21.15.60:0/3337680906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8832eaf30 0x55f8832eb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:50.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2370c2c640 1 --2- 172.21.15.60:0/3337680906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883334920 0x55f883334d00 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2360005cc0 tx=0x7f2360010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56711928dccc94b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f23567fc640 1 -- 172.21.15.60:0/3337680906 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360007560 con 0x55f883334920 2024-04-25T10:09:50.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f23567fc640 1 -- 172.21.15.60:0/3337680906 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2360007700 con 0x55f883334920 2024-04-25T10:09:50.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f23567fc640 1 -- 172.21.15.60:0/3337680906 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360014bb0 con 0x55f883334920 2024-04-25T10:09:50.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 -- 172.21.15.60:0/3337680906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883334920 msgr2=0x55f883334d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 --2- 172.21.15.60:0/3337680906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883334920 0x55f883334d00 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2360005cc0 tx=0x7f2360010000 comp rx=0 tx=0).stop 2024-04-25T10:09:50.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 -- 172.21.15.60:0/3337680906 shutdown_connections 2024-04-25T10:09:50.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 --2- 172.21.15.60:0/3337680906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f8832eaf30 0x55f8832eb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 --2- 172.21.15.60:0/3337680906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f883335240 0x55f8832ea9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 --2- 172.21.15.60:0/3337680906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883334920 0x55f883334d00 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 -- 172.21.15.60:0/3337680906 >> 172.21.15.60:0/3337680906 conn(0x55f8832f12a0 msgr2=0x55f8832e92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 -- 172.21.15.60:0/3337680906 shutdown_connections 2024-04-25T10:09:50.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 -- 172.21.15.60:0/3337680906 wait complete. 2024-04-25T10:09:50.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 -- start start 2024-04-25T10:09:50.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.121+0000 7f2375ffd000 1 Processor -- start 2024-04-25T10:09:50.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8832eaf30 0x55f88333bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f883334920 0x55f88333c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f883335240 0x55f8832eeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8830f4990 con 0x55f8832eaf30 2024-04-25T10:09:50.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f883135e40 con 0x55f883335240 2024-04-25T10:09:50.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8830c5900 con 0x55f883334920 2024-04-25T10:09:50.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 10 client.0 _ll_register_callbacks cb 0x55f8831ed1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:50.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2370c2c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8832eaf30 0x55f88333bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2370c2c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8832eaf30 0x55f88333bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36764/0 (socket says 172.21.15.60:36764) 2024-04-25T10:09:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2370c2c640 1 -- 172.21.15.60:0/2256322136 learned_addr learned my addr 172.21.15.60:0/2256322136 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f237142d640 1 --2- 172.21.15.60:0/2256322136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f883335240 0x55f8832eeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f236bfff640 1 --2- 172.21.15.60:0/2256322136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f883334920 0x55f88333c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2370c2c640 1 -- 172.21.15.60:0/2256322136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f883334920 msgr2=0x55f88333c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2370c2c640 1 --2- 172.21.15.60:0/2256322136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f883334920 0x55f88333c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2370c2c640 1 -- 172.21.15.60:0/2256322136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f883335240 msgr2=0x55f8832eeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2370c2c640 1 --2- 172.21.15.60:0/2256322136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f883335240 0x55f8832eeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2370c2c640 1 -- 172.21.15.60:0/2256322136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8830f2630 con 0x55f8832eaf30 2024-04-25T10:09:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f237142d640 1 --2- 172.21.15.60:0/2256322136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f883335240 0x55f8832eeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2370c2c640 1 --2- 172.21.15.60:0/2256322136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8832eaf30 0x55f88333bdf0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f23600078d0 tx=0x7f2360000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94c0382f5c7d820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f234b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:09:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f234b7fe640 1 -- 172.21.15.60:0/2256322136 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236002f060 con 0x55f8832eaf30 2024-04-25T10:09:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f234b7fe640 1 -- 172.21.15.60:0/2256322136 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2360020450 con 0x55f8832eaf30 2024-04-25T10:09:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 -- 172.21.15.60:0/2256322136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2360007700 con 0x55f8832eaf30 2024-04-25T10:09:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f234b7fe640 1 -- 172.21.15.60:0/2256322136 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2360035050 con 0x55f8832eaf30 2024-04-25T10:09:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 10 client.7014 fetch_fsmap learned FSMap version 39 2024-04-25T10:09:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 -- 172.21.15.60:0/2256322136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55f8830c23a0 con 0x55f8832eaf30 2024-04-25T10:09:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f234b7fe640 1 -- 172.21.15.60:0/2256322136 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2360007700 con 0x55f8832eaf30 2024-04-25T10:09:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f234b7fe640 1 -- 172.21.15.60:0/2256322136 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2360038020 con 0x55f8832eaf30 2024-04-25T10:09:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 10 client.7014 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T10:09:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 10 client.7014 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:09:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 20 client.7014 populate_metadata read hostname 'smithi060' 2024-04-25T10:09:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 --2- 172.21.15.60:0/2256322136 >> [v2:172.21.15.60:6833/1289503429,v1:172.21.15.60:6835/1289503429] conn(0x55f883334380 0x55f8832ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 4 client.7014 mds_command: new command op to 6873 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 -- 172.21.15.60:0/2256322136 --> [v2:172.21.15.60:6833/1289503429,v1:172.21.15.60:6835/1289503429] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8830d5a50 con 0x55f883334380 2024-04-25T10:09:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f236bfff640 1 --2- 172.21.15.60:0/2256322136 >> [v2:172.21.15.60:6833/1289503429,v1:172.21.15.60:6835/1289503429] conn(0x55f883334380 0x55f8832ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f234b7fe640 10 client.7014 ms_handle_connect on v2:172.21.15.60:6833/1289503429 2024-04-25T10:09:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f236bfff640 1 --2- 172.21.15.60:0/2256322136 >> [v2:172.21.15.60:6833/1289503429,v1:172.21.15.60:6835/1289503429] conn(0x55f883334380 0x55f8832ef4b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=279eedd1c7baf6 server_cookie=4f9b62bb8671d4f4 in_seq=0 out_seq=0 2024-04-25T10:09:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f234b7fe640 1 -- 172.21.15.60:0/2256322136 <== mds.0 v2:172.21.15.60:6833/1289503429 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f8830d5a50 con 0x55f883334380 2024-04-25T10:09:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f234b7fe640 10 client.7014 handle_command_reply: tid=0 2024-04-25T10:09:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 client.7014 shutdown 2024-04-25T10:09:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 -- 172.21.15.60:0/2256322136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8832eaf30 msgr2=0x55f88333bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 --2- 172.21.15.60:0/2256322136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8832eaf30 0x55f88333bdf0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f23600078d0 tx=0x7f2360000bd0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 -- 172.21.15.60:0/2256322136 shutdown_connections 2024-04-25T10:09:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 --2- 172.21.15.60:0/2256322136 >> [v2:172.21.15.60:6833/1289503429,v1:172.21.15.60:6835/1289503429] conn(0x55f883334380 0x55f8832ef4b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 --2- 172.21.15.60:0/2256322136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f883335240 0x55f8832eeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 --2- 172.21.15.60:0/2256322136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f883334920 0x55f88333c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 --2- 172.21.15.60:0/2256322136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f8832eaf30 0x55f88333bdf0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 -- 172.21.15.60:0/2256322136 >> 172.21.15.60:0/2256322136 conn(0x55f8832f12a0 msgr2=0x55f8832e99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f234b7fe640 0 client.7014 ms_handle_reset on v2:172.21.15.60:6833/1289503429 2024-04-25T10:09:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 -- 172.21.15.60:0/2256322136 shutdown_connections 2024-04-25T10:09:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 1 -- 172.21.15.60:0/2256322136 wait complete. 2024-04-25T10:09:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.125+0000 7f2375ffd000 20 client.7014 trim_cache size 0 max 16384 2024-04-25T10:09:50.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.141+0000 7f2375ffd000 1 Processor -- start 2024-04-25T10:09:50.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.141+0000 7f2375ffd000 1 -- start start 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.141+0000 7f2375ffd000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f8832eaf30 0x55f8832ee330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.141+0000 7f2375ffd000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f883334920 0x55f8832ee870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.141+0000 7f2375ffd000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883335240 0x55f88333ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.141+0000 7f2375ffd000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c201dc0 con 0x55f883335240 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.141+0000 7f2375ffd000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c2012c0 con 0x55f883334920 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.141+0000 7f2375ffd000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c2017c0 con 0x55f8832eaf30 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.141+0000 7f237142d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883335240 0x55f88333ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.141+0000 7f237142d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883335240 0x55f88333ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36774/0 (socket says 172.21.15.60:36774) 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.141+0000 7f237142d640 1 -- 172.21.15.60:0/4277813975 learned_addr learned my addr 172.21.15.60:0/4277813975 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2370c2c640 1 --2- 172.21.15.60:0/4277813975 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f8832eaf30 0x55f8832ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f236bfff640 1 --2- 172.21.15.60:0/4277813975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f883334920 0x55f8832ee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f237142d640 1 -- 172.21.15.60:0/4277813975 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f8832eaf30 msgr2=0x55f8832ee330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f237142d640 1 --2- 172.21.15.60:0/4277813975 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f8832eaf30 0x55f8832ee330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f237142d640 1 -- 172.21.15.60:0/4277813975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f883334920 msgr2=0x55f8832ee870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f237142d640 1 --2- 172.21.15.60:0/4277813975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f883334920 0x55f8832ee870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f237142d640 1 -- 172.21.15.60:0/4277813975 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8831bb600 con 0x55f883335240 2024-04-25T10:09:50.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f237142d640 1 --2- 172.21.15.60:0/4277813975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883335240 0x55f88333ca10 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f236401ce50 tx=0x7f23640f0140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e847ae0a52cb2d49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2355ffb640 1 -- 172.21.15.60:0/4277813975 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23640f4930 con 0x55f883335240 2024-04-25T10:09:50.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2355ffb640 1 -- 172.21.15.60:0/4277813975 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23640b3040 con 0x55f883335240 2024-04-25T10:09:50.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2355ffb640 1 -- 172.21.15.60:0/4277813975 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23640f8ef0 con 0x55f883335240 2024-04-25T10:09:50.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 -- 172.21.15.60:0/4277813975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883335240 msgr2=0x55f88333ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 --2- 172.21.15.60:0/4277813975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883335240 0x55f88333ca10 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f236401ce50 tx=0x7f23640f0140 comp rx=0 tx=0).stop 2024-04-25T10:09:50.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 -- 172.21.15.60:0/4277813975 shutdown_connections 2024-04-25T10:09:50.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 --2- 172.21.15.60:0/4277813975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883335240 0x55f88333ca10 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 --2- 172.21.15.60:0/4277813975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f883334920 0x55f8832ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 --2- 172.21.15.60:0/4277813975 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f8832eaf30 0x55f8832ee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 -- 172.21.15.60:0/4277813975 >> 172.21.15.60:0/4277813975 conn(0x55f8832f12a0 msgr2=0x55f8832f03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 -- 172.21.15.60:0/4277813975 shutdown_connections 2024-04-25T10:09:50.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 -- 172.21.15.60:0/4277813975 wait complete. 2024-04-25T10:09:50.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 -- start start 2024-04-25T10:09:50.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 Processor -- start 2024-04-25T10:09:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f8832eaf30 0x55f88333ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883334920 0x55f883350510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f883335240 0x55f883350a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88313e3d0 con 0x55f883334920 2024-04-25T10:09:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8831f5d50 con 0x55f883335240 2024-04-25T10:09:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8830d4980 con 0x55f8832eaf30 2024-04-25T10:09:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 10 client.0 _ll_register_callbacks cb 0x7f2360001300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f236bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883334920 0x55f883350510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f236bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883334920 0x55f883350510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36784/0 (socket says 172.21.15.60:36784) 2024-04-25T10:09:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f236bfff640 1 -- 172.21.15.60:0/3871970889 learned_addr learned my addr 172.21.15.60:0/3871970889 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2370c2c640 1 --2- 172.21.15.60:0/3871970889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f8832eaf30 0x55f88333ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f237142d640 1 --2- 172.21.15.60:0/3871970889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f883335240 0x55f883350a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f236bfff640 1 -- 172.21.15.60:0/3871970889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f8832eaf30 msgr2=0x55f88333ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f236bfff640 1 --2- 172.21.15.60:0/3871970889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f8832eaf30 0x55f88333ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f236bfff640 1 -- 172.21.15.60:0/3871970889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f883335240 msgr2=0x55f883350a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f236bfff640 1 --2- 172.21.15.60:0/3871970889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f883335240 0x55f883350a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f236bfff640 1 -- 172.21.15.60:0/3871970889 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8830ce9b0 con 0x55f883334920 2024-04-25T10:09:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f236bfff640 1 --2- 172.21.15.60:0/3871970889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883334920 0x55f883350510 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x55f883140290 tx=0x7f2358004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de782d21cdde8ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f234b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:09:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f234b7fe640 1 -- 172.21.15.60:0/3871970889 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358027060 con 0x55f883334920 2024-04-25T10:09:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f234b7fe640 1 -- 172.21.15.60:0/3871970889 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f235800de40 con 0x55f883334920 2024-04-25T10:09:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 -- 172.21.15.60:0/3871970889 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23640b3040 con 0x55f883334920 2024-04-25T10:09:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f234b7fe640 1 -- 172.21.15.60:0/3871970889 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f235802d050 con 0x55f883334920 2024-04-25T10:09:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 10 client.7020 fetch_fsmap learned FSMap version 39 2024-04-25T10:09:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.145+0000 7f2375ffd000 1 -- 172.21.15.60:0/3871970889 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55f8830e10b0 con 0x55f883334920 2024-04-25T10:09:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f234b7fe640 1 -- 172.21.15.60:0/3871970889 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23640b3040 con 0x55f883334920 2024-04-25T10:09:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f234b7fe640 1 -- 172.21.15.60:0/3871970889 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 39) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2358030020 con 0x55f883334920 2024-04-25T10:09:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 10 client.7020 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T10:09:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 10 client.7020 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:09:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 20 client.7020 populate_metadata read hostname 'smithi060' 2024-04-25T10:09:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 1 --2- 172.21.15.60:0/3871970889 >> [v2:172.21.15.60:6833/1289503429,v1:172.21.15.60:6835/1289503429] conn(0x55f883350f90 0x55f883395140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 4 client.7020 mds_command: new command op to 6873 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:09:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 1 -- 172.21.15.60:0/3871970889 --> [v2:172.21.15.60:6833/1289503429,v1:172.21.15.60:6835/1289503429] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f8831472e0 con 0x55f883350f90 2024-04-25T10:09:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2370c2c640 1 --2- 172.21.15.60:0/3871970889 >> [v2:172.21.15.60:6833/1289503429,v1:172.21.15.60:6835/1289503429] conn(0x55f883350f90 0x55f883395140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f234b7fe640 10 client.7020 ms_handle_connect on v2:172.21.15.60:6833/1289503429 2024-04-25T10:09:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2370c2c640 1 --2- 172.21.15.60:0/3871970889 >> [v2:172.21.15.60:6833/1289503429,v1:172.21.15.60:6835/1289503429] conn(0x55f883350f90 0x55f883395140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=955198f5eade7dd1 server_cookie=42ab4a54ec8e04e8 in_seq=0 out_seq=0 2024-04-25T10:09:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f234b7fe640 1 -- 172.21.15.60:0/3871970889 <== mds.0 v2:172.21.15.60:6833/1289503429 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55f8831472e0 con 0x55f883350f90 2024-04-25T10:09:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f234b7fe640 10 client.7020 handle_command_reply: tid=0 2024-04-25T10:09:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 1 client.7020 shutdown 2024-04-25T10:09:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 1 -- 172.21.15.60:0/3871970889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883334920 msgr2=0x55f883350510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 1 --2- 172.21.15.60:0/3871970889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883334920 0x55f883350510 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x55f883140290 tx=0x7f2358004410 comp rx=0 tx=0).stop 2024-04-25T10:09:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 1 -- 172.21.15.60:0/3871970889 shutdown_connections 2024-04-25T10:09:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 1 --2- 172.21.15.60:0/3871970889 >> [v2:172.21.15.60:6833/1289503429,v1:172.21.15.60:6835/1289503429] conn(0x55f883350f90 0x55f883395140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 1 --2- 172.21.15.60:0/3871970889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f883335240 0x55f883350a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f234b7fe640 0 client.7020 ms_handle_reset on v2:172.21.15.60:6833/1289503429 2024-04-25T10:09:50.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 1 --2- 172.21.15.60:0/3871970889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f883334920 0x55f883350510 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 1 --2- 172.21.15.60:0/3871970889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f8832eaf30 0x55f88333ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 1 -- 172.21.15.60:0/3871970889 >> 172.21.15.60:0/3871970889 conn(0x55f8832f12a0 msgr2=0x55f8832ed320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 1 -- 172.21.15.60:0/3871970889 shutdown_connections 2024-04-25T10:09:50.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 1 -- 172.21.15.60:0/3871970889 wait complete. 2024-04-25T10:09:50.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2375ffd000 20 client.7020 trim_cache size 0 max 16384 2024-04-25T10:09:50.164 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:09:50.164 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "0ea93863-a079-45c4-abed-6e83540ea9fe", 2024-04-25T10:09:50.164 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T10:09:50.165 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6873, 2024-04-25T10:09:50.165 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T10:09:50.165 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T10:09:50.165 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T10:09:50.165 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 5.7968878190000002, 2024-04-25T10:09:50.165 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 39, 2024-04-25T10:09:50.165 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 69, 2024-04-25T10:09:50.165 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 69, 2024-04-25T10:09:50.165 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 10.808985587 2024-04-25T10:09:50.165 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:09:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2372e90640 1 -- 172.21.15.60:0/2978045874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2350066f60 msgr2=0x7f2350069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2372e90640 1 --2- 172.21.15.60:0/2978045874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2350066f60 0x7f2350069420 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2360007fd0 tx=0x7f2360009040 comp rx=0 tx=0).stop 2024-04-25T10:09:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2372e90640 1 -- 172.21.15.60:0/2978045874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c153890 msgr2=0x7f236c0ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2372e90640 1 --2- 172.21.15.60:0/2978045874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c153890 0x7f236c0ac8a0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f23640b0040 tx=0x7f2364019710 comp rx=0 tx=0).stop 2024-04-25T10:09:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2372e90640 1 -- 172.21.15.60:0/2978045874 shutdown_connections 2024-04-25T10:09:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2372e90640 1 --2- 172.21.15.60:0/2978045874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2350066f60 0x7f2350069420 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2372e90640 1 --2- 172.21.15.60:0/2978045874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f236c153890 0x7f236c0ac8a0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2372e90640 1 --2- 172.21.15.60:0/2978045874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c15bc90 0x7f236c157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2372e90640 1 --2- 172.21.15.60:0/2978045874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f236c0ce7a0 0x7f236c1565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2372e90640 1 -- 172.21.15.60:0/2978045874 >> 172.21.15.60:0/2978045874 conn(0x7f236c0b12d0 msgr2=0x7f236c0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2372e90640 1 -- 172.21.15.60:0/2978045874 shutdown_connections 2024-04-25T10:09:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.149+0000 7f2372e90640 1 -- 172.21.15.60:0/2978045874 wait complete. 2024-04-25T10:09:50.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:09:50.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.273+0000 7f0c01819640 1 Processor -- start 2024-04-25T10:09:50.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.273+0000 7f0c01819640 1 -- start start 2024-04-25T10:09:50.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.273+0000 7f0c01819640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0bfc15deb0 0x7f0bfc0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.273+0000 7f0c01819640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bfc0ce9c0 0x7f0bfc0d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.273+0000 7f0c01819640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bfc06d200 con 0x7f0bfc0d0340 2024-04-25T10:09:50.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.273+0000 7f0c01819640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bfc0cf2c0 con 0x7f0bfc15deb0 2024-04-25T10:09:50.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.273+0000 7f0c01819640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc0cefe0 con 0x7f0bfc0ce9c0 2024-04-25T10:09:50.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.273+0000 7f0bfaffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0bfc15deb0 0x7f0bfc0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.273+0000 7f0bfaffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0bfc15deb0 0x7f0bfc0b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36794/0 (socket says 172.21.15.60:36794) 2024-04-25T10:09:50.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.273+0000 7f0bfaffd640 1 -- 172.21.15.60:0/449526866 learned_addr learned my addr 172.21.15.60:0/449526866 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:50.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.273+0000 7f0bfb7fe640 1 --2- 172.21.15.60:0/449526866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bfc0ce9c0 0x7f0bfc0d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.273+0000 7f0bfaffd640 1 -- 172.21.15.60:0/449526866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bfc0ce9c0 msgr2=0x7f0bfc0d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.273+0000 7f0bfaffd640 1 --2- 172.21.15.60:0/449526866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bfc0ce9c0 0x7f0bfc0d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.273+0000 7f0bfaffd640 1 -- 172.21.15.60:0/449526866 >> v1:172.21.15.60:6789/0 conn(0x7f0bfc0d0340 legacy=0x7f0bfc0ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.273+0000 7f0bfaffd640 1 -- 172.21.15.60:0/449526866 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc0d1d70 con 0x7f0bfc15deb0 2024-04-25T10:09:50.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0bfaffd640 1 --2- 172.21.15.60:0/449526866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc15deb0 0x7f0bfc0b6c10 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009870 tx=0x7f0bf401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0939f7d3581be1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0bf9ffb640 1 -- 172.21.15.60:0/449526866 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40b2070 con 0x7f0bfc15deb0 2024-04-25T10:09:50.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0bf9ffb640 1 -- 172.21.15.60:0/449526866 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf40a9070 con 0x7f0bfc15deb0 2024-04-25T10:09:50.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0bf9ffb640 1 -- 172.21.15.60:0/449526866 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40ad880 con 0x7f0bfc15deb0 2024-04-25T10:09:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0c01819640 1 -- 172.21.15.60:0/449526866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc15deb0 msgr2=0x7f0bfc0b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0c01819640 1 --2- 172.21.15.60:0/449526866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc15deb0 0x7f0bfc0b6c10 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009870 tx=0x7f0bf401c560 comp rx=0 tx=0).stop 2024-04-25T10:09:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0c01819640 1 -- 172.21.15.60:0/449526866 shutdown_connections 2024-04-25T10:09:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0c01819640 1 --2- 172.21.15.60:0/449526866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bfc0ce9c0 0x7f0bfc0d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0c01819640 1 --2- 172.21.15.60:0/449526866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc15deb0 0x7f0bfc0b6c10 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0c01819640 1 -- 172.21.15.60:0/449526866 >> 172.21.15.60:0/449526866 conn(0x7f0bfc06c090 msgr2=0x7f0bfc06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0c01819640 1 -- 172.21.15.60:0/449526866 shutdown_connections 2024-04-25T10:09:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0c01819640 1 -- 172.21.15.60:0/449526866 wait complete. 2024-04-25T10:09:50.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0c01819640 1 Processor -- start 2024-04-25T10:09:50.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0c01819640 1 -- start start 2024-04-25T10:09:50.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0c01819640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc0ce9c0 0x7f0bfc14e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0c01819640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bfc0d0340 0x7f0bfc15c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0c01819640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc15deb0 0x7f0bfc0c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0c01819640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc0d3470 con 0x7f0bfc0ce9c0 2024-04-25T10:09:50.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0c01819640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc0d32f0 con 0x7f0bfc15deb0 2024-04-25T10:09:50.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0c01819640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc0d35f0 con 0x7f0bfc0d0340 2024-04-25T10:09:50.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0bfaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc0ce9c0 0x7f0bfc14e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0bfaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc0ce9c0 0x7f0bfc14e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36806/0 (socket says 172.21.15.60:36806) 2024-04-25T10:09:50.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0bfaffd640 1 -- 172.21.15.60:0/3953042848 learned_addr learned my addr 172.21.15.60:0/3953042848 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:50.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0bfaffd640 1 -- 172.21.15.60:0/3953042848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bfc0d0340 msgr2=0x7f0bfc15c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0bfaffd640 1 --2- 172.21.15.60:0/3953042848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bfc0d0340 0x7f0bfc15c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0bfaffd640 1 -- 172.21.15.60:0/3953042848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc15deb0 msgr2=0x7f0bfc0c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0bfaffd640 1 --2- 172.21.15.60:0/3953042848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc15deb0 0x7f0bfc0c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0bfaffd640 1 -- 172.21.15.60:0/3953042848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc15ce70 con 0x7f0bfc0ce9c0 2024-04-25T10:09:50.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.277+0000 7f0bfaffd640 1 --2- 172.21.15.60:0/3953042848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc0ce9c0 0x7f0bfc14e0b0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40b0040 tx=0x7f0bf401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.281+0000 7f0be3fff640 1 -- 172.21.15.60:0/3953042848 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40b2040 con 0x7f0bfc0ce9c0 2024-04-25T10:09:50.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.281+0000 7f0be3fff640 1 -- 172.21.15.60:0/3953042848 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf40adbf0 con 0x7f0bfc0ce9c0 2024-04-25T10:09:50.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.281+0000 7f0c01819640 1 -- 172.21.15.60:0/3953042848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bfc15d140 con 0x7f0bfc0ce9c0 2024-04-25T10:09:50.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.281+0000 7f0be3fff640 1 -- 172.21.15.60:0/3953042848 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40b8420 con 0x7f0bfc0ce9c0 2024-04-25T10:09:50.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.281+0000 7f0c01819640 1 -- 172.21.15.60:0/3953042848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bfc0cc450 con 0x7f0bfc0ce9c0 2024-04-25T10:09:50.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.281+0000 7f0c01819640 1 -- 172.21.15.60:0/3953042848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bfc074770 con 0x7f0bfc0ce9c0 2024-04-25T10:09:50.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.281+0000 7f0be3fff640 1 -- 172.21.15.60:0/3953042848 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0bf40c0030 con 0x7f0bfc0ce9c0 2024-04-25T10:09:50.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.281+0000 7f0be3fff640 1 --2- 172.21.15.60:0/3953042848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0be8066f60 0x7f0be8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.281+0000 7f0be3fff640 1 -- 172.21.15.60:0/3953042848 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bf411eb20 con 0x7f0bfc0ce9c0 2024-04-25T10:09:50.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.281+0000 7f0bfa7fc640 1 --2- 172.21.15.60:0/3953042848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0be8066f60 0x7f0be8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.281+0000 7f0bfa7fc640 1 --2- 172.21.15.60:0/3953042848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0be8066f60 0x7f0be8069420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0007fd0 tx=0x7f0bf0007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.285+0000 7f0be3fff640 1 -- 172.21.15.60:0/3953042848 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0bf40c7020 con 0x7f0bfc0ce9c0 2024-04-25T10:09:50.314 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-04-25T10:09:50.305+0000 7ff101bd0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:09:50.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.477+0000 7f0c01819640 1 -- 172.21.15.60:0/3953042848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0bfc0d17a0 con 0x7f0bfc0ce9c0 2024-04-25T10:09:50.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.477+0000 7f0be3fff640 1 -- 172.21.15.60:0/3953042848 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0bf4016150 con 0x7f0bfc0ce9c0 2024-04-25T10:09:50.486 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:09:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.481+0000 7f0c01819640 1 -- 172.21.15.60:0/3953042848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0be8066f60 msgr2=0x7f0be8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.481+0000 7f0c01819640 1 --2- 172.21.15.60:0/3953042848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0be8066f60 0x7f0be8069420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0007fd0 tx=0x7f0bf0007d40 comp rx=0 tx=0).stop 2024-04-25T10:09:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.481+0000 7f0c01819640 1 -- 172.21.15.60:0/3953042848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc0ce9c0 msgr2=0x7f0bfc14e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.481+0000 7f0c01819640 1 --2- 172.21.15.60:0/3953042848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc0ce9c0 0x7f0bfc14e0b0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40b0040 tx=0x7f0bf401ada0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.481+0000 7f0c01819640 1 -- 172.21.15.60:0/3953042848 shutdown_connections 2024-04-25T10:09:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.481+0000 7f0c01819640 1 --2- 172.21.15.60:0/3953042848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0be8066f60 0x7f0be8069420 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.481+0000 7f0c01819640 1 --2- 172.21.15.60:0/3953042848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc15deb0 0x7f0bfc0c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.481+0000 7f0c01819640 1 --2- 172.21.15.60:0/3953042848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bfc0d0340 0x7f0bfc15c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.481+0000 7f0c01819640 1 --2- 172.21.15.60:0/3953042848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc0ce9c0 0x7f0bfc14e0b0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.481+0000 7f0c01819640 1 -- 172.21.15.60:0/3953042848 >> 172.21.15.60:0/3953042848 conn(0x7f0bfc06c090 msgr2=0x7f0bfc15e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.481+0000 7f0c01819640 1 -- 172.21.15.60:0/3953042848 shutdown_connections 2024-04-25T10:09:50.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:50.481+0000 7f0c01819640 1 -- 172.21.15.60:0/3953042848 wait complete. 2024-04-25T10:09:50.504 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:09:50.504 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:09:50.504 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:09:50.504 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:09:50.504 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:50.504 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:09:50.504 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:09:50.505 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:09:50.505 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:50.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:50.505 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:09:50.511 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:09:50.511 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:09:50.511 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:09:50.512 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:50.512 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:09:50.512 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:50.512 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:09:50.512 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:50.512 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:09:50.512 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:09:50.512 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:50.512 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:09:50.512 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:09:50.512 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:50.512 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:09:50.512 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:09:50.512 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42266sec preferred_lft 42266sec 2024-04-25T10:09:50.513 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:09:50.513 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:50.513 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:09:50.513 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:50.513 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:09:50.513 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:09:50.513 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:50.513 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:09:50.513 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:50.513 INFO:teuthology.orchestra.run.smithi060.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:09:50.513 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:50.513 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:09:50.513 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:50.513 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:09:50.513 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:50.514 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:09:50.514 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:50.514 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:09:50.514 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:50.514 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:09:50.514 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:50.514 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:09:50.514 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:50.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:50.514 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:09:50.565 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:09:50.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:50.566 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:09:50.614 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:09:50.614 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:09:50.614 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:09:50.614 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:09:50.614 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:09:50.614 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:09:50.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:50.615 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:09:50.615 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:09:50.615 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:09:50.615 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:09:50.615 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:09:50.615 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:09:50.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:50.689 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:09:50.737 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:09:50.737 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:09:50.738 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:50.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:50.738 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T10:09:50.794 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:09:50.794 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:09:50.794 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:50.841 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:50.841 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:09:50.842 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:50.889 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:09:50.947 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:09:50.999 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:09:50.999 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:09:50.999 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:09:50.999 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:09:50.999 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:09:50.999 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:09:51.000 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:09:51.000 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:09:51.000 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:09:51.000 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:09:51.000 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:09:51.000 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:09:51.000 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:09:51.000 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:09:51.000 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:09:51.000 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:09:51.000 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:09:51.000 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:09:51.000 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:09:51.000 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:09:51.000 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:09:51.001 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:09:51.001 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:09:51.001 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:09:51.001 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:09:51.001 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:09:51.001 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:09:51.001 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:09:51.001 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:09:51.001 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:51.001 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:51.001 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:51.001 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:51.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:51.002 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:09:51.053 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:09:51.053 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:09:51.098 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:09:51.112 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:09:51.161 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:09:51.161 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:09:51.161 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:09:51.162 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:09:51.162 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:09:51.162 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:09:51.162 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:09:51.162 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:09:51.162 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:09:51.163 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:09:51.163 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:09:51.163 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:09:51.163 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:09:51.163 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:09:51.163 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:09:51.164 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:09:51.164 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:09:51.164 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:09:51.164 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:09:51.164 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:09:51.164 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:09:51.164 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:09:51.165 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:09:51.165 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:09:51.165 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:09:51.165 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:09:51.165 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:09:51.165 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:09:51.166 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:09:51.166 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:51.166 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:51.166 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:51.166 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:51.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:51.167 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:09:51.182 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2024-04-25T10:09:51.173+0000 7f00b6e5e580 -1 init, newargv = 0x55a5f4d9ff50 newargc=15 2024-04-25T10:09:51.182 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[42402]: starting ceph client 2024-04-25T10:09:51.187 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[42402]: starting fuse 2024-04-25T10:09:52.289 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:09:52.306 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:09:52.355 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:09:52.355 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:09:52.355 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:09:52.356 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:09:52.356 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:09:52.356 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:09:52.356 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:09:52.356 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:09:52.356 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:09:52.356 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:09:52.356 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:09:52.356 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:09:52.356 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:09:52.356 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:09:52.356 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:09:52.356 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:09:52.357 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:09:52.357 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:09:52.357 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:09:52.357 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:09:52.357 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:09:52.357 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:09:52.357 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:09:52.357 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:09:52.357 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:09:52.357 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:09:52.357 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:09:52.357 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:09:52.357 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:09:52.357 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:52.358 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:52.358 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:52.358 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:52.358 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:52.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:52.359 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:09:52.409 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:09:52.409 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:09:52.410 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:09:52.410 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:09:52.410 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:09:52.410 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:09:52.410 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:09:52.410 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:09:52.410 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:09:52.410 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:09:52.410 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:09:52.410 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:09:52.410 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:09:52.410 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:09:52.410 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:09:52.410 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:09:52.411 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:09:52.411 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:09:52.411 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:09:52.411 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:09:52.411 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:09:52.411 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:09:52.411 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:09:52.411 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:09:52.411 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:09:52.411 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:09:52.411 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:09:52.411 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2024-04-25T10:09:52.411 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:09:52.537 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.42402.asok 2024-04-25T10:09:52.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:09:52.551 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42402.asok 2024-04-25T10:09:52.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:52.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42402.asok status 2024-04-25T10:09:52.703 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:09:52.703 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:09:52.703 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:09:52.703 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "42402", 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7038, 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 7038 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2814652439 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2814652439 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.7038 192.168.0.1:0/2814652439", 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/2814652439", 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 39, 2024-04-25T10:09:52.706 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 69, 2024-04-25T10:09:52.706 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:09:52.706 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:09:52.706 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:09:52.706 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:09:52.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:52.718 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:09:52.718 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:52.725 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:09:52.725 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:52.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:52.725 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:52.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:52.780 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-25T10:09:52.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:52.843 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-25T10:09:52.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:52.896 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-25T10:09:52.953 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:09:52.954 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:09:52.954 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:09:52.954 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:09:52.954 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-25T10:09:52.954 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:09:53.069 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627776 2024-04-25T10:09:53.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:09:53.080 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:09:53.080 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:53.130 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.297+0000 7feeacaff640 1 Processor -- start 2024-04-25T10:09:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.297+0000 7feeacaff640 1 -- start start 2024-04-25T10:09:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.297+0000 7feeacaff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feea80b6c10 0x7feea80ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.297+0000 7feeacaff640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea806d200 con 0x7feea815deb0 2024-04-25T10:09:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.297+0000 7feeacaff640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea80cefe0 con 0x7feea80d0a50 2024-04-25T10:09:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.297+0000 7feeacaff640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea80cf2c0 con 0x7feea80b6c10 2024-04-25T10:09:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.297+0000 7feea6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feea80b6c10 0x7feea80ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.297+0000 7feea6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feea80b6c10 0x7feea80ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36818/0 (socket says 172.21.15.60:36818) 2024-04-25T10:09:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.297+0000 7feea659c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7feea815deb0 0x7feea80d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44608/0 (socket says 172.21.15.60:44608) 2024-04-25T10:09:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.297+0000 7feea6d9d640 1 -- 172.21.15.60:0/2999954302 learned_addr learned my addr 172.21.15.60:0/2999954302 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.297+0000 7feea6d9d640 1 -- 172.21.15.60:0/2999954302 >> v1:172.21.15.100:6789/0 conn(0x7feea80d0a50 legacy=0x7feea80ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.297+0000 7feea6d9d640 1 -- 172.21.15.60:0/2999954302 >> v1:172.21.15.60:6789/0 conn(0x7feea815deb0 legacy=0x7feea80d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.297+0000 7feea6d9d640 1 -- 172.21.15.60:0/2999954302 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea80d1d70 con 0x7feea80b6c10 2024-04-25T10:09:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.301+0000 7feea6d9d640 1 --2- 172.21.15.60:0/2999954302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feea80b6c10 0x7feea80ce9c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7feea0009e50 tx=0x7feea001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9582355e58f52db8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.301+0000 7feea559a640 1 -- 172.21.15.60:0/2999954302 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea00a55f0 con 0x7feea80b6c10 2024-04-25T10:09:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.301+0000 7feea559a640 1 -- 172.21.15.60:0/2999954302 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea00a5790 con 0x7feea80b6c10 2024-04-25T10:09:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.301+0000 7feea559a640 1 -- 172.21.15.60:0/2999954302 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea00a5b60 con 0x7feea80b6c10 2024-04-25T10:09:53.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.301+0000 7feeacaff640 1 -- 172.21.15.60:0/2999954302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feea80b6c10 msgr2=0x7feea80ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.301+0000 7feeacaff640 1 --2- 172.21.15.60:0/2999954302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feea80b6c10 0x7feea80ce9c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7feea0009e50 tx=0x7feea001c880 comp rx=0 tx=0).stop 2024-04-25T10:09:53.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.301+0000 7feeacaff640 1 -- 172.21.15.60:0/2999954302 shutdown_connections 2024-04-25T10:09:53.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.301+0000 7feeacaff640 1 --2- 172.21.15.60:0/2999954302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feea80b6c10 0x7feea80ce9c0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.301+0000 7feeacaff640 1 -- 172.21.15.60:0/2999954302 >> 172.21.15.60:0/2999954302 conn(0x7feea806c090 msgr2=0x7feea806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:53.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.301+0000 7feeacaff640 1 -- 172.21.15.60:0/2999954302 shutdown_connections 2024-04-25T10:09:53.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.301+0000 7feeacaff640 1 -- 172.21.15.60:0/2999954302 wait complete. 2024-04-25T10:09:53.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.301+0000 7feeacaff640 1 Processor -- start 2024-04-25T10:09:53.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.301+0000 7feeacaff640 1 -- start start 2024-04-25T10:09:53.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feeacaff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feea80b6c10 0x7feea80c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feeacaff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feea80d0a50 0x7feea81480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feeacaff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea815deb0 0x7feea8148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feeacaff640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea816b7d0 con 0x7feea80d0a50 2024-04-25T10:09:53.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feeacaff640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea816b650 con 0x7feea815deb0 2024-04-25T10:09:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feeacaff640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea816b950 con 0x7feea80b6c10 2024-04-25T10:09:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feea5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feea80d0a50 0x7feea81480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feea5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feea80d0a50 0x7feea81480c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36830/0 (socket says 172.21.15.60:36830) 2024-04-25T10:09:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feea5d9b640 1 -- 172.21.15.60:0/1007758877 learned_addr learned my addr 172.21.15.60:0/1007758877 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feea6d9d640 1 --2- 172.21.15.60:0/1007758877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea815deb0 0x7feea8148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feea659c640 1 --2- 172.21.15.60:0/1007758877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feea80b6c10 0x7feea80c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feea5d9b640 1 -- 172.21.15.60:0/1007758877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feea80b6c10 msgr2=0x7feea80c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feea5d9b640 1 --2- 172.21.15.60:0/1007758877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feea80b6c10 0x7feea80c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feea5d9b640 1 -- 172.21.15.60:0/1007758877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea815deb0 msgr2=0x7feea8148600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feea5d9b640 1 --2- 172.21.15.60:0/1007758877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea815deb0 0x7feea8148600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feea5d9b640 1 -- 172.21.15.60:0/1007758877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea80c8000 con 0x7feea80d0a50 2024-04-25T10:09:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feea5d9b640 1 --2- 172.21.15.60:0/1007758877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feea80d0a50 0x7feea81480c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fee9800c450 tx=0x7fee9800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7fee9effd640 1 -- 172.21.15.60:0/1007758877 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98012830 con 0x7feea80d0a50 2024-04-25T10:09:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7fee9effd640 1 -- 172.21.15.60:0/1007758877 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee98017070 con 0x7feea80d0a50 2024-04-25T10:09:53.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feeacaff640 1 -- 172.21.15.60:0/1007758877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea814a4c0 con 0x7feea80d0a50 2024-04-25T10:09:53.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7fee9effd640 1 -- 172.21.15.60:0/1007758877 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98012c30 con 0x7feea80d0a50 2024-04-25T10:09:53.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feeacaff640 1 -- 172.21.15.60:0/1007758877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea814aa80 con 0x7feea80d0a50 2024-04-25T10:09:53.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feeacaff640 1 -- 172.21.15.60:0/1007758877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea8074770 con 0x7feea80d0a50 2024-04-25T10:09:53.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7fee9effd640 1 -- 172.21.15.60:0/1007758877 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee9801c460 con 0x7feea80d0a50 2024-04-25T10:09:53.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7fee9effd640 1 --2- 172.21.15.60:0/1007758877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fee88066f60 0x7fee88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7fee9effd640 1 -- 172.21.15.60:0/1007758877 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee98085930 con 0x7feea80d0a50 2024-04-25T10:09:53.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.305+0000 7feea659c640 1 --2- 172.21.15.60:0/1007758877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fee88066f60 0x7fee88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.309+0000 7feea659c640 1 --2- 172.21.15.60:0/1007758877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fee88066f60 0x7fee88069420 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fee90000f70 tx=0x7fee90007c70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.309+0000 7fee9effd640 1 -- 172.21.15.60:0/1007758877 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fee98056920 con 0x7feea80d0a50 2024-04-25T10:09:53.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.473+0000 7feeacaff640 1 -- 172.21.15.60:0/1007758877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feea80d17a0 con 0x7feea80d0a50 2024-04-25T10:09:53.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.473+0000 7fee9effd640 1 -- 172.21.15.60:0/1007758877 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16680 (secure 0 0 0) 0x7fee98056220 con 0x7feea80d0a50 2024-04-25T10:09:53.484 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:53.484 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":69,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:09:45.353654+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:42.005281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:42.710028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:53.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.477+0000 7feeacaff640 1 -- 172.21.15.60:0/1007758877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fee88066f60 msgr2=0x7fee88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.477+0000 7feeacaff640 1 --2- 172.21.15.60:0/1007758877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fee88066f60 0x7fee88069420 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fee90000f70 tx=0x7fee90007c70 comp rx=0 tx=0).stop 2024-04-25T10:09:53.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.477+0000 7feeacaff640 1 -- 172.21.15.60:0/1007758877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feea80d0a50 msgr2=0x7feea81480c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.477+0000 7feeacaff640 1 --2- 172.21.15.60:0/1007758877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feea80d0a50 0x7feea81480c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fee9800c450 tx=0x7fee9800c910 comp rx=0 tx=0).stop 2024-04-25T10:09:53.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.477+0000 7feeacaff640 1 -- 172.21.15.60:0/1007758877 shutdown_connections 2024-04-25T10:09:53.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.477+0000 7feeacaff640 1 --2- 172.21.15.60:0/1007758877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fee88066f60 0x7fee88069420 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.477+0000 7feeacaff640 1 --2- 172.21.15.60:0/1007758877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feea815deb0 0x7feea8148600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.477+0000 7feeacaff640 1 --2- 172.21.15.60:0/1007758877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feea80d0a50 0x7feea81480c0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.477+0000 7feeacaff640 1 --2- 172.21.15.60:0/1007758877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feea80b6c10 0x7feea80c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.477+0000 7feeacaff640 1 -- 172.21.15.60:0/1007758877 >> 172.21.15.60:0/1007758877 conn(0x7feea806c090 msgr2=0x7feea80b8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:53.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.477+0000 7feeacaff640 1 -- 172.21.15.60:0/1007758877 shutdown_connections 2024-04-25T10:09:53.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.477+0000 7feeacaff640 1 -- 172.21.15.60:0/1007758877 wait complete. 2024-04-25T10:09:53.503 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:09:42.005281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:09:42.710028+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:53.503 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:53.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fac00ce6640 1 Processor -- start 2024-04-25T10:09:53.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fac00ce6640 1 -- start start 2024-04-25T10:09:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fac00ce6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fabfc0b4b30 0x7fabfc15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fac00ce6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabfc0b3fa0 0x7fabfc15ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fac00ce6640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabfc15b860 con 0x7fabfc06bc20 2024-04-25T10:09:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fac00ce6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabfc15bb40 con 0x7fabfc0b4b30 2024-04-25T10:09:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fac00ce6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc15e670 con 0x7fabfc0b3fa0 2024-04-25T10:09:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fabfa59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fabfc0b4b30 0x7fabfc15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fabfad9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fabfc06bc20 0x7fabfc0ae750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44620/0 (socket says 172.21.15.60:44620) 2024-04-25T10:09:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fabfa59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fabfc0b4b30 0x7fabfc15afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36832/0 (socket says 172.21.15.60:36832) 2024-04-25T10:09:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fabfad9d640 1 -- 172.21.15.60:0/4240000550 learned_addr learned my addr 172.21.15.60:0/4240000550 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fabf3fff640 1 --2- 172.21.15.60:0/4240000550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabfc0b3fa0 0x7fabfc15ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fabfa59c640 1 -- 172.21.15.60:0/4240000550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabfc0b3fa0 msgr2=0x7fabfc15ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fabfa59c640 1 --2- 172.21.15.60:0/4240000550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabfc0b3fa0 0x7fabfc15ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fabfa59c640 1 -- 172.21.15.60:0/4240000550 >> v1:172.21.15.60:6789/0 conn(0x7fabfc06bc20 legacy=0x7fabfc0ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fabf9d9b640 1 -- 172.21.15.60:0/4240000550 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780169607 0 0) 0x7fabfc15b860 con 0x7fabfc06bc20 2024-04-25T10:09:53.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.621+0000 7fabfa59c640 1 -- 172.21.15.60:0/4240000550 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc15be20 con 0x7fabfc0b4b30 2024-04-25T10:09:53.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fabfa59c640 1 --2- 172.21.15.60:0/4240000550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabfc0b4b30 0x7fabfc15afe0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fabec0097c0 tx=0x7fabec01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7979da3c77e7cfd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fabf9d9b640 1 -- 172.21.15.60:0/4240000550 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec0b0070 con 0x7fabfc0b4b30 2024-04-25T10:09:53.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fabf9d9b640 1 -- 172.21.15.60:0/4240000550 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec0ab600 con 0x7fabfc0b4b30 2024-04-25T10:09:53.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fabf9d9b640 1 -- 172.21.15.60:0/4240000550 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec0ab9d0 con 0x7fabfc0b4b30 2024-04-25T10:09:53.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fac00ce6640 1 -- 172.21.15.60:0/4240000550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabfc0b4b30 msgr2=0x7fabfc15afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fac00ce6640 1 --2- 172.21.15.60:0/4240000550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabfc0b4b30 0x7fabfc15afe0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fabec0097c0 tx=0x7fabec01c380 comp rx=0 tx=0).stop 2024-04-25T10:09:53.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fac00ce6640 1 -- 172.21.15.60:0/4240000550 shutdown_connections 2024-04-25T10:09:53.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fac00ce6640 1 --2- 172.21.15.60:0/4240000550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabfc0b3fa0 0x7fabfc15ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fac00ce6640 1 --2- 172.21.15.60:0/4240000550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabfc0b4b30 0x7fabfc15afe0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fac00ce6640 1 -- 172.21.15.60:0/4240000550 >> 172.21.15.60:0/4240000550 conn(0x7fabfc071b00 msgr2=0x7fabfc071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:53.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fac00ce6640 1 -- 172.21.15.60:0/4240000550 shutdown_connections 2024-04-25T10:09:53.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fac00ce6640 1 -- 172.21.15.60:0/4240000550 wait complete. 2024-04-25T10:09:53.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fac00ce6640 1 Processor -- start 2024-04-25T10:09:53.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fac00ce6640 1 -- start start 2024-04-25T10:09:53.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fac00ce6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabfc06bc20 0x7fabfc099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fac00ce6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabfc0b3fa0 0x7fabfc165150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fac00ce6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabfc0b4b30 0x7fabfc1f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fac00ce6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc0b3010 con 0x7fabfc06bc20 2024-04-25T10:09:53.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fac00ce6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc0b2e90 con 0x7fabfc0b3fa0 2024-04-25T10:09:53.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fac00ce6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc0b3190 con 0x7fabfc0b4b30 2024-04-25T10:09:53.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fabfa59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabfc06bc20 0x7fabfc099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fabfa59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabfc06bc20 0x7fabfc099a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36840/0 (socket says 172.21.15.60:36840) 2024-04-25T10:09:53.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fabfa59c640 1 -- 172.21.15.60:0/764049466 learned_addr learned my addr 172.21.15.60:0/764049466 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:53.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fabfad9d640 1 --2- 172.21.15.60:0/764049466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabfc0b4b30 0x7fabfc1f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fabfa59c640 1 -- 172.21.15.60:0/764049466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabfc0b4b30 msgr2=0x7fabfc1f71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fabf3fff640 1 --2- 172.21.15.60:0/764049466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabfc0b3fa0 0x7fabfc165150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fabfa59c640 1 --2- 172.21.15.60:0/764049466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabfc0b4b30 0x7fabfc1f71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fabfa59c640 1 -- 172.21.15.60:0/764049466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabfc0b3fa0 msgr2=0x7fabfc165150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fabfa59c640 1 --2- 172.21.15.60:0/764049466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabfc0b3fa0 0x7fabfc165150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fabfa59c640 1 -- 172.21.15.60:0/764049466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc09a180 con 0x7fabfc06bc20 2024-04-25T10:09:53.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.625+0000 7fabfad9d640 1 --2- 172.21.15.60:0/764049466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabfc0b4b30 0x7fabfc1f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:53.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.629+0000 7fabfa59c640 1 --2- 172.21.15.60:0/764049466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabfc06bc20 0x7fabfc099a30 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fabec019fb0 tx=0x7fabec019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.629+0000 7fabf2ffd640 1 -- 172.21.15.60:0/764049466 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec0b8020 con 0x7fabfc06bc20 2024-04-25T10:09:53.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.629+0000 7fabf2ffd640 1 -- 172.21.15.60:0/764049466 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec0b4410 con 0x7fabfc06bc20 2024-04-25T10:09:53.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.629+0000 7fac00ce6640 1 -- 172.21.15.60:0/764049466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabfc09a460 con 0x7fabfc06bc20 2024-04-25T10:09:53.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.629+0000 7fabf2ffd640 1 -- 172.21.15.60:0/764049466 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec0b0040 con 0x7fabfc06bc20 2024-04-25T10:09:53.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.629+0000 7fac00ce6640 1 -- 172.21.15.60:0/764049466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabfc09aa20 con 0x7fabfc06bc20 2024-04-25T10:09:53.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.629+0000 7fac00ce6640 1 -- 172.21.15.60:0/764049466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabfc074770 con 0x7fabfc06bc20 2024-04-25T10:09:53.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.629+0000 7fabf2ffd640 1 -- 172.21.15.60:0/764049466 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fabec0bb020 con 0x7fabfc06bc20 2024-04-25T10:09:53.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.629+0000 7fabf2ffd640 1 --2- 172.21.15.60:0/764049466 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabdc066f60 0x7fabdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.629+0000 7fabf2ffd640 1 -- 172.21.15.60:0/764049466 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabec11ebf0 con 0x7fabfc06bc20 2024-04-25T10:09:53.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.629+0000 7fabf3fff640 1 --2- 172.21.15.60:0/764049466 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabdc066f60 0x7fabdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.629+0000 7fabf3fff640 1 --2- 172.21.15.60:0/764049466 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabdc066f60 0x7fabdc069420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009790 tx=0x7fabe4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.633+0000 7fabf2ffd640 1 -- 172.21.15.60:0/764049466 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fabec0c2030 con 0x7fabfc06bc20 2024-04-25T10:09:53.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.793+0000 7fac00ce6640 1 -- 172.21.15.60:0/764049466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fabfc06ec40 con 0x7fabfc06bc20 2024-04-25T10:09:53.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.793+0000 7fabf2ffd640 1 -- 172.21.15.60:0/764049466 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7fabec016090 con 0x7fabfc06bc20 2024-04-25T10:09:53.802 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:09:53.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.793+0000 7fac00ce6640 1 -- 172.21.15.60:0/764049466 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabdc066f60 msgr2=0x7fabdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.793+0000 7fac00ce6640 1 --2- 172.21.15.60:0/764049466 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabdc066f60 0x7fabdc069420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009790 tx=0x7fabe4008040 comp rx=0 tx=0).stop 2024-04-25T10:09:53.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.793+0000 7fac00ce6640 1 -- 172.21.15.60:0/764049466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabfc06bc20 msgr2=0x7fabfc099a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.793+0000 7fac00ce6640 1 --2- 172.21.15.60:0/764049466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabfc06bc20 0x7fabfc099a30 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fabec019fb0 tx=0x7fabec019120 comp rx=0 tx=0).stop 2024-04-25T10:09:53.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.797+0000 7fac00ce6640 1 -- 172.21.15.60:0/764049466 shutdown_connections 2024-04-25T10:09:53.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.797+0000 7fac00ce6640 1 --2- 172.21.15.60:0/764049466 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fabdc066f60 0x7fabdc069420 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.797+0000 7fac00ce6640 1 --2- 172.21.15.60:0/764049466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabfc0b4b30 0x7fabfc1f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.797+0000 7fac00ce6640 1 --2- 172.21.15.60:0/764049466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabfc0b3fa0 0x7fabfc165150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.797+0000 7fac00ce6640 1 --2- 172.21.15.60:0/764049466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabfc06bc20 0x7fabfc099a30 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.797+0000 7fac00ce6640 1 -- 172.21.15.60:0/764049466 >> 172.21.15.60:0/764049466 conn(0x7fabfc071b00 msgr2=0x7fabfc06fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.797+0000 7fac00ce6640 1 -- 172.21.15.60:0/764049466 shutdown_connections 2024-04-25T10:09:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.797+0000 7fac00ce6640 1 -- 172.21.15.60:0/764049466 wait complete. 2024-04-25T10:09:53.821 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:09:53.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.937+0000 7f59d950e640 1 Processor -- start 2024-04-25T10:09:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.937+0000 7f59d950e640 1 -- start start 2024-04-25T10:09:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.937+0000 7f59d950e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f59d4160080 0x7f59d415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.937+0000 7f59d950e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d4144a40 con 0x7f59d40cbcf0 2024-04-25T10:09:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.937+0000 7f59d950e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d40cc9f0 con 0x7f59d415fa20 2024-04-25T10:09:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.937+0000 7f59d950e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d415fe90 con 0x7f59d4160080 2024-04-25T10:09:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.937+0000 7f59d27fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f59d4160080 0x7f59d415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.937+0000 7f59d27fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f59d4160080 0x7f59d415f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36852/0 (socket says 172.21.15.60:36852) 2024-04-25T10:09:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.937+0000 7f59d27fc640 1 -- 172.21.15.60:0/405190109 learned_addr learned my addr 172.21.15.60:0/405190109 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.937+0000 7f59d27fc640 1 -- 172.21.15.60:0/405190109 >> v1:172.21.15.100:6789/0 conn(0x7f59d415fa20 legacy=0x7f59d40ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.937+0000 7f59d27fc640 1 -- 172.21.15.60:0/405190109 >> v1:172.21.15.60:6789/0 conn(0x7f59d40cbcf0 legacy=0x7f59d4071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.937+0000 7f59d27fc640 1 -- 172.21.15.60:0/405190109 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d40cccd0 con 0x7f59d4160080 2024-04-25T10:09:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.937+0000 7f59d1ffb640 1 -- 172.21.15.60:0/405190109 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632212871 0 0) 0x7f59d4144a40 con 0x7f59d40cbcf0 2024-04-25T10:09:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d27fc640 1 --2- 172.21.15.60:0/405190109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d4160080 0x7f59d415f4e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009870 tx=0x7f59c801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b9070dc41bfd1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d1ffb640 1 -- 172.21.15.60:0/405190109 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c80b0070 con 0x7f59d4160080 2024-04-25T10:09:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d1ffb640 1 -- 172.21.15.60:0/405190109 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c80ab600 con 0x7f59d4160080 2024-04-25T10:09:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d1ffb640 1 -- 172.21.15.60:0/405190109 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c80ab9d0 con 0x7f59d4160080 2024-04-25T10:09:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d950e640 1 -- 172.21.15.60:0/405190109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d4160080 msgr2=0x7f59d415f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d950e640 1 --2- 172.21.15.60:0/405190109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d4160080 0x7f59d415f4e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009870 tx=0x7f59c801c520 comp rx=0 tx=0).stop 2024-04-25T10:09:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d950e640 1 -- 172.21.15.60:0/405190109 shutdown_connections 2024-04-25T10:09:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d950e640 1 --2- 172.21.15.60:0/405190109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d4160080 0x7f59d415f4e0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d950e640 1 -- 172.21.15.60:0/405190109 >> 172.21.15.60:0/405190109 conn(0x7f59d40670b0 msgr2=0x7f59d406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d950e640 1 -- 172.21.15.60:0/405190109 shutdown_connections 2024-04-25T10:09:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d950e640 1 -- 172.21.15.60:0/405190109 wait complete. 2024-04-25T10:09:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d950e640 1 Processor -- start 2024-04-25T10:09:53.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d950e640 1 -- start start 2024-04-25T10:09:53.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d950e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d40cbcf0 0x7f59d4140db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d950e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59d415fa20 0x7f59d40ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d950e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59d4160080 0x7f59d4143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d950e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d40ce740 con 0x7f59d40cbcf0 2024-04-25T10:09:53.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d950e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d40ce5c0 con 0x7f59d415fa20 2024-04-25T10:09:53.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d950e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4169890 con 0x7f59d4160080 2024-04-25T10:09:53.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d40cbcf0 0x7f59d4140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d40cbcf0 0x7f59d4140db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36856/0 (socket says 172.21.15.60:36856) 2024-04-25T10:09:53.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d2ffd640 1 -- 172.21.15.60:0/2374215648 learned_addr learned my addr 172.21.15.60:0/2374215648 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:53.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d27fc640 1 --2- 172.21.15.60:0/2374215648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59d415fa20 0x7f59d40ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d37fe640 1 --2- 172.21.15.60:0/2374215648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59d4160080 0x7f59d4143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d2ffd640 1 -- 172.21.15.60:0/2374215648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59d4160080 msgr2=0x7f59d4143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d2ffd640 1 --2- 172.21.15.60:0/2374215648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59d4160080 0x7f59d4143110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d2ffd640 1 -- 172.21.15.60:0/2374215648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59d415fa20 msgr2=0x7f59d40ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d2ffd640 1 --2- 172.21.15.60:0/2374215648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59d415fa20 0x7f59d40ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d2ffd640 1 -- 172.21.15.60:0/2374215648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d40ae300 con 0x7f59d40cbcf0 2024-04-25T10:09:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d27fc640 1 --2- 172.21.15.60:0/2374215648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59d415fa20 0x7f59d40ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d2ffd640 1 --2- 172.21.15.60:0/2374215648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d40cbcf0 0x7f59d4140db0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f59c400c500 tx=0x7f59c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59bb7fe640 1 -- 172.21.15.60:0/2374215648 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4017070 con 0x7f59d40cbcf0 2024-04-25T10:09:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59bb7fe640 1 -- 172.21.15.60:0/2374215648 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c4003600 con 0x7f59d40cbcf0 2024-04-25T10:09:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.941+0000 7f59d950e640 1 -- 172.21.15.60:0/2374215648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d40ae5a0 con 0x7f59d40cbcf0 2024-04-25T10:09:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.945+0000 7f59bb7fe640 1 -- 172.21.15.60:0/2374215648 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40039f0 con 0x7f59d40cbcf0 2024-04-25T10:09:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.945+0000 7f59d950e640 1 -- 172.21.15.60:0/2374215648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d40ae880 con 0x7f59d40cbcf0 2024-04-25T10:09:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.945+0000 7f59d950e640 1 -- 172.21.15.60:0/2374215648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d4074770 con 0x7f59d40cbcf0 2024-04-25T10:09:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.945+0000 7f59bb7fe640 1 -- 172.21.15.60:0/2374215648 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59c401a070 con 0x7f59d40cbcf0 2024-04-25T10:09:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.945+0000 7f59bb7fe640 1 --2- 172.21.15.60:0/2374215648 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f59c0066f60 0x7f59c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.945+0000 7f59bb7fe640 1 -- 172.21.15.60:0/2374215648 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59c4084580 con 0x7f59d40cbcf0 2024-04-25T10:09:53.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.945+0000 7f59d27fc640 1 --2- 172.21.15.60:0/2374215648 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f59c0066f60 0x7f59c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.945+0000 7f59d27fc640 1 --2- 172.21.15.60:0/2374215648 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f59c0066f60 0x7f59c0069420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f59c80ad000 tx=0x7f59c80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:53.949+0000 7f59bb7fe640 1 -- 172.21.15.60:0/2374215648 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59c40555c0 con 0x7f59d40cbcf0 2024-04-25T10:09:54.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.105+0000 7f59d950e640 1 -- 172.21.15.60:0/2374215648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f59d413d2b0 con 0x7f59d40cbcf0 2024-04-25T10:09:54.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.105+0000 7f59bb7fe640 1 -- 172.21.15.60:0/2374215648 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f59c4054ec0 con 0x7f59d40cbcf0 2024-04-25T10:09:54.117 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:09:54.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.109+0000 7f59d950e640 1 -- 172.21.15.60:0/2374215648 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f59c0066f60 msgr2=0x7f59c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.109+0000 7f59d950e640 1 --2- 172.21.15.60:0/2374215648 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f59c0066f60 0x7f59c0069420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f59c80ad000 tx=0x7f59c80a6000 comp rx=0 tx=0).stop 2024-04-25T10:09:54.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.109+0000 7f59d950e640 1 -- 172.21.15.60:0/2374215648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d40cbcf0 msgr2=0x7f59d4140db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.109+0000 7f59d950e640 1 --2- 172.21.15.60:0/2374215648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d40cbcf0 0x7f59d4140db0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f59c400c500 tx=0x7f59c400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.109+0000 7f59d950e640 1 -- 172.21.15.60:0/2374215648 shutdown_connections 2024-04-25T10:09:54.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.109+0000 7f59d950e640 1 --2- 172.21.15.60:0/2374215648 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f59c0066f60 0x7f59c0069420 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.109+0000 7f59d950e640 1 --2- 172.21.15.60:0/2374215648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59d4160080 0x7f59d4143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.109+0000 7f59d950e640 1 --2- 172.21.15.60:0/2374215648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59d415fa20 0x7f59d40ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.109+0000 7f59d950e640 1 --2- 172.21.15.60:0/2374215648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d40cbcf0 0x7f59d4140db0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.109+0000 7f59d950e640 1 -- 172.21.15.60:0/2374215648 >> 172.21.15.60:0/2374215648 conn(0x7f59d40670b0 msgr2=0x7f59d40b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:54.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.109+0000 7f59d950e640 1 -- 172.21.15.60:0/2374215648 shutdown_connections 2024-04-25T10:09:54.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.109+0000 7f59d950e640 1 -- 172.21.15.60:0/2374215648 wait complete. 2024-04-25T10:09:54.135 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:09:54.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.241+0000 7ff36292f640 1 Processor -- start 2024-04-25T10:09:54.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.241+0000 7ff36292f640 1 -- start start 2024-04-25T10:09:54.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.241+0000 7ff36292f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff35c0b5870 0x7ff35c0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.241+0000 7ff36292f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff35c070b30 con 0x7ff35c15dc80 2024-04-25T10:09:54.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.241+0000 7ff36292f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff35c161790 con 0x7ff35c15d8a0 2024-04-25T10:09:54.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.241+0000 7ff36292f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff35c161a70 con 0x7ff35c0b5870 2024-04-25T10:09:54.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.241+0000 7ff35b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff35c15dc80 0x7ff35c0ce5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44638/0 (socket says 172.21.15.60:44638) 2024-04-25T10:09:54.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.241+0000 7ff35b7fe640 1 -- 172.21.15.60:0/2588540681 learned_addr learned my addr 172.21.15.60:0/2588540681 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:54.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.241+0000 7ff360ecc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff35c0b5870 0x7ff35c0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.241+0000 7ff35affd640 1 -- 172.21.15.60:0/2588540681 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975239923 0 0) 0x7ff35c070b30 con 0x7ff35c15dc80 2024-04-25T10:09:54.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.241+0000 7ff35affd640 1 -- 172.21.15.60:0/2588540681 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff344003530 con 0x7ff35c15dc80 2024-04-25T10:09:54.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.241+0000 7ff360ecc640 1 -- 172.21.15.60:0/2588540681 >> v1:172.21.15.100:6789/0 conn(0x7ff35c15d8a0 legacy=0x7ff35c0b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:54.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.241+0000 7ff360ecc640 1 -- 172.21.15.60:0/2588540681 >> v1:172.21.15.60:6789/0 conn(0x7ff35c15dc80 legacy=0x7ff35c0ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.241+0000 7ff360ecc640 1 -- 172.21.15.60:0/2588540681 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35c070800 con 0x7ff35c0b5870 2024-04-25T10:09:54.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff360ecc640 1 --2- 172.21.15.60:0/2588540681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff35c0b5870 0x7ff35c0ccd00 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7ff3540097c0 tx=0x7ff35401c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9caee29cc88644a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff35affd640 1 -- 172.21.15.60:0/2588540681 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3540b0070 con 0x7ff35c0b5870 2024-04-25T10:09:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff35affd640 1 -- 172.21.15.60:0/2588540681 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3540ab600 con 0x7ff35c0b5870 2024-04-25T10:09:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff35affd640 1 -- 172.21.15.60:0/2588540681 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3540ab9b0 con 0x7ff35c0b5870 2024-04-25T10:09:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff36292f640 1 -- 172.21.15.60:0/2588540681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff35c0b5870 msgr2=0x7ff35c0ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff36292f640 1 --2- 172.21.15.60:0/2588540681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff35c0b5870 0x7ff35c0ccd00 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7ff3540097c0 tx=0x7ff35401c370 comp rx=0 tx=0).stop 2024-04-25T10:09:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff36292f640 1 -- 172.21.15.60:0/2588540681 shutdown_connections 2024-04-25T10:09:54.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff36292f640 1 --2- 172.21.15.60:0/2588540681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff35c0b5870 0x7ff35c0ccd00 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff36292f640 1 -- 172.21.15.60:0/2588540681 >> 172.21.15.60:0/2588540681 conn(0x7ff35c06e400 msgr2=0x7ff35c06e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:54.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff36292f640 1 -- 172.21.15.60:0/2588540681 shutdown_connections 2024-04-25T10:09:54.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff36292f640 1 -- 172.21.15.60:0/2588540681 wait complete. 2024-04-25T10:09:54.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff36292f640 1 Processor -- start 2024-04-25T10:09:54.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff36292f640 1 -- start start 2024-04-25T10:09:54.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff36292f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff35c0b5870 0x7ff35c157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff36292f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff35c15d8a0 0x7ff35c15e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff36292f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35c15fd30 0x7ff35c160190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff36292f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c16bbb0 con 0x7ff35c0b5870 2024-04-25T10:09:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff36292f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c0cf510 con 0x7ff35c15fd30 2024-04-25T10:09:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff36292f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c16bd30 con 0x7ff35c15d8a0 2024-04-25T10:09:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff35bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff35c0b5870 0x7ff35c157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff35bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff35c0b5870 0x7ff35c157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36868/0 (socket says 172.21.15.60:36868) 2024-04-25T10:09:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff35bfff640 1 -- 172.21.15.60:0/2131846690 learned_addr learned my addr 172.21.15.60:0/2131846690 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff360ecc640 1 --2- 172.21.15.60:0/2131846690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35c15fd30 0x7ff35c160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff35bfff640 1 -- 172.21.15.60:0/2131846690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff35c15d8a0 msgr2=0x7ff35c15e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff35bfff640 1 --2- 172.21.15.60:0/2131846690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff35c15d8a0 0x7ff35c15e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff35bfff640 1 -- 172.21.15.60:0/2131846690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35c15fd30 msgr2=0x7ff35c160190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff35bfff640 1 --2- 172.21.15.60:0/2131846690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35c15fd30 0x7ff35c160190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.245+0000 7ff35bfff640 1 -- 172.21.15.60:0/2131846690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35c160810 con 0x7ff35c0b5870 2024-04-25T10:09:54.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.249+0000 7ff35bfff640 1 --2- 172.21.15.60:0/2131846690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff35c0b5870 0x7ff35c157250 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00c450 tx=0x7ff34c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.249+0000 7ff358ff9640 1 -- 172.21.15.60:0/2131846690 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c012830 con 0x7ff35c0b5870 2024-04-25T10:09:54.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.249+0000 7ff358ff9640 1 -- 172.21.15.60:0/2131846690 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34c017070 con 0x7ff35c0b5870 2024-04-25T10:09:54.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.249+0000 7ff36292f640 1 -- 172.21.15.60:0/2131846690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff35c160af0 con 0x7ff35c0b5870 2024-04-25T10:09:54.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.249+0000 7ff358ff9640 1 -- 172.21.15.60:0/2131846690 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c012c10 con 0x7ff35c0b5870 2024-04-25T10:09:54.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.249+0000 7ff36292f640 1 -- 172.21.15.60:0/2131846690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35c14d510 con 0x7ff35c0b5870 2024-04-25T10:09:54.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.249+0000 7ff36292f640 1 -- 172.21.15.60:0/2131846690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff35c074770 con 0x7ff35c0b5870 2024-04-25T10:09:54.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.249+0000 7ff358ff9640 1 -- 172.21.15.60:0/2131846690 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff34c01c460 con 0x7ff35c0b5870 2024-04-25T10:09:54.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.249+0000 7ff358ff9640 1 --2- 172.21.15.60:0/2131846690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff348066f60 0x7ff348069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.249+0000 7ff358ff9640 1 -- 172.21.15.60:0/2131846690 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff34c084ef0 con 0x7ff35c0b5870 2024-04-25T10:09:54.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.249+0000 7ff35b7fe640 1 --2- 172.21.15.60:0/2131846690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff348066f60 0x7ff348069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.249+0000 7ff35b7fe640 1 --2- 172.21.15.60:0/2131846690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff348066f60 0x7ff348069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff3500012f0 tx=0x7ff350001660 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.253+0000 7ff358ff9640 1 -- 172.21.15.60:0/2131846690 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff34c055e60 con 0x7ff35c0b5870 2024-04-25T10:09:54.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.405+0000 7ff36292f640 1 -- 172.21.15.60:0/2131846690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff35c0700d0 con 0x7ff35c0b5870 2024-04-25T10:09:54.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.409+0000 7ff358ff9640 1 -- 172.21.15.60:0/2131846690 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff34c055760 con 0x7ff35c0b5870 2024-04-25T10:09:54.416 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:09:54.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.409+0000 7ff36292f640 1 -- 172.21.15.60:0/2131846690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff348066f60 msgr2=0x7ff348069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.409+0000 7ff36292f640 1 --2- 172.21.15.60:0/2131846690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff348066f60 0x7ff348069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff3500012f0 tx=0x7ff350001660 comp rx=0 tx=0).stop 2024-04-25T10:09:54.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.409+0000 7ff36292f640 1 -- 172.21.15.60:0/2131846690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff35c0b5870 msgr2=0x7ff35c157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.409+0000 7ff36292f640 1 --2- 172.21.15.60:0/2131846690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff35c0b5870 0x7ff35c157250 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00c450 tx=0x7ff34c00c910 comp rx=0 tx=0).stop 2024-04-25T10:09:54.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.409+0000 7ff36292f640 1 -- 172.21.15.60:0/2131846690 shutdown_connections 2024-04-25T10:09:54.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.409+0000 7ff36292f640 1 --2- 172.21.15.60:0/2131846690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff348066f60 0x7ff348069420 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.409+0000 7ff36292f640 1 --2- 172.21.15.60:0/2131846690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35c15fd30 0x7ff35c160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.409+0000 7ff36292f640 1 --2- 172.21.15.60:0/2131846690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff35c15d8a0 0x7ff35c15e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.409+0000 7ff36292f640 1 --2- 172.21.15.60:0/2131846690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff35c0b5870 0x7ff35c157250 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.409+0000 7ff36292f640 1 -- 172.21.15.60:0/2131846690 >> 172.21.15.60:0/2131846690 conn(0x7ff35c06e400 msgr2=0x7ff35c0b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:54.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.409+0000 7ff36292f640 1 -- 172.21.15.60:0/2131846690 shutdown_connections 2024-04-25T10:09:54.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.409+0000 7ff36292f640 1 -- 172.21.15.60:0/2131846690 wait complete. 2024-04-25T10:09:54.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:09:54.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca87121640 1 Processor -- start 2024-04-25T10:09:54.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca87121640 1 -- start start 2024-04-25T10:09:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca87121640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fca8015dcc0 0x7fca800b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca87121640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca800b5310 con 0x7fca8006ac10 2024-04-25T10:09:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca87121640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca800b55b0 con 0x7fca800b1cc0 2024-04-25T10:09:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca87121640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca800b5b70 con 0x7fca8015dcc0 2024-04-25T10:09:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca84ebd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fca8006ac10 0x7fca8006aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44640/0 (socket says 172.21.15.60:44640) 2024-04-25T10:09:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca84ebd640 1 -- 172.21.15.60:0/2128239010 learned_addr learned my addr 172.21.15.60:0/2128239010 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7ffff640 1 --2- 172.21.15.60:0/2128239010 >> v2:172.21.15.100:3300/0 conn(0x7fca8015dcc0 0x7fca800b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7f7fe640 1 -- 172.21.15.60:0/2128239010 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2685387568 0 0) 0x7fca800b5310 con 0x7fca8006ac10 2024-04-25T10:09:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7f7fe640 1 -- 172.21.15.60:0/2128239010 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca70003530 con 0x7fca8006ac10 2024-04-25T10:09:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7f7fe640 1 -- 172.21.15.60:0/2128239010 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4000526198 0 0) 0x7fca70003530 con 0x7fca8006ac10 2024-04-25T10:09:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7f7fe640 1 -- 172.21.15.60:0/2128239010 >> v2:172.21.15.100:3300/0 conn(0x7fca8015dcc0 msgr2=0x7fca800b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7f7fe640 1 --2- 172.21.15.60:0/2128239010 >> v2:172.21.15.100:3300/0 conn(0x7fca8015dcc0 0x7fca800b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7f7fe640 1 -- 172.21.15.60:0/2128239010 >> v1:172.21.15.100:6789/0 conn(0x7fca800b1cc0 legacy=0x7fca800b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7f7fe640 1 -- 172.21.15.60:0/2128239010 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca800b5890 con 0x7fca8006ac10 2024-04-25T10:09:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7f7fe640 1 -- 172.21.15.60:0/2128239010 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fca74001680 con 0x7fca8006ac10 2024-04-25T10:09:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7f7fe640 1 -- 172.21.15.60:0/2128239010 >> v1:172.21.15.60:6789/0 conn(0x7fca8006ac10 legacy=0x7fca8006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7f7fe640 1 --2- 172.21.15.60:0/2128239010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca700039c0 0x7fca70003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7f7fe640 1 --2- 172.21.15.60:0/2128239010 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca70004810 0x7fca70004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7f7fe640 1 --2- 172.21.15.60:0/2128239010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca700051d0 0x7fca7000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7f7fe640 1 -- 172.21.15.60:0/2128239010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca800b5b70 con 0x7fca700051d0 2024-04-25T10:09:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7f7fe640 1 -- 172.21.15.60:0/2128239010 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7000c100 con 0x7fca700039c0 2024-04-25T10:09:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7f7fe640 1 -- 172.21.15.60:0/2128239010 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7000c300 con 0x7fca70004810 2024-04-25T10:09:54.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca856be640 1 --2- 172.21.15.60:0/2128239010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca700051d0 0x7fca7000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca84ebd640 1 --2- 172.21.15.60:0/2128239010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca700039c0 0x7fca70003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca7ffff640 1 --2- 172.21.15.60:0/2128239010 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca70004810 0x7fca70004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca856be640 1 -- 172.21.15.60:0/2128239010 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca70004810 msgr2=0x7fca70004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:54.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca856be640 1 --2- 172.21.15.60:0/2128239010 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca70004810 0x7fca70004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca856be640 1 -- 172.21.15.60:0/2128239010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca700039c0 msgr2=0x7fca70003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:54.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca856be640 1 --2- 172.21.15.60:0/2128239010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca700039c0 0x7fca70003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.553+0000 7fca856be640 1 -- 172.21.15.60:0/2128239010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca70003530 con 0x7fca700051d0 2024-04-25T10:09:54.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.557+0000 7fca856be640 1 -- 172.21.15.60:0/2128239010 reap_dead start 2024-04-25T10:09:54.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.557+0000 7fca856be640 1 --2- 172.21.15.60:0/2128239010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca700051d0 0x7fca7000b9d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fca78007fd0 tx=0x7fca7801c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e8f5fe5f3a1a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.557+0000 7fca7f7fe640 1 -- 172.21.15.60:0/2128239010 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780040b0 con 0x7fca700051d0 2024-04-25T10:09:54.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.557+0000 7fca7f7fe640 1 -- 172.21.15.60:0/2128239010 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca780a9680 con 0x7fca700051d0 2024-04-25T10:09:54.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.557+0000 7fca7f7fe640 1 -- 172.21.15.60:0/2128239010 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780a9a70 con 0x7fca700051d0 2024-04-25T10:09:54.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.557+0000 7fca87121640 1 -- 172.21.15.60:0/2128239010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca700051d0 msgr2=0x7fca7000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.557+0000 7fca87121640 1 --2- 172.21.15.60:0/2128239010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca700051d0 0x7fca7000b9d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fca78007fd0 tx=0x7fca7801c660 comp rx=0 tx=0).stop 2024-04-25T10:09:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.557+0000 7fca87121640 1 -- 172.21.15.60:0/2128239010 shutdown_connections 2024-04-25T10:09:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.557+0000 7fca87121640 1 --2- 172.21.15.60:0/2128239010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca700051d0 0x7fca7000b9d0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.557+0000 7fca87121640 1 -- 172.21.15.60:0/2128239010 >> 172.21.15.60:0/2128239010 conn(0x7fca80072920 msgr2=0x7fca80099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.557+0000 7fca87121640 1 -- 172.21.15.60:0/2128239010 shutdown_connections 2024-04-25T10:09:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.557+0000 7fca87121640 1 -- 172.21.15.60:0/2128239010 wait complete. 2024-04-25T10:09:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.557+0000 7fca87121640 1 Processor -- start 2024-04-25T10:09:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.557+0000 7fca87121640 1 -- start start 2024-04-25T10:09:54.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca87121640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca700051d0 0x7fca801533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca87121640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca80153910 0x7fca80151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca87121640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca801517d0 0x7fca8015e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca87121640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8006b840 con 0x7fca700051d0 2024-04-25T10:09:54.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca87121640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca800b2100 con 0x7fca80153910 2024-04-25T10:09:54.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca87121640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8006b9c0 con 0x7fca801517d0 2024-04-25T10:09:54.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca84ebd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca700051d0 0x7fca801533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca84ebd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca700051d0 0x7fca801533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36872/0 (socket says 172.21.15.60:36872) 2024-04-25T10:09:54.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca84ebd640 1 -- 172.21.15.60:0/747062110 learned_addr learned my addr 172.21.15.60:0/747062110 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:54.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca84ebd640 1 -- 172.21.15.60:0/747062110 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca801517d0 msgr2=0x7fca8015e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca856be640 1 --2- 172.21.15.60:0/747062110 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca801517d0 0x7fca8015e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca84ebd640 1 --2- 172.21.15.60:0/747062110 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca801517d0 0x7fca8015e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca84ebd640 1 -- 172.21.15.60:0/747062110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca80153910 msgr2=0x7fca80151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca7ffff640 1 --2- 172.21.15.60:0/747062110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca80153910 0x7fca80151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca84ebd640 1 --2- 172.21.15.60:0/747062110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca80153910 0x7fca80151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca84ebd640 1 -- 172.21.15.60:0/747062110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca80151ea0 con 0x7fca700051d0 2024-04-25T10:09:54.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca7ffff640 1 --2- 172.21.15.60:0/747062110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca80153910 0x7fca80151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:54.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca84ebd640 1 --2- 172.21.15.60:0/747062110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca700051d0 0x7fca801533d0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fca7400e780 tx=0x7fca7400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca7d7fa640 1 -- 172.21.15.60:0/747062110 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7400c6c0 con 0x7fca700051d0 2024-04-25T10:09:54.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca7d7fa640 1 -- 172.21.15.60:0/747062110 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca74018070 con 0x7fca700051d0 2024-04-25T10:09:54.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca7d7fa640 1 -- 172.21.15.60:0/747062110 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7400cac0 con 0x7fca700051d0 2024-04-25T10:09:54.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca87121640 1 -- 172.21.15.60:0/747062110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca80152180 con 0x7fca700051d0 2024-04-25T10:09:54.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca87121640 1 -- 172.21.15.60:0/747062110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca80152740 con 0x7fca700051d0 2024-04-25T10:09:54.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca87121640 1 -- 172.21.15.60:0/747062110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca80074770 con 0x7fca700051d0 2024-04-25T10:09:54.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca7d7fa640 1 -- 172.21.15.60:0/747062110 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca7400cc60 con 0x7fca700051d0 2024-04-25T10:09:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.561+0000 7fca7d7fa640 1 --2- 172.21.15.60:0/747062110 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fca64066f90 0x7fca64069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.565+0000 7fca7d7fa640 1 -- 172.21.15.60:0/747062110 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca74084e40 con 0x7fca700051d0 2024-04-25T10:09:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.565+0000 7fca7ffff640 1 --2- 172.21.15.60:0/747062110 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fca64066f90 0x7fca64069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.565+0000 7fca7ffff640 1 --2- 172.21.15.60:0/747062110 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fca64066f90 0x7fca64069450 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fca80153360 tx=0x7fca6c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.569+0000 7fca7d7fa640 1 -- 172.21.15.60:0/747062110 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca74055ee0 con 0x7fca700051d0 2024-04-25T10:09:54.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.737+0000 7fca87121640 1 -- 172.21.15.60:0/747062110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fca80153cf0 con 0x7fca700051d0 2024-04-25T10:09:54.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.737+0000 7fca7d7fa640 1 -- 172.21.15.60:0/747062110 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) v1 ==== 80+0+0 (secure 0 0 0) 0x7fca740557e0 con 0x7fca700051d0 2024-04-25T10:09:54.748 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:09:54.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.741+0000 7fca87121640 1 -- 172.21.15.60:0/747062110 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fca64066f90 msgr2=0x7fca64069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.741+0000 7fca87121640 1 --2- 172.21.15.60:0/747062110 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fca64066f90 0x7fca64069450 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fca80153360 tx=0x7fca6c008040 comp rx=0 tx=0).stop 2024-04-25T10:09:54.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.741+0000 7fca87121640 1 -- 172.21.15.60:0/747062110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca700051d0 msgr2=0x7fca801533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.741+0000 7fca87121640 1 --2- 172.21.15.60:0/747062110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca700051d0 0x7fca801533d0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fca7400e780 tx=0x7fca7400ec40 comp rx=0 tx=0).stop 2024-04-25T10:09:54.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.741+0000 7fca87121640 1 -- 172.21.15.60:0/747062110 shutdown_connections 2024-04-25T10:09:54.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.741+0000 7fca87121640 1 --2- 172.21.15.60:0/747062110 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fca64066f90 0x7fca64069450 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.741+0000 7fca87121640 1 --2- 172.21.15.60:0/747062110 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca801517d0 0x7fca8015e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.741+0000 7fca87121640 1 --2- 172.21.15.60:0/747062110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca80153910 0x7fca80151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.741+0000 7fca87121640 1 --2- 172.21.15.60:0/747062110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca700051d0 0x7fca801533d0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.741+0000 7fca87121640 1 -- 172.21.15.60:0/747062110 >> 172.21.15.60:0/747062110 conn(0x7fca80072920 msgr2=0x7fca8006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:54.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.741+0000 7fca87121640 1 -- 172.21.15.60:0/747062110 shutdown_connections 2024-04-25T10:09:54.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.741+0000 7fca87121640 1 -- 172.21.15.60:0/747062110 wait complete. 2024-04-25T10:09:54.767 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:09:54.767 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:54.773 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:54.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.929+0000 7f50115da640 1 Processor -- start 2024-04-25T10:09:54.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.929+0000 7f50115da640 1 -- start start 2024-04-25T10:09:54.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f50115da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f500c0ae7c0 0x7f500c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f50115da640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f500c071fd0 con 0x7f500c06d2f0 2024-04-25T10:09:54.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f50115da640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f500c0b2ce0 con 0x7f500c15dc90 2024-04-25T10:09:54.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f50115da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500c0accc0 con 0x7f500c0ae7c0 2024-04-25T10:09:54.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f500b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f500c06d2f0 0x7f500c0ad6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44656/0 (socket says 172.21.15.60:44656) 2024-04-25T10:09:54.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f500b7fe640 1 -- 172.21.15.60:0/1723543397 learned_addr learned my addr 172.21.15.60:0/1723543397 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:54.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f500a7fc640 1 --2- 172.21.15.60:0/1723543397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f500c0ae7c0 0x7f500c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f5009ffb640 1 -- 172.21.15.60:0/1723543397 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327768450 0 0) 0x7f500c071fd0 con 0x7f500c06d2f0 2024-04-25T10:09:54.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f5009ffb640 1 -- 172.21.15.60:0/1723543397 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ff8003530 con 0x7f500c06d2f0 2024-04-25T10:09:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f5009ffb640 1 -- 172.21.15.60:0/1723543397 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3813379928 0 0) 0x7f4ff8003530 con 0x7f500c06d2f0 2024-04-25T10:09:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f5009ffb640 1 -- 172.21.15.60:0/1723543397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f500c0ae7c0 msgr2=0x7f500c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f5009ffb640 1 --2- 172.21.15.60:0/1723543397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f500c0ae7c0 0x7f500c071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f5009ffb640 1 -- 172.21.15.60:0/1723543397 >> v1:172.21.15.100:6789/0 conn(0x7f500c15dc90 legacy=0x7f500c0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f5009ffb640 1 -- 172.21.15.60:0/1723543397 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c072290 con 0x7f500c06d2f0 2024-04-25T10:09:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f5009ffb640 1 -- 172.21.15.60:0/1723543397 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f5004001680 con 0x7f500c06d2f0 2024-04-25T10:09:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f5009ffb640 1 -- 172.21.15.60:0/1723543397 >> v1:172.21.15.60:6789/0 conn(0x7f500c06d2f0 legacy=0x7f500c0ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f5009ffb640 1 --2- 172.21.15.60:0/1723543397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff80039f0 0x7f4ff8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f5009ffb640 1 --2- 172.21.15.60:0/1723543397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f500c0ae7c0 0x7f4ff8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f5009ffb640 1 --2- 172.21.15.60:0/1723543397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ff8004d50 0x7f4ff80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f5009ffb640 1 -- 172.21.15.60:0/1723543397 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff800ba80 con 0x7f4ff8004d50 2024-04-25T10:09:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f5009ffb640 1 -- 172.21.15.60:0/1723543397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff800bc30 con 0x7f4ff80039f0 2024-04-25T10:09:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f5009ffb640 1 -- 172.21.15.60:0/1723543397 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff800be30 con 0x7f500c0ae7c0 2024-04-25T10:09:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f500b7fe640 1 --2- 172.21.15.60:0/1723543397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ff8004d50 0x7f4ff80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f500affd640 1 --2- 172.21.15.60:0/1723543397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff80039f0 0x7f4ff8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f500a7fc640 1 --2- 172.21.15.60:0/1723543397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f500c0ae7c0 0x7f4ff8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f500b7fe640 1 -- 172.21.15.60:0/1723543397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f500c0ae7c0 msgr2=0x7f4ff8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f500b7fe640 1 --2- 172.21.15.60:0/1723543397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f500c0ae7c0 0x7f4ff8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f500b7fe640 1 -- 172.21.15.60:0/1723543397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff80039f0 msgr2=0x7f4ff8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f500b7fe640 1 --2- 172.21.15.60:0/1723543397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff80039f0 0x7f4ff8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.933+0000 7f500b7fe640 1 -- 172.21.15.60:0/1723543397 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff8003530 con 0x7f4ff8004d50 2024-04-25T10:09:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f500b7fe640 1 --2- 172.21.15.60:0/1723543397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ff8004d50 0x7f4ff80051d0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f500400c160 tx=0x7f500401cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e110135e4bb9f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f5009ffb640 1 -- 172.21.15.60:0/1723543397 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5004007c70 con 0x7f4ff8004d50 2024-04-25T10:09:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f5009ffb640 1 -- 172.21.15.60:0/1723543397 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50040a9070 con 0x7f4ff8004d50 2024-04-25T10:09:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f5009ffb640 1 -- 172.21.15.60:0/1723543397 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5004005d60 con 0x7f4ff8004d50 2024-04-25T10:09:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f50115da640 1 -- 172.21.15.60:0/1723543397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ff8004d50 msgr2=0x7f4ff80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f50115da640 1 --2- 172.21.15.60:0/1723543397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ff8004d50 0x7f4ff80051d0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f500400c160 tx=0x7f500401cba0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f500b7fe640 1 -- 172.21.15.60:0/1723543397 reap_dead start 2024-04-25T10:09:54.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f50115da640 1 -- 172.21.15.60:0/1723543397 shutdown_connections 2024-04-25T10:09:54.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f50115da640 1 -- 172.21.15.60:0/1723543397 >> 172.21.15.60:0/1723543397 conn(0x7f500c0b12d0 msgr2=0x7f500c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:54.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f50115da640 1 -- 172.21.15.60:0/1723543397 shutdown_connections 2024-04-25T10:09:54.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f50115da640 1 -- 172.21.15.60:0/1723543397 wait complete. 2024-04-25T10:09:54.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f50115da640 1 Processor -- start 2024-04-25T10:09:54.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f50115da640 1 -- start start 2024-04-25T10:09:54.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f50115da640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ff8004d50 0x7f500c1f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f50115da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f500c1507e0 0x7f500c15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f50115da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f500c152800 0x7f500c152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f50115da640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c0b2010 con 0x7f4ff8004d50 2024-04-25T10:09:54.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f50115da640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c0ad260 con 0x7f500c1507e0 2024-04-25T10:09:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f50115da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500c0b2190 con 0x7f500c152800 2024-04-25T10:09:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f500affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ff8004d50 0x7f500c1f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f500affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ff8004d50 0x7f500c1f3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36898/0 (socket says 172.21.15.60:36898) 2024-04-25T10:09:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f500affd640 1 -- 172.21.15.60:0/491414182 learned_addr learned my addr 172.21.15.60:0/491414182 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f500a7fc640 1 --2- 172.21.15.60:0/491414182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f500c1507e0 0x7f500c15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f500b7fe640 1 --2- 172.21.15.60:0/491414182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f500c152800 0x7f500c152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f500affd640 1 -- 172.21.15.60:0/491414182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f500c152800 msgr2=0x7f500c152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f500affd640 1 --2- 172.21.15.60:0/491414182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f500c152800 0x7f500c152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f500affd640 1 -- 172.21.15.60:0/491414182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f500c1507e0 msgr2=0x7f500c15cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f500affd640 1 --2- 172.21.15.60:0/491414182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f500c1507e0 0x7f500c15cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f500affd640 1 -- 172.21.15.60:0/491414182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c156e80 con 0x7f4ff8004d50 2024-04-25T10:09:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f500affd640 1 --2- 172.21.15.60:0/491414182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ff8004d50 0x7f500c1f3f90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00e8f0 tx=0x7f4ffc00edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.937+0000 7f50037fe640 1 -- 172.21.15.60:0/491414182 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc003b90 con 0x7f4ff8004d50 2024-04-25T10:09:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.941+0000 7f50037fe640 1 -- 172.21.15.60:0/491414182 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ffc018070 con 0x7f4ff8004d50 2024-04-25T10:09:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.941+0000 7f50115da640 1 -- 172.21.15.60:0/491414182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500c150cc0 con 0x7f4ff8004d50 2024-04-25T10:09:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.941+0000 7f50037fe640 1 -- 172.21.15.60:0/491414182 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc0050d0 con 0x7f4ff8004d50 2024-04-25T10:09:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.941+0000 7f50115da640 1 -- 172.21.15.60:0/491414182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500c151280 con 0x7f4ff8004d50 2024-04-25T10:09:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.941+0000 7f50115da640 1 -- 172.21.15.60:0/491414182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500c074770 con 0x7f4ff8004d50 2024-04-25T10:09:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.941+0000 7f50037fe640 1 -- 172.21.15.60:0/491414182 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ffc005270 con 0x7f4ff8004d50 2024-04-25T10:09:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.941+0000 7f50037fe640 1 --2- 172.21.15.60:0/491414182 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fec066f60 0x7f4fec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.941+0000 7f50037fe640 1 -- 172.21.15.60:0/491414182 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ffc0850a0 con 0x7f4ff8004d50 2024-04-25T10:09:54.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.941+0000 7f500a7fc640 1 --2- 172.21.15.60:0/491414182 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fec066f60 0x7f4fec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.941+0000 7f500a7fc640 1 --2- 172.21.15.60:0/491414182 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fec066f60 0x7f4fec069420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f500c155c20 tx=0x7f4ff4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:54.945+0000 7f50037fe640 1 -- 172.21.15.60:0/491414182 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ffc0560b0 con 0x7f4ff8004d50 2024-04-25T10:09:55.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.109+0000 7f50115da640 1 -- 172.21.15.60:0/491414182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f500c0d0180 con 0x7f4ff8004d50 2024-04-25T10:09:55.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.109+0000 7f50037fe640 1 -- 172.21.15.60:0/491414182 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16680 (secure 0 0 0) 0x7f4ffc0559b0 con 0x7f4ff8004d50 2024-04-25T10:09:55.119 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:09:55.119 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":69,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:09:45.353654+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:42.005281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:42.710028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.113+0000 7f50115da640 1 -- 172.21.15.60:0/491414182 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fec066f60 msgr2=0x7f4fec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.113+0000 7f50115da640 1 --2- 172.21.15.60:0/491414182 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fec066f60 0x7f4fec069420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f500c155c20 tx=0x7f4ff4008040 comp rx=0 tx=0).stop 2024-04-25T10:09:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.113+0000 7f50115da640 1 -- 172.21.15.60:0/491414182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ff8004d50 msgr2=0x7f500c1f3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.113+0000 7f50115da640 1 --2- 172.21.15.60:0/491414182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ff8004d50 0x7f500c1f3f90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00e8f0 tx=0x7f4ffc00edb0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.113+0000 7f50115da640 1 -- 172.21.15.60:0/491414182 shutdown_connections 2024-04-25T10:09:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.113+0000 7f50115da640 1 --2- 172.21.15.60:0/491414182 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4fec066f60 0x7f4fec069420 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.113+0000 7f50115da640 1 --2- 172.21.15.60:0/491414182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f500c152800 0x7f500c152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.113+0000 7f50115da640 1 --2- 172.21.15.60:0/491414182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f500c1507e0 0x7f500c15cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.113+0000 7f50115da640 1 --2- 172.21.15.60:0/491414182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ff8004d50 0x7f500c1f3f90 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.113+0000 7f50115da640 1 -- 172.21.15.60:0/491414182 >> 172.21.15.60:0/491414182 conn(0x7f500c0b12d0 msgr2=0x7f500c0b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.113+0000 7f50115da640 1 -- 172.21.15.60:0/491414182 shutdown_connections 2024-04-25T10:09:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.113+0000 7f50115da640 1 -- 172.21.15.60:0/491414182 wait complete. 2024-04-25T10:09:55.139 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:09:42.005281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:09:42.710028+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:55.139 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ed62ec640 1 Processor -- start 2024-04-25T10:09:55.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ed62ec640 1 -- start start 2024-04-25T10:09:55.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ed62ec640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ed015dea0 0x7f3ed00ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ed62ec640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3ed00ceda0 0x7f3ed00d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ed62ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3ed00d0ea0 0x7f3ed00b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ed62ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed00b9670 con 0x7f3ed00d0ea0 2024-04-25T10:09:55.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ed62ec640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed00d1300 con 0x7f3ed00ceda0 2024-04-25T10:09:55.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ed62ec640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00d1bb0 con 0x7f3ed015dea0 2024-04-25T10:09:55.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ed4889640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3ed00d0ea0 0x7f3ed00b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ed4889640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3ed00d0ea0 0x7f3ed00b6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36912/0 (socket says 172.21.15.60:36912) 2024-04-25T10:09:55.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ed4889640 1 -- 172.21.15.60:0/2187129017 learned_addr learned my addr 172.21.15.60:0/2187129017 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:55.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ecffff640 1 --2- 172.21.15.60:0/2187129017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ed015dea0 0x7f3ed00ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ecf7fe640 1 --2- 172.21.15.60:0/2187129017 >> v2:172.21.15.100:3300/0 conn(0x7f3ed00ceda0 0x7f3ed00d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ed4889640 1 -- 172.21.15.60:0/2187129017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ed015dea0 msgr2=0x7f3ed00ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ed4889640 1 --2- 172.21.15.60:0/2187129017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ed015dea0 0x7f3ed00ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ed4889640 1 -- 172.21.15.60:0/2187129017 >> v2:172.21.15.100:3300/0 conn(0x7f3ed00ceda0 msgr2=0x7f3ed00d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ed4889640 1 --2- 172.21.15.60:0/2187129017 >> v2:172.21.15.100:3300/0 conn(0x7f3ed00ceda0 0x7f3ed00d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.261+0000 7f3ed4889640 1 -- 172.21.15.60:0/2187129017 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed00b85e0 con 0x7f3ed00d0ea0 2024-04-25T10:09:55.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed4889640 1 --2- 172.21.15.60:0/2187129017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed00d0ea0 0x7f3ed00b6ac0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3ed006d0b0 tx=0x7f3ec801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d11b9c39968dfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3eceffd640 1 -- 172.21.15.60:0/2187129017 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec80a5750 con 0x7f3ed00d0ea0 2024-04-25T10:09:55.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3eceffd640 1 -- 172.21.15.60:0/2187129017 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec80a58f0 con 0x7f3ed00d0ea0 2024-04-25T10:09:55.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3eceffd640 1 -- 172.21.15.60:0/2187129017 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec80a5cc0 con 0x7f3ed00d0ea0 2024-04-25T10:09:55.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 -- 172.21.15.60:0/2187129017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed00d0ea0 msgr2=0x7f3ed00b6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 --2- 172.21.15.60:0/2187129017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed00d0ea0 0x7f3ed00b6ac0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3ed006d0b0 tx=0x7f3ec801c9c0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 -- 172.21.15.60:0/2187129017 shutdown_connections 2024-04-25T10:09:55.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 --2- 172.21.15.60:0/2187129017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed00d0ea0 0x7f3ed00b6ac0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 --2- 172.21.15.60:0/2187129017 >> v2:172.21.15.100:3300/0 conn(0x7f3ed00ceda0 0x7f3ed00d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 --2- 172.21.15.60:0/2187129017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ed015dea0 0x7f3ed00ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 -- 172.21.15.60:0/2187129017 >> 172.21.15.60:0/2187129017 conn(0x7f3ed006c4d0 msgr2=0x7f3ed006c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:55.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 -- 172.21.15.60:0/2187129017 shutdown_connections 2024-04-25T10:09:55.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 -- 172.21.15.60:0/2187129017 wait complete. 2024-04-25T10:09:55.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 Processor -- start 2024-04-25T10:09:55.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 -- start start 2024-04-25T10:09:55.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ed00ceda0 0x7f3ed015e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ed00d0ea0 0x7f3ed00c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed01602f0 0x7f3ed00d3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00d2340 con 0x7f3ed01602f0 2024-04-25T10:09:55.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00b8e80 con 0x7f3ed00ceda0 2024-04-25T10:09:55.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed62ec640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00d24c0 con 0x7f3ed00d0ea0 2024-04-25T10:09:55.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed4889640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed01602f0 0x7f3ed00d3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed4889640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed01602f0 0x7f3ed00d3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36922/0 (socket says 172.21.15.60:36922) 2024-04-25T10:09:55.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed4889640 1 -- 172.21.15.60:0/398463050 learned_addr learned my addr 172.21.15.60:0/398463050 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:55.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ecffff640 1 --2- 172.21.15.60:0/398463050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ed00ceda0 0x7f3ed015e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ecf7fe640 1 --2- 172.21.15.60:0/398463050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ed00d0ea0 0x7f3ed00c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed4889640 1 -- 172.21.15.60:0/398463050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ed00d0ea0 msgr2=0x7f3ed00c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed4889640 1 --2- 172.21.15.60:0/398463050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ed00d0ea0 0x7f3ed00c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed4889640 1 -- 172.21.15.60:0/398463050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ed00ceda0 msgr2=0x7f3ed015e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed4889640 1 --2- 172.21.15.60:0/398463050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ed00ceda0 0x7f3ed015e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ed4889640 1 -- 172.21.15.60:0/398463050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed01607a0 con 0x7f3ed01602f0 2024-04-25T10:09:55.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.265+0000 7f3ecf7fe640 1 --2- 172.21.15.60:0/398463050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ed00d0ea0 0x7f3ed00c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:55.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.269+0000 7f3ed4889640 1 --2- 172.21.15.60:0/398463050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed01602f0 0x7f3ed00d3af0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3ec801b070 tx=0x7f3ec8019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.269+0000 7f3eccff9640 1 -- 172.21.15.60:0/398463050 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec80b8670 con 0x7f3ed01602f0 2024-04-25T10:09:55.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.269+0000 7f3eccff9640 1 -- 172.21.15.60:0/398463050 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec80be070 con 0x7f3ed01602f0 2024-04-25T10:09:55.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.269+0000 7f3ed62ec640 1 -- 172.21.15.60:0/398463050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed015e870 con 0x7f3ed01602f0 2024-04-25T10:09:55.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.269+0000 7f3eccff9640 1 -- 172.21.15.60:0/398463050 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec80b8a70 con 0x7f3ed01602f0 2024-04-25T10:09:55.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.269+0000 7f3ed62ec640 1 -- 172.21.15.60:0/398463050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed015ee30 con 0x7f3ed01602f0 2024-04-25T10:09:55.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.269+0000 7f3ed62ec640 1 -- 172.21.15.60:0/398463050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed0074770 con 0x7f3ed01602f0 2024-04-25T10:09:55.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.269+0000 7f3eccff9640 1 -- 172.21.15.60:0/398463050 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ec80b8c10 con 0x7f3ed01602f0 2024-04-25T10:09:55.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.269+0000 7f3eccff9640 1 --2- 172.21.15.60:0/398463050 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3eb0066f60 0x7f3eb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.269+0000 7f3eccff9640 1 -- 172.21.15.60:0/398463050 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ec8120290 con 0x7f3ed01602f0 2024-04-25T10:09:55.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.269+0000 7f3ecffff640 1 --2- 172.21.15.60:0/398463050 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3eb0066f60 0x7f3eb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.269+0000 7f3ecffff640 1 --2- 172.21.15.60:0/398463050 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3eb0066f60 0x7f3eb0069420 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000b930 tx=0x7f3ec0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.273+0000 7f3eccff9640 1 -- 172.21.15.60:0/398463050 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ec80165e0 con 0x7f3ed01602f0 2024-04-25T10:09:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.433+0000 7f3ed62ec640 1 -- 172.21.15.60:0/398463050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ed00b7db0 con 0x7f3ed01602f0 2024-04-25T10:09:55.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.437+0000 7f3eccff9640 1 -- 172.21.15.60:0/398463050 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ec80f1220 con 0x7f3ed01602f0 2024-04-25T10:09:55.444 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:09:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.437+0000 7f3ed62ec640 1 -- 172.21.15.60:0/398463050 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3eb0066f60 msgr2=0x7f3eb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.437+0000 7f3ed62ec640 1 --2- 172.21.15.60:0/398463050 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3eb0066f60 0x7f3eb0069420 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000b930 tx=0x7f3ec0008040 comp rx=0 tx=0).stop 2024-04-25T10:09:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.437+0000 7f3ed62ec640 1 -- 172.21.15.60:0/398463050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed01602f0 msgr2=0x7f3ed00d3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.437+0000 7f3ed62ec640 1 --2- 172.21.15.60:0/398463050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed01602f0 0x7f3ed00d3af0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3ec801b070 tx=0x7f3ec8019760 comp rx=0 tx=0).stop 2024-04-25T10:09:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.437+0000 7f3ed62ec640 1 -- 172.21.15.60:0/398463050 shutdown_connections 2024-04-25T10:09:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.437+0000 7f3ed62ec640 1 --2- 172.21.15.60:0/398463050 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3eb0066f60 0x7f3eb0069420 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.437+0000 7f3ed62ec640 1 --2- 172.21.15.60:0/398463050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ed01602f0 0x7f3ed00d3af0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.437+0000 7f3ed62ec640 1 --2- 172.21.15.60:0/398463050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ed00d0ea0 0x7f3ed00c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.437+0000 7f3ed62ec640 1 --2- 172.21.15.60:0/398463050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ed00ceda0 0x7f3ed015e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.437+0000 7f3ed62ec640 1 -- 172.21.15.60:0/398463050 >> 172.21.15.60:0/398463050 conn(0x7f3ed006c4d0 msgr2=0x7f3ed0154aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.437+0000 7f3ed62ec640 1 -- 172.21.15.60:0/398463050 shutdown_connections 2024-04-25T10:09:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.437+0000 7f3ed62ec640 1 -- 172.21.15.60:0/398463050 wait complete. 2024-04-25T10:09:55.463 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:09:55.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.585+0000 7f098819e640 1 Processor -- start 2024-04-25T10:09:55.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.585+0000 7f098819e640 1 -- start start 2024-04-25T10:09:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.589+0000 7f098819e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0980160080 0x7f098015f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.589+0000 7f098819e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f098015fa20 0x7f09800ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.589+0000 7f098819e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0980144a40 con 0x7f09800cbcf0 2024-04-25T10:09:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.589+0000 7f098819e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098015fe90 con 0x7f0980160080 2024-04-25T10:09:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.589+0000 7f098819e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09800b2dc0 con 0x7f098015fa20 2024-04-25T10:09:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.589+0000 7f0985739640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0980160080 0x7f098015f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.589+0000 7f0985739640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0980160080 0x7f098015f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36926/0 (socket says 172.21.15.60:36926) 2024-04-25T10:09:55.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.589+0000 7f0985739640 1 -- 172.21.15.60:0/2296830328 learned_addr learned my addr 172.21.15.60:0/2296830328 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:55.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.589+0000 7f098673b640 1 --2- 172.21.15.60:0/2296830328 >> v2:172.21.15.100:3300/0 conn(0x7f098015fa20 0x7f09800ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.589+0000 7f0985739640 1 -- 172.21.15.60:0/2296830328 >> v2:172.21.15.100:3300/0 conn(0x7f098015fa20 msgr2=0x7f09800ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.589+0000 7f0985739640 1 --2- 172.21.15.60:0/2296830328 >> v2:172.21.15.100:3300/0 conn(0x7f098015fa20 0x7f09800ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.589+0000 7f0985739640 1 -- 172.21.15.60:0/2296830328 >> v1:172.21.15.60:6789/0 conn(0x7f09800cbcf0 legacy=0x7f0980071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.589+0000 7f0985739640 1 -- 172.21.15.60:0/2296830328 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09800cc9f0 con 0x7f0980160080 2024-04-25T10:09:55.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.589+0000 7f0984f38640 1 -- 172.21.15.60:0/2296830328 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124246781 0 0) 0x7f0980144a40 con 0x7f09800cbcf0 2024-04-25T10:09:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.589+0000 7f098673b640 1 --2- 172.21.15.60:0/2296830328 >> v2:172.21.15.100:3300/0 conn(0x7f098015fa20 0x7f09800ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:55.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f0985739640 1 --2- 172.21.15.60:0/2296830328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0980160080 0x7f098015f4e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f09800d03a0 tx=0x7f097001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becc119f9404c927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f0984f38640 1 -- 172.21.15.60:0/2296830328 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09700ab610 con 0x7f0980160080 2024-04-25T10:09:55.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f0984f38640 1 -- 172.21.15.60:0/2296830328 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09700b0070 con 0x7f0980160080 2024-04-25T10:09:55.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f0984f38640 1 -- 172.21.15.60:0/2296830328 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09700ab9d0 con 0x7f0980160080 2024-04-25T10:09:55.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098819e640 1 -- 172.21.15.60:0/2296830328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0980160080 msgr2=0x7f098015f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098819e640 1 --2- 172.21.15.60:0/2296830328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0980160080 0x7f098015f4e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f09800d03a0 tx=0x7f097001c520 comp rx=0 tx=0).stop 2024-04-25T10:09:55.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098819e640 1 -- 172.21.15.60:0/2296830328 shutdown_connections 2024-04-25T10:09:55.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098819e640 1 --2- 172.21.15.60:0/2296830328 >> v2:172.21.15.100:3300/0 conn(0x7f098015fa20 0x7f09800ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098819e640 1 --2- 172.21.15.60:0/2296830328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0980160080 0x7f098015f4e0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098819e640 1 -- 172.21.15.60:0/2296830328 >> 172.21.15.60:0/2296830328 conn(0x7f09800670b0 msgr2=0x7f098006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:55.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098819e640 1 -- 172.21.15.60:0/2296830328 shutdown_connections 2024-04-25T10:09:55.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098819e640 1 -- 172.21.15.60:0/2296830328 wait complete. 2024-04-25T10:09:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098819e640 1 Processor -- start 2024-04-25T10:09:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098819e640 1 -- start start 2024-04-25T10:09:55.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098819e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09800cbcf0 0x7f0980158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098819e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f098015fa20 0x7f0980140a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098819e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0980160080 0x7f0980140f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098819e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09800ce5c0 con 0x7f09800cbcf0 2024-04-25T10:09:55.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098819e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09800cc110 con 0x7f0980160080 2024-04-25T10:09:55.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098819e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09800ce740 con 0x7f098015fa20 2024-04-25T10:09:55.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f0985f3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09800cbcf0 0x7f0980158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f0985f3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09800cbcf0 0x7f0980158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36934/0 (socket says 172.21.15.60:36934) 2024-04-25T10:09:55.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f0985f3a640 1 -- 172.21.15.60:0/3755836988 learned_addr learned my addr 172.21.15.60:0/3755836988 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:55.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f0985739640 1 --2- 172.21.15.60:0/3755836988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f098015fa20 0x7f0980140a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098673b640 1 --2- 172.21.15.60:0/3755836988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0980160080 0x7f0980140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f0985f3a640 1 -- 172.21.15.60:0/3755836988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f098015fa20 msgr2=0x7f0980140a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f0985f3a640 1 --2- 172.21.15.60:0/3755836988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f098015fa20 0x7f0980140a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f0985f3a640 1 -- 172.21.15.60:0/3755836988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0980160080 msgr2=0x7f0980140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f0985f3a640 1 --2- 172.21.15.60:0/3755836988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0980160080 0x7f0980140f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f0985f3a640 1 -- 172.21.15.60:0/3755836988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098013e6c0 con 0x7f09800cbcf0 2024-04-25T10:09:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f0985739640 1 --2- 172.21.15.60:0/3755836988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f098015fa20 0x7f0980140a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f098673b640 1 --2- 172.21.15.60:0/3755836988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0980160080 0x7f0980140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.593+0000 7f0985f3a640 1 --2- 172.21.15.60:0/3755836988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09800cbcf0 0x7f0980158da0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f097c001ec0 tx=0x7f097c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.597+0000 7f09767fc640 1 -- 172.21.15.60:0/3755836988 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c017070 con 0x7f09800cbcf0 2024-04-25T10:09:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.597+0000 7f09767fc640 1 -- 172.21.15.60:0/3755836988 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097c003600 con 0x7f09800cbcf0 2024-04-25T10:09:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.597+0000 7f09767fc640 1 -- 172.21.15.60:0/3755836988 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c0039f0 con 0x7f09800cbcf0 2024-04-25T10:09:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.597+0000 7f098819e640 1 -- 172.21.15.60:0/3755836988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0980139ac0 con 0x7f09800cbcf0 2024-04-25T10:09:55.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.597+0000 7f098819e640 1 -- 172.21.15.60:0/3755836988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098013a080 con 0x7f09800cbcf0 2024-04-25T10:09:55.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.597+0000 7f098819e640 1 -- 172.21.15.60:0/3755836988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098006bba0 con 0x7f09800cbcf0 2024-04-25T10:09:55.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.597+0000 7f09767fc640 1 -- 172.21.15.60:0/3755836988 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f097c01a070 con 0x7f09800cbcf0 2024-04-25T10:09:55.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.597+0000 7f09767fc640 1 --2- 172.21.15.60:0/3755836988 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0964066f60 0x7f0964069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.597+0000 7f09767fc640 1 -- 172.21.15.60:0/3755836988 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f097c084390 con 0x7f09800cbcf0 2024-04-25T10:09:55.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.597+0000 7f0985739640 1 --2- 172.21.15.60:0/3755836988 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0964066f60 0x7f0964069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.597+0000 7f0985739640 1 --2- 172.21.15.60:0/3755836988 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0964066f60 0x7f0964069420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f09700af000 tx=0x7f09700a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.601+0000 7f09767fc640 1 -- 172.21.15.60:0/3755836988 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f097c055430 con 0x7f09800cbcf0 2024-04-25T10:09:55.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.757+0000 7f098819e640 1 -- 172.21.15.60:0/3755836988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f09800b2970 con 0x7f09800cbcf0 2024-04-25T10:09:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.761+0000 7f09767fc640 1 -- 172.21.15.60:0/3755836988 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f097c054d30 con 0x7f09800cbcf0 2024-04-25T10:09:55.768 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:09:55.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.761+0000 7f098819e640 1 -- 172.21.15.60:0/3755836988 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0964066f60 msgr2=0x7f0964069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.761+0000 7f098819e640 1 --2- 172.21.15.60:0/3755836988 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0964066f60 0x7f0964069420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f09700af000 tx=0x7f09700a7000 comp rx=0 tx=0).stop 2024-04-25T10:09:55.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.761+0000 7f098819e640 1 -- 172.21.15.60:0/3755836988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09800cbcf0 msgr2=0x7f0980158da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.761+0000 7f098819e640 1 --2- 172.21.15.60:0/3755836988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09800cbcf0 0x7f0980158da0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f097c001ec0 tx=0x7f097c00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.761+0000 7f098819e640 1 -- 172.21.15.60:0/3755836988 shutdown_connections 2024-04-25T10:09:55.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.761+0000 7f098819e640 1 --2- 172.21.15.60:0/3755836988 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0964066f60 0x7f0964069420 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.761+0000 7f098819e640 1 --2- 172.21.15.60:0/3755836988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0980160080 0x7f0980140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.761+0000 7f098819e640 1 --2- 172.21.15.60:0/3755836988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f098015fa20 0x7f0980140a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.761+0000 7f098819e640 1 --2- 172.21.15.60:0/3755836988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09800cbcf0 0x7f0980158da0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.761+0000 7f098819e640 1 -- 172.21.15.60:0/3755836988 >> 172.21.15.60:0/3755836988 conn(0x7f09800670b0 msgr2=0x7f09800b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:55.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.761+0000 7f098819e640 1 -- 172.21.15.60:0/3755836988 shutdown_connections 2024-04-25T10:09:55.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.761+0000 7f098819e640 1 -- 172.21.15.60:0/3755836988 wait complete. 2024-04-25T10:09:55.788 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:09:55.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50fdc8d640 1 Processor -- start 2024-04-25T10:09:55.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50fdc8d640 1 -- start start 2024-04-25T10:09:55.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50fdc8d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f50f815de90 0x7f50f80b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50fdc8d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50f80d0320 0x7f50f80d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50fdc8d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f806d200 con 0x7f50f80ce290 2024-04-25T10:09:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50fdc8d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f80cf2a0 con 0x7f50f80d0320 2024-04-25T10:09:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50fdc8d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f80cefc0 con 0x7f50f815de90 2024-04-25T10:09:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50f6ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50f80d0320 0x7f50f80d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50f7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f50f80ce290 0x7f50f80ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44672/0 (socket says 172.21.15.60:44672) 2024-04-25T10:09:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50f7fff640 1 -- 172.21.15.60:0/1977303891 learned_addr learned my addr 172.21.15.60:0/1977303891 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50f77fe640 1 --2- 172.21.15.60:0/1977303891 >> v2:172.21.15.100:3300/0 conn(0x7f50f815de90 0x7f50f80b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50f67fc640 1 -- 172.21.15.60:0/1977303891 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140902183 0 0) 0x7f50f806d200 con 0x7f50f80ce290 2024-04-25T10:09:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50f67fc640 1 -- 172.21.15.60:0/1977303891 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50e0003530 con 0x7f50f80ce290 2024-04-25T10:09:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50f6ffd640 1 -- 172.21.15.60:0/1977303891 >> v2:172.21.15.100:3300/0 conn(0x7f50f815de90 msgr2=0x7f50f80b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50f6ffd640 1 --2- 172.21.15.60:0/1977303891 >> v2:172.21.15.100:3300/0 conn(0x7f50f815de90 0x7f50f80b6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50f6ffd640 1 -- 172.21.15.60:0/1977303891 >> v1:172.21.15.60:6789/0 conn(0x7f50f80ce290 legacy=0x7f50f80ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.901+0000 7f50f6ffd640 1 -- 172.21.15.60:0/1977303891 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f80d1d50 con 0x7f50f80d0320 2024-04-25T10:09:55.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50f6ffd640 1 --2- 172.21.15.60:0/1977303891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f80d0320 0x7f50f80d07a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f50f808bd20 tx=0x7f50ec01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e6a86d74e0826b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50f67fc640 1 -- 172.21.15.60:0/1977303891 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec0b0070 con 0x7f50f80d0320 2024-04-25T10:09:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50f67fc640 1 -- 172.21.15.60:0/1977303891 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ec0ab600 con 0x7f50f80d0320 2024-04-25T10:09:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50f67fc640 1 -- 172.21.15.60:0/1977303891 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec0ab9d0 con 0x7f50f80d0320 2024-04-25T10:09:55.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50fdc8d640 1 -- 172.21.15.60:0/1977303891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f80d0320 msgr2=0x7f50f80d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50fdc8d640 1 --2- 172.21.15.60:0/1977303891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f80d0320 0x7f50f80d07a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f50f808bd20 tx=0x7f50ec01c480 comp rx=0 tx=0).stop 2024-04-25T10:09:55.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50fdc8d640 1 -- 172.21.15.60:0/1977303891 shutdown_connections 2024-04-25T10:09:55.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50fdc8d640 1 --2- 172.21.15.60:0/1977303891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f80d0320 0x7f50f80d07a0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50fdc8d640 1 --2- 172.21.15.60:0/1977303891 >> v2:172.21.15.100:3300/0 conn(0x7f50f815de90 0x7f50f80b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50fdc8d640 1 -- 172.21.15.60:0/1977303891 >> 172.21.15.60:0/1977303891 conn(0x7f50f806c090 msgr2=0x7f50f806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:55.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50fdc8d640 1 -- 172.21.15.60:0/1977303891 shutdown_connections 2024-04-25T10:09:55.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50fdc8d640 1 -- 172.21.15.60:0/1977303891 wait complete. 2024-04-25T10:09:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50fdc8d640 1 Processor -- start 2024-04-25T10:09:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50fdc8d640 1 -- start start 2024-04-25T10:09:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50fdc8d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f80ce290 0x7f50f80c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50fdc8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f80d0320 0x7f50f80b9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50fdc8d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f815de90 0x7f50f80b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50fdc8d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f80d3410 con 0x7f50f815de90 2024-04-25T10:09:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50fdc8d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f80d3290 con 0x7f50f80d0320 2024-04-25T10:09:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50fdc8d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f80d3590 con 0x7f50f80ce290 2024-04-25T10:09:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50f7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f815de90 0x7f50f80b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50f7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f815de90 0x7f50f80b9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36956/0 (socket says 172.21.15.60:36956) 2024-04-25T10:09:55.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50f7fff640 1 -- 172.21.15.60:0/221010287 learned_addr learned my addr 172.21.15.60:0/221010287 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:55.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50f6ffd640 1 --2- 172.21.15.60:0/221010287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f80d0320 0x7f50f80b9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50f77fe640 1 --2- 172.21.15.60:0/221010287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f80ce290 0x7f50f80c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50f7fff640 1 -- 172.21.15.60:0/221010287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f80ce290 msgr2=0x7f50f80c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50f7fff640 1 --2- 172.21.15.60:0/221010287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f80ce290 0x7f50f80c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50f7fff640 1 -- 172.21.15.60:0/221010287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f80d0320 msgr2=0x7f50f80b9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50f7fff640 1 --2- 172.21.15.60:0/221010287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f80d0320 0x7f50f80b9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50f7fff640 1 -- 172.21.15.60:0/221010287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f80c99d0 con 0x7f50f815de90 2024-04-25T10:09:55.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50f6ffd640 1 --2- 172.21.15.60:0/221010287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f80d0320 0x7f50f80b9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:55.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.905+0000 7f50f77fe640 1 --2- 172.21.15.60:0/221010287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f80ce290 0x7f50f80c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:55.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.909+0000 7f50f7fff640 1 --2- 172.21.15.60:0/221010287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f815de90 0x7f50f80b9ff0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f50f000cb40 tx=0x7f50f000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.909+0000 7f50dffff640 1 -- 172.21.15.60:0/221010287 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0017070 con 0x7f50f815de90 2024-04-25T10:09:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.909+0000 7f50dffff640 1 -- 172.21.15.60:0/221010287 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f000aca0 con 0x7f50f815de90 2024-04-25T10:09:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.909+0000 7f50fdc8d640 1 -- 172.21.15.60:0/221010287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f80c9c40 con 0x7f50f815de90 2024-04-25T10:09:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.909+0000 7f50dffff640 1 -- 172.21.15.60:0/221010287 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0012620 con 0x7f50f815de90 2024-04-25T10:09:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.909+0000 7f50fdc8d640 1 -- 172.21.15.60:0/221010287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f8147ff0 con 0x7f50f815de90 2024-04-25T10:09:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.909+0000 7f50fdc8d640 1 -- 172.21.15.60:0/221010287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f8074770 con 0x7f50f815de90 2024-04-25T10:09:55.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.909+0000 7f50dffff640 1 -- 172.21.15.60:0/221010287 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f50f001a070 con 0x7f50f815de90 2024-04-25T10:09:55.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.909+0000 7f50dffff640 1 --2- 172.21.15.60:0/221010287 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f50e4066f90 0x7f50e4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.909+0000 7f50dffff640 1 -- 172.21.15.60:0/221010287 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50f0084e20 con 0x7f50f815de90 2024-04-25T10:09:55.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.909+0000 7f50f77fe640 1 --2- 172.21.15.60:0/221010287 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f50e4066f90 0x7f50e4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.909+0000 7f50f77fe640 1 --2- 172.21.15.60:0/221010287 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f50e4066f90 0x7f50e4069450 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f50f814d360 tx=0x7f50e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:55.913+0000 7f50dffff640 1 -- 172.21.15.60:0/221010287 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50f0055e30 con 0x7f50f815de90 2024-04-25T10:09:56.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.069+0000 7f50fdc8d640 1 -- 172.21.15.60:0/221010287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f50f80d1780 con 0x7f50f815de90 2024-04-25T10:09:56.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.069+0000 7f50dffff640 1 -- 172.21.15.60:0/221010287 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7f50f0055730 con 0x7f50f815de90 2024-04-25T10:09:56.077 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:09:56.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.069+0000 7f50fdc8d640 1 -- 172.21.15.60:0/221010287 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f50e4066f90 msgr2=0x7f50e4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.069+0000 7f50fdc8d640 1 --2- 172.21.15.60:0/221010287 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f50e4066f90 0x7f50e4069450 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f50f814d360 tx=0x7f50e8008040 comp rx=0 tx=0).stop 2024-04-25T10:09:56.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.069+0000 7f50fdc8d640 1 -- 172.21.15.60:0/221010287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f815de90 msgr2=0x7f50f80b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.069+0000 7f50fdc8d640 1 --2- 172.21.15.60:0/221010287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f815de90 0x7f50f80b9ff0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f50f000cb40 tx=0x7f50f000a060 comp rx=0 tx=0).stop 2024-04-25T10:09:56.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.069+0000 7f50fdc8d640 1 -- 172.21.15.60:0/221010287 shutdown_connections 2024-04-25T10:09:56.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.069+0000 7f50fdc8d640 1 --2- 172.21.15.60:0/221010287 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f50e4066f90 0x7f50e4069450 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.069+0000 7f50fdc8d640 1 --2- 172.21.15.60:0/221010287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f815de90 0x7f50f80b9ff0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.069+0000 7f50fdc8d640 1 --2- 172.21.15.60:0/221010287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f80d0320 0x7f50f80b9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.069+0000 7f50fdc8d640 1 --2- 172.21.15.60:0/221010287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f80ce290 0x7f50f80c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.069+0000 7f50fdc8d640 1 -- 172.21.15.60:0/221010287 >> 172.21.15.60:0/221010287 conn(0x7f50f806c090 msgr2=0x7f50f814a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:56.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.069+0000 7f50fdc8d640 1 -- 172.21.15.60:0/221010287 shutdown_connections 2024-04-25T10:09:56.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.069+0000 7f50fdc8d640 1 -- 172.21.15.60:0/221010287 wait complete. 2024-04-25T10:09:56.095 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:09:56.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f5370dce640 1 Processor -- start 2024-04-25T10:09:56.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f5370dce640 1 -- start start 2024-04-25T10:09:56.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f5370dce640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f536c0aed10 0x7f536c0ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f5370dce640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f536c0b2d90 con 0x7f536c06c330 2024-04-25T10:09:56.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f5370dce640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f536c0b3070 con 0x7f536c159e70 2024-04-25T10:09:56.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f5370dce640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536c0b4110 con 0x7f536c0aed10 2024-04-25T10:09:56.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f5369d9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f536c06c330 0x7f536c0acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44674/0 (socket says 172.21.15.60:44674) 2024-04-25T10:09:56.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f5369d9b640 1 -- 172.21.15.60:0/1182715803 learned_addr learned my addr 172.21.15.60:0/1182715803 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:09:56.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536ad9d640 1 --2- 172.21.15.60:0/1182715803 >> v2:172.21.15.100:3300/0 conn(0x7f536c0aed10 0x7f536c0ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536959a640 1 -- 172.21.15.60:0/1182715803 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787687045 0 0) 0x7f536c0b2d90 con 0x7f536c06c330 2024-04-25T10:09:56.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536959a640 1 -- 172.21.15.60:0/1182715803 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5358003530 con 0x7f536c06c330 2024-04-25T10:09:56.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536959a640 1 -- 172.21.15.60:0/1182715803 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3928781884 0 0) 0x7f5358003530 con 0x7f536c06c330 2024-04-25T10:09:56.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536959a640 1 -- 172.21.15.60:0/1182715803 >> v2:172.21.15.100:3300/0 conn(0x7f536c0aed10 msgr2=0x7f536c0ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:56.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536959a640 1 --2- 172.21.15.60:0/1182715803 >> v2:172.21.15.100:3300/0 conn(0x7f536c0aed10 0x7f536c0ad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536959a640 1 -- 172.21.15.60:0/1182715803 >> v1:172.21.15.100:6789/0 conn(0x7f536c159e70 legacy=0x7f536c06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536959a640 1 -- 172.21.15.60:0/1182715803 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c0b3350 con 0x7f536c06c330 2024-04-25T10:09:56.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536959a640 1 -- 172.21.15.60:0/1182715803 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f5354001a00 con 0x7f536c06c330 2024-04-25T10:09:56.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536959a640 1 -- 172.21.15.60:0/1182715803 >> v1:172.21.15.60:6789/0 conn(0x7f536c06c330 legacy=0x7f536c0acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536959a640 1 --2- 172.21.15.60:0/1182715803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53580039c0 0x7f5358003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536959a640 1 --2- 172.21.15.60:0/1182715803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5358004810 0x7f5358004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536959a640 1 --2- 172.21.15.60:0/1182715803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53580051d0 0x7f535800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536959a640 1 -- 172.21.15.60:0/1182715803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c0b4110 con 0x7f53580039c0 2024-04-25T10:09:56.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536959a640 1 -- 172.21.15.60:0/1182715803 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535800c070 con 0x7f5358004810 2024-04-25T10:09:56.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536959a640 1 -- 172.21.15.60:0/1182715803 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535800c270 con 0x7f53580051d0 2024-04-25T10:09:56.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536a59c640 1 --2- 172.21.15.60:0/1182715803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53580039c0 0x7f5358003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f5369d9b640 1 --2- 172.21.15.60:0/1182715803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5358004810 0x7f5358004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536ad9d640 1 --2- 172.21.15.60:0/1182715803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53580051d0 0x7f535800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536a59c640 1 -- 172.21.15.60:0/1182715803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53580051d0 msgr2=0x7f535800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:56.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536a59c640 1 --2- 172.21.15.60:0/1182715803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53580051d0 0x7f535800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536a59c640 1 -- 172.21.15.60:0/1182715803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5358004810 msgr2=0x7f5358004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:56.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536a59c640 1 --2- 172.21.15.60:0/1182715803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5358004810 0x7f5358004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536a59c640 1 -- 172.21.15.60:0/1182715803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5358003530 con 0x7f53580039c0 2024-04-25T10:09:56.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.213+0000 7f536ad9d640 1 -- 172.21.15.60:0/1182715803 reap_dead start 2024-04-25T10:09:56.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.217+0000 7f536a59c640 1 --2- 172.21.15.60:0/1182715803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53580039c0 0x7f5358003de0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f5364001780 tx=0x7f536401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862121f9b9fa5db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:56.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.217+0000 7f536959a640 1 -- 172.21.15.60:0/1182715803 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53640013a0 con 0x7f53580039c0 2024-04-25T10:09:56.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.217+0000 7f536959a640 1 -- 172.21.15.60:0/1182715803 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53640052f0 con 0x7f53580039c0 2024-04-25T10:09:56.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.217+0000 7f536959a640 1 -- 172.21.15.60:0/1182715803 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364003840 con 0x7f53580039c0 2024-04-25T10:09:56.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.217+0000 7f5370dce640 1 -- 172.21.15.60:0/1182715803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53580039c0 msgr2=0x7f5358003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.217+0000 7f5370dce640 1 --2- 172.21.15.60:0/1182715803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53580039c0 0x7f5358003de0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f5364001780 tx=0x7f536401c420 comp rx=0 tx=0).stop 2024-04-25T10:09:56.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.217+0000 7f5370dce640 1 -- 172.21.15.60:0/1182715803 shutdown_connections 2024-04-25T10:09:56.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.217+0000 7f5370dce640 1 --2- 172.21.15.60:0/1182715803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53580039c0 0x7f5358003de0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.217+0000 7f5370dce640 1 -- 172.21.15.60:0/1182715803 >> 172.21.15.60:0/1182715803 conn(0x7f536c071b90 msgr2=0x7f536c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:56.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.217+0000 7f5370dce640 1 -- 172.21.15.60:0/1182715803 shutdown_connections 2024-04-25T10:09:56.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.217+0000 7f5370dce640 1 -- 172.21.15.60:0/1182715803 wait complete. 2024-04-25T10:09:56.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.217+0000 7f5370dce640 1 Processor -- start 2024-04-25T10:09:56.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5370dce640 1 -- start start 2024-04-25T10:09:56.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5370dce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53580039c0 0x7f536c14ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5370dce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c06ac10 0x7f536c06b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5370dce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f536c1581f0 0x7f536c158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5370dce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c06c030 con 0x7f536c06ac10 2024-04-25T10:09:56.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5370dce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c0b3c80 con 0x7f53580039c0 2024-04-25T10:09:56.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5370dce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c06c1b0 con 0x7f536c1581f0 2024-04-25T10:09:56.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5369d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c06ac10 0x7f536c06b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5369d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c06ac10 0x7f536c06b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36984/0 (socket says 172.21.15.60:36984) 2024-04-25T10:09:56.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5369d9b640 1 -- 172.21.15.60:0/3940459035 learned_addr learned my addr 172.21.15.60:0/3940459035 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:09:56.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f536a59c640 1 --2- 172.21.15.60:0/3940459035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53580039c0 0x7f536c14ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f536ad9d640 1 --2- 172.21.15.60:0/3940459035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f536c1581f0 0x7f536c158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5369d9b640 1 -- 172.21.15.60:0/3940459035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f536c1581f0 msgr2=0x7f536c158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5369d9b640 1 --2- 172.21.15.60:0/3940459035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f536c1581f0 0x7f536c158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5369d9b640 1 -- 172.21.15.60:0/3940459035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53580039c0 msgr2=0x7f536c14ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5369d9b640 1 --2- 172.21.15.60:0/3940459035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53580039c0 0x7f536c14ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5369d9b640 1 -- 172.21.15.60:0/3940459035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c158f60 con 0x7f536c06ac10 2024-04-25T10:09:56.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f536ad9d640 1 --2- 172.21.15.60:0/3940459035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f536c1581f0 0x7f536c158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:56.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5369d9b640 1 --2- 172.21.15.60:0/3940459035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c06ac10 0x7f536c06b090 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f53540079e0 tx=0x7f535400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:56.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5362ffd640 1 -- 172.21.15.60:0/3940459035 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354007ad0 con 0x7f536c06ac10 2024-04-25T10:09:56.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5362ffd640 1 -- 172.21.15.60:0/3940459035 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5354018070 con 0x7f536c06ac10 2024-04-25T10:09:56.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5370dce640 1 -- 172.21.15.60:0/3940459035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536c1e59c0 con 0x7f536c06ac10 2024-04-25T10:09:56.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5362ffd640 1 -- 172.21.15.60:0/3940459035 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354013640 con 0x7f536c06ac10 2024-04-25T10:09:56.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5370dce640 1 -- 172.21.15.60:0/3940459035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536c1e5f80 con 0x7f536c06ac10 2024-04-25T10:09:56.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5370dce640 1 -- 172.21.15.60:0/3940459035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536c06b5d0 con 0x7f536c06ac10 2024-04-25T10:09:56.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5362ffd640 1 -- 172.21.15.60:0/3940459035 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f53540137e0 con 0x7f536c06ac10 2024-04-25T10:09:56.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5362ffd640 1 --2- 172.21.15.60:0/3940459035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f534c067100 0x7f534c0695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.221+0000 7f5362ffd640 1 -- 172.21.15.60:0/3940459035 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5354085570 con 0x7f536c06ac10 2024-04-25T10:09:56.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.225+0000 7f536a59c640 1 --2- 172.21.15.60:0/3940459035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f534c067100 0x7f534c0695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.225+0000 7f536a59c640 1 --2- 172.21.15.60:0/3940459035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f534c067100 0x7f534c0695c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5364001750 tx=0x7f53640011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:56.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.229+0000 7f5362ffd640 1 -- 172.21.15.60:0/3940459035 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5354056580 con 0x7f536c06ac10 2024-04-25T10:09:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.401+0000 7f5370dce640 1 -- 172.21.15.60:0/3940459035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f536c06fae0 con 0x7f536c06ac10 2024-04-25T10:09:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.401+0000 7f5362ffd640 1 -- 172.21.15.60:0/3940459035 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5354055e80 con 0x7f536c06ac10 2024-04-25T10:09:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:09:56.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.401+0000 7f5370dce640 1 -- 172.21.15.60:0/3940459035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f534c067100 msgr2=0x7f534c0695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.401+0000 7f5370dce640 1 --2- 172.21.15.60:0/3940459035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f534c067100 0x7f534c0695c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5364001750 tx=0x7f53640011a0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.401+0000 7f5370dce640 1 -- 172.21.15.60:0/3940459035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c06ac10 msgr2=0x7f536c06b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.401+0000 7f5370dce640 1 --2- 172.21.15.60:0/3940459035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c06ac10 0x7f536c06b090 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f53540079e0 tx=0x7f535400e530 comp rx=0 tx=0).stop 2024-04-25T10:09:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.401+0000 7f5370dce640 1 -- 172.21.15.60:0/3940459035 shutdown_connections 2024-04-25T10:09:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.401+0000 7f5370dce640 1 --2- 172.21.15.60:0/3940459035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f534c067100 0x7f534c0695c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.401+0000 7f5370dce640 1 --2- 172.21.15.60:0/3940459035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f536c1581f0 0x7f536c158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.401+0000 7f5370dce640 1 --2- 172.21.15.60:0/3940459035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c06ac10 0x7f536c06b090 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.401+0000 7f5370dce640 1 --2- 172.21.15.60:0/3940459035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53580039c0 0x7f536c14ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.401+0000 7f5370dce640 1 -- 172.21.15.60:0/3940459035 >> 172.21.15.60:0/3940459035 conn(0x7f536c071b90 msgr2=0x7f536c0b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.401+0000 7f5370dce640 1 -- 172.21.15.60:0/3940459035 shutdown_connections 2024-04-25T10:09:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:09:56.401+0000 7f5370dce640 1 -- 172.21.15.60:0/3940459035 wait complete. 2024-04-25T10:09:56.429 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:09:56.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:56.429 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:56.440 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[42402]: fuse finished with error 0 and tester_r 0 2024-04-25T10:09:56.442 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:10:02.544 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:10:02.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:02.545 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:10:02.553 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:02.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f7706d08640 1 Processor -- start 2024-04-25T10:10:02.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f7706d08640 1 -- start start 2024-04-25T10:10:02.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f7706d08640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77000719a0 0x7f77000ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:02.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f7706d08640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7700071fd0 con 0x7f770015dc90 2024-04-25T10:10:02.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f7706d08640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77000b2ce0 con 0x7f77000ae7c0 2024-04-25T10:10:02.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f7706d08640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77000accc0 con 0x7f77000719a0 2024-04-25T10:10:02.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f7704aa4640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f770015dc90 0x7f77000b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49218/0 (socket says 172.21.15.60:49218) 2024-04-25T10:10:02.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f7704aa4640 1 -- 172.21.15.60:0/3493710137 learned_addr learned my addr 172.21.15.60:0/3493710137 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:10:02.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f77052a5640 1 --2- 172.21.15.60:0/3493710137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77000719a0 0x7f77000ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:02.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76ff7fe640 1 -- 172.21.15.60:0/3493710137 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841738615 0 0) 0x7f7700071fd0 con 0x7f770015dc90 2024-04-25T10:10:02.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76ff7fe640 1 -- 172.21.15.60:0/3493710137 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76e8003530 con 0x7f770015dc90 2024-04-25T10:10:02.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76ff7fe640 1 -- 172.21.15.60:0/3493710137 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1617548422 0 0) 0x7f76e8003530 con 0x7f770015dc90 2024-04-25T10:10:02.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76ff7fe640 1 -- 172.21.15.60:0/3493710137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77000719a0 msgr2=0x7f77000ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:02.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76ff7fe640 1 --2- 172.21.15.60:0/3493710137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77000719a0 0x7f77000ad6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76ff7fe640 1 -- 172.21.15.60:0/3493710137 >> v1:172.21.15.100:6789/0 conn(0x7f77000ae7c0 legacy=0x7f7700071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76ff7fe640 1 -- 172.21.15.60:0/3493710137 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7700072290 con 0x7f770015dc90 2024-04-25T10:10:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76ff7fe640 1 -- 172.21.15.60:0/3493710137 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f76f0001680 con 0x7f770015dc90 2024-04-25T10:10:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76ff7fe640 1 -- 172.21.15.60:0/3493710137 >> v1:172.21.15.60:6789/0 conn(0x7f770015dc90 legacy=0x7f77000b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76ff7fe640 1 --2- 172.21.15.60:0/3493710137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e80039f0 0x7f76e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76ff7fe640 1 --2- 172.21.15.60:0/3493710137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76e8004810 0x7f76e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76ff7fe640 1 --2- 172.21.15.60:0/3493710137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77000719a0 0x7f76e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76ff7fe640 1 -- 172.21.15.60:0/3493710137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e800ba80 con 0x7f76e8004810 2024-04-25T10:10:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76ff7fe640 1 -- 172.21.15.60:0/3493710137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e800bc00 con 0x7f76e80039f0 2024-04-25T10:10:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76ff7fe640 1 -- 172.21.15.60:0/3493710137 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e800bdd0 con 0x7f77000719a0 2024-04-25T10:10:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76fffff640 1 --2- 172.21.15.60:0/3493710137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76e8004810 0x7f76e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f7704aa4640 1 --2- 172.21.15.60:0/3493710137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e80039f0 0x7f76e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f77052a5640 1 --2- 172.21.15.60:0/3493710137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77000719a0 0x7f76e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76fffff640 1 -- 172.21.15.60:0/3493710137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77000719a0 msgr2=0x7f76e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76fffff640 1 --2- 172.21.15.60:0/3493710137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77000719a0 0x7f76e80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76fffff640 1 -- 172.21.15.60:0/3493710137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e80039f0 msgr2=0x7f76e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76fffff640 1 --2- 172.21.15.60:0/3493710137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e80039f0 0x7f76e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.721+0000 7f76fffff640 1 -- 172.21.15.60:0/3493710137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e8003530 con 0x7f76e8004810 2024-04-25T10:10:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.725+0000 7f76fffff640 1 --2- 172.21.15.60:0/3493710137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76e8004810 0x7f76e8004c90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f76f4007fd0 tx=0x7f76f401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f335ea78e75b0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:02.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.725+0000 7f76ff7fe640 1 -- 172.21.15.60:0/3493710137 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f40b5070 con 0x7f76e8004810 2024-04-25T10:10:02.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.725+0000 7f76ff7fe640 1 -- 172.21.15.60:0/3493710137 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f40a8830 con 0x7f76e8004810 2024-04-25T10:10:02.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.725+0000 7f76ff7fe640 1 -- 172.21.15.60:0/3493710137 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4001460 con 0x7f76e8004810 2024-04-25T10:10:02.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.725+0000 7f7706d08640 1 -- 172.21.15.60:0/3493710137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76e8004810 msgr2=0x7f76e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:02.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.725+0000 7f7706d08640 1 --2- 172.21.15.60:0/3493710137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76e8004810 0x7f76e8004c90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f76f4007fd0 tx=0x7f76f401ec10 comp rx=0 tx=0).stop 2024-04-25T10:10:02.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.725+0000 7f77052a5640 1 -- 172.21.15.60:0/3493710137 reap_dead start 2024-04-25T10:10:02.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.725+0000 7f7706d08640 1 -- 172.21.15.60:0/3493710137 shutdown_connections 2024-04-25T10:10:02.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.725+0000 7f7706d08640 1 -- 172.21.15.60:0/3493710137 >> 172.21.15.60:0/3493710137 conn(0x7f77000b12d0 msgr2=0x7f77000b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:02.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.725+0000 7f7706d08640 1 -- 172.21.15.60:0/3493710137 shutdown_connections 2024-04-25T10:10:02.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.725+0000 7f7706d08640 1 -- 172.21.15.60:0/3493710137 wait complete. 2024-04-25T10:10:02.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.725+0000 7f7706d08640 1 Processor -- start 2024-04-25T10:10:02.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.725+0000 7f7706d08640 1 -- start start 2024-04-25T10:10:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f7706d08640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e8004810 0x7f7700153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f7706d08640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77001586e0 0x7f7700158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f7706d08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7700154c30 0x7f77001550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f7706d08640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77000b2010 con 0x7f7700154c30 2024-04-25T10:10:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f7706d08640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77000ad260 con 0x7f76e8004810 2024-04-25T10:10:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f7706d08640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77000b2190 con 0x7f77001586e0 2024-04-25T10:10:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f77052a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7700154c30 0x7f77001550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f77052a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7700154c30 0x7f77001550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50786/0 (socket says 172.21.15.60:50786) 2024-04-25T10:10:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f77052a5640 1 -- 172.21.15.60:0/3129147195 learned_addr learned my addr 172.21.15.60:0/3129147195 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f7704aa4640 1 --2- 172.21.15.60:0/3129147195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e8004810 0x7f7700153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f76fffff640 1 --2- 172.21.15.60:0/3129147195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77001586e0 0x7f7700158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f77052a5640 1 -- 172.21.15.60:0/3129147195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77001586e0 msgr2=0x7f7700158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f77052a5640 1 --2- 172.21.15.60:0/3129147195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77001586e0 0x7f7700158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f77052a5640 1 -- 172.21.15.60:0/3129147195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e8004810 msgr2=0x7f7700153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f77052a5640 1 --2- 172.21.15.60:0/3129147195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e8004810 0x7f7700153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f77052a5640 1 -- 172.21.15.60:0/3129147195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7700155d50 con 0x7f7700154c30 2024-04-25T10:10:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f76fffff640 1 --2- 172.21.15.60:0/3129147195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77001586e0 0x7f7700158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f77052a5640 1 --2- 172.21.15.60:0/3129147195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7700154c30 0x7f77001550b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009fd0 tx=0x7f76f800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f76fd7fa640 1 -- 172.21.15.60:0/3129147195 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f801c070 con 0x7f7700154c30 2024-04-25T10:10:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f76fd7fa640 1 -- 172.21.15.60:0/3129147195 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f8017400 con 0x7f7700154c30 2024-04-25T10:10:02.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f7706d08640 1 -- 172.21.15.60:0/3129147195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77001559c0 con 0x7f7700154c30 2024-04-25T10:10:02.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f76fd7fa640 1 -- 172.21.15.60:0/3129147195 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f80177f0 con 0x7f7700154c30 2024-04-25T10:10:02.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f7706d08640 1 -- 172.21.15.60:0/3129147195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77001562d0 con 0x7f7700154c30 2024-04-25T10:10:02.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f7706d08640 1 -- 172.21.15.60:0/3129147195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7700074770 con 0x7f7700154c30 2024-04-25T10:10:02.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f76fd7fa640 1 -- 172.21.15.60:0/3129147195 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76f801f070 con 0x7f7700154c30 2024-04-25T10:10:02.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f76fd7fa640 1 --2- 172.21.15.60:0/3129147195 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f76ec066f90 0x7f76ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:02.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f76fd7fa640 1 -- 172.21.15.60:0/3129147195 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f76f80852d0 con 0x7f7700154c30 2024-04-25T10:10:02.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.729+0000 7f7704aa4640 1 --2- 172.21.15.60:0/3129147195 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f76ec066f90 0x7f76ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:02.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.733+0000 7f7704aa4640 1 --2- 172.21.15.60:0/3129147195 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f76ec066f90 0x7f76ec069450 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f76f0001060 tx=0x7f76f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:02.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.733+0000 7f76fd7fa640 1 -- 172.21.15.60:0/3129147195 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76f8012070 con 0x7f7700154c30 2024-04-25T10:10:02.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.929+0000 7f7706d08640 1 -- 172.21.15.60:0/3129147195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77000af540 con 0x7f7700154c30 2024-04-25T10:10:02.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.933+0000 7f76fd7fa640 1 -- 172.21.15.60:0/3129147195 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f76f8056330 con 0x7f7700154c30 2024-04-25T10:10:02.941 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:10:02.941 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3185787428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3185787428},{"type":"v1","addr":"172.21.15.100:6836","nonce":3185787428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2667415244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2667415244},{"type":"v1","addr":"172.21.15.100:6837","nonce":2667415244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/56747111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":56747111},{"type":"v1","addr":"172.21.15.60:6834","nonce":56747111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:43.707831+0000","modified":"2024-04-25T10:09:46.733783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6873":{"gid":6873,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1289503429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1289503429},{"type":"v1","addr":"172.21.15.60:6835","nonce":1289503429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:10:02.941 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 39 2024-04-25T10:10:02.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.933+0000 7f7706d08640 1 -- 172.21.15.60:0/3129147195 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f76ec066f90 msgr2=0x7f76ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:02.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.933+0000 7f7706d08640 1 --2- 172.21.15.60:0/3129147195 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f76ec066f90 0x7f76ec069450 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f76f0001060 tx=0x7f76f0009040 comp rx=0 tx=0).stop 2024-04-25T10:10:02.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.933+0000 7f7706d08640 1 -- 172.21.15.60:0/3129147195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7700154c30 msgr2=0x7f77001550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:02.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.933+0000 7f7706d08640 1 --2- 172.21.15.60:0/3129147195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7700154c30 0x7f77001550b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009fd0 tx=0x7f76f800f4a0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.933+0000 7f7706d08640 1 -- 172.21.15.60:0/3129147195 shutdown_connections 2024-04-25T10:10:02.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.933+0000 7f7706d08640 1 --2- 172.21.15.60:0/3129147195 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f76ec066f90 0x7f76ec069450 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.933+0000 7f7706d08640 1 --2- 172.21.15.60:0/3129147195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7700154c30 0x7f77001550b0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.933+0000 7f7706d08640 1 --2- 172.21.15.60:0/3129147195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77001586e0 0x7f7700158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.933+0000 7f7706d08640 1 --2- 172.21.15.60:0/3129147195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76e8004810 0x7f7700153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.933+0000 7f7706d08640 1 -- 172.21.15.60:0/3129147195 >> 172.21.15.60:0/3129147195 conn(0x7f77000b12d0 msgr2=0x7f77000b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:02.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.933+0000 7f7706d08640 1 -- 172.21.15.60:0/3129147195 shutdown_connections 2024-04-25T10:10:02.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:02.933+0000 7f7706d08640 1 -- 172.21.15.60:0/3129147195 wait complete. 2024-04-25T10:10:02.959 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:10:03.117 INFO:teuthology.orchestra.run.smithi060.stdout:{"message":"","return_code":0} 2024-04-25T10:10:03.117 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:10:03.117 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:10:03.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b5ec32640 1 Processor -- start 2024-04-25T10:10:03.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b5ec32640 1 -- start start 2024-04-25T10:10:03.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b5ec32640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b5815bc90 0x7f5b5815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b5ec32640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5b580b4660 0x7f5b5815c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b5ec32640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b58071ca0 con 0x7f5b580ce7a0 2024-04-25T10:10:03.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b5ec32640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b580cc370 con 0x7f5b580b4660 2024-04-25T10:10:03.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b5ec32640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580cca40 con 0x7f5b5815bc90 2024-04-25T10:10:03.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b5c9ce640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5b580ce7a0 0x7f5b580ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49222/0 (socket says 172.21.15.60:49222) 2024-04-25T10:10:03.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b5c9ce640 1 -- 172.21.15.60:0/556865979 learned_addr learned my addr 172.21.15.60:0/556865979 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:10:03.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b57fff640 1 --2- 172.21.15.60:0/556865979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b5815bc90 0x7f5b5815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b577fe640 1 -- 172.21.15.60:0/556865979 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3314738272 0 0) 0x7f5b58071ca0 con 0x7f5b580ce7a0 2024-04-25T10:10:03.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b577fe640 1 -- 172.21.15.60:0/556865979 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b40003530 con 0x7f5b580ce7a0 2024-04-25T10:10:03.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b5d1cf640 1 --2- 172.21.15.60:0/556865979 >> v2:172.21.15.100:3300/0 conn(0x7f5b580b4660 0x7f5b5815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b577fe640 1 -- 172.21.15.60:0/556865979 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3295961539 0 0) 0x7f5b40003530 con 0x7f5b580ce7a0 2024-04-25T10:10:03.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b577fe640 1 -- 172.21.15.60:0/556865979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b5815bc90 msgr2=0x7f5b5815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:03.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b577fe640 1 --2- 172.21.15.60:0/556865979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b5815bc90 0x7f5b5815c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b577fe640 1 -- 172.21.15.60:0/556865979 >> v2:172.21.15.100:3300/0 conn(0x7f5b580b4660 msgr2=0x7f5b5815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:03.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b577fe640 1 --2- 172.21.15.60:0/556865979 >> v2:172.21.15.100:3300/0 conn(0x7f5b580b4660 0x7f5b5815c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b577fe640 1 -- 172.21.15.60:0/556865979 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b580ccd20 con 0x7f5b580ce7a0 2024-04-25T10:10:03.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b577fe640 1 -- 172.21.15.60:0/556865979 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f5b48001680 con 0x7f5b580ce7a0 2024-04-25T10:10:03.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b5d1cf640 1 --2- 172.21.15.60:0/556865979 >> v2:172.21.15.100:3300/0 conn(0x7f5b580b4660 0x7f5b5815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:03.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b577fe640 1 -- 172.21.15.60:0/556865979 >> v1:172.21.15.60:6789/0 conn(0x7f5b580ce7a0 legacy=0x7f5b580ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b577fe640 1 --2- 172.21.15.60:0/556865979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b5815bc90 0x7f5b400039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b577fe640 1 --2- 172.21.15.60:0/556865979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b40003f40 0x7f5b40004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.233+0000 7f5b577fe640 1 --2- 172.21.15.60:0/556865979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b40004d50 0x7f5b400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b577fe640 1 -- 172.21.15.60:0/556865979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580cca40 con 0x7f5b40003f40 2024-04-25T10:10:03.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b577fe640 1 -- 172.21.15.60:0/556865979 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4000b9d0 con 0x7f5b40004d50 2024-04-25T10:10:03.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b577fe640 1 -- 172.21.15.60:0/556865979 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4000bba0 con 0x7f5b5815bc90 2024-04-25T10:10:03.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b57fff640 1 --2- 172.21.15.60:0/556865979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b40003f40 0x7f5b40004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b57fff640 1 -- 172.21.15.60:0/556865979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b5815bc90 msgr2=0x7f5b400039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:03.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b57fff640 1 --2- 172.21.15.60:0/556865979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b5815bc90 0x7f5b400039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b5d1cf640 1 --2- 172.21.15.60:0/556865979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b40004d50 0x7f5b400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b57fff640 1 -- 172.21.15.60:0/556865979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b40004d50 msgr2=0x7f5b400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:03.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b57fff640 1 --2- 172.21.15.60:0/556865979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b40004d50 0x7f5b400051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b57fff640 1 -- 172.21.15.60:0/556865979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b40003530 con 0x7f5b40003f40 2024-04-25T10:10:03.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b57fff640 1 --2- 172.21.15.60:0/556865979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b40003f40 0x7f5b40004810 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009ae0 tx=0x7f5b4c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6f2e4d06d88ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b577fe640 1 -- 172.21.15.60:0/556865979 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c0b2070 con 0x7f5b40003f40 2024-04-25T10:10:03.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b577fe640 1 -- 172.21.15.60:0/556865979 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4c0a56a0 con 0x7f5b40003f40 2024-04-25T10:10:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b577fe640 1 -- 172.21.15.60:0/556865979 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c0096b0 con 0x7f5b40003f40 2024-04-25T10:10:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b5ec32640 1 -- 172.21.15.60:0/556865979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b40003f40 msgr2=0x7f5b40004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b5ec32640 1 --2- 172.21.15.60:0/556865979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b40003f40 0x7f5b40004810 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009ae0 tx=0x7f5b4c01cab0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b5d1cf640 1 -- 172.21.15.60:0/556865979 reap_dead start 2024-04-25T10:10:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b5ec32640 1 -- 172.21.15.60:0/556865979 shutdown_connections 2024-04-25T10:10:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b5ec32640 1 -- 172.21.15.60:0/556865979 >> 172.21.15.60:0/556865979 conn(0x7f5b580b12d0 msgr2=0x7f5b580b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b5ec32640 1 -- 172.21.15.60:0/556865979 shutdown_connections 2024-04-25T10:10:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.237+0000 7f5b5ec32640 1 -- 172.21.15.60:0/556865979 wait complete. 2024-04-25T10:10:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5ec32640 1 Processor -- start 2024-04-25T10:10:03.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5ec32640 1 -- start start 2024-04-25T10:10:03.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5ec32640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b40003f40 0x7f5b581545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5ec32640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b581e4210 0x7f5b581e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5ec32640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b581f49b0 0x7f5b581e6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5ec32640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580b3d20 con 0x7f5b40003f40 2024-04-25T10:10:03.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5ec32640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580b3ba0 con 0x7f5b581e4210 2024-04-25T10:10:03.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5ec32640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580b4280 con 0x7f5b581f49b0 2024-04-25T10:10:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5c9ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b40003f40 0x7f5b581545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5c9ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b40003f40 0x7f5b581545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50810/0 (socket says 172.21.15.60:50810) 2024-04-25T10:10:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5c9ce640 1 -- 172.21.15.60:0/317114144 learned_addr learned my addr 172.21.15.60:0/317114144 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b57fff640 1 --2- 172.21.15.60:0/317114144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b581e4210 0x7f5b581e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5d1cf640 1 --2- 172.21.15.60:0/317114144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b581f49b0 0x7f5b581e6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5c9ce640 1 -- 172.21.15.60:0/317114144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b581f49b0 msgr2=0x7f5b581e6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5c9ce640 1 --2- 172.21.15.60:0/317114144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b581f49b0 0x7f5b581e6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5c9ce640 1 -- 172.21.15.60:0/317114144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b581e4210 msgr2=0x7f5b581e65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5c9ce640 1 --2- 172.21.15.60:0/317114144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b581e4210 0x7f5b581e65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5c9ce640 1 -- 172.21.15.60:0/317114144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b581e4780 con 0x7f5b40003f40 2024-04-25T10:10:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b57fff640 1 --2- 172.21.15.60:0/317114144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b581e4210 0x7f5b581e65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5c9ce640 1 --2- 172.21.15.60:0/317114144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b40003f40 0x7f5b581545e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800f360 tx=0x7f5b4800f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b557fa640 1 -- 172.21.15.60:0/317114144 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48007d00 con 0x7f5b40003f40 2024-04-25T10:10:03.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b557fa640 1 -- 172.21.15.60:0/317114144 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4800a0d0 con 0x7f5b40003f40 2024-04-25T10:10:03.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5ec32640 1 -- 172.21.15.60:0/317114144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b581e4a60 con 0x7f5b40003f40 2024-04-25T10:10:03.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b557fa640 1 -- 172.21.15.60:0/317114144 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4800a4c0 con 0x7f5b40003f40 2024-04-25T10:10:03.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5ec32640 1 -- 172.21.15.60:0/317114144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b581e5020 con 0x7f5b40003f40 2024-04-25T10:10:03.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.241+0000 7f5b5ec32640 1 -- 172.21.15.60:0/317114144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b58074770 con 0x7f5b40003f40 2024-04-25T10:10:03.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.245+0000 7f5b557fa640 1 -- 172.21.15.60:0/317114144 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b48014070 con 0x7f5b40003f40 2024-04-25T10:10:03.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.245+0000 7f5b557fa640 1 --2- 172.21.15.60:0/317114144 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5b44066f60 0x7f5b44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.245+0000 7f5b557fa640 1 -- 172.21.15.60:0/317114144 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5b48088060 con 0x7f5b40003f40 2024-04-25T10:10:03.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.245+0000 7f5b57fff640 1 --2- 172.21.15.60:0/317114144 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5b44066f60 0x7f5b44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.245+0000 7f5b57fff640 1 --2- 172.21.15.60:0/317114144 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5b44066f60 0x7f5b44069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0af000 tx=0x7f5b4c0046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.249+0000 7f5b557fa640 1 -- 172.21.15.60:0/317114144 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b480591a0 con 0x7f5b40003f40 2024-04-25T10:10:03.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.445+0000 7f5b5ec32640 1 -- 172.21.15.60:0/317114144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5b580af7a0 con 0x7f5b40003f40 2024-04-25T10:10:03.454 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:10:03.445+0000 7f3a792d6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:10:03.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.449+0000 7f5b557fa640 1 -- 172.21.15.60:0/317114144 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5b48058aa0 con 0x7f5b40003f40 2024-04-25T10:10:03.457 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:10:03.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.449+0000 7f5b5ec32640 1 -- 172.21.15.60:0/317114144 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5b44066f60 msgr2=0x7f5b44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.449+0000 7f5b5ec32640 1 --2- 172.21.15.60:0/317114144 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5b44066f60 0x7f5b44069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0af000 tx=0x7f5b4c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.449+0000 7f5b5ec32640 1 -- 172.21.15.60:0/317114144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b40003f40 msgr2=0x7f5b581545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.449+0000 7f5b5ec32640 1 --2- 172.21.15.60:0/317114144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b40003f40 0x7f5b581545e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800f360 tx=0x7f5b4800f820 comp rx=0 tx=0).stop 2024-04-25T10:10:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.449+0000 7f5b5ec32640 1 -- 172.21.15.60:0/317114144 shutdown_connections 2024-04-25T10:10:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.449+0000 7f5b5ec32640 1 --2- 172.21.15.60:0/317114144 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5b44066f60 0x7f5b44069420 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.449+0000 7f5b5ec32640 1 --2- 172.21.15.60:0/317114144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b581f49b0 0x7f5b581e6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.449+0000 7f5b5ec32640 1 --2- 172.21.15.60:0/317114144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b581e4210 0x7f5b581e65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.449+0000 7f5b5ec32640 1 --2- 172.21.15.60:0/317114144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b40003f40 0x7f5b581545e0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.449+0000 7f5b5ec32640 1 -- 172.21.15.60:0/317114144 >> 172.21.15.60:0/317114144 conn(0x7f5b580b12d0 msgr2=0x7f5b5815a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.449+0000 7f5b5ec32640 1 -- 172.21.15.60:0/317114144 shutdown_connections 2024-04-25T10:10:03.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.449+0000 7f5b5ec32640 1 -- 172.21.15.60:0/317114144 wait complete. 2024-04-25T10:10:03.464 INFO:tasks.ceph.mds.c.smithi060.stderr: -3432> 2024-04-25T10:09:50.305+0000 7ff101bd0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:10:03.479 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T10:10:03.502 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:10:03.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 Processor -- start 2024-04-25T10:10:03.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 -- start start 2024-04-25T10:10:03.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6484003100 0x55985d847590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55985d847ad0 0x55985d8bdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55985d806000 0x55985d8be530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55985d7a99f0 con 0x55985d806000 2024-04-25T10:10:03.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55985d7ad950 con 0x55985d847ad0 2024-04-25T10:10:03.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55985d84d130 con 0x7f6484003100 2024-04-25T10:10:03.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f6499c11640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55985d806000 0x55985d8be530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f6499c11640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55985d806000 0x55985d8be530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50856/0 (socket says 172.21.15.60:50856) 2024-04-25T10:10:03.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f6499c11640 1 -- 172.21.15.60:0/2994747686 learned_addr learned my addr 172.21.15.60:0/2994747686 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:03.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f6499410640 1 --2- 172.21.15.60:0/2994747686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6484003100 0x55985d847590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f6498c0f640 1 --2- 172.21.15.60:0/2994747686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55985d847ad0 0x55985d8bdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f6499c11640 1 -- 172.21.15.60:0/2994747686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6484003100 msgr2=0x55985d847590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:03.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f6499c11640 1 --2- 172.21.15.60:0/2994747686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6484003100 0x55985d847590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f6499c11640 1 -- 172.21.15.60:0/2994747686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55985d847ad0 msgr2=0x55985d8bdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:03.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f6499c11640 1 --2- 172.21.15.60:0/2994747686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55985d847ad0 0x55985d8bdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f6499c11640 1 -- 172.21.15.60:0/2994747686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55985d8bdaf0 con 0x55985d806000 2024-04-25T10:10:03.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f6499410640 1 --2- 172.21.15.60:0/2994747686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6484003100 0x55985d847590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:03.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f6499c11640 1 --2- 172.21.15.60:0/2994747686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55985d806000 0x55985d8be530 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x55985d8077f0 tx=0x7f649400e7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcf6721cbbcb19b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f648b7fe640 1 -- 172.21.15.60:0/2994747686 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494013660 con 0x55985d806000 2024-04-25T10:10:03.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f648b7fe640 1 -- 172.21.15.60:0/2994747686 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6494019070 con 0x55985d806000 2024-04-25T10:10:03.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f648b7fe640 1 -- 172.21.15.60:0/2994747686 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494013a00 con 0x55985d806000 2024-04-25T10:10:03.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 -- 172.21.15.60:0/2994747686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55985d806000 msgr2=0x55985d8be530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 --2- 172.21.15.60:0/2994747686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55985d806000 0x55985d8be530 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x55985d8077f0 tx=0x7f649400e7b0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 -- 172.21.15.60:0/2994747686 shutdown_connections 2024-04-25T10:10:03.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 --2- 172.21.15.60:0/2994747686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55985d806000 0x55985d8be530 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 --2- 172.21.15.60:0/2994747686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55985d847ad0 0x55985d8bdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 --2- 172.21.15.60:0/2994747686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6484003100 0x55985d847590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 -- 172.21.15.60:0/2994747686 >> 172.21.15.60:0/2994747686 conn(0x55985d82de80 msgr2=0x55985d830e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:03.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 -- 172.21.15.60:0/2994747686 shutdown_connections 2024-04-25T10:10:03.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 -- 172.21.15.60:0/2994747686 wait complete. 2024-04-25T10:10:03.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 Processor -- start 2024-04-25T10:10:03.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 -- start start 2024-04-25T10:10:03.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55985d806000 0x55985d9759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55985d847ad0 0x55985d8dce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6484003100 0x55985d82c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55985d847f80 con 0x7f6484003100 2024-04-25T10:10:03.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55985d848220 con 0x55985d847ad0 2024-04-25T10:10:03.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.505+0000 7f649b8f2080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55985d805b60 con 0x55985d806000 2024-04-25T10:10:03.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f6499c11640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6484003100 0x55985d82c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f6499c11640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6484003100 0x55985d82c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50868/0 (socket says 172.21.15.60:50868) 2024-04-25T10:10:03.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f6499c11640 1 -- 172.21.15.60:0/1897565830 learned_addr learned my addr 172.21.15.60:0/1897565830 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:03.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f6499410640 1 --2- 172.21.15.60:0/1897565830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55985d806000 0x55985d9759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f6498c0f640 1 --2- 172.21.15.60:0/1897565830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55985d847ad0 0x55985d8dce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f6499c11640 1 -- 172.21.15.60:0/1897565830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55985d806000 msgr2=0x55985d9759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f6499c11640 1 --2- 172.21.15.60:0/1897565830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55985d806000 0x55985d9759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f6499c11640 1 -- 172.21.15.60:0/1897565830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55985d847ad0 msgr2=0x55985d8dce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f6499c11640 1 --2- 172.21.15.60:0/1897565830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55985d847ad0 0x55985d8dce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f6499c11640 1 -- 172.21.15.60:0/1897565830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55985d9761d0 con 0x7f6484003100 2024-04-25T10:10:03.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f6499c11640 1 --2- 172.21.15.60:0/1897565830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6484003100 0x55985d82c550 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f649400ecb0 tx=0x7f6494003340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f648a7fc640 1 -- 172.21.15.60:0/1897565830 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494028450 con 0x7f6484003100 2024-04-25T10:10:03.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f648a7fc640 1 -- 172.21.15.60:0/1897565830 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64940285f0 con 0x7f6484003100 2024-04-25T10:10:03.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f648a7fc640 1 -- 172.21.15.60:0/1897565830 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64940289c0 con 0x7f6484003100 2024-04-25T10:10:03.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f649b8f2080 1 -- 172.21.15.60:0/1897565830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55985d976470 con 0x7f6484003100 2024-04-25T10:10:03.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f649b8f2080 1 -- 172.21.15.60:0/1897565830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55985d9769b0 con 0x7f6484003100 2024-04-25T10:10:03.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f648a7fc640 1 -- 172.21.15.60:0/1897565830 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6494011070 con 0x7f6484003100 2024-04-25T10:10:03.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f648a7fc640 1 --2- 172.21.15.60:0/1897565830 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f647c065970 0x7f647c067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f648a7fc640 1 -- 172.21.15.60:0/1897565830 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6494085250 con 0x7f6484003100 2024-04-25T10:10:03.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f649b8f2080 1 --2- 172.21.15.60:0/1897565830 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55985d9646f0 0x55985d964ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f649b8f2080 1 -- 172.21.15.60:0/1897565830 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v9 -- 0x55985d8dd3b0 con 0x55985d9646f0 2024-04-25T10:10:03.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f6499410640 1 --2- 172.21.15.60:0/1897565830 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f647c065970 0x7f647c067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f6498c0f640 1 --2- 172.21.15.60:0/1897565830 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55985d9646f0 0x55985d964ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f6499410640 1 --2- 172.21.15.60:0/1897565830 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f647c065970 0x7f647c067e30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f648c00f740 tx=0x7f648c001650 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.509+0000 7f6498c0f640 1 --2- 172.21.15.60:0/1897565830 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55985d9646f0 0x55985d964ad0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.513+0000 7f6498c0f640 1 -- 172.21.15.60:0/1897565830 <== osd.4 v2:172.21.15.100:6808/3965541654 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v71'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f64900b3040 con 0x55985d9646f0 2024-04-25T10:10:03.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.513+0000 7f649b8f2080 1 -- 172.21.15.60:0/1897565830 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55985d9646f0 msgr2=0x55985d964ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.513+0000 7f649b8f2080 1 --2- 172.21.15.60:0/1897565830 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55985d9646f0 0x55985d964ad0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.513+0000 7f649b8f2080 1 -- 172.21.15.60:0/1897565830 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f647c065970 msgr2=0x7f647c067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.513+0000 7f649b8f2080 1 --2- 172.21.15.60:0/1897565830 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f647c065970 0x7f647c067e30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f648c00f740 tx=0x7f648c001650 comp rx=0 tx=0).stop 2024-04-25T10:10:03.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.513+0000 7f649b8f2080 1 -- 172.21.15.60:0/1897565830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6484003100 msgr2=0x55985d82c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.513+0000 7f649b8f2080 1 --2- 172.21.15.60:0/1897565830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6484003100 0x55985d82c550 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f649400ecb0 tx=0x7f6494003340 comp rx=0 tx=0).stop 2024-04-25T10:10:03.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.513+0000 7f6499c11640 1 -- 172.21.15.60:0/1897565830 reap_dead start 2024-04-25T10:10:03.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.513+0000 7f649b8f2080 1 -- 172.21.15.60:0/1897565830 shutdown_connections 2024-04-25T10:10:03.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.513+0000 7f649b8f2080 1 -- 172.21.15.60:0/1897565830 >> 172.21.15.60:0/1897565830 conn(0x55985d82de80 msgr2=0x55985d830e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:03.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.513+0000 7f649b8f2080 1 -- 172.21.15.60:0/1897565830 shutdown_connections 2024-04-25T10:10:03.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.513+0000 7f649b8f2080 1 -- 172.21.15.60:0/1897565830 wait complete. 2024-04-25T10:10:03.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:10:03.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c45e6640 1 Processor -- start 2024-04-25T10:10:03.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c45e6640 1 -- start start 2024-04-25T10:10:03.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c45e6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77bc15fbf0 0x7f77bc166870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c45e6640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77bc144a40 con 0x7f77bc160080 2024-04-25T10:10:03.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c45e6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77bc0cc9f0 con 0x7f77bc0cbcf0 2024-04-25T10:10:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c45e6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc0b2dc0 con 0x7f77bc15fbf0 2024-04-25T10:10:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1b81640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f77bc160080 0x7f77bc15f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49232/0 (socket says 172.21.15.60:49232) 2024-04-25T10:10:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1b81640 1 -- 172.21.15.60:0/3305172 learned_addr learned my addr 172.21.15.60:0/3305172 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:10:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c2b83640 1 --2- 172.21.15.60:0/3305172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77bc15fbf0 0x7f77bc166870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1380640 1 -- 172.21.15.60:0/3305172 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883082537 0 0) 0x7f77bc144a40 con 0x7f77bc160080 2024-04-25T10:10:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1380640 1 -- 172.21.15.60:0/3305172 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77a8003530 con 0x7f77bc160080 2024-04-25T10:10:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1380640 1 -- 172.21.15.60:0/3305172 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 327269594 0 0) 0x7f77a8003530 con 0x7f77bc160080 2024-04-25T10:10:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1380640 1 -- 172.21.15.60:0/3305172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77bc15fbf0 msgr2=0x7f77bc166870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1380640 1 --2- 172.21.15.60:0/3305172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77bc15fbf0 0x7f77bc166870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1380640 1 -- 172.21.15.60:0/3305172 >> v1:172.21.15.100:6789/0 conn(0x7f77bc0cbcf0 legacy=0x7f77bc071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1380640 1 -- 172.21.15.60:0/3305172 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc0cccd0 con 0x7f77bc160080 2024-04-25T10:10:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1380640 1 -- 172.21.15.60:0/3305172 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f77ac001680 con 0x7f77bc160080 2024-04-25T10:10:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1380640 1 -- 172.21.15.60:0/3305172 >> v1:172.21.15.60:6789/0 conn(0x7f77bc160080 legacy=0x7f77bc15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1380640 1 --2- 172.21.15.60:0/3305172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a80039f0 0x7f77a8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1380640 1 --2- 172.21.15.60:0/3305172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77a8004810 0x7f77a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1380640 1 --2- 172.21.15.60:0/3305172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77bc15fbf0 0x7f77a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1380640 1 -- 172.21.15.60:0/3305172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a800ba80 con 0x7f77a8004810 2024-04-25T10:10:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1380640 1 -- 172.21.15.60:0/3305172 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a800bc30 con 0x7f77a80039f0 2024-04-25T10:10:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1380640 1 -- 172.21.15.60:0/3305172 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a800be30 con 0x7f77bc15fbf0 2024-04-25T10:10:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1b81640 1 --2- 172.21.15.60:0/3305172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77a8004810 0x7f77a8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c2382640 1 --2- 172.21.15.60:0/3305172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a80039f0 0x7f77a8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1b81640 1 -- 172.21.15.60:0/3305172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77bc15fbf0 msgr2=0x7f77a80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c2b83640 1 --2- 172.21.15.60:0/3305172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77bc15fbf0 0x7f77a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1b81640 1 --2- 172.21.15.60:0/3305172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77bc15fbf0 0x7f77a80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1b81640 1 -- 172.21.15.60:0/3305172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a80039f0 msgr2=0x7f77a8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1b81640 1 --2- 172.21.15.60:0/3305172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a80039f0 0x7f77a8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.693+0000 7f77c1b81640 1 -- 172.21.15.60:0/3305172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a8003530 con 0x7f77a8004810 2024-04-25T10:10:03.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.697+0000 7f77c1b81640 1 --2- 172.21.15.60:0/3305172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77a8004810 0x7f77a8004c90 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00c240 tx=0x7f77ac01f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55787b415c2136a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.697+0000 7f77c1380640 1 -- 172.21.15.60:0/3305172 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac007d20 con 0x7f77a8004810 2024-04-25T10:10:03.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.697+0000 7f77c1380640 1 -- 172.21.15.60:0/3305172 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ac005ae0 con 0x7f77a8004810 2024-04-25T10:10:03.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.697+0000 7f77c1380640 1 -- 172.21.15.60:0/3305172 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac007070 con 0x7f77a8004810 2024-04-25T10:10:03.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.697+0000 7f77c45e6640 1 -- 172.21.15.60:0/3305172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77a8004810 msgr2=0x7f77a8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.697+0000 7f77c45e6640 1 --2- 172.21.15.60:0/3305172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77a8004810 0x7f77a8004c90 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00c240 tx=0x7f77ac01f3f0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.697+0000 7f77c2b83640 1 -- 172.21.15.60:0/3305172 reap_dead start 2024-04-25T10:10:03.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.697+0000 7f77c45e6640 1 -- 172.21.15.60:0/3305172 shutdown_connections 2024-04-25T10:10:03.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.697+0000 7f77c45e6640 1 -- 172.21.15.60:0/3305172 >> 172.21.15.60:0/3305172 conn(0x7f77bc0670b0 msgr2=0x7f77bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:03.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.697+0000 7f77c45e6640 1 -- 172.21.15.60:0/3305172 shutdown_connections 2024-04-25T10:10:03.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.697+0000 7f77c45e6640 1 -- 172.21.15.60:0/3305172 wait complete. 2024-04-25T10:10:03.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c45e6640 1 Processor -- start 2024-04-25T10:10:03.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c45e6640 1 -- start start 2024-04-25T10:10:03.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c45e6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a8004810 0x7f77bc13c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c45e6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77bc1408b0 0x7f77bc15b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c45e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77bc13ed20 0x7f77bc13f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c45e6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc0cc0d0 con 0x7f77bc13ed20 2024-04-25T10:10:03.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c45e6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc0ce9a0 con 0x7f77a8004810 2024-04-25T10:10:03.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c45e6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc0cfed0 con 0x7f77bc1408b0 2024-04-25T10:10:03.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c2b83640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77bc13ed20 0x7f77bc13f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c2b83640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77bc13ed20 0x7f77bc13f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50884/0 (socket says 172.21.15.60:50884) 2024-04-25T10:10:03.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c2b83640 1 -- 172.21.15.60:0/1618176407 learned_addr learned my addr 172.21.15.60:0/1618176407 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:03.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c1b81640 1 --2- 172.21.15.60:0/1618176407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77bc1408b0 0x7f77bc15b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c2382640 1 --2- 172.21.15.60:0/1618176407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a8004810 0x7f77bc13c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c2b83640 1 -- 172.21.15.60:0/1618176407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77bc1408b0 msgr2=0x7f77bc15b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c2b83640 1 --2- 172.21.15.60:0/1618176407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77bc1408b0 0x7f77bc15b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c2b83640 1 -- 172.21.15.60:0/1618176407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a8004810 msgr2=0x7f77bc13c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c2b83640 1 --2- 172.21.15.60:0/1618176407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a8004810 0x7f77bc13c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c2b83640 1 -- 172.21.15.60:0/1618176407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc140e00 con 0x7f77bc13ed20 2024-04-25T10:10:03.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c2b83640 1 --2- 172.21.15.60:0/1618176407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77bc13ed20 0x7f77bc13f1a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f77bc15cf20 tx=0x7f77b400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77b2ffd640 1 -- 172.21.15.60:0/1618176407 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b401c070 con 0x7f77bc13ed20 2024-04-25T10:10:03.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77b2ffd640 1 -- 172.21.15.60:0/1618176407 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b4017400 con 0x7f77bc13ed20 2024-04-25T10:10:03.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77b2ffd640 1 -- 172.21.15.60:0/1618176407 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40177f0 con 0x7f77bc13ed20 2024-04-25T10:10:03.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c45e6640 1 -- 172.21.15.60:0/1618176407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77bc1410e0 con 0x7f77bc13ed20 2024-04-25T10:10:03.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c45e6640 1 -- 172.21.15.60:0/1618176407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77bc1416a0 con 0x7f77bc13ed20 2024-04-25T10:10:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77c45e6640 1 -- 172.21.15.60:0/1618176407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77bc06bba0 con 0x7f77bc13ed20 2024-04-25T10:10:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77b2ffd640 1 -- 172.21.15.60:0/1618176407 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77b401f070 con 0x7f77bc13ed20 2024-04-25T10:10:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.701+0000 7f77b2ffd640 1 --2- 172.21.15.60:0/1618176407 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f77a0066f90 0x7f77a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.705+0000 7f77b2ffd640 1 -- 172.21.15.60:0/1618176407 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f77b4085190 con 0x7f77bc13ed20 2024-04-25T10:10:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.705+0000 7f77c2382640 1 --2- 172.21.15.60:0/1618176407 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f77a0066f90 0x7f77a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.705+0000 7f77c2382640 1 --2- 172.21.15.60:0/1618176407 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f77a0066f90 0x7f77a0069450 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f77bc15e990 tx=0x7f77b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.709+0000 7f77b2ffd640 1 -- 172.21.15.60:0/1618176407 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f77b4012070 con 0x7f77bc13ed20 2024-04-25T10:10:03.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:03.905+0000 7f77c45e6640 1 -- 172.21.15.60:0/1618176407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f77bc074770 con 0x7f77bc13ed20 2024-04-25T10:10:04.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.449+0000 7f77b2ffd640 1 -- 172.21.15.60:0/1618176407 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v41) v1 ==== 162+0+0 (secure 0 0 0) 0x7f77b4055ab0 con 0x7f77bc13ed20 2024-04-25T10:10:04.460 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:10:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.453+0000 7f77c45e6640 1 -- 172.21.15.60:0/1618176407 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f77a0066f90 msgr2=0x7f77a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.453+0000 7f77c45e6640 1 --2- 172.21.15.60:0/1618176407 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f77a0066f90 0x7f77a0069450 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f77bc15e990 tx=0x7f77b8009040 comp rx=0 tx=0).stop 2024-04-25T10:10:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.453+0000 7f77c45e6640 1 -- 172.21.15.60:0/1618176407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77bc13ed20 msgr2=0x7f77bc13f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.453+0000 7f77c45e6640 1 --2- 172.21.15.60:0/1618176407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77bc13ed20 0x7f77bc13f1a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f77bc15cf20 tx=0x7f77b400f4a0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.453+0000 7f77c45e6640 1 -- 172.21.15.60:0/1618176407 shutdown_connections 2024-04-25T10:10:04.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.453+0000 7f77c45e6640 1 --2- 172.21.15.60:0/1618176407 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f77a0066f90 0x7f77a0069450 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.453+0000 7f77c45e6640 1 --2- 172.21.15.60:0/1618176407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77bc13ed20 0x7f77bc13f1a0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.453+0000 7f77c45e6640 1 --2- 172.21.15.60:0/1618176407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77bc1408b0 0x7f77bc15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.453+0000 7f77c45e6640 1 --2- 172.21.15.60:0/1618176407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77a8004810 0x7f77bc13c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.453+0000 7f77c45e6640 1 -- 172.21.15.60:0/1618176407 >> 172.21.15.60:0/1618176407 conn(0x7f77bc0670b0 msgr2=0x7f77bc143f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:04.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.453+0000 7f77c45e6640 1 -- 172.21.15.60:0/1618176407 shutdown_connections 2024-04-25T10:10:04.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.453+0000 7f77c45e6640 1 -- 172.21.15.60:0/1618176407 wait complete. 2024-04-25T10:10:04.492 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:04.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.601+0000 7f928add8640 1 Processor -- start 2024-04-25T10:10:04.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.601+0000 7f928add8640 1 -- start start 2024-04-25T10:10:04.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.601+0000 7f928add8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f928415dc10 0x7f928406ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.601+0000 7f928add8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928406f4b0 0x7f92840ad560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.601+0000 7f928add8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92840b5050 con 0x7f928406ac10 2024-04-25T10:10:04.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.601+0000 7f928add8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92840b5660 con 0x7f928415dc10 2024-04-25T10:10:04.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.601+0000 7f928add8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9284070040 con 0x7f928406f4b0 2024-04-25T10:10:04.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.601+0000 7f9283fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f928415dc10 0x7f928406ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.601+0000 7f9289375640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928406f4b0 0x7f92840ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.601+0000 7f9283fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f928415dc10 0x7f928406ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:47998/0 (socket says 172.21.15.60:47998) 2024-04-25T10:10:04.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.601+0000 7f9283fff640 1 -- 172.21.15.60:0/3209179615 learned_addr learned my addr 172.21.15.60:0/3209179615 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:04.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.601+0000 7f9289375640 1 -- 172.21.15.60:0/3209179615 >> v2:172.21.15.100:3300/0 conn(0x7f928415dc10 msgr2=0x7f928406ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:04.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.601+0000 7f9289375640 1 --2- 172.21.15.60:0/3209179615 >> v2:172.21.15.100:3300/0 conn(0x7f928415dc10 0x7f928406ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.601+0000 7f9289375640 1 -- 172.21.15.60:0/3209179615 >> v1:172.21.15.100:6789/0 conn(0x7f928406ac10 legacy=0x7f92840b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.601+0000 7f9289375640 1 -- 172.21.15.60:0/3209179615 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9284070320 con 0x7f928406f4b0 2024-04-25T10:10:04.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.605+0000 7f9289375640 1 --2- 172.21.15.60:0/3209179615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928406f4b0 0x7f92840ad560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f927c009d90 tx=0x7f927c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c461b72c44fdd0a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.605+0000 7f92837fe640 1 -- 172.21.15.60:0/3209179615 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c0b2070 con 0x7f928406f4b0 2024-04-25T10:10:04.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.605+0000 7f92837fe640 1 -- 172.21.15.60:0/3209179615 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c0a5830 con 0x7f928406f4b0 2024-04-25T10:10:04.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.605+0000 7f92837fe640 1 -- 172.21.15.60:0/3209179615 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c0a5c00 con 0x7f928406f4b0 2024-04-25T10:10:04.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.605+0000 7f928add8640 1 -- 172.21.15.60:0/3209179615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928406f4b0 msgr2=0x7f92840ad560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.605+0000 7f928add8640 1 --2- 172.21.15.60:0/3209179615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928406f4b0 0x7f92840ad560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f927c009d90 tx=0x7f927c01c970 comp rx=0 tx=0).stop 2024-04-25T10:10:04.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.605+0000 7f928add8640 1 -- 172.21.15.60:0/3209179615 shutdown_connections 2024-04-25T10:10:04.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.605+0000 7f928add8640 1 --2- 172.21.15.60:0/3209179615 >> v2:172.21.15.100:3300/0 conn(0x7f928415dc10 0x7f928406ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.605+0000 7f928add8640 1 --2- 172.21.15.60:0/3209179615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928406f4b0 0x7f92840ad560 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.605+0000 7f928add8640 1 -- 172.21.15.60:0/3209179615 >> 172.21.15.60:0/3209179615 conn(0x7f92840b5aa0 msgr2=0x7f92840b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:04.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.605+0000 7f928add8640 1 -- 172.21.15.60:0/3209179615 shutdown_connections 2024-04-25T10:10:04.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.605+0000 7f928add8640 1 -- 172.21.15.60:0/3209179615 wait complete. 2024-04-25T10:10:04.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f928add8640 1 Processor -- start 2024-04-25T10:10:04.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f928add8640 1 -- start start 2024-04-25T10:10:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f928add8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928406ac10 0x7f92841f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f928add8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928406f4b0 0x7f928415cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f928add8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f928415dc10 0x7f92840d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f928add8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928416a740 con 0x7f928415dc10 2024-04-25T10:10:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f928add8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928406ba10 con 0x7f928406f4b0 2024-04-25T10:10:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f928add8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928416a8c0 con 0x7f928406ac10 2024-04-25T10:10:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f9289375640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f928415dc10 0x7f92840d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f9289375640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f928415dc10 0x7f92840d06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50900/0 (socket says 172.21.15.60:50900) 2024-04-25T10:10:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f9289375640 1 -- 172.21.15.60:0/3044604544 learned_addr learned my addr 172.21.15.60:0/3044604544 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f9288b74640 1 --2- 172.21.15.60:0/3044604544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928406ac10 0x7f92841f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f9283fff640 1 --2- 172.21.15.60:0/3044604544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928406f4b0 0x7f928415cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f9289375640 1 -- 172.21.15.60:0/3044604544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928406ac10 msgr2=0x7f92841f67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f9289375640 1 --2- 172.21.15.60:0/3044604544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928406ac10 0x7f92841f67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f9289375640 1 -- 172.21.15.60:0/3044604544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928406f4b0 msgr2=0x7f928415cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f9289375640 1 --2- 172.21.15.60:0/3044604544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928406f4b0 0x7f928415cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f9289375640 1 -- 172.21.15.60:0/3044604544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92840cd7c0 con 0x7f928415dc10 2024-04-25T10:10:04.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f9283fff640 1 --2- 172.21.15.60:0/3044604544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928406f4b0 0x7f928415cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:04.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f9289375640 1 --2- 172.21.15.60:0/3044604544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f928415dc10 0x7f92840d06c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f927c0b0040 tx=0x7f927c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f92817fa640 1 -- 172.21.15.60:0/3044604544 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c0b2050 con 0x7f928415dc10 2024-04-25T10:10:04.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f92817fa640 1 -- 172.21.15.60:0/3044604544 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c0be070 con 0x7f928415dc10 2024-04-25T10:10:04.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f928add8640 1 -- 172.21.15.60:0/3044604544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92840cdaa0 con 0x7f928415dc10 2024-04-25T10:10:04.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f92817fa640 1 -- 172.21.15.60:0/3044604544 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c009450 con 0x7f928415dc10 2024-04-25T10:10:04.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f928add8640 1 -- 172.21.15.60:0/3044604544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92840ce060 con 0x7f928415dc10 2024-04-25T10:10:04.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.609+0000 7f928add8640 1 -- 172.21.15.60:0/3044604544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9284074770 con 0x7f928415dc10 2024-04-25T10:10:04.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.613+0000 7f92817fa640 1 -- 172.21.15.60:0/3044604544 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f927c0adcc0 con 0x7f928415dc10 2024-04-25T10:10:04.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.613+0000 7f92817fa640 1 --2- 172.21.15.60:0/3044604544 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9264066f60 0x7f9264069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.613+0000 7f92817fa640 1 -- 172.21.15.60:0/3044604544 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f927c120480 con 0x7f928415dc10 2024-04-25T10:10:04.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.613+0000 7f9288b74640 1 --2- 172.21.15.60:0/3044604544 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9264066f60 0x7f9264069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.613+0000 7f9288b74640 1 --2- 172.21.15.60:0/3044604544 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9264066f60 0x7f9264069420 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f926c001650 tx=0x7f926c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.617+0000 7f92817fa640 1 -- 172.21.15.60:0/3044604544 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f927c016560 con 0x7f928415dc10 2024-04-25T10:10:04.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.813+0000 7f928add8640 1 -- 172.21.15.60:0/3044604544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92841f6d00 con 0x7f928415dc10 2024-04-25T10:10:04.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.813+0000 7f92817fa640 1 -- 172.21.15.60:0/3044604544 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f927c0f12e0 con 0x7f928415dc10 2024-04-25T10:10:04.822 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:10:04.822 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3185787428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3185787428},{"type":"v1","addr":"172.21.15.100:6836","nonce":3185787428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2667415244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2667415244},{"type":"v1","addr":"172.21.15.100:6837","nonce":2667415244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/56747111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":56747111},{"type":"v1","addr":"172.21.15.60:6834","nonce":56747111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:43.707831+0000","modified":"2024-04-25T10:10:04.459839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7134},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7134":{"gid":7134,"name":"c","rank":0,"incarnation":42,"state":"up:replay","state_seq":1,"addr":"172.21.15.60:6835/1295867731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1295867731},{"type":"v1","addr":"172.21.15.60:6835","nonce":1295867731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:10:04.822 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 42 2024-04-25T10:10:04.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.817+0000 7f928add8640 1 -- 172.21.15.60:0/3044604544 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9264066f60 msgr2=0x7f9264069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.817+0000 7f928add8640 1 --2- 172.21.15.60:0/3044604544 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9264066f60 0x7f9264069420 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f926c001650 tx=0x7f926c009040 comp rx=0 tx=0).stop 2024-04-25T10:10:04.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.817+0000 7f928add8640 1 -- 172.21.15.60:0/3044604544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f928415dc10 msgr2=0x7f92840d06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.817+0000 7f928add8640 1 --2- 172.21.15.60:0/3044604544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f928415dc10 0x7f92840d06c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f927c0b0040 tx=0x7f927c019710 comp rx=0 tx=0).stop 2024-04-25T10:10:04.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.817+0000 7f928add8640 1 -- 172.21.15.60:0/3044604544 shutdown_connections 2024-04-25T10:10:04.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.817+0000 7f928add8640 1 --2- 172.21.15.60:0/3044604544 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9264066f60 0x7f9264069420 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.817+0000 7f928add8640 1 --2- 172.21.15.60:0/3044604544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f928415dc10 0x7f92840d06c0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.817+0000 7f928add8640 1 --2- 172.21.15.60:0/3044604544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928406f4b0 0x7f928415cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.817+0000 7f928add8640 1 --2- 172.21.15.60:0/3044604544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928406ac10 0x7f92841f67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.817+0000 7f928add8640 1 -- 172.21.15.60:0/3044604544 >> 172.21.15.60:0/3044604544 conn(0x7f92840b5aa0 msgr2=0x7f92840b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:04.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.817+0000 7f928add8640 1 -- 172.21.15.60:0/3044604544 shutdown_connections 2024-04-25T10:10:04.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:04.817+0000 7f928add8640 1 -- 172.21.15.60:0/3044604544 wait complete. 2024-04-25T10:10:05.245 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:09:43.707831+0000', 'modified': '2024-04-25T10:10:04.459839+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7134}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7134': {'gid': 7134, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.60:6835/1295867731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 1295867731}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 1295867731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:10:05.245 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7134:up:replay 2024-04-25T10:10:06.246 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:06.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.373+0000 7f5d01494640 1 Processor -- start 2024-04-25T10:10:06.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.373+0000 7f5d01494640 1 -- start start 2024-04-25T10:10:06.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.373+0000 7f5d01494640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cfc0ce7a0 0x7f5cfc0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.373+0000 7f5d01494640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5cfc0b4660 0x7f5cfc0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.373+0000 7f5d01494640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cfc0cf0c0 con 0x7f5cfc15bc90 2024-04-25T10:10:06.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.373+0000 7f5d01494640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cfc0cc370 con 0x7f5cfc0b4660 2024-04-25T10:10:06.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.373+0000 7f5d01494640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc071ca0 con 0x7f5cfc0ce7a0 2024-04-25T10:10:06.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.373+0000 7f5cfb7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5cfc0b4660 0x7f5cfc0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.373+0000 7f5cfb7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5cfc0b4660 0x7f5cfc0b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50904/0 (socket says 172.21.15.60:50904) 2024-04-25T10:10:06.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.373+0000 7f5cfb7fe640 1 -- 172.21.15.60:0/1852103287 learned_addr learned my addr 172.21.15.60:0/1852103287 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:06.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.373+0000 7f5cfaffd640 1 --2- 172.21.15.60:0/1852103287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cfc0ce7a0 0x7f5cfc0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.373+0000 7f5cfb7fe640 1 -- 172.21.15.60:0/1852103287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cfc0ce7a0 msgr2=0x7f5cfc0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.373+0000 7f5cfb7fe640 1 --2- 172.21.15.60:0/1852103287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cfc0ce7a0 0x7f5cfc0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.373+0000 7f5cfb7fe640 1 -- 172.21.15.60:0/1852103287 >> v1:172.21.15.100:6789/0 conn(0x7f5cfc15bc90 legacy=0x7f5cfc0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.373+0000 7f5cfb7fe640 1 -- 172.21.15.60:0/1852103287 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc0cca40 con 0x7f5cfc0b4660 2024-04-25T10:10:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.373+0000 7f5cfb7fe640 1 --2- 172.21.15.60:0/1852103287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cfc0b4660 0x7f5cfc0b2ce0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009e80 tx=0x7f5cf401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44b9511ab24c7de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5cf9ffb640 1 -- 172.21.15.60:0/1852103287 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40a56e0 con 0x7f5cfc0b4660 2024-04-25T10:10:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5cf9ffb640 1 -- 172.21.15.60:0/1852103287 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf40a5880 con 0x7f5cfc0b4660 2024-04-25T10:10:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5cf9ffb640 1 -- 172.21.15.60:0/1852103287 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40a5c50 con 0x7f5cfc0b4660 2024-04-25T10:10:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 -- 172.21.15.60:0/1852103287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cfc0b4660 msgr2=0x7f5cfc0b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 --2- 172.21.15.60:0/1852103287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cfc0b4660 0x7f5cfc0b2ce0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009e80 tx=0x7f5cf401ca10 comp rx=0 tx=0).stop 2024-04-25T10:10:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 -- 172.21.15.60:0/1852103287 shutdown_connections 2024-04-25T10:10:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 --2- 172.21.15.60:0/1852103287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cfc0b4660 0x7f5cfc0b2ce0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 --2- 172.21.15.60:0/1852103287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cfc0ce7a0 0x7f5cfc0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 -- 172.21.15.60:0/1852103287 >> 172.21.15.60:0/1852103287 conn(0x7f5cfc0b12d0 msgr2=0x7f5cfc0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 -- 172.21.15.60:0/1852103287 shutdown_connections 2024-04-25T10:10:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 -- 172.21.15.60:0/1852103287 wait complete. 2024-04-25T10:10:06.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 Processor -- start 2024-04-25T10:10:06.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 -- start start 2024-04-25T10:10:06.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cfc0ce7a0 0x7f5cfc1599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cfc15bc90 0x7f5cfc149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cfc14b920 0x7f5cfc0b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc0b4280 con 0x7f5cfc14b920 2024-04-25T10:10:06.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc0b3db0 con 0x7f5cfc15bc90 2024-04-25T10:10:06.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc0b4400 con 0x7f5cfc0ce7a0 2024-04-25T10:10:06.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5cfb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cfc14b920 0x7f5cfc0b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5cfb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cfc14b920 0x7f5cfc0b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50906/0 (socket says 172.21.15.60:50906) 2024-04-25T10:10:06.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5cfb7fe640 1 -- 172.21.15.60:0/621300563 learned_addr learned my addr 172.21.15.60:0/621300563 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:06.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5cfaffd640 1 --2- 172.21.15.60:0/621300563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cfc0ce7a0 0x7f5cfc1599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5cfb7fe640 1 -- 172.21.15.60:0/621300563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cfc0ce7a0 msgr2=0x7f5cfc1599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5cfb7fe640 1 --2- 172.21.15.60:0/621300563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cfc0ce7a0 0x7f5cfc1599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5cfb7fe640 1 -- 172.21.15.60:0/621300563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cfc15bc90 msgr2=0x7f5cfc149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5cfa7fc640 1 --2- 172.21.15.60:0/621300563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cfc15bc90 0x7f5cfc149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5cfb7fe640 1 --2- 172.21.15.60:0/621300563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cfc15bc90 0x7f5cfc149360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5cfb7fe640 1 -- 172.21.15.60:0/621300563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc14b2b0 con 0x7f5cfc14b920 2024-04-25T10:10:06.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5cfaffd640 1 --2- 172.21.15.60:0/621300563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cfc0ce7a0 0x7f5cfc1599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:06.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5cfa7fc640 1 --2- 172.21.15.60:0/621300563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cfc15bc90 0x7f5cfc149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:06.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5cfb7fe640 1 --2- 172.21.15.60:0/621300563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cfc14b920 0x7f5cfc0b2c00 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40af040 tx=0x7f5cf40197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5ce37fe640 1 -- 172.21.15.60:0/621300563 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40b8670 con 0x7f5cfc14b920 2024-04-25T10:10:06.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5ce37fe640 1 -- 172.21.15.60:0/621300563 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf40be070 con 0x7f5cfc14b920 2024-04-25T10:10:06.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 -- 172.21.15.60:0/621300563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cfc14c510 con 0x7f5cfc14b920 2024-04-25T10:10:06.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5ce37fe640 1 -- 172.21.15.60:0/621300563 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40b8a70 con 0x7f5cfc14b920 2024-04-25T10:10:06.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.377+0000 7f5d01494640 1 -- 172.21.15.60:0/621300563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cfc14cad0 con 0x7f5cfc14b920 2024-04-25T10:10:06.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.381+0000 7f5d01494640 1 -- 172.21.15.60:0/621300563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cfc14cd10 con 0x7f5cfc14b920 2024-04-25T10:10:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.381+0000 7f5ce37fe640 1 -- 172.21.15.60:0/621300563 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5cf40b8c10 con 0x7f5cfc14b920 2024-04-25T10:10:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.381+0000 7f5ce37fe640 1 --2- 172.21.15.60:0/621300563 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5ce8066f90 0x7f5ce8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.381+0000 7f5ce37fe640 1 -- 172.21.15.60:0/621300563 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5cf411fa60 con 0x7f5cfc14b920 2024-04-25T10:10:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.381+0000 7f5cfaffd640 1 --2- 172.21.15.60:0/621300563 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5ce8066f90 0x7f5ce8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.381+0000 7f5cfaffd640 1 --2- 172.21.15.60:0/621300563 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5ce8066f90 0x7f5ce8069450 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0097a0 tx=0x7f5cec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.385+0000 7f5ce37fe640 1 -- 172.21.15.60:0/621300563 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5cfc14cd10 con 0x7f5cfc14b920 2024-04-25T10:10:06.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.573+0000 7f5d01494640 1 -- 172.21.15.60:0/621300563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cfc074770 con 0x7f5cfc14b920 2024-04-25T10:10:06.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.577+0000 7f5ce37fe640 1 -- 172.21.15.60:0/621300563 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f5cf40c7020 con 0x7f5cfc14b920 2024-04-25T10:10:06.584 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:10:06.584 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3185787428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3185787428},{"type":"v1","addr":"172.21.15.100:6836","nonce":3185787428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2667415244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2667415244},{"type":"v1","addr":"172.21.15.100:6837","nonce":2667415244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/56747111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":56747111},{"type":"v1","addr":"172.21.15.60:6834","nonce":56747111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:43.707831+0000","modified":"2024-04-25T10:10:05.473959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7134},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7134":{"gid":7134,"name":"c","rank":0,"incarnation":42,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.60:6835/1295867731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1295867731},{"type":"v1","addr":"172.21.15.60:6835","nonce":1295867731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:10:06.584 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-04-25T10:10:06.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.577+0000 7f5d01494640 1 -- 172.21.15.60:0/621300563 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5ce8066f90 msgr2=0x7f5ce8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.577+0000 7f5d01494640 1 --2- 172.21.15.60:0/621300563 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5ce8066f90 0x7f5ce8069450 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0097a0 tx=0x7f5cec008040 comp rx=0 tx=0).stop 2024-04-25T10:10:06.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.577+0000 7f5d01494640 1 -- 172.21.15.60:0/621300563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cfc14b920 msgr2=0x7f5cfc0b2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.577+0000 7f5d01494640 1 --2- 172.21.15.60:0/621300563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cfc14b920 0x7f5cfc0b2c00 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40af040 tx=0x7f5cf40197b0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.577+0000 7f5d01494640 1 -- 172.21.15.60:0/621300563 shutdown_connections 2024-04-25T10:10:06.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.577+0000 7f5d01494640 1 --2- 172.21.15.60:0/621300563 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5ce8066f90 0x7f5ce8069450 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.577+0000 7f5d01494640 1 --2- 172.21.15.60:0/621300563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cfc14b920 0x7f5cfc0b2c00 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.577+0000 7f5d01494640 1 --2- 172.21.15.60:0/621300563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cfc15bc90 0x7f5cfc149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.577+0000 7f5d01494640 1 --2- 172.21.15.60:0/621300563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cfc0ce7a0 0x7f5cfc1599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.577+0000 7f5d01494640 1 -- 172.21.15.60:0/621300563 >> 172.21.15.60:0/621300563 conn(0x7f5cfc0b12d0 msgr2=0x7f5cfc15de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.577+0000 7f5d01494640 1 -- 172.21.15.60:0/621300563 shutdown_connections 2024-04-25T10:10:06.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:06.577+0000 7f5d01494640 1 -- 172.21.15.60:0/621300563 wait complete. 2024-04-25T10:10:07.004 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:09:43.707831+0000', 'modified': '2024-04-25T10:10:05.473959+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7134}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7134': {'gid': 7134, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.60:6835/1295867731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 1295867731}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 1295867731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:10:07.004 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7134:up:rejoin 2024-04-25T10:10:08.005 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:08.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c36154640 1 Processor -- start 2024-04-25T10:10:08.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c36154640 1 -- start start 2024-04-25T10:10:08.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c36154640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c3006ec30 0x7f1c300cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c36154640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1c300b37e0 0x7f1c300b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c36154640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c3006f130 con 0x7f1c3015a7a0 2024-04-25T10:10:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c36154640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c300cdaa0 con 0x7f1c300b37e0 2024-04-25T10:10:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c36154640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3006d2f0 con 0x7f1c3006ec30 2024-04-25T10:10:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1c3015a7a0 0x7f1c30071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49242/0 (socket says 172.21.15.60:49242) 2024-04-25T10:10:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2f7fe640 1 -- 172.21.15.60:0/923599887 learned_addr learned my addr 172.21.15.60:0/923599887 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:10:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2ffff640 1 --2- 172.21.15.60:0/923599887 >> v2:172.21.15.100:3300/0 conn(0x7f1c300b37e0 0x7f1c300b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c26ffd640 1 --2- 172.21.15.60:0/923599887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c3006ec30 0x7f1c300cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2effd640 1 -- 172.21.15.60:0/923599887 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386785983 0 0) 0x7f1c3006f130 con 0x7f1c3015a7a0 2024-04-25T10:10:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2effd640 1 -- 172.21.15.60:0/923599887 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c1c003530 con 0x7f1c3015a7a0 2024-04-25T10:10:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2effd640 1 -- 172.21.15.60:0/923599887 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3461420058 0 0) 0x7f1c1c003530 con 0x7f1c3015a7a0 2024-04-25T10:10:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2effd640 1 -- 172.21.15.60:0/923599887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c3006ec30 msgr2=0x7f1c300cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2effd640 1 --2- 172.21.15.60:0/923599887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c3006ec30 0x7f1c300cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2effd640 1 -- 172.21.15.60:0/923599887 >> v2:172.21.15.100:3300/0 conn(0x7f1c300b37e0 msgr2=0x7f1c300b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2effd640 1 --2- 172.21.15.60:0/923599887 >> v2:172.21.15.100:3300/0 conn(0x7f1c300b37e0 0x7f1c300b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2effd640 1 -- 172.21.15.60:0/923599887 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3006f410 con 0x7f1c3015a7a0 2024-04-25T10:10:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2effd640 1 -- 172.21.15.60:0/923599887 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f1c20001680 con 0x7f1c3015a7a0 2024-04-25T10:10:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2effd640 1 -- 172.21.15.60:0/923599887 >> v1:172.21.15.60:6789/0 conn(0x7f1c3015a7a0 legacy=0x7f1c30071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2effd640 1 --2- 172.21.15.60:0/923599887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1c0039c0 0x7f1c1c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2effd640 1 --2- 172.21.15.60:0/923599887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c1c004810 0x7f1c1c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2effd640 1 --2- 172.21.15.60:0/923599887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c3006ec30 0x7f1c1c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2effd640 1 -- 172.21.15.60:0/923599887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3006d2f0 con 0x7f1c1c004810 2024-04-25T10:10:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2effd640 1 -- 172.21.15.60:0/923599887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c00b9d0 con 0x7f1c1c0039c0 2024-04-25T10:10:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2effd640 1 -- 172.21.15.60:0/923599887 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c00bb50 con 0x7f1c3006ec30 2024-04-25T10:10:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c26ffd640 1 --2- 172.21.15.60:0/923599887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c1c004810 0x7f1c1c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2f7fe640 1 --2- 172.21.15.60:0/923599887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1c0039c0 0x7f1c1c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c26ffd640 1 -- 172.21.15.60:0/923599887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c3006ec30 msgr2=0x7f1c1c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:08.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2ffff640 1 --2- 172.21.15.60:0/923599887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c3006ec30 0x7f1c1c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c26ffd640 1 --2- 172.21.15.60:0/923599887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c3006ec30 0x7f1c1c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c26ffd640 1 -- 172.21.15.60:0/923599887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1c0039c0 msgr2=0x7f1c1c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:08.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c26ffd640 1 --2- 172.21.15.60:0/923599887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1c0039c0 0x7f1c1c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c26ffd640 1 -- 172.21.15.60:0/923599887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c003530 con 0x7f1c1c004810 2024-04-25T10:10:08.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.133+0000 7f1c2f7fe640 1 --2- 172.21.15.60:0/923599887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1c0039c0 0x7f1c1c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:08.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.137+0000 7f1c26ffd640 1 --2- 172.21.15.60:0/923599887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c1c004810 0x7f1c1c004c90 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009ae0 tx=0x7f1c1801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149df807cb96211d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:08.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.137+0000 7f1c2effd640 1 -- 172.21.15.60:0/923599887 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c180b2070 con 0x7f1c1c004810 2024-04-25T10:10:08.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.137+0000 7f1c2effd640 1 -- 172.21.15.60:0/923599887 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c180a55a0 con 0x7f1c1c004810 2024-04-25T10:10:08.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.137+0000 7f1c2effd640 1 -- 172.21.15.60:0/923599887 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c180096b0 con 0x7f1c1c004810 2024-04-25T10:10:08.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.137+0000 7f1c36154640 1 -- 172.21.15.60:0/923599887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c1c004810 msgr2=0x7f1c1c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.137+0000 7f1c36154640 1 --2- 172.21.15.60:0/923599887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c1c004810 0x7f1c1c004c90 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009ae0 tx=0x7f1c1801ca90 comp rx=0 tx=0).stop 2024-04-25T10:10:08.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.137+0000 7f1c2ffff640 1 -- 172.21.15.60:0/923599887 reap_dead start 2024-04-25T10:10:08.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.137+0000 7f1c36154640 1 -- 172.21.15.60:0/923599887 shutdown_connections 2024-04-25T10:10:08.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.137+0000 7f1c36154640 1 -- 172.21.15.60:0/923599887 >> 172.21.15.60:0/923599887 conn(0x7f1c300b12d0 msgr2=0x7f1c300b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:08.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.137+0000 7f1c36154640 1 -- 172.21.15.60:0/923599887 shutdown_connections 2024-04-25T10:10:08.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.137+0000 7f1c36154640 1 -- 172.21.15.60:0/923599887 wait complete. 2024-04-25T10:10:08.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.137+0000 7f1c36154640 1 Processor -- start 2024-04-25T10:10:08.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.137+0000 7f1c36154640 1 -- start start 2024-04-25T10:10:08.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c36154640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c1c004810 0x7f1c30159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c36154640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c301563d0 0x7f1c3014efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c36154640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c30142b30 0x7f1c30142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c36154640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3015ab80 con 0x7f1c301563d0 2024-04-25T10:10:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c36154640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3006d8b0 con 0x7f1c30142b30 2024-04-25T10:10:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c36154640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3015ad00 con 0x7f1c1c004810 2024-04-25T10:10:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c26ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c301563d0 0x7f1c3014efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c26ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c301563d0 0x7f1c3014efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50928/0 (socket says 172.21.15.60:50928) 2024-04-25T10:10:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c26ffd640 1 -- 172.21.15.60:0/897443457 learned_addr learned my addr 172.21.15.60:0/897443457 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c26ffd640 1 -- 172.21.15.60:0/897443457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c1c004810 msgr2=0x7f1c30159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c26ffd640 1 --2- 172.21.15.60:0/897443457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c1c004810 0x7f1c30159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c26ffd640 1 -- 172.21.15.60:0/897443457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c30142b30 msgr2=0x7f1c30142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c26ffd640 1 --2- 172.21.15.60:0/897443457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c30142b30 0x7f1c30142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c26ffd640 1 -- 172.21.15.60:0/897443457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c30159880 con 0x7f1c301563d0 2024-04-25T10:10:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c26ffd640 1 --2- 172.21.15.60:0/897443457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c301563d0 0x7f1c3014efd0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f1c1801c8c0 tx=0x7f1c1c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:08.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c2cff9640 1 -- 172.21.15.60:0/897443457 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c180b2040 con 0x7f1c301563d0 2024-04-25T10:10:08.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c2cff9640 1 -- 172.21.15.60:0/897443457 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c18004020 con 0x7f1c301563d0 2024-04-25T10:10:08.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c36154640 1 -- 172.21.15.60:0/897443457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c30159b60 con 0x7f1c301563d0 2024-04-25T10:10:08.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c2cff9640 1 -- 172.21.15.60:0/897443457 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c180043d0 con 0x7f1c301563d0 2024-04-25T10:10:08.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c36154640 1 -- 172.21.15.60:0/897443457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3015a120 con 0x7f1c301563d0 2024-04-25T10:10:08.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c36154640 1 -- 172.21.15.60:0/897443457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c30074770 con 0x7f1c301563d0 2024-04-25T10:10:08.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c2cff9640 1 -- 172.21.15.60:0/897443457 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c180c0060 con 0x7f1c301563d0 2024-04-25T10:10:08.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c2cff9640 1 --2- 172.21.15.60:0/897443457 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c10066f60 0x7f1c10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c2cff9640 1 -- 172.21.15.60:0/897443457 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1c1811eb90 con 0x7f1c301563d0 2024-04-25T10:10:08.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.141+0000 7f1c2f7fe640 1 --2- 172.21.15.60:0/897443457 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c10066f60 0x7f1c10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.145+0000 7f1c2f7fe640 1 --2- 172.21.15.60:0/897443457 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c10066f60 0x7f1c10069420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1c20004880 tx=0x7f1c20000f00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:08.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.145+0000 7f1c2cff9640 1 -- 172.21.15.60:0/897443457 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c180167a0 con 0x7f1c301563d0 2024-04-25T10:10:08.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.333+0000 7f1c36154640 1 -- 172.21.15.60:0/897443457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c30151ae0 con 0x7f1c301563d0 2024-04-25T10:10:08.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.337+0000 7f1c2cff9640 1 -- 172.21.15.60:0/897443457 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f1c180efb10 con 0x7f1c301563d0 2024-04-25T10:10:08.344 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:10:08.345 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3185787428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3185787428},{"type":"v1","addr":"172.21.15.100:6836","nonce":3185787428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2667415244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2667415244},{"type":"v1","addr":"172.21.15.100:6837","nonce":2667415244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/56747111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":56747111},{"type":"v1","addr":"172.21.15.60:6834","nonce":56747111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:43.707831+0000","modified":"2024-04-25T10:10:06.476054+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7134},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7134":{"gid":7134,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/1295867731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1295867731},{"type":"v1","addr":"172.21.15.60:6835","nonce":1295867731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:10:08.345 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 45 2024-04-25T10:10:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.337+0000 7f1c36154640 1 -- 172.21.15.60:0/897443457 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c10066f60 msgr2=0x7f1c10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.337+0000 7f1c36154640 1 --2- 172.21.15.60:0/897443457 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c10066f60 0x7f1c10069420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1c20004880 tx=0x7f1c20000f00 comp rx=0 tx=0).stop 2024-04-25T10:10:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.337+0000 7f1c36154640 1 -- 172.21.15.60:0/897443457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c301563d0 msgr2=0x7f1c3014efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.337+0000 7f1c36154640 1 --2- 172.21.15.60:0/897443457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c301563d0 0x7f1c3014efd0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f1c1801c8c0 tx=0x7f1c1c005840 comp rx=0 tx=0).stop 2024-04-25T10:10:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.337+0000 7f1c36154640 1 -- 172.21.15.60:0/897443457 shutdown_connections 2024-04-25T10:10:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.337+0000 7f1c36154640 1 --2- 172.21.15.60:0/897443457 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c10066f60 0x7f1c10069420 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.337+0000 7f1c36154640 1 --2- 172.21.15.60:0/897443457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c30142b30 0x7f1c30142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.337+0000 7f1c36154640 1 --2- 172.21.15.60:0/897443457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c301563d0 0x7f1c3014efd0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.337+0000 7f1c36154640 1 --2- 172.21.15.60:0/897443457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c1c004810 0x7f1c30159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.337+0000 7f1c36154640 1 -- 172.21.15.60:0/897443457 >> 172.21.15.60:0/897443457 conn(0x7f1c300b12d0 msgr2=0x7f1c3015d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:08.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.337+0000 7f1c36154640 1 -- 172.21.15.60:0/897443457 shutdown_connections 2024-04-25T10:10:08.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.337+0000 7f1c36154640 1 -- 172.21.15.60:0/897443457 wait complete. 2024-04-25T10:10:08.766 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:09:43.707831+0000', 'modified': '2024-04-25T10:10:06.476054+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7134}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7134': {'gid': 7134, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.60:6835/1295867731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 1295867731}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 1295867731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:10:08.767 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:10:08.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:10:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.893+0000 7f1cfbdd3640 1 Processor -- start 2024-04-25T10:10:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.893+0000 7f1cfbdd3640 1 -- start start 2024-04-25T10:10:08.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.893+0000 7f1cfbdd3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1cf406e480 0x7f1cf406e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.893+0000 7f1cfbdd3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf415dc10 con 0x7f1cf4071b20 2024-04-25T10:10:08.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.893+0000 7f1cfbdd3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf415def0 con 0x7f1cf40cce50 2024-04-25T10:10:08.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.893+0000 7f1cfbdd3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf415e1d0 con 0x7f1cf406e480 2024-04-25T10:10:08.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.893+0000 7f1cf9b6f640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1cf4071b20 0x7f1cf4071f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55360/0 (socket says 172.21.15.60:55360) 2024-04-25T10:10:08.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.893+0000 7f1cf9b6f640 1 -- 172.21.15.60:0/367382778 learned_addr learned my addr 172.21.15.60:0/367382778 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:10:08.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.897+0000 7f1cf936e640 1 --2- 172.21.15.60:0/367382778 >> v2:172.21.15.60:3300/0 conn(0x7f1cf406e480 0x7f1cf406e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.897+0000 7f1cf8b6d640 1 -- 172.21.15.60:0/367382778 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167551568 0 0) 0x7f1cf415dc10 con 0x7f1cf4071b20 2024-04-25T10:10:08.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.897+0000 7f1cf8b6d640 1 -- 172.21.15.60:0/367382778 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ce0003530 con 0x7f1cf4071b20 2024-04-25T10:10:08.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.897+0000 7f1cf936e640 1 -- 172.21.15.60:0/367382778 >> v1:172.21.15.100:6789/0 conn(0x7f1cf40cce50 legacy=0x7f1cf40cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.897+0000 7f1cf936e640 1 -- 172.21.15.60:0/367382778 >> v1:172.21.15.60:6789/0 conn(0x7f1cf4071b20 legacy=0x7f1cf4071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.897+0000 7f1cf8b6d640 1 -- 172.21.15.60:0/367382778 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3844581866 0 0) 0x7f1ce0003530 con 0x7f1cf4071b20 2024-04-25T10:10:08.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.897+0000 7f1cf936e640 1 -- 172.21.15.60:0/367382778 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf415e4b0 con 0x7f1cf406e480 2024-04-25T10:10:08.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.897+0000 7f1cf936e640 1 --2- 172.21.15.60:0/367382778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cf406e480 0x7f1cf406e910 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400a960 tx=0x7f1ce401d700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a31ea1c1d3cab73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:08.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.897+0000 7f1cf8b6d640 1 -- 172.21.15.60:0/367382778 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0001680 con 0x7f1cf406e480 2024-04-25T10:10:08.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.897+0000 7f1cf8b6d640 1 -- 172.21.15.60:0/367382778 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce40a9450 con 0x7f1cf406e480 2024-04-25T10:10:08.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.897+0000 7f1cf8b6d640 1 -- 172.21.15.60:0/367382778 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce400b040 con 0x7f1cf406e480 2024-04-25T10:10:08.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cfbdd3640 1 -- 172.21.15.60:0/367382778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cf406e480 msgr2=0x7f1cf406e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cfbdd3640 1 --2- 172.21.15.60:0/367382778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cf406e480 0x7f1cf406e910 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400a960 tx=0x7f1ce401d700 comp rx=0 tx=0).stop 2024-04-25T10:10:08.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cfbdd3640 1 -- 172.21.15.60:0/367382778 shutdown_connections 2024-04-25T10:10:08.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cfbdd3640 1 --2- 172.21.15.60:0/367382778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cf406e480 0x7f1cf406e910 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cfbdd3640 1 -- 172.21.15.60:0/367382778 >> 172.21.15.60:0/367382778 conn(0x7f1cf40b12d0 msgr2=0x7f1cf40b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:08.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cfbdd3640 1 -- 172.21.15.60:0/367382778 shutdown_connections 2024-04-25T10:10:08.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cfbdd3640 1 -- 172.21.15.60:0/367382778 wait complete. 2024-04-25T10:10:08.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cfbdd3640 1 Processor -- start 2024-04-25T10:10:08.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cfbdd3640 1 -- start start 2024-04-25T10:10:08.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cfbdd3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cf406e480 0x7f1cf4129880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cfbdd3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cf4071b20 0x7f1cf4157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cfbdd3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf40cce50 0x7f1cf413d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cfbdd3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf40cdb10 con 0x7f1cf4071b20 2024-04-25T10:10:08.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cfbdd3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf40cd990 con 0x7f1cf40cce50 2024-04-25T10:10:08.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cfbdd3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf40cdc90 con 0x7f1cf406e480 2024-04-25T10:10:08.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cf936e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cf4071b20 0x7f1cf4157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cf936e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cf4071b20 0x7f1cf4157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47940/0 (socket says 172.21.15.60:47940) 2024-04-25T10:10:08.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cf936e640 1 -- 172.21.15.60:0/3878279087 learned_addr learned my addr 172.21.15.60:0/3878279087 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:08.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cfa370640 1 --2- 172.21.15.60:0/3878279087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf40cce50 0x7f1cf413d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cf9b6f640 1 --2- 172.21.15.60:0/3878279087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cf406e480 0x7f1cf4129880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cf936e640 1 -- 172.21.15.60:0/3878279087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cf406e480 msgr2=0x7f1cf4129880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cf936e640 1 --2- 172.21.15.60:0/3878279087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cf406e480 0x7f1cf4129880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cf936e640 1 -- 172.21.15.60:0/3878279087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf40cce50 msgr2=0x7f1cf413d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cf936e640 1 --2- 172.21.15.60:0/3878279087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf40cce50 0x7f1cf413d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cf936e640 1 -- 172.21.15.60:0/3878279087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf41585c0 con 0x7f1cf4071b20 2024-04-25T10:10:08.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cf9b6f640 1 --2- 172.21.15.60:0/3878279087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cf406e480 0x7f1cf4129880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:08.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.901+0000 7f1cf936e640 1 --2- 172.21.15.60:0/3878279087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cf4071b20 0x7f1cf4157b90 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400a960 tx=0x7f1ce401bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:08.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cea7fc640 1 -- 172.21.15.60:0/3878279087 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4006e20 con 0x7f1cf4071b20 2024-04-25T10:10:08.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cea7fc640 1 -- 172.21.15.60:0/3878279087 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce4004220 con 0x7f1cf4071b20 2024-04-25T10:10:08.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cea7fc640 1 -- 172.21.15.60:0/3878279087 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40a9e60 con 0x7f1cf4071b20 2024-04-25T10:10:08.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cfbdd3640 1 -- 172.21.15.60:0/3878279087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf40af330 con 0x7f1cf4071b20 2024-04-25T10:10:08.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cfbdd3640 1 -- 172.21.15.60:0/3878279087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf40af5d0 con 0x7f1cf4071b20 2024-04-25T10:10:08.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cea7fc640 1 -- 172.21.15.60:0/3878279087 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ce40c0060 con 0x7f1cf4071b20 2024-04-25T10:10:08.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cea7fc640 1 --2- 172.21.15.60:0/3878279087 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cd8066f60 0x7f1cd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cea7fc640 1 -- 172.21.15.60:0/3878279087 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1ce411ef80 con 0x7f1cf4071b20 2024-04-25T10:10:08.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cfef40000 1 Processor -- start 2024-04-25T10:10:08.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cf9b6f640 1 --2- 172.21.15.60:0/3878279087 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cd8066f60 0x7f1cd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cfef40000 1 -- start start 2024-04-25T10:10:08.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cfef40000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a50d80 0x559d89a51160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cfef40000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a516a0 0x559d89a071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cfef40000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a07720 0x559d89a07b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cfef40000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d898ea5f0 con 0x559d89a50d80 2024-04-25T10:10:08.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cfef40000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d89851da0 con 0x559d89a07720 2024-04-25T10:10:08.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cfef40000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d897da150 con 0x559d89a516a0 2024-04-25T10:10:08.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cf9b6f640 1 --2- 172.21.15.60:0/3878279087 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cd8066f60 0x7f1cd8069420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00011f0 tx=0x7f1cf0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:08.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.905+0000 7f1cf9b6f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a50d80 0x559d89a51160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a50d80 0x559d89a51160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47942/0 (socket says 172.21.15.60:47942) 2024-04-25T10:10:08.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 -- 172.21.15.60:0/2795532960 learned_addr learned my addr 172.21.15.60:0/2795532960 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:08.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf936e640 1 --2- 172.21.15.60:0/2795532960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a516a0 0x559d89a071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfa370640 1 --2- 172.21.15.60:0/2795532960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a07720 0x559d89a07b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 -- 172.21.15.60:0/2795532960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a516a0 msgr2=0x559d89a071e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:08.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 --2- 172.21.15.60:0/2795532960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a516a0 0x559d89a071e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 -- 172.21.15.60:0/2795532960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a07720 msgr2=0x559d89a07b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:08.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 --2- 172.21.15.60:0/2795532960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a07720 0x559d89a07b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 -- 172.21.15.60:0/2795532960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d897f3ca0 con 0x559d89a50d80 2024-04-25T10:10:08.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 --2- 172.21.15.60:0/2795532960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a50d80 0x559d89a51160 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0005d00 tx=0x7f1cf0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482bba4dbcfad387 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:08.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cd77fe640 1 -- 172.21.15.60:0/2795532960 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0007560 con 0x559d89a50d80 2024-04-25T10:10:08.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cd77fe640 1 -- 172.21.15.60:0/2795532960 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf0007700 con 0x559d89a50d80 2024-04-25T10:10:08.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cd77fe640 1 -- 172.21.15.60:0/2795532960 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0020650 con 0x559d89a50d80 2024-04-25T10:10:08.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 -- 172.21.15.60:0/2795532960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a50d80 msgr2=0x559d89a51160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 --2- 172.21.15.60:0/2795532960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a50d80 0x559d89a51160 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0005d00 tx=0x7f1cf0010000 comp rx=0 tx=0).stop 2024-04-25T10:10:08.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 -- 172.21.15.60:0/2795532960 shutdown_connections 2024-04-25T10:10:08.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 --2- 172.21.15.60:0/2795532960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a07720 0x559d89a07b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 --2- 172.21.15.60:0/2795532960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a516a0 0x559d89a071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 --2- 172.21.15.60:0/2795532960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a50d80 0x559d89a51160 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 -- 172.21.15.60:0/2795532960 >> 172.21.15.60:0/2795532960 conn(0x559d89a036d0 msgr2=0x559d89a05ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:08.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 -- 172.21.15.60:0/2795532960 shutdown_connections 2024-04-25T10:10:08.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 -- 172.21.15.60:0/2795532960 wait complete. 2024-04-25T10:10:08.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 -- start start 2024-04-25T10:10:08.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 Processor -- start 2024-04-25T10:10:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a07720 0x559d89a583d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a50d80 0x559d89a58910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a516a0 0x559d89a5d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8980c990 con 0x559d89a07720 2024-04-25T10:10:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d8984de40 con 0x559d89a50d80 2024-04-25T10:10:08.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d898510b0 con 0x559d89a516a0 2024-04-25T10:10:08.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 10 client.0 _ll_register_callbacks cb 0x559d899051c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:08.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a07720 0x559d89a583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a07720 0x559d89a583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47950/0 (socket says 172.21.15.60:47950) 2024-04-25T10:10:08.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 -- 172.21.15.60:0/2940249629 learned_addr learned my addr 172.21.15.60:0/2940249629 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:08.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfa370640 1 --2- 172.21.15.60:0/2940249629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a516a0 0x559d89a5d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf936e640 1 --2- 172.21.15.60:0/2940249629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a50d80 0x559d89a58910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 -- 172.21.15.60:0/2940249629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a516a0 msgr2=0x559d89a5d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:08.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 --2- 172.21.15.60:0/2940249629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a516a0 0x559d89a5d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 -- 172.21.15.60:0/2940249629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a50d80 msgr2=0x559d89a58910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:08.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 --2- 172.21.15.60:0/2940249629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a50d80 0x559d89a58910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 -- 172.21.15.60:0/2940249629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d8980a630 con 0x559d89a07720 2024-04-25T10:10:08.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfa370640 1 --2- 172.21.15.60:0/2940249629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a516a0 0x559d89a5d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:08.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf936e640 1 --2- 172.21.15.60:0/2940249629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a50d80 0x559d89a58910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:08.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cf9b6f640 1 --2- 172.21.15.60:0/2940249629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a07720 0x559d89a583d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0000c10 tx=0x7f1cf0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7baf41da713910 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:08.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:10:08.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cd4ff9640 1 -- 172.21.15.60:0/2940249629 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0007560 con 0x559d89a07720 2024-04-25T10:10:08.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cd4ff9640 1 -- 172.21.15.60:0/2940249629 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1cf0022cd0 con 0x559d89a07720 2024-04-25T10:10:08.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.909+0000 7f1cfef40000 1 -- 172.21.15.60:0/2940249629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1cf0007700 con 0x559d89a07720 2024-04-25T10:10:08.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cd4ff9640 1 -- 172.21.15.60:0/2940249629 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cf0033050 con 0x559d89a07720 2024-04-25T10:10:08.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 10 client.7179 fetch_fsmap learned FSMap version 45 2024-04-25T10:10:08.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 1 -- 172.21.15.60:0/2940249629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x559d897da3a0 con 0x559d89a07720 2024-04-25T10:10:08.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cd4ff9640 1 -- 172.21.15.60:0/2940249629 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf0007700 con 0x559d89a07720 2024-04-25T10:10:08.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cd4ff9640 1 -- 172.21.15.60:0/2940249629 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 45) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1cf0036020 con 0x559d89a07720 2024-04-25T10:10:08.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 10 client.7179 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T10:10:08.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 10 client.7179 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:10:08.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 20 client.7179 populate_metadata read hostname 'smithi060' 2024-04-25T10:10:08.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 1 --2- 172.21.15.60:0/2940249629 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x559d89a5ffa0 0x559d89a62390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 4 client.7179 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:08.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 1 -- 172.21.15.60:0/2940249629 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d897eda50 con 0x559d89a5ffa0 2024-04-25T10:10:08.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cf936e640 1 --2- 172.21.15.60:0/2940249629 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x559d89a5ffa0 0x559d89a62390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cf936e640 1 --2- 172.21.15.60:0/2940249629 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x559d89a5ffa0 0x559d89a62390 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96e156176c334f9a server_cookie=bc2536bd6dc79975 in_seq=0 out_seq=0 2024-04-25T10:10:08.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cd4ff9640 10 client.7179 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:08.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cd4ff9640 1 -- 172.21.15.60:0/2940249629 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559d897eda50 con 0x559d89a5ffa0 2024-04-25T10:10:08.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cd4ff9640 10 client.7179 handle_command_reply: tid=0 2024-04-25T10:10:08.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 1 client.7179 shutdown 2024-04-25T10:10:08.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 1 -- 172.21.15.60:0/2940249629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a07720 msgr2=0x559d89a583d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 1 --2- 172.21.15.60:0/2940249629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a07720 0x559d89a583d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0000c10 tx=0x7f1cf0012000 comp rx=0 tx=0).stop 2024-04-25T10:10:08.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 1 -- 172.21.15.60:0/2940249629 shutdown_connections 2024-04-25T10:10:08.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 1 --2- 172.21.15.60:0/2940249629 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x559d89a5ffa0 0x559d89a62390 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 1 --2- 172.21.15.60:0/2940249629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a516a0 0x559d89a5d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 1 --2- 172.21.15.60:0/2940249629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a50d80 0x559d89a58910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cd4ff9640 0 client.7179 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:08.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 1 --2- 172.21.15.60:0/2940249629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a07720 0x559d89a583d0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 1 -- 172.21.15.60:0/2940249629 >> 172.21.15.60:0/2940249629 conn(0x559d89a036d0 msgr2=0x559d89a08390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:08.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 1 -- 172.21.15.60:0/2940249629 shutdown_connections 2024-04-25T10:10:08.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 1 -- 172.21.15.60:0/2940249629 wait complete. 2024-04-25T10:10:08.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.913+0000 7f1cfef40000 20 client.7179 trim_cache size 0 max 16384 2024-04-25T10:10:08.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 Processor -- start 2024-04-25T10:10:08.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 -- start start 2024-04-25T10:10:08.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a036d0 0x559d89aaca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a07720 0x559d89aacf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a50d80 0x559d89aad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf4142820 con 0x559d89a07720 2024-04-25T10:10:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf4141c20 con 0x559d89a036d0 2024-04-25T10:10:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf4142220 con 0x559d89a50d80 2024-04-25T10:10:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cf936e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a07720 0x559d89aacf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cf936e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a07720 0x559d89aacf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47954/0 (socket says 172.21.15.60:47954) 2024-04-25T10:10:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cf936e640 1 -- 172.21.15.60:0/2597444480 learned_addr learned my addr 172.21.15.60:0/2597444480 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cf936e640 1 -- 172.21.15.60:0/2597444480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a50d80 msgr2=0x559d89aad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cf936e640 1 --2- 172.21.15.60:0/2597444480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a50d80 0x559d89aad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cf936e640 1 -- 172.21.15.60:0/2597444480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a036d0 msgr2=0x559d89aaca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cf936e640 1 --2- 172.21.15.60:0/2597444480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a036d0 0x559d89aaca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cf936e640 1 -- 172.21.15.60:0/2597444480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d898d3600 con 0x559d89a07720 2024-04-25T10:10:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cf936e640 1 --2- 172.21.15.60:0/2597444480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a07720 0x559d89aacf90 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40ebca0 tx=0x7f1ce400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd09a25ad2e0f286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cd6ffd640 1 -- 172.21.15.60:0/2597444480 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce411a020 con 0x559d89a07720 2024-04-25T10:10:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cd6ffd640 1 -- 172.21.15.60:0/2597444480 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce40f7fa0 con 0x559d89a07720 2024-04-25T10:10:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cd6ffd640 1 -- 172.21.15.60:0/2597444480 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40edb90 con 0x559d89a07720 2024-04-25T10:10:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 -- 172.21.15.60:0/2597444480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a07720 msgr2=0x559d89aacf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 --2- 172.21.15.60:0/2597444480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a07720 0x559d89aacf90 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40ebca0 tx=0x7f1ce400c040 comp rx=0 tx=0).stop 2024-04-25T10:10:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 -- 172.21.15.60:0/2597444480 shutdown_connections 2024-04-25T10:10:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 --2- 172.21.15.60:0/2597444480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a50d80 0x559d89aad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 --2- 172.21.15.60:0/2597444480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a07720 0x559d89aacf90 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 --2- 172.21.15.60:0/2597444480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a036d0 0x559d89aaca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 -- 172.21.15.60:0/2597444480 >> 172.21.15.60:0/2597444480 conn(0x559d89a06b80 msgr2=0x559d89a5c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 -- 172.21.15.60:0/2597444480 shutdown_connections 2024-04-25T10:10:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.929+0000 7f1cfef40000 1 -- 172.21.15.60:0/2597444480 wait complete. 2024-04-25T10:10:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 -- start start 2024-04-25T10:10:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 Processor -- start 2024-04-25T10:10:08.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a036d0 0x559d89aae560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a07720 0x559d89aaeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a50d80 0x559d89aaefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d897ea750 con 0x559d89a036d0 2024-04-25T10:10:08.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d898563d0 con 0x559d89a07720 2024-04-25T10:10:08.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d897f74b0 con 0x559d89a50d80 2024-04-25T10:10:08.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 10 client.0 _ll_register_callbacks cb 0x7f1cf00058e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:08.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cf9b6f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a036d0 0x559d89aae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cf9b6f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a036d0 0x559d89aae560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47960/0 (socket says 172.21.15.60:47960) 2024-04-25T10:10:08.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cf9b6f640 1 -- 172.21.15.60:0/1322068310 learned_addr learned my addr 172.21.15.60:0/1322068310 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:08.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cf9b6f640 1 -- 172.21.15.60:0/1322068310 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a50d80 msgr2=0x559d89aaefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:08.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cf9b6f640 1 --2- 172.21.15.60:0/1322068310 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a50d80 0x559d89aaefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cf9b6f640 1 -- 172.21.15.60:0/1322068310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a07720 msgr2=0x559d89aaeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:08.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cf936e640 1 --2- 172.21.15.60:0/1322068310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a07720 0x559d89aaeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cf9b6f640 1 --2- 172.21.15.60:0/1322068310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a07720 0x559d89aaeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cf9b6f640 1 -- 172.21.15.60:0/1322068310 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d897e69b0 con 0x559d89a036d0 2024-04-25T10:10:08.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cf9b6f640 1 --2- 172.21.15.60:0/1322068310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a036d0 0x559d89aae560 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0022440 tx=0x7f1cf0000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bd9450e9b6e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:08.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:10:08.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cd4ff9640 1 -- 172.21.15.60:0/1322068310 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf003b020 con 0x559d89a036d0 2024-04-25T10:10:08.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cd4ff9640 1 -- 172.21.15.60:0/1322068310 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1cf0014780 con 0x559d89a036d0 2024-04-25T10:10:08.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 -- 172.21.15.60:0/1322068310 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ce40f7fa0 con 0x559d89a036d0 2024-04-25T10:10:08.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cd4ff9640 1 -- 172.21.15.60:0/1322068310 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cf0022780 con 0x559d89a036d0 2024-04-25T10:10:08.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 10 client.7185 fetch_fsmap learned FSMap version 45 2024-04-25T10:10:08.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 -- 172.21.15.60:0/1322068310 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x559d897f90b0 con 0x559d89a036d0 2024-04-25T10:10:08.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cd4ff9640 1 -- 172.21.15.60:0/1322068310 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce40f7fa0 con 0x559d89a036d0 2024-04-25T10:10:08.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cd4ff9640 1 -- 172.21.15.60:0/1322068310 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 45) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1cf0036050 con 0x559d89a036d0 2024-04-25T10:10:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 10 client.7185 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T10:10:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 10 client.7185 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:10:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 20 client.7185 populate_metadata read hostname 'smithi060' 2024-04-25T10:10:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 --2- 172.21.15.60:0/1322068310 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x559d89a516a0 0x559d89aaf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 4 client.7185 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:10:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 -- 172.21.15.60:0/1322068310 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559d8985f2e0 con 0x559d89a516a0 2024-04-25T10:10:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cf936e640 1 --2- 172.21.15.60:0/1322068310 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x559d89a516a0 0x559d89aaf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cf936e640 1 --2- 172.21.15.60:0/1322068310 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x559d89a516a0 0x559d89aaf520 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fe78b6695461e48 server_cookie=3ed1d05b0645d0b0 in_seq=0 out_seq=0 2024-04-25T10:10:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cd4ff9640 10 client.7185 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cd4ff9640 1 -- 172.21.15.60:0/1322068310 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x559d8985f2e0 con 0x559d89a516a0 2024-04-25T10:10:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cd4ff9640 10 client.7185 handle_command_reply: tid=0 2024-04-25T10:10:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 client.7185 shutdown 2024-04-25T10:10:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 -- 172.21.15.60:0/1322068310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a036d0 msgr2=0x559d89aae560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 --2- 172.21.15.60:0/1322068310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a036d0 0x559d89aae560 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0022440 tx=0x7f1cf0000ee0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 -- 172.21.15.60:0/1322068310 shutdown_connections 2024-04-25T10:10:08.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 --2- 172.21.15.60:0/1322068310 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x559d89a516a0 0x559d89aaf520 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 --2- 172.21.15.60:0/1322068310 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559d89a50d80 0x559d89aaefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cd4ff9640 0 client.7185 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:08.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 --2- 172.21.15.60:0/1322068310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d89a07720 0x559d89aaeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 --2- 172.21.15.60:0/1322068310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559d89a036d0 0x559d89aae560 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 -- 172.21.15.60:0/1322068310 >> 172.21.15.60:0/1322068310 conn(0x559d89a06b80 msgr2=0x559d89a62390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:08.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 -- 172.21.15.60:0/1322068310 shutdown_connections 2024-04-25T10:10:08.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 1 -- 172.21.15.60:0/1322068310 wait complete. 2024-04-25T10:10:08.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.933+0000 7f1cfef40000 20 client.7185 trim_cache size 0 max 16384 2024-04-25T10:10:08.957 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:10:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "0ea93863-a079-45c4-abed-6e83540ea9fe", 2024-04-25T10:10:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T10:10:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7134, 2024-04-25T10:10:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T10:10:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T10:10:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T10:10:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 4.4739470399999997, 2024-04-25T10:10:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 45, 2024-04-25T10:10:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 71, 2024-04-25T10:10:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 71, 2024-04-25T10:10:08.959 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 5.4389782350000004 2024-04-25T10:10:08.959 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:10:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.937+0000 7f1cfbdd3640 1 -- 172.21.15.60:0/3878279087 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cd8066f60 msgr2=0x7f1cd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.937+0000 7f1cfbdd3640 1 --2- 172.21.15.60:0/3878279087 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cd8066f60 0x7f1cd8069420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00011f0 tx=0x7f1cf0009040 comp rx=0 tx=0).stop 2024-04-25T10:10:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.937+0000 7f1cfbdd3640 1 -- 172.21.15.60:0/3878279087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cf4071b20 msgr2=0x7f1cf4157b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.937+0000 7f1cfbdd3640 1 --2- 172.21.15.60:0/3878279087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cf4071b20 0x7f1cf4157b90 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400a960 tx=0x7f1ce401bf40 comp rx=0 tx=0).stop 2024-04-25T10:10:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.937+0000 7f1cfbdd3640 1 -- 172.21.15.60:0/3878279087 shutdown_connections 2024-04-25T10:10:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.937+0000 7f1cfbdd3640 1 --2- 172.21.15.60:0/3878279087 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cd8066f60 0x7f1cd8069420 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.937+0000 7f1cfbdd3640 1 --2- 172.21.15.60:0/3878279087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cf40cce50 0x7f1cf413d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.937+0000 7f1cfbdd3640 1 --2- 172.21.15.60:0/3878279087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cf4071b20 0x7f1cf4157b90 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.937+0000 7f1cfbdd3640 1 --2- 172.21.15.60:0/3878279087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cf406e480 0x7f1cf4129880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.937+0000 7f1cfbdd3640 1 -- 172.21.15.60:0/3878279087 >> 172.21.15.60:0/3878279087 conn(0x7f1cf40b12d0 msgr2=0x7f1cf40b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:08.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.937+0000 7f1cfbdd3640 1 -- 172.21.15.60:0/3878279087 shutdown_connections 2024-04-25T10:10:08.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:08.937+0000 7f1cfbdd3640 1 -- 172.21.15.60:0/3878279087 wait complete. 2024-04-25T10:10:08.961 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:10:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.069+0000 7f2127eb6640 1 Processor -- start 2024-04-25T10:10:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.069+0000 7f2127eb6640 1 -- start start 2024-04-25T10:10:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.069+0000 7f2127eb6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f212015b720 0x7f21200b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.069+0000 7f2127eb6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21200acf30 0x7f21200ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.069+0000 7f2127eb6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21200b5000 con 0x7f21200af060 2024-04-25T10:10:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.069+0000 7f2127eb6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21200ad8e0 con 0x7f21200acf30 2024-04-25T10:10:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.069+0000 7f2127eb6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21200b4440 con 0x7f212015b720 2024-04-25T10:10:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.069+0000 7f2126453640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21200acf30 0x7f21200ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.069+0000 7f2126453640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21200acf30 0x7f21200ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47974/0 (socket says 172.21.15.60:47974) 2024-04-25T10:10:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.069+0000 7f2126453640 1 -- 172.21.15.60:0/2568912025 learned_addr learned my addr 172.21.15.60:0/2568912025 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.069+0000 7f2125c52640 1 --2- 172.21.15.60:0/2568912025 >> v2:172.21.15.100:3300/0 conn(0x7f212015b720 0x7f21200b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.069+0000 7f2126453640 1 -- 172.21.15.60:0/2568912025 >> v2:172.21.15.100:3300/0 conn(0x7f212015b720 msgr2=0x7f21200b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.069+0000 7f2126453640 1 --2- 172.21.15.60:0/2568912025 >> v2:172.21.15.100:3300/0 conn(0x7f212015b720 0x7f21200b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.069+0000 7f2126453640 1 -- 172.21.15.60:0/2568912025 >> v1:172.21.15.100:6789/0 conn(0x7f21200af060 legacy=0x7f212015bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.069+0000 7f2126453640 1 -- 172.21.15.60:0/2568912025 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21200af5d0 con 0x7f21200acf30 2024-04-25T10:10:09.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.069+0000 7f2125c52640 1 --2- 172.21.15.60:0/2568912025 >> v2:172.21.15.100:3300/0 conn(0x7f212015b720 0x7f21200b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:09.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2126453640 1 --2- 172.21.15.60:0/2568912025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21200acf30 0x7f21200ad3a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f21200b30e0 tx=0x7f211801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc5e389dea5d145 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2124c50640 1 -- 172.21.15.60:0/2568912025 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21180a56c0 con 0x7f21200acf30 2024-04-25T10:10:09.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2124c50640 1 -- 172.21.15.60:0/2568912025 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21180a5860 con 0x7f21200acf30 2024-04-25T10:10:09.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2124c50640 1 -- 172.21.15.60:0/2568912025 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21180a5c30 con 0x7f21200acf30 2024-04-25T10:10:09.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2127eb6640 1 -- 172.21.15.60:0/2568912025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21200acf30 msgr2=0x7f21200ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2127eb6640 1 --2- 172.21.15.60:0/2568912025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21200acf30 0x7f21200ad3a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f21200b30e0 tx=0x7f211801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2127eb6640 1 -- 172.21.15.60:0/2568912025 shutdown_connections 2024-04-25T10:10:09.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2127eb6640 1 --2- 172.21.15.60:0/2568912025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21200acf30 0x7f21200ad3a0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2127eb6640 1 --2- 172.21.15.60:0/2568912025 >> v2:172.21.15.100:3300/0 conn(0x7f212015b720 0x7f21200b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2127eb6640 1 -- 172.21.15.60:0/2568912025 >> 172.21.15.60:0/2568912025 conn(0x7f21200cfe00 msgr2=0x7f21200d0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2127eb6640 1 -- 172.21.15.60:0/2568912025 shutdown_connections 2024-04-25T10:10:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2127eb6640 1 -- 172.21.15.60:0/2568912025 wait complete. 2024-04-25T10:10:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2127eb6640 1 Processor -- start 2024-04-25T10:10:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2127eb6640 1 -- start start 2024-04-25T10:10:09.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2127eb6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21200acf30 0x7f2120141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2127eb6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21200af060 0x7f212009d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2127eb6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f212015b720 0x7f212013ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2127eb6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212006c490 con 0x7f21200af060 2024-04-25T10:10:09.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2127eb6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212006c310 con 0x7f212015b720 2024-04-25T10:10:09.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2127eb6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2120168440 con 0x7f21200acf30 2024-04-25T10:10:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2125451640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21200af060 0x7f212009d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2125451640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21200af060 0x7f212009d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47986/0 (socket says 172.21.15.60:47986) 2024-04-25T10:10:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2125451640 1 -- 172.21.15.60:0/1637125028 learned_addr learned my addr 172.21.15.60:0/1637125028 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2125c52640 1 --2- 172.21.15.60:0/1637125028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21200acf30 0x7f2120141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2126453640 1 --2- 172.21.15.60:0/1637125028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f212015b720 0x7f212013ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2125451640 1 -- 172.21.15.60:0/1637125028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21200acf30 msgr2=0x7f2120141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2125451640 1 --2- 172.21.15.60:0/1637125028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21200acf30 0x7f2120141dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2125451640 1 -- 172.21.15.60:0/1637125028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f212015b720 msgr2=0x7f212013ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2125451640 1 --2- 172.21.15.60:0/1637125028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f212015b720 0x7f212013ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.073+0000 7f2125451640 1 -- 172.21.15.60:0/1637125028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212013ef50 con 0x7f21200af060 2024-04-25T10:10:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.077+0000 7f2125451640 1 --2- 172.21.15.60:0/1637125028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21200af060 0x7f212009d670 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f211000c500 tx=0x7f211000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.077+0000 7f21167fc640 1 -- 172.21.15.60:0/1637125028 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110016020 con 0x7f21200af060 2024-04-25T10:10:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.077+0000 7f21167fc640 1 -- 172.21.15.60:0/1637125028 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2110005220 con 0x7f21200af060 2024-04-25T10:10:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.077+0000 7f21167fc640 1 -- 172.21.15.60:0/1637125028 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110007030 con 0x7f21200af060 2024-04-25T10:10:09.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.077+0000 7f2127eb6640 1 -- 172.21.15.60:0/1637125028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212009de10 con 0x7f21200af060 2024-04-25T10:10:09.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.077+0000 7f2127eb6640 1 -- 172.21.15.60:0/1637125028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2120153640 con 0x7f21200af060 2024-04-25T10:10:09.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.077+0000 7f2127eb6640 1 -- 172.21.15.60:0/1637125028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2120074770 con 0x7f21200af060 2024-04-25T10:10:09.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.077+0000 7f21167fc640 1 -- 172.21.15.60:0/1637125028 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21100071f0 con 0x7f21200af060 2024-04-25T10:10:09.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.077+0000 7f21167fc640 1 --2- 172.21.15.60:0/1637125028 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2104066f60 0x7f2104069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.077+0000 7f21167fc640 1 -- 172.21.15.60:0/1637125028 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2110084a80 con 0x7f21200af060 2024-04-25T10:10:09.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.077+0000 7f2125c52640 1 --2- 172.21.15.60:0/1637125028 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2104066f60 0x7f2104069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.077+0000 7f2125c52640 1 --2- 172.21.15.60:0/1637125028 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2104066f60 0x7f2104069420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f212015b120 tx=0x7f211c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.081+0000 7f21167fc640 1 -- 172.21.15.60:0/1637125028 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2110020020 con 0x7f21200af060 2024-04-25T10:10:09.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.277+0000 7f2127eb6640 1 -- 172.21.15.60:0/1637125028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f21200d0520 con 0x7f21200af060 2024-04-25T10:10:09.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.277+0000 7f21167fc640 1 -- 172.21.15.60:0/1637125028 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) v1 ==== 53+0+73 (secure 0 0 0) 0x7f21200b5000 con 0x7f21200af060 2024-04-25T10:10:09.288 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:10:09.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.281+0000 7f2127eb6640 1 -- 172.21.15.60:0/1637125028 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2104066f60 msgr2=0x7f2104069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.281+0000 7f2127eb6640 1 --2- 172.21.15.60:0/1637125028 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2104066f60 0x7f2104069420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f212015b120 tx=0x7f211c008040 comp rx=0 tx=0).stop 2024-04-25T10:10:09.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.281+0000 7f2127eb6640 1 -- 172.21.15.60:0/1637125028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21200af060 msgr2=0x7f212009d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.281+0000 7f2127eb6640 1 --2- 172.21.15.60:0/1637125028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21200af060 0x7f212009d670 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f211000c500 tx=0x7f211000c9c0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.281+0000 7f2127eb6640 1 -- 172.21.15.60:0/1637125028 shutdown_connections 2024-04-25T10:10:09.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.281+0000 7f2127eb6640 1 --2- 172.21.15.60:0/1637125028 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2104066f60 0x7f2104069420 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.281+0000 7f2127eb6640 1 --2- 172.21.15.60:0/1637125028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f212015b720 0x7f212013ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.281+0000 7f2127eb6640 1 --2- 172.21.15.60:0/1637125028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21200af060 0x7f212009d670 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.281+0000 7f2127eb6640 1 --2- 172.21.15.60:0/1637125028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21200acf30 0x7f2120141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.281+0000 7f2127eb6640 1 -- 172.21.15.60:0/1637125028 >> 172.21.15.60:0/1637125028 conn(0x7f21200cfe00 msgr2=0x7f21200cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.281+0000 7f2127eb6640 1 -- 172.21.15.60:0/1637125028 shutdown_connections 2024-04-25T10:10:09.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:09.281+0000 7f2127eb6640 1 -- 172.21.15.60:0/1637125028 wait complete. 2024-04-25T10:10:09.307 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:10:09.307 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:10:09.308 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:10:09.308 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:10:09.308 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:09.308 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:10:09.308 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:10:09.308 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:10:09.308 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:09.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:09.308 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:10:09.314 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:10:09.441 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:10:09.441 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:10:09.441 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:09.441 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:10:09.441 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:09.441 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:10:09.442 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:09.442 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:10:09.442 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:10:09.442 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:09.442 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:10:09.442 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:10:09.442 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:09.442 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:10:09.442 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:10:09.442 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42248sec preferred_lft 42248sec 2024-04-25T10:10:09.442 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:10:09.442 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:09.442 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:10:09.443 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:09.443 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:10:09.443 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:10:09.443 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:09.443 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:10:09.443 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:09.443 INFO:teuthology.orchestra.run.smithi060.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:10:09.443 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:09.443 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:10:09.443 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:09.443 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:10:09.443 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:09.443 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:10:09.444 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:09.444 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:10:09.444 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:09.444 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:10:09.444 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:09.444 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:10:09.444 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:09.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:09.444 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:10:09.458 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:10:09.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:09.460 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:10:09.471 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-04-25T10:10:09.461+0000 7fd9f6d8d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:10:09.477 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:10:09.477 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:10:09.477 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:10:09.478 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:10:09.478 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:10:09.478 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:10:09.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:09.478 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:10:09.479 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:10:09.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:10:09.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:10:09.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:10:09.479 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:10:09.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:09.556 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:10:09.605 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:10:09.605 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:10:09.606 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:09.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:09.606 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T10:10:09.662 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:10:09.662 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:10:09.662 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:09.709 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:09.709 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:10:09.709 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:09.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:10:09.814 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:10:09.867 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:10:09.867 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:10:09.867 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:10:09.867 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:10:09.867 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:10:09.867 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:10:09.867 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:10:09.867 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:10:09.867 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:10:09.868 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:10:09.868 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:10:09.868 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:10:09.868 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:10:09.868 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:10:09.868 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:10:09.868 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:10:09.868 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:10:09.868 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:10:09.868 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:10:09.868 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:10:09.868 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:10:09.868 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:10:09.868 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:10:09.869 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:10:09.869 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:10:09.869 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:10:09.869 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:10:09.869 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:10:09.869 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:10:09.869 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:09.869 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:09.869 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:09.869 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:09.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:09.870 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:10:09.916 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:10:09.917 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:10:09.961 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:10:09.976 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:10:10.025 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:10:10.025 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:10:10.025 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:10:10.025 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:10:10.025 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:10:10.025 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:10:10.025 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:10:10.025 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:10:10.025 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:10:10.025 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:10:10.025 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:10:10.025 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:10:10.026 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:10:10.026 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:10:10.026 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:10:10.026 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:10:10.026 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:10:10.026 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:10:10.026 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:10:10.026 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:10:10.026 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:10:10.026 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:10:10.026 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:10:10.026 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:10:10.026 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:10:10.027 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:10:10.027 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:10:10.027 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:10:10.027 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:10:10.027 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:10.027 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:10.027 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:10.027 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:10.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:10.029 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:10:10.042 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2024-04-25T10:10:10.033+0000 7fe589028580 -1 init, newargv = 0x55c1a78b7470 newargc=15 2024-04-25T10:10:10.042 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[43052]: starting ceph client 2024-04-25T10:10:10.047 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[43052]: starting fuse 2024-04-25T10:10:11.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:10:11.165 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:10:11.215 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:10:11.215 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:10:11.215 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:10:11.215 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:10:11.215 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:10:11.215 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:10:11.215 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:10:11.215 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:10:11.215 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:10:11.216 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:10:11.216 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:10:11.216 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:10:11.216 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:10:11.216 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:10:11.216 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:10:11.216 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:10:11.216 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:10:11.216 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:10:11.216 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:10:11.216 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:10:11.216 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:10:11.216 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:10:11.216 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:10:11.216 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:10:11.217 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:10:11.217 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:10:11.217 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:10:11.217 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:10:11.217 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:10:11.217 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:11.217 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:11.217 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:11.217 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:11.217 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:11.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:11.218 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:10:11.265 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:10:11.265 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:10:11.265 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:10:11.265 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:10:11.265 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:10:11.265 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:10:11.266 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:10:11.266 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:10:11.266 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:10:11.266 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:10:11.266 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:10:11.266 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:10:11.266 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:10:11.266 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:10:11.266 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:10:11.266 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:10:11.266 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:10:11.266 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:10:11.266 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:10:11.266 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:10:11.266 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:10:11.267 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:10:11.267 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:10:11.267 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:10:11.267 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:10:11.267 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:10:11.267 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:10:11.267 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2024-04-25T10:10:11.267 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:10:11.394 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.43052.asok 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:10:11.404 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43052.asok 2024-04-25T10:10:11.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:11.405 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43052.asok status 2024-04-25T10:10:11.570 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:10:11.570 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:10:11.570 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:10:11.570 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:10:11.570 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2024-04-25T10:10:11.570 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:10:11.570 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:10:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "43052", 2024-04-25T10:10:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:10:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:10:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:10:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:10:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7197, 2024-04-25T10:10:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:10:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:10:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:10:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 7197 2024-04-25T10:10:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:10:11.571 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:10:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:10:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:10:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3402832230 2024-04-25T10:10:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:10:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:10:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:10:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:10:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:10:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3402832230 2024-04-25T10:10:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:10:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.7197 192.168.0.1:0/3402832230", 2024-04-25T10:10:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/3402832230", 2024-04-25T10:10:11.572 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:10:11.573 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 45, 2024-04-25T10:10:11.573 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 71, 2024-04-25T10:10:11.573 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:10:11.573 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:10:11.573 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:10:11.573 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:10:11.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:11.585 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:10:11.585 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:11.591 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:10:11.592 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:11.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:11.592 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:11.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:11.647 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-25T10:10:11.700 INFO:teuthology.orchestra.run.smithi060.stdout:file_undamaged 2024-04-25T10:10:11.701 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:10:11.702 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:10:11.702 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:10:11.702 DEBUG:teuthology.orchestra.run.smithi060:> import json 2024-04-25T10:10:11.702 DEBUG:teuthology.orchestra.run.smithi060:> import sys 2024-04-25T10:10:11.702 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:10:11.702 DEBUG:teuthology.orchestra.run.smithi060:> try: 2024-04-25T10:10:11.702 DEBUG:teuthology.orchestra.run.smithi060:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-25T10:10:11.702 DEBUG:teuthology.orchestra.run.smithi060:> except OSError as e: 2024-04-25T10:10:11.702 DEBUG:teuthology.orchestra.run.smithi060:> sys.exit(e.errno) 2024-04-25T10:10:11.702 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:10:11.702 DEBUG:teuthology.orchestra.run.smithi060:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:10:11.702 DEBUG:teuthology.orchestra.run.smithi060:> print(json.dumps( 2024-04-25T10:10:11.702 DEBUG:teuthology.orchestra.run.smithi060:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:10:11.703 DEBUG:teuthology.orchestra.run.smithi060:> indent=2)) 2024-04-25T10:10:11.703 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:10:11.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:11 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T10:10:11.823 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T10:10:11.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:11.824 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T10:10:11.886 INFO:teuthology.orchestra.run.smithi060.stdout:2 2024-04-25T10:10:11.887 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:10:11.887 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:11.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:12.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.101+0000 7fc75ba11640 1 Processor -- start 2024-04-25T10:10:12.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.101+0000 7fc75ba11640 1 -- start start 2024-04-25T10:10:12.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.101+0000 7fc75ba11640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7540b3610 0x7fc7540aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.101+0000 7fc75ba11640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7540b54d0 0x7fc7540af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.101+0000 7fc75ba11640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7540b3e60 con 0x7fc75406ac10 2024-04-25T10:10:12.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.101+0000 7fc75ba11640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7540b3b80 con 0x7fc7540b3610 2024-04-25T10:10:12.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.101+0000 7fc75ba11640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540ae130 con 0x7fc7540b54d0 2024-04-25T10:10:12.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.101+0000 7fc758fac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7540b3610 0x7fc7540aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.101+0000 7fc758fac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7540b3610 0x7fc7540aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47988/0 (socket says 172.21.15.60:47988) 2024-04-25T10:10:12.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.101+0000 7fc758fac640 1 -- 172.21.15.60:0/3214913901 learned_addr learned my addr 172.21.15.60:0/3214913901 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:12.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.101+0000 7fc759fae640 1 --2- 172.21.15.60:0/3214913901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7540b54d0 0x7fc7540af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.101+0000 7fc758fac640 1 -- 172.21.15.60:0/3214913901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7540b54d0 msgr2=0x7fc7540af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:12.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.101+0000 7fc758fac640 1 --2- 172.21.15.60:0/3214913901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7540b54d0 0x7fc7540af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.101+0000 7fc758fac640 1 -- 172.21.15.60:0/3214913901 >> v1:172.21.15.100:6789/0 conn(0x7fc75406ac10 legacy=0x7fc75406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:12.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.101+0000 7fc758fac640 1 -- 172.21.15.60:0/3214913901 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7540ae410 con 0x7fc7540b3610 2024-04-25T10:10:12.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.105+0000 7fc758fac640 1 --2- 172.21.15.60:0/3214913901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7540b3610 0x7fc7540aec10 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009870 tx=0x7fc74c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c28aac77327dfe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.105+0000 7fc74bfff640 1 -- 172.21.15.60:0/3214913901 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0a7070 con 0x7fc7540b3610 2024-04-25T10:10:12.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.105+0000 7fc74bfff640 1 -- 172.21.15.60:0/3214913901 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c0b0070 con 0x7fc7540b3610 2024-04-25T10:10:12.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.105+0000 7fc74bfff640 1 -- 172.21.15.60:0/3214913901 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0ab890 con 0x7fc7540b3610 2024-04-25T10:10:12.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.105+0000 7fc75ba11640 1 -- 172.21.15.60:0/3214913901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7540b3610 msgr2=0x7fc7540aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.105+0000 7fc75ba11640 1 --2- 172.21.15.60:0/3214913901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7540b3610 0x7fc7540aec10 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009870 tx=0x7fc74c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.105+0000 7fc75ba11640 1 -- 172.21.15.60:0/3214913901 shutdown_connections 2024-04-25T10:10:12.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.105+0000 7fc75ba11640 1 --2- 172.21.15.60:0/3214913901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7540b3610 0x7fc7540aec10 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.105+0000 7fc75ba11640 1 --2- 172.21.15.60:0/3214913901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7540b54d0 0x7fc7540af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.105+0000 7fc75ba11640 1 -- 172.21.15.60:0/3214913901 >> 172.21.15.60:0/3214913901 conn(0x7fc754072920 msgr2=0x7fc754099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:12.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.105+0000 7fc75ba11640 1 -- 172.21.15.60:0/3214913901 shutdown_connections 2024-04-25T10:10:12.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.105+0000 7fc75ba11640 1 -- 172.21.15.60:0/3214913901 wait complete. 2024-04-25T10:10:12.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.105+0000 7fc75ba11640 1 Processor -- start 2024-04-25T10:10:12.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.105+0000 7fc75ba11640 1 -- start start 2024-04-25T10:10:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc75ba11640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc75406ac10 0x7fc75406f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc75ba11640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7540b3610 0x7fc7541fa110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc75ba11640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7540b54d0 0x7fc7541fa650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc75ba11640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540b00a0 con 0x7fc7540b3610 2024-04-25T10:10:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc75ba11640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540aff20 con 0x7fc7540b54d0 2024-04-25T10:10:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc75ba11640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540b58e0 con 0x7fc75406ac10 2024-04-25T10:10:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc758fac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7540b3610 0x7fc7541fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc758fac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7540b3610 0x7fc7541fa110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47992/0 (socket says 172.21.15.60:47992) 2024-04-25T10:10:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc758fac640 1 -- 172.21.15.60:0/4090335729 learned_addr learned my addr 172.21.15.60:0/4090335729 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc759fae640 1 --2- 172.21.15.60:0/4090335729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7540b54d0 0x7fc7541fa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc7597ad640 1 --2- 172.21.15.60:0/4090335729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc75406ac10 0x7fc75406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc758fac640 1 -- 172.21.15.60:0/4090335729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc75406ac10 msgr2=0x7fc75406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc758fac640 1 --2- 172.21.15.60:0/4090335729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc75406ac10 0x7fc75406f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc758fac640 1 -- 172.21.15.60:0/4090335729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7540b54d0 msgr2=0x7fc7541fa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc758fac640 1 --2- 172.21.15.60:0/4090335729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7540b54d0 0x7fc7541fa650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc758fac640 1 -- 172.21.15.60:0/4090335729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75415c620 con 0x7fc7540b3610 2024-04-25T10:10:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc759fae640 1 --2- 172.21.15.60:0/4090335729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7540b54d0 0x7fc7541fa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc758fac640 1 --2- 172.21.15.60:0/4090335729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7540b3610 0x7fc7541fa110 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fc74c01c320 tx=0x7fc74c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc749ffb640 1 -- 172.21.15.60:0/4090335729 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0b4800 con 0x7fc7540b3610 2024-04-25T10:10:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc749ffb640 1 -- 172.21.15.60:0/4090335729 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c0b49a0 con 0x7fc7540b3610 2024-04-25T10:10:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc749ffb640 1 -- 172.21.15.60:0/4090335729 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0b4d90 con 0x7fc7540b3610 2024-04-25T10:10:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc75ba11640 1 -- 172.21.15.60:0/4090335729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7540cb3f0 con 0x7fc7540b3610 2024-04-25T10:10:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc75ba11640 1 -- 172.21.15.60:0/4090335729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75406fcd0 con 0x7fc7540b3610 2024-04-25T10:10:12.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc75ba11640 1 -- 172.21.15.60:0/4090335729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7540cc7c0 con 0x7fc7540b3610 2024-04-25T10:10:12.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc749ffb640 1 -- 172.21.15.60:0/4090335729 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc74c0bd3e0 con 0x7fc7540b3610 2024-04-25T10:10:12.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.109+0000 7fc749ffb640 1 --2- 172.21.15.60:0/4090335729 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc740066f60 0x7fc740069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.113+0000 7fc749ffb640 1 -- 172.21.15.60:0/4090335729 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc74c11f2d0 con 0x7fc7540b3610 2024-04-25T10:10:12.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.113+0000 7fc7597ad640 1 --2- 172.21.15.60:0/4090335729 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc740066f60 0x7fc740069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.113+0000 7fc7597ad640 1 --2- 172.21.15.60:0/4090335729 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc740066f60 0x7fc740069420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc744004450 tx=0x7fc744009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.117+0000 7fc749ffb640 1 -- 172.21.15.60:0/4090335729 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc74c016110 con 0x7fc7540b3610 2024-04-25T10:10:12.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.281+0000 7fc75ba11640 1 -- 172.21.15.60:0/4090335729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc754074770 con 0x7fc7540b3610 2024-04-25T10:10:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.285+0000 7fc749ffb640 1 -- 172.21.15.60:0/4090335729 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16907 (secure 0 0 0) 0x7fc74c0efc00 con 0x7fc7540b3610 2024-04-25T10:10:12.294 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:10:12.294 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:10:03.449677+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:42.005281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:42.710028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1e","mappings":[{"from":6,"to":0}]},{"pgid":"11.28","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/1289503429":"2024-04-26T10:10:03.449620+0000","172.21.15.60:6833/1289503429":"2024-04-26T10:10:03.449620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:12.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.285+0000 7fc75ba11640 1 -- 172.21.15.60:0/4090335729 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc740066f60 msgr2=0x7fc740069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.285+0000 7fc75ba11640 1 --2- 172.21.15.60:0/4090335729 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc740066f60 0x7fc740069420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc744004450 tx=0x7fc744009040 comp rx=0 tx=0).stop 2024-04-25T10:10:12.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.285+0000 7fc75ba11640 1 -- 172.21.15.60:0/4090335729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7540b3610 msgr2=0x7fc7541fa110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.285+0000 7fc75ba11640 1 --2- 172.21.15.60:0/4090335729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7540b3610 0x7fc7541fa110 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fc74c01c320 tx=0x7fc74c01ad30 comp rx=0 tx=0).stop 2024-04-25T10:10:12.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.285+0000 7fc75ba11640 1 -- 172.21.15.60:0/4090335729 shutdown_connections 2024-04-25T10:10:12.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.285+0000 7fc75ba11640 1 --2- 172.21.15.60:0/4090335729 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc740066f60 0x7fc740069420 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.285+0000 7fc75ba11640 1 --2- 172.21.15.60:0/4090335729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7540b54d0 0x7fc7541fa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.285+0000 7fc75ba11640 1 --2- 172.21.15.60:0/4090335729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7540b3610 0x7fc7541fa110 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.285+0000 7fc75ba11640 1 --2- 172.21.15.60:0/4090335729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc75406ac10 0x7fc75406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.285+0000 7fc75ba11640 1 -- 172.21.15.60:0/4090335729 >> 172.21.15.60:0/4090335729 conn(0x7fc754072920 msgr2=0x7fc75406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:12.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.289+0000 7fc75ba11640 1 -- 172.21.15.60:0/4090335729 shutdown_connections 2024-04-25T10:10:12.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.289+0000 7fc75ba11640 1 -- 172.21.15.60:0/4090335729 wait complete. 2024-04-25T10:10:12.313 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:09:42.005281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:09:42.710028+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:12.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:12.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc349a74640 1 Processor -- start 2024-04-25T10:10:12.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc349a74640 1 -- start start 2024-04-25T10:10:12.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc349a74640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc3440cbcf0 0x7fc344160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc349a74640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc344071c30 0x7fc34415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc349a74640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc344144a40 con 0x7fc34415fa20 2024-04-25T10:10:12.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc349a74640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3440b2dc0 con 0x7fc3440cbcf0 2024-04-25T10:10:12.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc349a74640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34415fe80 con 0x7fc344071c30 2024-04-25T10:10:12.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc3437fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc3440cbcf0 0x7fc344160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc342ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc344071c30 0x7fc34415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc342ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc344071c30 0x7fc34415f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:43628/0 (socket says 172.21.15.60:43628) 2024-04-25T10:10:12.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc343fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc34415fa20 0x7fc34415bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:41482/0 (socket says 172.21.15.60:41482) 2024-04-25T10:10:12.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc342ffd640 1 -- 172.21.15.60:0/4189423211 learned_addr learned my addr 172.21.15.60:0/4189423211 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:12.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc342ffd640 1 -- 172.21.15.60:0/4189423211 >> v2:172.21.15.100:3300/0 conn(0x7fc3440cbcf0 msgr2=0x7fc344160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:12.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc342ffd640 1 --2- 172.21.15.60:0/4189423211 >> v2:172.21.15.100:3300/0 conn(0x7fc3440cbcf0 0x7fc344160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc342ffd640 1 -- 172.21.15.60:0/4189423211 >> v1:172.21.15.100:6789/0 conn(0x7fc34415fa20 legacy=0x7fc34415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc342ffd640 1 -- 172.21.15.60:0/4189423211 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3440cc9f0 con 0x7fc344071c30 2024-04-25T10:10:12.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.429+0000 7fc3437fe640 1 --2- 172.21.15.60:0/4189423211 >> v2:172.21.15.100:3300/0 conn(0x7fc3440cbcf0 0x7fc344160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:10:12.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.433+0000 7fc342ffd640 1 --2- 172.21.15.60:0/4189423211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc344071c30 0x7fc34415f4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc334009870 tx=0x7fc33401c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a69a5a4124f68a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.433+0000 7fc3427fc640 1 -- 172.21.15.60:0/4189423211 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3340a93f0 con 0x7fc344071c30 2024-04-25T10:10:12.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.433+0000 7fc3427fc640 1 -- 172.21.15.60:0/4189423211 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3340a9590 con 0x7fc344071c30 2024-04-25T10:10:12.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.433+0000 7fc3427fc640 1 -- 172.21.15.60:0/4189423211 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3340a9960 con 0x7fc344071c30 2024-04-25T10:10:12.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.433+0000 7fc349a74640 1 -- 172.21.15.60:0/4189423211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc344071c30 msgr2=0x7fc34415f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.433+0000 7fc349a74640 1 --2- 172.21.15.60:0/4189423211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc344071c30 0x7fc34415f4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc334009870 tx=0x7fc33401c530 comp rx=0 tx=0).stop 2024-04-25T10:10:12.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.433+0000 7fc349a74640 1 -- 172.21.15.60:0/4189423211 shutdown_connections 2024-04-25T10:10:12.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.433+0000 7fc349a74640 1 --2- 172.21.15.60:0/4189423211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc344071c30 0x7fc34415f4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.433+0000 7fc349a74640 1 --2- 172.21.15.60:0/4189423211 >> v2:172.21.15.100:3300/0 conn(0x7fc3440cbcf0 0x7fc344160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.433+0000 7fc349a74640 1 -- 172.21.15.60:0/4189423211 >> 172.21.15.60:0/4189423211 conn(0x7fc3440670b0 msgr2=0x7fc34406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:12.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.433+0000 7fc349a74640 1 -- 172.21.15.60:0/4189423211 shutdown_connections 2024-04-25T10:10:12.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.433+0000 7fc349a74640 1 -- 172.21.15.60:0/4189423211 wait complete. 2024-04-25T10:10:12.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc349a74640 1 Processor -- start 2024-04-25T10:10:12.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc349a74640 1 -- start start 2024-04-25T10:10:12.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc349a74640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc344071c30 0x7fc3441f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc349a74640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3440cbcf0 0x7fc34413d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc349a74640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc34415fa20 0x7fc3440b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc349a74640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3440720b0 con 0x7fc3440cbcf0 2024-04-25T10:10:12.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc349a74640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3440cfed0 con 0x7fc344071c30 2024-04-25T10:10:12.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc349a74640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34413df90 con 0x7fc34415fa20 2024-04-25T10:10:12.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc342ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3440cbcf0 0x7fc34413d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc342ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3440cbcf0 0x7fc34413d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48000/0 (socket says 172.21.15.60:48000) 2024-04-25T10:10:12.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc342ffd640 1 -- 172.21.15.60:0/1915962815 learned_addr learned my addr 172.21.15.60:0/1915962815 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:12.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc343fff640 1 --2- 172.21.15.60:0/1915962815 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc34415fa20 0x7fc3440b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc3437fe640 1 --2- 172.21.15.60:0/1915962815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc344071c30 0x7fc3441f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc342ffd640 1 -- 172.21.15.60:0/1915962815 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc34415fa20 msgr2=0x7fc3440b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc342ffd640 1 --2- 172.21.15.60:0/1915962815 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc34415fa20 0x7fc3440b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc342ffd640 1 -- 172.21.15.60:0/1915962815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc344071c30 msgr2=0x7fc3441f2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc342ffd640 1 --2- 172.21.15.60:0/1915962815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc344071c30 0x7fc3441f2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc342ffd640 1 -- 172.21.15.60:0/1915962815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34413e270 con 0x7fc3440cbcf0 2024-04-25T10:10:12.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc3437fe640 1 --2- 172.21.15.60:0/1915962815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc344071c30 0x7fc3441f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:12.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc342ffd640 1 --2- 172.21.15.60:0/1915962815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3440cbcf0 0x7fc34413d1c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc3340ab040 tx=0x7fc33401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc33bfff640 1 -- 172.21.15.60:0/1915962815 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3340a93f0 con 0x7fc3440cbcf0 2024-04-25T10:10:12.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc33bfff640 1 -- 172.21.15.60:0/1915962815 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3340b1820 con 0x7fc3440cbcf0 2024-04-25T10:10:12.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc349a74640 1 -- 172.21.15.60:0/1915962815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34413e550 con 0x7fc3440cbcf0 2024-04-25T10:10:12.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc33bfff640 1 -- 172.21.15.60:0/1915962815 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3340b1c10 con 0x7fc3440cbcf0 2024-04-25T10:10:12.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc349a74640 1 -- 172.21.15.60:0/1915962815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34413eb10 con 0x7fc3440cbcf0 2024-04-25T10:10:12.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc349a74640 1 -- 172.21.15.60:0/1915962815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc34406bba0 con 0x7fc3440cbcf0 2024-04-25T10:10:12.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.437+0000 7fc33bfff640 1 -- 172.21.15.60:0/1915962815 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3340bc450 con 0x7fc3440cbcf0 2024-04-25T10:10:12.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.441+0000 7fc33bfff640 1 --2- 172.21.15.60:0/1915962815 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc324066f60 0x7fc324069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.441+0000 7fc33bfff640 1 -- 172.21.15.60:0/1915962815 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc33411fa90 con 0x7fc3440cbcf0 2024-04-25T10:10:12.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.441+0000 7fc3437fe640 1 --2- 172.21.15.60:0/1915962815 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc324066f60 0x7fc324069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.441+0000 7fc3437fe640 1 --2- 172.21.15.60:0/1915962815 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc324066f60 0x7fc324069420 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00dc00 tx=0x7fc32c00f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.445+0000 7fc33bfff640 1 -- 172.21.15.60:0/1915962815 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3340ad020 con 0x7fc3440cbcf0 2024-04-25T10:10:12.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.605+0000 7fc349a74640 1 -- 172.21.15.60:0/1915962815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc344074770 con 0x7fc3440cbcf0 2024-04-25T10:10:12.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.605+0000 7fc33bfff640 1 -- 172.21.15.60:0/1915962815 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc334016150 con 0x7fc3440cbcf0 2024-04-25T10:10:12.614 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:10:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.605+0000 7fc349a74640 1 -- 172.21.15.60:0/1915962815 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc324066f60 msgr2=0x7fc324069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.605+0000 7fc349a74640 1 --2- 172.21.15.60:0/1915962815 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc324066f60 0x7fc324069420 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00dc00 tx=0x7fc32c00f040 comp rx=0 tx=0).stop 2024-04-25T10:10:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.605+0000 7fc349a74640 1 -- 172.21.15.60:0/1915962815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3440cbcf0 msgr2=0x7fc34413d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.605+0000 7fc349a74640 1 --2- 172.21.15.60:0/1915962815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3440cbcf0 0x7fc34413d1c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc3340ab040 tx=0x7fc33401ad70 comp rx=0 tx=0).stop 2024-04-25T10:10:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.605+0000 7fc349a74640 1 -- 172.21.15.60:0/1915962815 shutdown_connections 2024-04-25T10:10:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.605+0000 7fc349a74640 1 --2- 172.21.15.60:0/1915962815 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc324066f60 0x7fc324069420 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.605+0000 7fc349a74640 1 --2- 172.21.15.60:0/1915962815 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc34415fa20 0x7fc3440b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.605+0000 7fc349a74640 1 --2- 172.21.15.60:0/1915962815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3440cbcf0 0x7fc34413d1c0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.605+0000 7fc349a74640 1 --2- 172.21.15.60:0/1915962815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc344071c30 0x7fc3441f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.605+0000 7fc349a74640 1 -- 172.21.15.60:0/1915962815 >> 172.21.15.60:0/1915962815 conn(0x7fc3440670b0 msgr2=0x7fc3440b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.605+0000 7fc349a74640 1 -- 172.21.15.60:0/1915962815 shutdown_connections 2024-04-25T10:10:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.605+0000 7fc349a74640 1 -- 172.21.15.60:0/1915962815 wait complete. 2024-04-25T10:10:12.632 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:10:12.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b21ff5640 1 Processor -- start 2024-04-25T10:10:12.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b21ff5640 1 -- start start 2024-04-25T10:10:12.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b21ff5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c0cefe0 0x7f3b1c15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b21ff5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1c15ce20 con 0x7f3b1c0cca60 2024-04-25T10:10:12.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b21ff5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1c15c580 con 0x7f3b1c0718a0 2024-04-25T10:10:12.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b21ff5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c071410 con 0x7f3b1c0cefe0 2024-04-25T10:10:12.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b13fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3b1c0cca60 0x7f3b1c0b30a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55362/0 (socket says 172.21.15.60:55362) 2024-04-25T10:10:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b13fff640 1 -- 172.21.15.60:0/344601334 learned_addr learned my addr 172.21.15.60:0/344601334 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:10:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1b7fe640 1 --2- 172.21.15.60:0/344601334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c0cefe0 0x7f3b1c15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1affd640 1 -- 172.21.15.60:0/344601334 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670730154 0 0) 0x7f3b1c15ce20 con 0x7f3b1c0cca60 2024-04-25T10:10:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1affd640 1 -- 172.21.15.60:0/344601334 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b08003530 con 0x7f3b1c0cca60 2024-04-25T10:10:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1affd640 1 -- 172.21.15.60:0/344601334 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4189308135 0 0) 0x7f3b08003530 con 0x7f3b1c0cca60 2024-04-25T10:10:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1affd640 1 -- 172.21.15.60:0/344601334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c0cefe0 msgr2=0x7f3b1c15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1affd640 1 --2- 172.21.15.60:0/344601334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c0cefe0 0x7f3b1c15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1affd640 1 -- 172.21.15.60:0/344601334 >> v1:172.21.15.100:6789/0 conn(0x7f3b1c0718a0 legacy=0x7f3b1c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1affd640 1 -- 172.21.15.60:0/344601334 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c15c860 con 0x7f3b1c0cca60 2024-04-25T10:10:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1affd640 1 -- 172.21.15.60:0/344601334 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f3b04001680 con 0x7f3b1c0cca60 2024-04-25T10:10:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1affd640 1 -- 172.21.15.60:0/344601334 >> v1:172.21.15.60:6789/0 conn(0x7f3b1c0cca60 legacy=0x7f3b1c0b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1affd640 1 --2- 172.21.15.60:0/344601334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c0cefe0 0x7f3b080039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1affd640 1 --2- 172.21.15.60:0/344601334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08003f70 0x7f3b08004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1affd640 1 --2- 172.21.15.60:0/344601334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b08004d50 0x7f3b080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1affd640 1 -- 172.21.15.60:0/344601334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c071410 con 0x7f3b08004d50 2024-04-25T10:10:12.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1affd640 1 -- 172.21.15.60:0/344601334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0800bb60 con 0x7f3b08003f70 2024-04-25T10:10:12.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1affd640 1 -- 172.21.15.60:0/344601334 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0800bd60 con 0x7f3b1c0cefe0 2024-04-25T10:10:12.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1bfff640 1 --2- 172.21.15.60:0/344601334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b08004d50 0x7f3b080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b13fff640 1 --2- 172.21.15.60:0/344601334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08003f70 0x7f3b08004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1b7fe640 1 --2- 172.21.15.60:0/344601334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c0cefe0 0x7f3b080039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1bfff640 1 -- 172.21.15.60:0/344601334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c0cefe0 msgr2=0x7f3b080039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:12.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1bfff640 1 --2- 172.21.15.60:0/344601334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c0cefe0 0x7f3b080039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1bfff640 1 -- 172.21.15.60:0/344601334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08003f70 msgr2=0x7f3b08004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:12.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1bfff640 1 --2- 172.21.15.60:0/344601334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08003f70 0x7f3b08004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.753+0000 7f3b1bfff640 1 -- 172.21.15.60:0/344601334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b08003530 con 0x7f3b08004d50 2024-04-25T10:10:12.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.757+0000 7f3b1bfff640 1 --2- 172.21.15.60:0/344601334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b08004d50 0x7f3b080051d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f3b14001e70 tx=0x7f3b1401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd171c86fc4b29c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.757+0000 7f3b1affd640 1 -- 172.21.15.60:0/344601334 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14001a90 con 0x7f3b08004d50 2024-04-25T10:10:12.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.757+0000 7f3b1affd640 1 -- 172.21.15.60:0/344601334 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b14003b80 con 0x7f3b08004d50 2024-04-25T10:10:12.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.757+0000 7f3b1affd640 1 -- 172.21.15.60:0/344601334 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14005080 con 0x7f3b08004d50 2024-04-25T10:10:12.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.757+0000 7f3b21ff5640 1 -- 172.21.15.60:0/344601334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b08004d50 msgr2=0x7f3b080051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.757+0000 7f3b21ff5640 1 --2- 172.21.15.60:0/344601334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b08004d50 0x7f3b080051d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f3b14001e70 tx=0x7f3b1401c780 comp rx=0 tx=0).stop 2024-04-25T10:10:12.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.757+0000 7f3b1bfff640 1 -- 172.21.15.60:0/344601334 reap_dead start 2024-04-25T10:10:12.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.757+0000 7f3b21ff5640 1 -- 172.21.15.60:0/344601334 shutdown_connections 2024-04-25T10:10:12.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.757+0000 7f3b21ff5640 1 -- 172.21.15.60:0/344601334 >> 172.21.15.60:0/344601334 conn(0x7f3b1c0670b0 msgr2=0x7f3b1c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:12.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.757+0000 7f3b21ff5640 1 -- 172.21.15.60:0/344601334 shutdown_connections 2024-04-25T10:10:12.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.757+0000 7f3b21ff5640 1 -- 172.21.15.60:0/344601334 wait complete. 2024-04-25T10:10:12.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.757+0000 7f3b21ff5640 1 Processor -- start 2024-04-25T10:10:12.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.757+0000 7f3b21ff5640 1 -- start start 2024-04-25T10:10:12.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b21ff5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08004d50 0x7f3b1c156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b21ff5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b1c157bc0 0x7f3b1c1f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b21ff5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c1f07f0 0x7f3b1c15ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b21ff5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0b37b0 con 0x7f3b1c157bc0 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b21ff5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0ccee0 con 0x7f3b08004d50 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b21ff5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c141770 con 0x7f3b1c1f07f0 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b13fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b1c157bc0 0x7f3b1c1f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b13fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b1c157bc0 0x7f3b1c1f5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48026/0 (socket says 172.21.15.60:48026) 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b13fff640 1 -- 172.21.15.60:0/3065390633 learned_addr learned my addr 172.21.15.60:0/3065390633 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b1b7fe640 1 --2- 172.21.15.60:0/3065390633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08004d50 0x7f3b1c156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b1bfff640 1 --2- 172.21.15.60:0/3065390633 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c1f07f0 0x7f3b1c15ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b13fff640 1 -- 172.21.15.60:0/3065390633 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c1f07f0 msgr2=0x7f3b1c15ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b13fff640 1 --2- 172.21.15.60:0/3065390633 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c1f07f0 0x7f3b1c15ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b13fff640 1 -- 172.21.15.60:0/3065390633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08004d50 msgr2=0x7f3b1c156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b13fff640 1 --2- 172.21.15.60:0/3065390633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08004d50 0x7f3b1c156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b13fff640 1 -- 172.21.15.60:0/3065390633 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c1f55c0 con 0x7f3b1c157bc0 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b1b7fe640 1 --2- 172.21.15.60:0/3065390633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08004d50 0x7f3b1c156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b13fff640 1 --2- 172.21.15.60:0/3065390633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b1c157bc0 0x7f3b1c1f5080 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f3b04011980 tx=0x7f3b04011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b18ff9640 1 -- 172.21.15.60:0/3065390633 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04010040 con 0x7f3b1c157bc0 2024-04-25T10:10:12.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b18ff9640 1 -- 172.21.15.60:0/3065390633 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0401a070 con 0x7f3b1c157bc0 2024-04-25T10:10:12.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b21ff5640 1 -- 172.21.15.60:0/3065390633 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1c156b20 con 0x7f3b1c157bc0 2024-04-25T10:10:12.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b18ff9640 1 -- 172.21.15.60:0/3065390633 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400fe40 con 0x7f3b1c157bc0 2024-04-25T10:10:12.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b21ff5640 1 -- 172.21.15.60:0/3065390633 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c1570e0 con 0x7f3b1c157bc0 2024-04-25T10:10:12.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b21ff5640 1 -- 172.21.15.60:0/3065390633 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1c06bba0 con 0x7f3b1c157bc0 2024-04-25T10:10:12.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b18ff9640 1 -- 172.21.15.60:0/3065390633 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b0401f460 con 0x7f3b1c157bc0 2024-04-25T10:10:12.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b18ff9640 1 --2- 172.21.15.60:0/3065390633 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3afc066f60 0x7f3afc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b18ff9640 1 -- 172.21.15.60:0/3065390633 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f3b04088350 con 0x7f3b1c157bc0 2024-04-25T10:10:12.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.761+0000 7f3b1b7fe640 1 --2- 172.21.15.60:0/3065390633 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3afc066f60 0x7f3afc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.765+0000 7f3b1b7fe640 1 --2- 172.21.15.60:0/3065390633 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3afc066f60 0x7f3afc069420 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009af0 tx=0x7f3b0c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.765+0000 7f3b18ff9640 1 -- 172.21.15.60:0/3065390633 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3b1c1f55c0 con 0x7f3b1c157bc0 2024-04-25T10:10:12.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.929+0000 7f3b21ff5640 1 -- 172.21.15.60:0/3065390633 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3b1c0b2970 con 0x7f3b1c157bc0 2024-04-25T10:10:12.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.929+0000 7f3b18ff9640 1 -- 172.21.15.60:0/3065390633 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3b040592d0 con 0x7f3b1c157bc0 2024-04-25T10:10:12.937 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:10:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.929+0000 7f3b21ff5640 1 -- 172.21.15.60:0/3065390633 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3afc066f60 msgr2=0x7f3afc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.929+0000 7f3b21ff5640 1 --2- 172.21.15.60:0/3065390633 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3afc066f60 0x7f3afc069420 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009af0 tx=0x7f3b0c008040 comp rx=0 tx=0).stop 2024-04-25T10:10:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.929+0000 7f3b21ff5640 1 -- 172.21.15.60:0/3065390633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b1c157bc0 msgr2=0x7f3b1c1f5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.929+0000 7f3b21ff5640 1 --2- 172.21.15.60:0/3065390633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b1c157bc0 0x7f3b1c1f5080 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f3b04011980 tx=0x7f3b04011e40 comp rx=0 tx=0).stop 2024-04-25T10:10:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.929+0000 7f3b21ff5640 1 -- 172.21.15.60:0/3065390633 shutdown_connections 2024-04-25T10:10:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.929+0000 7f3b21ff5640 1 --2- 172.21.15.60:0/3065390633 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3afc066f60 0x7f3afc069420 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.929+0000 7f3b21ff5640 1 --2- 172.21.15.60:0/3065390633 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c1f07f0 0x7f3b1c15ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.929+0000 7f3b21ff5640 1 --2- 172.21.15.60:0/3065390633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b1c157bc0 0x7f3b1c1f5080 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.929+0000 7f3b21ff5640 1 --2- 172.21.15.60:0/3065390633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b08004d50 0x7f3b1c156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.929+0000 7f3b21ff5640 1 -- 172.21.15.60:0/3065390633 >> 172.21.15.60:0/3065390633 conn(0x7f3b1c0670b0 msgr2=0x7f3b1c0b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.929+0000 7f3b21ff5640 1 -- 172.21.15.60:0/3065390633 shutdown_connections 2024-04-25T10:10:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:12.933+0000 7f3b21ff5640 1 -- 172.21.15.60:0/3065390633 wait complete. 2024-04-25T10:10:12.957 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:10:13.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84cc829640 1 Processor -- start 2024-04-25T10:10:13.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84cc829640 1 -- start start 2024-04-25T10:10:13.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84cc829640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84c80cefe0 0x7f84c815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84cc829640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f84c80cca60 0x7f84c80b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84cc829640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c815ce20 con 0x7f84c80b35e0 2024-04-25T10:10:13.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84cc829640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c8071410 con 0x7f84c80cefe0 2024-04-25T10:10:13.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84cc829640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c815c580 con 0x7f84c80cca60 2024-04-25T10:10:13.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84c659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84c80cefe0 0x7f84c815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84c659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84c80cefe0 0x7f84c815bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48034/0 (socket says 172.21.15.60:48034) 2024-04-25T10:10:13.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84c659c640 1 -- 172.21.15.60:0/1550021826 learned_addr learned my addr 172.21.15.60:0/1550021826 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:13.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84c659c640 1 -- 172.21.15.60:0/1550021826 >> v2:172.21.15.100:3300/0 conn(0x7f84c80cca60 msgr2=0x7f84c80b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:13.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84c5d9b640 1 --2- 172.21.15.60:0/1550021826 >> v2:172.21.15.100:3300/0 conn(0x7f84c80cca60 0x7f84c80b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84c659c640 1 --2- 172.21.15.60:0/1550021826 >> v2:172.21.15.100:3300/0 conn(0x7f84c80cca60 0x7f84c80b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84c559a640 1 -- 172.21.15.60:0/1550021826 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545030627 0 0) 0x7f84c815ce20 con 0x7f84c80b35e0 2024-04-25T10:10:13.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84c659c640 1 -- 172.21.15.60:0/1550021826 >> v1:172.21.15.60:6789/0 conn(0x7f84c80b35e0 legacy=0x7f84c80718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84c659c640 1 -- 172.21.15.60:0/1550021826 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c815c860 con 0x7f84c80cefe0 2024-04-25T10:10:13.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.069+0000 7f84c5d9b640 1 --2- 172.21.15.60:0/1550021826 >> v2:172.21.15.100:3300/0 conn(0x7f84c80cca60 0x7f84c80b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:13.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84c659c640 1 --2- 172.21.15.60:0/1550021826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80cefe0 0x7f84c815bea0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f84c8071590 tx=0x7f84bc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2fefc78392298c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:13.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84c559a640 1 -- 172.21.15.60:0/1550021826 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0ab680 con 0x7f84c80cefe0 2024-04-25T10:10:13.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84c559a640 1 -- 172.21.15.60:0/1550021826 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84bc0b0070 con 0x7f84c80cefe0 2024-04-25T10:10:13.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84c559a640 1 -- 172.21.15.60:0/1550021826 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0aba40 con 0x7f84c80cefe0 2024-04-25T10:10:13.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84cc829640 1 -- 172.21.15.60:0/1550021826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80cefe0 msgr2=0x7f84c815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84cc829640 1 --2- 172.21.15.60:0/1550021826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80cefe0 0x7f84c815bea0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f84c8071590 tx=0x7f84bc01c460 comp rx=0 tx=0).stop 2024-04-25T10:10:13.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84cc829640 1 -- 172.21.15.60:0/1550021826 shutdown_connections 2024-04-25T10:10:13.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84cc829640 1 --2- 172.21.15.60:0/1550021826 >> v2:172.21.15.100:3300/0 conn(0x7f84c80cca60 0x7f84c80b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84cc829640 1 --2- 172.21.15.60:0/1550021826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80cefe0 0x7f84c815bea0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84cc829640 1 -- 172.21.15.60:0/1550021826 >> 172.21.15.60:0/1550021826 conn(0x7f84c80670b0 msgr2=0x7f84c806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:13.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84cc829640 1 -- 172.21.15.60:0/1550021826 shutdown_connections 2024-04-25T10:10:13.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84cc829640 1 -- 172.21.15.60:0/1550021826 wait complete. 2024-04-25T10:10:13.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84cc829640 1 Processor -- start 2024-04-25T10:10:13.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84cc829640 1 -- start start 2024-04-25T10:10:13.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84cc829640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c80b35e0 0x7f84c8149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84cc829640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80cca60 0x7f84c814b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84cc829640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80cefe0 0x7f84c8154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84cc829640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c8168ba0 con 0x7f84c80cefe0 2024-04-25T10:10:13.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84cc829640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c8168a20 con 0x7f84c80b35e0 2024-04-25T10:10:13.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84cc829640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c8168d20 con 0x7f84c80cca60 2024-04-25T10:10:13.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84c6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80cefe0 0x7f84c8154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84c6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80cefe0 0x7f84c8154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48042/0 (socket says 172.21.15.60:48042) 2024-04-25T10:10:13.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.073+0000 7f84c6d9d640 1 -- 172.21.15.60:0/1525640671 learned_addr learned my addr 172.21.15.60:0/1525640671 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:13.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84c5d9b640 1 --2- 172.21.15.60:0/1525640671 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80cca60 0x7f84c814b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84c659c640 1 --2- 172.21.15.60:0/1525640671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c80b35e0 0x7f84c8149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84c6d9d640 1 -- 172.21.15.60:0/1525640671 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80cca60 msgr2=0x7f84c814b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84c6d9d640 1 --2- 172.21.15.60:0/1525640671 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80cca60 0x7f84c814b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84c6d9d640 1 -- 172.21.15.60:0/1525640671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c80b35e0 msgr2=0x7f84c8149860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84c6d9d640 1 --2- 172.21.15.60:0/1525640671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c80b35e0 0x7f84c8149860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84c6d9d640 1 -- 172.21.15.60:0/1525640671 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c814dfe0 con 0x7f84c80cefe0 2024-04-25T10:10:13.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84c6d9d640 1 --2- 172.21.15.60:0/1525640671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80cefe0 0x7f84c8154b90 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f84c0001570 tx=0x7f84c000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:13.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84b6ffd640 1 -- 172.21.15.60:0/1525640671 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0017070 con 0x7f84c80cefe0 2024-04-25T10:10:13.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84b6ffd640 1 -- 172.21.15.60:0/1525640671 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c0003db0 con 0x7f84c80cefe0 2024-04-25T10:10:13.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84cc829640 1 -- 172.21.15.60:0/1525640671 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c814e280 con 0x7f84c80cefe0 2024-04-25T10:10:13.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84b6ffd640 1 -- 172.21.15.60:0/1525640671 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c00052b0 con 0x7f84c80cefe0 2024-04-25T10:10:13.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84cc829640 1 -- 172.21.15.60:0/1525640671 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c814e840 con 0x7f84c80cefe0 2024-04-25T10:10:13.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84cc829640 1 -- 172.21.15.60:0/1525640671 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c806bba0 con 0x7f84c80cefe0 2024-04-25T10:10:13.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84b6ffd640 1 -- 172.21.15.60:0/1525640671 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f84c001a070 con 0x7f84c80cefe0 2024-04-25T10:10:13.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84b6ffd640 1 --2- 172.21.15.60:0/1525640671 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84a8066f90 0x7f84a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84b6ffd640 1 -- 172.21.15.60:0/1525640671 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f84c0084770 con 0x7f84c80cefe0 2024-04-25T10:10:13.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84c659c640 1 --2- 172.21.15.60:0/1525640671 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84a8066f90 0x7f84a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.077+0000 7f84c659c640 1 --2- 172.21.15.60:0/1525640671 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84a8066f90 0x7f84a8069450 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0af000 tx=0x7f84bc0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:13.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.081+0000 7f84b6ffd640 1 -- 172.21.15.60:0/1525640671 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f84c00556e0 con 0x7f84c80cefe0 2024-04-25T10:10:13.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.237+0000 7f84cc829640 1 -- 172.21.15.60:0/1525640671 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f84c806aed0 con 0x7f84c80cefe0 2024-04-25T10:10:13.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.237+0000 7f84b6ffd640 1 -- 172.21.15.60:0/1525640671 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f84c0054fe0 con 0x7f84c80cefe0 2024-04-25T10:10:13.247 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:10:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.241+0000 7f84cc829640 1 -- 172.21.15.60:0/1525640671 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84a8066f90 msgr2=0x7f84a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.241+0000 7f84cc829640 1 --2- 172.21.15.60:0/1525640671 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84a8066f90 0x7f84a8069450 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0af000 tx=0x7f84bc0a7000 comp rx=0 tx=0).stop 2024-04-25T10:10:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.241+0000 7f84cc829640 1 -- 172.21.15.60:0/1525640671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80cefe0 msgr2=0x7f84c8154b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.241+0000 7f84cc829640 1 --2- 172.21.15.60:0/1525640671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80cefe0 0x7f84c8154b90 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f84c0001570 tx=0x7f84c000cd20 comp rx=0 tx=0).stop 2024-04-25T10:10:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.241+0000 7f84cc829640 1 -- 172.21.15.60:0/1525640671 shutdown_connections 2024-04-25T10:10:13.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.241+0000 7f84cc829640 1 --2- 172.21.15.60:0/1525640671 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84a8066f90 0x7f84a8069450 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.241+0000 7f84cc829640 1 --2- 172.21.15.60:0/1525640671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80cefe0 0x7f84c8154b90 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.241+0000 7f84cc829640 1 --2- 172.21.15.60:0/1525640671 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80cca60 0x7f84c814b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.241+0000 7f84cc829640 1 --2- 172.21.15.60:0/1525640671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c80b35e0 0x7f84c8149860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.241+0000 7f84cc829640 1 -- 172.21.15.60:0/1525640671 >> 172.21.15.60:0/1525640671 conn(0x7f84c80670b0 msgr2=0x7f84c80af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:13.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.241+0000 7f84cc829640 1 -- 172.21.15.60:0/1525640671 shutdown_connections 2024-04-25T10:10:13.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.241+0000 7f84cc829640 1 -- 172.21.15.60:0/1525640671 wait complete. 2024-04-25T10:10:13.265 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:10:13.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.381+0000 7f4d4c3e0640 1 Processor -- start 2024-04-25T10:10:13.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.381+0000 7f4d4c3e0640 1 -- start start 2024-04-25T10:10:13.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.385+0000 7f4d4c3e0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d4415deb0 0x7f4d440b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.385+0000 7f4d4c3e0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4406d200 con 0x7f4d440ce9c0 2024-04-25T10:10:13.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.385+0000 7f4d4c3e0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d440cefe0 con 0x7f4d440d0340 2024-04-25T10:10:13.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.385+0000 7f4d4c3e0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d440cf2c0 con 0x7f4d4415deb0 2024-04-25T10:10:13.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.385+0000 7f4d4a17c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d4415deb0 0x7f4d440b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.385+0000 7f4d4a97d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4d440ce9c0 0x7f4d440d2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55382/0 (socket says 172.21.15.60:55382) 2024-04-25T10:10:13.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.385+0000 7f4d4a97d640 1 -- 172.21.15.60:0/3738178743 learned_addr learned my addr 172.21.15.60:0/3738178743 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:10:13.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.385+0000 7f4d4a17c640 1 -- 172.21.15.60:0/3738178743 >> v1:172.21.15.100:6789/0 conn(0x7f4d440d0340 legacy=0x7f4d440ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:13.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.385+0000 7f4d4a17c640 1 -- 172.21.15.60:0/3738178743 >> v1:172.21.15.60:6789/0 conn(0x7f4d440ce9c0 legacy=0x7f4d440d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.385+0000 7f4d4a17c640 1 -- 172.21.15.60:0/3738178743 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d440d1d70 con 0x7f4d4415deb0 2024-04-25T10:10:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.385+0000 7f4d4a17c640 1 --2- 172.21.15.60:0/3738178743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d4415deb0 0x7f4d440b6c10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f4d380097c0 tx=0x7f4d3801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57c5509f57769d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.385+0000 7f4d4917a640 1 -- 172.21.15.60:0/3738178743 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d380b0070 con 0x7f4d4415deb0 2024-04-25T10:10:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.385+0000 7f4d4917a640 1 -- 172.21.15.60:0/3738178743 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d380ab600 con 0x7f4d4415deb0 2024-04-25T10:10:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.385+0000 7f4d4917a640 1 -- 172.21.15.60:0/3738178743 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d380ab9d0 con 0x7f4d4415deb0 2024-04-25T10:10:13.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 -- 172.21.15.60:0/3738178743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d4415deb0 msgr2=0x7f4d440b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 --2- 172.21.15.60:0/3738178743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d4415deb0 0x7f4d440b6c10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f4d380097c0 tx=0x7f4d3801c460 comp rx=0 tx=0).stop 2024-04-25T10:10:13.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 -- 172.21.15.60:0/3738178743 shutdown_connections 2024-04-25T10:10:13.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 --2- 172.21.15.60:0/3738178743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d4415deb0 0x7f4d440b6c10 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 -- 172.21.15.60:0/3738178743 >> 172.21.15.60:0/3738178743 conn(0x7f4d4406c090 msgr2=0x7f4d4406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:13.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 -- 172.21.15.60:0/3738178743 shutdown_connections 2024-04-25T10:10:13.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 -- 172.21.15.60:0/3738178743 wait complete. 2024-04-25T10:10:13.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 Processor -- start 2024-04-25T10:10:13.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 -- start start 2024-04-25T10:10:13.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d440ce9c0 0x7f4d441fb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d440d0340 0x7f4d44149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d4415deb0 0x7f4d44151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4416bdc0 con 0x7f4d440ce9c0 2024-04-25T10:10:13.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4416bc40 con 0x7f4d440d0340 2024-04-25T10:10:13.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4416bf40 con 0x7f4d4415deb0 2024-04-25T10:10:13.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4a17c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d440ce9c0 0x7f4d441fb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4a17c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d440ce9c0 0x7f4d441fb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48062/0 (socket says 172.21.15.60:48062) 2024-04-25T10:10:13.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4a17c640 1 -- 172.21.15.60:0/4184725793 learned_addr learned my addr 172.21.15.60:0/4184725793 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:13.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4a97d640 1 --2- 172.21.15.60:0/4184725793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d4415deb0 0x7f4d44151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4997b640 1 --2- 172.21.15.60:0/4184725793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d440d0340 0x7f4d44149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4a17c640 1 -- 172.21.15.60:0/4184725793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d4415deb0 msgr2=0x7f4d44151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4a17c640 1 --2- 172.21.15.60:0/4184725793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d4415deb0 0x7f4d44151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4a17c640 1 -- 172.21.15.60:0/4184725793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d440d0340 msgr2=0x7f4d44149d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4a17c640 1 --2- 172.21.15.60:0/4184725793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d440d0340 0x7f4d44149d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4a17c640 1 -- 172.21.15.60:0/4184725793 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d44151170 con 0x7f4d440ce9c0 2024-04-25T10:10:13.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4a97d640 1 --2- 172.21.15.60:0/4184725793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d4415deb0 0x7f4d44151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:13.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4997b640 1 --2- 172.21.15.60:0/4184725793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d440d0340 0x7f4d44149d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:13.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4a17c640 1 --2- 172.21.15.60:0/4184725793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d440ce9c0 0x7f4d441fb3c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009710 tx=0x7f4d3801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:13.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d36ffd640 1 -- 172.21.15.60:0/4184725793 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d380b0050 con 0x7f4d440ce9c0 2024-04-25T10:10:13.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d36ffd640 1 -- 172.21.15.60:0/4184725793 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d380b1040 con 0x7f4d440ce9c0 2024-04-25T10:10:13.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 -- 172.21.15.60:0/4184725793 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d44151450 con 0x7f4d440ce9c0 2024-04-25T10:10:13.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d36ffd640 1 -- 172.21.15.60:0/4184725793 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d380b5610 con 0x7f4d440ce9c0 2024-04-25T10:10:13.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.389+0000 7f4d4c3e0640 1 -- 172.21.15.60:0/4184725793 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d441fabc0 con 0x7f4d440ce9c0 2024-04-25T10:10:13.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.393+0000 7f4d4c3e0640 1 -- 172.21.15.60:0/4184725793 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d44074770 con 0x7f4d440ce9c0 2024-04-25T10:10:13.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.393+0000 7f4d36ffd640 1 -- 172.21.15.60:0/4184725793 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d380be030 con 0x7f4d440ce9c0 2024-04-25T10:10:13.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.393+0000 7f4d36ffd640 1 --2- 172.21.15.60:0/4184725793 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4d28066f60 0x7f4d28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.393+0000 7f4d36ffd640 1 -- 172.21.15.60:0/4184725793 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4d3811eda0 con 0x7f4d440ce9c0 2024-04-25T10:10:13.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.393+0000 7f4d4997b640 1 --2- 172.21.15.60:0/4184725793 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4d28066f60 0x7f4d28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.393+0000 7f4d4997b640 1 --2- 172.21.15.60:0/4184725793 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4d28066f60 0x7f4d28069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4d441fba30 tx=0x7f4d40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:13.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.397+0000 7f4d36ffd640 1 -- 172.21.15.60:0/4184725793 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d380c5060 con 0x7f4d440ce9c0 2024-04-25T10:10:13.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.569+0000 7f4d4c3e0640 1 -- 172.21.15.60:0/4184725793 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4d440cd5e0 con 0x7f4d440ce9c0 2024-04-25T10:10:13.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.573+0000 7f4d36ffd640 1 -- 172.21.15.60:0/4184725793 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v71) v1 ==== 80+0+122 (secure 0 0 0) 0x7f4d38016050 con 0x7f4d440ce9c0 2024-04-25T10:10:13.580 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6835/1289503429 2024-04-26T10:10:03.449620+0000 2024-04-25T10:10:13.580 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6833/1289503429 2024-04-26T10:10:03.449620+0000 2024-04-25T10:10:13.581 INFO:teuthology.orchestra.run.smithi060.stderr:listed 2 entries 2024-04-25T10:10:13.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.573+0000 7f4d4c3e0640 1 -- 172.21.15.60:0/4184725793 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4d28066f60 msgr2=0x7f4d28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.573+0000 7f4d4c3e0640 1 --2- 172.21.15.60:0/4184725793 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4d28066f60 0x7f4d28069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4d441fba30 tx=0x7f4d40009040 comp rx=0 tx=0).stop 2024-04-25T10:10:13.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.573+0000 7f4d4c3e0640 1 -- 172.21.15.60:0/4184725793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d440ce9c0 msgr2=0x7f4d441fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.573+0000 7f4d4c3e0640 1 --2- 172.21.15.60:0/4184725793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d440ce9c0 0x7f4d441fb3c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009710 tx=0x7f4d3801aca0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.573+0000 7f4d4c3e0640 1 -- 172.21.15.60:0/4184725793 shutdown_connections 2024-04-25T10:10:13.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.573+0000 7f4d4c3e0640 1 --2- 172.21.15.60:0/4184725793 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4d28066f60 0x7f4d28069420 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.573+0000 7f4d4c3e0640 1 --2- 172.21.15.60:0/4184725793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d4415deb0 0x7f4d44151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.573+0000 7f4d4c3e0640 1 --2- 172.21.15.60:0/4184725793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d440d0340 0x7f4d44149d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.573+0000 7f4d4c3e0640 1 --2- 172.21.15.60:0/4184725793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d440ce9c0 0x7f4d441fb3c0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.573+0000 7f4d4c3e0640 1 -- 172.21.15.60:0/4184725793 >> 172.21.15.60:0/4184725793 conn(0x7f4d4406c090 msgr2=0x7f4d440d2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:13.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.573+0000 7f4d4c3e0640 1 -- 172.21.15.60:0/4184725793 shutdown_connections 2024-04-25T10:10:13.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.573+0000 7f4d4c3e0640 1 -- 172.21.15.60:0/4184725793 wait complete. 2024-04-25T10:10:13.600 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:10:13.600 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:13.605 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:13.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa32380640 1 Processor -- start 2024-04-25T10:10:13.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa32380640 1 -- start start 2024-04-25T10:10:13.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa32380640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faa2c0ce7a0 0x7faa2c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa32380640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2c15bc90 0x7faa2c0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa32380640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2c0cf0c0 con 0x7faa2c0b4660 2024-04-25T10:10:13.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa32380640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2c0cc370 con 0x7faa2c0ce7a0 2024-04-25T10:10:13.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa32380640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c071ca0 con 0x7faa2c15bc90 2024-04-25T10:10:13.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa2bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faa2c0ce7a0 0x7faa2c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa2bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faa2c0ce7a0 0x7faa2c0ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48076/0 (socket says 172.21.15.60:48076) 2024-04-25T10:10:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa3091d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7faa2c0b4660 0x7faa2c0b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55396/0 (socket says 172.21.15.60:55396) 2024-04-25T10:10:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa2bfff640 1 -- 172.21.15.60:0/1498172706 learned_addr learned my addr 172.21.15.60:0/1498172706 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa2b7fe640 1 --2- 172.21.15.60:0/1498172706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2c15bc90 0x7faa2c0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa2bfff640 1 -- 172.21.15.60:0/1498172706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2c15bc90 msgr2=0x7faa2c0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa2bfff640 1 --2- 172.21.15.60:0/1498172706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2c15bc90 0x7faa2c0b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa2bfff640 1 -- 172.21.15.60:0/1498172706 >> v1:172.21.15.60:6789/0 conn(0x7faa2c0b4660 legacy=0x7faa2c0b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa2affd640 1 -- 172.21.15.60:0/1498172706 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973964026 0 0) 0x7faa2c0cf0c0 con 0x7faa2c0b4660 2024-04-25T10:10:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa2bfff640 1 -- 172.21.15.60:0/1498172706 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c0cca40 con 0x7faa2c0ce7a0 2024-04-25T10:10:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.761+0000 7faa2b7fe640 1 --2- 172.21.15.60:0/1498172706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2c15bc90 0x7faa2c0b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:13.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.765+0000 7faa2bfff640 1 --2- 172.21.15.60:0/1498172706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa2c0ce7a0 0x7faa2c0ceb80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0098a0 tx=0x7faa1c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7573b01e26920364 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:13.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.765+0000 7faa2affd640 1 -- 172.21.15.60:0/1498172706 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0b0070 con 0x7faa2c0ce7a0 2024-04-25T10:10:13.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.765+0000 7faa2affd640 1 -- 172.21.15.60:0/1498172706 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1c0ab600 con 0x7faa2c0ce7a0 2024-04-25T10:10:13.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.765+0000 7faa2affd640 1 -- 172.21.15.60:0/1498172706 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0ab9d0 con 0x7faa2c0ce7a0 2024-04-25T10:10:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.765+0000 7faa32380640 1 -- 172.21.15.60:0/1498172706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa2c0ce7a0 msgr2=0x7faa2c0ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.765+0000 7faa32380640 1 --2- 172.21.15.60:0/1498172706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa2c0ce7a0 0x7faa2c0ceb80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0098a0 tx=0x7faa1c01c540 comp rx=0 tx=0).stop 2024-04-25T10:10:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.765+0000 7faa32380640 1 -- 172.21.15.60:0/1498172706 shutdown_connections 2024-04-25T10:10:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.765+0000 7faa32380640 1 --2- 172.21.15.60:0/1498172706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2c15bc90 0x7faa2c0b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.765+0000 7faa32380640 1 --2- 172.21.15.60:0/1498172706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa2c0ce7a0 0x7faa2c0ceb80 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.765+0000 7faa32380640 1 -- 172.21.15.60:0/1498172706 >> 172.21.15.60:0/1498172706 conn(0x7faa2c0b12d0 msgr2=0x7faa2c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.765+0000 7faa32380640 1 -- 172.21.15.60:0/1498172706 shutdown_connections 2024-04-25T10:10:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.765+0000 7faa32380640 1 -- 172.21.15.60:0/1498172706 wait complete. 2024-04-25T10:10:13.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.765+0000 7faa32380640 1 Processor -- start 2024-04-25T10:10:13.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.765+0000 7faa32380640 1 -- start start 2024-04-25T10:10:13.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa32380640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2c0ce7a0 0x7faa2c14c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa32380640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa2c15bc90 0x7faa2c0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa32380640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa2c14f590 0x7faa2c0b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa32380640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0b4280 con 0x7faa2c14f590 2024-04-25T10:10:13.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa32380640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0b3db0 con 0x7faa2c15bc90 2024-04-25T10:10:13.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa32380640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0b4400 con 0x7faa2c0ce7a0 2024-04-25T10:10:13.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa3091d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa2c14f590 0x7faa2c0b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa3091d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa2c14f590 0x7faa2c0b2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48090/0 (socket says 172.21.15.60:48090) 2024-04-25T10:10:13.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa3091d640 1 -- 172.21.15.60:0/1379672952 learned_addr learned my addr 172.21.15.60:0/1379672952 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:13.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa2b7fe640 1 --2- 172.21.15.60:0/1379672952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa2c15bc90 0x7faa2c0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa2bfff640 1 --2- 172.21.15.60:0/1379672952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2c0ce7a0 0x7faa2c14c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa3091d640 1 -- 172.21.15.60:0/1379672952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2c0ce7a0 msgr2=0x7faa2c14c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa3091d640 1 --2- 172.21.15.60:0/1379672952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2c0ce7a0 0x7faa2c14c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa3091d640 1 -- 172.21.15.60:0/1379672952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa2c15bc90 msgr2=0x7faa2c0b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa3091d640 1 --2- 172.21.15.60:0/1379672952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa2c15bc90 0x7faa2c0b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa3091d640 1 -- 172.21.15.60:0/1379672952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c151080 con 0x7faa2c14f590 2024-04-25T10:10:13.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa2bfff640 1 --2- 172.21.15.60:0/1379672952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2c0ce7a0 0x7faa2c14c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:13.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa3091d640 1 --2- 172.21.15.60:0/1379672952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa2c14f590 0x7faa2c0b2ee0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7faa2400c6a0 tx=0x7faa2400cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:13.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa28ff9640 1 -- 172.21.15.60:0/1379672952 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa24018070 con 0x7faa2c14f590 2024-04-25T10:10:13.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa28ff9640 1 -- 172.21.15.60:0/1379672952 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa24004d70 con 0x7faa2c14f590 2024-04-25T10:10:13.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa32380640 1 -- 172.21.15.60:0/1379672952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2c151360 con 0x7faa2c14f590 2024-04-25T10:10:13.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa28ff9640 1 -- 172.21.15.60:0/1379672952 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2400a880 con 0x7faa2c14f590 2024-04-25T10:10:13.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa32380640 1 -- 172.21.15.60:0/1379672952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2c14c690 con 0x7faa2c14f590 2024-04-25T10:10:13.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa32380640 1 -- 172.21.15.60:0/1379672952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2c074770 con 0x7faa2c14f590 2024-04-25T10:10:13.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa28ff9640 1 -- 172.21.15.60:0/1379672952 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa2401b070 con 0x7faa2c14f590 2024-04-25T10:10:13.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa28ff9640 1 --2- 172.21.15.60:0/1379672952 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faa18066f60 0x7faa18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa28ff9640 1 -- 172.21.15.60:0/1379672952 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7faa240845e0 con 0x7faa2c14f590 2024-04-25T10:10:13.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa2bfff640 1 --2- 172.21.15.60:0/1379672952 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faa18066f60 0x7faa18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.769+0000 7faa2bfff640 1 --2- 172.21.15.60:0/1379672952 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faa18066f60 0x7faa18069420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7faa1c009870 tx=0x7faa1c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:13.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.773+0000 7faa28ff9640 1 -- 172.21.15.60:0/1379672952 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa24055560 con 0x7faa2c14f590 2024-04-25T10:10:13.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.933+0000 7faa32380640 1 -- 172.21.15.60:0/1379672952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa2c0aeb80 con 0x7faa2c14f590 2024-04-25T10:10:13.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.937+0000 7faa28ff9640 1 -- 172.21.15.60:0/1379672952 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16907 (secure 0 0 0) 0x7faa24054e60 con 0x7faa2c14f590 2024-04-25T10:10:13.946 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:10:13.946 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:10:03.449677+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:42.005281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:42.710028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1e","mappings":[{"from":6,"to":0}]},{"pgid":"11.28","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/1289503429":"2024-04-26T10:10:03.449620+0000","172.21.15.60:6833/1289503429":"2024-04-26T10:10:03.449620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:13.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.941+0000 7faa32380640 1 -- 172.21.15.60:0/1379672952 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faa18066f60 msgr2=0x7faa18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.941+0000 7faa32380640 1 --2- 172.21.15.60:0/1379672952 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faa18066f60 0x7faa18069420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7faa1c009870 tx=0x7faa1c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:10:13.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.941+0000 7faa32380640 1 -- 172.21.15.60:0/1379672952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa2c14f590 msgr2=0x7faa2c0b2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.941+0000 7faa32380640 1 --2- 172.21.15.60:0/1379672952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa2c14f590 0x7faa2c0b2ee0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7faa2400c6a0 tx=0x7faa2400cb60 comp rx=0 tx=0).stop 2024-04-25T10:10:13.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.941+0000 7faa32380640 1 -- 172.21.15.60:0/1379672952 shutdown_connections 2024-04-25T10:10:13.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.941+0000 7faa32380640 1 --2- 172.21.15.60:0/1379672952 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faa18066f60 0x7faa18069420 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.941+0000 7faa32380640 1 --2- 172.21.15.60:0/1379672952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa2c14f590 0x7faa2c0b2ee0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.941+0000 7faa32380640 1 --2- 172.21.15.60:0/1379672952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa2c15bc90 0x7faa2c0b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.941+0000 7faa32380640 1 --2- 172.21.15.60:0/1379672952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2c0ce7a0 0x7faa2c14c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.941+0000 7faa32380640 1 -- 172.21.15.60:0/1379672952 >> 172.21.15.60:0/1379672952 conn(0x7faa2c0b12d0 msgr2=0x7faa2c15e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:13.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.941+0000 7faa32380640 1 -- 172.21.15.60:0/1379672952 shutdown_connections 2024-04-25T10:10:13.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:13.941+0000 7faa32380640 1 -- 172.21.15.60:0/1379672952 wait complete. 2024-04-25T10:10:13.965 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:09:42.005281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:09:42.710028+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:13.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:14.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.085+0000 7faf41050640 1 Processor -- start 2024-04-25T10:10:14.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.085+0000 7faf41050640 1 -- start start 2024-04-25T10:10:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.085+0000 7faf41050640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faf3c15dcd0 0x7faf3c06c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.085+0000 7faf41050640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3c06d220 con 0x7faf3c0b2860 2024-04-25T10:10:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.085+0000 7faf41050640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3c0cfa40 con 0x7faf3c0cbd60 2024-04-25T10:10:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.085+0000 7faf41050640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3c0cfce0 con 0x7faf3c15dcd0 2024-04-25T10:10:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.085+0000 7faf3ad9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faf3c15dcd0 0x7faf3c06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.085+0000 7faf3a59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7faf3c0b2860 0x7faf3c0d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55400/0 (socket says 172.21.15.60:55400) 2024-04-25T10:10:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.085+0000 7faf3a59c640 1 -- 172.21.15.60:0/3437464774 learned_addr learned my addr 172.21.15.60:0/3437464774 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:10:14.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.085+0000 7faf3ad9d640 1 -- 172.21.15.60:0/3437464774 >> v1:172.21.15.100:6789/0 conn(0x7faf3c0cbd60 legacy=0x7faf3c099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:14.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.085+0000 7faf3ad9d640 1 -- 172.21.15.60:0/3437464774 >> v1:172.21.15.60:6789/0 conn(0x7faf3c0b2860 legacy=0x7faf3c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.085+0000 7faf39d9b640 1 -- 172.21.15.60:0/3437464774 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145112320 0 0) 0x7faf3c06d220 con 0x7faf3c0b2860 2024-04-25T10:10:14.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.085+0000 7faf3ad9d640 1 -- 172.21.15.60:0/3437464774 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0cffc0 con 0x7faf3c15dcd0 2024-04-25T10:10:14.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.089+0000 7faf3ad9d640 1 --2- 172.21.15.60:0/3437464774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3c15dcd0 0x7faf3c06c940 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7faf300097c0 tx=0x7faf3001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38905042b2ca39b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.089+0000 7faf39d9b640 1 -- 172.21.15.60:0/3437464774 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf300b1070 con 0x7faf3c15dcd0 2024-04-25T10:10:14.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.089+0000 7faf39d9b640 1 -- 172.21.15.60:0/3437464774 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf300a8050 con 0x7faf3c15dcd0 2024-04-25T10:10:14.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.089+0000 7faf39d9b640 1 -- 172.21.15.60:0/3437464774 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf300ac860 con 0x7faf3c15dcd0 2024-04-25T10:10:14.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.089+0000 7faf41050640 1 -- 172.21.15.60:0/3437464774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3c15dcd0 msgr2=0x7faf3c06c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.089+0000 7faf41050640 1 --2- 172.21.15.60:0/3437464774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3c15dcd0 0x7faf3c06c940 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7faf300097c0 tx=0x7faf3001c3e0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.089+0000 7faf41050640 1 -- 172.21.15.60:0/3437464774 shutdown_connections 2024-04-25T10:10:14.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.089+0000 7faf41050640 1 --2- 172.21.15.60:0/3437464774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3c15dcd0 0x7faf3c06c940 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.089+0000 7faf41050640 1 -- 172.21.15.60:0/3437464774 >> 172.21.15.60:0/3437464774 conn(0x7faf3c0b3200 msgr2=0x7faf3c0b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:14.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.089+0000 7faf41050640 1 -- 172.21.15.60:0/3437464774 shutdown_connections 2024-04-25T10:10:14.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.089+0000 7faf41050640 1 -- 172.21.15.60:0/3437464774 wait complete. 2024-04-25T10:10:14.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf41050640 1 Processor -- start 2024-04-25T10:10:14.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf41050640 1 -- start start 2024-04-25T10:10:14.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf41050640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf3c0b2860 0x7faf3c14c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf41050640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf3c0cbd60 0x7faf3c15b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf41050640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3c15dcd0 0x7faf3c15b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf41050640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c06d000 con 0x7faf3c15dcd0 2024-04-25T10:10:14.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf41050640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c06ce80 con 0x7faf3c0b2860 2024-04-25T10:10:14.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf41050640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c15e0b0 con 0x7faf3c0cbd60 2024-04-25T10:10:14.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf3b59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3c15dcd0 0x7faf3c15b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf3b59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3c15dcd0 0x7faf3c15b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48108/0 (socket says 172.21.15.60:48108) 2024-04-25T10:10:14.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf3b59e640 1 -- 172.21.15.60:0/2249498796 learned_addr learned my addr 172.21.15.60:0/2249498796 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:14.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf3a59c640 1 --2- 172.21.15.60:0/2249498796 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf3c0cbd60 0x7faf3c15b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf3ad9d640 1 --2- 172.21.15.60:0/2249498796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf3c0b2860 0x7faf3c14c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf3b59e640 1 -- 172.21.15.60:0/2249498796 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf3c0cbd60 msgr2=0x7faf3c15b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf3b59e640 1 --2- 172.21.15.60:0/2249498796 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf3c0cbd60 0x7faf3c15b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf3b59e640 1 -- 172.21.15.60:0/2249498796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf3c0b2860 msgr2=0x7faf3c14c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf3b59e640 1 --2- 172.21.15.60:0/2249498796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf3c0b2860 0x7faf3c14c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf3b59e640 1 -- 172.21.15.60:0/2249498796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c15a9d0 con 0x7faf3c15dcd0 2024-04-25T10:10:14.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf3ad9d640 1 --2- 172.21.15.60:0/2249498796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf3c0b2860 0x7faf3c14c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:14.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf3b59e640 1 --2- 172.21.15.60:0/2249498796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3c15dcd0 0x7faf3c15b650 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7faf3400c7a0 tx=0x7faf3400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf237fe640 1 -- 172.21.15.60:0/2249498796 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34003b80 con 0x7faf3c15dcd0 2024-04-25T10:10:14.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf237fe640 1 -- 172.21.15.60:0/2249498796 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf34003d20 con 0x7faf3c15dcd0 2024-04-25T10:10:14.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf41050640 1 -- 172.21.15.60:0/2249498796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3c15acb0 con 0x7faf3c15dcd0 2024-04-25T10:10:14.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf237fe640 1 -- 172.21.15.60:0/2249498796 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3400aa90 con 0x7faf3c15dcd0 2024-04-25T10:10:14.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf41050640 1 -- 172.21.15.60:0/2249498796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3c14ffd0 con 0x7faf3c15dcd0 2024-04-25T10:10:14.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.093+0000 7faf41050640 1 -- 172.21.15.60:0/2249498796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c1502f0 con 0x7faf3c15dcd0 2024-04-25T10:10:14.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.097+0000 7faf237fe640 1 -- 172.21.15.60:0/2249498796 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf3400ac30 con 0x7faf3c15dcd0 2024-04-25T10:10:14.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.097+0000 7faf237fe640 1 --2- 172.21.15.60:0/2249498796 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faf28066f90 0x7faf28069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.097+0000 7faf237fe640 1 -- 172.21.15.60:0/2249498796 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7faf340850b0 con 0x7faf3c15dcd0 2024-04-25T10:10:14.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.097+0000 7faf3ad9d640 1 --2- 172.21.15.60:0/2249498796 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faf28066f90 0x7faf28069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.097+0000 7faf3ad9d640 1 --2- 172.21.15.60:0/2249498796 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faf28066f90 0x7faf28069450 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7faf300ae000 tx=0x7faf30008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.101+0000 7faf237fe640 1 -- 172.21.15.60:0/2249498796 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf34055f90 con 0x7faf3c15dcd0 2024-04-25T10:10:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.261+0000 7faf41050640 1 -- 172.21.15.60:0/2249498796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf3c153c80 con 0x7faf3c15dcd0 2024-04-25T10:10:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.261+0000 7faf237fe640 1 -- 172.21.15.60:0/2249498796 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf34055890 con 0x7faf3c15dcd0 2024-04-25T10:10:14.269 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:10:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.261+0000 7faf41050640 1 -- 172.21.15.60:0/2249498796 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faf28066f90 msgr2=0x7faf28069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.261+0000 7faf41050640 1 --2- 172.21.15.60:0/2249498796 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faf28066f90 0x7faf28069450 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7faf300ae000 tx=0x7faf30008040 comp rx=0 tx=0).stop 2024-04-25T10:10:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.261+0000 7faf41050640 1 -- 172.21.15.60:0/2249498796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3c15dcd0 msgr2=0x7faf3c15b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.261+0000 7faf41050640 1 --2- 172.21.15.60:0/2249498796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3c15dcd0 0x7faf3c15b650 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7faf3400c7a0 tx=0x7faf3400cc60 comp rx=0 tx=0).stop 2024-04-25T10:10:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.261+0000 7faf41050640 1 -- 172.21.15.60:0/2249498796 shutdown_connections 2024-04-25T10:10:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.261+0000 7faf41050640 1 --2- 172.21.15.60:0/2249498796 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7faf28066f90 0x7faf28069450 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.261+0000 7faf41050640 1 --2- 172.21.15.60:0/2249498796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf3c15dcd0 0x7faf3c15b650 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.261+0000 7faf41050640 1 --2- 172.21.15.60:0/2249498796 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf3c0cbd60 0x7faf3c15b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.261+0000 7faf41050640 1 --2- 172.21.15.60:0/2249498796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf3c0b2860 0x7faf3c14c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.261+0000 7faf41050640 1 -- 172.21.15.60:0/2249498796 >> 172.21.15.60:0/2249498796 conn(0x7faf3c0b3200 msgr2=0x7faf3c06f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.261+0000 7faf41050640 1 -- 172.21.15.60:0/2249498796 shutdown_connections 2024-04-25T10:10:14.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.261+0000 7faf41050640 1 -- 172.21.15.60:0/2249498796 wait complete. 2024-04-25T10:10:14.286 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:10:14.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fe2df8640 1 Processor -- start 2024-04-25T10:10:14.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fe2df8640 1 -- start start 2024-04-25T10:10:14.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fe2df8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2fdc06f030 0x7f2fdc0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fe2df8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fdc15ad30 con 0x7f2fdc071250 2024-04-25T10:10:14.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fe2df8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fdc15b010 con 0x7f2fdc0b3f60 2024-04-25T10:10:14.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fe2df8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fdc15aaf0 con 0x7f2fdc06f030 2024-04-25T10:10:14.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fe1395640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2fdc071250 0x7f2fdc15b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55408/0 (socket says 172.21.15.60:55408) 2024-04-25T10:10:14.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fe1395640 1 -- 172.21.15.60:0/1629750201 learned_addr learned my addr 172.21.15.60:0/1629750201 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:10:14.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdbfff640 1 --2- 172.21.15.60:0/1629750201 >> v2:172.21.15.100:3300/0 conn(0x7f2fdc06f030 0x7f2fdc0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdb7fe640 1 -- 172.21.15.60:0/1629750201 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440236488 0 0) 0x7f2fdc15ad30 con 0x7f2fdc071250 2024-04-25T10:10:14.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdb7fe640 1 -- 172.21.15.60:0/1629750201 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fcc003450 con 0x7f2fdc071250 2024-04-25T10:10:14.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdb7fe640 1 -- 172.21.15.60:0/1629750201 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 721530565 0 0) 0x7f2fcc003450 con 0x7f2fdc071250 2024-04-25T10:10:14.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdb7fe640 1 -- 172.21.15.60:0/1629750201 >> v2:172.21.15.100:3300/0 conn(0x7f2fdc06f030 msgr2=0x7f2fdc0b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:14.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdb7fe640 1 --2- 172.21.15.60:0/1629750201 >> v2:172.21.15.100:3300/0 conn(0x7f2fdc06f030 0x7f2fdc0b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdb7fe640 1 -- 172.21.15.60:0/1629750201 >> v1:172.21.15.100:6789/0 conn(0x7f2fdc0b3f60 legacy=0x7f2fdc15a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdb7fe640 1 -- 172.21.15.60:0/1629750201 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc15b2f0 con 0x7f2fdc071250 2024-04-25T10:10:14.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdb7fe640 1 -- 172.21.15.60:0/1629750201 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f2fd4001a00 con 0x7f2fdc071250 2024-04-25T10:10:14.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdb7fe640 1 -- 172.21.15.60:0/1629750201 >> v1:172.21.15.60:6789/0 conn(0x7f2fdc071250 legacy=0x7f2fdc15b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdb7fe640 1 --2- 172.21.15.60:0/1629750201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2fcc003ab0 0x7f2fcc003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdb7fe640 1 --2- 172.21.15.60:0/1629750201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fcc008860 0x7f2fcc00acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdb7fe640 1 --2- 172.21.15.60:0/1629750201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fcc00b570 0x7f2fcc00da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdb7fe640 1 -- 172.21.15.60:0/1629750201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc15aaf0 con 0x7f2fcc00b570 2024-04-25T10:10:14.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdb7fe640 1 -- 172.21.15.60:0/1629750201 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc00e310 con 0x7f2fcc008860 2024-04-25T10:10:14.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdb7fe640 1 -- 172.21.15.60:0/1629750201 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc00e510 con 0x7f2fcc003ab0 2024-04-25T10:10:14.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fe1395640 1 --2- 172.21.15.60:0/1629750201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fcc00b570 0x7f2fcc00da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fe0b94640 1 --2- 172.21.15.60:0/1629750201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2fcc003ab0 0x7f2fcc003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdbfff640 1 --2- 172.21.15.60:0/1629750201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fcc008860 0x7f2fcc00acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fe1395640 1 -- 172.21.15.60:0/1629750201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2fcc003ab0 msgr2=0x7f2fcc003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:14.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fe1395640 1 --2- 172.21.15.60:0/1629750201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2fcc003ab0 0x7f2fcc003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fe1395640 1 -- 172.21.15.60:0/1629750201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fcc008860 msgr2=0x7f2fcc00acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:14.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fe1395640 1 --2- 172.21.15.60:0/1629750201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fcc008860 0x7f2fcc00acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fe1395640 1 -- 172.21.15.60:0/1629750201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc003450 con 0x7f2fcc00b570 2024-04-25T10:10:14.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fdbfff640 1 --2- 172.21.15.60:0/1629750201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fcc008860 0x7f2fcc00acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:14.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.409+0000 7f2fe0b94640 1 --2- 172.21.15.60:0/1629750201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2fcc003ab0 0x7f2fcc003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe1395640 1 -- 172.21.15.60:0/1629750201 reap_dead start 2024-04-25T10:10:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe1395640 1 --2- 172.21.15.60:0/1629750201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fcc00b570 0x7f2fcc00da30 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400a840 tx=0x7f2fd401fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b585334d21c402a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fdb7fe640 1 -- 172.21.15.60:0/1629750201 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400a460 con 0x7f2fcc00b570 2024-04-25T10:10:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fdb7fe640 1 -- 172.21.15.60:0/1629750201 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd40aa710 con 0x7f2fcc00b570 2024-04-25T10:10:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fdb7fe640 1 -- 172.21.15.60:0/1629750201 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd40aab00 con 0x7f2fcc00b570 2024-04-25T10:10:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe2df8640 1 -- 172.21.15.60:0/1629750201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fcc00b570 msgr2=0x7f2fcc00da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe2df8640 1 --2- 172.21.15.60:0/1629750201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fcc00b570 0x7f2fcc00da30 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400a840 tx=0x7f2fd401fac0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe2df8640 1 -- 172.21.15.60:0/1629750201 shutdown_connections 2024-04-25T10:10:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe2df8640 1 --2- 172.21.15.60:0/1629750201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fcc00b570 0x7f2fcc00da30 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe2df8640 1 -- 172.21.15.60:0/1629750201 >> 172.21.15.60:0/1629750201 conn(0x7f2fdc0670b0 msgr2=0x7f2fdc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:14.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe2df8640 1 -- 172.21.15.60:0/1629750201 shutdown_connections 2024-04-25T10:10:14.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe2df8640 1 -- 172.21.15.60:0/1629750201 wait complete. 2024-04-25T10:10:14.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe2df8640 1 Processor -- start 2024-04-25T10:10:14.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe2df8640 1 -- start start 2024-04-25T10:10:14.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe2df8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fcc00b570 0x7f2fdc0ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe2df8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2fdc159590 0x7f2fdc0a8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe2df8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fdc0b0060 0x7f2fdc0b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe2df8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc0716e0 con 0x7f2fcc00b570 2024-04-25T10:10:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe2df8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc0b4340 con 0x7f2fdc0b0060 2024-04-25T10:10:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe2df8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc071860 con 0x7f2fdc159590 2024-04-25T10:10:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe0b94640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fcc00b570 0x7f2fdc0ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.413+0000 7f2fe0b94640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fcc00b570 0x7f2fdc0ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48120/0 (socket says 172.21.15.60:48120) 2024-04-25T10:10:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fe0b94640 1 -- 172.21.15.60:0/360717002 learned_addr learned my addr 172.21.15.60:0/360717002 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fdbfff640 1 --2- 172.21.15.60:0/360717002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2fdc159590 0x7f2fdc0a8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fe1395640 1 --2- 172.21.15.60:0/360717002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fdc0b0060 0x7f2fdc0b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fe0b94640 1 -- 172.21.15.60:0/360717002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2fdc159590 msgr2=0x7f2fdc0a8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fe0b94640 1 --2- 172.21.15.60:0/360717002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2fdc159590 0x7f2fdc0a8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fe0b94640 1 -- 172.21.15.60:0/360717002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fdc0b0060 msgr2=0x7f2fdc0b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fe0b94640 1 --2- 172.21.15.60:0/360717002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fdc0b0060 0x7f2fdc0b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fe0b94640 1 -- 172.21.15.60:0/360717002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc144690 con 0x7f2fcc00b570 2024-04-25T10:10:14.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fe0b94640 1 --2- 172.21.15.60:0/360717002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fcc00b570 0x7f2fdc0ae860 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000d570 tx=0x7f2fd000da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fd97fa640 1 -- 172.21.15.60:0/360717002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0005480 con 0x7f2fcc00b570 2024-04-25T10:10:14.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fd97fa640 1 -- 172.21.15.60:0/360717002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd0017070 con 0x7f2fcc00b570 2024-04-25T10:10:14.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fd97fa640 1 -- 172.21.15.60:0/360717002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0003840 con 0x7f2fcc00b570 2024-04-25T10:10:14.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fe2df8640 1 -- 172.21.15.60:0/360717002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc144c00 con 0x7f2fcc00b570 2024-04-25T10:10:14.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fe2df8640 1 -- 172.21.15.60:0/360717002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc0aeda0 con 0x7f2fcc00b570 2024-04-25T10:10:14.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fe2df8640 1 -- 172.21.15.60:0/360717002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fdc0a89d0 con 0x7f2fcc00b570 2024-04-25T10:10:14.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fd97fa640 1 -- 172.21.15.60:0/360717002 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2fd00039e0 con 0x7f2fcc00b570 2024-04-25T10:10:14.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fd97fa640 1 --2- 172.21.15.60:0/360717002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2fc0066f90 0x7f2fc0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fd97fa640 1 -- 172.21.15.60:0/360717002 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2fd0084f00 con 0x7f2fcc00b570 2024-04-25T10:10:14.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.417+0000 7f2fdbfff640 1 --2- 172.21.15.60:0/360717002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2fc0066f90 0x7f2fc0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.421+0000 7f2fdbfff640 1 --2- 172.21.15.60:0/360717002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2fc0066f90 0x7f2fc0069450 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc156280 tx=0x7f2fc8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.421+0000 7f2fd97fa640 1 -- 172.21.15.60:0/360717002 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2fd0019310 con 0x7f2fcc00b570 2024-04-25T10:10:14.560 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:10:14.553+0000 7f3a7badb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:10:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.585+0000 7f2fe2df8640 1 -- 172.21.15.60:0/360717002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2fdc074770 con 0x7f2fcc00b570 2024-04-25T10:10:14.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.585+0000 7f2fd97fa640 1 -- 172.21.15.60:0/360717002 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2fd0055e70 con 0x7f2fcc00b570 2024-04-25T10:10:14.594 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:10:14.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.589+0000 7f2fe2df8640 1 -- 172.21.15.60:0/360717002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2fc0066f90 msgr2=0x7f2fc0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.589+0000 7f2fe2df8640 1 --2- 172.21.15.60:0/360717002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2fc0066f90 0x7f2fc0069450 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc156280 tx=0x7f2fc8008040 comp rx=0 tx=0).stop 2024-04-25T10:10:14.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.589+0000 7f2fe2df8640 1 -- 172.21.15.60:0/360717002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fcc00b570 msgr2=0x7f2fdc0ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.589+0000 7f2fe2df8640 1 --2- 172.21.15.60:0/360717002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fcc00b570 0x7f2fdc0ae860 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000d570 tx=0x7f2fd000da30 comp rx=0 tx=0).stop 2024-04-25T10:10:14.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.589+0000 7f2fe2df8640 1 -- 172.21.15.60:0/360717002 shutdown_connections 2024-04-25T10:10:14.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.589+0000 7f2fe2df8640 1 --2- 172.21.15.60:0/360717002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2fc0066f90 0x7f2fc0069450 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.589+0000 7f2fe2df8640 1 --2- 172.21.15.60:0/360717002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fdc0b0060 0x7f2fdc0b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.589+0000 7f2fe2df8640 1 --2- 172.21.15.60:0/360717002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2fdc159590 0x7f2fdc0a8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.589+0000 7f2fe2df8640 1 --2- 172.21.15.60:0/360717002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2fcc00b570 0x7f2fdc0ae860 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.589+0000 7f2fe2df8640 1 -- 172.21.15.60:0/360717002 >> 172.21.15.60:0/360717002 conn(0x7f2fdc0670b0 msgr2=0x7f2fdc153a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:14.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.589+0000 7f2fe2df8640 1 -- 172.21.15.60:0/360717002 shutdown_connections 2024-04-25T10:10:14.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.589+0000 7f2fe2df8640 1 -- 172.21.15.60:0/360717002 wait complete. 2024-04-25T10:10:14.612 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:10:14.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472f9d5640 1 Processor -- start 2024-04-25T10:10:14.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472f9d5640 1 -- start start 2024-04-25T10:10:14.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472f9d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f472815d8a0 0x7f47280ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472f9d5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f472815dc80 0x7f47280b5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472f9d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47280afd60 0x7f47280b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472f9d5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4728070b30 con 0x7f472815dc80 2024-04-25T10:10:14.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472f9d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4728161790 con 0x7f472815d8a0 2024-04-25T10:10:14.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472f9d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4728161a70 con 0x7f47280afd60 2024-04-25T10:10:14.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472cf70640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f472815dc80 0x7f47280b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472cf70640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f472815dc80 0x7f47280b5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48126/0 (socket says 172.21.15.60:48126) 2024-04-25T10:10:14.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472cf70640 1 -- 172.21.15.60:0/257895353 learned_addr learned my addr 172.21.15.60:0/257895353 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:14.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472d771640 1 --2- 172.21.15.60:0/257895353 >> v2:172.21.15.100:3300/0 conn(0x7f472815d8a0 0x7f47280ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472df72640 1 --2- 172.21.15.60:0/257895353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47280afd60 0x7f47280b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472cf70640 1 -- 172.21.15.60:0/257895353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47280afd60 msgr2=0x7f47280b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:14.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472cf70640 1 --2- 172.21.15.60:0/257895353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47280afd60 0x7f47280b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472cf70640 1 -- 172.21.15.60:0/257895353 >> v2:172.21.15.100:3300/0 conn(0x7f472815d8a0 msgr2=0x7f47280ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:14.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472cf70640 1 --2- 172.21.15.60:0/257895353 >> v2:172.21.15.100:3300/0 conn(0x7f472815d8a0 0x7f47280ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472cf70640 1 -- 172.21.15.60:0/257895353 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4728070800 con 0x7f472815dc80 2024-04-25T10:10:14.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.733+0000 7f472df72640 1 --2- 172.21.15.60:0/257895353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47280afd60 0x7f47280b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:14.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.737+0000 7f472cf70640 1 --2- 172.21.15.60:0/257895353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f472815dc80 0x7f47280b5160 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f472806f080 tx=0x7f471801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21571548b310e5f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.737+0000 7f471ffff640 1 -- 172.21.15.60:0/257895353 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47180b0070 con 0x7f472815dc80 2024-04-25T10:10:14.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.737+0000 7f471ffff640 1 -- 172.21.15.60:0/257895353 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47180ab600 con 0x7f472815dc80 2024-04-25T10:10:14.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.737+0000 7f471ffff640 1 -- 172.21.15.60:0/257895353 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47180ab9d0 con 0x7f472815dc80 2024-04-25T10:10:14.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.737+0000 7f472f9d5640 1 -- 172.21.15.60:0/257895353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f472815dc80 msgr2=0x7f47280b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.737+0000 7f472f9d5640 1 --2- 172.21.15.60:0/257895353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f472815dc80 0x7f47280b5160 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f472806f080 tx=0x7f471801c3d0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.737+0000 7f472f9d5640 1 -- 172.21.15.60:0/257895353 shutdown_connections 2024-04-25T10:10:14.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.737+0000 7f472f9d5640 1 --2- 172.21.15.60:0/257895353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47280afd60 0x7f47280b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.737+0000 7f472f9d5640 1 --2- 172.21.15.60:0/257895353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f472815dc80 0x7f47280b5160 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.737+0000 7f472f9d5640 1 --2- 172.21.15.60:0/257895353 >> v2:172.21.15.100:3300/0 conn(0x7f472815d8a0 0x7f47280ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.737+0000 7f472f9d5640 1 -- 172.21.15.60:0/257895353 >> 172.21.15.60:0/257895353 conn(0x7f472806e400 msgr2=0x7f472806e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:14.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.737+0000 7f472f9d5640 1 -- 172.21.15.60:0/257895353 shutdown_connections 2024-04-25T10:10:14.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.737+0000 7f472f9d5640 1 -- 172.21.15.60:0/257895353 wait complete. 2024-04-25T10:10:14.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472f9d5640 1 Processor -- start 2024-04-25T10:10:14.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472f9d5640 1 -- start start 2024-04-25T10:10:14.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472f9d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47280afd60 0x7f4728157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472f9d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f472815d8a0 0x7f4728155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472f9d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4728151270 0x7f47280cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472f9d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4728072f10 con 0x7f472815d8a0 2024-04-25T10:10:14.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472f9d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4728072d90 con 0x7f4728151270 2024-04-25T10:10:14.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472f9d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4728073090 con 0x7f47280afd60 2024-04-25T10:10:14.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472cf70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f472815d8a0 0x7f4728155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472cf70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f472815d8a0 0x7f4728155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48132/0 (socket says 172.21.15.60:48132) 2024-04-25T10:10:14.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472cf70640 1 -- 172.21.15.60:0/1541232520 learned_addr learned my addr 172.21.15.60:0/1541232520 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:14.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472df72640 1 --2- 172.21.15.60:0/1541232520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4728151270 0x7f47280cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472d771640 1 --2- 172.21.15.60:0/1541232520 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47280afd60 0x7f4728157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472cf70640 1 -- 172.21.15.60:0/1541232520 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47280afd60 msgr2=0x7f4728157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472cf70640 1 --2- 172.21.15.60:0/1541232520 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47280afd60 0x7f4728157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472cf70640 1 -- 172.21.15.60:0/1541232520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4728151270 msgr2=0x7f47280cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472cf70640 1 --2- 172.21.15.60:0/1541232520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4728151270 0x7f47280cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472cf70640 1 -- 172.21.15.60:0/1541232520 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4728155fc0 con 0x7f472815d8a0 2024-04-25T10:10:14.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472cf70640 1 --2- 172.21.15.60:0/1541232520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f472815d8a0 0x7f4728155a80 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f47180ae040 tx=0x7f4718019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f471dffb640 1 -- 172.21.15.60:0/1541232520 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47180b0040 con 0x7f472815d8a0 2024-04-25T10:10:14.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f471dffb640 1 -- 172.21.15.60:0/1541232520 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47180b9070 con 0x7f472815d8a0 2024-04-25T10:10:14.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f471dffb640 1 -- 172.21.15.60:0/1541232520 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47180aa970 con 0x7f472815d8a0 2024-04-25T10:10:14.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472f9d5640 1 -- 172.21.15.60:0/1541232520 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4728156260 con 0x7f472815d8a0 2024-04-25T10:10:14.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472f9d5640 1 -- 172.21.15.60:0/1541232520 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4728156820 con 0x7f472815d8a0 2024-04-25T10:10:14.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f472f9d5640 1 -- 172.21.15.60:0/1541232520 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4728074770 con 0x7f472815d8a0 2024-04-25T10:10:14.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.741+0000 7f471dffb640 1 -- 172.21.15.60:0/1541232520 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f47180aab10 con 0x7f472815d8a0 2024-04-25T10:10:14.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.745+0000 7f471dffb640 1 --2- 172.21.15.60:0/1541232520 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f470c066f60 0x7f470c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.745+0000 7f471dffb640 1 -- 172.21.15.60:0/1541232520 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f471811f370 con 0x7f472815d8a0 2024-04-25T10:10:14.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.745+0000 7f472d771640 1 --2- 172.21.15.60:0/1541232520 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f470c066f60 0x7f470c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.745+0000 7f472d771640 1 --2- 172.21.15.60:0/1541232520 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f470c066f60 0x7f470c069420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4724003fd0 tx=0x7f47240039a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.749+0000 7f471dffb640 1 -- 172.21.15.60:0/1541232520 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f47180a4e20 con 0x7f472815d8a0 2024-04-25T10:10:14.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.909+0000 7f472f9d5640 1 -- 172.21.15.60:0/1541232520 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47280700d0 con 0x7f472815d8a0 2024-04-25T10:10:14.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.909+0000 7f471dffb640 1 -- 172.21.15.60:0/1541232520 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4718016110 con 0x7f472815d8a0 2024-04-25T10:10:14.917 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:10:14.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.909+0000 7f472f9d5640 1 -- 172.21.15.60:0/1541232520 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f470c066f60 msgr2=0x7f470c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.909+0000 7f472f9d5640 1 --2- 172.21.15.60:0/1541232520 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f470c066f60 0x7f470c069420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4724003fd0 tx=0x7f47240039a0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.909+0000 7f472f9d5640 1 -- 172.21.15.60:0/1541232520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f472815d8a0 msgr2=0x7f4728155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.909+0000 7f472f9d5640 1 --2- 172.21.15.60:0/1541232520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f472815d8a0 0x7f4728155a80 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f47180ae040 tx=0x7f4718019170 comp rx=0 tx=0).stop 2024-04-25T10:10:14.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.909+0000 7f472f9d5640 1 -- 172.21.15.60:0/1541232520 shutdown_connections 2024-04-25T10:10:14.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.909+0000 7f472f9d5640 1 --2- 172.21.15.60:0/1541232520 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f470c066f60 0x7f470c069420 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.909+0000 7f472f9d5640 1 --2- 172.21.15.60:0/1541232520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4728151270 0x7f47280cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.909+0000 7f472f9d5640 1 --2- 172.21.15.60:0/1541232520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f472815d8a0 0x7f4728155a80 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.909+0000 7f472f9d5640 1 --2- 172.21.15.60:0/1541232520 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47280afd60 0x7f4728157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.909+0000 7f472f9d5640 1 -- 172.21.15.60:0/1541232520 >> 172.21.15.60:0/1541232520 conn(0x7f472806e400 msgr2=0x7f47280b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:14.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.909+0000 7f472f9d5640 1 -- 172.21.15.60:0/1541232520 shutdown_connections 2024-04-25T10:10:14.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:14.909+0000 7f472f9d5640 1 -- 172.21.15.60:0/1541232520 wait complete. 2024-04-25T10:10:14.938 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:10:15.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.057+0000 7fbdcf256640 1 Processor -- start 2024-04-25T10:10:15.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.057+0000 7fbdcf256640 1 -- start start 2024-04-25T10:10:15.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.057+0000 7fbdcf256640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbdc815f480 0x7fbdc815f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.057+0000 7fbdcf256640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdc80cd390 0x7fbdc809a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.057+0000 7fbdcf256640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc815fda0 con 0x7fbdc809a930 2024-04-25T10:10:15.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.057+0000 7fbdcf256640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc80cc5d0 con 0x7fbdc815f480 2024-04-25T10:10:15.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.057+0000 7fbdcf256640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80cc8b0 con 0x7fbdc80cd390 2024-04-25T10:10:15.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.061+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdc80cd390 0x7fbdc809a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.061+0000 7fbdccff2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbdc815f480 0x7fbdc815f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.061+0000 7fbdcd7f3640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fbdc809a930 0x7fbdc80cf380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:41528/0 (socket says 172.21.15.60:41528) 2024-04-25T10:10:15.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.061+0000 7fbdcd7f3640 1 -- 172.21.15.60:0/3040787993 learned_addr learned my addr 172.21.15.60:0/3040787993 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:10:15.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.061+0000 7fbdccff2640 1 -- 172.21.15.60:0/3040787993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdc80cd390 msgr2=0x7fbdc809a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:15.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.061+0000 7fbdccff2640 1 --2- 172.21.15.60:0/3040787993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdc80cd390 0x7fbdc809a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.061+0000 7fbdccff2640 1 -- 172.21.15.60:0/3040787993 >> v1:172.21.15.100:6789/0 conn(0x7fbdc809a930 legacy=0x7fbdc80cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.061+0000 7fbdccff2640 1 -- 172.21.15.60:0/3040787993 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc80ccb90 con 0x7fbdc815f480 2024-04-25T10:10:15.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.061+0000 7fbdc7fff640 1 --2- 172.21.15.60:0/3040787993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdc80cd390 0x7fbdc809a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:10:15.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.061+0000 7fbdccff2640 1 --2- 172.21.15.60:0/3040787993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdc815f480 0x7fbdc815f860 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009d90 tx=0x7fbdb801c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90993c22b6136956 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:15.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.061+0000 7fbdc77fe640 1 -- 172.21.15.60:0/3040787993 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80b2070 con 0x7fbdc815f480 2024-04-25T10:10:15.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.061+0000 7fbdc77fe640 1 -- 172.21.15.60:0/3040787993 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb80a59e0 con 0x7fbdc815f480 2024-04-25T10:10:15.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.061+0000 7fbdc77fe640 1 -- 172.21.15.60:0/3040787993 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80a5db0 con 0x7fbdc815f480 2024-04-25T10:10:15.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 -- 172.21.15.60:0/3040787993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdc815f480 msgr2=0x7fbdc815f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 --2- 172.21.15.60:0/3040787993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdc815f480 0x7fbdc815f860 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009d90 tx=0x7fbdb801c940 comp rx=0 tx=0).stop 2024-04-25T10:10:15.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 -- 172.21.15.60:0/3040787993 shutdown_connections 2024-04-25T10:10:15.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 --2- 172.21.15.60:0/3040787993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdc80cd390 0x7fbdc809a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 --2- 172.21.15.60:0/3040787993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdc815f480 0x7fbdc815f860 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 -- 172.21.15.60:0/3040787993 >> 172.21.15.60:0/3040787993 conn(0x7fbdc80b32a0 msgr2=0x7fbdc80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:15.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 -- 172.21.15.60:0/3040787993 shutdown_connections 2024-04-25T10:10:15.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 -- 172.21.15.60:0/3040787993 wait complete. 2024-04-25T10:10:15.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 Processor -- start 2024-04-25T10:10:15.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 -- start start 2024-04-25T10:10:15.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbdc809a930 0x7fbdc8151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdc80cd390 0x7fbdc80aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdc815f480 0x7fbdc80af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc815cd40 con 0x7fbdc809a930 2024-04-25T10:10:15.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc815cbc0 con 0x7fbdc80cd390 2024-04-25T10:10:15.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc81687c0 con 0x7fbdc815f480 2024-04-25T10:10:15.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdccff2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbdc809a930 0x7fbdc8151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdccff2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbdc809a930 0x7fbdc8151310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48146/0 (socket says 172.21.15.60:48146) 2024-04-25T10:10:15.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdccff2640 1 -- 172.21.15.60:0/890418204 learned_addr learned my addr 172.21.15.60:0/890418204 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:15.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdc7fff640 1 --2- 172.21.15.60:0/890418204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdc80cd390 0x7fbdc80aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcd7f3640 1 --2- 172.21.15.60:0/890418204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdc815f480 0x7fbdc80af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdccff2640 1 -- 172.21.15.60:0/890418204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdc815f480 msgr2=0x7fbdc80af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdccff2640 1 --2- 172.21.15.60:0/890418204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdc815f480 0x7fbdc80af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdccff2640 1 -- 172.21.15.60:0/890418204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdc80cd390 msgr2=0x7fbdc80aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdccff2640 1 --2- 172.21.15.60:0/890418204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdc80cd390 0x7fbdc80aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdccff2640 1 -- 172.21.15.60:0/890418204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc8151850 con 0x7fbdc809a930 2024-04-25T10:10:15.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcd7f3640 1 --2- 172.21.15.60:0/890418204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdc815f480 0x7fbdc80af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:15.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdccff2640 1 --2- 172.21.15.60:0/890418204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbdc809a930 0x7fbdc8151310 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009d90 tx=0x7fbdb80196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:15.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdc57fa640 1 -- 172.21.15.60:0/890418204 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80bd020 con 0x7fbdc809a930 2024-04-25T10:10:15.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdc57fa640 1 -- 172.21.15.60:0/890418204 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb80ad6c0 con 0x7fbdc809a930 2024-04-25T10:10:15.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdc57fa640 1 -- 172.21.15.60:0/890418204 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80b2040 con 0x7fbdc809a930 2024-04-25T10:10:15.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 -- 172.21.15.60:0/890418204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc806bf30 con 0x7fbdc809a930 2024-04-25T10:10:15.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.065+0000 7fbdcf256640 1 -- 172.21.15.60:0/890418204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc806c4f0 con 0x7fbdc809a930 2024-04-25T10:10:15.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.069+0000 7fbdcf256640 1 -- 172.21.15.60:0/890418204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc8074770 con 0x7fbdc809a930 2024-04-25T10:10:15.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.069+0000 7fbdc57fa640 1 -- 172.21.15.60:0/890418204 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbdb80c0020 con 0x7fbdc809a930 2024-04-25T10:10:15.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.069+0000 7fbdc57fa640 1 --2- 172.21.15.60:0/890418204 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbdb4066f60 0x7fbdb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.069+0000 7fbdc57fa640 1 -- 172.21.15.60:0/890418204 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fbdb8120070 con 0x7fbdc809a930 2024-04-25T10:10:15.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.069+0000 7fbdc7fff640 1 --2- 172.21.15.60:0/890418204 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbdb4066f60 0x7fbdb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.069+0000 7fbdc7fff640 1 --2- 172.21.15.60:0/890418204 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbdb4066f60 0x7fbdb4069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009580 tx=0x7fbdbc009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:15.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.073+0000 7fbdc57fa640 1 -- 172.21.15.60:0/890418204 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbdb80c7030 con 0x7fbdc809a930 2024-04-25T10:10:15.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.241+0000 7fbdcf256640 1 -- 172.21.15.60:0/890418204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbdc80b40a0 con 0x7fbdc809a930 2024-04-25T10:10:15.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.245+0000 7fbdc57fa640 1 -- 172.21.15.60:0/890418204 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v71) v1 ==== 80+0+122 (secure 0 0 0) 0x7fbdb8016560 con 0x7fbdc809a930 2024-04-25T10:10:15.252 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6835/1289503429 2024-04-26T10:10:03.449620+0000 2024-04-25T10:10:15.253 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6833/1289503429 2024-04-26T10:10:03.449620+0000 2024-04-25T10:10:15.253 INFO:teuthology.orchestra.run.smithi060.stderr:listed 2 entries 2024-04-25T10:10:15.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.245+0000 7fbdcf256640 1 -- 172.21.15.60:0/890418204 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbdb4066f60 msgr2=0x7fbdb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.245+0000 7fbdcf256640 1 --2- 172.21.15.60:0/890418204 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbdb4066f60 0x7fbdb4069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009580 tx=0x7fbdbc009190 comp rx=0 tx=0).stop 2024-04-25T10:10:15.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.245+0000 7fbdcf256640 1 -- 172.21.15.60:0/890418204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbdc809a930 msgr2=0x7fbdc8151310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.245+0000 7fbdcf256640 1 --2- 172.21.15.60:0/890418204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbdc809a930 0x7fbdc8151310 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009d90 tx=0x7fbdb80196e0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.245+0000 7fbdcf256640 1 -- 172.21.15.60:0/890418204 shutdown_connections 2024-04-25T10:10:15.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.245+0000 7fbdcf256640 1 --2- 172.21.15.60:0/890418204 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbdb4066f60 0x7fbdb4069420 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.245+0000 7fbdcf256640 1 --2- 172.21.15.60:0/890418204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdc815f480 0x7fbdc80af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.245+0000 7fbdcf256640 1 --2- 172.21.15.60:0/890418204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdc80cd390 0x7fbdc80aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.245+0000 7fbdcf256640 1 --2- 172.21.15.60:0/890418204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbdc809a930 0x7fbdc8151310 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.245+0000 7fbdcf256640 1 -- 172.21.15.60:0/890418204 >> 172.21.15.60:0/890418204 conn(0x7fbdc80b32a0 msgr2=0x7fbdc80b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:15.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.245+0000 7fbdcf256640 1 -- 172.21.15.60:0/890418204 shutdown_connections 2024-04-25T10:10:15.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:15.245+0000 7fbdcf256640 1 -- 172.21.15.60:0/890418204 wait complete. 2024-04-25T10:10:15.272 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:10:15.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:15.272 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:15.284 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[43052]: fuse finished with error 0 and tester_r 0 2024-04-25T10:10:15.285 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:10:16.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:16.876 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:21.387 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:10:21.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:21.388 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:10:21.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:21.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-04-25T10:10:21.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.569+0000 7f20bc02d640 1 Processor -- start 2024-04-25T10:10:21.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.569+0000 7f20bc02d640 1 -- start start 2024-04-25T10:10:21.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.569+0000 7f20bc02d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f20b4071980 0x7f20b40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.569+0000 7f20bc02d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b40cf0a0 0x7f20b40b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.569+0000 7f20bc02d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b40cf810 con 0x7f20b4144e40 2024-04-25T10:10:21.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.569+0000 7f20bc02d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b40b3060 con 0x7f20b4071980 2024-04-25T10:10:21.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.569+0000 7f20bc02d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40cfaf0 con 0x7f20b40cf0a0 2024-04-25T10:10:21.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.573+0000 7f20b95c8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f20b4071980 0x7f20b40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.573+0000 7f20b95c8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f20b4071980 0x7f20b40ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51148/0 (socket says 172.21.15.60:51148) 2024-04-25T10:10:21.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.573+0000 7f20b95c8640 1 -- 172.21.15.60:0/3024285300 learned_addr learned my addr 172.21.15.60:0/3024285300 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:21.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.573+0000 7f20b9dc9640 1 --1- 172.21.15.60:0/3024285300 >> v1:172.21.15.100:6789/0 conn(0x7f20b4144e40 0x7f20b4071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:44388/0 (socket says 172.21.15.60:44388) 2024-04-25T10:10:21.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.573+0000 7f20ba5ca640 1 --2- 172.21.15.60:0/3024285300 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b40cf0a0 0x7f20b40b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.573+0000 7f20ba5ca640 1 -- 172.21.15.60:0/3024285300 >> v2:172.21.15.100:3300/0 conn(0x7f20b4071980 msgr2=0x7f20b40ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.573+0000 7f20ba5ca640 1 --2- 172.21.15.60:0/3024285300 >> v2:172.21.15.100:3300/0 conn(0x7f20b4071980 0x7f20b40ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.573+0000 7f20ba5ca640 1 -- 172.21.15.60:0/3024285300 >> v1:172.21.15.100:6789/0 conn(0x7f20b4144e40 legacy=0x7f20b4071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.573+0000 7f20ba5ca640 1 -- 172.21.15.60:0/3024285300 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b40cfdd0 con 0x7f20b40cf0a0 2024-04-25T10:10:21.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.573+0000 7f20ba5ca640 1 --2- 172.21.15.60:0/3024285300 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b40cf0a0 0x7f20b40b34f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009d90 tx=0x7f20ac01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e719b6f8f6140e2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.573+0000 7f20b8dc7640 1 -- 172.21.15.60:0/3024285300 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac0b2070 con 0x7f20b40cf0a0 2024-04-25T10:10:21.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20b8dc7640 1 -- 172.21.15.60:0/3024285300 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b4145430 con 0x7f20b40cf0a0 2024-04-25T10:10:21.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20b8dc7640 1 -- 172.21.15.60:0/3024285300 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac0a5c40 con 0x7f20b40cf0a0 2024-04-25T10:10:21.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 -- 172.21.15.60:0/3024285300 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b40cf0a0 msgr2=0x7f20b40b34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 --2- 172.21.15.60:0/3024285300 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b40cf0a0 0x7f20b40b34f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009d90 tx=0x7f20ac01c940 comp rx=0 tx=0).stop 2024-04-25T10:10:21.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 -- 172.21.15.60:0/3024285300 shutdown_connections 2024-04-25T10:10:21.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 --2- 172.21.15.60:0/3024285300 >> v2:172.21.15.100:3300/0 conn(0x7f20b4071980 0x7f20b40ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 --2- 172.21.15.60:0/3024285300 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b40cf0a0 0x7f20b40b34f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 -- 172.21.15.60:0/3024285300 >> 172.21.15.60:0/3024285300 conn(0x7f20b4066fa0 msgr2=0x7f20b406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 -- 172.21.15.60:0/3024285300 shutdown_connections 2024-04-25T10:10:21.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 -- 172.21.15.60:0/3024285300 wait complete. 2024-04-25T10:10:21.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 Processor -- start 2024-04-25T10:10:21.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 -- start start 2024-04-25T10:10:21.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b4071980 0x7f20b415acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20b40cf0a0 0x7f20b40af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b4144e40 0x7f20b40af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40b45d0 con 0x7f20b40cf0a0 2024-04-25T10:10:21.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40b4450 con 0x7f20b4144e40 2024-04-25T10:10:21.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40b4750 con 0x7f20b4071980 2024-04-25T10:10:21.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20b95c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20b40cf0a0 0x7f20b40af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20b95c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20b40cf0a0 0x7f20b40af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41164/0 (socket says 172.21.15.60:41164) 2024-04-25T10:10:21.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20b95c8640 1 -- 172.21.15.60:0/3927600934 learned_addr learned my addr 172.21.15.60:0/3927600934 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:21.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20ba5ca640 1 --2- 172.21.15.60:0/3927600934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b4144e40 0x7f20b40af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20b9dc9640 1 --2- 172.21.15.60:0/3927600934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b4071980 0x7f20b415acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20b95c8640 1 -- 172.21.15.60:0/3927600934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b4071980 msgr2=0x7f20b415acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20b95c8640 1 --2- 172.21.15.60:0/3927600934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b4071980 0x7f20b415acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20b95c8640 1 -- 172.21.15.60:0/3927600934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b4144e40 msgr2=0x7f20b40af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20b95c8640 1 --2- 172.21.15.60:0/3927600934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b4144e40 0x7f20b40af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20b95c8640 1 -- 172.21.15.60:0/3927600934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b415b230 con 0x7f20b40cf0a0 2024-04-25T10:10:21.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20ba5ca640 1 --2- 172.21.15.60:0/3927600934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b4144e40 0x7f20b40af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:21.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20b9dc9640 1 --2- 172.21.15.60:0/3927600934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b4071980 0x7f20b415acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:21.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20b95c8640 1 --2- 172.21.15.60:0/3927600934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20b40cf0a0 0x7f20b40af240 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f20a400e7f0 tx=0x7f20a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20aa7fc640 1 -- 172.21.15.60:0/3927600934 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4017070 con 0x7f20b40cf0a0 2024-04-25T10:10:21.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20aa7fc640 1 -- 172.21.15.60:0/3927600934 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a400c940 con 0x7f20b40cf0a0 2024-04-25T10:10:21.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20bc02d640 1 -- 172.21.15.60:0/3927600934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b406dec0 con 0x7f20b40cf0a0 2024-04-25T10:10:21.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.577+0000 7f20aa7fc640 1 -- 172.21.15.60:0/3927600934 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a400cd30 con 0x7f20b40cf0a0 2024-04-25T10:10:21.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20bc02d640 1 -- 172.21.15.60:0/3927600934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b406e480 con 0x7f20b40cf0a0 2024-04-25T10:10:21.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20aa7fc640 1 -- 172.21.15.60:0/3927600934 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20a401a070 con 0x7f20b40cf0a0 2024-04-25T10:10:21.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20aa7fc640 1 --2- 172.21.15.60:0/3927600934 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2098066f60 0x7f2098069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20aa7fc640 1 -- 172.21.15.60:0/3927600934 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f20a4084ff0 con 0x7f20b40cf0a0 2024-04-25T10:10:21.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20b9dc9640 1 --2- 172.21.15.60:0/3927600934 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2098066f60 0x7f2098069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20bf19a000 1 Processor -- start 2024-04-25T10:10:21.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20b9dc9640 1 --2- 172.21.15.60:0/3927600934 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2098066f60 0x7f2098069420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f20b0000f70 tx=0x7f20b0007d70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20bf19a000 1 -- start start 2024-04-25T10:10:21.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20bf19a000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa7d8c0 0x560b7aa7dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20bf19a000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7e1e0 0x560b7aa33110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20bf19a000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 0x560b7aa33a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20bf19a000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b7a881dd0 con 0x560b7aa7e1e0 2024-04-25T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20bf19a000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b7a80a010 con 0x560b7aa33650 2024-04-25T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20bf19a000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b7a87de70 con 0x560b7aa7d8c0 2024-04-25T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20b95c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7e1e0 0x560b7aa33110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20b95c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7e1e0 0x560b7aa33110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41172/0 (socket says 172.21.15.60:41172) 2024-04-25T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.581+0000 7f20b95c8640 1 -- 172.21.15.60:0/1646042922 learned_addr learned my addr 172.21.15.60:0/1646042922 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b9dc9640 1 --2- 172.21.15.60:0/1646042922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa7d8c0 0x560b7aa7dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20ba5ca640 1 --2- 172.21.15.60:0/1646042922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 0x560b7aa33a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b95c8640 1 -- 172.21.15.60:0/1646042922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa7d8c0 msgr2=0x560b7aa7dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b95c8640 1 --2- 172.21.15.60:0/1646042922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa7d8c0 0x560b7aa7dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b95c8640 1 -- 172.21.15.60:0/1646042922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 msgr2=0x560b7aa33a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b95c8640 1 --2- 172.21.15.60:0/1646042922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 0x560b7aa33a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b95c8640 1 -- 172.21.15.60:0/1646042922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b7a83a9c0 con 0x560b7aa7e1e0 2024-04-25T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20ba5ca640 1 --2- 172.21.15.60:0/1646042922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 0x560b7aa33a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b95c8640 1 --2- 172.21.15.60:0/1646042922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7e1e0 0x560b7aa33110 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f20a40096b0 tx=0x7f20a4055ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88780ec2e281aee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20977fe640 1 -- 172.21.15.60:0/1646042922 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4021040 con 0x560b7aa7e1e0 2024-04-25T10:10:21.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20977fe640 1 -- 172.21.15.60:0/1646042922 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a405df00 con 0x560b7aa7e1e0 2024-04-25T10:10:21.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20977fe640 1 -- 172.21.15.60:0/1646042922 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4051e60 con 0x560b7aa7e1e0 2024-04-25T10:10:21.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 -- 172.21.15.60:0/1646042922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7e1e0 msgr2=0x560b7aa33110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 --2- 172.21.15.60:0/1646042922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7e1e0 0x560b7aa33110 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f20a40096b0 tx=0x7f20a4055ef0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 -- 172.21.15.60:0/1646042922 shutdown_connections 2024-04-25T10:10:21.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 --2- 172.21.15.60:0/1646042922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 0x560b7aa33a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 --2- 172.21.15.60:0/1646042922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7e1e0 0x560b7aa33110 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 --2- 172.21.15.60:0/1646042922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa7d8c0 0x560b7aa7dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 -- 172.21.15.60:0/1646042922 >> 172.21.15.60:0/1646042922 conn(0x560b7a91b870 msgr2=0x560b7aa319f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 -- 172.21.15.60:0/1646042922 shutdown_connections 2024-04-25T10:10:21.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 -- 172.21.15.60:0/1646042922 wait complete. 2024-04-25T10:10:21.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 -- start start 2024-04-25T10:10:21.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 Processor -- start 2024-04-25T10:10:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 0x560b7aa37fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7d8c0 0x560b7aa384e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa7e1e0 0x560b7aa38a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b7a83c420 con 0x560b7aa7d8c0 2024-04-25T10:10:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b7aa3cc20 con 0x560b7aa33650 2024-04-25T10:10:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b7a886400 con 0x560b7aa7e1e0 2024-04-25T10:10:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 10 client.0 _ll_register_callbacks cb 0x560b7aa1ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b95c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7d8c0 0x560b7aa384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b95c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7d8c0 0x560b7aa384e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41184/0 (socket says 172.21.15.60:41184) 2024-04-25T10:10:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b95c8640 1 -- 172.21.15.60:0/4060960337 learned_addr learned my addr 172.21.15.60:0/4060960337 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b9dc9640 1 --2- 172.21.15.60:0/4060960337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 0x560b7aa37fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20ba5ca640 1 --2- 172.21.15.60:0/4060960337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa7e1e0 0x560b7aa38a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b95c8640 1 -- 172.21.15.60:0/4060960337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa7e1e0 msgr2=0x560b7aa38a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b95c8640 1 --2- 172.21.15.60:0/4060960337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa7e1e0 0x560b7aa38a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b95c8640 1 -- 172.21.15.60:0/4060960337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 msgr2=0x560b7aa37fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b95c8640 1 --2- 172.21.15.60:0/4060960337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 0x560b7aa37fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b95c8640 1 -- 172.21.15.60:0/4060960337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b7a80a260 con 0x560b7aa7d8c0 2024-04-25T10:10:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20ba5ca640 1 --2- 172.21.15.60:0/4060960337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa7e1e0 0x560b7aa38a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20b95c8640 1 --2- 172.21.15.60:0/4060960337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7d8c0 0x560b7aa384e0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f20a40560a0 tx=0x7f20a401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672c328ff0a7ffb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f2094ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:10:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f2094ff9640 1 -- 172.21.15.60:0/4060960337 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4081020 con 0x560b7aa7d8c0 2024-04-25T10:10:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 -- 172.21.15.60:0/4060960337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20a405df00 con 0x560b7aa7d8c0 2024-04-25T10:10:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f2094ff9640 1 -- 172.21.15.60:0/4060960337 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f20a4054c40 con 0x560b7aa7d8c0 2024-04-25T10:10:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f2094ff9640 1 -- 172.21.15.60:0/4060960337 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20a4021040 con 0x560b7aa7d8c0 2024-04-25T10:10:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 10 client.7272 fetch_fsmap learned FSMap version 46 2024-04-25T10:10:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.585+0000 7f20bf19a000 1 -- 172.21.15.60:0/4060960337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x560b7a83cb30 con 0x560b7aa7d8c0 2024-04-25T10:10:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f2094ff9640 1 -- 172.21.15.60:0/4060960337 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a405df00 con 0x560b7aa7d8c0 2024-04-25T10:10:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f2094ff9640 1 -- 172.21.15.60:0/4060960337 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f20a4024060 con 0x560b7aa7d8c0 2024-04-25T10:10:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 10 client.7272 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:10:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 10 client.7272 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:10:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 20 client.7272 populate_metadata read hostname 'smithi060' 2024-04-25T10:10:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 1 --2- 172.21.15.60:0/4060960337 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x560b7aa39430 0x560b7aa8e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 4 client.7272 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 1 -- 172.21.15.60:0/4060960337 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b7a81dc80 con 0x560b7aa39430 2024-04-25T10:10:21.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20b9dc9640 1 --2- 172.21.15.60:0/4060960337 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x560b7aa39430 0x560b7aa8e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f2094ff9640 10 client.7272 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20b9dc9640 1 --2- 172.21.15.60:0/4060960337 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x560b7aa39430 0x560b7aa8e960 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb8c1d3b6978b344 server_cookie=94e3dc8723b9384a in_seq=0 out_seq=0 2024-04-25T10:10:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f2094ff9640 1 -- 172.21.15.60:0/4060960337 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560b7a81dc80 con 0x560b7aa39430 2024-04-25T10:10:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f2094ff9640 10 client.7272 handle_command_reply: tid=0 2024-04-25T10:10:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 1 client.7272 shutdown 2024-04-25T10:10:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 1 -- 172.21.15.60:0/4060960337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7d8c0 msgr2=0x560b7aa384e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 1 --2- 172.21.15.60:0/4060960337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7d8c0 0x560b7aa384e0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f20a40560a0 tx=0x7f20a401d000 comp rx=0 tx=0).stop 2024-04-25T10:10:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 1 -- 172.21.15.60:0/4060960337 shutdown_connections 2024-04-25T10:10:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 1 --2- 172.21.15.60:0/4060960337 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x560b7aa39430 0x560b7aa8e960 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 1 --2- 172.21.15.60:0/4060960337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa7e1e0 0x560b7aa38a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 1 --2- 172.21.15.60:0/4060960337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7d8c0 0x560b7aa384e0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 1 --2- 172.21.15.60:0/4060960337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 0x560b7aa37fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f2094ff9640 0 client.7272 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 1 -- 172.21.15.60:0/4060960337 >> 172.21.15.60:0/4060960337 conn(0x560b7a91b870 msgr2=0x560b7aa319f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 1 -- 172.21.15.60:0/4060960337 shutdown_connections 2024-04-25T10:10:21.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 1 -- 172.21.15.60:0/4060960337 wait complete. 2024-04-25T10:10:21.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.589+0000 7f20bf19a000 20 client.7272 trim_cache size 0 max 16384 2024-04-25T10:10:21.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 Processor -- start 2024-04-25T10:10:21.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 -- start start 2024-04-25T10:10:21.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 0x560b7aa7c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa7d8c0 0x560b7aa7ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7e1e0 0x560b7aa8d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4202b10 con 0x560b7aa7e1e0 2024-04-25T10:10:21.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40ade60 con 0x560b7aa33650 2024-04-25T10:10:21.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b41f7cf0 con 0x560b7aa7d8c0 2024-04-25T10:10:21.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20ba5ca640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7e1e0 0x560b7aa8d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20ba5ca640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7e1e0 0x560b7aa8d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41194/0 (socket says 172.21.15.60:41194) 2024-04-25T10:10:21.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20ba5ca640 1 -- 172.21.15.60:0/3325329407 learned_addr learned my addr 172.21.15.60:0/3325329407 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:21.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20b9dc9640 1 --2- 172.21.15.60:0/3325329407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 0x560b7aa7c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20b95c8640 1 --2- 172.21.15.60:0/3325329407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa7d8c0 0x560b7aa7ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20ba5ca640 1 -- 172.21.15.60:0/3325329407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa7d8c0 msgr2=0x560b7aa7ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20ba5ca640 1 --2- 172.21.15.60:0/3325329407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa7d8c0 0x560b7aa7ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20ba5ca640 1 -- 172.21.15.60:0/3325329407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 msgr2=0x560b7aa7c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20ba5ca640 1 --2- 172.21.15.60:0/3325329407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 0x560b7aa7c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20ba5ca640 1 -- 172.21.15.60:0/3325329407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b7a9076d0 con 0x560b7aa7e1e0 2024-04-25T10:10:21.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20b9dc9640 1 --2- 172.21.15.60:0/3325329407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 0x560b7aa7c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:21.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20ba5ca640 1 --2- 172.21.15.60:0/3325329407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7e1e0 0x560b7aa8d960 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f20ac01a570 tx=0x7f20ac004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447cb00f1a23de2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f2096ffd640 1 -- 172.21.15.60:0/3325329407 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac0b2040 con 0x560b7aa7e1e0 2024-04-25T10:10:21.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f2096ffd640 1 -- 172.21.15.60:0/3325329407 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ac0a5b20 con 0x560b7aa7e1e0 2024-04-25T10:10:21.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f2096ffd640 1 -- 172.21.15.60:0/3325329407 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac0ad440 con 0x560b7aa7e1e0 2024-04-25T10:10:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 -- 172.21.15.60:0/3325329407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7e1e0 msgr2=0x560b7aa8d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 --2- 172.21.15.60:0/3325329407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7e1e0 0x560b7aa8d960 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f20ac01a570 tx=0x7f20ac004ac0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 -- 172.21.15.60:0/3325329407 shutdown_connections 2024-04-25T10:10:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 --2- 172.21.15.60:0/3325329407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7e1e0 0x560b7aa8d960 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 --2- 172.21.15.60:0/3325329407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa7d8c0 0x560b7aa7ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 --2- 172.21.15.60:0/3325329407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa33650 0x560b7aa7c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 -- 172.21.15.60:0/3325329407 >> 172.21.15.60:0/3325329407 conn(0x560b7a91b870 msgr2=0x560b7aa7c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 -- 172.21.15.60:0/3325329407 shutdown_connections 2024-04-25T10:10:21.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.605+0000 7f20bf19a000 1 -- 172.21.15.60:0/3325329407 wait complete. 2024-04-25T10:10:21.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 -- start start 2024-04-25T10:10:21.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 Processor -- start 2024-04-25T10:10:21.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa33650 0x560b7aadde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7d8c0 0x560b7aade390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa7e1e0 0x560b7aa983e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b7a8e4000 con 0x560b7aa7d8c0 2024-04-25T10:10:21.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40ad860 con 0x560b7aa7e1e0 2024-04-25T10:10:21.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b7a8810e0 con 0x560b7aa33650 2024-04-25T10:10:21.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 10 client.0 _ll_register_callbacks cb 0x560b7a81e200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:21.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20b95c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7d8c0 0x560b7aade390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20b95c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7d8c0 0x560b7aade390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41208/0 (socket says 172.21.15.60:41208) 2024-04-25T10:10:21.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20b95c8640 1 -- 172.21.15.60:0/3632718838 learned_addr learned my addr 172.21.15.60:0/3632718838 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:21.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20ba5ca640 1 --2- 172.21.15.60:0/3632718838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa7e1e0 0x560b7aa983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20b9dc9640 1 --2- 172.21.15.60:0/3632718838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa33650 0x560b7aadde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20b95c8640 1 -- 172.21.15.60:0/3632718838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa33650 msgr2=0x560b7aadde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20b95c8640 1 --2- 172.21.15.60:0/3632718838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa33650 0x560b7aadde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20b95c8640 1 -- 172.21.15.60:0/3632718838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa7e1e0 msgr2=0x560b7aa983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20b95c8640 1 --2- 172.21.15.60:0/3632718838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa7e1e0 0x560b7aa983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20b95c8640 1 -- 172.21.15.60:0/3632718838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b7a904900 con 0x560b7aa7d8c0 2024-04-25T10:10:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20b95c8640 1 --2- 172.21.15.60:0/3632718838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7d8c0 0x560b7aade390 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f20a4052170 tx=0x7f20a400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e74ee5da423f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f2094ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:10:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f2094ff9640 1 -- 172.21.15.60:0/3632718838 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4081070 con 0x560b7aa7d8c0 2024-04-25T10:10:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f2094ff9640 1 -- 172.21.15.60:0/3632718838 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f20a4054c40 con 0x560b7aa7d8c0 2024-04-25T10:10:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 -- 172.21.15.60:0/3632718838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20ac0a5b20 con 0x560b7aa7d8c0 2024-04-25T10:10:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f2094ff9640 1 -- 172.21.15.60:0/3632718838 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20a4059940 con 0x560b7aa7d8c0 2024-04-25T10:10:21.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 10 client.7278 fetch_fsmap learned FSMap version 46 2024-04-25T10:10:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 -- 172.21.15.60:0/3632718838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x560b7a828f20 con 0x560b7aa7d8c0 2024-04-25T10:10:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f2094ff9640 1 -- 172.21.15.60:0/3632718838 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ac0a5b20 con 0x560b7aa7d8c0 2024-04-25T10:10:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f2094ff9640 1 -- 172.21.15.60:0/3632718838 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f20a40546b0 con 0x560b7aa7d8c0 2024-04-25T10:10:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 10 client.7278 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:10:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 10 client.7278 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:10:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 20 client.7278 populate_metadata read hostname 'smithi060' 2024-04-25T10:10:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 --2- 172.21.15.60:0/3632718838 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x560b7aa98d70 0x560b7aa99180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 4 client.7278 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-04-25T10:10:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 -- 172.21.15.60:0/3632718838 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) v1 -- 0x560b7a88f310 con 0x560b7aa98d70 2024-04-25T10:10:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20b9dc9640 1 --2- 172.21.15.60:0/3632718838 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x560b7aa98d70 0x560b7aa99180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20b9dc9640 1 --2- 172.21.15.60:0/3632718838 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x560b7aa98d70 0x560b7aa99180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63862a556c7369e6 server_cookie=94d50b9e8a89db86 in_seq=0 out_seq=0 2024-04-25T10:10:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f2094ff9640 10 client.7278 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f2094ff9640 1 -- 172.21.15.60:0/3632718838 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560b7a88f310 con 0x560b7aa98d70 2024-04-25T10:10:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f2094ff9640 10 client.7278 handle_command_reply: tid=0 2024-04-25T10:10:21.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 client.7278 shutdown 2024-04-25T10:10:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 -- 172.21.15.60:0/3632718838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7d8c0 msgr2=0x560b7aade390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 --2- 172.21.15.60:0/3632718838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7d8c0 0x560b7aade390 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f20a4052170 tx=0x7f20a400cf50 comp rx=0 tx=0).stop 2024-04-25T10:10:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 -- 172.21.15.60:0/3632718838 shutdown_connections 2024-04-25T10:10:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 --2- 172.21.15.60:0/3632718838 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x560b7aa98d70 0x560b7aa99180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 --2- 172.21.15.60:0/3632718838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b7aa7e1e0 0x560b7aa983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 --2- 172.21.15.60:0/3632718838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560b7aa7d8c0 0x560b7aade390 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 --2- 172.21.15.60:0/3632718838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560b7aa33650 0x560b7aadde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f2094ff9640 0 client.7278 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 -- 172.21.15.60:0/3632718838 >> 172.21.15.60:0/3632718838 conn(0x560b7a91b870 msgr2=0x560b7aa7c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 -- 172.21.15.60:0/3632718838 shutdown_connections 2024-04-25T10:10:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 1 -- 172.21.15.60:0/3632718838 wait complete. 2024-04-25T10:10:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.609+0000 7f20bf19a000 20 client.7278 trim_cache size 0 max 16384 2024-04-25T10:10:21.625 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:10:21.625 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T10:10:21.625 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "51f07f81-1af7-4ffe-9bf9-f562b4b668ad", 2024-04-25T10:10:21.625 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T10:10:21.625 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:10:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.613+0000 7f20bc02d640 1 -- 172.21.15.60:0/3927600934 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2098066f60 msgr2=0x7f2098069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.613+0000 7f20bc02d640 1 --2- 172.21.15.60:0/3927600934 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2098066f60 0x7f2098069420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f20b0000f70 tx=0x7f20b0007d70 comp rx=0 tx=0).stop 2024-04-25T10:10:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.613+0000 7f20bc02d640 1 -- 172.21.15.60:0/3927600934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20b40cf0a0 msgr2=0x7f20b40af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.613+0000 7f20bc02d640 1 --2- 172.21.15.60:0/3927600934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20b40cf0a0 0x7f20b40af240 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f20a400e7f0 tx=0x7f20a400ecb0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.613+0000 7f20bc02d640 1 -- 172.21.15.60:0/3927600934 shutdown_connections 2024-04-25T10:10:21.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.613+0000 7f20bc02d640 1 --2- 172.21.15.60:0/3927600934 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2098066f60 0x7f2098069420 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.613+0000 7f20bc02d640 1 --2- 172.21.15.60:0/3927600934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b4144e40 0x7f20b40af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.613+0000 7f20bc02d640 1 --2- 172.21.15.60:0/3927600934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20b40cf0a0 0x7f20b40af240 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.613+0000 7f20bc02d640 1 --2- 172.21.15.60:0/3927600934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b4071980 0x7f20b415acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.613+0000 7f20bc02d640 1 -- 172.21.15.60:0/3927600934 >> 172.21.15.60:0/3927600934 conn(0x7f20b4066fa0 msgr2=0x7f20b406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.613+0000 7f20bc02d640 1 -- 172.21.15.60:0/3927600934 shutdown_connections 2024-04-25T10:10:21.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.613+0000 7f20bc02d640 1 -- 172.21.15.60:0/3927600934 wait complete. 2024-04-25T10:10:21.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:21.637 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:10:21.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.737+0000 7faba81ba640 1 Processor -- start 2024-04-25T10:10:21.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba81ba640 1 -- start start 2024-04-25T10:10:21.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba81ba640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faba015fbf0 0x7faba0166870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba81ba640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba0144a40 con 0x7faba00cbcf0 2024-04-25T10:10:21.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba81ba640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba00cc9f0 con 0x7faba0160080 2024-04-25T10:10:21.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba81ba640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba00b2dc0 con 0x7faba015fbf0 2024-04-25T10:10:21.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba5f56640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7faba00cbcf0 0x7faba0071c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60196/0 (socket says 172.21.15.60:60196) 2024-04-25T10:10:21.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba5f56640 1 -- 172.21.15.60:0/3358516945 learned_addr learned my addr 172.21.15.60:0/3358516945 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:10:21.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba6757640 1 --2- 172.21.15.60:0/3358516945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faba015fbf0 0x7faba0166870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba4f54640 1 -- 172.21.15.60:0/3358516945 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463763576 0 0) 0x7faba0144a40 con 0x7faba00cbcf0 2024-04-25T10:10:21.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba4f54640 1 -- 172.21.15.60:0/3358516945 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab8c003530 con 0x7faba00cbcf0 2024-04-25T10:10:21.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba4f54640 1 -- 172.21.15.60:0/3358516945 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1407688482 0 0) 0x7fab8c003530 con 0x7faba00cbcf0 2024-04-25T10:10:21.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba4f54640 1 -- 172.21.15.60:0/3358516945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faba015fbf0 msgr2=0x7faba0166870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba4f54640 1 --2- 172.21.15.60:0/3358516945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faba015fbf0 0x7faba0166870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba4f54640 1 -- 172.21.15.60:0/3358516945 >> v1:172.21.15.100:6789/0 conn(0x7faba0160080 legacy=0x7faba015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba4f54640 1 -- 172.21.15.60:0/3358516945 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba00cccd0 con 0x7faba00cbcf0 2024-04-25T10:10:21.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba4f54640 1 -- 172.21.15.60:0/3358516945 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fab9c001680 con 0x7faba00cbcf0 2024-04-25T10:10:21.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba4f54640 1 -- 172.21.15.60:0/3358516945 >> v1:172.21.15.60:6789/0 conn(0x7faba00cbcf0 legacy=0x7faba0071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba4f54640 1 --2- 172.21.15.60:0/3358516945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab8c0039f0 0x7fab8c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba4f54640 1 --2- 172.21.15.60:0/3358516945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab8c004810 0x7fab8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba4f54640 1 --2- 172.21.15.60:0/3358516945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faba015fbf0 0x7fab8c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba4f54640 1 -- 172.21.15.60:0/3358516945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c00ba80 con 0x7fab8c004810 2024-04-25T10:10:21.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba4f54640 1 -- 172.21.15.60:0/3358516945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c00bc30 con 0x7fab8c0039f0 2024-04-25T10:10:21.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba4f54640 1 -- 172.21.15.60:0/3358516945 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c00be30 con 0x7faba015fbf0 2024-04-25T10:10:21.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba5755640 1 --2- 172.21.15.60:0/3358516945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab8c004810 0x7fab8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba5f56640 1 --2- 172.21.15.60:0/3358516945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab8c0039f0 0x7fab8c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba6757640 1 --2- 172.21.15.60:0/3358516945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faba015fbf0 0x7fab8c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba5755640 1 -- 172.21.15.60:0/3358516945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faba015fbf0 msgr2=0x7fab8c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba5755640 1 --2- 172.21.15.60:0/3358516945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faba015fbf0 0x7fab8c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba5755640 1 -- 172.21.15.60:0/3358516945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab8c0039f0 msgr2=0x7fab8c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba5755640 1 --2- 172.21.15.60:0/3358516945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab8c0039f0 0x7fab8c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba5755640 1 -- 172.21.15.60:0/3358516945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c003530 con 0x7fab8c004810 2024-04-25T10:10:21.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.741+0000 7faba5f56640 1 --2- 172.21.15.60:0/3358516945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab8c0039f0 0x7fab8c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:21.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba5755640 1 --2- 172.21.15.60:0/3358516945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab8c004810 0x7fab8c004c90 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fab90001780 tx=0x7fab9001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e576c3aa1a7eb3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba4f54640 1 -- 172.21.15.60:0/3358516945 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab900013a0 con 0x7fab8c004810 2024-04-25T10:10:21.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba4f54640 1 -- 172.21.15.60:0/3358516945 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab900a9070 con 0x7fab8c004810 2024-04-25T10:10:21.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba4f54640 1 -- 172.21.15.60:0/3358516945 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab90003600 con 0x7fab8c004810 2024-04-25T10:10:21.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba81ba640 1 -- 172.21.15.60:0/3358516945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab8c004810 msgr2=0x7fab8c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba81ba640 1 --2- 172.21.15.60:0/3358516945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab8c004810 0x7fab8c004c90 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fab90001780 tx=0x7fab9001c420 comp rx=0 tx=0).stop 2024-04-25T10:10:21.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba6757640 1 -- 172.21.15.60:0/3358516945 reap_dead start 2024-04-25T10:10:21.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba81ba640 1 -- 172.21.15.60:0/3358516945 shutdown_connections 2024-04-25T10:10:21.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba81ba640 1 -- 172.21.15.60:0/3358516945 >> 172.21.15.60:0/3358516945 conn(0x7faba00670b0 msgr2=0x7faba006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba81ba640 1 -- 172.21.15.60:0/3358516945 shutdown_connections 2024-04-25T10:10:21.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba81ba640 1 -- 172.21.15.60:0/3358516945 wait complete. 2024-04-25T10:10:21.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba81ba640 1 Processor -- start 2024-04-25T10:10:21.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba81ba640 1 -- start start 2024-04-25T10:10:21.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba81ba640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab8c004810 0x7faba00b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba81ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faba00c59c0 0x7faba015b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba81ba640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faba00c66c0 0x7faba00c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba81ba640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba00cc0d0 con 0x7faba00c59c0 2024-04-25T10:10:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba81ba640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba00ce9a0 con 0x7fab8c004810 2024-04-25T10:10:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba81ba640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba00cfed0 con 0x7faba00c66c0 2024-04-25T10:10:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba5755640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faba00c59c0 0x7faba015b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba5755640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faba00c59c0 0x7faba015b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41224/0 (socket says 172.21.15.60:41224) 2024-04-25T10:10:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba5755640 1 -- 172.21.15.60:0/2174424149 learned_addr learned my addr 172.21.15.60:0/2174424149 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba5f56640 1 --2- 172.21.15.60:0/2174424149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab8c004810 0x7faba00b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba5755640 1 -- 172.21.15.60:0/2174424149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faba00c66c0 msgr2=0x7faba00c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba5755640 1 --2- 172.21.15.60:0/2174424149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faba00c66c0 0x7faba00c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba5755640 1 -- 172.21.15.60:0/2174424149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab8c004810 msgr2=0x7faba00b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba5755640 1 --2- 172.21.15.60:0/2174424149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab8c004810 0x7faba00b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.745+0000 7faba5755640 1 -- 172.21.15.60:0/2174424149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba00c4920 con 0x7faba00c59c0 2024-04-25T10:10:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7faba5755640 1 --2- 172.21.15.60:0/2174424149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faba00c59c0 0x7faba015b2a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fab90001780 tx=0x7fab8c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7fab967fc640 1 -- 172.21.15.60:0/2174424149 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab90007a40 con 0x7faba00c59c0 2024-04-25T10:10:21.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7fab967fc640 1 -- 172.21.15.60:0/2174424149 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab90007250 con 0x7faba00c59c0 2024-04-25T10:10:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7faba81ba640 1 -- 172.21.15.60:0/2174424149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba00c4c00 con 0x7faba00c59c0 2024-04-25T10:10:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7fab967fc640 1 -- 172.21.15.60:0/2174424149 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab90007420 con 0x7faba00c59c0 2024-04-25T10:10:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7faba81ba640 1 -- 172.21.15.60:0/2174424149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba00c51c0 con 0x7faba00c59c0 2024-04-25T10:10:21.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7fab967fc640 1 -- 172.21.15.60:0/2174424149 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab900be020 con 0x7faba00c59c0 2024-04-25T10:10:21.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7fab967fc640 1 --2- 172.21.15.60:0/2174424149 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fab84066f90 0x7fab84069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7fab967fc640 1 -- 172.21.15.60:0/2174424149 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fab9011e760 con 0x7faba00c59c0 2024-04-25T10:10:21.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7faba5f56640 1 --2- 172.21.15.60:0/2174424149 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fab84066f90 0x7fab84069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7fabab327000 1 Processor -- start 2024-04-25T10:10:21.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7fabab327000 1 -- start start 2024-04-25T10:10:21.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7faba5f56640 1 --2- 172.21.15.60:0/2174424149 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fab84066f90 0x7fab84069450 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fab9c001ec0 tx=0x7fab9c000ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7fabab327000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551889d0 0x55bd55188db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7fabab327000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551892f0 0x55bd5513eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7fabab327000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd5513f0a0 0x55bd5513f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7fabab327000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd54f8bff0 con 0x55bd551889d0 2024-04-25T10:10:21.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7fabab327000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd54f14150 con 0x55bd5513f0a0 2024-04-25T10:10:21.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7fabab327000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd54f88090 con 0x55bd551892f0 2024-04-25T10:10:21.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7faba5f56640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551889d0 0x55bd55188db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7faba5f56640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551889d0 0x55bd55188db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41234/0 (socket says 172.21.15.60:41234) 2024-04-25T10:10:21.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7faba5f56640 1 -- 172.21.15.60:0/1673699189 learned_addr learned my addr 172.21.15.60:0/1673699189 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:21.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7faba5755640 1 --2- 172.21.15.60:0/1673699189 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551892f0 0x55bd5513eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.749+0000 7faba6757640 1 --2- 172.21.15.60:0/1673699189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd5513f0a0 0x55bd5513f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba5f56640 1 -- 172.21.15.60:0/1673699189 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551892f0 msgr2=0x55bd5513eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba5f56640 1 --2- 172.21.15.60:0/1673699189 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551892f0 0x55bd5513eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba5f56640 1 -- 172.21.15.60:0/1673699189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd5513f0a0 msgr2=0x55bd5513f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba5f56640 1 --2- 172.21.15.60:0/1673699189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd5513f0a0 0x55bd5513f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba5f56640 1 -- 172.21.15.60:0/1673699189 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd551220d0 con 0x55bd551889d0 2024-04-25T10:10:21.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba5f56640 1 --2- 172.21.15.60:0/1673699189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551889d0 0x55bd55188db0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fab9c012aa0 tx=0x7fab9c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f68ab242413d88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fab837fe640 1 -- 172.21.15.60:0/1673699189 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c029070 con 0x55bd551889d0 2024-04-25T10:10:21.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fab837fe640 1 -- 172.21.15.60:0/1673699189 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c016a00 con 0x55bd551889d0 2024-04-25T10:10:21.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fab837fe640 1 -- 172.21.15.60:0/1673699189 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c016df0 con 0x55bd551889d0 2024-04-25T10:10:21.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 -- 172.21.15.60:0/1673699189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551889d0 msgr2=0x55bd55188db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 --2- 172.21.15.60:0/1673699189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551889d0 0x55bd55188db0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fab9c012aa0 tx=0x7fab9c015000 comp rx=0 tx=0).stop 2024-04-25T10:10:21.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 -- 172.21.15.60:0/1673699189 shutdown_connections 2024-04-25T10:10:21.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 --2- 172.21.15.60:0/1673699189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd5513f0a0 0x55bd5513f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 --2- 172.21.15.60:0/1673699189 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551892f0 0x55bd5513eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 --2- 172.21.15.60:0/1673699189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551889d0 0x55bd55188db0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 -- 172.21.15.60:0/1673699189 >> 172.21.15.60:0/1673699189 conn(0x55bd5513b020 msgr2=0x55bd5513d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 -- 172.21.15.60:0/1673699189 shutdown_connections 2024-04-25T10:10:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 -- 172.21.15.60:0/1673699189 wait complete. 2024-04-25T10:10:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 -- start start 2024-04-25T10:10:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 Processor -- start 2024-04-25T10:10:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd5513f0a0 0x55bd5516d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551889d0 0x55bd5516dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551892f0 0x55bd5516e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd54f50250 con 0x55bd551892f0 2024-04-25T10:10:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd54f46690 con 0x55bd5513f0a0 2024-04-25T10:10:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd54f14a40 con 0x55bd551889d0 2024-04-25T10:10:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 10 client.0 _ll_register_callbacks cb 0x55bd55129e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba6757640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551892f0 0x55bd5516e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba6757640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551892f0 0x55bd5516e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41242/0 (socket says 172.21.15.60:41242) 2024-04-25T10:10:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba6757640 1 -- 172.21.15.60:0/2542371976 learned_addr learned my addr 172.21.15.60:0/2542371976 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba5f56640 1 --2- 172.21.15.60:0/2542371976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd5513f0a0 0x55bd5516d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba5755640 1 --2- 172.21.15.60:0/2542371976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551889d0 0x55bd5516dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba6757640 1 -- 172.21.15.60:0/2542371976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551889d0 msgr2=0x55bd5516dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba6757640 1 --2- 172.21.15.60:0/2542371976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551889d0 0x55bd5516dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba6757640 1 -- 172.21.15.60:0/2542371976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd5513f0a0 msgr2=0x55bd5516d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba6757640 1 --2- 172.21.15.60:0/2542371976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd5513f0a0 0x55bd5516d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba6757640 1 -- 172.21.15.60:0/2542371976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd550391c0 con 0x55bd551892f0 2024-04-25T10:10:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7faba6757640 1 --2- 172.21.15.60:0/2542371976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551892f0 0x55bd5516e270 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7faba015cf20 tx=0x7fab98012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98fb03d8abb2565 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fab80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:10:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fab80ff9640 1 -- 172.21.15.60:0/2542371976 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98012e70 con 0x55bd551892f0 2024-04-25T10:10:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 -- 172.21.15.60:0/2542371976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab9c016a00 con 0x55bd551892f0 2024-04-25T10:10:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fab80ff9640 1 -- 172.21.15.60:0/2542371976 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fab980039a0 con 0x55bd551892f0 2024-04-25T10:10:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fab80ff9640 1 -- 172.21.15.60:0/2542371976 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab98025050 con 0x55bd551892f0 2024-04-25T10:10:21.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 10 client.7290 fetch_fsmap learned FSMap version 46 2024-04-25T10:10:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fabab327000 1 -- 172.21.15.60:0/2542371976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x55bd55035170 con 0x55bd551892f0 2024-04-25T10:10:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.753+0000 7fab80ff9640 1 -- 172.21.15.60:0/2542371976 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c016a00 con 0x55bd551892f0 2024-04-25T10:10:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fab80ff9640 1 -- 172.21.15.60:0/2542371976 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fab98028020 con 0x55bd551892f0 2024-04-25T10:10:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 10 client.7290 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:10:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 10 client.7290 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:10:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 20 client.7290 populate_metadata read hostname 'smithi060' 2024-04-25T10:10:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 1 --2- 172.21.15.60:0/2542371976 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55bd55198c80 0x55bd5519b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 4 client.7290 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 1 -- 172.21.15.60:0/2542371976 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd54f27c60 con 0x55bd55198c80 2024-04-25T10:10:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7faba5f56640 1 --2- 172.21.15.60:0/2542371976 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55bd55198c80 0x55bd5519b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fab80ff9640 10 client.7290 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7faba5f56640 1 --2- 172.21.15.60:0/2542371976 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55bd55198c80 0x55bd5519b070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=949e7c1f8f566398 server_cookie=4dab0e9afbece012 in_seq=0 out_seq=0 2024-04-25T10:10:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fab80ff9640 1 -- 172.21.15.60:0/2542371976 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bd54f27c60 con 0x55bd55198c80 2024-04-25T10:10:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fab80ff9640 10 client.7290 handle_command_reply: tid=0 2024-04-25T10:10:21.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 1 client.7290 shutdown 2024-04-25T10:10:21.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 1 -- 172.21.15.60:0/2542371976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551892f0 msgr2=0x55bd5516e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 1 --2- 172.21.15.60:0/2542371976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551892f0 0x55bd5516e270 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7faba015cf20 tx=0x7fab98012610 comp rx=0 tx=0).stop 2024-04-25T10:10:21.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 1 -- 172.21.15.60:0/2542371976 shutdown_connections 2024-04-25T10:10:21.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 1 --2- 172.21.15.60:0/2542371976 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55bd55198c80 0x55bd5519b070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 1 --2- 172.21.15.60:0/2542371976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551892f0 0x55bd5516e270 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 1 --2- 172.21.15.60:0/2542371976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551889d0 0x55bd5516dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fab80ff9640 0 client.7290 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:21.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 1 --2- 172.21.15.60:0/2542371976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd5513f0a0 0x55bd5516d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 1 -- 172.21.15.60:0/2542371976 >> 172.21.15.60:0/2542371976 conn(0x55bd5513b020 msgr2=0x55bd5513dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 1 -- 172.21.15.60:0/2542371976 shutdown_connections 2024-04-25T10:10:21.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 1 -- 172.21.15.60:0/2542371976 wait complete. 2024-04-25T10:10:21.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.757+0000 7fabab327000 20 client.7290 trim_cache size 0 max 16384 2024-04-25T10:10:21.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 Processor -- start 2024-04-25T10:10:21.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 -- start start 2024-04-25T10:10:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd5513f0a0 0x55bd55187de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551889d0 0x55bd5513e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd551892f0 0x55bd5513e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba0200390 con 0x55bd5513f0a0 2024-04-25T10:10:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba01ff510 con 0x55bd551892f0 2024-04-25T10:10:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba00af960 con 0x55bd551889d0 2024-04-25T10:10:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7faba5f56640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd5513f0a0 0x55bd55187de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7faba5f56640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd5513f0a0 0x55bd55187de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41250/0 (socket says 172.21.15.60:41250) 2024-04-25T10:10:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7faba5f56640 1 -- 172.21.15.60:0/1436444228 learned_addr learned my addr 172.21.15.60:0/1436444228 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7faba5f56640 1 -- 172.21.15.60:0/1436444228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551889d0 msgr2=0x55bd5513e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7faba5f56640 1 --2- 172.21.15.60:0/1436444228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551889d0 0x55bd5513e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7faba5f56640 1 -- 172.21.15.60:0/1436444228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd551892f0 msgr2=0x55bd5513e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7faba5f56640 1 --2- 172.21.15.60:0/1436444228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd551892f0 0x55bd5513e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7faba5f56640 1 -- 172.21.15.60:0/1436444228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd54fd2de0 con 0x55bd5513f0a0 2024-04-25T10:10:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7faba5f56640 1 --2- 172.21.15.60:0/1436444228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd5513f0a0 0x55bd55187de0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fab9c027040 tx=0x7fab9c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee369ee6cf46b06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fab82ffd640 1 -- 172.21.15.60:0/1436444228 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c029040 con 0x55bd5513f0a0 2024-04-25T10:10:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fab82ffd640 1 -- 172.21.15.60:0/1436444228 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c016a00 con 0x55bd5513f0a0 2024-04-25T10:10:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fab82ffd640 1 -- 172.21.15.60:0/1436444228 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c033370 con 0x55bd5513f0a0 2024-04-25T10:10:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 -- 172.21.15.60:0/1436444228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd5513f0a0 msgr2=0x55bd55187de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 --2- 172.21.15.60:0/1436444228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd5513f0a0 0x55bd55187de0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fab9c027040 tx=0x7fab9c016000 comp rx=0 tx=0).stop 2024-04-25T10:10:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 -- 172.21.15.60:0/1436444228 shutdown_connections 2024-04-25T10:10:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 --2- 172.21.15.60:0/1436444228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd551892f0 0x55bd5513e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 --2- 172.21.15.60:0/1436444228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551889d0 0x55bd5513e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 --2- 172.21.15.60:0/1436444228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd5513f0a0 0x55bd55187de0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 -- 172.21.15.60:0/1436444228 >> 172.21.15.60:0/1436444228 conn(0x55bd5513b020 msgr2=0x55bd551870a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 -- 172.21.15.60:0/1436444228 shutdown_connections 2024-04-25T10:10:21.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 -- 172.21.15.60:0/1436444228 wait complete. 2024-04-25T10:10:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 -- start start 2024-04-25T10:10:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 Processor -- start 2024-04-25T10:10:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd5513f0a0 0x55bd551e9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551889d0 0x55bd551e9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551892f0 0x55bd5519b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd54f26e00 con 0x55bd551892f0 2024-04-25T10:10:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba01fef40 con 0x55bd5513f0a0 2024-04-25T10:10:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd54f8b300 con 0x55bd551889d0 2024-04-25T10:10:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7fabab327000 10 client.0 _ll_register_callbacks cb 0x55bd5510a040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7faba6757640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551892f0 0x55bd5519b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.773+0000 7faba6757640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551892f0 0x55bd5519b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41252/0 (socket says 172.21.15.60:41252) 2024-04-25T10:10:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba6757640 1 -- 172.21.15.60:0/1547355943 learned_addr learned my addr 172.21.15.60:0/1547355943 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba5f56640 1 --2- 172.21.15.60:0/1547355943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd5513f0a0 0x55bd551e9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba5755640 1 --2- 172.21.15.60:0/1547355943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551889d0 0x55bd551e9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba6757640 1 -- 172.21.15.60:0/1547355943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551889d0 msgr2=0x55bd551e9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba6757640 1 --2- 172.21.15.60:0/1547355943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551889d0 0x55bd551e9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba6757640 1 -- 172.21.15.60:0/1547355943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd5513f0a0 msgr2=0x55bd551e9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba6757640 1 --2- 172.21.15.60:0/1547355943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd5513f0a0 0x55bd551e9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba6757640 1 -- 172.21.15.60:0/1547355943 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd54f2d8e0 con 0x55bd551892f0 2024-04-25T10:10:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba6757640 1 --2- 172.21.15.60:0/1547355943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551892f0 0x55bd5519b380 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fab98009c10 tx=0x7fab9800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2876a13666ca2a0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fab80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:10:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fab80ff9640 1 -- 172.21.15.60:0/1547355943 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98012e70 con 0x55bd551892f0 2024-04-25T10:10:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fab80ff9640 1 -- 172.21.15.60:0/1547355943 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fab98022b30 con 0x55bd551892f0 2024-04-25T10:10:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 1 -- 172.21.15.60:0/1547355943 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab9c016a00 con 0x55bd551892f0 2024-04-25T10:10:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fab80ff9640 1 -- 172.21.15.60:0/1547355943 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab98022d50 con 0x55bd551892f0 2024-04-25T10:10:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 10 client.7296 fetch_fsmap learned FSMap version 46 2024-04-25T10:10:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 1 -- 172.21.15.60:0/1547355943 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x55bd54fe58f0 con 0x55bd551892f0 2024-04-25T10:10:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fab80ff9640 1 -- 172.21.15.60:0/1547355943 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c016a00 con 0x55bd551892f0 2024-04-25T10:10:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fab80ff9640 1 -- 172.21.15.60:0/1547355943 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fab98028030 con 0x55bd551892f0 2024-04-25T10:10:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 10 client.7296 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:10:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 10 client.7296 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:10:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 20 client.7296 populate_metadata read hostname 'smithi060' 2024-04-25T10:10:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 1 --2- 172.21.15.60:0/1547355943 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55bd5519c090 0x55bd5519c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 4 client.7296 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:10:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 1 -- 172.21.15.60:0/1547355943 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bd54f99530 con 0x55bd5519c090 2024-04-25T10:10:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba5f56640 1 --2- 172.21.15.60:0/1547355943 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55bd5519c090 0x55bd5519c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fab80ff9640 10 client.7296 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba5f56640 1 --2- 172.21.15.60:0/1547355943 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55bd5519c090 0x55bd5519c4a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ecf64b6a87d3391 server_cookie=d0bb9d74d7d1c96b in_seq=0 out_seq=0 2024-04-25T10:10:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fab80ff9640 1 -- 172.21.15.60:0/1547355943 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bd54f99530 con 0x55bd5519c090 2024-04-25T10:10:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fab80ff9640 10 client.7296 handle_command_reply: tid=0 2024-04-25T10:10:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 1 client.7296 shutdown 2024-04-25T10:10:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 1 -- 172.21.15.60:0/1547355943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551892f0 msgr2=0x55bd5519b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 1 --2- 172.21.15.60:0/1547355943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551892f0 0x55bd5519b380 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fab98009c10 tx=0x7fab9800b040 comp rx=0 tx=0).stop 2024-04-25T10:10:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 1 -- 172.21.15.60:0/1547355943 shutdown_connections 2024-04-25T10:10:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 1 --2- 172.21.15.60:0/1547355943 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55bd5519c090 0x55bd5519c4a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 1 --2- 172.21.15.60:0/1547355943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd551892f0 0x55bd5519b380 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 1 --2- 172.21.15.60:0/1547355943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd551889d0 0x55bd551e9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fab80ff9640 0 client.7296 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 1 --2- 172.21.15.60:0/1547355943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd5513f0a0 0x55bd551e9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 1 -- 172.21.15.60:0/1547355943 >> 172.21.15.60:0/1547355943 conn(0x55bd5513b020 msgr2=0x55bd55187930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 1 -- 172.21.15.60:0/1547355943 shutdown_connections 2024-04-25T10:10:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 1 -- 172.21.15.60:0/1547355943 wait complete. 2024-04-25T10:10:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7fabab327000 20 client.7296 trim_cache size 0 max 16384 2024-04-25T10:10:21.791 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:10:21.791 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:10:21.791 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T10:10:21.791 INFO:teuthology.orchestra.run.smithi060.stdout: "51f07f81-1af7-4ffe-9bf9-f562b4b668ad": { 2024-04-25T10:10:21.791 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/subdir", 2024-04-25T10:10:21.791 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "51f07f81-1af7-4ffe-9bf9-f562b4b668ad", 2024-04-25T10:10:21.791 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive" 2024-04-25T10:10:21.791 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:10:21.791 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:10:21.791 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:10:21.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba81ba640 1 -- 172.21.15.60:0/2174424149 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fab84066f90 msgr2=0x7fab84069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba81ba640 1 --2- 172.21.15.60:0/2174424149 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fab84066f90 0x7fab84069450 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fab9c001ec0 tx=0x7fab9c000ee0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba81ba640 1 -- 172.21.15.60:0/2174424149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faba00c59c0 msgr2=0x7faba015b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba81ba640 1 --2- 172.21.15.60:0/2174424149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faba00c59c0 0x7faba015b2a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fab90001780 tx=0x7fab8c005840 comp rx=0 tx=0).stop 2024-04-25T10:10:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba81ba640 1 -- 172.21.15.60:0/2174424149 shutdown_connections 2024-04-25T10:10:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba81ba640 1 --2- 172.21.15.60:0/2174424149 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fab84066f90 0x7fab84069450 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba81ba640 1 --2- 172.21.15.60:0/2174424149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faba00c66c0 0x7faba00c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba81ba640 1 --2- 172.21.15.60:0/2174424149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faba00c59c0 0x7faba015b2a0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba81ba640 1 --2- 172.21.15.60:0/2174424149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab8c004810 0x7faba00b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba81ba640 1 -- 172.21.15.60:0/2174424149 >> 172.21.15.60:0/2174424149 conn(0x7faba00670b0 msgr2=0x7faba015e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba81ba640 1 -- 172.21.15.60:0/2174424149 shutdown_connections 2024-04-25T10:10:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:21.777+0000 7faba81ba640 1 -- 172.21.15.60:0/2174424149 wait complete. 2024-04-25T10:10:21.801 INFO:tasks.cephfs.filesystem:scrub status for tag:51f07f81-1af7-4ffe-9bf9-f562b4b668ad - {'path': '/subdir', 'tag': '51f07f81-1af7-4ffe-9bf9-f562b4b668ad', 'options': 'recursive'} 2024-04-25T10:10:46.986 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:46.992 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:51.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:51.803 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:10:51.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.933+0000 7f921f792640 1 Processor -- start 2024-04-25T10:10:51.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.933+0000 7f921f792640 1 -- start start 2024-04-25T10:10:51.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921f792640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f921815de90 0x7f92180b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921f792640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f92180d0320 0x7f92180d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921f792640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92180ce290 0x7f92180ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921f792640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92180cf2a0 con 0x7f921815de90 2024-04-25T10:10:51.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921f792640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92180cefc0 con 0x7f92180d0320 2024-04-25T10:10:51.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921f792640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92180d1d50 con 0x7f92180ce290 2024-04-25T10:10:51.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921d52e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f921815de90 0x7f92180b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921d52e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f921815de90 0x7f92180b6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57516/0 (socket says 172.21.15.60:57516) 2024-04-25T10:10:51.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921d52e640 1 -- 172.21.15.60:0/3267830880 learned_addr learned my addr 172.21.15.60:0/3267830880 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:51.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921cd2d640 1 --2- 172.21.15.60:0/3267830880 >> v2:172.21.15.100:3300/0 conn(0x7f92180d0320 0x7f92180d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921dd2f640 1 --2- 172.21.15.60:0/3267830880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92180ce290 0x7f92180ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921d52e640 1 -- 172.21.15.60:0/3267830880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92180ce290 msgr2=0x7f92180ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:51.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921d52e640 1 --2- 172.21.15.60:0/3267830880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92180ce290 0x7f92180ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921d52e640 1 -- 172.21.15.60:0/3267830880 >> v2:172.21.15.100:3300/0 conn(0x7f92180d0320 msgr2=0x7f92180d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:51.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921d52e640 1 --2- 172.21.15.60:0/3267830880 >> v2:172.21.15.100:3300/0 conn(0x7f92180d0320 0x7f92180d07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921d52e640 1 -- 172.21.15.60:0/3267830880 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921806d200 con 0x7f921815de90 2024-04-25T10:10:51.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921cd2d640 1 --2- 172.21.15.60:0/3267830880 >> v2:172.21.15.100:3300/0 conn(0x7f92180d0320 0x7f92180d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:51.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921dd2f640 1 --2- 172.21.15.60:0/3267830880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92180ce290 0x7f92180ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:51.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921d52e640 1 --2- 172.21.15.60:0/3267830880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f921815de90 0x7f92180b6bf0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f92180b6470 tx=0x7f920c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f215e5f52f04a246 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f920bfff640 1 -- 172.21.15.60:0/3267830880 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c0b0070 con 0x7f921815de90 2024-04-25T10:10:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f920bfff640 1 -- 172.21.15.60:0/3267830880 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920c0ab600 con 0x7f921815de90 2024-04-25T10:10:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f920bfff640 1 -- 172.21.15.60:0/3267830880 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c0ab9d0 con 0x7f921815de90 2024-04-25T10:10:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921f792640 1 -- 172.21.15.60:0/3267830880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f921815de90 msgr2=0x7f92180b6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921f792640 1 --2- 172.21.15.60:0/3267830880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f921815de90 0x7f92180b6bf0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f92180b6470 tx=0x7f920c01c520 comp rx=0 tx=0).stop 2024-04-25T10:10:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921f792640 1 -- 172.21.15.60:0/3267830880 shutdown_connections 2024-04-25T10:10:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921f792640 1 --2- 172.21.15.60:0/3267830880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92180ce290 0x7f92180ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921f792640 1 --2- 172.21.15.60:0/3267830880 >> v2:172.21.15.100:3300/0 conn(0x7f92180d0320 0x7f92180d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921f792640 1 --2- 172.21.15.60:0/3267830880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f921815de90 0x7f92180b6bf0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921f792640 1 -- 172.21.15.60:0/3267830880 >> 172.21.15.60:0/3267830880 conn(0x7f921806c090 msgr2=0x7f921806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:51.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921f792640 1 -- 172.21.15.60:0/3267830880 shutdown_connections 2024-04-25T10:10:51.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.937+0000 7f921f792640 1 -- 172.21.15.60:0/3267830880 wait complete. 2024-04-25T10:10:51.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921f792640 1 Processor -- start 2024-04-25T10:10:51.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921f792640 1 -- start start 2024-04-25T10:10:51.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921f792640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92180ce290 0x7f92180c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921f792640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92180d0320 0x7f921814b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921f792640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f921815de90 0x7f92180c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921f792640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92180b5d20 con 0x7f92180d0320 2024-04-25T10:10:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921f792640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92180b5ba0 con 0x7f92180ce290 2024-04-25T10:10:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921f792640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92180b5ea0 con 0x7f921815de90 2024-04-25T10:10:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921cd2d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92180d0320 0x7f921814b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921cd2d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92180d0320 0x7f921814b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57524/0 (socket says 172.21.15.60:57524) 2024-04-25T10:10:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921cd2d640 1 -- 172.21.15.60:0/1395049228 learned_addr learned my addr 172.21.15.60:0/1395049228 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921cd2d640 1 -- 172.21.15.60:0/1395049228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f921815de90 msgr2=0x7f92180c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921dd2f640 1 --2- 172.21.15.60:0/1395049228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f921815de90 0x7f92180c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921d52e640 1 --2- 172.21.15.60:0/1395049228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92180ce290 0x7f92180c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921cd2d640 1 --2- 172.21.15.60:0/1395049228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f921815de90 0x7f92180c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921cd2d640 1 -- 172.21.15.60:0/1395049228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92180ce290 msgr2=0x7f92180c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921cd2d640 1 --2- 172.21.15.60:0/1395049228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92180ce290 0x7f92180c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921cd2d640 1 -- 172.21.15.60:0/1395049228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92180c9a30 con 0x7f92180d0320 2024-04-25T10:10:51.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921cd2d640 1 --2- 172.21.15.60:0/1395049228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92180d0320 0x7f921814b4b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f921400cf90 tx=0x7f921400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:51.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f9209ffb640 1 -- 172.21.15.60:0/1395049228 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921401a070 con 0x7f92180d0320 2024-04-25T10:10:51.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f9209ffb640 1 -- 172.21.15.60:0/1395049228 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921400d840 con 0x7f92180d0320 2024-04-25T10:10:51.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921f792640 1 -- 172.21.15.60:0/1395049228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9218153840 con 0x7f92180d0320 2024-04-25T10:10:51.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f9209ffb640 1 -- 172.21.15.60:0/1395049228 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921400dc10 con 0x7f92180d0320 2024-04-25T10:10:51.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.941+0000 7f921f792640 1 -- 172.21.15.60:0/1395049228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9218153e00 con 0x7f92180d0320 2024-04-25T10:10:51.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f9209ffb640 1 -- 172.21.15.60:0/1395049228 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f921401d070 con 0x7f92180d0320 2024-04-25T10:10:51.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f9209ffb640 1 --2- 172.21.15.60:0/1395049228 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f91fc066f60 0x7f91fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f9209ffb640 1 -- 172.21.15.60:0/1395049228 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f92140857e0 con 0x7f92180d0320 2024-04-25T10:10:51.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f921d52e640 1 --2- 172.21.15.60:0/1395049228 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f91fc066f60 0x7f91fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f92228ff000 1 Processor -- start 2024-04-25T10:10:51.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f92228ff000 1 -- start start 2024-04-25T10:10:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f921d52e640 1 --2- 172.21.15.60:0/1395049228 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f91fc066f60 0x7f91fc069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f920c0ad000 tx=0x7f920c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f92228ff000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc78759d0 0x556fc7875db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f92228ff000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78762f0 0x556fc782bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f92228ff000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc782c0a0 0x556fc782c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f92228ff000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc7678ff0 con 0x556fc782c0a0 2024-04-25T10:10:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f92228ff000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc7601150 con 0x556fc78759d0 2024-04-25T10:10:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f92228ff000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556fc7675090 con 0x556fc78762f0 2024-04-25T10:10:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f921dd2f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc782c0a0 0x556fc782c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f921dd2f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc782c0a0 0x556fc782c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57530/0 (socket says 172.21.15.60:57530) 2024-04-25T10:10:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f921dd2f640 1 -- 172.21.15.60:0/1760805652 learned_addr learned my addr 172.21.15.60:0/1760805652 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f921cd2d640 1 --2- 172.21.15.60:0/1760805652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78762f0 0x556fc782bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f921d52e640 1 --2- 172.21.15.60:0/1760805652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc78759d0 0x556fc7875db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f921dd2f640 1 -- 172.21.15.60:0/1760805652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78762f0 msgr2=0x556fc782bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f921dd2f640 1 --2- 172.21.15.60:0/1760805652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78762f0 0x556fc782bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f921dd2f640 1 -- 172.21.15.60:0/1760805652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc78759d0 msgr2=0x556fc7875db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f921dd2f640 1 --2- 172.21.15.60:0/1760805652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc78759d0 0x556fc7875db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f921dd2f640 1 -- 172.21.15.60:0/1760805652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fc780f0d0 con 0x556fc782c0a0 2024-04-25T10:10:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f921d52e640 1 --2- 172.21.15.60:0/1760805652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc78759d0 0x556fc7875db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f921cd2d640 1 --2- 172.21.15.60:0/1760805652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78762f0 0x556fc782bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f921dd2f640 1 --2- 172.21.15.60:0/1760805652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc782c0a0 0x556fc782c480 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f921000b8c0 tx=0x7f921000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65048a38860c8db2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f9202ffd640 1 -- 172.21.15.60:0/1760805652 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9210016070 con 0x556fc782c0a0 2024-04-25T10:10:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f9202ffd640 1 -- 172.21.15.60:0/1760805652 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921000dbb0 con 0x556fc782c0a0 2024-04-25T10:10:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.945+0000 7f9202ffd640 1 -- 172.21.15.60:0/1760805652 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9210012460 con 0x556fc782c0a0 2024-04-25T10:10:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 -- 172.21.15.60:0/1760805652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc782c0a0 msgr2=0x556fc782c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 --2- 172.21.15.60:0/1760805652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc782c0a0 0x556fc782c480 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f921000b8c0 tx=0x7f921000bd80 comp rx=0 tx=0).stop 2024-04-25T10:10:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 -- 172.21.15.60:0/1760805652 shutdown_connections 2024-04-25T10:10:51.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 --2- 172.21.15.60:0/1760805652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc782c0a0 0x556fc782c480 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 --2- 172.21.15.60:0/1760805652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78762f0 0x556fc782bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 --2- 172.21.15.60:0/1760805652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc78759d0 0x556fc7875db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 -- 172.21.15.60:0/1760805652 >> 172.21.15.60:0/1760805652 conn(0x556fc7828020 msgr2=0x556fc782a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:51.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 -- 172.21.15.60:0/1760805652 shutdown_connections 2024-04-25T10:10:51.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 -- 172.21.15.60:0/1760805652 wait complete. 2024-04-25T10:10:51.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 -- start start 2024-04-25T10:10:51.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 Processor -- start 2024-04-25T10:10:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc782c0a0 0x556fc785a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78759d0 0x556fc785ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc78762f0 0x556fc785b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc763d250 con 0x556fc78762f0 2024-04-25T10:10:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc7633690 con 0x556fc782c0a0 2024-04-25T10:10:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556fc7601a40 con 0x556fc78759d0 2024-04-25T10:10:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 10 client.0 _ll_register_callbacks cb 0x556fc7816e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f921dd2f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc78762f0 0x556fc785b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f921dd2f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc78762f0 0x556fc785b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57536/0 (socket says 172.21.15.60:57536) 2024-04-25T10:10:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f921dd2f640 1 -- 172.21.15.60:0/3678764687 learned_addr learned my addr 172.21.15.60:0/3678764687 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f921cd2d640 1 --2- 172.21.15.60:0/3678764687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78759d0 0x556fc785ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f921d52e640 1 --2- 172.21.15.60:0/3678764687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc782c0a0 0x556fc785a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f921dd2f640 1 -- 172.21.15.60:0/3678764687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78759d0 msgr2=0x556fc785ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f921dd2f640 1 --2- 172.21.15.60:0/3678764687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78759d0 0x556fc785ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f921dd2f640 1 -- 172.21.15.60:0/3678764687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc782c0a0 msgr2=0x556fc785a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:51.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f921dd2f640 1 --2- 172.21.15.60:0/3678764687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc782c0a0 0x556fc785a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f921dd2f640 1 -- 172.21.15.60:0/3678764687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556fc77261c0 con 0x556fc78762f0 2024-04-25T10:10:51.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f921cd2d640 1 --2- 172.21.15.60:0/3678764687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78759d0 0x556fc785ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:51.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f921d52e640 1 --2- 172.21.15.60:0/3678764687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc782c0a0 0x556fc785a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:51.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f921dd2f640 1 --2- 172.21.15.60:0/3678764687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc78762f0 0x556fc785b270 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x556fc76353d0 tx=0x7f9210004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c2a33071c75d8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:51.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f91f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:10:51.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f91f7fff640 1 -- 172.21.15.60:0/3678764687 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9210016040 con 0x556fc78762f0 2024-04-25T10:10:51.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f91f7fff640 1 -- 172.21.15.60:0/3678764687 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9210012460 con 0x556fc78762f0 2024-04-25T10:10:51.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 -- 172.21.15.60:0/3678764687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f921000dbb0 con 0x556fc78762f0 2024-04-25T10:10:51.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f91f7fff640 1 -- 172.21.15.60:0/3678764687 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f921002f050 con 0x556fc78762f0 2024-04-25T10:10:51.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 10 client.7308 fetch_fsmap learned FSMap version 46 2024-04-25T10:10:51.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 -- 172.21.15.60:0/3678764687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x556fc7722170 con 0x556fc78762f0 2024-04-25T10:10:51.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f91f7fff640 1 -- 172.21.15.60:0/3678764687 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921000dbb0 con 0x556fc78762f0 2024-04-25T10:10:51.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f91f7fff640 1 -- 172.21.15.60:0/3678764687 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9210032020 con 0x556fc78762f0 2024-04-25T10:10:51.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 10 client.7308 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:10:51.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 10 client.7308 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:10:51.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 20 client.7308 populate_metadata read hostname 'smithi060' 2024-04-25T10:10:51.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 --2- 172.21.15.60:0/3678764687 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x556fc7885c80 0x556fc7888070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 4 client.7308 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:51.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f92228ff000 1 -- 172.21.15.60:0/3678764687 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556fc7614c60 con 0x556fc7885c80 2024-04-25T10:10:51.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.949+0000 7f921d52e640 1 --2- 172.21.15.60:0/3678764687 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x556fc7885c80 0x556fc7888070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f91f7fff640 10 client.7308 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:51.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f921d52e640 1 --2- 172.21.15.60:0/3678764687 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x556fc7885c80 0x556fc7888070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8118e04b0c26672 server_cookie=acc4b09a973cda91 in_seq=0 out_seq=0 2024-04-25T10:10:51.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f91f7fff640 1 -- 172.21.15.60:0/3678764687 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556fc7614c60 con 0x556fc7885c80 2024-04-25T10:10:51.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f91f7fff640 10 client.7308 handle_command_reply: tid=0 2024-04-25T10:10:51.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f92228ff000 1 client.7308 shutdown 2024-04-25T10:10:51.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f92228ff000 1 -- 172.21.15.60:0/3678764687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc78762f0 msgr2=0x556fc785b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f92228ff000 1 --2- 172.21.15.60:0/3678764687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc78762f0 0x556fc785b270 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x556fc76353d0 tx=0x7f9210004740 comp rx=0 tx=0).stop 2024-04-25T10:10:51.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f92228ff000 1 -- 172.21.15.60:0/3678764687 shutdown_connections 2024-04-25T10:10:51.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f92228ff000 1 --2- 172.21.15.60:0/3678764687 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x556fc7885c80 0x556fc7888070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f92228ff000 1 --2- 172.21.15.60:0/3678764687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc78762f0 0x556fc785b270 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f92228ff000 1 --2- 172.21.15.60:0/3678764687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78759d0 0x556fc785ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f91f7fff640 0 client.7308 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:51.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f92228ff000 1 --2- 172.21.15.60:0/3678764687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc782c0a0 0x556fc785a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f92228ff000 1 -- 172.21.15.60:0/3678764687 >> 172.21.15.60:0/3678764687 conn(0x556fc7828020 msgr2=0x556fc782aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:51.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f92228ff000 1 -- 172.21.15.60:0/3678764687 shutdown_connections 2024-04-25T10:10:51.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f92228ff000 1 -- 172.21.15.60:0/3678764687 wait complete. 2024-04-25T10:10:51.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.953+0000 7f92228ff000 20 client.7308 trim_cache size 0 max 16384 2024-04-25T10:10:51.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 Processor -- start 2024-04-25T10:10:51.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 -- start start 2024-04-25T10:10:51.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc782c0a0 0x556fc7874e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc78759d0 0x556fc782b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc78762f0 0x556fc782b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9218202c20 con 0x556fc78759d0 2024-04-25T10:10:51.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9218152d10 con 0x556fc78762f0 2024-04-25T10:10:51.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9218152110 con 0x556fc782c0a0 2024-04-25T10:10:51.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f921cd2d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc78759d0 0x556fc782b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f921cd2d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc78759d0 0x556fc782b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57550/0 (socket says 172.21.15.60:57550) 2024-04-25T10:10:51.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f921cd2d640 1 -- 172.21.15.60:0/668100004 learned_addr learned my addr 172.21.15.60:0/668100004 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:51.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f921cd2d640 1 -- 172.21.15.60:0/668100004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc782c0a0 msgr2=0x556fc7874e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:51.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f921cd2d640 1 --2- 172.21.15.60:0/668100004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc782c0a0 0x556fc7874e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f921cd2d640 1 -- 172.21.15.60:0/668100004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc78762f0 msgr2=0x556fc782b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:51.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f921cd2d640 1 --2- 172.21.15.60:0/668100004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc78762f0 0x556fc782b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f921cd2d640 1 -- 172.21.15.60:0/668100004 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fc76bfde0 con 0x556fc78759d0 2024-04-25T10:10:51.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f921cd2d640 1 --2- 172.21.15.60:0/668100004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc78759d0 0x556fc782b420 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f9214055ea0 tx=0x7f9214055c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb2ae3eeed35b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:51.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92027fc640 1 -- 172.21.15.60:0/668100004 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214082070 con 0x556fc78759d0 2024-04-25T10:10:51.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92027fc640 1 -- 172.21.15.60:0/668100004 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921400ddb0 con 0x556fc78759d0 2024-04-25T10:10:51.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92027fc640 1 -- 172.21.15.60:0/668100004 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214052810 con 0x556fc78759d0 2024-04-25T10:10:51.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 -- 172.21.15.60:0/668100004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc78759d0 msgr2=0x556fc782b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 --2- 172.21.15.60:0/668100004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc78759d0 0x556fc782b420 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f9214055ea0 tx=0x7f9214055c30 comp rx=0 tx=0).stop 2024-04-25T10:10:51.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 -- 172.21.15.60:0/668100004 shutdown_connections 2024-04-25T10:10:51.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 --2- 172.21.15.60:0/668100004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc78762f0 0x556fc782b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 --2- 172.21.15.60:0/668100004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc78759d0 0x556fc782b420 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 --2- 172.21.15.60:0/668100004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc782c0a0 0x556fc7874e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 -- 172.21.15.60:0/668100004 >> 172.21.15.60:0/668100004 conn(0x556fc7828020 msgr2=0x556fc78740d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:51.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 -- 172.21.15.60:0/668100004 shutdown_connections 2024-04-25T10:10:51.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 -- 172.21.15.60:0/668100004 wait complete. 2024-04-25T10:10:51.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 -- start start 2024-04-25T10:10:51.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 Processor -- start 2024-04-25T10:10:51.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc782c0a0 0x556fc78d6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc78759d0 0x556fc78d6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.969+0000 7f92228ff000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78762f0 0x556fc7888380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc7613e00 con 0x556fc782c0a0 2024-04-25T10:10:51.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9218152710 con 0x556fc78759d0 2024-04-25T10:10:51.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556fc7678300 con 0x556fc78762f0 2024-04-25T10:10:51.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 10 client.0 _ll_register_callbacks cb 0x556fc77f7040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:51.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921d52e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc782c0a0 0x556fc78d6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921d52e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc782c0a0 0x556fc78d6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57560/0 (socket says 172.21.15.60:57560) 2024-04-25T10:10:51.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921d52e640 1 -- 172.21.15.60:0/54369283 learned_addr learned my addr 172.21.15.60:0/54369283 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:51.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921cd2d640 1 --2- 172.21.15.60:0/54369283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc78759d0 0x556fc78d6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921dd2f640 1 --2- 172.21.15.60:0/54369283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78762f0 0x556fc7888380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921d52e640 1 -- 172.21.15.60:0/54369283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78762f0 msgr2=0x556fc7888380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:51.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921d52e640 1 --2- 172.21.15.60:0/54369283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78762f0 0x556fc7888380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921d52e640 1 -- 172.21.15.60:0/54369283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc78759d0 msgr2=0x556fc78d6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:51.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921d52e640 1 --2- 172.21.15.60:0/54369283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc78759d0 0x556fc78d6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921d52e640 1 -- 172.21.15.60:0/54369283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556fc761a8e0 con 0x556fc782c0a0 2024-04-25T10:10:51.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921dd2f640 1 --2- 172.21.15.60:0/54369283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78762f0 0x556fc7888380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:51.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921d52e640 1 --2- 172.21.15.60:0/54369283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc782c0a0 0x556fc78d6310 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f920c01cdc0 tx=0x7f920c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506b672134d30ba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:51.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f91f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:10:51.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f91f7fff640 1 -- 172.21.15.60:0/54369283 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c0b0040 con 0x556fc782c0a0 2024-04-25T10:10:51.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f91f7fff640 1 -- 172.21.15.60:0/54369283 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f920c0aba10 con 0x556fc782c0a0 2024-04-25T10:10:51.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 -- 172.21.15.60:0/54369283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f921400ddb0 con 0x556fc782c0a0 2024-04-25T10:10:51.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f91f7fff640 1 -- 172.21.15.60:0/54369283 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f920c0d6050 con 0x556fc782c0a0 2024-04-25T10:10:51.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 10 client.7314 fetch_fsmap learned FSMap version 46 2024-04-25T10:10:51.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 -- 172.21.15.60:0/54369283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x556fc76d28f0 con 0x556fc782c0a0 2024-04-25T10:10:51.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f91f7fff640 1 -- 172.21.15.60:0/54369283 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921400ddb0 con 0x556fc782c0a0 2024-04-25T10:10:51.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f91f7fff640 1 -- 172.21.15.60:0/54369283 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f920c0d9020 con 0x556fc782c0a0 2024-04-25T10:10:51.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 10 client.7314 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:10:51.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 10 client.7314 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:10:51.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 20 client.7314 populate_metadata read hostname 'smithi060' 2024-04-25T10:10:51.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 --2- 172.21.15.60:0/54369283 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x556fc78890e0 0x556fc78894f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 4 client.7314 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:10:51.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 -- 172.21.15.60:0/54369283 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556fc7686530 con 0x556fc78890e0 2024-04-25T10:10:51.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921cd2d640 1 --2- 172.21.15.60:0/54369283 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x556fc78890e0 0x556fc78894f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f91f7fff640 10 client.7314 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:51.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921cd2d640 1 --2- 172.21.15.60:0/54369283 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x556fc78890e0 0x556fc78894f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98706cd71833e30e server_cookie=4d3d23519cc03fe2 in_seq=0 out_seq=0 2024-04-25T10:10:51.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f91f7fff640 1 -- 172.21.15.60:0/54369283 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556fc7686530 con 0x556fc78890e0 2024-04-25T10:10:51.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f91f7fff640 10 client.7314 handle_command_reply: tid=0 2024-04-25T10:10:51.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 client.7314 shutdown 2024-04-25T10:10:51.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 -- 172.21.15.60:0/54369283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc782c0a0 msgr2=0x556fc78d6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 --2- 172.21.15.60:0/54369283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc782c0a0 0x556fc78d6310 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f920c01cdc0 tx=0x7f920c0192c0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 -- 172.21.15.60:0/54369283 shutdown_connections 2024-04-25T10:10:51.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 --2- 172.21.15.60:0/54369283 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x556fc78890e0 0x556fc78894f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 --2- 172.21.15.60:0/54369283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556fc78762f0 0x556fc7888380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 --2- 172.21.15.60:0/54369283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556fc78759d0 0x556fc78d6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 --2- 172.21.15.60:0/54369283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556fc782c0a0 0x556fc78d6310 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f91f7fff640 0 client.7314 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:51.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 -- 172.21.15.60:0/54369283 >> 172.21.15.60:0/54369283 conn(0x556fc7828020 msgr2=0x556fc7874960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:51.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 -- 172.21.15.60:0/54369283 shutdown_connections 2024-04-25T10:10:51.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 1 -- 172.21.15.60:0/54369283 wait complete. 2024-04-25T10:10:51.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f92228ff000 20 client.7314 trim_cache size 0 max 16384 2024-04-25T10:10:51.988 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:10:51.988 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T10:10:51.988 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T10:10:51.988 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:10:51.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921f792640 1 -- 172.21.15.60:0/1395049228 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f91fc066f60 msgr2=0x7f91fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921f792640 1 --2- 172.21.15.60:0/1395049228 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f91fc066f60 0x7f91fc069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f920c0ad000 tx=0x7f920c008040 comp rx=0 tx=0).stop 2024-04-25T10:10:51.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921f792640 1 -- 172.21.15.60:0/1395049228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92180d0320 msgr2=0x7f921814b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921f792640 1 --2- 172.21.15.60:0/1395049228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92180d0320 0x7f921814b4b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f921400cf90 tx=0x7f921400a7d0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921f792640 1 -- 172.21.15.60:0/1395049228 shutdown_connections 2024-04-25T10:10:51.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921f792640 1 --2- 172.21.15.60:0/1395049228 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f91fc066f60 0x7f91fc069420 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921f792640 1 --2- 172.21.15.60:0/1395049228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f921815de90 0x7f92180c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921f792640 1 --2- 172.21.15.60:0/1395049228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92180d0320 0x7f921814b4b0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921f792640 1 --2- 172.21.15.60:0/1395049228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92180ce290 0x7f92180c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921f792640 1 -- 172.21.15.60:0/1395049228 >> 172.21.15.60:0/1395049228 conn(0x7f921806c090 msgr2=0x7f921814d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:51.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921f792640 1 -- 172.21.15.60:0/1395049228 shutdown_connections 2024-04-25T10:10:51.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:51.973+0000 7f921f792640 1 -- 172.21.15.60:0/1395049228 wait complete. 2024-04-25T10:10:51.999 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:10:52.000 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:52.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.101+0000 7f53b0c44640 1 Processor -- start 2024-04-25T10:10:52.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.101+0000 7f53b0c44640 1 -- start start 2024-04-25T10:10:52.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53b0c44640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac0ce7a0 0x7f53ac0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53b0c44640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f53ac15bc90 0x7f53ac0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53b0c44640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53ac0cf0c0 con 0x7f53ac0b4660 2024-04-25T10:10:52.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53b0c44640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ac0cc370 con 0x7f53ac15bc90 2024-04-25T10:10:52.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53b0c44640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac071ca0 con 0x7f53ac0ce7a0 2024-04-25T10:10:52.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53a9d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f53ac15bc90 0x7f53ac0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53aa59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac0ce7a0 0x7f53ac0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53a9d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f53ac15bc90 0x7f53ac0b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:45624/0 (socket says 172.21.15.60:45624) 2024-04-25T10:10:52.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53aa59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac0ce7a0 0x7f53ac0ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:57570/0 (socket says 172.21.15.60:57570) 2024-04-25T10:10:52.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53a9d9b640 1 -- 172.21.15.60:0/600637750 learned_addr learned my addr 172.21.15.60:0/600637750 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:52.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53a9d9b640 1 -- 172.21.15.60:0/600637750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac0ce7a0 msgr2=0x7f53ac0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53a9d9b640 1 --2- 172.21.15.60:0/600637750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac0ce7a0 0x7f53ac0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53a9d9b640 1 -- 172.21.15.60:0/600637750 >> v1:172.21.15.100:6789/0 conn(0x7f53ac0b4660 legacy=0x7f53ac0b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53a9d9b640 1 -- 172.21.15.60:0/600637750 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac0cca40 con 0x7f53ac15bc90 2024-04-25T10:10:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53aa59c640 1 --2- 172.21.15.60:0/600637750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac0ce7a0 0x7f53ac0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:10:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53a9d9b640 1 --2- 172.21.15.60:0/600637750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ac15bc90 0x7f53ac0b25d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f53a00098a0 tx=0x7f53a001c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53e72de50d50cfdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.105+0000 7f53a959a640 1 -- 172.21.15.60:0/600637750 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a00b0070 con 0x7f53ac15bc90 2024-04-25T10:10:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53a959a640 1 -- 172.21.15.60:0/600637750 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a00ab600 con 0x7f53ac15bc90 2024-04-25T10:10:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53a959a640 1 -- 172.21.15.60:0/600637750 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a00ab9d0 con 0x7f53ac15bc90 2024-04-25T10:10:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53b0c44640 1 -- 172.21.15.60:0/600637750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ac15bc90 msgr2=0x7f53ac0b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53b0c44640 1 --2- 172.21.15.60:0/600637750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ac15bc90 0x7f53ac0b25d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f53a00098a0 tx=0x7f53a001c540 comp rx=0 tx=0).stop 2024-04-25T10:10:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53b0c44640 1 -- 172.21.15.60:0/600637750 shutdown_connections 2024-04-25T10:10:52.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53b0c44640 1 --2- 172.21.15.60:0/600637750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ac15bc90 0x7f53ac0b25d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53b0c44640 1 --2- 172.21.15.60:0/600637750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac0ce7a0 0x7f53ac0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53b0c44640 1 -- 172.21.15.60:0/600637750 >> 172.21.15.60:0/600637750 conn(0x7f53ac0b12d0 msgr2=0x7f53ac0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53b0c44640 1 -- 172.21.15.60:0/600637750 shutdown_connections 2024-04-25T10:10:52.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53b0c44640 1 -- 172.21.15.60:0/600637750 wait complete. 2024-04-25T10:10:52.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53b0c44640 1 Processor -- start 2024-04-25T10:10:52.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53b0c44640 1 -- start start 2024-04-25T10:10:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53b0c44640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac0ce7a0 0x7f53ac150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53b0c44640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac15bc90 0x7f53ac13bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53b0c44640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ac14b4d0 0x7f53ac158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53b0c44640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac0b4280 con 0x7f53ac0ce7a0 2024-04-25T10:10:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53b0c44640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac0b3db0 con 0x7f53ac14b4d0 2024-04-25T10:10:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53b0c44640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac0b4400 con 0x7f53ac15bc90 2024-04-25T10:10:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53aa59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac0ce7a0 0x7f53ac150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53aa59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac0ce7a0 0x7f53ac150230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57572/0 (socket says 172.21.15.60:57572) 2024-04-25T10:10:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53aa59c640 1 -- 172.21.15.60:0/2266017442 learned_addr learned my addr 172.21.15.60:0/2266017442 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53aad9d640 1 --2- 172.21.15.60:0/2266017442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ac14b4d0 0x7f53ac158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53a9d9b640 1 --2- 172.21.15.60:0/2266017442 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac15bc90 0x7f53ac13bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53aa59c640 1 -- 172.21.15.60:0/2266017442 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac15bc90 msgr2=0x7f53ac13bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53aa59c640 1 --2- 172.21.15.60:0/2266017442 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac15bc90 0x7f53ac13bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53aa59c640 1 -- 172.21.15.60:0/2266017442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ac14b4d0 msgr2=0x7f53ac158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53aa59c640 1 --2- 172.21.15.60:0/2266017442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ac14b4d0 0x7f53ac158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.109+0000 7f53aa59c640 1 -- 172.21.15.60:0/2266017442 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac151d20 con 0x7f53ac0ce7a0 2024-04-25T10:10:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.113+0000 7f53aa59c640 1 --2- 172.21.15.60:0/2266017442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac0ce7a0 0x7f53ac150230 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f539c00e980 tx=0x7f539c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.113+0000 7f539affd640 1 -- 172.21.15.60:0/2266017442 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c017070 con 0x7f53ac0ce7a0 2024-04-25T10:10:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.113+0000 7f539affd640 1 -- 172.21.15.60:0/2266017442 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539c00ca00 con 0x7f53ac0ce7a0 2024-04-25T10:10:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.113+0000 7f53b0c44640 1 -- 172.21.15.60:0/2266017442 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ac152000 con 0x7f53ac0ce7a0 2024-04-25T10:10:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.113+0000 7f539affd640 1 -- 172.21.15.60:0/2266017442 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c00cdd0 con 0x7f53ac0ce7a0 2024-04-25T10:10:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.113+0000 7f53b0c44640 1 -- 172.21.15.60:0/2266017442 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ac159430 con 0x7f53ac0ce7a0 2024-04-25T10:10:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.113+0000 7f53b0c44640 1 -- 172.21.15.60:0/2266017442 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ac074770 con 0x7f53ac0ce7a0 2024-04-25T10:10:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.113+0000 7f539affd640 1 -- 172.21.15.60:0/2266017442 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f539c01a070 con 0x7f53ac0ce7a0 2024-04-25T10:10:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.113+0000 7f539affd640 1 --2- 172.21.15.60:0/2266017442 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f538c066f60 0x7f538c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.113+0000 7f539affd640 1 -- 172.21.15.60:0/2266017442 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f539c085300 con 0x7f53ac0ce7a0 2024-04-25T10:10:52.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.113+0000 7f53a9d9b640 1 --2- 172.21.15.60:0/2266017442 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f538c066f60 0x7f538c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.113+0000 7f53a9d9b640 1 --2- 172.21.15.60:0/2266017442 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f538c066f60 0x7f538c069420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009870 tx=0x7f53a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.117+0000 7f539affd640 1 -- 172.21.15.60:0/2266017442 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f539c056310 con 0x7f53ac0ce7a0 2024-04-25T10:10:52.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.309+0000 7f53b0c44640 1 -- 172.21.15.60:0/2266017442 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53ac155460 con 0x7f53ac0ce7a0 2024-04-25T10:10:52.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.309+0000 7f539affd640 1 -- 172.21.15.60:0/2266017442 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f539c055c10 con 0x7f53ac0ce7a0 2024-04-25T10:10:52.317 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:10:52.317 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3185787428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3185787428},{"type":"v1","addr":"172.21.15.100:6836","nonce":3185787428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2667415244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2667415244},{"type":"v1","addr":"172.21.15.100:6837","nonce":2667415244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/56747111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":56747111},{"type":"v1","addr":"172.21.15.60:6834","nonce":56747111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:43.707831+0000","modified":"2024-04-25T10:10:14.475101+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7134},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7134":{"gid":7134,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":6,"addr":"172.21.15.60:6835/1295867731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1295867731},{"type":"v1","addr":"172.21.15.60:6835","nonce":1295867731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:10:52.317 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 46 2024-04-25T10:10:52.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.309+0000 7f53b0c44640 1 -- 172.21.15.60:0/2266017442 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f538c066f60 msgr2=0x7f538c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.309+0000 7f53b0c44640 1 --2- 172.21.15.60:0/2266017442 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f538c066f60 0x7f538c069420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009870 tx=0x7f53a0008040 comp rx=0 tx=0).stop 2024-04-25T10:10:52.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.309+0000 7f53b0c44640 1 -- 172.21.15.60:0/2266017442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac0ce7a0 msgr2=0x7f53ac150230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.309+0000 7f53b0c44640 1 --2- 172.21.15.60:0/2266017442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac0ce7a0 0x7f53ac150230 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f539c00e980 tx=0x7f539c00ee40 comp rx=0 tx=0).stop 2024-04-25T10:10:52.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.309+0000 7f53b0c44640 1 -- 172.21.15.60:0/2266017442 shutdown_connections 2024-04-25T10:10:52.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.309+0000 7f53b0c44640 1 --2- 172.21.15.60:0/2266017442 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f538c066f60 0x7f538c069420 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.309+0000 7f53b0c44640 1 --2- 172.21.15.60:0/2266017442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ac14b4d0 0x7f53ac158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.309+0000 7f53b0c44640 1 --2- 172.21.15.60:0/2266017442 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac15bc90 0x7f53ac13bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.309+0000 7f53b0c44640 1 --2- 172.21.15.60:0/2266017442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac0ce7a0 0x7f53ac150230 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.309+0000 7f53b0c44640 1 -- 172.21.15.60:0/2266017442 >> 172.21.15.60:0/2266017442 conn(0x7f53ac0b12d0 msgr2=0x7f53ac0cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.309+0000 7f53b0c44640 1 -- 172.21.15.60:0/2266017442 shutdown_connections 2024-04-25T10:10:52.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.309+0000 7f53b0c44640 1 -- 172.21.15.60:0/2266017442 wait complete. 2024-04-25T10:10:52.334 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T10:10:52.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.437+0000 7f66858c0640 1 Processor -- start 2024-04-25T10:10:52.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.437+0000 7f66858c0640 1 -- start start 2024-04-25T10:10:52.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.441+0000 7f66858c0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f66800ceef0 0x7f66800d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.441+0000 7f66858c0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668015bad0 con 0x7f66801620d0 2024-04-25T10:10:52.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.441+0000 7f66858c0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668015ebe0 con 0x7f668015b6f0 2024-04-25T10:10:52.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.441+0000 7f66858c0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6680161c40 con 0x7f66800ceef0 2024-04-25T10:10:52.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.441+0000 7f667e7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f66801620d0 0x7f66800ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34424/0 (socket says 172.21.15.60:34424) 2024-04-25T10:10:52.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.441+0000 7f667e7fc640 1 -- 172.21.15.60:0/3537066724 learned_addr learned my addr 172.21.15.60:0/3537066724 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:10:52.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.441+0000 7f667f7fe640 1 --2- 172.21.15.60:0/3537066724 >> v2:172.21.15.60:3300/0 conn(0x7f66800ceef0 0x7f66800d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.441+0000 7f667dffb640 1 -- 172.21.15.60:0/3537066724 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283322872 0 0) 0x7f668015bad0 con 0x7f66801620d0 2024-04-25T10:10:52.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.441+0000 7f667dffb640 1 -- 172.21.15.60:0/3537066724 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6668003530 con 0x7f66801620d0 2024-04-25T10:10:52.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.441+0000 7f667f7fe640 1 -- 172.21.15.60:0/3537066724 >> v1:172.21.15.100:6789/0 conn(0x7f668015b6f0 legacy=0x7f668015e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.441+0000 7f667f7fe640 1 -- 172.21.15.60:0/3537066724 >> v1:172.21.15.60:6789/0 conn(0x7f66801620d0 legacy=0x7f66800ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.441+0000 7f667f7fe640 1 -- 172.21.15.60:0/3537066724 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668015eec0 con 0x7f66800ceef0 2024-04-25T10:10:52.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.441+0000 7f667f7fe640 1 --2- 172.21.15.60:0/3537066724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66800ceef0 0x7f66800d1b30 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f6678009d90 tx=0x7f667801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3916af5f9c4d8f9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.441+0000 7f667dffb640 1 -- 172.21.15.60:0/3537066724 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66780b2070 con 0x7f66800ceef0 2024-04-25T10:10:52.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.441+0000 7f667dffb640 1 -- 172.21.15.60:0/3537066724 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66780a5900 con 0x7f66800ceef0 2024-04-25T10:10:52.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.441+0000 7f667dffb640 1 -- 172.21.15.60:0/3537066724 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66780a5cb0 con 0x7f66800ceef0 2024-04-25T10:10:52.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 -- 172.21.15.60:0/3537066724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66800ceef0 msgr2=0x7f66800d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 --2- 172.21.15.60:0/3537066724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66800ceef0 0x7f66800d1b30 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f6678009d90 tx=0x7f667801c930 comp rx=0 tx=0).stop 2024-04-25T10:10:52.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 -- 172.21.15.60:0/3537066724 shutdown_connections 2024-04-25T10:10:52.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 --2- 172.21.15.60:0/3537066724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66800ceef0 0x7f66800d1b30 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 -- 172.21.15.60:0/3537066724 >> 172.21.15.60:0/3537066724 conn(0x7f6680072340 msgr2=0x7f6680072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 -- 172.21.15.60:0/3537066724 shutdown_connections 2024-04-25T10:10:52.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 -- 172.21.15.60:0/3537066724 wait complete. 2024-04-25T10:10:52.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 Processor -- start 2024-04-25T10:10:52.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 -- start start 2024-04-25T10:10:52.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66800ceef0 0x7f668014e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668015b6f0 0x7f668014f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66801620d0 0x7f668014fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668015e960 con 0x7f668015b6f0 2024-04-25T10:10:52.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668015e7e0 con 0x7f66801620d0 2024-04-25T10:10:52.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668016ba70 con 0x7f66800ceef0 2024-04-25T10:10:52.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f667e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668015b6f0 0x7f668014f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f667e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668015b6f0 0x7f668014f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57582/0 (socket says 172.21.15.60:57582) 2024-04-25T10:10:52.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f667e7fc640 1 -- 172.21.15.60:0/1492038942 learned_addr learned my addr 172.21.15.60:0/1492038942 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:52.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f667f7fe640 1 --2- 172.21.15.60:0/1492038942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66801620d0 0x7f668014fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f667effd640 1 --2- 172.21.15.60:0/1492038942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66800ceef0 0x7f668014e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f667e7fc640 1 -- 172.21.15.60:0/1492038942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66800ceef0 msgr2=0x7f668014e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f667e7fc640 1 --2- 172.21.15.60:0/1492038942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66800ceef0 0x7f668014e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f667e7fc640 1 -- 172.21.15.60:0/1492038942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66801620d0 msgr2=0x7f668014fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f667e7fc640 1 --2- 172.21.15.60:0/1492038942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66801620d0 0x7f668014fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f667e7fc640 1 -- 172.21.15.60:0/1492038942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66800ce040 con 0x7f668015b6f0 2024-04-25T10:10:52.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f667f7fe640 1 --2- 172.21.15.60:0/1492038942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66801620d0 0x7f668014fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:52.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f667effd640 1 --2- 172.21.15.60:0/1492038942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66800ceef0 0x7f668014e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:52.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f667e7fc640 1 --2- 172.21.15.60:0/1492038942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668015b6f0 0x7f668014f780 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f667400c3c0 tx=0x7f667400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f6667fff640 1 -- 172.21.15.60:0/1492038942 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674017070 con 0x7f668015b6f0 2024-04-25T10:10:52.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f6667fff640 1 -- 172.21.15.60:0/1492038942 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6674005cf0 con 0x7f668015b6f0 2024-04-25T10:10:52.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f6667fff640 1 -- 172.21.15.60:0/1492038942 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66740072f0 con 0x7f668015b6f0 2024-04-25T10:10:52.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 -- 172.21.15.60:0/1492038942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66800b5690 con 0x7f668015b6f0 2024-04-25T10:10:52.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.445+0000 7f66858c0640 1 -- 172.21.15.60:0/1492038942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66800b5c20 con 0x7f668015b6f0 2024-04-25T10:10:52.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f6667fff640 1 -- 172.21.15.60:0/1492038942 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f667401a070 con 0x7f668015b6f0 2024-04-25T10:10:52.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f6667fff640 1 --2- 172.21.15.60:0/1492038942 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f666c066f60 0x7f666c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f6667fff640 1 -- 172.21.15.60:0/1492038942 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6674084490 con 0x7f668015b6f0 2024-04-25T10:10:52.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f6688a2d000 1 Processor -- start 2024-04-25T10:10:52.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f667effd640 1 --2- 172.21.15.60:0/1492038942 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f666c066f60 0x7f666c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f6688a2d000 1 -- start start 2024-04-25T10:10:52.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f6688a2d000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc92e0 0x55bc82dc96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f6688a2d000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc9c00 0x55bc82d7f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f6688a2d000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 0x55bc82d7fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f6688a2d000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc82c66ba0 con 0x55bc82dc9c00 2024-04-25T10:10:52.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f6688a2d000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc82bcc090 con 0x55bc82d7f8e0 2024-04-25T10:10:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f6688a2d000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc82b54160 con 0x55bc82dc92e0 2024-04-25T10:10:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f667effd640 1 --2- 172.21.15.60:0/1492038942 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f666c066f60 0x7f666c069420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6670003e40 tx=0x7f6670009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f667e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc9c00 0x55bc82d7f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f667e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc9c00 0x55bc82d7f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57588/0 (socket says 172.21.15.60:57588) 2024-04-25T10:10:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f667e7fc640 1 -- 172.21.15.60:0/4154399049 learned_addr learned my addr 172.21.15.60:0/4154399049 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f667effd640 1 --2- 172.21.15.60:0/4154399049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc92e0 0x55bc82dc96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f667f7fe640 1 --2- 172.21.15.60:0/4154399049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 0x55bc82d7fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f667e7fc640 1 -- 172.21.15.60:0/4154399049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc92e0 msgr2=0x55bc82dc96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f667e7fc640 1 --2- 172.21.15.60:0/4154399049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc92e0 0x55bc82dc96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f667e7fc640 1 -- 172.21.15.60:0/4154399049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 msgr2=0x55bc82d7fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f667e7fc640 1 --2- 172.21.15.60:0/4154399049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 0x55bc82d7fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f667e7fc640 1 -- 172.21.15.60:0/4154399049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc82c75c20 con 0x55bc82dc9c00 2024-04-25T10:10:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f667f7fe640 1 --2- 172.21.15.60:0/4154399049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 0x55bc82d7fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f667effd640 1 --2- 172.21.15.60:0/4154399049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc92e0 0x55bc82dc96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:52.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f667e7fc640 1 --2- 172.21.15.60:0/4154399049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc9c00 0x55bc82d7f3a0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f667405cee0 tx=0x7f6674054dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db03f73fd6aea9a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f6665ffb640 1 -- 172.21.15.60:0/4154399049 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674052da0 con 0x55bc82dc9c00 2024-04-25T10:10:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f6665ffb640 1 -- 172.21.15.60:0/4154399049 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6674053000 con 0x55bc82dc9c00 2024-04-25T10:10:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.449+0000 7f6665ffb640 1 -- 172.21.15.60:0/4154399049 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667405cf80 con 0x55bc82dc9c00 2024-04-25T10:10:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 -- 172.21.15.60:0/4154399049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc9c00 msgr2=0x55bc82d7f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 --2- 172.21.15.60:0/4154399049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc9c00 0x55bc82d7f3a0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f667405cee0 tx=0x7f6674054dc0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 -- 172.21.15.60:0/4154399049 shutdown_connections 2024-04-25T10:10:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 --2- 172.21.15.60:0/4154399049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 0x55bc82d7fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 --2- 172.21.15.60:0/4154399049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc9c00 0x55bc82d7f3a0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 --2- 172.21.15.60:0/4154399049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc92e0 0x55bc82dc96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 -- 172.21.15.60:0/4154399049 >> 172.21.15.60:0/4154399049 conn(0x55bc82d7b890 msgr2=0x55bc82d7dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 -- 172.21.15.60:0/4154399049 shutdown_connections 2024-04-25T10:10:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 -- 172.21.15.60:0/4154399049 wait complete. 2024-04-25T10:10:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 -- start start 2024-04-25T10:10:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 Processor -- start 2024-04-25T10:10:52.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 0x55bc82dd0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc92e0 0x55bc82dd0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc9c00 0x55bc82db7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc82bc8130 con 0x55bc82dc9c00 2024-04-25T10:10:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc82b90130 con 0x55bc82d7f8e0 2024-04-25T10:10:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc82c8a300 con 0x55bc82dc92e0 2024-04-25T10:10:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 10 client.0 _ll_register_callbacks cb 0x55bc82d6abc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f667f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc9c00 0x55bc82db7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f667f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc9c00 0x55bc82db7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57598/0 (socket says 172.21.15.60:57598) 2024-04-25T10:10:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f667f7fe640 1 -- 172.21.15.60:0/1459362608 learned_addr learned my addr 172.21.15.60:0/1459362608 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f667e7fc640 1 --2- 172.21.15.60:0/1459362608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc92e0 0x55bc82dd0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f667effd640 1 --2- 172.21.15.60:0/1459362608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 0x55bc82dd0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f667f7fe640 1 -- 172.21.15.60:0/1459362608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc92e0 msgr2=0x55bc82dd0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f667f7fe640 1 --2- 172.21.15.60:0/1459362608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc92e0 0x55bc82dd0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f667f7fe640 1 -- 172.21.15.60:0/1459362608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 msgr2=0x55bc82dd0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f667f7fe640 1 --2- 172.21.15.60:0/1459362608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 0x55bc82dd0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f667f7fe640 1 -- 172.21.15.60:0/1459362608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc82ba9060 con 0x55bc82dc9c00 2024-04-25T10:10:52.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f667e7fc640 1 --2- 172.21.15.60:0/1459362608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc92e0 0x55bc82dd0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:52.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f667f7fe640 1 --2- 172.21.15.60:0/1459362608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc9c00 0x55bc82db7090 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f667800c090 tx=0x7f667801b170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd7a2c22ca21a3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f665e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:10:52.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f665e7fc640 1 -- 172.21.15.60:0/1459362608 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66780be070 con 0x55bc82dc9c00 2024-04-25T10:10:52.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f665e7fc640 1 -- 172.21.15.60:0/1459362608 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f66780b8450 con 0x55bc82dc9c00 2024-04-25T10:10:52.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 -- 172.21.15.60:0/1459362608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6674053000 con 0x55bc82dc9c00 2024-04-25T10:10:52.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f665e7fc640 1 -- 172.21.15.60:0/1459362608 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66780c9050 con 0x55bc82dc9c00 2024-04-25T10:10:52.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 10 client.7332 fetch_fsmap learned FSMap version 46 2024-04-25T10:10:52.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 -- 172.21.15.60:0/1459362608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x55bc82b6d7c0 con 0x55bc82dc9c00 2024-04-25T10:10:52.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f665e7fc640 1 -- 172.21.15.60:0/1459362608 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6674053000 con 0x55bc82dc9c00 2024-04-25T10:10:52.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f665e7fc640 1 -- 172.21.15.60:0/1459362608 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f66780cc020 con 0x55bc82dc9c00 2024-04-25T10:10:52.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 10 client.7332 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:10:52.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 10 client.7332 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:10:52.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 20 client.7332 populate_metadata read hostname 'smithi060' 2024-04-25T10:10:52.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 --2- 172.21.15.60:0/1459362608 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55bc82db7e30 0x55bc82db8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 4 client.7332 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 -- 172.21.15.60:0/1459362608 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc82b67cd0 con 0x55bc82db7e30 2024-04-25T10:10:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f667effd640 1 --2- 172.21.15.60:0/1459362608 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55bc82db7e30 0x55bc82db8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f667effd640 1 --2- 172.21.15.60:0/1459362608 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55bc82db7e30 0x55bc82db8210 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9537d42733b5ee86 server_cookie=979c3bf745db631a in_seq=0 out_seq=0 2024-04-25T10:10:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f665e7fc640 10 client.7332 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f665e7fc640 1 -- 172.21.15.60:0/1459362608 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc82b67cd0 con 0x55bc82db7e30 2024-04-25T10:10:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f665e7fc640 10 client.7332 handle_command_reply: tid=0 2024-04-25T10:10:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.453+0000 7f6688a2d000 1 client.7332 shutdown 2024-04-25T10:10:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.457+0000 7f6688a2d000 1 -- 172.21.15.60:0/1459362608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc9c00 msgr2=0x55bc82db7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.457+0000 7f6688a2d000 1 --2- 172.21.15.60:0/1459362608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc9c00 0x55bc82db7090 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f667800c090 tx=0x7f667801b170 comp rx=0 tx=0).stop 2024-04-25T10:10:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.457+0000 7f6688a2d000 1 -- 172.21.15.60:0/1459362608 shutdown_connections 2024-04-25T10:10:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.457+0000 7f6688a2d000 1 --2- 172.21.15.60:0/1459362608 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55bc82db7e30 0x55bc82db8210 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.457+0000 7f6688a2d000 1 --2- 172.21.15.60:0/1459362608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc9c00 0x55bc82db7090 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.457+0000 7f6688a2d000 1 --2- 172.21.15.60:0/1459362608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc92e0 0x55bc82dd0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.457+0000 7f665e7fc640 0 client.7332 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.457+0000 7f6688a2d000 1 --2- 172.21.15.60:0/1459362608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 0x55bc82dd0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.457+0000 7f6688a2d000 1 -- 172.21.15.60:0/1459362608 >> 172.21.15.60:0/1459362608 conn(0x55bc82d7b890 msgr2=0x55bc82d7e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.457+0000 7f6688a2d000 1 -- 172.21.15.60:0/1459362608 shutdown_connections 2024-04-25T10:10:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.457+0000 7f6688a2d000 1 -- 172.21.15.60:0/1459362608 wait complete. 2024-04-25T10:10:52.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.457+0000 7f6688a2d000 20 client.7332 trim_cache size 0 max 16384 2024-04-25T10:10:52.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.465+0000 7f6688a2d000 1 Processor -- start 2024-04-25T10:10:52.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.465+0000 7f6688a2d000 1 -- start start 2024-04-25T10:10:52.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82d7f8e0 0x55bc82e25500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82dc92e0 0x55bc82e25a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc9c00 0x55bc82e25f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6680201db0 con 0x55bc82d7f8e0 2024-04-25T10:10:52.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66802011b0 con 0x55bc82dc92e0 2024-04-25T10:10:52.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66802017b0 con 0x55bc82dc9c00 2024-04-25T10:10:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82d7f8e0 0x55bc82e25500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82d7f8e0 0x55bc82e25500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57612/0 (socket says 172.21.15.60:57612) 2024-04-25T10:10:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667effd640 1 -- 172.21.15.60:0/259425386 learned_addr learned my addr 172.21.15.60:0/259425386 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667effd640 1 -- 172.21.15.60:0/259425386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc9c00 msgr2=0x55bc82e25f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667effd640 1 --2- 172.21.15.60:0/259425386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc9c00 0x55bc82e25f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667effd640 1 -- 172.21.15.60:0/259425386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82dc92e0 msgr2=0x55bc82e25a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667effd640 1 --2- 172.21.15.60:0/259425386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82dc92e0 0x55bc82e25a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667effd640 1 -- 172.21.15.60:0/259425386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc82b87fc0 con 0x55bc82d7f8e0 2024-04-25T10:10:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667effd640 1 --2- 172.21.15.60:0/259425386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82d7f8e0 0x55bc82e25500 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f667000dd80 tx=0x7f6670013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259d89bdfd06b4ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f66657fa640 1 -- 172.21.15.60:0/259425386 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66700248c0 con 0x55bc82d7f8e0 2024-04-25T10:10:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f66657fa640 1 -- 172.21.15.60:0/259425386 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667000ddb0 con 0x55bc82d7f8e0 2024-04-25T10:10:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f66657fa640 1 -- 172.21.15.60:0/259425386 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670014b40 con 0x55bc82d7f8e0 2024-04-25T10:10:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 -- 172.21.15.60:0/259425386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82d7f8e0 msgr2=0x55bc82e25500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 --2- 172.21.15.60:0/259425386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82d7f8e0 0x55bc82e25500 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f667000dd80 tx=0x7f6670013000 comp rx=0 tx=0).stop 2024-04-25T10:10:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 -- 172.21.15.60:0/259425386 shutdown_connections 2024-04-25T10:10:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 --2- 172.21.15.60:0/259425386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc9c00 0x55bc82e25f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 --2- 172.21.15.60:0/259425386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82dc92e0 0x55bc82e25a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 --2- 172.21.15.60:0/259425386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82d7f8e0 0x55bc82e25500 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 -- 172.21.15.60:0/259425386 >> 172.21.15.60:0/259425386 conn(0x55bc82d7b890 msgr2=0x55bc82dc8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 -- 172.21.15.60:0/259425386 shutdown_connections 2024-04-25T10:10:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 -- 172.21.15.60:0/259425386 wait complete. 2024-04-25T10:10:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 -- start start 2024-04-25T10:10:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 Processor -- start 2024-04-25T10:10:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 0x55bc82e29720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc92e0 0x55bc82e29c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc9c00 0x55bc82dcd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc82b66e70 con 0x55bc82dc92e0 2024-04-25T10:10:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc82b54a50 con 0x55bc82d7f8e0 2024-04-25T10:10:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc82bcb3a0 con 0x55bc82dc9c00 2024-04-25T10:10:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f6688a2d000 10 client.0 _ll_register_callbacks cb 0x55bc82b78240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc92e0 0x55bc82e29c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc92e0 0x55bc82e29c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57618/0 (socket says 172.21.15.60:57618) 2024-04-25T10:10:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667e7fc640 1 -- 172.21.15.60:0/2465893563 learned_addr learned my addr 172.21.15.60:0/2465893563 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667effd640 1 --2- 172.21.15.60:0/2465893563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 0x55bc82e29720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667f7fe640 1 --2- 172.21.15.60:0/2465893563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc9c00 0x55bc82dcd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667e7fc640 1 -- 172.21.15.60:0/2465893563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc9c00 msgr2=0x55bc82dcd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667e7fc640 1 --2- 172.21.15.60:0/2465893563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc9c00 0x55bc82dcd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667e7fc640 1 -- 172.21.15.60:0/2465893563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 msgr2=0x55bc82e29720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667e7fc640 1 --2- 172.21.15.60:0/2465893563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 0x55bc82e29720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667e7fc640 1 -- 172.21.15.60:0/2465893563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc82c259e0 con 0x55bc82dc92e0 2024-04-25T10:10:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667effd640 1 --2- 172.21.15.60:0/2465893563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 0x55bc82e29720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f665e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:10:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f667e7fc640 1 --2- 172.21.15.60:0/2465893563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc92e0 0x55bc82e29c60 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f667405cee0 tx=0x7f66740827f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f8c813fc4cce7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f665e7fc640 1 -- 172.21.15.60:0/2465893563 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674070030 con 0x55bc82dc92e0 2024-04-25T10:10:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.469+0000 7f665e7fc640 1 -- 172.21.15.60:0/2465893563 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6674053000 con 0x55bc82dc92e0 2024-04-25T10:10:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 1 -- 172.21.15.60:0/2465893563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f667000ddb0 con 0x55bc82dc92e0 2024-04-25T10:10:52.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f665e7fc640 1 -- 172.21.15.60:0/2465893563 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6674051370 con 0x55bc82dc92e0 2024-04-25T10:10:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 10 client.7338 fetch_fsmap learned FSMap version 46 2024-04-25T10:10:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 1 -- 172.21.15.60:0/2465893563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x55bc82b6b630 con 0x55bc82dc92e0 2024-04-25T10:10:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f665e7fc640 1 -- 172.21.15.60:0/2465893563 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667000ddb0 con 0x55bc82dc92e0 2024-04-25T10:10:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f665e7fc640 1 -- 172.21.15.60:0/2465893563 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6674007490 con 0x55bc82dc92e0 2024-04-25T10:10:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 10 client.7338 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:10:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 10 client.7338 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:10:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 20 client.7338 populate_metadata read hostname 'smithi060' 2024-04-25T10:10:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 1 --2- 172.21.15.60:0/2465893563 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55bc82dce2e0 0x55bc82dce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 4 client.7338 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:10:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 1 -- 172.21.15.60:0/2465893563 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55bc82bd95d0 con 0x55bc82dce2e0 2024-04-25T10:10:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f667effd640 1 --2- 172.21.15.60:0/2465893563 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55bc82dce2e0 0x55bc82dce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f665e7fc640 10 client.7338 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f667effd640 1 --2- 172.21.15.60:0/2465893563 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55bc82dce2e0 0x55bc82dce6c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2510c966cdea5472 server_cookie=2eb736d7c206d402 in_seq=0 out_seq=0 2024-04-25T10:10:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f665e7fc640 1 -- 172.21.15.60:0/2465893563 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+235 (crc 0 0 0) 0x55bc82bd95d0 con 0x55bc82dce2e0 2024-04-25T10:10:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f665e7fc640 10 client.7338 handle_command_reply: tid=0 2024-04-25T10:10:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 1 client.7338 shutdown 2024-04-25T10:10:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 1 -- 172.21.15.60:0/2465893563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc92e0 msgr2=0x55bc82e29c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 1 --2- 172.21.15.60:0/2465893563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc92e0 0x55bc82e29c60 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f667405cee0 tx=0x7f66740827f0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 1 -- 172.21.15.60:0/2465893563 shutdown_connections 2024-04-25T10:10:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 1 --2- 172.21.15.60:0/2465893563 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55bc82dce2e0 0x55bc82dce6c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 1 --2- 172.21.15.60:0/2465893563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc82dc9c00 0x55bc82dcd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 1 --2- 172.21.15.60:0/2465893563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc82dc92e0 0x55bc82e29c60 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 1 --2- 172.21.15.60:0/2465893563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc82d7f8e0 0x55bc82e29720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 1 -- 172.21.15.60:0/2465893563 >> 172.21.15.60:0/2465893563 conn(0x55bc82d7b890 msgr2=0x55bc82dc8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f665e7fc640 0 client.7338 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 1 -- 172.21.15.60:0/2465893563 shutdown_connections 2024-04-25T10:10:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 1 -- 172.21.15.60:0/2465893563 wait complete. 2024-04-25T10:10:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f6688a2d000 20 client.7338 trim_cache size 0 max 16384 2024-04-25T10:10:52.488 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:10:52.488 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T10:10:52.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:10:52.489 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "dentry", 2024-04-25T10:10:52.489 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 17397328, 2024-04-25T10:10:52.489 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511627776, 2024-04-25T10:10:52.489 INFO:teuthology.orchestra.run.smithi060.stdout: "frag": "*", 2024-04-25T10:10:52.489 INFO:teuthology.orchestra.run.smithi060.stdout: "dname": "file_to_be_damaged", 2024-04-25T10:10:52.489 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_id": "head", 2024-04-25T10:10:52.489 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-25T10:10:52.489 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:10:52.489 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T10:10:52.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f66858c0640 1 -- 172.21.15.60:0/1492038942 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f666c066f60 msgr2=0x7f666c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f66858c0640 1 --2- 172.21.15.60:0/1492038942 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f666c066f60 0x7f666c069420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6670003e40 tx=0x7f6670009040 comp rx=0 tx=0).stop 2024-04-25T10:10:52.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f66858c0640 1 -- 172.21.15.60:0/1492038942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668015b6f0 msgr2=0x7f668014f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f66858c0640 1 --2- 172.21.15.60:0/1492038942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668015b6f0 0x7f668014f780 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f667400c3c0 tx=0x7f667400c880 comp rx=0 tx=0).stop 2024-04-25T10:10:52.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f66858c0640 1 -- 172.21.15.60:0/1492038942 shutdown_connections 2024-04-25T10:10:52.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f66858c0640 1 --2- 172.21.15.60:0/1492038942 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f666c066f60 0x7f666c069420 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f66858c0640 1 --2- 172.21.15.60:0/1492038942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66801620d0 0x7f668014fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f66858c0640 1 --2- 172.21.15.60:0/1492038942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f668015b6f0 0x7f668014f780 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f66858c0640 1 --2- 172.21.15.60:0/1492038942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66800ceef0 0x7f668014e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f66858c0640 1 -- 172.21.15.60:0/1492038942 >> 172.21.15.60:0/1492038942 conn(0x7f6680072340 msgr2=0x7f66801536a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f66858c0640 1 -- 172.21.15.60:0/1492038942 shutdown_connections 2024-04-25T10:10:52.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.473+0000 7f66858c0640 1 -- 172.21.15.60:0/1492038942 wait complete. 2024-04-25T10:10:52.496 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:10:52.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc198fa0640 1 Processor -- start 2024-04-25T10:10:52.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc198fa0640 1 -- start start 2024-04-25T10:10:52.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc198fa0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc1940cbcf0 0x7fc1940cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc198fa0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc194071dc0 0x7fc1940b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc198fa0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1940cc0d0 con 0x7fc194071250 2024-04-25T10:10:52.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc198fa0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1940cce00 con 0x7fc1940cbcf0 2024-04-25T10:10:52.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc198fa0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1940cf170 con 0x7fc194071dc0 2024-04-25T10:10:52.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc191d9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc194071250 0x7fc1940716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34436/0 (socket says 172.21.15.60:34436) 2024-04-25T10:10:52.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc191d9b640 1 -- 172.21.15.60:0/1265217725 learned_addr learned my addr 172.21.15.60:0/1265217725 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:10:52.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc19259c640 1 --2- 172.21.15.60:0/1265217725 >> v2:172.21.15.100:3300/0 conn(0x7fc1940cbcf0 0x7fc1940cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc192d9d640 1 --2- 172.21.15.60:0/1265217725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc194071dc0 0x7fc1940b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc19159a640 1 -- 172.21.15.60:0/1265217725 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29891767 0 0) 0x7fc1940cc0d0 con 0x7fc194071250 2024-04-25T10:10:52.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc19159a640 1 -- 172.21.15.60:0/1265217725 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc180003530 con 0x7fc194071250 2024-04-25T10:10:52.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc19159a640 1 -- 172.21.15.60:0/1265217725 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1159370432 0 0) 0x7fc180003530 con 0x7fc194071250 2024-04-25T10:10:52.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc19159a640 1 -- 172.21.15.60:0/1265217725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc194071dc0 msgr2=0x7fc1940b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc19159a640 1 --2- 172.21.15.60:0/1265217725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc194071dc0 0x7fc1940b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc19159a640 1 -- 172.21.15.60:0/1265217725 >> v2:172.21.15.100:3300/0 conn(0x7fc1940cbcf0 msgr2=0x7fc1940cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc19159a640 1 --2- 172.21.15.60:0/1265217725 >> v2:172.21.15.100:3300/0 conn(0x7fc1940cbcf0 0x7fc1940cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc19159a640 1 -- 172.21.15.60:0/1265217725 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1940cf450 con 0x7fc194071250 2024-04-25T10:10:52.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc19159a640 1 -- 172.21.15.60:0/1265217725 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fc17c001680 con 0x7fc194071250 2024-04-25T10:10:52.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.593+0000 7fc19159a640 1 -- 172.21.15.60:0/1265217725 >> v1:172.21.15.60:6789/0 conn(0x7fc194071250 legacy=0x7fc1940716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc19159a640 1 --2- 172.21.15.60:0/1265217725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1800039c0 0x7fc180003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc19159a640 1 --2- 172.21.15.60:0/1265217725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc194071dc0 0x7fc180004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc19159a640 1 --2- 172.21.15.60:0/1265217725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc180004d50 0x7fc1800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc19159a640 1 -- 172.21.15.60:0/1265217725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1940cf170 con 0x7fc1800039c0 2024-04-25T10:10:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc19159a640 1 -- 172.21.15.60:0/1265217725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18000b9d0 con 0x7fc180004d50 2024-04-25T10:10:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc19159a640 1 -- 172.21.15.60:0/1265217725 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18000bb50 con 0x7fc194071dc0 2024-04-25T10:10:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc19259c640 1 --2- 172.21.15.60:0/1265217725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1800039c0 0x7fc180003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc19259c640 1 -- 172.21.15.60:0/1265217725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc194071dc0 msgr2=0x7fc180004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc191d9b640 1 --2- 172.21.15.60:0/1265217725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc194071dc0 0x7fc180004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc192d9d640 1 --2- 172.21.15.60:0/1265217725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc180004d50 0x7fc1800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc19259c640 1 --2- 172.21.15.60:0/1265217725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc194071dc0 0x7fc180004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc19259c640 1 -- 172.21.15.60:0/1265217725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc180004d50 msgr2=0x7fc1800051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc19259c640 1 --2- 172.21.15.60:0/1265217725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc180004d50 0x7fc1800051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc19259c640 1 -- 172.21.15.60:0/1265217725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc180003530 con 0x7fc1800039c0 2024-04-25T10:10:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc19259c640 1 --2- 172.21.15.60:0/1265217725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1800039c0 0x7fc180003de0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc184009fc0 tx=0x7fc18401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64b1a7960453fec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc19159a640 1 -- 172.21.15.60:0/1265217725 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1840b2070 con 0x7fc1800039c0 2024-04-25T10:10:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc19159a640 1 -- 172.21.15.60:0/1265217725 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1840a5ab0 con 0x7fc1800039c0 2024-04-25T10:10:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc19159a640 1 -- 172.21.15.60:0/1265217725 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184009c10 con 0x7fc1800039c0 2024-04-25T10:10:52.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc198fa0640 1 -- 172.21.15.60:0/1265217725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1800039c0 msgr2=0x7fc180003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc198fa0640 1 --2- 172.21.15.60:0/1265217725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1800039c0 0x7fc180003de0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc184009fc0 tx=0x7fc18401ccc0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc192d9d640 1 -- 172.21.15.60:0/1265217725 reap_dead start 2024-04-25T10:10:52.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc198fa0640 1 -- 172.21.15.60:0/1265217725 shutdown_connections 2024-04-25T10:10:52.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc198fa0640 1 -- 172.21.15.60:0/1265217725 >> 172.21.15.60:0/1265217725 conn(0x7fc194067080 msgr2=0x7fc19406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc198fa0640 1 -- 172.21.15.60:0/1265217725 shutdown_connections 2024-04-25T10:10:52.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.597+0000 7fc198fa0640 1 -- 172.21.15.60:0/1265217725 wait complete. 2024-04-25T10:10:52.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc198fa0640 1 Processor -- start 2024-04-25T10:10:52.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc198fa0640 1 -- start start 2024-04-25T10:10:52.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc198fa0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1800039c0 0x7fc19413d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc198fa0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc19413c140 0x7fc194139f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc198fa0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc19415e3e0 0x7fc194141a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc198fa0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1940b3750 con 0x7fc19415e3e0 2024-04-25T10:10:52.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc198fa0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1940b35d0 con 0x7fc19413c140 2024-04-25T10:10:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc198fa0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1940b38d0 con 0x7fc1800039c0 2024-04-25T10:10:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc19415e3e0 0x7fc194141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc192d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc19415e3e0 0x7fc194141a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57630/0 (socket says 172.21.15.60:57630) 2024-04-25T10:10:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc192d9d640 1 -- 172.21.15.60:0/2506150866 learned_addr learned my addr 172.21.15.60:0/2506150866 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc191d9b640 1 --2- 172.21.15.60:0/2506150866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc19413c140 0x7fc194139f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc19259c640 1 --2- 172.21.15.60:0/2506150866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1800039c0 0x7fc19413d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc192d9d640 1 -- 172.21.15.60:0/2506150866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1800039c0 msgr2=0x7fc19413d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc192d9d640 1 --2- 172.21.15.60:0/2506150866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1800039c0 0x7fc19413d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc192d9d640 1 -- 172.21.15.60:0/2506150866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc19413c140 msgr2=0x7fc194139f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc192d9d640 1 --2- 172.21.15.60:0/2506150866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc19413c140 0x7fc194139f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc192d9d640 1 -- 172.21.15.60:0/2506150866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19413c620 con 0x7fc19415e3e0 2024-04-25T10:10:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc192d9d640 1 --2- 172.21.15.60:0/2506150866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc19415e3e0 0x7fc194141a80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fc1940aefa0 tx=0x7fc18c010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc18affd640 1 -- 172.21.15.60:0/2506150866 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00f040 con 0x7fc19415e3e0 2024-04-25T10:10:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc18affd640 1 -- 172.21.15.60:0/2506150866 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c01c070 con 0x7fc19415e3e0 2024-04-25T10:10:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc198fa0640 1 -- 172.21.15.60:0/2506150866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19413c900 con 0x7fc19415e3e0 2024-04-25T10:10:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc18affd640 1 -- 172.21.15.60:0/2506150866 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00eae0 con 0x7fc19415e3e0 2024-04-25T10:10:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc198fa0640 1 -- 172.21.15.60:0/2506150866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19413cec0 con 0x7fc19415e3e0 2024-04-25T10:10:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc198fa0640 1 -- 172.21.15.60:0/2506150866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc194074770 con 0x7fc19415e3e0 2024-04-25T10:10:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.601+0000 7fc18affd640 1 -- 172.21.15.60:0/2506150866 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc18c00ec80 con 0x7fc19415e3e0 2024-04-25T10:10:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.605+0000 7fc18affd640 1 --2- 172.21.15.60:0/2506150866 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc174066f90 0x7fc174069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.605+0000 7fc18affd640 1 -- 172.21.15.60:0/2506150866 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc18c084fb0 con 0x7fc19415e3e0 2024-04-25T10:10:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.605+0000 7fc19259c640 1 --2- 172.21.15.60:0/2506150866 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc174066f90 0x7fc174069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.605+0000 7fc19259c640 1 --2- 172.21.15.60:0/2506150866 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc174066f90 0x7fc174069450 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc1840af000 tx=0x7fc1840a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.609+0000 7fc18affd640 1 -- 172.21.15.60:0/2506150866 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc18c055f30 con 0x7fc19415e3e0 2024-04-25T10:10:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.797+0000 7fc198fa0640 1 -- 172.21.15.60:0/2506150866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc194071b80 con 0x7fc19415e3e0 2024-04-25T10:10:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.797+0000 7fc18affd640 1 -- 172.21.15.60:0/2506150866 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fc18c055830 con 0x7fc19415e3e0 2024-04-25T10:10:52.805 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:10:52.806 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:10:52.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.801+0000 7fc198fa0640 1 -- 172.21.15.60:0/2506150866 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc174066f90 msgr2=0x7fc174069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.801+0000 7fc198fa0640 1 --2- 172.21.15.60:0/2506150866 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc174066f90 0x7fc174069450 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc1840af000 tx=0x7fc1840a7040 comp rx=0 tx=0).stop 2024-04-25T10:10:52.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.801+0000 7fc198fa0640 1 -- 172.21.15.60:0/2506150866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc19415e3e0 msgr2=0x7fc194141a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.801+0000 7fc198fa0640 1 --2- 172.21.15.60:0/2506150866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc19415e3e0 0x7fc194141a80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fc1940aefa0 tx=0x7fc18c010c20 comp rx=0 tx=0).stop 2024-04-25T10:10:52.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.801+0000 7fc198fa0640 1 -- 172.21.15.60:0/2506150866 shutdown_connections 2024-04-25T10:10:52.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.801+0000 7fc198fa0640 1 --2- 172.21.15.60:0/2506150866 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc174066f90 0x7fc174069450 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.801+0000 7fc198fa0640 1 --2- 172.21.15.60:0/2506150866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc19415e3e0 0x7fc194141a80 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.801+0000 7fc198fa0640 1 --2- 172.21.15.60:0/2506150866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc19413c140 0x7fc194139f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.801+0000 7fc198fa0640 1 --2- 172.21.15.60:0/2506150866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1800039c0 0x7fc19413d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.801+0000 7fc198fa0640 1 -- 172.21.15.60:0/2506150866 >> 172.21.15.60:0/2506150866 conn(0x7fc194067080 msgr2=0x7fc1940ae870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.801+0000 7fc198fa0640 1 -- 172.21.15.60:0/2506150866 shutdown_connections 2024-04-25T10:10:52.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.801+0000 7fc198fa0640 1 -- 172.21.15.60:0/2506150866 wait complete. 2024-04-25T10:10:52.823 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:10:52.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:52.823 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-04-25T10:10:52.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.941+0000 7fc51ff2d640 1 Processor -- start 2024-04-25T10:10:52.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.941+0000 7fc51ff2d640 1 -- start start 2024-04-25T10:10:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51ff2d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc5180d08f0 0x7fc518168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51ff2d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc518162660 con 0x7fc5180b9380 2024-04-25T10:10:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51ff2d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5181620a0 con 0x7fc51815d9f0 2024-04-25T10:10:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51ff2d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5180d2180 con 0x7fc5180d08f0 2024-04-25T10:10:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51d4c8640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc5180b9380 0x7fc5180d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34442/0 (socket says 172.21.15.60:34442) 2024-04-25T10:10:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51d4c8640 1 -- 172.21.15.60:0/1725222028 learned_addr learned my addr 172.21.15.60:0/1725222028 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:10:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51e4ca640 1 --2- 172.21.15.60:0/1725222028 >> v2:172.21.15.60:3300/0 conn(0x7fc5180d08f0 0x7fc518168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51ccc7640 1 -- 172.21.15.60:0/1725222028 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236800636 0 0) 0x7fc518162660 con 0x7fc5180b9380 2024-04-25T10:10:52.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51e4ca640 1 -- 172.21.15.60:0/1725222028 >> v1:172.21.15.100:6789/0 conn(0x7fc51815d9f0 legacy=0x7fc5181617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51e4ca640 1 -- 172.21.15.60:0/1725222028 >> v1:172.21.15.60:6789/0 conn(0x7fc5180b9380 legacy=0x7fc5180d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51e4ca640 1 -- 172.21.15.60:0/1725222028 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc518162380 con 0x7fc5180d08f0 2024-04-25T10:10:52.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51e4ca640 1 --2- 172.21.15.60:0/1725222028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5180d08f0 0x7fc518168ad0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fc510009d90 tx=0x7fc51001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26fbd0a020c2b3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51ccc7640 1 -- 172.21.15.60:0/1725222028 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5100b2070 con 0x7fc5180d08f0 2024-04-25T10:10:52.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51ccc7640 1 -- 172.21.15.60:0/1725222028 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5100a58b0 con 0x7fc5180d08f0 2024-04-25T10:10:52.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51ccc7640 1 -- 172.21.15.60:0/1725222028 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5100a5c80 con 0x7fc5180d08f0 2024-04-25T10:10:52.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51ff2d640 1 -- 172.21.15.60:0/1725222028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5180d08f0 msgr2=0x7fc518168ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51ff2d640 1 --2- 172.21.15.60:0/1725222028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5180d08f0 0x7fc518168ad0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fc510009d90 tx=0x7fc51001c970 comp rx=0 tx=0).stop 2024-04-25T10:10:52.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51ff2d640 1 -- 172.21.15.60:0/1725222028 shutdown_connections 2024-04-25T10:10:52.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51ff2d640 1 --2- 172.21.15.60:0/1725222028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5180d08f0 0x7fc518168ad0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51ff2d640 1 -- 172.21.15.60:0/1725222028 >> 172.21.15.60:0/1725222028 conn(0x7fc518072340 msgr2=0x7fc518072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51ff2d640 1 -- 172.21.15.60:0/1725222028 shutdown_connections 2024-04-25T10:10:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.945+0000 7fc51ff2d640 1 -- 172.21.15.60:0/1725222028 wait complete. 2024-04-25T10:10:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51ff2d640 1 Processor -- start 2024-04-25T10:10:52.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51ff2d640 1 -- start start 2024-04-25T10:10:52.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51ff2d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5180b9380 0x7fc5180cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51ff2d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5180d08f0 0x7fc5181fa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51ff2d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc51815d9f0 0x7fc5181fa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51ff2d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51816b9b0 con 0x7fc5180d08f0 2024-04-25T10:10:52.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51ff2d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51816b830 con 0x7fc5180b9380 2024-04-25T10:10:52.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51ff2d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51816bb30 con 0x7fc51815d9f0 2024-04-25T10:10:52.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51d4c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5180d08f0 0x7fc5181fa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51d4c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5180d08f0 0x7fc5181fa060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57656/0 (socket says 172.21.15.60:57656) 2024-04-25T10:10:52.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51d4c8640 1 -- 172.21.15.60:0/2272236719 learned_addr learned my addr 172.21.15.60:0/2272236719 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:52.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51d4c8640 1 -- 172.21.15.60:0/2272236719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc51815d9f0 msgr2=0x7fc5181fa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51d4c8640 1 --2- 172.21.15.60:0/2272236719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc51815d9f0 0x7fc5181fa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51d4c8640 1 -- 172.21.15.60:0/2272236719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5180b9380 msgr2=0x7fc5180cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51dcc9640 1 --2- 172.21.15.60:0/2272236719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5180b9380 0x7fc5180cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51d4c8640 1 --2- 172.21.15.60:0/2272236719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5180b9380 0x7fc5180cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51d4c8640 1 -- 172.21.15.60:0/2272236719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc518155fa0 con 0x7fc5180d08f0 2024-04-25T10:10:52.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51d4c8640 1 --2- 172.21.15.60:0/2272236719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5180d08f0 0x7fc5181fa060 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fc50800c480 tx=0x7fc50800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc50e7fc640 1 -- 172.21.15.60:0/2272236719 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc508018070 con 0x7fc5180d08f0 2024-04-25T10:10:52.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc50e7fc640 1 -- 172.21.15.60:0/2272236719 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5080074e0 con 0x7fc5180d08f0 2024-04-25T10:10:52.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51ff2d640 1 -- 172.21.15.60:0/2272236719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc518153ab0 con 0x7fc5180d08f0 2024-04-25T10:10:52.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc50e7fc640 1 -- 172.21.15.60:0/2272236719 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc508004a30 con 0x7fc5180d08f0 2024-04-25T10:10:52.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.949+0000 7fc51ff2d640 1 -- 172.21.15.60:0/2272236719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc518154070 con 0x7fc5180d08f0 2024-04-25T10:10:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc50e7fc640 1 -- 172.21.15.60:0/2272236719 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc50801b070 con 0x7fc5180d08f0 2024-04-25T10:10:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc50e7fc640 1 --2- 172.21.15.60:0/2272236719 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4fc066f90 0x7fc4fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc50e7fc640 1 -- 172.21.15.60:0/2272236719 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc508084470 con 0x7fc5180d08f0 2024-04-25T10:10:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc51dcc9640 1 --2- 172.21.15.60:0/2272236719 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4fc066f90 0x7fc4fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc52309a000 1 Processor -- start 2024-04-25T10:10:52.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc52309a000 1 -- start start 2024-04-25T10:10:52.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc51dcc9640 1 --2- 172.21.15.60:0/2272236719 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4fc066f90 0x7fc4fc069450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc514000f70 tx=0x7fc514009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc52309a000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8d9d40 0x55c1ab8da120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc52309a000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8da660 0x55c1ab8900b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc52309a000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8905f0 0x55c1ab8909d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc52309a000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab6dec40 con 0x55c1ab8905f0 2024-04-25T10:10:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc52309a000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab6671d0 con 0x55c1ab8da660 2024-04-25T10:10:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc52309a000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab6dace0 con 0x55c1ab8d9d40 2024-04-25T10:10:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc51e4ca640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8905f0 0x55c1ab8909d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc51e4ca640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8905f0 0x55c1ab8909d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57660/0 (socket says 172.21.15.60:57660) 2024-04-25T10:10:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc51e4ca640 1 -- 172.21.15.60:0/3916326962 learned_addr learned my addr 172.21.15.60:0/3916326962 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc51d4c8640 1 --2- 172.21.15.60:0/3916326962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8da660 0x55c1ab8900b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc51dcc9640 1 --2- 172.21.15.60:0/3916326962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8d9d40 0x55c1ab8da120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc51e4ca640 1 -- 172.21.15.60:0/3916326962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8d9d40 msgr2=0x55c1ab8da120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc51e4ca640 1 --2- 172.21.15.60:0/3916326962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8d9d40 0x55c1ab8da120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc51e4ca640 1 -- 172.21.15.60:0/3916326962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8da660 msgr2=0x55c1ab8900b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc51e4ca640 1 --2- 172.21.15.60:0/3916326962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8da660 0x55c1ab8900b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc51e4ca640 1 -- 172.21.15.60:0/3916326962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1ab7242a0 con 0x55c1ab8905f0 2024-04-25T10:10:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc51e4ca640 1 --2- 172.21.15.60:0/3916326962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8905f0 0x55c1ab8909d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fc5100b0040 tx=0x7fc510004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb82ffd01cdad7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc4fb7fe640 1 -- 172.21.15.60:0/3916326962 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5100bd020 con 0x55c1ab8905f0 2024-04-25T10:10:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc4fb7fe640 1 -- 172.21.15.60:0/3916326962 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5100b3040 con 0x55c1ab8905f0 2024-04-25T10:10:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc4fb7fe640 1 -- 172.21.15.60:0/3916326962 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5100b2050 con 0x55c1ab8905f0 2024-04-25T10:10:52.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc52309a000 1 -- 172.21.15.60:0/3916326962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8905f0 msgr2=0x55c1ab8909d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.953+0000 7fc52309a000 1 --2- 172.21.15.60:0/3916326962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8905f0 0x55c1ab8909d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fc5100b0040 tx=0x7fc510004a50 comp rx=0 tx=0).stop 2024-04-25T10:10:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 -- 172.21.15.60:0/3916326962 shutdown_connections 2024-04-25T10:10:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 --2- 172.21.15.60:0/3916326962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8905f0 0x55c1ab8909d0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 --2- 172.21.15.60:0/3916326962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8da660 0x55c1ab8900b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 --2- 172.21.15.60:0/3916326962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8d9d40 0x55c1ab8da120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 -- 172.21.15.60:0/3916326962 >> 172.21.15.60:0/3916326962 conn(0x55c1ab88c5a0 msgr2=0x55c1ab88e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 -- 172.21.15.60:0/3916326962 shutdown_connections 2024-04-25T10:10:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 -- 172.21.15.60:0/3916326962 wait complete. 2024-04-25T10:10:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 -- start start 2024-04-25T10:10:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 Processor -- start 2024-04-25T10:10:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8905f0 0x55c1ab8959e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8d9d40 0x55c1ab895f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8da660 0x55c1ab8bed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab699a90 con 0x55c1ab8d9d40 2024-04-25T10:10:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab88c420 con 0x55c1ab8da660 2024-04-25T10:10:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab6e0660 con 0x55c1ab8905f0 2024-04-25T10:10:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 10 client.0 _ll_register_callbacks cb 0x55c1ab87b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc51d4c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8d9d40 0x55c1ab895f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc51d4c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8d9d40 0x55c1ab895f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57676/0 (socket says 172.21.15.60:57676) 2024-04-25T10:10:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc51d4c8640 1 -- 172.21.15.60:0/3551655343 learned_addr learned my addr 172.21.15.60:0/3551655343 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc51dcc9640 1 --2- 172.21.15.60:0/3551655343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8905f0 0x55c1ab8959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc51e4ca640 1 --2- 172.21.15.60:0/3551655343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8da660 0x55c1ab8bed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc51d4c8640 1 -- 172.21.15.60:0/3551655343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8905f0 msgr2=0x55c1ab8959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc51d4c8640 1 --2- 172.21.15.60:0/3551655343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8905f0 0x55c1ab8959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc51d4c8640 1 -- 172.21.15.60:0/3551655343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8da660 msgr2=0x55c1ab8bed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc51d4c8640 1 --2- 172.21.15.60:0/3551655343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8da660 0x55c1ab8bed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc51d4c8640 1 -- 172.21.15.60:0/3551655343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1ab697730 con 0x55c1ab8d9d40 2024-04-25T10:10:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc51e4ca640 1 --2- 172.21.15.60:0/3551655343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8da660 0x55c1ab8bed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc51d4c8640 1 --2- 172.21.15.60:0/3551655343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8d9d40 0x55c1ab895f20 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fc5080010c0 tx=0x7fc508055520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d33f14e457fae45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc4f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:10:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc4f8ff9640 1 -- 172.21.15.60:0/3551655343 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5080840b0 con 0x55c1ab8d9d40 2024-04-25T10:10:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc4f8ff9640 1 -- 172.21.15.60:0/3551655343 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc50805cf60 con 0x55c1ab8d9d40 2024-04-25T10:10:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 -- 172.21.15.60:0/3551655343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5100b3040 con 0x55c1ab8d9d40 2024-04-25T10:10:52.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc4f8ff9640 1 -- 172.21.15.60:0/3551655343 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc508084250 con 0x55c1ab8d9d40 2024-04-25T10:10:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 10 client.7359 fetch_fsmap learned FSMap version 46 2024-04-25T10:10:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 -- 172.21.15.60:0/3551655343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x55c1ab667420 con 0x55c1ab8d9d40 2024-04-25T10:10:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc4f8ff9640 1 -- 172.21.15.60:0/3551655343 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5100b3040 con 0x55c1ab8d9d40 2024-04-25T10:10:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc4f8ff9640 1 -- 172.21.15.60:0/3551655343 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc50807e020 con 0x55c1ab8d9d40 2024-04-25T10:10:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 10 client.7359 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:10:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 10 client.7359 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:10:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 20 client.7359 populate_metadata read hostname 'smithi060' 2024-04-25T10:10:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 --2- 172.21.15.60:0/3551655343 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55c1ab8bf780 0x55c1ab8ec000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 4 client.7359 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc52309a000 1 -- 172.21.15.60:0/3551655343 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1ab67ab50 con 0x55c1ab8bf780 2024-04-25T10:10:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc51dcc9640 1 --2- 172.21.15.60:0/3551655343 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55c1ab8bf780 0x55c1ab8ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc4f8ff9640 10 client.7359 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc51dcc9640 1 --2- 172.21.15.60:0/3551655343 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55c1ab8bf780 0x55c1ab8ec000 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ee8eee98197e389 server_cookie=ba36e5a76b65e51e in_seq=0 out_seq=0 2024-04-25T10:10:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc4f8ff9640 1 -- 172.21.15.60:0/3551655343 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c1ab67ab50 con 0x55c1ab8bf780 2024-04-25T10:10:52.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.957+0000 7fc4f8ff9640 10 client.7359 handle_command_reply: tid=0 2024-04-25T10:10:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.961+0000 7fc52309a000 1 client.7359 shutdown 2024-04-25T10:10:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.961+0000 7fc52309a000 1 -- 172.21.15.60:0/3551655343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8d9d40 msgr2=0x55c1ab895f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.961+0000 7fc52309a000 1 --2- 172.21.15.60:0/3551655343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8d9d40 0x55c1ab895f20 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fc5080010c0 tx=0x7fc508055520 comp rx=0 tx=0).stop 2024-04-25T10:10:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.961+0000 7fc52309a000 1 -- 172.21.15.60:0/3551655343 shutdown_connections 2024-04-25T10:10:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.961+0000 7fc52309a000 1 --2- 172.21.15.60:0/3551655343 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55c1ab8bf780 0x55c1ab8ec000 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.961+0000 7fc52309a000 1 --2- 172.21.15.60:0/3551655343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8da660 0x55c1ab8bed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.961+0000 7fc52309a000 1 --2- 172.21.15.60:0/3551655343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8d9d40 0x55c1ab895f20 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.961+0000 7fc4f8ff9640 0 client.7359 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.961+0000 7fc52309a000 1 --2- 172.21.15.60:0/3551655343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8905f0 0x55c1ab8959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.961+0000 7fc52309a000 1 -- 172.21.15.60:0/3551655343 >> 172.21.15.60:0/3551655343 conn(0x55c1ab88c5a0 msgr2=0x55c1ab88f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.961+0000 7fc52309a000 1 -- 172.21.15.60:0/3551655343 shutdown_connections 2024-04-25T10:10:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.961+0000 7fc52309a000 1 -- 172.21.15.60:0/3551655343 wait complete. 2024-04-25T10:10:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.961+0000 7fc52309a000 20 client.7359 trim_cache size 0 max 16384 2024-04-25T10:10:52.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.973+0000 7fc52309a000 1 Processor -- start 2024-04-25T10:10:52.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.973+0000 7fc52309a000 1 -- start start 2024-04-25T10:10:52.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8be090 0x55c1ab88f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab88c5a0 0x55c1ab88fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8905f0 0x55c1ab8ec000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc518202f60 con 0x55c1ab8905f0 2024-04-25T10:10:52.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5181f79a0 con 0x55c1ab88c5a0 2024-04-25T10:10:52.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5181f6da0 con 0x55c1ab8be090 2024-04-25T10:10:52.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51e4ca640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8905f0 0x55c1ab8ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51e4ca640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8905f0 0x55c1ab8ec000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57680/0 (socket says 172.21.15.60:57680) 2024-04-25T10:10:52.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51e4ca640 1 -- 172.21.15.60:0/991209114 learned_addr learned my addr 172.21.15.60:0/991209114 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51d4c8640 1 --2- 172.21.15.60:0/991209114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab88c5a0 0x55c1ab88fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51e4ca640 1 -- 172.21.15.60:0/991209114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8be090 msgr2=0x55c1ab88f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51e4ca640 1 --2- 172.21.15.60:0/991209114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8be090 0x55c1ab88f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51e4ca640 1 -- 172.21.15.60:0/991209114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab88c5a0 msgr2=0x55c1ab88fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51e4ca640 1 --2- 172.21.15.60:0/991209114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab88c5a0 0x55c1ab88fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51e4ca640 1 -- 172.21.15.60:0/991209114 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1ab763550 con 0x55c1ab8905f0 2024-04-25T10:10:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51e4ca640 1 --2- 172.21.15.60:0/991209114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8905f0 0x55c1ab8ec000 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fc510009bd0 tx=0x7fc5100a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fb31eba72897c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc4faffd640 1 -- 172.21.15.60:0/991209114 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5100bd070 con 0x55c1ab8905f0 2024-04-25T10:10:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc4faffd640 1 -- 172.21.15.60:0/991209114 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc510003c20 con 0x55c1ab8905f0 2024-04-25T10:10:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc4faffd640 1 -- 172.21.15.60:0/991209114 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5100b2040 con 0x55c1ab8905f0 2024-04-25T10:10:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 -- 172.21.15.60:0/991209114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8905f0 msgr2=0x55c1ab8ec000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 --2- 172.21.15.60:0/991209114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8905f0 0x55c1ab8ec000 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fc510009bd0 tx=0x7fc5100a7070 comp rx=0 tx=0).stop 2024-04-25T10:10:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 -- 172.21.15.60:0/991209114 shutdown_connections 2024-04-25T10:10:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 --2- 172.21.15.60:0/991209114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab8905f0 0x55c1ab8ec000 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 --2- 172.21.15.60:0/991209114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab88c5a0 0x55c1ab88fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 --2- 172.21.15.60:0/991209114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8be090 0x55c1ab88f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 -- 172.21.15.60:0/991209114 >> 172.21.15.60:0/991209114 conn(0x55c1ab8bcd60 msgr2=0x55c1ab8e7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 -- 172.21.15.60:0/991209114 shutdown_connections 2024-04-25T10:10:52.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 -- 172.21.15.60:0/991209114 wait complete. 2024-04-25T10:10:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 -- start start 2024-04-25T10:10:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 Processor -- start 2024-04-25T10:10:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab88c5a0 0x55c1ab93bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8905f0 0x55c1ab93c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8be090 0x55c1ab938600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab6e3270 con 0x55c1ab88c5a0 2024-04-25T10:10:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5181f73a0 con 0x55c1ab8905f0 2024-04-25T10:10:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab699a90 con 0x55c1ab8be090 2024-04-25T10:10:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc52309a000 10 client.0 _ll_register_callbacks cb 0x7fc5100b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51dcc9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab88c5a0 0x55c1ab93bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51dcc9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab88c5a0 0x55c1ab93bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57690/0 (socket says 172.21.15.60:57690) 2024-04-25T10:10:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51dcc9640 1 -- 172.21.15.60:0/3544983521 learned_addr learned my addr 172.21.15.60:0/3544983521 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51d4c8640 1 --2- 172.21.15.60:0/3544983521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8905f0 0x55c1ab93c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51e4ca640 1 --2- 172.21.15.60:0/3544983521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8be090 0x55c1ab938600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51dcc9640 1 -- 172.21.15.60:0/3544983521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8be090 msgr2=0x55c1ab938600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51dcc9640 1 --2- 172.21.15.60:0/3544983521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8be090 0x55c1ab938600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51dcc9640 1 -- 172.21.15.60:0/3544983521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8905f0 msgr2=0x55c1ab93c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51dcc9640 1 --2- 172.21.15.60:0/3544983521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8905f0 0x55c1ab93c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51dcc9640 1 -- 172.21.15.60:0/3544983521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1ab691490 con 0x55c1ab88c5a0 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51d4c8640 1 --2- 172.21.15.60:0/3544983521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8905f0 0x55c1ab93c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc51dcc9640 1 --2- 172.21.15.60:0/3544983521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab88c5a0 0x55c1ab93bf60 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x55c1ab6e29b0 tx=0x7fc514000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3b6109bfd29bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.977+0000 7fc4f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc4f8ff9640 1 -- 172.21.15.60:0/3544983521 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc514003d00 con 0x55c1ab88c5a0 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc4f8ff9640 1 -- 172.21.15.60:0/3544983521 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc514016b40 con 0x55c1ab88c5a0 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 1 -- 172.21.15.60:0/3544983521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc510003c20 con 0x55c1ab88c5a0 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc4f8ff9640 1 -- 172.21.15.60:0/3544983521 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc514016d60 con 0x55c1ab88c5a0 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 10 client.7365 fetch_fsmap learned FSMap version 46 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 1 -- 172.21.15.60:0/3544983521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x55c1ab6861b0 con 0x55c1ab88c5a0 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc4f8ff9640 1 -- 172.21.15.60:0/3544983521 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc510003c20 con 0x55c1ab88c5a0 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc4f8ff9640 1 -- 172.21.15.60:0/3544983521 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc514034070 con 0x55c1ab88c5a0 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 10 client.7365 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 10 client.7365 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 20 client.7365 populate_metadata read hostname 'smithi060' 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 1 --2- 172.21.15.60:0/3544983521 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55c1ab8d9d40 0x55c1ab939490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 4 client.7365 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 1 -- 172.21.15.60:0/3544983521 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) v1 -- 0x55c1ab6ec180 con 0x55c1ab8d9d40 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc51d4c8640 1 --2- 172.21.15.60:0/3544983521 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55c1ab8d9d40 0x55c1ab939490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc51d4c8640 1 --2- 172.21.15.60:0/3544983521 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55c1ab8d9d40 0x55c1ab939490 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853c3e43d748de38 server_cookie=524ca92d04bccc6 in_seq=0 out_seq=0 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc4f8ff9640 10 client.7365 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc4f8ff9640 1 -- 172.21.15.60:0/3544983521 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c1ab6ec180 con 0x55c1ab8d9d40 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc4f8ff9640 10 client.7365 handle_command_reply: tid=0 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 1 client.7365 shutdown 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 1 -- 172.21.15.60:0/3544983521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab88c5a0 msgr2=0x55c1ab93bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 1 --2- 172.21.15.60:0/3544983521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab88c5a0 0x55c1ab93bf60 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x55c1ab6e29b0 tx=0x7fc514000ee0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 1 -- 172.21.15.60:0/3544983521 shutdown_connections 2024-04-25T10:10:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 1 --2- 172.21.15.60:0/3544983521 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x55c1ab8d9d40 0x55c1ab939490 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 1 --2- 172.21.15.60:0/3544983521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c1ab8be090 0x55c1ab938600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 1 --2- 172.21.15.60:0/3544983521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1ab8905f0 0x55c1ab93c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 1 --2- 172.21.15.60:0/3544983521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c1ab88c5a0 0x55c1ab93bf60 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 1 -- 172.21.15.60:0/3544983521 >> 172.21.15.60:0/3544983521 conn(0x55c1ab8bcd60 msgr2=0x55c1ab88e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc4f8ff9640 0 client.7365 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 1 -- 172.21.15.60:0/3544983521 shutdown_connections 2024-04-25T10:10:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 1 -- 172.21.15.60:0/3544983521 wait complete. 2024-04-25T10:10:52.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc52309a000 20 client.7365 trim_cache size 0 max 16384 2024-04-25T10:10:52.994 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:10:52.995 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T10:10:52.995 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "79846d77-eabf-4acf-8fbd-861323055c36", 2024-04-25T10:10:52.995 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T10:10:52.995 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc51ff2d640 1 -- 172.21.15.60:0/2272236719 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4fc066f90 msgr2=0x7fc4fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc51ff2d640 1 --2- 172.21.15.60:0/2272236719 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4fc066f90 0x7fc4fc069450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc514000f70 tx=0x7fc514009040 comp rx=0 tx=0).stop 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc51ff2d640 1 -- 172.21.15.60:0/2272236719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5180d08f0 msgr2=0x7fc5181fa060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc51ff2d640 1 --2- 172.21.15.60:0/2272236719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5180d08f0 0x7fc5181fa060 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fc50800c480 tx=0x7fc50800c940 comp rx=0 tx=0).stop 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc51ff2d640 1 -- 172.21.15.60:0/2272236719 shutdown_connections 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc51ff2d640 1 --2- 172.21.15.60:0/2272236719 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4fc066f90 0x7fc4fc069450 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc51ff2d640 1 --2- 172.21.15.60:0/2272236719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc51815d9f0 0x7fc5181fa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc51ff2d640 1 --2- 172.21.15.60:0/2272236719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5180d08f0 0x7fc5181fa060 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc51ff2d640 1 --2- 172.21.15.60:0/2272236719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5180b9380 0x7fc5180cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc51ff2d640 1 -- 172.21.15.60:0/2272236719 >> 172.21.15.60:0/2272236719 conn(0x7fc518072340 msgr2=0x7fc518151d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc51ff2d640 1 -- 172.21.15.60:0/2272236719 shutdown_connections 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:52.981+0000 7fc51ff2d640 1 -- 172.21.15.60:0/2272236719 wait complete. 2024-04-25T10:10:53.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:53.006 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:10:53.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.117+0000 7fbd064d6640 1 Processor -- start 2024-04-25T10:10:53.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.117+0000 7fbd064d6640 1 -- start start 2024-04-25T10:10:53.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.117+0000 7fbd064d6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd000b35f0 0x7fbd000ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.117+0000 7fbd064d6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd000aec10 0x7fbd000ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.117+0000 7fbd064d6640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd0015def0 con 0x7fbd0015db10 2024-04-25T10:10:53.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.117+0000 7fbd064d6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd000accc0 con 0x7fbd000b35f0 2024-04-25T10:10:53.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.117+0000 7fbd064d6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd00071630 con 0x7fbd000aec10 2024-04-25T10:10:53.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.117+0000 7fbcfffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbd0015db10 0x7fbd000b2ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34454/0 (socket says 172.21.15.60:34454) 2024-04-25T10:10:53.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.117+0000 7fbcfffff640 1 -- 172.21.15.60:0/285703857 learned_addr learned my addr 172.21.15.60:0/285703857 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:10:53.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.117+0000 7fbcff7fe640 1 --2- 172.21.15.60:0/285703857 >> v2:172.21.15.60:3300/0 conn(0x7fbd000b35f0 0x7fbd000ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.117+0000 7fbd04a73640 1 --2- 172.21.15.60:0/285703857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd000aec10 0x7fbd000ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.117+0000 7fbcff7fe640 1 -- 172.21.15.60:0/285703857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd000aec10 msgr2=0x7fbd000ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.117+0000 7fbcff7fe640 1 --2- 172.21.15.60:0/285703857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd000aec10 0x7fbd000ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.117+0000 7fbcff7fe640 1 -- 172.21.15.60:0/285703857 >> v1:172.21.15.60:6789/0 conn(0x7fbd0015db10 legacy=0x7fbd000b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.117+0000 7fbcff7fe640 1 -- 172.21.15.60:0/285703857 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd00071910 con 0x7fbd000b35f0 2024-04-25T10:10:53.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.121+0000 7fbcff7fe640 1 --2- 172.21.15.60:0/285703857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd000b35f0 0x7fbd000ad130 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40097c0 tx=0x7fbcf401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb50c4da5a4fde4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.121+0000 7fbcfeffd640 1 -- 172.21.15.60:0/285703857 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40aa600 con 0x7fbd000b35f0 2024-04-25T10:10:53.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.121+0000 7fbcfeffd640 1 -- 172.21.15.60:0/285703857 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf40af070 con 0x7fbd000b35f0 2024-04-25T10:10:53.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.121+0000 7fbcfeffd640 1 -- 172.21.15.60:0/285703857 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40aa9c0 con 0x7fbd000b35f0 2024-04-25T10:10:53.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.121+0000 7fbd064d6640 1 -- 172.21.15.60:0/285703857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd000b35f0 msgr2=0x7fbd000ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.121+0000 7fbd064d6640 1 --2- 172.21.15.60:0/285703857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd000b35f0 0x7fbd000ad130 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40097c0 tx=0x7fbcf401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.121+0000 7fbd064d6640 1 -- 172.21.15.60:0/285703857 shutdown_connections 2024-04-25T10:10:53.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.121+0000 7fbd064d6640 1 --2- 172.21.15.60:0/285703857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd000aec10 0x7fbd000ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.121+0000 7fbd064d6640 1 --2- 172.21.15.60:0/285703857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd000b35f0 0x7fbd000ad130 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.121+0000 7fbd064d6640 1 -- 172.21.15.60:0/285703857 >> 172.21.15.60:0/285703857 conn(0x7fbd000670b0 msgr2=0x7fbd0006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.121+0000 7fbd064d6640 1 -- 172.21.15.60:0/285703857 shutdown_connections 2024-04-25T10:10:53.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.121+0000 7fbd064d6640 1 -- 172.21.15.60:0/285703857 wait complete. 2024-04-25T10:10:53.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.121+0000 7fbd064d6640 1 Processor -- start 2024-04-25T10:10:53.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.121+0000 7fbd064d6640 1 -- start start 2024-04-25T10:10:53.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd064d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd000aec10 0x7fbd000afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd064d6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd000b35f0 0x7fbd00151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd064d6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd0015db10 0x7fbd00140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd064d6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd000713d0 con 0x7fbd0015db10 2024-04-25T10:10:53.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd064d6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd00071250 con 0x7fbd000aec10 2024-04-25T10:10:53.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd064d6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd000adfc0 con 0x7fbd000b35f0 2024-04-25T10:10:53.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd04a73640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd0015db10 0x7fbd00140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd04a73640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd0015db10 0x7fbd00140b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57710/0 (socket says 172.21.15.60:57710) 2024-04-25T10:10:53.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd04a73640 1 -- 172.21.15.60:0/258558757 learned_addr learned my addr 172.21.15.60:0/258558757 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:53.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbcfffff640 1 --2- 172.21.15.60:0/258558757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd000aec10 0x7fbd000afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbcff7fe640 1 --2- 172.21.15.60:0/258558757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd000b35f0 0x7fbd00151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd04a73640 1 -- 172.21.15.60:0/258558757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd000b35f0 msgr2=0x7fbd00151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd04a73640 1 --2- 172.21.15.60:0/258558757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd000b35f0 0x7fbd00151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd04a73640 1 -- 172.21.15.60:0/258558757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd000aec10 msgr2=0x7fbd000afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd04a73640 1 --2- 172.21.15.60:0/258558757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd000aec10 0x7fbd000afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd04a73640 1 -- 172.21.15.60:0/258558757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0014f3d0 con 0x7fbd0015db10 2024-04-25T10:10:53.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbcff7fe640 1 --2- 172.21.15.60:0/258558757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd000b35f0 0x7fbd00151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:53.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbcfffff640 1 --2- 172.21.15.60:0/258558757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd000aec10 0x7fbd000afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:53.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd04a73640 1 --2- 172.21.15.60:0/258558757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd0015db10 0x7fbd00140b00 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0007e60 tx=0x7fbcf000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbcfcff9640 1 -- 172.21.15.60:0/258558757 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0014420 con 0x7fbd0015db10 2024-04-25T10:10:53.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbcfcff9640 1 -- 172.21.15.60:0/258558757 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf0019070 con 0x7fbd0015db10 2024-04-25T10:10:53.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd064d6640 1 -- 172.21.15.60:0/258558757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0014f6b0 con 0x7fbd0015db10 2024-04-25T10:10:53.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbcfcff9640 1 -- 172.21.15.60:0/258558757 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0014820 con 0x7fbd0015db10 2024-04-25T10:10:53.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd064d6640 1 -- 172.21.15.60:0/258558757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0014fc70 con 0x7fbd0015db10 2024-04-25T10:10:53.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbcfcff9640 1 -- 172.21.15.60:0/258558757 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbcf00149c0 con 0x7fbd0015db10 2024-04-25T10:10:53.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbcfcff9640 1 --2- 172.21.15.60:0/258558757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbcec066f90 0x7fbcec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbcfcff9640 1 -- 172.21.15.60:0/258558757 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fbcf0085910 con 0x7fbd0015db10 2024-04-25T10:10:53.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbcfffff640 1 --2- 172.21.15.60:0/258558757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbcec066f90 0x7fbcec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd09643000 1 Processor -- start 2024-04-25T10:10:53.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbd09643000 1 -- start start 2024-04-25T10:10:53.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.125+0000 7fbcfffff640 1 --2- 172.21.15.60:0/258558757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbcec066f90 0x7fbcec069450 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8007fd0 tx=0x7fbcf8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b29d0 0x563b994b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b32f0 0x563b99468b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 0x563b99469480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b992b5ff0 con 0x563b994b32f0 2024-04-25T10:10:53.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b9923e150 con 0x563b994b29d0 2024-04-25T10:10:53.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b992b2090 con 0x563b994690a0 2024-04-25T10:10:53.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbcff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b32f0 0x563b99468b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbcff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b32f0 0x563b99468b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57712/0 (socket says 172.21.15.60:57712) 2024-04-25T10:10:53.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbcff7fe640 1 -- 172.21.15.60:0/677921144 learned_addr learned my addr 172.21.15.60:0/677921144 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:53.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbcff7fe640 1 -- 172.21.15.60:0/677921144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 msgr2=0x563b99469480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbcff7fe640 1 --2- 172.21.15.60:0/677921144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 0x563b99469480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbcff7fe640 1 -- 172.21.15.60:0/677921144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b29d0 msgr2=0x563b994b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbcfffff640 1 --2- 172.21.15.60:0/677921144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b29d0 0x563b994b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbcff7fe640 1 --2- 172.21.15.60:0/677921144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b29d0 0x563b994b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbcff7fe640 1 -- 172.21.15.60:0/677921144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b9944c0d0 con 0x563b994b32f0 2024-04-25T10:10:53.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbcff7fe640 1 --2- 172.21.15.60:0/677921144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b32f0 0x563b99468b60 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40ac040 tx=0x7fbcf40a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c26b206b049cc06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbce5ffb640 1 -- 172.21.15.60:0/677921144 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40aa780 con 0x563b994b32f0 2024-04-25T10:10:53.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbce5ffb640 1 -- 172.21.15.60:0/677921144 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf40af040 con 0x563b994b32f0 2024-04-25T10:10:53.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbce5ffb640 1 -- 172.21.15.60:0/677921144 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40aab40 con 0x563b994b32f0 2024-04-25T10:10:53.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 -- 172.21.15.60:0/677921144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b32f0 msgr2=0x563b99468b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 --2- 172.21.15.60:0/677921144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b32f0 0x563b99468b60 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40ac040 tx=0x7fbcf40a7000 comp rx=0 tx=0).stop 2024-04-25T10:10:53.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 -- 172.21.15.60:0/677921144 shutdown_connections 2024-04-25T10:10:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 --2- 172.21.15.60:0/677921144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 0x563b99469480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 --2- 172.21.15.60:0/677921144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b32f0 0x563b99468b60 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 --2- 172.21.15.60:0/677921144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b29d0 0x563b994b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 -- 172.21.15.60:0/677921144 >> 172.21.15.60:0/677921144 conn(0x563b99465020 msgr2=0x563b99467440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 -- 172.21.15.60:0/677921144 shutdown_connections 2024-04-25T10:10:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 -- 172.21.15.60:0/677921144 wait complete. 2024-04-25T10:10:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 -- start start 2024-04-25T10:10:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 Processor -- start 2024-04-25T10:10:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 0x563b994977f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b29d0 0x563b99497d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b32f0 0x563b99498270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b9927a250 con 0x563b994b32f0 2024-04-25T10:10:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b99270690 con 0x563b994b29d0 2024-04-25T10:10:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b9923ea40 con 0x563b994690a0 2024-04-25T10:10:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd09643000 10 client.0 _ll_register_callbacks cb 0x563b99453e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:53.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd04a73640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b32f0 0x563b99498270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd04a73640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b32f0 0x563b99498270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57724/0 (socket says 172.21.15.60:57724) 2024-04-25T10:10:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.129+0000 7fbd04a73640 1 -- 172.21.15.60:0/3558206262 learned_addr learned my addr 172.21.15.60:0/3558206262 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd04a73640 1 -- 172.21.15.60:0/3558206262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 msgr2=0x563b994977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbcfffff640 1 --2- 172.21.15.60:0/3558206262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 0x563b994977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd04a73640 1 --2- 172.21.15.60:0/3558206262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 0x563b994977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbcff7fe640 1 --2- 172.21.15.60:0/3558206262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b29d0 0x563b99497d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd04a73640 1 -- 172.21.15.60:0/3558206262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b29d0 msgr2=0x563b99497d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd04a73640 1 --2- 172.21.15.60:0/3558206262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b29d0 0x563b99497d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd04a73640 1 -- 172.21.15.60:0/3558206262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b993631c0 con 0x563b994b32f0 2024-04-25T10:10:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd04a73640 1 --2- 172.21.15.60:0/3558206262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b32f0 0x563b99498270 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fbcf005e370 tx=0x7fbcf0056250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545f4ae826e1bc34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbcdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:10:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbcdeffd640 1 -- 172.21.15.60:0/3558206262 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf005e8a0 con 0x563b994b32f0 2024-04-25T10:10:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbcdeffd640 1 -- 172.21.15.60:0/3558206262 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fbcf0012dc0 con 0x563b994b32f0 2024-04-25T10:10:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 1 -- 172.21.15.60:0/3558206262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcf40af040 con 0x563b994b32f0 2024-04-25T10:10:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbcdeffd640 1 -- 172.21.15.60:0/3558206262 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcf0052800 con 0x563b994b32f0 2024-04-25T10:10:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 10 client.7380 fetch_fsmap learned FSMap version 46 2024-04-25T10:10:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 1 -- 172.21.15.60:0/3558206262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x563b9935f170 con 0x563b994b32f0 2024-04-25T10:10:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbcdeffd640 1 -- 172.21.15.60:0/3558206262 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf40af040 con 0x563b994b32f0 2024-04-25T10:10:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbcdeffd640 1 -- 172.21.15.60:0/3558206262 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbcf0074120 con 0x563b994b32f0 2024-04-25T10:10:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 10 client.7380 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:10:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 10 client.7380 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:10:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 20 client.7380 populate_metadata read hostname 'smithi060' 2024-04-25T10:10:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 1 --2- 172.21.15.60:0/3558206262 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x563b994c2c80 0x563b994c5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 4 client.7380 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 1 -- 172.21.15.60:0/3558206262 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b99251c60 con 0x563b994c2c80 2024-04-25T10:10:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbcfffff640 1 --2- 172.21.15.60:0/3558206262 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x563b994c2c80 0x563b994c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbcdeffd640 10 client.7380 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbcfffff640 1 --2- 172.21.15.60:0/3558206262 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x563b994c2c80 0x563b994c5070 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=147feb186247d543 server_cookie=be5e7eb3e2298c7e in_seq=0 out_seq=0 2024-04-25T10:10:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbcdeffd640 1 -- 172.21.15.60:0/3558206262 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b99251c60 con 0x563b994c2c80 2024-04-25T10:10:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbcdeffd640 10 client.7380 handle_command_reply: tid=0 2024-04-25T10:10:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 1 client.7380 shutdown 2024-04-25T10:10:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 1 -- 172.21.15.60:0/3558206262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b32f0 msgr2=0x563b99498270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 1 --2- 172.21.15.60:0/3558206262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b32f0 0x563b99498270 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fbcf005e370 tx=0x7fbcf0056250 comp rx=0 tx=0).stop 2024-04-25T10:10:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 1 -- 172.21.15.60:0/3558206262 shutdown_connections 2024-04-25T10:10:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 1 --2- 172.21.15.60:0/3558206262 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x563b994c2c80 0x563b994c5070 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 1 --2- 172.21.15.60:0/3558206262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b32f0 0x563b99498270 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 1 --2- 172.21.15.60:0/3558206262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b29d0 0x563b99497d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 1 --2- 172.21.15.60:0/3558206262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 0x563b994977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbcdeffd640 0 client.7380 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:53.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 1 -- 172.21.15.60:0/3558206262 >> 172.21.15.60:0/3558206262 conn(0x563b99465020 msgr2=0x563b99467ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 1 -- 172.21.15.60:0/3558206262 shutdown_connections 2024-04-25T10:10:53.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 1 -- 172.21.15.60:0/3558206262 wait complete. 2024-04-25T10:10:53.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.133+0000 7fbd09643000 20 client.7380 trim_cache size 0 max 16384 2024-04-25T10:10:53.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 Processor -- start 2024-04-25T10:10:53.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 -- start start 2024-04-25T10:10:53.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 0x563b99468a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b29d0 0x563b994bfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b32f0 0x563b994c0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd00203550 con 0x563b994b29d0 2024-04-25T10:10:53.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd000b1940 con 0x563b994b32f0 2024-04-25T10:10:53.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd000cd5e0 con 0x563b994690a0 2024-04-25T10:10:53.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbcff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b29d0 0x563b994bfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbcff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b29d0 0x563b994bfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57740/0 (socket says 172.21.15.60:57740) 2024-04-25T10:10:53.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbcff7fe640 1 -- 172.21.15.60:0/4273604646 learned_addr learned my addr 172.21.15.60:0/4273604646 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:53.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbcff7fe640 1 -- 172.21.15.60:0/4273604646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 msgr2=0x563b99468a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbcff7fe640 1 --2- 172.21.15.60:0/4273604646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 0x563b99468a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbcff7fe640 1 -- 172.21.15.60:0/4273604646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b32f0 msgr2=0x563b994c0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbcff7fe640 1 --2- 172.21.15.60:0/4273604646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b32f0 0x563b994c0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbcff7fe640 1 -- 172.21.15.60:0/4273604646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b992fcde0 con 0x563b994b29d0 2024-04-25T10:10:53.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbcff7fe640 1 --2- 172.21.15.60:0/4273604646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b29d0 0x563b994bfe40 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40190b0 tx=0x7fbcf4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc4b97df9cce080 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbce57fa640 1 -- 172.21.15.60:0/4273604646 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4004240 con 0x563b994b29d0 2024-04-25T10:10:53.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbce57fa640 1 -- 172.21.15.60:0/4273604646 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf40c2070 con 0x563b994b29d0 2024-04-25T10:10:53.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbce57fa640 1 -- 172.21.15.60:0/4273604646 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4003620 con 0x563b994b29d0 2024-04-25T10:10:53.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 -- 172.21.15.60:0/4273604646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b29d0 msgr2=0x563b994bfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 --2- 172.21.15.60:0/4273604646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b29d0 0x563b994bfe40 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40190b0 tx=0x7fbcf4004740 comp rx=0 tx=0).stop 2024-04-25T10:10:53.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 -- 172.21.15.60:0/4273604646 shutdown_connections 2024-04-25T10:10:53.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 --2- 172.21.15.60:0/4273604646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b32f0 0x563b994c0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 --2- 172.21.15.60:0/4273604646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b29d0 0x563b994bfe40 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 --2- 172.21.15.60:0/4273604646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 0x563b99468a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 -- 172.21.15.60:0/4273604646 >> 172.21.15.60:0/4273604646 conn(0x563b99465020 msgr2=0x563b994bf730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 -- 172.21.15.60:0/4273604646 shutdown_connections 2024-04-25T10:10:53.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 -- 172.21.15.60:0/4273604646 wait complete. 2024-04-25T10:10:53.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.149+0000 7fbd09643000 1 -- start start 2024-04-25T10:10:53.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 Processor -- start 2024-04-25T10:10:53.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 0x563b99512510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b29d0 0x563b99512a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b32f0 0x563b994c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b99250e00 con 0x563b994b29d0 2024-04-25T10:10:53.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd000b1340 con 0x563b994b32f0 2024-04-25T10:10:53.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b992b5300 con 0x563b994690a0 2024-04-25T10:10:53.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 10 client.0 _ll_register_callbacks cb 0x563b99434040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:53.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b29d0 0x563b99512a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b29d0 0x563b99512a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57754/0 (socket says 172.21.15.60:57754) 2024-04-25T10:10:53.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcff7fe640 1 -- 172.21.15.60:0/350818032 learned_addr learned my addr 172.21.15.60:0/350818032 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:10:53.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcff7fe640 1 -- 172.21.15.60:0/350818032 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 msgr2=0x563b99512510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcff7fe640 1 --2- 172.21.15.60:0/350818032 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 0x563b99512510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcff7fe640 1 -- 172.21.15.60:0/350818032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b32f0 msgr2=0x563b994c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcff7fe640 1 --2- 172.21.15.60:0/350818032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b32f0 0x563b994c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcff7fe640 1 -- 172.21.15.60:0/350818032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b992578e0 con 0x563b994b29d0 2024-04-25T10:10:53.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcff7fe640 1 --2- 172.21.15.60:0/350818032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b29d0 0x563b99512a50 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fbcf401c4e0 tx=0x7fbcf401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21c6543440a3806 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:10:53.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcdeffd640 1 -- 172.21.15.60:0/350818032 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40b8040 con 0x563b994b29d0 2024-04-25T10:10:53.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcdeffd640 1 -- 172.21.15.60:0/350818032 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fbcf40cd7b0 con 0x563b994b29d0 2024-04-25T10:10:53.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 -- 172.21.15.60:0/350818032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcf40c2070 con 0x563b994b29d0 2024-04-25T10:10:53.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcdeffd640 1 -- 172.21.15.60:0/350818032 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcf40cd9b0 con 0x563b994b29d0 2024-04-25T10:10:53.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 10 client.7386 fetch_fsmap learned FSMap version 46 2024-04-25T10:10:53.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 -- 172.21.15.60:0/350818032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x563b9930f8f0 con 0x563b994b29d0 2024-04-25T10:10:53.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcdeffd640 1 -- 172.21.15.60:0/350818032 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf40c2070 con 0x563b994b29d0 2024-04-25T10:10:53.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcdeffd640 1 -- 172.21.15.60:0/350818032 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbcf40cdb50 con 0x563b994b29d0 2024-04-25T10:10:53.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 10 client.7386 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:10:53.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 10 client.7386 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:10:53.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 20 client.7386 populate_metadata read hostname 'smithi060' 2024-04-25T10:10:53.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 --2- 172.21.15.60:0/350818032 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x563b994c5380 0x563b994c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 4 client.7386 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:10:53.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 -- 172.21.15.60:0/350818032 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563b992c3530 con 0x563b994c5380 2024-04-25T10:10:53.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcfffff640 1 --2- 172.21.15.60:0/350818032 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x563b994c5380 0x563b994c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcdeffd640 10 client.7386 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:53.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcfffff640 1 --2- 172.21.15.60:0/350818032 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x563b994c5380 0x563b994c5790 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90bf6f0fcd2daaa5 server_cookie=c09d15278f3ea9fb in_seq=0 out_seq=0 2024-04-25T10:10:53.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcdeffd640 1 -- 172.21.15.60:0/350818032 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x563b992c3530 con 0x563b994c5380 2024-04-25T10:10:53.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcdeffd640 10 client.7386 handle_command_reply: tid=0 2024-04-25T10:10:53.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 client.7386 shutdown 2024-04-25T10:10:53.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 -- 172.21.15.60:0/350818032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b29d0 msgr2=0x563b99512a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 --2- 172.21.15.60:0/350818032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b29d0 0x563b99512a50 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fbcf401c4e0 tx=0x7fbcf401ace0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 -- 172.21.15.60:0/350818032 shutdown_connections 2024-04-25T10:10:53.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 --2- 172.21.15.60:0/350818032 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x563b994c5380 0x563b994c5790 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 --2- 172.21.15.60:0/350818032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563b994b32f0 0x563b994c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 --2- 172.21.15.60:0/350818032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563b994b29d0 0x563b99512a50 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 --2- 172.21.15.60:0/350818032 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563b994690a0 0x563b99512510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbcdeffd640 0 client.7386 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:10:53.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 -- 172.21.15.60:0/350818032 >> 172.21.15.60:0/350818032 conn(0x563b99465020 msgr2=0x563b994bffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 -- 172.21.15.60:0/350818032 shutdown_connections 2024-04-25T10:10:53.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 1 -- 172.21.15.60:0/350818032 wait complete. 2024-04-25T10:10:53.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd09643000 20 client.7386 trim_cache size 0 max 16384 2024-04-25T10:10:53.174 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:10:53.174 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:10:53.174 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T10:10:53.174 INFO:teuthology.orchestra.run.smithi060.stdout: "79846d77-eabf-4acf-8fbd-861323055c36": { 2024-04-25T10:10:53.174 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/subdir", 2024-04-25T10:10:53.174 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "79846d77-eabf-4acf-8fbd-861323055c36", 2024-04-25T10:10:53.174 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,repair" 2024-04-25T10:10:53.174 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:10:53.174 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:10:53.174 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:10:53.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd064d6640 1 -- 172.21.15.60:0/258558757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbcec066f90 msgr2=0x7fbcec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd064d6640 1 --2- 172.21.15.60:0/258558757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbcec066f90 0x7fbcec069450 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8007fd0 tx=0x7fbcf8009040 comp rx=0 tx=0).stop 2024-04-25T10:10:53.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd064d6640 1 -- 172.21.15.60:0/258558757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd0015db10 msgr2=0x7fbd00140b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd064d6640 1 --2- 172.21.15.60:0/258558757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd0015db10 0x7fbd00140b00 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0007e60 tx=0x7fbcf000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd064d6640 1 -- 172.21.15.60:0/258558757 shutdown_connections 2024-04-25T10:10:53.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd064d6640 1 --2- 172.21.15.60:0/258558757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbcec066f90 0x7fbcec069450 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd064d6640 1 --2- 172.21.15.60:0/258558757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd0015db10 0x7fbd00140b00 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd064d6640 1 --2- 172.21.15.60:0/258558757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd000b35f0 0x7fbd00151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd064d6640 1 --2- 172.21.15.60:0/258558757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd000aec10 0x7fbd000afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd064d6640 1 -- 172.21.15.60:0/258558757 >> 172.21.15.60:0/258558757 conn(0x7fbd000670b0 msgr2=0x7fbd0015fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd064d6640 1 -- 172.21.15.60:0/258558757 shutdown_connections 2024-04-25T10:10:53.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:10:53.153+0000 7fbd064d6640 1 -- 172.21.15.60:0/258558757 wait complete. 2024-04-25T10:10:53.179 INFO:tasks.cephfs.filesystem:scrub status for tag:79846d77-eabf-4acf-8fbd-861323055c36 - {'path': '/subdir', 'tag': '79846d77-eabf-4acf-8fbd-861323055c36', 'options': 'recursive,repair'} 2024-04-25T10:11:17.067 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:17.072 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:23.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:23.181 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:11:23.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41f658640 1 Processor -- start 2024-04-25T10:11:23.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41f658640 1 -- start start 2024-04-25T10:11:23.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41f658640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa418161f80 0x7fa4180af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41f658640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa418166330 0x7fa41806ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41f658640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa418162360 con 0x7fa4181667b0 2024-04-25T10:11:23.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41f658640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa418165ea0 con 0x7fa418161f80 2024-04-25T10:11:23.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41f658640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4180cfcf0 con 0x7fa418166330 2024-04-25T10:11:23.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41dbf5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa4181667b0 0x7fa4180722a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53068/0 (socket says 172.21.15.60:53068) 2024-04-25T10:11:23.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41dbf5640 1 -- 172.21.15.60:0/326276467 learned_addr learned my addr 172.21.15.60:0/326276467 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:23.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41cbf3640 1 --2- 172.21.15.60:0/326276467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa418166330 0x7fa41806ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41d3f4640 1 --2- 172.21.15.60:0/326276467 >> v2:172.21.15.100:3300/0 conn(0x7fa418161f80 0x7fa4180af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa40ffff640 1 -- 172.21.15.60:0/326276467 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1055350488 0 0) 0x7fa418162360 con 0x7fa4181667b0 2024-04-25T10:11:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa40ffff640 1 -- 172.21.15.60:0/326276467 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa404003530 con 0x7fa4181667b0 2024-04-25T10:11:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa40ffff640 1 -- 172.21.15.60:0/326276467 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4261749646 0 0) 0x7fa404003530 con 0x7fa4181667b0 2024-04-25T10:11:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa40ffff640 1 -- 172.21.15.60:0/326276467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa418166330 msgr2=0x7fa41806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa40ffff640 1 --2- 172.21.15.60:0/326276467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa418166330 0x7fa41806ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa40ffff640 1 -- 172.21.15.60:0/326276467 >> v2:172.21.15.100:3300/0 conn(0x7fa418161f80 msgr2=0x7fa4180af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa40ffff640 1 --2- 172.21.15.60:0/326276467 >> v2:172.21.15.100:3300/0 conn(0x7fa418161f80 0x7fa4180af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa40ffff640 1 -- 172.21.15.60:0/326276467 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4180cffd0 con 0x7fa4181667b0 2024-04-25T10:11:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa40ffff640 1 -- 172.21.15.60:0/326276467 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fa414001680 con 0x7fa4181667b0 2024-04-25T10:11:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa40ffff640 1 -- 172.21.15.60:0/326276467 >> v1:172.21.15.60:6789/0 conn(0x7fa4181667b0 legacy=0x7fa4180722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa40ffff640 1 --2- 172.21.15.60:0/326276467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4040039c0 0x7fa404003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa40ffff640 1 --2- 172.21.15.60:0/326276467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa418166330 0x7fa404004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa40ffff640 1 --2- 172.21.15.60:0/326276467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa404004d50 0x7fa4040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa40ffff640 1 -- 172.21.15.60:0/326276467 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa418165ea0 con 0x7fa404004d50 2024-04-25T10:11:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa40ffff640 1 -- 172.21.15.60:0/326276467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4180cfcf0 con 0x7fa4040039c0 2024-04-25T10:11:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa40ffff640 1 -- 172.21.15.60:0/326276467 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40400b9d0 con 0x7fa418166330 2024-04-25T10:11:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41dbf5640 1 --2- 172.21.15.60:0/326276467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa404004d50 0x7fa4040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41cbf3640 1 --2- 172.21.15.60:0/326276467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa418166330 0x7fa404004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41d3f4640 1 --2- 172.21.15.60:0/326276467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4040039c0 0x7fa404003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41dbf5640 1 -- 172.21.15.60:0/326276467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa418166330 msgr2=0x7fa404004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41dbf5640 1 --2- 172.21.15.60:0/326276467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa418166330 0x7fa404004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41dbf5640 1 -- 172.21.15.60:0/326276467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4040039c0 msgr2=0x7fa404003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41dbf5640 1 --2- 172.21.15.60:0/326276467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4040039c0 0x7fa404003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41dbf5640 1 -- 172.21.15.60:0/326276467 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa404003530 con 0x7fa404004d50 2024-04-25T10:11:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.312+0000 7fa41cbf3640 1 --2- 172.21.15.60:0/326276467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa418166330 0x7fa404004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:23.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.316+0000 7fa41dbf5640 1 --2- 172.21.15.60:0/326276467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa404004d50 0x7fa4040051d0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa41400c210 tx=0x7fa41401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4edfafcf43cba80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.316+0000 7fa40ffff640 1 -- 172.21.15.60:0/326276467 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414007d40 con 0x7fa404004d50 2024-04-25T10:11:23.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.316+0000 7fa40ffff640 1 -- 172.21.15.60:0/326276467 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4140a9020 con 0x7fa404004d50 2024-04-25T10:11:23.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.316+0000 7fa40ffff640 1 -- 172.21.15.60:0/326276467 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4140a5ba0 con 0x7fa404004d50 2024-04-25T10:11:23.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.316+0000 7fa41f658640 1 -- 172.21.15.60:0/326276467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa404004d50 msgr2=0x7fa4040051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.316+0000 7fa41f658640 1 --2- 172.21.15.60:0/326276467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa404004d50 0x7fa4040051d0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa41400c210 tx=0x7fa41401cc20 comp rx=0 tx=0).stop 2024-04-25T10:11:23.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.316+0000 7fa41dbf5640 1 -- 172.21.15.60:0/326276467 reap_dead start 2024-04-25T10:11:23.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.316+0000 7fa41f658640 1 -- 172.21.15.60:0/326276467 shutdown_connections 2024-04-25T10:11:23.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.316+0000 7fa41f658640 1 -- 172.21.15.60:0/326276467 >> 172.21.15.60:0/326276467 conn(0x7fa4180b2360 msgr2=0x7fa4180b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.316+0000 7fa41f658640 1 -- 172.21.15.60:0/326276467 shutdown_connections 2024-04-25T10:11:23.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.316+0000 7fa41f658640 1 -- 172.21.15.60:0/326276467 wait complete. 2024-04-25T10:11:23.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.316+0000 7fa41f658640 1 Processor -- start 2024-04-25T10:11:23.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.316+0000 7fa41f658640 1 -- start start 2024-04-25T10:11:23.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41f658640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa41815ffe0 0x7fa41815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41f658640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa418153ff0 0x7fa418148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41f658640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4181603c0 0x7fa418148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41f658640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41806b950 con 0x7fa4181603c0 2024-04-25T10:11:23.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41f658640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4180d12c0 con 0x7fa41815ffe0 2024-04-25T10:11:23.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41f658640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41806bad0 con 0x7fa418153ff0 2024-04-25T10:11:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41dbf5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4181603c0 0x7fa418148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41dbf5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4181603c0 0x7fa418148950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60760/0 (socket says 172.21.15.60:60760) 2024-04-25T10:11:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41dbf5640 1 -- 172.21.15.60:0/465116625 learned_addr learned my addr 172.21.15.60:0/465116625 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41d3f4640 1 --2- 172.21.15.60:0/465116625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa41815ffe0 0x7fa41815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41cbf3640 1 --2- 172.21.15.60:0/465116625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa418153ff0 0x7fa418148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41dbf5640 1 -- 172.21.15.60:0/465116625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa418153ff0 msgr2=0x7fa418148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41dbf5640 1 --2- 172.21.15.60:0/465116625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa418153ff0 0x7fa418148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41dbf5640 1 -- 172.21.15.60:0/465116625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa41815ffe0 msgr2=0x7fa41815bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41dbf5640 1 --2- 172.21.15.60:0/465116625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa41815ffe0 0x7fa41815bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41dbf5640 1 -- 172.21.15.60:0/465116625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4181544b0 con 0x7fa4181603c0 2024-04-25T10:11:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41cbf3640 1 --2- 172.21.15.60:0/465116625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa418153ff0 0x7fa418148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41dbf5640 1 --2- 172.21.15.60:0/465116625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4181603c0 0x7fa418148950 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fa41401ca50 tx=0x7fa404005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa40dffb640 1 -- 172.21.15.60:0/465116625 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414007d40 con 0x7fa4181603c0 2024-04-25T10:11:23.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa40dffb640 1 -- 172.21.15.60:0/465116625 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4140048b0 con 0x7fa4181603c0 2024-04-25T10:11:23.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41f658640 1 -- 172.21.15.60:0/465116625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa418154790 con 0x7fa4181603c0 2024-04-25T10:11:23.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa40dffb640 1 -- 172.21.15.60:0/465116625 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414001c80 con 0x7fa4181603c0 2024-04-25T10:11:23.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa41f658640 1 -- 172.21.15.60:0/465116625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa418154d50 con 0x7fa4181603c0 2024-04-25T10:11:23.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa40dffb640 1 -- 172.21.15.60:0/465116625 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa4140ae430 con 0x7fa4181603c0 2024-04-25T10:11:23.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa40dffb640 1 --2- 172.21.15.60:0/465116625 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa3fc066f60 0x7fa3fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.320+0000 7fa40dffb640 1 -- 172.21.15.60:0/465116625 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa41411e740 con 0x7fa4181603c0 2024-04-25T10:11:23.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa41d3f4640 1 --2- 172.21.15.60:0/465116625 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa3fc066f60 0x7fa3fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 Processor -- start 2024-04-25T10:11:23.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 -- start start 2024-04-25T10:11:23.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa41d3f4640 1 --2- 172.21.15.60:0/465116625 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa3fc066f60 0x7fa3fc069420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa410009fd0 tx=0x7fa41000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c79d0 0x564ad34c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad34c82f0 0x564ad347db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad347e0a0 0x564ad347e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad32caff0 con 0x564ad34c79d0 2024-04-25T10:11:23.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad3253150 con 0x564ad34c82f0 2024-04-25T10:11:23.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x564ad32c7090 con 0x564ad347e0a0 2024-04-25T10:11:23.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa41d3f4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c79d0 0x564ad34c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa41d3f4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c79d0 0x564ad34c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60764/0 (socket says 172.21.15.60:60764) 2024-04-25T10:11:23.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa41d3f4640 1 -- 172.21.15.60:0/4161137789 learned_addr learned my addr 172.21.15.60:0/4161137789 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:23.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa41cbf3640 1 --2- 172.21.15.60:0/4161137789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad34c82f0 0x564ad347db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa41dbf5640 1 --2- 172.21.15.60:0/4161137789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad347e0a0 0x564ad347e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa41d3f4640 1 -- 172.21.15.60:0/4161137789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad347e0a0 msgr2=0x564ad347e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa41d3f4640 1 --2- 172.21.15.60:0/4161137789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad347e0a0 0x564ad347e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa41d3f4640 1 -- 172.21.15.60:0/4161137789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad34c82f0 msgr2=0x564ad347db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa41d3f4640 1 --2- 172.21.15.60:0/4161137789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad34c82f0 0x564ad347db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa41d3f4640 1 -- 172.21.15.60:0/4161137789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ad34610d0 con 0x564ad34c79d0 2024-04-25T10:11:23.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa41cbf3640 1 --2- 172.21.15.60:0/4161137789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad34c82f0 0x564ad347db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:23.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa41d3f4640 1 --2- 172.21.15.60:0/4161137789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c79d0 0x564ad34c7db0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fa410003e10 tx=0x7fa410015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93ddc9093c374a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa402ffd640 1 -- 172.21.15.60:0/4161137789 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa410018880 con 0x564ad34c79d0 2024-04-25T10:11:23.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa402ffd640 1 -- 172.21.15.60:0/4161137789 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa410018a20 con 0x564ad34c79d0 2024-04-25T10:11:23.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa402ffd640 1 -- 172.21.15.60:0/4161137789 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa410018dd0 con 0x564ad34c79d0 2024-04-25T10:11:23.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 -- 172.21.15.60:0/4161137789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c79d0 msgr2=0x564ad34c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 --2- 172.21.15.60:0/4161137789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c79d0 0x564ad34c7db0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fa410003e10 tx=0x7fa410015000 comp rx=0 tx=0).stop 2024-04-25T10:11:23.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 -- 172.21.15.60:0/4161137789 shutdown_connections 2024-04-25T10:11:23.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 --2- 172.21.15.60:0/4161137789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad347e0a0 0x564ad347e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 --2- 172.21.15.60:0/4161137789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad34c82f0 0x564ad347db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 --2- 172.21.15.60:0/4161137789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c79d0 0x564ad34c7db0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 -- 172.21.15.60:0/4161137789 >> 172.21.15.60:0/4161137789 conn(0x564ad347a020 msgr2=0x564ad347c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 -- 172.21.15.60:0/4161137789 shutdown_connections 2024-04-25T10:11:23.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 -- 172.21.15.60:0/4161137789 wait complete. 2024-04-25T10:11:23.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 -- start start 2024-04-25T10:11:23.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.324+0000 7fa4227c5000 1 Processor -- start 2024-04-25T10:11:23.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad347e0a0 0x564ad34ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c79d0 0x564ad34acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad34c82f0 0x564ad34ad270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad328f250 con 0x564ad34c79d0 2024-04-25T10:11:23.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad3285690 con 0x564ad347e0a0 2024-04-25T10:11:23.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x564ad3253a40 con 0x564ad34c82f0 2024-04-25T10:11:23.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 10 client.0 _ll_register_callbacks cb 0x564ad3468e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:23.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa41cbf3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c79d0 0x564ad34acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa41cbf3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c79d0 0x564ad34acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60770/0 (socket says 172.21.15.60:60770) 2024-04-25T10:11:23.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa41cbf3640 1 -- 172.21.15.60:0/3624972429 learned_addr learned my addr 172.21.15.60:0/3624972429 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:23.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa41dbf5640 1 --2- 172.21.15.60:0/3624972429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad34c82f0 0x564ad34ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa41d3f4640 1 --2- 172.21.15.60:0/3624972429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad347e0a0 0x564ad34ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa41cbf3640 1 -- 172.21.15.60:0/3624972429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad34c82f0 msgr2=0x564ad34ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa41cbf3640 1 --2- 172.21.15.60:0/3624972429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad34c82f0 0x564ad34ad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa41cbf3640 1 -- 172.21.15.60:0/3624972429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad347e0a0 msgr2=0x564ad34ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa41cbf3640 1 --2- 172.21.15.60:0/3624972429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad347e0a0 0x564ad34ac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa41cbf3640 1 -- 172.21.15.60:0/3624972429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ad33781c0 con 0x564ad34c79d0 2024-04-25T10:11:23.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa41cbf3640 1 --2- 172.21.15.60:0/3624972429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c79d0 0x564ad34acd30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fa408013670 tx=0x7fa408013b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8727c6666c33384 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa3f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:11:23.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa3f7fff640 1 -- 172.21.15.60:0/3624972429 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408011a40 con 0x564ad34c79d0 2024-04-25T10:11:23.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 -- 172.21.15.60:0/3624972429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa410018a20 con 0x564ad34c79d0 2024-04-25T10:11:23.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa3f7fff640 1 -- 172.21.15.60:0/3624972429 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa408011be0 con 0x564ad34c79d0 2024-04-25T10:11:23.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa3f7fff640 1 -- 172.21.15.60:0/3624972429 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa408013e70 con 0x564ad34c79d0 2024-04-25T10:11:23.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 10 client.7398 fetch_fsmap learned FSMap version 47 2024-04-25T10:11:23.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 -- 172.21.15.60:0/3624972429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x564ad3374170 con 0x564ad34c79d0 2024-04-25T10:11:23.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa3f7fff640 1 -- 172.21.15.60:0/3624972429 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa410018a20 con 0x564ad34c79d0 2024-04-25T10:11:23.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa3f7fff640 1 -- 172.21.15.60:0/3624972429 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa40802a080 con 0x564ad34c79d0 2024-04-25T10:11:23.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 10 client.7398 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T10:11:23.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 10 client.7398 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:11:23.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 20 client.7398 populate_metadata read hostname 'smithi060' 2024-04-25T10:11:23.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 --2- 172.21.15.60:0/3624972429 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x564ad34d7c80 0x564ad34da070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 4 client.7398 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:23.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 -- 172.21.15.60:0/3624972429 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ad3266c60 con 0x564ad34d7c80 2024-04-25T10:11:23.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa41d3f4640 1 --2- 172.21.15.60:0/3624972429 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x564ad34d7c80 0x564ad34da070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa41d3f4640 1 --2- 172.21.15.60:0/3624972429 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x564ad34d7c80 0x564ad34da070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3712b36cfd39604c server_cookie=a00a539a8bc6eb83 in_seq=0 out_seq=0 2024-04-25T10:11:23.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa3f7fff640 10 client.7398 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:11:23.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa3f7fff640 1 -- 172.21.15.60:0/3624972429 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564ad3266c60 con 0x564ad34d7c80 2024-04-25T10:11:23.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa3f7fff640 10 client.7398 handle_command_reply: tid=0 2024-04-25T10:11:23.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 client.7398 shutdown 2024-04-25T10:11:23.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 -- 172.21.15.60:0/3624972429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c79d0 msgr2=0x564ad34acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 --2- 172.21.15.60:0/3624972429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c79d0 0x564ad34acd30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fa408013670 tx=0x7fa408013b30 comp rx=0 tx=0).stop 2024-04-25T10:11:23.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 -- 172.21.15.60:0/3624972429 shutdown_connections 2024-04-25T10:11:23.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 --2- 172.21.15.60:0/3624972429 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x564ad34d7c80 0x564ad34da070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 --2- 172.21.15.60:0/3624972429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad34c82f0 0x564ad34ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 --2- 172.21.15.60:0/3624972429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c79d0 0x564ad34acd30 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 --2- 172.21.15.60:0/3624972429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad347e0a0 0x564ad34ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa3f7fff640 0 client.7398 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:11:23.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 -- 172.21.15.60:0/3624972429 >> 172.21.15.60:0/3624972429 conn(0x564ad347a020 msgr2=0x564ad347cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 -- 172.21.15.60:0/3624972429 shutdown_connections 2024-04-25T10:11:23.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 1 -- 172.21.15.60:0/3624972429 wait complete. 2024-04-25T10:11:23.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.328+0000 7fa4227c5000 20 client.7398 trim_cache size 0 max 16384 2024-04-25T10:11:23.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.344+0000 7fa4227c5000 1 Processor -- start 2024-04-25T10:11:23.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.344+0000 7fa4227c5000 1 -- start start 2024-04-25T10:11:23.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.344+0000 7fa4227c5000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad347e0a0 0x564ad347c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.344+0000 7fa4227c5000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad34c79d0 0x564ad347c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.344+0000 7fa4227c5000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c82f0 0x564ad347ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.344+0000 7fa4227c5000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa418204b40 con 0x564ad34c82f0 2024-04-25T10:11:23.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.344+0000 7fa4227c5000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa418203d10 con 0x564ad34c79d0 2024-04-25T10:11:23.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.344+0000 7fa4227c5000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4181f8db0 con 0x564ad347e0a0 2024-04-25T10:11:23.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.344+0000 7fa41dbf5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c82f0 0x564ad347ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.344+0000 7fa41dbf5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c82f0 0x564ad347ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60786/0 (socket says 172.21.15.60:60786) 2024-04-25T10:11:23.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.344+0000 7fa41dbf5640 1 -- 172.21.15.60:0/3974989376 learned_addr learned my addr 172.21.15.60:0/3974989376 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:23.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41dbf5640 1 -- 172.21.15.60:0/3974989376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad347e0a0 msgr2=0x564ad347c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41dbf5640 1 --2- 172.21.15.60:0/3974989376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad347e0a0 0x564ad347c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41dbf5640 1 -- 172.21.15.60:0/3974989376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad34c79d0 msgr2=0x564ad347c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41cbf3640 1 --2- 172.21.15.60:0/3974989376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad34c79d0 0x564ad347c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41dbf5640 1 --2- 172.21.15.60:0/3974989376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad34c79d0 0x564ad347c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41dbf5640 1 -- 172.21.15.60:0/3974989376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ad3311de0 con 0x564ad34c82f0 2024-04-25T10:11:23.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41dbf5640 1 --2- 172.21.15.60:0/3974989376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c82f0 0x564ad347ce90 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fa414001c10 tx=0x7fa41411e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635a1d95d531bbde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4027fc640 1 -- 172.21.15.60:0/3974989376 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4140eef30 con 0x564ad34c82f0 2024-04-25T10:11:23.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4027fc640 1 -- 172.21.15.60:0/3974989376 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4140a9040 con 0x564ad34c82f0 2024-04-25T10:11:23.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4027fc640 1 -- 172.21.15.60:0/3974989376 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4140f3080 con 0x564ad34c82f0 2024-04-25T10:11:23.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 -- 172.21.15.60:0/3974989376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c82f0 msgr2=0x564ad347ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 --2- 172.21.15.60:0/3974989376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c82f0 0x564ad347ce90 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fa414001c10 tx=0x7fa41411e040 comp rx=0 tx=0).stop 2024-04-25T10:11:23.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 -- 172.21.15.60:0/3974989376 shutdown_connections 2024-04-25T10:11:23.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 --2- 172.21.15.60:0/3974989376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c82f0 0x564ad347ce90 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 --2- 172.21.15.60:0/3974989376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad34c79d0 0x564ad347c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 --2- 172.21.15.60:0/3974989376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad347e0a0 0x564ad347c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 -- 172.21.15.60:0/3974989376 >> 172.21.15.60:0/3974989376 conn(0x564ad347a020 msgr2=0x564ad34d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 -- 172.21.15.60:0/3974989376 shutdown_connections 2024-04-25T10:11:23.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 -- 172.21.15.60:0/3974989376 wait complete. 2024-04-25T10:11:23.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 -- start start 2024-04-25T10:11:23.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 Processor -- start 2024-04-25T10:11:23.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad347e0a0 0x564ad3527720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad34c79d0 0x564ad3527c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c82f0 0x564ad34d9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ad3265e00 con 0x564ad34c82f0 2024-04-25T10:11:23.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa418203710 con 0x564ad34c79d0 2024-04-25T10:11:23.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x564ad32ca300 con 0x564ad347e0a0 2024-04-25T10:11:23.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 10 client.0 _ll_register_callbacks cb 0x564ad3449040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:23.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41dbf5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c82f0 0x564ad34d9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41dbf5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c82f0 0x564ad34d9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60798/0 (socket says 172.21.15.60:60798) 2024-04-25T10:11:23.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41dbf5640 1 -- 172.21.15.60:0/431720532 learned_addr learned my addr 172.21.15.60:0/431720532 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:23.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41d3f4640 1 --2- 172.21.15.60:0/431720532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad347e0a0 0x564ad3527720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41cbf3640 1 --2- 172.21.15.60:0/431720532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad34c79d0 0x564ad3527c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41dbf5640 1 -- 172.21.15.60:0/431720532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad347e0a0 msgr2=0x564ad3527720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41dbf5640 1 --2- 172.21.15.60:0/431720532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad347e0a0 0x564ad3527720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41dbf5640 1 -- 172.21.15.60:0/431720532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad34c79d0 msgr2=0x564ad3527c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41dbf5640 1 --2- 172.21.15.60:0/431720532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad34c79d0 0x564ad3527c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41dbf5640 1 -- 172.21.15.60:0/431720532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ad326c8e0 con 0x564ad34c82f0 2024-04-25T10:11:23.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa41dbf5640 1 --2- 172.21.15.60:0/431720532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c82f0 0x564ad34d9fc0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fa4140986a0 tx=0x7fa414000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151658db07ee1796 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa3f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:11:23.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa3f7fff640 1 -- 172.21.15.60:0/431720532 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4140eef30 con 0x564ad34c82f0 2024-04-25T10:11:23.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa3f7fff640 1 -- 172.21.15.60:0/431720532 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa414117c80 con 0x564ad34c82f0 2024-04-25T10:11:23.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 -- 172.21.15.60:0/431720532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4140a9040 con 0x564ad34c82f0 2024-04-25T10:11:23.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa3f7fff640 1 -- 172.21.15.60:0/431720532 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa414117e80 con 0x564ad34c82f0 2024-04-25T10:11:23.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 10 client.7404 fetch_fsmap learned FSMap version 47 2024-04-25T10:11:23.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa4227c5000 1 -- 172.21.15.60:0/431720532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x564ad33248f0 con 0x564ad34c82f0 2024-04-25T10:11:23.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.348+0000 7fa3f7fff640 1 -- 172.21.15.60:0/431720532 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4140a9040 con 0x564ad34c82f0 2024-04-25T10:11:23.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa3f7fff640 1 -- 172.21.15.60:0/431720532 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa4140be030 con 0x564ad34c82f0 2024-04-25T10:11:23.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 10 client.7404 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T10:11:23.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 10 client.7404 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:11:23.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 20 client.7404 populate_metadata read hostname 'smithi060' 2024-04-25T10:11:23.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 1 --2- 172.21.15.60:0/431720532 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x564ad34da920 0x564ad34dad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 4 client.7404 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:11:23.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 1 -- 172.21.15.60:0/431720532 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564ad32d8530 con 0x564ad34da920 2024-04-25T10:11:23.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa41d3f4640 1 --2- 172.21.15.60:0/431720532 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x564ad34da920 0x564ad34dad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa3f7fff640 10 client.7404 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:11:23.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa41d3f4640 1 --2- 172.21.15.60:0/431720532 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x564ad34da920 0x564ad34dad00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=328134beb757b36 server_cookie=882a44ee7d804fa8 in_seq=0 out_seq=0 2024-04-25T10:11:23.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa3f7fff640 1 -- 172.21.15.60:0/431720532 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564ad32d8530 con 0x564ad34da920 2024-04-25T10:11:23.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa3f7fff640 10 client.7404 handle_command_reply: tid=0 2024-04-25T10:11:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 1 client.7404 shutdown 2024-04-25T10:11:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 1 -- 172.21.15.60:0/431720532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c82f0 msgr2=0x564ad34d9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 1 --2- 172.21.15.60:0/431720532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c82f0 0x564ad34d9fc0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fa4140986a0 tx=0x7fa414000bd0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 1 -- 172.21.15.60:0/431720532 shutdown_connections 2024-04-25T10:11:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 1 --2- 172.21.15.60:0/431720532 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x564ad34da920 0x564ad34dad00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 1 --2- 172.21.15.60:0/431720532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564ad34c82f0 0x564ad34d9fc0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 1 --2- 172.21.15.60:0/431720532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564ad34c79d0 0x564ad3527c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa3f7fff640 0 client.7404 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:11:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 1 --2- 172.21.15.60:0/431720532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564ad347e0a0 0x564ad3527720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 1 -- 172.21.15.60:0/431720532 >> 172.21.15.60:0/431720532 conn(0x564ad347a020 msgr2=0x564ad34c60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 1 -- 172.21.15.60:0/431720532 shutdown_connections 2024-04-25T10:11:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 1 -- 172.21.15.60:0/431720532 wait complete. 2024-04-25T10:11:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa4227c5000 20 client.7404 trim_cache size 0 max 16384 2024-04-25T10:11:23.367 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:11:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T10:11:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T10:11:23.367 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:11:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa41f658640 1 -- 172.21.15.60:0/465116625 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa3fc066f60 msgr2=0x7fa3fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa41f658640 1 --2- 172.21.15.60:0/465116625 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa3fc066f60 0x7fa3fc069420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa410009fd0 tx=0x7fa41000b040 comp rx=0 tx=0).stop 2024-04-25T10:11:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa41f658640 1 -- 172.21.15.60:0/465116625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4181603c0 msgr2=0x7fa418148950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa41f658640 1 --2- 172.21.15.60:0/465116625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4181603c0 0x7fa418148950 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fa41401ca50 tx=0x7fa404005840 comp rx=0 tx=0).stop 2024-04-25T10:11:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa41f658640 1 -- 172.21.15.60:0/465116625 shutdown_connections 2024-04-25T10:11:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa41f658640 1 --2- 172.21.15.60:0/465116625 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa3fc066f60 0x7fa3fc069420 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa41f658640 1 --2- 172.21.15.60:0/465116625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4181603c0 0x7fa418148950 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa41f658640 1 --2- 172.21.15.60:0/465116625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa418153ff0 0x7fa418148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa41f658640 1 --2- 172.21.15.60:0/465116625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa41815ffe0 0x7fa41815bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa41f658640 1 -- 172.21.15.60:0/465116625 >> 172.21.15.60:0/465116625 conn(0x7fa4180b2360 msgr2=0x7fa4180b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa41f658640 1 -- 172.21.15.60:0/465116625 shutdown_connections 2024-04-25T10:11:23.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.352+0000 7fa41f658640 1 -- 172.21.15.60:0/465116625 wait complete. 2024-04-25T10:11:23.378 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:11:23.378 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:23.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.484+0000 7fc50f774640 1 Processor -- start 2024-04-25T10:11:23.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.484+0000 7fc50f774640 1 -- start start 2024-04-25T10:11:23.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.484+0000 7fc50f774640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc508161290 0x7fc5080b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.484+0000 7fc50f774640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc5080b6fb0 0x7fc5080b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.484+0000 7fc50f774640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5080b9900 con 0x7fc508160eb0 2024-04-25T10:11:23.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.484+0000 7fc50f774640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5080ce9c0 con 0x7fc508161290 2024-04-25T10:11:23.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.484+0000 7fc50f774640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5080ceca0 con 0x7fc5080b6fb0 2024-04-25T10:11:23.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.484+0000 7fc50cd0f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc508161290 0x7fc5080b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.484+0000 7fc50cd0f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc508161290 0x7fc5080b6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60810/0 (socket says 172.21.15.60:60810) 2024-04-25T10:11:23.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.484+0000 7fc50cd0f640 1 -- 172.21.15.60:0/2168218706 learned_addr learned my addr 172.21.15.60:0/2168218706 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:23.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.484+0000 7fc50dd11640 1 --2- 172.21.15.60:0/2168218706 >> v2:172.21.15.100:3300/0 conn(0x7fc5080b6fb0 0x7fc5080b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.484+0000 7fc50cd0f640 1 -- 172.21.15.60:0/2168218706 >> v2:172.21.15.100:3300/0 conn(0x7fc5080b6fb0 msgr2=0x7fc5080b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.484+0000 7fc50cd0f640 1 --2- 172.21.15.60:0/2168218706 >> v2:172.21.15.100:3300/0 conn(0x7fc5080b6fb0 0x7fc5080b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.484+0000 7fc50cd0f640 1 -- 172.21.15.60:0/2168218706 >> v1:172.21.15.100:6789/0 conn(0x7fc508160eb0 legacy=0x7fc5080b91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.484+0000 7fc50cd0f640 1 -- 172.21.15.60:0/2168218706 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5080d2490 con 0x7fc508161290 2024-04-25T10:11:23.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50cd0f640 1 --2- 172.21.15.60:0/2168218706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc508161290 0x7fc5080b6a70 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009870 tx=0x7fc4fc01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f64de7ff1049553 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc4fbfff640 1 -- 172.21.15.60:0/2168218706 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc0b5070 con 0x7fc508161290 2024-04-25T10:11:23.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc4fbfff640 1 -- 172.21.15.60:0/2168218706 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4fc0a89d0 con 0x7fc508161290 2024-04-25T10:11:23.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc4fbfff640 1 -- 172.21.15.60:0/2168218706 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc0a8da0 con 0x7fc508161290 2024-04-25T10:11:23.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50f774640 1 -- 172.21.15.60:0/2168218706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc508161290 msgr2=0x7fc5080b6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50f774640 1 --2- 172.21.15.60:0/2168218706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc508161290 0x7fc5080b6a70 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009870 tx=0x7fc4fc01ecd0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50f774640 1 -- 172.21.15.60:0/2168218706 shutdown_connections 2024-04-25T10:11:23.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50f774640 1 --2- 172.21.15.60:0/2168218706 >> v2:172.21.15.100:3300/0 conn(0x7fc5080b6fb0 0x7fc5080b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50f774640 1 --2- 172.21.15.60:0/2168218706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc508161290 0x7fc5080b6a70 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50f774640 1 -- 172.21.15.60:0/2168218706 >> 172.21.15.60:0/2168218706 conn(0x7fc5080d05c0 msgr2=0x7fc5080d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50f774640 1 -- 172.21.15.60:0/2168218706 shutdown_connections 2024-04-25T10:11:23.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50f774640 1 -- 172.21.15.60:0/2168218706 wait complete. 2024-04-25T10:11:23.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50f774640 1 Processor -- start 2024-04-25T10:11:23.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50f774640 1 -- start start 2024-04-25T10:11:23.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50f774640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5080b6fb0 0x7fc50815ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50f774640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc508160eb0 0x7fc5081e8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50f774640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc508161290 0x7fc5081f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50f774640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5080cef80 con 0x7fc508160eb0 2024-04-25T10:11:23.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50f774640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5080b2e70 con 0x7fc5080b6fb0 2024-04-25T10:11:23.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50f774640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5080b5930 con 0x7fc508161290 2024-04-25T10:11:23.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50cd0f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc508160eb0 0x7fc5081e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50cd0f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc508160eb0 0x7fc5081e8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60824/0 (socket says 172.21.15.60:60824) 2024-04-25T10:11:23.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.488+0000 7fc50cd0f640 1 -- 172.21.15.60:0/2964332319 learned_addr learned my addr 172.21.15.60:0/2964332319 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:23.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc50d510640 1 --2- 172.21.15.60:0/2964332319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5080b6fb0 0x7fc50815ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc50dd11640 1 --2- 172.21.15.60:0/2964332319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc508161290 0x7fc5081f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc50cd0f640 1 -- 172.21.15.60:0/2964332319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc508161290 msgr2=0x7fc5081f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc50cd0f640 1 --2- 172.21.15.60:0/2964332319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc508161290 0x7fc5081f38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc50cd0f640 1 -- 172.21.15.60:0/2964332319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5080b6fb0 msgr2=0x7fc50815ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc50cd0f640 1 --2- 172.21.15.60:0/2964332319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5080b6fb0 0x7fc50815ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc50cd0f640 1 -- 172.21.15.60:0/2964332319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5081e77d0 con 0x7fc508160eb0 2024-04-25T10:11:23.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc50d510640 1 --2- 172.21.15.60:0/2964332319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5080b6fb0 0x7fc50815ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:23.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc50cd0f640 1 --2- 172.21.15.60:0/2964332319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc508160eb0 0x7fc5081e8870 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc01c900 tx=0x7fc4fc01ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc4f9ffb640 1 -- 172.21.15.60:0/2964332319 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc0b5070 con 0x7fc508160eb0 2024-04-25T10:11:23.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc4f9ffb640 1 -- 172.21.15.60:0/2964332319 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4fc0b0a90 con 0x7fc508160eb0 2024-04-25T10:11:23.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc50f774640 1 -- 172.21.15.60:0/2964332319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5081e7ab0 con 0x7fc508160eb0 2024-04-25T10:11:23.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc4f9ffb640 1 -- 172.21.15.60:0/2964332319 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc0bb410 con 0x7fc508160eb0 2024-04-25T10:11:23.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc50f774640 1 -- 172.21.15.60:0/2964332319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5081e8070 con 0x7fc508160eb0 2024-04-25T10:11:23.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc50f774640 1 -- 172.21.15.60:0/2964332319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc508074770 con 0x7fc508160eb0 2024-04-25T10:11:23.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc4f9ffb640 1 -- 172.21.15.60:0/2964332319 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4fc0c2060 con 0x7fc508160eb0 2024-04-25T10:11:23.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc4f9ffb640 1 --2- 172.21.15.60:0/2964332319 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4ec066f60 0x7fc4ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc4f9ffb640 1 -- 172.21.15.60:0/2964332319 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc4fc122540 con 0x7fc508160eb0 2024-04-25T10:11:23.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc50d510640 1 --2- 172.21.15.60:0/2964332319 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4ec066f60 0x7fc4ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.492+0000 7fc50d510640 1 --2- 172.21.15.60:0/2964332319 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4ec066f60 0x7fc4ec069420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc5080cda00 tx=0x7fc504009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.496+0000 7fc4f9ffb640 1 -- 172.21.15.60:0/2964332319 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4fc018830 con 0x7fc508160eb0 2024-04-25T10:11:23.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.688+0000 7fc50f774640 1 -- 172.21.15.60:0/2964332319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5080b2150 con 0x7fc508160eb0 2024-04-25T10:11:23.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.688+0000 7fc4f9ffb640 1 -- 172.21.15.60:0/2964332319 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fc4fc0f35a0 con 0x7fc508160eb0 2024-04-25T10:11:23.698 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:23.698 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3185787428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3185787428},{"type":"v1","addr":"172.21.15.100:6836","nonce":3185787428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2667415244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2667415244},{"type":"v1","addr":"172.21.15.100:6837","nonce":2667415244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/56747111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":56747111},{"type":"v1","addr":"172.21.15.60:6834","nonce":56747111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:43.707831+0000","modified":"2024-04-25T10:10:54.473950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7134},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7134":{"gid":7134,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":16,"addr":"172.21.15.60:6835/1295867731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1295867731},{"type":"v1","addr":"172.21.15.60:6835","nonce":1295867731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:11:23.699 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 47 2024-04-25T10:11:23.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.692+0000 7fc50f774640 1 -- 172.21.15.60:0/2964332319 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4ec066f60 msgr2=0x7fc4ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.692+0000 7fc50f774640 1 --2- 172.21.15.60:0/2964332319 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4ec066f60 0x7fc4ec069420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc5080cda00 tx=0x7fc504009040 comp rx=0 tx=0).stop 2024-04-25T10:11:23.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.692+0000 7fc50f774640 1 -- 172.21.15.60:0/2964332319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc508160eb0 msgr2=0x7fc5081e8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.692+0000 7fc50f774640 1 --2- 172.21.15.60:0/2964332319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc508160eb0 0x7fc5081e8870 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc01c900 tx=0x7fc4fc01ba70 comp rx=0 tx=0).stop 2024-04-25T10:11:23.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.692+0000 7fc50f774640 1 -- 172.21.15.60:0/2964332319 shutdown_connections 2024-04-25T10:11:23.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.692+0000 7fc50f774640 1 --2- 172.21.15.60:0/2964332319 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4ec066f60 0x7fc4ec069420 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.692+0000 7fc50f774640 1 --2- 172.21.15.60:0/2964332319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc508161290 0x7fc5081f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.692+0000 7fc50f774640 1 --2- 172.21.15.60:0/2964332319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc508160eb0 0x7fc5081e8870 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.692+0000 7fc50f774640 1 --2- 172.21.15.60:0/2964332319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5080b6fb0 0x7fc50815ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.692+0000 7fc50f774640 1 -- 172.21.15.60:0/2964332319 >> 172.21.15.60:0/2964332319 conn(0x7fc5080d05c0 msgr2=0x7fc5080b7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.692+0000 7fc50f774640 1 -- 172.21.15.60:0/2964332319 shutdown_connections 2024-04-25T10:11:23.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.692+0000 7fc50f774640 1 -- 172.21.15.60:0/2964332319 wait complete. 2024-04-25T10:11:23.716 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T10:11:23.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.820+0000 7f87c3e5d640 1 Processor -- start 2024-04-25T10:11:23.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.820+0000 7f87c3e5d640 1 -- start start 2024-04-25T10:11:23.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.824+0000 7f87c3e5d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f87bc15db10 0x7f87bc0b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.824+0000 7f87c3e5d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc0cfcf0 0x7f87bc0b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.824+0000 7f87c3e5d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87bc0ae140 con 0x7f87bc0aec10 2024-04-25T10:11:23.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.824+0000 7f87c3e5d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87bc0accc0 con 0x7f87bc15db10 2024-04-25T10:11:23.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.824+0000 7f87c3e5d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc15def0 con 0x7f87bc0cfcf0 2024-04-25T10:11:23.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.824+0000 7f87c1bf9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f87bc15db10 0x7f87bc0b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.824+0000 7f87c23fa640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc0cfcf0 0x7f87bc0b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.824+0000 7f87c23fa640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc0cfcf0 0x7f87bc0b3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:60934/0 (socket says 172.21.15.60:60934) 2024-04-25T10:11:23.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.824+0000 7f87c23fa640 1 -- 172.21.15.60:0/4246251161 learned_addr learned my addr 172.21.15.60:0/4246251161 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:23.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.824+0000 7f87c1bf9640 1 -- 172.21.15.60:0/4246251161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc0cfcf0 msgr2=0x7f87bc0b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.824+0000 7f87c1bf9640 1 --2- 172.21.15.60:0/4246251161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc0cfcf0 0x7f87bc0b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.824+0000 7f87c1bf9640 1 -- 172.21.15.60:0/4246251161 >> v1:172.21.15.100:6789/0 conn(0x7f87bc0aec10 legacy=0x7f87bc161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.824+0000 7f87c1bf9640 1 -- 172.21.15.60:0/4246251161 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc071630 con 0x7f87bc15db10 2024-04-25T10:11:23.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.824+0000 7f87c23fa640 1 --2- 172.21.15.60:0/4246251161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc0cfcf0 0x7f87bc0b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:23.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c1bf9640 1 --2- 172.21.15.60:0/4246251161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc15db10 0x7f87bc0b2ee0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f87b80097c0 tx=0x7f87b801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74799017dfe1aea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c0bf7640 1 -- 172.21.15.60:0/4246251161 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b80b1070 con 0x7f87bc15db10 2024-04-25T10:11:23.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c0bf7640 1 -- 172.21.15.60:0/4246251161 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b80ac620 con 0x7f87bc15db10 2024-04-25T10:11:23.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c0bf7640 1 -- 172.21.15.60:0/4246251161 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b80ac9f0 con 0x7f87bc15db10 2024-04-25T10:11:23.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 -- 172.21.15.60:0/4246251161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc15db10 msgr2=0x7f87bc0b2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 --2- 172.21.15.60:0/4246251161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc15db10 0x7f87bc0b2ee0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f87b80097c0 tx=0x7f87b801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 -- 172.21.15.60:0/4246251161 shutdown_connections 2024-04-25T10:11:23.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 --2- 172.21.15.60:0/4246251161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc0cfcf0 0x7f87bc0b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 --2- 172.21.15.60:0/4246251161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc15db10 0x7f87bc0b2ee0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 -- 172.21.15.60:0/4246251161 >> 172.21.15.60:0/4246251161 conn(0x7f87bc0670b0 msgr2=0x7f87bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 -- 172.21.15.60:0/4246251161 shutdown_connections 2024-04-25T10:11:23.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 -- 172.21.15.60:0/4246251161 wait complete. 2024-04-25T10:11:23.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 Processor -- start 2024-04-25T10:11:23.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 -- start start 2024-04-25T10:11:23.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0aec10 0x7f87bc14e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc0cfcf0 0x7f87bc14cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc15db10 0x7f87bc14d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc06dcf0 con 0x7f87bc0aec10 2024-04-25T10:11:23.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc06db70 con 0x7f87bc0cfcf0 2024-04-25T10:11:23.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc071250 con 0x7f87bc15db10 2024-04-25T10:11:23.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c1bf9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0aec10 0x7f87bc14e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c1bf9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0aec10 0x7f87bc14e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60836/0 (socket says 172.21.15.60:60836) 2024-04-25T10:11:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c1bf9640 1 -- 172.21.15.60:0/3400747242 learned_addr learned my addr 172.21.15.60:0/3400747242 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c23fa640 1 --2- 172.21.15.60:0/3400747242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc15db10 0x7f87bc14d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c1bf9640 1 -- 172.21.15.60:0/3400747242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc15db10 msgr2=0x7f87bc14d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c1bf9640 1 --2- 172.21.15.60:0/3400747242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc15db10 0x7f87bc14d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c1bf9640 1 -- 172.21.15.60:0/3400747242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc0cfcf0 msgr2=0x7f87bc14cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c13f8640 1 --2- 172.21.15.60:0/3400747242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc0cfcf0 0x7f87bc14cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c1bf9640 1 --2- 172.21.15.60:0/3400747242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc0cfcf0 0x7f87bc14cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c1bf9640 1 -- 172.21.15.60:0/3400747242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc0ce710 con 0x7f87bc0aec10 2024-04-25T10:11:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c1bf9640 1 --2- 172.21.15.60:0/3400747242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0aec10 0x7f87bc14e320 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f87b80af040 tx=0x7f87b801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87b27fc640 1 -- 172.21.15.60:0/3400747242 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b80b1050 con 0x7f87bc0aec10 2024-04-25T10:11:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87b27fc640 1 -- 172.21.15.60:0/3400747242 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b80b2040 con 0x7f87bc0aec10 2024-04-25T10:11:23.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 -- 172.21.15.60:0/3400747242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87bc0ce9f0 con 0x7f87bc0aec10 2024-04-25T10:11:23.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87b27fc640 1 -- 172.21.15.60:0/3400747242 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8006e20 con 0x7f87bc0aec10 2024-04-25T10:11:23.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.828+0000 7f87c3e5d640 1 -- 172.21.15.60:0/3400747242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87bc150d70 con 0x7f87bc0aec10 2024-04-25T10:11:23.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87b27fc640 1 -- 172.21.15.60:0/3400747242 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87b80bf030 con 0x7f87bc0aec10 2024-04-25T10:11:23.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87b27fc640 1 --2- 172.21.15.60:0/3400747242 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87a0066f60 0x7f87a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87b27fc640 1 -- 172.21.15.60:0/3400747242 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f87b811ec00 con 0x7f87bc0aec10 2024-04-25T10:11:23.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c13f8640 1 --2- 172.21.15.60:0/3400747242 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87a0066f60 0x7f87a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c6fca000 1 Processor -- start 2024-04-25T10:11:23.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c6fca000 1 -- start start 2024-04-25T10:11:23.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c13f8640 1 --2- 172.21.15.60:0/3400747242 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87a0066f60 0x7f87a0069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f87ac007fd0 tx=0x7f87ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c6fca000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95e02e0 0x5652c95e06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c6fca000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95e0c00 0x5652c95963a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c6fca000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95968e0 0x5652c9596cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c6fca000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c947dba0 con 0x5652c95968e0 2024-04-25T10:11:23.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c6fca000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c93e3090 con 0x5652c95e0c00 2024-04-25T10:11:23.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c6fca000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5652c936b160 con 0x5652c95e02e0 2024-04-25T10:11:23.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c23fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95968e0 0x5652c9596cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c23fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95968e0 0x5652c9596cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60840/0 (socket says 172.21.15.60:60840) 2024-04-25T10:11:23.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c23fa640 1 -- 172.21.15.60:0/1867126600 learned_addr learned my addr 172.21.15.60:0/1867126600 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:23.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c13f8640 1 --2- 172.21.15.60:0/1867126600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95e0c00 0x5652c95963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c23fa640 1 -- 172.21.15.60:0/1867126600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95e02e0 msgr2=0x5652c95e06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c23fa640 1 --2- 172.21.15.60:0/1867126600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95e02e0 0x5652c95e06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c23fa640 1 -- 172.21.15.60:0/1867126600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95e0c00 msgr2=0x5652c95963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c23fa640 1 --2- 172.21.15.60:0/1867126600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95e0c00 0x5652c95963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c23fa640 1 -- 172.21.15.60:0/1867126600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652c948cc20 con 0x5652c95968e0 2024-04-25T10:11:23.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f87c23fa640 1 --2- 172.21.15.60:0/1867126600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95968e0 0x5652c9596cc0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009210 tx=0x7f87b4009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35faf04f8382e026 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f879f7fe640 1 -- 172.21.15.60:0/1867126600 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4013780 con 0x5652c95968e0 2024-04-25T10:11:23.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f879f7fe640 1 -- 172.21.15.60:0/1867126600 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b4019070 con 0x5652c95968e0 2024-04-25T10:11:23.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.832+0000 7f879f7fe640 1 -- 172.21.15.60:0/1867126600 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4013b20 con 0x5652c95968e0 2024-04-25T10:11:23.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 -- 172.21.15.60:0/1867126600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95968e0 msgr2=0x5652c9596cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 --2- 172.21.15.60:0/1867126600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95968e0 0x5652c9596cc0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009210 tx=0x7f87b4009810 comp rx=0 tx=0).stop 2024-04-25T10:11:23.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 -- 172.21.15.60:0/1867126600 shutdown_connections 2024-04-25T10:11:23.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 --2- 172.21.15.60:0/1867126600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95968e0 0x5652c9596cc0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 --2- 172.21.15.60:0/1867126600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95e0c00 0x5652c95963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 --2- 172.21.15.60:0/1867126600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95e02e0 0x5652c95e06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 -- 172.21.15.60:0/1867126600 >> 172.21.15.60:0/1867126600 conn(0x5652c9592890 msgr2=0x5652c9594c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 -- 172.21.15.60:0/1867126600 shutdown_connections 2024-04-25T10:11:23.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 -- 172.21.15.60:0/1867126600 wait complete. 2024-04-25T10:11:23.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 -- start start 2024-04-25T10:11:23.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 Processor -- start 2024-04-25T10:11:23.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95968e0 0x5652c95e7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95e02e0 0x5652c95e7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95e0c00 0x5652c95a1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c93df130 con 0x5652c95e02e0 2024-04-25T10:11:23.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c93a7130 con 0x5652c95e0c00 2024-04-25T10:11:23.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5652c94a1300 con 0x5652c95968e0 2024-04-25T10:11:23.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 10 client.0 _ll_register_callbacks cb 0x5652c9581bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:23.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c13f8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95e02e0 0x5652c95e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c13f8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95e02e0 0x5652c95e7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60854/0 (socket says 172.21.15.60:60854) 2024-04-25T10:11:23.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c13f8640 1 -- 172.21.15.60:0/1847390809 learned_addr learned my addr 172.21.15.60:0/1847390809 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:23.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c1bf9640 1 --2- 172.21.15.60:0/1847390809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95968e0 0x5652c95e7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c23fa640 1 --2- 172.21.15.60:0/1847390809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95e0c00 0x5652c95a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c13f8640 1 -- 172.21.15.60:0/1847390809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95968e0 msgr2=0x5652c95e7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c13f8640 1 --2- 172.21.15.60:0/1847390809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95968e0 0x5652c95e7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c13f8640 1 -- 172.21.15.60:0/1847390809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95e0c00 msgr2=0x5652c95a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c13f8640 1 --2- 172.21.15.60:0/1847390809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95e0c00 0x5652c95a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c13f8640 1 -- 172.21.15.60:0/1847390809 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652c93c0060 con 0x5652c95e02e0 2024-04-25T10:11:23.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c13f8640 1 --2- 172.21.15.60:0/1847390809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95e02e0 0x5652c95e7970 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f87ac014b30 tx=0x7f87ac000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca80d33a7588dcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f879cff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:11:23.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f879cff9640 1 -- 172.21.15.60:0/1847390809 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac012e20 con 0x5652c95e02e0 2024-04-25T10:11:23.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f879cff9640 1 -- 172.21.15.60:0/1847390809 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f87ac01f410 con 0x5652c95e02e0 2024-04-25T10:11:23.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 -- 172.21.15.60:0/1847390809 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87b4019070 con 0x5652c95e02e0 2024-04-25T10:11:23.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f879cff9640 1 -- 172.21.15.60:0/1847390809 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87ac02c050 con 0x5652c95e02e0 2024-04-25T10:11:23.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 10 client.7419 fetch_fsmap learned FSMap version 47 2024-04-25T10:11:23.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 -- 172.21.15.60:0/1847390809 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x5652c93847c0 con 0x5652c95e02e0 2024-04-25T10:11:23.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f879cff9640 1 -- 172.21.15.60:0/1847390809 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b4019070 con 0x5652c95e02e0 2024-04-25T10:11:23.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f879cff9640 1 -- 172.21.15.60:0/1847390809 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f87ac02f020 con 0x5652c95e02e0 2024-04-25T10:11:23.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 10 client.7419 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T10:11:23.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 10 client.7419 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:11:23.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 20 client.7419 populate_metadata read hostname 'smithi060' 2024-04-25T10:11:23.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 --2- 172.21.15.60:0/1847390809 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x5652c95a1e30 0x5652c95a2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 4 client.7419 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:23.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c6fca000 1 -- 172.21.15.60:0/1847390809 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652c937ecd0 con 0x5652c95a1e30 2024-04-25T10:11:23.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c1bf9640 1 --2- 172.21.15.60:0/1847390809 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x5652c95a1e30 0x5652c95a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f879cff9640 10 client.7419 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:11:23.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.836+0000 7f87c1bf9640 1 --2- 172.21.15.60:0/1847390809 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x5652c95a1e30 0x5652c95a2210 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3cef9fac3c32db8 server_cookie=420821243f504abc in_seq=0 out_seq=0 2024-04-25T10:11:23.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.840+0000 7f879cff9640 1 -- 172.21.15.60:0/1847390809 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5652c937ecd0 con 0x5652c95a1e30 2024-04-25T10:11:23.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.840+0000 7f879cff9640 10 client.7419 handle_command_reply: tid=0 2024-04-25T10:11:23.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.840+0000 7f87c6fca000 1 client.7419 shutdown 2024-04-25T10:11:23.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.840+0000 7f87c6fca000 1 -- 172.21.15.60:0/1847390809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95e02e0 msgr2=0x5652c95e7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.840+0000 7f87c6fca000 1 --2- 172.21.15.60:0/1847390809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95e02e0 0x5652c95e7970 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f87ac014b30 tx=0x7f87ac000bd0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.840+0000 7f87c6fca000 1 -- 172.21.15.60:0/1847390809 shutdown_connections 2024-04-25T10:11:23.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.840+0000 7f87c6fca000 1 --2- 172.21.15.60:0/1847390809 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x5652c95a1e30 0x5652c95a2210 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.840+0000 7f87c6fca000 1 --2- 172.21.15.60:0/1847390809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95e0c00 0x5652c95a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.840+0000 7f879cff9640 0 client.7419 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:11:23.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.840+0000 7f87c6fca000 1 --2- 172.21.15.60:0/1847390809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95e02e0 0x5652c95e7970 secure :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f87ac014b30 tx=0x7f87ac000bd0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.840+0000 7f87c6fca000 1 --2- 172.21.15.60:0/1847390809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95968e0 0x5652c95e7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.840+0000 7f87c6fca000 1 -- 172.21.15.60:0/1847390809 >> 172.21.15.60:0/1847390809 conn(0x5652c9592890 msgr2=0x5652c9595510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.840+0000 7f87c6fca000 1 -- 172.21.15.60:0/1847390809 shutdown_connections 2024-04-25T10:11:23.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.840+0000 7f87c6fca000 1 -- 172.21.15.60:0/1847390809 wait complete. 2024-04-25T10:11:23.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.840+0000 7f87c6fca000 20 client.7419 trim_cache size 0 max 16384 2024-04-25T10:11:23.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 Processor -- start 2024-04-25T10:11:23.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 -- start start 2024-04-25T10:11:23.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95968e0 0x5652c9595c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95e0c00 0x5652c95961d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95de3f0 0x5652c95de7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc202020 con 0x5652c95e0c00 2024-04-25T10:11:23.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc201420 con 0x5652c95de3f0 2024-04-25T10:11:23.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc201a20 con 0x5652c95968e0 2024-04-25T10:11:23.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c13f8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95e0c00 0x5652c95961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c13f8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95e0c00 0x5652c95961d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60868/0 (socket says 172.21.15.60:60868) 2024-04-25T10:11:23.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c13f8640 1 -- 172.21.15.60:0/1501054063 learned_addr learned my addr 172.21.15.60:0/1501054063 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:23.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c13f8640 1 -- 172.21.15.60:0/1501054063 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95968e0 msgr2=0x5652c9595c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c13f8640 1 --2- 172.21.15.60:0/1501054063 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95968e0 0x5652c9595c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c13f8640 1 -- 172.21.15.60:0/1501054063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95de3f0 msgr2=0x5652c95de7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c13f8640 1 --2- 172.21.15.60:0/1501054063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95de3f0 0x5652c95de7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c13f8640 1 -- 172.21.15.60:0/1501054063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652c939efc0 con 0x5652c95e0c00 2024-04-25T10:11:23.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c13f8640 1 --2- 172.21.15.60:0/1501054063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95e0c00 0x5652c95961d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f87ac001230 tx=0x7f87ac01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28af29b5e4fadda4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f879effd640 1 -- 172.21.15.60:0/1501054063 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac012070 con 0x5652c95e0c00 2024-04-25T10:11:23.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f879effd640 1 -- 172.21.15.60:0/1501054063 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac024070 con 0x5652c95e0c00 2024-04-25T10:11:23.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f879effd640 1 -- 172.21.15.60:0/1501054063 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac012e20 con 0x5652c95e0c00 2024-04-25T10:11:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 -- 172.21.15.60:0/1501054063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95e0c00 msgr2=0x5652c95961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 --2- 172.21.15.60:0/1501054063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95e0c00 0x5652c95961d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f87ac001230 tx=0x7f87ac01d000 comp rx=0 tx=0).stop 2024-04-25T10:11:23.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 -- 172.21.15.60:0/1501054063 shutdown_connections 2024-04-25T10:11:23.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 --2- 172.21.15.60:0/1501054063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95de3f0 0x5652c95de7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 --2- 172.21.15.60:0/1501054063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95e0c00 0x5652c95961d0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 --2- 172.21.15.60:0/1501054063 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95968e0 0x5652c9595c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 -- 172.21.15.60:0/1501054063 >> 172.21.15.60:0/1501054063 conn(0x5652c9592890 msgr2=0x5652c95edb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 -- 172.21.15.60:0/1501054063 shutdown_connections 2024-04-25T10:11:23.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 -- 172.21.15.60:0/1501054063 wait complete. 2024-04-25T10:11:23.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 -- start start 2024-04-25T10:11:23.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 Processor -- start 2024-04-25T10:11:23.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95968e0 0x5652c9640750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95de3f0 0x5652c9640c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95e0c00 0x5652c95e4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c937de70 con 0x5652c95968e0 2024-04-25T10:11:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c936ba50 con 0x5652c95de3f0 2024-04-25T10:11:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5652c93e23a0 con 0x5652c95e0c00 2024-04-25T10:11:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c6fca000 10 client.0 _ll_register_callbacks cb 0x5652c938f240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c1bf9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95968e0 0x5652c9640750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c1bf9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95968e0 0x5652c9640750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60872/0 (socket says 172.21.15.60:60872) 2024-04-25T10:11:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c1bf9640 1 -- 172.21.15.60:0/321421147 learned_addr learned my addr 172.21.15.60:0/321421147 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c1bf9640 1 -- 172.21.15.60:0/321421147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95e0c00 msgr2=0x5652c95e4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c1bf9640 1 --2- 172.21.15.60:0/321421147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95e0c00 0x5652c95e4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c1bf9640 1 -- 172.21.15.60:0/321421147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95de3f0 msgr2=0x5652c9640c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c1bf9640 1 --2- 172.21.15.60:0/321421147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95de3f0 0x5652c9640c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.852+0000 7f87c1bf9640 1 -- 172.21.15.60:0/321421147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652c943c9e0 con 0x5652c95968e0 2024-04-25T10:11:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c1bf9640 1 --2- 172.21.15.60:0/321421147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95968e0 0x5652c9640750 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f87b801a0d0 tx=0x7f87b80efc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e178743fd05a8a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f879cff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:11:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f879cff9640 1 -- 172.21.15.60:0/321421147 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b810d040 con 0x5652c95968e0 2024-04-25T10:11:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f879cff9640 1 -- 172.21.15.60:0/321421147 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f87b80ebaa0 con 0x5652c95968e0 2024-04-25T10:11:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 1 -- 172.21.15.60:0/321421147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87ac024070 con 0x5652c95968e0 2024-04-25T10:11:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f879cff9640 1 -- 172.21.15.60:0/321421147 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87b80f7b40 con 0x5652c95968e0 2024-04-25T10:11:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 10 client.7425 fetch_fsmap learned FSMap version 47 2024-04-25T10:11:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 1 -- 172.21.15.60:0/321421147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x5652c9382630 con 0x5652c95968e0 2024-04-25T10:11:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f879cff9640 1 -- 172.21.15.60:0/321421147 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac024070 con 0x5652c95968e0 2024-04-25T10:11:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f879cff9640 1 -- 172.21.15.60:0/321421147 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f87b80f3150 con 0x5652c95968e0 2024-04-25T10:11:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 10 client.7425 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T10:11:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 10 client.7425 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:11:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 20 client.7425 populate_metadata read hostname 'smithi060' 2024-04-25T10:11:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 1 --2- 172.21.15.60:0/321421147 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x5652c95e5490 0x5652c95e5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 4 client.7425 mds_command: new command op to 7134 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:11:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 1 -- 172.21.15.60:0/321421147 --> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5652c93f05d0 con 0x5652c95e5490 2024-04-25T10:11:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c13f8640 1 --2- 172.21.15.60:0/321421147 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x5652c95e5490 0x5652c95e5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c13f8640 1 --2- 172.21.15.60:0/321421147 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x5652c95e5490 0x5652c95e5870 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d1010c2df3a69e5 server_cookie=93387ad87532aea3 in_seq=0 out_seq=0 2024-04-25T10:11:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f879cff9640 10 client.7425 ms_handle_connect on v2:172.21.15.60:6833/1295867731 2024-04-25T10:11:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f879cff9640 1 -- 172.21.15.60:0/321421147 <== mds.0 v2:172.21.15.60:6833/1295867731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5652c93f05d0 con 0x5652c95e5490 2024-04-25T10:11:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f879cff9640 10 client.7425 handle_command_reply: tid=0 2024-04-25T10:11:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 1 client.7425 shutdown 2024-04-25T10:11:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 1 -- 172.21.15.60:0/321421147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95968e0 msgr2=0x5652c9640750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 1 --2- 172.21.15.60:0/321421147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95968e0 0x5652c9640750 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f87b801a0d0 tx=0x7f87b80efc90 comp rx=0 tx=0).stop 2024-04-25T10:11:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 1 -- 172.21.15.60:0/321421147 shutdown_connections 2024-04-25T10:11:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 1 --2- 172.21.15.60:0/321421147 >> [v2:172.21.15.60:6833/1295867731,v1:172.21.15.60:6835/1295867731] conn(0x5652c95e5490 0x5652c95e5870 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 1 --2- 172.21.15.60:0/321421147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5652c95e0c00 0x5652c95e4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 1 --2- 172.21.15.60:0/321421147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5652c95de3f0 0x5652c9640c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 1 --2- 172.21.15.60:0/321421147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5652c95968e0 0x5652c9640750 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f879cff9640 0 client.7425 ms_handle_reset on v2:172.21.15.60:6833/1295867731 2024-04-25T10:11:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 1 -- 172.21.15.60:0/321421147 >> 172.21.15.60:0/321421147 conn(0x5652c9592890 msgr2=0x5652c95de7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 1 -- 172.21.15.60:0/321421147 shutdown_connections 2024-04-25T10:11:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 1 -- 172.21.15.60:0/321421147 wait complete. 2024-04-25T10:11:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c6fca000 20 client.7425 trim_cache size 0 max 16384 2024-04-25T10:11:23.871 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:23.872 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-04-25T10:11:23.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c3e5d640 1 -- 172.21.15.60:0/3400747242 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87a0066f60 msgr2=0x7f87a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c3e5d640 1 --2- 172.21.15.60:0/3400747242 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87a0066f60 0x7f87a0069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f87ac007fd0 tx=0x7f87ac009040 comp rx=0 tx=0).stop 2024-04-25T10:11:23.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c3e5d640 1 -- 172.21.15.60:0/3400747242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0aec10 msgr2=0x7f87bc14e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c3e5d640 1 --2- 172.21.15.60:0/3400747242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0aec10 0x7f87bc14e320 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f87b80af040 tx=0x7f87b801ace0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c3e5d640 1 -- 172.21.15.60:0/3400747242 shutdown_connections 2024-04-25T10:11:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c3e5d640 1 --2- 172.21.15.60:0/3400747242 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f87a0066f60 0x7f87a0069420 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c3e5d640 1 --2- 172.21.15.60:0/3400747242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc15db10 0x7f87bc14d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c3e5d640 1 --2- 172.21.15.60:0/3400747242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc0cfcf0 0x7f87bc14cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c3e5d640 1 --2- 172.21.15.60:0/3400747242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0aec10 0x7f87bc14e320 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c3e5d640 1 -- 172.21.15.60:0/3400747242 >> 172.21.15.60:0/3400747242 conn(0x7f87bc0670b0 msgr2=0x7f87bc1600d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c3e5d640 1 -- 172.21.15.60:0/3400747242 shutdown_connections 2024-04-25T10:11:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.856+0000 7f87c3e5d640 1 -- 172.21.15.60:0/3400747242 wait complete. 2024-04-25T10:11:23.881 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:11:23.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.980+0000 7fec6b5d7640 1 Processor -- start 2024-04-25T10:11:23.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.980+0000 7fec6b5d7640 1 -- start start 2024-04-25T10:11:23.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.980+0000 7fec6b5d7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fec6415b9a0 0x7fec640cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.980+0000 7fec6b5d7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fec640cc2c0 0x7fec6415f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.980+0000 7fec6b5d7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6415fe40 con 0x7fec6415f930 2024-04-25T10:11:23.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.980+0000 7fec6b5d7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec64160120 con 0x7fec6415b9a0 2024-04-25T10:11:23.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.980+0000 7fec6b5d7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec640c6980 con 0x7fec640cc2c0 2024-04-25T10:11:23.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.980+0000 7fec69373640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fec6415b9a0 0x7fec640cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.980+0000 7fec69373640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fec6415b9a0 0x7fec640cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60876/0 (socket says 172.21.15.60:60876) 2024-04-25T10:11:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.980+0000 7fec69373640 1 -- 172.21.15.60:0/3497629547 learned_addr learned my addr 172.21.15.60:0/3497629547 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.980+0000 7fec68b72640 1 --2- 172.21.15.60:0/3497629547 >> v2:172.21.15.100:3300/0 conn(0x7fec640cc2c0 0x7fec6415f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.980+0000 7fec69373640 1 -- 172.21.15.60:0/3497629547 >> v2:172.21.15.100:3300/0 conn(0x7fec640cc2c0 msgr2=0x7fec6415f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.980+0000 7fec69373640 1 --2- 172.21.15.60:0/3497629547 >> v2:172.21.15.100:3300/0 conn(0x7fec640cc2c0 0x7fec6415f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.980+0000 7fec69373640 1 -- 172.21.15.60:0/3497629547 >> v1:172.21.15.100:6789/0 conn(0x7fec6415f930 legacy=0x7fec640b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.980+0000 7fec69373640 1 -- 172.21.15.60:0/3497629547 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec640c6c60 con 0x7fec6415b9a0 2024-04-25T10:11:23.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.980+0000 7fec68b72640 1 --2- 172.21.15.60:0/3497629547 >> v2:172.21.15.100:3300/0 conn(0x7fec640cc2c0 0x7fec6415f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.984+0000 7fec69373640 1 --2- 172.21.15.60:0/3497629547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6415b9a0 0x7fec640cbd80 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fec640b4070 tx=0x7fec6001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2890eb49624b4ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.984+0000 7fec5bfff640 1 -- 172.21.15.60:0/3497629547 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec600b0070 con 0x7fec6415b9a0 2024-04-25T10:11:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.984+0000 7fec5bfff640 1 -- 172.21.15.60:0/3497629547 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec600ab600 con 0x7fec6415b9a0 2024-04-25T10:11:23.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.984+0000 7fec5bfff640 1 -- 172.21.15.60:0/3497629547 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec600ab9d0 con 0x7fec6415b9a0 2024-04-25T10:11:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.984+0000 7fec6b5d7640 1 -- 172.21.15.60:0/3497629547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6415b9a0 msgr2=0x7fec640cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.984+0000 7fec6b5d7640 1 --2- 172.21.15.60:0/3497629547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6415b9a0 0x7fec640cbd80 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fec640b4070 tx=0x7fec6001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.984+0000 7fec6b5d7640 1 -- 172.21.15.60:0/3497629547 shutdown_connections 2024-04-25T10:11:23.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.984+0000 7fec6b5d7640 1 --2- 172.21.15.60:0/3497629547 >> v2:172.21.15.100:3300/0 conn(0x7fec640cc2c0 0x7fec6415f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.984+0000 7fec6b5d7640 1 --2- 172.21.15.60:0/3497629547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6415b9a0 0x7fec640cbd80 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.984+0000 7fec6b5d7640 1 -- 172.21.15.60:0/3497629547 >> 172.21.15.60:0/3497629547 conn(0x7fec640669a0 msgr2=0x7fec64066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.984+0000 7fec6b5d7640 1 -- 172.21.15.60:0/3497629547 shutdown_connections 2024-04-25T10:11:23.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.984+0000 7fec6b5d7640 1 -- 172.21.15.60:0/3497629547 wait complete. 2024-04-25T10:11:23.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec6b5d7640 1 Processor -- start 2024-04-25T10:11:23.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec6b5d7640 1 -- start start 2024-04-25T10:11:23.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec6b5d7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec640cc2c0 0x7fec6414c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec6b5d7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6415b9a0 0x7fec640b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec6b5d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6415f930 0x7fec640b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec6b5d7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec640c7990 con 0x7fec6415f930 2024-04-25T10:11:23.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec6b5d7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec640b3c10 con 0x7fec6415b9a0 2024-04-25T10:11:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec6b5d7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec640c7b10 con 0x7fec640cc2c0 2024-04-25T10:11:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec69b74640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6415f930 0x7fec640b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec69b74640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6415f930 0x7fec640b15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60890/0 (socket says 172.21.15.60:60890) 2024-04-25T10:11:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec69b74640 1 -- 172.21.15.60:0/3773948730 learned_addr learned my addr 172.21.15.60:0/3773948730 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec68b72640 1 --2- 172.21.15.60:0/3773948730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6415b9a0 0x7fec640b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec69373640 1 --2- 172.21.15.60:0/3773948730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec640cc2c0 0x7fec6414c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec69b74640 1 -- 172.21.15.60:0/3773948730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec640cc2c0 msgr2=0x7fec6414c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec69b74640 1 --2- 172.21.15.60:0/3773948730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec640cc2c0 0x7fec6414c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec69b74640 1 -- 172.21.15.60:0/3773948730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6415b9a0 msgr2=0x7fec640b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec69b74640 1 --2- 172.21.15.60:0/3773948730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6415b9a0 0x7fec640b1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec69b74640 1 -- 172.21.15.60:0/3773948730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec640d22d0 con 0x7fec6415f930 2024-04-25T10:11:23.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec68b72640 1 --2- 172.21.15.60:0/3773948730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6415b9a0 0x7fec640b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec69b74640 1 --2- 172.21.15.60:0/3773948730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6415f930 0x7fec640b15a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00c780 tx=0x7fec5c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec59ffb640 1 -- 172.21.15.60:0/3773948730 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c004b70 con 0x7fec6415f930 2024-04-25T10:11:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec59ffb640 1 -- 172.21.15.60:0/3773948730 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c004d10 con 0x7fec6415f930 2024-04-25T10:11:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec6b5d7640 1 -- 172.21.15.60:0/3773948730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec641511f0 con 0x7fec6415f930 2024-04-25T10:11:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec59ffb640 1 -- 172.21.15.60:0/3773948730 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00aa90 con 0x7fec6415f930 2024-04-25T10:11:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec6b5d7640 1 -- 172.21.15.60:0/3773948730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec64150c30 con 0x7fec6415f930 2024-04-25T10:11:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec6b5d7640 1 -- 172.21.15.60:0/3773948730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec64074770 con 0x7fec6415f930 2024-04-25T10:11:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec59ffb640 1 -- 172.21.15.60:0/3773948730 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec5c00ac30 con 0x7fec6415f930 2024-04-25T10:11:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec59ffb640 1 --2- 172.21.15.60:0/3773948730 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec48066f60 0x7fec48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec59ffb640 1 -- 172.21.15.60:0/3773948730 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fec5c085380 con 0x7fec6415f930 2024-04-25T10:11:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.988+0000 7fec69373640 1 --2- 172.21.15.60:0/3773948730 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec48066f60 0x7fec48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.992+0000 7fec69373640 1 --2- 172.21.15.60:0/3773948730 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec48066f60 0x7fec48069420 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fec600ad000 tx=0x7fec60008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:23.992+0000 7fec59ffb640 1 -- 172.21.15.60:0/3773948730 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec5c020070 con 0x7fec6415f930 2024-04-25T10:11:24.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.184+0000 7fec6b5d7640 1 -- 172.21.15.60:0/3773948730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fec640c85c0 con 0x7fec6415f930 2024-04-25T10:11:24.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.184+0000 7fec59ffb640 1 -- 172.21.15.60:0/3773948730 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fec5c056300 con 0x7fec6415f930 2024-04-25T10:11:24.195 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:24.195 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:11:24.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.188+0000 7fec6b5d7640 1 -- 172.21.15.60:0/3773948730 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec48066f60 msgr2=0x7fec48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.188+0000 7fec6b5d7640 1 --2- 172.21.15.60:0/3773948730 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec48066f60 0x7fec48069420 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fec600ad000 tx=0x7fec60008040 comp rx=0 tx=0).stop 2024-04-25T10:11:24.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.188+0000 7fec6b5d7640 1 -- 172.21.15.60:0/3773948730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6415f930 msgr2=0x7fec640b15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.188+0000 7fec6b5d7640 1 --2- 172.21.15.60:0/3773948730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6415f930 0x7fec640b15a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00c780 tx=0x7fec5c00cc40 comp rx=0 tx=0).stop 2024-04-25T10:11:24.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.188+0000 7fec6b5d7640 1 -- 172.21.15.60:0/3773948730 shutdown_connections 2024-04-25T10:11:24.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.188+0000 7fec6b5d7640 1 --2- 172.21.15.60:0/3773948730 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fec48066f60 0x7fec48069420 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.188+0000 7fec6b5d7640 1 --2- 172.21.15.60:0/3773948730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6415f930 0x7fec640b15a0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.188+0000 7fec6b5d7640 1 --2- 172.21.15.60:0/3773948730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6415b9a0 0x7fec640b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.188+0000 7fec6b5d7640 1 --2- 172.21.15.60:0/3773948730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec640cc2c0 0x7fec6414c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.188+0000 7fec6b5d7640 1 -- 172.21.15.60:0/3773948730 >> 172.21.15.60:0/3773948730 conn(0x7fec640669a0 msgr2=0x7fec640d16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:24.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.188+0000 7fec6b5d7640 1 -- 172.21.15.60:0/3773948730 shutdown_connections 2024-04-25T10:11:24.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.188+0000 7fec6b5d7640 1 -- 172.21.15.60:0/3773948730 wait complete. 2024-04-25T10:11:24.212 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:11:24.213 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:11:24.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.324+0000 7fdc337e4640 1 Processor -- start 2024-04-25T10:11:24.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.324+0000 7fdc337e4640 1 -- start start 2024-04-25T10:11:24.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.324+0000 7fdc337e4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2c0ae7c0 0x7fdc2c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.324+0000 7fdc337e4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdc2c06d2f0 0x7fdc2c0717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.324+0000 7fdc337e4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2c071fd0 con 0x7fdc2c15dc90 2024-04-25T10:11:24.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.324+0000 7fdc337e4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2c0accc0 con 0x7fdc2c06d2f0 2024-04-25T10:11:24.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.324+0000 7fdc337e4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0ad260 con 0x7fdc2c0ae7c0 2024-04-25T10:11:24.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.324+0000 7fdc31580640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdc2c15dc90 0x7fdc2c0b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53080/0 (socket says 172.21.15.60:53080) 2024-04-25T10:11:24.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.324+0000 7fdc31580640 1 -- 172.21.15.60:0/2837939268 learned_addr learned my addr 172.21.15.60:0/2837939268 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:24.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.324+0000 7fdc31d81640 1 --2- 172.21.15.60:0/2837939268 >> v2:172.21.15.100:3300/0 conn(0x7fdc2c06d2f0 0x7fdc2c0717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 -- 172.21.15.60:0/2837939268 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407532201 0 0) 0x7fdc2c071fd0 con 0x7fdc2c15dc90 2024-04-25T10:11:24.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.324+0000 7fdc30d7f640 1 --2- 172.21.15.60:0/2837939268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2c0ae7c0 0x7fdc2c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 -- 172.21.15.60:0/2837939268 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc18003530 con 0x7fdc2c15dc90 2024-04-25T10:11:24.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 -- 172.21.15.60:0/2837939268 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2977827014 0 0) 0x7fdc18003530 con 0x7fdc2c15dc90 2024-04-25T10:11:24.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 -- 172.21.15.60:0/2837939268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2c0ae7c0 msgr2=0x7fdc2c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:24.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 --2- 172.21.15.60:0/2837939268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2c0ae7c0 0x7fdc2c071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 -- 172.21.15.60:0/2837939268 >> v2:172.21.15.100:3300/0 conn(0x7fdc2c06d2f0 msgr2=0x7fdc2c0717d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:24.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 --2- 172.21.15.60:0/2837939268 >> v2:172.21.15.100:3300/0 conn(0x7fdc2c06d2f0 0x7fdc2c0717d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 -- 172.21.15.60:0/2837939268 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c0b2ce0 con 0x7fdc2c15dc90 2024-04-25T10:11:24.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 -- 172.21.15.60:0/2837939268 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fdc28001a00 con 0x7fdc2c15dc90 2024-04-25T10:11:24.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 -- 172.21.15.60:0/2837939268 >> v1:172.21.15.60:6789/0 conn(0x7fdc2c15dc90 legacy=0x7fdc2c0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 --2- 172.21.15.60:0/2837939268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2c0ae7c0 0x7fdc180039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 --2- 172.21.15.60:0/2837939268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc18003f90 0x7fdc18004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 --2- 172.21.15.60:0/2837939268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18004d50 0x7fdc180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 -- 172.21.15.60:0/2837939268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0ad260 con 0x7fdc18003f90 2024-04-25T10:11:24.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 -- 172.21.15.60:0/2837939268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1800b9d0 con 0x7fdc18004d50 2024-04-25T10:11:24.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 -- 172.21.15.60:0/2837939268 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1800bb50 con 0x7fdc2c0ae7c0 2024-04-25T10:11:24.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc30d7f640 1 --2- 172.21.15.60:0/2837939268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc18003f90 0x7fdc18004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc30d7f640 1 -- 172.21.15.60:0/2837939268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2c0ae7c0 msgr2=0x7fdc180039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:24.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc31d81640 1 --2- 172.21.15.60:0/2837939268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18004d50 0x7fdc180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc31580640 1 --2- 172.21.15.60:0/2837939268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2c0ae7c0 0x7fdc180039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc30d7f640 1 --2- 172.21.15.60:0/2837939268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2c0ae7c0 0x7fdc180039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc30d7f640 1 -- 172.21.15.60:0/2837939268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18004d50 msgr2=0x7fdc180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:24.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc30d7f640 1 --2- 172.21.15.60:0/2837939268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18004d50 0x7fdc180051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc30d7f640 1 -- 172.21.15.60:0/2837939268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc18003530 con 0x7fdc18003f90 2024-04-25T10:11:24.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc31d81640 1 --2- 172.21.15.60:0/2837939268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18004d50 0x7fdc180051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:24.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc31580640 1 --2- 172.21.15.60:0/2837939268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2c0ae7c0 0x7fdc180039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:24.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc30d7f640 1 --2- 172.21.15.60:0/2837939268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc18003f90 0x7fdc18004810 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009b10 tx=0x7fdc1c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8450bc498fb25d03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 -- 172.21.15.60:0/2837939268 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0b2070 con 0x7fdc18003f90 2024-04-25T10:11:24.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 -- 172.21.15.60:0/2837939268 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c0a5660 con 0x7fdc18003f90 2024-04-25T10:11:24.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc23fff640 1 -- 172.21.15.60:0/2837939268 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0096b0 con 0x7fdc18003f90 2024-04-25T10:11:24.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc337e4640 1 -- 172.21.15.60:0/2837939268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc18003f90 msgr2=0x7fdc18004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc337e4640 1 --2- 172.21.15.60:0/2837939268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc18003f90 0x7fdc18004810 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009b10 tx=0x7fdc1c01ca70 comp rx=0 tx=0).stop 2024-04-25T10:11:24.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc31d81640 1 -- 172.21.15.60:0/2837939268 reap_dead start 2024-04-25T10:11:24.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc337e4640 1 -- 172.21.15.60:0/2837939268 shutdown_connections 2024-04-25T10:11:24.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc337e4640 1 -- 172.21.15.60:0/2837939268 >> 172.21.15.60:0/2837939268 conn(0x7fdc2c0b12d0 msgr2=0x7fdc2c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:24.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc337e4640 1 -- 172.21.15.60:0/2837939268 shutdown_connections 2024-04-25T10:11:24.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.328+0000 7fdc337e4640 1 -- 172.21.15.60:0/2837939268 wait complete. 2024-04-25T10:11:24.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc337e4640 1 Processor -- start 2024-04-25T10:11:24.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc337e4640 1 -- start start 2024-04-25T10:11:24.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc337e4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18003f90 0x7fdc2c15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc337e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc2c15bf60 0x7fdc2c15a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc337e4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2c15ab10 0x7fdc2c1447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc337e4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0b2010 con 0x7fdc2c15bf60 2024-04-25T10:11:24.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc337e4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0acef0 con 0x7fdc18003f90 2024-04-25T10:11:24.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc337e4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0b2190 con 0x7fdc2c15ab10 2024-04-25T10:11:24.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc30d7f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc2c15bf60 0x7fdc2c15a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc30d7f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc2c15bf60 0x7fdc2c15a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60908/0 (socket says 172.21.15.60:60908) 2024-04-25T10:11:24.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc30d7f640 1 -- 172.21.15.60:0/1359350535 learned_addr learned my addr 172.21.15.60:0/1359350535 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:24.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc31d81640 1 --2- 172.21.15.60:0/1359350535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2c15ab10 0x7fdc2c1447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc31580640 1 --2- 172.21.15.60:0/1359350535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18003f90 0x7fdc2c15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc30d7f640 1 -- 172.21.15.60:0/1359350535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2c15ab10 msgr2=0x7fdc2c1447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc30d7f640 1 --2- 172.21.15.60:0/1359350535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2c15ab10 0x7fdc2c1447d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc30d7f640 1 -- 172.21.15.60:0/1359350535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18003f90 msgr2=0x7fdc2c15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc30d7f640 1 --2- 172.21.15.60:0/1359350535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18003f90 0x7fdc2c15ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc30d7f640 1 -- 172.21.15.60:0/1359350535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c15b5f0 con 0x7fdc2c15bf60 2024-04-25T10:11:24.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc31580640 1 --2- 172.21.15.60:0/1359350535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18003f90 0x7fdc2c15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:24.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc30d7f640 1 --2- 172.21.15.60:0/1359350535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc2c15bf60 0x7fdc2c15a5d0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009b10 tx=0x7fdc18005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc21ffb640 1 -- 172.21.15.60:0/1359350535 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0b2040 con 0x7fdc2c15bf60 2024-04-25T10:11:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc21ffb640 1 -- 172.21.15.60:0/1359350535 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c00f000 con 0x7fdc2c15bf60 2024-04-25T10:11:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc337e4640 1 -- 172.21.15.60:0/1359350535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2c15b310 con 0x7fdc2c15bf60 2024-04-25T10:11:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc21ffb640 1 -- 172.21.15.60:0/1359350535 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc280020d0 con 0x7fdc2c15bf60 2024-04-25T10:11:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc337e4640 1 -- 172.21.15.60:0/1359350535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c153fe0 con 0x7fdc2c15bf60 2024-04-25T10:11:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.332+0000 7fdc337e4640 1 -- 172.21.15.60:0/1359350535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2c074770 con 0x7fdc2c15bf60 2024-04-25T10:11:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.336+0000 7fdc21ffb640 1 -- 172.21.15.60:0/1359350535 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc1c0bd020 con 0x7fdc2c15bf60 2024-04-25T10:11:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.336+0000 7fdc21ffb640 1 --2- 172.21.15.60:0/1359350535 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc10066f90 0x7fdc10069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.336+0000 7fdc21ffb640 1 -- 172.21.15.60:0/1359350535 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fdc1c11e7d0 con 0x7fdc2c15bf60 2024-04-25T10:11:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.336+0000 7fdc31580640 1 --2- 172.21.15.60:0/1359350535 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc10066f90 0x7fdc10069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.336+0000 7fdc31580640 1 --2- 172.21.15.60:0/1359350535 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc10066f90 0x7fdc10069450 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c1f6c60 tx=0x7fdc28001840 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.340+0000 7fdc21ffb640 1 -- 172.21.15.60:0/1359350535 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc1c0167c0 con 0x7fdc2c15bf60 2024-04-25T10:11:24.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.536+0000 7fdc337e4640 1 -- 172.21.15.60:0/1359350535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdc2c06e350 con 0x7fdc2c15bf60 2024-04-25T10:11:24.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.536+0000 7fdc21ffb640 1 -- 172.21.15.60:0/1359350535 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdc1c0ef900 con 0x7fdc2c15bf60 2024-04-25T10:11:24.544 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:11:24.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.536+0000 7fdc337e4640 1 -- 172.21.15.60:0/1359350535 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc10066f90 msgr2=0x7fdc10069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.536+0000 7fdc337e4640 1 --2- 172.21.15.60:0/1359350535 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc10066f90 0x7fdc10069450 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c1f6c60 tx=0x7fdc28001840 comp rx=0 tx=0).stop 2024-04-25T10:11:24.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.536+0000 7fdc337e4640 1 -- 172.21.15.60:0/1359350535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc2c15bf60 msgr2=0x7fdc2c15a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.536+0000 7fdc337e4640 1 --2- 172.21.15.60:0/1359350535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc2c15bf60 0x7fdc2c15a5d0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009b10 tx=0x7fdc18005840 comp rx=0 tx=0).stop 2024-04-25T10:11:24.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.536+0000 7fdc337e4640 1 -- 172.21.15.60:0/1359350535 shutdown_connections 2024-04-25T10:11:24.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.536+0000 7fdc337e4640 1 --2- 172.21.15.60:0/1359350535 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc10066f90 0x7fdc10069450 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.536+0000 7fdc337e4640 1 --2- 172.21.15.60:0/1359350535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2c15ab10 0x7fdc2c1447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.536+0000 7fdc337e4640 1 --2- 172.21.15.60:0/1359350535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc2c15bf60 0x7fdc2c15a5d0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.536+0000 7fdc337e4640 1 --2- 172.21.15.60:0/1359350535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc18003f90 0x7fdc2c15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.536+0000 7fdc337e4640 1 -- 172.21.15.60:0/1359350535 >> 172.21.15.60:0/1359350535 conn(0x7fdc2c0b12d0 msgr2=0x7fdc2c0af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:24.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.536+0000 7fdc337e4640 1 -- 172.21.15.60:0/1359350535 shutdown_connections 2024-04-25T10:11:24.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:24.536+0000 7fdc337e4640 1 -- 172.21.15.60:0/1359350535 wait complete. 2024-04-25T10:11:24.563 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:11:24.563 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:11:24.563 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:11:24.563 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:11:24.563 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:24.563 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:11:24.563 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:11:24.563 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:11:24.563 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:24.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:24.564 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:11:24.570 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:11:24.570 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:11:24.570 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:11:24.570 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:24.570 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:11:24.570 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:24.570 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:11:24.571 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:24.571 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:11:24.571 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:11:24.571 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:24.571 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:11:24.571 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:11:24.572 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:24.572 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:11:24.572 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:11:24.572 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42172sec preferred_lft 42172sec 2024-04-25T10:11:24.572 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:11:24.572 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:24.572 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:11:24.572 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:24.572 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:11:24.572 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:11:24.572 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:24.572 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:11:24.572 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:24.572 INFO:teuthology.orchestra.run.smithi060.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:11:24.572 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:24.573 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:11:24.573 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:24.573 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:11:24.573 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:24.573 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:24.573 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:24.573 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:11:24.573 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:24.573 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:24.573 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:24.573 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:11:24.573 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:24.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:24.574 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:11:24.620 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:11:24.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:24.622 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:11:24.669 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:11:24.669 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:11:24.669 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:11:24.669 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:11:24.669 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:11:24.669 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:11:24.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:24.670 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:11:24.670 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:11:24.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:11:24.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:11:24.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:11:24.670 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:11:24.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:24.743 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:11:24.792 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:11:24.792 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:11:24.793 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:24.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:24.793 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T10:11:24.847 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:11:24.847 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:24.847 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:24.896 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:24.897 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:24.897 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:24.943 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:11:25.001 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:11:25.049 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:11:25.049 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:11:25.049 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:11:25.049 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:11:25.049 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:11:25.049 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:11:25.049 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:11:25.049 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:11:25.049 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:11:25.049 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:11:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:11:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:11:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:11:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:11:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:11:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:11:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:11:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:11:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:11:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:11:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:11:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:11:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:11:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:11:25.051 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:11:25.051 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:11:25.051 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:11:25.051 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:11:25.051 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:11:25.051 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:25.051 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:25.051 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:25.051 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:25.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:25.052 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:11:25.100 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:11:25.100 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:11:25.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:11:25.160 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:11:25.208 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:11:25.208 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:11:25.208 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:11:25.208 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:11:25.208 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:11:25.208 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:11:25.208 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:11:25.209 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:11:25.209 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:11:25.209 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:11:25.209 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:11:25.209 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:11:25.209 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:11:25.209 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:11:25.209 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:11:25.209 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:11:25.209 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:11:25.209 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:11:25.209 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:11:25.209 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:11:25.209 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:11:25.210 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:11:25.210 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:11:25.210 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:11:25.210 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:11:25.210 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:11:25.210 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:11:25.210 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:11:25.210 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:11:25.210 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:25.210 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:25.210 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:25.210 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:25.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:25.211 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:11:25.222 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2024-04-25T10:11:25.212+0000 7f2167e50580 -1 init, newargv = 0x56115cb00f40 newargc=15 2024-04-25T10:11:25.222 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[43948]: starting ceph client 2024-04-25T10:11:25.227 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[43948]: starting fuse 2024-04-25T10:11:26.328 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:11:26.345 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:11:26.394 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:11:26.394 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:11:26.394 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:11:26.395 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:11:26.395 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:11:26.395 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:11:26.395 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:11:26.395 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:11:26.395 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:11:26.395 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:11:26.395 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:11:26.395 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:11:26.395 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:11:26.395 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:11:26.395 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:11:26.395 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:11:26.395 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:11:26.396 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:11:26.396 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:11:26.396 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:11:26.397 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:11:26.397 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:11:26.397 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:11:26.397 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:11:26.397 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:11:26.397 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:11:26.397 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:11:26.397 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:11:26.397 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:11:26.398 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:26.398 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:26.398 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:26.398 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:26.398 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:26.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:26.399 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:11:26.448 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:11:26.448 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:11:26.449 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:11:26.449 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:11:26.449 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:11:26.449 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:11:26.449 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:11:26.449 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:11:26.449 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:11:26.449 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:11:26.449 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:11:26.449 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:11:26.449 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:11:26.449 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:11:26.449 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:11:26.449 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:11:26.449 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:11:26.450 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:11:26.450 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:11:26.450 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:11:26.450 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:11:26.450 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:11:26.450 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:11:26.450 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:11:26.450 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:11:26.450 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:11:26.450 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:11:26.450 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2024-04-25T10:11:26.450 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:11:26.575 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.43948.asok 2024-04-25T10:11:26.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:11:26.588 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43948.asok 2024-04-25T10:11:26.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:26.588 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43948.asok status 2024-04-25T10:11:26.748 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:11:26.748 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:11:26.748 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:11:26.748 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:11:26.748 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2024-04-25T10:11:26.749 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:11:26.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:11:26.749 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "43948", 2024-04-25T10:11:26.749 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:11:26.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:26.749 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:11:26.749 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:11:26.749 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7443, 2024-04-25T10:11:26.749 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:11:26.749 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:11:26.749 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:11:26.749 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 7443 2024-04-25T10:11:26.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:26.749 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:11:26.750 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:11:26.750 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:11:26.750 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1713903035 2024-04-25T10:11:26.750 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:26.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:26.750 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:11:26.750 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:11:26.750 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:11:26.750 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1713903035 2024-04-25T10:11:26.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:26.750 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.7443 192.168.0.1:0/1713903035", 2024-04-25T10:11:26.750 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/1713903035", 2024-04-25T10:11:26.751 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:11:26.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 47, 2024-04-25T10:11:26.751 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 71, 2024-04-25T10:11:26.751 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:11:26.751 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:11:26.751 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:11:26.751 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:11:26.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:26.763 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:11:26.763 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:26.770 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:11:26.770 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:26.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:26.770 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:26.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:26.821 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T10:11:26.872 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:11:26.873 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T10:11:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.940+0000 7f41757b8080 1 Processor -- start 2024-04-25T10:11:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.940+0000 7f41757b8080 1 -- start start 2024-04-25T10:11:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f415c0022b0 0x5624810c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624810d9d90 0x562481048380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56248104ded0 0x5624810c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562480fa99f0 con 0x5624810d9d90 2024-04-25T10:11:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562480fad950 con 0x56248104ded0 2024-04-25T10:11:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624810040b0 con 0x7f415c0022b0 2024-04-25T10:11:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4172ad5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624810d9d90 0x562481048380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4172ad5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624810d9d90 0x562481048380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60918/0 (socket says 172.21.15.60:60918) 2024-04-25T10:11:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4172ad5640 1 -- 172.21.15.60:0/944821298 learned_addr learned my addr 172.21.15.60:0/944821298 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4173ad7640 1 --2- 172.21.15.60:0/944821298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56248104ded0 0x5624810c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4172ad5640 1 -- 172.21.15.60:0/944821298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f415c0022b0 msgr2=0x5624810c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4172ad5640 1 --2- 172.21.15.60:0/944821298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f415c0022b0 0x5624810c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4172ad5640 1 -- 172.21.15.60:0/944821298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56248104ded0 msgr2=0x5624810c44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4172ad5640 1 --2- 172.21.15.60:0/944821298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56248104ded0 0x5624810c44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4172ad5640 1 -- 172.21.15.60:0/944821298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562481033810 con 0x5624810d9d90 2024-04-25T10:11:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4173ad7640 1 --2- 172.21.15.60:0/944821298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56248104ded0 0x5624810c44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4172ad5640 1 --2- 172.21.15.60:0/944821298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624810d9d90 0x562481048380 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f416800d570 tx=0x7f416800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b5163626b97848 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4171ad3640 1 -- 172.21.15.60:0/944821298 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416800b990 con 0x5624810d9d90 2024-04-25T10:11:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4171ad3640 1 -- 172.21.15.60:0/944821298 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4168012030 con 0x5624810d9d90 2024-04-25T10:11:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4171ad3640 1 -- 172.21.15.60:0/944821298 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416800bd50 con 0x5624810d9d90 2024-04-25T10:11:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 -- 172.21.15.60:0/944821298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624810d9d90 msgr2=0x562481048380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 --2- 172.21.15.60:0/944821298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624810d9d90 0x562481048380 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f416800d570 tx=0x7f416800da30 comp rx=0 tx=0).stop 2024-04-25T10:11:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 -- 172.21.15.60:0/944821298 shutdown_connections 2024-04-25T10:11:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 --2- 172.21.15.60:0/944821298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56248104ded0 0x5624810c44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 --2- 172.21.15.60:0/944821298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624810d9d90 0x562481048380 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 --2- 172.21.15.60:0/944821298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f415c0022b0 0x5624810c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 -- 172.21.15.60:0/944821298 >> 172.21.15.60:0/944821298 conn(0x56248102fe50 msgr2=0x562481031df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 -- 172.21.15.60:0/944821298 shutdown_connections 2024-04-25T10:11:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 -- 172.21.15.60:0/944821298 wait complete. 2024-04-25T10:11:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 Processor -- start 2024-04-25T10:11:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 -- start start 2024-04-25T10:11:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56248104ded0 0x5624810c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624810d9d90 0x562481175fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f415c0022b0 0x5624810be500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562481033ab0 con 0x5624810d9d90 2024-04-25T10:11:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562481033d50 con 0x7f415c0022b0 2024-04-25T10:11:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41757b8080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x562481033ff0 con 0x56248104ded0 2024-04-25T10:11:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4172ad5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624810d9d90 0x562481175fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4172ad5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624810d9d90 0x562481175fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60926/0 (socket says 172.21.15.60:60926) 2024-04-25T10:11:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4172ad5640 1 -- 172.21.15.60:0/1542119625 learned_addr learned my addr 172.21.15.60:0/1542119625 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f41732d6640 1 --2- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56248104ded0 0x5624810c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.944+0000 7f4173ad7640 1 --2- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f415c0022b0 0x5624810be500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f4172ad5640 1 -- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56248104ded0 msgr2=0x5624810c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f4172ad5640 1 --2- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56248104ded0 0x5624810c4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f4172ad5640 1 -- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f415c0022b0 msgr2=0x5624810be500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f4172ad5640 1 --2- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f415c0022b0 0x5624810be500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f4172ad5640 1 -- 172.21.15.60:0/1542119625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562481176760 con 0x5624810d9d90 2024-04-25T10:11:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f41732d6640 1 --2- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56248104ded0 0x5624810c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f4172ad5640 1 --2- 172.21.15.60:0/1542119625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624810d9d90 0x562481175fe0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f416800bc00 tx=0x7f4168001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f4170ad1640 1 -- 172.21.15.60:0/1542119625 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168017620 con 0x5624810d9d90 2024-04-25T10:11:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f4170ad1640 1 -- 172.21.15.60:0/1542119625 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41680177c0 con 0x5624810d9d90 2024-04-25T10:11:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f41757b8080 1 -- 172.21.15.60:0/1542119625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562481176a00 con 0x5624810d9d90 2024-04-25T10:11:26.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f4170ad1640 1 -- 172.21.15.60:0/1542119625 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168017b70 con 0x5624810d9d90 2024-04-25T10:11:26.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f41757b8080 1 -- 172.21.15.60:0/1542119625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562481176f40 con 0x5624810d9d90 2024-04-25T10:11:26.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f4170ad1640 1 -- 172.21.15.60:0/1542119625 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4168021070 con 0x5624810d9d90 2024-04-25T10:11:26.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f4170ad1640 1 --2- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4160065970 0x7f4160067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f4170ad1640 1 -- 172.21.15.60:0/1542119625 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f41680856d0 con 0x5624810d9d90 2024-04-25T10:11:26.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f41757b8080 1 --2- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x562481165f40 0x562481166320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f41732d6640 1 --2- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4160065970 0x7f4160067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f41757b8080 1 -- 172.21.15.60:0/1542119625 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) v9 -- 0x562481166860 con 0x562481165f40 2024-04-25T10:11:26.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f4173ad7640 1 --2- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x562481165f40 0x562481166320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.948+0000 7f41732d6640 1 --2- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4160065970 0x7f4160067e30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f41640ab000 tx=0x7f416400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.952+0000 7f4173ad7640 1 --2- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x562481165f40 0x562481166320 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.952+0000 7f4173ad7640 1 -- 172.21.15.60:0/1542119625 <== osd.4 v2:172.21.15.100:6808/3965541654 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v71'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f416c00f550 con 0x562481165f40 2024-04-25T10:11:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.952+0000 7f41757b8080 1 -- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x562481165f40 msgr2=0x562481166320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.952+0000 7f41757b8080 1 --2- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x562481165f40 0x562481166320 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.952+0000 7f41757b8080 1 -- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4160065970 msgr2=0x7f4160067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.952+0000 7f41757b8080 1 --2- 172.21.15.60:0/1542119625 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4160065970 0x7f4160067e30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f41640ab000 tx=0x7f416400b040 comp rx=0 tx=0).stop 2024-04-25T10:11:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.952+0000 7f41757b8080 1 -- 172.21.15.60:0/1542119625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624810d9d90 msgr2=0x562481175fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.952+0000 7f41757b8080 1 --2- 172.21.15.60:0/1542119625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624810d9d90 0x562481175fe0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f416800bc00 tx=0x7f4168001a50 comp rx=0 tx=0).stop 2024-04-25T10:11:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.952+0000 7f4173ad7640 1 -- 172.21.15.60:0/1542119625 reap_dead start 2024-04-25T10:11:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.952+0000 7f41757b8080 1 -- 172.21.15.60:0/1542119625 shutdown_connections 2024-04-25T10:11:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.952+0000 7f41757b8080 1 -- 172.21.15.60:0/1542119625 >> 172.21.15.60:0/1542119625 conn(0x56248102fe50 msgr2=0x56248104d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.952+0000 7f41757b8080 1 -- 172.21.15.60:0/1542119625 shutdown_connections 2024-04-25T10:11:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:26.952+0000 7f41757b8080 1 -- 172.21.15.60:0/1542119625 wait complete. 2024-04-25T10:11:26.966 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:11:26.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:27.065 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:27.107 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:11:27.108 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:11:27.108 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:27.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:27.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb9101f7640 1 Processor -- start 2024-04-25T10:11:27.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb9101f7640 1 -- start start 2024-04-25T10:11:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb9101f7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb90815a7a0 0x7fb9080cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb9101f7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90806ec30 0x7fb908071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb9101f7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb9080b37e0 0x7fb9080b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb9101f7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9080cdaa0 con 0x7fb9080b37e0 2024-04-25T10:11:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb9101f7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb90806d2f0 con 0x7fb90815a7a0 2024-04-25T10:11:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb9101f7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90806d8b0 con 0x7fb90806ec30 2024-04-25T10:11:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb90e794640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb9080b37e0 0x7fb9080b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb90e794640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb9080b37e0 0x7fb9080b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60930/0 (socket says 172.21.15.60:60930) 2024-04-25T10:11:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb90e794640 1 -- 172.21.15.60:0/1702436228 learned_addr learned my addr 172.21.15.60:0/1702436228 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb90d792640 1 --2- 172.21.15.60:0/1702436228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90806ec30 0x7fb908071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb90df93640 1 --2- 172.21.15.60:0/1702436228 >> v2:172.21.15.100:3300/0 conn(0x7fb90815a7a0 0x7fb9080cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb90e794640 1 -- 172.21.15.60:0/1702436228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90806ec30 msgr2=0x7fb908071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb90e794640 1 --2- 172.21.15.60:0/1702436228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90806ec30 0x7fb908071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb90e794640 1 -- 172.21.15.60:0/1702436228 >> v2:172.21.15.100:3300/0 conn(0x7fb90815a7a0 msgr2=0x7fb9080cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb90e794640 1 --2- 172.21.15.60:0/1702436228 >> v2:172.21.15.100:3300/0 conn(0x7fb90815a7a0 0x7fb9080cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb90e794640 1 -- 172.21.15.60:0/1702436228 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9080721e0 con 0x7fb9080b37e0 2024-04-25T10:11:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.284+0000 7fb90df93640 1 --2- 172.21.15.60:0/1702436228 >> v2:172.21.15.100:3300/0 conn(0x7fb90815a7a0 0x7fb9080cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb90e794640 1 --2- 172.21.15.60:0/1702436228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080b37e0 0x7fb9080b3c90 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb90804f5d0 tx=0x7fb90001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a556b1eb282e529 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb90cf91640 1 -- 172.21.15.60:0/1702436228 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9000a5780 con 0x7fb9080b37e0 2024-04-25T10:11:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb90cf91640 1 -- 172.21.15.60:0/1702436228 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9000a5920 con 0x7fb9080b37e0 2024-04-25T10:11:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb90cf91640 1 -- 172.21.15.60:0/1702436228 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9000a5cf0 con 0x7fb9080b37e0 2024-04-25T10:11:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 -- 172.21.15.60:0/1702436228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080b37e0 msgr2=0x7fb9080b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 --2- 172.21.15.60:0/1702436228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080b37e0 0x7fb9080b3c90 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb90804f5d0 tx=0x7fb90001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 -- 172.21.15.60:0/1702436228 shutdown_connections 2024-04-25T10:11:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 --2- 172.21.15.60:0/1702436228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080b37e0 0x7fb9080b3c90 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 --2- 172.21.15.60:0/1702436228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90806ec30 0x7fb908071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 --2- 172.21.15.60:0/1702436228 >> v2:172.21.15.100:3300/0 conn(0x7fb90815a7a0 0x7fb9080cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 -- 172.21.15.60:0/1702436228 >> 172.21.15.60:0/1702436228 conn(0x7fb9080b12d0 msgr2=0x7fb9080b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 -- 172.21.15.60:0/1702436228 shutdown_connections 2024-04-25T10:11:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 -- 172.21.15.60:0/1702436228 wait complete. 2024-04-25T10:11:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 Processor -- start 2024-04-25T10:11:27.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 -- start start 2024-04-25T10:11:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90806ec30 0x7fb9080c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080b37e0 0x7fb9080c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb90815a7a0 0x7fb908143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90815ad00 con 0x7fb9080b37e0 2024-04-25T10:11:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90815ab80 con 0x7fb90815a7a0 2024-04-25T10:11:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb9101f7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90806e530 con 0x7fb90806ec30 2024-04-25T10:11:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb90d792640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080b37e0 0x7fb9080c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb90d792640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080b37e0 0x7fb9080c4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60934/0 (socket says 172.21.15.60:60934) 2024-04-25T10:11:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb90d792640 1 -- 172.21.15.60:0/516163199 learned_addr learned my addr 172.21.15.60:0/516163199 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb90e794640 1 --2- 172.21.15.60:0/516163199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb90815a7a0 0x7fb908143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.288+0000 7fb90df93640 1 --2- 172.21.15.60:0/516163199 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90806ec30 0x7fb9080c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.292+0000 7fb90d792640 1 -- 172.21.15.60:0/516163199 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90806ec30 msgr2=0x7fb9080c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.292+0000 7fb90d792640 1 --2- 172.21.15.60:0/516163199 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90806ec30 0x7fb9080c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.292+0000 7fb90d792640 1 -- 172.21.15.60:0/516163199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb90815a7a0 msgr2=0x7fb908143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.292+0000 7fb90d792640 1 --2- 172.21.15.60:0/516163199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb90815a7a0 0x7fb908143ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.292+0000 7fb90d792640 1 -- 172.21.15.60:0/516163199 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb908153c10 con 0x7fb9080b37e0 2024-04-25T10:11:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.292+0000 7fb90d792640 1 --2- 172.21.15.60:0/516163199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080b37e0 0x7fb9080c4f50 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800d800 tx=0x7fb8f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.292+0000 7fb8fe7fc640 1 -- 172.21.15.60:0/516163199 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800b6c0 con 0x7fb9080b37e0 2024-04-25T10:11:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.292+0000 7fb8fe7fc640 1 -- 172.21.15.60:0/516163199 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f8019070 con 0x7fb9080b37e0 2024-04-25T10:11:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.292+0000 7fb9101f7640 1 -- 172.21.15.60:0/516163199 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9080c55c0 con 0x7fb9080b37e0 2024-04-25T10:11:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.292+0000 7fb8fe7fc640 1 -- 172.21.15.60:0/516163199 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800ba60 con 0x7fb9080b37e0 2024-04-25T10:11:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.292+0000 7fb9101f7640 1 -- 172.21.15.60:0/516163199 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb908158bf0 con 0x7fb9080b37e0 2024-04-25T10:11:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.292+0000 7fb9101f7640 1 -- 172.21.15.60:0/516163199 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb908074770 con 0x7fb9080b37e0 2024-04-25T10:11:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.292+0000 7fb8fe7fc640 1 -- 172.21.15.60:0/516163199 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8f800bc00 con 0x7fb9080b37e0 2024-04-25T10:11:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.292+0000 7fb8fe7fc640 1 --2- 172.21.15.60:0/516163199 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8ec066f60 0x7fb8ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.292+0000 7fb8fe7fc640 1 -- 172.21.15.60:0/516163199 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb8f8085510 con 0x7fb9080b37e0 2024-04-25T10:11:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.292+0000 7fb90df93640 1 --2- 172.21.15.60:0/516163199 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8ec066f60 0x7fb8ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.296+0000 7fb90df93640 1 --2- 172.21.15.60:0/516163199 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8ec066f60 0x7fb8ec069420 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb904006fd0 tx=0x7fb904008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.296+0000 7fb8fe7fc640 1 -- 172.21.15.60:0/516163199 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb908153c10 con 0x7fb9080b37e0 2024-04-25T10:11:27.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.464+0000 7fb9101f7640 1 -- 172.21.15.60:0/516163199 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb90815eec0 con 0x7fb9080b37e0 2024-04-25T10:11:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.464+0000 7fb8fe7fc640 1 -- 172.21.15.60:0/516163199 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16907 (secure 0 0 0) 0x7fb8f8056480 con 0x7fb9080b37e0 2024-04-25T10:11:27.474 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:27.474 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:10:03.449677+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:42.005281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:42.710028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1e","mappings":[{"from":6,"to":0}]},{"pgid":"11.28","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/1289503429":"2024-04-26T10:10:03.449620+0000","172.21.15.60:6833/1289503429":"2024-04-26T10:10:03.449620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.468+0000 7fb9101f7640 1 -- 172.21.15.60:0/516163199 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8ec066f60 msgr2=0x7fb8ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.468+0000 7fb9101f7640 1 --2- 172.21.15.60:0/516163199 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8ec066f60 0x7fb8ec069420 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb904006fd0 tx=0x7fb904008040 comp rx=0 tx=0).stop 2024-04-25T10:11:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.468+0000 7fb9101f7640 1 -- 172.21.15.60:0/516163199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080b37e0 msgr2=0x7fb9080c4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.468+0000 7fb9101f7640 1 --2- 172.21.15.60:0/516163199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080b37e0 0x7fb9080c4f50 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800d800 tx=0x7fb8f800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.468+0000 7fb9101f7640 1 -- 172.21.15.60:0/516163199 shutdown_connections 2024-04-25T10:11:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.468+0000 7fb9101f7640 1 --2- 172.21.15.60:0/516163199 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8ec066f60 0x7fb8ec069420 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.468+0000 7fb9101f7640 1 --2- 172.21.15.60:0/516163199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb90815a7a0 0x7fb908143ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.468+0000 7fb9101f7640 1 --2- 172.21.15.60:0/516163199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080b37e0 0x7fb9080c4f50 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.468+0000 7fb9101f7640 1 --2- 172.21.15.60:0/516163199 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90806ec30 0x7fb9080c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.468+0000 7fb9101f7640 1 -- 172.21.15.60:0/516163199 >> 172.21.15.60:0/516163199 conn(0x7fb9080b12d0 msgr2=0x7fb9081638a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.468+0000 7fb9101f7640 1 -- 172.21.15.60:0/516163199 shutdown_connections 2024-04-25T10:11:27.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.468+0000 7fb9101f7640 1 -- 172.21.15.60:0/516163199 wait complete. 2024-04-25T10:11:27.494 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:09:42.005281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:09:42.710028+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:27.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:27.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.616+0000 7efda06a3640 1 Processor -- start 2024-04-25T10:11:27.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.620+0000 7efda06a3640 1 -- start start 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.620+0000 7efda06a3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efd9815db10 0x7efd980b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.620+0000 7efda06a3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efd980cfcf0 0x7efd980b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.620+0000 7efda06a3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd980ae140 con 0x7efd980aec10 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.620+0000 7efda06a3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd980accc0 con 0x7efd980cfcf0 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.620+0000 7efda06a3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9815def0 con 0x7efd9815db10 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.620+0000 7efd9ec40640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efd980cfcf0 0x7efd980b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.620+0000 7efd9dc3e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efd980aec10 0x7efd98161660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53100/0 (socket says 172.21.15.60:53100) 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.620+0000 7efd9dc3e640 1 -- 172.21.15.60:0/956901857 learned_addr learned my addr 172.21.15.60:0/956901857 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.620+0000 7efd9e43f640 1 --2- 172.21.15.60:0/956901857 >> v2:172.21.15.100:3300/0 conn(0x7efd9815db10 0x7efd980b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.620+0000 7efd9ec40640 1 -- 172.21.15.60:0/956901857 >> v2:172.21.15.100:3300/0 conn(0x7efd9815db10 msgr2=0x7efd980b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.620+0000 7efd9ec40640 1 --2- 172.21.15.60:0/956901857 >> v2:172.21.15.100:3300/0 conn(0x7efd9815db10 0x7efd980b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.620+0000 7efd9ec40640 1 -- 172.21.15.60:0/956901857 >> v1:172.21.15.60:6789/0 conn(0x7efd980aec10 legacy=0x7efd98161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.620+0000 7efd9d43d640 1 -- 172.21.15.60:0/956901857 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369396504 0 0) 0x7efd88001120 con 0x7efd980aec10 2024-04-25T10:11:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.620+0000 7efd9ec40640 1 -- 172.21.15.60:0/956901857 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd98071630 con 0x7efd980cfcf0 2024-04-25T10:11:27.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efd9ec40640 1 --2- 172.21.15.60:0/956901857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd980cfcf0 0x7efd980b3420 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7efd9806f5d0 tx=0x7efd9001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1d497d3112d752 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efd9d43d640 1 -- 172.21.15.60:0/956901857 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd900b2070 con 0x7efd980cfcf0 2024-04-25T10:11:27.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efd9d43d640 1 -- 172.21.15.60:0/956901857 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd900a5910 con 0x7efd980cfcf0 2024-04-25T10:11:27.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efd9d43d640 1 -- 172.21.15.60:0/956901857 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd900a5ce0 con 0x7efd980cfcf0 2024-04-25T10:11:27.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efda06a3640 1 -- 172.21.15.60:0/956901857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd980cfcf0 msgr2=0x7efd980b3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efda06a3640 1 --2- 172.21.15.60:0/956901857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd980cfcf0 0x7efd980b3420 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7efd9806f5d0 tx=0x7efd9001c930 comp rx=0 tx=0).stop 2024-04-25T10:11:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efda06a3640 1 -- 172.21.15.60:0/956901857 shutdown_connections 2024-04-25T10:11:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efda06a3640 1 --2- 172.21.15.60:0/956901857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd980cfcf0 0x7efd980b3420 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efda06a3640 1 --2- 172.21.15.60:0/956901857 >> v2:172.21.15.100:3300/0 conn(0x7efd9815db10 0x7efd980b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efda06a3640 1 -- 172.21.15.60:0/956901857 >> 172.21.15.60:0/956901857 conn(0x7efd980670b0 msgr2=0x7efd9806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efda06a3640 1 -- 172.21.15.60:0/956901857 shutdown_connections 2024-04-25T10:11:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efda06a3640 1 -- 172.21.15.60:0/956901857 wait complete. 2024-04-25T10:11:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efda06a3640 1 Processor -- start 2024-04-25T10:11:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efda06a3640 1 -- start start 2024-04-25T10:11:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efda06a3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd980aec10 0x7efd980ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efda06a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd980cfcf0 0x7efd98150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efda06a3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd9815db10 0x7efd9814e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efda06a3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9806dcf0 con 0x7efd980cfcf0 2024-04-25T10:11:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efda06a3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9806db70 con 0x7efd9815db10 2024-04-25T10:11:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efda06a3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd98071250 con 0x7efd980aec10 2024-04-25T10:11:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efd9dc3e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd980cfcf0 0x7efd98150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efd9dc3e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd980cfcf0 0x7efd98150040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60954/0 (socket says 172.21.15.60:60954) 2024-04-25T10:11:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efd9dc3e640 1 -- 172.21.15.60:0/883598022 learned_addr learned my addr 172.21.15.60:0/883598022 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efd9ec40640 1 --2- 172.21.15.60:0/883598022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd9815db10 0x7efd9814e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efd9dc3e640 1 -- 172.21.15.60:0/883598022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd980aec10 msgr2=0x7efd980ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efd9e43f640 1 --2- 172.21.15.60:0/883598022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd980aec10 0x7efd980ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efd9dc3e640 1 --2- 172.21.15.60:0/883598022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd980aec10 0x7efd980ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efd9dc3e640 1 -- 172.21.15.60:0/883598022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd9815db10 msgr2=0x7efd9814e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efd9dc3e640 1 --2- 172.21.15.60:0/883598022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd9815db10 0x7efd9814e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efd9dc3e640 1 -- 172.21.15.60:0/883598022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9814ecc0 con 0x7efd980cfcf0 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efd9ec40640 1 --2- 172.21.15.60:0/883598022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd9815db10 0x7efd9814e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.624+0000 7efd9dc3e640 1 --2- 172.21.15.60:0/883598022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd980cfcf0 0x7efd98150040 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7efd8800c4b0 tx=0x7efd8800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.628+0000 7efd8effd640 1 -- 172.21.15.60:0/883598022 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88018070 con 0x7efd980cfcf0 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.628+0000 7efd8effd640 1 -- 172.21.15.60:0/883598022 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd88007500 con 0x7efd980cfcf0 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.628+0000 7efda06a3640 1 -- 172.21.15.60:0/883598022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9814efa0 con 0x7efd980cfcf0 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.628+0000 7efd8effd640 1 -- 172.21.15.60:0/883598022 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88004a30 con 0x7efd980cfcf0 2024-04-25T10:11:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.628+0000 7efda06a3640 1 -- 172.21.15.60:0/883598022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9814f560 con 0x7efd980cfcf0 2024-04-25T10:11:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.628+0000 7efda06a3640 1 -- 172.21.15.60:0/883598022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9806bba0 con 0x7efd980cfcf0 2024-04-25T10:11:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.628+0000 7efd8effd640 1 -- 172.21.15.60:0/883598022 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd8801b070 con 0x7efd980cfcf0 2024-04-25T10:11:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.628+0000 7efd8effd640 1 --2- 172.21.15.60:0/883598022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efd7c066f60 0x7efd7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.628+0000 7efd8effd640 1 -- 172.21.15.60:0/883598022 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7efd880843b0 con 0x7efd980cfcf0 2024-04-25T10:11:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.628+0000 7efd9e43f640 1 --2- 172.21.15.60:0/883598022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efd7c066f60 0x7efd7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.628+0000 7efd9e43f640 1 --2- 172.21.15.60:0/883598022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efd7c066f60 0x7efd7c069420 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efd98152c50 tx=0x7efd94008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.632+0000 7efd8effd640 1 -- 172.21.15.60:0/883598022 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efd980ae140 con 0x7efd980cfcf0 2024-04-25T10:11:27.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.796+0000 7efda06a3640 1 -- 172.21.15.60:0/883598022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd98074770 con 0x7efd980cfcf0 2024-04-25T10:11:27.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.796+0000 7efd8effd640 1 -- 172.21.15.60:0/883598022 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd88055330 con 0x7efd980cfcf0 2024-04-25T10:11:27.805 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:11:27.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.796+0000 7efda06a3640 1 -- 172.21.15.60:0/883598022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efd7c066f60 msgr2=0x7efd7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.796+0000 7efda06a3640 1 --2- 172.21.15.60:0/883598022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efd7c066f60 0x7efd7c069420 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efd98152c50 tx=0x7efd94008040 comp rx=0 tx=0).stop 2024-04-25T10:11:27.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.796+0000 7efda06a3640 1 -- 172.21.15.60:0/883598022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd980cfcf0 msgr2=0x7efd98150040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.796+0000 7efda06a3640 1 --2- 172.21.15.60:0/883598022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd980cfcf0 0x7efd98150040 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7efd8800c4b0 tx=0x7efd8800c970 comp rx=0 tx=0).stop 2024-04-25T10:11:27.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.800+0000 7efda06a3640 1 -- 172.21.15.60:0/883598022 shutdown_connections 2024-04-25T10:11:27.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.800+0000 7efda06a3640 1 --2- 172.21.15.60:0/883598022 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efd7c066f60 0x7efd7c069420 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.800+0000 7efda06a3640 1 --2- 172.21.15.60:0/883598022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd9815db10 0x7efd9814e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.800+0000 7efda06a3640 1 --2- 172.21.15.60:0/883598022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd980cfcf0 0x7efd98150040 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.800+0000 7efda06a3640 1 --2- 172.21.15.60:0/883598022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd980aec10 0x7efd980ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.800+0000 7efda06a3640 1 -- 172.21.15.60:0/883598022 >> 172.21.15.60:0/883598022 conn(0x7efd980670b0 msgr2=0x7efd9806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:27.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.800+0000 7efda06a3640 1 -- 172.21.15.60:0/883598022 shutdown_connections 2024-04-25T10:11:27.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.800+0000 7efda06a3640 1 -- 172.21.15.60:0/883598022 wait complete. 2024-04-25T10:11:27.824 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:27.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9dca63640 1 Processor -- start 2024-04-25T10:11:27.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9dca63640 1 -- start start 2024-04-25T10:11:27.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9dca63640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb9d80cbcf0 0x7fb9d8160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9dca63640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d8144a40 con 0x7fb9d815fbf0 2024-04-25T10:11:27.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9dca63640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d80cc9f0 con 0x7fb9d8071c30 2024-04-25T10:11:27.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9dca63640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d80b2dc0 con 0x7fb9d80cbcf0 2024-04-25T10:11:27.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d6d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb9d815fbf0 0x7fb9d80ceec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53114/0 (socket says 172.21.15.60:53114) 2024-04-25T10:11:27.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d6d9d640 1 -- 172.21.15.60:0/1525704857 learned_addr learned my addr 172.21.15.60:0/1525704857 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:27.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d659c640 1 --2- 172.21.15.60:0/1525704857 >> v2:172.21.15.100:3300/0 conn(0x7fb9d80cbcf0 0x7fb9d8160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d559a640 1 -- 172.21.15.60:0/1525704857 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454595694 0 0) 0x7fb9d8144a40 con 0x7fb9d815fbf0 2024-04-25T10:11:27.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d559a640 1 -- 172.21.15.60:0/1525704857 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9c0003530 con 0x7fb9d815fbf0 2024-04-25T10:11:27.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d559a640 1 -- 172.21.15.60:0/1525704857 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2148033131 0 0) 0x7fb9c0003530 con 0x7fb9d815fbf0 2024-04-25T10:11:27.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d559a640 1 -- 172.21.15.60:0/1525704857 >> v2:172.21.15.100:3300/0 conn(0x7fb9d80cbcf0 msgr2=0x7fb9d8160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:27.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d559a640 1 --2- 172.21.15.60:0/1525704857 >> v2:172.21.15.100:3300/0 conn(0x7fb9d80cbcf0 0x7fb9d8160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d559a640 1 -- 172.21.15.60:0/1525704857 >> v1:172.21.15.100:6789/0 conn(0x7fb9d8071c30 legacy=0x7fb9d815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d559a640 1 -- 172.21.15.60:0/1525704857 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d80cccd0 con 0x7fb9d815fbf0 2024-04-25T10:11:27.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d559a640 1 -- 172.21.15.60:0/1525704857 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fb9d0001a00 con 0x7fb9d815fbf0 2024-04-25T10:11:27.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d559a640 1 -- 172.21.15.60:0/1525704857 >> v1:172.21.15.60:6789/0 conn(0x7fb9d815fbf0 legacy=0x7fb9d80ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d559a640 1 --2- 172.21.15.60:0/1525704857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c00039c0 0x7fb9c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d559a640 1 --2- 172.21.15.60:0/1525704857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9c0004810 0x7fb9c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d559a640 1 --2- 172.21.15.60:0/1525704857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9c00051d0 0x7fb9c000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d559a640 1 -- 172.21.15.60:0/1525704857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c000c040 con 0x7fb9c00051d0 2024-04-25T10:11:27.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d559a640 1 -- 172.21.15.60:0/1525704857 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c000c1f0 con 0x7fb9c00039c0 2024-04-25T10:11:27.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d559a640 1 -- 172.21.15.60:0/1525704857 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c000c3f0 con 0x7fb9c0004810 2024-04-25T10:11:27.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d6d9d640 1 --2- 172.21.15.60:0/1525704857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9c00051d0 0x7fb9c000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d659c640 1 --2- 172.21.15.60:0/1525704857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c00039c0 0x7fb9c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d5d9b640 1 --2- 172.21.15.60:0/1525704857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9c0004810 0x7fb9c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d6d9d640 1 -- 172.21.15.60:0/1525704857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9c0004810 msgr2=0x7fb9c0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:27.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d6d9d640 1 --2- 172.21.15.60:0/1525704857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9c0004810 0x7fb9c0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d6d9d640 1 -- 172.21.15.60:0/1525704857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c00039c0 msgr2=0x7fb9c0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:27.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d6d9d640 1 --2- 172.21.15.60:0/1525704857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c00039c0 0x7fb9c0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.948+0000 7fb9d6d9d640 1 -- 172.21.15.60:0/1525704857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c0003530 con 0x7fb9c00051d0 2024-04-25T10:11:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.952+0000 7fb9d6d9d640 1 -- 172.21.15.60:0/1525704857 reap_dead start 2024-04-25T10:11:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.952+0000 7fb9d6d9d640 1 --2- 172.21.15.60:0/1525704857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9c00051d0 0x7fb9c000b9d0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000a840 tx=0x7fb9d001d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d366520a77a32b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.952+0000 7fb9d559a640 1 -- 172.21.15.60:0/1525704857 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d000a460 con 0x7fb9c00051d0 2024-04-25T10:11:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.952+0000 7fb9d559a640 1 -- 172.21.15.60:0/1525704857 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d00ac640 con 0x7fb9c00051d0 2024-04-25T10:11:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.952+0000 7fb9d559a640 1 -- 172.21.15.60:0/1525704857 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00aca30 con 0x7fb9c00051d0 2024-04-25T10:11:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.952+0000 7fb9dca63640 1 -- 172.21.15.60:0/1525704857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9c00051d0 msgr2=0x7fb9c000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.952+0000 7fb9dca63640 1 --2- 172.21.15.60:0/1525704857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9c00051d0 0x7fb9c000b9d0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000a840 tx=0x7fb9d001d360 comp rx=0 tx=0).stop 2024-04-25T10:11:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.952+0000 7fb9dca63640 1 -- 172.21.15.60:0/1525704857 shutdown_connections 2024-04-25T10:11:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.952+0000 7fb9dca63640 1 --2- 172.21.15.60:0/1525704857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9c00051d0 0x7fb9c000b9d0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.952+0000 7fb9dca63640 1 -- 172.21.15.60:0/1525704857 >> 172.21.15.60:0/1525704857 conn(0x7fb9d80670b0 msgr2=0x7fb9d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.952+0000 7fb9dca63640 1 -- 172.21.15.60:0/1525704857 shutdown_connections 2024-04-25T10:11:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.952+0000 7fb9dca63640 1 -- 172.21.15.60:0/1525704857 wait complete. 2024-04-25T10:11:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.952+0000 7fb9dca63640 1 Processor -- start 2024-04-25T10:11:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.952+0000 7fb9dca63640 1 -- start start 2024-04-25T10:11:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9dca63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c00051d0 0x7fb9d8139150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9dca63640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9d813e350 0x7fb9d814d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9dca63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d813f890 0x7fb9d813fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9dca63640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80cc0d0 con 0x7fb9d813f890 2024-04-25T10:11:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9dca63640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80ce9a0 con 0x7fb9c00051d0 2024-04-25T10:11:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9dca63640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80720b0 con 0x7fb9d813e350 2024-04-25T10:11:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9d6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d813f890 0x7fb9d813fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9d6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d813f890 0x7fb9d813fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60970/0 (socket says 172.21.15.60:60970) 2024-04-25T10:11:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9d6d9d640 1 -- 172.21.15.60:0/1152772340 learned_addr learned my addr 172.21.15.60:0/1152772340 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9d5d9b640 1 --2- 172.21.15.60:0/1152772340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9d813e350 0x7fb9d814d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9d659c640 1 --2- 172.21.15.60:0/1152772340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c00051d0 0x7fb9d8139150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9d6d9d640 1 -- 172.21.15.60:0/1152772340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9d813e350 msgr2=0x7fb9d814d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9d6d9d640 1 --2- 172.21.15.60:0/1152772340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9d813e350 0x7fb9d814d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9d6d9d640 1 -- 172.21.15.60:0/1152772340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c00051d0 msgr2=0x7fb9d8139150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9d6d9d640 1 --2- 172.21.15.60:0/1152772340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c00051d0 0x7fb9d8139150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9d6d9d640 1 -- 172.21.15.60:0/1152772340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d814dcc0 con 0x7fb9d813f890 2024-04-25T10:11:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9d6d9d640 1 --2- 172.21.15.60:0/1152772340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d813f890 0x7fb9d813fd40 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fb9d001dc00 tx=0x7fb9c0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9c6ffd640 1 -- 172.21.15.60:0/1152772340 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00019c0 con 0x7fb9d813f890 2024-04-25T10:11:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9c6ffd640 1 -- 172.21.15.60:0/1152772340 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d0001260 con 0x7fb9d813f890 2024-04-25T10:11:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9dca63640 1 -- 172.21.15.60:0/1152772340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d813e7f0 con 0x7fb9d813f890 2024-04-25T10:11:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9c6ffd640 1 -- 172.21.15.60:0/1152772340 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0004d70 con 0x7fb9d813f890 2024-04-25T10:11:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9dca63640 1 -- 172.21.15.60:0/1152772340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d813edb0 con 0x7fb9d813f890 2024-04-25T10:11:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9dca63640 1 -- 172.21.15.60:0/1152772340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d806bba0 con 0x7fb9d813f890 2024-04-25T10:11:27.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9c6ffd640 1 -- 172.21.15.60:0/1152772340 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9d00070e0 con 0x7fb9d813f890 2024-04-25T10:11:27.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9c6ffd640 1 --2- 172.21.15.60:0/1152772340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9b8066f60 0x7fb9b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9c6ffd640 1 -- 172.21.15.60:0/1152772340 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb9d011e1a0 con 0x7fb9d813f890 2024-04-25T10:11:27.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.956+0000 7fb9d659c640 1 --2- 172.21.15.60:0/1152772340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9b8066f60 0x7fb9b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.960+0000 7fb9d659c640 1 --2- 172.21.15.60:0/1152772340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9b8066f60 0x7fb9b8069420 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009910 tx=0x7fb9c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:27.960+0000 7fb9c6ffd640 1 -- 172.21.15.60:0/1152772340 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb9d0017070 con 0x7fb9d813f890 2024-04-25T10:11:28.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.120+0000 7fb9dca63640 1 -- 172.21.15.60:0/1152772340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb9d8074770 con 0x7fb9d813f890 2024-04-25T10:11:28.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.120+0000 7fb9c6ffd640 1 -- 172.21.15.60:0/1152772340 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb9d00ef080 con 0x7fb9d813f890 2024-04-25T10:11:28.129 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:11:28.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.124+0000 7fb9dca63640 1 -- 172.21.15.60:0/1152772340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9b8066f60 msgr2=0x7fb9b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.124+0000 7fb9dca63640 1 --2- 172.21.15.60:0/1152772340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9b8066f60 0x7fb9b8069420 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009910 tx=0x7fb9c8008040 comp rx=0 tx=0).stop 2024-04-25T10:11:28.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.124+0000 7fb9dca63640 1 -- 172.21.15.60:0/1152772340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d813f890 msgr2=0x7fb9d813fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.124+0000 7fb9dca63640 1 --2- 172.21.15.60:0/1152772340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d813f890 0x7fb9d813fd40 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fb9d001dc00 tx=0x7fb9c0005840 comp rx=0 tx=0).stop 2024-04-25T10:11:28.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.124+0000 7fb9dca63640 1 -- 172.21.15.60:0/1152772340 shutdown_connections 2024-04-25T10:11:28.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.124+0000 7fb9dca63640 1 --2- 172.21.15.60:0/1152772340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9b8066f60 0x7fb9b8069420 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.124+0000 7fb9dca63640 1 --2- 172.21.15.60:0/1152772340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d813f890 0x7fb9d813fd40 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.124+0000 7fb9dca63640 1 --2- 172.21.15.60:0/1152772340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9d813e350 0x7fb9d814d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.124+0000 7fb9dca63640 1 --2- 172.21.15.60:0/1152772340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9c00051d0 0x7fb9d8139150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.124+0000 7fb9dca63640 1 -- 172.21.15.60:0/1152772340 >> 172.21.15.60:0/1152772340 conn(0x7fb9d80670b0 msgr2=0x7fb9d80cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:28.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.124+0000 7fb9dca63640 1 -- 172.21.15.60:0/1152772340 shutdown_connections 2024-04-25T10:11:28.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.124+0000 7fb9dca63640 1 -- 172.21.15.60:0/1152772340 wait complete. 2024-04-25T10:11:28.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:28.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a296ea640 1 Processor -- start 2024-04-25T10:11:28.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a296ea640 1 -- start start 2024-04-25T10:11:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a296ea640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a2406c330 0x7f9a2415ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a296ea640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a240cfb90 con 0x7f9a240b3660 2024-04-25T10:11:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a296ea640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a24072690 con 0x7f9a2415b810 2024-04-25T10:11:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a296ea640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a24072950 con 0x7f9a2406c330 2024-04-25T10:11:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a237fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9a240b3660 0x7f9a2406dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53116/0 (socket says 172.21.15.60:53116) 2024-04-25T10:11:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a237fe640 1 -- 172.21.15.60:0/1481384050 learned_addr learned my addr 172.21.15.60:0/1481384050 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a227fc640 1 --2- 172.21.15.60:0/1481384050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a2406c330 0x7f9a2415ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a21ffb640 1 -- 172.21.15.60:0/1481384050 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893018806 0 0) 0x7f9a240cfb90 con 0x7f9a240b3660 2024-04-25T10:11:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a21ffb640 1 -- 172.21.15.60:0/1481384050 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a0c003530 con 0x7f9a240b3660 2024-04-25T10:11:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a21ffb640 1 -- 172.21.15.60:0/1481384050 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1933623644 0 0) 0x7f9a0c003530 con 0x7f9a240b3660 2024-04-25T10:11:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a21ffb640 1 -- 172.21.15.60:0/1481384050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a2406c330 msgr2=0x7f9a2415ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a21ffb640 1 --2- 172.21.15.60:0/1481384050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a2406c330 0x7f9a2415ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a21ffb640 1 -- 172.21.15.60:0/1481384050 >> v1:172.21.15.100:6789/0 conn(0x7f9a2415b810 legacy=0x7f9a2406bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a21ffb640 1 -- 172.21.15.60:0/1481384050 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a24072c30 con 0x7f9a240b3660 2024-04-25T10:11:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a21ffb640 1 -- 172.21.15.60:0/1481384050 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f9a1c001680 con 0x7f9a240b3660 2024-04-25T10:11:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a21ffb640 1 -- 172.21.15.60:0/1481384050 >> v1:172.21.15.60:6789/0 conn(0x7f9a240b3660 legacy=0x7f9a2406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a21ffb640 1 --2- 172.21.15.60:0/1481384050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c0039f0 0x7f9a0c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a21ffb640 1 --2- 172.21.15.60:0/1481384050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a2406c330 0x7f9a0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a21ffb640 1 --2- 172.21.15.60:0/1481384050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a0c004d50 0x7f9a0c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a21ffb640 1 -- 172.21.15.60:0/1481384050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a24072950 con 0x7f9a0c004d50 2024-04-25T10:11:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a21ffb640 1 -- 172.21.15.60:0/1481384050 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c00bb00 con 0x7f9a0c0039f0 2024-04-25T10:11:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a21ffb640 1 -- 172.21.15.60:0/1481384050 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c00bd00 con 0x7f9a2406c330 2024-04-25T10:11:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a237fe640 1 --2- 172.21.15.60:0/1481384050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a0c004d50 0x7f9a0c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a22ffd640 1 --2- 172.21.15.60:0/1481384050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c0039f0 0x7f9a0c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a227fc640 1 --2- 172.21.15.60:0/1481384050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a2406c330 0x7f9a0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a237fe640 1 -- 172.21.15.60:0/1481384050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a2406c330 msgr2=0x7f9a0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:28.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a237fe640 1 --2- 172.21.15.60:0/1481384050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a2406c330 0x7f9a0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a237fe640 1 -- 172.21.15.60:0/1481384050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c0039f0 msgr2=0x7f9a0c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:28.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a237fe640 1 --2- 172.21.15.60:0/1481384050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c0039f0 0x7f9a0c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.268+0000 7f9a237fe640 1 -- 172.21.15.60:0/1481384050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c003530 con 0x7f9a0c004d50 2024-04-25T10:11:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.272+0000 7f9a237fe640 1 --2- 172.21.15.60:0/1481384050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a0c004d50 0x7f9a0c0051d0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00c210 tx=0x7f9a1c01f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903bb90e40f8492f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.272+0000 7f9a21ffb640 1 -- 172.21.15.60:0/1481384050 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c007d40 con 0x7f9a0c004d50 2024-04-25T10:11:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.272+0000 7f9a21ffb640 1 -- 172.21.15.60:0/1481384050 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1c005b00 con 0x7f9a0c004d50 2024-04-25T10:11:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.272+0000 7f9a21ffb640 1 -- 172.21.15.60:0/1481384050 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c007070 con 0x7f9a0c004d50 2024-04-25T10:11:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.272+0000 7f9a296ea640 1 -- 172.21.15.60:0/1481384050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a0c004d50 msgr2=0x7f9a0c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.272+0000 7f9a296ea640 1 --2- 172.21.15.60:0/1481384050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a0c004d50 0x7f9a0c0051d0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00c210 tx=0x7f9a1c01f3f0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.272+0000 7f9a237fe640 1 -- 172.21.15.60:0/1481384050 reap_dead start 2024-04-25T10:11:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.272+0000 7f9a296ea640 1 -- 172.21.15.60:0/1481384050 shutdown_connections 2024-04-25T10:11:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.272+0000 7f9a296ea640 1 -- 172.21.15.60:0/1481384050 >> 172.21.15.60:0/1481384050 conn(0x7f9a24071b70 msgr2=0x7f9a24071f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.272+0000 7f9a296ea640 1 -- 172.21.15.60:0/1481384050 shutdown_connections 2024-04-25T10:11:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.272+0000 7f9a296ea640 1 -- 172.21.15.60:0/1481384050 wait complete. 2024-04-25T10:11:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.272+0000 7f9a296ea640 1 Processor -- start 2024-04-25T10:11:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.272+0000 7f9a296ea640 1 -- start start 2024-04-25T10:11:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a296ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c004d50 0x7f9a24146980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a296ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a241e76f0 0x7f9a241e8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a296ea640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a24159480 0x7f9a24159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a296ea640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2415ebd0 con 0x7f9a241e76f0 2024-04-25T10:11:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a296ea640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2415ea50 con 0x7f9a0c004d50 2024-04-25T10:11:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a296ea640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2415ed50 con 0x7f9a24159480 2024-04-25T10:11:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a227fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a241e76f0 0x7f9a241e8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a227fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a241e76f0 0x7f9a241e8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60980/0 (socket says 172.21.15.60:60980) 2024-04-25T10:11:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a227fc640 1 -- 172.21.15.60:0/3900600624 learned_addr learned my addr 172.21.15.60:0/3900600624 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a237fe640 1 --2- 172.21.15.60:0/3900600624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a24159480 0x7f9a24159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a22ffd640 1 --2- 172.21.15.60:0/3900600624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c004d50 0x7f9a24146980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a227fc640 1 -- 172.21.15.60:0/3900600624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a24159480 msgr2=0x7f9a24159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a227fc640 1 --2- 172.21.15.60:0/3900600624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a24159480 0x7f9a24159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a227fc640 1 -- 172.21.15.60:0/3900600624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c004d50 msgr2=0x7f9a24146980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a227fc640 1 --2- 172.21.15.60:0/3900600624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c004d50 0x7f9a24146980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a227fc640 1 -- 172.21.15.60:0/3900600624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a241579a0 con 0x7f9a241e76f0 2024-04-25T10:11:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a237fe640 1 --2- 172.21.15.60:0/3900600624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a24159480 0x7f9a24159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a22ffd640 1 --2- 172.21.15.60:0/3900600624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c004d50 0x7f9a24146980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a227fc640 1 --2- 172.21.15.60:0/3900600624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a241e76f0 0x7f9a241e8280 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f9a18011610 tx=0x7f9a18011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a0b7fe640 1 -- 172.21.15.60:0/3900600624 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1800f8f0 con 0x7f9a241e76f0 2024-04-25T10:11:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a0b7fe640 1 -- 172.21.15.60:0/3900600624 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1800fa90 con 0x7f9a241e76f0 2024-04-25T10:11:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a296ea640 1 -- 172.21.15.60:0/3900600624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a241e6650 con 0x7f9a241e76f0 2024-04-25T10:11:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a0b7fe640 1 -- 172.21.15.60:0/3900600624 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1800fe40 con 0x7f9a241e76f0 2024-04-25T10:11:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a296ea640 1 -- 172.21.15.60:0/3900600624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a241e6c10 con 0x7f9a241e76f0 2024-04-25T10:11:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a296ea640 1 -- 172.21.15.60:0/3900600624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a24074770 con 0x7f9a241e76f0 2024-04-25T10:11:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a0b7fe640 1 -- 172.21.15.60:0/3900600624 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a18015460 con 0x7f9a241e76f0 2024-04-25T10:11:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a0b7fe640 1 --2- 172.21.15.60:0/3900600624 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a10066f60 0x7f9a10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a0b7fe640 1 -- 172.21.15.60:0/3900600624 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9a18085a80 con 0x7f9a241e76f0 2024-04-25T10:11:28.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.276+0000 7f9a22ffd640 1 --2- 172.21.15.60:0/3900600624 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a10066f60 0x7f9a10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.280+0000 7f9a22ffd640 1 --2- 172.21.15.60:0/3900600624 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a10066f60 0x7f9a10069420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9a14001dd0 tx=0x7f9a14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.280+0000 7f9a0b7fe640 1 -- 172.21.15.60:0/3900600624 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a241579a0 con 0x7f9a241e76f0 2024-04-25T10:11:28.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.444+0000 7f9a296ea640 1 -- 172.21.15.60:0/3900600624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9a24066b50 con 0x7f9a241e76f0 2024-04-25T10:11:28.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.448+0000 7f9a0b7fe640 1 -- 172.21.15.60:0/3900600624 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9a18019030 con 0x7f9a241e76f0 2024-04-25T10:11:28.455 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:11:28.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.448+0000 7f9a296ea640 1 -- 172.21.15.60:0/3900600624 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a10066f60 msgr2=0x7f9a10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.448+0000 7f9a296ea640 1 --2- 172.21.15.60:0/3900600624 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a10066f60 0x7f9a10069420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9a14001dd0 tx=0x7f9a14009040 comp rx=0 tx=0).stop 2024-04-25T10:11:28.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.448+0000 7f9a296ea640 1 -- 172.21.15.60:0/3900600624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a241e76f0 msgr2=0x7f9a241e8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.448+0000 7f9a296ea640 1 --2- 172.21.15.60:0/3900600624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a241e76f0 0x7f9a241e8280 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f9a18011610 tx=0x7f9a18011ad0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.448+0000 7f9a296ea640 1 -- 172.21.15.60:0/3900600624 shutdown_connections 2024-04-25T10:11:28.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.448+0000 7f9a296ea640 1 --2- 172.21.15.60:0/3900600624 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a10066f60 0x7f9a10069420 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.448+0000 7f9a296ea640 1 --2- 172.21.15.60:0/3900600624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a24159480 0x7f9a24159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.448+0000 7f9a296ea640 1 --2- 172.21.15.60:0/3900600624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a241e76f0 0x7f9a241e8280 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.448+0000 7f9a296ea640 1 --2- 172.21.15.60:0/3900600624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a0c004d50 0x7f9a24146980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.448+0000 7f9a296ea640 1 -- 172.21.15.60:0/3900600624 >> 172.21.15.60:0/3900600624 conn(0x7f9a24071b70 msgr2=0x7f9a2415fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:28.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.448+0000 7f9a296ea640 1 -- 172.21.15.60:0/3900600624 shutdown_connections 2024-04-25T10:11:28.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.448+0000 7f9a296ea640 1 -- 172.21.15.60:0/3900600624 wait complete. 2024-04-25T10:11:28.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:11:28.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.584+0000 7f35f51b9640 1 Processor -- start 2024-04-25T10:11:28.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.584+0000 7f35f51b9640 1 -- start start 2024-04-25T10:11:28.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.588+0000 7f35f51b9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f35f015deb0 0x7f35f00b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.588+0000 7f35f51b9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35f00ce9c0 0x7f35f00d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.588+0000 7f35f51b9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f006d200 con 0x7f35f00d0340 2024-04-25T10:11:28.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.588+0000 7f35f51b9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f00cf2c0 con 0x7f35f015deb0 2024-04-25T10:11:28.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.588+0000 7f35f51b9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00cefe0 con 0x7f35f00ce9c0 2024-04-25T10:11:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.588+0000 7f35eed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f35f015deb0 0x7f35f00b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.588+0000 7f35eed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f35f015deb0 0x7f35f00b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60986/0 (socket says 172.21.15.60:60986) 2024-04-25T10:11:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.588+0000 7f35eed9d640 1 -- 172.21.15.60:0/1468764444 learned_addr learned my addr 172.21.15.60:0/1468764444 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.588+0000 7f35ef59e640 1 --2- 172.21.15.60:0/1468764444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35f00ce9c0 0x7f35f00d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.588+0000 7f35eed9d640 1 -- 172.21.15.60:0/1468764444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35f00ce9c0 msgr2=0x7f35f00d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.588+0000 7f35eed9d640 1 --2- 172.21.15.60:0/1468764444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35f00ce9c0 0x7f35f00d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.588+0000 7f35eed9d640 1 -- 172.21.15.60:0/1468764444 >> v1:172.21.15.100:6789/0 conn(0x7f35f00d0340 legacy=0x7f35f00ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.588+0000 7f35eed9d640 1 -- 172.21.15.60:0/1468764444 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f00d1d70 con 0x7f35f015deb0 2024-04-25T10:11:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.588+0000 7f35ef59e640 1 --2- 172.21.15.60:0/1468764444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35f00ce9c0 0x7f35f00d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:28.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.588+0000 7f35eed9d640 1 --2- 172.21.15.60:0/1468764444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35f015deb0 0x7f35f00b6c10 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009870 tx=0x7f35e001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0412ab56bb44d03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35edd9b640 1 -- 172.21.15.60:0/1468764444 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e00b0070 con 0x7f35f015deb0 2024-04-25T10:11:28.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35edd9b640 1 -- 172.21.15.60:0/1468764444 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e00ab600 con 0x7f35f015deb0 2024-04-25T10:11:28.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35edd9b640 1 -- 172.21.15.60:0/1468764444 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e00ab9d0 con 0x7f35f015deb0 2024-04-25T10:11:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 -- 172.21.15.60:0/1468764444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35f015deb0 msgr2=0x7f35f00b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 --2- 172.21.15.60:0/1468764444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35f015deb0 0x7f35f00b6c10 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009870 tx=0x7f35e001c520 comp rx=0 tx=0).stop 2024-04-25T10:11:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 -- 172.21.15.60:0/1468764444 shutdown_connections 2024-04-25T10:11:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 --2- 172.21.15.60:0/1468764444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35f00ce9c0 0x7f35f00d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 --2- 172.21.15.60:0/1468764444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35f015deb0 0x7f35f00b6c10 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 -- 172.21.15.60:0/1468764444 >> 172.21.15.60:0/1468764444 conn(0x7f35f006c090 msgr2=0x7f35f006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 -- 172.21.15.60:0/1468764444 shutdown_connections 2024-04-25T10:11:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 -- 172.21.15.60:0/1468764444 wait complete. 2024-04-25T10:11:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 Processor -- start 2024-04-25T10:11:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 -- start start 2024-04-25T10:11:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f00ce9c0 0x7f35f015bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35f00d0340 0x7f35f014abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35f015deb0 0x7f35f01607f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00d3470 con 0x7f35f015deb0 2024-04-25T10:11:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00d32f0 con 0x7f35f00ce9c0 2024-04-25T10:11:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00d35f0 con 0x7f35f00d0340 2024-04-25T10:11:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35ef59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35f015deb0 0x7f35f01607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35ef59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35f015deb0 0x7f35f01607f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32768/0 (socket says 172.21.15.60:32768) 2024-04-25T10:11:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35ef59e640 1 -- 172.21.15.60:0/1391551328 learned_addr learned my addr 172.21.15.60:0/1391551328 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35eed9d640 1 --2- 172.21.15.60:0/1391551328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f00ce9c0 0x7f35f015bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35ee59c640 1 --2- 172.21.15.60:0/1391551328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35f00d0340 0x7f35f014abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35ef59e640 1 -- 172.21.15.60:0/1391551328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35f00d0340 msgr2=0x7f35f014abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35ef59e640 1 --2- 172.21.15.60:0/1391551328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35f00d0340 0x7f35f014abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35ef59e640 1 -- 172.21.15.60:0/1391551328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f00ce9c0 msgr2=0x7f35f015bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35ef59e640 1 --2- 172.21.15.60:0/1391551328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f00ce9c0 0x7f35f015bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35ef59e640 1 -- 172.21.15.60:0/1391551328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f015c350 con 0x7f35f015deb0 2024-04-25T10:11:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35eed9d640 1 --2- 172.21.15.60:0/1391551328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f00ce9c0 0x7f35f015bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35ef59e640 1 --2- 172.21.15.60:0/1391551328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35f015deb0 0x7f35f01607f0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f35e800cef0 tx=0x7f35e800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35d77fe640 1 -- 172.21.15.60:0/1391551328 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e801a070 con 0x7f35f015deb0 2024-04-25T10:11:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35d77fe640 1 -- 172.21.15.60:0/1391551328 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e800d860 con 0x7f35f015deb0 2024-04-25T10:11:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 -- 172.21.15.60:0/1391551328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f015c630 con 0x7f35f015deb0 2024-04-25T10:11:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35d77fe640 1 -- 172.21.15.60:0/1391551328 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e800dc50 con 0x7f35f015deb0 2024-04-25T10:11:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.592+0000 7f35f51b9640 1 -- 172.21.15.60:0/1391551328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f01522d0 con 0x7f35f015deb0 2024-04-25T10:11:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.596+0000 7f35f51b9640 1 -- 172.21.15.60:0/1391551328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f0074770 con 0x7f35f015deb0 2024-04-25T10:11:28.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.596+0000 7f35d77fe640 1 -- 172.21.15.60:0/1391551328 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f35e801d070 con 0x7f35f015deb0 2024-04-25T10:11:28.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.596+0000 7f35d77fe640 1 --2- 172.21.15.60:0/1391551328 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f35dc066f60 0x7f35dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.596+0000 7f35d77fe640 1 -- 172.21.15.60:0/1391551328 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f35e8085980 con 0x7f35f015deb0 2024-04-25T10:11:28.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.596+0000 7f35eed9d640 1 --2- 172.21.15.60:0/1391551328 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f35dc066f60 0x7f35dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.596+0000 7f35eed9d640 1 --2- 172.21.15.60:0/1391551328 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f35dc066f60 0x7f35dc069420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f35e00ad000 tx=0x7f35e00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.600+0000 7f35d77fe640 1 -- 172.21.15.60:0/1391551328 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f35e80569b0 con 0x7f35f015deb0 2024-04-25T10:11:28.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.764+0000 7f35f51b9640 1 -- 172.21.15.60:0/1391551328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f35f00d17a0 con 0x7f35f015deb0 2024-04-25T10:11:28.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.764+0000 7f35d77fe640 1 -- 172.21.15.60:0/1391551328 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v71) v1 ==== 80+0+122 (secure 0 0 0) 0x7f35e80562b0 con 0x7f35f015deb0 2024-04-25T10:11:28.774 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6835/1289503429 2024-04-26T10:10:03.449620+0000 2024-04-25T10:11:28.774 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6833/1289503429 2024-04-26T10:10:03.449620+0000 2024-04-25T10:11:28.774 INFO:teuthology.orchestra.run.smithi060.stderr:listed 2 entries 2024-04-25T10:11:28.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.768+0000 7f35f51b9640 1 -- 172.21.15.60:0/1391551328 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f35dc066f60 msgr2=0x7f35dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.768+0000 7f35f51b9640 1 --2- 172.21.15.60:0/1391551328 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f35dc066f60 0x7f35dc069420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f35e00ad000 tx=0x7f35e00a6000 comp rx=0 tx=0).stop 2024-04-25T10:11:28.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.768+0000 7f35f51b9640 1 -- 172.21.15.60:0/1391551328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35f015deb0 msgr2=0x7f35f01607f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.768+0000 7f35f51b9640 1 --2- 172.21.15.60:0/1391551328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35f015deb0 0x7f35f01607f0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f35e800cef0 tx=0x7f35e800a710 comp rx=0 tx=0).stop 2024-04-25T10:11:28.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.768+0000 7f35f51b9640 1 -- 172.21.15.60:0/1391551328 shutdown_connections 2024-04-25T10:11:28.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.768+0000 7f35f51b9640 1 --2- 172.21.15.60:0/1391551328 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f35dc066f60 0x7f35dc069420 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.768+0000 7f35f51b9640 1 --2- 172.21.15.60:0/1391551328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35f015deb0 0x7f35f01607f0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.768+0000 7f35f51b9640 1 --2- 172.21.15.60:0/1391551328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35f00d0340 0x7f35f014abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.768+0000 7f35f51b9640 1 --2- 172.21.15.60:0/1391551328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f00ce9c0 0x7f35f015bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.768+0000 7f35f51b9640 1 -- 172.21.15.60:0/1391551328 >> 172.21.15.60:0/1391551328 conn(0x7f35f006c090 msgr2=0x7f35f015f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:28.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.768+0000 7f35f51b9640 1 -- 172.21.15.60:0/1391551328 shutdown_connections 2024-04-25T10:11:28.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:28.768+0000 7f35f51b9640 1 -- 172.21.15.60:0/1391551328 wait complete. 2024-04-25T10:11:28.791 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:11:28.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:28.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:28.800 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[43948]: fuse finished with error 0 and tester_r 0 2024-04-25T10:11:28.802 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:11:28.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:28.802 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:11:28.970 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:11:28.970 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:28.976 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:28.977 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:11:28.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:28.977 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:11:29.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:11:29.023 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:11:29.023 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:11:29.023 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:29.029 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:11:29.029 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:11:29.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:29.030 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:11:29.075 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:11:29.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:11:29.076 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:11:29.076 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:29.082 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:29.082 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:11:29.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:29.082 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:11:29.130 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:11:29.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:11:29.131 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:29.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.256+0000 7f6c5b2ad640 1 Processor -- start 2024-04-25T10:11:29.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.256+0000 7f6c5b2ad640 1 -- start start 2024-04-25T10:11:29.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.256+0000 7f6c5b2ad640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c5415bb30 0x7f6c540cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.256+0000 7f6c5b2ad640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c5415bf10 0x7f6c54099b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.256+0000 7f6c5b2ad640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5409a560 con 0x7f6c5409a050 2024-04-25T10:11:29.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.256+0000 7f6c5b2ad640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c540cce10 con 0x7f6c5415bf10 2024-04-25T10:11:29.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.256+0000 7f6c5b2ad640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540cc4e0 con 0x7f6c5415bb30 2024-04-25T10:11:29.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.256+0000 7f6c58848640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c5415bf10 0x7f6c54099b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.256+0000 7f6c5984a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6c5409a050 0x7f6c540cfe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39414/0 (socket says 172.21.15.60:39414) 2024-04-25T10:11:29.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.256+0000 7f6c5984a640 1 -- 172.21.15.60:0/1892577673 learned_addr learned my addr 172.21.15.60:0/1892577673 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:29.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.256+0000 7f6c59049640 1 --2- 172.21.15.60:0/1892577673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c5415bb30 0x7f6c540cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.256+0000 7f6c58848640 1 -- 172.21.15.60:0/1892577673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c5415bb30 msgr2=0x7f6c540cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:29.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.256+0000 7f6c58848640 1 --2- 172.21.15.60:0/1892577673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c5415bb30 0x7f6c540cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.256+0000 7f6c58848640 1 -- 172.21.15.60:0/1892577673 >> v1:172.21.15.60:6789/0 conn(0x7f6c5409a050 legacy=0x7f6c540cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.256+0000 7f6c47fff640 1 -- 172.21.15.60:0/1892577673 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561534420 0 0) 0x7f6c5409a560 con 0x7f6c5409a050 2024-04-25T10:11:29.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.256+0000 7f6c58848640 1 -- 172.21.15.60:0/1892577673 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c540cc7c0 con 0x7f6c5415bf10 2024-04-25T10:11:29.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.260+0000 7f6c58848640 1 --2- 172.21.15.60:0/1892577673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5415bf10 0x7f6c54099b10 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009d90 tx=0x7f6c4801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebcac0fce984a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.260+0000 7f6c47fff640 1 -- 172.21.15.60:0/1892577673 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480b2070 con 0x7f6c5415bf10 2024-04-25T10:11:29.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.260+0000 7f6c47fff640 1 -- 172.21.15.60:0/1892577673 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c480a5940 con 0x7f6c5415bf10 2024-04-25T10:11:29.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.260+0000 7f6c47fff640 1 -- 172.21.15.60:0/1892577673 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480a5d10 con 0x7f6c5415bf10 2024-04-25T10:11:29.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.260+0000 7f6c5b2ad640 1 -- 172.21.15.60:0/1892577673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5415bf10 msgr2=0x7f6c54099b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.260+0000 7f6c5b2ad640 1 --2- 172.21.15.60:0/1892577673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5415bf10 0x7f6c54099b10 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009d90 tx=0x7f6c4801c970 comp rx=0 tx=0).stop 2024-04-25T10:11:29.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.260+0000 7f6c5b2ad640 1 -- 172.21.15.60:0/1892577673 shutdown_connections 2024-04-25T10:11:29.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.260+0000 7f6c5b2ad640 1 --2- 172.21.15.60:0/1892577673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5415bf10 0x7f6c54099b10 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.260+0000 7f6c5b2ad640 1 --2- 172.21.15.60:0/1892577673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c5415bb30 0x7f6c540cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.260+0000 7f6c5b2ad640 1 -- 172.21.15.60:0/1892577673 >> 172.21.15.60:0/1892577673 conn(0x7f6c540b3320 msgr2=0x7f6c540b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:29.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.260+0000 7f6c5b2ad640 1 -- 172.21.15.60:0/1892577673 shutdown_connections 2024-04-25T10:11:29.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.260+0000 7f6c5b2ad640 1 -- 172.21.15.60:0/1892577673 wait complete. 2024-04-25T10:11:29.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.260+0000 7f6c5b2ad640 1 Processor -- start 2024-04-25T10:11:29.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.260+0000 7f6c5b2ad640 1 -- start start 2024-04-25T10:11:29.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c5b2ad640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5409a050 0x7f6c54155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c5b2ad640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c5415bb30 0x7f6c541f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c5b2ad640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c5415bf10 0x7f6c540b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c5b2ad640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540b1590 con 0x7f6c5409a050 2024-04-25T10:11:29.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c5b2ad640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540d0730 con 0x7f6c5415bf10 2024-04-25T10:11:29.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c5b2ad640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540b1710 con 0x7f6c5415bb30 2024-04-25T10:11:29.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c59049640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5409a050 0x7f6c54155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c59049640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5409a050 0x7f6c54155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36826/0 (socket says 172.21.15.60:36826) 2024-04-25T10:11:29.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c59049640 1 -- 172.21.15.60:0/256571192 learned_addr learned my addr 172.21.15.60:0/256571192 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:29.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c58848640 1 --2- 172.21.15.60:0/256571192 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c5415bb30 0x7f6c541f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c5984a640 1 --2- 172.21.15.60:0/256571192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c5415bf10 0x7f6c540b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c59049640 1 -- 172.21.15.60:0/256571192 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c5415bb30 msgr2=0x7f6c541f7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c59049640 1 --2- 172.21.15.60:0/256571192 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c5415bb30 0x7f6c541f7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c59049640 1 -- 172.21.15.60:0/256571192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c5415bf10 msgr2=0x7f6c540b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c59049640 1 --2- 172.21.15.60:0/256571192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c5415bf10 0x7f6c540b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c59049640 1 -- 172.21.15.60:0/256571192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c541f85b0 con 0x7f6c5409a050 2024-04-25T10:11:29.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c5984a640 1 --2- 172.21.15.60:0/256571192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c5415bf10 0x7f6c540b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:29.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c59049640 1 --2- 172.21.15.60:0/256571192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5409a050 0x7f6c54155c50 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000d800 tx=0x7f6c5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c45ffb640 1 -- 172.21.15.60:0/256571192 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000b6c0 con 0x7f6c5409a050 2024-04-25T10:11:29.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c45ffb640 1 -- 172.21.15.60:0/256571192 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c50019070 con 0x7f6c5409a050 2024-04-25T10:11:29.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c5b2ad640 1 -- 172.21.15.60:0/256571192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c541f8890 con 0x7f6c5409a050 2024-04-25T10:11:29.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c45ffb640 1 -- 172.21.15.60:0/256571192 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000ba60 con 0x7f6c5409a050 2024-04-25T10:11:29.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c5b2ad640 1 -- 172.21.15.60:0/256571192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c54156940 con 0x7f6c5409a050 2024-04-25T10:11:29.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c5b2ad640 1 -- 172.21.15.60:0/256571192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c54074770 con 0x7f6c5409a050 2024-04-25T10:11:29.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c45ffb640 1 -- 172.21.15.60:0/256571192 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c5000bc00 con 0x7f6c5409a050 2024-04-25T10:11:29.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c45ffb640 1 --2- 172.21.15.60:0/256571192 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6c38066f90 0x7f6c38069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.264+0000 7f6c45ffb640 1 -- 172.21.15.60:0/256571192 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6c50085ed0 con 0x7f6c5409a050 2024-04-25T10:11:29.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.268+0000 7f6c58848640 1 --2- 172.21.15.60:0/256571192 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6c38066f90 0x7f6c38069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.268+0000 7f6c58848640 1 --2- 172.21.15.60:0/256571192 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6c38066f90 0x7f6c38069450 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6c480af000 tx=0x7f6c480a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.268+0000 7f6c45ffb640 1 -- 172.21.15.60:0/256571192 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c50056fc0 con 0x7f6c5409a050 2024-04-25T10:11:29.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.468+0000 7f6c5b2ad640 1 -- 172.21.15.60:0/256571192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c5415f580 con 0x7f6c5409a050 2024-04-25T10:11:29.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.468+0000 7f6c45ffb640 1 -- 172.21.15.60:0/256571192 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f6c500568c0 con 0x7f6c5409a050 2024-04-25T10:11:29.477 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:29.477 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3185787428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3185787428},{"type":"v1","addr":"172.21.15.100:6836","nonce":3185787428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2667415244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2667415244},{"type":"v1","addr":"172.21.15.100:6837","nonce":2667415244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/56747111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":56747111},{"type":"v1","addr":"172.21.15.60:6834","nonce":56747111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:43.707831+0000","modified":"2024-04-25T10:10:54.473950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7134},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7134":{"gid":7134,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":16,"addr":"172.21.15.60:6835/1295867731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1295867731},{"type":"v1","addr":"172.21.15.60:6835","nonce":1295867731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:11:29.477 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 47 2024-04-25T10:11:29.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.468+0000 7f6c5b2ad640 1 -- 172.21.15.60:0/256571192 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6c38066f90 msgr2=0x7f6c38069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.468+0000 7f6c5b2ad640 1 --2- 172.21.15.60:0/256571192 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6c38066f90 0x7f6c38069450 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6c480af000 tx=0x7f6c480a7070 comp rx=0 tx=0).stop 2024-04-25T10:11:29.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.472+0000 7f6c5b2ad640 1 -- 172.21.15.60:0/256571192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5409a050 msgr2=0x7f6c54155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.472+0000 7f6c5b2ad640 1 --2- 172.21.15.60:0/256571192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5409a050 0x7f6c54155c50 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000d800 tx=0x7f6c5000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.472+0000 7f6c5b2ad640 1 -- 172.21.15.60:0/256571192 shutdown_connections 2024-04-25T10:11:29.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.472+0000 7f6c5b2ad640 1 --2- 172.21.15.60:0/256571192 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6c38066f90 0x7f6c38069450 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.472+0000 7f6c5b2ad640 1 --2- 172.21.15.60:0/256571192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c5415bf10 0x7f6c540b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.472+0000 7f6c5b2ad640 1 --2- 172.21.15.60:0/256571192 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c5415bb30 0x7f6c541f7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.472+0000 7f6c5b2ad640 1 --2- 172.21.15.60:0/256571192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5409a050 0x7f6c54155c50 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.472+0000 7f6c5b2ad640 1 -- 172.21.15.60:0/256571192 >> 172.21.15.60:0/256571192 conn(0x7f6c540b3320 msgr2=0x7f6c54070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:29.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.472+0000 7f6c5b2ad640 1 -- 172.21.15.60:0/256571192 shutdown_connections 2024-04-25T10:11:29.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.472+0000 7f6c5b2ad640 1 -- 172.21.15.60:0/256571192 wait complete. 2024-04-25T10:11:29.496 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:29.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.604+0000 7fb90a7b5640 1 Processor -- start 2024-04-25T10:11:29.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.604+0000 7fb90a7b5640 1 -- start start 2024-04-25T10:11:29.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb90a7b5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9040ce7a0 0x7fb9040ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb90a7b5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb90415bc90 0x7fb9040b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb90a7b5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb9040b4660 0x7fb9040b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb90a7b5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9040cc370 con 0x7fb9040b4660 2024-04-25T10:11:29.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb90a7b5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9040cf0c0 con 0x7fb90415bc90 2024-04-25T10:11:29.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb90a7b5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb904071ca0 con 0x7fb9040ce7a0 2024-04-25T10:11:29.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb908d52640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb9040b4660 0x7fb9040b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb908d52640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb9040b4660 0x7fb9040b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36832/0 (socket says 172.21.15.60:36832) 2024-04-25T10:11:29.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb908d52640 1 -- 172.21.15.60:0/3296390883 learned_addr learned my addr 172.21.15.60:0/3296390883 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:29.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb9037fe640 1 --2- 172.21.15.60:0/3296390883 >> v2:172.21.15.100:3300/0 conn(0x7fb90415bc90 0x7fb9040b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb903fff640 1 --2- 172.21.15.60:0/3296390883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9040ce7a0 0x7fb9040ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb908d52640 1 -- 172.21.15.60:0/3296390883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9040ce7a0 msgr2=0x7fb9040ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:29.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb908d52640 1 --2- 172.21.15.60:0/3296390883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9040ce7a0 0x7fb9040ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb908d52640 1 -- 172.21.15.60:0/3296390883 >> v2:172.21.15.100:3300/0 conn(0x7fb90415bc90 msgr2=0x7fb9040b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:29.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb908d52640 1 --2- 172.21.15.60:0/3296390883 >> v2:172.21.15.100:3300/0 conn(0x7fb90415bc90 0x7fb9040b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb908d52640 1 -- 172.21.15.60:0/3296390883 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9040cca40 con 0x7fb9040b4660 2024-04-25T10:11:29.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb908d52640 1 --2- 172.21.15.60:0/3296390883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9040b4660 0x7fb9040b2b10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009e50 tx=0x7fb8fc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc92e6a1ae7e5f73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb902ffd640 1 -- 172.21.15.60:0/3296390883 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc0a5670 con 0x7fb9040b4660 2024-04-25T10:11:29.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb902ffd640 1 -- 172.21.15.60:0/3296390883 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8fc0a5810 con 0x7fb9040b4660 2024-04-25T10:11:29.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.608+0000 7fb902ffd640 1 -- 172.21.15.60:0/3296390883 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc0a5be0 con 0x7fb9040b4660 2024-04-25T10:11:29.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 -- 172.21.15.60:0/3296390883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9040b4660 msgr2=0x7fb9040b2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 --2- 172.21.15.60:0/3296390883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9040b4660 0x7fb9040b2b10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009e50 tx=0x7fb8fc01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 -- 172.21.15.60:0/3296390883 shutdown_connections 2024-04-25T10:11:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 --2- 172.21.15.60:0/3296390883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9040b4660 0x7fb9040b2b10 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 --2- 172.21.15.60:0/3296390883 >> v2:172.21.15.100:3300/0 conn(0x7fb90415bc90 0x7fb9040b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 --2- 172.21.15.60:0/3296390883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9040ce7a0 0x7fb9040ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 -- 172.21.15.60:0/3296390883 >> 172.21.15.60:0/3296390883 conn(0x7fb9040b12d0 msgr2=0x7fb9040b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 -- 172.21.15.60:0/3296390883 shutdown_connections 2024-04-25T10:11:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 -- 172.21.15.60:0/3296390883 wait complete. 2024-04-25T10:11:29.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 Processor -- start 2024-04-25T10:11:29.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 -- start start 2024-04-25T10:11:29.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9040ce7a0 0x7fb904145f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb90415bc90 0x7fb904159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb904156950 0x7fb90415dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040b3db0 con 0x7fb90415bc90 2024-04-25T10:11:29.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040b3460 con 0x7fb9040ce7a0 2024-04-25T10:11:29.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040b4280 con 0x7fb904156950 2024-04-25T10:11:29.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb9037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb90415bc90 0x7fb904159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb9037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb90415bc90 0x7fb904159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36848/0 (socket says 172.21.15.60:36848) 2024-04-25T10:11:29.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb9037fe640 1 -- 172.21.15.60:0/2545074145 learned_addr learned my addr 172.21.15.60:0/2545074145 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:29.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb903fff640 1 --2- 172.21.15.60:0/2545074145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9040ce7a0 0x7fb904145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb908d52640 1 --2- 172.21.15.60:0/2545074145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb904156950 0x7fb90415dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb9037fe640 1 -- 172.21.15.60:0/2545074145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb904156950 msgr2=0x7fb90415dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb9037fe640 1 --2- 172.21.15.60:0/2545074145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb904156950 0x7fb90415dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb9037fe640 1 -- 172.21.15.60:0/2545074145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9040ce7a0 msgr2=0x7fb904145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb9037fe640 1 --2- 172.21.15.60:0/2545074145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9040ce7a0 0x7fb904145f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb9037fe640 1 -- 172.21.15.60:0/2545074145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb904152d50 con 0x7fb90415bc90 2024-04-25T10:11:29.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb9037fe640 1 --2- 172.21.15.60:0/2545074145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb90415bc90 0x7fb904159f80 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800d5f0 tx=0x7fb8f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb900ff9640 1 -- 172.21.15.60:0/2545074145 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f80125f0 con 0x7fb90415bc90 2024-04-25T10:11:29.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb900ff9640 1 -- 172.21.15.60:0/2545074145 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f8019070 con 0x7fb90415bc90 2024-04-25T10:11:29.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb900ff9640 1 -- 172.21.15.60:0/2545074145 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f80129b0 con 0x7fb90415bc90 2024-04-25T10:11:29.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 -- 172.21.15.60:0/2545074145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb904156e20 con 0x7fb90415bc90 2024-04-25T10:11:29.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.612+0000 7fb90a7b5640 1 -- 172.21.15.60:0/2545074145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9041564e0 con 0x7fb90415bc90 2024-04-25T10:11:29.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.616+0000 7fb90a7b5640 1 -- 172.21.15.60:0/2545074145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb904155ff0 con 0x7fb90415bc90 2024-04-25T10:11:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.616+0000 7fb900ff9640 1 -- 172.21.15.60:0/2545074145 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8f8012b50 con 0x7fb90415bc90 2024-04-25T10:11:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.616+0000 7fb900ff9640 1 --2- 172.21.15.60:0/2545074145 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8f0066f60 0x7fb8f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.616+0000 7fb900ff9640 1 -- 172.21.15.60:0/2545074145 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb8f80854e0 con 0x7fb90415bc90 2024-04-25T10:11:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.616+0000 7fb903fff640 1 --2- 172.21.15.60:0/2545074145 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8f0066f60 0x7fb8f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.616+0000 7fb903fff640 1 --2- 172.21.15.60:0/2545074145 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8f0066f60 0x7fb8f0069420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb90414bf20 tx=0x7fb8f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.620+0000 7fb900ff9640 1 -- 172.21.15.60:0/2545074145 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb904152d50 con 0x7fb90415bc90 2024-04-25T10:11:29.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.776+0000 7fb90a7b5640 1 -- 172.21.15.60:0/2545074145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb904074770 con 0x7fb90415bc90 2024-04-25T10:11:29.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.776+0000 7fb900ff9640 1 -- 172.21.15.60:0/2545074145 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16907 (secure 0 0 0) 0x7fb8f8056460 con 0x7fb90415bc90 2024-04-25T10:11:29.786 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:29.787 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:10:03.449677+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:42.005281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:42.710028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1e","mappings":[{"from":6,"to":0}]},{"pgid":"11.28","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/1289503429":"2024-04-26T10:10:03.449620+0000","172.21.15.60:6833/1289503429":"2024-04-26T10:10:03.449620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:29.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.780+0000 7fb90a7b5640 1 -- 172.21.15.60:0/2545074145 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8f0066f60 msgr2=0x7fb8f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.780+0000 7fb90a7b5640 1 --2- 172.21.15.60:0/2545074145 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8f0066f60 0x7fb8f0069420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb90414bf20 tx=0x7fb8f4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:29.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.780+0000 7fb90a7b5640 1 -- 172.21.15.60:0/2545074145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb90415bc90 msgr2=0x7fb904159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.780+0000 7fb90a7b5640 1 --2- 172.21.15.60:0/2545074145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb90415bc90 0x7fb904159f80 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800d5f0 tx=0x7fb8f800dab0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.780+0000 7fb90a7b5640 1 -- 172.21.15.60:0/2545074145 shutdown_connections 2024-04-25T10:11:29.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.780+0000 7fb90a7b5640 1 --2- 172.21.15.60:0/2545074145 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb8f0066f60 0x7fb8f0069420 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.780+0000 7fb90a7b5640 1 --2- 172.21.15.60:0/2545074145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb904156950 0x7fb90415dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.780+0000 7fb90a7b5640 1 --2- 172.21.15.60:0/2545074145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb90415bc90 0x7fb904159f80 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.780+0000 7fb90a7b5640 1 --2- 172.21.15.60:0/2545074145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9040ce7a0 0x7fb904145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.780+0000 7fb90a7b5640 1 -- 172.21.15.60:0/2545074145 >> 172.21.15.60:0/2545074145 conn(0x7fb9040b12d0 msgr2=0x7fb9040b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:29.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.780+0000 7fb90a7b5640 1 -- 172.21.15.60:0/2545074145 shutdown_connections 2024-04-25T10:11:29.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.780+0000 7fb90a7b5640 1 -- 172.21.15.60:0/2545074145 wait complete. 2024-04-25T10:11:29.804 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:09:42.005281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:09:42.710028+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:29.805 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:29.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.912+0000 7f066f482640 1 Processor -- start 2024-04-25T10:11:29.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.912+0000 7f066f482640 1 -- start start 2024-04-25T10:11:29.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.912+0000 7f066f482640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06680cbcf0 0x7f0668160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.912+0000 7f066f482640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0668071c30 0x7f066815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066f482640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0668144a40 con 0x7f066815fa20 2024-04-25T10:11:29.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066f482640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06680b2dc0 con 0x7f06680cbcf0 2024-04-25T10:11:29.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066f482640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066815fe80 con 0x7f0668071c30 2024-04-25T10:11:29.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066d21e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06680cbcf0 0x7f0668160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066d21e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06680cbcf0 0x7f0668160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36862/0 (socket says 172.21.15.60:36862) 2024-04-25T10:11:29.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066d21e640 1 -- 172.21.15.60:0/1338514602 learned_addr learned my addr 172.21.15.60:0/1338514602 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:29.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066ca1d640 1 --2- 172.21.15.60:0/1338514602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0668071c30 0x7f066815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066d21e640 1 -- 172.21.15.60:0/1338514602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0668071c30 msgr2=0x7f066815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:29.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066d21e640 1 --2- 172.21.15.60:0/1338514602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0668071c30 0x7f066815f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066d21e640 1 -- 172.21.15.60:0/1338514602 >> v1:172.21.15.100:6789/0 conn(0x7f066815fa20 legacy=0x7f066815bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:29.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066d21e640 1 -- 172.21.15.60:0/1338514602 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06680cc9f0 con 0x7f06680cbcf0 2024-04-25T10:11:29.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066ca1d640 1 --2- 172.21.15.60:0/1338514602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0668071c30 0x7f066815f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:29.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066d21e640 1 --2- 172.21.15.60:0/1338514602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06680cbcf0 0x7f0668160080 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f0664009870 tx=0x7f066401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8fa29075b7af84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f065ffff640 1 -- 172.21.15.60:0/1338514602 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06640aa610 con 0x7f06680cbcf0 2024-04-25T10:11:29.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f065ffff640 1 -- 172.21.15.60:0/1338514602 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06640af070 con 0x7f06680cbcf0 2024-04-25T10:11:29.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f065ffff640 1 -- 172.21.15.60:0/1338514602 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06640aa9d0 con 0x7f06680cbcf0 2024-04-25T10:11:29.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066f482640 1 -- 172.21.15.60:0/1338514602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06680cbcf0 msgr2=0x7f0668160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066f482640 1 --2- 172.21.15.60:0/1338514602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06680cbcf0 0x7f0668160080 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f0664009870 tx=0x7f066401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066f482640 1 -- 172.21.15.60:0/1338514602 shutdown_connections 2024-04-25T10:11:29.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066f482640 1 --2- 172.21.15.60:0/1338514602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0668071c30 0x7f066815f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066f482640 1 --2- 172.21.15.60:0/1338514602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06680cbcf0 0x7f0668160080 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066f482640 1 -- 172.21.15.60:0/1338514602 >> 172.21.15.60:0/1338514602 conn(0x7f06680670b0 msgr2=0x7f066806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:29.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066f482640 1 -- 172.21.15.60:0/1338514602 shutdown_connections 2024-04-25T10:11:29.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.916+0000 7f066f482640 1 -- 172.21.15.60:0/1338514602 wait complete. 2024-04-25T10:11:29.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066f482640 1 Processor -- start 2024-04-25T10:11:29.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066f482640 1 -- start start 2024-04-25T10:11:29.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066f482640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0668071c30 0x7f066813d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066f482640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06680cbcf0 0x7f066813bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066f482640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f066815fa20 0x7f066813dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066f482640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06680cfed0 con 0x7f0668071c30 2024-04-25T10:11:29.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066f482640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06680cea30 con 0x7f066815fa20 2024-04-25T10:11:29.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066f482640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066806f2e0 con 0x7f06680cbcf0 2024-04-25T10:11:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066d21e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0668071c30 0x7f066813d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066d21e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0668071c30 0x7f066813d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36872/0 (socket says 172.21.15.60:36872) 2024-04-25T10:11:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066d21e640 1 -- 172.21.15.60:0/754542742 learned_addr learned my addr 172.21.15.60:0/754542742 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066ca1d640 1 --2- 172.21.15.60:0/754542742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06680cbcf0 0x7f066813bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066da1f640 1 --2- 172.21.15.60:0/754542742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f066815fa20 0x7f066813dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066d21e640 1 -- 172.21.15.60:0/754542742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06680cbcf0 msgr2=0x7f066813bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066d21e640 1 --2- 172.21.15.60:0/754542742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06680cbcf0 0x7f066813bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066d21e640 1 -- 172.21.15.60:0/754542742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f066815fa20 msgr2=0x7f066813dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066d21e640 1 --2- 172.21.15.60:0/754542742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f066815fa20 0x7f066813dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066d21e640 1 -- 172.21.15.60:0/754542742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066806ea40 con 0x7f0668071c30 2024-04-25T10:11:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066da1f640 1 --2- 172.21.15.60:0/754542742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f066815fa20 0x7f066813dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066d21e640 1 --2- 172.21.15.60:0/754542742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0668071c30 0x7f066813d460 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f06640ac040 tx=0x7f066401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f065dffb640 1 -- 172.21.15.60:0/754542742 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06640b3800 con 0x7f0668071c30 2024-04-25T10:11:29.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f065dffb640 1 -- 172.21.15.60:0/754542742 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06640b39a0 con 0x7f0668071c30 2024-04-25T10:11:29.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066f482640 1 -- 172.21.15.60:0/754542742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066806ed20 con 0x7f0668071c30 2024-04-25T10:11:29.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f065dffb640 1 -- 172.21.15.60:0/754542742 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06640b3d90 con 0x7f0668071c30 2024-04-25T10:11:29.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.920+0000 7f066f482640 1 -- 172.21.15.60:0/754542742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066813c4a0 con 0x7f0668071c30 2024-04-25T10:11:29.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.924+0000 7f066f482640 1 -- 172.21.15.60:0/754542742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066806bba0 con 0x7f0668071c30 2024-04-25T10:11:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.924+0000 7f065dffb640 1 -- 172.21.15.60:0/754542742 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06640c6390 con 0x7f0668071c30 2024-04-25T10:11:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.924+0000 7f065dffb640 1 --2- 172.21.15.60:0/754542742 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f064c066f60 0x7f064c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.924+0000 7f065dffb640 1 -- 172.21.15.60:0/754542742 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f066411f830 con 0x7f0668071c30 2024-04-25T10:11:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.924+0000 7f066ca1d640 1 --2- 172.21.15.60:0/754542742 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f064c066f60 0x7f064c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.924+0000 7f066ca1d640 1 --2- 172.21.15.60:0/754542742 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f064c066f60 0x7f064c069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f066813cbb0 tx=0x7f0658008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:29.928+0000 7f065dffb640 1 -- 172.21.15.60:0/754542742 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f06640c3060 con 0x7f0668071c30 2024-04-25T10:11:30.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.088+0000 7f066f482640 1 -- 172.21.15.60:0/754542742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f066806bba0 con 0x7f0668071c30 2024-04-25T10:11:30.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.092+0000 7f065dffb640 1 -- 172.21.15.60:0/754542742 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06640a7070 con 0x7f0668071c30 2024-04-25T10:11:30.100 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:11:30.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.092+0000 7f066f482640 1 -- 172.21.15.60:0/754542742 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f064c066f60 msgr2=0x7f064c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.092+0000 7f066f482640 1 --2- 172.21.15.60:0/754542742 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f064c066f60 0x7f064c069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f066813cbb0 tx=0x7f0658008040 comp rx=0 tx=0).stop 2024-04-25T10:11:30.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.092+0000 7f066f482640 1 -- 172.21.15.60:0/754542742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0668071c30 msgr2=0x7f066813d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.092+0000 7f066f482640 1 --2- 172.21.15.60:0/754542742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0668071c30 0x7f066813d460 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f06640ac040 tx=0x7f066401ad30 comp rx=0 tx=0).stop 2024-04-25T10:11:30.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.092+0000 7f066f482640 1 -- 172.21.15.60:0/754542742 shutdown_connections 2024-04-25T10:11:30.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.092+0000 7f066f482640 1 --2- 172.21.15.60:0/754542742 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f064c066f60 0x7f064c069420 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.092+0000 7f066f482640 1 --2- 172.21.15.60:0/754542742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f066815fa20 0x7f066813dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.092+0000 7f066f482640 1 --2- 172.21.15.60:0/754542742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06680cbcf0 0x7f066813bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.092+0000 7f066f482640 1 --2- 172.21.15.60:0/754542742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0668071c30 0x7f066813d460 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.092+0000 7f066f482640 1 -- 172.21.15.60:0/754542742 >> 172.21.15.60:0/754542742 conn(0x7f06680670b0 msgr2=0x7f0668143bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:30.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.092+0000 7f066f482640 1 -- 172.21.15.60:0/754542742 shutdown_connections 2024-04-25T10:11:30.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.092+0000 7f066f482640 1 -- 172.21.15.60:0/754542742 wait complete. 2024-04-25T10:11:30.119 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:30.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb41068640 1 Processor -- start 2024-04-25T10:11:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb41068640 1 -- start start 2024-04-25T10:11:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb41068640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb3c15fbf0 0x7ffb3c166870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb41068640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3c144a40 con 0x7ffb3c0cbcf0 2024-04-25T10:11:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb41068640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3c0cc9f0 con 0x7ffb3c160080 2024-04-25T10:11:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb41068640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c0b2dc0 con 0x7ffb3c15fbf0 2024-04-25T10:11:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb3ad9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ffb3c0cbcf0 0x7ffb3c071c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39430/0 (socket says 172.21.15.60:39430) 2024-04-25T10:11:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb3ad9d640 1 -- 172.21.15.60:0/2483798273 learned_addr learned my addr 172.21.15.60:0/2483798273 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb3b59e640 1 --2- 172.21.15.60:0/2483798273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb3c15fbf0 0x7ffb3c166870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb39d9b640 1 -- 172.21.15.60:0/2483798273 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3291737350 0 0) 0x7ffb3c144a40 con 0x7ffb3c0cbcf0 2024-04-25T10:11:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb39d9b640 1 -- 172.21.15.60:0/2483798273 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb24003530 con 0x7ffb3c0cbcf0 2024-04-25T10:11:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb39d9b640 1 -- 172.21.15.60:0/2483798273 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2654358955 0 0) 0x7ffb24003530 con 0x7ffb3c0cbcf0 2024-04-25T10:11:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb39d9b640 1 -- 172.21.15.60:0/2483798273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb3c15fbf0 msgr2=0x7ffb3c166870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb39d9b640 1 --2- 172.21.15.60:0/2483798273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb3c15fbf0 0x7ffb3c166870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb39d9b640 1 -- 172.21.15.60:0/2483798273 >> v1:172.21.15.100:6789/0 conn(0x7ffb3c160080 legacy=0x7ffb3c15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb39d9b640 1 -- 172.21.15.60:0/2483798273 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c0cccd0 con 0x7ffb3c0cbcf0 2024-04-25T10:11:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb39d9b640 1 -- 172.21.15.60:0/2483798273 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7ffb2c001680 con 0x7ffb3c0cbcf0 2024-04-25T10:11:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb39d9b640 1 -- 172.21.15.60:0/2483798273 >> v1:172.21.15.60:6789/0 conn(0x7ffb3c0cbcf0 legacy=0x7ffb3c071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb39d9b640 1 --2- 172.21.15.60:0/2483798273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb240039f0 0x7ffb24003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb39d9b640 1 --2- 172.21.15.60:0/2483798273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb3c15fbf0 0x7ffb24004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb39d9b640 1 --2- 172.21.15.60:0/2483798273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb24004d50 0x7ffb240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb39d9b640 1 -- 172.21.15.60:0/2483798273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2400ba80 con 0x7ffb240039f0 2024-04-25T10:11:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb39d9b640 1 -- 172.21.15.60:0/2483798273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2400bc30 con 0x7ffb24004d50 2024-04-25T10:11:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb39d9b640 1 -- 172.21.15.60:0/2483798273 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2400be30 con 0x7ffb3c15fbf0 2024-04-25T10:11:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb3ad9d640 1 --2- 172.21.15.60:0/2483798273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb240039f0 0x7ffb24003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb3a59c640 1 --2- 172.21.15.60:0/2483798273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb3c15fbf0 0x7ffb24004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb3ad9d640 1 -- 172.21.15.60:0/2483798273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb3c15fbf0 msgr2=0x7ffb24004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb3ad9d640 1 --2- 172.21.15.60:0/2483798273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb3c15fbf0 0x7ffb24004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb3ad9d640 1 -- 172.21.15.60:0/2483798273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb24004d50 msgr2=0x7ffb240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb3ad9d640 1 --2- 172.21.15.60:0/2483798273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb24004d50 0x7ffb240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.244+0000 7ffb3ad9d640 1 -- 172.21.15.60:0/2483798273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb24003530 con 0x7ffb240039f0 2024-04-25T10:11:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.248+0000 7ffb3ad9d640 1 --2- 172.21.15.60:0/2483798273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb240039f0 0x7ffb24003e10 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00c240 tx=0x7ffb2c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a929d0e14bfc3ebf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.248+0000 7ffb39d9b640 1 -- 172.21.15.60:0/2483798273 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c007d20 con 0x7ffb240039f0 2024-04-25T10:11:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.248+0000 7ffb39d9b640 1 -- 172.21.15.60:0/2483798273 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c0a9070 con 0x7ffb240039f0 2024-04-25T10:11:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.248+0000 7ffb39d9b640 1 -- 172.21.15.60:0/2483798273 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c005d40 con 0x7ffb240039f0 2024-04-25T10:11:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.248+0000 7ffb41068640 1 -- 172.21.15.60:0/2483798273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb240039f0 msgr2=0x7ffb24003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.248+0000 7ffb41068640 1 --2- 172.21.15.60:0/2483798273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb240039f0 0x7ffb24003e10 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00c240 tx=0x7ffb2c01cc80 comp rx=0 tx=0).stop 2024-04-25T10:11:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.248+0000 7ffb3b59e640 1 -- 172.21.15.60:0/2483798273 reap_dead start 2024-04-25T10:11:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.248+0000 7ffb41068640 1 -- 172.21.15.60:0/2483798273 shutdown_connections 2024-04-25T10:11:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.248+0000 7ffb41068640 1 -- 172.21.15.60:0/2483798273 >> 172.21.15.60:0/2483798273 conn(0x7ffb3c0670b0 msgr2=0x7ffb3c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.248+0000 7ffb41068640 1 -- 172.21.15.60:0/2483798273 shutdown_connections 2024-04-25T10:11:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.248+0000 7ffb41068640 1 -- 172.21.15.60:0/2483798273 wait complete. 2024-04-25T10:11:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.248+0000 7ffb41068640 1 Processor -- start 2024-04-25T10:11:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.248+0000 7ffb41068640 1 -- start start 2024-04-25T10:11:30.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb41068640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb240039f0 0x7ffb3c0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb41068640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb3c0b0e30 0x7ffb3c163b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb41068640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb3c164090 0x7ffb3c0aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb41068640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c0cfed0 con 0x7ffb3c0b0e30 2024-04-25T10:11:30.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb41068640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c0ce5c0 con 0x7ffb240039f0 2024-04-25T10:11:30.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb41068640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c1f4960 con 0x7ffb3c164090 2024-04-25T10:11:30.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb3a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb3c0b0e30 0x7ffb3c163b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb3a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb3c0b0e30 0x7ffb3c163b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36884/0 (socket says 172.21.15.60:36884) 2024-04-25T10:11:30.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb3a59c640 1 -- 172.21.15.60:0/3922210221 learned_addr learned my addr 172.21.15.60:0/3922210221 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:30.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb3ad9d640 1 --2- 172.21.15.60:0/3922210221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb240039f0 0x7ffb3c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb3b59e640 1 --2- 172.21.15.60:0/3922210221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb3c164090 0x7ffb3c0aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb3a59c640 1 -- 172.21.15.60:0/3922210221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb3c164090 msgr2=0x7ffb3c0aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb3a59c640 1 --2- 172.21.15.60:0/3922210221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb3c164090 0x7ffb3c0aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb3a59c640 1 -- 172.21.15.60:0/3922210221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb240039f0 msgr2=0x7ffb3c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb3a59c640 1 --2- 172.21.15.60:0/3922210221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb240039f0 0x7ffb3c0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb3a59c640 1 -- 172.21.15.60:0/3922210221 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c161bc0 con 0x7ffb3c0b0e30 2024-04-25T10:11:30.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb3b59e640 1 --2- 172.21.15.60:0/3922210221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb3c164090 0x7ffb3c0aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:30.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb3a59c640 1 --2- 172.21.15.60:0/3922210221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb3c0b0e30 0x7ffb3c163b50 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000e4b0 tx=0x7ffb3000e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb237fe640 1 -- 172.21.15.60:0/3922210221 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb300052e0 con 0x7ffb3c0b0e30 2024-04-25T10:11:30.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb237fe640 1 -- 172.21.15.60:0/3922210221 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb30018070 con 0x7ffb3c0b0e30 2024-04-25T10:11:30.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb41068640 1 -- 172.21.15.60:0/3922210221 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3c161ea0 con 0x7ffb3c0b0e30 2024-04-25T10:11:30.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb237fe640 1 -- 172.21.15.60:0/3922210221 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30005480 con 0x7ffb3c0b0e30 2024-04-25T10:11:30.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb41068640 1 -- 172.21.15.60:0/3922210221 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3c164510 con 0x7ffb3c0b0e30 2024-04-25T10:11:30.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb41068640 1 -- 172.21.15.60:0/3922210221 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3c06bba0 con 0x7ffb3c0b0e30 2024-04-25T10:11:30.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb237fe640 1 -- 172.21.15.60:0/3922210221 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb30003840 con 0x7ffb3c0b0e30 2024-04-25T10:11:30.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb237fe640 1 --2- 172.21.15.60:0/3922210221 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb28066f60 0x7ffb28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb237fe640 1 -- 172.21.15.60:0/3922210221 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ffb30084c30 con 0x7ffb3c0b0e30 2024-04-25T10:11:30.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.252+0000 7ffb3ad9d640 1 --2- 172.21.15.60:0/3922210221 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb28066f60 0x7ffb28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.256+0000 7ffb3ad9d640 1 --2- 172.21.15.60:0/3922210221 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb28066f60 0x7ffb28069420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0ab000 tx=0x7ffb2c0b1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.256+0000 7ffb237fe640 1 -- 172.21.15.60:0/3922210221 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb30055c40 con 0x7ffb3c0b0e30 2024-04-25T10:11:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.416+0000 7ffb41068640 1 -- 172.21.15.60:0/3922210221 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffb3c0b2970 con 0x7ffb3c0b0e30 2024-04-25T10:11:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.420+0000 7ffb237fe640 1 -- 172.21.15.60:0/3922210221 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffb30055540 con 0x7ffb3c0b0e30 2024-04-25T10:11:30.428 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:11:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.420+0000 7ffb41068640 1 -- 172.21.15.60:0/3922210221 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb28066f60 msgr2=0x7ffb28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.420+0000 7ffb41068640 1 --2- 172.21.15.60:0/3922210221 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb28066f60 0x7ffb28069420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0ab000 tx=0x7ffb2c0b1040 comp rx=0 tx=0).stop 2024-04-25T10:11:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.420+0000 7ffb41068640 1 -- 172.21.15.60:0/3922210221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb3c0b0e30 msgr2=0x7ffb3c163b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.420+0000 7ffb41068640 1 --2- 172.21.15.60:0/3922210221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb3c0b0e30 0x7ffb3c163b50 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000e4b0 tx=0x7ffb3000e970 comp rx=0 tx=0).stop 2024-04-25T10:11:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.420+0000 7ffb41068640 1 -- 172.21.15.60:0/3922210221 shutdown_connections 2024-04-25T10:11:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.420+0000 7ffb41068640 1 --2- 172.21.15.60:0/3922210221 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb28066f60 0x7ffb28069420 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.420+0000 7ffb41068640 1 --2- 172.21.15.60:0/3922210221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb3c164090 0x7ffb3c0aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.420+0000 7ffb41068640 1 --2- 172.21.15.60:0/3922210221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb3c0b0e30 0x7ffb3c163b50 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.420+0000 7ffb41068640 1 --2- 172.21.15.60:0/3922210221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb240039f0 0x7ffb3c0b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.420+0000 7ffb41068640 1 -- 172.21.15.60:0/3922210221 >> 172.21.15.60:0/3922210221 conn(0x7ffb3c0670b0 msgr2=0x7ffb3c145040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.420+0000 7ffb41068640 1 -- 172.21.15.60:0/3922210221 shutdown_connections 2024-04-25T10:11:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.420+0000 7ffb41068640 1 -- 172.21.15.60:0/3922210221 wait complete. 2024-04-25T10:11:30.448 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96b1267640 1 Processor -- start 2024-04-25T10:11:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96b1267640 1 -- start start 2024-04-25T10:11:30.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96b1267640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f96ac15fbf0 0x7f96ac166870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96b1267640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ac144a40 con 0x7f96ac0cbcf0 2024-04-25T10:11:30.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96b1267640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ac0cc9f0 con 0x7f96ac160080 2024-04-25T10:11:30.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96b1267640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ac0b2dc0 con 0x7f96ac15fbf0 2024-04-25T10:11:30.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96aaffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f96ac0cbcf0 0x7f96ac071c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39442/0 (socket says 172.21.15.60:39442) 2024-04-25T10:11:30.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96aaffd640 1 -- 172.21.15.60:0/1662919527 learned_addr learned my addr 172.21.15.60:0/1662919527 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96ab7fe640 1 --2- 172.21.15.60:0/1662919527 >> v2:172.21.15.100:3300/0 conn(0x7f96ac15fbf0 0x7f96ac166870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96a9ffb640 1 -- 172.21.15.60:0/1662919527 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189678854 0 0) 0x7f96ac144a40 con 0x7f96ac0cbcf0 2024-04-25T10:11:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96a9ffb640 1 -- 172.21.15.60:0/1662919527 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9694003530 con 0x7f96ac0cbcf0 2024-04-25T10:11:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96a9ffb640 1 -- 172.21.15.60:0/1662919527 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 824015073 0 0) 0x7f9694003530 con 0x7f96ac0cbcf0 2024-04-25T10:11:30.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96a9ffb640 1 -- 172.21.15.60:0/1662919527 >> v2:172.21.15.100:3300/0 conn(0x7f96ac15fbf0 msgr2=0x7f96ac166870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96a9ffb640 1 --2- 172.21.15.60:0/1662919527 >> v2:172.21.15.100:3300/0 conn(0x7f96ac15fbf0 0x7f96ac166870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96a9ffb640 1 -- 172.21.15.60:0/1662919527 >> v1:172.21.15.100:6789/0 conn(0x7f96ac160080 legacy=0x7f96ac15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96a9ffb640 1 -- 172.21.15.60:0/1662919527 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ac0cccd0 con 0x7f96ac0cbcf0 2024-04-25T10:11:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96a9ffb640 1 -- 172.21.15.60:0/1662919527 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f969c001680 con 0x7f96ac0cbcf0 2024-04-25T10:11:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96a9ffb640 1 -- 172.21.15.60:0/1662919527 >> v1:172.21.15.60:6789/0 conn(0x7f96ac0cbcf0 legacy=0x7f96ac071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96a9ffb640 1 --2- 172.21.15.60:0/1662919527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96940039c0 0x7f9694003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96a9ffb640 1 --2- 172.21.15.60:0/1662919527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9694004810 0x7f9694004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96a9ffb640 1 --2- 172.21.15.60:0/1662919527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96940051d0 0x7f969400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96a9ffb640 1 -- 172.21.15.60:0/1662919527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969400c040 con 0x7f9694004810 2024-04-25T10:11:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96a9ffb640 1 -- 172.21.15.60:0/1662919527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969400c1f0 con 0x7f96940051d0 2024-04-25T10:11:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96a9ffb640 1 -- 172.21.15.60:0/1662919527 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969400c3f0 con 0x7f96940039c0 2024-04-25T10:11:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96aa7fc640 1 --2- 172.21.15.60:0/1662919527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9694004810 0x7f9694004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96ab7fe640 1 --2- 172.21.15.60:0/1662919527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96940051d0 0x7f969400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96aaffd640 1 --2- 172.21.15.60:0/1662919527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96940039c0 0x7f9694003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96aa7fc640 1 -- 172.21.15.60:0/1662919527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96940039c0 msgr2=0x7f9694003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96aa7fc640 1 --2- 172.21.15.60:0/1662919527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96940039c0 0x7f9694003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96aa7fc640 1 -- 172.21.15.60:0/1662919527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96940051d0 msgr2=0x7f969400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96aa7fc640 1 --2- 172.21.15.60:0/1662919527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96940051d0 0x7f969400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96ab7fe640 1 -- 172.21.15.60:0/1662919527 reap_dead start 2024-04-25T10:11:30.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.564+0000 7f96aa7fc640 1 -- 172.21.15.60:0/1662919527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9694003530 con 0x7f9694004810 2024-04-25T10:11:30.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.568+0000 7f96aa7fc640 1 --2- 172.21.15.60:0/1662919527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9694004810 0x7f9694004c90 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f96a0001780 tx=0x7f96a001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755ebe2f68b708de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.568+0000 7f96a9ffb640 1 -- 172.21.15.60:0/1662919527 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a00013a0 con 0x7f9694004810 2024-04-25T10:11:30.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.568+0000 7f96a9ffb640 1 -- 172.21.15.60:0/1662919527 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a00052f0 con 0x7f9694004810 2024-04-25T10:11:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.568+0000 7f96a9ffb640 1 -- 172.21.15.60:0/1662919527 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a0003840 con 0x7f9694004810 2024-04-25T10:11:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.568+0000 7f96b1267640 1 -- 172.21.15.60:0/1662919527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9694004810 msgr2=0x7f9694004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.568+0000 7f96b1267640 1 --2- 172.21.15.60:0/1662919527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9694004810 0x7f9694004c90 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f96a0001780 tx=0x7f96a001c400 comp rx=0 tx=0).stop 2024-04-25T10:11:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.568+0000 7f96b1267640 1 -- 172.21.15.60:0/1662919527 shutdown_connections 2024-04-25T10:11:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.568+0000 7f96b1267640 1 --2- 172.21.15.60:0/1662919527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9694004810 0x7f9694004c90 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.568+0000 7f96b1267640 1 -- 172.21.15.60:0/1662919527 >> 172.21.15.60:0/1662919527 conn(0x7f96ac0670b0 msgr2=0x7f96ac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.568+0000 7f96b1267640 1 -- 172.21.15.60:0/1662919527 shutdown_connections 2024-04-25T10:11:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.568+0000 7f96b1267640 1 -- 172.21.15.60:0/1662919527 wait complete. 2024-04-25T10:11:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.568+0000 7f96b1267640 1 Processor -- start 2024-04-25T10:11:30.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.568+0000 7f96b1267640 1 -- start start 2024-04-25T10:11:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96b1267640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9694004810 0x7f96ac1f1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96b1267640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96ac13ea80 0x7f96ac15b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96b1267640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96ac1f2440 0x7f96ac15e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96b1267640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac0cc0d0 con 0x7f9694004810 2024-04-25T10:11:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96b1267640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac0ce9a0 con 0x7f96ac1f2440 2024-04-25T10:11:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96b1267640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac0ce5c0 con 0x7f96ac13ea80 2024-04-25T10:11:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96aaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9694004810 0x7f96ac1f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96aaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9694004810 0x7f96ac1f1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36912/0 (socket says 172.21.15.60:36912) 2024-04-25T10:11:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96aaffd640 1 -- 172.21.15.60:0/3686008140 learned_addr learned my addr 172.21.15.60:0/3686008140 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96ab7fe640 1 --2- 172.21.15.60:0/3686008140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96ac1f2440 0x7f96ac15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96aa7fc640 1 --2- 172.21.15.60:0/3686008140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96ac13ea80 0x7f96ac15b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96aaffd640 1 -- 172.21.15.60:0/3686008140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96ac13ea80 msgr2=0x7f96ac15b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96aaffd640 1 --2- 172.21.15.60:0/3686008140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96ac13ea80 0x7f96ac15b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96aaffd640 1 -- 172.21.15.60:0/3686008140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96ac1f2440 msgr2=0x7f96ac15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96aaffd640 1 --2- 172.21.15.60:0/3686008140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96ac1f2440 0x7f96ac15e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96aaffd640 1 -- 172.21.15.60:0/3686008140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ac1f19a0 con 0x7f9694004810 2024-04-25T10:11:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96aaffd640 1 --2- 172.21.15.60:0/3686008140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9694004810 0x7f96ac1f1330 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f969c0119b0 tx=0x7f969c011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96937fe640 1 -- 172.21.15.60:0/3686008140 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c010040 con 0x7f9694004810 2024-04-25T10:11:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96937fe640 1 -- 172.21.15.60:0/3686008140 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969c01a070 con 0x7f9694004810 2024-04-25T10:11:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96b1267640 1 -- 172.21.15.60:0/3686008140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ac1f1c80 con 0x7f9694004810 2024-04-25T10:11:30.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96937fe640 1 -- 172.21.15.60:0/3686008140 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c00fe00 con 0x7f9694004810 2024-04-25T10:11:30.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96b1267640 1 -- 172.21.15.60:0/3686008140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ac15f9a0 con 0x7f9694004810 2024-04-25T10:11:30.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96b1267640 1 -- 172.21.15.60:0/3686008140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ac06bba0 con 0x7f9694004810 2024-04-25T10:11:30.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96937fe640 1 -- 172.21.15.60:0/3686008140 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f969c01f460 con 0x7f9694004810 2024-04-25T10:11:30.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96937fe640 1 --2- 172.21.15.60:0/3686008140 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9698066f60 0x7f9698069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96937fe640 1 -- 172.21.15.60:0/3686008140 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f969c088320 con 0x7f9694004810 2024-04-25T10:11:30.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.572+0000 7f96aa7fc640 1 --2- 172.21.15.60:0/3686008140 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9698066f60 0x7f9698069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.576+0000 7f96aa7fc640 1 --2- 172.21.15.60:0/3686008140 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9698066f60 0x7f9698069420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f96a0001750 tx=0x7f96a00011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.576+0000 7f96937fe640 1 -- 172.21.15.60:0/3686008140 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f969c0593f0 con 0x7f9694004810 2024-04-25T10:11:30.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.732+0000 7f96b1267640 1 -- 172.21.15.60:0/3686008140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f96ac0b2970 con 0x7f9694004810 2024-04-25T10:11:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.732+0000 7f96937fe640 1 -- 172.21.15.60:0/3686008140 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f969c058cf0 con 0x7f9694004810 2024-04-25T10:11:30.742 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:11:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.736+0000 7f96b1267640 1 -- 172.21.15.60:0/3686008140 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9698066f60 msgr2=0x7f9698069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.736+0000 7f96b1267640 1 --2- 172.21.15.60:0/3686008140 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9698066f60 0x7f9698069420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f96a0001750 tx=0x7f96a00011a0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.736+0000 7f96b1267640 1 -- 172.21.15.60:0/3686008140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9694004810 msgr2=0x7f96ac1f1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.736+0000 7f96b1267640 1 --2- 172.21.15.60:0/3686008140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9694004810 0x7f96ac1f1330 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f969c0119b0 tx=0x7f969c011e70 comp rx=0 tx=0).stop 2024-04-25T10:11:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.736+0000 7f96b1267640 1 -- 172.21.15.60:0/3686008140 shutdown_connections 2024-04-25T10:11:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.736+0000 7f96b1267640 1 --2- 172.21.15.60:0/3686008140 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9698066f60 0x7f9698069420 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.736+0000 7f96b1267640 1 --2- 172.21.15.60:0/3686008140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96ac1f2440 0x7f96ac15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.736+0000 7f96b1267640 1 --2- 172.21.15.60:0/3686008140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96ac13ea80 0x7f96ac15b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.736+0000 7f96b1267640 1 --2- 172.21.15.60:0/3686008140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9694004810 0x7f96ac1f1330 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.736+0000 7f96b1267640 1 -- 172.21.15.60:0/3686008140 >> 172.21.15.60:0/3686008140 conn(0x7f96ac0670b0 msgr2=0x7f96ac0b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.736+0000 7f96b1267640 1 -- 172.21.15.60:0/3686008140 shutdown_connections 2024-04-25T10:11:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.736+0000 7f96b1267640 1 -- 172.21.15.60:0/3686008140 wait complete. 2024-04-25T10:11:30.761 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc290443640 1 Processor -- start 2024-04-25T10:11:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc290443640 1 -- start start 2024-04-25T10:11:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc290443640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc28815a7a0 0x7fc2880cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc290443640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc2880b37e0 0x7fc2880b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc290443640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28806f130 con 0x7fc28806ec30 2024-04-25T10:11:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc290443640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2880cdaa0 con 0x7fc2880b37e0 2024-04-25T10:11:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc290443640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28806d2f0 con 0x7fc28815a7a0 2024-04-25T10:11:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc28e9e0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc2880b37e0 0x7fc2880b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc28d9de640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc28806ec30 0x7fc288071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39450/0 (socket says 172.21.15.60:39450) 2024-04-25T10:11:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc28d9de640 1 -- 172.21.15.60:0/3943335459 learned_addr learned my addr 172.21.15.60:0/3943335459 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc28e1df640 1 --2- 172.21.15.60:0/3943335459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc28815a7a0 0x7fc2880cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc28e9e0640 1 -- 172.21.15.60:0/3943335459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc28815a7a0 msgr2=0x7fc2880cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc28e9e0640 1 --2- 172.21.15.60:0/3943335459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc28815a7a0 0x7fc2880cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc28e9e0640 1 -- 172.21.15.60:0/3943335459 >> v1:172.21.15.60:6789/0 conn(0x7fc28806ec30 legacy=0x7fc288071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc28d1dd640 1 -- 172.21.15.60:0/3943335459 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074229596 0 0) 0x7fc28806f130 con 0x7fc28806ec30 2024-04-25T10:11:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc28e9e0640 1 -- 172.21.15.60:0/3943335459 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28806f410 con 0x7fc2880b37e0 2024-04-25T10:11:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc28e1df640 1 --2- 172.21.15.60:0/3943335459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc28815a7a0 0x7fc2880cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.864+0000 7fc28e9e0640 1 --2- 172.21.15.60:0/3943335459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2880b37e0 0x7fc2880b3c90 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc2800097c0 tx=0x7fc28001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f426292e1a1e3dde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc28d1dd640 1 -- 172.21.15.60:0/3943335459 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2800b1070 con 0x7fc2880b37e0 2024-04-25T10:11:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc28d1dd640 1 -- 172.21.15.60:0/3943335459 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2800ac410 con 0x7fc2880b37e0 2024-04-25T10:11:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc28d1dd640 1 -- 172.21.15.60:0/3943335459 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2800ac7e0 con 0x7fc2880b37e0 2024-04-25T10:11:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc290443640 1 -- 172.21.15.60:0/3943335459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2880b37e0 msgr2=0x7fc2880b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc290443640 1 --2- 172.21.15.60:0/3943335459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2880b37e0 0x7fc2880b3c90 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc2800097c0 tx=0x7fc28001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc290443640 1 -- 172.21.15.60:0/3943335459 shutdown_connections 2024-04-25T10:11:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc290443640 1 --2- 172.21.15.60:0/3943335459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2880b37e0 0x7fc2880b3c90 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc290443640 1 --2- 172.21.15.60:0/3943335459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc28815a7a0 0x7fc2880cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc290443640 1 -- 172.21.15.60:0/3943335459 >> 172.21.15.60:0/3943335459 conn(0x7fc2880b12d0 msgr2=0x7fc2880b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc290443640 1 -- 172.21.15.60:0/3943335459 shutdown_connections 2024-04-25T10:11:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc290443640 1 -- 172.21.15.60:0/3943335459 wait complete. 2024-04-25T10:11:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc290443640 1 Processor -- start 2024-04-25T10:11:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc290443640 1 -- start start 2024-04-25T10:11:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc290443640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc28806ec30 0x7fc28814ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc290443640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2880b37e0 0x7fc28814f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc290443640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28815a7a0 0x7fc28815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc290443640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28806e8c0 con 0x7fc2880b37e0 2024-04-25T10:11:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc290443640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28806e740 con 0x7fc28815a7a0 2024-04-25T10:11:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc290443640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28806d8b0 con 0x7fc28806ec30 2024-04-25T10:11:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc28d9de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2880b37e0 0x7fc28814f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc28d9de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2880b37e0 0x7fc28814f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36934/0 (socket says 172.21.15.60:36934) 2024-04-25T10:11:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc28d9de640 1 -- 172.21.15.60:0/3367168997 learned_addr learned my addr 172.21.15.60:0/3367168997 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc28e9e0640 1 --2- 172.21.15.60:0/3367168997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28815a7a0 0x7fc28815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc28e1df640 1 --2- 172.21.15.60:0/3367168997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc28806ec30 0x7fc28814ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc28d9de640 1 -- 172.21.15.60:0/3367168997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc28806ec30 msgr2=0x7fc28814ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc28d9de640 1 --2- 172.21.15.60:0/3367168997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc28806ec30 0x7fc28814ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc28d9de640 1 -- 172.21.15.60:0/3367168997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28815a7a0 msgr2=0x7fc28815f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc28d9de640 1 --2- 172.21.15.60:0/3367168997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28815a7a0 0x7fc28815f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc28d9de640 1 -- 172.21.15.60:0/3367168997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2881630f0 con 0x7fc2880b37e0 2024-04-25T10:11:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc28e9e0640 1 --2- 172.21.15.60:0/3367168997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28815a7a0 0x7fc28815f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.868+0000 7fc28d9de640 1 --2- 172.21.15.60:0/3367168997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2880b37e0 0x7fc28814f2a0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc2780011f0 tx=0x7fc27800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.872+0000 7fc27effd640 1 -- 172.21.15.60:0/3367168997 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278018070 con 0x7fc2880b37e0 2024-04-25T10:11:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.872+0000 7fc27effd640 1 -- 172.21.15.60:0/3367168997 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2780074e0 con 0x7fc2880b37e0 2024-04-25T10:11:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.872+0000 7fc27effd640 1 -- 172.21.15.60:0/3367168997 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278004a10 con 0x7fc2880b37e0 2024-04-25T10:11:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.872+0000 7fc290443640 1 -- 172.21.15.60:0/3367168997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2881633d0 con 0x7fc2880b37e0 2024-04-25T10:11:30.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.872+0000 7fc290443640 1 -- 172.21.15.60:0/3367168997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc288163990 con 0x7fc2880b37e0 2024-04-25T10:11:30.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.872+0000 7fc290443640 1 -- 172.21.15.60:0/3367168997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc288074770 con 0x7fc2880b37e0 2024-04-25T10:11:30.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.872+0000 7fc27effd640 1 -- 172.21.15.60:0/3367168997 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc278004bb0 con 0x7fc2880b37e0 2024-04-25T10:11:30.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.872+0000 7fc27effd640 1 --2- 172.21.15.60:0/3367168997 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc26c066f60 0x7fc26c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.872+0000 7fc27effd640 1 -- 172.21.15.60:0/3367168997 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc278084370 con 0x7fc2880b37e0 2024-04-25T10:11:30.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.872+0000 7fc28e1df640 1 --2- 172.21.15.60:0/3367168997 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc26c066f60 0x7fc26c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.872+0000 7fc28e1df640 1 --2- 172.21.15.60:0/3367168997 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc26c066f60 0x7fc26c069420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc288159470 tx=0x7fc284008450 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:30.876+0000 7fc27effd640 1 -- 172.21.15.60:0/3367168997 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc278055380 con 0x7fc2880b37e0 2024-04-25T10:11:31.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.072+0000 7fc290443640 1 -- 172.21.15.60:0/3367168997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc288151a00 con 0x7fc2880b37e0 2024-04-25T10:11:31.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.072+0000 7fc27effd640 1 -- 172.21.15.60:0/3367168997 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fc27805b080 con 0x7fc2880b37e0 2024-04-25T10:11:31.082 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:31.083 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3185787428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3185787428},{"type":"v1","addr":"172.21.15.100:6836","nonce":3185787428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2667415244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2667415244},{"type":"v1","addr":"172.21.15.100:6837","nonce":2667415244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/56747111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":56747111},{"type":"v1","addr":"172.21.15.60:6834","nonce":56747111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:43.707831+0000","modified":"2024-04-25T10:10:54.473950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7134},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7134":{"gid":7134,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":16,"addr":"172.21.15.60:6835/1295867731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1295867731},{"type":"v1","addr":"172.21.15.60:6835","nonce":1295867731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:11:31.083 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 47 2024-04-25T10:11:31.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.076+0000 7fc290443640 1 -- 172.21.15.60:0/3367168997 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc26c066f60 msgr2=0x7fc26c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.076+0000 7fc290443640 1 --2- 172.21.15.60:0/3367168997 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc26c066f60 0x7fc26c069420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc288159470 tx=0x7fc284008450 comp rx=0 tx=0).stop 2024-04-25T10:11:31.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.076+0000 7fc290443640 1 -- 172.21.15.60:0/3367168997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2880b37e0 msgr2=0x7fc28814f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.076+0000 7fc290443640 1 --2- 172.21.15.60:0/3367168997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2880b37e0 0x7fc28814f2a0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fc2780011f0 tx=0x7fc27800c9b0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.076+0000 7fc290443640 1 -- 172.21.15.60:0/3367168997 shutdown_connections 2024-04-25T10:11:31.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.076+0000 7fc290443640 1 --2- 172.21.15.60:0/3367168997 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc26c066f60 0x7fc26c069420 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.076+0000 7fc290443640 1 --2- 172.21.15.60:0/3367168997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc28815a7a0 0x7fc28815f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.076+0000 7fc290443640 1 --2- 172.21.15.60:0/3367168997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2880b37e0 0x7fc28814f2a0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.076+0000 7fc290443640 1 --2- 172.21.15.60:0/3367168997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc28806ec30 0x7fc28814ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.076+0000 7fc290443640 1 -- 172.21.15.60:0/3367168997 >> 172.21.15.60:0/3367168997 conn(0x7fc2880b12d0 msgr2=0x7fc288150af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:31.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.076+0000 7fc290443640 1 -- 172.21.15.60:0/3367168997 shutdown_connections 2024-04-25T10:11:31.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.076+0000 7fc290443640 1 -- 172.21.15.60:0/3367168997 wait complete. 2024-04-25T10:11:31.101 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:31.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.216+0000 7f4ba76ed640 1 Processor -- start 2024-04-25T10:11:31.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.216+0000 7f4ba76ed640 1 -- start start 2024-04-25T10:11:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba76ed640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4ba00ce7a0 0x7f4ba00ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba76ed640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ba015bc90 0x7f4ba00b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba76ed640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba00cf0c0 con 0x7f4ba00b4660 2024-04-25T10:11:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba76ed640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba00cc370 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba76ed640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba0071ca0 con 0x7f4ba015bc90 2024-04-25T10:11:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba5c8a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4ba00b4660 0x7f4ba00b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39462/0 (socket says 172.21.15.60:39462) 2024-04-25T10:11:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba5489640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4ba00ce7a0 0x7f4ba00ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba5c8a640 1 -- 172.21.15.60:0/163216552 learned_addr learned my addr 172.21.15.60:0/163216552 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba4c88640 1 --2- 172.21.15.60:0/163216552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ba015bc90 0x7f4ba00b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba5489640 1 -- 172.21.15.60:0/163216552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ba015bc90 msgr2=0x7f4ba00b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba5489640 1 --2- 172.21.15.60:0/163216552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ba015bc90 0x7f4ba00b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba5489640 1 -- 172.21.15.60:0/163216552 >> v1:172.21.15.60:6789/0 conn(0x7f4ba00b4660 legacy=0x7f4ba00b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4b93fff640 1 -- 172.21.15.60:0/163216552 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761979886 0 0) 0x7f4ba00cf0c0 con 0x7f4ba00b4660 2024-04-25T10:11:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba5489640 1 -- 172.21.15.60:0/163216552 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba00cca40 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba5489640 1 --2- 172.21.15.60:0/163216552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba00ce7a0 0x7f4ba00ceb80 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0097f0 tx=0x7f4b9c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48de0dbdb139429e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4b93fff640 1 -- 172.21.15.60:0/163216552 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c0b0070 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4b93fff640 1 -- 172.21.15.60:0/163216552 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9c0ab600 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4b93fff640 1 -- 172.21.15.60:0/163216552 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c0ab9d0 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba76ed640 1 -- 172.21.15.60:0/163216552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba00ce7a0 msgr2=0x7f4ba00ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba76ed640 1 --2- 172.21.15.60:0/163216552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba00ce7a0 0x7f4ba00ceb80 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0097f0 tx=0x7f4b9c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba76ed640 1 -- 172.21.15.60:0/163216552 shutdown_connections 2024-04-25T10:11:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba76ed640 1 --2- 172.21.15.60:0/163216552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ba015bc90 0x7f4ba00b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba76ed640 1 --2- 172.21.15.60:0/163216552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba00ce7a0 0x7f4ba00ceb80 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba76ed640 1 -- 172.21.15.60:0/163216552 >> 172.21.15.60:0/163216552 conn(0x7f4ba00b12d0 msgr2=0x7f4ba00b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:31.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba76ed640 1 -- 172.21.15.60:0/163216552 shutdown_connections 2024-04-25T10:11:31.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.220+0000 7f4ba76ed640 1 -- 172.21.15.60:0/163216552 wait complete. 2024-04-25T10:11:31.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba76ed640 1 Processor -- start 2024-04-25T10:11:31.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba76ed640 1 -- start start 2024-04-25T10:11:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba76ed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba00ce7a0 0x7f4ba0140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba76ed640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ba015bc90 0x7f4ba00ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba76ed640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba015a9f0 0x7f4ba006dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba76ed640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba00b4280 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba76ed640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba00b3db0 con 0x7f4ba015a9f0 2024-04-25T10:11:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba76ed640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba00b4400 con 0x7f4ba015bc90 2024-04-25T10:11:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba5489640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba00ce7a0 0x7f4ba0140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba5489640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba00ce7a0 0x7f4ba0140ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36944/0 (socket says 172.21.15.60:36944) 2024-04-25T10:11:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba5489640 1 -- 172.21.15.60:0/1301197006 learned_addr learned my addr 172.21.15.60:0/1301197006 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba5489640 1 -- 172.21.15.60:0/1301197006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ba015bc90 msgr2=0x7f4ba00ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba5489640 1 --2- 172.21.15.60:0/1301197006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ba015bc90 0x7f4ba00ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba5489640 1 -- 172.21.15.60:0/1301197006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba015a9f0 msgr2=0x7f4ba006dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba5489640 1 --2- 172.21.15.60:0/1301197006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba015a9f0 0x7f4ba006dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba5489640 1 -- 172.21.15.60:0/1301197006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba01525b0 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba5489640 1 --2- 172.21.15.60:0/1301197006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba00ce7a0 0x7f4ba0140ea0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0ae040 tx=0x7f4b9c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4b91ffb640 1 -- 172.21.15.60:0/1301197006 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c0b0050 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4b91ffb640 1 -- 172.21.15.60:0/1301197006 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9c0bc070 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba76ed640 1 -- 172.21.15.60:0/1301197006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba00adb70 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4b91ffb640 1 -- 172.21.15.60:0/1301197006 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c0b5850 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba76ed640 1 -- 172.21.15.60:0/1301197006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba0152820 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.224+0000 7f4ba76ed640 1 -- 172.21.15.60:0/1301197006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba0074770 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.228+0000 7f4b91ffb640 1 -- 172.21.15.60:0/1301197006 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b9c0b59f0 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.228+0000 7f4b91ffb640 1 --2- 172.21.15.60:0/1301197006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b84066f60 0x7f4b84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.228+0000 7f4b91ffb640 1 -- 172.21.15.60:0/1301197006 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4b9c11f880 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.228+0000 7f4ba4c88640 1 --2- 172.21.15.60:0/1301197006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b84066f60 0x7f4b84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.228+0000 7f4ba4c88640 1 --2- 172.21.15.60:0/1301197006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b84066f60 0x7f4b84069420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4b940098f0 tx=0x7f4b94008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.232+0000 7f4b91ffb640 1 -- 172.21.15.60:0/1301197006 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b9c016090 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.392+0000 7f4ba76ed640 1 -- 172.21.15.60:0/1301197006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ba00aecc0 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.396+0000 7f4b91ffb640 1 -- 172.21.15.60:0/1301197006 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16907 (secure 0 0 0) 0x7f4b9c0f0890 con 0x7f4ba00ce7a0 2024-04-25T10:11:31.563 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:31.563 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:10:03.449677+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:42.005281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:42.710028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1e","mappings":[{"from":6,"to":0}]},{"pgid":"11.28","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/1289503429":"2024-04-26T10:10:03.449620+0000","172.21.15.60:6833/1289503429":"2024-04-26T10:10:03.449620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.400+0000 7f4ba76ed640 1 -- 172.21.15.60:0/1301197006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b84066f60 msgr2=0x7f4b84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.400+0000 7f4ba76ed640 1 --2- 172.21.15.60:0/1301197006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b84066f60 0x7f4b84069420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4b940098f0 tx=0x7f4b94008040 comp rx=0 tx=0).stop 2024-04-25T10:11:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.400+0000 7f4ba76ed640 1 -- 172.21.15.60:0/1301197006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba00ce7a0 msgr2=0x7f4ba0140ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.400+0000 7f4ba76ed640 1 --2- 172.21.15.60:0/1301197006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba00ce7a0 0x7f4ba0140ea0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0ae040 tx=0x7f4b9c01ace0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.400+0000 7f4ba76ed640 1 -- 172.21.15.60:0/1301197006 shutdown_connections 2024-04-25T10:11:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.400+0000 7f4ba76ed640 1 --2- 172.21.15.60:0/1301197006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b84066f60 0x7f4b84069420 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.400+0000 7f4ba76ed640 1 --2- 172.21.15.60:0/1301197006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba015a9f0 0x7f4ba006dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.400+0000 7f4ba76ed640 1 --2- 172.21.15.60:0/1301197006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ba015bc90 0x7f4ba00ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.400+0000 7f4ba76ed640 1 --2- 172.21.15.60:0/1301197006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba00ce7a0 0x7f4ba0140ea0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.400+0000 7f4ba76ed640 1 -- 172.21.15.60:0/1301197006 >> 172.21.15.60:0/1301197006 conn(0x7f4ba00b12d0 msgr2=0x7f4ba0154df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.400+0000 7f4ba76ed640 1 -- 172.21.15.60:0/1301197006 shutdown_connections 2024-04-25T10:11:31.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.400+0000 7f4ba76ed640 1 -- 172.21.15.60:0/1301197006 wait complete. 2024-04-25T10:11:31.566 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:11:31.567 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dc12fa640 1 Processor -- start 2024-04-25T10:11:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dc12fa640 1 -- start start 2024-04-25T10:11:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dc12fa640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7dbc0ce7a0 0x7f7dbc0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dc12fa640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dbc15bc90 0x7f7dbc0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dc12fa640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dbc0cf0c0 con 0x7f7dbc0b4660 2024-04-25T10:11:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dc12fa640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dbc0cc370 con 0x7f7dbc0ce7a0 2024-04-25T10:11:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dc12fa640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc071ca0 con 0x7f7dbc15bc90 2024-04-25T10:11:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dbaffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7dbc0ce7a0 0x7f7dbc0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dba7fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dbc15bc90 0x7f7dbc0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dbaffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7dbc0ce7a0 0x7f7dbc0ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:52394/0 (socket says 172.21.15.60:52394) 2024-04-25T10:11:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dba7fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dbc15bc90 0x7f7dbc0b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:48666/0 (socket says 172.21.15.60:48666) 2024-04-25T10:11:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dbb7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f7dbc0b4660 0x7f7dbc0b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:33240/0 (socket says 172.21.15.60:33240) 2024-04-25T10:11:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dbaffd640 1 -- 172.21.15.60:0/297893730 learned_addr learned my addr 172.21.15.60:0/297893730 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dbaffd640 1 -- 172.21.15.60:0/297893730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dbc15bc90 msgr2=0x7f7dbc0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dbaffd640 1 --2- 172.21.15.60:0/297893730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dbc15bc90 0x7f7dbc0b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dbaffd640 1 -- 172.21.15.60:0/297893730 >> v1:172.21.15.100:6789/0 conn(0x7f7dbc0b4660 legacy=0x7f7dbc0b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.688+0000 7f7dbaffd640 1 -- 172.21.15.60:0/297893730 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc0cca40 con 0x7f7dbc0ce7a0 2024-04-25T10:11:31.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.692+0000 7f7dbaffd640 1 --2- 172.21.15.60:0/297893730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dbc0ce7a0 0x7f7dbc0ceb80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0098a0 tx=0x7f7dac01c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caa93b3c2e3e33b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.692+0000 7f7db9ffb640 1 -- 172.21.15.60:0/297893730 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac0b0070 con 0x7f7dbc0ce7a0 2024-04-25T10:11:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.692+0000 7f7db9ffb640 1 -- 172.21.15.60:0/297893730 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dac0ab600 con 0x7f7dbc0ce7a0 2024-04-25T10:11:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.692+0000 7f7db9ffb640 1 -- 172.21.15.60:0/297893730 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac0ab9d0 con 0x7f7dbc0ce7a0 2024-04-25T10:11:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.692+0000 7f7dc12fa640 1 -- 172.21.15.60:0/297893730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dbc0ce7a0 msgr2=0x7f7dbc0ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.692+0000 7f7dc12fa640 1 --2- 172.21.15.60:0/297893730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dbc0ce7a0 0x7f7dbc0ceb80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0098a0 tx=0x7f7dac01c540 comp rx=0 tx=0).stop 2024-04-25T10:11:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.692+0000 7f7dc12fa640 1 -- 172.21.15.60:0/297893730 shutdown_connections 2024-04-25T10:11:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.692+0000 7f7dc12fa640 1 --2- 172.21.15.60:0/297893730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dbc15bc90 0x7f7dbc0b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.692+0000 7f7dc12fa640 1 --2- 172.21.15.60:0/297893730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dbc0ce7a0 0x7f7dbc0ceb80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.692+0000 7f7dc12fa640 1 -- 172.21.15.60:0/297893730 >> 172.21.15.60:0/297893730 conn(0x7f7dbc0b12d0 msgr2=0x7f7dbc0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.692+0000 7f7dc12fa640 1 -- 172.21.15.60:0/297893730 shutdown_connections 2024-04-25T10:11:31.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.692+0000 7f7dc12fa640 1 -- 172.21.15.60:0/297893730 wait complete. 2024-04-25T10:11:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.692+0000 7f7dc12fa640 1 Processor -- start 2024-04-25T10:11:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.692+0000 7f7dc12fa640 1 -- start start 2024-04-25T10:11:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.692+0000 7f7dc12fa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dbc0ce7a0 0x7f7dbc165f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.692+0000 7f7dc12fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dbc15bc90 0x7f7dbc154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dc12fa640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dbc1551a0 0x7f7dbc15df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dc12fa640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc0b4280 con 0x7f7dbc15bc90 2024-04-25T10:11:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dc12fa640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc0b3db0 con 0x7f7dbc0ce7a0 2024-04-25T10:11:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dc12fa640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc0b4400 con 0x7f7dbc1551a0 2024-04-25T10:11:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dba7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dbc15bc90 0x7f7dbc154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dba7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dbc15bc90 0x7f7dbc154820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36948/0 (socket says 172.21.15.60:36948) 2024-04-25T10:11:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dba7fc640 1 -- 172.21.15.60:0/2797952804 learned_addr learned my addr 172.21.15.60:0/2797952804 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dbb7fe640 1 --2- 172.21.15.60:0/2797952804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dbc1551a0 0x7f7dbc15df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dbaffd640 1 --2- 172.21.15.60:0/2797952804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dbc0ce7a0 0x7f7dbc165f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dba7fc640 1 -- 172.21.15.60:0/2797952804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dbc1551a0 msgr2=0x7f7dbc15df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dba7fc640 1 --2- 172.21.15.60:0/2797952804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dbc1551a0 0x7f7dbc15df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dba7fc640 1 -- 172.21.15.60:0/2797952804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dbc0ce7a0 msgr2=0x7f7dbc165f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dba7fc640 1 --2- 172.21.15.60:0/2797952804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dbc0ce7a0 0x7f7dbc165f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dba7fc640 1 -- 172.21.15.60:0/2797952804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc157900 con 0x7f7dbc15bc90 2024-04-25T10:11:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dba7fc640 1 --2- 172.21.15.60:0/2797952804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dbc15bc90 0x7f7dbc154820 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f7db000e870 tx=0x7f7db000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7da37fe640 1 -- 172.21.15.60:0/2797952804 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db0017070 con 0x7f7dbc15bc90 2024-04-25T10:11:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7da37fe640 1 -- 172.21.15.60:0/2797952804 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db000c940 con 0x7f7dbc15bc90 2024-04-25T10:11:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7da37fe640 1 -- 172.21.15.60:0/2797952804 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db000ccf0 con 0x7f7dbc15bc90 2024-04-25T10:11:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dc12fa640 1 -- 172.21.15.60:0/2797952804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dbc06f300 con 0x7f7dbc15bc90 2024-04-25T10:11:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dc12fa640 1 -- 172.21.15.60:0/2797952804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dbc156b40 con 0x7f7dbc15bc90 2024-04-25T10:11:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dc12fa640 1 -- 172.21.15.60:0/2797952804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dbc074770 con 0x7f7dbc15bc90 2024-04-25T10:11:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7da37fe640 1 -- 172.21.15.60:0/2797952804 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7db001a070 con 0x7f7dbc15bc90 2024-04-25T10:11:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7da37fe640 1 --2- 172.21.15.60:0/2797952804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7da8066f60 0x7f7da8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7da37fe640 1 -- 172.21.15.60:0/2797952804 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7db0084ff0 con 0x7f7dbc15bc90 2024-04-25T10:11:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.696+0000 7f7dbaffd640 1 --2- 172.21.15.60:0/2797952804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7da8066f60 0x7f7da8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.700+0000 7f7dbaffd640 1 --2- 172.21.15.60:0/2797952804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7da8066f60 0x7f7da8069420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009870 tx=0x7f7dac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.700+0000 7f7da37fe640 1 -- 172.21.15.60:0/2797952804 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7dbc157900 con 0x7f7dbc15bc90 2024-04-25T10:11:31.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.888+0000 7f7dc12fa640 1 -- 172.21.15.60:0/2797952804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7dbc0b35c0 con 0x7f7dbc15bc90 2024-04-25T10:11:31.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.888+0000 7f7da37fe640 1 -- 172.21.15.60:0/2797952804 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f7db0055ed0 con 0x7f7dbc15bc90 2024-04-25T10:11:31.898 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:31.898 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3185787428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3185787428},{"type":"v1","addr":"172.21.15.100:6836","nonce":3185787428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2667415244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2667415244},{"type":"v1","addr":"172.21.15.100:6837","nonce":2667415244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/56747111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":56747111},{"type":"v1","addr":"172.21.15.60:6834","nonce":56747111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:43.707831+0000","modified":"2024-04-25T10:10:54.473950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7134},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7134":{"gid":7134,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":16,"addr":"172.21.15.60:6835/1295867731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1295867731},{"type":"v1","addr":"172.21.15.60:6835","nonce":1295867731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:11:31.898 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 47 2024-04-25T10:11:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.892+0000 7f7dc12fa640 1 -- 172.21.15.60:0/2797952804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7da8066f60 msgr2=0x7f7da8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.892+0000 7f7dc12fa640 1 --2- 172.21.15.60:0/2797952804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7da8066f60 0x7f7da8069420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009870 tx=0x7f7dac008040 comp rx=0 tx=0).stop 2024-04-25T10:11:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.892+0000 7f7dc12fa640 1 -- 172.21.15.60:0/2797952804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dbc15bc90 msgr2=0x7f7dbc154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.892+0000 7f7dc12fa640 1 --2- 172.21.15.60:0/2797952804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dbc15bc90 0x7f7dbc154820 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f7db000e870 tx=0x7f7db000ec70 comp rx=0 tx=0).stop 2024-04-25T10:11:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.892+0000 7f7dc12fa640 1 -- 172.21.15.60:0/2797952804 shutdown_connections 2024-04-25T10:11:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.892+0000 7f7dc12fa640 1 --2- 172.21.15.60:0/2797952804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7da8066f60 0x7f7da8069420 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.892+0000 7f7dc12fa640 1 --2- 172.21.15.60:0/2797952804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7dbc1551a0 0x7f7dbc15df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.892+0000 7f7dc12fa640 1 --2- 172.21.15.60:0/2797952804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7dbc15bc90 0x7f7dbc154820 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.892+0000 7f7dc12fa640 1 --2- 172.21.15.60:0/2797952804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dbc0ce7a0 0x7f7dbc165f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.892+0000 7f7dc12fa640 1 -- 172.21.15.60:0/2797952804 >> 172.21.15.60:0/2797952804 conn(0x7f7dbc0b12d0 msgr2=0x7f7dbc15b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.892+0000 7f7dc12fa640 1 -- 172.21.15.60:0/2797952804 shutdown_connections 2024-04-25T10:11:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:31.892+0000 7f7dc12fa640 1 -- 172.21.15.60:0/2797952804 wait complete. 2024-04-25T10:11:31.916 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:32.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.028+0000 7f53b066b640 1 Processor -- start 2024-04-25T10:11:32.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.028+0000 7f53b066b640 1 -- start start 2024-04-25T10:11:32.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.032+0000 7f53b066b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f53a815deb0 0x7f53a80b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.032+0000 7f53b066b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a806d200 con 0x7f53a80ce9c0 2024-04-25T10:11:32.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.032+0000 7f53b066b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a80cefe0 con 0x7f53a80d0340 2024-04-25T10:11:32.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.032+0000 7f53b066b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a80cf2c0 con 0x7f53a815deb0 2024-04-25T10:11:32.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.032+0000 7f53ae407640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f53a815deb0 0x7f53a80b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.032+0000 7f53aec08640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f53a80ce9c0 0x7f53a80d2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39464/0 (socket says 172.21.15.60:39464) 2024-04-25T10:11:32.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.032+0000 7f53ae407640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f53a815deb0 0x7f53a80b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36954/0 (socket says 172.21.15.60:36954) 2024-04-25T10:11:32.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.032+0000 7f53aec08640 1 -- 172.21.15.60:0/2241358779 learned_addr learned my addr 172.21.15.60:0/2241358779 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:32.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.032+0000 7f53ae407640 1 -- 172.21.15.60:0/2241358779 >> v1:172.21.15.100:6789/0 conn(0x7f53a80d0340 legacy=0x7f53a80ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.032+0000 7f53ae407640 1 -- 172.21.15.60:0/2241358779 >> v1:172.21.15.60:6789/0 conn(0x7f53a80ce9c0 legacy=0x7f53a80d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.032+0000 7f53ad405640 1 -- 172.21.15.60:0/2241358779 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454785532 0 0) 0x7f53a806d200 con 0x7f53a80ce9c0 2024-04-25T10:11:32.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.032+0000 7f53ae407640 1 -- 172.21.15.60:0/2241358779 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a80d1d70 con 0x7f53a815deb0 2024-04-25T10:11:32.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.032+0000 7f53ae407640 1 --2- 172.21.15.60:0/2241358779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53a815deb0 0x7f53a80b6c10 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f53a40097c0 tx=0x7f53a401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb734ab8aebbd81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.032+0000 7f53ad405640 1 -- 172.21.15.60:0/2241358779 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a40b0070 con 0x7f53a815deb0 2024-04-25T10:11:32.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.032+0000 7f53ad405640 1 -- 172.21.15.60:0/2241358779 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a40ab600 con 0x7f53a815deb0 2024-04-25T10:11:32.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.032+0000 7f53ad405640 1 -- 172.21.15.60:0/2241358779 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a40ab9d0 con 0x7f53a815deb0 2024-04-25T10:11:32.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53b066b640 1 -- 172.21.15.60:0/2241358779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53a815deb0 msgr2=0x7f53a80b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53b066b640 1 --2- 172.21.15.60:0/2241358779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53a815deb0 0x7f53a80b6c10 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f53a40097c0 tx=0x7f53a401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53b066b640 1 -- 172.21.15.60:0/2241358779 shutdown_connections 2024-04-25T10:11:32.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53b066b640 1 --2- 172.21.15.60:0/2241358779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53a815deb0 0x7f53a80b6c10 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53b066b640 1 -- 172.21.15.60:0/2241358779 >> 172.21.15.60:0/2241358779 conn(0x7f53a806c090 msgr2=0x7f53a806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:32.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53b066b640 1 -- 172.21.15.60:0/2241358779 shutdown_connections 2024-04-25T10:11:32.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53b066b640 1 -- 172.21.15.60:0/2241358779 wait complete. 2024-04-25T10:11:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53b066b640 1 Processor -- start 2024-04-25T10:11:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53b066b640 1 -- start start 2024-04-25T10:11:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53b066b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53a80ce9c0 0x7f53a8148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53b066b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53a80d0340 0x7f53a80c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53b066b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53a815deb0 0x7f53a814e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53b066b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a816bdc0 con 0x7f53a815deb0 2024-04-25T10:11:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53b066b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a816bc40 con 0x7f53a80d0340 2024-04-25T10:11:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53b066b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a816bf40 con 0x7f53a80ce9c0 2024-04-25T10:11:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53aec08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53a815deb0 0x7f53a814e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53aec08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53a815deb0 0x7f53a814e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36968/0 (socket says 172.21.15.60:36968) 2024-04-25T10:11:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53aec08640 1 -- 172.21.15.60:0/1824825582 learned_addr learned my addr 172.21.15.60:0/1824825582 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53adc06640 1 --2- 172.21.15.60:0/1824825582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53a80d0340 0x7f53a80c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53ae407640 1 --2- 172.21.15.60:0/1824825582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53a80ce9c0 0x7f53a8148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53aec08640 1 -- 172.21.15.60:0/1824825582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53a80ce9c0 msgr2=0x7f53a8148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53aec08640 1 --2- 172.21.15.60:0/1824825582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53a80ce9c0 0x7f53a8148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53aec08640 1 -- 172.21.15.60:0/1824825582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53a80d0340 msgr2=0x7f53a80c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53aec08640 1 --2- 172.21.15.60:0/1824825582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53a80d0340 0x7f53a80c7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53aec08640 1 -- 172.21.15.60:0/1824825582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a8149820 con 0x7f53a815deb0 2024-04-25T10:11:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f53aec08640 1 --2- 172.21.15.60:0/1824825582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53a815deb0 0x7f53a814e9c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f53a000c7a0 tx=0x7f53a000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f539effd640 1 -- 172.21.15.60:0/1824825582 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0018070 con 0x7f53a815deb0 2024-04-25T10:11:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.036+0000 7f539effd640 1 -- 172.21.15.60:0/1824825582 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a000a640 con 0x7f53a815deb0 2024-04-25T10:11:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.040+0000 7f53b066b640 1 -- 172.21.15.60:0/1824825582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a8149b00 con 0x7f53a815deb0 2024-04-25T10:11:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.040+0000 7f539effd640 1 -- 172.21.15.60:0/1824825582 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a000aab0 con 0x7f53a815deb0 2024-04-25T10:11:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.040+0000 7f53b066b640 1 -- 172.21.15.60:0/1824825582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a80c7b90 con 0x7f53a815deb0 2024-04-25T10:11:32.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.040+0000 7f53b066b640 1 -- 172.21.15.60:0/1824825582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a8074770 con 0x7f53a815deb0 2024-04-25T10:11:32.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.040+0000 7f539effd640 1 -- 172.21.15.60:0/1824825582 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f53a001b070 con 0x7f53a815deb0 2024-04-25T10:11:32.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.040+0000 7f539effd640 1 --2- 172.21.15.60:0/1824825582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f538c066f60 0x7f538c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.040+0000 7f539effd640 1 -- 172.21.15.60:0/1824825582 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f53a0084670 con 0x7f53a815deb0 2024-04-25T10:11:32.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.040+0000 7f53ae407640 1 --2- 172.21.15.60:0/1824825582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f538c066f60 0x7f538c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.040+0000 7f53ae407640 1 --2- 172.21.15.60:0/1824825582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f538c066f60 0x7f538c069420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f53a40ad000 tx=0x7f53a40a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.044+0000 7f539effd640 1 -- 172.21.15.60:0/1824825582 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f53a0055660 con 0x7f53a815deb0 2024-04-25T10:11:32.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.240+0000 7f53b066b640 1 -- 172.21.15.60:0/1824825582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53a80d17a0 con 0x7f53a815deb0 2024-04-25T10:11:32.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.240+0000 7f539effd640 1 -- 172.21.15.60:0/1824825582 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f53a0054f60 con 0x7f53a815deb0 2024-04-25T10:11:32.249 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:32.249 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3185787428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3185787428},{"type":"v1","addr":"172.21.15.100:6836","nonce":3185787428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2667415244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2667415244},{"type":"v1","addr":"172.21.15.100:6837","nonce":2667415244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/56747111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":56747111},{"type":"v1","addr":"172.21.15.60:6834","nonce":56747111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:43.707831+0000","modified":"2024-04-25T10:10:54.473950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7134},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7134":{"gid":7134,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":16,"addr":"172.21.15.60:6835/1295867731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1295867731},{"type":"v1","addr":"172.21.15.60:6835","nonce":1295867731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:11:32.249 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 47 2024-04-25T10:11:32.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.240+0000 7f53b066b640 1 -- 172.21.15.60:0/1824825582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f538c066f60 msgr2=0x7f538c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.240+0000 7f53b066b640 1 --2- 172.21.15.60:0/1824825582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f538c066f60 0x7f538c069420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f53a40ad000 tx=0x7f53a40a6000 comp rx=0 tx=0).stop 2024-04-25T10:11:32.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.240+0000 7f53b066b640 1 -- 172.21.15.60:0/1824825582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53a815deb0 msgr2=0x7f53a814e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.240+0000 7f53b066b640 1 --2- 172.21.15.60:0/1824825582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53a815deb0 0x7f53a814e9c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f53a000c7a0 tx=0x7f53a000cc60 comp rx=0 tx=0).stop 2024-04-25T10:11:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.240+0000 7f53b066b640 1 -- 172.21.15.60:0/1824825582 shutdown_connections 2024-04-25T10:11:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.240+0000 7f53b066b640 1 --2- 172.21.15.60:0/1824825582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f538c066f60 0x7f538c069420 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.240+0000 7f53b066b640 1 --2- 172.21.15.60:0/1824825582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53a815deb0 0x7f53a814e9c0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.240+0000 7f53b066b640 1 --2- 172.21.15.60:0/1824825582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53a80d0340 0x7f53a80c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.240+0000 7f53b066b640 1 --2- 172.21.15.60:0/1824825582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53a80ce9c0 0x7f53a8148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.240+0000 7f53b066b640 1 -- 172.21.15.60:0/1824825582 >> 172.21.15.60:0/1824825582 conn(0x7f53a806c090 msgr2=0x7f53a814cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.240+0000 7f53b066b640 1 -- 172.21.15.60:0/1824825582 shutdown_connections 2024-04-25T10:11:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.240+0000 7f53b066b640 1 -- 172.21.15.60:0/1824825582 wait complete. 2024-04-25T10:11:32.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:32.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.384+0000 7f2cf05a0640 1 Processor -- start 2024-04-25T10:11:32.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.384+0000 7f2cf05a0640 1 -- start start 2024-04-25T10:11:32.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.384+0000 7f2cf05a0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2ce815b9f0 0x7f2ce80cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.384+0000 7f2cf05a0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2ce80c5950 0x7f2ce815f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.384+0000 7f2cf05a0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce8099ad0 con 0x7f2ce809a850 2024-04-25T10:11:32.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.384+0000 7f2cf05a0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce815bdd0 con 0x7f2ce80c5950 2024-04-25T10:11:32.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.384+0000 7f2cf05a0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce8099d70 con 0x7f2ce815b9f0 2024-04-25T10:11:32.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.384+0000 7f2ceeb3d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2ce80c5950 0x7f2ce815f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.384+0000 7f2cedb3b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2ce809a850 0x7f2ce80c5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39468/0 (socket says 172.21.15.60:39468) 2024-04-25T10:11:32.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.384+0000 7f2cedb3b640 1 -- 172.21.15.60:0/159147996 learned_addr learned my addr 172.21.15.60:0/159147996 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:32.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.384+0000 7f2cee33c640 1 --2- 172.21.15.60:0/159147996 >> v2:172.21.15.100:3300/0 conn(0x7f2ce815b9f0 0x7f2ce80cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.384+0000 7f2ceeb3d640 1 -- 172.21.15.60:0/159147996 >> v2:172.21.15.100:3300/0 conn(0x7f2ce815b9f0 msgr2=0x7f2ce80cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:32.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.384+0000 7f2ceeb3d640 1 --2- 172.21.15.60:0/159147996 >> v2:172.21.15.100:3300/0 conn(0x7f2ce815b9f0 0x7f2ce80cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.384+0000 7f2ceeb3d640 1 -- 172.21.15.60:0/159147996 >> v1:172.21.15.60:6789/0 conn(0x7f2ce809a850 legacy=0x7f2ce80c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.384+0000 7f2ced33a640 1 -- 172.21.15.60:0/159147996 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188651144 0 0) 0x7f2ce8099ad0 con 0x7f2ce809a850 2024-04-25T10:11:32.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.384+0000 7f2ceeb3d640 1 -- 172.21.15.60:0/159147996 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce809a050 con 0x7f2ce80c5950 2024-04-25T10:11:32.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.388+0000 7f2ceeb3d640 1 --2- 172.21.15.60:0/159147996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80c5950 0x7f2ce815f460 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8073140 tx=0x7f2ce001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d8d4290f3fae88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.388+0000 7f2ced33a640 1 -- 172.21.15.60:0/159147996 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce00b2070 con 0x7f2ce80c5950 2024-04-25T10:11:32.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.388+0000 7f2ced33a640 1 -- 172.21.15.60:0/159147996 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce00a5760 con 0x7f2ce80c5950 2024-04-25T10:11:32.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.388+0000 7f2ced33a640 1 -- 172.21.15.60:0/159147996 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce00a5b30 con 0x7f2ce80c5950 2024-04-25T10:11:32.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.388+0000 7f2cf05a0640 1 -- 172.21.15.60:0/159147996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80c5950 msgr2=0x7f2ce815f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.388+0000 7f2cf05a0640 1 --2- 172.21.15.60:0/159147996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80c5950 0x7f2ce815f460 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8073140 tx=0x7f2ce001c8a0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.388+0000 7f2cf05a0640 1 -- 172.21.15.60:0/159147996 shutdown_connections 2024-04-25T10:11:32.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.388+0000 7f2cf05a0640 1 --2- 172.21.15.60:0/159147996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80c5950 0x7f2ce815f460 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.388+0000 7f2cf05a0640 1 --2- 172.21.15.60:0/159147996 >> v2:172.21.15.100:3300/0 conn(0x7f2ce815b9f0 0x7f2ce80cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.388+0000 7f2cf05a0640 1 -- 172.21.15.60:0/159147996 >> 172.21.15.60:0/159147996 conn(0x7f2ce80b31e0 msgr2=0x7f2ce80b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:32.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.388+0000 7f2cf05a0640 1 -- 172.21.15.60:0/159147996 shutdown_connections 2024-04-25T10:11:32.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.388+0000 7f2cf05a0640 1 -- 172.21.15.60:0/159147996 wait complete. 2024-04-25T10:11:32.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cf05a0640 1 Processor -- start 2024-04-25T10:11:32.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cf05a0640 1 -- start start 2024-04-25T10:11:32.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cf05a0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ce809a850 0x7f2ce8157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cf05a0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80c5950 0x7f2ce80c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cf05a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce815b9f0 0x7f2ce814fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cf05a0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce81667f0 con 0x7f2ce80c5950 2024-04-25T10:11:32.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cf05a0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce809aed0 con 0x7f2ce815b9f0 2024-04-25T10:11:32.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cf05a0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8166970 con 0x7f2ce809a850 2024-04-25T10:11:32.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cedb3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80c5950 0x7f2ce80c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cedb3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80c5950 0x7f2ce80c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36980/0 (socket says 172.21.15.60:36980) 2024-04-25T10:11:32.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cedb3b640 1 -- 172.21.15.60:0/511574929 learned_addr learned my addr 172.21.15.60:0/511574929 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:32.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2ceeb3d640 1 --2- 172.21.15.60:0/511574929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce815b9f0 0x7f2ce814fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cee33c640 1 --2- 172.21.15.60:0/511574929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ce809a850 0x7f2ce8157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cedb3b640 1 -- 172.21.15.60:0/511574929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ce809a850 msgr2=0x7f2ce8157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cedb3b640 1 --2- 172.21.15.60:0/511574929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ce809a850 0x7f2ce8157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cedb3b640 1 -- 172.21.15.60:0/511574929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce815b9f0 msgr2=0x7f2ce814fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cedb3b640 1 --2- 172.21.15.60:0/511574929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce815b9f0 0x7f2ce814fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cedb3b640 1 -- 172.21.15.60:0/511574929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce815a110 con 0x7f2ce80c5950 2024-04-25T10:11:32.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2ceeb3d640 1 --2- 172.21.15.60:0/511574929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce815b9f0 0x7f2ce814fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:32.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cedb3b640 1 --2- 172.21.15.60:0/511574929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80c5950 0x7f2ce80c51e0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400c4a0 tx=0x7f2ce400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cdaffd640 1 -- 172.21.15.60:0/511574929 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4018070 con 0x7f2ce80c5950 2024-04-25T10:11:32.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cdaffd640 1 -- 172.21.15.60:0/511574929 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce40074e0 con 0x7f2ce80c5950 2024-04-25T10:11:32.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cdaffd640 1 -- 172.21.15.60:0/511574929 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4004a30 con 0x7f2ce80c5950 2024-04-25T10:11:32.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cf05a0640 1 -- 172.21.15.60:0/511574929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce8152c20 con 0x7f2ce80c5950 2024-04-25T10:11:32.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cf05a0640 1 -- 172.21.15.60:0/511574929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce815a3d0 con 0x7f2ce80c5950 2024-04-25T10:11:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.392+0000 7f2cf05a0640 1 -- 172.21.15.60:0/511574929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce8074770 con 0x7f2ce80c5950 2024-04-25T10:11:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.396+0000 7f2cdaffd640 1 -- 172.21.15.60:0/511574929 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ce401b070 con 0x7f2ce80c5950 2024-04-25T10:11:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.396+0000 7f2cdaffd640 1 --2- 172.21.15.60:0/511574929 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ccc066f60 0x7f2ccc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.396+0000 7f2cdaffd640 1 -- 172.21.15.60:0/511574929 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2ce4084340 con 0x7f2ce80c5950 2024-04-25T10:11:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.396+0000 7f2cee33c640 1 --2- 172.21.15.60:0/511574929 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ccc066f60 0x7f2ccc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.396+0000 7f2cee33c640 1 --2- 172.21.15.60:0/511574929 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ccc066f60 0x7f2ccc069420 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009a50 tx=0x7f2cdc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.400+0000 7f2cdaffd640 1 -- 172.21.15.60:0/511574929 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ce4055350 con 0x7f2ce80c5950 2024-04-25T10:11:32.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.556+0000 7f2cf05a0640 1 -- 172.21.15.60:0/511574929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ce80ced00 con 0x7f2ce80c5950 2024-04-25T10:11:32.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.556+0000 7f2cdaffd640 1 -- 172.21.15.60:0/511574929 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16907 (secure 0 0 0) 0x7f2ce4054c50 con 0x7f2ce80c5950 2024-04-25T10:11:32.566 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:32.566 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:10:03.449677+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:42.005281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:42.710028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1e","mappings":[{"from":6,"to":0}]},{"pgid":"11.28","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/1289503429":"2024-04-26T10:10:03.449620+0000","172.21.15.60:6833/1289503429":"2024-04-26T10:10:03.449620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:32.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.560+0000 7f2cf05a0640 1 -- 172.21.15.60:0/511574929 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ccc066f60 msgr2=0x7f2ccc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.560+0000 7f2cf05a0640 1 --2- 172.21.15.60:0/511574929 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ccc066f60 0x7f2ccc069420 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009a50 tx=0x7f2cdc008040 comp rx=0 tx=0).stop 2024-04-25T10:11:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.560+0000 7f2cf05a0640 1 -- 172.21.15.60:0/511574929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80c5950 msgr2=0x7f2ce80c51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.560+0000 7f2cf05a0640 1 --2- 172.21.15.60:0/511574929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80c5950 0x7f2ce80c51e0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400c4a0 tx=0x7f2ce400c960 comp rx=0 tx=0).stop 2024-04-25T10:11:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.560+0000 7f2cf05a0640 1 -- 172.21.15.60:0/511574929 shutdown_connections 2024-04-25T10:11:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.560+0000 7f2cf05a0640 1 --2- 172.21.15.60:0/511574929 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ccc066f60 0x7f2ccc069420 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.560+0000 7f2cf05a0640 1 --2- 172.21.15.60:0/511574929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce815b9f0 0x7f2ce814fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.560+0000 7f2cf05a0640 1 --2- 172.21.15.60:0/511574929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ce80c5950 0x7f2ce80c51e0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.560+0000 7f2cf05a0640 1 --2- 172.21.15.60:0/511574929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ce809a850 0x7f2ce8157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.560+0000 7f2cf05a0640 1 -- 172.21.15.60:0/511574929 >> 172.21.15.60:0/511574929 conn(0x7f2ce80b31e0 msgr2=0x7f2ce8166cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.560+0000 7f2cf05a0640 1 -- 172.21.15.60:0/511574929 shutdown_connections 2024-04-25T10:11:32.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.560+0000 7f2cf05a0640 1 -- 172.21.15.60:0/511574929 wait complete. 2024-04-25T10:11:32.584 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:11:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.700+0000 7fb5ec81d640 1 Processor -- start 2024-04-25T10:11:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.700+0000 7fb5ec81d640 1 -- start start 2024-04-25T10:11:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.700+0000 7fb5ec81d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb5e80ae7c0 0x7fb5e8071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.700+0000 7fb5ec81d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e8071fd0 con 0x7fb5e815dc90 2024-04-25T10:11:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.700+0000 7fb5ec81d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e80b2ce0 con 0x7fb5e806d2f0 2024-04-25T10:11:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.700+0000 7fb5ec81d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e80accc0 con 0x7fb5e80ae7c0 2024-04-25T10:11:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.700+0000 7fb5e659c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb5e815dc90 0x7fb5e80b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39472/0 (socket says 172.21.15.60:39472) 2024-04-25T10:11:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.700+0000 7fb5e659c640 1 -- 172.21.15.60:0/2068085539 learned_addr learned my addr 172.21.15.60:0/2068085539 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.700+0000 7fb5e5d9b640 1 --2- 172.21.15.60:0/2068085539 >> v2:172.21.15.60:3300/0 conn(0x7fb5e80ae7c0 0x7fb5e8071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.700+0000 7fb5e559a640 1 -- 172.21.15.60:0/2068085539 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078926240 0 0) 0x7fb5e8071fd0 con 0x7fb5e815dc90 2024-04-25T10:11:32.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.700+0000 7fb5e559a640 1 -- 172.21.15.60:0/2068085539 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d0003530 con 0x7fb5e815dc90 2024-04-25T10:11:32.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.700+0000 7fb5e5d9b640 1 -- 172.21.15.60:0/2068085539 >> v1:172.21.15.100:6789/0 conn(0x7fb5e806d2f0 legacy=0x7fb5e80ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.700+0000 7fb5e5d9b640 1 -- 172.21.15.60:0/2068085539 >> v1:172.21.15.60:6789/0 conn(0x7fb5e815dc90 legacy=0x7fb5e80b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.700+0000 7fb5e5d9b640 1 -- 172.21.15.60:0/2068085539 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e8072290 con 0x7fb5e80ae7c0 2024-04-25T10:11:32.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.704+0000 7fb5e5d9b640 1 --2- 172.21.15.60:0/2068085539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5e80ae7c0 0x7fb5e8071290 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80097c0 tx=0x7fb5d801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b65c3005e35fa2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.704+0000 7fb5e559a640 1 -- 172.21.15.60:0/2068085539 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80b1070 con 0x7fb5e80ae7c0 2024-04-25T10:11:32.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.704+0000 7fb5e559a640 1 -- 172.21.15.60:0/2068085539 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d80ac410 con 0x7fb5e80ae7c0 2024-04-25T10:11:32.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.704+0000 7fb5e559a640 1 -- 172.21.15.60:0/2068085539 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80ac7e0 con 0x7fb5e80ae7c0 2024-04-25T10:11:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.704+0000 7fb5ec81d640 1 -- 172.21.15.60:0/2068085539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5e80ae7c0 msgr2=0x7fb5e8071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.704+0000 7fb5ec81d640 1 --2- 172.21.15.60:0/2068085539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5e80ae7c0 0x7fb5e8071290 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80097c0 tx=0x7fb5d801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.704+0000 7fb5ec81d640 1 -- 172.21.15.60:0/2068085539 shutdown_connections 2024-04-25T10:11:32.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.704+0000 7fb5ec81d640 1 --2- 172.21.15.60:0/2068085539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5e80ae7c0 0x7fb5e8071290 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.704+0000 7fb5ec81d640 1 -- 172.21.15.60:0/2068085539 >> 172.21.15.60:0/2068085539 conn(0x7fb5e80b12d0 msgr2=0x7fb5e80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:32.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.704+0000 7fb5ec81d640 1 -- 172.21.15.60:0/2068085539 shutdown_connections 2024-04-25T10:11:32.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.704+0000 7fb5ec81d640 1 -- 172.21.15.60:0/2068085539 wait complete. 2024-04-25T10:11:32.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.704+0000 7fb5ec81d640 1 Processor -- start 2024-04-25T10:11:32.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.704+0000 7fb5ec81d640 1 -- start start 2024-04-25T10:11:32.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5ec81d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5e806d2f0 0x7fb5e814cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5ec81d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e80ae7c0 0x7fb5e814c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5ec81d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5e815dc90 0x7fb5e815a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5ec81d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80ad260 con 0x7fb5e806d2f0 2024-04-25T10:11:32.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5ec81d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80b2220 con 0x7fb5e80ae7c0 2024-04-25T10:11:32.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5ec81d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e8071be0 con 0x7fb5e815dc90 2024-04-25T10:11:32.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5e659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5e806d2f0 0x7fb5e814cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5e659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5e806d2f0 0x7fb5e814cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36988/0 (socket says 172.21.15.60:36988) 2024-04-25T10:11:32.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5e659c640 1 -- 172.21.15.60:0/1527559689 learned_addr learned my addr 172.21.15.60:0/1527559689 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:32.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5e5d9b640 1 --2- 172.21.15.60:0/1527559689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e80ae7c0 0x7fb5e814c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5e6d9d640 1 --2- 172.21.15.60:0/1527559689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5e815dc90 0x7fb5e815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5e659c640 1 -- 172.21.15.60:0/1527559689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5e815dc90 msgr2=0x7fb5e815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5e659c640 1 --2- 172.21.15.60:0/1527559689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5e815dc90 0x7fb5e815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5e659c640 1 -- 172.21.15.60:0/1527559689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e80ae7c0 msgr2=0x7fb5e814c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5e659c640 1 --2- 172.21.15.60:0/1527559689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e80ae7c0 0x7fb5e814c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5e659c640 1 -- 172.21.15.60:0/1527559689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e8156720 con 0x7fb5e806d2f0 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5e6d9d640 1 --2- 172.21.15.60:0/1527559689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5e815dc90 0x7fb5e815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5e659c640 1 --2- 172.21.15.60:0/1527559689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5e806d2f0 0x7fb5e814cd70 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00c4d0 tx=0x7fb5dc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5d6ffd640 1 -- 172.21.15.60:0/1527559689 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc017070 con 0x7fb5e806d2f0 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5d6ffd640 1 -- 172.21.15.60:0/1527559689 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5dc005cf0 con 0x7fb5e806d2f0 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5d6ffd640 1 -- 172.21.15.60:0/1527559689 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc007270 con 0x7fb5e806d2f0 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5ec81d640 1 -- 172.21.15.60:0/1527559689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e8156a00 con 0x7fb5e806d2f0 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5ec81d640 1 -- 172.21.15.60:0/1527559689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e8156fc0 con 0x7fb5e806d2f0 2024-04-25T10:11:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5ec81d640 1 -- 172.21.15.60:0/1527559689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e8074770 con 0x7fb5e806d2f0 2024-04-25T10:11:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5d6ffd640 1 -- 172.21.15.60:0/1527559689 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5dc01a070 con 0x7fb5e806d2f0 2024-04-25T10:11:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5d6ffd640 1 --2- 172.21.15.60:0/1527559689 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb5c8066f60 0x7fb5c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5d6ffd640 1 -- 172.21.15.60:0/1527559689 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb5dc084420 con 0x7fb5e806d2f0 2024-04-25T10:11:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.708+0000 7fb5e5d9b640 1 --2- 172.21.15.60:0/1527559689 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb5c8066f60 0x7fb5c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.712+0000 7fb5e5d9b640 1 --2- 172.21.15.60:0/1527559689 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb5c8066f60 0x7fb5c8069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80ae000 tx=0x7fb5d80a6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.712+0000 7fb5d6ffd640 1 -- 172.21.15.60:0/1527559689 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb5dc054d40 con 0x7fb5e806d2f0 2024-04-25T10:11:32.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.908+0000 7fb5ec81d640 1 -- 172.21.15.60:0/1527559689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb5e81519e0 con 0x7fb5e806d2f0 2024-04-25T10:11:32.916 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:11:32.908+0000 7f3a792d6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:11:32.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.908+0000 7fb5d6ffd640 1 -- 172.21.15.60:0/1527559689 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb5dc058de0 con 0x7fb5e806d2f0 2024-04-25T10:11:32.919 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:11:32.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.916+0000 7fb5ec81d640 1 -- 172.21.15.60:0/1527559689 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb5c8066f60 msgr2=0x7fb5c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.916+0000 7fb5ec81d640 1 --2- 172.21.15.60:0/1527559689 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb5c8066f60 0x7fb5c8069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80ae000 tx=0x7fb5d80a6040 comp rx=0 tx=0).stop 2024-04-25T10:11:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.916+0000 7fb5ec81d640 1 -- 172.21.15.60:0/1527559689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5e806d2f0 msgr2=0x7fb5e814cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.916+0000 7fb5ec81d640 1 --2- 172.21.15.60:0/1527559689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5e806d2f0 0x7fb5e814cd70 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00c4d0 tx=0x7fb5dc00c990 comp rx=0 tx=0).stop 2024-04-25T10:11:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.916+0000 7fb5ec81d640 1 -- 172.21.15.60:0/1527559689 shutdown_connections 2024-04-25T10:11:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.916+0000 7fb5ec81d640 1 --2- 172.21.15.60:0/1527559689 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb5c8066f60 0x7fb5c8069420 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.916+0000 7fb5ec81d640 1 --2- 172.21.15.60:0/1527559689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5e815dc90 0x7fb5e815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.916+0000 7fb5ec81d640 1 --2- 172.21.15.60:0/1527559689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e80ae7c0 0x7fb5e814c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.916+0000 7fb5ec81d640 1 --2- 172.21.15.60:0/1527559689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5e806d2f0 0x7fb5e814cd70 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.916+0000 7fb5ec81d640 1 -- 172.21.15.60:0/1527559689 >> 172.21.15.60:0/1527559689 conn(0x7fb5e80b12d0 msgr2=0x7fb5e806dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.916+0000 7fb5ec81d640 1 -- 172.21.15.60:0/1527559689 shutdown_connections 2024-04-25T10:11:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:32.916+0000 7fb5ec81d640 1 -- 172.21.15.60:0/1527559689 wait complete. 2024-04-25T10:11:32.929 INFO:tasks.ceph.mds.c.smithi060.stderr: -4511> 2024-04-25T10:10:09.461+0000 7fd9f6d8d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:32.939 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:11:32.963 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:11:33.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.044+0000 7f81ecdf3640 1 Processor -- start 2024-04-25T10:11:33.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.044+0000 7f81ecdf3640 1 -- start start 2024-04-25T10:11:33.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.044+0000 7f81ecdf3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f81e8162450 0x7f81e806db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.044+0000 7f81ecdf3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e8162910 con 0x7f81e80b2f70 2024-04-25T10:11:33.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.044+0000 7f81ecdf3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e8162bf0 con 0x7f81e80aeaa0 2024-04-25T10:11:33.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.044+0000 7f81ecdf3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e8162ed0 con 0x7f81e8162450 2024-04-25T10:11:33.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.044+0000 7f81e5d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f81e8162450 0x7f81e806db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.044+0000 7f81e5d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f81e8162450 0x7f81e806db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37016/0 (socket says 172.21.15.60:37016) 2024-04-25T10:11:33.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.044+0000 7f81e5d9b640 1 -- 172.21.15.60:0/2508442189 learned_addr learned my addr 172.21.15.60:0/2508442189 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:33.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.044+0000 7f81e6d9d640 1 --1- 172.21.15.60:0/2508442189 >> v1:172.21.15.60:6789/0 conn(0x7f81e80b2f70 0x7f81e8071610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39488/0 (socket says 172.21.15.60:39488) 2024-04-25T10:11:33.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.048+0000 7f81e5d9b640 1 -- 172.21.15.60:0/2508442189 >> v1:172.21.15.100:6789/0 conn(0x7f81e80aeaa0 legacy=0x7f81e8166450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.048+0000 7f81e5d9b640 1 -- 172.21.15.60:0/2508442189 >> v1:172.21.15.60:6789/0 conn(0x7f81e80b2f70 legacy=0x7f81e8071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.048+0000 7f81e5d9b640 1 -- 172.21.15.60:0/2508442189 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e81631b0 con 0x7f81e8162450 2024-04-25T10:11:33.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.048+0000 7f81e5d9b640 1 --2- 172.21.15.60:0/2508442189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e8162450 0x7f81e806db70 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009870 tx=0x7f81dc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45671cfdd3d1c4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.048+0000 7f81e559a640 1 -- 172.21.15.60:0/2508442189 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc0ab610 con 0x7f81e8162450 2024-04-25T10:11:33.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.048+0000 7f81e559a640 1 -- 172.21.15.60:0/2508442189 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc0b0070 con 0x7f81e8162450 2024-04-25T10:11:33.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.048+0000 7f81e559a640 1 -- 172.21.15.60:0/2508442189 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc0ab9d0 con 0x7f81e8162450 2024-04-25T10:11:33.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.048+0000 7f81ecdf3640 1 -- 172.21.15.60:0/2508442189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e8162450 msgr2=0x7f81e806db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.048+0000 7f81ecdf3640 1 --2- 172.21.15.60:0/2508442189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e8162450 0x7f81e806db70 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009870 tx=0x7f81dc01c520 comp rx=0 tx=0).stop 2024-04-25T10:11:33.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.048+0000 7f81ecdf3640 1 -- 172.21.15.60:0/2508442189 shutdown_connections 2024-04-25T10:11:33.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.048+0000 7f81ecdf3640 1 --2- 172.21.15.60:0/2508442189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e8162450 0x7f81e806db70 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.048+0000 7f81ecdf3640 1 -- 172.21.15.60:0/2508442189 >> 172.21.15.60:0/2508442189 conn(0x7f81e80670b0 msgr2=0x7f81e806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:33.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.048+0000 7f81ecdf3640 1 -- 172.21.15.60:0/2508442189 shutdown_connections 2024-04-25T10:11:33.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.048+0000 7f81ecdf3640 1 -- 172.21.15.60:0/2508442189 wait complete. 2024-04-25T10:11:33.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81ecdf3640 1 Processor -- start 2024-04-25T10:11:33.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81ecdf3640 1 -- start start 2024-04-25T10:11:33.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81ecdf3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e80aeaa0 0x7f81e815aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81ecdf3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80b2f70 0x7f81e815b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81ecdf3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81e8162450 0x7f81e80b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81ecdf3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e80b3790 con 0x7f81e80b2f70 2024-04-25T10:11:33.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81ecdf3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e80aee80 con 0x7f81e8162450 2024-04-25T10:11:33.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81ecdf3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e8157eb0 con 0x7f81e80aeaa0 2024-04-25T10:11:33.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81e5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80b2f70 0x7f81e815b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81e5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80b2f70 0x7f81e815b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37028/0 (socket says 172.21.15.60:37028) 2024-04-25T10:11:33.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81e5d9b640 1 -- 172.21.15.60:0/343905477 learned_addr learned my addr 172.21.15.60:0/343905477 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:33.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81e5d9b640 1 -- 172.21.15.60:0/343905477 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e80aeaa0 msgr2=0x7f81e815aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81e5d9b640 1 --2- 172.21.15.60:0/343905477 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e80aeaa0 0x7f81e815aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81e5d9b640 1 -- 172.21.15.60:0/343905477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81e8162450 msgr2=0x7f81e80b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81e5d9b640 1 --2- 172.21.15.60:0/343905477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81e8162450 0x7f81e80b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81e5d9b640 1 -- 172.21.15.60:0/343905477 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e815c990 con 0x7f81e80b2f70 2024-04-25T10:11:33.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81e5d9b640 1 --2- 172.21.15.60:0/343905477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80b2f70 0x7f81e815b400 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0ad040 tx=0x7f81dc01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81d6ffd640 1 -- 172.21.15.60:0/343905477 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc0b4800 con 0x7f81e80b2f70 2024-04-25T10:11:33.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81d6ffd640 1 -- 172.21.15.60:0/343905477 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc0b49a0 con 0x7f81e80b2f70 2024-04-25T10:11:33.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81ecdf3640 1 -- 172.21.15.60:0/343905477 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e815cc70 con 0x7f81e80b2f70 2024-04-25T10:11:33.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81d6ffd640 1 -- 172.21.15.60:0/343905477 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc0b4d90 con 0x7f81e80b2f70 2024-04-25T10:11:33.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81ecdf3640 1 -- 172.21.15.60:0/343905477 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e815d230 con 0x7f81e80b2f70 2024-04-25T10:11:33.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81ecdf3640 1 -- 172.21.15.60:0/343905477 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e806bba0 con 0x7f81e80b2f70 2024-04-25T10:11:33.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81d6ffd640 1 -- 172.21.15.60:0/343905477 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f81dc0c7390 con 0x7f81e80b2f70 2024-04-25T10:11:33.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81d6ffd640 1 --2- 172.21.15.60:0/343905477 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81c8066f60 0x7f81c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.052+0000 7f81d6ffd640 1 -- 172.21.15.60:0/343905477 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f81dc11f970 con 0x7f81e80b2f70 2024-04-25T10:11:33.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.056+0000 7f81e659c640 1 --2- 172.21.15.60:0/343905477 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81c8066f60 0x7f81c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.056+0000 7f81e659c640 1 --2- 172.21.15.60:0/343905477 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81c8066f60 0x7f81c8069420 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f81e80b2be0 tx=0x7f81e0009230 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.056+0000 7f81d6ffd640 1 -- 172.21.15.60:0/343905477 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f81dc0c4060 con 0x7f81e80b2f70 2024-04-25T10:11:33.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.252+0000 7f81ecdf3640 1 -- 172.21.15.60:0/343905477 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f81e806bba0 con 0x7f81e80b2f70 2024-04-25T10:11:33.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.912+0000 7f81d6ffd640 1 -- 172.21.15.60:0/343905477 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f81dc0a8050 con 0x7f81e80b2f70 2024-04-25T10:11:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.916+0000 7f81ecdf3640 1 -- 172.21.15.60:0/343905477 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81c8066f60 msgr2=0x7f81c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.916+0000 7f81ecdf3640 1 --2- 172.21.15.60:0/343905477 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81c8066f60 0x7f81c8069420 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f81e80b2be0 tx=0x7f81e0009230 comp rx=0 tx=0).stop 2024-04-25T10:11:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.916+0000 7f81ecdf3640 1 -- 172.21.15.60:0/343905477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80b2f70 msgr2=0x7f81e815b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.916+0000 7f81ecdf3640 1 --2- 172.21.15.60:0/343905477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80b2f70 0x7f81e815b400 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0ad040 tx=0x7f81dc01ad60 comp rx=0 tx=0).stop 2024-04-25T10:11:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.916+0000 7f81ecdf3640 1 -- 172.21.15.60:0/343905477 shutdown_connections 2024-04-25T10:11:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.916+0000 7f81ecdf3640 1 --2- 172.21.15.60:0/343905477 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f81c8066f60 0x7f81c8069420 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.916+0000 7f81ecdf3640 1 --2- 172.21.15.60:0/343905477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81e8162450 0x7f81e80b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.916+0000 7f81ecdf3640 1 --2- 172.21.15.60:0/343905477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80b2f70 0x7f81e815b400 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.916+0000 7f81ecdf3640 1 --2- 172.21.15.60:0/343905477 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e80aeaa0 0x7f81e815aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.916+0000 7f81ecdf3640 1 -- 172.21.15.60:0/343905477 >> 172.21.15.60:0/343905477 conn(0x7f81e80670b0 msgr2=0x7f81e8071610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.916+0000 7f81ecdf3640 1 -- 172.21.15.60:0/343905477 shutdown_connections 2024-04-25T10:11:33.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:33.916+0000 7f81ecdf3640 1 -- 172.21.15.60:0/343905477 wait complete. 2024-04-25T10:11:33.949 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:11:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.116+0000 7f19edff7640 1 Processor -- start 2024-04-25T10:11:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.116+0000 7f19edff7640 1 -- start start 2024-04-25T10:11:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.120+0000 7f19edff7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f19e8071980 0x7f19e80ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.120+0000 7f19edff7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19e80cf0a0 0x7f19e80b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.120+0000 7f19edff7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e80cf810 con 0x7f19e8144e40 2024-04-25T10:11:34.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.120+0000 7f19edff7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e80b3060 con 0x7f19e8071980 2024-04-25T10:11:34.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.120+0000 7f19edff7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e80cfaf0 con 0x7f19e80cf0a0 2024-04-25T10:11:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.120+0000 7f19e6ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f19e8071980 0x7f19e80ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.120+0000 7f19e77fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f19e8144e40 0x7f19e8071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39502/0 (socket says 172.21.15.60:39502) 2024-04-25T10:11:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.120+0000 7f19e77fe640 1 -- 172.21.15.60:0/3849352813 learned_addr learned my addr 172.21.15.60:0/3849352813 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.120+0000 7f19e7fff640 1 --2- 172.21.15.60:0/3849352813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19e80cf0a0 0x7f19e80b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.120+0000 7f19e6ffd640 1 -- 172.21.15.60:0/3849352813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19e80cf0a0 msgr2=0x7f19e80b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.120+0000 7f19e6ffd640 1 --2- 172.21.15.60:0/3849352813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19e80cf0a0 0x7f19e80b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.120+0000 7f19e6ffd640 1 -- 172.21.15.60:0/3849352813 >> v1:172.21.15.60:6789/0 conn(0x7f19e8144e40 legacy=0x7f19e8071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.120+0000 7f19e6ffd640 1 -- 172.21.15.60:0/3849352813 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e80cfdd0 con 0x7f19e8071980 2024-04-25T10:11:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.120+0000 7f19e67fc640 1 -- 172.21.15.60:0/3849352813 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2092153143 0 0) 0x7f19e80cf810 con 0x7f19e8144e40 2024-04-25T10:11:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19e6ffd640 1 --2- 172.21.15.60:0/3849352813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e8071980 0x7f19e80ccc60 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0097c0 tx=0x7f19dc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3bb21beb6604a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19e67fc640 1 -- 172.21.15.60:0/3849352813 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc0aa5f0 con 0x7f19e8071980 2024-04-25T10:11:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19e67fc640 1 -- 172.21.15.60:0/3849352813 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc0af070 con 0x7f19e8071980 2024-04-25T10:11:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19e67fc640 1 -- 172.21.15.60:0/3849352813 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc0aa9b0 con 0x7f19e8071980 2024-04-25T10:11:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19edff7640 1 -- 172.21.15.60:0/3849352813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e8071980 msgr2=0x7f19e80ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19edff7640 1 --2- 172.21.15.60:0/3849352813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e8071980 0x7f19e80ccc60 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0097c0 tx=0x7f19dc01c470 comp rx=0 tx=0).stop 2024-04-25T10:11:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19edff7640 1 -- 172.21.15.60:0/3849352813 shutdown_connections 2024-04-25T10:11:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19edff7640 1 --2- 172.21.15.60:0/3849352813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19e80cf0a0 0x7f19e80b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19edff7640 1 --2- 172.21.15.60:0/3849352813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e8071980 0x7f19e80ccc60 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19edff7640 1 -- 172.21.15.60:0/3849352813 >> 172.21.15.60:0/3849352813 conn(0x7f19e8066fa0 msgr2=0x7f19e806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19edff7640 1 -- 172.21.15.60:0/3849352813 shutdown_connections 2024-04-25T10:11:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19edff7640 1 -- 172.21.15.60:0/3849352813 wait complete. 2024-04-25T10:11:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19edff7640 1 Processor -- start 2024-04-25T10:11:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19edff7640 1 -- start start 2024-04-25T10:11:34.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19edff7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e8071980 0x7f19e80cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19edff7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19e80cf0a0 0x7f19e813df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19edff7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e8144e40 0x7f19e80afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19edff7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e80b4650 con 0x7f19e8144e40 2024-04-25T10:11:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19edff7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e80b44d0 con 0x7f19e8071980 2024-04-25T10:11:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19edff7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e80b47d0 con 0x7f19e80cf0a0 2024-04-25T10:11:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e8144e40 0x7f19e80afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19e7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e8144e40 0x7f19e80afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37054/0 (socket says 172.21.15.60:37054) 2024-04-25T10:11:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19e7fff640 1 -- 172.21.15.60:0/2668047698 learned_addr learned my addr 172.21.15.60:0/2668047698 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19e6ffd640 1 --2- 172.21.15.60:0/2668047698 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19e80cf0a0 0x7f19e813df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19e77fe640 1 --2- 172.21.15.60:0/2668047698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e8071980 0x7f19e80cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19e7fff640 1 -- 172.21.15.60:0/2668047698 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19e80cf0a0 msgr2=0x7f19e813df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19e7fff640 1 --2- 172.21.15.60:0/2668047698 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19e80cf0a0 0x7f19e813df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19e7fff640 1 -- 172.21.15.60:0/2668047698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e8071980 msgr2=0x7f19e80cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19e7fff640 1 --2- 172.21.15.60:0/2668047698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e8071980 0x7f19e80cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19e7fff640 1 -- 172.21.15.60:0/2668047698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e813e4b0 con 0x7f19e8144e40 2024-04-25T10:11:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19e77fe640 1 --2- 172.21.15.60:0/2668047698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e8071980 0x7f19e80cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.124+0000 7f19e7fff640 1 --2- 172.21.15.60:0/2668047698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e8144e40 0x7f19e80afab0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f19e000efc0 tx=0x7f19e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.128+0000 7f19cffff640 1 -- 172.21.15.60:0/2668047698 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0018020 con 0x7f19e8144e40 2024-04-25T10:11:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.128+0000 7f19cffff640 1 -- 172.21.15.60:0/2668047698 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e0014480 con 0x7f19e8144e40 2024-04-25T10:11:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.128+0000 7f19edff7640 1 -- 172.21.15.60:0/2668047698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e815ce60 con 0x7f19e8144e40 2024-04-25T10:11:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.128+0000 7f19cffff640 1 -- 172.21.15.60:0/2668047698 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0014870 con 0x7f19e8144e40 2024-04-25T10:11:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.128+0000 7f19edff7640 1 -- 172.21.15.60:0/2668047698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e815a7e0 con 0x7f19e8144e40 2024-04-25T10:11:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.128+0000 7f19edff7640 1 -- 172.21.15.60:0/2668047698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e8074770 con 0x7f19e8144e40 2024-04-25T10:11:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.128+0000 7f19cffff640 1 -- 172.21.15.60:0/2668047698 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19e001b020 con 0x7f19e8144e40 2024-04-25T10:11:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.128+0000 7f19cffff640 1 --2- 172.21.15.60:0/2668047698 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f19d4066f60 0x7f19d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.128+0000 7f19cffff640 1 -- 172.21.15.60:0/2668047698 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f19e0085bf0 con 0x7f19e8144e40 2024-04-25T10:11:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.128+0000 7f19e77fe640 1 --2- 172.21.15.60:0/2668047698 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f19d4066f60 0x7f19d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.128+0000 7f19e77fe640 1 --2- 172.21.15.60:0/2668047698 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f19d4066f60 0x7f19d4069420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f19d8000f70 tx=0x7f19d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.132+0000 7f19cffff640 1 -- 172.21.15.60:0/2668047698 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f19e0022070 con 0x7f19e8144e40 2024-04-25T10:11:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.292+0000 7f19edff7640 1 -- 172.21.15.60:0/2668047698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f19e80cbce0 con 0x7f19e8144e40 2024-04-25T10:11:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.916+0000 7f19cffff640 1 -- 172.21.15.60:0/2668047698 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) v1 ==== 181+0+0 (secure 0 0 0) 0x7f19e0010040 con 0x7f19e8144e40 2024-04-25T10:11:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:11:34.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.920+0000 7f19edff7640 1 -- 172.21.15.60:0/2668047698 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f19d4066f60 msgr2=0x7f19d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.920+0000 7f19edff7640 1 --2- 172.21.15.60:0/2668047698 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f19d4066f60 0x7f19d4069420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f19d8000f70 tx=0x7f19d8009040 comp rx=0 tx=0).stop 2024-04-25T10:11:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.920+0000 7f19edff7640 1 -- 172.21.15.60:0/2668047698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e8144e40 msgr2=0x7f19e80afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.920+0000 7f19edff7640 1 --2- 172.21.15.60:0/2668047698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e8144e40 0x7f19e80afab0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f19e000efc0 tx=0x7f19e000c490 comp rx=0 tx=0).stop 2024-04-25T10:11:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.920+0000 7f19edff7640 1 -- 172.21.15.60:0/2668047698 shutdown_connections 2024-04-25T10:11:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.920+0000 7f19edff7640 1 --2- 172.21.15.60:0/2668047698 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f19d4066f60 0x7f19d4069420 secure :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f19d8000f70 tx=0x7f19d8009040 comp rx=0 tx=0).stop 2024-04-25T10:11:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.920+0000 7f19edff7640 1 --2- 172.21.15.60:0/2668047698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e8144e40 0x7f19e80afab0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.920+0000 7f19edff7640 1 --2- 172.21.15.60:0/2668047698 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19e80cf0a0 0x7f19e813df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.920+0000 7f19edff7640 1 --2- 172.21.15.60:0/2668047698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e8071980 0x7f19e80cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.920+0000 7f19edff7640 1 -- 172.21.15.60:0/2668047698 >> 172.21.15.60:0/2668047698 conn(0x7f19e8066fa0 msgr2=0x7f19e80af240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.928+0000 7f19edff7640 1 -- 172.21.15.60:0/2668047698 shutdown_connections 2024-04-25T10:11:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:34.928+0000 7f19edff7640 1 -- 172.21.15.60:0/2668047698 wait complete. 2024-04-25T10:11:34.961 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:11:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c42e63640 1 Processor -- start 2024-04-25T10:11:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c42e63640 1 -- start start 2024-04-25T10:11:35.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c42e63640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2c3c0b0b30 0x7f2c3c099e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c42e63640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c3c0cfa10 0x7f2c3c0cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c42e63640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c3c09a350 con 0x7f2c3c099a30 2024-04-25T10:11:35.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c42e63640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3c15ef50 con 0x7f2c3c0b0b30 2024-04-25T10:11:35.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c42e63640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c06bec0 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c3bfff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2c3c0b0b30 0x7f2c3c099e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c41400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c3c0cfa10 0x7f2c3c0cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c3bfff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2c3c0b0b30 0x7f2c3c099e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:52456/0 (socket says 172.21.15.60:52456) 2024-04-25T10:11:35.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c40bff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2c3c099a30 0x7f2c3c161fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:33284/0 (socket says 172.21.15.60:33284) 2024-04-25T10:11:35.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c3bfff640 1 -- 172.21.15.60:0/1518926768 learned_addr learned my addr 172.21.15.60:0/1518926768 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:35.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c41400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c3c0cfa10 0x7f2c3c0cfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:48748/0 (socket says 172.21.15.60:48748) 2024-04-25T10:11:35.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c41400640 1 -- 172.21.15.60:0/1518926768 >> v2:172.21.15.100:3300/0 conn(0x7f2c3c0b0b30 msgr2=0x7f2c3c099e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:35.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c41400640 1 --2- 172.21.15.60:0/1518926768 >> v2:172.21.15.100:3300/0 conn(0x7f2c3c0b0b30 0x7f2c3c099e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c41400640 1 -- 172.21.15.60:0/1518926768 >> v1:172.21.15.100:6789/0 conn(0x7f2c3c099a30 legacy=0x7f2c3c161fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c41400640 1 -- 172.21.15.60:0/1518926768 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c06c1a0 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.092+0000 7f2c3bfff640 1 --2- 172.21.15.60:0/1518926768 >> v2:172.21.15.100:3300/0 conn(0x7f2c3c0b0b30 0x7f2c3c099e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:35.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.096+0000 7f2c41400640 1 --2- 172.21.15.60:0/1518926768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c3c0cfa10 0x7f2c3c0cfec0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009d90 tx=0x7f2c3401c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1cd3ee2ec8620c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:35.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.096+0000 7f2c3b7fe640 1 -- 172.21.15.60:0/1518926768 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c340b2070 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.096+0000 7f2c3b7fe640 1 -- 172.21.15.60:0/1518926768 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c340a5940 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.096+0000 7f2c3b7fe640 1 -- 172.21.15.60:0/1518926768 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c340a5d10 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.096+0000 7f2c42e63640 1 -- 172.21.15.60:0/1518926768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c3c0cfa10 msgr2=0x7f2c3c0cfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.096+0000 7f2c42e63640 1 --2- 172.21.15.60:0/1518926768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c3c0cfa10 0x7f2c3c0cfec0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009d90 tx=0x7f2c3401c880 comp rx=0 tx=0).stop 2024-04-25T10:11:35.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.096+0000 7f2c42e63640 1 -- 172.21.15.60:0/1518926768 shutdown_connections 2024-04-25T10:11:35.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.096+0000 7f2c42e63640 1 --2- 172.21.15.60:0/1518926768 >> v2:172.21.15.100:3300/0 conn(0x7f2c3c0b0b30 0x7f2c3c099e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.096+0000 7f2c42e63640 1 --2- 172.21.15.60:0/1518926768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c3c0cfa10 0x7f2c3c0cfec0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.096+0000 7f2c42e63640 1 -- 172.21.15.60:0/1518926768 >> 172.21.15.60:0/1518926768 conn(0x7f2c3c0d17d0 msgr2=0x7f2c3c0d1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:35.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.096+0000 7f2c42e63640 1 -- 172.21.15.60:0/1518926768 shutdown_connections 2024-04-25T10:11:35.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.096+0000 7f2c42e63640 1 -- 172.21.15.60:0/1518926768 wait complete. 2024-04-25T10:11:35.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c42e63640 1 Processor -- start 2024-04-25T10:11:35.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c42e63640 1 -- start start 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c42e63640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c3c099a30 0x7f2c3c1556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c42e63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c3c0b0b30 0x7f2c3c156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c42e63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c3c0cfa10 0x7f2c3c15f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c42e63640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c16a970 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c42e63640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c06cf60 con 0x7f2c3c0b0b30 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c42e63640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c16aaf0 con 0x7f2c3c099a30 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c41400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c3c0cfa10 0x7f2c3c15f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c41400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c3c0cfa10 0x7f2c3c15f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37068/0 (socket says 172.21.15.60:37068) 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c41400640 1 -- 172.21.15.60:0/2988585465 learned_addr learned my addr 172.21.15.60:0/2988585465 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c40bff640 1 --2- 172.21.15.60:0/2988585465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c3c099a30 0x7f2c3c1556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c3bfff640 1 --2- 172.21.15.60:0/2988585465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c3c0b0b30 0x7f2c3c156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c41400640 1 -- 172.21.15.60:0/2988585465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c3c099a30 msgr2=0x7f2c3c1556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c41400640 1 --2- 172.21.15.60:0/2988585465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c3c099a30 0x7f2c3c1556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c41400640 1 -- 172.21.15.60:0/2988585465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c3c0b0b30 msgr2=0x7f2c3c156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c41400640 1 --2- 172.21.15.60:0/2988585465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c3c0b0b30 0x7f2c3c156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c41400640 1 -- 172.21.15.60:0/2988585465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c1605c0 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c41400640 1 --2- 172.21.15.60:0/2988585465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c3c0cfa10 0x7f2c3c15f410 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f2c3401a4b0 tx=0x7f2c34019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c397fa640 1 -- 172.21.15.60:0/2988585465 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c340b2050 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c397fa640 1 -- 172.21.15.60:0/2988585465 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c340be070 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c42e63640 1 -- 172.21.15.60:0/2988585465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3c160860 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c397fa640 1 -- 172.21.15.60:0/2988585465 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c340ada90 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.100+0000 7f2c42e63640 1 -- 172.21.15.60:0/2988585465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3c15f950 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.104+0000 7f2c42e63640 1 -- 172.21.15.60:0/2988585465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3c074770 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.104+0000 7f2c397fa640 1 -- 172.21.15.60:0/2988585465 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c340adc30 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.104+0000 7f2c397fa640 1 --2- 172.21.15.60:0/2988585465 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2c1c066f60 0x7f2c1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.104+0000 7f2c397fa640 1 -- 172.21.15.60:0/2988585465 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f2c3411ff50 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.104+0000 7f2c40bff640 1 --2- 172.21.15.60:0/2988585465 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2c1c066f60 0x7f2c1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.104+0000 7f2c40bff640 1 --2- 172.21.15.60:0/2988585465 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2c1c066f60 0x7f2c1c069420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c000f70 tx=0x7f2c2c007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:35.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.108+0000 7f2c397fa640 1 -- 172.21.15.60:0/2988585465 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c34016560 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.268+0000 7f2c42e63640 1 -- 172.21.15.60:0/2988585465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2c3c0d2600 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.932+0000 7f2c397fa640 1 -- 172.21.15.60:0/2988585465 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2c340f09d0 con 0x7f2c3c0cfa10 2024-04-25T10:11:35.941 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T10:11:35.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.936+0000 7f2c42e63640 1 -- 172.21.15.60:0/2988585465 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2c1c066f60 msgr2=0x7f2c1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.936+0000 7f2c42e63640 1 --2- 172.21.15.60:0/2988585465 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2c1c066f60 0x7f2c1c069420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c000f70 tx=0x7f2c2c007e90 comp rx=0 tx=0).stop 2024-04-25T10:11:35.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.936+0000 7f2c42e63640 1 -- 172.21.15.60:0/2988585465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c3c0cfa10 msgr2=0x7f2c3c15f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.936+0000 7f2c42e63640 1 --2- 172.21.15.60:0/2988585465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c3c0cfa10 0x7f2c3c15f410 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f2c3401a4b0 tx=0x7f2c34019620 comp rx=0 tx=0).stop 2024-04-25T10:11:35.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.940+0000 7f2c42e63640 1 -- 172.21.15.60:0/2988585465 shutdown_connections 2024-04-25T10:11:35.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.940+0000 7f2c42e63640 1 --2- 172.21.15.60:0/2988585465 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2c1c066f60 0x7f2c1c069420 secure :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c000f70 tx=0x7f2c2c007e90 comp rx=0 tx=0).stop 2024-04-25T10:11:35.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.940+0000 7f2c42e63640 1 --2- 172.21.15.60:0/2988585465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c3c0cfa10 0x7f2c3c15f410 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.940+0000 7f2c42e63640 1 --2- 172.21.15.60:0/2988585465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c3c0b0b30 0x7f2c3c156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.940+0000 7f2c42e63640 1 --2- 172.21.15.60:0/2988585465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c3c099a30 0x7f2c3c1556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.940+0000 7f2c42e63640 1 -- 172.21.15.60:0/2988585465 >> 172.21.15.60:0/2988585465 conn(0x7f2c3c0d17d0 msgr2=0x7f2c3c0b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:35.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.940+0000 7f2c42e63640 1 -- 172.21.15.60:0/2988585465 shutdown_connections 2024-04-25T10:11:35.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:35.940+0000 7f2c42e63640 1 -- 172.21.15.60:0/2988585465 wait complete. 2024-04-25T10:11:35.976 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:11:36.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f64a7088640 1 Processor -- start 2024-04-25T10:11:36.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f64a7088640 1 -- start start 2024-04-25T10:11:36.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f64a7088640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64a0160080 0x7f64a015f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f64a7088640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64a015fa20 0x7f64a00ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f64a7088640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a0144a40 con 0x7f64a00cbcf0 2024-04-25T10:11:36.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f64a7088640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a00b2dc0 con 0x7f64a015fa20 2024-04-25T10:11:36.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f64a7088640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00cc9f0 con 0x7f64a0160080 2024-04-25T10:11:36.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f64a4e24640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f64a00cbcf0 0x7f64a0071c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39508/0 (socket says 172.21.15.60:39508) 2024-04-25T10:11:36.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f64a4e24640 1 -- 172.21.15.60:0/629952212 learned_addr learned my addr 172.21.15.60:0/629952212 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:36.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f649ffff640 1 --2- 172.21.15.60:0/629952212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64a0160080 0x7f64a015f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f64a5625640 1 --2- 172.21.15.60:0/629952212 >> v2:172.21.15.60:3300/0 conn(0x7f64a015fa20 0x7f64a00ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f649f7fe640 1 -- 172.21.15.60:0/629952212 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331510777 0 0) 0x7f64a0144a40 con 0x7f64a00cbcf0 2024-04-25T10:11:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f649f7fe640 1 -- 172.21.15.60:0/629952212 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6488003530 con 0x7f64a00cbcf0 2024-04-25T10:11:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f64a5625640 1 -- 172.21.15.60:0/629952212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64a0160080 msgr2=0x7f64a015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f64a5625640 1 --2- 172.21.15.60:0/629952212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64a0160080 0x7f64a015f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f64a5625640 1 -- 172.21.15.60:0/629952212 >> v1:172.21.15.60:6789/0 conn(0x7f64a00cbcf0 legacy=0x7f64a0071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f649f7fe640 1 -- 172.21.15.60:0/629952212 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3842829069 0 0) 0x7f6488003530 con 0x7f64a00cbcf0 2024-04-25T10:11:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.088+0000 7f64a5625640 1 -- 172.21.15.60:0/629952212 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a00cccd0 con 0x7f64a015fa20 2024-04-25T10:11:36.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.092+0000 7f64a5625640 1 --2- 172.21.15.60:0/629952212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64a015fa20 0x7f64a00ceec0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f64980097c0 tx=0x7f649801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315392c28642a99e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:36.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.096+0000 7f649f7fe640 1 -- 172.21.15.60:0/629952212 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64980b0070 con 0x7f64a015fa20 2024-04-25T10:11:36.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.096+0000 7f649f7fe640 1 -- 172.21.15.60:0/629952212 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64980ab400 con 0x7f64a015fa20 2024-04-25T10:11:36.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.096+0000 7f649f7fe640 1 -- 172.21.15.60:0/629952212 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490001a00 con 0x7f64a015fa20 2024-04-25T10:11:36.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.096+0000 7f64a7088640 1 -- 172.21.15.60:0/629952212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64a015fa20 msgr2=0x7f64a00ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.096+0000 7f64a7088640 1 --2- 172.21.15.60:0/629952212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64a015fa20 0x7f64a00ceec0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f64980097c0 tx=0x7f649801c3c0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.096+0000 7f64a7088640 1 -- 172.21.15.60:0/629952212 shutdown_connections 2024-04-25T10:11:36.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.096+0000 7f64a7088640 1 --2- 172.21.15.60:0/629952212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64a015fa20 0x7f64a00ceec0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.096+0000 7f64a7088640 1 --2- 172.21.15.60:0/629952212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64a0160080 0x7f64a015f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.096+0000 7f64a7088640 1 -- 172.21.15.60:0/629952212 >> 172.21.15.60:0/629952212 conn(0x7f64a00670b0 msgr2=0x7f64a006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:36.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.096+0000 7f64a7088640 1 -- 172.21.15.60:0/629952212 shutdown_connections 2024-04-25T10:11:36.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.100+0000 7f64a7088640 1 -- 172.21.15.60:0/629952212 wait complete. 2024-04-25T10:11:36.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.100+0000 7f64a7088640 1 Processor -- start 2024-04-25T10:11:36.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.100+0000 7f64a7088640 1 -- start start 2024-04-25T10:11:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.100+0000 7f64a7088640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a00cbcf0 0x7f64a01423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.100+0000 7f64a7088640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64a015fa20 0x7f64a00a92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.100+0000 7f64a7088640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64a0160080 0x7f64a00a9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.100+0000 7f64a7088640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00ce740 con 0x7f64a0160080 2024-04-25T10:11:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.100+0000 7f64a7088640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00ce5c0 con 0x7f64a00cbcf0 2024-04-25T10:11:36.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.100+0000 7f64a7088640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00cfc10 con 0x7f64a015fa20 2024-04-25T10:11:36.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.104+0000 7f64a5625640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64a0160080 0x7f64a00a9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.104+0000 7f64a5625640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64a0160080 0x7f64a00a9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37086/0 (socket says 172.21.15.60:37086) 2024-04-25T10:11:36.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.104+0000 7f64a5625640 1 -- 172.21.15.60:0/1057252870 learned_addr learned my addr 172.21.15.60:0/1057252870 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:36.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.104+0000 7f64a5625640 1 -- 172.21.15.60:0/1057252870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64a015fa20 msgr2=0x7f64a00a92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.104+0000 7f64a5625640 1 --2- 172.21.15.60:0/1057252870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64a015fa20 0x7f64a00a92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.104+0000 7f64a5625640 1 -- 172.21.15.60:0/1057252870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a00cbcf0 msgr2=0x7f64a01423f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.104+0000 7f64a5625640 1 --2- 172.21.15.60:0/1057252870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a00cbcf0 0x7f64a01423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.104+0000 7f64a5625640 1 -- 172.21.15.60:0/1057252870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a0142b60 con 0x7f64a0160080 2024-04-25T10:11:36.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.104+0000 7f64a5625640 1 --2- 172.21.15.60:0/1057252870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64a0160080 0x7f64a00a9820 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f649801aa70 tx=0x7f6498019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:36.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.104+0000 7f649d7fa640 1 -- 172.21.15.60:0/1057252870 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64980b0040 con 0x7f64a0160080 2024-04-25T10:11:36.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.104+0000 7f649d7fa640 1 -- 172.21.15.60:0/1057252870 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64980bc070 con 0x7f64a0160080 2024-04-25T10:11:36.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.104+0000 7f649d7fa640 1 -- 172.21.15.60:0/1057252870 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64980b5680 con 0x7f64a0160080 2024-04-25T10:11:36.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.104+0000 7f64a7088640 1 -- 172.21.15.60:0/1057252870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a0142e40 con 0x7f64a0160080 2024-04-25T10:11:36.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.104+0000 7f64a7088640 1 -- 172.21.15.60:0/1057252870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a0143400 con 0x7f64a0160080 2024-04-25T10:11:36.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.108+0000 7f649d7fa640 1 -- 172.21.15.60:0/1057252870 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64980b3070 con 0x7f64a0160080 2024-04-25T10:11:36.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.108+0000 7f64a7088640 1 -- 172.21.15.60:0/1057252870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a006bba0 con 0x7f64a0160080 2024-04-25T10:11:36.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.108+0000 7f649d7fa640 1 --2- 172.21.15.60:0/1057252870 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f648c066f60 0x7f648c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.108+0000 7f649d7fa640 1 -- 172.21.15.60:0/1057252870 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f649811eb40 con 0x7f64a0160080 2024-04-25T10:11:36.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.108+0000 7f64a4e24640 1 --2- 172.21.15.60:0/1057252870 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f648c066f60 0x7f648c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.108+0000 7f64a4e24640 1 --2- 172.21.15.60:0/1057252870 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f648c066f60 0x7f648c069420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6490000f90 tx=0x7f6490001530 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.112+0000 7f649d7fa640 1 -- 172.21.15.60:0/1057252870 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6498015fb0 con 0x7f64a0160080 2024-04-25T10:11:36.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.272+0000 7f64a7088640 1 -- 172.21.15.60:0/1057252870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f64a0074770 con 0x7f64a0160080 2024-04-25T10:11:36.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.276+0000 7f649d7fa640 1 -- 172.21.15.60:0/1057252870 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f64980efdf0 con 0x7f64a0160080 2024-04-25T10:11:36.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.280+0000 7f64a7088640 1 -- 172.21.15.60:0/1057252870 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f648c066f60 msgr2=0x7f648c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.280+0000 7f64a7088640 1 --2- 172.21.15.60:0/1057252870 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f648c066f60 0x7f648c069420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6490000f90 tx=0x7f6490001530 comp rx=0 tx=0).stop 2024-04-25T10:11:36.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.280+0000 7f64a7088640 1 -- 172.21.15.60:0/1057252870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64a0160080 msgr2=0x7f64a00a9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.280+0000 7f64a7088640 1 --2- 172.21.15.60:0/1057252870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64a0160080 0x7f64a00a9820 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f649801aa70 tx=0x7f6498019160 comp rx=0 tx=0).stop 2024-04-25T10:11:36.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.280+0000 7f64a7088640 1 -- 172.21.15.60:0/1057252870 shutdown_connections 2024-04-25T10:11:36.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.280+0000 7f64a7088640 1 --2- 172.21.15.60:0/1057252870 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f648c066f60 0x7f648c069420 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.280+0000 7f64a7088640 1 --2- 172.21.15.60:0/1057252870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64a0160080 0x7f64a00a9820 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.280+0000 7f64a7088640 1 --2- 172.21.15.60:0/1057252870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64a015fa20 0x7f64a00a92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.280+0000 7f64a7088640 1 --2- 172.21.15.60:0/1057252870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64a00cbcf0 0x7f64a01423f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.280+0000 7f64a7088640 1 -- 172.21.15.60:0/1057252870 >> 172.21.15.60:0/1057252870 conn(0x7f64a00670b0 msgr2=0x7f64a015d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:36.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.280+0000 7f64a7088640 1 -- 172.21.15.60:0/1057252870 shutdown_connections 2024-04-25T10:11:36.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.280+0000 7f64a7088640 1 -- 172.21.15.60:0/1057252870 wait complete. 2024-04-25T10:11:36.306 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-25T10:11:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d4792640 1 Processor -- start 2024-04-25T10:11:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d4792640 1 -- start start 2024-04-25T10:11:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d4792640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83cc145290 0x7f83cc0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d4792640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83cc070cf0 0x7f83cc0b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d4792640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f83cc0b6340 0x7f83cc0d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d4792640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83cc0b5970 con 0x7f83cc145290 2024-04-25T10:11:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d4792640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83cc071ea0 con 0x7f83cc0b6340 2024-04-25T10:11:36.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d4792640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc0b4d60 con 0x7f83cc070cf0 2024-04-25T10:11:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d252e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83cc145290 0x7f83cc0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d252e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83cc145290 0x7f83cc0b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37096/0 (socket says 172.21.15.60:37096) 2024-04-25T10:11:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d252e640 1 -- 172.21.15.60:0/1803435939 learned_addr learned my addr 172.21.15.60:0/1803435939 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d2d2f640 1 --2- 172.21.15.60:0/1803435939 >> v2:172.21.15.100:3300/0 conn(0x7f83cc0b6340 0x7f83cc0d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d1d2d640 1 --2- 172.21.15.60:0/1803435939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83cc070cf0 0x7f83cc0b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d252e640 1 -- 172.21.15.60:0/1803435939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83cc070cf0 msgr2=0x7f83cc0b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d252e640 1 --2- 172.21.15.60:0/1803435939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83cc070cf0 0x7f83cc0b5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d252e640 1 -- 172.21.15.60:0/1803435939 >> v2:172.21.15.100:3300/0 conn(0x7f83cc0b6340 msgr2=0x7f83cc0d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d252e640 1 --2- 172.21.15.60:0/1803435939 >> v2:172.21.15.100:3300/0 conn(0x7f83cc0b6340 0x7f83cc0d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d252e640 1 -- 172.21.15.60:0/1803435939 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83cc0b5040 con 0x7f83cc145290 2024-04-25T10:11:36.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d2d2f640 1 --2- 172.21.15.60:0/1803435939 >> v2:172.21.15.100:3300/0 conn(0x7f83cc0b6340 0x7f83cc0d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:36.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.424+0000 7f83d1d2d640 1 --2- 172.21.15.60:0/1803435939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83cc070cf0 0x7f83cc0b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.428+0000 7f83d252e640 1 --2- 172.21.15.60:0/1803435939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83cc145290 0x7f83cc0b46d0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f83cc06bb50 tx=0x7f83c401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f517223daf524d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.428+0000 7f83d152c640 1 -- 172.21.15.60:0/1803435939 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c40b4070 con 0x7f83cc145290 2024-04-25T10:11:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.428+0000 7f83d152c640 1 -- 172.21.15.60:0/1803435939 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c40af400 con 0x7f83cc145290 2024-04-25T10:11:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.428+0000 7f83d152c640 1 -- 172.21.15.60:0/1803435939 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c40af7d0 con 0x7f83cc145290 2024-04-25T10:11:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.428+0000 7f83d4792640 1 -- 172.21.15.60:0/1803435939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83cc145290 msgr2=0x7f83cc0b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.428+0000 7f83d4792640 1 --2- 172.21.15.60:0/1803435939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83cc145290 0x7f83cc0b46d0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f83cc06bb50 tx=0x7f83c401f1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.428+0000 7f83d4792640 1 -- 172.21.15.60:0/1803435939 shutdown_connections 2024-04-25T10:11:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.428+0000 7f83d4792640 1 --2- 172.21.15.60:0/1803435939 >> v2:172.21.15.100:3300/0 conn(0x7f83cc0b6340 0x7f83cc0d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.428+0000 7f83d4792640 1 --2- 172.21.15.60:0/1803435939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83cc070cf0 0x7f83cc0b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.428+0000 7f83d4792640 1 --2- 172.21.15.60:0/1803435939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83cc145290 0x7f83cc0b46d0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.428+0000 7f83d4792640 1 -- 172.21.15.60:0/1803435939 >> 172.21.15.60:0/1803435939 conn(0x7f83cc06f550 msgr2=0x7f83cc06f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.428+0000 7f83d4792640 1 -- 172.21.15.60:0/1803435939 shutdown_connections 2024-04-25T10:11:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.428+0000 7f83d4792640 1 -- 172.21.15.60:0/1803435939 wait complete. 2024-04-25T10:11:36.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d4792640 1 Processor -- start 2024-04-25T10:11:36.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d4792640 1 -- start start 2024-04-25T10:11:36.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d4792640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83cc070cf0 0x7f83cc13a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d4792640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83cc0b6340 0x7f83cc13ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d4792640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83cc145290 0x7f83cc13ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d4792640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc145670 con 0x7f83cc070cf0 2024-04-25T10:11:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d4792640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc0b5600 con 0x7f83cc0b6340 2024-04-25T10:11:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d4792640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc0d0d50 con 0x7f83cc145290 2024-04-25T10:11:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d252e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83cc070cf0 0x7f83cc13a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d252e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83cc070cf0 0x7f83cc13a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37104/0 (socket says 172.21.15.60:37104) 2024-04-25T10:11:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d252e640 1 -- 172.21.15.60:0/933164850 learned_addr learned my addr 172.21.15.60:0/933164850 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d1d2d640 1 --2- 172.21.15.60:0/933164850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83cc0b6340 0x7f83cc13ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d2d2f640 1 --2- 172.21.15.60:0/933164850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83cc145290 0x7f83cc13ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d252e640 1 -- 172.21.15.60:0/933164850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83cc145290 msgr2=0x7f83cc13ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d252e640 1 --2- 172.21.15.60:0/933164850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83cc145290 0x7f83cc13ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d252e640 1 -- 172.21.15.60:0/933164850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83cc0b6340 msgr2=0x7f83cc13ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d252e640 1 --2- 172.21.15.60:0/933164850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83cc0b6340 0x7f83cc13ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d252e640 1 -- 172.21.15.60:0/933164850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83cc1f58a0 con 0x7f83cc070cf0 2024-04-25T10:11:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d2d2f640 1 --2- 172.21.15.60:0/933164850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83cc145290 0x7f83cc13ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d252e640 1 --2- 172.21.15.60:0/933164850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83cc070cf0 0x7f83cc13a130 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f83c401d890 tx=0x7f83c401bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83caffd640 1 -- 172.21.15.60:0/933164850 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c40b4040 con 0x7f83cc070cf0 2024-04-25T10:11:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83caffd640 1 -- 172.21.15.60:0/933164850 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c40c0070 con 0x7f83cc070cf0 2024-04-25T10:11:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d4792640 1 -- 172.21.15.60:0/933164850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83cc1f5b80 con 0x7f83cc070cf0 2024-04-25T10:11:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83caffd640 1 -- 172.21.15.60:0/933164850 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4009510 con 0x7f83cc070cf0 2024-04-25T10:11:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d4792640 1 -- 172.21.15.60:0/933164850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83cc1f6140 con 0x7f83cc070cf0 2024-04-25T10:11:36.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83d4792640 1 -- 172.21.15.60:0/933164850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83cc074770 con 0x7f83cc070cf0 2024-04-25T10:11:36.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.432+0000 7f83caffd640 1 -- 172.21.15.60:0/933164850 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83c40ba660 con 0x7f83cc070cf0 2024-04-25T10:11:36.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.436+0000 7f83caffd640 1 --2- 172.21.15.60:0/933164850 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83b8066f60 0x7f83b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.436+0000 7f83caffd640 1 -- 172.21.15.60:0/933164850 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f83c41231c0 con 0x7f83cc070cf0 2024-04-25T10:11:36.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.436+0000 7f83d1d2d640 1 --2- 172.21.15.60:0/933164850 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83b8066f60 0x7f83b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.436+0000 7f83d1d2d640 1 --2- 172.21.15.60:0/933164850 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83b8066f60 0x7f83b8069420 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f83cc13d440 tx=0x7f83c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:36.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.440+0000 7f83caffd640 1 -- 172.21.15.60:0/933164850 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83c4018d40 con 0x7f83cc070cf0 2024-04-25T10:11:36.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.652+0000 7f83d4792640 1 -- 172.21.15.60:0/933164850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7f83cc0701f0 con 0x7f83cc070cf0 2024-04-25T10:11:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.936+0000 7f83caffd640 1 -- 172.21.15.60:0/933164850 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v303) v1 ==== 166+0+0 (secure 0 0 0) 0x7f83c40f4500 con 0x7f83cc070cf0 2024-04-25T10:11:36.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.940+0000 7f83d4792640 1 -- 172.21.15.60:0/933164850 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83b8066f60 msgr2=0x7f83b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.940+0000 7f83d4792640 1 --2- 172.21.15.60:0/933164850 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83b8066f60 0x7f83b8069420 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f83cc13d440 tx=0x7f83c0008040 comp rx=0 tx=0).stop 2024-04-25T10:11:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.940+0000 7f83d4792640 1 -- 172.21.15.60:0/933164850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83cc070cf0 msgr2=0x7f83cc13a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.940+0000 7f83d4792640 1 --2- 172.21.15.60:0/933164850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83cc070cf0 0x7f83cc13a130 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f83c401d890 tx=0x7f83c401bf80 comp rx=0 tx=0).stop 2024-04-25T10:11:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.940+0000 7f83d4792640 1 -- 172.21.15.60:0/933164850 shutdown_connections 2024-04-25T10:11:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.940+0000 7f83d4792640 1 --2- 172.21.15.60:0/933164850 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83b8066f60 0x7f83b8069420 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.940+0000 7f83d4792640 1 --2- 172.21.15.60:0/933164850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83cc145290 0x7f83cc13ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.940+0000 7f83d4792640 1 --2- 172.21.15.60:0/933164850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83cc0b6340 0x7f83cc13ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.940+0000 7f83d4792640 1 --2- 172.21.15.60:0/933164850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83cc070cf0 0x7f83cc13a130 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.940+0000 7f83d4792640 1 -- 172.21.15.60:0/933164850 >> 172.21.15.60:0/933164850 conn(0x7f83cc06f550 msgr2=0x7f83cc143080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:36.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.940+0000 7f83d4792640 1 -- 172.21.15.60:0/933164850 shutdown_connections 2024-04-25T10:11:36.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:36.940+0000 7f83d4792640 1 -- 172.21.15.60:0/933164850 wait complete. 2024-04-25T10:11:36.965 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T10:11:36.965 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T10:11:36.965 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:11:36.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.084+0000 7f92861ec640 1 Processor -- start 2024-04-25T10:11:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.084+0000 7f92861ec640 1 -- start start 2024-04-25T10:11:37.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f92861ec640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928015bc90 0x7f928015c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f92861ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f92800b4660 0x7f928015c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f92861ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9280071ca0 con 0x7f92800ce7a0 2024-04-25T10:11:37.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f92861ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92800cc370 con 0x7f92800b4660 2024-04-25T10:11:37.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f92861ec640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92800cca40 con 0x7f928015bc90 2024-04-25T10:11:37.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f927ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f92800b4660 0x7f928015c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f927ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f92800b4660 0x7f928015c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37120/0 (socket says 172.21.15.60:37120) 2024-04-25T10:11:37.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f927f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f92800ce7a0 0x7f92800ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39510/0 (socket says 172.21.15.60:39510) 2024-04-25T10:11:37.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f927ffff640 1 -- 172.21.15.60:0/3058778387 learned_addr learned my addr 172.21.15.60:0/3058778387 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:37.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f927effd640 1 --2- 172.21.15.60:0/3058778387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928015bc90 0x7f928015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f927ffff640 1 -- 172.21.15.60:0/3058778387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928015bc90 msgr2=0x7f928015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f927ffff640 1 --2- 172.21.15.60:0/3058778387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928015bc90 0x7f928015c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f927ffff640 1 -- 172.21.15.60:0/3058778387 >> v1:172.21.15.60:6789/0 conn(0x7f92800ce7a0 legacy=0x7f92800ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f927ffff640 1 -- 172.21.15.60:0/3058778387 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92800ccd20 con 0x7f92800b4660 2024-04-25T10:11:37.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f927effd640 1 --2- 172.21.15.60:0/3058778387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928015bc90 0x7f928015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:37.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f927ffff640 1 --2- 172.21.15.60:0/3058778387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92800b4660 0x7f928015c760 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f9278009e50 tx=0x7f927801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7d1cdfa1f41959 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f927e7fc640 1 -- 172.21.15.60:0/3058778387 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92780a5700 con 0x7f92800b4660 2024-04-25T10:11:37.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f927e7fc640 1 -- 172.21.15.60:0/3058778387 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92780a58a0 con 0x7f92800b4660 2024-04-25T10:11:37.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.088+0000 7f927e7fc640 1 -- 172.21.15.60:0/3058778387 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92780a5c70 con 0x7f92800b4660 2024-04-25T10:11:37.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f92861ec640 1 -- 172.21.15.60:0/3058778387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92800b4660 msgr2=0x7f928015c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f92861ec640 1 --2- 172.21.15.60:0/3058778387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92800b4660 0x7f928015c760 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f9278009e50 tx=0x7f927801ca30 comp rx=0 tx=0).stop 2024-04-25T10:11:37.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f92861ec640 1 -- 172.21.15.60:0/3058778387 shutdown_connections 2024-04-25T10:11:37.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f92861ec640 1 --2- 172.21.15.60:0/3058778387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92800b4660 0x7f928015c760 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f92861ec640 1 --2- 172.21.15.60:0/3058778387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f928015bc90 0x7f928015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f92861ec640 1 -- 172.21.15.60:0/3058778387 >> 172.21.15.60:0/3058778387 conn(0x7f92800b12d0 msgr2=0x7f92800b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:37.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f92861ec640 1 -- 172.21.15.60:0/3058778387 shutdown_connections 2024-04-25T10:11:37.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f92861ec640 1 -- 172.21.15.60:0/3058778387 wait complete. 2024-04-25T10:11:37.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f92861ec640 1 Processor -- start 2024-04-25T10:11:37.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f92861ec640 1 -- start start 2024-04-25T10:11:37.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f92861ec640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92800ce7a0 0x7f92800ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f92861ec640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928015bc90 0x7f9280155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f92861ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9280148f40 0x7f92800b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f92861ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92800b3db0 con 0x7f9280148f40 2024-04-25T10:11:37.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f92861ec640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92800b4490 con 0x7f928015bc90 2024-04-25T10:11:37.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f92861ec640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92801685c0 con 0x7f92800ce7a0 2024-04-25T10:11:37.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f927ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9280148f40 0x7f92800b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f927ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9280148f40 0x7f92800b2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37128/0 (socket says 172.21.15.60:37128) 2024-04-25T10:11:37.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f927ffff640 1 -- 172.21.15.60:0/1795088302 learned_addr learned my addr 172.21.15.60:0/1795088302 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:37.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f927effd640 1 --2- 172.21.15.60:0/1795088302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928015bc90 0x7f9280155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.092+0000 7f927f7fe640 1 --2- 172.21.15.60:0/1795088302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92800ce7a0 0x7f92800ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.096+0000 7f927ffff640 1 -- 172.21.15.60:0/1795088302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92800ce7a0 msgr2=0x7f92800ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.096+0000 7f927ffff640 1 --2- 172.21.15.60:0/1795088302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92800ce7a0 0x7f92800ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.096+0000 7f927ffff640 1 -- 172.21.15.60:0/1795088302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928015bc90 msgr2=0x7f9280155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.096+0000 7f927ffff640 1 --2- 172.21.15.60:0/1795088302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928015bc90 0x7f9280155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.096+0000 7f927ffff640 1 -- 172.21.15.60:0/1795088302 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92801578b0 con 0x7f9280148f40 2024-04-25T10:11:37.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.096+0000 7f927ffff640 1 --2- 172.21.15.60:0/1795088302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9280148f40 0x7f92800b2150 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f92780af040 tx=0x7f92780197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.096+0000 7f9267fff640 1 -- 172.21.15.60:0/1795088302 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92780b6680 con 0x7f9280148f40 2024-04-25T10:11:37.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.096+0000 7f9267fff640 1 -- 172.21.15.60:0/1795088302 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92780bc070 con 0x7f9280148f40 2024-04-25T10:11:37.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.096+0000 7f92861ec640 1 -- 172.21.15.60:0/1795088302 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9280157b90 con 0x7f9280148f40 2024-04-25T10:11:37.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.096+0000 7f9267fff640 1 -- 172.21.15.60:0/1795088302 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92780b6a40 con 0x7f9280148f40 2024-04-25T10:11:37.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.096+0000 7f92861ec640 1 -- 172.21.15.60:0/1795088302 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92800b2690 con 0x7f9280148f40 2024-04-25T10:11:37.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.096+0000 7f92861ec640 1 -- 172.21.15.60:0/1795088302 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9280074770 con 0x7f9280148f40 2024-04-25T10:11:37.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.096+0000 7f9267fff640 1 -- 172.21.15.60:0/1795088302 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92780b6be0 con 0x7f9280148f40 2024-04-25T10:11:37.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.096+0000 7f9267fff640 1 --2- 172.21.15.60:0/1795088302 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f926c066f90 0x7f926c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.096+0000 7f9267fff640 1 -- 172.21.15.60:0/1795088302 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f9278120400 con 0x7f9280148f40 2024-04-25T10:11:37.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.096+0000 7f927f7fe640 1 --2- 172.21.15.60:0/1795088302 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f926c066f90 0x7f926c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.100+0000 7f927f7fe640 1 --2- 172.21.15.60:0/1795088302 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f926c066f90 0x7f926c069450 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f92800b0d10 tx=0x7f9270007cf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.100+0000 7f9267fff640 1 -- 172.21.15.60:0/1795088302 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9278016620 con 0x7f9280148f40 2024-04-25T10:11:37.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.264+0000 7f92861ec640 1 -- 172.21.15.60:0/1795088302 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9280157d90 con 0x7f9280148f40 2024-04-25T10:11:37.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.264+0000 7f9267fff640 1 -- 172.21.15.60:0/1795088302 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+13163 (secure 0 0 0) 0x7f92780f1820 con 0x7f9280148f40 2024-04-25T10:11:37.273 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:37.273 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":74,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:11:35.935225+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/1295867731":"2024-04-26T10:11:32.913025+0000","172.21.15.60:6833/1295867731":"2024-04-26T10:11:32.913025+0000","172.21.15.60:6835/1289503429":"2024-04-26T10:10:03.449620+0000","172.21.15.60:6833/1289503429":"2024-04-26T10:10:03.449620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:37.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.264+0000 7f92861ec640 1 -- 172.21.15.60:0/1795088302 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f926c066f90 msgr2=0x7f926c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.264+0000 7f92861ec640 1 --2- 172.21.15.60:0/1795088302 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f926c066f90 0x7f926c069450 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f92800b0d10 tx=0x7f9270007cf0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.264+0000 7f92861ec640 1 -- 172.21.15.60:0/1795088302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9280148f40 msgr2=0x7f92800b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.264+0000 7f92861ec640 1 --2- 172.21.15.60:0/1795088302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9280148f40 0x7f92800b2150 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f92780af040 tx=0x7f92780197d0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.264+0000 7f92861ec640 1 -- 172.21.15.60:0/1795088302 shutdown_connections 2024-04-25T10:11:37.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.264+0000 7f92861ec640 1 --2- 172.21.15.60:0/1795088302 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f926c066f90 0x7f926c069450 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.264+0000 7f92861ec640 1 --2- 172.21.15.60:0/1795088302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9280148f40 0x7f92800b2150 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.264+0000 7f92861ec640 1 --2- 172.21.15.60:0/1795088302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f928015bc90 0x7f9280155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.264+0000 7f92861ec640 1 --2- 172.21.15.60:0/1795088302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92800ce7a0 0x7f92800ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.264+0000 7f92861ec640 1 -- 172.21.15.60:0/1795088302 >> 172.21.15.60:0/1795088302 conn(0x7f92800b12d0 msgr2=0x7f9280154b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:37.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.268+0000 7f92861ec640 1 -- 172.21.15.60:0/1795088302 shutdown_connections 2024-04-25T10:11:37.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.268+0000 7f92861ec640 1 -- 172.21.15.60:0/1795088302 wait complete. 2024-04-25T10:11:37.294 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:37.294 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:37.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0af506e640 1 Processor -- start 2024-04-25T10:11:37.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0af506e640 1 -- start start 2024-04-25T10:11:37.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0af506e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0af00b3f60 0x7f0af015a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0af506e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af00b3510 con 0x7f0af006f030 2024-04-25T10:11:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0af506e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af015ad30 con 0x7f0af0071250 2024-04-25T10:11:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0af506e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af015b010 con 0x7f0af00b3f60 2024-04-25T10:11:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aee59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0af006f030 0x7f0af00b2e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39514/0 (socket says 172.21.15.60:39514) 2024-04-25T10:11:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aee59c640 1 -- 172.21.15.60:0/1380511572 learned_addr learned my addr 172.21.15.60:0/1380511572 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aeed9d640 1 --2- 172.21.15.60:0/1380511572 >> v2:172.21.15.100:3300/0 conn(0x7f0af00b3f60 0x7f0af015a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aedd9b640 1 -- 172.21.15.60:0/1380511572 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763138451 0 0) 0x7f0af00b3510 con 0x7f0af006f030 2024-04-25T10:11:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aedd9b640 1 -- 172.21.15.60:0/1380511572 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ad8003450 con 0x7f0af006f030 2024-04-25T10:11:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aedd9b640 1 -- 172.21.15.60:0/1380511572 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4043984394 0 0) 0x7f0ad8003450 con 0x7f0af006f030 2024-04-25T10:11:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aedd9b640 1 -- 172.21.15.60:0/1380511572 >> v2:172.21.15.100:3300/0 conn(0x7f0af00b3f60 msgr2=0x7f0af015a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aedd9b640 1 --2- 172.21.15.60:0/1380511572 >> v2:172.21.15.100:3300/0 conn(0x7f0af00b3f60 0x7f0af015a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aedd9b640 1 -- 172.21.15.60:0/1380511572 >> v1:172.21.15.100:6789/0 conn(0x7f0af0071250 legacy=0x7f0af015b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aedd9b640 1 -- 172.21.15.60:0/1380511572 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af015b2f0 con 0x7f0af006f030 2024-04-25T10:11:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aedd9b640 1 -- 172.21.15.60:0/1380511572 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f0ae4001680 con 0x7f0af006f030 2024-04-25T10:11:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aedd9b640 1 -- 172.21.15.60:0/1380511572 >> v1:172.21.15.60:6789/0 conn(0x7f0af006f030 legacy=0x7f0af00b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aedd9b640 1 --2- 172.21.15.60:0/1380511572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ad8003ab0 0x7f0ad8003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aedd9b640 1 --2- 172.21.15.60:0/1380511572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ad8008860 0x7f0ad800acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aedd9b640 1 --2- 172.21.15.60:0/1380511572 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ad800b570 0x7f0ad800da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aedd9b640 1 -- 172.21.15.60:0/1380511572 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af015b010 con 0x7f0ad8003ab0 2024-04-25T10:11:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aedd9b640 1 -- 172.21.15.60:0/1380511572 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad800e310 con 0x7f0ad8008860 2024-04-25T10:11:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aedd9b640 1 -- 172.21.15.60:0/1380511572 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad800e510 con 0x7f0ad800b570 2024-04-25T10:11:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aeed9d640 1 --2- 172.21.15.60:0/1380511572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ad8003ab0 0x7f0ad8003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aee59c640 1 --2- 172.21.15.60:0/1380511572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ad8008860 0x7f0ad800acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aeed9d640 1 -- 172.21.15.60:0/1380511572 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ad800b570 msgr2=0x7f0ad800da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aeed9d640 1 --2- 172.21.15.60:0/1380511572 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ad800b570 0x7f0ad800da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aeed9d640 1 -- 172.21.15.60:0/1380511572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ad8008860 msgr2=0x7f0ad800acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aeed9d640 1 --2- 172.21.15.60:0/1380511572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ad8008860 0x7f0ad800acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aeed9d640 1 -- 172.21.15.60:0/1380511572 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad8003450 con 0x7f0ad8003ab0 2024-04-25T10:11:37.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aee59c640 1 --2- 172.21.15.60:0/1380511572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ad8008860 0x7f0ad800acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:37.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.416+0000 7f0aef59e640 1 -- 172.21.15.60:0/1380511572 reap_dead start 2024-04-25T10:11:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.420+0000 7f0aeed9d640 1 --2- 172.21.15.60:0/1380511572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ad8003ab0 0x7f0ad8003ed0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009ae0 tx=0x7f0ae001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b985c26b16f18d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.420+0000 7f0aedd9b640 1 -- 172.21.15.60:0/1380511572 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00b2070 con 0x7f0ad8003ab0 2024-04-25T10:11:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.420+0000 7f0aedd9b640 1 -- 172.21.15.60:0/1380511572 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae00a5570 con 0x7f0ad8003ab0 2024-04-25T10:11:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.420+0000 7f0aedd9b640 1 -- 172.21.15.60:0/1380511572 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00096b0 con 0x7f0ad8003ab0 2024-04-25T10:11:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.420+0000 7f0af506e640 1 -- 172.21.15.60:0/1380511572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ad8003ab0 msgr2=0x7f0ad8003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.420+0000 7f0af506e640 1 --2- 172.21.15.60:0/1380511572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ad8003ab0 0x7f0ad8003ed0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009ae0 tx=0x7f0ae001ca90 comp rx=0 tx=0).stop 2024-04-25T10:11:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.420+0000 7f0af506e640 1 -- 172.21.15.60:0/1380511572 shutdown_connections 2024-04-25T10:11:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.420+0000 7f0af506e640 1 --2- 172.21.15.60:0/1380511572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ad8003ab0 0x7f0ad8003ed0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.420+0000 7f0af506e640 1 -- 172.21.15.60:0/1380511572 >> 172.21.15.60:0/1380511572 conn(0x7f0af00670b0 msgr2=0x7f0af006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.420+0000 7f0af506e640 1 -- 172.21.15.60:0/1380511572 shutdown_connections 2024-04-25T10:11:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.420+0000 7f0af506e640 1 -- 172.21.15.60:0/1380511572 wait complete. 2024-04-25T10:11:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.420+0000 7f0af506e640 1 Processor -- start 2024-04-25T10:11:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.420+0000 7f0af506e640 1 -- start start 2024-04-25T10:11:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0af506e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ad8003ab0 0x7f0af01f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0af506e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0148000 0x7f0af01f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0af506e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0af01f3f20 0x7f0af015d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0af506e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af0071bc0 con 0x7f0af0148000 2024-04-25T10:11:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0af506e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af0071a40 con 0x7f0af01f3f20 2024-04-25T10:11:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0af506e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af015a3e0 con 0x7f0ad8003ab0 2024-04-25T10:11:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0aee59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0148000 0x7f0af01f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0aee59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0148000 0x7f0af01f39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37138/0 (socket says 172.21.15.60:37138) 2024-04-25T10:11:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0aee59c640 1 -- 172.21.15.60:0/4006763304 learned_addr learned my addr 172.21.15.60:0/4006763304 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0aef59e640 1 --2- 172.21.15.60:0/4006763304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0af01f3f20 0x7f0af015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0aeed9d640 1 --2- 172.21.15.60:0/4006763304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ad8003ab0 0x7f0af01f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0aee59c640 1 -- 172.21.15.60:0/4006763304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ad8003ab0 msgr2=0x7f0af01f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0aee59c640 1 --2- 172.21.15.60:0/4006763304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ad8003ab0 0x7f0af01f5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0aee59c640 1 -- 172.21.15.60:0/4006763304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0af01f3f20 msgr2=0x7f0af015d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0aee59c640 1 --2- 172.21.15.60:0/4006763304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0af01f3f20 0x7f0af015d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0aee59c640 1 -- 172.21.15.60:0/4006763304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af00b0f30 con 0x7f0af0148000 2024-04-25T10:11:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0aee59c640 1 --2- 172.21.15.60:0/4006763304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0148000 0x7f0af01f39e0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400d8b0 tx=0x7f0ae400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0ad77fe640 1 -- 172.21.15.60:0/4006763304 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4019070 con 0x7f0af0148000 2024-04-25T10:11:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0ad77fe640 1 -- 172.21.15.60:0/4006763304 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae40074e0 con 0x7f0af0148000 2024-04-25T10:11:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0af506e640 1 -- 172.21.15.60:0/4006763304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af01f45f0 con 0x7f0af0148000 2024-04-25T10:11:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0ad77fe640 1 -- 172.21.15.60:0/4006763304 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4004a30 con 0x7f0af0148000 2024-04-25T10:11:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0af506e640 1 -- 172.21.15.60:0/4006763304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af01f4bb0 con 0x7f0af0148000 2024-04-25T10:11:37.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0af506e640 1 -- 172.21.15.60:0/4006763304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af0074770 con 0x7f0af0148000 2024-04-25T10:11:37.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0ad77fe640 1 -- 172.21.15.60:0/4006763304 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ae401c070 con 0x7f0af0148000 2024-04-25T10:11:37.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0ad77fe640 1 --2- 172.21.15.60:0/4006763304 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0adc066f60 0x7f0adc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.424+0000 7f0ad77fe640 1 -- 172.21.15.60:0/4006763304 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f0ae4084ec0 con 0x7f0af0148000 2024-04-25T10:11:37.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.428+0000 7f0aeed9d640 1 --2- 172.21.15.60:0/4006763304 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0adc066f60 0x7f0adc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.428+0000 7f0aeed9d640 1 --2- 172.21.15.60:0/4006763304 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0adc066f60 0x7f0adc069420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00af000 tx=0x7f0ae00046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.428+0000 7f0ad77fe640 1 -- 172.21.15.60:0/4006763304 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0af00b0f30 con 0x7f0af0148000 2024-04-25T10:11:37.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.584+0000 7f0af506e640 1 -- 172.21.15.60:0/4006763304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0af0148480 con 0x7f0af0148000 2024-04-25T10:11:37.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.584+0000 7f0ad77fe640 1 -- 172.21.15.60:0/4006763304 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ae4055c00 con 0x7f0af0148000 2024-04-25T10:11:37.594 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:11:37.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.588+0000 7f0af506e640 1 -- 172.21.15.60:0/4006763304 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0adc066f60 msgr2=0x7f0adc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.588+0000 7f0af506e640 1 --2- 172.21.15.60:0/4006763304 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0adc066f60 0x7f0adc069420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00af000 tx=0x7f0ae00046d0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.588+0000 7f0af506e640 1 -- 172.21.15.60:0/4006763304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0148000 msgr2=0x7f0af01f39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.588+0000 7f0af506e640 1 --2- 172.21.15.60:0/4006763304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0148000 0x7f0af01f39e0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400d8b0 tx=0x7f0ae400dd70 comp rx=0 tx=0).stop 2024-04-25T10:11:37.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.588+0000 7f0af506e640 1 -- 172.21.15.60:0/4006763304 shutdown_connections 2024-04-25T10:11:37.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.588+0000 7f0af506e640 1 --2- 172.21.15.60:0/4006763304 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0adc066f60 0x7f0adc069420 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.588+0000 7f0af506e640 1 --2- 172.21.15.60:0/4006763304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0af01f3f20 0x7f0af015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.588+0000 7f0af506e640 1 --2- 172.21.15.60:0/4006763304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0af0148000 0x7f0af01f39e0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.588+0000 7f0af506e640 1 --2- 172.21.15.60:0/4006763304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ad8003ab0 0x7f0af01f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.588+0000 7f0af506e640 1 -- 172.21.15.60:0/4006763304 >> 172.21.15.60:0/4006763304 conn(0x7f0af00670b0 msgr2=0x7f0af00af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:37.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.588+0000 7f0af506e640 1 -- 172.21.15.60:0/4006763304 shutdown_connections 2024-04-25T10:11:37.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.588+0000 7f0af506e640 1 -- 172.21.15.60:0/4006763304 wait complete. 2024-04-25T10:11:37.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-25T10:11:37.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.716+0000 7f51ab279640 1 Processor -- start 2024-04-25T10:11:37.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.716+0000 7f51ab279640 1 -- start start 2024-04-25T10:11:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.716+0000 7f51ab279640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f51a415bc90 0x7f51a415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.716+0000 7f51ab279640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f51a40b4660 0x7f51a415c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.716+0000 7f51ab279640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a4071ca0 con 0x7f51a40ce7a0 2024-04-25T10:11:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.716+0000 7f51ab279640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a40cc370 con 0x7f51a415bc90 2024-04-25T10:11:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.716+0000 7f51ab279640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a40cca40 con 0x7f51a40b4660 2024-04-25T10:11:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.716+0000 7f51a8814640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f51a415bc90 0x7f51a415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.716+0000 7f51a8814640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f51a415bc90 0x7f51a415c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37154/0 (socket says 172.21.15.60:37154) 2024-04-25T10:11:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.716+0000 7f51a8814640 1 -- 172.21.15.60:0/4209998777 learned_addr learned my addr 172.21.15.60:0/4209998777 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.716+0000 7f51a9816640 1 --2- 172.21.15.60:0/4209998777 >> v2:172.21.15.100:3300/0 conn(0x7f51a40b4660 0x7f51a415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.716+0000 7f51a8814640 1 -- 172.21.15.60:0/4209998777 >> v2:172.21.15.100:3300/0 conn(0x7f51a40b4660 msgr2=0x7f51a415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.716+0000 7f51a8814640 1 --2- 172.21.15.60:0/4209998777 >> v2:172.21.15.100:3300/0 conn(0x7f51a40b4660 0x7f51a415c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.716+0000 7f51a8814640 1 -- 172.21.15.60:0/4209998777 >> v1:172.21.15.100:6789/0 conn(0x7f51a40ce7a0 legacy=0x7f51a40ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.716+0000 7f51a8814640 1 -- 172.21.15.60:0/4209998777 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a40ccd20 con 0x7f51a415bc90 2024-04-25T10:11:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51a8814640 1 --2- 172.21.15.60:0/4209998777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a415bc90 0x7f51a415c220 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f51a4072bf0 tx=0x7f519401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548fe3e99339bb55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f519bfff640 1 -- 172.21.15.60:0/4209998777 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51940b0070 con 0x7f51a415bc90 2024-04-25T10:11:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f519bfff640 1 -- 172.21.15.60:0/4209998777 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51940ab600 con 0x7f51a415bc90 2024-04-25T10:11:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f519bfff640 1 -- 172.21.15.60:0/4209998777 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51940ab9d0 con 0x7f51a415bc90 2024-04-25T10:11:37.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 -- 172.21.15.60:0/4209998777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a415bc90 msgr2=0x7f51a415c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 --2- 172.21.15.60:0/4209998777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a415bc90 0x7f51a415c220 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f51a4072bf0 tx=0x7f519401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 -- 172.21.15.60:0/4209998777 shutdown_connections 2024-04-25T10:11:37.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 --2- 172.21.15.60:0/4209998777 >> v2:172.21.15.100:3300/0 conn(0x7f51a40b4660 0x7f51a415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 --2- 172.21.15.60:0/4209998777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a415bc90 0x7f51a415c220 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 -- 172.21.15.60:0/4209998777 >> 172.21.15.60:0/4209998777 conn(0x7f51a40b12d0 msgr2=0x7f51a40b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:37.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 -- 172.21.15.60:0/4209998777 shutdown_connections 2024-04-25T10:11:37.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 -- 172.21.15.60:0/4209998777 wait complete. 2024-04-25T10:11:37.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 Processor -- start 2024-04-25T10:11:37.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 -- start start 2024-04-25T10:11:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a40ce7a0 0x7f51a41448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a415bc90 0x7f51a4148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51a41510e0 0x7f51a414b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a41685c0 con 0x7f51a40ce7a0 2024-04-25T10:11:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a40b3db0 con 0x7f51a415bc90 2024-04-25T10:11:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a4168740 con 0x7f51a41510e0 2024-04-25T10:11:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51a9015640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a40ce7a0 0x7f51a41448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51a9015640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a40ce7a0 0x7f51a41448f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37164/0 (socket says 172.21.15.60:37164) 2024-04-25T10:11:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51a9015640 1 -- 172.21.15.60:0/672934529 learned_addr learned my addr 172.21.15.60:0/672934529 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51a9015640 1 -- 172.21.15.60:0/672934529 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51a41510e0 msgr2=0x7f51a414b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51a9015640 1 --2- 172.21.15.60:0/672934529 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51a41510e0 0x7f51a414b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51a9015640 1 -- 172.21.15.60:0/672934529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a415bc90 msgr2=0x7f51a4148620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51a9015640 1 --2- 172.21.15.60:0/672934529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a415bc90 0x7f51a4148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51a9015640 1 -- 172.21.15.60:0/672934529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a4148f20 con 0x7f51a40ce7a0 2024-04-25T10:11:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51a9015640 1 --2- 172.21.15.60:0/672934529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a40ce7a0 0x7f51a41448f0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f51a000b410 tx=0x7f51a000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f5199ffb640 1 -- 172.21.15.60:0/672934529 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0015030 con 0x7f51a40ce7a0 2024-04-25T10:11:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f5199ffb640 1 -- 172.21.15.60:0/672934529 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a0004020 con 0x7f51a40ce7a0 2024-04-25T10:11:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 -- 172.21.15.60:0/672934529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a4149200 con 0x7f51a40ce7a0 2024-04-25T10:11:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f5199ffb640 1 -- 172.21.15.60:0/672934529 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0004450 con 0x7f51a40ce7a0 2024-04-25T10:11:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.720+0000 7f51ab279640 1 -- 172.21.15.60:0/672934529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a41497c0 con 0x7f51a40ce7a0 2024-04-25T10:11:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.724+0000 7f51ab279640 1 -- 172.21.15.60:0/672934529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a4074770 con 0x7f51a40ce7a0 2024-04-25T10:11:37.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.724+0000 7f5199ffb640 1 -- 172.21.15.60:0/672934529 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51a0018070 con 0x7f51a40ce7a0 2024-04-25T10:11:37.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.724+0000 7f5199ffb640 1 --2- 172.21.15.60:0/672934529 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5188066f60 0x7f5188069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.724+0000 7f5199ffb640 1 -- 172.21.15.60:0/672934529 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f51a00849d0 con 0x7f51a40ce7a0 2024-04-25T10:11:37.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.724+0000 7f51a8814640 1 --2- 172.21.15.60:0/672934529 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5188066f60 0x7f5188069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.724+0000 7f51a8814640 1 --2- 172.21.15.60:0/672934529 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5188066f60 0x7f5188069420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f51940ad000 tx=0x7f5194004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.728+0000 7f5199ffb640 1 -- 172.21.15.60:0/672934529 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51a0055be0 con 0x7f51a40ce7a0 2024-04-25T10:11:37.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:37.940+0000 7f51ab279640 1 -- 172.21.15.60:0/672934529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7f51a4073220 con 0x7f51a40ce7a0 2024-04-25T10:11:38.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:38.944+0000 7f5199ffb640 1 -- 172.21.15.60:0/672934529 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v305) v1 ==== 170+0+0 (secure 0 0 0) 0x7f51a00554e0 con 0x7f51a40ce7a0 2024-04-25T10:11:38.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:38.948+0000 7f51ab279640 1 -- 172.21.15.60:0/672934529 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5188066f60 msgr2=0x7f5188069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:38.948+0000 7f51ab279640 1 --2- 172.21.15.60:0/672934529 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5188066f60 0x7f5188069420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f51940ad000 tx=0x7f5194004740 comp rx=0 tx=0).stop 2024-04-25T10:11:38.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:38.948+0000 7f51ab279640 1 -- 172.21.15.60:0/672934529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a40ce7a0 msgr2=0x7f51a41448f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:38.948+0000 7f51ab279640 1 --2- 172.21.15.60:0/672934529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a40ce7a0 0x7f51a41448f0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f51a000b410 tx=0x7f51a000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:38.948+0000 7f51ab279640 1 -- 172.21.15.60:0/672934529 shutdown_connections 2024-04-25T10:11:38.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:38.948+0000 7f51ab279640 1 --2- 172.21.15.60:0/672934529 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5188066f60 0x7f5188069420 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:38.948+0000 7f51ab279640 1 --2- 172.21.15.60:0/672934529 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51a41510e0 0x7f51a414b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:38.948+0000 7f51ab279640 1 --2- 172.21.15.60:0/672934529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a415bc90 0x7f51a4148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:38.948+0000 7f51ab279640 1 --2- 172.21.15.60:0/672934529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a40ce7a0 0x7f51a41448f0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:38.948+0000 7f51ab279640 1 -- 172.21.15.60:0/672934529 >> 172.21.15.60:0/672934529 conn(0x7f51a40b12d0 msgr2=0x7f51a40ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:38.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:38.948+0000 7f51ab279640 1 -- 172.21.15.60:0/672934529 shutdown_connections 2024-04-25T10:11:38.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:38.948+0000 7f51ab279640 1 -- 172.21.15.60:0/672934529 wait complete. 2024-04-25T10:11:38.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:11:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f8606981640 1 Processor -- start 2024-04-25T10:11:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f8606981640 1 -- start start 2024-04-25T10:11:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f8606981640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f860015fbf0 0x7f8600166870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f8606981640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8600144a40 con 0x7f8600160080 2024-04-25T10:11:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f8606981640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86000cc9f0 con 0x7f86000cbcf0 2024-04-25T10:11:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f8606981640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86000b2dc0 con 0x7f860015fbf0 2024-04-25T10:11:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85ff7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8600160080 0x7f860015f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50724/0 (socket says 172.21.15.60:50724) 2024-04-25T10:11:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85ff7fe640 1 -- 172.21.15.60:0/2386606820 learned_addr learned my addr 172.21.15.60:0/2386606820 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f8604f1e640 1 --2- 172.21.15.60:0/2386606820 >> v2:172.21.15.100:3300/0 conn(0x7f860015fbf0 0x7f8600166870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85feffd640 1 -- 172.21.15.60:0/2386606820 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794309821 0 0) 0x7f8600144a40 con 0x7f8600160080 2024-04-25T10:11:39.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85feffd640 1 -- 172.21.15.60:0/2386606820 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85e8003530 con 0x7f8600160080 2024-04-25T10:11:39.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85feffd640 1 -- 172.21.15.60:0/2386606820 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 747748700 0 0) 0x7f85e8003530 con 0x7f8600160080 2024-04-25T10:11:39.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85feffd640 1 -- 172.21.15.60:0/2386606820 >> v2:172.21.15.100:3300/0 conn(0x7f860015fbf0 msgr2=0x7f8600166870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85feffd640 1 --2- 172.21.15.60:0/2386606820 >> v2:172.21.15.100:3300/0 conn(0x7f860015fbf0 0x7f8600166870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85feffd640 1 -- 172.21.15.60:0/2386606820 >> v1:172.21.15.100:6789/0 conn(0x7f86000cbcf0 legacy=0x7f8600071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85feffd640 1 -- 172.21.15.60:0/2386606820 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86000cccd0 con 0x7f8600160080 2024-04-25T10:11:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85feffd640 1 -- 172.21.15.60:0/2386606820 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f85f4001680 con 0x7f8600160080 2024-04-25T10:11:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85feffd640 1 -- 172.21.15.60:0/2386606820 >> v1:172.21.15.60:6789/0 conn(0x7f8600160080 legacy=0x7f860015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85feffd640 1 --2- 172.21.15.60:0/2386606820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85e80039c0 0x7f85e8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85feffd640 1 --2- 172.21.15.60:0/2386606820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85e8004810 0x7f85e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85feffd640 1 --2- 172.21.15.60:0/2386606820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85e80051d0 0x7f85e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85feffd640 1 -- 172.21.15.60:0/2386606820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e800c040 con 0x7f85e80039c0 2024-04-25T10:11:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85feffd640 1 -- 172.21.15.60:0/2386606820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e800c1f0 con 0x7f85e8004810 2024-04-25T10:11:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85feffd640 1 -- 172.21.15.60:0/2386606820 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e800c3f0 con 0x7f85e80051d0 2024-04-25T10:11:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85fffff640 1 --2- 172.21.15.60:0/2386606820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85e80039c0 0x7f85e8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85ff7fe640 1 --2- 172.21.15.60:0/2386606820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85e8004810 0x7f85e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f8604f1e640 1 --2- 172.21.15.60:0/2386606820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85e80051d0 0x7f85e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85fffff640 1 -- 172.21.15.60:0/2386606820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85e80051d0 msgr2=0x7f85e800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:39.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85fffff640 1 --2- 172.21.15.60:0/2386606820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85e80051d0 0x7f85e800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85fffff640 1 -- 172.21.15.60:0/2386606820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85e8004810 msgr2=0x7f85e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:39.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85fffff640 1 --2- 172.21.15.60:0/2386606820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85e8004810 0x7f85e8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f85fffff640 1 -- 172.21.15.60:0/2386606820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e8003530 con 0x7f85e80039c0 2024-04-25T10:11:39.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.108+0000 7f8604f1e640 1 -- 172.21.15.60:0/2386606820 reap_dead start 2024-04-25T10:11:39.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.112+0000 7f85fffff640 1 --2- 172.21.15.60:0/2386606820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85e80039c0 0x7f85e8003de0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f85f0001780 tx=0x7f85f001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80521a9f9b5d9fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.112+0000 7f85feffd640 1 -- 172.21.15.60:0/2386606820 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f00013a0 con 0x7f85e80039c0 2024-04-25T10:11:39.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.112+0000 7f85feffd640 1 -- 172.21.15.60:0/2386606820 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f00052f0 con 0x7f85e80039c0 2024-04-25T10:11:39.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.112+0000 7f85feffd640 1 -- 172.21.15.60:0/2386606820 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f0003840 con 0x7f85e80039c0 2024-04-25T10:11:39.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.112+0000 7f8606981640 1 -- 172.21.15.60:0/2386606820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85e80039c0 msgr2=0x7f85e8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.112+0000 7f8606981640 1 --2- 172.21.15.60:0/2386606820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85e80039c0 0x7f85e8003de0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f85f0001780 tx=0x7f85f001c400 comp rx=0 tx=0).stop 2024-04-25T10:11:39.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.112+0000 7f8606981640 1 -- 172.21.15.60:0/2386606820 shutdown_connections 2024-04-25T10:11:39.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.112+0000 7f8606981640 1 --2- 172.21.15.60:0/2386606820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85e80039c0 0x7f85e8003de0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.112+0000 7f8606981640 1 -- 172.21.15.60:0/2386606820 >> 172.21.15.60:0/2386606820 conn(0x7f86000670b0 msgr2=0x7f860006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:39.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.112+0000 7f8606981640 1 -- 172.21.15.60:0/2386606820 shutdown_connections 2024-04-25T10:11:39.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.112+0000 7f8606981640 1 -- 172.21.15.60:0/2386606820 wait complete. 2024-04-25T10:11:39.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.112+0000 7f8606981640 1 Processor -- start 2024-04-25T10:11:39.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.112+0000 7f8606981640 1 -- start start 2024-04-25T10:11:39.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f8606981640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85e80039c0 0x7f860012eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f8606981640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f860015b280 0x7f8600130210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f8606981640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8600130750 0x7f86000ab610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f8606981640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860015fa70 con 0x7f85e80039c0 2024-04-25T10:11:39.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f8606981640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860015f8f0 con 0x7f8600130750 2024-04-25T10:11:39.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f8606981640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8600072040 con 0x7f860015b280 2024-04-25T10:11:39.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85fffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85e80039c0 0x7f860012eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85fffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85e80039c0 0x7f860012eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45336/0 (socket says 172.21.15.60:45336) 2024-04-25T10:11:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85fffff640 1 -- 172.21.15.60:0/1433988712 learned_addr learned my addr 172.21.15.60:0/1433988712 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85ff7fe640 1 --2- 172.21.15.60:0/1433988712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f860015b280 0x7f8600130210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f8604f1e640 1 --2- 172.21.15.60:0/1433988712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8600130750 0x7f86000ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85fffff640 1 -- 172.21.15.60:0/1433988712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f860015b280 msgr2=0x7f8600130210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85fffff640 1 --2- 172.21.15.60:0/1433988712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f860015b280 0x7f8600130210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85fffff640 1 -- 172.21.15.60:0/1433988712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8600130750 msgr2=0x7f86000ab610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85fffff640 1 --2- 172.21.15.60:0/1433988712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8600130750 0x7f86000ab610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85fffff640 1 -- 172.21.15.60:0/1433988712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860012f170 con 0x7f85e80039c0 2024-04-25T10:11:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85ff7fe640 1 --2- 172.21.15.60:0/1433988712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f860015b280 0x7f8600130210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85fffff640 1 --2- 172.21.15.60:0/1433988712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85e80039c0 0x7f860012eb90 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f85f0005040 tx=0x7f85e8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85fcff9640 1 -- 172.21.15.60:0/1433988712 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f0007a40 con 0x7f85e80039c0 2024-04-25T10:11:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85fcff9640 1 -- 172.21.15.60:0/1433988712 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f00ba070 con 0x7f85e80039c0 2024-04-25T10:11:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f8606981640 1 -- 172.21.15.60:0/1433988712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f860012f450 con 0x7f85e80039c0 2024-04-25T10:11:39.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85fcff9640 1 -- 172.21.15.60:0/1433988712 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f00054a0 con 0x7f85e80039c0 2024-04-25T10:11:39.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f8606981640 1 -- 172.21.15.60:0/1433988712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860012fa10 con 0x7f85e80039c0 2024-04-25T10:11:39.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f8606981640 1 -- 172.21.15.60:0/1433988712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f860006bba0 con 0x7f85e80039c0 2024-04-25T10:11:39.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85fcff9640 1 -- 172.21.15.60:0/1433988712 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85f0007030 con 0x7f85e80039c0 2024-04-25T10:11:39.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85fcff9640 1 --2- 172.21.15.60:0/1433988712 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f85ec066f60 0x7f85ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85fcff9640 1 -- 172.21.15.60:0/1433988712 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f85f011f710 con 0x7f85e80039c0 2024-04-25T10:11:39.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.116+0000 7f85ff7fe640 1 --2- 172.21.15.60:0/1433988712 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f85ec066f60 0x7f85ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.120+0000 7f85ff7fe640 1 --2- 172.21.15.60:0/1433988712 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f85ec066f60 0x7f85ec069420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f86000b2280 tx=0x7f85f4000ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.120+0000 7f85fcff9640 1 -- 172.21.15.60:0/1433988712 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f85f0016110 con 0x7f85e80039c0 2024-04-25T10:11:39.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.280+0000 7f8606981640 1 -- 172.21.15.60:0/1433988712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8600074770 con 0x7f85e80039c0 2024-04-25T10:11:39.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.284+0000 7f85fcff9640 1 -- 172.21.15.60:0/1433988712 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f85f00f0aa0 con 0x7f85e80039c0 2024-04-25T10:11:39.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.288+0000 7f8606981640 1 -- 172.21.15.60:0/1433988712 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f85ec066f60 msgr2=0x7f85ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.288+0000 7f8606981640 1 --2- 172.21.15.60:0/1433988712 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f85ec066f60 0x7f85ec069420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f86000b2280 tx=0x7f85f4000ee0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.288+0000 7f8606981640 1 -- 172.21.15.60:0/1433988712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85e80039c0 msgr2=0x7f860012eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.288+0000 7f8606981640 1 --2- 172.21.15.60:0/1433988712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85e80039c0 0x7f860012eb90 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f85f0005040 tx=0x7f85e8005840 comp rx=0 tx=0).stop 2024-04-25T10:11:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.288+0000 7f8606981640 1 -- 172.21.15.60:0/1433988712 shutdown_connections 2024-04-25T10:11:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.288+0000 7f8606981640 1 --2- 172.21.15.60:0/1433988712 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f85ec066f60 0x7f85ec069420 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.288+0000 7f8606981640 1 --2- 172.21.15.60:0/1433988712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8600130750 0x7f86000ab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.288+0000 7f8606981640 1 --2- 172.21.15.60:0/1433988712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f860015b280 0x7f8600130210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.288+0000 7f8606981640 1 --2- 172.21.15.60:0/1433988712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85e80039c0 0x7f860012eb90 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.288+0000 7f8606981640 1 -- 172.21.15.60:0/1433988712 >> 172.21.15.60:0/1433988712 conn(0x7f86000670b0 msgr2=0x7f860015d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.288+0000 7f8606981640 1 -- 172.21.15.60:0/1433988712 shutdown_connections 2024-04-25T10:11:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.288+0000 7f8606981640 1 -- 172.21.15.60:0/1433988712 wait complete. 2024-04-25T10:11:39.313 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:11:39.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:39.361 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:39.407 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:11:39.407 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:11:39.407 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:39.413 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:11:39.413 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:11:39.413 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:39.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:39.419 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:11:39.419 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:39.460 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:11:39.461 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:11:39.461 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:39.467 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:39.467 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:11:39.467 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:39.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.628+0000 7f2ba2c55640 1 Processor -- start 2024-04-25T10:11:39.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.628+0000 7f2ba2c55640 1 -- start start 2024-04-25T10:11:39.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.628+0000 7f2ba2c55640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2b9c15d9f0 0x7f2b9c0af710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.628+0000 7f2ba2c55640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b9c0afc50 0x7f2b9c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.628+0000 7f2ba2c55640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9c06b5e0 con 0x7f2b9c06b150 2024-04-25T10:11:39.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.628+0000 7f2ba2c55640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9c0ce480 con 0x7f2b9c15d9f0 2024-04-25T10:11:39.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.628+0000 7f2ba2c55640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c0b00b0 con 0x7f2b9c0afc50 2024-04-25T10:11:39.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.628+0000 7f2ba09f1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2b9c15d9f0 0x7f2b9c0af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.628+0000 7f2ba09f1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2b9c15d9f0 0x7f2b9c0af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45348/0 (socket says 172.21.15.60:45348) 2024-04-25T10:11:39.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.628+0000 7f2ba09f1640 1 -- 172.21.15.60:0/2383727977 learned_addr learned my addr 172.21.15.60:0/2383727977 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:39.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.628+0000 7f2b9bfff640 1 --2- 172.21.15.60:0/2383727977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b9c0afc50 0x7f2b9c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.628+0000 7f2ba09f1640 1 -- 172.21.15.60:0/2383727977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b9c0afc50 msgr2=0x7f2b9c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:39.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.628+0000 7f2ba09f1640 1 --2- 172.21.15.60:0/2383727977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b9c0afc50 0x7f2b9c06ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.628+0000 7f2ba09f1640 1 -- 172.21.15.60:0/2383727977 >> v1:172.21.15.100:6789/0 conn(0x7f2b9c06b150 legacy=0x7f2b9c161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.628+0000 7f2ba09f1640 1 -- 172.21.15.60:0/2383727977 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c0724a0 con 0x7f2b9c15d9f0 2024-04-25T10:11:39.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.632+0000 7f2ba09f1640 1 --2- 172.21.15.60:0/2383727977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c15d9f0 0x7f2b9c0af710 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009870 tx=0x7f2b8c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09515e4b54ef375 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.632+0000 7f2b9b7fe640 1 -- 172.21.15.60:0/2383727977 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c0aa600 con 0x7f2b9c15d9f0 2024-04-25T10:11:39.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.632+0000 7f2b9b7fe640 1 -- 172.21.15.60:0/2383727977 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8c0af070 con 0x7f2b9c15d9f0 2024-04-25T10:11:39.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.632+0000 7f2b9b7fe640 1 -- 172.21.15.60:0/2383727977 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c0aa9c0 con 0x7f2b9c15d9f0 2024-04-25T10:11:39.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.632+0000 7f2ba2c55640 1 -- 172.21.15.60:0/2383727977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c15d9f0 msgr2=0x7f2b9c0af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.632+0000 7f2ba2c55640 1 --2- 172.21.15.60:0/2383727977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c15d9f0 0x7f2b9c0af710 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009870 tx=0x7f2b8c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.632+0000 7f2ba2c55640 1 -- 172.21.15.60:0/2383727977 shutdown_connections 2024-04-25T10:11:39.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.632+0000 7f2ba2c55640 1 --2- 172.21.15.60:0/2383727977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b9c0afc50 0x7f2b9c06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.632+0000 7f2ba2c55640 1 --2- 172.21.15.60:0/2383727977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c15d9f0 0x7f2b9c0af710 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.632+0000 7f2ba2c55640 1 -- 172.21.15.60:0/2383727977 >> 172.21.15.60:0/2383727977 conn(0x7f2b9c0b2360 msgr2=0x7f2b9c0b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:39.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.632+0000 7f2ba2c55640 1 -- 172.21.15.60:0/2383727977 shutdown_connections 2024-04-25T10:11:39.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.632+0000 7f2ba2c55640 1 -- 172.21.15.60:0/2383727977 wait complete. 2024-04-25T10:11:39.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2ba2c55640 1 Processor -- start 2024-04-25T10:11:39.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2ba2c55640 1 -- start start 2024-04-25T10:11:39.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2ba2c55640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b9c06b150 0x7f2b9c13bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2ba2c55640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c0afc50 0x7f2b9c0b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2ba2c55640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b9c15d9f0 0x7f2b9c0b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2ba2c55640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c15ddd0 con 0x7f2b9c0afc50 2024-04-25T10:11:39.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2ba2c55640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c073020 con 0x7f2b9c06b150 2024-04-25T10:11:39.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2ba2c55640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c14a2a0 con 0x7f2b9c15d9f0 2024-04-25T10:11:39.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2b9bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c0afc50 0x7f2b9c0b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2b9bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c0afc50 0x7f2b9c0b0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45350/0 (socket says 172.21.15.60:45350) 2024-04-25T10:11:39.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2b9bfff640 1 -- 172.21.15.60:0/3811227741 learned_addr learned my addr 172.21.15.60:0/3811227741 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:39.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2ba09f1640 1 --2- 172.21.15.60:0/3811227741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b9c06b150 0x7f2b9c13bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2ba11f2640 1 --2- 172.21.15.60:0/3811227741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b9c15d9f0 0x7f2b9c0b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2b9bfff640 1 -- 172.21.15.60:0/3811227741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b9c15d9f0 msgr2=0x7f2b9c0b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2b9bfff640 1 --2- 172.21.15.60:0/3811227741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b9c15d9f0 0x7f2b9c0b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2b9bfff640 1 -- 172.21.15.60:0/3811227741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b9c06b150 msgr2=0x7f2b9c13bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2b9bfff640 1 --2- 172.21.15.60:0/3811227741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b9c06b150 0x7f2b9c13bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2b9bfff640 1 -- 172.21.15.60:0/3811227741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c14a580 con 0x7f2b9c0afc50 2024-04-25T10:11:39.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2b9bfff640 1 --2- 172.21.15.60:0/3811227741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c0afc50 0x7f2b9c0b0240 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000d800 tx=0x7f2b9000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2b997fa640 1 -- 172.21.15.60:0/3811227741 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9000b6c0 con 0x7f2b9c0afc50 2024-04-25T10:11:39.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2b997fa640 1 -- 172.21.15.60:0/3811227741 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b90019070 con 0x7f2b9c0afc50 2024-04-25T10:11:39.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2ba2c55640 1 -- 172.21.15.60:0/3811227741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9c14a860 con 0x7f2b9c0afc50 2024-04-25T10:11:39.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2b997fa640 1 -- 172.21.15.60:0/3811227741 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9000ba60 con 0x7f2b9c0afc50 2024-04-25T10:11:39.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2ba2c55640 1 -- 172.21.15.60:0/3811227741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9c14ae20 con 0x7f2b9c0afc50 2024-04-25T10:11:39.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2ba2c55640 1 -- 172.21.15.60:0/3811227741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9c06ccb0 con 0x7f2b9c0afc50 2024-04-25T10:11:39.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.636+0000 7f2b997fa640 1 -- 172.21.15.60:0/3811227741 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2b9000bc00 con 0x7f2b9c0afc50 2024-04-25T10:11:39.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.640+0000 7f2b997fa640 1 --2- 172.21.15.60:0/3811227741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b88066f60 0x7f2b88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.640+0000 7f2b997fa640 1 -- 172.21.15.60:0/3811227741 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f2b90085310 con 0x7f2b9c0afc50 2024-04-25T10:11:39.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.640+0000 7f2ba09f1640 1 --2- 172.21.15.60:0/3811227741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b88066f60 0x7f2b88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.640+0000 7f2ba09f1640 1 --2- 172.21.15.60:0/3811227741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b88066f60 0x7f2b88069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0ae000 tx=0x7f2b8c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.644+0000 7f2b997fa640 1 -- 172.21.15.60:0/3811227741 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2b90056520 con 0x7f2b9c0afc50 2024-04-25T10:11:39.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.836+0000 7f2ba2c55640 1 -- 172.21.15.60:0/3811227741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b9c1687d0 con 0x7f2b9c0afc50 2024-04-25T10:11:39.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.836+0000 7f2b997fa640 1 -- 172.21.15.60:0/3811227741 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f2b90055e20 con 0x7f2b9c0afc50 2024-04-25T10:11:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:39.846 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5995,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3185787428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3185787428},{"type":"v1","addr":"172.21.15.100:6836","nonce":3185787428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5998,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2667415244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2667415244},{"type":"v1","addr":"172.21.15.100:6837","nonce":2667415244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/56747111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":56747111},{"type":"v1","addr":"172.21.15.60:6834","nonce":56747111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":7575,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1987160192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1987160192},{"type":"v1","addr":"172.21.15.60:6835","nonce":1987160192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[]} 2024-04-25T10:11:39.846 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 49 2024-04-25T10:11:39.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.840+0000 7f2ba2c55640 1 -- 172.21.15.60:0/3811227741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b88066f60 msgr2=0x7f2b88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.840+0000 7f2ba2c55640 1 --2- 172.21.15.60:0/3811227741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b88066f60 0x7f2b88069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0ae000 tx=0x7f2b8c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:11:39.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.840+0000 7f2ba2c55640 1 -- 172.21.15.60:0/3811227741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c0afc50 msgr2=0x7f2b9c0b0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.840+0000 7f2ba2c55640 1 --2- 172.21.15.60:0/3811227741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c0afc50 0x7f2b9c0b0240 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000d800 tx=0x7f2b9000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.840+0000 7f2ba2c55640 1 -- 172.21.15.60:0/3811227741 shutdown_connections 2024-04-25T10:11:39.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.840+0000 7f2ba2c55640 1 --2- 172.21.15.60:0/3811227741 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b88066f60 0x7f2b88069420 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.840+0000 7f2ba2c55640 1 --2- 172.21.15.60:0/3811227741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b9c15d9f0 0x7f2b9c0b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.840+0000 7f2ba2c55640 1 --2- 172.21.15.60:0/3811227741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c0afc50 0x7f2b9c0b0240 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.840+0000 7f2ba2c55640 1 --2- 172.21.15.60:0/3811227741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b9c06b150 0x7f2b9c13bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.840+0000 7f2ba2c55640 1 -- 172.21.15.60:0/3811227741 >> 172.21.15.60:0/3811227741 conn(0x7f2b9c0b2360 msgr2=0x7f2b9c156e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:39.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.840+0000 7f2ba2c55640 1 -- 172.21.15.60:0/3811227741 shutdown_connections 2024-04-25T10:11:39.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:39.840+0000 7f2ba2c55640 1 -- 172.21.15.60:0/3811227741 wait complete. 2024-04-25T10:11:39.865 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:11:39.866 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:11:39.866 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:11:39.866 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:39.866 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:11:39.866 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:11:39.866 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:11:39.866 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:39.867 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:11:39.867 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:11:39.867 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:11:39.867 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:39.867 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:11:39.867 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:11:39.867 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:11:39.868 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:39.876 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:11:39.877 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:11:39.878 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:11:39.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:11:39.885 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:11:39.885 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:11:39.887 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:11:39.887 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:11:39.924 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:11:39.924 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:39.925 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:11:39.925 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:39.930 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:11:39.931 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:39.932 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:11:39.932 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:39.932 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:11:39.988 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T10:11:39.988 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:11:39.999 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T10:11:40.001 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T10:11:40.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.048+0000 7f29dfdd9640 1 Processor -- start 2024-04-25T10:11:40.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.048+0000 7f29dfdd9640 1 -- start start 2024-04-25T10:11:40.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.048+0000 7f29dfdd9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f29d815bc90 0x7f29d815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.048+0000 7f29dfdd9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d8071ca0 con 0x7f29d80b4660 2024-04-25T10:11:40.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.048+0000 7f29dfdd9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d80cca40 con 0x7f29d80ce7a0 2024-04-25T10:11:40.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.048+0000 7f29dfdd9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d80cc370 con 0x7f29d815bc90 2024-04-25T10:11:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.048+0000 7f29dd374640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f29d815bc90 0x7f29d815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.048+0000 7f29de376640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f29d80b4660 0x7f29d815c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50740/0 (socket says 172.21.15.60:50740) 2024-04-25T10:11:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.048+0000 7f29de376640 1 -- 172.21.15.60:0/3092327710 learned_addr learned my addr 172.21.15.60:0/3092327710 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.048+0000 7f29dd374640 1 -- 172.21.15.60:0/3092327710 >> v1:172.21.15.100:6789/0 conn(0x7f29d80ce7a0 legacy=0x7f29d80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.048+0000 7f29dcb73640 1 -- 172.21.15.60:0/3092327710 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507301206 0 0) 0x7f29d8071ca0 con 0x7f29d80b4660 2024-04-25T10:11:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.048+0000 7f29dd374640 1 -- 172.21.15.60:0/3092327710 >> v1:172.21.15.60:6789/0 conn(0x7f29d80b4660 legacy=0x7f29d815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.048+0000 7f29dd374640 1 -- 172.21.15.60:0/3092327710 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d80ccd20 con 0x7f29d815bc90 2024-04-25T10:11:40.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dd374640 1 --2- 172.21.15.60:0/3092327710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29d815bc90 0x7f29d815c220 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f29c80097c0 tx=0x7f29c801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f013933369ffafc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:40.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dcb73640 1 -- 172.21.15.60:0/3092327710 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c80b0070 con 0x7f29d815bc90 2024-04-25T10:11:40.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dcb73640 1 -- 172.21.15.60:0/3092327710 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c80ab600 con 0x7f29d815bc90 2024-04-25T10:11:40.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dcb73640 1 -- 172.21.15.60:0/3092327710 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c80ab9d0 con 0x7f29d815bc90 2024-04-25T10:11:40.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dfdd9640 1 -- 172.21.15.60:0/3092327710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29d815bc90 msgr2=0x7f29d815c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dfdd9640 1 --2- 172.21.15.60:0/3092327710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29d815bc90 0x7f29d815c220 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f29c80097c0 tx=0x7f29c801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dfdd9640 1 -- 172.21.15.60:0/3092327710 shutdown_connections 2024-04-25T10:11:40.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dfdd9640 1 --2- 172.21.15.60:0/3092327710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29d815bc90 0x7f29d815c220 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dfdd9640 1 -- 172.21.15.60:0/3092327710 >> 172.21.15.60:0/3092327710 conn(0x7f29d80b12d0 msgr2=0x7f29d80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:40.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dfdd9640 1 -- 172.21.15.60:0/3092327710 shutdown_connections 2024-04-25T10:11:40.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dfdd9640 1 -- 172.21.15.60:0/3092327710 wait complete. 2024-04-25T10:11:40.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dfdd9640 1 Processor -- start 2024-04-25T10:11:40.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dfdd9640 1 -- start start 2024-04-25T10:11:40.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dfdd9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29d80ce7a0 0x7f29d814b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dfdd9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d815bc90 0x7f29d814de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dfdd9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29d814e3d0 0x7f29d81499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dfdd9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d81695d0 con 0x7f29d80ce7a0 2024-04-25T10:11:40.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dfdd9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d80b3db0 con 0x7f29d815bc90 2024-04-25T10:11:40.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dfdd9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d8169750 con 0x7f29d814e3d0 2024-04-25T10:11:40.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29ddb75640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29d80ce7a0 0x7f29d814b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29ddb75640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29d80ce7a0 0x7f29d814b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45398/0 (socket says 172.21.15.60:45398) 2024-04-25T10:11:40.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29ddb75640 1 -- 172.21.15.60:0/1197209502 learned_addr learned my addr 172.21.15.60:0/1197209502 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:40.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29de376640 1 --2- 172.21.15.60:0/1197209502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29d814e3d0 0x7f29d81499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.052+0000 7f29dd374640 1 --2- 172.21.15.60:0/1197209502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d815bc90 0x7f29d814de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29ddb75640 1 -- 172.21.15.60:0/1197209502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29d814e3d0 msgr2=0x7f29d81499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29ddb75640 1 --2- 172.21.15.60:0/1197209502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29d814e3d0 0x7f29d81499c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29ddb75640 1 -- 172.21.15.60:0/1197209502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d815bc90 msgr2=0x7f29d814de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29ddb75640 1 --2- 172.21.15.60:0/1197209502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d815bc90 0x7f29d814de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29ddb75640 1 -- 172.21.15.60:0/1197209502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d814ebf0 con 0x7f29d80ce7a0 2024-04-25T10:11:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29ddb75640 1 --2- 172.21.15.60:0/1197209502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29d80ce7a0 0x7f29d814b680 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f29d400c7a0 tx=0x7f29d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29ce7fc640 1 -- 172.21.15.60:0/1197209502 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4003b80 con 0x7f29d80ce7a0 2024-04-25T10:11:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29ce7fc640 1 -- 172.21.15.60:0/1197209502 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d4003d20 con 0x7f29d80ce7a0 2024-04-25T10:11:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29dfdd9640 1 -- 172.21.15.60:0/1197209502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d8150d30 con 0x7f29d80ce7a0 2024-04-25T10:11:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29ce7fc640 1 -- 172.21.15.60:0/1197209502 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d400aa90 con 0x7f29d80ce7a0 2024-04-25T10:11:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29dfdd9640 1 -- 172.21.15.60:0/1197209502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d8156160 con 0x7f29d80ce7a0 2024-04-25T10:11:40.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29dfdd9640 1 -- 172.21.15.60:0/1197209502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d8155f80 con 0x7f29d80ce7a0 2024-04-25T10:11:40.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29ce7fc640 1 -- 172.21.15.60:0/1197209502 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29d400ac30 con 0x7f29d80ce7a0 2024-04-25T10:11:40.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29ce7fc640 1 --2- 172.21.15.60:0/1197209502 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f29bc066f60 0x7f29bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29ce7fc640 1 -- 172.21.15.60:0/1197209502 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f29d4084bb0 con 0x7f29d80ce7a0 2024-04-25T10:11:40.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29dd374640 1 --2- 172.21.15.60:0/1197209502 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f29bc066f60 0x7f29bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.056+0000 7f29dd374640 1 --2- 172.21.15.60:0/1197209502 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f29bc066f60 0x7f29bc069420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f29c80ad000 tx=0x7f29c8004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:40.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.060+0000 7f29ce7fc640 1 -- 172.21.15.60:0/1197209502 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f29d8155f80 con 0x7f29d80ce7a0 2024-04-25T10:11:40.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.232+0000 7f29dfdd9640 1 -- 172.21.15.60:0/1197209502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f29d814e7e0 con 0x7f29d80ce7a0 2024-04-25T10:11:40.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.964+0000 7f29ce7fc640 1 -- 172.21.15.60:0/1197209502 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v76) v1 ==== 97+0+0 (secure 0 0 0) 0x7f29d814e7e0 con 0x7f29d80ce7a0 2024-04-25T10:11:40.973 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-04-25T10:11:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.968+0000 7f29dfdd9640 1 -- 172.21.15.60:0/1197209502 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f29bc066f60 msgr2=0x7f29bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.968+0000 7f29dfdd9640 1 --2- 172.21.15.60:0/1197209502 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f29bc066f60 0x7f29bc069420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f29c80ad000 tx=0x7f29c8004740 comp rx=0 tx=0).stop 2024-04-25T10:11:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.968+0000 7f29dfdd9640 1 -- 172.21.15.60:0/1197209502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29d80ce7a0 msgr2=0x7f29d814b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.968+0000 7f29dfdd9640 1 --2- 172.21.15.60:0/1197209502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29d80ce7a0 0x7f29d814b680 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f29d400c7a0 tx=0x7f29d400cc60 comp rx=0 tx=0).stop 2024-04-25T10:11:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.968+0000 7f29dfdd9640 1 -- 172.21.15.60:0/1197209502 shutdown_connections 2024-04-25T10:11:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.968+0000 7f29dfdd9640 1 --2- 172.21.15.60:0/1197209502 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f29bc066f60 0x7f29bc069420 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.968+0000 7f29dfdd9640 1 --2- 172.21.15.60:0/1197209502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29d814e3d0 0x7f29d81499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.968+0000 7f29dfdd9640 1 --2- 172.21.15.60:0/1197209502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d815bc90 0x7f29d814de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.968+0000 7f29dfdd9640 1 --2- 172.21.15.60:0/1197209502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29d80ce7a0 0x7f29d814b680 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.968+0000 7f29dfdd9640 1 -- 172.21.15.60:0/1197209502 >> 172.21.15.60:0/1197209502 conn(0x7f29d80b12d0 msgr2=0x7f29d80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:40.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.968+0000 7f29dfdd9640 1 -- 172.21.15.60:0/1197209502 shutdown_connections 2024-04-25T10:11:40.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:40.968+0000 7f29dfdd9640 1 -- 172.21.15.60:0/1197209502 wait complete. 2024-04-25T10:11:40.998 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:11:41.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.112+0000 7f3f875ea640 1 Processor -- start 2024-04-25T10:11:41.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.112+0000 7f3f875ea640 1 -- start start 2024-04-25T10:11:41.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f875ea640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3f800ce7a0 0x7f3f800ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f875ea640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3f800b4660 0x7f3f800b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f875ea640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f800cf0c0 con 0x7f3f8015bc90 2024-04-25T10:11:41.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f875ea640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f800cc370 con 0x7f3f800ce7a0 2024-04-25T10:11:41.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f875ea640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f80071ca0 con 0x7f3f800b4660 2024-04-25T10:11:41.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f85386640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3f800ce7a0 0x7f3f800ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f85386640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3f800ce7a0 0x7f3f800ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45412/0 (socket says 172.21.15.60:45412) 2024-04-25T10:11:41.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f85386640 1 -- 172.21.15.60:0/3196828138 learned_addr learned my addr 172.21.15.60:0/3196828138 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:41.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f84b85640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3f8015bc90 0x7f3f800b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50754/0 (socket says 172.21.15.60:50754) 2024-04-25T10:11:41.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f85386640 1 -- 172.21.15.60:0/3196828138 >> v2:172.21.15.100:3300/0 conn(0x7f3f800b4660 msgr2=0x7f3f800b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:41.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f85386640 1 --2- 172.21.15.60:0/3196828138 >> v2:172.21.15.100:3300/0 conn(0x7f3f800b4660 0x7f3f800b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f85386640 1 -- 172.21.15.60:0/3196828138 >> v1:172.21.15.60:6789/0 conn(0x7f3f8015bc90 legacy=0x7f3f800b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f85386640 1 -- 172.21.15.60:0/3196828138 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f800cca40 con 0x7f3f800ce7a0 2024-04-25T10:11:41.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f85386640 1 --2- 172.21.15.60:0/3196828138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f800ce7a0 0x7f3f800ceb80 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f3f8006cf50 tx=0x7f3f7c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ba77eca146ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:41.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f77fff640 1 -- 172.21.15.60:0/3196828138 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0a56c0 con 0x7f3f800ce7a0 2024-04-25T10:11:41.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f77fff640 1 -- 172.21.15.60:0/3196828138 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7c0a5860 con 0x7f3f800ce7a0 2024-04-25T10:11:41.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.116+0000 7f3f77fff640 1 -- 172.21.15.60:0/3196828138 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0a5c30 con 0x7f3f800ce7a0 2024-04-25T10:11:41.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 -- 172.21.15.60:0/3196828138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f800ce7a0 msgr2=0x7f3f800ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 --2- 172.21.15.60:0/3196828138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f800ce7a0 0x7f3f800ceb80 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f3f8006cf50 tx=0x7f3f7c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 -- 172.21.15.60:0/3196828138 shutdown_connections 2024-04-25T10:11:41.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 --2- 172.21.15.60:0/3196828138 >> v2:172.21.15.100:3300/0 conn(0x7f3f800b4660 0x7f3f800b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 --2- 172.21.15.60:0/3196828138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f800ce7a0 0x7f3f800ceb80 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 -- 172.21.15.60:0/3196828138 >> 172.21.15.60:0/3196828138 conn(0x7f3f800b12d0 msgr2=0x7f3f800b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:41.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 -- 172.21.15.60:0/3196828138 shutdown_connections 2024-04-25T10:11:41.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 -- 172.21.15.60:0/3196828138 wait complete. 2024-04-25T10:11:41.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 Processor -- start 2024-04-25T10:11:41.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 -- start start 2024-04-25T10:11:41.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f800ce7a0 0x7f3f80161d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8015bc90 0x7f3f80156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80156bd0 0x7f3f8015f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800b4400 con 0x7f3f80156bd0 2024-04-25T10:11:41.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800b4280 con 0x7f3f8015bc90 2024-04-25T10:11:41.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800b3220 con 0x7f3f800ce7a0 2024-04-25T10:11:41.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f85b87640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80156bd0 0x7f3f8015f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f85b87640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80156bd0 0x7f3f8015f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45414/0 (socket says 172.21.15.60:45414) 2024-04-25T10:11:41.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f85b87640 1 -- 172.21.15.60:0/2369031886 learned_addr learned my addr 172.21.15.60:0/2369031886 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:41.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f84b85640 1 --2- 172.21.15.60:0/2369031886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8015bc90 0x7f3f80156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f85386640 1 --2- 172.21.15.60:0/2369031886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f800ce7a0 0x7f3f80161d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f85b87640 1 -- 172.21.15.60:0/2369031886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f800ce7a0 msgr2=0x7f3f80161d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f85b87640 1 --2- 172.21.15.60:0/2369031886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f800ce7a0 0x7f3f80161d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f85b87640 1 -- 172.21.15.60:0/2369031886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8015bc90 msgr2=0x7f3f80156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f85b87640 1 --2- 172.21.15.60:0/2369031886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8015bc90 0x7f3f80156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f85b87640 1 -- 172.21.15.60:0/2369031886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f80160080 con 0x7f3f80156bd0 2024-04-25T10:11:41.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f85b87640 1 --2- 172.21.15.60:0/2369031886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80156bd0 0x7f3f8015f310 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800b430 tx=0x7f3f7800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:41.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f75ffb640 1 -- 172.21.15.60:0/2369031886 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78015020 con 0x7f3f80156bd0 2024-04-25T10:11:41.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f75ffb640 1 -- 172.21.15.60:0/2369031886 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f78011830 con 0x7f3f80156bd0 2024-04-25T10:11:41.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 -- 172.21.15.60:0/2369031886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f80160360 con 0x7f3f80156bd0 2024-04-25T10:11:41.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f75ffb640 1 -- 172.21.15.60:0/2369031886 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78011c20 con 0x7f3f80156bd0 2024-04-25T10:11:41.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.120+0000 7f3f875ea640 1 -- 172.21.15.60:0/2369031886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f80160920 con 0x7f3f80156bd0 2024-04-25T10:11:41.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.124+0000 7f3f875ea640 1 -- 172.21.15.60:0/2369031886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f800af0c0 con 0x7f3f80156bd0 2024-04-25T10:11:41.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.124+0000 7f3f75ffb640 1 -- 172.21.15.60:0/2369031886 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f78018020 con 0x7f3f80156bd0 2024-04-25T10:11:41.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.124+0000 7f3f75ffb640 1 --2- 172.21.15.60:0/2369031886 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3f64066f60 0x7f3f64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.124+0000 7f3f75ffb640 1 -- 172.21.15.60:0/2369031886 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3f78084c20 con 0x7f3f80156bd0 2024-04-25T10:11:41.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.124+0000 7f3f85386640 1 --2- 172.21.15.60:0/2369031886 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3f64066f60 0x7f3f64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.124+0000 7f3f85386640 1 --2- 172.21.15.60:0/2369031886 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3f64066f60 0x7f3f64069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0ae000 tx=0x7f3f7c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:41.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.128+0000 7f3f75ffb640 1 -- 172.21.15.60:0/2369031886 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f7801f020 con 0x7f3f80156bd0 2024-04-25T10:11:41.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.336+0000 7f3f875ea640 1 -- 172.21.15.60:0/2369031886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3f80074770 con 0x7f3f80156bd0 2024-04-25T10:11:41.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.336+0000 7f3f75ffb640 1 -- 172.21.15.60:0/2369031886 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3f780172c0 con 0x7f3f80156bd0 2024-04-25T10:11:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:41.347 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:11:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-04-25T10:11:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-04-25T10:11:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmKr12EhAAnHHZ5VjG0cywMrIv6VElVg==", 2024-04-25T10:11:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:11:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:11:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:11:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:11:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-04-25T10:11:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmU39rIRAAk6Ca0tKG4LpmR4pNxjzv0w==", 2024-04-25T10:11:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:11:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:11:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:11:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:11:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-04-25T10:11:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmVsshGxAAttXv0jYEGKgo0sPhIRTuqQ==", 2024-04-25T10:11:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:11:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:11:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:11:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:11:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-04-25T10:11:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmQtXUKRAAI3Iq02h3KH/PZbWKvJD9Jg==", 2024-04-25T10:11:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:11:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:11:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:11:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:11:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCkKipmE+UIBRAAc8ctpHoL3PkkbkMRCAJUGw==", 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-04-25T10:11:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCnKipmbAnYCBAALLsPH2FjUpjoIZwTw+8aOA==", 2024-04-25T10:11:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:11:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:11:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-04-25T10:11:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCqKipmqrNSDxAAZdgFrcyeVwRQMKJJYIM98g==", 2024-04-25T10:11:41.353 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.353 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:41.353 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:11:41.353 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:11:41.353 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.353 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-04-25T10:11:41.353 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCtKipm1L+hFxAAPnPWlCm1Fmi0mN3tyReeQw==", 2024-04-25T10:11:41.354 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.354 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:41.354 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:11:41.354 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:11:41.354 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.354 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-04-25T10:11:41.354 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCzKipm2WyDFRAAmyfg6qu5drE9yMaR2FKd/A==", 2024-04-25T10:11:41.354 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.355 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:41.355 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:11:41.355 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:11:41.355 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.355 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-04-25T10:11:41.355 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC2KipmZLDhGRAAcgaz8YaPGUMIfxOLtWObyA==", 2024-04-25T10:11:41.355 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.355 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:41.356 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:11:41.356 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:11:41.356 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.356 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-04-25T10:11:41.356 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC5Kipm/3VMIBAA+rhGCyA/7d+s8m7ZWTzX/Q==", 2024-04-25T10:11:41.356 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.356 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:41.356 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:11:41.357 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:11:41.357 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.357 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-04-25T10:11:41.357 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8KipmFNCCJxAAv0OSTpAG+xIHhLqYXAZazQ==", 2024-04-25T10:11:41.357 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.357 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:41.357 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:11:41.357 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:11:41.357 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.358 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-04-25T10:11:41.358 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmv65dLxAA6/QpPrqE9aT+CtZqkyutog==", 2024-04-25T10:11:41.358 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.358 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:11:41.358 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-04-25T10:11:41.358 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:11:41.358 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.358 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-04-25T10:11:41.358 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipm2NiONxAAd3X4zx1JnBXKHJvdZ/zMdQ==", 2024-04-25T10:11:41.358 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.358 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:11:41.358 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:11:41.359 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:11:41.359 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:11:41.359 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.359 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-04-25T10:11:41.359 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmZRONNBAAa+XTv5YIBJWeUvf5dfoWxw==", 2024-04-25T10:11:41.359 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.359 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:11:41.359 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:11:41.359 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:11:41.359 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:11:41.359 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.360 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-04-25T10:11:41.360 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCjKipmgvfiABAApLvTC9U6HzeAefCvX4tiSQ==", 2024-04-25T10:11:41.360 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.360 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:11:41.360 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:11:41.360 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:11:41.360 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:11:41.360 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.360 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-04-25T10:11:41.360 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQChKipmumzYNhAAlCRYMpSNaTo7bz8P6x0W9g==", 2024-04-25T10:11:41.361 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.361 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:11:41.361 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:11:41.361 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:11:41.361 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:11:41.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.361 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-04-25T10:11:41.361 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmhxlLDxAAegBrJWdMNz6ZC3oXT2Qo0A==", 2024-04-25T10:11:41.361 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.361 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:11:41.362 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T10:11:41.362 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:11:41.362 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:41.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:41.362 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-04-25T10:11:41.362 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmpl1cDBAAGqx2kIak9NoWpkcdHczcFA==", 2024-04-25T10:11:41.362 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:11:41.362 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:11:41.362 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T10:11:41.362 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:11:41.362 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.362 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:41.363 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T10:11:41.363 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:11:41.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.340+0000 7f3f875ea640 1 -- 172.21.15.60:0/2369031886 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3f64066f60 msgr2=0x7f3f64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.340+0000 7f3f875ea640 1 --2- 172.21.15.60:0/2369031886 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3f64066f60 0x7f3f64069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0ae000 tx=0x7f3f7c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:11:41.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.340+0000 7f3f875ea640 1 -- 172.21.15.60:0/2369031886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80156bd0 msgr2=0x7f3f8015f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.340+0000 7f3f875ea640 1 --2- 172.21.15.60:0/2369031886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80156bd0 0x7f3f8015f310 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800b430 tx=0x7f3f7800b8f0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.340+0000 7f3f875ea640 1 -- 172.21.15.60:0/2369031886 shutdown_connections 2024-04-25T10:11:41.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.340+0000 7f3f875ea640 1 --2- 172.21.15.60:0/2369031886 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3f64066f60 0x7f3f64069420 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.340+0000 7f3f875ea640 1 --2- 172.21.15.60:0/2369031886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80156bd0 0x7f3f8015f310 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.340+0000 7f3f875ea640 1 --2- 172.21.15.60:0/2369031886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8015bc90 0x7f3f80156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.340+0000 7f3f875ea640 1 --2- 172.21.15.60:0/2369031886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f800ce7a0 0x7f3f80161d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.340+0000 7f3f875ea640 1 -- 172.21.15.60:0/2369031886 >> 172.21.15.60:0/2369031886 conn(0x7f3f800b12d0 msgr2=0x7f3f8015b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:41.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.340+0000 7f3f875ea640 1 -- 172.21.15.60:0/2369031886 shutdown_connections 2024-04-25T10:11:41.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.340+0000 7f3f875ea640 1 -- 172.21.15.60:0/2369031886 wait complete. 2024-04-25T10:11:41.365 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.472+0000 7f795dbf8640 1 Processor -- start 2024-04-25T10:11:41.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.472+0000 7f795dbf8640 1 -- start start 2024-04-25T10:11:41.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f795dbf8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f79580ce7a0 0x7f79580ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f795dbf8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79580b33f0 con 0x7f795815bc90 2024-04-25T10:11:41.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f795dbf8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79580cf0c0 con 0x7f79580b4660 2024-04-25T10:11:41.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f795dbf8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79580cc370 con 0x7f79580ce7a0 2024-04-25T10:11:41.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f7956ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f795815bc90 0x7f79580b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50770/0 (socket says 172.21.15.60:50770) 2024-04-25T10:11:41.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f7956ffd640 1 -- 172.21.15.60:0/2592082820 learned_addr learned my addr 172.21.15.60:0/2592082820 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:41.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79567fc640 1 -- 172.21.15.60:0/2592082820 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 990326868 0 0) 0x7f79580b33f0 con 0x7f795815bc90 2024-04-25T10:11:41.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79577fe640 1 --2- 172.21.15.60:0/2592082820 >> v2:172.21.15.100:3300/0 conn(0x7f79580ce7a0 0x7f79580ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79567fc640 1 -- 172.21.15.60:0/2592082820 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7940003530 con 0x7f795815bc90 2024-04-25T10:11:41.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79567fc640 1 -- 172.21.15.60:0/2592082820 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2113205064 0 0) 0x7f7940003530 con 0x7f795815bc90 2024-04-25T10:11:41.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79567fc640 1 -- 172.21.15.60:0/2592082820 >> v2:172.21.15.100:3300/0 conn(0x7f79580ce7a0 msgr2=0x7f79580ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:41.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79567fc640 1 --2- 172.21.15.60:0/2592082820 >> v2:172.21.15.100:3300/0 conn(0x7f79580ce7a0 0x7f79580ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79567fc640 1 -- 172.21.15.60:0/2592082820 >> v1:172.21.15.100:6789/0 conn(0x7f79580b4660 legacy=0x7f79580b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79567fc640 1 -- 172.21.15.60:0/2592082820 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7958071ca0 con 0x7f795815bc90 2024-04-25T10:11:41.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79567fc640 1 -- 172.21.15.60:0/2592082820 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f794c001680 con 0x7f795815bc90 2024-04-25T10:11:41.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79567fc640 1 -- 172.21.15.60:0/2592082820 >> v1:172.21.15.60:6789/0 conn(0x7f795815bc90 legacy=0x7f79580b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79567fc640 1 --2- 172.21.15.60:0/2592082820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79400039c0 0x7f7940003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79567fc640 1 --2- 172.21.15.60:0/2592082820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7940004810 0x7f7940004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79567fc640 1 --2- 172.21.15.60:0/2592082820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79400051d0 0x7f794000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79567fc640 1 -- 172.21.15.60:0/2592082820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794000c040 con 0x7f79400039c0 2024-04-25T10:11:41.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79567fc640 1 -- 172.21.15.60:0/2592082820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794000c1f0 con 0x7f7940004810 2024-04-25T10:11:41.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79567fc640 1 -- 172.21.15.60:0/2592082820 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794000c3f0 con 0x7f79400051d0 2024-04-25T10:11:41.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79577fe640 1 --2- 172.21.15.60:0/2592082820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79400039c0 0x7f7940003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f7956ffd640 1 --2- 172.21.15.60:0/2592082820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7940004810 0x7f7940004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f7957fff640 1 --2- 172.21.15.60:0/2592082820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79400051d0 0x7f794000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79577fe640 1 -- 172.21.15.60:0/2592082820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79400051d0 msgr2=0x7f794000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:41.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79577fe640 1 --2- 172.21.15.60:0/2592082820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79400051d0 0x7f794000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79577fe640 1 -- 172.21.15.60:0/2592082820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7940004810 msgr2=0x7f7940004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:41.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79577fe640 1 --2- 172.21.15.60:0/2592082820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7940004810 0x7f7940004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f7957fff640 1 -- 172.21.15.60:0/2592082820 reap_dead start 2024-04-25T10:11:41.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.476+0000 7f79577fe640 1 -- 172.21.15.60:0/2592082820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7940003530 con 0x7f79400039c0 2024-04-25T10:11:41.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f79577fe640 1 --2- 172.21.15.60:0/2592082820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79400039c0 0x7f7940003de0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f7948009ae0 tx=0x7f794801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2bdc8dc08c85c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:41.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f79567fc640 1 -- 172.21.15.60:0/2592082820 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79480096b0 con 0x7f79400039c0 2024-04-25T10:11:41.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f79567fc640 1 -- 172.21.15.60:0/2592082820 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79480a6420 con 0x7f79400039c0 2024-04-25T10:11:41.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f79567fc640 1 -- 172.21.15.60:0/2592082820 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79480a6810 con 0x7f79400039c0 2024-04-25T10:11:41.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f795dbf8640 1 -- 172.21.15.60:0/2592082820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79400039c0 msgr2=0x7f7940003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f795dbf8640 1 --2- 172.21.15.60:0/2592082820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79400039c0 0x7f7940003de0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f7948009ae0 tx=0x7f794801ca70 comp rx=0 tx=0).stop 2024-04-25T10:11:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f795dbf8640 1 -- 172.21.15.60:0/2592082820 shutdown_connections 2024-04-25T10:11:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f795dbf8640 1 --2- 172.21.15.60:0/2592082820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79400039c0 0x7f7940003de0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f795dbf8640 1 -- 172.21.15.60:0/2592082820 >> 172.21.15.60:0/2592082820 conn(0x7f79580b12d0 msgr2=0x7f79580b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f795dbf8640 1 -- 172.21.15.60:0/2592082820 shutdown_connections 2024-04-25T10:11:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f795dbf8640 1 -- 172.21.15.60:0/2592082820 wait complete. 2024-04-25T10:11:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f795dbf8640 1 Processor -- start 2024-04-25T10:11:41.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f795dbf8640 1 -- start start 2024-04-25T10:11:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f795dbf8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79400039c0 0x7f795815a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f795dbf8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7958148cc0 0x7f7958149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f795dbf8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795806d4f0 0x7f795806d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f795dbf8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79580b3d20 con 0x7f795806d4f0 2024-04-25T10:11:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.480+0000 7f795dbf8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79580b3ba0 con 0x7f7958148cc0 2024-04-25T10:11:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f795dbf8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79580b4280 con 0x7f79400039c0 2024-04-25T10:11:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f7957fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795806d4f0 0x7f795806d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f7957fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795806d4f0 0x7f795806d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45426/0 (socket says 172.21.15.60:45426) 2024-04-25T10:11:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f7957fff640 1 -- 172.21.15.60:0/3501835661 learned_addr learned my addr 172.21.15.60:0/3501835661 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f79577fe640 1 --2- 172.21.15.60:0/3501835661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79400039c0 0x7f795815a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f7956ffd640 1 --2- 172.21.15.60:0/3501835661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7958148cc0 0x7f7958149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f7957fff640 1 -- 172.21.15.60:0/3501835661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79400039c0 msgr2=0x7f795815a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f7957fff640 1 --2- 172.21.15.60:0/3501835661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79400039c0 0x7f795815a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f7957fff640 1 -- 172.21.15.60:0/3501835661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7958148cc0 msgr2=0x7f7958149140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f7957fff640 1 --2- 172.21.15.60:0/3501835661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7958148cc0 0x7f7958149140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f7957fff640 1 -- 172.21.15.60:0/3501835661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795806f2c0 con 0x7f795806d4f0 2024-04-25T10:11:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f7956ffd640 1 --2- 172.21.15.60:0/3501835661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7958148cc0 0x7f7958149140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f7957fff640 1 --2- 172.21.15.60:0/3501835661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795806d4f0 0x7f795806d9a0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f7958072bf0 tx=0x7f795000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f793ffff640 1 -- 172.21.15.60:0/3501835661 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7950018070 con 0x7f795806d4f0 2024-04-25T10:11:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f793ffff640 1 -- 172.21.15.60:0/3501835661 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7950003b80 con 0x7f795806d4f0 2024-04-25T10:11:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f795dbf8640 1 -- 172.21.15.60:0/3501835661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7958149680 con 0x7f795806d4f0 2024-04-25T10:11:41.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f793ffff640 1 -- 172.21.15.60:0/3501835661 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7950005070 con 0x7f795806d4f0 2024-04-25T10:11:41.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f795dbf8640 1 -- 172.21.15.60:0/3501835661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795806e500 con 0x7f795806d4f0 2024-04-25T10:11:41.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f795dbf8640 1 -- 172.21.15.60:0/3501835661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7958074770 con 0x7f795806d4f0 2024-04-25T10:11:41.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f793ffff640 1 -- 172.21.15.60:0/3501835661 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f795001b070 con 0x7f795806d4f0 2024-04-25T10:11:41.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f793ffff640 1 --2- 172.21.15.60:0/3501835661 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7944066f60 0x7f7944069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f793ffff640 1 -- 172.21.15.60:0/3501835661 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f79500842b0 con 0x7f795806d4f0 2024-04-25T10:11:41.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.484+0000 7f79577fe640 1 --2- 172.21.15.60:0/3501835661 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7944066f60 0x7f7944069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.488+0000 7f79577fe640 1 --2- 172.21.15.60:0/3501835661 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7944066f60 0x7f7944069420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f79480ad000 tx=0x7f79480a9040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.488+0000 7f793ffff640 1 -- 172.21.15.60:0/3501835661 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7950054fc0 con 0x7f795806d4f0 2024-04-25T10:11:41.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.648+0000 7f795dbf8640 1 -- 172.21.15.60:0/3501835661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79580b33a0 con 0x7f795806d4f0 2024-04-25T10:11:41.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.652+0000 7f793ffff640 1 -- 172.21.15.60:0/3501835661 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+12904 (secure 0 0 0) 0x7f7950059060 con 0x7f795806d4f0 2024-04-25T10:11:41.659 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:41.659 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":78,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:11:40.981044+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:41.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.652+0000 7f795dbf8640 1 -- 172.21.15.60:0/3501835661 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7944066f60 msgr2=0x7f7944069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.652+0000 7f795dbf8640 1 --2- 172.21.15.60:0/3501835661 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7944066f60 0x7f7944069420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f79480ad000 tx=0x7f79480a9040 comp rx=0 tx=0).stop 2024-04-25T10:11:41.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.652+0000 7f795dbf8640 1 -- 172.21.15.60:0/3501835661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795806d4f0 msgr2=0x7f795806d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.652+0000 7f795dbf8640 1 --2- 172.21.15.60:0/3501835661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795806d4f0 0x7f795806d9a0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f7958072bf0 tx=0x7f795000dc00 comp rx=0 tx=0).stop 2024-04-25T10:11:41.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.652+0000 7f795dbf8640 1 -- 172.21.15.60:0/3501835661 shutdown_connections 2024-04-25T10:11:41.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.652+0000 7f795dbf8640 1 --2- 172.21.15.60:0/3501835661 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7944066f60 0x7f7944069420 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.652+0000 7f795dbf8640 1 --2- 172.21.15.60:0/3501835661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795806d4f0 0x7f795806d9a0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.652+0000 7f795dbf8640 1 --2- 172.21.15.60:0/3501835661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7958148cc0 0x7f7958149140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.652+0000 7f795dbf8640 1 --2- 172.21.15.60:0/3501835661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79400039c0 0x7f795815a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.652+0000 7f795dbf8640 1 -- 172.21.15.60:0/3501835661 >> 172.21.15.60:0/3501835661 conn(0x7f79580b12d0 msgr2=0x7f795815e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:41.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.652+0000 7f795dbf8640 1 -- 172.21.15.60:0/3501835661 shutdown_connections 2024-04-25T10:11:41.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.652+0000 7f795dbf8640 1 -- 172.21.15.60:0/3501835661 wait complete. 2024-04-25T10:11:41.677 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:41.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.788+0000 7f55c260a640 1 Processor -- start 2024-04-25T10:11:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.788+0000 7f55c260a640 1 -- start start 2024-04-25T10:11:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.788+0000 7f55c260a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f55bc0cbcf0 0x7f55bc160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.788+0000 7f55c260a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc144a40 con 0x7f55bc15fbf0 2024-04-25T10:11:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.788+0000 7f55c260a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc0cc9f0 con 0x7f55bc071c30 2024-04-25T10:11:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.788+0000 7f55c260a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55bc0b2dc0 con 0x7f55bc0cbcf0 2024-04-25T10:11:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.788+0000 7f55bbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f55bc0cbcf0 0x7f55bc160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.788+0000 7f55c0ba7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f55bc15fbf0 0x7f55bc0ceec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50784/0 (socket says 172.21.15.60:50784) 2024-04-25T10:11:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.788+0000 7f55bbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f55bc0cbcf0 0x7f55bc160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45440/0 (socket says 172.21.15.60:45440) 2024-04-25T10:11:41.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.788+0000 7f55c0ba7640 1 -- 172.21.15.60:0/3707868354 learned_addr learned my addr 172.21.15.60:0/3707868354 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.788+0000 7f55bbfff640 1 -- 172.21.15.60:0/3707868354 >> v1:172.21.15.100:6789/0 conn(0x7f55bc071c30 legacy=0x7f55bc15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.788+0000 7f55baffd640 1 -- 172.21.15.60:0/3707868354 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3671180977 0 0) 0x7f55bc144a40 con 0x7f55bc15fbf0 2024-04-25T10:11:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.788+0000 7f55bbfff640 1 -- 172.21.15.60:0/3707868354 >> v1:172.21.15.60:6789/0 conn(0x7f55bc15fbf0 legacy=0x7f55bc0ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.788+0000 7f55bbfff640 1 -- 172.21.15.60:0/3707868354 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc0cccd0 con 0x7f55bc0cbcf0 2024-04-25T10:11:41.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55bbfff640 1 --2- 172.21.15.60:0/3707868354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55bc0cbcf0 0x7f55bc160080 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0097c0 tx=0x7f55ac01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41e3e0cea810aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:41.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55baffd640 1 -- 172.21.15.60:0/3707868354 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac0a7070 con 0x7f55bc0cbcf0 2024-04-25T10:11:41.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55baffd640 1 -- 172.21.15.60:0/3707868354 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac0b0070 con 0x7f55bc0cbcf0 2024-04-25T10:11:41.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55baffd640 1 -- 172.21.15.60:0/3707868354 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac0ab890 con 0x7f55bc0cbcf0 2024-04-25T10:11:41.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 -- 172.21.15.60:0/3707868354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55bc0cbcf0 msgr2=0x7f55bc160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 --2- 172.21.15.60:0/3707868354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55bc0cbcf0 0x7f55bc160080 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0097c0 tx=0x7f55ac01c470 comp rx=0 tx=0).stop 2024-04-25T10:11:41.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 -- 172.21.15.60:0/3707868354 shutdown_connections 2024-04-25T10:11:41.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 --2- 172.21.15.60:0/3707868354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55bc0cbcf0 0x7f55bc160080 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 -- 172.21.15.60:0/3707868354 >> 172.21.15.60:0/3707868354 conn(0x7f55bc0670b0 msgr2=0x7f55bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:41.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 -- 172.21.15.60:0/3707868354 shutdown_connections 2024-04-25T10:11:41.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 -- 172.21.15.60:0/3707868354 wait complete. 2024-04-25T10:11:41.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 Processor -- start 2024-04-25T10:11:41.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 -- start start 2024-04-25T10:11:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55bc071c30 0x7f55bc0aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55bc0cbcf0 0x7f55bc13cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55bc15fbf0 0x7f55bc0ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0ce740 con 0x7f55bc071c30 2024-04-25T10:11:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0ce5c0 con 0x7f55bc15fbf0 2024-04-25T10:11:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc072030 con 0x7f55bc0cbcf0 2024-04-25T10:11:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55bc071c30 0x7f55bc0aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55bc071c30 0x7f55bc0aa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45454/0 (socket says 172.21.15.60:45454) 2024-04-25T10:11:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55bbfff640 1 -- 172.21.15.60:0/2177809461 learned_addr learned my addr 172.21.15.60:0/2177809461 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55bbfff640 1 -- 172.21.15.60:0/2177809461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55bc0cbcf0 msgr2=0x7f55bc13cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55bbfff640 1 --2- 172.21.15.60:0/2177809461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55bc0cbcf0 0x7f55bc13cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c0ba7640 1 --2- 172.21.15.60:0/2177809461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55bc15fbf0 0x7f55bc0ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55bbfff640 1 -- 172.21.15.60:0/2177809461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55bc15fbf0 msgr2=0x7f55bc0ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55bbfff640 1 --2- 172.21.15.60:0/2177809461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55bc15fbf0 0x7f55bc0ad920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55bbfff640 1 -- 172.21.15.60:0/2177809461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc0ac8c0 con 0x7f55bc071c30 2024-04-25T10:11:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55bbfff640 1 --2- 172.21.15.60:0/2177809461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55bc071c30 0x7f55bc0aa4a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f55ac019080 tx=0x7f55ac01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55b8ff9640 1 -- 172.21.15.60:0/2177809461 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac0bf610 con 0x7f55bc071c30 2024-04-25T10:11:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55b8ff9640 1 -- 172.21.15.60:0/2177809461 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac0bf7b0 con 0x7f55bc071c30 2024-04-25T10:11:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 -- 172.21.15.60:0/2177809461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc0acb60 con 0x7f55bc071c30 2024-04-25T10:11:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55b8ff9640 1 -- 172.21.15.60:0/2177809461 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac0bfb80 con 0x7f55bc071c30 2024-04-25T10:11:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.792+0000 7f55c260a640 1 -- 172.21.15.60:0/2177809461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc0ad120 con 0x7f55bc071c30 2024-04-25T10:11:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.796+0000 7f55c260a640 1 -- 172.21.15.60:0/2177809461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc06bba0 con 0x7f55bc071c30 2024-04-25T10:11:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.796+0000 7f55b8ff9640 1 -- 172.21.15.60:0/2177809461 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f55ac0bd020 con 0x7f55bc071c30 2024-04-25T10:11:41.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.796+0000 7f55b8ff9640 1 --2- 172.21.15.60:0/2177809461 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f55a8066f60 0x7f55a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.796+0000 7f55b8ff9640 1 -- 172.21.15.60:0/2177809461 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f55ac11f7a0 con 0x7f55bc071c30 2024-04-25T10:11:41.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.796+0000 7f55bb7fe640 1 --2- 172.21.15.60:0/2177809461 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f55a8066f60 0x7f55a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.796+0000 7f55bb7fe640 1 --2- 172.21.15.60:0/2177809461 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f55a8066f60 0x7f55a8069420 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f55b0000f70 tx=0x7f55b00079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.800+0000 7f55b8ff9640 1 -- 172.21.15.60:0/2177809461 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f55ac016090 con 0x7f55bc071c30 2024-04-25T10:11:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.952+0000 7f55c260a640 1 -- 172.21.15.60:0/2177809461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f55bc0b2970 con 0x7f55bc071c30 2024-04-25T10:11:41.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.956+0000 7f55b8ff9640 1 -- 172.21.15.60:0/2177809461 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55ac0f0c20 con 0x7f55bc071c30 2024-04-25T10:11:41.966 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:11:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.960+0000 7f55c260a640 1 -- 172.21.15.60:0/2177809461 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f55a8066f60 msgr2=0x7f55a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.960+0000 7f55c260a640 1 --2- 172.21.15.60:0/2177809461 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f55a8066f60 0x7f55a8069420 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f55b0000f70 tx=0x7f55b00079c0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.960+0000 7f55c260a640 1 -- 172.21.15.60:0/2177809461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55bc071c30 msgr2=0x7f55bc0aa4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.960+0000 7f55c260a640 1 --2- 172.21.15.60:0/2177809461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55bc071c30 0x7f55bc0aa4a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f55ac019080 tx=0x7f55ac01acb0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.960+0000 7f55c260a640 1 -- 172.21.15.60:0/2177809461 shutdown_connections 2024-04-25T10:11:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.960+0000 7f55c260a640 1 --2- 172.21.15.60:0/2177809461 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f55a8066f60 0x7f55a8069420 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.960+0000 7f55c260a640 1 --2- 172.21.15.60:0/2177809461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55bc15fbf0 0x7f55bc0ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.960+0000 7f55c260a640 1 --2- 172.21.15.60:0/2177809461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55bc0cbcf0 0x7f55bc13cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.960+0000 7f55c260a640 1 --2- 172.21.15.60:0/2177809461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55bc071c30 0x7f55bc0aa4a0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.960+0000 7f55c260a640 1 -- 172.21.15.60:0/2177809461 >> 172.21.15.60:0/2177809461 conn(0x7f55bc0670b0 msgr2=0x7f55bc140410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:41.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.960+0000 7f55c260a640 1 -- 172.21.15.60:0/2177809461 shutdown_connections 2024-04-25T10:11:41.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:41.960+0000 7f55c260a640 1 -- 172.21.15.60:0/2177809461 wait complete. 2024-04-25T10:11:41.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:11:42.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f48147c8640 1 Processor -- start 2024-04-25T10:11:42.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f48147c8640 1 -- start start 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f48147c8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f480c15a7a0 0x7f480c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f48147c8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f480c06ec30 0x7f480c071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f48147c8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f480c0b37e0 0x7f480c0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f48147c8640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480c0cdaa0 con 0x7f480c06ec30 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f48147c8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480c06d2f0 con 0x7f480c15a7a0 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f48147c8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480c06d8b0 con 0x7f480c0b37e0 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f4811d63640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f480c06ec30 0x7f480c071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f4811d63640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f480c06ec30 0x7f480c071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45464/0 (socket says 172.21.15.60:45464) 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f4811d63640 1 -- 172.21.15.60:0/852184035 learned_addr learned my addr 172.21.15.60:0/852184035 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f4812d65640 1 --2- 172.21.15.60:0/852184035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f480c0b37e0 0x7f480c0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f4812564640 1 --2- 172.21.15.60:0/852184035 >> v2:172.21.15.100:3300/0 conn(0x7f480c15a7a0 0x7f480c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f4811d63640 1 -- 172.21.15.60:0/852184035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f480c0b37e0 msgr2=0x7f480c0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:42.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f4811d63640 1 --2- 172.21.15.60:0/852184035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f480c0b37e0 0x7f480c0b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f4811d63640 1 -- 172.21.15.60:0/852184035 >> v2:172.21.15.100:3300/0 conn(0x7f480c15a7a0 msgr2=0x7f480c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:42.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f4811d63640 1 --2- 172.21.15.60:0/852184035 >> v2:172.21.15.100:3300/0 conn(0x7f480c15a7a0 0x7f480c0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f4811d63640 1 -- 172.21.15.60:0/852184035 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c0721e0 con 0x7f480c06ec30 2024-04-25T10:11:42.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.092+0000 7f4812d65640 1 --2- 172.21.15.60:0/852184035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f480c0b37e0 0x7f480c0b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:42.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f4811d63640 1 --2- 172.21.15.60:0/852184035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c06ec30 0x7f480c071ca0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f480c04f5d0 tx=0x7f47fc01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a06b943f6bfb601 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:42.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f4811562640 1 -- 172.21.15.60:0/852184035 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc0b4070 con 0x7f480c06ec30 2024-04-25T10:11:42.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f4811562640 1 -- 172.21.15.60:0/852184035 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47fc0a7b90 con 0x7f480c06ec30 2024-04-25T10:11:42.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f4811562640 1 -- 172.21.15.60:0/852184035 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc0af440 con 0x7f480c06ec30 2024-04-25T10:11:42.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 -- 172.21.15.60:0/852184035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c06ec30 msgr2=0x7f480c071ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 --2- 172.21.15.60:0/852184035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c06ec30 0x7f480c071ca0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f480c04f5d0 tx=0x7f47fc01ebf0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 -- 172.21.15.60:0/852184035 shutdown_connections 2024-04-25T10:11:42.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 --2- 172.21.15.60:0/852184035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f480c0b37e0 0x7f480c0b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 --2- 172.21.15.60:0/852184035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c06ec30 0x7f480c071ca0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 --2- 172.21.15.60:0/852184035 >> v2:172.21.15.100:3300/0 conn(0x7f480c15a7a0 0x7f480c0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 -- 172.21.15.60:0/852184035 >> 172.21.15.60:0/852184035 conn(0x7f480c0b12d0 msgr2=0x7f480c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:42.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 -- 172.21.15.60:0/852184035 shutdown_connections 2024-04-25T10:11:42.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 -- 172.21.15.60:0/852184035 wait complete. 2024-04-25T10:11:42.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 Processor -- start 2024-04-25T10:11:42.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 -- start start 2024-04-25T10:11:42.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f480c06ec30 0x7f480c15c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f480c0b37e0 0x7f480c141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c15a7a0 0x7f480c15d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c15ad00 con 0x7f480c15a7a0 2024-04-25T10:11:42.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c15ab80 con 0x7f480c0b37e0 2024-04-25T10:11:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f48147c8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480c06e530 con 0x7f480c06ec30 2024-04-25T10:11:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f4812d65640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c15a7a0 0x7f480c15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f4812d65640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c15a7a0 0x7f480c15d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45480/0 (socket says 172.21.15.60:45480) 2024-04-25T10:11:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f4812d65640 1 -- 172.21.15.60:0/3792526897 learned_addr learned my addr 172.21.15.60:0/3792526897 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f4811d63640 1 --2- 172.21.15.60:0/3792526897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f480c0b37e0 0x7f480c141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f4812564640 1 --2- 172.21.15.60:0/3792526897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f480c06ec30 0x7f480c15c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f4812d65640 1 -- 172.21.15.60:0/3792526897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f480c06ec30 msgr2=0x7f480c15c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f4812d65640 1 --2- 172.21.15.60:0/3792526897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f480c06ec30 0x7f480c15c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f4812d65640 1 -- 172.21.15.60:0/3792526897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f480c0b37e0 msgr2=0x7f480c141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f4812d65640 1 --2- 172.21.15.60:0/3792526897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f480c0b37e0 0x7f480c141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.096+0000 7f4812d65640 1 -- 172.21.15.60:0/3792526897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c15b3a0 con 0x7f480c15a7a0 2024-04-25T10:11:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.100+0000 7f4812d65640 1 --2- 172.21.15.60:0/3792526897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c15a7a0 0x7f480c15d420 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f480400cef0 tx=0x7f480400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.100+0000 7f4802ffd640 1 -- 172.21.15.60:0/3792526897 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480401a070 con 0x7f480c15a7a0 2024-04-25T10:11:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.100+0000 7f4802ffd640 1 -- 172.21.15.60:0/3792526897 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480400d860 con 0x7f480c15a7a0 2024-04-25T10:11:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.100+0000 7f48147c8640 1 -- 172.21.15.60:0/3792526897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480c15b680 con 0x7f480c15a7a0 2024-04-25T10:11:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.100+0000 7f4802ffd640 1 -- 172.21.15.60:0/3792526897 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480400dc50 con 0x7f480c15a7a0 2024-04-25T10:11:42.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.100+0000 7f48147c8640 1 -- 172.21.15.60:0/3792526897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480c15bc40 con 0x7f480c15a7a0 2024-04-25T10:11:42.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.100+0000 7f48147c8640 1 -- 172.21.15.60:0/3792526897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f480c074770 con 0x7f480c15a7a0 2024-04-25T10:11:42.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.100+0000 7f4802ffd640 1 -- 172.21.15.60:0/3792526897 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f480401d070 con 0x7f480c15a7a0 2024-04-25T10:11:42.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.100+0000 7f4802ffd640 1 --2- 172.21.15.60:0/3792526897 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f47f0066f90 0x7f47f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.100+0000 7f4802ffd640 1 -- 172.21.15.60:0/3792526897 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4804085560 con 0x7f480c15a7a0 2024-04-25T10:11:42.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.100+0000 7f4812564640 1 --2- 172.21.15.60:0/3792526897 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f47f0066f90 0x7f47f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.100+0000 7f4812564640 1 --2- 172.21.15.60:0/3792526897 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f47f0066f90 0x7f47f0069450 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f480800d8d0 tx=0x7f4808008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:42.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.104+0000 7f4802ffd640 1 -- 172.21.15.60:0/3792526897 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4804056970 con 0x7f480c15a7a0 2024-04-25T10:11:42.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.264+0000 7f48147c8640 1 -- 172.21.15.60:0/3792526897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f480c15e330 con 0x7f480c15a7a0 2024-04-25T10:11:42.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.264+0000 7f4802ffd640 1 -- 172.21.15.60:0/3792526897 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4804056270 con 0x7f480c15a7a0 2024-04-25T10:11:42.273 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:42.273 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T10:11:42.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:11:42.273 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T10:11:42.273 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T10:11:42.273 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:42.273 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T10:11:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.264+0000 7f48147c8640 1 -- 172.21.15.60:0/3792526897 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f47f0066f90 msgr2=0x7f47f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.264+0000 7f48147c8640 1 --2- 172.21.15.60:0/3792526897 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f47f0066f90 0x7f47f0069450 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f480800d8d0 tx=0x7f4808008040 comp rx=0 tx=0).stop 2024-04-25T10:11:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.264+0000 7f48147c8640 1 -- 172.21.15.60:0/3792526897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c15a7a0 msgr2=0x7f480c15d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.264+0000 7f48147c8640 1 --2- 172.21.15.60:0/3792526897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c15a7a0 0x7f480c15d420 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f480400cef0 tx=0x7f480400a710 comp rx=0 tx=0).stop 2024-04-25T10:11:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.264+0000 7f48147c8640 1 -- 172.21.15.60:0/3792526897 shutdown_connections 2024-04-25T10:11:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.264+0000 7f48147c8640 1 --2- 172.21.15.60:0/3792526897 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f47f0066f90 0x7f47f0069450 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.264+0000 7f48147c8640 1 --2- 172.21.15.60:0/3792526897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f480c15a7a0 0x7f480c15d420 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.264+0000 7f48147c8640 1 --2- 172.21.15.60:0/3792526897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f480c0b37e0 0x7f480c141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.264+0000 7f48147c8640 1 --2- 172.21.15.60:0/3792526897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f480c06ec30 0x7f480c15c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.264+0000 7f48147c8640 1 -- 172.21.15.60:0/3792526897 >> 172.21.15.60:0/3792526897 conn(0x7f480c0b12d0 msgr2=0x7f480c164590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.264+0000 7f48147c8640 1 -- 172.21.15.60:0/3792526897 shutdown_connections 2024-04-25T10:11:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.264+0000 7f48147c8640 1 -- 172.21.15.60:0/3792526897 wait complete. 2024-04-25T10:11:42.291 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:11:42.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:11:42.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.404+0000 7f5590249640 1 Processor -- start 2024-04-25T10:11:42.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.404+0000 7f5590249640 1 -- start start 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.404+0000 7f5590249640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f558815da40 0x7f55880b95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.404+0000 7f5590249640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f558815de20 0x7f55880ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.404+0000 7f5590249640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55880d1a70 con 0x7f558815e2b0 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.404+0000 7f5590249640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558815e7f0 con 0x7f558815da40 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.404+0000 7f5590249640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558815ead0 con 0x7f558815de20 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f558dfe5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f558815da40 0x7f55880b95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f558dfe5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f558815da40 0x7f55880b95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45482/0 (socket says 172.21.15.60:45482) 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f558e7e6640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f558815e2b0 0x7f5588072340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50790/0 (socket says 172.21.15.60:50790) 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f558dfe5640 1 -- 172.21.15.60:0/2229115091 learned_addr learned my addr 172.21.15.60:0/2229115091 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:42.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f558dfe5640 1 -- 172.21.15.60:0/2229115091 >> v2:172.21.15.100:3300/0 conn(0x7f558815de20 msgr2=0x7f55880ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:42.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f558d7e4640 1 --2- 172.21.15.60:0/2229115091 >> v2:172.21.15.100:3300/0 conn(0x7f558815de20 0x7f55880ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f558dfe5640 1 --2- 172.21.15.60:0/2229115091 >> v2:172.21.15.100:3300/0 conn(0x7f558815de20 0x7f55880ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f558dfe5640 1 -- 172.21.15.60:0/2229115091 >> v1:172.21.15.60:6789/0 conn(0x7f558815e2b0 legacy=0x7f5588072340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f558cfe3640 1 -- 172.21.15.60:0/2229115091 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824300453 0 0) 0x7f55880d1a70 con 0x7f558815e2b0 2024-04-25T10:11:42.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f558dfe5640 1 -- 172.21.15.60:0/2229115091 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558815edb0 con 0x7f558815da40 2024-04-25T10:11:42.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f558d7e4640 1 --2- 172.21.15.60:0/2229115091 >> v2:172.21.15.100:3300/0 conn(0x7f558815de20 0x7f55880ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:42.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f558dfe5640 1 --2- 172.21.15.60:0/2229115091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558815da40 0x7f55880b95b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f55880b65b0 tx=0x7f558401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654ff7eb34fc9b18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:42.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f558cfe3640 1 -- 172.21.15.60:0/2229115091 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55840aa670 con 0x7f558815da40 2024-04-25T10:11:42.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f558cfe3640 1 -- 172.21.15.60:0/2229115091 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55840af070 con 0x7f558815da40 2024-04-25T10:11:42.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f558cfe3640 1 -- 172.21.15.60:0/2229115091 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55840aaa30 con 0x7f558815da40 2024-04-25T10:11:42.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f5590249640 1 -- 172.21.15.60:0/2229115091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558815da40 msgr2=0x7f55880b95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f5590249640 1 --2- 172.21.15.60:0/2229115091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558815da40 0x7f55880b95b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f55880b65b0 tx=0x7f558401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f5590249640 1 -- 172.21.15.60:0/2229115091 shutdown_connections 2024-04-25T10:11:42.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f5590249640 1 --2- 172.21.15.60:0/2229115091 >> v2:172.21.15.100:3300/0 conn(0x7f558815de20 0x7f55880ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f5590249640 1 --2- 172.21.15.60:0/2229115091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558815da40 0x7f55880b95b0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f5590249640 1 -- 172.21.15.60:0/2229115091 >> 172.21.15.60:0/2229115091 conn(0x7f55880d0470 msgr2=0x7f55880d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:42.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f5590249640 1 -- 172.21.15.60:0/2229115091 shutdown_connections 2024-04-25T10:11:42.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.408+0000 7f5590249640 1 -- 172.21.15.60:0/2229115091 wait complete. 2024-04-25T10:11:42.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f5590249640 1 Processor -- start 2024-04-25T10:11:42.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f5590249640 1 -- start start 2024-04-25T10:11:42.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f5590249640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f558815da40 0x7f55880c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f5590249640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558815de20 0x7f55880c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f5590249640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558815e2b0 0x7f558815f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f5590249640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558816a900 con 0x7f558815de20 2024-04-25T10:11:42.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f5590249640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558816a780 con 0x7f558815e2b0 2024-04-25T10:11:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f5590249640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558816aa80 con 0x7f558815da40 2024-04-25T10:11:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f558d7e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558815de20 0x7f55880c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f558d7e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558815de20 0x7f55880c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45498/0 (socket says 172.21.15.60:45498) 2024-04-25T10:11:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f558d7e4640 1 -- 172.21.15.60:0/2219757579 learned_addr learned my addr 172.21.15.60:0/2219757579 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f558e7e6640 1 --2- 172.21.15.60:0/2219757579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558815e2b0 0x7f558815f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f558dfe5640 1 --2- 172.21.15.60:0/2219757579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f558815da40 0x7f55880c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f558d7e4640 1 -- 172.21.15.60:0/2219757579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f558815da40 msgr2=0x7f55880c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f558d7e4640 1 --2- 172.21.15.60:0/2219757579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f558815da40 0x7f55880c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f558d7e4640 1 -- 172.21.15.60:0/2219757579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558815e2b0 msgr2=0x7f558815f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f558d7e4640 1 --2- 172.21.15.60:0/2219757579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558815e2b0 0x7f558815f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f558d7e4640 1 -- 172.21.15.60:0/2219757579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55880c94b0 con 0x7f558815de20 2024-04-25T10:11:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f558e7e6640 1 --2- 172.21.15.60:0/2219757579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558815e2b0 0x7f558815f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f558dfe5640 1 --2- 172.21.15.60:0/2219757579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f558815da40 0x7f55880c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f558d7e4640 1 --2- 172.21.15.60:0/2219757579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558815de20 0x7f55880c8640 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f557800c7c0 tx=0x7f557800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f557e7fc640 1 -- 172.21.15.60:0/2219757579 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5578007a30 con 0x7f558815de20 2024-04-25T10:11:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f557e7fc640 1 -- 172.21.15.60:0/2219757579 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5578007bd0 con 0x7f558815de20 2024-04-25T10:11:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f5590249640 1 -- 172.21.15.60:0/2219757579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55880c9790 con 0x7f558815de20 2024-04-25T10:11:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f557e7fc640 1 -- 172.21.15.60:0/2219757579 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55780113f0 con 0x7f558815de20 2024-04-25T10:11:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f5590249640 1 -- 172.21.15.60:0/2219757579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558815f770 con 0x7f558815de20 2024-04-25T10:11:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.412+0000 7f5590249640 1 -- 172.21.15.60:0/2219757579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5588074770 con 0x7f558815de20 2024-04-25T10:11:42.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.416+0000 7f557e7fc640 1 -- 172.21.15.60:0/2219757579 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5578011590 con 0x7f558815de20 2024-04-25T10:11:42.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.416+0000 7f557e7fc640 1 --2- 172.21.15.60:0/2219757579 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f556c066f60 0x7f556c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.416+0000 7f557e7fc640 1 -- 172.21.15.60:0/2219757579 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5578084e90 con 0x7f558815de20 2024-04-25T10:11:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.416+0000 7f558dfe5640 1 --2- 172.21.15.60:0/2219757579 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f556c066f60 0x7f556c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.416+0000 7f558dfe5640 1 --2- 172.21.15.60:0/2219757579 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f556c066f60 0x7f556c069420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f55840ae000 tx=0x7f5584008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.420+0000 7f557e7fc640 1 -- 172.21.15.60:0/2219757579 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5578015020 con 0x7f558815de20 2024-04-25T10:11:42.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.576+0000 7f5590249640 1 -- 172.21.15.60:0/2219757579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f558815fc80 con 0x7f558815de20 2024-04-25T10:11:42.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.968+0000 7f557e7fc640 1 -- 172.21.15.60:0/2219757579 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5578056280 con 0x7f558815de20 2024-04-25T10:11:42.977 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T10:11:42.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.972+0000 7f5590249640 1 -- 172.21.15.60:0/2219757579 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f556c066f60 msgr2=0x7f556c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.972+0000 7f5590249640 1 --2- 172.21.15.60:0/2219757579 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f556c066f60 0x7f556c069420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f55840ae000 tx=0x7f5584008040 comp rx=0 tx=0).stop 2024-04-25T10:11:42.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.972+0000 7f5590249640 1 -- 172.21.15.60:0/2219757579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558815de20 msgr2=0x7f55880c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.972+0000 7f5590249640 1 --2- 172.21.15.60:0/2219757579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558815de20 0x7f55880c8640 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f557800c7c0 tx=0x7f557800cc80 comp rx=0 tx=0).stop 2024-04-25T10:11:42.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.972+0000 7f5590249640 1 -- 172.21.15.60:0/2219757579 shutdown_connections 2024-04-25T10:11:42.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.972+0000 7f5590249640 1 --2- 172.21.15.60:0/2219757579 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f556c066f60 0x7f556c069420 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.972+0000 7f5590249640 1 --2- 172.21.15.60:0/2219757579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f558815e2b0 0x7f558815f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.972+0000 7f5590249640 1 --2- 172.21.15.60:0/2219757579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f558815de20 0x7f55880c8640 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.972+0000 7f5590249640 1 --2- 172.21.15.60:0/2219757579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f558815da40 0x7f55880c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.972+0000 7f5590249640 1 -- 172.21.15.60:0/2219757579 >> 172.21.15.60:0/2219757579 conn(0x7f55880d0470 msgr2=0x7f5588153630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:42.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.972+0000 7f5590249640 1 -- 172.21.15.60:0/2219757579 shutdown_connections 2024-04-25T10:11:42.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:42.972+0000 7f5590249640 1 -- 172.21.15.60:0/2219757579 wait complete. 2024-04-25T10:11:43.006 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:11:43.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.128+0000 7f1b1950b640 1 Processor -- start 2024-04-25T10:11:43.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.128+0000 7f1b1950b640 1 -- start start 2024-04-25T10:11:43.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.128+0000 7f1b1950b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1b14161250 0x7f1b140b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:43.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.128+0000 7f1b1950b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b140b5940 0x7f1b140d2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:43.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.128+0000 7f1b1950b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b14162200 con 0x7f1b14160e70 2024-04-25T10:11:43.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.128+0000 7f1b1950b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b140d1a30 con 0x7f1b14161250 2024-04-25T10:11:43.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.128+0000 7f1b1950b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b141624e0 con 0x7f1b140b5940 2024-04-25T10:11:43.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.128+0000 7f1b127fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1b14161250 0x7f1b140b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:43.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.128+0000 7f1b137fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b140b5940 0x7f1b140d2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:43.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.128+0000 7f1b12ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1b14160e70 0x7f1b14161760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:38166/0 (socket says 172.21.15.60:38166) 2024-04-25T10:11:43.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.128+0000 7f1b12ffd640 1 -- 172.21.15.60:0/2865646546 learned_addr learned my addr 172.21.15.60:0/2865646546 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:43.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.132+0000 7f1b137fe640 1 -- 172.21.15.60:0/2865646546 >> v2:172.21.15.100:3300/0 conn(0x7f1b14161250 msgr2=0x7f1b140b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:43.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.132+0000 7f1b137fe640 1 --2- 172.21.15.60:0/2865646546 >> v2:172.21.15.100:3300/0 conn(0x7f1b14161250 0x7f1b140b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:43.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.132+0000 7f1b137fe640 1 -- 172.21.15.60:0/2865646546 >> v1:172.21.15.100:6789/0 conn(0x7f1b14160e70 legacy=0x7f1b14161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:43.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.132+0000 7f1b137fe640 1 -- 172.21.15.60:0/2865646546 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b140d1c80 con 0x7f1b140b5940 2024-04-25T10:11:43.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.132+0000 7f1b127fc640 1 --2- 172.21.15.60:0/2865646546 >> v2:172.21.15.100:3300/0 conn(0x7f1b14161250 0x7f1b140b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:43.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.132+0000 7f1b137fe640 1 --2- 172.21.15.60:0/2865646546 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b140b5940 0x7f1b140d2480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009d90 tx=0x7f1b0c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b945f38c59dc1759 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:43.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.132+0000 7f1b11ffb640 1 -- 172.21.15.60:0/2865646546 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c0b2070 con 0x7f1b140b5940 2024-04-25T10:11:43.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.132+0000 7f1b11ffb640 1 -- 172.21.15.60:0/2865646546 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c0a58b0 con 0x7f1b140b5940 2024-04-25T10:11:43.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.132+0000 7f1b11ffb640 1 -- 172.21.15.60:0/2865646546 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c0a5c80 con 0x7f1b140b5940 2024-04-25T10:11:43.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b1950b640 1 -- 172.21.15.60:0/2865646546 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b140b5940 msgr2=0x7f1b140d2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:43.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b1950b640 1 --2- 172.21.15.60:0/2865646546 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b140b5940 0x7f1b140d2480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009d90 tx=0x7f1b0c01c8e0 comp rx=0 tx=0).stop 2024-04-25T10:11:43.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b1950b640 1 -- 172.21.15.60:0/2865646546 shutdown_connections 2024-04-25T10:11:43.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b1950b640 1 --2- 172.21.15.60:0/2865646546 >> v2:172.21.15.100:3300/0 conn(0x7f1b14161250 0x7f1b140b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:43.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b1950b640 1 --2- 172.21.15.60:0/2865646546 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b140b5940 0x7f1b140d2480 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:43.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b1950b640 1 -- 172.21.15.60:0/2865646546 >> 172.21.15.60:0/2865646546 conn(0x7f1b140d05d0 msgr2=0x7f1b140d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:43.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b1950b640 1 -- 172.21.15.60:0/2865646546 shutdown_connections 2024-04-25T10:11:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b1950b640 1 -- 172.21.15.60:0/2865646546 wait complete. 2024-04-25T10:11:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b1950b640 1 Processor -- start 2024-04-25T10:11:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b1950b640 1 -- start start 2024-04-25T10:11:43.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b1950b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b140b5940 0x7f1b14153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:43.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b1950b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b14160e70 0x7f1b1414c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b1950b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b14161250 0x7f1b14150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b1950b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b140d3350 con 0x7f1b14161250 2024-04-25T10:11:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b1950b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b140d31d0 con 0x7f1b140b5940 2024-04-25T10:11:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b1950b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b140d34d0 con 0x7f1b14160e70 2024-04-25T10:11:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b137fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b14161250 0x7f1b14150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b137fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b14161250 0x7f1b14150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45506/0 (socket says 172.21.15.60:45506) 2024-04-25T10:11:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b137fe640 1 -- 172.21.15.60:0/2429579004 learned_addr learned my addr 172.21.15.60:0/2429579004 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b12ffd640 1 --2- 172.21.15.60:0/2429579004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b140b5940 0x7f1b14153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b127fc640 1 --2- 172.21.15.60:0/2429579004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b14160e70 0x7f1b1414c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b137fe640 1 -- 172.21.15.60:0/2429579004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b14160e70 msgr2=0x7f1b1414c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b137fe640 1 --2- 172.21.15.60:0/2429579004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b14160e70 0x7f1b1414c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b137fe640 1 -- 172.21.15.60:0/2429579004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b140b5940 msgr2=0x7f1b14153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b137fe640 1 --2- 172.21.15.60:0/2429579004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b140b5940 0x7f1b14153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b137fe640 1 -- 172.21.15.60:0/2429579004 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b14150ac0 con 0x7f1b14161250 2024-04-25T10:11:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.136+0000 7f1b137fe640 1 --2- 172.21.15.60:0/2429579004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b14161250 0x7f1b14150350 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0b0040 tx=0x7f1b0c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.140+0000 7f1afb7fe640 1 -- 172.21.15.60:0/2429579004 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c0b2050 con 0x7f1b14161250 2024-04-25T10:11:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.140+0000 7f1afb7fe640 1 -- 172.21.15.60:0/2429579004 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c0be070 con 0x7f1b14161250 2024-04-25T10:11:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.140+0000 7f1afb7fe640 1 -- 172.21.15.60:0/2429579004 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c0ada90 con 0x7f1b14161250 2024-04-25T10:11:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.140+0000 7f1b1950b640 1 -- 172.21.15.60:0/2429579004 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b14150da0 con 0x7f1b14161250 2024-04-25T10:11:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.140+0000 7f1b1950b640 1 -- 172.21.15.60:0/2429579004 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b14151360 con 0x7f1b14161250 2024-04-25T10:11:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.140+0000 7f1b1950b640 1 -- 172.21.15.60:0/2429579004 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b14074770 con 0x7f1b14161250 2024-04-25T10:11:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.140+0000 7f1afb7fe640 1 -- 172.21.15.60:0/2429579004 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b0c0a5730 con 0x7f1b14161250 2024-04-25T10:11:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.140+0000 7f1afb7fe640 1 --2- 172.21.15.60:0/2429579004 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1b00066f60 0x7f1b00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.140+0000 7f1afb7fe640 1 -- 172.21.15.60:0/2429579004 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f1b0c11fea0 con 0x7f1b14161250 2024-04-25T10:11:43.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.140+0000 7f1b12ffd640 1 --2- 172.21.15.60:0/2429579004 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1b00066f60 0x7f1b00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:43.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.140+0000 7f1b12ffd640 1 --2- 172.21.15.60:0/2429579004 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1b00066f60 0x7f1b00069420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1b04000f70 tx=0x7f1b04007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.144+0000 7f1afb7fe640 1 -- 172.21.15.60:0/2429579004 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b0c016560 con 0x7f1b14161250 2024-04-25T10:11:43.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.304+0000 7f1b1950b640 1 -- 172.21.15.60:0/2429579004 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1b140c7250 con 0x7f1b14161250 2024-04-25T10:11:43.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.972+0000 7f1afb7fe640 1 -- 172.21.15.60:0/2429579004 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1b0c0f0ad0 con 0x7f1b14161250 2024-04-25T10:11:43.981 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T10:11:43.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.976+0000 7f1b1950b640 1 -- 172.21.15.60:0/2429579004 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1b00066f60 msgr2=0x7f1b00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:43.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.976+0000 7f1b1950b640 1 --2- 172.21.15.60:0/2429579004 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1b00066f60 0x7f1b00069420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1b04000f70 tx=0x7f1b04007e90 comp rx=0 tx=0).stop 2024-04-25T10:11:43.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.976+0000 7f1b1950b640 1 -- 172.21.15.60:0/2429579004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b14161250 msgr2=0x7f1b14150350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:43.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.976+0000 7f1b1950b640 1 --2- 172.21.15.60:0/2429579004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b14161250 0x7f1b14150350 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0b0040 tx=0x7f1b0c019680 comp rx=0 tx=0).stop 2024-04-25T10:11:43.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.976+0000 7f1b1950b640 1 -- 172.21.15.60:0/2429579004 shutdown_connections 2024-04-25T10:11:43.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.976+0000 7f1b1950b640 1 --2- 172.21.15.60:0/2429579004 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1b00066f60 0x7f1b00069420 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:43.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.976+0000 7f1b1950b640 1 --2- 172.21.15.60:0/2429579004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b14161250 0x7f1b14150350 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:43.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.976+0000 7f1b1950b640 1 --2- 172.21.15.60:0/2429579004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b14160e70 0x7f1b1414c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:43.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.976+0000 7f1b1950b640 1 --2- 172.21.15.60:0/2429579004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b140b5940 0x7f1b14153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:43.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.976+0000 7f1b1950b640 1 -- 172.21.15.60:0/2429579004 >> 172.21.15.60:0/2429579004 conn(0x7f1b140d05d0 msgr2=0x7f1b140b7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:43.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.976+0000 7f1b1950b640 1 -- 172.21.15.60:0/2429579004 shutdown_connections 2024-04-25T10:11:43.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:43.976+0000 7f1b1950b640 1 -- 172.21.15.60:0/2429579004 wait complete. 2024-04-25T10:11:44.016 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:11:44.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548f7bc640 1 Processor -- start 2024-04-25T10:11:44.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548f7bc640 1 -- start start 2024-04-25T10:11:44.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548f7bc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548815dd30 0x7f548815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548f7bc640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54880cd890 con 0x7f54880cf8a0 2024-04-25T10:11:44.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548f7bc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54880cdb70 con 0x7f54880cd3d0 2024-04-25T10:11:44.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548f7bc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54880ce4a0 con 0x7f548815dd30 2024-04-25T10:11:44.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548dd59640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f54880cf8a0 0x7f548815be20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50794/0 (socket says 172.21.15.60:50794) 2024-04-25T10:11:44.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548dd59640 1 -- 172.21.15.60:0/3088128409 learned_addr learned my addr 172.21.15.60:0/3088128409 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548d558640 1 --2- 172.21.15.60:0/3088128409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548815dd30 0x7f548815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f547bfff640 1 -- 172.21.15.60:0/3088128409 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631300536 0 0) 0x7f54880cd890 con 0x7f54880cf8a0 2024-04-25T10:11:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f547bfff640 1 -- 172.21.15.60:0/3088128409 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5474003530 con 0x7f54880cf8a0 2024-04-25T10:11:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f547bfff640 1 -- 172.21.15.60:0/3088128409 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3123068554 0 0) 0x7f5474003530 con 0x7f54880cf8a0 2024-04-25T10:11:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f547bfff640 1 -- 172.21.15.60:0/3088128409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548815dd30 msgr2=0x7f548815ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f547bfff640 1 --2- 172.21.15.60:0/3088128409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548815dd30 0x7f548815ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f547bfff640 1 -- 172.21.15.60:0/3088128409 >> v1:172.21.15.100:6789/0 conn(0x7f54880cd3d0 legacy=0x7f54880cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f547bfff640 1 -- 172.21.15.60:0/3088128409 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54880cde50 con 0x7f54880cf8a0 2024-04-25T10:11:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f547bfff640 1 -- 172.21.15.60:0/3088128409 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f5480001a00 con 0x7f54880cf8a0 2024-04-25T10:11:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f547bfff640 1 -- 172.21.15.60:0/3088128409 >> v1:172.21.15.60:6789/0 conn(0x7f54880cf8a0 legacy=0x7f548815be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f547bfff640 1 --2- 172.21.15.60:0/3088128409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548815dd30 0x7f54740039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f547bfff640 1 --2- 172.21.15.60:0/3088128409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5474003f70 0x7f5474004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f547bfff640 1 --2- 172.21.15.60:0/3088128409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5474004d50 0x7f54740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f547bfff640 1 -- 172.21.15.60:0/3088128409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54880ce4a0 con 0x7f5474003f70 2024-04-25T10:11:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f547bfff640 1 -- 172.21.15.60:0/3088128409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547400bb60 con 0x7f5474004d50 2024-04-25T10:11:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f547bfff640 1 -- 172.21.15.60:0/3088128409 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547400bd60 con 0x7f548815dd30 2024-04-25T10:11:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548cd57640 1 --2- 172.21.15.60:0/3088128409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5474003f70 0x7f5474004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548d558640 1 --2- 172.21.15.60:0/3088128409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548815dd30 0x7f54740039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548dd59640 1 --2- 172.21.15.60:0/3088128409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5474004d50 0x7f54740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548cd57640 1 -- 172.21.15.60:0/3088128409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548815dd30 msgr2=0x7f54740039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548cd57640 1 --2- 172.21.15.60:0/3088128409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548815dd30 0x7f54740039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548cd57640 1 -- 172.21.15.60:0/3088128409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5474004d50 msgr2=0x7f54740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548cd57640 1 --2- 172.21.15.60:0/3088128409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5474004d50 0x7f54740051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.140+0000 7f548cd57640 1 -- 172.21.15.60:0/3088128409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5474003530 con 0x7f5474003f70 2024-04-25T10:11:44.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.144+0000 7f548cd57640 1 --2- 172.21.15.60:0/3088128409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5474003f70 0x7f5474004810 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5484001780 tx=0x7f548401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185551c7318314c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.144+0000 7f547bfff640 1 -- 172.21.15.60:0/3088128409 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54840013a0 con 0x7f5474003f70 2024-04-25T10:11:44.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.144+0000 7f547bfff640 1 -- 172.21.15.60:0/3088128409 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54840052f0 con 0x7f5474003f70 2024-04-25T10:11:44.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.144+0000 7f547bfff640 1 -- 172.21.15.60:0/3088128409 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484003840 con 0x7f5474003f70 2024-04-25T10:11:44.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.144+0000 7f548f7bc640 1 -- 172.21.15.60:0/3088128409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5474003f70 msgr2=0x7f5474004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.144+0000 7f548f7bc640 1 --2- 172.21.15.60:0/3088128409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5474003f70 0x7f5474004810 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5484001780 tx=0x7f548401c3f0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.144+0000 7f548dd59640 1 -- 172.21.15.60:0/3088128409 reap_dead start 2024-04-25T10:11:44.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.144+0000 7f548f7bc640 1 -- 172.21.15.60:0/3088128409 shutdown_connections 2024-04-25T10:11:44.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.144+0000 7f548f7bc640 1 -- 172.21.15.60:0/3088128409 >> 172.21.15.60:0/3088128409 conn(0x7f54880670b0 msgr2=0x7f548806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.144+0000 7f548f7bc640 1 -- 172.21.15.60:0/3088128409 shutdown_connections 2024-04-25T10:11:44.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.144+0000 7f548f7bc640 1 -- 172.21.15.60:0/3088128409 wait complete. 2024-04-25T10:11:44.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.144+0000 7f548f7bc640 1 Processor -- start 2024-04-25T10:11:44.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.144+0000 7f548f7bc640 1 -- start start 2024-04-25T10:11:44.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548f7bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5474003f70 0x7f54881596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548f7bc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5488164ef0 0x7f5488145a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548f7bc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5488165350 0x7f5488145fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548f7bc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548815d0d0 con 0x7f5474003f70 2024-04-25T10:11:44.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548f7bc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548815cf50 con 0x7f5488164ef0 2024-04-25T10:11:44.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548f7bc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54881662a0 con 0x7f5488165350 2024-04-25T10:11:44.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548d558640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5474003f70 0x7f54881596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548d558640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5474003f70 0x7f54881596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45534/0 (socket says 172.21.15.60:45534) 2024-04-25T10:11:44.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548d558640 1 -- 172.21.15.60:0/419327074 learned_addr learned my addr 172.21.15.60:0/419327074 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:44.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548cd57640 1 --2- 172.21.15.60:0/419327074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5488164ef0 0x7f5488145a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548dd59640 1 --2- 172.21.15.60:0/419327074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5488165350 0x7f5488145fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548d558640 1 -- 172.21.15.60:0/419327074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5488165350 msgr2=0x7f5488145fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548d558640 1 --2- 172.21.15.60:0/419327074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5488165350 0x7f5488145fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548d558640 1 -- 172.21.15.60:0/419327074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5488164ef0 msgr2=0x7f5488145a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548d558640 1 --2- 172.21.15.60:0/419327074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5488164ef0 0x7f5488145a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548d558640 1 -- 172.21.15.60:0/419327074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488154640 con 0x7f5474003f70 2024-04-25T10:11:44.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548cd57640 1 --2- 172.21.15.60:0/419327074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5488164ef0 0x7f5488145a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:44.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548dd59640 1 --2- 172.21.15.60:0/419327074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5488165350 0x7f5488145fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:44.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548d558640 1 --2- 172.21.15.60:0/419327074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5474003f70 0x7f54881596c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f547c0115e0 tx=0x7f547c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f5479ffb640 1 -- 172.21.15.60:0/419327074 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c00f8f0 con 0x7f5474003f70 2024-04-25T10:11:44.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f5479ffb640 1 -- 172.21.15.60:0/419327074 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c00fa90 con 0x7f5474003f70 2024-04-25T10:11:44.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548f7bc640 1 -- 172.21.15.60:0/419327074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5488154920 con 0x7f5474003f70 2024-04-25T10:11:44.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f5479ffb640 1 -- 172.21.15.60:0/419327074 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c00fe60 con 0x7f5474003f70 2024-04-25T10:11:44.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548f7bc640 1 -- 172.21.15.60:0/419327074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5488154ee0 con 0x7f5474003f70 2024-04-25T10:11:44.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f548f7bc640 1 -- 172.21.15.60:0/419327074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5488074770 con 0x7f5474003f70 2024-04-25T10:11:44.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f5479ffb640 1 -- 172.21.15.60:0/419327074 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f547c015460 con 0x7f5474003f70 2024-04-25T10:11:44.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f5479ffb640 1 --2- 172.21.15.60:0/419327074 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f546c066f60 0x7f546c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.148+0000 7f5479ffb640 1 -- 172.21.15.60:0/419327074 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f547c085ce0 con 0x7f5474003f70 2024-04-25T10:11:44.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.152+0000 7f548cd57640 1 --2- 172.21.15.60:0/419327074 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f546c066f60 0x7f546c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.152+0000 7f548cd57640 1 --2- 172.21.15.60:0/419327074 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f546c066f60 0x7f546c069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f54840ac000 tx=0x7f54840011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.152+0000 7f5479ffb640 1 -- 172.21.15.60:0/419327074 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f547c019030 con 0x7f5474003f70 2024-04-25T10:11:44.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:44.352+0000 7f548f7bc640 1 -- 172.21.15.60:0/419327074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f548815a580 con 0x7f5474003f70 2024-04-25T10:11:45.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.036+0000 7f5479ffb640 1 -- 172.21.15.60:0/419327074 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v52) v1 ==== 174+0+0 (secure 0 0 0) 0x7f547c056ec0 con 0x7f5474003f70 2024-04-25T10:11:45.049 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 12 and data pool 13 2024-04-25T10:11:45.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.064+0000 7f548f7bc640 1 -- 172.21.15.60:0/419327074 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f546c066f60 msgr2=0x7f546c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.064+0000 7f548f7bc640 1 --2- 172.21.15.60:0/419327074 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f546c066f60 0x7f546c069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f54840ac000 tx=0x7f54840011a0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.064+0000 7f548f7bc640 1 -- 172.21.15.60:0/419327074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5474003f70 msgr2=0x7f54881596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.064+0000 7f548f7bc640 1 --2- 172.21.15.60:0/419327074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5474003f70 0x7f54881596c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f547c0115e0 tx=0x7f547c011aa0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.064+0000 7f548f7bc640 1 -- 172.21.15.60:0/419327074 shutdown_connections 2024-04-25T10:11:45.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.064+0000 7f548f7bc640 1 --2- 172.21.15.60:0/419327074 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f546c066f60 0x7f546c069420 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.064+0000 7f548f7bc640 1 --2- 172.21.15.60:0/419327074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5488165350 0x7f5488145fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.064+0000 7f548f7bc640 1 --2- 172.21.15.60:0/419327074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5488164ef0 0x7f5488145a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.064+0000 7f548f7bc640 1 --2- 172.21.15.60:0/419327074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5474003f70 0x7f54881596c0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.064+0000 7f548f7bc640 1 -- 172.21.15.60:0/419327074 >> 172.21.15.60:0/419327074 conn(0x7f54880670b0 msgr2=0x7f548806da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:45.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.064+0000 7f548f7bc640 1 -- 172.21.15.60:0/419327074 shutdown_connections 2024-04-25T10:11:45.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.064+0000 7f548f7bc640 1 -- 172.21.15.60:0/419327074 wait complete. 2024-04-25T10:11:45.094 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:45.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.200+0000 7f5c2738c640 1 Processor -- start 2024-04-25T10:11:45.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.200+0000 7f5c2738c640 1 -- start start 2024-04-25T10:11:45.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c2738c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c200ce7a0 0x7f5c200ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c2738c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5c200b4660 0x7f5c200b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c2738c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c200cf0c0 con 0x7f5c2015bc90 2024-04-25T10:11:45.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c2738c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c200cc370 con 0x7f5c200b4660 2024-04-25T10:11:45.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c2738c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c20071ca0 con 0x7f5c200ce7a0 2024-04-25T10:11:45.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c25929640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5c200b4660 0x7f5c200b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c25929640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5c200b4660 0x7f5c200b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45550/0 (socket says 172.21.15.60:45550) 2024-04-25T10:11:45.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c25929640 1 -- 172.21.15.60:0/960869273 learned_addr learned my addr 172.21.15.60:0/960869273 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:45.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c25128640 1 --2- 172.21.15.60:0/960869273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c200ce7a0 0x7f5c200ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c25929640 1 -- 172.21.15.60:0/960869273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c200ce7a0 msgr2=0x7f5c200ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:45.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c25929640 1 --2- 172.21.15.60:0/960869273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c200ce7a0 0x7f5c200ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c25929640 1 -- 172.21.15.60:0/960869273 >> v1:172.21.15.60:6789/0 conn(0x7f5c2015bc90 legacy=0x7f5c200b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c25929640 1 -- 172.21.15.60:0/960869273 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c200cca40 con 0x7f5c200b4660 2024-04-25T10:11:45.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c17fff640 1 -- 172.21.15.60:0/960869273 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3144010604 0 0) 0x7f5c200cf0c0 con 0x7f5c2015bc90 2024-04-25T10:11:45.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c25128640 1 --2- 172.21.15.60:0/960869273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c200ce7a0 0x7f5c200ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:45.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c25929640 1 --2- 172.21.15.60:0/960869273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c200b4660 0x7f5c200b2ce0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009e80 tx=0x7f5c1801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ebc602307e029b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c17fff640 1 -- 172.21.15.60:0/960869273 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c180a56e0 con 0x7f5c200b4660 2024-04-25T10:11:45.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c17fff640 1 -- 172.21.15.60:0/960869273 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c180a5880 con 0x7f5c200b4660 2024-04-25T10:11:45.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.204+0000 7f5c17fff640 1 -- 172.21.15.60:0/960869273 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c180a5c50 con 0x7f5c200b4660 2024-04-25T10:11:45.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 -- 172.21.15.60:0/960869273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c200b4660 msgr2=0x7f5c200b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 --2- 172.21.15.60:0/960869273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c200b4660 0x7f5c200b2ce0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009e80 tx=0x7f5c1801ca10 comp rx=0 tx=0).stop 2024-04-25T10:11:45.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 -- 172.21.15.60:0/960869273 shutdown_connections 2024-04-25T10:11:45.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 --2- 172.21.15.60:0/960869273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c200b4660 0x7f5c200b2ce0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 --2- 172.21.15.60:0/960869273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c200ce7a0 0x7f5c200ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 -- 172.21.15.60:0/960869273 >> 172.21.15.60:0/960869273 conn(0x7f5c200b12d0 msgr2=0x7f5c200b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:45.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 -- 172.21.15.60:0/960869273 shutdown_connections 2024-04-25T10:11:45.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 -- 172.21.15.60:0/960869273 wait complete. 2024-04-25T10:11:45.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 Processor -- start 2024-04-25T10:11:45.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 -- start start 2024-04-25T10:11:45.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c200ce7a0 0x7f5c20153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c2015bc90 0x7f5c2014da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c201f4d00 0x7f5c20156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200b4280 con 0x7f5c2015bc90 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200b3db0 con 0x7f5c201f4d00 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200b4400 con 0x7f5c200ce7a0 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c24927640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c2015bc90 0x7f5c2014da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c24927640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c2015bc90 0x7f5c2014da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45558/0 (socket says 172.21.15.60:45558) 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c24927640 1 -- 172.21.15.60:0/2841625710 learned_addr learned my addr 172.21.15.60:0/2841625710 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c25128640 1 --2- 172.21.15.60:0/2841625710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c200ce7a0 0x7f5c20153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c25929640 1 --2- 172.21.15.60:0/2841625710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c201f4d00 0x7f5c20156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c24927640 1 -- 172.21.15.60:0/2841625710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c200ce7a0 msgr2=0x7f5c20153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c24927640 1 --2- 172.21.15.60:0/2841625710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c200ce7a0 0x7f5c20153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c24927640 1 -- 172.21.15.60:0/2841625710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c201f4d00 msgr2=0x7f5c20156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c24927640 1 --2- 172.21.15.60:0/2841625710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c201f4d00 0x7f5c20156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c24927640 1 -- 172.21.15.60:0/2841625710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c20156d70 con 0x7f5c2015bc90 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c24927640 1 --2- 172.21.15.60:0/2841625710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c2015bc90 0x7f5c2014da70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00c440 tx=0x7f5c1c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c15ffb640 1 -- 172.21.15.60:0/2841625710 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c017070 con 0x7f5c2015bc90 2024-04-25T10:11:45.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c15ffb640 1 -- 172.21.15.60:0/2841625710 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c003600 con 0x7f5c2015bc90 2024-04-25T10:11:45.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 -- 172.21.15.60:0/2841625710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c20157050 con 0x7f5c2015bc90 2024-04-25T10:11:45.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c15ffb640 1 -- 172.21.15.60:0/2841625710 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c0039f0 con 0x7f5c2015bc90 2024-04-25T10:11:45.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.208+0000 7f5c2738c640 1 -- 172.21.15.60:0/2841625710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c20158ea0 con 0x7f5c2015bc90 2024-04-25T10:11:45.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.212+0000 7f5c2738c640 1 -- 172.21.15.60:0/2841625710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c20074770 con 0x7f5c2015bc90 2024-04-25T10:11:45.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.212+0000 7f5c15ffb640 1 -- 172.21.15.60:0/2841625710 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c1c01a070 con 0x7f5c2015bc90 2024-04-25T10:11:45.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.212+0000 7f5c15ffb640 1 --2- 172.21.15.60:0/2841625710 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5c04066f90 0x7f5c04069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.212+0000 7f5c15ffb640 1 -- 172.21.15.60:0/2841625710 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c1c0842b0 con 0x7f5c2015bc90 2024-04-25T10:11:45.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.212+0000 7f5c25128640 1 --2- 172.21.15.60:0/2841625710 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5c04066f90 0x7f5c04069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.212+0000 7f5c25128640 1 --2- 172.21.15.60:0/2841625710 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5c04066f90 0x7f5c04069450 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5c2015eb20 tx=0x7f5c10008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.216+0000 7f5c15ffb640 1 -- 172.21.15.60:0/2841625710 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c1c055350 con 0x7f5c2015bc90 2024-04-25T10:11:45.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.380+0000 7f5c2738c640 1 -- 172.21.15.60:0/2841625710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c20159070 con 0x7f5c2015bc90 2024-04-25T10:11:45.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.384+0000 7f5c15ffb640 1 -- 172.21.15.60:0/2841625710 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16685 (secure 0 0 0) 0x7f5c1c054c50 con 0x7f5c2015bc90 2024-04-25T10:11:45.391 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:45.391 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":81,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:11:44.357442+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:42.621502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:43.352679+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:45.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.384+0000 7f5c2738c640 1 -- 172.21.15.60:0/2841625710 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5c04066f90 msgr2=0x7f5c04069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.384+0000 7f5c2738c640 1 --2- 172.21.15.60:0/2841625710 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5c04066f90 0x7f5c04069450 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5c2015eb20 tx=0x7f5c10008040 comp rx=0 tx=0).stop 2024-04-25T10:11:45.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.384+0000 7f5c2738c640 1 -- 172.21.15.60:0/2841625710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c2015bc90 msgr2=0x7f5c2014da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.384+0000 7f5c2738c640 1 --2- 172.21.15.60:0/2841625710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c2015bc90 0x7f5c2014da70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00c440 tx=0x7f5c1c00c900 comp rx=0 tx=0).stop 2024-04-25T10:11:45.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.384+0000 7f5c2738c640 1 -- 172.21.15.60:0/2841625710 shutdown_connections 2024-04-25T10:11:45.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.384+0000 7f5c2738c640 1 --2- 172.21.15.60:0/2841625710 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5c04066f90 0x7f5c04069450 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.384+0000 7f5c2738c640 1 --2- 172.21.15.60:0/2841625710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c201f4d00 0x7f5c20156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.384+0000 7f5c2738c640 1 --2- 172.21.15.60:0/2841625710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c2015bc90 0x7f5c2014da70 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.384+0000 7f5c2738c640 1 --2- 172.21.15.60:0/2841625710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c200ce7a0 0x7f5c20153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.384+0000 7f5c2738c640 1 -- 172.21.15.60:0/2841625710 >> 172.21.15.60:0/2841625710 conn(0x7f5c200b12d0 msgr2=0x7f5c200b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:45.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.384+0000 7f5c2738c640 1 -- 172.21.15.60:0/2841625710 shutdown_connections 2024-04-25T10:11:45.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.384+0000 7f5c2738c640 1 -- 172.21.15.60:0/2841625710 wait complete. 2024-04-25T10:11:45.411 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.528+0000 7f38e1fc0640 1 Processor -- start 2024-04-25T10:11:45.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.528+0000 7f38e1fc0640 1 -- start start 2024-04-25T10:11:45.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.528+0000 7f38e1fc0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38dc06ec30 0x7f38dc0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.528+0000 7f38e1fc0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38dc0b37e0 0x7f38dc0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.528+0000 7f38e1fc0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38dc06f130 con 0x7f38dc15a7a0 2024-04-25T10:11:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.528+0000 7f38e1fc0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38dc0cdaa0 con 0x7f38dc06ec30 2024-04-25T10:11:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.528+0000 7f38e1fc0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc06d2f0 con 0x7f38dc0b37e0 2024-04-25T10:11:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.528+0000 7f38db7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f38dc15a7a0 0x7f38dc071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50810/0 (socket says 172.21.15.60:50810) 2024-04-25T10:11:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.528+0000 7f38db7fe640 1 -- 172.21.15.60:0/2172165833 learned_addr learned my addr 172.21.15.60:0/2172165833 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.528+0000 7f38daffd640 1 --2- 172.21.15.60:0/2172165833 >> v2:172.21.15.60:3300/0 conn(0x7f38dc06ec30 0x7f38dc0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.528+0000 7f38dbfff640 1 --2- 172.21.15.60:0/2172165833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38dc0b37e0 0x7f38dc0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.528+0000 7f38daffd640 1 -- 172.21.15.60:0/2172165833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38dc0b37e0 msgr2=0x7f38dc0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.528+0000 7f38daffd640 1 --2- 172.21.15.60:0/2172165833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38dc0b37e0 0x7f38dc0b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.528+0000 7f38daffd640 1 -- 172.21.15.60:0/2172165833 >> v1:172.21.15.60:6789/0 conn(0x7f38dc15a7a0 legacy=0x7f38dc071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.528+0000 7f38da7fc640 1 -- 172.21.15.60:0/2172165833 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659083837 0 0) 0x7f38dc06f130 con 0x7f38dc15a7a0 2024-04-25T10:11:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.528+0000 7f38daffd640 1 -- 172.21.15.60:0/2172165833 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc06f410 con 0x7f38dc06ec30 2024-04-25T10:11:45.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.532+0000 7f38daffd640 1 --2- 172.21.15.60:0/2172165833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38dc06ec30 0x7f38dc0cd560 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f38d00097f0 tx=0x7f38d001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc3ba0098c5caa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.532+0000 7f38da7fc640 1 -- 172.21.15.60:0/2172165833 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d00b1070 con 0x7f38dc06ec30 2024-04-25T10:11:45.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.532+0000 7f38da7fc640 1 -- 172.21.15.60:0/2172165833 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d00ac410 con 0x7f38dc06ec30 2024-04-25T10:11:45.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.532+0000 7f38da7fc640 1 -- 172.21.15.60:0/2172165833 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d00ac7e0 con 0x7f38dc06ec30 2024-04-25T10:11:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.532+0000 7f38e1fc0640 1 -- 172.21.15.60:0/2172165833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38dc06ec30 msgr2=0x7f38dc0cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.532+0000 7f38e1fc0640 1 --2- 172.21.15.60:0/2172165833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38dc06ec30 0x7f38dc0cd560 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f38d00097f0 tx=0x7f38d001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.532+0000 7f38e1fc0640 1 -- 172.21.15.60:0/2172165833 shutdown_connections 2024-04-25T10:11:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.532+0000 7f38e1fc0640 1 --2- 172.21.15.60:0/2172165833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38dc0b37e0 0x7f38dc0b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.532+0000 7f38e1fc0640 1 --2- 172.21.15.60:0/2172165833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38dc06ec30 0x7f38dc0cd560 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.532+0000 7f38e1fc0640 1 -- 172.21.15.60:0/2172165833 >> 172.21.15.60:0/2172165833 conn(0x7f38dc0b12d0 msgr2=0x7f38dc0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.532+0000 7f38e1fc0640 1 -- 172.21.15.60:0/2172165833 shutdown_connections 2024-04-25T10:11:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.532+0000 7f38e1fc0640 1 -- 172.21.15.60:0/2172165833 wait complete. 2024-04-25T10:11:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.532+0000 7f38e1fc0640 1 Processor -- start 2024-04-25T10:11:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.532+0000 7f38e1fc0640 1 -- start start 2024-04-25T10:11:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38e1fc0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38dc06ec30 0x7f38dc14de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38e1fc0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38dc0b37e0 0x7f38dc160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38e1fc0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38dc15a7a0 0x7f38dc159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38e1fc0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc06e8e0 con 0x7f38dc06ec30 2024-04-25T10:11:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38e1fc0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc06e760 con 0x7f38dc0b37e0 2024-04-25T10:11:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38e1fc0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc06d8b0 con 0x7f38dc15a7a0 2024-04-25T10:11:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38db7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38dc06ec30 0x7f38dc14de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38db7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38dc06ec30 0x7f38dc14de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45578/0 (socket says 172.21.15.60:45578) 2024-04-25T10:11:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38db7fe640 1 -- 172.21.15.60:0/1178870861 learned_addr learned my addr 172.21.15.60:0/1178870861 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38dbfff640 1 --2- 172.21.15.60:0/1178870861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38dc15a7a0 0x7f38dc159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38db7fe640 1 -- 172.21.15.60:0/1178870861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38dc15a7a0 msgr2=0x7f38dc159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38db7fe640 1 --2- 172.21.15.60:0/1178870861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38dc15a7a0 0x7f38dc159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38db7fe640 1 -- 172.21.15.60:0/1178870861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38dc0b37e0 msgr2=0x7f38dc160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38db7fe640 1 --2- 172.21.15.60:0/1178870861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38dc0b37e0 0x7f38dc160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38db7fe640 1 -- 172.21.15.60:0/1178870861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc159ba0 con 0x7f38dc06ec30 2024-04-25T10:11:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38dbfff640 1 --2- 172.21.15.60:0/1178870861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38dc15a7a0 0x7f38dc159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38db7fe640 1 --2- 172.21.15.60:0/1178870861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38dc06ec30 0x7f38dc14de40 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00c480 tx=0x7f38cc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38c3fff640 1 -- 172.21.15.60:0/1178870861 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc018070 con 0x7f38dc06ec30 2024-04-25T10:11:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38c3fff640 1 -- 172.21.15.60:0/1178870861 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc007500 con 0x7f38dc06ec30 2024-04-25T10:11:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38e1fc0640 1 -- 172.21.15.60:0/1178870861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38dc156ac0 con 0x7f38dc06ec30 2024-04-25T10:11:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38c3fff640 1 -- 172.21.15.60:0/1178870861 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc004a30 con 0x7f38dc06ec30 2024-04-25T10:11:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38e1fc0640 1 -- 172.21.15.60:0/1178870861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38dc157360 con 0x7f38dc06ec30 2024-04-25T10:11:45.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38e1fc0640 1 -- 172.21.15.60:0/1178870861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38dc156eb0 con 0x7f38dc06ec30 2024-04-25T10:11:45.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38c3fff640 1 -- 172.21.15.60:0/1178870861 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38cc01b070 con 0x7f38dc06ec30 2024-04-25T10:11:45.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38c3fff640 1 --2- 172.21.15.60:0/1178870861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f38c8066f60 0x7f38c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.536+0000 7f38c3fff640 1 -- 172.21.15.60:0/1178870861 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38cc084390 con 0x7f38dc06ec30 2024-04-25T10:11:45.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.540+0000 7f38daffd640 1 --2- 172.21.15.60:0/1178870861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f38c8066f60 0x7f38c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.540+0000 7f38daffd640 1 --2- 172.21.15.60:0/1178870861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f38c8066f60 0x7f38c8069420 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f38d00ae000 tx=0x7f38d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.544+0000 7f38c3fff640 1 -- 172.21.15.60:0/1178870861 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38cc055410 con 0x7f38dc06ec30 2024-04-25T10:11:45.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.704+0000 7f38e1fc0640 1 -- 172.21.15.60:0/1178870861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38dc074770 con 0x7f38dc06ec30 2024-04-25T10:11:45.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.704+0000 7f38c3fff640 1 -- 172.21.15.60:0/1178870861 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16685 (secure 0 0 0) 0x7f38cc054d10 con 0x7f38dc06ec30 2024-04-25T10:11:45.714 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:45.714 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":81,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:11:44.357442+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:42.621502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:43.352679+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:45.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.708+0000 7f38e1fc0640 1 -- 172.21.15.60:0/1178870861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f38c8066f60 msgr2=0x7f38c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.708+0000 7f38e1fc0640 1 --2- 172.21.15.60:0/1178870861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f38c8066f60 0x7f38c8069420 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f38d00ae000 tx=0x7f38d0008040 comp rx=0 tx=0).stop 2024-04-25T10:11:45.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.708+0000 7f38e1fc0640 1 -- 172.21.15.60:0/1178870861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38dc06ec30 msgr2=0x7f38dc14de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.708+0000 7f38e1fc0640 1 --2- 172.21.15.60:0/1178870861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38dc06ec30 0x7f38dc14de40 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00c480 tx=0x7f38cc00c940 comp rx=0 tx=0).stop 2024-04-25T10:11:45.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.708+0000 7f38e1fc0640 1 -- 172.21.15.60:0/1178870861 shutdown_connections 2024-04-25T10:11:45.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.708+0000 7f38e1fc0640 1 --2- 172.21.15.60:0/1178870861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f38c8066f60 0x7f38c8069420 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.708+0000 7f38e1fc0640 1 --2- 172.21.15.60:0/1178870861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38dc15a7a0 0x7f38dc159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.708+0000 7f38e1fc0640 1 --2- 172.21.15.60:0/1178870861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38dc0b37e0 0x7f38dc160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.708+0000 7f38e1fc0640 1 --2- 172.21.15.60:0/1178870861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38dc06ec30 0x7f38dc14de40 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.708+0000 7f38e1fc0640 1 -- 172.21.15.60:0/1178870861 >> 172.21.15.60:0/1178870861 conn(0x7f38dc0b12d0 msgr2=0x7f38dc071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.708+0000 7f38e1fc0640 1 -- 172.21.15.60:0/1178870861 shutdown_connections 2024-04-25T10:11:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.708+0000 7f38e1fc0640 1 -- 172.21.15.60:0/1178870861 wait complete. 2024-04-25T10:11:45.735 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:11:45.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9ff2e2640 1 Processor -- start 2024-04-25T10:11:45.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9ff2e2640 1 -- start start 2024-04-25T10:11:45.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9ff2e2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe9f815f0d0 0x7fe9f80cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9ff2e2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f80cca60 con 0x7fe9f809a290 2024-04-25T10:11:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9ff2e2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f80ccd40 con 0x7fe9f80cdb00 2024-04-25T10:11:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9ff2e2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f80cc830 con 0x7fe9f815f0d0 2024-04-25T10:11:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9fd87f640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe9f809a290 0x7fe9f80b1170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50818/0 (socket says 172.21.15.60:50818) 2024-04-25T10:11:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9fd87f640 1 -- 172.21.15.60:0/439714040 learned_addr learned my addr 172.21.15.60:0/439714040 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9fd07e640 1 --2- 172.21.15.60:0/439714040 >> v2:172.21.15.100:3300/0 conn(0x7fe9f815f0d0 0x7fe9f80cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9effff640 1 -- 172.21.15.60:0/439714040 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099587658 0 0) 0x7fe9f80cca60 con 0x7fe9f809a290 2024-04-25T10:11:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9effff640 1 -- 172.21.15.60:0/439714040 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9e4003530 con 0x7fe9f809a290 2024-04-25T10:11:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9effff640 1 -- 172.21.15.60:0/439714040 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2573352064 0 0) 0x7fe9e4003530 con 0x7fe9f809a290 2024-04-25T10:11:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9effff640 1 -- 172.21.15.60:0/439714040 >> v2:172.21.15.100:3300/0 conn(0x7fe9f815f0d0 msgr2=0x7fe9f80cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9effff640 1 --2- 172.21.15.60:0/439714040 >> v2:172.21.15.100:3300/0 conn(0x7fe9f815f0d0 0x7fe9f80cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9effff640 1 -- 172.21.15.60:0/439714040 >> v1:172.21.15.100:6789/0 conn(0x7fe9f80cdb00 legacy=0x7fe9f8099b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9effff640 1 -- 172.21.15.60:0/439714040 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f80cd020 con 0x7fe9f809a290 2024-04-25T10:11:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9effff640 1 -- 172.21.15.60:0/439714040 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fe9f0001a00 con 0x7fe9f809a290 2024-04-25T10:11:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9effff640 1 -- 172.21.15.60:0/439714040 >> v1:172.21.15.60:6789/0 conn(0x7fe9f809a290 legacy=0x7fe9f80b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9effff640 1 --2- 172.21.15.60:0/439714040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e40039c0 0x7fe9e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9effff640 1 --2- 172.21.15.60:0/439714040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e4004810 0x7fe9e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9effff640 1 --2- 172.21.15.60:0/439714040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9e40051d0 0x7fe9e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9effff640 1 -- 172.21.15.60:0/439714040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e400c140 con 0x7fe9e40039c0 2024-04-25T10:11:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9effff640 1 -- 172.21.15.60:0/439714040 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e400c2f0 con 0x7fe9e40051d0 2024-04-25T10:11:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9effff640 1 -- 172.21.15.60:0/439714040 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e400c4f0 con 0x7fe9e4004810 2024-04-25T10:11:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9fd07e640 1 --2- 172.21.15.60:0/439714040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e40039c0 0x7fe9e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9fd87f640 1 --2- 172.21.15.60:0/439714040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9e40051d0 0x7fe9e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.856+0000 7fe9fc87d640 1 --2- 172.21.15.60:0/439714040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e4004810 0x7fe9e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.860+0000 7fe9fd07e640 1 -- 172.21.15.60:0/439714040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e4004810 msgr2=0x7fe9e4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.860+0000 7fe9fd07e640 1 --2- 172.21.15.60:0/439714040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e4004810 0x7fe9e4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.860+0000 7fe9fd07e640 1 -- 172.21.15.60:0/439714040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9e40051d0 msgr2=0x7fe9e400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.860+0000 7fe9fd07e640 1 --2- 172.21.15.60:0/439714040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9e40051d0 0x7fe9e400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.860+0000 7fe9fd87f640 1 -- 172.21.15.60:0/439714040 reap_dead start 2024-04-25T10:11:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.860+0000 7fe9fd07e640 1 -- 172.21.15.60:0/439714040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e4003530 con 0x7fe9e40039c0 2024-04-25T10:11:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.860+0000 7fe9fc87d640 1 --2- 172.21.15.60:0/439714040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e4004810 0x7fe9e4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:45.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.860+0000 7fe9fd07e640 1 --2- 172.21.15.60:0/439714040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e40039c0 0x7fe9e4003de0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009ae0 tx=0x7fe9f401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229b928d9cf76c33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9effff640 1 -- 172.21.15.60:0/439714040 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40096b0 con 0x7fe9e40039c0 2024-04-25T10:11:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9effff640 1 -- 172.21.15.60:0/439714040 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f40a6450 con 0x7fe9e40039c0 2024-04-25T10:11:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9effff640 1 -- 172.21.15.60:0/439714040 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40a6840 con 0x7fe9e40039c0 2024-04-25T10:11:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9ff2e2640 1 -- 172.21.15.60:0/439714040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e40039c0 msgr2=0x7fe9e4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9ff2e2640 1 --2- 172.21.15.60:0/439714040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e40039c0 0x7fe9e4003de0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009ae0 tx=0x7fe9f401ca70 comp rx=0 tx=0).stop 2024-04-25T10:11:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9ff2e2640 1 -- 172.21.15.60:0/439714040 shutdown_connections 2024-04-25T10:11:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9ff2e2640 1 --2- 172.21.15.60:0/439714040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e40039c0 0x7fe9e4003de0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9ff2e2640 1 -- 172.21.15.60:0/439714040 >> 172.21.15.60:0/439714040 conn(0x7fe9f80b34d0 msgr2=0x7fe9f80b38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9ff2e2640 1 -- 172.21.15.60:0/439714040 shutdown_connections 2024-04-25T10:11:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9ff2e2640 1 -- 172.21.15.60:0/439714040 wait complete. 2024-04-25T10:11:45.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9ff2e2640 1 Processor -- start 2024-04-25T10:11:45.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9ff2e2640 1 -- start start 2024-04-25T10:11:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9ff2e2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e40039c0 0x7fe9f8152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9ff2e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f80c59a0 0x7fe9f80c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9ff2e2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80b6170 0x7fe9f80b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9ff2e2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f8099f50 con 0x7fe9e40039c0 2024-04-25T10:11:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9ff2e2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f8099dd0 con 0x7fe9f80c59a0 2024-04-25T10:11:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9ff2e2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f809a0d0 con 0x7fe9f80b6170 2024-04-25T10:11:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9fd07e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e40039c0 0x7fe9f8152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9fd07e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e40039c0 0x7fe9f8152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45596/0 (socket says 172.21.15.60:45596) 2024-04-25T10:11:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9fd07e640 1 -- 172.21.15.60:0/669642324 learned_addr learned my addr 172.21.15.60:0/669642324 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9fd87f640 1 --2- 172.21.15.60:0/669642324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80b6170 0x7fe9f80b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9fc87d640 1 --2- 172.21.15.60:0/669642324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f80c59a0 0x7fe9f80c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9fd07e640 1 -- 172.21.15.60:0/669642324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80b6170 msgr2=0x7fe9f80b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9fd07e640 1 --2- 172.21.15.60:0/669642324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80b6170 0x7fe9f80b65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9fd07e640 1 -- 172.21.15.60:0/669642324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f80c59a0 msgr2=0x7fe9f80c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9fd07e640 1 --2- 172.21.15.60:0/669642324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f80c59a0 0x7fe9f80c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9fd07e640 1 -- 172.21.15.60:0/669642324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f80ca0d0 con 0x7fe9e40039c0 2024-04-25T10:11:45.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9fd87f640 1 --2- 172.21.15.60:0/669642324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80b6170 0x7fe9f80b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.864+0000 7fe9fd07e640 1 --2- 172.21.15.60:0/669642324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e40039c0 0x7fe9f8152e10 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fe9f401c8a0 tx=0x7fe9f4019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.868+0000 7fe9edffb640 1 -- 172.21.15.60:0/669642324 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40bd070 con 0x7fe9e40039c0 2024-04-25T10:11:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.868+0000 7fe9edffb640 1 -- 172.21.15.60:0/669642324 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f40a6c80 con 0x7fe9e40039c0 2024-04-25T10:11:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.868+0000 7fe9ff2e2640 1 -- 172.21.15.60:0/669642324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f80c6510 con 0x7fe9e40039c0 2024-04-25T10:11:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.868+0000 7fe9edffb640 1 -- 172.21.15.60:0/669642324 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40a7650 con 0x7fe9e40039c0 2024-04-25T10:11:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.868+0000 7fe9ff2e2640 1 -- 172.21.15.60:0/669642324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f80c8d90 con 0x7fe9e40039c0 2024-04-25T10:11:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.868+0000 7fe9ff2e2640 1 -- 172.21.15.60:0/669642324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f8074770 con 0x7fe9e40039c0 2024-04-25T10:11:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.868+0000 7fe9edffb640 1 -- 172.21.15.60:0/669642324 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9f40c0070 con 0x7fe9e40039c0 2024-04-25T10:11:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.868+0000 7fe9edffb640 1 --2- 172.21.15.60:0/669642324 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f90 0x7fe9dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.868+0000 7fe9edffb640 1 -- 172.21.15.60:0/669642324 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9f411ec40 con 0x7fe9e40039c0 2024-04-25T10:11:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.868+0000 7fe9fc87d640 1 --2- 172.21.15.60:0/669642324 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f90 0x7fe9dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.868+0000 7fe9fc87d640 1 --2- 172.21.15.60:0/669642324 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f90 0x7fe9dc069450 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8000c10 tx=0x7fe9e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:45.872+0000 7fe9edffb640 1 -- 172.21.15.60:0/669642324 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9f40b5070 con 0x7fe9e40039c0 2024-04-25T10:11:45.977 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-04-25T10:11:45.968+0000 7f0ae804b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:46.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:46.068+0000 7fe9ff2e2640 1 -- 172.21.15.60:0/669642324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe9f80b7960 con 0x7fe9e40039c0 2024-04-25T10:11:47.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.036+0000 7fe9edffb640 1 -- 172.21.15.60:0/669642324 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe9f40ac020 con 0x7fe9e40039c0 2024-04-25T10:11:47.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.040+0000 7fe9ff2e2640 1 -- 172.21.15.60:0/669642324 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f90 msgr2=0x7fe9dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.040+0000 7fe9ff2e2640 1 --2- 172.21.15.60:0/669642324 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f90 0x7fe9dc069450 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8000c10 tx=0x7fe9e8009040 comp rx=0 tx=0).stop 2024-04-25T10:11:47.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.040+0000 7fe9ff2e2640 1 -- 172.21.15.60:0/669642324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e40039c0 msgr2=0x7fe9f8152e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.040+0000 7fe9ff2e2640 1 --2- 172.21.15.60:0/669642324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e40039c0 0x7fe9f8152e10 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fe9f401c8a0 tx=0x7fe9f4019810 comp rx=0 tx=0).stop 2024-04-25T10:11:47.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.040+0000 7fe9ff2e2640 1 -- 172.21.15.60:0/669642324 shutdown_connections 2024-04-25T10:11:47.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.040+0000 7fe9ff2e2640 1 --2- 172.21.15.60:0/669642324 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe9dc066f90 0x7fe9dc069450 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.040+0000 7fe9ff2e2640 1 --2- 172.21.15.60:0/669642324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9f80b6170 0x7fe9f80b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.040+0000 7fe9ff2e2640 1 --2- 172.21.15.60:0/669642324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9f80c59a0 0x7fe9f80c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.040+0000 7fe9ff2e2640 1 --2- 172.21.15.60:0/669642324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e40039c0 0x7fe9f8152e10 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.040+0000 7fe9ff2e2640 1 -- 172.21.15.60:0/669642324 >> 172.21.15.60:0/669642324 conn(0x7fe9f80b34d0 msgr2=0x7fe9f80aed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:47.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.040+0000 7fe9ff2e2640 1 -- 172.21.15.60:0/669642324 shutdown_connections 2024-04-25T10:11:47.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.040+0000 7fe9ff2e2640 1 -- 172.21.15.60:0/669642324 wait complete. 2024-04-25T10:11:47.072 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:11:47.072 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:11:47.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:47.137 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:47.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d0d0fc640 1 Processor -- start 2024-04-25T10:11:47.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d0d0fc640 1 -- start start 2024-04-25T10:11:47.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d0d0fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1d0815de90 0x7f1d080b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d0d0fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1d080ce9a0 0x7f1d080d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d0d0fc640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d0806d200 con 0x7f1d080d0320 2024-04-25T10:11:47.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d0d0fc640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d080cf2a0 con 0x7f1d0815de90 2024-04-25T10:11:47.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d0d0fc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d080cefc0 con 0x7f1d080ce9a0 2024-04-25T10:11:47.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d06d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1d0815de90 0x7f1d080b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d0659c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1d080d0320 0x7f1d080ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50824/0 (socket says 172.21.15.60:50824) 2024-04-25T10:11:47.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d06d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1d0815de90 0x7f1d080b6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45606/0 (socket says 172.21.15.60:45606) 2024-04-25T10:11:47.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d0659c640 1 -- 172.21.15.60:0/2471615067 learned_addr learned my addr 172.21.15.60:0/2471615067 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d06d9d640 1 -- 172.21.15.60:0/2471615067 >> v2:172.21.15.100:3300/0 conn(0x7f1d080ce9a0 msgr2=0x7f1d080d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d0759e640 1 --2- 172.21.15.60:0/2471615067 >> v2:172.21.15.100:3300/0 conn(0x7f1d080ce9a0 0x7f1d080d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d05d9b640 1 -- 172.21.15.60:0/2471615067 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720233019 0 0) 0x7f1d0806d200 con 0x7f1d080d0320 2024-04-25T10:11:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d06d9d640 1 --2- 172.21.15.60:0/2471615067 >> v2:172.21.15.100:3300/0 conn(0x7f1d080ce9a0 0x7f1d080d07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d06d9d640 1 -- 172.21.15.60:0/2471615067 >> v1:172.21.15.60:6789/0 conn(0x7f1d080d0320 legacy=0x7f1d080ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d06d9d640 1 -- 172.21.15.60:0/2471615067 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d080d1d50 con 0x7f1d0815de90 2024-04-25T10:11:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.196+0000 7f1d0759e640 1 --2- 172.21.15.60:0/2471615067 >> v2:172.21.15.100:3300/0 conn(0x7f1d080ce9a0 0x7f1d080d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:47.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.200+0000 7f1d06d9d640 1 --2- 172.21.15.60:0/2471615067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d0815de90 0x7f1d080b6bf0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f1d080b99e0 tx=0x7f1d0001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa43d0460b31d914 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:47.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.200+0000 7f1d05d9b640 1 -- 172.21.15.60:0/2471615067 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d000b2070 con 0x7f1d0815de90 2024-04-25T10:11:47.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.200+0000 7f1d05d9b640 1 -- 172.21.15.60:0/2471615067 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d000a57f0 con 0x7f1d0815de90 2024-04-25T10:11:47.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.200+0000 7f1d05d9b640 1 -- 172.21.15.60:0/2471615067 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d000a5bc0 con 0x7f1d0815de90 2024-04-25T10:11:47.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.200+0000 7f1d0d0fc640 1 -- 172.21.15.60:0/2471615067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d0815de90 msgr2=0x7f1d080b6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.200+0000 7f1d0d0fc640 1 --2- 172.21.15.60:0/2471615067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d0815de90 0x7f1d080b6bf0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f1d080b99e0 tx=0x7f1d0001c930 comp rx=0 tx=0).stop 2024-04-25T10:11:47.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.200+0000 7f1d0d0fc640 1 -- 172.21.15.60:0/2471615067 shutdown_connections 2024-04-25T10:11:47.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.200+0000 7f1d0d0fc640 1 --2- 172.21.15.60:0/2471615067 >> v2:172.21.15.100:3300/0 conn(0x7f1d080ce9a0 0x7f1d080d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.200+0000 7f1d0d0fc640 1 --2- 172.21.15.60:0/2471615067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d0815de90 0x7f1d080b6bf0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.200+0000 7f1d0d0fc640 1 -- 172.21.15.60:0/2471615067 >> 172.21.15.60:0/2471615067 conn(0x7f1d0806c090 msgr2=0x7f1d0806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:47.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.200+0000 7f1d0d0fc640 1 -- 172.21.15.60:0/2471615067 shutdown_connections 2024-04-25T10:11:47.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.200+0000 7f1d0d0fc640 1 -- 172.21.15.60:0/2471615067 wait complete. 2024-04-25T10:11:47.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.200+0000 7f1d0d0fc640 1 Processor -- start 2024-04-25T10:11:47.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.200+0000 7f1d0d0fc640 1 -- start start 2024-04-25T10:11:47.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0d0fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d080ce9a0 0x7f1d080c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0d0fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080d0320 0x7f1d080cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0d0fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d0815de90 0x7f1d080b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0d0fc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080d3410 con 0x7f1d080d0320 2024-04-25T10:11:47.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0d0fc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080d3290 con 0x7f1d080ce9a0 2024-04-25T10:11:47.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0d0fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080d3590 con 0x7f1d0815de90 2024-04-25T10:11:47.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080d0320 0x7f1d080cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080d0320 0x7f1d080cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45618/0 (socket says 172.21.15.60:45618) 2024-04-25T10:11:47.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0659c640 1 -- 172.21.15.60:0/1093072849 learned_addr learned my addr 172.21.15.60:0/1093072849 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:47.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d06d9d640 1 --2- 172.21.15.60:0/1093072849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d080ce9a0 0x7f1d080c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0759e640 1 --2- 172.21.15.60:0/1093072849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d0815de90 0x7f1d080b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0659c640 1 -- 172.21.15.60:0/1093072849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d0815de90 msgr2=0x7f1d080b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0659c640 1 --2- 172.21.15.60:0/1093072849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d0815de90 0x7f1d080b7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0659c640 1 -- 172.21.15.60:0/1093072849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d080ce9a0 msgr2=0x7f1d080c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0659c640 1 --2- 172.21.15.60:0/1093072849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d080ce9a0 0x7f1d080c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0659c640 1 -- 172.21.15.60:0/1093072849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d080c8a20 con 0x7f1d080d0320 2024-04-25T10:11:47.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0659c640 1 --2- 172.21.15.60:0/1093072849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080d0320 0x7f1d080cb7d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0011f0 tx=0x7f1cfc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:47.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1cef7fe640 1 -- 172.21.15.60:0/1093072849 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc018070 con 0x7f1d080d0320 2024-04-25T10:11:47.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1cef7fe640 1 -- 172.21.15.60:0/1093072849 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc0074e0 con 0x7f1d080d0320 2024-04-25T10:11:47.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1cef7fe640 1 -- 172.21.15.60:0/1093072849 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc004a10 con 0x7f1d080d0320 2024-04-25T10:11:47.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0d0fc640 1 -- 172.21.15.60:0/1093072849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d080c8d40 con 0x7f1d080d0320 2024-04-25T10:11:47.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0d0fc640 1 -- 172.21.15.60:0/1093072849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d080c9300 con 0x7f1d080d0320 2024-04-25T10:11:47.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d0d0fc640 1 -- 172.21.15.60:0/1093072849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d08074770 con 0x7f1d080d0320 2024-04-25T10:11:47.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1cef7fe640 1 -- 172.21.15.60:0/1093072849 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1cfc01b070 con 0x7f1d080d0320 2024-04-25T10:11:47.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1cef7fe640 1 --2- 172.21.15.60:0/1093072849 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cf4066f60 0x7f1cf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1cef7fe640 1 -- 172.21.15.60:0/1093072849 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cfc0842e0 con 0x7f1d080d0320 2024-04-25T10:11:47.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.204+0000 7f1d06d9d640 1 --2- 172.21.15.60:0/1093072849 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cf4066f60 0x7f1cf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.208+0000 7f1d06d9d640 1 --2- 172.21.15.60:0/1093072849 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cf4066f60 0x7f1cf4069420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1d000af000 tx=0x7f1d000a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:47.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.208+0000 7f1cef7fe640 1 -- 172.21.15.60:0/1093072849 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1cfc055380 con 0x7f1d080d0320 2024-04-25T10:11:47.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:47.404+0000 7f1d0d0fc640 1 -- 172.21.15.60:0/1093072849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1d080d1780 con 0x7f1d080d0320 2024-04-25T10:11:48.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.044+0000 7f1cef7fe640 1 -- 172.21.15.60:0/1093072849 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1cfc05b080 con 0x7f1d080d0320 2024-04-25T10:11:48.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.048+0000 7f1d0d0fc640 1 -- 172.21.15.60:0/1093072849 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cf4066f60 msgr2=0x7f1cf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.048+0000 7f1d0d0fc640 1 --2- 172.21.15.60:0/1093072849 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cf4066f60 0x7f1cf4069420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1d000af000 tx=0x7f1d000a7070 comp rx=0 tx=0).stop 2024-04-25T10:11:48.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.048+0000 7f1d0d0fc640 1 -- 172.21.15.60:0/1093072849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080d0320 msgr2=0x7f1d080cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.048+0000 7f1d0d0fc640 1 --2- 172.21.15.60:0/1093072849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080d0320 0x7f1d080cb7d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0011f0 tx=0x7f1cfc00c9b0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.048+0000 7f1d0d0fc640 1 -- 172.21.15.60:0/1093072849 shutdown_connections 2024-04-25T10:11:48.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.048+0000 7f1d0d0fc640 1 --2- 172.21.15.60:0/1093072849 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cf4066f60 0x7f1cf4069420 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.048+0000 7f1d0d0fc640 1 --2- 172.21.15.60:0/1093072849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d0815de90 0x7f1d080b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.048+0000 7f1d0d0fc640 1 --2- 172.21.15.60:0/1093072849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080d0320 0x7f1d080cb7d0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.048+0000 7f1d0d0fc640 1 --2- 172.21.15.60:0/1093072849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d080ce9a0 0x7f1d080c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.048+0000 7f1d0d0fc640 1 -- 172.21.15.60:0/1093072849 >> 172.21.15.60:0/1093072849 conn(0x7f1d0806c090 msgr2=0x7f1d0815f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:48.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.048+0000 7f1d0d0fc640 1 -- 172.21.15.60:0/1093072849 shutdown_connections 2024-04-25T10:11:48.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.048+0000 7f1d0d0fc640 1 -- 172.21.15.60:0/1093072849 wait complete. 2024-04-25T10:11:48.079 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:11:48.079 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:48.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.244+0000 7f79fba0b640 1 Processor -- start 2024-04-25T10:11:48.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.244+0000 7f79fba0b640 1 -- start start 2024-04-25T10:11:48.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79fba0b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79f40d0a50 0x7f79f40b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79fba0b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f79f40ce2b0 0x7f79f40ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79fba0b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f406d200 con 0x7f79f415deb0 2024-04-25T10:11:48.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79fba0b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f40cf2c0 con 0x7f79f40d0a50 2024-04-25T10:11:48.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79fba0b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f40cefe0 con 0x7f79f40ce2b0 2024-04-25T10:11:48.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79f8fa6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79f40d0a50 0x7f79f40b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79f8fa6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79f40d0a50 0x7f79f40b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45620/0 (socket says 172.21.15.60:45620) 2024-04-25T10:11:48.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79f8fa6640 1 -- 172.21.15.60:0/1181998708 learned_addr learned my addr 172.21.15.60:0/1181998708 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:48.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79f97a7640 1 --1- 172.21.15.60:0/1181998708 >> v1:172.21.15.60:6789/0 conn(0x7f79f415deb0 0x7f79f40d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50828/0 (socket says 172.21.15.60:50828) 2024-04-25T10:11:48.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79f9fa8640 1 --2- 172.21.15.60:0/1181998708 >> v2:172.21.15.100:3300/0 conn(0x7f79f40ce2b0 0x7f79f40ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79f8fa6640 1 -- 172.21.15.60:0/1181998708 >> v2:172.21.15.100:3300/0 conn(0x7f79f40ce2b0 msgr2=0x7f79f40ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:48.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79f8fa6640 1 --2- 172.21.15.60:0/1181998708 >> v2:172.21.15.100:3300/0 conn(0x7f79f40ce2b0 0x7f79f40ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79f8fa6640 1 -- 172.21.15.60:0/1181998708 >> v1:172.21.15.60:6789/0 conn(0x7f79f415deb0 legacy=0x7f79f40d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79f8fa6640 1 -- 172.21.15.60:0/1181998708 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f40d1d70 con 0x7f79f40d0a50 2024-04-25T10:11:48.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79ebfff640 1 -- 172.21.15.60:0/1181998708 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741667569 0 0) 0x7f79f406d200 con 0x7f79f415deb0 2024-04-25T10:11:48.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79f8fa6640 1 --2- 172.21.15.60:0/1181998708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f40d0a50 0x7f79f40b6c10 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f79f40d1590 tx=0x7f79e401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa7165174b9ee1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79ebfff640 1 -- 172.21.15.60:0/1181998708 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e40ab610 con 0x7f79f40d0a50 2024-04-25T10:11:48.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79ebfff640 1 -- 172.21.15.60:0/1181998708 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e40b0070 con 0x7f79f40d0a50 2024-04-25T10:11:48.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79ebfff640 1 -- 172.21.15.60:0/1181998708 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e40ab9d0 con 0x7f79f40d0a50 2024-04-25T10:11:48.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.248+0000 7f79fba0b640 1 -- 172.21.15.60:0/1181998708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f40d0a50 msgr2=0x7f79f40b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 --2- 172.21.15.60:0/1181998708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f40d0a50 0x7f79f40b6c10 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f79f40d1590 tx=0x7f79e401c520 comp rx=0 tx=0).stop 2024-04-25T10:11:48.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 -- 172.21.15.60:0/1181998708 shutdown_connections 2024-04-25T10:11:48.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 --2- 172.21.15.60:0/1181998708 >> v2:172.21.15.100:3300/0 conn(0x7f79f40ce2b0 0x7f79f40ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 --2- 172.21.15.60:0/1181998708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f40d0a50 0x7f79f40b6c10 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 -- 172.21.15.60:0/1181998708 >> 172.21.15.60:0/1181998708 conn(0x7f79f406c090 msgr2=0x7f79f406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:48.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 -- 172.21.15.60:0/1181998708 shutdown_connections 2024-04-25T10:11:48.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 -- 172.21.15.60:0/1181998708 wait complete. 2024-04-25T10:11:48.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 Processor -- start 2024-04-25T10:11:48.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 -- start start 2024-04-25T10:11:48.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f40ce2b0 0x7f79f40b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79f40d0a50 0x7f79f40b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f415deb0 0x7f79f40c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40d31e0 con 0x7f79f40ce2b0 2024-04-25T10:11:48.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40d3060 con 0x7f79f40d0a50 2024-04-25T10:11:48.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40d3360 con 0x7f79f415deb0 2024-04-25T10:11:48.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79f97a7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f40ce2b0 0x7f79f40b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79f97a7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f40ce2b0 0x7f79f40b6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45626/0 (socket says 172.21.15.60:45626) 2024-04-25T10:11:48.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79f97a7640 1 -- 172.21.15.60:0/2273800937 learned_addr learned my addr 172.21.15.60:0/2273800937 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:48.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79f8fa6640 1 --2- 172.21.15.60:0/2273800937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79f40d0a50 0x7f79f40b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79f9fa8640 1 --2- 172.21.15.60:0/2273800937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f415deb0 0x7f79f40c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79f97a7640 1 -- 172.21.15.60:0/2273800937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f415deb0 msgr2=0x7f79f40c4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79f97a7640 1 --2- 172.21.15.60:0/2273800937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f415deb0 0x7f79f40c4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79f97a7640 1 -- 172.21.15.60:0/2273800937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79f40d0a50 msgr2=0x7f79f40b82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79f97a7640 1 --2- 172.21.15.60:0/2273800937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79f40d0a50 0x7f79f40b82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79f97a7640 1 -- 172.21.15.60:0/2273800937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f40c46d0 con 0x7f79f40ce2b0 2024-04-25T10:11:48.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79f9fa8640 1 --2- 172.21.15.60:0/2273800937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f415deb0 0x7f79f40c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:48.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79f8fa6640 1 --2- 172.21.15.60:0/2273800937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79f40d0a50 0x7f79f40b82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:48.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79f97a7640 1 --2- 172.21.15.60:0/2273800937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f40ce2b0 0x7f79f40b6bd0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c7c0 tx=0x7f79ec00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79e9ffb640 1 -- 172.21.15.60:0/2273800937 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec018070 con 0x7f79f40ce2b0 2024-04-25T10:11:48.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79e9ffb640 1 -- 172.21.15.60:0/2273800937 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec00a640 con 0x7f79f40ce2b0 2024-04-25T10:11:48.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 -- 172.21.15.60:0/2273800937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f40b7240 con 0x7f79f40ce2b0 2024-04-25T10:11:48.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79e9ffb640 1 -- 172.21.15.60:0/2273800937 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00aa30 con 0x7f79f40ce2b0 2024-04-25T10:11:48.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 -- 172.21.15.60:0/2273800937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f40b7800 con 0x7f79f40ce2b0 2024-04-25T10:11:48.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.252+0000 7f79fba0b640 1 -- 172.21.15.60:0/2273800937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f4074770 con 0x7f79f40ce2b0 2024-04-25T10:11:48.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.256+0000 7f79e9ffb640 1 -- 172.21.15.60:0/2273800937 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79ec01b070 con 0x7f79f40ce2b0 2024-04-25T10:11:48.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.256+0000 7f79e9ffb640 1 --2- 172.21.15.60:0/2273800937 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79d8066f60 0x7f79d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.256+0000 7f79e9ffb640 1 -- 172.21.15.60:0/2273800937 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79ec084630 con 0x7f79f40ce2b0 2024-04-25T10:11:48.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.256+0000 7f79f8fa6640 1 --2- 172.21.15.60:0/2273800937 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79d8066f60 0x7f79d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.256+0000 7f79f8fa6640 1 --2- 172.21.15.60:0/2273800937 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79d8066f60 0x7f79d8069420 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f79e40af000 tx=0x7f79e4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.260+0000 7f79e9ffb640 1 -- 172.21.15.60:0/2273800937 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79ec055640 con 0x7f79f40ce2b0 2024-04-25T10:11:48.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.460+0000 7f79fba0b640 1 -- 172.21.15.60:0/2273800937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79f40d17a0 con 0x7f79f40ce2b0 2024-04-25T10:11:48.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.460+0000 7f79e9ffb640 1 -- 172.21.15.60:0/2273800937 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f79ec054f40 con 0x7f79f40ce2b0 2024-04-25T10:11:48.469 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:48.469 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:11:47.412210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7653":{"gid":7653,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2455562778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2455562778},{"type":"v1","addr":"172.21.15.60:6835","nonce":2455562778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:48.469 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 55 2024-04-25T10:11:48.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.464+0000 7f79fba0b640 1 -- 172.21.15.60:0/2273800937 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79d8066f60 msgr2=0x7f79d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.464+0000 7f79fba0b640 1 --2- 172.21.15.60:0/2273800937 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79d8066f60 0x7f79d8069420 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f79e40af000 tx=0x7f79e4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.464+0000 7f79fba0b640 1 -- 172.21.15.60:0/2273800937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f40ce2b0 msgr2=0x7f79f40b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.464+0000 7f79fba0b640 1 --2- 172.21.15.60:0/2273800937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f40ce2b0 0x7f79f40b6bd0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c7c0 tx=0x7f79ec00cc80 comp rx=0 tx=0).stop 2024-04-25T10:11:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.464+0000 7f79fba0b640 1 -- 172.21.15.60:0/2273800937 shutdown_connections 2024-04-25T10:11:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.464+0000 7f79fba0b640 1 --2- 172.21.15.60:0/2273800937 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79d8066f60 0x7f79d8069420 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.464+0000 7f79fba0b640 1 --2- 172.21.15.60:0/2273800937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f415deb0 0x7f79f40c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.464+0000 7f79fba0b640 1 --2- 172.21.15.60:0/2273800937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79f40d0a50 0x7f79f40b82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.464+0000 7f79fba0b640 1 --2- 172.21.15.60:0/2273800937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f40ce2b0 0x7f79f40b6bd0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.464+0000 7f79fba0b640 1 -- 172.21.15.60:0/2273800937 >> 172.21.15.60:0/2273800937 conn(0x7f79f406c090 msgr2=0x7f79f414a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.464+0000 7f79fba0b640 1 -- 172.21.15.60:0/2273800937 shutdown_connections 2024-04-25T10:11:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.464+0000 7f79fba0b640 1 -- 172.21.15.60:0/2273800937 wait complete. 2024-04-25T10:11:48.489 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9d5605640 1 Processor -- start 2024-04-25T10:11:48.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9d5605640 1 -- start start 2024-04-25T10:11:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9d5605640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb9d015bc90 0x7fb9d015c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9d5605640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d0071ca0 con 0x7fb9d00ce7a0 2024-04-25T10:11:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9d5605640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d00cca40 con 0x7fb9d00b4660 2024-04-25T10:11:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9d5605640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d00cc370 con 0x7fb9d015bc90 2024-04-25T10:11:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9ceffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb9d00ce7a0 0x7fb9d00ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50836/0 (socket says 172.21.15.60:50836) 2024-04-25T10:11:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9ceffd640 1 -- 172.21.15.60:0/1653438677 learned_addr learned my addr 172.21.15.60:0/1653438677 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9ce7fc640 1 --2- 172.21.15.60:0/1653438677 >> v2:172.21.15.100:3300/0 conn(0x7fb9d015bc90 0x7fb9d015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cdffb640 1 -- 172.21.15.60:0/1653438677 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235502508 0 0) 0x7fb9d0071ca0 con 0x7fb9d00ce7a0 2024-04-25T10:11:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cdffb640 1 -- 172.21.15.60:0/1653438677 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9b8003530 con 0x7fb9d00ce7a0 2024-04-25T10:11:48.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cdffb640 1 -- 172.21.15.60:0/1653438677 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1370197228 0 0) 0x7fb9b8003530 con 0x7fb9d00ce7a0 2024-04-25T10:11:48.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cdffb640 1 -- 172.21.15.60:0/1653438677 >> v2:172.21.15.100:3300/0 conn(0x7fb9d015bc90 msgr2=0x7fb9d015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:48.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cdffb640 1 --2- 172.21.15.60:0/1653438677 >> v2:172.21.15.100:3300/0 conn(0x7fb9d015bc90 0x7fb9d015c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cdffb640 1 -- 172.21.15.60:0/1653438677 >> v1:172.21.15.100:6789/0 conn(0x7fb9d00b4660 legacy=0x7fb9d015c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cdffb640 1 -- 172.21.15.60:0/1653438677 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d00ccd20 con 0x7fb9d00ce7a0 2024-04-25T10:11:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cdffb640 1 -- 172.21.15.60:0/1653438677 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fb9c0001680 con 0x7fb9d00ce7a0 2024-04-25T10:11:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cdffb640 1 -- 172.21.15.60:0/1653438677 >> v1:172.21.15.60:6789/0 conn(0x7fb9d00ce7a0 legacy=0x7fb9d00ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cdffb640 1 --2- 172.21.15.60:0/1653438677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b80039c0 0x7fb9b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cdffb640 1 --2- 172.21.15.60:0/1653438677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b8004810 0x7fb9b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cdffb640 1 --2- 172.21.15.60:0/1653438677 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9b80051d0 0x7fb9b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cdffb640 1 -- 172.21.15.60:0/1653438677 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b800c040 con 0x7fb9b80039c0 2024-04-25T10:11:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cdffb640 1 -- 172.21.15.60:0/1653438677 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b800c1f0 con 0x7fb9b8004810 2024-04-25T10:11:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cdffb640 1 -- 172.21.15.60:0/1653438677 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b800c3f0 con 0x7fb9b80051d0 2024-04-25T10:11:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9ceffd640 1 --2- 172.21.15.60:0/1653438677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b80039c0 0x7fb9b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9ce7fc640 1 --2- 172.21.15.60:0/1653438677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b8004810 0x7fb9b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cf7fe640 1 --2- 172.21.15.60:0/1653438677 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9b80051d0 0x7fb9b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9ceffd640 1 -- 172.21.15.60:0/1653438677 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9b80051d0 msgr2=0x7fb9b800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9ceffd640 1 --2- 172.21.15.60:0/1653438677 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9b80051d0 0x7fb9b800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9ceffd640 1 -- 172.21.15.60:0/1653438677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b8004810 msgr2=0x7fb9b8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9ceffd640 1 --2- 172.21.15.60:0/1653438677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b8004810 0x7fb9b8004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9ceffd640 1 -- 172.21.15.60:0/1653438677 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b8003530 con 0x7fb9b80039c0 2024-04-25T10:11:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.596+0000 7fb9cf7fe640 1 -- 172.21.15.60:0/1653438677 reap_dead start 2024-04-25T10:11:48.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.600+0000 7fb9ceffd640 1 --2- 172.21.15.60:0/1653438677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b80039c0 0x7fb9b8003de0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000c240 tx=0x7fb9c001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321e577183851204 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.600+0000 7fb9cdffb640 1 -- 172.21.15.60:0/1653438677 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0007d20 con 0x7fb9b80039c0 2024-04-25T10:11:48.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.600+0000 7fb9cdffb640 1 -- 172.21.15.60:0/1653438677 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c00a9070 con 0x7fb9b80039c0 2024-04-25T10:11:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.600+0000 7fb9cdffb640 1 -- 172.21.15.60:0/1653438677 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0005da0 con 0x7fb9b80039c0 2024-04-25T10:11:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.600+0000 7fb9d5605640 1 -- 172.21.15.60:0/1653438677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b80039c0 msgr2=0x7fb9b8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.600+0000 7fb9d5605640 1 --2- 172.21.15.60:0/1653438677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b80039c0 0x7fb9b8003de0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000c240 tx=0x7fb9c001cc80 comp rx=0 tx=0).stop 2024-04-25T10:11:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.600+0000 7fb9d5605640 1 -- 172.21.15.60:0/1653438677 shutdown_connections 2024-04-25T10:11:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.600+0000 7fb9d5605640 1 --2- 172.21.15.60:0/1653438677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b80039c0 0x7fb9b8003de0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.600+0000 7fb9d5605640 1 -- 172.21.15.60:0/1653438677 >> 172.21.15.60:0/1653438677 conn(0x7fb9d00b12d0 msgr2=0x7fb9d00b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.600+0000 7fb9d5605640 1 -- 172.21.15.60:0/1653438677 shutdown_connections 2024-04-25T10:11:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.600+0000 7fb9d5605640 1 -- 172.21.15.60:0/1653438677 wait complete. 2024-04-25T10:11:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.600+0000 7fb9d5605640 1 Processor -- start 2024-04-25T10:11:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.600+0000 7fb9d5605640 1 -- start start 2024-04-25T10:11:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9d5605640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b80039c0 0x7fb9d013fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9d5605640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9d014d620 0x7fb9d014e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9d5605640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d014da80 0x7fb9d015e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9d5605640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d00b4280 con 0x7fb9d014da80 2024-04-25T10:11:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9d5605640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d00b3db0 con 0x7fb9b80039c0 2024-04-25T10:11:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9d5605640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d00b4400 con 0x7fb9d014d620 2024-04-25T10:11:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9cf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d014da80 0x7fb9d015e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9cf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d014da80 0x7fb9d015e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45640/0 (socket says 172.21.15.60:45640) 2024-04-25T10:11:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9cf7fe640 1 -- 172.21.15.60:0/3668463121 learned_addr learned my addr 172.21.15.60:0/3668463121 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9ceffd640 1 --2- 172.21.15.60:0/3668463121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b80039c0 0x7fb9d013fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9ce7fc640 1 --2- 172.21.15.60:0/3668463121 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9d014d620 0x7fb9d014e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9cf7fe640 1 -- 172.21.15.60:0/3668463121 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9d014d620 msgr2=0x7fb9d014e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9cf7fe640 1 --2- 172.21.15.60:0/3668463121 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9d014d620 0x7fb9d014e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9cf7fe640 1 -- 172.21.15.60:0/3668463121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b80039c0 msgr2=0x7fb9d013fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9cf7fe640 1 --2- 172.21.15.60:0/3668463121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b80039c0 0x7fb9d013fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9cf7fe640 1 -- 172.21.15.60:0/3668463121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d014f880 con 0x7fb9d014da80 2024-04-25T10:11:48.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9ceffd640 1 --2- 172.21.15.60:0/3668463121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b80039c0 0x7fb9d013fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:48.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9cf7fe640 1 --2- 172.21.15.60:0/3668463121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d014da80 0x7fb9d015e2b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fb9d006bbb0 tx=0x7fb9c800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9b77fe640 1 -- 172.21.15.60:0/3668463121 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8018070 con 0x7fb9d014da80 2024-04-25T10:11:48.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9b77fe640 1 -- 172.21.15.60:0/3668463121 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c8003db0 con 0x7fb9d014da80 2024-04-25T10:11:48.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9d5605640 1 -- 172.21.15.60:0/3668463121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d014f2c0 con 0x7fb9d014da80 2024-04-25T10:11:48.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9b77fe640 1 -- 172.21.15.60:0/3668463121 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80052b0 con 0x7fb9d014da80 2024-04-25T10:11:48.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9d5605640 1 -- 172.21.15.60:0/3668463121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d014ce20 con 0x7fb9d014da80 2024-04-25T10:11:48.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9d5605640 1 -- 172.21.15.60:0/3668463121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d0074770 con 0x7fb9d014da80 2024-04-25T10:11:48.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9b77fe640 1 -- 172.21.15.60:0/3668463121 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9c801b070 con 0x7fb9d014da80 2024-04-25T10:11:48.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9b77fe640 1 --2- 172.21.15.60:0/3668463121 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9bc066f90 0x7fb9bc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9b77fe640 1 -- 172.21.15.60:0/3668463121 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9c8084670 con 0x7fb9d014da80 2024-04-25T10:11:48.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.604+0000 7fb9ceffd640 1 --2- 172.21.15.60:0/3668463121 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9bc066f90 0x7fb9bc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.608+0000 7fb9ceffd640 1 --2- 172.21.15.60:0/3668463121 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9bc066f90 0x7fb9bc069450 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00ab000 tx=0x7fb9c00b0040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.608+0000 7fb9b77fe640 1 -- 172.21.15.60:0/3668463121 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb9c8055680 con 0x7fb9d014da80 2024-04-25T10:11:48.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.768+0000 7fb9d5605640 1 -- 172.21.15.60:0/3668463121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9d00b0080 con 0x7fb9d014da80 2024-04-25T10:11:48.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.772+0000 7fb9b77fe640 1 -- 172.21.15.60:0/3668463121 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16672 (secure 0 0 0) 0x7fb9c8054f80 con 0x7fb9d014da80 2024-04-25T10:11:48.781 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:48.781 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:11:46.035912+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:42.621502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:43.352679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:48.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.772+0000 7fb9d5605640 1 -- 172.21.15.60:0/3668463121 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9bc066f90 msgr2=0x7fb9bc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.772+0000 7fb9d5605640 1 --2- 172.21.15.60:0/3668463121 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9bc066f90 0x7fb9bc069450 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00ab000 tx=0x7fb9c00b0040 comp rx=0 tx=0).stop 2024-04-25T10:11:48.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.776+0000 7fb9d5605640 1 -- 172.21.15.60:0/3668463121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d014da80 msgr2=0x7fb9d015e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.776+0000 7fb9d5605640 1 --2- 172.21.15.60:0/3668463121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d014da80 0x7fb9d015e2b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fb9d006bbb0 tx=0x7fb9c800d9d0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.776+0000 7fb9d5605640 1 -- 172.21.15.60:0/3668463121 shutdown_connections 2024-04-25T10:11:48.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.776+0000 7fb9d5605640 1 --2- 172.21.15.60:0/3668463121 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9bc066f90 0x7fb9bc069450 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.776+0000 7fb9d5605640 1 --2- 172.21.15.60:0/3668463121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d014da80 0x7fb9d015e2b0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.776+0000 7fb9d5605640 1 --2- 172.21.15.60:0/3668463121 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9d014d620 0x7fb9d014e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.776+0000 7fb9d5605640 1 --2- 172.21.15.60:0/3668463121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b80039c0 0x7fb9d013fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.776+0000 7fb9d5605640 1 -- 172.21.15.60:0/3668463121 >> 172.21.15.60:0/3668463121 conn(0x7fb9d00b12d0 msgr2=0x7fb9d015c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:48.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.776+0000 7fb9d5605640 1 -- 172.21.15.60:0/3668463121 shutdown_connections 2024-04-25T10:11:48.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.776+0000 7fb9d5605640 1 -- 172.21.15.60:0/3668463121 wait complete. 2024-04-25T10:11:48.800 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:11:48.800 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:48.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.908+0000 7f584bf12640 1 Processor -- start 2024-04-25T10:11:48.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.908+0000 7f584bf12640 1 -- start start 2024-04-25T10:11:48.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.908+0000 7f584bf12640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5844071b20 0x7f5844071f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.908+0000 7f584bf12640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f584406e480 0x7f584406e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.908+0000 7f584bf12640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5844072440 con 0x7f584406d2f0 2024-04-25T10:11:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.908+0000 7f584bf12640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584415dc10 con 0x7f584406e480 2024-04-25T10:11:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.908+0000 7f584bf12640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584415def0 con 0x7f5844071b20 2024-04-25T10:11:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.908+0000 7f58494ad640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f584406e480 0x7f584406e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.908+0000 7f58494ad640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f584406e480 0x7f584406e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46794/0 (socket says 172.21.15.60:46794) 2024-04-25T10:11:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.908+0000 7f58494ad640 1 -- 172.21.15.60:0/2276572729 learned_addr learned my addr 172.21.15.60:0/2276572729 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.908+0000 7f5849cae640 1 --2- 172.21.15.60:0/2276572729 >> v2:172.21.15.100:3300/0 conn(0x7f5844071b20 0x7f5844071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.908+0000 7f58494ad640 1 -- 172.21.15.60:0/2276572729 >> v2:172.21.15.100:3300/0 conn(0x7f5844071b20 msgr2=0x7f5844071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.908+0000 7f58494ad640 1 --2- 172.21.15.60:0/2276572729 >> v2:172.21.15.100:3300/0 conn(0x7f5844071b20 0x7f5844071f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.908+0000 7f58494ad640 1 -- 172.21.15.60:0/2276572729 >> v1:172.21.15.100:6789/0 conn(0x7f584406d2f0 legacy=0x7f584406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.908+0000 7f58494ad640 1 -- 172.21.15.60:0/2276572729 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584415e1d0 con 0x7f584406e480 2024-04-25T10:11:48.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f58494ad640 1 --2- 172.21.15.60:0/2276572729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584406e480 0x7f584406e910 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f58440d05d0 tx=0x7f583401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa0ab712a2d62bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f5848cac640 1 -- 172.21.15.60:0/2276572729 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58340b0070 con 0x7f584406e480 2024-04-25T10:11:48.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f5848cac640 1 -- 172.21.15.60:0/2276572729 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58340ab600 con 0x7f584406e480 2024-04-25T10:11:48.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f5848cac640 1 -- 172.21.15.60:0/2276572729 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58340ab9d0 con 0x7f584406e480 2024-04-25T10:11:48.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584bf12640 1 -- 172.21.15.60:0/2276572729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584406e480 msgr2=0x7f584406e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584bf12640 1 --2- 172.21.15.60:0/2276572729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584406e480 0x7f584406e910 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f58440d05d0 tx=0x7f583401c450 comp rx=0 tx=0).stop 2024-04-25T10:11:48.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584bf12640 1 -- 172.21.15.60:0/2276572729 shutdown_connections 2024-04-25T10:11:48.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584bf12640 1 --2- 172.21.15.60:0/2276572729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584406e480 0x7f584406e910 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584bf12640 1 --2- 172.21.15.60:0/2276572729 >> v2:172.21.15.100:3300/0 conn(0x7f5844071b20 0x7f5844071f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584bf12640 1 -- 172.21.15.60:0/2276572729 >> 172.21.15.60:0/2276572729 conn(0x7f58440b12d0 msgr2=0x7f58440b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:48.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584bf12640 1 -- 172.21.15.60:0/2276572729 shutdown_connections 2024-04-25T10:11:48.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584bf12640 1 -- 172.21.15.60:0/2276572729 wait complete. 2024-04-25T10:11:48.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584bf12640 1 Processor -- start 2024-04-25T10:11:48.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584bf12640 1 -- start start 2024-04-25T10:11:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584bf12640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f584406d2f0 0x7f58440b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584bf12640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584406e480 0x7f584413df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584bf12640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5844071b20 0x7f5844154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584bf12640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58440cf070 con 0x7f584406e480 2024-04-25T10:11:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584bf12640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58440ceef0 con 0x7f584406d2f0 2024-04-25T10:11:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584bf12640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5844167670 con 0x7f5844071b20 2024-04-25T10:11:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f58494ad640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584406e480 0x7f584413df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f58494ad640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584406e480 0x7f584413df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46810/0 (socket says 172.21.15.60:46810) 2024-04-25T10:11:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f58494ad640 1 -- 172.21.15.60:0/278432207 learned_addr learned my addr 172.21.15.60:0/278432207 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f5849cae640 1 --2- 172.21.15.60:0/278432207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f584406d2f0 0x7f58440b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.912+0000 7f584a4af640 1 --2- 172.21.15.60:0/278432207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5844071b20 0x7f5844154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f58494ad640 1 -- 172.21.15.60:0/278432207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5844071b20 msgr2=0x7f5844154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f58494ad640 1 --2- 172.21.15.60:0/278432207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5844071b20 0x7f5844154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f58494ad640 1 -- 172.21.15.60:0/278432207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f584406d2f0 msgr2=0x7f58440b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f58494ad640 1 --2- 172.21.15.60:0/278432207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f584406d2f0 0x7f58440b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f58494ad640 1 -- 172.21.15.60:0/278432207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58440b2440 con 0x7f584406e480 2024-04-25T10:11:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f584a4af640 1 --2- 172.21.15.60:0/278432207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5844071b20 0x7f5844154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f5849cae640 1 --2- 172.21.15.60:0/278432207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f584406d2f0 0x7f58440b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f58494ad640 1 --2- 172.21.15.60:0/278432207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584406e480 0x7f584413df00 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f583401ab00 tx=0x7f58340191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f583a7fc640 1 -- 172.21.15.60:0/278432207 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58340b9070 con 0x7f584406e480 2024-04-25T10:11:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f583a7fc640 1 -- 172.21.15.60:0/278432207 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58340b4420 con 0x7f584406e480 2024-04-25T10:11:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f583a7fc640 1 -- 172.21.15.60:0/278432207 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58340b0040 con 0x7f584406e480 2024-04-25T10:11:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f584bf12640 1 -- 172.21.15.60:0/278432207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58440b26b0 con 0x7f584406e480 2024-04-25T10:11:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f584bf12640 1 -- 172.21.15.60:0/278432207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584413e440 con 0x7f584406e480 2024-04-25T10:11:48.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f584bf12640 1 -- 172.21.15.60:0/278432207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58440b2b60 con 0x7f584406e480 2024-04-25T10:11:48.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f583a7fc640 1 -- 172.21.15.60:0/278432207 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58340aa3a0 con 0x7f584406e480 2024-04-25T10:11:48.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f583a7fc640 1 --2- 172.21.15.60:0/278432207 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5828066f60 0x7f5828069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f583a7fc640 1 -- 172.21.15.60:0/278432207 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f583411e170 con 0x7f584406e480 2024-04-25T10:11:48.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f5849cae640 1 --2- 172.21.15.60:0/278432207 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5828066f60 0x7f5828069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.916+0000 7f5849cae640 1 --2- 172.21.15.60:0/278432207 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5828066f60 0x7f5828069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f583c009f50 tx=0x7f583c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:48.920+0000 7f583a7fc640 1 -- 172.21.15.60:0/278432207 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58440b2b60 con 0x7f584406e480 2024-04-25T10:11:49.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.072+0000 7f584bf12640 1 -- 172.21.15.60:0/278432207 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5844074770 con 0x7f5828066f60 2024-04-25T10:11:49.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.088+0000 7f583a7fc640 1 -- 172.21.15.60:0/278432207 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252078 (secure 0 0 0) 0x7f583c0588c0 con 0x7f5828066f60 2024-04-25T10:11:49.102 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:49.104 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:11:49.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.092+0000 7f584bf12640 1 -- 172.21.15.60:0/278432207 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5828066f60 msgr2=0x7f5828069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.092+0000 7f584bf12640 1 --2- 172.21.15.60:0/278432207 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5828066f60 0x7f5828069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f583c009f50 tx=0x7f583c00b040 comp rx=0 tx=0).stop 2024-04-25T10:11:49.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.092+0000 7f584bf12640 1 -- 172.21.15.60:0/278432207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584406e480 msgr2=0x7f584413df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.092+0000 7f584bf12640 1 --2- 172.21.15.60:0/278432207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584406e480 0x7f584413df00 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f583401ab00 tx=0x7f58340191f0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.092+0000 7f584bf12640 1 -- 172.21.15.60:0/278432207 shutdown_connections 2024-04-25T10:11:49.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.092+0000 7f584bf12640 1 --2- 172.21.15.60:0/278432207 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5828066f60 0x7f5828069420 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.092+0000 7f584bf12640 1 --2- 172.21.15.60:0/278432207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5844071b20 0x7f5844154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.092+0000 7f584bf12640 1 --2- 172.21.15.60:0/278432207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584406e480 0x7f584413df00 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.092+0000 7f584bf12640 1 --2- 172.21.15.60:0/278432207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f584406d2f0 0x7f58440b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.092+0000 7f584bf12640 1 -- 172.21.15.60:0/278432207 >> 172.21.15.60:0/278432207 conn(0x7f58440b12d0 msgr2=0x7f58440b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:49.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.092+0000 7f584bf12640 1 -- 172.21.15.60:0/278432207 shutdown_connections 2024-04-25T10:11:49.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.092+0000 7f584bf12640 1 -- 172.21.15.60:0/278432207 wait complete. 2024-04-25T10:11:49.117 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-04-25T10:11:48.522575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266824,"kb_used_data":9032,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754707896,"statfs":{"total":773094113280,"available":772820885504,"internally_reserved":0,"allocated":9248768,"data_stored":4585852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.545989"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335072+0000","last_change":"2024-04-25T10:11:45.063671+0000","last_active":"2024-04-25T10:11:46.335072+0000","last_peered":"2024-04-25T10:11:46.335072+0000","last_clean":"2024-04-25T10:11:46.335072+0000","last_became_active":"2024-04-25T10:11:45.059505+0000","last_became_peered":"2024-04-25T10:11:45.059505+0000","last_unstale":"2024-04-25T10:11:46.335072+0000","last_undegraded":"2024-04-25T10:11:46.335072+0000","last_fullsized":"2024-04-25T10:11:46.335072+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:53.161374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.042635+0000","last_change":"2024-04-25T10:11:45.061143+0000","last_active":"2024-04-25T10:11:46.042635+0000","last_peered":"2024-04-25T10:11:46.042635+0000","last_clean":"2024-04-25T10:11:46.042635+0000","last_became_active":"2024-04-25T10:11:45.060863+0000","last_became_peered":"2024-04-25T10:11:45.060863+0000","last_unstale":"2024-04-25T10:11:46.042635+0000","last_undegraded":"2024-04-25T10:11:46.042635+0000","last_fullsized":"2024-04-25T10:11:46.042635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:17.431486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207253+0000","last_change":"2024-04-25T10:11:45.077970+0000","last_active":"2024-04-25T10:11:46.207253+0000","last_peered":"2024-04-25T10:11:46.207253+0000","last_clean":"2024-04-25T10:11:46.207253+0000","last_became_active":"2024-04-25T10:11:45.077282+0000","last_became_peered":"2024-04-25T10:11:45.077282+0000","last_unstale":"2024-04-25T10:11:46.207253+0000","last_undegraded":"2024-04-25T10:11:46.207253+0000","last_fullsized":"2024-04-25T10:11:46.207253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:30.887901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334292+0000","last_change":"2024-04-25T10:11:45.049379+0000","last_active":"2024-04-25T10:11:46.334292+0000","last_peered":"2024-04-25T10:11:46.334292+0000","last_clean":"2024-04-25T10:11:46.334292+0000","last_became_active":"2024-04-25T10:11:45.048301+0000","last_became_peered":"2024-04-25T10:11:45.048301+0000","last_unstale":"2024-04-25T10:11:46.334292+0000","last_undegraded":"2024-04-25T10:11:46.334292+0000","last_fullsized":"2024-04-25T10:11:46.334292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:07.189560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047076+0000","last_change":"2024-04-25T10:11:45.045185+0000","last_active":"2024-04-25T10:11:46.047076+0000","last_peered":"2024-04-25T10:11:46.047076+0000","last_clean":"2024-04-25T10:11:46.047076+0000","last_became_active":"2024-04-25T10:11:45.044393+0000","last_became_peered":"2024-04-25T10:11:45.044393+0000","last_unstale":"2024-04-25T10:11:46.047076+0000","last_undegraded":"2024-04-25T10:11:46.047076+0000","last_fullsized":"2024-04-25T10:11:46.047076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:36.716191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046159+0000","last_change":"2024-04-25T10:11:45.061227+0000","last_active":"2024-04-25T10:11:46.046159+0000","last_peered":"2024-04-25T10:11:46.046159+0000","last_clean":"2024-04-25T10:11:46.046159+0000","last_became_active":"2024-04-25T10:11:45.060477+0000","last_became_peered":"2024-04-25T10:11:45.060477+0000","last_unstale":"2024-04-25T10:11:46.046159+0000","last_undegraded":"2024-04-25T10:11:46.046159+0000","last_fullsized":"2024-04-25T10:11:46.046159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:10.367189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.333393+0000","last_change":"2024-04-25T10:11:45.050209+0000","last_active":"2024-04-25T10:11:46.333393+0000","last_peered":"2024-04-25T10:11:46.333393+0000","last_clean":"2024-04-25T10:11:46.333393+0000","last_became_active":"2024-04-25T10:11:45.049856+0000","last_became_peered":"2024-04-25T10:11:45.049856+0000","last_unstale":"2024-04-25T10:11:46.333393+0000","last_undegraded":"2024-04-25T10:11:46.333393+0000","last_fullsized":"2024-04-25T10:11:46.333393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:57.205394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046214+0000","last_change":"2024-04-25T10:11:45.038897+0000","last_active":"2024-04-25T10:11:46.046214+0000","last_peered":"2024-04-25T10:11:46.046214+0000","last_clean":"2024-04-25T10:11:46.046214+0000","last_became_active":"2024-04-25T10:11:45.037970+0000","last_became_peered":"2024-04-25T10:11:45.037970+0000","last_unstale":"2024-04-25T10:11:46.046214+0000","last_undegraded":"2024-04-25T10:11:46.046214+0000","last_fullsized":"2024-04-25T10:11:46.046214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:53.161374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.048076+0000","last_change":"2024-04-25T10:11:45.039946+0000","last_active":"2024-04-25T10:11:46.048076+0000","last_peered":"2024-04-25T10:11:46.048076+0000","last_clean":"2024-04-25T10:11:46.048076+0000","last_became_active":"2024-04-25T10:11:45.039000+0000","last_became_peered":"2024-04-25T10:11:45.039000+0000","last_unstale":"2024-04-25T10:11:46.048076+0000","last_undegraded":"2024-04-25T10:11:46.048076+0000","last_fullsized":"2024-04-25T10:11:46.048076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:31.443603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335344+0000","last_change":"2024-04-25T10:11:45.058201+0000","last_active":"2024-04-25T10:11:46.335344+0000","last_peered":"2024-04-25T10:11:46.335344+0000","last_clean":"2024-04-25T10:11:46.335344+0000","last_became_active":"2024-04-25T10:11:45.050910+0000","last_became_peered":"2024-04-25T10:11:45.050910+0000","last_unstale":"2024-04-25T10:11:46.335344+0000","last_undegraded":"2024-04-25T10:11:46.335344+0000","last_fullsized":"2024-04-25T10:11:46.335344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:31.443603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.042321+0000","last_change":"2024-04-25T10:11:45.039883+0000","last_active":"2024-04-25T10:11:46.042321+0000","last_peered":"2024-04-25T10:11:46.042321+0000","last_clean":"2024-04-25T10:11:46.042321+0000","last_became_active":"2024-04-25T10:11:45.039047+0000","last_became_peered":"2024-04-25T10:11:45.039047+0000","last_unstale":"2024-04-25T10:11:46.042321+0000","last_undegraded":"2024-04-25T10:11:46.042321+0000","last_fullsized":"2024-04-25T10:11:46.042321+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:57.829518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334546+0000","last_change":"2024-04-25T10:11:45.060984+0000","last_active":"2024-04-25T10:11:46.334546+0000","last_peered":"2024-04-25T10:11:46.334546+0000","last_clean":"2024-04-25T10:11:46.334546+0000","last_became_active":"2024-04-25T10:11:45.059253+0000","last_became_peered":"2024-04-25T10:11:45.059253+0000","last_unstale":"2024-04-25T10:11:46.334546+0000","last_undegraded":"2024-04-25T10:11:46.334546+0000","last_fullsized":"2024-04-25T10:11:46.334546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:41.651037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207206+0000","last_change":"2024-04-25T10:11:45.051484+0000","last_active":"2024-04-25T10:11:46.207206+0000","last_peered":"2024-04-25T10:11:46.207206+0000","last_clean":"2024-04-25T10:11:46.207206+0000","last_became_active":"2024-04-25T10:11:45.049730+0000","last_became_peered":"2024-04-25T10:11:45.049730+0000","last_unstale":"2024-04-25T10:11:46.207206+0000","last_undegraded":"2024-04-25T10:11:46.207206+0000","last_fullsized":"2024-04-25T10:11:46.207206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:27.884490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.333825+0000","last_change":"2024-04-25T10:11:45.049988+0000","last_active":"2024-04-25T10:11:46.333825+0000","last_peered":"2024-04-25T10:11:46.333825+0000","last_clean":"2024-04-25T10:11:46.333825+0000","last_became_active":"2024-04-25T10:11:45.049300+0000","last_became_peered":"2024-04-25T10:11:45.049300+0000","last_unstale":"2024-04-25T10:11:46.333825+0000","last_undegraded":"2024-04-25T10:11:46.333825+0000","last_fullsized":"2024-04-25T10:11:46.333825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:35.241173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.332110+0000","last_change":"2024-04-25T10:11:45.041503+0000","last_active":"2024-04-25T10:11:46.332110+0000","last_peered":"2024-04-25T10:11:46.332110+0000","last_clean":"2024-04-25T10:11:46.332110+0000","last_became_active":"2024-04-25T10:11:45.039646+0000","last_became_peered":"2024-04-25T10:11:45.039646+0000","last_unstale":"2024-04-25T10:11:46.332110+0000","last_undegraded":"2024-04-25T10:11:46.332110+0000","last_fullsized":"2024-04-25T10:11:46.332110+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:01.752158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047480+0000","last_change":"2024-04-25T10:11:45.035192+0000","last_active":"2024-04-25T10:11:46.047480+0000","last_peered":"2024-04-25T10:11:46.047480+0000","last_clean":"2024-04-25T10:11:46.047480+0000","last_became_active":"2024-04-25T10:11:45.034440+0000","last_became_peered":"2024-04-25T10:11:45.034440+0000","last_unstale":"2024-04-25T10:11:46.047480+0000","last_undegraded":"2024-04-25T10:11:46.047480+0000","last_fullsized":"2024-04-25T10:11:46.047480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:48.597040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047207+0000","last_change":"2024-04-25T10:11:45.060439+0000","last_active":"2024-04-25T10:11:46.047207+0000","last_peered":"2024-04-25T10:11:46.047207+0000","last_clean":"2024-04-25T10:11:46.047207+0000","last_became_active":"2024-04-25T10:11:45.059732+0000","last_became_peered":"2024-04-25T10:11:45.059732+0000","last_unstale":"2024-04-25T10:11:46.047207+0000","last_undegraded":"2024-04-25T10:11:46.047207+0000","last_fullsized":"2024-04-25T10:11:46.047207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:33.434871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.332995+0000","last_change":"2024-04-25T10:11:45.048635+0000","last_active":"2024-04-25T10:11:46.332995+0000","last_peered":"2024-04-25T10:11:46.332995+0000","last_clean":"2024-04-25T10:11:46.332995+0000","last_became_active":"2024-04-25T10:11:45.045910+0000","last_became_peered":"2024-04-25T10:11:45.045910+0000","last_unstale":"2024-04-25T10:11:46.332995+0000","last_undegraded":"2024-04-25T10:11:46.332995+0000","last_fullsized":"2024-04-25T10:11:46.332995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:11.618132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.060732+0000","last_change":"2024-04-25T10:11:45.046575+0000","last_active":"2024-04-25T10:11:46.060732+0000","last_peered":"2024-04-25T10:11:46.060732+0000","last_clean":"2024-04-25T10:11:46.060732+0000","last_became_active":"2024-04-25T10:11:45.045722+0000","last_became_peered":"2024-04-25T10:11:45.045722+0000","last_unstale":"2024-04-25T10:11:46.060732+0000","last_undegraded":"2024-04-25T10:11:46.060732+0000","last_fullsized":"2024-04-25T10:11:46.060732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:46.497467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334222+0000","last_change":"2024-04-25T10:11:45.052267+0000","last_active":"2024-04-25T10:11:46.334222+0000","last_peered":"2024-04-25T10:11:46.334222+0000","last_clean":"2024-04-25T10:11:46.334222+0000","last_became_active":"2024-04-25T10:11:45.050259+0000","last_became_peered":"2024-04-25T10:11:45.050259+0000","last_unstale":"2024-04-25T10:11:46.334222+0000","last_undegraded":"2024-04-25T10:11:46.334222+0000","last_fullsized":"2024-04-25T10:11:46.334222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:37.283701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334085+0000","last_change":"2024-04-25T10:11:45.049528+0000","last_active":"2024-04-25T10:11:46.334085+0000","last_peered":"2024-04-25T10:11:46.334085+0000","last_clean":"2024-04-25T10:11:46.334085+0000","last_became_active":"2024-04-25T10:11:45.048998+0000","last_became_peered":"2024-04-25T10:11:45.048998+0000","last_unstale":"2024-04-25T10:11:46.334085+0000","last_undegraded":"2024-04-25T10:11:46.334085+0000","last_fullsized":"2024-04-25T10:11:46.334085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:42.843817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.043229+0000","last_change":"2024-04-25T10:11:45.042201+0000","last_active":"2024-04-25T10:11:46.043229+0000","last_peered":"2024-04-25T10:11:46.043229+0000","last_clean":"2024-04-25T10:11:46.043229+0000","last_became_active":"2024-04-25T10:11:45.040941+0000","last_became_peered":"2024-04-25T10:11:45.040941+0000","last_unstale":"2024-04-25T10:11:46.043229+0000","last_undegraded":"2024-04-25T10:11:46.043229+0000","last_fullsized":"2024-04-25T10:11:46.043229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:59.172072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334269+0000","last_change":"2024-04-25T10:11:45.062858+0000","last_active":"2024-04-25T10:11:46.334269+0000","last_peered":"2024-04-25T10:11:46.334269+0000","last_clean":"2024-04-25T10:11:46.334269+0000","last_became_active":"2024-04-25T10:11:45.061828+0000","last_became_peered":"2024-04-25T10:11:45.061828+0000","last_unstale":"2024-04-25T10:11:46.334269+0000","last_undegraded":"2024-04-25T10:11:46.334269+0000","last_fullsized":"2024-04-25T10:11:46.334269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:38.898303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.333773+0000","last_change":"2024-04-25T10:11:45.054556+0000","last_active":"2024-04-25T10:11:46.333773+0000","last_peered":"2024-04-25T10:11:46.333773+0000","last_clean":"2024-04-25T10:11:46.333773+0000","last_became_active":"2024-04-25T10:11:45.053843+0000","last_became_peered":"2024-04-25T10:11:45.053843+0000","last_unstale":"2024-04-25T10:11:46.333773+0000","last_undegraded":"2024-04-25T10:11:46.333773+0000","last_fullsized":"2024-04-25T10:11:46.333773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:13.636120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.042430+0000","last_change":"2024-04-25T10:11:45.041749+0000","last_active":"2024-04-25T10:11:46.042430+0000","last_peered":"2024-04-25T10:11:46.042430+0000","last_clean":"2024-04-25T10:11:46.042430+0000","last_became_active":"2024-04-25T10:11:45.040929+0000","last_became_peered":"2024-04-25T10:11:45.040929+0000","last_unstale":"2024-04-25T10:11:46.042430+0000","last_undegraded":"2024-04-25T10:11:46.042430+0000","last_fullsized":"2024-04-25T10:11:46.042430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:12.437425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207197+0000","last_change":"2024-04-25T10:11:45.057490+0000","last_active":"2024-04-25T10:11:46.207197+0000","last_peered":"2024-04-25T10:11:46.207197+0000","last_clean":"2024-04-25T10:11:46.207197+0000","last_became_active":"2024-04-25T10:11:45.056404+0000","last_became_peered":"2024-04-25T10:11:45.056404+0000","last_unstale":"2024-04-25T10:11:46.207197+0000","last_undegraded":"2024-04-25T10:11:46.207197+0000","last_fullsized":"2024-04-25T10:11:46.207197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:08.880560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047047+0000","last_change":"2024-04-25T10:11:45.032175+0000","last_active":"2024-04-25T10:11:46.047047+0000","last_peered":"2024-04-25T10:11:46.047047+0000","last_clean":"2024-04-25T10:11:46.047047+0000","last_became_active":"2024-04-25T10:11:45.031392+0000","last_became_peered":"2024-04-25T10:11:45.031392+0000","last_unstale":"2024-04-25T10:11:46.047047+0000","last_undegraded":"2024-04-25T10:11:46.047047+0000","last_fullsized":"2024-04-25T10:11:46.047047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:25.865771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334706+0000","last_change":"2024-04-25T10:11:45.058024+0000","last_active":"2024-04-25T10:11:46.334706+0000","last_peered":"2024-04-25T10:11:46.334706+0000","last_clean":"2024-04-25T10:11:46.334706+0000","last_became_active":"2024-04-25T10:11:45.057406+0000","last_became_peered":"2024-04-25T10:11:45.057406+0000","last_unstale":"2024-04-25T10:11:46.334706+0000","last_undegraded":"2024-04-25T10:11:46.334706+0000","last_fullsized":"2024-04-25T10:11:46.334706+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:03.603847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.042272+0000","last_change":"2024-04-25T10:11:45.041831+0000","last_active":"2024-04-25T10:11:46.042272+0000","last_peered":"2024-04-25T10:11:46.042272+0000","last_clean":"2024-04-25T10:11:46.042272+0000","last_became_active":"2024-04-25T10:11:45.040626+0000","last_became_peered":"2024-04-25T10:11:45.040626+0000","last_unstale":"2024-04-25T10:11:46.042272+0000","last_undegraded":"2024-04-25T10:11:46.042272+0000","last_fullsized":"2024-04-25T10:11:46.042272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:25.649590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.042965+0000","last_change":"2024-04-25T10:11:45.057574+0000","last_active":"2024-04-25T10:11:46.042965+0000","last_peered":"2024-04-25T10:11:46.042965+0000","last_clean":"2024-04-25T10:11:46.042965+0000","last_became_active":"2024-04-25T10:11:45.057208+0000","last_became_peered":"2024-04-25T10:11:45.057208+0000","last_unstale":"2024-04-25T10:11:46.042965+0000","last_undegraded":"2024-04-25T10:11:46.042965+0000","last_fullsized":"2024-04-25T10:11:46.042965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:58.307282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.048082+0000","last_change":"2024-04-25T10:11:45.054418+0000","last_active":"2024-04-25T10:11:46.048082+0000","last_peered":"2024-04-25T10:11:46.048082+0000","last_clean":"2024-04-25T10:11:46.048082+0000","last_became_active":"2024-04-25T10:11:45.053638+0000","last_became_peered":"2024-04-25T10:11:45.053638+0000","last_unstale":"2024-04-25T10:11:46.048082+0000","last_undegraded":"2024-04-25T10:11:46.048082+0000","last_fullsized":"2024-04-25T10:11:46.048082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:21.579326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334075+0000","last_change":"2024-04-25T10:11:45.042287+0000","last_active":"2024-04-25T10:11:46.334075+0000","last_peered":"2024-04-25T10:11:46.334075+0000","last_clean":"2024-04-25T10:11:46.334075+0000","last_became_active":"2024-04-25T10:11:45.041446+0000","last_became_peered":"2024-04-25T10:11:45.041446+0000","last_unstale":"2024-04-25T10:11:46.334075+0000","last_undegraded":"2024-04-25T10:11:46.334075+0000","last_fullsized":"2024-04-25T10:11:46.334075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:17.846024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047988+0000","last_change":"2024-04-25T10:11:45.047629+0000","last_active":"2024-04-25T10:11:46.047988+0000","last_peered":"2024-04-25T10:11:46.047988+0000","last_clean":"2024-04-25T10:11:46.047988+0000","last_became_active":"2024-04-25T10:11:45.046855+0000","last_became_peered":"2024-04-25T10:11:45.046855+0000","last_unstale":"2024-04-25T10:11:46.047988+0000","last_undegraded":"2024-04-25T10:11:46.047988+0000","last_fullsized":"2024-04-25T10:11:46.047988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:34.255770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335539+0000","last_change":"2024-04-25T10:11:44.015887+0000","last_active":"2024-04-25T10:11:46.335539+0000","last_peered":"2024-04-25T10:11:46.335539+0000","last_clean":"2024-04-25T10:11:46.335539+0000","last_became_active":"2024-04-25T10:11:44.015192+0000","last_became_peered":"2024-04-25T10:11:44.015192+0000","last_unstale":"2024-04-25T10:11:46.335539+0000","last_undegraded":"2024-04-25T10:11:46.335539+0000","last_fullsized":"2024-04-25T10:11:46.335539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:58.640565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335244+0000","last_change":"2024-04-25T10:11:45.053014+0000","last_active":"2024-04-25T10:11:46.335244+0000","last_peered":"2024-04-25T10:11:46.335244+0000","last_clean":"2024-04-25T10:11:46.335244+0000","last_became_active":"2024-04-25T10:11:45.052232+0000","last_became_peered":"2024-04-25T10:11:45.052232+0000","last_unstale":"2024-04-25T10:11:46.335244+0000","last_undegraded":"2024-04-25T10:11:46.335244+0000","last_fullsized":"2024-04-25T10:11:46.335244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:40.527351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.048127+0000","last_change":"2024-04-25T10:11:44.016115+0000","last_active":"2024-04-25T10:11:46.048127+0000","last_peered":"2024-04-25T10:11:46.048127+0000","last_clean":"2024-04-25T10:11:46.048127+0000","last_became_active":"2024-04-25T10:11:44.015408+0000","last_became_peered":"2024-04-25T10:11:44.015408+0000","last_unstale":"2024-04-25T10:11:46.048127+0000","last_undegraded":"2024-04-25T10:11:46.048127+0000","last_fullsized":"2024-04-25T10:11:46.048127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:11.430066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335067+0000","last_change":"2024-04-25T10:11:45.056604+0000","last_active":"2024-04-25T10:11:46.335067+0000","last_peered":"2024-04-25T10:11:46.335067+0000","last_clean":"2024-04-25T10:11:46.335067+0000","last_became_active":"2024-04-25T10:11:45.055735+0000","last_became_peered":"2024-04-25T10:11:45.055735+0000","last_unstale":"2024-04-25T10:11:46.335067+0000","last_undegraded":"2024-04-25T10:11:46.335067+0000","last_fullsized":"2024-04-25T10:11:46.335067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:10.367189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.060915+0000","last_change":"2024-04-25T10:11:44.001270+0000","last_active":"2024-04-25T10:11:46.060915+0000","last_peered":"2024-04-25T10:11:46.060915+0000","last_clean":"2024-04-25T10:11:46.060915+0000","last_became_active":"2024-04-25T10:11:44.000933+0000","last_became_peered":"2024-04-25T10:11:44.000933+0000","last_unstale":"2024-04-25T10:11:46.060915+0000","last_undegraded":"2024-04-25T10:11:46.060915+0000","last_fullsized":"2024-04-25T10:11:46.060915+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:26.219939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207334+0000","last_change":"2024-04-25T10:11:45.061058+0000","last_active":"2024-04-25T10:11:46.207334+0000","last_peered":"2024-04-25T10:11:46.207334+0000","last_clean":"2024-04-25T10:11:46.207334+0000","last_became_active":"2024-04-25T10:11:45.060406+0000","last_became_peered":"2024-04-25T10:11:45.060406+0000","last_unstale":"2024-04-25T10:11:46.207334+0000","last_undegraded":"2024-04-25T10:11:46.207334+0000","last_fullsized":"2024-04-25T10:11:46.207334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:23.214924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.061241+0000","last_change":"2024-04-25T10:11:44.001285+0000","last_active":"2024-04-25T10:11:46.061241+0000","last_peered":"2024-04-25T10:11:46.061241+0000","last_clean":"2024-04-25T10:11:46.061241+0000","last_became_active":"2024-04-25T10:11:44.000926+0000","last_became_peered":"2024-04-25T10:11:44.000926+0000","last_unstale":"2024-04-25T10:11:46.061241+0000","last_undegraded":"2024-04-25T10:11:46.061241+0000","last_fullsized":"2024-04-25T10:11:46.061241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:40.643678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.043181+0000","last_change":"2024-04-25T10:11:45.056307+0000","last_active":"2024-04-25T10:11:46.043181+0000","last_peered":"2024-04-25T10:11:46.043181+0000","last_clean":"2024-04-25T10:11:46.043181+0000","last_became_active":"2024-04-25T10:11:45.055925+0000","last_became_peered":"2024-04-25T10:11:45.055925+0000","last_unstale":"2024-04-25T10:11:46.043181+0000","last_undegraded":"2024-04-25T10:11:46.043181+0000","last_fullsized":"2024-04-25T10:11:46.043181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:44.618524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.048060+0000","last_change":"2024-04-25T10:11:44.023047+0000","last_active":"2024-04-25T10:11:46.048060+0000","last_peered":"2024-04-25T10:11:46.048060+0000","last_clean":"2024-04-25T10:11:46.048060+0000","last_became_active":"2024-04-25T10:11:44.022221+0000","last_became_peered":"2024-04-25T10:11:44.022221+0000","last_unstale":"2024-04-25T10:11:46.048060+0000","last_undegraded":"2024-04-25T10:11:46.048060+0000","last_fullsized":"2024-04-25T10:11:46.048060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:56.822160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047905+0000","last_change":"2024-04-25T10:11:45.058562+0000","last_active":"2024-04-25T10:11:46.047905+0000","last_peered":"2024-04-25T10:11:46.047905+0000","last_clean":"2024-04-25T10:11:46.047905+0000","last_became_active":"2024-04-25T10:11:45.057926+0000","last_became_peered":"2024-04-25T10:11:45.057926+0000","last_unstale":"2024-04-25T10:11:46.047905+0000","last_undegraded":"2024-04-25T10:11:46.047905+0000","last_fullsized":"2024-04-25T10:11:46.047905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:05.516707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046653+0000","last_change":"2024-04-25T10:11:44.017757+0000","last_active":"2024-04-25T10:11:46.046653+0000","last_peered":"2024-04-25T10:11:46.046653+0000","last_clean":"2024-04-25T10:11:46.046653+0000","last_became_active":"2024-04-25T10:11:44.016976+0000","last_became_peered":"2024-04-25T10:11:44.016976+0000","last_unstale":"2024-04-25T10:11:46.046653+0000","last_undegraded":"2024-04-25T10:11:46.046653+0000","last_fullsized":"2024-04-25T10:11:46.046653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:29.023830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.206675+0000","last_change":"2024-04-25T10:11:45.051305+0000","last_active":"2024-04-25T10:11:46.206675+0000","last_peered":"2024-04-25T10:11:46.206675+0000","last_clean":"2024-04-25T10:11:46.206675+0000","last_became_active":"2024-04-25T10:11:45.049893+0000","last_became_peered":"2024-04-25T10:11:45.049893+0000","last_unstale":"2024-04-25T10:11:46.206675+0000","last_undegraded":"2024-04-25T10:11:46.206675+0000","last_fullsized":"2024-04-25T10:11:46.206675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:10.057697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.061589+0000","last_change":"2024-04-25T10:11:44.012240+0000","last_active":"2024-04-25T10:11:46.061589+0000","last_peered":"2024-04-25T10:11:46.061589+0000","last_clean":"2024-04-25T10:11:46.061589+0000","last_became_active":"2024-04-25T10:11:44.011969+0000","last_became_peered":"2024-04-25T10:11:44.011969+0000","last_unstale":"2024-04-25T10:11:46.061589+0000","last_undegraded":"2024-04-25T10:11:46.061589+0000","last_fullsized":"2024-04-25T10:11:46.061589+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:24.512219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.332509+0000","last_change":"2024-04-25T10:11:45.040153+0000","last_active":"2024-04-25T10:11:46.332509+0000","last_peered":"2024-04-25T10:11:46.332509+0000","last_clean":"2024-04-25T10:11:46.332509+0000","last_became_active":"2024-04-25T10:11:45.039005+0000","last_became_peered":"2024-04-25T10:11:45.039005+0000","last_unstale":"2024-04-25T10:11:46.332509+0000","last_undegraded":"2024-04-25T10:11:46.332509+0000","last_fullsized":"2024-04-25T10:11:46.332509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:45.876545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.041785+0000","last_change":"2024-04-25T10:11:44.008870+0000","last_active":"2024-04-25T10:11:46.041785+0000","last_peered":"2024-04-25T10:11:46.041785+0000","last_clean":"2024-04-25T10:11:46.041785+0000","last_became_active":"2024-04-25T10:11:44.008466+0000","last_became_peered":"2024-04-25T10:11:44.008466+0000","last_unstale":"2024-04-25T10:11:46.041785+0000","last_undegraded":"2024-04-25T10:11:46.041785+0000","last_fullsized":"2024-04-25T10:11:46.041785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:33.581181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.333426+0000","last_change":"2024-04-25T10:11:45.042616+0000","last_active":"2024-04-25T10:11:46.333426+0000","last_peered":"2024-04-25T10:11:46.333426+0000","last_clean":"2024-04-25T10:11:46.333426+0000","last_became_active":"2024-04-25T10:11:45.041908+0000","last_became_peered":"2024-04-25T10:11:45.041908+0000","last_unstale":"2024-04-25T10:11:46.333426+0000","last_undegraded":"2024-04-25T10:11:46.333426+0000","last_fullsized":"2024-04-25T10:11:46.333426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:56.943947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335806+0000","last_change":"2024-04-25T10:11:44.029573+0000","last_active":"2024-04-25T10:11:46.335806+0000","last_peered":"2024-04-25T10:11:46.335806+0000","last_clean":"2024-04-25T10:11:46.335806+0000","last_became_active":"2024-04-25T10:11:44.028845+0000","last_became_peered":"2024-04-25T10:11:44.028845+0000","last_unstale":"2024-04-25T10:11:46.335806+0000","last_undegraded":"2024-04-25T10:11:46.335806+0000","last_fullsized":"2024-04-25T10:11:46.335806+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:54.498636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.048171+0000","last_change":"2024-04-25T10:11:45.061066+0000","last_active":"2024-04-25T10:11:46.048171+0000","last_peered":"2024-04-25T10:11:46.048171+0000","last_clean":"2024-04-25T10:11:46.048171+0000","last_became_active":"2024-04-25T10:11:45.060314+0000","last_became_peered":"2024-04-25T10:11:45.060314+0000","last_unstale":"2024-04-25T10:11:46.048171+0000","last_undegraded":"2024-04-25T10:11:46.048171+0000","last_fullsized":"2024-04-25T10:11:46.048171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:01.882798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335354+0000","last_change":"2024-04-25T10:11:44.015020+0000","last_active":"2024-04-25T10:11:46.335354+0000","last_peered":"2024-04-25T10:11:46.335354+0000","last_clean":"2024-04-25T10:11:46.335354+0000","last_became_active":"2024-04-25T10:11:44.014329+0000","last_became_peered":"2024-04-25T10:11:44.014329+0000","last_unstale":"2024-04-25T10:11:46.335354+0000","last_undegraded":"2024-04-25T10:11:46.335354+0000","last_fullsized":"2024-04-25T10:11:46.335354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:45.766870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.061009+0000","last_change":"2024-04-25T10:11:45.055583+0000","last_active":"2024-04-25T10:11:46.061009+0000","last_peered":"2024-04-25T10:11:46.061009+0000","last_clean":"2024-04-25T10:11:46.061009+0000","last_became_active":"2024-04-25T10:11:45.054621+0000","last_became_peered":"2024-04-25T10:11:45.054621+0000","last_unstale":"2024-04-25T10:11:46.061009+0000","last_undegraded":"2024-04-25T10:11:46.061009+0000","last_fullsized":"2024-04-25T10:11:46.061009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:33.024532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335021+0000","last_change":"2024-04-25T10:11:44.026968+0000","last_active":"2024-04-25T10:11:46.335021+0000","last_peered":"2024-04-25T10:11:46.335021+0000","last_clean":"2024-04-25T10:11:46.335021+0000","last_became_active":"2024-04-25T10:11:44.026321+0000","last_became_peered":"2024-04-25T10:11:44.026321+0000","last_unstale":"2024-04-25T10:11:46.335021+0000","last_undegraded":"2024-04-25T10:11:46.335021+0000","last_fullsized":"2024-04-25T10:11:46.335021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:28.871503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047847+0000","last_change":"2024-04-25T10:11:45.053749+0000","last_active":"2024-04-25T10:11:46.047847+0000","last_peered":"2024-04-25T10:11:46.047847+0000","last_clean":"2024-04-25T10:11:46.047847+0000","last_became_active":"2024-04-25T10:11:45.053039+0000","last_became_peered":"2024-04-25T10:11:45.053039+0000","last_unstale":"2024-04-25T10:11:46.047847+0000","last_undegraded":"2024-04-25T10:11:46.047847+0000","last_fullsized":"2024-04-25T10:11:46.047847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:06.439806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"81'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047265+0000","last_change":"2024-04-25T10:11:44.006862+0000","last_active":"2024-04-25T10:11:46.047265+0000","last_peered":"2024-04-25T10:11:46.047265+0000","last_clean":"2024-04-25T10:11:46.047265+0000","last_became_active":"2024-04-25T10:11:44.006146+0000","last_became_peered":"2024-04-25T10:11:44.006146+0000","last_unstale":"2024-04-25T10:11:46.047265+0000","last_undegraded":"2024-04-25T10:11:46.047265+0000","last_fullsized":"2024-04-25T10:11:46.047265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:06.738500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.060723+0000","last_change":"2024-04-25T10:11:45.053114+0000","last_active":"2024-04-25T10:11:46.060723+0000","last_peered":"2024-04-25T10:11:46.060723+0000","last_clean":"2024-04-25T10:11:46.060723+0000","last_became_active":"2024-04-25T10:11:45.052733+0000","last_became_peered":"2024-04-25T10:11:45.052733+0000","last_unstale":"2024-04-25T10:11:46.060723+0000","last_undegraded":"2024-04-25T10:11:46.060723+0000","last_fullsized":"2024-04-25T10:11:46.060723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:22.178842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.042533+0000","last_change":"2024-04-25T10:11:44.014482+0000","last_active":"2024-04-25T10:11:46.042533+0000","last_peered":"2024-04-25T10:11:46.042533+0000","last_clean":"2024-04-25T10:11:46.042533+0000","last_became_active":"2024-04-25T10:11:44.013943+0000","last_became_peered":"2024-04-25T10:11:44.013943+0000","last_unstale":"2024-04-25T10:11:46.042533+0000","last_undegraded":"2024-04-25T10:11:46.042533+0000","last_fullsized":"2024-04-25T10:11:46.042533+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:22.265695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207169+0000","last_change":"2024-04-25T10:11:45.056116+0000","last_active":"2024-04-25T10:11:46.207169+0000","last_peered":"2024-04-25T10:11:46.207169+0000","last_clean":"2024-04-25T10:11:46.207169+0000","last_became_active":"2024-04-25T10:11:45.050722+0000","last_became_peered":"2024-04-25T10:11:45.050722+0000","last_unstale":"2024-04-25T10:11:46.207169+0000","last_undegraded":"2024-04-25T10:11:46.207169+0000","last_fullsized":"2024-04-25T10:11:46.207169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:57.429333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.332801+0000","last_change":"2024-04-25T10:11:44.016162+0000","last_active":"2024-04-25T10:11:46.332801+0000","last_peered":"2024-04-25T10:11:46.332801+0000","last_clean":"2024-04-25T10:11:46.332801+0000","last_became_active":"2024-04-25T10:11:44.014948+0000","last_became_peered":"2024-04-25T10:11:44.014948+0000","last_unstale":"2024-04-25T10:11:46.332801+0000","last_undegraded":"2024-04-25T10:11:46.332801+0000","last_fullsized":"2024-04-25T10:11:46.332801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:32.017173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.041623+0000","last_change":"2024-04-25T10:11:45.057510+0000","last_active":"2024-04-25T10:11:46.041623+0000","last_peered":"2024-04-25T10:11:46.041623+0000","last_clean":"2024-04-25T10:11:46.041623+0000","last_became_active":"2024-04-25T10:11:45.057077+0000","last_became_peered":"2024-04-25T10:11:45.057077+0000","last_unstale":"2024-04-25T10:11:46.041623+0000","last_undegraded":"2024-04-25T10:11:46.041623+0000","last_fullsized":"2024-04-25T10:11:46.041623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:30.031189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046137+0000","last_change":"2024-04-25T10:11:44.008928+0000","last_active":"2024-04-25T10:11:46.046137+0000","last_peered":"2024-04-25T10:11:46.046137+0000","last_clean":"2024-04-25T10:11:46.046137+0000","last_became_active":"2024-04-25T10:11:44.007992+0000","last_became_peered":"2024-04-25T10:11:44.007992+0000","last_unstale":"2024-04-25T10:11:46.046137+0000","last_undegraded":"2024-04-25T10:11:46.046137+0000","last_fullsized":"2024-04-25T10:11:46.046137+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:59.862550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.041827+0000","last_change":"2024-04-25T10:11:45.033718+0000","last_active":"2024-04-25T10:11:46.041827+0000","last_peered":"2024-04-25T10:11:46.041827+0000","last_clean":"2024-04-25T10:11:46.041827+0000","last_became_active":"2024-04-25T10:11:45.032911+0000","last_became_peered":"2024-04-25T10:11:45.032911+0000","last_unstale":"2024-04-25T10:11:46.041827+0000","last_undegraded":"2024-04-25T10:11:46.041827+0000","last_fullsized":"2024-04-25T10:11:46.041827+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:41.700969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.060355+0000","last_change":"2024-04-25T10:11:44.010323+0000","last_active":"2024-04-25T10:11:46.060355+0000","last_peered":"2024-04-25T10:11:46.060355+0000","last_clean":"2024-04-25T10:11:46.060355+0000","last_became_active":"2024-04-25T10:11:44.009951+0000","last_became_peered":"2024-04-25T10:11:44.009951+0000","last_unstale":"2024-04-25T10:11:46.060355+0000","last_undegraded":"2024-04-25T10:11:46.060355+0000","last_fullsized":"2024-04-25T10:11:46.060355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:37.890944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207401+0000","last_change":"2024-04-25T10:11:45.050441+0000","last_active":"2024-04-25T10:11:46.207401+0000","last_peered":"2024-04-25T10:11:46.207401+0000","last_clean":"2024-04-25T10:11:46.207401+0000","last_became_active":"2024-04-25T10:11:45.049635+0000","last_became_peered":"2024-04-25T10:11:45.049635+0000","last_unstale":"2024-04-25T10:11:46.207401+0000","last_undegraded":"2024-04-25T10:11:46.207401+0000","last_fullsized":"2024-04-25T10:11:46.207401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:28.530096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335958+0000","last_change":"2024-04-25T10:11:44.027297+0000","last_active":"2024-04-25T10:11:46.335958+0000","last_peered":"2024-04-25T10:11:46.335958+0000","last_clean":"2024-04-25T10:11:46.335958+0000","last_became_active":"2024-04-25T10:11:44.026474+0000","last_became_peered":"2024-04-25T10:11:44.026474+0000","last_unstale":"2024-04-25T10:11:46.335958+0000","last_undegraded":"2024-04-25T10:11:46.335958+0000","last_fullsized":"2024-04-25T10:11:46.335958+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:11.465700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335548+0000","last_change":"2024-04-25T10:11:45.066030+0000","last_active":"2024-04-25T10:11:46.335548+0000","last_peered":"2024-04-25T10:11:46.335548+0000","last_clean":"2024-04-25T10:11:46.335548+0000","last_became_active":"2024-04-25T10:11:45.064286+0000","last_became_peered":"2024-04-25T10:11:45.064286+0000","last_unstale":"2024-04-25T10:11:46.335548+0000","last_undegraded":"2024-04-25T10:11:46.335548+0000","last_fullsized":"2024-04-25T10:11:46.335548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:01.882798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.060873+0000","last_change":"2024-04-25T10:11:44.007340+0000","last_active":"2024-04-25T10:11:46.060873+0000","last_peered":"2024-04-25T10:11:46.060873+0000","last_clean":"2024-04-25T10:11:46.060873+0000","last_became_active":"2024-04-25T10:11:44.006878+0000","last_became_peered":"2024-04-25T10:11:44.006878+0000","last_unstale":"2024-04-25T10:11:46.060873+0000","last_undegraded":"2024-04-25T10:11:46.060873+0000","last_fullsized":"2024-04-25T10:11:46.060873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:52.154015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047580+0000","last_change":"2024-04-25T10:11:45.044021+0000","last_active":"2024-04-25T10:11:46.047580+0000","last_peered":"2024-04-25T10:11:46.047580+0000","last_clean":"2024-04-25T10:11:46.047580+0000","last_became_active":"2024-04-25T10:11:45.043292+0000","last_became_peered":"2024-04-25T10:11:45.043292+0000","last_unstale":"2024-04-25T10:11:46.047580+0000","last_undegraded":"2024-04-25T10:11:46.047580+0000","last_fullsized":"2024-04-25T10:11:46.047580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:16.016307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.061231+0000","last_change":"2024-04-25T10:11:44.012098+0000","last_active":"2024-04-25T10:11:46.061231+0000","last_peered":"2024-04-25T10:11:46.061231+0000","last_clean":"2024-04-25T10:11:46.061231+0000","last_became_active":"2024-04-25T10:11:44.011745+0000","last_became_peered":"2024-04-25T10:11:44.011745+0000","last_unstale":"2024-04-25T10:11:46.061231+0000","last_undegraded":"2024-04-25T10:11:46.061231+0000","last_fullsized":"2024-04-25T10:11:46.061231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:02.596488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.333913+0000","last_change":"2024-04-25T10:11:45.049708+0000","last_active":"2024-04-25T10:11:46.333913+0000","last_peered":"2024-04-25T10:11:46.333913+0000","last_clean":"2024-04-25T10:11:46.333913+0000","last_became_active":"2024-04-25T10:11:45.048821+0000","last_became_peered":"2024-04-25T10:11:45.048821+0000","last_unstale":"2024-04-25T10:11:46.333913+0000","last_undegraded":"2024-04-25T10:11:46.333913+0000","last_fullsized":"2024-04-25T10:11:46.333913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:14.885624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.332901+0000","last_change":"2024-04-25T10:11:44.001767+0000","last_active":"2024-04-25T10:11:46.332901+0000","last_peered":"2024-04-25T10:11:46.332901+0000","last_clean":"2024-04-25T10:11:46.332901+0000","last_became_active":"2024-04-25T10:11:44.001184+0000","last_became_peered":"2024-04-25T10:11:44.001184+0000","last_unstale":"2024-04-25T10:11:46.332901+0000","last_undegraded":"2024-04-25T10:11:46.332901+0000","last_fullsized":"2024-04-25T10:11:46.332901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:49.671623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046574+0000","last_change":"2024-04-25T10:11:45.041234+0000","last_active":"2024-04-25T10:11:46.046574+0000","last_peered":"2024-04-25T10:11:46.046574+0000","last_clean":"2024-04-25T10:11:46.046574+0000","last_became_active":"2024-04-25T10:11:45.039578+0000","last_became_peered":"2024-04-25T10:11:45.039578+0000","last_unstale":"2024-04-25T10:11:46.046574+0000","last_undegraded":"2024-04-25T10:11:46.046574+0000","last_fullsized":"2024-04-25T10:11:46.046574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:40.527351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334069+0000","last_change":"2024-04-25T10:11:43.999418+0000","last_active":"2024-04-25T10:11:46.334069+0000","last_peered":"2024-04-25T10:11:46.334069+0000","last_clean":"2024-04-25T10:11:46.334069+0000","last_became_active":"2024-04-25T10:11:43.998847+0000","last_became_peered":"2024-04-25T10:11:43.998847+0000","last_unstale":"2024-04-25T10:11:46.334069+0000","last_undegraded":"2024-04-25T10:11:46.334069+0000","last_fullsized":"2024-04-25T10:11:46.334069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:30.670903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.332364+0000","last_change":"2024-04-25T10:11:45.054251+0000","last_active":"2024-04-25T10:11:46.332364+0000","last_peered":"2024-04-25T10:11:46.332364+0000","last_clean":"2024-04-25T10:11:46.332364+0000","last_became_active":"2024-04-25T10:11:45.050664+0000","last_became_peered":"2024-04-25T10:11:45.050664+0000","last_unstale":"2024-04-25T10:11:46.332364+0000","last_undegraded":"2024-04-25T10:11:46.332364+0000","last_fullsized":"2024-04-25T10:11:46.332364+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:52.177015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.332417+0000","last_change":"2024-04-25T10:11:44.012408+0000","last_active":"2024-04-25T10:11:46.332417+0000","last_peered":"2024-04-25T10:11:46.332417+0000","last_clean":"2024-04-25T10:11:46.332417+0000","last_became_active":"2024-04-25T10:11:44.012121+0000","last_became_peered":"2024-04-25T10:11:44.012121+0000","last_unstale":"2024-04-25T10:11:46.332417+0000","last_undegraded":"2024-04-25T10:11:46.332417+0000","last_fullsized":"2024-04-25T10:11:46.332417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:13.420380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046063+0000","last_change":"2024-04-25T10:11:45.050580+0000","last_active":"2024-04-25T10:11:46.046063+0000","last_peered":"2024-04-25T10:11:46.046063+0000","last_clean":"2024-04-25T10:11:46.046063+0000","last_became_active":"2024-04-25T10:11:45.049858+0000","last_became_peered":"2024-04-25T10:11:45.049858+0000","last_unstale":"2024-04-25T10:11:46.046063+0000","last_undegraded":"2024-04-25T10:11:46.046063+0000","last_fullsized":"2024-04-25T10:11:46.046063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:03.603847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.331870+0000","last_change":"2024-04-25T10:11:44.002773+0000","last_active":"2024-04-25T10:11:46.331870+0000","last_peered":"2024-04-25T10:11:46.331870+0000","last_clean":"2024-04-25T10:11:46.331870+0000","last_became_active":"2024-04-25T10:11:44.002034+0000","last_became_peered":"2024-04-25T10:11:44.002034+0000","last_unstale":"2024-04-25T10:11:46.331870+0000","last_undegraded":"2024-04-25T10:11:46.331870+0000","last_fullsized":"2024-04-25T10:11:46.331870+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:45.490109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.048533+0000","last_change":"2024-04-25T10:11:45.027583+0000","last_active":"2024-04-25T10:11:46.048533+0000","last_peered":"2024-04-25T10:11:46.048533+0000","last_clean":"2024-04-25T10:11:46.048533+0000","last_became_active":"2024-04-25T10:11:45.026819+0000","last_became_peered":"2024-04-25T10:11:45.026819+0000","last_unstale":"2024-04-25T10:11:46.048533+0000","last_undegraded":"2024-04-25T10:11:46.048533+0000","last_fullsized":"2024-04-25T10:11:46.048533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:32.038199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.042670+0000","last_change":"2024-04-25T10:11:44.015314+0000","last_active":"2024-04-25T10:11:46.042670+0000","last_peered":"2024-04-25T10:11:46.042670+0000","last_clean":"2024-04-25T10:11:46.042670+0000","last_became_active":"2024-04-25T10:11:44.014485+0000","last_became_peered":"2024-04-25T10:11:44.014485+0000","last_unstale":"2024-04-25T10:11:46.042670+0000","last_undegraded":"2024-04-25T10:11:46.042670+0000","last_fullsized":"2024-04-25T10:11:46.042670+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:42.645985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207483+0000","last_change":"2024-04-25T10:11:45.060657+0000","last_active":"2024-04-25T10:11:46.207483+0000","last_peered":"2024-04-25T10:11:46.207483+0000","last_clean":"2024-04-25T10:11:46.207483+0000","last_became_active":"2024-04-25T10:11:45.058200+0000","last_became_peered":"2024-04-25T10:11:45.058200+0000","last_unstale":"2024-04-25T10:11:46.207483+0000","last_undegraded":"2024-04-25T10:11:46.207483+0000","last_fullsized":"2024-04-25T10:11:46.207483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:36.488666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.058571+0000","last_change":"2024-04-25T10:11:44.010083+0000","last_active":"2024-04-25T10:11:46.058571+0000","last_peered":"2024-04-25T10:11:46.058571+0000","last_clean":"2024-04-25T10:11:46.058571+0000","last_became_active":"2024-04-25T10:11:44.009518+0000","last_became_peered":"2024-04-25T10:11:44.009518+0000","last_unstale":"2024-04-25T10:11:46.058571+0000","last_undegraded":"2024-04-25T10:11:46.058571+0000","last_fullsized":"2024-04-25T10:11:46.058571+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:05.696828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335209+0000","last_change":"2024-04-25T10:11:45.067192+0000","last_active":"2024-04-25T10:11:46.335209+0000","last_peered":"2024-04-25T10:11:46.335209+0000","last_clean":"2024-04-25T10:11:46.335209+0000","last_became_active":"2024-04-25T10:11:45.064677+0000","last_became_peered":"2024-04-25T10:11:45.064677+0000","last_unstale":"2024-04-25T10:11:46.335209+0000","last_undegraded":"2024-04-25T10:11:46.335209+0000","last_fullsized":"2024-04-25T10:11:46.335209+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:27.227297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334719+0000","last_change":"2024-04-25T10:11:44.013908+0000","last_active":"2024-04-25T10:11:46.334719+0000","last_peered":"2024-04-25T10:11:46.334719+0000","last_clean":"2024-04-25T10:11:46.334719+0000","last_became_active":"2024-04-25T10:11:44.013090+0000","last_became_peered":"2024-04-25T10:11:44.013090+0000","last_unstale":"2024-04-25T10:11:46.334719+0000","last_undegraded":"2024-04-25T10:11:46.334719+0000","last_fullsized":"2024-04-25T10:11:46.334719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:03.261623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.048869+0000","last_change":"2024-04-25T10:11:45.042444+0000","last_active":"2024-04-25T10:11:46.048869+0000","last_peered":"2024-04-25T10:11:46.048869+0000","last_clean":"2024-04-25T10:11:46.048869+0000","last_became_active":"2024-04-25T10:11:45.040388+0000","last_became_peered":"2024-04-25T10:11:45.040388+0000","last_unstale":"2024-04-25T10:11:46.048869+0000","last_undegraded":"2024-04-25T10:11:46.048869+0000","last_fullsized":"2024-04-25T10:11:46.048869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:54.691413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335422+0000","last_change":"2024-04-25T10:11:44.023684+0000","last_active":"2024-04-25T10:11:46.335422+0000","last_peered":"2024-04-25T10:11:46.335422+0000","last_clean":"2024-04-25T10:11:46.335422+0000","last_became_active":"2024-04-25T10:11:44.021325+0000","last_became_peered":"2024-04-25T10:11:44.021325+0000","last_unstale":"2024-04-25T10:11:46.335422+0000","last_undegraded":"2024-04-25T10:11:46.335422+0000","last_fullsized":"2024-04-25T10:11:46.335422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:11.873020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046371+0000","last_change":"2024-04-25T10:11:45.026993+0000","last_active":"2024-04-25T10:11:46.046371+0000","last_peered":"2024-04-25T10:11:46.046371+0000","last_clean":"2024-04-25T10:11:46.046371+0000","last_became_active":"2024-04-25T10:11:45.026188+0000","last_became_peered":"2024-04-25T10:11:45.026188+0000","last_unstale":"2024-04-25T10:11:46.046371+0000","last_undegraded":"2024-04-25T10:11:46.046371+0000","last_fullsized":"2024-04-25T10:11:46.046371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:27.227297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.061328+0000","last_change":"2024-04-25T10:11:44.010253+0000","last_active":"2024-04-25T10:11:46.061328+0000","last_peered":"2024-04-25T10:11:46.061328+0000","last_clean":"2024-04-25T10:11:46.061328+0000","last_became_active":"2024-04-25T10:11:44.009937+0000","last_became_peered":"2024-04-25T10:11:44.009937+0000","last_unstale":"2024-04-25T10:11:46.061328+0000","last_undegraded":"2024-04-25T10:11:46.061328+0000","last_fullsized":"2024-04-25T10:11:46.061328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:09.359831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.061179+0000","last_change":"2024-04-25T10:11:45.054354+0000","last_active":"2024-04-25T10:11:46.061179+0000","last_peered":"2024-04-25T10:11:46.061179+0000","last_clean":"2024-04-25T10:11:46.061179+0000","last_became_active":"2024-04-25T10:11:45.054042+0000","last_became_peered":"2024-04-25T10:11:45.054042+0000","last_unstale":"2024-04-25T10:11:46.061179+0000","last_undegraded":"2024-04-25T10:11:46.061179+0000","last_fullsized":"2024-04-25T10:11:46.061179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:50.678982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046943+0000","last_change":"2024-04-25T10:11:44.016859+0000","last_active":"2024-04-25T10:11:46.046943+0000","last_peered":"2024-04-25T10:11:46.046943+0000","last_clean":"2024-04-25T10:11:46.046943+0000","last_became_active":"2024-04-25T10:11:44.015863+0000","last_became_peered":"2024-04-25T10:11:44.015863+0000","last_unstale":"2024-04-25T10:11:46.046943+0000","last_undegraded":"2024-04-25T10:11:46.046943+0000","last_fullsized":"2024-04-25T10:11:46.046943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:24.642231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207427+0000","last_change":"2024-04-25T10:11:45.056740+0000","last_active":"2024-04-25T10:11:46.207427+0000","last_peered":"2024-04-25T10:11:46.207427+0000","last_clean":"2024-04-25T10:11:46.207427+0000","last_became_active":"2024-04-25T10:11:45.050875+0000","last_became_peered":"2024-04-25T10:11:45.050875+0000","last_unstale":"2024-04-25T10:11:46.207427+0000","last_undegraded":"2024-04-25T10:11:46.207427+0000","last_fullsized":"2024-04-25T10:11:46.207427+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:46.087055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334743+0000","last_change":"2024-04-25T10:11:44.023581+0000","last_active":"2024-04-25T10:11:46.334743+0000","last_peered":"2024-04-25T10:11:46.334743+0000","last_clean":"2024-04-25T10:11:46.334743+0000","last_became_active":"2024-04-25T10:11:44.022751+0000","last_became_peered":"2024-04-25T10:11:44.022751+0000","last_unstale":"2024-04-25T10:11:46.334743+0000","last_undegraded":"2024-04-25T10:11:46.334743+0000","last_fullsized":"2024-04-25T10:11:46.334743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:26.853442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334474+0000","last_change":"2024-04-25T10:11:45.065388+0000","last_active":"2024-04-25T10:11:46.334474+0000","last_peered":"2024-04-25T10:11:46.334474+0000","last_clean":"2024-04-25T10:11:46.334474+0000","last_became_active":"2024-04-25T10:11:45.063569+0000","last_became_peered":"2024-04-25T10:11:45.063569+0000","last_unstale":"2024-04-25T10:11:46.334474+0000","last_undegraded":"2024-04-25T10:11:46.334474+0000","last_fullsized":"2024-04-25T10:11:46.334474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:25.519578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046415+0000","last_change":"2024-04-25T10:11:44.030136+0000","last_active":"2024-04-25T10:11:46.046415+0000","last_peered":"2024-04-25T10:11:46.046415+0000","last_clean":"2024-04-25T10:11:46.046415+0000","last_became_active":"2024-04-25T10:11:44.029433+0000","last_became_peered":"2024-04-25T10:11:44.029433+0000","last_unstale":"2024-04-25T10:11:46.046415+0000","last_undegraded":"2024-04-25T10:11:46.046415+0000","last_fullsized":"2024-04-25T10:11:46.046415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:40.693610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.042450+0000","last_change":"2024-04-25T10:11:45.025591+0000","last_active":"2024-04-25T10:11:46.042450+0000","last_peered":"2024-04-25T10:11:46.042450+0000","last_clean":"2024-04-25T10:11:46.042450+0000","last_became_active":"2024-04-25T10:11:45.024672+0000","last_became_peered":"2024-04-25T10:11:45.024672+0000","last_unstale":"2024-04-25T10:11:46.042450+0000","last_undegraded":"2024-04-25T10:11:46.042450+0000","last_fullsized":"2024-04-25T10:11:46.042450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:01.799712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207395+0000","last_change":"2024-04-25T10:11:44.022906+0000","last_active":"2024-04-25T10:11:46.207395+0000","last_peered":"2024-04-25T10:11:46.207395+0000","last_clean":"2024-04-25T10:11:46.207395+0000","last_became_active":"2024-04-25T10:11:44.022169+0000","last_became_peered":"2024-04-25T10:11:44.022169+0000","last_unstale":"2024-04-25T10:11:46.207395+0000","last_undegraded":"2024-04-25T10:11:46.207395+0000","last_fullsized":"2024-04-25T10:11:46.207395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:05.432447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":281,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.206822+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:11:46.206822+0000","last_peered":"2024-04-25T10:11:46.206822+0000","last_clean":"2024-04-25T10:11:46.206822+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:11:46.206822+0000","last_undegraded":"2024-04-25T10:11:46.206822+0000","last_fullsized":"2024-04-25T10:11:46.206822+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509524,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1597440,"data_stored":1015475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509523,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":425037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542226,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1011712,"data_stored":425731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542227,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1015808,"data_stored":426671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542227,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1011712,"data_stored":426649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542227,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36320,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335520,"statfs":{"total":96636764160,"available":96599572480,"internally_reserved":0,"allocated":1003520,"data_stored":425619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542227,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36336,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335504,"statfs":{"total":96636764160,"available":96599556096,"internally_reserved":0,"allocated":1011712,"data_stored":425231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542228,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36908,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334932,"statfs":{"total":96636764160,"available":96598970368,"internally_reserved":0,"allocated":1597440,"data_stored":1015439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:49.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:49.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6dd749640 1 Processor -- start 2024-04-25T10:11:49.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6dd749640 1 -- start start 2024-04-25T10:11:49.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6dd749640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe6d80ce7a0 0x7fe6d80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6dd749640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6d815bc90 0x7fe6d80b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6dd749640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d80cf0c0 con 0x7fe6d80b4660 2024-04-25T10:11:49.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6dd749640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6d80cc370 con 0x7fe6d80ce7a0 2024-04-25T10:11:49.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6dd749640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d8071ca0 con 0x7fe6d815bc90 2024-04-25T10:11:49.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6cffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6d815bc90 0x7fe6d80b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6d6ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe6d80ce7a0 0x7fe6d80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6d6ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe6d80ce7a0 0x7fe6d80ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58008/0 (socket says 172.21.15.60:58008) 2024-04-25T10:11:49.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6d77fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe6d80b4660 0x7fe6d80b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:59652/0 (socket says 172.21.15.60:59652) 2024-04-25T10:11:49.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6d77fe640 1 -- 172.21.15.60:0/2842664526 learned_addr learned my addr 172.21.15.60:0/2842664526 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:49.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6cffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6d815bc90 0x7fe6d80b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:51580/0 (socket says 172.21.15.60:51580) 2024-04-25T10:11:49.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6cffff640 1 -- 172.21.15.60:0/2842664526 >> v2:172.21.15.100:3300/0 conn(0x7fe6d80ce7a0 msgr2=0x7fe6d80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:49.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6cffff640 1 --2- 172.21.15.60:0/2842664526 >> v2:172.21.15.100:3300/0 conn(0x7fe6d80ce7a0 0x7fe6d80ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6cffff640 1 -- 172.21.15.60:0/2842664526 >> v1:172.21.15.100:6789/0 conn(0x7fe6d80b4660 legacy=0x7fe6d80b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6cffff640 1 -- 172.21.15.60:0/2842664526 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d80cca40 con 0x7fe6d815bc90 2024-04-25T10:11:49.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.268+0000 7fe6d6ffd640 1 --2- 172.21.15.60:0/2842664526 >> v2:172.21.15.100:3300/0 conn(0x7fe6d80ce7a0 0x7fe6d80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:49.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.272+0000 7fe6cffff640 1 --2- 172.21.15.60:0/2842664526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6d815bc90 0x7fe6d80b25d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00097f0 tx=0x7fe6c001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72230fe9ef4914f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:49.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.272+0000 7fe6d67fc640 1 -- 172.21.15.60:0/2842664526 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c00b0070 con 0x7fe6d815bc90 2024-04-25T10:11:49.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.272+0000 7fe6d67fc640 1 -- 172.21.15.60:0/2842664526 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c00ab600 con 0x7fe6d815bc90 2024-04-25T10:11:49.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.272+0000 7fe6d67fc640 1 -- 172.21.15.60:0/2842664526 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c00ab9d0 con 0x7fe6d815bc90 2024-04-25T10:11:49.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.272+0000 7fe6dd749640 1 -- 172.21.15.60:0/2842664526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6d815bc90 msgr2=0x7fe6d80b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.272+0000 7fe6dd749640 1 --2- 172.21.15.60:0/2842664526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6d815bc90 0x7fe6d80b25d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00097f0 tx=0x7fe6c001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.272+0000 7fe6dd749640 1 -- 172.21.15.60:0/2842664526 shutdown_connections 2024-04-25T10:11:49.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.272+0000 7fe6dd749640 1 --2- 172.21.15.60:0/2842664526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6d815bc90 0x7fe6d80b25d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.272+0000 7fe6dd749640 1 --2- 172.21.15.60:0/2842664526 >> v2:172.21.15.100:3300/0 conn(0x7fe6d80ce7a0 0x7fe6d80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.272+0000 7fe6dd749640 1 -- 172.21.15.60:0/2842664526 >> 172.21.15.60:0/2842664526 conn(0x7fe6d80b12d0 msgr2=0x7fe6d80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:49.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.272+0000 7fe6dd749640 1 -- 172.21.15.60:0/2842664526 shutdown_connections 2024-04-25T10:11:49.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.272+0000 7fe6dd749640 1 -- 172.21.15.60:0/2842664526 wait complete. 2024-04-25T10:11:49.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6dd749640 1 Processor -- start 2024-04-25T10:11:49.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6dd749640 1 -- start start 2024-04-25T10:11:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6dd749640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d80ce7a0 0x7fe6d8156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6dd749640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6d815bc90 0x7fe6d8141a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6dd749640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d8141fb0 0x7fe6d81530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6dd749640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d80b4280 con 0x7fe6d80ce7a0 2024-04-25T10:11:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6dd749640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d80b3db0 con 0x7fe6d8141fb0 2024-04-25T10:11:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6dd749640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d80b4400 con 0x7fe6d815bc90 2024-04-25T10:11:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d80ce7a0 0x7fe6d8156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d80ce7a0 0x7fe6d8156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46820/0 (socket says 172.21.15.60:46820) 2024-04-25T10:11:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6d6ffd640 1 -- 172.21.15.60:0/1083551930 learned_addr learned my addr 172.21.15.60:0/1083551930 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6cffff640 1 --2- 172.21.15.60:0/1083551930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6d815bc90 0x7fe6d8141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6d77fe640 1 --2- 172.21.15.60:0/1083551930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d8141fb0 0x7fe6d81530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6d6ffd640 1 -- 172.21.15.60:0/1083551930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6d815bc90 msgr2=0x7fe6d8141a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6d6ffd640 1 --2- 172.21.15.60:0/1083551930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6d815bc90 0x7fe6d8141a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6d6ffd640 1 -- 172.21.15.60:0/1083551930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d8141fb0 msgr2=0x7fe6d81530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6d6ffd640 1 --2- 172.21.15.60:0/1083551930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d8141fb0 0x7fe6d81530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6d6ffd640 1 -- 172.21.15.60:0/1083551930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d8153e50 con 0x7fe6d80ce7a0 2024-04-25T10:11:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6d6ffd640 1 --2- 172.21.15.60:0/1083551930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d80ce7a0 0x7fe6d8156a80 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e980 tx=0x7fe6c800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6cf7fe640 1 -- 172.21.15.60:0/1083551930 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8017070 con 0x7fe6d80ce7a0 2024-04-25T10:11:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6cf7fe640 1 -- 172.21.15.60:0/1083551930 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c800ca00 con 0x7fe6d80ce7a0 2024-04-25T10:11:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6cf7fe640 1 -- 172.21.15.60:0/1083551930 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c800cdd0 con 0x7fe6d80ce7a0 2024-04-25T10:11:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6dd749640 1 -- 172.21.15.60:0/1083551930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d8154130 con 0x7fe6d80ce7a0 2024-04-25T10:11:49.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6dd749640 1 -- 172.21.15.60:0/1083551930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d81546f0 con 0x7fe6d80ce7a0 2024-04-25T10:11:49.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.276+0000 7fe6dd749640 1 -- 172.21.15.60:0/1083551930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d8074770 con 0x7fe6d80ce7a0 2024-04-25T10:11:49.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.280+0000 7fe6cf7fe640 1 -- 172.21.15.60:0/1083551930 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6c801a070 con 0x7fe6d80ce7a0 2024-04-25T10:11:49.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.280+0000 7fe6cf7fe640 1 --2- 172.21.15.60:0/1083551930 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe6b8066f60 0x7fe6b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.280+0000 7fe6cf7fe640 1 -- 172.21.15.60:0/1083551930 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6c80851e0 con 0x7fe6d80ce7a0 2024-04-25T10:11:49.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.280+0000 7fe6cffff640 1 --2- 172.21.15.60:0/1083551930 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe6b8066f60 0x7fe6b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.280+0000 7fe6cffff640 1 --2- 172.21.15.60:0/1083551930 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe6b8066f60 0x7fe6b8069420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00ad000 tx=0x7fe6c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:49.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.284+0000 7fe6cf7fe640 1 -- 172.21.15.60:0/1083551930 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6c8055b90 con 0x7fe6d80ce7a0 2024-04-25T10:11:49.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.440+0000 7fe6dd749640 1 -- 172.21.15.60:0/1083551930 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6d80b2770 con 0x7fe6b8066f60 2024-04-25T10:11:49.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.456+0000 7fe6cf7fe640 1 -- 172.21.15.60:0/1083551930 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252078 (secure 0 0 0) 0x7fe6d80b2770 con 0x7fe6b8066f60 2024-04-25T10:11:49.466 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:49.469 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:11:49.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.460+0000 7fe6dd749640 1 -- 172.21.15.60:0/1083551930 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe6b8066f60 msgr2=0x7fe6b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.460+0000 7fe6dd749640 1 --2- 172.21.15.60:0/1083551930 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe6b8066f60 0x7fe6b8069420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00ad000 tx=0x7fe6c0008040 comp rx=0 tx=0).stop 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.460+0000 7fe6dd749640 1 -- 172.21.15.60:0/1083551930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d80ce7a0 msgr2=0x7fe6d8156a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.460+0000 7fe6dd749640 1 --2- 172.21.15.60:0/1083551930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d80ce7a0 0x7fe6d8156a80 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e980 tx=0x7fe6c800ee40 comp rx=0 tx=0).stop 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.460+0000 7fe6dd749640 1 -- 172.21.15.60:0/1083551930 shutdown_connections 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.460+0000 7fe6dd749640 1 --2- 172.21.15.60:0/1083551930 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe6b8066f60 0x7fe6b8069420 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.460+0000 7fe6dd749640 1 --2- 172.21.15.60:0/1083551930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6d8141fb0 0x7fe6d81530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.460+0000 7fe6dd749640 1 --2- 172.21.15.60:0/1083551930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6d815bc90 0x7fe6d8141a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.460+0000 7fe6dd749640 1 --2- 172.21.15.60:0/1083551930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6d80ce7a0 0x7fe6d8156a80 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.460+0000 7fe6dd749640 1 -- 172.21.15.60:0/1083551930 >> 172.21.15.60:0/1083551930 conn(0x7fe6d80b12d0 msgr2=0x7fe6d80b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.460+0000 7fe6dd749640 1 -- 172.21.15.60:0/1083551930 shutdown_connections 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.460+0000 7fe6dd749640 1 -- 172.21.15.60:0/1083551930 wait complete. 2024-04-25T10:11:49.485 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-04-25T10:11:48.522575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266824,"kb_used_data":9032,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754707896,"statfs":{"total":773094113280,"available":772820885504,"internally_reserved":0,"allocated":9248768,"data_stored":4585852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.545989"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335072+0000","last_change":"2024-04-25T10:11:45.063671+0000","last_active":"2024-04-25T10:11:46.335072+0000","last_peered":"2024-04-25T10:11:46.335072+0000","last_clean":"2024-04-25T10:11:46.335072+0000","last_became_active":"2024-04-25T10:11:45.059505+0000","last_became_peered":"2024-04-25T10:11:45.059505+0000","last_unstale":"2024-04-25T10:11:46.335072+0000","last_undegraded":"2024-04-25T10:11:46.335072+0000","last_fullsized":"2024-04-25T10:11:46.335072+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:53.161374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.042635+0000","last_change":"2024-04-25T10:11:45.061143+0000","last_active":"2024-04-25T10:11:46.042635+0000","last_peered":"2024-04-25T10:11:46.042635+0000","last_clean":"2024-04-25T10:11:46.042635+0000","last_became_active":"2024-04-25T10:11:45.060863+0000","last_became_peered":"2024-04-25T10:11:45.060863+0000","last_unstale":"2024-04-25T10:11:46.042635+0000","last_undegraded":"2024-04-25T10:11:46.042635+0000","last_fullsized":"2024-04-25T10:11:46.042635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:17.431486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207253+0000","last_change":"2024-04-25T10:11:45.077970+0000","last_active":"2024-04-25T10:11:46.207253+0000","last_peered":"2024-04-25T10:11:46.207253+0000","last_clean":"2024-04-25T10:11:46.207253+0000","last_became_active":"2024-04-25T10:11:45.077282+0000","last_became_peered":"2024-04-25T10:11:45.077282+0000","last_unstale":"2024-04-25T10:11:46.207253+0000","last_undegraded":"2024-04-25T10:11:46.207253+0000","last_fullsized":"2024-04-25T10:11:46.207253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:30.887901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334292+0000","last_change":"2024-04-25T10:11:45.049379+0000","last_active":"2024-04-25T10:11:46.334292+0000","last_peered":"2024-04-25T10:11:46.334292+0000","last_clean":"2024-04-25T10:11:46.334292+0000","last_became_active":"2024-04-25T10:11:45.048301+0000","last_became_peered":"2024-04-25T10:11:45.048301+0000","last_unstale":"2024-04-25T10:11:46.334292+0000","last_undegraded":"2024-04-25T10:11:46.334292+0000","last_fullsized":"2024-04-25T10:11:46.334292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:07.189560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047076+0000","last_change":"2024-04-25T10:11:45.045185+0000","last_active":"2024-04-25T10:11:46.047076+0000","last_peered":"2024-04-25T10:11:46.047076+0000","last_clean":"2024-04-25T10:11:46.047076+0000","last_became_active":"2024-04-25T10:11:45.044393+0000","last_became_peered":"2024-04-25T10:11:45.044393+0000","last_unstale":"2024-04-25T10:11:46.047076+0000","last_undegraded":"2024-04-25T10:11:46.047076+0000","last_fullsized":"2024-04-25T10:11:46.047076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:36.716191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046159+0000","last_change":"2024-04-25T10:11:45.061227+0000","last_active":"2024-04-25T10:11:46.046159+0000","last_peered":"2024-04-25T10:11:46.046159+0000","last_clean":"2024-04-25T10:11:46.046159+0000","last_became_active":"2024-04-25T10:11:45.060477+0000","last_became_peered":"2024-04-25T10:11:45.060477+0000","last_unstale":"2024-04-25T10:11:46.046159+0000","last_undegraded":"2024-04-25T10:11:46.046159+0000","last_fullsized":"2024-04-25T10:11:46.046159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:10.367189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.333393+0000","last_change":"2024-04-25T10:11:45.050209+0000","last_active":"2024-04-25T10:11:46.333393+0000","last_peered":"2024-04-25T10:11:46.333393+0000","last_clean":"2024-04-25T10:11:46.333393+0000","last_became_active":"2024-04-25T10:11:45.049856+0000","last_became_peered":"2024-04-25T10:11:45.049856+0000","last_unstale":"2024-04-25T10:11:46.333393+0000","last_undegraded":"2024-04-25T10:11:46.333393+0000","last_fullsized":"2024-04-25T10:11:46.333393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:57.205394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046214+0000","last_change":"2024-04-25T10:11:45.038897+0000","last_active":"2024-04-25T10:11:46.046214+0000","last_peered":"2024-04-25T10:11:46.046214+0000","last_clean":"2024-04-25T10:11:46.046214+0000","last_became_active":"2024-04-25T10:11:45.037970+0000","last_became_peered":"2024-04-25T10:11:45.037970+0000","last_unstale":"2024-04-25T10:11:46.046214+0000","last_undegraded":"2024-04-25T10:11:46.046214+0000","last_fullsized":"2024-04-25T10:11:46.046214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:53.161374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.048076+0000","last_change":"2024-04-25T10:11:45.039946+0000","last_active":"2024-04-25T10:11:46.048076+0000","last_peered":"2024-04-25T10:11:46.048076+0000","last_clean":"2024-04-25T10:11:46.048076+0000","last_became_active":"2024-04-25T10:11:45.039000+0000","last_became_peered":"2024-04-25T10:11:45.039000+0000","last_unstale":"2024-04-25T10:11:46.048076+0000","last_undegraded":"2024-04-25T10:11:46.048076+0000","last_fullsized":"2024-04-25T10:11:46.048076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:31.443603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335344+0000","last_change":"2024-04-25T10:11:45.058201+0000","last_active":"2024-04-25T10:11:46.335344+0000","last_peered":"2024-04-25T10:11:46.335344+0000","last_clean":"2024-04-25T10:11:46.335344+0000","last_became_active":"2024-04-25T10:11:45.050910+0000","last_became_peered":"2024-04-25T10:11:45.050910+0000","last_unstale":"2024-04-25T10:11:46.335344+0000","last_undegraded":"2024-04-25T10:11:46.335344+0000","last_fullsized":"2024-04-25T10:11:46.335344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:31.443603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.042321+0000","last_change":"2024-04-25T10:11:45.039883+0000","last_active":"2024-04-25T10:11:46.042321+0000","last_peered":"2024-04-25T10:11:46.042321+0000","last_clean":"2024-04-25T10:11:46.042321+0000","last_became_active":"2024-04-25T10:11:45.039047+0000","last_became_peered":"2024-04-25T10:11:45.039047+0000","last_unstale":"2024-04-25T10:11:46.042321+0000","last_undegraded":"2024-04-25T10:11:46.042321+0000","last_fullsized":"2024-04-25T10:11:46.042321+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:57.829518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334546+0000","last_change":"2024-04-25T10:11:45.060984+0000","last_active":"2024-04-25T10:11:46.334546+0000","last_peered":"2024-04-25T10:11:46.334546+0000","last_clean":"2024-04-25T10:11:46.334546+0000","last_became_active":"2024-04-25T10:11:45.059253+0000","last_became_peered":"2024-04-25T10:11:45.059253+0000","last_unstale":"2024-04-25T10:11:46.334546+0000","last_undegraded":"2024-04-25T10:11:46.334546+0000","last_fullsized":"2024-04-25T10:11:46.334546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:41.651037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207206+0000","last_change":"2024-04-25T10:11:45.051484+0000","last_active":"2024-04-25T10:11:46.207206+0000","last_peered":"2024-04-25T10:11:46.207206+0000","last_clean":"2024-04-25T10:11:46.207206+0000","last_became_active":"2024-04-25T10:11:45.049730+0000","last_became_peered":"2024-04-25T10:11:45.049730+0000","last_unstale":"2024-04-25T10:11:46.207206+0000","last_undegraded":"2024-04-25T10:11:46.207206+0000","last_fullsized":"2024-04-25T10:11:46.207206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:27.884490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.333825+0000","last_change":"2024-04-25T10:11:45.049988+0000","last_active":"2024-04-25T10:11:46.333825+0000","last_peered":"2024-04-25T10:11:46.333825+0000","last_clean":"2024-04-25T10:11:46.333825+0000","last_became_active":"2024-04-25T10:11:45.049300+0000","last_became_peered":"2024-04-25T10:11:45.049300+0000","last_unstale":"2024-04-25T10:11:46.333825+0000","last_undegraded":"2024-04-25T10:11:46.333825+0000","last_fullsized":"2024-04-25T10:11:46.333825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:35.241173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.332110+0000","last_change":"2024-04-25T10:11:45.041503+0000","last_active":"2024-04-25T10:11:46.332110+0000","last_peered":"2024-04-25T10:11:46.332110+0000","last_clean":"2024-04-25T10:11:46.332110+0000","last_became_active":"2024-04-25T10:11:45.039646+0000","last_became_peered":"2024-04-25T10:11:45.039646+0000","last_unstale":"2024-04-25T10:11:46.332110+0000","last_undegraded":"2024-04-25T10:11:46.332110+0000","last_fullsized":"2024-04-25T10:11:46.332110+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:01.752158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047480+0000","last_change":"2024-04-25T10:11:45.035192+0000","last_active":"2024-04-25T10:11:46.047480+0000","last_peered":"2024-04-25T10:11:46.047480+0000","last_clean":"2024-04-25T10:11:46.047480+0000","last_became_active":"2024-04-25T10:11:45.034440+0000","last_became_peered":"2024-04-25T10:11:45.034440+0000","last_unstale":"2024-04-25T10:11:46.047480+0000","last_undegraded":"2024-04-25T10:11:46.047480+0000","last_fullsized":"2024-04-25T10:11:46.047480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:48.597040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047207+0000","last_change":"2024-04-25T10:11:45.060439+0000","last_active":"2024-04-25T10:11:46.047207+0000","last_peered":"2024-04-25T10:11:46.047207+0000","last_clean":"2024-04-25T10:11:46.047207+0000","last_became_active":"2024-04-25T10:11:45.059732+0000","last_became_peered":"2024-04-25T10:11:45.059732+0000","last_unstale":"2024-04-25T10:11:46.047207+0000","last_undegraded":"2024-04-25T10:11:46.047207+0000","last_fullsized":"2024-04-25T10:11:46.047207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:33.434871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.332995+0000","last_change":"2024-04-25T10:11:45.048635+0000","last_active":"2024-04-25T10:11:46.332995+0000","last_peered":"2024-04-25T10:11:46.332995+0000","last_clean":"2024-04-25T10:11:46.332995+0000","last_became_active":"2024-04-25T10:11:45.045910+0000","last_became_peered":"2024-04-25T10:11:45.045910+0000","last_unstale":"2024-04-25T10:11:46.332995+0000","last_undegraded":"2024-04-25T10:11:46.332995+0000","last_fullsized":"2024-04-25T10:11:46.332995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:11.618132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.060732+0000","last_change":"2024-04-25T10:11:45.046575+0000","last_active":"2024-04-25T10:11:46.060732+0000","last_peered":"2024-04-25T10:11:46.060732+0000","last_clean":"2024-04-25T10:11:46.060732+0000","last_became_active":"2024-04-25T10:11:45.045722+0000","last_became_peered":"2024-04-25T10:11:45.045722+0000","last_unstale":"2024-04-25T10:11:46.060732+0000","last_undegraded":"2024-04-25T10:11:46.060732+0000","last_fullsized":"2024-04-25T10:11:46.060732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:46.497467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334222+0000","last_change":"2024-04-25T10:11:45.052267+0000","last_active":"2024-04-25T10:11:46.334222+0000","last_peered":"2024-04-25T10:11:46.334222+0000","last_clean":"2024-04-25T10:11:46.334222+0000","last_became_active":"2024-04-25T10:11:45.050259+0000","last_became_peered":"2024-04-25T10:11:45.050259+0000","last_unstale":"2024-04-25T10:11:46.334222+0000","last_undegraded":"2024-04-25T10:11:46.334222+0000","last_fullsized":"2024-04-25T10:11:46.334222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:37.283701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334085+0000","last_change":"2024-04-25T10:11:45.049528+0000","last_active":"2024-04-25T10:11:46.334085+0000","last_peered":"2024-04-25T10:11:46.334085+0000","last_clean":"2024-04-25T10:11:46.334085+0000","last_became_active":"2024-04-25T10:11:45.048998+0000","last_became_peered":"2024-04-25T10:11:45.048998+0000","last_unstale":"2024-04-25T10:11:46.334085+0000","last_undegraded":"2024-04-25T10:11:46.334085+0000","last_fullsized":"2024-04-25T10:11:46.334085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:42.843817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.043229+0000","last_change":"2024-04-25T10:11:45.042201+0000","last_active":"2024-04-25T10:11:46.043229+0000","last_peered":"2024-04-25T10:11:46.043229+0000","last_clean":"2024-04-25T10:11:46.043229+0000","last_became_active":"2024-04-25T10:11:45.040941+0000","last_became_peered":"2024-04-25T10:11:45.040941+0000","last_unstale":"2024-04-25T10:11:46.043229+0000","last_undegraded":"2024-04-25T10:11:46.043229+0000","last_fullsized":"2024-04-25T10:11:46.043229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:59.172072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334269+0000","last_change":"2024-04-25T10:11:45.062858+0000","last_active":"2024-04-25T10:11:46.334269+0000","last_peered":"2024-04-25T10:11:46.334269+0000","last_clean":"2024-04-25T10:11:46.334269+0000","last_became_active":"2024-04-25T10:11:45.061828+0000","last_became_peered":"2024-04-25T10:11:45.061828+0000","last_unstale":"2024-04-25T10:11:46.334269+0000","last_undegraded":"2024-04-25T10:11:46.334269+0000","last_fullsized":"2024-04-25T10:11:46.334269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:38.898303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.333773+0000","last_change":"2024-04-25T10:11:45.054556+0000","last_active":"2024-04-25T10:11:46.333773+0000","last_peered":"2024-04-25T10:11:46.333773+0000","last_clean":"2024-04-25T10:11:46.333773+0000","last_became_active":"2024-04-25T10:11:45.053843+0000","last_became_peered":"2024-04-25T10:11:45.053843+0000","last_unstale":"2024-04-25T10:11:46.333773+0000","last_undegraded":"2024-04-25T10:11:46.333773+0000","last_fullsized":"2024-04-25T10:11:46.333773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:13.636120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.042430+0000","last_change":"2024-04-25T10:11:45.041749+0000","last_active":"2024-04-25T10:11:46.042430+0000","last_peered":"2024-04-25T10:11:46.042430+0000","last_clean":"2024-04-25T10:11:46.042430+0000","last_became_active":"2024-04-25T10:11:45.040929+0000","last_became_peered":"2024-04-25T10:11:45.040929+0000","last_unstale":"2024-04-25T10:11:46.042430+0000","last_undegraded":"2024-04-25T10:11:46.042430+0000","last_fullsized":"2024-04-25T10:11:46.042430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:12.437425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207197+0000","last_change":"2024-04-25T10:11:45.057490+0000","last_active":"2024-04-25T10:11:46.207197+0000","last_peered":"2024-04-25T10:11:46.207197+0000","last_clean":"2024-04-25T10:11:46.207197+0000","last_became_active":"2024-04-25T10:11:45.056404+0000","last_became_peered":"2024-04-25T10:11:45.056404+0000","last_unstale":"2024-04-25T10:11:46.207197+0000","last_undegraded":"2024-04-25T10:11:46.207197+0000","last_fullsized":"2024-04-25T10:11:46.207197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:08.880560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047047+0000","last_change":"2024-04-25T10:11:45.032175+0000","last_active":"2024-04-25T10:11:46.047047+0000","last_peered":"2024-04-25T10:11:46.047047+0000","last_clean":"2024-04-25T10:11:46.047047+0000","last_became_active":"2024-04-25T10:11:45.031392+0000","last_became_peered":"2024-04-25T10:11:45.031392+0000","last_unstale":"2024-04-25T10:11:46.047047+0000","last_undegraded":"2024-04-25T10:11:46.047047+0000","last_fullsized":"2024-04-25T10:11:46.047047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:25.865771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334706+0000","last_change":"2024-04-25T10:11:45.058024+0000","last_active":"2024-04-25T10:11:46.334706+0000","last_peered":"2024-04-25T10:11:46.334706+0000","last_clean":"2024-04-25T10:11:46.334706+0000","last_became_active":"2024-04-25T10:11:45.057406+0000","last_became_peered":"2024-04-25T10:11:45.057406+0000","last_unstale":"2024-04-25T10:11:46.334706+0000","last_undegraded":"2024-04-25T10:11:46.334706+0000","last_fullsized":"2024-04-25T10:11:46.334706+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:03.603847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.042272+0000","last_change":"2024-04-25T10:11:45.041831+0000","last_active":"2024-04-25T10:11:46.042272+0000","last_peered":"2024-04-25T10:11:46.042272+0000","last_clean":"2024-04-25T10:11:46.042272+0000","last_became_active":"2024-04-25T10:11:45.040626+0000","last_became_peered":"2024-04-25T10:11:45.040626+0000","last_unstale":"2024-04-25T10:11:46.042272+0000","last_undegraded":"2024-04-25T10:11:46.042272+0000","last_fullsized":"2024-04-25T10:11:46.042272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:25.649590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.042965+0000","last_change":"2024-04-25T10:11:45.057574+0000","last_active":"2024-04-25T10:11:46.042965+0000","last_peered":"2024-04-25T10:11:46.042965+0000","last_clean":"2024-04-25T10:11:46.042965+0000","last_became_active":"2024-04-25T10:11:45.057208+0000","last_became_peered":"2024-04-25T10:11:45.057208+0000","last_unstale":"2024-04-25T10:11:46.042965+0000","last_undegraded":"2024-04-25T10:11:46.042965+0000","last_fullsized":"2024-04-25T10:11:46.042965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:58.307282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.048082+0000","last_change":"2024-04-25T10:11:45.054418+0000","last_active":"2024-04-25T10:11:46.048082+0000","last_peered":"2024-04-25T10:11:46.048082+0000","last_clean":"2024-04-25T10:11:46.048082+0000","last_became_active":"2024-04-25T10:11:45.053638+0000","last_became_peered":"2024-04-25T10:11:45.053638+0000","last_unstale":"2024-04-25T10:11:46.048082+0000","last_undegraded":"2024-04-25T10:11:46.048082+0000","last_fullsized":"2024-04-25T10:11:46.048082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:21.579326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334075+0000","last_change":"2024-04-25T10:11:45.042287+0000","last_active":"2024-04-25T10:11:46.334075+0000","last_peered":"2024-04-25T10:11:46.334075+0000","last_clean":"2024-04-25T10:11:46.334075+0000","last_became_active":"2024-04-25T10:11:45.041446+0000","last_became_peered":"2024-04-25T10:11:45.041446+0000","last_unstale":"2024-04-25T10:11:46.334075+0000","last_undegraded":"2024-04-25T10:11:46.334075+0000","last_fullsized":"2024-04-25T10:11:46.334075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:17.846024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047988+0000","last_change":"2024-04-25T10:11:45.047629+0000","last_active":"2024-04-25T10:11:46.047988+0000","last_peered":"2024-04-25T10:11:46.047988+0000","last_clean":"2024-04-25T10:11:46.047988+0000","last_became_active":"2024-04-25T10:11:45.046855+0000","last_became_peered":"2024-04-25T10:11:45.046855+0000","last_unstale":"2024-04-25T10:11:46.047988+0000","last_undegraded":"2024-04-25T10:11:46.047988+0000","last_fullsized":"2024-04-25T10:11:46.047988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:34.255770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335539+0000","last_change":"2024-04-25T10:11:44.015887+0000","last_active":"2024-04-25T10:11:46.335539+0000","last_peered":"2024-04-25T10:11:46.335539+0000","last_clean":"2024-04-25T10:11:46.335539+0000","last_became_active":"2024-04-25T10:11:44.015192+0000","last_became_peered":"2024-04-25T10:11:44.015192+0000","last_unstale":"2024-04-25T10:11:46.335539+0000","last_undegraded":"2024-04-25T10:11:46.335539+0000","last_fullsized":"2024-04-25T10:11:46.335539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:58.640565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335244+0000","last_change":"2024-04-25T10:11:45.053014+0000","last_active":"2024-04-25T10:11:46.335244+0000","last_peered":"2024-04-25T10:11:46.335244+0000","last_clean":"2024-04-25T10:11:46.335244+0000","last_became_active":"2024-04-25T10:11:45.052232+0000","last_became_peered":"2024-04-25T10:11:45.052232+0000","last_unstale":"2024-04-25T10:11:46.335244+0000","last_undegraded":"2024-04-25T10:11:46.335244+0000","last_fullsized":"2024-04-25T10:11:46.335244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:40.527351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.048127+0000","last_change":"2024-04-25T10:11:44.016115+0000","last_active":"2024-04-25T10:11:46.048127+0000","last_peered":"2024-04-25T10:11:46.048127+0000","last_clean":"2024-04-25T10:11:46.048127+0000","last_became_active":"2024-04-25T10:11:44.015408+0000","last_became_peered":"2024-04-25T10:11:44.015408+0000","last_unstale":"2024-04-25T10:11:46.048127+0000","last_undegraded":"2024-04-25T10:11:46.048127+0000","last_fullsized":"2024-04-25T10:11:46.048127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:11.430066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335067+0000","last_change":"2024-04-25T10:11:45.056604+0000","last_active":"2024-04-25T10:11:46.335067+0000","last_peered":"2024-04-25T10:11:46.335067+0000","last_clean":"2024-04-25T10:11:46.335067+0000","last_became_active":"2024-04-25T10:11:45.055735+0000","last_became_peered":"2024-04-25T10:11:45.055735+0000","last_unstale":"2024-04-25T10:11:46.335067+0000","last_undegraded":"2024-04-25T10:11:46.335067+0000","last_fullsized":"2024-04-25T10:11:46.335067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:10.367189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.060915+0000","last_change":"2024-04-25T10:11:44.001270+0000","last_active":"2024-04-25T10:11:46.060915+0000","last_peered":"2024-04-25T10:11:46.060915+0000","last_clean":"2024-04-25T10:11:46.060915+0000","last_became_active":"2024-04-25T10:11:44.000933+0000","last_became_peered":"2024-04-25T10:11:44.000933+0000","last_unstale":"2024-04-25T10:11:46.060915+0000","last_undegraded":"2024-04-25T10:11:46.060915+0000","last_fullsized":"2024-04-25T10:11:46.060915+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:26.219939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207334+0000","last_change":"2024-04-25T10:11:45.061058+0000","last_active":"2024-04-25T10:11:46.207334+0000","last_peered":"2024-04-25T10:11:46.207334+0000","last_clean":"2024-04-25T10:11:46.207334+0000","last_became_active":"2024-04-25T10:11:45.060406+0000","last_became_peered":"2024-04-25T10:11:45.060406+0000","last_unstale":"2024-04-25T10:11:46.207334+0000","last_undegraded":"2024-04-25T10:11:46.207334+0000","last_fullsized":"2024-04-25T10:11:46.207334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:23.214924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.061241+0000","last_change":"2024-04-25T10:11:44.001285+0000","last_active":"2024-04-25T10:11:46.061241+0000","last_peered":"2024-04-25T10:11:46.061241+0000","last_clean":"2024-04-25T10:11:46.061241+0000","last_became_active":"2024-04-25T10:11:44.000926+0000","last_became_peered":"2024-04-25T10:11:44.000926+0000","last_unstale":"2024-04-25T10:11:46.061241+0000","last_undegraded":"2024-04-25T10:11:46.061241+0000","last_fullsized":"2024-04-25T10:11:46.061241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:40.643678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.043181+0000","last_change":"2024-04-25T10:11:45.056307+0000","last_active":"2024-04-25T10:11:46.043181+0000","last_peered":"2024-04-25T10:11:46.043181+0000","last_clean":"2024-04-25T10:11:46.043181+0000","last_became_active":"2024-04-25T10:11:45.055925+0000","last_became_peered":"2024-04-25T10:11:45.055925+0000","last_unstale":"2024-04-25T10:11:46.043181+0000","last_undegraded":"2024-04-25T10:11:46.043181+0000","last_fullsized":"2024-04-25T10:11:46.043181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:44.618524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.048060+0000","last_change":"2024-04-25T10:11:44.023047+0000","last_active":"2024-04-25T10:11:46.048060+0000","last_peered":"2024-04-25T10:11:46.048060+0000","last_clean":"2024-04-25T10:11:46.048060+0000","last_became_active":"2024-04-25T10:11:44.022221+0000","last_became_peered":"2024-04-25T10:11:44.022221+0000","last_unstale":"2024-04-25T10:11:46.048060+0000","last_undegraded":"2024-04-25T10:11:46.048060+0000","last_fullsized":"2024-04-25T10:11:46.048060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:56.822160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047905+0000","last_change":"2024-04-25T10:11:45.058562+0000","last_active":"2024-04-25T10:11:46.047905+0000","last_peered":"2024-04-25T10:11:46.047905+0000","last_clean":"2024-04-25T10:11:46.047905+0000","last_became_active":"2024-04-25T10:11:45.057926+0000","last_became_peered":"2024-04-25T10:11:45.057926+0000","last_unstale":"2024-04-25T10:11:46.047905+0000","last_undegraded":"2024-04-25T10:11:46.047905+0000","last_fullsized":"2024-04-25T10:11:46.047905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:05.516707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046653+0000","last_change":"2024-04-25T10:11:44.017757+0000","last_active":"2024-04-25T10:11:46.046653+0000","last_peered":"2024-04-25T10:11:46.046653+0000","last_clean":"2024-04-25T10:11:46.046653+0000","last_became_active":"2024-04-25T10:11:44.016976+0000","last_became_peered":"2024-04-25T10:11:44.016976+0000","last_unstale":"2024-04-25T10:11:46.046653+0000","last_undegraded":"2024-04-25T10:11:46.046653+0000","last_fullsized":"2024-04-25T10:11:46.046653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:29.023830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.206675+0000","last_change":"2024-04-25T10:11:45.051305+0000","last_active":"2024-04-25T10:11:46.206675+0000","last_peered":"2024-04-25T10:11:46.206675+0000","last_clean":"2024-04-25T10:11:46.206675+0000","last_became_active":"2024-04-25T10:11:45.049893+0000","last_became_peered":"2024-04-25T10:11:45.049893+0000","last_unstale":"2024-04-25T10:11:46.206675+0000","last_undegraded":"2024-04-25T10:11:46.206675+0000","last_fullsized":"2024-04-25T10:11:46.206675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:10.057697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.061589+0000","last_change":"2024-04-25T10:11:44.012240+0000","last_active":"2024-04-25T10:11:46.061589+0000","last_peered":"2024-04-25T10:11:46.061589+0000","last_clean":"2024-04-25T10:11:46.061589+0000","last_became_active":"2024-04-25T10:11:44.011969+0000","last_became_peered":"2024-04-25T10:11:44.011969+0000","last_unstale":"2024-04-25T10:11:46.061589+0000","last_undegraded":"2024-04-25T10:11:46.061589+0000","last_fullsized":"2024-04-25T10:11:46.061589+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:24.512219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.332509+0000","last_change":"2024-04-25T10:11:45.040153+0000","last_active":"2024-04-25T10:11:46.332509+0000","last_peered":"2024-04-25T10:11:46.332509+0000","last_clean":"2024-04-25T10:11:46.332509+0000","last_became_active":"2024-04-25T10:11:45.039005+0000","last_became_peered":"2024-04-25T10:11:45.039005+0000","last_unstale":"2024-04-25T10:11:46.332509+0000","last_undegraded":"2024-04-25T10:11:46.332509+0000","last_fullsized":"2024-04-25T10:11:46.332509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:45.876545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.041785+0000","last_change":"2024-04-25T10:11:44.008870+0000","last_active":"2024-04-25T10:11:46.041785+0000","last_peered":"2024-04-25T10:11:46.041785+0000","last_clean":"2024-04-25T10:11:46.041785+0000","last_became_active":"2024-04-25T10:11:44.008466+0000","last_became_peered":"2024-04-25T10:11:44.008466+0000","last_unstale":"2024-04-25T10:11:46.041785+0000","last_undegraded":"2024-04-25T10:11:46.041785+0000","last_fullsized":"2024-04-25T10:11:46.041785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:33.581181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.333426+0000","last_change":"2024-04-25T10:11:45.042616+0000","last_active":"2024-04-25T10:11:46.333426+0000","last_peered":"2024-04-25T10:11:46.333426+0000","last_clean":"2024-04-25T10:11:46.333426+0000","last_became_active":"2024-04-25T10:11:45.041908+0000","last_became_peered":"2024-04-25T10:11:45.041908+0000","last_unstale":"2024-04-25T10:11:46.333426+0000","last_undegraded":"2024-04-25T10:11:46.333426+0000","last_fullsized":"2024-04-25T10:11:46.333426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:56.943947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335806+0000","last_change":"2024-04-25T10:11:44.029573+0000","last_active":"2024-04-25T10:11:46.335806+0000","last_peered":"2024-04-25T10:11:46.335806+0000","last_clean":"2024-04-25T10:11:46.335806+0000","last_became_active":"2024-04-25T10:11:44.028845+0000","last_became_peered":"2024-04-25T10:11:44.028845+0000","last_unstale":"2024-04-25T10:11:46.335806+0000","last_undegraded":"2024-04-25T10:11:46.335806+0000","last_fullsized":"2024-04-25T10:11:46.335806+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:54.498636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.048171+0000","last_change":"2024-04-25T10:11:45.061066+0000","last_active":"2024-04-25T10:11:46.048171+0000","last_peered":"2024-04-25T10:11:46.048171+0000","last_clean":"2024-04-25T10:11:46.048171+0000","last_became_active":"2024-04-25T10:11:45.060314+0000","last_became_peered":"2024-04-25T10:11:45.060314+0000","last_unstale":"2024-04-25T10:11:46.048171+0000","last_undegraded":"2024-04-25T10:11:46.048171+0000","last_fullsized":"2024-04-25T10:11:46.048171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:01.882798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335354+0000","last_change":"2024-04-25T10:11:44.015020+0000","last_active":"2024-04-25T10:11:46.335354+0000","last_peered":"2024-04-25T10:11:46.335354+0000","last_clean":"2024-04-25T10:11:46.335354+0000","last_became_active":"2024-04-25T10:11:44.014329+0000","last_became_peered":"2024-04-25T10:11:44.014329+0000","last_unstale":"2024-04-25T10:11:46.335354+0000","last_undegraded":"2024-04-25T10:11:46.335354+0000","last_fullsized":"2024-04-25T10:11:46.335354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:45.766870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.061009+0000","last_change":"2024-04-25T10:11:45.055583+0000","last_active":"2024-04-25T10:11:46.061009+0000","last_peered":"2024-04-25T10:11:46.061009+0000","last_clean":"2024-04-25T10:11:46.061009+0000","last_became_active":"2024-04-25T10:11:45.054621+0000","last_became_peered":"2024-04-25T10:11:45.054621+0000","last_unstale":"2024-04-25T10:11:46.061009+0000","last_undegraded":"2024-04-25T10:11:46.061009+0000","last_fullsized":"2024-04-25T10:11:46.061009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:33.024532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335021+0000","last_change":"2024-04-25T10:11:44.026968+0000","last_active":"2024-04-25T10:11:46.335021+0000","last_peered":"2024-04-25T10:11:46.335021+0000","last_clean":"2024-04-25T10:11:46.335021+0000","last_became_active":"2024-04-25T10:11:44.026321+0000","last_became_peered":"2024-04-25T10:11:44.026321+0000","last_unstale":"2024-04-25T10:11:46.335021+0000","last_undegraded":"2024-04-25T10:11:46.335021+0000","last_fullsized":"2024-04-25T10:11:46.335021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:28.871503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047847+0000","last_change":"2024-04-25T10:11:45.053749+0000","last_active":"2024-04-25T10:11:46.047847+0000","last_peered":"2024-04-25T10:11:46.047847+0000","last_clean":"2024-04-25T10:11:46.047847+0000","last_became_active":"2024-04-25T10:11:45.053039+0000","last_became_peered":"2024-04-25T10:11:45.053039+0000","last_unstale":"2024-04-25T10:11:46.047847+0000","last_undegraded":"2024-04-25T10:11:46.047847+0000","last_fullsized":"2024-04-25T10:11:46.047847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:06.439806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"81'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047265+0000","last_change":"2024-04-25T10:11:44.006862+0000","last_active":"2024-04-25T10:11:46.047265+0000","last_peered":"2024-04-25T10:11:46.047265+0000","last_clean":"2024-04-25T10:11:46.047265+0000","last_became_active":"2024-04-25T10:11:44.006146+0000","last_became_peered":"2024-04-25T10:11:44.006146+0000","last_unstale":"2024-04-25T10:11:46.047265+0000","last_undegraded":"2024-04-25T10:11:46.047265+0000","last_fullsized":"2024-04-25T10:11:46.047265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:06.738500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.060723+0000","last_change":"2024-04-25T10:11:45.053114+0000","last_active":"2024-04-25T10:11:46.060723+0000","last_peered":"2024-04-25T10:11:46.060723+0000","last_clean":"2024-04-25T10:11:46.060723+0000","last_became_active":"2024-04-25T10:11:45.052733+0000","last_became_peered":"2024-04-25T10:11:45.052733+0000","last_unstale":"2024-04-25T10:11:46.060723+0000","last_undegraded":"2024-04-25T10:11:46.060723+0000","last_fullsized":"2024-04-25T10:11:46.060723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:22.178842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.042533+0000","last_change":"2024-04-25T10:11:44.014482+0000","last_active":"2024-04-25T10:11:46.042533+0000","last_peered":"2024-04-25T10:11:46.042533+0000","last_clean":"2024-04-25T10:11:46.042533+0000","last_became_active":"2024-04-25T10:11:44.013943+0000","last_became_peered":"2024-04-25T10:11:44.013943+0000","last_unstale":"2024-04-25T10:11:46.042533+0000","last_undegraded":"2024-04-25T10:11:46.042533+0000","last_fullsized":"2024-04-25T10:11:46.042533+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:22.265695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207169+0000","last_change":"2024-04-25T10:11:45.056116+0000","last_active":"2024-04-25T10:11:46.207169+0000","last_peered":"2024-04-25T10:11:46.207169+0000","last_clean":"2024-04-25T10:11:46.207169+0000","last_became_active":"2024-04-25T10:11:45.050722+0000","last_became_peered":"2024-04-25T10:11:45.050722+0000","last_unstale":"2024-04-25T10:11:46.207169+0000","last_undegraded":"2024-04-25T10:11:46.207169+0000","last_fullsized":"2024-04-25T10:11:46.207169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:57.429333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.332801+0000","last_change":"2024-04-25T10:11:44.016162+0000","last_active":"2024-04-25T10:11:46.332801+0000","last_peered":"2024-04-25T10:11:46.332801+0000","last_clean":"2024-04-25T10:11:46.332801+0000","last_became_active":"2024-04-25T10:11:44.014948+0000","last_became_peered":"2024-04-25T10:11:44.014948+0000","last_unstale":"2024-04-25T10:11:46.332801+0000","last_undegraded":"2024-04-25T10:11:46.332801+0000","last_fullsized":"2024-04-25T10:11:46.332801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:32.017173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.041623+0000","last_change":"2024-04-25T10:11:45.057510+0000","last_active":"2024-04-25T10:11:46.041623+0000","last_peered":"2024-04-25T10:11:46.041623+0000","last_clean":"2024-04-25T10:11:46.041623+0000","last_became_active":"2024-04-25T10:11:45.057077+0000","last_became_peered":"2024-04-25T10:11:45.057077+0000","last_unstale":"2024-04-25T10:11:46.041623+0000","last_undegraded":"2024-04-25T10:11:46.041623+0000","last_fullsized":"2024-04-25T10:11:46.041623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:30.031189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046137+0000","last_change":"2024-04-25T10:11:44.008928+0000","last_active":"2024-04-25T10:11:46.046137+0000","last_peered":"2024-04-25T10:11:46.046137+0000","last_clean":"2024-04-25T10:11:46.046137+0000","last_became_active":"2024-04-25T10:11:44.007992+0000","last_became_peered":"2024-04-25T10:11:44.007992+0000","last_unstale":"2024-04-25T10:11:46.046137+0000","last_undegraded":"2024-04-25T10:11:46.046137+0000","last_fullsized":"2024-04-25T10:11:46.046137+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:59.862550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.041827+0000","last_change":"2024-04-25T10:11:45.033718+0000","last_active":"2024-04-25T10:11:46.041827+0000","last_peered":"2024-04-25T10:11:46.041827+0000","last_clean":"2024-04-25T10:11:46.041827+0000","last_became_active":"2024-04-25T10:11:45.032911+0000","last_became_peered":"2024-04-25T10:11:45.032911+0000","last_unstale":"2024-04-25T10:11:46.041827+0000","last_undegraded":"2024-04-25T10:11:46.041827+0000","last_fullsized":"2024-04-25T10:11:46.041827+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:41.700969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.060355+0000","last_change":"2024-04-25T10:11:44.010323+0000","last_active":"2024-04-25T10:11:46.060355+0000","last_peered":"2024-04-25T10:11:46.060355+0000","last_clean":"2024-04-25T10:11:46.060355+0000","last_became_active":"2024-04-25T10:11:44.009951+0000","last_became_peered":"2024-04-25T10:11:44.009951+0000","last_unstale":"2024-04-25T10:11:46.060355+0000","last_undegraded":"2024-04-25T10:11:46.060355+0000","last_fullsized":"2024-04-25T10:11:46.060355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:37.890944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207401+0000","last_change":"2024-04-25T10:11:45.050441+0000","last_active":"2024-04-25T10:11:46.207401+0000","last_peered":"2024-04-25T10:11:46.207401+0000","last_clean":"2024-04-25T10:11:46.207401+0000","last_became_active":"2024-04-25T10:11:45.049635+0000","last_became_peered":"2024-04-25T10:11:45.049635+0000","last_unstale":"2024-04-25T10:11:46.207401+0000","last_undegraded":"2024-04-25T10:11:46.207401+0000","last_fullsized":"2024-04-25T10:11:46.207401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:28.530096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335958+0000","last_change":"2024-04-25T10:11:44.027297+0000","last_active":"2024-04-25T10:11:46.335958+0000","last_peered":"2024-04-25T10:11:46.335958+0000","last_clean":"2024-04-25T10:11:46.335958+0000","last_became_active":"2024-04-25T10:11:44.026474+0000","last_became_peered":"2024-04-25T10:11:44.026474+0000","last_unstale":"2024-04-25T10:11:46.335958+0000","last_undegraded":"2024-04-25T10:11:46.335958+0000","last_fullsized":"2024-04-25T10:11:46.335958+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:11.465700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335548+0000","last_change":"2024-04-25T10:11:45.066030+0000","last_active":"2024-04-25T10:11:46.335548+0000","last_peered":"2024-04-25T10:11:46.335548+0000","last_clean":"2024-04-25T10:11:46.335548+0000","last_became_active":"2024-04-25T10:11:45.064286+0000","last_became_peered":"2024-04-25T10:11:45.064286+0000","last_unstale":"2024-04-25T10:11:46.335548+0000","last_undegraded":"2024-04-25T10:11:46.335548+0000","last_fullsized":"2024-04-25T10:11:46.335548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:01.882798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.060873+0000","last_change":"2024-04-25T10:11:44.007340+0000","last_active":"2024-04-25T10:11:46.060873+0000","last_peered":"2024-04-25T10:11:46.060873+0000","last_clean":"2024-04-25T10:11:46.060873+0000","last_became_active":"2024-04-25T10:11:44.006878+0000","last_became_peered":"2024-04-25T10:11:44.006878+0000","last_unstale":"2024-04-25T10:11:46.060873+0000","last_undegraded":"2024-04-25T10:11:46.060873+0000","last_fullsized":"2024-04-25T10:11:46.060873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:52.154015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.047580+0000","last_change":"2024-04-25T10:11:45.044021+0000","last_active":"2024-04-25T10:11:46.047580+0000","last_peered":"2024-04-25T10:11:46.047580+0000","last_clean":"2024-04-25T10:11:46.047580+0000","last_became_active":"2024-04-25T10:11:45.043292+0000","last_became_peered":"2024-04-25T10:11:45.043292+0000","last_unstale":"2024-04-25T10:11:46.047580+0000","last_undegraded":"2024-04-25T10:11:46.047580+0000","last_fullsized":"2024-04-25T10:11:46.047580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:16.016307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.061231+0000","last_change":"2024-04-25T10:11:44.012098+0000","last_active":"2024-04-25T10:11:46.061231+0000","last_peered":"2024-04-25T10:11:46.061231+0000","last_clean":"2024-04-25T10:11:46.061231+0000","last_became_active":"2024-04-25T10:11:44.011745+0000","last_became_peered":"2024-04-25T10:11:44.011745+0000","last_unstale":"2024-04-25T10:11:46.061231+0000","last_undegraded":"2024-04-25T10:11:46.061231+0000","last_fullsized":"2024-04-25T10:11:46.061231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:02.596488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.333913+0000","last_change":"2024-04-25T10:11:45.049708+0000","last_active":"2024-04-25T10:11:46.333913+0000","last_peered":"2024-04-25T10:11:46.333913+0000","last_clean":"2024-04-25T10:11:46.333913+0000","last_became_active":"2024-04-25T10:11:45.048821+0000","last_became_peered":"2024-04-25T10:11:45.048821+0000","last_unstale":"2024-04-25T10:11:46.333913+0000","last_undegraded":"2024-04-25T10:11:46.333913+0000","last_fullsized":"2024-04-25T10:11:46.333913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:14.885624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.332901+0000","last_change":"2024-04-25T10:11:44.001767+0000","last_active":"2024-04-25T10:11:46.332901+0000","last_peered":"2024-04-25T10:11:46.332901+0000","last_clean":"2024-04-25T10:11:46.332901+0000","last_became_active":"2024-04-25T10:11:44.001184+0000","last_became_peered":"2024-04-25T10:11:44.001184+0000","last_unstale":"2024-04-25T10:11:46.332901+0000","last_undegraded":"2024-04-25T10:11:46.332901+0000","last_fullsized":"2024-04-25T10:11:46.332901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:49.671623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046574+0000","last_change":"2024-04-25T10:11:45.041234+0000","last_active":"2024-04-25T10:11:46.046574+0000","last_peered":"2024-04-25T10:11:46.046574+0000","last_clean":"2024-04-25T10:11:46.046574+0000","last_became_active":"2024-04-25T10:11:45.039578+0000","last_became_peered":"2024-04-25T10:11:45.039578+0000","last_unstale":"2024-04-25T10:11:46.046574+0000","last_undegraded":"2024-04-25T10:11:46.046574+0000","last_fullsized":"2024-04-25T10:11:46.046574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:40.527351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334069+0000","last_change":"2024-04-25T10:11:43.999418+0000","last_active":"2024-04-25T10:11:46.334069+0000","last_peered":"2024-04-25T10:11:46.334069+0000","last_clean":"2024-04-25T10:11:46.334069+0000","last_became_active":"2024-04-25T10:11:43.998847+0000","last_became_peered":"2024-04-25T10:11:43.998847+0000","last_unstale":"2024-04-25T10:11:46.334069+0000","last_undegraded":"2024-04-25T10:11:46.334069+0000","last_fullsized":"2024-04-25T10:11:46.334069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:30.670903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.332364+0000","last_change":"2024-04-25T10:11:45.054251+0000","last_active":"2024-04-25T10:11:46.332364+0000","last_peered":"2024-04-25T10:11:46.332364+0000","last_clean":"2024-04-25T10:11:46.332364+0000","last_became_active":"2024-04-25T10:11:45.050664+0000","last_became_peered":"2024-04-25T10:11:45.050664+0000","last_unstale":"2024-04-25T10:11:46.332364+0000","last_undegraded":"2024-04-25T10:11:46.332364+0000","last_fullsized":"2024-04-25T10:11:46.332364+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:52.177015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.332417+0000","last_change":"2024-04-25T10:11:44.012408+0000","last_active":"2024-04-25T10:11:46.332417+0000","last_peered":"2024-04-25T10:11:46.332417+0000","last_clean":"2024-04-25T10:11:46.332417+0000","last_became_active":"2024-04-25T10:11:44.012121+0000","last_became_peered":"2024-04-25T10:11:44.012121+0000","last_unstale":"2024-04-25T10:11:46.332417+0000","last_undegraded":"2024-04-25T10:11:46.332417+0000","last_fullsized":"2024-04-25T10:11:46.332417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:13.420380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046063+0000","last_change":"2024-04-25T10:11:45.050580+0000","last_active":"2024-04-25T10:11:46.046063+0000","last_peered":"2024-04-25T10:11:46.046063+0000","last_clean":"2024-04-25T10:11:46.046063+0000","last_became_active":"2024-04-25T10:11:45.049858+0000","last_became_peered":"2024-04-25T10:11:45.049858+0000","last_unstale":"2024-04-25T10:11:46.046063+0000","last_undegraded":"2024-04-25T10:11:46.046063+0000","last_fullsized":"2024-04-25T10:11:46.046063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:03.603847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.331870+0000","last_change":"2024-04-25T10:11:44.002773+0000","last_active":"2024-04-25T10:11:46.331870+0000","last_peered":"2024-04-25T10:11:46.331870+0000","last_clean":"2024-04-25T10:11:46.331870+0000","last_became_active":"2024-04-25T10:11:44.002034+0000","last_became_peered":"2024-04-25T10:11:44.002034+0000","last_unstale":"2024-04-25T10:11:46.331870+0000","last_undegraded":"2024-04-25T10:11:46.331870+0000","last_fullsized":"2024-04-25T10:11:46.331870+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:45.490109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.048533+0000","last_change":"2024-04-25T10:11:45.027583+0000","last_active":"2024-04-25T10:11:46.048533+0000","last_peered":"2024-04-25T10:11:46.048533+0000","last_clean":"2024-04-25T10:11:46.048533+0000","last_became_active":"2024-04-25T10:11:45.026819+0000","last_became_peered":"2024-04-25T10:11:45.026819+0000","last_unstale":"2024-04-25T10:11:46.048533+0000","last_undegraded":"2024-04-25T10:11:46.048533+0000","last_fullsized":"2024-04-25T10:11:46.048533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:32.038199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.042670+0000","last_change":"2024-04-25T10:11:44.015314+0000","last_active":"2024-04-25T10:11:46.042670+0000","last_peered":"2024-04-25T10:11:46.042670+0000","last_clean":"2024-04-25T10:11:46.042670+0000","last_became_active":"2024-04-25T10:11:44.014485+0000","last_became_peered":"2024-04-25T10:11:44.014485+0000","last_unstale":"2024-04-25T10:11:46.042670+0000","last_undegraded":"2024-04-25T10:11:46.042670+0000","last_fullsized":"2024-04-25T10:11:46.042670+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:42.645985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207483+0000","last_change":"2024-04-25T10:11:45.060657+0000","last_active":"2024-04-25T10:11:46.207483+0000","last_peered":"2024-04-25T10:11:46.207483+0000","last_clean":"2024-04-25T10:11:46.207483+0000","last_became_active":"2024-04-25T10:11:45.058200+0000","last_became_peered":"2024-04-25T10:11:45.058200+0000","last_unstale":"2024-04-25T10:11:46.207483+0000","last_undegraded":"2024-04-25T10:11:46.207483+0000","last_fullsized":"2024-04-25T10:11:46.207483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:36.488666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.058571+0000","last_change":"2024-04-25T10:11:44.010083+0000","last_active":"2024-04-25T10:11:46.058571+0000","last_peered":"2024-04-25T10:11:46.058571+0000","last_clean":"2024-04-25T10:11:46.058571+0000","last_became_active":"2024-04-25T10:11:44.009518+0000","last_became_peered":"2024-04-25T10:11:44.009518+0000","last_unstale":"2024-04-25T10:11:46.058571+0000","last_undegraded":"2024-04-25T10:11:46.058571+0000","last_fullsized":"2024-04-25T10:11:46.058571+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:05.696828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335209+0000","last_change":"2024-04-25T10:11:45.067192+0000","last_active":"2024-04-25T10:11:46.335209+0000","last_peered":"2024-04-25T10:11:46.335209+0000","last_clean":"2024-04-25T10:11:46.335209+0000","last_became_active":"2024-04-25T10:11:45.064677+0000","last_became_peered":"2024-04-25T10:11:45.064677+0000","last_unstale":"2024-04-25T10:11:46.335209+0000","last_undegraded":"2024-04-25T10:11:46.335209+0000","last_fullsized":"2024-04-25T10:11:46.335209+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:27.227297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334719+0000","last_change":"2024-04-25T10:11:44.013908+0000","last_active":"2024-04-25T10:11:46.334719+0000","last_peered":"2024-04-25T10:11:46.334719+0000","last_clean":"2024-04-25T10:11:46.334719+0000","last_became_active":"2024-04-25T10:11:44.013090+0000","last_became_peered":"2024-04-25T10:11:44.013090+0000","last_unstale":"2024-04-25T10:11:46.334719+0000","last_undegraded":"2024-04-25T10:11:46.334719+0000","last_fullsized":"2024-04-25T10:11:46.334719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:03.261623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.048869+0000","last_change":"2024-04-25T10:11:45.042444+0000","last_active":"2024-04-25T10:11:46.048869+0000","last_peered":"2024-04-25T10:11:46.048869+0000","last_clean":"2024-04-25T10:11:46.048869+0000","last_became_active":"2024-04-25T10:11:45.040388+0000","last_became_peered":"2024-04-25T10:11:45.040388+0000","last_unstale":"2024-04-25T10:11:46.048869+0000","last_undegraded":"2024-04-25T10:11:46.048869+0000","last_fullsized":"2024-04-25T10:11:46.048869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:54.691413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.335422+0000","last_change":"2024-04-25T10:11:44.023684+0000","last_active":"2024-04-25T10:11:46.335422+0000","last_peered":"2024-04-25T10:11:46.335422+0000","last_clean":"2024-04-25T10:11:46.335422+0000","last_became_active":"2024-04-25T10:11:44.021325+0000","last_became_peered":"2024-04-25T10:11:44.021325+0000","last_unstale":"2024-04-25T10:11:46.335422+0000","last_undegraded":"2024-04-25T10:11:46.335422+0000","last_fullsized":"2024-04-25T10:11:46.335422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:11.873020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046371+0000","last_change":"2024-04-25T10:11:45.026993+0000","last_active":"2024-04-25T10:11:46.046371+0000","last_peered":"2024-04-25T10:11:46.046371+0000","last_clean":"2024-04-25T10:11:46.046371+0000","last_became_active":"2024-04-25T10:11:45.026188+0000","last_became_peered":"2024-04-25T10:11:45.026188+0000","last_unstale":"2024-04-25T10:11:46.046371+0000","last_undegraded":"2024-04-25T10:11:46.046371+0000","last_fullsized":"2024-04-25T10:11:46.046371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:27.227297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.061328+0000","last_change":"2024-04-25T10:11:44.010253+0000","last_active":"2024-04-25T10:11:46.061328+0000","last_peered":"2024-04-25T10:11:46.061328+0000","last_clean":"2024-04-25T10:11:46.061328+0000","last_became_active":"2024-04-25T10:11:44.009937+0000","last_became_peered":"2024-04-25T10:11:44.009937+0000","last_unstale":"2024-04-25T10:11:46.061328+0000","last_undegraded":"2024-04-25T10:11:46.061328+0000","last_fullsized":"2024-04-25T10:11:46.061328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:09.359831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.061179+0000","last_change":"2024-04-25T10:11:45.054354+0000","last_active":"2024-04-25T10:11:46.061179+0000","last_peered":"2024-04-25T10:11:46.061179+0000","last_clean":"2024-04-25T10:11:46.061179+0000","last_became_active":"2024-04-25T10:11:45.054042+0000","last_became_peered":"2024-04-25T10:11:45.054042+0000","last_unstale":"2024-04-25T10:11:46.061179+0000","last_undegraded":"2024-04-25T10:11:46.061179+0000","last_fullsized":"2024-04-25T10:11:46.061179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:50.678982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046943+0000","last_change":"2024-04-25T10:11:44.016859+0000","last_active":"2024-04-25T10:11:46.046943+0000","last_peered":"2024-04-25T10:11:46.046943+0000","last_clean":"2024-04-25T10:11:46.046943+0000","last_became_active":"2024-04-25T10:11:44.015863+0000","last_became_peered":"2024-04-25T10:11:44.015863+0000","last_unstale":"2024-04-25T10:11:46.046943+0000","last_undegraded":"2024-04-25T10:11:46.046943+0000","last_fullsized":"2024-04-25T10:11:46.046943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:24.642231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207427+0000","last_change":"2024-04-25T10:11:45.056740+0000","last_active":"2024-04-25T10:11:46.207427+0000","last_peered":"2024-04-25T10:11:46.207427+0000","last_clean":"2024-04-25T10:11:46.207427+0000","last_became_active":"2024-04-25T10:11:45.050875+0000","last_became_peered":"2024-04-25T10:11:45.050875+0000","last_unstale":"2024-04-25T10:11:46.207427+0000","last_undegraded":"2024-04-25T10:11:46.207427+0000","last_fullsized":"2024-04-25T10:11:46.207427+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:46.087055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334743+0000","last_change":"2024-04-25T10:11:44.023581+0000","last_active":"2024-04-25T10:11:46.334743+0000","last_peered":"2024-04-25T10:11:46.334743+0000","last_clean":"2024-04-25T10:11:46.334743+0000","last_became_active":"2024-04-25T10:11:44.022751+0000","last_became_peered":"2024-04-25T10:11:44.022751+0000","last_unstale":"2024-04-25T10:11:46.334743+0000","last_undegraded":"2024-04-25T10:11:46.334743+0000","last_fullsized":"2024-04-25T10:11:46.334743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:26.853442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.334474+0000","last_change":"2024-04-25T10:11:45.065388+0000","last_active":"2024-04-25T10:11:46.334474+0000","last_peered":"2024-04-25T10:11:46.334474+0000","last_clean":"2024-04-25T10:11:46.334474+0000","last_became_active":"2024-04-25T10:11:45.063569+0000","last_became_peered":"2024-04-25T10:11:45.063569+0000","last_unstale":"2024-04-25T10:11:46.334474+0000","last_undegraded":"2024-04-25T10:11:46.334474+0000","last_fullsized":"2024-04-25T10:11:46.334474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:25.519578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.046415+0000","last_change":"2024-04-25T10:11:44.030136+0000","last_active":"2024-04-25T10:11:46.046415+0000","last_peered":"2024-04-25T10:11:46.046415+0000","last_clean":"2024-04-25T10:11:46.046415+0000","last_became_active":"2024-04-25T10:11:44.029433+0000","last_became_peered":"2024-04-25T10:11:44.029433+0000","last_unstale":"2024-04-25T10:11:46.046415+0000","last_undegraded":"2024-04-25T10:11:46.046415+0000","last_fullsized":"2024-04-25T10:11:46.046415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:40.693610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.042450+0000","last_change":"2024-04-25T10:11:45.025591+0000","last_active":"2024-04-25T10:11:46.042450+0000","last_peered":"2024-04-25T10:11:46.042450+0000","last_clean":"2024-04-25T10:11:46.042450+0000","last_became_active":"2024-04-25T10:11:45.024672+0000","last_became_peered":"2024-04-25T10:11:45.024672+0000","last_unstale":"2024-04-25T10:11:46.042450+0000","last_undegraded":"2024-04-25T10:11:46.042450+0000","last_fullsized":"2024-04-25T10:11:46.042450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:43.972918+0000","last_clean_scrub_stamp":"2024-04-25T10:11:43.972918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:01.799712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.207395+0000","last_change":"2024-04-25T10:11:44.022906+0000","last_active":"2024-04-25T10:11:46.207395+0000","last_peered":"2024-04-25T10:11:46.207395+0000","last_clean":"2024-04-25T10:11:46.207395+0000","last_became_active":"2024-04-25T10:11:44.022169+0000","last_became_peered":"2024-04-25T10:11:44.022169+0000","last_unstale":"2024-04-25T10:11:46.207395+0000","last_undegraded":"2024-04-25T10:11:46.207395+0000","last_fullsized":"2024-04-25T10:11:46.207395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:42.965559+0000","last_clean_scrub_stamp":"2024-04-25T10:11:42.965559+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:05.432447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":281,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T10:11:46.206822+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:11:46.206822+0000","last_peered":"2024-04-25T10:11:46.206822+0000","last_clean":"2024-04-25T10:11:46.206822+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:11:46.206822+0000","last_undegraded":"2024-04-25T10:11:46.206822+0000","last_fullsized":"2024-04-25T10:11:46.206822+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509524,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1597440,"data_stored":1015475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509523,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":425037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542226,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1011712,"data_stored":425731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542227,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1015808,"data_stored":426671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542227,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":1011712,"data_stored":426649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542227,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36320,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335520,"statfs":{"total":96636764160,"available":96599572480,"internally_reserved":0,"allocated":1003520,"data_stored":425619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542227,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36336,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335504,"statfs":{"total":96636764160,"available":96599556096,"internally_reserved":0,"allocated":1011712,"data_stored":425231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542228,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36908,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334932,"statfs":{"total":96636764160,"available":96598970368,"internally_reserved":0,"allocated":1597440,"data_stored":1015439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:49.489 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:11:49.489 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:11:49.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.660+0000 7fc4a4227640 1 Processor -- start 2024-04-25T10:11:49.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.660+0000 7fc4a4227640 1 -- start start 2024-04-25T10:11:49.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.660+0000 7fc4a4227640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49c0cbf80 0x7fc49c0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.660+0000 7fc4a4227640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49c0cc360 con 0x7fc49c15c470 2024-04-25T10:11:49.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.660+0000 7fc4a4227640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49c15c9b0 con 0x7fc49c071320 2024-04-25T10:11:49.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.660+0000 7fc4a4227640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c15cc90 con 0x7fc49c0cbf80 2024-04-25T10:11:49.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.660+0000 7fc4a27c4640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc49c15c470 0x7fc49c15cf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54772/0 (socket says 172.21.15.60:54772) 2024-04-25T10:11:49.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.660+0000 7fc4a27c4640 1 -- 172.21.15.60:0/1412690443 learned_addr learned my addr 172.21.15.60:0/1412690443 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:49.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.660+0000 7fc4a1fc3640 1 --2- 172.21.15.60:0/1412690443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49c0cbf80 0x7fc49c0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.660+0000 7fc4a0fc1640 1 -- 172.21.15.60:0/1412690443 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493458629 0 0) 0x7fc49c0cc360 con 0x7fc49c15c470 2024-04-25T10:11:49.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 -- 172.21.15.60:0/1412690443 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc488003530 con 0x7fc49c15c470 2024-04-25T10:11:49.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 -- 172.21.15.60:0/1412690443 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4048831562 0 0) 0x7fc488003530 con 0x7fc49c15c470 2024-04-25T10:11:49.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 -- 172.21.15.60:0/1412690443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49c0cbf80 msgr2=0x7fc49c0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:49.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 --2- 172.21.15.60:0/1412690443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49c0cbf80 0x7fc49c0cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 -- 172.21.15.60:0/1412690443 >> v1:172.21.15.100:6789/0 conn(0x7fc49c071320 legacy=0x7fc49c15bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 -- 172.21.15.60:0/1412690443 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c0cd460 con 0x7fc49c15c470 2024-04-25T10:11:49.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 -- 172.21.15.60:0/1412690443 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fc494001a00 con 0x7fc49c15c470 2024-04-25T10:11:49.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 -- 172.21.15.60:0/1412690443 >> v1:172.21.15.60:6789/0 conn(0x7fc49c15c470 legacy=0x7fc49c15cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 --2- 172.21.15.60:0/1412690443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49c0cbf80 0x7fc4880039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 --2- 172.21.15.60:0/1412690443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc488003f70 0x7fc488004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 --2- 172.21.15.60:0/1412690443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc488004d50 0x7fc4880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 -- 172.21.15.60:0/1412690443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c15cc90 con 0x7fc488004d50 2024-04-25T10:11:49.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 -- 172.21.15.60:0/1412690443 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48800bab0 con 0x7fc488003f70 2024-04-25T10:11:49.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 -- 172.21.15.60:0/1412690443 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48800bcb0 con 0x7fc49c0cbf80 2024-04-25T10:11:49.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a27c4640 1 --2- 172.21.15.60:0/1412690443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc488004d50 0x7fc4880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a1fc3640 1 --2- 172.21.15.60:0/1412690443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49c0cbf80 0x7fc4880039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a17c2640 1 --2- 172.21.15.60:0/1412690443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc488003f70 0x7fc488004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a27c4640 1 -- 172.21.15.60:0/1412690443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49c0cbf80 msgr2=0x7fc4880039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:49.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a27c4640 1 --2- 172.21.15.60:0/1412690443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49c0cbf80 0x7fc4880039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a27c4640 1 -- 172.21.15.60:0/1412690443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc488003f70 msgr2=0x7fc488004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:49.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a27c4640 1 --2- 172.21.15.60:0/1412690443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc488003f70 0x7fc488004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a27c4640 1 -- 172.21.15.60:0/1412690443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc488003530 con 0x7fc488004d50 2024-04-25T10:11:49.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a17c2640 1 --2- 172.21.15.60:0/1412690443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc488003f70 0x7fc488004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a27c4640 1 --2- 172.21.15.60:0/1412690443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc488004d50 0x7fc4880051d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fc49400a850 tx=0x7fc49401d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7832ebc8d0fd37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 -- 172.21.15.60:0/1412690443 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49400a4a0 con 0x7fc488004d50 2024-04-25T10:11:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 -- 172.21.15.60:0/1412690443 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4940ad650 con 0x7fc488004d50 2024-04-25T10:11:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a0fc1640 1 -- 172.21.15.60:0/1412690443 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4940ada80 con 0x7fc488004d50 2024-04-25T10:11:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a4227640 1 -- 172.21.15.60:0/1412690443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc488004d50 msgr2=0x7fc4880051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a4227640 1 --2- 172.21.15.60:0/1412690443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc488004d50 0x7fc4880051d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fc49400a850 tx=0x7fc49401d3a0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a27c4640 1 -- 172.21.15.60:0/1412690443 reap_dead start 2024-04-25T10:11:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a4227640 1 -- 172.21.15.60:0/1412690443 shutdown_connections 2024-04-25T10:11:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a4227640 1 -- 172.21.15.60:0/1412690443 >> 172.21.15.60:0/1412690443 conn(0x7fc49c067060 msgr2=0x7fc49c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.664+0000 7fc4a4227640 1 -- 172.21.15.60:0/1412690443 shutdown_connections 2024-04-25T10:11:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a4227640 1 -- 172.21.15.60:0/1412690443 wait complete. 2024-04-25T10:11:49.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a4227640 1 Processor -- start 2024-04-25T10:11:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a4227640 1 -- start start 2024-04-25T10:11:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a4227640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc488004d50 0x7fc49c13d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a4227640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc49c0aec90 0x7fc49c1556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a4227640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49c154320 0x7fc49c1547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a4227640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0cdf40 con 0x7fc49c0aec90 2024-04-25T10:11:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a4227640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0cda20 con 0x7fc488004d50 2024-04-25T10:11:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a4227640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0ce0c0 con 0x7fc49c154320 2024-04-25T10:11:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a17c2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc49c0aec90 0x7fc49c1556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a17c2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc49c0aec90 0x7fc49c1556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46840/0 (socket says 172.21.15.60:46840) 2024-04-25T10:11:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a17c2640 1 -- 172.21.15.60:0/3841431635 learned_addr learned my addr 172.21.15.60:0/3841431635 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a1fc3640 1 --2- 172.21.15.60:0/3841431635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc488004d50 0x7fc49c13d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a27c4640 1 --2- 172.21.15.60:0/3841431635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49c154320 0x7fc49c1547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a17c2640 1 -- 172.21.15.60:0/3841431635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49c154320 msgr2=0x7fc49c1547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a17c2640 1 --2- 172.21.15.60:0/3841431635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49c154320 0x7fc49c1547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a17c2640 1 -- 172.21.15.60:0/3841431635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc488004d50 msgr2=0x7fc49c13d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a17c2640 1 --2- 172.21.15.60:0/3841431635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc488004d50 0x7fc49c13d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a17c2640 1 -- 172.21.15.60:0/3841431635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c0ae300 con 0x7fc49c0aec90 2024-04-25T10:11:49.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a17c2640 1 --2- 172.21.15.60:0/3841431635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc49c0aec90 0x7fc49c1556d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00d7a0 tx=0x7fc48c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:49.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4927fc640 1 -- 172.21.15.60:0/3841431635 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c005070 con 0x7fc49c0aec90 2024-04-25T10:11:49.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4927fc640 1 -- 172.21.15.60:0/3841431635 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c005210 con 0x7fc49c0aec90 2024-04-25T10:11:49.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a4227640 1 -- 172.21.15.60:0/3841431635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49c0b4340 con 0x7fc49c0aec90 2024-04-25T10:11:49.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4927fc640 1 -- 172.21.15.60:0/3841431635 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c007030 con 0x7fc49c0aec90 2024-04-25T10:11:49.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.668+0000 7fc4a4227640 1 -- 172.21.15.60:0/3841431635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49c0adb00 con 0x7fc49c0aec90 2024-04-25T10:11:49.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.672+0000 7fc4a4227640 1 -- 172.21.15.60:0/3841431635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49c074770 con 0x7fc49c0aec90 2024-04-25T10:11:49.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.672+0000 7fc4927fc640 1 -- 172.21.15.60:0/3841431635 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc48c01a020 con 0x7fc49c0aec90 2024-04-25T10:11:49.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.672+0000 7fc4927fc640 1 --2- 172.21.15.60:0/3841431635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc480066f90 0x7fc480069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.672+0000 7fc4927fc640 1 -- 172.21.15.60:0/3841431635 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc48c084930 con 0x7fc49c0aec90 2024-04-25T10:11:49.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.672+0000 7fc4a1fc3640 1 --2- 172.21.15.60:0/3841431635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc480066f90 0x7fc480069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.672+0000 7fc4a1fc3640 1 --2- 172.21.15.60:0/3841431635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc480066f90 0x7fc480069450 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc498009a30 tx=0x7fc498008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:49.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.676+0000 7fc4927fc640 1 -- 172.21.15.60:0/3841431635 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc48c055810 con 0x7fc49c0aec90 2024-04-25T10:11:49.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.884+0000 7fc4a4227640 1 -- 172.21.15.60:0/3841431635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc49c0acd30 con 0x7fc49c0aec90 2024-04-25T10:11:49.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.888+0000 7fc4927fc640 1 -- 172.21.15.60:0/3841431635 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc48c055110 con 0x7fc49c0aec90 2024-04-25T10:11:49.897 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T10:11:49.897 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCiKipmv65dLxAA6/QpPrqE9aT+CtZqkyutog== 2024-04-25T10:11:49.897 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-04-25T10:11:49.898 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-04-25T10:11:49.898 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:11:49.898 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-04-25T10:11:49.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.892+0000 7fc4a4227640 1 -- 172.21.15.60:0/3841431635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc480066f90 msgr2=0x7fc480069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.892+0000 7fc4a4227640 1 --2- 172.21.15.60:0/3841431635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc480066f90 0x7fc480069450 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc498009a30 tx=0x7fc498008040 comp rx=0 tx=0).stop 2024-04-25T10:11:49.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.892+0000 7fc4a4227640 1 -- 172.21.15.60:0/3841431635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc49c0aec90 msgr2=0x7fc49c1556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.892+0000 7fc4a4227640 1 --2- 172.21.15.60:0/3841431635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc49c0aec90 0x7fc49c1556d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00d7a0 tx=0x7fc48c00dc60 comp rx=0 tx=0).stop 2024-04-25T10:11:49.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.892+0000 7fc4a4227640 1 -- 172.21.15.60:0/3841431635 shutdown_connections 2024-04-25T10:11:49.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.892+0000 7fc4a4227640 1 --2- 172.21.15.60:0/3841431635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc480066f90 0x7fc480069450 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.892+0000 7fc4a4227640 1 --2- 172.21.15.60:0/3841431635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc49c154320 0x7fc49c1547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.892+0000 7fc4a4227640 1 --2- 172.21.15.60:0/3841431635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc49c0aec90 0x7fc49c1556d0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.892+0000 7fc4a4227640 1 --2- 172.21.15.60:0/3841431635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc488004d50 0x7fc49c13d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.892+0000 7fc4a4227640 1 -- 172.21.15.60:0/3841431635 >> 172.21.15.60:0/3841431635 conn(0x7fc49c067060 msgr2=0x7fc49c0b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:49.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.892+0000 7fc4a4227640 1 -- 172.21.15.60:0/3841431635 shutdown_connections 2024-04-25T10:11:49.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:49.892+0000 7fc4a4227640 1 -- 172.21.15.60:0/3841431635 wait complete. 2024-04-25T10:11:49.915 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:50.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.028+0000 7f1f63671640 1 Processor -- start 2024-04-25T10:11:50.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.028+0000 7f1f63671640 1 -- start start 2024-04-25T10:11:50.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.028+0000 7f1f63671640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f5c0ce7a0 0x7f1f5c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.028+0000 7f1f63671640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1f5c0b4660 0x7f1f5c0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.028+0000 7f1f63671640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c0cf0c0 con 0x7f1f5c15bc90 2024-04-25T10:11:50.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.028+0000 7f1f63671640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f5c0cc370 con 0x7f1f5c0b4660 2024-04-25T10:11:50.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.028+0000 7f1f63671640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c071ca0 con 0x7f1f5c0ce7a0 2024-04-25T10:11:50.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.028+0000 7f1f61c0e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1f5c0b4660 0x7f1f5c0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.028+0000 7f1f6140d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f5c0ce7a0 0x7f1f5c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.028+0000 7f1f61c0e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1f5c0b4660 0x7f1f5c0b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58060/0 (socket says 172.21.15.60:58060) 2024-04-25T10:11:50.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.028+0000 7f1f60c0c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1f5c15bc90 0x7f1f5c0b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:59680/0 (socket says 172.21.15.60:59680) 2024-04-25T10:11:50.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.028+0000 7f1f61c0e640 1 -- 172.21.15.60:0/3067503550 learned_addr learned my addr 172.21.15.60:0/3067503550 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:50.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.028+0000 7f1f6140d640 1 -- 172.21.15.60:0/3067503550 >> v2:172.21.15.100:3300/0 conn(0x7f1f5c0b4660 msgr2=0x7f1f5c0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.028+0000 7f1f6140d640 1 --2- 172.21.15.60:0/3067503550 >> v2:172.21.15.100:3300/0 conn(0x7f1f5c0b4660 0x7f1f5c0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.028+0000 7f1f6140d640 1 -- 172.21.15.60:0/3067503550 >> v1:172.21.15.100:6789/0 conn(0x7f1f5c15bc90 legacy=0x7f1f5c0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.028+0000 7f1f6140d640 1 -- 172.21.15.60:0/3067503550 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c0cca40 con 0x7f1f5c0ce7a0 2024-04-25T10:11:50.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.032+0000 7f1f6140d640 1 --2- 172.21.15.60:0/3067503550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f5c0ce7a0 0x7f1f5c0ceb80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1f580097f0 tx=0x7f1f5801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea01809c694304e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.032+0000 7f1f53fff640 1 -- 172.21.15.60:0/3067503550 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f580b0070 con 0x7f1f5c0ce7a0 2024-04-25T10:11:50.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.032+0000 7f1f53fff640 1 -- 172.21.15.60:0/3067503550 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f580ab600 con 0x7f1f5c0ce7a0 2024-04-25T10:11:50.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.032+0000 7f1f53fff640 1 -- 172.21.15.60:0/3067503550 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f580ab9d0 con 0x7f1f5c0ce7a0 2024-04-25T10:11:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.032+0000 7f1f63671640 1 -- 172.21.15.60:0/3067503550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f5c0ce7a0 msgr2=0x7f1f5c0ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.032+0000 7f1f63671640 1 --2- 172.21.15.60:0/3067503550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f5c0ce7a0 0x7f1f5c0ceb80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1f580097f0 tx=0x7f1f5801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.032+0000 7f1f63671640 1 -- 172.21.15.60:0/3067503550 shutdown_connections 2024-04-25T10:11:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.032+0000 7f1f63671640 1 --2- 172.21.15.60:0/3067503550 >> v2:172.21.15.100:3300/0 conn(0x7f1f5c0b4660 0x7f1f5c0b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.032+0000 7f1f63671640 1 --2- 172.21.15.60:0/3067503550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f5c0ce7a0 0x7f1f5c0ceb80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.032+0000 7f1f63671640 1 -- 172.21.15.60:0/3067503550 >> 172.21.15.60:0/3067503550 conn(0x7f1f5c0b12d0 msgr2=0x7f1f5c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.032+0000 7f1f63671640 1 -- 172.21.15.60:0/3067503550 shutdown_connections 2024-04-25T10:11:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.032+0000 7f1f63671640 1 -- 172.21.15.60:0/3067503550 wait complete. 2024-04-25T10:11:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.032+0000 7f1f63671640 1 Processor -- start 2024-04-25T10:11:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.032+0000 7f1f63671640 1 -- start start 2024-04-25T10:11:50.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f63671640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f5c0ce7a0 0x7f1f5c06d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f63671640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f5c15bc90 0x7f1f5c1609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f63671640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f5c1556a0 0x7f1f5c15c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f63671640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0b4280 con 0x7f1f5c15bc90 2024-04-25T10:11:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f63671640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0b3db0 con 0x7f1f5c0ce7a0 2024-04-25T10:11:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f63671640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0b4400 con 0x7f1f5c1556a0 2024-04-25T10:11:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f60c0c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f5c15bc90 0x7f1f5c1609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f60c0c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f5c15bc90 0x7f1f5c1609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46854/0 (socket says 172.21.15.60:46854) 2024-04-25T10:11:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f60c0c640 1 -- 172.21.15.60:0/305958580 learned_addr learned my addr 172.21.15.60:0/305958580 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f61c0e640 1 --2- 172.21.15.60:0/305958580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f5c1556a0 0x7f1f5c15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f6140d640 1 --2- 172.21.15.60:0/305958580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f5c0ce7a0 0x7f1f5c06d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f60c0c640 1 -- 172.21.15.60:0/305958580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f5c1556a0 msgr2=0x7f1f5c15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f60c0c640 1 --2- 172.21.15.60:0/305958580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f5c1556a0 0x7f1f5c15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f60c0c640 1 -- 172.21.15.60:0/305958580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f5c0ce7a0 msgr2=0x7f1f5c06d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f60c0c640 1 --2- 172.21.15.60:0/305958580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f5c0ce7a0 0x7f1f5c06d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f60c0c640 1 -- 172.21.15.60:0/305958580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c160ef0 con 0x7f1f5c15bc90 2024-04-25T10:11:50.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f60c0c640 1 --2- 172.21.15.60:0/305958580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f5c15bc90 0x7f1f5c1609b0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00c490 tx=0x7f1f4c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f51ffb640 1 -- 172.21.15.60:0/305958580 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c018070 con 0x7f1f5c15bc90 2024-04-25T10:11:50.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f51ffb640 1 -- 172.21.15.60:0/305958580 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c007480 con 0x7f1f5c15bc90 2024-04-25T10:11:50.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f51ffb640 1 -- 172.21.15.60:0/305958580 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c004a10 con 0x7f1f5c15bc90 2024-04-25T10:11:50.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f63671640 1 -- 172.21.15.60:0/305958580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c06dfd0 con 0x7f1f5c15bc90 2024-04-25T10:11:50.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f63671640 1 -- 172.21.15.60:0/305958580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c06e590 con 0x7f1f5c15bc90 2024-04-25T10:11:50.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f63671640 1 -- 172.21.15.60:0/305958580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5c074770 con 0x7f1f5c15bc90 2024-04-25T10:11:50.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f51ffb640 1 -- 172.21.15.60:0/305958580 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f4c01b070 con 0x7f1f5c15bc90 2024-04-25T10:11:50.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f51ffb640 1 --2- 172.21.15.60:0/305958580 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f40066f60 0x7f1f40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.036+0000 7f1f51ffb640 1 -- 172.21.15.60:0/305958580 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f4c084170 con 0x7f1f5c15bc90 2024-04-25T10:11:50.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.040+0000 7f1f6140d640 1 --2- 172.21.15.60:0/305958580 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f40066f60 0x7f1f40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.040+0000 7f1f6140d640 1 --2- 172.21.15.60:0/305958580 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f40066f60 0x7f1f40069420 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1f580ad000 tx=0x7f1f580a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.040+0000 7f1f51ffb640 1 -- 172.21.15.60:0/305958580 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f4c054b30 con 0x7f1f5c15bc90 2024-04-25T10:11:50.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.228+0000 7f1f63671640 1 -- 172.21.15.60:0/305958580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f5c15b6c0 con 0x7f1f5c15bc90 2024-04-25T10:11:50.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.232+0000 7f1f51ffb640 1 -- 172.21.15.60:0/305958580 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f1f4c058bd0 con 0x7f1f5c15bc90 2024-04-25T10:11:50.240 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:50.240 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:11:47.412210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7653":{"gid":7653,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2455562778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2455562778},{"type":"v1","addr":"172.21.15.60:6835","nonce":2455562778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:50.240 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 55 2024-04-25T10:11:50.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.232+0000 7f1f63671640 1 -- 172.21.15.60:0/305958580 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f40066f60 msgr2=0x7f1f40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.232+0000 7f1f63671640 1 --2- 172.21.15.60:0/305958580 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f40066f60 0x7f1f40069420 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1f580ad000 tx=0x7f1f580a6000 comp rx=0 tx=0).stop 2024-04-25T10:11:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.232+0000 7f1f63671640 1 -- 172.21.15.60:0/305958580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f5c15bc90 msgr2=0x7f1f5c1609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.232+0000 7f1f63671640 1 --2- 172.21.15.60:0/305958580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f5c15bc90 0x7f1f5c1609b0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00c490 tx=0x7f1f4c00c950 comp rx=0 tx=0).stop 2024-04-25T10:11:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.232+0000 7f1f63671640 1 -- 172.21.15.60:0/305958580 shutdown_connections 2024-04-25T10:11:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.232+0000 7f1f63671640 1 --2- 172.21.15.60:0/305958580 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f40066f60 0x7f1f40069420 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.232+0000 7f1f63671640 1 --2- 172.21.15.60:0/305958580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f5c1556a0 0x7f1f5c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.232+0000 7f1f63671640 1 --2- 172.21.15.60:0/305958580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f5c15bc90 0x7f1f5c1609b0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.232+0000 7f1f63671640 1 --2- 172.21.15.60:0/305958580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f5c0ce7a0 0x7f1f5c06d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.232+0000 7f1f63671640 1 -- 172.21.15.60:0/305958580 >> 172.21.15.60:0/305958580 conn(0x7f1f5c0b12d0 msgr2=0x7f1f5c0b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.232+0000 7f1f63671640 1 -- 172.21.15.60:0/305958580 shutdown_connections 2024-04-25T10:11:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.232+0000 7f1f63671640 1 -- 172.21.15.60:0/305958580 wait complete. 2024-04-25T10:11:50.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:44.357874+0000', 'modified': '2024-04-25T10:11:47.412210+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7653}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7653': {'gid': 7653, 'name': 'c', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/2455562778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 2455562778}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 2455562778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:50.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:11:50.661 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:11:50.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc73a7640 1 Processor -- start 2024-04-25T10:11:50.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc73a7640 1 -- start start 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc73a7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc00b53f0 0x7fbbc00b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc73a7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc00b9900 con 0x7fbbc0160eb0 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc73a7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc00ce9c0 con 0x7fbbc0161290 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc73a7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00ceca0 con 0x7fbbc00b53f0 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc5143640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbbc0160eb0 0x7fbbc00b91f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54788/0 (socket says 172.21.15.60:54788) 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc5143640 1 -- 172.21.15.60:0/812407708 learned_addr learned my addr 172.21.15.60:0/812407708 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc5944640 1 --2- 172.21.15.60:0/812407708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc00b53f0 0x7fbbc00b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbb7fff640 1 -- 172.21.15.60:0/812407708 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1187992729 0 0) 0x7fbbc00b9900 con 0x7fbbc0160eb0 2024-04-25T10:11:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbb7fff640 1 -- 172.21.15.60:0/812407708 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbac003530 con 0x7fbbc0160eb0 2024-04-25T10:11:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbb7fff640 1 -- 172.21.15.60:0/812407708 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4058423403 0 0) 0x7fbbac003530 con 0x7fbbc0160eb0 2024-04-25T10:11:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbb7fff640 1 -- 172.21.15.60:0/812407708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc00b53f0 msgr2=0x7fbbc00b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbb7fff640 1 --2- 172.21.15.60:0/812407708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc00b53f0 0x7fbbc00b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbb7fff640 1 -- 172.21.15.60:0/812407708 >> v1:172.21.15.100:6789/0 conn(0x7fbbc0161290 legacy=0x7fbbc00b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbb7fff640 1 -- 172.21.15.60:0/812407708 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc00d2490 con 0x7fbbc0160eb0 2024-04-25T10:11:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbb7fff640 1 -- 172.21.15.60:0/812407708 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fbbbc001680 con 0x7fbbc0160eb0 2024-04-25T10:11:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbb7fff640 1 -- 172.21.15.60:0/812407708 >> v1:172.21.15.60:6789/0 conn(0x7fbbc0160eb0 legacy=0x7fbbc00b91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbb7fff640 1 --2- 172.21.15.60:0/812407708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbac0039f0 0x7fbbac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbb7fff640 1 --2- 172.21.15.60:0/812407708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbac0043a0 0x7fbbac004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbb7fff640 1 --2- 172.21.15.60:0/812407708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc00b53f0 0x7fbbac0050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbb7fff640 1 -- 172.21.15.60:0/812407708 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00ceca0 con 0x7fbbac0043a0 2024-04-25T10:11:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbb7fff640 1 -- 172.21.15.60:0/812407708 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac00b9d0 con 0x7fbbac0039f0 2024-04-25T10:11:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbb7fff640 1 -- 172.21.15.60:0/812407708 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac00bbd0 con 0x7fbbc00b53f0 2024-04-25T10:11:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc4942640 1 --2- 172.21.15.60:0/812407708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbac0043a0 0x7fbbac004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc5143640 1 --2- 172.21.15.60:0/812407708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbac0039f0 0x7fbbac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc5944640 1 --2- 172.21.15.60:0/812407708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc00b53f0 0x7fbbac0050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc4942640 1 -- 172.21.15.60:0/812407708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc00b53f0 msgr2=0x7fbbac0050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc4942640 1 --2- 172.21.15.60:0/812407708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc00b53f0 0x7fbbac0050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc4942640 1 -- 172.21.15.60:0/812407708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbac0039f0 msgr2=0x7fbbac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc4942640 1 --2- 172.21.15.60:0/812407708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbac0039f0 0x7fbbac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc4942640 1 -- 172.21.15.60:0/812407708 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac003530 con 0x7fbbac0043a0 2024-04-25T10:11:50.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.788+0000 7fbbc5143640 1 --2- 172.21.15.60:0/812407708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbac0039f0 0x7fbbac003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.792+0000 7fbbc4942640 1 --2- 172.21.15.60:0/812407708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbac0043a0 0x7fbbac004870 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0001860 tx=0x7fbbb001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122e1deadaa43b3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.792+0000 7fbbb7fff640 1 -- 172.21.15.60:0/812407708 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0001460 con 0x7fbbac0043a0 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.792+0000 7fbbb7fff640 1 -- 172.21.15.60:0/812407708 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb00b0070 con 0x7fbbac0043a0 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.792+0000 7fbbb7fff640 1 -- 172.21.15.60:0/812407708 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb00ab880 con 0x7fbbac0043a0 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.792+0000 7fbbc73a7640 1 -- 172.21.15.60:0/812407708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbac0043a0 msgr2=0x7fbbac004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.792+0000 7fbbc73a7640 1 --2- 172.21.15.60:0/812407708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbac0043a0 0x7fbbac004870 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0001860 tx=0x7fbbb001c500 comp rx=0 tx=0).stop 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.792+0000 7fbbc5944640 1 -- 172.21.15.60:0/812407708 reap_dead start 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.792+0000 7fbbc73a7640 1 -- 172.21.15.60:0/812407708 shutdown_connections 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.792+0000 7fbbc73a7640 1 -- 172.21.15.60:0/812407708 >> 172.21.15.60:0/812407708 conn(0x7fbbc00d05c0 msgr2=0x7fbbc00d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.792+0000 7fbbc73a7640 1 -- 172.21.15.60:0/812407708 shutdown_connections 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.792+0000 7fbbc73a7640 1 -- 172.21.15.60:0/812407708 wait complete. 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc73a7640 1 Processor -- start 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc73a7640 1 -- start start 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc73a7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbc0150eb0 0x7fbbc00ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc73a7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc0151290 0x7fbbc015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc73a7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00cd420 0x7fbbc00cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc73a7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00b7180 con 0x7fbbc00cd420 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc73a7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00b2e70 con 0x7fbbc0150eb0 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc73a7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00cef80 con 0x7fbbc0151290 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc5944640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00cd420 0x7fbbc00cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc5944640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00cd420 0x7fbbc00cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46868/0 (socket says 172.21.15.60:46868) 2024-04-25T10:11:50.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc5944640 1 -- 172.21.15.60:0/813024357 learned_addr learned my addr 172.21.15.60:0/813024357 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:50.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc5143640 1 --2- 172.21.15.60:0/813024357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbc0150eb0 0x7fbbc00ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc4942640 1 --2- 172.21.15.60:0/813024357 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc0151290 0x7fbbc015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc5944640 1 -- 172.21.15.60:0/813024357 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc0151290 msgr2=0x7fbbc015adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc5944640 1 --2- 172.21.15.60:0/813024357 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc0151290 0x7fbbc015adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc5944640 1 -- 172.21.15.60:0/813024357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbc0150eb0 msgr2=0x7fbbc00ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc5944640 1 --2- 172.21.15.60:0/813024357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbc0150eb0 0x7fbbc00ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc5944640 1 -- 172.21.15.60:0/813024357 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc00cac50 con 0x7fbbc00cd420 2024-04-25T10:11:50.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc5944640 1 --2- 172.21.15.60:0/813024357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00cd420 0x7fbbc00cba10 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00b60b0 tx=0x7fbbb80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbb5ffb640 1 -- 172.21.15.60:0/813024357 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb80039a0 con 0x7fbbc00cd420 2024-04-25T10:11:50.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbb5ffb640 1 -- 172.21.15.60:0/813024357 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb801b070 con 0x7fbbc00cd420 2024-04-25T10:11:50.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc73a7640 1 -- 172.21.15.60:0/813024357 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc00caf30 con 0x7fbbc00cd420 2024-04-25T10:11:50.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbb5ffb640 1 -- 172.21.15.60:0/813024357 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8003b80 con 0x7fbbc00cd420 2024-04-25T10:11:50.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.796+0000 7fbbc73a7640 1 -- 172.21.15.60:0/813024357 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc00cb4f0 con 0x7fbbc00cd420 2024-04-25T10:11:50.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbb5ffb640 1 -- 172.21.15.60:0/813024357 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbbb8004030 con 0x7fbbc00cd420 2024-04-25T10:11:50.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbb5ffb640 1 --2- 172.21.15.60:0/813024357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbba4066f60 0x7fbba4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbb5ffb640 1 -- 172.21.15.60:0/813024357 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbb80859e0 con 0x7fbbc00cd420 2024-04-25T10:11:50.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbc5143640 1 --2- 172.21.15.60:0/813024357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbba4066f60 0x7fbba4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 Processor -- start 2024-04-25T10:11:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 -- start start 2024-04-25T10:11:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbc5143640 1 --2- 172.21.15.60:0/813024357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbba4066f60 0x7fbba4069420 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc001ec0 tx=0x7fbbbc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644eb920 0x5558644ebd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644ec240 0x5558644a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644a1f30 0x5558644a2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558643895f0 con 0x5558644eb920 2024-04-25T10:11:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558642f0da0 con 0x5558644ec240 2024-04-25T10:11:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x555864279150 con 0x5558644a1f30 2024-04-25T10:11:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbc5143640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644eb920 0x5558644ebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbc5143640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644eb920 0x5558644ebd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46878/0 (socket says 172.21.15.60:46878) 2024-04-25T10:11:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbc5143640 1 -- 172.21.15.60:0/728174504 learned_addr learned my addr 172.21.15.60:0/728174504 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbc4942640 1 --2- 172.21.15.60:0/728174504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644ec240 0x5558644a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbc5944640 1 --2- 172.21.15.60:0/728174504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644a1f30 0x5558644a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbc5143640 1 -- 172.21.15.60:0/728174504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644a1f30 msgr2=0x5558644a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbc5143640 1 --2- 172.21.15.60:0/728174504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644a1f30 0x5558644a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbc5143640 1 -- 172.21.15.60:0/728174504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644ec240 msgr2=0x5558644a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbc5143640 1 --2- 172.21.15.60:0/728174504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644ec240 0x5558644a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbc5143640 1 -- 172.21.15.60:0/728174504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555864292ca0 con 0x5558644eb920 2024-04-25T10:11:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbc5143640 1 --2- 172.21.15.60:0/728174504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644eb920 0x5558644ebd00 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc001770 tx=0x7fbbbc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b63e6b669f6893 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbaaffd640 1 -- 172.21.15.60:0/728174504 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc029070 con 0x5558644eb920 2024-04-25T10:11:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbaaffd640 1 -- 172.21.15.60:0/728174504 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbbc016b70 con 0x5558644eb920 2024-04-25T10:11:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbaaffd640 1 -- 172.21.15.60:0/728174504 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc010040 con 0x5558644eb920 2024-04-25T10:11:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 -- 172.21.15.60:0/728174504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644eb920 msgr2=0x5558644ebd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 --2- 172.21.15.60:0/728174504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644eb920 0x5558644ebd00 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc001770 tx=0x7fbbbc016000 comp rx=0 tx=0).stop 2024-04-25T10:11:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 -- 172.21.15.60:0/728174504 shutdown_connections 2024-04-25T10:11:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 --2- 172.21.15.60:0/728174504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644a1f30 0x5558644a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 --2- 172.21.15.60:0/728174504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644ec240 0x5558644a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 --2- 172.21.15.60:0/728174504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644eb920 0x5558644ebd00 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 -- 172.21.15.60:0/728174504 >> 172.21.15.60:0/728174504 conn(0x5558644a82a0 msgr2=0x5558644a02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 -- 172.21.15.60:0/728174504 shutdown_connections 2024-04-25T10:11:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.800+0000 7fbbca514000 1 -- 172.21.15.60:0/728174504 wait complete. 2024-04-25T10:11:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 1 -- start start 2024-04-25T10:11:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 1 Processor -- start 2024-04-25T10:11:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644a1f30 0x5558644f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644eb920 0x5558644f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644ec240 0x5558644a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558642ab990 con 0x5558644a1f30 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558642ece40 con 0x5558644ec240 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55586427c900 con 0x5558644eb920 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 10 client.0 _ll_register_callbacks cb 0x5558643a41c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbc5143640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644a1f30 0x5558644f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbc5143640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644a1f30 0x5558644f2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46892/0 (socket says 172.21.15.60:46892) 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbc5143640 1 -- 172.21.15.60:0/2047675470 learned_addr learned my addr 172.21.15.60:0/2047675470 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbc4942640 1 --2- 172.21.15.60:0/2047675470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644eb920 0x5558644f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbc5143640 1 -- 172.21.15.60:0/2047675470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644eb920 msgr2=0x5558644f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbc5944640 1 --2- 172.21.15.60:0/2047675470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644ec240 0x5558644a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbc5143640 1 --2- 172.21.15.60:0/2047675470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644eb920 0x5558644f31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbc5143640 1 -- 172.21.15.60:0/2047675470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644ec240 msgr2=0x5558644a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbc5143640 1 --2- 172.21.15.60:0/2047675470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644ec240 0x5558644a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbc5143640 1 -- 172.21.15.60:0/2047675470 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558642a9630 con 0x5558644a1f30 2024-04-25T10:11:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbc5944640 1 --2- 172.21.15.60:0/2047675470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644ec240 0x5558644a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbc5143640 1 --2- 172.21.15.60:0/2047675470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644a1f30 0x5558644f2c80 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc001000 tx=0x7fbbc00b6950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10138ed8a7c06639 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbb9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:11:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbb9ffff640 1 -- 172.21.15.60:0/2047675470 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc034060 con 0x5558644a1f30 2024-04-25T10:11:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 1 -- 172.21.15.60:0/2047675470 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbbc016b70 con 0x5558644a1f30 2024-04-25T10:11:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbb9ffff640 1 -- 172.21.15.60:0/2047675470 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbbc023830 con 0x5558644a1f30 2024-04-25T10:11:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbb9ffff640 1 -- 172.21.15.60:0/2047675470 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbbc03a050 con 0x5558644a1f30 2024-04-25T10:11:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 10 client.7803 fetch_fsmap learned FSMap version 55 2024-04-25T10:11:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 1 -- 172.21.15.60:0/2047675470 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x5558642793a0 con 0x5558644a1f30 2024-04-25T10:11:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbb9ffff640 1 -- 172.21.15.60:0/2047675470 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbbc016b70 con 0x5558644a1f30 2024-04-25T10:11:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbb9ffff640 1 -- 172.21.15.60:0/2047675470 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbbbc023a70 con 0x5558644a1f30 2024-04-25T10:11:50.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 10 client.7803 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T10:11:50.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 10 client.7803 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:11:50.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 20 client.7803 populate_metadata read hostname 'smithi060' 2024-04-25T10:11:50.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 1 --2- 172.21.15.60:0/2047675470 >> [v2:172.21.15.60:6833/2455562778,v1:172.21.15.60:6835/2455562778] conn(0x5558644a7160 0x5558644fcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 4 client.7803 mds_command: new command op to 7653 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:50.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbca514000 1 -- 172.21.15.60:0/2047675470 --> [v2:172.21.15.60:6833/2455562778,v1:172.21.15.60:6835/2455562778] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55586428ca50 con 0x5558644a7160 2024-04-25T10:11:50.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbc4942640 1 --2- 172.21.15.60:0/2047675470 >> [v2:172.21.15.60:6833/2455562778,v1:172.21.15.60:6835/2455562778] conn(0x5558644a7160 0x5558644fcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbb9ffff640 10 client.7803 ms_handle_connect on v2:172.21.15.60:6833/2455562778 2024-04-25T10:11:50.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbbc4942640 1 --2- 172.21.15.60:0/2047675470 >> [v2:172.21.15.60:6833/2455562778,v1:172.21.15.60:6835/2455562778] conn(0x5558644a7160 0x5558644fcce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6646c388c685101c server_cookie=647069f58eb8f0bc in_seq=0 out_seq=0 2024-04-25T10:11:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbb9ffff640 1 -- 172.21.15.60:0/2047675470 <== mds.0 v2:172.21.15.60:6833/2455562778 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55586428ca50 con 0x5558644a7160 2024-04-25T10:11:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.804+0000 7fbb9ffff640 10 client.7803 handle_command_reply: tid=0 2024-04-25T10:11:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.808+0000 7fbbca514000 1 client.7803 shutdown 2024-04-25T10:11:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.808+0000 7fbbca514000 1 -- 172.21.15.60:0/2047675470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644a1f30 msgr2=0x5558644f2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.808+0000 7fbbca514000 1 --2- 172.21.15.60:0/2047675470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644a1f30 0x5558644f2c80 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc001000 tx=0x7fbbc00b6950 comp rx=0 tx=0).stop 2024-04-25T10:11:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.808+0000 7fbbca514000 1 -- 172.21.15.60:0/2047675470 shutdown_connections 2024-04-25T10:11:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.808+0000 7fbbca514000 1 --2- 172.21.15.60:0/2047675470 >> [v2:172.21.15.60:6833/2455562778,v1:172.21.15.60:6835/2455562778] conn(0x5558644a7160 0x5558644fcce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.808+0000 7fbbca514000 1 --2- 172.21.15.60:0/2047675470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644ec240 0x5558644a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.808+0000 7fbbca514000 1 --2- 172.21.15.60:0/2047675470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644eb920 0x5558644f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.808+0000 7fbb9ffff640 0 client.7803 ms_handle_reset on v2:172.21.15.60:6833/2455562778 2024-04-25T10:11:50.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.808+0000 7fbbca514000 1 --2- 172.21.15.60:0/2047675470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644a1f30 0x5558644f2c80 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.808+0000 7fbbca514000 1 -- 172.21.15.60:0/2047675470 >> 172.21.15.60:0/2047675470 conn(0x5558644a82a0 msgr2=0x5558644a09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.808+0000 7fbbca514000 1 -- 172.21.15.60:0/2047675470 shutdown_connections 2024-04-25T10:11:50.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.808+0000 7fbbca514000 1 -- 172.21.15.60:0/2047675470 wait complete. 2024-04-25T10:11:50.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.808+0000 7fbbca514000 20 client.7803 trim_cache size 0 max 16384 2024-04-25T10:11:50.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.820+0000 7fbbca514000 1 Processor -- start 2024-04-25T10:11:50.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.820+0000 7fbbca514000 1 -- start start 2024-04-25T10:11:50.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644a1f30 0x555864547190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644eb920 0x5558645476d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644ec240 0x555864547c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc01ff7c0 con 0x5558644ec240 2024-04-25T10:11:50.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc01f7860 con 0x5558644a1f30 2024-04-25T10:11:50.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc01ff1c0 con 0x5558644eb920 2024-04-25T10:11:50.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc5944640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644ec240 0x555864547c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc5944640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644ec240 0x555864547c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46894/0 (socket says 172.21.15.60:46894) 2024-04-25T10:11:50.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc5944640 1 -- 172.21.15.60:0/1464028906 learned_addr learned my addr 172.21.15.60:0/1464028906 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:50.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc4942640 1 --2- 172.21.15.60:0/1464028906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644eb920 0x5558645476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc5944640 1 -- 172.21.15.60:0/1464028906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644eb920 msgr2=0x5558645476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc5143640 1 --2- 172.21.15.60:0/1464028906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644a1f30 0x555864547190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc5944640 1 --2- 172.21.15.60:0/1464028906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644eb920 0x5558645476d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc5944640 1 -- 172.21.15.60:0/1464028906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644a1f30 msgr2=0x555864547190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc5944640 1 --2- 172.21.15.60:0/1464028906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644a1f30 0x555864547190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc5944640 1 -- 172.21.15.60:0/1464028906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558644ec070 con 0x5558644ec240 2024-04-25T10:11:50.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc5944640 1 --2- 172.21.15.60:0/1464028906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644ec240 0x555864547c10 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8004b00 tx=0x7fbbb80561f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7ffe0367fbf6d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbaa7fc640 1 -- 172.21.15.60:0/1464028906 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb805ea00 con 0x5558644ec240 2024-04-25T10:11:50.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbaa7fc640 1 -- 172.21.15.60:0/1464028906 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb80545f0 con 0x5558644ec240 2024-04-25T10:11:50.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbaa7fc640 1 -- 172.21.15.60:0/1464028906 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb805a0c0 con 0x5558644ec240 2024-04-25T10:11:50.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 -- 172.21.15.60:0/1464028906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644ec240 msgr2=0x555864547c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 --2- 172.21.15.60:0/1464028906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644ec240 0x555864547c10 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8004b00 tx=0x7fbbb80561f0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 -- 172.21.15.60:0/1464028906 shutdown_connections 2024-04-25T10:11:50.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 --2- 172.21.15.60:0/1464028906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644ec240 0x555864547c10 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 --2- 172.21.15.60:0/1464028906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644eb920 0x5558645476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 --2- 172.21.15.60:0/1464028906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644a1f30 0x555864547190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 -- 172.21.15.60:0/1464028906 >> 172.21.15.60:0/1464028906 conn(0x5558644a82a0 msgr2=0x5558644ea0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 -- 172.21.15.60:0/1464028906 shutdown_connections 2024-04-25T10:11:50.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 -- 172.21.15.60:0/1464028906 wait complete. 2024-04-25T10:11:50.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 -- start start 2024-04-25T10:11:50.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 Processor -- start 2024-04-25T10:11:50.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644a1f30 0x55586454c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644eb920 0x55586454ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644ec240 0x555864533760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558642f53d0 con 0x5558644eb920 2024-04-25T10:11:50.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558643acd50 con 0x5558644ec240 2024-04-25T10:11:50.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55586428b980 con 0x5558644a1f30 2024-04-25T10:11:50.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbca514000 10 client.0 _ll_register_callbacks cb 0x7fbbbc001560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:50.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc4942640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644eb920 0x55586454ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc4942640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644eb920 0x55586454ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46900/0 (socket says 172.21.15.60:46900) 2024-04-25T10:11:50.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc4942640 1 -- 172.21.15.60:0/3598422184 learned_addr learned my addr 172.21.15.60:0/3598422184 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:50.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc5143640 1 --2- 172.21.15.60:0/3598422184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644a1f30 0x55586454c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc5944640 1 --2- 172.21.15.60:0/3598422184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644ec240 0x555864533760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc4942640 1 -- 172.21.15.60:0/3598422184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644a1f30 msgr2=0x55586454c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc4942640 1 --2- 172.21.15.60:0/3598422184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644a1f30 0x55586454c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc4942640 1 -- 172.21.15.60:0/3598422184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644ec240 msgr2=0x555864533760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc4942640 1 --2- 172.21.15.60:0/3598422184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644ec240 0x555864533760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.824+0000 7fbbc4942640 1 -- 172.21.15.60:0/3598422184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555864372600 con 0x5558644eb920 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbc4942640 1 --2- 172.21.15.60:0/3598422184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644eb920 0x55586454ccf0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00ad040 tx=0x7fbbb001ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4974177da39554b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbb9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbb9ffff640 1 -- 172.21.15.60:0/3598422184 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb00c3060 con 0x5558644eb920 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbb9ffff640 1 -- 172.21.15.60:0/3598422184 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbb00beb00 con 0x5558644eb920 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 1 -- 172.21.15.60:0/3598422184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbb80545f0 con 0x5558644eb920 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbb9ffff640 1 -- 172.21.15.60:0/3598422184 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbb00c9050 con 0x5558644eb920 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 10 client.7809 fetch_fsmap learned FSMap version 55 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 1 -- 172.21.15.60:0/3598422184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x5558642859b0 con 0x5558644eb920 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbb9ffff640 1 -- 172.21.15.60:0/3598422184 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb80545f0 con 0x5558644eb920 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbb9ffff640 1 -- 172.21.15.60:0/3598422184 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbbb00cc020 con 0x5558644eb920 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 10 client.7809 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 10 client.7809 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 20 client.7809 populate_metadata read hostname 'smithi060' 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 1 --2- 172.21.15.60:0/3598422184 >> [v2:172.21.15.60:6833/2455562778,v1:172.21.15.60:6835/2455562778] conn(0x555864533ca0 0x555864534080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 4 client.7809 mds_command: new command op to 7653 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 1 -- 172.21.15.60:0/3598422184 --> [v2:172.21.15.60:6833/2455562778,v1:172.21.15.60:6835/2455562778] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5558642fe2e0 con 0x555864533ca0 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbc5143640 1 --2- 172.21.15.60:0/3598422184 >> [v2:172.21.15.60:6833/2455562778,v1:172.21.15.60:6835/2455562778] conn(0x555864533ca0 0x555864534080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbc5143640 1 --2- 172.21.15.60:0/3598422184 >> [v2:172.21.15.60:6833/2455562778,v1:172.21.15.60:6835/2455562778] conn(0x555864533ca0 0x555864534080 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf878725d758e859 server_cookie=f6a8e303b9007a4e in_seq=0 out_seq=0 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbb9ffff640 10 client.7809 ms_handle_connect on v2:172.21.15.60:6833/2455562778 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbb9ffff640 1 -- 172.21.15.60:0/3598422184 <== mds.0 v2:172.21.15.60:6833/2455562778 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5558642fe2e0 con 0x555864533ca0 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbb9ffff640 10 client.7809 handle_command_reply: tid=0 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 1 client.7809 shutdown 2024-04-25T10:11:50.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 1 -- 172.21.15.60:0/3598422184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644eb920 msgr2=0x55586454ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 1 --2- 172.21.15.60:0/3598422184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644eb920 0x55586454ccf0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00ad040 tx=0x7fbbb001ad40 comp rx=0 tx=0).stop 2024-04-25T10:11:50.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 1 -- 172.21.15.60:0/3598422184 shutdown_connections 2024-04-25T10:11:50.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 1 --2- 172.21.15.60:0/3598422184 >> [v2:172.21.15.60:6833/2455562778,v1:172.21.15.60:6835/2455562778] conn(0x555864533ca0 0x555864534080 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 1 --2- 172.21.15.60:0/3598422184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558644ec240 0x555864533760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 1 --2- 172.21.15.60:0/3598422184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558644eb920 0x55586454ccf0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 1 --2- 172.21.15.60:0/3598422184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558644a1f30 0x55586454c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbb9ffff640 0 client.7809 ms_handle_reset on v2:172.21.15.60:6833/2455562778 2024-04-25T10:11:50.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 1 -- 172.21.15.60:0/3598422184 >> 172.21.15.60:0/3598422184 conn(0x5558644a82a0 msgr2=0x5558644ea7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 1 -- 172.21.15.60:0/3598422184 shutdown_connections 2024-04-25T10:11:50.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 1 -- 172.21.15.60:0/3598422184 wait complete. 2024-04-25T10:11:50.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbca514000 20 client.7809 trim_cache size 0 max 16384 2024-04-25T10:11:50.847 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:11:50.847 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "0ea93863-a079-45c4-abed-6e83540ea9fe", 2024-04-25T10:11:50.847 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T10:11:50.847 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7653, 2024-04-25T10:11:50.847 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T10:11:50.847 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T10:11:50.847 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T10:11:50.847 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 5.7837876279999998, 2024-04-25T10:11:50.847 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 55, 2024-04-25T10:11:50.847 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 82, 2024-04-25T10:11:50.847 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 82, 2024-04-25T10:11:50.847 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 10.84657878 2024-04-25T10:11:50.848 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:11:50.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbc73a7640 1 -- 172.21.15.60:0/813024357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbba4066f60 msgr2=0x7fbba4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbc73a7640 1 --2- 172.21.15.60:0/813024357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbba4066f60 0x7fbba4069420 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc001ec0 tx=0x7fbbbc009040 comp rx=0 tx=0).stop 2024-04-25T10:11:50.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbc73a7640 1 -- 172.21.15.60:0/813024357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00cd420 msgr2=0x7fbbc00cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbc73a7640 1 --2- 172.21.15.60:0/813024357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00cd420 0x7fbbc00cba10 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00b60b0 tx=0x7fbbb80115a0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbc73a7640 1 -- 172.21.15.60:0/813024357 shutdown_connections 2024-04-25T10:11:50.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbc73a7640 1 --2- 172.21.15.60:0/813024357 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbba4066f60 0x7fbba4069420 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbc73a7640 1 --2- 172.21.15.60:0/813024357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00cd420 0x7fbbc00cba10 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbc73a7640 1 --2- 172.21.15.60:0/813024357 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc0151290 0x7fbbc015adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbc73a7640 1 --2- 172.21.15.60:0/813024357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbc0150eb0 0x7fbbc00ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbc73a7640 1 -- 172.21.15.60:0/813024357 >> 172.21.15.60:0/813024357 conn(0x7fbbc00d05c0 msgr2=0x7fbbc00b7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbc73a7640 1 -- 172.21.15.60:0/813024357 shutdown_connections 2024-04-25T10:11:50.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.828+0000 7fbbc73a7640 1 -- 172.21.15.60:0/813024357 wait complete. 2024-04-25T10:11:50.853 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:11:50.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.956+0000 7f876a3ec640 1 Processor -- start 2024-04-25T10:11:50.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.956+0000 7f876a3ec640 1 -- start start 2024-04-25T10:11:50.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f876a3ec640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87640ce7a0 0x7f87640ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f876a3ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87640b33f0 con 0x7f876415bc90 2024-04-25T10:11:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f876a3ec640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87640cf0c0 con 0x7f87640b4660 2024-04-25T10:11:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f876a3ec640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87640cc370 con 0x7f87640ce7a0 2024-04-25T10:11:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f87637fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f876415bc90 0x7f87640b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54790/0 (socket says 172.21.15.60:54790) 2024-04-25T10:11:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f87637fe640 1 -- 172.21.15.60:0/1763736578 learned_addr learned my addr 172.21.15.60:0/1763736578 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8763fff640 1 --2- 172.21.15.60:0/1763736578 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87640ce7a0 0x7f87640ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8762ffd640 1 -- 172.21.15.60:0/1763736578 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453502952 0 0) 0x7f87640b33f0 con 0x7f876415bc90 2024-04-25T10:11:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8762ffd640 1 -- 172.21.15.60:0/1763736578 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f874c003530 con 0x7f876415bc90 2024-04-25T10:11:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8762ffd640 1 -- 172.21.15.60:0/1763736578 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2965732042 0 0) 0x7f874c003530 con 0x7f876415bc90 2024-04-25T10:11:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8762ffd640 1 -- 172.21.15.60:0/1763736578 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87640ce7a0 msgr2=0x7f87640ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8762ffd640 1 --2- 172.21.15.60:0/1763736578 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87640ce7a0 0x7f87640ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8762ffd640 1 -- 172.21.15.60:0/1763736578 >> v1:172.21.15.100:6789/0 conn(0x7f87640b4660 legacy=0x7f87640b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8762ffd640 1 -- 172.21.15.60:0/1763736578 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8764071ca0 con 0x7f876415bc90 2024-04-25T10:11:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8762ffd640 1 -- 172.21.15.60:0/1763736578 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f8758001680 con 0x7f876415bc90 2024-04-25T10:11:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8762ffd640 1 -- 172.21.15.60:0/1763736578 >> v1:172.21.15.60:6789/0 conn(0x7f876415bc90 legacy=0x7f87640b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8762ffd640 1 --2- 172.21.15.60:0/1763736578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f874c0039f0 0x7f874c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8762ffd640 1 --2- 172.21.15.60:0/1763736578 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87640ce7a0 0x7f874c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8762ffd640 1 --2- 172.21.15.60:0/1763736578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f874c004d50 0x7f874c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8762ffd640 1 -- 172.21.15.60:0/1763736578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c00ba80 con 0x7f874c004d50 2024-04-25T10:11:50.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8762ffd640 1 -- 172.21.15.60:0/1763736578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c00bc00 con 0x7f874c0039f0 2024-04-25T10:11:50.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8762ffd640 1 -- 172.21.15.60:0/1763736578 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c00be00 con 0x7f87640ce7a0 2024-04-25T10:11:50.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8768989640 1 --2- 172.21.15.60:0/1763736578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f874c004d50 0x7f874c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8763fff640 1 --2- 172.21.15.60:0/1763736578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f874c0039f0 0x7f874c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f87637fe640 1 --2- 172.21.15.60:0/1763736578 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87640ce7a0 0x7f874c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8768989640 1 -- 172.21.15.60:0/1763736578 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87640ce7a0 msgr2=0x7f874c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8768989640 1 --2- 172.21.15.60:0/1763736578 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87640ce7a0 0x7f874c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8768989640 1 -- 172.21.15.60:0/1763736578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f874c0039f0 msgr2=0x7f874c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8768989640 1 --2- 172.21.15.60:0/1763736578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f874c0039f0 0x7f874c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.960+0000 7f8768989640 1 -- 172.21.15.60:0/1763736578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c003530 con 0x7f874c004d50 2024-04-25T10:11:50.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f8768989640 1 --2- 172.21.15.60:0/1763736578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f874c004d50 0x7f874c0051d0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f8764072c10 tx=0x7f875c01c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ab7b4eafad95ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f8762ffd640 1 -- 172.21.15.60:0/1763736578 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c001a90 con 0x7f874c004d50 2024-04-25T10:11:50.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f8762ffd640 1 -- 172.21.15.60:0/1763736578 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c003b80 con 0x7f874c004d50 2024-04-25T10:11:50.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f8762ffd640 1 -- 172.21.15.60:0/1763736578 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c0050d0 con 0x7f874c004d50 2024-04-25T10:11:50.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f876a3ec640 1 -- 172.21.15.60:0/1763736578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f874c004d50 msgr2=0x7f874c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f876a3ec640 1 --2- 172.21.15.60:0/1763736578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f874c004d50 0x7f874c0051d0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f8764072c10 tx=0x7f875c01c700 comp rx=0 tx=0).stop 2024-04-25T10:11:50.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f8768989640 1 -- 172.21.15.60:0/1763736578 reap_dead start 2024-04-25T10:11:50.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f876a3ec640 1 -- 172.21.15.60:0/1763736578 shutdown_connections 2024-04-25T10:11:50.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f876a3ec640 1 -- 172.21.15.60:0/1763736578 >> 172.21.15.60:0/1763736578 conn(0x7f87640b12d0 msgr2=0x7f87640b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f876a3ec640 1 -- 172.21.15.60:0/1763736578 shutdown_connections 2024-04-25T10:11:50.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f876a3ec640 1 -- 172.21.15.60:0/1763736578 wait complete. 2024-04-25T10:11:50.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f876a3ec640 1 Processor -- start 2024-04-25T10:11:50.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f876a3ec640 1 -- start start 2024-04-25T10:11:50.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f876a3ec640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8764147fa0 0x7f87641536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f876a3ec640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87641586a0 0x7f8764158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f876a3ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87640af390 0x7f87640af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f876a3ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8764071f80 con 0x7f87640af390 2024-04-25T10:11:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f876a3ec640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640ccd20 con 0x7f87641586a0 2024-04-25T10:11:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f876a3ec640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87640b3ba0 con 0x7f8764147fa0 2024-04-25T10:11:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f8768989640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87640af390 0x7f87640af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f8768989640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87640af390 0x7f87640af810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46918/0 (socket says 172.21.15.60:46918) 2024-04-25T10:11:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f8768989640 1 -- 172.21.15.60:0/1617534986 learned_addr learned my addr 172.21.15.60:0/1617534986 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f8763fff640 1 --2- 172.21.15.60:0/1617534986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8764147fa0 0x7f87641536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f87637fe640 1 --2- 172.21.15.60:0/1617534986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87641586a0 0x7f8764158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f8768989640 1 -- 172.21.15.60:0/1617534986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8764147fa0 msgr2=0x7f87641536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f8768989640 1 --2- 172.21.15.60:0/1617534986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8764147fa0 0x7f87641536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f8768989640 1 -- 172.21.15.60:0/1617534986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87641586a0 msgr2=0x7f8764158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f8768989640 1 --2- 172.21.15.60:0/1617534986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87641586a0 0x7f8764158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f8768989640 1 -- 172.21.15.60:0/1617534986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8764153bf0 con 0x7f87640af390 2024-04-25T10:11:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.964+0000 7f8763fff640 1 --2- 172.21.15.60:0/1617534986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8764147fa0 0x7f87641536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.968+0000 7f8768989640 1 --2- 172.21.15.60:0/1617534986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87640af390 0x7f87640af810 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f875c01cc70 tx=0x7f874c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.968+0000 7f8760ff9640 1 -- 172.21.15.60:0/1617534986 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c001a90 con 0x7f87640af390 2024-04-25T10:11:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.968+0000 7f8760ff9640 1 -- 172.21.15.60:0/1617534986 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c0012e0 con 0x7f87640af390 2024-04-25T10:11:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.968+0000 7f876a3ec640 1 -- 172.21.15.60:0/1617534986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8764159160 con 0x7f87640af390 2024-04-25T10:11:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.968+0000 7f8760ff9640 1 -- 172.21.15.60:0/1617534986 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c007dc0 con 0x7f87640af390 2024-04-25T10:11:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.968+0000 7f876a3ec640 1 -- 172.21.15.60:0/1617534986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8764152bd0 con 0x7f87640af390 2024-04-25T10:11:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.968+0000 7f876a3ec640 1 -- 172.21.15.60:0/1617534986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8764074770 con 0x7f87640af390 2024-04-25T10:11:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.968+0000 7f8760ff9640 1 -- 172.21.15.60:0/1617534986 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f875c0a8680 con 0x7f87640af390 2024-04-25T10:11:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.968+0000 7f8760ff9640 1 --2- 172.21.15.60:0/1617534986 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8750066f60 0x7f8750069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.968+0000 7f8760ff9640 1 -- 172.21.15.60:0/1617534986 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f875c11ee10 con 0x7f87640af390 2024-04-25T10:11:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.968+0000 7f8763fff640 1 --2- 172.21.15.60:0/1617534986 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8750066f60 0x7f8750069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.968+0000 7f8763fff640 1 --2- 172.21.15.60:0/1617534986 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8750066f60 0x7f8750069420 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8754009af0 tx=0x7f8754008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:50.972+0000 7f8760ff9640 1 -- 172.21.15.60:0/1617534986 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f875c122030 con 0x7f87640af390 2024-04-25T10:11:51.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:51.164+0000 7f876a3ec640 1 -- 172.21.15.60:0/1617534986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f87640a8ed0 con 0x7f87640af390 2024-04-25T10:11:51.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:51.164+0000 7f8760ff9640 1 -- 172.21.15.60:0/1617534986 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) v1 ==== 53+0+73 (secure 0 0 0) 0x7f875c0be070 con 0x7f87640af390 2024-04-25T10:11:51.174 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:11:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:51.168+0000 7f876a3ec640 1 -- 172.21.15.60:0/1617534986 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8750066f60 msgr2=0x7f8750069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:51.168+0000 7f876a3ec640 1 --2- 172.21.15.60:0/1617534986 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8750066f60 0x7f8750069420 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8754009af0 tx=0x7f8754008040 comp rx=0 tx=0).stop 2024-04-25T10:11:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:51.168+0000 7f876a3ec640 1 -- 172.21.15.60:0/1617534986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87640af390 msgr2=0x7f87640af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:51.168+0000 7f876a3ec640 1 --2- 172.21.15.60:0/1617534986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87640af390 0x7f87640af810 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f875c01cc70 tx=0x7f874c005840 comp rx=0 tx=0).stop 2024-04-25T10:11:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:51.168+0000 7f876a3ec640 1 -- 172.21.15.60:0/1617534986 shutdown_connections 2024-04-25T10:11:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:51.168+0000 7f876a3ec640 1 --2- 172.21.15.60:0/1617534986 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8750066f60 0x7f8750069420 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:51.168+0000 7f876a3ec640 1 --2- 172.21.15.60:0/1617534986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87640af390 0x7f87640af810 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:51.168+0000 7f876a3ec640 1 --2- 172.21.15.60:0/1617534986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87641586a0 0x7f8764158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:51.168+0000 7f876a3ec640 1 --2- 172.21.15.60:0/1617534986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8764147fa0 0x7f87641536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:51.168+0000 7f876a3ec640 1 -- 172.21.15.60:0/1617534986 >> 172.21.15.60:0/1617534986 conn(0x7f87640b12d0 msgr2=0x7f87640a85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:51.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:51.168+0000 7f876a3ec640 1 -- 172.21.15.60:0/1617534986 shutdown_connections 2024-04-25T10:11:51.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:51.168+0000 7f876a3ec640 1 -- 172.21.15.60:0/1617534986 wait complete. 2024-04-25T10:11:51.191 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:11:51.191 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:11:51.191 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:11:51.191 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:11:51.191 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:51.191 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:11:51.191 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:11:51.191 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:11:51.191 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:51.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:51.192 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:11:51.197 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:11:51.197 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:11:51.197 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:11:51.197 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:51.197 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:11:51.198 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:51.198 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:11:51.198 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:51.198 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:11:51.198 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:11:51.198 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:51.198 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:11:51.198 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:11:51.198 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:51.198 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:11:51.198 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:11:51.198 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42146sec preferred_lft 42146sec 2024-04-25T10:11:51.198 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:11:51.199 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:51.199 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:11:51.199 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:51.199 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:11:51.199 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:11:51.199 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:51.199 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:11:51.199 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:51.199 INFO:teuthology.orchestra.run.smithi060.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:11:51.199 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:51.200 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:11:51.200 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:51.200 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:11:51.200 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:51.200 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:51.200 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:51.200 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:11:51.200 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:51.200 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:51.200 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:51.200 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:11:51.200 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:51.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:51.201 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:11:51.253 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:11:51.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:51.255 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:11:51.308 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:11:51.308 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:11:51.309 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:11:51.309 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:11:51.309 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:11:51.309 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:11:51.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:51.309 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:11:51.309 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:11:51.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:11:51.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:11:51.310 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:11:51.310 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:11:51.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:51.387 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:11:51.436 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:11:51.436 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:11:51.436 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:51.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:51.437 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T10:11:51.492 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:11:51.492 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:51.492 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:51.544 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:51.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:51.544 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:51.592 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:11:51.649 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:11:51.697 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:11:51.697 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:11:51.697 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:11:51.697 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:11:51.697 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:11:51.697 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:11:51.697 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:11:51.697 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:11:51.697 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:11:51.697 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:11:51.697 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:11:51.697 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:11:51.698 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:11:51.698 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:11:51.698 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:11:51.698 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:11:51.698 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:11:51.698 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:11:51.698 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:11:51.698 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:11:51.698 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:11:51.698 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:11:51.698 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:11:51.698 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:11:51.698 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:11:51.698 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:11:51.699 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:11:51.699 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:11:51.699 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:11:51.699 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:51.699 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:51.699 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:51.699 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:51.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:51.700 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:11:51.748 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:11:51.748 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:11:51.793 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:11:51.807 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:11:51.859 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:11:51.859 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:11:51.859 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:11:51.859 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:11:51.859 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:11:51.859 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:11:51.859 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:11:51.860 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:11:51.860 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:11:51.860 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:11:51.860 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:11:51.860 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:11:51.860 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:11:51.860 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:11:51.860 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:11:51.860 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:11:51.860 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:11:51.860 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:11:51.861 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:11:51.861 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:11:51.861 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:11:51.861 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:11:51.861 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:11:51.861 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:11:51.861 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:11:51.861 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:11:51.861 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:11:51.861 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:11:51.861 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:11:51.861 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:51.862 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:51.862 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:51.862 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:51.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:51.862 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:11:51.877 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2024-04-25T10:11:51.868+0000 7f71ec45e580 -1 init, newargv = 0x55f1a240cdd0 newargc=15 2024-04-25T10:11:51.877 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[45379]: starting ceph client 2024-04-25T10:11:51.883 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[45379]: starting fuse 2024-04-25T10:11:52.983 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:11:53.000 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:11:53.049 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:11:53.049 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:11:53.049 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:11:53.049 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:11:53.049 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:11:53.049 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:11:53.049 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:11:53.049 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:11:53.049 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:11:53.049 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:11:53.049 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:11:53.049 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:11:53.050 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:11:53.050 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:11:53.050 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:11:53.050 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:11:53.050 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:11:53.050 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:11:53.050 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:11:53.050 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:11:53.050 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:11:53.050 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:11:53.050 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:11:53.050 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:11:53.050 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:11:53.051 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:11:53.051 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:11:53.051 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:11:53.051 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:11:53.051 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:53.051 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:53.051 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:53.051 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:53.051 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:53.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:53.052 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:11:53.100 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:11:53.100 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:11:53.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:11:53.100 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:11:53.101 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:11:53.101 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:11:53.101 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:11:53.101 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:11:53.101 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:11:53.101 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:11:53.101 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:11:53.101 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:11:53.101 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:11:53.101 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:11:53.101 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:11:53.101 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:11:53.101 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:11:53.101 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:11:53.101 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:11:53.101 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:11:53.102 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:11:53.102 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:11:53.102 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:11:53.102 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:11:53.102 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:11:53.102 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:11:53.102 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:11:53.102 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2024-04-25T10:11:53.102 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:11:53.227 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.45379.asok 2024-04-25T10:11:53.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:11:53.239 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45379.asok 2024-04-25T10:11:53.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:53.240 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45379.asok status 2024-04-25T10:11:53.400 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:11:53.400 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:11:53.400 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:11:53.400 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:11:53.400 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2024-04-25T10:11:53.400 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:11:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:11:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "45379", 2024-04-25T10:11:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:11:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:11:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:11:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7824, 2024-04-25T10:11:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:11:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:11:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:11:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 7824 2024-04-25T10:11:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:11:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:11:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:11:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 737827456 2024-04-25T10:11:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:11:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:11:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:11:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:11:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 737827456 2024-04-25T10:11:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:11:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.7824 192.168.0.1:0/737827456", 2024-04-25T10:11:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/737827456", 2024-04-25T10:11:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:11:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 55, 2024-04-25T10:11:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 82, 2024-04-25T10:11:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:11:53.403 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:11:53.403 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:11:53.403 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:11:53.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:53.414 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:11:53.414 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:53.420 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:11:53.421 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:53.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:53.421 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:53.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:53.476 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-04-25T10:11:53.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:53.536 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-04-25T10:11:53.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:53.591 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-25T10:11:53.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:53.644 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-04-25T10:11:53.702 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:11:53.702 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:11:53.702 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:11:53.702 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:11:53.702 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-04-25T10:11:53.702 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:11:53.806 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627776 2024-04-25T10:11:53.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:11:53.817 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:11:53.817 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:53.864 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:54.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.032+0000 7fc95685a640 1 Processor -- start 2024-04-25T10:11:54.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.032+0000 7fc95685a640 1 -- start start 2024-04-25T10:11:54.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.032+0000 7fc95685a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9500b3ef0 0x7fc9500cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.032+0000 7fc95685a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95006f130 con 0x7fc95006ec30 2024-04-25T10:11:54.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.032+0000 7fc95685a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95006f410 con 0x7fc95015a7a0 2024-04-25T10:11:54.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.032+0000 7fc95685a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500cdaa0 con 0x7fc9500b3ef0 2024-04-25T10:11:54.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.032+0000 7fc94f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc95006ec30 0x7fc9500b37e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54800/0 (socket says 172.21.15.60:54800) 2024-04-25T10:11:54.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94f7fe640 1 -- 172.21.15.60:0/1659435354 learned_addr learned my addr 172.21.15.60:0/1659435354 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:54.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc954df7640 1 --2- 172.21.15.60:0/1659435354 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9500b3ef0 0x7fc9500cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94effd640 1 -- 172.21.15.60:0/1659435354 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700420124 0 0) 0x7fc95006f130 con 0x7fc95006ec30 2024-04-25T10:11:54.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94effd640 1 -- 172.21.15.60:0/1659435354 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc938003530 con 0x7fc95006ec30 2024-04-25T10:11:54.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94effd640 1 -- 172.21.15.60:0/1659435354 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1546860353 0 0) 0x7fc938003530 con 0x7fc95006ec30 2024-04-25T10:11:54.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94effd640 1 -- 172.21.15.60:0/1659435354 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9500b3ef0 msgr2=0x7fc9500cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:54.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94effd640 1 --2- 172.21.15.60:0/1659435354 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9500b3ef0 0x7fc9500cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94effd640 1 -- 172.21.15.60:0/1659435354 >> v1:172.21.15.100:6789/0 conn(0x7fc95015a7a0 legacy=0x7fc950071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94effd640 1 -- 172.21.15.60:0/1659435354 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95006d540 con 0x7fc95006ec30 2024-04-25T10:11:54.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94effd640 1 -- 172.21.15.60:0/1659435354 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fc944001680 con 0x7fc95006ec30 2024-04-25T10:11:54.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94effd640 1 -- 172.21.15.60:0/1659435354 >> v1:172.21.15.60:6789/0 conn(0x7fc95006ec30 legacy=0x7fc9500b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94effd640 1 --2- 172.21.15.60:0/1659435354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9380039f0 0x7fc938003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94effd640 1 --2- 172.21.15.60:0/1659435354 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9500b3ef0 0x7fc938004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94effd640 1 --2- 172.21.15.60:0/1659435354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc938004d50 0x7fc9380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94effd640 1 -- 172.21.15.60:0/1659435354 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93800ba80 con 0x7fc938004d50 2024-04-25T10:11:54.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94effd640 1 -- 172.21.15.60:0/1659435354 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93800bc30 con 0x7fc9380039f0 2024-04-25T10:11:54.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94effd640 1 -- 172.21.15.60:0/1659435354 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93800be30 con 0x7fc9500b3ef0 2024-04-25T10:11:54.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc954df7640 1 --2- 172.21.15.60:0/1659435354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc938004d50 0x7fc9380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94ffff640 1 --2- 172.21.15.60:0/1659435354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9380039f0 0x7fc938003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94f7fe640 1 --2- 172.21.15.60:0/1659435354 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9500b3ef0 0x7fc938004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc954df7640 1 -- 172.21.15.60:0/1659435354 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9500b3ef0 msgr2=0x7fc938004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc954df7640 1 --2- 172.21.15.60:0/1659435354 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9500b3ef0 0x7fc938004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc954df7640 1 -- 172.21.15.60:0/1659435354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9380039f0 msgr2=0x7fc938003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc954df7640 1 --2- 172.21.15.60:0/1659435354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9380039f0 0x7fc938003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc954df7640 1 -- 172.21.15.60:0/1659435354 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc938003530 con 0x7fc938004d50 2024-04-25T10:11:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94f7fe640 1 --2- 172.21.15.60:0/1659435354 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9500b3ef0 0x7fc938004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.036+0000 7fc94ffff640 1 --2- 172.21.15.60:0/1659435354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9380039f0 0x7fc938003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:54.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc954df7640 1 --2- 172.21.15.60:0/1659435354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc938004d50 0x7fc9380051d0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc950072b70 tx=0x7fc94801eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb690e8ecf0e559 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc94effd640 1 -- 172.21.15.60:0/1659435354 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9480b4070 con 0x7fc938004d50 2024-04-25T10:11:54.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc94effd640 1 -- 172.21.15.60:0/1659435354 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9480a77a0 con 0x7fc938004d50 2024-04-25T10:11:54.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc94effd640 1 -- 172.21.15.60:0/1659435354 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948009690 con 0x7fc938004d50 2024-04-25T10:11:54.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc95685a640 1 -- 172.21.15.60:0/1659435354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc938004d50 msgr2=0x7fc9380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc95685a640 1 --2- 172.21.15.60:0/1659435354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc938004d50 0x7fc9380051d0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc950072b70 tx=0x7fc94801eba0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc954df7640 1 -- 172.21.15.60:0/1659435354 reap_dead start 2024-04-25T10:11:54.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc95685a640 1 -- 172.21.15.60:0/1659435354 shutdown_connections 2024-04-25T10:11:54.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc95685a640 1 -- 172.21.15.60:0/1659435354 >> 172.21.15.60:0/1659435354 conn(0x7fc9500b12d0 msgr2=0x7fc9500b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:54.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc95685a640 1 -- 172.21.15.60:0/1659435354 shutdown_connections 2024-04-25T10:11:54.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc95685a640 1 -- 172.21.15.60:0/1659435354 wait complete. 2024-04-25T10:11:54.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc95685a640 1 Processor -- start 2024-04-25T10:11:54.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc95685a640 1 -- start start 2024-04-25T10:11:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc95685a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc938004d50 0x7fc95014e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc95685a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9500cb230 0x7fc9500b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc95685a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc950154110 0x7fc9500b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc95685a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95006d8b0 con 0x7fc938004d50 2024-04-25T10:11:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc95685a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95006d2f0 con 0x7fc9500cb230 2024-04-25T10:11:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc95685a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500cd560 con 0x7fc950154110 2024-04-25T10:11:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc94ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc938004d50 0x7fc95014e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc94ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc938004d50 0x7fc95014e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46930/0 (socket says 172.21.15.60:46930) 2024-04-25T10:11:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc94ffff640 1 -- 172.21.15.60:0/1690857951 learned_addr learned my addr 172.21.15.60:0/1690857951 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc954df7640 1 --2- 172.21.15.60:0/1690857951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc950154110 0x7fc9500b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc94f7fe640 1 --2- 172.21.15.60:0/1690857951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9500cb230 0x7fc9500b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc94ffff640 1 -- 172.21.15.60:0/1690857951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc950154110 msgr2=0x7fc9500b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc94ffff640 1 --2- 172.21.15.60:0/1690857951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc950154110 0x7fc9500b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc94ffff640 1 -- 172.21.15.60:0/1690857951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9500cb230 msgr2=0x7fc9500b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc94ffff640 1 --2- 172.21.15.60:0/1690857951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9500cb230 0x7fc9500b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc94ffff640 1 -- 172.21.15.60:0/1690857951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9500ca190 con 0x7fc938004d50 2024-04-25T10:11:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.040+0000 7fc94ffff640 1 --2- 172.21.15.60:0/1690857951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc938004d50 0x7fc95014e100 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fc94000e8f0 tx=0x7fc94000edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.044+0000 7fc94cff9640 1 -- 172.21.15.60:0/1690857951 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940003b90 con 0x7fc938004d50 2024-04-25T10:11:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.044+0000 7fc94cff9640 1 -- 172.21.15.60:0/1690857951 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc940018070 con 0x7fc938004d50 2024-04-25T10:11:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.044+0000 7fc94cff9640 1 -- 172.21.15.60:0/1690857951 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9400050d0 con 0x7fc938004d50 2024-04-25T10:11:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.044+0000 7fc95685a640 1 -- 172.21.15.60:0/1690857951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9500ca470 con 0x7fc938004d50 2024-04-25T10:11:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.044+0000 7fc95685a640 1 -- 172.21.15.60:0/1690857951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9500caa30 con 0x7fc938004d50 2024-04-25T10:11:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.044+0000 7fc95685a640 1 -- 172.21.15.60:0/1690857951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc950074770 con 0x7fc938004d50 2024-04-25T10:11:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.044+0000 7fc94cff9640 1 -- 172.21.15.60:0/1690857951 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc940005270 con 0x7fc938004d50 2024-04-25T10:11:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.044+0000 7fc94cff9640 1 --2- 172.21.15.60:0/1690857951 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc93c066f60 0x7fc93c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.044+0000 7fc94cff9640 1 -- 172.21.15.60:0/1690857951 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc940085090 con 0x7fc938004d50 2024-04-25T10:11:54.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.044+0000 7fc94f7fe640 1 --2- 172.21.15.60:0/1690857951 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc93c066f60 0x7fc93c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.044+0000 7fc94f7fe640 1 --2- 172.21.15.60:0/1690857951 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc93c066f60 0x7fc93c069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc944001ec0 tx=0x7fc944009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.048+0000 7fc94cff9640 1 -- 172.21.15.60:0/1690857951 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc940056000 con 0x7fc938004d50 2024-04-25T10:11:54.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.208+0000 7fc95685a640 1 -- 172.21.15.60:0/1690857951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc950155790 con 0x7fc938004d50 2024-04-25T10:11:54.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.212+0000 7fc94cff9640 1 -- 172.21.15.60:0/1690857951 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16672 (secure 0 0 0) 0x7fc940055900 con 0x7fc938004d50 2024-04-25T10:11:54.220 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:54.220 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:11:46.035912+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:42.621502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:43.352679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.212+0000 7fc95685a640 1 -- 172.21.15.60:0/1690857951 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc93c066f60 msgr2=0x7fc93c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.212+0000 7fc95685a640 1 --2- 172.21.15.60:0/1690857951 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc93c066f60 0x7fc93c069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc944001ec0 tx=0x7fc944009040 comp rx=0 tx=0).stop 2024-04-25T10:11:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.212+0000 7fc95685a640 1 -- 172.21.15.60:0/1690857951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc938004d50 msgr2=0x7fc95014e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.212+0000 7fc95685a640 1 --2- 172.21.15.60:0/1690857951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc938004d50 0x7fc95014e100 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fc94000e8f0 tx=0x7fc94000edb0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.212+0000 7fc95685a640 1 -- 172.21.15.60:0/1690857951 shutdown_connections 2024-04-25T10:11:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.212+0000 7fc95685a640 1 --2- 172.21.15.60:0/1690857951 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc93c066f60 0x7fc93c069420 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.212+0000 7fc95685a640 1 --2- 172.21.15.60:0/1690857951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc950154110 0x7fc9500b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.212+0000 7fc95685a640 1 --2- 172.21.15.60:0/1690857951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9500cb230 0x7fc9500b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.212+0000 7fc95685a640 1 --2- 172.21.15.60:0/1690857951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc938004d50 0x7fc95014e100 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.212+0000 7fc95685a640 1 -- 172.21.15.60:0/1690857951 >> 172.21.15.60:0/1690857951 conn(0x7fc9500b12d0 msgr2=0x7fc9500cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:54.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.212+0000 7fc95685a640 1 -- 172.21.15.60:0/1690857951 shutdown_connections 2024-04-25T10:11:54.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.212+0000 7fc95685a640 1 -- 172.21.15.60:0/1690857951 wait complete. 2024-04-25T10:11:54.239 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:42.621502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:43.352679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:54.240 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:54.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f49085b3640 1 Processor -- start 2024-04-25T10:11:54.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f49085b3640 1 -- start start 2024-04-25T10:11:54.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f49085b3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f49000b3f60 0x7f490015a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f49085b3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4900071250 0x7f49000716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f49085b3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49000b3510 con 0x7f490006f030 2024-04-25T10:11:54.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f49085b3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490015ad30 con 0x7f49000b3f60 2024-04-25T10:11:54.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f49085b3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490015b010 con 0x7f4900071250 2024-04-25T10:11:54.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f4905b4e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f490006f030 0x7f49000b2e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54808/0 (socket says 172.21.15.60:54808) 2024-04-25T10:11:54.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f4905b4e640 1 -- 172.21.15.60:0/2034571817 learned_addr learned my addr 172.21.15.60:0/2034571817 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:54.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f490634f640 1 --2- 172.21.15.60:0/2034571817 >> v2:172.21.15.60:3300/0 conn(0x7f49000b3f60 0x7f490015a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f4906b50640 1 --2- 172.21.15.60:0/2034571817 >> v2:172.21.15.100:3300/0 conn(0x7f4900071250 0x7f49000716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f490534d640 1 -- 172.21.15.60:0/2034571817 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650127767 0 0) 0x7f49000b3510 con 0x7f490006f030 2024-04-25T10:11:54.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f490534d640 1 -- 172.21.15.60:0/2034571817 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48ec005740 con 0x7f490006f030 2024-04-25T10:11:54.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f490634f640 1 -- 172.21.15.60:0/2034571817 >> v2:172.21.15.100:3300/0 conn(0x7f4900071250 msgr2=0x7f49000716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:54.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f490634f640 1 --2- 172.21.15.60:0/2034571817 >> v2:172.21.15.100:3300/0 conn(0x7f4900071250 0x7f49000716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f490634f640 1 -- 172.21.15.60:0/2034571817 >> v1:172.21.15.60:6789/0 conn(0x7f490006f030 legacy=0x7f49000b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.364+0000 7f490634f640 1 -- 172.21.15.60:0/2034571817 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490015b2f0 con 0x7f49000b3f60 2024-04-25T10:11:54.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.368+0000 7f490634f640 1 --2- 172.21.15.60:0/2034571817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49000b3f60 0x7f490015a3e0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f49000cc3d0 tx=0x7f48fc01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c1fbc225e8c902 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.368+0000 7f490534d640 1 -- 172.21.15.60:0/2034571817 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc0a6510 con 0x7f49000b3f60 2024-04-25T10:11:54.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.368+0000 7f490534d640 1 -- 172.21.15.60:0/2034571817 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc0a66b0 con 0x7f49000b3f60 2024-04-25T10:11:54.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.368+0000 7f490534d640 1 -- 172.21.15.60:0/2034571817 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc0a6a60 con 0x7f49000b3f60 2024-04-25T10:11:54.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.368+0000 7f49085b3640 1 -- 172.21.15.60:0/2034571817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49000b3f60 msgr2=0x7f490015a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.368+0000 7f49085b3640 1 --2- 172.21.15.60:0/2034571817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49000b3f60 0x7f490015a3e0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f49000cc3d0 tx=0x7f48fc01c8c0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.368+0000 7f49085b3640 1 -- 172.21.15.60:0/2034571817 shutdown_connections 2024-04-25T10:11:54.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.368+0000 7f49085b3640 1 --2- 172.21.15.60:0/2034571817 >> v2:172.21.15.100:3300/0 conn(0x7f4900071250 0x7f49000716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.368+0000 7f49085b3640 1 --2- 172.21.15.60:0/2034571817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49000b3f60 0x7f490015a3e0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.368+0000 7f49085b3640 1 -- 172.21.15.60:0/2034571817 >> 172.21.15.60:0/2034571817 conn(0x7f49000670b0 msgr2=0x7f490006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:54.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.368+0000 7f49085b3640 1 -- 172.21.15.60:0/2034571817 shutdown_connections 2024-04-25T10:11:54.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.368+0000 7f49085b3640 1 -- 172.21.15.60:0/2034571817 wait complete. 2024-04-25T10:11:54.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.368+0000 7f49085b3640 1 Processor -- start 2024-04-25T10:11:54.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.368+0000 7f49085b3640 1 -- start start 2024-04-25T10:11:54.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f49085b3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f490006f030 0x7f49001457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f49085b3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4900071250 0x7f4900144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f49085b3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49000b3f60 0x7f49001449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f49085b3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490015b720 con 0x7f490006f030 2024-04-25T10:11:54.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f49085b3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490015b5a0 con 0x7f4900071250 2024-04-25T10:11:54.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f49085b3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490015b8a0 con 0x7f49000b3f60 2024-04-25T10:11:54.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f490634f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f490006f030 0x7f49001457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f490634f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f490006f030 0x7f49001457b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46942/0 (socket says 172.21.15.60:46942) 2024-04-25T10:11:54.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f490634f640 1 -- 172.21.15.60:0/3725335814 learned_addr learned my addr 172.21.15.60:0/3725335814 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:54.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f4905b4e640 1 --2- 172.21.15.60:0/3725335814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4900071250 0x7f4900144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f4906b50640 1 --2- 172.21.15.60:0/3725335814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49000b3f60 0x7f49001449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f490634f640 1 -- 172.21.15.60:0/3725335814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49000b3f60 msgr2=0x7f49001449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f490634f640 1 --2- 172.21.15.60:0/3725335814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49000b3f60 0x7f49001449c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f490634f640 1 -- 172.21.15.60:0/3725335814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4900071250 msgr2=0x7f4900144480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f490634f640 1 --2- 172.21.15.60:0/3725335814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4900071250 0x7f4900144480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f490634f640 1 -- 172.21.15.60:0/3725335814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490014df40 con 0x7f490006f030 2024-04-25T10:11:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f4905b4e640 1 --2- 172.21.15.60:0/3725335814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4900071250 0x7f4900144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f4906b50640 1 --2- 172.21.15.60:0/3725335814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49000b3f60 0x7f49001449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f490634f640 1 --2- 172.21.15.60:0/3725335814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f490006f030 0x7f49001457b0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f48fc01a4f0 tx=0x7f48fc019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f48f6ffd640 1 -- 172.21.15.60:0/3725335814 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc003bb0 con 0x7f490006f030 2024-04-25T10:11:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f48f6ffd640 1 -- 172.21.15.60:0/3725335814 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc003d50 con 0x7f490006f030 2024-04-25T10:11:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f49085b3640 1 -- 172.21.15.60:0/3725335814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490014e220 con 0x7f490006f030 2024-04-25T10:11:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f48f6ffd640 1 -- 172.21.15.60:0/3725335814 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc0a6a60 con 0x7f490006f030 2024-04-25T10:11:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f49085b3640 1 -- 172.21.15.60:0/3725335814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4900154340 con 0x7f490006f030 2024-04-25T10:11:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f49085b3640 1 -- 172.21.15.60:0/3725335814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4900074770 con 0x7f490006f030 2024-04-25T10:11:54.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f48f6ffd640 1 -- 172.21.15.60:0/3725335814 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f48fc0a6c00 con 0x7f490006f030 2024-04-25T10:11:54.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f48f6ffd640 1 --2- 172.21.15.60:0/3725335814 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f48e4066f60 0x7f48e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f48f6ffd640 1 -- 172.21.15.60:0/3725335814 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48fc11f180 con 0x7f490006f030 2024-04-25T10:11:54.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.372+0000 7f4905b4e640 1 --2- 172.21.15.60:0/3725335814 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f48e4066f60 0x7f48e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.376+0000 7f4905b4e640 1 --2- 172.21.15.60:0/3725335814 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f48e4066f60 0x7f48e4069420 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4900071f00 tx=0x7f48f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.376+0000 7f48f6ffd640 1 -- 172.21.15.60:0/3725335814 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48fc0bf060 con 0x7f490006f030 2024-04-25T10:11:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.540+0000 7f49085b3640 1 -- 172.21.15.60:0/3725335814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4900159a80 con 0x7f490006f030 2024-04-25T10:11:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.540+0000 7f48f6ffd640 1 -- 172.21.15.60:0/3725335814 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48fc016540 con 0x7f490006f030 2024-04-25T10:11:54.548 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:11:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.540+0000 7f49085b3640 1 -- 172.21.15.60:0/3725335814 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f48e4066f60 msgr2=0x7f48e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.540+0000 7f49085b3640 1 --2- 172.21.15.60:0/3725335814 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f48e4066f60 0x7f48e4069420 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4900071f00 tx=0x7f48f0009040 comp rx=0 tx=0).stop 2024-04-25T10:11:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.540+0000 7f49085b3640 1 -- 172.21.15.60:0/3725335814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f490006f030 msgr2=0x7f49001457b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.540+0000 7f49085b3640 1 --2- 172.21.15.60:0/3725335814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f490006f030 0x7f49001457b0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f48fc01a4f0 tx=0x7f48fc019660 comp rx=0 tx=0).stop 2024-04-25T10:11:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.540+0000 7f49085b3640 1 -- 172.21.15.60:0/3725335814 shutdown_connections 2024-04-25T10:11:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.540+0000 7f49085b3640 1 --2- 172.21.15.60:0/3725335814 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f48e4066f60 0x7f48e4069420 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.540+0000 7f49085b3640 1 --2- 172.21.15.60:0/3725335814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49000b3f60 0x7f49001449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.540+0000 7f49085b3640 1 --2- 172.21.15.60:0/3725335814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4900071250 0x7f4900144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.540+0000 7f49085b3640 1 --2- 172.21.15.60:0/3725335814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f490006f030 0x7f49001457b0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.540+0000 7f49085b3640 1 -- 172.21.15.60:0/3725335814 >> 172.21.15.60:0/3725335814 conn(0x7f49000670b0 msgr2=0x7f49000b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.540+0000 7f49085b3640 1 -- 172.21.15.60:0/3725335814 shutdown_connections 2024-04-25T10:11:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.544+0000 7f49085b3640 1 -- 172.21.15.60:0/3725335814 wait complete. 2024-04-25T10:11:54.568 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:54.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdade432640 1 Processor -- start 2024-04-25T10:11:54.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdade432640 1 -- start start 2024-04-25T10:11:54.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdade432640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdad815b910 0x7fdad806fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdade432640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdad80ccf60 0x7fdad806ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdade432640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad815bcf0 con 0x7fdad806b150 2024-04-25T10:11:54.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdade432640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad8072b30 con 0x7fdad80ccf60 2024-04-25T10:11:54.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdade432640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad8073340 con 0x7fdad815b910 2024-04-25T10:11:54.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdadc9cf640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdad806b150 0x7fdad815f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54822/0 (socket says 172.21.15.60:54822) 2024-04-25T10:11:54.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdadc9cf640 1 -- 172.21.15.60:0/236722664 learned_addr learned my addr 172.21.15.60:0/236722664 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:54.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdad77fe640 1 --2- 172.21.15.60:0/236722664 >> v2:172.21.15.60:3300/0 conn(0x7fdad80ccf60 0x7fdad806ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdad7fff640 1 --2- 172.21.15.60:0/236722664 >> v2:172.21.15.100:3300/0 conn(0x7fdad815b910 0x7fdad806fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdad6ffd640 1 -- 172.21.15.60:0/236722664 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393975397 0 0) 0x7fdad815bcf0 con 0x7fdad806b150 2024-04-25T10:11:54.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdad6ffd640 1 -- 172.21.15.60:0/236722664 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdac0003530 con 0x7fdad806b150 2024-04-25T10:11:54.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdad77fe640 1 -- 172.21.15.60:0/236722664 >> v2:172.21.15.100:3300/0 conn(0x7fdad815b910 msgr2=0x7fdad806fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:54.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdad77fe640 1 --2- 172.21.15.60:0/236722664 >> v2:172.21.15.100:3300/0 conn(0x7fdad815b910 0x7fdad806fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdad77fe640 1 -- 172.21.15.60:0/236722664 >> v1:172.21.15.60:6789/0 conn(0x7fdad806b150 legacy=0x7fdad815f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.688+0000 7fdad77fe640 1 -- 172.21.15.60:0/236722664 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad8070b10 con 0x7fdad80ccf60 2024-04-25T10:11:54.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.692+0000 7fdad77fe640 1 --2- 172.21.15.60:0/236722664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdad80ccf60 0x7fdad806ac10 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fdad80d05d0 tx=0x7fdacc01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb56358bc491b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.692+0000 7fdad6ffd640 1 -- 172.21.15.60:0/236722664 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0b0070 con 0x7fdad80ccf60 2024-04-25T10:11:54.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.692+0000 7fdad6ffd640 1 -- 172.21.15.60:0/236722664 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdacc0ab600 con 0x7fdad80ccf60 2024-04-25T10:11:54.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.692+0000 7fdad6ffd640 1 -- 172.21.15.60:0/236722664 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0ab9d0 con 0x7fdad80ccf60 2024-04-25T10:11:54.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.692+0000 7fdade432640 1 -- 172.21.15.60:0/236722664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdad80ccf60 msgr2=0x7fdad806ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.692+0000 7fdade432640 1 --2- 172.21.15.60:0/236722664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdad80ccf60 0x7fdad806ac10 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fdad80d05d0 tx=0x7fdacc01c360 comp rx=0 tx=0).stop 2024-04-25T10:11:54.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.692+0000 7fdade432640 1 -- 172.21.15.60:0/236722664 shutdown_connections 2024-04-25T10:11:54.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.692+0000 7fdade432640 1 --2- 172.21.15.60:0/236722664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdad80ccf60 0x7fdad806ac10 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.692+0000 7fdade432640 1 --2- 172.21.15.60:0/236722664 >> v2:172.21.15.100:3300/0 conn(0x7fdad815b910 0x7fdad806fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.692+0000 7fdade432640 1 -- 172.21.15.60:0/236722664 >> 172.21.15.60:0/236722664 conn(0x7fdad80b2360 msgr2=0x7fdad80b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:54.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.692+0000 7fdade432640 1 -- 172.21.15.60:0/236722664 shutdown_connections 2024-04-25T10:11:54.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.692+0000 7fdade432640 1 -- 172.21.15.60:0/236722664 wait complete. 2024-04-25T10:11:54.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.692+0000 7fdade432640 1 Processor -- start 2024-04-25T10:11:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdade432640 1 -- start start 2024-04-25T10:11:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdade432640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdad806b150 0x7fdad8142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdade432640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad80ccf60 0x7fdad815bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdade432640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdad815b910 0x7fdad8146460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdade432640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad80cd3f0 con 0x7fdad815b910 2024-04-25T10:11:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdade432640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad8070450 con 0x7fdad80ccf60 2024-04-25T10:11:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdade432640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad8070770 con 0x7fdad806b150 2024-04-25T10:11:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdadc9cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdad815b910 0x7fdad8146460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdadc9cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdad815b910 0x7fdad8146460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46958/0 (socket says 172.21.15.60:46958) 2024-04-25T10:11:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdadc9cf640 1 -- 172.21.15.60:0/2676910038 learned_addr learned my addr 172.21.15.60:0/2676910038 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdad7fff640 1 --2- 172.21.15.60:0/2676910038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdad806b150 0x7fdad8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdad77fe640 1 --2- 172.21.15.60:0/2676910038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad80ccf60 0x7fdad815bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdadc9cf640 1 -- 172.21.15.60:0/2676910038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdad806b150 msgr2=0x7fdad8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdadc9cf640 1 --2- 172.21.15.60:0/2676910038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdad806b150 0x7fdad8142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdadc9cf640 1 -- 172.21.15.60:0/2676910038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad80ccf60 msgr2=0x7fdad815bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdadc9cf640 1 --2- 172.21.15.60:0/2676910038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad80ccf60 0x7fdad815bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdadc9cf640 1 -- 172.21.15.60:0/2676910038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad8145af0 con 0x7fdad815b910 2024-04-25T10:11:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdad77fe640 1 --2- 172.21.15.60:0/2676910038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad80ccf60 0x7fdad815bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdadc9cf640 1 --2- 172.21.15.60:0/2676910038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdad815b910 0x7fdad8146460 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fdad000cad0 tx=0x7fdad000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdad4ff9640 1 -- 172.21.15.60:0/2676910038 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0017070 con 0x7fdad815b910 2024-04-25T10:11:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdad4ff9640 1 -- 172.21.15.60:0/2676910038 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad000ac30 con 0x7fdad815b910 2024-04-25T10:11:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdade432640 1 -- 172.21.15.60:0/2676910038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad8148d50 con 0x7fdad815b910 2024-04-25T10:11:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdad4ff9640 1 -- 172.21.15.60:0/2676910038 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0012400 con 0x7fdad815b910 2024-04-25T10:11:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdade432640 1 -- 172.21.15.60:0/2676910038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad8143c10 con 0x7fdad815b910 2024-04-25T10:11:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdade432640 1 -- 172.21.15.60:0/2676910038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad8149270 con 0x7fdad815b910 2024-04-25T10:11:54.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.696+0000 7fdad4ff9640 1 -- 172.21.15.60:0/2676910038 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdad001a070 con 0x7fdad815b910 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.700+0000 7fdad4ff9640 1 --2- 172.21.15.60:0/2676910038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdac4066f60 0x7fdac4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.700+0000 7fdad4ff9640 1 -- 172.21.15.60:0/2676910038 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdad0084d30 con 0x7fdad815b910 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.700+0000 7fdad7fff640 1 --2- 172.21.15.60:0/2676910038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdac4066f60 0x7fdac4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.700+0000 7fdad7fff640 1 --2- 172.21.15.60:0/2676910038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdac4066f60 0x7fdac4069420 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdac800b9d0 tx=0x7fdac8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.704+0000 7fdad4ff9640 1 -- 172.21.15.60:0/2676910038 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdad0055dd0 con 0x7fdad815b910 2024-04-25T10:11:54.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.864+0000 7fdade432640 1 -- 172.21.15.60:0/2676910038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdad8074770 con 0x7fdad815b910 2024-04-25T10:11:54.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.864+0000 7fdad4ff9640 1 -- 172.21.15.60:0/2676910038 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdad00556d0 con 0x7fdad815b910 2024-04-25T10:11:54.872 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:11:54.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.864+0000 7fdade432640 1 -- 172.21.15.60:0/2676910038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdac4066f60 msgr2=0x7fdac4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.864+0000 7fdade432640 1 --2- 172.21.15.60:0/2676910038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdac4066f60 0x7fdac4069420 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdac800b9d0 tx=0x7fdac8008040 comp rx=0 tx=0).stop 2024-04-25T10:11:54.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.864+0000 7fdade432640 1 -- 172.21.15.60:0/2676910038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdad815b910 msgr2=0x7fdad8146460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.864+0000 7fdade432640 1 --2- 172.21.15.60:0/2676910038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdad815b910 0x7fdad8146460 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fdad000cad0 tx=0x7fdad000cf90 comp rx=0 tx=0).stop 2024-04-25T10:11:54.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.864+0000 7fdade432640 1 -- 172.21.15.60:0/2676910038 shutdown_connections 2024-04-25T10:11:54.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.864+0000 7fdade432640 1 --2- 172.21.15.60:0/2676910038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdac4066f60 0x7fdac4069420 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.864+0000 7fdade432640 1 --2- 172.21.15.60:0/2676910038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdad815b910 0x7fdad8146460 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.864+0000 7fdade432640 1 --2- 172.21.15.60:0/2676910038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdad80ccf60 0x7fdad815bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.864+0000 7fdade432640 1 --2- 172.21.15.60:0/2676910038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdad806b150 0x7fdad8142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.864+0000 7fdade432640 1 -- 172.21.15.60:0/2676910038 >> 172.21.15.60:0/2676910038 conn(0x7fdad80b2360 msgr2=0x7fdad80cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:54.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.864+0000 7fdade432640 1 -- 172.21.15.60:0/2676910038 shutdown_connections 2024-04-25T10:11:54.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:54.864+0000 7fdade432640 1 -- 172.21.15.60:0/2676910038 wait complete. 2024-04-25T10:11:54.892 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:55.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a61160640 1 Processor -- start 2024-04-25T10:11:55.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a61160640 1 -- start start 2024-04-25T10:11:55.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a61160640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3a5c0b0ac0 0x7f3a5c06c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a61160640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3a5c0cf2e0 0x7f3a5c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a61160640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a5c0c46e0 0x7f3a5c0c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a61160640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a5c0c5090 con 0x7f3a5c0cf2e0 2024-04-25T10:11:55.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a61160640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a5c099bb0 con 0x7f3a5c0b0ac0 2024-04-25T10:11:55.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a61160640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c0b17c0 con 0x7f3a5c0c46e0 2024-04-25T10:11:55.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a5a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3a5c0cf2e0 0x7f3a5c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a5a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3a5c0cf2e0 0x7f3a5c0b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46960/0 (socket says 172.21.15.60:46960) 2024-04-25T10:11:55.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a5a59c640 1 -- 172.21.15.60:0/2846919213 learned_addr learned my addr 172.21.15.60:0/2846919213 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:55.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a5ad9d640 1 --2- 172.21.15.60:0/2846919213 >> v2:172.21.15.100:3300/0 conn(0x7f3a5c0b0ac0 0x7f3a5c06c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a5b59e640 1 --2- 172.21.15.60:0/2846919213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a5c0c46e0 0x7f3a5c0c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a5a59c640 1 -- 172.21.15.60:0/2846919213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a5c0c46e0 msgr2=0x7f3a5c0c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a5a59c640 1 --2- 172.21.15.60:0/2846919213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a5c0c46e0 0x7f3a5c0c4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a5a59c640 1 -- 172.21.15.60:0/2846919213 >> v2:172.21.15.100:3300/0 conn(0x7f3a5c0b0ac0 msgr2=0x7f3a5c06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a5a59c640 1 --2- 172.21.15.60:0/2846919213 >> v2:172.21.15.100:3300/0 conn(0x7f3a5c0b0ac0 0x7f3a5c06c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.012+0000 7f3a5a59c640 1 -- 172.21.15.60:0/2846919213 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c0b1aa0 con 0x7f3a5c0cf2e0 2024-04-25T10:11:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.016+0000 7f3a5a59c640 1 --2- 172.21.15.60:0/2846919213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a5c0cf2e0 0x7f3a5c0b0ea0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c072ab0 tx=0x7f3a4401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342199b5c35f4962 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.016+0000 7f3a59d9b640 1 -- 172.21.15.60:0/2846919213 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a440b2070 con 0x7f3a5c0cf2e0 2024-04-25T10:11:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.016+0000 7f3a59d9b640 1 -- 172.21.15.60:0/2846919213 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a440a9070 con 0x7f3a5c0cf2e0 2024-04-25T10:11:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.016+0000 7f3a59d9b640 1 -- 172.21.15.60:0/2846919213 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a440ad880 con 0x7f3a5c0cf2e0 2024-04-25T10:11:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.016+0000 7f3a61160640 1 -- 172.21.15.60:0/2846919213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a5c0cf2e0 msgr2=0x7f3a5c0b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.016+0000 7f3a61160640 1 --2- 172.21.15.60:0/2846919213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a5c0cf2e0 0x7f3a5c0b0ea0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c072ab0 tx=0x7f3a4401c490 comp rx=0 tx=0).stop 2024-04-25T10:11:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.016+0000 7f3a61160640 1 -- 172.21.15.60:0/2846919213 shutdown_connections 2024-04-25T10:11:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.016+0000 7f3a61160640 1 --2- 172.21.15.60:0/2846919213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a5c0c46e0 0x7f3a5c0c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.016+0000 7f3a61160640 1 --2- 172.21.15.60:0/2846919213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a5c0cf2e0 0x7f3a5c0b0ea0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.016+0000 7f3a61160640 1 --2- 172.21.15.60:0/2846919213 >> v2:172.21.15.100:3300/0 conn(0x7f3a5c0b0ac0 0x7f3a5c06c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.016+0000 7f3a61160640 1 -- 172.21.15.60:0/2846919213 >> 172.21.15.60:0/2846919213 conn(0x7f3a5c0b3200 msgr2=0x7f3a5c0b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.016+0000 7f3a61160640 1 -- 172.21.15.60:0/2846919213 shutdown_connections 2024-04-25T10:11:55.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.016+0000 7f3a61160640 1 -- 172.21.15.60:0/2846919213 wait complete. 2024-04-25T10:11:55.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.016+0000 7f3a61160640 1 Processor -- start 2024-04-25T10:11:55.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.016+0000 7f3a61160640 1 -- start start 2024-04-25T10:11:55.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a61160640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a5c0b0ac0 0x7f3a5c1497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a61160640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a5c0c46e0 0x7f3a5c0b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a61160640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a5c0cf2e0 0x7f3a5c15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a61160640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c09aa40 con 0x7f3a5c0c46e0 2024-04-25T10:11:55.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a61160640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c09a8c0 con 0x7f3a5c0b0ac0 2024-04-25T10:11:55.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a61160640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c09abc0 con 0x7f3a5c0cf2e0 2024-04-25T10:11:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a5a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a5c0c46e0 0x7f3a5c0b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a5a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a5c0c46e0 0x7f3a5c0b58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46970/0 (socket says 172.21.15.60:46970) 2024-04-25T10:11:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a5a59c640 1 -- 172.21.15.60:0/1864049297 learned_addr learned my addr 172.21.15.60:0/1864049297 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a5b59e640 1 --2- 172.21.15.60:0/1864049297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a5c0cf2e0 0x7f3a5c15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a5ad9d640 1 --2- 172.21.15.60:0/1864049297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a5c0b0ac0 0x7f3a5c1497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a5a59c640 1 -- 172.21.15.60:0/1864049297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a5c0cf2e0 msgr2=0x7f3a5c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a5a59c640 1 --2- 172.21.15.60:0/1864049297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a5c0cf2e0 0x7f3a5c15a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a5a59c640 1 -- 172.21.15.60:0/1864049297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a5c0b0ac0 msgr2=0x7f3a5c1497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a5a59c640 1 --2- 172.21.15.60:0/1864049297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a5c0b0ac0 0x7f3a5c1497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a5a59c640 1 -- 172.21.15.60:0/1864049297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c1658b0 con 0x7f3a5c0c46e0 2024-04-25T10:11:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a5b59e640 1 --2- 172.21.15.60:0/1864049297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a5c0cf2e0 0x7f3a5c15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a5a59c640 1 --2- 172.21.15.60:0/1864049297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a5c0c46e0 0x7f3a5c0b58d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f3a4401ab40 tx=0x7f3a44019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a537fe640 1 -- 172.21.15.60:0/1864049297 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a440bb070 con 0x7f3a5c0c46e0 2024-04-25T10:11:55.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a537fe640 1 -- 172.21.15.60:0/1864049297 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a440add70 con 0x7f3a5c0c46e0 2024-04-25T10:11:55.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a537fe640 1 -- 172.21.15.60:0/1864049297 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a440b2040 con 0x7f3a5c0c46e0 2024-04-25T10:11:55.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a61160640 1 -- 172.21.15.60:0/1864049297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5c0b63e0 con 0x7f3a5c0c46e0 2024-04-25T10:11:55.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a61160640 1 -- 172.21.15.60:0/1864049297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5c0b69a0 con 0x7f3a5c0c46e0 2024-04-25T10:11:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a61160640 1 -- 172.21.15.60:0/1864049297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5c074770 con 0x7f3a5c0c46e0 2024-04-25T10:11:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a537fe640 1 -- 172.21.15.60:0/1864049297 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a440be070 con 0x7f3a5c0c46e0 2024-04-25T10:11:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a537fe640 1 --2- 172.21.15.60:0/1864049297 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a3c066f60 0x7f3a3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a537fe640 1 -- 172.21.15.60:0/1864049297 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a4411df30 con 0x7f3a5c0c46e0 2024-04-25T10:11:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.020+0000 7f3a5ad9d640 1 --2- 172.21.15.60:0/1864049297 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a3c066f60 0x7f3a3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.024+0000 7f3a5ad9d640 1 --2- 172.21.15.60:0/1864049297 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a3c066f60 0x7f3a3c069420 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3a540039f0 tx=0x7f3a54009790 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.028+0000 7f3a537fe640 1 -- 172.21.15.60:0/1864049297 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a44016110 con 0x7f3a5c0c46e0 2024-04-25T10:11:55.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.188+0000 7f3a61160640 1 -- 172.21.15.60:0/1864049297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3a5c0b2880 con 0x7f3a5c0c46e0 2024-04-25T10:11:55.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.188+0000 7f3a537fe640 1 -- 172.21.15.60:0/1864049297 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3a440eeaa0 con 0x7f3a5c0c46e0 2024-04-25T10:11:55.197 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:11:55.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.188+0000 7f3a61160640 1 -- 172.21.15.60:0/1864049297 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a3c066f60 msgr2=0x7f3a3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.188+0000 7f3a61160640 1 --2- 172.21.15.60:0/1864049297 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a3c066f60 0x7f3a3c069420 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3a540039f0 tx=0x7f3a54009790 comp rx=0 tx=0).stop 2024-04-25T10:11:55.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.188+0000 7f3a61160640 1 -- 172.21.15.60:0/1864049297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a5c0c46e0 msgr2=0x7f3a5c0b58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.188+0000 7f3a61160640 1 --2- 172.21.15.60:0/1864049297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a5c0c46e0 0x7f3a5c0b58d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f3a4401ab40 tx=0x7f3a44019230 comp rx=0 tx=0).stop 2024-04-25T10:11:55.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.188+0000 7f3a61160640 1 -- 172.21.15.60:0/1864049297 shutdown_connections 2024-04-25T10:11:55.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.188+0000 7f3a61160640 1 --2- 172.21.15.60:0/1864049297 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3a3c066f60 0x7f3a3c069420 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.188+0000 7f3a61160640 1 --2- 172.21.15.60:0/1864049297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a5c0cf2e0 0x7f3a5c15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.188+0000 7f3a61160640 1 --2- 172.21.15.60:0/1864049297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a5c0c46e0 0x7f3a5c0b58d0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.188+0000 7f3a61160640 1 --2- 172.21.15.60:0/1864049297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a5c0b0ac0 0x7f3a5c1497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.188+0000 7f3a61160640 1 -- 172.21.15.60:0/1864049297 >> 172.21.15.60:0/1864049297 conn(0x7f3a5c0b3200 msgr2=0x7f3a5c0cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.192+0000 7f3a61160640 1 -- 172.21.15.60:0/1864049297 shutdown_connections 2024-04-25T10:11:55.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.192+0000 7f3a61160640 1 -- 172.21.15.60:0/1864049297 wait complete. 2024-04-25T10:11:55.216 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:11:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20beeab640 1 Processor -- start 2024-04-25T10:11:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20beeab640 1 -- start start 2024-04-25T10:11:55.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20beeab640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f20b806ac10 0x7f20b806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20beeab640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b80b54d0 0x7f20b80af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20beeab640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b806b530 con 0x7f20b80b3610 2024-04-25T10:11:55.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20beeab640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b80b3e60 con 0x7f20b806ac10 2024-04-25T10:11:55.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20beeab640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b80b3b80 con 0x7f20b80b54d0 2024-04-25T10:11:55.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20bcc47640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f20b806ac10 0x7f20b806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20bd448640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b80b54d0 0x7f20b80af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20bcc47640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f20b806ac10 0x7f20b806aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58242/0 (socket says 172.21.15.60:58242) 2024-04-25T10:11:55.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20bcc47640 1 -- 172.21.15.60:0/3562468461 learned_addr learned my addr 172.21.15.60:0/3562468461 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:55.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20b77fe640 1 -- 172.21.15.60:0/3562468461 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792639739 0 0) 0x7f20b806b530 con 0x7f20b80b3610 2024-04-25T10:11:55.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20b77fe640 1 -- 172.21.15.60:0/3562468461 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a4003530 con 0x7f20b80b3610 2024-04-25T10:11:55.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20bd448640 1 -- 172.21.15.60:0/3562468461 >> v2:172.21.15.100:3300/0 conn(0x7f20b806ac10 msgr2=0x7f20b806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20bd448640 1 --2- 172.21.15.60:0/3562468461 >> v2:172.21.15.100:3300/0 conn(0x7f20b806ac10 0x7f20b806aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20bd448640 1 -- 172.21.15.60:0/3562468461 >> v1:172.21.15.100:6789/0 conn(0x7f20b80b3610 legacy=0x7f20b80aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.336+0000 7f20bd448640 1 -- 172.21.15.60:0/3562468461 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b80ae130 con 0x7f20b80b54d0 2024-04-25T10:11:55.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.340+0000 7f20bd448640 1 --2- 172.21.15.60:0/3562468461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b80b54d0 0x7f20b80af5d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20b00097c0 tx=0x7f20b001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8173ba73a3df5037 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.340+0000 7f20b77fe640 1 -- 172.21.15.60:0/3562468461 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b00a7070 con 0x7f20b80b54d0 2024-04-25T10:11:55.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.340+0000 7f20b77fe640 1 -- 172.21.15.60:0/3562468461 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b00b0070 con 0x7f20b80b54d0 2024-04-25T10:11:55.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.340+0000 7f20b77fe640 1 -- 172.21.15.60:0/3562468461 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b00ab870 con 0x7f20b80b54d0 2024-04-25T10:11:55.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.340+0000 7f20beeab640 1 -- 172.21.15.60:0/3562468461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b80b54d0 msgr2=0x7f20b80af5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.340+0000 7f20beeab640 1 --2- 172.21.15.60:0/3562468461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b80b54d0 0x7f20b80af5d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20b00097c0 tx=0x7f20b001c470 comp rx=0 tx=0).stop 2024-04-25T10:11:55.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.340+0000 7f20beeab640 1 -- 172.21.15.60:0/3562468461 shutdown_connections 2024-04-25T10:11:55.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.340+0000 7f20beeab640 1 --2- 172.21.15.60:0/3562468461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b80b54d0 0x7f20b80af5d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.340+0000 7f20beeab640 1 --2- 172.21.15.60:0/3562468461 >> v2:172.21.15.100:3300/0 conn(0x7f20b806ac10 0x7f20b806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.340+0000 7f20beeab640 1 -- 172.21.15.60:0/3562468461 >> 172.21.15.60:0/3562468461 conn(0x7f20b8072920 msgr2=0x7f20b8099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.340+0000 7f20beeab640 1 -- 172.21.15.60:0/3562468461 shutdown_connections 2024-04-25T10:11:55.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.340+0000 7f20beeab640 1 -- 172.21.15.60:0/3562468461 wait complete. 2024-04-25T10:11:55.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20beeab640 1 Processor -- start 2024-04-25T10:11:55.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20beeab640 1 -- start start 2024-04-25T10:11:55.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20beeab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b806ac10 0x7f20b814d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20beeab640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b80b3610 0x7f20b80ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20beeab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20b80b54d0 0x7f20b80cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20beeab640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b815df00 con 0x7f20b80b54d0 2024-04-25T10:11:55.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20beeab640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b815dd80 con 0x7f20b806ac10 2024-04-25T10:11:55.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20beeab640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b80aff20 con 0x7f20b80b3610 2024-04-25T10:11:55.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20bd448640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20b80b54d0 0x7f20b80cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20bd448640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20b80b54d0 0x7f20b80cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46982/0 (socket says 172.21.15.60:46982) 2024-04-25T10:11:55.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20bd448640 1 -- 172.21.15.60:0/1300476250 learned_addr learned my addr 172.21.15.60:0/1300476250 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:55.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20b7fff640 1 --2- 172.21.15.60:0/1300476250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b80b3610 0x7f20b80ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20bcc47640 1 --2- 172.21.15.60:0/1300476250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b806ac10 0x7f20b814d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20bd448640 1 -- 172.21.15.60:0/1300476250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b80b3610 msgr2=0x7f20b80ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20bd448640 1 --2- 172.21.15.60:0/1300476250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b80b3610 0x7f20b80ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20bd448640 1 -- 172.21.15.60:0/1300476250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b806ac10 msgr2=0x7f20b814d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20bd448640 1 --2- 172.21.15.60:0/1300476250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b806ac10 0x7f20b814d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20bd448640 1 -- 172.21.15.60:0/1300476250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b80cc220 con 0x7f20b80b54d0 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20bd448640 1 --2- 172.21.15.60:0/1300476250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20b80b54d0 0x7f20b80cab00 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f20b001ab20 tx=0x7f20b0019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20b57fa640 1 -- 172.21.15.60:0/1300476250 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b00abac0 con 0x7f20b80b54d0 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20b57fa640 1 -- 172.21.15.60:0/1300476250 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b00abc60 con 0x7f20b80b54d0 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20b57fa640 1 -- 172.21.15.60:0/1300476250 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b00bf670 con 0x7f20b80b54d0 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20beeab640 1 -- 172.21.15.60:0/1300476250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b80cb180 con 0x7f20b80b54d0 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20beeab640 1 -- 172.21.15.60:0/1300476250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b80cb740 con 0x7f20b80b54d0 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.344+0000 7f20beeab640 1 -- 172.21.15.60:0/1300476250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b8074770 con 0x7f20b80b54d0 2024-04-25T10:11:55.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.348+0000 7f20b57fa640 1 -- 172.21.15.60:0/1300476250 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20b00c9430 con 0x7f20b80b54d0 2024-04-25T10:11:55.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.348+0000 7f20b57fa640 1 --2- 172.21.15.60:0/1300476250 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2098066f60 0x7f2098069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.348+0000 7f20b57fa640 1 -- 172.21.15.60:0/1300476250 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20b011fd00 con 0x7f20b80b54d0 2024-04-25T10:11:55.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.348+0000 7f20bcc47640 1 --2- 172.21.15.60:0/1300476250 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2098066f60 0x7f2098069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.348+0000 7f20bcc47640 1 --2- 172.21.15.60:0/1300476250 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2098066f60 0x7f2098069420 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009ce0 tx=0x7f20a800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.352+0000 7f20b57fa640 1 -- 172.21.15.60:0/1300476250 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20b0016090 con 0x7f20b80b54d0 2024-04-25T10:11:55.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.524+0000 7f20beeab640 1 -- 172.21.15.60:0/1300476250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f20b815c5c0 con 0x7f20b80b54d0 2024-04-25T10:11:55.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.524+0000 7f20b57fa640 1 -- 172.21.15.60:0/1300476250 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v82) v1 ==== 80+0+0 (secure 0 0 0) 0x7f20b00f0cf0 con 0x7f20b80b54d0 2024-04-25T10:11:55.534 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:11:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.528+0000 7f20beeab640 1 -- 172.21.15.60:0/1300476250 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2098066f60 msgr2=0x7f2098069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.528+0000 7f20beeab640 1 --2- 172.21.15.60:0/1300476250 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2098066f60 0x7f2098069420 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009ce0 tx=0x7f20a800b040 comp rx=0 tx=0).stop 2024-04-25T10:11:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.528+0000 7f20beeab640 1 -- 172.21.15.60:0/1300476250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20b80b54d0 msgr2=0x7f20b80cab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.528+0000 7f20beeab640 1 --2- 172.21.15.60:0/1300476250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20b80b54d0 0x7f20b80cab00 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f20b001ab20 tx=0x7f20b0019210 comp rx=0 tx=0).stop 2024-04-25T10:11:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.528+0000 7f20beeab640 1 -- 172.21.15.60:0/1300476250 shutdown_connections 2024-04-25T10:11:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.528+0000 7f20beeab640 1 --2- 172.21.15.60:0/1300476250 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2098066f60 0x7f2098069420 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.528+0000 7f20beeab640 1 --2- 172.21.15.60:0/1300476250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20b80b54d0 0x7f20b80cab00 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.528+0000 7f20beeab640 1 --2- 172.21.15.60:0/1300476250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20b80b3610 0x7f20b80ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.528+0000 7f20beeab640 1 --2- 172.21.15.60:0/1300476250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20b806ac10 0x7f20b814d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.528+0000 7f20beeab640 1 -- 172.21.15.60:0/1300476250 >> 172.21.15.60:0/1300476250 conn(0x7f20b8072920 msgr2=0x7f20b80b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.528+0000 7f20beeab640 1 -- 172.21.15.60:0/1300476250 shutdown_connections 2024-04-25T10:11:55.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.528+0000 7f20beeab640 1 -- 172.21.15.60:0/1300476250 wait complete. 2024-04-25T10:11:55.553 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:11:55.553 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:55.559 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:55.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.724+0000 7f66a41bd640 1 Processor -- start 2024-04-25T10:11:55.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.724+0000 7f66a41bd640 1 -- start start 2024-04-25T10:11:55.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a41bd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f669c15f480 0x7f669c15f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a41bd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f669c0cd390 0x7f669c09a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a41bd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669c15fda0 con 0x7f669c09a930 2024-04-25T10:11:55.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a41bd640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f669c0cc5d0 con 0x7f669c15f480 2024-04-25T10:11:55.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a41bd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f669c0cc8b0 con 0x7f669c0cd390 2024-04-25T10:11:55.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a1f59640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f669c15f480 0x7f669c15f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a1f59640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f669c15f480 0x7f669c15f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46992/0 (socket says 172.21.15.60:46992) 2024-04-25T10:11:55.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a1f59640 1 -- 172.21.15.60:0/3305868747 learned_addr learned my addr 172.21.15.60:0/3305868747 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:55.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a1758640 1 --2- 172.21.15.60:0/3305868747 >> v2:172.21.15.100:3300/0 conn(0x7f669c0cd390 0x7f669c09a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a1f59640 1 -- 172.21.15.60:0/3305868747 >> v2:172.21.15.100:3300/0 conn(0x7f669c0cd390 msgr2=0x7f669c09a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a1f59640 1 --2- 172.21.15.60:0/3305868747 >> v2:172.21.15.100:3300/0 conn(0x7f669c0cd390 0x7f669c09a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a1f59640 1 -- 172.21.15.60:0/3305868747 >> v1:172.21.15.100:6789/0 conn(0x7f669c09a930 legacy=0x7f669c0cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a1f59640 1 -- 172.21.15.60:0/3305868747 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c0ccb90 con 0x7f669c15f480 2024-04-25T10:11:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a1f59640 1 --2- 172.21.15.60:0/3305868747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669c15f480 0x7f669c15f860 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f669c0726d0 tx=0x7f669801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387ad3a7add0e3cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a0f57640 1 -- 172.21.15.60:0/3305868747 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66980aa5f0 con 0x7f669c15f480 2024-04-25T10:11:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a0f57640 1 -- 172.21.15.60:0/3305868747 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66980af070 con 0x7f669c15f480 2024-04-25T10:11:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a0f57640 1 -- 172.21.15.60:0/3305868747 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66980aa9b0 con 0x7f669c15f480 2024-04-25T10:11:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a41bd640 1 -- 172.21.15.60:0/3305868747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669c15f480 msgr2=0x7f669c15f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a41bd640 1 --2- 172.21.15.60:0/3305868747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669c15f480 0x7f669c15f860 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f669c0726d0 tx=0x7f669801c4b0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a41bd640 1 -- 172.21.15.60:0/3305868747 shutdown_connections 2024-04-25T10:11:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a41bd640 1 --2- 172.21.15.60:0/3305868747 >> v2:172.21.15.100:3300/0 conn(0x7f669c0cd390 0x7f669c09a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a41bd640 1 --2- 172.21.15.60:0/3305868747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669c15f480 0x7f669c15f860 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a41bd640 1 -- 172.21.15.60:0/3305868747 >> 172.21.15.60:0/3305868747 conn(0x7f669c0b32a0 msgr2=0x7f669c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a41bd640 1 -- 172.21.15.60:0/3305868747 shutdown_connections 2024-04-25T10:11:55.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.728+0000 7f66a41bd640 1 -- 172.21.15.60:0/3305868747 wait complete. 2024-04-25T10:11:55.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a41bd640 1 Processor -- start 2024-04-25T10:11:55.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a41bd640 1 -- start start 2024-04-25T10:11:55.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a41bd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669c09a930 0x7f669c0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a41bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669c0cd390 0x7f669c14a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a41bd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669c15f480 0x7f669c0b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a41bd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c15cd40 con 0x7f669c15f480 2024-04-25T10:11:55.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a41bd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c15cbc0 con 0x7f669c0cd390 2024-04-25T10:11:55.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a41bd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669c1687c0 con 0x7f669c09a930 2024-04-25T10:11:55.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a275a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669c15f480 0x7f669c0b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a275a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669c15f480 0x7f669c0b2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47000/0 (socket says 172.21.15.60:47000) 2024-04-25T10:11:55.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a275a640 1 -- 172.21.15.60:0/2432994335 learned_addr learned my addr 172.21.15.60:0/2432994335 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:55.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a1f59640 1 --2- 172.21.15.60:0/2432994335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669c09a930 0x7f669c0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a1758640 1 --2- 172.21.15.60:0/2432994335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669c0cd390 0x7f669c14a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a275a640 1 -- 172.21.15.60:0/2432994335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669c09a930 msgr2=0x7f669c0afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a275a640 1 --2- 172.21.15.60:0/2432994335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669c09a930 0x7f669c0afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a275a640 1 -- 172.21.15.60:0/2432994335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669c0cd390 msgr2=0x7f669c14a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a275a640 1 --2- 172.21.15.60:0/2432994335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669c0cd390 0x7f669c14a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a275a640 1 -- 172.21.15.60:0/2432994335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c156a90 con 0x7f669c15f480 2024-04-25T10:11:55.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a1f59640 1 --2- 172.21.15.60:0/2432994335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669c09a930 0x7f669c0afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:55.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a275a640 1 --2- 172.21.15.60:0/2432994335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669c15f480 0x7f669c0b2330 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f669400c7b0 tx=0x7f669400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66927fc640 1 -- 172.21.15.60:0/2432994335 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6694016020 con 0x7f669c15f480 2024-04-25T10:11:55.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a41bd640 1 -- 172.21.15.60:0/2432994335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669c15a090 con 0x7f669c15f480 2024-04-25T10:11:55.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66927fc640 1 -- 172.21.15.60:0/2432994335 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669400aa90 con 0x7f669c15f480 2024-04-25T10:11:55.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66a41bd640 1 -- 172.21.15.60:0/2432994335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669c15a650 con 0x7f669c15f480 2024-04-25T10:11:55.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.732+0000 7f66927fc640 1 -- 172.21.15.60:0/2432994335 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669400ce60 con 0x7f669c15f480 2024-04-25T10:11:55.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.736+0000 7f66a41bd640 1 -- 172.21.15.60:0/2432994335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669c074770 con 0x7f669c15f480 2024-04-25T10:11:55.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.736+0000 7f66927fc640 1 -- 172.21.15.60:0/2432994335 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6694019020 con 0x7f669c15f480 2024-04-25T10:11:55.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.736+0000 7f66927fc640 1 --2- 172.21.15.60:0/2432994335 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6680066f90 0x7f6680069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.736+0000 7f66927fc640 1 -- 172.21.15.60:0/2432994335 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6694085490 con 0x7f669c15f480 2024-04-25T10:11:55.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.736+0000 7f66a1f59640 1 --2- 172.21.15.60:0/2432994335 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6680066f90 0x7f6680069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.736+0000 7f66a1f59640 1 --2- 172.21.15.60:0/2432994335 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6680066f90 0x7f6680069450 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f66980ae000 tx=0x7f66980a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.740+0000 7f66927fc640 1 -- 172.21.15.60:0/2432994335 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6694020030 con 0x7f669c15f480 2024-04-25T10:11:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.904+0000 7f66a41bd640 1 -- 172.21.15.60:0/2432994335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f669c0b1e10 con 0x7f669c15f480 2024-04-25T10:11:55.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.904+0000 7f66927fc640 1 -- 172.21.15.60:0/2432994335 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16672 (secure 0 0 0) 0x7f66940102e0 con 0x7f669c15f480 2024-04-25T10:11:55.914 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:55.914 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:11:46.035912+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:42.621502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:43.352679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:55.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.908+0000 7f66a41bd640 1 -- 172.21.15.60:0/2432994335 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6680066f90 msgr2=0x7f6680069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.908+0000 7f66a41bd640 1 --2- 172.21.15.60:0/2432994335 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6680066f90 0x7f6680069450 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f66980ae000 tx=0x7f66980a7000 comp rx=0 tx=0).stop 2024-04-25T10:11:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.908+0000 7f66a41bd640 1 -- 172.21.15.60:0/2432994335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669c15f480 msgr2=0x7f669c0b2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.908+0000 7f66a41bd640 1 --2- 172.21.15.60:0/2432994335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669c15f480 0x7f669c0b2330 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f669400c7b0 tx=0x7f669400cc70 comp rx=0 tx=0).stop 2024-04-25T10:11:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.908+0000 7f66a41bd640 1 -- 172.21.15.60:0/2432994335 shutdown_connections 2024-04-25T10:11:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.908+0000 7f66a41bd640 1 --2- 172.21.15.60:0/2432994335 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6680066f90 0x7f6680069450 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.908+0000 7f66a41bd640 1 --2- 172.21.15.60:0/2432994335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f669c15f480 0x7f669c0b2330 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.908+0000 7f66a41bd640 1 --2- 172.21.15.60:0/2432994335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669c0cd390 0x7f669c14a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.908+0000 7f66a41bd640 1 --2- 172.21.15.60:0/2432994335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669c09a930 0x7f669c0afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.908+0000 7f66a41bd640 1 -- 172.21.15.60:0/2432994335 >> 172.21.15.60:0/2432994335 conn(0x7f669c0b32a0 msgr2=0x7f669c0b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.908+0000 7f66a41bd640 1 -- 172.21.15.60:0/2432994335 shutdown_connections 2024-04-25T10:11:55.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:55.908+0000 7f66a41bd640 1 -- 172.21.15.60:0/2432994335 wait complete. 2024-04-25T10:11:55.934 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:42.621502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:43.352679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:55.934 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:56.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dfdce640 1 Processor -- start 2024-04-25T10:11:56.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dfdce640 1 -- start start 2024-04-25T10:11:56.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dfdce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9d80cbcf0 0x7ff9d8160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dfdce640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff9d815fbf0 0x7ff9d80ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dfdce640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d8144a40 con 0x7ff9d8071c30 2024-04-25T10:11:56.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dfdce640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d80b2dc0 con 0x7ff9d815fbf0 2024-04-25T10:11:56.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dfdce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d80cc9f0 con 0x7ff9d80cbcf0 2024-04-25T10:11:56.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dd369640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff9d8071c30 0x7ff9d815f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54826/0 (socket says 172.21.15.60:54826) 2024-04-25T10:11:56.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dd369640 1 -- 172.21.15.60:0/226113196 learned_addr learned my addr 172.21.15.60:0/226113196 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:56.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9de36b640 1 --2- 172.21.15.60:0/226113196 >> v2:172.21.15.100:3300/0 conn(0x7ff9d815fbf0 0x7ff9d80ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9ddb6a640 1 --2- 172.21.15.60:0/226113196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9d80cbcf0 0x7ff9d8160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dcb68640 1 -- 172.21.15.60:0/226113196 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493827084 0 0) 0x7ff9d8144a40 con 0x7ff9d8071c30 2024-04-25T10:11:56.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dcb68640 1 -- 172.21.15.60:0/226113196 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9c4003530 con 0x7ff9d8071c30 2024-04-25T10:11:56.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dcb68640 1 -- 172.21.15.60:0/226113196 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 296508834 0 0) 0x7ff9c4003530 con 0x7ff9d8071c30 2024-04-25T10:11:56.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dcb68640 1 -- 172.21.15.60:0/226113196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9d80cbcf0 msgr2=0x7ff9d8160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dcb68640 1 --2- 172.21.15.60:0/226113196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9d80cbcf0 0x7ff9d8160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dcb68640 1 -- 172.21.15.60:0/226113196 >> v2:172.21.15.100:3300/0 conn(0x7ff9d815fbf0 msgr2=0x7ff9d80ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dcb68640 1 --2- 172.21.15.60:0/226113196 >> v2:172.21.15.100:3300/0 conn(0x7ff9d815fbf0 0x7ff9d80ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dcb68640 1 -- 172.21.15.60:0/226113196 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d80cccd0 con 0x7ff9d8071c30 2024-04-25T10:11:56.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dcb68640 1 -- 172.21.15.60:0/226113196 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7ff9c8001680 con 0x7ff9d8071c30 2024-04-25T10:11:56.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dcb68640 1 -- 172.21.15.60:0/226113196 >> v1:172.21.15.60:6789/0 conn(0x7ff9d8071c30 legacy=0x7ff9d815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dcb68640 1 --2- 172.21.15.60:0/226113196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9d80cbcf0 0x7ff9c40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dcb68640 1 --2- 172.21.15.60:0/226113196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c4003f40 0x7ff9c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dcb68640 1 --2- 172.21.15.60:0/226113196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c4004d50 0x7ff9c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dcb68640 1 -- 172.21.15.60:0/226113196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d80cc9f0 con 0x7ff9c4003f40 2024-04-25T10:11:56.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dcb68640 1 -- 172.21.15.60:0/226113196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c400b9d0 con 0x7ff9c4004d50 2024-04-25T10:11:56.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dcb68640 1 -- 172.21.15.60:0/226113196 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c400bb50 con 0x7ff9d80cbcf0 2024-04-25T10:11:56.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dd369640 1 --2- 172.21.15.60:0/226113196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c4003f40 0x7ff9c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9ddb6a640 1 --2- 172.21.15.60:0/226113196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9d80cbcf0 0x7ff9c40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9de36b640 1 --2- 172.21.15.60:0/226113196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c4004d50 0x7ff9c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dd369640 1 -- 172.21.15.60:0/226113196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9d80cbcf0 msgr2=0x7ff9c40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dd369640 1 --2- 172.21.15.60:0/226113196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9d80cbcf0 0x7ff9c40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dd369640 1 -- 172.21.15.60:0/226113196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c4004d50 msgr2=0x7ff9c40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dd369640 1 --2- 172.21.15.60:0/226113196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c4004d50 0x7ff9c40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9dd369640 1 -- 172.21.15.60:0/226113196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c4003530 con 0x7ff9c4003f40 2024-04-25T10:11:56.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.056+0000 7ff9de36b640 1 --2- 172.21.15.60:0/226113196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c4004d50 0x7ff9c40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:56.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.060+0000 7ff9dd369640 1 --2- 172.21.15.60:0/226113196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c4003f40 0x7ff9c4004810 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800c210 tx=0x7ff9c801f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1acb56ebcfacf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.060+0000 7ff9dcb68640 1 -- 172.21.15.60:0/226113196 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8007d60 con 0x7ff9c4003f40 2024-04-25T10:11:56.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.060+0000 7ff9dcb68640 1 -- 172.21.15.60:0/226113196 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c8005ae0 con 0x7ff9c4003f40 2024-04-25T10:11:56.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.060+0000 7ff9dcb68640 1 -- 172.21.15.60:0/226113196 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8007070 con 0x7ff9c4003f40 2024-04-25T10:11:56.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.060+0000 7ff9dfdce640 1 -- 172.21.15.60:0/226113196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c4003f40 msgr2=0x7ff9c4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.060+0000 7ff9dfdce640 1 --2- 172.21.15.60:0/226113196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c4003f40 0x7ff9c4004810 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800c210 tx=0x7ff9c801f390 comp rx=0 tx=0).stop 2024-04-25T10:11:56.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.060+0000 7ff9de36b640 1 -- 172.21.15.60:0/226113196 reap_dead start 2024-04-25T10:11:56.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.060+0000 7ff9dfdce640 1 -- 172.21.15.60:0/226113196 shutdown_connections 2024-04-25T10:11:56.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.060+0000 7ff9dfdce640 1 -- 172.21.15.60:0/226113196 >> 172.21.15.60:0/226113196 conn(0x7ff9d80670b0 msgr2=0x7ff9d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.060+0000 7ff9dfdce640 1 -- 172.21.15.60:0/226113196 shutdown_connections 2024-04-25T10:11:56.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.060+0000 7ff9dfdce640 1 -- 172.21.15.60:0/226113196 wait complete. 2024-04-25T10:11:56.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.060+0000 7ff9dfdce640 1 Processor -- start 2024-04-25T10:11:56.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.060+0000 7ff9dfdce640 1 -- start start 2024-04-25T10:11:56.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9dfdce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c4003f40 0x7ff9d80afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9dfdce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9d813cd70 0x7ff9d813d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9dfdce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d814ceb0 0x7ff9d814d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9dfdce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d80cc0d0 con 0x7ff9c4003f40 2024-04-25T10:11:56.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9dfdce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d80ce9a0 con 0x7ff9d814ceb0 2024-04-25T10:11:56.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9dfdce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d80720b0 con 0x7ff9d813cd70 2024-04-25T10:11:56.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9ddb6a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c4003f40 0x7ff9d80afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9ddb6a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c4003f40 0x7ff9d80afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47018/0 (socket says 172.21.15.60:47018) 2024-04-25T10:11:56.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9ddb6a640 1 -- 172.21.15.60:0/1864501694 learned_addr learned my addr 172.21.15.60:0/1864501694 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:56.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9dd369640 1 --2- 172.21.15.60:0/1864501694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9d813cd70 0x7ff9d813d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9de36b640 1 --2- 172.21.15.60:0/1864501694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d814ceb0 0x7ff9d814d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9ddb6a640 1 -- 172.21.15.60:0/1864501694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9d813cd70 msgr2=0x7ff9d813d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9ddb6a640 1 --2- 172.21.15.60:0/1864501694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9d813cd70 0x7ff9d813d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9ddb6a640 1 -- 172.21.15.60:0/1864501694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d814ceb0 msgr2=0x7ff9d814d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9ddb6a640 1 --2- 172.21.15.60:0/1864501694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d814ceb0 0x7ff9d814d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9ddb6a640 1 -- 172.21.15.60:0/1864501694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d80b06e0 con 0x7ff9c4003f40 2024-04-25T10:11:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9ddb6a640 1 --2- 172.21.15.60:0/1864501694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c4003f40 0x7ff9d80afe90 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4011560 tx=0x7ff9d4011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9ce7fc640 1 -- 172.21.15.60:0/1864501694 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4015930 con 0x7ff9c4003f40 2024-04-25T10:11:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9ce7fc640 1 -- 172.21.15.60:0/1864501694 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d4015ad0 con 0x7ff9c4003f40 2024-04-25T10:11:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9dfdce640 1 -- 172.21.15.60:0/1864501694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d80b09c0 con 0x7ff9c4003f40 2024-04-25T10:11:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9ce7fc640 1 -- 172.21.15.60:0/1864501694 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400f460 con 0x7ff9c4003f40 2024-04-25T10:11:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9dfdce640 1 -- 172.21.15.60:0/1864501694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d80b0f80 con 0x7ff9c4003f40 2024-04-25T10:11:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9dfdce640 1 -- 172.21.15.60:0/1864501694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d8074770 con 0x7ff9c4003f40 2024-04-25T10:11:56.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9ce7fc640 1 -- 172.21.15.60:0/1864501694 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9d400f600 con 0x7ff9c4003f40 2024-04-25T10:11:56.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9ce7fc640 1 --2- 172.21.15.60:0/1864501694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff9bc066f90 0x7ff9bc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9ce7fc640 1 -- 172.21.15.60:0/1864501694 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9d4085810 con 0x7ff9c4003f40 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.064+0000 7ff9dd369640 1 --2- 172.21.15.60:0/1864501694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff9bc066f90 0x7ff9bc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.068+0000 7ff9dd369640 1 --2- 172.21.15.60:0/1864501694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff9bc066f90 0x7ff9bc069450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff9c80ae000 tx=0x7ff9c8000ff0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.068+0000 7ff9ce7fc640 1 -- 172.21.15.60:0/1864501694 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff9d4019030 con 0x7ff9c4003f40 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.236+0000 7ff9dfdce640 1 -- 172.21.15.60:0/1864501694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9d8142300 con 0x7ff9c4003f40 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.236+0000 7ff9ce7fc640 1 -- 172.21.15.60:0/1864501694 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9d4056820 con 0x7ff9c4003f40 2024-04-25T10:11:56.245 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:11:56.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.240+0000 7ff9dfdce640 1 -- 172.21.15.60:0/1864501694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff9bc066f90 msgr2=0x7ff9bc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.240+0000 7ff9dfdce640 1 --2- 172.21.15.60:0/1864501694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff9bc066f90 0x7ff9bc069450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff9c80ae000 tx=0x7ff9c8000ff0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.240+0000 7ff9dfdce640 1 -- 172.21.15.60:0/1864501694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c4003f40 msgr2=0x7ff9d80afe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.240+0000 7ff9dfdce640 1 --2- 172.21.15.60:0/1864501694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c4003f40 0x7ff9d80afe90 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4011560 tx=0x7ff9d4011a20 comp rx=0 tx=0).stop 2024-04-25T10:11:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.240+0000 7ff9dfdce640 1 -- 172.21.15.60:0/1864501694 shutdown_connections 2024-04-25T10:11:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.240+0000 7ff9dfdce640 1 --2- 172.21.15.60:0/1864501694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff9bc066f90 0x7ff9bc069450 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.240+0000 7ff9dfdce640 1 --2- 172.21.15.60:0/1864501694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d814ceb0 0x7ff9d814d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.240+0000 7ff9dfdce640 1 --2- 172.21.15.60:0/1864501694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9d813cd70 0x7ff9d813d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.240+0000 7ff9dfdce640 1 --2- 172.21.15.60:0/1864501694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c4003f40 0x7ff9d80afe90 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.240+0000 7ff9dfdce640 1 -- 172.21.15.60:0/1864501694 >> 172.21.15.60:0/1864501694 conn(0x7ff9d80670b0 msgr2=0x7ff9d815d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.240+0000 7ff9dfdce640 1 -- 172.21.15.60:0/1864501694 shutdown_connections 2024-04-25T10:11:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.240+0000 7ff9dfdce640 1 -- 172.21.15.60:0/1864501694 wait complete. 2024-04-25T10:11:56.265 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:56.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.384+0000 7fe234288640 1 Processor -- start 2024-04-25T10:11:56.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.384+0000 7fe234288640 1 -- start start 2024-04-25T10:11:56.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.384+0000 7fe234288640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22c15bea0 0x7fe22c0b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.384+0000 7fe234288640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe22c0b35e0 0x7fe22c0718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.384+0000 7fe234288640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22c15ce20 con 0x7fe22c0cefe0 2024-04-25T10:11:56.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.384+0000 7fe234288640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22c15c350 con 0x7fe22c0b35e0 2024-04-25T10:11:56.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.384+0000 7fe234288640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c071410 con 0x7fe22c15bea0 2024-04-25T10:11:56.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.384+0000 7fe232825640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe22c0b35e0 0x7fe22c0718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.384+0000 7fe232825640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe22c0b35e0 0x7fe22c0718a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47028/0 (socket says 172.21.15.60:47028) 2024-04-25T10:11:56.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.384+0000 7fe232825640 1 -- 172.21.15.60:0/3800551842 learned_addr learned my addr 172.21.15.60:0/3800551842 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:56.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.384+0000 7fe231823640 1 --2- 172.21.15.60:0/3800551842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22c15bea0 0x7fe22c0b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.384+0000 7fe232825640 1 -- 172.21.15.60:0/3800551842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22c15bea0 msgr2=0x7fe22c0b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.384+0000 7fe232825640 1 --2- 172.21.15.60:0/3800551842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22c15bea0 0x7fe22c0b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.384+0000 7fe232825640 1 -- 172.21.15.60:0/3800551842 >> v1:172.21.15.100:6789/0 conn(0x7fe22c0cefe0 legacy=0x7fe22c0cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.384+0000 7fe232825640 1 -- 172.21.15.60:0/3800551842 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c15c580 con 0x7fe22c0b35e0 2024-04-25T10:11:56.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.388+0000 7fe232825640 1 --2- 172.21.15.60:0/3800551842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe22c0b35e0 0x7fe22c0718a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe224009e50 tx=0x7fe22401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521748e8250b25dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.388+0000 7fe231022640 1 -- 172.21.15.60:0/3800551842 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2240a55e0 con 0x7fe22c0b35e0 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.388+0000 7fe231022640 1 -- 172.21.15.60:0/3800551842 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2240a5780 con 0x7fe22c0b35e0 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.388+0000 7fe231022640 1 -- 172.21.15.60:0/3800551842 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2240a5b50 con 0x7fe22c0b35e0 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.388+0000 7fe234288640 1 -- 172.21.15.60:0/3800551842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe22c0b35e0 msgr2=0x7fe22c0718a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.388+0000 7fe234288640 1 --2- 172.21.15.60:0/3800551842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe22c0b35e0 0x7fe22c0718a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe224009e50 tx=0x7fe22401c960 comp rx=0 tx=0).stop 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.388+0000 7fe234288640 1 -- 172.21.15.60:0/3800551842 shutdown_connections 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.388+0000 7fe234288640 1 --2- 172.21.15.60:0/3800551842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe22c0b35e0 0x7fe22c0718a0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.388+0000 7fe234288640 1 --2- 172.21.15.60:0/3800551842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22c15bea0 0x7fe22c0b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.388+0000 7fe234288640 1 -- 172.21.15.60:0/3800551842 >> 172.21.15.60:0/3800551842 conn(0x7fe22c0670b0 msgr2=0x7fe22c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.388+0000 7fe234288640 1 -- 172.21.15.60:0/3800551842 shutdown_connections 2024-04-25T10:11:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.388+0000 7fe234288640 1 -- 172.21.15.60:0/3800551842 wait complete. 2024-04-25T10:11:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.388+0000 7fe234288640 1 Processor -- start 2024-04-25T10:11:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.388+0000 7fe234288640 1 -- start start 2024-04-25T10:11:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe234288640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe22c0b35e0 0x7fe22c152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe234288640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe22c0cefe0 0x7fe22c151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe234288640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22c15bea0 0x7fe22c0b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe234288640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c168ba0 con 0x7fe22c0b35e0 2024-04-25T10:11:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe234288640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c168a20 con 0x7fe22c0cefe0 2024-04-25T10:11:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe234288640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c168d20 con 0x7fe22c15bea0 2024-04-25T10:11:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe232024640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe22c0b35e0 0x7fe22c152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe232024640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe22c0b35e0 0x7fe22c152ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47044/0 (socket says 172.21.15.60:47044) 2024-04-25T10:11:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe232024640 1 -- 172.21.15.60:0/3922424088 learned_addr learned my addr 172.21.15.60:0/3922424088 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe231823640 1 --2- 172.21.15.60:0/3922424088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe22c0cefe0 0x7fe22c151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe232825640 1 --2- 172.21.15.60:0/3922424088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22c15bea0 0x7fe22c0b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe232024640 1 -- 172.21.15.60:0/3922424088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22c15bea0 msgr2=0x7fe22c0b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe232024640 1 --2- 172.21.15.60:0/3922424088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22c15bea0 0x7fe22c0b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe232024640 1 -- 172.21.15.60:0/3922424088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe22c0cefe0 msgr2=0x7fe22c151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe232024640 1 --2- 172.21.15.60:0/3922424088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe22c0cefe0 0x7fe22c151440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe232024640 1 -- 172.21.15.60:0/3922424088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c1f92b0 con 0x7fe22c0b35e0 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe232825640 1 --2- 172.21.15.60:0/3922424088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22c15bea0 0x7fe22c0b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe232024640 1 --2- 172.21.15.60:0/3922424088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe22c0b35e0 0x7fe22c152ac0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c450 tx=0x7fe22800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe222ffd640 1 -- 172.21.15.60:0/3922424088 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228012830 con 0x7fe22c0b35e0 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe222ffd640 1 -- 172.21.15.60:0/3922424088 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe228017070 con 0x7fe22c0b35e0 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe222ffd640 1 -- 172.21.15.60:0/3922424088 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228012c30 con 0x7fe22c0b35e0 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe234288640 1 -- 172.21.15.60:0/3922424088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22c151d00 con 0x7fe22c0b35e0 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe234288640 1 -- 172.21.15.60:0/3922424088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22c1522c0 con 0x7fe22c0b35e0 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe234288640 1 -- 172.21.15.60:0/3922424088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22c06bba0 con 0x7fe22c0b35e0 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe222ffd640 1 -- 172.21.15.60:0/3922424088 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe22801c460 con 0x7fe22c0b35e0 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe222ffd640 1 --2- 172.21.15.60:0/3922424088 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe210066f60 0x7fe210069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe222ffd640 1 -- 172.21.15.60:0/3922424088 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe228084d10 con 0x7fe22c0b35e0 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.392+0000 7fe231823640 1 --2- 172.21.15.60:0/3922424088 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe210066f60 0x7fe210069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.396+0000 7fe231823640 1 --2- 172.21.15.60:0/3922424088 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe210066f60 0x7fe210069420 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0af360 tx=0x7fe21c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.396+0000 7fe222ffd640 1 -- 172.21.15.60:0/3922424088 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe228055db0 con 0x7fe22c0b35e0 2024-04-25T10:11:56.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.552+0000 7fe234288640 1 -- 172.21.15.60:0/3922424088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe22c074770 con 0x7fe22c0b35e0 2024-04-25T10:11:56.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.556+0000 7fe222ffd640 1 -- 172.21.15.60:0/3922424088 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2280556b0 con 0x7fe22c0b35e0 2024-04-25T10:11:56.563 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:11:56.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.556+0000 7fe234288640 1 -- 172.21.15.60:0/3922424088 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe210066f60 msgr2=0x7fe210069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.556+0000 7fe234288640 1 --2- 172.21.15.60:0/3922424088 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe210066f60 0x7fe210069420 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0af360 tx=0x7fe21c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:56.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.556+0000 7fe234288640 1 -- 172.21.15.60:0/3922424088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe22c0b35e0 msgr2=0x7fe22c152ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.556+0000 7fe234288640 1 --2- 172.21.15.60:0/3922424088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe22c0b35e0 0x7fe22c152ac0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c450 tx=0x7fe22800c910 comp rx=0 tx=0).stop 2024-04-25T10:11:56.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.556+0000 7fe234288640 1 -- 172.21.15.60:0/3922424088 shutdown_connections 2024-04-25T10:11:56.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.556+0000 7fe234288640 1 --2- 172.21.15.60:0/3922424088 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe210066f60 0x7fe210069420 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.556+0000 7fe234288640 1 --2- 172.21.15.60:0/3922424088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22c15bea0 0x7fe22c0b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.556+0000 7fe234288640 1 --2- 172.21.15.60:0/3922424088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe22c0cefe0 0x7fe22c151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.556+0000 7fe234288640 1 --2- 172.21.15.60:0/3922424088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe22c0b35e0 0x7fe22c152ac0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.556+0000 7fe234288640 1 -- 172.21.15.60:0/3922424088 >> 172.21.15.60:0/3922424088 conn(0x7fe22c0670b0 msgr2=0x7fe22c15b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.556+0000 7fe234288640 1 -- 172.21.15.60:0/3922424088 shutdown_connections 2024-04-25T10:11:56.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.556+0000 7fe234288640 1 -- 172.21.15.60:0/3922424088 wait complete. 2024-04-25T10:11:56.580 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:56.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65b6e3c640 1 Processor -- start 2024-04-25T10:11:56.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65b6e3c640 1 -- start start 2024-04-25T10:11:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65b6e3c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65b00b4dd0 0x7f65b015be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65b6e3c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f65b00d05a0 0x7f65b00b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65b6e3c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f65b006ac10 0x7f65b0070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65b6e3c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b006fb70 con 0x7f65b00d05a0 2024-04-25T10:11:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65b6e3c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b015c370 con 0x7f65b006ac10 2024-04-25T10:11:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65b6e3c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b00b56f0 con 0x7f65b00b4dd0 2024-04-25T10:11:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65affff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f65b00d05a0 0x7f65b00b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65affff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f65b00d05a0 0x7f65b00b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47060/0 (socket says 172.21.15.60:47060) 2024-04-25T10:11:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65affff640 1 -- 172.21.15.60:0/3406630121 learned_addr learned my addr 172.21.15.60:0/3406630121 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65b53d9640 1 --2- 172.21.15.60:0/3406630121 >> v2:172.21.15.100:3300/0 conn(0x7f65b006ac10 0x7f65b0070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65b4bd8640 1 --2- 172.21.15.60:0/3406630121 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65b00b4dd0 0x7f65b015be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65affff640 1 -- 172.21.15.60:0/3406630121 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65b00b4dd0 msgr2=0x7f65b015be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65affff640 1 --2- 172.21.15.60:0/3406630121 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65b00b4dd0 0x7f65b015be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65affff640 1 -- 172.21.15.60:0/3406630121 >> v2:172.21.15.100:3300/0 conn(0x7f65b006ac10 msgr2=0x7f65b0070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65affff640 1 --2- 172.21.15.60:0/3406630121 >> v2:172.21.15.100:3300/0 conn(0x7f65b006ac10 0x7f65b0070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.692+0000 7f65affff640 1 -- 172.21.15.60:0/3406630121 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b006b1d0 con 0x7f65b00d05a0 2024-04-25T10:11:56.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65affff640 1 --2- 172.21.15.60:0/3406630121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65b00d05a0 0x7f65b00b51b0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f65b00ce170 tx=0x7f659c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b28a322053ae185 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65af7fe640 1 -- 172.21.15.60:0/3406630121 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c0ab620 con 0x7f65b00d05a0 2024-04-25T10:11:56.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65af7fe640 1 -- 172.21.15.60:0/3406630121 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c0b0070 con 0x7f65b00d05a0 2024-04-25T10:11:56.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65af7fe640 1 -- 172.21.15.60:0/3406630121 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c0ab9e0 con 0x7f65b00d05a0 2024-04-25T10:11:56.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 -- 172.21.15.60:0/3406630121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65b00d05a0 msgr2=0x7f65b00b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 --2- 172.21.15.60:0/3406630121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65b00d05a0 0x7f65b00b51b0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f65b00ce170 tx=0x7f659c01c460 comp rx=0 tx=0).stop 2024-04-25T10:11:56.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 -- 172.21.15.60:0/3406630121 shutdown_connections 2024-04-25T10:11:56.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 --2- 172.21.15.60:0/3406630121 >> v2:172.21.15.100:3300/0 conn(0x7f65b006ac10 0x7f65b0070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 --2- 172.21.15.60:0/3406630121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65b00d05a0 0x7f65b00b51b0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 --2- 172.21.15.60:0/3406630121 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65b00b4dd0 0x7f65b015be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 -- 172.21.15.60:0/3406630121 >> 172.21.15.60:0/3406630121 conn(0x7f65b00b2360 msgr2=0x7f65b00b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 -- 172.21.15.60:0/3406630121 shutdown_connections 2024-04-25T10:11:56.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 -- 172.21.15.60:0/3406630121 wait complete. 2024-04-25T10:11:56.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 Processor -- start 2024-04-25T10:11:56.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 -- start start 2024-04-25T10:11:56.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65b006ac10 0x7f65b013b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b00b4dd0 0x7f65b00ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65b00d05a0 0x7f65b015d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b0145390 con 0x7f65b00d05a0 2024-04-25T10:11:56.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b006ba70 con 0x7f65b00b4dd0 2024-04-25T10:11:56.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b6e3c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b0145510 con 0x7f65b006ac10 2024-04-25T10:11:56.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b53d9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65b00d05a0 0x7f65b015d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b53d9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65b00d05a0 0x7f65b015d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47076/0 (socket says 172.21.15.60:47076) 2024-04-25T10:11:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b53d9640 1 -- 172.21.15.60:0/3578350693 learned_addr learned my addr 172.21.15.60:0/3578350693 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b53d9640 1 -- 172.21.15.60:0/3578350693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65b006ac10 msgr2=0x7f65b013b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b4bd8640 1 --2- 172.21.15.60:0/3578350693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65b006ac10 0x7f65b013b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b53d9640 1 --2- 172.21.15.60:0/3578350693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65b006ac10 0x7f65b013b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b53d9640 1 -- 172.21.15.60:0/3578350693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b00b4dd0 msgr2=0x7f65b00ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b53d9640 1 --2- 172.21.15.60:0/3578350693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b00b4dd0 0x7f65b00ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b53d9640 1 -- 172.21.15.60:0/3578350693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b00ce280 con 0x7f65b00d05a0 2024-04-25T10:11:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65b53d9640 1 --2- 172.21.15.60:0/3578350693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65b00d05a0 0x7f65b015d570 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f65a8007d20 tx=0x7f65a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.696+0000 7f65ad7fa640 1 -- 172.21.15.60:0/3578350693 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a8014420 con 0x7f65b00d05a0 2024-04-25T10:11:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.700+0000 7f65ad7fa640 1 -- 172.21.15.60:0/3578350693 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a8019070 con 0x7f65b00d05a0 2024-04-25T10:11:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.700+0000 7f65b6e3c640 1 -- 172.21.15.60:0/3578350693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b015e070 con 0x7f65b00d05a0 2024-04-25T10:11:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.700+0000 7f65ad7fa640 1 -- 172.21.15.60:0/3578350693 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a8014820 con 0x7f65b00d05a0 2024-04-25T10:11:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.700+0000 7f65b6e3c640 1 -- 172.21.15.60:0/3578350693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b00ce800 con 0x7f65b00d05a0 2024-04-25T10:11:56.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.700+0000 7f65b6e3c640 1 -- 172.21.15.60:0/3578350693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b0074770 con 0x7f65b00d05a0 2024-04-25T10:11:56.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.700+0000 7f65ad7fa640 1 -- 172.21.15.60:0/3578350693 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65a80149c0 con 0x7f65b00d05a0 2024-04-25T10:11:56.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.700+0000 7f65ad7fa640 1 --2- 172.21.15.60:0/3578350693 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6594066f60 0x7f6594069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.700+0000 7f65ad7fa640 1 -- 172.21.15.60:0/3578350693 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65a8085990 con 0x7f65b00d05a0 2024-04-25T10:11:56.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.700+0000 7f65b4bd8640 1 --2- 172.21.15.60:0/3578350693 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6594066f60 0x7f6594069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.700+0000 7f65b4bd8640 1 --2- 172.21.15.60:0/3578350693 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6594066f60 0x7f6594069420 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f65a40096f0 tx=0x7f65a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.704+0000 7f65ad7fa640 1 -- 172.21.15.60:0/3578350693 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65a80569c0 con 0x7f65b00d05a0 2024-04-25T10:11:56.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.860+0000 7f65b6e3c640 1 -- 172.21.15.60:0/3578350693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65b015ed50 con 0x7f65b00d05a0 2024-04-25T10:11:56.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.860+0000 7f65ad7fa640 1 -- 172.21.15.60:0/3578350693 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65a80562c0 con 0x7f65b00d05a0 2024-04-25T10:11:56.869 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.860+0000 7f65b6e3c640 1 -- 172.21.15.60:0/3578350693 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6594066f60 msgr2=0x7f6594069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.860+0000 7f65b6e3c640 1 --2- 172.21.15.60:0/3578350693 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6594066f60 0x7f6594069420 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f65a40096f0 tx=0x7f65a4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.860+0000 7f65b6e3c640 1 -- 172.21.15.60:0/3578350693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65b00d05a0 msgr2=0x7f65b015d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.860+0000 7f65b6e3c640 1 --2- 172.21.15.60:0/3578350693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65b00d05a0 0x7f65b015d570 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f65a8007d20 tx=0x7f65a800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.860+0000 7f65b6e3c640 1 -- 172.21.15.60:0/3578350693 shutdown_connections 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.860+0000 7f65b6e3c640 1 --2- 172.21.15.60:0/3578350693 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6594066f60 0x7f6594069420 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.860+0000 7f65b6e3c640 1 --2- 172.21.15.60:0/3578350693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65b00d05a0 0x7f65b015d570 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.860+0000 7f65b6e3c640 1 --2- 172.21.15.60:0/3578350693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65b00b4dd0 0x7f65b00ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.860+0000 7f65b6e3c640 1 --2- 172.21.15.60:0/3578350693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65b006ac10 0x7f65b013b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.860+0000 7f65b6e3c640 1 -- 172.21.15.60:0/3578350693 >> 172.21.15.60:0/3578350693 conn(0x7f65b00b2360 msgr2=0x7f65b0143bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.860+0000 7f65b6e3c640 1 -- 172.21.15.60:0/3578350693 shutdown_connections 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.860+0000 7f65b6e3c640 1 -- 172.21.15.60:0/3578350693 wait complete. 2024-04-25T10:11:56.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:11:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.988+0000 7f591174c640 1 Processor -- start 2024-04-25T10:11:56.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.988+0000 7f591174c640 1 -- start start 2024-04-25T10:11:56.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.988+0000 7f591174c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f590c15a7a0 0x7f590c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.988+0000 7f591174c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f590c06ec30 0x7f590c071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.988+0000 7f591174c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590c0721e0 con 0x7f590c0b37e0 2024-04-25T10:11:56.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.988+0000 7f591174c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f590c0cdaa0 con 0x7f590c06ec30 2024-04-25T10:11:56.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.988+0000 7f591174c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f590c06d2f0 con 0x7f590c15a7a0 2024-04-25T10:11:56.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.988+0000 7f590a7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f590c06ec30 0x7f590c071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.988+0000 7f590b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f590c0b37e0 0x7f590c06dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54842/0 (socket says 172.21.15.60:54842) 2024-04-25T10:11:56.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.988+0000 7f590b7fe640 1 -- 172.21.15.60:0/3508828267 learned_addr learned my addr 172.21.15.60:0/3508828267 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:56.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.988+0000 7f590affd640 1 --2- 172.21.15.60:0/3508828267 >> v2:172.21.15.100:3300/0 conn(0x7f590c15a7a0 0x7f590c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.988+0000 7f590a7fc640 1 -- 172.21.15.60:0/3508828267 >> v2:172.21.15.100:3300/0 conn(0x7f590c15a7a0 msgr2=0x7f590c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.988+0000 7f590a7fc640 1 --2- 172.21.15.60:0/3508828267 >> v2:172.21.15.100:3300/0 conn(0x7f590c15a7a0 0x7f590c0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.988+0000 7f590a7fc640 1 -- 172.21.15.60:0/3508828267 >> v1:172.21.15.60:6789/0 conn(0x7f590c0b37e0 legacy=0x7f590c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.988+0000 7f5909ffb640 1 -- 172.21.15.60:0/3508828267 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762708438 0 0) 0x7f590c0721e0 con 0x7f590c0b37e0 2024-04-25T10:11:56.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.988+0000 7f590a7fc640 1 -- 172.21.15.60:0/3508828267 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c06f130 con 0x7f590c06ec30 2024-04-25T10:11:57.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.992+0000 7f590a7fc640 1 --2- 172.21.15.60:0/3508828267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c06ec30 0x7f590c071ca0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f590c04f5d0 tx=0x7f590001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d954804712f65e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.992+0000 7f5909ffb640 1 -- 172.21.15.60:0/3508828267 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59000b1070 con 0x7f590c06ec30 2024-04-25T10:11:57.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.992+0000 7f5909ffb640 1 -- 172.21.15.60:0/3508828267 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59000ac5f0 con 0x7f590c06ec30 2024-04-25T10:11:57.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.992+0000 7f5909ffb640 1 -- 172.21.15.60:0/3508828267 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59000ac9c0 con 0x7f590c06ec30 2024-04-25T10:11:57.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.992+0000 7f591174c640 1 -- 172.21.15.60:0/3508828267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c06ec30 msgr2=0x7f590c071ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.992+0000 7f591174c640 1 --2- 172.21.15.60:0/3508828267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c06ec30 0x7f590c071ca0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f590c04f5d0 tx=0x7f590001c480 comp rx=0 tx=0).stop 2024-04-25T10:11:57.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.992+0000 7f591174c640 1 -- 172.21.15.60:0/3508828267 shutdown_connections 2024-04-25T10:11:57.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.992+0000 7f591174c640 1 --2- 172.21.15.60:0/3508828267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c06ec30 0x7f590c071ca0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.992+0000 7f591174c640 1 --2- 172.21.15.60:0/3508828267 >> v2:172.21.15.100:3300/0 conn(0x7f590c15a7a0 0x7f590c0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.992+0000 7f591174c640 1 -- 172.21.15.60:0/3508828267 >> 172.21.15.60:0/3508828267 conn(0x7f590c0b12d0 msgr2=0x7f590c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.992+0000 7f591174c640 1 -- 172.21.15.60:0/3508828267 shutdown_connections 2024-04-25T10:11:57.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.992+0000 7f591174c640 1 -- 172.21.15.60:0/3508828267 wait complete. 2024-04-25T10:11:57.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f591174c640 1 Processor -- start 2024-04-25T10:11:57.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f591174c640 1 -- start start 2024-04-25T10:11:57.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f591174c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f590c06ec30 0x7f590c1f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f591174c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f590c0b37e0 0x7f590c157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f591174c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c15a7a0 0x7f590c0ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f591174c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c15ad00 con 0x7f590c15a7a0 2024-04-25T10:11:57.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f591174c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c15ab80 con 0x7f590c06ec30 2024-04-25T10:11:57.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f591174c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590c06e860 con 0x7f590c0b37e0 2024-04-25T10:11:57.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f590b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c15a7a0 0x7f590c0ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f590b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c15a7a0 0x7f590c0ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47086/0 (socket says 172.21.15.60:47086) 2024-04-25T10:11:57.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f590b7fe640 1 -- 172.21.15.60:0/1936851002 learned_addr learned my addr 172.21.15.60:0/1936851002 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:57.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f590affd640 1 --2- 172.21.15.60:0/1936851002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f590c06ec30 0x7f590c1f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f590a7fc640 1 --2- 172.21.15.60:0/1936851002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f590c0b37e0 0x7f590c157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f590b7fe640 1 -- 172.21.15.60:0/1936851002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f590c0b37e0 msgr2=0x7f590c157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f590b7fe640 1 --2- 172.21.15.60:0/1936851002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f590c0b37e0 0x7f590c157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f590b7fe640 1 -- 172.21.15.60:0/1936851002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f590c06ec30 msgr2=0x7f590c1f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f590b7fe640 1 --2- 172.21.15.60:0/1936851002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f590c06ec30 0x7f590c1f7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f590b7fe640 1 -- 172.21.15.60:0/1936851002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c1f7a00 con 0x7f590c15a7a0 2024-04-25T10:11:57.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f590b7fe640 1 --2- 172.21.15.60:0/1936851002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c15a7a0 0x7f590c0ce0a0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f590400c810 tx=0x7f590400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f58f37fe640 1 -- 172.21.15.60:0/1936851002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904018070 con 0x7f590c15a7a0 2024-04-25T10:11:57.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f58f37fe640 1 -- 172.21.15.60:0/1936851002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590400a640 con 0x7f590c15a7a0 2024-04-25T10:11:57.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f58f37fe640 1 -- 172.21.15.60:0/1936851002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590400aa30 con 0x7f590c15a7a0 2024-04-25T10:11:57.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f591174c640 1 -- 172.21.15.60:0/1936851002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590c157d60 con 0x7f590c15a7a0 2024-04-25T10:11:57.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f591174c640 1 -- 172.21.15.60:0/1936851002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590c1f7c70 con 0x7f590c15a7a0 2024-04-25T10:11:57.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:56.996+0000 7f591174c640 1 -- 172.21.15.60:0/1936851002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590c1f8160 con 0x7f590c15a7a0 2024-04-25T10:11:57.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.000+0000 7f58f37fe640 1 -- 172.21.15.60:0/1936851002 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f590401b070 con 0x7f590c15a7a0 2024-04-25T10:11:57.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.000+0000 7f58f37fe640 1 --2- 172.21.15.60:0/1936851002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f58f8066f60 0x7f58f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.000+0000 7f58f37fe640 1 -- 172.21.15.60:0/1936851002 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59040845b0 con 0x7f590c15a7a0 2024-04-25T10:11:57.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.000+0000 7f590affd640 1 --2- 172.21.15.60:0/1936851002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f58f8066f60 0x7f58f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.000+0000 7f590affd640 1 --2- 172.21.15.60:0/1936851002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f58f8066f60 0x7f58f8069420 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00b930 tx=0x7f58fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.004+0000 7f58f37fe640 1 -- 172.21.15.60:0/1936851002 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f590c157d60 con 0x7f590c15a7a0 2024-04-25T10:11:57.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.172+0000 7f591174c640 1 -- 172.21.15.60:0/1936851002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f590c06de70 con 0x7f590c15a7a0 2024-04-25T10:11:57.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.172+0000 7f58f37fe640 1 -- 172.21.15.60:0/1936851002 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v82) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5904055650 con 0x7f590c15a7a0 2024-04-25T10:11:57.180 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:11:57.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.172+0000 7f591174c640 1 -- 172.21.15.60:0/1936851002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f58f8066f60 msgr2=0x7f58f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.172+0000 7f591174c640 1 --2- 172.21.15.60:0/1936851002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f58f8066f60 0x7f58f8069420 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00b930 tx=0x7f58fc008040 comp rx=0 tx=0).stop 2024-04-25T10:11:57.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.172+0000 7f591174c640 1 -- 172.21.15.60:0/1936851002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c15a7a0 msgr2=0x7f590c0ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.172+0000 7f591174c640 1 --2- 172.21.15.60:0/1936851002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c15a7a0 0x7f590c0ce0a0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f590400c810 tx=0x7f590400ccd0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.172+0000 7f591174c640 1 -- 172.21.15.60:0/1936851002 shutdown_connections 2024-04-25T10:11:57.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.172+0000 7f591174c640 1 --2- 172.21.15.60:0/1936851002 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f58f8066f60 0x7f58f8069420 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.172+0000 7f591174c640 1 --2- 172.21.15.60:0/1936851002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c15a7a0 0x7f590c0ce0a0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.172+0000 7f591174c640 1 --2- 172.21.15.60:0/1936851002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f590c0b37e0 0x7f590c157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.172+0000 7f591174c640 1 --2- 172.21.15.60:0/1936851002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f590c06ec30 0x7f590c1f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.172+0000 7f591174c640 1 -- 172.21.15.60:0/1936851002 >> 172.21.15.60:0/1936851002 conn(0x7f590c0b12d0 msgr2=0x7f590c164560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.172+0000 7f591174c640 1 -- 172.21.15.60:0/1936851002 shutdown_connections 2024-04-25T10:11:57.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.172+0000 7f591174c640 1 -- 172.21.15.60:0/1936851002 wait complete. 2024-04-25T10:11:57.197 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:11:57.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:57.198 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:57.208 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[45379]: fuse finished with error 0 and tester_r 0 2024-04-25T10:11:57.210 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:57.267 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:11:57.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:57.268 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:11:57.273 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:57.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.416+0000 7f4136c34640 1 Processor -- start 2024-04-25T10:11:57.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.416+0000 7f4136c34640 1 -- start start 2024-04-25T10:11:57.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f4136c34640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f413015dc90 0x7f41300b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f4136c34640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f41300b2b10 0x7f4130071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f4136c34640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4130071fd0 con 0x7f41300ae7c0 2024-04-25T10:11:57.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f4136c34640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41300accc0 con 0x7f41300b2b10 2024-04-25T10:11:57.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f4136c34640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41300ad260 con 0x7f413015dc90 2024-04-25T10:11:57.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f412ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f41300b2b10 0x7f4130071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f412ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f41300b2b10 0x7f4130071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47098/0 (socket says 172.21.15.60:47098) 2024-04-25T10:11:57.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f412ffff640 1 -- 172.21.15.60:0/1770256501 learned_addr learned my addr 172.21.15.60:0/1770256501 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:57.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f41349d0640 1 --2- 172.21.15.60:0/1770256501 >> v2:172.21.15.100:3300/0 conn(0x7f413015dc90 0x7f41300b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f412ffff640 1 -- 172.21.15.60:0/1770256501 >> v2:172.21.15.100:3300/0 conn(0x7f413015dc90 msgr2=0x7f41300b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f412ffff640 1 --2- 172.21.15.60:0/1770256501 >> v2:172.21.15.100:3300/0 conn(0x7f413015dc90 0x7f41300b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f412ffff640 1 -- 172.21.15.60:0/1770256501 >> v1:172.21.15.100:6789/0 conn(0x7f41300ae7c0 legacy=0x7f41300ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f412ffff640 1 -- 172.21.15.60:0/1770256501 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4130072290 con 0x7f41300b2b10 2024-04-25T10:11:57.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f412ffff640 1 --2- 172.21.15.60:0/1770256501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41300b2b10 0x7f4130071290 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f4130070f80 tx=0x7f412401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f860368987bbf127 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f412f7fe640 1 -- 172.21.15.60:0/1770256501 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41240ab5f0 con 0x7f41300b2b10 2024-04-25T10:11:57.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f412f7fe640 1 -- 172.21.15.60:0/1770256501 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41240b0070 con 0x7f41300b2b10 2024-04-25T10:11:57.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f412f7fe640 1 -- 172.21.15.60:0/1770256501 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41240ab9b0 con 0x7f41300b2b10 2024-04-25T10:11:57.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f4136c34640 1 -- 172.21.15.60:0/1770256501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41300b2b10 msgr2=0x7f4130071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.420+0000 7f4136c34640 1 --2- 172.21.15.60:0/1770256501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41300b2b10 0x7f4130071290 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f4130070f80 tx=0x7f412401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 -- 172.21.15.60:0/1770256501 shutdown_connections 2024-04-25T10:11:57.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 --2- 172.21.15.60:0/1770256501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41300b2b10 0x7f4130071290 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 --2- 172.21.15.60:0/1770256501 >> v2:172.21.15.100:3300/0 conn(0x7f413015dc90 0x7f41300b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 -- 172.21.15.60:0/1770256501 >> 172.21.15.60:0/1770256501 conn(0x7f41300b12d0 msgr2=0x7f41300b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 -- 172.21.15.60:0/1770256501 shutdown_connections 2024-04-25T10:11:57.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 -- 172.21.15.60:0/1770256501 wait complete. 2024-04-25T10:11:57.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 Processor -- start 2024-04-25T10:11:57.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 -- start start 2024-04-25T10:11:57.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41300ae7c0 0x7f4130138e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f413015dc90 0x7f413014e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4130155340 0x7f41300b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4130071ba0 con 0x7f41300ae7c0 2024-04-25T10:11:57.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4130071a20 con 0x7f413015dc90 2024-04-25T10:11:57.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41300b2220 con 0x7f4130155340 2024-04-25T10:11:57.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f41349d0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41300ae7c0 0x7f4130138e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f41349d0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41300ae7c0 0x7f4130138e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47100/0 (socket says 172.21.15.60:47100) 2024-04-25T10:11:57.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f41349d0640 1 -- 172.21.15.60:0/1109647556 learned_addr learned my addr 172.21.15.60:0/1109647556 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:57.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f412ffff640 1 --2- 172.21.15.60:0/1109647556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f413015dc90 0x7f413014e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f41349d0640 1 -- 172.21.15.60:0/1109647556 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4130155340 msgr2=0x7f41300b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f41349d0640 1 --2- 172.21.15.60:0/1109647556 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4130155340 0x7f41300b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f41349d0640 1 -- 172.21.15.60:0/1109647556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f413015dc90 msgr2=0x7f413014e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f41349d0640 1 --2- 172.21.15.60:0/1109647556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f413015dc90 0x7f413014e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f41349d0640 1 -- 172.21.15.60:0/1109647556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413014ea90 con 0x7f41300ae7c0 2024-04-25T10:11:57.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f41349d0640 1 --2- 172.21.15.60:0/1109647556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41300ae7c0 0x7f4130138e40 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4120007f20 tx=0x7f412000e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f412d7fa640 1 -- 172.21.15.60:0/1109647556 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120014620 con 0x7f41300ae7c0 2024-04-25T10:11:57.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f412d7fa640 1 -- 172.21.15.60:0/1109647556 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4120019070 con 0x7f41300ae7c0 2024-04-25T10:11:57.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 -- 172.21.15.60:0/1109647556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4130150210 con 0x7f41300ae7c0 2024-04-25T10:11:57.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f412d7fa640 1 -- 172.21.15.60:0/1109647556 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120014a00 con 0x7f41300ae7c0 2024-04-25T10:11:57.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 -- 172.21.15.60:0/1109647556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41301507d0 con 0x7f41300ae7c0 2024-04-25T10:11:57.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.424+0000 7f4136c34640 1 -- 172.21.15.60:0/1109647556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4130074770 con 0x7f41300ae7c0 2024-04-25T10:11:57.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.428+0000 7f412d7fa640 1 -- 172.21.15.60:0/1109647556 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4120014ba0 con 0x7f41300ae7c0 2024-04-25T10:11:57.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.428+0000 7f412d7fa640 1 --2- 172.21.15.60:0/1109647556 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f411c066f90 0x7f411c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.428+0000 7f412d7fa640 1 -- 172.21.15.60:0/1109647556 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41200858f0 con 0x7f41300ae7c0 2024-04-25T10:11:57.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.428+0000 7f412ffff640 1 --2- 172.21.15.60:0/1109647556 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f411c066f90 0x7f411c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.428+0000 7f412ffff640 1 --2- 172.21.15.60:0/1109647556 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f411c066f90 0x7f411c069450 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f41240af000 tx=0x7f4124008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.432+0000 7f412d7fa640 1 -- 172.21.15.60:0/1109647556 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4120056900 con 0x7f41300ae7c0 2024-04-25T10:11:57.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.628+0000 7f4136c34640 1 -- 172.21.15.60:0/1109647556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f413015fd30 con 0x7f41300ae7c0 2024-04-25T10:11:57.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.628+0000 7f412d7fa640 1 -- 172.21.15.60:0/1109647556 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f4120056200 con 0x7f41300ae7c0 2024-04-25T10:11:57.637 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:57.637 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:11:47.412210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7653":{"gid":7653,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2455562778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2455562778},{"type":"v1","addr":"172.21.15.60:6835","nonce":2455562778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:57.637 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 55 2024-04-25T10:11:57.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.632+0000 7f4136c34640 1 -- 172.21.15.60:0/1109647556 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f411c066f90 msgr2=0x7f411c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.632+0000 7f4136c34640 1 --2- 172.21.15.60:0/1109647556 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f411c066f90 0x7f411c069450 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f41240af000 tx=0x7f4124008040 comp rx=0 tx=0).stop 2024-04-25T10:11:57.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.632+0000 7f4136c34640 1 -- 172.21.15.60:0/1109647556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41300ae7c0 msgr2=0x7f4130138e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.632+0000 7f4136c34640 1 --2- 172.21.15.60:0/1109647556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41300ae7c0 0x7f4130138e40 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4120007f20 tx=0x7f412000e570 comp rx=0 tx=0).stop 2024-04-25T10:11:57.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.632+0000 7f4136c34640 1 -- 172.21.15.60:0/1109647556 shutdown_connections 2024-04-25T10:11:57.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.632+0000 7f4136c34640 1 --2- 172.21.15.60:0/1109647556 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f411c066f90 0x7f411c069450 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.632+0000 7f4136c34640 1 --2- 172.21.15.60:0/1109647556 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4130155340 0x7f41300b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.632+0000 7f4136c34640 1 --2- 172.21.15.60:0/1109647556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f413015dc90 0x7f413014e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.632+0000 7f4136c34640 1 --2- 172.21.15.60:0/1109647556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41300ae7c0 0x7f4130138e40 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.632+0000 7f4136c34640 1 -- 172.21.15.60:0/1109647556 >> 172.21.15.60:0/1109647556 conn(0x7f41300b12d0 msgr2=0x7f413006dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.632+0000 7f4136c34640 1 -- 172.21.15.60:0/1109647556 shutdown_connections 2024-04-25T10:11:57.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.632+0000 7f4136c34640 1 -- 172.21.15.60:0/1109647556 wait complete. 2024-04-25T10:11:57.657 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:11:57.813 INFO:teuthology.orchestra.run.smithi060.stdout:{"message":"","return_code":0} 2024-04-25T10:11:57.814 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:11:57.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:11:57.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.928+0000 7f736ed25640 1 Processor -- start 2024-04-25T10:11:57.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.928+0000 7f736ed25640 1 -- start start 2024-04-25T10:11:57.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.928+0000 7f736ed25640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f736815a7a0 0x7f73680cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.928+0000 7f736ed25640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f736806ec30 0x7f7368071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.928+0000 7f736ed25640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73680721e0 con 0x7f73680b37e0 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.928+0000 7f736ed25640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73680cdaa0 con 0x7f736815a7a0 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.928+0000 7f736ed25640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f736806d2f0 con 0x7f736806ec30 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.928+0000 7f736cac1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f736815a7a0 0x7f73680cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.928+0000 7f736cac1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f736815a7a0 0x7f73680cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47108/0 (socket says 172.21.15.60:47108) 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.928+0000 7f736cac1640 1 -- 172.21.15.60:0/3524248144 learned_addr learned my addr 172.21.15.60:0/3524248144 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.928+0000 7f7367fff640 1 --2- 172.21.15.60:0/3524248144 >> v2:172.21.15.100:3300/0 conn(0x7f736806ec30 0x7f7368071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.928+0000 7f736cac1640 1 -- 172.21.15.60:0/3524248144 >> v2:172.21.15.100:3300/0 conn(0x7f736806ec30 msgr2=0x7f7368071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.928+0000 7f736cac1640 1 --2- 172.21.15.60:0/3524248144 >> v2:172.21.15.100:3300/0 conn(0x7f736806ec30 0x7f7368071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.928+0000 7f736cac1640 1 -- 172.21.15.60:0/3524248144 >> v1:172.21.15.100:6789/0 conn(0x7f73680b37e0 legacy=0x7f736806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.928+0000 7f736cac1640 1 -- 172.21.15.60:0/3524248144 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736806f130 con 0x7f736815a7a0 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.928+0000 7f7367fff640 1 --2- 172.21.15.60:0/3524248144 >> v2:172.21.15.100:3300/0 conn(0x7f736806ec30 0x7f7368071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:57.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736cac1640 1 --2- 172.21.15.60:0/3524248144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 0x7f73680cd560 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f7368072bf0 tx=0x7f735801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275ee0e2ee81d64d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f73677fe640 1 -- 172.21.15.60:0/3524248144 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73580a5780 con 0x7f736815a7a0 2024-04-25T10:11:57.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f73677fe640 1 -- 172.21.15.60:0/3524248144 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73580a5920 con 0x7f736815a7a0 2024-04-25T10:11:57.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f73677fe640 1 -- 172.21.15.60:0/3524248144 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73580a5cf0 con 0x7f736815a7a0 2024-04-25T10:11:57.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736ed25640 1 -- 172.21.15.60:0/3524248144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 msgr2=0x7f73680cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736ed25640 1 --2- 172.21.15.60:0/3524248144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 0x7f73680cd560 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f7368072bf0 tx=0x7f735801c9b0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736ed25640 1 -- 172.21.15.60:0/3524248144 shutdown_connections 2024-04-25T10:11:57.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736ed25640 1 --2- 172.21.15.60:0/3524248144 >> v2:172.21.15.100:3300/0 conn(0x7f736806ec30 0x7f7368071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736ed25640 1 --2- 172.21.15.60:0/3524248144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 0x7f73680cd560 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736ed25640 1 -- 172.21.15.60:0/3524248144 >> 172.21.15.60:0/3524248144 conn(0x7f73680b12d0 msgr2=0x7f73680b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736ed25640 1 -- 172.21.15.60:0/3524248144 shutdown_connections 2024-04-25T10:11:57.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736ed25640 1 -- 172.21.15.60:0/3524248144 wait complete. 2024-04-25T10:11:57.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736ed25640 1 Processor -- start 2024-04-25T10:11:57.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736ed25640 1 -- start start 2024-04-25T10:11:57.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736ed25640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f736806ec30 0x7f736814c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736ed25640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680b37e0 0x7f736814ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736ed25640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 0x7f7368155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736ed25640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736815ad00 con 0x7f736815a7a0 2024-04-25T10:11:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736ed25640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736815ab80 con 0x7f73680b37e0 2024-04-25T10:11:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736ed25640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736806e860 con 0x7f736806ec30 2024-04-25T10:11:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736d2c2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 0x7f7368155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736d2c2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 0x7f7368155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47120/0 (socket says 172.21.15.60:47120) 2024-04-25T10:11:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.932+0000 7f736d2c2640 1 -- 172.21.15.60:0/2738264970 learned_addr learned my addr 172.21.15.60:0/2738264970 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f7367fff640 1 --2- 172.21.15.60:0/2738264970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680b37e0 0x7f736814ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f736cac1640 1 --2- 172.21.15.60:0/2738264970 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f736806ec30 0x7f736814c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f736d2c2640 1 -- 172.21.15.60:0/2738264970 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f736806ec30 msgr2=0x7f736814c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f736d2c2640 1 --2- 172.21.15.60:0/2738264970 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f736806ec30 0x7f736814c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f736d2c2640 1 -- 172.21.15.60:0/2738264970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680b37e0 msgr2=0x7f736814ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f736d2c2640 1 --2- 172.21.15.60:0/2738264970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680b37e0 0x7f736814ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f736d2c2640 1 -- 172.21.15.60:0/2738264970 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73681511d0 con 0x7f736815a7a0 2024-04-25T10:11:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f736d2c2640 1 --2- 172.21.15.60:0/2738264970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 0x7f7368155d50 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f736000c3c0 tx=0x7f736000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f73657fa640 1 -- 172.21.15.60:0/2738264970 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360011790 con 0x7f736815a7a0 2024-04-25T10:11:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f73657fa640 1 -- 172.21.15.60:0/2738264970 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7360017070 con 0x7f736815a7a0 2024-04-25T10:11:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f736ed25640 1 -- 172.21.15.60:0/2738264970 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7368150930 con 0x7f736815a7a0 2024-04-25T10:11:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f73657fa640 1 -- 172.21.15.60:0/2738264970 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360011b70 con 0x7f736815a7a0 2024-04-25T10:11:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f736ed25640 1 -- 172.21.15.60:0/2738264970 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7368150ef0 con 0x7f736815a7a0 2024-04-25T10:11:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f736ed25640 1 -- 172.21.15.60:0/2738264970 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f736815ea60 con 0x7f736815a7a0 2024-04-25T10:11:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f73657fa640 1 -- 172.21.15.60:0/2738264970 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7360011d10 con 0x7f736815a7a0 2024-04-25T10:11:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f73657fa640 1 --2- 172.21.15.60:0/2738264970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7354066f90 0x7f7354069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f73657fa640 1 -- 172.21.15.60:0/2738264970 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7360084d50 con 0x7f736815a7a0 2024-04-25T10:11:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f736cac1640 1 --2- 172.21.15.60:0/2738264970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7354066f90 0x7f7354069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.936+0000 7f736cac1640 1 --2- 172.21.15.60:0/2738264970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7354066f90 0x7f7354069450 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f73580ae000 tx=0x7f73580a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:57.940+0000 7f73657fa640 1 -- 172.21.15.60:0/2738264970 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7360088030 con 0x7f736815a7a0 2024-04-25T10:11:58.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.132+0000 7f736ed25640 1 -- 172.21.15.60:0/2738264970 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f73680cef80 con 0x7f736815a7a0 2024-04-25T10:11:58.140 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:11:58.132+0000 7f3a792d6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:11:58.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.132+0000 7f73657fa640 1 -- 172.21.15.60:0/2738264970 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) v1 ==== 161+0+0 (secure 0 0 0) 0x7f73681511d0 con 0x7f736815a7a0 2024-04-25T10:11:58.142 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:11:58.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.140+0000 7f736ed25640 1 -- 172.21.15.60:0/2738264970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7354066f90 msgr2=0x7f7354069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.140+0000 7f736ed25640 1 --2- 172.21.15.60:0/2738264970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7354066f90 0x7f7354069450 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f73580ae000 tx=0x7f73580a7070 comp rx=0 tx=0).stop 2024-04-25T10:11:58.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.140+0000 7f736ed25640 1 -- 172.21.15.60:0/2738264970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 msgr2=0x7f7368155d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.140+0000 7f736ed25640 1 --2- 172.21.15.60:0/2738264970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 0x7f7368155d50 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f736000c3c0 tx=0x7f736000c880 comp rx=0 tx=0).stop 2024-04-25T10:11:58.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.140+0000 7f736ed25640 1 -- 172.21.15.60:0/2738264970 shutdown_connections 2024-04-25T10:11:58.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.140+0000 7f736ed25640 1 --2- 172.21.15.60:0/2738264970 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7354066f90 0x7f7354069450 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.140+0000 7f736ed25640 1 --2- 172.21.15.60:0/2738264970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f736815a7a0 0x7f7368155d50 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.140+0000 7f736ed25640 1 --2- 172.21.15.60:0/2738264970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73680b37e0 0x7f736814ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.140+0000 7f736ed25640 1 --2- 172.21.15.60:0/2738264970 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f736806ec30 0x7f736814c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.140+0000 7f736ed25640 1 -- 172.21.15.60:0/2738264970 >> 172.21.15.60:0/2738264970 conn(0x7f73680b12d0 msgr2=0x7f736815ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:58.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.140+0000 7f736ed25640 1 -- 172.21.15.60:0/2738264970 shutdown_connections 2024-04-25T10:11:58.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.140+0000 7f736ed25640 1 -- 172.21.15.60:0/2738264970 wait complete. 2024-04-25T10:11:58.149 INFO:tasks.ceph.mds.c.smithi060.stderr: -3895> 2024-04-25T10:11:45.968+0000 7f0ae804b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:58.164 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-25T10:11:58.186 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:11:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.184+0000 7f8883e4a080 1 Processor -- start 2024-04-25T10:11:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.184+0000 7f8883e4a080 1 -- start start 2024-04-25T10:11:58.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5573e4cc8dc0 0x5573e4da5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5573e4d12060 0x5573e4cf7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573e4d12440 0x5573e4da2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573e4cc8140 con 0x5573e4cc8dc0 2024-04-25T10:11:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573e4c6e9f0 con 0x5573e4d12440 2024-04-25T10:11:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573e4cc83e0 con 0x5573e4d12060 2024-04-25T10:11:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881968640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5573e4cc8dc0 0x5573e4da5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881968640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5573e4cc8dc0 0x5573e4da5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47162/0 (socket says 172.21.15.60:47162) 2024-04-25T10:11:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881968640 1 -- 172.21.15.60:0/3238574196 learned_addr learned my addr 172.21.15.60:0/3238574196 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881968640 1 -- 172.21.15.60:0/3238574196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5573e4d12060 msgr2=0x5573e4cf7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881968640 1 --2- 172.21.15.60:0/3238574196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5573e4d12060 0x5573e4cf7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881968640 1 -- 172.21.15.60:0/3238574196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573e4d12440 msgr2=0x5573e4da2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881968640 1 --2- 172.21.15.60:0/3238574196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573e4d12440 0x5573e4da2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881968640 1 -- 172.21.15.60:0/3238574196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573e4da6300 con 0x5573e4cc8dc0 2024-04-25T10:11:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881968640 1 --2- 172.21.15.60:0/3238574196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5573e4cc8dc0 0x5573e4da5b80 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f88740ae040 tx=0x7f8874003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900167285833ea77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8873fff640 1 -- 172.21.15.60:0/3238574196 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88740b0040 con 0x5573e4cc8dc0 2024-04-25T10:11:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8873fff640 1 -- 172.21.15.60:0/3238574196 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88740ba070 con 0x5573e4cc8dc0 2024-04-25T10:11:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8873fff640 1 -- 172.21.15.60:0/3238574196 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88740ab960 con 0x5573e4cc8dc0 2024-04-25T10:11:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 -- 172.21.15.60:0/3238574196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5573e4cc8dc0 msgr2=0x5573e4da5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 --2- 172.21.15.60:0/3238574196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5573e4cc8dc0 0x5573e4da5b80 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f88740ae040 tx=0x7f8874003030 comp rx=0 tx=0).stop 2024-04-25T10:11:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 -- 172.21.15.60:0/3238574196 shutdown_connections 2024-04-25T10:11:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 --2- 172.21.15.60:0/3238574196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573e4d12440 0x5573e4da2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 --2- 172.21.15.60:0/3238574196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5573e4d12060 0x5573e4cf7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 --2- 172.21.15.60:0/3238574196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5573e4cc8dc0 0x5573e4da5b80 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 -- 172.21.15.60:0/3238574196 >> 172.21.15.60:0/3238574196 conn(0x5573e4cf3f20 msgr2=0x5573e4d869a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 -- 172.21.15.60:0/3238574196 shutdown_connections 2024-04-25T10:11:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 -- 172.21.15.60:0/3238574196 wait complete. 2024-04-25T10:11:58.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 Processor -- start 2024-04-25T10:11:58.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 -- start start 2024-04-25T10:11:58.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5573e4cc8dc0 0x5573e4d867f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5573e4d12060 0x5573e4d81470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573e4d12440 0x5573e4d819b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573e4da65a0 con 0x5573e4d12060 2024-04-25T10:11:58.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573e4da60c0 con 0x5573e4d12440 2024-04-25T10:11:58.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573e4da6840 con 0x5573e4cc8dc0 2024-04-25T10:11:58.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881167640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5573e4d12060 0x5573e4d81470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881167640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5573e4d12060 0x5573e4d81470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47168/0 (socket says 172.21.15.60:47168) 2024-04-25T10:11:58.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881167640 1 -- 172.21.15.60:0/1142168031 learned_addr learned my addr 172.21.15.60:0/1142168031 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:58.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8882169640 1 --2- 172.21.15.60:0/1142168031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573e4d12440 0x5573e4d819b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881167640 1 -- 172.21.15.60:0/1142168031 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5573e4cc8dc0 msgr2=0x5573e4d867f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881167640 1 --2- 172.21.15.60:0/1142168031 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5573e4cc8dc0 0x5573e4d867f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881167640 1 -- 172.21.15.60:0/1142168031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573e4d12440 msgr2=0x5573e4d819b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881167640 1 --2- 172.21.15.60:0/1142168031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5573e4d12440 0x5573e4d819b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881167640 1 -- 172.21.15.60:0/1142168031 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573e4d91630 con 0x5573e4d12060 2024-04-25T10:11:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8881167640 1 --2- 172.21.15.60:0/1142168031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5573e4d12060 0x5573e4d81470 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f88780063e0 tx=0x7f88780068a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8872ffd640 1 -- 172.21.15.60:0/1142168031 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8878017070 con 0x5573e4d12060 2024-04-25T10:11:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8872ffd640 1 -- 172.21.15.60:0/1142168031 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8878012410 con 0x5573e4d12060 2024-04-25T10:11:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 -- 172.21.15.60:0/1142168031 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5573e4d918d0 con 0x5573e4d12060 2024-04-25T10:11:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8872ffd640 1 -- 172.21.15.60:0/1142168031 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8878012800 con 0x5573e4d12060 2024-04-25T10:11:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.188+0000 7f8883e4a080 1 -- 172.21.15.60:0/1142168031 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573e4d91e10 con 0x5573e4d12060 2024-04-25T10:11:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.192+0000 7f8872ffd640 1 -- 172.21.15.60:0/1142168031 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f887801a070 con 0x5573e4d12060 2024-04-25T10:11:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.192+0000 7f8872ffd640 1 --2- 172.21.15.60:0/1142168031 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8864065970 0x7f8864067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.192+0000 7f8872ffd640 1 -- 172.21.15.60:0/1142168031 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f88780830c0 con 0x5573e4d12060 2024-04-25T10:11:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.192+0000 7f8883e4a080 1 --2- 172.21.15.60:0/1142168031 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5573e4d86d30 0x5573e4e2ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.192+0000 7f8883e4a080 1 -- 172.21.15.60:0/1142168031 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e83) v9 -- 0x5573e4e2f450 con 0x5573e4d86d30 2024-04-25T10:11:58.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.192+0000 7f8882169640 1 --2- 172.21.15.60:0/1142168031 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5573e4d86d30 0x5573e4e2ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.192+0000 7f8881968640 1 --2- 172.21.15.60:0/1142168031 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8864065970 0x7f8864067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.192+0000 7f8882169640 1 --2- 172.21.15.60:0/1142168031 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5573e4d86d30 0x5573e4e2ef10 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.192+0000 7f8881968640 1 --2- 172.21.15.60:0/1142168031 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8864065970 0x7f8864067e30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f88740b9000 tx=0x7f8874005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.196+0000 7f8882169640 1 -- 172.21.15.60:0/1142168031 <== osd.0 v2:172.21.15.60:6801/4056167511 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v83'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f887c00b4a0 con 0x5573e4d86d30 2024-04-25T10:11:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.196+0000 7f8883e4a080 1 -- 172.21.15.60:0/1142168031 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5573e4d86d30 msgr2=0x5573e4e2ef10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.196+0000 7f8883e4a080 1 --2- 172.21.15.60:0/1142168031 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5573e4d86d30 0x5573e4e2ef10 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.196+0000 7f8883e4a080 1 -- 172.21.15.60:0/1142168031 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8864065970 msgr2=0x7f8864067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.196+0000 7f8883e4a080 1 --2- 172.21.15.60:0/1142168031 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8864065970 0x7f8864067e30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f88740b9000 tx=0x7f8874005ad0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.196+0000 7f8883e4a080 1 -- 172.21.15.60:0/1142168031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5573e4d12060 msgr2=0x5573e4d81470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.196+0000 7f8883e4a080 1 --2- 172.21.15.60:0/1142168031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5573e4d12060 0x5573e4d81470 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f88780063e0 tx=0x7f88780068a0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.196+0000 7f8882169640 1 -- 172.21.15.60:0/1142168031 reap_dead start 2024-04-25T10:11:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.196+0000 7f8883e4a080 1 -- 172.21.15.60:0/1142168031 shutdown_connections 2024-04-25T10:11:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.196+0000 7f8883e4a080 1 -- 172.21.15.60:0/1142168031 >> 172.21.15.60:0/1142168031 conn(0x5573e4cf3f20 msgr2=0x5573e4cc91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.196+0000 7f8883e4a080 1 -- 172.21.15.60:0/1142168031 shutdown_connections 2024-04-25T10:11:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.196+0000 7f8883e4a080 1 -- 172.21.15.60:0/1142168031 wait complete. 2024-04-25T10:11:58.208 DEBUG:teuthology.orchestra.run.smithi060:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-25T10:11:58.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a676dd400 1 -- start start 2024-04-25T10:11:58.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a676dd400 1 Processor -- start 2024-04-25T10:11:58.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a676dd400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604afb7000 0x55604b170580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a676dd400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604afb7400 0x55604b170b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a676dd400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604afb7800 0x55604b171080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a676dd400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604bd80180 con 0x55604afb7000 2024-04-25T10:11:58.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a676dd400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604bd80300 con 0x55604afb7800 2024-04-25T10:11:58.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a676dd400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55604bd80480 con 0x55604afb7400 2024-04-25T10:11:58.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a65e9c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604afb7000 0x55604b170580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a65e9c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604afb7000 0x55604b170580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47192/0 (socket says 172.21.15.60:47192) 2024-04-25T10:11:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a65e9c640 1 -- 172.21.15.60:0/339467835 learned_addr learned my addr 172.21.15.60:0/339467835 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a65e9c640 1 -- 172.21.15.60:0/339467835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604afb7400 msgr2=0x55604b170b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a6669d640 1 --2- 172.21.15.60:0/339467835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604afb7800 0x55604b171080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a6569b640 1 --2- 172.21.15.60:0/339467835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604afb7400 0x55604b170b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a65e9c640 1 --2- 172.21.15.60:0/339467835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604afb7400 0x55604b170b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a65e9c640 1 -- 172.21.15.60:0/339467835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604afb7800 msgr2=0x55604b171080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a65e9c640 1 --2- 172.21.15.60:0/339467835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604afb7800 0x55604b171080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a65e9c640 1 -- 172.21.15.60:0/339467835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55604b076b40 con 0x55604afb7000 2024-04-25T10:11:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a6569b640 1 --2- 172.21.15.60:0/339467835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604afb7400 0x55604b170b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a65e9c640 1 --2- 172.21.15.60:0/339467835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604afb7000 0x55604b170580 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x55604be213e0 tx=0x55604bd82000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5d946533ffa753 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a63697640 1 -- 172.21.15.60:0/339467835 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55604afec9c0 con 0x55604afb7000 2024-04-25T10:11:58.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a63697640 1 -- 172.21.15.60:0/339467835 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55604b0001c0 con 0x55604afb7000 2024-04-25T10:11:58.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a63697640 1 -- 172.21.15.60:0/339467835 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55604afec4e0 con 0x55604afb7000 2024-04-25T10:11:58.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a676dd400 1 -- 172.21.15.60:0/339467835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55604b076d20 con 0x55604afb7000 2024-04-25T10:11:58.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a676dd400 1 -- 172.21.15.60:0/339467835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55604b076f00 con 0x55604afb7000 2024-04-25T10:11:58.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a676dd400 4 waiting for MDS map... 2024-04-25T10:11:58.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a63697640 1 -- 172.21.15.60:0/339467835 <== mon.0 v2:172.21.15.60:3300/0 4 ==== osd_map(83..83 src has 1..83) v4 ==== 5795+0+0 (secure 0 0 0) 0x55604b075a00 con 0x55604afb7000 2024-04-25T10:11:58.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a63697640 1 -- 172.21.15.60:0/339467835 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 56) v1 ==== 2723+0+0 (secure 0 0 0) 0x55604b00d180 con 0x55604afb7000 2024-04-25T10:11:58.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a676dd400 4 Got MDS map 56 2024-04-25T10:11:58.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a676dd400 10 main: JournalTool::main 2024-04-25T10:11:58.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.272+0000 7f9a676dd400 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:11:58.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 Processor -- start 2024-04-25T10:11:58.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 -- start start 2024-04-25T10:11:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604bd9b800 0x55604b171600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604bd9bc00 0x55604b171b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bf36000 0x55604b172100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604bd80600 con 0x55604bf36000 2024-04-25T10:11:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604bd80780 con 0x55604bd9bc00 2024-04-25T10:11:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55604bd80900 con 0x55604bd9b800 2024-04-25T10:11:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6569b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bf36000 0x55604b172100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6569b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bf36000 0x55604b172100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47208/0 (socket says 172.21.15.60:47208) 2024-04-25T10:11:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6569b640 1 -- 172.21.15.60:0/133141351 learned_addr learned my addr 172.21.15.60:0/133141351 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a65e9c640 1 --2- 172.21.15.60:0/133141351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604bd9bc00 0x55604b171b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6669d640 1 --2- 172.21.15.60:0/133141351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604bd9b800 0x55604b171600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6569b640 1 -- 172.21.15.60:0/133141351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604bd9b800 msgr2=0x55604b171600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6569b640 1 --2- 172.21.15.60:0/133141351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604bd9b800 0x55604b171600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6569b640 1 -- 172.21.15.60:0/133141351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604bd9bc00 msgr2=0x55604b171b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6569b640 1 --2- 172.21.15.60:0/133141351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604bd9bc00 0x55604b171b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6569b640 1 -- 172.21.15.60:0/133141351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55604b0761e0 con 0x55604bf36000 2024-04-25T10:11:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6569b640 1 --2- 172.21.15.60:0/133141351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bf36000 0x55604b172100 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x55604bedc4b0 tx=0x55604bf3e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce300ddc313afb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a61693640 1 -- 172.21.15.60:0/133141351 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55604afec9c0 con 0x55604bf36000 2024-04-25T10:11:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a61693640 1 -- 172.21.15.60:0/133141351 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55604b000e00 con 0x55604bf36000 2024-04-25T10:11:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a61693640 1 -- 172.21.15.60:0/133141351 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55604afec4e0 con 0x55604bf36000 2024-04-25T10:11:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 -- 172.21.15.60:0/133141351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bf36000 msgr2=0x55604b172100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 --2- 172.21.15.60:0/133141351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bf36000 0x55604b172100 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x55604bedc4b0 tx=0x55604bf3e0c0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 -- 172.21.15.60:0/133141351 shutdown_connections 2024-04-25T10:11:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 --2- 172.21.15.60:0/133141351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bf36000 0x55604b172100 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 --2- 172.21.15.60:0/133141351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604bd9bc00 0x55604b171b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 --2- 172.21.15.60:0/133141351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604bd9b800 0x55604b171600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 -- 172.21.15.60:0/133141351 >> 172.21.15.60:0/133141351 conn(0x55604bd9b400 msgr2=0x55604afe4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:58.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 -- 172.21.15.60:0/133141351 shutdown_connections 2024-04-25T10:11:58.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 -- 172.21.15.60:0/133141351 wait complete. 2024-04-25T10:11:58.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 Processor -- start 2024-04-25T10:11:58.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 -- start start 2024-04-25T10:11:58.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bd9bc00 0x55604b173180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604bd9b800 0x55604b172c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604bf37000 0x55604b172680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604bd81c80 con 0x55604bd9bc00 2024-04-25T10:11:58.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604bd81b00 con 0x55604bf37000 2024-04-25T10:11:58.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a676dd400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55604bd81980 con 0x55604bd9b800 2024-04-25T10:11:58.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6669d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bd9bc00 0x55604b173180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6669d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bd9bc00 0x55604b173180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47220/0 (socket says 172.21.15.60:47220) 2024-04-25T10:11:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6669d640 1 -- 172.21.15.60:0/3954295862 learned_addr learned my addr 172.21.15.60:0/3954295862 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6569b640 1 --2- 172.21.15.60:0/3954295862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604bf37000 0x55604b172680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a65e9c640 1 --2- 172.21.15.60:0/3954295862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604bd9b800 0x55604b172c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6669d640 1 -- 172.21.15.60:0/3954295862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604bd9b800 msgr2=0x55604b172c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6669d640 1 --2- 172.21.15.60:0/3954295862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604bd9b800 0x55604b172c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6669d640 1 -- 172.21.15.60:0/3954295862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604bf37000 msgr2=0x55604b172680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6669d640 1 --2- 172.21.15.60:0/3954295862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604bf37000 0x55604b172680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6669d640 1 -- 172.21.15.60:0/3954295862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55604b077a40 con 0x55604bd9bc00 2024-04-25T10:11:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6569b640 1 --2- 172.21.15.60:0/3954295862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604bf37000 0x55604b172680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.276+0000 7f9a6669d640 1 --2- 172.21.15.60:0/3954295862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bd9bc00 0x55604b173180 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x55604bedd7a0 tx=0x55604bf3e1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a5fe90640 1 -- 172.21.15.60:0/3954295862 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55604afec9c0 con 0x55604bd9bc00 2024-04-25T10:11:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a5fe90640 1 -- 172.21.15.60:0/3954295862 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55604b000c40 con 0x55604bd9bc00 2024-04-25T10:11:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a676dd400 1 -- 172.21.15.60:0/3954295862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55604b077c20 con 0x55604bd9bc00 2024-04-25T10:11:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a5fe90640 1 -- 172.21.15.60:0/3954295862 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55604afec4e0 con 0x55604bd9bc00 2024-04-25T10:11:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a676dd400 1 -- 172.21.15.60:0/3954295862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55604b077a40 con 0x55604bd9bc00 2024-04-25T10:11:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a676dd400 4 main: JournalTool: resolving pool 12 2024-04-25T10:11:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a5fe90640 1 -- 172.21.15.60:0/3954295862 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x55604bf2cc00 con 0x55604bd9bc00 2024-04-25T10:11:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a5fe90640 1 --2- 172.21.15.60:0/3954295862 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55604b2f2c00 0x55604b173700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a5fe90640 1 -- 172.21.15.60:0/3954295862 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5795+0+0 (secure 0 0 0) 0x55604b2f4a00 con 0x55604bd9bc00 2024-04-25T10:11:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a676dd400 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:11:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a676dd400 4 main: Executing for rank 0 2024-04-25T10:11:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a65e9c640 1 --2- 172.21.15.60:0/3954295862 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55604b2f2c00 0x55604b173700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a676dd400 1 -- start start 2024-04-25T10:11:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a676dd400 1 Processor -- start 2024-04-25T10:11:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a65e9c640 1 --2- 172.21.15.60:0/3954295862 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55604b2f2c00 0x55604b173700 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55604be218c0 tx=0x55604bd823c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a676dd400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bf36800 0x55604b172100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a676dd400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604bf36400 0x55604b171b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a676dd400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604bf36000 0x55604b171600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a676dd400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604bd81680 con 0x55604bf36800 2024-04-25T10:11:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a676dd400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604bd80600 con 0x55604bf36400 2024-04-25T10:11:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a676dd400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55604bd80780 con 0x55604bf36000 2024-04-25T10:11:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a6669d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bf36800 0x55604b172100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a6669d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bf36800 0x55604b172100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47228/0 (socket says 172.21.15.60:47228) 2024-04-25T10:11:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a6669d640 1 -- 172.21.15.60:0/3900161176 learned_addr learned my addr 172.21.15.60:0/3900161176 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a65e9c640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604bf36400 0x55604b171b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a6569b640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604bf36000 0x55604b171600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a6669d640 1 -- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604bf36000 msgr2=0x55604b171600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a6669d640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604bf36000 0x55604b171600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a6669d640 1 -- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604bf36400 msgr2=0x55604b171b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a6669d640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604bf36400 0x55604b171b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a6669d640 1 -- 172.21.15.60:0/3900161176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55604b426780 con 0x55604bf36800 2024-04-25T10:11:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a6569b640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604bf36000 0x55604b171600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a6669d640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bf36800 0x55604b172100 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x55604b3893b0 tx=0x55604bf3eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0a485b48034095 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a5ce8a640 1 -- 172.21.15.60:0/3900161176 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55604b36fba0 con 0x55604bf36800 2024-04-25T10:11:58.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a5ce8a640 1 -- 172.21.15.60:0/3900161176 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55604b000a80 con 0x55604bf36800 2024-04-25T10:11:58.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a5ce8a640 1 -- 172.21.15.60:0/3900161176 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55604b36fd40 con 0x55604bf36800 2024-04-25T10:11:58.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55604b426960 con 0x55604bf36800 2024-04-25T10:11:58.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55604b426b40 con 0x55604bf36800 2024-04-25T10:11:58.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.280+0000 7f9a676dd400 4 waiting for MDS map... 2024-04-25T10:11:58.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.284+0000 7f9a5ce8a640 1 -- 172.21.15.60:0/3900161176 <== mon.0 v2:172.21.15.60:3300/0 4 ==== osd_map(83..83 src has 1..83) v4 ==== 5795+0+0 (secure 0 0 0) 0x55604b43c200 con 0x55604bf36800 2024-04-25T10:11:58.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.284+0000 7f9a5ce8a640 1 -- 172.21.15.60:0/3900161176 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 56) v1 ==== 2723+0+0 (secure 0 0 0) 0x55604b444000 con 0x55604bf36800 2024-04-25T10:11:58.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.284+0000 7f9a676dd400 4 Got MDS map 56 2024-04-25T10:11:58.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.284+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x55604b43b000 0x55604b450000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.284+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83) v9 -- 0x55604b43b400 con 0x55604b43b000 2024-04-25T10:11:58.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.284+0000 7f9a65e9c640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x55604b43b000 0x55604b450000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.284+0000 7f9a65e9c640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x55604b43b000 0x55604b450000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=8e0cf8edb323da67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.288+0000 7f9a65e9c640 1 -- 172.21.15.60:0/3900161176 <== osd.6 v2:172.21.15.100:6824/4086390550 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x55604b132000 con 0x55604b43b000 2024-04-25T10:11:58.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.288+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55604b43b800 0x55604b450580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.288+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:2 12.14 12:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83) v9 -- 0x55604b43bc00 con 0x55604b43b800 2024-04-25T10:11:58.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.292+0000 7f9a6569b640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55604b43b800 0x55604b450580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.292+0000 7f9a6569b640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55604b43b800 0x55604b450580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=52ba3d89b754c359 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.292+0000 7f9a6569b640 1 -- 172.21.15.60:0/3900161176 <== osd.5 v2:172.21.15.100:6800/3542120824 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv19 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x55604b132500 con 0x55604b43b800 2024-04-25T10:11:58.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.292+0000 7f9a5b687640 1 -- 172.21.15.60:0/3900161176 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:3 12.14 12:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83) v9 -- 0x55604bf37c00 con 0x55604b43b800 2024-04-25T10:11:58.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.292+0000 7f9a5b687640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55604bf37400 0x55604b452680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.292+0000 7f9a5b687640 1 -- 172.21.15.60:0/3900161176 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:4 12.1b 12:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83) v9 -- 0x55604bf37800 con 0x55604bf37400 2024-04-25T10:11:58.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.292+0000 7f9a6669d640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55604bf37400 0x55604b452680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.292+0000 7f9a6569b640 1 -- 172.21.15.60:0/3900161176 <== osd.5 v2:172.21.15.100:6800/3542120824 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x55604b132500 con 0x55604b43b800 2024-04-25T10:11:58.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.292+0000 7f9a6669d640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55604bf37400 0x55604b452680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=b66f8adbd04f7feb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.292+0000 7f9a6669d640 1 -- 172.21.15.60:0/3900161176 <== osd.4 v2:172.21.15.100:6808/3965541654 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55604b132780 con 0x55604bf37400 2024-04-25T10:11:58.303 INFO:teuthology.orchestra.run.smithi060.stdout:old journal was 4208696~2030 2024-04-25T10:11:58.303 INFO:teuthology.orchestra.run.smithi060.stdout:new journal start will be 8388608 (4177882 bytes past old end) 2024-04-25T10:11:58.303 INFO:teuthology.orchestra.run.smithi060.stdout:writing journal head 2024-04-25T10:11:58.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.292+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:5 12.14 12:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) v9 -- 0x55604b43bc00 con 0x55604b43b800 2024-04-25T10:11:58.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.296+0000 7f9a6569b640 1 -- 172.21.15.60:0/3900161176 <== osd.5 v2:172.21.15.100:6800/3542120824 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v83'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55604b132500 con 0x55604b43b800 2024-04-25T10:11:58.304 INFO:teuthology.orchestra.run.smithi060.stdout:writing EResetJournal entry 2024-04-25T10:11:58.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.296+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:6 12.1b 12:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) v9 -- 0x55604b46a000 con 0x55604bf37400 2024-04-25T10:11:58.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.296+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55604b46a400 0x55604b450b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.296+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:7 12.b 12:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) v9 -- 0x55604b46a800 con 0x55604b46a400 2024-04-25T10:11:58.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.296+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:8 12.1f 12:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) v9 -- 0x55604b46ac00 con 0x55604b43b000 2024-04-25T10:11:58.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.296+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:9 12.1a 12:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) v9 -- 0x55604b46b000 con 0x55604bf37400 2024-04-25T10:11:58.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.296+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:10 12.1e 12:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) v9 -- 0x55604b46b400 con 0x55604bf37400 2024-04-25T10:11:58.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.296+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x55604b46b800 0x55604b451080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.296+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:11 12.1d 12:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) v9 -- 0x55604b46bc00 con 0x55604b46b800 2024-04-25T10:11:58.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.296+0000 7f9a6569b640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x55604b46b800 0x55604b451080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.296+0000 7f9a65e9c640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55604b46a400 0x55604b450b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.296+0000 7f9a6569b640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x55604b46b800 0x55604b451080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=7ece9fcc6381a0ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.296+0000 7f9a65e9c640 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55604b46a400 0x55604b450b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=dfdd81558f39dafb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.296+0000 7f9a6569b640 1 -- 172.21.15.60:0/3900161176 <== osd.0 v2:172.21.15.60:6801/4056167511 1 ==== osd_op_reply(11 200.00000007 [delete] v83'3 uv2 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55604b132500 con 0x55604b46b800 2024-04-25T10:11:58.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a65e9c640 1 -- 172.21.15.60:0/3900161176 <== osd.3 v2:172.21.15.60:6814/1867957156 1 ==== osd_op_reply(7 200.00000003 [delete] v83'4 uv3 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55604b132000 con 0x55604b46a400 2024-04-25T10:11:58.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a6669d640 1 -- 172.21.15.60:0/3900161176 <== osd.4 v2:172.21.15.100:6808/3965541654 2 ==== osd_op_reply(6 200.00000002 [delete] v83'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55604b132780 con 0x55604bf37400 2024-04-25T10:11:58.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a5b687640 1 -- 172.21.15.60:0/3900161176 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:12 12.1b 12:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) v9 -- 0x55604b2f3c00 con 0x55604bf37400 2024-04-25T10:11:58.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a6669d640 1 -- 172.21.15.60:0/3900161176 <== osd.4 v2:172.21.15.100:6808/3965541654 3 ==== osd_op_reply(9 200.00000005 [delete] v83'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55604b132780 con 0x55604bf37400 2024-04-25T10:11:58.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a6669d640 1 -- 172.21.15.60:0/3900161176 <== osd.4 v2:172.21.15.100:6808/3965541654 4 ==== osd_op_reply(10 200.00000006 [delete] v83'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55604b132780 con 0x55604bf37400 2024-04-25T10:11:58.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a65e9c640 1 -- 172.21.15.60:0/3900161176 <== osd.6 v2:172.21.15.100:6824/4086390550 2 ==== osd_op_reply(8 200.00000004 [delete] v83'7 uv6 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55604b132000 con 0x55604b43b000 2024-04-25T10:11:58.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a6669d640 1 -- 172.21.15.60:0/3900161176 <== osd.4 v2:172.21.15.100:6808/3965541654 5 ==== osd_op_reply(12 200.00000002 [write 0~48] v83'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55604b132780 con 0x55604bf37400 2024-04-25T10:11:58.309 INFO:teuthology.orchestra.run.smithi060.stdout:done 2024-04-25T10:11:58.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x55604b46b800 msgr2=0x55604b451080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x55604b46b800 0x55604b451080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55604b46a400 msgr2=0x55604b450b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55604b46a400 0x55604b450b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55604bf37400 msgr2=0x55604b452680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55604bf37400 0x55604b452680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55604b43b800 msgr2=0x55604b450580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55604b43b800 0x55604b450580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x55604b43b000 msgr2=0x55604b450000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a6569b640 1 -- 172.21.15.60:0/3900161176 reap_dead start 2024-04-25T10:11:58.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x55604b43b000 0x55604b450000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a6569b640 1 -- 172.21.15.60:0/3900161176 reap_dead start 2024-04-25T10:11:58.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bf36800 msgr2=0x55604b172100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bf36800 0x55604b172100 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x55604b3893b0 tx=0x55604bf3eb40 comp rx=0 tx=0).stop 2024-04-25T10:11:58.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 shutdown_connections 2024-04-25T10:11:58.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3900161176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bf36800 0x55604b172100 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 >> 172.21.15.60:0/3900161176 conn(0x55604bf36c00 msgr2=0x55604b422000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:58.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 shutdown_connections 2024-04-25T10:11:58.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/3900161176 wait complete. 2024-04-25T10:11:58.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/3954295862 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55604b2f2c00 msgr2=0x55604b173700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3954295862 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55604b2f2c00 0x55604b173700 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55604be218c0 tx=0x55604bd823c0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/3954295862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bd9bc00 msgr2=0x55604b173180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3954295862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bd9bc00 0x55604b173180 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x55604bedd7a0 tx=0x55604bf3e1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/3954295862 shutdown_connections 2024-04-25T10:11:58.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3954295862 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55604b2f2c00 0x55604b173700 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3954295862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604bf37000 0x55604b172680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3954295862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604bd9b800 0x55604b172c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/3954295862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604bd9bc00 0x55604b173180 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/3954295862 >> 172.21.15.60:0/3954295862 conn(0x55604bd9b400 msgr2=0x55604afe4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:58.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/3954295862 shutdown_connections 2024-04-25T10:11:58.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/3954295862 wait complete. 2024-04-25T10:11:58.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/339467835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604afb7000 msgr2=0x55604b170580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/339467835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604afb7000 0x55604b170580 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x55604be213e0 tx=0x55604bd82000 comp rx=0 tx=0).stop 2024-04-25T10:11:58.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/339467835 shutdown_connections 2024-04-25T10:11:58.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/339467835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55604afb7800 0x55604b171080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/339467835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55604afb7400 0x55604b170b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 --2- 172.21.15.60:0/339467835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55604afb7000 0x55604b170580 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.300+0000 7f9a676dd400 1 -- 172.21.15.60:0/339467835 >> 172.21.15.60:0/339467835 conn(0x55604afb6c00 msgr2=0x55604afe3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:58.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.304+0000 7f9a676dd400 1 -- 172.21.15.60:0/339467835 shutdown_connections 2024-04-25T10:11:58.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.304+0000 7f9a676dd400 1 -- 172.21.15.60:0/339467835 wait complete. 2024-04-25T10:11:58.318 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.109394, result: old journal was 4208696~2030 new journal start will be 8388608 (4177882 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-25T10:11:58.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:11:58.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd3fce640 1 Processor -- start 2024-04-25T10:11:58.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd3fce640 1 -- start start 2024-04-25T10:11:58.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd3fce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fcc0b1f30 0x7f7fcc15f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd3fce640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7fcc15f920 0x7f7fcc06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd3fce640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7fcc06b150 0x7f7fcc15bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd3fce640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fcc15fd60 con 0x7f7fcc15f920 2024-04-25T10:11:58.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd3fce640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fcc160230 con 0x7f7fcc06b150 2024-04-25T10:11:58.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd3fce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc06bad0 con 0x7f7fcc0b1f30 2024-04-25T10:11:58.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd1569640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7fcc15f920 0x7f7fcc06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd1569640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7fcc15f920 0x7f7fcc06ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47242/0 (socket says 172.21.15.60:47242) 2024-04-25T10:11:58.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd1569640 1 -- 172.21.15.60:0/2701344997 learned_addr learned my addr 172.21.15.60:0/2701344997 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:58.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd1d6a640 1 --2- 172.21.15.60:0/2701344997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fcc0b1f30 0x7f7fcc15f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd256b640 1 --2- 172.21.15.60:0/2701344997 >> v2:172.21.15.100:3300/0 conn(0x7f7fcc06b150 0x7f7fcc15bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd1569640 1 -- 172.21.15.60:0/2701344997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fcc0b1f30 msgr2=0x7f7fcc15f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd1569640 1 --2- 172.21.15.60:0/2701344997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fcc0b1f30 0x7f7fcc15f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd1569640 1 -- 172.21.15.60:0/2701344997 >> v2:172.21.15.100:3300/0 conn(0x7f7fcc06b150 msgr2=0x7f7fcc15bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd1569640 1 --2- 172.21.15.60:0/2701344997 >> v2:172.21.15.100:3300/0 conn(0x7f7fcc06b150 0x7f7fcc15bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd1569640 1 -- 172.21.15.60:0/2701344997 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc06b7f0 con 0x7f7fcc15f920 2024-04-25T10:11:58.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.480+0000 7f7fd1d6a640 1 --2- 172.21.15.60:0/2701344997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fcc0b1f30 0x7f7fcc15f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:58.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.484+0000 7f7fd1569640 1 --2- 172.21.15.60:0/2701344997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fcc15f920 0x7f7fcc06ac10 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc071950 tx=0x7f7fbc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e20d75963e5ee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.484+0000 7f7fd0d68640 1 -- 172.21.15.60:0/2701344997 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0b1070 con 0x7f7fcc15f920 2024-04-25T10:11:58.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.484+0000 7f7fd0d68640 1 -- 172.21.15.60:0/2701344997 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fbc0ac630 con 0x7f7fcc15f920 2024-04-25T10:11:58.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.484+0000 7f7fd0d68640 1 -- 172.21.15.60:0/2701344997 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0a8050 con 0x7f7fcc15f920 2024-04-25T10:11:58.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.484+0000 7f7fd3fce640 1 -- 172.21.15.60:0/2701344997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fcc15f920 msgr2=0x7f7fcc06ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.484+0000 7f7fd3fce640 1 --2- 172.21.15.60:0/2701344997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fcc15f920 0x7f7fcc06ac10 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc071950 tx=0x7f7fbc01c520 comp rx=0 tx=0).stop 2024-04-25T10:11:58.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.484+0000 7f7fd3fce640 1 -- 172.21.15.60:0/2701344997 shutdown_connections 2024-04-25T10:11:58.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.484+0000 7f7fd3fce640 1 --2- 172.21.15.60:0/2701344997 >> v2:172.21.15.100:3300/0 conn(0x7f7fcc06b150 0x7f7fcc15bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.484+0000 7f7fd3fce640 1 --2- 172.21.15.60:0/2701344997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fcc15f920 0x7f7fcc06ac10 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.484+0000 7f7fd3fce640 1 --2- 172.21.15.60:0/2701344997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fcc0b1f30 0x7f7fcc15f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.484+0000 7f7fd3fce640 1 -- 172.21.15.60:0/2701344997 >> 172.21.15.60:0/2701344997 conn(0x7f7fcc072900 msgr2=0x7f7fcc099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:58.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.484+0000 7f7fd3fce640 1 -- 172.21.15.60:0/2701344997 shutdown_connections 2024-04-25T10:11:58.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.484+0000 7f7fd3fce640 1 -- 172.21.15.60:0/2701344997 wait complete. 2024-04-25T10:11:58.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.484+0000 7f7fd3fce640 1 Processor -- start 2024-04-25T10:11:58.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.484+0000 7f7fd3fce640 1 -- start start 2024-04-25T10:11:58.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd3fce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fcc06b150 0x7f7fcc146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd3fce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fcc0b1f30 0x7f7fcc0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd3fce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fcc15f920 0x7f7fcc1fa730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd3fce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc15dc20 con 0x7f7fcc06b150 2024-04-25T10:11:58.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd3fce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc15daa0 con 0x7f7fcc15f920 2024-04-25T10:11:58.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd3fce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc15dda0 con 0x7f7fcc0b1f30 2024-04-25T10:11:58.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd1d6a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fcc06b150 0x7f7fcc146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd1d6a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fcc06b150 0x7f7fcc146e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47254/0 (socket says 172.21.15.60:47254) 2024-04-25T10:11:58.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd1d6a640 1 -- 172.21.15.60:0/3283263727 learned_addr learned my addr 172.21.15.60:0/3283263727 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:58.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd256b640 1 --2- 172.21.15.60:0/3283263727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fcc15f920 0x7f7fcc1fa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd1569640 1 --2- 172.21.15.60:0/3283263727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fcc0b1f30 0x7f7fcc0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd1d6a640 1 -- 172.21.15.60:0/3283263727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fcc0b1f30 msgr2=0x7f7fcc0b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd1d6a640 1 --2- 172.21.15.60:0/3283263727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fcc0b1f30 0x7f7fcc0b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd1d6a640 1 -- 172.21.15.60:0/3283263727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fcc15f920 msgr2=0x7f7fcc1fa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd1d6a640 1 --2- 172.21.15.60:0/3283263727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fcc15f920 0x7f7fcc1fa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd1d6a640 1 -- 172.21.15.60:0/3283263727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc0b42a0 con 0x7f7fcc06b150 2024-04-25T10:11:58.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd1d6a640 1 --2- 172.21.15.60:0/3283263727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fcc06b150 0x7f7fcc146e90 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400cf90 tx=0x7f7fc400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fc27fc640 1 -- 172.21.15.60:0/3283263727 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc401a070 con 0x7f7fcc06b150 2024-04-25T10:11:58.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fc27fc640 1 -- 172.21.15.60:0/3283263727 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc400d840 con 0x7f7fcc06b150 2024-04-25T10:11:58.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd3fce640 1 -- 172.21.15.60:0/3283263727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fcc0b4580 con 0x7f7fcc06b150 2024-04-25T10:11:58.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fc27fc640 1 -- 172.21.15.60:0/3283263727 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc400dc10 con 0x7f7fcc06b150 2024-04-25T10:11:58.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd3fce640 1 -- 172.21.15.60:0/3283263727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fcc1f9f30 con 0x7f7fcc06b150 2024-04-25T10:11:58.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd3fce640 1 -- 172.21.15.60:0/3283263727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fcc06ccb0 con 0x7f7fcc06b150 2024-04-25T10:11:58.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fc27fc640 1 -- 172.21.15.60:0/3283263727 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7fc401d070 con 0x7f7fcc06b150 2024-04-25T10:11:58.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fc27fc640 1 --2- 172.21.15.60:0/3283263727 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7fb0066f60 0x7f7fb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fc27fc640 1 -- 172.21.15.60:0/3283263727 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7fc40857e0 con 0x7f7fcc06b150 2024-04-25T10:11:58.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.488+0000 7f7fd1569640 1 --2- 172.21.15.60:0/3283263727 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7fb0066f60 0x7f7fb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.492+0000 7f7fd1569640 1 --2- 172.21.15.60:0/3283263727 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7fb0066f60 0x7f7fb0069420 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0ae000 tx=0x7f7fbc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.492+0000 7f7fc27fc640 1 -- 172.21.15.60:0/3283263727 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7fc40561d0 con 0x7f7fcc06b150 2024-04-25T10:11:58.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:58.688+0000 7f7fd3fce640 1 -- 172.21.15.60:0/3283263727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f7fcc074770 con 0x7f7fcc06b150 2024-04-25T10:11:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.136+0000 7f7fc27fc640 1 -- 172.21.15.60:0/3283263727 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v57) v1 ==== 162+0+0 (secure 0 0 0) 0x7f7fc405a270 con 0x7f7fcc06b150 2024-04-25T10:11:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:11:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.140+0000 7f7fd3fce640 1 -- 172.21.15.60:0/3283263727 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7fb0066f60 msgr2=0x7f7fb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.140+0000 7f7fd3fce640 1 --2- 172.21.15.60:0/3283263727 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7fb0066f60 0x7f7fb0069420 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0ae000 tx=0x7f7fbc008040 comp rx=0 tx=0).stop 2024-04-25T10:11:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.140+0000 7f7fd3fce640 1 -- 172.21.15.60:0/3283263727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fcc06b150 msgr2=0x7f7fcc146e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.140+0000 7f7fd3fce640 1 --2- 172.21.15.60:0/3283263727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fcc06b150 0x7f7fcc146e90 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400cf90 tx=0x7f7fc400a7d0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.140+0000 7f7fd3fce640 1 -- 172.21.15.60:0/3283263727 shutdown_connections 2024-04-25T10:11:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.140+0000 7f7fd3fce640 1 --2- 172.21.15.60:0/3283263727 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7fb0066f60 0x7f7fb0069420 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.140+0000 7f7fd3fce640 1 --2- 172.21.15.60:0/3283263727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fcc15f920 0x7f7fcc1fa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.140+0000 7f7fd3fce640 1 --2- 172.21.15.60:0/3283263727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fcc0b1f30 0x7f7fcc0b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.140+0000 7f7fd3fce640 1 --2- 172.21.15.60:0/3283263727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fcc06b150 0x7f7fcc146e90 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.140+0000 7f7fd3fce640 1 -- 172.21.15.60:0/3283263727 >> 172.21.15.60:0/3283263727 conn(0x7f7fcc072900 msgr2=0x7f7fcc0aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.140+0000 7f7fd3fce640 1 -- 172.21.15.60:0/3283263727 shutdown_connections 2024-04-25T10:11:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.140+0000 7f7fd3fce640 1 -- 172.21.15.60:0/3283263727 wait complete. 2024-04-25T10:11:59.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:59.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.284+0000 7fc792263640 1 Processor -- start 2024-04-25T10:11:59.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.284+0000 7fc792263640 1 -- start start 2024-04-25T10:11:59.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.284+0000 7fc792263640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc78c15bc90 0x7fc78c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.284+0000 7fc792263640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc78c071ca0 con 0x7fc78c0ce7a0 2024-04-25T10:11:59.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.284+0000 7fc792263640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc78c0cca40 con 0x7fc78c0b4660 2024-04-25T10:11:59.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.284+0000 7fc792263640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc78c0cc370 con 0x7fc78c15bc90 2024-04-25T10:11:59.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.284+0000 7fc78bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc78c0ce7a0 0x7fc78c0ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41552/0 (socket says 172.21.15.60:41552) 2024-04-25T10:11:59.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.284+0000 7fc78b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc78c15bc90 0x7fc78c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.284+0000 7fc78bfff640 1 -- 172.21.15.60:0/1925579824 learned_addr learned my addr 172.21.15.60:0/1925579824 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:11:59.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.284+0000 7fc78affd640 1 -- 172.21.15.60:0/1925579824 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675966471 0 0) 0x7fc78c071ca0 con 0x7fc78c0ce7a0 2024-04-25T10:11:59.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.284+0000 7fc78affd640 1 -- 172.21.15.60:0/1925579824 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc76c003530 con 0x7fc78c0ce7a0 2024-04-25T10:11:59.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.284+0000 7fc78b7fe640 1 -- 172.21.15.60:0/1925579824 >> v1:172.21.15.100:6789/0 conn(0x7fc78c0b4660 legacy=0x7fc78c15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.284+0000 7fc78b7fe640 1 -- 172.21.15.60:0/1925579824 >> v1:172.21.15.60:6789/0 conn(0x7fc78c0ce7a0 legacy=0x7fc78c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.284+0000 7fc78b7fe640 1 -- 172.21.15.60:0/1925579824 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c0ccd20 con 0x7fc78c15bc90 2024-04-25T10:11:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc78b7fe640 1 --2- 172.21.15.60:0/1925579824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c15bc90 0x7fc78c15c220 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fc774009d90 tx=0x7fc77401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411aa1f6f2953e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc78affd640 1 -- 172.21.15.60:0/1925579824 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7740b2070 con 0x7fc78c15bc90 2024-04-25T10:11:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc78affd640 1 -- 172.21.15.60:0/1925579824 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7740a5830 con 0x7fc78c15bc90 2024-04-25T10:11:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc78affd640 1 -- 172.21.15.60:0/1925579824 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7740a5c00 con 0x7fc78c15bc90 2024-04-25T10:11:59.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc792263640 1 -- 172.21.15.60:0/1925579824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c15bc90 msgr2=0x7fc78c15c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc792263640 1 --2- 172.21.15.60:0/1925579824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c15bc90 0x7fc78c15c220 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fc774009d90 tx=0x7fc77401c970 comp rx=0 tx=0).stop 2024-04-25T10:11:59.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc792263640 1 -- 172.21.15.60:0/1925579824 shutdown_connections 2024-04-25T10:11:59.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc792263640 1 --2- 172.21.15.60:0/1925579824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c15bc90 0x7fc78c15c220 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc792263640 1 -- 172.21.15.60:0/1925579824 >> 172.21.15.60:0/1925579824 conn(0x7fc78c0b12d0 msgr2=0x7fc78c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:59.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc792263640 1 -- 172.21.15.60:0/1925579824 shutdown_connections 2024-04-25T10:11:59.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc792263640 1 -- 172.21.15.60:0/1925579824 wait complete. 2024-04-25T10:11:59.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc792263640 1 Processor -- start 2024-04-25T10:11:59.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc792263640 1 -- start start 2024-04-25T10:11:59.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc792263640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c0ce7a0 0x7fc78c149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc792263640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc78c15bc90 0x7fc78c14a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc792263640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78c14ae50 0x7fc78c14ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc792263640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c1695d0 con 0x7fc78c0ce7a0 2024-04-25T10:11:59.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc792263640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c0b3db0 con 0x7fc78c15bc90 2024-04-25T10:11:59.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc792263640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c169750 con 0x7fc78c14ae50 2024-04-25T10:11:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc78bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c0ce7a0 0x7fc78c149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc78bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c0ce7a0 0x7fc78c149dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53562/0 (socket says 172.21.15.60:53562) 2024-04-25T10:11:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc78bfff640 1 -- 172.21.15.60:0/3752300036 learned_addr learned my addr 172.21.15.60:0/3752300036 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:11:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc78b7fe640 1 --2- 172.21.15.60:0/3752300036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc78c15bc90 0x7fc78c14a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc790800640 1 --2- 172.21.15.60:0/3752300036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78c14ae50 0x7fc78c14ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc78bfff640 1 -- 172.21.15.60:0/3752300036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78c14ae50 msgr2=0x7fc78c14ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc78bfff640 1 --2- 172.21.15.60:0/3752300036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78c14ae50 0x7fc78c14ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc78bfff640 1 -- 172.21.15.60:0/3752300036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc78c15bc90 msgr2=0x7fc78c14a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc78bfff640 1 --2- 172.21.15.60:0/3752300036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc78c15bc90 0x7fc78c14a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc78bfff640 1 -- 172.21.15.60:0/3752300036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c1596a0 con 0x7fc78c0ce7a0 2024-04-25T10:11:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc78b7fe640 1 --2- 172.21.15.60:0/3752300036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc78c15bc90 0x7fc78c14a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc78bfff640 1 --2- 172.21.15.60:0/3752300036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c0ce7a0 0x7fc78c149dd0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00c420 tx=0x7fc77c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc788ff9640 1 -- 172.21.15.60:0/3752300036 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c017070 con 0x7fc78c0ce7a0 2024-04-25T10:11:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.288+0000 7fc788ff9640 1 -- 172.21.15.60:0/3752300036 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c005cf0 con 0x7fc78c0ce7a0 2024-04-25T10:11:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.292+0000 7fc792263640 1 -- 172.21.15.60:0/3752300036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78c14d700 con 0x7fc78c0ce7a0 2024-04-25T10:11:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.292+0000 7fc788ff9640 1 -- 172.21.15.60:0/3752300036 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c0072d0 con 0x7fc78c0ce7a0 2024-04-25T10:11:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.292+0000 7fc792263640 1 -- 172.21.15.60:0/3752300036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78c159940 con 0x7fc78c0ce7a0 2024-04-25T10:11:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.292+0000 7fc792263640 1 -- 172.21.15.60:0/3752300036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78c14b260 con 0x7fc78c0ce7a0 2024-04-25T10:11:59.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.292+0000 7fc788ff9640 1 -- 172.21.15.60:0/3752300036 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc77c01a070 con 0x7fc78c0ce7a0 2024-04-25T10:11:59.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.292+0000 7fc788ff9640 1 --2- 172.21.15.60:0/3752300036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc770066f60 0x7fc770069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.292+0000 7fc788ff9640 1 -- 172.21.15.60:0/3752300036 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc77c084400 con 0x7fc78c0ce7a0 2024-04-25T10:11:59.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.292+0000 7fc78b7fe640 1 --2- 172.21.15.60:0/3752300036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc770066f60 0x7fc770069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.292+0000 7fc78b7fe640 1 --2- 172.21.15.60:0/3752300036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc770066f60 0x7fc770069420 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc7740af000 tx=0x7fc774004ac0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.296+0000 7fc788ff9640 1 -- 172.21.15.60:0/3752300036 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc78c14b260 con 0x7fc78c0ce7a0 2024-04-25T10:11:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.488+0000 7fc792263640 1 -- 172.21.15.60:0/3752300036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc78c074770 con 0x7fc78c0ce7a0 2024-04-25T10:11:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.488+0000 7fc788ff9640 1 -- 172.21.15.60:0/3752300036 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fc77c054cc0 con 0x7fc78c0ce7a0 2024-04-25T10:11:59.498 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:11:59.499 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:11:59.146880+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7908":{"gid":7908,"name":"c","rank":0,"incarnation":58,"state":"up:replay","state_seq":1,"addr":"172.21.15.60:6835/2817879567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2817879567},{"type":"v1","addr":"172.21.15.60:6835","nonce":2817879567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 58 2024-04-25T10:11:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.492+0000 7fc792263640 1 -- 172.21.15.60:0/3752300036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc770066f60 msgr2=0x7fc770069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.492+0000 7fc792263640 1 --2- 172.21.15.60:0/3752300036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc770066f60 0x7fc770069420 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc7740af000 tx=0x7fc774004ac0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.492+0000 7fc792263640 1 -- 172.21.15.60:0/3752300036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c0ce7a0 msgr2=0x7fc78c149dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.492+0000 7fc792263640 1 --2- 172.21.15.60:0/3752300036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c0ce7a0 0x7fc78c149dd0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00c420 tx=0x7fc77c00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.492+0000 7fc792263640 1 -- 172.21.15.60:0/3752300036 shutdown_connections 2024-04-25T10:11:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.492+0000 7fc792263640 1 --2- 172.21.15.60:0/3752300036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc770066f60 0x7fc770069420 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.492+0000 7fc792263640 1 --2- 172.21.15.60:0/3752300036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78c14ae50 0x7fc78c14ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.492+0000 7fc792263640 1 --2- 172.21.15.60:0/3752300036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc78c15bc90 0x7fc78c14a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.492+0000 7fc792263640 1 --2- 172.21.15.60:0/3752300036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78c0ce7a0 0x7fc78c149dd0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.492+0000 7fc792263640 1 -- 172.21.15.60:0/3752300036 >> 172.21.15.60:0/3752300036 conn(0x7fc78c0b12d0 msgr2=0x7fc78c15c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.492+0000 7fc792263640 1 -- 172.21.15.60:0/3752300036 shutdown_connections 2024-04-25T10:11:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:11:59.492+0000 7fc792263640 1 -- 172.21.15.60:0/3752300036 wait complete. 2024-04-25T10:11:59.918 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:44.357874+0000', 'modified': '2024-04-25T10:11:59.146880+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7908}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7908': {'gid': 7908, 'name': 'c', 'rank': 0, 'incarnation': 58, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.60:6835/2817879567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 2817879567}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 2817879567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:59.918 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7908:up:replay 2024-04-25T10:12:00.920 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:01.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa19e1b1640 1 Processor -- start 2024-04-25T10:12:01.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa19e1b1640 1 -- start start 2024-04-25T10:12:01.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa19e1b1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa1980b6c10 0x7fa1980ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa19e1b1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa19806d200 con 0x7fa1980d0a50 2024-04-25T10:12:01.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa19e1b1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1980cefe0 con 0x7fa19815deb0 2024-04-25T10:12:01.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa19e1b1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1980cf2c0 con 0x7fa1980b6c10 2024-04-25T10:12:01.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa196ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa1980d0a50 0x7fa1980ce2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41554/0 (socket says 172.21.15.60:41554) 2024-04-25T10:12:01.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa196ffd640 1 -- 172.21.15.60:0/3703235951 learned_addr learned my addr 172.21.15.60:0/3703235951 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:12:01.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa197fff640 1 --2- 172.21.15.60:0/3703235951 >> v2:172.21.15.100:3300/0 conn(0x7fa1980b6c10 0x7fa1980ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa1967fc640 1 -- 172.21.15.60:0/3703235951 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017803642 0 0) 0x7fa19806d200 con 0x7fa1980d0a50 2024-04-25T10:12:01.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa1967fc640 1 -- 172.21.15.60:0/3703235951 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa180003530 con 0x7fa1980d0a50 2024-04-25T10:12:01.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa1967fc640 1 -- 172.21.15.60:0/3703235951 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4054910172 0 0) 0x7fa180003530 con 0x7fa1980d0a50 2024-04-25T10:12:01.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa1967fc640 1 -- 172.21.15.60:0/3703235951 >> v2:172.21.15.100:3300/0 conn(0x7fa1980b6c10 msgr2=0x7fa1980ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:01.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa1967fc640 1 --2- 172.21.15.60:0/3703235951 >> v2:172.21.15.100:3300/0 conn(0x7fa1980b6c10 0x7fa1980ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa1967fc640 1 -- 172.21.15.60:0/3703235951 >> v1:172.21.15.100:6789/0 conn(0x7fa19815deb0 legacy=0x7fa1980d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa1967fc640 1 -- 172.21.15.60:0/3703235951 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1980d1d70 con 0x7fa1980d0a50 2024-04-25T10:12:01.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa1967fc640 1 -- 172.21.15.60:0/3703235951 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fa18c001a00 con 0x7fa1980d0a50 2024-04-25T10:12:01.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa1967fc640 1 -- 172.21.15.60:0/3703235951 >> v1:172.21.15.60:6789/0 conn(0x7fa1980d0a50 legacy=0x7fa1980ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa1967fc640 1 --2- 172.21.15.60:0/3703235951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1800039c0 0x7fa180003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa1967fc640 1 --2- 172.21.15.60:0/3703235951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa180004810 0x7fa180004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa1967fc640 1 --2- 172.21.15.60:0/3703235951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1800051d0 0x7fa18000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa1967fc640 1 -- 172.21.15.60:0/3703235951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980cf2c0 con 0x7fa1800051d0 2024-04-25T10:12:01.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa1967fc640 1 -- 172.21.15.60:0/3703235951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18000c070 con 0x7fa1800039c0 2024-04-25T10:12:01.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.048+0000 7fa1967fc640 1 -- 172.21.15.60:0/3703235951 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18000c270 con 0x7fa180004810 2024-04-25T10:12:01.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa197fff640 1 --2- 172.21.15.60:0/3703235951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1800051d0 0x7fa18000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa1977fe640 1 --2- 172.21.15.60:0/3703235951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1800039c0 0x7fa180003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa196ffd640 1 --2- 172.21.15.60:0/3703235951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa180004810 0x7fa180004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa197fff640 1 -- 172.21.15.60:0/3703235951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa180004810 msgr2=0x7fa180004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:01.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa197fff640 1 --2- 172.21.15.60:0/3703235951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa180004810 0x7fa180004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa197fff640 1 -- 172.21.15.60:0/3703235951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1800039c0 msgr2=0x7fa180003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:01.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa197fff640 1 --2- 172.21.15.60:0/3703235951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1800039c0 0x7fa180003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa197fff640 1 -- 172.21.15.60:0/3703235951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa180003530 con 0x7fa1800051d0 2024-04-25T10:12:01.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa1977fe640 1 --2- 172.21.15.60:0/3703235951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1800039c0 0x7fa180003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa197fff640 1 -- 172.21.15.60:0/3703235951 reap_dead start 2024-04-25T10:12:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa197fff640 1 --2- 172.21.15.60:0/3703235951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1800051d0 0x7fa18000b9d0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa190009ae0 tx=0x7fa19001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d8d37290e40ffa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa1967fc640 1 -- 172.21.15.60:0/3703235951 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1900096b0 con 0x7fa1800051d0 2024-04-25T10:12:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa1967fc640 1 -- 172.21.15.60:0/3703235951 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa190004020 con 0x7fa1800051d0 2024-04-25T10:12:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa1967fc640 1 -- 172.21.15.60:0/3703235951 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa190004410 con 0x7fa1800051d0 2024-04-25T10:12:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa19e1b1640 1 -- 172.21.15.60:0/3703235951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1800051d0 msgr2=0x7fa18000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa19e1b1640 1 --2- 172.21.15.60:0/3703235951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1800051d0 0x7fa18000b9d0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa190009ae0 tx=0x7fa19001cab0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa19e1b1640 1 -- 172.21.15.60:0/3703235951 shutdown_connections 2024-04-25T10:12:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa19e1b1640 1 --2- 172.21.15.60:0/3703235951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1800051d0 0x7fa18000b9d0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa19e1b1640 1 -- 172.21.15.60:0/3703235951 >> 172.21.15.60:0/3703235951 conn(0x7fa19806c090 msgr2=0x7fa19806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa19e1b1640 1 -- 172.21.15.60:0/3703235951 shutdown_connections 2024-04-25T10:12:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.052+0000 7fa19e1b1640 1 -- 172.21.15.60:0/3703235951 wait complete. 2024-04-25T10:12:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa19e1b1640 1 Processor -- start 2024-04-25T10:12:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa19e1b1640 1 -- start start 2024-04-25T10:12:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa19e1b1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1800051d0 0x7fa1980c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa19e1b1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa198160e60 0x7fa1981f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa19e1b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1980c8b60 0x7fa1981f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa19e1b1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980d0340 con 0x7fa198160e60 2024-04-25T10:12:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa19e1b1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980d2330 con 0x7fa1980c8b60 2024-04-25T10:12:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa19e1b1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980d04c0 con 0x7fa1800051d0 2024-04-25T10:12:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa196ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa198160e60 0x7fa1981f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa196ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa198160e60 0x7fa1981f6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53590/0 (socket says 172.21.15.60:53590) 2024-04-25T10:12:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa196ffd640 1 -- 172.21.15.60:0/3664660259 learned_addr learned my addr 172.21.15.60:0/3664660259 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa197fff640 1 --2- 172.21.15.60:0/3664660259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1980c8b60 0x7fa1981f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa1977fe640 1 --2- 172.21.15.60:0/3664660259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1800051d0 0x7fa1980c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa196ffd640 1 -- 172.21.15.60:0/3664660259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1800051d0 msgr2=0x7fa1980c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa196ffd640 1 --2- 172.21.15.60:0/3664660259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1800051d0 0x7fa1980c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa196ffd640 1 -- 172.21.15.60:0/3664660259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1980c8b60 msgr2=0x7fa1981f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa196ffd640 1 --2- 172.21.15.60:0/3664660259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1980c8b60 0x7fa1981f6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa196ffd640 1 -- 172.21.15.60:0/3664660259 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1980c90e0 con 0x7fa198160e60 2024-04-25T10:12:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa197fff640 1 --2- 172.21.15.60:0/3664660259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1980c8b60 0x7fa1981f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa196ffd640 1 --2- 172.21.15.60:0/3664660259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa198160e60 0x7fa1981f6aa0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0079a0 tx=0x7fa18c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa17ffff640 1 -- 172.21.15.60:0/3664660259 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18c00ee50 con 0x7fa198160e60 2024-04-25T10:12:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa17ffff640 1 -- 172.21.15.60:0/3664660259 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18c018070 con 0x7fa198160e60 2024-04-25T10:12:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa19e1b1640 1 -- 172.21.15.60:0/3664660259 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1980c93c0 con 0x7fa198160e60 2024-04-25T10:12:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa17ffff640 1 -- 172.21.15.60:0/3664660259 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18c013660 con 0x7fa198160e60 2024-04-25T10:12:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa19e1b1640 1 -- 172.21.15.60:0/3664660259 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1980c9980 con 0x7fa198160e60 2024-04-25T10:12:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.056+0000 7fa19e1b1640 1 -- 172.21.15.60:0/3664660259 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa198074770 con 0x7fa198160e60 2024-04-25T10:12:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.060+0000 7fa17ffff640 1 -- 172.21.15.60:0/3664660259 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa18c013800 con 0x7fa198160e60 2024-04-25T10:12:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.060+0000 7fa17ffff640 1 --2- 172.21.15.60:0/3664660259 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa184066f90 0x7fa184069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.060+0000 7fa17ffff640 1 -- 172.21.15.60:0/3664660259 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa18c0856e0 con 0x7fa198160e60 2024-04-25T10:12:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.060+0000 7fa1977fe640 1 --2- 172.21.15.60:0/3664660259 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa184066f90 0x7fa184069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.060+0000 7fa1977fe640 1 --2- 172.21.15.60:0/3664660259 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa184066f90 0x7fa184069450 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa188005450 tx=0x7fa188009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.064+0000 7fa17ffff640 1 -- 172.21.15.60:0/3664660259 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa18c056690 con 0x7fa198160e60 2024-04-25T10:12:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.260+0000 7fa19e1b1640 1 -- 172.21.15.60:0/3664660259 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1980b8210 con 0x7fa198160e60 2024-04-25T10:12:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.260+0000 7fa17ffff640 1 -- 172.21.15.60:0/3664660259 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fa18c055f90 con 0x7fa198160e60 2024-04-25T10:12:01.270 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:12:01.271 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:12:00.157610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7908":{"gid":7908,"name":"c","rank":0,"incarnation":58,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.60:6835/2817879567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2817879567},{"type":"v1","addr":"172.21.15.60:6835","nonce":2817879567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:12:01.271 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 60 2024-04-25T10:12:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.260+0000 7fa19e1b1640 1 -- 172.21.15.60:0/3664660259 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa184066f90 msgr2=0x7fa184069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.260+0000 7fa19e1b1640 1 --2- 172.21.15.60:0/3664660259 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa184066f90 0x7fa184069450 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa188005450 tx=0x7fa188009040 comp rx=0 tx=0).stop 2024-04-25T10:12:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.260+0000 7fa19e1b1640 1 -- 172.21.15.60:0/3664660259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa198160e60 msgr2=0x7fa1981f6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.260+0000 7fa19e1b1640 1 --2- 172.21.15.60:0/3664660259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa198160e60 0x7fa1981f6aa0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0079a0 tx=0x7fa18c00e530 comp rx=0 tx=0).stop 2024-04-25T10:12:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.264+0000 7fa19e1b1640 1 -- 172.21.15.60:0/3664660259 shutdown_connections 2024-04-25T10:12:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.264+0000 7fa19e1b1640 1 --2- 172.21.15.60:0/3664660259 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa184066f90 0x7fa184069450 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.264+0000 7fa19e1b1640 1 --2- 172.21.15.60:0/3664660259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1980c8b60 0x7fa1981f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.264+0000 7fa19e1b1640 1 --2- 172.21.15.60:0/3664660259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa198160e60 0x7fa1981f6aa0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.264+0000 7fa19e1b1640 1 --2- 172.21.15.60:0/3664660259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1800051d0 0x7fa1980c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.264+0000 7fa19e1b1640 1 -- 172.21.15.60:0/3664660259 >> 172.21.15.60:0/3664660259 conn(0x7fa19806c090 msgr2=0x7fa198157790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.264+0000 7fa19e1b1640 1 -- 172.21.15.60:0/3664660259 shutdown_connections 2024-04-25T10:12:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:01.264+0000 7fa19e1b1640 1 -- 172.21.15.60:0/3664660259 wait complete. 2024-04-25T10:12:01.690 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:44.357874+0000', 'modified': '2024-04-25T10:12:00.157610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7908}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7908': {'gid': 7908, 'name': 'c', 'rank': 0, 'incarnation': 58, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.60:6835/2817879567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 2817879567}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 2817879567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:12:01.691 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7908:up:rejoin 2024-04-25T10:12:02.691 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.816+0000 7fae8a93b640 1 Processor -- start 2024-04-25T10:12:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.816+0000 7fae8a93b640 1 -- start start 2024-04-25T10:12:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae8a93b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fae840b4660 0x7fae8415c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae8a93b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae84071ca0 con 0x7fae8415bc90 2024-04-25T10:12:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae8a93b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae840cca40 con 0x7fae840ce7a0 2024-04-25T10:12:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae8a93b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae840cc370 con 0x7fae840b4660 2024-04-25T10:12:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae7bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fae8415bc90 0x7fae8415c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41560/0 (socket says 172.21.15.60:41560) 2024-04-25T10:12:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae7bfff640 1 -- 172.21.15.60:0/576861158 learned_addr learned my addr 172.21.15.60:0/576861158 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:12:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae88ed8640 1 --2- 172.21.15.60:0/576861158 >> v2:172.21.15.100:3300/0 conn(0x7fae840b4660 0x7fae8415c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae837fe640 1 -- 172.21.15.60:0/576861158 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386402843 0 0) 0x7fae84071ca0 con 0x7fae8415bc90 2024-04-25T10:12:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae837fe640 1 -- 172.21.15.60:0/576861158 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae70003530 con 0x7fae8415bc90 2024-04-25T10:12:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae837fe640 1 -- 172.21.15.60:0/576861158 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1583610467 0 0) 0x7fae70003530 con 0x7fae8415bc90 2024-04-25T10:12:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae837fe640 1 -- 172.21.15.60:0/576861158 >> v2:172.21.15.100:3300/0 conn(0x7fae840b4660 msgr2=0x7fae8415c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae837fe640 1 --2- 172.21.15.60:0/576861158 >> v2:172.21.15.100:3300/0 conn(0x7fae840b4660 0x7fae8415c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae837fe640 1 -- 172.21.15.60:0/576861158 >> v1:172.21.15.100:6789/0 conn(0x7fae840ce7a0 legacy=0x7fae840ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae837fe640 1 -- 172.21.15.60:0/576861158 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae840ccd20 con 0x7fae8415bc90 2024-04-25T10:12:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae837fe640 1 -- 172.21.15.60:0/576861158 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fae6c001680 con 0x7fae8415bc90 2024-04-25T10:12:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae837fe640 1 -- 172.21.15.60:0/576861158 >> v1:172.21.15.60:6789/0 conn(0x7fae8415bc90 legacy=0x7fae8415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae837fe640 1 --2- 172.21.15.60:0/576861158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae700039c0 0x7fae70003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae837fe640 1 --2- 172.21.15.60:0/576861158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae70004810 0x7fae70004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae837fe640 1 --2- 172.21.15.60:0/576861158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae700051d0 0x7fae7000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae837fe640 1 -- 172.21.15.60:0/576861158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7000c040 con 0x7fae700051d0 2024-04-25T10:12:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae837fe640 1 -- 172.21.15.60:0/576861158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7000c1f0 con 0x7fae70004810 2024-04-25T10:12:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae837fe640 1 -- 172.21.15.60:0/576861158 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7000c3f0 con 0x7fae700039c0 2024-04-25T10:12:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae88ed8640 1 --2- 172.21.15.60:0/576861158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae700051d0 0x7fae7000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae83fff640 1 --2- 172.21.15.60:0/576861158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae700039c0 0x7fae70003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae7bfff640 1 --2- 172.21.15.60:0/576861158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae70004810 0x7fae70004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae88ed8640 1 -- 172.21.15.60:0/576861158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae700039c0 msgr2=0x7fae70003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae88ed8640 1 --2- 172.21.15.60:0/576861158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae700039c0 0x7fae70003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae88ed8640 1 -- 172.21.15.60:0/576861158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae70004810 msgr2=0x7fae70004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae88ed8640 1 --2- 172.21.15.60:0/576861158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae70004810 0x7fae70004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae88ed8640 1 -- 172.21.15.60:0/576861158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae70003530 con 0x7fae700051d0 2024-04-25T10:12:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.820+0000 7fae83fff640 1 --2- 172.21.15.60:0/576861158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae700039c0 0x7fae70003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae88ed8640 1 -- 172.21.15.60:0/576861158 reap_dead start 2024-04-25T10:12:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae88ed8640 1 --2- 172.21.15.60:0/576861158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae700051d0 0x7fae7000b9d0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009f90 tx=0x7fae7c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd358c13983c344 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae837fe640 1 -- 172.21.15.60:0/576861158 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c0b2070 con 0x7fae700051d0 2024-04-25T10:12:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae837fe640 1 -- 172.21.15.60:0/576861158 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7c0a5c90 con 0x7fae700051d0 2024-04-25T10:12:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae837fe640 1 -- 172.21.15.60:0/576861158 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c009c10 con 0x7fae700051d0 2024-04-25T10:12:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae8a93b640 1 -- 172.21.15.60:0/576861158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae700051d0 msgr2=0x7fae7000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae8a93b640 1 --2- 172.21.15.60:0/576861158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae700051d0 0x7fae7000b9d0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009f90 tx=0x7fae7c01cc80 comp rx=0 tx=0).stop 2024-04-25T10:12:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae8a93b640 1 -- 172.21.15.60:0/576861158 shutdown_connections 2024-04-25T10:12:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae8a93b640 1 --2- 172.21.15.60:0/576861158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae700051d0 0x7fae7000b9d0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae8a93b640 1 -- 172.21.15.60:0/576861158 >> 172.21.15.60:0/576861158 conn(0x7fae840b12d0 msgr2=0x7fae840b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae8a93b640 1 -- 172.21.15.60:0/576861158 shutdown_connections 2024-04-25T10:12:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae8a93b640 1 -- 172.21.15.60:0/576861158 wait complete. 2024-04-25T10:12:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae8a93b640 1 Processor -- start 2024-04-25T10:12:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae8a93b640 1 -- start start 2024-04-25T10:12:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae8a93b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae700051d0 0x7fae84145290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae8a93b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae840b16b0 0x7fae840b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae8a93b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae841571e0 0x7fae84157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae8a93b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae840b4400 con 0x7fae840b16b0 2024-04-25T10:12:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae8a93b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae840b4280 con 0x7fae700051d0 2024-04-25T10:12:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae8a93b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae840b3ba0 con 0x7fae841571e0 2024-04-25T10:12:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae7bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae840b16b0 0x7fae840b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae7bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae840b16b0 0x7fae840b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53606/0 (socket says 172.21.15.60:53606) 2024-04-25T10:12:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae7bfff640 1 -- 172.21.15.60:0/3476256426 learned_addr learned my addr 172.21.15.60:0/3476256426 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae83fff640 1 --2- 172.21.15.60:0/3476256426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae700051d0 0x7fae84145290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.824+0000 7fae88ed8640 1 --2- 172.21.15.60:0/3476256426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae841571e0 0x7fae84157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae7bfff640 1 -- 172.21.15.60:0/3476256426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae841571e0 msgr2=0x7fae84157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae7bfff640 1 --2- 172.21.15.60:0/3476256426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae841571e0 0x7fae84157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae7bfff640 1 -- 172.21.15.60:0/3476256426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae700051d0 msgr2=0x7fae84145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae7bfff640 1 --2- 172.21.15.60:0/3476256426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae700051d0 0x7fae84145290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae7bfff640 1 -- 172.21.15.60:0/3476256426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae84157bd0 con 0x7fae840b16b0 2024-04-25T10:12:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae83fff640 1 --2- 172.21.15.60:0/3476256426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae700051d0 0x7fae84145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae88ed8640 1 --2- 172.21.15.60:0/3476256426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae841571e0 0x7fae84157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae7bfff640 1 --2- 172.21.15.60:0/3476256426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae840b16b0 0x7fae840b1b30 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00d7e0 tx=0x7fae6c00dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae817fa640 1 -- 172.21.15.60:0/3476256426 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c019070 con 0x7fae840b16b0 2024-04-25T10:12:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae817fa640 1 -- 172.21.15.60:0/3476256426 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6c0074e0 con 0x7fae840b16b0 2024-04-25T10:12:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae817fa640 1 -- 172.21.15.60:0/3476256426 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c0049f0 con 0x7fae840b16b0 2024-04-25T10:12:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae8a93b640 1 -- 172.21.15.60:0/3476256426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae84148950 con 0x7fae840b16b0 2024-04-25T10:12:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae8a93b640 1 -- 172.21.15.60:0/3476256426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae84147e70 con 0x7fae840b16b0 2024-04-25T10:12:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae8a93b640 1 -- 172.21.15.60:0/3476256426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae84074770 con 0x7fae840b16b0 2024-04-25T10:12:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae817fa640 1 -- 172.21.15.60:0/3476256426 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae6c01c070 con 0x7fae840b16b0 2024-04-25T10:12:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae817fa640 1 --2- 172.21.15.60:0/3476256426 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae64066f90 0x7fae64069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae817fa640 1 -- 172.21.15.60:0/3476256426 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fae6c0849f0 con 0x7fae840b16b0 2024-04-25T10:12:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae83fff640 1 --2- 172.21.15.60:0/3476256426 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae64066f90 0x7fae64069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.828+0000 7fae83fff640 1 --2- 172.21.15.60:0/3476256426 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae64066f90 0x7fae64069450 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fae74000f70 tx=0x7fae74009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:02.832+0000 7fae817fa640 1 -- 172.21.15.60:0/3476256426 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae6c055a20 con 0x7fae840b16b0 2024-04-25T10:12:03.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.028+0000 7fae8a93b640 1 -- 172.21.15.60:0/3476256426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae840cc4a0 con 0x7fae840b16b0 2024-04-25T10:12:03.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.028+0000 7fae817fa640 1 -- 172.21.15.60:0/3476256426 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fae6c055320 con 0x7fae840b16b0 2024-04-25T10:12:03.038 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:12:03.038 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:12:01.161388+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7908":{"gid":7908,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/2817879567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2817879567},{"type":"v1","addr":"172.21.15.60:6835","nonce":2817879567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:12:03.038 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 61 2024-04-25T10:12:03.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.032+0000 7fae8a93b640 1 -- 172.21.15.60:0/3476256426 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae64066f90 msgr2=0x7fae64069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.032+0000 7fae8a93b640 1 --2- 172.21.15.60:0/3476256426 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae64066f90 0x7fae64069450 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fae74000f70 tx=0x7fae74009040 comp rx=0 tx=0).stop 2024-04-25T10:12:03.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.032+0000 7fae8a93b640 1 -- 172.21.15.60:0/3476256426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae840b16b0 msgr2=0x7fae840b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.032+0000 7fae8a93b640 1 --2- 172.21.15.60:0/3476256426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae840b16b0 0x7fae840b1b30 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00d7e0 tx=0x7fae6c00dca0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.032+0000 7fae8a93b640 1 -- 172.21.15.60:0/3476256426 shutdown_connections 2024-04-25T10:12:03.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.032+0000 7fae8a93b640 1 --2- 172.21.15.60:0/3476256426 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae64066f90 0x7fae64069450 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.032+0000 7fae8a93b640 1 --2- 172.21.15.60:0/3476256426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae841571e0 0x7fae84157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.032+0000 7fae8a93b640 1 --2- 172.21.15.60:0/3476256426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae840b16b0 0x7fae840b1b30 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.032+0000 7fae8a93b640 1 --2- 172.21.15.60:0/3476256426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae700051d0 0x7fae84145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.032+0000 7fae8a93b640 1 -- 172.21.15.60:0/3476256426 >> 172.21.15.60:0/3476256426 conn(0x7fae840b12d0 msgr2=0x7fae8406e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:03.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.032+0000 7fae8a93b640 1 -- 172.21.15.60:0/3476256426 shutdown_connections 2024-04-25T10:12:03.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.032+0000 7fae8a93b640 1 -- 172.21.15.60:0/3476256426 wait complete. 2024-04-25T10:12:03.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:44.357874+0000', 'modified': '2024-04-25T10:12:01.161388+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7908}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7908': {'gid': 7908, 'name': 'c', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.60:6835/2817879567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 2817879567}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 2817879567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:12:03.461 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:12:03.461 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:12:03.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.588+0000 7f8efd3bf640 1 Processor -- start 2024-04-25T10:12:03.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.588+0000 7f8efd3bf640 1 -- start start 2024-04-25T10:12:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.588+0000 7f8efd3bf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8ef806c0e0 0x7f8ef80d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.588+0000 7f8efd3bf640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef815f5a0 con 0x7f8ef8099b10 2024-04-25T10:12:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.588+0000 7f8efd3bf640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef80d14d0 con 0x7f8ef80d0930 2024-04-25T10:12:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.588+0000 7f8efd3bf640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef806c570 con 0x7f8ef806c0e0 2024-04-25T10:12:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.588+0000 7f8ef67fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8ef806c0e0 0x7f8ef80d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.588+0000 7f8ef6ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8ef8099b10 0x7f8ef8099ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41572/0 (socket says 172.21.15.60:41572) 2024-04-25T10:12:03.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.588+0000 7f8ef6ffd640 1 -- 172.21.15.60:0/3848911102 learned_addr learned my addr 172.21.15.60:0/3848911102 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:12:03.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.592+0000 7f8ef67fc640 1 -- 172.21.15.60:0/3848911102 >> v1:172.21.15.100:6789/0 conn(0x7f8ef80d0930 legacy=0x7f8ef80d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.592+0000 7f8ef67fc640 1 -- 172.21.15.60:0/3848911102 >> v1:172.21.15.60:6789/0 conn(0x7f8ef8099b10 legacy=0x7f8ef8099ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.592+0000 7f8ef67fc640 1 -- 172.21.15.60:0/3848911102 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef815ea60 con 0x7f8ef806c0e0 2024-04-25T10:12:03.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.592+0000 7f8ef5ffb640 1 -- 172.21.15.60:0/3848911102 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530750924 0 0) 0x7f8ef00014a0 con 0x7f8ef8099b10 2024-04-25T10:12:03.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.592+0000 7f8ef67fc640 1 --2- 172.21.15.60:0/3848911102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ef806c0e0 0x7f8ef80d03f0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00097c0 tx=0x7f8ee001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deafdd2c5d06646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:03.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.592+0000 7f8ef5ffb640 1 -- 172.21.15.60:0/3848911102 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee00b1070 con 0x7f8ef806c0e0 2024-04-25T10:12:03.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.592+0000 7f8ef5ffb640 1 -- 172.21.15.60:0/3848911102 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee00a8050 con 0x7f8ef806c0e0 2024-04-25T10:12:03.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.592+0000 7f8ef5ffb640 1 -- 172.21.15.60:0/3848911102 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee00ac860 con 0x7f8ef806c0e0 2024-04-25T10:12:03.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.592+0000 7f8efd3bf640 1 -- 172.21.15.60:0/3848911102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ef806c0e0 msgr2=0x7f8ef80d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.592+0000 7f8efd3bf640 1 --2- 172.21.15.60:0/3848911102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ef806c0e0 0x7f8ef80d03f0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00097c0 tx=0x7f8ee001c3e0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.592+0000 7f8efd3bf640 1 -- 172.21.15.60:0/3848911102 shutdown_connections 2024-04-25T10:12:03.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.592+0000 7f8efd3bf640 1 --2- 172.21.15.60:0/3848911102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ef806c0e0 0x7f8ef80d03f0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.592+0000 7f8efd3bf640 1 -- 172.21.15.60:0/3848911102 >> 172.21.15.60:0/3848911102 conn(0x7f8ef80b3320 msgr2=0x7f8ef80b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:03.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.592+0000 7f8efd3bf640 1 -- 172.21.15.60:0/3848911102 shutdown_connections 2024-04-25T10:12:03.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.592+0000 7f8efd3bf640 1 -- 172.21.15.60:0/3848911102 wait complete. 2024-04-25T10:12:03.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8efd3bf640 1 Processor -- start 2024-04-25T10:12:03.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8efd3bf640 1 -- start start 2024-04-25T10:12:03.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8efd3bf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef806c0e0 0x7f8ef81f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8efd3bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ef8099b10 0x7f8ef815a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8efd3bf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ef80d0930 0x7f8ef8141580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8efd3bf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef8161390 con 0x7f8ef8099b10 2024-04-25T10:12:03.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8efd3bf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80cfba0 con 0x7f8ef806c0e0 2024-04-25T10:12:03.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8efd3bf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef8161510 con 0x7f8ef80d0930 2024-04-25T10:12:03.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8ef67fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ef8099b10 0x7f8ef815a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8ef67fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ef8099b10 0x7f8ef815a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53632/0 (socket says 172.21.15.60:53632) 2024-04-25T10:12:03.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8ef67fc640 1 -- 172.21.15.60:0/2276191863 learned_addr learned my addr 172.21.15.60:0/2276191863 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:03.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8ef6ffd640 1 --2- 172.21.15.60:0/2276191863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef806c0e0 0x7f8ef81f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8ef77fe640 1 --2- 172.21.15.60:0/2276191863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ef80d0930 0x7f8ef8141580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8ef67fc640 1 -- 172.21.15.60:0/2276191863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ef80d0930 msgr2=0x7f8ef8141580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8ef67fc640 1 --2- 172.21.15.60:0/2276191863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ef80d0930 0x7f8ef8141580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8ef67fc640 1 -- 172.21.15.60:0/2276191863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef806c0e0 msgr2=0x7f8ef81f8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8ef67fc640 1 --2- 172.21.15.60:0/2276191863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef806c0e0 0x7f8ef81f8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8ef67fc640 1 -- 172.21.15.60:0/2276191863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef81f76d0 con 0x7f8ef8099b10 2024-04-25T10:12:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8ef6ffd640 1 --2- 172.21.15.60:0/2276191863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef806c0e0 0x7f8ef81f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8ef67fc640 1 --2- 172.21.15.60:0/2276191863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ef8099b10 0x7f8ef815a2a0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f8ee001a010 tx=0x7f8ee001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8eef7fe640 1 -- 172.21.15.60:0/2276191863 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee00b1040 con 0x7f8ef8099b10 2024-04-25T10:12:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8eef7fe640 1 -- 172.21.15.60:0/2276191863 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee00ace20 con 0x7f8ef8099b10 2024-04-25T10:12:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8efd3bf640 1 -- 172.21.15.60:0/2276191863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef81422b0 con 0x7f8ef8099b10 2024-04-25T10:12:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8eef7fe640 1 -- 172.21.15.60:0/2276191863 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0004220 con 0x7f8ef8099b10 2024-04-25T10:12:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.596+0000 7f8efd3bf640 1 -- 172.21.15.60:0/2276191863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef815a7e0 con 0x7f8ef8099b10 2024-04-25T10:12:03.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8eef7fe640 1 -- 172.21.15.60:0/2276191863 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ee00bf060 con 0x7f8ef8099b10 2024-04-25T10:12:03.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8eef7fe640 1 --2- 172.21.15.60:0/2276191863 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ed8066f60 0x7f8ed8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8eef7fe640 1 -- 172.21.15.60:0/2276191863 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f8ee011eae0 con 0x7f8ef8099b10 2024-04-25T10:12:03.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8ef6ffd640 1 --2- 172.21.15.60:0/2276191863 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ed8066f60 0x7f8ed8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8f0052c000 1 Processor -- start 2024-04-25T10:12:03.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8f0052c000 1 -- start start 2024-04-25T10:12:03.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8ef6ffd640 1 --2- 172.21.15.60:0/2276191863 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ed8066f60 0x7f8ed8069420 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0007fd0 tx=0x7f8ef0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:03.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8f0052c000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d91d920 0x55891d91dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8f0052c000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d91e240 0x55891d8d39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8f0052c000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d8d3f30 0x55891d8d4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8f0052c000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891d7bb5f0 con 0x55891d8d3f30 2024-04-25T10:12:03.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8f0052c000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891d722da0 con 0x55891d91d920 2024-04-25T10:12:03.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8f0052c000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55891d6ab150 con 0x55891d91e240 2024-04-25T10:12:03.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8ef77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d8d3f30 0x55891d8d4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8ef77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d8d3f30 0x55891d8d4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53636/0 (socket says 172.21.15.60:53636) 2024-04-25T10:12:03.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8ef77fe640 1 -- 172.21.15.60:0/1411727387 learned_addr learned my addr 172.21.15.60:0/1411727387 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:03.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8ef6ffd640 1 --2- 172.21.15.60:0/1411727387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d91d920 0x55891d91dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8ef67fc640 1 --2- 172.21.15.60:0/1411727387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d91e240 0x55891d8d39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8ef77fe640 1 -- 172.21.15.60:0/1411727387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d91e240 msgr2=0x55891d8d39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8ef77fe640 1 --2- 172.21.15.60:0/1411727387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d91e240 0x55891d8d39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8ef77fe640 1 -- 172.21.15.60:0/1411727387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d91d920 msgr2=0x55891d91dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8ef77fe640 1 --2- 172.21.15.60:0/1411727387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d91d920 0x55891d91dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8ef77fe640 1 -- 172.21.15.60:0/1411727387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55891d6c4ca0 con 0x55891d8d3f30 2024-04-25T10:12:03.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8ef77fe640 1 --2- 172.21.15.60:0/1411727387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d8d3f30 0x55891d8d4310 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80015c0 tx=0x7f8ee8001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e77510d9d0eaae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:03.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8eecff9640 1 -- 172.21.15.60:0/1411727387 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80052d0 con 0x55891d8d3f30 2024-04-25T10:12:03.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8eecff9640 1 -- 172.21.15.60:0/1411727387 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee8005470 con 0x55891d8d3f30 2024-04-25T10:12:03.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8eecff9640 1 -- 172.21.15.60:0/1411727387 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800d950 con 0x55891d8d3f30 2024-04-25T10:12:03.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8f0052c000 1 -- 172.21.15.60:0/1411727387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d8d3f30 msgr2=0x55891d8d4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8f0052c000 1 --2- 172.21.15.60:0/1411727387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d8d3f30 0x55891d8d4310 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80015c0 tx=0x7f8ee8001a80 comp rx=0 tx=0).stop 2024-04-25T10:12:03.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8f0052c000 1 -- 172.21.15.60:0/1411727387 shutdown_connections 2024-04-25T10:12:03.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8f0052c000 1 --2- 172.21.15.60:0/1411727387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d8d3f30 0x55891d8d4310 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8f0052c000 1 --2- 172.21.15.60:0/1411727387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d91e240 0x55891d8d39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8f0052c000 1 --2- 172.21.15.60:0/1411727387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d91d920 0x55891d91dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8f0052c000 1 -- 172.21.15.60:0/1411727387 >> 172.21.15.60:0/1411727387 conn(0x55891d8da2a0 msgr2=0x55891d8d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:03.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.600+0000 7f8f0052c000 1 -- 172.21.15.60:0/1411727387 shutdown_connections 2024-04-25T10:12:03.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 1 -- 172.21.15.60:0/1411727387 wait complete. 2024-04-25T10:12:03.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 1 -- start start 2024-04-25T10:12:03.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 1 Processor -- start 2024-04-25T10:12:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d8d3f30 0x55891d924cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d91d920 0x55891d925230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91e240 0x55891d8d7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891d6dd990 con 0x55891d91e240 2024-04-25T10:12:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891d71ee40 con 0x55891d8d3f30 2024-04-25T10:12:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55891d6ae900 con 0x55891d91d920 2024-04-25T10:12:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 10 client.0 _ll_register_callbacks cb 0x55891d7d61c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ef77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91e240 0x55891d8d7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ef77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91e240 0x55891d8d7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53648/0 (socket says 172.21.15.60:53648) 2024-04-25T10:12:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ef77fe640 1 -- 172.21.15.60:0/1180330338 learned_addr learned my addr 172.21.15.60:0/1180330338 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ef67fc640 1 --2- 172.21.15.60:0/1180330338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d91d920 0x55891d925230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ef6ffd640 1 --2- 172.21.15.60:0/1180330338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d8d3f30 0x55891d924cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ef77fe640 1 -- 172.21.15.60:0/1180330338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d91d920 msgr2=0x55891d925230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ef77fe640 1 --2- 172.21.15.60:0/1180330338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d91d920 0x55891d925230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ef77fe640 1 -- 172.21.15.60:0/1180330338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d8d3f30 msgr2=0x55891d924cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ef77fe640 1 --2- 172.21.15.60:0/1180330338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d8d3f30 0x55891d924cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ef77fe640 1 -- 172.21.15.60:0/1180330338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55891d6db630 con 0x55891d91e240 2024-04-25T10:12:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ef77fe640 1 --2- 172.21.15.60:0/1180330338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91e240 0x55891d8d7aa0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8000c10 tx=0x7f8ee8000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726fa17433e4c528 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ed5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:12:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ed5ffb640 1 -- 172.21.15.60:0/1180330338 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8007450 con 0x55891d91e240 2024-04-25T10:12:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ed5ffb640 1 -- 172.21.15.60:0/1180330338 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f8ee80075f0 con 0x55891d91e240 2024-04-25T10:12:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 1 -- 172.21.15.60:0/1180330338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ee8005470 con 0x55891d91e240 2024-04-25T10:12:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ed5ffb640 1 -- 172.21.15.60:0/1180330338 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ee802e050 con 0x55891d91e240 2024-04-25T10:12:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 10 client.7974 fetch_fsmap learned FSMap version 61 2024-04-25T10:12:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 1 -- 172.21.15.60:0/1180330338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55891d6ab3a0 con 0x55891d91e240 2024-04-25T10:12:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ed5ffb640 1 -- 172.21.15.60:0/1180330338 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee8005470 con 0x55891d91e240 2024-04-25T10:12:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ed5ffb640 1 -- 172.21.15.60:0/1180330338 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8ee8031020 con 0x55891d91e240 2024-04-25T10:12:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 10 client.7974 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T10:12:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 10 client.7974 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:12:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 20 client.7974 populate_metadata read hostname 'smithi060' 2024-04-25T10:12:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 1 --2- 172.21.15.60:0/1180330338 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55891d8d9160 0x55891d92ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 4 client.7974 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8f0052c000 1 -- 172.21.15.60:0/1180330338 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55891d6bea50 con 0x55891d8d9160 2024-04-25T10:12:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ef6ffd640 1 --2- 172.21.15.60:0/1180330338 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55891d8d9160 0x55891d92ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ef6ffd640 1 --2- 172.21.15.60:0/1180330338 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55891d8d9160 0x55891d92ed50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0a650738b421475 server_cookie=453a061d33e7e8fd in_seq=0 out_seq=0 2024-04-25T10:12:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.604+0000 7f8ed5ffb640 10 client.7974 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.608+0000 7f8ed5ffb640 1 -- 172.21.15.60:0/1180330338 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55891d6bea50 con 0x55891d8d9160 2024-04-25T10:12:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.608+0000 7f8ed5ffb640 10 client.7974 handle_command_reply: tid=0 2024-04-25T10:12:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.608+0000 7f8f0052c000 1 client.7974 shutdown 2024-04-25T10:12:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.608+0000 7f8f0052c000 1 -- 172.21.15.60:0/1180330338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91e240 msgr2=0x55891d8d7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.608+0000 7f8f0052c000 1 --2- 172.21.15.60:0/1180330338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91e240 0x55891d8d7aa0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8000c10 tx=0x7f8ee8000ee0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.608+0000 7f8f0052c000 1 -- 172.21.15.60:0/1180330338 shutdown_connections 2024-04-25T10:12:03.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.608+0000 7f8f0052c000 1 --2- 172.21.15.60:0/1180330338 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55891d8d9160 0x55891d92ed50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.608+0000 7f8f0052c000 1 --2- 172.21.15.60:0/1180330338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91e240 0x55891d8d7aa0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.608+0000 7f8f0052c000 1 --2- 172.21.15.60:0/1180330338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d91d920 0x55891d925230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.608+0000 7f8f0052c000 1 --2- 172.21.15.60:0/1180330338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d8d3f30 0x55891d924cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.608+0000 7f8ed5ffb640 0 client.7974 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:03.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.608+0000 7f8f0052c000 1 -- 172.21.15.60:0/1180330338 >> 172.21.15.60:0/1180330338 conn(0x55891d8da2a0 msgr2=0x55891d8d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:03.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.608+0000 7f8f0052c000 1 -- 172.21.15.60:0/1180330338 shutdown_connections 2024-04-25T10:12:03.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.608+0000 7f8f0052c000 1 -- 172.21.15.60:0/1180330338 wait complete. 2024-04-25T10:12:03.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.608+0000 7f8f0052c000 20 client.7974 trim_cache size 0 max 16384 2024-04-25T10:12:03.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 Processor -- start 2024-04-25T10:12:03.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 -- start start 2024-04-25T10:12:03.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d8d3f30 0x55891d8d72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91d920 0x55891d8d7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d91e240 0x55891d926920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80c6e20 con 0x55891d91d920 2024-04-25T10:12:03.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef81fb8b0 con 0x55891d91e240 2024-04-25T10:12:03.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80c6820 con 0x55891d8d3f30 2024-04-25T10:12:03.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8ef67fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91d920 0x55891d8d7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8ef67fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91d920 0x55891d8d7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53662/0 (socket says 172.21.15.60:53662) 2024-04-25T10:12:03.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8ef67fc640 1 -- 172.21.15.60:0/1685794540 learned_addr learned my addr 172.21.15.60:0/1685794540 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:03.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8ef67fc640 1 -- 172.21.15.60:0/1685794540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d8d3f30 msgr2=0x55891d8d72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8ef6ffd640 1 --2- 172.21.15.60:0/1685794540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d8d3f30 0x55891d8d72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8ef67fc640 1 --2- 172.21.15.60:0/1685794540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d8d3f30 0x55891d8d72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8ef67fc640 1 -- 172.21.15.60:0/1685794540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d91e240 msgr2=0x55891d926920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8ef77fe640 1 --2- 172.21.15.60:0/1685794540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d91e240 0x55891d926920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8ef67fc640 1 --2- 172.21.15.60:0/1685794540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d91e240 0x55891d926920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8ef67fc640 1 -- 172.21.15.60:0/1685794540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55891d91e070 con 0x55891d91d920 2024-04-25T10:12:03.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8ef67fc640 1 --2- 172.21.15.60:0/1685794540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91d920 0x55891d8d7800 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00eb7e0 tx=0x7f8ee0009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41935e555de00d6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:03.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8ed7fff640 1 -- 172.21.15.60:0/1685794540 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee00f7a90 con 0x55891d91d920 2024-04-25T10:12:03.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8ed7fff640 1 -- 172.21.15.60:0/1685794540 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee00a8020 con 0x55891d91d920 2024-04-25T10:12:03.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8ed7fff640 1 -- 172.21.15.60:0/1685794540 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee00043c0 con 0x55891d91d920 2024-04-25T10:12:03.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 -- 172.21.15.60:0/1685794540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91d920 msgr2=0x55891d8d7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 --2- 172.21.15.60:0/1685794540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91d920 0x55891d8d7800 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00eb7e0 tx=0x7f8ee0009190 comp rx=0 tx=0).stop 2024-04-25T10:12:03.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 -- 172.21.15.60:0/1685794540 shutdown_connections 2024-04-25T10:12:03.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 --2- 172.21.15.60:0/1685794540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d91e240 0x55891d926920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 --2- 172.21.15.60:0/1685794540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91d920 0x55891d8d7800 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 --2- 172.21.15.60:0/1685794540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d8d3f30 0x55891d8d72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 -- 172.21.15.60:0/1685794540 >> 172.21.15.60:0/1685794540 conn(0x55891d8da2a0 msgr2=0x55891d8d6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:03.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 -- 172.21.15.60:0/1685794540 shutdown_connections 2024-04-25T10:12:03.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 -- 172.21.15.60:0/1685794540 wait complete. 2024-04-25T10:12:03.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 -- start start 2024-04-25T10:12:03.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.624+0000 7f8f0052c000 1 Processor -- start 2024-04-25T10:12:03.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d8d3f30 0x55891d929ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d91d920 0x55891d92a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91e240 0x55891d962090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891d7273d0 con 0x55891d91e240 2024-04-25T10:12:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891d7ded50 con 0x55891d8d3f30 2024-04-25T10:12:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55891d6bd980 con 0x55891d91d920 2024-04-25T10:12:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 10 client.0 _ll_register_callbacks cb 0x7f8ee8007d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ef77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91e240 0x55891d962090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ef77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91e240 0x55891d962090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53668/0 (socket says 172.21.15.60:53668) 2024-04-25T10:12:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ef77fe640 1 -- 172.21.15.60:0/955859871 learned_addr learned my addr 172.21.15.60:0/955859871 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ef67fc640 1 --2- 172.21.15.60:0/955859871 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d91d920 0x55891d92a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ef6ffd640 1 --2- 172.21.15.60:0/955859871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d8d3f30 0x55891d929ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ef77fe640 1 -- 172.21.15.60:0/955859871 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d91d920 msgr2=0x55891d92a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ef77fe640 1 --2- 172.21.15.60:0/955859871 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d91d920 0x55891d92a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ef77fe640 1 -- 172.21.15.60:0/955859871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d8d3f30 msgr2=0x55891d929ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ef77fe640 1 --2- 172.21.15.60:0/955859871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d8d3f30 0x55891d929ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ef77fe640 1 -- 172.21.15.60:0/955859871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55891d7a4600 con 0x55891d91e240 2024-04-25T10:12:03.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ef67fc640 1 --2- 172.21.15.60:0/955859871 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d91d920 0x55891d92a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:03.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ef77fe640 1 --2- 172.21.15.60:0/955859871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91e240 0x55891d962090 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8001be0 tx=0x7f8ee8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a79d59dff88fd13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:03.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ed5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:12:03.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ed5ffb640 1 -- 172.21.15.60:0/955859871 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800dc70 con 0x55891d91e240 2024-04-25T10:12:03.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ed5ffb640 1 -- 172.21.15.60:0/955859871 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f8ee800de10 con 0x55891d91e240 2024-04-25T10:12:03.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 -- 172.21.15.60:0/955859871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ee00a8020 con 0x55891d91e240 2024-04-25T10:12:03.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ed5ffb640 1 -- 172.21.15.60:0/955859871 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ee8043050 con 0x55891d91e240 2024-04-25T10:12:03.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 10 client.7980 fetch_fsmap learned FSMap version 61 2024-04-25T10:12:03.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 -- 172.21.15.60:0/955859871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55891d6b79b0 con 0x55891d91e240 2024-04-25T10:12:03.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ed5ffb640 1 -- 172.21.15.60:0/955859871 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee00a8020 con 0x55891d91e240 2024-04-25T10:12:03.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ed5ffb640 1 -- 172.21.15.60:0/955859871 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8ee8031040 con 0x55891d91e240 2024-04-25T10:12:03.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 10 client.7980 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T10:12:03.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 10 client.7980 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:12:03.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 20 client.7980 populate_metadata read hostname 'smithi060' 2024-04-25T10:12:03.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 --2- 172.21.15.60:0/955859871 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55891d9625d0 0x55891d9629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 4 client.7980 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:12:03.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 -- 172.21.15.60:0/955859871 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55891d7302e0 con 0x55891d9625d0 2024-04-25T10:12:03.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ef6ffd640 1 --2- 172.21.15.60:0/955859871 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55891d9625d0 0x55891d9629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ef6ffd640 1 --2- 172.21.15.60:0/955859871 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55891d9625d0 0x55891d9629b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb499c0ace23897b server_cookie=dec3df479fe3d7ce in_seq=0 out_seq=0 2024-04-25T10:12:03.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ed5ffb640 10 client.7980 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:03.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ed5ffb640 1 -- 172.21.15.60:0/955859871 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x55891d7302e0 con 0x55891d9625d0 2024-04-25T10:12:03.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ed5ffb640 10 client.7980 handle_command_reply: tid=0 2024-04-25T10:12:03.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 client.7980 shutdown 2024-04-25T10:12:03.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 -- 172.21.15.60:0/955859871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91e240 msgr2=0x55891d962090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 --2- 172.21.15.60:0/955859871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91e240 0x55891d962090 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8001be0 tx=0x7f8ee8009040 comp rx=0 tx=0).stop 2024-04-25T10:12:03.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 -- 172.21.15.60:0/955859871 shutdown_connections 2024-04-25T10:12:03.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 --2- 172.21.15.60:0/955859871 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55891d9625d0 0x55891d9629b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 --2- 172.21.15.60:0/955859871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55891d91e240 0x55891d962090 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 --2- 172.21.15.60:0/955859871 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55891d91d920 0x55891d92a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8ed5ffb640 0 client.7980 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:03.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 --2- 172.21.15.60:0/955859871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55891d8d3f30 0x55891d929ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 -- 172.21.15.60:0/955859871 >> 172.21.15.60:0/955859871 conn(0x55891d8da2a0 msgr2=0x55891d8d72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:03.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 -- 172.21.15.60:0/955859871 shutdown_connections 2024-04-25T10:12:03.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 1 -- 172.21.15.60:0/955859871 wait complete. 2024-04-25T10:12:03.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8f0052c000 20 client.7980 trim_cache size 0 max 16384 2024-04-25T10:12:03.643 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:12:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "0ea93863-a079-45c4-abed-6e83540ea9fe", 2024-04-25T10:12:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T10:12:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7908, 2024-04-25T10:12:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T10:12:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T10:12:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T10:12:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 4.4809042750000003, 2024-04-25T10:12:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 61, 2024-04-25T10:12:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 85, 2024-04-25T10:12:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 83, 2024-04-25T10:12:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 5.44960106 2024-04-25T10:12:03.644 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:12:03.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8efd3bf640 1 -- 172.21.15.60:0/2276191863 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ed8066f60 msgr2=0x7f8ed8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8efd3bf640 1 --2- 172.21.15.60:0/2276191863 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ed8066f60 0x7f8ed8069420 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0007fd0 tx=0x7f8ef0009040 comp rx=0 tx=0).stop 2024-04-25T10:12:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8efd3bf640 1 -- 172.21.15.60:0/2276191863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ef8099b10 msgr2=0x7f8ef815a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8efd3bf640 1 --2- 172.21.15.60:0/2276191863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ef8099b10 0x7f8ef815a2a0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f8ee001a010 tx=0x7f8ee001ac20 comp rx=0 tx=0).stop 2024-04-25T10:12:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8efd3bf640 1 -- 172.21.15.60:0/2276191863 shutdown_connections 2024-04-25T10:12:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8efd3bf640 1 --2- 172.21.15.60:0/2276191863 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8ed8066f60 0x7f8ed8069420 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8efd3bf640 1 --2- 172.21.15.60:0/2276191863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ef80d0930 0x7f8ef8141580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8efd3bf640 1 --2- 172.21.15.60:0/2276191863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ef8099b10 0x7f8ef815a2a0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8efd3bf640 1 --2- 172.21.15.60:0/2276191863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ef806c0e0 0x7f8ef81f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8efd3bf640 1 -- 172.21.15.60:0/2276191863 >> 172.21.15.60:0/2276191863 conn(0x7f8ef80b3320 msgr2=0x7f8ef809a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8efd3bf640 1 -- 172.21.15.60:0/2276191863 shutdown_connections 2024-04-25T10:12:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.628+0000 7f8efd3bf640 1 -- 172.21.15.60:0/2276191863 wait complete. 2024-04-25T10:12:03.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:12:03.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.760+0000 7fdb685b5640 1 Processor -- start 2024-04-25T10:12:03.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.760+0000 7fdb685b5640 1 -- start start 2024-04-25T10:12:03.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb685b5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb6015bc90 0x7fdb6015c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb685b5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb600b4660 0x7fdb6015c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb685b5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb60071ca0 con 0x7fdb600ce7a0 2024-04-25T10:12:03.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb685b5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb600cc370 con 0x7fdb6015bc90 2024-04-25T10:12:03.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb685b5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600cca40 con 0x7fdb600b4660 2024-04-25T10:12:03.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb65b50640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb6015bc90 0x7fdb6015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb65b50640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb6015bc90 0x7fdb6015c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53674/0 (socket says 172.21.15.60:53674) 2024-04-25T10:12:03.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb65b50640 1 -- 172.21.15.60:0/3281986721 learned_addr learned my addr 172.21.15.60:0/3281986721 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:03.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb66b52640 1 --2- 172.21.15.60:0/3281986721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb600b4660 0x7fdb6015c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb65b50640 1 -- 172.21.15.60:0/3281986721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb600b4660 msgr2=0x7fdb6015c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb65b50640 1 --2- 172.21.15.60:0/3281986721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb600b4660 0x7fdb6015c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb65b50640 1 -- 172.21.15.60:0/3281986721 >> v1:172.21.15.100:6789/0 conn(0x7fdb600ce7a0 legacy=0x7fdb600ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb65b50640 1 -- 172.21.15.60:0/3281986721 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb600ccd20 con 0x7fdb6015bc90 2024-04-25T10:12:03.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb65b50640 1 --2- 172.21.15.60:0/3281986721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb6015bc90 0x7fdb6015c220 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009870 tx=0x7fdb5001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e571f01ba11a90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:03.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb6534f640 1 -- 172.21.15.60:0/3281986721 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb500b2070 con 0x7fdb6015bc90 2024-04-25T10:12:03.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb6534f640 1 -- 172.21.15.60:0/3281986721 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb500a9070 con 0x7fdb6015bc90 2024-04-25T10:12:03.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.764+0000 7fdb6534f640 1 -- 172.21.15.60:0/3281986721 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb500ad880 con 0x7fdb6015bc90 2024-04-25T10:12:03.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb685b5640 1 -- 172.21.15.60:0/3281986721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb6015bc90 msgr2=0x7fdb6015c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb685b5640 1 --2- 172.21.15.60:0/3281986721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb6015bc90 0x7fdb6015c220 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009870 tx=0x7fdb5001c560 comp rx=0 tx=0).stop 2024-04-25T10:12:03.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb685b5640 1 -- 172.21.15.60:0/3281986721 shutdown_connections 2024-04-25T10:12:03.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb685b5640 1 --2- 172.21.15.60:0/3281986721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb6015bc90 0x7fdb6015c220 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb685b5640 1 --2- 172.21.15.60:0/3281986721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb600b4660 0x7fdb6015c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb685b5640 1 -- 172.21.15.60:0/3281986721 >> 172.21.15.60:0/3281986721 conn(0x7fdb600b12d0 msgr2=0x7fdb600b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:03.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb685b5640 1 -- 172.21.15.60:0/3281986721 shutdown_connections 2024-04-25T10:12:03.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb685b5640 1 -- 172.21.15.60:0/3281986721 wait complete. 2024-04-25T10:12:03.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb685b5640 1 Processor -- start 2024-04-25T10:12:03.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb685b5640 1 -- start start 2024-04-25T10:12:03.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb685b5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb600ce7a0 0x7fdb601575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb685b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb6015bc90 0x7fdb60159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb685b5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb60148f90 0x7fdb600c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb685b5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb601685c0 con 0x7fdb600ce7a0 2024-04-25T10:12:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb685b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600b3db0 con 0x7fdb6015bc90 2024-04-25T10:12:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb685b5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb60168740 con 0x7fdb60148f90 2024-04-25T10:12:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb66351640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb600ce7a0 0x7fdb601575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb66351640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb600ce7a0 0x7fdb601575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53686/0 (socket says 172.21.15.60:53686) 2024-04-25T10:12:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb66351640 1 -- 172.21.15.60:0/3733624385 learned_addr learned my addr 172.21.15.60:0/3733624385 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb65b50640 1 --2- 172.21.15.60:0/3733624385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb6015bc90 0x7fdb60159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb66b52640 1 --2- 172.21.15.60:0/3733624385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb60148f90 0x7fdb600c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb66351640 1 -- 172.21.15.60:0/3733624385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb60148f90 msgr2=0x7fdb600c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb66351640 1 --2- 172.21.15.60:0/3733624385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb60148f90 0x7fdb600c59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb66351640 1 -- 172.21.15.60:0/3733624385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb6015bc90 msgr2=0x7fdb60159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.768+0000 7fdb66351640 1 --2- 172.21.15.60:0/3733624385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb6015bc90 0x7fdb60159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.772+0000 7fdb66351640 1 -- 172.21.15.60:0/3733624385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb6015a2c0 con 0x7fdb600ce7a0 2024-04-25T10:12:03.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.772+0000 7fdb66b52640 1 --2- 172.21.15.60:0/3733624385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb60148f90 0x7fdb600c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:03.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.772+0000 7fdb66351640 1 --2- 172.21.15.60:0/3733624385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb600ce7a0 0x7fdb601575d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00c3a0 tx=0x7fdb5c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:03.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.772+0000 7fdb56ffd640 1 -- 172.21.15.60:0/3733624385 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c0052d0 con 0x7fdb600ce7a0 2024-04-25T10:12:03.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.772+0000 7fdb56ffd640 1 -- 172.21.15.60:0/3733624385 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5c005470 con 0x7fdb600ce7a0 2024-04-25T10:12:03.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.772+0000 7fdb56ffd640 1 -- 172.21.15.60:0/3733624385 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c007260 con 0x7fdb600ce7a0 2024-04-25T10:12:03.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.772+0000 7fdb685b5640 1 -- 172.21.15.60:0/3733624385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb60157bb0 con 0x7fdb600ce7a0 2024-04-25T10:12:03.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.772+0000 7fdb685b5640 1 -- 172.21.15.60:0/3733624385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb60158170 con 0x7fdb600ce7a0 2024-04-25T10:12:03.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.772+0000 7fdb685b5640 1 -- 172.21.15.60:0/3733624385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb60074770 con 0x7fdb600ce7a0 2024-04-25T10:12:03.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.772+0000 7fdb56ffd640 1 -- 172.21.15.60:0/3733624385 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdb5c007400 con 0x7fdb600ce7a0 2024-04-25T10:12:03.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.772+0000 7fdb56ffd640 1 --2- 172.21.15.60:0/3733624385 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb44066f60 0x7fdb44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.772+0000 7fdb56ffd640 1 -- 172.21.15.60:0/3733624385 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fdb5c0844e0 con 0x7fdb600ce7a0 2024-04-25T10:12:03.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.772+0000 7fdb65b50640 1 --2- 172.21.15.60:0/3733624385 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb44066f60 0x7fdb44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.772+0000 7fdb65b50640 1 --2- 172.21.15.60:0/3733624385 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb44066f60 0x7fdb44069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdb500af000 tx=0x7fdb500a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:03.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.776+0000 7fdb56ffd640 1 -- 172.21.15.60:0/3733624385 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdb5c055480 con 0x7fdb600ce7a0 2024-04-25T10:12:03.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.968+0000 7fdb685b5640 1 -- 172.21.15.60:0/3733624385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdb60071800 con 0x7fdb600ce7a0 2024-04-25T10:12:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.968+0000 7fdb56ffd640 1 -- 172.21.15.60:0/3733624385 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdb5c054d80 con 0x7fdb600ce7a0 2024-04-25T10:12:03.976 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:12:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.968+0000 7fdb685b5640 1 -- 172.21.15.60:0/3733624385 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb44066f60 msgr2=0x7fdb44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.968+0000 7fdb685b5640 1 --2- 172.21.15.60:0/3733624385 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb44066f60 0x7fdb44069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdb500af000 tx=0x7fdb500a7000 comp rx=0 tx=0).stop 2024-04-25T10:12:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.968+0000 7fdb685b5640 1 -- 172.21.15.60:0/3733624385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb600ce7a0 msgr2=0x7fdb601575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.968+0000 7fdb685b5640 1 --2- 172.21.15.60:0/3733624385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb600ce7a0 0x7fdb601575d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00c3a0 tx=0x7fdb5c00c860 comp rx=0 tx=0).stop 2024-04-25T10:12:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.968+0000 7fdb685b5640 1 -- 172.21.15.60:0/3733624385 shutdown_connections 2024-04-25T10:12:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.968+0000 7fdb685b5640 1 --2- 172.21.15.60:0/3733624385 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb44066f60 0x7fdb44069420 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.968+0000 7fdb685b5640 1 --2- 172.21.15.60:0/3733624385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb60148f90 0x7fdb600c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.968+0000 7fdb685b5640 1 --2- 172.21.15.60:0/3733624385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb6015bc90 0x7fdb60159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.968+0000 7fdb685b5640 1 --2- 172.21.15.60:0/3733624385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb600ce7a0 0x7fdb601575d0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.968+0000 7fdb685b5640 1 -- 172.21.15.60:0/3733624385 >> 172.21.15.60:0/3733624385 conn(0x7fdb600b12d0 msgr2=0x7fdb6015b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:03.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.968+0000 7fdb685b5640 1 -- 172.21.15.60:0/3733624385 shutdown_connections 2024-04-25T10:12:03.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:03.968+0000 7fdb685b5640 1 -- 172.21.15.60:0/3733624385 wait complete. 2024-04-25T10:12:03.993 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:12:03.993 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:12:03.993 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:12:03.993 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:12:03.993 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:03.993 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:12:03.993 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:12:03.993 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:12:03.994 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:03.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:03.994 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:12:04.000 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:12:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:12:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:12:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:12:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:04.001 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:12:04.001 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:12:04.001 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:12:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:12:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:12:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42133sec preferred_lft 42133sec 2024-04-25T10:12:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:12:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:04.001 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:12:04.002 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:12:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:12:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:04.002 INFO:teuthology.orchestra.run.smithi060.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:12:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:12:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:12:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:04.002 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:12:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:12:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:04.003 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:12:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:12:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:04.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:04.003 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:12:04.056 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:12:04.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:04.058 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:12:04.104 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:12:04.104 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:12:04.104 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:12:04.104 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:12:04.104 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:12:04.104 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:12:04.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:04.105 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:12:04.105 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:12:04.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:12:04.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:12:04.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:12:04.105 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:12:04.158 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-04-25T10:12:04.148+0000 7ff06ad61640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:12:04.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:04.177 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:12:04.224 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:12:04.224 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:12:04.224 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:04.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:04.224 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T10:12:04.280 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:12:04.280 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:04.280 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:04.327 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:04.328 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:04.328 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:04.375 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:12:04.433 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:12:04.480 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:12:04.480 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:12:04.481 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:12:04.481 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:12:04.481 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:12:04.481 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:12:04.481 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:12:04.481 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:12:04.481 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:12:04.481 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:12:04.481 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:12:04.481 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:12:04.481 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:12:04.481 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:12:04.481 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:12:04.481 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:12:04.482 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:12:04.482 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:12:04.482 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:12:04.482 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:12:04.482 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:12:04.482 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:12:04.482 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:12:04.482 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:12:04.482 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:12:04.482 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:12:04.482 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:12:04.482 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:12:04.482 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:12:04.483 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:04.483 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:04.483 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:04.483 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:04.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:04.484 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:12:04.531 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:12:04.532 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:12:04.580 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:12:04.596 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:12:04.644 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:12:04.644 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:12:04.644 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:12:04.644 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:12:04.644 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:12:04.644 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:12:04.644 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:12:04.644 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:12:04.644 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:12:04.644 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:12:04.644 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:12:04.644 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:12:04.645 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:12:04.645 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:12:04.645 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:12:04.645 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:12:04.645 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:12:04.645 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:12:04.645 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:12:04.645 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:12:04.645 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:12:04.645 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:12:04.645 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:12:04.645 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:12:04.645 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:12:04.645 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:12:04.646 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:12:04.646 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:12:04.646 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:12:04.646 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:04.646 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:04.646 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:04.646 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:04.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:04.647 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:12:04.665 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2024-04-25T10:12:04.656+0000 7f6e9c59e580 -1 init, newargv = 0x56387a242d70 newargc=15 2024-04-25T10:12:04.665 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[46062]: starting ceph client 2024-04-25T10:12:04.671 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[46062]: starting fuse 2024-04-25T10:12:05.771 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:12:05.788 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:12:05.837 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:12:05.837 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:12:05.837 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:12:05.837 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:12:05.837 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:12:05.837 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:12:05.837 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:12:05.837 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:12:05.838 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:12:05.838 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:12:05.838 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:12:05.838 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:12:05.838 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:12:05.838 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:12:05.838 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:12:05.838 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:12:05.838 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:12:05.838 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:12:05.838 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:12:05.838 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:12:05.838 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:12:05.839 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:12:05.839 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:12:05.839 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:12:05.839 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:12:05.839 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:12:05.839 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:12:05.839 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:12:05.839 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:12:05.839 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:05.839 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:05.839 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:05.839 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:05.839 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:05.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:05.840 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:12:05.888 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:12:05.888 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:12:05.888 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:12:05.889 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:12:05.889 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:12:05.889 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:12:05.889 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:12:05.889 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:12:05.889 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:12:05.889 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:12:05.889 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:12:05.889 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:12:05.889 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:12:05.889 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:12:05.889 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:12:05.889 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:12:05.889 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:12:05.889 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:12:05.889 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:12:05.890 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:12:05.890 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:12:05.890 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:12:05.890 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:12:05.890 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:12:05.890 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:12:05.890 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:12:05.890 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:12:05.890 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2024-04-25T10:12:05.890 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:12:06.015 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.46062.asok 2024-04-25T10:12:06.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:12:06.027 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46062.asok 2024-04-25T10:12:06.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:06.028 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46062.asok status 2024-04-25T10:12:06.189 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:12:06.189 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:12:06.189 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:12:06.189 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:12:06.189 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2024-04-25T10:12:06.189 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:12:06.189 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:12:06.190 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "46062", 2024-04-25T10:12:06.190 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:12:06.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:12:06.190 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:12:06.190 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:12:06.190 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7995, 2024-04-25T10:12:06.190 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:12:06.190 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:12:06.190 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:12:06.190 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 7995 2024-04-25T10:12:06.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:12:06.190 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:12:06.190 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:12:06.190 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:12:06.191 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1029829677 2024-04-25T10:12:06.191 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:12:06.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:12:06.191 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:12:06.191 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:12:06.191 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:12:06.191 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1029829677 2024-04-25T10:12:06.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:12:06.191 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.7995 192.168.0.1:0/1029829677", 2024-04-25T10:12:06.191 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/1029829677", 2024-04-25T10:12:06.191 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:12:06.191 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 61, 2024-04-25T10:12:06.191 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 85, 2024-04-25T10:12:06.191 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:12:06.192 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:12:06.192 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:12:06.192 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:12:06.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:06.204 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:12:06.204 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:06.210 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:12:06.210 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:06.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:06.211 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:06.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:06.267 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-04-25T10:12:06.321 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-04-25T10:12:06.312+0000 7ff066d59640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-25T10:12:06.323 INFO:teuthology.orchestra.run.smithi060.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-04-25T10:12:06.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:12:06.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:06.324 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-04-25T10:12:06.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.496+0000 7fac46bef640 1 Processor -- start 2024-04-25T10:12:06.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.496+0000 7fac46bef640 1 -- start start 2024-04-25T10:12:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac46bef640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fac400cfb90 0x7fac401655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac46bef640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac400b2670 con 0x7fac4015a680 2024-04-25T10:12:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac46bef640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac400b2950 con 0x7fac400cbc10 2024-04-25T10:12:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac46bef640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac400b2c30 con 0x7fac400cfb90 2024-04-25T10:12:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac4498b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fac4015a680 0x7fac400b3770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41582/0 (socket says 172.21.15.60:41582) 2024-04-25T10:12:06.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac4498b640 1 -- 172.21.15.60:0/1486594726 learned_addr learned my addr 172.21.15.60:0/1486594726 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:12:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac4518c640 1 --2- 172.21.15.60:0/1486594726 >> v2:172.21.15.100:3300/0 conn(0x7fac400cfb90 0x7fac401655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3f7fe640 1 -- 172.21.15.60:0/1486594726 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150218820 0 0) 0x7fac400b2670 con 0x7fac4015a680 2024-04-25T10:12:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3f7fe640 1 -- 172.21.15.60:0/1486594726 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac28003530 con 0x7fac4015a680 2024-04-25T10:12:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3f7fe640 1 -- 172.21.15.60:0/1486594726 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1584069342 0 0) 0x7fac28003530 con 0x7fac4015a680 2024-04-25T10:12:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3f7fe640 1 -- 172.21.15.60:0/1486594726 >> v2:172.21.15.100:3300/0 conn(0x7fac400cfb90 msgr2=0x7fac401655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3f7fe640 1 --2- 172.21.15.60:0/1486594726 >> v2:172.21.15.100:3300/0 conn(0x7fac400cfb90 0x7fac401655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3f7fe640 1 -- 172.21.15.60:0/1486594726 >> v1:172.21.15.100:6789/0 conn(0x7fac400cbc10 legacy=0x7fac400cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3f7fe640 1 -- 172.21.15.60:0/1486594726 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac400b2f10 con 0x7fac4015a680 2024-04-25T10:12:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3f7fe640 1 -- 172.21.15.60:0/1486594726 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fac30001680 con 0x7fac4015a680 2024-04-25T10:12:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3f7fe640 1 -- 172.21.15.60:0/1486594726 >> v1:172.21.15.60:6789/0 conn(0x7fac4015a680 legacy=0x7fac400b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3f7fe640 1 --2- 172.21.15.60:0/1486594726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac280039c0 0x7fac28003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3f7fe640 1 --2- 172.21.15.60:0/1486594726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac28004810 0x7fac28004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3f7fe640 1 --2- 172.21.15.60:0/1486594726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac280051d0 0x7fac2800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3f7fe640 1 -- 172.21.15.60:0/1486594726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac400b2c30 con 0x7fac280051d0 2024-04-25T10:12:06.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3f7fe640 1 -- 172.21.15.60:0/1486594726 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2800c070 con 0x7fac280039c0 2024-04-25T10:12:06.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3f7fe640 1 -- 172.21.15.60:0/1486594726 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2800c270 con 0x7fac28004810 2024-04-25T10:12:06.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac4518c640 1 --2- 172.21.15.60:0/1486594726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac280051d0 0x7fac2800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac4498b640 1 --2- 172.21.15.60:0/1486594726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac280039c0 0x7fac28003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3ffff640 1 --2- 172.21.15.60:0/1486594726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac28004810 0x7fac28004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac4518c640 1 -- 172.21.15.60:0/1486594726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac28004810 msgr2=0x7fac28004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac4518c640 1 --2- 172.21.15.60:0/1486594726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac28004810 0x7fac28004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac4518c640 1 -- 172.21.15.60:0/1486594726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac280039c0 msgr2=0x7fac28003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac4518c640 1 --2- 172.21.15.60:0/1486594726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac280039c0 0x7fac28003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac4518c640 1 -- 172.21.15.60:0/1486594726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac28003530 con 0x7fac280051d0 2024-04-25T10:12:06.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac3ffff640 1 --2- 172.21.15.60:0/1486594726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac28004810 0x7fac28004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:06.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac4518c640 1 -- 172.21.15.60:0/1486594726 reap_dead start 2024-04-25T10:12:06.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.500+0000 7fac4518c640 1 --2- 172.21.15.60:0/1486594726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac280051d0 0x7fac2800b9d0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fac38009fc0 tx=0x7fac3801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaea824ae878c730 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac3f7fe640 1 -- 172.21.15.60:0/1486594726 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38009c10 con 0x7fac280051d0 2024-04-25T10:12:06.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac3f7fe640 1 -- 172.21.15.60:0/1486594726 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac380a5d10 con 0x7fac280051d0 2024-04-25T10:12:06.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac3f7fe640 1 -- 172.21.15.60:0/1486594726 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38003be0 con 0x7fac280051d0 2024-04-25T10:12:06.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac46bef640 1 -- 172.21.15.60:0/1486594726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac280051d0 msgr2=0x7fac2800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac46bef640 1 --2- 172.21.15.60:0/1486594726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac280051d0 0x7fac2800b9d0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fac38009fc0 tx=0x7fac3801ccc0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac46bef640 1 -- 172.21.15.60:0/1486594726 shutdown_connections 2024-04-25T10:12:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac46bef640 1 --2- 172.21.15.60:0/1486594726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac280051d0 0x7fac2800b9d0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac46bef640 1 -- 172.21.15.60:0/1486594726 >> 172.21.15.60:0/1486594726 conn(0x7fac40066fa0 msgr2=0x7fac4006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac46bef640 1 -- 172.21.15.60:0/1486594726 shutdown_connections 2024-04-25T10:12:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac46bef640 1 -- 172.21.15.60:0/1486594726 wait complete. 2024-04-25T10:12:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac46bef640 1 Processor -- start 2024-04-25T10:12:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac46bef640 1 -- start start 2024-04-25T10:12:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac46bef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac280051d0 0x7fac401f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac46bef640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac4015ada0 0x7fac40151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac46bef640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac400b1470 0x7fac400ada20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac46bef640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac400cd4f0 con 0x7fac280051d0 2024-04-25T10:12:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac46bef640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4015aa60 con 0x7fac400b1470 2024-04-25T10:12:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac46bef640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac400cd670 con 0x7fac4015ada0 2024-04-25T10:12:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac4498b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac280051d0 0x7fac401f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac4498b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac280051d0 0x7fac401f33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53704/0 (socket says 172.21.15.60:53704) 2024-04-25T10:12:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac4498b640 1 -- 172.21.15.60:0/3438139791 learned_addr learned my addr 172.21.15.60:0/3438139791 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac3ffff640 1 --2- 172.21.15.60:0/3438139791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac4015ada0 0x7fac40151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac4518c640 1 --2- 172.21.15.60:0/3438139791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac400b1470 0x7fac400ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac4498b640 1 -- 172.21.15.60:0/3438139791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac4015ada0 msgr2=0x7fac40151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac4498b640 1 --2- 172.21.15.60:0/3438139791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac4015ada0 0x7fac40151f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac4498b640 1 -- 172.21.15.60:0/3438139791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac400b1470 msgr2=0x7fac400ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac4498b640 1 --2- 172.21.15.60:0/3438139791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac400b1470 0x7fac400ada20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac4498b640 1 -- 172.21.15.60:0/3438139791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac401f39e0 con 0x7fac280051d0 2024-04-25T10:12:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac3ffff640 1 --2- 172.21.15.60:0/3438139791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac4015ada0 0x7fac40151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac4498b640 1 --2- 172.21.15.60:0/3438139791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac280051d0 0x7fac401f33b0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fac3000e7e0 tx=0x7fac3000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.504+0000 7fac3d7fa640 1 -- 172.21.15.60:0/3438139791 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3000c660 con 0x7fac280051d0 2024-04-25T10:12:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac3d7fa640 1 -- 172.21.15.60:0/3438139791 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac30018070 con 0x7fac280051d0 2024-04-25T10:12:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac46bef640 1 -- 172.21.15.60:0/3438139791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac401f3cc0 con 0x7fac280051d0 2024-04-25T10:12:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac3d7fa640 1 -- 172.21.15.60:0/3438139791 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3000ca40 con 0x7fac280051d0 2024-04-25T10:12:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac46bef640 1 -- 172.21.15.60:0/3438139791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac401f4280 con 0x7fac280051d0 2024-04-25T10:12:06.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac3d7fa640 1 -- 172.21.15.60:0/3438139791 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac3000cbe0 con 0x7fac280051d0 2024-04-25T10:12:06.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac3d7fa640 1 --2- 172.21.15.60:0/3438139791 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fac2c066f90 0x7fac2c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac3d7fa640 1 -- 172.21.15.60:0/3438139791 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fac30084aa0 con 0x7fac280051d0 2024-04-25T10:12:06.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac3ffff640 1 --2- 172.21.15.60:0/3438139791 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fac2c066f90 0x7fac2c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac49d5c000 1 Processor -- start 2024-04-25T10:12:06.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac49d5c000 1 -- start start 2024-04-25T10:12:06.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac3ffff640 1 --2- 172.21.15.60:0/3438139791 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fac2c066f90 0x7fac2c069450 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fac400ce4d0 tx=0x7fac34009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac49d5c000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab671b0 0x55f83ab67590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac49d5c000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab67ad0 0x55f83ab4b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac49d5c000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 0x55f83ab4c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac49d5c000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83a96afd0 con 0x55f83ab67ad0 2024-04-25T10:12:06.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac49d5c000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83a8f3170 con 0x55f83ab4bf10 2024-04-25T10:12:06.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac49d5c000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f83a967070 con 0x55f83ab671b0 2024-04-25T10:12:06.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac3ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab67ad0 0x55f83ab4b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac3ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab67ad0 0x55f83ab4b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53708/0 (socket says 172.21.15.60:53708) 2024-04-25T10:12:06.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.508+0000 7fac3ffff640 1 -- 172.21.15.60:0/45777805 learned_addr learned my addr 172.21.15.60:0/45777805 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:06.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac4518c640 1 --2- 172.21.15.60:0/45777805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 0x55f83ab4c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac4498b640 1 --2- 172.21.15.60:0/45777805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab671b0 0x55f83ab67590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac3ffff640 1 -- 172.21.15.60:0/45777805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab671b0 msgr2=0x55f83ab67590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac3ffff640 1 --2- 172.21.15.60:0/45777805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab671b0 0x55f83ab67590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac3ffff640 1 -- 172.21.15.60:0/45777805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 msgr2=0x55f83ab4c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac3ffff640 1 --2- 172.21.15.60:0/45777805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 0x55f83ab4c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac3ffff640 1 -- 172.21.15.60:0/45777805 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f83a9b1d20 con 0x55f83ab67ad0 2024-04-25T10:12:06.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac4498b640 1 --2- 172.21.15.60:0/45777805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab671b0 0x55f83ab67590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:06.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac4518c640 1 --2- 172.21.15.60:0/45777805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 0x55f83ab4c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:06.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac3ffff640 1 --2- 172.21.15.60:0/45777805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab67ad0 0x55f83ab4b9d0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fac3400e8d0 tx=0x7fac34011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fe5e9623f01f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac267fc640 1 -- 172.21.15.60:0/45777805 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3401a050 con 0x55f83ab67ad0 2024-04-25T10:12:06.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac267fc640 1 -- 172.21.15.60:0/45777805 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac34024070 con 0x55f83ab67ad0 2024-04-25T10:12:06.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac267fc640 1 -- 172.21.15.60:0/45777805 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac34013b10 con 0x55f83ab67ad0 2024-04-25T10:12:06.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 -- 172.21.15.60:0/45777805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab67ad0 msgr2=0x55f83ab4b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 --2- 172.21.15.60:0/45777805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab67ad0 0x55f83ab4b9d0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fac3400e8d0 tx=0x7fac34011000 comp rx=0 tx=0).stop 2024-04-25T10:12:06.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 -- 172.21.15.60:0/45777805 shutdown_connections 2024-04-25T10:12:06.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 --2- 172.21.15.60:0/45777805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 0x55f83ab4c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 --2- 172.21.15.60:0/45777805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab67ad0 0x55f83ab4b9d0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 --2- 172.21.15.60:0/45777805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab671b0 0x55f83ab67590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 -- 172.21.15.60:0/45777805 >> 172.21.15.60:0/45777805 conn(0x55f83ab23710 msgr2=0x55f83ab23af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 -- 172.21.15.60:0/45777805 shutdown_connections 2024-04-25T10:12:06.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 -- 172.21.15.60:0/45777805 wait complete. 2024-04-25T10:12:06.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 -- start start 2024-04-25T10:12:06.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 Processor -- start 2024-04-25T10:12:06.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 0x55f83ab6e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab671b0 0x55f83ab6ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab67ad0 0x55f83ab4c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83aa28c10 con 0x55f83ab671b0 2024-04-25T10:12:06.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83a925680 con 0x55f83ab4bf10 2024-04-25T10:12:06.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f83a96f600 con 0x55f83ab67ad0 2024-04-25T10:12:06.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 10 client.0 _ll_register_callbacks cb 0x55f83aa20080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:06.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac3ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab671b0 0x55f83ab6ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac3ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab671b0 0x55f83ab6ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53716/0 (socket says 172.21.15.60:53716) 2024-04-25T10:12:06.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac3ffff640 1 -- 172.21.15.60:0/2734900348 learned_addr learned my addr 172.21.15.60:0/2734900348 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:06.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac4498b640 1 --2- 172.21.15.60:0/2734900348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 0x55f83ab6e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac4518c640 1 --2- 172.21.15.60:0/2734900348 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab67ad0 0x55f83ab4c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac3ffff640 1 -- 172.21.15.60:0/2734900348 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab67ad0 msgr2=0x55f83ab4c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac3ffff640 1 --2- 172.21.15.60:0/2734900348 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab67ad0 0x55f83ab4c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac3ffff640 1 -- 172.21.15.60:0/2734900348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 msgr2=0x55f83ab6e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac3ffff640 1 --2- 172.21.15.60:0/2734900348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 0x55f83ab6e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac3ffff640 1 -- 172.21.15.60:0/2734900348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f83a942d30 con 0x55f83ab671b0 2024-04-25T10:12:06.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac3ffff640 1 --2- 172.21.15.60:0/2734900348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab671b0 0x55f83ab6ec60 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x55f83a968600 tx=0x7fac340011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b5a4dfefe2acd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:12:06.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac1f7fe640 1 -- 172.21.15.60:0/2734900348 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3401f6c0 con 0x55f83ab671b0 2024-04-25T10:12:06.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac1f7fe640 1 -- 172.21.15.60:0/2734900348 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fac34034b50 con 0x55f83ab671b0 2024-04-25T10:12:06.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.512+0000 7fac49d5c000 1 -- 172.21.15.60:0/2734900348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac34024070 con 0x55f83ab671b0 2024-04-25T10:12:06.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac1f7fe640 1 -- 172.21.15.60:0/2734900348 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac3401a020 con 0x55f83ab671b0 2024-04-25T10:12:06.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 10 client.8007 fetch_fsmap learned FSMap version 61 2024-04-25T10:12:06.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 1 -- 172.21.15.60:0/2734900348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55f83a8f33c0 con 0x55f83ab671b0 2024-04-25T10:12:06.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac1f7fe640 1 -- 172.21.15.60:0/2734900348 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac34024070 con 0x55f83ab671b0 2024-04-25T10:12:06.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac1f7fe640 1 -- 172.21.15.60:0/2734900348 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fac3401f860 con 0x55f83ab671b0 2024-04-25T10:12:06.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 10 client.8007 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T10:12:06.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 10 client.8007 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:12:06.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 20 client.8007 populate_metadata read hostname 'smithi060' 2024-04-25T10:12:06.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 1 --2- 172.21.15.60:0/2734900348 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55f83ab4cfd0 0x55f83ab4d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 4 client.8007 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:06.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 1 -- 172.21.15.60:0/2734900348 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f83a906e00 con 0x55f83ab4cfd0 2024-04-25T10:12:06.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac4498b640 1 --2- 172.21.15.60:0/2734900348 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55f83ab4cfd0 0x55f83ab4d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac1f7fe640 10 client.8007 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:06.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac4498b640 1 --2- 172.21.15.60:0/2734900348 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55f83ab4cfd0 0x55f83ab4d3e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6271869d8cb52f40 server_cookie=d49962bb71dfe743 in_seq=0 out_seq=0 2024-04-25T10:12:06.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac1f7fe640 1 -- 172.21.15.60:0/2734900348 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f83a906e00 con 0x55f83ab4cfd0 2024-04-25T10:12:06.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac1f7fe640 10 client.8007 handle_command_reply: tid=0 2024-04-25T10:12:06.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 1 client.8007 shutdown 2024-04-25T10:12:06.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 1 -- 172.21.15.60:0/2734900348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab671b0 msgr2=0x55f83ab6ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 1 --2- 172.21.15.60:0/2734900348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab671b0 0x55f83ab6ec60 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x55f83a968600 tx=0x7fac340011a0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 1 -- 172.21.15.60:0/2734900348 shutdown_connections 2024-04-25T10:12:06.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 1 --2- 172.21.15.60:0/2734900348 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55f83ab4cfd0 0x55f83ab4d3e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 1 --2- 172.21.15.60:0/2734900348 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab67ad0 0x55f83ab4c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 1 --2- 172.21.15.60:0/2734900348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab671b0 0x55f83ab6ec60 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 1 --2- 172.21.15.60:0/2734900348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 0x55f83ab6e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 1 -- 172.21.15.60:0/2734900348 >> 172.21.15.60:0/2734900348 conn(0x55f83ab23710 msgr2=0x55f83ab1b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac1f7fe640 0 client.8007 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:06.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 1 -- 172.21.15.60:0/2734900348 shutdown_connections 2024-04-25T10:12:06.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 1 -- 172.21.15.60:0/2734900348 wait complete. 2024-04-25T10:12:06.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.516+0000 7fac49d5c000 20 client.8007 trim_cache size 0 max 16384 2024-04-25T10:12:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 Processor -- start 2024-04-25T10:12:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 -- start start 2024-04-25T10:12:06.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab4bf10 0x55f83ab22e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab671b0 0x55f83ab65470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab67ad0 0x55f83ab659b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac40200280 con 0x55f83ab4bf10 2024-04-25T10:12:06.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac401ff480 con 0x55f83ab671b0 2024-04-25T10:12:06.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac40154e20 con 0x55f83ab67ad0 2024-04-25T10:12:06.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac4498b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab4bf10 0x55f83ab22e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac4498b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab4bf10 0x55f83ab22e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53724/0 (socket says 172.21.15.60:53724) 2024-04-25T10:12:06.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac4498b640 1 -- 172.21.15.60:0/189443641 learned_addr learned my addr 172.21.15.60:0/189443641 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:06.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac4518c640 1 --2- 172.21.15.60:0/189443641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab67ad0 0x55f83ab659b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac3ffff640 1 --2- 172.21.15.60:0/189443641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab671b0 0x55f83ab65470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac4498b640 1 -- 172.21.15.60:0/189443641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab67ad0 msgr2=0x55f83ab659b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac4498b640 1 --2- 172.21.15.60:0/189443641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab67ad0 0x55f83ab659b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac4498b640 1 -- 172.21.15.60:0/189443641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab671b0 msgr2=0x55f83ab65470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac4498b640 1 --2- 172.21.15.60:0/189443641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab671b0 0x55f83ab65470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac4498b640 1 -- 172.21.15.60:0/189443641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f83ab4c660 con 0x55f83ab4bf10 2024-04-25T10:12:06.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac4518c640 1 --2- 172.21.15.60:0/189443641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab67ad0 0x55f83ab659b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:06.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac4498b640 1 --2- 172.21.15.60:0/189443641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab4bf10 0x55f83ab22e60 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fac30001bf0 tx=0x7fac3001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da539ff7670b547b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac25ffb640 1 -- 172.21.15.60:0/189443641 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3005da50 con 0x55f83ab4bf10 2024-04-25T10:12:06.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac25ffb640 1 -- 172.21.15.60:0/189443641 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac30070070 con 0x55f83ab4bf10 2024-04-25T10:12:06.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac25ffb640 1 -- 172.21.15.60:0/189443641 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30053640 con 0x55f83ab4bf10 2024-04-25T10:12:06.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 -- 172.21.15.60:0/189443641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab4bf10 msgr2=0x55f83ab22e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 --2- 172.21.15.60:0/189443641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab4bf10 0x55f83ab22e60 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fac30001bf0 tx=0x7fac3001e040 comp rx=0 tx=0).stop 2024-04-25T10:12:06.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 -- 172.21.15.60:0/189443641 shutdown_connections 2024-04-25T10:12:06.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 --2- 172.21.15.60:0/189443641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab67ad0 0x55f83ab659b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 --2- 172.21.15.60:0/189443641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab671b0 0x55f83ab65470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 --2- 172.21.15.60:0/189443641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab4bf10 0x55f83ab22e60 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 -- 172.21.15.60:0/189443641 >> 172.21.15.60:0/189443641 conn(0x55f83ab23710 msgr2=0x55f83ab4af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 -- 172.21.15.60:0/189443641 shutdown_connections 2024-04-25T10:12:06.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.532+0000 7fac49d5c000 1 -- 172.21.15.60:0/189443641 wait complete. 2024-04-25T10:12:06.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 -- start start 2024-04-25T10:12:06.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 Processor -- start 2024-04-25T10:12:06.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 0x55f83abc75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab671b0 0x55f83abc7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab67ad0 0x55f83ab1c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83ab26d50 con 0x55f83ab671b0 2024-04-25T10:12:06.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac401fee80 con 0x55f83ab4bf10 2024-04-25T10:12:06.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f83ab4bc70 con 0x55f83ab67ad0 2024-04-25T10:12:06.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 10 client.0 _ll_register_callbacks cb 0x55f83a908b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:06.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac3ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab671b0 0x55f83abc7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac3ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab671b0 0x55f83abc7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53728/0 (socket says 172.21.15.60:53728) 2024-04-25T10:12:06.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac3ffff640 1 -- 172.21.15.60:0/1229941239 learned_addr learned my addr 172.21.15.60:0/1229941239 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:06.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac4498b640 1 --2- 172.21.15.60:0/1229941239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 0x55f83abc75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac4518c640 1 --2- 172.21.15.60:0/1229941239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab67ad0 0x55f83ab1c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac3ffff640 1 -- 172.21.15.60:0/1229941239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab67ad0 msgr2=0x55f83ab1c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac3ffff640 1 --2- 172.21.15.60:0/1229941239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab67ad0 0x55f83ab1c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac3ffff640 1 -- 172.21.15.60:0/1229941239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 msgr2=0x55f83abc75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac3ffff640 1 --2- 172.21.15.60:0/1229941239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 0x55f83abc75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac3ffff640 1 -- 172.21.15.60:0/1229941239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f83a9ed0f0 con 0x55f83ab671b0 2024-04-25T10:12:06.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac4498b640 1 --2- 172.21.15.60:0/1229941239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 0x55f83abc75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:06.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac3ffff640 1 --2- 172.21.15.60:0/1229941239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab671b0 0x55f83abc7b30 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fac3401f6f0 tx=0x7fac34014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc373f77ad769589 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:12:06.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac1f7fe640 1 -- 172.21.15.60:0/1229941239 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac34041030 con 0x55f83ab671b0 2024-04-25T10:12:06.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac1f7fe640 1 -- 172.21.15.60:0/1229941239 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fac3401fe00 con 0x55f83ab671b0 2024-04-25T10:12:06.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 -- 172.21.15.60:0/1229941239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac30070070 con 0x55f83ab671b0 2024-04-25T10:12:06.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac1f7fe640 1 -- 172.21.15.60:0/1229941239 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac3404a050 con 0x55f83ab671b0 2024-04-25T10:12:06.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 10 client.8013 fetch_fsmap learned FSMap version 61 2024-04-25T10:12:06.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 -- 172.21.15.60:0/1229941239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55f83a8ffd20 con 0x55f83ab671b0 2024-04-25T10:12:06.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac1f7fe640 1 -- 172.21.15.60:0/1229941239 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac30070070 con 0x55f83ab671b0 2024-04-25T10:12:06.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac1f7fe640 1 -- 172.21.15.60:0/1229941239 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fac3404d020 con 0x55f83ab671b0 2024-04-25T10:12:06.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 10 client.8013 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T10:12:06.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 10 client.8013 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:12:06.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 20 client.8013 populate_metadata read hostname 'smithi060' 2024-04-25T10:12:06.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 --2- 172.21.15.60:0/1229941239 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55f83ab1cfb0 0x55f83ab1d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 4 client.8013 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-04-25T10:12:06.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 -- 172.21.15.60:0/1229941239 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) v1 -- 0x55f83a978510 con 0x55f83ab1cfb0 2024-04-25T10:12:06.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac4498b640 1 --2- 172.21.15.60:0/1229941239 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55f83ab1cfb0 0x55f83ab1d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac4498b640 1 --2- 172.21.15.60:0/1229941239 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55f83ab1cfb0 0x55f83ab1d3c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2d7c5631c637c41 server_cookie=6c9da79c92594065 in_seq=0 out_seq=0 2024-04-25T10:12:06.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac1f7fe640 10 client.8013 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:06.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac1f7fe640 1 -- 172.21.15.60:0/1229941239 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f83a978510 con 0x55f83ab1cfb0 2024-04-25T10:12:06.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac1f7fe640 10 client.8013 handle_command_reply: tid=0 2024-04-25T10:12:06.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 client.8013 shutdown 2024-04-25T10:12:06.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 -- 172.21.15.60:0/1229941239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab671b0 msgr2=0x55f83abc7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 --2- 172.21.15.60:0/1229941239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab671b0 0x55f83abc7b30 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fac3401f6f0 tx=0x7fac34014000 comp rx=0 tx=0).stop 2024-04-25T10:12:06.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 -- 172.21.15.60:0/1229941239 shutdown_connections 2024-04-25T10:12:06.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 --2- 172.21.15.60:0/1229941239 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55f83ab1cfb0 0x55f83ab1d3c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 --2- 172.21.15.60:0/1229941239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f83ab67ad0 0x55f83ab1c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 --2- 172.21.15.60:0/1229941239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f83ab671b0 0x55f83abc7b30 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 --2- 172.21.15.60:0/1229941239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f83ab4bf10 0x55f83abc75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.536+0000 7fac49d5c000 1 -- 172.21.15.60:0/1229941239 >> 172.21.15.60:0/1229941239 conn(0x55f83ab23710 msgr2=0x55f83ab21a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.540+0000 7fac1f7fe640 0 client.8013 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:06.553 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-04-25T10:12:06.540+0000 7ff066d59640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-04-25T10:11:53.583631+0000 1=1+0) n(v0 rc2024-04-25T10:11:53.694129+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x55e103712100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:11:53.583631+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:11:53.694129+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-25T10:12:06.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.540+0000 7fac49d5c000 1 -- 172.21.15.60:0/1229941239 shutdown_connections 2024-04-25T10:12:06.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.540+0000 7fac49d5c000 1 -- 172.21.15.60:0/1229941239 wait complete. 2024-04-25T10:12:06.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.540+0000 7fac49d5c000 20 client.8013 trim_cache size 0 max 16384 2024-04-25T10:12:06.554 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:12:06.554 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T10:12:06.554 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "d02aaf26-bed9-49c2-8b6f-899fe995a92f", 2024-04-25T10:12:06.555 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T10:12:06.555 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:12:06.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.540+0000 7fac46bef640 1 -- 172.21.15.60:0/3438139791 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fac2c066f90 msgr2=0x7fac2c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.540+0000 7fac46bef640 1 --2- 172.21.15.60:0/3438139791 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fac2c066f90 0x7fac2c069450 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fac400ce4d0 tx=0x7fac34009040 comp rx=0 tx=0).stop 2024-04-25T10:12:06.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.540+0000 7fac46bef640 1 -- 172.21.15.60:0/3438139791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac280051d0 msgr2=0x7fac401f33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.540+0000 7fac46bef640 1 --2- 172.21.15.60:0/3438139791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac280051d0 0x7fac401f33b0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fac3000e7e0 tx=0x7fac3000eca0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.540+0000 7fac46bef640 1 -- 172.21.15.60:0/3438139791 shutdown_connections 2024-04-25T10:12:06.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.540+0000 7fac46bef640 1 --2- 172.21.15.60:0/3438139791 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fac2c066f90 0x7fac2c069450 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.540+0000 7fac46bef640 1 --2- 172.21.15.60:0/3438139791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac400b1470 0x7fac400ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.540+0000 7fac46bef640 1 --2- 172.21.15.60:0/3438139791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac4015ada0 0x7fac40151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.540+0000 7fac46bef640 1 --2- 172.21.15.60:0/3438139791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac280051d0 0x7fac401f33b0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.540+0000 7fac46bef640 1 -- 172.21.15.60:0/3438139791 >> 172.21.15.60:0/3438139791 conn(0x7fac40066fa0 msgr2=0x7fac4006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.540+0000 7fac46bef640 1 -- 172.21.15.60:0/3438139791 shutdown_connections 2024-04-25T10:12:06.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.540+0000 7fac46bef640 1 -- 172.21.15.60:0/3438139791 wait complete. 2024-04-25T10:12:06.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:06.565 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:12:06.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.676+0000 7f2a37193640 1 Processor -- start 2024-04-25T10:12:06.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.676+0000 7f2a37193640 1 -- start start 2024-04-25T10:12:06.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.676+0000 7f2a37193640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a3015dea0 0x7f2a300ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.676+0000 7f2a37193640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2a300ceda0 0x7f2a300d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.676+0000 7f2a37193640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a300d1bb0 con 0x7f2a300d0ea0 2024-04-25T10:12:06.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.676+0000 7f2a37193640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a300b9670 con 0x7f2a300ceda0 2024-04-25T10:12:06.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.676+0000 7f2a37193640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300d1300 con 0x7f2a3015dea0 2024-04-25T10:12:06.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a2ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2a300ceda0 0x7f2a300d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a2ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2a300ceda0 0x7f2a300d0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53744/0 (socket says 172.21.15.60:53744) 2024-04-25T10:12:06.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a2ffff640 1 -- 172.21.15.60:0/2662611609 learned_addr learned my addr 172.21.15.60:0/2662611609 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:06.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a34f2f640 1 --2- 172.21.15.60:0/2662611609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a3015dea0 0x7f2a300ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a2ffff640 1 -- 172.21.15.60:0/2662611609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a3015dea0 msgr2=0x7f2a300ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a2ffff640 1 --2- 172.21.15.60:0/2662611609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a3015dea0 0x7f2a300ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a2ffff640 1 -- 172.21.15.60:0/2662611609 >> v1:172.21.15.100:6789/0 conn(0x7f2a300d0ea0 legacy=0x7f2a300b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a2ffff640 1 -- 172.21.15.60:0/2662611609 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a300b85e0 con 0x7f2a300ceda0 2024-04-25T10:12:06.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a2ffff640 1 --2- 172.21.15.60:0/2662611609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a300ceda0 0x7f2a300d0960 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009870 tx=0x7f2a2401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcecd2090c1ac5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a2f7fe640 1 -- 172.21.15.60:0/2662611609 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a240aa610 con 0x7f2a300ceda0 2024-04-25T10:12:06.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a2f7fe640 1 -- 172.21.15.60:0/2662611609 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a240af070 con 0x7f2a300ceda0 2024-04-25T10:12:06.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a2f7fe640 1 -- 172.21.15.60:0/2662611609 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a240aa9d0 con 0x7f2a300ceda0 2024-04-25T10:12:06.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a37193640 1 -- 172.21.15.60:0/2662611609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a300ceda0 msgr2=0x7f2a300d0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a37193640 1 --2- 172.21.15.60:0/2662611609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a300ceda0 0x7f2a300d0960 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009870 tx=0x7f2a2401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a37193640 1 -- 172.21.15.60:0/2662611609 shutdown_connections 2024-04-25T10:12:06.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a37193640 1 --2- 172.21.15.60:0/2662611609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a300ceda0 0x7f2a300d0960 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a37193640 1 --2- 172.21.15.60:0/2662611609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a3015dea0 0x7f2a300ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a37193640 1 -- 172.21.15.60:0/2662611609 >> 172.21.15.60:0/2662611609 conn(0x7f2a3006c4d0 msgr2=0x7f2a3006c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a37193640 1 -- 172.21.15.60:0/2662611609 shutdown_connections 2024-04-25T10:12:06.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.680+0000 7f2a37193640 1 -- 172.21.15.60:0/2662611609 wait complete. 2024-04-25T10:12:06.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a37193640 1 Processor -- start 2024-04-25T10:12:06.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a37193640 1 -- start start 2024-04-25T10:12:06.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a37193640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a300ceda0 0x7f2a300c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a37193640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a300d0ea0 0x7f2a301598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a37193640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a30159e30 0x7f2a3015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a37193640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3016a850 con 0x7f2a30159e30 2024-04-25T10:12:06.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a37193640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300b8e80 con 0x7f2a300d0ea0 2024-04-25T10:12:06.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a37193640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3016a9d0 con 0x7f2a300ceda0 2024-04-25T10:12:06.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a35730640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a30159e30 0x7f2a3015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a35730640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a30159e30 0x7f2a3015ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53752/0 (socket says 172.21.15.60:53752) 2024-04-25T10:12:06.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a35730640 1 -- 172.21.15.60:0/354439823 learned_addr learned my addr 172.21.15.60:0/354439823 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:06.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a34f2f640 1 --2- 172.21.15.60:0/354439823 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a300ceda0 0x7f2a300c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a2ffff640 1 --2- 172.21.15.60:0/354439823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a300d0ea0 0x7f2a301598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a35730640 1 -- 172.21.15.60:0/354439823 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a300ceda0 msgr2=0x7f2a300c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a35730640 1 --2- 172.21.15.60:0/354439823 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a300ceda0 0x7f2a300c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a35730640 1 -- 172.21.15.60:0/354439823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a300d0ea0 msgr2=0x7f2a301598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a35730640 1 --2- 172.21.15.60:0/354439823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a300d0ea0 0x7f2a301598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a35730640 1 -- 172.21.15.60:0/354439823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a300cb8a0 con 0x7f2a30159e30 2024-04-25T10:12:06.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a35730640 1 --2- 172.21.15.60:0/354439823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a30159e30 0x7f2a3015ac60 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800c3f0 tx=0x7f2a2800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a2d7fa640 1 -- 172.21.15.60:0/354439823 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28011790 con 0x7f2a30159e30 2024-04-25T10:12:06.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a2d7fa640 1 -- 172.21.15.60:0/354439823 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a28017070 con 0x7f2a30159e30 2024-04-25T10:12:06.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a37193640 1 -- 172.21.15.60:0/354439823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3015a4b0 con 0x7f2a30159e30 2024-04-25T10:12:06.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a2d7fa640 1 -- 172.21.15.60:0/354439823 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28011b70 con 0x7f2a30159e30 2024-04-25T10:12:06.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.684+0000 7f2a37193640 1 -- 172.21.15.60:0/354439823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a300cbe20 con 0x7f2a30159e30 2024-04-25T10:12:06.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a2d7fa640 1 -- 172.21.15.60:0/354439823 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a28011d10 con 0x7f2a30159e30 2024-04-25T10:12:06.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a2d7fa640 1 --2- 172.21.15.60:0/354439823 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2a1c066f60 0x7f2a1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a2d7fa640 1 -- 172.21.15.60:0/354439823 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f2a28084ef0 con 0x7f2a30159e30 2024-04-25T10:12:06.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a34f2f640 1 --2- 172.21.15.60:0/354439823 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2a1c066f60 0x7f2a1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 Processor -- start 2024-04-25T10:12:06.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 -- start start 2024-04-25T10:12:06.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a34f2f640 1 --2- 172.21.15.60:0/354439823 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2a1c066f60 0x7f2a1c069420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2a20003fd0 tx=0x7f2a200039a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a599d0 0x563732a59db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a5a2f0 0x563732a0fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a100a0 0x563732a10480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56373285cff0 con 0x563732a599d0 2024-04-25T10:12:06.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637327e5150 con 0x563732a100a0 2024-04-25T10:12:06.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x563732859090 con 0x563732a5a2f0 2024-04-25T10:12:06.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a34f2f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a599d0 0x563732a59db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a34f2f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a599d0 0x563732a59db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53762/0 (socket says 172.21.15.60:53762) 2024-04-25T10:12:06.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a34f2f640 1 -- 172.21.15.60:0/923948134 learned_addr learned my addr 172.21.15.60:0/923948134 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:06.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a34f2f640 1 -- 172.21.15.60:0/923948134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a5a2f0 msgr2=0x563732a0fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a2ffff640 1 --2- 172.21.15.60:0/923948134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a5a2f0 0x563732a0fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a35730640 1 --2- 172.21.15.60:0/923948134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a100a0 0x563732a10480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a34f2f640 1 --2- 172.21.15.60:0/923948134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a5a2f0 0x563732a0fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a34f2f640 1 -- 172.21.15.60:0/923948134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a100a0 msgr2=0x563732a10480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a34f2f640 1 --2- 172.21.15.60:0/923948134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a100a0 0x563732a10480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a34f2f640 1 -- 172.21.15.60:0/923948134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637329f30d0 con 0x563732a599d0 2024-04-25T10:12:06.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a35730640 1 --2- 172.21.15.60:0/923948134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a100a0 0x563732a10480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:06.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a34f2f640 1 --2- 172.21.15.60:0/923948134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a599d0 0x563732a59db0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f2a200049e0 tx=0x7f2a20012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5174d0da52a588d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a167fc640 1 -- 172.21.15.60:0/923948134 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20026070 con 0x563732a599d0 2024-04-25T10:12:06.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a167fc640 1 -- 172.21.15.60:0/923948134 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a200147e0 con 0x563732a599d0 2024-04-25T10:12:06.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a167fc640 1 -- 172.21.15.60:0/923948134 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20014b90 con 0x563732a599d0 2024-04-25T10:12:06.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 -- 172.21.15.60:0/923948134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a599d0 msgr2=0x563732a59db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 --2- 172.21.15.60:0/923948134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a599d0 0x563732a59db0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f2a200049e0 tx=0x7f2a20012000 comp rx=0 tx=0).stop 2024-04-25T10:12:06.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 -- 172.21.15.60:0/923948134 shutdown_connections 2024-04-25T10:12:06.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 --2- 172.21.15.60:0/923948134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a100a0 0x563732a10480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 --2- 172.21.15.60:0/923948134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a5a2f0 0x563732a0fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 --2- 172.21.15.60:0/923948134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a599d0 0x563732a59db0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 -- 172.21.15.60:0/923948134 >> 172.21.15.60:0/923948134 conn(0x563732a0c020 msgr2=0x563732a0e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 -- 172.21.15.60:0/923948134 shutdown_connections 2024-04-25T10:12:06.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.688+0000 7f2a3a300000 1 -- 172.21.15.60:0/923948134 wait complete. 2024-04-25T10:12:06.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 1 -- start start 2024-04-25T10:12:06.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 1 Processor -- start 2024-04-25T10:12:06.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a100a0 0x563732a3e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a599d0 0x563732a3ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a5a2f0 0x563732a3f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563732821250 con 0x563732a5a2f0 2024-04-25T10:12:06.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563732817690 con 0x563732a599d0 2024-04-25T10:12:06.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5637327e5a40 con 0x563732a100a0 2024-04-25T10:12:06.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 10 client.0 _ll_register_callbacks cb 0x5637329fae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:06.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a35730640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a5a2f0 0x563732a3f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a35730640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a5a2f0 0x563732a3f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53766/0 (socket says 172.21.15.60:53766) 2024-04-25T10:12:06.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a35730640 1 -- 172.21.15.60:0/2037828417 learned_addr learned my addr 172.21.15.60:0/2037828417 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:06.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a34f2f640 1 --2- 172.21.15.60:0/2037828417 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a100a0 0x563732a3e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a2ffff640 1 --2- 172.21.15.60:0/2037828417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a599d0 0x563732a3ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a35730640 1 -- 172.21.15.60:0/2037828417 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a100a0 msgr2=0x563732a3e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a35730640 1 --2- 172.21.15.60:0/2037828417 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a100a0 0x563732a3e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a35730640 1 -- 172.21.15.60:0/2037828417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a599d0 msgr2=0x563732a3ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a35730640 1 --2- 172.21.15.60:0/2037828417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a599d0 0x563732a3ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a35730640 1 -- 172.21.15.60:0/2037828417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56373290a1c0 con 0x563732a5a2f0 2024-04-25T10:12:06.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a35730640 1 --2- 172.21.15.60:0/2037828417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a5a2f0 0x563732a3f270 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f2a28007fa0 tx=0x7f2a280557c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a1945bc5a3719a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:12:06.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a0f7fe640 1 -- 172.21.15.60:0/2037828417 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2806d170 con 0x563732a5a2f0 2024-04-25T10:12:06.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 1 -- 172.21.15.60:0/2037828417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a200147e0 con 0x563732a5a2f0 2024-04-25T10:12:06.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a0f7fe640 1 -- 172.21.15.60:0/2037828417 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f2a2805da10 con 0x563732a5a2f0 2024-04-25T10:12:06.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a0f7fe640 1 -- 172.21.15.60:0/2037828417 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a28073050 con 0x563732a5a2f0 2024-04-25T10:12:06.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 10 client.8025 fetch_fsmap learned FSMap version 61 2024-04-25T10:12:06.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 1 -- 172.21.15.60:0/2037828417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x563732906170 con 0x563732a5a2f0 2024-04-25T10:12:06.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a0f7fe640 1 -- 172.21.15.60:0/2037828417 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a200147e0 con 0x563732a5a2f0 2024-04-25T10:12:06.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a0f7fe640 1 -- 172.21.15.60:0/2037828417 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2a28054b50 con 0x563732a5a2f0 2024-04-25T10:12:06.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 10 client.8025 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T10:12:06.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 10 client.8025 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:12:06.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 20 client.8025 populate_metadata read hostname 'smithi060' 2024-04-25T10:12:06.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 1 --2- 172.21.15.60:0/2037828417 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x563732a69c80 0x563732a6c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 4 client.8025 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:06.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 1 -- 172.21.15.60:0/2037828417 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637327f8c60 con 0x563732a69c80 2024-04-25T10:12:06.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a34f2f640 1 --2- 172.21.15.60:0/2037828417 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x563732a69c80 0x563732a6c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a34f2f640 1 --2- 172.21.15.60:0/2037828417 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x563732a69c80 0x563732a6c070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4385308dac59bd42 server_cookie=42fd038e55f5fc5b in_seq=0 out_seq=0 2024-04-25T10:12:06.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a0f7fe640 10 client.8025 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:06.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a0f7fe640 1 -- 172.21.15.60:0/2037828417 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5637327f8c60 con 0x563732a69c80 2024-04-25T10:12:06.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a0f7fe640 10 client.8025 handle_command_reply: tid=0 2024-04-25T10:12:06.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.692+0000 7f2a3a300000 1 client.8025 shutdown 2024-04-25T10:12:06.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.696+0000 7f2a3a300000 1 -- 172.21.15.60:0/2037828417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a5a2f0 msgr2=0x563732a3f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.696+0000 7f2a3a300000 1 --2- 172.21.15.60:0/2037828417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a5a2f0 0x563732a3f270 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f2a28007fa0 tx=0x7f2a280557c0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.696+0000 7f2a3a300000 1 -- 172.21.15.60:0/2037828417 shutdown_connections 2024-04-25T10:12:06.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.696+0000 7f2a3a300000 1 --2- 172.21.15.60:0/2037828417 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x563732a69c80 0x563732a6c070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.696+0000 7f2a3a300000 1 --2- 172.21.15.60:0/2037828417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a5a2f0 0x563732a3f270 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.696+0000 7f2a3a300000 1 --2- 172.21.15.60:0/2037828417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a599d0 0x563732a3ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.696+0000 7f2a0f7fe640 0 client.8025 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:06.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.696+0000 7f2a3a300000 1 --2- 172.21.15.60:0/2037828417 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a100a0 0x563732a3e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.696+0000 7f2a3a300000 1 -- 172.21.15.60:0/2037828417 >> 172.21.15.60:0/2037828417 conn(0x563732a0c020 msgr2=0x563732a0eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.696+0000 7f2a3a300000 1 -- 172.21.15.60:0/2037828417 shutdown_connections 2024-04-25T10:12:06.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.696+0000 7f2a3a300000 1 -- 172.21.15.60:0/2037828417 wait complete. 2024-04-25T10:12:06.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.696+0000 7f2a3a300000 20 client.8025 trim_cache size 0 max 16384 2024-04-25T10:12:06.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.708+0000 7f2a3a300000 1 Processor -- start 2024-04-25T10:12:06.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.708+0000 7f2a3a300000 1 -- start start 2024-04-25T10:12:06.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a100a0 0x563732ab5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a599d0 0x563732ab6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a5a2f0 0x563732ab6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a302031a0 con 0x563732a599d0 2024-04-25T10:12:06.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a302023a0 con 0x563732a100a0 2024-04-25T10:12:06.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a302017a0 con 0x563732a5a2f0 2024-04-25T10:12:06.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a2ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a599d0 0x563732ab6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a2ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a599d0 0x563732ab6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53780/0 (socket says 172.21.15.60:53780) 2024-04-25T10:12:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a2ffff640 1 -- 172.21.15.60:0/679125119 learned_addr learned my addr 172.21.15.60:0/679125119 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a34f2f640 1 --2- 172.21.15.60:0/679125119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a100a0 0x563732ab5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a35730640 1 --2- 172.21.15.60:0/679125119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a5a2f0 0x563732ab6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a2ffff640 1 -- 172.21.15.60:0/679125119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a5a2f0 msgr2=0x563732ab6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a2ffff640 1 --2- 172.21.15.60:0/679125119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a5a2f0 0x563732ab6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a2ffff640 1 -- 172.21.15.60:0/679125119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a100a0 msgr2=0x563732ab5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a2ffff640 1 --2- 172.21.15.60:0/679125119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a100a0 0x563732ab5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a2ffff640 1 -- 172.21.15.60:0/679125119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637328a3de0 con 0x563732a599d0 2024-04-25T10:12:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a35730640 1 --2- 172.21.15.60:0/679125119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a5a2f0 0x563732ab6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a34f2f640 1 --2- 172.21.15.60:0/679125119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a100a0 0x563732ab5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a2ffff640 1 --2- 172.21.15.60:0/679125119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a599d0 0x563732ab6420 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f2a2401c530 tx=0x7f2a240a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72ae522a5aa9251 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a15ffb640 1 -- 172.21.15.60:0/679125119 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a240aa600 con 0x563732a599d0 2024-04-25T10:12:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a15ffb640 1 -- 172.21.15.60:0/679125119 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a240af040 con 0x563732a599d0 2024-04-25T10:12:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a15ffb640 1 -- 172.21.15.60:0/679125119 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a240aa9e0 con 0x563732a599d0 2024-04-25T10:12:06.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 -- 172.21.15.60:0/679125119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a599d0 msgr2=0x563732ab6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 --2- 172.21.15.60:0/679125119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a599d0 0x563732ab6420 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f2a2401c530 tx=0x7f2a240a7000 comp rx=0 tx=0).stop 2024-04-25T10:12:06.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 -- 172.21.15.60:0/679125119 shutdown_connections 2024-04-25T10:12:06.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 --2- 172.21.15.60:0/679125119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a5a2f0 0x563732ab6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 --2- 172.21.15.60:0/679125119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a599d0 0x563732ab6420 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 --2- 172.21.15.60:0/679125119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a100a0 0x563732ab5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 -- 172.21.15.60:0/679125119 >> 172.21.15.60:0/679125119 conn(0x563732a0c020 msgr2=0x563732a580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 -- 172.21.15.60:0/679125119 shutdown_connections 2024-04-25T10:12:06.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 -- 172.21.15.60:0/679125119 wait complete. 2024-04-25T10:12:06.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 -- start start 2024-04-25T10:12:06.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 Processor -- start 2024-04-25T10:12:06.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a100a0 0x563732a6cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a599d0 0x563732a6d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a5a2f0 0x563732a6d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637327f7e00 con 0x563732a100a0 2024-04-25T10:12:06.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a30201da0 con 0x563732a599d0 2024-04-25T10:12:06.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56373285c300 con 0x563732a5a2f0 2024-04-25T10:12:06.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a3a300000 10 client.0 _ll_register_callbacks cb 0x5637329db040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:06.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a34f2f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a100a0 0x563732a6cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a34f2f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a100a0 0x563732a6cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53784/0 (socket says 172.21.15.60:53784) 2024-04-25T10:12:06.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a34f2f640 1 -- 172.21.15.60:0/3285362425 learned_addr learned my addr 172.21.15.60:0/3285362425 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:06.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a35730640 1 --2- 172.21.15.60:0/3285362425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a5a2f0 0x563732a6d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.712+0000 7f2a2ffff640 1 --2- 172.21.15.60:0/3285362425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a599d0 0x563732a6d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a34f2f640 1 -- 172.21.15.60:0/3285362425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a5a2f0 msgr2=0x563732a6d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a34f2f640 1 --2- 172.21.15.60:0/3285362425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a5a2f0 0x563732a6d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a34f2f640 1 -- 172.21.15.60:0/3285362425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a599d0 msgr2=0x563732a6d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a34f2f640 1 --2- 172.21.15.60:0/3285362425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a599d0 0x563732a6d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a34f2f640 1 -- 172.21.15.60:0/3285362425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637327fe8e0 con 0x563732a100a0 2024-04-25T10:12:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a34f2f640 1 --2- 172.21.15.60:0/3285362425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a100a0 0x563732a6cbd0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2a20004410 tx=0x7f2a20004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8de3619a5ef04a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:12:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a0f7fe640 1 -- 172.21.15.60:0/3285362425 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20026040 con 0x563732a100a0 2024-04-25T10:12:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a0f7fe640 1 -- 172.21.15.60:0/3285362425 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f2a200146c0 con 0x563732a100a0 2024-04-25T10:12:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 1 -- 172.21.15.60:0/3285362425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a240af040 con 0x563732a100a0 2024-04-25T10:12:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a0f7fe640 1 -- 172.21.15.60:0/3285362425 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a2003f050 con 0x563732a100a0 2024-04-25T10:12:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 10 client.8031 fetch_fsmap learned FSMap version 61 2024-04-25T10:12:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 1 -- 172.21.15.60:0/3285362425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x5637328b68f0 con 0x563732a100a0 2024-04-25T10:12:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a0f7fe640 1 -- 172.21.15.60:0/3285362425 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a240af040 con 0x563732a100a0 2024-04-25T10:12:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a0f7fe640 1 -- 172.21.15.60:0/3285362425 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2a20042020 con 0x563732a100a0 2024-04-25T10:12:06.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 10 client.8031 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T10:12:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 10 client.8031 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:12:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 20 client.8031 populate_metadata read hostname 'smithi060' 2024-04-25T10:12:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 1 --2- 172.21.15.60:0/3285362425 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x563732abff50 0x563732ac2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 4 client.8031 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:12:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 1 -- 172.21.15.60:0/3285362425 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56373286a530 con 0x563732abff50 2024-04-25T10:12:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a2ffff640 1 --2- 172.21.15.60:0/3285362425 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x563732abff50 0x563732ac2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a2ffff640 1 --2- 172.21.15.60:0/3285362425 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x563732abff50 0x563732ac2340 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7800ca1f17e4c6d server_cookie=856aa7e8a4c2d71a in_seq=0 out_seq=0 2024-04-25T10:12:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a0f7fe640 10 client.8031 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a0f7fe640 1 -- 172.21.15.60:0/3285362425 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+256 (crc 0 0 0) 0x56373286a530 con 0x563732abff50 2024-04-25T10:12:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a0f7fe640 10 client.8031 handle_command_reply: tid=0 2024-04-25T10:12:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 1 client.8031 shutdown 2024-04-25T10:12:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 1 -- 172.21.15.60:0/3285362425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a100a0 msgr2=0x563732a6cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 1 --2- 172.21.15.60:0/3285362425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a100a0 0x563732a6cbd0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2a20004410 tx=0x7f2a20004ac0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 1 -- 172.21.15.60:0/3285362425 shutdown_connections 2024-04-25T10:12:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 1 --2- 172.21.15.60:0/3285362425 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x563732abff50 0x563732ac2340 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 1 --2- 172.21.15.60:0/3285362425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563732a5a2f0 0x563732a6d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 1 --2- 172.21.15.60:0/3285362425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563732a599d0 0x563732a6d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a0f7fe640 0 client.8031 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:06.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 1 --2- 172.21.15.60:0/3285362425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563732a100a0 0x563732a6cbd0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 1 -- 172.21.15.60:0/3285362425 >> 172.21.15.60:0/3285362425 conn(0x563732a0c020 msgr2=0x563732a58930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 1 -- 172.21.15.60:0/3285362425 shutdown_connections 2024-04-25T10:12:06.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 1 -- 172.21.15.60:0/3285362425 wait complete. 2024-04-25T10:12:06.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a3a300000 20 client.8031 trim_cache size 0 max 16384 2024-04-25T10:12:06.730 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:12:06.731 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:12:06.731 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T10:12:06.731 INFO:teuthology.orchestra.run.smithi060.stdout: "d02aaf26-bed9-49c2-8b6f-899fe995a92f": { 2024-04-25T10:12:06.731 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/dir", 2024-04-25T10:12:06.731 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "d02aaf26-bed9-49c2-8b6f-899fe995a92f", 2024-04-25T10:12:06.731 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive" 2024-04-25T10:12:06.731 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:12:06.731 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:12:06.731 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:12:06.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a37193640 1 -- 172.21.15.60:0/354439823 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2a1c066f60 msgr2=0x7f2a1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a37193640 1 --2- 172.21.15.60:0/354439823 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2a1c066f60 0x7f2a1c069420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2a20003fd0 tx=0x7f2a200039a0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a37193640 1 -- 172.21.15.60:0/354439823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a30159e30 msgr2=0x7f2a3015ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a37193640 1 --2- 172.21.15.60:0/354439823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a30159e30 0x7f2a3015ac60 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800c3f0 tx=0x7f2a2800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a37193640 1 -- 172.21.15.60:0/354439823 shutdown_connections 2024-04-25T10:12:06.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a37193640 1 --2- 172.21.15.60:0/354439823 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2a1c066f60 0x7f2a1c069420 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a37193640 1 --2- 172.21.15.60:0/354439823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a30159e30 0x7f2a3015ac60 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a37193640 1 --2- 172.21.15.60:0/354439823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a300d0ea0 0x7f2a301598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a37193640 1 --2- 172.21.15.60:0/354439823 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a300ceda0 0x7f2a300c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a37193640 1 -- 172.21.15.60:0/354439823 >> 172.21.15.60:0/354439823 conn(0x7f2a3006c4d0 msgr2=0x7f2a300d3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a37193640 1 -- 172.21.15.60:0/354439823 shutdown_connections 2024-04-25T10:12:06.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:06.716+0000 7f2a37193640 1 -- 172.21.15.60:0/354439823 wait complete. 2024-04-25T10:12:06.742 INFO:tasks.cephfs.filesystem:scrub status for tag:d02aaf26-bed9-49c2-8b6f-899fe995a92f - {'path': '/dir', 'tag': 'd02aaf26-bed9-49c2-8b6f-899fe995a92f', 'options': 'recursive'} 2024-04-25T10:12:09.587 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:12:09.580+0000 7f3a7badb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:12:17.243 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:17.246 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:36.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:36.743 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:12:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efefdc7c640 1 Processor -- start 2024-04-25T10:12:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efefdc7c640 1 -- start start 2024-04-25T10:12:36.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efefdc7c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efef80b5470 0x7efef80b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efefdc7c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef815e160 con 0x7efef80b2860 2024-04-25T10:12:36.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efefdc7c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef80d1b20 con 0x7efef815dd80 2024-04-25T10:12:36.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efefdc7c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef80b5e20 con 0x7efef80b5470 2024-04-25T10:12:36.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef6ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efef80b2860 0x7efef80b9310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41198/0 (socket says 172.21.15.60:41198) 2024-04-25T10:12:36.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef6ffd640 1 -- 172.21.15.60:0/2949721314 learned_addr learned my addr 172.21.15.60:0/2949721314 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:12:36.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef7fff640 1 --2- 172.21.15.60:0/2949721314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efef80b5470 0x7efef80b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef67fc640 1 -- 172.21.15.60:0/2949721314 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227954048 0 0) 0x7efef815e160 con 0x7efef80b2860 2024-04-25T10:12:36.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef67fc640 1 -- 172.21.15.60:0/2949721314 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efee0003530 con 0x7efef80b2860 2024-04-25T10:12:36.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef67fc640 1 -- 172.21.15.60:0/2949721314 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 54452710 0 0) 0x7efee0003530 con 0x7efef80b2860 2024-04-25T10:12:36.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef67fc640 1 -- 172.21.15.60:0/2949721314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efef80b5470 msgr2=0x7efef80b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef67fc640 1 --2- 172.21.15.60:0/2949721314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efef80b5470 0x7efef80b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef67fc640 1 -- 172.21.15.60:0/2949721314 >> v1:172.21.15.100:6789/0 conn(0x7efef815dd80 legacy=0x7efef80b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef67fc640 1 -- 172.21.15.60:0/2949721314 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef80d1e00 con 0x7efef80b2860 2024-04-25T10:12:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef67fc640 1 -- 172.21.15.60:0/2949721314 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7efeec001680 con 0x7efef80b2860 2024-04-25T10:12:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef67fc640 1 -- 172.21.15.60:0/2949721314 >> v1:172.21.15.60:6789/0 conn(0x7efef80b2860 legacy=0x7efef80b9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef67fc640 1 --2- 172.21.15.60:0/2949721314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efef80b5470 0x7efee00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef67fc640 1 --2- 172.21.15.60:0/2949721314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee0003f70 0x7efee0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef67fc640 1 --2- 172.21.15.60:0/2949721314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efee0004d50 0x7efee00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef67fc640 1 -- 172.21.15.60:0/2949721314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef80b5e20 con 0x7efee0003f70 2024-04-25T10:12:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef67fc640 1 -- 172.21.15.60:0/2949721314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee000bab0 con 0x7efee0004d50 2024-04-25T10:12:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef67fc640 1 -- 172.21.15.60:0/2949721314 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee000bcb0 con 0x7efef80b5470 2024-04-25T10:12:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef6ffd640 1 --2- 172.21.15.60:0/2949721314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee0003f70 0x7efee0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef77fe640 1 --2- 172.21.15.60:0/2949721314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efef80b5470 0x7efee00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef7fff640 1 --2- 172.21.15.60:0/2949721314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efee0004d50 0x7efee00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef6ffd640 1 -- 172.21.15.60:0/2949721314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efef80b5470 msgr2=0x7efee00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef6ffd640 1 --2- 172.21.15.60:0/2949721314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efef80b5470 0x7efee00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef6ffd640 1 -- 172.21.15.60:0/2949721314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efee0004d50 msgr2=0x7efee00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef6ffd640 1 --2- 172.21.15.60:0/2949721314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efee0004d50 0x7efee00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.876+0000 7efef6ffd640 1 -- 172.21.15.60:0/2949721314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee0003530 con 0x7efee0003f70 2024-04-25T10:12:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efef6ffd640 1 --2- 172.21.15.60:0/2949721314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee0003f70 0x7efee0004810 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7efef80b2c80 tx=0x7efeec01f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6b4374f888090 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efef67fc640 1 -- 172.21.15.60:0/2949721314 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec007d40 con 0x7efee0003f70 2024-04-25T10:12:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efef67fc640 1 -- 172.21.15.60:0/2949721314 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec005b00 con 0x7efee0003f70 2024-04-25T10:12:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efef67fc640 1 -- 172.21.15.60:0/2949721314 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec007070 con 0x7efee0003f70 2024-04-25T10:12:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efefdc7c640 1 -- 172.21.15.60:0/2949721314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee0003f70 msgr2=0x7efee0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efefdc7c640 1 --2- 172.21.15.60:0/2949721314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee0003f70 0x7efee0004810 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7efef80b2c80 tx=0x7efeec01f3d0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efef7fff640 1 -- 172.21.15.60:0/2949721314 reap_dead start 2024-04-25T10:12:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efefdc7c640 1 -- 172.21.15.60:0/2949721314 shutdown_connections 2024-04-25T10:12:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efefdc7c640 1 -- 172.21.15.60:0/2949721314 >> 172.21.15.60:0/2949721314 conn(0x7efef80d0470 msgr2=0x7efef80d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efefdc7c640 1 -- 172.21.15.60:0/2949721314 shutdown_connections 2024-04-25T10:12:36.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efefdc7c640 1 -- 172.21.15.60:0/2949721314 wait complete. 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efefdc7c640 1 Processor -- start 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efefdc7c640 1 -- start start 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efefdc7c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efee0003f70 0x7efef814deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efefdc7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efef80ca220 0x7efef80cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efefdc7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efef815e360 0x7efef8160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.880+0000 7efefdc7c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef80d26a0 con 0x7efef80ca220 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efefdc7c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef80d23c0 con 0x7efef815e360 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efefdc7c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef80b58e0 con 0x7efee0003f70 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efef6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efef80ca220 0x7efef80cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efef6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efef80ca220 0x7efef80cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33938/0 (socket says 172.21.15.60:33938) 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efef6ffd640 1 -- 172.21.15.60:0/3304694368 learned_addr learned my addr 172.21.15.60:0/3304694368 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efef7fff640 1 --2- 172.21.15.60:0/3304694368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efef815e360 0x7efef8160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efef77fe640 1 --2- 172.21.15.60:0/3304694368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efee0003f70 0x7efef814deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efef6ffd640 1 -- 172.21.15.60:0/3304694368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efee0003f70 msgr2=0x7efef814deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efef6ffd640 1 --2- 172.21.15.60:0/3304694368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efee0003f70 0x7efef814deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efef6ffd640 1 -- 172.21.15.60:0/3304694368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efef815e360 msgr2=0x7efef8160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efef6ffd640 1 --2- 172.21.15.60:0/3304694368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efef815e360 0x7efef8160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efef6ffd640 1 -- 172.21.15.60:0/3304694368 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef80ccd10 con 0x7efef80ca220 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efef7fff640 1 --2- 172.21.15.60:0/3304694368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efef815e360 0x7efef8160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efef6ffd640 1 --2- 172.21.15.60:0/3304694368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efef80ca220 0x7efef80cc3c0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7efeec026040 tx=0x7efee0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efedffff640 1 -- 172.21.15.60:0/3304694368 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec007d20 con 0x7efef80ca220 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efedffff640 1 -- 172.21.15.60:0/3304694368 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec0b6040 con 0x7efef80ca220 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efedffff640 1 -- 172.21.15.60:0/3304694368 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec0049f0 con 0x7efef80ca220 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efefdc7c640 1 -- 172.21.15.60:0/3304694368 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef80c7650 con 0x7efef80ca220 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efefdc7c640 1 -- 172.21.15.60:0/3304694368 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef80c7c10 con 0x7efef80ca220 2024-04-25T10:12:36.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efedffff640 1 -- 172.21.15.60:0/3304694368 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efeec004b90 con 0x7efef80ca220 2024-04-25T10:12:36.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efedffff640 1 --2- 172.21.15.60:0/3304694368 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efee4066f60 0x7efee4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efedffff640 1 -- 172.21.15.60:0/3304694368 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7efeec122250 con 0x7efef80ca220 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7efef77fe640 1 --2- 172.21.15.60:0/3304694368 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efee4066f60 0x7efee4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.884+0000 7eff00de9000 1 Processor -- start 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 -- start start 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7efef77fe640 1 --2- 172.21.15.60:0/3304694368 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efee4066f60 0x7efee4069420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efee80015e0 tx=0x7efee8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b79d0 0x55e9221b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e9221b82f0 0x55e92216db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e92216e0a0 0x55e92216e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e921fbaff0 con 0x55e9221b79d0 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e921f43150 con 0x55e92216e0a0 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e921fb7090 con 0x55e9221b82f0 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7efef77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b79d0 0x55e9221b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7efef77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b79d0 0x55e9221b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33948/0 (socket says 172.21.15.60:33948) 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7efef77fe640 1 -- 172.21.15.60:0/3043468328 learned_addr learned my addr 172.21.15.60:0/3043468328 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7efef7fff640 1 --2- 172.21.15.60:0/3043468328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e92216e0a0 0x55e92216e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7efef6ffd640 1 --2- 172.21.15.60:0/3043468328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e9221b82f0 0x55e92216db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7efef77fe640 1 -- 172.21.15.60:0/3043468328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e9221b82f0 msgr2=0x55e92216db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7efef77fe640 1 --2- 172.21.15.60:0/3043468328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e9221b82f0 0x55e92216db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7efef77fe640 1 -- 172.21.15.60:0/3043468328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e92216e0a0 msgr2=0x55e92216e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7efef77fe640 1 --2- 172.21.15.60:0/3043468328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e92216e0a0 0x55e92216e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7efef77fe640 1 -- 172.21.15.60:0/3043468328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9221510d0 con 0x55e9221b79d0 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7efef77fe640 1 --2- 172.21.15.60:0/3043468328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b79d0 0x55e9221b7db0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7efee800f870 tx=0x7efee8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24f7c0a9775ee37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7efedd7fa640 1 -- 172.21.15.60:0/3043468328 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8025070 con 0x55e9221b79d0 2024-04-25T10:12:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7efedd7fa640 1 -- 172.21.15.60:0/3043468328 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee801b050 con 0x55e9221b79d0 2024-04-25T10:12:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7efedd7fa640 1 -- 172.21.15.60:0/3043468328 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8014c10 con 0x55e9221b79d0 2024-04-25T10:12:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 -- 172.21.15.60:0/3043468328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b79d0 msgr2=0x55e9221b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 --2- 172.21.15.60:0/3043468328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b79d0 0x55e9221b7db0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7efee800f870 tx=0x7efee8012000 comp rx=0 tx=0).stop 2024-04-25T10:12:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 -- 172.21.15.60:0/3043468328 shutdown_connections 2024-04-25T10:12:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 --2- 172.21.15.60:0/3043468328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e92216e0a0 0x55e92216e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 --2- 172.21.15.60:0/3043468328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e9221b82f0 0x55e92216db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 --2- 172.21.15.60:0/3043468328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b79d0 0x55e9221b7db0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 -- 172.21.15.60:0/3043468328 >> 172.21.15.60:0/3043468328 conn(0x55e92216a020 msgr2=0x55e92216c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 -- 172.21.15.60:0/3043468328 shutdown_connections 2024-04-25T10:12:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 -- 172.21.15.60:0/3043468328 wait complete. 2024-04-25T10:12:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 -- start start 2024-04-25T10:12:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.888+0000 7eff00de9000 1 Processor -- start 2024-04-25T10:12:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e92216e0a0 0x55e92219c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b79d0 0x55e92219cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9221b82f0 0x55e92219d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e921f7f250 con 0x55e9221b79d0 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e921f75690 con 0x55e9221b82f0 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e921f43a40 con 0x55e92216e0a0 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 10 client.0 _ll_register_callbacks cb 0x55e922158e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efef6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b79d0 0x55e92219cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efef6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b79d0 0x55e92219cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33960/0 (socket says 172.21.15.60:33960) 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efef6ffd640 1 -- 172.21.15.60:0/600397375 learned_addr learned my addr 172.21.15.60:0/600397375 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efef77fe640 1 --2- 172.21.15.60:0/600397375 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e92216e0a0 0x55e92219c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efef7fff640 1 --2- 172.21.15.60:0/600397375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9221b82f0 0x55e92219d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efef6ffd640 1 -- 172.21.15.60:0/600397375 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e92216e0a0 msgr2=0x55e92219c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efef6ffd640 1 --2- 172.21.15.60:0/600397375 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e92216e0a0 0x55e92219c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efef6ffd640 1 -- 172.21.15.60:0/600397375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9221b82f0 msgr2=0x55e92219d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efef6ffd640 1 --2- 172.21.15.60:0/600397375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9221b82f0 0x55e92219d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efef6ffd640 1 -- 172.21.15.60:0/600397375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9220681c0 con 0x55e9221b79d0 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efef6ffd640 1 --2- 172.21.15.60:0/600397375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b79d0 0x55e92219cd30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7efeec0f2a90 tx=0x7efeec0f2640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6061b01cd95d28d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efed67fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efed67fc640 1 -- 172.21.15.60:0/600397375 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec0f2b00 con 0x55e9221b79d0 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 -- 172.21.15.60:0/600397375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efee801b050 con 0x55e9221b79d0 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efed67fc640 1 -- 172.21.15.60:0/600397375 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7efeec0ef170 con 0x55e9221b79d0 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efed67fc640 1 -- 172.21.15.60:0/600397375 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7efeec0f6c50 con 0x55e9221b79d0 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 10 client.8043 fetch_fsmap learned FSMap version 62 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 -- 172.21.15.60:0/600397375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55e922064170 con 0x55e9221b79d0 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efed67fc640 1 -- 172.21.15.60:0/600397375 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee801b050 con 0x55e9221b79d0 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efed67fc640 1 -- 172.21.15.60:0/600397375 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efeec0a8ce0 con 0x55e9221b79d0 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 10 client.8043 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 10 client.8043 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 20 client.8043 populate_metadata read hostname 'smithi060' 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 --2- 172.21.15.60:0/600397375 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55e9221c7c80 0x55e9221ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 4 client.8043 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 -- 172.21.15.60:0/600397375 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e921f56c60 con 0x55e9221c7c80 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efef77fe640 1 --2- 172.21.15.60:0/600397375 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55e9221c7c80 0x55e9221ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efef77fe640 1 --2- 172.21.15.60:0/600397375 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55e9221c7c80 0x55e9221ca070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15bbec26f22a9038 server_cookie=5973752570536fb1 in_seq=0 out_seq=0 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efed67fc640 10 client.8043 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efed67fc640 1 -- 172.21.15.60:0/600397375 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e921f56c60 con 0x55e9221c7c80 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efed67fc640 10 client.8043 handle_command_reply: tid=0 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 client.8043 shutdown 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 -- 172.21.15.60:0/600397375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b79d0 msgr2=0x55e92219cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 --2- 172.21.15.60:0/600397375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b79d0 0x55e92219cd30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7efeec0f2a90 tx=0x7efeec0f2640 comp rx=0 tx=0).stop 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 -- 172.21.15.60:0/600397375 shutdown_connections 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 --2- 172.21.15.60:0/600397375 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55e9221c7c80 0x55e9221ca070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 --2- 172.21.15.60:0/600397375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9221b82f0 0x55e92219d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 --2- 172.21.15.60:0/600397375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b79d0 0x55e92219cd30 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 --2- 172.21.15.60:0/600397375 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e92216e0a0 0x55e92219c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7efed67fc640 0 client.8043 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 -- 172.21.15.60:0/600397375 >> 172.21.15.60:0/600397375 conn(0x55e92216a020 msgr2=0x55e92216cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 -- 172.21.15.60:0/600397375 shutdown_connections 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 1 -- 172.21.15.60:0/600397375 wait complete. 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.892+0000 7eff00de9000 20 client.8043 trim_cache size 0 max 16384 2024-04-25T10:12:36.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7eff00de9000 1 Processor -- start 2024-04-25T10:12:36.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7eff00de9000 1 -- start start 2024-04-25T10:12:36.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7eff00de9000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e92216e0a0 0x55e9221b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7eff00de9000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9221b79d0 0x55e92216d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7eff00de9000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b82f0 0x55e92216d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7eff00de9000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef8200520 con 0x55e9221b82f0 2024-04-25T10:12:36.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7eff00de9000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef81ff700 con 0x55e9221b79d0 2024-04-25T10:12:36.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7eff00de9000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef81f77d0 con 0x55e92216e0a0 2024-04-25T10:12:36.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7efef7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b82f0 0x55e92216d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7efef7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b82f0 0x55e92216d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33966/0 (socket says 172.21.15.60:33966) 2024-04-25T10:12:36.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7efef7fff640 1 -- 172.21.15.60:0/2537207466 learned_addr learned my addr 172.21.15.60:0/2537207466 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:36.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7efef7fff640 1 -- 172.21.15.60:0/2537207466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e92216e0a0 msgr2=0x55e9221b6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7efef7fff640 1 --2- 172.21.15.60:0/2537207466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e92216e0a0 0x55e9221b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7efef7fff640 1 -- 172.21.15.60:0/2537207466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9221b79d0 msgr2=0x55e92216d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7efef7fff640 1 --2- 172.21.15.60:0/2537207466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9221b79d0 0x55e92216d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7efef7fff640 1 -- 172.21.15.60:0/2537207466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e922001de0 con 0x55e9221b82f0 2024-04-25T10:12:36.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7efef7fff640 1 --2- 172.21.15.60:0/2537207466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b82f0 0x55e92216d960 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7efef80d4ac0 tx=0x7efef0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c991efa9f20137 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7efedcff9640 1 -- 172.21.15.60:0/2537207466 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef001c070 con 0x55e9221b82f0 2024-04-25T10:12:36.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7efedcff9640 1 -- 172.21.15.60:0/2537207466 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef00188f0 con 0x55e9221b82f0 2024-04-25T10:12:36.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.908+0000 7efedcff9640 1 -- 172.21.15.60:0/2537207466 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef0018ce0 con 0x55e9221b82f0 2024-04-25T10:12:36.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- 172.21.15.60:0/2537207466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b82f0 msgr2=0x55e92216d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 --2- 172.21.15.60:0/2537207466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b82f0 0x55e92216d960 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7efef80d4ac0 tx=0x7efef0009750 comp rx=0 tx=0).stop 2024-04-25T10:12:36.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- 172.21.15.60:0/2537207466 shutdown_connections 2024-04-25T10:12:36.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 --2- 172.21.15.60:0/2537207466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e9221b82f0 0x55e92216d960 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 --2- 172.21.15.60:0/2537207466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9221b79d0 0x55e92216d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 --2- 172.21.15.60:0/2537207466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e92216e0a0 0x55e9221b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- 172.21.15.60:0/2537207466 >> 172.21.15.60:0/2537207466 conn(0x55e92216a020 msgr2=0x55e9221b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- 172.21.15.60:0/2537207466 shutdown_connections 2024-04-25T10:12:36.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- 172.21.15.60:0/2537207466 wait complete. 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- start start 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 Processor -- start 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e92216e0a0 0x55e922220d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9221b79d0 0x55e922221270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e9221b82f0 0x55e9221ca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e921f55e00 con 0x55e92216e0a0 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef81ff100 con 0x55e9221b79d0 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e921fba300 con 0x55e9221b82f0 2024-04-25T10:12:36.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 10 client.0 _ll_register_callbacks cb 0x55e922139040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:36.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efef77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e92216e0a0 0x55e922220d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efef77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e92216e0a0 0x55e922220d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33974/0 (socket says 172.21.15.60:33974) 2024-04-25T10:12:36.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efef77fe640 1 -- 172.21.15.60:0/3721350061 learned_addr learned my addr 172.21.15.60:0/3721350061 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:36.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efef77fe640 1 -- 172.21.15.60:0/3721350061 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e9221b82f0 msgr2=0x55e9221ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efef77fe640 1 --2- 172.21.15.60:0/3721350061 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e9221b82f0 0x55e9221ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efef77fe640 1 -- 172.21.15.60:0/3721350061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9221b79d0 msgr2=0x55e922221270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efef77fe640 1 --2- 172.21.15.60:0/3721350061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9221b79d0 0x55e922221270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efef77fe640 1 -- 172.21.15.60:0/3721350061 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e921f5c8e0 con 0x55e92216e0a0 2024-04-25T10:12:36.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efef77fe640 1 --2- 172.21.15.60:0/3721350061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e92216e0a0 0x55e922220d30 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7efee8009010 tx=0x7efee8001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1996774b64b3820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efed67fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:12:36.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efed67fc640 1 -- 172.21.15.60:0/3721350061 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8038060 con 0x55e92216e0a0 2024-04-25T10:12:36.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efed67fc640 1 -- 172.21.15.60:0/3721350061 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7efee802d440 con 0x55e92216e0a0 2024-04-25T10:12:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- 172.21.15.60:0/3721350061 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efef00188f0 con 0x55e92216e0a0 2024-04-25T10:12:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efed67fc640 1 -- 172.21.15.60:0/3721350061 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7efee8025040 con 0x55e92216e0a0 2024-04-25T10:12:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 10 client.8049 fetch_fsmap learned FSMap version 62 2024-04-25T10:12:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- 172.21.15.60:0/3721350061 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55e9220148f0 con 0x55e92216e0a0 2024-04-25T10:12:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efed67fc640 1 -- 172.21.15.60:0/3721350061 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef00188f0 con 0x55e92216e0a0 2024-04-25T10:12:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efed67fc640 1 -- 172.21.15.60:0/3721350061 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efee802b040 con 0x55e92216e0a0 2024-04-25T10:12:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 10 client.8049 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:12:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 10 client.8049 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:12:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 20 client.8049 populate_metadata read hostname 'smithi060' 2024-04-25T10:12:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 --2- 172.21.15.60:0/3721350061 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55e9221cad40 0x55e9221cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 4 client.8049 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:12:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- 172.21.15.60:0/3721350061 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e921fc8530 con 0x55e9221cad40 2024-04-25T10:12:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efef6ffd640 1 --2- 172.21.15.60:0/3721350061 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55e9221cad40 0x55e9221cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efed67fc640 10 client.8049 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efef6ffd640 1 --2- 172.21.15.60:0/3721350061 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55e9221cad40 0x55e9221cb120 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=641fb02c28511faa server_cookie=78684c322d25a1da in_seq=0 out_seq=0 2024-04-25T10:12:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efed67fc640 1 -- 172.21.15.60:0/3721350061 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e921fc8530 con 0x55e9221cad40 2024-04-25T10:12:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efed67fc640 10 client.8049 handle_command_reply: tid=0 2024-04-25T10:12:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 client.8049 shutdown 2024-04-25T10:12:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- 172.21.15.60:0/3721350061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e92216e0a0 msgr2=0x55e922220d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 --2- 172.21.15.60:0/3721350061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e92216e0a0 0x55e922220d30 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7efee8009010 tx=0x7efee8001130 comp rx=0 tx=0).stop 2024-04-25T10:12:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- 172.21.15.60:0/3721350061 shutdown_connections 2024-04-25T10:12:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 --2- 172.21.15.60:0/3721350061 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55e9221cad40 0x55e9221cb120 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 --2- 172.21.15.60:0/3721350061 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e9221b82f0 0x55e9221ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 --2- 172.21.15.60:0/3721350061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9221b79d0 0x55e922221270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7efed67fc640 0 client.8049 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 --2- 172.21.15.60:0/3721350061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e92216e0a0 0x55e922220d30 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- 172.21.15.60:0/3721350061 >> 172.21.15.60:0/3721350061 conn(0x55e92216a020 msgr2=0x55e9221b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- 172.21.15.60:0/3721350061 shutdown_connections 2024-04-25T10:12:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.912+0000 7eff00de9000 1 -- 172.21.15.60:0/3721350061 wait complete. 2024-04-25T10:12:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.916+0000 7eff00de9000 20 client.8049 trim_cache size 0 max 16384 2024-04-25T10:12:36.929 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:12:36.929 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T10:12:36.929 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T10:12:36.929 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:12:36.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.916+0000 7efefdc7c640 1 -- 172.21.15.60:0/3304694368 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efee4066f60 msgr2=0x7efee4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.916+0000 7efefdc7c640 1 --2- 172.21.15.60:0/3304694368 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efee4066f60 0x7efee4069420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efee80015e0 tx=0x7efee8009040 comp rx=0 tx=0).stop 2024-04-25T10:12:36.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.916+0000 7efefdc7c640 1 -- 172.21.15.60:0/3304694368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efef80ca220 msgr2=0x7efef80cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.916+0000 7efefdc7c640 1 --2- 172.21.15.60:0/3304694368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efef80ca220 0x7efef80cc3c0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7efeec026040 tx=0x7efee0005840 comp rx=0 tx=0).stop 2024-04-25T10:12:36.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.916+0000 7efefdc7c640 1 -- 172.21.15.60:0/3304694368 shutdown_connections 2024-04-25T10:12:36.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.916+0000 7efefdc7c640 1 --2- 172.21.15.60:0/3304694368 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efee4066f60 0x7efee4069420 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.916+0000 7efefdc7c640 1 --2- 172.21.15.60:0/3304694368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efef815e360 0x7efef8160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.916+0000 7efefdc7c640 1 --2- 172.21.15.60:0/3304694368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efef80ca220 0x7efef80cc3c0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.916+0000 7efefdc7c640 1 --2- 172.21.15.60:0/3304694368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efee0003f70 0x7efef814deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.916+0000 7efefdc7c640 1 -- 172.21.15.60:0/3304694368 >> 172.21.15.60:0/3304694368 conn(0x7efef80d0470 msgr2=0x7efef80b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.916+0000 7efefdc7c640 1 -- 172.21.15.60:0/3304694368 shutdown_connections 2024-04-25T10:12:36.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:36.916+0000 7efefdc7c640 1 -- 172.21.15.60:0/3304694368 wait complete. 2024-04-25T10:12:36.940 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:12:36.940 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.048+0000 7f2819d58640 1 Processor -- start 2024-04-25T10:12:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.048+0000 7f2819d58640 1 -- start start 2024-04-25T10:12:37.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.048+0000 7f2819d58640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ce7a0 0x7f28140ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.048+0000 7f2819d58640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f281415bc90 0x7f28140b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.048+0000 7f2819d58640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28140cf0c0 con 0x7f28140b4660 2024-04-25T10:12:37.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.048+0000 7f2819d58640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28140cc370 con 0x7f281415bc90 2024-04-25T10:12:37.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.048+0000 7f2819d58640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2814071ca0 con 0x7f28140ce7a0 2024-04-25T10:12:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.048+0000 7f2812ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f281415bc90 0x7f28140b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.048+0000 7f28137fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ce7a0 0x7f28140ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f28137fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ce7a0 0x7f28140ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:34008/0 (socket says 172.21.15.60:34008) 2024-04-25T10:12:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f2812ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f281415bc90 0x7f28140b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:47096/0 (socket says 172.21.15.60:47096) 2024-04-25T10:12:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f28137fe640 1 -- 172.21.15.60:0/2809332341 learned_addr learned my addr 172.21.15.60:0/2809332341 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f2813fff640 1 --1- 172.21.15.60:0/2809332341 >> v1:172.21.15.100:6789/0 conn(0x7f28140b4660 0x7f28140b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:56278/0 (socket says 172.21.15.60:56278) 2024-04-25T10:12:37.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f2812ffd640 1 -- 172.21.15.60:0/2809332341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ce7a0 msgr2=0x7f28140ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f2812ffd640 1 --2- 172.21.15.60:0/2809332341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ce7a0 0x7f28140ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f2812ffd640 1 -- 172.21.15.60:0/2809332341 >> v1:172.21.15.100:6789/0 conn(0x7f28140b4660 legacy=0x7f28140b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f2812ffd640 1 -- 172.21.15.60:0/2809332341 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28140cca40 con 0x7f281415bc90 2024-04-25T10:12:37.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f2812ffd640 1 --2- 172.21.15.60:0/2809332341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281415bc90 0x7f28140b25d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f280800dc10 tx=0x7f28080207b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1945a202f008b971 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f28127fc640 1 -- 172.21.15.60:0/2809332341 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28080b0020 con 0x7f281415bc90 2024-04-25T10:12:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f28127fc640 1 -- 172.21.15.60:0/2809332341 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280800d110 con 0x7f281415bc90 2024-04-25T10:12:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f28127fc640 1 -- 172.21.15.60:0/2809332341 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28080aa6b0 con 0x7f281415bc90 2024-04-25T10:12:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f2819d58640 1 -- 172.21.15.60:0/2809332341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281415bc90 msgr2=0x7f28140b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f2819d58640 1 --2- 172.21.15.60:0/2809332341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281415bc90 0x7f28140b25d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f280800dc10 tx=0x7f28080207b0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f2819d58640 1 -- 172.21.15.60:0/2809332341 shutdown_connections 2024-04-25T10:12:37.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f2819d58640 1 --2- 172.21.15.60:0/2809332341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281415bc90 0x7f28140b25d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f2819d58640 1 --2- 172.21.15.60:0/2809332341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ce7a0 0x7f28140ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.052+0000 7f2819d58640 1 -- 172.21.15.60:0/2809332341 >> 172.21.15.60:0/2809332341 conn(0x7f28140b12d0 msgr2=0x7f28140b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2819d58640 1 -- 172.21.15.60:0/2809332341 shutdown_connections 2024-04-25T10:12:37.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2819d58640 1 -- 172.21.15.60:0/2809332341 wait complete. 2024-04-25T10:12:37.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2819d58640 1 Processor -- start 2024-04-25T10:12:37.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2819d58640 1 -- start start 2024-04-25T10:12:37.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2819d58640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ce7a0 0x7f281414ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2819d58640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f281415bc90 0x7f281414b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2819d58640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281414e160 0x7f281414e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2819d58640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28140b4280 con 0x7f281415bc90 2024-04-25T10:12:37.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2819d58640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28140b3db0 con 0x7f281414e160 2024-04-25T10:12:37.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2819d58640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28140b4400 con 0x7f28140ce7a0 2024-04-25T10:12:37.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2812ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f281415bc90 0x7f281414b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2812ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f281415bc90 0x7f281414b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33990/0 (socket says 172.21.15.60:33990) 2024-04-25T10:12:37.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2812ffd640 1 -- 172.21.15.60:0/2353121195 learned_addr learned my addr 172.21.15.60:0/2353121195 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:37.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f28137fe640 1 --2- 172.21.15.60:0/2353121195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ce7a0 0x7f281414ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2813fff640 1 --2- 172.21.15.60:0/2353121195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281414e160 0x7f281414e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2812ffd640 1 -- 172.21.15.60:0/2353121195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ce7a0 msgr2=0x7f281414ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2812ffd640 1 --2- 172.21.15.60:0/2353121195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ce7a0 0x7f281414ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2812ffd640 1 -- 172.21.15.60:0/2353121195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281414e160 msgr2=0x7f281414e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2812ffd640 1 --2- 172.21.15.60:0/2353121195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281414e160 0x7f281414e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2812ffd640 1 -- 172.21.15.60:0/2353121195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2814159880 con 0x7f281415bc90 2024-04-25T10:12:37.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2812ffd640 1 --2- 172.21.15.60:0/2353121195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f281415bc90 0x7f281414b1d0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f28080af040 tx=0x7f280801d550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f27fbfff640 1 -- 172.21.15.60:0/2353121195 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28080b0040 con 0x7f281415bc90 2024-04-25T10:12:37.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f27fbfff640 1 -- 172.21.15.60:0/2353121195 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280800d2c0 con 0x7f281415bc90 2024-04-25T10:12:37.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2819d58640 1 -- 172.21.15.60:0/2353121195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2814159b60 con 0x7f281415bc90 2024-04-25T10:12:37.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f27fbfff640 1 -- 172.21.15.60:0/2353121195 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808006e20 con 0x7f281415bc90 2024-04-25T10:12:37.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.056+0000 7f2819d58640 1 -- 172.21.15.60:0/2353121195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281415a120 con 0x7f281415bc90 2024-04-25T10:12:37.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.060+0000 7f2819d58640 1 -- 172.21.15.60:0/2353121195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2814074770 con 0x7f281415bc90 2024-04-25T10:12:37.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.060+0000 7f27fbfff640 1 -- 172.21.15.60:0/2353121195 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f28080b3020 con 0x7f281415bc90 2024-04-25T10:12:37.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.060+0000 7f27fbfff640 1 --2- 172.21.15.60:0/2353121195 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2800066f90 0x7f2800069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.060+0000 7f27fbfff640 1 -- 172.21.15.60:0/2353121195 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f28081231a0 con 0x7f281415bc90 2024-04-25T10:12:37.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.060+0000 7f28137fe640 1 --2- 172.21.15.60:0/2353121195 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2800066f90 0x7f2800069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.060+0000 7f28137fe640 1 --2- 172.21.15.60:0/2353121195 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2800066f90 0x7f2800069450 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f28140b0460 tx=0x7f2804009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.064+0000 7f27fbfff640 1 -- 172.21.15.60:0/2353121195 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28080c5060 con 0x7f281415bc90 2024-04-25T10:12:37.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.260+0000 7f2819d58640 1 -- 172.21.15.60:0/2353121195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f281414b710 con 0x7f281415bc90 2024-04-25T10:12:37.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.260+0000 7f27fbfff640 1 -- 172.21.15.60:0/2353121195 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f280801a3a0 con 0x7f281415bc90 2024-04-25T10:12:37.268 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:12:37.268 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:12:09.160298+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7908":{"gid":7908,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":6,"addr":"172.21.15.60:6835/2817879567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2817879567},{"type":"v1","addr":"172.21.15.60:6835","nonce":2817879567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:12:37.268 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T10:12:37.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.260+0000 7f2819d58640 1 -- 172.21.15.60:0/2353121195 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2800066f90 msgr2=0x7f2800069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.260+0000 7f2819d58640 1 --2- 172.21.15.60:0/2353121195 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2800066f90 0x7f2800069450 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f28140b0460 tx=0x7f2804009240 comp rx=0 tx=0).stop 2024-04-25T10:12:37.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.260+0000 7f2819d58640 1 -- 172.21.15.60:0/2353121195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f281415bc90 msgr2=0x7f281414b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.260+0000 7f2819d58640 1 --2- 172.21.15.60:0/2353121195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f281415bc90 0x7f281414b1d0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f28080af040 tx=0x7f280801d550 comp rx=0 tx=0).stop 2024-04-25T10:12:37.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.260+0000 7f2819d58640 1 -- 172.21.15.60:0/2353121195 shutdown_connections 2024-04-25T10:12:37.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.260+0000 7f2819d58640 1 --2- 172.21.15.60:0/2353121195 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2800066f90 0x7f2800069450 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.260+0000 7f2819d58640 1 --2- 172.21.15.60:0/2353121195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f281414e160 0x7f281414e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.260+0000 7f2819d58640 1 --2- 172.21.15.60:0/2353121195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f281415bc90 0x7f281414b1d0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.260+0000 7f2819d58640 1 --2- 172.21.15.60:0/2353121195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ce7a0 0x7f281414ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.260+0000 7f2819d58640 1 -- 172.21.15.60:0/2353121195 >> 172.21.15.60:0/2353121195 conn(0x7f28140b12d0 msgr2=0x7f28140b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.260+0000 7f2819d58640 1 -- 172.21.15.60:0/2353121195 shutdown_connections 2024-04-25T10:12:37.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.260+0000 7f2819d58640 1 -- 172.21.15.60:0/2353121195 wait complete. 2024-04-25T10:12:37.285 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T10:12:37.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.388+0000 7f8685c7f640 1 Processor -- start 2024-04-25T10:12:37.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.388+0000 7f8685c7f640 1 -- start start 2024-04-25T10:12:37.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f8685c7f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f86800cf8a0 0x7f868015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f8685c7f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f86800cd3d0 0x7f868015be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f8685c7f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86800cd890 con 0x7f868015dd30 2024-04-25T10:12:37.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f8685c7f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86800ce4a0 con 0x7f86800cd3d0 2024-04-25T10:12:37.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f8685c7f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86800cdb70 con 0x7f86800cf8a0 2024-04-25T10:12:37.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f867f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f868015dd30 0x7f86800cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41204/0 (socket says 172.21.15.60:41204) 2024-04-25T10:12:37.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f867ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f86800cd3d0 0x7f868015be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f867f7fe640 1 -- 172.21.15.60:0/4059400140 learned_addr learned my addr 172.21.15.60:0/4059400140 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:12:37.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f867effd640 1 --2- 172.21.15.60:0/4059400140 >> v2:172.21.15.100:3300/0 conn(0x7f86800cf8a0 0x7f868015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f867ffff640 1 -- 172.21.15.60:0/4059400140 >> v2:172.21.15.100:3300/0 conn(0x7f86800cf8a0 msgr2=0x7f868015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f867ffff640 1 --2- 172.21.15.60:0/4059400140 >> v2:172.21.15.100:3300/0 conn(0x7f86800cf8a0 0x7f868015ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f867ffff640 1 -- 172.21.15.60:0/4059400140 >> v1:172.21.15.60:6789/0 conn(0x7f868015dd30 legacy=0x7f86800cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f867e7fc640 1 -- 172.21.15.60:0/4059400140 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281798646 0 0) 0x7f86800cd890 con 0x7f868015dd30 2024-04-25T10:12:37.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f867ffff640 1 -- 172.21.15.60:0/4059400140 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86800cde50 con 0x7f86800cd3d0 2024-04-25T10:12:37.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f867ffff640 1 --2- 172.21.15.60:0/4059400140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86800cd3d0 0x7f868015be20 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f8680072550 tx=0x7f867801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a54225569e46c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f867e7fc640 1 -- 172.21.15.60:0/4059400140 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780b2070 con 0x7f86800cd3d0 2024-04-25T10:12:37.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f867e7fc640 1 -- 172.21.15.60:0/4059400140 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86780a5950 con 0x7f86800cd3d0 2024-04-25T10:12:37.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f867e7fc640 1 -- 172.21.15.60:0/4059400140 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780a5d20 con 0x7f86800cd3d0 2024-04-25T10:12:37.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f8685c7f640 1 -- 172.21.15.60:0/4059400140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86800cd3d0 msgr2=0x7f868015be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f8685c7f640 1 --2- 172.21.15.60:0/4059400140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86800cd3d0 0x7f868015be20 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f8680072550 tx=0x7f867801c870 comp rx=0 tx=0).stop 2024-04-25T10:12:37.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f8685c7f640 1 -- 172.21.15.60:0/4059400140 shutdown_connections 2024-04-25T10:12:37.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f8685c7f640 1 --2- 172.21.15.60:0/4059400140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86800cd3d0 0x7f868015be20 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f8685c7f640 1 --2- 172.21.15.60:0/4059400140 >> v2:172.21.15.100:3300/0 conn(0x7f86800cf8a0 0x7f868015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.392+0000 7f8685c7f640 1 -- 172.21.15.60:0/4059400140 >> 172.21.15.60:0/4059400140 conn(0x7f86800670b0 msgr2=0x7f868006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f8685c7f640 1 -- 172.21.15.60:0/4059400140 shutdown_connections 2024-04-25T10:12:37.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f8685c7f640 1 -- 172.21.15.60:0/4059400140 wait complete. 2024-04-25T10:12:37.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f8685c7f640 1 Processor -- start 2024-04-25T10:12:37.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f8685c7f640 1 -- start start 2024-04-25T10:12:37.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f8685c7f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86800cd3d0 0x7f8680142760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f8685c7f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86800cf8a0 0x7f8680164b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f8685c7f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f868015dd30 0x7f8680153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f8685c7f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868015cf00 con 0x7f86800cd3d0 2024-04-25T10:12:37.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f8685c7f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868015cd80 con 0x7f868015dd30 2024-04-25T10:12:37.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f8685c7f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868015d080 con 0x7f86800cf8a0 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f867f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86800cd3d0 0x7f8680142760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f867f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86800cd3d0 0x7f8680142760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34020/0 (socket says 172.21.15.60:34020) 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f867f7fe640 1 -- 172.21.15.60:0/840964157 learned_addr learned my addr 172.21.15.60:0/840964157 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f867f7fe640 1 -- 172.21.15.60:0/840964157 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86800cf8a0 msgr2=0x7f8680164b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f867f7fe640 1 --2- 172.21.15.60:0/840964157 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86800cf8a0 0x7f8680164b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f867f7fe640 1 -- 172.21.15.60:0/840964157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f868015dd30 msgr2=0x7f8680153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f867f7fe640 1 --2- 172.21.15.60:0/840964157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f868015dd30 0x7f8680153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f867f7fe640 1 -- 172.21.15.60:0/840964157 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86801650b0 con 0x7f86800cd3d0 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f867f7fe640 1 --2- 172.21.15.60:0/840964157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86800cd3d0 0x7f8680142760 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f867000c520 tx=0x7f867000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f8677fff640 1 -- 172.21.15.60:0/840964157 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670018070 con 0x7f86800cd3d0 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f8677fff640 1 -- 172.21.15.60:0/840964157 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86700074e0 con 0x7f86800cd3d0 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f8685c7f640 1 -- 172.21.15.60:0/840964157 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8680154250 con 0x7f86800cd3d0 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f8677fff640 1 -- 172.21.15.60:0/840964157 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86700049f0 con 0x7f86800cd3d0 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.396+0000 7f8685c7f640 1 -- 172.21.15.60:0/840964157 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8680154d50 con 0x7f86800cd3d0 2024-04-25T10:12:37.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8677fff640 1 -- 172.21.15.60:0/840964157 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f867001b070 con 0x7f86800cd3d0 2024-04-25T10:12:37.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8677fff640 1 --2- 172.21.15.60:0/840964157 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8660066f90 0x7f8660069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8677fff640 1 -- 172.21.15.60:0/840964157 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f8670084450 con 0x7f86800cd3d0 2024-04-25T10:12:37.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f867effd640 1 --2- 172.21.15.60:0/840964157 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8660066f90 0x7f8660069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 Processor -- start 2024-04-25T10:12:37.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 -- start start 2024-04-25T10:12:37.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f867effd640 1 --2- 172.21.15.60:0/840964157 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8660066f90 0x7f8660069450 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f866800a8b0 tx=0x7f8668008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887b22e0 0x560a887b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887b2c00 0x560a887683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887688e0 0x560a88768cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a8864fba0 con 0x560a887b2c00 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a885b5090 con 0x560a887688e0 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a8853d160 con 0x560a887b22e0 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f867effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887b2c00 0x560a887683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f867effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887b2c00 0x560a887683a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34034/0 (socket says 172.21.15.60:34034) 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f867effd640 1 -- 172.21.15.60:0/3798131790 learned_addr learned my addr 172.21.15.60:0/3798131790 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f867ffff640 1 --2- 172.21.15.60:0/3798131790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887688e0 0x560a88768cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f867f7fe640 1 --2- 172.21.15.60:0/3798131790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887b22e0 0x560a887b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f867effd640 1 -- 172.21.15.60:0/3798131790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887b22e0 msgr2=0x560a887b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f867effd640 1 --2- 172.21.15.60:0/3798131790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887b22e0 0x560a887b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f867effd640 1 -- 172.21.15.60:0/3798131790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887688e0 msgr2=0x560a88768cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f867effd640 1 --2- 172.21.15.60:0/3798131790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887688e0 0x560a88768cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f867effd640 1 -- 172.21.15.60:0/3798131790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a8865ec20 con 0x560a887b2c00 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f867ffff640 1 --2- 172.21.15.60:0/3798131790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887688e0 0x560a88768cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f867effd640 1 --2- 172.21.15.60:0/3798131790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887b2c00 0x560a887683a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f866800cab0 tx=0x7f866800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c85a5402232250e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f86757fa640 1 -- 172.21.15.60:0/3798131790 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668023070 con 0x560a887b2c00 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f86757fa640 1 -- 172.21.15.60:0/3798131790 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866801e400 con 0x560a887b2c00 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f86757fa640 1 -- 172.21.15.60:0/3798131790 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866801e7b0 con 0x560a887b2c00 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 -- 172.21.15.60:0/3798131790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887b2c00 msgr2=0x560a887683a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 --2- 172.21.15.60:0/3798131790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887b2c00 0x560a887683a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f866800cab0 tx=0x7f866800f000 comp rx=0 tx=0).stop 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 -- 172.21.15.60:0/3798131790 shutdown_connections 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 --2- 172.21.15.60:0/3798131790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887688e0 0x560a88768cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 --2- 172.21.15.60:0/3798131790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887b2c00 0x560a887683a0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 --2- 172.21.15.60:0/3798131790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887b22e0 0x560a887b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 -- 172.21.15.60:0/3798131790 >> 172.21.15.60:0/3798131790 conn(0x560a88764890 msgr2=0x560a88766c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 -- 172.21.15.60:0/3798131790 shutdown_connections 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 -- 172.21.15.60:0/3798131790 wait complete. 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.400+0000 7f8688dec000 1 -- start start 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 Processor -- start 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887688e0 0x560a887b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887b22e0 0x560a887b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887b2c00 0x560a887a0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a885b1130 con 0x560a887b22e0 2024-04-25T10:12:37.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a88579130 con 0x560a887b2c00 2024-04-25T10:12:37.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a88673300 con 0x560a887688e0 2024-04-25T10:12:37.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 10 client.0 _ll_register_callbacks cb 0x560a88753bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:37.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f867effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887b22e0 0x560a887b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f867effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887b22e0 0x560a887b9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34046/0 (socket says 172.21.15.60:34046) 2024-04-25T10:12:37.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f867effd640 1 -- 172.21.15.60:0/2049919828 learned_addr learned my addr 172.21.15.60:0/2049919828 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:37.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f867f7fe640 1 --2- 172.21.15.60:0/2049919828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887688e0 0x560a887b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f867ffff640 1 --2- 172.21.15.60:0/2049919828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887b2c00 0x560a887a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f867effd640 1 -- 172.21.15.60:0/2049919828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887688e0 msgr2=0x560a887b9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f867effd640 1 --2- 172.21.15.60:0/2049919828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887688e0 0x560a887b9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f867effd640 1 -- 172.21.15.60:0/2049919828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887b2c00 msgr2=0x560a887a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f867effd640 1 --2- 172.21.15.60:0/2049919828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887b2c00 0x560a887a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f867effd640 1 -- 172.21.15.60:0/2049919828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a88592060 con 0x560a887b22e0 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f867f7fe640 1 --2- 172.21.15.60:0/2049919828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887688e0 0x560a887b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f867effd640 1 --2- 172.21.15.60:0/2049919828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887b22e0 0x560a887b9b80 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f8668021040 tx=0x7f8668013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a14846009abc027 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f865e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f865e7fc640 1 -- 172.21.15.60:0/2049919828 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866802e020 con 0x560a887b22e0 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 -- 172.21.15.60:0/2049919828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f866801e400 con 0x560a887b22e0 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f865e7fc640 1 -- 172.21.15.60:0/2049919828 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f866801ed80 con 0x560a887b22e0 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f865e7fc640 1 -- 172.21.15.60:0/2049919828 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8668038050 con 0x560a887b22e0 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 10 client.8067 fetch_fsmap learned FSMap version 62 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 -- 172.21.15.60:0/2049919828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x560a885567c0 con 0x560a887b22e0 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f865e7fc640 1 -- 172.21.15.60:0/2049919828 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866801e400 con 0x560a887b22e0 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f865e7fc640 1 -- 172.21.15.60:0/2049919828 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8668029470 con 0x560a887b22e0 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 10 client.8067 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 10 client.8067 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 20 client.8067 populate_metadata read hostname 'smithi060' 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 --2- 172.21.15.60:0/2049919828 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x560a887a0e00 0x560a887a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 4 client.8067 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 -- 172.21.15.60:0/2049919828 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a88550cd0 con 0x560a887a0e00 2024-04-25T10:12:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f867f7fe640 1 --2- 172.21.15.60:0/2049919828 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x560a887a0e00 0x560a887a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f867f7fe640 1 --2- 172.21.15.60:0/2049919828 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x560a887a0e00 0x560a887a11e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e217b911813bc5 server_cookie=587a35c81cda452a in_seq=0 out_seq=0 2024-04-25T10:12:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f865e7fc640 10 client.8067 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f865e7fc640 1 -- 172.21.15.60:0/2049919828 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a88550cd0 con 0x560a887a0e00 2024-04-25T10:12:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f865e7fc640 10 client.8067 handle_command_reply: tid=0 2024-04-25T10:12:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 client.8067 shutdown 2024-04-25T10:12:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 -- 172.21.15.60:0/2049919828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887b22e0 msgr2=0x560a887b9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 --2- 172.21.15.60:0/2049919828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887b22e0 0x560a887b9b80 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f8668021040 tx=0x7f8668013000 comp rx=0 tx=0).stop 2024-04-25T10:12:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 -- 172.21.15.60:0/2049919828 shutdown_connections 2024-04-25T10:12:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 --2- 172.21.15.60:0/2049919828 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x560a887a0e00 0x560a887a11e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 --2- 172.21.15.60:0/2049919828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887b2c00 0x560a887a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 --2- 172.21.15.60:0/2049919828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887b22e0 0x560a887b9b80 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 --2- 172.21.15.60:0/2049919828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887688e0 0x560a887b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 -- 172.21.15.60:0/2049919828 >> 172.21.15.60:0/2049919828 conn(0x560a88764890 msgr2=0x560a88767510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f865e7fc640 0 client.8067 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:37.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 -- 172.21.15.60:0/2049919828 shutdown_connections 2024-04-25T10:12:37.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 1 -- 172.21.15.60:0/2049919828 wait complete. 2024-04-25T10:12:37.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.404+0000 7f8688dec000 20 client.8067 trim_cache size 0 max 16384 2024-04-25T10:12:37.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.416+0000 7f8688dec000 1 Processor -- start 2024-04-25T10:12:37.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.416+0000 7f8688dec000 1 -- start start 2024-04-25T10:12:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.416+0000 7f8688dec000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887688e0 0x560a887682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.416+0000 7f8688dec000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887b22e0 0x560a88766c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.416+0000 7f8688dec000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887b2c00 0x560a887671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.416+0000 7f8688dec000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8680201cd0 con 0x560a887688e0 2024-04-25T10:12:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.416+0000 7f8688dec000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86800b2460 con 0x560a887b22e0 2024-04-25T10:12:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.416+0000 7f8688dec000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86800b2a60 con 0x560a887b2c00 2024-04-25T10:12:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887688e0 0x560a887682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887688e0 0x560a887682c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34060/0 (socket says 172.21.15.60:34060) 2024-04-25T10:12:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 -- 172.21.15.60:0/3288769397 learned_addr learned my addr 172.21.15.60:0/3288769397 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 -- 172.21.15.60:0/3288769397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887b2c00 msgr2=0x560a887671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 --2- 172.21.15.60:0/3288769397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887b2c00 0x560a887671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 -- 172.21.15.60:0/3288769397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887b22e0 msgr2=0x560a88766c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 --2- 172.21.15.60:0/3288769397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887b22e0 0x560a88766c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 -- 172.21.15.60:0/3288769397 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a88570fc0 con 0x560a887688e0 2024-04-25T10:12:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 --2- 172.21.15.60:0/3288769397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887688e0 0x560a887682c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f867005cf60 tx=0x7f8670082870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67b59bdde96f34b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8674ff9640 1 -- 172.21.15.60:0/3288769397 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670053080 con 0x560a887688e0 2024-04-25T10:12:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8674ff9640 1 -- 172.21.15.60:0/3288769397 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867005b0c0 con 0x560a887688e0 2024-04-25T10:12:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8674ff9640 1 -- 172.21.15.60:0/3288769397 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86700840b0 con 0x560a887688e0 2024-04-25T10:12:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 -- 172.21.15.60:0/3288769397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887688e0 msgr2=0x560a887682c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 --2- 172.21.15.60:0/3288769397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887688e0 0x560a887682c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f867005cf60 tx=0x7f8670082870 comp rx=0 tx=0).stop 2024-04-25T10:12:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 -- 172.21.15.60:0/3288769397 shutdown_connections 2024-04-25T10:12:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 --2- 172.21.15.60:0/3288769397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887b2c00 0x560a887671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 --2- 172.21.15.60:0/3288769397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887b22e0 0x560a88766c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 --2- 172.21.15.60:0/3288769397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887688e0 0x560a887682c0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 -- 172.21.15.60:0/3288769397 >> 172.21.15.60:0/3288769397 conn(0x560a88764890 msgr2=0x560a887bfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 -- 172.21.15.60:0/3288769397 shutdown_connections 2024-04-25T10:12:37.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 -- 172.21.15.60:0/3288769397 wait complete. 2024-04-25T10:12:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 -- start start 2024-04-25T10:12:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 Processor -- start 2024-04-25T10:12:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887688e0 0x560a88811f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887b22e0 0x560a88812490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887b2c00 0x560a887b6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a8854fe70 con 0x560a887688e0 2024-04-25T10:12:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a8853da50 con 0x560a887b2c00 2024-04-25T10:12:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a885b43a0 con 0x560a887b22e0 2024-04-25T10:12:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 10 client.0 _ll_register_callbacks cb 0x560a88561240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887688e0 0x560a88811f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887688e0 0x560a88811f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34068/0 (socket says 172.21.15.60:34068) 2024-04-25T10:12:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 -- 172.21.15.60:0/2434960955 learned_addr learned my addr 172.21.15.60:0/2434960955 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 -- 172.21.15.60:0/2434960955 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887b22e0 msgr2=0x560a88812490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 --2- 172.21.15.60:0/2434960955 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887b22e0 0x560a88812490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 -- 172.21.15.60:0/2434960955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887b2c00 msgr2=0x560a887b6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 --2- 172.21.15.60:0/2434960955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887b2c00 0x560a887b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 -- 172.21.15.60:0/2434960955 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a8860e9e0 con 0x560a887688e0 2024-04-25T10:12:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867f7fe640 1 --2- 172.21.15.60:0/2434960955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887688e0 0x560a88811f50 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f8670054d10 tx=0x7f8670084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295988f944f9b0b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f865e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:12:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f865e7fc640 1 -- 172.21.15.60:0/2434960955 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86700548c0 con 0x560a887688e0 2024-04-25T10:12:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f865e7fc640 1 -- 172.21.15.60:0/2434960955 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f86700513f0 con 0x560a887688e0 2024-04-25T10:12:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 -- 172.21.15.60:0/2434960955 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f867005b0c0 con 0x560a887688e0 2024-04-25T10:12:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f865e7fc640 1 -- 172.21.15.60:0/2434960955 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f867007c0d0 con 0x560a887688e0 2024-04-25T10:12:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 10 client.8073 fetch_fsmap learned FSMap version 62 2024-04-25T10:12:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 -- 172.21.15.60:0/2434960955 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x560a88554630 con 0x560a887688e0 2024-04-25T10:12:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f865e7fc640 1 -- 172.21.15.60:0/2434960955 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867005b0c0 con 0x560a887688e0 2024-04-25T10:12:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f865e7fc640 1 -- 172.21.15.60:0/2434960955 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f86700840b0 con 0x560a887688e0 2024-04-25T10:12:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 10 client.8073 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:12:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 10 client.8073 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:12:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 20 client.8073 populate_metadata read hostname 'smithi060' 2024-04-25T10:12:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 --2- 172.21.15.60:0/2434960955 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x560a887b6970 0x560a887b6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 4 client.8073 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:12:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f8688dec000 1 -- 172.21.15.60:0/2434960955 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x560a885c25d0 con 0x560a887b6970 2024-04-25T10:12:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.420+0000 7f867effd640 1 --2- 172.21.15.60:0/2434960955 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x560a887b6970 0x560a887b6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f865e7fc640 10 client.8073 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f867effd640 1 --2- 172.21.15.60:0/2434960955 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x560a887b6970 0x560a887b6d50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9055b9bd9706c22 server_cookie=91fbeec09e01e66e in_seq=0 out_seq=0 2024-04-25T10:12:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f865e7fc640 1 -- 172.21.15.60:0/2434960955 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+418 (crc 0 0 0) 0x560a885c25d0 con 0x560a887b6970 2024-04-25T10:12:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f865e7fc640 10 client.8073 handle_command_reply: tid=0 2024-04-25T10:12:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8688dec000 1 client.8073 shutdown 2024-04-25T10:12:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8688dec000 1 -- 172.21.15.60:0/2434960955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887688e0 msgr2=0x560a88811f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8688dec000 1 --2- 172.21.15.60:0/2434960955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887688e0 0x560a88811f50 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f8670054d10 tx=0x7f8670084040 comp rx=0 tx=0).stop 2024-04-25T10:12:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8688dec000 1 -- 172.21.15.60:0/2434960955 shutdown_connections 2024-04-25T10:12:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8688dec000 1 --2- 172.21.15.60:0/2434960955 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x560a887b6970 0x560a887b6d50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8688dec000 1 --2- 172.21.15.60:0/2434960955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a887b2c00 0x560a887b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8688dec000 1 --2- 172.21.15.60:0/2434960955 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560a887b22e0 0x560a88812490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f865e7fc640 0 client.8073 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8688dec000 1 --2- 172.21.15.60:0/2434960955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560a887688e0 0x560a88811f50 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8688dec000 1 -- 172.21.15.60:0/2434960955 >> 172.21.15.60:0/2434960955 conn(0x560a88764890 msgr2=0x560a887bfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8688dec000 1 -- 172.21.15.60:0/2434960955 shutdown_connections 2024-04-25T10:12:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8688dec000 1 -- 172.21.15.60:0/2434960955 wait complete. 2024-04-25T10:12:37.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8688dec000 20 client.8073 trim_cache size 0 max 16384 2024-04-25T10:12:37.436 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:12:37.437 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T10:12:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:12:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "dir_frag", 2024-04-25T10:12:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 215513721, 2024-04-25T10:12:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511627776, 2024-04-25T10:12:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "frag": "*", 2024-04-25T10:12:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/dir" 2024-04-25T10:12:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:12:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:12:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-04-25T10:12:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 454532660, 2024-04-25T10:12:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511627776, 2024-04-25T10:12:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/dir" 2024-04-25T10:12:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:12:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:12:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-04-25T10:12:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2204635862, 2024-04-25T10:12:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511627777, 2024-04-25T10:12:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/testdir/hardlink" 2024-04-25T10:12:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:12:37.438 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T10:12:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8685c7f640 1 -- 172.21.15.60:0/840964157 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8660066f90 msgr2=0x7f8660069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8685c7f640 1 --2- 172.21.15.60:0/840964157 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8660066f90 0x7f8660069450 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f866800a8b0 tx=0x7f8668008040 comp rx=0 tx=0).stop 2024-04-25T10:12:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8685c7f640 1 -- 172.21.15.60:0/840964157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86800cd3d0 msgr2=0x7f8680142760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8685c7f640 1 --2- 172.21.15.60:0/840964157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86800cd3d0 0x7f8680142760 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f867000c520 tx=0x7f867000c9e0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8685c7f640 1 -- 172.21.15.60:0/840964157 shutdown_connections 2024-04-25T10:12:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8685c7f640 1 --2- 172.21.15.60:0/840964157 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8660066f90 0x7f8660069450 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8685c7f640 1 --2- 172.21.15.60:0/840964157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f868015dd30 0x7f8680153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8685c7f640 1 --2- 172.21.15.60:0/840964157 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86800cf8a0 0x7f8680164b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8685c7f640 1 --2- 172.21.15.60:0/840964157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86800cd3d0 0x7f8680142760 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8685c7f640 1 -- 172.21.15.60:0/840964157 >> 172.21.15.60:0/840964157 conn(0x7f86800670b0 msgr2=0x7f86800ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8685c7f640 1 -- 172.21.15.60:0/840964157 shutdown_connections 2024-04-25T10:12:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.424+0000 7f8685c7f640 1 -- 172.21.15.60:0/840964157 wait complete. 2024-04-25T10:12:37.447 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:12:37.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.548+0000 7f266b854640 1 Processor -- start 2024-04-25T10:12:37.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.548+0000 7f266b854640 1 -- start start 2024-04-25T10:12:37.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f266b854640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f266406c330 0x7f266406fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f266b854640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f26640cc010 0x7f26640b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f266b854640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26640af460 con 0x7f266415b8b0 2024-04-25T10:12:37.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f266b854640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266415bc90 con 0x7f266406c330 2024-04-25T10:12:37.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f266b854640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26640b2f30 con 0x7f26640cc010 2024-04-25T10:12:37.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f26695f0640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f266415b8b0 0x7f266406bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41208/0 (socket says 172.21.15.60:41208) 2024-04-25T10:12:37.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f26695f0640 1 -- 172.21.15.60:0/730583597 learned_addr learned my addr 172.21.15.60:0/730583597 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:12:37.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f2668def640 1 --2- 172.21.15.60:0/730583597 >> v2:172.21.15.60:3300/0 conn(0x7f266406c330 0x7f266406fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f2669df1640 1 --2- 172.21.15.60:0/730583597 >> v2:172.21.15.100:3300/0 conn(0x7f26640cc010 0x7f26640b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f2668def640 1 -- 172.21.15.60:0/730583597 >> v2:172.21.15.100:3300/0 conn(0x7f26640cc010 msgr2=0x7f26640b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f2668def640 1 --2- 172.21.15.60:0/730583597 >> v2:172.21.15.100:3300/0 conn(0x7f26640cc010 0x7f26640b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f2668def640 1 -- 172.21.15.60:0/730583597 >> v1:172.21.15.60:6789/0 conn(0x7f266415b8b0 legacy=0x7f266406bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f2668def640 1 -- 172.21.15.60:0/730583597 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26640b3210 con 0x7f266406c330 2024-04-25T10:12:37.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f265bfff640 1 -- 172.21.15.60:0/730583597 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014897589 0 0) 0x7f26640af460 con 0x7f266415b8b0 2024-04-25T10:12:37.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f2668def640 1 --2- 172.21.15.60:0/730583597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f266406c330 0x7f266406fed0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f2664072b10 tx=0x7f265401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b1dfaa88e037f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f265bfff640 1 -- 172.21.15.60:0/730583597 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540b0070 con 0x7f266406c330 2024-04-25T10:12:37.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f265bfff640 1 -- 172.21.15.60:0/730583597 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26540ab600 con 0x7f266406c330 2024-04-25T10:12:37.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.552+0000 7f265bfff640 1 -- 172.21.15.60:0/730583597 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540ab9d0 con 0x7f266406c330 2024-04-25T10:12:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 -- 172.21.15.60:0/730583597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f266406c330 msgr2=0x7f266406fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 --2- 172.21.15.60:0/730583597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f266406c330 0x7f266406fed0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f2664072b10 tx=0x7f265401c480 comp rx=0 tx=0).stop 2024-04-25T10:12:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 -- 172.21.15.60:0/730583597 shutdown_connections 2024-04-25T10:12:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 --2- 172.21.15.60:0/730583597 >> v2:172.21.15.100:3300/0 conn(0x7f26640cc010 0x7f26640b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 --2- 172.21.15.60:0/730583597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f266406c330 0x7f266406fed0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 -- 172.21.15.60:0/730583597 >> 172.21.15.60:0/730583597 conn(0x7f2664071b30 msgr2=0x7f2664071f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 -- 172.21.15.60:0/730583597 shutdown_connections 2024-04-25T10:12:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 -- 172.21.15.60:0/730583597 wait complete. 2024-04-25T10:12:37.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 Processor -- start 2024-04-25T10:12:37.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 -- start start 2024-04-25T10:12:37.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f266406c330 0x7f2664157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26640cc010 0x7f26640acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f266415b8b0 0x7f266413d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26640b4b30 con 0x7f266406c330 2024-04-25T10:12:37.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26640b49b0 con 0x7f26640cc010 2024-04-25T10:12:37.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26640b4cb0 con 0x7f266415b8b0 2024-04-25T10:12:37.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f26695f0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f266406c330 0x7f2664157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f26695f0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f266406c330 0x7f2664157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34082/0 (socket says 172.21.15.60:34082) 2024-04-25T10:12:37.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f26695f0640 1 -- 172.21.15.60:0/909598276 learned_addr learned my addr 172.21.15.60:0/909598276 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:37.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f2668def640 1 --2- 172.21.15.60:0/909598276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26640cc010 0x7f26640acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f2669df1640 1 --2- 172.21.15.60:0/909598276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f266415b8b0 0x7f266413d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f26695f0640 1 -- 172.21.15.60:0/909598276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f266415b8b0 msgr2=0x7f266413d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f26695f0640 1 --2- 172.21.15.60:0/909598276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f266415b8b0 0x7f266413d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f26695f0640 1 -- 172.21.15.60:0/909598276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26640cc010 msgr2=0x7f26640acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f26695f0640 1 --2- 172.21.15.60:0/909598276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26640cc010 0x7f26640acd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f26695f0640 1 -- 172.21.15.60:0/909598276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26640ad2b0 con 0x7f266406c330 2024-04-25T10:12:37.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f26695f0640 1 --2- 172.21.15.60:0/909598276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f266406c330 0x7f2664157630 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f266000c890 tx=0x7f266000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f2659ffb640 1 -- 172.21.15.60:0/909598276 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660018070 con 0x7f266406c330 2024-04-25T10:12:37.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f2659ffb640 1 -- 172.21.15.60:0/909598276 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266000a640 con 0x7f266406c330 2024-04-25T10:12:37.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 -- 172.21.15.60:0/909598276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26640ad550 con 0x7f266406c330 2024-04-25T10:12:37.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f2659ffb640 1 -- 172.21.15.60:0/909598276 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266000aa30 con 0x7f266406c330 2024-04-25T10:12:37.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.556+0000 7f266b854640 1 -- 172.21.15.60:0/909598276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26640adb10 con 0x7f266406c330 2024-04-25T10:12:37.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.560+0000 7f266b854640 1 -- 172.21.15.60:0/909598276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2664074770 con 0x7f266406c330 2024-04-25T10:12:37.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.560+0000 7f2659ffb640 1 -- 172.21.15.60:0/909598276 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f266001b070 con 0x7f266406c330 2024-04-25T10:12:37.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.560+0000 7f2659ffb640 1 --2- 172.21.15.60:0/909598276 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2648066f90 0x7f2648069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.560+0000 7f2659ffb640 1 -- 172.21.15.60:0/909598276 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f2660084710 con 0x7f266406c330 2024-04-25T10:12:37.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.560+0000 7f2668def640 1 --2- 172.21.15.60:0/909598276 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2648066f90 0x7f2648069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.560+0000 7f2668def640 1 --2- 172.21.15.60:0/909598276 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2648066f90 0x7f2648069450 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f26540ad000 tx=0x7f2654008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.564+0000 7f2659ffb640 1 -- 172.21.15.60:0/909598276 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f26600556b0 con 0x7f266406c330 2024-04-25T10:12:37.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.756+0000 7f266b854640 1 -- 172.21.15.60:0/909598276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f266409dd10 con 0x7f266406c330 2024-04-25T10:12:37.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.756+0000 7f2659ffb640 1 -- 172.21.15.60:0/909598276 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f2660054fb0 con 0x7f266406c330 2024-04-25T10:12:37.764 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:12:37.764 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:12:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.756+0000 7f266b854640 1 -- 172.21.15.60:0/909598276 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2648066f90 msgr2=0x7f2648069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.756+0000 7f266b854640 1 --2- 172.21.15.60:0/909598276 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2648066f90 0x7f2648069450 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f26540ad000 tx=0x7f2654008040 comp rx=0 tx=0).stop 2024-04-25T10:12:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.756+0000 7f266b854640 1 -- 172.21.15.60:0/909598276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f266406c330 msgr2=0x7f2664157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.756+0000 7f266b854640 1 --2- 172.21.15.60:0/909598276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f266406c330 0x7f2664157630 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f266000c890 tx=0x7f266000cd50 comp rx=0 tx=0).stop 2024-04-25T10:12:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.760+0000 7f266b854640 1 -- 172.21.15.60:0/909598276 shutdown_connections 2024-04-25T10:12:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.760+0000 7f266b854640 1 --2- 172.21.15.60:0/909598276 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2648066f90 0x7f2648069450 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.760+0000 7f266b854640 1 --2- 172.21.15.60:0/909598276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f266415b8b0 0x7f266413d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.760+0000 7f266b854640 1 --2- 172.21.15.60:0/909598276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26640cc010 0x7f26640acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.760+0000 7f266b854640 1 --2- 172.21.15.60:0/909598276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f266406c330 0x7f2664157630 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.760+0000 7f266b854640 1 -- 172.21.15.60:0/909598276 >> 172.21.15.60:0/909598276 conn(0x7f2664071b30 msgr2=0x7f266414bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.760+0000 7f266b854640 1 -- 172.21.15.60:0/909598276 shutdown_connections 2024-04-25T10:12:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.760+0000 7f266b854640 1 -- 172.21.15.60:0/909598276 wait complete. 2024-04-25T10:12:37.782 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:12:37.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:37.782 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-25T10:12:37.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.900+0000 7f4614dbb640 1 Processor -- start 2024-04-25T10:12:37.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.900+0000 7f4614dbb640 1 -- start start 2024-04-25T10:12:37.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.900+0000 7f4614dbb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4610071980 0x7f46100ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.900+0000 7f4614dbb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46100cf810 con 0x7f4610144e40 2024-04-25T10:12:37.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.900+0000 7f4614dbb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46100cfaf0 con 0x7f46100cf0a0 2024-04-25T10:12:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.900+0000 7f4614dbb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46100b3060 con 0x7f4610071980 2024-04-25T10:12:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.900+0000 7f460e59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4610144e40 0x7f4610071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41210/0 (socket says 172.21.15.60:41210) 2024-04-25T10:12:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.900+0000 7f460e59c640 1 -- 172.21.15.60:0/1372072985 learned_addr learned my addr 172.21.15.60:0/1372072985 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:12:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.900+0000 7f460dd9b640 1 --2- 172.21.15.60:0/1372072985 >> v2:172.21.15.60:3300/0 conn(0x7f4610071980 0x7f46100ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.900+0000 7f460d59a640 1 -- 172.21.15.60:0/1372072985 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602690668 0 0) 0x7f46100cf810 con 0x7f4610144e40 2024-04-25T10:12:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.904+0000 7f460d59a640 1 -- 172.21.15.60:0/1372072985 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45fc003530 con 0x7f4610144e40 2024-04-25T10:12:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.904+0000 7f460dd9b640 1 -- 172.21.15.60:0/1372072985 >> v1:172.21.15.100:6789/0 conn(0x7f46100cf0a0 legacy=0x7f46100b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.904+0000 7f460dd9b640 1 -- 172.21.15.60:0/1372072985 >> v1:172.21.15.60:6789/0 conn(0x7f4610144e40 legacy=0x7f4610071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.904+0000 7f460dd9b640 1 -- 172.21.15.60:0/1372072985 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46100cfdd0 con 0x7f4610071980 2024-04-25T10:12:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.904+0000 7f460dd9b640 1 --2- 172.21.15.60:0/1372072985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4610071980 0x7f46100ccc60 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f46100b31e0 tx=0x7f45f801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86994153d4bcf23e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.904+0000 7f460d59a640 1 -- 172.21.15.60:0/1372072985 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80aa620 con 0x7f4610071980 2024-04-25T10:12:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.904+0000 7f460d59a640 1 -- 172.21.15.60:0/1372072985 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f80af070 con 0x7f4610071980 2024-04-25T10:12:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.904+0000 7f460d59a640 1 -- 172.21.15.60:0/1372072985 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80aa9e0 con 0x7f4610071980 2024-04-25T10:12:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.904+0000 7f4614dbb640 1 -- 172.21.15.60:0/1372072985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4610071980 msgr2=0x7f46100ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.904+0000 7f4614dbb640 1 --2- 172.21.15.60:0/1372072985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4610071980 0x7f46100ccc60 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f46100b31e0 tx=0x7f45f801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.904+0000 7f4614dbb640 1 -- 172.21.15.60:0/1372072985 shutdown_connections 2024-04-25T10:12:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.904+0000 7f4614dbb640 1 --2- 172.21.15.60:0/1372072985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4610071980 0x7f46100ccc60 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.904+0000 7f4614dbb640 1 -- 172.21.15.60:0/1372072985 >> 172.21.15.60:0/1372072985 conn(0x7f4610066fa0 msgr2=0x7f461006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.904+0000 7f4614dbb640 1 -- 172.21.15.60:0/1372072985 shutdown_connections 2024-04-25T10:12:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.904+0000 7f4614dbb640 1 -- 172.21.15.60:0/1372072985 wait complete. 2024-04-25T10:12:37.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f4614dbb640 1 Processor -- start 2024-04-25T10:12:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f4614dbb640 1 -- start start 2024-04-25T10:12:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f4614dbb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4610071980 0x7f461015b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f4614dbb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46100cf0a0 0x7f46100a8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f4614dbb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4610144e40 0x7f461015e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f4614dbb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46100b4820 con 0x7f46100cf0a0 2024-04-25T10:12:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f4614dbb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46100b46a0 con 0x7f4610071980 2024-04-25T10:12:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f4614dbb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46101696f0 con 0x7f4610144e40 2024-04-25T10:12:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f460dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46100cf0a0 0x7f46100a8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f460dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46100cf0a0 0x7f46100a8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34100/0 (socket says 172.21.15.60:34100) 2024-04-25T10:12:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f460dd9b640 1 -- 172.21.15.60:0/3375728677 learned_addr learned my addr 172.21.15.60:0/3375728677 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f460e59c640 1 --2- 172.21.15.60:0/3375728677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4610071980 0x7f461015b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f460ed9d640 1 --2- 172.21.15.60:0/3375728677 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4610144e40 0x7f461015e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f460dd9b640 1 -- 172.21.15.60:0/3375728677 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4610144e40 msgr2=0x7f461015e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f460dd9b640 1 --2- 172.21.15.60:0/3375728677 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4610144e40 0x7f461015e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f460dd9b640 1 -- 172.21.15.60:0/3375728677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4610071980 msgr2=0x7f461015b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f460dd9b640 1 --2- 172.21.15.60:0/3375728677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4610071980 0x7f461015b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f460dd9b640 1 -- 172.21.15.60:0/3375728677 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46100a8ce0 con 0x7f46100cf0a0 2024-04-25T10:12:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f460dd9b640 1 --2- 172.21.15.60:0/3375728677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46100cf0a0 0x7f46100a8490 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f45f80ac040 tx=0x7f45f8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f4606ffd640 1 -- 172.21.15.60:0/3375728677 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80aac40 con 0x7f46100cf0a0 2024-04-25T10:12:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f4606ffd640 1 -- 172.21.15.60:0/3375728677 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4610145430 con 0x7f46100cf0a0 2024-04-25T10:12:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f4614dbb640 1 -- 172.21.15.60:0/3375728677 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46100a8fc0 con 0x7f46100cf0a0 2024-04-25T10:12:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f4606ffd640 1 -- 172.21.15.60:0/3375728677 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80be410 con 0x7f46100cf0a0 2024-04-25T10:12:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.908+0000 7f4614dbb640 1 -- 172.21.15.60:0/3375728677 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46100a89d0 con 0x7f46100cf0a0 2024-04-25T10:12:37.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4606ffd640 1 -- 172.21.15.60:0/3375728677 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45f80bc020 con 0x7f46100cf0a0 2024-04-25T10:12:37.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4606ffd640 1 --2- 172.21.15.60:0/3375728677 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f45f0066f90 0x7f45f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4606ffd640 1 -- 172.21.15.60:0/3375728677 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f45f811f0d0 con 0x7f46100cf0a0 2024-04-25T10:12:37.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f460e59c640 1 --2- 172.21.15.60:0/3375728677 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f45f0066f90 0x7f45f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 Processor -- start 2024-04-25T10:12:37.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 -- start start 2024-04-25T10:12:37.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f460e59c640 1 --2- 172.21.15.60:0/3375728677 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f45f0066f90 0x7f45f0069450 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4600001570 tx=0x7f4600009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bd5a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd5aa10 0x558d3bd10160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd106a0 0x558d3bd10a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3bbf2f40 con 0x558d3bd5a0f0 2024-04-25T10:12:37.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3bb5a070 con 0x558d3bd5aa10 2024-04-25T10:12:37.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d3bae21a0 con 0x558d3bd106a0 2024-04-25T10:12:37.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f460e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bd5a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f460e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bd5a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34116/0 (socket says 172.21.15.60:34116) 2024-04-25T10:12:37.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f460e59c640 1 -- 172.21.15.60:0/4038894029 learned_addr learned my addr 172.21.15.60:0/4038894029 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:37.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f460ed9d640 1 --2- 172.21.15.60:0/4038894029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd106a0 0x558d3bd10a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f460dd9b640 1 --2- 172.21.15.60:0/4038894029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd5aa10 0x558d3bd10160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f460e59c640 1 -- 172.21.15.60:0/4038894029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd106a0 msgr2=0x558d3bd10a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f460e59c640 1 --2- 172.21.15.60:0/4038894029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd106a0 0x558d3bd10a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f460e59c640 1 -- 172.21.15.60:0/4038894029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd5aa10 msgr2=0x558d3bd10160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f460e59c640 1 --2- 172.21.15.60:0/4038894029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd5aa10 0x558d3bd10160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f460e59c640 1 -- 172.21.15.60:0/4038894029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d3bae0510 con 0x558d3bd5a0f0 2024-04-25T10:12:37.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f460e59c640 1 --2- 172.21.15.60:0/4038894029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bd5a4d0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f4600002240 tx=0x7f4600011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9124eca8d9c94447 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f45effff640 1 -- 172.21.15.60:0/4038894029 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460000ee60 con 0x558d3bd5a0f0 2024-04-25T10:12:37.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f45effff640 1 -- 172.21.15.60:0/4038894029 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4600013630 con 0x558d3bd5a0f0 2024-04-25T10:12:37.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f45effff640 1 -- 172.21.15.60:0/4038894029 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600013a00 con 0x558d3bd5a0f0 2024-04-25T10:12:37.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 -- 172.21.15.60:0/4038894029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 msgr2=0x558d3bd5a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 --2- 172.21.15.60:0/4038894029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bd5a4d0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f4600002240 tx=0x7f4600011000 comp rx=0 tx=0).stop 2024-04-25T10:12:37.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 -- 172.21.15.60:0/4038894029 shutdown_connections 2024-04-25T10:12:37.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 --2- 172.21.15.60:0/4038894029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd106a0 0x558d3bd10a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 --2- 172.21.15.60:0/4038894029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd5aa10 0x558d3bd10160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 --2- 172.21.15.60:0/4038894029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bd5a4d0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 -- 172.21.15.60:0/4038894029 >> 172.21.15.60:0/4038894029 conn(0x558d3bd08240 msgr2=0x558d3bd0a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 -- 172.21.15.60:0/4038894029 shutdown_connections 2024-04-25T10:12:37.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.912+0000 7f4617f28000 1 -- 172.21.15.60:0/4038894029 wait complete. 2024-04-25T10:12:37.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 -- start start 2024-04-25T10:12:37.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 Processor -- start 2024-04-25T10:12:37.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd106a0 0x558d3bd61450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd5a0f0 0x558d3bd61990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd5aa10 0x558d3bd47d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3bb146c0 con 0x558d3bd106a0 2024-04-25T10:12:37.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3bb56110 con 0x558d3bd5aa10 2024-04-25T10:12:37.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d3bd15200 con 0x558d3bd5a0f0 2024-04-25T10:12:37.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 10 client.0 _ll_register_callbacks cb 0x558d3bcf7570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:37.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f460e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd106a0 0x558d3bd61450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f460e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd106a0 0x558d3bd61450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34130/0 (socket says 172.21.15.60:34130) 2024-04-25T10:12:37.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f460e59c640 1 -- 172.21.15.60:0/105867301 learned_addr learned my addr 172.21.15.60:0/105867301 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:37.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f460ed9d640 1 --2- 172.21.15.60:0/105867301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd5aa10 0x558d3bd47d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f460dd9b640 1 --2- 172.21.15.60:0/105867301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd5a0f0 0x558d3bd61990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f460e59c640 1 -- 172.21.15.60:0/105867301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd5a0f0 msgr2=0x558d3bd61990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f460e59c640 1 --2- 172.21.15.60:0/105867301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd5a0f0 0x558d3bd61990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f460e59c640 1 -- 172.21.15.60:0/105867301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd5aa10 msgr2=0x558d3bd47d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f460e59c640 1 --2- 172.21.15.60:0/105867301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd5aa10 0x558d3bd47d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f460e59c640 1 -- 172.21.15.60:0/105867301 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d3bae23b0 con 0x558d3bd106a0 2024-04-25T10:12:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f460e59c640 1 --2- 172.21.15.60:0/105867301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd106a0 0x558d3bd61450 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f4600007930 tx=0x7f4600007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117a321b14f6a062 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f45ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:12:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f45ed7fa640 1 -- 172.21.15.60:0/105867301 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600013e50 con 0x558d3bd106a0 2024-04-25T10:12:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f45ed7fa640 1 -- 172.21.15.60:0/105867301 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f4600004b70 con 0x558d3bd106a0 2024-04-25T10:12:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 -- 172.21.15.60:0/105867301 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4600013630 con 0x558d3bd106a0 2024-04-25T10:12:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f45ed7fa640 1 -- 172.21.15.60:0/105867301 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4600032050 con 0x558d3bd106a0 2024-04-25T10:12:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 10 client.8097 fetch_fsmap learned FSMap version 62 2024-04-25T10:12:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 -- 172.21.15.60:0/105867301 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x558d3bbdbe00 con 0x558d3bd106a0 2024-04-25T10:12:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f45ed7fa640 1 -- 172.21.15.60:0/105867301 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4600013630 con 0x558d3bd106a0 2024-04-25T10:12:37.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f45ed7fa640 1 -- 172.21.15.60:0/105867301 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4600035020 con 0x558d3bd106a0 2024-04-25T10:12:37.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 10 client.8097 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:12:37.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 10 client.8097 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:12:37.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 20 client.8097 populate_metadata read hostname 'smithi060' 2024-04-25T10:12:37.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 --2- 172.21.15.60:0/105867301 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x558d3bd48b30 0x558d3bd48f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 4 client.8097 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:37.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 -- 172.21.15.60:0/105867301 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d3baf5d60 con 0x558d3bd48b30 2024-04-25T10:12:37.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f460dd9b640 1 --2- 172.21.15.60:0/105867301 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x558d3bd48b30 0x558d3bd48f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f460dd9b640 1 --2- 172.21.15.60:0/105867301 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x558d3bd48b30 0x558d3bd48f40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e1dd7f09a35f53 server_cookie=df6382827e8ff6da in_seq=0 out_seq=0 2024-04-25T10:12:37.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f45ed7fa640 10 client.8097 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:37.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f45ed7fa640 1 -- 172.21.15.60:0/105867301 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d3baf5d60 con 0x558d3bd48b30 2024-04-25T10:12:37.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f45ed7fa640 10 client.8097 handle_command_reply: tid=0 2024-04-25T10:12:37.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 client.8097 shutdown 2024-04-25T10:12:37.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 -- 172.21.15.60:0/105867301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd106a0 msgr2=0x558d3bd61450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 --2- 172.21.15.60:0/105867301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd106a0 0x558d3bd61450 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f4600007930 tx=0x7f4600007000 comp rx=0 tx=0).stop 2024-04-25T10:12:37.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 -- 172.21.15.60:0/105867301 shutdown_connections 2024-04-25T10:12:37.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 --2- 172.21.15.60:0/105867301 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x558d3bd48b30 0x558d3bd48f40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 --2- 172.21.15.60:0/105867301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd5aa10 0x558d3bd47d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 --2- 172.21.15.60:0/105867301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd5a0f0 0x558d3bd61990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f45ed7fa640 0 client.8097 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:37.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 --2- 172.21.15.60:0/105867301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd106a0 0x558d3bd61450 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 -- 172.21.15.60:0/105867301 >> 172.21.15.60:0/105867301 conn(0x558d3bd08240 msgr2=0x558d3bd11310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 -- 172.21.15.60:0/105867301 shutdown_connections 2024-04-25T10:12:37.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.916+0000 7f4617f28000 1 -- 172.21.15.60:0/105867301 wait complete. 2024-04-25T10:12:37.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.920+0000 7f4617f28000 20 client.8097 trim_cache size 0 max 16384 2024-04-25T10:12:37.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.932+0000 7f4617f28000 1 Processor -- start 2024-04-25T10:12:37.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.932+0000 7f4617f28000 1 -- start start 2024-04-25T10:12:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.932+0000 7f4617f28000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd106a0 0x558d3bd0a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.932+0000 7f4617f28000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bd0ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.932+0000 7f4617f28000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd5aa10 0x558d3bd5ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.932+0000 7f4617f28000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46101fe5e0 con 0x558d3bd5a0f0 2024-04-25T10:12:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.932+0000 7f4617f28000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461006e810 con 0x558d3bd5aa10 2024-04-25T10:12:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.932+0000 7f4617f28000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46100ab550 con 0x558d3bd106a0 2024-04-25T10:12:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.932+0000 7f460dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bd0ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.932+0000 7f460dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bd0ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34140/0 (socket says 172.21.15.60:34140) 2024-04-25T10:12:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.932+0000 7f460dd9b640 1 -- 172.21.15.60:0/1081363362 learned_addr learned my addr 172.21.15.60:0/1081363362 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f460dd9b640 1 -- 172.21.15.60:0/1081363362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd106a0 msgr2=0x558d3bd0a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f460dd9b640 1 --2- 172.21.15.60:0/1081363362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd106a0 0x558d3bd0a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f460dd9b640 1 -- 172.21.15.60:0/1081363362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd5aa10 msgr2=0x558d3bd5ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f460dd9b640 1 --2- 172.21.15.60:0/1081363362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd5aa10 0x558d3bd5ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f460dd9b640 1 -- 172.21.15.60:0/1081363362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d3bbb1c50 con 0x558d3bd5a0f0 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f460dd9b640 1 --2- 172.21.15.60:0/1081363362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bd0ab70 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f45f80ef900 tx=0x7f45f811f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8c5e5cf2de467d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f45ef7fe640 1 -- 172.21.15.60:0/1081363362 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80edc70 con 0x558d3bd5a0f0 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f45ef7fe640 1 -- 172.21.15.60:0/1081363362 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f80af040 con 0x558d3bd5a0f0 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f45ef7fe640 1 -- 172.21.15.60:0/1081363362 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80f7bf0 con 0x558d3bd5a0f0 2024-04-25T10:12:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 -- 172.21.15.60:0/1081363362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 msgr2=0x558d3bd0ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 --2- 172.21.15.60:0/1081363362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bd0ab70 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f45f80ef900 tx=0x7f45f811f040 comp rx=0 tx=0).stop 2024-04-25T10:12:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 -- 172.21.15.60:0/1081363362 shutdown_connections 2024-04-25T10:12:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 --2- 172.21.15.60:0/1081363362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd5aa10 0x558d3bd5ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 --2- 172.21.15.60:0/1081363362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bd0ab70 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 --2- 172.21.15.60:0/1081363362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd106a0 0x558d3bd0a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 -- 172.21.15.60:0/1081363362 >> 172.21.15.60:0/1081363362 conn(0x558d3bd08240 msgr2=0x558d3bd58f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 -- 172.21.15.60:0/1081363362 shutdown_connections 2024-04-25T10:12:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 -- 172.21.15.60:0/1081363362 wait complete. 2024-04-25T10:12:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 -- start start 2024-04-25T10:12:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 Processor -- start 2024-04-25T10:12:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd106a0 0x558d3bdbad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bdbb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd5aa10 0x558d3bd5f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d3bd080c0 con 0x558d3bd5a0f0 2024-04-25T10:12:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46100abb50 con 0x558d3bd106a0 2024-04-25T10:12:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d3bafcbd0 con 0x558d3bd5aa10 2024-04-25T10:12:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 10 client.0 _ll_register_callbacks cb 0x7f4600001a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f460dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bdbb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f460dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bdbb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34146/0 (socket says 172.21.15.60:34146) 2024-04-25T10:12:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f460dd9b640 1 -- 172.21.15.60:0/43556168 learned_addr learned my addr 172.21.15.60:0/43556168 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f460dd9b640 1 -- 172.21.15.60:0/43556168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd5aa10 msgr2=0x558d3bd5f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f460dd9b640 1 --2- 172.21.15.60:0/43556168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd5aa10 0x558d3bd5f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f460dd9b640 1 -- 172.21.15.60:0/43556168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd106a0 msgr2=0x558d3bdbad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f460dd9b640 1 --2- 172.21.15.60:0/43556168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd106a0 0x558d3bdbad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f460dd9b640 1 -- 172.21.15.60:0/43556168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d3bb22120 con 0x558d3bd5a0f0 2024-04-25T10:12:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f45ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:12:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f460dd9b640 1 --2- 172.21.15.60:0/43556168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bdbb2b0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f45f80190b0 tx=0x7f4600016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2c72bcad924e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f45ed7fa640 1 -- 172.21.15.60:0/43556168 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80f7bf0 con 0x558d3bd5a0f0 2024-04-25T10:12:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f45ed7fa640 1 -- 172.21.15.60:0/43556168 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f45f80f7d90 con 0x558d3bd5a0f0 2024-04-25T10:12:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 -- 172.21.15.60:0/43556168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45f80af040 con 0x558d3bd5a0f0 2024-04-25T10:12:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f45ed7fa640 1 -- 172.21.15.60:0/43556168 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45f80eedc0 con 0x558d3bd5a0f0 2024-04-25T10:12:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 10 client.8103 fetch_fsmap learned FSMap version 62 2024-04-25T10:12:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 -- 172.21.15.60:0/43556168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x558d3bb01170 con 0x558d3bd5a0f0 2024-04-25T10:12:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f45ed7fa640 1 -- 172.21.15.60:0/43556168 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f810d030 con 0x558d3bd5a0f0 2024-04-25T10:12:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f45ed7fa640 1 -- 172.21.15.60:0/43556168 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f45f80c6030 con 0x558d3bd5a0f0 2024-04-25T10:12:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 10 client.8103 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:12:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 10 client.8103 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:12:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 20 client.8103 populate_metadata read hostname 'smithi060' 2024-04-25T10:12:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 --2- 172.21.15.60:0/43556168 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x558d3bd5fb40 0x558d3bd5ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 4 client.8103 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:12:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f4617f28000 1 -- 172.21.15.60:0/43556168 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x558d3bb675b0 con 0x558d3bd5fb40 2024-04-25T10:12:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.936+0000 7f460e59c640 1 --2- 172.21.15.60:0/43556168 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x558d3bd5fb40 0x558d3bd5ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f45ed7fa640 10 client.8103 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f460e59c640 1 --2- 172.21.15.60:0/43556168 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x558d3bd5fb40 0x558d3bd5ff50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=191c2d30ad457e93 server_cookie=3eb45eadec972bea in_seq=0 out_seq=0 2024-04-25T10:12:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f45ed7fa640 1 -- 172.21.15.60:0/43556168 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558d3bb675b0 con 0x558d3bd5fb40 2024-04-25T10:12:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f45ed7fa640 10 client.8103 handle_command_reply: tid=0 2024-04-25T10:12:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4617f28000 1 client.8103 shutdown 2024-04-25T10:12:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4617f28000 1 -- 172.21.15.60:0/43556168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 msgr2=0x558d3bdbb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4617f28000 1 --2- 172.21.15.60:0/43556168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bdbb2b0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f45f80190b0 tx=0x7f4600016040 comp rx=0 tx=0).stop 2024-04-25T10:12:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4617f28000 1 -- 172.21.15.60:0/43556168 shutdown_connections 2024-04-25T10:12:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4617f28000 1 --2- 172.21.15.60:0/43556168 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x558d3bd5fb40 0x558d3bd5ff50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4617f28000 1 --2- 172.21.15.60:0/43556168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d3bd5aa10 0x558d3bd5f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4617f28000 1 --2- 172.21.15.60:0/43556168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d3bd5a0f0 0x558d3bdbb2b0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4617f28000 1 --2- 172.21.15.60:0/43556168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d3bd106a0 0x558d3bdbad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4617f28000 1 -- 172.21.15.60:0/43556168 >> 172.21.15.60:0/43556168 conn(0x558d3bd08240 msgr2=0x558d3bd10a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f45ed7fa640 0 client.8103 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4617f28000 1 -- 172.21.15.60:0/43556168 shutdown_connections 2024-04-25T10:12:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4617f28000 1 -- 172.21.15.60:0/43556168 wait complete. 2024-04-25T10:12:37.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4617f28000 20 client.8103 trim_cache size 0 max 16384 2024-04-25T10:12:37.954 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-04-25T10:12:37.940+0000 7ff066d59640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv16 ap=3 DIRTYPARENT f(v0 m2024-04-25T10:11:53.583631+0000 1=1+0)->f(v1 m2024-04-25T10:11:53.583631+0000) n(v0 rc2024-04-25T10:11:53.694129+0000 2=1+1)->n(v0 rc2024-04-25T10:11:53.694129+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55e103712100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:11:53.583631+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:11:53.694129+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-25T10:12:37.954 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:12:37.954 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T10:12:37.954 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "2b05146a-5276-4760-aafa-b7e556f5b7ef", 2024-04-25T10:12:37.954 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T10:12:37.954 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:12:37.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4614dbb640 1 -- 172.21.15.60:0/3375728677 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f45f0066f90 msgr2=0x7f45f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4614dbb640 1 --2- 172.21.15.60:0/3375728677 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f45f0066f90 0x7f45f0069450 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4600001570 tx=0x7f4600009040 comp rx=0 tx=0).stop 2024-04-25T10:12:37.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4614dbb640 1 -- 172.21.15.60:0/3375728677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46100cf0a0 msgr2=0x7f46100a8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4614dbb640 1 --2- 172.21.15.60:0/3375728677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46100cf0a0 0x7f46100a8490 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f45f80ac040 tx=0x7f45f8019240 comp rx=0 tx=0).stop 2024-04-25T10:12:37.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4614dbb640 1 -- 172.21.15.60:0/3375728677 shutdown_connections 2024-04-25T10:12:37.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4614dbb640 1 --2- 172.21.15.60:0/3375728677 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f45f0066f90 0x7f45f0069450 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4614dbb640 1 --2- 172.21.15.60:0/3375728677 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4610144e40 0x7f461015e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4614dbb640 1 --2- 172.21.15.60:0/3375728677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46100cf0a0 0x7f46100a8490 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4614dbb640 1 --2- 172.21.15.60:0/3375728677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4610071980 0x7f461015b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4614dbb640 1 -- 172.21.15.60:0/3375728677 >> 172.21.15.60:0/3375728677 conn(0x7f4610066fa0 msgr2=0x7f46100b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4614dbb640 1 -- 172.21.15.60:0/3375728677 shutdown_connections 2024-04-25T10:12:37.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:37.940+0000 7f4614dbb640 1 -- 172.21.15.60:0/3375728677 wait complete. 2024-04-25T10:12:37.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:37.964 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:12:38.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.068+0000 7f61bd1af640 1 Processor -- start 2024-04-25T10:12:38.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61bd1af640 1 -- start start 2024-04-25T10:12:38.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61bd1af640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f61b80b3b20 0x7f61b815c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61bd1af640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f61b815c630 0x7f61b80c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61bd1af640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b815f5c0 con 0x7f61b81595a0 2024-04-25T10:12:38.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61bd1af640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b8160380 con 0x7f61b815c630 2024-04-25T10:12:38.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61bd1af640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b815f8a0 con 0x7f61b80b3b20 2024-04-25T10:12:38.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61b759e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f61b815c630 0x7f61b80c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61b6d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f61b81595a0 0x7f61b80b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41218/0 (socket says 172.21.15.60:41218) 2024-04-25T10:12:38.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61b6d9d640 1 -- 172.21.15.60:0/2081016732 learned_addr learned my addr 172.21.15.60:0/2081016732 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:12:38.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61b659c640 1 --2- 172.21.15.60:0/2081016732 >> v2:172.21.15.100:3300/0 conn(0x7f61b80b3b20 0x7f61b815c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61b5d9b640 1 -- 172.21.15.60:0/2081016732 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807347190 0 0) 0x7f61b815f5c0 con 0x7f61b81595a0 2024-04-25T10:12:38.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61b5d9b640 1 -- 172.21.15.60:0/2081016732 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61a0003530 con 0x7f61b81595a0 2024-04-25T10:12:38.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61b759e640 1 -- 172.21.15.60:0/2081016732 >> v2:172.21.15.100:3300/0 conn(0x7f61b80b3b20 msgr2=0x7f61b815c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61b759e640 1 --2- 172.21.15.60:0/2081016732 >> v2:172.21.15.100:3300/0 conn(0x7f61b80b3b20 0x7f61b815c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61b759e640 1 -- 172.21.15.60:0/2081016732 >> v1:172.21.15.60:6789/0 conn(0x7f61b81595a0 legacy=0x7f61b80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61b759e640 1 -- 172.21.15.60:0/2081016732 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b815fb80 con 0x7f61b815c630 2024-04-25T10:12:38.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.072+0000 7f61b759e640 1 --2- 172.21.15.60:0/2081016732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61b815c630 0x7f61b80c6a60 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f61b80b40a0 tx=0x7f61b001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edafe2b635de708d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61b5d9b640 1 -- 172.21.15.60:0/2081016732 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00b2070 con 0x7f61b815c630 2024-04-25T10:12:38.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61b5d9b640 1 -- 172.21.15.60:0/2081016732 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b00a5900 con 0x7f61b815c630 2024-04-25T10:12:38.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61b5d9b640 1 -- 172.21.15.60:0/2081016732 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00a5cb0 con 0x7f61b815c630 2024-04-25T10:12:38.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 -- 172.21.15.60:0/2081016732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61b815c630 msgr2=0x7f61b80c6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 --2- 172.21.15.60:0/2081016732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61b815c630 0x7f61b80c6a60 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f61b80b40a0 tx=0x7f61b001c930 comp rx=0 tx=0).stop 2024-04-25T10:12:38.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 -- 172.21.15.60:0/2081016732 shutdown_connections 2024-04-25T10:12:38.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 --2- 172.21.15.60:0/2081016732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61b815c630 0x7f61b80c6a60 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 --2- 172.21.15.60:0/2081016732 >> v2:172.21.15.100:3300/0 conn(0x7f61b80b3b20 0x7f61b815c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 -- 172.21.15.60:0/2081016732 >> 172.21.15.60:0/2081016732 conn(0x7f61b80669a0 msgr2=0x7f61b8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 -- 172.21.15.60:0/2081016732 shutdown_connections 2024-04-25T10:12:38.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 -- 172.21.15.60:0/2081016732 wait complete. 2024-04-25T10:12:38.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 Processor -- start 2024-04-25T10:12:38.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 -- start start 2024-04-25T10:12:38.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61b80b3b20 0x7f61b8158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61b81595a0 0x7f61b81f68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b815c630 0x7f61b81f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b80c7b80 con 0x7f61b81595a0 2024-04-25T10:12:38.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b80c7a00 con 0x7f61b815c630 2024-04-25T10:12:38.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b8168750 con 0x7f61b80b3b20 2024-04-25T10:12:38.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61b659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61b81595a0 0x7f61b81f68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61b659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61b81595a0 0x7f61b81f68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34170/0 (socket says 172.21.15.60:34170) 2024-04-25T10:12:38.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61b659c640 1 -- 172.21.15.60:0/19969770 learned_addr learned my addr 172.21.15.60:0/19969770 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:38.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61b6d9d640 1 --2- 172.21.15.60:0/19969770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61b80b3b20 0x7f61b8158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61b759e640 1 --2- 172.21.15.60:0/19969770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b815c630 0x7f61b81f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61b659c640 1 -- 172.21.15.60:0/19969770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61b80b3b20 msgr2=0x7f61b8158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61b659c640 1 --2- 172.21.15.60:0/19969770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61b80b3b20 0x7f61b8158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61b659c640 1 -- 172.21.15.60:0/19969770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b815c630 msgr2=0x7f61b81f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61b659c640 1 --2- 172.21.15.60:0/19969770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b815c630 0x7f61b81f7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61b659c640 1 -- 172.21.15.60:0/19969770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b8156fe0 con 0x7f61b81595a0 2024-04-25T10:12:38.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61b6d9d640 1 --2- 172.21.15.60:0/19969770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61b80b3b20 0x7f61b8158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:38.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61b759e640 1 --2- 172.21.15.60:0/19969770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b815c630 0x7f61b81f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:38.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61b659c640 1 --2- 172.21.15.60:0/19969770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61b81595a0 0x7f61b81f68b0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00d5f0 tx=0x7f61ac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f619f7fe640 1 -- 172.21.15.60:0/19969770 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac0125f0 con 0x7f61b81595a0 2024-04-25T10:12:38.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f619f7fe640 1 -- 172.21.15.60:0/19969770 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac019070 con 0x7f61b81595a0 2024-04-25T10:12:38.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 -- 172.21.15.60:0/19969770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b81572c0 con 0x7f61b81595a0 2024-04-25T10:12:38.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f619f7fe640 1 -- 172.21.15.60:0/19969770 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac0129b0 con 0x7f61b81595a0 2024-04-25T10:12:38.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.076+0000 7f61bd1af640 1 -- 172.21.15.60:0/19969770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b8157880 con 0x7f61b81595a0 2024-04-25T10:12:38.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f619f7fe640 1 -- 172.21.15.60:0/19969770 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61ac012b50 con 0x7f61b81595a0 2024-04-25T10:12:38.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f619f7fe640 1 --2- 172.21.15.60:0/19969770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f61a4066f60 0x7f61a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f619f7fe640 1 -- 172.21.15.60:0/19969770 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f61ac0854e0 con 0x7f61b81595a0 2024-04-25T10:12:38.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f61b6d9d640 1 --2- 172.21.15.60:0/19969770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f61a4066f60 0x7f61a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f61c031c000 1 Processor -- start 2024-04-25T10:12:38.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f61c031c000 1 -- start start 2024-04-25T10:12:38.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f61b6d9d640 1 --2- 172.21.15.60:0/19969770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f61a4066f60 0x7f61a4069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f61a8005890 tx=0x7f61a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f61c031c000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578dc09d0 0x559578dc0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f61c031c000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578dc12f0 0x559578d76b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f61c031c000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578d770a0 0x559578d77480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f61c031c000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559578bc3ff0 con 0x559578d770a0 2024-04-25T10:12:38.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f61c031c000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559578b4c150 con 0x559578dc12f0 2024-04-25T10:12:38.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f61c031c000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559578bc0090 con 0x559578dc09d0 2024-04-25T10:12:38.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f61b759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578d770a0 0x559578d77480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f61b759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578d770a0 0x559578d77480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34182/0 (socket says 172.21.15.60:34182) 2024-04-25T10:12:38.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f61b759e640 1 -- 172.21.15.60:0/488733467 learned_addr learned my addr 172.21.15.60:0/488733467 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:38.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f61b6d9d640 1 --2- 172.21.15.60:0/488733467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578dc09d0 0x559578dc0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.080+0000 7f61b659c640 1 --2- 172.21.15.60:0/488733467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578dc12f0 0x559578d76b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b759e640 1 -- 172.21.15.60:0/488733467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578dc09d0 msgr2=0x559578dc0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b759e640 1 --2- 172.21.15.60:0/488733467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578dc09d0 0x559578dc0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b759e640 1 -- 172.21.15.60:0/488733467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578dc12f0 msgr2=0x559578d76b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b759e640 1 --2- 172.21.15.60:0/488733467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578dc12f0 0x559578d76b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b759e640 1 -- 172.21.15.60:0/488733467 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559578d5a0d0 con 0x559578d770a0 2024-04-25T10:12:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b759e640 1 --2- 172.21.15.60:0/488733467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578d770a0 0x559578d77480 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f61b0019540 tx=0x7f61b0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98e33cdc79fdd03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f619cff9640 1 -- 172.21.15.60:0/488733467 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00b2040 con 0x559578d770a0 2024-04-25T10:12:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f619cff9640 1 -- 172.21.15.60:0/488733467 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b00bc070 con 0x559578d770a0 2024-04-25T10:12:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f619cff9640 1 -- 172.21.15.60:0/488733467 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0003e40 con 0x559578d770a0 2024-04-25T10:12:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 -- 172.21.15.60:0/488733467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578d770a0 msgr2=0x559578d77480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 --2- 172.21.15.60:0/488733467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578d770a0 0x559578d77480 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f61b0019540 tx=0x7f61b0004ac0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 -- 172.21.15.60:0/488733467 shutdown_connections 2024-04-25T10:12:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 --2- 172.21.15.60:0/488733467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578d770a0 0x559578d77480 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 --2- 172.21.15.60:0/488733467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578dc12f0 0x559578d76b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 --2- 172.21.15.60:0/488733467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578dc09d0 0x559578dc0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 -- 172.21.15.60:0/488733467 >> 172.21.15.60:0/488733467 conn(0x559578d73020 msgr2=0x559578d75440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 -- 172.21.15.60:0/488733467 shutdown_connections 2024-04-25T10:12:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 -- 172.21.15.60:0/488733467 wait complete. 2024-04-25T10:12:38.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 -- start start 2024-04-25T10:12:38.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 Processor -- start 2024-04-25T10:12:38.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578d770a0 0x559578da57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578dc09d0 0x559578da5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578dc12f0 0x559578da6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559578b88250 con 0x559578d770a0 2024-04-25T10:12:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559578b7e690 con 0x559578dc09d0 2024-04-25T10:12:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559578b4ca40 con 0x559578dc12f0 2024-04-25T10:12:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 10 client.0 _ll_register_callbacks cb 0x559578d61e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578d770a0 0x559578da57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578d770a0 0x559578da57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34184/0 (socket says 172.21.15.60:34184) 2024-04-25T10:12:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b6d9d640 1 -- 172.21.15.60:0/47253296 learned_addr learned my addr 172.21.15.60:0/47253296 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b659c640 1 --2- 172.21.15.60:0/47253296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578dc09d0 0x559578da5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b759e640 1 --2- 172.21.15.60:0/47253296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578dc12f0 0x559578da6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b6d9d640 1 -- 172.21.15.60:0/47253296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578dc12f0 msgr2=0x559578da6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b6d9d640 1 --2- 172.21.15.60:0/47253296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578dc12f0 0x559578da6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b6d9d640 1 -- 172.21.15.60:0/47253296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578dc09d0 msgr2=0x559578da5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b6d9d640 1 --2- 172.21.15.60:0/47253296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578dc09d0 0x559578da5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b6d9d640 1 -- 172.21.15.60:0/47253296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559578c711c0 con 0x559578d770a0 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b6d9d640 1 --2- 172.21.15.60:0/47253296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578d770a0 0x559578da57f0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f61a8007370 tx=0x7f61a8000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4641b58feeeb8e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f6195ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f6195ffb640 1 -- 172.21.15.60:0/47253296 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a8027060 con 0x559578d770a0 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f6195ffb640 1 -- 172.21.15.60:0/47253296 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f61a800de40 con 0x559578d770a0 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 -- 172.21.15.60:0/47253296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61b00bc070 con 0x559578d770a0 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f6195ffb640 1 -- 172.21.15.60:0/47253296 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61a8012620 con 0x559578d770a0 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 10 client.8118 fetch_fsmap learned FSMap version 62 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 -- 172.21.15.60:0/47253296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x559578c6d170 con 0x559578d770a0 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f6195ffb640 1 -- 172.21.15.60:0/47253296 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b00bc070 con 0x559578d770a0 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f6195ffb640 1 -- 172.21.15.60:0/47253296 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f61a80127c0 con 0x559578d770a0 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 10 client.8118 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 10 client.8118 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:12:38.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 20 client.8118 populate_metadata read hostname 'smithi060' 2024-04-25T10:12:38.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 --2- 172.21.15.60:0/47253296 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x559578dd0c80 0x559578dd3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 4 client.8118 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:38.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61c031c000 1 -- 172.21.15.60:0/47253296 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559578b5fc60 con 0x559578dd0c80 2024-04-25T10:12:38.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.084+0000 7f61b659c640 1 --2- 172.21.15.60:0/47253296 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x559578dd0c80 0x559578dd3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f6195ffb640 10 client.8118 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:38.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f61b659c640 1 --2- 172.21.15.60:0/47253296 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x559578dd0c80 0x559578dd3070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32822c08b62a0cb6 server_cookie=14fc17e61119f1a9 in_seq=0 out_seq=0 2024-04-25T10:12:38.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f6195ffb640 1 -- 172.21.15.60:0/47253296 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559578b5fc60 con 0x559578dd0c80 2024-04-25T10:12:38.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f6195ffb640 10 client.8118 handle_command_reply: tid=0 2024-04-25T10:12:38.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f61c031c000 1 client.8118 shutdown 2024-04-25T10:12:38.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f61c031c000 1 -- 172.21.15.60:0/47253296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578d770a0 msgr2=0x559578da57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f61c031c000 1 --2- 172.21.15.60:0/47253296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578d770a0 0x559578da57f0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f61a8007370 tx=0x7f61a8000ee0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f61c031c000 1 -- 172.21.15.60:0/47253296 shutdown_connections 2024-04-25T10:12:38.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f61c031c000 1 --2- 172.21.15.60:0/47253296 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x559578dd0c80 0x559578dd3070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f61c031c000 1 --2- 172.21.15.60:0/47253296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578dc12f0 0x559578da6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f61c031c000 1 --2- 172.21.15.60:0/47253296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578dc09d0 0x559578da5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f6195ffb640 0 client.8118 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:38.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f61c031c000 1 --2- 172.21.15.60:0/47253296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578d770a0 0x559578da57f0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f61c031c000 1 -- 172.21.15.60:0/47253296 >> 172.21.15.60:0/47253296 conn(0x559578d73020 msgr2=0x559578d75ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f61c031c000 1 -- 172.21.15.60:0/47253296 shutdown_connections 2024-04-25T10:12:38.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f61c031c000 1 -- 172.21.15.60:0/47253296 wait complete. 2024-04-25T10:12:38.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.088+0000 7f61c031c000 20 client.8118 trim_cache size 0 max 16384 2024-04-25T10:12:38.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.100+0000 7f61c031c000 1 Processor -- start 2024-04-25T10:12:38.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.100+0000 7f61c031c000 1 -- start start 2024-04-25T10:12:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578d770a0 0x559578d76a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578dc09d0 0x559578dcde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578dc12f0 0x559578dce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b8202a90 con 0x559578dc12f0 2024-04-25T10:12:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b8201cc0 con 0x559578d770a0 2024-04-25T10:12:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b80cd5a0 con 0x559578dc09d0 2024-04-25T10:12:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578dc12f0 0x559578dce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578dc12f0 0x559578dce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34196/0 (socket says 172.21.15.60:34196) 2024-04-25T10:12:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 -- 172.21.15.60:0/366856059 learned_addr learned my addr 172.21.15.60:0/366856059 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 -- 172.21.15.60:0/366856059 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578dc09d0 msgr2=0x559578dcde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 --2- 172.21.15.60:0/366856059 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578dc09d0 0x559578dcde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 -- 172.21.15.60:0/366856059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578d770a0 msgr2=0x559578d76a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 --2- 172.21.15.60:0/366856059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578d770a0 0x559578d76a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 -- 172.21.15.60:0/366856059 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559578c0ade0 con 0x559578dc12f0 2024-04-25T10:12:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 --2- 172.21.15.60:0/366856059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578dc12f0 0x559578dce380 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f61b0019540 tx=0x7f61b00a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5421274f48ea00f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f6197fff640 1 -- 172.21.15.60:0/366856059 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00c4070 con 0x559578dc12f0 2024-04-25T10:12:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f6197fff640 1 -- 172.21.15.60:0/366856059 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b00bc040 con 0x559578dc12f0 2024-04-25T10:12:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f6197fff640 1 -- 172.21.15.60:0/366856059 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00b2050 con 0x559578dc12f0 2024-04-25T10:12:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 -- 172.21.15.60:0/366856059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578dc12f0 msgr2=0x559578dce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 --2- 172.21.15.60:0/366856059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578dc12f0 0x559578dce380 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f61b0019540 tx=0x7f61b00a7000 comp rx=0 tx=0).stop 2024-04-25T10:12:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 -- 172.21.15.60:0/366856059 shutdown_connections 2024-04-25T10:12:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 --2- 172.21.15.60:0/366856059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578dc12f0 0x559578dce380 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 --2- 172.21.15.60:0/366856059 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578dc09d0 0x559578dcde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 --2- 172.21.15.60:0/366856059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578d770a0 0x559578d76a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 -- 172.21.15.60:0/366856059 >> 172.21.15.60:0/366856059 conn(0x559578d73020 msgr2=0x559578dcd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 -- 172.21.15.60:0/366856059 shutdown_connections 2024-04-25T10:12:38.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 -- 172.21.15.60:0/366856059 wait complete. 2024-04-25T10:12:38.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 -- start start 2024-04-25T10:12:38.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 Processor -- start 2024-04-25T10:12:38.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578d770a0 0x559578e20510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578dc09d0 0x559578e20a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578dc12f0 0x559578dd2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559578b5ee00 con 0x559578dc12f0 2024-04-25T10:12:38.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b80b1fb0 con 0x559578dc09d0 2024-04-25T10:12:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559578bc3300 con 0x559578d770a0 2024-04-25T10:12:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 10 client.0 _ll_register_callbacks cb 0x559578d42040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578dc12f0 0x559578dd2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578dc12f0 0x559578dd2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34202/0 (socket says 172.21.15.60:34202) 2024-04-25T10:12:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 -- 172.21.15.60:0/2466781131 learned_addr learned my addr 172.21.15.60:0/2466781131 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:12:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 -- 172.21.15.60:0/2466781131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578d770a0 msgr2=0x559578e20510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 --2- 172.21.15.60:0/2466781131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578d770a0 0x559578e20510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 -- 172.21.15.60:0/2466781131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578dc09d0 msgr2=0x559578e20a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b659c640 1 --2- 172.21.15.60:0/2466781131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578dc09d0 0x559578e20a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 --2- 172.21.15.60:0/2466781131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578dc09d0 0x559578e20a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 -- 172.21.15.60:0/2466781131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559578b658e0 con 0x559578dc12f0 2024-04-25T10:12:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61b759e640 1 --2- 172.21.15.60:0/2466781131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578dc12f0 0x559578dd2a20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f61b001a560 tx=0x7f61b001b170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e832eedd3270f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f6195ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:12:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f6195ffb640 1 -- 172.21.15.60:0/2466781131 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00cc070 con 0x559578dc12f0 2024-04-25T10:12:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f6195ffb640 1 -- 172.21.15.60:0/2466781131 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f61b00a5700 con 0x559578dc12f0 2024-04-25T10:12:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 -- 172.21.15.60:0/2466781131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61b00bc040 con 0x559578dc12f0 2024-04-25T10:12:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f6195ffb640 1 -- 172.21.15.60:0/2466781131 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61b00d7050 con 0x559578dc12f0 2024-04-25T10:12:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 10 client.8124 fetch_fsmap learned FSMap version 62 2024-04-25T10:12:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.104+0000 7f61c031c000 1 -- 172.21.15.60:0/2466781131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x559578c1d8f0 con 0x559578dc12f0 2024-04-25T10:12:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f6195ffb640 1 -- 172.21.15.60:0/2466781131 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b00bc040 con 0x559578dc12f0 2024-04-25T10:12:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f6195ffb640 1 -- 172.21.15.60:0/2466781131 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f61b00a5900 con 0x559578dc12f0 2024-04-25T10:12:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 10 client.8124 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:12:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 10 client.8124 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:12:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 20 client.8124 populate_metadata read hostname 'smithi060' 2024-04-25T10:12:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 1 --2- 172.21.15.60:0/2466781131 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x559578dd33b0 0x559578dd37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 4 client.8124 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:12:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 1 -- 172.21.15.60:0/2466781131 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559578bd1530 con 0x559578dd33b0 2024-04-25T10:12:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61b6d9d640 1 --2- 172.21.15.60:0/2466781131 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x559578dd33b0 0x559578dd37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f6195ffb640 10 client.8124 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:38.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61b6d9d640 1 --2- 172.21.15.60:0/2466781131 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x559578dd33b0 0x559578dd37c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b87326ca3e2b866 server_cookie=3f5fcf0e58a709ff in_seq=0 out_seq=0 2024-04-25T10:12:38.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f6195ffb640 1 -- 172.21.15.60:0/2466781131 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x559578bd1530 con 0x559578dd33b0 2024-04-25T10:12:38.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f6195ffb640 10 client.8124 handle_command_reply: tid=0 2024-04-25T10:12:38.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 1 client.8124 shutdown 2024-04-25T10:12:38.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 1 -- 172.21.15.60:0/2466781131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578dc12f0 msgr2=0x559578dd2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 1 --2- 172.21.15.60:0/2466781131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578dc12f0 0x559578dd2a20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f61b001a560 tx=0x7f61b001b170 comp rx=0 tx=0).stop 2024-04-25T10:12:38.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 1 -- 172.21.15.60:0/2466781131 shutdown_connections 2024-04-25T10:12:38.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 1 --2- 172.21.15.60:0/2466781131 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x559578dd33b0 0x559578dd37c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 1 --2- 172.21.15.60:0/2466781131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559578dc12f0 0x559578dd2a20 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 1 --2- 172.21.15.60:0/2466781131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559578dc09d0 0x559578e20a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 1 --2- 172.21.15.60:0/2466781131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559578d770a0 0x559578e20510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 1 -- 172.21.15.60:0/2466781131 >> 172.21.15.60:0/2466781131 conn(0x559578d73020 msgr2=0x559578dcdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f6195ffb640 0 client.8124 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:12:38.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 1 -- 172.21.15.60:0/2466781131 shutdown_connections 2024-04-25T10:12:38.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 1 -- 172.21.15.60:0/2466781131 wait complete. 2024-04-25T10:12:38.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61c031c000 20 client.8124 trim_cache size 0 max 16384 2024-04-25T10:12:38.122 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:12:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:12:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T10:12:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "2b05146a-5276-4760-aafa-b7e556f5b7ef": { 2024-04-25T10:12:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/dir", 2024-04-25T10:12:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "2b05146a-5276-4760-aafa-b7e556f5b7ef", 2024-04-25T10:12:38.122 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,repair" 2024-04-25T10:12:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:12:38.123 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:12:38.123 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:12:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61bd1af640 1 -- 172.21.15.60:0/19969770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f61a4066f60 msgr2=0x7f61a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61bd1af640 1 --2- 172.21.15.60:0/19969770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f61a4066f60 0x7f61a4069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f61a8005890 tx=0x7f61a8009040 comp rx=0 tx=0).stop 2024-04-25T10:12:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61bd1af640 1 -- 172.21.15.60:0/19969770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61b81595a0 msgr2=0x7f61b81f68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61bd1af640 1 --2- 172.21.15.60:0/19969770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61b81595a0 0x7f61b81f68b0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00d5f0 tx=0x7f61ac00dab0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61bd1af640 1 -- 172.21.15.60:0/19969770 shutdown_connections 2024-04-25T10:12:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61bd1af640 1 --2- 172.21.15.60:0/19969770 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f61a4066f60 0x7f61a4069420 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61bd1af640 1 --2- 172.21.15.60:0/19969770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b815c630 0x7f61b81f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61bd1af640 1 --2- 172.21.15.60:0/19969770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61b81595a0 0x7f61b81f68b0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61bd1af640 1 --2- 172.21.15.60:0/19969770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61b80b3b20 0x7f61b8158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61bd1af640 1 -- 172.21.15.60:0/19969770 >> 172.21.15.60:0/19969770 conn(0x7f61b80669a0 msgr2=0x7f61b806f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61bd1af640 1 -- 172.21.15.60:0/19969770 shutdown_connections 2024-04-25T10:12:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:12:38.108+0000 7f61bd1af640 1 -- 172.21.15.60:0/19969770 wait complete. 2024-04-25T10:12:38.131 INFO:tasks.cephfs.filesystem:scrub status for tag:2b05146a-5276-4760-aafa-b7e556f5b7ef - {'path': '/dir', 'tag': '2b05146a-5276-4760-aafa-b7e556f5b7ef', 'options': 'recursive,repair'} 2024-04-25T10:12:47.351 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:47.355 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:08.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:08.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:13:08.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f744034b640 1 Processor -- start 2024-04-25T10:13:08.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f744034b640 1 -- start start 2024-04-25T10:13:08.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f744034b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74380cbcf0 0x7f7438160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f744034b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7438071c30 0x7f743815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f744034b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7438144a40 con 0x7f743815fa20 2024-04-25T10:13:08.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f744034b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74380b2dc0 con 0x7f7438071c30 2024-04-25T10:13:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f744034b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743815fe80 con 0x7f74380cbcf0 2024-04-25T10:13:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f743d8e6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7438071c30 0x7f743815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f743e8e8640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f743815fa20 0x7f743815bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48018/0 (socket says 172.21.15.60:48018) 2024-04-25T10:13:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f743e8e8640 1 -- 172.21.15.60:0/2002416872 learned_addr learned my addr 172.21.15.60:0/2002416872 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:13:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f743e0e7640 1 --2- 172.21.15.60:0/2002416872 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74380cbcf0 0x7f7438160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f743d0e5640 1 -- 172.21.15.60:0/2002416872 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3109981159 0 0) 0x7f7438144a40 con 0x7f743815fa20 2024-04-25T10:13:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f743d0e5640 1 -- 172.21.15.60:0/2002416872 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7424003530 con 0x7f743815fa20 2024-04-25T10:13:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f743d8e6640 1 -- 172.21.15.60:0/2002416872 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74380cbcf0 msgr2=0x7f7438160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f743d8e6640 1 --2- 172.21.15.60:0/2002416872 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74380cbcf0 0x7f7438160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f743d8e6640 1 -- 172.21.15.60:0/2002416872 >> v1:172.21.15.60:6789/0 conn(0x7f743815fa20 legacy=0x7f743815bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.267+0000 7f743d8e6640 1 -- 172.21.15.60:0/2002416872 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74380cc9f0 con 0x7f7438071c30 2024-04-25T10:13:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f743d8e6640 1 --2- 172.21.15.60:0/2002416872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7438071c30 0x7f743815f4e0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f74340097c0 tx=0x7f743401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7764a99e05e6a462 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f743d0e5640 1 -- 172.21.15.60:0/2002416872 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74340a7070 con 0x7f7438071c30 2024-04-25T10:13:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f743d0e5640 1 -- 172.21.15.60:0/2002416872 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74340b0070 con 0x7f7438071c30 2024-04-25T10:13:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f743d0e5640 1 -- 172.21.15.60:0/2002416872 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74340ab890 con 0x7f7438071c30 2024-04-25T10:13:08.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f744034b640 1 -- 172.21.15.60:0/2002416872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7438071c30 msgr2=0x7f743815f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f744034b640 1 --2- 172.21.15.60:0/2002416872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7438071c30 0x7f743815f4e0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f74340097c0 tx=0x7f743401c470 comp rx=0 tx=0).stop 2024-04-25T10:13:08.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f744034b640 1 -- 172.21.15.60:0/2002416872 shutdown_connections 2024-04-25T10:13:08.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f744034b640 1 --2- 172.21.15.60:0/2002416872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7438071c30 0x7f743815f4e0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f744034b640 1 --2- 172.21.15.60:0/2002416872 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74380cbcf0 0x7f7438160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f744034b640 1 -- 172.21.15.60:0/2002416872 >> 172.21.15.60:0/2002416872 conn(0x7f74380670b0 msgr2=0x7f743806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f744034b640 1 -- 172.21.15.60:0/2002416872 shutdown_connections 2024-04-25T10:13:08.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f744034b640 1 -- 172.21.15.60:0/2002416872 wait complete. 2024-04-25T10:13:08.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f744034b640 1 Processor -- start 2024-04-25T10:13:08.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f744034b640 1 -- start start 2024-04-25T10:13:08.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f744034b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7438071c30 0x7f74380ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f744034b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74380cbcf0 0x7f743813a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f744034b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743815fa20 0x7f7438143500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f744034b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74380720b0 con 0x7f7438071c30 2024-04-25T10:13:08.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f744034b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74380cfed0 con 0x7f743815fa20 2024-04-25T10:13:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f744034b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7438143e50 con 0x7f74380cbcf0 2024-04-25T10:13:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f743e0e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7438071c30 0x7f74380ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f743e0e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7438071c30 0x7f74380ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32942/0 (socket says 172.21.15.60:32942) 2024-04-25T10:13:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f743e0e7640 1 -- 172.21.15.60:0/2534626394 learned_addr learned my addr 172.21.15.60:0/2534626394 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.271+0000 7f743d8e6640 1 --2- 172.21.15.60:0/2534626394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74380cbcf0 0x7f743813a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f743e8e8640 1 --2- 172.21.15.60:0/2534626394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743815fa20 0x7f7438143500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f743e0e7640 1 -- 172.21.15.60:0/2534626394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74380cbcf0 msgr2=0x7f743813a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f743e0e7640 1 --2- 172.21.15.60:0/2534626394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74380cbcf0 0x7f743813a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f743e0e7640 1 -- 172.21.15.60:0/2534626394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743815fa20 msgr2=0x7f7438143500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f743e0e7640 1 --2- 172.21.15.60:0/2534626394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743815fa20 0x7f7438143500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f743e0e7640 1 -- 172.21.15.60:0/2534626394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743806dc80 con 0x7f7438071c30 2024-04-25T10:13:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f743e0e7640 1 --2- 172.21.15.60:0/2534626394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7438071c30 0x7f74380ac8a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f742800d800 tx=0x7f742800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f742effd640 1 -- 172.21.15.60:0/2534626394 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742800b6c0 con 0x7f7438071c30 2024-04-25T10:13:08.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f742effd640 1 -- 172.21.15.60:0/2534626394 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7428019070 con 0x7f7438071c30 2024-04-25T10:13:08.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f744034b640 1 -- 172.21.15.60:0/2534626394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743806df60 con 0x7f7438071c30 2024-04-25T10:13:08.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f742effd640 1 -- 172.21.15.60:0/2534626394 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742800ba60 con 0x7f7438071c30 2024-04-25T10:13:08.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f744034b640 1 -- 172.21.15.60:0/2534626394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743813a8f0 con 0x7f7438071c30 2024-04-25T10:13:08.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f742effd640 1 -- 172.21.15.60:0/2534626394 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f742800bc00 con 0x7f7438071c30 2024-04-25T10:13:08.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f742effd640 1 --2- 172.21.15.60:0/2534626394 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f741c066f90 0x7f741c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f742effd640 1 -- 172.21.15.60:0/2534626394 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f7428085450 con 0x7f7438071c30 2024-04-25T10:13:08.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f743d8e6640 1 --2- 172.21.15.60:0/2534626394 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f741c066f90 0x7f741c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f74434b8000 1 Processor -- start 2024-04-25T10:13:08.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.275+0000 7f74434b8000 1 -- start start 2024-04-25T10:13:08.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743d8e6640 1 --2- 172.21.15.60:0/2534626394 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f741c066f90 0x7f741c069450 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f74340af000 tx=0x7f74340a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f7679d0 0x55637f767db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f71db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f71e0a0 0x55637f71e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637f56aff0 con 0x55637f7682f0 2024-04-25T10:13:08.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637f4f3150 con 0x55637f71e0a0 2024-04-25T10:13:08.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55637f567090 con 0x55637f7679d0 2024-04-25T10:13:08.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743d8e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f71db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743d8e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f71db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32950/0 (socket says 172.21.15.60:32950) 2024-04-25T10:13:08.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743d8e6640 1 -- 172.21.15.60:0/1872803507 learned_addr learned my addr 172.21.15.60:0/1872803507 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:08.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743e0e7640 1 --2- 172.21.15.60:0/1872803507 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f7679d0 0x55637f767db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743e8e8640 1 --2- 172.21.15.60:0/1872803507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f71e0a0 0x55637f71e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743d8e6640 1 -- 172.21.15.60:0/1872803507 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f7679d0 msgr2=0x55637f767db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743d8e6640 1 --2- 172.21.15.60:0/1872803507 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f7679d0 0x55637f767db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743d8e6640 1 -- 172.21.15.60:0/1872803507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f71e0a0 msgr2=0x55637f71e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743d8e6640 1 --2- 172.21.15.60:0/1872803507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f71e0a0 0x55637f71e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743d8e6640 1 -- 172.21.15.60:0/1872803507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55637f7010d0 con 0x55637f7682f0 2024-04-25T10:13:08.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743e8e8640 1 --2- 172.21.15.60:0/1872803507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f71e0a0 0x55637f71e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:08.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743d8e6640 1 --2- 172.21.15.60:0/1872803507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f71db60 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f74340097c0 tx=0x7f7434004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2909fd6ff9cf3af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f741bfff640 1 -- 172.21.15.60:0/1872803507 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74340bc070 con 0x55637f7682f0 2024-04-25T10:13:08.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f741bfff640 1 -- 172.21.15.60:0/1872803507 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74340b0040 con 0x55637f7682f0 2024-04-25T10:13:08.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f741bfff640 1 -- 172.21.15.60:0/1872803507 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74340abda0 con 0x55637f7682f0 2024-04-25T10:13:08.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 -- 172.21.15.60:0/1872803507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 msgr2=0x55637f71db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 --2- 172.21.15.60:0/1872803507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f71db60 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f74340097c0 tx=0x7f7434004740 comp rx=0 tx=0).stop 2024-04-25T10:13:08.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 -- 172.21.15.60:0/1872803507 shutdown_connections 2024-04-25T10:13:08.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 --2- 172.21.15.60:0/1872803507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f71e0a0 0x55637f71e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 --2- 172.21.15.60:0/1872803507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f71db60 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 --2- 172.21.15.60:0/1872803507 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f7679d0 0x55637f767db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 -- 172.21.15.60:0/1872803507 >> 172.21.15.60:0/1872803507 conn(0x55637f71a020 msgr2=0x55637f71c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 -- 172.21.15.60:0/1872803507 shutdown_connections 2024-04-25T10:13:08.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 -- 172.21.15.60:0/1872803507 wait complete. 2024-04-25T10:13:08.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 -- start start 2024-04-25T10:13:08.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 Processor -- start 2024-04-25T10:13:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f71e0a0 0x55637f74c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f7679d0 0x55637f74cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f74d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637f52f250 con 0x55637f7682f0 2024-04-25T10:13:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637f525690 con 0x55637f7679d0 2024-04-25T10:13:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55637f4f3a40 con 0x55637f71e0a0 2024-04-25T10:13:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f74434b8000 10 client.0 _ll_register_callbacks cb 0x55637f708e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743e8e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f74d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743e8e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f74d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32962/0 (socket says 172.21.15.60:32962) 2024-04-25T10:13:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743e8e8640 1 -- 172.21.15.60:0/2105915532 learned_addr learned my addr 172.21.15.60:0/2105915532 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.279+0000 7f743e0e7640 1 --2- 172.21.15.60:0/2105915532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f71e0a0 0x55637f74c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f743d8e6640 1 --2- 172.21.15.60:0/2105915532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f7679d0 0x55637f74cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f743e8e8640 1 -- 172.21.15.60:0/2105915532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f71e0a0 msgr2=0x55637f74c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f743e8e8640 1 --2- 172.21.15.60:0/2105915532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f71e0a0 0x55637f74c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f743e8e8640 1 -- 172.21.15.60:0/2105915532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f7679d0 msgr2=0x55637f74cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f743e8e8640 1 --2- 172.21.15.60:0/2105915532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f7679d0 0x55637f74cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f743e8e8640 1 -- 172.21.15.60:0/2105915532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55637f6181c0 con 0x55637f7682f0 2024-04-25T10:13:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f743e8e8640 1 --2- 172.21.15.60:0/2105915532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f74d270 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f743000f800 tx=0x7f743000fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb812137c78fc2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74197fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:13:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74197fa640 1 -- 172.21.15.60:0/2105915532 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430017020 con 0x55637f7682f0 2024-04-25T10:13:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74197fa640 1 -- 172.21.15.60:0/2105915532 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f743000d8e0 con 0x55637f7682f0 2024-04-25T10:13:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 1 -- 172.21.15.60:0/2105915532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74340b0040 con 0x55637f7682f0 2024-04-25T10:13:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74197fa640 1 -- 172.21.15.60:0/2105915532 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7430021050 con 0x55637f7682f0 2024-04-25T10:13:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 10 client.8139 fetch_fsmap learned FSMap version 62 2024-04-25T10:13:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 1 -- 172.21.15.60:0/2105915532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55637f614170 con 0x55637f7682f0 2024-04-25T10:13:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74197fa640 1 -- 172.21.15.60:0/2105915532 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74340b0040 con 0x55637f7682f0 2024-04-25T10:13:08.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74197fa640 1 -- 172.21.15.60:0/2105915532 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7430024020 con 0x55637f7682f0 2024-04-25T10:13:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 10 client.8139 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:13:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 10 client.8139 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:13:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 20 client.8139 populate_metadata read hostname 'smithi060' 2024-04-25T10:13:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 1 --2- 172.21.15.60:0/2105915532 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55637f777c80 0x55637f77a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 4 client.8139 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 1 -- 172.21.15.60:0/2105915532 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55637f506c60 con 0x55637f777c80 2024-04-25T10:13:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f743e0e7640 1 --2- 172.21.15.60:0/2105915532 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55637f777c80 0x55637f77a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f743e0e7640 1 --2- 172.21.15.60:0/2105915532 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55637f777c80 0x55637f77a070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6f5d2050d8b037d server_cookie=cd663de1b13baadc in_seq=0 out_seq=0 2024-04-25T10:13:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74197fa640 10 client.8139 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:13:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74197fa640 1 -- 172.21.15.60:0/2105915532 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55637f506c60 con 0x55637f777c80 2024-04-25T10:13:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74197fa640 10 client.8139 handle_command_reply: tid=0 2024-04-25T10:13:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 1 client.8139 shutdown 2024-04-25T10:13:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 1 -- 172.21.15.60:0/2105915532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 msgr2=0x55637f74d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 1 --2- 172.21.15.60:0/2105915532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f74d270 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f743000f800 tx=0x7f743000fcc0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 1 -- 172.21.15.60:0/2105915532 shutdown_connections 2024-04-25T10:13:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 1 --2- 172.21.15.60:0/2105915532 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55637f777c80 0x55637f77a070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 1 --2- 172.21.15.60:0/2105915532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f74d270 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 1 --2- 172.21.15.60:0/2105915532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f7679d0 0x55637f74cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 1 --2- 172.21.15.60:0/2105915532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f71e0a0 0x55637f74c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74197fa640 0 client.8139 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:13:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 1 -- 172.21.15.60:0/2105915532 >> 172.21.15.60:0/2105915532 conn(0x55637f71a020 msgr2=0x55637f71cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 1 -- 172.21.15.60:0/2105915532 shutdown_connections 2024-04-25T10:13:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 1 -- 172.21.15.60:0/2105915532 wait complete. 2024-04-25T10:13:08.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.283+0000 7f74434b8000 20 client.8139 trim_cache size 0 max 16384 2024-04-25T10:13:08.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f74434b8000 1 Processor -- start 2024-04-25T10:13:08.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f74434b8000 1 -- start start 2024-04-25T10:13:08.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f74434b8000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f71e0a0 0x55637f7c4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f74434b8000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7679d0 0x55637f7c4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f74434b8000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f7682f0 0x55637f7c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f74434b8000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7438202910 con 0x55637f7679d0 2024-04-25T10:13:08.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f74434b8000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74381f73a0 con 0x55637f71e0a0 2024-04-25T10:13:08.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f74434b8000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74381fa760 con 0x55637f7682f0 2024-04-25T10:13:08.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f743d8e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7679d0 0x55637f7c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f743d8e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7679d0 0x55637f7c4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32978/0 (socket says 172.21.15.60:32978) 2024-04-25T10:13:08.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f743d8e6640 1 -- 172.21.15.60:0/744272826 learned_addr learned my addr 172.21.15.60:0/744272826 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:08.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f743e8e8640 1 --2- 172.21.15.60:0/744272826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f7682f0 0x55637f7c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f743e0e7640 1 --2- 172.21.15.60:0/744272826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f71e0a0 0x55637f7c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f743d8e6640 1 -- 172.21.15.60:0/744272826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f7682f0 msgr2=0x55637f7c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f743d8e6640 1 --2- 172.21.15.60:0/744272826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f7682f0 0x55637f7c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f743d8e6640 1 -- 172.21.15.60:0/744272826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f71e0a0 msgr2=0x55637f7c4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f743d8e6640 1 --2- 172.21.15.60:0/744272826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f71e0a0 0x55637f7c4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f743d8e6640 1 -- 172.21.15.60:0/744272826 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55637f5b1de0 con 0x55637f7679d0 2024-04-25T10:13:08.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f743e0e7640 1 --2- 172.21.15.60:0/744272826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f71e0a0 0x55637f7c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f743d8e6640 1 --2- 172.21.15.60:0/744272826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7679d0 0x55637f7c4840 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f743401c950 tx=0x7f74340b8070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e56e410fbee504 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f741b7fe640 1 -- 172.21.15.60:0/744272826 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74340bc040 con 0x55637f7679d0 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f741b7fe640 1 -- 172.21.15.60:0/744272826 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74340d0070 con 0x55637f7679d0 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.299+0000 7f741b7fe640 1 -- 172.21.15.60:0/744272826 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74340cc480 con 0x55637f7679d0 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 -- 172.21.15.60:0/744272826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7679d0 msgr2=0x55637f7c4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 --2- 172.21.15.60:0/744272826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7679d0 0x55637f7c4840 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f743401c950 tx=0x7f74340b8070 comp rx=0 tx=0).stop 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 -- 172.21.15.60:0/744272826 shutdown_connections 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 --2- 172.21.15.60:0/744272826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f7682f0 0x55637f7c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 --2- 172.21.15.60:0/744272826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7679d0 0x55637f7c4840 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 --2- 172.21.15.60:0/744272826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f71e0a0 0x55637f7c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 -- 172.21.15.60:0/744272826 >> 172.21.15.60:0/744272826 conn(0x55637f71a020 msgr2=0x55637f7660d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 -- 172.21.15.60:0/744272826 shutdown_connections 2024-04-25T10:13:08.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 -- 172.21.15.60:0/744272826 wait complete. 2024-04-25T10:13:08.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 -- start start 2024-04-25T10:13:08.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 Processor -- start 2024-04-25T10:13:08.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f71e0a0 0x55637f7c8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f7679d0 0x55637f7c8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f77a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637f505e00 con 0x55637f7682f0 2024-04-25T10:13:08.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74381f6da0 con 0x55637f7679d0 2024-04-25T10:13:08.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55637f56a300 con 0x55637f71e0a0 2024-04-25T10:13:08.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 10 client.0 _ll_register_callbacks cb 0x55637f6e9040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:08.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f743e8e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f77a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f743e8e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f77a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32990/0 (socket says 172.21.15.60:32990) 2024-04-25T10:13:08.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f743e8e8640 1 -- 172.21.15.60:0/4145130480 learned_addr learned my addr 172.21.15.60:0/4145130480 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:08.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f743d8e6640 1 --2- 172.21.15.60:0/4145130480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f7679d0 0x55637f7c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f743e0e7640 1 --2- 172.21.15.60:0/4145130480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f71e0a0 0x55637f7c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f743e8e8640 1 -- 172.21.15.60:0/4145130480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f71e0a0 msgr2=0x55637f7c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f743e8e8640 1 --2- 172.21.15.60:0/4145130480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f71e0a0 0x55637f7c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f743e8e8640 1 -- 172.21.15.60:0/4145130480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f7679d0 msgr2=0x55637f7c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f743e8e8640 1 --2- 172.21.15.60:0/4145130480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f7679d0 0x55637f7c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f743e8e8640 1 -- 172.21.15.60:0/4145130480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55637f50c8e0 con 0x55637f7682f0 2024-04-25T10:13:08.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f743e8e8640 1 --2- 172.21.15.60:0/4145130480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f77a7e0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f7430011010 tx=0x7f7430012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44a87d3a11039f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74197fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:13:08.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74197fa640 1 -- 172.21.15.60:0/4145130480 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430017070 con 0x55637f7682f0 2024-04-25T10:13:08.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74197fa640 1 -- 172.21.15.60:0/4145130480 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f74300074e0 con 0x55637f7682f0 2024-04-25T10:13:08.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 -- 172.21.15.60:0/4145130480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74340d0070 con 0x55637f7682f0 2024-04-25T10:13:08.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74197fa640 1 -- 172.21.15.60:0/4145130480 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7430035050 con 0x55637f7682f0 2024-04-25T10:13:08.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 10 client.8145 fetch_fsmap learned FSMap version 62 2024-04-25T10:13:08.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 -- 172.21.15.60:0/4145130480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55637f5c48f0 con 0x55637f7682f0 2024-04-25T10:13:08.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74197fa640 1 -- 172.21.15.60:0/4145130480 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74340d0070 con 0x55637f7682f0 2024-04-25T10:13:08.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74197fa640 1 -- 172.21.15.60:0/4145130480 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7430038020 con 0x55637f7682f0 2024-04-25T10:13:08.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 10 client.8145 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:13:08.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 10 client.8145 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:13:08.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 20 client.8145 populate_metadata read hostname 'smithi060' 2024-04-25T10:13:08.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 --2- 172.21.15.60:0/4145130480 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55637f77b460 0x55637f77b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 4 client.8145 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:13:08.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74434b8000 1 -- 172.21.15.60:0/4145130480 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55637f578530 con 0x55637f77b460 2024-04-25T10:13:08.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f743e0e7640 1 --2- 172.21.15.60:0/4145130480 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55637f77b460 0x55637f77b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f74197fa640 10 client.8145 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:13:08.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.303+0000 7f743e0e7640 1 --2- 172.21.15.60:0/4145130480 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55637f77b460 0x55637f77b870 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dfd190c2efb0387 server_cookie=183a4259d0cb9685 in_seq=0 out_seq=0 2024-04-25T10:13:08.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f74197fa640 1 -- 172.21.15.60:0/4145130480 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55637f578530 con 0x55637f77b460 2024-04-25T10:13:08.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f74197fa640 10 client.8145 handle_command_reply: tid=0 2024-04-25T10:13:08.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f74434b8000 1 client.8145 shutdown 2024-04-25T10:13:08.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f74434b8000 1 -- 172.21.15.60:0/4145130480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 msgr2=0x55637f77a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f74434b8000 1 --2- 172.21.15.60:0/4145130480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f77a7e0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f7430011010 tx=0x7f7430012000 comp rx=0 tx=0).stop 2024-04-25T10:13:08.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f74434b8000 1 -- 172.21.15.60:0/4145130480 shutdown_connections 2024-04-25T10:13:08.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f74434b8000 1 --2- 172.21.15.60:0/4145130480 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x55637f77b460 0x55637f77b870 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f74434b8000 1 --2- 172.21.15.60:0/4145130480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55637f7682f0 0x55637f77a7e0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f74434b8000 1 --2- 172.21.15.60:0/4145130480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55637f7679d0 0x55637f7c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f74197fa640 0 client.8145 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:13:08.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f74434b8000 1 --2- 172.21.15.60:0/4145130480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55637f71e0a0 0x55637f7c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f74434b8000 1 -- 172.21.15.60:0/4145130480 >> 172.21.15.60:0/4145130480 conn(0x55637f71a020 msgr2=0x55637f766960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f74434b8000 1 -- 172.21.15.60:0/4145130480 shutdown_connections 2024-04-25T10:13:08.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f74434b8000 1 -- 172.21.15.60:0/4145130480 wait complete. 2024-04-25T10:13:08.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f74434b8000 20 client.8145 trim_cache size 0 max 16384 2024-04-25T10:13:08.322 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:13:08.322 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T10:13:08.322 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T10:13:08.322 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:13:08.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f744034b640 1 -- 172.21.15.60:0/2534626394 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f741c066f90 msgr2=0x7f741c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f744034b640 1 --2- 172.21.15.60:0/2534626394 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f741c066f90 0x7f741c069450 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f74340af000 tx=0x7f74340a7000 comp rx=0 tx=0).stop 2024-04-25T10:13:08.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f744034b640 1 -- 172.21.15.60:0/2534626394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7438071c30 msgr2=0x7f74380ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f744034b640 1 --2- 172.21.15.60:0/2534626394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7438071c30 0x7f74380ac8a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f742800d800 tx=0x7f742800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f744034b640 1 -- 172.21.15.60:0/2534626394 shutdown_connections 2024-04-25T10:13:08.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f744034b640 1 --2- 172.21.15.60:0/2534626394 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f741c066f90 0x7f741c069450 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f744034b640 1 --2- 172.21.15.60:0/2534626394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f743815fa20 0x7f7438143500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f744034b640 1 --2- 172.21.15.60:0/2534626394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74380cbcf0 0x7f743813a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f744034b640 1 --2- 172.21.15.60:0/2534626394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7438071c30 0x7f74380ac8a0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f744034b640 1 -- 172.21.15.60:0/2534626394 >> 172.21.15.60:0/2534626394 conn(0x7f74380670b0 msgr2=0x7f743815d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f744034b640 1 -- 172.21.15.60:0/2534626394 shutdown_connections 2024-04-25T10:13:08.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.307+0000 7f744034b640 1 -- 172.21.15.60:0/2534626394 wait complete. 2024-04-25T10:13:08.332 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:13:08.333 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:08.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe156ec4640 1 Processor -- start 2024-04-25T10:13:08.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe156ec4640 1 -- start start 2024-04-25T10:13:08.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe156ec4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1500d0a50 0x7fe1500b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe156ec4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe1500ce2b0 0x7fe1500ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe156ec4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe15006d200 con 0x7fe15015deb0 2024-04-25T10:13:08.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe156ec4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1500cf2c0 con 0x7fe1500ce2b0 2024-04-25T10:13:08.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe156ec4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500cefe0 con 0x7fe1500d0a50 2024-04-25T10:13:08.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe155461640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe1500ce2b0 0x7fe1500ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe14ffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1500d0a50 0x7fe1500b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe155461640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe1500ce2b0 0x7fe1500ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56728/0 (socket says 172.21.15.60:56728) 2024-04-25T10:13:08.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe14ffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1500d0a50 0x7fe1500b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:49604/0 (socket says 172.21.15.60:49604) 2024-04-25T10:13:08.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe154c60640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe15015deb0 0x7fe1500d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:60738/0 (socket says 172.21.15.60:60738) 2024-04-25T10:13:08.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe155461640 1 -- 172.21.15.60:0/3452386538 learned_addr learned my addr 172.21.15.60:0/3452386538 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:08.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe14ffff640 1 -- 172.21.15.60:0/3452386538 >> v2:172.21.15.100:3300/0 conn(0x7fe1500ce2b0 msgr2=0x7fe1500ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe14ffff640 1 --2- 172.21.15.60:0/3452386538 >> v2:172.21.15.100:3300/0 conn(0x7fe1500ce2b0 0x7fe1500ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe14ffff640 1 -- 172.21.15.60:0/3452386538 >> v1:172.21.15.100:6789/0 conn(0x7fe15015deb0 legacy=0x7fe1500d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe14ffff640 1 -- 172.21.15.60:0/3452386538 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1500d1d70 con 0x7fe1500d0a50 2024-04-25T10:13:08.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe14ffff640 1 --2- 172.21.15.60:0/3452386538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1500d0a50 0x7fe1500b6c10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe1380097c0 tx=0x7fe13801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10e3bd736bc730dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe14f7fe640 1 -- 172.21.15.60:0/3452386538 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1380b2070 con 0x7fe1500d0a50 2024-04-25T10:13:08.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe14f7fe640 1 -- 172.21.15.60:0/3452386538 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1380ad640 con 0x7fe1500d0a50 2024-04-25T10:13:08.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.439+0000 7fe14f7fe640 1 -- 172.21.15.60:0/3452386538 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1380ada10 con 0x7fe1500d0a50 2024-04-25T10:13:08.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 -- 172.21.15.60:0/3452386538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1500d0a50 msgr2=0x7fe1500b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 --2- 172.21.15.60:0/3452386538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1500d0a50 0x7fe1500b6c10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe1380097c0 tx=0x7fe13801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 -- 172.21.15.60:0/3452386538 shutdown_connections 2024-04-25T10:13:08.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 --2- 172.21.15.60:0/3452386538 >> v2:172.21.15.100:3300/0 conn(0x7fe1500ce2b0 0x7fe1500ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 --2- 172.21.15.60:0/3452386538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1500d0a50 0x7fe1500b6c10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 -- 172.21.15.60:0/3452386538 >> 172.21.15.60:0/3452386538 conn(0x7fe15006c090 msgr2=0x7fe15006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 -- 172.21.15.60:0/3452386538 shutdown_connections 2024-04-25T10:13:08.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 -- 172.21.15.60:0/3452386538 wait complete. 2024-04-25T10:13:08.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 Processor -- start 2024-04-25T10:13:08.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 -- start start 2024-04-25T10:13:08.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1500ce2b0 0x7fe1500c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1500d0a50 0x7fe150152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe15015deb0 0x7fe150153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500d31e0 con 0x7fe1500ce2b0 2024-04-25T10:13:08.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500d3060 con 0x7fe1500d0a50 2024-04-25T10:13:08.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500d3360 con 0x7fe15015deb0 2024-04-25T10:13:08.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe154c60640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1500ce2b0 0x7fe1500c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe154c60640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1500ce2b0 0x7fe1500c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32994/0 (socket says 172.21.15.60:32994) 2024-04-25T10:13:08.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe154c60640 1 -- 172.21.15.60:0/639091497 learned_addr learned my addr 172.21.15.60:0/639091497 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:08.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe155461640 1 --2- 172.21.15.60:0/639091497 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe15015deb0 0x7fe150153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe14ffff640 1 --2- 172.21.15.60:0/639091497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1500d0a50 0x7fe150152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe154c60640 1 -- 172.21.15.60:0/639091497 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe15015deb0 msgr2=0x7fe150153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe154c60640 1 --2- 172.21.15.60:0/639091497 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe15015deb0 0x7fe150153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe154c60640 1 -- 172.21.15.60:0/639091497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1500d0a50 msgr2=0x7fe150152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe154c60640 1 --2- 172.21.15.60:0/639091497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1500d0a50 0x7fe150152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe154c60640 1 -- 172.21.15.60:0/639091497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1500b8e40 con 0x7fe1500ce2b0 2024-04-25T10:13:08.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe155461640 1 --2- 172.21.15.60:0/639091497 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe15015deb0 0x7fe150153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:08.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe154c60640 1 --2- 172.21.15.60:0/639091497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1500ce2b0 0x7fe1500c8c00 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fe14000c370 tx=0x7fe14000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe14d7fa640 1 -- 172.21.15.60:0/639091497 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1400074e0 con 0x7fe1500ce2b0 2024-04-25T10:13:08.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe14d7fa640 1 -- 172.21.15.60:0/639091497 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe140007680 con 0x7fe1500ce2b0 2024-04-25T10:13:08.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 -- 172.21.15.60:0/639091497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15014e8d0 con 0x7fe1500ce2b0 2024-04-25T10:13:08.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe14d7fa640 1 -- 172.21.15.60:0/639091497 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe140004a30 con 0x7fe1500ce2b0 2024-04-25T10:13:08.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.443+0000 7fe156ec4640 1 -- 172.21.15.60:0/639091497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15014ee90 con 0x7fe1500ce2b0 2024-04-25T10:13:08.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.447+0000 7fe156ec4640 1 -- 172.21.15.60:0/639091497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe150074770 con 0x7fe1500ce2b0 2024-04-25T10:13:08.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.447+0000 7fe14d7fa640 1 -- 172.21.15.60:0/639091497 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe14001a020 con 0x7fe1500ce2b0 2024-04-25T10:13:08.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.447+0000 7fe14d7fa640 1 --2- 172.21.15.60:0/639091497 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe130066f60 0x7fe130069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.447+0000 7fe14d7fa640 1 -- 172.21.15.60:0/639091497 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe140085050 con 0x7fe1500ce2b0 2024-04-25T10:13:08.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.447+0000 7fe14ffff640 1 --2- 172.21.15.60:0/639091497 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe130066f60 0x7fe130069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.447+0000 7fe14ffff640 1 --2- 172.21.15.60:0/639091497 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe130066f60 0x7fe130069420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe1380af000 tx=0x7fe138008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.455+0000 7fe14d7fa640 1 -- 172.21.15.60:0/639091497 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe140021070 con 0x7fe1500ce2b0 2024-04-25T10:13:08.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.643+0000 7fe156ec4640 1 -- 172.21.15.60:0/639091497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1500d17a0 con 0x7fe1500ce2b0 2024-04-25T10:13:08.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.647+0000 7fe14d7fa640 1 -- 172.21.15.60:0/639091497 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fe140055ff0 con 0x7fe1500ce2b0 2024-04-25T10:13:08.716 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:13:08.716 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:12:09.160298+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7908":{"gid":7908,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":6,"addr":"172.21.15.60:6835/2817879567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2817879567},{"type":"v1","addr":"172.21.15.60:6835","nonce":2817879567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:13:08.718 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T10:13:08.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.647+0000 7fe156ec4640 1 -- 172.21.15.60:0/639091497 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe130066f60 msgr2=0x7fe130069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.647+0000 7fe156ec4640 1 --2- 172.21.15.60:0/639091497 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe130066f60 0x7fe130069420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe1380af000 tx=0x7fe138008040 comp rx=0 tx=0).stop 2024-04-25T10:13:08.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.647+0000 7fe156ec4640 1 -- 172.21.15.60:0/639091497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1500ce2b0 msgr2=0x7fe1500c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.647+0000 7fe156ec4640 1 --2- 172.21.15.60:0/639091497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1500ce2b0 0x7fe1500c8c00 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fe14000c370 tx=0x7fe14000c830 comp rx=0 tx=0).stop 2024-04-25T10:13:08.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.647+0000 7fe156ec4640 1 -- 172.21.15.60:0/639091497 shutdown_connections 2024-04-25T10:13:08.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.647+0000 7fe156ec4640 1 --2- 172.21.15.60:0/639091497 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe130066f60 0x7fe130069420 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.647+0000 7fe156ec4640 1 --2- 172.21.15.60:0/639091497 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe15015deb0 0x7fe150153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.647+0000 7fe156ec4640 1 --2- 172.21.15.60:0/639091497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1500d0a50 0x7fe150152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.647+0000 7fe156ec4640 1 --2- 172.21.15.60:0/639091497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1500ce2b0 0x7fe1500c8c00 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.647+0000 7fe156ec4640 1 -- 172.21.15.60:0/639091497 >> 172.21.15.60:0/639091497 conn(0x7fe15006c090 msgr2=0x7fe1500c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.647+0000 7fe156ec4640 1 -- 172.21.15.60:0/639091497 shutdown_connections 2024-04-25T10:13:08.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.647+0000 7fe156ec4640 1 -- 172.21.15.60:0/639091497 wait complete. 2024-04-25T10:13:08.720 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T10:13:08.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.835+0000 7f13add08640 1 Processor -- start 2024-04-25T10:13:08.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.835+0000 7f13add08640 1 -- start start 2024-04-25T10:13:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f13a815dd30 0x7f13a815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13a80cd3d0 0x7f13a80cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a80cd890 con 0x7f13a806f310 2024-04-25T10:13:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a80ce4a0 con 0x7f13a815dd30 2024-04-25T10:13:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a80cdb70 con 0x7f13a80cd3d0 2024-04-25T10:13:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f13a806f310 0x7f13a806d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48030/0 (socket says 172.21.15.60:48030) 2024-04-25T10:13:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a7fff640 1 -- 172.21.15.60:0/222302337 learned_addr learned my addr 172.21.15.60:0/222302337 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:13:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f139ffff640 1 --2- 172.21.15.60:0/222302337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13a80cd3d0 0x7f13a80cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 -- 172.21.15.60:0/222302337 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4121826940 0 0) 0x7f13a80cd890 con 0x7f13a806f310 2024-04-25T10:13:08.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a77fe640 1 --2- 172.21.15.60:0/222302337 >> v2:172.21.15.100:3300/0 conn(0x7f13a815dd30 0x7f13a815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 -- 172.21.15.60:0/222302337 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1394003530 con 0x7f13a806f310 2024-04-25T10:13:08.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 -- 172.21.15.60:0/222302337 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 237739374 0 0) 0x7f1394003530 con 0x7f13a806f310 2024-04-25T10:13:08.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 -- 172.21.15.60:0/222302337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13a80cd3d0 msgr2=0x7f13a80cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 --2- 172.21.15.60:0/222302337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13a80cd3d0 0x7f13a80cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 -- 172.21.15.60:0/222302337 >> v2:172.21.15.100:3300/0 conn(0x7f13a815dd30 msgr2=0x7f13a815ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 --2- 172.21.15.60:0/222302337 >> v2:172.21.15.100:3300/0 conn(0x7f13a815dd30 0x7f13a815ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 -- 172.21.15.60:0/222302337 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a80cde50 con 0x7f13a806f310 2024-04-25T10:13:08.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 -- 172.21.15.60:0/222302337 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f13a0001a00 con 0x7f13a806f310 2024-04-25T10:13:08.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 -- 172.21.15.60:0/222302337 >> v1:172.21.15.60:6789/0 conn(0x7f13a806f310 legacy=0x7f13a806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 --2- 172.21.15.60:0/222302337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13a80cd3d0 0x7f13940039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 --2- 172.21.15.60:0/222302337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1394003f60 0x7f1394004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 --2- 172.21.15.60:0/222302337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1394004d50 0x7f13940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 -- 172.21.15.60:0/222302337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a80ce4a0 con 0x7f1394004d50 2024-04-25T10:13:08.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 -- 172.21.15.60:0/222302337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a80cdb70 con 0x7f1394003f60 2024-04-25T10:13:08.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 -- 172.21.15.60:0/222302337 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139400bb00 con 0x7f13a80cd3d0 2024-04-25T10:13:08.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a7fff640 1 --2- 172.21.15.60:0/222302337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1394004d50 0x7f13940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f139ffff640 1 --2- 172.21.15.60:0/222302337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1394003f60 0x7f1394004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a77fe640 1 --2- 172.21.15.60:0/222302337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13a80cd3d0 0x7f13940039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a7fff640 1 -- 172.21.15.60:0/222302337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13a80cd3d0 msgr2=0x7f13940039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a7fff640 1 --2- 172.21.15.60:0/222302337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13a80cd3d0 0x7f13940039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a7fff640 1 -- 172.21.15.60:0/222302337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1394003f60 msgr2=0x7f1394004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a7fff640 1 --2- 172.21.15.60:0/222302337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1394003f60 0x7f1394004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a7fff640 1 -- 172.21.15.60:0/222302337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1394003530 con 0x7f1394004d50 2024-04-25T10:13:08.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f139ffff640 1 --2- 172.21.15.60:0/222302337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1394003f60 0x7f1394004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:08.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a7fff640 1 --2- 172.21.15.60:0/222302337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1394004d50 0x7f13940051d0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f13a000a880 tx=0x7f13a001fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a45471deb6d420c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 -- 172.21.15.60:0/222302337 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a000a4a0 con 0x7f1394004d50 2024-04-25T10:13:08.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 -- 172.21.15.60:0/222302337 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a00ad020 con 0x7f1394004d50 2024-04-25T10:13:08.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a6ffd640 1 -- 172.21.15.60:0/222302337 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a00a9740 con 0x7f1394004d50 2024-04-25T10:13:08.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 -- 172.21.15.60:0/222302337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1394004d50 msgr2=0x7f13940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 --2- 172.21.15.60:0/222302337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1394004d50 0x7f13940051d0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f13a000a880 tx=0x7f13a001fb00 comp rx=0 tx=0).stop 2024-04-25T10:13:08.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a7fff640 1 -- 172.21.15.60:0/222302337 reap_dead start 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 -- 172.21.15.60:0/222302337 shutdown_connections 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 -- 172.21.15.60:0/222302337 >> 172.21.15.60:0/222302337 conn(0x7f13a80670b0 msgr2=0x7f13a806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 -- 172.21.15.60:0/222302337 shutdown_connections 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 -- 172.21.15.60:0/222302337 wait complete. 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 Processor -- start 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 -- start start 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13a8151820 0x7f13a8150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13a80cb0b0 0x7f13a8153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13a8154410 0x7f13a814ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a815cfb0 con 0x7f13a8151820 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a815ce30 con 0x7f13a80cb0b0 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a806d960 con 0x7f13a8154410 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13a8151820 0x7f13a8150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13a8151820 0x7f13a8150600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33012/0 (socket says 172.21.15.60:33012) 2024-04-25T10:13:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a77fe640 1 -- 172.21.15.60:0/481870395 learned_addr learned my addr 172.21.15.60:0/481870395 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a7fff640 1 --2- 172.21.15.60:0/481870395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13a8154410 0x7f13a814ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f139ffff640 1 --2- 172.21.15.60:0/481870395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13a80cb0b0 0x7f13a8153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a77fe640 1 -- 172.21.15.60:0/481870395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13a8154410 msgr2=0x7f13a814ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a77fe640 1 --2- 172.21.15.60:0/481870395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13a8154410 0x7f13a814ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a77fe640 1 -- 172.21.15.60:0/481870395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13a80cb0b0 msgr2=0x7f13a8153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a77fe640 1 --2- 172.21.15.60:0/481870395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13a80cb0b0 0x7f13a8153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a77fe640 1 -- 172.21.15.60:0/481870395 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a80c8a60 con 0x7f13a8151820 2024-04-25T10:13:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a77fe640 1 --2- 172.21.15.60:0/481870395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13a8151820 0x7f13a8150600 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f139800e840 tx=0x7f139800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a4ff9640 1 -- 172.21.15.60:0/481870395 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398017070 con 0x7f13a8151820 2024-04-25T10:13:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a4ff9640 1 -- 172.21.15.60:0/481870395 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139800c6d0 con 0x7f13a8151820 2024-04-25T10:13:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 -- 172.21.15.60:0/481870395 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a80c8d20 con 0x7f13a8151820 2024-04-25T10:13:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13a4ff9640 1 -- 172.21.15.60:0/481870395 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139800f040 con 0x7f13a8151820 2024-04-25T10:13:08.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.839+0000 7f13add08640 1 -- 172.21.15.60:0/481870395 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a80c92e0 con 0x7f13a8151820 2024-04-25T10:13:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13a4ff9640 1 -- 172.21.15.60:0/481870395 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f139801a070 con 0x7f13a8151820 2024-04-25T10:13:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13a4ff9640 1 --2- 172.21.15.60:0/481870395 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1388066f60 0x7f1388069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13a4ff9640 1 -- 172.21.15.60:0/481870395 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f1398084cb0 con 0x7f13a8151820 2024-04-25T10:13:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f139ffff640 1 --2- 172.21.15.60:0/481870395 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1388066f60 0x7f1388069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13b0e75000 1 Processor -- start 2024-04-25T10:13:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13b0e75000 1 -- start start 2024-04-25T10:13:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f139ffff640 1 --2- 172.21.15.60:0/481870395 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1388066f60 0x7f1388069420 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f13a80ca8c0 tx=0x7f1390008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13b0e75000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562421a122e0 0x562421a126c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13b0e75000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562421a12c00 0x5624219c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13b0e75000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5624219c88e0 0x5624219c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13b0e75000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624218afba0 con 0x562421a122e0 2024-04-25T10:13:08.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13b0e75000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562421815090 con 0x562421a12c00 2024-04-25T10:13:08.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13b0e75000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56242179d160 con 0x5624219c88e0 2024-04-25T10:13:08.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13a77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562421a122e0 0x562421a126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13a77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562421a122e0 0x562421a126c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33020/0 (socket says 172.21.15.60:33020) 2024-04-25T10:13:08.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13a77fe640 1 -- 172.21.15.60:0/3201344648 learned_addr learned my addr 172.21.15.60:0/3201344648 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:08.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f139ffff640 1 --2- 172.21.15.60:0/3201344648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562421a12c00 0x5624219c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13a7fff640 1 --2- 172.21.15.60:0/3201344648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5624219c88e0 0x5624219c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13a77fe640 1 -- 172.21.15.60:0/3201344648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5624219c88e0 msgr2=0x5624219c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13a77fe640 1 --2- 172.21.15.60:0/3201344648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5624219c88e0 0x5624219c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13a77fe640 1 -- 172.21.15.60:0/3201344648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562421a12c00 msgr2=0x5624219c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13a77fe640 1 --2- 172.21.15.60:0/3201344648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562421a12c00 0x5624219c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13a77fe640 1 -- 172.21.15.60:0/3201344648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624218bec20 con 0x562421a122e0 2024-04-25T10:13:08.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13a7fff640 1 --2- 172.21.15.60:0/3201344648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5624219c88e0 0x5624219c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:08.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f13a77fe640 1 --2- 172.21.15.60:0/3201344648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562421a122e0 0x562421a126c0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f139805d730 tx=0x7f13980551c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12dd1f9e9b50752 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f139d7fa640 1 -- 172.21.15.60:0/3201344648 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398021040 con 0x562421a122e0 2024-04-25T10:13:08.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f139d7fa640 1 -- 172.21.15.60:0/3201344648 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1398053850 con 0x562421a122e0 2024-04-25T10:13:08.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.843+0000 7f139d7fa640 1 -- 172.21.15.60:0/3201344648 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398059170 con 0x562421a122e0 2024-04-25T10:13:08.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 -- 172.21.15.60:0/3201344648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562421a122e0 msgr2=0x562421a126c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 --2- 172.21.15.60:0/3201344648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562421a122e0 0x562421a126c0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f139805d730 tx=0x7f13980551c0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 -- 172.21.15.60:0/3201344648 shutdown_connections 2024-04-25T10:13:08.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 --2- 172.21.15.60:0/3201344648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5624219c88e0 0x5624219c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 --2- 172.21.15.60:0/3201344648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562421a12c00 0x5624219c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 --2- 172.21.15.60:0/3201344648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562421a122e0 0x562421a126c0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 -- 172.21.15.60:0/3201344648 >> 172.21.15.60:0/3201344648 conn(0x5624219c4890 msgr2=0x5624219c6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 -- 172.21.15.60:0/3201344648 shutdown_connections 2024-04-25T10:13:08.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 -- 172.21.15.60:0/3201344648 wait complete. 2024-04-25T10:13:08.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 -- start start 2024-04-25T10:13:08.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 Processor -- start 2024-04-25T10:13:08.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624219c88e0 0x562421a19640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562421a122e0 0x562421a19b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562421a12c00 0x562421a00090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562421811130 con 0x562421a122e0 2024-04-25T10:13:08.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624217d9130 con 0x5624219c88e0 2024-04-25T10:13:08.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624218d3300 con 0x562421a12c00 2024-04-25T10:13:08.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 10 client.0 _ll_register_callbacks cb 0x5624219b3bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:08.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f139ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562421a122e0 0x562421a19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f139ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562421a122e0 0x562421a19b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33026/0 (socket says 172.21.15.60:33026) 2024-04-25T10:13:08.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f139ffff640 1 -- 172.21.15.60:0/1499318527 learned_addr learned my addr 172.21.15.60:0/1499318527 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:08.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13a77fe640 1 --2- 172.21.15.60:0/1499318527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624219c88e0 0x562421a19640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13a7fff640 1 --2- 172.21.15.60:0/1499318527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562421a12c00 0x562421a00090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f139ffff640 1 -- 172.21.15.60:0/1499318527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562421a12c00 msgr2=0x562421a00090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f139ffff640 1 --2- 172.21.15.60:0/1499318527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562421a12c00 0x562421a00090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f139ffff640 1 -- 172.21.15.60:0/1499318527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624219c88e0 msgr2=0x562421a19640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f139ffff640 1 --2- 172.21.15.60:0/1499318527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624219c88e0 0x562421a19640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f139ffff640 1 -- 172.21.15.60:0/1499318527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624217f2060 con 0x562421a122e0 2024-04-25T10:13:08.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f139ffff640 1 --2- 172.21.15.60:0/1499318527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562421a122e0 0x562421a19b80 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f1390006a80 tx=0x7f1390004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5428c0082e0cf73c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13867fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:13:08.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13867fc640 1 -- 172.21.15.60:0/1499318527 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390026070 con 0x562421a122e0 2024-04-25T10:13:08.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13867fc640 1 -- 172.21.15.60:0/1499318527 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f1390022620 con 0x562421a122e0 2024-04-25T10:13:08.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 -- 172.21.15.60:0/1499318527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1398053850 con 0x562421a122e0 2024-04-25T10:13:08.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13867fc640 1 -- 172.21.15.60:0/1499318527 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1390031050 con 0x562421a122e0 2024-04-25T10:13:08.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 10 client.8160 fetch_fsmap learned FSMap version 62 2024-04-25T10:13:08.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 -- 172.21.15.60:0/1499318527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x5624217b67c0 con 0x562421a122e0 2024-04-25T10:13:08.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13867fc640 1 -- 172.21.15.60:0/1499318527 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1398053850 con 0x562421a122e0 2024-04-25T10:13:08.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13867fc640 1 -- 172.21.15.60:0/1499318527 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1390034020 con 0x562421a122e0 2024-04-25T10:13:08.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 10 client.8160 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:13:08.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 10 client.8160 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:08.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 20 client.8160 populate_metadata read hostname 'smithi060' 2024-04-25T10:13:08.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 --2- 172.21.15.60:0/1499318527 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x562421a00e30 0x562421a01210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 4 client.8160 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:08.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13b0e75000 1 -- 172.21.15.60:0/1499318527 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624217b0cd0 con 0x562421a00e30 2024-04-25T10:13:08.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13a77fe640 1 --2- 172.21.15.60:0/1499318527 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x562421a00e30 0x562421a01210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13a77fe640 1 --2- 172.21.15.60:0/1499318527 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x562421a00e30 0x562421a01210 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b445dc9c5c18a0d8 server_cookie=9ef70277d9ac8d4a in_seq=0 out_seq=0 2024-04-25T10:13:08.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13867fc640 10 client.8160 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:13:08.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13867fc640 1 -- 172.21.15.60:0/1499318527 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5624217b0cd0 con 0x562421a00e30 2024-04-25T10:13:08.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.847+0000 7f13867fc640 10 client.8160 handle_command_reply: tid=0 2024-04-25T10:13:08.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.851+0000 7f13b0e75000 1 client.8160 shutdown 2024-04-25T10:13:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.851+0000 7f13b0e75000 1 -- 172.21.15.60:0/1499318527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562421a122e0 msgr2=0x562421a19b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.851+0000 7f13b0e75000 1 --2- 172.21.15.60:0/1499318527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562421a122e0 0x562421a19b80 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f1390006a80 tx=0x7f1390004740 comp rx=0 tx=0).stop 2024-04-25T10:13:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.851+0000 7f13b0e75000 1 -- 172.21.15.60:0/1499318527 shutdown_connections 2024-04-25T10:13:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.851+0000 7f13b0e75000 1 --2- 172.21.15.60:0/1499318527 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x562421a00e30 0x562421a01210 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.851+0000 7f13b0e75000 1 --2- 172.21.15.60:0/1499318527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562421a12c00 0x562421a00090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.851+0000 7f13b0e75000 1 --2- 172.21.15.60:0/1499318527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562421a122e0 0x562421a19b80 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.851+0000 7f13867fc640 0 client.8160 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:13:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.851+0000 7f13b0e75000 1 --2- 172.21.15.60:0/1499318527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624219c88e0 0x562421a19640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.851+0000 7f13b0e75000 1 -- 172.21.15.60:0/1499318527 >> 172.21.15.60:0/1499318527 conn(0x5624219c4890 msgr2=0x5624219c7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.851+0000 7f13b0e75000 1 -- 172.21.15.60:0/1499318527 shutdown_connections 2024-04-25T10:13:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.851+0000 7f13b0e75000 1 -- 172.21.15.60:0/1499318527 wait complete. 2024-04-25T10:13:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.851+0000 7f13b0e75000 20 client.8160 trim_cache size 0 max 16384 2024-04-25T10:13:08.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.859+0000 7f13b0e75000 1 Processor -- start 2024-04-25T10:13:08.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.859+0000 7f13b0e75000 1 -- start start 2024-04-25T10:13:08.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624219c88e0 0x562421a6e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562421a122e0 0x562421a6ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562421a12c00 0x562421a6ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a81ffbb0 con 0x5624219c88e0 2024-04-25T10:13:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a81fefe0 con 0x562421a122e0 2024-04-25T10:13:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a81ff5b0 con 0x562421a12c00 2024-04-25T10:13:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624219c88e0 0x562421a6e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624219c88e0 0x562421a6e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33034/0 (socket says 172.21.15.60:33034) 2024-04-25T10:13:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 -- 172.21.15.60:0/3959849116 learned_addr learned my addr 172.21.15.60:0/3959849116 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 -- 172.21.15.60:0/3959849116 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562421a12c00 msgr2=0x562421a6ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 --2- 172.21.15.60:0/3959849116 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562421a12c00 0x562421a6ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 -- 172.21.15.60:0/3959849116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562421a122e0 msgr2=0x562421a6ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 --2- 172.21.15.60:0/3959849116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562421a122e0 0x562421a6ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 -- 172.21.15.60:0/3959849116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624217d0fc0 con 0x5624219c88e0 2024-04-25T10:13:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 --2- 172.21.15.60:0/3959849116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624219c88e0 0x562421a6e500 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f1398084fd0 tx=0x7f139801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b47d0f342b07587 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f139cff9640 1 -- 172.21.15.60:0/3959849116 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398021040 con 0x5624219c88e0 2024-04-25T10:13:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f139cff9640 1 -- 172.21.15.60:0/3959849116 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1398051bc0 con 0x5624219c88e0 2024-04-25T10:13:08.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f139cff9640 1 -- 172.21.15.60:0/3959849116 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398059390 con 0x5624219c88e0 2024-04-25T10:13:08.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 -- 172.21.15.60:0/3959849116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624219c88e0 msgr2=0x562421a6e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 --2- 172.21.15.60:0/3959849116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624219c88e0 0x562421a6e500 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f1398084fd0 tx=0x7f139801e000 comp rx=0 tx=0).stop 2024-04-25T10:13:08.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 -- 172.21.15.60:0/3959849116 shutdown_connections 2024-04-25T10:13:08.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 --2- 172.21.15.60:0/3959849116 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562421a12c00 0x562421a6ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 --2- 172.21.15.60:0/3959849116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562421a122e0 0x562421a6ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 --2- 172.21.15.60:0/3959849116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624219c88e0 0x562421a6e500 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 -- 172.21.15.60:0/3959849116 >> 172.21.15.60:0/3959849116 conn(0x5624219c4890 msgr2=0x562421a11290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 -- 172.21.15.60:0/3959849116 shutdown_connections 2024-04-25T10:13:08.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 -- 172.21.15.60:0/3959849116 wait complete. 2024-04-25T10:13:08.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 -- start start 2024-04-25T10:13:08.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 Processor -- start 2024-04-25T10:13:08.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624219c88e0 0x562421a72720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562421a122e0 0x562421a72c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562421a12c00 0x562421a169b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624217afe70 con 0x5624219c88e0 2024-04-25T10:13:08.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242179da50 con 0x562421a122e0 2024-04-25T10:13:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624218143a0 con 0x562421a12c00 2024-04-25T10:13:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 10 client.0 _ll_register_callbacks cb 0x5624217c1240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624219c88e0 0x562421a72720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624219c88e0 0x562421a72720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33042/0 (socket says 172.21.15.60:33042) 2024-04-25T10:13:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 -- 172.21.15.60:0/4020264495 learned_addr learned my addr 172.21.15.60:0/4020264495 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 -- 172.21.15.60:0/4020264495 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562421a12c00 msgr2=0x562421a169b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 --2- 172.21.15.60:0/4020264495 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562421a12c00 0x562421a169b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 -- 172.21.15.60:0/4020264495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562421a122e0 msgr2=0x562421a72c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 --2- 172.21.15.60:0/4020264495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562421a122e0 0x562421a72c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 -- 172.21.15.60:0/4020264495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56242186e9e0 con 0x5624219c88e0 2024-04-25T10:13:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13a77fe640 1 --2- 172.21.15.60:0/4020264495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624219c88e0 0x562421a72720 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f13980519b0 tx=0x7f1398000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e9ebb6d53ed91b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13867fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:13:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13867fc640 1 -- 172.21.15.60:0/4020264495 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398021070 con 0x5624219c88e0 2024-04-25T10:13:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13867fc640 1 -- 172.21.15.60:0/4020264495 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f1398059370 con 0x5624219c88e0 2024-04-25T10:13:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 -- 172.21.15.60:0/4020264495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1398051bc0 con 0x5624219c88e0 2024-04-25T10:13:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13867fc640 1 -- 172.21.15.60:0/4020264495 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1398055090 con 0x5624219c88e0 2024-04-25T10:13:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 10 client.8166 fetch_fsmap learned FSMap version 62 2024-04-25T10:13:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13b0e75000 1 -- 172.21.15.60:0/4020264495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x5624217b4630 con 0x5624219c88e0 2024-04-25T10:13:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.863+0000 7f13867fc640 1 -- 172.21.15.60:0/4020264495 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1398051bc0 con 0x5624219c88e0 2024-04-25T10:13:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13867fc640 1 -- 172.21.15.60:0/4020264495 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1398012050 con 0x5624219c88e0 2024-04-25T10:13:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 10 client.8166 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:13:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 10 client.8166 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 20 client.8166 populate_metadata read hostname 'smithi060' 2024-04-25T10:13:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 1 --2- 172.21.15.60:0/4020264495 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x562421a17370 0x562421a17750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 4 client.8166 mds_command: new command op to 7908 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:13:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 1 -- 172.21.15.60:0/4020264495 --> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5624218225d0 con 0x562421a17370 2024-04-25T10:13:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f139ffff640 1 --2- 172.21.15.60:0/4020264495 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x562421a17370 0x562421a17750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13867fc640 10 client.8166 ms_handle_connect on v2:172.21.15.60:6833/2817879567 2024-04-25T10:13:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f139ffff640 1 --2- 172.21.15.60:0/4020264495 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x562421a17370 0x562421a17750 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe4ed462597ba2f7 server_cookie=e88a02f7d76d42b4 in_seq=0 out_seq=0 2024-04-25T10:13:08.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13867fc640 1 -- 172.21.15.60:0/4020264495 <== mds.0 v2:172.21.15.60:6833/2817879567 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+144 (crc 0 0 0) 0x5624218225d0 con 0x562421a17370 2024-04-25T10:13:08.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13867fc640 10 client.8166 handle_command_reply: tid=0 2024-04-25T10:13:08.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 1 client.8166 shutdown 2024-04-25T10:13:08.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 1 -- 172.21.15.60:0/4020264495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624219c88e0 msgr2=0x562421a72720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 1 --2- 172.21.15.60:0/4020264495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624219c88e0 0x562421a72720 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f13980519b0 tx=0x7f1398000bd0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 1 -- 172.21.15.60:0/4020264495 shutdown_connections 2024-04-25T10:13:08.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 1 --2- 172.21.15.60:0/4020264495 >> [v2:172.21.15.60:6833/2817879567,v1:172.21.15.60:6835/2817879567] conn(0x562421a17370 0x562421a17750 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 1 --2- 172.21.15.60:0/4020264495 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562421a12c00 0x562421a169b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13867fc640 0 client.8166 ms_handle_reset on v2:172.21.15.60:6833/2817879567 2024-04-25T10:13:08.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 1 --2- 172.21.15.60:0/4020264495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562421a122e0 0x562421a72c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 1 --2- 172.21.15.60:0/4020264495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5624219c88e0 0x562421a72720 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 1 -- 172.21.15.60:0/4020264495 >> 172.21.15.60:0/4020264495 conn(0x5624219c4890 msgr2=0x562421a11b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 1 -- 172.21.15.60:0/4020264495 shutdown_connections 2024-04-25T10:13:08.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 1 -- 172.21.15.60:0/4020264495 wait complete. 2024-04-25T10:13:08.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13b0e75000 20 client.8166 trim_cache size 0 max 16384 2024-04-25T10:13:08.882 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:13:08.882 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T10:13:08.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:13:08.882 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-04-25T10:13:08.882 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2204635862, 2024-04-25T10:13:08.882 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511627777, 2024-04-25T10:13:08.882 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/testdir/hardlink" 2024-04-25T10:13:08.882 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:13:08.883 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T10:13:08.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13add08640 1 -- 172.21.15.60:0/481870395 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1388066f60 msgr2=0x7f1388069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13add08640 1 --2- 172.21.15.60:0/481870395 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1388066f60 0x7f1388069420 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f13a80ca8c0 tx=0x7f1390008040 comp rx=0 tx=0).stop 2024-04-25T10:13:08.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13add08640 1 -- 172.21.15.60:0/481870395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13a8151820 msgr2=0x7f13a8150600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13add08640 1 --2- 172.21.15.60:0/481870395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13a8151820 0x7f13a8150600 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f139800e840 tx=0x7f139800ed00 comp rx=0 tx=0).stop 2024-04-25T10:13:08.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13add08640 1 -- 172.21.15.60:0/481870395 shutdown_connections 2024-04-25T10:13:08.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13add08640 1 --2- 172.21.15.60:0/481870395 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1388066f60 0x7f1388069420 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13add08640 1 --2- 172.21.15.60:0/481870395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13a8154410 0x7f13a814ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13add08640 1 --2- 172.21.15.60:0/481870395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13a80cb0b0 0x7f13a8153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13add08640 1 --2- 172.21.15.60:0/481870395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13a8151820 0x7f13a8150600 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13add08640 1 -- 172.21.15.60:0/481870395 >> 172.21.15.60:0/481870395 conn(0x7f13a80670b0 msgr2=0x7f13a815b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13add08640 1 -- 172.21.15.60:0/481870395 shutdown_connections 2024-04-25T10:13:08.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:08.867+0000 7f13add08640 1 -- 172.21.15.60:0/481870395 wait complete. 2024-04-25T10:13:08.891 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:13:08.891 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:08.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:09.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 Processor -- start 2024-04-25T10:13:09.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 -- start start 2024-04-25T10:13:09.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f291806ec30 0x7f29180cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29180b37e0 0x7f29180b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f291806f130 con 0x7f291815a7a0 2024-04-25T10:13:09.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29180cdaa0 con 0x7f291806ec30 2024-04-25T10:13:09.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291806d2f0 con 0x7f29180b37e0 2024-04-25T10:13:09.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f290ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f291806ec30 0x7f29180cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f290ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f291806ec30 0x7f29180cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59958/0 (socket says 172.21.15.60:59958) 2024-04-25T10:13:09.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f290ffff640 1 -- 172.21.15.60:0/4202689044 learned_addr learned my addr 172.21.15.60:0/4202689044 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:09.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f29177fe640 1 --2- 172.21.15.60:0/4202689044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29180b37e0 0x7f29180b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f290ffff640 1 -- 172.21.15.60:0/4202689044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29180b37e0 msgr2=0x7f29180b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f290ffff640 1 --2- 172.21.15.60:0/4202689044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29180b37e0 0x7f29180b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f290ffff640 1 -- 172.21.15.60:0/4202689044 >> v1:172.21.15.100:6789/0 conn(0x7f291815a7a0 legacy=0x7f2918071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f290ffff640 1 -- 172.21.15.60:0/4202689044 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291806f410 con 0x7f291806ec30 2024-04-25T10:13:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f290ffff640 1 --2- 172.21.15.60:0/4202689044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291806ec30 0x7f29180cd560 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f29000098a0 tx=0x7f290001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fada8f795a9c7912 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f29167fc640 1 -- 172.21.15.60:0/4202689044 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000b1070 con 0x7f291806ec30 2024-04-25T10:13:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f29167fc640 1 -- 172.21.15.60:0/4202689044 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29000ac630 con 0x7f291806ec30 2024-04-25T10:13:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f29167fc640 1 -- 172.21.15.60:0/4202689044 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000a8050 con 0x7f291806ec30 2024-04-25T10:13:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 -- 172.21.15.60:0/4202689044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291806ec30 msgr2=0x7f29180cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 --2- 172.21.15.60:0/4202689044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291806ec30 0x7f29180cd560 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f29000098a0 tx=0x7f290001c500 comp rx=0 tx=0).stop 2024-04-25T10:13:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 -- 172.21.15.60:0/4202689044 shutdown_connections 2024-04-25T10:13:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 --2- 172.21.15.60:0/4202689044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291806ec30 0x7f29180cd560 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 --2- 172.21.15.60:0/4202689044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29180b37e0 0x7f29180b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 -- 172.21.15.60:0/4202689044 >> 172.21.15.60:0/4202689044 conn(0x7f29180b12d0 msgr2=0x7f29180b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 -- 172.21.15.60:0/4202689044 shutdown_connections 2024-04-25T10:13:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 -- 172.21.15.60:0/4202689044 wait complete. 2024-04-25T10:13:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 Processor -- start 2024-04-25T10:13:09.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 -- start start 2024-04-25T10:13:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.051+0000 7f291d95c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f291806ec30 0x7f2918160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f291d95c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29180b37e0 0x7f2918141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f291d95c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291815a7a0 0x7f2918157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f291d95c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291806e8e0 con 0x7f291815a7a0 2024-04-25T10:13:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f291d95c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291806e760 con 0x7f291806ec30 2024-04-25T10:13:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f291d95c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291806d8b0 con 0x7f29180b37e0 2024-04-25T10:13:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f29177fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291815a7a0 0x7f2918157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f29177fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291815a7a0 0x7f2918157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59968/0 (socket says 172.21.15.60:59968) 2024-04-25T10:13:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f29177fe640 1 -- 172.21.15.60:0/3957976316 learned_addr learned my addr 172.21.15.60:0/3957976316 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f29177fe640 1 -- 172.21.15.60:0/3957976316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29180b37e0 msgr2=0x7f2918141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f290ffff640 1 --2- 172.21.15.60:0/3957976316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29180b37e0 0x7f2918141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f29177fe640 1 --2- 172.21.15.60:0/3957976316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29180b37e0 0x7f2918141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f29177fe640 1 -- 172.21.15.60:0/3957976316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f291806ec30 msgr2=0x7f2918160810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f29177fe640 1 --2- 172.21.15.60:0/3957976316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f291806ec30 0x7f2918160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f29177fe640 1 -- 172.21.15.60:0/3957976316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29181422e0 con 0x7f291815a7a0 2024-04-25T10:13:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f290ffff640 1 --2- 172.21.15.60:0/3957976316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29180b37e0 0x7f2918141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:09.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f29177fe640 1 --2- 172.21.15.60:0/3957976316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291815a7a0 0x7f2918157f30 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f2910007e70 tx=0x7f291000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f2914ff9640 1 -- 172.21.15.60:0/3957976316 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910014620 con 0x7f291815a7a0 2024-04-25T10:13:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f2914ff9640 1 -- 172.21.15.60:0/3957976316 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2910019070 con 0x7f291815a7a0 2024-04-25T10:13:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f291d95c640 1 -- 172.21.15.60:0/3957976316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291815f490 con 0x7f291815a7a0 2024-04-25T10:13:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f2914ff9640 1 -- 172.21.15.60:0/3957976316 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910014a20 con 0x7f291815a7a0 2024-04-25T10:13:09.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f291d95c640 1 -- 172.21.15.60:0/3957976316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291815fa50 con 0x7f291815a7a0 2024-04-25T10:13:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f291d95c640 1 -- 172.21.15.60:0/3957976316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2918074770 con 0x7f291815a7a0 2024-04-25T10:13:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f2914ff9640 1 -- 172.21.15.60:0/3957976316 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2910014bc0 con 0x7f291815a7a0 2024-04-25T10:13:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f2914ff9640 1 --2- 172.21.15.60:0/3957976316 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f28f8066f90 0x7f28f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f2914ff9640 1 -- 172.21.15.60:0/3957976316 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f29100868f0 con 0x7f291815a7a0 2024-04-25T10:13:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f2916ffd640 1 --2- 172.21.15.60:0/3957976316 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f28f8066f90 0x7f28f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f2916ffd640 1 --2- 172.21.15.60:0/3957976316 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f28f8066f90 0x7f28f8069450 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2908009670 tx=0x7f2908006bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.059+0000 7f2914ff9640 1 -- 172.21.15.60:0/3957976316 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2910057800 con 0x7f291815a7a0 2024-04-25T10:13:09.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.219+0000 7f291d95c640 1 -- 172.21.15.60:0/3957976316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f291815ea80 con 0x7f291815a7a0 2024-04-25T10:13:09.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.219+0000 7f2914ff9640 1 -- 172.21.15.60:0/3957976316 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16851 (secure 0 0 0) 0x7f2910057100 con 0x7f291815a7a0 2024-04-25T10:13:09.230 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:13:09.230 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:12:03.323694+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:42.621502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:43.352679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/2455562778":"2024-04-26T10:11:58.136789+0000","172.21.15.60:6833/2455562778":"2024-04-26T10:11:58.136789+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:09.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.223+0000 7f291d95c640 1 -- 172.21.15.60:0/3957976316 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f28f8066f90 msgr2=0x7f28f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.223+0000 7f291d95c640 1 --2- 172.21.15.60:0/3957976316 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f28f8066f90 0x7f28f8069450 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2908009670 tx=0x7f2908006bd0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.223+0000 7f291d95c640 1 -- 172.21.15.60:0/3957976316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291815a7a0 msgr2=0x7f2918157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.223+0000 7f291d95c640 1 --2- 172.21.15.60:0/3957976316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291815a7a0 0x7f2918157f30 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f2910007e70 tx=0x7f291000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.223+0000 7f291d95c640 1 -- 172.21.15.60:0/3957976316 shutdown_connections 2024-04-25T10:13:09.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.223+0000 7f291d95c640 1 --2- 172.21.15.60:0/3957976316 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f28f8066f90 0x7f28f8069450 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.223+0000 7f291d95c640 1 --2- 172.21.15.60:0/3957976316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f291815a7a0 0x7f2918157f30 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.223+0000 7f291d95c640 1 --2- 172.21.15.60:0/3957976316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29180b37e0 0x7f2918141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.223+0000 7f291d95c640 1 --2- 172.21.15.60:0/3957976316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f291806ec30 0x7f2918160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.223+0000 7f291d95c640 1 -- 172.21.15.60:0/3957976316 >> 172.21.15.60:0/3957976316 conn(0x7f29180b12d0 msgr2=0x7f29180cdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.223+0000 7f291d95c640 1 -- 172.21.15.60:0/3957976316 shutdown_connections 2024-04-25T10:13:09.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.223+0000 7f291d95c640 1 -- 172.21.15.60:0/3957976316 wait complete. 2024-04-25T10:13:09.248 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:42.621502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:43.352679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:09.248 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:09.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01aa358640 1 Processor -- start 2024-04-25T10:13:09.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01aa358640 1 -- start start 2024-04-25T10:13:09.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01aa358640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f01a415b6f0 0x7f01a41620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01aa358640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01a40ceef0 0x7f01a415e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01aa358640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a415bad0 con 0x7f01a415e0d0 2024-04-25T10:13:09.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01aa358640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a4162610 con 0x7f01a415b6f0 2024-04-25T10:13:09.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01aa358640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a4161c40 con 0x7f01a40ceef0 2024-04-25T10:13:09.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f01a415e0d0 0x7f01a40ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58490/0 (socket says 172.21.15.60:58490) 2024-04-25T10:13:09.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a37fe640 1 -- 172.21.15.60:0/3895655539 learned_addr learned my addr 172.21.15.60:0/3895655539 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:13:09.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a88f5640 1 --2- 172.21.15.60:0/3895655539 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01a40ceef0 0x7f01a415e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a3fff640 1 --2- 172.21.15.60:0/3895655539 >> v2:172.21.15.100:3300/0 conn(0x7f01a415b6f0 0x7f01a41620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a2ffd640 1 -- 172.21.15.60:0/3895655539 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254274265 0 0) 0x7f01a415bad0 con 0x7f01a415e0d0 2024-04-25T10:13:09.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a2ffd640 1 -- 172.21.15.60:0/3895655539 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f018c003530 con 0x7f01a415e0d0 2024-04-25T10:13:09.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a2ffd640 1 -- 172.21.15.60:0/3895655539 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2641810998 0 0) 0x7f018c003530 con 0x7f01a415e0d0 2024-04-25T10:13:09.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a2ffd640 1 -- 172.21.15.60:0/3895655539 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01a40ceef0 msgr2=0x7f01a415e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a2ffd640 1 --2- 172.21.15.60:0/3895655539 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01a40ceef0 0x7f01a415e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a2ffd640 1 -- 172.21.15.60:0/3895655539 >> v2:172.21.15.100:3300/0 conn(0x7f01a415b6f0 msgr2=0x7f01a41620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a2ffd640 1 --2- 172.21.15.60:0/3895655539 >> v2:172.21.15.100:3300/0 conn(0x7f01a415b6f0 0x7f01a41620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a2ffd640 1 -- 172.21.15.60:0/3895655539 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a415ebe0 con 0x7f01a415e0d0 2024-04-25T10:13:09.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a2ffd640 1 -- 172.21.15.60:0/3895655539 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f0198001680 con 0x7f01a415e0d0 2024-04-25T10:13:09.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a2ffd640 1 -- 172.21.15.60:0/3895655539 >> v1:172.21.15.60:6789/0 conn(0x7f01a415e0d0 legacy=0x7f01a40ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a2ffd640 1 --2- 172.21.15.60:0/3895655539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f018c0039c0 0x7f018c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a2ffd640 1 --2- 172.21.15.60:0/3895655539 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01a40ceef0 0x7f018c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a2ffd640 1 --2- 172.21.15.60:0/3895655539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f018c004d50 0x7f018c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a2ffd640 1 -- 172.21.15.60:0/3895655539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a4162610 con 0x7f018c004d50 2024-04-25T10:13:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a2ffd640 1 -- 172.21.15.60:0/3895655539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a4161c40 con 0x7f018c0039c0 2024-04-25T10:13:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a2ffd640 1 -- 172.21.15.60:0/3895655539 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018c00b9d0 con 0x7f01a40ceef0 2024-04-25T10:13:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a88f5640 1 --2- 172.21.15.60:0/3895655539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f018c004d50 0x7f018c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a3fff640 1 --2- 172.21.15.60:0/3895655539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f018c0039c0 0x7f018c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a37fe640 1 --2- 172.21.15.60:0/3895655539 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01a40ceef0 0x7f018c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a88f5640 1 -- 172.21.15.60:0/3895655539 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01a40ceef0 msgr2=0x7f018c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a88f5640 1 --2- 172.21.15.60:0/3895655539 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01a40ceef0 0x7f018c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a88f5640 1 -- 172.21.15.60:0/3895655539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f018c0039c0 msgr2=0x7f018c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a88f5640 1 --2- 172.21.15.60:0/3895655539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f018c0039c0 0x7f018c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.367+0000 7f01a88f5640 1 -- 172.21.15.60:0/3895655539 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018c003530 con 0x7f018c004d50 2024-04-25T10:13:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.371+0000 7f01a88f5640 1 --2- 172.21.15.60:0/3895655539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f018c004d50 0x7f018c0051d0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f019c009f90 tx=0x7f019c01cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332ba516a018f3ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.371+0000 7f01a2ffd640 1 -- 172.21.15.60:0/3895655539 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c0b2070 con 0x7f018c004d50 2024-04-25T10:13:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.371+0000 7f01a2ffd640 1 -- 172.21.15.60:0/3895655539 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c0a5c10 con 0x7f018c004d50 2024-04-25T10:13:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.371+0000 7f01a2ffd640 1 -- 172.21.15.60:0/3895655539 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c009c10 con 0x7f018c004d50 2024-04-25T10:13:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.371+0000 7f01aa358640 1 -- 172.21.15.60:0/3895655539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f018c004d50 msgr2=0x7f018c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.371+0000 7f01aa358640 1 --2- 172.21.15.60:0/3895655539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f018c004d50 0x7f018c0051d0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f019c009f90 tx=0x7f019c01cc00 comp rx=0 tx=0).stop 2024-04-25T10:13:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.371+0000 7f01a88f5640 1 -- 172.21.15.60:0/3895655539 reap_dead start 2024-04-25T10:13:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.371+0000 7f01aa358640 1 -- 172.21.15.60:0/3895655539 shutdown_connections 2024-04-25T10:13:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.371+0000 7f01aa358640 1 -- 172.21.15.60:0/3895655539 >> 172.21.15.60:0/3895655539 conn(0x7f01a4072340 msgr2=0x7f01a4072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.371+0000 7f01aa358640 1 -- 172.21.15.60:0/3895655539 shutdown_connections 2024-04-25T10:13:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.371+0000 7f01aa358640 1 -- 172.21.15.60:0/3895655539 wait complete. 2024-04-25T10:13:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.371+0000 7f01aa358640 1 Processor -- start 2024-04-25T10:13:09.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.371+0000 7f01aa358640 1 -- start start 2024-04-25T10:13:09.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01aa358640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01a4152bd0 0x7f01a40cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01aa358640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01a40d3100 0x7f01a40c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01aa358640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01a40c8ef0 0x7f01a40d1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01aa358640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a40b9550 con 0x7f01a4152bd0 2024-04-25T10:13:09.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01aa358640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a41618d0 con 0x7f01a40d3100 2024-04-25T10:13:09.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01aa358640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a40b96d0 con 0x7f01a40c8ef0 2024-04-25T10:13:09.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01a4152bd0 0x7f01a40cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01a4152bd0 0x7f01a40cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59994/0 (socket says 172.21.15.60:59994) 2024-04-25T10:13:09.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a3fff640 1 -- 172.21.15.60:0/3617521120 learned_addr learned my addr 172.21.15.60:0/3617521120 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:09.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a37fe640 1 --2- 172.21.15.60:0/3617521120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01a40d3100 0x7f01a40c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a88f5640 1 --2- 172.21.15.60:0/3617521120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01a40c8ef0 0x7f01a40d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a3fff640 1 -- 172.21.15.60:0/3617521120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01a40c8ef0 msgr2=0x7f01a40d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a3fff640 1 --2- 172.21.15.60:0/3617521120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01a40c8ef0 0x7f01a40d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a3fff640 1 -- 172.21.15.60:0/3617521120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01a40d3100 msgr2=0x7f01a40c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a3fff640 1 --2- 172.21.15.60:0/3617521120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01a40d3100 0x7f01a40c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a3fff640 1 -- 172.21.15.60:0/3617521120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a40d3580 con 0x7f01a4152bd0 2024-04-25T10:13:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a37fe640 1 --2- 172.21.15.60:0/3617521120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01a40d3100 0x7f01a40c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a3fff640 1 --2- 172.21.15.60:0/3617521120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01a4152bd0 0x7f01a40cc110 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f0194011610 tx=0x7f0194011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a0ff9640 1 -- 172.21.15.60:0/3617521120 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019400f8f0 con 0x7f01a4152bd0 2024-04-25T10:13:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a0ff9640 1 -- 172.21.15.60:0/3617521120 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019400fa90 con 0x7f01a4152bd0 2024-04-25T10:13:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01aa358640 1 -- 172.21.15.60:0/3617521120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a40d3850 con 0x7f01a4152bd0 2024-04-25T10:13:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a0ff9640 1 -- 172.21.15.60:0/3617521120 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019400fe40 con 0x7f01a4152bd0 2024-04-25T10:13:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01aa358640 1 -- 172.21.15.60:0/3617521120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a40d3e10 con 0x7f01a4152bd0 2024-04-25T10:13:09.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01aa358640 1 -- 172.21.15.60:0/3617521120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a4074770 con 0x7f01a4152bd0 2024-04-25T10:13:09.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a0ff9640 1 -- 172.21.15.60:0/3617521120 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0194015460 con 0x7f01a4152bd0 2024-04-25T10:13:09.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a0ff9640 1 --2- 172.21.15.60:0/3617521120 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0190066f90 0x7f0190069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a0ff9640 1 -- 172.21.15.60:0/3617521120 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f0194085ac0 con 0x7f01a4152bd0 2024-04-25T10:13:09.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a37fe640 1 --2- 172.21.15.60:0/3617521120 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0190066f90 0x7f0190069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a37fe640 1 --2- 172.21.15.60:0/3617521120 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0190066f90 0x7f0190069450 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f01980017f0 tx=0x7f0198001ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.379+0000 7f01a0ff9640 1 -- 172.21.15.60:0/3617521120 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0194019030 con 0x7f01a4152bd0 2024-04-25T10:13:09.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.535+0000 7f01aa358640 1 -- 172.21.15.60:0/3617521120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01a40c92d0 con 0x7f01a4152bd0 2024-04-25T10:13:09.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.535+0000 7f01a0ff9640 1 -- 172.21.15.60:0/3617521120 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0194056a60 con 0x7f01a4152bd0 2024-04-25T10:13:09.546 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:13:09.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.539+0000 7f01aa358640 1 -- 172.21.15.60:0/3617521120 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0190066f90 msgr2=0x7f0190069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.539+0000 7f01aa358640 1 --2- 172.21.15.60:0/3617521120 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0190066f90 0x7f0190069450 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f01980017f0 tx=0x7f0198001ca0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.539+0000 7f01aa358640 1 -- 172.21.15.60:0/3617521120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01a4152bd0 msgr2=0x7f01a40cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.539+0000 7f01aa358640 1 --2- 172.21.15.60:0/3617521120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01a4152bd0 0x7f01a40cc110 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f0194011610 tx=0x7f0194011ad0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.539+0000 7f01aa358640 1 -- 172.21.15.60:0/3617521120 shutdown_connections 2024-04-25T10:13:09.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.539+0000 7f01aa358640 1 --2- 172.21.15.60:0/3617521120 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0190066f90 0x7f0190069450 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.539+0000 7f01aa358640 1 --2- 172.21.15.60:0/3617521120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01a40c8ef0 0x7f01a40d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.539+0000 7f01aa358640 1 --2- 172.21.15.60:0/3617521120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01a40d3100 0x7f01a40c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.539+0000 7f01aa358640 1 --2- 172.21.15.60:0/3617521120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01a4152bd0 0x7f01a40cc110 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.539+0000 7f01aa358640 1 -- 172.21.15.60:0/3617521120 >> 172.21.15.60:0/3617521120 conn(0x7f01a4072340 msgr2=0x7f01a40ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.539+0000 7f01aa358640 1 -- 172.21.15.60:0/3617521120 shutdown_connections 2024-04-25T10:13:09.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.539+0000 7f01aa358640 1 -- 172.21.15.60:0/3617521120 wait complete. 2024-04-25T10:13:09.564 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:09.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.679+0000 7f20277ee640 1 Processor -- start 2024-04-25T10:13:09.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f20277ee640 1 -- start start 2024-04-25T10:13:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f20277ee640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f202015ef20 0x7f20200b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f20277ee640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f20200cfcf0 0x7f20200b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f20277ee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2020099a30 0x7f2020099ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f20277ee640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20200cc320 con 0x7f202015ef20 2024-04-25T10:13:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f20277ee640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20200cc5c0 con 0x7f20200cfcf0 2024-04-25T10:13:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f20277ee640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20200d07b0 con 0x7f2020099a30 2024-04-25T10:13:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f202558a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f202015ef20 0x7f20200b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f202558a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f202015ef20 0x7f20200b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59998/0 (socket says 172.21.15.60:59998) 2024-04-25T10:13:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f202558a640 1 -- 172.21.15.60:0/1259573673 learned_addr learned my addr 172.21.15.60:0/1259573673 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:09.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f2025d8b640 1 --2- 172.21.15.60:0/1259573673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2020099a30 0x7f2020099ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f2024d89640 1 --2- 172.21.15.60:0/1259573673 >> v2:172.21.15.100:3300/0 conn(0x7f20200cfcf0 0x7f20200b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f202558a640 1 -- 172.21.15.60:0/1259573673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2020099a30 msgr2=0x7f2020099ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f202558a640 1 --2- 172.21.15.60:0/1259573673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2020099a30 0x7f2020099ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f202558a640 1 -- 172.21.15.60:0/1259573673 >> v2:172.21.15.100:3300/0 conn(0x7f20200cfcf0 msgr2=0x7f20200b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f202558a640 1 --2- 172.21.15.60:0/1259573673 >> v2:172.21.15.100:3300/0 conn(0x7f20200cfcf0 0x7f20200b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.683+0000 7f202558a640 1 -- 172.21.15.60:0/1259573673 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20200d04d0 con 0x7f202015ef20 2024-04-25T10:13:09.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f202558a640 1 --2- 172.21.15.60:0/1259573673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f202015ef20 0x7f20200b2560 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f202006b860 tx=0x7f201c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a0799ce8a0404f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f2017fff640 1 -- 172.21.15.60:0/1259573673 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c0b0070 con 0x7f202015ef20 2024-04-25T10:13:09.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f2017fff640 1 -- 172.21.15.60:0/1259573673 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201c0ab600 con 0x7f202015ef20 2024-04-25T10:13:09.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f2017fff640 1 -- 172.21.15.60:0/1259573673 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c0ab9d0 con 0x7f202015ef20 2024-04-25T10:13:09.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 -- 172.21.15.60:0/1259573673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f202015ef20 msgr2=0x7f20200b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 --2- 172.21.15.60:0/1259573673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f202015ef20 0x7f20200b2560 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f202006b860 tx=0x7f201c01c520 comp rx=0 tx=0).stop 2024-04-25T10:13:09.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 -- 172.21.15.60:0/1259573673 shutdown_connections 2024-04-25T10:13:09.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 --2- 172.21.15.60:0/1259573673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2020099a30 0x7f2020099ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 --2- 172.21.15.60:0/1259573673 >> v2:172.21.15.100:3300/0 conn(0x7f20200cfcf0 0x7f20200b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 --2- 172.21.15.60:0/1259573673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f202015ef20 0x7f20200b2560 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 -- 172.21.15.60:0/1259573673 >> 172.21.15.60:0/1259573673 conn(0x7f20200b3200 msgr2=0x7f20200b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 -- 172.21.15.60:0/1259573673 shutdown_connections 2024-04-25T10:13:09.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 -- 172.21.15.60:0/1259573673 wait complete. 2024-04-25T10:13:09.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 Processor -- start 2024-04-25T10:13:09.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 -- start start 2024-04-25T10:13:09.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2020099a30 0x7f2020158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20200cfcf0 0x7f20200c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f202015ef20 0x7f202015df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20200d0250 con 0x7f20200cfcf0 2024-04-25T10:13:09.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20200d00d0 con 0x7f202015ef20 2024-04-25T10:13:09.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f20277ee640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202015f300 con 0x7f2020099a30 2024-04-25T10:13:09.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f2024d89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20200cfcf0 0x7f20200c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f2024d89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20200cfcf0 0x7f20200c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60010/0 (socket says 172.21.15.60:60010) 2024-04-25T10:13:09.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f2024d89640 1 -- 172.21.15.60:0/938604793 learned_addr learned my addr 172.21.15.60:0/938604793 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f202558a640 1 --2- 172.21.15.60:0/938604793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2020099a30 0x7f2020158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f2025d8b640 1 --2- 172.21.15.60:0/938604793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f202015ef20 0x7f202015df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f2024d89640 1 -- 172.21.15.60:0/938604793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2020099a30 msgr2=0x7f2020158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f2024d89640 1 --2- 172.21.15.60:0/938604793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2020099a30 0x7f2020158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f2024d89640 1 -- 172.21.15.60:0/938604793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f202015ef20 msgr2=0x7f202015df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f2024d89640 1 --2- 172.21.15.60:0/938604793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f202015ef20 0x7f202015df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f2024d89640 1 -- 172.21.15.60:0/938604793 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2020157ab0 con 0x7f20200cfcf0 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.687+0000 7f2024d89640 1 --2- 172.21.15.60:0/938604793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20200cfcf0 0x7f20200c7c10 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f201000d800 tx=0x7f201000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.691+0000 7f2015ffb640 1 -- 172.21.15.60:0/938604793 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201000b6c0 con 0x7f20200cfcf0 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.691+0000 7f2015ffb640 1 -- 172.21.15.60:0/938604793 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2010019070 con 0x7f20200cfcf0 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.691+0000 7f20277ee640 1 -- 172.21.15.60:0/938604793 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2020157d90 con 0x7f20200cfcf0 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.691+0000 7f2015ffb640 1 -- 172.21.15.60:0/938604793 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201000ba60 con 0x7f20200cfcf0 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.691+0000 7f20277ee640 1 -- 172.21.15.60:0/938604793 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2020158350 con 0x7f20200cfcf0 2024-04-25T10:13:09.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.691+0000 7f20277ee640 1 -- 172.21.15.60:0/938604793 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2020074770 con 0x7f20200cfcf0 2024-04-25T10:13:09.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.691+0000 7f2015ffb640 1 -- 172.21.15.60:0/938604793 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f201000bc00 con 0x7f20200cfcf0 2024-04-25T10:13:09.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.691+0000 7f2015ffb640 1 --2- 172.21.15.60:0/938604793 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2004066f60 0x7f2004069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.691+0000 7f2015ffb640 1 -- 172.21.15.60:0/938604793 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f20100854b0 con 0x7f20200cfcf0 2024-04-25T10:13:09.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.691+0000 7f202558a640 1 --2- 172.21.15.60:0/938604793 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2004066f60 0x7f2004069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.691+0000 7f202558a640 1 --2- 172.21.15.60:0/938604793 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2004066f60 0x7f2004069420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f201c0ad000 tx=0x7f201c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.695+0000 7f2015ffb640 1 -- 172.21.15.60:0/938604793 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20100564e0 con 0x7f20200cfcf0 2024-04-25T10:13:09.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.851+0000 7f20277ee640 1 -- 172.21.15.60:0/938604793 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20200b4070 con 0x7f20200cfcf0 2024-04-25T10:13:09.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.851+0000 7f2015ffb640 1 -- 172.21.15.60:0/938604793 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2010055de0 con 0x7f20200cfcf0 2024-04-25T10:13:09.860 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:13:09.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.859+0000 7f20277ee640 1 -- 172.21.15.60:0/938604793 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2004066f60 msgr2=0x7f2004069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.859+0000 7f20277ee640 1 --2- 172.21.15.60:0/938604793 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2004066f60 0x7f2004069420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f201c0ad000 tx=0x7f201c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:09.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.859+0000 7f20277ee640 1 -- 172.21.15.60:0/938604793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20200cfcf0 msgr2=0x7f20200c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.859+0000 7f20277ee640 1 --2- 172.21.15.60:0/938604793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20200cfcf0 0x7f20200c7c10 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f201000d800 tx=0x7f201000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.859+0000 7f20277ee640 1 -- 172.21.15.60:0/938604793 shutdown_connections 2024-04-25T10:13:09.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.859+0000 7f20277ee640 1 --2- 172.21.15.60:0/938604793 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2004066f60 0x7f2004069420 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.859+0000 7f20277ee640 1 --2- 172.21.15.60:0/938604793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f202015ef20 0x7f202015df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.859+0000 7f20277ee640 1 --2- 172.21.15.60:0/938604793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20200cfcf0 0x7f20200c7c10 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.859+0000 7f20277ee640 1 --2- 172.21.15.60:0/938604793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2020099a30 0x7f2020158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.859+0000 7f20277ee640 1 -- 172.21.15.60:0/938604793 >> 172.21.15.60:0/938604793 conn(0x7f20200b3200 msgr2=0x7f202006cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.859+0000 7f20277ee640 1 -- 172.21.15.60:0/938604793 shutdown_connections 2024-04-25T10:13:09.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.859+0000 7f20277ee640 1 -- 172.21.15.60:0/938604793 wait complete. 2024-04-25T10:13:09.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:09.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.991+0000 7f21ec714640 1 Processor -- start 2024-04-25T10:13:09.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.991+0000 7f21ec714640 1 -- start start 2024-04-25T10:13:09.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.991+0000 7f21ec714640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21e40b4610 0x7f21e415ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.991+0000 7f21ec714640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21e415b1d0 0x7f21e415ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.991+0000 7f21ec714640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e415b9a0 con 0x7f21e415a8b0 2024-04-25T10:13:09.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.991+0000 7f21ec714640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e415bc80 con 0x7f21e415b1d0 2024-04-25T10:13:09.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.991+0000 7f21ec714640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e415bf60 con 0x7f21e40b4610 2024-04-25T10:13:09.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.991+0000 7f21eacb1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21e415b1d0 0x7f21e415ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.991+0000 7f21eacb1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21e415b1d0 0x7f21e415ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60020/0 (socket says 172.21.15.60:60020) 2024-04-25T10:13:09.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.991+0000 7f21eacb1640 1 -- 172.21.15.60:0/533055877 learned_addr learned my addr 172.21.15.60:0/533055877 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:09.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.991+0000 7f21e9caf640 1 --2- 172.21.15.60:0/533055877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21e40b4610 0x7f21e415ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.991+0000 7f21eacb1640 1 -- 172.21.15.60:0/533055877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21e40b4610 msgr2=0x7f21e415ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.991+0000 7f21eacb1640 1 --2- 172.21.15.60:0/533055877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21e40b4610 0x7f21e415ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.991+0000 7f21eacb1640 1 -- 172.21.15.60:0/533055877 >> v1:172.21.15.100:6789/0 conn(0x7f21e415a8b0 legacy=0x7f21e40b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.991+0000 7f21eacb1640 1 -- 172.21.15.60:0/533055877 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e415c240 con 0x7f21e415b1d0 2024-04-25T10:13:09.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.991+0000 7f21e9caf640 1 --2- 172.21.15.60:0/533055877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21e40b4610 0x7f21e415ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:09.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21eacb1640 1 --2- 172.21.15.60:0/533055877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21e415b1d0 0x7f21e415ca40 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009e50 tx=0x7f21dc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941ff2fdaa965942 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21e94ae640 1 -- 172.21.15.60:0/533055877 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc0a5780 con 0x7f21e415b1d0 2024-04-25T10:13:09.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21e94ae640 1 -- 172.21.15.60:0/533055877 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21dc0a5920 con 0x7f21e415b1d0 2024-04-25T10:13:09.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21e94ae640 1 -- 172.21.15.60:0/533055877 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc0a5cf0 con 0x7f21e415b1d0 2024-04-25T10:13:09.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21ec714640 1 -- 172.21.15.60:0/533055877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21e415b1d0 msgr2=0x7f21e415ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21ec714640 1 --2- 172.21.15.60:0/533055877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21e415b1d0 0x7f21e415ca40 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009e50 tx=0x7f21dc01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21ec714640 1 -- 172.21.15.60:0/533055877 shutdown_connections 2024-04-25T10:13:09.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21ec714640 1 --2- 172.21.15.60:0/533055877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21e415b1d0 0x7f21e415ca40 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21ec714640 1 --2- 172.21.15.60:0/533055877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21e40b4610 0x7f21e415ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21ec714640 1 -- 172.21.15.60:0/533055877 >> 172.21.15.60:0/533055877 conn(0x7f21e4072900 msgr2=0x7f21e4099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21ec714640 1 -- 172.21.15.60:0/533055877 shutdown_connections 2024-04-25T10:13:10.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21ec714640 1 -- 172.21.15.60:0/533055877 wait complete. 2024-04-25T10:13:10.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21ec714640 1 Processor -- start 2024-04-25T10:13:10.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21ec714640 1 -- start start 2024-04-25T10:13:10.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21ec714640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e40b4610 0x7f21e4160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21ec714640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21e415a8b0 0x7f21e4146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21ec714640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21e415b1d0 0x7f21e4164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21ec714640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e415db60 con 0x7f21e415a8b0 2024-04-25T10:13:10.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21ec714640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e415d9e0 con 0x7f21e40b4610 2024-04-25T10:13:10.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21ec714640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e415dce0 con 0x7f21e415b1d0 2024-04-25T10:13:10.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21e9caf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21e415a8b0 0x7f21e4146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.995+0000 7f21e9caf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21e415a8b0 0x7f21e4146220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60028/0 (socket says 172.21.15.60:60028) 2024-04-25T10:13:10.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21e9caf640 1 -- 172.21.15.60:0/1907165423 learned_addr learned my addr 172.21.15.60:0/1907165423 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:10.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21eacb1640 1 --2- 172.21.15.60:0/1907165423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21e415b1d0 0x7f21e4164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21ea4b0640 1 --2- 172.21.15.60:0/1907165423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e40b4610 0x7f21e4160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21e9caf640 1 -- 172.21.15.60:0/1907165423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21e415b1d0 msgr2=0x7f21e4164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21e9caf640 1 --2- 172.21.15.60:0/1907165423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21e415b1d0 0x7f21e4164bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21e9caf640 1 -- 172.21.15.60:0/1907165423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e40b4610 msgr2=0x7f21e4160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21e9caf640 1 --2- 172.21.15.60:0/1907165423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e40b4610 0x7f21e4160220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21e9caf640 1 -- 172.21.15.60:0/1907165423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e4160890 con 0x7f21e415a8b0 2024-04-25T10:13:10.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21eacb1640 1 --2- 172.21.15.60:0/1907165423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21e415b1d0 0x7f21e4164bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:10.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21e9caf640 1 --2- 172.21.15.60:0/1907165423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21e415a8b0 0x7f21e4146220 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f21d400c7f0 tx=0x7f21d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21daffd640 1 -- 172.21.15.60:0/1907165423 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d4015070 con 0x7f21e415a8b0 2024-04-25T10:13:10.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21daffd640 1 -- 172.21.15.60:0/1907165423 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d4007820 con 0x7f21e415a8b0 2024-04-25T10:13:10.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21ec714640 1 -- 172.21.15.60:0/1907165423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e4160b70 con 0x7f21e415a8b0 2024-04-25T10:13:10.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21daffd640 1 -- 172.21.15.60:0/1907165423 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d4007c10 con 0x7f21e415a8b0 2024-04-25T10:13:10.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21ec714640 1 -- 172.21.15.60:0/1907165423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e4161130 con 0x7f21e415a8b0 2024-04-25T10:13:10.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21ec714640 1 -- 172.21.15.60:0/1907165423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e4074770 con 0x7f21e415a8b0 2024-04-25T10:13:10.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21daffd640 1 -- 172.21.15.60:0/1907165423 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21d4018070 con 0x7f21e415a8b0 2024-04-25T10:13:10.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21daffd640 1 --2- 172.21.15.60:0/1907165423 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f21c8066f90 0x7f21c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21daffd640 1 -- 172.21.15.60:0/1907165423 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f21d4085060 con 0x7f21e415a8b0 2024-04-25T10:13:10.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21ea4b0640 1 --2- 172.21.15.60:0/1907165423 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f21c8066f90 0x7f21c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:09.999+0000 7f21ea4b0640 1 --2- 172.21.15.60:0/1907165423 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f21c8066f90 0x7f21c8069450 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f21e0007fd0 tx=0x7f21e0007e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.003+0000 7f21daffd640 1 -- 172.21.15.60:0/1907165423 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f21d40559c0 con 0x7f21e415a8b0 2024-04-25T10:13:10.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.163+0000 7f21ec714640 1 -- 172.21.15.60:0/1907165423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f21e406f0f0 con 0x7f21e415a8b0 2024-04-25T10:13:10.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.163+0000 7f21daffd640 1 -- 172.21.15.60:0/1907165423 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f21d4059a60 con 0x7f21e415a8b0 2024-04-25T10:13:10.167 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.163+0000 7f21ec714640 1 -- 172.21.15.60:0/1907165423 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f21c8066f90 msgr2=0x7f21c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.163+0000 7f21ec714640 1 --2- 172.21.15.60:0/1907165423 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f21c8066f90 0x7f21c8069450 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f21e0007fd0 tx=0x7f21e0007e40 comp rx=0 tx=0).stop 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.163+0000 7f21ec714640 1 -- 172.21.15.60:0/1907165423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21e415a8b0 msgr2=0x7f21e4146220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.163+0000 7f21ec714640 1 --2- 172.21.15.60:0/1907165423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21e415a8b0 0x7f21e4146220 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f21d400c7f0 tx=0x7f21d400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.163+0000 7f21ec714640 1 -- 172.21.15.60:0/1907165423 shutdown_connections 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.163+0000 7f21ec714640 1 --2- 172.21.15.60:0/1907165423 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f21c8066f90 0x7f21c8069450 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.163+0000 7f21ec714640 1 --2- 172.21.15.60:0/1907165423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21e415b1d0 0x7f21e4164bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.163+0000 7f21ec714640 1 --2- 172.21.15.60:0/1907165423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21e415a8b0 0x7f21e4146220 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.163+0000 7f21ec714640 1 --2- 172.21.15.60:0/1907165423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21e40b4610 0x7f21e4160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.163+0000 7f21ec714640 1 -- 172.21.15.60:0/1907165423 >> 172.21.15.60:0/1907165423 conn(0x7f21e4072900 msgr2=0x7f21e40b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.163+0000 7f21ec714640 1 -- 172.21.15.60:0/1907165423 shutdown_connections 2024-04-25T10:13:10.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.163+0000 7f21ec714640 1 -- 172.21.15.60:0/1907165423 wait complete. 2024-04-25T10:13:10.187 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:13:10.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.299+0000 7f553d737640 1 Processor -- start 2024-04-25T10:13:10.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.299+0000 7f553d737640 1 -- start start 2024-04-25T10:13:10.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.299+0000 7f553d737640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f55380ce7a0 0x7f55380ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.299+0000 7f553d737640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55380b4660 0x7f55380b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.299+0000 7f553d737640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55380cf0c0 con 0x7f553815bc90 2024-04-25T10:13:10.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.299+0000 7f553d737640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55380cc370 con 0x7f55380ce7a0 2024-04-25T10:13:10.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.299+0000 7f553d737640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5538071ca0 con 0x7f55380b4660 2024-04-25T10:13:10.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.299+0000 7f5536ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f55380ce7a0 0x7f55380ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.299+0000 7f55367fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f553815bc90 0x7f55380b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58492/0 (socket says 172.21.15.60:58492) 2024-04-25T10:13:10.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.299+0000 7f55367fc640 1 -- 172.21.15.60:0/777874725 learned_addr learned my addr 172.21.15.60:0/777874725 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:13:10.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.299+0000 7f55377fe640 1 --2- 172.21.15.60:0/777874725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55380b4660 0x7f55380b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.299+0000 7f5536ffd640 1 -- 172.21.15.60:0/777874725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55380b4660 msgr2=0x7f55380b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:10.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.299+0000 7f5536ffd640 1 --2- 172.21.15.60:0/777874725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55380b4660 0x7f55380b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.299+0000 7f5536ffd640 1 -- 172.21.15.60:0/777874725 >> v1:172.21.15.60:6789/0 conn(0x7f553815bc90 legacy=0x7f55380b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.299+0000 7f5536ffd640 1 -- 172.21.15.60:0/777874725 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55380cca40 con 0x7f55380ce7a0 2024-04-25T10:13:10.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.299+0000 7f55377fe640 1 --2- 172.21.15.60:0/777874725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55380b4660 0x7f55380b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:10.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f5536ffd640 1 --2- 172.21.15.60:0/777874725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55380ce7a0 0x7f55380ceb80 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f5528009d90 tx=0x7f552801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd39e5d01359932f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f5535ffb640 1 -- 172.21.15.60:0/777874725 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55280b2070 con 0x7f55380ce7a0 2024-04-25T10:13:10.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f5535ffb640 1 -- 172.21.15.60:0/777874725 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55280a5830 con 0x7f55380ce7a0 2024-04-25T10:13:10.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f5535ffb640 1 -- 172.21.15.60:0/777874725 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55280a5c00 con 0x7f55380ce7a0 2024-04-25T10:13:10.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f553d737640 1 -- 172.21.15.60:0/777874725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55380ce7a0 msgr2=0x7f55380ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f553d737640 1 --2- 172.21.15.60:0/777874725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55380ce7a0 0x7f55380ceb80 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f5528009d90 tx=0x7f552801c970 comp rx=0 tx=0).stop 2024-04-25T10:13:10.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f553d737640 1 -- 172.21.15.60:0/777874725 shutdown_connections 2024-04-25T10:13:10.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f553d737640 1 --2- 172.21.15.60:0/777874725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55380b4660 0x7f55380b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f553d737640 1 --2- 172.21.15.60:0/777874725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55380ce7a0 0x7f55380ceb80 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f553d737640 1 -- 172.21.15.60:0/777874725 >> 172.21.15.60:0/777874725 conn(0x7f55380b12d0 msgr2=0x7f55380b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f553d737640 1 -- 172.21.15.60:0/777874725 shutdown_connections 2024-04-25T10:13:10.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f553d737640 1 -- 172.21.15.60:0/777874725 wait complete. 2024-04-25T10:13:10.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f553d737640 1 Processor -- start 2024-04-25T10:13:10.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f553d737640 1 -- start start 2024-04-25T10:13:10.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f553d737640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55380ce7a0 0x7f55381447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f553d737640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f553815bc90 0x7f55380cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f553d737640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5538144f10 0x7f5538142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f553d737640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55380b4280 con 0x7f5538144f10 2024-04-25T10:13:10.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f553d737640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55380b3db0 con 0x7f55380ce7a0 2024-04-25T10:13:10.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.303+0000 7f553d737640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55380b4400 con 0x7f553815bc90 2024-04-25T10:13:10.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f55377fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5538144f10 0x7f5538142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f55377fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5538144f10 0x7f5538142da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60048/0 (socket says 172.21.15.60:60048) 2024-04-25T10:13:10.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f55377fe640 1 -- 172.21.15.60:0/4030650794 learned_addr learned my addr 172.21.15.60:0/4030650794 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:10.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f55367fc640 1 --2- 172.21.15.60:0/4030650794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f553815bc90 0x7f55380cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f55377fe640 1 -- 172.21.15.60:0/4030650794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f553815bc90 msgr2=0x7f55380cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f55377fe640 1 --2- 172.21.15.60:0/4030650794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f553815bc90 0x7f55380cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f5536ffd640 1 --2- 172.21.15.60:0/4030650794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55380ce7a0 0x7f55381447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f55377fe640 1 -- 172.21.15.60:0/4030650794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55380ce7a0 msgr2=0x7f55381447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f55377fe640 1 --2- 172.21.15.60:0/4030650794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55380ce7a0 0x7f55381447b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f55377fe640 1 -- 172.21.15.60:0/4030650794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55380cb4e0 con 0x7f5538144f10 2024-04-25T10:13:10.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f5536ffd640 1 --2- 172.21.15.60:0/4030650794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55380ce7a0 0x7f55381447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:10.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f55377fe640 1 --2- 172.21.15.60:0/4030650794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5538144f10 0x7f5538142da0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f553000c5f0 tx=0x7f553000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f551f7fe640 1 -- 172.21.15.60:0/4030650794 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5530015070 con 0x7f5538144f10 2024-04-25T10:13:10.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f551f7fe640 1 -- 172.21.15.60:0/4030650794 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5530011820 con 0x7f5538144f10 2024-04-25T10:13:10.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f551f7fe640 1 -- 172.21.15.60:0/4030650794 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5530011c10 con 0x7f5538144f10 2024-04-25T10:13:10.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f553d737640 1 -- 172.21.15.60:0/4030650794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5538143710 con 0x7f5538144f10 2024-04-25T10:13:10.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f553d737640 1 -- 172.21.15.60:0/4030650794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5538143cd0 con 0x7f5538144f10 2024-04-25T10:13:10.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f553d737640 1 -- 172.21.15.60:0/4030650794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5538074770 con 0x7f5538144f10 2024-04-25T10:13:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f551f7fe640 1 -- 172.21.15.60:0/4030650794 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5530018070 con 0x7f5538144f10 2024-04-25T10:13:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f551f7fe640 1 --2- 172.21.15.60:0/4030650794 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5524066f60 0x7f5524069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f551f7fe640 1 -- 172.21.15.60:0/4030650794 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f5530084ed0 con 0x7f5538144f10 2024-04-25T10:13:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f5536ffd640 1 --2- 172.21.15.60:0/4030650794 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5524066f60 0x7f5524069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.307+0000 7f5536ffd640 1 --2- 172.21.15.60:0/4030650794 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5524066f60 0x7f5524069420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f55280af000 tx=0x7f55280a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.311+0000 7f551f7fe640 1 -- 172.21.15.60:0/4030650794 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5530055e70 con 0x7f5538144f10 2024-04-25T10:13:10.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.487+0000 7f553d737640 1 -- 172.21.15.60:0/4030650794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f55380b1040 con 0x7f5538144f10 2024-04-25T10:13:10.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.487+0000 7f551f7fe640 1 -- 172.21.15.60:0/4030650794 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v85) v1 ==== 80+0+122 (secure 0 0 0) 0x7f5530055770 con 0x7f5538144f10 2024-04-25T10:13:10.493 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6835/2455562778 2024-04-26T10:11:58.136789+0000 2024-04-25T10:13:10.493 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6833/2455562778 2024-04-26T10:11:58.136789+0000 2024-04-25T10:13:10.493 INFO:teuthology.orchestra.run.smithi060.stderr:listed 2 entries 2024-04-25T10:13:10.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.491+0000 7f553d737640 1 -- 172.21.15.60:0/4030650794 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5524066f60 msgr2=0x7f5524069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.491+0000 7f553d737640 1 --2- 172.21.15.60:0/4030650794 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5524066f60 0x7f5524069420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f55280af000 tx=0x7f55280a7070 comp rx=0 tx=0).stop 2024-04-25T10:13:10.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.491+0000 7f553d737640 1 -- 172.21.15.60:0/4030650794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5538144f10 msgr2=0x7f5538142da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.491+0000 7f553d737640 1 --2- 172.21.15.60:0/4030650794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5538144f10 0x7f5538142da0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f553000c5f0 tx=0x7f553000cab0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.491+0000 7f553d737640 1 -- 172.21.15.60:0/4030650794 shutdown_connections 2024-04-25T10:13:10.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.491+0000 7f553d737640 1 --2- 172.21.15.60:0/4030650794 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5524066f60 0x7f5524069420 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.491+0000 7f553d737640 1 --2- 172.21.15.60:0/4030650794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5538144f10 0x7f5538142da0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.491+0000 7f553d737640 1 --2- 172.21.15.60:0/4030650794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f553815bc90 0x7f55380cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.491+0000 7f553d737640 1 --2- 172.21.15.60:0/4030650794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55380ce7a0 0x7f55381447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.491+0000 7f553d737640 1 -- 172.21.15.60:0/4030650794 >> 172.21.15.60:0/4030650794 conn(0x7f55380b12d0 msgr2=0x7f553815c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.491+0000 7f553d737640 1 -- 172.21.15.60:0/4030650794 shutdown_connections 2024-04-25T10:13:10.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.491+0000 7f553d737640 1 -- 172.21.15.60:0/4030650794 wait complete. 2024-04-25T10:13:10.512 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:13:10.512 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:10.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:10.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.683+0000 7fef619da640 1 Processor -- start 2024-04-25T10:13:10.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.683+0000 7fef619da640 1 -- start start 2024-04-25T10:13:10.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.683+0000 7fef619da640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fef5c15d8a0 0x7fef5c0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.683+0000 7fef619da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef5c15dc80 0x7fef5c0b5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.683+0000 7fef619da640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef5c070b30 con 0x7fef5c0afd60 2024-04-25T10:13:10.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.683+0000 7fef619da640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef5c161790 con 0x7fef5c15d8a0 2024-04-25T10:13:10.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.683+0000 7fef619da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c161a70 con 0x7fef5c15dc80 2024-04-25T10:13:10.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.683+0000 7fef5a7fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef5c15dc80 0x7fef5c0b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.683+0000 7fef5affd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fef5c15d8a0 0x7fef5c0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.683+0000 7fef5affd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fef5c15d8a0 0x7fef5c0ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49968/0 (socket says 172.21.15.60:49968) 2024-04-25T10:13:10.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.683+0000 7fef5affd640 1 -- 172.21.15.60:0/2570290961 learned_addr learned my addr 172.21.15.60:0/2570290961 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:10.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.683+0000 7fef5a7fc640 1 -- 172.21.15.60:0/2570290961 >> v2:172.21.15.100:3300/0 conn(0x7fef5c15d8a0 msgr2=0x7fef5c0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:10.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.683+0000 7fef5a7fc640 1 --2- 172.21.15.60:0/2570290961 >> v2:172.21.15.100:3300/0 conn(0x7fef5c15d8a0 0x7fef5c0ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.683+0000 7fef5a7fc640 1 -- 172.21.15.60:0/2570290961 >> v1:172.21.15.100:6789/0 conn(0x7fef5c0afd60 legacy=0x7fef5c072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.683+0000 7fef5a7fc640 1 -- 172.21.15.60:0/2570290961 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c070800 con 0x7fef5c15dc80 2024-04-25T10:13:10.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.687+0000 7fef5a7fc640 1 --2- 172.21.15.60:0/2570290961 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef5c15dc80 0x7fef5c0b5160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fef500097c0 tx=0x7fef5001c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a3b5003af17ec30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.687+0000 7fef59ffb640 1 -- 172.21.15.60:0/2570290961 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef500b0070 con 0x7fef5c15dc80 2024-04-25T10:13:10.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.687+0000 7fef59ffb640 1 -- 172.21.15.60:0/2570290961 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef500ab600 con 0x7fef5c15dc80 2024-04-25T10:13:10.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.687+0000 7fef59ffb640 1 -- 172.21.15.60:0/2570290961 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef500ab9d0 con 0x7fef5c15dc80 2024-04-25T10:13:10.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.687+0000 7fef619da640 1 -- 172.21.15.60:0/2570290961 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef5c15dc80 msgr2=0x7fef5c0b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.687+0000 7fef619da640 1 --2- 172.21.15.60:0/2570290961 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef5c15dc80 0x7fef5c0b5160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fef500097c0 tx=0x7fef5001c350 comp rx=0 tx=0).stop 2024-04-25T10:13:10.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.687+0000 7fef619da640 1 -- 172.21.15.60:0/2570290961 shutdown_connections 2024-04-25T10:13:10.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.687+0000 7fef619da640 1 --2- 172.21.15.60:0/2570290961 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef5c15dc80 0x7fef5c0b5160 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.687+0000 7fef619da640 1 --2- 172.21.15.60:0/2570290961 >> v2:172.21.15.100:3300/0 conn(0x7fef5c15d8a0 0x7fef5c0ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.687+0000 7fef619da640 1 -- 172.21.15.60:0/2570290961 >> 172.21.15.60:0/2570290961 conn(0x7fef5c06e400 msgr2=0x7fef5c06e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.687+0000 7fef619da640 1 -- 172.21.15.60:0/2570290961 shutdown_connections 2024-04-25T10:13:10.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.687+0000 7fef619da640 1 -- 172.21.15.60:0/2570290961 wait complete. 2024-04-25T10:13:10.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.687+0000 7fef619da640 1 Processor -- start 2024-04-25T10:13:10.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.687+0000 7fef619da640 1 -- start start 2024-04-25T10:13:10.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef619da640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef5c0afd60 0x7fef5c0cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef619da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef5c15d8a0 0x7fef5c14da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef619da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef5c0cc2f0 0x7fef5c15b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef619da640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c16ad20 con 0x7fef5c0afd60 2024-04-25T10:13:10.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef619da640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c16aba0 con 0x7fef5c15d8a0 2024-04-25T10:13:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef619da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c16aea0 con 0x7fef5c0cc2f0 2024-04-25T10:13:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef5affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef5c0afd60 0x7fef5c0cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef5affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef5c0afd60 0x7fef5c0cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60060/0 (socket says 172.21.15.60:60060) 2024-04-25T10:13:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef5affd640 1 -- 172.21.15.60:0/403200046 learned_addr learned my addr 172.21.15.60:0/403200046 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef5a7fc640 1 --2- 172.21.15.60:0/403200046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef5c15d8a0 0x7fef5c14da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef5b7fe640 1 --2- 172.21.15.60:0/403200046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef5c0cc2f0 0x7fef5c15b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef5affd640 1 -- 172.21.15.60:0/403200046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef5c0cc2f0 msgr2=0x7fef5c15b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef5affd640 1 --2- 172.21.15.60:0/403200046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef5c0cc2f0 0x7fef5c15b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef5affd640 1 -- 172.21.15.60:0/403200046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef5c15d8a0 msgr2=0x7fef5c14da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef5affd640 1 --2- 172.21.15.60:0/403200046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef5c15d8a0 0x7fef5c14da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef5affd640 1 -- 172.21.15.60:0/403200046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c14dfc0 con 0x7fef5c0afd60 2024-04-25T10:13:10.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef5affd640 1 --2- 172.21.15.60:0/403200046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef5c0afd60 0x7fef5c0cab00 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00e900 tx=0x7fef4c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef43fff640 1 -- 172.21.15.60:0/403200046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c017070 con 0x7fef5c0afd60 2024-04-25T10:13:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef43fff640 1 -- 172.21.15.60:0/403200046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4c00ca00 con 0x7fef5c0afd60 2024-04-25T10:13:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef619da640 1 -- 172.21.15.60:0/403200046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5c0cb250 con 0x7fef5c0afd60 2024-04-25T10:13:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef43fff640 1 -- 172.21.15.60:0/403200046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c00cdf0 con 0x7fef5c0afd60 2024-04-25T10:13:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef619da640 1 -- 172.21.15.60:0/403200046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5c0cb810 con 0x7fef5c0afd60 2024-04-25T10:13:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef619da640 1 -- 172.21.15.60:0/403200046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef5c074770 con 0x7fef5c0afd60 2024-04-25T10:13:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef43fff640 1 -- 172.21.15.60:0/403200046 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef4c01a070 con 0x7fef5c0afd60 2024-04-25T10:13:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef43fff640 1 --2- 172.21.15.60:0/403200046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fef48066f60 0x7fef48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef43fff640 1 -- 172.21.15.60:0/403200046 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fef4c085970 con 0x7fef5c0afd60 2024-04-25T10:13:10.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.691+0000 7fef5a7fc640 1 --2- 172.21.15.60:0/403200046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fef48066f60 0x7fef48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.695+0000 7fef5a7fc640 1 --2- 172.21.15.60:0/403200046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fef48066f60 0x7fef48069420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fef500ad000 tx=0x7fef500a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.695+0000 7fef43fff640 1 -- 172.21.15.60:0/403200046 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef4c056910 con 0x7fef5c0afd60 2024-04-25T10:13:10.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.867+0000 7fef619da640 1 -- 172.21.15.60:0/403200046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef5c1576e0 con 0x7fef5c0afd60 2024-04-25T10:13:10.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.871+0000 7fef43fff640 1 -- 172.21.15.60:0/403200046 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16851 (secure 0 0 0) 0x7fef4c056210 con 0x7fef5c0afd60 2024-04-25T10:13:10.875 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:13:10.876 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:12:03.323694+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:42.621502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:43.352679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/2455562778":"2024-04-26T10:11:58.136789+0000","172.21.15.60:6833/2455562778":"2024-04-26T10:11:58.136789+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:10.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.871+0000 7fef619da640 1 -- 172.21.15.60:0/403200046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fef48066f60 msgr2=0x7fef48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.871+0000 7fef619da640 1 --2- 172.21.15.60:0/403200046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fef48066f60 0x7fef48069420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fef500ad000 tx=0x7fef500a6000 comp rx=0 tx=0).stop 2024-04-25T10:13:10.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.871+0000 7fef619da640 1 -- 172.21.15.60:0/403200046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef5c0afd60 msgr2=0x7fef5c0cab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.871+0000 7fef619da640 1 --2- 172.21.15.60:0/403200046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef5c0afd60 0x7fef5c0cab00 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00e900 tx=0x7fef4c00edc0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.871+0000 7fef619da640 1 -- 172.21.15.60:0/403200046 shutdown_connections 2024-04-25T10:13:10.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.871+0000 7fef619da640 1 --2- 172.21.15.60:0/403200046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fef48066f60 0x7fef48069420 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.871+0000 7fef619da640 1 --2- 172.21.15.60:0/403200046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef5c0cc2f0 0x7fef5c15b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.871+0000 7fef619da640 1 --2- 172.21.15.60:0/403200046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef5c15d8a0 0x7fef5c14da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.871+0000 7fef619da640 1 --2- 172.21.15.60:0/403200046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef5c0afd60 0x7fef5c0cab00 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.871+0000 7fef619da640 1 -- 172.21.15.60:0/403200046 >> 172.21.15.60:0/403200046 conn(0x7fef5c06e400 msgr2=0x7fef5c0b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.871+0000 7fef619da640 1 -- 172.21.15.60:0/403200046 shutdown_connections 2024-04-25T10:13:10.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:10.871+0000 7fef619da640 1 -- 172.21.15.60:0/403200046 wait complete. 2024-04-25T10:13:10.895 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:42.621502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:43.352679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:10.895 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:11.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.015+0000 7fe017aa0640 1 Processor -- start 2024-04-25T10:13:11.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.015+0000 7fe017aa0640 1 -- start start 2024-04-25T10:13:11.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.015+0000 7fe017aa0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe01006ac10 0x7fe0100af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.015+0000 7fe017aa0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe01006aff0 0x7fe01006b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.015+0000 7fe017aa0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0100aff30 con 0x7fe0100d05a0 2024-04-25T10:13:11.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.015+0000 7fe017aa0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01015bf10 con 0x7fe01006aff0 2024-04-25T10:13:11.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.015+0000 7fe017aa0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0100b0210 con 0x7fe01006ac10 2024-04-25T10:13:11.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.015+0000 7fe01503b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe01006aff0 0x7fe01006b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.015+0000 7fe01503b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe01006aff0 0x7fe01006b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60068/0 (socket says 172.21.15.60:60068) 2024-04-25T10:13:11.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.015+0000 7fe01503b640 1 -- 172.21.15.60:0/2291444524 learned_addr learned my addr 172.21.15.60:0/2291444524 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:11.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.015+0000 7fe01583c640 1 --2- 172.21.15.60:0/2291444524 >> v2:172.21.15.100:3300/0 conn(0x7fe01006ac10 0x7fe0100af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.015+0000 7fe01503b640 1 -- 172.21.15.60:0/2291444524 >> v2:172.21.15.100:3300/0 conn(0x7fe01006ac10 msgr2=0x7fe0100af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.015+0000 7fe01503b640 1 --2- 172.21.15.60:0/2291444524 >> v2:172.21.15.100:3300/0 conn(0x7fe01006ac10 0x7fe0100af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.015+0000 7fe01503b640 1 -- 172.21.15.60:0/2291444524 >> v1:172.21.15.100:6789/0 conn(0x7fe0100d05a0 legacy=0x7fe0100d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.015+0000 7fe01503b640 1 -- 172.21.15.60:0/2291444524 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0100b04f0 con 0x7fe01006aff0 2024-04-25T10:13:11.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe01503b640 1 --2- 172.21.15.60:0/2291444524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01006aff0 0x7fe01006b470 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fe010071f50 tx=0x7fe00c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba579f4ab3f046d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe01483a640 1 -- 172.21.15.60:0/2291444524 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c0a5760 con 0x7fe01006aff0 2024-04-25T10:13:11.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe01483a640 1 -- 172.21.15.60:0/2291444524 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00c0a5900 con 0x7fe01006aff0 2024-04-25T10:13:11.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe01483a640 1 -- 172.21.15.60:0/2291444524 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c0a5cd0 con 0x7fe01006aff0 2024-04-25T10:13:11.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe017aa0640 1 -- 172.21.15.60:0/2291444524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01006aff0 msgr2=0x7fe01006b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe017aa0640 1 --2- 172.21.15.60:0/2291444524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01006aff0 0x7fe01006b470 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fe010071f50 tx=0x7fe00c01c9b0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe017aa0640 1 -- 172.21.15.60:0/2291444524 shutdown_connections 2024-04-25T10:13:11.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe017aa0640 1 --2- 172.21.15.60:0/2291444524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01006aff0 0x7fe01006b470 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe017aa0640 1 --2- 172.21.15.60:0/2291444524 >> v2:172.21.15.100:3300/0 conn(0x7fe01006ac10 0x7fe0100af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe017aa0640 1 -- 172.21.15.60:0/2291444524 >> 172.21.15.60:0/2291444524 conn(0x7fe010072920 msgr2=0x7fe010099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe017aa0640 1 -- 172.21.15.60:0/2291444524 shutdown_connections 2024-04-25T10:13:11.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe017aa0640 1 -- 172.21.15.60:0/2291444524 wait complete. 2024-04-25T10:13:11.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe017aa0640 1 Processor -- start 2024-04-25T10:13:11.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe017aa0640 1 -- start start 2024-04-25T10:13:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe017aa0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe01006ac10 0x7fe01013a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe017aa0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01006aff0 0x7fe0100cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe017aa0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0100d05a0 0x7fe0101591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe017aa0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100b2160 con 0x7fe01006aff0 2024-04-25T10:13:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe017aa0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100b1fe0 con 0x7fe0100d05a0 2024-04-25T10:13:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe017aa0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01006b9b0 con 0x7fe01006ac10 2024-04-25T10:13:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe01503b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01006aff0 0x7fe0100cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe01503b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01006aff0 0x7fe0100cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60070/0 (socket says 172.21.15.60:60070) 2024-04-25T10:13:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe01503b640 1 -- 172.21.15.60:0/725535740 learned_addr learned my addr 172.21.15.60:0/725535740 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe01583c640 1 --2- 172.21.15.60:0/725535740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe01006ac10 0x7fe01013a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe01503b640 1 -- 172.21.15.60:0/725535740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe01006ac10 msgr2=0x7fe01013a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.019+0000 7fe01603d640 1 --2- 172.21.15.60:0/725535740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0100d05a0 0x7fe0101591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe01503b640 1 --2- 172.21.15.60:0/725535740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe01006ac10 0x7fe01013a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe01503b640 1 -- 172.21.15.60:0/725535740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0100d05a0 msgr2=0x7fe0101591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe01503b640 1 --2- 172.21.15.60:0/725535740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0100d05a0 0x7fe0101591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe01503b640 1 -- 172.21.15.60:0/725535740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0100cd710 con 0x7fe01006aff0 2024-04-25T10:13:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe01503b640 1 --2- 172.21.15.60:0/725535740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01006aff0 0x7fe0100cf1a0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0af040 tx=0x7fe00c019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe0027fc640 1 -- 172.21.15.60:0/725535740 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c0bc020 con 0x7fe01006aff0 2024-04-25T10:13:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe0027fc640 1 -- 172.21.15.60:0/725535740 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00c0b8670 con 0x7fe01006aff0 2024-04-25T10:13:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe017aa0640 1 -- 172.21.15.60:0/725535740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0100cd9f0 con 0x7fe01006aff0 2024-04-25T10:13:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe0027fc640 1 -- 172.21.15.60:0/725535740 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c0b8a60 con 0x7fe01006aff0 2024-04-25T10:13:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe017aa0640 1 -- 172.21.15.60:0/725535740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0100cdcd0 con 0x7fe01006aff0 2024-04-25T10:13:11.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe017aa0640 1 -- 172.21.15.60:0/725535740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe010074770 con 0x7fe01006aff0 2024-04-25T10:13:11.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe0027fc640 1 -- 172.21.15.60:0/725535740 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe00c0bf020 con 0x7fe01006aff0 2024-04-25T10:13:11.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe0027fc640 1 --2- 172.21.15.60:0/725535740 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdff4066f60 0x7fdff4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe0027fc640 1 -- 172.21.15.60:0/725535740 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe00c120270 con 0x7fe01006aff0 2024-04-25T10:13:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe01583c640 1 --2- 172.21.15.60:0/725535740 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdff4066f60 0x7fdff4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.023+0000 7fe01583c640 1 --2- 172.21.15.60:0/725535740 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdff4066f60 0x7fdff4069420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe008009790 tx=0x7fe008008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.027+0000 7fe0027fc640 1 -- 172.21.15.60:0/725535740 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe00c0c6030 con 0x7fe01006aff0 2024-04-25T10:13:11.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.187+0000 7fe017aa0640 1 -- 172.21.15.60:0/725535740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe01013f6a0 con 0x7fe01006aff0 2024-04-25T10:13:11.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.187+0000 7fe0027fc640 1 -- 172.21.15.60:0/725535740 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe00c0165a0 con 0x7fe01006aff0 2024-04-25T10:13:11.193 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:13:11.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.191+0000 7fe017aa0640 1 -- 172.21.15.60:0/725535740 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdff4066f60 msgr2=0x7fdff4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.191+0000 7fe017aa0640 1 --2- 172.21.15.60:0/725535740 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdff4066f60 0x7fdff4069420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe008009790 tx=0x7fe008008040 comp rx=0 tx=0).stop 2024-04-25T10:13:11.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.191+0000 7fe017aa0640 1 -- 172.21.15.60:0/725535740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01006aff0 msgr2=0x7fe0100cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.191+0000 7fe017aa0640 1 --2- 172.21.15.60:0/725535740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01006aff0 0x7fe0100cf1a0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0af040 tx=0x7fe00c019750 comp rx=0 tx=0).stop 2024-04-25T10:13:11.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.191+0000 7fe017aa0640 1 -- 172.21.15.60:0/725535740 shutdown_connections 2024-04-25T10:13:11.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.191+0000 7fe017aa0640 1 --2- 172.21.15.60:0/725535740 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdff4066f60 0x7fdff4069420 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.191+0000 7fe017aa0640 1 --2- 172.21.15.60:0/725535740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0100d05a0 0x7fe0101591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.191+0000 7fe017aa0640 1 --2- 172.21.15.60:0/725535740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01006aff0 0x7fe0100cf1a0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.191+0000 7fe017aa0640 1 --2- 172.21.15.60:0/725535740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe01006ac10 0x7fe01013a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.191+0000 7fe017aa0640 1 -- 172.21.15.60:0/725535740 >> 172.21.15.60:0/725535740 conn(0x7fe010072920 msgr2=0x7fe01013bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.191+0000 7fe017aa0640 1 -- 172.21.15.60:0/725535740 shutdown_connections 2024-04-25T10:13:11.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.191+0000 7fe017aa0640 1 -- 172.21.15.60:0/725535740 wait complete. 2024-04-25T10:13:11.211 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:11.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.335+0000 7f632cbf4640 1 Processor -- start 2024-04-25T10:13:11.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.335+0000 7f632cbf4640 1 -- start start 2024-04-25T10:13:11.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632cbf4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f632815fbf0 0x7f6328166870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632cbf4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6328144a40 con 0x7f63280cbcf0 2024-04-25T10:13:11.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632cbf4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63280cc9f0 con 0x7f6328160080 2024-04-25T10:13:11.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632cbf4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63280b2dc0 con 0x7f632815fbf0 2024-04-25T10:13:11.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632659c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f63280cbcf0 0x7f6328071c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58500/0 (socket says 172.21.15.60:58500) 2024-04-25T10:13:11.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632659c640 1 -- 172.21.15.60:0/551638887 learned_addr learned my addr 172.21.15.60:0/551638887 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:13:11.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f6326d9d640 1 --2- 172.21.15.60:0/551638887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f632815fbf0 0x7f6328166870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632559a640 1 -- 172.21.15.60:0/551638887 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296152031 0 0) 0x7f6328144a40 con 0x7f63280cbcf0 2024-04-25T10:13:11.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632559a640 1 -- 172.21.15.60:0/551638887 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6310003530 con 0x7f63280cbcf0 2024-04-25T10:13:11.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632559a640 1 -- 172.21.15.60:0/551638887 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1408934621 0 0) 0x7f6310003530 con 0x7f63280cbcf0 2024-04-25T10:13:11.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632559a640 1 -- 172.21.15.60:0/551638887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f632815fbf0 msgr2=0x7f6328166870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632559a640 1 --2- 172.21.15.60:0/551638887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f632815fbf0 0x7f6328166870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632559a640 1 -- 172.21.15.60:0/551638887 >> v1:172.21.15.100:6789/0 conn(0x7f6328160080 legacy=0x7f632815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632559a640 1 -- 172.21.15.60:0/551638887 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280cccd0 con 0x7f63280cbcf0 2024-04-25T10:13:11.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632559a640 1 -- 172.21.15.60:0/551638887 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f6318001680 con 0x7f63280cbcf0 2024-04-25T10:13:11.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632559a640 1 -- 172.21.15.60:0/551638887 >> v1:172.21.15.60:6789/0 conn(0x7f63280cbcf0 legacy=0x7f6328071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632559a640 1 --2- 172.21.15.60:0/551638887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63100039f0 0x7f6310003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632559a640 1 --2- 172.21.15.60:0/551638887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f632815fbf0 0x7f6310004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632559a640 1 --2- 172.21.15.60:0/551638887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6310004d50 0x7f63100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632559a640 1 -- 172.21.15.60:0/551638887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631000ba80 con 0x7f63100039f0 2024-04-25T10:13:11.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632559a640 1 -- 172.21.15.60:0/551638887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631000bc30 con 0x7f6310004d50 2024-04-25T10:13:11.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632559a640 1 -- 172.21.15.60:0/551638887 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631000be30 con 0x7f632815fbf0 2024-04-25T10:13:11.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632659c640 1 --2- 172.21.15.60:0/551638887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63100039f0 0x7f6310003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f6325d9b640 1 --2- 172.21.15.60:0/551638887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f632815fbf0 0x7f6310004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f6326d9d640 1 --2- 172.21.15.60:0/551638887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6310004d50 0x7f63100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632659c640 1 -- 172.21.15.60:0/551638887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f632815fbf0 msgr2=0x7f6310004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632659c640 1 --2- 172.21.15.60:0/551638887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f632815fbf0 0x7f6310004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632659c640 1 -- 172.21.15.60:0/551638887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6310004d50 msgr2=0x7f63100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632659c640 1 --2- 172.21.15.60:0/551638887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6310004d50 0x7f63100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f632659c640 1 -- 172.21.15.60:0/551638887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6310003530 con 0x7f63100039f0 2024-04-25T10:13:11.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.339+0000 7f6326d9d640 1 --2- 172.21.15.60:0/551638887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6310004d50 0x7f63100051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:11.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632659c640 1 --2- 172.21.15.60:0/551638887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63100039f0 0x7f6310003e10 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f631800c210 tx=0x7f631801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155574cee57a085f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632559a640 1 -- 172.21.15.60:0/551638887 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318007d20 con 0x7f63100039f0 2024-04-25T10:13:11.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632559a640 1 -- 172.21.15.60:0/551638887 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63180a9020 con 0x7f63100039f0 2024-04-25T10:13:11.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632559a640 1 -- 172.21.15.60:0/551638887 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318005b00 con 0x7f63100039f0 2024-04-25T10:13:11.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632cbf4640 1 -- 172.21.15.60:0/551638887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63100039f0 msgr2=0x7f6310003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632cbf4640 1 --2- 172.21.15.60:0/551638887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63100039f0 0x7f6310003e10 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f631800c210 tx=0x7f631801cc60 comp rx=0 tx=0).stop 2024-04-25T10:13:11.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f6326d9d640 1 -- 172.21.15.60:0/551638887 reap_dead start 2024-04-25T10:13:11.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632cbf4640 1 -- 172.21.15.60:0/551638887 shutdown_connections 2024-04-25T10:13:11.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632cbf4640 1 -- 172.21.15.60:0/551638887 >> 172.21.15.60:0/551638887 conn(0x7f63280670b0 msgr2=0x7f632806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632cbf4640 1 -- 172.21.15.60:0/551638887 shutdown_connections 2024-04-25T10:13:11.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632cbf4640 1 -- 172.21.15.60:0/551638887 wait complete. 2024-04-25T10:13:11.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632cbf4640 1 Processor -- start 2024-04-25T10:13:11.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632cbf4640 1 -- start start 2024-04-25T10:13:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632cbf4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63100039f0 0x7f63280af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632cbf4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f632806efd0 0x7f632806d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632cbf4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f632813dcf0 0x7f632813e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632cbf4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63280cfed0 con 0x7f632813dcf0 2024-04-25T10:13:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632cbf4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63280ce5c0 con 0x7f63100039f0 2024-04-25T10:13:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632cbf4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632806e160 con 0x7f632806efd0 2024-04-25T10:13:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f6326d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f632813dcf0 0x7f632813e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f6326d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f632813dcf0 0x7f632813e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60092/0 (socket says 172.21.15.60:60092) 2024-04-25T10:13:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f6326d9d640 1 -- 172.21.15.60:0/3035065685 learned_addr learned my addr 172.21.15.60:0/3035065685 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f632659c640 1 --2- 172.21.15.60:0/3035065685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63100039f0 0x7f63280af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f6325d9b640 1 --2- 172.21.15.60:0/3035065685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f632806efd0 0x7f632806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f6326d9d640 1 -- 172.21.15.60:0/3035065685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f632806efd0 msgr2=0x7f632806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f6326d9d640 1 --2- 172.21.15.60:0/3035065685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f632806efd0 0x7f632806d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f6326d9d640 1 -- 172.21.15.60:0/3035065685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63100039f0 msgr2=0x7f63280af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f6326d9d640 1 --2- 172.21.15.60:0/3035065685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63100039f0 0x7f63280af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.343+0000 7f6326d9d640 1 -- 172.21.15.60:0/3035065685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632806dba0 con 0x7f632813dcf0 2024-04-25T10:13:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.347+0000 7f6326d9d640 1 --2- 172.21.15.60:0/3035065685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f632813dcf0 0x7f632813e1a0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f63280cc5b0 tx=0x7f632000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.347+0000 7f6316ffd640 1 -- 172.21.15.60:0/3035065685 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632001c070 con 0x7f632813dcf0 2024-04-25T10:13:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.347+0000 7f6316ffd640 1 -- 172.21.15.60:0/3035065685 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6320017400 con 0x7f632813dcf0 2024-04-25T10:13:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.347+0000 7f632cbf4640 1 -- 172.21.15.60:0/3035065685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632806de80 con 0x7f632813dcf0 2024-04-25T10:13:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.347+0000 7f6316ffd640 1 -- 172.21.15.60:0/3035065685 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63200177f0 con 0x7f632813dcf0 2024-04-25T10:13:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.347+0000 7f632cbf4640 1 -- 172.21.15.60:0/3035065685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63280acc20 con 0x7f632813dcf0 2024-04-25T10:13:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.347+0000 7f632cbf4640 1 -- 172.21.15.60:0/3035065685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f632806bba0 con 0x7f632813dcf0 2024-04-25T10:13:11.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.347+0000 7f6316ffd640 1 -- 172.21.15.60:0/3035065685 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f632001f070 con 0x7f632813dcf0 2024-04-25T10:13:11.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.347+0000 7f6316ffd640 1 --2- 172.21.15.60:0/3035065685 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6308066f90 0x7f6308069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.347+0000 7f6316ffd640 1 -- 172.21.15.60:0/3035065685 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f6320084ff0 con 0x7f632813dcf0 2024-04-25T10:13:11.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.347+0000 7f632659c640 1 --2- 172.21.15.60:0/3035065685 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6308066f90 0x7f6308069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.347+0000 7f632659c640 1 --2- 172.21.15.60:0/3035065685 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6308066f90 0x7f6308069450 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f63180aa000 tx=0x7f6318000ff0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.351+0000 7f6316ffd640 1 -- 172.21.15.60:0/3035065685 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6320012070 con 0x7f632813dcf0 2024-04-25T10:13:11.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.511+0000 7f632cbf4640 1 -- 172.21.15.60:0/3035065685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6328074770 con 0x7f632813dcf0 2024-04-25T10:13:11.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.511+0000 7f6316ffd640 1 -- 172.21.15.60:0/3035065685 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6320055f90 con 0x7f632813dcf0 2024-04-25T10:13:11.516 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:13:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.511+0000 7f632cbf4640 1 -- 172.21.15.60:0/3035065685 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6308066f90 msgr2=0x7f6308069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.511+0000 7f632cbf4640 1 --2- 172.21.15.60:0/3035065685 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6308066f90 0x7f6308069450 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f63180aa000 tx=0x7f6318000ff0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.511+0000 7f632cbf4640 1 -- 172.21.15.60:0/3035065685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f632813dcf0 msgr2=0x7f632813e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.511+0000 7f632cbf4640 1 --2- 172.21.15.60:0/3035065685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f632813dcf0 0x7f632813e1a0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f63280cc5b0 tx=0x7f632000f4a0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.511+0000 7f632cbf4640 1 -- 172.21.15.60:0/3035065685 shutdown_connections 2024-04-25T10:13:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.511+0000 7f632cbf4640 1 --2- 172.21.15.60:0/3035065685 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6308066f90 0x7f6308069450 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.511+0000 7f632cbf4640 1 --2- 172.21.15.60:0/3035065685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f632813dcf0 0x7f632813e1a0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.511+0000 7f632cbf4640 1 --2- 172.21.15.60:0/3035065685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f632806efd0 0x7f632806d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.511+0000 7f632cbf4640 1 --2- 172.21.15.60:0/3035065685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63100039f0 0x7f63280af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.511+0000 7f632cbf4640 1 -- 172.21.15.60:0/3035065685 >> 172.21.15.60:0/3035065685 conn(0x7f63280670b0 msgr2=0x7f63280b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.515+0000 7f632cbf4640 1 -- 172.21.15.60:0/3035065685 shutdown_connections 2024-04-25T10:13:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.515+0000 7f632cbf4640 1 -- 172.21.15.60:0/3035065685 wait complete. 2024-04-25T10:13:11.535 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:11.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b3459640 1 Processor -- start 2024-04-25T10:13:11.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b3459640 1 -- start start 2024-04-25T10:13:11.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b3459640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f60ac15ef20 0x7f60ac0b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b3459640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60ac0cfcf0 0x7f60ac0b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b3459640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ac0cc320 con 0x7f60ac099a30 2024-04-25T10:13:11.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b3459640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ac0cc5c0 con 0x7f60ac15ef20 2024-04-25T10:13:11.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b3459640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac0d07b0 con 0x7f60ac0cfcf0 2024-04-25T10:13:11.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b09f4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60ac0cfcf0 0x7f60ac0b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b11f5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f60ac15ef20 0x7f60ac0b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b09f4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60ac0cfcf0 0x7f60ac0b4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:59194/0 (socket says 172.21.15.60:59194) 2024-04-25T10:13:11.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b11f5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f60ac15ef20 0x7f60ac0b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:50034/0 (socket says 172.21.15.60:50034) 2024-04-25T10:13:11.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b09f4640 1 -- 172.21.15.60:0/92957160 learned_addr learned my addr 172.21.15.60:0/92957160 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:11.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b19f6640 1 --1- 172.21.15.60:0/92957160 >> v1:172.21.15.100:6789/0 conn(0x7f60ac099a30 0x7f60ac0cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:51484/0 (socket says 172.21.15.60:51484) 2024-04-25T10:13:11.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b09f4640 1 -- 172.21.15.60:0/92957160 >> v2:172.21.15.100:3300/0 conn(0x7f60ac15ef20 msgr2=0x7f60ac0b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b09f4640 1 --2- 172.21.15.60:0/92957160 >> v2:172.21.15.100:3300/0 conn(0x7f60ac15ef20 0x7f60ac0b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b09f4640 1 -- 172.21.15.60:0/92957160 >> v1:172.21.15.100:6789/0 conn(0x7f60ac099a30 legacy=0x7f60ac0cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.659+0000 7f60b09f4640 1 -- 172.21.15.60:0/92957160 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac0d04d0 con 0x7f60ac0cfcf0 2024-04-25T10:13:11.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.663+0000 7f60b09f4640 1 --2- 172.21.15.60:0/92957160 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60ac0cfcf0 0x7f60ac0b4560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f609c009870 tx=0x7f609c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b1dfa1cada98e0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.663+0000 7f60a3fff640 1 -- 172.21.15.60:0/92957160 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c0b2070 con 0x7f60ac0cfcf0 2024-04-25T10:13:11.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.663+0000 7f60a3fff640 1 -- 172.21.15.60:0/92957160 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609c0a9070 con 0x7f60ac0cfcf0 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.663+0000 7f60a3fff640 1 -- 172.21.15.60:0/92957160 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c0ad880 con 0x7f60ac0cfcf0 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.663+0000 7f60b3459640 1 -- 172.21.15.60:0/92957160 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60ac0cfcf0 msgr2=0x7f60ac0b4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.663+0000 7f60b3459640 1 --2- 172.21.15.60:0/92957160 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60ac0cfcf0 0x7f60ac0b4560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f609c009870 tx=0x7f609c01c560 comp rx=0 tx=0).stop 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.663+0000 7f60b3459640 1 -- 172.21.15.60:0/92957160 shutdown_connections 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.663+0000 7f60b3459640 1 --2- 172.21.15.60:0/92957160 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60ac0cfcf0 0x7f60ac0b4560 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.663+0000 7f60b3459640 1 --2- 172.21.15.60:0/92957160 >> v2:172.21.15.100:3300/0 conn(0x7f60ac15ef20 0x7f60ac0b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.663+0000 7f60b3459640 1 -- 172.21.15.60:0/92957160 >> 172.21.15.60:0/92957160 conn(0x7f60ac0b3200 msgr2=0x7f60ac0b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.663+0000 7f60b3459640 1 -- 172.21.15.60:0/92957160 shutdown_connections 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.663+0000 7f60b3459640 1 -- 172.21.15.60:0/92957160 wait complete. 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.663+0000 7f60b3459640 1 Processor -- start 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.663+0000 7f60b3459640 1 -- start start 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b3459640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ac099a30 0x7f60ac147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b3459640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60ac0cfcf0 0x7f60ac06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b3459640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60ac15ef20 0x7f60ac15cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b3459640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac09a250 con 0x7f60ac15ef20 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b3459640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac0d0170 con 0x7f60ac099a30 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b3459640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac06cc40 con 0x7f60ac0cfcf0 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b19f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60ac15ef20 0x7f60ac15cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b19f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60ac15ef20 0x7f60ac15cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60104/0 (socket says 172.21.15.60:60104) 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b19f6640 1 -- 172.21.15.60:0/2263681078 learned_addr learned my addr 172.21.15.60:0/2263681078 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b09f4640 1 --2- 172.21.15.60:0/2263681078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60ac0cfcf0 0x7f60ac06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b11f5640 1 --2- 172.21.15.60:0/2263681078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ac099a30 0x7f60ac147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b19f6640 1 -- 172.21.15.60:0/2263681078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60ac0cfcf0 msgr2=0x7f60ac06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b19f6640 1 --2- 172.21.15.60:0/2263681078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60ac0cfcf0 0x7f60ac06c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b19f6640 1 -- 172.21.15.60:0/2263681078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ac099a30 msgr2=0x7f60ac147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b19f6640 1 --2- 172.21.15.60:0/2263681078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ac099a30 0x7f60ac147bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b19f6640 1 -- 172.21.15.60:0/2263681078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac06cf20 con 0x7f60ac15ef20 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b19f6640 1 --2- 172.21.15.60:0/2263681078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60ac15ef20 0x7f60ac15cd00 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f60a400c740 tx=0x7f60a400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60a1ffb640 1 -- 172.21.15.60:0/2263681078 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a4017070 con 0x7f60ac15ef20 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60a1ffb640 1 -- 172.21.15.60:0/2263681078 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a400ce50 con 0x7f60ac15ef20 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b3459640 1 -- 172.21.15.60:0/2263681078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ac06d200 con 0x7f60ac15ef20 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60a1ffb640 1 -- 172.21.15.60:0/2263681078 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a400a860 con 0x7f60ac15ef20 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b3459640 1 -- 172.21.15.60:0/2263681078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac06d7c0 con 0x7f60ac15ef20 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b3459640 1 -- 172.21.15.60:0/2263681078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ac0b7600 con 0x7f60ac15ef20 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60a1ffb640 1 -- 172.21.15.60:0/2263681078 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f60a401a070 con 0x7f60ac15ef20 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60a1ffb640 1 --2- 172.21.15.60:0/2263681078 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6090066f60 0x7f6090069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60a1ffb640 1 -- 172.21.15.60:0/2263681078 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f60a4084650 con 0x7f60ac15ef20 2024-04-25T10:13:11.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.667+0000 7f60b11f5640 1 --2- 172.21.15.60:0/2263681078 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6090066f60 0x7f6090069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.671+0000 7f60b11f5640 1 --2- 172.21.15.60:0/2263681078 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6090066f60 0x7f6090069420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009a50 tx=0x7f60a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.671+0000 7f60a1ffb640 1 -- 172.21.15.60:0/2263681078 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f60a4054f20 con 0x7f60ac15ef20 2024-04-25T10:13:11.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.831+0000 7f60b3459640 1 -- 172.21.15.60:0/2263681078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f60ac074770 con 0x7f60ac15ef20 2024-04-25T10:13:11.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.831+0000 7f60a1ffb640 1 -- 172.21.15.60:0/2263681078 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f60a4058fc0 con 0x7f60ac15ef20 2024-04-25T10:13:11.836 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:13:11.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.831+0000 7f60b3459640 1 -- 172.21.15.60:0/2263681078 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6090066f60 msgr2=0x7f6090069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.831+0000 7f60b3459640 1 --2- 172.21.15.60:0/2263681078 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6090066f60 0x7f6090069420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009a50 tx=0x7f60a8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.831+0000 7f60b3459640 1 -- 172.21.15.60:0/2263681078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60ac15ef20 msgr2=0x7f60ac15cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.831+0000 7f60b3459640 1 --2- 172.21.15.60:0/2263681078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60ac15ef20 0x7f60ac15cd00 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f60a400c740 tx=0x7f60a400cc00 comp rx=0 tx=0).stop 2024-04-25T10:13:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.831+0000 7f60b3459640 1 -- 172.21.15.60:0/2263681078 shutdown_connections 2024-04-25T10:13:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.831+0000 7f60b3459640 1 --2- 172.21.15.60:0/2263681078 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6090066f60 0x7f6090069420 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.831+0000 7f60b3459640 1 --2- 172.21.15.60:0/2263681078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60ac15ef20 0x7f60ac15cd00 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.831+0000 7f60b3459640 1 --2- 172.21.15.60:0/2263681078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60ac0cfcf0 0x7f60ac06c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.831+0000 7f60b3459640 1 --2- 172.21.15.60:0/2263681078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ac099a30 0x7f60ac147bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.831+0000 7f60b3459640 1 -- 172.21.15.60:0/2263681078 >> 172.21.15.60:0/2263681078 conn(0x7f60ac0b3200 msgr2=0x7f60ac0cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.831+0000 7f60b3459640 1 -- 172.21.15.60:0/2263681078 shutdown_connections 2024-04-25T10:13:11.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.835+0000 7f60b3459640 1 -- 172.21.15.60:0/2263681078 wait complete. 2024-04-25T10:13:11.854 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:13:11.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.963+0000 7f70bdcf3640 1 Processor -- start 2024-04-25T10:13:11.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.963+0000 7f70bdcf3640 1 -- start start 2024-04-25T10:13:11.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.963+0000 7f70bdcf3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f70b815deb0 0x7f70b80b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.963+0000 7f70bdcf3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b806d200 con 0x7f70b80ce9c0 2024-04-25T10:13:11.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.963+0000 7f70bdcf3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b80cefe0 con 0x7f70b80d0340 2024-04-25T10:13:11.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.963+0000 7f70bdcf3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b80cf2c0 con 0x7f70b815deb0 2024-04-25T10:13:11.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.963+0000 7f70b7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f70b80ce9c0 0x7f70b80d2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58516/0 (socket says 172.21.15.60:58516) 2024-04-25T10:13:11.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.963+0000 7f70b7fff640 1 -- 172.21.15.60:0/3580068992 learned_addr learned my addr 172.21.15.60:0/3580068992 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:13:11.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b77fe640 1 --2- 172.21.15.60:0/3580068992 >> v2:172.21.15.100:3300/0 conn(0x7f70b815deb0 0x7f70b80b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 -- 172.21.15.60:0/3580068992 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864151943 0 0) 0x7f70b806d200 con 0x7f70b80ce9c0 2024-04-25T10:13:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 -- 172.21.15.60:0/3580068992 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70a0003530 con 0x7f70b80ce9c0 2024-04-25T10:13:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 -- 172.21.15.60:0/3580068992 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1049356305 0 0) 0x7f70a0003530 con 0x7f70b80ce9c0 2024-04-25T10:13:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 -- 172.21.15.60:0/3580068992 >> v2:172.21.15.100:3300/0 conn(0x7f70b815deb0 msgr2=0x7f70b80b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 --2- 172.21.15.60:0/3580068992 >> v2:172.21.15.100:3300/0 conn(0x7f70b815deb0 0x7f70b80b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 -- 172.21.15.60:0/3580068992 >> v1:172.21.15.100:6789/0 conn(0x7f70b80d0340 legacy=0x7f70b80ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 -- 172.21.15.60:0/3580068992 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b80d1d70 con 0x7f70b80ce9c0 2024-04-25T10:13:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 -- 172.21.15.60:0/3580068992 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f70b0001a00 con 0x7f70b80ce9c0 2024-04-25T10:13:11.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 -- 172.21.15.60:0/3580068992 >> v1:172.21.15.60:6789/0 conn(0x7f70b80ce9c0 legacy=0x7f70b80d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 --2- 172.21.15.60:0/3580068992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a0003a10 0x7f70a0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 --2- 172.21.15.60:0/3580068992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a0004810 0x7f70a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 --2- 172.21.15.60:0/3580068992 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70a00051d0 0x7f70a000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 -- 172.21.15.60:0/3580068992 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b80cf2c0 con 0x7f70a0004810 2024-04-25T10:13:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 -- 172.21.15.60:0/3580068992 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a000c0d0 con 0x7f70a0003a10 2024-04-25T10:13:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 -- 172.21.15.60:0/3580068992 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a000c2d0 con 0x7f70a00051d0 2024-04-25T10:13:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b6ffd640 1 --2- 172.21.15.60:0/3580068992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a0004810 0x7f70a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b77fe640 1 --2- 172.21.15.60:0/3580068992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a0003a10 0x7f70a0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b7fff640 1 --2- 172.21.15.60:0/3580068992 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70a00051d0 0x7f70a000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b6ffd640 1 -- 172.21.15.60:0/3580068992 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70a00051d0 msgr2=0x7f70a000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b6ffd640 1 --2- 172.21.15.60:0/3580068992 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70a00051d0 0x7f70a000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b6ffd640 1 -- 172.21.15.60:0/3580068992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a0003a10 msgr2=0x7f70a0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b6ffd640 1 --2- 172.21.15.60:0/3580068992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a0003a10 0x7f70a0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b6ffd640 1 -- 172.21.15.60:0/3580068992 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a0003530 con 0x7f70a0004810 2024-04-25T10:13:11.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b7fff640 1 --2- 172.21.15.60:0/3580068992 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70a00051d0 0x7f70a000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:11.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b77fe640 1 --2- 172.21.15.60:0/3580068992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a0003a10 0x7f70a0003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:11.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b7fff640 1 -- 172.21.15.60:0/3580068992 reap_dead start 2024-04-25T10:13:11.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b6ffd640 1 --2- 172.21.15.60:0/3580068992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a0004810 0x7f70a0004c90 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f70a8001780 tx=0x7f70a801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a675fe607f4c738 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 -- 172.21.15.60:0/3580068992 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a80013a0 con 0x7f70a0004810 2024-04-25T10:13:11.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 -- 172.21.15.60:0/3580068992 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a80050d0 con 0x7f70a0004810 2024-04-25T10:13:11.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.967+0000 7f70b67fc640 1 -- 172.21.15.60:0/3580068992 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8003600 con 0x7f70a0004810 2024-04-25T10:13:11.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 -- 172.21.15.60:0/3580068992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a0004810 msgr2=0x7f70a0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 --2- 172.21.15.60:0/3580068992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a0004810 0x7f70a0004c90 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f70a8001780 tx=0x7f70a801c400 comp rx=0 tx=0).stop 2024-04-25T10:13:11.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 -- 172.21.15.60:0/3580068992 shutdown_connections 2024-04-25T10:13:11.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 --2- 172.21.15.60:0/3580068992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a0004810 0x7f70a0004c90 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 -- 172.21.15.60:0/3580068992 >> 172.21.15.60:0/3580068992 conn(0x7f70b806c090 msgr2=0x7f70b806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 -- 172.21.15.60:0/3580068992 shutdown_connections 2024-04-25T10:13:11.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 -- 172.21.15.60:0/3580068992 wait complete. 2024-04-25T10:13:11.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 Processor -- start 2024-04-25T10:13:11.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 -- start start 2024-04-25T10:13:11.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a0004810 0x7f70b81514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70b814ede0 0x7f70b80b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70b80b86e0 0x7f70b80cc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b80b6c10 con 0x7f70b80b86e0 2024-04-25T10:13:11.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b80d2330 con 0x7f70a0004810 2024-04-25T10:13:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b80b6d90 con 0x7f70b814ede0 2024-04-25T10:13:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70b7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70b80b86e0 0x7f70b80cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70b7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70b80b86e0 0x7f70b80cc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60118/0 (socket says 172.21.15.60:60118) 2024-04-25T10:13:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70b7fff640 1 -- 172.21.15.60:0/1817905050 learned_addr learned my addr 172.21.15.60:0/1817905050 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70b77fe640 1 --2- 172.21.15.60:0/1817905050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a0004810 0x7f70b81514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70b6ffd640 1 --2- 172.21.15.60:0/1817905050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70b814ede0 0x7f70b80b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70b7fff640 1 -- 172.21.15.60:0/1817905050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70b814ede0 msgr2=0x7f70b80b81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70b7fff640 1 --2- 172.21.15.60:0/1817905050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70b814ede0 0x7f70b80b81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70b7fff640 1 -- 172.21.15.60:0/1817905050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a0004810 msgr2=0x7f70b81514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70b7fff640 1 --2- 172.21.15.60:0/1817905050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a0004810 0x7f70b81514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70b7fff640 1 -- 172.21.15.60:0/1817905050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b81f49e0 con 0x7f70b80b86e0 2024-04-25T10:13:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70b7fff640 1 --2- 172.21.15.60:0/1817905050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70b80b86e0 0x7f70b80cc050 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f70b0001340 tx=0x7f70b0010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f709ffff640 1 -- 172.21.15.60:0/1817905050 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b001a070 con 0x7f70b80b86e0 2024-04-25T10:13:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f709ffff640 1 -- 172.21.15.60:0/1817905050 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b0011040 con 0x7f70b80b86e0 2024-04-25T10:13:11.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 -- 172.21.15.60:0/1817905050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b8150700 con 0x7f70b80b86e0 2024-04-25T10:13:11.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f709ffff640 1 -- 172.21.15.60:0/1817905050 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b000eb10 con 0x7f70b80b86e0 2024-04-25T10:13:11.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 -- 172.21.15.60:0/1817905050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b8150ca0 con 0x7f70b80b86e0 2024-04-25T10:13:11.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.971+0000 7f70bdcf3640 1 -- 172.21.15.60:0/1817905050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b8074770 con 0x7f70b80b86e0 2024-04-25T10:13:11.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.975+0000 7f709ffff640 1 -- 172.21.15.60:0/1817905050 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f70b001d070 con 0x7f70b80b86e0 2024-04-25T10:13:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.975+0000 7f709ffff640 1 --2- 172.21.15.60:0/1817905050 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f70a4066f90 0x7f70a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.975+0000 7f709ffff640 1 -- 172.21.15.60:0/1817905050 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f70b0087fa0 con 0x7f70b80b86e0 2024-04-25T10:13:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.975+0000 7f70b77fe640 1 --2- 172.21.15.60:0/1817905050 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f70a4066f90 0x7f70a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.975+0000 7f70b77fe640 1 --2- 172.21.15.60:0/1817905050 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f70a4066f90 0x7f70a4069450 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f70b806a4c0 tx=0x7f70ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:11.979+0000 7f709ffff640 1 -- 172.21.15.60:0/1817905050 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f70b0024070 con 0x7f70b80b86e0 2024-04-25T10:13:12.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:12.147+0000 7f70bdcf3640 1 -- 172.21.15.60:0/1817905050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f70b80d17a0 con 0x7f70b80b86e0 2024-04-25T10:13:12.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:12.147+0000 7f709ffff640 1 -- 172.21.15.60:0/1817905050 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v85) v1 ==== 80+0+122 (secure 0 0 0) 0x7f70b81f49e0 con 0x7f70b80b86e0 2024-04-25T10:13:12.154 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6835/2455562778 2024-04-26T10:11:58.136789+0000 2024-04-25T10:13:12.154 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6833/2455562778 2024-04-26T10:11:58.136789+0000 2024-04-25T10:13:12.155 INFO:teuthology.orchestra.run.smithi060.stderr:listed 2 entries 2024-04-25T10:13:12.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:12.151+0000 7f70bdcf3640 1 -- 172.21.15.60:0/1817905050 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f70a4066f90 msgr2=0x7f70a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:12.151+0000 7f70bdcf3640 1 --2- 172.21.15.60:0/1817905050 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f70a4066f90 0x7f70a4069450 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f70b806a4c0 tx=0x7f70ac008040 comp rx=0 tx=0).stop 2024-04-25T10:13:12.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:12.151+0000 7f70bdcf3640 1 -- 172.21.15.60:0/1817905050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70b80b86e0 msgr2=0x7f70b80cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:12.151+0000 7f70bdcf3640 1 --2- 172.21.15.60:0/1817905050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70b80b86e0 0x7f70b80cc050 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f70b0001340 tx=0x7f70b0010c30 comp rx=0 tx=0).stop 2024-04-25T10:13:12.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:12.151+0000 7f70bdcf3640 1 -- 172.21.15.60:0/1817905050 shutdown_connections 2024-04-25T10:13:12.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:12.151+0000 7f70bdcf3640 1 --2- 172.21.15.60:0/1817905050 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f70a4066f90 0x7f70a4069450 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:12.151+0000 7f70bdcf3640 1 --2- 172.21.15.60:0/1817905050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70b80b86e0 0x7f70b80cc050 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:12.151+0000 7f70bdcf3640 1 --2- 172.21.15.60:0/1817905050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70b814ede0 0x7f70b80b81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:12.151+0000 7f70bdcf3640 1 --2- 172.21.15.60:0/1817905050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a0004810 0x7f70b81514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:12.151+0000 7f70bdcf3640 1 -- 172.21.15.60:0/1817905050 >> 172.21.15.60:0/1817905050 conn(0x7f70b806c090 msgr2=0x7f70b806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:12.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:12.151+0000 7f70bdcf3640 1 -- 172.21.15.60:0/1817905050 shutdown_connections 2024-04-25T10:13:12.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:12.151+0000 7f70bdcf3640 1 -- 172.21.15.60:0/1817905050 wait complete. 2024-04-25T10:13:12.171 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:13:12.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:12.172 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:12.183 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[46062]: fuse finished with error 0 and tester_r 0 2024-04-25T10:13:12.184 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:17.469 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:17.473 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:18.287 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:13:18.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:18.287 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:13:18.295 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:18.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.467+0000 7fa535c03640 1 Processor -- start 2024-04-25T10:13:18.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.467+0000 7fa535c03640 1 -- start start 2024-04-25T10:13:18.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.471+0000 7fa535c03640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa53015a7a0 0x7fa5300cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.471+0000 7fa535c03640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa53006ec30 0x7fa530071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.471+0000 7fa535c03640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5300721e0 con 0x7fa5300b37e0 2024-04-25T10:13:18.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.471+0000 7fa535c03640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5300cdaa0 con 0x7fa53015a7a0 2024-04-25T10:13:18.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.471+0000 7fa535c03640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53006d2f0 con 0x7fa53006ec30 2024-04-25T10:13:18.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.471+0000 7fa52f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa53015a7a0 0x7fa5300cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.471+0000 7fa52f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa53015a7a0 0x7fa5300cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60128/0 (socket says 172.21.15.60:60128) 2024-04-25T10:13:18.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.471+0000 7fa52f7fe640 1 -- 172.21.15.60:0/4028717447 learned_addr learned my addr 172.21.15.60:0/4028717447 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:18.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.471+0000 7fa526ffd640 1 --2- 172.21.15.60:0/4028717447 >> v2:172.21.15.100:3300/0 conn(0x7fa53006ec30 0x7fa530071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.471+0000 7fa52f7fe640 1 -- 172.21.15.60:0/4028717447 >> v2:172.21.15.100:3300/0 conn(0x7fa53006ec30 msgr2=0x7fa530071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:18.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.471+0000 7fa52f7fe640 1 --2- 172.21.15.60:0/4028717447 >> v2:172.21.15.100:3300/0 conn(0x7fa53006ec30 0x7fa530071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.471+0000 7fa52f7fe640 1 -- 172.21.15.60:0/4028717447 >> v1:172.21.15.100:6789/0 conn(0x7fa5300b37e0 legacy=0x7fa53006dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.471+0000 7fa52f7fe640 1 -- 172.21.15.60:0/4028717447 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53006f130 con 0x7fa53015a7a0 2024-04-25T10:13:18.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa52f7fe640 1 --2- 172.21.15.60:0/4028717447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53015a7a0 0x7fa5300cd560 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa530072bf0 tx=0x7fa52001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5918eedf8372cd18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa52effd640 1 -- 172.21.15.60:0/4028717447 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5200ab430 con 0x7fa53015a7a0 2024-04-25T10:13:18.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa52effd640 1 -- 172.21.15.60:0/4028717447 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5200b0070 con 0x7fa53015a7a0 2024-04-25T10:13:18.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa52effd640 1 -- 172.21.15.60:0/4028717447 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5200ab7f0 con 0x7fa53015a7a0 2024-04-25T10:13:18.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa535c03640 1 -- 172.21.15.60:0/4028717447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53015a7a0 msgr2=0x7fa5300cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa535c03640 1 --2- 172.21.15.60:0/4028717447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53015a7a0 0x7fa5300cd560 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa530072bf0 tx=0x7fa52001c520 comp rx=0 tx=0).stop 2024-04-25T10:13:18.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa535c03640 1 -- 172.21.15.60:0/4028717447 shutdown_connections 2024-04-25T10:13:18.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa535c03640 1 --2- 172.21.15.60:0/4028717447 >> v2:172.21.15.100:3300/0 conn(0x7fa53006ec30 0x7fa530071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa535c03640 1 --2- 172.21.15.60:0/4028717447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53015a7a0 0x7fa5300cd560 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa535c03640 1 -- 172.21.15.60:0/4028717447 >> 172.21.15.60:0/4028717447 conn(0x7fa5300b12d0 msgr2=0x7fa5300b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:18.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa535c03640 1 -- 172.21.15.60:0/4028717447 shutdown_connections 2024-04-25T10:13:18.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa535c03640 1 -- 172.21.15.60:0/4028717447 wait complete. 2024-04-25T10:13:18.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa535c03640 1 Processor -- start 2024-04-25T10:13:18.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa535c03640 1 -- start start 2024-04-25T10:13:18.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa535c03640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa53006ec30 0x7fa53015aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa535c03640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5300b37e0 0x7fa530155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa535c03640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53015a7a0 0x7fa530163b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa535c03640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53015ad00 con 0x7fa53015a7a0 2024-04-25T10:13:18.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa535c03640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53015ab80 con 0x7fa5300b37e0 2024-04-25T10:13:18.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa535c03640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53006e860 con 0x7fa53006ec30 2024-04-25T10:13:18.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa52ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53015a7a0 0x7fa530163b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa52ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53015a7a0 0x7fa530163b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60140/0 (socket says 172.21.15.60:60140) 2024-04-25T10:13:18.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa52ffff640 1 -- 172.21.15.60:0/1804880310 learned_addr learned my addr 172.21.15.60:0/1804880310 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa52f7fe640 1 --2- 172.21.15.60:0/1804880310 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa53006ec30 0x7fa53015aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa526ffd640 1 --2- 172.21.15.60:0/1804880310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5300b37e0 0x7fa530155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa52ffff640 1 -- 172.21.15.60:0/1804880310 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa53006ec30 msgr2=0x7fa53015aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa52ffff640 1 --2- 172.21.15.60:0/1804880310 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa53006ec30 0x7fa53015aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa52ffff640 1 -- 172.21.15.60:0/1804880310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5300b37e0 msgr2=0x7fa530155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa52ffff640 1 --2- 172.21.15.60:0/1804880310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5300b37e0 0x7fa530155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa52ffff640 1 -- 172.21.15.60:0/1804880310 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53015b530 con 0x7fa53015a7a0 2024-04-25T10:13:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.475+0000 7fa52f7fe640 1 --2- 172.21.15.60:0/1804880310 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa53006ec30 0x7fa53015aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.479+0000 7fa52ffff640 1 --2- 172.21.15.60:0/1804880310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53015a7a0 0x7fa530163b00 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fa52800c770 tx=0x7fa52800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.479+0000 7fa52cff9640 1 -- 172.21.15.60:0/1804880310 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa528003b80 con 0x7fa53015a7a0 2024-04-25T10:13:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.479+0000 7fa52cff9640 1 -- 172.21.15.60:0/1804880310 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa528003d20 con 0x7fa53015a7a0 2024-04-25T10:13:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.479+0000 7fa535c03640 1 -- 172.21.15.60:0/1804880310 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53015b810 con 0x7fa53015a7a0 2024-04-25T10:13:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.479+0000 7fa52cff9640 1 -- 172.21.15.60:0/1804880310 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52800aa90 con 0x7fa53015a7a0 2024-04-25T10:13:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.479+0000 7fa535c03640 1 -- 172.21.15.60:0/1804880310 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53015bdd0 con 0x7fa53015a7a0 2024-04-25T10:13:18.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.479+0000 7fa535c03640 1 -- 172.21.15.60:0/1804880310 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa530074770 con 0x7fa53015a7a0 2024-04-25T10:13:18.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.479+0000 7fa52cff9640 1 -- 172.21.15.60:0/1804880310 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa52800ac30 con 0x7fa53015a7a0 2024-04-25T10:13:18.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.479+0000 7fa52cff9640 1 --2- 172.21.15.60:0/1804880310 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa514066f60 0x7fa514069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.479+0000 7fa52cff9640 1 -- 172.21.15.60:0/1804880310 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fa5280859f0 con 0x7fa53015a7a0 2024-04-25T10:13:18.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.479+0000 7fa52f7fe640 1 --2- 172.21.15.60:0/1804880310 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa514066f60 0x7fa514069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.483+0000 7fa52f7fe640 1 --2- 172.21.15.60:0/1804880310 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa514066f60 0x7fa514069420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa5200af000 tx=0x7fa5200a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.483+0000 7fa52cff9640 1 -- 172.21.15.60:0/1804880310 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa528056a90 con 0x7fa53015a7a0 2024-04-25T10:13:18.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.683+0000 7fa535c03640 1 -- 172.21.15.60:0/1804880310 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5300cd6c0 con 0x7fa53015a7a0 2024-04-25T10:13:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.683+0000 7fa52cff9640 1 -- 172.21.15.60:0/1804880310 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fa528056390 con 0x7fa53015a7a0 2024-04-25T10:13:18.688 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:13:18.688 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:12:09.160298+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7908},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7908":{"gid":7908,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":6,"addr":"172.21.15.60:6835/2817879567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2817879567},{"type":"v1","addr":"172.21.15.60:6835","nonce":2817879567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:13:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T10:13:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.683+0000 7fa535c03640 1 -- 172.21.15.60:0/1804880310 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa514066f60 msgr2=0x7fa514069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.683+0000 7fa535c03640 1 --2- 172.21.15.60:0/1804880310 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa514066f60 0x7fa514069420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa5200af000 tx=0x7fa5200a7000 comp rx=0 tx=0).stop 2024-04-25T10:13:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.683+0000 7fa535c03640 1 -- 172.21.15.60:0/1804880310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53015a7a0 msgr2=0x7fa530163b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.683+0000 7fa535c03640 1 --2- 172.21.15.60:0/1804880310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53015a7a0 0x7fa530163b00 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fa52800c770 tx=0x7fa52800cc30 comp rx=0 tx=0).stop 2024-04-25T10:13:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.683+0000 7fa535c03640 1 -- 172.21.15.60:0/1804880310 shutdown_connections 2024-04-25T10:13:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.683+0000 7fa535c03640 1 --2- 172.21.15.60:0/1804880310 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa514066f60 0x7fa514069420 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.683+0000 7fa535c03640 1 --2- 172.21.15.60:0/1804880310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa53015a7a0 0x7fa530163b00 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.683+0000 7fa535c03640 1 --2- 172.21.15.60:0/1804880310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5300b37e0 0x7fa530155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.683+0000 7fa535c03640 1 --2- 172.21.15.60:0/1804880310 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa53006ec30 0x7fa53015aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.683+0000 7fa535c03640 1 -- 172.21.15.60:0/1804880310 >> 172.21.15.60:0/1804880310 conn(0x7fa5300b12d0 msgr2=0x7fa5300ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.683+0000 7fa535c03640 1 -- 172.21.15.60:0/1804880310 shutdown_connections 2024-04-25T10:13:18.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.683+0000 7fa535c03640 1 -- 172.21.15.60:0/1804880310 wait complete. 2024-04-25T10:13:18.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:13:18.861 INFO:teuthology.orchestra.run.smithi060.stdout:{"message":"","return_code":0} 2024-04-25T10:13:18.861 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:13:18.861 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:13:18.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.979+0000 7f42ff2a2640 1 Processor -- start 2024-04-25T10:13:18.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.979+0000 7f42ff2a2640 1 -- start start 2024-04-25T10:13:18.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.979+0000 7f42ff2a2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42f815bc90 0x7f42f815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.979+0000 7f42ff2a2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f8071ca0 con 0x7f42f80ce7a0 2024-04-25T10:13:18.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.979+0000 7f42ff2a2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f80cca40 con 0x7f42f80b4660 2024-04-25T10:13:18.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.979+0000 7f42ff2a2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80cc370 con 0x7f42f815bc90 2024-04-25T10:13:18.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.979+0000 7f42fd03e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f42f80ce7a0 0x7f42f80ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56628/0 (socket says 172.21.15.60:56628) 2024-04-25T10:13:18.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.979+0000 7f42fd03e640 1 -- 172.21.15.60:0/3345598194 learned_addr learned my addr 172.21.15.60:0/3345598194 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:13:18.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42fc83d640 1 --2- 172.21.15.60:0/3345598194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42f815bc90 0x7f42f815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 -- 172.21.15.60:0/3345598194 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167436482 0 0) 0x7f42f8071ca0 con 0x7f42f80ce7a0 2024-04-25T10:13:18.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 -- 172.21.15.60:0/3345598194 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42e4003530 con 0x7f42f80ce7a0 2024-04-25T10:13:18.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 -- 172.21.15.60:0/3345598194 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 343414908 0 0) 0x7f42e4003530 con 0x7f42f80ce7a0 2024-04-25T10:13:18.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 -- 172.21.15.60:0/3345598194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42f815bc90 msgr2=0x7f42f815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:18.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 --2- 172.21.15.60:0/3345598194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42f815bc90 0x7f42f815c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 -- 172.21.15.60:0/3345598194 >> v1:172.21.15.100:6789/0 conn(0x7f42f80b4660 legacy=0x7f42f815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 -- 172.21.15.60:0/3345598194 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f80ccd20 con 0x7f42f80ce7a0 2024-04-25T10:13:18.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 -- 172.21.15.60:0/3345598194 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f42f4001680 con 0x7f42f80ce7a0 2024-04-25T10:13:18.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 -- 172.21.15.60:0/3345598194 >> v1:172.21.15.60:6789/0 conn(0x7f42f80ce7a0 legacy=0x7f42f80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 --2- 172.21.15.60:0/3345598194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42f815bc90 0x7f42e40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 --2- 172.21.15.60:0/3345598194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e4003f70 0x7f42e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 --2- 172.21.15.60:0/3345598194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42e4004d50 0x7f42e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 -- 172.21.15.60:0/3345598194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e400ba80 con 0x7f42e4003f70 2024-04-25T10:13:18.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 -- 172.21.15.60:0/3345598194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e400bc30 con 0x7f42e4004d50 2024-04-25T10:13:18.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 -- 172.21.15.60:0/3345598194 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e400be30 con 0x7f42f815bc90 2024-04-25T10:13:18.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42fc83d640 1 --2- 172.21.15.60:0/3345598194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e4003f70 0x7f42e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42fd83f640 1 --2- 172.21.15.60:0/3345598194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42e4004d50 0x7f42e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42fd03e640 1 --2- 172.21.15.60:0/3345598194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42f815bc90 0x7f42e40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42fc83d640 1 -- 172.21.15.60:0/3345598194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42f815bc90 msgr2=0x7f42e40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:18.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42fc83d640 1 --2- 172.21.15.60:0/3345598194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42f815bc90 0x7f42e40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42fc83d640 1 -- 172.21.15.60:0/3345598194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42e4004d50 msgr2=0x7f42e40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42fc83d640 1 --2- 172.21.15.60:0/3345598194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42e4004d50 0x7f42e40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42fc83d640 1 -- 172.21.15.60:0/3345598194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e4003530 con 0x7f42e4003f70 2024-04-25T10:13:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42fd83f640 1 --2- 172.21.15.60:0/3345598194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42e4004d50 0x7f42e40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42fd03e640 1 --2- 172.21.15.60:0/3345598194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42f815bc90 0x7f42e40039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:18.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42fc83d640 1 --2- 172.21.15.60:0/3345598194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e4003f70 0x7f42e4004810 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009ae0 tx=0x7f42e801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78437c2d4c0ef142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 -- 172.21.15.60:0/3345598194 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e80b2070 con 0x7f42e4003f70 2024-04-25T10:13:18.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 -- 172.21.15.60:0/3345598194 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e80a5660 con 0x7f42e4003f70 2024-04-25T10:13:18.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42effff640 1 -- 172.21.15.60:0/3345598194 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e80096b0 con 0x7f42e4003f70 2024-04-25T10:13:18.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42ff2a2640 1 -- 172.21.15.60:0/3345598194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e4003f70 msgr2=0x7f42e4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42ff2a2640 1 --2- 172.21.15.60:0/3345598194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e4003f70 0x7f42e4004810 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009ae0 tx=0x7f42e801ca70 comp rx=0 tx=0).stop 2024-04-25T10:13:18.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.983+0000 7f42fd83f640 1 -- 172.21.15.60:0/3345598194 reap_dead start 2024-04-25T10:13:18.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42ff2a2640 1 -- 172.21.15.60:0/3345598194 shutdown_connections 2024-04-25T10:13:18.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42ff2a2640 1 -- 172.21.15.60:0/3345598194 >> 172.21.15.60:0/3345598194 conn(0x7f42f80b12d0 msgr2=0x7f42f80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42ff2a2640 1 -- 172.21.15.60:0/3345598194 shutdown_connections 2024-04-25T10:13:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42ff2a2640 1 -- 172.21.15.60:0/3345598194 wait complete. 2024-04-25T10:13:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42ff2a2640 1 Processor -- start 2024-04-25T10:13:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42ff2a2640 1 -- start start 2024-04-25T10:13:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42ff2a2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e4003f70 0x7f42f8150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42ff2a2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f81f3420 0x7f42f81f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42ff2a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f8152720 0x7f42f8152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42ff2a2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80b3d20 con 0x7f42f8152720 2024-04-25T10:13:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42ff2a2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80b3ba0 con 0x7f42f81f3420 2024-04-25T10:13:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42ff2a2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80b4280 con 0x7f42e4003f70 2024-04-25T10:13:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42fd83f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f8152720 0x7f42f8152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42fd83f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f8152720 0x7f42f8152ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54902/0 (socket says 172.21.15.60:54902) 2024-04-25T10:13:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42fd83f640 1 -- 172.21.15.60:0/2245570647 learned_addr learned my addr 172.21.15.60:0/2245570647 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42fd03e640 1 --2- 172.21.15.60:0/2245570647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e4003f70 0x7f42f8150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42fc83d640 1 --2- 172.21.15.60:0/2245570647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f81f3420 0x7f42f81f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42fd83f640 1 -- 172.21.15.60:0/2245570647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e4003f70 msgr2=0x7f42f8150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42fd83f640 1 --2- 172.21.15.60:0/2245570647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e4003f70 0x7f42f8150bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42fd83f640 1 -- 172.21.15.60:0/2245570647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f81f3420 msgr2=0x7f42f81f38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42fd83f640 1 --2- 172.21.15.60:0/2245570647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f81f3420 0x7f42f81f38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42fd83f640 1 -- 172.21.15.60:0/2245570647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f8151100 con 0x7f42f8152720 2024-04-25T10:13:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42fd83f640 1 --2- 172.21.15.60:0/2245570647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f8152720 0x7f42f8152ba0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f42f80b4010 tx=0x7f42f000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42edffb640 1 -- 172.21.15.60:0/2245570647 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0018070 con 0x7f42f8152720 2024-04-25T10:13:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42edffb640 1 -- 172.21.15.60:0/2245570647 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f0003b80 con 0x7f42f8152720 2024-04-25T10:13:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42ff2a2640 1 -- 172.21.15.60:0/2245570647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f81f3f30 con 0x7f42f8152720 2024-04-25T10:13:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42edffb640 1 -- 172.21.15.60:0/2245570647 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0005070 con 0x7f42f8152720 2024-04-25T10:13:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.987+0000 7f42ff2a2640 1 -- 172.21.15.60:0/2245570647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f80acab0 con 0x7f42f8152720 2024-04-25T10:13:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.991+0000 7f42ff2a2640 1 -- 172.21.15.60:0/2245570647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f80ac8a0 con 0x7f42f8152720 2024-04-25T10:13:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.991+0000 7f42edffb640 1 -- 172.21.15.60:0/2245570647 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f42f0005210 con 0x7f42f8152720 2024-04-25T10:13:18.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.991+0000 7f42edffb640 1 --2- 172.21.15.60:0/2245570647 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f42dc066f90 0x7f42dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.991+0000 7f42edffb640 1 -- 172.21.15.60:0/2245570647 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f42f0084710 con 0x7f42f8152720 2024-04-25T10:13:18.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.991+0000 7f42fd03e640 1 --2- 172.21.15.60:0/2245570647 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f42dc066f90 0x7f42dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.991+0000 7f42fd03e640 1 --2- 172.21.15.60:0/2245570647 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f42dc066f90 0x7f42dc069450 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f42f4001000 tx=0x7f42f40014d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:18.995+0000 7f42edffb640 1 -- 172.21.15.60:0/2245570647 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f42f0055630 con 0x7f42f8152720 2024-04-25T10:13:19.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.191+0000 7f42ff2a2640 1 -- 172.21.15.60:0/2245570647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f42f8074770 con 0x7f42f8152720 2024-04-25T10:13:19.195 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:13:19.191+0000 7f3a792d6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:13:19.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.191+0000 7f42edffb640 1 -- 172.21.15.60:0/2245570647 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7f42f0054f30 con 0x7f42f8152720 2024-04-25T10:13:19.198 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.195+0000 7f42ff2a2640 1 -- 172.21.15.60:0/2245570647 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f42dc066f90 msgr2=0x7f42dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.195+0000 7f42ff2a2640 1 --2- 172.21.15.60:0/2245570647 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f42dc066f90 0x7f42dc069450 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f42f4001000 tx=0x7f42f40014d0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.195+0000 7f42ff2a2640 1 -- 172.21.15.60:0/2245570647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f8152720 msgr2=0x7f42f8152ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.195+0000 7f42ff2a2640 1 --2- 172.21.15.60:0/2245570647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f8152720 0x7f42f8152ba0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f42f80b4010 tx=0x7f42f000dc00 comp rx=0 tx=0).stop 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.195+0000 7f42ff2a2640 1 -- 172.21.15.60:0/2245570647 shutdown_connections 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.195+0000 7f42ff2a2640 1 --2- 172.21.15.60:0/2245570647 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f42dc066f90 0x7f42dc069450 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.195+0000 7f42ff2a2640 1 --2- 172.21.15.60:0/2245570647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f8152720 0x7f42f8152ba0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.195+0000 7f42ff2a2640 1 --2- 172.21.15.60:0/2245570647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f81f3420 0x7f42f81f38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.195+0000 7f42ff2a2640 1 --2- 172.21.15.60:0/2245570647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e4003f70 0x7f42f8150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.195+0000 7f42ff2a2640 1 -- 172.21.15.60:0/2245570647 >> 172.21.15.60:0/2245570647 conn(0x7f42f80b12d0 msgr2=0x7f42f8071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.195+0000 7f42ff2a2640 1 -- 172.21.15.60:0/2245570647 shutdown_connections 2024-04-25T10:13:19.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.195+0000 7f42ff2a2640 1 -- 172.21.15.60:0/2245570647 wait complete. 2024-04-25T10:13:19.214 INFO:tasks.ceph.mds.c.smithi060.stderr: -4765> 2024-04-25T10:12:04.148+0000 7ff06ad61640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:19.216 INFO:tasks.ceph.mds.c.smithi060.stderr: -4378> 2024-04-25T10:12:06.312+0000 7ff066d59640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-25T10:13:19.216 INFO:tasks.ceph.mds.c.smithi060.stderr: -4216> 2024-04-25T10:12:06.540+0000 7ff066d59640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-04-25T10:11:53.583631+0000 1=1+0) n(v0 rc2024-04-25T10:11:53.694129+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x55e103712100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:11:53.583631+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:11:53.694129+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-25T10:13:19.219 DEBUG:teuthology.orchestra.run.smithi060:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-25T10:13:19.221 INFO:tasks.ceph.mds.c.smithi060.stderr: -2496> 2024-04-25T10:12:37.940+0000 7ff066d59640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv16 ap=3 DIRTYPARENT f(v0 m2024-04-25T10:11:53.583631+0000 1=1+0)->f(v1 m2024-04-25T10:11:53.583631+0000) n(v0 rc2024-04-25T10:11:53.694129+0000 2=1+1)->n(v0 rc2024-04-25T10:11:53.694129+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55e103712100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:11:53.583631+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:11:53.694129+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-25T10:13:19.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.247+0000 7f0897d8b8c0 1 -- start start 2024-04-25T10:13:19.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.247+0000 7f0897d8b8c0 1 Processor -- start 2024-04-25T10:13:19.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.247+0000 7f0897d8b8c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56271d361800 0x56271d51a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.247+0000 7f0897d8b8c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271d361400 0x56271d51ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.247+0000 7f0897d8b8c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271d361000 0x56271d51b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.247+0000 7f0897d8b8c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271e264180 con 0x56271d361000 2024-04-25T10:13:19.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.247+0000 7f0897d8b8c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271e264300 con 0x56271d361400 2024-04-25T10:13:19.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.247+0000 7f0897d8b8c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56271e264480 con 0x56271d361800 2024-04-25T10:13:19.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.247+0000 7f0896d4b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271d361000 0x56271d51b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.247+0000 7f0896d4b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271d361000 0x56271d51b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54920/0 (socket says 172.21.15.60:54920) 2024-04-25T10:13:19.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.247+0000 7f0896d4b640 1 -- 172.21.15.60:0/3502374876 learned_addr learned my addr 172.21.15.60:0/3502374876 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:19.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0896d4b640 1 -- 172.21.15.60:0/3502374876 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56271d361800 msgr2=0x56271d51a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0896d4b640 1 --2- 172.21.15.60:0/3502374876 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56271d361800 0x56271d51a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0896d4b640 1 -- 172.21.15.60:0/3502374876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271d361400 msgr2=0x56271d51ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0896d4b640 1 --2- 172.21.15.60:0/3502374876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271d361400 0x56271d51ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0896d4b640 1 -- 172.21.15.60:0/3502374876 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56271d3861e0 con 0x56271d361000 2024-04-25T10:13:19.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0896d4b640 1 --2- 172.21.15.60:0/3502374876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271d361000 0x56271d51b080 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x56271e1a9b60 tx=0x56271d43a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d7d896f03ca650 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0893d45640 1 -- 172.21.15.60:0/3502374876 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56271d3964e0 con 0x56271d361000 2024-04-25T10:13:19.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0893d45640 1 -- 172.21.15.60:0/3502374876 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56271d3aa1c0 con 0x56271d361000 2024-04-25T10:13:19.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0893d45640 1 -- 172.21.15.60:0/3502374876 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56271d396b60 con 0x56271d361000 2024-04-25T10:13:19.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3502374876 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56271d3874a0 con 0x56271d361000 2024-04-25T10:13:19.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3502374876 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56271d387a40 con 0x56271d361000 2024-04-25T10:13:19.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 4 waiting for MDS map... 2024-04-25T10:13:19.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0893d45640 1 -- 172.21.15.60:0/3502374876 <== mon.0 v2:172.21.15.60:3300/0 4 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x56271e272600 con 0x56271d361000 2024-04-25T10:13:19.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0893d45640 1 -- 172.21.15.60:0/3502374876 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 63) v1 ==== 2723+0+0 (secure 0 0 0) 0x56271d3b7180 con 0x56271d361000 2024-04-25T10:13:19.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 4 Got MDS map 63 2024-04-25T10:13:19.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 4 datascan.main: Using metadata pool output 2024-04-25T10:13:19.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 4 datascan.main: connecting to RADOS... 2024-04-25T10:13:19.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 Processor -- start 2024-04-25T10:13:19.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 -- start start 2024-04-25T10:13:19.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271e276c00 0x56271d51b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56271e277000 0x56271d51bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271e277400 0x56271d51c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271e264600 con 0x56271e277400 2024-04-25T10:13:19.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271e264780 con 0x56271e276c00 2024-04-25T10:13:19.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56271e264900 con 0x56271e277000 2024-04-25T10:13:19.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0895d49640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271e277400 0x56271d51c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0895d49640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271e277400 0x56271d51c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54926/0 (socket says 172.21.15.60:54926) 2024-04-25T10:13:19.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0895d49640 1 -- 172.21.15.60:0/630693868 learned_addr learned my addr 172.21.15.60:0/630693868 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:19.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0895d49640 1 -- 172.21.15.60:0/630693868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56271e277000 msgr2=0x56271d51bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0895d49640 1 --2- 172.21.15.60:0/630693868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56271e277000 0x56271d51bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0895d49640 1 -- 172.21.15.60:0/630693868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271e276c00 msgr2=0x56271d51b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0895d49640 1 --2- 172.21.15.60:0/630693868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271e276c00 0x56271d51b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0895d49640 1 -- 172.21.15.60:0/630693868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56271d387c20 con 0x56271e277400 2024-04-25T10:13:19.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0895d49640 1 --2- 172.21.15.60:0/630693868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271e277400 0x56271d51c100 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x56271e282fc0 tx=0x56271d692000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daffb081253ebb10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0891d41640 1 -- 172.21.15.60:0/630693868 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56271d3964e0 con 0x56271e277400 2024-04-25T10:13:19.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0891d41640 1 -- 172.21.15.60:0/630693868 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56271d3aac40 con 0x56271e277400 2024-04-25T10:13:19.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0891d41640 1 -- 172.21.15.60:0/630693868 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56271d396b60 con 0x56271e277400 2024-04-25T10:13:19.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/630693868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271e277400 msgr2=0x56271d51c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/630693868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271e277400 0x56271d51c100 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x56271e282fc0 tx=0x56271d692000 comp rx=0 tx=0).stop 2024-04-25T10:13:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/630693868 shutdown_connections 2024-04-25T10:13:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/630693868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271e277400 0x56271d51c100 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/630693868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56271e277000 0x56271d51bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/630693868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271e276c00 0x56271d51b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/630693868 >> 172.21.15.60:0/630693868 conn(0x56271d360c00 msgr2=0x56271d38f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/630693868 shutdown_connections 2024-04-25T10:13:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/630693868 wait complete. 2024-04-25T10:13:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.251+0000 7f0897d8b8c0 1 Processor -- start 2024-04-25T10:13:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0897d8b8c0 1 -- start start 2024-04-25T10:13:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0897d8b8c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56271e276c00 0x56271d51d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0897d8b8c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271e277000 0x56271d51cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0897d8b8c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271e277400 0x56271d51c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0897d8b8c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271e265c80 con 0x56271e277000 2024-04-25T10:13:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0897d8b8c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56271e265b00 con 0x56271e277400 2024-04-25T10:13:19.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0897d8b8c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56271e265980 con 0x56271e276c00 2024-04-25T10:13:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f089654a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271e277000 0x56271d51cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f089654a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271e277000 0x56271d51cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54928/0 (socket says 172.21.15.60:54928) 2024-04-25T10:13:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 learned_addr learned my addr 172.21.15.60:0/3823634439 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0895d49640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271e277400 0x56271d51c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0895d49640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56271e276c00 0x56271d51d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56271e276c00 msgr2=0x56271d51d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f089654a640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56271e276c00 0x56271d51d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271e277400 msgr2=0x56271d51c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f089654a640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271e277400 0x56271d51c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56271d3861e0 con 0x56271e277000 2024-04-25T10:13:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0895d49640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271e277400 0x56271d51c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f089654a640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271e277000 0x56271d51cc00 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x56271e1a9140 tx=0x56271e124000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f089053e640 1 -- 172.21.15.60:0/3823634439 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56271d3969c0 con 0x56271e277000 2024-04-25T10:13:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f089053e640 1 -- 172.21.15.60:0/3823634439 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56271d3aaa80 con 0x56271e277000 2024-04-25T10:13:19.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f089053e640 1 -- 172.21.15.60:0/3823634439 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56271d3964e0 con 0x56271e277000 2024-04-25T10:13:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56271d3874a0 con 0x56271e277000 2024-04-25T10:13:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56271d387a40 con 0x56271e277000 2024-04-25T10:13:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0897d8b8c0 4 datascan.init: resolving metadata pool 12 2024-04-25T10:13:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f089053e640 1 -- 172.21.15.60:0/3823634439 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x56271d698c00 con 0x56271e277000 2024-04-25T10:13:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f089053e640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x56271d690c00 0x56271d51d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f089053e640 1 -- 172.21.15.60:0/3823634439 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x56271d68ea00 con 0x56271e277000 2024-04-25T10:13:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0897d8b8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:13:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0897d8b8c0 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-04-25T10:13:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0897d8b8c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:13:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x56271e277800 0x56271d51c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.255+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271e277c00 con 0x56271e277800 2024-04-25T10:13:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.259+0000 7f0895d49640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x56271d690c00 0x56271d51d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.259+0000 7f0895d49640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x56271e277800 0x56271d51c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.259+0000 7f0895d49640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x56271e277800 0x56271d51c100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.259+0000 7f0895d49640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x56271d690c00 0x56271d51d700 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x56271e283110 tx=0x56271d693080 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.267 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:13:19.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.263+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.7 v2:172.21.15.100:6816/2771485684 1 ==== osd_op_reply(1 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271e277800 2024-04-25T10:13:19.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.263+0000 7f0891540640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x56271d691000 0x56271d78e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.263+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:2 13.20 13:04000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d691400 con 0x56271d691000 2024-04-25T10:13:19.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.263+0000 7f0896d4b640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x56271d691000 0x56271d78e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.263+0000 7f0896d4b640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x56271d691000 0x56271d78e000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.267+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.6 v2:172.21.15.100:6824/4086390550 1 ==== osd_op_reply(2 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d691000 2024-04-25T10:13:19.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.267+0000 7f0891d41640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x56271d691c00 0x56271d78e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.267+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:3 13.10 13:08000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79c000 con 0x56271d691c00 2024-04-25T10:13:19.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.267+0000 7f089654a640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x56271d691c00 0x56271d78e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.267+0000 7f089654a640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x56271d691c00 0x56271d78e580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.271+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 <== osd.1 v2:172.21.15.60:6824/3313552458 1 ==== osd_op_reply(3 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470780 con 0x56271d691c00 2024-04-25T10:13:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.271+0000 7f0891540640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x56271d691800 0x56271d78eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.271+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:4 13.30 13:0c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79c800 con 0x56271d691800 2024-04-25T10:13:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.271+0000 7f0895d49640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x56271d691800 0x56271d78eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.271+0000 7f0895d49640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x56271d691800 0x56271d78eb00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.275+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.4 v2:172.21.15.100:6808/3965541654 1 ==== osd_op_reply(4 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271d691800 2024-04-25T10:13:19.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.275+0000 7f0891d41640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x56271d79c400 0x56271d78f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.275+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:5 13.8 13:10000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79d400 con 0x56271d79c400 2024-04-25T10:13:19.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.275+0000 7f0896d4b640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x56271d79c400 0x56271d78f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.275+0000 7f0896d4b640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x56271d79c400 0x56271d78f600 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.275+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.2 v2:172.21.15.60:6800/1771147646 1 ==== osd_op_reply(5 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d79c400 2024-04-25T10:13:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.275+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:6 13.28 13:14000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79cc00 con 0x56271d79c400 2024-04-25T10:13:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.279+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.2 v2:172.21.15.60:6800/1771147646 2 ==== osd_op_reply(6 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d79c400 2024-04-25T10:13:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.279+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:7 13.18 13:18000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79d800 con 0x56271e277800 2024-04-25T10:13:19.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.279+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.7 v2:172.21.15.100:6816/2771485684 2 ==== osd_op_reply(7 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271e277800 2024-04-25T10:13:19.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.279+0000 7f0891540640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x56271d79d000 0x56271d78f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.279+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:8 13.38 13:1c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d690800 con 0x56271d79d000 2024-04-25T10:13:19.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.279+0000 7f089654a640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x56271d79d000 0x56271d78f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.279+0000 7f089654a640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x56271d79d000 0x56271d78f080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.279+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 <== osd.5 v2:172.21.15.100:6800/3542120824 1 ==== osd_op_reply(8 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470780 con 0x56271d79d000 2024-04-25T10:13:19.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.279+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:9 13.4 13:20000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79dc00 con 0x56271d691000 2024-04-25T10:13:19.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.279+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.6 v2:172.21.15.100:6824/4086390550 2 ==== osd_op_reply(9 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d691000 2024-04-25T10:13:19.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.279+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:10 13.24 13:24000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271e277c00 con 0x56271d691000 2024-04-25T10:13:19.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.279+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.6 v2:172.21.15.100:6824/4086390550 3 ==== osd_op_reply(10 [pgnls start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d691000 2024-04-25T10:13:19.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.279+0000 7f0891d41640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x56271d79c800 0x56271d78fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.279+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:11 13.14 13:28000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79d800 con 0x56271d79c800 2024-04-25T10:13:19.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0895d49640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x56271d79c800 0x56271d78fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0895d49640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x56271d79c800 0x56271d78fb80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.0 v2:172.21.15.60:6801/4056167511 1 ==== osd_op_reply(11 [pgnls start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271d79c800 2024-04-25T10:13:19.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:12 13.34 13:2c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271e276800 con 0x56271d691000 2024-04-25T10:13:19.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.6 v2:172.21.15.100:6824/4086390550 4 ==== osd_op_reply(12 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d691000 2024-04-25T10:13:19.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:13 13.c 13:30000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79d400 con 0x56271d691c00 2024-04-25T10:13:19.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 <== osd.1 v2:172.21.15.60:6824/3313552458 2 ==== osd_op_reply(13 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470780 con 0x56271d691c00 2024-04-25T10:13:19.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:14 13.2c 13:34000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271e276400 con 0x56271d79c400 2024-04-25T10:13:19.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.2 v2:172.21.15.60:6800/1771147646 3 ==== osd_op_reply(14 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d79c400 2024-04-25T10:13:19.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:15 13.1c 13:38000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d691400 con 0x56271d691000 2024-04-25T10:13:19.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.6 v2:172.21.15.100:6824/4086390550 5 ==== osd_op_reply(15 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d691000 2024-04-25T10:13:19.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:16 13.3c 13:3c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79cc00 con 0x56271d79c400 2024-04-25T10:13:19.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.2 v2:172.21.15.60:6800/1771147646 4 ==== osd_op_reply(16 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d79c400 2024-04-25T10:13:19.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:17 13.2 13:40000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79dc00 con 0x56271d79d000 2024-04-25T10:13:19.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 <== osd.5 v2:172.21.15.100:6800/3542120824 2 ==== osd_op_reply(17 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470780 con 0x56271d79d000 2024-04-25T10:13:19.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:18 13.22 13:44000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79c000 con 0x56271d691c00 2024-04-25T10:13:19.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 <== osd.1 v2:172.21.15.60:6824/3313552458 3 ==== osd_op_reply(18 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470780 con 0x56271d691c00 2024-04-25T10:13:19.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:19 13.12 13:48000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d691400 con 0x56271d79c800 2024-04-25T10:13:19.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.0 v2:172.21.15.60:6801/4056167511 2 ==== osd_op_reply(19 [pgnls start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271d79c800 2024-04-25T10:13:19.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:20 13.32 13:4c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79d400 con 0x56271d79c400 2024-04-25T10:13:19.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.2 v2:172.21.15.60:6800/1771147646 5 ==== osd_op_reply(20 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d79c400 2024-04-25T10:13:19.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.283+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:21 13.a 13:50000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271e277c00 con 0x56271d79c400 2024-04-25T10:13:19.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.287+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.2 v2:172.21.15.60:6800/1771147646 6 ==== osd_op_reply(21 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d79c400 2024-04-25T10:13:19.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.287+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:22 13.2a 13:54000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79dc00 con 0x56271d691c00 2024-04-25T10:13:19.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.287+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 <== osd.1 v2:172.21.15.60:6824/3313552458 4 ==== osd_op_reply(22 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470780 con 0x56271d691c00 2024-04-25T10:13:19.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.287+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:23 13.1a 13:58000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271e276800 con 0x56271d691c00 2024-04-25T10:13:19.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.287+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 <== osd.1 v2:172.21.15.60:6824/3313552458 5 ==== osd_op_reply(23 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470780 con 0x56271d691c00 2024-04-25T10:13:19.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.287+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:24 13.3a 13:5c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79cc00 con 0x56271d79d000 2024-04-25T10:13:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.287+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 <== osd.5 v2:172.21.15.100:6800/3542120824 3 ==== osd_op_reply(24 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470780 con 0x56271d79d000 2024-04-25T10:13:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.287+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:25 13.6 13:60000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271e276400 con 0x56271d691800 2024-04-25T10:13:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.287+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.4 v2:172.21.15.100:6808/3965541654 2 ==== osd_op_reply(25 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271d691800 2024-04-25T10:13:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.287+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:26 13.26 13:64000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d690800 con 0x56271e277800 2024-04-25T10:13:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.287+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.7 v2:172.21.15.100:6816/2771485684 3 ==== osd_op_reply(26 [pgnls start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271e277800 2024-04-25T10:13:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.287+0000 7f0891d41640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x56271d7a2000 0x56271d790100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.287+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:27 13.16 13:68000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a2400 con 0x56271d7a2000 2024-04-25T10:13:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.287+0000 7f0896d4b640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x56271d7a2000 0x56271d790100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.287+0000 7f0896d4b640 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x56271d7a2000 0x56271d790100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.291+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.3 v2:172.21.15.60:6814/1867957156 1 ==== osd_op_reply(27 [pgnls start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d7a2000 2024-04-25T10:13:19.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.291+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:28 13.36 13:6c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271e139400 con 0x56271d691000 2024-04-25T10:13:19.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.291+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.6 v2:172.21.15.100:6824/4086390550 6 ==== osd_op_reply(28 [pgnls start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d691000 2024-04-25T10:13:19.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.291+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:29 13.e 13:70000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a2800 con 0x56271e277800 2024-04-25T10:13:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.291+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.7 v2:172.21.15.100:6816/2771485684 4 ==== osd_op_reply(29 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271e277800 2024-04-25T10:13:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.291+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:30 13.2e 13:74000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a2800 con 0x56271d79c400 2024-04-25T10:13:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.291+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.2 v2:172.21.15.60:6800/1771147646 7 ==== osd_op_reply(30 [pgnls start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d79c400 2024-04-25T10:13:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.291+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:31 13.1e 13:78000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a2c00 con 0x56271d691800 2024-04-25T10:13:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.291+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.4 v2:172.21.15.100:6808/3965541654 3 ==== osd_op_reply(31 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271d691800 2024-04-25T10:13:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.291+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:32 13.3e 13:7c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a3800 con 0x56271d691c00 2024-04-25T10:13:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.291+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 <== osd.1 v2:172.21.15.60:6824/3313552458 6 ==== osd_op_reply(32 [pgnls start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470780 con 0x56271d691c00 2024-04-25T10:13:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.291+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:33 13.1 13:80000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a3000 con 0x56271d691000 2024-04-25T10:13:19.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.295+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.6 v2:172.21.15.100:6824/4086390550 7 ==== osd_op_reply(33 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d691000 2024-04-25T10:13:19.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.295+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:34 13.21 13:84000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a3c00 con 0x56271d691800 2024-04-25T10:13:19.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.295+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.4 v2:172.21.15.100:6808/3965541654 4 ==== osd_op_reply(34 [pgnls start_epoch 86 out=49b] v82'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271d691800 2024-04-25T10:13:19.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.295+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:35 13.11 13:88000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a3400 con 0x56271d691c00 2024-04-25T10:13:19.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.295+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 <== osd.1 v2:172.21.15.60:6824/3313552458 7 ==== osd_op_reply(35 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470780 con 0x56271d691c00 2024-04-25T10:13:19.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.295+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:36 13.31 13:8c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d691400 con 0x56271d7a2000 2024-04-25T10:13:19.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.295+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.3 v2:172.21.15.60:6814/1867957156 2 ==== osd_op_reply(36 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d7a2000 2024-04-25T10:13:19.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.295+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:37 13.9 13:90000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79d400 con 0x56271d79d000 2024-04-25T10:13:19.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.295+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 <== osd.5 v2:172.21.15.100:6800/3542120824 4 ==== osd_op_reply(37 [pgnls start_epoch 86 out=49b] v82'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470780 con 0x56271d79d000 2024-04-25T10:13:19.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.295+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:38 13.29 13:94000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d690800 con 0x56271d691000 2024-04-25T10:13:19.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.295+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.6 v2:172.21.15.100:6824/4086390550 8 ==== osd_op_reply(38 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d691000 2024-04-25T10:13:19.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.295+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:39 13.19 13:98000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79c000 con 0x56271d7a2000 2024-04-25T10:13:19.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.295+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.3 v2:172.21.15.60:6814/1867957156 3 ==== osd_op_reply(39 [pgnls start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d7a2000 2024-04-25T10:13:19.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.295+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:40 13.39 13:9c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271e276400 con 0x56271d79c400 2024-04-25T10:13:19.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.295+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.2 v2:172.21.15.60:6800/1771147646 8 ==== osd_op_reply(40 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d79c400 2024-04-25T10:13:19.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.295+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:41 13.5 13:a0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79dc00 con 0x56271d691800 2024-04-25T10:13:19.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.4 v2:172.21.15.100:6808/3965541654 5 ==== osd_op_reply(41 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271d691800 2024-04-25T10:13:19.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:42 13.25 13:a4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79d800 con 0x56271d691800 2024-04-25T10:13:19.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.4 v2:172.21.15.100:6808/3965541654 6 ==== osd_op_reply(42 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271d691800 2024-04-25T10:13:19.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:43 13.15 13:a8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271e277c00 con 0x56271d691800 2024-04-25T10:13:19.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.4 v2:172.21.15.100:6808/3965541654 7 ==== osd_op_reply(43 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271d691800 2024-04-25T10:13:19.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:44 13.35 13:ac000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a2400 con 0x56271d7a2000 2024-04-25T10:13:19.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.3 v2:172.21.15.60:6814/1867957156 4 ==== osd_op_reply(44 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d7a2000 2024-04-25T10:13:19.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:45 13.d 13:b0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271e276800 con 0x56271d691800 2024-04-25T10:13:19.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.4 v2:172.21.15.100:6808/3965541654 8 ==== osd_op_reply(45 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271d691800 2024-04-25T10:13:19.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:46 13.2d 13:b4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a2800 con 0x56271d79c800 2024-04-25T10:13:19.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.0 v2:172.21.15.60:6801/4056167511 3 ==== osd_op_reply(46 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271d79c800 2024-04-25T10:13:19.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:47 13.1d 13:b8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d691400 con 0x56271e277800 2024-04-25T10:13:19.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.7 v2:172.21.15.100:6816/2771485684 5 ==== osd_op_reply(47 [pgnls start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271e277800 2024-04-25T10:13:19.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:48 13.3d 13:bc000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a3000 con 0x56271e277800 2024-04-25T10:13:19.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.7 v2:172.21.15.100:6816/2771485684 6 ==== osd_op_reply(48 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271e277800 2024-04-25T10:13:19.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:49 13.3 13:c0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271e139400 con 0x56271d79c800 2024-04-25T10:13:19.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.0 v2:172.21.15.60:6801/4056167511 4 ==== osd_op_reply(49 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271d79c800 2024-04-25T10:13:19.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.299+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:50 13.23 13:c4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a3800 con 0x56271d691c00 2024-04-25T10:13:19.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.303+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 <== osd.1 v2:172.21.15.60:6824/3313552458 8 ==== osd_op_reply(50 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470780 con 0x56271d691c00 2024-04-25T10:13:19.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.303+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:51 13.13 13:c8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271e277c00 con 0x56271e277800 2024-04-25T10:13:19.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.303+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.7 v2:172.21.15.100:6816/2771485684 7 ==== osd_op_reply(51 [pgnls start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271e277800 2024-04-25T10:13:19.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.303+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:52 13.33 13:cc000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a3400 con 0x56271e277800 2024-04-25T10:13:19.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.303+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.7 v2:172.21.15.100:6816/2771485684 8 ==== osd_op_reply(52 [pgnls start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271e277800 2024-04-25T10:13:19.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.303+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:53 13.b 13:d0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271e276800 con 0x56271d79d000 2024-04-25T10:13:19.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.303+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 <== osd.5 v2:172.21.15.100:6800/3542120824 5 ==== osd_op_reply(53 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470780 con 0x56271d79d000 2024-04-25T10:13:19.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.303+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:54 13.2b 13:d4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79cc00 con 0x56271d79c400 2024-04-25T10:13:19.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.303+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.2 v2:172.21.15.60:6800/1771147646 9 ==== osd_op_reply(54 [pgnls start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d79c400 2024-04-25T10:13:19.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.303+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:55 13.1b 13:d8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79dc00 con 0x56271d691800 2024-04-25T10:13:19.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.303+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.4 v2:172.21.15.100:6808/3965541654 9 ==== osd_op_reply(55 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271d691800 2024-04-25T10:13:19.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.303+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:56 13.3b 13:dc000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79dc00 con 0x56271d691800 2024-04-25T10:13:19.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.303+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.4 v2:172.21.15.100:6808/3965541654 10 ==== osd_op_reply(56 [pgnls start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271d691800 2024-04-25T10:13:19.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.303+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:57 13.7 13:e0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d79d800 con 0x56271e277800 2024-04-25T10:13:19.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.303+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.7 v2:172.21.15.100:6816/2771485684 9 ==== osd_op_reply(57 [pgnls start_epoch 86 out=81b] v82'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56271d470000 con 0x56271e277800 2024-04-25T10:13:19.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.307+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:58 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7be000 con 0x56271e277800 2024-04-25T10:13:19.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.307+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.7 v2:172.21.15.100:6816/2771485684 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x56271d470000 con 0x56271e277800 2024-04-25T10:13:19.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.307+0000 7f0897d8b8c0 10 datascan.operator(): handling object 1099511627777.0 2024-04-25T10:13:19.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.307+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:59 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7be400 con 0x56271e277800 2024-04-25T10:13:19.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.311+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.7 v2:172.21.15.100:6816/2771485684 11 ==== osd_op_reply(59 10000000001.00000000 [call] v86'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56271d470000 con 0x56271e277800 2024-04-25T10:13:19.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.311+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:60 13.27 13:e4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7be800 con 0x56271d691c00 2024-04-25T10:13:19.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.311+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 <== osd.1 v2:172.21.15.60:6824/3313552458 9 ==== osd_op_reply(60 [pgnls start_epoch 86 out=49b] v82'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470780 con 0x56271d691c00 2024-04-25T10:13:19.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.311+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:61 13.17 13:e8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a2c00 con 0x56271d79d000 2024-04-25T10:13:19.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.311+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 <== osd.5 v2:172.21.15.100:6800/3542120824 6 ==== osd_op_reply(61 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470780 con 0x56271d79d000 2024-04-25T10:13:19.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.311+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:62 13.37 13:ec000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a2800 con 0x56271d79d000 2024-04-25T10:13:19.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.311+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 <== osd.5 v2:172.21.15.100:6800/3542120824 7 ==== osd_op_reply(62 [pgnls start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470780 con 0x56271d79d000 2024-04-25T10:13:19.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.311+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:63 13.f 13:f0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a3c00 con 0x56271d691000 2024-04-25T10:13:19.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.311+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.6 v2:172.21.15.100:6824/4086390550 9 ==== osd_op_reply(63 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d691000 2024-04-25T10:13:19.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.311+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:64 13.2f 13:f4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a3000 con 0x56271d691800 2024-04-25T10:13:19.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.311+0000 7f0895d49640 1 -- 172.21.15.60:0/3823634439 <== osd.4 v2:172.21.15.100:6808/3965541654 11 ==== osd_op_reply(64 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470000 con 0x56271d691800 2024-04-25T10:13:19.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.311+0000 7f0891d41640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:65 13.1f 13:f8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7bfc00 con 0x56271d691000 2024-04-25T10:13:19.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.6 v2:172.21.15.100:6824/4086390550 10 ==== osd_op_reply(65 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d691000 2024-04-25T10:13:19.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0891540640 1 -- 172.21.15.60:0/3823634439 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:66 13.3f 13:fc000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x56271d7a3400 con 0x56271d691000 2024-04-25T10:13:19.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0896d4b640 1 -- 172.21.15.60:0/3823634439 <== osd.6 v2:172.21.15.100:6824/4086390550 11 ==== osd_op_reply(66 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56271d470500 con 0x56271d691000 2024-04-25T10:13:19.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x56271d79c800 msgr2=0x56271d78fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x56271d79c800 0x56271d78fb80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x56271d691c00 msgr2=0x56271d78e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x56271d691c00 0x56271d78e580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x56271d79c400 msgr2=0x56271d78f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x56271d79c400 0x56271d78f600 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x56271d7a2000 msgr2=0x56271d790100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x56271d7a2000 0x56271d790100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x56271d691800 msgr2=0x56271d78eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x56271d691800 0x56271d78eb00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x56271d79d000 msgr2=0x56271d78f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 reap_dead start 2024-04-25T10:13:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x56271d79d000 0x56271d78f080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x56271d691000 msgr2=0x56271d78e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x56271d691000 0x56271d78e000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x56271e277800 msgr2=0x56271d51c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x56271e277800 0x56271d51c100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x56271d690c00 msgr2=0x56271d51d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x56271d690c00 0x56271d51d700 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x56271e283110 tx=0x56271d693080 comp rx=0 tx=0).stop 2024-04-25T10:13:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271e277000 msgr2=0x56271d51cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/3823634439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271e277000 0x56271d51cc00 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x56271e1a9140 tx=0x56271e124000 comp rx=0 tx=0).stop 2024-04-25T10:13:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f089654a640 1 -- 172.21.15.60:0/3823634439 reap_dead start 2024-04-25T10:13:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 shutdown_connections 2024-04-25T10:13:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 >> 172.21.15.60:0/3823634439 conn(0x56271d360c00 msgr2=0x56271d38f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 shutdown_connections 2024-04-25T10:13:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3823634439 wait complete. 2024-04-25T10:13:19.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3502374876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271d361000 msgr2=0x56271d51b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/3502374876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271d361000 0x56271d51b080 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x56271e1a9b60 tx=0x56271d43a9c0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3502374876 shutdown_connections 2024-04-25T10:13:19.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/3502374876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56271d361000 0x56271d51b080 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/3502374876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56271d361400 0x56271d51ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 --2- 172.21.15.60:0/3502374876 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56271d361800 0x56271d51a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3502374876 >> 172.21.15.60:0/3502374876 conn(0x56271d361c00 msgr2=0x56271d38f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3502374876 shutdown_connections 2024-04-25T10:13:19.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.315+0000 7f0897d8b8c0 1 -- 172.21.15.60:0/3502374876 wait complete. 2024-04-25T10:13:19.324 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.105394, result: 2024-04-25T10:13:19.325 DEBUG:teuthology.orchestra.run.smithi060:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-04-25T10:13:19.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0fadd98c0 1 -- start start 2024-04-25T10:13:19.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0fadd98c0 1 Processor -- start 2024-04-25T10:13:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0fadd98c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea0e52000 0x55eea0249080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0fadd98c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea008fc00 0x55eea0248b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0fadd98c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea0e53000 0x55eea0249600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0fadd98c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea0f9e000 con 0x55eea0e52000 2024-04-25T10:13:19.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0fadd98c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea0f9e180 con 0x55eea0e53000 2024-04-25T10:13:19.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0fadd98c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eea0f9e300 con 0x55eea008fc00 2024-04-25T10:13:19.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0f9598640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea0e52000 0x55eea0249080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0f9598640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea0e52000 0x55eea0249080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54956/0 (socket says 172.21.15.60:54956) 2024-04-25T10:13:19.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0f9598640 1 -- 172.21.15.60:0/2602282541 learned_addr learned my addr 172.21.15.60:0/2602282541 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:19.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0f8d97640 1 --2- 172.21.15.60:0/2602282541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea008fc00 0x55eea0248b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0f9d99640 1 --2- 172.21.15.60:0/2602282541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea0e53000 0x55eea0249600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0f9598640 1 -- 172.21.15.60:0/2602282541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea008fc00 msgr2=0x55eea0248b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0f9598640 1 --2- 172.21.15.60:0/2602282541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea008fc00 0x55eea0248b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0f9598640 1 -- 172.21.15.60:0/2602282541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea0e53000 msgr2=0x55eea0249600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0f9598640 1 --2- 172.21.15.60:0/2602282541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea0e53000 0x55eea0249600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0f9598640 1 -- 172.21.15.60:0/2602282541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eea00b41e0 con 0x55eea0e52000 2024-04-25T10:13:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0f9598640 1 --2- 172.21.15.60:0/2602282541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea0e52000 0x55eea0249080 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x55eea0edba10 tx=0x55eea018c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44aef923644a219 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0f6d93640 1 -- 172.21.15.60:0/2602282541 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eea00c4b60 con 0x55eea0e52000 2024-04-25T10:13:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0f6d93640 1 -- 172.21.15.60:0/2602282541 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eea00d8540 con 0x55eea0e52000 2024-04-25T10:13:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0f6d93640 1 -- 172.21.15.60:0/2602282541 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eea00c41a0 con 0x55eea0e52000 2024-04-25T10:13:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/2602282541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eea00b5c20 con 0x55eea0e52000 2024-04-25T10:13:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/2602282541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55eea00b54a0 con 0x55eea0e52000 2024-04-25T10:13:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0fadd98c0 4 waiting for MDS map... 2024-04-25T10:13:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0f6d93640 1 -- 172.21.15.60:0/2602282541 <== mon.0 v2:172.21.15.60:3300/0 4 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x55eea0fa0400 con 0x55eea0e52000 2024-04-25T10:13:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0f6d93640 1 -- 172.21.15.60:0/2602282541 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 63) v1 ==== 2723+0+0 (secure 0 0 0) 0x55eea00e5180 con 0x55eea0e52000 2024-04-25T10:13:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0fadd98c0 4 Got MDS map 63 2024-04-25T10:13:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0fadd98c0 4 datascan.main: Using metadata pool output 2024-04-25T10:13:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.395+0000 7fe0fadd98c0 4 datascan.main: connecting to RADOS... 2024-04-25T10:13:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 Processor -- start 2024-04-25T10:13:19.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 -- start start 2024-04-25T10:13:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea0e53800 0x55eea0248580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea0e53c00 0x55eea024a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea1002000 0x55eea024a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea0f9e480 con 0x55eea1002000 2024-04-25T10:13:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea0f9e600 con 0x55eea0e53800 2024-04-25T10:13:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eea0f9e780 con 0x55eea0e53c00 2024-04-25T10:13:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea1002000 0x55eea024a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea1002000 0x55eea024a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54960/0 (socket says 172.21.15.60:54960) 2024-04-25T10:13:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 -- 172.21.15.60:0/2697970156 learned_addr learned my addr 172.21.15.60:0/2697970156 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f9d99640 1 --2- 172.21.15.60:0/2697970156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea0e53800 0x55eea0248580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f9598640 1 --2- 172.21.15.60:0/2697970156 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea0e53c00 0x55eea024a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 -- 172.21.15.60:0/2697970156 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea0e53c00 msgr2=0x55eea024a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 --2- 172.21.15.60:0/2697970156 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea0e53c00 0x55eea024a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 -- 172.21.15.60:0/2697970156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea0e53800 msgr2=0x55eea0248580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 --2- 172.21.15.60:0/2697970156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea0e53800 0x55eea0248580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 -- 172.21.15.60:0/2697970156 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eea00b41e0 con 0x55eea1002000 2024-04-25T10:13:19.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 --2- 172.21.15.60:0/2697970156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea1002000 0x55eea024a680 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x55eea0facf90 tx=0x55eea10100c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a62d81d355cb43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/2697970156 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eea00c4b60 con 0x55eea1002000 2024-04-25T10:13:19.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/2697970156 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eea00d8c40 con 0x55eea1002000 2024-04-25T10:13:19.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/2697970156 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eea00c41a0 con 0x55eea1002000 2024-04-25T10:13:19.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/2697970156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea1002000 msgr2=0x55eea024a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/2697970156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea1002000 0x55eea024a680 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x55eea0facf90 tx=0x55eea10100c0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/2697970156 shutdown_connections 2024-04-25T10:13:19.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/2697970156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea1002000 0x55eea024a680 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/2697970156 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea0e53c00 0x55eea024a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/2697970156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea0e53800 0x55eea0248580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/2697970156 >> 172.21.15.60:0/2697970156 conn(0x55eea0e53400 msgr2=0x55eea00bd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/2697970156 shutdown_connections 2024-04-25T10:13:19.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/2697970156 wait complete. 2024-04-25T10:13:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 Processor -- start 2024-04-25T10:13:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 -- start start 2024-04-25T10:13:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea0e53c00 0x55eea024b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea0e53800 0x55eea024b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea1003000 0x55eea024ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea0f9fe00 con 0x55eea1003000 2024-04-25T10:13:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea0f9fc80 con 0x55eea0e53c00 2024-04-25T10:13:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0fadd98c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eea0f9fb00 con 0x55eea0e53800 2024-04-25T10:13:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea1003000 0x55eea024ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea1003000 0x55eea024ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54970/0 (socket says 172.21.15.60:54970) 2024-04-25T10:13:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 learned_addr learned my addr 172.21.15.60:0/3667315397 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f9d99640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea0e53c00 0x55eea024b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f9598640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea0e53800 0x55eea024b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea0e53800 msgr2=0x55eea024b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea0e53800 0x55eea024b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea0e53c00 msgr2=0x55eea024b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea0e53c00 0x55eea024b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eea00b52c0 con 0x55eea1003000 2024-04-25T10:13:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.399+0000 7fe0f8d97640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea1003000 0x55eea024ac00 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x55eea0fadc50 tx=0x55eea1010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0f358c640 1 -- 172.21.15.60:0/3667315397 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eea00c4b60 con 0x55eea1003000 2024-04-25T10:13:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0f358c640 1 -- 172.21.15.60:0/3667315397 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eea00d8a80 con 0x55eea1003000 2024-04-25T10:13:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eea00b5e00 con 0x55eea1003000 2024-04-25T10:13:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0f358c640 1 -- 172.21.15.60:0/3667315397 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55eea00c41a0 con 0x55eea1003000 2024-04-25T10:13:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eea00b5a40 con 0x55eea1003000 2024-04-25T10:13:19.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0fadd98c0 4 datascan.init: resolving metadata pool 12 2024-04-25T10:13:19.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0f358c640 1 -- 172.21.15.60:0/3667315397 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x55eea03bcc00 con 0x55eea1003000 2024-04-25T10:13:19.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0f358c640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55eea04a6000 0x55eea04a8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0f358c640 1 -- 172.21.15.60:0/3667315397 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x55eea0e50000 con 0x55eea1003000 2024-04-25T10:13:19.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0fadd98c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:13:19.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0fadd98c0 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-04-25T10:13:19.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0fadd98c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:13:19.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55eea1002c00 0x55eea024a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea1002800 con 0x55eea1002c00 2024-04-25T10:13:19.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0f9d99640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55eea04a6000 0x55eea04a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0f9598640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55eea1002c00 0x55eea024a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0f9598640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55eea1002c00 0x55eea024a680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0f9d99640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55eea04a6000 0x55eea04a8000 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x55eea016a150 tx=0x55eea1010420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.5 v2:172.21.15.100:6800/3542120824 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv24 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55eea0142000 con 0x55eea1002c00 2024-04-25T10:13:19.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x55eea1002400 0x55eea024a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:2 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea1002000 con 0x55eea1002400 2024-04-25T10:13:19.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.403+0000 7fe0f9d99640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x55eea1002400 0x55eea024a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.407+0000 7fe0f9d99640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x55eea1002400 0x55eea024a100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.407+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.407+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x55eea04a7000 0x55eea0248580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.407+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:3 13.0 13:00000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04a7400 con 0x55eea04a7000 2024-04-25T10:13:19.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.407+0000 7fe0f9598640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x55eea04a7000 0x55eea0248580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.407+0000 7fe0f9598640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x55eea04a7000 0x55eea0248580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.407+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.407+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:4 13.20 13:04000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c6000 con 0x55eea1002400 2024-04-25T10:13:19.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.407+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.407+0000 7fe0f4d8f640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55eea04c6400 0x55eea04a8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.407+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:5 13.10 13:08000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c6800 con 0x55eea04c6400 2024-04-25T10:13:19.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.407+0000 7fe0f8d97640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55eea04c6400 0x55eea04a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.407+0000 7fe0f8d97640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55eea04c6400 0x55eea04a8580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.407+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.407+0000 7fe0f458e640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55eea04c7000 0x55eea04a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.407+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:6 13.30 13:0c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7400 con 0x55eea04c7000 2024-04-25T10:13:19.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.411+0000 7fe0f9d99640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55eea04c7000 0x55eea04a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.411+0000 7fe0f9d99640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55eea04c7000 0x55eea04a8b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.411+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.411+0000 7fe0f4d8f640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x55eea04c6c00 0x55eea04a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.411+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:7 13.8 13:10000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7800 con 0x55eea04c6c00 2024-04-25T10:13:19.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.411+0000 7fe0f9598640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x55eea04c6c00 0x55eea04a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.411+0000 7fe0f9598640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x55eea04c6c00 0x55eea04a9080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.411+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.411+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:8 13.28 13:14000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da000 con 0x55eea04c6c00 2024-04-25T10:13:19.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.411+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.411+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:9 13.18 13:18000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7c00 con 0x55eea04a7000 2024-04-25T10:13:19.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.411+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.411+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:10 13.38 13:1c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da400 con 0x55eea1002c00 2024-04-25T10:13:19.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.411+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.5 v2:172.21.15.100:6800/3542120824 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea1002c00 2024-04-25T10:13:19.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.411+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:11 13.4 13:20000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7800 con 0x55eea1002400 2024-04-25T10:13:19.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:12 13.24 13:24000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da800 con 0x55eea1002400 2024-04-25T10:13:19.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f4d8f640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x55eea04a7400 0x55eea04a9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:13 13.14 13:28000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea1002800 con 0x55eea04a7400 2024-04-25T10:13:19.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f8d97640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x55eea04a7400 0x55eea04a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f8d97640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x55eea04a7400 0x55eea04a9600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.0 v2:172.21.15.60:6801/4056167511 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04a7400 2024-04-25T10:13:19.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:14 13.34 13:2c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c6000 con 0x55eea1002400 2024-04-25T10:13:19.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:15 13.c 13:30000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea0e52c00 con 0x55eea04c6400 2024-04-25T10:13:19.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 2 ==== osd_op_reply(15 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:16 13.2c 13:34000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7400 con 0x55eea04c6c00 2024-04-25T10:13:19.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:17 13.1c 13:38000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da000 con 0x55eea1002400 2024-04-25T10:13:19.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:18 13.3c 13:3c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea1002000 con 0x55eea04c6c00 2024-04-25T10:13:19.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:19 13.2 13:40000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04dac00 con 0x55eea1002c00 2024-04-25T10:13:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.415+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.5 v2:172.21.15.100:6800/3542120824 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea1002c00 2024-04-25T10:13:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:20 13.22 13:44000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04a6c00 con 0x55eea04c6400 2024-04-25T10:13:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:21 13.12 13:48000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04db000 con 0x55eea04a7400 2024-04-25T10:13:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.0 v2:172.21.15.60:6801/4056167511 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04a7400 2024-04-25T10:13:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:22 13.32 13:4c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04db800 con 0x55eea04c6c00 2024-04-25T10:13:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:23 13.a 13:50000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04db400 con 0x55eea04c6c00 2024-04-25T10:13:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 6 ==== osd_op_reply(23 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:24 13.2a 13:54000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04a6c00 con 0x55eea04c6400 2024-04-25T10:13:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:25 13.1a 13:58000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7800 con 0x55eea04c6400 2024-04-25T10:13:19.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:26 13.3a 13:5c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea0e52c00 con 0x55eea1002c00 2024-04-25T10:13:19.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.5 v2:172.21.15.100:6800/3542120824 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea1002c00 2024-04-25T10:13:19.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:27 13.6 13:60000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da400 con 0x55eea04c7000 2024-04-25T10:13:19.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:28 13.26 13:64000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea1002800 con 0x55eea04a7000 2024-04-25T10:13:19.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f4d8f640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55eea04dac00 0x55eea04a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:29 13.16 13:68000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04db800 con 0x55eea04dac00 2024-04-25T10:13:19.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f9d99640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55eea04dac00 0x55eea04a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.419+0000 7fe0f9d99640 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55eea04dac00 0x55eea04a9b80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.423+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.3 v2:172.21.15.60:6814/1867957156 1 ==== osd_op_reply(29 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04dac00 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.423+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:30 13.36 13:6c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c6800 con 0x55eea1002400 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.423+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 7 ==== osd_op_reply(30 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.423+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:31 13.e 13:70000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea1002000 con 0x55eea04a7000 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.423+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.423+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:32 13.2e 13:74000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da800 con 0x55eea04c6c00 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.423+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 7 ==== osd_op_reply(32 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.423+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:33 13.1e 13:78000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7c00 con 0x55eea04c7000 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.423+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 3 ==== osd_op_reply(33 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.423+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:34 13.3e 13:7c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da000 con 0x55eea04c6400 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.423+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.423+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:35 13.1 13:80000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7400 con 0x55eea1002400 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.423+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 8 ==== osd_op_reply(35 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.423+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:36 13.21 13:84000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da400 con 0x55eea04c7000 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.427+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 86 out=49b] v82'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.427+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:37 13.11 13:88000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04dbc00 con 0x55eea04c6400 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.427+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.427+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:38 13.31 13:8c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04db800 con 0x55eea04dac00 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.427+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.3 v2:172.21.15.60:6814/1867957156 2 ==== osd_op_reply(38 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04dac00 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.427+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:39 13.9 13:90000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04db000 con 0x55eea1002c00 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.427+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.5 v2:172.21.15.100:6800/3542120824 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 86 out=49b] v82'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea1002c00 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.427+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:40 13.29 13:94000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c6000 con 0x55eea1002400 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.427+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 9 ==== osd_op_reply(40 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.427+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:41 13.19 13:98000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da000 con 0x55eea04dac00 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.427+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.3 v2:172.21.15.60:6814/1867957156 3 ==== osd_op_reply(41 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04dac00 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.427+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:42 13.39 13:9c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c6800 con 0x55eea04c6c00 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.427+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 8 ==== osd_op_reply(42 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.427+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:43 13.5 13:a0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04dbc00 con 0x55eea04c7000 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.427+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.427+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:44 13.25 13:a4000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da400 con 0x55eea04c7000 2024-04-25T10:13:19.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.431+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 6 ==== osd_op_reply(44 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.431+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:45 13.15 13:a8000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7800 con 0x55eea04c7000 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.431+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 7 ==== osd_op_reply(45 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.431+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:46 13.35 13:ac000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04db800 con 0x55eea04dac00 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.431+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.3 v2:172.21.15.60:6814/1867957156 4 ==== osd_op_reply(46 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04dac00 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.431+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:47 13.d 13:b0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04a6c00 con 0x55eea04c7000 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.431+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 8 ==== osd_op_reply(47 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.431+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:48 13.2d 13:b4000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da000 con 0x55eea04a7400 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.431+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.0 v2:172.21.15.60:6801/4056167511 3 ==== osd_op_reply(48 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04a7400 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.431+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:49 13.1d 13:b8000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7c00 con 0x55eea04a7000 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.431+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.431+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:50 13.3d 13:bc000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04dbc00 con 0x55eea04a7000 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.431+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 6 ==== osd_op_reply(50 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.431+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:51 13.3 13:c0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7400 con 0x55eea04a7400 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.431+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.0 v2:172.21.15.60:6801/4056167511 4 ==== osd_op_reply(51 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04a7400 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.431+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:52 13.23 13:c4000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea1002800 con 0x55eea04c6400 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.435+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 8 ==== osd_op_reply(52 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.435+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:53 13.13 13:c8000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c6000 con 0x55eea04a7000 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.435+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.435+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:54 13.33 13:cc000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea1002000 con 0x55eea04a7000 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.435+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.435+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:55 13.b 13:d0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c6800 con 0x55eea1002c00 2024-04-25T10:13:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.435+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.5 v2:172.21.15.100:6800/3542120824 6 ==== osd_op_reply(55 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea1002c00 2024-04-25T10:13:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.435+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:56 13.2b 13:d4000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea0e52c00 con 0x55eea04c6c00 2024-04-25T10:13:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.435+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 9 ==== osd_op_reply(56 [pgnls-filter start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.435+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:57 13.1b 13:d8000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04db400 con 0x55eea04c7000 2024-04-25T10:13:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.435+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 9 ==== osd_op_reply(57 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.435+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:58 13.3b 13:dc000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04ea000 con 0x55eea04c7000 2024-04-25T10:13:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.435+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 10 ==== osd_op_reply(58 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.435+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:59 13.7 13:e0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da800 con 0x55eea04a7000 2024-04-25T10:13:19.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.439+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 9 ==== osd_op_reply(59 [pgnls-filter start_epoch 86 out=81b] v86'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.439+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:60 13.0 13:00000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04a7800 con 0x55eea04a7000 2024-04-25T10:13:19.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.439+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 10 ==== osd_op_reply(60 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.439+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:61 13.20 13:04000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04db000 con 0x55eea1002400 2024-04-25T10:13:19.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.439+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 10 ==== osd_op_reply(61 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.439+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:62 13.10 13:08000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04ea400 con 0x55eea04c6400 2024-04-25T10:13:19.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.439+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.439+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:63 13.30 13:0c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04a7800 con 0x55eea04c7000 2024-04-25T10:13:19.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.439+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 11 ==== osd_op_reply(63 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.439+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:64 13.8 13:10000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04ea800 con 0x55eea04c6c00 2024-04-25T10:13:19.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.439+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 10 ==== osd_op_reply(64 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.439+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:65 13.28 13:14000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04dbc00 con 0x55eea04c6c00 2024-04-25T10:13:19.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.439+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 11 ==== osd_op_reply(65 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.439+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:66 13.18 13:18000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04eac00 con 0x55eea04a7000 2024-04-25T10:13:19.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.443+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 11 ==== osd_op_reply(66 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.443+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:67 13.38 13:1c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da800 con 0x55eea1002c00 2024-04-25T10:13:19.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.443+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.5 v2:172.21.15.100:6800/3542120824 7 ==== osd_op_reply(67 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea1002c00 2024-04-25T10:13:19.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.443+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:68 13.4 13:20000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04eb000 con 0x55eea1002400 2024-04-25T10:13:19.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.443+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 11 ==== osd_op_reply(68 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.443+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:69 13.24 13:24000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea0e52c00 con 0x55eea1002400 2024-04-25T10:13:19.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.443+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.443+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:70 13.14 13:28000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04eb400 con 0x55eea04a7400 2024-04-25T10:13:19.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.443+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.0 v2:172.21.15.60:6801/4056167511 5 ==== osd_op_reply(70 [pgnls-filter start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04a7400 2024-04-25T10:13:19.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.443+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:71 13.34 13:2c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7c00 con 0x55eea1002400 2024-04-25T10:13:19.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.443+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 13 ==== osd_op_reply(71 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.443+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:72 13.c 13:30000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04eb800 con 0x55eea04c6400 2024-04-25T10:13:19.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.443+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 10 ==== osd_op_reply(72 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.443+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:73 13.2c 13:34000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c6000 con 0x55eea04c6c00 2024-04-25T10:13:19.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.443+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.443+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04ebc00 con 0x55eea1002400 2024-04-25T10:13:19.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 14 ==== osd_op_reply(74 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:75 13.3c 13:3c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c6800 con 0x55eea04c6c00 2024-04-25T10:13:19.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 13 ==== osd_op_reply(75 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:76 13.2 13:40000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da000 con 0x55eea1002c00 2024-04-25T10:13:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.5 v2:172.21.15.100:6800/3542120824 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea1002c00 2024-04-25T10:13:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:77 13.22 13:44000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea1002000 con 0x55eea04c6400 2024-04-25T10:13:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 11 ==== osd_op_reply(77 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:78 13.12 13:48000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da400 con 0x55eea04a7400 2024-04-25T10:13:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.0 v2:172.21.15.60:6801/4056167511 6 ==== osd_op_reply(78 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04a7400 2024-04-25T10:13:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:79 13.32 13:4c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04a6c00 con 0x55eea04c6c00 2024-04-25T10:13:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:80 13.a 13:50000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04ea400 con 0x55eea04c6c00 2024-04-25T10:13:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 15 ==== osd_op_reply(80 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:81 13.2a 13:54000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04a7800 con 0x55eea04c6400 2024-04-25T10:13:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 12 ==== osd_op_reply(81 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:82 13.1a 13:58000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04eb400 con 0x55eea04c6400 2024-04-25T10:13:19.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 13 ==== osd_op_reply(82 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:83 13.3a 13:5c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04db800 con 0x55eea1002c00 2024-04-25T10:13:19.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.5 v2:172.21.15.100:6800/3542120824 9 ==== osd_op_reply(83 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea1002c00 2024-04-25T10:13:19.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.447+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:84 13.6 13:60000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04eb800 con 0x55eea04c7000 2024-04-25T10:13:19.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.451+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 12 ==== osd_op_reply(84 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.451+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:85 13.26 13:64000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04db400 con 0x55eea04a7000 2024-04-25T10:13:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.451+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.451+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:86 13.16 13:68000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04ea000 con 0x55eea04dac00 2024-04-25T10:13:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.451+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.3 v2:172.21.15.60:6814/1867957156 5 ==== osd_op_reply(86 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04dac00 2024-04-25T10:13:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.451+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:87 13.36 13:6c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04db000 con 0x55eea1002400 2024-04-25T10:13:19.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.451+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 15 ==== osd_op_reply(87 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.451+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:88 13.e 13:70000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7400 con 0x55eea04a7000 2024-04-25T10:13:19.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.451+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.451+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:89 13.2e 13:74000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04ea800 con 0x55eea04c6c00 2024-04-25T10:13:19.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.451+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.451+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:90 13.1e 13:78000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7800 con 0x55eea04c7000 2024-04-25T10:13:19.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.451+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 13 ==== osd_op_reply(90 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.451+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:91 13.3e 13:7c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04eac00 con 0x55eea04c6400 2024-04-25T10:13:19.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.451+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 14 ==== osd_op_reply(91 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.451+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:92 13.1 13:80000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea1002800 con 0x55eea1002400 2024-04-25T10:13:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.455+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 16 ==== osd_op_reply(92 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.455+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:93 13.21 13:84000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04ea400 con 0x55eea04c7000 2024-04-25T10:13:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.455+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 14 ==== osd_op_reply(93 [pgnls-filter start_epoch 86 out=49b] v82'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.455+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:94 13.11 13:88000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea1002800 con 0x55eea04c6400 2024-04-25T10:13:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.455+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 15 ==== osd_op_reply(94 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.455+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:95 13.31 13:8c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04a6c00 con 0x55eea04dac00 2024-04-25T10:13:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.455+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.3 v2:172.21.15.60:6814/1867957156 6 ==== osd_op_reply(95 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04dac00 2024-04-25T10:13:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.455+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:96 13.9 13:90000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7c00 con 0x55eea1002c00 2024-04-25T10:13:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.455+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.5 v2:172.21.15.100:6800/3542120824 10 ==== osd_op_reply(96 [pgnls-filter start_epoch 86 out=49b] v82'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea1002c00 2024-04-25T10:13:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.455+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:97 13.29 13:94000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04dbc00 con 0x55eea1002400 2024-04-25T10:13:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.455+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 17 ==== osd_op_reply(97 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.455+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:98 13.19 13:98000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7800 con 0x55eea04dac00 2024-04-25T10:13:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.455+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.3 v2:172.21.15.60:6814/1867957156 7 ==== osd_op_reply(98 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04dac00 2024-04-25T10:13:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.455+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:99 13.39 13:9c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da800 con 0x55eea04c6c00 2024-04-25T10:13:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.459+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 17 ==== osd_op_reply(99 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.459+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:100 13.5 13:a0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04db000 con 0x55eea04c7000 2024-04-25T10:13:19.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.459+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 15 ==== osd_op_reply(100 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.459+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:101 13.25 13:a4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da000 con 0x55eea04c7000 2024-04-25T10:13:19.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.459+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 16 ==== osd_op_reply(101 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.459+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:102 13.15 13:a8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04eb000 con 0x55eea04c7000 2024-04-25T10:13:19.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.459+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 17 ==== osd_op_reply(102 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.459+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:103 13.35 13:ac000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c6000 con 0x55eea04dac00 2024-04-25T10:13:19.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.459+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.3 v2:172.21.15.60:6814/1867957156 8 ==== osd_op_reply(103 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04dac00 2024-04-25T10:13:19.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.459+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:104 13.d 13:b0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04ebc00 con 0x55eea04c7000 2024-04-25T10:13:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.459+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 18 ==== osd_op_reply(104 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.459+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:105 13.2d 13:b4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c6800 con 0x55eea04a7400 2024-04-25T10:13:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.459+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.0 v2:172.21.15.60:6801/4056167511 7 ==== osd_op_reply(105 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04a7400 2024-04-25T10:13:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.459+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:106 13.1d 13:b8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04eb800 con 0x55eea04a7000 2024-04-25T10:13:19.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.459+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 14 ==== osd_op_reply(106 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.459+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:107 13.3d 13:bc000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c6800 con 0x55eea04a7000 2024-04-25T10:13:19.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.463+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.463+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:108 13.3 13:c0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04ea000 con 0x55eea04a7400 2024-04-25T10:13:19.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.463+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.0 v2:172.21.15.60:6801/4056167511 8 ==== osd_op_reply(108 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04a7400 2024-04-25T10:13:19.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.463+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:109 13.23 13:c4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea1002000 con 0x55eea04c6400 2024-04-25T10:13:19.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.463+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 16 ==== osd_op_reply(109 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.463+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:110 13.13 13:c8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea0e52c00 con 0x55eea04a7000 2024-04-25T10:13:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.463+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 16 ==== osd_op_reply(110 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.463+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:111 13.33 13:cc000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea1002800 con 0x55eea04a7000 2024-04-25T10:13:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.463+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 17 ==== osd_op_reply(111 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.463+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:112 13.b 13:d0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea1002800 con 0x55eea1002c00 2024-04-25T10:13:19.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.463+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.5 v2:172.21.15.100:6800/3542120824 11 ==== osd_op_reply(112 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea1002c00 2024-04-25T10:13:19.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.463+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:113 13.2b 13:d4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04eb400 con 0x55eea04c6c00 2024-04-25T10:13:19.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.463+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.2 v2:172.21.15.60:6800/1771147646 18 ==== osd_op_reply(113 [pgnls-filter start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea04c6c00 2024-04-25T10:13:19.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.463+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:114 13.1b 13:d8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea0e52c00 con 0x55eea04c7000 2024-04-25T10:13:19.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.467+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 19 ==== osd_op_reply(114 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.467+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:115 13.3b 13:dc000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04eac00 con 0x55eea04c7000 2024-04-25T10:13:19.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.467+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 20 ==== osd_op_reply(115 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.467+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:116 13.7 13:e0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7400 con 0x55eea04a7000 2024-04-25T10:13:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.467+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 18 ==== osd_op_reply(116 [pgnls-filter start_epoch 86 out=81b] v86'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.467+0000 7fe0fadd98c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-04-25T10:13:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.467+0000 7fe0fadd98c0 10 datascan.operator(): handling object 10000000001.0 2024-04-25T10:13:19.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.467+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:117 13.7 13:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04ea400 con 0x55eea04a7000 2024-04-25T10:13:19.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.467+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.7 v2:172.21.15.100:6816/2771485684 19 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x55eea0142000 con 0x55eea04a7000 2024-04-25T10:13:19.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.467+0000 7fe0fadd98c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-04-25T10:13:19.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.467+0000 7fe0fadd98c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-04-25T10:13:19.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.467+0000 7fe0fadd98c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-04-25T10:13:19.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.467+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:118 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04eb000 con 0x55eea04a7400 2024-04-25T10:13:19.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.467+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.0 v2:172.21.15.60:6801/4056167511 9 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+59 (crc 0 0 0) 0x55eea0142780 con 0x55eea04a7400 2024-04-25T10:13:19.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.467+0000 7fe0fadd98c0 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-04-25T10:13:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.467+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04ebc00 con 0x55eea1002400 2024-04-25T10:13:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 18 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+520 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0fadd98c0 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-04-25T10:13:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0fadd98c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-04-25T10:13:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0fadd98c0 20 datascan.get_frag_of: frag is * 2024-04-25T10:13:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:120 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7800 con 0x55eea04a7400 2024-04-25T10:13:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.0 v2:172.21.15.60:6801/4056167511 10 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55eea0142780 con 0x55eea04a7400 2024-04-25T10:13:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0fadd98c0 10 datascan.find_or_create_dirfrag: read_version = 17 2024-04-25T10:13:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0fadd98c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-04-25T10:13:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:121 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c6000 con 0x55eea04a7400 2024-04-25T10:13:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.0 v2:172.21.15.60:6801/4056167511 11 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55eea0142780 con 0x55eea04a7400 2024-04-25T10:13:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0fadd98c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:13:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0fadd98c0 20 datascan.read_dentry: file_head not found in result 2024-04-25T10:13:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0fadd98c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-04-25T10:13:19.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:122 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04dbc00 con 0x55eea04a7400 2024-04-25T10:13:19.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.0 v2:172.21.15.60:6801/4056167511 12 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v86'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55eea0142780 con 0x55eea04a7400 2024-04-25T10:13:19.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0fadd98c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:13:19.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:123 13.27 13:e4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04db000 con 0x55eea04c6400 2024-04-25T10:13:19.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 <== osd.1 v2:172.21.15.60:6824/3313552458 17 ==== osd_op_reply(123 [pgnls-filter start_epoch 86 out=49b] v82'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142780 con 0x55eea04c6400 2024-04-25T10:13:19.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.471+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:124 13.17 13:e8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c7c00 con 0x55eea1002c00 2024-04-25T10:13:19.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.475+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.5 v2:172.21.15.100:6800/3542120824 12 ==== osd_op_reply(124 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea1002c00 2024-04-25T10:13:19.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.475+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:125 13.37 13:ec000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04a7800 con 0x55eea1002c00 2024-04-25T10:13:19.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.475+0000 7fe0f9598640 1 -- 172.21.15.60:0/3667315397 <== osd.5 v2:172.21.15.100:6800/3542120824 13 ==== osd_op_reply(125 [pgnls-filter start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142000 con 0x55eea1002c00 2024-04-25T10:13:19.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.475+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:126 13.f 13:f0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04c6800 con 0x55eea1002400 2024-04-25T10:13:19.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.475+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 19 ==== osd_op_reply(126 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.475+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:127 13.2f 13:f4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04da400 con 0x55eea04c7000 2024-04-25T10:13:19.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.475+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.4 v2:172.21.15.100:6808/3965541654 21 ==== osd_op_reply(127 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea04c7000 2024-04-25T10:13:19.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.475+0000 7fe0f458e640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:128 13.1f 13:f8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea04ea800 con 0x55eea1002400 2024-04-25T10:13:19.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.475+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 20 ==== osd_op_reply(128 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0f4d8f640 1 -- 172.21.15.60:0/3667315397 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:129 13.3f 13:fc000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x55eea0e52c00 con 0x55eea1002400 2024-04-25T10:13:19.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0f9d99640 1 -- 172.21.15.60:0/3667315397 <== osd.6 v2:172.21.15.100:6824/4086390550 21 ==== osd_op_reply(129 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eea0142500 con 0x55eea1002400 2024-04-25T10:13:19.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x55eea04a7400 msgr2=0x55eea04a9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x55eea04a7400 0x55eea04a9600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55eea04c6400 msgr2=0x55eea04a8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55eea04c6400 0x55eea04a8580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x55eea04c6c00 msgr2=0x55eea04a9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x55eea04c6c00 0x55eea04a9080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55eea04dac00 msgr2=0x55eea04a9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55eea04dac00 0x55eea04a9b80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55eea04c7000 msgr2=0x55eea04a8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 reap_dead start 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55eea04c7000 0x55eea04a8b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 reap_dead start 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55eea1002c00 msgr2=0x55eea024a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55eea1002c00 0x55eea024a680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x55eea1002400 msgr2=0x55eea024a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x55eea1002400 0x55eea024a100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x55eea04a7000 msgr2=0x55eea0248580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x55eea04a7000 0x55eea0248580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55eea04a6000 msgr2=0x55eea04a8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55eea04a6000 0x55eea04a8000 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x55eea016a150 tx=0x55eea1010420 comp rx=0 tx=0).stop 2024-04-25T10:13:19.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0f8d97640 1 -- 172.21.15.60:0/3667315397 reap_dead start 2024-04-25T10:13:19.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea1003000 msgr2=0x55eea024ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea1003000 0x55eea024ac00 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x55eea0fadc50 tx=0x55eea1010000 comp rx=0 tx=0).stop 2024-04-25T10:13:19.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 shutdown_connections 2024-04-25T10:13:19.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/3667315397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea1003000 0x55eea024ac00 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 >> 172.21.15.60:0/3667315397 conn(0x55eea0e53400 msgr2=0x55eea00bd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 shutdown_connections 2024-04-25T10:13:19.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/3667315397 wait complete. 2024-04-25T10:13:19.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/2602282541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea0e52000 msgr2=0x55eea0249080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/2602282541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea0e52000 0x55eea0249080 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x55eea0edba10 tx=0x55eea018c180 comp rx=0 tx=0).stop 2024-04-25T10:13:19.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/2602282541 shutdown_connections 2024-04-25T10:13:19.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/2602282541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea0e53000 0x55eea0249600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/2602282541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea008fc00 0x55eea0248b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 --2- 172.21.15.60:0/2602282541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea0e52000 0x55eea0249080 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/2602282541 >> 172.21.15.60:0/2602282541 conn(0x55eea008ec00 msgr2=0x55eea00bc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/2602282541 shutdown_connections 2024-04-25T10:13:19.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.479+0000 7fe0fadd98c0 1 -- 172.21.15.60:0/2602282541 wait complete. 2024-04-25T10:13:19.490 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.164964, result: 2024-04-25T10:13:19.490 DEBUG:teuthology.orchestra.run.smithi060:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-25T10:13:19.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.563+0000 7f4fc8c8e8c0 1 -- start start 2024-04-25T10:13:19.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.563+0000 7f4fc8c8e8c0 1 Processor -- start 2024-04-25T10:13:19.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.563+0000 7f4fc8c8e8c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c1d21000 0x5619c1eda580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.563+0000 7f4fc8c8e8c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c1d21400 0x5619c1edab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.563+0000 7f4fc8c8e8c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5619c1d21800 0x5619c1edb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.563+0000 7f4fc8c8e8c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c2b00600 con 0x5619c1d21400 2024-04-25T10:13:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.563+0000 7f4fc8c8e8c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c2b00480 con 0x5619c1d21000 2024-04-25T10:13:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.563+0000 7f4fc8c8e8c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619c2b00300 con 0x5619c1d21800 2024-04-25T10:13:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc6c4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c1d21400 0x5619c1edab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc6c4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c1d21400 0x5619c1edab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54976/0 (socket says 172.21.15.60:54976) 2024-04-25T10:13:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/3459795979 learned_addr learned my addr 172.21.15.60:0/3459795979 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc744d640 1 --2- 172.21.15.60:0/3459795979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c1d21000 0x5619c1eda580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc7c4e640 1 --2- 172.21.15.60:0/3459795979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5619c1d21800 0x5619c1edb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/3459795979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5619c1d21800 msgr2=0x5619c1edb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc6c4c640 1 --2- 172.21.15.60:0/3459795979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5619c1d21800 0x5619c1edb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/3459795979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c1d21000 msgr2=0x5619c1eda580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc6c4c640 1 --2- 172.21.15.60:0/3459795979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c1d21000 0x5619c1eda580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/3459795979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c1d461e0 con 0x5619c1d21400 2024-04-25T10:13:19.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc6c4c640 1 --2- 172.21.15.60:0/3459795979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c1d21400 0x5619c1edab00 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x5619c2b734d0 tx=0x5619c2af4600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25efbab8a4035d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc4c48640 1 -- 172.21.15.60:0/3459795979 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5619c1d564e0 con 0x5619c1d21400 2024-04-25T10:13:19.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc4c48640 1 -- 172.21.15.60:0/3459795979 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619c1d6a1c0 con 0x5619c1d21400 2024-04-25T10:13:19.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc4c48640 1 -- 172.21.15.60:0/3459795979 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5619c1d56b60 con 0x5619c1d21400 2024-04-25T10:13:19.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/3459795979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619c1d472c0 con 0x5619c1d21400 2024-04-25T10:13:19.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/3459795979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5619c1d47a40 con 0x5619c1d21400 2024-04-25T10:13:19.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 4 waiting for MDS map... 2024-04-25T10:13:19.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc4c48640 1 -- 172.21.15.60:0/3459795979 <== mon.0 v2:172.21.15.60:3300/0 4 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x5619c1db5000 con 0x5619c1d21400 2024-04-25T10:13:19.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc4c48640 1 -- 172.21.15.60:0/3459795979 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 63) v1 ==== 2723+0+0 (secure 0 0 0) 0x5619c1d77180 con 0x5619c1d21400 2024-04-25T10:13:19.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 4 Got MDS map 63 2024-04-25T10:13:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 4 datascan.main: Using metadata pool output 2024-04-25T10:13:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 4 datascan.main: connecting to RADOS... 2024-04-25T10:13:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 Processor -- start 2024-04-25T10:13:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 -- start start 2024-04-25T10:13:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c2c94400 0x5619c1edb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5619c2c94800 0x5619c1edbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c2c94c00 0x5619c1edc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c2b00180 con 0x5619c2c94400 2024-04-25T10:13:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c2b00900 con 0x5619c2c94c00 2024-04-25T10:13:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619c1d80900 con 0x5619c2c94800 2024-04-25T10:13:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc7c4e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c2c94400 0x5619c1edb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc7c4e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c2c94400 0x5619c1edb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54984/0 (socket says 172.21.15.60:54984) 2024-04-25T10:13:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/4096847929 learned_addr learned my addr 172.21.15.60:0/4096847929 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:19.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc6c4c640 1 --2- 172.21.15.60:0/4096847929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c2c94c00 0x5619c1edc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/4096847929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5619c2c94800 msgr2=0x5619c1edbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc7c4e640 1 --2- 172.21.15.60:0/4096847929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5619c2c94800 0x5619c1edbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/4096847929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c2c94c00 msgr2=0x5619c1edc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc7c4e640 1 --2- 172.21.15.60:0/4096847929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c2c94c00 0x5619c1edc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/4096847929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c1d47c20 con 0x5619c2c94400 2024-04-25T10:13:19.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc7c4e640 1 --2- 172.21.15.60:0/4096847929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c2c94400 0x5619c1edb600 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x5619c2c50840 tx=0x5619c2af4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e128cb40f74afd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc2c44640 1 -- 172.21.15.60:0/4096847929 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5619c1d569c0 con 0x5619c2c94400 2024-04-25T10:13:19.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc2c44640 1 -- 172.21.15.60:0/4096847929 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619c1d6ae00 con 0x5619c2c94400 2024-04-25T10:13:19.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc2c44640 1 -- 172.21.15.60:0/4096847929 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5619c1d564e0 con 0x5619c2c94400 2024-04-25T10:13:19.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/4096847929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c2c94400 msgr2=0x5619c1edb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/4096847929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c2c94400 0x5619c1edb600 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x5619c2c50840 tx=0x5619c2af4000 comp rx=0 tx=0).stop 2024-04-25T10:13:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/4096847929 shutdown_connections 2024-04-25T10:13:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/4096847929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c2c94c00 0x5619c1edc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/4096847929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5619c2c94800 0x5619c1edbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/4096847929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c2c94400 0x5619c1edb600 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/4096847929 >> 172.21.15.60:0/4096847929 conn(0x5619c2c94000 msgr2=0x5619c1d4e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/4096847929 shutdown_connections 2024-04-25T10:13:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.567+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/4096847929 wait complete. 2024-04-25T10:13:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc8c8e8c0 1 Processor -- start 2024-04-25T10:13:19.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc8c8e8c0 1 -- start start 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc8c8e8c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5619c2c94400 0x5619c1edd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc8c8e8c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c2c94800 0x5619c1edcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc8c8e8c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c2c94c00 0x5619c1edc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc8c8e8c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c2b01380 con 0x5619c2c94c00 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc8c8e8c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c2b00900 con 0x5619c2c94800 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc8c8e8c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619c2b01500 con 0x5619c2c94400 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc6c4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c2c94c00 0x5619c1edc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc6c4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c2c94c00 0x5619c1edc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55000/0 (socket says 172.21.15.60:55000) 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 learned_addr learned my addr 172.21.15.60:0/2382168501 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc7c4e640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5619c2c94400 0x5619c1edd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc744d640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c2c94800 0x5619c1edcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5619c2c94400 msgr2=0x5619c1edd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc6c4c640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5619c2c94400 0x5619c1edd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c2c94800 msgr2=0x5619c1edcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc6c4c640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c2c94800 0x5619c1edcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c1d474a0 con 0x5619c2c94c00 2024-04-25T10:13:19.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc744d640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c2c94800 0x5619c1edcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:19.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc6c4c640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c2c94c00 0x5619c1edc680 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x5619c2c51c50 tx=0x5619c2af4540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc1441640 1 -- 172.21.15.60:0/2382168501 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5619c1d56340 con 0x5619c2c94c00 2024-04-25T10:13:19.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc1441640 1 -- 172.21.15.60:0/2382168501 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619c1d6ac40 con 0x5619c2c94c00 2024-04-25T10:13:19.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc1441640 1 -- 172.21.15.60:0/2382168501 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5619c1d569c0 con 0x5619c2c94c00 2024-04-25T10:13:19.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5619c1d463c0 con 0x5619c2c94c00 2024-04-25T10:13:19.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619c1d47c20 con 0x5619c2c94c00 2024-04-25T10:13:19.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc8c8e8c0 4 datascan.init: resolving metadata pool 12 2024-04-25T10:13:19.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc1441640 1 -- 172.21.15.60:0/2382168501 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x5619c2c36c00 con 0x5619c2c94c00 2024-04-25T10:13:19.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc1441640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5619c2c95c00 0x5619c1edd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc1441640 1 -- 172.21.15.60:0/2382168501 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x5619c2050200 con 0x5619c2c94c00 2024-04-25T10:13:19.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc8c8e8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:13:19.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc8c8e8c0 4 datascan.main: resolving metadata pool 12 2024-04-25T10:13:19.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x5619c213c000 0x5619c1edc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213c400 con 0x5619c213c000 2024-04-25T10:13:19.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc7c4e640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5619c2c95c00 0x5619c1edd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.571+0000 7f4fc744d640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x5619c213c000 0x5619c1edc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc7c4e640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5619c2c95c00 0x5619c1edd700 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x5619c2c50840 tx=0x5619c2af4360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc744d640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x5619c213c000 0x5619c1edc100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.4 v2:172.21.15.100:6808/3965541654 1 ==== osd_op_reply(1 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c213c000 2024-04-25T10:13:19.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc744d640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5619c213c400 0x5619c2146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213d400 con 0x5619c213c400 2024-04-25T10:13:19.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc7c4e640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5619c213c400 0x5619c2146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc7c4e640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5619c213c400 0x5619c2146000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 1 ==== osd_op_reply(2 [pgnls start_epoch 86 out=177b] v86'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc7c4e640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x5619c213d400 0x5619c2146580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2b04c00 con 0x5619c213d400 2024-04-25T10:13:19.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc744d640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x5619c213d400 0x5619c2146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc744d640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x5619c213d400 0x5619c2146580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.3 v2:172.21.15.60:6814/1867957156 1 ==== osd_op_reply(3 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c213d400 2024-04-25T10:13:19.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc744d640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5619c2b04c00 0x5619c2146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213d800 con 0x5619c2b04c00 2024-04-25T10:13:19.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc6c4c640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5619c2b04c00 0x5619c2146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc6c4c640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5619c2b04c00 0x5619c2146b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.1 v2:172.21.15.60:6824/3313552458 1 ==== osd_op_reply(4 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b04c00 2024-04-25T10:13:19.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc6c4c640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x5619c213d800 0x5619c2147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215a000 con 0x5619c213d800 2024-04-25T10:13:19.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.575+0000 7f4fc7c4e640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x5619c213d800 0x5619c2147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.579+0000 7f4fc7c4e640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x5619c213d800 0x5619c2147600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.579+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 1 ==== osd_op_reply(5 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.579+0000 7f4fc7c4e640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x5619c215a000 0x5619c2147b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.579+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2b05000 con 0x5619c215a000 2024-04-25T10:13:19.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.579+0000 7f4fc744d640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x5619c215a000 0x5619c2147b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.579+0000 7f4fc744d640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x5619c215a000 0x5619c2147b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.579+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.5 v2:172.21.15.100:6800/3542120824 1 ==== osd_op_reply(6 [pgnls start_epoch 86 out=149b] v85'27 uv27 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c215a000 2024-04-25T10:13:19.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.579+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2b05000 con 0x5619c213c400 2024-04-25T10:13:19.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.579+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 2 ==== osd_op_reply(7 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.579+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2b05000 con 0x5619c213c400 2024-04-25T10:13:19.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.579+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 3 ==== osd_op_reply(8 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.579+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2b05000 con 0x5619c213c000 2024-04-25T10:13:19.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.4 v2:172.21.15.100:6808/3965541654 2 ==== osd_op_reply(9 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c213c000 2024-04-25T10:13:19.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2b05000 con 0x5619c213d400 2024-04-25T10:13:19.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.3 v2:172.21.15.60:6814/1867957156 2 ==== osd_op_reply(10 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c213d400 2024-04-25T10:13:19.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc744d640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x5619c2b05000 0x5619c2147080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213dc00 con 0x5619c2b05000 2024-04-25T10:13:19.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc6c4c640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x5619c2b05000 0x5619c2147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc6c4c640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x5619c2b05000 0x5619c2147080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.2 v2:172.21.15.60:6800/1771147646 1 ==== osd_op_reply(11 [pgnls start_epoch 86 out=81b] v85'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b05000 2024-04-25T10:13:19.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213dc00 con 0x5619c213c000 2024-04-25T10:13:19.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.4 v2:172.21.15.100:6808/3965541654 3 ==== osd_op_reply(12 [pgnls start_epoch 86 out=49b] v83'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c213c000 2024-04-25T10:13:19.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213dc00 con 0x5619c2b04c00 2024-04-25T10:13:19.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.1 v2:172.21.15.60:6824/3313552458 2 ==== osd_op_reply(13 [pgnls start_epoch 86 out=77b] v85'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b04c00 2024-04-25T10:13:19.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213dc00 con 0x5619c213d800 2024-04-25T10:13:19.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 2 ==== osd_op_reply(14 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213dc00 con 0x5619c213c400 2024-04-25T10:13:19.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 4 ==== osd_op_reply(15 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213dc00 con 0x5619c213c000 2024-04-25T10:13:19.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.4 v2:172.21.15.100:6808/3965541654 4 ==== osd_op_reply(16 [pgnls start_epoch 86 out=49b] v83'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c213c000 2024-04-25T10:13:19.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.583+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213dc00 con 0x5619c213d800 2024-04-25T10:13:19.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.587+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 3 ==== osd_op_reply(17 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.587+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213dc00 con 0x5619c215a000 2024-04-25T10:13:19.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.587+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.5 v2:172.21.15.100:6800/3542120824 2 ==== osd_op_reply(18 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c215a000 2024-04-25T10:13:19.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.587+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213dc00 con 0x5619c2b05000 2024-04-25T10:13:19.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.587+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.2 v2:172.21.15.60:6800/1771147646 2 ==== osd_op_reply(19 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b05000 2024-04-25T10:13:19.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.587+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213dc00 con 0x5619c213c400 2024-04-25T10:13:19.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.587+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 5 ==== osd_op_reply(20 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.587+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213dc00 con 0x5619c213d800 2024-04-25T10:13:19.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.587+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 4 ==== osd_op_reply(21 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.587+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213dc00 con 0x5619c213d800 2024-04-25T10:13:19.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.587+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 5 ==== osd_op_reply(22 [pgnls start_epoch 86 out=97b] v85'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.587+0000 7f4fc7c4e640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5619c213dc00 0x5619c2148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.587+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b000 con 0x5619c213dc00 2024-04-25T10:13:19.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.587+0000 7f4fc7c4e640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5619c213dc00 0x5619c2148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc7c4e640 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5619c213dc00 0x5619c2148100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.7 v2:172.21.15.100:6816/2771485684 1 ==== osd_op_reply(23 [pgnls start_epoch 86 out=74b] v81'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213dc00 2024-04-25T10:13:19.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b000 con 0x5619c213c400 2024-04-25T10:13:19.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 6 ==== osd_op_reply(24 [pgnls start_epoch 86 out=73b] v85'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b000 con 0x5619c213c400 2024-04-25T10:13:19.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 7 ==== osd_op_reply(25 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b000 con 0x5619c2b04c00 2024-04-25T10:13:19.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.1 v2:172.21.15.60:6824/3313552458 3 ==== osd_op_reply(26 [pgnls start_epoch 86 out=98b] v85'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b04c00 2024-04-25T10:13:19.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b000 con 0x5619c213d400 2024-04-25T10:13:19.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.3 v2:172.21.15.60:6814/1867957156 3 ==== osd_op_reply(27 [pgnls start_epoch 86 out=76b] v85'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c213d400 2024-04-25T10:13:19.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b000 con 0x5619c213c000 2024-04-25T10:13:19.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.4 v2:172.21.15.100:6808/3965541654 5 ==== osd_op_reply(28 [pgnls start_epoch 86 out=73b] v85'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c213c000 2024-04-25T10:13:19.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b000 con 0x5619c2b04c00 2024-04-25T10:13:19.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.1 v2:172.21.15.60:6824/3313552458 4 ==== osd_op_reply(29 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b04c00 2024-04-25T10:13:19.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b000 con 0x5619c213d800 2024-04-25T10:13:19.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 6 ==== osd_op_reply(30 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.591+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b000 con 0x5619c213d800 2024-04-25T10:13:19.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 7 ==== osd_op_reply(31 [pgnls start_epoch 86 out=73b] v81'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b000 con 0x5619c213d800 2024-04-25T10:13:19.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 8 ==== osd_op_reply(32 [pgnls start_epoch 86 out=101b] v85'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-25T10:13:19.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213c800 con 0x5619c213c400 2024-04-25T10:13:19.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-25T10:13:19.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:34 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213cc00 con 0x5619c213c400 2024-04-25T10:13:19.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T10:13:19.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:35 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213d000 con 0x5619c213c400 2024-04-25T10:13:19.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 20 datascan.scan_links: injected ino detected 2024-04-25T10:13:19.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-25T10:13:19.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:36 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213c800 con 0x5619c213c400 2024-04-25T10:13:19.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-25T10:13:19.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:37 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213cc00 con 0x5619c213c400 2024-04-25T10:13:19.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T10:13:19.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:13:19.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-25T10:13:19.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:13:19.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-25T10:13:19.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:13:19.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-25T10:13:19.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:38 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215bc00 con 0x5619c215a000 2024-04-25T10:13:19.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.5 v2:172.21.15.100:6800/3542120824 3 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv25 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c215a000 2024-04-25T10:13:19.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-25T10:13:19.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:39 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b800 con 0x5619c213c400 2024-04-25T10:13:19.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-04-25T10:13:19.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:40 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b400 con 0x5619c2b05000 2024-04-25T10:13:19.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.595+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.2 v2:172.21.15.60:6800/1771147646 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b05000 2024-04-25T10:13:19.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-25T10:13:19.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:13:19.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-25T10:13:19.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:41 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b000 con 0x5619c213c400 2024-04-25T10:13:19.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-25T10:13:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:42 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2176000 con 0x5619c213d800 2024-04-25T10:13:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-25T10:13:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:13:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T10:13:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:13:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-25T10:13:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:43 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2176400 con 0x5619c213c400 2024-04-25T10:13:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T10:13:19.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:13:19.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T10:13:19.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:44 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2176800 con 0x5619c2b04c00 2024-04-25T10:13:19.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.1 v2:172.21.15.60:6824/3313552458 5 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b04c00 2024-04-25T10:13:19.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T10:13:19.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:13:19.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 200.00000002 2024-04-25T10:13:19.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-25T10:13:19.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-25T10:13:19.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:45 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2176c00 con 0x5619c2b04c00 2024-04-25T10:13:19.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.1 v2:172.21.15.60:6824/3313552458 6 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b04c00 2024-04-25T10:13:19.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-25T10:13:19.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:13:19.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T10:13:19.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:13:19.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T10:13:19.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:46 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177000 con 0x5619c213d800 2024-04-25T10:13:19.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 10 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1041 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:47 12.0 12:00000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213c000 2024-04-25T10:13:19.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.4 v2:172.21.15.100:6808/3965541654 6 ==== osd_op_reply(47 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c213c000 2024-04-25T10:13:19.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.599+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:48 12.10 12:08000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213c400 2024-04-25T10:13:19.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 16 ==== osd_op_reply(48 [pgnls start_epoch 86 out=177b] v86'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:49 12.8 12:10000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213d400 2024-04-25T10:13:19.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.3 v2:172.21.15.60:6814/1867957156 4 ==== osd_op_reply(49 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c213d400 2024-04-25T10:13:19.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:50 12.18 12:18000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c2b04c00 2024-04-25T10:13:19.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.1 v2:172.21.15.60:6824/3313552458 7 ==== osd_op_reply(50 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b04c00 2024-04-25T10:13:19.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:51 12.4 12:20000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213d800 2024-04-25T10:13:19.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 11 ==== osd_op_reply(51 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:52 12.14 12:28000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c215a000 2024-04-25T10:13:19.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.5 v2:172.21.15.100:6800/3542120824 4 ==== osd_op_reply(52 [pgnls start_epoch 86 out=149b] v85'27 uv27 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c215a000 2024-04-25T10:13:19.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:53 12.c 12:30000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213c400 2024-04-25T10:13:19.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 17 ==== osd_op_reply(53 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:54 12.1c 12:38000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213c400 2024-04-25T10:13:19.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 18 ==== osd_op_reply(54 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:55 12.2 12:40000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213c000 2024-04-25T10:13:19.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.4 v2:172.21.15.100:6808/3965541654 7 ==== osd_op_reply(55 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c213c000 2024-04-25T10:13:19.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:56 12.12 12:48000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213d400 2024-04-25T10:13:19.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.3 v2:172.21.15.60:6814/1867957156 5 ==== osd_op_reply(56 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c213d400 2024-04-25T10:13:19.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.603+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:57 12.a 12:50000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c2b05000 2024-04-25T10:13:19.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.607+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.2 v2:172.21.15.60:6800/1771147646 4 ==== osd_op_reply(57 [pgnls start_epoch 86 out=81b] v85'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b05000 2024-04-25T10:13:19.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.607+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:58 12.1a 12:58000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213c000 2024-04-25T10:13:19.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.607+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.4 v2:172.21.15.100:6808/3965541654 8 ==== osd_op_reply(58 [pgnls start_epoch 86 out=49b] v83'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c213c000 2024-04-25T10:13:19.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.607+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:59 12.6 12:60000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c2b04c00 2024-04-25T10:13:19.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.607+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.1 v2:172.21.15.60:6824/3313552458 8 ==== osd_op_reply(59 [pgnls start_epoch 86 out=77b] v85'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b04c00 2024-04-25T10:13:19.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.607+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:60 12.16 12:68000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213d800 2024-04-25T10:13:19.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.607+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 12 ==== osd_op_reply(60 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.607+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:61 12.e 12:70000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213c400 2024-04-25T10:13:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.607+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 19 ==== osd_op_reply(61 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.607+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:62 12.1e 12:78000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213c000 2024-04-25T10:13:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.607+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.4 v2:172.21.15.100:6808/3965541654 9 ==== osd_op_reply(62 [pgnls start_epoch 86 out=49b] v83'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c213c000 2024-04-25T10:13:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.607+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:63 12.1 12:80000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213d800 2024-04-25T10:13:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.607+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 13 ==== osd_op_reply(63 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.607+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:64 12.11 12:88000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c215a000 2024-04-25T10:13:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.611+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.5 v2:172.21.15.100:6800/3542120824 5 ==== osd_op_reply(64 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c215a000 2024-04-25T10:13:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.611+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:65 12.9 12:90000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c2b05000 2024-04-25T10:13:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.611+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.2 v2:172.21.15.60:6800/1771147646 5 ==== osd_op_reply(65 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b05000 2024-04-25T10:13:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.611+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:66 12.19 12:98000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213c400 2024-04-25T10:13:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.611+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 20 ==== osd_op_reply(66 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.611+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:67 12.5 12:a0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213d800 2024-04-25T10:13:19.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.611+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 14 ==== osd_op_reply(67 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.611+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:68 12.15 12:a8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213d800 2024-04-25T10:13:19.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.611+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 15 ==== osd_op_reply(68 [pgnls start_epoch 86 out=97b] v85'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.611+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:69 12.d 12:b0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213dc00 2024-04-25T10:13:19.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.611+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.7 v2:172.21.15.100:6816/2771485684 2 ==== osd_op_reply(69 [pgnls start_epoch 86 out=74b] v81'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213dc00 2024-04-25T10:13:19.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.611+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:70 12.1d 12:b8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213c400 2024-04-25T10:13:19.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.611+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 21 ==== osd_op_reply(70 [pgnls start_epoch 86 out=73b] v85'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.611+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:71 12.3 12:c0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213c400 2024-04-25T10:13:19.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.611+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 22 ==== osd_op_reply(71 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.611+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:72 12.13 12:c8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c2b04c00 2024-04-25T10:13:19.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.615+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.1 v2:172.21.15.60:6824/3313552458 9 ==== osd_op_reply(72 [pgnls start_epoch 86 out=98b] v85'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b04c00 2024-04-25T10:13:19.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.615+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:73 12.b 12:d0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213d400 2024-04-25T10:13:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.615+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.3 v2:172.21.15.60:6814/1867957156 6 ==== osd_op_reply(73 [pgnls start_epoch 86 out=76b] v85'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c213d400 2024-04-25T10:13:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.615+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:74 12.1b 12:d8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213c000 2024-04-25T10:13:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.615+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.4 v2:172.21.15.100:6808/3965541654 10 ==== osd_op_reply(74 [pgnls start_epoch 86 out=73b] v85'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c213c000 2024-04-25T10:13:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.615+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:75 12.7 12:e0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c2b04c00 2024-04-25T10:13:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.615+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.1 v2:172.21.15.60:6824/3313552458 10 ==== osd_op_reply(75 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b04c00 2024-04-25T10:13:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.615+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:76 12.17 12:e8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213d800 2024-04-25T10:13:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.615+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 16 ==== osd_op_reply(76 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.615+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:77 12.f 12:f0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213d800 2024-04-25T10:13:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.615+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 17 ==== osd_op_reply(77 [pgnls start_epoch 86 out=73b] v81'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.615+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:78 12.1f 12:f8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213d800 2024-04-25T10:13:19.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.615+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 18 ==== osd_op_reply(78 [pgnls start_epoch 86 out=101b] v85'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.615+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-25T10:13:19.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.615+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:79 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177800 con 0x5619c213c400 2024-04-25T10:13:19.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 23 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-25T10:13:19.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:80 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215ac00 con 0x5619c213c400 2024-04-25T10:13:19.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 24 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T10:13:19.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:81 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215a800 con 0x5619c213c400 2024-04-25T10:13:19.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 25 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 20 datascan.scan_links: injected ino detected 2024-04-25T10:13:19.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-04-25T10:13:19.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-04-25T10:13:19.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-25T10:13:19.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:82 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215a400 con 0x5619c213c400 2024-04-25T10:13:19.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 26 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-25T10:13:19.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:83 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b400 con 0x5619c213c400 2024-04-25T10:13:19.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 27 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T10:13:19.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:13:19.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-25T10:13:19.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:13:19.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-25T10:13:19.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:13:19.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-25T10:13:19.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:84 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b800 con 0x5619c215a000 2024-04-25T10:13:19.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 <== osd.5 v2:172.21.15.100:6800/3542120824 6 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv25 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e000 con 0x5619c215a000 2024-04-25T10:13:19.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-25T10:13:19.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:85 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213d000 con 0x5619c213c400 2024-04-25T10:13:19.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 28 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-04-25T10:13:19.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.619+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:86 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213c800 con 0x5619c2b05000 2024-04-25T10:13:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.2 v2:172.21.15.60:6800/1771147646 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b05000 2024-04-25T10:13:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-25T10:13:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:13:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-25T10:13:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:87 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213cc00 con 0x5619c213c400 2024-04-25T10:13:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 29 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-25T10:13:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:88 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215ac00 con 0x5619c213d800 2024-04-25T10:13:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 19 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-25T10:13:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:13:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T10:13:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:13:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-25T10:13:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215a800 con 0x5619c213c400 2024-04-25T10:13:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 30 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T10:13:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:13:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T10:13:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:90 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215a400 con 0x5619c2b04c00 2024-04-25T10:13:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.1 v2:172.21.15.60:6824/3313552458 11 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b04c00 2024-04-25T10:13:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T10:13:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:13:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 200.00000002 2024-04-25T10:13:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-25T10:13:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-25T10:13:19.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:91 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b400 con 0x5619c2b04c00 2024-04-25T10:13:19.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.1 v2:172.21.15.60:6824/3313552458 12 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b04c00 2024-04-25T10:13:19.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-25T10:13:19.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:13:19.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T10:13:19.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:13:19.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T10:13:19.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.623+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:92 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c215b000 con 0x5619c213d800 2024-04-25T10:13:19.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.627+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.6 v2:172.21.15.100:6824/4086390550 20 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1041 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213d800 2024-04-25T10:13:19.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.627+0000 7f4fc8c8e8c0 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-04-25T10:13:19.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.627+0000 7f4fc8c8e8c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T10:13:19.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.627+0000 7f4fc8c8e8c0 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-04-25T10:13:19.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.627+0000 7f4fc8c8e8c0 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-04-25T10:13:19.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.627+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:93 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c213d000 con 0x5619c213c400 2024-04-25T10:13:19.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.627+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 31 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.627+0000 7f4fc8c8e8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:13:19.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.627+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:94 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2176000 con 0x5619c213c400 2024-04-25T10:13:19.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.631+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 32 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v86'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.631+0000 7f4fc8c8e8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:13:19.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.631+0000 7f4fc8c8e8c0 10 datascan.scan_links: processing 1 injected_inos 2024-04-25T10:13:19.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.631+0000 7f4fc8c8e8c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-04-25T10:13:19.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.631+0000 7f4fc8c8e8c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-04-25T10:13:19.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.631+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:95 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2176400 con 0x5619c213c400 2024-04-25T10:13:19.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.631+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 33 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.631+0000 7f4fc8c8e8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:13:19.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.631+0000 7f4fc8c8e8c0 20 datascan.scan_links: first is now 2 2024-04-25T10:13:19.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.631+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:96 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2176800 con 0x5619c213c400 2024-04-25T10:13:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.631+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.0 v2:172.21.15.60:6801/4056167511 34 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v86'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5619c1e1e500 con 0x5619c213c400 2024-04-25T10:13:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.631+0000 7f4fc8c8e8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:13:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.631+0000 7f4fc8c8e8c0 10 datascan.scan_links: updating inotable 2024-04-25T10:13:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.631+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:97 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2176c00 con 0x5619c2b04c00 2024-04-25T10:13:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.631+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.1 v2:172.21.15.60:6824/3313552458 13 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b04c00 2024-04-25T10:13:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-04-25T10:13:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-04-25T10:13:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:98 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177000 con 0x5619c2b04c00 2024-04-25T10:13:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc6c4c640 1 -- 172.21.15.60:0/2382168501 <== osd.1 v2:172.21.15.60:6824/3313552458 14 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v86'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x5619c1e1e780 con 0x5619c2b04c00 2024-04-25T10:13:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 10 datascan.scan_links: updating snaptable 2024-04-25T10:13:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:99 12.d 12:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v9 -- 0x5619c2177400 con 0x5619c213dc00 2024-04-25T10:13:19.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc7c4e640 1 -- 172.21.15.60:0/2382168501 <== osd.7 v2:172.21.15.100:6816/2771485684 3 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x5619c1e1e280 con 0x5619c213dc00 2024-04-25T10:13:19.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5619c213c400 msgr2=0x5619c2146000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5619c213c400 0x5619c2146000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5619c2b04c00 msgr2=0x5619c2146b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5619c2b04c00 0x5619c2146b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x5619c2b05000 msgr2=0x5619c2147080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x5619c2b05000 0x5619c2147080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x5619c213d400 msgr2=0x5619c2146580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x5619c213d400 0x5619c2146580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x5619c213c000 msgr2=0x5619c1edc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x5619c213c000 0x5619c1edc100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x5619c215a000 msgr2=0x5619c2147b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x5619c215a000 0x5619c2147b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x5619c213d800 msgr2=0x5619c2147600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x5619c213d800 0x5619c2147600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5619c213dc00 msgr2=0x5619c2148100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5619c213dc00 0x5619c2148100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 reap_dead start 2024-04-25T10:13:19.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5619c2c95c00 msgr2=0x5619c1edd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5619c2c95c00 0x5619c1edd700 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x5619c2c50840 tx=0x5619c2af4360 comp rx=0 tx=0).stop 2024-04-25T10:13:19.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 reap_dead start 2024-04-25T10:13:19.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c2c94c00 msgr2=0x5619c1edc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/2382168501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c2c94c00 0x5619c1edc680 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x5619c2c51c50 tx=0x5619c2af4540 comp rx=0 tx=0).stop 2024-04-25T10:13:19.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 reap_dead start 2024-04-25T10:13:19.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 reap_dead start 2024-04-25T10:13:19.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 shutdown_connections 2024-04-25T10:13:19.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 >> 172.21.15.60:0/2382168501 conn(0x5619c2c94000 msgr2=0x5619c1d4e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc744d640 1 -- 172.21.15.60:0/2382168501 reap_dead start 2024-04-25T10:13:19.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 shutdown_connections 2024-04-25T10:13:19.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.635+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/2382168501 wait complete. 2024-04-25T10:13:19.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.639+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/3459795979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c1d21400 msgr2=0x5619c1edab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.639+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/3459795979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c1d21400 0x5619c1edab00 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x5619c2b734d0 tx=0x5619c2af4600 comp rx=0 tx=0).stop 2024-04-25T10:13:19.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.639+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/3459795979 shutdown_connections 2024-04-25T10:13:19.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.639+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/3459795979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5619c1d21800 0x5619c1edb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.639+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/3459795979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5619c1d21400 0x5619c1edab00 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.639+0000 7f4fc8c8e8c0 1 --2- 172.21.15.60:0/3459795979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5619c1d21000 0x5619c1eda580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.639+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/3459795979 >> 172.21.15.60:0/3459795979 conn(0x5619c1d20c00 msgr2=0x5619c1d4d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.639+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/3459795979 shutdown_connections 2024-04-25T10:13:19.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.639+0000 7f4fc8c8e8c0 1 -- 172.21.15.60:0/3459795979 wait complete. 2024-04-25T10:13:19.646 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.156327, result: 2024-04-25T10:13:19.646 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:13:19.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4dd6bb640 1 Processor -- start 2024-04-25T10:13:19.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4dd6bb640 1 -- start start 2024-04-25T10:13:19.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4dd6bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d815db10 0x7fc4d80b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4dd6bb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc4d80cfcf0 0x7fc4d80b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4dd6bb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d80ae140 con 0x7fc4d80aec10 2024-04-25T10:13:19.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4dd6bb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d80accc0 con 0x7fc4d80cfcf0 2024-04-25T10:13:19.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4dd6bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d815def0 con 0x7fc4d815db10 2024-04-25T10:13:19.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d67fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc4d80aec10 0x7fc4d8161660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56660/0 (socket says 172.21.15.60:56660) 2024-04-25T10:13:19.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d67fc640 1 -- 172.21.15.60:0/1575072019 learned_addr learned my addr 172.21.15.60:0/1575072019 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:13:19.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d6ffd640 1 --2- 172.21.15.60:0/1575072019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d815db10 0x7fc4d80b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d77fe640 1 --2- 172.21.15.60:0/1575072019 >> v2:172.21.15.100:3300/0 conn(0x7fc4d80cfcf0 0x7fc4d80b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d5ffb640 1 -- 172.21.15.60:0/1575072019 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 673148348 0 0) 0x7fc4cc001120 con 0x7fc4d80aec10 2024-04-25T10:13:19.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d5ffb640 1 -- 172.21.15.60:0/1575072019 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4c0003530 con 0x7fc4d80aec10 2024-04-25T10:13:19.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d5ffb640 1 -- 172.21.15.60:0/1575072019 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2628738216 0 0) 0x7fc4c0003530 con 0x7fc4d80aec10 2024-04-25T10:13:19.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d5ffb640 1 -- 172.21.15.60:0/1575072019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d815db10 msgr2=0x7fc4d80b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d5ffb640 1 --2- 172.21.15.60:0/1575072019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d815db10 0x7fc4d80b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d5ffb640 1 -- 172.21.15.60:0/1575072019 >> v2:172.21.15.100:3300/0 conn(0x7fc4d80cfcf0 msgr2=0x7fc4d80b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d5ffb640 1 --2- 172.21.15.60:0/1575072019 >> v2:172.21.15.100:3300/0 conn(0x7fc4d80cfcf0 0x7fc4d80b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d5ffb640 1 -- 172.21.15.60:0/1575072019 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d8071630 con 0x7fc4d80aec10 2024-04-25T10:13:19.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d5ffb640 1 -- 172.21.15.60:0/1575072019 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fc4cc001850 con 0x7fc4d80aec10 2024-04-25T10:13:19.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d5ffb640 1 -- 172.21.15.60:0/1575072019 >> v1:172.21.15.60:6789/0 conn(0x7fc4d80aec10 legacy=0x7fc4d8161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d5ffb640 1 --2- 172.21.15.60:0/1575072019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c00039c0 0x7fc4c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d5ffb640 1 --2- 172.21.15.60:0/1575072019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c0004810 0x7fc4c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d5ffb640 1 --2- 172.21.15.60:0/1575072019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d815db10 0x7fc4c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d5ffb640 1 -- 172.21.15.60:0/1575072019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d815def0 con 0x7fc4c0004810 2024-04-25T10:13:19.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d5ffb640 1 -- 172.21.15.60:0/1575072019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c000b9d0 con 0x7fc4c00039c0 2024-04-25T10:13:19.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d5ffb640 1 -- 172.21.15.60:0/1575072019 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c000bb50 con 0x7fc4d815db10 2024-04-25T10:13:19.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d67fc640 1 --2- 172.21.15.60:0/1575072019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c0004810 0x7fc4c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d6ffd640 1 --2- 172.21.15.60:0/1575072019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c00039c0 0x7fc4c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d77fe640 1 --2- 172.21.15.60:0/1575072019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d815db10 0x7fc4c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d67fc640 1 -- 172.21.15.60:0/1575072019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d815db10 msgr2=0x7fc4c00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d67fc640 1 --2- 172.21.15.60:0/1575072019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d815db10 0x7fc4c00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d67fc640 1 -- 172.21.15.60:0/1575072019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c00039c0 msgr2=0x7fc4c0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d67fc640 1 --2- 172.21.15.60:0/1575072019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c00039c0 0x7fc4c0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.819+0000 7fc4d67fc640 1 -- 172.21.15.60:0/1575072019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c0003530 con 0x7fc4c0004810 2024-04-25T10:13:19.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.823+0000 7fc4d67fc640 1 --2- 172.21.15.60:0/1575072019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c0004810 0x7fc4c0004c90 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00c4c0 tx=0x7fc4cc01f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92dcfb6788b3964 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.823+0000 7fc4d5ffb640 1 -- 172.21.15.60:0/1575072019 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc00c0e0 con 0x7fc4c0004810 2024-04-25T10:13:19.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.823+0000 7fc4d5ffb640 1 -- 172.21.15.60:0/1575072019 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4cc005b00 con 0x7fc4c0004810 2024-04-25T10:13:19.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.823+0000 7fc4d5ffb640 1 -- 172.21.15.60:0/1575072019 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc007070 con 0x7fc4c0004810 2024-04-25T10:13:19.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.823+0000 7fc4dd6bb640 1 -- 172.21.15.60:0/1575072019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c0004810 msgr2=0x7fc4c0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.823+0000 7fc4dd6bb640 1 --2- 172.21.15.60:0/1575072019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c0004810 0x7fc4c0004c90 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00c4c0 tx=0x7fc4cc01f440 comp rx=0 tx=0).stop 2024-04-25T10:13:19.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.823+0000 7fc4d77fe640 1 -- 172.21.15.60:0/1575072019 reap_dead start 2024-04-25T10:13:19.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.823+0000 7fc4dd6bb640 1 -- 172.21.15.60:0/1575072019 shutdown_connections 2024-04-25T10:13:19.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.823+0000 7fc4dd6bb640 1 -- 172.21.15.60:0/1575072019 >> 172.21.15.60:0/1575072019 conn(0x7fc4d80670b0 msgr2=0x7fc4d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.823+0000 7fc4dd6bb640 1 -- 172.21.15.60:0/1575072019 shutdown_connections 2024-04-25T10:13:19.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.823+0000 7fc4dd6bb640 1 -- 172.21.15.60:0/1575072019 wait complete. 2024-04-25T10:13:19.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.823+0000 7fc4dd6bb640 1 Processor -- start 2024-04-25T10:13:19.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4dd6bb640 1 -- start start 2024-04-25T10:13:19.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4dd6bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c0004810 0x7fc4d814dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4dd6bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d815fc10 0x7fc4d8151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4dd6bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d8151960 0x7fc4d80aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4dd6bb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d8071ed0 con 0x7fc4c0004810 2024-04-25T10:13:19.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4dd6bb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d8071bf0 con 0x7fc4d815fc10 2024-04-25T10:13:19.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4dd6bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d806db70 con 0x7fc4d8151960 2024-04-25T10:13:19.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c0004810 0x7fc4d814dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c0004810 0x7fc4d814dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55020/0 (socket says 172.21.15.60:55020) 2024-04-25T10:13:19.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4d6ffd640 1 -- 172.21.15.60:0/4263627292 learned_addr learned my addr 172.21.15.60:0/4263627292 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:19.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4d67fc640 1 --2- 172.21.15.60:0/4263627292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d815fc10 0x7fc4d8151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4d77fe640 1 --2- 172.21.15.60:0/4263627292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d8151960 0x7fc4d80aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4d6ffd640 1 -- 172.21.15.60:0/4263627292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d8151960 msgr2=0x7fc4d80aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4d6ffd640 1 --2- 172.21.15.60:0/4263627292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d8151960 0x7fc4d80aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4d6ffd640 1 -- 172.21.15.60:0/4263627292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d815fc10 msgr2=0x7fc4d8151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4d6ffd640 1 --2- 172.21.15.60:0/4263627292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d815fc10 0x7fc4d8151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4d6ffd640 1 -- 172.21.15.60:0/4263627292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d80ced80 con 0x7fc4c0004810 2024-04-25T10:13:19.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4d6ffd640 1 --2- 172.21.15.60:0/4263627292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c0004810 0x7fc4d814dba0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8011930 tx=0x7fc4c8011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4bf7fe640 1 -- 172.21.15.60:0/4263627292 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c800faf0 con 0x7fc4c0004810 2024-04-25T10:13:19.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4bf7fe640 1 -- 172.21.15.60:0/4263627292 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c800fc90 con 0x7fc4c0004810 2024-04-25T10:13:19.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4dd6bb640 1 -- 172.21.15.60:0/4263627292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d80af420 con 0x7fc4c0004810 2024-04-25T10:13:19.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4bf7fe640 1 -- 172.21.15.60:0/4263627292 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8014680 con 0x7fc4c0004810 2024-04-25T10:13:19.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4dd6bb640 1 -- 172.21.15.60:0/4263627292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d80af9e0 con 0x7fc4c0004810 2024-04-25T10:13:19.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4dd6bb640 1 -- 172.21.15.60:0/4263627292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d806bba0 con 0x7fc4c0004810 2024-04-25T10:13:19.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4bf7fe640 1 -- 172.21.15.60:0/4263627292 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4c8012040 con 0x7fc4c0004810 2024-04-25T10:13:19.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4bf7fe640 1 --2- 172.21.15.60:0/4263627292 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4c4066f90 0x7fc4c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.827+0000 7fc4bf7fe640 1 -- 172.21.15.60:0/4263627292 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fc4c80856e0 con 0x7fc4c0004810 2024-04-25T10:13:19.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.831+0000 7fc4d67fc640 1 --2- 172.21.15.60:0/4263627292 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4c4066f90 0x7fc4c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.831+0000 7fc4d67fc640 1 --2- 172.21.15.60:0/4263627292 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4c4066f90 0x7fc4c4069450 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0ae000 tx=0x7fc4cc001c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:19.835+0000 7fc4bf7fe640 1 -- 172.21.15.60:0/4263627292 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4c8019030 con 0x7fc4c0004810 2024-04-25T10:13:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.031+0000 7fc4dd6bb640 1 -- 172.21.15.60:0/4263627292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fc4d8151d40 con 0x7fc4c0004810 2024-04-25T10:13:20.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.195+0000 7fc4bf7fe640 1 -- 172.21.15.60:0/4263627292 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v64) v1 ==== 162+0+0 (secure 0 0 0) 0x7fc4c8055fd0 con 0x7fc4c0004810 2024-04-25T10:13:20.201 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:13:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.199+0000 7fc4dd6bb640 1 -- 172.21.15.60:0/4263627292 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4c4066f90 msgr2=0x7fc4c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.199+0000 7fc4dd6bb640 1 --2- 172.21.15.60:0/4263627292 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4c4066f90 0x7fc4c4069450 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0ae000 tx=0x7fc4cc001c90 comp rx=0 tx=0).stop 2024-04-25T10:13:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.199+0000 7fc4dd6bb640 1 -- 172.21.15.60:0/4263627292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c0004810 msgr2=0x7fc4d814dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.199+0000 7fc4dd6bb640 1 --2- 172.21.15.60:0/4263627292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c0004810 0x7fc4d814dba0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8011930 tx=0x7fc4c8011df0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.199+0000 7fc4dd6bb640 1 -- 172.21.15.60:0/4263627292 shutdown_connections 2024-04-25T10:13:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.199+0000 7fc4dd6bb640 1 --2- 172.21.15.60:0/4263627292 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc4c4066f90 0x7fc4c4069450 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.199+0000 7fc4dd6bb640 1 --2- 172.21.15.60:0/4263627292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d8151960 0x7fc4d80aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.199+0000 7fc4dd6bb640 1 --2- 172.21.15.60:0/4263627292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d815fc10 0x7fc4d8151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.199+0000 7fc4dd6bb640 1 --2- 172.21.15.60:0/4263627292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c0004810 0x7fc4d814dba0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.199+0000 7fc4dd6bb640 1 -- 172.21.15.60:0/4263627292 >> 172.21.15.60:0/4263627292 conn(0x7fc4d80670b0 msgr2=0x7fc4d81581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.199+0000 7fc4dd6bb640 1 -- 172.21.15.60:0/4263627292 shutdown_connections 2024-04-25T10:13:20.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.199+0000 7fc4dd6bb640 1 -- 172.21.15.60:0/4263627292 wait complete. 2024-04-25T10:13:20.230 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:20.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.343+0000 7f1f3274c640 1 Processor -- start 2024-04-25T10:13:20.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.343+0000 7f1f3274c640 1 -- start start 2024-04-25T10:13:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.343+0000 7f1f3274c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1f2c0b4660 0x7f1f2c15c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.343+0000 7f1f3274c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2c071ca0 con 0x7f1f2c0ce7a0 2024-04-25T10:13:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.343+0000 7f1f3274c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2c0cca40 con 0x7f1f2c15bc90 2024-04-25T10:13:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.343+0000 7f1f3274c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f2c0cc370 con 0x7f1f2c0b4660 2024-04-25T10:13:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.343+0000 7f1f30ce9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1f2c0b4660 0x7f1f2c15c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.343+0000 7f1f2bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1f2c0ce7a0 0x7f1f2c0ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56668/0 (socket says 172.21.15.60:56668) 2024-04-25T10:13:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.343+0000 7f1f30ce9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1f2c0b4660 0x7f1f2c15c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55034/0 (socket says 172.21.15.60:55034) 2024-04-25T10:13:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.343+0000 7f1f2bfff640 1 -- 172.21.15.60:0/3493343990 learned_addr learned my addr 172.21.15.60:0/3493343990 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:13:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.347+0000 7f1f30ce9640 1 -- 172.21.15.60:0/3493343990 >> v1:172.21.15.100:6789/0 conn(0x7f1f2c15bc90 legacy=0x7f1f2c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.347+0000 7f1f30ce9640 1 -- 172.21.15.60:0/3493343990 >> v1:172.21.15.60:6789/0 conn(0x7f1f2c0ce7a0 legacy=0x7f1f2c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.347+0000 7f1f30ce9640 1 -- 172.21.15.60:0/3493343990 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c0ccd20 con 0x7f1f2c0b4660 2024-04-25T10:13:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.347+0000 7f1f2affd640 1 -- 172.21.15.60:0/3493343990 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543936653 0 0) 0x7f1f2c071ca0 con 0x7f1f2c0ce7a0 2024-04-25T10:13:20.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.347+0000 7f1f30ce9640 1 --2- 172.21.15.60:0/3493343990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2c0b4660 0x7f1f2c15c930 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f1f240097c0 tx=0x7f1f2401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44462cb3a52b5b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.347+0000 7f1f2affd640 1 -- 172.21.15.60:0/3493343990 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f240b0070 con 0x7f1f2c0b4660 2024-04-25T10:13:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.347+0000 7f1f2affd640 1 -- 172.21.15.60:0/3493343990 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f240ab600 con 0x7f1f2c0b4660 2024-04-25T10:13:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.347+0000 7f1f2affd640 1 -- 172.21.15.60:0/3493343990 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f240ab9d0 con 0x7f1f2c0b4660 2024-04-25T10:13:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.347+0000 7f1f3274c640 1 -- 172.21.15.60:0/3493343990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2c0b4660 msgr2=0x7f1f2c15c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.347+0000 7f1f3274c640 1 --2- 172.21.15.60:0/3493343990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2c0b4660 0x7f1f2c15c930 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f1f240097c0 tx=0x7f1f2401c460 comp rx=0 tx=0).stop 2024-04-25T10:13:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.347+0000 7f1f3274c640 1 -- 172.21.15.60:0/3493343990 shutdown_connections 2024-04-25T10:13:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.347+0000 7f1f3274c640 1 --2- 172.21.15.60:0/3493343990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2c0b4660 0x7f1f2c15c930 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.347+0000 7f1f3274c640 1 -- 172.21.15.60:0/3493343990 >> 172.21.15.60:0/3493343990 conn(0x7f1f2c0b12d0 msgr2=0x7f1f2c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.347+0000 7f1f3274c640 1 -- 172.21.15.60:0/3493343990 shutdown_connections 2024-04-25T10:13:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.347+0000 7f1f3274c640 1 -- 172.21.15.60:0/3493343990 wait complete. 2024-04-25T10:13:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f3274c640 1 Processor -- start 2024-04-25T10:13:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f3274c640 1 -- start start 2024-04-25T10:13:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f3274c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f2c0ce7a0 0x7f1f2c151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f3274c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c15bc90 0x7f1f2c06dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f3274c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2c06e3e0 0x7f1f2c149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f3274c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1695d0 con 0x7f1f2c06e3e0 2024-04-25T10:13:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f3274c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c0b3db0 con 0x7f1f2c15bc90 2024-04-25T10:13:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f3274c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c169750 con 0x7f1f2c0ce7a0 2024-04-25T10:13:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f30ce9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2c06e3e0 0x7f1f2c149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f30ce9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2c06e3e0 0x7f1f2c149920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55046/0 (socket says 172.21.15.60:55046) 2024-04-25T10:13:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f30ce9640 1 -- 172.21.15.60:0/3844799723 learned_addr learned my addr 172.21.15.60:0/3844799723 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f2b7fe640 1 --2- 172.21.15.60:0/3844799723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c15bc90 0x7f1f2c06dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f30ce9640 1 -- 172.21.15.60:0/3844799723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f2c0ce7a0 msgr2=0x7f1f2c151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f30ce9640 1 --2- 172.21.15.60:0/3844799723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f2c0ce7a0 0x7f1f2c151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f30ce9640 1 -- 172.21.15.60:0/3844799723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c15bc90 msgr2=0x7f1f2c06dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f30ce9640 1 --2- 172.21.15.60:0/3844799723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c15bc90 0x7f1f2c06dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f30ce9640 1 -- 172.21.15.60:0/3844799723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c152050 con 0x7f1f2c06e3e0 2024-04-25T10:13:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f30ce9640 1 --2- 172.21.15.60:0/3844799723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2c06e3e0 0x7f1f2c149920 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f1f240ae040 tx=0x7f1f2c15d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f28ff9640 1 -- 172.21.15.60:0/3844799723 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f240b0040 con 0x7f1f2c06e3e0 2024-04-25T10:13:20.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f28ff9640 1 -- 172.21.15.60:0/3844799723 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f240ba070 con 0x7f1f2c06e3e0 2024-04-25T10:13:20.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f3274c640 1 -- 172.21.15.60:0/3844799723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2c152330 con 0x7f1f2c06e3e0 2024-04-25T10:13:20.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f28ff9640 1 -- 172.21.15.60:0/3844799723 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f240b4630 con 0x7f1f2c06e3e0 2024-04-25T10:13:20.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f3274c640 1 -- 172.21.15.60:0/3844799723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c1528f0 con 0x7f1f2c06e3e0 2024-04-25T10:13:20.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f3274c640 1 -- 172.21.15.60:0/3844799723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2c074770 con 0x7f1f2c06e3e0 2024-04-25T10:13:20.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f28ff9640 1 -- 172.21.15.60:0/3844799723 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f240b47d0 con 0x7f1f2c06e3e0 2024-04-25T10:13:20.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.351+0000 7f1f28ff9640 1 --2- 172.21.15.60:0/3844799723 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f18066f60 0x7f1f18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.355+0000 7f1f28ff9640 1 -- 172.21.15.60:0/3844799723 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f1f2411edf0 con 0x7f1f2c06e3e0 2024-04-25T10:13:20.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.355+0000 7f1f2bfff640 1 --2- 172.21.15.60:0/3844799723 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f18066f60 0x7f1f18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.355+0000 7f1f2bfff640 1 --2- 172.21.15.60:0/3844799723 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f18066f60 0x7f1f18069420 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c151790 tx=0x7f1f1c007cf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.359+0000 7f1f28ff9640 1 -- 172.21.15.60:0/3844799723 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f24016050 con 0x7f1f2c06e3e0 2024-04-25T10:13:20.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.551+0000 7f1f3274c640 1 -- 172.21.15.60:0/3844799723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f2c156fc0 con 0x7f1f2c06e3e0 2024-04-25T10:13:20.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.551+0000 7f1f28ff9640 1 -- 172.21.15.60:0/3844799723 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f1f240efd30 con 0x7f1f2c06e3e0 2024-04-25T10:13:20.557 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:13:20.557 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:13:20.202792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8256":{"gid":8256,"name":"c","rank":0,"incarnation":65,"state":"up:replay","state_seq":1,"addr":"172.21.15.60:6835/3807131024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3807131024},{"type":"v1","addr":"172.21.15.60:6835","nonce":3807131024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:13:20.558 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 65 2024-04-25T10:13:20.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.555+0000 7f1f3274c640 1 -- 172.21.15.60:0/3844799723 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f18066f60 msgr2=0x7f1f18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.555+0000 7f1f3274c640 1 --2- 172.21.15.60:0/3844799723 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f18066f60 0x7f1f18069420 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c151790 tx=0x7f1f1c007cf0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.555+0000 7f1f3274c640 1 -- 172.21.15.60:0/3844799723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2c06e3e0 msgr2=0x7f1f2c149920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.555+0000 7f1f3274c640 1 --2- 172.21.15.60:0/3844799723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2c06e3e0 0x7f1f2c149920 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f1f240ae040 tx=0x7f1f2c15d680 comp rx=0 tx=0).stop 2024-04-25T10:13:20.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.555+0000 7f1f3274c640 1 -- 172.21.15.60:0/3844799723 shutdown_connections 2024-04-25T10:13:20.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.555+0000 7f1f3274c640 1 --2- 172.21.15.60:0/3844799723 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f18066f60 0x7f1f18069420 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.555+0000 7f1f3274c640 1 --2- 172.21.15.60:0/3844799723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f2c06e3e0 0x7f1f2c149920 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.555+0000 7f1f3274c640 1 --2- 172.21.15.60:0/3844799723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c15bc90 0x7f1f2c06dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.555+0000 7f1f3274c640 1 --2- 172.21.15.60:0/3844799723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f2c0ce7a0 0x7f1f2c151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.555+0000 7f1f3274c640 1 -- 172.21.15.60:0/3844799723 >> 172.21.15.60:0/3844799723 conn(0x7f1f2c0b12d0 msgr2=0x7f1f2c1574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.555+0000 7f1f3274c640 1 -- 172.21.15.60:0/3844799723 shutdown_connections 2024-04-25T10:13:20.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:20.555+0000 7f1f3274c640 1 -- 172.21.15.60:0/3844799723 wait complete. 2024-04-25T10:13:20.979 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:44.357874+0000', 'modified': '2024-04-25T10:13:20.202792+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8256}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8256': {'gid': 8256, 'name': 'c', 'rank': 0, 'incarnation': 65, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.60:6835/3807131024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 3807131024}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 3807131024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:20.979 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8256:up:replay 2024-04-25T10:13:21.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:22.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.111+0000 7fef44654640 1 Processor -- start 2024-04-25T10:13:22.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.111+0000 7fef44654640 1 -- start start 2024-04-25T10:13:22.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.111+0000 7fef44654640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fef3c15bc90 0x7fef3c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.111+0000 7fef44654640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef3c0b4660 0x7fef3c15c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.111+0000 7fef44654640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3c071ca0 con 0x7fef3c0ce7a0 2024-04-25T10:13:22.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.111+0000 7fef44654640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef3c0cc370 con 0x7fef3c15bc90 2024-04-25T10:13:22.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.111+0000 7fef44654640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c0cca40 con 0x7fef3c0b4660 2024-04-25T10:13:22.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.111+0000 7fef41bef640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fef3c15bc90 0x7fef3c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.111+0000 7fef42bf1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef3c0b4660 0x7fef3c15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.111+0000 7fef42bf1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef3c0b4660 0x7fef3c15c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:53458/0 (socket says 172.21.15.60:53458) 2024-04-25T10:13:22.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.111+0000 7fef42bf1640 1 -- 172.21.15.60:0/1989325108 learned_addr learned my addr 172.21.15.60:0/1989325108 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:22.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.111+0000 7fef423f0640 1 --1- 172.21.15.60:0/1989325108 >> v1:172.21.15.100:6789/0 conn(0x7fef3c0ce7a0 0x7fef3c0ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:49378/0 (socket says 172.21.15.60:49378) 2024-04-25T10:13:22.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.111+0000 7fef42bf1640 1 -- 172.21.15.60:0/1989325108 >> v2:172.21.15.100:3300/0 conn(0x7fef3c15bc90 msgr2=0x7fef3c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.111+0000 7fef42bf1640 1 --2- 172.21.15.60:0/1989325108 >> v2:172.21.15.100:3300/0 conn(0x7fef3c15bc90 0x7fef3c15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.111+0000 7fef42bf1640 1 -- 172.21.15.60:0/1989325108 >> v1:172.21.15.100:6789/0 conn(0x7fef3c0ce7a0 legacy=0x7fef3c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.111+0000 7fef42bf1640 1 -- 172.21.15.60:0/1989325108 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c0ccd20 con 0x7fef3c0b4660 2024-04-25T10:13:22.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.115+0000 7fef42bf1640 1 --2- 172.21.15.60:0/1989325108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef3c0b4660 0x7fef3c15c760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef38009e50 tx=0x7fef3801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc6b4c6b5f2b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.115+0000 7fef413ee640 1 -- 172.21.15.60:0/1989325108 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef380a5700 con 0x7fef3c0b4660 2024-04-25T10:13:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.115+0000 7fef413ee640 1 -- 172.21.15.60:0/1989325108 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef380a58a0 con 0x7fef3c0b4660 2024-04-25T10:13:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.115+0000 7fef413ee640 1 -- 172.21.15.60:0/1989325108 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef380a5c70 con 0x7fef3c0b4660 2024-04-25T10:13:22.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.115+0000 7fef44654640 1 -- 172.21.15.60:0/1989325108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef3c0b4660 msgr2=0x7fef3c15c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.115+0000 7fef44654640 1 --2- 172.21.15.60:0/1989325108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef3c0b4660 0x7fef3c15c760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef38009e50 tx=0x7fef3801ca30 comp rx=0 tx=0).stop 2024-04-25T10:13:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.115+0000 7fef44654640 1 -- 172.21.15.60:0/1989325108 shutdown_connections 2024-04-25T10:13:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.115+0000 7fef44654640 1 --2- 172.21.15.60:0/1989325108 >> v2:172.21.15.100:3300/0 conn(0x7fef3c15bc90 0x7fef3c15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.115+0000 7fef44654640 1 --2- 172.21.15.60:0/1989325108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef3c0b4660 0x7fef3c15c760 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.115+0000 7fef44654640 1 -- 172.21.15.60:0/1989325108 >> 172.21.15.60:0/1989325108 conn(0x7fef3c0b12d0 msgr2=0x7fef3c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.115+0000 7fef44654640 1 -- 172.21.15.60:0/1989325108 shutdown_connections 2024-04-25T10:13:22.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.115+0000 7fef44654640 1 -- 172.21.15.60:0/1989325108 wait complete. 2024-04-25T10:13:22.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef44654640 1 Processor -- start 2024-04-25T10:13:22.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef44654640 1 -- start start 2024-04-25T10:13:22.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef44654640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef3c0ce7a0 0x7fef3c14adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef44654640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef3c15bc90 0x7fef3c15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef44654640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef3c14c9a0 0x7fef3c14bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef44654640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c1685c0 con 0x7fef3c15bc90 2024-04-25T10:13:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef44654640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c0b3db0 con 0x7fef3c14c9a0 2024-04-25T10:13:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef44654640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c168740 con 0x7fef3c0ce7a0 2024-04-25T10:13:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef41bef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef3c15bc90 0x7fef3c15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef41bef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef3c15bc90 0x7fef3c15b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55058/0 (socket says 172.21.15.60:55058) 2024-04-25T10:13:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef41bef640 1 -- 172.21.15.60:0/1963215180 learned_addr learned my addr 172.21.15.60:0/1963215180 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef42bf1640 1 --2- 172.21.15.60:0/1963215180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef3c14c9a0 0x7fef3c14bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef423f0640 1 --2- 172.21.15.60:0/1963215180 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef3c0ce7a0 0x7fef3c14adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef41bef640 1 -- 172.21.15.60:0/1963215180 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef3c0ce7a0 msgr2=0x7fef3c14adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef41bef640 1 --2- 172.21.15.60:0/1963215180 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef3c0ce7a0 0x7fef3c14adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef41bef640 1 -- 172.21.15.60:0/1963215180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef3c14c9a0 msgr2=0x7fef3c14bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef41bef640 1 --2- 172.21.15.60:0/1963215180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef3c14c9a0 0x7fef3c14bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef41bef640 1 -- 172.21.15.60:0/1963215180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c14ce30 con 0x7fef3c15bc90 2024-04-25T10:13:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef41bef640 1 --2- 172.21.15.60:0/1963215180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef3c15bc90 0x7fef3c15b3f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00e930 tx=0x7fef2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef32ffd640 1 -- 172.21.15.60:0/1963215180 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c017070 con 0x7fef3c15bc90 2024-04-25T10:13:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef32ffd640 1 -- 172.21.15.60:0/1963215180 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2c00ca00 con 0x7fef3c15bc90 2024-04-25T10:13:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef44654640 1 -- 172.21.15.60:0/1963215180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3c14c100 con 0x7fef3c15bc90 2024-04-25T10:13:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef32ffd640 1 -- 172.21.15.60:0/1963215180 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c00cdb0 con 0x7fef3c15bc90 2024-04-25T10:13:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef44654640 1 -- 172.21.15.60:0/1963215180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3c14d0a0 con 0x7fef3c15bc90 2024-04-25T10:13:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef44654640 1 -- 172.21.15.60:0/1963215180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef3c074770 con 0x7fef3c15bc90 2024-04-25T10:13:22.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.119+0000 7fef32ffd640 1 -- 172.21.15.60:0/1963215180 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef2c01a070 con 0x7fef3c15bc90 2024-04-25T10:13:22.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.123+0000 7fef32ffd640 1 --2- 172.21.15.60:0/1963215180 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fef20066f90 0x7fef20069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.123+0000 7fef32ffd640 1 -- 172.21.15.60:0/1963215180 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fef2c085110 con 0x7fef3c15bc90 2024-04-25T10:13:22.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.123+0000 7fef423f0640 1 --2- 172.21.15.60:0/1963215180 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fef20066f90 0x7fef20069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.123+0000 7fef423f0640 1 --2- 172.21.15.60:0/1963215180 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fef20066f90 0x7fef20069450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fef34000f70 tx=0x7fef34007cf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.127+0000 7fef32ffd640 1 -- 172.21.15.60:0/1963215180 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef2c056120 con 0x7fef3c15bc90 2024-04-25T10:13:22.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.323+0000 7fef44654640 1 -- 172.21.15.60:0/1963215180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef3c0b0890 con 0x7fef3c15bc90 2024-04-25T10:13:22.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.323+0000 7fef32ffd640 1 -- 172.21.15.60:0/1963215180 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fef2c055a20 con 0x7fef3c15bc90 2024-04-25T10:13:22.329 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:13:22.329 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:13:21.218349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8256":{"gid":8256,"name":"c","rank":0,"incarnation":65,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.60:6835/3807131024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3807131024},{"type":"v1","addr":"172.21.15.60:6835","nonce":3807131024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:13:22.330 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 67 2024-04-25T10:13:22.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.327+0000 7fef44654640 1 -- 172.21.15.60:0/1963215180 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fef20066f90 msgr2=0x7fef20069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.327+0000 7fef44654640 1 --2- 172.21.15.60:0/1963215180 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fef20066f90 0x7fef20069450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fef34000f70 tx=0x7fef34007cf0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.327+0000 7fef44654640 1 -- 172.21.15.60:0/1963215180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef3c15bc90 msgr2=0x7fef3c15b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.327+0000 7fef44654640 1 --2- 172.21.15.60:0/1963215180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef3c15bc90 0x7fef3c15b3f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00e930 tx=0x7fef2c00edf0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.327+0000 7fef44654640 1 -- 172.21.15.60:0/1963215180 shutdown_connections 2024-04-25T10:13:22.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.327+0000 7fef44654640 1 --2- 172.21.15.60:0/1963215180 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fef20066f90 0x7fef20069450 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.327+0000 7fef44654640 1 --2- 172.21.15.60:0/1963215180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef3c14c9a0 0x7fef3c14bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.327+0000 7fef44654640 1 --2- 172.21.15.60:0/1963215180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef3c15bc90 0x7fef3c15b3f0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.327+0000 7fef44654640 1 --2- 172.21.15.60:0/1963215180 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef3c0ce7a0 0x7fef3c14adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.327+0000 7fef44654640 1 -- 172.21.15.60:0/1963215180 >> 172.21.15.60:0/1963215180 conn(0x7fef3c0b12d0 msgr2=0x7fef3c0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.327+0000 7fef44654640 1 -- 172.21.15.60:0/1963215180 shutdown_connections 2024-04-25T10:13:22.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:22.327+0000 7fef44654640 1 -- 172.21.15.60:0/1963215180 wait complete. 2024-04-25T10:13:22.752 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:44.357874+0000', 'modified': '2024-04-25T10:13:21.218349+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8256}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8256': {'gid': 8256, 'name': 'c', 'rank': 0, 'incarnation': 65, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.60:6835/3807131024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 3807131024}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 3807131024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:22.752 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8256:up:rejoin 2024-04-25T10:13:23.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:23.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32f164c640 1 Processor -- start 2024-04-25T10:13:23.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32f164c640 1 -- start start 2024-04-25T10:13:23.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32f164c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f32ec162250 0x7f32ec0b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32f164c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32ec0b34f0 0x7f32ec071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32f164c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f32ec0717d0 0x7f32ec06d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32f164c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ec0af0c0 con 0x7f32ec162250 2024-04-25T10:13:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32f164c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ec071c40 con 0x7f32ec0717d0 2024-04-25T10:13:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32f164c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec0b3aa0 con 0x7f32ec0b34f0 2024-04-25T10:13:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32eaffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f32ec162250 0x7f32ec0b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32eaffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f32ec162250 0x7f32ec0b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55064/0 (socket says 172.21.15.60:55064) 2024-04-25T10:13:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32eaffd640 1 -- 172.21.15.60:0/1983596833 learned_addr learned my addr 172.21.15.60:0/1983596833 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32ea7fc640 1 --2- 172.21.15.60:0/1983596833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32ec0b34f0 0x7f32ec071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32eb7fe640 1 --2- 172.21.15.60:0/1983596833 >> v2:172.21.15.100:3300/0 conn(0x7f32ec0717d0 0x7f32ec06d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32eaffd640 1 -- 172.21.15.60:0/1983596833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32ec0b34f0 msgr2=0x7f32ec071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32eaffd640 1 --2- 172.21.15.60:0/1983596833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32ec0b34f0 0x7f32ec071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32eaffd640 1 -- 172.21.15.60:0/1983596833 >> v2:172.21.15.100:3300/0 conn(0x7f32ec0717d0 msgr2=0x7f32ec06d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:23.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32eaffd640 1 --2- 172.21.15.60:0/1983596833 >> v2:172.21.15.100:3300/0 conn(0x7f32ec0717d0 0x7f32ec06d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32eaffd640 1 -- 172.21.15.60:0/1983596833 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec0b3d80 con 0x7f32ec162250 2024-04-25T10:13:23.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.883+0000 7f32eb7fe640 1 --2- 172.21.15.60:0/1983596833 >> v2:172.21.15.100:3300/0 conn(0x7f32ec0717d0 0x7f32ec06d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:23.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.887+0000 7f32eaffd640 1 --2- 172.21.15.60:0/1983596833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32ec162250 0x7f32ec0b2fb0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0cf4b0 tx=0x7f32dc01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd560508df7472b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:23.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.887+0000 7f32e9ffb640 1 -- 172.21.15.60:0/1983596833 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0b5070 con 0x7f32ec162250 2024-04-25T10:13:23.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.887+0000 7f32e9ffb640 1 -- 172.21.15.60:0/1983596833 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc0a89d0 con 0x7f32ec162250 2024-04-25T10:13:23.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.887+0000 7f32e9ffb640 1 -- 172.21.15.60:0/1983596833 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0a8da0 con 0x7f32ec162250 2024-04-25T10:13:23.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.887+0000 7f32f164c640 1 -- 172.21.15.60:0/1983596833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32ec162250 msgr2=0x7f32ec0b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.887+0000 7f32f164c640 1 --2- 172.21.15.60:0/1983596833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32ec162250 0x7f32ec0b2fb0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0cf4b0 tx=0x7f32dc01ed10 comp rx=0 tx=0).stop 2024-04-25T10:13:23.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.887+0000 7f32f164c640 1 -- 172.21.15.60:0/1983596833 shutdown_connections 2024-04-25T10:13:23.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.887+0000 7f32f164c640 1 --2- 172.21.15.60:0/1983596833 >> v2:172.21.15.100:3300/0 conn(0x7f32ec0717d0 0x7f32ec06d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.887+0000 7f32f164c640 1 --2- 172.21.15.60:0/1983596833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32ec0b34f0 0x7f32ec071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.887+0000 7f32f164c640 1 --2- 172.21.15.60:0/1983596833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32ec162250 0x7f32ec0b2fb0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.887+0000 7f32f164c640 1 -- 172.21.15.60:0/1983596833 >> 172.21.15.60:0/1983596833 conn(0x7f32ec0b12d0 msgr2=0x7f32ec0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:23.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.887+0000 7f32f164c640 1 -- 172.21.15.60:0/1983596833 shutdown_connections 2024-04-25T10:13:23.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.887+0000 7f32f164c640 1 -- 172.21.15.60:0/1983596833 wait complete. 2024-04-25T10:13:23.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.887+0000 7f32f164c640 1 Processor -- start 2024-04-25T10:13:23.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.887+0000 7f32f164c640 1 -- start start 2024-04-25T10:13:23.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32f164c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32ec0717d0 0x7f32ec15d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32f164c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32ec0b34f0 0x7f32ec1f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32f164c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32ec162250 0x7f32ec15a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32f164c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec06e5a0 con 0x7f32ec0717d0 2024-04-25T10:13:23.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32f164c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec0b4340 con 0x7f32ec0b34f0 2024-04-25T10:13:23.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32f164c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec0cc440 con 0x7f32ec162250 2024-04-25T10:13:23.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32eaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32ec0717d0 0x7f32ec15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32eaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32ec0717d0 0x7f32ec15d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55074/0 (socket says 172.21.15.60:55074) 2024-04-25T10:13:23.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32eaffd640 1 -- 172.21.15.60:0/1273032340 learned_addr learned my addr 172.21.15.60:0/1273032340 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:23.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32eb7fe640 1 --2- 172.21.15.60:0/1273032340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32ec162250 0x7f32ec15a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32ea7fc640 1 --2- 172.21.15.60:0/1273032340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32ec0b34f0 0x7f32ec1f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32eaffd640 1 -- 172.21.15.60:0/1273032340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32ec162250 msgr2=0x7f32ec15a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32eaffd640 1 --2- 172.21.15.60:0/1273032340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32ec162250 0x7f32ec15a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32eaffd640 1 -- 172.21.15.60:0/1273032340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32ec0b34f0 msgr2=0x7f32ec1f7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32eaffd640 1 --2- 172.21.15.60:0/1273032340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32ec0b34f0 0x7f32ec1f7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32eaffd640 1 -- 172.21.15.60:0/1273032340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec1f77c0 con 0x7f32ec0717d0 2024-04-25T10:13:23.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32ea7fc640 1 --2- 172.21.15.60:0/1273032340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32ec0b34f0 0x7f32ec1f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:23.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32eaffd640 1 --2- 172.21.15.60:0/1273032340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32ec0717d0 0x7f32ec15d360 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0b3040 tx=0x7f32dc01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:23.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32d37fe640 1 -- 172.21.15.60:0/1273032340 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0b5040 con 0x7f32ec0717d0 2024-04-25T10:13:23.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32d37fe640 1 -- 172.21.15.60:0/1273032340 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc0c0070 con 0x7f32ec0717d0 2024-04-25T10:13:23.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32f164c640 1 -- 172.21.15.60:0/1273032340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ec0adf00 con 0x7f32ec0717d0 2024-04-25T10:13:23.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32d37fe640 1 -- 172.21.15.60:0/1273032340 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0b08e0 con 0x7f32ec0717d0 2024-04-25T10:13:23.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32f164c640 1 -- 172.21.15.60:0/1273032340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ec0ae4c0 con 0x7f32ec0717d0 2024-04-25T10:13:23.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32f164c640 1 -- 172.21.15.60:0/1273032340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ec074770 con 0x7f32ec0717d0 2024-04-25T10:13:23.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32d37fe640 1 -- 172.21.15.60:0/1273032340 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32dc0b0a80 con 0x7f32ec0717d0 2024-04-25T10:13:23.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32d37fe640 1 --2- 172.21.15.60:0/1273032340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f32d8066f60 0x7f32d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32d37fe640 1 -- 172.21.15.60:0/1273032340 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f32dc122570 con 0x7f32ec0717d0 2024-04-25T10:13:23.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.891+0000 7f32ea7fc640 1 --2- 172.21.15.60:0/1273032340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f32d8066f60 0x7f32d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.895+0000 7f32ea7fc640 1 --2- 172.21.15.60:0/1273032340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f32d8066f60 0x7f32d8069420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f32ec161ef0 tx=0x7f32e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:23.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:23.895+0000 7f32d37fe640 1 -- 172.21.15.60:0/1273032340 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f32dc018870 con 0x7f32ec0717d0 2024-04-25T10:13:24.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.091+0000 7f32f164c640 1 -- 172.21.15.60:0/1273032340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32ec06c250 con 0x7f32ec0717d0 2024-04-25T10:13:24.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.091+0000 7f32d37fe640 1 -- 172.21.15.60:0/1273032340 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f32dc0f3420 con 0x7f32ec0717d0 2024-04-25T10:13:24.096 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:13:24.096 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:13:22.221010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8256":{"gid":8256,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/3807131024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3807131024},{"type":"v1","addr":"172.21.15.60:6835","nonce":3807131024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:13:24.096 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 68 2024-04-25T10:13:24.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.091+0000 7f32f164c640 1 -- 172.21.15.60:0/1273032340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f32d8066f60 msgr2=0x7f32d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.091+0000 7f32f164c640 1 --2- 172.21.15.60:0/1273032340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f32d8066f60 0x7f32d8069420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f32ec161ef0 tx=0x7f32e0008040 comp rx=0 tx=0).stop 2024-04-25T10:13:24.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.091+0000 7f32f164c640 1 -- 172.21.15.60:0/1273032340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32ec0717d0 msgr2=0x7f32ec15d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.091+0000 7f32f164c640 1 --2- 172.21.15.60:0/1273032340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32ec0717d0 0x7f32ec15d360 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0b3040 tx=0x7f32dc01bab0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.091+0000 7f32f164c640 1 -- 172.21.15.60:0/1273032340 shutdown_connections 2024-04-25T10:13:24.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.091+0000 7f32f164c640 1 --2- 172.21.15.60:0/1273032340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f32d8066f60 0x7f32d8069420 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.091+0000 7f32f164c640 1 --2- 172.21.15.60:0/1273032340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32ec162250 0x7f32ec15a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.091+0000 7f32f164c640 1 --2- 172.21.15.60:0/1273032340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32ec0b34f0 0x7f32ec1f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.091+0000 7f32f164c640 1 --2- 172.21.15.60:0/1273032340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32ec0717d0 0x7f32ec15d360 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.091+0000 7f32f164c640 1 -- 172.21.15.60:0/1273032340 >> 172.21.15.60:0/1273032340 conn(0x7f32ec0b12d0 msgr2=0x7f32ec0af240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.091+0000 7f32f164c640 1 -- 172.21.15.60:0/1273032340 shutdown_connections 2024-04-25T10:13:24.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.091+0000 7f32f164c640 1 -- 172.21.15.60:0/1273032340 wait complete. 2024-04-25T10:13:24.517 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:44.357874+0000', 'modified': '2024-04-25T10:13:22.221010+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8256}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8256': {'gid': 8256, 'name': 'c', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.60:6835/3807131024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 3807131024}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 3807131024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:24.517 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:13:24.517 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:13:24.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.651+0000 7f7236187640 1 Processor -- start 2024-04-25T10:13:24.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.651+0000 7f7236187640 1 -- start start 2024-04-25T10:13:24.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.651+0000 7f7236187640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7230071b20 0x7f7230071f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.651+0000 7f7236187640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72300cce50 0x7f72300cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.651+0000 7f7236187640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7230072440 con 0x7f723006e480 2024-04-25T10:13:24.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.651+0000 7f7236187640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723015dc10 con 0x7f7230071b20 2024-04-25T10:13:24.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.651+0000 7f7236187640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723015def0 con 0x7f72300cce50 2024-04-25T10:13:24.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.651+0000 7f722f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7230071b20 0x7f7230071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.651+0000 7f722f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7230071b20 0x7f7230071f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55088/0 (socket says 172.21.15.60:55088) 2024-04-25T10:13:24.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.651+0000 7f722f7fe640 1 -- 172.21.15.60:0/1564994833 learned_addr learned my addr 172.21.15.60:0/1564994833 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:24.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.651+0000 7f722ffff640 1 --2- 172.21.15.60:0/1564994833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72300cce50 0x7f72300cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.651+0000 7f722f7fe640 1 -- 172.21.15.60:0/1564994833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72300cce50 msgr2=0x7f72300cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.651+0000 7f722f7fe640 1 --2- 172.21.15.60:0/1564994833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72300cce50 0x7f72300cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.651+0000 7f722f7fe640 1 -- 172.21.15.60:0/1564994833 >> v1:172.21.15.100:6789/0 conn(0x7f723006e480 legacy=0x7f723006e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.651+0000 7f722f7fe640 1 -- 172.21.15.60:0/1564994833 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723015e1d0 con 0x7f7230071b20 2024-04-25T10:13:24.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f722f7fe640 1 --2- 172.21.15.60:0/1564994833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7230071b20 0x7f7230071f00 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f722000bb10 tx=0x7f722001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeae5258b28ade0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f722e7fc640 1 -- 172.21.15.60:0/1564994833 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72200b4070 con 0x7f7230071b20 2024-04-25T10:13:24.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f722e7fc640 1 -- 172.21.15.60:0/1564994833 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72200a7970 con 0x7f7230071b20 2024-04-25T10:13:24.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f722e7fc640 1 -- 172.21.15.60:0/1564994833 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72200a7d40 con 0x7f7230071b20 2024-04-25T10:13:24.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f7236187640 1 -- 172.21.15.60:0/1564994833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7230071b20 msgr2=0x7f7230071f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f7236187640 1 --2- 172.21.15.60:0/1564994833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7230071b20 0x7f7230071f00 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f722000bb10 tx=0x7f722001eb60 comp rx=0 tx=0).stop 2024-04-25T10:13:24.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f7236187640 1 -- 172.21.15.60:0/1564994833 shutdown_connections 2024-04-25T10:13:24.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f7236187640 1 --2- 172.21.15.60:0/1564994833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72300cce50 0x7f72300cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f7236187640 1 --2- 172.21.15.60:0/1564994833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7230071b20 0x7f7230071f00 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f7236187640 1 -- 172.21.15.60:0/1564994833 >> 172.21.15.60:0/1564994833 conn(0x7f72300b12d0 msgr2=0x7f72300b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f7236187640 1 -- 172.21.15.60:0/1564994833 shutdown_connections 2024-04-25T10:13:24.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f7236187640 1 -- 172.21.15.60:0/1564994833 wait complete. 2024-04-25T10:13:24.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f7236187640 1 Processor -- start 2024-04-25T10:13:24.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f7236187640 1 -- start start 2024-04-25T10:13:24.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f7236187640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f723006e480 0x7f723015bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f7236187640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7230071b20 0x7f723015cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f7236187640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72300cce50 0x7f7230158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f7236187640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72300cd940 con 0x7f72300cce50 2024-04-25T10:13:24.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f7236187640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72300cd7c0 con 0x7f723006e480 2024-04-25T10:13:24.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.655+0000 7f7236187640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72300cdac0 con 0x7f7230071b20 2024-04-25T10:13:24.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f722ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72300cce50 0x7f7230158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f722ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72300cce50 0x7f7230158480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55098/0 (socket says 172.21.15.60:55098) 2024-04-25T10:13:24.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f722ffff640 1 -- 172.21.15.60:0/3722211864 learned_addr learned my addr 172.21.15.60:0/3722211864 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:24.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f722effd640 1 --2- 172.21.15.60:0/3722211864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7230071b20 0x7f723015cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f722f7fe640 1 --2- 172.21.15.60:0/3722211864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f723006e480 0x7f723015bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f722ffff640 1 -- 172.21.15.60:0/3722211864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7230071b20 msgr2=0x7f723015cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f722ffff640 1 --2- 172.21.15.60:0/3722211864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7230071b20 0x7f723015cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f722ffff640 1 -- 172.21.15.60:0/3722211864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f723006e480 msgr2=0x7f723015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f722ffff640 1 --2- 172.21.15.60:0/3722211864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f723006e480 0x7f723015bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f722ffff640 1 -- 172.21.15.60:0/3722211864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72301589c0 con 0x7f72300cce50 2024-04-25T10:13:24.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f722effd640 1 --2- 172.21.15.60:0/3722211864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7230071b20 0x7f723015cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:24.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f722f7fe640 1 --2- 172.21.15.60:0/3722211864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f723006e480 0x7f723015bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:24.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f722ffff640 1 --2- 172.21.15.60:0/3722211864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72300cce50 0x7f7230158480 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f7228007e60 tx=0x7f722800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f7217fff640 1 -- 172.21.15.60:0/3722211864 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7228014420 con 0x7f72300cce50 2024-04-25T10:13:24.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f7217fff640 1 -- 172.21.15.60:0/3722211864 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7228019070 con 0x7f72300cce50 2024-04-25T10:13:24.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f7236187640 1 -- 172.21.15.60:0/3722211864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72301576c0 con 0x7f72300cce50 2024-04-25T10:13:24.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f7217fff640 1 -- 172.21.15.60:0/3722211864 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7228014820 con 0x7f72300cce50 2024-04-25T10:13:24.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f7236187640 1 -- 172.21.15.60:0/3722211864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7230157c80 con 0x7f72300cce50 2024-04-25T10:13:24.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f7217fff640 1 -- 172.21.15.60:0/3722211864 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72280149c0 con 0x7f72300cce50 2024-04-25T10:13:24.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f7217fff640 1 --2- 172.21.15.60:0/3722211864 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f721c066f60 0x7f721c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f7217fff640 1 -- 172.21.15.60:0/3722211864 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f7228085910 con 0x7f72300cce50 2024-04-25T10:13:24.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f722f7fe640 1 --2- 172.21.15.60:0/3722211864 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f721c066f60 0x7f721c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f72392f4000 1 Processor -- start 2024-04-25T10:13:24.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.659+0000 7f72392f4000 1 -- start start 2024-04-25T10:13:24.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722f7fe640 1 --2- 172.21.15.60:0/3722211864 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f721c066f60 0x7f721c069420 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f72200b1000 tx=0x7f7220008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2ee920 0x5569ff2eed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ef240 0x5569ff2a49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff2a5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ff18c5f0 con 0x5569ff2a4f30 2024-04-25T10:13:24.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ff0f3da0 con 0x5569ff2ee920 2024-04-25T10:13:24.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569ff07c150 con 0x5569ff2ef240 2024-04-25T10:13:24.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff2a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff2a5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55110/0 (socket says 172.21.15.60:55110) 2024-04-25T10:13:24.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722ffff640 1 -- 172.21.15.60:0/3421309414 learned_addr learned my addr 172.21.15.60:0/3421309414 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:24.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722effd640 1 --2- 172.21.15.60:0/3421309414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ef240 0x5569ff2a49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722f7fe640 1 --2- 172.21.15.60:0/3421309414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2ee920 0x5569ff2eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722ffff640 1 -- 172.21.15.60:0/3421309414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ef240 msgr2=0x5569ff2a49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722ffff640 1 --2- 172.21.15.60:0/3421309414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ef240 0x5569ff2a49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722ffff640 1 -- 172.21.15.60:0/3421309414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2ee920 msgr2=0x5569ff2eed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722ffff640 1 --2- 172.21.15.60:0/3421309414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2ee920 0x5569ff2eed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722ffff640 1 -- 172.21.15.60:0/3421309414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569ff095ca0 con 0x5569ff2a4f30 2024-04-25T10:13:24.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722ffff640 1 --2- 172.21.15.60:0/3421309414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff2a5310 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f722805e3c0 tx=0x7f7228055c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9265c0dd631d7a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72157fa640 1 -- 172.21.15.60:0/3421309414 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722805e8f0 con 0x5569ff2a4f30 2024-04-25T10:13:24.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72157fa640 1 -- 172.21.15.60:0/3421309414 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72280544e0 con 0x5569ff2a4f30 2024-04-25T10:13:24.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72157fa640 1 -- 172.21.15.60:0/3421309414 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7228052850 con 0x5569ff2a4f30 2024-04-25T10:13:24.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 -- 172.21.15.60:0/3421309414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 msgr2=0x5569ff2a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 --2- 172.21.15.60:0/3421309414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff2a5310 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f722805e3c0 tx=0x7f7228055c70 comp rx=0 tx=0).stop 2024-04-25T10:13:24.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 -- 172.21.15.60:0/3421309414 shutdown_connections 2024-04-25T10:13:24.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 --2- 172.21.15.60:0/3421309414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff2a5310 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 --2- 172.21.15.60:0/3421309414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ef240 0x5569ff2a49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 --2- 172.21.15.60:0/3421309414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2ee920 0x5569ff2eed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 -- 172.21.15.60:0/3421309414 >> 172.21.15.60:0/3421309414 conn(0x5569ff2ab2a0 msgr2=0x5569ff2a32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 -- 172.21.15.60:0/3421309414 shutdown_connections 2024-04-25T10:13:24.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 -- 172.21.15.60:0/3421309414 wait complete. 2024-04-25T10:13:24.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 -- start start 2024-04-25T10:13:24.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 Processor -- start 2024-04-25T10:13:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2a4f30 0x5569ff2f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2ee920 0x5569ff2f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ef240 0x5569ff2a8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ff0ae990 con 0x5569ff2ee920 2024-04-25T10:13:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ff0efe40 con 0x5569ff2a4f30 2024-04-25T10:13:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569ff07f900 con 0x5569ff2ef240 2024-04-25T10:13:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f72392f4000 10 client.0 _ll_register_callbacks cb 0x5569ff1a71c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2ee920 0x5569ff2f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2ee920 0x5569ff2f61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55116/0 (socket says 172.21.15.60:55116) 2024-04-25T10:13:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722effd640 1 -- 172.21.15.60:0/1632711399 learned_addr learned my addr 172.21.15.60:0/1632711399 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722f7fe640 1 --2- 172.21.15.60:0/1632711399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2a4f30 0x5569ff2f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722ffff640 1 --2- 172.21.15.60:0/1632711399 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ef240 0x5569ff2a8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722effd640 1 -- 172.21.15.60:0/1632711399 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ef240 msgr2=0x5569ff2a8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722effd640 1 --2- 172.21.15.60:0/1632711399 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ef240 0x5569ff2a8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722effd640 1 -- 172.21.15.60:0/1632711399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2a4f30 msgr2=0x5569ff2f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722effd640 1 --2- 172.21.15.60:0/1632711399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2a4f30 0x5569ff2f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722effd640 1 -- 172.21.15.60:0/1632711399 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569ff0ac630 con 0x5569ff2ee920 2024-04-25T10:13:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722ffff640 1 --2- 172.21.15.60:0/1632711399 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ef240 0x5569ff2a8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.663+0000 7f722f7fe640 1 --2- 172.21.15.60:0/1632711399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2a4f30 0x5569ff2f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f722effd640 1 --2- 172.21.15.60:0/1632711399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2ee920 0x5569ff2f61c0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f7224007cc0 tx=0x7f722400f6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e0fcfa08807c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f720e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:13:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f720e7fc640 1 -- 172.21.15.60:0/1632711399 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7224018020 con 0x5569ff2ee920 2024-04-25T10:13:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f720e7fc640 1 -- 172.21.15.60:0/1632711399 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f7224014660 con 0x5569ff2ee920 2024-04-25T10:13:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 1 -- 172.21.15.60:0/1632711399 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72280544e0 con 0x5569ff2ee920 2024-04-25T10:13:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f720e7fc640 1 -- 172.21.15.60:0/1632711399 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7224022050 con 0x5569ff2ee920 2024-04-25T10:13:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 10 client.8313 fetch_fsmap learned FSMap version 68 2024-04-25T10:13:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 1 -- 172.21.15.60:0/1632711399 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x5569ff07c3a0 con 0x5569ff2ee920 2024-04-25T10:13:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f720e7fc640 1 -- 172.21.15.60:0/1632711399 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72280544e0 con 0x5569ff2ee920 2024-04-25T10:13:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f720e7fc640 1 -- 172.21.15.60:0/1632711399 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7224025020 con 0x5569ff2ee920 2024-04-25T10:13:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 10 client.8313 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:13:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 10 client.8313 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 20 client.8313 populate_metadata read hostname 'smithi060' 2024-04-25T10:13:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 1 --2- 172.21.15.60:0/1632711399 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x5569ff2aa160 0x5569ff2ffce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 4 client.8313 mds_command: new command op to 8256 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 1 -- 172.21.15.60:0/1632711399 --> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569ff08fa50 con 0x5569ff2aa160 2024-04-25T10:13:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f722f7fe640 1 --2- 172.21.15.60:0/1632711399 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x5569ff2aa160 0x5569ff2ffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f720e7fc640 10 client.8313 ms_handle_connect on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f722f7fe640 1 --2- 172.21.15.60:0/1632711399 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x5569ff2aa160 0x5569ff2ffce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a636fdfa3027e5 server_cookie=3a2b0286d2ba8efd in_seq=0 out_seq=0 2024-04-25T10:13:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f720e7fc640 1 -- 172.21.15.60:0/1632711399 <== mds.0 v2:172.21.15.60:6833/3807131024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5569ff08fa50 con 0x5569ff2aa160 2024-04-25T10:13:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f720e7fc640 10 client.8313 handle_command_reply: tid=0 2024-04-25T10:13:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 1 client.8313 shutdown 2024-04-25T10:13:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 1 -- 172.21.15.60:0/1632711399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2ee920 msgr2=0x5569ff2f61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 1 --2- 172.21.15.60:0/1632711399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2ee920 0x5569ff2f61c0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f7224007cc0 tx=0x7f722400f6e0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 1 -- 172.21.15.60:0/1632711399 shutdown_connections 2024-04-25T10:13:24.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 1 --2- 172.21.15.60:0/1632711399 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x5569ff2aa160 0x5569ff2ffce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 1 --2- 172.21.15.60:0/1632711399 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ef240 0x5569ff2a8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f720e7fc640 0 client.8313 ms_handle_reset on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:24.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 1 --2- 172.21.15.60:0/1632711399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2ee920 0x5569ff2f61c0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 1 --2- 172.21.15.60:0/1632711399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2a4f30 0x5569ff2f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 1 -- 172.21.15.60:0/1632711399 >> 172.21.15.60:0/1632711399 conn(0x5569ff2ab2a0 msgr2=0x5569ff2a39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 1 -- 172.21.15.60:0/1632711399 shutdown_connections 2024-04-25T10:13:24.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 1 -- 172.21.15.60:0/1632711399 wait complete. 2024-04-25T10:13:24.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.667+0000 7f72392f4000 20 client.8313 trim_cache size 0 max 16384 2024-04-25T10:13:24.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f72392f4000 1 Processor -- start 2024-04-25T10:13:24.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f72392f4000 1 -- start start 2024-04-25T10:13:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f72392f4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2ab2a0 0x5569ff34a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f72392f4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff34a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f72392f4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ee920 0x5569ff34ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f72392f4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72301fae30 con 0x5569ff2a4f30 2024-04-25T10:13:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f72392f4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72301fa230 con 0x5569ff2ab2a0 2024-04-25T10:13:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f72392f4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72301fa830 con 0x5569ff2ee920 2024-04-25T10:13:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f722effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff34a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f722effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff34a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55128/0 (socket says 172.21.15.60:55128) 2024-04-25T10:13:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f722effd640 1 -- 172.21.15.60:0/2537655248 learned_addr learned my addr 172.21.15.60:0/2537655248 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f722ffff640 1 --2- 172.21.15.60:0/2537655248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ee920 0x5569ff34ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f722effd640 1 -- 172.21.15.60:0/2537655248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ee920 msgr2=0x5569ff34ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f722effd640 1 --2- 172.21.15.60:0/2537655248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ee920 0x5569ff34ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f722effd640 1 -- 172.21.15.60:0/2537655248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2ab2a0 msgr2=0x5569ff34a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f722effd640 1 --2- 172.21.15.60:0/2537655248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2ab2a0 0x5569ff34a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f722effd640 1 -- 172.21.15.60:0/2537655248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569ff2ef070 con 0x5569ff2a4f30 2024-04-25T10:13:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f722effd640 1 --2- 172.21.15.60:0/2537655248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff34a8e0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f7224000c10 tx=0x7f722400b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c86f5646ecefb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f7214ff9640 1 -- 172.21.15.60:0/2537655248 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7224018070 con 0x5569ff2a4f30 2024-04-25T10:13:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f7214ff9640 1 -- 172.21.15.60:0/2537655248 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7224000e50 con 0x5569ff2a4f30 2024-04-25T10:13:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.683+0000 7f7214ff9640 1 -- 172.21.15.60:0/2537655248 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7224014cf0 con 0x5569ff2a4f30 2024-04-25T10:13:24.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 -- 172.21.15.60:0/2537655248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 msgr2=0x5569ff34a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 --2- 172.21.15.60:0/2537655248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff34a8e0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f7224000c10 tx=0x7f722400b000 comp rx=0 tx=0).stop 2024-04-25T10:13:24.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 -- 172.21.15.60:0/2537655248 shutdown_connections 2024-04-25T10:13:24.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 --2- 172.21.15.60:0/2537655248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ee920 0x5569ff34ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 --2- 172.21.15.60:0/2537655248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff34a8e0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 --2- 172.21.15.60:0/2537655248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2ab2a0 0x5569ff34a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 -- 172.21.15.60:0/2537655248 >> 172.21.15.60:0/2537655248 conn(0x5569ff2a8510 msgr2=0x5569ff2ed200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 -- 172.21.15.60:0/2537655248 shutdown_connections 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 -- 172.21.15.60:0/2537655248 wait complete. 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 -- start start 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 Processor -- start 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff332de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2ab2a0 0x5569ff333730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ee920 0x5569ff333c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ff0f83d0 con 0x5569ff2a4f30 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569ff1afd50 con 0x5569ff2ab2a0 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569ff08e980 con 0x5569ff2ee920 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 10 client.0 _ll_register_callbacks cb 0x7f7228055fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f722f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff332de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f722f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff332de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55134/0 (socket says 172.21.15.60:55134) 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f722f7fe640 1 -- 172.21.15.60:0/4285775412 learned_addr learned my addr 172.21.15.60:0/4285775412 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f722ffff640 1 --2- 172.21.15.60:0/4285775412 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ee920 0x5569ff333c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f722effd640 1 --2- 172.21.15.60:0/4285775412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2ab2a0 0x5569ff333730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f722f7fe640 1 -- 172.21.15.60:0/4285775412 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ee920 msgr2=0x5569ff333c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f722f7fe640 1 --2- 172.21.15.60:0/4285775412 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ee920 0x5569ff333c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f722f7fe640 1 -- 172.21.15.60:0/4285775412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2ab2a0 msgr2=0x5569ff333730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f722f7fe640 1 --2- 172.21.15.60:0/4285775412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2ab2a0 0x5569ff333730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f722f7fe640 1 -- 172.21.15.60:0/4285775412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569ff175600 con 0x5569ff2a4f30 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f722f7fe640 1 --2- 172.21.15.60:0/4285775412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff332de0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f7220000c10 tx=0x7f722001b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1b6700b9a2b06b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f720e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f720e7fc640 1 -- 172.21.15.60:0/4285775412 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72200b4070 con 0x5569ff2a4f30 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f720e7fc640 1 -- 172.21.15.60:0/4285775412 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f72200c7690 con 0x5569ff2a4f30 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 -- 172.21.15.60:0/4285775412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7224000e50 con 0x5569ff2a4f30 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f720e7fc640 1 -- 172.21.15.60:0/4285775412 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72200c7890 con 0x5569ff2a4f30 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 10 client.8319 fetch_fsmap learned FSMap version 68 2024-04-25T10:13:24.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 -- 172.21.15.60:0/4285775412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x5569ff0889b0 con 0x5569ff2a4f30 2024-04-25T10:13:24.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f720e7fc640 1 -- 172.21.15.60:0/4285775412 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7224000e50 con 0x5569ff2a4f30 2024-04-25T10:13:24.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f720e7fc640 1 -- 172.21.15.60:0/4285775412 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f72200c7ce0 con 0x5569ff2a4f30 2024-04-25T10:13:24.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 10 client.8319 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:13:24.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 10 client.8319 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:24.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 20 client.8319 populate_metadata read hostname 'smithi060' 2024-04-25T10:13:24.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 --2- 172.21.15.60:0/4285775412 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x5569ff2ef240 0x5569ff3341b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 4 client.8319 mds_command: new command op to 8256 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:13:24.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 -- 172.21.15.60:0/4285775412 --> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5569ff1012e0 con 0x5569ff2ef240 2024-04-25T10:13:24.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f722effd640 1 --2- 172.21.15.60:0/4285775412 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x5569ff2ef240 0x5569ff3341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f722effd640 1 --2- 172.21.15.60:0/4285775412 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x5569ff2ef240 0x5569ff3341b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6381dc4637432c8c server_cookie=ef543f30cab41ecb in_seq=0 out_seq=0 2024-04-25T10:13:24.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f720e7fc640 10 client.8319 ms_handle_connect on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:24.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f720e7fc640 1 -- 172.21.15.60:0/4285775412 <== mds.0 v2:172.21.15.60:6833/3807131024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5569ff1012e0 con 0x5569ff2ef240 2024-04-25T10:13:24.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f720e7fc640 10 client.8319 handle_command_reply: tid=0 2024-04-25T10:13:24.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 client.8319 shutdown 2024-04-25T10:13:24.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 -- 172.21.15.60:0/4285775412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 msgr2=0x5569ff332de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 --2- 172.21.15.60:0/4285775412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff332de0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f7220000c10 tx=0x7f722001b900 comp rx=0 tx=0).stop 2024-04-25T10:13:24.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 -- 172.21.15.60:0/4285775412 shutdown_connections 2024-04-25T10:13:24.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 --2- 172.21.15.60:0/4285775412 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x5569ff2ef240 0x5569ff3341b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 --2- 172.21.15.60:0/4285775412 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569ff2ee920 0x5569ff333c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 --2- 172.21.15.60:0/4285775412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569ff2ab2a0 0x5569ff333730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f720e7fc640 0 client.8319 ms_handle_reset on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:24.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 --2- 172.21.15.60:0/4285775412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569ff2a4f30 0x5569ff332de0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 -- 172.21.15.60:0/4285775412 >> 172.21.15.60:0/4285775412 conn(0x5569ff2a8510 msgr2=0x5569ff2ed910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.687+0000 7f72392f4000 1 -- 172.21.15.60:0/4285775412 shutdown_connections 2024-04-25T10:13:24.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.691+0000 7f72392f4000 1 -- 172.21.15.60:0/4285775412 wait complete. 2024-04-25T10:13:24.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.691+0000 7f72392f4000 20 client.8319 trim_cache size 0 max 16384 2024-04-25T10:13:24.700 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:13:24.700 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "0ea93863-a079-45c4-abed-6e83540ea9fe", 2024-04-25T10:13:24.700 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T10:13:24.700 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 8256, 2024-04-25T10:13:24.700 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T10:13:24.700 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T10:13:24.700 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T10:13:24.700 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 4.4821161590000003, 2024-04-25T10:13:24.700 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 68, 2024-04-25T10:13:24.700 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 86, 2024-04-25T10:13:24.700 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T10:13:24.700 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 5.4283480050000001 2024-04-25T10:13:24.700 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:13:24.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.691+0000 7f7236187640 1 -- 172.21.15.60:0/3722211864 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f721c066f60 msgr2=0x7f721c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.691+0000 7f7236187640 1 --2- 172.21.15.60:0/3722211864 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f721c066f60 0x7f721c069420 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f72200b1000 tx=0x7f7220008040 comp rx=0 tx=0).stop 2024-04-25T10:13:24.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.691+0000 7f7236187640 1 -- 172.21.15.60:0/3722211864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72300cce50 msgr2=0x7f7230158480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.691+0000 7f7236187640 1 --2- 172.21.15.60:0/3722211864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72300cce50 0x7f7230158480 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f7228007e60 tx=0x7f722800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.691+0000 7f7236187640 1 -- 172.21.15.60:0/3722211864 shutdown_connections 2024-04-25T10:13:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.691+0000 7f7236187640 1 --2- 172.21.15.60:0/3722211864 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f721c066f60 0x7f721c069420 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.691+0000 7f7236187640 1 --2- 172.21.15.60:0/3722211864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72300cce50 0x7f7230158480 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.691+0000 7f7236187640 1 --2- 172.21.15.60:0/3722211864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7230071b20 0x7f723015cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.691+0000 7f7236187640 1 --2- 172.21.15.60:0/3722211864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f723006e480 0x7f723015bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.691+0000 7f7236187640 1 -- 172.21.15.60:0/3722211864 >> 172.21.15.60:0/3722211864 conn(0x7f72300b12d0 msgr2=0x7f72300cb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.691+0000 7f7236187640 1 -- 172.21.15.60:0/3722211864 shutdown_connections 2024-04-25T10:13:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.691+0000 7f7236187640 1 -- 172.21.15.60:0/3722211864 wait complete. 2024-04-25T10:13:24.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:13:24.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.815+0000 7f7762978640 1 Processor -- start 2024-04-25T10:13:24.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.815+0000 7f7762978640 1 -- start start 2024-04-25T10:13:24.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f7762978640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f775c15bc90 0x7f775c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f7762978640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f775c071ca0 con 0x7f775c0b4660 2024-04-25T10:13:24.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f7762978640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f775c0cca40 con 0x7f775c0ce7a0 2024-04-25T10:13:24.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f7762978640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f775c0cc370 con 0x7f775c15bc90 2024-04-25T10:13:24.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f775b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f775c15bc90 0x7f775c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f7760f15640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f775c0b4660 0x7f775c15c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56670/0 (socket says 172.21.15.60:56670) 2024-04-25T10:13:24.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f7760f15640 1 -- 172.21.15.60:0/3796995150 learned_addr learned my addr 172.21.15.60:0/3796995150 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:13:24.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f775affd640 1 -- 172.21.15.60:0/3796995150 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331034110 0 0) 0x7f775c071ca0 con 0x7f775c0b4660 2024-04-25T10:13:24.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f775affd640 1 -- 172.21.15.60:0/3796995150 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7744003530 con 0x7f775c0b4660 2024-04-25T10:13:24.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f775b7fe640 1 -- 172.21.15.60:0/3796995150 >> v1:172.21.15.100:6789/0 conn(0x7f775c0ce7a0 legacy=0x7f775c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f775b7fe640 1 -- 172.21.15.60:0/3796995150 >> v1:172.21.15.60:6789/0 conn(0x7f775c0b4660 legacy=0x7f775c15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f775b7fe640 1 -- 172.21.15.60:0/3796995150 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c0ccd20 con 0x7f775c15bc90 2024-04-25T10:13:24.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f775b7fe640 1 --2- 172.21.15.60:0/3796995150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775c15bc90 0x7f775c15c220 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f77500097c0 tx=0x7f775001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c40f2d0c73c03a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f775affd640 1 -- 172.21.15.60:0/3796995150 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77500b0070 con 0x7f775c15bc90 2024-04-25T10:13:24.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f775affd640 1 -- 172.21.15.60:0/3796995150 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77500ab600 con 0x7f775c15bc90 2024-04-25T10:13:24.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f775affd640 1 -- 172.21.15.60:0/3796995150 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77500ab9d0 con 0x7f775c15bc90 2024-04-25T10:13:24.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f7762978640 1 -- 172.21.15.60:0/3796995150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775c15bc90 msgr2=0x7f775c15c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f7762978640 1 --2- 172.21.15.60:0/3796995150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775c15bc90 0x7f775c15c220 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f77500097c0 tx=0x7f775001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f7762978640 1 -- 172.21.15.60:0/3796995150 shutdown_connections 2024-04-25T10:13:24.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f7762978640 1 --2- 172.21.15.60:0/3796995150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775c15bc90 0x7f775c15c220 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f7762978640 1 -- 172.21.15.60:0/3796995150 >> 172.21.15.60:0/3796995150 conn(0x7f775c0b12d0 msgr2=0x7f775c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f7762978640 1 -- 172.21.15.60:0/3796995150 shutdown_connections 2024-04-25T10:13:24.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.819+0000 7f7762978640 1 -- 172.21.15.60:0/3796995150 wait complete. 2024-04-25T10:13:24.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7762978640 1 Processor -- start 2024-04-25T10:13:24.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7762978640 1 -- start start 2024-04-25T10:13:24.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7762978640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f775c0ce7a0 0x7f775c06eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7762978640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f775c15bc90 0x7f775c149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7762978640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775c0b0780 0x7f775c1524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7762978640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c1695d0 con 0x7f775c0b0780 2024-04-25T10:13:24.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7762978640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c0b3db0 con 0x7f775c0ce7a0 2024-04-25T10:13:24.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7762978640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775c169750 con 0x7f775c15bc90 2024-04-25T10:13:24.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7760f15640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775c0b0780 0x7f775c1524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7760f15640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775c0b0780 0x7f775c1524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55148/0 (socket says 172.21.15.60:55148) 2024-04-25T10:13:24.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7760f15640 1 -- 172.21.15.60:0/3053533966 learned_addr learned my addr 172.21.15.60:0/3053533966 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:24.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7760f15640 1 -- 172.21.15.60:0/3053533966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f775c15bc90 msgr2=0x7f775c149d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7760f15640 1 --2- 172.21.15.60:0/3053533966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f775c15bc90 0x7f775c149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7760f15640 1 -- 172.21.15.60:0/3053533966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f775c0ce7a0 msgr2=0x7f775c06eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7760f15640 1 --2- 172.21.15.60:0/3053533966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f775c0ce7a0 0x7f775c06eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7760f15640 1 -- 172.21.15.60:0/3053533966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c14ab40 con 0x7f775c0b0780 2024-04-25T10:13:24.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7760f15640 1 --2- 172.21.15.60:0/3053533966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775c0b0780 0x7f775c1524f0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f775400ca70 tx=0x7f775400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7758ff9640 1 -- 172.21.15.60:0/3053533966 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754017070 con 0x7f775c0b0780 2024-04-25T10:13:24.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7758ff9640 1 -- 172.21.15.60:0/3053533966 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775400abd0 con 0x7f775c0b0780 2024-04-25T10:13:24.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7762978640 1 -- 172.21.15.60:0/3053533966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775c06e070 con 0x7f775c0b0780 2024-04-25T10:13:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7758ff9640 1 -- 172.21.15.60:0/3053533966 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754012400 con 0x7f775c0b0780 2024-04-25T10:13:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7762978640 1 -- 172.21.15.60:0/3053533966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775c14a240 con 0x7f775c0b0780 2024-04-25T10:13:24.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.823+0000 7f7762978640 1 -- 172.21.15.60:0/3053533966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775c074770 con 0x7f775c0b0780 2024-04-25T10:13:24.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.827+0000 7f7758ff9640 1 -- 172.21.15.60:0/3053533966 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f775401a070 con 0x7f775c0b0780 2024-04-25T10:13:24.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.827+0000 7f7758ff9640 1 --2- 172.21.15.60:0/3053533966 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7748066f90 0x7f7748069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.827+0000 7f7758ff9640 1 -- 172.21.15.60:0/3053533966 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f7754084ce0 con 0x7f775c0b0780 2024-04-25T10:13:24.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.827+0000 7f775bfff640 1 --2- 172.21.15.60:0/3053533966 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7748066f90 0x7f7748069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.827+0000 7f775bfff640 1 --2- 172.21.15.60:0/3053533966 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7748066f90 0x7f7748069450 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f774c000f70 tx=0x7f774c0079e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:24.831+0000 7f7758ff9640 1 -- 172.21.15.60:0/3053533966 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f77540555d0 con 0x7f775c0b0780 2024-04-25T10:13:25.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:25.019+0000 7f7762978640 1 -- 172.21.15.60:0/3053533966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f775c06e590 con 0x7f775c0b0780 2024-04-25T10:13:25.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:25.019+0000 7f7758ff9640 1 -- 172.21.15.60:0/3053533966 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7754059670 con 0x7f775c0b0780 2024-04-25T10:13:25.025 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:13:25.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:25.023+0000 7f7762978640 1 -- 172.21.15.60:0/3053533966 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7748066f90 msgr2=0x7f7748069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:25.023+0000 7f7762978640 1 --2- 172.21.15.60:0/3053533966 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7748066f90 0x7f7748069450 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f774c000f70 tx=0x7f774c0079e0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:25.023+0000 7f7762978640 1 -- 172.21.15.60:0/3053533966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775c0b0780 msgr2=0x7f775c1524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:25.023+0000 7f7762978640 1 --2- 172.21.15.60:0/3053533966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775c0b0780 0x7f775c1524f0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f775400ca70 tx=0x7f775400cf30 comp rx=0 tx=0).stop 2024-04-25T10:13:25.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:25.023+0000 7f7762978640 1 -- 172.21.15.60:0/3053533966 shutdown_connections 2024-04-25T10:13:25.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:25.023+0000 7f7762978640 1 --2- 172.21.15.60:0/3053533966 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7748066f90 0x7f7748069450 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:25.023+0000 7f7762978640 1 --2- 172.21.15.60:0/3053533966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f775c0b0780 0x7f775c1524f0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:25.023+0000 7f7762978640 1 --2- 172.21.15.60:0/3053533966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f775c15bc90 0x7f775c149d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:25.023+0000 7f7762978640 1 --2- 172.21.15.60:0/3053533966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f775c0ce7a0 0x7f775c06eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:25.023+0000 7f7762978640 1 -- 172.21.15.60:0/3053533966 >> 172.21.15.60:0/3053533966 conn(0x7f775c0b12d0 msgr2=0x7f775c0b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:25.023+0000 7f7762978640 1 -- 172.21.15.60:0/3053533966 shutdown_connections 2024-04-25T10:13:25.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:25.023+0000 7f7762978640 1 -- 172.21.15.60:0/3053533966 wait complete. 2024-04-25T10:13:25.043 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:13:25.043 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:13:25.043 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:13:25.043 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:13:25.043 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:25.043 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:13:25.043 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:13:25.043 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:13:25.043 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:25.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:25.044 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:13:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:13:25.050 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:13:25.050 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:13:25.050 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:25.050 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:13:25.050 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:25.050 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:25.050 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:13:25.050 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:25.050 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:25.051 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:13:25.051 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:13:25.051 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:25.051 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:13:25.051 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:13:25.051 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42052sec preferred_lft 42052sec 2024-04-25T10:13:25.051 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:13:25.051 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:25.051 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:25.051 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:25.051 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:13:25.051 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:13:25.051 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:25.051 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:13:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:25.052 INFO:teuthology.orchestra.run.smithi060.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:13:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:13:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:13:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:25.052 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:13:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:25.052 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:25.052 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:13:25.053 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:25.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:25.053 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:13:25.102 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:13:25.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:25.104 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:13:25.156 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:13:25.156 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:13:25.156 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:13:25.156 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:13:25.156 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:13:25.156 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:13:25.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:25.157 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:13:25.157 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:13:25.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:13:25.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:13:25.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:13:25.157 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:13:25.213 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-04-25T10:13:25.207+0000 7f1d83d39640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:25.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:25.231 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:13:25.279 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:13:25.279 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:13:25.279 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:25.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:25.280 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T10:13:25.334 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:13:25.334 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:25.334 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:25.383 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:25.383 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:25.383 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:25.431 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:13:25.488 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:13:25.537 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:13:25.538 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:13:25.538 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:13:25.538 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:13:25.538 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:13:25.538 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:13:25.538 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:13:25.538 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:13:25.538 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:13:25.538 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:13:25.538 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:13:25.538 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:13:25.538 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:13:25.538 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:13:25.539 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:13:25.539 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:13:25.539 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:13:25.539 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:13:25.539 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:13:25.539 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:13:25.539 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:13:25.539 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:13:25.539 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:13:25.539 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:13:25.539 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:13:25.539 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:13:25.539 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:13:25.539 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:13:25.539 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:13:25.540 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:25.540 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:25.540 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:25.540 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:25.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:25.541 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:13:25.587 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:13:25.587 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:13:25.632 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:13:25.648 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:13:25.695 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:13:25.695 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:13:25.695 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:13:25.695 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:13:25.695 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:13:25.696 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:13:25.696 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:13:25.696 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:13:25.696 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:13:25.696 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:13:25.696 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:13:25.696 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:13:25.696 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:13:25.696 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:13:25.696 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:13:25.696 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:13:25.696 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:13:25.697 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:13:25.697 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:13:25.697 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:13:25.697 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:13:25.697 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:13:25.697 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:13:25.697 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:13:25.697 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:13:25.697 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:13:25.697 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:13:25.697 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:13:25.697 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:13:25.697 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:25.697 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:25.697 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:25.698 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:25.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:25.699 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:13:25.714 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2024-04-25T10:13:25.707+0000 7f2af012b580 -1 init, newargv = 0x55bb06f1a360 newargc=15 2024-04-25T10:13:25.714 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[47250]: starting ceph client 2024-04-25T10:13:25.720 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[47250]: starting fuse 2024-04-25T10:13:26.819 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:13:26.836 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:13:26.885 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:13:26.885 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:13:26.887 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:13:26.887 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:13:26.887 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:13:26.887 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:13:26.887 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:13:26.887 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:13:26.887 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:13:26.887 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:13:26.887 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:13:26.887 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:13:26.887 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:13:26.887 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:13:26.887 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:13:26.887 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:13:26.888 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:13:26.888 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:26.888 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:26.888 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:26.888 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:26.888 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:26.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:26.889 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:13:26.935 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:13:26.935 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:13:26.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:13:26.935 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:13:26.935 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:13:26.935 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:13:26.936 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:13:26.937 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:13:26.937 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:13:26.937 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:13:26.937 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:13:26.937 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2024-04-25T10:13:26.937 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:13:27.063 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.47250.asok 2024-04-25T10:13:27.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:27.075 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47250.asok 2024-04-25T10:13:27.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:27.075 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47250.asok status 2024-04-25T10:13:27.234 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:13:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:13:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:13:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:13:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2024-04-25T10:13:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:13:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:13:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "47250", 2024-04-25T10:13:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:13:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:13:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:13:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:13:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 8334, 2024-04-25T10:13:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:13:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:13:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:13:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 8334 2024-04-25T10:13:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:13:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:13:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:13:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:13:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3749336275 2024-04-25T10:13:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:13:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:13:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:13:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:13:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:13:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3749336275 2024-04-25T10:13:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:13:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.8334 192.168.0.1:0/3749336275", 2024-04-25T10:13:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/3749336275", 2024-04-25T10:13:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:13:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 68, 2024-04-25T10:13:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 86, 2024-04-25T10:13:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:13:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:13:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:13:27.237 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:13:27.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:27.249 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:13:27.249 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:27.255 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:13:27.255 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:27.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:27.255 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:27.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:27.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-25T10:13:27.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.479+0000 7f67ccb67640 1 Processor -- start 2024-04-25T10:13:27.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.479+0000 7f67ccb67640 1 -- start start 2024-04-25T10:13:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67ccb67640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f67c80b93a0 0x7f67c80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67ccb67640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c80cebb0 0x7f67c80d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67ccb67640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c806c8d0 con 0x7f67c815db10 2024-04-25T10:13:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67ccb67640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c80d1b10 con 0x7f67c80b93a0 2024-04-25T10:13:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67ccb67640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80cf000 con 0x7f67c80cebb0 2024-04-25T10:13:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67bdd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f67c80b93a0 0x7f67c80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67bdd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f67c80b93a0 0x7f67c80ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55152/0 (socket says 172.21.15.60:55152) 2024-04-25T10:13:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67bdd9b640 1 -- 172.21.15.60:0/1982743346 learned_addr learned my addr 172.21.15.60:0/1982743346 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67bdd9b640 1 -- 172.21.15.60:0/1982743346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c80cebb0 msgr2=0x7f67c80d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67c6d9d640 1 --2- 172.21.15.60:0/1982743346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c80cebb0 0x7f67c80d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67bdd9b640 1 --2- 172.21.15.60:0/1982743346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c80cebb0 0x7f67c80d2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67bdd9b640 1 -- 172.21.15.60:0/1982743346 >> v1:172.21.15.100:6789/0 conn(0x7f67c815db10 legacy=0x7f67c806c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67bdd9b640 1 -- 172.21.15.60:0/1982743346 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c806cbb0 con 0x7f67c80b93a0 2024-04-25T10:13:27.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67c6d9d640 1 --2- 172.21.15.60:0/1982743346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c80cebb0 0x7f67c80d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:27.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67bdd9b640 1 --2- 172.21.15.60:0/1982743346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c80b93a0 0x7f67c80ce670 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f67b00098a0 tx=0x7f67b001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba52f01bccda8868 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67c5d9b640 1 -- 172.21.15.60:0/1982743346 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b00ab680 con 0x7f67c80b93a0 2024-04-25T10:13:27.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67c5d9b640 1 -- 172.21.15.60:0/1982743346 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b00b0070 con 0x7f67c80b93a0 2024-04-25T10:13:27.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.483+0000 7f67c5d9b640 1 -- 172.21.15.60:0/1982743346 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b00aba40 con 0x7f67c80b93a0 2024-04-25T10:13:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 -- 172.21.15.60:0/1982743346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c80b93a0 msgr2=0x7f67c80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 --2- 172.21.15.60:0/1982743346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c80b93a0 0x7f67c80ce670 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f67b00098a0 tx=0x7f67b001c480 comp rx=0 tx=0).stop 2024-04-25T10:13:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 -- 172.21.15.60:0/1982743346 shutdown_connections 2024-04-25T10:13:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 --2- 172.21.15.60:0/1982743346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c80b93a0 0x7f67c80ce670 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 --2- 172.21.15.60:0/1982743346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c80cebb0 0x7f67c80d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 -- 172.21.15.60:0/1982743346 >> 172.21.15.60:0/1982743346 conn(0x7f67c80d0420 msgr2=0x7f67c80d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 -- 172.21.15.60:0/1982743346 shutdown_connections 2024-04-25T10:13:27.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 -- 172.21.15.60:0/1982743346 wait complete. 2024-04-25T10:13:27.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 Processor -- start 2024-04-25T10:13:27.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 -- start start 2024-04-25T10:13:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c80b93a0 0x7f67c815c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c80cebb0 0x7f67c80c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c815db10 0x7f67c80c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80d3130 con 0x7f67c80b93a0 2024-04-25T10:13:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80d2fb0 con 0x7f67c815db10 2024-04-25T10:13:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80d32b0 con 0x7f67c80cebb0 2024-04-25T10:13:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67c659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c80b93a0 0x7f67c815c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67c659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c80b93a0 0x7f67c815c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55158/0 (socket says 172.21.15.60:55158) 2024-04-25T10:13:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67c659c640 1 -- 172.21.15.60:0/3162662965 learned_addr learned my addr 172.21.15.60:0/3162662965 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:27.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67bdd9b640 1 --2- 172.21.15.60:0/3162662965 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c80cebb0 0x7f67c80c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67c6d9d640 1 --2- 172.21.15.60:0/3162662965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c815db10 0x7f67c80c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67c659c640 1 -- 172.21.15.60:0/3162662965 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c80cebb0 msgr2=0x7f67c80c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67c659c640 1 --2- 172.21.15.60:0/3162662965 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c80cebb0 0x7f67c80c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67c659c640 1 -- 172.21.15.60:0/3162662965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c815db10 msgr2=0x7f67c80c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67c659c640 1 --2- 172.21.15.60:0/3162662965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c815db10 0x7f67c80c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67c659c640 1 -- 172.21.15.60:0/3162662965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c8156440 con 0x7f67c80b93a0 2024-04-25T10:13:27.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67c659c640 1 --2- 172.21.15.60:0/3162662965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c80b93a0 0x7f67c815c5f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f67b800c3c0 tx=0x7f67b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67bf7fe640 1 -- 172.21.15.60:0/3162662965 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8017070 con 0x7f67c80b93a0 2024-04-25T10:13:27.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67bf7fe640 1 -- 172.21.15.60:0/3162662965 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b80052d0 con 0x7f67c80b93a0 2024-04-25T10:13:27.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 -- 172.21.15.60:0/3162662965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c8156720 con 0x7f67c80b93a0 2024-04-25T10:13:27.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67bf7fe640 1 -- 172.21.15.60:0/3162662965 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8003840 con 0x7f67c80b93a0 2024-04-25T10:13:27.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.487+0000 7f67ccb67640 1 -- 172.21.15.60:0/3162662965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c8154d60 con 0x7f67c80b93a0 2024-04-25T10:13:27.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67bf7fe640 1 -- 172.21.15.60:0/3162662965 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67b801a070 con 0x7f67c80b93a0 2024-04-25T10:13:27.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67bf7fe640 1 --2- 172.21.15.60:0/3162662965 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67a8066f90 0x7f67a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67bf7fe640 1 -- 172.21.15.60:0/3162662965 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f67b80842d0 con 0x7f67c80b93a0 2024-04-25T10:13:27.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67bdd9b640 1 --2- 172.21.15.60:0/3162662965 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67a8066f90 0x7f67a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67cfcd4000 1 Processor -- start 2024-04-25T10:13:27.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67cfcd4000 1 -- start start 2024-04-25T10:13:27.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67bdd9b640 1 --2- 172.21.15.60:0/3162662965 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67a8066f90 0x7f67a8069450 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f67b00af000 tx=0x7f67b00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67cfcd4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa71c90 0x5622cfa72070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67cfcd4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa725b0 0x5622cfa27d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67cfcd4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa28290 0x5622cfa28670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67cfcd4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622cf90ef40 con 0x5622cfa28290 2024-04-25T10:13:27.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67cfcd4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622cf876070 con 0x5622cfa71c90 2024-04-25T10:13:27.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67cfcd4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622cf7fe1a0 con 0x5622cfa725b0 2024-04-25T10:13:27.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67c6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa28290 0x5622cfa28670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67c6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa28290 0x5622cfa28670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55174/0 (socket says 172.21.15.60:55174) 2024-04-25T10:13:27.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67c6d9d640 1 -- 172.21.15.60:0/3257556986 learned_addr learned my addr 172.21.15.60:0/3257556986 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:27.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67bdd9b640 1 --2- 172.21.15.60:0/3257556986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa725b0 0x5622cfa27d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.491+0000 7f67c659c640 1 --2- 172.21.15.60:0/3257556986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa71c90 0x5622cfa72070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c6d9d640 1 -- 172.21.15.60:0/3257556986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa725b0 msgr2=0x5622cfa27d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c6d9d640 1 --2- 172.21.15.60:0/3257556986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa725b0 0x5622cfa27d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c6d9d640 1 -- 172.21.15.60:0/3257556986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa71c90 msgr2=0x5622cfa72070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c6d9d640 1 --2- 172.21.15.60:0/3257556986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa71c90 0x5622cfa72070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c6d9d640 1 -- 172.21.15.60:0/3257556986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622cf7fc510 con 0x5622cfa28290 2024-04-25T10:13:27.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c6d9d640 1 --2- 172.21.15.60:0/3257556986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa28290 0x5622cfa28670 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f67c000e4f0 tx=0x7f67c000e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95bc029e7db24273 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67a7fff640 1 -- 172.21.15.60:0/3257556986 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c000c9d0 con 0x5622cfa28290 2024-04-25T10:13:27.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67a7fff640 1 -- 172.21.15.60:0/3257556986 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c000cb70 con 0x5622cfa28290 2024-04-25T10:13:27.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67a7fff640 1 -- 172.21.15.60:0/3257556986 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c000a410 con 0x5622cfa28290 2024-04-25T10:13:27.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3257556986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa28290 msgr2=0x5622cfa28670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/3257556986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa28290 0x5622cfa28670 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f67c000e4f0 tx=0x7f67c000e9b0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3257556986 shutdown_connections 2024-04-25T10:13:27.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/3257556986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa28290 0x5622cfa28670 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/3257556986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa725b0 0x5622cfa27d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/3257556986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa71c90 0x5622cfa72070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3257556986 >> 172.21.15.60:0/3257556986 conn(0x5622cfa24240 msgr2=0x5622cfa26630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3257556986 shutdown_connections 2024-04-25T10:13:27.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3257556986 wait complete. 2024-04-25T10:13:27.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 -- start start 2024-04-25T10:13:27.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 Processor -- start 2024-04-25T10:13:27.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa28290 0x5622cfa32e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa71c90 0x5622cfa33370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa725b0 0x5622cfa338b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622cf8306c0 con 0x5622cfa725b0 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622cf872110 con 0x5622cfa28290 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622cf92cf20 con 0x5622cfa71c90 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 10 client.0 _ll_register_callbacks cb 0x5622cfa13570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa725b0 0x5622cfa338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa725b0 0x5622cfa338b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55176/0 (socket says 172.21.15.60:55176) 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c6d9d640 1 -- 172.21.15.60:0/3143917569 learned_addr learned my addr 172.21.15.60:0/3143917569 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67bdd9b640 1 --2- 172.21.15.60:0/3143917569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa71c90 0x5622cfa33370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c659c640 1 --2- 172.21.15.60:0/3143917569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa28290 0x5622cfa32e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c6d9d640 1 -- 172.21.15.60:0/3143917569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa71c90 msgr2=0x5622cfa33370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c6d9d640 1 --2- 172.21.15.60:0/3143917569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa71c90 0x5622cfa33370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c6d9d640 1 -- 172.21.15.60:0/3143917569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa28290 msgr2=0x5622cfa32e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c6d9d640 1 --2- 172.21.15.60:0/3143917569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa28290 0x5622cfa32e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c6d9d640 1 -- 172.21.15.60:0/3143917569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622cf7fe3b0 con 0x5622cfa725b0 2024-04-25T10:13:27.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c659c640 1 --2- 172.21.15.60:0/3143917569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa28290 0x5622cfa32e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:27.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67bdd9b640 1 --2- 172.21.15.60:0/3143917569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa71c90 0x5622cfa33370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:27.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67c6d9d640 1 --2- 172.21.15.60:0/3143917569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa725b0 0x5622cfa338b0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f67c001a040 tx=0x7f67c0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc020ee1e661043 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:13:27.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67a57fa640 1 -- 172.21.15.60:0/3143917569 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0027070 con 0x5622cfa725b0 2024-04-25T10:13:27.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67a57fa640 1 -- 172.21.15.60:0/3143917569 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f67c0003db0 con 0x5622cfa725b0 2024-04-25T10:13:27.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3143917569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67c000cb70 con 0x5622cfa725b0 2024-04-25T10:13:27.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67a57fa640 1 -- 172.21.15.60:0/3143917569 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67c0032050 con 0x5622cfa725b0 2024-04-25T10:13:27.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 10 client.8346 fetch_fsmap learned FSMap version 68 2024-04-25T10:13:27.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3143917569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x5622cf8f7e00 con 0x5622cfa725b0 2024-04-25T10:13:27.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.495+0000 7f67a57fa640 1 -- 172.21.15.60:0/3143917569 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c000cb70 con 0x5622cfa725b0 2024-04-25T10:13:27.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67a57fa640 1 -- 172.21.15.60:0/3143917569 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f67c0035020 con 0x5622cfa725b0 2024-04-25T10:13:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 10 client.8346 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:13:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 10 client.8346 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:13:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 20 client.8346 populate_metadata read hostname 'smithi060' 2024-04-25T10:13:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/3143917569 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x5622cfa34180 0x5622cfa83410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 4 client.8346 mds_command: new command op to 8256 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3143917569 --> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622cf811d60 con 0x5622cfa34180 2024-04-25T10:13:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67c659c640 1 --2- 172.21.15.60:0/3143917569 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x5622cfa34180 0x5622cfa83410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67a57fa640 10 client.8346 ms_handle_connect on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67c659c640 1 --2- 172.21.15.60:0/3143917569 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x5622cfa34180 0x5622cfa83410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce5a23c1465d7cf1 server_cookie=28f11e4c38345343 in_seq=0 out_seq=0 2024-04-25T10:13:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67a57fa640 1 -- 172.21.15.60:0/3143917569 <== mds.0 v2:172.21.15.60:6833/3807131024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5622cf811d60 con 0x5622cfa34180 2024-04-25T10:13:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67a57fa640 10 client.8346 handle_command_reply: tid=0 2024-04-25T10:13:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 1 client.8346 shutdown 2024-04-25T10:13:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3143917569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa725b0 msgr2=0x5622cfa338b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/3143917569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa725b0 0x5622cfa338b0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f67c001a040 tx=0x7f67c0004ac0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3143917569 shutdown_connections 2024-04-25T10:13:27.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/3143917569 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x5622cfa34180 0x5622cfa83410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/3143917569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa725b0 0x5622cfa338b0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/3143917569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa71c90 0x5622cfa33370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67a57fa640 0 client.8346 ms_handle_reset on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:27.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/3143917569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa28290 0x5622cfa32e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3143917569 >> 172.21.15.60:0/3143917569 conn(0x5622cfa24240 msgr2=0x5622cfa26ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3143917569 shutdown_connections 2024-04-25T10:13:27.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3143917569 wait complete. 2024-04-25T10:13:27.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.499+0000 7f67cfcd4000 20 client.8346 trim_cache size 0 max 16384 2024-04-25T10:13:27.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 Processor -- start 2024-04-25T10:13:27.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 -- start start 2024-04-25T10:13:27.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa28290 0x5622cfa27640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa71c90 0x5622cfa27b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa725b0 0x5622cfa83410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c82029b0 con 0x5622cfa725b0 2024-04-25T10:13:27.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c8201bc0 con 0x5622cfa71c90 2024-04-25T10:13:27.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c8153c00 con 0x5622cfa28290 2024-04-25T10:13:27.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67c6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa725b0 0x5622cfa83410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67c6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa725b0 0x5622cfa83410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55192/0 (socket says 172.21.15.60:55192) 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67c6d9d640 1 -- 172.21.15.60:0/3509048476 learned_addr learned my addr 172.21.15.60:0/3509048476 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67c6d9d640 1 -- 172.21.15.60:0/3509048476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa28290 msgr2=0x5622cfa27640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67c6d9d640 1 --2- 172.21.15.60:0/3509048476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa28290 0x5622cfa27640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67c6d9d640 1 -- 172.21.15.60:0/3509048476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa71c90 msgr2=0x5622cfa27b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67c6d9d640 1 --2- 172.21.15.60:0/3509048476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa71c90 0x5622cfa27b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67c6d9d640 1 -- 172.21.15.60:0/3509048476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622cf8cdc50 con 0x5622cfa725b0 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67c6d9d640 1 --2- 172.21.15.60:0/3509048476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa725b0 0x5622cfa83410 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f67c0007cb0 tx=0x7f67c0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9b837b453d97bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67a77fe640 1 -- 172.21.15.60:0/3509048476 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c003b070 con 0x5622cfa725b0 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67a77fe640 1 -- 172.21.15.60:0/3509048476 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c000cb70 con 0x5622cfa725b0 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67a77fe640 1 -- 172.21.15.60:0/3509048476 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0027040 con 0x5622cfa725b0 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3509048476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa725b0 msgr2=0x5622cfa83410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/3509048476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa725b0 0x5622cfa83410 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f67c0007cb0 tx=0x7f67c0020000 comp rx=0 tx=0).stop 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3509048476 shutdown_connections 2024-04-25T10:13:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/3509048476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa725b0 0x5622cfa83410 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/3509048476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa71c90 0x5622cfa27b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/3509048476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa28290 0x5622cfa27640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3509048476 >> 172.21.15.60:0/3509048476 conn(0x5622cfa24240 msgr2=0x5622cfa2cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3509048476 shutdown_connections 2024-04-25T10:13:27.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 -- 172.21.15.60:0/3509048476 wait complete. 2024-04-25T10:13:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 -- start start 2024-04-25T10:13:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.515+0000 7f67cfcd4000 1 Processor -- start 2024-04-25T10:13:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa28290 0x5622cfa854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa71c90 0x5622cfa8dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa725b0 0x5622cfa8e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622cfa2db70 con 0x5622cfa28290 2024-04-25T10:13:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c8154200 con 0x5622cfa725b0 2024-04-25T10:13:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622cfa31200 con 0x5622cfa71c90 2024-04-25T10:13:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 10 client.0 _ll_register_callbacks cb 0x7f67c0007840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67c659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa28290 0x5622cfa854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67c659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa28290 0x5622cfa854d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55200/0 (socket says 172.21.15.60:55200) 2024-04-25T10:13:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67c659c640 1 -- 172.21.15.60:0/2231566985 learned_addr learned my addr 172.21.15.60:0/2231566985 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67bdd9b640 1 --2- 172.21.15.60:0/2231566985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa71c90 0x5622cfa8dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67c6d9d640 1 --2- 172.21.15.60:0/2231566985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa725b0 0x5622cfa8e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67c659c640 1 -- 172.21.15.60:0/2231566985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa71c90 msgr2=0x5622cfa8dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67c659c640 1 --2- 172.21.15.60:0/2231566985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa71c90 0x5622cfa8dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67c659c640 1 -- 172.21.15.60:0/2231566985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa725b0 msgr2=0x5622cfa8e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67c659c640 1 --2- 172.21.15.60:0/2231566985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa725b0 0x5622cfa8e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67c659c640 1 -- 172.21.15.60:0/2231566985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622cf83e120 con 0x5622cfa28290 2024-04-25T10:13:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67c6d9d640 1 --2- 172.21.15.60:0/2231566985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa725b0 0x5622cfa8e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67c659c640 1 --2- 172.21.15.60:0/2231566985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa28290 0x5622cfa854d0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f67b80827c0 tx=0x7f67b80545b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b5536e661d1757 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:13:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67a57fa640 1 -- 172.21.15.60:0/2231566985 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b807e030 con 0x5622cfa28290 2024-04-25T10:13:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67a57fa640 1 -- 172.21.15.60:0/2231566985 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f67b8052cf0 con 0x5622cfa28290 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 -- 172.21.15.60:0/2231566985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67c000cb70 con 0x5622cfa28290 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67a57fa640 1 -- 172.21.15.60:0/2231566985 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67b8052f50 con 0x5622cfa28290 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 10 client.8352 fetch_fsmap learned FSMap version 68 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 -- 172.21.15.60:0/2231566985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x5622cf81d170 con 0x5622cfa28290 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67a57fa640 1 -- 172.21.15.60:0/2231566985 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c000cb70 con 0x5622cfa28290 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67a57fa640 1 -- 172.21.15.60:0/2231566985 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f67c000cd20 con 0x5622cfa28290 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 10 client.8352 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 10 client.8352 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 20 client.8352 populate_metadata read hostname 'smithi060' 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/2231566985 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x5622cfad1d60 0x5622cfad2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 4 client.8352 mds_command: new command op to 8256 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 -- 172.21.15.60:0/2231566985 --> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x5622cf8835b0 con 0x5622cfad1d60 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67bdd9b640 1 --2- 172.21.15.60:0/2231566985 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x5622cfad1d60 0x5622cfad2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67bdd9b640 1 --2- 172.21.15.60:0/2231566985 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x5622cfad1d60 0x5622cfad2170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ad2f7e175d4cb52 server_cookie=5cb901fef981766b in_seq=0 out_seq=0 2024-04-25T10:13:27.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67a57fa640 10 client.8352 ms_handle_connect on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67a57fa640 1 -- 172.21.15.60:0/2231566985 <== mds.0 v2:172.21.15.60:6833/3807131024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5622cf8835b0 con 0x5622cfad1d60 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67a57fa640 10 client.8352 handle_command_reply: tid=0 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 client.8352 shutdown 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 -- 172.21.15.60:0/2231566985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa28290 msgr2=0x5622cfa854d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/2231566985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa28290 0x5622cfa854d0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f67b80827c0 tx=0x7f67b80545b0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 -- 172.21.15.60:0/2231566985 shutdown_connections 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/2231566985 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x5622cfad1d60 0x5622cfad2170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/2231566985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622cfa725b0 0x5622cfa8e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/2231566985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622cfa71c90 0x5622cfa8dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 --2- 172.21.15.60:0/2231566985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622cfa28290 0x5622cfa854d0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67a57fa640 0 client.8352 ms_handle_reset on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 -- 172.21.15.60:0/2231566985 >> 172.21.15.60:0/2231566985 conn(0x5622cfa24240 msgr2=0x5622cfa26630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 -- 172.21.15.60:0/2231566985 shutdown_connections 2024-04-25T10:13:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 1 -- 172.21.15.60:0/2231566985 wait complete. 2024-04-25T10:13:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.519+0000 7f67cfcd4000 20 client.8352 trim_cache size 0 max 16384 2024-04-25T10:13:27.531 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:13:27.531 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T10:13:27.531 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "d639c8cf-e1f7-4d15-9ca6-a9735785f374", 2024-04-25T10:13:27.531 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T10:13:27.531 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:13:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.523+0000 7f67ccb67640 1 -- 172.21.15.60:0/3162662965 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67a8066f90 msgr2=0x7f67a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.523+0000 7f67ccb67640 1 --2- 172.21.15.60:0/3162662965 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67a8066f90 0x7f67a8069450 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f67b00af000 tx=0x7f67b00a7000 comp rx=0 tx=0).stop 2024-04-25T10:13:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.523+0000 7f67ccb67640 1 -- 172.21.15.60:0/3162662965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c80b93a0 msgr2=0x7f67c815c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.523+0000 7f67ccb67640 1 --2- 172.21.15.60:0/3162662965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c80b93a0 0x7f67c815c5f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f67b800c3c0 tx=0x7f67b800c880 comp rx=0 tx=0).stop 2024-04-25T10:13:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.523+0000 7f67ccb67640 1 -- 172.21.15.60:0/3162662965 shutdown_connections 2024-04-25T10:13:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.523+0000 7f67ccb67640 1 --2- 172.21.15.60:0/3162662965 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67a8066f90 0x7f67a8069450 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.523+0000 7f67ccb67640 1 --2- 172.21.15.60:0/3162662965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c815db10 0x7f67c80c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.523+0000 7f67ccb67640 1 --2- 172.21.15.60:0/3162662965 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c80cebb0 0x7f67c80c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.523+0000 7f67ccb67640 1 --2- 172.21.15.60:0/3162662965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c80b93a0 0x7f67c815c5f0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.523+0000 7f67ccb67640 1 -- 172.21.15.60:0/3162662965 >> 172.21.15.60:0/3162662965 conn(0x7f67c80d0420 msgr2=0x7f67c81680d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.523+0000 7f67ccb67640 1 -- 172.21.15.60:0/3162662965 shutdown_connections 2024-04-25T10:13:27.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.523+0000 7f67ccb67640 1 -- 172.21.15.60:0/3162662965 wait complete. 2024-04-25T10:13:27.533 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-04-25T10:13:27.523+0000 7f1d7fd31640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5644a4940680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:13:27.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:27.543 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:13:27.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.655+0000 7f5da6ca0640 1 Processor -- start 2024-04-25T10:13:27.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.659+0000 7f5da6ca0640 1 -- start start 2024-04-25T10:13:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.659+0000 7f5da6ca0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5da006f030 0x7f5da00b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.659+0000 7f5da6ca0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5da0071250 0x7f5da00716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.659+0000 7f5da6ca0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da015ad30 con 0x7f5da00b3f60 2024-04-25T10:13:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.659+0000 7f5da6ca0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da015aaf0 con 0x7f5da006f030 2024-04-25T10:13:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.659+0000 7f5da6ca0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da015b010 con 0x7f5da0071250 2024-04-25T10:13:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.659+0000 7f5d97fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5da006f030 0x7f5da00b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.659+0000 7f5d97fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5da006f030 0x7f5da00b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55208/0 (socket says 172.21.15.60:55208) 2024-04-25T10:13:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.659+0000 7f5d97fff640 1 -- 172.21.15.60:0/2315313016 learned_addr learned my addr 172.21.15.60:0/2315313016 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.659+0000 7f5d97fff640 1 -- 172.21.15.60:0/2315313016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5da0071250 msgr2=0x7f5da00716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.659+0000 7f5d97fff640 1 --2- 172.21.15.60:0/2315313016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5da0071250 0x7f5da00716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.659+0000 7f5d97fff640 1 -- 172.21.15.60:0/2315313016 >> v1:172.21.15.100:6789/0 conn(0x7f5da00b3f60 legacy=0x7f5da015a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.659+0000 7f5d97fff640 1 -- 172.21.15.60:0/2315313016 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da015b2f0 con 0x7f5da006f030 2024-04-25T10:13:27.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.659+0000 7f5d97fff640 1 --2- 172.21.15.60:0/2315313016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da006f030 0x7f5da00b2e00 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009870 tx=0x7f5d9001d5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0562b0990f10f50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5d977fe640 1 -- 172.21.15.60:0/2315313016 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900b1070 con 0x7f5da006f030 2024-04-25T10:13:27.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5d977fe640 1 -- 172.21.15.60:0/2315313016 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d900ac600 con 0x7f5da006f030 2024-04-25T10:13:27.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5d977fe640 1 -- 172.21.15.60:0/2315313016 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900ac9d0 con 0x7f5da006f030 2024-04-25T10:13:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5da6ca0640 1 -- 172.21.15.60:0/2315313016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da006f030 msgr2=0x7f5da00b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5da6ca0640 1 --2- 172.21.15.60:0/2315313016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da006f030 0x7f5da00b2e00 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009870 tx=0x7f5d9001d5a0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5da6ca0640 1 -- 172.21.15.60:0/2315313016 shutdown_connections 2024-04-25T10:13:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5da6ca0640 1 --2- 172.21.15.60:0/2315313016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da006f030 0x7f5da00b2e00 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5da6ca0640 1 --2- 172.21.15.60:0/2315313016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5da0071250 0x7f5da00716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5da6ca0640 1 -- 172.21.15.60:0/2315313016 >> 172.21.15.60:0/2315313016 conn(0x7f5da00670b0 msgr2=0x7f5da006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5da6ca0640 1 -- 172.21.15.60:0/2315313016 shutdown_connections 2024-04-25T10:13:27.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5da6ca0640 1 -- 172.21.15.60:0/2315313016 wait complete. 2024-04-25T10:13:27.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5da6ca0640 1 Processor -- start 2024-04-25T10:13:27.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5da6ca0640 1 -- start start 2024-04-25T10:13:27.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5da6ca0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5da006f030 0x7f5da0148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5da6ca0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0071250 0x7f5da0148340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5da6ca0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5da00b3f60 0x7f5da01585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5da6ca0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da015b5a0 con 0x7f5da0071250 2024-04-25T10:13:27.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5da6ca0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da015b720 con 0x7f5da00b3f60 2024-04-25T10:13:27.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5da6ca0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da015b8a0 con 0x7f5da006f030 2024-04-25T10:13:27.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.663+0000 7f5d97fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0071250 0x7f5da0148340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5d97fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0071250 0x7f5da0148340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55212/0 (socket says 172.21.15.60:55212) 2024-04-25T10:13:27.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5d97fff640 1 -- 172.21.15.60:0/745926236 learned_addr learned my addr 172.21.15.60:0/745926236 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:27.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5da4a3c640 1 --2- 172.21.15.60:0/745926236 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5da006f030 0x7f5da0148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5da523d640 1 --2- 172.21.15.60:0/745926236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5da00b3f60 0x7f5da01585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5d97fff640 1 -- 172.21.15.60:0/745926236 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5da006f030 msgr2=0x7f5da0148fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5d97fff640 1 --2- 172.21.15.60:0/745926236 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5da006f030 0x7f5da0148fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5d97fff640 1 -- 172.21.15.60:0/745926236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5da00b3f60 msgr2=0x7f5da01585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5d97fff640 1 --2- 172.21.15.60:0/745926236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5da00b3f60 0x7f5da01585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5d97fff640 1 -- 172.21.15.60:0/745926236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da014cf80 con 0x7f5da0071250 2024-04-25T10:13:27.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5d97fff640 1 --2- 172.21.15.60:0/745926236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0071250 0x7f5da0148340 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f5d900af040 tx=0x7f5d9001a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5d957fa640 1 -- 172.21.15.60:0/745926236 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900b1040 con 0x7f5da0071250 2024-04-25T10:13:27.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5d957fa640 1 -- 172.21.15.60:0/745926236 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d900ba070 con 0x7f5da0071250 2024-04-25T10:13:27.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5d957fa640 1 -- 172.21.15.60:0/745926236 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900b58a0 con 0x7f5da0071250 2024-04-25T10:13:27.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5da6ca0640 1 -- 172.21.15.60:0/745926236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da01593c0 con 0x7f5da0071250 2024-04-25T10:13:27.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5da6ca0640 1 -- 172.21.15.60:0/745926236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da0158e00 con 0x7f5da0071250 2024-04-25T10:13:27.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5d957fa640 1 -- 172.21.15.60:0/745926236 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d900ac4c0 con 0x7f5da0071250 2024-04-25T10:13:27.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5d957fa640 1 --2- 172.21.15.60:0/745926236 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5d8c066f60 0x7f5d8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5d957fa640 1 -- 172.21.15.60:0/745926236 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f5d9011eda0 con 0x7f5da0071250 2024-04-25T10:13:27.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5da4a3c640 1 --2- 172.21.15.60:0/745926236 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5d8c066f60 0x7f5d8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.667+0000 7f5da9e0d000 1 Processor -- start 2024-04-25T10:13:27.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 -- start start 2024-04-25T10:13:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da4a3c640 1 --2- 172.21.15.60:0/745926236 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5d8c066f60 0x7f5d8c069420 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5da014f200 tx=0x7f5d9c009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e5e30 0x557e557e6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e557e6750 0x557e5579bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e5579c4b0 0x557e5579c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e555e4ff0 con 0x557e557e5e30 2024-04-25T10:13:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e5556d150 con 0x557e557e6750 2024-04-25T10:13:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e555e1090 con 0x557e5579c4b0 2024-04-25T10:13:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da4a3c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e5e30 0x557e557e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da4a3c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e5e30 0x557e557e6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55222/0 (socket says 172.21.15.60:55222) 2024-04-25T10:13:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da4a3c640 1 -- 172.21.15.60:0/1614126189 learned_addr learned my addr 172.21.15.60:0/1614126189 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5d97fff640 1 --2- 172.21.15.60:0/1614126189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e557e6750 0x557e5579bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da523d640 1 --2- 172.21.15.60:0/1614126189 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e5579c4b0 0x557e5579c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da4a3c640 1 -- 172.21.15.60:0/1614126189 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e5579c4b0 msgr2=0x557e5579c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da4a3c640 1 --2- 172.21.15.60:0/1614126189 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e5579c4b0 0x557e5579c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da4a3c640 1 -- 172.21.15.60:0/1614126189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e557e6750 msgr2=0x557e5579bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da4a3c640 1 --2- 172.21.15.60:0/1614126189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e557e6750 0x557e5579bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da4a3c640 1 -- 172.21.15.60:0/1614126189 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e5577b0d0 con 0x557e557e5e30 2024-04-25T10:13:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5d97fff640 1 --2- 172.21.15.60:0/1614126189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e557e6750 0x557e5579bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da4a3c640 1 --2- 172.21.15.60:0/1614126189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e5e30 0x557e557e6210 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00eba0 tx=0x7f5d9c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50b5b1644e7f0f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5d867fc640 1 -- 172.21.15.60:0/1614126189 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c01b040 con 0x557e557e5e30 2024-04-25T10:13:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5d867fc640 1 -- 172.21.15.60:0/1614126189 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9c024070 con 0x557e557e5e30 2024-04-25T10:13:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5d867fc640 1 -- 172.21.15.60:0/1614126189 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c015cc0 con 0x557e557e5e30 2024-04-25T10:13:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 -- 172.21.15.60:0/1614126189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e5e30 msgr2=0x557e557e6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/1614126189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e5e30 0x557e557e6210 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00eba0 tx=0x7f5d9c011000 comp rx=0 tx=0).stop 2024-04-25T10:13:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 -- 172.21.15.60:0/1614126189 shutdown_connections 2024-04-25T10:13:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/1614126189 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e5579c4b0 0x557e5579c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/1614126189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e557e6750 0x557e5579bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/1614126189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e5e30 0x557e557e6210 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 -- 172.21.15.60:0/1614126189 >> 172.21.15.60:0/1614126189 conn(0x557e55794020 msgr2=0x557e55796440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 -- 172.21.15.60:0/1614126189 shutdown_connections 2024-04-25T10:13:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 -- 172.21.15.60:0/1614126189 wait complete. 2024-04-25T10:13:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 -- start start 2024-04-25T10:13:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 Processor -- start 2024-04-25T10:13:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e5579c4b0 0x557e557e57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e557e5e30 0x557e557a2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e6750 0x557e557a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.671+0000 7f5da9e0d000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e555e9620 con 0x557e557e6750 2024-04-25T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e5559f690 con 0x557e557e5e30 2024-04-25T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e5556da40 con 0x557e5579c4b0 2024-04-25T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 10 client.0 _ll_register_callbacks cb 0x557e55782e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da523d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e6750 0x557e557a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da523d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e6750 0x557e557a28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55238/0 (socket says 172.21.15.60:55238) 2024-04-25T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da523d640 1 -- 172.21.15.60:0/745565043 learned_addr learned my addr 172.21.15.60:0/745565043 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da4a3c640 1 --2- 172.21.15.60:0/745565043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e5579c4b0 0x557e557e57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da523d640 1 -- 172.21.15.60:0/745565043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e5579c4b0 msgr2=0x557e557e57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da523d640 1 --2- 172.21.15.60:0/745565043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e5579c4b0 0x557e557e57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5d97fff640 1 --2- 172.21.15.60:0/745565043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e557e5e30 0x557e557a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da523d640 1 -- 172.21.15.60:0/745565043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e557e5e30 msgr2=0x557e557a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da523d640 1 --2- 172.21.15.60:0/745565043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e557e5e30 0x557e557a2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da523d640 1 -- 172.21.15.60:0/745565043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e556921c0 con 0x557e557e6750 2024-04-25T10:13:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da523d640 1 --2- 172.21.15.60:0/745565043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e6750 0x557e557a28a0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009fd0 tx=0x7f5d98007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabb40fbec651d49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5d7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:13:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5d7f7fe640 1 -- 172.21.15.60:0/745565043 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d980155f0 con 0x557e557e6750 2024-04-25T10:13:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5d7f7fe640 1 -- 172.21.15.60:0/745565043 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f5d98015790 con 0x557e557e6750 2024-04-25T10:13:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 -- 172.21.15.60:0/745565043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d9c024070 con 0x557e557e6750 2024-04-25T10:13:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5d7f7fe640 1 -- 172.21.15.60:0/745565043 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d98007e30 con 0x557e557e6750 2024-04-25T10:13:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 10 client.8364 fetch_fsmap learned FSMap version 68 2024-04-25T10:13:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 -- 172.21.15.60:0/745565043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x557e5568e170 con 0x557e557e6750 2024-04-25T10:13:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5d7f7fe640 1 -- 172.21.15.60:0/745565043 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9c024070 con 0x557e557e6750 2024-04-25T10:13:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5d7f7fe640 1 -- 172.21.15.60:0/745565043 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5d98025080 con 0x557e557e6750 2024-04-25T10:13:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 10 client.8364 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:13:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 10 client.8364 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:13:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 20 client.8364 populate_metadata read hostname 'smithi060' 2024-04-25T10:13:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/745565043 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x557e557a3640 0x557e557f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 4 client.8364 mds_command: new command op to 8256 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 -- 172.21.15.60:0/745565043 --> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e55580c60 con 0x557e557a3640 2024-04-25T10:13:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da4a3c640 1 --2- 172.21.15.60:0/745565043 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x557e557a3640 0x557e557f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5d7f7fe640 10 client.8364 ms_handle_connect on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da4a3c640 1 --2- 172.21.15.60:0/745565043 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x557e557a3640 0x557e557f80f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d06a288891796da server_cookie=edf9084d393f937e in_seq=0 out_seq=0 2024-04-25T10:13:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5d7f7fe640 1 -- 172.21.15.60:0/745565043 <== mds.0 v2:172.21.15.60:6833/3807131024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557e55580c60 con 0x557e557a3640 2024-04-25T10:13:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5d7f7fe640 10 client.8364 handle_command_reply: tid=0 2024-04-25T10:13:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 client.8364 shutdown 2024-04-25T10:13:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 -- 172.21.15.60:0/745565043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e6750 msgr2=0x557e557a28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/745565043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e6750 0x557e557a28a0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009fd0 tx=0x7f5d98007680 comp rx=0 tx=0).stop 2024-04-25T10:13:27.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 -- 172.21.15.60:0/745565043 shutdown_connections 2024-04-25T10:13:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/745565043 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x557e557a3640 0x557e557f80f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/745565043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e6750 0x557e557a28a0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/745565043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e557e5e30 0x557e557a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/745565043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e5579c4b0 0x557e557e57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5d7f7fe640 0 client.8364 ms_handle_reset on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 -- 172.21.15.60:0/745565043 >> 172.21.15.60:0/745565043 conn(0x557e55794020 msgr2=0x557e5579d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 -- 172.21.15.60:0/745565043 shutdown_connections 2024-04-25T10:13:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 1 -- 172.21.15.60:0/745565043 wait complete. 2024-04-25T10:13:27.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.675+0000 7f5da9e0d000 20 client.8364 trim_cache size 0 max 16384 2024-04-25T10:13:27.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 Processor -- start 2024-04-25T10:13:27.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 -- start start 2024-04-25T10:13:27.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e55794020 0x557e557fa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e5579c4b0 0x557e557fa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e557e5e30 0x557e557f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0157650 con 0x557e55794020 2024-04-25T10:13:27.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0156830 con 0x557e5579c4b0 2024-04-25T10:13:27.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0155c30 con 0x557e557e5e30 2024-04-25T10:13:27.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da4a3c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e55794020 0x557e557fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da4a3c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e55794020 0x557e557fa110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55240/0 (socket says 172.21.15.60:55240) 2024-04-25T10:13:27.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da4a3c640 1 -- 172.21.15.60:0/1622085006 learned_addr learned my addr 172.21.15.60:0/1622085006 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:27.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da4a3c640 1 -- 172.21.15.60:0/1622085006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e557e5e30 msgr2=0x557e557f32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da4a3c640 1 --2- 172.21.15.60:0/1622085006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e557e5e30 0x557e557f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da4a3c640 1 -- 172.21.15.60:0/1622085006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e5579c4b0 msgr2=0x557e557fa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da4a3c640 1 --2- 172.21.15.60:0/1622085006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e5579c4b0 0x557e557fa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da4a3c640 1 -- 172.21.15.60:0/1622085006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e5562bde0 con 0x557e55794020 2024-04-25T10:13:27.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da4a3c640 1 --2- 172.21.15.60:0/1622085006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e55794020 0x557e557fa110 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00e5a0 tx=0x7f5d9c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b941c18834569a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5d85ffb640 1 -- 172.21.15.60:0/1622085006 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c032b80 con 0x557e55794020 2024-04-25T10:13:27.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5d85ffb640 1 -- 172.21.15.60:0/1622085006 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9c024040 con 0x557e55794020 2024-04-25T10:13:27.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5d85ffb640 1 -- 172.21.15.60:0/1622085006 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c004040 con 0x557e55794020 2024-04-25T10:13:27.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 -- 172.21.15.60:0/1622085006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e55794020 msgr2=0x557e557fa110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/1622085006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e55794020 0x557e557fa110 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00e5a0 tx=0x7f5d9c013000 comp rx=0 tx=0).stop 2024-04-25T10:13:27.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 -- 172.21.15.60:0/1622085006 shutdown_connections 2024-04-25T10:13:27.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/1622085006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e557e5e30 0x557e557f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/1622085006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e5579c4b0 0x557e557fa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/1622085006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e55794020 0x557e557fa110 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 -- 172.21.15.60:0/1622085006 >> 172.21.15.60:0/1622085006 conn(0x557e55793b00 msgr2=0x557e557f2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 -- 172.21.15.60:0/1622085006 shutdown_connections 2024-04-25T10:13:27.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.691+0000 7f5da9e0d000 1 -- 172.21.15.60:0/1622085006 wait complete. 2024-04-25T10:13:27.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 -- start start 2024-04-25T10:13:27.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 Processor -- start 2024-04-25T10:13:27.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e55794020 0x557e557b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e5579c4b0 0x557e557b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e5e30 0x557e557f9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e5557fe00 con 0x557e557e5e30 2024-04-25T10:13:27.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0156230 con 0x557e5579c4b0 2024-04-25T10:13:27.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e555e4300 con 0x557e55794020 2024-04-25T10:13:27.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 10 client.0 _ll_register_callbacks cb 0x557e55763040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:27.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da523d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e5e30 0x557e557f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da523d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e5e30 0x557e557f9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55242/0 (socket says 172.21.15.60:55242) 2024-04-25T10:13:27.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da523d640 1 -- 172.21.15.60:0/161387188 learned_addr learned my addr 172.21.15.60:0/161387188 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:27.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5d97fff640 1 --2- 172.21.15.60:0/161387188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e5579c4b0 0x557e557b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da523d640 1 -- 172.21.15.60:0/161387188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e55794020 msgr2=0x557e557b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da4a3c640 1 --2- 172.21.15.60:0/161387188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e55794020 0x557e557b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da523d640 1 --2- 172.21.15.60:0/161387188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e55794020 0x557e557b2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da523d640 1 -- 172.21.15.60:0/161387188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e5579c4b0 msgr2=0x557e557b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da523d640 1 --2- 172.21.15.60:0/161387188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e5579c4b0 0x557e557b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da523d640 1 -- 172.21.15.60:0/161387188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e555868e0 con 0x557e557e5e30 2024-04-25T10:13:27.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5d97fff640 1 --2- 172.21.15.60:0/161387188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e5579c4b0 0x557e557b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:27.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da523d640 1 --2- 172.21.15.60:0/161387188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e5e30 0x557e557f9140 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f5d98008010 tx=0x7f5d98004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60f7443e8fa25c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5d7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:13:27.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5d7f7fe640 1 -- 172.21.15.60:0/161387188 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9802e630 con 0x557e557e5e30 2024-04-25T10:13:27.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5d7f7fe640 1 -- 172.21.15.60:0/161387188 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f5d9801fb40 con 0x557e557e5e30 2024-04-25T10:13:27.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 -- 172.21.15.60:0/161387188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d9c024040 con 0x557e557e5e30 2024-04-25T10:13:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5d7f7fe640 1 -- 172.21.15.60:0/161387188 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d98007000 con 0x557e557e5e30 2024-04-25T10:13:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 10 client.8370 fetch_fsmap learned FSMap version 68 2024-04-25T10:13:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 -- 172.21.15.60:0/161387188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x557e5563e8f0 con 0x557e557e5e30 2024-04-25T10:13:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5d7f7fe640 1 -- 172.21.15.60:0/161387188 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9c024040 con 0x557e557e5e30 2024-04-25T10:13:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5d7f7fe640 1 -- 172.21.15.60:0/161387188 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5d9801f410 con 0x557e557e5e30 2024-04-25T10:13:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 10 client.8370 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:13:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 10 client.8370 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:13:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 20 client.8370 populate_metadata read hostname 'smithi060' 2024-04-25T10:13:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/161387188 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x557e557e6750 0x557e5584dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 4 client.8370 mds_command: new command op to 8256 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:13:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 -- 172.21.15.60:0/161387188 --> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557e555f2530 con 0x557e557e6750 2024-04-25T10:13:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da4a3c640 1 --2- 172.21.15.60:0/161387188 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x557e557e6750 0x557e5584dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da4a3c640 1 --2- 172.21.15.60:0/161387188 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x557e557e6750 0x557e5584dfe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84b7ec717b2ac0fd server_cookie=e0793d8aed8d61ec in_seq=0 out_seq=0 2024-04-25T10:13:27.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5d7f7fe640 10 client.8370 ms_handle_connect on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:27.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5d7f7fe640 1 -- 172.21.15.60:0/161387188 <== mds.0 v2:172.21.15.60:6833/3807131024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x557e555f2530 con 0x557e557e6750 2024-04-25T10:13:27.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5d7f7fe640 10 client.8370 handle_command_reply: tid=0 2024-04-25T10:13:27.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 client.8370 shutdown 2024-04-25T10:13:27.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 -- 172.21.15.60:0/161387188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e5e30 msgr2=0x557e557f9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/161387188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e5e30 0x557e557f9140 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f5d98008010 tx=0x7f5d98004b60 comp rx=0 tx=0).stop 2024-04-25T10:13:27.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 -- 172.21.15.60:0/161387188 shutdown_connections 2024-04-25T10:13:27.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/161387188 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x557e557e6750 0x557e5584dfe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/161387188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557e557e5e30 0x557e557f9140 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/161387188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557e5579c4b0 0x557e557b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 --2- 172.21.15.60:0/161387188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557e55794020 0x557e557b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5d7f7fe640 0 client.8370 ms_handle_reset on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:27.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 -- 172.21.15.60:0/161387188 >> 172.21.15.60:0/161387188 conn(0x557e55793b00 msgr2=0x557e557f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 -- 172.21.15.60:0/161387188 shutdown_connections 2024-04-25T10:13:27.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 1 -- 172.21.15.60:0/161387188 wait complete. 2024-04-25T10:13:27.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.695+0000 7f5da9e0d000 20 client.8370 trim_cache size 0 max 16384 2024-04-25T10:13:27.708 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:13:27.708 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:13:27.708 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T10:13:27.708 INFO:teuthology.orchestra.run.smithi060.stdout: "d639c8cf-e1f7-4d15-9ca6-a9735785f374": { 2024-04-25T10:13:27.708 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/dir", 2024-04-25T10:13:27.708 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "d639c8cf-e1f7-4d15-9ca6-a9735785f374", 2024-04-25T10:13:27.708 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,repair" 2024-04-25T10:13:27.708 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:13:27.709 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:13:27.709 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:13:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.699+0000 7f5da6ca0640 1 -- 172.21.15.60:0/745926236 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5d8c066f60 msgr2=0x7f5d8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.699+0000 7f5da6ca0640 1 --2- 172.21.15.60:0/745926236 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5d8c066f60 0x7f5d8c069420 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5da014f200 tx=0x7f5d9c009060 comp rx=0 tx=0).stop 2024-04-25T10:13:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.699+0000 7f5da6ca0640 1 -- 172.21.15.60:0/745926236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0071250 msgr2=0x7f5da0148340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.699+0000 7f5da6ca0640 1 --2- 172.21.15.60:0/745926236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0071250 0x7f5da0148340 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f5d900af040 tx=0x7f5d9001a340 comp rx=0 tx=0).stop 2024-04-25T10:13:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.699+0000 7f5da6ca0640 1 -- 172.21.15.60:0/745926236 shutdown_connections 2024-04-25T10:13:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.699+0000 7f5da6ca0640 1 --2- 172.21.15.60:0/745926236 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5d8c066f60 0x7f5d8c069420 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.699+0000 7f5da6ca0640 1 --2- 172.21.15.60:0/745926236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5da00b3f60 0x7f5da01585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.699+0000 7f5da6ca0640 1 --2- 172.21.15.60:0/745926236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0071250 0x7f5da0148340 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.699+0000 7f5da6ca0640 1 --2- 172.21.15.60:0/745926236 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5da006f030 0x7f5da0148fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.699+0000 7f5da6ca0640 1 -- 172.21.15.60:0/745926236 >> 172.21.15.60:0/745926236 conn(0x7f5da00670b0 msgr2=0x7f5da00cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.699+0000 7f5da6ca0640 1 -- 172.21.15.60:0/745926236 shutdown_connections 2024-04-25T10:13:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:27.699+0000 7f5da6ca0640 1 -- 172.21.15.60:0/745926236 wait complete. 2024-04-25T10:13:27.718 INFO:tasks.cephfs.filesystem:scrub status for tag:d639c8cf-e1f7-4d15-9ca6-a9735785f374 - {'path': '/dir', 'tag': 'd639c8cf-e1f7-4d15-9ca6-a9735785f374', 'options': 'recursive,repair'} 2024-04-25T10:13:47.525 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:47.528 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:57.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:57.719 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:13:57.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.851+0000 7f83e35bc640 1 Processor -- start 2024-04-25T10:13:57.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.851+0000 7f83e35bc640 1 -- start start 2024-04-25T10:13:57.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e35bc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83dc15db10 0x7f83dc0b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e35bc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83dc0aec10 0x7f83dc0b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e35bc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f83dc0cfcf0 0x7f83dc161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e35bc640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83dc0accc0 con 0x7f83dc0aec10 2024-04-25T10:13:57.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e35bc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83dc15def0 con 0x7f83dc0cfcf0 2024-04-25T10:13:57.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e35bc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc071630 con 0x7f83dc15db10 2024-04-25T10:13:57.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e0b57640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83dc0aec10 0x7f83dc0b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e0b57640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83dc0aec10 0x7f83dc0b3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41620/0 (socket says 172.21.15.60:41620) 2024-04-25T10:13:57.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e0b57640 1 -- 172.21.15.60:0/2670454234 learned_addr learned my addr 172.21.15.60:0/2670454234 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:57.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e1358640 1 --2- 172.21.15.60:0/2670454234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83dc15db10 0x7f83dc0b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e1b59640 1 --2- 172.21.15.60:0/2670454234 >> v2:172.21.15.100:3300/0 conn(0x7f83dc0cfcf0 0x7f83dc161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e0b57640 1 -- 172.21.15.60:0/2670454234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83dc15db10 msgr2=0x7f83dc0b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e0b57640 1 --2- 172.21.15.60:0/2670454234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83dc15db10 0x7f83dc0b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e0b57640 1 -- 172.21.15.60:0/2670454234 >> v2:172.21.15.100:3300/0 conn(0x7f83dc0cfcf0 msgr2=0x7f83dc161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e0b57640 1 --2- 172.21.15.60:0/2670454234 >> v2:172.21.15.100:3300/0 conn(0x7f83dc0cfcf0 0x7f83dc161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e0b57640 1 -- 172.21.15.60:0/2670454234 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc071910 con 0x7f83dc0aec10 2024-04-25T10:13:57.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e1b59640 1 --2- 172.21.15.60:0/2670454234 >> v2:172.21.15.100:3300/0 conn(0x7f83dc0cfcf0 0x7f83dc161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:57.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e1358640 1 --2- 172.21.15.60:0/2670454234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83dc15db10 0x7f83dc0b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:57.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83e0b57640 1 --2- 172.21.15.60:0/2670454234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83dc0aec10 0x7f83dc0b3420 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0b4ad0 tx=0x7f83d801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab29f407f2035f13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83d3fff640 1 -- 172.21.15.60:0/2670454234 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d80b0070 con 0x7f83dc0aec10 2024-04-25T10:13:57.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.855+0000 7f83d3fff640 1 -- 172.21.15.60:0/2670454234 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d80ab600 con 0x7f83dc0aec10 2024-04-25T10:13:57.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83d3fff640 1 -- 172.21.15.60:0/2670454234 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d80ab9d0 con 0x7f83dc0aec10 2024-04-25T10:13:57.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 -- 172.21.15.60:0/2670454234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83dc0aec10 msgr2=0x7f83dc0b3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 --2- 172.21.15.60:0/2670454234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83dc0aec10 0x7f83dc0b3420 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0b4ad0 tx=0x7f83d801c260 comp rx=0 tx=0).stop 2024-04-25T10:13:57.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 -- 172.21.15.60:0/2670454234 shutdown_connections 2024-04-25T10:13:57.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 --2- 172.21.15.60:0/2670454234 >> v2:172.21.15.100:3300/0 conn(0x7f83dc0cfcf0 0x7f83dc161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 --2- 172.21.15.60:0/2670454234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83dc0aec10 0x7f83dc0b3420 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 --2- 172.21.15.60:0/2670454234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83dc15db10 0x7f83dc0b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 -- 172.21.15.60:0/2670454234 >> 172.21.15.60:0/2670454234 conn(0x7f83dc0670b0 msgr2=0x7f83dc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 -- 172.21.15.60:0/2670454234 shutdown_connections 2024-04-25T10:13:57.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 -- 172.21.15.60:0/2670454234 wait complete. 2024-04-25T10:13:57.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 Processor -- start 2024-04-25T10:13:57.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 -- start start 2024-04-25T10:13:57.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83dc0aec10 0x7f83dc0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83dc0cfcf0 0x7f83dc151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83dc15db10 0x7f83dc151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc0713d0 con 0x7f83dc0cfcf0 2024-04-25T10:13:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc071250 con 0x7f83dc15db10 2024-04-25T10:13:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc0ad110 con 0x7f83dc0aec10 2024-04-25T10:13:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e0b57640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83dc0cfcf0 0x7f83dc151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e0b57640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83dc0cfcf0 0x7f83dc151930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41636/0 (socket says 172.21.15.60:41636) 2024-04-25T10:13:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e0b57640 1 -- 172.21.15.60:0/1194297529 learned_addr learned my addr 172.21.15.60:0/1194297529 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e1b59640 1 --2- 172.21.15.60:0/1194297529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83dc15db10 0x7f83dc151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e1358640 1 --2- 172.21.15.60:0/1194297529 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83dc0aec10 0x7f83dc0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e0b57640 1 -- 172.21.15.60:0/1194297529 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83dc0aec10 msgr2=0x7f83dc0cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e0b57640 1 --2- 172.21.15.60:0/1194297529 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83dc0aec10 0x7f83dc0cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e0b57640 1 -- 172.21.15.60:0/1194297529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83dc15db10 msgr2=0x7f83dc151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e0b57640 1 --2- 172.21.15.60:0/1194297529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83dc15db10 0x7f83dc151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e0b57640 1 -- 172.21.15.60:0/1194297529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc152790 con 0x7f83dc0cfcf0 2024-04-25T10:13:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e0b57640 1 --2- 172.21.15.60:0/1194297529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83dc0cfcf0 0x7f83dc151930 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f83d80ae040 tx=0x7f83d8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83d1ffb640 1 -- 172.21.15.60:0/1194297529 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d80b9070 con 0x7f83dc0cfcf0 2024-04-25T10:13:57.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83d1ffb640 1 -- 172.21.15.60:0/1194297529 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d80b4600 con 0x7f83dc0cfcf0 2024-04-25T10:13:57.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 -- 172.21.15.60:0/1194297529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83dc153070 con 0x7f83dc0cfcf0 2024-04-25T10:13:57.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83d1ffb640 1 -- 172.21.15.60:0/1194297529 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d80b0040 con 0x7f83dc0cfcf0 2024-04-25T10:13:57.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.859+0000 7f83e35bc640 1 -- 172.21.15.60:0/1194297529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83dc153630 con 0x7f83dc0cfcf0 2024-04-25T10:13:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83d1ffb640 1 -- 172.21.15.60:0/1194297529 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83d80bc070 con 0x7f83dc0cfcf0 2024-04-25T10:13:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83d1ffb640 1 --2- 172.21.15.60:0/1194297529 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83c0066f60 0x7f83c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83d1ffb640 1 -- 172.21.15.60:0/1194297529 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f83d811e150 con 0x7f83dc0cfcf0 2024-04-25T10:13:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e1358640 1 --2- 172.21.15.60:0/1194297529 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83c0066f60 0x7f83c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e6729000 1 Processor -- start 2024-04-25T10:13:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e6729000 1 -- start start 2024-04-25T10:13:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e1358640 1 --2- 172.21.15.60:0/1194297529 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83c0066f60 0x7f83c0069420 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f83cc006fd0 tx=0x7f83cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e6729000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aefe9d0 0x56552aefedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e6729000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aeff2f0 0x56552aeb4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e6729000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeb50a0 0x56552aeb5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e6729000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56552ad01ff0 con 0x56552aeff2f0 2024-04-25T10:13:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e6729000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56552ac8a150 con 0x56552aefe9d0 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e6729000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56552acfe090 con 0x56552aeb50a0 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e0b57640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aeff2f0 0x56552aeb4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e0b57640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aeff2f0 0x56552aeb4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41644/0 (socket says 172.21.15.60:41644) 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e0b57640 1 -- 172.21.15.60:0/1839205902 learned_addr learned my addr 172.21.15.60:0/1839205902 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e1358640 1 --2- 172.21.15.60:0/1839205902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aefe9d0 0x56552aefedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e1b59640 1 --2- 172.21.15.60:0/1839205902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeb50a0 0x56552aeb5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e0b57640 1 -- 172.21.15.60:0/1839205902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeb50a0 msgr2=0x56552aeb5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e0b57640 1 --2- 172.21.15.60:0/1839205902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeb50a0 0x56552aeb5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e0b57640 1 -- 172.21.15.60:0/1839205902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aefe9d0 msgr2=0x56552aefedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e0b57640 1 --2- 172.21.15.60:0/1839205902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aefe9d0 0x56552aefedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e0b57640 1 -- 172.21.15.60:0/1839205902 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56552ae980d0 con 0x56552aeff2f0 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.863+0000 7f83e1b59640 1 --2- 172.21.15.60:0/1839205902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeb50a0 0x56552aeb5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e0b57640 1 --2- 172.21.15.60:0/1839205902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aeff2f0 0x56552aeb4b60 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f83d80eafa0 tx=0x7f83d80ef240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07c4dc95fbfff17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83c6ffd640 1 -- 172.21.15.60:0/1839205902 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d80f2c90 con 0x56552aeff2f0 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83c6ffd640 1 -- 172.21.15.60:0/1839205902 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d80f7250 con 0x56552aeff2f0 2024-04-25T10:13:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83c6ffd640 1 -- 172.21.15.60:0/1839205902 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d80ab4c0 con 0x56552aeff2f0 2024-04-25T10:13:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 -- 172.21.15.60:0/1839205902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aeff2f0 msgr2=0x56552aeb4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 --2- 172.21.15.60:0/1839205902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aeff2f0 0x56552aeb4b60 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f83d80eafa0 tx=0x7f83d80ef240 comp rx=0 tx=0).stop 2024-04-25T10:13:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 -- 172.21.15.60:0/1839205902 shutdown_connections 2024-04-25T10:13:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 --2- 172.21.15.60:0/1839205902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeb50a0 0x56552aeb5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 --2- 172.21.15.60:0/1839205902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aeff2f0 0x56552aeb4b60 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 --2- 172.21.15.60:0/1839205902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aefe9d0 0x56552aefedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 -- 172.21.15.60:0/1839205902 >> 172.21.15.60:0/1839205902 conn(0x56552aeb1020 msgr2=0x56552aeb3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 -- 172.21.15.60:0/1839205902 shutdown_connections 2024-04-25T10:13:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 -- 172.21.15.60:0/1839205902 wait complete. 2024-04-25T10:13:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 -- start start 2024-04-25T10:13:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 Processor -- start 2024-04-25T10:13:57.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aeb50a0 0x56552aee37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aefe9d0 0x56552aee3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeff2f0 0x56552aee4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56552acc6250 con 0x56552aefe9d0 2024-04-25T10:13:57.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56552acbc690 con 0x56552aeb50a0 2024-04-25T10:13:57.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56552ac8aa40 con 0x56552aeff2f0 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 10 client.0 _ll_register_callbacks cb 0x56552ae9fe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e0b57640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aefe9d0 0x56552aee3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e0b57640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aefe9d0 0x56552aee3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41646/0 (socket says 172.21.15.60:41646) 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e0b57640 1 -- 172.21.15.60:0/1401246104 learned_addr learned my addr 172.21.15.60:0/1401246104 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e1358640 1 --2- 172.21.15.60:0/1401246104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aeb50a0 0x56552aee37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e1b59640 1 --2- 172.21.15.60:0/1401246104 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeff2f0 0x56552aee4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e0b57640 1 -- 172.21.15.60:0/1401246104 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeff2f0 msgr2=0x56552aee4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e0b57640 1 --2- 172.21.15.60:0/1401246104 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeff2f0 0x56552aee4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e0b57640 1 -- 172.21.15.60:0/1401246104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aeb50a0 msgr2=0x56552aee37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e0b57640 1 --2- 172.21.15.60:0/1401246104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aeb50a0 0x56552aee37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e0b57640 1 -- 172.21.15.60:0/1401246104 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56552adaf1c0 con 0x56552aefe9d0 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e0b57640 1 --2- 172.21.15.60:0/1401246104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aefe9d0 0x56552aee3d30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f83d811c690 tx=0x7f83d8009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4357b13f5b05c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83bbfff640 1 -- 172.21.15.60:0/1401246104 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d811c120 con 0x56552aefe9d0 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83bbfff640 1 -- 172.21.15.60:0/1401246104 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f83d811c2c0 con 0x56552aefe9d0 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 -- 172.21.15.60:0/1401246104 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83d80f7250 con 0x56552aefe9d0 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83bbfff640 1 -- 172.21.15.60:0/1401246104 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83d80fc050 con 0x56552aefe9d0 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 10 client.8382 fetch_fsmap learned FSMap version 68 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 -- 172.21.15.60:0/1401246104 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x56552adab170 con 0x56552aefe9d0 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83bbfff640 1 -- 172.21.15.60:0/1401246104 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d80f7250 con 0x56552aefe9d0 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83bbfff640 1 -- 172.21.15.60:0/1401246104 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f83d80ff080 con 0x56552aefe9d0 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 10 client.8382 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 10 client.8382 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 20 client.8382 populate_metadata read hostname 'smithi060' 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 --2- 172.21.15.60:0/1401246104 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x56552af0ec80 0x56552af11070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 4 client.8382 mds_command: new command op to 8256 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e6729000 1 -- 172.21.15.60:0/1401246104 --> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56552ac9dc60 con 0x56552af0ec80 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.867+0000 7f83e1358640 1 --2- 172.21.15.60:0/1401246104 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x56552af0ec80 0x56552af11070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83e1358640 1 --2- 172.21.15.60:0/1401246104 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x56552af0ec80 0x56552af11070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd7e490225e0d5e9 server_cookie=a7aef9ad825aadd8 in_seq=0 out_seq=0 2024-04-25T10:13:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83bbfff640 10 client.8382 ms_handle_connect on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83bbfff640 1 -- 172.21.15.60:0/1401246104 <== mds.0 v2:172.21.15.60:6833/3807131024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56552ac9dc60 con 0x56552af0ec80 2024-04-25T10:13:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83bbfff640 10 client.8382 handle_command_reply: tid=0 2024-04-25T10:13:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83e6729000 1 client.8382 shutdown 2024-04-25T10:13:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83e6729000 1 -- 172.21.15.60:0/1401246104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aefe9d0 msgr2=0x56552aee3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83e6729000 1 --2- 172.21.15.60:0/1401246104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aefe9d0 0x56552aee3d30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f83d811c690 tx=0x7f83d8009240 comp rx=0 tx=0).stop 2024-04-25T10:13:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83e6729000 1 -- 172.21.15.60:0/1401246104 shutdown_connections 2024-04-25T10:13:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83e6729000 1 --2- 172.21.15.60:0/1401246104 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x56552af0ec80 0x56552af11070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83e6729000 1 --2- 172.21.15.60:0/1401246104 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeff2f0 0x56552aee4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83e6729000 1 --2- 172.21.15.60:0/1401246104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aefe9d0 0x56552aee3d30 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83e6729000 1 --2- 172.21.15.60:0/1401246104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aeb50a0 0x56552aee37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83e6729000 1 -- 172.21.15.60:0/1401246104 >> 172.21.15.60:0/1401246104 conn(0x56552aeb1020 msgr2=0x56552aeb3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83bbfff640 0 client.8382 ms_handle_reset on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:57.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83e6729000 1 -- 172.21.15.60:0/1401246104 shutdown_connections 2024-04-25T10:13:57.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83e6729000 1 -- 172.21.15.60:0/1401246104 wait complete. 2024-04-25T10:13:57.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.871+0000 7f83e6729000 20 client.8382 trim_cache size 0 max 16384 2024-04-25T10:13:57.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 Processor -- start 2024-04-25T10:13:57.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 -- start start 2024-04-25T10:13:57.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aeb50a0 0x56552aeb4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aefe9d0 0x56552af0be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeff2f0 0x56552af0c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc202dc0 con 0x56552aefe9d0 2024-04-25T10:13:57.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc201fc0 con 0x56552aeb50a0 2024-04-25T10:13:57.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc1fa900 con 0x56552aeff2f0 2024-04-25T10:13:57.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e0b57640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aefe9d0 0x56552af0be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e0b57640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aefe9d0 0x56552af0be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41654/0 (socket says 172.21.15.60:41654) 2024-04-25T10:13:57.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e0b57640 1 -- 172.21.15.60:0/759829346 learned_addr learned my addr 172.21.15.60:0/759829346 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:57.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e0b57640 1 -- 172.21.15.60:0/759829346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeff2f0 msgr2=0x56552af0c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e0b57640 1 --2- 172.21.15.60:0/759829346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeff2f0 0x56552af0c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e0b57640 1 -- 172.21.15.60:0/759829346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aeb50a0 msgr2=0x56552aeb4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e0b57640 1 --2- 172.21.15.60:0/759829346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aeb50a0 0x56552aeb4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e0b57640 1 -- 172.21.15.60:0/759829346 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56552ad48de0 con 0x56552aefe9d0 2024-04-25T10:13:57.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e0b57640 1 --2- 172.21.15.60:0/759829346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aefe9d0 0x56552af0be40 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f83d80eaf50 tx=0x7f83d8119000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4573f072e78f666f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83c67fc640 1 -- 172.21.15.60:0/759829346 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d80ee910 con 0x56552aefe9d0 2024-04-25T10:13:57.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83c67fc640 1 -- 172.21.15.60:0/759829346 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d80eeda0 con 0x56552aefe9d0 2024-04-25T10:13:57.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83c67fc640 1 -- 172.21.15.60:0/759829346 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d80f7250 con 0x56552aefe9d0 2024-04-25T10:13:57.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 -- 172.21.15.60:0/759829346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aefe9d0 msgr2=0x56552af0be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 --2- 172.21.15.60:0/759829346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aefe9d0 0x56552af0be40 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f83d80eaf50 tx=0x7f83d8119000 comp rx=0 tx=0).stop 2024-04-25T10:13:57.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 -- 172.21.15.60:0/759829346 shutdown_connections 2024-04-25T10:13:57.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 --2- 172.21.15.60:0/759829346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeff2f0 0x56552af0c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 --2- 172.21.15.60:0/759829346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aefe9d0 0x56552af0be40 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 --2- 172.21.15.60:0/759829346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aeb50a0 0x56552aeb4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 -- 172.21.15.60:0/759829346 >> 172.21.15.60:0/759829346 conn(0x56552aeb1020 msgr2=0x56552af0b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 -- 172.21.15.60:0/759829346 shutdown_connections 2024-04-25T10:13:57.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 -- 172.21.15.60:0/759829346 wait complete. 2024-04-25T10:13:57.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 -- start start 2024-04-25T10:13:57.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.887+0000 7f83e6729000 1 Processor -- start 2024-04-25T10:13:57.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aeb50a0 0x56552af10a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aefe9d0 0x56552af10f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeff2f0 0x56552af114b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56552ac9ce00 con 0x56552aeb50a0 2024-04-25T10:13:57.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc150630 con 0x56552aefe9d0 2024-04-25T10:13:57.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56552ad01300 con 0x56552aeff2f0 2024-04-25T10:13:57.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 10 client.0 _ll_register_callbacks cb 0x56552ae80040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:57.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e1358640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aeb50a0 0x56552af10a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e1358640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aeb50a0 0x56552af10a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41660/0 (socket says 172.21.15.60:41660) 2024-04-25T10:13:57.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e1358640 1 -- 172.21.15.60:0/3142824172 learned_addr learned my addr 172.21.15.60:0/3142824172 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:57.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e1b59640 1 --2- 172.21.15.60:0/3142824172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeff2f0 0x56552af114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e0b57640 1 --2- 172.21.15.60:0/3142824172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aefe9d0 0x56552af10f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e1358640 1 -- 172.21.15.60:0/3142824172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeff2f0 msgr2=0x56552af114b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e1358640 1 --2- 172.21.15.60:0/3142824172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeff2f0 0x56552af114b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e1358640 1 -- 172.21.15.60:0/3142824172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aefe9d0 msgr2=0x56552af10f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e1358640 1 --2- 172.21.15.60:0/3142824172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aefe9d0 0x56552af10f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e1358640 1 -- 172.21.15.60:0/3142824172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56552aca38e0 con 0x56552aeb50a0 2024-04-25T10:13:57.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e1b59640 1 --2- 172.21.15.60:0/3142824172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeff2f0 0x56552af114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:57.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e0b57640 1 --2- 172.21.15.60:0/3142824172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aefe9d0 0x56552af10f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:57.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e1358640 1 --2- 172.21.15.60:0/3142824172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aeb50a0 0x56552af10a30 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00ee60 tx=0x7f83cc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd85ed4ff129ec6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:13:57.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83bbfff640 1 -- 172.21.15.60:0/3142824172 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc01a040 con 0x56552aeb50a0 2024-04-25T10:13:57.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83bbfff640 1 -- 172.21.15.60:0/3142824172 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f83cc0059a0 con 0x56552aeb50a0 2024-04-25T10:13:57.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 -- 172.21.15.60:0/3142824172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56552ad0f530 con 0x56552aeb50a0 2024-04-25T10:13:57.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83bbfff640 1 -- 172.21.15.60:0/3142824172 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83cc034050 con 0x56552aeb50a0 2024-04-25T10:13:57.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 10 client.8388 fetch_fsmap learned FSMap version 68 2024-04-25T10:13:57.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 -- 172.21.15.60:0/3142824172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x56552ad5b8f0 con 0x56552aeb50a0 2024-04-25T10:13:57.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83bbfff640 1 -- 172.21.15.60:0/3142824172 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56552ad0f530 con 0x56552aeb50a0 2024-04-25T10:13:57.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83bbfff640 1 -- 172.21.15.60:0/3142824172 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f83cc037020 con 0x56552aeb50a0 2024-04-25T10:13:57.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 10 client.8388 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:13:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 10 client.8388 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:13:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 20 client.8388 populate_metadata read hostname 'smithi060' 2024-04-25T10:13:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 --2- 172.21.15.60:0/3142824172 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x56552af63f40 0x56552af66330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 4 client.8388 mds_command: new command op to 8256 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:13:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 -- 172.21.15.60:0/3142824172 --> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56552ad09880 con 0x56552af63f40 2024-04-25T10:13:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e0b57640 1 --2- 172.21.15.60:0/3142824172 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x56552af63f40 0x56552af66330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e0b57640 1 --2- 172.21.15.60:0/3142824172 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x56552af63f40 0x56552af66330 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=831074574e4c7821 server_cookie=ffab434c0af57201 in_seq=0 out_seq=0 2024-04-25T10:13:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83bbfff640 10 client.8388 ms_handle_connect on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83bbfff640 1 -- 172.21.15.60:0/3142824172 <== mds.0 v2:172.21.15.60:6833/3807131024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56552ad09880 con 0x56552af63f40 2024-04-25T10:13:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83bbfff640 10 client.8388 handle_command_reply: tid=0 2024-04-25T10:13:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 client.8388 shutdown 2024-04-25T10:13:57.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 -- 172.21.15.60:0/3142824172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aeb50a0 msgr2=0x56552af10a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 --2- 172.21.15.60:0/3142824172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aeb50a0 0x56552af10a30 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00ee60 tx=0x7f83cc0046d0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 -- 172.21.15.60:0/3142824172 shutdown_connections 2024-04-25T10:13:57.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 --2- 172.21.15.60:0/3142824172 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x56552af63f40 0x56552af66330 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 --2- 172.21.15.60:0/3142824172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56552aeff2f0 0x56552af114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 --2- 172.21.15.60:0/3142824172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56552aefe9d0 0x56552af10f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 --2- 172.21.15.60:0/3142824172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56552aeb50a0 0x56552af10a30 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83bbfff640 0 client.8388 ms_handle_reset on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:57.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 -- 172.21.15.60:0/3142824172 >> 172.21.15.60:0/3142824172 conn(0x56552aeb1020 msgr2=0x56552af0bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 -- 172.21.15.60:0/3142824172 shutdown_connections 2024-04-25T10:13:57.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 1 -- 172.21.15.60:0/3142824172 wait complete. 2024-04-25T10:13:57.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e6729000 20 client.8388 trim_cache size 0 max 16384 2024-04-25T10:13:57.904 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:13:57.904 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T10:13:57.904 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T10:13:57.904 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:13:57.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e35bc640 1 -- 172.21.15.60:0/1194297529 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83c0066f60 msgr2=0x7f83c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e35bc640 1 --2- 172.21.15.60:0/1194297529 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83c0066f60 0x7f83c0069420 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f83cc006fd0 tx=0x7f83cc008040 comp rx=0 tx=0).stop 2024-04-25T10:13:57.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e35bc640 1 -- 172.21.15.60:0/1194297529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83dc0cfcf0 msgr2=0x7f83dc151930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e35bc640 1 --2- 172.21.15.60:0/1194297529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83dc0cfcf0 0x7f83dc151930 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f83d80ae040 tx=0x7f83d8019000 comp rx=0 tx=0).stop 2024-04-25T10:13:57.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e35bc640 1 -- 172.21.15.60:0/1194297529 shutdown_connections 2024-04-25T10:13:57.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e35bc640 1 --2- 172.21.15.60:0/1194297529 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f83c0066f60 0x7f83c0069420 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e35bc640 1 --2- 172.21.15.60:0/1194297529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83dc15db10 0x7f83dc151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e35bc640 1 --2- 172.21.15.60:0/1194297529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83dc0cfcf0 0x7f83dc151930 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e35bc640 1 --2- 172.21.15.60:0/1194297529 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83dc0aec10 0x7f83dc0cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e35bc640 1 -- 172.21.15.60:0/1194297529 >> 172.21.15.60:0/1194297529 conn(0x7f83dc0670b0 msgr2=0x7f83dc0cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e35bc640 1 -- 172.21.15.60:0/1194297529 shutdown_connections 2024-04-25T10:13:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:57.891+0000 7f83e35bc640 1 -- 172.21.15.60:0/1194297529 wait complete. 2024-04-25T10:13:57.915 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:13:57.915 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.027+0000 7f89922e7640 1 Processor -- start 2024-04-25T10:13:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.027+0000 7f89922e7640 1 -- start start 2024-04-25T10:13:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.027+0000 7f89922e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f898c06ec30 0x7f898c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.027+0000 7f89922e7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f898c0b37e0 0x7f898c0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.027+0000 7f89922e7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898c06f130 con 0x7f898c15a7a0 2024-04-25T10:13:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.027+0000 7f89922e7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898c0cdaa0 con 0x7f898c0b37e0 2024-04-25T10:13:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.027+0000 7f89922e7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c06d2f0 con 0x7f898c06ec30 2024-04-25T10:13:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.027+0000 7f8990884640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f898c0b37e0 0x7f898c0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.027+0000 7f898bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f898c15a7a0 0x7f898c071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58170/0 (socket says 172.21.15.60:58170) 2024-04-25T10:13:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.027+0000 7f898bfff640 1 -- 172.21.15.60:0/3753979913 learned_addr learned my addr 172.21.15.60:0/3753979913 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:13:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.027+0000 7f898b7fe640 1 --2- 172.21.15.60:0/3753979913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f898c06ec30 0x7f898c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.027+0000 7f898affd640 1 -- 172.21.15.60:0/3753979913 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2460303813 0 0) 0x7f898c06f130 con 0x7f898c15a7a0 2024-04-25T10:13:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.027+0000 7f8990884640 1 -- 172.21.15.60:0/3753979913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f898c06ec30 msgr2=0x7f898c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.027+0000 7f8990884640 1 --2- 172.21.15.60:0/3753979913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f898c06ec30 0x7f898c0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.027+0000 7f8990884640 1 -- 172.21.15.60:0/3753979913 >> v1:172.21.15.60:6789/0 conn(0x7f898c15a7a0 legacy=0x7f898c071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.027+0000 7f8990884640 1 -- 172.21.15.60:0/3753979913 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c06f410 con 0x7f898c0b37e0 2024-04-25T10:13:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f8990884640 1 --2- 172.21.15.60:0/3753979913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f898c0b37e0 0x7f898c0b3c90 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f89840097c0 tx=0x7f898401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b598db87b688d4c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f898affd640 1 -- 172.21.15.60:0/3753979913 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89840b1070 con 0x7f898c0b37e0 2024-04-25T10:13:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f898affd640 1 -- 172.21.15.60:0/3753979913 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89840ac410 con 0x7f898c0b37e0 2024-04-25T10:13:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f898affd640 1 -- 172.21.15.60:0/3753979913 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89840ac7e0 con 0x7f898c0b37e0 2024-04-25T10:13:58.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f89922e7640 1 -- 172.21.15.60:0/3753979913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f898c0b37e0 msgr2=0x7f898c0b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f89922e7640 1 --2- 172.21.15.60:0/3753979913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f898c0b37e0 0x7f898c0b3c90 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f89840097c0 tx=0x7f898401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f89922e7640 1 -- 172.21.15.60:0/3753979913 shutdown_connections 2024-04-25T10:13:58.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f89922e7640 1 --2- 172.21.15.60:0/3753979913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f898c0b37e0 0x7f898c0b3c90 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f89922e7640 1 --2- 172.21.15.60:0/3753979913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f898c06ec30 0x7f898c0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f89922e7640 1 -- 172.21.15.60:0/3753979913 >> 172.21.15.60:0/3753979913 conn(0x7f898c0b12d0 msgr2=0x7f898c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f89922e7640 1 -- 172.21.15.60:0/3753979913 shutdown_connections 2024-04-25T10:13:58.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f89922e7640 1 -- 172.21.15.60:0/3753979913 wait complete. 2024-04-25T10:13:58.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f89922e7640 1 Processor -- start 2024-04-25T10:13:58.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f89922e7640 1 -- start start 2024-04-25T10:13:58.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f89922e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f898c06ec30 0x7f898c14dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f89922e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f898c0b37e0 0x7f898c14c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f89922e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f898c15a7a0 0x7f898c14cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f89922e7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c06e8e0 con 0x7f898c06ec30 2024-04-25T10:13:58.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f89922e7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c06e760 con 0x7f898c15a7a0 2024-04-25T10:13:58.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f89922e7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c06d8b0 con 0x7f898c0b37e0 2024-04-25T10:13:58.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f898bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f898c06ec30 0x7f898c14dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f898bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f898c06ec30 0x7f898c14dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41670/0 (socket says 172.21.15.60:41670) 2024-04-25T10:13:58.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f898bfff640 1 -- 172.21.15.60:0/830669340 learned_addr learned my addr 172.21.15.60:0/830669340 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:58.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f8990884640 1 --2- 172.21.15.60:0/830669340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f898c15a7a0 0x7f898c14cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f898b7fe640 1 --2- 172.21.15.60:0/830669340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f898c0b37e0 0x7f898c14c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f898bfff640 1 -- 172.21.15.60:0/830669340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f898c0b37e0 msgr2=0x7f898c14c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f898bfff640 1 --2- 172.21.15.60:0/830669340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f898c0b37e0 0x7f898c14c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f898bfff640 1 -- 172.21.15.60:0/830669340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f898c15a7a0 msgr2=0x7f898c14cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f898bfff640 1 --2- 172.21.15.60:0/830669340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f898c15a7a0 0x7f898c14cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f898bfff640 1 -- 172.21.15.60:0/830669340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c158e20 con 0x7f898c06ec30 2024-04-25T10:13:58.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f8990884640 1 --2- 172.21.15.60:0/830669340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f898c15a7a0 0x7f898c14cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:58.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.031+0000 7f898bfff640 1 --2- 172.21.15.60:0/830669340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f898c06ec30 0x7f898c14dd90 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f897c00c7c0 tx=0x7f897c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.035+0000 7f8988ff9640 1 -- 172.21.15.60:0/830669340 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c018070 con 0x7f898c06ec30 2024-04-25T10:13:58.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.035+0000 7f8988ff9640 1 -- 172.21.15.60:0/830669340 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c00a640 con 0x7f898c06ec30 2024-04-25T10:13:58.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.035+0000 7f89922e7640 1 -- 172.21.15.60:0/830669340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898c15c080 con 0x7f898c06ec30 2024-04-25T10:13:58.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.035+0000 7f8988ff9640 1 -- 172.21.15.60:0/830669340 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c00aa30 con 0x7f898c06ec30 2024-04-25T10:13:58.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.035+0000 7f89922e7640 1 -- 172.21.15.60:0/830669340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c14d120 con 0x7f898c06ec30 2024-04-25T10:13:58.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.035+0000 7f89922e7640 1 -- 172.21.15.60:0/830669340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898c074770 con 0x7f898c06ec30 2024-04-25T10:13:58.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.035+0000 7f8988ff9640 1 -- 172.21.15.60:0/830669340 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f897c01b070 con 0x7f898c06ec30 2024-04-25T10:13:58.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.035+0000 7f8988ff9640 1 --2- 172.21.15.60:0/830669340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8978066f90 0x7f8978069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.035+0000 7f8988ff9640 1 -- 172.21.15.60:0/830669340 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f897c084670 con 0x7f898c06ec30 2024-04-25T10:13:58.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.035+0000 7f898b7fe640 1 --2- 172.21.15.60:0/830669340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8978066f90 0x7f8978069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.035+0000 7f898b7fe640 1 --2- 172.21.15.60:0/830669340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8978066f90 0x7f8978069450 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f898c14d3d0 tx=0x7f8980008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.039+0000 7f8988ff9640 1 -- 172.21.15.60:0/830669340 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f897c055640 con 0x7f898c06ec30 2024-04-25T10:13:58.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.227+0000 7f89922e7640 1 -- 172.21.15.60:0/830669340 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f898c0b3490 con 0x7f898c06ec30 2024-04-25T10:13:58.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.227+0000 7f8988ff9640 1 -- 172.21.15.60:0/830669340 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f897c054f40 con 0x7f898c06ec30 2024-04-25T10:13:58.233 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:13:58.234 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:13:22.221010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8256":{"gid":8256,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/3807131024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3807131024},{"type":"v1","addr":"172.21.15.60:6835","nonce":3807131024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:13:58.234 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 68 2024-04-25T10:13:58.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.231+0000 7f89922e7640 1 -- 172.21.15.60:0/830669340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8978066f90 msgr2=0x7f8978069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.231+0000 7f89922e7640 1 --2- 172.21.15.60:0/830669340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8978066f90 0x7f8978069450 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f898c14d3d0 tx=0x7f8980008040 comp rx=0 tx=0).stop 2024-04-25T10:13:58.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.231+0000 7f89922e7640 1 -- 172.21.15.60:0/830669340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f898c06ec30 msgr2=0x7f898c14dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.231+0000 7f89922e7640 1 --2- 172.21.15.60:0/830669340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f898c06ec30 0x7f898c14dd90 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f897c00c7c0 tx=0x7f897c00cc80 comp rx=0 tx=0).stop 2024-04-25T10:13:58.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.231+0000 7f89922e7640 1 -- 172.21.15.60:0/830669340 shutdown_connections 2024-04-25T10:13:58.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.231+0000 7f89922e7640 1 --2- 172.21.15.60:0/830669340 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8978066f90 0x7f8978069450 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.231+0000 7f89922e7640 1 --2- 172.21.15.60:0/830669340 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f898c15a7a0 0x7f898c14cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.231+0000 7f89922e7640 1 --2- 172.21.15.60:0/830669340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f898c0b37e0 0x7f898c14c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.231+0000 7f89922e7640 1 --2- 172.21.15.60:0/830669340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f898c06ec30 0x7f898c14dd90 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.231+0000 7f89922e7640 1 -- 172.21.15.60:0/830669340 >> 172.21.15.60:0/830669340 conn(0x7f898c0b12d0 msgr2=0x7f898c15e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.231+0000 7f89922e7640 1 -- 172.21.15.60:0/830669340 shutdown_connections 2024-04-25T10:13:58.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.231+0000 7f89922e7640 1 -- 172.21.15.60:0/830669340 wait complete. 2024-04-25T10:13:58.251 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T10:13:58.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.359+0000 7f23bb0d5640 1 Processor -- start 2024-04-25T10:13:58.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23bb0d5640 1 -- start start 2024-04-25T10:13:58.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23bb0d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f23b40cbcf0 0x7f23b4160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23bb0d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23b4071c30 0x7f23b415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23bb0d5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b4144a40 con 0x7f23b415fa20 2024-04-25T10:13:58.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23bb0d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b40b2dc0 con 0x7f23b40cbcf0 2024-04-25T10:13:58.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23bb0d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b415fe80 con 0x7f23b4071c30 2024-04-25T10:13:58.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23b9672640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f23b415fa20 0x7f23b415bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58174/0 (socket says 172.21.15.60:58174) 2024-04-25T10:13:58.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23b9672640 1 -- 172.21.15.60:0/3177880359 learned_addr learned my addr 172.21.15.60:0/3177880359 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:13:58.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23ab7fe640 1 -- 172.21.15.60:0/3177880359 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295310215 0 0) 0x7f23b4144a40 con 0x7f23b415fa20 2024-04-25T10:13:58.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23ab7fe640 1 -- 172.21.15.60:0/3177880359 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f239c003530 con 0x7f23b415fa20 2024-04-25T10:13:58.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23abfff640 1 --2- 172.21.15.60:0/3177880359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23b4071c30 0x7f23b415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23b8e71640 1 --2- 172.21.15.60:0/3177880359 >> v2:172.21.15.100:3300/0 conn(0x7f23b40cbcf0 0x7f23b4160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23ab7fe640 1 -- 172.21.15.60:0/3177880359 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 413918822 0 0) 0x7f239c003530 con 0x7f23b415fa20 2024-04-25T10:13:58.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23ab7fe640 1 -- 172.21.15.60:0/3177880359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23b4071c30 msgr2=0x7f23b415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23ab7fe640 1 --2- 172.21.15.60:0/3177880359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23b4071c30 0x7f23b415f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23ab7fe640 1 -- 172.21.15.60:0/3177880359 >> v2:172.21.15.100:3300/0 conn(0x7f23b40cbcf0 msgr2=0x7f23b4160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23ab7fe640 1 --2- 172.21.15.60:0/3177880359 >> v2:172.21.15.100:3300/0 conn(0x7f23b40cbcf0 0x7f23b4160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23ab7fe640 1 -- 172.21.15.60:0/3177880359 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b40cc9f0 con 0x7f23b415fa20 2024-04-25T10:13:58.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23ab7fe640 1 -- 172.21.15.60:0/3177880359 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f23ac001a00 con 0x7f23b415fa20 2024-04-25T10:13:58.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23ab7fe640 1 -- 172.21.15.60:0/3177880359 >> v1:172.21.15.60:6789/0 conn(0x7f23b415fa20 legacy=0x7f23b415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23ab7fe640 1 --2- 172.21.15.60:0/3177880359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23b4071c30 0x7f239c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23ab7fe640 1 --2- 172.21.15.60:0/3177880359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f239c003ff0 0x7f239c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23ab7fe640 1 --2- 172.21.15.60:0/3177880359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f239c004d50 0x7f239c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23ab7fe640 1 -- 172.21.15.60:0/3177880359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c00ba80 con 0x7f239c004d50 2024-04-25T10:13:58.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23ab7fe640 1 -- 172.21.15.60:0/3177880359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c00bc00 con 0x7f239c003ff0 2024-04-25T10:13:58.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23ab7fe640 1 -- 172.21.15.60:0/3177880359 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239c00bdd0 con 0x7f23b4071c30 2024-04-25T10:13:58.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23b9672640 1 --2- 172.21.15.60:0/3177880359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f239c004d50 0x7f239c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23b8e71640 1 --2- 172.21.15.60:0/3177880359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23b4071c30 0x7f239c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23abfff640 1 --2- 172.21.15.60:0/3177880359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f239c003ff0 0x7f239c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23b9672640 1 -- 172.21.15.60:0/3177880359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23b4071c30 msgr2=0x7f239c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23b9672640 1 --2- 172.21.15.60:0/3177880359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23b4071c30 0x7f239c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23b9672640 1 -- 172.21.15.60:0/3177880359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f239c003ff0 msgr2=0x7f239c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23b9672640 1 --2- 172.21.15.60:0/3177880359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f239c003ff0 0x7f239c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23b9672640 1 -- 172.21.15.60:0/3177880359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239c003530 con 0x7f239c004d50 2024-04-25T10:13:58.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.363+0000 7f23abfff640 1 --2- 172.21.15.60:0/3177880359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f239c003ff0 0x7f239c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:58.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.367+0000 7f23b9672640 1 --2- 172.21.15.60:0/3177880359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f239c004d50 0x7f239c0051d0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00a880 tx=0x7f23ac01d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad3b2533574c168 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.367+0000 7f23ab7fe640 1 -- 172.21.15.60:0/3177880359 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac00a4a0 con 0x7f239c004d50 2024-04-25T10:13:58.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.367+0000 7f23ab7fe640 1 -- 172.21.15.60:0/3177880359 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac0ad630 con 0x7f239c004d50 2024-04-25T10:13:58.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.367+0000 7f23ab7fe640 1 -- 172.21.15.60:0/3177880359 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0ada20 con 0x7f239c004d50 2024-04-25T10:13:58.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.367+0000 7f23bb0d5640 1 -- 172.21.15.60:0/3177880359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f239c004d50 msgr2=0x7f239c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.367+0000 7f23bb0d5640 1 --2- 172.21.15.60:0/3177880359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f239c004d50 0x7f239c0051d0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00a880 tx=0x7f23ac01d3a0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.367+0000 7f23b9672640 1 -- 172.21.15.60:0/3177880359 reap_dead start 2024-04-25T10:13:58.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.367+0000 7f23bb0d5640 1 -- 172.21.15.60:0/3177880359 shutdown_connections 2024-04-25T10:13:58.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.367+0000 7f23bb0d5640 1 -- 172.21.15.60:0/3177880359 >> 172.21.15.60:0/3177880359 conn(0x7f23b40670b0 msgr2=0x7f23b406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.367+0000 7f23bb0d5640 1 -- 172.21.15.60:0/3177880359 shutdown_connections 2024-04-25T10:13:58.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.367+0000 7f23bb0d5640 1 -- 172.21.15.60:0/3177880359 wait complete. 2024-04-25T10:13:58.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.367+0000 7f23bb0d5640 1 Processor -- start 2024-04-25T10:13:58.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.367+0000 7f23bb0d5640 1 -- start start 2024-04-25T10:13:58.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23bb0d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23b40cf7e0 0x7f23b41f15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23bb0d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23b40b1700 0x7f23b40b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23bb0d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23b40b0bd0 0x7f23b413bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23bb0d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40ce9a0 con 0x7f23b40cf7e0 2024-04-25T10:13:58.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23bb0d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40ccfb0 con 0x7f23b40b0bd0 2024-04-25T10:13:58.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23bb0d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40cc0d0 con 0x7f23b40b1700 2024-04-25T10:13:58.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23b8e71640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23b40cf7e0 0x7f23b41f15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23b8e71640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23b40cf7e0 0x7f23b41f15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41686/0 (socket says 172.21.15.60:41686) 2024-04-25T10:13:58.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23b8e71640 1 -- 172.21.15.60:0/484006372 learned_addr learned my addr 172.21.15.60:0/484006372 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:58.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23abfff640 1 --2- 172.21.15.60:0/484006372 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23b40b1700 0x7f23b40b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23b8e71640 1 -- 172.21.15.60:0/484006372 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23b40b1700 msgr2=0x7f23b40b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23b9672640 1 --2- 172.21.15.60:0/484006372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23b40b0bd0 0x7f23b413bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23b8e71640 1 --2- 172.21.15.60:0/484006372 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23b40b1700 0x7f23b40b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23b8e71640 1 -- 172.21.15.60:0/484006372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23b40b0bd0 msgr2=0x7f23b413bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23b8e71640 1 --2- 172.21.15.60:0/484006372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23b40b0bd0 0x7f23b413bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23b8e71640 1 -- 172.21.15.60:0/484006372 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b40afed0 con 0x7f23b40cf7e0 2024-04-25T10:13:58.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23abfff640 1 --2- 172.21.15.60:0/484006372 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23b40b1700 0x7f23b40b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:58.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23b8e71640 1 --2- 172.21.15.60:0/484006372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23b40cf7e0 0x7f23b41f15e0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f23b000f600 tx=0x7f23b000fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23a97fa640 1 -- 172.21.15.60:0/484006372 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0019020 con 0x7f23b40cf7e0 2024-04-25T10:13:58.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23bb0d5640 1 -- 172.21.15.60:0/484006372 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b41f1ea0 con 0x7f23b40cf7e0 2024-04-25T10:13:58.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23a97fa640 1 -- 172.21.15.60:0/484006372 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b0014870 con 0x7f23b40cf7e0 2024-04-25T10:13:58.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23a97fa640 1 -- 172.21.15.60:0/484006372 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0014c60 con 0x7f23b40cf7e0 2024-04-25T10:13:58.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23bb0d5640 1 -- 172.21.15.60:0/484006372 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b41f2460 con 0x7f23b40cf7e0 2024-04-25T10:13:58.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23a97fa640 1 -- 172.21.15.60:0/484006372 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23b001c020 con 0x7f23b40cf7e0 2024-04-25T10:13:58.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23a97fa640 1 --2- 172.21.15.60:0/484006372 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f23a0066f90 0x7f23a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23a97fa640 1 -- 172.21.15.60:0/484006372 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f23b0085a90 con 0x7f23b40cf7e0 2024-04-25T10:13:58.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.371+0000 7f23abfff640 1 --2- 172.21.15.60:0/484006372 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f23a0066f90 0x7f23a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 Processor -- start 2024-04-25T10:13:58.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 -- start start 2024-04-25T10:13:58.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23abfff640 1 --2- 172.21.15.60:0/484006372 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f23a0066f90 0x7f23a0069450 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f23b4141850 tx=0x7f23a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1be2e0 0x55b0bc1be6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1bec00 0x55b0bc1743a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1748e0 0x55b0bc174cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bc05bba0 con 0x55b0bc1748e0 2024-04-25T10:13:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bbfc1090 con 0x55b0bc1bec00 2024-04-25T10:13:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0bbf49160 con 0x55b0bc1be2e0 2024-04-25T10:13:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23b9672640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1748e0 0x55b0bc174cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23b9672640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1748e0 0x55b0bc174cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41690/0 (socket says 172.21.15.60:41690) 2024-04-25T10:13:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23b9672640 1 -- 172.21.15.60:0/646090685 learned_addr learned my addr 172.21.15.60:0/646090685 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23abfff640 1 --2- 172.21.15.60:0/646090685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1bec00 0x55b0bc1743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23b8e71640 1 --2- 172.21.15.60:0/646090685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1be2e0 0x55b0bc1be6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23b9672640 1 -- 172.21.15.60:0/646090685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1be2e0 msgr2=0x55b0bc1be6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23b9672640 1 --2- 172.21.15.60:0/646090685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1be2e0 0x55b0bc1be6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23b9672640 1 -- 172.21.15.60:0/646090685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1bec00 msgr2=0x55b0bc1743a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23b9672640 1 --2- 172.21.15.60:0/646090685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1bec00 0x55b0bc1743a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23b9672640 1 -- 172.21.15.60:0/646090685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0bc06ac20 con 0x55b0bc1748e0 2024-04-25T10:13:58.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23abfff640 1 --2- 172.21.15.60:0/646090685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1bec00 0x55b0bc1743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:58.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23b9672640 1 --2- 172.21.15.60:0/646090685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1748e0 0x55b0bc174cc0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f23ac098dc0 tx=0x7f23b40ccbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b2dedc6fc2a9da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f239a7fc640 1 -- 172.21.15.60:0/646090685 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac002000 con 0x55b0bc1748e0 2024-04-25T10:13:58.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f239a7fc640 1 -- 172.21.15.60:0/646090685 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac00a4a0 con 0x55b0bc1748e0 2024-04-25T10:13:58.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f239a7fc640 1 -- 172.21.15.60:0/646090685 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0019c0 con 0x55b0bc1748e0 2024-04-25T10:13:58.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 -- 172.21.15.60:0/646090685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1748e0 msgr2=0x55b0bc174cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 --2- 172.21.15.60:0/646090685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1748e0 0x55b0bc174cc0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f23ac098dc0 tx=0x7f23b40ccbc0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 -- 172.21.15.60:0/646090685 shutdown_connections 2024-04-25T10:13:58.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 --2- 172.21.15.60:0/646090685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1748e0 0x55b0bc174cc0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 --2- 172.21.15.60:0/646090685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1bec00 0x55b0bc1743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 --2- 172.21.15.60:0/646090685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1be2e0 0x55b0bc1be6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 -- 172.21.15.60:0/646090685 >> 172.21.15.60:0/646090685 conn(0x55b0bc170890 msgr2=0x55b0bc172c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 -- 172.21.15.60:0/646090685 shutdown_connections 2024-04-25T10:13:58.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 -- 172.21.15.60:0/646090685 wait complete. 2024-04-25T10:13:58.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 -- start start 2024-04-25T10:13:58.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 Processor -- start 2024-04-25T10:13:58.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1748e0 0x55b0bc1c5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1be2e0 0x55b0bc1c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1bec00 0x55b0bc1ac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bbfbd130 con 0x55b0bc1bec00 2024-04-25T10:13:58.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bbf85130 con 0x55b0bc1748e0 2024-04-25T10:13:58.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0bc07f300 con 0x55b0bc1be2e0 2024-04-25T10:13:58.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23be242000 10 client.0 _ll_register_callbacks cb 0x55b0bc15fbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:58.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23b9672640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1bec00 0x55b0bc1ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23b9672640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1bec00 0x55b0bc1ac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41702/0 (socket says 172.21.15.60:41702) 2024-04-25T10:13:58.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23b9672640 1 -- 172.21.15.60:0/1348963794 learned_addr learned my addr 172.21.15.60:0/1348963794 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:58.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.375+0000 7f23b8e71640 1 --2- 172.21.15.60:0/1348963794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1748e0 0x55b0bc1c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23b9672640 1 -- 172.21.15.60:0/1348963794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1be2e0 msgr2=0x55b0bc1c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23abfff640 1 --2- 172.21.15.60:0/1348963794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1be2e0 0x55b0bc1c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23b9672640 1 --2- 172.21.15.60:0/1348963794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1be2e0 0x55b0bc1c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23b9672640 1 -- 172.21.15.60:0/1348963794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1748e0 msgr2=0x55b0bc1c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23b9672640 1 --2- 172.21.15.60:0/1348963794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1748e0 0x55b0bc1c5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23b9672640 1 -- 172.21.15.60:0/1348963794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0bbf9e060 con 0x55b0bc1bec00 2024-04-25T10:13:58.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23b8e71640 1 --2- 172.21.15.60:0/1348963794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1748e0 0x55b0bc1c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:58.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23b9672640 1 --2- 172.21.15.60:0/1348963794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1bec00 0x55b0bc1ac090 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f23ac01ce10 tx=0x7f23ac01a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef5906c5be615ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23937fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:13:58.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23937fe640 1 -- 172.21.15.60:0/1348963794 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0c3020 con 0x55b0bc1bec00 2024-04-25T10:13:58.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23937fe640 1 -- 172.21.15.60:0/1348963794 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f23ac0b2040 con 0x55b0bc1bec00 2024-04-25T10:13:58.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 1 -- 172.21.15.60:0/1348963794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23ac00a4a0 con 0x55b0bc1bec00 2024-04-25T10:13:58.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23937fe640 1 -- 172.21.15.60:0/1348963794 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23ac0017c0 con 0x55b0bc1bec00 2024-04-25T10:13:58.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 10 client.8409 fetch_fsmap learned FSMap version 68 2024-04-25T10:13:58.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 1 -- 172.21.15.60:0/1348963794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55b0bbf627c0 con 0x55b0bc1bec00 2024-04-25T10:13:58.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23937fe640 1 -- 172.21.15.60:0/1348963794 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac00a4a0 con 0x55b0bc1bec00 2024-04-25T10:13:58.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23937fe640 1 -- 172.21.15.60:0/1348963794 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f23ac0070c0 con 0x55b0bc1bec00 2024-04-25T10:13:58.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 10 client.8409 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:13:58.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 10 client.8409 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:58.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 20 client.8409 populate_metadata read hostname 'smithi060' 2024-04-25T10:13:58.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 1 --2- 172.21.15.60:0/1348963794 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x55b0bc1ace30 0x55b0bc1ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 4 client.8409 mds_command: new command op to 8256 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:58.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 1 -- 172.21.15.60:0/1348963794 --> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0bbf5ccd0 con 0x55b0bc1ace30 2024-04-25T10:13:58.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23b8e71640 1 --2- 172.21.15.60:0/1348963794 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x55b0bc1ace30 0x55b0bc1ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23937fe640 10 client.8409 ms_handle_connect on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:58.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23b8e71640 1 --2- 172.21.15.60:0/1348963794 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x55b0bc1ace30 0x55b0bc1ad210 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3de9faad20ea0176 server_cookie=2a347b198e2a5076 in_seq=0 out_seq=0 2024-04-25T10:13:58.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23937fe640 1 -- 172.21.15.60:0/1348963794 <== mds.0 v2:172.21.15.60:6833/3807131024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b0bbf5ccd0 con 0x55b0bc1ace30 2024-04-25T10:13:58.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23937fe640 10 client.8409 handle_command_reply: tid=0 2024-04-25T10:13:58.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 1 client.8409 shutdown 2024-04-25T10:13:58.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 1 -- 172.21.15.60:0/1348963794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1bec00 msgr2=0x55b0bc1ac090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 1 --2- 172.21.15.60:0/1348963794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1bec00 0x55b0bc1ac090 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f23ac01ce10 tx=0x7f23ac01a140 comp rx=0 tx=0).stop 2024-04-25T10:13:58.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 1 -- 172.21.15.60:0/1348963794 shutdown_connections 2024-04-25T10:13:58.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 1 --2- 172.21.15.60:0/1348963794 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x55b0bc1ace30 0x55b0bc1ad210 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 1 --2- 172.21.15.60:0/1348963794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1bec00 0x55b0bc1ac090 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 1 --2- 172.21.15.60:0/1348963794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1be2e0 0x55b0bc1c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23937fe640 0 client.8409 ms_handle_reset on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:58.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 1 --2- 172.21.15.60:0/1348963794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1748e0 0x55b0bc1c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 1 -- 172.21.15.60:0/1348963794 >> 172.21.15.60:0/1348963794 conn(0x55b0bc170890 msgr2=0x55b0bc173510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 1 -- 172.21.15.60:0/1348963794 shutdown_connections 2024-04-25T10:13:58.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 1 -- 172.21.15.60:0/1348963794 wait complete. 2024-04-25T10:13:58.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.379+0000 7f23be242000 20 client.8409 trim_cache size 0 max 16384 2024-04-25T10:13:58.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23be242000 1 Processor -- start 2024-04-25T10:13:58.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23be242000 1 -- start start 2024-04-25T10:13:58.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23be242000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1748e0 0x55b0bc1ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23be242000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1be2e0 0x55b0bc1ab730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23be242000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1bec00 0x55b0bc172c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23be242000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b41fd510 con 0x55b0bc1748e0 2024-04-25T10:13:58.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23be242000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b41f55b0 con 0x55b0bc1be2e0 2024-04-25T10:13:58.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23be242000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b41fcf40 con 0x55b0bc1bec00 2024-04-25T10:13:58.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23b8e71640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1748e0 0x55b0bc1ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23b8e71640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1748e0 0x55b0bc1ab1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41712/0 (socket says 172.21.15.60:41712) 2024-04-25T10:13:58.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23b8e71640 1 -- 172.21.15.60:0/2828971472 learned_addr learned my addr 172.21.15.60:0/2828971472 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:58.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23b8e71640 1 -- 172.21.15.60:0/2828971472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1bec00 msgr2=0x55b0bc172c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23b8e71640 1 --2- 172.21.15.60:0/2828971472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1bec00 0x55b0bc172c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23b8e71640 1 -- 172.21.15.60:0/2828971472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1be2e0 msgr2=0x55b0bc1ab730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23b8e71640 1 --2- 172.21.15.60:0/2828971472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1be2e0 0x55b0bc1ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23b8e71640 1 -- 172.21.15.60:0/2828971472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0bbf7cfc0 con 0x55b0bc1748e0 2024-04-25T10:13:58.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23b8e71640 1 --2- 172.21.15.60:0/2828971472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1748e0 0x55b0bc1ab1f0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f23b0056180 tx=0x7f23b000d260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e9f0ee03ed28aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f2399ffb640 1 -- 172.21.15.60:0/2828971472 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0072070 con 0x55b0bc1748e0 2024-04-25T10:13:58.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f2399ffb640 1 -- 172.21.15.60:0/2828971472 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b005a340 con 0x55b0bc1748e0 2024-04-25T10:13:58.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f2399ffb640 1 -- 172.21.15.60:0/2828971472 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b005e900 con 0x55b0bc1748e0 2024-04-25T10:13:58.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23be242000 1 -- 172.21.15.60:0/2828971472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1748e0 msgr2=0x55b0bc1ab1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.391+0000 7f23be242000 1 --2- 172.21.15.60:0/2828971472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1748e0 0x55b0bc1ab1f0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f23b0056180 tx=0x7f23b000d260 comp rx=0 tx=0).stop 2024-04-25T10:13:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 -- 172.21.15.60:0/2828971472 shutdown_connections 2024-04-25T10:13:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 --2- 172.21.15.60:0/2828971472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1bec00 0x55b0bc172c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 --2- 172.21.15.60:0/2828971472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1be2e0 0x55b0bc1ab730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 --2- 172.21.15.60:0/2828971472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1748e0 0x55b0bc1ab1f0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 -- 172.21.15.60:0/2828971472 >> 172.21.15.60:0/2828971472 conn(0x55b0bc170890 msgr2=0x55b0bc1cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 -- 172.21.15.60:0/2828971472 shutdown_connections 2024-04-25T10:13:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 -- 172.21.15.60:0/2828971472 wait complete. 2024-04-25T10:13:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 -- start start 2024-04-25T10:13:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 Processor -- start 2024-04-25T10:13:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1748e0 0x55b0bc1c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1be2e0 0x55b0bc1c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1bec00 0x55b0bc1c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bbf5be70 con 0x55b0bc1bec00 2024-04-25T10:13:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0bbf49a50 con 0x55b0bc1be2e0 2024-04-25T10:13:58.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0bbfc03a0 con 0x55b0bc1748e0 2024-04-25T10:13:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 10 client.0 _ll_register_callbacks cb 0x55b0bbf6d240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23b9672640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1bec00 0x55b0bc1c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23b9672640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1bec00 0x55b0bc1c2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41718/0 (socket says 172.21.15.60:41718) 2024-04-25T10:13:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23b9672640 1 -- 172.21.15.60:0/2249457963 learned_addr learned my addr 172.21.15.60:0/2249457963 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23b9672640 1 -- 172.21.15.60:0/2249457963 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1748e0 msgr2=0x55b0bc1c24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23b9672640 1 --2- 172.21.15.60:0/2249457963 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1748e0 0x55b0bc1c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23b9672640 1 -- 172.21.15.60:0/2249457963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1be2e0 msgr2=0x55b0bc1c29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23b9672640 1 --2- 172.21.15.60:0/2249457963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1be2e0 0x55b0bc1c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23b9672640 1 -- 172.21.15.60:0/2249457963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0bc01a9e0 con 0x55b0bc1bec00 2024-04-25T10:13:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23b9672640 1 --2- 172.21.15.60:0/2249457963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1bec00 0x55b0bc1c2f20 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f23ac01a180 tx=0x7f239c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74b965d09cecd45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23937fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:13:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23937fe640 1 -- 172.21.15.60:0/2249457963 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0c3070 con 0x55b0bc1bec00 2024-04-25T10:13:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23937fe640 1 -- 172.21.15.60:0/2249457963 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f23ac007910 con 0x55b0bc1bec00 2024-04-25T10:13:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 -- 172.21.15.60:0/2249457963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23b005a340 con 0x55b0bc1bec00 2024-04-25T10:13:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23937fe640 1 -- 172.21.15.60:0/2249457963 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23ac0d6050 con 0x55b0bc1bec00 2024-04-25T10:13:58.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 10 client.8415 fetch_fsmap learned FSMap version 68 2024-04-25T10:13:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 -- 172.21.15.60:0/2249457963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55b0bbf60630 con 0x55b0bc1bec00 2024-04-25T10:13:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23937fe640 1 -- 172.21.15.60:0/2249457963 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b005a340 con 0x55b0bc1bec00 2024-04-25T10:13:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23937fe640 1 -- 172.21.15.60:0/2249457963 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f23ac0d9020 con 0x55b0bc1bec00 2024-04-25T10:13:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 10 client.8415 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:13:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 10 client.8415 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 20 client.8415 populate_metadata read hostname 'smithi060' 2024-04-25T10:13:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 --2- 172.21.15.60:0/2249457963 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x55b0bc1c3850 0x55b0bc22a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 4 client.8415 mds_command: new command op to 8256 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:13:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 -- 172.21.15.60:0/2249457963 --> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55b0bbfce5d0 con 0x55b0bc1c3850 2024-04-25T10:13:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23b8e71640 1 --2- 172.21.15.60:0/2249457963 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x55b0bc1c3850 0x55b0bc22a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23937fe640 10 client.8415 ms_handle_connect on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23b8e71640 1 --2- 172.21.15.60:0/2249457963 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x55b0bc1c3850 0x55b0bc22a570 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5f4faa055141043 server_cookie=23353af36a8e0aa7 in_seq=0 out_seq=0 2024-04-25T10:13:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23937fe640 1 -- 172.21.15.60:0/2249457963 <== mds.0 v2:172.21.15.60:6833/3807131024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b0bbfce5d0 con 0x55b0bc1c3850 2024-04-25T10:13:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23937fe640 10 client.8415 handle_command_reply: tid=0 2024-04-25T10:13:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 client.8415 shutdown 2024-04-25T10:13:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 -- 172.21.15.60:0/2249457963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1bec00 msgr2=0x55b0bc1c2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 --2- 172.21.15.60:0/2249457963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1bec00 0x55b0bc1c2f20 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f23ac01a180 tx=0x7f239c005840 comp rx=0 tx=0).stop 2024-04-25T10:13:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 -- 172.21.15.60:0/2249457963 shutdown_connections 2024-04-25T10:13:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 --2- 172.21.15.60:0/2249457963 >> [v2:172.21.15.60:6833/3807131024,v1:172.21.15.60:6835/3807131024] conn(0x55b0bc1c3850 0x55b0bc22a570 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 --2- 172.21.15.60:0/2249457963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55b0bc1bec00 0x55b0bc1c2f20 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23937fe640 0 client.8415 ms_handle_reset on v2:172.21.15.60:6833/3807131024 2024-04-25T10:13:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 --2- 172.21.15.60:0/2249457963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b0bc1be2e0 0x55b0bc1c29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 --2- 172.21.15.60:0/2249457963 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55b0bc1748e0 0x55b0bc1c24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 -- 172.21.15.60:0/2249457963 >> 172.21.15.60:0/2249457963 conn(0x55b0bc170890 msgr2=0x55b0bc1bd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 -- 172.21.15.60:0/2249457963 shutdown_connections 2024-04-25T10:13:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 1 -- 172.21.15.60:0/2249457963 wait complete. 2024-04-25T10:13:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.395+0000 7f23be242000 20 client.8415 trim_cache size 0 max 16384 2024-04-25T10:13:58.407 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:13:58.407 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-04-25T10:13:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.399+0000 7f23bb0d5640 1 -- 172.21.15.60:0/484006372 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f23a0066f90 msgr2=0x7f23a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.399+0000 7f23bb0d5640 1 --2- 172.21.15.60:0/484006372 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f23a0066f90 0x7f23a0069450 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f23b4141850 tx=0x7f23a4008040 comp rx=0 tx=0).stop 2024-04-25T10:13:58.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.399+0000 7f23bb0d5640 1 -- 172.21.15.60:0/484006372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23b40cf7e0 msgr2=0x7f23b41f15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.399+0000 7f23bb0d5640 1 --2- 172.21.15.60:0/484006372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23b40cf7e0 0x7f23b41f15e0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f23b000f600 tx=0x7f23b000fac0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.399+0000 7f23bb0d5640 1 -- 172.21.15.60:0/484006372 shutdown_connections 2024-04-25T10:13:58.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.399+0000 7f23bb0d5640 1 --2- 172.21.15.60:0/484006372 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f23a0066f90 0x7f23a0069450 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.399+0000 7f23bb0d5640 1 --2- 172.21.15.60:0/484006372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23b40b0bd0 0x7f23b413bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.399+0000 7f23bb0d5640 1 --2- 172.21.15.60:0/484006372 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23b40b1700 0x7f23b40b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.399+0000 7f23bb0d5640 1 --2- 172.21.15.60:0/484006372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23b40cf7e0 0x7f23b41f15e0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.399+0000 7f23bb0d5640 1 -- 172.21.15.60:0/484006372 >> 172.21.15.60:0/484006372 conn(0x7f23b40670b0 msgr2=0x7f23b415def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.399+0000 7f23bb0d5640 1 -- 172.21.15.60:0/484006372 shutdown_connections 2024-04-25T10:13:58.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.399+0000 7f23bb0d5640 1 -- 172.21.15.60:0/484006372 wait complete. 2024-04-25T10:13:58.417 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:13:58.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.519+0000 7f6811de6640 1 Processor -- start 2024-04-25T10:13:58.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.519+0000 7f6811de6640 1 -- start start 2024-04-25T10:13:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f6811de6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f680c162040 0x7f680c0716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f6811de6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f680c0b4810 0x7f680c071c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f6811de6640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680c06f040 con 0x7f680c06eaa0 2024-04-25T10:13:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f6811de6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680c0ccc40 con 0x7f680c162040 2024-04-25T10:13:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f6811de6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0ccf20 con 0x7f680c0b4810 2024-04-25T10:13:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f680c06eaa0 0x7f680c0b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58188/0 (socket says 172.21.15.60:58188) 2024-04-25T10:13:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680bfff640 1 -- 172.21.15.60:0/2325688755 learned_addr learned my addr 172.21.15.60:0/2325688755 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:13:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680b7fe640 1 --2- 172.21.15.60:0/2325688755 >> v2:172.21.15.100:3300/0 conn(0x7f680c162040 0x7f680c0716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680affd640 1 --2- 172.21.15.60:0/2325688755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f680c0b4810 0x7f680c071c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680a7fc640 1 -- 172.21.15.60:0/2325688755 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409102136 0 0) 0x7f680c06f040 con 0x7f680c06eaa0 2024-04-25T10:13:58.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680a7fc640 1 -- 172.21.15.60:0/2325688755 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67f4003530 con 0x7f680c06eaa0 2024-04-25T10:13:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680a7fc640 1 -- 172.21.15.60:0/2325688755 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3307941477 0 0) 0x7f67f4003530 con 0x7f680c06eaa0 2024-04-25T10:13:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680a7fc640 1 -- 172.21.15.60:0/2325688755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f680c0b4810 msgr2=0x7f680c071c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680a7fc640 1 --2- 172.21.15.60:0/2325688755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f680c0b4810 0x7f680c071c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680a7fc640 1 -- 172.21.15.60:0/2325688755 >> v2:172.21.15.100:3300/0 conn(0x7f680c162040 msgr2=0x7f680c0716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680a7fc640 1 --2- 172.21.15.60:0/2325688755 >> v2:172.21.15.100:3300/0 conn(0x7f680c162040 0x7f680c0716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680a7fc640 1 -- 172.21.15.60:0/2325688755 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c0cd200 con 0x7f680c06eaa0 2024-04-25T10:13:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680a7fc640 1 -- 172.21.15.60:0/2325688755 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f67fc001680 con 0x7f680c06eaa0 2024-04-25T10:13:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680a7fc640 1 -- 172.21.15.60:0/2325688755 >> v1:172.21.15.60:6789/0 conn(0x7f680c06eaa0 legacy=0x7f680c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680a7fc640 1 --2- 172.21.15.60:0/2325688755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f680c0b4810 0x7f67f40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680a7fc640 1 --2- 172.21.15.60:0/2325688755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f4003f90 0x7f67f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680a7fc640 1 --2- 172.21.15.60:0/2325688755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f4004d50 0x7f67f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680a7fc640 1 -- 172.21.15.60:0/2325688755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0ccc40 con 0x7f67f4004d50 2024-04-25T10:13:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680a7fc640 1 -- 172.21.15.60:0/2325688755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0ccf20 con 0x7f67f4003f90 2024-04-25T10:13:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680a7fc640 1 -- 172.21.15.60:0/2325688755 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f400b9d0 con 0x7f680c0b4810 2024-04-25T10:13:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680bfff640 1 --2- 172.21.15.60:0/2325688755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f4004d50 0x7f67f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680b7fe640 1 --2- 172.21.15.60:0/2325688755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f680c0b4810 0x7f67f40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680affd640 1 --2- 172.21.15.60:0/2325688755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f4003f90 0x7f67f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680bfff640 1 -- 172.21.15.60:0/2325688755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f680c0b4810 msgr2=0x7f67f40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680bfff640 1 --2- 172.21.15.60:0/2325688755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f680c0b4810 0x7f67f40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680bfff640 1 -- 172.21.15.60:0/2325688755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f4003f90 msgr2=0x7f67f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680bfff640 1 --2- 172.21.15.60:0/2325688755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f4003f90 0x7f67f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680bfff640 1 -- 172.21.15.60:0/2325688755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f4003530 con 0x7f67f4004d50 2024-04-25T10:13:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.523+0000 7f680affd640 1 --2- 172.21.15.60:0/2325688755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f4003f90 0x7f67f4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f680bfff640 1 --2- 172.21.15.60:0/2325688755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f4004d50 0x7f67f40051d0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00c160 tx=0x7f67fc01cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36a2b71b0ef6f20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f680a7fc640 1 -- 172.21.15.60:0/2325688755 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc007cb0 con 0x7f67f4004d50 2024-04-25T10:13:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f680a7fc640 1 -- 172.21.15.60:0/2325688755 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc0a9020 con 0x7f67f4004d50 2024-04-25T10:13:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f680a7fc640 1 -- 172.21.15.60:0/2325688755 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc005b20 con 0x7f67f4004d50 2024-04-25T10:13:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f6811de6640 1 -- 172.21.15.60:0/2325688755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f4004d50 msgr2=0x7f67f40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f6811de6640 1 --2- 172.21.15.60:0/2325688755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f4004d50 0x7f67f40051d0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00c160 tx=0x7f67fc01cbc0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f680bfff640 1 -- 172.21.15.60:0/2325688755 reap_dead start 2024-04-25T10:13:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f6811de6640 1 -- 172.21.15.60:0/2325688755 shutdown_connections 2024-04-25T10:13:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f6811de6640 1 -- 172.21.15.60:0/2325688755 >> 172.21.15.60:0/2325688755 conn(0x7f680c0cf9d0 msgr2=0x7f680c0cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f6811de6640 1 -- 172.21.15.60:0/2325688755 shutdown_connections 2024-04-25T10:13:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f6811de6640 1 -- 172.21.15.60:0/2325688755 wait complete. 2024-04-25T10:13:58.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f6811de6640 1 Processor -- start 2024-04-25T10:13:58.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f6811de6640 1 -- start start 2024-04-25T10:13:58.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f6811de6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f4004d50 0x7f680c1fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f6811de6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f680c15b570 0x7f680c154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f6811de6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f680c163d50 0x7f680c1641d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f6811de6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0b45d0 con 0x7f67f4004d50 2024-04-25T10:13:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f6811de6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0b4450 con 0x7f680c15b570 2024-04-25T10:13:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f6811de6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0ce730 con 0x7f680c163d50 2024-04-25T10:13:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f680b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f4004d50 0x7f680c1fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f680b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f4004d50 0x7f680c1fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41742/0 (socket says 172.21.15.60:41742) 2024-04-25T10:13:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f680b7fe640 1 -- 172.21.15.60:0/3726919538 learned_addr learned my addr 172.21.15.60:0/3726919538 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f680affd640 1 --2- 172.21.15.60:0/3726919538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f680c15b570 0x7f680c154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f680b7fe640 1 -- 172.21.15.60:0/3726919538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f680c163d50 msgr2=0x7f680c1641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f680b7fe640 1 --2- 172.21.15.60:0/3726919538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f680c163d50 0x7f680c1641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f680b7fe640 1 -- 172.21.15.60:0/3726919538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f680c15b570 msgr2=0x7f680c154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f680b7fe640 1 --2- 172.21.15.60:0/3726919538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f680c15b570 0x7f680c154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f680b7fe640 1 -- 172.21.15.60:0/3726919538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c1fc290 con 0x7f67f4004d50 2024-04-25T10:13:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f680b7fe640 1 --2- 172.21.15.60:0/3726919538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f4004d50 0x7f680c1fbd50 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f68040116c0 tx=0x7f6804011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.527+0000 7f67f3fff640 1 -- 172.21.15.60:0/3726919538 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680400f940 con 0x7f67f4004d50 2024-04-25T10:13:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.531+0000 7f67f3fff640 1 -- 172.21.15.60:0/3726919538 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680400fae0 con 0x7f67f4004d50 2024-04-25T10:13:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.531+0000 7f6811de6640 1 -- 172.21.15.60:0/3726919538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680c1552e0 con 0x7f67f4004d50 2024-04-25T10:13:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.531+0000 7f67f3fff640 1 -- 172.21.15.60:0/3726919538 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680400fe90 con 0x7f67f4004d50 2024-04-25T10:13:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.531+0000 7f6811de6640 1 -- 172.21.15.60:0/3726919538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680c155640 con 0x7f67f4004d50 2024-04-25T10:13:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.531+0000 7f6811de6640 1 -- 172.21.15.60:0/3726919538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680c074770 con 0x7f67f4004d50 2024-04-25T10:13:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.531+0000 7f67f3fff640 1 -- 172.21.15.60:0/3726919538 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6804015460 con 0x7f67f4004d50 2024-04-25T10:13:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.531+0000 7f67f3fff640 1 --2- 172.21.15.60:0/3726919538 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67f8067130 0x7f67f80695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.531+0000 7f67f3fff640 1 -- 172.21.15.60:0/3726919538 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f68040859c0 con 0x7f67f4004d50 2024-04-25T10:13:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.531+0000 7f680affd640 1 --2- 172.21.15.60:0/3726919538 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67f8067130 0x7f67f80695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.531+0000 7f680affd640 1 --2- 172.21.15.60:0/3726919538 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67f8067130 0x7f67f80695f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f680c1649d0 tx=0x7f680000d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.535+0000 7f67f3fff640 1 -- 172.21.15.60:0/3726919538 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6804087e20 con 0x7f67f4004d50 2024-04-25T10:13:58.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.723+0000 7f6811de6640 1 -- 172.21.15.60:0/3726919538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f680c165310 con 0x7f67f4004d50 2024-04-25T10:13:58.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.723+0000 7f67f3fff640 1 -- 172.21.15.60:0/3726919538 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f6804019030 con 0x7f67f4004d50 2024-04-25T10:13:58.727 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:13:58.727 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:13:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.723+0000 7f6811de6640 1 -- 172.21.15.60:0/3726919538 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67f8067130 msgr2=0x7f67f80695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.723+0000 7f6811de6640 1 --2- 172.21.15.60:0/3726919538 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67f8067130 0x7f67f80695f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f680c1649d0 tx=0x7f680000d040 comp rx=0 tx=0).stop 2024-04-25T10:13:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.723+0000 7f6811de6640 1 -- 172.21.15.60:0/3726919538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f4004d50 msgr2=0x7f680c1fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.723+0000 7f6811de6640 1 --2- 172.21.15.60:0/3726919538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f4004d50 0x7f680c1fbd50 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f68040116c0 tx=0x7f6804011b80 comp rx=0 tx=0).stop 2024-04-25T10:13:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.723+0000 7f6811de6640 1 -- 172.21.15.60:0/3726919538 shutdown_connections 2024-04-25T10:13:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.723+0000 7f6811de6640 1 --2- 172.21.15.60:0/3726919538 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f67f8067130 0x7f67f80695f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.723+0000 7f6811de6640 1 --2- 172.21.15.60:0/3726919538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f680c163d50 0x7f680c1641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.723+0000 7f6811de6640 1 --2- 172.21.15.60:0/3726919538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f680c15b570 0x7f680c154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.723+0000 7f6811de6640 1 --2- 172.21.15.60:0/3726919538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67f4004d50 0x7f680c1fbd50 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.723+0000 7f6811de6640 1 -- 172.21.15.60:0/3726919538 >> 172.21.15.60:0/3726919538 conn(0x7f680c0cf9d0 msgr2=0x7f680c0b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.723+0000 7f6811de6640 1 -- 172.21.15.60:0/3726919538 shutdown_connections 2024-04-25T10:13:58.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:58.723+0000 7f6811de6640 1 -- 172.21.15.60:0/3726919538 wait complete. 2024-04-25T10:13:58.744 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:13:58.745 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:13:58.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:58.796 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:58.840 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:13:58.840 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:13:58.840 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:58.845 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:59.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.003+0000 7f4be73be640 1 Processor -- start 2024-04-25T10:13:59.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.003+0000 7f4be73be640 1 -- start start 2024-04-25T10:13:59.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4be73be640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be00ae7c0 0x7f4be0071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4be73be640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4be006d2f0 0x7f4be00717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4be73be640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be0071fd0 con 0x7f4be015dc90 2024-04-25T10:13:59.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4be73be640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be00accc0 con 0x7f4be006d2f0 2024-04-25T10:13:59.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4be73be640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00ad260 con 0x7f4be00ae7c0 2024-04-25T10:13:59.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4be595b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4be006d2f0 0x7f4be00717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4be595b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4be006d2f0 0x7f4be00717d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47678/0 (socket says 172.21.15.60:47678) 2024-04-25T10:13:59.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4be595b640 1 -- 172.21.15.60:0/914454084 learned_addr learned my addr 172.21.15.60:0/914454084 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:59.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4be4959640 1 --2- 172.21.15.60:0/914454084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be00ae7c0 0x7f4be0071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4be595b640 1 -- 172.21.15.60:0/914454084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be00ae7c0 msgr2=0x7f4be0071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:59.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4be595b640 1 --2- 172.21.15.60:0/914454084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be00ae7c0 0x7f4be0071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4be595b640 1 -- 172.21.15.60:0/914454084 >> v1:172.21.15.100:6789/0 conn(0x7f4be015dc90 legacy=0x7f4be00b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:59.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4be595b640 1 -- 172.21.15.60:0/914454084 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be00b2ce0 con 0x7f4be006d2f0 2024-04-25T10:13:59.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4be595b640 1 --2- 172.21.15.60:0/914454084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be006d2f0 0x7f4be00717d0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009e50 tx=0x7f4bd801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77182f284d211d9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4bd7fff640 1 -- 172.21.15.60:0/914454084 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80a5810 con 0x7f4be006d2f0 2024-04-25T10:13:59.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4bd7fff640 1 -- 172.21.15.60:0/914454084 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd80a59b0 con 0x7f4be006d2f0 2024-04-25T10:13:59.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.007+0000 7f4bd7fff640 1 -- 172.21.15.60:0/914454084 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80a5d80 con 0x7f4be006d2f0 2024-04-25T10:13:59.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 -- 172.21.15.60:0/914454084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be006d2f0 msgr2=0x7f4be00717d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 --2- 172.21.15.60:0/914454084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be006d2f0 0x7f4be00717d0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009e50 tx=0x7f4bd801ca30 comp rx=0 tx=0).stop 2024-04-25T10:13:59.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 -- 172.21.15.60:0/914454084 shutdown_connections 2024-04-25T10:13:59.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 --2- 172.21.15.60:0/914454084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be006d2f0 0x7f4be00717d0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 --2- 172.21.15.60:0/914454084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be00ae7c0 0x7f4be0071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 -- 172.21.15.60:0/914454084 >> 172.21.15.60:0/914454084 conn(0x7f4be00b12d0 msgr2=0x7f4be00b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 -- 172.21.15.60:0/914454084 shutdown_connections 2024-04-25T10:13:59.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 -- 172.21.15.60:0/914454084 wait complete. 2024-04-25T10:13:59.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 Processor -- start 2024-04-25T10:13:59.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 -- start start 2024-04-25T10:13:59.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be006d2f0 0x7f4be013eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be00ae7c0 0x7f4be00af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be015dc90 0x7f4be00af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be016a7e0 con 0x7f4be015dc90 2024-04-25T10:13:59.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00b2220 con 0x7f4be00ae7c0 2024-04-25T10:13:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be016a960 con 0x7f4be006d2f0 2024-04-25T10:13:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be595b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be015dc90 0x7f4be00af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be595b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be015dc90 0x7f4be00af780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47680/0 (socket says 172.21.15.60:47680) 2024-04-25T10:13:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be595b640 1 -- 172.21.15.60:0/15937049 learned_addr learned my addr 172.21.15.60:0/15937049 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be4959640 1 --2- 172.21.15.60:0/15937049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be00ae7c0 0x7f4be00af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be515a640 1 --2- 172.21.15.60:0/15937049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be006d2f0 0x7f4be013eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be595b640 1 -- 172.21.15.60:0/15937049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be006d2f0 msgr2=0x7f4be013eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be595b640 1 --2- 172.21.15.60:0/15937049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be006d2f0 0x7f4be013eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be595b640 1 -- 172.21.15.60:0/15937049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be00ae7c0 msgr2=0x7f4be00af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be595b640 1 --2- 172.21.15.60:0/15937049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be00ae7c0 0x7f4be00af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be595b640 1 -- 172.21.15.60:0/15937049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be014d130 con 0x7f4be015dc90 2024-04-25T10:13:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be595b640 1 --2- 172.21.15.60:0/15937049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be015dc90 0x7f4be00af780 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80af040 tx=0x7f4bd80197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4bd5ffb640 1 -- 172.21.15.60:0/15937049 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80a5d60 con 0x7f4be015dc90 2024-04-25T10:13:59.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4bd5ffb640 1 -- 172.21.15.60:0/15937049 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd80be070 con 0x7f4be015dc90 2024-04-25T10:13:59.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4bd5ffb640 1 -- 172.21.15.60:0/15937049 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80b86b0 con 0x7f4be015dc90 2024-04-25T10:13:59.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 -- 172.21.15.60:0/15937049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be014d410 con 0x7f4be015dc90 2024-04-25T10:13:59.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.011+0000 7f4be73be640 1 -- 172.21.15.60:0/15937049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be014d9d0 con 0x7f4be015dc90 2024-04-25T10:13:59.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.015+0000 7f4be73be640 1 -- 172.21.15.60:0/15937049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be0074770 con 0x7f4be015dc90 2024-04-25T10:13:59.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.015+0000 7f4bd5ffb640 1 -- 172.21.15.60:0/15937049 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4bd80b8850 con 0x7f4be015dc90 2024-04-25T10:13:59.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.015+0000 7f4bd5ffb640 1 --2- 172.21.15.60:0/15937049 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4bc4066f90 0x7f4bc4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.015+0000 7f4bd5ffb640 1 -- 172.21.15.60:0/15937049 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f4bd811fe90 con 0x7f4be015dc90 2024-04-25T10:13:59.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.015+0000 7f4be515a640 1 --2- 172.21.15.60:0/15937049 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4bc4066f90 0x7f4bc4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.015+0000 7f4be515a640 1 --2- 172.21.15.60:0/15937049 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4bc4066f90 0x7f4bc4069450 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0015b0 tx=0x7f4bdc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.019+0000 7f4bd5ffb640 1 -- 172.21.15.60:0/15937049 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4bd8016620 con 0x7f4be015dc90 2024-04-25T10:13:59.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.179+0000 7f4be73be640 1 -- 172.21.15.60:0/15937049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4be00cec00 con 0x7f4be015dc90 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.179+0000 7f4bd5ffb640 1 -- 172.21.15.60:0/15937049 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16981 (secure 0 0 0) 0x7f4bd80f0e00 con 0x7f4be015dc90 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":86,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:13:19.192820+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:42.621502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:43.352679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/2817879567":"2024-04-26T10:13:19.192767+0000","172.21.15.60:6833/2817879567":"2024-04-26T10:13:19.192767+0000","172.21.15.60:6835/2455562778":"2024-04-26T10:11:58.136789+0000","172.21.15.60:6833/2455562778":"2024-04-26T10:11:58.136789+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:59.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.183+0000 7f4be73be640 1 -- 172.21.15.60:0/15937049 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4bc4066f90 msgr2=0x7f4bc4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.183+0000 7f4be73be640 1 --2- 172.21.15.60:0/15937049 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4bc4066f90 0x7f4bc4069450 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0015b0 tx=0x7f4bdc009040 comp rx=0 tx=0).stop 2024-04-25T10:13:59.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.183+0000 7f4be73be640 1 -- 172.21.15.60:0/15937049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be015dc90 msgr2=0x7f4be00af780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.183+0000 7f4be73be640 1 --2- 172.21.15.60:0/15937049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be015dc90 0x7f4be00af780 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80af040 tx=0x7f4bd80197d0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.183+0000 7f4be73be640 1 -- 172.21.15.60:0/15937049 shutdown_connections 2024-04-25T10:13:59.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.183+0000 7f4be73be640 1 --2- 172.21.15.60:0/15937049 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4bc4066f90 0x7f4bc4069450 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.183+0000 7f4be73be640 1 --2- 172.21.15.60:0/15937049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4be015dc90 0x7f4be00af780 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.183+0000 7f4be73be640 1 --2- 172.21.15.60:0/15937049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4be00ae7c0 0x7f4be00af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.183+0000 7f4be73be640 1 --2- 172.21.15.60:0/15937049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4be006d2f0 0x7f4be013eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.183+0000 7f4be73be640 1 -- 172.21.15.60:0/15937049 >> 172.21.15.60:0/15937049 conn(0x7f4be00b12d0 msgr2=0x7f4be0153060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.183+0000 7f4be73be640 1 -- 172.21.15.60:0/15937049 shutdown_connections 2024-04-25T10:13:59.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.183+0000 7f4be73be640 1 -- 172.21.15.60:0/15937049 wait complete. 2024-04-25T10:13:59.204 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:42.621502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:43.352679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:59.204 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:59.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.319+0000 7f2f56a45640 1 Processor -- start 2024-04-25T10:13:59.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.319+0000 7f2f56a45640 1 -- start start 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.319+0000 7f2f56a45640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2f500d0960 0x7f2f500d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.319+0000 7f2f56a45640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f500d1bb0 con 0x7f2f500d2340 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.319+0000 7f2f56a45640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f500b85e0 con 0x7f2f5015dea0 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.319+0000 7f2f56a45640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f500b9670 con 0x7f2f500d0960 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.319+0000 7f2f47fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2f500d0960 0x7f2f500d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.319+0000 7f2f54fe2640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2f500d2340 0x7f2f500d27d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38566/0 (socket says 172.21.15.60:38566) 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.319+0000 7f2f54fe2640 1 -- 172.21.15.60:0/131180821 learned_addr learned my addr 172.21.15.60:0/131180821 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.319+0000 7f2f47fff640 1 -- 172.21.15.60:0/131180821 >> v1:172.21.15.100:6789/0 conn(0x7f2f5015dea0 legacy=0x7f2f500ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.319+0000 7f2f47fff640 1 -- 172.21.15.60:0/131180821 >> v1:172.21.15.60:6789/0 conn(0x7f2f500d2340 legacy=0x7f2f500d27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.319+0000 7f2f47fff640 1 -- 172.21.15.60:0/131180821 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f500b88c0 con 0x7f2f500d0960 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.319+0000 7f2f4f7fe640 1 -- 172.21.15.60:0/131180821 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748297199 0 0) 0x7f2f500d1bb0 con 0x7f2f500d2340 2024-04-25T10:13:59.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f47fff640 1 --2- 172.21.15.60:0/131180821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500d0960 0x7f2f500d0de0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f2f380097f0 tx=0x7f2f3801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79591a68ac472212 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f4f7fe640 1 -- 172.21.15.60:0/131180821 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f380b0070 con 0x7f2f500d0960 2024-04-25T10:13:59.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f4f7fe640 1 -- 172.21.15.60:0/131180821 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f38004230 con 0x7f2f500d0960 2024-04-25T10:13:59.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f4f7fe640 1 -- 172.21.15.60:0/131180821 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38003620 con 0x7f2f500d0960 2024-04-25T10:13:59.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f56a45640 1 -- 172.21.15.60:0/131180821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500d0960 msgr2=0x7f2f500d0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f56a45640 1 --2- 172.21.15.60:0/131180821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500d0960 0x7f2f500d0de0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f2f380097f0 tx=0x7f2f3801c460 comp rx=0 tx=0).stop 2024-04-25T10:13:59.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f56a45640 1 -- 172.21.15.60:0/131180821 shutdown_connections 2024-04-25T10:13:59.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f56a45640 1 --2- 172.21.15.60:0/131180821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500d0960 0x7f2f500d0de0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f56a45640 1 -- 172.21.15.60:0/131180821 >> 172.21.15.60:0/131180821 conn(0x7f2f5006c4d0 msgr2=0x7f2f5006c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f56a45640 1 -- 172.21.15.60:0/131180821 shutdown_connections 2024-04-25T10:13:59.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f56a45640 1 -- 172.21.15.60:0/131180821 wait complete. 2024-04-25T10:13:59.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f56a45640 1 Processor -- start 2024-04-25T10:13:59.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f56a45640 1 -- start start 2024-04-25T10:13:59.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f56a45640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500d0960 0x7f2f500c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f56a45640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f500d2340 0x7f2f50158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f56a45640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f500c7b10 0x7f2f501561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f56a45640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500cf0f0 con 0x7f2f500d0960 2024-04-25T10:13:59.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f56a45640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500cef70 con 0x7f2f500c7b10 2024-04-25T10:13:59.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f56a45640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5016b860 con 0x7f2f500d2340 2024-04-25T10:13:59.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f4ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500d0960 0x7f2f500c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f4ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500d0960 0x7f2f500c9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47692/0 (socket says 172.21.15.60:47692) 2024-04-25T10:13:59.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f4ffff640 1 -- 172.21.15.60:0/2637704197 learned_addr learned my addr 172.21.15.60:0/2637704197 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:59.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f47fff640 1 --2- 172.21.15.60:0/2637704197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f500d2340 0x7f2f50158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f54fe2640 1 --2- 172.21.15.60:0/2637704197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f500c7b10 0x7f2f501561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f4ffff640 1 -- 172.21.15.60:0/2637704197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f500d2340 msgr2=0x7f2f50158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f4ffff640 1 --2- 172.21.15.60:0/2637704197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f500d2340 0x7f2f50158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f4ffff640 1 -- 172.21.15.60:0/2637704197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f500c7b10 msgr2=0x7f2f501561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f4ffff640 1 --2- 172.21.15.60:0/2637704197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f500c7b10 0x7f2f501561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f4ffff640 1 -- 172.21.15.60:0/2637704197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5014b4a0 con 0x7f2f500d0960 2024-04-25T10:13:59.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.323+0000 7f2f4ffff640 1 --2- 172.21.15.60:0/2637704197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500d0960 0x7f2f500c9850 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000b440 tx=0x7f2f4000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.327+0000 7f2f4d7fa640 1 -- 172.21.15.60:0/2637704197 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40015020 con 0x7f2f500d0960 2024-04-25T10:13:59.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.327+0000 7f2f4d7fa640 1 -- 172.21.15.60:0/2637704197 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f40011890 con 0x7f2f500d0960 2024-04-25T10:13:59.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.327+0000 7f2f4d7fa640 1 -- 172.21.15.60:0/2637704197 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40011c80 con 0x7f2f500d0960 2024-04-25T10:13:59.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.327+0000 7f2f56a45640 1 -- 172.21.15.60:0/2637704197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f500c8fb0 con 0x7f2f500d0960 2024-04-25T10:13:59.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.327+0000 7f2f56a45640 1 -- 172.21.15.60:0/2637704197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f500c9570 con 0x7f2f500d0960 2024-04-25T10:13:59.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.327+0000 7f2f56a45640 1 -- 172.21.15.60:0/2637704197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f50074770 con 0x7f2f500d0960 2024-04-25T10:13:59.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.327+0000 7f2f4d7fa640 1 -- 172.21.15.60:0/2637704197 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f40018020 con 0x7f2f500d0960 2024-04-25T10:13:59.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.327+0000 7f2f4d7fa640 1 --2- 172.21.15.60:0/2637704197 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2f30066f60 0x7f2f30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.327+0000 7f2f4d7fa640 1 -- 172.21.15.60:0/2637704197 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f2f40084d50 con 0x7f2f500d0960 2024-04-25T10:13:59.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.327+0000 7f2f47fff640 1 --2- 172.21.15.60:0/2637704197 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2f30066f60 0x7f2f30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.327+0000 7f2f47fff640 1 --2- 172.21.15.60:0/2637704197 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2f30066f60 0x7f2f30069420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2f380ad000 tx=0x7f2f38008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.331+0000 7f2f4d7fa640 1 -- 172.21.15.60:0/2637704197 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f40088030 con 0x7f2f500d0960 2024-04-25T10:13:59.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.487+0000 7f2f56a45640 1 -- 172.21.15.60:0/2637704197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f500b3be0 con 0x7f2f500d0960 2024-04-25T10:13:59.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.491+0000 7f2f4d7fa640 1 -- 172.21.15.60:0/2637704197 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f4001f030 con 0x7f2f500d0960 2024-04-25T10:13:59.494 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:13:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.491+0000 7f2f56a45640 1 -- 172.21.15.60:0/2637704197 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2f30066f60 msgr2=0x7f2f30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.491+0000 7f2f56a45640 1 --2- 172.21.15.60:0/2637704197 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2f30066f60 0x7f2f30069420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2f380ad000 tx=0x7f2f38008040 comp rx=0 tx=0).stop 2024-04-25T10:13:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.491+0000 7f2f56a45640 1 -- 172.21.15.60:0/2637704197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500d0960 msgr2=0x7f2f500c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.491+0000 7f2f56a45640 1 --2- 172.21.15.60:0/2637704197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500d0960 0x7f2f500c9850 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000b440 tx=0x7f2f4000b900 comp rx=0 tx=0).stop 2024-04-25T10:13:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.491+0000 7f2f56a45640 1 -- 172.21.15.60:0/2637704197 shutdown_connections 2024-04-25T10:13:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.491+0000 7f2f56a45640 1 --2- 172.21.15.60:0/2637704197 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2f30066f60 0x7f2f30069420 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.491+0000 7f2f56a45640 1 --2- 172.21.15.60:0/2637704197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f500c7b10 0x7f2f501561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.491+0000 7f2f56a45640 1 --2- 172.21.15.60:0/2637704197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f500d2340 0x7f2f50158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.491+0000 7f2f56a45640 1 --2- 172.21.15.60:0/2637704197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500d0960 0x7f2f500c9850 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.491+0000 7f2f56a45640 1 -- 172.21.15.60:0/2637704197 >> 172.21.15.60:0/2637704197 conn(0x7f2f5006c4d0 msgr2=0x7f2f50150e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.491+0000 7f2f56a45640 1 -- 172.21.15.60:0/2637704197 shutdown_connections 2024-04-25T10:13:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.491+0000 7f2f56a45640 1 -- 172.21.15.60:0/2637704197 wait complete. 2024-04-25T10:13:59.512 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:59.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6beba47640 1 Processor -- start 2024-04-25T10:13:59.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6beba47640 1 -- start start 2024-04-25T10:13:59.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6beba47640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6be40cbcf0 0x7f6be4160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6beba47640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6be4071c30 0x7f6be415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6beba47640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be4144a40 con 0x7f6be415fa20 2024-04-25T10:13:59.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6beba47640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be40b2dc0 con 0x7f6be40cbcf0 2024-04-25T10:13:59.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6beba47640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be415fe80 con 0x7f6be4071c30 2024-04-25T10:13:59.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6be9fe4640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6be415fa20 0x7f6be415bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38570/0 (socket says 172.21.15.60:38570) 2024-04-25T10:13:59.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6be9fe4640 1 -- 172.21.15.60:0/3058794527 learned_addr learned my addr 172.21.15.60:0/3058794527 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:13:59.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6be8fe2640 1 --2- 172.21.15.60:0/3058794527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6be4071c30 0x7f6be415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6be97e3640 1 --2- 172.21.15.60:0/3058794527 >> v2:172.21.15.100:3300/0 conn(0x7f6be40cbcf0 0x7f6be4160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6bdbfff640 1 -- 172.21.15.60:0/3058794527 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501959761 0 0) 0x7f6be4144a40 con 0x7f6be415fa20 2024-04-25T10:13:59.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6bdbfff640 1 -- 172.21.15.60:0/3058794527 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bd0003530 con 0x7f6be415fa20 2024-04-25T10:13:59.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6bdbfff640 1 -- 172.21.15.60:0/3058794527 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1558546805 0 0) 0x7f6bd0003530 con 0x7f6be415fa20 2024-04-25T10:13:59.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6bdbfff640 1 -- 172.21.15.60:0/3058794527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6be4071c30 msgr2=0x7f6be415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:59.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6bdbfff640 1 --2- 172.21.15.60:0/3058794527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6be4071c30 0x7f6be415f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6bdbfff640 1 -- 172.21.15.60:0/3058794527 >> v2:172.21.15.100:3300/0 conn(0x7f6be40cbcf0 msgr2=0x7f6be4160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:59.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6bdbfff640 1 --2- 172.21.15.60:0/3058794527 >> v2:172.21.15.100:3300/0 conn(0x7f6be40cbcf0 0x7f6be4160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6bdbfff640 1 -- 172.21.15.60:0/3058794527 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be40cc9f0 con 0x7f6be415fa20 2024-04-25T10:13:59.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6bdbfff640 1 -- 172.21.15.60:0/3058794527 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f6bdc001a00 con 0x7f6be415fa20 2024-04-25T10:13:59.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6bdbfff640 1 -- 172.21.15.60:0/3058794527 >> v1:172.21.15.60:6789/0 conn(0x7f6be415fa20 legacy=0x7f6be415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6bdbfff640 1 --2- 172.21.15.60:0/3058794527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bd00039c0 0x7f6bd0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6bdbfff640 1 --2- 172.21.15.60:0/3058794527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6be4071c30 0x7f6bd0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6bdbfff640 1 --2- 172.21.15.60:0/3058794527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bd0004d50 0x7f6bd00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6bdbfff640 1 -- 172.21.15.60:0/3058794527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd000b9d0 con 0x7f6bd0004d50 2024-04-25T10:13:59.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6bdbfff640 1 -- 172.21.15.60:0/3058794527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd000bb50 con 0x7f6bd00039c0 2024-04-25T10:13:59.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6bdbfff640 1 -- 172.21.15.60:0/3058794527 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd000bcd0 con 0x7f6be4071c30 2024-04-25T10:13:59.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6be9fe4640 1 --2- 172.21.15.60:0/3058794527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bd0004d50 0x7f6bd00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6be97e3640 1 --2- 172.21.15.60:0/3058794527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bd00039c0 0x7f6bd0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6be8fe2640 1 --2- 172.21.15.60:0/3058794527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6be4071c30 0x7f6bd0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6be9fe4640 1 -- 172.21.15.60:0/3058794527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6be4071c30 msgr2=0x7f6bd0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:59.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6be9fe4640 1 --2- 172.21.15.60:0/3058794527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6be4071c30 0x7f6bd0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6be9fe4640 1 -- 172.21.15.60:0/3058794527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bd00039c0 msgr2=0x7f6bd0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:59.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6be9fe4640 1 --2- 172.21.15.60:0/3058794527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bd00039c0 0x7f6bd0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6be9fe4640 1 -- 172.21.15.60:0/3058794527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd0003530 con 0x7f6bd0004d50 2024-04-25T10:13:59.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.623+0000 7f6be97e3640 1 --2- 172.21.15.60:0/3058794527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bd00039c0 0x7f6bd0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:59.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.627+0000 7f6be9fe4640 1 --2- 172.21.15.60:0/3058794527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bd0004d50 0x7f6bd00051d0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00a880 tx=0x7f6bdc01d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3476182b9c7f5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.627+0000 7f6bdbfff640 1 -- 172.21.15.60:0/3058794527 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc00a4a0 con 0x7f6bd0004d50 2024-04-25T10:13:59.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.627+0000 7f6bdbfff640 1 -- 172.21.15.60:0/3058794527 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc0ad630 con 0x7f6bd0004d50 2024-04-25T10:13:59.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.627+0000 7f6bdbfff640 1 -- 172.21.15.60:0/3058794527 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc0ada60 con 0x7f6bd0004d50 2024-04-25T10:13:59.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.627+0000 7f6beba47640 1 -- 172.21.15.60:0/3058794527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bd0004d50 msgr2=0x7f6bd00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.627+0000 7f6beba47640 1 --2- 172.21.15.60:0/3058794527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bd0004d50 0x7f6bd00051d0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00a880 tx=0x7f6bdc01d3a0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.627+0000 7f6be9fe4640 1 -- 172.21.15.60:0/3058794527 reap_dead start 2024-04-25T10:13:59.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.627+0000 7f6beba47640 1 -- 172.21.15.60:0/3058794527 shutdown_connections 2024-04-25T10:13:59.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.627+0000 7f6beba47640 1 -- 172.21.15.60:0/3058794527 >> 172.21.15.60:0/3058794527 conn(0x7f6be40670b0 msgr2=0x7f6be406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.627+0000 7f6beba47640 1 -- 172.21.15.60:0/3058794527 shutdown_connections 2024-04-25T10:13:59.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.627+0000 7f6beba47640 1 -- 172.21.15.60:0/3058794527 wait complete. 2024-04-25T10:13:59.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.627+0000 7f6beba47640 1 Processor -- start 2024-04-25T10:13:59.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.627+0000 7f6beba47640 1 -- start start 2024-04-25T10:13:59.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6beba47640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6bd0004d50 0x7f6be40a8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6beba47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6be41f5530 0x7f6be4143e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6beba47640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6be41f1520 0x7f6be415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6beba47640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be40ce9a0 con 0x7f6be41f5530 2024-04-25T10:13:59.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6beba47640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be40ccfb0 con 0x7f6be41f1520 2024-04-25T10:13:59.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6beba47640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be40cc0d0 con 0x7f6bd0004d50 2024-04-25T10:13:59.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6be8fe2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6be41f5530 0x7f6be4143e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6be8fe2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6be41f5530 0x7f6be4143e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47708/0 (socket says 172.21.15.60:47708) 2024-04-25T10:13:59.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6be8fe2640 1 -- 172.21.15.60:0/3893392164 learned_addr learned my addr 172.21.15.60:0/3893392164 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:59.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6be8fe2640 1 -- 172.21.15.60:0/3893392164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6bd0004d50 msgr2=0x7f6be40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6be8fe2640 1 --2- 172.21.15.60:0/3893392164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6bd0004d50 0x7f6be40a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6be8fe2640 1 -- 172.21.15.60:0/3893392164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6be41f1520 msgr2=0x7f6be415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6be8fe2640 1 --2- 172.21.15.60:0/3893392164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6be41f1520 0x7f6be415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6be8fe2640 1 -- 172.21.15.60:0/3893392164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be413d150 con 0x7f6be41f5530 2024-04-25T10:13:59.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6be8fe2640 1 --2- 172.21.15.60:0/3893392164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6be41f5530 0x7f6be4143e00 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40115e0 tx=0x7f6bd4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6bd9ffb640 1 -- 172.21.15.60:0/3893392164 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4015870 con 0x7f6be41f5530 2024-04-25T10:13:59.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6bd9ffb640 1 -- 172.21.15.60:0/3893392164 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd4015a10 con 0x7f6be41f5530 2024-04-25T10:13:59.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6beba47640 1 -- 172.21.15.60:0/3893392164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be413d430 con 0x7f6be41f5530 2024-04-25T10:13:59.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6bd9ffb640 1 -- 172.21.15.60:0/3893392164 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4015dc0 con 0x7f6be41f5530 2024-04-25T10:13:59.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6beba47640 1 -- 172.21.15.60:0/3893392164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be413d9f0 con 0x7f6be41f5530 2024-04-25T10:13:59.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6beba47640 1 -- 172.21.15.60:0/3893392164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be4074770 con 0x7f6be41f5530 2024-04-25T10:13:59.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6bd9ffb640 1 -- 172.21.15.60:0/3893392164 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6bd400f460 con 0x7f6be41f5530 2024-04-25T10:13:59.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6bd9ffb640 1 --2- 172.21.15.60:0/3893392164 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6bc8066f90 0x7f6bc8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6bd9ffb640 1 -- 172.21.15.60:0/3893392164 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f6bd40859a0 con 0x7f6be41f5530 2024-04-25T10:13:59.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.631+0000 7f6be97e3640 1 --2- 172.21.15.60:0/3893392164 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6bc8066f90 0x7f6bc8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.635+0000 7f6be97e3640 1 --2- 172.21.15.60:0/3893392164 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6bc8066f90 0x7f6bc8069450 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6be00098d0 tx=0x7f6be0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.635+0000 7f6bd9ffb640 1 -- 172.21.15.60:0/3893392164 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6bd4019030 con 0x7f6be41f5530 2024-04-25T10:13:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.795+0000 7f6beba47640 1 -- 172.21.15.60:0/3893392164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6be40cf1d0 con 0x7f6be41f5530 2024-04-25T10:13:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.795+0000 7f6bd9ffb640 1 -- 172.21.15.60:0/3893392164 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6bd40568e0 con 0x7f6be41f5530 2024-04-25T10:13:59.800 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:13:59.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.795+0000 7f6beba47640 1 -- 172.21.15.60:0/3893392164 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6bc8066f90 msgr2=0x7f6bc8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.795+0000 7f6beba47640 1 --2- 172.21.15.60:0/3893392164 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6bc8066f90 0x7f6bc8069450 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6be00098d0 tx=0x7f6be0008040 comp rx=0 tx=0).stop 2024-04-25T10:13:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.795+0000 7f6beba47640 1 -- 172.21.15.60:0/3893392164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6be41f5530 msgr2=0x7f6be4143e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.795+0000 7f6beba47640 1 --2- 172.21.15.60:0/3893392164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6be41f5530 0x7f6be4143e00 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40115e0 tx=0x7f6bd4011aa0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.795+0000 7f6beba47640 1 -- 172.21.15.60:0/3893392164 shutdown_connections 2024-04-25T10:13:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.795+0000 7f6beba47640 1 --2- 172.21.15.60:0/3893392164 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6bc8066f90 0x7f6bc8069450 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.795+0000 7f6beba47640 1 --2- 172.21.15.60:0/3893392164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6be41f1520 0x7f6be415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.795+0000 7f6beba47640 1 --2- 172.21.15.60:0/3893392164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6be41f5530 0x7f6be4143e00 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.795+0000 7f6beba47640 1 --2- 172.21.15.60:0/3893392164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6bd0004d50 0x7f6be40a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.795+0000 7f6beba47640 1 -- 172.21.15.60:0/3893392164 >> 172.21.15.60:0/3893392164 conn(0x7f6be40670b0 msgr2=0x7f6be415d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.795+0000 7f6beba47640 1 -- 172.21.15.60:0/3893392164 shutdown_connections 2024-04-25T10:13:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.795+0000 7f6beba47640 1 -- 172.21.15.60:0/3893392164 wait complete. 2024-04-25T10:13:59.817 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:59.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d0e115640 1 Processor -- start 2024-04-25T10:13:59.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d0e115640 1 -- start start 2024-04-25T10:13:59.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d0e115640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1d080cf8a0 0x7f1d0815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d0e115640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1d080cd3d0 0x7f1d0815be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d0e115640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d080cd890 con 0x7f1d0815dd30 2024-04-25T10:13:59.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d0e115640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d080ce4a0 con 0x7f1d080cd3d0 2024-04-25T10:13:59.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d0e115640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d080cdb70 con 0x7f1d080cf8a0 2024-04-25T10:13:59.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d077fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1d0815dd30 0x7f1d080cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38576/0 (socket says 172.21.15.60:38576) 2024-04-25T10:13:59.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d077fe640 1 -- 172.21.15.60:0/735368007 learned_addr learned my addr 172.21.15.60:0/735368007 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:13:59.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d07fff640 1 --2- 172.21.15.60:0/735368007 >> v2:172.21.15.60:3300/0 conn(0x7f1d080cd3d0 0x7f1d0815be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d067fc640 1 -- 172.21.15.60:0/735368007 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276360550 0 0) 0x7f1d080cd890 con 0x7f1d0815dd30 2024-04-25T10:13:59.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d067fc640 1 -- 172.21.15.60:0/735368007 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cf0003530 con 0x7f1d0815dd30 2024-04-25T10:13:59.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d06ffd640 1 --2- 172.21.15.60:0/735368007 >> v2:172.21.15.100:3300/0 conn(0x7f1d080cf8a0 0x7f1d0815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d07fff640 1 -- 172.21.15.60:0/735368007 >> v2:172.21.15.100:3300/0 conn(0x7f1d080cf8a0 msgr2=0x7f1d0815ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:59.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d07fff640 1 --2- 172.21.15.60:0/735368007 >> v2:172.21.15.100:3300/0 conn(0x7f1d080cf8a0 0x7f1d0815ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d07fff640 1 -- 172.21.15.60:0/735368007 >> v1:172.21.15.60:6789/0 conn(0x7f1d0815dd30 legacy=0x7f1d080cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d07fff640 1 -- 172.21.15.60:0/735368007 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d080cde50 con 0x7f1d080cd3d0 2024-04-25T10:13:59.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.935+0000 7f1d06ffd640 1 --2- 172.21.15.60:0/735368007 >> v2:172.21.15.100:3300/0 conn(0x7f1d080cf8a0 0x7f1d0815ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:59.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d07fff640 1 --2- 172.21.15.60:0/735368007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080cd3d0 0x7f1d0815be20 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f1d08072550 tx=0x7f1d0001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c9221782984781 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d067fc640 1 -- 172.21.15.60:0/735368007 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d000b2070 con 0x7f1d080cd3d0 2024-04-25T10:13:59.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d067fc640 1 -- 172.21.15.60:0/735368007 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d000a5950 con 0x7f1d080cd3d0 2024-04-25T10:13:59.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d067fc640 1 -- 172.21.15.60:0/735368007 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d000a5d20 con 0x7f1d080cd3d0 2024-04-25T10:13:59.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d0e115640 1 -- 172.21.15.60:0/735368007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080cd3d0 msgr2=0x7f1d0815be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d0e115640 1 --2- 172.21.15.60:0/735368007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080cd3d0 0x7f1d0815be20 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f1d08072550 tx=0x7f1d0001c870 comp rx=0 tx=0).stop 2024-04-25T10:13:59.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d0e115640 1 -- 172.21.15.60:0/735368007 shutdown_connections 2024-04-25T10:13:59.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d0e115640 1 --2- 172.21.15.60:0/735368007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080cd3d0 0x7f1d0815be20 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d0e115640 1 --2- 172.21.15.60:0/735368007 >> v2:172.21.15.100:3300/0 conn(0x7f1d080cf8a0 0x7f1d0815ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d0e115640 1 -- 172.21.15.60:0/735368007 >> 172.21.15.60:0/735368007 conn(0x7f1d080670b0 msgr2=0x7f1d0806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d0e115640 1 -- 172.21.15.60:0/735368007 shutdown_connections 2024-04-25T10:13:59.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d0e115640 1 -- 172.21.15.60:0/735368007 wait complete. 2024-04-25T10:13:59.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d0e115640 1 Processor -- start 2024-04-25T10:13:59.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d0e115640 1 -- start start 2024-04-25T10:13:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d0e115640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080cd3d0 0x7f1d0813f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d0e115640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d080cf8a0 0x7f1d0813ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d0e115640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d0815dd30 0x7f1d080a8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d0e115640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0815cf00 con 0x7f1d080cd3d0 2024-04-25T10:13:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d0e115640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0815cd80 con 0x7f1d0815dd30 2024-04-25T10:13:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d0e115640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0815d080 con 0x7f1d080cf8a0 2024-04-25T10:13:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080cd3d0 0x7f1d0813f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080cd3d0 0x7f1d0813f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47726/0 (socket says 172.21.15.60:47726) 2024-04-25T10:13:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.939+0000 7f1d077fe640 1 -- 172.21.15.60:0/2228906534 learned_addr learned my addr 172.21.15.60:0/2228906534 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:13:59.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1d07fff640 1 --2- 172.21.15.60:0/2228906534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d0815dd30 0x7f1d080a8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1d06ffd640 1 --2- 172.21.15.60:0/2228906534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d080cf8a0 0x7f1d0813ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1d077fe640 1 -- 172.21.15.60:0/2228906534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d080cf8a0 msgr2=0x7f1d0813ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1d077fe640 1 --2- 172.21.15.60:0/2228906534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d080cf8a0 0x7f1d0813ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1d077fe640 1 -- 172.21.15.60:0/2228906534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d0815dd30 msgr2=0x7f1d080a8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1d077fe640 1 --2- 172.21.15.60:0/2228906534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d0815dd30 0x7f1d080a8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1d077fe640 1 -- 172.21.15.60:0/2228906534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0813f440 con 0x7f1d080cd3d0 2024-04-25T10:13:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1d06ffd640 1 --2- 172.21.15.60:0/2228906534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d080cf8a0 0x7f1d0813ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1d077fe640 1 --2- 172.21.15.60:0/2228906534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080cd3d0 0x7f1d0813f730 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800c3c0 tx=0x7f1cf800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1ceffff640 1 -- 172.21.15.60:0/2228906534 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8017070 con 0x7f1d080cd3d0 2024-04-25T10:13:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1ceffff640 1 -- 172.21.15.60:0/2228906534 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf8005cf0 con 0x7f1d080cd3d0 2024-04-25T10:13:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1ceffff640 1 -- 172.21.15.60:0/2228906534 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80072f0 con 0x7f1d080cd3d0 2024-04-25T10:13:59.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1d0e115640 1 -- 172.21.15.60:0/2228906534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d080a9430 con 0x7f1d080cd3d0 2024-04-25T10:13:59.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1d0e115640 1 -- 172.21.15.60:0/2228906534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d080a99f0 con 0x7f1d080cd3d0 2024-04-25T10:13:59.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1d0e115640 1 -- 172.21.15.60:0/2228906534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d08074770 con 0x7f1d080cd3d0 2024-04-25T10:13:59.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1ceffff640 1 -- 172.21.15.60:0/2228906534 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1cf801a070 con 0x7f1d080cd3d0 2024-04-25T10:13:59.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1ceffff640 1 --2- 172.21.15.60:0/2228906534 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cf4066f60 0x7f1cf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1ceffff640 1 -- 172.21.15.60:0/2228906534 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f1cf8084520 con 0x7f1d080cd3d0 2024-04-25T10:13:59.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1d06ffd640 1 --2- 172.21.15.60:0/2228906534 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cf4066f60 0x7f1cf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.943+0000 7f1d06ffd640 1 --2- 172.21.15.60:0/2228906534 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cf4066f60 0x7f1cf4069420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1d08142850 tx=0x7f1cfc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:13:59.947+0000 7f1ceffff640 1 -- 172.21.15.60:0/2228906534 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1cf8055440 con 0x7f1d080cd3d0 2024-04-25T10:14:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.107+0000 7f1d0e115640 1 -- 172.21.15.60:0/2228906534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d080ae4b0 con 0x7f1d080cd3d0 2024-04-25T10:14:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.107+0000 7f1ceffff640 1 -- 172.21.15.60:0/2228906534 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1cf8054d40 con 0x7f1d080cd3d0 2024-04-25T10:14:00.113 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:14:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.111+0000 7f1d0e115640 1 -- 172.21.15.60:0/2228906534 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cf4066f60 msgr2=0x7f1cf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.111+0000 7f1d0e115640 1 --2- 172.21.15.60:0/2228906534 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cf4066f60 0x7f1cf4069420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1d08142850 tx=0x7f1cfc008040 comp rx=0 tx=0).stop 2024-04-25T10:14:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.111+0000 7f1d0e115640 1 -- 172.21.15.60:0/2228906534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080cd3d0 msgr2=0x7f1d0813f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.111+0000 7f1d0e115640 1 --2- 172.21.15.60:0/2228906534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080cd3d0 0x7f1d0813f730 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800c3c0 tx=0x7f1cf800c880 comp rx=0 tx=0).stop 2024-04-25T10:14:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.111+0000 7f1d0e115640 1 -- 172.21.15.60:0/2228906534 shutdown_connections 2024-04-25T10:14:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.111+0000 7f1d0e115640 1 --2- 172.21.15.60:0/2228906534 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cf4066f60 0x7f1cf4069420 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.111+0000 7f1d0e115640 1 --2- 172.21.15.60:0/2228906534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d0815dd30 0x7f1d080a8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.111+0000 7f1d0e115640 1 --2- 172.21.15.60:0/2228906534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d080cf8a0 0x7f1d0813ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.111+0000 7f1d0e115640 1 --2- 172.21.15.60:0/2228906534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d080cd3d0 0x7f1d0813f730 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.111+0000 7f1d0e115640 1 -- 172.21.15.60:0/2228906534 >> 172.21.15.60:0/2228906534 conn(0x7f1d080670b0 msgr2=0x7f1d08071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.111+0000 7f1d0e115640 1 -- 172.21.15.60:0/2228906534 shutdown_connections 2024-04-25T10:14:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.111+0000 7f1d0e115640 1 -- 172.21.15.60:0/2228906534 wait complete. 2024-04-25T10:14:00.131 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:14:00.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7ae207a640 1 Processor -- start 2024-04-25T10:14:00.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7ae207a640 1 -- start start 2024-04-25T10:14:00.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7ae207a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7adc15bc90 0x7f7adc15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7ae207a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7adc0b4660 0x7f7adc15c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7ae207a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7adc071ca0 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7ae207a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7adc0cc370 con 0x7f7adc0b4660 2024-04-25T10:14:00.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7ae207a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc0cca40 con 0x7f7adc15bc90 2024-04-25T10:14:00.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7adb7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7adc0ce7a0 0x7f7adc0ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38586/0 (socket says 172.21.15.60:38586) 2024-04-25T10:14:00.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7adb7fe640 1 -- 172.21.15.60:0/2393145380 learned_addr learned my addr 172.21.15.60:0/2393145380 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:00.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7adaffd640 1 --2- 172.21.15.60:0/2393145380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7adc15bc90 0x7f7adc15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7adbfff640 1 --2- 172.21.15.60:0/2393145380 >> v2:172.21.15.100:3300/0 conn(0x7f7adc0b4660 0x7f7adc15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7ada7fc640 1 -- 172.21.15.60:0/2393145380 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3844937795 0 0) 0x7f7adc071ca0 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7ada7fc640 1 -- 172.21.15.60:0/2393145380 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ac8003530 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7ada7fc640 1 -- 172.21.15.60:0/2393145380 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 387623148 0 0) 0x7f7ac8003530 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7adbfff640 1 -- 172.21.15.60:0/2393145380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7adc15bc90 msgr2=0x7f7adc15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:00.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7adbfff640 1 --2- 172.21.15.60:0/2393145380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7adc15bc90 0x7f7adc15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7adbfff640 1 -- 172.21.15.60:0/2393145380 >> v1:172.21.15.60:6789/0 conn(0x7f7adc0ce7a0 legacy=0x7f7adc0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.243+0000 7f7adbfff640 1 -- 172.21.15.60:0/2393145380 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc0ccd20 con 0x7f7adc0b4660 2024-04-25T10:14:00.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.247+0000 7f7adbfff640 1 --2- 172.21.15.60:0/2393145380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc0b4660 0x7f7adc15c760 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009d90 tx=0x7f7ad401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c5121571385839f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:00.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.247+0000 7f7ada7fc640 1 -- 172.21.15.60:0/2393145380 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40ac070 con 0x7f7adc0b4660 2024-04-25T10:14:00.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.247+0000 7f7ada7fc640 1 -- 172.21.15.60:0/2393145380 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad40a64a0 con 0x7f7adc0b4660 2024-04-25T10:14:00.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.247+0000 7f7ada7fc640 1 -- 172.21.15.60:0/2393145380 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc001680 con 0x7f7adc0b4660 2024-04-25T10:14:00.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.247+0000 7f7ae207a640 1 -- 172.21.15.60:0/2393145380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc0b4660 msgr2=0x7f7adc15c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.247+0000 7f7ae207a640 1 --2- 172.21.15.60:0/2393145380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc0b4660 0x7f7adc15c760 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009d90 tx=0x7f7ad401c890 comp rx=0 tx=0).stop 2024-04-25T10:14:00.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.247+0000 7f7ae207a640 1 -- 172.21.15.60:0/2393145380 shutdown_connections 2024-04-25T10:14:00.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.247+0000 7f7ae207a640 1 --2- 172.21.15.60:0/2393145380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc0b4660 0x7f7adc15c760 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.247+0000 7f7ae207a640 1 --2- 172.21.15.60:0/2393145380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7adc15bc90 0x7f7adc15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.247+0000 7f7ae207a640 1 -- 172.21.15.60:0/2393145380 >> 172.21.15.60:0/2393145380 conn(0x7f7adc0b12d0 msgr2=0x7f7adc0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.247+0000 7f7ae207a640 1 -- 172.21.15.60:0/2393145380 shutdown_connections 2024-04-25T10:14:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.247+0000 7f7ae207a640 1 -- 172.21.15.60:0/2393145380 wait complete. 2024-04-25T10:14:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.247+0000 7f7ae207a640 1 Processor -- start 2024-04-25T10:14:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.247+0000 7f7ae207a640 1 -- start start 2024-04-25T10:14:00.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7ae207a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7adc0ce7a0 0x7f7adc13cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7ae207a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7adc15bc90 0x7f7adc1573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7ae207a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc1510a0 0x7f7adc151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7ae207a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc0b3db0 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7ae207a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc0b4490 con 0x7f7adc1510a0 2024-04-25T10:14:00.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7ae207a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc1685c0 con 0x7f7adc15bc90 2024-04-25T10:14:00.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7adb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7adc0ce7a0 0x7f7adc13cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7adb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7adc0ce7a0 0x7f7adc13cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47728/0 (socket says 172.21.15.60:47728) 2024-04-25T10:14:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7adb7fe640 1 -- 172.21.15.60:0/1246346681 learned_addr learned my addr 172.21.15.60:0/1246346681 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7adaffd640 1 --2- 172.21.15.60:0/1246346681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7adc15bc90 0x7f7adc1573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7adbfff640 1 --2- 172.21.15.60:0/1246346681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc1510a0 0x7f7adc151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7adb7fe640 1 -- 172.21.15.60:0/1246346681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7adc15bc90 msgr2=0x7f7adc1573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7adb7fe640 1 --2- 172.21.15.60:0/1246346681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7adc15bc90 0x7f7adc1573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7adb7fe640 1 -- 172.21.15.60:0/1246346681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc1510a0 msgr2=0x7f7adc151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7adb7fe640 1 --2- 172.21.15.60:0/1246346681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc1510a0 0x7f7adc151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7adb7fe640 1 -- 172.21.15.60:0/1246346681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc157aa0 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7adb7fe640 1 --2- 172.21.15.60:0/1246346681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7adc0ce7a0 0x7f7adc13cf60 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00c6d0 tx=0x7f7acc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7ad3fff640 1 -- 172.21.15.60:0/1246346681 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc017070 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7ad3fff640 1 -- 172.21.15.60:0/1246346681 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7acc005ae0 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7ae207a640 1 -- 172.21.15.60:0/1246346681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7adc157d80 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7ad3fff640 1 -- 172.21.15.60:0/1246346681 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc007070 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7ae207a640 1 -- 172.21.15.60:0/1246346681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7adc158550 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7ae207a640 1 -- 172.21.15.60:0/1246346681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7adc074770 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7ad3fff640 1 -- 172.21.15.60:0/1246346681 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7acc01a070 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7ad3fff640 1 --2- 172.21.15.60:0/1246346681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7abc066f60 0x7f7abc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7ad3fff640 1 -- 172.21.15.60:0/1246346681 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f7acc0849e0 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.251+0000 7f7adaffd640 1 --2- 172.21.15.60:0/1246346681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7abc066f60 0x7f7abc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.255+0000 7f7adaffd640 1 --2- 172.21.15.60:0/1246346681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7abc066f60 0x7f7abc069420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0afc10 tx=0x7f7ac4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:00.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.255+0000 7f7ad3fff640 1 -- 172.21.15.60:0/1246346681 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7acc021020 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.427+0000 7f7ae207a640 1 -- 172.21.15.60:0/1246346681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7adc157f50 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.427+0000 7f7ad3fff640 1 -- 172.21.15.60:0/1246346681 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v86) v1 ==== 80+0+244 (secure 0 0 0) 0x7f7acc055970 con 0x7f7adc0ce7a0 2024-04-25T10:14:00.431 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6835/2817879567 2024-04-26T10:13:19.192767+0000 2024-04-25T10:14:00.432 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6833/2817879567 2024-04-26T10:13:19.192767+0000 2024-04-25T10:14:00.432 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6835/2455562778 2024-04-26T10:11:58.136789+0000 2024-04-25T10:14:00.432 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6833/2455562778 2024-04-26T10:11:58.136789+0000 2024-04-25T10:14:00.432 INFO:teuthology.orchestra.run.smithi060.stderr:listed 4 entries 2024-04-25T10:14:00.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.427+0000 7f7ae207a640 1 -- 172.21.15.60:0/1246346681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7abc066f60 msgr2=0x7f7abc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.427+0000 7f7ae207a640 1 --2- 172.21.15.60:0/1246346681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7abc066f60 0x7f7abc069420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0afc10 tx=0x7f7ac4008040 comp rx=0 tx=0).stop 2024-04-25T10:14:00.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.427+0000 7f7ae207a640 1 -- 172.21.15.60:0/1246346681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7adc0ce7a0 msgr2=0x7f7adc13cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.427+0000 7f7ae207a640 1 --2- 172.21.15.60:0/1246346681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7adc0ce7a0 0x7f7adc13cf60 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00c6d0 tx=0x7f7acc00cb90 comp rx=0 tx=0).stop 2024-04-25T10:14:00.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.427+0000 7f7ae207a640 1 -- 172.21.15.60:0/1246346681 shutdown_connections 2024-04-25T10:14:00.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.427+0000 7f7ae207a640 1 --2- 172.21.15.60:0/1246346681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7abc066f60 0x7f7abc069420 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.427+0000 7f7ae207a640 1 --2- 172.21.15.60:0/1246346681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7adc1510a0 0x7f7adc151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.427+0000 7f7ae207a640 1 --2- 172.21.15.60:0/1246346681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7adc15bc90 0x7f7adc1573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.427+0000 7f7ae207a640 1 --2- 172.21.15.60:0/1246346681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7adc0ce7a0 0x7f7adc13cf60 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.427+0000 7f7ae207a640 1 -- 172.21.15.60:0/1246346681 >> 172.21.15.60:0/1246346681 conn(0x7f7adc0b12d0 msgr2=0x7f7adc071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:00.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.427+0000 7f7ae207a640 1 -- 172.21.15.60:0/1246346681 shutdown_connections 2024-04-25T10:14:00.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.431+0000 7f7ae207a640 1 -- 172.21.15.60:0/1246346681 wait complete. 2024-04-25T10:14:00.451 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:14:00.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:00.452 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:00.462 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[47250]: fuse finished with error 0 and tester_r 0 2024-04-25T10:14:00.463 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:14:00.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:00.464 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:14:00.555 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:14:00.555 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:00.561 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:00.561 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:14:00.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:00.561 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:14:00.607 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:14:00.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:00.608 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:14:00.609 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:00.614 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:00.614 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:14:00.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:00.615 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:14:00.664 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:14:00.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:00.666 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:14:00.666 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:00.672 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:00.672 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:14:00.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:00.673 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:14:00.723 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:14:00.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:00.725 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:00.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb2de0d640 1 Processor -- start 2024-04-25T10:14:00.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb2de0d640 1 -- start start 2024-04-25T10:14:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb2de0d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb2815dc90 0x7ffb280b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb2de0d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffb280b2b10 0x7ffb28071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb2de0d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffb280ae7c0 0x7ffb280717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb2de0d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb280accc0 con 0x7ffb280b2b10 2024-04-25T10:14:00.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb2de0d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb280ad260 con 0x7ffb280ae7c0 2024-04-25T10:14:00.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb2de0d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb28071fd0 con 0x7ffb2815dc90 2024-04-25T10:14:00.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb26ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffb280b2b10 0x7ffb28071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb26ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffb280b2b10 0x7ffb28071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47744/0 (socket says 172.21.15.60:47744) 2024-04-25T10:14:00.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb26ffd640 1 -- 172.21.15.60:0/832688616 learned_addr learned my addr 172.21.15.60:0/832688616 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:00.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb277fe640 1 --2- 172.21.15.60:0/832688616 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb2815dc90 0x7ffb280b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb27fff640 1 --2- 172.21.15.60:0/832688616 >> v2:172.21.15.100:3300/0 conn(0x7ffb280ae7c0 0x7ffb280717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb26ffd640 1 -- 172.21.15.60:0/832688616 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb2815dc90 msgr2=0x7ffb280b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:00.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb26ffd640 1 --2- 172.21.15.60:0/832688616 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb2815dc90 0x7ffb280b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb26ffd640 1 -- 172.21.15.60:0/832688616 >> v2:172.21.15.100:3300/0 conn(0x7ffb280ae7c0 msgr2=0x7ffb280717d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:00.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb26ffd640 1 --2- 172.21.15.60:0/832688616 >> v2:172.21.15.100:3300/0 conn(0x7ffb280ae7c0 0x7ffb280717d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb26ffd640 1 -- 172.21.15.60:0/832688616 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb28072290 con 0x7ffb280b2b10 2024-04-25T10:14:00.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.855+0000 7ffb27fff640 1 --2- 172.21.15.60:0/832688616 >> v2:172.21.15.100:3300/0 conn(0x7ffb280ae7c0 0x7ffb280717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:00.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb26ffd640 1 --2- 172.21.15.60:0/832688616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb280b2b10 0x7ffb28071290 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7ffb2806cf50 tx=0x7ffb1801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bcd00e3b0d52c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:00.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb267fc640 1 -- 172.21.15.60:0/832688616 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180ab5f0 con 0x7ffb280b2b10 2024-04-25T10:14:00.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb267fc640 1 -- 172.21.15.60:0/832688616 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb180b0070 con 0x7ffb280b2b10 2024-04-25T10:14:00.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb267fc640 1 -- 172.21.15.60:0/832688616 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180ab9b0 con 0x7ffb280b2b10 2024-04-25T10:14:00.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 -- 172.21.15.60:0/832688616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb280b2b10 msgr2=0x7ffb28071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 --2- 172.21.15.60:0/832688616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb280b2b10 0x7ffb28071290 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7ffb2806cf50 tx=0x7ffb1801c520 comp rx=0 tx=0).stop 2024-04-25T10:14:00.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 -- 172.21.15.60:0/832688616 shutdown_connections 2024-04-25T10:14:00.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 --2- 172.21.15.60:0/832688616 >> v2:172.21.15.100:3300/0 conn(0x7ffb280ae7c0 0x7ffb280717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 --2- 172.21.15.60:0/832688616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb280b2b10 0x7ffb28071290 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 --2- 172.21.15.60:0/832688616 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb2815dc90 0x7ffb280b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 -- 172.21.15.60:0/832688616 >> 172.21.15.60:0/832688616 conn(0x7ffb280b12d0 msgr2=0x7ffb280b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:00.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 -- 172.21.15.60:0/832688616 shutdown_connections 2024-04-25T10:14:00.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 -- 172.21.15.60:0/832688616 wait complete. 2024-04-25T10:14:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 Processor -- start 2024-04-25T10:14:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 -- start start 2024-04-25T10:14:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb280ae7c0 0x7ffb28145900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb2815dc90 0x7ffb281fa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb281fab00 0x7ffb28149420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb28161a20 con 0x7ffb281fab00 2024-04-25T10:14:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb281618a0 con 0x7ffb2815dc90 2024-04-25T10:14:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.859+0000 7ffb2de0d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb28161ba0 con 0x7ffb280ae7c0 2024-04-25T10:14:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb27fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb281fab00 0x7ffb28149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb27fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb281fab00 0x7ffb28149420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47758/0 (socket says 172.21.15.60:47758) 2024-04-25T10:14:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb27fff640 1 -- 172.21.15.60:0/2631120555 learned_addr learned my addr 172.21.15.60:0/2631120555 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb26ffd640 1 --2- 172.21.15.60:0/2631120555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb2815dc90 0x7ffb281fa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb277fe640 1 --2- 172.21.15.60:0/2631120555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb280ae7c0 0x7ffb28145900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb27fff640 1 -- 172.21.15.60:0/2631120555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb280ae7c0 msgr2=0x7ffb28145900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb27fff640 1 --2- 172.21.15.60:0/2631120555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb280ae7c0 0x7ffb28145900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb27fff640 1 -- 172.21.15.60:0/2631120555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb2815dc90 msgr2=0x7ffb281fa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb27fff640 1 --2- 172.21.15.60:0/2631120555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb2815dc90 0x7ffb281fa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb27fff640 1 -- 172.21.15.60:0/2631120555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb281f6e70 con 0x7ffb281fab00 2024-04-25T10:14:00.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb27fff640 1 --2- 172.21.15.60:0/2631120555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb281fab00 0x7ffb28149420 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000ce70 tx=0x7ffb2000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:00.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb1ffff640 1 -- 172.21.15.60:0/2631120555 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2001a070 con 0x7ffb281fab00 2024-04-25T10:14:00.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb1ffff640 1 -- 172.21.15.60:0/2631120555 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2000d840 con 0x7ffb281fab00 2024-04-25T10:14:00.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb2de0d640 1 -- 172.21.15.60:0/2631120555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2814c570 con 0x7ffb281fab00 2024-04-25T10:14:00.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb1ffff640 1 -- 172.21.15.60:0/2631120555 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2000dc30 con 0x7ffb281fab00 2024-04-25T10:14:00.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb2de0d640 1 -- 172.21.15.60:0/2631120555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2814cb30 con 0x7ffb281fab00 2024-04-25T10:14:00.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb2de0d640 1 -- 172.21.15.60:0/2631120555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb28074770 con 0x7ffb281fab00 2024-04-25T10:14:00.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb1ffff640 1 -- 172.21.15.60:0/2631120555 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb2001d070 con 0x7ffb281fab00 2024-04-25T10:14:00.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb1ffff640 1 --2- 172.21.15.60:0/2631120555 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb08066f90 0x7ffb08069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb1ffff640 1 -- 172.21.15.60:0/2631120555 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7ffb200858e0 con 0x7ffb281fab00 2024-04-25T10:14:00.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb277fe640 1 --2- 172.21.15.60:0/2631120555 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb08066f90 0x7ffb08069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.863+0000 7ffb277fe640 1 --2- 172.21.15.60:0/2631120555 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb08066f90 0x7ffb08069450 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000b930 tx=0x7ffb10008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:00.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:00.867+0000 7ffb1ffff640 1 -- 172.21.15.60:0/2631120555 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb20056780 con 0x7ffb281fab00 2024-04-25T10:14:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.067+0000 7ffb2de0d640 1 -- 172.21.15.60:0/2631120555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb28153300 con 0x7ffb281fab00 2024-04-25T10:14:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.067+0000 7ffb1ffff640 1 -- 172.21.15.60:0/2631120555 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4944 (secure 0 0 0) 0x7ffb20056080 con 0x7ffb281fab00 2024-04-25T10:14:01.072 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:01.072 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:13:22.221010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8256":{"gid":8256,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/3807131024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3807131024},{"type":"v1","addr":"172.21.15.60:6835","nonce":3807131024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:14:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 68 2024-04-25T10:14:01.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.067+0000 7ffb2de0d640 1 -- 172.21.15.60:0/2631120555 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb08066f90 msgr2=0x7ffb08069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.067+0000 7ffb2de0d640 1 --2- 172.21.15.60:0/2631120555 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb08066f90 0x7ffb08069450 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000b930 tx=0x7ffb10008040 comp rx=0 tx=0).stop 2024-04-25T10:14:01.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.067+0000 7ffb2de0d640 1 -- 172.21.15.60:0/2631120555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb281fab00 msgr2=0x7ffb28149420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.067+0000 7ffb2de0d640 1 --2- 172.21.15.60:0/2631120555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb281fab00 0x7ffb28149420 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000ce70 tx=0x7ffb2000a710 comp rx=0 tx=0).stop 2024-04-25T10:14:01.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.067+0000 7ffb2de0d640 1 -- 172.21.15.60:0/2631120555 shutdown_connections 2024-04-25T10:14:01.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.067+0000 7ffb2de0d640 1 --2- 172.21.15.60:0/2631120555 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb08066f90 0x7ffb08069450 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.067+0000 7ffb2de0d640 1 --2- 172.21.15.60:0/2631120555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb281fab00 0x7ffb28149420 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.067+0000 7ffb2de0d640 1 --2- 172.21.15.60:0/2631120555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb2815dc90 0x7ffb281fa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.067+0000 7ffb2de0d640 1 --2- 172.21.15.60:0/2631120555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb280ae7c0 0x7ffb28145900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.067+0000 7ffb2de0d640 1 -- 172.21.15.60:0/2631120555 >> 172.21.15.60:0/2631120555 conn(0x7ffb280b12d0 msgr2=0x7ffb280cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:01.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.071+0000 7ffb2de0d640 1 -- 172.21.15.60:0/2631120555 shutdown_connections 2024-04-25T10:14:01.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.071+0000 7ffb2de0d640 1 -- 172.21.15.60:0/2631120555 wait complete. 2024-04-25T10:14:01.092 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:01.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f9af7640 1 Processor -- start 2024-04-25T10:14:01.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f9af7640 1 -- start start 2024-04-25T10:14:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f9af7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f95f40aecb0 0x7f95f406b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f9af7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95f406fea0 0x7f95f415e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f9af7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f95f415ec80 0x7f95f406e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f9af7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f40af090 con 0x7f95f40aecb0 2024-04-25T10:14:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f9af7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f415e2b0 con 0x7f95f415ec80 2024-04-25T10:14:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f9af7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f415dfc0 con 0x7f95f406fea0 2024-04-25T10:14:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f95f40aecb0 0x7f95f406b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f95f40aecb0 0x7f95f406b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47768/0 (socket says 172.21.15.60:47768) 2024-04-25T10:14:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f37fe640 1 -- 172.21.15.60:0/2851505115 learned_addr learned my addr 172.21.15.60:0/2851505115 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f2ffd640 1 --2- 172.21.15.60:0/2851505115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95f406fea0 0x7f95f415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f3fff640 1 --2- 172.21.15.60:0/2851505115 >> v2:172.21.15.100:3300/0 conn(0x7f95f415ec80 0x7f95f406e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f37fe640 1 -- 172.21.15.60:0/2851505115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95f406fea0 msgr2=0x7f95f415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:01.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f37fe640 1 --2- 172.21.15.60:0/2851505115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95f406fea0 0x7f95f415e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f37fe640 1 -- 172.21.15.60:0/2851505115 >> v2:172.21.15.100:3300/0 conn(0x7f95f415ec80 msgr2=0x7f95f406e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:01.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f37fe640 1 --2- 172.21.15.60:0/2851505115 >> v2:172.21.15.100:3300/0 conn(0x7f95f415ec80 0x7f95f406e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f37fe640 1 -- 172.21.15.60:0/2851505115 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f40b5360 con 0x7f95f40aecb0 2024-04-25T10:14:01.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f2ffd640 1 --2- 172.21.15.60:0/2851505115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95f406fea0 0x7f95f415e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:01.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.211+0000 7f95f3fff640 1 --2- 172.21.15.60:0/2851505115 >> v2:172.21.15.100:3300/0 conn(0x7f95f415ec80 0x7f95f406e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:01.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f37fe640 1 --2- 172.21.15.60:0/2851505115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95f40aecb0 0x7f95f406b620 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f95f406cd30 tx=0x7f95e401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10088ebc0383c871 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:01.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f27fc640 1 -- 172.21.15.60:0/2851505115 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e40b0070 con 0x7f95f40aecb0 2024-04-25T10:14:01.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f27fc640 1 -- 172.21.15.60:0/2851505115 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e40ab600 con 0x7f95f40aecb0 2024-04-25T10:14:01.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f27fc640 1 -- 172.21.15.60:0/2851505115 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e40ab9d0 con 0x7f95f40aecb0 2024-04-25T10:14:01.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f9af7640 1 -- 172.21.15.60:0/2851505115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95f40aecb0 msgr2=0x7f95f406b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f9af7640 1 --2- 172.21.15.60:0/2851505115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95f40aecb0 0x7f95f406b620 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f95f406cd30 tx=0x7f95e401c520 comp rx=0 tx=0).stop 2024-04-25T10:14:01.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f9af7640 1 -- 172.21.15.60:0/2851505115 shutdown_connections 2024-04-25T10:14:01.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f9af7640 1 --2- 172.21.15.60:0/2851505115 >> v2:172.21.15.100:3300/0 conn(0x7f95f415ec80 0x7f95f406e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f9af7640 1 --2- 172.21.15.60:0/2851505115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95f406fea0 0x7f95f415e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f9af7640 1 --2- 172.21.15.60:0/2851505115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95f40aecb0 0x7f95f406b620 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f9af7640 1 -- 172.21.15.60:0/2851505115 >> 172.21.15.60:0/2851505115 conn(0x7f95f40b5aa0 msgr2=0x7f95f40b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:01.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f9af7640 1 -- 172.21.15.60:0/2851505115 shutdown_connections 2024-04-25T10:14:01.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f9af7640 1 -- 172.21.15.60:0/2851505115 wait complete. 2024-04-25T10:14:01.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f9af7640 1 Processor -- start 2024-04-25T10:14:01.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f9af7640 1 -- start start 2024-04-25T10:14:01.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f9af7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95f406fea0 0x7f95f4148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f9af7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95f40aecb0 0x7f95f4154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f9af7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95f415ec80 0x7f95f414b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.215+0000 7f95f9af7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f4157a30 con 0x7f95f406fea0 2024-04-25T10:14:01.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f9af7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f41578b0 con 0x7f95f40aecb0 2024-04-25T10:14:01.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f9af7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f4157bb0 con 0x7f95f415ec80 2024-04-25T10:14:01.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95f406fea0 0x7f95f4148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95f406fea0 0x7f95f4148c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47774/0 (socket says 172.21.15.60:47774) 2024-04-25T10:14:01.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f37fe640 1 -- 172.21.15.60:0/3970056750 learned_addr learned my addr 172.21.15.60:0/3970056750 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:01.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f2ffd640 1 --2- 172.21.15.60:0/3970056750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95f40aecb0 0x7f95f4154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f3fff640 1 --2- 172.21.15.60:0/3970056750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95f415ec80 0x7f95f414b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f37fe640 1 -- 172.21.15.60:0/3970056750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95f415ec80 msgr2=0x7f95f414b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f37fe640 1 --2- 172.21.15.60:0/3970056750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95f415ec80 0x7f95f414b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f37fe640 1 -- 172.21.15.60:0/3970056750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95f40aecb0 msgr2=0x7f95f4154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f37fe640 1 --2- 172.21.15.60:0/3970056750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95f40aecb0 0x7f95f4154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f37fe640 1 -- 172.21.15.60:0/3970056750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f4154980 con 0x7f95f406fea0 2024-04-25T10:14:01.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f2ffd640 1 --2- 172.21.15.60:0/3970056750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95f40aecb0 0x7f95f4154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:01.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f37fe640 1 --2- 172.21.15.60:0/3970056750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95f406fea0 0x7f95f4148c50 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f95e40ae040 tx=0x7f95e401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:01.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95dbfff640 1 -- 172.21.15.60:0/3970056750 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e40b0040 con 0x7f95f406fea0 2024-04-25T10:14:01.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95dbfff640 1 -- 172.21.15.60:0/3970056750 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e40bc070 con 0x7f95f406fea0 2024-04-25T10:14:01.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95dbfff640 1 -- 172.21.15.60:0/3970056750 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e40b6880 con 0x7f95f406fea0 2024-04-25T10:14:01.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f9af7640 1 -- 172.21.15.60:0/3970056750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f4159130 con 0x7f95f406fea0 2024-04-25T10:14:01.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f9af7640 1 -- 172.21.15.60:0/3970056750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f414a850 con 0x7f95f406fea0 2024-04-25T10:14:01.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f9af7640 1 -- 172.21.15.60:0/3970056750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f4074770 con 0x7f95f406fea0 2024-04-25T10:14:01.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95dbfff640 1 -- 172.21.15.60:0/3970056750 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95e40ab4c0 con 0x7f95f406fea0 2024-04-25T10:14:01.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95dbfff640 1 --2- 172.21.15.60:0/3970056750 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f95e0066f60 0x7f95e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95dbfff640 1 -- 172.21.15.60:0/3970056750 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f95e411f540 con 0x7f95f406fea0 2024-04-25T10:14:01.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.219+0000 7f95f2ffd640 1 --2- 172.21.15.60:0/3970056750 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f95e0066f60 0x7f95e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.223+0000 7f95f2ffd640 1 --2- 172.21.15.60:0/3970056750 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f95e0066f60 0x7f95e0069420 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f95f41f9d00 tx=0x7f95e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:01.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.223+0000 7f95dbfff640 1 -- 172.21.15.60:0/3970056750 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95e4016110 con 0x7f95f406fea0 2024-04-25T10:14:01.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.383+0000 7f95f9af7640 1 -- 172.21.15.60:0/3970056750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95f414b590 con 0x7f95f406fea0 2024-04-25T10:14:01.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.387+0000 7f95dbfff640 1 -- 172.21.15.60:0/3970056750 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16981 (secure 0 0 0) 0x7f95e40f0480 con 0x7f95f406fea0 2024-04-25T10:14:01.391 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:01.392 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":86,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:13:19.192820+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:42.621502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:43.352679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/2817879567":"2024-04-26T10:13:19.192767+0000","172.21.15.60:6833/2817879567":"2024-04-26T10:13:19.192767+0000","172.21.15.60:6835/2455562778":"2024-04-26T10:11:58.136789+0000","172.21.15.60:6833/2455562778":"2024-04-26T10:11:58.136789+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:01.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.387+0000 7f95f9af7640 1 -- 172.21.15.60:0/3970056750 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f95e0066f60 msgr2=0x7f95e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.387+0000 7f95f9af7640 1 --2- 172.21.15.60:0/3970056750 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f95e0066f60 0x7f95e0069420 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f95f41f9d00 tx=0x7f95e8008040 comp rx=0 tx=0).stop 2024-04-25T10:14:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.387+0000 7f95f9af7640 1 -- 172.21.15.60:0/3970056750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95f406fea0 msgr2=0x7f95f4148c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.387+0000 7f95f9af7640 1 --2- 172.21.15.60:0/3970056750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95f406fea0 0x7f95f4148c50 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f95e40ae040 tx=0x7f95e401ad60 comp rx=0 tx=0).stop 2024-04-25T10:14:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.387+0000 7f95f9af7640 1 -- 172.21.15.60:0/3970056750 shutdown_connections 2024-04-25T10:14:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.387+0000 7f95f9af7640 1 --2- 172.21.15.60:0/3970056750 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f95e0066f60 0x7f95e0069420 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.387+0000 7f95f9af7640 1 --2- 172.21.15.60:0/3970056750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95f415ec80 0x7f95f414b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.387+0000 7f95f9af7640 1 --2- 172.21.15.60:0/3970056750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95f40aecb0 0x7f95f4154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.387+0000 7f95f9af7640 1 --2- 172.21.15.60:0/3970056750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95f406fea0 0x7f95f4148c50 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.387+0000 7f95f9af7640 1 -- 172.21.15.60:0/3970056750 >> 172.21.15.60:0/3970056750 conn(0x7f95f40b5aa0 msgr2=0x7f95f406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.387+0000 7f95f9af7640 1 -- 172.21.15.60:0/3970056750 shutdown_connections 2024-04-25T10:14:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.387+0000 7f95f9af7640 1 -- 172.21.15.60:0/3970056750 wait complete. 2024-04-25T10:14:01.412 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:42.621502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:43.352679+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:01.412 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:01.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.531+0000 7f6d29ab5640 1 Processor -- start 2024-04-25T10:14:01.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.531+0000 7f6d29ab5640 1 -- start start 2024-04-25T10:14:01.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.531+0000 7f6d29ab5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d2415b6f0 0x7f6d241620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.531+0000 7f6d29ab5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6d240ceef0 0x7f6d2415e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.531+0000 7f6d29ab5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d2415bad0 con 0x7f6d2415e0d0 2024-04-25T10:14:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.531+0000 7f6d29ab5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d24162610 con 0x7f6d240ceef0 2024-04-25T10:14:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.531+0000 7f6d29ab5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d24161c40 con 0x7f6d2415b6f0 2024-04-25T10:14:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.531+0000 7f6d23fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6d240ceef0 0x7f6d2415e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.531+0000 7f6d23fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6d240ceef0 0x7f6d2415e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47776/0 (socket says 172.21.15.60:47776) 2024-04-25T10:14:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.531+0000 7f6d23fff640 1 -- 172.21.15.60:0/2870278334 learned_addr learned my addr 172.21.15.60:0/2870278334 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.531+0000 7f6d237fe640 1 --2- 172.21.15.60:0/2870278334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d2415b6f0 0x7f6d241620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.531+0000 7f6d23fff640 1 -- 172.21.15.60:0/2870278334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d2415b6f0 msgr2=0x7f6d241620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.531+0000 7f6d23fff640 1 --2- 172.21.15.60:0/2870278334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d2415b6f0 0x7f6d241620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.531+0000 7f6d23fff640 1 -- 172.21.15.60:0/2870278334 >> v1:172.21.15.100:6789/0 conn(0x7f6d2415e0d0 legacy=0x7f6d240ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.531+0000 7f6d23fff640 1 -- 172.21.15.60:0/2870278334 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2415ebe0 con 0x7f6d240ceef0 2024-04-25T10:14:01.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.535+0000 7f6d23fff640 1 --2- 172.21.15.60:0/2870278334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d240ceef0 0x7f6d2415e510 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009e50 tx=0x7f6d1c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1cb602b17138e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:01.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.535+0000 7f6d227fc640 1 -- 172.21.15.60:0/2870278334 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c0a57f0 con 0x7f6d240ceef0 2024-04-25T10:14:01.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.535+0000 7f6d227fc640 1 -- 172.21.15.60:0/2870278334 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1c0a5990 con 0x7f6d240ceef0 2024-04-25T10:14:01.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.535+0000 7f6d227fc640 1 -- 172.21.15.60:0/2870278334 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c0a5d60 con 0x7f6d240ceef0 2024-04-25T10:14:01.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.535+0000 7f6d29ab5640 1 -- 172.21.15.60:0/2870278334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d240ceef0 msgr2=0x7f6d2415e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.535+0000 7f6d29ab5640 1 --2- 172.21.15.60:0/2870278334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d240ceef0 0x7f6d2415e510 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009e50 tx=0x7f6d1c01ca30 comp rx=0 tx=0).stop 2024-04-25T10:14:01.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.535+0000 7f6d29ab5640 1 -- 172.21.15.60:0/2870278334 shutdown_connections 2024-04-25T10:14:01.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.535+0000 7f6d29ab5640 1 --2- 172.21.15.60:0/2870278334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d240ceef0 0x7f6d2415e510 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.535+0000 7f6d29ab5640 1 --2- 172.21.15.60:0/2870278334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d2415b6f0 0x7f6d241620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.535+0000 7f6d29ab5640 1 -- 172.21.15.60:0/2870278334 >> 172.21.15.60:0/2870278334 conn(0x7f6d24072340 msgr2=0x7f6d24072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:01.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.535+0000 7f6d29ab5640 1 -- 172.21.15.60:0/2870278334 shutdown_connections 2024-04-25T10:14:01.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.535+0000 7f6d29ab5640 1 -- 172.21.15.60:0/2870278334 wait complete. 2024-04-25T10:14:01.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.535+0000 7f6d29ab5640 1 Processor -- start 2024-04-25T10:14:01.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.535+0000 7f6d29ab5640 1 -- start start 2024-04-25T10:14:01.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.535+0000 7f6d29ab5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d240ceef0 0x7f6d24151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d29ab5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d2415b6f0 0x7f6d240c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d29ab5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d2415e0d0 0x7f6d24152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d29ab5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240d24c0 con 0x7f6d240ceef0 2024-04-25T10:14:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d29ab5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240d2340 con 0x7f6d2415e0d0 2024-04-25T10:14:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d29ab5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240d2640 con 0x7f6d2415b6f0 2024-04-25T10:14:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d237fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d240ceef0 0x7f6d24151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d237fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d240ceef0 0x7f6d24151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47780/0 (socket says 172.21.15.60:47780) 2024-04-25T10:14:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d237fe640 1 -- 172.21.15.60:0/604540635 learned_addr learned my addr 172.21.15.60:0/604540635 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d22ffd640 1 --2- 172.21.15.60:0/604540635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d2415b6f0 0x7f6d240c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d23fff640 1 --2- 172.21.15.60:0/604540635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d2415e0d0 0x7f6d24152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d237fe640 1 -- 172.21.15.60:0/604540635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d2415b6f0 msgr2=0x7f6d240c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d237fe640 1 --2- 172.21.15.60:0/604540635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d2415b6f0 0x7f6d240c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d237fe640 1 -- 172.21.15.60:0/604540635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d2415e0d0 msgr2=0x7f6d24152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d237fe640 1 --2- 172.21.15.60:0/604540635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d2415e0d0 0x7f6d24152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d237fe640 1 -- 172.21.15.60:0/604540635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d240cec90 con 0x7f6d240ceef0 2024-04-25T10:14:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d23fff640 1 --2- 172.21.15.60:0/604540635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d2415e0d0 0x7f6d24152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:01.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d237fe640 1 --2- 172.21.15.60:0/604540635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d240ceef0 0x7f6d24151da0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00d830 tx=0x7f6d0c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:01.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d1bfff640 1 -- 172.21.15.60:0/604540635 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c00b6c0 con 0x7f6d240ceef0 2024-04-25T10:14:01.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d1bfff640 1 -- 172.21.15.60:0/604540635 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c019070 con 0x7f6d240ceef0 2024-04-25T10:14:01.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d29ab5640 1 -- 172.21.15.60:0/604540635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d240c6b00 con 0x7f6d240ceef0 2024-04-25T10:14:01.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d1bfff640 1 -- 172.21.15.60:0/604540635 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c00ba60 con 0x7f6d240ceef0 2024-04-25T10:14:01.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d29ab5640 1 -- 172.21.15.60:0/604540635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d24153980 con 0x7f6d240ceef0 2024-04-25T10:14:01.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d29ab5640 1 -- 172.21.15.60:0/604540635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d24074770 con 0x7f6d240ceef0 2024-04-25T10:14:01.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d1bfff640 1 -- 172.21.15.60:0/604540635 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d0c01e460 con 0x7f6d240ceef0 2024-04-25T10:14:01.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d1bfff640 1 --2- 172.21.15.60:0/604540635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6d04066f60 0x7f6d04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d1bfff640 1 -- 172.21.15.60:0/604540635 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f6d0c085870 con 0x7f6d240ceef0 2024-04-25T10:14:01.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.539+0000 7f6d22ffd640 1 --2- 172.21.15.60:0/604540635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6d04066f60 0x7f6d04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.543+0000 7f6d22ffd640 1 --2- 172.21.15.60:0/604540635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6d04066f60 0x7f6d04069420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6d240b8f10 tx=0x7f6d14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:01.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.543+0000 7f6d1bfff640 1 -- 172.21.15.60:0/604540635 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6d0c056720 con 0x7f6d240ceef0 2024-04-25T10:14:01.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.707+0000 7f6d29ab5640 1 -- 172.21.15.60:0/604540635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d240d16e0 con 0x7f6d240ceef0 2024-04-25T10:14:01.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.707+0000 7f6d1bfff640 1 -- 172.21.15.60:0/604540635 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d0c056020 con 0x7f6d240ceef0 2024-04-25T10:14:01.711 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:14:01.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.707+0000 7f6d29ab5640 1 -- 172.21.15.60:0/604540635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6d04066f60 msgr2=0x7f6d04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.707+0000 7f6d29ab5640 1 --2- 172.21.15.60:0/604540635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6d04066f60 0x7f6d04069420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6d240b8f10 tx=0x7f6d14009040 comp rx=0 tx=0).stop 2024-04-25T10:14:01.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.707+0000 7f6d29ab5640 1 -- 172.21.15.60:0/604540635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d240ceef0 msgr2=0x7f6d24151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.707+0000 7f6d29ab5640 1 --2- 172.21.15.60:0/604540635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d240ceef0 0x7f6d24151da0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00d830 tx=0x7f6d0c00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.707+0000 7f6d29ab5640 1 -- 172.21.15.60:0/604540635 shutdown_connections 2024-04-25T10:14:01.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.707+0000 7f6d29ab5640 1 --2- 172.21.15.60:0/604540635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6d04066f60 0x7f6d04069420 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.707+0000 7f6d29ab5640 1 --2- 172.21.15.60:0/604540635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d2415e0d0 0x7f6d24152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.707+0000 7f6d29ab5640 1 --2- 172.21.15.60:0/604540635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d2415b6f0 0x7f6d240c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.707+0000 7f6d29ab5640 1 --2- 172.21.15.60:0/604540635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d240ceef0 0x7f6d24151da0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.707+0000 7f6d29ab5640 1 -- 172.21.15.60:0/604540635 >> 172.21.15.60:0/604540635 conn(0x7f6d24072340 msgr2=0x7f6d24160c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:01.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.707+0000 7f6d29ab5640 1 -- 172.21.15.60:0/604540635 shutdown_connections 2024-04-25T10:14:01.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.707+0000 7f6d29ab5640 1 -- 172.21.15.60:0/604540635 wait complete. 2024-04-25T10:14:01.731 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13d1912640 1 Processor -- start 2024-04-25T10:14:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13d1912640 1 -- start start 2024-04-25T10:14:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13d1912640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f13cc160080 0x7f13cc15f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13d1912640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f13cc15fa20 0x7f13cc0ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13d1912640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13cc144a40 con 0x7f13cc0cbcf0 2024-04-25T10:14:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13d1912640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13cc15fe90 con 0x7f13cc160080 2024-04-25T10:14:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13d1912640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13cc0b2dc0 con 0x7f13cc15fa20 2024-04-25T10:14:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13caffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f13cc0cbcf0 0x7f13cc071c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38602/0 (socket says 172.21.15.60:38602) 2024-04-25T10:14:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13caffd640 1 -- 172.21.15.60:0/4222420281 learned_addr learned my addr 172.21.15.60:0/4222420281 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13ca7fc640 1 --2- 172.21.15.60:0/4222420281 >> v2:172.21.15.60:3300/0 conn(0x7f13cc160080 0x7f13cc15f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13cb7fe640 1 --2- 172.21.15.60:0/4222420281 >> v2:172.21.15.100:3300/0 conn(0x7f13cc15fa20 0x7f13cc0ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13c9ffb640 1 -- 172.21.15.60:0/4222420281 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490088579 0 0) 0x7f13cc144a40 con 0x7f13cc0cbcf0 2024-04-25T10:14:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13c9ffb640 1 -- 172.21.15.60:0/4222420281 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13b4003530 con 0x7f13cc0cbcf0 2024-04-25T10:14:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13ca7fc640 1 -- 172.21.15.60:0/4222420281 >> v2:172.21.15.100:3300/0 conn(0x7f13cc15fa20 msgr2=0x7f13cc0ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13ca7fc640 1 --2- 172.21.15.60:0/4222420281 >> v2:172.21.15.100:3300/0 conn(0x7f13cc15fa20 0x7f13cc0ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13ca7fc640 1 -- 172.21.15.60:0/4222420281 >> v1:172.21.15.60:6789/0 conn(0x7f13cc0cbcf0 legacy=0x7f13cc071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.859+0000 7f13ca7fc640 1 -- 172.21.15.60:0/4222420281 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc0cc9f0 con 0x7f13cc160080 2024-04-25T10:14:01.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.863+0000 7f13ca7fc640 1 --2- 172.21.15.60:0/4222420281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13cc160080 0x7f13cc15f4e0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0d03a0 tx=0x7f13c001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84de8e62cb33e3ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:01.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.863+0000 7f13c9ffb640 1 -- 172.21.15.60:0/4222420281 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c00b0070 con 0x7f13cc160080 2024-04-25T10:14:01.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.863+0000 7f13c9ffb640 1 -- 172.21.15.60:0/4222420281 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c00ab600 con 0x7f13cc160080 2024-04-25T10:14:01.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.863+0000 7f13c9ffb640 1 -- 172.21.15.60:0/4222420281 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c00ab9d0 con 0x7f13cc160080 2024-04-25T10:14:01.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.863+0000 7f13d1912640 1 -- 172.21.15.60:0/4222420281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13cc160080 msgr2=0x7f13cc15f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.863+0000 7f13d1912640 1 --2- 172.21.15.60:0/4222420281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13cc160080 0x7f13cc15f4e0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0d03a0 tx=0x7f13c001c480 comp rx=0 tx=0).stop 2024-04-25T10:14:01.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.863+0000 7f13d1912640 1 -- 172.21.15.60:0/4222420281 shutdown_connections 2024-04-25T10:14:01.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.863+0000 7f13d1912640 1 --2- 172.21.15.60:0/4222420281 >> v2:172.21.15.100:3300/0 conn(0x7f13cc15fa20 0x7f13cc0ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.863+0000 7f13d1912640 1 --2- 172.21.15.60:0/4222420281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13cc160080 0x7f13cc15f4e0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.863+0000 7f13d1912640 1 -- 172.21.15.60:0/4222420281 >> 172.21.15.60:0/4222420281 conn(0x7f13cc0670b0 msgr2=0x7f13cc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.863+0000 7f13d1912640 1 -- 172.21.15.60:0/4222420281 shutdown_connections 2024-04-25T10:14:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.863+0000 7f13d1912640 1 -- 172.21.15.60:0/4222420281 wait complete. 2024-04-25T10:14:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.863+0000 7f13d1912640 1 Processor -- start 2024-04-25T10:14:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.863+0000 7f13d1912640 1 -- start start 2024-04-25T10:14:01.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13d1912640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13cc0cbcf0 0x7f13cc13dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13d1912640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13cc15fa20 0x7f13cc0ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13d1912640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13cc160080 0x7f13cc0b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13d1912640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc0ce5c0 con 0x7f13cc15fa20 2024-04-25T10:14:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13d1912640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc0cc110 con 0x7f13cc0cbcf0 2024-04-25T10:14:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13d1912640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc0ce740 con 0x7f13cc160080 2024-04-25T10:14:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13ca7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13cc15fa20 0x7f13cc0ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13ca7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13cc15fa20 0x7f13cc0ab9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47810/0 (socket says 172.21.15.60:47810) 2024-04-25T10:14:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13ca7fc640 1 -- 172.21.15.60:0/1194407205 learned_addr learned my addr 172.21.15.60:0/1194407205 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13caffd640 1 --2- 172.21.15.60:0/1194407205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13cc0cbcf0 0x7f13cc13dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13cb7fe640 1 --2- 172.21.15.60:0/1194407205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13cc160080 0x7f13cc0b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13ca7fc640 1 -- 172.21.15.60:0/1194407205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13cc160080 msgr2=0x7f13cc0b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13ca7fc640 1 --2- 172.21.15.60:0/1194407205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13cc160080 0x7f13cc0b1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13ca7fc640 1 -- 172.21.15.60:0/1194407205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13cc0cbcf0 msgr2=0x7f13cc13dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13ca7fc640 1 --2- 172.21.15.60:0/1194407205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13cc0cbcf0 0x7f13cc13dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13ca7fc640 1 -- 172.21.15.60:0/1194407205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc0a85c0 con 0x7f13cc15fa20 2024-04-25T10:14:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13ca7fc640 1 --2- 172.21.15.60:0/1194407205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13cc15fa20 0x7f13cc0ab9d0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f13c0019090 tx=0x7f13c0019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13b3fff640 1 -- 172.21.15.60:0/1194407205 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c00ba020 con 0x7f13cc15fa20 2024-04-25T10:14:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13b3fff640 1 -- 172.21.15.60:0/1194407205 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c00b5460 con 0x7f13cc15fa20 2024-04-25T10:14:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13d1912640 1 -- 172.21.15.60:0/1194407205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13cc0a88a0 con 0x7f13cc15fa20 2024-04-25T10:14:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13b3fff640 1 -- 172.21.15.60:0/1194407205 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c00b0040 con 0x7f13cc15fa20 2024-04-25T10:14:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13d1912640 1 -- 172.21.15.60:0/1194407205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13cc0a8e60 con 0x7f13cc15fa20 2024-04-25T10:14:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13d1912640 1 -- 172.21.15.60:0/1194407205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13cc06bba0 con 0x7f13cc15fa20 2024-04-25T10:14:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.867+0000 7f13b3fff640 1 -- 172.21.15.60:0/1194407205 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13c00bd020 con 0x7f13cc15fa20 2024-04-25T10:14:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.871+0000 7f13b3fff640 1 --2- 172.21.15.60:0/1194407205 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13b8066f60 0x7f13b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.871+0000 7f13b3fff640 1 -- 172.21.15.60:0/1194407205 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f13c011f890 con 0x7f13cc15fa20 2024-04-25T10:14:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.871+0000 7f13caffd640 1 --2- 172.21.15.60:0/1194407205 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13b8066f60 0x7f13b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:01.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.871+0000 7f13caffd640 1 --2- 172.21.15.60:0/1194407205 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13b8066f60 0x7f13b8069420 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f13bc005890 tx=0x7f13bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:01.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:01.871+0000 7f13b3fff640 1 -- 172.21.15.60:0/1194407205 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f13c00c4030 con 0x7f13cc15fa20 2024-04-25T10:14:02.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.035+0000 7f13d1912640 1 -- 172.21.15.60:0/1194407205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13cc074770 con 0x7f13cc15fa20 2024-04-25T10:14:02.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.035+0000 7f13b3fff640 1 -- 172.21.15.60:0/1194407205 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f13c00f6070 con 0x7f13cc15fa20 2024-04-25T10:14:02.040 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:14:02.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.035+0000 7f13d1912640 1 -- 172.21.15.60:0/1194407205 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13b8066f60 msgr2=0x7f13b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.035+0000 7f13d1912640 1 --2- 172.21.15.60:0/1194407205 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13b8066f60 0x7f13b8069420 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f13bc005890 tx=0x7f13bc009040 comp rx=0 tx=0).stop 2024-04-25T10:14:02.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.035+0000 7f13d1912640 1 -- 172.21.15.60:0/1194407205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13cc15fa20 msgr2=0x7f13cc0ab9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.035+0000 7f13d1912640 1 --2- 172.21.15.60:0/1194407205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13cc15fa20 0x7f13cc0ab9d0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f13c0019090 tx=0x7f13c0019220 comp rx=0 tx=0).stop 2024-04-25T10:14:02.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.035+0000 7f13d1912640 1 -- 172.21.15.60:0/1194407205 shutdown_connections 2024-04-25T10:14:02.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.035+0000 7f13d1912640 1 --2- 172.21.15.60:0/1194407205 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13b8066f60 0x7f13b8069420 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.039+0000 7f13d1912640 1 --2- 172.21.15.60:0/1194407205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13cc160080 0x7f13cc0b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.039+0000 7f13d1912640 1 --2- 172.21.15.60:0/1194407205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13cc15fa20 0x7f13cc0ab9d0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.039+0000 7f13d1912640 1 --2- 172.21.15.60:0/1194407205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13cc0cbcf0 0x7f13cc13dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.039+0000 7f13d1912640 1 -- 172.21.15.60:0/1194407205 >> 172.21.15.60:0/1194407205 conn(0x7f13cc0670b0 msgr2=0x7f13cc0ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:02.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.039+0000 7f13d1912640 1 -- 172.21.15.60:0/1194407205 shutdown_connections 2024-04-25T10:14:02.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.039+0000 7f13d1912640 1 -- 172.21.15.60:0/1194407205 wait complete. 2024-04-25T10:14:02.060 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:02.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a1655d640 1 Processor -- start 2024-04-25T10:14:02.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a1655d640 1 -- start start 2024-04-25T10:14:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a1655d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a1015ef20 0x7f9a100b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a1655d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a100cc320 con 0x7f9a100cfcf0 2024-04-25T10:14:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a1655d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a100cc5c0 con 0x7f9a10099a30 2024-04-25T10:14:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a1655d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a100d07b0 con 0x7f9a1015ef20 2024-04-25T10:14:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9a100cfcf0 0x7f9a100b4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38606/0 (socket says 172.21.15.60:38606) 2024-04-25T10:14:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0f7fe640 1 -- 172.21.15.60:0/3561664623 learned_addr learned my addr 172.21.15.60:0/3561664623 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0ffff640 1 --2- 172.21.15.60:0/3561664623 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a1015ef20 0x7f9a100b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0effd640 1 -- 172.21.15.60:0/3561664623 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679715637 0 0) 0x7f9a100cc320 con 0x7f9a100cfcf0 2024-04-25T10:14:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0effd640 1 -- 172.21.15.60:0/3561664623 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99f8003530 con 0x7f9a100cfcf0 2024-04-25T10:14:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0effd640 1 -- 172.21.15.60:0/3561664623 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3553180341 0 0) 0x7f99f8003530 con 0x7f9a100cfcf0 2024-04-25T10:14:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0effd640 1 -- 172.21.15.60:0/3561664623 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a1015ef20 msgr2=0x7f9a100b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0effd640 1 --2- 172.21.15.60:0/3561664623 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a1015ef20 0x7f9a100b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0effd640 1 -- 172.21.15.60:0/3561664623 >> v1:172.21.15.100:6789/0 conn(0x7f9a10099a30 legacy=0x7f9a100cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0effd640 1 -- 172.21.15.60:0/3561664623 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a100d04d0 con 0x7f9a100cfcf0 2024-04-25T10:14:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0effd640 1 -- 172.21.15.60:0/3561664623 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f9a04001680 con 0x7f9a100cfcf0 2024-04-25T10:14:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0effd640 1 -- 172.21.15.60:0/3561664623 >> v1:172.21.15.60:6789/0 conn(0x7f9a100cfcf0 legacy=0x7f9a100b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0effd640 1 --2- 172.21.15.60:0/3561664623 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a1015ef20 0x7f99f80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0effd640 1 --2- 172.21.15.60:0/3561664623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f8003f30 0x7f99f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0effd640 1 --2- 172.21.15.60:0/3561664623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f8004d50 0x7f99f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0effd640 1 -- 172.21.15.60:0/3561664623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a100d07b0 con 0x7f99f8004d50 2024-04-25T10:14:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0effd640 1 -- 172.21.15.60:0/3561664623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f800ba30 con 0x7f99f8003f30 2024-04-25T10:14:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0effd640 1 -- 172.21.15.60:0/3561664623 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f800bc30 con 0x7f9a1015ef20 2024-04-25T10:14:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a14afa640 1 --2- 172.21.15.60:0/3561664623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f8004d50 0x7f99f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0f7fe640 1 --2- 172.21.15.60:0/3561664623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f8003f30 0x7f99f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a0ffff640 1 --2- 172.21.15.60:0/3561664623 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a1015ef20 0x7f99f80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a14afa640 1 -- 172.21.15.60:0/3561664623 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a1015ef20 msgr2=0x7f99f80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:02.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a14afa640 1 --2- 172.21.15.60:0/3561664623 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a1015ef20 0x7f99f80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a14afa640 1 -- 172.21.15.60:0/3561664623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f8003f30 msgr2=0x7f99f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:02.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a14afa640 1 --2- 172.21.15.60:0/3561664623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f8003f30 0x7f99f8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.179+0000 7f9a14afa640 1 -- 172.21.15.60:0/3561664623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f8003530 con 0x7f99f8004d50 2024-04-25T10:14:02.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.183+0000 7f9a14afa640 1 --2- 172.21.15.60:0/3561664623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f8004d50 0x7f99f80051d0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f9a08002010 tx=0x7f9a0801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fc43eb93ad79b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:02.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.183+0000 7f9a0effd640 1 -- 172.21.15.60:0/3561664623 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a08001c10 con 0x7f99f8004d50 2024-04-25T10:14:02.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.183+0000 7f9a0effd640 1 -- 172.21.15.60:0/3561664623 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a080a66f0 con 0x7f99f8004d50 2024-04-25T10:14:02.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.183+0000 7f9a0effd640 1 -- 172.21.15.60:0/3561664623 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a080a6ae0 con 0x7f99f8004d50 2024-04-25T10:14:02.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.183+0000 7f9a1655d640 1 -- 172.21.15.60:0/3561664623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f8004d50 msgr2=0x7f99f80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.183+0000 7f9a1655d640 1 --2- 172.21.15.60:0/3561664623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99f8004d50 0x7f99f80051d0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f9a08002010 tx=0x7f9a0801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.183+0000 7f9a14afa640 1 -- 172.21.15.60:0/3561664623 reap_dead start 2024-04-25T10:14:02.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.183+0000 7f9a1655d640 1 -- 172.21.15.60:0/3561664623 shutdown_connections 2024-04-25T10:14:02.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.183+0000 7f9a1655d640 1 -- 172.21.15.60:0/3561664623 >> 172.21.15.60:0/3561664623 conn(0x7f9a100b3200 msgr2=0x7f9a100b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:02.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.183+0000 7f9a1655d640 1 -- 172.21.15.60:0/3561664623 shutdown_connections 2024-04-25T10:14:02.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.183+0000 7f9a1655d640 1 -- 172.21.15.60:0/3561664623 wait complete. 2024-04-25T10:14:02.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.183+0000 7f9a1655d640 1 Processor -- start 2024-04-25T10:14:02.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.183+0000 7f9a1655d640 1 -- start start 2024-04-25T10:14:02.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a1655d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f8004d50 0x7f9a1014fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a1655d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a101578e0 0x7f9a101597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a1655d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1015c3e0 0x7f9a1015c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a1655d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a100d0170 con 0x7f9a1015c3e0 2024-04-25T10:14:02.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a1655d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a100ccb40 con 0x7f99f8004d50 2024-04-25T10:14:02.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a1655d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1015f300 con 0x7f9a101578e0 2024-04-25T10:14:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a14afa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1015c3e0 0x7f9a1015c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a14afa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1015c3e0 0x7f9a1015c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47830/0 (socket says 172.21.15.60:47830) 2024-04-25T10:14:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a14afa640 1 -- 172.21.15.60:0/2547051919 learned_addr learned my addr 172.21.15.60:0/2547051919 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a0f7fe640 1 --2- 172.21.15.60:0/2547051919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a101578e0 0x7f9a101597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a0ffff640 1 --2- 172.21.15.60:0/2547051919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f8004d50 0x7f9a1014fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a14afa640 1 -- 172.21.15.60:0/2547051919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a101578e0 msgr2=0x7f9a101597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a14afa640 1 --2- 172.21.15.60:0/2547051919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a101578e0 0x7f9a101597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a14afa640 1 -- 172.21.15.60:0/2547051919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f8004d50 msgr2=0x7f9a1014fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a14afa640 1 --2- 172.21.15.60:0/2547051919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f8004d50 0x7f9a1014fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a14afa640 1 -- 172.21.15.60:0/2547051919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a10159d20 con 0x7f9a1015c3e0 2024-04-25T10:14:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a0f7fe640 1 --2- 172.21.15.60:0/2547051919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a101578e0 0x7f9a101597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a0ffff640 1 --2- 172.21.15.60:0/2547051919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f8004d50 0x7f9a1014fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a14afa640 1 --2- 172.21.15.60:0/2547051919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1015c3e0 0x7f9a1015c860 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f9a080b0040 tx=0x7f99f8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a0cff9640 1 -- 172.21.15.60:0/2547051919 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a080bd070 con 0x7f9a1015c3e0 2024-04-25T10:14:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a0cff9640 1 -- 172.21.15.60:0/2547051919 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a08006d60 con 0x7f9a1015c3e0 2024-04-25T10:14:02.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a1655d640 1 -- 172.21.15.60:0/2547051919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a10157f70 con 0x7f9a1015c3e0 2024-04-25T10:14:02.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a0cff9640 1 -- 172.21.15.60:0/2547051919 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a08003e60 con 0x7f9a1015c3e0 2024-04-25T10:14:02.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a1655d640 1 -- 172.21.15.60:0/2547051919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a10158530 con 0x7f9a1015c3e0 2024-04-25T10:14:02.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a1655d640 1 -- 172.21.15.60:0/2547051919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a10074770 con 0x7f9a1015c3e0 2024-04-25T10:14:02.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a0cff9640 1 -- 172.21.15.60:0/2547051919 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a080c0070 con 0x7f9a1015c3e0 2024-04-25T10:14:02.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a0cff9640 1 --2- 172.21.15.60:0/2547051919 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f99fc066f60 0x7f99fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a0cff9640 1 -- 172.21.15.60:0/2547051919 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f9a0811ed50 con 0x7f9a1015c3e0 2024-04-25T10:14:02.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.187+0000 7f9a0ffff640 1 --2- 172.21.15.60:0/2547051919 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f99fc066f60 0x7f99fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.191+0000 7f9a0ffff640 1 --2- 172.21.15.60:0/2547051919 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f99fc066f60 0x7f99fc069420 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009a20 tx=0x7f9a00008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:02.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.191+0000 7f9a0cff9640 1 -- 172.21.15.60:0/2547051919 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a10159d20 con 0x7f9a1015c3e0 2024-04-25T10:14:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.347+0000 7f9a1655d640 1 -- 172.21.15.60:0/2547051919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9a10157cc0 con 0x7f9a1015c3e0 2024-04-25T10:14:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.347+0000 7f9a0cff9640 1 -- 172.21.15.60:0/2547051919 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9a080165f0 con 0x7f9a1015c3e0 2024-04-25T10:14:02.353 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:14:02.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.351+0000 7f9a1655d640 1 -- 172.21.15.60:0/2547051919 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f99fc066f60 msgr2=0x7f99fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.351+0000 7f9a1655d640 1 --2- 172.21.15.60:0/2547051919 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f99fc066f60 0x7f99fc069420 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009a20 tx=0x7f9a00008040 comp rx=0 tx=0).stop 2024-04-25T10:14:02.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.351+0000 7f9a1655d640 1 -- 172.21.15.60:0/2547051919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1015c3e0 msgr2=0x7f9a1015c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.351+0000 7f9a1655d640 1 --2- 172.21.15.60:0/2547051919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1015c3e0 0x7f9a1015c860 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f9a080b0040 tx=0x7f99f8005840 comp rx=0 tx=0).stop 2024-04-25T10:14:02.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.351+0000 7f9a1655d640 1 -- 172.21.15.60:0/2547051919 shutdown_connections 2024-04-25T10:14:02.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.351+0000 7f9a1655d640 1 --2- 172.21.15.60:0/2547051919 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f99fc066f60 0x7f99fc069420 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.351+0000 7f9a1655d640 1 --2- 172.21.15.60:0/2547051919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1015c3e0 0x7f9a1015c860 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.351+0000 7f9a1655d640 1 --2- 172.21.15.60:0/2547051919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a101578e0 0x7f9a101597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.351+0000 7f9a1655d640 1 --2- 172.21.15.60:0/2547051919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99f8004d50 0x7f9a1014fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.351+0000 7f9a1655d640 1 -- 172.21.15.60:0/2547051919 >> 172.21.15.60:0/2547051919 conn(0x7f9a100b3200 msgr2=0x7f9a100b4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:02.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.351+0000 7f9a1655d640 1 -- 172.21.15.60:0/2547051919 shutdown_connections 2024-04-25T10:14:02.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.351+0000 7f9a1655d640 1 -- 172.21.15.60:0/2547051919 wait complete. 2024-04-25T10:14:02.372 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:02.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.483+0000 7faea62bc640 1 Processor -- start 2024-04-25T10:14:02.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.483+0000 7faea62bc640 1 -- start start 2024-04-25T10:14:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.483+0000 7faea62bc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faea015dc90 0x7faea00b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.483+0000 7faea62bc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faea00719a0 0x7faea00ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.483+0000 7faea62bc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea0071fd0 con 0x7faea00b2b10 2024-04-25T10:14:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.483+0000 7faea62bc640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea00accc0 con 0x7faea015dc90 2024-04-25T10:14:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.483+0000 7faea62bc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea00ad260 con 0x7faea00719a0 2024-04-25T10:14:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.483+0000 7fae9ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faea015dc90 0x7faea00b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.483+0000 7fae9ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faea015dc90 0x7faea00b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47844/0 (socket says 172.21.15.60:47844) 2024-04-25T10:14:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.483+0000 7fae9ffff640 1 -- 172.21.15.60:0/3056222362 learned_addr learned my addr 172.21.15.60:0/3056222362 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.483+0000 7faea4859640 1 --2- 172.21.15.60:0/3056222362 >> v2:172.21.15.100:3300/0 conn(0x7faea00719a0 0x7faea00ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.483+0000 7fae9ffff640 1 -- 172.21.15.60:0/3056222362 >> v2:172.21.15.100:3300/0 conn(0x7faea00719a0 msgr2=0x7faea00ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.483+0000 7fae9ffff640 1 --2- 172.21.15.60:0/3056222362 >> v2:172.21.15.100:3300/0 conn(0x7faea00719a0 0x7faea00ad6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.483+0000 7fae9ffff640 1 -- 172.21.15.60:0/3056222362 >> v1:172.21.15.100:6789/0 conn(0x7faea00b2b10 legacy=0x7faea0071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.483+0000 7fae9ffff640 1 -- 172.21.15.60:0/3056222362 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea0072290 con 0x7faea015dc90 2024-04-25T10:14:02.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7fae9ffff640 1 --2- 172.21.15.60:0/3056222362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea015dc90 0x7faea00b25d0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7faea0072ab0 tx=0x7fae9001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33121533044841c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:02.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7fae9f7fe640 1 -- 172.21.15.60:0/3056222362 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae900ab5f0 con 0x7faea015dc90 2024-04-25T10:14:02.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7fae9f7fe640 1 -- 172.21.15.60:0/3056222362 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae900b0070 con 0x7faea015dc90 2024-04-25T10:14:02.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7fae9f7fe640 1 -- 172.21.15.60:0/3056222362 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae900ab9b0 con 0x7faea015dc90 2024-04-25T10:14:02.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7faea62bc640 1 -- 172.21.15.60:0/3056222362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea015dc90 msgr2=0x7faea00b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7faea62bc640 1 --2- 172.21.15.60:0/3056222362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea015dc90 0x7faea00b25d0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7faea0072ab0 tx=0x7fae9001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7faea62bc640 1 -- 172.21.15.60:0/3056222362 shutdown_connections 2024-04-25T10:14:02.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7faea62bc640 1 --2- 172.21.15.60:0/3056222362 >> v2:172.21.15.100:3300/0 conn(0x7faea00719a0 0x7faea00ad6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7faea62bc640 1 --2- 172.21.15.60:0/3056222362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea015dc90 0x7faea00b25d0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7faea62bc640 1 -- 172.21.15.60:0/3056222362 >> 172.21.15.60:0/3056222362 conn(0x7faea00b12d0 msgr2=0x7faea00b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:02.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7faea62bc640 1 -- 172.21.15.60:0/3056222362 shutdown_connections 2024-04-25T10:14:02.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7faea62bc640 1 -- 172.21.15.60:0/3056222362 wait complete. 2024-04-25T10:14:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7faea62bc640 1 Processor -- start 2024-04-25T10:14:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7faea62bc640 1 -- start start 2024-04-25T10:14:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7faea62bc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faea00719a0 0x7faea0159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7faea62bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea015dc90 0x7faea014ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7faea62bc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faea015fea0 0x7faea0151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7faea62bc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea00adfc0 con 0x7faea015dc90 2024-04-25T10:14:02.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7faea62bc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea00ade40 con 0x7faea00719a0 2024-04-25T10:14:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.487+0000 7faea62bc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea00b2220 con 0x7faea015fea0 2024-04-25T10:14:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae97fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea015dc90 0x7faea014ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae97fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea015dc90 0x7faea014ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47852/0 (socket says 172.21.15.60:47852) 2024-04-25T10:14:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae97fff640 1 -- 172.21.15.60:0/3207981731 learned_addr learned my addr 172.21.15.60:0/3207981731 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7faea4859640 1 --2- 172.21.15.60:0/3207981731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faea015fea0 0x7faea0151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae9ffff640 1 --2- 172.21.15.60:0/3207981731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faea00719a0 0x7faea0159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae97fff640 1 -- 172.21.15.60:0/3207981731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faea015fea0 msgr2=0x7faea0151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae97fff640 1 --2- 172.21.15.60:0/3207981731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faea015fea0 0x7faea0151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae97fff640 1 -- 172.21.15.60:0/3207981731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faea00719a0 msgr2=0x7faea0159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae97fff640 1 --2- 172.21.15.60:0/3207981731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faea00719a0 0x7faea0159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae97fff640 1 -- 172.21.15.60:0/3207981731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea014fb00 con 0x7faea015dc90 2024-04-25T10:14:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7faea4859640 1 --2- 172.21.15.60:0/3207981731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faea015fea0 0x7faea0151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae9ffff640 1 --2- 172.21.15.60:0/3207981731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faea00719a0 0x7faea0159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae97fff640 1 --2- 172.21.15.60:0/3207981731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea015dc90 0x7faea014ec50 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fae8800c490 tx=0x7fae88001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:02.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae9d7fa640 1 -- 172.21.15.60:0/3207981731 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88017070 con 0x7faea015dc90 2024-04-25T10:14:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae9d7fa640 1 -- 172.21.15.60:0/3207981731 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae88003600 con 0x7faea015dc90 2024-04-25T10:14:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae9d7fa640 1 -- 172.21.15.60:0/3207981731 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88003a30 con 0x7faea015dc90 2024-04-25T10:14:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7faea62bc640 1 -- 172.21.15.60:0/3207981731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea014fda0 con 0x7faea015dc90 2024-04-25T10:14:02.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7faea62bc640 1 -- 172.21.15.60:0/3207981731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea0150360 con 0x7faea015dc90 2024-04-25T10:14:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7faea62bc640 1 -- 172.21.15.60:0/3207981731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea0074770 con 0x7faea015dc90 2024-04-25T10:14:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae9d7fa640 1 -- 172.21.15.60:0/3207981731 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae88003bd0 con 0x7faea015dc90 2024-04-25T10:14:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae9d7fa640 1 --2- 172.21.15.60:0/3207981731 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae80066f60 0x7fae80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae9d7fa640 1 -- 172.21.15.60:0/3207981731 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fae880843c0 con 0x7faea015dc90 2024-04-25T10:14:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae9ffff640 1 --2- 172.21.15.60:0/3207981731 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae80066f60 0x7fae80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.491+0000 7fae9ffff640 1 --2- 172.21.15.60:0/3207981731 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae80066f60 0x7fae80069420 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fae900af000 tx=0x7fae90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:02.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.495+0000 7fae9d7fa640 1 -- 172.21.15.60:0/3207981731 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae88055300 con 0x7faea015dc90 2024-04-25T10:14:02.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.683+0000 7faea62bc640 1 -- 172.21.15.60:0/3207981731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faea0067c90 con 0x7faea015dc90 2024-04-25T10:14:02.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.683+0000 7fae9d7fa640 1 -- 172.21.15.60:0/3207981731 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fae88054c00 con 0x7faea015dc90 2024-04-25T10:14:02.689 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:02.689 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:13:22.221010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8256":{"gid":8256,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/3807131024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3807131024},{"type":"v1","addr":"172.21.15.60:6835","nonce":3807131024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:14:02.689 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 68 2024-04-25T10:14:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.687+0000 7faea62bc640 1 -- 172.21.15.60:0/3207981731 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae80066f60 msgr2=0x7fae80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.687+0000 7faea62bc640 1 --2- 172.21.15.60:0/3207981731 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae80066f60 0x7fae80069420 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fae900af000 tx=0x7fae90008040 comp rx=0 tx=0).stop 2024-04-25T10:14:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.687+0000 7faea62bc640 1 -- 172.21.15.60:0/3207981731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea015dc90 msgr2=0x7faea014ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.687+0000 7faea62bc640 1 --2- 172.21.15.60:0/3207981731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea015dc90 0x7faea014ec50 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fae8800c490 tx=0x7fae88001e80 comp rx=0 tx=0).stop 2024-04-25T10:14:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.687+0000 7faea62bc640 1 -- 172.21.15.60:0/3207981731 shutdown_connections 2024-04-25T10:14:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.687+0000 7faea62bc640 1 --2- 172.21.15.60:0/3207981731 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae80066f60 0x7fae80069420 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.687+0000 7faea62bc640 1 --2- 172.21.15.60:0/3207981731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faea015fea0 0x7faea0151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.687+0000 7faea62bc640 1 --2- 172.21.15.60:0/3207981731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea015dc90 0x7faea014ec50 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.687+0000 7faea62bc640 1 --2- 172.21.15.60:0/3207981731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faea00719a0 0x7faea0159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.687+0000 7faea62bc640 1 -- 172.21.15.60:0/3207981731 >> 172.21.15.60:0/3207981731 conn(0x7faea00b12d0 msgr2=0x7faea00cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:02.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.687+0000 7faea62bc640 1 -- 172.21.15.60:0/3207981731 shutdown_connections 2024-04-25T10:14:02.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.687+0000 7faea62bc640 1 -- 172.21.15.60:0/3207981731 wait complete. 2024-04-25T10:14:02.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:02.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.819+0000 7fb2f77fd640 1 Processor -- start 2024-04-25T10:14:02.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.819+0000 7fb2f77fd640 1 -- start start 2024-04-25T10:14:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.823+0000 7fb2f77fd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb2f015a7a0 0x7fb2f00cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.823+0000 7fb2f77fd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb2f00b37e0 0x7fb2f00b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.823+0000 7fb2f77fd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f006f130 con 0x7fb2f006ec30 2024-04-25T10:14:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.823+0000 7fb2f77fd640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f00b4190 con 0x7fb2f015a7a0 2024-04-25T10:14:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.823+0000 7fb2f77fd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f00cdaa0 con 0x7fb2f00b37e0 2024-04-25T10:14:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.823+0000 7fb2f5599640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb2f015a7a0 0x7fb2f00cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.823+0000 7fb2f5599640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb2f015a7a0 0x7fb2f00cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47858/0 (socket says 172.21.15.60:47858) 2024-04-25T10:14:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.823+0000 7fb2f5599640 1 -- 172.21.15.60:0/3862881532 learned_addr learned my addr 172.21.15.60:0/3862881532 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.823+0000 7fb2f5d9a640 1 --2- 172.21.15.60:0/3862881532 >> v2:172.21.15.100:3300/0 conn(0x7fb2f00b37e0 0x7fb2f00b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.823+0000 7fb2f5599640 1 -- 172.21.15.60:0/3862881532 >> v2:172.21.15.100:3300/0 conn(0x7fb2f00b37e0 msgr2=0x7fb2f00b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.823+0000 7fb2f5599640 1 --2- 172.21.15.60:0/3862881532 >> v2:172.21.15.100:3300/0 conn(0x7fb2f00b37e0 0x7fb2f00b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.823+0000 7fb2f5599640 1 -- 172.21.15.60:0/3862881532 >> v1:172.21.15.100:6789/0 conn(0x7fb2f006ec30 legacy=0x7fb2f0071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.823+0000 7fb2f5599640 1 -- 172.21.15.60:0/3862881532 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f006f410 con 0x7fb2f015a7a0 2024-04-25T10:14:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f5599640 1 --2- 172.21.15.60:0/3862881532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f015a7a0 0x7fb2f00cd560 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00bb10 tx=0x7fb2ec01ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9f4446effc6e56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2e7fff640 1 -- 172.21.15.60:0/3862881532 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0b4070 con 0x7fb2f015a7a0 2024-04-25T10:14:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2e7fff640 1 -- 172.21.15.60:0/3862881532 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ec0a7b90 con 0x7fb2f015a7a0 2024-04-25T10:14:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2e7fff640 1 -- 172.21.15.60:0/3862881532 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0af440 con 0x7fb2f015a7a0 2024-04-25T10:14:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 -- 172.21.15.60:0/3862881532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f015a7a0 msgr2=0x7fb2f00cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 --2- 172.21.15.60:0/3862881532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f015a7a0 0x7fb2f00cd560 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00bb10 tx=0x7fb2ec01ebb0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 -- 172.21.15.60:0/3862881532 shutdown_connections 2024-04-25T10:14:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 --2- 172.21.15.60:0/3862881532 >> v2:172.21.15.100:3300/0 conn(0x7fb2f00b37e0 0x7fb2f00b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 --2- 172.21.15.60:0/3862881532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f015a7a0 0x7fb2f00cd560 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 -- 172.21.15.60:0/3862881532 >> 172.21.15.60:0/3862881532 conn(0x7fb2f00b12d0 msgr2=0x7fb2f00b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 -- 172.21.15.60:0/3862881532 shutdown_connections 2024-04-25T10:14:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 -- 172.21.15.60:0/3862881532 wait complete. 2024-04-25T10:14:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 Processor -- start 2024-04-25T10:14:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 -- start start 2024-04-25T10:14:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f006ec30 0x7fb2f01525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f00b37e0 0x7fb2f0152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2f015a7a0 0x7fb2f00b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f006e8e0 con 0x7fb2f00b37e0 2024-04-25T10:14:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f006e760 con 0x7fb2f006ec30 2024-04-25T10:14:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f006d8b0 con 0x7fb2f015a7a0 2024-04-25T10:14:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f4d98640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f00b37e0 0x7fb2f0152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f4d98640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f00b37e0 0x7fb2f0152020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47868/0 (socket says 172.21.15.60:47868) 2024-04-25T10:14:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f4d98640 1 -- 172.21.15.60:0/2789294153 learned_addr learned my addr 172.21.15.60:0/2789294153 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f5599640 1 --2- 172.21.15.60:0/2789294153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f006ec30 0x7fb2f01525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f4d98640 1 -- 172.21.15.60:0/2789294153 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2f015a7a0 msgr2=0x7fb2f00b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f4d98640 1 --2- 172.21.15.60:0/2789294153 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2f015a7a0 0x7fb2f00b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f4d98640 1 -- 172.21.15.60:0/2789294153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f006ec30 msgr2=0x7fb2f01525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f4d98640 1 --2- 172.21.15.60:0/2789294153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f006ec30 0x7fb2f01525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f4d98640 1 -- 172.21.15.60:0/2789294153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f01f6450 con 0x7fb2f00b37e0 2024-04-25T10:14:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f4d98640 1 --2- 172.21.15.60:0/2789294153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f00b37e0 0x7fb2f0152020 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000c410 tx=0x7fb2e000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2e5ffb640 1 -- 172.21.15.60:0/2789294153 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0011790 con 0x7fb2f00b37e0 2024-04-25T10:14:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2e5ffb640 1 -- 172.21.15.60:0/2789294153 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e0017070 con 0x7fb2f00b37e0 2024-04-25T10:14:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 -- 172.21.15.60:0/2789294153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f0154740 con 0x7fb2f00b37e0 2024-04-25T10:14:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2e5ffb640 1 -- 172.21.15.60:0/2789294153 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0011bb0 con 0x7fb2f00b37e0 2024-04-25T10:14:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.827+0000 7fb2f77fd640 1 -- 172.21.15.60:0/2789294153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f0154d00 con 0x7fb2f00b37e0 2024-04-25T10:14:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.831+0000 7fb2f77fd640 1 -- 172.21.15.60:0/2789294153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f0152df0 con 0x7fb2f00b37e0 2024-04-25T10:14:02.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.831+0000 7fb2e5ffb640 1 -- 172.21.15.60:0/2789294153 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2e001c460 con 0x7fb2f00b37e0 2024-04-25T10:14:02.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.831+0000 7fb2e5ffb640 1 --2- 172.21.15.60:0/2789294153 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb2d4066f60 0x7fb2d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:02.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.831+0000 7fb2e5ffb640 1 -- 172.21.15.60:0/2789294153 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fb2e0084e40 con 0x7fb2f00b37e0 2024-04-25T10:14:02.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.831+0000 7fb2f5599640 1 --2- 172.21.15.60:0/2789294153 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb2d4066f60 0x7fb2d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:02.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.831+0000 7fb2f5599640 1 --2- 172.21.15.60:0/2789294153 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb2d4066f60 0x7fb2d4069420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0b1000 tx=0x7fb2ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:02.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.835+0000 7fb2e5ffb640 1 -- 172.21.15.60:0/2789294153 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2e0021020 con 0x7fb2f00b37e0 2024-04-25T10:14:02.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.991+0000 7fb2f77fd640 1 -- 172.21.15.60:0/2789294153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2f0074770 con 0x7fb2f00b37e0 2024-04-25T10:14:02.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.995+0000 7fb2e5ffb640 1 -- 172.21.15.60:0/2789294153 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16981 (secure 0 0 0) 0x7fb2e0055ce0 con 0x7fb2f00b37e0 2024-04-25T10:14:02.998 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:02.998 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":86,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:13:19.192820+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:42.621502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:43.352679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/2817879567":"2024-04-26T10:13:19.192767+0000","172.21.15.60:6833/2817879567":"2024-04-26T10:13:19.192767+0000","172.21.15.60:6835/2455562778":"2024-04-26T10:11:58.136789+0000","172.21.15.60:6833/2455562778":"2024-04-26T10:11:58.136789+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:03.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.995+0000 7fb2f77fd640 1 -- 172.21.15.60:0/2789294153 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb2d4066f60 msgr2=0x7fb2d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.995+0000 7fb2f77fd640 1 --2- 172.21.15.60:0/2789294153 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb2d4066f60 0x7fb2d4069420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0b1000 tx=0x7fb2ec008040 comp rx=0 tx=0).stop 2024-04-25T10:14:03.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.995+0000 7fb2f77fd640 1 -- 172.21.15.60:0/2789294153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f00b37e0 msgr2=0x7fb2f0152020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.995+0000 7fb2f77fd640 1 --2- 172.21.15.60:0/2789294153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f00b37e0 0x7fb2f0152020 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000c410 tx=0x7fb2e000c8d0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.995+0000 7fb2f77fd640 1 -- 172.21.15.60:0/2789294153 shutdown_connections 2024-04-25T10:14:03.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.995+0000 7fb2f77fd640 1 --2- 172.21.15.60:0/2789294153 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb2d4066f60 0x7fb2d4069420 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.995+0000 7fb2f77fd640 1 --2- 172.21.15.60:0/2789294153 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2f015a7a0 0x7fb2f00b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.995+0000 7fb2f77fd640 1 --2- 172.21.15.60:0/2789294153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2f00b37e0 0x7fb2f0152020 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.995+0000 7fb2f77fd640 1 --2- 172.21.15.60:0/2789294153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f006ec30 0x7fb2f01525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.995+0000 7fb2f77fd640 1 -- 172.21.15.60:0/2789294153 >> 172.21.15.60:0/2789294153 conn(0x7fb2f00b12d0 msgr2=0x7fb2f00cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:03.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.995+0000 7fb2f77fd640 1 -- 172.21.15.60:0/2789294153 shutdown_connections 2024-04-25T10:14:03.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:02.995+0000 7fb2f77fd640 1 -- 172.21.15.60:0/2789294153 wait complete. 2024-04-25T10:14:03.016 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:14:03.017 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:03.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.131+0000 7ff894686640 1 Processor -- start 2024-04-25T10:14:03.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.131+0000 7ff894686640 1 -- start start 2024-04-25T10:14:03.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.135+0000 7ff894686640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff88c15dc90 0x7ff88c0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.135+0000 7ff894686640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88c071fd0 con 0x7ff88c0719a0 2024-04-25T10:14:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.135+0000 7ff894686640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88c072290 con 0x7ff88c0b2b10 2024-04-25T10:14:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.135+0000 7ff894686640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff88c0accc0 con 0x7ff88c15dc90 2024-04-25T10:14:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.135+0000 7ff892c23640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff88c0719a0 0x7ff88c0ad6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38608/0 (socket says 172.21.15.60:38608) 2024-04-25T10:14:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.135+0000 7ff892c23640 1 -- 172.21.15.60:0/2493492055 learned_addr learned my addr 172.21.15.60:0/2493492055 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.135+0000 7ff892422640 1 --2- 172.21.15.60:0/2493492055 >> v2:172.21.15.60:3300/0 conn(0x7ff88c15dc90 0x7ff88c0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.135+0000 7ff891420640 1 -- 172.21.15.60:0/2493492055 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185942383 0 0) 0x7ff88c071fd0 con 0x7ff88c0719a0 2024-04-25T10:14:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.135+0000 7ff891420640 1 -- 172.21.15.60:0/2493492055 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff878003530 con 0x7ff88c0719a0 2024-04-25T10:14:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.135+0000 7ff892422640 1 -- 172.21.15.60:0/2493492055 >> v1:172.21.15.100:6789/0 conn(0x7ff88c0b2b10 legacy=0x7ff88c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.135+0000 7ff892422640 1 -- 172.21.15.60:0/2493492055 >> v1:172.21.15.60:6789/0 conn(0x7ff88c0719a0 legacy=0x7ff88c0ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.135+0000 7ff892422640 1 -- 172.21.15.60:0/2493492055 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c0acef0 con 0x7ff88c15dc90 2024-04-25T10:14:03.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff892422640 1 --2- 172.21.15.60:0/2493492055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff88c15dc90 0x7ff88c0b25d0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7ff8880097c0 tx=0x7ff88801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f09eafc0c1f912b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:03.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff891420640 1 -- 172.21.15.60:0/2493492055 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8880b0070 con 0x7ff88c15dc90 2024-04-25T10:14:03.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff891420640 1 -- 172.21.15.60:0/2493492055 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8880ab600 con 0x7ff88c15dc90 2024-04-25T10:14:03.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff891420640 1 -- 172.21.15.60:0/2493492055 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8880ab9d0 con 0x7ff88c15dc90 2024-04-25T10:14:03.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 -- 172.21.15.60:0/2493492055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff88c15dc90 msgr2=0x7ff88c0b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 --2- 172.21.15.60:0/2493492055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff88c15dc90 0x7ff88c0b25d0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7ff8880097c0 tx=0x7ff88801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 -- 172.21.15.60:0/2493492055 shutdown_connections 2024-04-25T10:14:03.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 --2- 172.21.15.60:0/2493492055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff88c15dc90 0x7ff88c0b25d0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 -- 172.21.15.60:0/2493492055 >> 172.21.15.60:0/2493492055 conn(0x7ff88c0b12d0 msgr2=0x7ff88c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:03.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 -- 172.21.15.60:0/2493492055 shutdown_connections 2024-04-25T10:14:03.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 -- 172.21.15.60:0/2493492055 wait complete. 2024-04-25T10:14:03.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 Processor -- start 2024-04-25T10:14:03.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 -- start start 2024-04-25T10:14:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff88c0719a0 0x7ff88c145e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff88c15dc90 0x7ff88c163a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff88c161700 0x7ff88c0cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c0b2220 con 0x7ff88c0719a0 2024-04-25T10:14:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c06d540 con 0x7ff88c15dc90 2024-04-25T10:14:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c0ad260 con 0x7ff88c161700 2024-04-25T10:14:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff892422640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff88c0719a0 0x7ff88c145e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff892422640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff88c0719a0 0x7ff88c145e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47886/0 (socket says 172.21.15.60:47886) 2024-04-25T10:14:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff892422640 1 -- 172.21.15.60:0/590452094 learned_addr learned my addr 172.21.15.60:0/590452094 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff891c21640 1 --2- 172.21.15.60:0/590452094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff88c15dc90 0x7ff88c163a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff892422640 1 -- 172.21.15.60:0/590452094 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff88c161700 msgr2=0x7ff88c0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff892c23640 1 --2- 172.21.15.60:0/590452094 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff88c161700 0x7ff88c0cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff892422640 1 --2- 172.21.15.60:0/590452094 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff88c161700 0x7ff88c0cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff892422640 1 -- 172.21.15.60:0/590452094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff88c15dc90 msgr2=0x7ff88c163a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff892422640 1 --2- 172.21.15.60:0/590452094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff88c15dc90 0x7ff88c163a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff892422640 1 -- 172.21.15.60:0/590452094 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c161bc0 con 0x7ff88c0719a0 2024-04-25T10:14:03.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff891c21640 1 --2- 172.21.15.60:0/590452094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff88c15dc90 0x7ff88c163a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:03.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff892422640 1 --2- 172.21.15.60:0/590452094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff88c0719a0 0x7ff88c145e50 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7ff8880097c0 tx=0x7ff888019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:03.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff882ffd640 1 -- 172.21.15.60:0/590452094 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8880b0040 con 0x7ff88c0719a0 2024-04-25T10:14:03.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff882ffd640 1 -- 172.21.15.60:0/590452094 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8880b9070 con 0x7ff88c0719a0 2024-04-25T10:14:03.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 -- 172.21.15.60:0/590452094 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff88c161ea0 con 0x7ff88c0719a0 2024-04-25T10:14:03.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff882ffd640 1 -- 172.21.15.60:0/590452094 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8880b4850 con 0x7ff88c0719a0 2024-04-25T10:14:03.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.139+0000 7ff894686640 1 -- 172.21.15.60:0/590452094 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88c162460 con 0x7ff88c0719a0 2024-04-25T10:14:03.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.143+0000 7ff894686640 1 -- 172.21.15.60:0/590452094 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff88c074770 con 0x7ff88c0719a0 2024-04-25T10:14:03.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.143+0000 7ff882ffd640 1 -- 172.21.15.60:0/590452094 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8880b49f0 con 0x7ff88c0719a0 2024-04-25T10:14:03.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.143+0000 7ff882ffd640 1 --2- 172.21.15.60:0/590452094 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff870066f60 0x7ff870069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.143+0000 7ff882ffd640 1 -- 172.21.15.60:0/590452094 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7ff88811ed10 con 0x7ff88c0719a0 2024-04-25T10:14:03.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.143+0000 7ff891c21640 1 --2- 172.21.15.60:0/590452094 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff870066f60 0x7ff870069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.143+0000 7ff891c21640 1 --2- 172.21.15.60:0/590452094 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff870066f60 0x7ff870069420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0d0100 tx=0x7ff87c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:03.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.147+0000 7ff882ffd640 1 -- 172.21.15.60:0/590452094 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff888016090 con 0x7ff88c0719a0 2024-04-25T10:14:03.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.335+0000 7ff894686640 1 -- 172.21.15.60:0/590452094 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff88c071fd0 con 0x7ff88c0719a0 2024-04-25T10:14:03.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.335+0000 7ff882ffd640 1 -- 172.21.15.60:0/590452094 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4944 (secure 0 0 0) 0x7ff8880efc50 con 0x7ff88c0719a0 2024-04-25T10:14:03.339 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:03.339 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:13:22.221010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8256":{"gid":8256,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/3807131024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3807131024},{"type":"v1","addr":"172.21.15.60:6835","nonce":3807131024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:14:03.339 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 68 2024-04-25T10:14:03.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.335+0000 7ff894686640 1 -- 172.21.15.60:0/590452094 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff870066f60 msgr2=0x7ff870069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.335+0000 7ff894686640 1 --2- 172.21.15.60:0/590452094 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff870066f60 0x7ff870069420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0d0100 tx=0x7ff87c009040 comp rx=0 tx=0).stop 2024-04-25T10:14:03.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.335+0000 7ff894686640 1 -- 172.21.15.60:0/590452094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff88c0719a0 msgr2=0x7ff88c145e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.335+0000 7ff894686640 1 --2- 172.21.15.60:0/590452094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff88c0719a0 0x7ff88c145e50 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7ff8880097c0 tx=0x7ff888019240 comp rx=0 tx=0).stop 2024-04-25T10:14:03.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.335+0000 7ff894686640 1 -- 172.21.15.60:0/590452094 shutdown_connections 2024-04-25T10:14:03.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.335+0000 7ff894686640 1 --2- 172.21.15.60:0/590452094 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff870066f60 0x7ff870069420 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.335+0000 7ff894686640 1 --2- 172.21.15.60:0/590452094 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff88c161700 0x7ff88c0cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.335+0000 7ff894686640 1 --2- 172.21.15.60:0/590452094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff88c15dc90 0x7ff88c163a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.335+0000 7ff894686640 1 --2- 172.21.15.60:0/590452094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff88c0719a0 0x7ff88c145e50 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.335+0000 7ff894686640 1 -- 172.21.15.60:0/590452094 >> 172.21.15.60:0/590452094 conn(0x7ff88c0b12d0 msgr2=0x7ff88c0b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:03.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.335+0000 7ff894686640 1 -- 172.21.15.60:0/590452094 shutdown_connections 2024-04-25T10:14:03.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.335+0000 7ff894686640 1 -- 172.21.15.60:0/590452094 wait complete. 2024-04-25T10:14:03.357 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:03.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.463+0000 7f0db6937640 1 Processor -- start 2024-04-25T10:14:03.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.463+0000 7f0db6937640 1 -- start start 2024-04-25T10:14:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.463+0000 7f0db6937640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0db00ae7c0 0x7f0db0071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.463+0000 7f0db6937640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0db006d2f0 0x7f0db00717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.463+0000 7f0db6937640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db0071fd0 con 0x7f0db015dc90 2024-04-25T10:14:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.463+0000 7f0db6937640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db00accc0 con 0x7f0db00ae7c0 2024-04-25T10:14:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.463+0000 7f0db6937640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db00ad260 con 0x7f0db006d2f0 2024-04-25T10:14:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.463+0000 7f0daf7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0db00ae7c0 0x7f0db0071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.463+0000 7f0daffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0db015dc90 0x7f0db00b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38622/0 (socket says 172.21.15.60:38622) 2024-04-25T10:14:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.463+0000 7f0daffff640 1 -- 172.21.15.60:0/2961191722 learned_addr learned my addr 172.21.15.60:0/2961191722 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.463+0000 7f0db4ed4640 1 --2- 172.21.15.60:0/2961191722 >> v2:172.21.15.100:3300/0 conn(0x7f0db006d2f0 0x7f0db00717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.463+0000 7f0daf7fe640 1 -- 172.21.15.60:0/2961191722 >> v2:172.21.15.100:3300/0 conn(0x7f0db006d2f0 msgr2=0x7f0db00717d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.463+0000 7f0daf7fe640 1 --2- 172.21.15.60:0/2961191722 >> v2:172.21.15.100:3300/0 conn(0x7f0db006d2f0 0x7f0db00717d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.463+0000 7f0daf7fe640 1 -- 172.21.15.60:0/2961191722 >> v1:172.21.15.60:6789/0 conn(0x7f0db015dc90 legacy=0x7f0db00b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.463+0000 7f0daeffd640 1 -- 172.21.15.60:0/2961191722 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354424085 0 0) 0x7f0db0071fd0 con 0x7f0db015dc90 2024-04-25T10:14:03.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.463+0000 7f0daf7fe640 1 -- 172.21.15.60:0/2961191722 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db00b2ce0 con 0x7f0db00ae7c0 2024-04-25T10:14:03.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0daf7fe640 1 --2- 172.21.15.60:0/2961191722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0db00ae7c0 0x7f0db0071290 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f0db0068540 tx=0x7f0da001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e70726e771158f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:03.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0daeffd640 1 -- 172.21.15.60:0/2961191722 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da00b1070 con 0x7f0db00ae7c0 2024-04-25T10:14:03.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0daeffd640 1 -- 172.21.15.60:0/2961191722 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da00ac5f0 con 0x7f0db00ae7c0 2024-04-25T10:14:03.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0daeffd640 1 -- 172.21.15.60:0/2961191722 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da00ac9c0 con 0x7f0db00ae7c0 2024-04-25T10:14:03.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 -- 172.21.15.60:0/2961191722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0db00ae7c0 msgr2=0x7f0db0071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 --2- 172.21.15.60:0/2961191722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0db00ae7c0 0x7f0db0071290 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f0db0068540 tx=0x7f0da001c480 comp rx=0 tx=0).stop 2024-04-25T10:14:03.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 -- 172.21.15.60:0/2961191722 shutdown_connections 2024-04-25T10:14:03.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 --2- 172.21.15.60:0/2961191722 >> v2:172.21.15.100:3300/0 conn(0x7f0db006d2f0 0x7f0db00717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 --2- 172.21.15.60:0/2961191722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0db00ae7c0 0x7f0db0071290 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 -- 172.21.15.60:0/2961191722 >> 172.21.15.60:0/2961191722 conn(0x7f0db00b12d0 msgr2=0x7f0db00b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:03.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 -- 172.21.15.60:0/2961191722 shutdown_connections 2024-04-25T10:14:03.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 -- 172.21.15.60:0/2961191722 wait complete. 2024-04-25T10:14:03.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 Processor -- start 2024-04-25T10:14:03.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 -- start start 2024-04-25T10:14:03.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db006d2f0 0x7f0db014d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0db00ae7c0 0x7f0db015be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0db015dc90 0x7f0db0154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db016a7e0 con 0x7f0db015dc90 2024-04-25T10:14:03.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db00b2220 con 0x7f0db006d2f0 2024-04-25T10:14:03.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db016a960 con 0x7f0db00ae7c0 2024-04-25T10:14:03.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db4ed4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0db015dc90 0x7f0db0154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db4ed4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0db015dc90 0x7f0db0154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47910/0 (socket says 172.21.15.60:47910) 2024-04-25T10:14:03.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db4ed4640 1 -- 172.21.15.60:0/248703091 learned_addr learned my addr 172.21.15.60:0/248703091 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0daffff640 1 --2- 172.21.15.60:0/248703091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db006d2f0 0x7f0db014d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db4ed4640 1 -- 172.21.15.60:0/248703091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0db00ae7c0 msgr2=0x7f0db015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db4ed4640 1 --2- 172.21.15.60:0/248703091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0db00ae7c0 0x7f0db015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db4ed4640 1 -- 172.21.15.60:0/248703091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db006d2f0 msgr2=0x7f0db014d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db4ed4640 1 --2- 172.21.15.60:0/248703091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db006d2f0 0x7f0db014d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db4ed4640 1 -- 172.21.15.60:0/248703091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db015b090 con 0x7f0db015dc90 2024-04-25T10:14:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db4ed4640 1 --2- 172.21.15.60:0/248703091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0db015dc90 0x7f0db0154ba0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f0da800cc30 tx=0x7f0da800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0dacff9640 1 -- 172.21.15.60:0/248703091 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da800ce40 con 0x7f0db015dc90 2024-04-25T10:14:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0dacff9640 1 -- 172.21.15.60:0/248703091 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da80039a0 con 0x7f0db015dc90 2024-04-25T10:14:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 -- 172.21.15.60:0/248703091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db015b370 con 0x7f0db015dc90 2024-04-25T10:14:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0dacff9640 1 -- 172.21.15.60:0/248703091 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8003d50 con 0x7f0db015dc90 2024-04-25T10:14:03.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.467+0000 7f0db6937640 1 -- 172.21.15.60:0/248703091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db014de30 con 0x7f0db015dc90 2024-04-25T10:14:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.471+0000 7f0db6937640 1 -- 172.21.15.60:0/248703091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db0071fd0 con 0x7f0db015dc90 2024-04-25T10:14:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.471+0000 7f0dacff9640 1 -- 172.21.15.60:0/248703091 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0da8003f70 con 0x7f0db015dc90 2024-04-25T10:14:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.471+0000 7f0dacff9640 1 --2- 172.21.15.60:0/248703091 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0d9c066f60 0x7f0d9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.471+0000 7f0dacff9640 1 -- 172.21.15.60:0/248703091 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f0da8085560 con 0x7f0db015dc90 2024-04-25T10:14:03.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.471+0000 7f0daffff640 1 --2- 172.21.15.60:0/248703091 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0d9c066f60 0x7f0d9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.471+0000 7f0daffff640 1 --2- 172.21.15.60:0/248703091 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0d9c066f60 0x7f0d9c069420 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0da4000f70 tx=0x7f0da4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:03.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.475+0000 7f0dacff9640 1 -- 172.21.15.60:0/248703091 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0da8014030 con 0x7f0db015dc90 2024-04-25T10:14:03.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.667+0000 7f0db6937640 1 -- 172.21.15.60:0/248703091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0db0156a60 con 0x7f0db015dc90 2024-04-25T10:14:03.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.667+0000 7f0dacff9640 1 -- 172.21.15.60:0/248703091 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f0da8056530 con 0x7f0db015dc90 2024-04-25T10:14:03.672 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:03.672 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:44.357874+0000","modified":"2024-04-25T10:13:22.221010+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8256":{"gid":8256,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/3807131024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3807131024},{"type":"v1","addr":"172.21.15.60:6835","nonce":3807131024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:14:03.672 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 68 2024-04-25T10:14:03.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.667+0000 7f0db6937640 1 -- 172.21.15.60:0/248703091 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0d9c066f60 msgr2=0x7f0d9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.667+0000 7f0db6937640 1 --2- 172.21.15.60:0/248703091 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0d9c066f60 0x7f0d9c069420 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0da4000f70 tx=0x7f0da4009040 comp rx=0 tx=0).stop 2024-04-25T10:14:03.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.667+0000 7f0db6937640 1 -- 172.21.15.60:0/248703091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0db015dc90 msgr2=0x7f0db0154ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.667+0000 7f0db6937640 1 --2- 172.21.15.60:0/248703091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0db015dc90 0x7f0db0154ba0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f0da800cc30 tx=0x7f0da800a5c0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.671+0000 7f0db6937640 1 -- 172.21.15.60:0/248703091 shutdown_connections 2024-04-25T10:14:03.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.671+0000 7f0db6937640 1 --2- 172.21.15.60:0/248703091 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0d9c066f60 0x7f0d9c069420 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.671+0000 7f0db6937640 1 --2- 172.21.15.60:0/248703091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0db015dc90 0x7f0db0154ba0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.671+0000 7f0db6937640 1 --2- 172.21.15.60:0/248703091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0db00ae7c0 0x7f0db015be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.671+0000 7f0db6937640 1 --2- 172.21.15.60:0/248703091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db006d2f0 0x7f0db014d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.671+0000 7f0db6937640 1 -- 172.21.15.60:0/248703091 >> 172.21.15.60:0/248703091 conn(0x7f0db00b12d0 msgr2=0x7f0db0156f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:03.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.671+0000 7f0db6937640 1 -- 172.21.15.60:0/248703091 shutdown_connections 2024-04-25T10:14:03.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.671+0000 7f0db6937640 1 -- 172.21.15.60:0/248703091 wait complete. 2024-04-25T10:14:03.690 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.807+0000 7fc3c30ef640 1 Processor -- start 2024-04-25T10:14:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.807+0000 7fc3c30ef640 1 -- start start 2024-04-25T10:14:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.807+0000 7fc3c30ef640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc3bc0ae7c0 0x7fc3bc071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.807+0000 7fc3c30ef640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3bc071fd0 con 0x7fc3bc15dc90 2024-04-25T10:14:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.807+0000 7fc3c30ef640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3bc0b2ce0 con 0x7fc3bc06d2f0 2024-04-25T10:14:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.807+0000 7fc3c30ef640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3bc0accc0 con 0x7fc3bc0ae7c0 2024-04-25T10:14:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.807+0000 7fc3c0e8b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc3bc15dc90 0x7fc3bc0b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38634/0 (socket says 172.21.15.60:38634) 2024-04-25T10:14:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.807+0000 7fc3c0e8b640 1 -- 172.21.15.60:0/1547895290 learned_addr learned my addr 172.21.15.60:0/1547895290 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bbfff640 1 --2- 172.21.15.60:0/1547895290 >> v2:172.21.15.100:3300/0 conn(0x7fc3bc0ae7c0 0x7fc3bc071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bb7fe640 1 -- 172.21.15.60:0/1547895290 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3980999844 0 0) 0x7fc3bc071fd0 con 0x7fc3bc15dc90 2024-04-25T10:14:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bb7fe640 1 -- 172.21.15.60:0/1547895290 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3ac003530 con 0x7fc3bc15dc90 2024-04-25T10:14:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bb7fe640 1 -- 172.21.15.60:0/1547895290 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1250680564 0 0) 0x7fc3ac003530 con 0x7fc3bc15dc90 2024-04-25T10:14:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bb7fe640 1 -- 172.21.15.60:0/1547895290 >> v2:172.21.15.100:3300/0 conn(0x7fc3bc0ae7c0 msgr2=0x7fc3bc071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bb7fe640 1 --2- 172.21.15.60:0/1547895290 >> v2:172.21.15.100:3300/0 conn(0x7fc3bc0ae7c0 0x7fc3bc071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bb7fe640 1 -- 172.21.15.60:0/1547895290 >> v1:172.21.15.100:6789/0 conn(0x7fc3bc06d2f0 legacy=0x7fc3bc0ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bb7fe640 1 -- 172.21.15.60:0/1547895290 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc072290 con 0x7fc3bc15dc90 2024-04-25T10:14:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bb7fe640 1 -- 172.21.15.60:0/1547895290 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fc3b0001680 con 0x7fc3bc15dc90 2024-04-25T10:14:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bb7fe640 1 -- 172.21.15.60:0/1547895290 >> v1:172.21.15.60:6789/0 conn(0x7fc3bc15dc90 legacy=0x7fc3bc0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bb7fe640 1 --2- 172.21.15.60:0/1547895290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3ac0039c0 0x7fc3ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bb7fe640 1 --2- 172.21.15.60:0/1547895290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3ac004810 0x7fc3ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bb7fe640 1 --2- 172.21.15.60:0/1547895290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3ac0051d0 0x7fc3ac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bb7fe640 1 -- 172.21.15.60:0/1547895290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac00c040 con 0x7fc3ac0051d0 2024-04-25T10:14:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bb7fe640 1 -- 172.21.15.60:0/1547895290 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac00c1f0 con 0x7fc3ac0039c0 2024-04-25T10:14:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bb7fe640 1 -- 172.21.15.60:0/1547895290 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac00c3f0 con 0x7fc3ac004810 2024-04-25T10:14:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3c168c640 1 --2- 172.21.15.60:0/1547895290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3ac0051d0 0x7fc3ac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3c0e8b640 1 --2- 172.21.15.60:0/1547895290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3ac0039c0 0x7fc3ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3bbfff640 1 --2- 172.21.15.60:0/1547895290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3ac004810 0x7fc3ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3c168c640 1 -- 172.21.15.60:0/1547895290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3ac004810 msgr2=0x7fc3ac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3c168c640 1 --2- 172.21.15.60:0/1547895290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3ac004810 0x7fc3ac004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3c168c640 1 -- 172.21.15.60:0/1547895290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3ac0039c0 msgr2=0x7fc3ac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3c168c640 1 --2- 172.21.15.60:0/1547895290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3ac0039c0 0x7fc3ac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.811+0000 7fc3c168c640 1 -- 172.21.15.60:0/1547895290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ac003530 con 0x7fc3ac0051d0 2024-04-25T10:14:03.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c168c640 1 -- 172.21.15.60:0/1547895290 reap_dead start 2024-04-25T10:14:03.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c168c640 1 --2- 172.21.15.60:0/1547895290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3ac0051d0 0x7fc3ac00b9d0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4001e70 tx=0x7fc3b401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fb1e3dd1133ec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:03.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3bb7fe640 1 -- 172.21.15.60:0/1547895290 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4001a90 con 0x7fc3ac0051d0 2024-04-25T10:14:03.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3bb7fe640 1 -- 172.21.15.60:0/1547895290 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b4003bc0 con 0x7fc3ac0051d0 2024-04-25T10:14:03.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3bb7fe640 1 -- 172.21.15.60:0/1547895290 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4005020 con 0x7fc3ac0051d0 2024-04-25T10:14:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c30ef640 1 -- 172.21.15.60:0/1547895290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3ac0051d0 msgr2=0x7fc3ac00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c30ef640 1 --2- 172.21.15.60:0/1547895290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3ac0051d0 0x7fc3ac00b9d0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4001e70 tx=0x7fc3b401c780 comp rx=0 tx=0).stop 2024-04-25T10:14:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c30ef640 1 -- 172.21.15.60:0/1547895290 shutdown_connections 2024-04-25T10:14:03.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c30ef640 1 --2- 172.21.15.60:0/1547895290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3ac0051d0 0x7fc3ac00b9d0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c30ef640 1 -- 172.21.15.60:0/1547895290 >> 172.21.15.60:0/1547895290 conn(0x7fc3bc0b12d0 msgr2=0x7fc3bc0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:03.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c30ef640 1 -- 172.21.15.60:0/1547895290 shutdown_connections 2024-04-25T10:14:03.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c30ef640 1 -- 172.21.15.60:0/1547895290 wait complete. 2024-04-25T10:14:03.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c30ef640 1 Processor -- start 2024-04-25T10:14:03.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c30ef640 1 -- start start 2024-04-25T10:14:03.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c30ef640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3ac0051d0 0x7fc3bc1582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c30ef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3bc0c5a00 0x7fc3bc1f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c30ef640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3bc1f39b0 0x7fc3bc15bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c30ef640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0b2010 con 0x7fc3bc0c5a00 2024-04-25T10:14:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c30ef640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0ad260 con 0x7fc3bc1f39b0 2024-04-25T10:14:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c30ef640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0b2190 con 0x7fc3ac0051d0 2024-04-25T10:14:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3bc0c5a00 0x7fc3bc1f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3bc0c5a00 0x7fc3bc1f3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47932/0 (socket says 172.21.15.60:47932) 2024-04-25T10:14:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3bbfff640 1 -- 172.21.15.60:0/2705840351 learned_addr learned my addr 172.21.15.60:0/2705840351 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c0e8b640 1 --2- 172.21.15.60:0/2705840351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3ac0051d0 0x7fc3bc1582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c168c640 1 --2- 172.21.15.60:0/2705840351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3bc1f39b0 0x7fc3bc15bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3bbfff640 1 -- 172.21.15.60:0/2705840351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3ac0051d0 msgr2=0x7fc3bc1582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3bbfff640 1 --2- 172.21.15.60:0/2705840351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3ac0051d0 0x7fc3bc1582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3bbfff640 1 -- 172.21.15.60:0/2705840351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3bc1f39b0 msgr2=0x7fc3bc15bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3bbfff640 1 --2- 172.21.15.60:0/2705840351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3bc1f39b0 0x7fc3bc15bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3bbfff640 1 -- 172.21.15.60:0/2705840351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc0c4540 con 0x7fc3bc0c5a00 2024-04-25T10:14:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c0e8b640 1 --2- 172.21.15.60:0/2705840351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3ac0051d0 0x7fc3bc1582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3c168c640 1 --2- 172.21.15.60:0/2705840351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3bc1f39b0 0x7fc3bc15bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.815+0000 7fc3bbfff640 1 --2- 172.21.15.60:0/2705840351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3bc0c5a00 0x7fc3bc1f3470 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80115c0 tx=0x7fc3a8011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.819+0000 7fc3b97fa640 1 -- 172.21.15.60:0/2705840351 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8015870 con 0x7fc3bc0c5a00 2024-04-25T10:14:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.819+0000 7fc3b97fa640 1 -- 172.21.15.60:0/2705840351 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a8015a10 con 0x7fc3bc0c5a00 2024-04-25T10:14:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.819+0000 7fc3b97fa640 1 -- 172.21.15.60:0/2705840351 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8015dc0 con 0x7fc3bc0c5a00 2024-04-25T10:14:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.819+0000 7fc3c30ef640 1 -- 172.21.15.60:0/2705840351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3bc0c63e0 con 0x7fc3bc0c5a00 2024-04-25T10:14:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.819+0000 7fc3c30ef640 1 -- 172.21.15.60:0/2705840351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3bc0c69a0 con 0x7fc3bc0c5a00 2024-04-25T10:14:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.819+0000 7fc3c30ef640 1 -- 172.21.15.60:0/2705840351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3bc074770 con 0x7fc3bc0c5a00 2024-04-25T10:14:03.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.819+0000 7fc3b97fa640 1 -- 172.21.15.60:0/2705840351 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3a800f460 con 0x7fc3bc0c5a00 2024-04-25T10:14:03.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.819+0000 7fc3b97fa640 1 --2- 172.21.15.60:0/2705840351 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc3a0066f90 0x7fc3a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:03.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.819+0000 7fc3b97fa640 1 -- 172.21.15.60:0/2705840351 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fc3a8085540 con 0x7fc3bc0c5a00 2024-04-25T10:14:03.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.819+0000 7fc3c0e8b640 1 --2- 172.21.15.60:0/2705840351 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc3a0066f90 0x7fc3a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:03.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.819+0000 7fc3c0e8b640 1 --2- 172.21.15.60:0/2705840351 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc3a0066f90 0x7fc3a0069450 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0001ec0 tx=0x7fc3b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:03.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.823+0000 7fc3b97fa640 1 -- 172.21.15.60:0/2705840351 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3a8019030 con 0x7fc3bc0c5a00 2024-04-25T10:14:03.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.979+0000 7fc3c30ef640 1 -- 172.21.15.60:0/2705840351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3bc0b29e0 con 0x7fc3bc0c5a00 2024-04-25T10:14:03.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.979+0000 7fc3b97fa640 1 -- 172.21.15.60:0/2705840351 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16981 (secure 0 0 0) 0x7fc3a8055e30 con 0x7fc3bc0c5a00 2024-04-25T10:14:03.984 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:03.984 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":86,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:13:19.192820+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:42.621502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:43.352679+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/2817879567":"2024-04-26T10:13:19.192767+0000","172.21.15.60:6833/2817879567":"2024-04-26T10:13:19.192767+0000","172.21.15.60:6835/2455562778":"2024-04-26T10:11:58.136789+0000","172.21.15.60:6833/2455562778":"2024-04-26T10:11:58.136789+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:03.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.979+0000 7fc3c30ef640 1 -- 172.21.15.60:0/2705840351 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc3a0066f90 msgr2=0x7fc3a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.979+0000 7fc3c30ef640 1 --2- 172.21.15.60:0/2705840351 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc3a0066f90 0x7fc3a0069450 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0001ec0 tx=0x7fc3b0009040 comp rx=0 tx=0).stop 2024-04-25T10:14:03.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.979+0000 7fc3c30ef640 1 -- 172.21.15.60:0/2705840351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3bc0c5a00 msgr2=0x7fc3bc1f3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:03.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.979+0000 7fc3c30ef640 1 --2- 172.21.15.60:0/2705840351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3bc0c5a00 0x7fc3bc1f3470 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80115c0 tx=0x7fc3a8011a80 comp rx=0 tx=0).stop 2024-04-25T10:14:03.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.983+0000 7fc3c30ef640 1 -- 172.21.15.60:0/2705840351 shutdown_connections 2024-04-25T10:14:03.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.983+0000 7fc3c30ef640 1 --2- 172.21.15.60:0/2705840351 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc3a0066f90 0x7fc3a0069450 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.983+0000 7fc3c30ef640 1 --2- 172.21.15.60:0/2705840351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3bc1f39b0 0x7fc3bc15bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.983+0000 7fc3c30ef640 1 --2- 172.21.15.60:0/2705840351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3bc0c5a00 0x7fc3bc1f3470 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.983+0000 7fc3c30ef640 1 --2- 172.21.15.60:0/2705840351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3ac0051d0 0x7fc3bc1582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:03.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.983+0000 7fc3c30ef640 1 -- 172.21.15.60:0/2705840351 >> 172.21.15.60:0/2705840351 conn(0x7fc3bc0b12d0 msgr2=0x7fc3bc0adb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:03.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.983+0000 7fc3c30ef640 1 -- 172.21.15.60:0/2705840351 shutdown_connections 2024-04-25T10:14:03.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:03.983+0000 7fc3c30ef640 1 -- 172.21.15.60:0/2705840351 wait complete. 2024-04-25T10:14:04.002 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:14:04.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3dfa608640 1 Processor -- start 2024-04-25T10:14:04.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3dfa608640 1 -- start start 2024-04-25T10:14:04.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3dfa608640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3df415bc90 0x7f3df415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3dfa608640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df4071ca0 con 0x7f3df40ce7a0 2024-04-25T10:14:04.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3dfa608640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df40cca40 con 0x7f3df40b4660 2024-04-25T10:14:04.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3dfa608640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40cc370 con 0x7f3df415bc90 2024-04-25T10:14:04.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3df3fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3df40ce7a0 0x7f3df40ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38644/0 (socket says 172.21.15.60:38644) 2024-04-25T10:14:04.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3df3fff640 1 -- 172.21.15.60:0/441822511 learned_addr learned my addr 172.21.15.60:0/441822511 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:04.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3df37fe640 1 --2- 172.21.15.60:0/441822511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3df415bc90 0x7f3df415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3df2ffd640 1 -- 172.21.15.60:0/441822511 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591407254 0 0) 0x7f3df4071ca0 con 0x7f3df40ce7a0 2024-04-25T10:14:04.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3df2ffd640 1 -- 172.21.15.60:0/441822511 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3de0003530 con 0x7f3df40ce7a0 2024-04-25T10:14:04.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3df2ffd640 1 -- 172.21.15.60:0/441822511 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2515916634 0 0) 0x7f3de0003530 con 0x7f3df40ce7a0 2024-04-25T10:14:04.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3df2ffd640 1 -- 172.21.15.60:0/441822511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3df415bc90 msgr2=0x7f3df415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:04.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3df2ffd640 1 --2- 172.21.15.60:0/441822511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3df415bc90 0x7f3df415c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3df2ffd640 1 -- 172.21.15.60:0/441822511 >> v1:172.21.15.100:6789/0 conn(0x7f3df40b4660 legacy=0x7f3df415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3df2ffd640 1 -- 172.21.15.60:0/441822511 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df40ccd20 con 0x7f3df40ce7a0 2024-04-25T10:14:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3df2ffd640 1 -- 172.21.15.60:0/441822511 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f3de4001680 con 0x7f3df40ce7a0 2024-04-25T10:14:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3df2ffd640 1 -- 172.21.15.60:0/441822511 >> v1:172.21.15.60:6789/0 conn(0x7f3df40ce7a0 legacy=0x7f3df40ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.119+0000 7f3df2ffd640 1 --2- 172.21.15.60:0/441822511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de00039f0 0x7f3de0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df2ffd640 1 --2- 172.21.15.60:0/441822511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3df415bc90 0x7f3de0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df2ffd640 1 --2- 172.21.15.60:0/441822511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de0004d50 0x7f3de00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df2ffd640 1 -- 172.21.15.60:0/441822511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de000bb60 con 0x7f3de00039f0 2024-04-25T10:14:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df2ffd640 1 -- 172.21.15.60:0/441822511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de000bce0 con 0x7f3de0004d50 2024-04-25T10:14:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df2ffd640 1 -- 172.21.15.60:0/441822511 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de000bee0 con 0x7f3df415bc90 2024-04-25T10:14:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df3fff640 1 --2- 172.21.15.60:0/441822511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de00039f0 0x7f3de0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df8ba5640 1 --2- 172.21.15.60:0/441822511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de0004d50 0x7f3de00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df37fe640 1 --2- 172.21.15.60:0/441822511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3df415bc90 0x7f3de0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df3fff640 1 -- 172.21.15.60:0/441822511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3df415bc90 msgr2=0x7f3de0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:04.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df3fff640 1 --2- 172.21.15.60:0/441822511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3df415bc90 0x7f3de0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df3fff640 1 -- 172.21.15.60:0/441822511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de0004d50 msgr2=0x7f3de00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:04.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df3fff640 1 --2- 172.21.15.60:0/441822511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de0004d50 0x7f3de00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df3fff640 1 -- 172.21.15.60:0/441822511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de0003530 con 0x7f3de00039f0 2024-04-25T10:14:04.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df37fe640 1 --2- 172.21.15.60:0/441822511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3df415bc90 0x7f3de0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:04.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df8ba5640 1 --2- 172.21.15.60:0/441822511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de0004d50 0x7f3de00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:04.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df3fff640 1 --2- 172.21.15.60:0/441822511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de00039f0 0x7f3de0003e10 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f3de400c240 tx=0x7f3df415d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5e7d83838c594d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:04.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df2ffd640 1 -- 172.21.15.60:0/441822511 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4007d20 con 0x7f3de00039f0 2024-04-25T10:14:04.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df2ffd640 1 -- 172.21.15.60:0/441822511 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de40a9020 con 0x7f3de00039f0 2024-04-25T10:14:04.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df2ffd640 1 -- 172.21.15.60:0/441822511 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4005b00 con 0x7f3de00039f0 2024-04-25T10:14:04.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3dfa608640 1 -- 172.21.15.60:0/441822511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de00039f0 msgr2=0x7f3de0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3dfa608640 1 --2- 172.21.15.60:0/441822511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de00039f0 0x7f3de0003e10 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f3de400c240 tx=0x7f3df415d680 comp rx=0 tx=0).stop 2024-04-25T10:14:04.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3df8ba5640 1 -- 172.21.15.60:0/441822511 reap_dead start 2024-04-25T10:14:04.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3dfa608640 1 -- 172.21.15.60:0/441822511 shutdown_connections 2024-04-25T10:14:04.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3dfa608640 1 -- 172.21.15.60:0/441822511 >> 172.21.15.60:0/441822511 conn(0x7f3df40b12d0 msgr2=0x7f3df40b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:04.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3dfa608640 1 -- 172.21.15.60:0/441822511 shutdown_connections 2024-04-25T10:14:04.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.123+0000 7f3dfa608640 1 -- 172.21.15.60:0/441822511 wait complete. 2024-04-25T10:14:04.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3dfa608640 1 Processor -- start 2024-04-25T10:14:04.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3dfa608640 1 -- start start 2024-04-25T10:14:04.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3dfa608640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de00039f0 0x7f3df4159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3dfa608640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df41f4700 0x7f3df415f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3dfa608640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3df4153c80 0x7f3df40b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3dfa608640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40b3db0 con 0x7f3de00039f0 2024-04-25T10:14:04.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3dfa608640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40b4490 con 0x7f3df41f4700 2024-04-25T10:14:04.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3dfa608640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41675c0 con 0x7f3df4153c80 2024-04-25T10:14:04.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de00039f0 0x7f3df4159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de00039f0 0x7f3df4159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47954/0 (socket says 172.21.15.60:47954) 2024-04-25T10:14:04.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df3fff640 1 -- 172.21.15.60:0/4196836799 learned_addr learned my addr 172.21.15.60:0/4196836799 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:04.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df37fe640 1 --2- 172.21.15.60:0/4196836799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df41f4700 0x7f3df415f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df8ba5640 1 --2- 172.21.15.60:0/4196836799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3df4153c80 0x7f3df40b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df3fff640 1 -- 172.21.15.60:0/4196836799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3df4153c80 msgr2=0x7f3df40b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df3fff640 1 --2- 172.21.15.60:0/4196836799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3df4153c80 0x7f3df40b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df3fff640 1 -- 172.21.15.60:0/4196836799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df41f4700 msgr2=0x7f3df415f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df3fff640 1 --2- 172.21.15.60:0/4196836799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df41f4700 0x7f3df415f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df3fff640 1 -- 172.21.15.60:0/4196836799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df4154100 con 0x7f3de00039f0 2024-04-25T10:14:04.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df8ba5640 1 --2- 172.21.15.60:0/4196836799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3df4153c80 0x7f3df40b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:04.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df37fe640 1 --2- 172.21.15.60:0/4196836799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df41f4700 0x7f3df415f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:04.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df3fff640 1 --2- 172.21.15.60:0/4196836799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de00039f0 0x7f3df4159b10 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f3de40ab040 tx=0x7f3de0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:04.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df0ff9640 1 -- 172.21.15.60:0/4196836799 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4004ab0 con 0x7f3de00039f0 2024-04-25T10:14:04.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df0ff9640 1 -- 172.21.15.60:0/4196836799 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de4004c50 con 0x7f3de00039f0 2024-04-25T10:14:04.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3dfa608640 1 -- 172.21.15.60:0/4196836799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df415fc00 con 0x7f3de00039f0 2024-04-25T10:14:04.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df0ff9640 1 -- 172.21.15.60:0/4196836799 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de40070d0 con 0x7f3de00039f0 2024-04-25T10:14:04.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3dfa608640 1 -- 172.21.15.60:0/4196836799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df41601c0 con 0x7f3de00039f0 2024-04-25T10:14:04.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3dfa608640 1 -- 172.21.15.60:0/4196836799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df4074770 con 0x7f3de00039f0 2024-04-25T10:14:04.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.127+0000 7f3df0ff9640 1 -- 172.21.15.60:0/4196836799 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3de4007290 con 0x7f3de00039f0 2024-04-25T10:14:04.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.131+0000 7f3df0ff9640 1 --2- 172.21.15.60:0/4196836799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3dd4066f90 0x7f3dd4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.131+0000 7f3df0ff9640 1 -- 172.21.15.60:0/4196836799 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f3de411e8f0 con 0x7f3de00039f0 2024-04-25T10:14:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.131+0000 7f3df37fe640 1 --2- 172.21.15.60:0/4196836799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3dd4066f90 0x7f3dd4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.131+0000 7f3df37fe640 1 --2- 172.21.15.60:0/4196836799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3dd4066f90 0x7f3dd4069450 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3df415f140 tx=0x7f3ddc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.135+0000 7f3df0ff9640 1 -- 172.21.15.60:0/4196836799 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3de4016990 con 0x7f3de00039f0 2024-04-25T10:14:04.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.327+0000 7f3dfa608640 1 -- 172.21.15.60:0/4196836799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3df40cbfc0 con 0x7f3de00039f0 2024-04-25T10:14:04.333 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:14:04.327+0000 7f3a792d6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:14:04.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.331+0000 7f3df0ff9640 1 -- 172.21.15.60:0/4196836799 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3de40c03a0 con 0x7f3de00039f0 2024-04-25T10:14:04.336 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:14:04.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.335+0000 7f3dfa608640 1 -- 172.21.15.60:0/4196836799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3dd4066f90 msgr2=0x7f3dd4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.335+0000 7f3dfa608640 1 --2- 172.21.15.60:0/4196836799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3dd4066f90 0x7f3dd4069450 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3df415f140 tx=0x7f3ddc008040 comp rx=0 tx=0).stop 2024-04-25T10:14:04.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.335+0000 7f3dfa608640 1 -- 172.21.15.60:0/4196836799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de00039f0 msgr2=0x7f3df4159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.335+0000 7f3dfa608640 1 --2- 172.21.15.60:0/4196836799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de00039f0 0x7f3df4159b10 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f3de40ab040 tx=0x7f3de0005840 comp rx=0 tx=0).stop 2024-04-25T10:14:04.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.335+0000 7f3dfa608640 1 -- 172.21.15.60:0/4196836799 shutdown_connections 2024-04-25T10:14:04.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.335+0000 7f3dfa608640 1 --2- 172.21.15.60:0/4196836799 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3dd4066f90 0x7f3dd4069450 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.335+0000 7f3dfa608640 1 --2- 172.21.15.60:0/4196836799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3df4153c80 0x7f3df40b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.335+0000 7f3dfa608640 1 --2- 172.21.15.60:0/4196836799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df41f4700 0x7f3df415f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.335+0000 7f3dfa608640 1 --2- 172.21.15.60:0/4196836799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de00039f0 0x7f3df4159b10 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.335+0000 7f3dfa608640 1 -- 172.21.15.60:0/4196836799 >> 172.21.15.60:0/4196836799 conn(0x7f3df40b12d0 msgr2=0x7f3df40b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:04.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.335+0000 7f3dfa608640 1 -- 172.21.15.60:0/4196836799 shutdown_connections 2024-04-25T10:14:04.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.335+0000 7f3dfa608640 1 -- 172.21.15.60:0/4196836799 wait complete. 2024-04-25T10:14:04.343 INFO:tasks.ceph.mds.c.smithi060.stderr: -2339> 2024-04-25T10:13:25.207+0000 7f1d83d39640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:14:04.343 INFO:tasks.ceph.mds.c.smithi060.stderr: -1916> 2024-04-25T10:13:27.523+0000 7f1d7fd31640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5644a4940680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:14:04.359 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:14:04.371 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:14:04.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.471+0000 7f4413d5a640 1 Processor -- start 2024-04-25T10:14:04.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.471+0000 7f4413d5a640 1 -- start start 2024-04-25T10:14:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.471+0000 7f4413d5a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f440c0cbcf0 0x7f440c160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.471+0000 7f4413d5a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f440c071c30 0x7f440c15f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.471+0000 7f4413d5a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440c144a40 con 0x7f440c15fa20 2024-04-25T10:14:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.471+0000 7f4413d5a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f440c0b2dc0 con 0x7f440c0cbcf0 2024-04-25T10:14:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.471+0000 7f4413d5a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c15fe80 con 0x7f440c071c30 2024-04-25T10:14:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f44122f7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f440c15fa20 0x7f440c15bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38648/0 (socket says 172.21.15.60:38648) 2024-04-25T10:14:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f44122f7640 1 -- 172.21.15.60:0/943413166 learned_addr learned my addr 172.21.15.60:0/943413166 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4411af6640 1 --2- 172.21.15.60:0/943413166 >> v2:172.21.15.100:3300/0 conn(0x7f440c0cbcf0 0x7f440c160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f44112f5640 1 --2- 172.21.15.60:0/943413166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f440c071c30 0x7f440c15f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 -- 172.21.15.60:0/943413166 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737575487 0 0) 0x7f440c144a40 con 0x7f440c15fa20 2024-04-25T10:14:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 -- 172.21.15.60:0/943413166 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43f8003530 con 0x7f440c15fa20 2024-04-25T10:14:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 -- 172.21.15.60:0/943413166 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3467151696 0 0) 0x7f43f8003530 con 0x7f440c15fa20 2024-04-25T10:14:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 -- 172.21.15.60:0/943413166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f440c071c30 msgr2=0x7f440c15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 --2- 172.21.15.60:0/943413166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f440c071c30 0x7f440c15f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 -- 172.21.15.60:0/943413166 >> v2:172.21.15.100:3300/0 conn(0x7f440c0cbcf0 msgr2=0x7f440c160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 --2- 172.21.15.60:0/943413166 >> v2:172.21.15.100:3300/0 conn(0x7f440c0cbcf0 0x7f440c160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 -- 172.21.15.60:0/943413166 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c0cc9f0 con 0x7f440c15fa20 2024-04-25T10:14:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 -- 172.21.15.60:0/943413166 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f4404001a00 con 0x7f440c15fa20 2024-04-25T10:14:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 -- 172.21.15.60:0/943413166 >> v1:172.21.15.60:6789/0 conn(0x7f440c15fa20 legacy=0x7f440c15bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 --2- 172.21.15.60:0/943413166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f80039c0 0x7f43f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 --2- 172.21.15.60:0/943413166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43f8004810 0x7f43f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 --2- 172.21.15.60:0/943413166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f440c071c30 0x7f43f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 -- 172.21.15.60:0/943413166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f800b9d0 con 0x7f43f8004810 2024-04-25T10:14:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 -- 172.21.15.60:0/943413166 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f800bb50 con 0x7f43f80039c0 2024-04-25T10:14:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 -- 172.21.15.60:0/943413166 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f800bcd0 con 0x7f440c071c30 2024-04-25T10:14:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f44112f5640 1 --2- 172.21.15.60:0/943413166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43f8004810 0x7f43f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4411af6640 1 --2- 172.21.15.60:0/943413166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f80039c0 0x7f43f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f44122f7640 1 --2- 172.21.15.60:0/943413166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f440c071c30 0x7f43f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f44112f5640 1 -- 172.21.15.60:0/943413166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f440c071c30 msgr2=0x7f43f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:04.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f44112f5640 1 --2- 172.21.15.60:0/943413166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f440c071c30 0x7f43f80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f44112f5640 1 -- 172.21.15.60:0/943413166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f80039c0 msgr2=0x7f43f8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:04.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f44112f5640 1 --2- 172.21.15.60:0/943413166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43f80039c0 0x7f43f8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f44112f5640 1 -- 172.21.15.60:0/943413166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f8003530 con 0x7f43f8004810 2024-04-25T10:14:04.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f44122f7640 1 --2- 172.21.15.60:0/943413166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f440c071c30 0x7f43f80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:04.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f44112f5640 1 --2- 172.21.15.60:0/943413166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43f8004810 0x7f43f8004c90 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009ac0 tx=0x7f43fc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771c2bbbd8879ac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:04.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 -- 172.21.15.60:0/943413166 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0b2070 con 0x7f43f8004810 2024-04-25T10:14:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 -- 172.21.15.60:0/943413166 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc0a5560 con 0x7f43f8004810 2024-04-25T10:14:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.475+0000 7f4410af4640 1 -- 172.21.15.60:0/943413166 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc009690 con 0x7f43f8004810 2024-04-25T10:14:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 -- 172.21.15.60:0/943413166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43f8004810 msgr2=0x7f43f8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 --2- 172.21.15.60:0/943413166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43f8004810 0x7f43f8004c90 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009ac0 tx=0x7f43fc01ca50 comp rx=0 tx=0).stop 2024-04-25T10:14:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f44122f7640 1 -- 172.21.15.60:0/943413166 reap_dead start 2024-04-25T10:14:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 -- 172.21.15.60:0/943413166 shutdown_connections 2024-04-25T10:14:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 -- 172.21.15.60:0/943413166 >> 172.21.15.60:0/943413166 conn(0x7f440c0670b0 msgr2=0x7f440c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 -- 172.21.15.60:0/943413166 shutdown_connections 2024-04-25T10:14:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 -- 172.21.15.60:0/943413166 wait complete. 2024-04-25T10:14:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 Processor -- start 2024-04-25T10:14:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 -- start start 2024-04-25T10:14:04.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43f8004810 0x7f440c15b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f440c139950 0x7f440c139db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f440c13c1e0 0x7f440c13c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c0ce9a0 con 0x7f440c13c1e0 2024-04-25T10:14:04.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c0ccfb0 con 0x7f440c139950 2024-04-25T10:14:04.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c0cc0d0 con 0x7f43f8004810 2024-04-25T10:14:04.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f44122f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f440c13c1e0 0x7f440c13c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f44122f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f440c13c1e0 0x7f440c13c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48004/0 (socket says 172.21.15.60:48004) 2024-04-25T10:14:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f44122f7640 1 -- 172.21.15.60:0/983587181 learned_addr learned my addr 172.21.15.60:0/983587181 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4411af6640 1 --2- 172.21.15.60:0/983587181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43f8004810 0x7f440c15b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f44112f5640 1 --2- 172.21.15.60:0/983587181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f440c139950 0x7f440c139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f44122f7640 1 -- 172.21.15.60:0/983587181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43f8004810 msgr2=0x7f440c15b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f44122f7640 1 --2- 172.21.15.60:0/983587181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43f8004810 0x7f440c15b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f44122f7640 1 -- 172.21.15.60:0/983587181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f440c139950 msgr2=0x7f440c139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f44122f7640 1 --2- 172.21.15.60:0/983587181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f440c139950 0x7f440c139db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f44122f7640 1 -- 172.21.15.60:0/983587181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c13a3f0 con 0x7f440c13c1e0 2024-04-25T10:14:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f44112f5640 1 --2- 172.21.15.60:0/983587181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f440c139950 0x7f440c139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f44122f7640 1 --2- 172.21.15.60:0/983587181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f440c13c1e0 0x7f440c13c670 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f440c0aee10 tx=0x7f4404010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f44027fc640 1 -- 172.21.15.60:0/983587181 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440401a070 con 0x7f440c13c1e0 2024-04-25T10:14:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f44027fc640 1 -- 172.21.15.60:0/983587181 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4404011040 con 0x7f440c13c1e0 2024-04-25T10:14:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 -- 172.21.15.60:0/983587181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440c13a6d0 con 0x7f440c13c1e0 2024-04-25T10:14:04.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f44027fc640 1 -- 172.21.15.60:0/983587181 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440400ecc0 con 0x7f440c13c1e0 2024-04-25T10:14:04.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 -- 172.21.15.60:0/983587181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440c13ac90 con 0x7f440c13c1e0 2024-04-25T10:14:04.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.479+0000 7f4413d5a640 1 -- 172.21.15.60:0/983587181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f440c06bba0 con 0x7f440c13c1e0 2024-04-25T10:14:04.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.483+0000 7f44027fc640 1 -- 172.21.15.60:0/983587181 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f440401d070 con 0x7f440c13c1e0 2024-04-25T10:14:04.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.483+0000 7f44027fc640 1 --2- 172.21.15.60:0/983587181 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f43f0066f90 0x7f43f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.483+0000 7f44027fc640 1 -- 172.21.15.60:0/983587181 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5996+0+0 (secure 0 0 0) 0x7f44040878c0 con 0x7f440c13c1e0 2024-04-25T10:14:04.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.483+0000 7f4411af6640 1 --2- 172.21.15.60:0/983587181 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f43f0066f90 0x7f43f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.483+0000 7f4411af6640 1 --2- 172.21.15.60:0/983587181 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f43f0066f90 0x7f43f0069450 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4408009ab0 tx=0x7f4408008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:04.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.487+0000 7f44027fc640 1 -- 172.21.15.60:0/983587181 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f440400ee60 con 0x7f440c13c1e0 2024-04-25T10:14:04.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:04.675+0000 7f4413d5a640 1 -- 172.21.15.60:0/983587181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f440c0b2970 con 0x7f440c13c1e0 2024-04-25T10:14:05.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.335+0000 7f44027fc640 1 -- 172.21.15.60:0/983587181 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v70) v1 ==== 104+0+0 (secure 0 0 0) 0x7f44040580c0 con 0x7f440c13c1e0 2024-04-25T10:14:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.335+0000 7f4413d5a640 1 -- 172.21.15.60:0/983587181 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f43f0066f90 msgr2=0x7f43f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.335+0000 7f4413d5a640 1 --2- 172.21.15.60:0/983587181 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f43f0066f90 0x7f43f0069450 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4408009ab0 tx=0x7f4408008040 comp rx=0 tx=0).stop 2024-04-25T10:14:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.335+0000 7f4413d5a640 1 -- 172.21.15.60:0/983587181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f440c13c1e0 msgr2=0x7f440c13c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.335+0000 7f4413d5a640 1 --2- 172.21.15.60:0/983587181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f440c13c1e0 0x7f440c13c670 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f440c0aee10 tx=0x7f4404010d70 comp rx=0 tx=0).stop 2024-04-25T10:14:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.339+0000 7f4413d5a640 1 -- 172.21.15.60:0/983587181 shutdown_connections 2024-04-25T10:14:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.339+0000 7f4413d5a640 1 --2- 172.21.15.60:0/983587181 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f43f0066f90 0x7f43f0069450 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:05.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.339+0000 7f4413d5a640 1 --2- 172.21.15.60:0/983587181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f440c13c1e0 0x7f440c13c670 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.339+0000 7f4413d5a640 1 --2- 172.21.15.60:0/983587181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f440c139950 0x7f440c139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.339+0000 7f4413d5a640 1 --2- 172.21.15.60:0/983587181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43f8004810 0x7f440c15b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.339+0000 7f4413d5a640 1 -- 172.21.15.60:0/983587181 >> 172.21.15.60:0/983587181 conn(0x7f440c0670b0 msgr2=0x7f440c144eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.339+0000 7f4413d5a640 1 -- 172.21.15.60:0/983587181 shutdown_connections 2024-04-25T10:14:05.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.339+0000 7f4413d5a640 1 -- 172.21.15.60:0/983587181 wait complete. 2024-04-25T10:14:05.364 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:14:05.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.535+0000 7fa7e213a640 1 Processor -- start 2024-04-25T10:14:05.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.535+0000 7fa7e213a640 1 -- start start 2024-04-25T10:14:05.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.535+0000 7fa7e213a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7dc071980 0x7fa7dc0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.535+0000 7fa7e213a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7dc0cf810 con 0x7fa7dc144e40 2024-04-25T10:14:05.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.535+0000 7fa7e213a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7dc0cfaf0 con 0x7fa7dc0cf0a0 2024-04-25T10:14:05.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.535+0000 7fa7e213a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc0b3060 con 0x7fa7dc071980 2024-04-25T10:14:05.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.535+0000 7fa7db7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa7dc144e40 0x7fa7dc071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38654/0 (socket says 172.21.15.60:38654) 2024-04-25T10:14:05.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.535+0000 7fa7db7fe640 1 -- 172.21.15.60:0/2035430513 learned_addr learned my addr 172.21.15.60:0/2035430513 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:05.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.535+0000 7fa7daffd640 1 --2- 172.21.15.60:0/2035430513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7dc071980 0x7fa7dc0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:05.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.535+0000 7fa7da7fc640 1 -- 172.21.15.60:0/2035430513 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186157444 0 0) 0x7fa7dc0cf810 con 0x7fa7dc144e40 2024-04-25T10:14:05.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7da7fc640 1 -- 172.21.15.60:0/2035430513 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7c4003530 con 0x7fa7dc144e40 2024-04-25T10:14:05.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7da7fc640 1 -- 172.21.15.60:0/2035430513 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2697150345 0 0) 0x7fa7c4003530 con 0x7fa7dc144e40 2024-04-25T10:14:05.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7da7fc640 1 -- 172.21.15.60:0/2035430513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7dc071980 msgr2=0x7fa7dc0ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:05.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7da7fc640 1 --2- 172.21.15.60:0/2035430513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7dc071980 0x7fa7dc0ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:05.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7da7fc640 1 -- 172.21.15.60:0/2035430513 >> v1:172.21.15.100:6789/0 conn(0x7fa7dc0cf0a0 legacy=0x7fa7dc0b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:05.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7da7fc640 1 -- 172.21.15.60:0/2035430513 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc0cfdd0 con 0x7fa7dc144e40 2024-04-25T10:14:05.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7da7fc640 1 -- 172.21.15.60:0/2035430513 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fa7cc001680 con 0x7fa7dc144e40 2024-04-25T10:14:05.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7da7fc640 1 -- 172.21.15.60:0/2035430513 >> v1:172.21.15.60:6789/0 conn(0x7fa7dc144e40 legacy=0x7fa7dc071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:05.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7da7fc640 1 --2- 172.21.15.60:0/2035430513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7c40039f0 0x7fa7c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7da7fc640 1 --2- 172.21.15.60:0/2035430513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7dc071980 0x7fa7c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7da7fc640 1 --2- 172.21.15.60:0/2035430513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c4004d50 0x7fa7c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7da7fc640 1 -- 172.21.15.60:0/2035430513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc0b3060 con 0x7fa7c4004d50 2024-04-25T10:14:05.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7da7fc640 1 -- 172.21.15.60:0/2035430513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c400b9d0 con 0x7fa7c40039f0 2024-04-25T10:14:05.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7da7fc640 1 -- 172.21.15.60:0/2035430513 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c400bbd0 con 0x7fa7dc071980 2024-04-25T10:14:05.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7dbfff640 1 --2- 172.21.15.60:0/2035430513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c4004d50 0x7fa7c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:05.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7daffd640 1 --2- 172.21.15.60:0/2035430513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7dc071980 0x7fa7c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:05.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7db7fe640 1 --2- 172.21.15.60:0/2035430513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7c40039f0 0x7fa7c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:05.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7dbfff640 1 -- 172.21.15.60:0/2035430513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7dc071980 msgr2=0x7fa7c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:05.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7dbfff640 1 --2- 172.21.15.60:0/2035430513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7dc071980 0x7fa7c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:05.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7dbfff640 1 -- 172.21.15.60:0/2035430513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7c40039f0 msgr2=0x7fa7c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:05.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7dbfff640 1 --2- 172.21.15.60:0/2035430513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7c40039f0 0x7fa7c4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:05.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7dbfff640 1 -- 172.21.15.60:0/2035430513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c4003530 con 0x7fa7c4004d50 2024-04-25T10:14:05.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.539+0000 7fa7dbfff640 1 --2- 172.21.15.60:0/2035430513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c4004d50 0x7fa7c40051d0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4002010 tx=0x7fa7d401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a34041eb1256a7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:05.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7da7fc640 1 -- 172.21.15.60:0/2035430513 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40b2070 con 0x7fa7c4004d50 2024-04-25T10:14:05.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7da7fc640 1 -- 172.21.15.60:0/2035430513 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d40a57f0 con 0x7fa7c4004d50 2024-04-25T10:14:05.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7da7fc640 1 -- 172.21.15.60:0/2035430513 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4001c10 con 0x7fa7c4004d50 2024-04-25T10:14:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7e213a640 1 -- 172.21.15.60:0/2035430513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c4004d50 msgr2=0x7fa7c40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7e213a640 1 --2- 172.21.15.60:0/2035430513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c4004d50 0x7fa7c40051d0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4002010 tx=0x7fa7d401c8e0 comp rx=0 tx=0).stop 2024-04-25T10:14:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7dbfff640 1 -- 172.21.15.60:0/2035430513 reap_dead start 2024-04-25T10:14:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7e213a640 1 -- 172.21.15.60:0/2035430513 shutdown_connections 2024-04-25T10:14:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7e213a640 1 -- 172.21.15.60:0/2035430513 >> 172.21.15.60:0/2035430513 conn(0x7fa7dc066fa0 msgr2=0x7fa7dc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7e213a640 1 -- 172.21.15.60:0/2035430513 shutdown_connections 2024-04-25T10:14:05.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7e213a640 1 -- 172.21.15.60:0/2035430513 wait complete. 2024-04-25T10:14:05.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7e213a640 1 Processor -- start 2024-04-25T10:14:05.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7e213a640 1 -- start start 2024-04-25T10:14:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7e213a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c4004d50 0x7fa7dc13efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7e213a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7dc0ad5e0 0x7fa7dc15af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7e213a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7dc0aaab0 0x7fa7dc0aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7e213a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc1453a0 con 0x7fa7c4004d50 2024-04-25T10:14:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7e213a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc145220 con 0x7fa7dc0aaab0 2024-04-25T10:14:05.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7e213a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc0ccc60 con 0x7fa7dc0ad5e0 2024-04-25T10:14:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7db7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c4004d50 0x7fa7dc13efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7db7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c4004d50 0x7fa7dc13efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48012/0 (socket says 172.21.15.60:48012) 2024-04-25T10:14:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7db7fe640 1 -- 172.21.15.60:0/2488592032 learned_addr learned my addr 172.21.15.60:0/2488592032 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7dbfff640 1 --2- 172.21.15.60:0/2488592032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7dc0aaab0 0x7fa7dc0aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7daffd640 1 --2- 172.21.15.60:0/2488592032 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7dc0ad5e0 0x7fa7dc15af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7db7fe640 1 -- 172.21.15.60:0/2488592032 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7dc0ad5e0 msgr2=0x7fa7dc15af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7db7fe640 1 --2- 172.21.15.60:0/2488592032 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7dc0ad5e0 0x7fa7dc15af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7db7fe640 1 -- 172.21.15.60:0/2488592032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7dc0aaab0 msgr2=0x7fa7dc0aaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7db7fe640 1 --2- 172.21.15.60:0/2488592032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7dc0aaab0 0x7fa7dc0aaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7db7fe640 1 -- 172.21.15.60:0/2488592032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc15b490 con 0x7fa7c4004d50 2024-04-25T10:14:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7dbfff640 1 --2- 172.21.15.60:0/2488592032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7dc0aaab0 0x7fa7dc0aaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7db7fe640 1 --2- 172.21.15.60:0/2488592032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c4004d50 0x7fa7dc13efc0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc001b10 tx=0x7fa7cc012770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:05.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7c3fff640 1 -- 172.21.15.60:0/2488592032 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc01c070 con 0x7fa7c4004d50 2024-04-25T10:14:05.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7c3fff640 1 -- 172.21.15.60:0/2488592032 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc014750 con 0x7fa7c4004d50 2024-04-25T10:14:05.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.543+0000 7fa7c3fff640 1 -- 172.21.15.60:0/2488592032 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc014b00 con 0x7fa7c4004d50 2024-04-25T10:14:05.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.547+0000 7fa7e213a640 1 -- 172.21.15.60:0/2488592032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7dc0ada60 con 0x7fa7c4004d50 2024-04-25T10:14:05.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.547+0000 7fa7e213a640 1 -- 172.21.15.60:0/2488592032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7dc0ae010 con 0x7fa7c4004d50 2024-04-25T10:14:05.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.547+0000 7fa7e213a640 1 -- 172.21.15.60:0/2488592032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7dc074770 con 0x7fa7c4004d50 2024-04-25T10:14:05.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.547+0000 7fa7c3fff640 1 -- 172.21.15.60:0/2488592032 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7cc014ca0 con 0x7fa7c4004d50 2024-04-25T10:14:05.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.547+0000 7fa7c3fff640 1 --2- 172.21.15.60:0/2488592032 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa7c8066f60 0x7fa7c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.547+0000 7fa7c3fff640 1 -- 172.21.15.60:0/2488592032 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5996+0+0 (secure 0 0 0) 0x7fa7cc087a60 con 0x7fa7c4004d50 2024-04-25T10:14:05.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.547+0000 7fa7daffd640 1 --2- 172.21.15.60:0/2488592032 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa7c8066f60 0x7fa7c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:05.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.547+0000 7fa7daffd640 1 --2- 172.21.15.60:0/2488592032 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa7c8066f60 0x7fa7c8069420 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc143490 tx=0x7fa7d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:05.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.551+0000 7fa7c3fff640 1 -- 172.21.15.60:0/2488592032 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7cc0589e0 con 0x7fa7c4004d50 2024-04-25T10:14:05.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:05.711+0000 7fa7e213a640 1 -- 172.21.15.60:0/2488592032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa7dc0b29e0 con 0x7fa7c4004d50 2024-04-25T10:14:06.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.723+0000 7fa7c3fff640 1 -- 172.21.15.60:0/2488592032 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v88) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa7cc0582e0 con 0x7fa7c4004d50 2024-04-25T10:14:06.727 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:14:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.723+0000 7fa7e213a640 1 -- 172.21.15.60:0/2488592032 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa7c8066f60 msgr2=0x7fa7c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.723+0000 7fa7e213a640 1 --2- 172.21.15.60:0/2488592032 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa7c8066f60 0x7fa7c8069420 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc143490 tx=0x7fa7d0008040 comp rx=0 tx=0).stop 2024-04-25T10:14:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.723+0000 7fa7e213a640 1 -- 172.21.15.60:0/2488592032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c4004d50 msgr2=0x7fa7dc13efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.723+0000 7fa7e213a640 1 --2- 172.21.15.60:0/2488592032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c4004d50 0x7fa7dc13efc0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc001b10 tx=0x7fa7cc012770 comp rx=0 tx=0).stop 2024-04-25T10:14:06.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.723+0000 7fa7e213a640 1 -- 172.21.15.60:0/2488592032 shutdown_connections 2024-04-25T10:14:06.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.723+0000 7fa7e213a640 1 --2- 172.21.15.60:0/2488592032 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa7c8066f60 0x7fa7c8069420 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.723+0000 7fa7e213a640 1 --2- 172.21.15.60:0/2488592032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7dc0aaab0 0x7fa7dc0aaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.723+0000 7fa7e213a640 1 --2- 172.21.15.60:0/2488592032 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7dc0ad5e0 0x7fa7dc15af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.723+0000 7fa7e213a640 1 --2- 172.21.15.60:0/2488592032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c4004d50 0x7fa7dc13efc0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.723+0000 7fa7e213a640 1 -- 172.21.15.60:0/2488592032 >> 172.21.15.60:0/2488592032 conn(0x7fa7dc066fa0 msgr2=0x7fa7dc0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.727+0000 7fa7e213a640 1 -- 172.21.15.60:0/2488592032 shutdown_connections 2024-04-25T10:14:06.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.727+0000 7fa7e213a640 1 -- 172.21.15.60:0/2488592032 wait complete. 2024-04-25T10:14:06.755 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:14:06.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.883+0000 7f8ed3a2b640 1 Processor -- start 2024-04-25T10:14:06.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.883+0000 7f8ed3a2b640 1 -- start start 2024-04-25T10:14:06.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ed3a2b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8ecc166950 0x7f8ecc168d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ed3a2b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ecc06bec0 con 0x7f8ecc0b0b30 2024-04-25T10:14:06.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ed3a2b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ecc06c1a0 con 0x7f8ecc099a30 2024-04-25T10:14:06.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ed3a2b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ecc15ef50 con 0x7f8ecc166950 2024-04-25T10:14:06.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ed0fc6640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8ecc0b0b30 0x7f8ecc099e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38662/0 (socket says 172.21.15.60:38662) 2024-04-25T10:14:06.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ed0fc6640 1 -- 172.21.15.60:0/3625295709 learned_addr learned my addr 172.21.15.60:0/3625295709 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:06.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ed1fc8640 1 --2- 172.21.15.60:0/3625295709 >> v2:172.21.15.100:3300/0 conn(0x7f8ecc166950 0x7f8ecc168d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ec3fff640 1 -- 172.21.15.60:0/3625295709 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189801235 0 0) 0x7f8ecc06bec0 con 0x7f8ecc0b0b30 2024-04-25T10:14:06.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ec3fff640 1 -- 172.21.15.60:0/3625295709 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8eb8003530 con 0x7f8ecc0b0b30 2024-04-25T10:14:06.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ec3fff640 1 -- 172.21.15.60:0/3625295709 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2482127209 0 0) 0x7f8eb8003530 con 0x7f8ecc0b0b30 2024-04-25T10:14:06.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ec3fff640 1 -- 172.21.15.60:0/3625295709 >> v2:172.21.15.100:3300/0 conn(0x7f8ecc166950 msgr2=0x7f8ecc168d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ec3fff640 1 --2- 172.21.15.60:0/3625295709 >> v2:172.21.15.100:3300/0 conn(0x7f8ecc166950 0x7f8ecc168d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ec3fff640 1 -- 172.21.15.60:0/3625295709 >> v1:172.21.15.100:6789/0 conn(0x7f8ecc099a30 legacy=0x7f8ecc161fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ec3fff640 1 -- 172.21.15.60:0/3625295709 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc06c480 con 0x7f8ecc0b0b30 2024-04-25T10:14:06.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ec3fff640 1 -- 172.21.15.60:0/3625295709 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f8ebc001680 con 0x7f8ecc0b0b30 2024-04-25T10:14:06.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ec3fff640 1 -- 172.21.15.60:0/3625295709 >> v1:172.21.15.60:6789/0 conn(0x7f8ecc0b0b30 legacy=0x7f8ecc099e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ec3fff640 1 --2- 172.21.15.60:0/3625295709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8eb80039c0 0x7f8eb8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ec3fff640 1 --2- 172.21.15.60:0/3625295709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8eb8004810 0x7f8eb8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ec3fff640 1 --2- 172.21.15.60:0/3625295709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8eb80051d0 0x7f8eb800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ec3fff640 1 -- 172.21.15.60:0/3625295709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb800c040 con 0x7f8eb80039c0 2024-04-25T10:14:06.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ec3fff640 1 -- 172.21.15.60:0/3625295709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb800c1f0 con 0x7f8eb8004810 2024-04-25T10:14:06.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ec3fff640 1 -- 172.21.15.60:0/3625295709 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb800c3f0 con 0x7f8eb80051d0 2024-04-25T10:14:06.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ed17c7640 1 --2- 172.21.15.60:0/3625295709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8eb80039c0 0x7f8eb8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ed0fc6640 1 --2- 172.21.15.60:0/3625295709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8eb8004810 0x7f8eb8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ed1fc8640 1 --2- 172.21.15.60:0/3625295709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8eb80051d0 0x7f8eb800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ed17c7640 1 -- 172.21.15.60:0/3625295709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8eb80051d0 msgr2=0x7f8eb800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ed17c7640 1 --2- 172.21.15.60:0/3625295709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8eb80051d0 0x7f8eb800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ed17c7640 1 -- 172.21.15.60:0/3625295709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8eb8004810 msgr2=0x7f8eb8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ed17c7640 1 --2- 172.21.15.60:0/3625295709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8eb8004810 0x7f8eb8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ed17c7640 1 -- 172.21.15.60:0/3625295709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb8003530 con 0x7f8eb80039c0 2024-04-25T10:14:06.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.887+0000 7f8ed1fc8640 1 -- 172.21.15.60:0/3625295709 reap_dead start 2024-04-25T10:14:06.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.891+0000 7f8ed17c7640 1 --2- 172.21.15.60:0/3625295709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8eb80039c0 0x7f8eb8003de0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8001780 tx=0x7f8ec801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275c6e19ad0f8b8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.891+0000 7f8ec3fff640 1 -- 172.21.15.60:0/3625295709 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80013a0 con 0x7f8eb80039c0 2024-04-25T10:14:06.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.891+0000 7f8ec3fff640 1 -- 172.21.15.60:0/3625295709 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec80052f0 con 0x7f8eb80039c0 2024-04-25T10:14:06.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.891+0000 7f8ec3fff640 1 -- 172.21.15.60:0/3625295709 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8003840 con 0x7f8eb80039c0 2024-04-25T10:14:06.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.891+0000 7f8ed3a2b640 1 -- 172.21.15.60:0/3625295709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8eb80039c0 msgr2=0x7f8eb8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.891+0000 7f8ed3a2b640 1 --2- 172.21.15.60:0/3625295709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8eb80039c0 0x7f8eb8003de0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8001780 tx=0x7f8ec801c3d0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.891+0000 7f8ed3a2b640 1 -- 172.21.15.60:0/3625295709 shutdown_connections 2024-04-25T10:14:06.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.891+0000 7f8ed3a2b640 1 --2- 172.21.15.60:0/3625295709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8eb80039c0 0x7f8eb8003de0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.891+0000 7f8ed3a2b640 1 -- 172.21.15.60:0/3625295709 >> 172.21.15.60:0/3625295709 conn(0x7f8ecc0d17d0 msgr2=0x7f8ecc0d1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.891+0000 7f8ed3a2b640 1 -- 172.21.15.60:0/3625295709 shutdown_connections 2024-04-25T10:14:06.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.891+0000 7f8ed3a2b640 1 -- 172.21.15.60:0/3625295709 wait complete. 2024-04-25T10:14:06.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.891+0000 7f8ed3a2b640 1 Processor -- start 2024-04-25T10:14:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.891+0000 7f8ed3a2b640 1 -- start start 2024-04-25T10:14:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed3a2b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8eb80039c0 0x7f8ecc0b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed3a2b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc158050 0x7f8ecc15f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed3a2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ecc165630 0x7f8ecc165ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed3a2b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc1623f0 con 0x7f8ecc158050 2024-04-25T10:14:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed3a2b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc06cf60 con 0x7f8ecc165630 2024-04-25T10:14:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed3a2b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc162570 con 0x7f8eb80039c0 2024-04-25T10:14:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed0fc6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc158050 0x7f8ecc15f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed0fc6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc158050 0x7f8ecc15f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48032/0 (socket says 172.21.15.60:48032) 2024-04-25T10:14:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed0fc6640 1 -- 172.21.15.60:0/346885759 learned_addr learned my addr 172.21.15.60:0/346885759 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed17c7640 1 --2- 172.21.15.60:0/346885759 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8eb80039c0 0x7f8ecc0b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed1fc8640 1 --2- 172.21.15.60:0/346885759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ecc165630 0x7f8ecc165ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed0fc6640 1 -- 172.21.15.60:0/346885759 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8eb80039c0 msgr2=0x7f8ecc0b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed0fc6640 1 --2- 172.21.15.60:0/346885759 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8eb80039c0 0x7f8ecc0b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed0fc6640 1 -- 172.21.15.60:0/346885759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ecc165630 msgr2=0x7f8ecc165ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed0fc6640 1 --2- 172.21.15.60:0/346885759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ecc165630 0x7f8ecc165ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed0fc6640 1 -- 172.21.15.60:0/346885759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc1f71d0 con 0x7f8ecc158050 2024-04-25T10:14:06.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed17c7640 1 --2- 172.21.15.60:0/346885759 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8eb80039c0 0x7f8ecc0b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:06.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed1fc8640 1 --2- 172.21.15.60:0/346885759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ecc165630 0x7f8ecc165ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:06.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed0fc6640 1 --2- 172.21.15.60:0/346885759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc158050 0x7f8ecc15f1c0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00e7b0 tx=0x7f8ebc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ec1ffb640 1 -- 172.21.15.60:0/346885759 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc00c6c0 con 0x7f8ecc158050 2024-04-25T10:14:06.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ec1ffb640 1 -- 172.21.15.60:0/346885759 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ebc018070 con 0x7f8ecc158050 2024-04-25T10:14:06.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed3a2b640 1 -- 172.21.15.60:0/346885759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ecc1f74b0 con 0x7f8ecc158050 2024-04-25T10:14:06.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ec1ffb640 1 -- 172.21.15.60:0/346885759 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc00caa0 con 0x7f8ecc158050 2024-04-25T10:14:06.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed3a2b640 1 -- 172.21.15.60:0/346885759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc1f7a70 con 0x7f8ecc158050 2024-04-25T10:14:06.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed3a2b640 1 -- 172.21.15.60:0/346885759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ecc074770 con 0x7f8ecc158050 2024-04-25T10:14:06.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ec1ffb640 1 -- 172.21.15.60:0/346885759 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ebc00cc40 con 0x7f8ecc158050 2024-04-25T10:14:06.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ec1ffb640 1 --2- 172.21.15.60:0/346885759 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8eb0066f60 0x7f8eb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ec1ffb640 1 -- 172.21.15.60:0/346885759 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5558+0+0 (secure 0 0 0) 0x7f8ebc084d50 con 0x7f8ecc158050 2024-04-25T10:14:06.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.895+0000 7f8ed17c7640 1 --2- 172.21.15.60:0/346885759 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8eb0066f60 0x7f8eb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.899+0000 7f8ed17c7640 1 --2- 172.21.15.60:0/346885759 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8eb0066f60 0x7f8eb0069420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8001750 tx=0x7f8ec80011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:06.899+0000 7f8ec1ffb640 1 -- 172.21.15.60:0/346885759 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ebc088030 con 0x7f8ecc158050 2024-04-25T10:14:07.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.059+0000 7f8ed3a2b640 1 -- 172.21.15.60:0/346885759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8ecc0d1550 con 0x7f8ecc158050 2024-04-25T10:14:07.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.731+0000 7f8ec1ffb640 1 -- 172.21.15.60:0/346885759 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v89) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8ebc055df0 con 0x7f8ecc158050 2024-04-25T10:14:07.735 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T10:14:07.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.735+0000 7f8ed3a2b640 1 -- 172.21.15.60:0/346885759 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8eb0066f60 msgr2=0x7f8eb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.735+0000 7f8ed3a2b640 1 --2- 172.21.15.60:0/346885759 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8eb0066f60 0x7f8eb0069420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8001750 tx=0x7f8ec80011a0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.735+0000 7f8ed3a2b640 1 -- 172.21.15.60:0/346885759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc158050 msgr2=0x7f8ecc15f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.735+0000 7f8ed3a2b640 1 --2- 172.21.15.60:0/346885759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc158050 0x7f8ecc15f1c0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00e7b0 tx=0x7f8ebc00ec70 comp rx=0 tx=0).stop 2024-04-25T10:14:07.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.739+0000 7f8ed3a2b640 1 -- 172.21.15.60:0/346885759 shutdown_connections 2024-04-25T10:14:07.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.739+0000 7f8ed3a2b640 1 --2- 172.21.15.60:0/346885759 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8eb0066f60 0x7f8eb0069420 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.739+0000 7f8ed3a2b640 1 --2- 172.21.15.60:0/346885759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ecc165630 0x7f8ecc165ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.739+0000 7f8ed3a2b640 1 --2- 172.21.15.60:0/346885759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ecc158050 0x7f8ecc15f1c0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.739+0000 7f8ed3a2b640 1 --2- 172.21.15.60:0/346885759 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8eb80039c0 0x7f8ecc0b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.739+0000 7f8ed3a2b640 1 -- 172.21.15.60:0/346885759 >> 172.21.15.60:0/346885759 conn(0x7f8ecc0d17d0 msgr2=0x7f8ecc15d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:07.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.739+0000 7f8ed3a2b640 1 -- 172.21.15.60:0/346885759 shutdown_connections 2024-04-25T10:14:07.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.739+0000 7f8ed3a2b640 1 -- 172.21.15.60:0/346885759 wait complete. 2024-04-25T10:14:07.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:14:07.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.891+0000 7fa636695640 1 Processor -- start 2024-04-25T10:14:07.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.891+0000 7fa636695640 1 -- start start 2024-04-25T10:14:07.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.891+0000 7fa636695640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa6300cefe0 0x7fa63015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.891+0000 7fa636695640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6300cca60 0x7fa6300b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.891+0000 7fa636695640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63015ce20 con 0x7fa6300b35e0 2024-04-25T10:14:07.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.891+0000 7fa636695640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa630071410 con 0x7fa6300cefe0 2024-04-25T10:14:07.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.891+0000 7fa636695640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63015c580 con 0x7fa6300cca60 2024-04-25T10:14:07.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.891+0000 7fa62ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa6300cefe0 0x7fa63015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.891+0000 7fa62ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa6300cefe0 0x7fa63015bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48046/0 (socket says 172.21.15.60:48046) 2024-04-25T10:14:07.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.891+0000 7fa62ffff640 1 -- 172.21.15.60:0/866030065 learned_addr learned my addr 172.21.15.60:0/866030065 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:07.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.891+0000 7fa62f7fe640 1 --2- 172.21.15.60:0/866030065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6300cca60 0x7fa6300b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.891+0000 7fa62ffff640 1 -- 172.21.15.60:0/866030065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6300cca60 msgr2=0x7fa6300b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.891+0000 7fa62ffff640 1 --2- 172.21.15.60:0/866030065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6300cca60 0x7fa6300b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.891+0000 7fa62ffff640 1 -- 172.21.15.60:0/866030065 >> v1:172.21.15.100:6789/0 conn(0x7fa6300b35e0 legacy=0x7fa6300718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.891+0000 7fa62ffff640 1 -- 172.21.15.60:0/866030065 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63015c860 con 0x7fa6300cefe0 2024-04-25T10:14:07.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.891+0000 7fa62f7fe640 1 --2- 172.21.15.60:0/866030065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6300cca60 0x7fa6300b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:07.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.895+0000 7fa62ffff640 1 --2- 172.21.15.60:0/866030065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6300cefe0 0x7fa63015bea0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fa624009870 tx=0x7fa62401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788bbc2259ad575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.895+0000 7fa62effd640 1 -- 172.21.15.60:0/866030065 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6240ab640 con 0x7fa6300cefe0 2024-04-25T10:14:07.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.895+0000 7fa62effd640 1 -- 172.21.15.60:0/866030065 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6240b0070 con 0x7fa6300cefe0 2024-04-25T10:14:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.895+0000 7fa62effd640 1 -- 172.21.15.60:0/866030065 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6240aba00 con 0x7fa6300cefe0 2024-04-25T10:14:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.895+0000 7fa636695640 1 -- 172.21.15.60:0/866030065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6300cefe0 msgr2=0x7fa63015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.895+0000 7fa636695640 1 --2- 172.21.15.60:0/866030065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6300cefe0 0x7fa63015bea0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fa624009870 tx=0x7fa62401c460 comp rx=0 tx=0).stop 2024-04-25T10:14:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.895+0000 7fa636695640 1 -- 172.21.15.60:0/866030065 shutdown_connections 2024-04-25T10:14:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.895+0000 7fa636695640 1 --2- 172.21.15.60:0/866030065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6300cca60 0x7fa6300b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.895+0000 7fa636695640 1 --2- 172.21.15.60:0/866030065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6300cefe0 0x7fa63015bea0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.895+0000 7fa636695640 1 -- 172.21.15.60:0/866030065 >> 172.21.15.60:0/866030065 conn(0x7fa6300670b0 msgr2=0x7fa63006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.895+0000 7fa636695640 1 -- 172.21.15.60:0/866030065 shutdown_connections 2024-04-25T10:14:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.895+0000 7fa636695640 1 -- 172.21.15.60:0/866030065 wait complete. 2024-04-25T10:14:07.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa636695640 1 Processor -- start 2024-04-25T10:14:07.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa636695640 1 -- start start 2024-04-25T10:14:07.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa636695640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6300b35e0 0x7fa63014d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa636695640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6300cca60 0x7fa6300b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa636695640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6300cefe0 0x7fa6300b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa636695640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa630168ba0 con 0x7fa6300cca60 2024-04-25T10:14:07.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa636695640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa630168a20 con 0x7fa6300cefe0 2024-04-25T10:14:07.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa636695640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa630168d20 con 0x7fa6300b35e0 2024-04-25T10:14:07.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa62f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6300cca60 0x7fa6300b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa62f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6300cca60 0x7fa6300b1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48048/0 (socket says 172.21.15.60:48048) 2024-04-25T10:14:07.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa62f7fe640 1 -- 172.21.15.60:0/1538534733 learned_addr learned my addr 172.21.15.60:0/1538534733 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:07.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa62ffff640 1 --2- 172.21.15.60:0/1538534733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6300b35e0 0x7fa63014d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa634c32640 1 --2- 172.21.15.60:0/1538534733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6300cefe0 0x7fa6300b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa62f7fe640 1 -- 172.21.15.60:0/1538534733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6300b35e0 msgr2=0x7fa63014d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa62f7fe640 1 --2- 172.21.15.60:0/1538534733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6300b35e0 0x7fa63014d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa62f7fe640 1 -- 172.21.15.60:0/1538534733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6300cefe0 msgr2=0x7fa6300b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa62f7fe640 1 --2- 172.21.15.60:0/1538534733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6300cefe0 0x7fa6300b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa62f7fe640 1 -- 172.21.15.60:0/1538534733 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6300af320 con 0x7fa6300cca60 2024-04-25T10:14:07.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa634c32640 1 --2- 172.21.15.60:0/1538534733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6300cefe0 0x7fa6300b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:07.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa62f7fe640 1 --2- 172.21.15.60:0/1538534733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6300cca60 0x7fa6300b1610 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fa62000c7c0 tx=0x7fa62000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa62cff9640 1 -- 172.21.15.60:0/1538534733 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620007a30 con 0x7fa6300cca60 2024-04-25T10:14:07.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa62cff9640 1 -- 172.21.15.60:0/1538534733 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa620007bd0 con 0x7fa6300cca60 2024-04-25T10:14:07.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa62cff9640 1 -- 172.21.15.60:0/1538534733 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6200113f0 con 0x7fa6300cca60 2024-04-25T10:14:07.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa636695640 1 -- 172.21.15.60:0/1538534733 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6300af600 con 0x7fa6300cca60 2024-04-25T10:14:07.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa636695640 1 -- 172.21.15.60:0/1538534733 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6300afbc0 con 0x7fa6300cca60 2024-04-25T10:14:07.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa636695640 1 -- 172.21.15.60:0/1538534733 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa630074770 con 0x7fa6300cca60 2024-04-25T10:14:07.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.899+0000 7fa62cff9640 1 -- 172.21.15.60:0/1538534733 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa620011590 con 0x7fa6300cca60 2024-04-25T10:14:07.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.903+0000 7fa62cff9640 1 --2- 172.21.15.60:0/1538534733 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa61c066f60 0x7fa61c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.903+0000 7fa62cff9640 1 -- 172.21.15.60:0/1538534733 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fa620084f50 con 0x7fa6300cca60 2024-04-25T10:14:07.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.903+0000 7fa62ffff640 1 --2- 172.21.15.60:0/1538534733 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa61c066f60 0x7fa61c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.903+0000 7fa62ffff640 1 --2- 172.21.15.60:0/1538534733 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa61c066f60 0x7fa61c069420 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa6240af000 tx=0x7fa6240a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:07.907+0000 7fa62cff9640 1 -- 172.21.15.60:0/1538534733 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa620015020 con 0x7fa6300cca60 2024-04-25T10:14:08.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.071+0000 7fa636695640 1 -- 172.21.15.60:0/1538534733 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa63015d240 con 0x7fa6300cca60 2024-04-25T10:14:08.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.075+0000 7fa62cff9640 1 -- 172.21.15.60:0/1538534733 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa620056240 con 0x7fa6300cca60 2024-04-25T10:14:08.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.079+0000 7fa636695640 1 -- 172.21.15.60:0/1538534733 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa61c066f60 msgr2=0x7fa61c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.079+0000 7fa636695640 1 --2- 172.21.15.60:0/1538534733 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa61c066f60 0x7fa61c069420 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa6240af000 tx=0x7fa6240a7000 comp rx=0 tx=0).stop 2024-04-25T10:14:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.079+0000 7fa636695640 1 -- 172.21.15.60:0/1538534733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6300cca60 msgr2=0x7fa6300b1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.079+0000 7fa636695640 1 --2- 172.21.15.60:0/1538534733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6300cca60 0x7fa6300b1610 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fa62000c7c0 tx=0x7fa62000cc80 comp rx=0 tx=0).stop 2024-04-25T10:14:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.079+0000 7fa636695640 1 -- 172.21.15.60:0/1538534733 shutdown_connections 2024-04-25T10:14:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.079+0000 7fa636695640 1 --2- 172.21.15.60:0/1538534733 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa61c066f60 0x7fa61c069420 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.079+0000 7fa636695640 1 --2- 172.21.15.60:0/1538534733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6300cefe0 0x7fa6300b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.079+0000 7fa636695640 1 --2- 172.21.15.60:0/1538534733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6300cca60 0x7fa6300b1610 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.079+0000 7fa636695640 1 --2- 172.21.15.60:0/1538534733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6300b35e0 0x7fa63014d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.079+0000 7fa636695640 1 -- 172.21.15.60:0/1538534733 >> 172.21.15.60:0/1538534733 conn(0x7fa6300670b0 msgr2=0x7fa63015ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.079+0000 7fa636695640 1 -- 172.21.15.60:0/1538534733 shutdown_connections 2024-04-25T10:14:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.079+0000 7fa636695640 1 -- 172.21.15.60:0/1538534733 wait complete. 2024-04-25T10:14:08.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-25T10:14:08.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762ff85640 1 Processor -- start 2024-04-25T10:14:08.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762ff85640 1 -- start start 2024-04-25T10:14:08.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762ff85640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f762806ec30 0x7f76280cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762ff85640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76280b37e0 0x7f76280b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762ff85640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f762806f130 con 0x7f762815a7a0 2024-04-25T10:14:08.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762ff85640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76280cdaa0 con 0x7f762806ec30 2024-04-25T10:14:08.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762ff85640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762806d2f0 con 0x7f76280b37e0 2024-04-25T10:14:08.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762dd21640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f762815a7a0 0x7f7628071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38676/0 (socket says 172.21.15.60:38676) 2024-04-25T10:14:08.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762dd21640 1 -- 172.21.15.60:0/2892934981 learned_addr learned my addr 172.21.15.60:0/2892934981 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:08.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762e522640 1 --2- 172.21.15.60:0/2892934981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76280b37e0 0x7f76280b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762cd1f640 1 -- 172.21.15.60:0/2892934981 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2049149814 0 0) 0x7f762806f130 con 0x7f762815a7a0 2024-04-25T10:14:08.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762cd1f640 1 -- 172.21.15.60:0/2892934981 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7614003530 con 0x7f762815a7a0 2024-04-25T10:14:08.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762d520640 1 --2- 172.21.15.60:0/2892934981 >> v2:172.21.15.100:3300/0 conn(0x7f762806ec30 0x7f76280cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762cd1f640 1 -- 172.21.15.60:0/2892934981 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3169816085 0 0) 0x7f7614003530 con 0x7f762815a7a0 2024-04-25T10:14:08.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762cd1f640 1 -- 172.21.15.60:0/2892934981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76280b37e0 msgr2=0x7f76280b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:08.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762cd1f640 1 --2- 172.21.15.60:0/2892934981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76280b37e0 0x7f76280b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762cd1f640 1 -- 172.21.15.60:0/2892934981 >> v2:172.21.15.100:3300/0 conn(0x7f762806ec30 msgr2=0x7f76280cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762cd1f640 1 --2- 172.21.15.60:0/2892934981 >> v2:172.21.15.100:3300/0 conn(0x7f762806ec30 0x7f76280cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762cd1f640 1 -- 172.21.15.60:0/2892934981 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762806f410 con 0x7f762815a7a0 2024-04-25T10:14:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762cd1f640 1 -- 172.21.15.60:0/2892934981 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f7624001680 con 0x7f762815a7a0 2024-04-25T10:14:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762cd1f640 1 -- 172.21.15.60:0/2892934981 >> v1:172.21.15.60:6789/0 conn(0x7f762815a7a0 legacy=0x7f7628071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762cd1f640 1 --2- 172.21.15.60:0/2892934981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76140039c0 0x7f7614003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762cd1f640 1 --2- 172.21.15.60:0/2892934981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76280b37e0 0x7f7614004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762cd1f640 1 --2- 172.21.15.60:0/2892934981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7614004d50 0x7f76140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762cd1f640 1 -- 172.21.15.60:0/2892934981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762806d2f0 con 0x7f76140039c0 2024-04-25T10:14:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762cd1f640 1 -- 172.21.15.60:0/2892934981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761400b9d0 con 0x7f7614004d50 2024-04-25T10:14:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762cd1f640 1 -- 172.21.15.60:0/2892934981 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761400bb50 con 0x7f76280b37e0 2024-04-25T10:14:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762dd21640 1 --2- 172.21.15.60:0/2892934981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76140039c0 0x7f7614003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762d520640 1 --2- 172.21.15.60:0/2892934981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76280b37e0 0x7f7614004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762e522640 1 --2- 172.21.15.60:0/2892934981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7614004d50 0x7f76140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762dd21640 1 -- 172.21.15.60:0/2892934981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76280b37e0 msgr2=0x7f7614004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:08.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762dd21640 1 --2- 172.21.15.60:0/2892934981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76280b37e0 0x7f7614004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762dd21640 1 -- 172.21.15.60:0/2892934981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7614004d50 msgr2=0x7f76140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:08.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762dd21640 1 --2- 172.21.15.60:0/2892934981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7614004d50 0x7f76140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762dd21640 1 -- 172.21.15.60:0/2892934981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7614003530 con 0x7f76140039c0 2024-04-25T10:14:08.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762e522640 1 --2- 172.21.15.60:0/2892934981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7614004d50 0x7f76140051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:08.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.223+0000 7f762d520640 1 --2- 172.21.15.60:0/2892934981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76280b37e0 0x7f7614004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:08.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.227+0000 7f762dd21640 1 --2- 172.21.15.60:0/2892934981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76140039c0 0x7f7614003de0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f762400c240 tx=0x7f762401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe00af8ac60ab5dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:08.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.227+0000 7f762cd1f640 1 -- 172.21.15.60:0/2892934981 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624007d20 con 0x7f76140039c0 2024-04-25T10:14:08.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.227+0000 7f762cd1f640 1 -- 172.21.15.60:0/2892934981 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76240a9070 con 0x7f76140039c0 2024-04-25T10:14:08.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.227+0000 7f762cd1f640 1 -- 172.21.15.60:0/2892934981 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624005d40 con 0x7f76140039c0 2024-04-25T10:14:08.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.227+0000 7f762ff85640 1 -- 172.21.15.60:0/2892934981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76140039c0 msgr2=0x7f7614003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:08.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.227+0000 7f762ff85640 1 --2- 172.21.15.60:0/2892934981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76140039c0 0x7f7614003de0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f762400c240 tx=0x7f762401cc80 comp rx=0 tx=0).stop 2024-04-25T10:14:08.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.227+0000 7f762e522640 1 -- 172.21.15.60:0/2892934981 reap_dead start 2024-04-25T10:14:08.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.227+0000 7f762ff85640 1 -- 172.21.15.60:0/2892934981 shutdown_connections 2024-04-25T10:14:08.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.227+0000 7f762ff85640 1 -- 172.21.15.60:0/2892934981 >> 172.21.15.60:0/2892934981 conn(0x7f76280b12d0 msgr2=0x7f76280b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:08.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.227+0000 7f762ff85640 1 -- 172.21.15.60:0/2892934981 shutdown_connections 2024-04-25T10:14:08.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.227+0000 7f762ff85640 1 -- 172.21.15.60:0/2892934981 wait complete. 2024-04-25T10:14:08.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.227+0000 7f762ff85640 1 Processor -- start 2024-04-25T10:14:08.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.227+0000 7f762ff85640 1 -- start start 2024-04-25T10:14:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762ff85640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76140039c0 0x7f762814c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762ff85640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f762814d460 0x7f762813cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762ff85640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f762814ee30 0x7f762814f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762ff85640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762806e6b0 con 0x7f762814d460 2024-04-25T10:14:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762ff85640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762806e530 con 0x7f762814ee30 2024-04-25T10:14:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762ff85640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762806e830 con 0x7f76140039c0 2024-04-25T10:14:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762d520640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f762814d460 0x7f762813cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762d520640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f762814d460 0x7f762813cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48066/0 (socket says 172.21.15.60:48066) 2024-04-25T10:14:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762d520640 1 -- 172.21.15.60:0/302733694 learned_addr learned my addr 172.21.15.60:0/302733694 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762dd21640 1 --2- 172.21.15.60:0/302733694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76140039c0 0x7f762814c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762e522640 1 --2- 172.21.15.60:0/302733694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f762814ee30 0x7f762814f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762d520640 1 -- 172.21.15.60:0/302733694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76140039c0 msgr2=0x7f762814c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762d520640 1 --2- 172.21.15.60:0/302733694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76140039c0 0x7f762814c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762d520640 1 -- 172.21.15.60:0/302733694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f762814ee30 msgr2=0x7f762814f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762d520640 1 --2- 172.21.15.60:0/302733694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f762814ee30 0x7f762814f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762d520640 1 -- 172.21.15.60:0/302733694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7628158c60 con 0x7f762814d460 2024-04-25T10:14:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762e522640 1 --2- 172.21.15.60:0/302733694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f762814ee30 0x7f762814f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762dd21640 1 --2- 172.21.15.60:0/302733694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76140039c0 0x7f762814c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762d520640 1 --2- 172.21.15.60:0/302733694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f762814d460 0x7f762813cdc0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f76180116d0 tx=0x7f7618011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f761e7fc640 1 -- 172.21.15.60:0/302733694 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618004220 con 0x7f762814d460 2024-04-25T10:14:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f761e7fc640 1 -- 172.21.15.60:0/302733694 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76180043c0 con 0x7f762814d460 2024-04-25T10:14:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762ff85640 1 -- 172.21.15.60:0/302733694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7628158f40 con 0x7f762814d460 2024-04-25T10:14:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f761e7fc640 1 -- 172.21.15.60:0/302733694 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761800f9c0 con 0x7f762814d460 2024-04-25T10:14:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762ff85640 1 -- 172.21.15.60:0/302733694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7628159500 con 0x7f762814d460 2024-04-25T10:14:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f762ff85640 1 -- 172.21.15.60:0/302733694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7628074770 con 0x7f762814d460 2024-04-25T10:14:08.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.231+0000 7f761e7fc640 1 -- 172.21.15.60:0/302733694 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f761800fb60 con 0x7f762814d460 2024-04-25T10:14:08.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.235+0000 7f761e7fc640 1 --2- 172.21.15.60:0/302733694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f760c066f60 0x7f760c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.235+0000 7f761e7fc640 1 -- 172.21.15.60:0/302733694 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f7618085a20 con 0x7f762814d460 2024-04-25T10:14:08.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.235+0000 7f762dd21640 1 --2- 172.21.15.60:0/302733694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f760c066f60 0x7f760c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.235+0000 7f762dd21640 1 --2- 172.21.15.60:0/302733694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f760c066f60 0x7f760c069420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f76240ab000 tx=0x7f76240a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:08.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.235+0000 7f761e7fc640 1 -- 172.21.15.60:0/302733694 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7618019030 con 0x7f762814d460 2024-04-25T10:14:08.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.455+0000 7f762ff85640 1 -- 172.21.15.60:0/302733694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7f7628153b20 con 0x7f762814d460 2024-04-25T10:14:08.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.731+0000 7f761e7fc640 1 -- 172.21.15.60:0/302733694 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v405) v1 ==== 167+0+0 (secure 0 0 0) 0x7f7618056c60 con 0x7f762814d460 2024-04-25T10:14:08.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.735+0000 7f762ff85640 1 -- 172.21.15.60:0/302733694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f760c066f60 msgr2=0x7f760c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:08.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.735+0000 7f762ff85640 1 --2- 172.21.15.60:0/302733694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f760c066f60 0x7f760c069420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f76240ab000 tx=0x7f76240a7070 comp rx=0 tx=0).stop 2024-04-25T10:14:08.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.735+0000 7f762ff85640 1 -- 172.21.15.60:0/302733694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f762814d460 msgr2=0x7f762813cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:08.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.735+0000 7f762ff85640 1 --2- 172.21.15.60:0/302733694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f762814d460 0x7f762813cdc0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f76180116d0 tx=0x7f7618011b90 comp rx=0 tx=0).stop 2024-04-25T10:14:08.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.735+0000 7f762ff85640 1 -- 172.21.15.60:0/302733694 shutdown_connections 2024-04-25T10:14:08.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.735+0000 7f762ff85640 1 --2- 172.21.15.60:0/302733694 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f760c066f60 0x7f760c069420 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.735+0000 7f762ff85640 1 --2- 172.21.15.60:0/302733694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f762814ee30 0x7f762814f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.735+0000 7f762ff85640 1 --2- 172.21.15.60:0/302733694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f762814d460 0x7f762813cdc0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.735+0000 7f762ff85640 1 --2- 172.21.15.60:0/302733694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76140039c0 0x7f762814c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.735+0000 7f762ff85640 1 -- 172.21.15.60:0/302733694 >> 172.21.15.60:0/302733694 conn(0x7f76280b12d0 msgr2=0x7f7628152e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:08.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.735+0000 7f762ff85640 1 -- 172.21.15.60:0/302733694 shutdown_connections 2024-04-25T10:14:08.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.735+0000 7f762ff85640 1 -- 172.21.15.60:0/302733694 wait complete. 2024-04-25T10:14:08.762 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T10:14:08.767 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:14:08.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:08.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.883+0000 7f15f43a8640 1 Processor -- start 2024-04-25T10:14:08.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.883+0000 7f15f43a8640 1 -- start start 2024-04-25T10:14:08.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.887+0000 7f15f43a8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f15ec160f80 0x7f15ec0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.887+0000 7f15f43a8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f15ec161d80 0x7f15ec0b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.887+0000 7f15f43a8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ec0b5b60 con 0x7f15ec06b620 2024-04-25T10:14:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.887+0000 7f15f43a8640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ec0b3bf0 con 0x7f15ec160f80 2024-04-25T10:14:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.887+0000 7f15f43a8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ec0b55a0 con 0x7f15ec161d80 2024-04-25T10:14:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.887+0000 7f15f2144640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f15ec160f80 0x7f15ec0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.887+0000 7f15f2144640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f15ec160f80 0x7f15ec0b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48074/0 (socket says 172.21.15.60:48074) 2024-04-25T10:14:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.887+0000 7f15f2144640 1 -- 172.21.15.60:0/3399459401 learned_addr learned my addr 172.21.15.60:0/3399459401 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.887+0000 7f15f2945640 1 --2- 172.21.15.60:0/3399459401 >> v2:172.21.15.100:3300/0 conn(0x7f15ec161d80 0x7f15ec0b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.887+0000 7f15f2144640 1 -- 172.21.15.60:0/3399459401 >> v2:172.21.15.100:3300/0 conn(0x7f15ec161d80 msgr2=0x7f15ec0b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.887+0000 7f15f2144640 1 --2- 172.21.15.60:0/3399459401 >> v2:172.21.15.100:3300/0 conn(0x7f15ec161d80 0x7f15ec0b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.887+0000 7f15f2144640 1 -- 172.21.15.60:0/3399459401 >> v1:172.21.15.60:6789/0 conn(0x7f15ec06b620 legacy=0x7f15ec161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.887+0000 7f15f1142640 1 -- 172.21.15.60:0/3399459401 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936644229 0 0) 0x7f15ec0b5b60 con 0x7f15ec06b620 2024-04-25T10:14:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.887+0000 7f15f2144640 1 -- 172.21.15.60:0/3399459401 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec0b5880 con 0x7f15ec160f80 2024-04-25T10:14:08.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f2144640 1 --2- 172.21.15.60:0/3399459401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec160f80 0x7f15ec0b3610 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f15ec15bbe0 tx=0x7f15e001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b729fd796ee1a3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:08.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f1142640 1 -- 172.21.15.60:0/3399459401 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e00b1070 con 0x7f15ec160f80 2024-04-25T10:14:08.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f1142640 1 -- 172.21.15.60:0/3399459401 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e00ac600 con 0x7f15ec160f80 2024-04-25T10:14:08.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f1142640 1 -- 172.21.15.60:0/3399459401 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e00ac9d0 con 0x7f15ec160f80 2024-04-25T10:14:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f43a8640 1 -- 172.21.15.60:0/3399459401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec160f80 msgr2=0x7f15ec0b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f43a8640 1 --2- 172.21.15.60:0/3399459401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec160f80 0x7f15ec0b3610 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f15ec15bbe0 tx=0x7f15e001c490 comp rx=0 tx=0).stop 2024-04-25T10:14:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f43a8640 1 -- 172.21.15.60:0/3399459401 shutdown_connections 2024-04-25T10:14:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f43a8640 1 --2- 172.21.15.60:0/3399459401 >> v2:172.21.15.100:3300/0 conn(0x7f15ec161d80 0x7f15ec0b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f43a8640 1 --2- 172.21.15.60:0/3399459401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec160f80 0x7f15ec0b3610 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f43a8640 1 -- 172.21.15.60:0/3399459401 >> 172.21.15.60:0/3399459401 conn(0x7f15ec0728e0 msgr2=0x7f15ec099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f43a8640 1 -- 172.21.15.60:0/3399459401 shutdown_connections 2024-04-25T10:14:08.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f43a8640 1 -- 172.21.15.60:0/3399459401 wait complete. 2024-04-25T10:14:08.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f43a8640 1 Processor -- start 2024-04-25T10:14:08.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f43a8640 1 -- start start 2024-04-25T10:14:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f43a8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec06b620 0x7f15ec15cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f43a8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec160f80 0x7f15ec0b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f43a8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15ec161d80 0x7f15ec146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f43a8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec15f680 con 0x7f15ec06b620 2024-04-25T10:14:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f43a8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec15f500 con 0x7f15ec160f80 2024-04-25T10:14:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f43a8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec15f800 con 0x7f15ec161d80 2024-04-25T10:14:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f2144640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec06b620 0x7f15ec15cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f2144640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec06b620 0x7f15ec15cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38282/0 (socket says 172.21.15.60:38282) 2024-04-25T10:14:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f2144640 1 -- 172.21.15.60:0/1486708098 learned_addr learned my addr 172.21.15.60:0/1486708098 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f1943640 1 --2- 172.21.15.60:0/1486708098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec160f80 0x7f15ec0b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.891+0000 7f15f2945640 1 --2- 172.21.15.60:0/1486708098 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15ec161d80 0x7f15ec146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15f2144640 1 -- 172.21.15.60:0/1486708098 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15ec161d80 msgr2=0x7f15ec146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:08.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15f2144640 1 --2- 172.21.15.60:0/1486708098 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15ec161d80 0x7f15ec146610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15f2144640 1 -- 172.21.15.60:0/1486708098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec160f80 msgr2=0x7f15ec0b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:08.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15f2144640 1 --2- 172.21.15.60:0/1486708098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec160f80 0x7f15ec0b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15f2144640 1 -- 172.21.15.60:0/1486708098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec0c9990 con 0x7f15ec06b620 2024-04-25T10:14:08.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15f2144640 1 --2- 172.21.15.60:0/1486708098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec06b620 0x7f15ec15cb40 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f15e00190a0 tx=0x7f15e001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:08.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15deffd640 1 -- 172.21.15.60:0/1486708098 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e00b1040 con 0x7f15ec06b620 2024-04-25T10:14:08.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15deffd640 1 -- 172.21.15.60:0/1486708098 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e00b7450 con 0x7f15ec06b620 2024-04-25T10:14:08.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15deffd640 1 -- 172.21.15.60:0/1486708098 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e00b7800 con 0x7f15ec06b620 2024-04-25T10:14:08.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15f43a8640 1 -- 172.21.15.60:0/1486708098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ec0c9c70 con 0x7f15ec06b620 2024-04-25T10:14:08.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15f43a8640 1 -- 172.21.15.60:0/1486708098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ec0ca230 con 0x7f15ec06b620 2024-04-25T10:14:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15f43a8640 1 -- 172.21.15.60:0/1486708098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ec06ccb0 con 0x7f15ec06b620 2024-04-25T10:14:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15deffd640 1 -- 172.21.15.60:0/1486708098 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f15e00be030 con 0x7f15ec06b620 2024-04-25T10:14:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15deffd640 1 --2- 172.21.15.60:0/1486708098 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f15d0066f60 0x7f15d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15deffd640 1 -- 172.21.15.60:0/1486708098 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f15e011e7e0 con 0x7f15ec06b620 2024-04-25T10:14:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15f1943640 1 --2- 172.21.15.60:0/1486708098 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f15d0066f60 0x7f15d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.895+0000 7f15f1943640 1 --2- 172.21.15.60:0/1486708098 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f15d0066f60 0x7f15d0069420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f15ec1f9e20 tx=0x7f15e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:08.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:08.899+0000 7f15deffd640 1 -- 172.21.15.60:0/1486708098 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f15e00c5020 con 0x7f15ec06b620 2024-04-25T10:14:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.063+0000 7f15f43a8640 1 -- 172.21.15.60:0/1486708098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15ec074770 con 0x7f15ec06b620 2024-04-25T10:14:09.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.063+0000 7f15deffd640 1 -- 172.21.15.60:0/1486708098 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+13293 (secure 0 0 0) 0x7f15e0016110 con 0x7f15ec06b620 2024-04-25T10:14:09.068 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:09.068 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:14:07.724289+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/3807131024":"2024-04-26T10:14:04.330602+0000","172.21.15.60:6833/3807131024":"2024-04-26T10:14:04.330602+0000","172.21.15.60:6835/2817879567":"2024-04-26T10:13:19.192767+0000","172.21.15.60:6833/2817879567":"2024-04-26T10:13:19.192767+0000","172.21.15.60:6835/2455562778":"2024-04-26T10:11:58.136789+0000","172.21.15.60:6833/2455562778":"2024-04-26T10:11:58.136789+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:09.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.067+0000 7f15f43a8640 1 -- 172.21.15.60:0/1486708098 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f15d0066f60 msgr2=0x7f15d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.067+0000 7f15f43a8640 1 --2- 172.21.15.60:0/1486708098 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f15d0066f60 0x7f15d0069420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f15ec1f9e20 tx=0x7f15e8009040 comp rx=0 tx=0).stop 2024-04-25T10:14:09.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.067+0000 7f15f43a8640 1 -- 172.21.15.60:0/1486708098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec06b620 msgr2=0x7f15ec15cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.067+0000 7f15f43a8640 1 --2- 172.21.15.60:0/1486708098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec06b620 0x7f15ec15cb40 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f15e00190a0 tx=0x7f15e001acd0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.067+0000 7f15f43a8640 1 -- 172.21.15.60:0/1486708098 shutdown_connections 2024-04-25T10:14:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.067+0000 7f15f43a8640 1 --2- 172.21.15.60:0/1486708098 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f15d0066f60 0x7f15d0069420 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.067+0000 7f15f43a8640 1 --2- 172.21.15.60:0/1486708098 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15ec161d80 0x7f15ec146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.067+0000 7f15f43a8640 1 --2- 172.21.15.60:0/1486708098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec160f80 0x7f15ec0b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.067+0000 7f15f43a8640 1 --2- 172.21.15.60:0/1486708098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15ec06b620 0x7f15ec15cb40 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.067+0000 7f15f43a8640 1 -- 172.21.15.60:0/1486708098 >> 172.21.15.60:0/1486708098 conn(0x7f15ec0728e0 msgr2=0x7f15ec15de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.067+0000 7f15f43a8640 1 -- 172.21.15.60:0/1486708098 shutdown_connections 2024-04-25T10:14:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.067+0000 7f15f43a8640 1 -- 172.21.15.60:0/1486708098 wait complete. 2024-04-25T10:14:09.088 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:09.088 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:09.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.211+0000 7f9db3454640 1 Processor -- start 2024-04-25T10:14:09.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.211+0000 7f9db3454640 1 -- start start 2024-04-25T10:14:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.211+0000 7f9db3454640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9dac0b3f60 0x7f9dac15a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.211+0000 7f9db3454640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9dac071250 0x7f9dac0716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.211+0000 7f9db3454640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dac0b3510 con 0x7f9dac06f030 2024-04-25T10:14:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.211+0000 7f9db3454640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dac15ad30 con 0x7f9dac071250 2024-04-25T10:14:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.211+0000 7f9db3454640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dac15b010 con 0x7f9dac0b3f60 2024-04-25T10:14:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.211+0000 7f9db19f1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9dac071250 0x7f9dac0716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.211+0000 7f9db19f1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9dac071250 0x7f9dac0716e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38294/0 (socket says 172.21.15.60:38294) 2024-04-25T10:14:09.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.211+0000 7f9db19f1640 1 -- 172.21.15.60:0/3378682071 learned_addr learned my addr 172.21.15.60:0/3378682071 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:09.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.211+0000 7f9db11f0640 1 --2- 172.21.15.60:0/3378682071 >> v2:172.21.15.100:3300/0 conn(0x7f9dac0b3f60 0x7f9dac15a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.211+0000 7f9db19f1640 1 -- 172.21.15.60:0/3378682071 >> v2:172.21.15.100:3300/0 conn(0x7f9dac0b3f60 msgr2=0x7f9dac15a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:09.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.211+0000 7f9db19f1640 1 --2- 172.21.15.60:0/3378682071 >> v2:172.21.15.100:3300/0 conn(0x7f9dac0b3f60 0x7f9dac15a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.211+0000 7f9db19f1640 1 -- 172.21.15.60:0/3378682071 >> v1:172.21.15.100:6789/0 conn(0x7f9dac06f030 legacy=0x7f9dac0b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:09.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.211+0000 7f9db19f1640 1 -- 172.21.15.60:0/3378682071 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac15b2f0 con 0x7f9dac071250 2024-04-25T10:14:09.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db19f1640 1 --2- 172.21.15.60:0/3378682071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac071250 0x7f9dac0716e0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0b36e0 tx=0x7f9da401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d4bc3429cfbd66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:09.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9da3fff640 1 -- 172.21.15.60:0/3378682071 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da40a55e0 con 0x7f9dac071250 2024-04-25T10:14:09.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9da3fff640 1 -- 172.21.15.60:0/3378682071 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da40a5780 con 0x7f9dac071250 2024-04-25T10:14:09.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9da3fff640 1 -- 172.21.15.60:0/3378682071 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da40a5b50 con 0x7f9dac071250 2024-04-25T10:14:09.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db3454640 1 -- 172.21.15.60:0/3378682071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac071250 msgr2=0x7f9dac0716e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db3454640 1 --2- 172.21.15.60:0/3378682071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac071250 0x7f9dac0716e0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0b36e0 tx=0x7f9da401c920 comp rx=0 tx=0).stop 2024-04-25T10:14:09.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db3454640 1 -- 172.21.15.60:0/3378682071 shutdown_connections 2024-04-25T10:14:09.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db3454640 1 --2- 172.21.15.60:0/3378682071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac071250 0x7f9dac0716e0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db3454640 1 --2- 172.21.15.60:0/3378682071 >> v2:172.21.15.100:3300/0 conn(0x7f9dac0b3f60 0x7f9dac15a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db3454640 1 -- 172.21.15.60:0/3378682071 >> 172.21.15.60:0/3378682071 conn(0x7f9dac0670b0 msgr2=0x7f9dac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:09.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db3454640 1 -- 172.21.15.60:0/3378682071 shutdown_connections 2024-04-25T10:14:09.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db3454640 1 -- 172.21.15.60:0/3378682071 wait complete. 2024-04-25T10:14:09.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db3454640 1 Processor -- start 2024-04-25T10:14:09.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db3454640 1 -- start start 2024-04-25T10:14:09.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db3454640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dac06f030 0x7f9dac0cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db3454640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac071250 0x7f9dac0ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db3454640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dac0b3f60 0x7f9dac1fa130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db3454640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac166ba0 con 0x7f9dac071250 2024-04-25T10:14:09.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db3454640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac166a20 con 0x7f9dac06f030 2024-04-25T10:14:09.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db3454640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac166d20 con 0x7f9dac0b3f60 2024-04-25T10:14:09.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db09ef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac071250 0x7f9dac0ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db09ef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac071250 0x7f9dac0ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38296/0 (socket says 172.21.15.60:38296) 2024-04-25T10:14:09.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db09ef640 1 -- 172.21.15.60:0/1257826626 learned_addr learned my addr 172.21.15.60:0/1257826626 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:09.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db19f1640 1 --2- 172.21.15.60:0/1257826626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dac0b3f60 0x7f9dac1fa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db11f0640 1 --2- 172.21.15.60:0/1257826626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dac06f030 0x7f9dac0cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db09ef640 1 -- 172.21.15.60:0/1257826626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dac0b3f60 msgr2=0x7f9dac1fa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db09ef640 1 --2- 172.21.15.60:0/1257826626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dac0b3f60 0x7f9dac1fa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db09ef640 1 -- 172.21.15.60:0/1257826626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dac06f030 msgr2=0x7f9dac0cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db09ef640 1 --2- 172.21.15.60:0/1257826626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dac06f030 0x7f9dac0cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.215+0000 7f9db09ef640 1 -- 172.21.15.60:0/1257826626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac1429e0 con 0x7f9dac071250 2024-04-25T10:14:09.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.219+0000 7f9db11f0640 1 --2- 172.21.15.60:0/1257826626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dac06f030 0x7f9dac0cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:09.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.219+0000 7f9db09ef640 1 --2- 172.21.15.60:0/1257826626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac071250 0x7f9dac0ce270 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00b410 tx=0x7f9d9c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:09.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.219+0000 7f9da1ffb640 1 -- 172.21.15.60:0/1257826626 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c015030 con 0x7f9dac071250 2024-04-25T10:14:09.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.219+0000 7f9da1ffb640 1 -- 172.21.15.60:0/1257826626 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9c004020 con 0x7f9dac071250 2024-04-25T10:14:09.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.219+0000 7f9db3454640 1 -- 172.21.15.60:0/1257826626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dac13f9f0 con 0x7f9dac071250 2024-04-25T10:14:09.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.219+0000 7f9da1ffb640 1 -- 172.21.15.60:0/1257826626 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c004450 con 0x7f9dac071250 2024-04-25T10:14:09.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.219+0000 7f9db3454640 1 -- 172.21.15.60:0/1257826626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dac141f00 con 0x7f9dac071250 2024-04-25T10:14:09.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.219+0000 7f9db3454640 1 -- 172.21.15.60:0/1257826626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dac074770 con 0x7f9dac071250 2024-04-25T10:14:09.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.219+0000 7f9da1ffb640 1 -- 172.21.15.60:0/1257826626 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d9c018070 con 0x7f9dac071250 2024-04-25T10:14:09.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.219+0000 7f9da1ffb640 1 --2- 172.21.15.60:0/1257826626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d90066f60 0x7f9d90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.219+0000 7f9da1ffb640 1 -- 172.21.15.60:0/1257826626 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f9d9c084920 con 0x7f9dac071250 2024-04-25T10:14:09.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.219+0000 7f9db11f0640 1 --2- 172.21.15.60:0/1257826626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d90066f60 0x7f9d90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.219+0000 7f9db11f0640 1 --2- 172.21.15.60:0/1257826626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d90066f60 0x7f9d90069420 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0723f0 tx=0x7f9da8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:09.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.223+0000 7f9da1ffb640 1 -- 172.21.15.60:0/1257826626 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d9c055b80 con 0x7f9dac071250 2024-04-25T10:14:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.375+0000 7f9db3454640 1 -- 172.21.15.60:0/1257826626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9dac157160 con 0x7f9dac071250 2024-04-25T10:14:09.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.379+0000 7f9da1ffb640 1 -- 172.21.15.60:0/1257826626 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d9c055480 con 0x7f9dac071250 2024-04-25T10:14:09.384 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:14:09.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.379+0000 7f9db3454640 1 -- 172.21.15.60:0/1257826626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d90066f60 msgr2=0x7f9d90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.379+0000 7f9db3454640 1 --2- 172.21.15.60:0/1257826626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d90066f60 0x7f9d90069420 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0723f0 tx=0x7f9da8008040 comp rx=0 tx=0).stop 2024-04-25T10:14:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.379+0000 7f9db3454640 1 -- 172.21.15.60:0/1257826626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac071250 msgr2=0x7f9dac0ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.379+0000 7f9db3454640 1 --2- 172.21.15.60:0/1257826626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac071250 0x7f9dac0ce270 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00b410 tx=0x7f9d9c00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.379+0000 7f9db3454640 1 -- 172.21.15.60:0/1257826626 shutdown_connections 2024-04-25T10:14:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.379+0000 7f9db3454640 1 --2- 172.21.15.60:0/1257826626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9d90066f60 0x7f9d90069420 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.379+0000 7f9db3454640 1 --2- 172.21.15.60:0/1257826626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dac0b3f60 0x7f9dac1fa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.379+0000 7f9db3454640 1 --2- 172.21.15.60:0/1257826626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac071250 0x7f9dac0ce270 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.379+0000 7f9db3454640 1 --2- 172.21.15.60:0/1257826626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dac06f030 0x7f9dac0cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.379+0000 7f9db3454640 1 -- 172.21.15.60:0/1257826626 >> 172.21.15.60:0/1257826626 conn(0x7f9dac0670b0 msgr2=0x7f9dac0b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:09.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.379+0000 7f9db3454640 1 -- 172.21.15.60:0/1257826626 shutdown_connections 2024-04-25T10:14:09.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.379+0000 7f9db3454640 1 -- 172.21.15.60:0/1257826626 wait complete. 2024-04-25T10:14:09.401 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-25T10:14:09.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.511+0000 7f1407b20640 1 Processor -- start 2024-04-25T10:14:09.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.511+0000 7f1407b20640 1 -- start start 2024-04-25T10:14:09.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f1407b20640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1400160eb0 0x7f14000b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f1407b20640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f14000b9730 0x7f14000b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f1407b20640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14000ce9c0 con 0x7f1400161290 2024-04-25T10:14:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f1407b20640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14000ceca0 con 0x7f14000b9730 2024-04-25T10:14:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f1407b20640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14000d2490 con 0x7f1400160eb0 2024-04-25T10:14:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f14060bd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f14000b9730 0x7f14000b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f14050bb640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1400161290 0x7f14000b6a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45848/0 (socket says 172.21.15.60:45848) 2024-04-25T10:14:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f14050bb640 1 -- 172.21.15.60:0/1298157700 learned_addr learned my addr 172.21.15.60:0/1298157700 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f14058bc640 1 --2- 172.21.15.60:0/1298157700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1400160eb0 0x7f14000b91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f14048ba640 1 -- 172.21.15.60:0/1298157700 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506205796 0 0) 0x7f14000ce9c0 con 0x7f1400161290 2024-04-25T10:14:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f14060bd640 1 -- 172.21.15.60:0/1298157700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1400160eb0 msgr2=0x7f14000b91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f14060bd640 1 --2- 172.21.15.60:0/1298157700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1400160eb0 0x7f14000b91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f14060bd640 1 -- 172.21.15.60:0/1298157700 >> v1:172.21.15.60:6789/0 conn(0x7f1400161290 legacy=0x7f14000b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f14060bd640 1 -- 172.21.15.60:0/1298157700 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14000d2770 con 0x7f14000b9730 2024-04-25T10:14:09.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f14060bd640 1 --2- 172.21.15.60:0/1298157700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14000b9730 0x7f14000b53f0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009d90 tx=0x7f13f801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378f91d48720a137 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:09.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f14048ba640 1 -- 172.21.15.60:0/1298157700 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f80b4070 con 0x7f14000b9730 2024-04-25T10:14:09.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f14048ba640 1 -- 172.21.15.60:0/1298157700 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f80af600 con 0x7f14000b9730 2024-04-25T10:14:09.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f14048ba640 1 -- 172.21.15.60:0/1298157700 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f80af9d0 con 0x7f14000b9730 2024-04-25T10:14:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f1407b20640 1 -- 172.21.15.60:0/1298157700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14000b9730 msgr2=0x7f14000b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f1407b20640 1 --2- 172.21.15.60:0/1298157700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14000b9730 0x7f14000b53f0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009d90 tx=0x7f13f801f150 comp rx=0 tx=0).stop 2024-04-25T10:14:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f1407b20640 1 -- 172.21.15.60:0/1298157700 shutdown_connections 2024-04-25T10:14:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f1407b20640 1 --2- 172.21.15.60:0/1298157700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14000b9730 0x7f14000b53f0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f1407b20640 1 --2- 172.21.15.60:0/1298157700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1400160eb0 0x7f14000b91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f1407b20640 1 -- 172.21.15.60:0/1298157700 >> 172.21.15.60:0/1298157700 conn(0x7f14000d05c0 msgr2=0x7f14000d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f1407b20640 1 -- 172.21.15.60:0/1298157700 shutdown_connections 2024-04-25T10:14:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.515+0000 7f1407b20640 1 -- 172.21.15.60:0/1298157700 wait complete. 2024-04-25T10:14:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f1407b20640 1 Processor -- start 2024-04-25T10:14:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f1407b20640 1 -- start start 2024-04-25T10:14:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f1407b20640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14000b9730 0x7f140015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f1407b20640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1400160eb0 0x7f14000aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f1407b20640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1400161290 0x7f1400150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f1407b20640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14000cef80 con 0x7f14000b9730 2024-04-25T10:14:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f1407b20640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14000b7180 con 0x7f1400161290 2024-04-25T10:14:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f1407b20640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14000b5930 con 0x7f1400160eb0 2024-04-25T10:14:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f14058bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14000b9730 0x7f140015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f14058bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14000b9730 0x7f140015a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38328/0 (socket says 172.21.15.60:38328) 2024-04-25T10:14:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f14058bc640 1 -- 172.21.15.60:0/1266991257 learned_addr learned my addr 172.21.15.60:0/1266991257 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f14050bb640 1 --2- 172.21.15.60:0/1266991257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1400160eb0 0x7f14000aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f14060bd640 1 --2- 172.21.15.60:0/1266991257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1400161290 0x7f1400150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f14058bc640 1 -- 172.21.15.60:0/1266991257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1400160eb0 msgr2=0x7f14000aa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f14058bc640 1 --2- 172.21.15.60:0/1266991257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1400160eb0 0x7f14000aa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f14058bc640 1 -- 172.21.15.60:0/1266991257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1400161290 msgr2=0x7f1400150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f14058bc640 1 --2- 172.21.15.60:0/1266991257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1400161290 0x7f1400150dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f14058bc640 1 -- 172.21.15.60:0/1266991257 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14000aa830 con 0x7f14000b9730 2024-04-25T10:14:09.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f14058bc640 1 --2- 172.21.15.60:0/1266991257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14000b9730 0x7f140015a970 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00d800 tx=0x7f13fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f13f67fc640 1 -- 172.21.15.60:0/1266991257 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc00b6c0 con 0x7f14000b9730 2024-04-25T10:14:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f13f67fc640 1 -- 172.21.15.60:0/1266991257 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13fc019070 con 0x7f14000b9730 2024-04-25T10:14:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f1407b20640 1 -- 172.21.15.60:0/1266991257 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140015af50 con 0x7f14000b9730 2024-04-25T10:14:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f13f67fc640 1 -- 172.21.15.60:0/1266991257 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc00ba60 con 0x7f14000b9730 2024-04-25T10:14:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f1407b20640 1 -- 172.21.15.60:0/1266991257 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140015b510 con 0x7f14000b9730 2024-04-25T10:14:09.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f1407b20640 1 -- 172.21.15.60:0/1266991257 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1400074770 con 0x7f14000b9730 2024-04-25T10:14:09.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.519+0000 7f13f67fc640 1 -- 172.21.15.60:0/1266991257 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13fc00bc00 con 0x7f14000b9730 2024-04-25T10:14:09.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.523+0000 7f13f67fc640 1 --2- 172.21.15.60:0/1266991257 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13e4066f60 0x7f13e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.523+0000 7f13f67fc640 1 -- 172.21.15.60:0/1266991257 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f13fc0851c0 con 0x7f14000b9730 2024-04-25T10:14:09.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.523+0000 7f14050bb640 1 --2- 172.21.15.60:0/1266991257 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13e4066f60 0x7f13e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.523+0000 7f14050bb640 1 --2- 172.21.15.60:0/1266991257 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13e4066f60 0x7f13e4069420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f13f0007fd0 tx=0x7f13f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:09.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.527+0000 7f13f67fc640 1 -- 172.21.15.60:0/1266991257 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f13fc0564b0 con 0x7f14000b9730 2024-04-25T10:14:09.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:09.735+0000 7f1407b20640 1 -- 172.21.15.60:0/1266991257 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f14000d15b0 con 0x7f14000b9730 2024-04-25T10:14:10.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.735+0000 7f13f67fc640 1 -- 172.21.15.60:0/1266991257 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v407) v1 ==== 149+0+0 (secure 0 0 0) 0x7f13fc05ae20 con 0x7f14000b9730 2024-04-25T10:14:10.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.739+0000 7f1407b20640 1 -- 172.21.15.60:0/1266991257 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13e4066f60 msgr2=0x7f13e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.739+0000 7f1407b20640 1 --2- 172.21.15.60:0/1266991257 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13e4066f60 0x7f13e4069420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f13f0007fd0 tx=0x7f13f0009040 comp rx=0 tx=0).stop 2024-04-25T10:14:10.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.739+0000 7f1407b20640 1 -- 172.21.15.60:0/1266991257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14000b9730 msgr2=0x7f140015a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.739+0000 7f1407b20640 1 --2- 172.21.15.60:0/1266991257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14000b9730 0x7f140015a970 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00d800 tx=0x7f13fc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.739+0000 7f1407b20640 1 -- 172.21.15.60:0/1266991257 shutdown_connections 2024-04-25T10:14:10.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.739+0000 7f1407b20640 1 --2- 172.21.15.60:0/1266991257 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13e4066f60 0x7f13e4069420 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.739+0000 7f1407b20640 1 --2- 172.21.15.60:0/1266991257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1400161290 0x7f1400150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.739+0000 7f1407b20640 1 --2- 172.21.15.60:0/1266991257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1400160eb0 0x7f14000aa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.739+0000 7f1407b20640 1 --2- 172.21.15.60:0/1266991257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14000b9730 0x7f140015a970 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.739+0000 7f1407b20640 1 -- 172.21.15.60:0/1266991257 >> 172.21.15.60:0/1266991257 conn(0x7f14000d05c0 msgr2=0x7f14000d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.739+0000 7f1407b20640 1 -- 172.21.15.60:0/1266991257 shutdown_connections 2024-04-25T10:14:10.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.739+0000 7f1407b20640 1 -- 172.21.15.60:0/1266991257 wait complete. 2024-04-25T10:14:10.765 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:14:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdacd5b1640 1 Processor -- start 2024-04-25T10:14:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdacd5b1640 1 -- start start 2024-04-25T10:14:10.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdacd5b1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdac80cbcf0 0x7fdac8160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdacd5b1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdac8071c30 0x7fdac815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdacd5b1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac8144a40 con 0x7fdac815fa20 2024-04-25T10:14:10.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdacd5b1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac80b2dc0 con 0x7fdac8071c30 2024-04-25T10:14:10.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdacd5b1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac80cc9f0 con 0x7fdac80cbcf0 2024-04-25T10:14:10.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdac67fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdac8071c30 0x7fdac815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdac77fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdac815fa20 0x7fdac815bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45862/0 (socket says 172.21.15.60:45862) 2024-04-25T10:14:10.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdac77fe640 1 -- 172.21.15.60:0/1336022554 learned_addr learned my addr 172.21.15.60:0/1336022554 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:10.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdac6ffd640 1 --2- 172.21.15.60:0/1336022554 >> v2:172.21.15.100:3300/0 conn(0x7fdac80cbcf0 0x7fdac8160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdac67fc640 1 -- 172.21.15.60:0/1336022554 >> v2:172.21.15.100:3300/0 conn(0x7fdac80cbcf0 msgr2=0x7fdac8160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdac67fc640 1 --2- 172.21.15.60:0/1336022554 >> v2:172.21.15.100:3300/0 conn(0x7fdac80cbcf0 0x7fdac8160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdac67fc640 1 -- 172.21.15.60:0/1336022554 >> v1:172.21.15.60:6789/0 conn(0x7fdac815fa20 legacy=0x7fdac815bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdac67fc640 1 -- 172.21.15.60:0/1336022554 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac80cccd0 con 0x7fdac8071c30 2024-04-25T10:14:10.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdac6ffd640 1 --2- 172.21.15.60:0/1336022554 >> v2:172.21.15.100:3300/0 conn(0x7fdac80cbcf0 0x7fdac8160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:10.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.895+0000 7fdac67fc640 1 --2- 172.21.15.60:0/1336022554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac8071c30 0x7fdac815f4e0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fdac80b4c50 tx=0x7fdabc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad138849067ea06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdac5ffb640 1 -- 172.21.15.60:0/1336022554 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0b0070 con 0x7fdac8071c30 2024-04-25T10:14:10.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdac5ffb640 1 -- 172.21.15.60:0/1336022554 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdabc0ab600 con 0x7fdac8071c30 2024-04-25T10:14:10.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdac5ffb640 1 -- 172.21.15.60:0/1336022554 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0ab9d0 con 0x7fdac8071c30 2024-04-25T10:14:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdacd5b1640 1 -- 172.21.15.60:0/1336022554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac8071c30 msgr2=0x7fdac815f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdacd5b1640 1 --2- 172.21.15.60:0/1336022554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac8071c30 0x7fdac815f4e0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fdac80b4c50 tx=0x7fdabc01c480 comp rx=0 tx=0).stop 2024-04-25T10:14:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdacd5b1640 1 -- 172.21.15.60:0/1336022554 shutdown_connections 2024-04-25T10:14:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdacd5b1640 1 --2- 172.21.15.60:0/1336022554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac8071c30 0x7fdac815f4e0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdacd5b1640 1 --2- 172.21.15.60:0/1336022554 >> v2:172.21.15.100:3300/0 conn(0x7fdac80cbcf0 0x7fdac8160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdacd5b1640 1 -- 172.21.15.60:0/1336022554 >> 172.21.15.60:0/1336022554 conn(0x7fdac80670b0 msgr2=0x7fdac806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdacd5b1640 1 -- 172.21.15.60:0/1336022554 shutdown_connections 2024-04-25T10:14:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdacd5b1640 1 -- 172.21.15.60:0/1336022554 wait complete. 2024-04-25T10:14:10.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdacd5b1640 1 Processor -- start 2024-04-25T10:14:10.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdacd5b1640 1 -- start start 2024-04-25T10:14:10.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdacd5b1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdac8071c30 0x7fdac8160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdacd5b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac80cbcf0 0x7fdac815b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdacd5b1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac815fa20 0x7fdac8165910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdacd5b1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac80cfed0 con 0x7fdac815fa20 2024-04-25T10:14:10.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdacd5b1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac80720b0 con 0x7fdac80cbcf0 2024-04-25T10:14:10.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdacd5b1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac8166480 con 0x7fdac8071c30 2024-04-25T10:14:10.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdac77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac815fa20 0x7fdac8165910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdac77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac815fa20 0x7fdac8165910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38338/0 (socket says 172.21.15.60:38338) 2024-04-25T10:14:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdac77fe640 1 -- 172.21.15.60:0/1493032757 learned_addr learned my addr 172.21.15.60:0/1493032757 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdac67fc640 1 --2- 172.21.15.60:0/1493032757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac80cbcf0 0x7fdac815b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdac6ffd640 1 --2- 172.21.15.60:0/1493032757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdac8071c30 0x7fdac8160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdac77fe640 1 -- 172.21.15.60:0/1493032757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdac8071c30 msgr2=0x7fdac8160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdac77fe640 1 --2- 172.21.15.60:0/1493032757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdac8071c30 0x7fdac8160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdac77fe640 1 -- 172.21.15.60:0/1493032757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac80cbcf0 msgr2=0x7fdac815b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdac77fe640 1 --2- 172.21.15.60:0/1493032757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac80cbcf0 0x7fdac815b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdac77fe640 1 -- 172.21.15.60:0/1493032757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac80af860 con 0x7fdac815fa20 2024-04-25T10:14:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdac6ffd640 1 --2- 172.21.15.60:0/1493032757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdac8071c30 0x7fdac8160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdac67fc640 1 --2- 172.21.15.60:0/1493032757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac80cbcf0 0x7fdac815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.899+0000 7fdac77fe640 1 --2- 172.21.15.60:0/1493032757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac815fa20 0x7fdac8165910 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fdac000cad0 tx=0x7fdac000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.903+0000 7fdaaf7fe640 1 -- 172.21.15.60:0/1493032757 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0017070 con 0x7fdac815fa20 2024-04-25T10:14:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.903+0000 7fdaaf7fe640 1 -- 172.21.15.60:0/1493032757 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac000ac30 con 0x7fdac815fa20 2024-04-25T10:14:10.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.903+0000 7fdacd5b1640 1 -- 172.21.15.60:0/1493032757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac80afb40 con 0x7fdac815fa20 2024-04-25T10:14:10.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.903+0000 7fdaaf7fe640 1 -- 172.21.15.60:0/1493032757 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0012620 con 0x7fdac815fa20 2024-04-25T10:14:10.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.903+0000 7fdacd5b1640 1 -- 172.21.15.60:0/1493032757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac80b0100 con 0x7fdac815fa20 2024-04-25T10:14:10.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.903+0000 7fdacd5b1640 1 -- 172.21.15.60:0/1493032757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac806bba0 con 0x7fdac815fa20 2024-04-25T10:14:10.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.903+0000 7fdaaf7fe640 1 -- 172.21.15.60:0/1493032757 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdac001a070 con 0x7fdac815fa20 2024-04-25T10:14:10.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.903+0000 7fdaaf7fe640 1 --2- 172.21.15.60:0/1493032757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdab4066f60 0x7fdab4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.903+0000 7fdaaf7fe640 1 -- 172.21.15.60:0/1493032757 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fdac0084ff0 con 0x7fdac815fa20 2024-04-25T10:14:10.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.903+0000 7fdac6ffd640 1 --2- 172.21.15.60:0/1493032757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdab4066f60 0x7fdab4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.903+0000 7fdac6ffd640 1 --2- 172.21.15.60:0/1493032757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdab4066f60 0x7fdab4069420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdab8008930 tx=0x7fdab8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:10.907+0000 7fdaaf7fe640 1 -- 172.21.15.60:0/1493032757 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdac0021020 con 0x7fdac815fa20 2024-04-25T10:14:11.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.067+0000 7fdacd5b1640 1 -- 172.21.15.60:0/1493032757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdac8074770 con 0x7fdac815fa20 2024-04-25T10:14:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.071+0000 7fdaaf7fe640 1 -- 172.21.15.60:0/1493032757 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdac000ae20 con 0x7fdac815fa20 2024-04-25T10:14:11.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.075+0000 7fdacd5b1640 1 -- 172.21.15.60:0/1493032757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdab4066f60 msgr2=0x7fdab4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.075+0000 7fdacd5b1640 1 --2- 172.21.15.60:0/1493032757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdab4066f60 0x7fdab4069420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdab8008930 tx=0x7fdab8008040 comp rx=0 tx=0).stop 2024-04-25T10:14:11.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.075+0000 7fdacd5b1640 1 -- 172.21.15.60:0/1493032757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac815fa20 msgr2=0x7fdac8165910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.075+0000 7fdacd5b1640 1 --2- 172.21.15.60:0/1493032757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac815fa20 0x7fdac8165910 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fdac000cad0 tx=0x7fdac000cf90 comp rx=0 tx=0).stop 2024-04-25T10:14:11.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.075+0000 7fdacd5b1640 1 -- 172.21.15.60:0/1493032757 shutdown_connections 2024-04-25T10:14:11.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.075+0000 7fdacd5b1640 1 --2- 172.21.15.60:0/1493032757 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdab4066f60 0x7fdab4069420 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.075+0000 7fdacd5b1640 1 --2- 172.21.15.60:0/1493032757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac815fa20 0x7fdac8165910 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.075+0000 7fdacd5b1640 1 --2- 172.21.15.60:0/1493032757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac80cbcf0 0x7fdac815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.075+0000 7fdacd5b1640 1 --2- 172.21.15.60:0/1493032757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdac8071c30 0x7fdac8160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.075+0000 7fdacd5b1640 1 -- 172.21.15.60:0/1493032757 >> 172.21.15.60:0/1493032757 conn(0x7fdac80670b0 msgr2=0x7fdac813eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.075+0000 7fdacd5b1640 1 -- 172.21.15.60:0/1493032757 shutdown_connections 2024-04-25T10:14:11.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.075+0000 7fdacd5b1640 1 -- 172.21.15.60:0/1493032757 wait complete. 2024-04-25T10:14:11.096 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:14:11.096 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:11.148 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:11.199 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:14:11.200 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:14:11.200 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:11.205 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:11.205 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:14:11.205 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:11.211 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:11.211 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:14:11.211 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:11.251 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:11.251 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:14:11.251 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:11.257 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:11.257 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:14:11.258 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:11.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.419+0000 7fc44bcd8640 1 Processor -- start 2024-04-25T10:14:11.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.419+0000 7fc44bcd8640 1 -- start start 2024-04-25T10:14:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.419+0000 7fc44bcd8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc44415a7a0 0x7fc4440cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.419+0000 7fc44bcd8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc44406ec30 0x7fc444071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.419+0000 7fc44bcd8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4440721e0 con 0x7fc4440b37e0 2024-04-25T10:14:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.419+0000 7fc44bcd8640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4440cdaa0 con 0x7fc44415a7a0 2024-04-25T10:14:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.419+0000 7fc44bcd8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44406d2f0 con 0x7fc44406ec30 2024-04-25T10:14:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.419+0000 7fc449a74640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc44415a7a0 0x7fc4440cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.419+0000 7fc449a74640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc44415a7a0 0x7fc4440cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38350/0 (socket says 172.21.15.60:38350) 2024-04-25T10:14:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.419+0000 7fc449a74640 1 -- 172.21.15.60:0/4123935932 learned_addr learned my addr 172.21.15.60:0/4123935932 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.419+0000 7fc449273640 1 --2- 172.21.15.60:0/4123935932 >> v2:172.21.15.100:3300/0 conn(0x7fc44406ec30 0x7fc444071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.419+0000 7fc449a74640 1 -- 172.21.15.60:0/4123935932 >> v2:172.21.15.100:3300/0 conn(0x7fc44406ec30 msgr2=0x7fc444071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.419+0000 7fc449a74640 1 --2- 172.21.15.60:0/4123935932 >> v2:172.21.15.100:3300/0 conn(0x7fc44406ec30 0x7fc444071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.419+0000 7fc449a74640 1 -- 172.21.15.60:0/4123935932 >> v1:172.21.15.100:6789/0 conn(0x7fc4440b37e0 legacy=0x7fc44406dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.419+0000 7fc449a74640 1 -- 172.21.15.60:0/4123935932 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44406f130 con 0x7fc44415a7a0 2024-04-25T10:14:11.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.423+0000 7fc449a74640 1 --2- 172.21.15.60:0/4123935932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc44415a7a0 0x7fc4440cd560 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fc444072bf0 tx=0x7fc44001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648da5000c626163 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.423+0000 7fc448a72640 1 -- 172.21.15.60:0/4123935932 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4400ab430 con 0x7fc44415a7a0 2024-04-25T10:14:11.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.423+0000 7fc448a72640 1 -- 172.21.15.60:0/4123935932 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4400b0070 con 0x7fc44415a7a0 2024-04-25T10:14:11.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.423+0000 7fc448a72640 1 -- 172.21.15.60:0/4123935932 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4400ab7f0 con 0x7fc44415a7a0 2024-04-25T10:14:11.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.423+0000 7fc44bcd8640 1 -- 172.21.15.60:0/4123935932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc44415a7a0 msgr2=0x7fc4440cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.423+0000 7fc44bcd8640 1 --2- 172.21.15.60:0/4123935932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc44415a7a0 0x7fc4440cd560 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fc444072bf0 tx=0x7fc44001c520 comp rx=0 tx=0).stop 2024-04-25T10:14:11.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.423+0000 7fc44bcd8640 1 -- 172.21.15.60:0/4123935932 shutdown_connections 2024-04-25T10:14:11.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.423+0000 7fc44bcd8640 1 --2- 172.21.15.60:0/4123935932 >> v2:172.21.15.100:3300/0 conn(0x7fc44406ec30 0x7fc444071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.423+0000 7fc44bcd8640 1 --2- 172.21.15.60:0/4123935932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc44415a7a0 0x7fc4440cd560 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.423+0000 7fc44bcd8640 1 -- 172.21.15.60:0/4123935932 >> 172.21.15.60:0/4123935932 conn(0x7fc4440b12d0 msgr2=0x7fc4440b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.423+0000 7fc44bcd8640 1 -- 172.21.15.60:0/4123935932 shutdown_connections 2024-04-25T10:14:11.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.423+0000 7fc44bcd8640 1 -- 172.21.15.60:0/4123935932 wait complete. 2024-04-25T10:14:11.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.423+0000 7fc44bcd8640 1 Processor -- start 2024-04-25T10:14:11.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.423+0000 7fc44bcd8640 1 -- start start 2024-04-25T10:14:11.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44bcd8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc44406ec30 0x7fc444162e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44bcd8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440b37e0 0x7fc444155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44bcd8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc44415a7a0 0x7fc444163400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44bcd8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44415ad00 con 0x7fc44415a7a0 2024-04-25T10:14:11.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44bcd8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44415ab80 con 0x7fc4440b37e0 2024-04-25T10:14:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44bcd8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44406e860 con 0x7fc44406ec30 2024-04-25T10:14:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44a275640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc44415a7a0 0x7fc444163400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44a275640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc44415a7a0 0x7fc444163400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38356/0 (socket says 172.21.15.60:38356) 2024-04-25T10:14:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44a275640 1 -- 172.21.15.60:0/2837464531 learned_addr learned my addr 172.21.15.60:0/2837464531 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc449a74640 1 --2- 172.21.15.60:0/2837464531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc44406ec30 0x7fc444162e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc449273640 1 --2- 172.21.15.60:0/2837464531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440b37e0 0x7fc444155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44a275640 1 -- 172.21.15.60:0/2837464531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc44406ec30 msgr2=0x7fc444162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44a275640 1 --2- 172.21.15.60:0/2837464531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc44406ec30 0x7fc444162e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44a275640 1 -- 172.21.15.60:0/2837464531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440b37e0 msgr2=0x7fc444155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44a275640 1 --2- 172.21.15.60:0/2837464531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440b37e0 0x7fc444155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44a275640 1 -- 172.21.15.60:0/2837464531 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc444163ae0 con 0x7fc44415a7a0 2024-04-25T10:14:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44a275640 1 --2- 172.21.15.60:0/2837464531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc44415a7a0 0x7fc444163400 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00c710 tx=0x7fc43c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc43a7fc640 1 -- 172.21.15.60:0/2837464531 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c016020 con 0x7fc44415a7a0 2024-04-25T10:14:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc43a7fc640 1 -- 172.21.15.60:0/2837464531 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c006d60 con 0x7fc44415a7a0 2024-04-25T10:14:11.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44bcd8640 1 -- 172.21.15.60:0/2837464531 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc444162350 con 0x7fc44415a7a0 2024-04-25T10:14:11.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc43a7fc640 1 -- 172.21.15.60:0/2837464531 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c003bc0 con 0x7fc44415a7a0 2024-04-25T10:14:11.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44bcd8640 1 -- 172.21.15.60:0/2837464531 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc444162910 con 0x7fc44415a7a0 2024-04-25T10:14:11.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc44bcd8640 1 -- 172.21.15.60:0/2837464531 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc444162b10 con 0x7fc44415a7a0 2024-04-25T10:14:11.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc43a7fc640 1 -- 172.21.15.60:0/2837464531 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc43c00a850 con 0x7fc44415a7a0 2024-04-25T10:14:11.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc43a7fc640 1 --2- 172.21.15.60:0/2837464531 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc428066f60 0x7fc428069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc43a7fc640 1 -- 172.21.15.60:0/2837464531 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fc43c085750 con 0x7fc44415a7a0 2024-04-25T10:14:11.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.427+0000 7fc449a74640 1 --2- 172.21.15.60:0/2837464531 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc428066f60 0x7fc428069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.431+0000 7fc449a74640 1 --2- 172.21.15.60:0/2837464531 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc428066f60 0x7fc428069420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc4400af000 tx=0x7fc4400a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.431+0000 7fc43a7fc640 1 -- 172.21.15.60:0/2837464531 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc444162b10 con 0x7fc44415a7a0 2024-04-25T10:14:11.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.627+0000 7fc44bcd8640 1 -- 172.21.15.60:0/2837464531 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc444074770 con 0x7fc44415a7a0 2024-04-25T10:14:11.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.627+0000 7fc43a7fc640 1 -- 172.21.15.60:0/2837464531 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fc43c020020 con 0x7fc44415a7a0 2024-04-25T10:14:11.632 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:11.632 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":70,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6436,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1902473049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1902473049},{"type":"v1","addr":"172.21.15.100:6835","nonce":1902473049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6439,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2602504105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2602504105},{"type":"v1","addr":"172.21.15.100:6837","nonce":2602504105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7650,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/433798940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":433798940},{"type":"v1","addr":"172.21.15.60:6834","nonce":433798940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":8538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3071786480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3071786480},{"type":"v1","addr":"172.21.15.60:6835","nonce":3071786480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[]} 2024-04-25T10:14:11.632 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 70 2024-04-25T10:14:11.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.627+0000 7fc44bcd8640 1 -- 172.21.15.60:0/2837464531 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc428066f60 msgr2=0x7fc428069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.627+0000 7fc44bcd8640 1 --2- 172.21.15.60:0/2837464531 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc428066f60 0x7fc428069420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc4400af000 tx=0x7fc4400a7000 comp rx=0 tx=0).stop 2024-04-25T10:14:11.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.627+0000 7fc44bcd8640 1 -- 172.21.15.60:0/2837464531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc44415a7a0 msgr2=0x7fc444163400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.627+0000 7fc44bcd8640 1 --2- 172.21.15.60:0/2837464531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc44415a7a0 0x7fc444163400 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00c710 tx=0x7fc43c00cbd0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.627+0000 7fc44bcd8640 1 -- 172.21.15.60:0/2837464531 shutdown_connections 2024-04-25T10:14:11.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.627+0000 7fc44bcd8640 1 --2- 172.21.15.60:0/2837464531 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc428066f60 0x7fc428069420 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.627+0000 7fc44bcd8640 1 --2- 172.21.15.60:0/2837464531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc44415a7a0 0x7fc444163400 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.627+0000 7fc44bcd8640 1 --2- 172.21.15.60:0/2837464531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440b37e0 0x7fc444155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.627+0000 7fc44bcd8640 1 --2- 172.21.15.60:0/2837464531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc44406ec30 0x7fc444162e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.627+0000 7fc44bcd8640 1 -- 172.21.15.60:0/2837464531 >> 172.21.15.60:0/2837464531 conn(0x7fc4440b12d0 msgr2=0x7fc444159820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.631+0000 7fc44bcd8640 1 -- 172.21.15.60:0/2837464531 shutdown_connections 2024-04-25T10:14:11.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.631+0000 7fc44bcd8640 1 -- 172.21.15.60:0/2837464531 wait complete. 2024-04-25T10:14:11.652 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:14:11.652 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:14:11.652 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:14:11.652 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:11.652 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:14:11.653 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:14:11.653 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:14:11.653 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:11.653 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:14:11.653 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:14:11.653 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:14:11.653 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:11.653 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:14:11.654 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:14:11.654 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:14:11.654 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:11.665 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:14:11.666 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:14:11.667 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:14:11.667 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:14:11.668 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:14:11.668 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:14:11.669 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:14:11.670 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:14:11.712 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:14:11.712 DEBUG:teuthology.parallel:result is None 2024-04-25T10:14:11.713 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:14:11.713 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:14:11.713 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:14:11.714 DEBUG:teuthology.parallel:result is None 2024-04-25T10:14:11.714 DEBUG:teuthology.parallel:result is None 2024-04-25T10:14:11.714 DEBUG:teuthology.parallel:result is None 2024-04-25T10:14:11.714 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:14:11.775 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:14:11.782 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T10:14:11.785 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T10:14:11.786 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T10:14:11.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.835+0000 7f45dff08640 1 Processor -- start 2024-04-25T10:14:11.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.835+0000 7f45dff08640 1 -- start start 2024-04-25T10:14:11.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.835+0000 7f45dff08640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f45d815deb0 0x7f45d80b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.835+0000 7f45dff08640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f45d80ce9c0 0x7f45d80d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.835+0000 7f45dff08640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d806d200 con 0x7f45d80d0340 2024-04-25T10:14:11.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.835+0000 7f45dff08640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d80cf2c0 con 0x7f45d80ce9c0 2024-04-25T10:14:11.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.835+0000 7f45dff08640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d80cefe0 con 0x7f45d815deb0 2024-04-25T10:14:11.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.835+0000 7f45de4a5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f45d80ce9c0 0x7f45d80d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.835+0000 7f45dd4a3640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f45d80d0340 0x7f45d80ce2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45882/0 (socket says 172.21.15.60:45882) 2024-04-25T10:14:11.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.835+0000 7f45dd4a3640 1 -- 172.21.15.60:0/2197229755 learned_addr learned my addr 172.21.15.60:0/2197229755 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:11.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.835+0000 7f45ddca4640 1 --2- 172.21.15.60:0/2197229755 >> v2:172.21.15.100:3300/0 conn(0x7f45d815deb0 0x7f45d80b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.835+0000 7f45de4a5640 1 -- 172.21.15.60:0/2197229755 >> v2:172.21.15.100:3300/0 conn(0x7f45d815deb0 msgr2=0x7f45d80b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:11.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.835+0000 7f45de4a5640 1 --2- 172.21.15.60:0/2197229755 >> v2:172.21.15.100:3300/0 conn(0x7f45d815deb0 0x7f45d80b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.835+0000 7f45de4a5640 1 -- 172.21.15.60:0/2197229755 >> v1:172.21.15.60:6789/0 conn(0x7f45d80d0340 legacy=0x7f45d80ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.835+0000 7f45de4a5640 1 -- 172.21.15.60:0/2197229755 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d80d1d70 con 0x7f45d80ce9c0 2024-04-25T10:14:11.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.835+0000 7f45ddca4640 1 --2- 172.21.15.60:0/2197229755 >> v2:172.21.15.100:3300/0 conn(0x7f45d815deb0 0x7f45d80b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:11.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.839+0000 7f45de4a5640 1 --2- 172.21.15.60:0/2197229755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80ce9c0 0x7f45d80d07c0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f45d808bd20 tx=0x7f45d001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c250f8f21acf23c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.839+0000 7f45dcca2640 1 -- 172.21.15.60:0/2197229755 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d00b0070 con 0x7f45d80ce9c0 2024-04-25T10:14:11.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.839+0000 7f45dcca2640 1 -- 172.21.15.60:0/2197229755 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d00ab600 con 0x7f45d80ce9c0 2024-04-25T10:14:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.839+0000 7f45dcca2640 1 -- 172.21.15.60:0/2197229755 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d00ab9d0 con 0x7f45d80ce9c0 2024-04-25T10:14:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.839+0000 7f45dff08640 1 -- 172.21.15.60:0/2197229755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80ce9c0 msgr2=0x7f45d80d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.839+0000 7f45dff08640 1 --2- 172.21.15.60:0/2197229755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80ce9c0 0x7f45d80d07c0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f45d808bd20 tx=0x7f45d001c480 comp rx=0 tx=0).stop 2024-04-25T10:14:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.839+0000 7f45dff08640 1 -- 172.21.15.60:0/2197229755 shutdown_connections 2024-04-25T10:14:11.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.839+0000 7f45dff08640 1 --2- 172.21.15.60:0/2197229755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80ce9c0 0x7f45d80d07c0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.839+0000 7f45dff08640 1 --2- 172.21.15.60:0/2197229755 >> v2:172.21.15.100:3300/0 conn(0x7f45d815deb0 0x7f45d80b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.839+0000 7f45dff08640 1 -- 172.21.15.60:0/2197229755 >> 172.21.15.60:0/2197229755 conn(0x7f45d806c090 msgr2=0x7f45d806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.839+0000 7f45dff08640 1 -- 172.21.15.60:0/2197229755 shutdown_connections 2024-04-25T10:14:11.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.839+0000 7f45dff08640 1 -- 172.21.15.60:0/2197229755 wait complete. 2024-04-25T10:14:11.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.839+0000 7f45dff08640 1 Processor -- start 2024-04-25T10:14:11.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.839+0000 7f45dff08640 1 -- start start 2024-04-25T10:14:11.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dff08640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45d80ce9c0 0x7f45d8147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dff08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80d0340 0x7f45d814ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dff08640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45d815deb0 0x7f45d81549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dff08640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d80d3430 con 0x7f45d80d0340 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dff08640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d80d32b0 con 0x7f45d80ce9c0 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dff08640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d80d35b0 con 0x7f45d815deb0 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dd4a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80d0340 0x7f45d814ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dd4a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80d0340 0x7f45d814ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38400/0 (socket says 172.21.15.60:38400) 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dd4a3640 1 -- 172.21.15.60:0/3546849230 learned_addr learned my addr 172.21.15.60:0/3546849230 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45ddca4640 1 --2- 172.21.15.60:0/3546849230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45d80ce9c0 0x7f45d8147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45de4a5640 1 --2- 172.21.15.60:0/3546849230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45d815deb0 0x7f45d81549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dd4a3640 1 -- 172.21.15.60:0/3546849230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45d815deb0 msgr2=0x7f45d81549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dd4a3640 1 --2- 172.21.15.60:0/3546849230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45d815deb0 0x7f45d81549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dd4a3640 1 -- 172.21.15.60:0/3546849230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45d80ce9c0 msgr2=0x7f45d8147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dd4a3640 1 --2- 172.21.15.60:0/3546849230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45d80ce9c0 0x7f45d8147fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dd4a3640 1 -- 172.21.15.60:0/3546849230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d80d0fd0 con 0x7f45d80d0340 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dd4a3640 1 --2- 172.21.15.60:0/3546849230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80d0340 0x7f45d814ed40 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00c420 tx=0x7f45cc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45ca7fc640 1 -- 172.21.15.60:0/3546849230 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc017070 con 0x7f45d80d0340 2024-04-25T10:14:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45ca7fc640 1 -- 172.21.15.60:0/3546849230 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45cc005cf0 con 0x7f45d80d0340 2024-04-25T10:14:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dff08640 1 -- 172.21.15.60:0/3546849230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d8148520 con 0x7f45d80d0340 2024-04-25T10:14:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45ca7fc640 1 -- 172.21.15.60:0/3546849230 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0072f0 con 0x7f45d80d0340 2024-04-25T10:14:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dff08640 1 -- 172.21.15.60:0/3546849230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d8148d80 con 0x7f45d80d0340 2024-04-25T10:14:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45dff08640 1 -- 172.21.15.60:0/3546849230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d8074770 con 0x7f45d80d0340 2024-04-25T10:14:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45ca7fc640 1 -- 172.21.15.60:0/3546849230 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45cc01a070 con 0x7f45d80d0340 2024-04-25T10:14:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45ca7fc640 1 --2- 172.21.15.60:0/3546849230 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f45bc066f90 0x7f45bc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45ca7fc640 1 -- 172.21.15.60:0/3546849230 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f45cc0840f0 con 0x7f45d80d0340 2024-04-25T10:14:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.843+0000 7f45ddca4640 1 --2- 172.21.15.60:0/3546849230 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f45bc066f90 0x7f45bc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.847+0000 7f45ddca4640 1 --2- 172.21.15.60:0/3546849230 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f45bc066f90 0x7f45bc069450 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f45d400d620 tx=0x7f45d4007810 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:11.847+0000 7f45ca7fc640 1 -- 172.21.15.60:0/3546849230 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45cc054d00 con 0x7f45d80d0340 2024-04-25T10:14:12.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.019+0000 7f45dff08640 1 -- 172.21.15.60:0/3546849230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f45d815baa0 con 0x7f45d80d0340 2024-04-25T10:14:12.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.755+0000 7f45ca7fc640 1 -- 172.21.15.60:0/3546849230 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v91) v1 ==== 97+0+0 (secure 0 0 0) 0x7f45cc058da0 con 0x7f45d80d0340 2024-04-25T10:14:12.760 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-04-25T10:14:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.759+0000 7f45dff08640 1 -- 172.21.15.60:0/3546849230 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f45bc066f90 msgr2=0x7f45bc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.759+0000 7f45dff08640 1 --2- 172.21.15.60:0/3546849230 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f45bc066f90 0x7f45bc069450 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f45d400d620 tx=0x7f45d4007810 comp rx=0 tx=0).stop 2024-04-25T10:14:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.759+0000 7f45dff08640 1 -- 172.21.15.60:0/3546849230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80d0340 msgr2=0x7f45d814ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.759+0000 7f45dff08640 1 --2- 172.21.15.60:0/3546849230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80d0340 0x7f45d814ed40 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00c420 tx=0x7f45cc00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.759+0000 7f45dff08640 1 -- 172.21.15.60:0/3546849230 shutdown_connections 2024-04-25T10:14:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.759+0000 7f45dff08640 1 --2- 172.21.15.60:0/3546849230 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f45bc066f90 0x7f45bc069450 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.759+0000 7f45dff08640 1 --2- 172.21.15.60:0/3546849230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45d815deb0 0x7f45d81549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.759+0000 7f45dff08640 1 --2- 172.21.15.60:0/3546849230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80d0340 0x7f45d814ed40 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.759+0000 7f45dff08640 1 --2- 172.21.15.60:0/3546849230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45d80ce9c0 0x7f45d8147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.759+0000 7f45dff08640 1 -- 172.21.15.60:0/3546849230 >> 172.21.15.60:0/3546849230 conn(0x7f45d806c090 msgr2=0x7f45d80c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.759+0000 7f45dff08640 1 -- 172.21.15.60:0/3546849230 shutdown_connections 2024-04-25T10:14:12.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.759+0000 7f45dff08640 1 -- 172.21.15.60:0/3546849230 wait complete. 2024-04-25T10:14:12.786 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:14:12.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.903+0000 7f4aa3122640 1 Processor -- start 2024-04-25T10:14:12.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.903+0000 7f4aa3122640 1 -- start start 2024-04-25T10:14:12.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.903+0000 7f4aa3122640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4a9c15f480 0x7f4a9c15f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.903+0000 7f4aa3122640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a9c15fda0 con 0x7f4a9c0cf380 2024-04-25T10:14:12.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4aa3122640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a9c0cc5d0 con 0x7f4a9c0cd390 2024-04-25T10:14:12.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4aa3122640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a9c0cc8b0 con 0x7f4a9c15f480 2024-04-25T10:14:12.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4aa16bf640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4a9c0cf380 0x7f4a9c0cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45898/0 (socket says 172.21.15.60:45898) 2024-04-25T10:14:12.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4aa16bf640 1 -- 172.21.15.60:0/739976250 learned_addr learned my addr 172.21.15.60:0/739976250 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:12.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4aa0ebe640 1 --2- 172.21.15.60:0/739976250 >> v2:172.21.15.100:3300/0 conn(0x7f4a9c15f480 0x7f4a9c15f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/739976250 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486269820 0 0) 0x7f4a9c15fda0 con 0x7f4a9c0cf380 2024-04-25T10:14:12.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/739976250 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a8c003530 con 0x7f4a9c0cf380 2024-04-25T10:14:12.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/739976250 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2149355729 0 0) 0x7f4a8c003530 con 0x7f4a9c0cf380 2024-04-25T10:14:12.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/739976250 >> v2:172.21.15.100:3300/0 conn(0x7f4a9c15f480 msgr2=0x7f4a9c15f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:12.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 --2- 172.21.15.60:0/739976250 >> v2:172.21.15.100:3300/0 conn(0x7f4a9c15f480 0x7f4a9c15f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/739976250 >> v1:172.21.15.100:6789/0 conn(0x7f4a9c0cd390 legacy=0x7f4a9c09a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/739976250 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9c0ccb90 con 0x7f4a9c0cf380 2024-04-25T10:14:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/739976250 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f4a94001a00 con 0x7f4a9c0cf380 2024-04-25T10:14:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/739976250 >> v1:172.21.15.60:6789/0 conn(0x7f4a9c0cf380 legacy=0x7f4a9c0cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 --2- 172.21.15.60:0/739976250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a8c003a10 0x7f4a8c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 --2- 172.21.15.60:0/739976250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a8c004810 0x7f4a8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 --2- 172.21.15.60:0/739976250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a8c0099c0 0x7f4a8c005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/739976250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c0cc8b0 con 0x7f4a8c004810 2024-04-25T10:14:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/739976250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c014900 con 0x7f4a8c0099c0 2024-04-25T10:14:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/739976250 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c014b00 con 0x7f4a8c003a10 2024-04-25T10:14:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9bfff640 1 --2- 172.21.15.60:0/739976250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a8c004810 0x7f4a8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4aa16bf640 1 --2- 172.21.15.60:0/739976250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a8c0099c0 0x7f4a8c005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4aa0ebe640 1 --2- 172.21.15.60:0/739976250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a8c003a10 0x7f4a8c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9bfff640 1 -- 172.21.15.60:0/739976250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a8c003a10 msgr2=0x7f4a8c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9bfff640 1 --2- 172.21.15.60:0/739976250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a8c003a10 0x7f4a8c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9bfff640 1 -- 172.21.15.60:0/739976250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a8c0099c0 msgr2=0x7f4a8c005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9bfff640 1 --2- 172.21.15.60:0/739976250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a8c0099c0 0x7f4a8c005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9bfff640 1 -- 172.21.15.60:0/739976250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c003530 con 0x7f4a8c004810 2024-04-25T10:14:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4aa16bf640 1 -- 172.21.15.60:0/739976250 reap_dead start 2024-04-25T10:14:12.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9bfff640 1 --2- 172.21.15.60:0/739976250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a8c004810 0x7f4a8c004c90 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f4a88001780 tx=0x7f4a8801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f6a6afc6999ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/739976250 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a880013a0 con 0x7f4a8c004810 2024-04-25T10:14:12.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/739976250 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a880052f0 con 0x7f4a8c004810 2024-04-25T10:14:12.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.907+0000 7f4a9b7fe640 1 -- 172.21.15.60:0/739976250 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a88003840 con 0x7f4a8c004810 2024-04-25T10:14:12.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 -- 172.21.15.60:0/739976250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a8c004810 msgr2=0x7f4a8c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 --2- 172.21.15.60:0/739976250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a8c004810 0x7f4a8c004c90 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f4a88001780 tx=0x7f4a8801c3e0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 -- 172.21.15.60:0/739976250 shutdown_connections 2024-04-25T10:14:12.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 --2- 172.21.15.60:0/739976250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a8c004810 0x7f4a8c004c90 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 -- 172.21.15.60:0/739976250 >> 172.21.15.60:0/739976250 conn(0x7f4a9c0b32a0 msgr2=0x7f4a9c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:12.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 -- 172.21.15.60:0/739976250 shutdown_connections 2024-04-25T10:14:12.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 -- 172.21.15.60:0/739976250 wait complete. 2024-04-25T10:14:12.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 Processor -- start 2024-04-25T10:14:12.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 -- start start 2024-04-25T10:14:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a8c004810 0x7f4a9c154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9c0b4740 0x7f4a9c155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c0b4bc0 0x7f4a9c1564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c160170 con 0x7f4a9c0b4bc0 2024-04-25T10:14:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c0cc2c0 con 0x7f4a8c004810 2024-04-25T10:14:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c1602f0 con 0x7f4a9c0b4740 2024-04-25T10:14:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa16bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c0b4bc0 0x7f4a9c1564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa16bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c0b4bc0 0x7f4a9c1564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38416/0 (socket says 172.21.15.60:38416) 2024-04-25T10:14:12.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa16bf640 1 -- 172.21.15.60:0/2618744531 learned_addr learned my addr 172.21.15.60:0/2618744531 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:12.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa0ebe640 1 --2- 172.21.15.60:0/2618744531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a8c004810 0x7f4a9c154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4a9bfff640 1 --2- 172.21.15.60:0/2618744531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9c0b4740 0x7f4a9c155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa16bf640 1 -- 172.21.15.60:0/2618744531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9c0b4740 msgr2=0x7f4a9c155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa16bf640 1 --2- 172.21.15.60:0/2618744531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9c0b4740 0x7f4a9c155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa16bf640 1 -- 172.21.15.60:0/2618744531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a8c004810 msgr2=0x7f4a9c154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa16bf640 1 --2- 172.21.15.60:0/2618744531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a8c004810 0x7f4a9c154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa16bf640 1 -- 172.21.15.60:0/2618744531 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9c0c53f0 con 0x7f4a9c0b4bc0 2024-04-25T10:14:12.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa0ebe640 1 --2- 172.21.15.60:0/2618744531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a8c004810 0x7f4a9c154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:12.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4a9bfff640 1 --2- 172.21.15.60:0/2618744531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9c0b4740 0x7f4a9c155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:12.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa16bf640 1 --2- 172.21.15.60:0/2618744531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c0b4bc0 0x7f4a9c1564b0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c072680 tx=0x7f4a94011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4a997fa640 1 -- 172.21.15.60:0/2618744531 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94010040 con 0x7f4a9c0b4bc0 2024-04-25T10:14:12.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4a997fa640 1 -- 172.21.15.60:0/2618744531 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9401a070 con 0x7f4a9c0b4bc0 2024-04-25T10:14:12.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 -- 172.21.15.60:0/2618744531 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9c0c56d0 con 0x7f4a9c0b4bc0 2024-04-25T10:14:12.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4a997fa640 1 -- 172.21.15.60:0/2618744531 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9400faa0 con 0x7f4a9c0b4bc0 2024-04-25T10:14:12.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.911+0000 7f4aa3122640 1 -- 172.21.15.60:0/2618744531 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9c0c5c90 con 0x7f4a9c0b4bc0 2024-04-25T10:14:12.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.915+0000 7f4aa3122640 1 -- 172.21.15.60:0/2618744531 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a9c074770 con 0x7f4a9c0b4bc0 2024-04-25T10:14:12.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.915+0000 7f4a997fa640 1 -- 172.21.15.60:0/2618744531 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a9400fc40 con 0x7f4a9c0b4bc0 2024-04-25T10:14:12.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.915+0000 7f4a997fa640 1 --2- 172.21.15.60:0/2618744531 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a80066f60 0x7f4a80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.915+0000 7f4a997fa640 1 -- 172.21.15.60:0/2618744531 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4a94087c10 con 0x7f4a9c0b4bc0 2024-04-25T10:14:12.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.915+0000 7f4aa0ebe640 1 --2- 172.21.15.60:0/2618744531 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a80066f60 0x7f4a80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.915+0000 7f4aa0ebe640 1 --2- 172.21.15.60:0/2618744531 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a80066f60 0x7f4a80069420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009b70 tx=0x7f4a90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:12.919+0000 7f4a997fa640 1 -- 172.21.15.60:0/2618744531 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4a94059000 con 0x7f4a9c0b4bc0 2024-04-25T10:14:13.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.135+0000 7f4aa3122640 1 -- 172.21.15.60:0/2618744531 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4a9c0b7730 con 0x7f4a9c0b4bc0 2024-04-25T10:14:13.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.139+0000 7f4a997fa640 1 -- 172.21.15.60:0/2618744531 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4a94058900 con 0x7f4a9c0b4bc0 2024-04-25T10:14:13.143 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:13.143 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:14:13.143 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-04-25T10:14:13.143 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.143 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-04-25T10:14:13.143 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmKr12EhAAnHHZ5VjG0cywMrIv6VElVg==", 2024-04-25T10:14:13.143 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.143 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:14:13.143 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:14:13.144 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:14:13.144 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:14:13.144 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.144 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.144 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.144 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-04-25T10:14:13.144 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmU39rIRAAk6Ca0tKG4LpmR4pNxjzv0w==", 2024-04-25T10:14:13.144 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.144 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:14:13.144 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:14:13.144 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:14:13.144 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:14:13.144 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.145 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.145 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.145 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-04-25T10:14:13.145 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmVsshGxAAttXv0jYEGKgo0sPhIRTuqQ==", 2024-04-25T10:14:13.145 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.145 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:14:13.145 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:14:13.145 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:14:13.145 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:14:13.145 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.145 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.145 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.145 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-04-25T10:14:13.145 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmQtXUKRAAI3Iq02h3KH/PZbWKvJD9Jg==", 2024-04-25T10:14:13.146 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.146 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:14:13.146 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:14:13.146 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:14:13.146 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:14:13.146 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.146 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.146 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.146 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-04-25T10:14:13.146 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCkKipmE+UIBRAAc8ctpHoL3PkkbkMRCAJUGw==", 2024-04-25T10:14:13.146 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.146 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:13.146 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:14:13.146 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:14:13.147 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.147 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.147 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-04-25T10:14:13.147 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCnKipmbAnYCBAALLsPH2FjUpjoIZwTw+8aOA==", 2024-04-25T10:14:13.147 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.147 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:13.147 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:14:13.147 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:14:13.147 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.147 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.147 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-04-25T10:14:13.147 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCqKipmqrNSDxAAZdgFrcyeVwRQMKJJYIM98g==", 2024-04-25T10:14:13.147 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.147 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:13.148 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:14:13.148 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:14:13.148 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.148 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.148 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-04-25T10:14:13.148 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCtKipm1L+hFxAAPnPWlCm1Fmi0mN3tyReeQw==", 2024-04-25T10:14:13.148 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.148 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:13.148 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:14:13.148 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:14:13.148 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.148 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.148 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-04-25T10:14:13.149 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCzKipm2WyDFRAAmyfg6qu5drE9yMaR2FKd/A==", 2024-04-25T10:14:13.149 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.149 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:13.149 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:14:13.149 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:14:13.149 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.149 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.149 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-04-25T10:14:13.149 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC2KipmZLDhGRAAcgaz8YaPGUMIfxOLtWObyA==", 2024-04-25T10:14:13.149 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.149 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:13.149 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:14:13.149 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:14:13.150 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.150 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.150 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-04-25T10:14:13.150 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC5Kipm/3VMIBAA+rhGCyA/7d+s8m7ZWTzX/Q==", 2024-04-25T10:14:13.150 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.150 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:13.150 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:14:13.150 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:14:13.150 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.150 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.150 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-04-25T10:14:13.150 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8KipmFNCCJxAAv0OSTpAG+xIHhLqYXAZazQ==", 2024-04-25T10:14:13.150 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.151 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:13.151 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:14:13.151 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:14:13.151 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.151 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-04-25T10:14:13.151 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmv65dLxAA6/QpPrqE9aT+CtZqkyutog==", 2024-04-25T10:14:13.151 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.151 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:14:13.151 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-04-25T10:14:13.152 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:14:13.152 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.152 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.152 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-04-25T10:14:13.152 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipm2NiONxAAd3X4zx1JnBXKHJvdZ/zMdQ==", 2024-04-25T10:14:13.152 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.152 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:14:13.152 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:14:13.152 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:14:13.152 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:14:13.152 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmZRONNBAAa+XTv5YIBJWeUvf5dfoWxw==", 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCjKipmgvfiABAApLvTC9U6HzeAefCvX4tiSQ==", 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:14:13.154 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:14:13.154 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:14:13.154 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:14:13.154 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.154 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-04-25T10:14:13.154 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQChKipmumzYNhAAlCRYMpSNaTo7bz8P6x0W9g==", 2024-04-25T10:14:13.154 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.154 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:14:13.154 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:14:13.154 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:14:13.154 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:14:13.155 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.155 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-04-25T10:14:13.155 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmhxlLDxAAegBrJWdMNz6ZC3oXT2Qo0A==", 2024-04-25T10:14:13.155 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.155 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:14:13.155 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T10:14:13.155 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:14:13.155 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:13.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:13.155 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-04-25T10:14:13.156 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmpl1cDBAAGqx2kIak9NoWpkcdHczcFA==", 2024-04-25T10:14:13.156 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:14:13.156 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:14:13.156 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T10:14:13.156 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:14:13.156 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.156 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:13.156 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T10:14:13.156 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:14:13.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.139+0000 7f4aa3122640 1 -- 172.21.15.60:0/2618744531 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a80066f60 msgr2=0x7f4a80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.139+0000 7f4aa3122640 1 --2- 172.21.15.60:0/2618744531 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a80066f60 0x7f4a80069420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009b70 tx=0x7f4a90008040 comp rx=0 tx=0).stop 2024-04-25T10:14:13.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.139+0000 7f4aa3122640 1 -- 172.21.15.60:0/2618744531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c0b4bc0 msgr2=0x7f4a9c1564b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.139+0000 7f4aa3122640 1 --2- 172.21.15.60:0/2618744531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c0b4bc0 0x7f4a9c1564b0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c072680 tx=0x7f4a94011c20 comp rx=0 tx=0).stop 2024-04-25T10:14:13.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.139+0000 7f4aa3122640 1 -- 172.21.15.60:0/2618744531 shutdown_connections 2024-04-25T10:14:13.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.139+0000 7f4aa3122640 1 --2- 172.21.15.60:0/2618744531 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a80066f60 0x7f4a80069420 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.139+0000 7f4aa3122640 1 --2- 172.21.15.60:0/2618744531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c0b4bc0 0x7f4a9c1564b0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.139+0000 7f4aa3122640 1 --2- 172.21.15.60:0/2618744531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9c0b4740 0x7f4a9c155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.139+0000 7f4aa3122640 1 --2- 172.21.15.60:0/2618744531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a8c004810 0x7f4a9c154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.139+0000 7f4aa3122640 1 -- 172.21.15.60:0/2618744531 >> 172.21.15.60:0/2618744531 conn(0x7f4a9c0b32a0 msgr2=0x7f4a9c070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.139+0000 7f4aa3122640 1 -- 172.21.15.60:0/2618744531 shutdown_connections 2024-04-25T10:14:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.139+0000 7f4aa3122640 1 -- 172.21.15.60:0/2618744531 wait complete. 2024-04-25T10:14:13.163 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:13.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f950f15f640 1 Processor -- start 2024-04-25T10:14:13.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f950f15f640 1 -- start start 2024-04-25T10:14:13.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f950f15f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f950815deb0 0x7f95080b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f950f15f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f950806d200 con 0x7f95080d0340 2024-04-25T10:14:13.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f950f15f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95080cefe0 con 0x7f95080ce9c0 2024-04-25T10:14:13.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f950f15f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95080cf2c0 con 0x7f950815deb0 2024-04-25T10:14:13.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f9507fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f95080d0340 0x7f95080ce2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45900/0 (socket says 172.21.15.60:45900) 2024-04-25T10:14:13.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f9507fff640 1 -- 172.21.15.60:0/1159340772 learned_addr learned my addr 172.21.15.60:0/1159340772 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:13.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f950cefb640 1 --2- 172.21.15.60:0/1159340772 >> v2:172.21.15.100:3300/0 conn(0x7f950815deb0 0x7f95080b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f95077fe640 1 -- 172.21.15.60:0/1159340772 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679234812 0 0) 0x7f950806d200 con 0x7f95080d0340 2024-04-25T10:14:13.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f95077fe640 1 -- 172.21.15.60:0/1159340772 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94f8003530 con 0x7f95080d0340 2024-04-25T10:14:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f95077fe640 1 -- 172.21.15.60:0/1159340772 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1565299822 0 0) 0x7f94f8003530 con 0x7f95080d0340 2024-04-25T10:14:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f95077fe640 1 -- 172.21.15.60:0/1159340772 >> v2:172.21.15.100:3300/0 conn(0x7f950815deb0 msgr2=0x7f95080b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f95077fe640 1 --2- 172.21.15.60:0/1159340772 >> v2:172.21.15.100:3300/0 conn(0x7f950815deb0 0x7f95080b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f95077fe640 1 -- 172.21.15.60:0/1159340772 >> v1:172.21.15.100:6789/0 conn(0x7f95080ce9c0 legacy=0x7f95080d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f95077fe640 1 -- 172.21.15.60:0/1159340772 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95080d1d70 con 0x7f95080d0340 2024-04-25T10:14:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f95077fe640 1 -- 172.21.15.60:0/1159340772 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f94f4001680 con 0x7f95080d0340 2024-04-25T10:14:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f95077fe640 1 -- 172.21.15.60:0/1159340772 >> v1:172.21.15.60:6789/0 conn(0x7f95080d0340 legacy=0x7f95080ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f95077fe640 1 --2- 172.21.15.60:0/1159340772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f80039c0 0x7f94f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f95077fe640 1 --2- 172.21.15.60:0/1159340772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94f8004810 0x7f94f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f95077fe640 1 --2- 172.21.15.60:0/1159340772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94f80051d0 0x7f94f800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f95077fe640 1 -- 172.21.15.60:0/1159340772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95080cf2c0 con 0x7f94f80051d0 2024-04-25T10:14:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f95077fe640 1 -- 172.21.15.60:0/1159340772 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f800c070 con 0x7f94f80039c0 2024-04-25T10:14:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f95077fe640 1 -- 172.21.15.60:0/1159340772 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f800c270 con 0x7f94f8004810 2024-04-25T10:14:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f950d6fc640 1 --2- 172.21.15.60:0/1159340772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94f80051d0 0x7f94f800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f950cefb640 1 --2- 172.21.15.60:0/1159340772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f80039c0 0x7f94f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f9507fff640 1 --2- 172.21.15.60:0/1159340772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94f8004810 0x7f94f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f950d6fc640 1 -- 172.21.15.60:0/1159340772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94f8004810 msgr2=0x7f94f8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f950d6fc640 1 --2- 172.21.15.60:0/1159340772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94f8004810 0x7f94f8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f950d6fc640 1 -- 172.21.15.60:0/1159340772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f80039c0 msgr2=0x7f94f8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f950d6fc640 1 --2- 172.21.15.60:0/1159340772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94f80039c0 0x7f94f8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.279+0000 7f950d6fc640 1 -- 172.21.15.60:0/1159340772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f8003530 con 0x7f94f80051d0 2024-04-25T10:14:13.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950d6fc640 1 -- 172.21.15.60:0/1159340772 reap_dead start 2024-04-25T10:14:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950d6fc640 1 --2- 172.21.15.60:0/1159340772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94f80051d0 0x7f94f800b9d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9500001e70 tx=0x7f950001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7eb9088b511b30b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f95077fe640 1 -- 172.21.15.60:0/1159340772 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500001a90 con 0x7f94f80051d0 2024-04-25T10:14:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f95077fe640 1 -- 172.21.15.60:0/1159340772 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9500003bc0 con 0x7f94f80051d0 2024-04-25T10:14:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f95077fe640 1 -- 172.21.15.60:0/1159340772 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500005020 con 0x7f94f80051d0 2024-04-25T10:14:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950f15f640 1 -- 172.21.15.60:0/1159340772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94f80051d0 msgr2=0x7f94f800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950f15f640 1 --2- 172.21.15.60:0/1159340772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94f80051d0 0x7f94f800b9d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9500001e70 tx=0x7f950001c780 comp rx=0 tx=0).stop 2024-04-25T10:14:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950f15f640 1 -- 172.21.15.60:0/1159340772 shutdown_connections 2024-04-25T10:14:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950f15f640 1 --2- 172.21.15.60:0/1159340772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94f80051d0 0x7f94f800b9d0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950f15f640 1 -- 172.21.15.60:0/1159340772 >> 172.21.15.60:0/1159340772 conn(0x7f950806c090 msgr2=0x7f950806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950f15f640 1 -- 172.21.15.60:0/1159340772 shutdown_connections 2024-04-25T10:14:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950f15f640 1 -- 172.21.15.60:0/1159340772 wait complete. 2024-04-25T10:14:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950f15f640 1 Processor -- start 2024-04-25T10:14:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950f15f640 1 -- start start 2024-04-25T10:14:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950f15f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94f80051d0 0x7f95081f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950f15f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95081f6af0 0x7f95081f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950f15f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f950815f1f0 0x7f950815f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950f15f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95080d2330 con 0x7f95081f6af0 2024-04-25T10:14:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950f15f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95080cee30 con 0x7f950815f1f0 2024-04-25T10:14:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f950f15f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95080b6c10 con 0x7f94f80051d0 2024-04-25T10:14:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f9507fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95081f6af0 0x7f95081f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f9507fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95081f6af0 0x7f95081f6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38426/0 (socket says 172.21.15.60:38426) 2024-04-25T10:14:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.283+0000 7f9507fff640 1 -- 172.21.15.60:0/2254187639 learned_addr learned my addr 172.21.15.60:0/2254187639 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f950cefb640 1 --2- 172.21.15.60:0/2254187639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94f80051d0 0x7f95081f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f950d6fc640 1 --2- 172.21.15.60:0/2254187639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f950815f1f0 0x7f950815f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f9507fff640 1 -- 172.21.15.60:0/2254187639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94f80051d0 msgr2=0x7f95081f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f9507fff640 1 --2- 172.21.15.60:0/2254187639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94f80051d0 0x7f95081f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f9507fff640 1 -- 172.21.15.60:0/2254187639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f950815f1f0 msgr2=0x7f950815f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f9507fff640 1 --2- 172.21.15.60:0/2254187639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f950815f1f0 0x7f950815f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f9507fff640 1 -- 172.21.15.60:0/2254187639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950815fbb0 con 0x7f95081f6af0 2024-04-25T10:14:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f9507fff640 1 --2- 172.21.15.60:0/2254187639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95081f6af0 0x7f95081f6f70 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f94f400e780 tx=0x7f94f400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f95057fa640 1 -- 172.21.15.60:0/2254187639 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f400ee90 con 0x7f95081f6af0 2024-04-25T10:14:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f95057fa640 1 -- 172.21.15.60:0/2254187639 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f4018070 con 0x7f95081f6af0 2024-04-25T10:14:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f950f15f640 1 -- 172.21.15.60:0/2254187639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f950815e430 con 0x7f95081f6af0 2024-04-25T10:14:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f95057fa640 1 -- 172.21.15.60:0/2254187639 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f400ca90 con 0x7f95081f6af0 2024-04-25T10:14:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f950f15f640 1 -- 172.21.15.60:0/2254187639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f950815e9f0 con 0x7f95081f6af0 2024-04-25T10:14:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f950f15f640 1 -- 172.21.15.60:0/2254187639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9508074770 con 0x7f95081f6af0 2024-04-25T10:14:13.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f95057fa640 1 -- 172.21.15.60:0/2254187639 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f94f400cc30 con 0x7f95081f6af0 2024-04-25T10:14:13.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f95057fa640 1 --2- 172.21.15.60:0/2254187639 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f94ec067100 0x7f94ec0695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f95057fa640 1 -- 172.21.15.60:0/2254187639 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f94f4084a40 con 0x7f95081f6af0 2024-04-25T10:14:13.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.287+0000 7f950cefb640 1 --2- 172.21.15.60:0/2254187639 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f94ec067100 0x7f94ec0695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.291+0000 7f950cefb640 1 --2- 172.21.15.60:0/2254187639 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f94ec067100 0x7f94ec0695c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9508155a90 tx=0x7f94fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.291+0000 7f95057fa640 1 -- 172.21.15.60:0/2254187639 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f94f4055750 con 0x7f95081f6af0 2024-04-25T10:14:13.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.451+0000 7f950f15f640 1 -- 172.21.15.60:0/2254187639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95080d17a0 con 0x7f95081f6af0 2024-04-25T10:14:13.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.455+0000 7f95057fa640 1 -- 172.21.15.60:0/2254187639 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+12904 (secure 0 0 0) 0x7f94f4084e20 con 0x7f95081f6af0 2024-04-25T10:14:13.458 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:13.459 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:14:12.771718+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.455+0000 7f950f15f640 1 -- 172.21.15.60:0/2254187639 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f94ec067100 msgr2=0x7f94ec0695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.455+0000 7f950f15f640 1 --2- 172.21.15.60:0/2254187639 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f94ec067100 0x7f94ec0695c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9508155a90 tx=0x7f94fc008040 comp rx=0 tx=0).stop 2024-04-25T10:14:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.455+0000 7f950f15f640 1 -- 172.21.15.60:0/2254187639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95081f6af0 msgr2=0x7f95081f6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.455+0000 7f950f15f640 1 --2- 172.21.15.60:0/2254187639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95081f6af0 0x7f95081f6f70 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f94f400e780 tx=0x7f94f400ec40 comp rx=0 tx=0).stop 2024-04-25T10:14:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.455+0000 7f950f15f640 1 -- 172.21.15.60:0/2254187639 shutdown_connections 2024-04-25T10:14:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.455+0000 7f950f15f640 1 --2- 172.21.15.60:0/2254187639 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f94ec067100 0x7f94ec0695c0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.455+0000 7f950f15f640 1 --2- 172.21.15.60:0/2254187639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f950815f1f0 0x7f950815f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.455+0000 7f950f15f640 1 --2- 172.21.15.60:0/2254187639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95081f6af0 0x7f95081f6f70 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.455+0000 7f950f15f640 1 --2- 172.21.15.60:0/2254187639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94f80051d0 0x7f95081f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.455+0000 7f950f15f640 1 -- 172.21.15.60:0/2254187639 >> 172.21.15.60:0/2254187639 conn(0x7f950806c090 msgr2=0x7f95080c7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.455+0000 7f950f15f640 1 -- 172.21.15.60:0/2254187639 shutdown_connections 2024-04-25T10:14:13.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.455+0000 7f950f15f640 1 -- 172.21.15.60:0/2254187639 wait complete. 2024-04-25T10:14:13.477 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:13.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:13.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.591+0000 7f7832b3b640 1 Processor -- start 2024-04-25T10:14:13.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.591+0000 7f7832b3b640 1 -- start start 2024-04-25T10:14:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.591+0000 7f7832b3b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f782c0cfaa0 0x7f782c1654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.591+0000 7f7832b3b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782c15e410 con 0x7f782c0b2560 2024-04-25T10:14:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.591+0000 7f7832b3b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782c15e6f0 con 0x7f782c154c10 2024-04-25T10:14:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.591+0000 7f7832b3b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f782c15e9d0 con 0x7f782c0cfaa0 2024-04-25T10:14:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.591+0000 7f78310d8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f782c0cfaa0 0x7f782c1654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.591+0000 7f782bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f782c0b2560 0x7f782c0cf390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45906/0 (socket says 172.21.15.60:45906) 2024-04-25T10:14:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.591+0000 7f782bfff640 1 -- 172.21.15.60:0/205629802 learned_addr learned my addr 172.21.15.60:0/205629802 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.591+0000 7f78310d8640 1 -- 172.21.15.60:0/205629802 >> v1:172.21.15.100:6789/0 conn(0x7f782c154c10 legacy=0x7f782c0b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.591+0000 7f78310d8640 1 -- 172.21.15.60:0/205629802 >> v1:172.21.15.60:6789/0 conn(0x7f782c0b2560 legacy=0x7f782c0cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.591+0000 7f78310d8640 1 -- 172.21.15.60:0/205629802 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c15ecb0 con 0x7f782c0cfaa0 2024-04-25T10:14:13.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f78310d8640 1 --2- 172.21.15.60:0/205629802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f782c0cfaa0 0x7f782c1654d0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f7824009d90 tx=0x7f782401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2832980b841e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f782b7fe640 1 -- 172.21.15.60:0/205629802 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78240b2070 con 0x7f782c0cfaa0 2024-04-25T10:14:13.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f782b7fe640 1 -- 172.21.15.60:0/205629802 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78240a5870 con 0x7f782c0cfaa0 2024-04-25T10:14:13.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f782b7fe640 1 -- 172.21.15.60:0/205629802 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78240a5c40 con 0x7f782c0cfaa0 2024-04-25T10:14:13.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f7832b3b640 1 -- 172.21.15.60:0/205629802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f782c0cfaa0 msgr2=0x7f782c1654d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f7832b3b640 1 --2- 172.21.15.60:0/205629802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f782c0cfaa0 0x7f782c1654d0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f7824009d90 tx=0x7f782401c8a0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f7832b3b640 1 -- 172.21.15.60:0/205629802 shutdown_connections 2024-04-25T10:14:13.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f7832b3b640 1 --2- 172.21.15.60:0/205629802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f782c0cfaa0 0x7f782c1654d0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f7832b3b640 1 -- 172.21.15.60:0/205629802 >> 172.21.15.60:0/205629802 conn(0x7f782c0670b0 msgr2=0x7f782c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f7832b3b640 1 -- 172.21.15.60:0/205629802 shutdown_connections 2024-04-25T10:14:13.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f7832b3b640 1 -- 172.21.15.60:0/205629802 wait complete. 2024-04-25T10:14:13.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f7832b3b640 1 Processor -- start 2024-04-25T10:14:13.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f7832b3b640 1 -- start start 2024-04-25T10:14:13.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f7832b3b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f782c0b2560 0x7f782c155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f7832b3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f782c0cfaa0 0x7f782c1431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f7832b3b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f782c154c10 0x7f782c157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f7832b3b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c168640 con 0x7f782c0cfaa0 2024-04-25T10:14:13.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f7832b3b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c155200 con 0x7f782c154c10 2024-04-25T10:14:13.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f7832b3b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782c1687c0 con 0x7f782c0b2560 2024-04-25T10:14:13.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f782bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f782c0cfaa0 0x7f782c1431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f782bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f782c0cfaa0 0x7f782c1431f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38446/0 (socket says 172.21.15.60:38446) 2024-04-25T10:14:13.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f782bfff640 1 -- 172.21.15.60:0/605376164 learned_addr learned my addr 172.21.15.60:0/605376164 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:13.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f78310d8640 1 --2- 172.21.15.60:0/605376164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f782c154c10 0x7f782c157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f78308d7640 1 --2- 172.21.15.60:0/605376164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f782c0b2560 0x7f782c155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f782bfff640 1 -- 172.21.15.60:0/605376164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f782c0b2560 msgr2=0x7f782c155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f782bfff640 1 --2- 172.21.15.60:0/605376164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f782c0b2560 0x7f782c155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f782bfff640 1 -- 172.21.15.60:0/605376164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f782c154c10 msgr2=0x7f782c157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f782bfff640 1 --2- 172.21.15.60:0/605376164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f782c154c10 0x7f782c157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.595+0000 7f782bfff640 1 -- 172.21.15.60:0/605376164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c1522d0 con 0x7f782c0cfaa0 2024-04-25T10:14:13.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.599+0000 7f782bfff640 1 --2- 172.21.15.60:0/605376164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f782c0cfaa0 0x7f782c1431f0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f782000c420 tx=0x7f782000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.599+0000 7f78297fa640 1 -- 172.21.15.60:0/605376164 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820017070 con 0x7f782c0cfaa0 2024-04-25T10:14:13.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.599+0000 7f78297fa640 1 -- 172.21.15.60:0/605376164 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7820005cf0 con 0x7f782c0cfaa0 2024-04-25T10:14:13.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.599+0000 7f7832b3b640 1 -- 172.21.15.60:0/605376164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782c1525b0 con 0x7f782c0cfaa0 2024-04-25T10:14:13.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.599+0000 7f78297fa640 1 -- 172.21.15.60:0/605376164 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78200072f0 con 0x7f782c0cfaa0 2024-04-25T10:14:13.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.599+0000 7f7832b3b640 1 -- 172.21.15.60:0/605376164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782c152b70 con 0x7f782c0cfaa0 2024-04-25T10:14:13.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.599+0000 7f7832b3b640 1 -- 172.21.15.60:0/605376164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f782c06bba0 con 0x7f782c0cfaa0 2024-04-25T10:14:13.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.599+0000 7f78297fa640 1 -- 172.21.15.60:0/605376164 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f782001a070 con 0x7f782c0cfaa0 2024-04-25T10:14:13.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.599+0000 7f78297fa640 1 --2- 172.21.15.60:0/605376164 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7818066f60 0x7f7818069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.599+0000 7f78297fa640 1 -- 172.21.15.60:0/605376164 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7820084090 con 0x7f782c0cfaa0 2024-04-25T10:14:13.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.599+0000 7f78308d7640 1 --2- 172.21.15.60:0/605376164 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7818066f60 0x7f7818069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.599+0000 7f78308d7640 1 --2- 172.21.15.60:0/605376164 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7818066f60 0x7f7818069420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f781c00a750 tx=0x7f781c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.603+0000 7f78297fa640 1 -- 172.21.15.60:0/605376164 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7820055400 con 0x7f782c0cfaa0 2024-04-25T10:14:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.767+0000 7f7832b3b640 1 -- 172.21.15.60:0/605376164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f782c0b2fc0 con 0x7f782c0cfaa0 2024-04-25T10:14:13.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.767+0000 7f78297fa640 1 -- 172.21.15.60:0/605376164 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7820054d00 con 0x7f782c0cfaa0 2024-04-25T10:14:13.771 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:14:13.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.767+0000 7f7832b3b640 1 -- 172.21.15.60:0/605376164 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7818066f60 msgr2=0x7f7818069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.767+0000 7f7832b3b640 1 --2- 172.21.15.60:0/605376164 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7818066f60 0x7f7818069420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f781c00a750 tx=0x7f781c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.767+0000 7f7832b3b640 1 -- 172.21.15.60:0/605376164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f782c0cfaa0 msgr2=0x7f782c1431f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.767+0000 7f7832b3b640 1 --2- 172.21.15.60:0/605376164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f782c0cfaa0 0x7f782c1431f0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f782000c420 tx=0x7f782000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.767+0000 7f7832b3b640 1 -- 172.21.15.60:0/605376164 shutdown_connections 2024-04-25T10:14:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.767+0000 7f7832b3b640 1 --2- 172.21.15.60:0/605376164 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7818066f60 0x7f7818069420 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.767+0000 7f7832b3b640 1 --2- 172.21.15.60:0/605376164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f782c154c10 0x7f782c157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.767+0000 7f7832b3b640 1 --2- 172.21.15.60:0/605376164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f782c0cfaa0 0x7f782c1431f0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.767+0000 7f7832b3b640 1 --2- 172.21.15.60:0/605376164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f782c0b2560 0x7f782c155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.767+0000 7f7832b3b640 1 -- 172.21.15.60:0/605376164 >> 172.21.15.60:0/605376164 conn(0x7f782c0670b0 msgr2=0x7f782c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.767+0000 7f7832b3b640 1 -- 172.21.15.60:0/605376164 shutdown_connections 2024-04-25T10:14:13.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.767+0000 7f7832b3b640 1 -- 172.21.15.60:0/605376164 wait complete. 2024-04-25T10:14:13.790 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:14:13.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.911+0000 7f38428e0640 1 Processor -- start 2024-04-25T10:14:13.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.911+0000 7f38428e0640 1 -- start start 2024-04-25T10:14:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.911+0000 7f38428e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383c0cdc30 0x7f383c06ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.911+0000 7f38428e0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f383c0cff00 0x7f383c0cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.911+0000 7f38428e0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f383c0d03b0 con 0x7f383c0cfae0 2024-04-25T10:14:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.911+0000 7f38428e0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f383c0d0690 con 0x7f383c0cff00 2024-04-25T10:14:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.911+0000 7f38428e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383c0d0970 con 0x7f383c0cdc30 2024-04-25T10:14:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.911+0000 7f3840e7d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f383c0cff00 0x7f383c0cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.911+0000 7f3840e7d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f383c0cff00 0x7f383c0cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38454/0 (socket says 172.21.15.60:38454) 2024-04-25T10:14:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.911+0000 7f3840e7d640 1 -- 172.21.15.60:0/1124749884 learned_addr learned my addr 172.21.15.60:0/1124749884 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.911+0000 7f383bfff640 1 --2- 172.21.15.60:0/1124749884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383c0cdc30 0x7f383c06ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.911+0000 7f3840e7d640 1 -- 172.21.15.60:0/1124749884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383c0cdc30 msgr2=0x7f383c06ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.911+0000 7f3840e7d640 1 --2- 172.21.15.60:0/1124749884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383c0cdc30 0x7f383c06ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.911+0000 7f3840e7d640 1 -- 172.21.15.60:0/1124749884 >> v1:172.21.15.100:6789/0 conn(0x7f383c0cfae0 legacy=0x7f383c099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.911+0000 7f3840e7d640 1 -- 172.21.15.60:0/1124749884 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383c0d0c50 con 0x7f383c0cff00 2024-04-25T10:14:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f3840e7d640 1 --2- 172.21.15.60:0/1124749884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c0cff00 0x7f383c0cbe00 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f3834009e50 tx=0x7f383401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da65c3ed34b9956 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f383affd640 1 -- 172.21.15.60:0/1124749884 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38340a5700 con 0x7f383c0cff00 2024-04-25T10:14:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f383affd640 1 -- 172.21.15.60:0/1124749884 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38340a58a0 con 0x7f383c0cff00 2024-04-25T10:14:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f383affd640 1 -- 172.21.15.60:0/1124749884 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38340a5c70 con 0x7f383c0cff00 2024-04-25T10:14:13.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f38428e0640 1 -- 172.21.15.60:0/1124749884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c0cff00 msgr2=0x7f383c0cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f38428e0640 1 --2- 172.21.15.60:0/1124749884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c0cff00 0x7f383c0cbe00 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f3834009e50 tx=0x7f383401ca30 comp rx=0 tx=0).stop 2024-04-25T10:14:13.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f38428e0640 1 -- 172.21.15.60:0/1124749884 shutdown_connections 2024-04-25T10:14:13.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f38428e0640 1 --2- 172.21.15.60:0/1124749884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c0cff00 0x7f383c0cbe00 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f38428e0640 1 --2- 172.21.15.60:0/1124749884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383c0cdc30 0x7f383c06ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f38428e0640 1 -- 172.21.15.60:0/1124749884 >> 172.21.15.60:0/1124749884 conn(0x7f383c0b32a0 msgr2=0x7f383c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f38428e0640 1 -- 172.21.15.60:0/1124749884 shutdown_connections 2024-04-25T10:14:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f38428e0640 1 -- 172.21.15.60:0/1124749884 wait complete. 2024-04-25T10:14:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f38428e0640 1 Processor -- start 2024-04-25T10:14:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f38428e0640 1 -- start start 2024-04-25T10:14:13.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f38428e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c0cdc30 0x7f383c14dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f38428e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f383c0cfae0 0x7f383c09cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f38428e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383c0cff00 0x7f383c09b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f38428e0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383c0ccb50 con 0x7f383c0cdc30 2024-04-25T10:14:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f38428e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383c09a350 con 0x7f383c0cfae0 2024-04-25T10:14:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f38428e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383c0cccd0 con 0x7f383c0cff00 2024-04-25T10:14:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f383bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c0cdc30 0x7f383c14dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f383bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c0cdc30 0x7f383c14dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38464/0 (socket says 172.21.15.60:38464) 2024-04-25T10:14:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f383bfff640 1 -- 172.21.15.60:0/4102062583 learned_addr learned my addr 172.21.15.60:0/4102062583 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f383b7fe640 1 --2- 172.21.15.60:0/4102062583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f383c0cfae0 0x7f383c09cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.915+0000 7f3840e7d640 1 --2- 172.21.15.60:0/4102062583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383c0cff00 0x7f383c09b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f383bfff640 1 -- 172.21.15.60:0/4102062583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383c0cff00 msgr2=0x7f383c09b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f383bfff640 1 --2- 172.21.15.60:0/4102062583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383c0cff00 0x7f383c09b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f383bfff640 1 -- 172.21.15.60:0/4102062583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f383c0cfae0 msgr2=0x7f383c09cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f383bfff640 1 --2- 172.21.15.60:0/4102062583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f383c0cfae0 0x7f383c09cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f383bfff640 1 -- 172.21.15.60:0/4102062583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383c09be40 con 0x7f383c0cdc30 2024-04-25T10:14:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f383bfff640 1 --2- 172.21.15.60:0/4102062583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c0cdc30 0x7f383c14dce0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f382c00d800 tx=0x7f382c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f3838ff9640 1 -- 172.21.15.60:0/4102062583 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c00b6c0 con 0x7f383c0cdc30 2024-04-25T10:14:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f3838ff9640 1 -- 172.21.15.60:0/4102062583 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c019070 con 0x7f383c0cdc30 2024-04-25T10:14:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f38428e0640 1 -- 172.21.15.60:0/4102062583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f383c09c120 con 0x7f383c0cdc30 2024-04-25T10:14:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f3838ff9640 1 -- 172.21.15.60:0/4102062583 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c00ba60 con 0x7f383c0cdc30 2024-04-25T10:14:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f38428e0640 1 -- 172.21.15.60:0/4102062583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f383c09c6e0 con 0x7f383c0cdc30 2024-04-25T10:14:13.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f38428e0640 1 -- 172.21.15.60:0/4102062583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f383c074770 con 0x7f383c0cdc30 2024-04-25T10:14:13.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f3838ff9640 1 -- 172.21.15.60:0/4102062583 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f382c00bc00 con 0x7f383c0cdc30 2024-04-25T10:14:13.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f3838ff9640 1 --2- 172.21.15.60:0/4102062583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3828066f60 0x7f3828069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f3838ff9640 1 -- 172.21.15.60:0/4102062583 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f382c085120 con 0x7f383c0cdc30 2024-04-25T10:14:13.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f383b7fe640 1 --2- 172.21.15.60:0/4102062583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3828066f60 0x7f3828069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.919+0000 7f383b7fe640 1 --2- 172.21.15.60:0/4102062583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3828066f60 0x7f3828069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f383c0b76e0 tx=0x7f3830009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:13.923+0000 7f3838ff9640 1 -- 172.21.15.60:0/4102062583 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f382c056480 con 0x7f383c0cdc30 2024-04-25T10:14:14.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.079+0000 7f38428e0640 1 -- 172.21.15.60:0/4102062583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f383c0b7730 con 0x7f383c0cdc30 2024-04-25T10:14:14.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.079+0000 7f3838ff9640 1 -- 172.21.15.60:0/4102062583 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v93) v1 ==== 84+0+65 (secure 0 0 0) 0x7f382c055d80 con 0x7f383c0cdc30 2024-04-25T10:14:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:14.085 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T10:14:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:14:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T10:14:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T10:14:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:14.085 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T10:14:14.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.083+0000 7f38428e0640 1 -- 172.21.15.60:0/4102062583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3828066f60 msgr2=0x7f3828069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.083+0000 7f38428e0640 1 --2- 172.21.15.60:0/4102062583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3828066f60 0x7f3828069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f383c0b76e0 tx=0x7f3830009040 comp rx=0 tx=0).stop 2024-04-25T10:14:14.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.083+0000 7f38428e0640 1 -- 172.21.15.60:0/4102062583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c0cdc30 msgr2=0x7f383c14dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.083+0000 7f38428e0640 1 --2- 172.21.15.60:0/4102062583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c0cdc30 0x7f383c14dce0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f382c00d800 tx=0x7f382c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.083+0000 7f38428e0640 1 -- 172.21.15.60:0/4102062583 shutdown_connections 2024-04-25T10:14:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.083+0000 7f38428e0640 1 --2- 172.21.15.60:0/4102062583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3828066f60 0x7f3828069420 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.083+0000 7f38428e0640 1 --2- 172.21.15.60:0/4102062583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383c0cff00 0x7f383c09b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.083+0000 7f38428e0640 1 --2- 172.21.15.60:0/4102062583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f383c0cfae0 0x7f383c09cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.083+0000 7f38428e0640 1 --2- 172.21.15.60:0/4102062583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c0cdc30 0x7f383c14dce0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.083+0000 7f38428e0640 1 -- 172.21.15.60:0/4102062583 >> 172.21.15.60:0/4102062583 conn(0x7f383c0b32a0 msgr2=0x7f383c06f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.083+0000 7f38428e0640 1 -- 172.21.15.60:0/4102062583 shutdown_connections 2024-04-25T10:14:14.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.083+0000 7f38428e0640 1 -- 172.21.15.60:0/4102062583 wait complete. 2024-04-25T10:14:14.102 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:14:14.102 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:14:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.223+0000 7fbbe0392640 1 Processor -- start 2024-04-25T10:14:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.223+0000 7fbbe0392640 1 -- start start 2024-04-25T10:14:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.223+0000 7fbbe0392640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbd80ad290 0x7fbbd8161c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.223+0000 7fbbe0392640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd815e6d0 con 0x7fbbd8154c20 2024-04-25T10:14:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.223+0000 7fbbe0392640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd815e9b0 con 0x7fbbd815e0c0 2024-04-25T10:14:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.223+0000 7fbbe0392640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd815ec90 con 0x7fbbd80ad290 2024-04-25T10:14:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.223+0000 7fbbde12e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbbd8154c20 0x7fbbd80aead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45910/0 (socket says 172.21.15.60:45910) 2024-04-25T10:14:14.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.223+0000 7fbbde12e640 1 -- 172.21.15.60:0/262197218 learned_addr learned my addr 172.21.15.60:0/262197218 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:14.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.223+0000 7fbbde92f640 1 --2- 172.21.15.60:0/262197218 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbd80ad290 0x7fbbd8161c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.223+0000 7fbbdd12c640 1 -- 172.21.15.60:0/262197218 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552710754 0 0) 0x7fbbd815e6d0 con 0x7fbbd8154c20 2024-04-25T10:14:14.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.223+0000 7fbbdd12c640 1 -- 172.21.15.60:0/262197218 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbc0003530 con 0x7fbbd8154c20 2024-04-25T10:14:14.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd12c640 1 -- 172.21.15.60:0/262197218 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3171064092 0 0) 0x7fbbc0003530 con 0x7fbbd8154c20 2024-04-25T10:14:14.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd12c640 1 -- 172.21.15.60:0/262197218 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbd80ad290 msgr2=0x7fbbd8161c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd12c640 1 --2- 172.21.15.60:0/262197218 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbd80ad290 0x7fbbd8161c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd12c640 1 -- 172.21.15.60:0/262197218 >> v1:172.21.15.100:6789/0 conn(0x7fbbd815e0c0 legacy=0x7fbbd80acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd12c640 1 -- 172.21.15.60:0/262197218 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd815ef70 con 0x7fbbd8154c20 2024-04-25T10:14:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd12c640 1 -- 172.21.15.60:0/262197218 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fbbc8001680 con 0x7fbbd8154c20 2024-04-25T10:14:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd12c640 1 -- 172.21.15.60:0/262197218 >> v1:172.21.15.60:6789/0 conn(0x7fbbd8154c20 legacy=0x7fbbd80aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd12c640 1 --2- 172.21.15.60:0/262197218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00039f0 0x7fbbc0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd12c640 1 --2- 172.21.15.60:0/262197218 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc0004810 0x7fbbc0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd12c640 1 --2- 172.21.15.60:0/262197218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbc0005190 0x7fbbc000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd12c640 1 -- 172.21.15.60:0/262197218 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd815ec90 con 0x7fbbc00039f0 2024-04-25T10:14:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd12c640 1 -- 172.21.15.60:0/262197218 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc000bf10 con 0x7fbbc0005190 2024-04-25T10:14:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd12c640 1 -- 172.21.15.60:0/262197218 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc000c0e0 con 0x7fbbc0004810 2024-04-25T10:14:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbde12e640 1 --2- 172.21.15.60:0/262197218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00039f0 0x7fbbc0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd92d640 1 --2- 172.21.15.60:0/262197218 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc0004810 0x7fbbc0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbde92f640 1 --2- 172.21.15.60:0/262197218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbc0005190 0x7fbbc000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbde12e640 1 -- 172.21.15.60:0/262197218 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc0004810 msgr2=0x7fbbc0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbde12e640 1 --2- 172.21.15.60:0/262197218 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc0004810 0x7fbbc0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbde12e640 1 -- 172.21.15.60:0/262197218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbc0005190 msgr2=0x7fbbc000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbde12e640 1 --2- 172.21.15.60:0/262197218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbc0005190 0x7fbbc000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbde12e640 1 -- 172.21.15.60:0/262197218 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc0003530 con 0x7fbbc00039f0 2024-04-25T10:14:14.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbde92f640 1 --2- 172.21.15.60:0/262197218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbc0005190 0x7fbbc000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:14.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbde12e640 1 --2- 172.21.15.60:0/262197218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00039f0 0x7fbbc0003e10 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800c210 tx=0x7fbbc801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bbbea8eed7ff09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd12c640 1 -- 172.21.15.60:0/262197218 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8007d40 con 0x7fbbc00039f0 2024-04-25T10:14:14.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd12c640 1 -- 172.21.15.60:0/262197218 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc80a9070 con 0x7fbbc00039f0 2024-04-25T10:14:14.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbdd12c640 1 -- 172.21.15.60:0/262197218 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8005d40 con 0x7fbbc00039f0 2024-04-25T10:14:14.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbe0392640 1 -- 172.21.15.60:0/262197218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00039f0 msgr2=0x7fbbc0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbe0392640 1 --2- 172.21.15.60:0/262197218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00039f0 0x7fbbc0003e10 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800c210 tx=0x7fbbc801cca0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbde92f640 1 -- 172.21.15.60:0/262197218 reap_dead start 2024-04-25T10:14:14.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbe0392640 1 -- 172.21.15.60:0/262197218 shutdown_connections 2024-04-25T10:14:14.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbe0392640 1 -- 172.21.15.60:0/262197218 >> 172.21.15.60:0/262197218 conn(0x7fbbd8066fa0 msgr2=0x7fbbd806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbe0392640 1 -- 172.21.15.60:0/262197218 shutdown_connections 2024-04-25T10:14:14.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.227+0000 7fbbe0392640 1 -- 172.21.15.60:0/262197218 wait complete. 2024-04-25T10:14:14.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbe0392640 1 Processor -- start 2024-04-25T10:14:14.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbe0392640 1 -- start start 2024-04-25T10:14:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbe0392640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00039f0 0x7fbbd80713d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbe0392640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbd814ac10 0x7fbbd814b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbe0392640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd813bb50 0x7fbbd813bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbe0392640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd80cf5b0 con 0x7fbbc00039f0 2024-04-25T10:14:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbe0392640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd80cf430 con 0x7fbbd813bb50 2024-04-25T10:14:14.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbe0392640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd80cf730 con 0x7fbbd814ac10 2024-04-25T10:14:14.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbde12e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00039f0 0x7fbbd80713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbde12e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00039f0 0x7fbbd80713d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38484/0 (socket says 172.21.15.60:38484) 2024-04-25T10:14:14.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbde12e640 1 -- 172.21.15.60:0/1877589457 learned_addr learned my addr 172.21.15.60:0/1877589457 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:14.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbde92f640 1 --2- 172.21.15.60:0/1877589457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd813bb50 0x7fbbd813bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbdd92d640 1 --2- 172.21.15.60:0/1877589457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbd814ac10 0x7fbbd814b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbde12e640 1 -- 172.21.15.60:0/1877589457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbd814ac10 msgr2=0x7fbbd814b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbde12e640 1 --2- 172.21.15.60:0/1877589457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbd814ac10 0x7fbbd814b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbde12e640 1 -- 172.21.15.60:0/1877589457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd813bb50 msgr2=0x7fbbd813bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbde12e640 1 --2- 172.21.15.60:0/1877589457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd813bb50 0x7fbbd813bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbde12e640 1 -- 172.21.15.60:0/1877589457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd8155360 con 0x7fbbc00039f0 2024-04-25T10:14:14.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbde12e640 1 --2- 172.21.15.60:0/1877589457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00039f0 0x7fbbd80713d0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fbbc801cad0 tx=0x7fbbc0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbd6ffd640 1 -- 172.21.15.60:0/1877589457 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8007d20 con 0x7fbbc00039f0 2024-04-25T10:14:14.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbd6ffd640 1 -- 172.21.15.60:0/1877589457 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc8004a00 con 0x7fbbc00039f0 2024-04-25T10:14:14.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbe0392640 1 -- 172.21.15.60:0/1877589457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd8155640 con 0x7fbbc00039f0 2024-04-25T10:14:14.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbd6ffd640 1 -- 172.21.15.60:0/1877589457 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8004db0 con 0x7fbbc00039f0 2024-04-25T10:14:14.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.231+0000 7fbbe0392640 1 -- 172.21.15.60:0/1877589457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd8153990 con 0x7fbbc00039f0 2024-04-25T10:14:14.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.235+0000 7fbbe0392640 1 -- 172.21.15.60:0/1877589457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd8074770 con 0x7fbbc00039f0 2024-04-25T10:14:14.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.235+0000 7fbbd6ffd640 1 -- 172.21.15.60:0/1877589457 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbbd8162460 con 0x7fbbc00039f0 2024-04-25T10:14:14.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.235+0000 7fbbd6ffd640 1 --2- 172.21.15.60:0/1877589457 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbbc4066f60 0x7fbbc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.235+0000 7fbbd6ffd640 1 -- 172.21.15.60:0/1877589457 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbbc811e3f0 con 0x7fbbc00039f0 2024-04-25T10:14:14.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.235+0000 7fbbdd92d640 1 --2- 172.21.15.60:0/1877589457 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbbc4066f60 0x7fbbc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.235+0000 7fbbdd92d640 1 --2- 172.21.15.60:0/1877589457 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbbc4066f60 0x7fbbc4069420 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbbd81688f0 tx=0x7fbbcc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.239+0000 7fbbd6ffd640 1 -- 172.21.15.60:0/1877589457 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbbc80169b0 con 0x7fbbc00039f0 2024-04-25T10:14:14.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.399+0000 7fbbe0392640 1 -- 172.21.15.60:0/1877589457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbbd8150e00 con 0x7fbbc00039f0 2024-04-25T10:14:14.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.759+0000 7fbbd6ffd640 1 -- 172.21.15.60:0/1877589457 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v94) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbbc80b8380 con 0x7fbbc00039f0 2024-04-25T10:14:14.766 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T10:14:14.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.763+0000 7fbbe0392640 1 -- 172.21.15.60:0/1877589457 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbbc4066f60 msgr2=0x7fbbc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.763+0000 7fbbe0392640 1 --2- 172.21.15.60:0/1877589457 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbbc4066f60 0x7fbbc4069420 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbbd81688f0 tx=0x7fbbcc009040 comp rx=0 tx=0).stop 2024-04-25T10:14:14.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.763+0000 7fbbe0392640 1 -- 172.21.15.60:0/1877589457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00039f0 msgr2=0x7fbbd80713d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.763+0000 7fbbe0392640 1 --2- 172.21.15.60:0/1877589457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00039f0 0x7fbbd80713d0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fbbc801cad0 tx=0x7fbbc0005840 comp rx=0 tx=0).stop 2024-04-25T10:14:14.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.767+0000 7fbbe0392640 1 -- 172.21.15.60:0/1877589457 shutdown_connections 2024-04-25T10:14:14.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.767+0000 7fbbe0392640 1 --2- 172.21.15.60:0/1877589457 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbbc4066f60 0x7fbbc4069420 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.767+0000 7fbbe0392640 1 --2- 172.21.15.60:0/1877589457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd813bb50 0x7fbbd813bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.767+0000 7fbbe0392640 1 --2- 172.21.15.60:0/1877589457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbd814ac10 0x7fbbd814b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.767+0000 7fbbe0392640 1 --2- 172.21.15.60:0/1877589457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc00039f0 0x7fbbd80713d0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.767+0000 7fbbe0392640 1 -- 172.21.15.60:0/1877589457 >> 172.21.15.60:0/1877589457 conn(0x7fbbd8066fa0 msgr2=0x7fbbd80afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.767+0000 7fbbe0392640 1 -- 172.21.15.60:0/1877589457 shutdown_connections 2024-04-25T10:14:14.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.767+0000 7fbbe0392640 1 -- 172.21.15.60:0/1877589457 wait complete. 2024-04-25T10:14:14.798 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:14:14.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.931+0000 7f179670b640 1 Processor -- start 2024-04-25T10:14:14.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.931+0000 7f179670b640 1 -- start start 2024-04-25T10:14:14.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.931+0000 7f179670b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f17900aee30 0x7f17900b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.931+0000 7f179670b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17900b3020 0x7f179015e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.931+0000 7f179670b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17900ceb90 con 0x7f17900b3480 2024-04-25T10:14:14.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.931+0000 7f179670b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17900af210 con 0x7f17900aee30 2024-04-25T10:14:14.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.931+0000 7f179670b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17900cee70 con 0x7f17900b3020 2024-04-25T10:14:14.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.931+0000 7f178ffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f17900aee30 0x7f17900b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.931+0000 7f178f7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17900b3020 0x7f179015e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.931+0000 7f1794ca8640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f17900b3480 0x7f179006e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:47138/0 (socket says 172.21.15.60:47138) 2024-04-25T10:14:14.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.931+0000 7f1794ca8640 1 -- 172.21.15.60:0/2524116084 learned_addr learned my addr 172.21.15.60:0/2524116084 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:14.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.931+0000 7f178ffff640 1 -- 172.21.15.60:0/2524116084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17900b3020 msgr2=0x7f179015e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.931+0000 7f178ffff640 1 --2- 172.21.15.60:0/2524116084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17900b3020 0x7f179015e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.931+0000 7f178ffff640 1 -- 172.21.15.60:0/2524116084 >> v1:172.21.15.100:6789/0 conn(0x7f17900b3480 legacy=0x7f179006e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.931+0000 7f178ffff640 1 -- 172.21.15.60:0/2524116084 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17900cf150 con 0x7f17900aee30 2024-04-25T10:14:14.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.931+0000 7f178f7fe640 1 --2- 172.21.15.60:0/2524116084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17900b3020 0x7f179015e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:14.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.935+0000 7f178ffff640 1 --2- 172.21.15.60:0/2524116084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17900aee30 0x7f17900b4560 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f17800097c0 tx=0x7f178001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e148b57053e07b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.935+0000 7f178effd640 1 -- 172.21.15.60:0/2524116084 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17800b0070 con 0x7f17900aee30 2024-04-25T10:14:14.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.935+0000 7f178effd640 1 -- 172.21.15.60:0/2524116084 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1780004230 con 0x7f17900aee30 2024-04-25T10:14:14.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.935+0000 7f178effd640 1 -- 172.21.15.60:0/2524116084 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780003620 con 0x7f17900aee30 2024-04-25T10:14:14.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.935+0000 7f179670b640 1 -- 172.21.15.60:0/2524116084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17900aee30 msgr2=0x7f17900b4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.935+0000 7f179670b640 1 --2- 172.21.15.60:0/2524116084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17900aee30 0x7f17900b4560 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f17800097c0 tx=0x7f178001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.935+0000 7f179670b640 1 -- 172.21.15.60:0/2524116084 shutdown_connections 2024-04-25T10:14:14.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.935+0000 7f179670b640 1 --2- 172.21.15.60:0/2524116084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17900b3020 0x7f179015e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.935+0000 7f179670b640 1 --2- 172.21.15.60:0/2524116084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17900aee30 0x7f17900b4560 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.935+0000 7f179670b640 1 -- 172.21.15.60:0/2524116084 >> 172.21.15.60:0/2524116084 conn(0x7f1790067060 msgr2=0x7f179006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.935+0000 7f179670b640 1 -- 172.21.15.60:0/2524116084 shutdown_connections 2024-04-25T10:14:14.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.935+0000 7f179670b640 1 -- 172.21.15.60:0/2524116084 wait complete. 2024-04-25T10:14:14.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f179670b640 1 Processor -- start 2024-04-25T10:14:14.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f179670b640 1 -- start start 2024-04-25T10:14:14.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f179670b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17900aee30 0x7f17901f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f179670b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17900b3020 0x7f179013d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f179670b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17900b3480 0x7f1790158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f179670b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17900ce930 con 0x7f17900b3020 2024-04-25T10:14:14.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f179670b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17900ce7b0 con 0x7f17900b3480 2024-04-25T10:14:14.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f179670b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17900b4aa0 con 0x7f17900aee30 2024-04-25T10:14:14.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f178f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17900b3020 0x7f179013d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f178f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17900b3020 0x7f179013d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38486/0 (socket says 172.21.15.60:38486) 2024-04-25T10:14:14.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f178f7fe640 1 -- 172.21.15.60:0/979310682 learned_addr learned my addr 172.21.15.60:0/979310682 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:14.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f178ffff640 1 --2- 172.21.15.60:0/979310682 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17900aee30 0x7f17901f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f1794ca8640 1 --2- 172.21.15.60:0/979310682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17900b3480 0x7f1790158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f178f7fe640 1 -- 172.21.15.60:0/979310682 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17900aee30 msgr2=0x7f17901f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f178f7fe640 1 --2- 172.21.15.60:0/979310682 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17900aee30 0x7f17901f4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f178f7fe640 1 -- 172.21.15.60:0/979310682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17900b3480 msgr2=0x7f1790158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f178f7fe640 1 --2- 172.21.15.60:0/979310682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17900b3480 0x7f1790158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f178f7fe640 1 -- 172.21.15.60:0/979310682 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179013dca0 con 0x7f17900b3020 2024-04-25T10:14:14.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f1794ca8640 1 --2- 172.21.15.60:0/979310682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17900b3480 0x7f1790158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:14.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f178f7fe640 1 --2- 172.21.15.60:0/979310682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17900b3020 0x7f179013d760 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f178400e9b0 tx=0x7f178400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f178cff9640 1 -- 172.21.15.60:0/979310682 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784017070 con 0x7f17900b3020 2024-04-25T10:14:14.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f178cff9640 1 -- 172.21.15.60:0/979310682 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178400ca00 con 0x7f17900b3020 2024-04-25T10:14:14.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.939+0000 7f178cff9640 1 -- 172.21.15.60:0/979310682 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178400cdd0 con 0x7f17900b3020 2024-04-25T10:14:14.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.943+0000 7f179670b640 1 -- 172.21.15.60:0/979310682 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17901f54f0 con 0x7f17900b3020 2024-04-25T10:14:14.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.943+0000 7f179670b640 1 -- 172.21.15.60:0/979310682 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17901f5ab0 con 0x7f17900b3020 2024-04-25T10:14:14.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.943+0000 7f179670b640 1 -- 172.21.15.60:0/979310682 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1790074770 con 0x7f17900b3020 2024-04-25T10:14:14.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.943+0000 7f178cff9640 1 -- 172.21.15.60:0/979310682 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f178401a070 con 0x7f17900b3020 2024-04-25T10:14:14.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.943+0000 7f178cff9640 1 --2- 172.21.15.60:0/979310682 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f177c066f60 0x7f177c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.943+0000 7f178cff9640 1 -- 172.21.15.60:0/979310682 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f1784084c10 con 0x7f17900b3020 2024-04-25T10:14:14.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.943+0000 7f178ffff640 1 --2- 172.21.15.60:0/979310682 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f177c066f60 0x7f177c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.943+0000 7f178ffff640 1 --2- 172.21.15.60:0/979310682 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f177c066f60 0x7f177c069420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f17800ad000 tx=0x7f1780008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:14.947+0000 7f178cff9640 1 -- 172.21.15.60:0/979310682 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f17840557b0 con 0x7f17900b3020 2024-04-25T10:14:15.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.111+0000 7f179670b640 1 -- 172.21.15.60:0/979310682 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f17900b2bd0 con 0x7f17900b3020 2024-04-25T10:14:15.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.767+0000 7f178cff9640 1 -- 172.21.15.60:0/979310682 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v95) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1784059850 con 0x7f17900b3020 2024-04-25T10:14:15.771 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T10:14:15.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.767+0000 7f179670b640 1 -- 172.21.15.60:0/979310682 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f177c066f60 msgr2=0x7f177c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.767+0000 7f179670b640 1 --2- 172.21.15.60:0/979310682 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f177c066f60 0x7f177c069420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f17800ad000 tx=0x7f1780008040 comp rx=0 tx=0).stop 2024-04-25T10:14:15.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.767+0000 7f179670b640 1 -- 172.21.15.60:0/979310682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17900b3020 msgr2=0x7f179013d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.767+0000 7f179670b640 1 --2- 172.21.15.60:0/979310682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17900b3020 0x7f179013d760 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f178400e9b0 tx=0x7f178400ee70 comp rx=0 tx=0).stop 2024-04-25T10:14:15.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.771+0000 7f179670b640 1 -- 172.21.15.60:0/979310682 shutdown_connections 2024-04-25T10:14:15.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.771+0000 7f179670b640 1 --2- 172.21.15.60:0/979310682 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f177c066f60 0x7f177c069420 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.771+0000 7f179670b640 1 --2- 172.21.15.60:0/979310682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17900b3480 0x7f1790158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.771+0000 7f179670b640 1 --2- 172.21.15.60:0/979310682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17900b3020 0x7f179013d760 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.771+0000 7f179670b640 1 --2- 172.21.15.60:0/979310682 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17900aee30 0x7f17901f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.771+0000 7f179670b640 1 -- 172.21.15.60:0/979310682 >> 172.21.15.60:0/979310682 conn(0x7f1790067060 msgr2=0x7f17900af400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.771+0000 7f179670b640 1 -- 172.21.15.60:0/979310682 shutdown_connections 2024-04-25T10:14:15.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.771+0000 7f179670b640 1 -- 172.21.15.60:0/979310682 wait complete. 2024-04-25T10:14:15.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:14:15.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3231614640 1 Processor -- start 2024-04-25T10:14:15.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3231614640 1 -- start start 2024-04-25T10:14:15.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3231614640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f322c06f030 0x7f322c0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3231614640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c15ad30 con 0x7f322c0cf4f0 2024-04-25T10:14:15.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3231614640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c15b010 con 0x7f322c0b3f60 2024-04-25T10:14:15.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3231614640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c15aaf0 con 0x7f322c06f030 2024-04-25T10:14:15.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f322b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f322c0cf4f0 0x7f322c071250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45918/0 (socket says 172.21.15.60:45918) 2024-04-25T10:14:15.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f322b7fe640 1 -- 172.21.15.60:0/2668882358 learned_addr learned my addr 172.21.15.60:0/2668882358 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:15.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f322a7fc640 1 --2- 172.21.15.60:0/2668882358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f322c06f030 0x7f322c0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 -- 172.21.15.60:0/2668882358 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608740277 0 0) 0x7f322c15ad30 con 0x7f322c0cf4f0 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 -- 172.21.15.60:0/2668882358 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3214003530 con 0x7f322c0cf4f0 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 -- 172.21.15.60:0/2668882358 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674786633 0 0) 0x7f322c15b010 con 0x7f322c0b3f60 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 -- 172.21.15.60:0/2668882358 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f322c15ad30 con 0x7f322c0b3f60 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 -- 172.21.15.60:0/2668882358 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 59905810 0 0) 0x7f3214003530 con 0x7f322c0cf4f0 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 -- 172.21.15.60:0/2668882358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f322c06f030 msgr2=0x7f322c0b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 --2- 172.21.15.60:0/2668882358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f322c06f030 0x7f322c0b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 -- 172.21.15.60:0/2668882358 >> v1:172.21.15.100:6789/0 conn(0x7f322c0b3f60 legacy=0x7f322c15a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 -- 172.21.15.60:0/2668882358 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c15b2f0 con 0x7f322c0cf4f0 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 -- 172.21.15.60:0/2668882358 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f3224001a00 con 0x7f322c0cf4f0 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 -- 172.21.15.60:0/2668882358 >> v1:172.21.15.60:6789/0 conn(0x7f322c0cf4f0 legacy=0x7f322c071250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 --2- 172.21.15.60:0/2668882358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3214003b80 0x7f3214003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 --2- 172.21.15.60:0/2668882358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f322c06f030 0x7f3214004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 --2- 172.21.15.60:0/2668882358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3214004d50 0x7f32140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 -- 172.21.15.60:0/2668882358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c15aaf0 con 0x7f3214004d50 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 -- 172.21.15.60:0/2668882358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321400dd70 con 0x7f3214003b80 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f3229ffb640 1 -- 172.21.15.60:0/2668882358 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321400df70 con 0x7f322c06f030 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f322b7fe640 1 --2- 172.21.15.60:0/2668882358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3214004d50 0x7f32140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f322affd640 1 --2- 172.21.15.60:0/2668882358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3214003b80 0x7f3214003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f322a7fc640 1 --2- 172.21.15.60:0/2668882358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f322c06f030 0x7f3214004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f322b7fe640 1 -- 172.21.15.60:0/2668882358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f322c06f030 msgr2=0x7f3214004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f322b7fe640 1 --2- 172.21.15.60:0/2668882358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f322c06f030 0x7f3214004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f322b7fe640 1 -- 172.21.15.60:0/2668882358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3214003b80 msgr2=0x7f3214003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f322b7fe640 1 --2- 172.21.15.60:0/2668882358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3214003b80 0x7f3214003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f322b7fe640 1 -- 172.21.15.60:0/2668882358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3214003530 con 0x7f3214004d50 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.931+0000 7f322affd640 1 --2- 172.21.15.60:0/2668882358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3214003b80 0x7f3214003f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:15.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.935+0000 7f322b7fe640 1 --2- 172.21.15.60:0/2668882358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3214004d50 0x7f32140051d0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f322400a730 tx=0x7f322401d220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7cbab1ed997c97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.935+0000 7f3229ffb640 1 -- 172.21.15.60:0/2668882358 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322400a350 con 0x7f3214004d50 2024-04-25T10:14:15.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.935+0000 7f3229ffb640 1 -- 172.21.15.60:0/2668882358 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32240ad430 con 0x7f3214004d50 2024-04-25T10:14:15.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.935+0000 7f3229ffb640 1 -- 172.21.15.60:0/2668882358 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32240ad800 con 0x7f3214004d50 2024-04-25T10:14:15.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.935+0000 7f3231614640 1 -- 172.21.15.60:0/2668882358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3214004d50 msgr2=0x7f32140051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.935+0000 7f3231614640 1 --2- 172.21.15.60:0/2668882358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3214004d50 0x7f32140051d0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f322400a730 tx=0x7f322401d220 comp rx=0 tx=0).stop 2024-04-25T10:14:15.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.935+0000 7f322b7fe640 1 -- 172.21.15.60:0/2668882358 reap_dead start 2024-04-25T10:14:15.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.935+0000 7f3231614640 1 -- 172.21.15.60:0/2668882358 shutdown_connections 2024-04-25T10:14:15.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.935+0000 7f3231614640 1 -- 172.21.15.60:0/2668882358 >> 172.21.15.60:0/2668882358 conn(0x7f322c0670b0 msgr2=0x7f322c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.935+0000 7f3231614640 1 -- 172.21.15.60:0/2668882358 shutdown_connections 2024-04-25T10:14:15.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.935+0000 7f3231614640 1 -- 172.21.15.60:0/2668882358 wait complete. 2024-04-25T10:14:15.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.935+0000 7f3231614640 1 Processor -- start 2024-04-25T10:14:15.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.935+0000 7f3231614640 1 -- start start 2024-04-25T10:14:15.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f3231614640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322c153340 0x7f322c149600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f3231614640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f322c153720 0x7f322c14add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f3231614640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f322c14b310 0x7f322c138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f3231614640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0cf980 con 0x7f322c153720 2024-04-25T10:14:15.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f3231614640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0b4340 con 0x7f322c153340 2024-04-25T10:14:15.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f3231614640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0cfb00 con 0x7f322c14b310 2024-04-25T10:14:15.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f322a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f322c153720 0x7f322c14add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f322a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f322c153720 0x7f322c14add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38508/0 (socket says 172.21.15.60:38508) 2024-04-25T10:14:15.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f322a7fc640 1 -- 172.21.15.60:0/1515203199 learned_addr learned my addr 172.21.15.60:0/1515203199 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:15.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f322affd640 1 --2- 172.21.15.60:0/1515203199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322c153340 0x7f322c149600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f322b7fe640 1 --2- 172.21.15.60:0/1515203199 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f322c14b310 0x7f322c138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f322a7fc640 1 -- 172.21.15.60:0/1515203199 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f322c14b310 msgr2=0x7f322c138cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f322a7fc640 1 --2- 172.21.15.60:0/1515203199 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f322c14b310 0x7f322c138cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f322a7fc640 1 -- 172.21.15.60:0/1515203199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322c153340 msgr2=0x7f322c149600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f322a7fc640 1 --2- 172.21.15.60:0/1515203199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322c153340 0x7f322c149600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f322a7fc640 1 -- 172.21.15.60:0/1515203199 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c14b790 con 0x7f322c153720 2024-04-25T10:14:15.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f322affd640 1 --2- 172.21.15.60:0/1515203199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322c153340 0x7f322c149600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:15.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f322a7fc640 1 --2- 172.21.15.60:0/1515203199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f322c153720 0x7f322c14add0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f3220010890 tx=0x7f3220010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f32137fe640 1 -- 172.21.15.60:0/1515203199 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322000f040 con 0x7f322c153720 2024-04-25T10:14:15.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f32137fe640 1 -- 172.21.15.60:0/1515203199 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322000eb60 con 0x7f322c153720 2024-04-25T10:14:15.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f3231614640 1 -- 172.21.15.60:0/1515203199 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c14ba70 con 0x7f322c153720 2024-04-25T10:14:15.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f32137fe640 1 -- 172.21.15.60:0/1515203199 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322001c400 con 0x7f322c153720 2024-04-25T10:14:15.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f3231614640 1 -- 172.21.15.60:0/1515203199 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c14c030 con 0x7f322c153720 2024-04-25T10:14:15.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f3231614640 1 -- 172.21.15.60:0/1515203199 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c074770 con 0x7f322c153720 2024-04-25T10:14:15.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f32137fe640 1 -- 172.21.15.60:0/1515203199 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f322001c5a0 con 0x7f322c153720 2024-04-25T10:14:15.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f32137fe640 1 --2- 172.21.15.60:0/1515203199 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3218066f90 0x7f3218069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f32137fe640 1 -- 172.21.15.60:0/1515203199 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3220085630 con 0x7f322c153720 2024-04-25T10:14:15.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.939+0000 7f322affd640 1 --2- 172.21.15.60:0/1515203199 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3218066f90 0x7f3218069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.943+0000 7f322affd640 1 --2- 172.21.15.60:0/1515203199 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3218066f90 0x7f3218069450 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f321c007af0 tx=0x7f321c007a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:15.943+0000 7f32137fe640 1 -- 172.21.15.60:0/1515203199 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3220018030 con 0x7f322c153720 2024-04-25T10:14:16.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.143+0000 7f3231614640 1 -- 172.21.15.60:0/1515203199 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f322c159aa0 con 0x7f322c153720 2024-04-25T10:14:16.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.843+0000 7f32137fe640 1 -- 172.21.15.60:0/1515203199 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v73) v1 ==== 174+0+0 (secure 0 0 0) 0x7f32200566a0 con 0x7f322c153720 2024-04-25T10:14:16.849 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 14 and data pool 15 2024-04-25T10:14:16.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.847+0000 7f3231614640 1 -- 172.21.15.60:0/1515203199 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3218066f90 msgr2=0x7f3218069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.847+0000 7f3231614640 1 --2- 172.21.15.60:0/1515203199 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3218066f90 0x7f3218069450 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f321c007af0 tx=0x7f321c007a80 comp rx=0 tx=0).stop 2024-04-25T10:14:16.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.847+0000 7f3231614640 1 -- 172.21.15.60:0/1515203199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f322c153720 msgr2=0x7f322c14add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.847+0000 7f3231614640 1 --2- 172.21.15.60:0/1515203199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f322c153720 0x7f322c14add0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f3220010890 tx=0x7f3220010d50 comp rx=0 tx=0).stop 2024-04-25T10:14:16.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.847+0000 7f3231614640 1 -- 172.21.15.60:0/1515203199 shutdown_connections 2024-04-25T10:14:16.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.847+0000 7f3231614640 1 --2- 172.21.15.60:0/1515203199 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3218066f90 0x7f3218069450 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.847+0000 7f3231614640 1 --2- 172.21.15.60:0/1515203199 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f322c14b310 0x7f322c138cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.847+0000 7f3231614640 1 --2- 172.21.15.60:0/1515203199 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f322c153720 0x7f322c14add0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.847+0000 7f3231614640 1 --2- 172.21.15.60:0/1515203199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f322c153340 0x7f322c149600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.847+0000 7f3231614640 1 -- 172.21.15.60:0/1515203199 >> 172.21.15.60:0/1515203199 conn(0x7f322c0670b0 msgr2=0x7f322c1501b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:16.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.847+0000 7f3231614640 1 -- 172.21.15.60:0/1515203199 shutdown_connections 2024-04-25T10:14:16.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.847+0000 7f3231614640 1 -- 172.21.15.60:0/1515203199 wait complete. 2024-04-25T10:14:16.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:16.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2238095640 1 Processor -- start 2024-04-25T10:14:16.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2238095640 1 -- start start 2024-04-25T10:14:16.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2238095640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f223006ac10 0x7f223006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2238095640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f22300b54d0 0x7f22300af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2238095640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f223006b530 con 0x7f22300b3610 2024-04-25T10:14:16.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2238095640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22300b3e60 con 0x7f22300b54d0 2024-04-25T10:14:16.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2238095640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22300b3b80 con 0x7f223006ac10 2024-04-25T10:14:16.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2235630640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f22300b3610 0x7f22300aec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45930/0 (socket says 172.21.15.60:45930) 2024-04-25T10:14:16.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2235630640 1 -- 172.21.15.60:0/3668432676 learned_addr learned my addr 172.21.15.60:0/3668432676 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:16.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2236632640 1 --2- 172.21.15.60:0/3668432676 >> v2:172.21.15.100:3300/0 conn(0x7f22300b54d0 0x7f22300af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2235e31640 1 --2- 172.21.15.60:0/3668432676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f223006ac10 0x7f223006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2234e2f640 1 -- 172.21.15.60:0/3668432676 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687262383 0 0) 0x7f223006b530 con 0x7f22300b3610 2024-04-25T10:14:16.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2234e2f640 1 -- 172.21.15.60:0/3668432676 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f221c003530 con 0x7f22300b3610 2024-04-25T10:14:16.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2234e2f640 1 -- 172.21.15.60:0/3668432676 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4272054819 0 0) 0x7f221c003530 con 0x7f22300b3610 2024-04-25T10:14:16.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2234e2f640 1 -- 172.21.15.60:0/3668432676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f223006ac10 msgr2=0x7f223006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:16.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2234e2f640 1 --2- 172.21.15.60:0/3668432676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f223006ac10 0x7f223006aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2234e2f640 1 -- 172.21.15.60:0/3668432676 >> v2:172.21.15.100:3300/0 conn(0x7f22300b54d0 msgr2=0x7f22300af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:16.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2234e2f640 1 --2- 172.21.15.60:0/3668432676 >> v2:172.21.15.100:3300/0 conn(0x7f22300b54d0 0x7f22300af5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2234e2f640 1 -- 172.21.15.60:0/3668432676 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22300ae130 con 0x7f22300b3610 2024-04-25T10:14:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2234e2f640 1 -- 172.21.15.60:0/3668432676 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f2220001680 con 0x7f22300b3610 2024-04-25T10:14:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2234e2f640 1 -- 172.21.15.60:0/3668432676 >> v1:172.21.15.60:6789/0 conn(0x7f22300b3610 legacy=0x7f22300aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2234e2f640 1 --2- 172.21.15.60:0/3668432676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c0039c0 0x7f221c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2234e2f640 1 --2- 172.21.15.60:0/3668432676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f223006ac10 0x7f221c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2234e2f640 1 --2- 172.21.15.60:0/3668432676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d50 0x7f221c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2234e2f640 1 -- 172.21.15.60:0/3668432676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22300b3e60 con 0x7f221c004d50 2024-04-25T10:14:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2234e2f640 1 -- 172.21.15.60:0/3668432676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22300b3b80 con 0x7f221c0039c0 2024-04-25T10:14:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2234e2f640 1 -- 172.21.15.60:0/3668432676 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221c00b9d0 con 0x7f223006ac10 2024-04-25T10:14:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2236632640 1 --2- 172.21.15.60:0/3668432676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d50 0x7f221c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2235e31640 1 --2- 172.21.15.60:0/3668432676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c0039c0 0x7f221c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2235630640 1 --2- 172.21.15.60:0/3668432676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f223006ac10 0x7f221c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2236632640 1 -- 172.21.15.60:0/3668432676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f223006ac10 msgr2=0x7f221c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2236632640 1 --2- 172.21.15.60:0/3668432676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f223006ac10 0x7f221c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2236632640 1 -- 172.21.15.60:0/3668432676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c0039c0 msgr2=0x7f221c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2236632640 1 --2- 172.21.15.60:0/3668432676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c0039c0 0x7f221c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2236632640 1 -- 172.21.15.60:0/3668432676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c003530 con 0x7f221c004d50 2024-04-25T10:14:17.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.991+0000 7f2235e31640 1 --2- 172.21.15.60:0/3668432676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c0039c0 0x7f221c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.995+0000 7f2236632640 1 --2- 172.21.15.60:0/3668432676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d50 0x7f221c0051d0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f2228009fc0 tx=0x7f222801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d9332c0bfba920 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.995+0000 7f2234e2f640 1 -- 172.21.15.60:0/3668432676 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22280b2070 con 0x7f221c004d50 2024-04-25T10:14:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.995+0000 7f2234e2f640 1 -- 172.21.15.60:0/3668432676 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22280a5cd0 con 0x7f221c004d50 2024-04-25T10:14:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.995+0000 7f2234e2f640 1 -- 172.21.15.60:0/3668432676 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228009c10 con 0x7f221c004d50 2024-04-25T10:14:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.995+0000 7f2238095640 1 -- 172.21.15.60:0/3668432676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d50 msgr2=0x7f221c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.995+0000 7f2238095640 1 --2- 172.21.15.60:0/3668432676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d50 0x7f221c0051d0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f2228009fc0 tx=0x7f222801ccc0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.995+0000 7f2236632640 1 -- 172.21.15.60:0/3668432676 reap_dead start 2024-04-25T10:14:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.995+0000 7f2238095640 1 -- 172.21.15.60:0/3668432676 shutdown_connections 2024-04-25T10:14:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.995+0000 7f2238095640 1 -- 172.21.15.60:0/3668432676 >> 172.21.15.60:0/3668432676 conn(0x7f2230072920 msgr2=0x7f2230099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.995+0000 7f2238095640 1 -- 172.21.15.60:0/3668432676 shutdown_connections 2024-04-25T10:14:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.995+0000 7f2238095640 1 -- 172.21.15.60:0/3668432676 wait complete. 2024-04-25T10:14:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2238095640 1 Processor -- start 2024-04-25T10:14:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2238095640 1 -- start start 2024-04-25T10:14:17.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2238095640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d50 0x7f2230157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2238095640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300d0ac0 0x7f22301f3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2238095640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2230154450 0x7f22301548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2238095640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22300b5960 con 0x7f221c004d50 2024-04-25T10:14:17.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2238095640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22300ae6f0 con 0x7f2230154450 2024-04-25T10:14:17.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2238095640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22300b5ae0 con 0x7f22300d0ac0 2024-04-25T10:14:17.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2235e31640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d50 0x7f2230157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2235e31640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d50 0x7f2230157a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38536/0 (socket says 172.21.15.60:38536) 2024-04-25T10:14:17.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2235e31640 1 -- 172.21.15.60:0/1740601386 learned_addr learned my addr 172.21.15.60:0/1740601386 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:17.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2235630640 1 --2- 172.21.15.60:0/1740601386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300d0ac0 0x7f22301f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2236632640 1 --2- 172.21.15.60:0/1740601386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2230154450 0x7f22301548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2235e31640 1 -- 172.21.15.60:0/1740601386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300d0ac0 msgr2=0x7f22301f3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2235e31640 1 --2- 172.21.15.60:0/1740601386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300d0ac0 0x7f22301f3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2235e31640 1 -- 172.21.15.60:0/1740601386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2230154450 msgr2=0x7f22301548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2235e31640 1 --2- 172.21.15.60:0/1740601386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2230154450 0x7f22301548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2235e31640 1 -- 172.21.15.60:0/1740601386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22301f3c10 con 0x7f221c004d50 2024-04-25T10:14:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2235e31640 1 --2- 172.21.15.60:0/1740601386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d50 0x7f2230157a90 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f222c009fd0 tx=0x7f222c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f22267fc640 1 -- 172.21.15.60:0/1740601386 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c004220 con 0x7f221c004d50 2024-04-25T10:14:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f22267fc640 1 -- 172.21.15.60:0/1740601386 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222c0043c0 con 0x7f221c004d50 2024-04-25T10:14:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2238095640 1 -- 172.21.15.60:0/1740601386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22301f3ef0 con 0x7f221c004d50 2024-04-25T10:14:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f22267fc640 1 -- 172.21.15.60:0/1740601386 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c015890 con 0x7f221c004d50 2024-04-25T10:14:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f2238095640 1 -- 172.21.15.60:0/1740601386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22300cac20 con 0x7f221c004d50 2024-04-25T10:14:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:16.999+0000 7f22267fc640 1 -- 172.21.15.60:0/1740601386 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f222c015a30 con 0x7f221c004d50 2024-04-25T10:14:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.003+0000 7f2238095640 1 -- 172.21.15.60:0/1740601386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2230074770 con 0x7f221c004d50 2024-04-25T10:14:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.003+0000 7f22267fc640 1 --2- 172.21.15.60:0/1740601386 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2214066f60 0x7f2214069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.003+0000 7f22267fc640 1 -- 172.21.15.60:0/1740601386 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f222c085660 con 0x7f221c004d50 2024-04-25T10:14:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.003+0000 7f2235630640 1 --2- 172.21.15.60:0/1740601386 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2214066f60 0x7f2214069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.003+0000 7f2235630640 1 --2- 172.21.15.60:0/1740601386 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2214066f60 0x7f2214069420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2220000c10 tx=0x7f2220009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.007+0000 7f22267fc640 1 -- 172.21.15.60:0/1740601386 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f222c019030 con 0x7f221c004d50 2024-04-25T10:14:17.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.167+0000 7f2238095640 1 -- 172.21.15.60:0/1740601386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22300700a0 con 0x7f221c004d50 2024-04-25T10:14:17.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.167+0000 7f22267fc640 1 -- 172.21.15.60:0/1740601386 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16633 (secure 0 0 0) 0x7f222c056670 con 0x7f221c004d50 2024-04-25T10:14:17.173 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:17.173 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":96,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:14:16.145711+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:14.444411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:15.153386+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.171+0000 7f2238095640 1 -- 172.21.15.60:0/1740601386 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2214066f60 msgr2=0x7f2214069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.171+0000 7f2238095640 1 --2- 172.21.15.60:0/1740601386 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2214066f60 0x7f2214069420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2220000c10 tx=0x7f2220009040 comp rx=0 tx=0).stop 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.171+0000 7f2238095640 1 -- 172.21.15.60:0/1740601386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d50 msgr2=0x7f2230157a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.171+0000 7f2238095640 1 --2- 172.21.15.60:0/1740601386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d50 0x7f2230157a90 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f222c009fd0 tx=0x7f222c011a60 comp rx=0 tx=0).stop 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.171+0000 7f2238095640 1 -- 172.21.15.60:0/1740601386 shutdown_connections 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.171+0000 7f2238095640 1 --2- 172.21.15.60:0/1740601386 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2214066f60 0x7f2214069420 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.171+0000 7f2238095640 1 --2- 172.21.15.60:0/1740601386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2230154450 0x7f22301548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.171+0000 7f2238095640 1 --2- 172.21.15.60:0/1740601386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300d0ac0 0x7f22301f3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.171+0000 7f2238095640 1 --2- 172.21.15.60:0/1740601386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d50 0x7f2230157a90 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.171+0000 7f2238095640 1 -- 172.21.15.60:0/1740601386 >> 172.21.15.60:0/1740601386 conn(0x7f2230072920 msgr2=0x7f22300b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.171+0000 7f2238095640 1 -- 172.21.15.60:0/1740601386 shutdown_connections 2024-04-25T10:14:17.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.171+0000 7f2238095640 1 -- 172.21.15.60:0/1740601386 wait complete. 2024-04-25T10:14:17.193 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:17.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.319+0000 7f4bc5aab640 1 Processor -- start 2024-04-25T10:14:17.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.319+0000 7f4bc5aab640 1 -- start start 2024-04-25T10:14:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.319+0000 7f4bc5aab640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc00b5380 0x7f4bc00b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.319+0000 7f4bc5aab640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4bc00d2460 0x7f4bc00d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.319+0000 7f4bc5aab640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc00b6a30 con 0x7f4bc015dd00 2024-04-25T10:14:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.319+0000 7f4bc5aab640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc015e0e0 con 0x7f4bc00d2460 2024-04-25T10:14:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.319+0000 7f4bc5aab640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00b5d20 con 0x7f4bc00b5380 2024-04-25T10:14:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.319+0000 7f4bbf7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4bc015dd00 0x7f4bc0161770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45944/0 (socket says 172.21.15.60:45944) 2024-04-25T10:14:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.319+0000 7f4bbf7fe640 1 -- 172.21.15.60:0/2935040643 learned_addr learned my addr 172.21.15.60:0/2935040643 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbffff640 1 --2- 172.21.15.60:0/2935040643 >> v2:172.21.15.100:3300/0 conn(0x7f4bc00d2460 0x7f4bc00d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbeffd640 1 --2- 172.21.15.60:0/2935040643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc00b5380 0x7f4bc00b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbe7fc640 1 -- 172.21.15.60:0/2935040643 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287479937 0 0) 0x7f4bb0001120 con 0x7f4bc015dd00 2024-04-25T10:14:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbe7fc640 1 -- 172.21.15.60:0/2935040643 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ba8003530 con 0x7f4bc015dd00 2024-04-25T10:14:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbe7fc640 1 -- 172.21.15.60:0/2935040643 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2672488486 0 0) 0x7f4ba8003530 con 0x7f4bc015dd00 2024-04-25T10:14:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbe7fc640 1 -- 172.21.15.60:0/2935040643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc00b5380 msgr2=0x7f4bc00b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbe7fc640 1 --2- 172.21.15.60:0/2935040643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc00b5380 0x7f4bc00b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbe7fc640 1 -- 172.21.15.60:0/2935040643 >> v2:172.21.15.100:3300/0 conn(0x7f4bc00d2460 msgr2=0x7f4bc00d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbe7fc640 1 --2- 172.21.15.60:0/2935040643 >> v2:172.21.15.100:3300/0 conn(0x7f4bc00d2460 0x7f4bc00d28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbe7fc640 1 -- 172.21.15.60:0/2935040643 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc0162210 con 0x7f4bc015dd00 2024-04-25T10:14:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbe7fc640 1 -- 172.21.15.60:0/2935040643 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f4bb0001850 con 0x7f4bc015dd00 2024-04-25T10:14:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbe7fc640 1 -- 172.21.15.60:0/2935040643 >> v1:172.21.15.60:6789/0 conn(0x7f4bc015dd00 legacy=0x7f4bc0161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbe7fc640 1 --2- 172.21.15.60:0/2935040643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba80039c0 0x7f4ba8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbe7fc640 1 --2- 172.21.15.60:0/2935040643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba8004810 0x7f4ba8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbe7fc640 1 --2- 172.21.15.60:0/2935040643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc00b5380 0x7f4ba80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbe7fc640 1 -- 172.21.15.60:0/2935040643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba800b9d0 con 0x7f4ba8004810 2024-04-25T10:14:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbe7fc640 1 -- 172.21.15.60:0/2935040643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba800bb50 con 0x7f4ba80039c0 2024-04-25T10:14:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbe7fc640 1 -- 172.21.15.60:0/2935040643 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba800bd00 con 0x7f4bc00b5380 2024-04-25T10:14:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbeffd640 1 --2- 172.21.15.60:0/2935040643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba8004810 0x7f4ba8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbf7fe640 1 --2- 172.21.15.60:0/2935040643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba80039c0 0x7f4ba8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbffff640 1 --2- 172.21.15.60:0/2935040643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc00b5380 0x7f4ba80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbeffd640 1 -- 172.21.15.60:0/2935040643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc00b5380 msgr2=0x7f4ba80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbeffd640 1 --2- 172.21.15.60:0/2935040643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc00b5380 0x7f4ba80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbeffd640 1 -- 172.21.15.60:0/2935040643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba80039c0 msgr2=0x7f4ba8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbeffd640 1 --2- 172.21.15.60:0/2935040643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba80039c0 0x7f4ba8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbeffd640 1 -- 172.21.15.60:0/2935040643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba8003530 con 0x7f4ba8004810 2024-04-25T10:14:17.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbf7fe640 1 --2- 172.21.15.60:0/2935040643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ba80039c0 0x7f4ba8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.323+0000 7f4bbeffd640 1 --2- 172.21.15.60:0/2935040643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba8004810 0x7f4ba8004c90 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009ac0 tx=0x7f4bb401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38376ef49dcf357 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bbe7fc640 1 -- 172.21.15.60:0/2935040643 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4009690 con 0x7f4ba8004810 2024-04-25T10:14:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bbe7fc640 1 -- 172.21.15.60:0/2935040643 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb40a9450 con 0x7f4ba8004810 2024-04-25T10:14:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bbe7fc640 1 -- 172.21.15.60:0/2935040643 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40a9820 con 0x7f4ba8004810 2024-04-25T10:14:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bc5aab640 1 -- 172.21.15.60:0/2935040643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba8004810 msgr2=0x7f4ba8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bc5aab640 1 --2- 172.21.15.60:0/2935040643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ba8004810 0x7f4ba8004c90 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009ac0 tx=0x7f4bb401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bbffff640 1 -- 172.21.15.60:0/2935040643 reap_dead start 2024-04-25T10:14:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bc5aab640 1 -- 172.21.15.60:0/2935040643 shutdown_connections 2024-04-25T10:14:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bc5aab640 1 -- 172.21.15.60:0/2935040643 >> 172.21.15.60:0/2935040643 conn(0x7f4bc00d0380 msgr2=0x7f4bc00d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bc5aab640 1 -- 172.21.15.60:0/2935040643 shutdown_connections 2024-04-25T10:14:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bc5aab640 1 -- 172.21.15.60:0/2935040643 wait complete. 2024-04-25T10:14:17.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bc5aab640 1 Processor -- start 2024-04-25T10:14:17.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bc5aab640 1 -- start start 2024-04-25T10:14:17.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bc5aab640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc01572a0 0x7f4bc0157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bc5aab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc0157680 0x7f4bc015e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bc5aab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bc015fd50 0x7f4bc0160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bc5aab640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00b9770 con 0x7f4bc0157680 2024-04-25T10:14:17.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bc5aab640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc0161e80 con 0x7f4bc015fd50 2024-04-25T10:14:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bc5aab640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00b98f0 con 0x7f4bc01572a0 2024-04-25T10:14:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bbeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc0157680 0x7f4bc015e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bbeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc0157680 0x7f4bc015e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38548/0 (socket says 172.21.15.60:38548) 2024-04-25T10:14:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bbeffd640 1 -- 172.21.15.60:0/2477077877 learned_addr learned my addr 172.21.15.60:0/2477077877 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bbffff640 1 --2- 172.21.15.60:0/2477077877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bc015fd50 0x7f4bc0160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bbf7fe640 1 --2- 172.21.15.60:0/2477077877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc01572a0 0x7f4bc0157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bbeffd640 1 -- 172.21.15.60:0/2477077877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc01572a0 msgr2=0x7f4bc0157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bbeffd640 1 --2- 172.21.15.60:0/2477077877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc01572a0 0x7f4bc0157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bbeffd640 1 -- 172.21.15.60:0/2477077877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bc015fd50 msgr2=0x7f4bc0160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bbeffd640 1 --2- 172.21.15.60:0/2477077877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bc015fd50 0x7f4bc0160200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bbeffd640 1 -- 172.21.15.60:0/2477077877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc01598d0 con 0x7f4bc0157680 2024-04-25T10:14:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bbeffd640 1 --2- 172.21.15.60:0/2477077877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc0157680 0x7f4bc015e0e0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f4bb401c820 tx=0x7f4bb40197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4ba7fff640 1 -- 172.21.15.60:0/2477077877 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40bf070 con 0x7f4bc0157680 2024-04-25T10:14:17.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4ba7fff640 1 -- 172.21.15.60:0/2477077877 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb400f000 con 0x7f4bc0157680 2024-04-25T10:14:17.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4bc5aab640 1 -- 172.21.15.60:0/2477077877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc0159bb0 con 0x7f4bc0157680 2024-04-25T10:14:17.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.327+0000 7f4ba7fff640 1 -- 172.21.15.60:0/2477077877 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40ba3f0 con 0x7f4bc0157680 2024-04-25T10:14:17.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.331+0000 7f4bc5aab640 1 -- 172.21.15.60:0/2477077877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc015a170 con 0x7f4bc0157680 2024-04-25T10:14:17.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.331+0000 7f4bc5aab640 1 -- 172.21.15.60:0/2477077877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc0074770 con 0x7f4bc0157680 2024-04-25T10:14:17.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.331+0000 7f4ba7fff640 1 -- 172.21.15.60:0/2477077877 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4bb40c2070 con 0x7f4bc0157680 2024-04-25T10:14:17.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.331+0000 7f4ba7fff640 1 --2- 172.21.15.60:0/2477077877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4bac066f90 0x7f4bac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.331+0000 7f4ba7fff640 1 -- 172.21.15.60:0/2477077877 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bb411eba0 con 0x7f4bc0157680 2024-04-25T10:14:17.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.331+0000 7f4bbf7fe640 1 --2- 172.21.15.60:0/2477077877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4bac066f90 0x7f4bac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.331+0000 7f4bbf7fe640 1 --2- 172.21.15.60:0/2477077877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4bac066f90 0x7f4bac069450 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0004850 tx=0x7f4bb0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.335+0000 7f4ba7fff640 1 -- 172.21.15.60:0/2477077877 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4bb40ad020 con 0x7f4bc0157680 2024-04-25T10:14:17.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.499+0000 7f4bc5aab640 1 -- 172.21.15.60:0/2477077877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bc00d1530 con 0x7f4bc0157680 2024-04-25T10:14:17.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.499+0000 7f4ba7fff640 1 -- 172.21.15.60:0/2477077877 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16633 (secure 0 0 0) 0x7f4bb4016760 con 0x7f4bc0157680 2024-04-25T10:14:17.504 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:17.504 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":96,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:14:16.145711+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:14.444411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:15.153386+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:17.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.503+0000 7f4bc5aab640 1 -- 172.21.15.60:0/2477077877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4bac066f90 msgr2=0x7f4bac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.503+0000 7f4bc5aab640 1 --2- 172.21.15.60:0/2477077877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4bac066f90 0x7f4bac069450 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0004850 tx=0x7f4bb0009040 comp rx=0 tx=0).stop 2024-04-25T10:14:17.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.503+0000 7f4bc5aab640 1 -- 172.21.15.60:0/2477077877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc0157680 msgr2=0x7f4bc015e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.503+0000 7f4bc5aab640 1 --2- 172.21.15.60:0/2477077877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc0157680 0x7f4bc015e0e0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f4bb401c820 tx=0x7f4bb40197f0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.503+0000 7f4bc5aab640 1 -- 172.21.15.60:0/2477077877 shutdown_connections 2024-04-25T10:14:17.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.503+0000 7f4bc5aab640 1 --2- 172.21.15.60:0/2477077877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4bac066f90 0x7f4bac069450 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.503+0000 7f4bc5aab640 1 --2- 172.21.15.60:0/2477077877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bc015fd50 0x7f4bc0160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.503+0000 7f4bc5aab640 1 --2- 172.21.15.60:0/2477077877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc0157680 0x7f4bc015e0e0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.503+0000 7f4bc5aab640 1 --2- 172.21.15.60:0/2477077877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc01572a0 0x7f4bc0157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.503+0000 7f4bc5aab640 1 -- 172.21.15.60:0/2477077877 >> 172.21.15.60:0/2477077877 conn(0x7f4bc00d0380 msgr2=0x7f4bc015ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.503+0000 7f4bc5aab640 1 -- 172.21.15.60:0/2477077877 shutdown_connections 2024-04-25T10:14:17.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.503+0000 7f4bc5aab640 1 -- 172.21.15.60:0/2477077877 wait complete. 2024-04-25T10:14:17.525 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:14:17.572 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:17.575 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:17.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.651+0000 7f5883fc0640 1 Processor -- start 2024-04-25T10:14:17.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.651+0000 7f5883fc0640 1 -- start start 2024-04-25T10:14:17.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.651+0000 7f5883fc0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f587c0ceef0 0x7f587c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.651+0000 7f5883fc0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587c15bad0 con 0x7f587c1620d0 2024-04-25T10:14:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.651+0000 7f5883fc0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587c15ebe0 con 0x7f587c15b6f0 2024-04-25T10:14:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.651+0000 7f5883fc0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587c161c40 con 0x7f587c0ceef0 2024-04-25T10:14:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.651+0000 7f588155b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f587c1620d0 0x7f587c0ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45960/0 (socket says 172.21.15.60:45960) 2024-04-25T10:14:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.651+0000 7f588155b640 1 -- 172.21.15.60:0/2527288058 learned_addr learned my addr 172.21.15.60:0/2527288058 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.651+0000 7f588255d640 1 --2- 172.21.15.60:0/2527288058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f587c0ceef0 0x7f587c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 -- 172.21.15.60:0/2527288058 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882567709 0 0) 0x7f587c15bad0 con 0x7f587c1620d0 2024-04-25T10:14:17.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 -- 172.21.15.60:0/2527288058 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5868003530 con 0x7f587c1620d0 2024-04-25T10:14:17.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 -- 172.21.15.60:0/2527288058 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1301160706 0 0) 0x7f587c15ebe0 con 0x7f587c15b6f0 2024-04-25T10:14:17.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 -- 172.21.15.60:0/2527288058 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f587c15bad0 con 0x7f587c15b6f0 2024-04-25T10:14:17.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 -- 172.21.15.60:0/2527288058 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1154680437 0 0) 0x7f5868003530 con 0x7f587c1620d0 2024-04-25T10:14:17.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 -- 172.21.15.60:0/2527288058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f587c0ceef0 msgr2=0x7f587c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 --2- 172.21.15.60:0/2527288058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f587c0ceef0 0x7f587c0d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 -- 172.21.15.60:0/2527288058 >> v1:172.21.15.100:6789/0 conn(0x7f587c15b6f0 legacy=0x7f587c15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 -- 172.21.15.60:0/2527288058 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c15eec0 con 0x7f587c1620d0 2024-04-25T10:14:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 -- 172.21.15.60:0/2527288058 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f586c001680 con 0x7f587c1620d0 2024-04-25T10:14:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 -- 172.21.15.60:0/2527288058 >> v1:172.21.15.60:6789/0 conn(0x7f587c1620d0 legacy=0x7f587c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 --2- 172.21.15.60:0/2527288058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5868003b80 0x7f5868003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 --2- 172.21.15.60:0/2527288058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f587c0ceef0 0x7f5868004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 --2- 172.21.15.60:0/2527288058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5868004d50 0x7f58680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 -- 172.21.15.60:0/2527288058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587c161c40 con 0x7f5868003b80 2024-04-25T10:14:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 -- 172.21.15.60:0/2527288058 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586800ba80 con 0x7f5868004d50 2024-04-25T10:14:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 -- 172.21.15.60:0/2527288058 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586800bc80 con 0x7f587c0ceef0 2024-04-25T10:14:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5881d5c640 1 --2- 172.21.15.60:0/2527288058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5868003b80 0x7f5868003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f588155b640 1 --2- 172.21.15.60:0/2527288058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f587c0ceef0 0x7f5868004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f588255d640 1 --2- 172.21.15.60:0/2527288058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5868004d50 0x7f58680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5881d5c640 1 -- 172.21.15.60:0/2527288058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f587c0ceef0 msgr2=0x7f5868004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5881d5c640 1 --2- 172.21.15.60:0/2527288058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f587c0ceef0 0x7f5868004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5881d5c640 1 -- 172.21.15.60:0/2527288058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5868004d50 msgr2=0x7f58680051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5881d5c640 1 --2- 172.21.15.60:0/2527288058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5868004d50 0x7f58680051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5881d5c640 1 -- 172.21.15.60:0/2527288058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5868003530 con 0x7f5868003b80 2024-04-25T10:14:17.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f588155b640 1 --2- 172.21.15.60:0/2527288058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f587c0ceef0 0x7f5868004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:17.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f588255d640 1 --2- 172.21.15.60:0/2527288058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5868004d50 0x7f58680051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5881d5c640 1 --2- 172.21.15.60:0/2527288058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5868003b80 0x7f5868003f80 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f58780018d0 tx=0x7f587801c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b0f6bdfb6ae8ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 -- 172.21.15.60:0/2527288058 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878001520 con 0x7f5868003b80 2024-04-25T10:14:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 -- 172.21.15.60:0/2527288058 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58780050d0 con 0x7f5868003b80 2024-04-25T10:14:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5880d5a640 1 -- 172.21.15.60:0/2527288058 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58780054a0 con 0x7f5868003b80 2024-04-25T10:14:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5883fc0640 1 -- 172.21.15.60:0/2527288058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5868003b80 msgr2=0x7f5868003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.655+0000 7f5883fc0640 1 --2- 172.21.15.60:0/2527288058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5868003b80 0x7f5868003f80 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f58780018d0 tx=0x7f587801c580 comp rx=0 tx=0).stop 2024-04-25T10:14:17.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f588255d640 1 -- 172.21.15.60:0/2527288058 reap_dead start 2024-04-25T10:14:17.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f5883fc0640 1 -- 172.21.15.60:0/2527288058 shutdown_connections 2024-04-25T10:14:17.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f5883fc0640 1 -- 172.21.15.60:0/2527288058 >> 172.21.15.60:0/2527288058 conn(0x7f587c072340 msgr2=0x7f587c072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f5883fc0640 1 -- 172.21.15.60:0/2527288058 shutdown_connections 2024-04-25T10:14:17.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f5883fc0640 1 -- 172.21.15.60:0/2527288058 wait complete. 2024-04-25T10:14:17.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f5883fc0640 1 Processor -- start 2024-04-25T10:14:17.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f5883fc0640 1 -- start start 2024-04-25T10:14:17.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f5883fc0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5868003b80 0x7f587c0c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f5883fc0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587c0d3f30 0x7f587c15f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f5883fc0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587c150a30 0x7f587c150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f5883fc0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587c15e960 con 0x7f587c150a30 2024-04-25T10:14:17.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f5883fc0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587c15e7e0 con 0x7f587c0d3f30 2024-04-25T10:14:17.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f5883fc0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587c0d2340 con 0x7f5868003b80 2024-04-25T10:14:17.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f588255d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587c150a30 0x7f587c150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f588255d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587c150a30 0x7f587c150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38578/0 (socket says 172.21.15.60:38578) 2024-04-25T10:14:17.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f588255d640 1 -- 172.21.15.60:0/2839678912 learned_addr learned my addr 172.21.15.60:0/2839678912 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:17.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f588155b640 1 --2- 172.21.15.60:0/2839678912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587c0d3f30 0x7f587c15f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f5881d5c640 1 --2- 172.21.15.60:0/2839678912 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5868003b80 0x7f587c0c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f588255d640 1 -- 172.21.15.60:0/2839678912 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5868003b80 msgr2=0x7f587c0c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f588255d640 1 --2- 172.21.15.60:0/2839678912 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5868003b80 0x7f587c0c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f588255d640 1 -- 172.21.15.60:0/2839678912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587c0d3f30 msgr2=0x7f587c15f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f588255d640 1 --2- 172.21.15.60:0/2839678912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587c0d3f30 0x7f587c15f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f588255d640 1 -- 172.21.15.60:0/2839678912 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c0c7a80 con 0x7f587c150a30 2024-04-25T10:14:17.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f588255d640 1 --2- 172.21.15.60:0/2839678912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587c150a30 0x7f587c150ec0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f587c073030 tx=0x7f587400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f58727fc640 1 -- 172.21.15.60:0/2839678912 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587401c070 con 0x7f587c150a30 2024-04-25T10:14:17.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f58727fc640 1 -- 172.21.15.60:0/2839678912 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5874017400 con 0x7f587c150a30 2024-04-25T10:14:17.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f58727fc640 1 -- 172.21.15.60:0/2839678912 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58740177f0 con 0x7f587c150a30 2024-04-25T10:14:17.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f5883fc0640 1 -- 172.21.15.60:0/2839678912 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f587c151400 con 0x7f587c150a30 2024-04-25T10:14:17.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.659+0000 7f5883fc0640 1 -- 172.21.15.60:0/2839678912 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587c0c8000 con 0x7f587c150a30 2024-04-25T10:14:17.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.663+0000 7f5883fc0640 1 -- 172.21.15.60:0/2839678912 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f587c074770 con 0x7f587c150a30 2024-04-25T10:14:17.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.663+0000 7f58727fc640 1 -- 172.21.15.60:0/2839678912 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f587401f070 con 0x7f587c150a30 2024-04-25T10:14:17.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.663+0000 7f58727fc640 1 --2- 172.21.15.60:0/2839678912 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5860066f90 0x7f5860069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.663+0000 7f58727fc640 1 -- 172.21.15.60:0/2839678912 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5874085180 con 0x7f587c150a30 2024-04-25T10:14:17.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.663+0000 7f5881d5c640 1 --2- 172.21.15.60:0/2839678912 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5860066f90 0x7f5860069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.663+0000 7f5881d5c640 1 --2- 172.21.15.60:0/2839678912 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5860066f90 0x7f5860069450 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f58780ad000 tx=0x7f58780a6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.667+0000 7f58727fc640 1 -- 172.21.15.60:0/2839678912 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5874012070 con 0x7f587c150a30 2024-04-25T10:14:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:17.867+0000 7f5883fc0640 1 -- 172.21.15.60:0/2839678912 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f587c0d16e0 con 0x7f587c150a30 2024-04-25T10:14:18.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:18.871+0000 7f58727fc640 1 -- 172.21.15.60:0/2839678912 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v76) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5874055aa0 con 0x7f587c150a30 2024-04-25T10:14:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:18.875+0000 7f5883fc0640 1 -- 172.21.15.60:0/2839678912 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5860066f90 msgr2=0x7f5860069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:18.875+0000 7f5883fc0640 1 --2- 172.21.15.60:0/2839678912 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5860066f90 0x7f5860069450 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f58780ad000 tx=0x7f58780a6040 comp rx=0 tx=0).stop 2024-04-25T10:14:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:18.875+0000 7f5883fc0640 1 -- 172.21.15.60:0/2839678912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587c150a30 msgr2=0x7f587c150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:18.875+0000 7f5883fc0640 1 --2- 172.21.15.60:0/2839678912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587c150a30 0x7f587c150ec0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f587c073030 tx=0x7f587400f4a0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:18.875+0000 7f5883fc0640 1 -- 172.21.15.60:0/2839678912 shutdown_connections 2024-04-25T10:14:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:18.875+0000 7f5883fc0640 1 --2- 172.21.15.60:0/2839678912 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5860066f90 0x7f5860069450 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:18.875+0000 7f5883fc0640 1 --2- 172.21.15.60:0/2839678912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587c150a30 0x7f587c150ec0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:18.875+0000 7f5883fc0640 1 --2- 172.21.15.60:0/2839678912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587c0d3f30 0x7f587c15f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:18.875+0000 7f5883fc0640 1 --2- 172.21.15.60:0/2839678912 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5868003b80 0x7f587c0c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:18.875+0000 7f5883fc0640 1 -- 172.21.15.60:0/2839678912 >> 172.21.15.60:0/2839678912 conn(0x7f587c072340 msgr2=0x7f587c152020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:18.875+0000 7f5883fc0640 1 -- 172.21.15.60:0/2839678912 shutdown_connections 2024-04-25T10:14:18.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:18.875+0000 7f5883fc0640 1 -- 172.21.15.60:0/2839678912 wait complete. 2024-04-25T10:14:18.907 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:14:18.908 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:14:19.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.083+0000 7ff592e20640 1 Processor -- start 2024-04-25T10:14:19.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.083+0000 7ff592e20640 1 -- start start 2024-04-25T10:14:19.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff592e20640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff58c15dd30 0x7ff58c15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff592e20640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff58c0cf8a0 0x7ff58c15be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff592e20640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58c0cd890 con 0x7ff58c0cd3d0 2024-04-25T10:14:19.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff592e20640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff58c0ce4a0 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff592e20640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff58c0cdb70 con 0x7ff58c15dd30 2024-04-25T10:14:19.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff5913bd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff58c0cf8a0 0x7ff58c15be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff5913bd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff58c0cf8a0 0x7ff58c15be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47428/0 (socket says 172.21.15.60:47428) 2024-04-25T10:14:19.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff5913bd640 1 -- 172.21.15.60:0/1425661597 learned_addr learned my addr 172.21.15.60:0/1425661597 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:19.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff58bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff58c0cd3d0 0x7ff58c0cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55460/0 (socket says 172.21.15.60:55460) 2024-04-25T10:14:19.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff590bbc640 1 --2- 172.21.15.60:0/1425661597 >> v2:172.21.15.100:3300/0 conn(0x7ff58c15dd30 0x7ff58c15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff5913bd640 1 -- 172.21.15.60:0/1425661597 >> v2:172.21.15.100:3300/0 conn(0x7ff58c15dd30 msgr2=0x7ff58c15ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff5913bd640 1 --2- 172.21.15.60:0/1425661597 >> v2:172.21.15.100:3300/0 conn(0x7ff58c15dd30 0x7ff58c15ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff5913bd640 1 -- 172.21.15.60:0/1425661597 >> v1:172.21.15.60:6789/0 conn(0x7ff58c0cd3d0 legacy=0x7ff58c0cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff5913bd640 1 -- 172.21.15.60:0/1425661597 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c0cde50 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff58b7fe640 1 -- 172.21.15.60:0/1425661597 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480925179 0 0) 0x7ff58c0cd890 con 0x7ff58c0cd3d0 2024-04-25T10:14:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff5913bd640 1 --2- 172.21.15.60:0/1425661597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58c0cf8a0 0x7ff58c15be20 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7ff58c06bbf0 tx=0x7ff57c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1222a482f9fba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff58b7fe640 1 -- 172.21.15.60:0/1425661597 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0ab680 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff58b7fe640 1 -- 172.21.15.60:0/1425661597 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c0b0070 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.087+0000 7ff58b7fe640 1 -- 172.21.15.60:0/1425661597 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0aba40 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 -- 172.21.15.60:0/1425661597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58c0cf8a0 msgr2=0x7ff58c15be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 --2- 172.21.15.60:0/1425661597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58c0cf8a0 0x7ff58c15be20 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7ff58c06bbf0 tx=0x7ff57c01c460 comp rx=0 tx=0).stop 2024-04-25T10:14:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 -- 172.21.15.60:0/1425661597 shutdown_connections 2024-04-25T10:14:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 --2- 172.21.15.60:0/1425661597 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58c0cf8a0 0x7ff58c15be20 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 --2- 172.21.15.60:0/1425661597 >> v2:172.21.15.100:3300/0 conn(0x7ff58c15dd30 0x7ff58c15ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 -- 172.21.15.60:0/1425661597 >> 172.21.15.60:0/1425661597 conn(0x7ff58c0670b0 msgr2=0x7ff58c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 -- 172.21.15.60:0/1425661597 shutdown_connections 2024-04-25T10:14:19.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 -- 172.21.15.60:0/1425661597 wait complete. 2024-04-25T10:14:19.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 Processor -- start 2024-04-25T10:14:19.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 -- start start 2024-04-25T10:14:19.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff58c0cd3d0 0x7ff58c13eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58c0cf8a0 0x7ff58c0a8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff58c15dd30 0x7ff58c141c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c15cf00 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c15cd80 con 0x7ff58c0cd3d0 2024-04-25T10:14:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c15d080 con 0x7ff58c15dd30 2024-04-25T10:14:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff58bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58c0cf8a0 0x7ff58c0a8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff58bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58c0cf8a0 0x7ff58c0a8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47432/0 (socket says 172.21.15.60:47432) 2024-04-25T10:14:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff58bfff640 1 -- 172.21.15.60:0/1115406082 learned_addr learned my addr 172.21.15.60:0/1115406082 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff590bbc640 1 --2- 172.21.15.60:0/1115406082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff58c0cd3d0 0x7ff58c13eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff5913bd640 1 --2- 172.21.15.60:0/1115406082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff58c15dd30 0x7ff58c141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff58bfff640 1 -- 172.21.15.60:0/1115406082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff58c15dd30 msgr2=0x7ff58c141c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff58bfff640 1 --2- 172.21.15.60:0/1115406082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff58c15dd30 0x7ff58c141c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff58bfff640 1 -- 172.21.15.60:0/1115406082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff58c0cd3d0 msgr2=0x7ff58c13eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff58bfff640 1 --2- 172.21.15.60:0/1115406082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff58c0cd3d0 0x7ff58c13eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff58bfff640 1 -- 172.21.15.60:0/1115406082 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c0a89d0 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff58bfff640 1 --2- 172.21.15.60:0/1115406082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58c0cf8a0 0x7ff58c0a8490 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7ff58000c890 tx=0x7ff58000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff5897fa640 1 -- 172.21.15.60:0/1115406082 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580018070 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff5897fa640 1 -- 172.21.15.60:0/1115406082 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58000a640 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 -- 172.21.15.60:0/1115406082 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58c142170 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff5897fa640 1 -- 172.21.15.60:0/1115406082 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58000aa30 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.091+0000 7ff592e20640 1 -- 172.21.15.60:0/1115406082 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58c13f630 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.095+0000 7ff592e20640 1 -- 172.21.15.60:0/1115406082 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58c074770 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.095+0000 7ff5897fa640 1 -- 172.21.15.60:0/1115406082 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff58001b070 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.095+0000 7ff5897fa640 1 --2- 172.21.15.60:0/1115406082 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff578066f90 0x7ff578069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.095+0000 7ff5897fa640 1 -- 172.21.15.60:0/1115406082 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff580084740 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.095+0000 7ff590bbc640 1 --2- 172.21.15.60:0/1115406082 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff578066f90 0x7ff578069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.095+0000 7ff590bbc640 1 --2- 172.21.15.60:0/1115406082 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff578066f90 0x7ff578069450 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff58c1f7be0 tx=0x7ff584008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.099+0000 7ff5897fa640 1 -- 172.21.15.60:0/1115406082 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff58c142170 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.299+0000 7ff592e20640 1 -- 172.21.15.60:0/1115406082 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff58c0cec50 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.875+0000 7ff5897fa640 1 -- 172.21.15.60:0/1115406082 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v77) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff5800556b0 con 0x7ff58c0cf8a0 2024-04-25T10:14:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.879+0000 7ff592e20640 1 -- 172.21.15.60:0/1115406082 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff578066f90 msgr2=0x7ff578069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.879+0000 7ff592e20640 1 --2- 172.21.15.60:0/1115406082 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff578066f90 0x7ff578069450 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff58c1f7be0 tx=0x7ff584008040 comp rx=0 tx=0).stop 2024-04-25T10:14:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.879+0000 7ff592e20640 1 -- 172.21.15.60:0/1115406082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58c0cf8a0 msgr2=0x7ff58c0a8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.879+0000 7ff592e20640 1 --2- 172.21.15.60:0/1115406082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58c0cf8a0 0x7ff58c0a8490 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7ff58000c890 tx=0x7ff58000cd50 comp rx=0 tx=0).stop 2024-04-25T10:14:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.879+0000 7ff592e20640 1 -- 172.21.15.60:0/1115406082 shutdown_connections 2024-04-25T10:14:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.879+0000 7ff592e20640 1 --2- 172.21.15.60:0/1115406082 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff578066f90 0x7ff578069450 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.879+0000 7ff592e20640 1 --2- 172.21.15.60:0/1115406082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff58c15dd30 0x7ff58c141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.879+0000 7ff592e20640 1 --2- 172.21.15.60:0/1115406082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58c0cf8a0 0x7ff58c0a8490 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.879+0000 7ff592e20640 1 --2- 172.21.15.60:0/1115406082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff58c0cd3d0 0x7ff58c13eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.879+0000 7ff592e20640 1 -- 172.21.15.60:0/1115406082 >> 172.21.15.60:0/1115406082 conn(0x7ff58c0670b0 msgr2=0x7ff58c15d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:19.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.879+0000 7ff592e20640 1 -- 172.21.15.60:0/1115406082 shutdown_connections 2024-04-25T10:14:19.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:19.879+0000 7ff592e20640 1 -- 172.21.15.60:0/1115406082 wait complete. 2024-04-25T10:14:19.903 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:14:19.904 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:20.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.027+0000 7f4cc4ef3640 1 Processor -- start 2024-04-25T10:14:20.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.027+0000 7f4cc4ef3640 1 -- start start 2024-04-25T10:14:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.031+0000 7f4cc4ef3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4cc015b560 0x7f4cc00701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.031+0000 7f4cc4ef3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4cc015b940 0x7f4cc006dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.031+0000 7f4cc4ef3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cc006e470 con 0x7f4cc006bc20 2024-04-25T10:14:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.031+0000 7f4cc4ef3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc00b31a0 con 0x7f4cc015b560 2024-04-25T10:14:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.031+0000 7f4cc4ef3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc006e210 con 0x7f4cc015b940 2024-04-25T10:14:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.031+0000 7f4cbe59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4cc015b560 0x7f4cc00701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.031+0000 7f4cbed9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4cc006bc20 0x7f4cc015f350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55464/0 (socket says 172.21.15.60:55464) 2024-04-25T10:14:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.031+0000 7f4cbed9d640 1 -- 172.21.15.60:0/1731573510 learned_addr learned my addr 172.21.15.60:0/1731573510 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.031+0000 7f4cbdd9b640 1 --2- 172.21.15.60:0/1731573510 >> v2:172.21.15.100:3300/0 conn(0x7f4cc015b940 0x7f4cc006dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.031+0000 7f4cbe59c640 1 -- 172.21.15.60:0/1731573510 >> v2:172.21.15.100:3300/0 conn(0x7f4cc015b940 msgr2=0x7f4cc006dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.031+0000 7f4cbe59c640 1 --2- 172.21.15.60:0/1731573510 >> v2:172.21.15.100:3300/0 conn(0x7f4cc015b940 0x7f4cc006dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.031+0000 7f4cbe59c640 1 -- 172.21.15.60:0/1731573510 >> v1:172.21.15.60:6789/0 conn(0x7f4cc006bc20 legacy=0x7f4cc015f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.031+0000 7f4cbe59c640 1 -- 172.21.15.60:0/1731573510 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc006e750 con 0x7f4cc015b560 2024-04-25T10:14:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.031+0000 7f4cbd59a640 1 -- 172.21.15.60:0/1731573510 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2581542411 0 0) 0x7f4cc006e470 con 0x7f4cc006bc20 2024-04-25T10:14:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cbe59c640 1 --2- 172.21.15.60:0/1731573510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cc015b560 0x7f4cc00701e0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0067420 tx=0x7f4cb001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180f10da29fb08fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cbd59a640 1 -- 172.21.15.60:0/1731573510 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb00a7070 con 0x7f4cc015b560 2024-04-25T10:14:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cbd59a640 1 -- 172.21.15.60:0/1731573510 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb00b0070 con 0x7f4cc015b560 2024-04-25T10:14:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cbd59a640 1 -- 172.21.15.60:0/1731573510 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb00ab890 con 0x7f4cc015b560 2024-04-25T10:14:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cc4ef3640 1 -- 172.21.15.60:0/1731573510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cc015b560 msgr2=0x7f4cc00701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cc4ef3640 1 --2- 172.21.15.60:0/1731573510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cc015b560 0x7f4cc00701e0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0067420 tx=0x7f4cb001c450 comp rx=0 tx=0).stop 2024-04-25T10:14:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cc4ef3640 1 -- 172.21.15.60:0/1731573510 shutdown_connections 2024-04-25T10:14:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cc4ef3640 1 --2- 172.21.15.60:0/1731573510 >> v2:172.21.15.100:3300/0 conn(0x7f4cc015b940 0x7f4cc006dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cc4ef3640 1 --2- 172.21.15.60:0/1731573510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cc015b560 0x7f4cc00701e0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cc4ef3640 1 -- 172.21.15.60:0/1731573510 >> 172.21.15.60:0/1731573510 conn(0x7f4cc0071b90 msgr2=0x7f4cc0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cc4ef3640 1 -- 172.21.15.60:0/1731573510 shutdown_connections 2024-04-25T10:14:20.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cc4ef3640 1 -- 172.21.15.60:0/1731573510 wait complete. 2024-04-25T10:14:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cc4ef3640 1 Processor -- start 2024-04-25T10:14:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cc4ef3640 1 -- start start 2024-04-25T10:14:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cc4ef3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cc006bc20 0x7f4cc00b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cc4ef3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4cc015b560 0x7f4cc0141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cc4ef3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cc015b940 0x7f4cc0156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cc4ef3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc0168660 con 0x7f4cc015b940 2024-04-25T10:14:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cc4ef3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc01684e0 con 0x7f4cc006bc20 2024-04-25T10:14:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cc4ef3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc01687e0 con 0x7f4cc015b560 2024-04-25T10:14:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cbed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cc015b940 0x7f4cc0156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cbed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cc015b940 0x7f4cc0156980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47438/0 (socket says 172.21.15.60:47438) 2024-04-25T10:14:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cbed9d640 1 -- 172.21.15.60:0/3422777037 learned_addr learned my addr 172.21.15.60:0/3422777037 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cbe59c640 1 --2- 172.21.15.60:0/3422777037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cc006bc20 0x7f4cc00b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cbdd9b640 1 --2- 172.21.15.60:0/3422777037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4cc015b560 0x7f4cc0141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cbed9d640 1 -- 172.21.15.60:0/3422777037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4cc015b560 msgr2=0x7f4cc0141ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cbed9d640 1 --2- 172.21.15.60:0/3422777037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4cc015b560 0x7f4cc0141ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cbed9d640 1 -- 172.21.15.60:0/3422777037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cc006bc20 msgr2=0x7f4cc00b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cbed9d640 1 --2- 172.21.15.60:0/3422777037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cc006bc20 0x7f4cc00b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cbed9d640 1 -- 172.21.15.60:0/3422777037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc0142910 con 0x7f4cc015b940 2024-04-25T10:14:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4cbed9d640 1 --2- 172.21.15.60:0/3422777037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cc015b940 0x7f4cc0156980 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800c810 tx=0x7f4cb800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.035+0000 7f4caeffd640 1 -- 172.21.15.60:0/3422777037 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8018070 con 0x7f4cc015b940 2024-04-25T10:14:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.039+0000 7f4caeffd640 1 -- 172.21.15.60:0/3422777037 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb800a640 con 0x7f4cc015b940 2024-04-25T10:14:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.039+0000 7f4cc4ef3640 1 -- 172.21.15.60:0/3422777037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc00b09f0 con 0x7f4cc015b940 2024-04-25T10:14:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.039+0000 7f4caeffd640 1 -- 172.21.15.60:0/3422777037 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb800aa30 con 0x7f4cc015b940 2024-04-25T10:14:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.039+0000 7f4cc4ef3640 1 -- 172.21.15.60:0/3422777037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc00b0fb0 con 0x7f4cc015b940 2024-04-25T10:14:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.039+0000 7f4cc4ef3640 1 -- 172.21.15.60:0/3422777037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc0074770 con 0x7f4cc015b940 2024-04-25T10:14:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.039+0000 7f4caeffd640 1 -- 172.21.15.60:0/3422777037 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4cb801b070 con 0x7f4cc015b940 2024-04-25T10:14:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.039+0000 7f4caeffd640 1 --2- 172.21.15.60:0/3422777037 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4ca0066f60 0x7f4ca0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.039+0000 7f4caeffd640 1 -- 172.21.15.60:0/3422777037 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4cb80848e0 con 0x7f4cc015b940 2024-04-25T10:14:20.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.039+0000 7f4cbe59c640 1 --2- 172.21.15.60:0/3422777037 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4ca0066f60 0x7f4ca0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.039+0000 7f4cbe59c640 1 --2- 172.21.15.60:0/3422777037 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4ca0066f60 0x7f4ca0069420 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00af000 tx=0x7f4cb00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.043+0000 7f4caeffd640 1 -- 172.21.15.60:0/3422777037 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4cb80559d0 con 0x7f4cc015b940 2024-04-25T10:14:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.239+0000 7f4cc4ef3640 1 -- 172.21.15.60:0/3422777037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cc006cb50 con 0x7f4cc015b940 2024-04-25T10:14:20.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.239+0000 7f4caeffd640 1 -- 172.21.15.60:0/3422777037 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f4cb80552d0 con 0x7f4cc015b940 2024-04-25T10:14:20.245 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:20.245 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1491228634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1491228634},{"type":"v1","addr":"172.21.15.100:6837","nonce":1491228634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/209898872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":209898872},{"type":"v1","addr":"172.21.15.100:6835","nonce":209898872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1793343287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1793343287},{"type":"v1","addr":"172.21.15.60:6833","nonce":1793343287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:16.146481+0000","modified":"2024-04-25T10:14:19.300954+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2002755047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2002755047},{"type":"v1","addr":"172.21.15.60:6835","nonce":2002755047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:14:20.245 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 77 2024-04-25T10:14:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.243+0000 7f4cc4ef3640 1 -- 172.21.15.60:0/3422777037 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4ca0066f60 msgr2=0x7f4ca0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.243+0000 7f4cc4ef3640 1 --2- 172.21.15.60:0/3422777037 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4ca0066f60 0x7f4ca0069420 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00af000 tx=0x7f4cb00a7000 comp rx=0 tx=0).stop 2024-04-25T10:14:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.243+0000 7f4cc4ef3640 1 -- 172.21.15.60:0/3422777037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cc015b940 msgr2=0x7f4cc0156980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.243+0000 7f4cc4ef3640 1 --2- 172.21.15.60:0/3422777037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cc015b940 0x7f4cc0156980 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800c810 tx=0x7f4cb800ccd0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.243+0000 7f4cc4ef3640 1 -- 172.21.15.60:0/3422777037 shutdown_connections 2024-04-25T10:14:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.243+0000 7f4cc4ef3640 1 --2- 172.21.15.60:0/3422777037 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4ca0066f60 0x7f4ca0069420 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.243+0000 7f4cc4ef3640 1 --2- 172.21.15.60:0/3422777037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cc015b940 0x7f4cc0156980 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.243+0000 7f4cc4ef3640 1 --2- 172.21.15.60:0/3422777037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4cc015b560 0x7f4cc0141ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.243+0000 7f4cc4ef3640 1 --2- 172.21.15.60:0/3422777037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cc006bc20 0x7f4cc00b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.243+0000 7f4cc4ef3640 1 -- 172.21.15.60:0/3422777037 >> 172.21.15.60:0/3422777037 conn(0x7f4cc0071b90 msgr2=0x7f4cc015be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.243+0000 7f4cc4ef3640 1 -- 172.21.15.60:0/3422777037 shutdown_connections 2024-04-25T10:14:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.243+0000 7f4cc4ef3640 1 -- 172.21.15.60:0/3422777037 wait complete. 2024-04-25T10:14:20.265 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:20.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.387+0000 7f15924fb640 1 Processor -- start 2024-04-25T10:14:20.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.387+0000 7f15924fb640 1 -- start start 2024-04-25T10:14:20.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.387+0000 7f15924fb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f158c162250 0x7f158c0b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.387+0000 7f15924fb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f158c0719a0 0x7f158c0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.387+0000 7f15924fb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158c0b3aa0 con 0x7f158c0b34f0 2024-04-25T10:14:20.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.387+0000 7f15924fb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158c0af0c0 con 0x7f158c162250 2024-04-25T10:14:20.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.387+0000 7f15924fb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158c0b3d80 con 0x7f158c0719a0 2024-04-25T10:14:20.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.387+0000 7f158bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f158c162250 0x7f158c0b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.387+0000 7f158bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f158c162250 0x7f158c0b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47440/0 (socket says 172.21.15.60:47440) 2024-04-25T10:14:20.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.387+0000 7f15837fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f158c0b34f0 0x7f158c071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55476/0 (socket says 172.21.15.60:55476) 2024-04-25T10:14:20.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.387+0000 7f158bfff640 1 -- 172.21.15.60:0/2822006577 learned_addr learned my addr 172.21.15.60:0/2822006577 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:20.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.387+0000 7f1590a98640 1 --2- 172.21.15.60:0/2822006577 >> v2:172.21.15.100:3300/0 conn(0x7f158c0719a0 0x7f158c0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f158bfff640 1 -- 172.21.15.60:0/2822006577 >> v2:172.21.15.100:3300/0 conn(0x7f158c0719a0 msgr2=0x7f158c0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f158bfff640 1 --2- 172.21.15.60:0/2822006577 >> v2:172.21.15.100:3300/0 conn(0x7f158c0719a0 0x7f158c0cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f158bfff640 1 -- 172.21.15.60:0/2822006577 >> v1:172.21.15.60:6789/0 conn(0x7f158c0b34f0 legacy=0x7f158c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f158bfff640 1 -- 172.21.15.60:0/2822006577 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c0b4060 con 0x7f158c162250 2024-04-25T10:14:20.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f158b7fe640 1 -- 172.21.15.60:0/2822006577 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203977943 0 0) 0x7f158c0b3aa0 con 0x7f158c0b34f0 2024-04-25T10:14:20.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f158bfff640 1 --2- 172.21.15.60:0/2822006577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f158c162250 0x7f158c0b2fb0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f158c068540 tx=0x7f157c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fc5675f13019c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f158b7fe640 1 -- 172.21.15.60:0/2822006577 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c0b5070 con 0x7f158c162250 2024-04-25T10:14:20.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f158b7fe640 1 -- 172.21.15.60:0/2822006577 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c0a8a10 con 0x7f158c162250 2024-04-25T10:14:20.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f158b7fe640 1 -- 172.21.15.60:0/2822006577 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c0a8de0 con 0x7f158c162250 2024-04-25T10:14:20.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f15924fb640 1 -- 172.21.15.60:0/2822006577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f158c162250 msgr2=0x7f158c0b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f15924fb640 1 --2- 172.21.15.60:0/2822006577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f158c162250 0x7f158c0b2fb0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f158c068540 tx=0x7f157c01ed10 comp rx=0 tx=0).stop 2024-04-25T10:14:20.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f15924fb640 1 -- 172.21.15.60:0/2822006577 shutdown_connections 2024-04-25T10:14:20.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f15924fb640 1 --2- 172.21.15.60:0/2822006577 >> v2:172.21.15.100:3300/0 conn(0x7f158c0719a0 0x7f158c0cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f15924fb640 1 --2- 172.21.15.60:0/2822006577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f158c162250 0x7f158c0b2fb0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f15924fb640 1 -- 172.21.15.60:0/2822006577 >> 172.21.15.60:0/2822006577 conn(0x7f158c0b12d0 msgr2=0x7f158c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f15924fb640 1 -- 172.21.15.60:0/2822006577 shutdown_connections 2024-04-25T10:14:20.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.391+0000 7f15924fb640 1 -- 172.21.15.60:0/2822006577 wait complete. 2024-04-25T10:14:20.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15924fb640 1 Processor -- start 2024-04-25T10:14:20.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15924fb640 1 -- start start 2024-04-25T10:14:20.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15924fb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f158c0719a0 0x7f158c1598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15924fb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f158c0b34f0 0x7f158c15de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15924fb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f158c162250 0x7f158c15e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15924fb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c0cc440 con 0x7f158c0719a0 2024-04-25T10:14:20.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15924fb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c06e5a0 con 0x7f158c162250 2024-04-25T10:14:20.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15924fb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c0cced0 con 0x7f158c0b34f0 2024-04-25T10:14:20.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f158bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f158c0719a0 0x7f158c1598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f158bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f158c0719a0 0x7f158c1598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47444/0 (socket says 172.21.15.60:47444) 2024-04-25T10:14:20.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f158bfff640 1 -- 172.21.15.60:0/2731842595 learned_addr learned my addr 172.21.15.60:0/2731842595 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:20.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f158bfff640 1 -- 172.21.15.60:0/2731842595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f158c0b34f0 msgr2=0x7f158c15de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f158bfff640 1 --2- 172.21.15.60:0/2731842595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f158c0b34f0 0x7f158c15de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f158bfff640 1 -- 172.21.15.60:0/2731842595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f158c162250 msgr2=0x7f158c15e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f1590a98640 1 --2- 172.21.15.60:0/2731842595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f158c162250 0x7f158c15e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f158bfff640 1 --2- 172.21.15.60:0/2731842595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f158c162250 0x7f158c15e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f158bfff640 1 -- 172.21.15.60:0/2731842595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c163820 con 0x7f158c0719a0 2024-04-25T10:14:20.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f1590a98640 1 --2- 172.21.15.60:0/2731842595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f158c162250 0x7f158c15e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:20.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f158bfff640 1 --2- 172.21.15.60:0/2731842595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f158c0719a0 0x7f158c1598b0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f157c01b920 tx=0x7f157c01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15897fa640 1 -- 172.21.15.60:0/2731842595 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c0b5070 con 0x7f158c0719a0 2024-04-25T10:14:20.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15897fa640 1 -- 172.21.15.60:0/2731842595 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c0b0ab0 con 0x7f158c0719a0 2024-04-25T10:14:20.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15924fb640 1 -- 172.21.15.60:0/2731842595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158c163b00 con 0x7f158c0719a0 2024-04-25T10:14:20.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15897fa640 1 -- 172.21.15.60:0/2731842595 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c0bb410 con 0x7f158c0719a0 2024-04-25T10:14:20.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15924fb640 1 -- 172.21.15.60:0/2731842595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158c1640c0 con 0x7f158c0719a0 2024-04-25T10:14:20.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15924fb640 1 -- 172.21.15.60:0/2731842595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158c074770 con 0x7f158c0719a0 2024-04-25T10:14:20.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15897fa640 1 -- 172.21.15.60:0/2731842595 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f157c0c2060 con 0x7f158c0719a0 2024-04-25T10:14:20.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15897fa640 1 --2- 172.21.15.60:0/2731842595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f156c066f60 0x7f156c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.395+0000 7f15897fa640 1 -- 172.21.15.60:0/2731842595 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f157c122010 con 0x7f158c0719a0 2024-04-25T10:14:20.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.399+0000 7f15837fe640 1 --2- 172.21.15.60:0/2731842595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f156c066f60 0x7f156c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.399+0000 7f15837fe640 1 --2- 172.21.15.60:0/2731842595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f156c066f60 0x7f156c069420 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f15740011f0 tx=0x7f1574007930 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.403+0000 7f15897fa640 1 -- 172.21.15.60:0/2731842595 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f157c018870 con 0x7f158c0719a0 2024-04-25T10:14:20.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.563+0000 7f15924fb640 1 -- 172.21.15.60:0/2731842595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f158c0b4580 con 0x7f158c0719a0 2024-04-25T10:14:20.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.563+0000 7f15897fa640 1 -- 172.21.15.60:0/2731842595 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f157c0f30b0 con 0x7f158c0719a0 2024-04-25T10:14:20.569 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:20.570 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":97,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:14:17.788631+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:14.444411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:15.153386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.567+0000 7f15924fb640 1 -- 172.21.15.60:0/2731842595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f156c066f60 msgr2=0x7f156c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.567+0000 7f15924fb640 1 --2- 172.21.15.60:0/2731842595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f156c066f60 0x7f156c069420 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f15740011f0 tx=0x7f1574007930 comp rx=0 tx=0).stop 2024-04-25T10:14:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.567+0000 7f15924fb640 1 -- 172.21.15.60:0/2731842595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f158c0719a0 msgr2=0x7f158c1598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.567+0000 7f15924fb640 1 --2- 172.21.15.60:0/2731842595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f158c0719a0 0x7f158c1598b0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f157c01b920 tx=0x7f157c01bab0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.567+0000 7f15924fb640 1 -- 172.21.15.60:0/2731842595 shutdown_connections 2024-04-25T10:14:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.567+0000 7f15924fb640 1 --2- 172.21.15.60:0/2731842595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f156c066f60 0x7f156c069420 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.567+0000 7f15924fb640 1 --2- 172.21.15.60:0/2731842595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f158c162250 0x7f158c15e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.567+0000 7f15924fb640 1 --2- 172.21.15.60:0/2731842595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f158c0b34f0 0x7f158c15de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.567+0000 7f15924fb640 1 --2- 172.21.15.60:0/2731842595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f158c0719a0 0x7f158c1598b0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.567+0000 7f15924fb640 1 -- 172.21.15.60:0/2731842595 >> 172.21.15.60:0/2731842595 conn(0x7f158c0b12d0 msgr2=0x7f158c0b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.567+0000 7f15924fb640 1 -- 172.21.15.60:0/2731842595 shutdown_connections 2024-04-25T10:14:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.567+0000 7f15924fb640 1 -- 172.21.15.60:0/2731842595 wait complete. 2024-04-25T10:14:20.589 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:14:20.590 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:14:20.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c694a1640 1 Processor -- start 2024-04-25T10:14:20.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c694a1640 1 -- start start 2024-04-25T10:14:20.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c694a1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8c6406ec30 0x7f8c640cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c694a1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c6406f130 con 0x7f8c6415a7a0 2024-04-25T10:14:20.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c694a1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c6406f410 con 0x7f8c640b37e0 2024-04-25T10:14:20.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c694a1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c640cdaa0 con 0x7f8c6406ec30 2024-04-25T10:14:20.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c62ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8c6415a7a0 0x7f8c64071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55492/0 (socket says 172.21.15.60:55492) 2024-04-25T10:14:20.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c62ffd640 1 -- 172.21.15.60:0/807529921 learned_addr learned my addr 172.21.15.60:0/807529921 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:20.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c627fc640 1 --2- 172.21.15.60:0/807529921 >> v2:172.21.15.100:3300/0 conn(0x7f8c6406ec30 0x7f8c640cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c61ffb640 1 -- 172.21.15.60:0/807529921 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163633587 0 0) 0x7f8c6406f130 con 0x7f8c6415a7a0 2024-04-25T10:14:20.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c61ffb640 1 -- 172.21.15.60:0/807529921 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c4c003530 con 0x7f8c6415a7a0 2024-04-25T10:14:20.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c61ffb640 1 -- 172.21.15.60:0/807529921 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4216851885 0 0) 0x7f8c4c003530 con 0x7f8c6415a7a0 2024-04-25T10:14:20.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c61ffb640 1 -- 172.21.15.60:0/807529921 >> v2:172.21.15.100:3300/0 conn(0x7f8c6406ec30 msgr2=0x7f8c640cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c61ffb640 1 --2- 172.21.15.60:0/807529921 >> v2:172.21.15.100:3300/0 conn(0x7f8c6406ec30 0x7f8c640cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c61ffb640 1 -- 172.21.15.60:0/807529921 >> v1:172.21.15.100:6789/0 conn(0x7f8c640b37e0 legacy=0x7f8c6406dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c61ffb640 1 -- 172.21.15.60:0/807529921 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6406d540 con 0x7f8c6415a7a0 2024-04-25T10:14:20.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c61ffb640 1 -- 172.21.15.60:0/807529921 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f8c54001680 con 0x7f8c6415a7a0 2024-04-25T10:14:20.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c61ffb640 1 -- 172.21.15.60:0/807529921 >> v1:172.21.15.60:6789/0 conn(0x7f8c6415a7a0 legacy=0x7f8c64071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c61ffb640 1 --2- 172.21.15.60:0/807529921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c003a40 0x7f8c4c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c61ffb640 1 --2- 172.21.15.60:0/807529921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c4c004810 0x7f8c4c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c61ffb640 1 --2- 172.21.15.60:0/807529921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c4c0051d0 0x7f8c4c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c61ffb640 1 -- 172.21.15.60:0/807529921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c00c0d0 con 0x7f8c4c003a40 2024-04-25T10:14:20.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c61ffb640 1 -- 172.21.15.60:0/807529921 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c00c280 con 0x7f8c4c004810 2024-04-25T10:14:20.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c61ffb640 1 -- 172.21.15.60:0/807529921 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c00c480 con 0x7f8c4c0051d0 2024-04-25T10:14:20.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c62ffd640 1 --2- 172.21.15.60:0/807529921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c003a40 0x7f8c4c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c627fc640 1 --2- 172.21.15.60:0/807529921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c4c004810 0x7f8c4c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c62ffd640 1 -- 172.21.15.60:0/807529921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c4c0051d0 msgr2=0x7f8c4c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c62ffd640 1 --2- 172.21.15.60:0/807529921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c4c0051d0 0x7f8c4c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c62ffd640 1 -- 172.21.15.60:0/807529921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c4c004810 msgr2=0x7f8c4c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c62ffd640 1 --2- 172.21.15.60:0/807529921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c4c004810 0x7f8c4c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c62ffd640 1 -- 172.21.15.60:0/807529921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c003530 con 0x7f8c4c003a40 2024-04-25T10:14:20.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.703+0000 7f8c637fe640 1 -- 172.21.15.60:0/807529921 reap_dead start 2024-04-25T10:14:20.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.707+0000 7f8c62ffd640 1 --2- 172.21.15.60:0/807529921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c003a40 0x7f8c4c003e60 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400c110 tx=0x7f8c5401cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23edb98308a65105 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.707+0000 7f8c61ffb640 1 -- 172.21.15.60:0/807529921 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54007c50 con 0x7f8c4c003a40 2024-04-25T10:14:20.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.707+0000 7f8c61ffb640 1 -- 172.21.15.60:0/807529921 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c540a9070 con 0x7f8c4c003a40 2024-04-25T10:14:20.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.707+0000 7f8c61ffb640 1 -- 172.21.15.60:0/807529921 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54005d80 con 0x7f8c4c003a40 2024-04-25T10:14:20.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.707+0000 7f8c694a1640 1 -- 172.21.15.60:0/807529921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c003a40 msgr2=0x7f8c4c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.707+0000 7f8c694a1640 1 --2- 172.21.15.60:0/807529921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c003a40 0x7f8c4c003e60 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400c110 tx=0x7f8c5401cbc0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.707+0000 7f8c694a1640 1 -- 172.21.15.60:0/807529921 shutdown_connections 2024-04-25T10:14:20.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.707+0000 7f8c694a1640 1 --2- 172.21.15.60:0/807529921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c003a40 0x7f8c4c003e60 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.707+0000 7f8c694a1640 1 -- 172.21.15.60:0/807529921 >> 172.21.15.60:0/807529921 conn(0x7f8c640b12d0 msgr2=0x7f8c640b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.707+0000 7f8c694a1640 1 -- 172.21.15.60:0/807529921 shutdown_connections 2024-04-25T10:14:20.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.707+0000 7f8c694a1640 1 -- 172.21.15.60:0/807529921 wait complete. 2024-04-25T10:14:20.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.707+0000 7f8c694a1640 1 Processor -- start 2024-04-25T10:14:20.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.707+0000 7f8c694a1640 1 -- start start 2024-04-25T10:14:20.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c694a1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c003a40 0x7f8c641f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c694a1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c641f9c90 0x7f8c6414e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c694a1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6414ebc0 0x7f8c641f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c694a1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6406e8c0 con 0x7f8c4c003a40 2024-04-25T10:14:20.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c694a1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6406e740 con 0x7f8c6414ebc0 2024-04-25T10:14:20.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c694a1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6406d2f0 con 0x7f8c641f9c90 2024-04-25T10:14:20.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c62ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c003a40 0x7f8c641f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c62ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c003a40 0x7f8c641f8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47460/0 (socket says 172.21.15.60:47460) 2024-04-25T10:14:20.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c62ffd640 1 -- 172.21.15.60:0/1873719787 learned_addr learned my addr 172.21.15.60:0/1873719787 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:20.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c637fe640 1 --2- 172.21.15.60:0/1873719787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6414ebc0 0x7f8c641f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c627fc640 1 --2- 172.21.15.60:0/1873719787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c641f9c90 0x7f8c6414e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c62ffd640 1 -- 172.21.15.60:0/1873719787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c641f9c90 msgr2=0x7f8c6414e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c62ffd640 1 --2- 172.21.15.60:0/1873719787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c641f9c90 0x7f8c6414e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c62ffd640 1 -- 172.21.15.60:0/1873719787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6414ebc0 msgr2=0x7f8c641f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c62ffd640 1 --2- 172.21.15.60:0/1873719787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6414ebc0 0x7f8c641f6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c62ffd640 1 -- 172.21.15.60:0/1873719787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c641f7c20 con 0x7f8c4c003a40 2024-04-25T10:14:20.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c627fc640 1 --2- 172.21.15.60:0/1873719787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c641f9c90 0x7f8c6414e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:20.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c62ffd640 1 --2- 172.21.15.60:0/1873719787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c003a40 0x7f8c641f8000 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f8c5401c630 tx=0x7f8c4c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c4b7fe640 1 -- 172.21.15.60:0/1873719787 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54005e60 con 0x7f8c4c003a40 2024-04-25T10:14:20.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c4b7fe640 1 -- 172.21.15.60:0/1873719787 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c54000ee0 con 0x7f8c4c003a40 2024-04-25T10:14:20.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c4b7fe640 1 -- 172.21.15.60:0/1873719787 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c540072f0 con 0x7f8c4c003a40 2024-04-25T10:14:20.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c694a1640 1 -- 172.21.15.60:0/1873719787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c640acaf0 con 0x7f8c4c003a40 2024-04-25T10:14:20.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c694a1640 1 -- 172.21.15.60:0/1873719787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c641fa110 con 0x7f8c4c003a40 2024-04-25T10:14:20.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c694a1640 1 -- 172.21.15.60:0/1873719787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c64074770 con 0x7f8c4c003a40 2024-04-25T10:14:20.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c4b7fe640 1 -- 172.21.15.60:0/1873719787 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c540b7030 con 0x7f8c4c003a40 2024-04-25T10:14:20.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c4b7fe640 1 --2- 172.21.15.60:0/1873719787 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8c50066f90 0x7f8c50069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c4b7fe640 1 -- 172.21.15.60:0/1873719787 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c5411e140 con 0x7f8c4c003a40 2024-04-25T10:14:20.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.711+0000 7f8c627fc640 1 --2- 172.21.15.60:0/1873719787 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8c50066f90 0x7f8c50069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.715+0000 7f8c627fc640 1 --2- 172.21.15.60:0/1873719787 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8c50066f90 0x7f8c50069450 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8c6414e610 tx=0x7f8c58008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.715+0000 7f8c4b7fe640 1 -- 172.21.15.60:0/1873719787 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c540168d0 con 0x7f8c4c003a40 2024-04-25T10:14:20.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.867+0000 7f8c694a1640 1 -- 172.21.15.60:0/1873719787 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8c6414efa0 con 0x7f8c50066f90 2024-04-25T10:14:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.887+0000 7f8c4b7fe640 1 -- 172.21.15.60:0/1873719787 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252034 (secure 0 0 0) 0x7f8c6414efa0 con 0x7f8c50066f90 2024-04-25T10:14:20.895 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:20.896 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:14:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.891+0000 7f8c694a1640 1 -- 172.21.15.60:0/1873719787 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8c50066f90 msgr2=0x7f8c50069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.891+0000 7f8c694a1640 1 --2- 172.21.15.60:0/1873719787 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8c50066f90 0x7f8c50069450 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8c6414e610 tx=0x7f8c58008040 comp rx=0 tx=0).stop 2024-04-25T10:14:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.891+0000 7f8c694a1640 1 -- 172.21.15.60:0/1873719787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c003a40 msgr2=0x7f8c641f8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.891+0000 7f8c694a1640 1 --2- 172.21.15.60:0/1873719787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c003a40 0x7f8c641f8000 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f8c5401c630 tx=0x7f8c4c005840 comp rx=0 tx=0).stop 2024-04-25T10:14:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.891+0000 7f8c694a1640 1 -- 172.21.15.60:0/1873719787 shutdown_connections 2024-04-25T10:14:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.891+0000 7f8c694a1640 1 --2- 172.21.15.60:0/1873719787 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8c50066f90 0x7f8c50069450 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.891+0000 7f8c694a1640 1 --2- 172.21.15.60:0/1873719787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6414ebc0 0x7f8c641f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.891+0000 7f8c694a1640 1 --2- 172.21.15.60:0/1873719787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c641f9c90 0x7f8c6414e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.891+0000 7f8c694a1640 1 --2- 172.21.15.60:0/1873719787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c4c003a40 0x7f8c641f8000 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.891+0000 7f8c694a1640 1 -- 172.21.15.60:0/1873719787 >> 172.21.15.60:0/1873719787 conn(0x7f8c640b12d0 msgr2=0x7f8c640cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.891+0000 7f8c694a1640 1 -- 172.21.15.60:0/1873719787 shutdown_connections 2024-04-25T10:14:20.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:20.891+0000 7f8c694a1640 1 -- 172.21.15.60:0/1873719787 wait complete. 2024-04-25T10:14:20.912 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-04-25T10:14:20.576410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276520,"kb_used_data":10404,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754698200,"statfs":{"total":773094113280,"available":772810956800,"internally_reserved":0,"allocated":10653696,"data_stored":5228461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.806690"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.828185+0000","last_change":"2024-04-25T10:14:16.815098+0000","last_active":"2024-04-25T10:14:16.828185+0000","last_peered":"2024-04-25T10:14:16.828185+0000","last_clean":"2024-04-25T10:14:16.828185+0000","last_became_active":"2024-04-25T10:14:16.814339+0000","last_became_peered":"2024-04-25T10:14:16.814339+0000","last_unstale":"2024-04-25T10:14:16.828185+0000","last_undegraded":"2024-04-25T10:14:16.828185+0000","last_fullsized":"2024-04-25T10:14:16.828185+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:00.499368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.844682+0000","last_change":"2024-04-25T10:14:16.839651+0000","last_active":"2024-04-25T10:14:16.844682+0000","last_peered":"2024-04-25T10:14:16.844682+0000","last_clean":"2024-04-25T10:14:16.844682+0000","last_became_active":"2024-04-25T10:14:16.838355+0000","last_became_peered":"2024-04-25T10:14:16.838355+0000","last_unstale":"2024-04-25T10:14:16.844682+0000","last_undegraded":"2024-04-25T10:14:16.844682+0000","last_fullsized":"2024-04-25T10:14:16.844682+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:32.610149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.818475+0000","last_change":"2024-04-25T10:14:16.813762+0000","last_active":"2024-04-25T10:14:16.818475+0000","last_peered":"2024-04-25T10:14:16.818475+0000","last_clean":"2024-04-25T10:14:16.818475+0000","last_became_active":"2024-04-25T10:14:16.812971+0000","last_became_peered":"2024-04-25T10:14:16.812971+0000","last_unstale":"2024-04-25T10:14:16.818475+0000","last_undegraded":"2024-04-25T10:14:16.818475+0000","last_fullsized":"2024-04-25T10:14:16.818475+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:32.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.838906+0000","last_change":"2024-04-25T10:14:16.830279+0000","last_active":"2024-04-25T10:14:16.838906+0000","last_peered":"2024-04-25T10:14:16.838906+0000","last_clean":"2024-04-25T10:14:16.838906+0000","last_became_active":"2024-04-25T10:14:16.829625+0000","last_became_peered":"2024-04-25T10:14:16.829625+0000","last_unstale":"2024-04-25T10:14:16.838906+0000","last_undegraded":"2024-04-25T10:14:16.838906+0000","last_fullsized":"2024-04-25T10:14:16.838906+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:53.990178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.848145+0000","last_change":"2024-04-25T10:14:16.845675+0000","last_active":"2024-04-25T10:14:16.848145+0000","last_peered":"2024-04-25T10:14:16.848145+0000","last_clean":"2024-04-25T10:14:16.848145+0000","last_became_active":"2024-04-25T10:14:16.844616+0000","last_became_peered":"2024-04-25T10:14:16.844616+0000","last_unstale":"2024-04-25T10:14:16.848145+0000","last_undegraded":"2024-04-25T10:14:16.848145+0000","last_fullsized":"2024-04-25T10:14:16.848145+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:15.358398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063452+0000","last_change":"2024-04-25T10:14:16.842235+0000","last_active":"2024-04-25T10:14:18.063452+0000","last_peered":"2024-04-25T10:14:18.063452+0000","last_clean":"2024-04-25T10:14:18.063452+0000","last_became_active":"2024-04-25T10:14:16.841382+0000","last_became_peered":"2024-04-25T10:14:16.841382+0000","last_unstale":"2024-04-25T10:14:18.063452+0000","last_undegraded":"2024-04-25T10:14:18.063452+0000","last_fullsized":"2024-04-25T10:14:18.063452+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:05.727695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063511+0000","last_change":"2024-04-25T10:14:16.842333+0000","last_active":"2024-04-25T10:14:18.063511+0000","last_peered":"2024-04-25T10:14:18.063511+0000","last_clean":"2024-04-25T10:14:18.063511+0000","last_became_active":"2024-04-25T10:14:16.841611+0000","last_became_peered":"2024-04-25T10:14:16.841611+0000","last_unstale":"2024-04-25T10:14:18.063511+0000","last_undegraded":"2024-04-25T10:14:18.063511+0000","last_fullsized":"2024-04-25T10:14:18.063511+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:26.423483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.841925+0000","last_change":"2024-04-25T10:14:16.836100+0000","last_active":"2024-04-25T10:14:16.841925+0000","last_peered":"2024-04-25T10:14:16.841925+0000","last_clean":"2024-04-25T10:14:16.841925+0000","last_became_active":"2024-04-25T10:14:16.831230+0000","last_became_peered":"2024-04-25T10:14:16.831230+0000","last_unstale":"2024-04-25T10:14:16.841925+0000","last_undegraded":"2024-04-25T10:14:16.841925+0000","last_fullsized":"2024-04-25T10:14:16.841925+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:52.305296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.803811+0000","last_change":"2024-04-25T10:14:16.844498+0000","last_active":"2024-04-25T10:14:17.803811+0000","last_peered":"2024-04-25T10:14:17.803811+0000","last_clean":"2024-04-25T10:14:17.803811+0000","last_became_active":"2024-04-25T10:14:16.843783+0000","last_became_peered":"2024-04-25T10:14:16.843783+0000","last_unstale":"2024-04-25T10:14:17.803811+0000","last_undegraded":"2024-04-25T10:14:17.803811+0000","last_fullsized":"2024-04-25T10:14:17.803811+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:37.215360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.832853+0000","last_change":"2024-04-25T10:14:16.825791+0000","last_active":"2024-04-25T10:14:16.832853+0000","last_peered":"2024-04-25T10:14:16.832853+0000","last_clean":"2024-04-25T10:14:16.832853+0000","last_became_active":"2024-04-25T10:14:16.824962+0000","last_became_peered":"2024-04-25T10:14:16.824962+0000","last_unstale":"2024-04-25T10:14:16.832853+0000","last_undegraded":"2024-04-25T10:14:16.832853+0000","last_fullsized":"2024-04-25T10:14:16.832853+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:15.211114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063989+0000","last_change":"2024-04-25T10:14:16.834336+0000","last_active":"2024-04-25T10:14:18.063989+0000","last_peered":"2024-04-25T10:14:18.063989+0000","last_clean":"2024-04-25T10:14:18.063989+0000","last_became_active":"2024-04-25T10:14:16.833561+0000","last_became_peered":"2024-04-25T10:14:16.833561+0000","last_unstale":"2024-04-25T10:14:18.063989+0000","last_undegraded":"2024-04-25T10:14:18.063989+0000","last_fullsized":"2024-04-25T10:14:18.063989+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:33.492135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.803014+0000","last_change":"2024-04-25T10:14:16.824569+0000","last_active":"2024-04-25T10:14:17.803014+0000","last_peered":"2024-04-25T10:14:17.803014+0000","last_clean":"2024-04-25T10:14:17.803014+0000","last_became_active":"2024-04-25T10:14:16.823752+0000","last_became_peered":"2024-04-25T10:14:16.823752+0000","last_unstale":"2024-04-25T10:14:17.803014+0000","last_undegraded":"2024-04-25T10:14:17.803014+0000","last_fullsized":"2024-04-25T10:14:17.803014+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:33.765433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.839015+0000","last_change":"2024-04-25T10:14:16.830478+0000","last_active":"2024-04-25T10:14:16.839015+0000","last_peered":"2024-04-25T10:14:16.839015+0000","last_clean":"2024-04-25T10:14:16.839015+0000","last_became_active":"2024-04-25T10:14:16.829702+0000","last_became_peered":"2024-04-25T10:14:16.829702+0000","last_unstale":"2024-04-25T10:14:16.839015+0000","last_undegraded":"2024-04-25T10:14:16.839015+0000","last_fullsized":"2024-04-25T10:14:16.839015+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:52.536706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.829266+0000","last_change":"2024-04-25T10:14:16.809416+0000","last_active":"2024-04-25T10:14:16.829266+0000","last_peered":"2024-04-25T10:14:16.829266+0000","last_clean":"2024-04-25T10:14:16.829266+0000","last_became_active":"2024-04-25T10:14:16.808672+0000","last_became_peered":"2024-04-25T10:14:16.808672+0000","last_unstale":"2024-04-25T10:14:16.829266+0000","last_undegraded":"2024-04-25T10:14:16.829266+0000","last_fullsized":"2024-04-25T10:14:16.829266+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:29.820849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.845446+0000","last_change":"2024-04-25T10:14:16.840401+0000","last_active":"2024-04-25T10:14:16.845446+0000","last_peered":"2024-04-25T10:14:16.845446+0000","last_clean":"2024-04-25T10:14:16.845446+0000","last_became_active":"2024-04-25T10:14:16.837968+0000","last_became_peered":"2024-04-25T10:14:16.837968+0000","last_unstale":"2024-04-25T10:14:16.845446+0000","last_undegraded":"2024-04-25T10:14:16.845446+0000","last_fullsized":"2024-04-25T10:14:16.845446+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:33.763115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063068+0000","last_change":"2024-04-25T10:14:16.828675+0000","last_active":"2024-04-25T10:14:18.063068+0000","last_peered":"2024-04-25T10:14:18.063068+0000","last_clean":"2024-04-25T10:14:18.063068+0000","last_became_active":"2024-04-25T10:14:16.827961+0000","last_became_peered":"2024-04-25T10:14:16.827961+0000","last_unstale":"2024-04-25T10:14:18.063068+0000","last_undegraded":"2024-04-25T10:14:18.063068+0000","last_fullsized":"2024-04-25T10:14:18.063068+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:09.929605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.845936+0000","last_change":"2024-04-25T10:14:16.842579+0000","last_active":"2024-04-25T10:14:16.845936+0000","last_peered":"2024-04-25T10:14:16.845936+0000","last_clean":"2024-04-25T10:14:16.845936+0000","last_became_active":"2024-04-25T10:14:16.841857+0000","last_became_peered":"2024-04-25T10:14:16.841857+0000","last_unstale":"2024-04-25T10:14:16.845936+0000","last_undegraded":"2024-04-25T10:14:16.845936+0000","last_fullsized":"2024-04-25T10:14:16.845936+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:16.094182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.839061+0000","last_change":"2024-04-25T10:14:16.834470+0000","last_active":"2024-04-25T10:14:16.839061+0000","last_peered":"2024-04-25T10:14:16.839061+0000","last_clean":"2024-04-25T10:14:16.839061+0000","last_became_active":"2024-04-25T10:14:16.833645+0000","last_became_peered":"2024-04-25T10:14:16.833645+0000","last_unstale":"2024-04-25T10:14:16.839061+0000","last_undegraded":"2024-04-25T10:14:16.839061+0000","last_fullsized":"2024-04-25T10:14:16.839061+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:11.241056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.828761+0000","last_change":"2024-04-25T10:14:16.817847+0000","last_active":"2024-04-25T10:14:16.828761+0000","last_peered":"2024-04-25T10:14:16.828761+0000","last_clean":"2024-04-25T10:14:16.828761+0000","last_became_active":"2024-04-25T10:14:16.817086+0000","last_became_peered":"2024-04-25T10:14:16.817086+0000","last_unstale":"2024-04-25T10:14:16.828761+0000","last_undegraded":"2024-04-25T10:14:16.828761+0000","last_fullsized":"2024-04-25T10:14:16.828761+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:20.632609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.803239+0000","last_change":"2024-04-25T10:14:16.839117+0000","last_active":"2024-04-25T10:14:17.803239+0000","last_peered":"2024-04-25T10:14:17.803239+0000","last_clean":"2024-04-25T10:14:17.803239+0000","last_became_active":"2024-04-25T10:14:16.838446+0000","last_became_peered":"2024-04-25T10:14:16.838446+0000","last_unstale":"2024-04-25T10:14:17.803239+0000","last_undegraded":"2024-04-25T10:14:17.803239+0000","last_fullsized":"2024-04-25T10:14:17.803239+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:56.219741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.823877+0000","last_change":"2024-04-25T10:14:16.802268+0000","last_active":"2024-04-25T10:14:16.823877+0000","last_peered":"2024-04-25T10:14:16.823877+0000","last_clean":"2024-04-25T10:14:16.823877+0000","last_became_active":"2024-04-25T10:14:16.801628+0000","last_became_peered":"2024-04-25T10:14:16.801628+0000","last_unstale":"2024-04-25T10:14:16.823877+0000","last_undegraded":"2024-04-25T10:14:16.823877+0000","last_fullsized":"2024-04-25T10:14:16.823877+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:22.469287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.841473+0000","last_change":"2024-04-25T10:14:16.836272+0000","last_active":"2024-04-25T10:14:16.841473+0000","last_peered":"2024-04-25T10:14:16.841473+0000","last_clean":"2024-04-25T10:14:16.841473+0000","last_became_active":"2024-04-25T10:14:16.831376+0000","last_became_peered":"2024-04-25T10:14:16.831376+0000","last_unstale":"2024-04-25T10:14:16.841473+0000","last_undegraded":"2024-04-25T10:14:16.841473+0000","last_fullsized":"2024-04-25T10:14:16.841473+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:26.205046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.842271+0000","last_change":"2024-04-25T10:14:16.837060+0000","last_active":"2024-04-25T10:14:16.842271+0000","last_peered":"2024-04-25T10:14:16.842271+0000","last_clean":"2024-04-25T10:14:16.842271+0000","last_became_active":"2024-04-25T10:14:16.831663+0000","last_became_peered":"2024-04-25T10:14:16.831663+0000","last_unstale":"2024-04-25T10:14:16.842271+0000","last_undegraded":"2024-04-25T10:14:16.842271+0000","last_fullsized":"2024-04-25T10:14:16.842271+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:59.352831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063264+0000","last_change":"2024-04-25T10:14:16.832957+0000","last_active":"2024-04-25T10:14:18.063264+0000","last_peered":"2024-04-25T10:14:18.063264+0000","last_clean":"2024-04-25T10:14:18.063264+0000","last_became_active":"2024-04-25T10:14:16.832078+0000","last_became_peered":"2024-04-25T10:14:16.832078+0000","last_unstale":"2024-04-25T10:14:18.063264+0000","last_undegraded":"2024-04-25T10:14:18.063264+0000","last_fullsized":"2024-04-25T10:14:18.063264+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:39.379522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.835429+0000","last_change":"2024-04-25T10:14:16.830004+0000","last_active":"2024-04-25T10:14:16.835429+0000","last_peered":"2024-04-25T10:14:16.835429+0000","last_clean":"2024-04-25T10:14:16.835429+0000","last_became_active":"2024-04-25T10:14:16.828640+0000","last_became_peered":"2024-04-25T10:14:16.828640+0000","last_unstale":"2024-04-25T10:14:16.835429+0000","last_undegraded":"2024-04-25T10:14:16.835429+0000","last_fullsized":"2024-04-25T10:14:16.835429+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:53.877666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.842000+0000","last_change":"2024-04-25T10:14:16.823761+0000","last_active":"2024-04-25T10:14:16.842000+0000","last_peered":"2024-04-25T10:14:16.842000+0000","last_clean":"2024-04-25T10:14:16.842000+0000","last_became_active":"2024-04-25T10:14:16.822949+0000","last_became_peered":"2024-04-25T10:14:16.822949+0000","last_unstale":"2024-04-25T10:14:16.842000+0000","last_undegraded":"2024-04-25T10:14:16.842000+0000","last_fullsized":"2024-04-25T10:14:16.842000+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:14.648790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.064317+0000","last_change":"2024-04-25T10:14:16.840130+0000","last_active":"2024-04-25T10:14:18.064317+0000","last_peered":"2024-04-25T10:14:18.064317+0000","last_clean":"2024-04-25T10:14:18.064317+0000","last_became_active":"2024-04-25T10:14:16.838781+0000","last_became_peered":"2024-04-25T10:14:16.838781+0000","last_unstale":"2024-04-25T10:14:18.064317+0000","last_undegraded":"2024-04-25T10:14:18.064317+0000","last_fullsized":"2024-04-25T10:14:18.064317+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:36.699661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.848070+0000","last_change":"2024-04-25T10:14:16.844672+0000","last_active":"2024-04-25T10:14:16.848070+0000","last_peered":"2024-04-25T10:14:16.848070+0000","last_clean":"2024-04-25T10:14:16.848070+0000","last_became_active":"2024-04-25T10:14:16.843854+0000","last_became_peered":"2024-04-25T10:14:16.843854+0000","last_unstale":"2024-04-25T10:14:16.848070+0000","last_undegraded":"2024-04-25T10:14:16.848070+0000","last_fullsized":"2024-04-25T10:14:16.848070+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:56.590783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.837244+0000","last_change":"2024-04-25T10:14:16.823917+0000","last_active":"2024-04-25T10:14:16.837244+0000","last_peered":"2024-04-25T10:14:16.837244+0000","last_clean":"2024-04-25T10:14:16.837244+0000","last_became_active":"2024-04-25T10:14:16.823098+0000","last_became_peered":"2024-04-25T10:14:16.823098+0000","last_unstale":"2024-04-25T10:14:16.837244+0000","last_undegraded":"2024-04-25T10:14:16.837244+0000","last_fullsized":"2024-04-25T10:14:16.837244+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:15.050081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.829318+0000","last_change":"2024-04-25T10:14:16.816406+0000","last_active":"2024-04-25T10:14:16.829318+0000","last_peered":"2024-04-25T10:14:16.829318+0000","last_clean":"2024-04-25T10:14:16.829318+0000","last_became_active":"2024-04-25T10:14:16.815619+0000","last_became_peered":"2024-04-25T10:14:16.815619+0000","last_unstale":"2024-04-25T10:14:16.829318+0000","last_undegraded":"2024-04-25T10:14:16.829318+0000","last_fullsized":"2024-04-25T10:14:16.829318+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:33.722277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.845500+0000","last_change":"2024-04-25T10:14:16.840606+0000","last_active":"2024-04-25T10:14:16.845500+0000","last_peered":"2024-04-25T10:14:16.845500+0000","last_clean":"2024-04-25T10:14:16.845500+0000","last_became_active":"2024-04-25T10:14:16.839604+0000","last_became_peered":"2024-04-25T10:14:16.839604+0000","last_unstale":"2024-04-25T10:14:16.845500+0000","last_undegraded":"2024-04-25T10:14:16.845500+0000","last_fullsized":"2024-04-25T10:14:16.845500+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:33.944668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.842999+0000","last_change":"2024-04-25T10:14:16.823664+0000","last_active":"2024-04-25T10:14:16.842999+0000","last_peered":"2024-04-25T10:14:16.842999+0000","last_clean":"2024-04-25T10:14:16.842999+0000","last_became_active":"2024-04-25T10:14:16.822926+0000","last_became_peered":"2024-04-25T10:14:16.822926+0000","last_unstale":"2024-04-25T10:14:16.842999+0000","last_undegraded":"2024-04-25T10:14:16.842999+0000","last_fullsized":"2024-04-25T10:14:16.842999+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:43.890875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.831144+0000","last_change":"2024-04-25T10:14:16.822544+0000","last_active":"2024-04-25T10:14:16.831144+0000","last_peered":"2024-04-25T10:14:16.831144+0000","last_clean":"2024-04-25T10:14:16.831144+0000","last_became_active":"2024-04-25T10:14:16.821713+0000","last_became_peered":"2024-04-25T10:14:16.821713+0000","last_unstale":"2024-04-25T10:14:16.831144+0000","last_undegraded":"2024-04-25T10:14:16.831144+0000","last_fullsized":"2024-04-25T10:14:16.831144+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:28.734252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"96'4","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.864092+0000","last_change":"2024-04-25T10:14:15.805877+0000","last_active":"2024-04-25T10:14:16.864092+0000","last_peered":"2024-04-25T10:14:16.864092+0000","last_clean":"2024-04-25T10:14:16.864092+0000","last_became_active":"2024-04-25T10:14:15.805542+0000","last_became_peered":"2024-04-25T10:14:15.805542+0000","last_unstale":"2024-04-25T10:14:16.864092+0000","last_undegraded":"2024-04-25T10:14:16.864092+0000","last_fullsized":"2024-04-25T10:14:16.864092+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:10.983038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.842890+0000","last_change":"2024-04-25T10:14:16.837300+0000","last_active":"2024-04-25T10:14:16.842890+0000","last_peered":"2024-04-25T10:14:16.842890+0000","last_clean":"2024-04-25T10:14:16.842890+0000","last_became_active":"2024-04-25T10:14:16.833190+0000","last_became_peered":"2024-04-25T10:14:16.833190+0000","last_unstale":"2024-04-25T10:14:16.842890+0000","last_undegraded":"2024-04-25T10:14:16.842890+0000","last_fullsized":"2024-04-25T10:14:16.842890+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:34.351067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063309+0000","last_change":"2024-04-25T10:14:15.809678+0000","last_active":"2024-04-25T10:14:18.063309+0000","last_peered":"2024-04-25T10:14:18.063309+0000","last_clean":"2024-04-25T10:14:18.063309+0000","last_became_active":"2024-04-25T10:14:15.808934+0000","last_became_peered":"2024-04-25T10:14:15.808934+0000","last_unstale":"2024-04-25T10:14:18.063309+0000","last_undegraded":"2024-04-25T10:14:18.063309+0000","last_fullsized":"2024-04-25T10:14:18.063309+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:32.755352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.844567+0000","last_change":"2024-04-25T10:14:16.827136+0000","last_active":"2024-04-25T10:14:16.844567+0000","last_peered":"2024-04-25T10:14:16.844567+0000","last_clean":"2024-04-25T10:14:16.844567+0000","last_became_active":"2024-04-25T10:14:16.825869+0000","last_became_peered":"2024-04-25T10:14:16.825869+0000","last_unstale":"2024-04-25T10:14:16.844567+0000","last_undegraded":"2024-04-25T10:14:16.844567+0000","last_fullsized":"2024-04-25T10:14:16.844567+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:39.741353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"96'2","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.858743+0000","last_change":"2024-04-25T10:14:15.800268+0000","last_active":"2024-04-25T10:14:16.858743+0000","last_peered":"2024-04-25T10:14:16.858743+0000","last_clean":"2024-04-25T10:14:16.858743+0000","last_became_active":"2024-04-25T10:14:15.799860+0000","last_became_peered":"2024-04-25T10:14:15.799860+0000","last_unstale":"2024-04-25T10:14:16.858743+0000","last_undegraded":"2024-04-25T10:14:16.858743+0000","last_fullsized":"2024-04-25T10:14:16.858743+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:14.348317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.803223+0000","last_change":"2024-04-25T10:14:16.823722+0000","last_active":"2024-04-25T10:14:17.803223+0000","last_peered":"2024-04-25T10:14:17.803223+0000","last_clean":"2024-04-25T10:14:17.803223+0000","last_became_active":"2024-04-25T10:14:16.822623+0000","last_became_peered":"2024-04-25T10:14:16.822623+0000","last_unstale":"2024-04-25T10:14:17.803223+0000","last_undegraded":"2024-04-25T10:14:17.803223+0000","last_fullsized":"2024-04-25T10:14:17.803223+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:18.001521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"96'2","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.863850+0000","last_change":"2024-04-25T10:14:15.811285+0000","last_active":"2024-04-25T10:14:16.863850+0000","last_peered":"2024-04-25T10:14:16.863850+0000","last_clean":"2024-04-25T10:14:16.863850+0000","last_became_active":"2024-04-25T10:14:15.810949+0000","last_became_peered":"2024-04-25T10:14:15.810949+0000","last_unstale":"2024-04-25T10:14:16.863850+0000","last_undegraded":"2024-04-25T10:14:16.863850+0000","last_fullsized":"2024-04-25T10:14:16.863850+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:52.980097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.842474+0000","last_change":"2024-04-25T10:14:16.840014+0000","last_active":"2024-04-25T10:14:16.842474+0000","last_peered":"2024-04-25T10:14:16.842474+0000","last_clean":"2024-04-25T10:14:16.842474+0000","last_became_active":"2024-04-25T10:14:16.839106+0000","last_became_peered":"2024-04-25T10:14:16.839106+0000","last_unstale":"2024-04-25T10:14:16.842474+0000","last_undegraded":"2024-04-25T10:14:16.842474+0000","last_fullsized":"2024-04-25T10:14:16.842474+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:25.477777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"96'1","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.863768+0000","last_change":"2024-04-25T10:14:15.797291+0000","last_active":"2024-04-25T10:14:16.863768+0000","last_peered":"2024-04-25T10:14:16.863768+0000","last_clean":"2024-04-25T10:14:16.863768+0000","last_became_active":"2024-04-25T10:14:15.796842+0000","last_became_peered":"2024-04-25T10:14:15.796842+0000","last_unstale":"2024-04-25T10:14:16.863768+0000","last_undegraded":"2024-04-25T10:14:16.863768+0000","last_fullsized":"2024-04-25T10:14:16.863768+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:34.581645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063893+0000","last_change":"2024-04-25T10:14:16.836280+0000","last_active":"2024-04-25T10:14:18.063893+0000","last_peered":"2024-04-25T10:14:18.063893+0000","last_clean":"2024-04-25T10:14:18.063893+0000","last_became_active":"2024-04-25T10:14:16.835426+0000","last_became_peered":"2024-04-25T10:14:16.835426+0000","last_unstale":"2024-04-25T10:14:18.063893+0000","last_undegraded":"2024-04-25T10:14:18.063893+0000","last_fullsized":"2024-04-25T10:14:18.063893+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:48.818995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"96'1","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.859581+0000","last_change":"2024-04-25T10:14:15.789221+0000","last_active":"2024-04-25T10:14:16.859581+0000","last_peered":"2024-04-25T10:14:16.859581+0000","last_clean":"2024-04-25T10:14:16.859581+0000","last_became_active":"2024-04-25T10:14:15.788949+0000","last_became_peered":"2024-04-25T10:14:15.788949+0000","last_unstale":"2024-04-25T10:14:16.859581+0000","last_undegraded":"2024-04-25T10:14:16.859581+0000","last_fullsized":"2024-04-25T10:14:16.859581+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:32.753034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.839141+0000","last_change":"2024-04-25T10:14:16.834847+0000","last_active":"2024-04-25T10:14:16.839141+0000","last_peered":"2024-04-25T10:14:16.839141+0000","last_clean":"2024-04-25T10:14:16.839141+0000","last_became_active":"2024-04-25T10:14:16.834104+0000","last_became_peered":"2024-04-25T10:14:16.834104+0000","last_unstale":"2024-04-25T10:14:16.839141+0000","last_undegraded":"2024-04-25T10:14:16.839141+0000","last_fullsized":"2024-04-25T10:14:16.839141+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:05.447159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.814406+0000","last_change":"2024-04-25T10:14:15.804657+0000","last_active":"2024-04-25T10:14:16.814406+0000","last_peered":"2024-04-25T10:14:16.814406+0000","last_clean":"2024-04-25T10:14:16.814406+0000","last_became_active":"2024-04-25T10:14:15.803934+0000","last_became_peered":"2024-04-25T10:14:15.803934+0000","last_unstale":"2024-04-25T10:14:16.814406+0000","last_undegraded":"2024-04-25T10:14:16.814406+0000","last_fullsized":"2024-04-25T10:14:16.814406+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:37.484119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.801535+0000","last_change":"2024-04-25T10:14:16.812686+0000","last_active":"2024-04-25T10:14:17.801535+0000","last_peered":"2024-04-25T10:14:17.801535+0000","last_clean":"2024-04-25T10:14:17.801535+0000","last_became_active":"2024-04-25T10:14:16.810803+0000","last_became_peered":"2024-04-25T10:14:16.810803+0000","last_unstale":"2024-04-25T10:14:17.801535+0000","last_undegraded":"2024-04-25T10:14:17.801535+0000","last_fullsized":"2024-04-25T10:14:17.801535+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:53.331240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.804241+0000","last_change":"2024-04-25T10:14:15.798425+0000","last_active":"2024-04-25T10:14:16.804241+0000","last_peered":"2024-04-25T10:14:16.804241+0000","last_clean":"2024-04-25T10:14:16.804241+0000","last_became_active":"2024-04-25T10:14:15.798030+0000","last_became_peered":"2024-04-25T10:14:15.798030+0000","last_unstale":"2024-04-25T10:14:16.804241+0000","last_undegraded":"2024-04-25T10:14:16.804241+0000","last_fullsized":"2024-04-25T10:14:16.804241+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:32.884443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.842056+0000","last_change":"2024-04-25T10:14:16.827231+0000","last_active":"2024-04-25T10:14:16.842056+0000","last_peered":"2024-04-25T10:14:16.842056+0000","last_clean":"2024-04-25T10:14:16.842056+0000","last_became_active":"2024-04-25T10:14:16.826423+0000","last_became_peered":"2024-04-25T10:14:16.826423+0000","last_unstale":"2024-04-25T10:14:16.842056+0000","last_undegraded":"2024-04-25T10:14:16.842056+0000","last_fullsized":"2024-04-25T10:14:16.842056+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:14.354255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.813881+0000","last_change":"2024-04-25T10:14:15.793711+0000","last_active":"2024-04-25T10:14:16.813881+0000","last_peered":"2024-04-25T10:14:16.813881+0000","last_clean":"2024-04-25T10:14:16.813881+0000","last_became_active":"2024-04-25T10:14:15.793312+0000","last_became_peered":"2024-04-25T10:14:15.793312+0000","last_unstale":"2024-04-25T10:14:16.813881+0000","last_undegraded":"2024-04-25T10:14:16.813881+0000","last_fullsized":"2024-04-25T10:14:16.813881+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:58.969202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.803867+0000","last_change":"2024-04-25T10:14:16.814806+0000","last_active":"2024-04-25T10:14:17.803867+0000","last_peered":"2024-04-25T10:14:17.803867+0000","last_clean":"2024-04-25T10:14:17.803867+0000","last_became_active":"2024-04-25T10:14:16.814036+0000","last_became_peered":"2024-04-25T10:14:16.814036+0000","last_unstale":"2024-04-25T10:14:17.803867+0000","last_undegraded":"2024-04-25T10:14:17.803867+0000","last_fullsized":"2024-04-25T10:14:17.803867+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:41.386848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.817157+0000","last_change":"2024-04-25T10:14:15.793546+0000","last_active":"2024-04-25T10:14:16.817157+0000","last_peered":"2024-04-25T10:14:16.817157+0000","last_clean":"2024-04-25T10:14:16.817157+0000","last_became_active":"2024-04-25T10:14:15.793087+0000","last_became_peered":"2024-04-25T10:14:15.793087+0000","last_unstale":"2024-04-25T10:14:16.817157+0000","last_undegraded":"2024-04-25T10:14:16.817157+0000","last_fullsized":"2024-04-25T10:14:16.817157+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:27.671299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.835914+0000","last_change":"2024-04-25T10:14:16.829166+0000","last_active":"2024-04-25T10:14:16.835914+0000","last_peered":"2024-04-25T10:14:16.835914+0000","last_clean":"2024-04-25T10:14:16.835914+0000","last_became_active":"2024-04-25T10:14:16.828439+0000","last_became_peered":"2024-04-25T10:14:16.828439+0000","last_unstale":"2024-04-25T10:14:16.835914+0000","last_undegraded":"2024-04-25T10:14:16.835914+0000","last_fullsized":"2024-04-25T10:14:16.835914+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:01.715936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"96'3","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.064997+0000","last_change":"2024-04-25T10:14:15.815968+0000","last_active":"2024-04-25T10:14:18.064997+0000","last_peered":"2024-04-25T10:14:18.064997+0000","last_clean":"2024-04-25T10:14:18.064997+0000","last_became_active":"2024-04-25T10:14:15.815092+0000","last_became_peered":"2024-04-25T10:14:15.815092+0000","last_unstale":"2024-04-25T10:14:18.064997+0000","last_undegraded":"2024-04-25T10:14:18.064997+0000","last_fullsized":"2024-04-25T10:14:18.064997+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:16.991440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.064731+0000","last_change":"2024-04-25T10:14:16.839865+0000","last_active":"2024-04-25T10:14:18.064731+0000","last_peered":"2024-04-25T10:14:18.064731+0000","last_clean":"2024-04-25T10:14:18.064731+0000","last_became_active":"2024-04-25T10:14:16.831654+0000","last_became_peered":"2024-04-25T10:14:16.831654+0000","last_unstale":"2024-04-25T10:14:18.064731+0000","last_undegraded":"2024-04-25T10:14:18.064731+0000","last_fullsized":"2024-04-25T10:14:18.064731+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:57.140722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"96'6","reported_seq":20,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.803484+0000","last_change":"2024-04-25T10:14:15.804890+0000","last_active":"2024-04-25T10:14:17.803484+0000","last_peered":"2024-04-25T10:14:17.803484+0000","last_clean":"2024-04-25T10:14:17.803484+0000","last_became_active":"2024-04-25T10:14:15.803941+0000","last_became_peered":"2024-04-25T10:14:15.803941+0000","last_unstale":"2024-04-25T10:14:17.803484+0000","last_undegraded":"2024-04-25T10:14:17.803484+0000","last_fullsized":"2024-04-25T10:14:17.803484+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:25.029340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.845308+0000","last_change":"2024-04-25T10:14:16.838331+0000","last_active":"2024-04-25T10:14:16.845308+0000","last_peered":"2024-04-25T10:14:16.845308+0000","last_clean":"2024-04-25T10:14:16.845308+0000","last_became_active":"2024-04-25T10:14:16.835377+0000","last_became_peered":"2024-04-25T10:14:16.835377+0000","last_unstale":"2024-04-25T10:14:16.845308+0000","last_undegraded":"2024-04-25T10:14:16.845308+0000","last_fullsized":"2024-04-25T10:14:16.845308+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:11.993119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"96'2","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.864354+0000","last_change":"2024-04-25T10:14:15.798437+0000","last_active":"2024-04-25T10:14:16.864354+0000","last_peered":"2024-04-25T10:14:16.864354+0000","last_clean":"2024-04-25T10:14:16.864354+0000","last_became_active":"2024-04-25T10:14:15.797924+0000","last_became_peered":"2024-04-25T10:14:15.797924+0000","last_unstale":"2024-04-25T10:14:16.864354+0000","last_undegraded":"2024-04-25T10:14:16.864354+0000","last_fullsized":"2024-04-25T10:14:16.864354+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:17.277691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.844633+0000","last_change":"2024-04-25T10:14:16.828535+0000","last_active":"2024-04-25T10:14:16.844633+0000","last_peered":"2024-04-25T10:14:16.844633+0000","last_clean":"2024-04-25T10:14:16.844633+0000","last_became_active":"2024-04-25T10:14:16.827773+0000","last_became_peered":"2024-04-25T10:14:16.827773+0000","last_unstale":"2024-04-25T10:14:16.844633+0000","last_undegraded":"2024-04-25T10:14:16.844633+0000","last_fullsized":"2024-04-25T10:14:16.844633+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:28.448033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.796597+0000","last_change":"2024-04-25T10:14:15.796462+0000","last_active":"2024-04-25T10:14:16.796597+0000","last_peered":"2024-04-25T10:14:16.796597+0000","last_clean":"2024-04-25T10:14:16.796597+0000","last_became_active":"2024-04-25T10:14:15.795901+0000","last_became_peered":"2024-04-25T10:14:15.795901+0000","last_unstale":"2024-04-25T10:14:16.796597+0000","last_undegraded":"2024-04-25T10:14:16.796597+0000","last_fullsized":"2024-04-25T10:14:16.796597+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:03.804755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.801067+0000","last_change":"2024-04-25T10:14:16.813690+0000","last_active":"2024-04-25T10:14:17.801067+0000","last_peered":"2024-04-25T10:14:17.801067+0000","last_clean":"2024-04-25T10:14:17.801067+0000","last_became_active":"2024-04-25T10:14:16.812956+0000","last_became_peered":"2024-04-25T10:14:16.812956+0000","last_unstale":"2024-04-25T10:14:17.801067+0000","last_undegraded":"2024-04-25T10:14:17.801067+0000","last_fullsized":"2024-04-25T10:14:17.801067+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:43.857050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.801615+0000","last_change":"2024-04-25T10:14:15.816673+0000","last_active":"2024-04-25T10:14:17.801615+0000","last_peered":"2024-04-25T10:14:17.801615+0000","last_clean":"2024-04-25T10:14:17.801615+0000","last_became_active":"2024-04-25T10:14:15.815942+0000","last_became_peered":"2024-04-25T10:14:15.815942+0000","last_unstale":"2024-04-25T10:14:17.801615+0000","last_undegraded":"2024-04-25T10:14:17.801615+0000","last_fullsized":"2024-04-25T10:14:17.801615+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:42.517601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.829110+0000","last_change":"2024-04-25T10:14:16.820291+0000","last_active":"2024-04-25T10:14:16.829110+0000","last_peered":"2024-04-25T10:14:16.829110+0000","last_clean":"2024-04-25T10:14:16.829110+0000","last_became_active":"2024-04-25T10:14:16.819514+0000","last_became_peered":"2024-04-25T10:14:16.819514+0000","last_unstale":"2024-04-25T10:14:16.829110+0000","last_undegraded":"2024-04-25T10:14:16.829110+0000","last_fullsized":"2024-04-25T10:14:16.829110+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.666850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"96'8","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.862810+0000","last_change":"2024-04-25T10:14:15.803375+0000","last_active":"2024-04-25T10:14:16.862810+0000","last_peered":"2024-04-25T10:14:16.862810+0000","last_clean":"2024-04-25T10:14:16.862810+0000","last_became_active":"2024-04-25T10:14:15.802900+0000","last_became_peered":"2024-04-25T10:14:15.802900+0000","last_unstale":"2024-04-25T10:14:16.862810+0000","last_undegraded":"2024-04-25T10:14:16.862810+0000","last_fullsized":"2024-04-25T10:14:16.862810+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:17.802904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.835484+0000","last_change":"2024-04-25T10:14:16.831758+0000","last_active":"2024-04-25T10:14:16.835484+0000","last_peered":"2024-04-25T10:14:16.835484+0000","last_clean":"2024-04-25T10:14:16.835484+0000","last_became_active":"2024-04-25T10:14:16.830919+0000","last_became_peered":"2024-04-25T10:14:16.830919+0000","last_unstale":"2024-04-25T10:14:16.835484+0000","last_undegraded":"2024-04-25T10:14:16.835484+0000","last_fullsized":"2024-04-25T10:14:16.835484+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:26.181804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.803550+0000","last_change":"2024-04-25T10:14:15.803968+0000","last_active":"2024-04-25T10:14:17.803550+0000","last_peered":"2024-04-25T10:14:17.803550+0000","last_clean":"2024-04-25T10:14:17.803550+0000","last_became_active":"2024-04-25T10:14:15.803251+0000","last_became_peered":"2024-04-25T10:14:15.803251+0000","last_unstale":"2024-04-25T10:14:17.803550+0000","last_undegraded":"2024-04-25T10:14:17.803550+0000","last_fullsized":"2024-04-25T10:14:17.803550+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:24.192595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.834271+0000","last_change":"2024-04-25T10:14:16.817653+0000","last_active":"2024-04-25T10:14:16.834271+0000","last_peered":"2024-04-25T10:14:16.834271+0000","last_clean":"2024-04-25T10:14:16.834271+0000","last_became_active":"2024-04-25T10:14:16.816917+0000","last_became_peered":"2024-04-25T10:14:16.816917+0000","last_unstale":"2024-04-25T10:14:16.834271+0000","last_undegraded":"2024-04-25T10:14:16.834271+0000","last_fullsized":"2024-04-25T10:14:16.834271+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:22.150303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"96'1","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.868034+0000","last_change":"2024-04-25T10:14:15.811035+0000","last_active":"2024-04-25T10:14:16.868034+0000","last_peered":"2024-04-25T10:14:16.868034+0000","last_clean":"2024-04-25T10:14:16.868034+0000","last_became_active":"2024-04-25T10:14:15.810333+0000","last_became_peered":"2024-04-25T10:14:15.810333+0000","last_unstale":"2024-04-25T10:14:16.868034+0000","last_undegraded":"2024-04-25T10:14:16.868034+0000","last_fullsized":"2024-04-25T10:14:16.868034+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:24.467696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.064733+0000","last_change":"2024-04-25T10:14:16.839536+0000","last_active":"2024-04-25T10:14:18.064733+0000","last_peered":"2024-04-25T10:14:18.064733+0000","last_clean":"2024-04-25T10:14:18.064733+0000","last_became_active":"2024-04-25T10:14:16.837753+0000","last_became_peered":"2024-04-25T10:14:16.837753+0000","last_unstale":"2024-04-25T10:14:18.064733+0000","last_undegraded":"2024-04-25T10:14:18.064733+0000","last_fullsized":"2024-04-25T10:14:18.064733+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:13.267284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.810386+0000","last_change":"2024-04-25T10:14:15.810532+0000","last_active":"2024-04-25T10:14:16.810386+0000","last_peered":"2024-04-25T10:14:16.810386+0000","last_clean":"2024-04-25T10:14:16.810386+0000","last_became_active":"2024-04-25T10:14:15.810236+0000","last_became_peered":"2024-04-25T10:14:15.810236+0000","last_unstale":"2024-04-25T10:14:16.810386+0000","last_undegraded":"2024-04-25T10:14:16.810386+0000","last_fullsized":"2024-04-25T10:14:16.810386+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:04.437078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.845612+0000","last_change":"2024-04-25T10:14:16.841170+0000","last_active":"2024-04-25T10:14:16.845612+0000","last_peered":"2024-04-25T10:14:16.845612+0000","last_clean":"2024-04-25T10:14:16.845612+0000","last_became_active":"2024-04-25T10:14:16.840499+0000","last_became_peered":"2024-04-25T10:14:16.840499+0000","last_unstale":"2024-04-25T10:14:16.845612+0000","last_undegraded":"2024-04-25T10:14:16.845612+0000","last_fullsized":"2024-04-25T10:14:16.845612+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:45.631223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"96'2","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.859697+0000","last_change":"2024-04-25T10:14:15.804460+0000","last_active":"2024-04-25T10:14:16.859697+0000","last_peered":"2024-04-25T10:14:16.859697+0000","last_clean":"2024-04-25T10:14:16.859697+0000","last_became_active":"2024-04-25T10:14:15.803915+0000","last_became_peered":"2024-04-25T10:14:15.803915+0000","last_unstale":"2024-04-25T10:14:16.859697+0000","last_undegraded":"2024-04-25T10:14:16.859697+0000","last_fullsized":"2024-04-25T10:14:16.859697+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:27.671299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.828875+0000","last_change":"2024-04-25T10:14:16.824226+0000","last_active":"2024-04-25T10:14:16.828875+0000","last_peered":"2024-04-25T10:14:16.828875+0000","last_clean":"2024-04-25T10:14:16.828875+0000","last_became_active":"2024-04-25T10:14:16.823343+0000","last_became_peered":"2024-04-25T10:14:16.823343+0000","last_unstale":"2024-04-25T10:14:16.828875+0000","last_undegraded":"2024-04-25T10:14:16.828875+0000","last_fullsized":"2024-04-25T10:14:16.828875+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:56.579231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.800661+0000","last_change":"2024-04-25T10:14:15.795520+0000","last_active":"2024-04-25T10:14:16.800661+0000","last_peered":"2024-04-25T10:14:16.800661+0000","last_clean":"2024-04-25T10:14:16.800661+0000","last_became_active":"2024-04-25T10:14:15.794156+0000","last_became_peered":"2024-04-25T10:14:15.794156+0000","last_unstale":"2024-04-25T10:14:16.800661+0000","last_undegraded":"2024-04-25T10:14:16.800661+0000","last_fullsized":"2024-04-25T10:14:16.800661+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:44.783969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.844001+0000","last_change":"2024-04-25T10:14:16.840252+0000","last_active":"2024-04-25T10:14:16.844001+0000","last_peered":"2024-04-25T10:14:16.844001+0000","last_clean":"2024-04-25T10:14:16.844001+0000","last_became_active":"2024-04-25T10:14:16.839437+0000","last_became_peered":"2024-04-25T10:14:16.839437+0000","last_unstale":"2024-04-25T10:14:16.844001+0000","last_undegraded":"2024-04-25T10:14:16.844001+0000","last_fullsized":"2024-04-25T10:14:16.844001+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:50.764059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.799654+0000","last_change":"2024-04-25T10:14:15.805543+0000","last_active":"2024-04-25T10:14:16.799654+0000","last_peered":"2024-04-25T10:14:16.799654+0000","last_clean":"2024-04-25T10:14:16.799654+0000","last_became_active":"2024-04-25T10:14:15.804838+0000","last_became_peered":"2024-04-25T10:14:15.804838+0000","last_unstale":"2024-04-25T10:14:16.799654+0000","last_undegraded":"2024-04-25T10:14:16.799654+0000","last_fullsized":"2024-04-25T10:14:16.799654+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:11.333769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.829223+0000","last_change":"2024-04-25T10:14:16.822757+0000","last_active":"2024-04-25T10:14:16.829223+0000","last_peered":"2024-04-25T10:14:16.829223+0000","last_clean":"2024-04-25T10:14:16.829223+0000","last_became_active":"2024-04-25T10:14:16.822019+0000","last_became_peered":"2024-04-25T10:14:16.822019+0000","last_unstale":"2024-04-25T10:14:16.829223+0000","last_undegraded":"2024-04-25T10:14:16.829223+0000","last_fullsized":"2024-04-25T10:14:16.829223+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:52.450601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.810395+0000","last_change":"2024-04-25T10:14:15.788583+0000","last_active":"2024-04-25T10:14:16.810395+0000","last_peered":"2024-04-25T10:14:16.810395+0000","last_clean":"2024-04-25T10:14:16.810395+0000","last_became_active":"2024-04-25T10:14:15.788230+0000","last_became_peered":"2024-04-25T10:14:15.788230+0000","last_unstale":"2024-04-25T10:14:16.810395+0000","last_undegraded":"2024-04-25T10:14:16.810395+0000","last_fullsized":"2024-04-25T10:14:16.810395+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:33.340986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.842353+0000","last_change":"2024-04-25T10:14:16.839074+0000","last_active":"2024-04-25T10:14:16.842353+0000","last_peered":"2024-04-25T10:14:16.842353+0000","last_clean":"2024-04-25T10:14:16.842353+0000","last_became_active":"2024-04-25T10:14:16.837444+0000","last_became_peered":"2024-04-25T10:14:16.837444+0000","last_unstale":"2024-04-25T10:14:16.842353+0000","last_undegraded":"2024-04-25T10:14:16.842353+0000","last_fullsized":"2024-04-25T10:14:16.842353+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:03.178522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"96'2","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.869329+0000","last_change":"2024-04-25T10:14:15.812784+0000","last_active":"2024-04-25T10:14:16.869329+0000","last_peered":"2024-04-25T10:14:16.869329+0000","last_clean":"2024-04-25T10:14:16.869329+0000","last_became_active":"2024-04-25T10:14:15.812062+0000","last_became_peered":"2024-04-25T10:14:15.812062+0000","last_unstale":"2024-04-25T10:14:16.869329+0000","last_undegraded":"2024-04-25T10:14:16.869329+0000","last_fullsized":"2024-04-25T10:14:16.869329+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:37.484119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.844763+0000","last_change":"2024-04-25T10:14:16.833231+0000","last_active":"2024-04-25T10:14:16.844763+0000","last_peered":"2024-04-25T10:14:16.844763+0000","last_clean":"2024-04-25T10:14:16.844763+0000","last_became_active":"2024-04-25T10:14:16.832464+0000","last_became_peered":"2024-04-25T10:14:16.832464+0000","last_unstale":"2024-04-25T10:14:16.844763+0000","last_undegraded":"2024-04-25T10:14:16.844763+0000","last_fullsized":"2024-04-25T10:14:16.844763+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:21.298499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.812079+0000","last_change":"2024-04-25T10:14:15.800905+0000","last_active":"2024-04-25T10:14:16.812079+0000","last_peered":"2024-04-25T10:14:16.812079+0000","last_clean":"2024-04-25T10:14:16.812079+0000","last_became_active":"2024-04-25T10:14:15.800508+0000","last_became_peered":"2024-04-25T10:14:15.800508+0000","last_unstale":"2024-04-25T10:14:16.812079+0000","last_undegraded":"2024-04-25T10:14:16.812079+0000","last_fullsized":"2024-04-25T10:14:16.812079+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:32.934586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.844513+0000","last_change":"2024-04-25T10:14:16.825900+0000","last_active":"2024-04-25T10:14:16.844513+0000","last_peered":"2024-04-25T10:14:16.844513+0000","last_clean":"2024-04-25T10:14:16.844513+0000","last_became_active":"2024-04-25T10:14:16.825191+0000","last_became_peered":"2024-04-25T10:14:16.825191+0000","last_unstale":"2024-04-25T10:14:16.844513+0000","last_undegraded":"2024-04-25T10:14:16.844513+0000","last_fullsized":"2024-04-25T10:14:16.844513+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:19.184363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.810272+0000","last_change":"2024-04-25T10:14:15.807382+0000","last_active":"2024-04-25T10:14:16.810272+0000","last_peered":"2024-04-25T10:14:16.810272+0000","last_clean":"2024-04-25T10:14:16.810272+0000","last_became_active":"2024-04-25T10:14:15.807067+0000","last_became_peered":"2024-04-25T10:14:15.807067+0000","last_unstale":"2024-04-25T10:14:16.810272+0000","last_undegraded":"2024-04-25T10:14:16.810272+0000","last_fullsized":"2024-04-25T10:14:16.810272+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:10.230974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.848080+0000","last_change":"2024-04-25T10:14:16.845116+0000","last_active":"2024-04-25T10:14:16.848080+0000","last_peered":"2024-04-25T10:14:16.848080+0000","last_clean":"2024-04-25T10:14:16.848080+0000","last_became_active":"2024-04-25T10:14:16.844276+0000","last_became_peered":"2024-04-25T10:14:16.844276+0000","last_unstale":"2024-04-25T10:14:16.848080+0000","last_undegraded":"2024-04-25T10:14:16.848080+0000","last_fullsized":"2024-04-25T10:14:16.848080+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:28.681380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.815637+0000","last_change":"2024-04-25T10:14:15.809526+0000","last_active":"2024-04-25T10:14:16.815637+0000","last_peered":"2024-04-25T10:14:16.815637+0000","last_clean":"2024-04-25T10:14:16.815637+0000","last_became_active":"2024-04-25T10:14:15.808949+0000","last_became_peered":"2024-04-25T10:14:15.808949+0000","last_unstale":"2024-04-25T10:14:16.815637+0000","last_undegraded":"2024-04-25T10:14:16.815637+0000","last_fullsized":"2024-04-25T10:14:16.815637+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:21.140221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.829367+0000","last_change":"2024-04-25T10:14:16.820141+0000","last_active":"2024-04-25T10:14:16.829367+0000","last_peered":"2024-04-25T10:14:16.829367+0000","last_clean":"2024-04-25T10:14:16.829367+0000","last_became_active":"2024-04-25T10:14:16.819364+0000","last_became_peered":"2024-04-25T10:14:16.819364+0000","last_unstale":"2024-04-25T10:14:16.829367+0000","last_undegraded":"2024-04-25T10:14:16.829367+0000","last_fullsized":"2024-04-25T10:14:16.829367+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:32.622159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.810135+0000","last_change":"2024-04-25T10:14:15.799281+0000","last_active":"2024-04-25T10:14:16.810135+0000","last_peered":"2024-04-25T10:14:16.810135+0000","last_clean":"2024-04-25T10:14:16.810135+0000","last_became_active":"2024-04-25T10:14:15.798941+0000","last_became_peered":"2024-04-25T10:14:15.798941+0000","last_unstale":"2024-04-25T10:14:16.810135+0000","last_undegraded":"2024-04-25T10:14:16.810135+0000","last_fullsized":"2024-04-25T10:14:16.810135+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:51.526625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.802453+0000","last_change":"2024-04-25T10:14:16.833945+0000","last_active":"2024-04-25T10:14:17.802453+0000","last_peered":"2024-04-25T10:14:17.802453+0000","last_clean":"2024-04-25T10:14:17.802453+0000","last_became_active":"2024-04-25T10:14:16.830624+0000","last_became_peered":"2024-04-25T10:14:16.830624+0000","last_unstale":"2024-04-25T10:14:17.802453+0000","last_undegraded":"2024-04-25T10:14:17.802453+0000","last_fullsized":"2024-04-25T10:14:17.802453+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:22.496567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.814185+0000","last_change":"2024-04-25T10:14:15.802215+0000","last_active":"2024-04-25T10:14:16.814185+0000","last_peered":"2024-04-25T10:14:16.814185+0000","last_clean":"2024-04-25T10:14:16.814185+0000","last_became_active":"2024-04-25T10:14:15.801581+0000","last_became_peered":"2024-04-25T10:14:15.801581+0000","last_unstale":"2024-04-25T10:14:16.814185+0000","last_undegraded":"2024-04-25T10:14:16.814185+0000","last_fullsized":"2024-04-25T10:14:16.814185+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:44.783969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.828928+0000","last_change":"2024-04-25T10:14:16.820230+0000","last_active":"2024-04-25T10:14:16.828928+0000","last_peered":"2024-04-25T10:14:16.828928+0000","last_clean":"2024-04-25T10:14:16.828928+0000","last_became_active":"2024-04-25T10:14:16.819448+0000","last_became_peered":"2024-04-25T10:14:16.819448+0000","last_unstale":"2024-04-25T10:14:16.828928+0000","last_undegraded":"2024-04-25T10:14:16.828928+0000","last_fullsized":"2024-04-25T10:14:16.828928+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:33.542463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.827993+0000","last_change":"2024-04-25T10:14:15.795044+0000","last_active":"2024-04-25T10:14:16.827993+0000","last_peered":"2024-04-25T10:14:16.827993+0000","last_clean":"2024-04-25T10:14:16.827993+0000","last_became_active":"2024-04-25T10:14:15.794668+0000","last_became_peered":"2024-04-25T10:14:15.794668+0000","last_unstale":"2024-04-25T10:14:16.827993+0000","last_undegraded":"2024-04-25T10:14:16.827993+0000","last_fullsized":"2024-04-25T10:14:16.827993+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:14.348317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.850162+0000","last_change":"2024-04-25T10:14:16.849341+0000","last_active":"2024-04-25T10:14:16.850162+0000","last_peered":"2024-04-25T10:14:16.850162+0000","last_clean":"2024-04-25T10:14:16.850162+0000","last_became_active":"2024-04-25T10:14:16.848791+0000","last_became_peered":"2024-04-25T10:14:16.848791+0000","last_unstale":"2024-04-25T10:14:16.850162+0000","last_undegraded":"2024-04-25T10:14:16.850162+0000","last_fullsized":"2024-04-25T10:14:16.850162+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:53.990178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.794706+0000","last_change":"2024-04-25T10:14:15.796624+0000","last_active":"2024-04-25T10:14:16.794706+0000","last_peered":"2024-04-25T10:14:16.794706+0000","last_clean":"2024-04-25T10:14:16.794706+0000","last_became_active":"2024-04-25T10:14:15.796285+0000","last_became_peered":"2024-04-25T10:14:15.796285+0000","last_unstale":"2024-04-25T10:14:16.794706+0000","last_undegraded":"2024-04-25T10:14:16.794706+0000","last_fullsized":"2024-04-25T10:14:16.794706+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:52.959065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.845418+0000","last_change":"2024-04-25T10:14:16.842021+0000","last_active":"2024-04-25T10:14:16.845418+0000","last_peered":"2024-04-25T10:14:16.845418+0000","last_clean":"2024-04-25T10:14:16.845418+0000","last_became_active":"2024-04-25T10:14:16.841008+0000","last_became_peered":"2024-04-25T10:14:16.841008+0000","last_unstale":"2024-04-25T10:14:16.845418+0000","last_undegraded":"2024-04-25T10:14:16.845418+0000","last_fullsized":"2024-04-25T10:14:16.845418+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:20.352191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063400+0000","last_change":"2024-04-25T10:14:15.811616+0000","last_active":"2024-04-25T10:14:18.063400+0000","last_peered":"2024-04-25T10:14:18.063400+0000","last_clean":"2024-04-25T10:14:18.063400+0000","last_became_active":"2024-04-25T10:14:15.810966+0000","last_became_peered":"2024-04-25T10:14:15.810966+0000","last_unstale":"2024-04-25T10:14:18.063400+0000","last_undegraded":"2024-04-25T10:14:18.063400+0000","last_fullsized":"2024-04-25T10:14:18.063400+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:55.209660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":309,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063458+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:14:18.063458+0000","last_peered":"2024-04-25T10:14:18.063458+0000","last_clean":"2024-04-25T10:14:18.063458+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:14:18.063458+0000","last_undegraded":"2024-04-25T10:14:18.063458+0000","last_fullsized":"2024-04-25T10:14:18.063458+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509554,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32928,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1781760,"data_stored":1097682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509553,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1212416,"data_stored":515238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542256,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1167360,"data_stored":500427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542257,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1183744,"data_stored":507272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542257,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36508,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335332,"statfs":{"total":96636764160,"available":96599379968,"internally_reserved":0,"allocated":1196032,"data_stored":508836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542257,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36488,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335352,"statfs":{"total":96636764160,"available":96599400448,"internally_reserved":0,"allocated":1167360,"data_stored":500427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542257,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36484,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335356,"statfs":{"total":96636764160,"available":96599404544,"internally_reserved":0,"allocated":1167360,"data_stored":500427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542258,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37084,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334756,"statfs":{"total":96636764160,"available":96598790144,"internally_reserved":0,"allocated":1777664,"data_stored":1098152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:14:20.916 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:14:21.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.075+0000 7effe6fad640 1 Processor -- start 2024-04-25T10:14:21.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.075+0000 7effe6fad640 1 -- start start 2024-04-25T10:14:21.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effe6fad640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe015f480 0x7effe015f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effe6fad640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7effe00cd390 0x7effe009a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effe6fad640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe015fda0 con 0x7effe009a930 2024-04-25T10:14:21.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effe6fad640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe00cc5d0 con 0x7effe00cd390 2024-04-25T10:14:21.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effe6fad640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe00cc8b0 con 0x7effe015f480 2024-04-25T10:14:21.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effe554a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7effe009a930 0x7effe00cf380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55494/0 (socket says 172.21.15.60:55494) 2024-04-25T10:14:21.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effe554a640 1 -- 172.21.15.60:0/2458001409 learned_addr learned my addr 172.21.15.60:0/2458001409 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:21.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdffff640 1 --2- 172.21.15.60:0/2458001409 >> v2:172.21.15.100:3300/0 conn(0x7effe00cd390 0x7effe009a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effe4d49640 1 --2- 172.21.15.60:0/2458001409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe015f480 0x7effe015f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdf7fe640 1 -- 172.21.15.60:0/2458001409 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1981464521 0 0) 0x7effe015fda0 con 0x7effe009a930 2024-04-25T10:14:21.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdf7fe640 1 -- 172.21.15.60:0/2458001409 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effc8003530 con 0x7effe009a930 2024-04-25T10:14:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdf7fe640 1 -- 172.21.15.60:0/2458001409 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1163285807 0 0) 0x7effc8003530 con 0x7effe009a930 2024-04-25T10:14:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdf7fe640 1 -- 172.21.15.60:0/2458001409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe015f480 msgr2=0x7effe015f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdf7fe640 1 --2- 172.21.15.60:0/2458001409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe015f480 0x7effe015f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdf7fe640 1 -- 172.21.15.60:0/2458001409 >> v2:172.21.15.100:3300/0 conn(0x7effe00cd390 msgr2=0x7effe009a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdf7fe640 1 --2- 172.21.15.60:0/2458001409 >> v2:172.21.15.100:3300/0 conn(0x7effe00cd390 0x7effe009a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdf7fe640 1 -- 172.21.15.60:0/2458001409 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe00ccb90 con 0x7effe009a930 2024-04-25T10:14:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdf7fe640 1 -- 172.21.15.60:0/2458001409 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7effd8001a00 con 0x7effe009a930 2024-04-25T10:14:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdf7fe640 1 -- 172.21.15.60:0/2458001409 >> v1:172.21.15.60:6789/0 conn(0x7effe009a930 legacy=0x7effe00cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdf7fe640 1 --2- 172.21.15.60:0/2458001409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc80039c0 0x7effc8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdf7fe640 1 --2- 172.21.15.60:0/2458001409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe015f480 0x7effc8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdf7fe640 1 --2- 172.21.15.60:0/2458001409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc8004d50 0x7effc80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdf7fe640 1 -- 172.21.15.60:0/2458001409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe00cc5d0 con 0x7effc8004d50 2024-04-25T10:14:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdf7fe640 1 -- 172.21.15.60:0/2458001409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe00cc8b0 con 0x7effc80039c0 2024-04-25T10:14:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdf7fe640 1 -- 172.21.15.60:0/2458001409 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc800b9d0 con 0x7effe015f480 2024-04-25T10:14:21.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effe554a640 1 --2- 172.21.15.60:0/2458001409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc8004d50 0x7effc80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effe4d49640 1 --2- 172.21.15.60:0/2458001409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc80039c0 0x7effc8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effdffff640 1 --2- 172.21.15.60:0/2458001409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe015f480 0x7effc8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effe554a640 1 -- 172.21.15.60:0/2458001409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe015f480 msgr2=0x7effc8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effe554a640 1 --2- 172.21.15.60:0/2458001409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe015f480 0x7effc8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effe554a640 1 -- 172.21.15.60:0/2458001409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc80039c0 msgr2=0x7effc8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effe554a640 1 --2- 172.21.15.60:0/2458001409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc80039c0 0x7effc8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.079+0000 7effe554a640 1 -- 172.21.15.60:0/2458001409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc8003530 con 0x7effc8004d50 2024-04-25T10:14:21.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe554a640 1 --2- 172.21.15.60:0/2458001409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc8004d50 0x7effc80051d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7effd80013e0 tx=0x7effd801d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77103332b4e3a81e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effdf7fe640 1 -- 172.21.15.60:0/2458001409 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd800a4e0 con 0x7effc8004d50 2024-04-25T10:14:21.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effdf7fe640 1 -- 172.21.15.60:0/2458001409 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd80ad650 con 0x7effc8004d50 2024-04-25T10:14:21.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effdf7fe640 1 -- 172.21.15.60:0/2458001409 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd80ada40 con 0x7effc8004d50 2024-04-25T10:14:21.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe6fad640 1 -- 172.21.15.60:0/2458001409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc8004d50 msgr2=0x7effc80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe6fad640 1 --2- 172.21.15.60:0/2458001409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc8004d50 0x7effc80051d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7effd80013e0 tx=0x7effd801d380 comp rx=0 tx=0).stop 2024-04-25T10:14:21.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe554a640 1 -- 172.21.15.60:0/2458001409 reap_dead start 2024-04-25T10:14:21.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe6fad640 1 -- 172.21.15.60:0/2458001409 shutdown_connections 2024-04-25T10:14:21.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe6fad640 1 -- 172.21.15.60:0/2458001409 >> 172.21.15.60:0/2458001409 conn(0x7effe00b32a0 msgr2=0x7effe00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe6fad640 1 -- 172.21.15.60:0/2458001409 shutdown_connections 2024-04-25T10:14:21.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe6fad640 1 -- 172.21.15.60:0/2458001409 wait complete. 2024-04-25T10:14:21.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe6fad640 1 Processor -- start 2024-04-25T10:14:21.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe6fad640 1 -- start start 2024-04-25T10:14:21.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe6fad640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effe0155130 0x7effe00c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe6fad640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe00c6080 0x7effe00c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe6fad640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effe00b4600 0x7effe00b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe6fad640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe0160170 con 0x7effe0155130 2024-04-25T10:14:21.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe6fad640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe00cc2c0 con 0x7effe00b4600 2024-04-25T10:14:21.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe6fad640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe01602f0 con 0x7effe00c6080 2024-04-25T10:14:21.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe4d49640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effe0155130 0x7effe00c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe4d49640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effe0155130 0x7effe00c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47472/0 (socket says 172.21.15.60:47472) 2024-04-25T10:14:21.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe4d49640 1 -- 172.21.15.60:0/349213027 learned_addr learned my addr 172.21.15.60:0/349213027 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:21.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effdffff640 1 --2- 172.21.15.60:0/349213027 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe00c6080 0x7effe00c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.083+0000 7effe554a640 1 --2- 172.21.15.60:0/349213027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effe00b4600 0x7effe00b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effe4d49640 1 -- 172.21.15.60:0/349213027 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe00c6080 msgr2=0x7effe00c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effe4d49640 1 --2- 172.21.15.60:0/349213027 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe00c6080 0x7effe00c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effe4d49640 1 -- 172.21.15.60:0/349213027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effe00b4600 msgr2=0x7effe00b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effe4d49640 1 --2- 172.21.15.60:0/349213027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effe00b4600 0x7effe00b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effe4d49640 1 -- 172.21.15.60:0/349213027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe006c730 con 0x7effe0155130 2024-04-25T10:14:21.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effe554a640 1 --2- 172.21.15.60:0/349213027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effe00b4600 0x7effe00b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:21.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effe4d49640 1 --2- 172.21.15.60:0/349213027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effe0155130 0x7effe00c5b40 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7effd0011960 tx=0x7effd0011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effdd7fa640 1 -- 172.21.15.60:0/349213027 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd000fa90 con 0x7effe0155130 2024-04-25T10:14:21.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effdd7fa640 1 -- 172.21.15.60:0/349213027 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd000fc30 con 0x7effe0155130 2024-04-25T10:14:21.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effdd7fa640 1 -- 172.21.15.60:0/349213027 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0014440 con 0x7effe0155130 2024-04-25T10:14:21.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effe6fad640 1 -- 172.21.15.60:0/349213027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe006ca10 con 0x7effe0155130 2024-04-25T10:14:21.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effe6fad640 1 -- 172.21.15.60:0/349213027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe006cfd0 con 0x7effe0155130 2024-04-25T10:14:21.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effe6fad640 1 -- 172.21.15.60:0/349213027 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe0074770 con 0x7effe0155130 2024-04-25T10:14:21.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effdd7fa640 1 -- 172.21.15.60:0/349213027 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7effd0012040 con 0x7effe0155130 2024-04-25T10:14:21.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effdd7fa640 1 --2- 172.21.15.60:0/349213027 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7effcc066f60 0x7effcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effdd7fa640 1 -- 172.21.15.60:0/349213027 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effd00855e0 con 0x7effe0155130 2024-04-25T10:14:21.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effdffff640 1 --2- 172.21.15.60:0/349213027 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7effcc066f60 0x7effcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.087+0000 7effdffff640 1 --2- 172.21.15.60:0/349213027 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7effcc066f60 0x7effcc069420 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7effe00cd9d0 tx=0x7effd4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.091+0000 7effdd7fa640 1 -- 172.21.15.60:0/349213027 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7effd0019030 con 0x7effe0155130 2024-04-25T10:14:21.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.247+0000 7effe6fad640 1 -- 172.21.15.60:0/349213027 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7effe00b7730 con 0x7effcc066f60 2024-04-25T10:14:21.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.267+0000 7effdd7fa640 1 -- 172.21.15.60:0/349213027 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252034 (secure 0 0 0) 0x7effe00b7730 con 0x7effcc066f60 2024-04-25T10:14:21.273 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:21.274 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:14:21.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.271+0000 7effe6fad640 1 -- 172.21.15.60:0/349213027 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7effcc066f60 msgr2=0x7effcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.271+0000 7effe6fad640 1 --2- 172.21.15.60:0/349213027 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7effcc066f60 0x7effcc069420 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7effe00cd9d0 tx=0x7effd4008040 comp rx=0 tx=0).stop 2024-04-25T10:14:21.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.271+0000 7effe6fad640 1 -- 172.21.15.60:0/349213027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effe0155130 msgr2=0x7effe00c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.271+0000 7effe6fad640 1 --2- 172.21.15.60:0/349213027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effe0155130 0x7effe00c5b40 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7effd0011960 tx=0x7effd0011e20 comp rx=0 tx=0).stop 2024-04-25T10:14:21.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.271+0000 7effe6fad640 1 -- 172.21.15.60:0/349213027 shutdown_connections 2024-04-25T10:14:21.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.271+0000 7effe6fad640 1 --2- 172.21.15.60:0/349213027 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7effcc066f60 0x7effcc069420 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.271+0000 7effe6fad640 1 --2- 172.21.15.60:0/349213027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effe00b4600 0x7effe00b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.271+0000 7effe6fad640 1 --2- 172.21.15.60:0/349213027 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effe00c6080 0x7effe00c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.271+0000 7effe6fad640 1 --2- 172.21.15.60:0/349213027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effe0155130 0x7effe00c5b40 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.271+0000 7effe6fad640 1 -- 172.21.15.60:0/349213027 >> 172.21.15.60:0/349213027 conn(0x7effe00b32a0 msgr2=0x7effe015c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.271+0000 7effe6fad640 1 -- 172.21.15.60:0/349213027 shutdown_connections 2024-04-25T10:14:21.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.271+0000 7effe6fad640 1 -- 172.21.15.60:0/349213027 wait complete. 2024-04-25T10:14:21.291 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-04-25T10:14:20.576410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276520,"kb_used_data":10404,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754698200,"statfs":{"total":773094113280,"available":772810956800,"internally_reserved":0,"allocated":10653696,"data_stored":5228461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.806690"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.828185+0000","last_change":"2024-04-25T10:14:16.815098+0000","last_active":"2024-04-25T10:14:16.828185+0000","last_peered":"2024-04-25T10:14:16.828185+0000","last_clean":"2024-04-25T10:14:16.828185+0000","last_became_active":"2024-04-25T10:14:16.814339+0000","last_became_peered":"2024-04-25T10:14:16.814339+0000","last_unstale":"2024-04-25T10:14:16.828185+0000","last_undegraded":"2024-04-25T10:14:16.828185+0000","last_fullsized":"2024-04-25T10:14:16.828185+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:00.499368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.844682+0000","last_change":"2024-04-25T10:14:16.839651+0000","last_active":"2024-04-25T10:14:16.844682+0000","last_peered":"2024-04-25T10:14:16.844682+0000","last_clean":"2024-04-25T10:14:16.844682+0000","last_became_active":"2024-04-25T10:14:16.838355+0000","last_became_peered":"2024-04-25T10:14:16.838355+0000","last_unstale":"2024-04-25T10:14:16.844682+0000","last_undegraded":"2024-04-25T10:14:16.844682+0000","last_fullsized":"2024-04-25T10:14:16.844682+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:32.610149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.818475+0000","last_change":"2024-04-25T10:14:16.813762+0000","last_active":"2024-04-25T10:14:16.818475+0000","last_peered":"2024-04-25T10:14:16.818475+0000","last_clean":"2024-04-25T10:14:16.818475+0000","last_became_active":"2024-04-25T10:14:16.812971+0000","last_became_peered":"2024-04-25T10:14:16.812971+0000","last_unstale":"2024-04-25T10:14:16.818475+0000","last_undegraded":"2024-04-25T10:14:16.818475+0000","last_fullsized":"2024-04-25T10:14:16.818475+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:32.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.838906+0000","last_change":"2024-04-25T10:14:16.830279+0000","last_active":"2024-04-25T10:14:16.838906+0000","last_peered":"2024-04-25T10:14:16.838906+0000","last_clean":"2024-04-25T10:14:16.838906+0000","last_became_active":"2024-04-25T10:14:16.829625+0000","last_became_peered":"2024-04-25T10:14:16.829625+0000","last_unstale":"2024-04-25T10:14:16.838906+0000","last_undegraded":"2024-04-25T10:14:16.838906+0000","last_fullsized":"2024-04-25T10:14:16.838906+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:53.990178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.848145+0000","last_change":"2024-04-25T10:14:16.845675+0000","last_active":"2024-04-25T10:14:16.848145+0000","last_peered":"2024-04-25T10:14:16.848145+0000","last_clean":"2024-04-25T10:14:16.848145+0000","last_became_active":"2024-04-25T10:14:16.844616+0000","last_became_peered":"2024-04-25T10:14:16.844616+0000","last_unstale":"2024-04-25T10:14:16.848145+0000","last_undegraded":"2024-04-25T10:14:16.848145+0000","last_fullsized":"2024-04-25T10:14:16.848145+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:15.358398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063452+0000","last_change":"2024-04-25T10:14:16.842235+0000","last_active":"2024-04-25T10:14:18.063452+0000","last_peered":"2024-04-25T10:14:18.063452+0000","last_clean":"2024-04-25T10:14:18.063452+0000","last_became_active":"2024-04-25T10:14:16.841382+0000","last_became_peered":"2024-04-25T10:14:16.841382+0000","last_unstale":"2024-04-25T10:14:18.063452+0000","last_undegraded":"2024-04-25T10:14:18.063452+0000","last_fullsized":"2024-04-25T10:14:18.063452+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:05.727695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063511+0000","last_change":"2024-04-25T10:14:16.842333+0000","last_active":"2024-04-25T10:14:18.063511+0000","last_peered":"2024-04-25T10:14:18.063511+0000","last_clean":"2024-04-25T10:14:18.063511+0000","last_became_active":"2024-04-25T10:14:16.841611+0000","last_became_peered":"2024-04-25T10:14:16.841611+0000","last_unstale":"2024-04-25T10:14:18.063511+0000","last_undegraded":"2024-04-25T10:14:18.063511+0000","last_fullsized":"2024-04-25T10:14:18.063511+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:26.423483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.841925+0000","last_change":"2024-04-25T10:14:16.836100+0000","last_active":"2024-04-25T10:14:16.841925+0000","last_peered":"2024-04-25T10:14:16.841925+0000","last_clean":"2024-04-25T10:14:16.841925+0000","last_became_active":"2024-04-25T10:14:16.831230+0000","last_became_peered":"2024-04-25T10:14:16.831230+0000","last_unstale":"2024-04-25T10:14:16.841925+0000","last_undegraded":"2024-04-25T10:14:16.841925+0000","last_fullsized":"2024-04-25T10:14:16.841925+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:52.305296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.803811+0000","last_change":"2024-04-25T10:14:16.844498+0000","last_active":"2024-04-25T10:14:17.803811+0000","last_peered":"2024-04-25T10:14:17.803811+0000","last_clean":"2024-04-25T10:14:17.803811+0000","last_became_active":"2024-04-25T10:14:16.843783+0000","last_became_peered":"2024-04-25T10:14:16.843783+0000","last_unstale":"2024-04-25T10:14:17.803811+0000","last_undegraded":"2024-04-25T10:14:17.803811+0000","last_fullsized":"2024-04-25T10:14:17.803811+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:37.215360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.832853+0000","last_change":"2024-04-25T10:14:16.825791+0000","last_active":"2024-04-25T10:14:16.832853+0000","last_peered":"2024-04-25T10:14:16.832853+0000","last_clean":"2024-04-25T10:14:16.832853+0000","last_became_active":"2024-04-25T10:14:16.824962+0000","last_became_peered":"2024-04-25T10:14:16.824962+0000","last_unstale":"2024-04-25T10:14:16.832853+0000","last_undegraded":"2024-04-25T10:14:16.832853+0000","last_fullsized":"2024-04-25T10:14:16.832853+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:15.211114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063989+0000","last_change":"2024-04-25T10:14:16.834336+0000","last_active":"2024-04-25T10:14:18.063989+0000","last_peered":"2024-04-25T10:14:18.063989+0000","last_clean":"2024-04-25T10:14:18.063989+0000","last_became_active":"2024-04-25T10:14:16.833561+0000","last_became_peered":"2024-04-25T10:14:16.833561+0000","last_unstale":"2024-04-25T10:14:18.063989+0000","last_undegraded":"2024-04-25T10:14:18.063989+0000","last_fullsized":"2024-04-25T10:14:18.063989+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:33.492135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.803014+0000","last_change":"2024-04-25T10:14:16.824569+0000","last_active":"2024-04-25T10:14:17.803014+0000","last_peered":"2024-04-25T10:14:17.803014+0000","last_clean":"2024-04-25T10:14:17.803014+0000","last_became_active":"2024-04-25T10:14:16.823752+0000","last_became_peered":"2024-04-25T10:14:16.823752+0000","last_unstale":"2024-04-25T10:14:17.803014+0000","last_undegraded":"2024-04-25T10:14:17.803014+0000","last_fullsized":"2024-04-25T10:14:17.803014+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:33.765433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.839015+0000","last_change":"2024-04-25T10:14:16.830478+0000","last_active":"2024-04-25T10:14:16.839015+0000","last_peered":"2024-04-25T10:14:16.839015+0000","last_clean":"2024-04-25T10:14:16.839015+0000","last_became_active":"2024-04-25T10:14:16.829702+0000","last_became_peered":"2024-04-25T10:14:16.829702+0000","last_unstale":"2024-04-25T10:14:16.839015+0000","last_undegraded":"2024-04-25T10:14:16.839015+0000","last_fullsized":"2024-04-25T10:14:16.839015+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:52.536706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.829266+0000","last_change":"2024-04-25T10:14:16.809416+0000","last_active":"2024-04-25T10:14:16.829266+0000","last_peered":"2024-04-25T10:14:16.829266+0000","last_clean":"2024-04-25T10:14:16.829266+0000","last_became_active":"2024-04-25T10:14:16.808672+0000","last_became_peered":"2024-04-25T10:14:16.808672+0000","last_unstale":"2024-04-25T10:14:16.829266+0000","last_undegraded":"2024-04-25T10:14:16.829266+0000","last_fullsized":"2024-04-25T10:14:16.829266+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:29.820849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.845446+0000","last_change":"2024-04-25T10:14:16.840401+0000","last_active":"2024-04-25T10:14:16.845446+0000","last_peered":"2024-04-25T10:14:16.845446+0000","last_clean":"2024-04-25T10:14:16.845446+0000","last_became_active":"2024-04-25T10:14:16.837968+0000","last_became_peered":"2024-04-25T10:14:16.837968+0000","last_unstale":"2024-04-25T10:14:16.845446+0000","last_undegraded":"2024-04-25T10:14:16.845446+0000","last_fullsized":"2024-04-25T10:14:16.845446+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:33.763115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063068+0000","last_change":"2024-04-25T10:14:16.828675+0000","last_active":"2024-04-25T10:14:18.063068+0000","last_peered":"2024-04-25T10:14:18.063068+0000","last_clean":"2024-04-25T10:14:18.063068+0000","last_became_active":"2024-04-25T10:14:16.827961+0000","last_became_peered":"2024-04-25T10:14:16.827961+0000","last_unstale":"2024-04-25T10:14:18.063068+0000","last_undegraded":"2024-04-25T10:14:18.063068+0000","last_fullsized":"2024-04-25T10:14:18.063068+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:09.929605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.845936+0000","last_change":"2024-04-25T10:14:16.842579+0000","last_active":"2024-04-25T10:14:16.845936+0000","last_peered":"2024-04-25T10:14:16.845936+0000","last_clean":"2024-04-25T10:14:16.845936+0000","last_became_active":"2024-04-25T10:14:16.841857+0000","last_became_peered":"2024-04-25T10:14:16.841857+0000","last_unstale":"2024-04-25T10:14:16.845936+0000","last_undegraded":"2024-04-25T10:14:16.845936+0000","last_fullsized":"2024-04-25T10:14:16.845936+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:16.094182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.839061+0000","last_change":"2024-04-25T10:14:16.834470+0000","last_active":"2024-04-25T10:14:16.839061+0000","last_peered":"2024-04-25T10:14:16.839061+0000","last_clean":"2024-04-25T10:14:16.839061+0000","last_became_active":"2024-04-25T10:14:16.833645+0000","last_became_peered":"2024-04-25T10:14:16.833645+0000","last_unstale":"2024-04-25T10:14:16.839061+0000","last_undegraded":"2024-04-25T10:14:16.839061+0000","last_fullsized":"2024-04-25T10:14:16.839061+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:11.241056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.828761+0000","last_change":"2024-04-25T10:14:16.817847+0000","last_active":"2024-04-25T10:14:16.828761+0000","last_peered":"2024-04-25T10:14:16.828761+0000","last_clean":"2024-04-25T10:14:16.828761+0000","last_became_active":"2024-04-25T10:14:16.817086+0000","last_became_peered":"2024-04-25T10:14:16.817086+0000","last_unstale":"2024-04-25T10:14:16.828761+0000","last_undegraded":"2024-04-25T10:14:16.828761+0000","last_fullsized":"2024-04-25T10:14:16.828761+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:20.632609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.803239+0000","last_change":"2024-04-25T10:14:16.839117+0000","last_active":"2024-04-25T10:14:17.803239+0000","last_peered":"2024-04-25T10:14:17.803239+0000","last_clean":"2024-04-25T10:14:17.803239+0000","last_became_active":"2024-04-25T10:14:16.838446+0000","last_became_peered":"2024-04-25T10:14:16.838446+0000","last_unstale":"2024-04-25T10:14:17.803239+0000","last_undegraded":"2024-04-25T10:14:17.803239+0000","last_fullsized":"2024-04-25T10:14:17.803239+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:56.219741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.823877+0000","last_change":"2024-04-25T10:14:16.802268+0000","last_active":"2024-04-25T10:14:16.823877+0000","last_peered":"2024-04-25T10:14:16.823877+0000","last_clean":"2024-04-25T10:14:16.823877+0000","last_became_active":"2024-04-25T10:14:16.801628+0000","last_became_peered":"2024-04-25T10:14:16.801628+0000","last_unstale":"2024-04-25T10:14:16.823877+0000","last_undegraded":"2024-04-25T10:14:16.823877+0000","last_fullsized":"2024-04-25T10:14:16.823877+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:22.469287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.841473+0000","last_change":"2024-04-25T10:14:16.836272+0000","last_active":"2024-04-25T10:14:16.841473+0000","last_peered":"2024-04-25T10:14:16.841473+0000","last_clean":"2024-04-25T10:14:16.841473+0000","last_became_active":"2024-04-25T10:14:16.831376+0000","last_became_peered":"2024-04-25T10:14:16.831376+0000","last_unstale":"2024-04-25T10:14:16.841473+0000","last_undegraded":"2024-04-25T10:14:16.841473+0000","last_fullsized":"2024-04-25T10:14:16.841473+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:26.205046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.842271+0000","last_change":"2024-04-25T10:14:16.837060+0000","last_active":"2024-04-25T10:14:16.842271+0000","last_peered":"2024-04-25T10:14:16.842271+0000","last_clean":"2024-04-25T10:14:16.842271+0000","last_became_active":"2024-04-25T10:14:16.831663+0000","last_became_peered":"2024-04-25T10:14:16.831663+0000","last_unstale":"2024-04-25T10:14:16.842271+0000","last_undegraded":"2024-04-25T10:14:16.842271+0000","last_fullsized":"2024-04-25T10:14:16.842271+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:59.352831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063264+0000","last_change":"2024-04-25T10:14:16.832957+0000","last_active":"2024-04-25T10:14:18.063264+0000","last_peered":"2024-04-25T10:14:18.063264+0000","last_clean":"2024-04-25T10:14:18.063264+0000","last_became_active":"2024-04-25T10:14:16.832078+0000","last_became_peered":"2024-04-25T10:14:16.832078+0000","last_unstale":"2024-04-25T10:14:18.063264+0000","last_undegraded":"2024-04-25T10:14:18.063264+0000","last_fullsized":"2024-04-25T10:14:18.063264+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:39.379522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.835429+0000","last_change":"2024-04-25T10:14:16.830004+0000","last_active":"2024-04-25T10:14:16.835429+0000","last_peered":"2024-04-25T10:14:16.835429+0000","last_clean":"2024-04-25T10:14:16.835429+0000","last_became_active":"2024-04-25T10:14:16.828640+0000","last_became_peered":"2024-04-25T10:14:16.828640+0000","last_unstale":"2024-04-25T10:14:16.835429+0000","last_undegraded":"2024-04-25T10:14:16.835429+0000","last_fullsized":"2024-04-25T10:14:16.835429+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:53.877666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.842000+0000","last_change":"2024-04-25T10:14:16.823761+0000","last_active":"2024-04-25T10:14:16.842000+0000","last_peered":"2024-04-25T10:14:16.842000+0000","last_clean":"2024-04-25T10:14:16.842000+0000","last_became_active":"2024-04-25T10:14:16.822949+0000","last_became_peered":"2024-04-25T10:14:16.822949+0000","last_unstale":"2024-04-25T10:14:16.842000+0000","last_undegraded":"2024-04-25T10:14:16.842000+0000","last_fullsized":"2024-04-25T10:14:16.842000+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:14.648790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.064317+0000","last_change":"2024-04-25T10:14:16.840130+0000","last_active":"2024-04-25T10:14:18.064317+0000","last_peered":"2024-04-25T10:14:18.064317+0000","last_clean":"2024-04-25T10:14:18.064317+0000","last_became_active":"2024-04-25T10:14:16.838781+0000","last_became_peered":"2024-04-25T10:14:16.838781+0000","last_unstale":"2024-04-25T10:14:18.064317+0000","last_undegraded":"2024-04-25T10:14:18.064317+0000","last_fullsized":"2024-04-25T10:14:18.064317+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:36.699661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.848070+0000","last_change":"2024-04-25T10:14:16.844672+0000","last_active":"2024-04-25T10:14:16.848070+0000","last_peered":"2024-04-25T10:14:16.848070+0000","last_clean":"2024-04-25T10:14:16.848070+0000","last_became_active":"2024-04-25T10:14:16.843854+0000","last_became_peered":"2024-04-25T10:14:16.843854+0000","last_unstale":"2024-04-25T10:14:16.848070+0000","last_undegraded":"2024-04-25T10:14:16.848070+0000","last_fullsized":"2024-04-25T10:14:16.848070+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:56.590783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.837244+0000","last_change":"2024-04-25T10:14:16.823917+0000","last_active":"2024-04-25T10:14:16.837244+0000","last_peered":"2024-04-25T10:14:16.837244+0000","last_clean":"2024-04-25T10:14:16.837244+0000","last_became_active":"2024-04-25T10:14:16.823098+0000","last_became_peered":"2024-04-25T10:14:16.823098+0000","last_unstale":"2024-04-25T10:14:16.837244+0000","last_undegraded":"2024-04-25T10:14:16.837244+0000","last_fullsized":"2024-04-25T10:14:16.837244+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:15.050081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.829318+0000","last_change":"2024-04-25T10:14:16.816406+0000","last_active":"2024-04-25T10:14:16.829318+0000","last_peered":"2024-04-25T10:14:16.829318+0000","last_clean":"2024-04-25T10:14:16.829318+0000","last_became_active":"2024-04-25T10:14:16.815619+0000","last_became_peered":"2024-04-25T10:14:16.815619+0000","last_unstale":"2024-04-25T10:14:16.829318+0000","last_undegraded":"2024-04-25T10:14:16.829318+0000","last_fullsized":"2024-04-25T10:14:16.829318+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:33.722277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.845500+0000","last_change":"2024-04-25T10:14:16.840606+0000","last_active":"2024-04-25T10:14:16.845500+0000","last_peered":"2024-04-25T10:14:16.845500+0000","last_clean":"2024-04-25T10:14:16.845500+0000","last_became_active":"2024-04-25T10:14:16.839604+0000","last_became_peered":"2024-04-25T10:14:16.839604+0000","last_unstale":"2024-04-25T10:14:16.845500+0000","last_undegraded":"2024-04-25T10:14:16.845500+0000","last_fullsized":"2024-04-25T10:14:16.845500+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:33.944668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.842999+0000","last_change":"2024-04-25T10:14:16.823664+0000","last_active":"2024-04-25T10:14:16.842999+0000","last_peered":"2024-04-25T10:14:16.842999+0000","last_clean":"2024-04-25T10:14:16.842999+0000","last_became_active":"2024-04-25T10:14:16.822926+0000","last_became_peered":"2024-04-25T10:14:16.822926+0000","last_unstale":"2024-04-25T10:14:16.842999+0000","last_undegraded":"2024-04-25T10:14:16.842999+0000","last_fullsized":"2024-04-25T10:14:16.842999+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:43.890875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.831144+0000","last_change":"2024-04-25T10:14:16.822544+0000","last_active":"2024-04-25T10:14:16.831144+0000","last_peered":"2024-04-25T10:14:16.831144+0000","last_clean":"2024-04-25T10:14:16.831144+0000","last_became_active":"2024-04-25T10:14:16.821713+0000","last_became_peered":"2024-04-25T10:14:16.821713+0000","last_unstale":"2024-04-25T10:14:16.831144+0000","last_undegraded":"2024-04-25T10:14:16.831144+0000","last_fullsized":"2024-04-25T10:14:16.831144+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:28.734252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"96'4","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.864092+0000","last_change":"2024-04-25T10:14:15.805877+0000","last_active":"2024-04-25T10:14:16.864092+0000","last_peered":"2024-04-25T10:14:16.864092+0000","last_clean":"2024-04-25T10:14:16.864092+0000","last_became_active":"2024-04-25T10:14:15.805542+0000","last_became_peered":"2024-04-25T10:14:15.805542+0000","last_unstale":"2024-04-25T10:14:16.864092+0000","last_undegraded":"2024-04-25T10:14:16.864092+0000","last_fullsized":"2024-04-25T10:14:16.864092+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:10.983038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.842890+0000","last_change":"2024-04-25T10:14:16.837300+0000","last_active":"2024-04-25T10:14:16.842890+0000","last_peered":"2024-04-25T10:14:16.842890+0000","last_clean":"2024-04-25T10:14:16.842890+0000","last_became_active":"2024-04-25T10:14:16.833190+0000","last_became_peered":"2024-04-25T10:14:16.833190+0000","last_unstale":"2024-04-25T10:14:16.842890+0000","last_undegraded":"2024-04-25T10:14:16.842890+0000","last_fullsized":"2024-04-25T10:14:16.842890+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:34.351067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063309+0000","last_change":"2024-04-25T10:14:15.809678+0000","last_active":"2024-04-25T10:14:18.063309+0000","last_peered":"2024-04-25T10:14:18.063309+0000","last_clean":"2024-04-25T10:14:18.063309+0000","last_became_active":"2024-04-25T10:14:15.808934+0000","last_became_peered":"2024-04-25T10:14:15.808934+0000","last_unstale":"2024-04-25T10:14:18.063309+0000","last_undegraded":"2024-04-25T10:14:18.063309+0000","last_fullsized":"2024-04-25T10:14:18.063309+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:32.755352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.844567+0000","last_change":"2024-04-25T10:14:16.827136+0000","last_active":"2024-04-25T10:14:16.844567+0000","last_peered":"2024-04-25T10:14:16.844567+0000","last_clean":"2024-04-25T10:14:16.844567+0000","last_became_active":"2024-04-25T10:14:16.825869+0000","last_became_peered":"2024-04-25T10:14:16.825869+0000","last_unstale":"2024-04-25T10:14:16.844567+0000","last_undegraded":"2024-04-25T10:14:16.844567+0000","last_fullsized":"2024-04-25T10:14:16.844567+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:39.741353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"96'2","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.858743+0000","last_change":"2024-04-25T10:14:15.800268+0000","last_active":"2024-04-25T10:14:16.858743+0000","last_peered":"2024-04-25T10:14:16.858743+0000","last_clean":"2024-04-25T10:14:16.858743+0000","last_became_active":"2024-04-25T10:14:15.799860+0000","last_became_peered":"2024-04-25T10:14:15.799860+0000","last_unstale":"2024-04-25T10:14:16.858743+0000","last_undegraded":"2024-04-25T10:14:16.858743+0000","last_fullsized":"2024-04-25T10:14:16.858743+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:14.348317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.803223+0000","last_change":"2024-04-25T10:14:16.823722+0000","last_active":"2024-04-25T10:14:17.803223+0000","last_peered":"2024-04-25T10:14:17.803223+0000","last_clean":"2024-04-25T10:14:17.803223+0000","last_became_active":"2024-04-25T10:14:16.822623+0000","last_became_peered":"2024-04-25T10:14:16.822623+0000","last_unstale":"2024-04-25T10:14:17.803223+0000","last_undegraded":"2024-04-25T10:14:17.803223+0000","last_fullsized":"2024-04-25T10:14:17.803223+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:18.001521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"96'2","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.863850+0000","last_change":"2024-04-25T10:14:15.811285+0000","last_active":"2024-04-25T10:14:16.863850+0000","last_peered":"2024-04-25T10:14:16.863850+0000","last_clean":"2024-04-25T10:14:16.863850+0000","last_became_active":"2024-04-25T10:14:15.810949+0000","last_became_peered":"2024-04-25T10:14:15.810949+0000","last_unstale":"2024-04-25T10:14:16.863850+0000","last_undegraded":"2024-04-25T10:14:16.863850+0000","last_fullsized":"2024-04-25T10:14:16.863850+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:52.980097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.842474+0000","last_change":"2024-04-25T10:14:16.840014+0000","last_active":"2024-04-25T10:14:16.842474+0000","last_peered":"2024-04-25T10:14:16.842474+0000","last_clean":"2024-04-25T10:14:16.842474+0000","last_became_active":"2024-04-25T10:14:16.839106+0000","last_became_peered":"2024-04-25T10:14:16.839106+0000","last_unstale":"2024-04-25T10:14:16.842474+0000","last_undegraded":"2024-04-25T10:14:16.842474+0000","last_fullsized":"2024-04-25T10:14:16.842474+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:25.477777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"96'1","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.863768+0000","last_change":"2024-04-25T10:14:15.797291+0000","last_active":"2024-04-25T10:14:16.863768+0000","last_peered":"2024-04-25T10:14:16.863768+0000","last_clean":"2024-04-25T10:14:16.863768+0000","last_became_active":"2024-04-25T10:14:15.796842+0000","last_became_peered":"2024-04-25T10:14:15.796842+0000","last_unstale":"2024-04-25T10:14:16.863768+0000","last_undegraded":"2024-04-25T10:14:16.863768+0000","last_fullsized":"2024-04-25T10:14:16.863768+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:34.581645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063893+0000","last_change":"2024-04-25T10:14:16.836280+0000","last_active":"2024-04-25T10:14:18.063893+0000","last_peered":"2024-04-25T10:14:18.063893+0000","last_clean":"2024-04-25T10:14:18.063893+0000","last_became_active":"2024-04-25T10:14:16.835426+0000","last_became_peered":"2024-04-25T10:14:16.835426+0000","last_unstale":"2024-04-25T10:14:18.063893+0000","last_undegraded":"2024-04-25T10:14:18.063893+0000","last_fullsized":"2024-04-25T10:14:18.063893+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:48.818995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"96'1","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.859581+0000","last_change":"2024-04-25T10:14:15.789221+0000","last_active":"2024-04-25T10:14:16.859581+0000","last_peered":"2024-04-25T10:14:16.859581+0000","last_clean":"2024-04-25T10:14:16.859581+0000","last_became_active":"2024-04-25T10:14:15.788949+0000","last_became_peered":"2024-04-25T10:14:15.788949+0000","last_unstale":"2024-04-25T10:14:16.859581+0000","last_undegraded":"2024-04-25T10:14:16.859581+0000","last_fullsized":"2024-04-25T10:14:16.859581+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:32.753034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.839141+0000","last_change":"2024-04-25T10:14:16.834847+0000","last_active":"2024-04-25T10:14:16.839141+0000","last_peered":"2024-04-25T10:14:16.839141+0000","last_clean":"2024-04-25T10:14:16.839141+0000","last_became_active":"2024-04-25T10:14:16.834104+0000","last_became_peered":"2024-04-25T10:14:16.834104+0000","last_unstale":"2024-04-25T10:14:16.839141+0000","last_undegraded":"2024-04-25T10:14:16.839141+0000","last_fullsized":"2024-04-25T10:14:16.839141+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:05.447159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.814406+0000","last_change":"2024-04-25T10:14:15.804657+0000","last_active":"2024-04-25T10:14:16.814406+0000","last_peered":"2024-04-25T10:14:16.814406+0000","last_clean":"2024-04-25T10:14:16.814406+0000","last_became_active":"2024-04-25T10:14:15.803934+0000","last_became_peered":"2024-04-25T10:14:15.803934+0000","last_unstale":"2024-04-25T10:14:16.814406+0000","last_undegraded":"2024-04-25T10:14:16.814406+0000","last_fullsized":"2024-04-25T10:14:16.814406+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:37.484119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.801535+0000","last_change":"2024-04-25T10:14:16.812686+0000","last_active":"2024-04-25T10:14:17.801535+0000","last_peered":"2024-04-25T10:14:17.801535+0000","last_clean":"2024-04-25T10:14:17.801535+0000","last_became_active":"2024-04-25T10:14:16.810803+0000","last_became_peered":"2024-04-25T10:14:16.810803+0000","last_unstale":"2024-04-25T10:14:17.801535+0000","last_undegraded":"2024-04-25T10:14:17.801535+0000","last_fullsized":"2024-04-25T10:14:17.801535+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:53.331240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.804241+0000","last_change":"2024-04-25T10:14:15.798425+0000","last_active":"2024-04-25T10:14:16.804241+0000","last_peered":"2024-04-25T10:14:16.804241+0000","last_clean":"2024-04-25T10:14:16.804241+0000","last_became_active":"2024-04-25T10:14:15.798030+0000","last_became_peered":"2024-04-25T10:14:15.798030+0000","last_unstale":"2024-04-25T10:14:16.804241+0000","last_undegraded":"2024-04-25T10:14:16.804241+0000","last_fullsized":"2024-04-25T10:14:16.804241+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:32.884443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.842056+0000","last_change":"2024-04-25T10:14:16.827231+0000","last_active":"2024-04-25T10:14:16.842056+0000","last_peered":"2024-04-25T10:14:16.842056+0000","last_clean":"2024-04-25T10:14:16.842056+0000","last_became_active":"2024-04-25T10:14:16.826423+0000","last_became_peered":"2024-04-25T10:14:16.826423+0000","last_unstale":"2024-04-25T10:14:16.842056+0000","last_undegraded":"2024-04-25T10:14:16.842056+0000","last_fullsized":"2024-04-25T10:14:16.842056+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:14.354255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.813881+0000","last_change":"2024-04-25T10:14:15.793711+0000","last_active":"2024-04-25T10:14:16.813881+0000","last_peered":"2024-04-25T10:14:16.813881+0000","last_clean":"2024-04-25T10:14:16.813881+0000","last_became_active":"2024-04-25T10:14:15.793312+0000","last_became_peered":"2024-04-25T10:14:15.793312+0000","last_unstale":"2024-04-25T10:14:16.813881+0000","last_undegraded":"2024-04-25T10:14:16.813881+0000","last_fullsized":"2024-04-25T10:14:16.813881+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:58.969202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.803867+0000","last_change":"2024-04-25T10:14:16.814806+0000","last_active":"2024-04-25T10:14:17.803867+0000","last_peered":"2024-04-25T10:14:17.803867+0000","last_clean":"2024-04-25T10:14:17.803867+0000","last_became_active":"2024-04-25T10:14:16.814036+0000","last_became_peered":"2024-04-25T10:14:16.814036+0000","last_unstale":"2024-04-25T10:14:17.803867+0000","last_undegraded":"2024-04-25T10:14:17.803867+0000","last_fullsized":"2024-04-25T10:14:17.803867+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:41.386848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.817157+0000","last_change":"2024-04-25T10:14:15.793546+0000","last_active":"2024-04-25T10:14:16.817157+0000","last_peered":"2024-04-25T10:14:16.817157+0000","last_clean":"2024-04-25T10:14:16.817157+0000","last_became_active":"2024-04-25T10:14:15.793087+0000","last_became_peered":"2024-04-25T10:14:15.793087+0000","last_unstale":"2024-04-25T10:14:16.817157+0000","last_undegraded":"2024-04-25T10:14:16.817157+0000","last_fullsized":"2024-04-25T10:14:16.817157+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:27.671299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.835914+0000","last_change":"2024-04-25T10:14:16.829166+0000","last_active":"2024-04-25T10:14:16.835914+0000","last_peered":"2024-04-25T10:14:16.835914+0000","last_clean":"2024-04-25T10:14:16.835914+0000","last_became_active":"2024-04-25T10:14:16.828439+0000","last_became_peered":"2024-04-25T10:14:16.828439+0000","last_unstale":"2024-04-25T10:14:16.835914+0000","last_undegraded":"2024-04-25T10:14:16.835914+0000","last_fullsized":"2024-04-25T10:14:16.835914+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:01.715936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"96'3","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.064997+0000","last_change":"2024-04-25T10:14:15.815968+0000","last_active":"2024-04-25T10:14:18.064997+0000","last_peered":"2024-04-25T10:14:18.064997+0000","last_clean":"2024-04-25T10:14:18.064997+0000","last_became_active":"2024-04-25T10:14:15.815092+0000","last_became_peered":"2024-04-25T10:14:15.815092+0000","last_unstale":"2024-04-25T10:14:18.064997+0000","last_undegraded":"2024-04-25T10:14:18.064997+0000","last_fullsized":"2024-04-25T10:14:18.064997+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:16.991440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.064731+0000","last_change":"2024-04-25T10:14:16.839865+0000","last_active":"2024-04-25T10:14:18.064731+0000","last_peered":"2024-04-25T10:14:18.064731+0000","last_clean":"2024-04-25T10:14:18.064731+0000","last_became_active":"2024-04-25T10:14:16.831654+0000","last_became_peered":"2024-04-25T10:14:16.831654+0000","last_unstale":"2024-04-25T10:14:18.064731+0000","last_undegraded":"2024-04-25T10:14:18.064731+0000","last_fullsized":"2024-04-25T10:14:18.064731+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:57.140722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"96'6","reported_seq":20,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.803484+0000","last_change":"2024-04-25T10:14:15.804890+0000","last_active":"2024-04-25T10:14:17.803484+0000","last_peered":"2024-04-25T10:14:17.803484+0000","last_clean":"2024-04-25T10:14:17.803484+0000","last_became_active":"2024-04-25T10:14:15.803941+0000","last_became_peered":"2024-04-25T10:14:15.803941+0000","last_unstale":"2024-04-25T10:14:17.803484+0000","last_undegraded":"2024-04-25T10:14:17.803484+0000","last_fullsized":"2024-04-25T10:14:17.803484+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:25.029340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.845308+0000","last_change":"2024-04-25T10:14:16.838331+0000","last_active":"2024-04-25T10:14:16.845308+0000","last_peered":"2024-04-25T10:14:16.845308+0000","last_clean":"2024-04-25T10:14:16.845308+0000","last_became_active":"2024-04-25T10:14:16.835377+0000","last_became_peered":"2024-04-25T10:14:16.835377+0000","last_unstale":"2024-04-25T10:14:16.845308+0000","last_undegraded":"2024-04-25T10:14:16.845308+0000","last_fullsized":"2024-04-25T10:14:16.845308+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:11.993119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"96'2","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.864354+0000","last_change":"2024-04-25T10:14:15.798437+0000","last_active":"2024-04-25T10:14:16.864354+0000","last_peered":"2024-04-25T10:14:16.864354+0000","last_clean":"2024-04-25T10:14:16.864354+0000","last_became_active":"2024-04-25T10:14:15.797924+0000","last_became_peered":"2024-04-25T10:14:15.797924+0000","last_unstale":"2024-04-25T10:14:16.864354+0000","last_undegraded":"2024-04-25T10:14:16.864354+0000","last_fullsized":"2024-04-25T10:14:16.864354+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:17.277691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.844633+0000","last_change":"2024-04-25T10:14:16.828535+0000","last_active":"2024-04-25T10:14:16.844633+0000","last_peered":"2024-04-25T10:14:16.844633+0000","last_clean":"2024-04-25T10:14:16.844633+0000","last_became_active":"2024-04-25T10:14:16.827773+0000","last_became_peered":"2024-04-25T10:14:16.827773+0000","last_unstale":"2024-04-25T10:14:16.844633+0000","last_undegraded":"2024-04-25T10:14:16.844633+0000","last_fullsized":"2024-04-25T10:14:16.844633+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:28.448033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.796597+0000","last_change":"2024-04-25T10:14:15.796462+0000","last_active":"2024-04-25T10:14:16.796597+0000","last_peered":"2024-04-25T10:14:16.796597+0000","last_clean":"2024-04-25T10:14:16.796597+0000","last_became_active":"2024-04-25T10:14:15.795901+0000","last_became_peered":"2024-04-25T10:14:15.795901+0000","last_unstale":"2024-04-25T10:14:16.796597+0000","last_undegraded":"2024-04-25T10:14:16.796597+0000","last_fullsized":"2024-04-25T10:14:16.796597+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:03.804755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.801067+0000","last_change":"2024-04-25T10:14:16.813690+0000","last_active":"2024-04-25T10:14:17.801067+0000","last_peered":"2024-04-25T10:14:17.801067+0000","last_clean":"2024-04-25T10:14:17.801067+0000","last_became_active":"2024-04-25T10:14:16.812956+0000","last_became_peered":"2024-04-25T10:14:16.812956+0000","last_unstale":"2024-04-25T10:14:17.801067+0000","last_undegraded":"2024-04-25T10:14:17.801067+0000","last_fullsized":"2024-04-25T10:14:17.801067+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:43.857050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.801615+0000","last_change":"2024-04-25T10:14:15.816673+0000","last_active":"2024-04-25T10:14:17.801615+0000","last_peered":"2024-04-25T10:14:17.801615+0000","last_clean":"2024-04-25T10:14:17.801615+0000","last_became_active":"2024-04-25T10:14:15.815942+0000","last_became_peered":"2024-04-25T10:14:15.815942+0000","last_unstale":"2024-04-25T10:14:17.801615+0000","last_undegraded":"2024-04-25T10:14:17.801615+0000","last_fullsized":"2024-04-25T10:14:17.801615+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:42.517601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.829110+0000","last_change":"2024-04-25T10:14:16.820291+0000","last_active":"2024-04-25T10:14:16.829110+0000","last_peered":"2024-04-25T10:14:16.829110+0000","last_clean":"2024-04-25T10:14:16.829110+0000","last_became_active":"2024-04-25T10:14:16.819514+0000","last_became_peered":"2024-04-25T10:14:16.819514+0000","last_unstale":"2024-04-25T10:14:16.829110+0000","last_undegraded":"2024-04-25T10:14:16.829110+0000","last_fullsized":"2024-04-25T10:14:16.829110+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.666850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"96'8","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.862810+0000","last_change":"2024-04-25T10:14:15.803375+0000","last_active":"2024-04-25T10:14:16.862810+0000","last_peered":"2024-04-25T10:14:16.862810+0000","last_clean":"2024-04-25T10:14:16.862810+0000","last_became_active":"2024-04-25T10:14:15.802900+0000","last_became_peered":"2024-04-25T10:14:15.802900+0000","last_unstale":"2024-04-25T10:14:16.862810+0000","last_undegraded":"2024-04-25T10:14:16.862810+0000","last_fullsized":"2024-04-25T10:14:16.862810+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:17.802904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.835484+0000","last_change":"2024-04-25T10:14:16.831758+0000","last_active":"2024-04-25T10:14:16.835484+0000","last_peered":"2024-04-25T10:14:16.835484+0000","last_clean":"2024-04-25T10:14:16.835484+0000","last_became_active":"2024-04-25T10:14:16.830919+0000","last_became_peered":"2024-04-25T10:14:16.830919+0000","last_unstale":"2024-04-25T10:14:16.835484+0000","last_undegraded":"2024-04-25T10:14:16.835484+0000","last_fullsized":"2024-04-25T10:14:16.835484+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:26.181804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.803550+0000","last_change":"2024-04-25T10:14:15.803968+0000","last_active":"2024-04-25T10:14:17.803550+0000","last_peered":"2024-04-25T10:14:17.803550+0000","last_clean":"2024-04-25T10:14:17.803550+0000","last_became_active":"2024-04-25T10:14:15.803251+0000","last_became_peered":"2024-04-25T10:14:15.803251+0000","last_unstale":"2024-04-25T10:14:17.803550+0000","last_undegraded":"2024-04-25T10:14:17.803550+0000","last_fullsized":"2024-04-25T10:14:17.803550+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:24.192595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.834271+0000","last_change":"2024-04-25T10:14:16.817653+0000","last_active":"2024-04-25T10:14:16.834271+0000","last_peered":"2024-04-25T10:14:16.834271+0000","last_clean":"2024-04-25T10:14:16.834271+0000","last_became_active":"2024-04-25T10:14:16.816917+0000","last_became_peered":"2024-04-25T10:14:16.816917+0000","last_unstale":"2024-04-25T10:14:16.834271+0000","last_undegraded":"2024-04-25T10:14:16.834271+0000","last_fullsized":"2024-04-25T10:14:16.834271+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:22.150303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"96'1","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.868034+0000","last_change":"2024-04-25T10:14:15.811035+0000","last_active":"2024-04-25T10:14:16.868034+0000","last_peered":"2024-04-25T10:14:16.868034+0000","last_clean":"2024-04-25T10:14:16.868034+0000","last_became_active":"2024-04-25T10:14:15.810333+0000","last_became_peered":"2024-04-25T10:14:15.810333+0000","last_unstale":"2024-04-25T10:14:16.868034+0000","last_undegraded":"2024-04-25T10:14:16.868034+0000","last_fullsized":"2024-04-25T10:14:16.868034+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:24.467696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.064733+0000","last_change":"2024-04-25T10:14:16.839536+0000","last_active":"2024-04-25T10:14:18.064733+0000","last_peered":"2024-04-25T10:14:18.064733+0000","last_clean":"2024-04-25T10:14:18.064733+0000","last_became_active":"2024-04-25T10:14:16.837753+0000","last_became_peered":"2024-04-25T10:14:16.837753+0000","last_unstale":"2024-04-25T10:14:18.064733+0000","last_undegraded":"2024-04-25T10:14:18.064733+0000","last_fullsized":"2024-04-25T10:14:18.064733+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:13.267284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.810386+0000","last_change":"2024-04-25T10:14:15.810532+0000","last_active":"2024-04-25T10:14:16.810386+0000","last_peered":"2024-04-25T10:14:16.810386+0000","last_clean":"2024-04-25T10:14:16.810386+0000","last_became_active":"2024-04-25T10:14:15.810236+0000","last_became_peered":"2024-04-25T10:14:15.810236+0000","last_unstale":"2024-04-25T10:14:16.810386+0000","last_undegraded":"2024-04-25T10:14:16.810386+0000","last_fullsized":"2024-04-25T10:14:16.810386+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:04.437078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.845612+0000","last_change":"2024-04-25T10:14:16.841170+0000","last_active":"2024-04-25T10:14:16.845612+0000","last_peered":"2024-04-25T10:14:16.845612+0000","last_clean":"2024-04-25T10:14:16.845612+0000","last_became_active":"2024-04-25T10:14:16.840499+0000","last_became_peered":"2024-04-25T10:14:16.840499+0000","last_unstale":"2024-04-25T10:14:16.845612+0000","last_undegraded":"2024-04-25T10:14:16.845612+0000","last_fullsized":"2024-04-25T10:14:16.845612+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:45.631223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"96'2","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.859697+0000","last_change":"2024-04-25T10:14:15.804460+0000","last_active":"2024-04-25T10:14:16.859697+0000","last_peered":"2024-04-25T10:14:16.859697+0000","last_clean":"2024-04-25T10:14:16.859697+0000","last_became_active":"2024-04-25T10:14:15.803915+0000","last_became_peered":"2024-04-25T10:14:15.803915+0000","last_unstale":"2024-04-25T10:14:16.859697+0000","last_undegraded":"2024-04-25T10:14:16.859697+0000","last_fullsized":"2024-04-25T10:14:16.859697+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:27.671299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.828875+0000","last_change":"2024-04-25T10:14:16.824226+0000","last_active":"2024-04-25T10:14:16.828875+0000","last_peered":"2024-04-25T10:14:16.828875+0000","last_clean":"2024-04-25T10:14:16.828875+0000","last_became_active":"2024-04-25T10:14:16.823343+0000","last_became_peered":"2024-04-25T10:14:16.823343+0000","last_unstale":"2024-04-25T10:14:16.828875+0000","last_undegraded":"2024-04-25T10:14:16.828875+0000","last_fullsized":"2024-04-25T10:14:16.828875+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:56.579231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.800661+0000","last_change":"2024-04-25T10:14:15.795520+0000","last_active":"2024-04-25T10:14:16.800661+0000","last_peered":"2024-04-25T10:14:16.800661+0000","last_clean":"2024-04-25T10:14:16.800661+0000","last_became_active":"2024-04-25T10:14:15.794156+0000","last_became_peered":"2024-04-25T10:14:15.794156+0000","last_unstale":"2024-04-25T10:14:16.800661+0000","last_undegraded":"2024-04-25T10:14:16.800661+0000","last_fullsized":"2024-04-25T10:14:16.800661+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:44.783969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.844001+0000","last_change":"2024-04-25T10:14:16.840252+0000","last_active":"2024-04-25T10:14:16.844001+0000","last_peered":"2024-04-25T10:14:16.844001+0000","last_clean":"2024-04-25T10:14:16.844001+0000","last_became_active":"2024-04-25T10:14:16.839437+0000","last_became_peered":"2024-04-25T10:14:16.839437+0000","last_unstale":"2024-04-25T10:14:16.844001+0000","last_undegraded":"2024-04-25T10:14:16.844001+0000","last_fullsized":"2024-04-25T10:14:16.844001+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:50.764059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.799654+0000","last_change":"2024-04-25T10:14:15.805543+0000","last_active":"2024-04-25T10:14:16.799654+0000","last_peered":"2024-04-25T10:14:16.799654+0000","last_clean":"2024-04-25T10:14:16.799654+0000","last_became_active":"2024-04-25T10:14:15.804838+0000","last_became_peered":"2024-04-25T10:14:15.804838+0000","last_unstale":"2024-04-25T10:14:16.799654+0000","last_undegraded":"2024-04-25T10:14:16.799654+0000","last_fullsized":"2024-04-25T10:14:16.799654+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:11.333769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.829223+0000","last_change":"2024-04-25T10:14:16.822757+0000","last_active":"2024-04-25T10:14:16.829223+0000","last_peered":"2024-04-25T10:14:16.829223+0000","last_clean":"2024-04-25T10:14:16.829223+0000","last_became_active":"2024-04-25T10:14:16.822019+0000","last_became_peered":"2024-04-25T10:14:16.822019+0000","last_unstale":"2024-04-25T10:14:16.829223+0000","last_undegraded":"2024-04-25T10:14:16.829223+0000","last_fullsized":"2024-04-25T10:14:16.829223+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:52.450601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.810395+0000","last_change":"2024-04-25T10:14:15.788583+0000","last_active":"2024-04-25T10:14:16.810395+0000","last_peered":"2024-04-25T10:14:16.810395+0000","last_clean":"2024-04-25T10:14:16.810395+0000","last_became_active":"2024-04-25T10:14:15.788230+0000","last_became_peered":"2024-04-25T10:14:15.788230+0000","last_unstale":"2024-04-25T10:14:16.810395+0000","last_undegraded":"2024-04-25T10:14:16.810395+0000","last_fullsized":"2024-04-25T10:14:16.810395+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:33.340986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.842353+0000","last_change":"2024-04-25T10:14:16.839074+0000","last_active":"2024-04-25T10:14:16.842353+0000","last_peered":"2024-04-25T10:14:16.842353+0000","last_clean":"2024-04-25T10:14:16.842353+0000","last_became_active":"2024-04-25T10:14:16.837444+0000","last_became_peered":"2024-04-25T10:14:16.837444+0000","last_unstale":"2024-04-25T10:14:16.842353+0000","last_undegraded":"2024-04-25T10:14:16.842353+0000","last_fullsized":"2024-04-25T10:14:16.842353+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:03.178522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"96'2","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.869329+0000","last_change":"2024-04-25T10:14:15.812784+0000","last_active":"2024-04-25T10:14:16.869329+0000","last_peered":"2024-04-25T10:14:16.869329+0000","last_clean":"2024-04-25T10:14:16.869329+0000","last_became_active":"2024-04-25T10:14:15.812062+0000","last_became_peered":"2024-04-25T10:14:15.812062+0000","last_unstale":"2024-04-25T10:14:16.869329+0000","last_undegraded":"2024-04-25T10:14:16.869329+0000","last_fullsized":"2024-04-25T10:14:16.869329+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:37.484119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.844763+0000","last_change":"2024-04-25T10:14:16.833231+0000","last_active":"2024-04-25T10:14:16.844763+0000","last_peered":"2024-04-25T10:14:16.844763+0000","last_clean":"2024-04-25T10:14:16.844763+0000","last_became_active":"2024-04-25T10:14:16.832464+0000","last_became_peered":"2024-04-25T10:14:16.832464+0000","last_unstale":"2024-04-25T10:14:16.844763+0000","last_undegraded":"2024-04-25T10:14:16.844763+0000","last_fullsized":"2024-04-25T10:14:16.844763+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:21.298499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.812079+0000","last_change":"2024-04-25T10:14:15.800905+0000","last_active":"2024-04-25T10:14:16.812079+0000","last_peered":"2024-04-25T10:14:16.812079+0000","last_clean":"2024-04-25T10:14:16.812079+0000","last_became_active":"2024-04-25T10:14:15.800508+0000","last_became_peered":"2024-04-25T10:14:15.800508+0000","last_unstale":"2024-04-25T10:14:16.812079+0000","last_undegraded":"2024-04-25T10:14:16.812079+0000","last_fullsized":"2024-04-25T10:14:16.812079+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:32.934586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.844513+0000","last_change":"2024-04-25T10:14:16.825900+0000","last_active":"2024-04-25T10:14:16.844513+0000","last_peered":"2024-04-25T10:14:16.844513+0000","last_clean":"2024-04-25T10:14:16.844513+0000","last_became_active":"2024-04-25T10:14:16.825191+0000","last_became_peered":"2024-04-25T10:14:16.825191+0000","last_unstale":"2024-04-25T10:14:16.844513+0000","last_undegraded":"2024-04-25T10:14:16.844513+0000","last_fullsized":"2024-04-25T10:14:16.844513+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:19.184363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.810272+0000","last_change":"2024-04-25T10:14:15.807382+0000","last_active":"2024-04-25T10:14:16.810272+0000","last_peered":"2024-04-25T10:14:16.810272+0000","last_clean":"2024-04-25T10:14:16.810272+0000","last_became_active":"2024-04-25T10:14:15.807067+0000","last_became_peered":"2024-04-25T10:14:15.807067+0000","last_unstale":"2024-04-25T10:14:16.810272+0000","last_undegraded":"2024-04-25T10:14:16.810272+0000","last_fullsized":"2024-04-25T10:14:16.810272+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:10.230974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.848080+0000","last_change":"2024-04-25T10:14:16.845116+0000","last_active":"2024-04-25T10:14:16.848080+0000","last_peered":"2024-04-25T10:14:16.848080+0000","last_clean":"2024-04-25T10:14:16.848080+0000","last_became_active":"2024-04-25T10:14:16.844276+0000","last_became_peered":"2024-04-25T10:14:16.844276+0000","last_unstale":"2024-04-25T10:14:16.848080+0000","last_undegraded":"2024-04-25T10:14:16.848080+0000","last_fullsized":"2024-04-25T10:14:16.848080+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:28.681380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.815637+0000","last_change":"2024-04-25T10:14:15.809526+0000","last_active":"2024-04-25T10:14:16.815637+0000","last_peered":"2024-04-25T10:14:16.815637+0000","last_clean":"2024-04-25T10:14:16.815637+0000","last_became_active":"2024-04-25T10:14:15.808949+0000","last_became_peered":"2024-04-25T10:14:15.808949+0000","last_unstale":"2024-04-25T10:14:16.815637+0000","last_undegraded":"2024-04-25T10:14:16.815637+0000","last_fullsized":"2024-04-25T10:14:16.815637+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:21.140221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.829367+0000","last_change":"2024-04-25T10:14:16.820141+0000","last_active":"2024-04-25T10:14:16.829367+0000","last_peered":"2024-04-25T10:14:16.829367+0000","last_clean":"2024-04-25T10:14:16.829367+0000","last_became_active":"2024-04-25T10:14:16.819364+0000","last_became_peered":"2024-04-25T10:14:16.819364+0000","last_unstale":"2024-04-25T10:14:16.829367+0000","last_undegraded":"2024-04-25T10:14:16.829367+0000","last_fullsized":"2024-04-25T10:14:16.829367+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:32.622159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.810135+0000","last_change":"2024-04-25T10:14:15.799281+0000","last_active":"2024-04-25T10:14:16.810135+0000","last_peered":"2024-04-25T10:14:16.810135+0000","last_clean":"2024-04-25T10:14:16.810135+0000","last_became_active":"2024-04-25T10:14:15.798941+0000","last_became_peered":"2024-04-25T10:14:15.798941+0000","last_unstale":"2024-04-25T10:14:16.810135+0000","last_undegraded":"2024-04-25T10:14:16.810135+0000","last_fullsized":"2024-04-25T10:14:16.810135+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:51.526625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:17.802453+0000","last_change":"2024-04-25T10:14:16.833945+0000","last_active":"2024-04-25T10:14:17.802453+0000","last_peered":"2024-04-25T10:14:17.802453+0000","last_clean":"2024-04-25T10:14:17.802453+0000","last_became_active":"2024-04-25T10:14:16.830624+0000","last_became_peered":"2024-04-25T10:14:16.830624+0000","last_unstale":"2024-04-25T10:14:17.802453+0000","last_undegraded":"2024-04-25T10:14:17.802453+0000","last_fullsized":"2024-04-25T10:14:17.802453+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:22.496567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.814185+0000","last_change":"2024-04-25T10:14:15.802215+0000","last_active":"2024-04-25T10:14:16.814185+0000","last_peered":"2024-04-25T10:14:16.814185+0000","last_clean":"2024-04-25T10:14:16.814185+0000","last_became_active":"2024-04-25T10:14:15.801581+0000","last_became_peered":"2024-04-25T10:14:15.801581+0000","last_unstale":"2024-04-25T10:14:16.814185+0000","last_undegraded":"2024-04-25T10:14:16.814185+0000","last_fullsized":"2024-04-25T10:14:16.814185+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:44.783969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.828928+0000","last_change":"2024-04-25T10:14:16.820230+0000","last_active":"2024-04-25T10:14:16.828928+0000","last_peered":"2024-04-25T10:14:16.828928+0000","last_clean":"2024-04-25T10:14:16.828928+0000","last_became_active":"2024-04-25T10:14:16.819448+0000","last_became_peered":"2024-04-25T10:14:16.819448+0000","last_unstale":"2024-04-25T10:14:16.828928+0000","last_undegraded":"2024-04-25T10:14:16.828928+0000","last_fullsized":"2024-04-25T10:14:16.828928+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:33.542463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.827993+0000","last_change":"2024-04-25T10:14:15.795044+0000","last_active":"2024-04-25T10:14:16.827993+0000","last_peered":"2024-04-25T10:14:16.827993+0000","last_clean":"2024-04-25T10:14:16.827993+0000","last_became_active":"2024-04-25T10:14:15.794668+0000","last_became_peered":"2024-04-25T10:14:15.794668+0000","last_unstale":"2024-04-25T10:14:16.827993+0000","last_undegraded":"2024-04-25T10:14:16.827993+0000","last_fullsized":"2024-04-25T10:14:16.827993+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:14.348317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.850162+0000","last_change":"2024-04-25T10:14:16.849341+0000","last_active":"2024-04-25T10:14:16.850162+0000","last_peered":"2024-04-25T10:14:16.850162+0000","last_clean":"2024-04-25T10:14:16.850162+0000","last_became_active":"2024-04-25T10:14:16.848791+0000","last_became_peered":"2024-04-25T10:14:16.848791+0000","last_unstale":"2024-04-25T10:14:16.850162+0000","last_undegraded":"2024-04-25T10:14:16.850162+0000","last_fullsized":"2024-04-25T10:14:16.850162+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:53.990178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.794706+0000","last_change":"2024-04-25T10:14:15.796624+0000","last_active":"2024-04-25T10:14:16.794706+0000","last_peered":"2024-04-25T10:14:16.794706+0000","last_clean":"2024-04-25T10:14:16.794706+0000","last_became_active":"2024-04-25T10:14:15.796285+0000","last_became_peered":"2024-04-25T10:14:15.796285+0000","last_unstale":"2024-04-25T10:14:16.794706+0000","last_undegraded":"2024-04-25T10:14:16.794706+0000","last_fullsized":"2024-04-25T10:14:16.794706+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:52.959065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:14:16.845418+0000","last_change":"2024-04-25T10:14:16.842021+0000","last_active":"2024-04-25T10:14:16.845418+0000","last_peered":"2024-04-25T10:14:16.845418+0000","last_clean":"2024-04-25T10:14:16.845418+0000","last_became_active":"2024-04-25T10:14:16.841008+0000","last_became_peered":"2024-04-25T10:14:16.841008+0000","last_unstale":"2024-04-25T10:14:16.845418+0000","last_undegraded":"2024-04-25T10:14:16.845418+0000","last_fullsized":"2024-04-25T10:14:16.845418+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:15.763223+0000","last_clean_scrub_stamp":"2024-04-25T10:14:15.763223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:20.352191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063400+0000","last_change":"2024-04-25T10:14:15.811616+0000","last_active":"2024-04-25T10:14:18.063400+0000","last_peered":"2024-04-25T10:14:18.063400+0000","last_clean":"2024-04-25T10:14:18.063400+0000","last_became_active":"2024-04-25T10:14:15.810966+0000","last_became_peered":"2024-04-25T10:14:15.810966+0000","last_unstale":"2024-04-25T10:14:18.063400+0000","last_undegraded":"2024-04-25T10:14:18.063400+0000","last_fullsized":"2024-04-25T10:14:18.063400+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:14.753142+0000","last_clean_scrub_stamp":"2024-04-25T10:14:14.753142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:55.209660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":309,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:14:18.063458+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:14:18.063458+0000","last_peered":"2024-04-25T10:14:18.063458+0000","last_clean":"2024-04-25T10:14:18.063458+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:14:18.063458+0000","last_undegraded":"2024-04-25T10:14:18.063458+0000","last_fullsized":"2024-04-25T10:14:18.063458+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509554,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32928,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1781760,"data_stored":1097682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509553,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1212416,"data_stored":515238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542256,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1167360,"data_stored":500427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542257,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1183744,"data_stored":507272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542257,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36508,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335332,"statfs":{"total":96636764160,"available":96599379968,"internally_reserved":0,"allocated":1196032,"data_stored":508836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542257,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36488,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335352,"statfs":{"total":96636764160,"available":96599400448,"internally_reserved":0,"allocated":1167360,"data_stored":500427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542257,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36484,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335356,"statfs":{"total":96636764160,"available":96599404544,"internally_reserved":0,"allocated":1167360,"data_stored":500427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542258,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37084,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334756,"statfs":{"total":96636764160,"available":96598790144,"internally_reserved":0,"allocated":1777664,"data_stored":1098152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:14:21.295 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:14:21.295 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:14:21.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.467+0000 7f3d23bc9640 1 Processor -- start 2024-04-25T10:14:21.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.467+0000 7f3d23bc9640 1 -- start start 2024-04-25T10:14:21.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.467+0000 7f3d23bc9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3d1c071320 0x7f3d1c071700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.467+0000 7f3d23bc9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d1c0cc130 0x7f3d1c0cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.467+0000 7f3d23bc9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1c06e980 con 0x7f3d1c06d380 2024-04-25T10:14:21.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.467+0000 7f3d23bc9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1c0a8630 con 0x7f3d1c071320 2024-04-25T10:14:21.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.467+0000 7f3d23bc9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c0ccad0 con 0x7f3d1c0cc130 2024-04-25T10:14:21.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.467+0000 7f3d21965640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3d1c071320 0x7f3d1c071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.467+0000 7f3d21965640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3d1c071320 0x7f3d1c071700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:36896/0 (socket says 172.21.15.60:36896) 2024-04-25T10:14:21.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.467+0000 7f3d21965640 1 -- 172.21.15.60:0/2355970836 learned_addr learned my addr 172.21.15.60:0/2355970836 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:21.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.467+0000 7f3d21164640 1 --2- 172.21.15.60:0/2355970836 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d1c0cc130 0x7f3d1c0cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.467+0000 7f3d21965640 1 -- 172.21.15.60:0/2355970836 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d1c0cc130 msgr2=0x7f3d1c0cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.467+0000 7f3d21965640 1 --2- 172.21.15.60:0/2355970836 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d1c0cc130 0x7f3d1c0cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.467+0000 7f3d21965640 1 -- 172.21.15.60:0/2355970836 >> v1:172.21.15.100:6789/0 conn(0x7f3d1c06d380 legacy=0x7f3d1c06d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.467+0000 7f3d21965640 1 -- 172.21.15.60:0/2355970836 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1c071c40 con 0x7f3d1c071320 2024-04-25T10:14:21.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.471+0000 7f3d21965640 1 --2- 172.21.15.60:0/2355970836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c071320 0x7f3d1c071700 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009870 tx=0x7f3d1801ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f449576e24fdd98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.471+0000 7f3d20963640 1 -- 172.21.15.60:0/2355970836 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d180b5070 con 0x7f3d1c071320 2024-04-25T10:14:21.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.471+0000 7f3d20963640 1 -- 172.21.15.60:0/2355970836 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d180a8a50 con 0x7f3d1c071320 2024-04-25T10:14:21.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.471+0000 7f3d20963640 1 -- 172.21.15.60:0/2355970836 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d180a8e20 con 0x7f3d1c071320 2024-04-25T10:14:21.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.471+0000 7f3d23bc9640 1 -- 172.21.15.60:0/2355970836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c071320 msgr2=0x7f3d1c071700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.471+0000 7f3d23bc9640 1 --2- 172.21.15.60:0/2355970836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c071320 0x7f3d1c071700 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009870 tx=0x7f3d1801ed50 comp rx=0 tx=0).stop 2024-04-25T10:14:21.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.471+0000 7f3d23bc9640 1 -- 172.21.15.60:0/2355970836 shutdown_connections 2024-04-25T10:14:21.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.471+0000 7f3d23bc9640 1 --2- 172.21.15.60:0/2355970836 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d1c0cc130 0x7f3d1c0cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.471+0000 7f3d23bc9640 1 --2- 172.21.15.60:0/2355970836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c071320 0x7f3d1c071700 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.471+0000 7f3d23bc9640 1 -- 172.21.15.60:0/2355970836 >> 172.21.15.60:0/2355970836 conn(0x7f3d1c067060 msgr2=0x7f3d1c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.471+0000 7f3d23bc9640 1 -- 172.21.15.60:0/2355970836 shutdown_connections 2024-04-25T10:14:21.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.471+0000 7f3d23bc9640 1 -- 172.21.15.60:0/2355970836 wait complete. 2024-04-25T10:14:21.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.471+0000 7f3d23bc9640 1 Processor -- start 2024-04-25T10:14:21.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.471+0000 7f3d23bc9640 1 -- start start 2024-04-25T10:14:21.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d23bc9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d1c06d380 0x7f3d1c13b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d23bc9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c071320 0x7f3d1c138890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d23bc9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d1c0cc130 0x7f3d1c0acbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d23bc9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c0ccd10 con 0x7f3d1c06d380 2024-04-25T10:14:21.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d23bc9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c15c5c0 con 0x7f3d1c071320 2024-04-25T10:14:21.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d23bc9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c15bf70 con 0x7f3d1c0cc130 2024-04-25T10:14:21.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d21965640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d1c06d380 0x7f3d1c13b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d21965640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d1c06d380 0x7f3d1c13b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47474/0 (socket says 172.21.15.60:47474) 2024-04-25T10:14:21.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d21965640 1 -- 172.21.15.60:0/229074540 learned_addr learned my addr 172.21.15.60:0/229074540 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:21.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d21164640 1 --2- 172.21.15.60:0/229074540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c071320 0x7f3d1c138890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d22166640 1 --2- 172.21.15.60:0/229074540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d1c0cc130 0x7f3d1c0acbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d21965640 1 -- 172.21.15.60:0/229074540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d1c0cc130 msgr2=0x7f3d1c0acbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d21965640 1 --2- 172.21.15.60:0/229074540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d1c0cc130 0x7f3d1c0acbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d21965640 1 -- 172.21.15.60:0/229074540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c071320 msgr2=0x7f3d1c138890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d21965640 1 --2- 172.21.15.60:0/229074540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c071320 0x7f3d1c138890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d21965640 1 -- 172.21.15.60:0/229074540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1c0ad2b0 con 0x7f3d1c06d380 2024-04-25T10:14:21.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d21164640 1 --2- 172.21.15.60:0/229074540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c071320 0x7f3d1c138890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:21.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d22166640 1 --2- 172.21.15.60:0/229074540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d1c0cc130 0x7f3d1c0acbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:21.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d21965640 1 --2- 172.21.15.60:0/229074540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d1c06d380 0x7f3d1c13b9b0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f3d1801c980 tx=0x7f3d1801baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d0e7fc640 1 -- 172.21.15.60:0/229074540 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d180b5040 con 0x7f3d1c06d380 2024-04-25T10:14:21.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d0e7fc640 1 -- 172.21.15.60:0/229074540 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d180bf070 con 0x7f3d1c06d380 2024-04-25T10:14:21.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d23bc9640 1 -- 172.21.15.60:0/229074540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1c0ad590 con 0x7f3d1c06d380 2024-04-25T10:14:21.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d0e7fc640 1 -- 172.21.15.60:0/229074540 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d180b0d90 con 0x7f3d1c06d380 2024-04-25T10:14:21.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d23bc9640 1 -- 172.21.15.60:0/229074540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1c0adb50 con 0x7f3d1c06d380 2024-04-25T10:14:21.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d23bc9640 1 -- 172.21.15.60:0/229074540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1c074770 con 0x7f3d1c06d380 2024-04-25T10:14:21.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d0e7fc640 1 -- 172.21.15.60:0/229074540 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d180c4460 con 0x7f3d1c06d380 2024-04-25T10:14:21.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d0e7fc640 1 --2- 172.21.15.60:0/229074540 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3d00066f90 0x7f3d00069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d0e7fc640 1 -- 172.21.15.60:0/229074540 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d18122840 con 0x7f3d1c06d380 2024-04-25T10:14:21.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.475+0000 7f3d21164640 1 --2- 172.21.15.60:0/229074540 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3d00066f90 0x7f3d00069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.479+0000 7f3d21164640 1 --2- 172.21.15.60:0/229074540 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3d00066f90 0x7f3d00069450 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0ae800 tx=0x7f3d10008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.479+0000 7f3d0e7fc640 1 -- 172.21.15.60:0/229074540 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d180188b0 con 0x7f3d1c06d380 2024-04-25T10:14:21.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.699+0000 7f3d23bc9640 1 -- 172.21.15.60:0/229074540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3d1c0cc760 con 0x7f3d1c06d380 2024-04-25T10:14:21.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.699+0000 7f3d0e7fc640 1 -- 172.21.15.60:0/229074540 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3d180f3850 con 0x7f3d1c06d380 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCiKipmv65dLxAA6/QpPrqE9aT+CtZqkyutog== 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-04-25T10:14:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.703+0000 7f3d23bc9640 1 -- 172.21.15.60:0/229074540 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3d00066f90 msgr2=0x7f3d00069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.703+0000 7f3d23bc9640 1 --2- 172.21.15.60:0/229074540 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3d00066f90 0x7f3d00069450 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0ae800 tx=0x7f3d10008040 comp rx=0 tx=0).stop 2024-04-25T10:14:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.703+0000 7f3d23bc9640 1 -- 172.21.15.60:0/229074540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d1c06d380 msgr2=0x7f3d1c13b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.703+0000 7f3d23bc9640 1 --2- 172.21.15.60:0/229074540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d1c06d380 0x7f3d1c13b9b0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f3d1801c980 tx=0x7f3d1801baf0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.703+0000 7f3d23bc9640 1 -- 172.21.15.60:0/229074540 shutdown_connections 2024-04-25T10:14:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.703+0000 7f3d23bc9640 1 --2- 172.21.15.60:0/229074540 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3d00066f90 0x7f3d00069450 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.703+0000 7f3d23bc9640 1 --2- 172.21.15.60:0/229074540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d1c0cc130 0x7f3d1c0acbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.703+0000 7f3d23bc9640 1 --2- 172.21.15.60:0/229074540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d1c071320 0x7f3d1c138890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.703+0000 7f3d23bc9640 1 --2- 172.21.15.60:0/229074540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d1c06d380 0x7f3d1c13b9b0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.703+0000 7f3d23bc9640 1 -- 172.21.15.60:0/229074540 >> 172.21.15.60:0/229074540 conn(0x7f3d1c067060 msgr2=0x7f3d1c0af3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.703+0000 7f3d23bc9640 1 -- 172.21.15.60:0/229074540 shutdown_connections 2024-04-25T10:14:21.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.703+0000 7f3d23bc9640 1 -- 172.21.15.60:0/229074540 wait complete. 2024-04-25T10:14:21.725 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:21.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.839+0000 7fd457df0640 1 Processor -- start 2024-04-25T10:14:21.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.839+0000 7fd457df0640 1 -- start start 2024-04-25T10:14:21.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd457df0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd45015bc90 0x7fd45015c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd457df0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4500b4660 0x7fd45015c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd457df0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd450071ca0 con 0x7fd4500ce7a0 2024-04-25T10:14:21.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd457df0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4500cc370 con 0x7fd4500b4660 2024-04-25T10:14:21.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd457df0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4500cca40 con 0x7fd45015bc90 2024-04-25T10:14:21.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd45638d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4500b4660 0x7fd45015c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd455b8c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd4500ce7a0 0x7fd4500ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55500/0 (socket says 172.21.15.60:55500) 2024-04-25T10:14:21.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd455b8c640 1 -- 172.21.15.60:0/3805461050 learned_addr learned my addr 172.21.15.60:0/3805461050 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:21.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd45538b640 1 --2- 172.21.15.60:0/3805461050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd45015bc90 0x7fd45015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd454b8a640 1 -- 172.21.15.60:0/3805461050 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070707650 0 0) 0x7fd450071ca0 con 0x7fd4500ce7a0 2024-04-25T10:14:21.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd454b8a640 1 -- 172.21.15.60:0/3805461050 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd43c003530 con 0x7fd4500ce7a0 2024-04-25T10:14:21.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd45638d640 1 -- 172.21.15.60:0/3805461050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd45015bc90 msgr2=0x7fd45015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd45638d640 1 --2- 172.21.15.60:0/3805461050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd45015bc90 0x7fd45015c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd45638d640 1 -- 172.21.15.60:0/3805461050 >> v1:172.21.15.60:6789/0 conn(0x7fd4500ce7a0 legacy=0x7fd4500ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd45638d640 1 -- 172.21.15.60:0/3805461050 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4500ccd20 con 0x7fd4500b4660 2024-04-25T10:14:21.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd45638d640 1 --2- 172.21.15.60:0/3805461050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4500b4660 0x7fd45015c760 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fd448009d90 tx=0x7fd44801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7186bd5843d8c1c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd454b8a640 1 -- 172.21.15.60:0/3805461050 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4480b2070 con 0x7fd4500b4660 2024-04-25T10:14:21.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd454b8a640 1 -- 172.21.15.60:0/3805461050 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4480a5830 con 0x7fd4500b4660 2024-04-25T10:14:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.843+0000 7fd454b8a640 1 -- 172.21.15.60:0/3805461050 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4480a5be0 con 0x7fd4500b4660 2024-04-25T10:14:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 -- 172.21.15.60:0/3805461050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4500b4660 msgr2=0x7fd45015c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 --2- 172.21.15.60:0/3805461050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4500b4660 0x7fd45015c760 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fd448009d90 tx=0x7fd44801c970 comp rx=0 tx=0).stop 2024-04-25T10:14:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 -- 172.21.15.60:0/3805461050 shutdown_connections 2024-04-25T10:14:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 --2- 172.21.15.60:0/3805461050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4500b4660 0x7fd45015c760 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 --2- 172.21.15.60:0/3805461050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd45015bc90 0x7fd45015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 -- 172.21.15.60:0/3805461050 >> 172.21.15.60:0/3805461050 conn(0x7fd4500b12d0 msgr2=0x7fd4500b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 -- 172.21.15.60:0/3805461050 shutdown_connections 2024-04-25T10:14:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 -- 172.21.15.60:0/3805461050 wait complete. 2024-04-25T10:14:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 Processor -- start 2024-04-25T10:14:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 -- start start 2024-04-25T10:14:21.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4500ce7a0 0x7fd45014ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd45015bc90 0x7fd450151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4501586c0 0x7fd450153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4500b3db0 con 0x7fd4500ce7a0 2024-04-25T10:14:21.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4500b4490 con 0x7fd4501586c0 2024-04-25T10:14:21.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4501685c0 con 0x7fd45015bc90 2024-04-25T10:14:21.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd455b8c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4500ce7a0 0x7fd45014ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd455b8c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4500ce7a0 0x7fd45014ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47504/0 (socket says 172.21.15.60:47504) 2024-04-25T10:14:21.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd455b8c640 1 -- 172.21.15.60:0/1495108819 learned_addr learned my addr 172.21.15.60:0/1495108819 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:21.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd45638d640 1 --2- 172.21.15.60:0/1495108819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4501586c0 0x7fd450153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd45538b640 1 --2- 172.21.15.60:0/1495108819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd45015bc90 0x7fd450151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd455b8c640 1 -- 172.21.15.60:0/1495108819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd45015bc90 msgr2=0x7fd450151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd455b8c640 1 --2- 172.21.15.60:0/1495108819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd45015bc90 0x7fd450151cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd455b8c640 1 -- 172.21.15.60:0/1495108819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4501586c0 msgr2=0x7fd450153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd455b8c640 1 --2- 172.21.15.60:0/1495108819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4501586c0 0x7fd450153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd455b8c640 1 -- 172.21.15.60:0/1495108819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45014b660 con 0x7fd4500ce7a0 2024-04-25T10:14:21.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd45638d640 1 --2- 172.21.15.60:0/1495108819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4501586c0 0x7fd450153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:21.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd455b8c640 1 --2- 172.21.15.60:0/1495108819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4500ce7a0 0x7fd45014ab60 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00c3f0 tx=0x7fd44c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd4467fc640 1 -- 172.21.15.60:0/1495108819 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c017070 con 0x7fd4500ce7a0 2024-04-25T10:14:21.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd4467fc640 1 -- 172.21.15.60:0/1495108819 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c005cf0 con 0x7fd4500ce7a0 2024-04-25T10:14:21.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 -- 172.21.15.60:0/1495108819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd450157620 con 0x7fd4500ce7a0 2024-04-25T10:14:21.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd4467fc640 1 -- 172.21.15.60:0/1495108819 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0072f0 con 0x7fd4500ce7a0 2024-04-25T10:14:21.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.847+0000 7fd457df0640 1 -- 172.21.15.60:0/1495108819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd450157be0 con 0x7fd4500ce7a0 2024-04-25T10:14:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.851+0000 7fd457df0640 1 -- 172.21.15.60:0/1495108819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd450074770 con 0x7fd4500ce7a0 2024-04-25T10:14:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.851+0000 7fd4467fc640 1 -- 172.21.15.60:0/1495108819 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd44c01a070 con 0x7fd4500ce7a0 2024-04-25T10:14:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.851+0000 7fd4467fc640 1 --2- 172.21.15.60:0/1495108819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd434066f60 0x7fd434069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.851+0000 7fd4467fc640 1 -- 172.21.15.60:0/1495108819 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd44c084400 con 0x7fd4500ce7a0 2024-04-25T10:14:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.851+0000 7fd45538b640 1 --2- 172.21.15.60:0/1495108819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd434066f60 0x7fd434069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.851+0000 7fd45538b640 1 --2- 172.21.15.60:0/1495108819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd434066f60 0x7fd434069420 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd45014ba90 tx=0x7fd440008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:21.855+0000 7fd4467fc640 1 -- 172.21.15.60:0/1495108819 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd45014b660 con 0x7fd4500ce7a0 2024-04-25T10:14:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.051+0000 7fd457df0640 1 -- 172.21.15.60:0/1495108819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd450154a30 con 0x7fd4500ce7a0 2024-04-25T10:14:22.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.051+0000 7fd4467fc640 1 -- 172.21.15.60:0/1495108819 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd44c055410 con 0x7fd4500ce7a0 2024-04-25T10:14:22.056 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:22.056 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1491228634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1491228634},{"type":"v1","addr":"172.21.15.100:6837","nonce":1491228634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/209898872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":209898872},{"type":"v1","addr":"172.21.15.100:6835","nonce":209898872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1793343287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1793343287},{"type":"v1","addr":"172.21.15.60:6833","nonce":1793343287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:16.146481+0000","modified":"2024-04-25T10:14:19.300954+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2002755047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2002755047},{"type":"v1","addr":"172.21.15.60:6835","nonce":2002755047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:14:22.057 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 77 2024-04-25T10:14:22.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.055+0000 7fd457df0640 1 -- 172.21.15.60:0/1495108819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd434066f60 msgr2=0x7fd434069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.055+0000 7fd457df0640 1 --2- 172.21.15.60:0/1495108819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd434066f60 0x7fd434069420 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd45014ba90 tx=0x7fd440008040 comp rx=0 tx=0).stop 2024-04-25T10:14:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.055+0000 7fd457df0640 1 -- 172.21.15.60:0/1495108819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4500ce7a0 msgr2=0x7fd45014ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.055+0000 7fd457df0640 1 --2- 172.21.15.60:0/1495108819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4500ce7a0 0x7fd45014ab60 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00c3f0 tx=0x7fd44c00c8b0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.055+0000 7fd457df0640 1 -- 172.21.15.60:0/1495108819 shutdown_connections 2024-04-25T10:14:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.055+0000 7fd457df0640 1 --2- 172.21.15.60:0/1495108819 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd434066f60 0x7fd434069420 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.055+0000 7fd457df0640 1 --2- 172.21.15.60:0/1495108819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4501586c0 0x7fd450153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.055+0000 7fd457df0640 1 --2- 172.21.15.60:0/1495108819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd45015bc90 0x7fd450151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.055+0000 7fd457df0640 1 --2- 172.21.15.60:0/1495108819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4500ce7a0 0x7fd45014ab60 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.055+0000 7fd457df0640 1 -- 172.21.15.60:0/1495108819 >> 172.21.15.60:0/1495108819 conn(0x7fd4500b12d0 msgr2=0x7fd4500ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.055+0000 7fd457df0640 1 -- 172.21.15.60:0/1495108819 shutdown_connections 2024-04-25T10:14:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.055+0000 7fd457df0640 1 -- 172.21.15.60:0/1495108819 wait complete. 2024-04-25T10:14:22.478 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:14:16.146481+0000', 'modified': '2024-04-25T10:14:19.300954+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8631}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8631': {'gid': 8631, 'name': 'a', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/2002755047', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 2002755047}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 2002755047}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:22.478 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:14:22.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:14:22.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.611+0000 7fdb3f983640 1 Processor -- start 2024-04-25T10:14:22.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.611+0000 7fdb3f983640 1 -- start start 2024-04-25T10:14:22.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.611+0000 7fdb3f983640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb38099b10 0x7fdb3806c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.611+0000 7fdb3f983640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdb38099ef0 0x7fdb381617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.611+0000 7fdb3f983640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb3815f5a0 con 0x7fdb38161ce0 2024-04-25T10:14:22.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.611+0000 7fdb3f983640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3815ea60 con 0x7fdb38099b10 2024-04-25T10:14:22.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.611+0000 7fdb3f983640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3815ed40 con 0x7fdb38099ef0 2024-04-25T10:14:22.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.611+0000 7fdb3d71f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb38099b10 0x7fdb3806c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.611+0000 7fdb3d71f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb38099b10 0x7fdb3806c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47508/0 (socket says 172.21.15.60:47508) 2024-04-25T10:14:22.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.611+0000 7fdb3d71f640 1 -- 172.21.15.60:0/1298515529 learned_addr learned my addr 172.21.15.60:0/1298515529 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:22.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.611+0000 7fdb3cf1e640 1 --2- 172.21.15.60:0/1298515529 >> v2:172.21.15.100:3300/0 conn(0x7fdb38099ef0 0x7fdb381617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.611+0000 7fdb3d71f640 1 -- 172.21.15.60:0/1298515529 >> v2:172.21.15.100:3300/0 conn(0x7fdb38099ef0 msgr2=0x7fdb381617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.611+0000 7fdb3d71f640 1 --2- 172.21.15.60:0/1298515529 >> v2:172.21.15.100:3300/0 conn(0x7fdb38099ef0 0x7fdb381617a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.611+0000 7fdb3d71f640 1 -- 172.21.15.60:0/1298515529 >> v1:172.21.15.100:6789/0 conn(0x7fdb38161ce0 legacy=0x7fdb380d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.611+0000 7fdb3d71f640 1 -- 172.21.15.60:0/1298515529 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3815f020 con 0x7fdb38099b10 2024-04-25T10:14:22.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.615+0000 7fdb3d71f640 1 --2- 172.21.15.60:0/1298515529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb38099b10 0x7fdb3806c0e0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fdb38072c00 tx=0x7fdb3001c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e545653748de2435 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.615+0000 7fdb2ffff640 1 -- 172.21.15.60:0/1298515529 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb300a5570 con 0x7fdb38099b10 2024-04-25T10:14:22.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.615+0000 7fdb2ffff640 1 -- 172.21.15.60:0/1298515529 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb300a5710 con 0x7fdb38099b10 2024-04-25T10:14:22.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.615+0000 7fdb2ffff640 1 -- 172.21.15.60:0/1298515529 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb300a5ae0 con 0x7fdb38099b10 2024-04-25T10:14:22.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.615+0000 7fdb3f983640 1 -- 172.21.15.60:0/1298515529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb38099b10 msgr2=0x7fdb3806c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.615+0000 7fdb3f983640 1 --2- 172.21.15.60:0/1298515529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb38099b10 0x7fdb3806c0e0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fdb38072c00 tx=0x7fdb3001c710 comp rx=0 tx=0).stop 2024-04-25T10:14:22.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.615+0000 7fdb3f983640 1 -- 172.21.15.60:0/1298515529 shutdown_connections 2024-04-25T10:14:22.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.615+0000 7fdb3f983640 1 --2- 172.21.15.60:0/1298515529 >> v2:172.21.15.100:3300/0 conn(0x7fdb38099ef0 0x7fdb381617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.615+0000 7fdb3f983640 1 --2- 172.21.15.60:0/1298515529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb38099b10 0x7fdb3806c0e0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.615+0000 7fdb3f983640 1 -- 172.21.15.60:0/1298515529 >> 172.21.15.60:0/1298515529 conn(0x7fdb380b3320 msgr2=0x7fdb380b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.615+0000 7fdb3f983640 1 -- 172.21.15.60:0/1298515529 shutdown_connections 2024-04-25T10:14:22.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.615+0000 7fdb3f983640 1 -- 172.21.15.60:0/1298515529 wait complete. 2024-04-25T10:14:22.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.615+0000 7fdb3f983640 1 Processor -- start 2024-04-25T10:14:22.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.615+0000 7fdb3f983640 1 -- start start 2024-04-25T10:14:22.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3f983640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb38099b10 0x7fdb3814f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3f983640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb38099ef0 0x7fdb380c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3f983640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb38161ce0 0x7fdb380c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3f983640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3809a480 con 0x7fdb38161ce0 2024-04-25T10:14:22.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3f983640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3809a300 con 0x7fdb38099b10 2024-04-25T10:14:22.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3f983640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb38161390 con 0x7fdb38099ef0 2024-04-25T10:14:22.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3df20640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb38161ce0 0x7fdb380c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3df20640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb38161ce0 0x7fdb380c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47522/0 (socket says 172.21.15.60:47522) 2024-04-25T10:14:22.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3df20640 1 -- 172.21.15.60:0/3003219486 learned_addr learned my addr 172.21.15.60:0/3003219486 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:22.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3df20640 1 -- 172.21.15.60:0/3003219486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb38099ef0 msgr2=0x7fdb380c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3d71f640 1 --2- 172.21.15.60:0/3003219486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb38099b10 0x7fdb3814f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3cf1e640 1 --2- 172.21.15.60:0/3003219486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb38099ef0 0x7fdb380c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3df20640 1 --2- 172.21.15.60:0/3003219486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb38099ef0 0x7fdb380c69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3df20640 1 -- 172.21.15.60:0/3003219486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb38099b10 msgr2=0x7fdb3814f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3df20640 1 --2- 172.21.15.60:0/3003219486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb38099b10 0x7fdb3814f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3df20640 1 -- 172.21.15.60:0/3003219486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb380c5360 con 0x7fdb38161ce0 2024-04-25T10:14:22.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3d71f640 1 --2- 172.21.15.60:0/3003219486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb38099b10 0x7fdb3814f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:22.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3df20640 1 --2- 172.21.15.60:0/3003219486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb38161ce0 0x7fdb380c6ee0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400b440 tx=0x7fdb3400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb2dffb640 1 -- 172.21.15.60:0/3003219486 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34015020 con 0x7fdb38161ce0 2024-04-25T10:14:22.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb2dffb640 1 -- 172.21.15.60:0/3003219486 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb34011850 con 0x7fdb38161ce0 2024-04-25T10:14:22.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3f983640 1 -- 172.21.15.60:0/3003219486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3814fc50 con 0x7fdb38161ce0 2024-04-25T10:14:22.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb2dffb640 1 -- 172.21.15.60:0/3003219486 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34011c40 con 0x7fdb38161ce0 2024-04-25T10:14:22.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb3f983640 1 -- 172.21.15.60:0/3003219486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb38150210 con 0x7fdb38161ce0 2024-04-25T10:14:22.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb2dffb640 1 -- 172.21.15.60:0/3003219486 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdb34018020 con 0x7fdb38161ce0 2024-04-25T10:14:22.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb2dffb640 1 --2- 172.21.15.60:0/3003219486 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb1c066f60 0x7fdb1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.619+0000 7fdb2dffb640 1 -- 172.21.15.60:0/3003219486 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb34084b70 con 0x7fdb38161ce0 2024-04-25T10:14:22.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb3d71f640 1 --2- 172.21.15.60:0/3003219486 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb1c066f60 0x7fdb1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 Processor -- start 2024-04-25T10:14:22.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 -- start start 2024-04-25T10:14:22.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb3d71f640 1 --2- 172.21.15.60:0/3003219486 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb1c066f60 0x7fdb1c069420 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdb300ae000 tx=0x7fdb300a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 0x5602a1aabd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1aac240 0x5602a1a619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1a62310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a19495f0 con 0x5602a1a61f30 2024-04-25T10:14:22.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a18b0da0 con 0x5602a1aab920 2024-04-25T10:14:22.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602a1839150 con 0x5602a1aac240 2024-04-25T10:14:22.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb3df20640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1a62310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb3df20640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1a62310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47532/0 (socket says 172.21.15.60:47532) 2024-04-25T10:14:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb3df20640 1 -- 172.21.15.60:0/1917738975 learned_addr learned my addr 172.21.15.60:0/1917738975 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb3cf1e640 1 --2- 172.21.15.60:0/1917738975 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1aac240 0x5602a1a619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb3d71f640 1 --2- 172.21.15.60:0/1917738975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 0x5602a1aabd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb3df20640 1 -- 172.21.15.60:0/1917738975 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1aac240 msgr2=0x5602a1a619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb3df20640 1 --2- 172.21.15.60:0/1917738975 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1aac240 0x5602a1a619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb3df20640 1 -- 172.21.15.60:0/1917738975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 msgr2=0x5602a1aabd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb3df20640 1 --2- 172.21.15.60:0/1917738975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 0x5602a1aabd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb3df20640 1 -- 172.21.15.60:0/1917738975 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602a1852ca0 con 0x5602a1a61f30 2024-04-25T10:14:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb3df20640 1 --2- 172.21.15.60:0/1917738975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1a62310 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fdb340518d0 tx=0x7fdb340595c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bca19a5f38bba24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb22ffd640 1 -- 172.21.15.60:0/1917738975 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34081070 con 0x5602a1a61f30 2024-04-25T10:14:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb22ffd640 1 -- 172.21.15.60:0/1917738975 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb34053510 con 0x5602a1a61f30 2024-04-25T10:14:22.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb22ffd640 1 -- 172.21.15.60:0/1917738975 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34020d90 con 0x5602a1a61f30 2024-04-25T10:14:22.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 -- 172.21.15.60:0/1917738975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 msgr2=0x5602a1a62310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 --2- 172.21.15.60:0/1917738975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1a62310 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fdb340518d0 tx=0x7fdb340595c0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 -- 172.21.15.60:0/1917738975 shutdown_connections 2024-04-25T10:14:22.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 --2- 172.21.15.60:0/1917738975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1a62310 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 --2- 172.21.15.60:0/1917738975 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1aac240 0x5602a1a619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 --2- 172.21.15.60:0/1917738975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 0x5602a1aabd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 -- 172.21.15.60:0/1917738975 >> 172.21.15.60:0/1917738975 conn(0x5602a1a682a0 msgr2=0x5602a1a602d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 -- 172.21.15.60:0/1917738975 shutdown_connections 2024-04-25T10:14:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 -- 172.21.15.60:0/1917738975 wait complete. 2024-04-25T10:14:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 -- start start 2024-04-25T10:14:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.623+0000 7fdb42af0000 1 Processor -- start 2024-04-25T10:14:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1ab2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 0x5602a1ab32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1aac240 0x5602a1a65aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a186b990 con 0x5602a1a61f30 2024-04-25T10:14:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a18ace40 con 0x5602a1aab920 2024-04-25T10:14:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602a183c900 con 0x5602a1aac240 2024-04-25T10:14:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 10 client.0 _ll_register_callbacks cb 0x5602a19641c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb3d71f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1ab2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb3d71f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1ab2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47538/0 (socket says 172.21.15.60:47538) 2024-04-25T10:14:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb3d71f640 1 -- 172.21.15.60:0/1673296129 learned_addr learned my addr 172.21.15.60:0/1673296129 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb3cf1e640 1 --2- 172.21.15.60:0/1673296129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 0x5602a1ab32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb3df20640 1 --2- 172.21.15.60:0/1673296129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1aac240 0x5602a1a65aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb3d71f640 1 -- 172.21.15.60:0/1673296129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1aac240 msgr2=0x5602a1a65aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb3d71f640 1 --2- 172.21.15.60:0/1673296129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1aac240 0x5602a1a65aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb3d71f640 1 -- 172.21.15.60:0/1673296129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 msgr2=0x5602a1ab32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb3d71f640 1 --2- 172.21.15.60:0/1673296129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 0x5602a1ab32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb3d71f640 1 -- 172.21.15.60:0/1673296129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602a1869630 con 0x5602a1a61f30 2024-04-25T10:14:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb3df20640 1 --2- 172.21.15.60:0/1673296129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1aac240 0x5602a1a65aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb3d71f640 1 --2- 172.21.15.60:0/1673296129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1ab2d60 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009c90 tx=0x7fdb3001af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92cf2946722d726 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb17fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:14:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb17fff640 1 -- 172.21.15.60:0/1673296129 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb300b9510 con 0x5602a1a61f30 2024-04-25T10:14:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb17fff640 1 -- 172.21.15.60:0/1673296129 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb300b96b0 con 0x5602a1a61f30 2024-04-25T10:14:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 -- 172.21.15.60:0/1673296129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb34053510 con 0x5602a1a61f30 2024-04-25T10:14:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb17fff640 1 -- 172.21.15.60:0/1673296129 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb300cf050 con 0x5602a1a61f30 2024-04-25T10:14:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 10 client.8769 fetch_fsmap learned FSMap version 77 2024-04-25T10:14:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 -- 172.21.15.60:0/1673296129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x5602a18393a0 con 0x5602a1a61f30 2024-04-25T10:14:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb17fff640 1 -- 172.21.15.60:0/1673296129 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb34053510 con 0x5602a1a61f30 2024-04-25T10:14:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb17fff640 1 -- 172.21.15.60:0/1673296129 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdb300d2020 con 0x5602a1a61f30 2024-04-25T10:14:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 10 client.8769 fetch_fsmap finished waiting for FSMap version 77 2024-04-25T10:14:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 10 client.8769 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:14:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 20 client.8769 populate_metadata read hostname 'smithi060' 2024-04-25T10:14:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 --2- 172.21.15.60:0/1673296129 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x5602a1a67160 0x5602a1abcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 4 client.8769 mds_command: new command op to 8631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 -- 172.21.15.60:0/1673296129 --> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602a184ca50 con 0x5602a1a67160 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb3cf1e640 1 --2- 172.21.15.60:0/1673296129 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x5602a1a67160 0x5602a1abcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb17fff640 10 client.8769 ms_handle_connect on v2:172.21.15.60:6834/2002755047 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb3cf1e640 1 --2- 172.21.15.60:0/1673296129 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x5602a1a67160 0x5602a1abcdc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=846db9cc6dcd6c59 server_cookie=8b81eff6515c99c0 in_seq=0 out_seq=0 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb17fff640 1 -- 172.21.15.60:0/1673296129 <== mds.0 v2:172.21.15.60:6834/2002755047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5602a184ca50 con 0x5602a1a67160 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb17fff640 10 client.8769 handle_command_reply: tid=0 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 client.8769 shutdown 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 -- 172.21.15.60:0/1673296129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 msgr2=0x5602a1ab2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 --2- 172.21.15.60:0/1673296129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1ab2d60 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009c90 tx=0x7fdb3001af50 comp rx=0 tx=0).stop 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 -- 172.21.15.60:0/1673296129 shutdown_connections 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 --2- 172.21.15.60:0/1673296129 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x5602a1a67160 0x5602a1abcdc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 --2- 172.21.15.60:0/1673296129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1aac240 0x5602a1a65aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 --2- 172.21.15.60:0/1673296129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 0x5602a1ab32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb17fff640 0 client.8769 ms_handle_reset on v2:172.21.15.60:6834/2002755047 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 --2- 172.21.15.60:0/1673296129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1ab2d60 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 -- 172.21.15.60:0/1673296129 >> 172.21.15.60:0/1673296129 conn(0x5602a1a682a0 msgr2=0x5602a1a609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 -- 172.21.15.60:0/1673296129 shutdown_connections 2024-04-25T10:14:22.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 1 -- 172.21.15.60:0/1673296129 wait complete. 2024-04-25T10:14:22.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.627+0000 7fdb42af0000 20 client.8769 trim_cache size 0 max 16384 2024-04-25T10:14:22.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.643+0000 7fdb42af0000 1 Processor -- start 2024-04-25T10:14:22.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.643+0000 7fdb42af0000 1 -- start start 2024-04-25T10:14:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.643+0000 7fdb42af0000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1a65330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.643+0000 7fdb42af0000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 0x5602a1a65870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.643+0000 7fdb42af0000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1aac240 0x5602a1ab5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.643+0000 7fdb42af0000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb382024a0 con 0x5602a1a61f30 2024-04-25T10:14:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.643+0000 7fdb42af0000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3813df20 con 0x5602a1aab920 2024-04-25T10:14:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.643+0000 7fdb42af0000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb38201ed0 con 0x5602a1aac240 2024-04-25T10:14:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.643+0000 7fdb3d71f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1a65330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.643+0000 7fdb3d71f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1a65330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47544/0 (socket says 172.21.15.60:47544) 2024-04-25T10:14:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.643+0000 7fdb3d71f640 1 -- 172.21.15.60:0/2463410609 learned_addr learned my addr 172.21.15.60:0/2463410609 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3d71f640 1 -- 172.21.15.60:0/2463410609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1aac240 msgr2=0x5602a1ab5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3d71f640 1 --2- 172.21.15.60:0/2463410609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1aac240 0x5602a1ab5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3d71f640 1 -- 172.21.15.60:0/2463410609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 msgr2=0x5602a1a65870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3cf1e640 1 --2- 172.21.15.60:0/2463410609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 0x5602a1a65870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3d71f640 1 --2- 172.21.15.60:0/2463410609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 0x5602a1a65870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3d71f640 1 -- 172.21.15.60:0/2463410609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602a1aac070 con 0x5602a1a61f30 2024-04-25T10:14:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3d71f640 1 --2- 172.21.15.60:0/2463410609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1a65330 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009dd0 tx=0x7fdb300bc070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7451487637955550 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb227fc640 1 -- 172.21.15.60:0/2463410609 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb300b9510 con 0x5602a1a61f30 2024-04-25T10:14:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb227fc640 1 -- 172.21.15.60:0/2463410609 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb300b96b0 con 0x5602a1a61f30 2024-04-25T10:14:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb227fc640 1 -- 172.21.15.60:0/2463410609 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb300b9cc0 con 0x5602a1a61f30 2024-04-25T10:14:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 -- 172.21.15.60:0/2463410609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 msgr2=0x5602a1a65330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 --2- 172.21.15.60:0/2463410609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1a65330 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009dd0 tx=0x7fdb300bc070 comp rx=0 tx=0).stop 2024-04-25T10:14:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 -- 172.21.15.60:0/2463410609 shutdown_connections 2024-04-25T10:14:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 --2- 172.21.15.60:0/2463410609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1aac240 0x5602a1ab5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 --2- 172.21.15.60:0/2463410609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 0x5602a1a65870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 --2- 172.21.15.60:0/2463410609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1a61f30 0x5602a1a65330 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 -- 172.21.15.60:0/2463410609 >> 172.21.15.60:0/2463410609 conn(0x5602a1a682a0 msgr2=0x5602a1a98170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 -- 172.21.15.60:0/2463410609 shutdown_connections 2024-04-25T10:14:22.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 -- 172.21.15.60:0/2463410609 wait complete. 2024-04-25T10:14:22.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 -- start start 2024-04-25T10:14:22.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 Processor -- start 2024-04-25T10:14:22.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1a61f30 0x5602a1ab8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 0x5602a1ab8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1aac240 0x5602a1ac6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a18b53d0 con 0x5602a1aac240 2024-04-25T10:14:22.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602a196cd50 con 0x5602a1aab920 2024-04-25T10:14:22.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602a184b980 con 0x5602a1a61f30 2024-04-25T10:14:22.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 10 client.0 _ll_register_callbacks cb 0x7fdb34055240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:22.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3df20640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1aac240 0x5602a1ac6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3df20640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1aac240 0x5602a1ac6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47546/0 (socket says 172.21.15.60:47546) 2024-04-25T10:14:22.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3df20640 1 -- 172.21.15.60:0/2090423511 learned_addr learned my addr 172.21.15.60:0/2090423511 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:22.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3cf1e640 1 --2- 172.21.15.60:0/2090423511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 0x5602a1ab8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3d71f640 1 --2- 172.21.15.60:0/2090423511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1a61f30 0x5602a1ab8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3df20640 1 -- 172.21.15.60:0/2090423511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1a61f30 msgr2=0x5602a1ab8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3df20640 1 --2- 172.21.15.60:0/2090423511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1a61f30 0x5602a1ab8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3df20640 1 -- 172.21.15.60:0/2090423511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 msgr2=0x5602a1ab8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3df20640 1 --2- 172.21.15.60:0/2090423511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 0x5602a1ab8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3df20640 1 -- 172.21.15.60:0/2090423511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602a1932600 con 0x5602a1aac240 2024-04-25T10:14:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb3df20640 1 --2- 172.21.15.60:0/2090423511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1aac240 0x5602a1ac6840 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fdb3405d650 tx=0x7fdb34082f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b885a853bc448d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb17fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:14:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb17fff640 1 -- 172.21.15.60:0/2090423511 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34081070 con 0x5602a1aac240 2024-04-25T10:14:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb17fff640 1 -- 172.21.15.60:0/2090423511 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb34051ae0 con 0x5602a1aac240 2024-04-25T10:14:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 -- 172.21.15.60:0/2090423511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb300b96b0 con 0x5602a1aac240 2024-04-25T10:14:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb17fff640 1 -- 172.21.15.60:0/2090423511 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb34059170 con 0x5602a1aac240 2024-04-25T10:14:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 10 client.8775 fetch_fsmap learned FSMap version 77 2024-04-25T10:14:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 -- 172.21.15.60:0/2090423511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x5602a18459b0 con 0x5602a1aac240 2024-04-25T10:14:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb17fff640 1 -- 172.21.15.60:0/2090423511 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb300b96b0 con 0x5602a1aac240 2024-04-25T10:14:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb17fff640 1 -- 172.21.15.60:0/2090423511 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdb340548c0 con 0x5602a1aac240 2024-04-25T10:14:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 10 client.8775 fetch_fsmap finished waiting for FSMap version 77 2024-04-25T10:14:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 10 client.8775 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:14:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 20 client.8775 populate_metadata read hostname 'smithi060' 2024-04-25T10:14:22.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 --2- 172.21.15.60:0/2090423511 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x5602a1af3160 0x5602a1ac6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 4 client.8775 mds_command: new command op to 8631 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:14:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.647+0000 7fdb42af0000 1 -- 172.21.15.60:0/2090423511 --> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5602a18be2e0 con 0x5602a1af3160 2024-04-25T10:14:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb3d71f640 1 --2- 172.21.15.60:0/2090423511 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x5602a1af3160 0x5602a1ac6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb17fff640 10 client.8775 ms_handle_connect on v2:172.21.15.60:6834/2002755047 2024-04-25T10:14:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb3d71f640 1 --2- 172.21.15.60:0/2090423511 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x5602a1af3160 0x5602a1ac6d80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbfa9a2e93596c99 server_cookie=132430a10d58c6d3 in_seq=0 out_seq=0 2024-04-25T10:14:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb17fff640 1 -- 172.21.15.60:0/2090423511 <== mds.0 v2:172.21.15.60:6834/2002755047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5602a18be2e0 con 0x5602a1af3160 2024-04-25T10:14:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb17fff640 10 client.8775 handle_command_reply: tid=0 2024-04-25T10:14:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb42af0000 1 client.8775 shutdown 2024-04-25T10:14:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb42af0000 1 -- 172.21.15.60:0/2090423511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1aac240 msgr2=0x5602a1ac6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb42af0000 1 --2- 172.21.15.60:0/2090423511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1aac240 0x5602a1ac6840 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fdb3405d650 tx=0x7fdb34082f60 comp rx=0 tx=0).stop 2024-04-25T10:14:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb42af0000 1 -- 172.21.15.60:0/2090423511 shutdown_connections 2024-04-25T10:14:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb42af0000 1 --2- 172.21.15.60:0/2090423511 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x5602a1af3160 0x5602a1ac6d80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb42af0000 1 --2- 172.21.15.60:0/2090423511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5602a1aac240 0x5602a1ac6840 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb42af0000 1 --2- 172.21.15.60:0/2090423511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5602a1aab920 0x5602a1ab8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb17fff640 0 client.8775 ms_handle_reset on v2:172.21.15.60:6834/2002755047 2024-04-25T10:14:22.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb42af0000 1 --2- 172.21.15.60:0/2090423511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5602a1a61f30 0x5602a1ab8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb42af0000 1 -- 172.21.15.60:0/2090423511 >> 172.21.15.60:0/2090423511 conn(0x5602a1a682a0 msgr2=0x5602a1a64320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb42af0000 1 -- 172.21.15.60:0/2090423511 shutdown_connections 2024-04-25T10:14:22.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb42af0000 1 -- 172.21.15.60:0/2090423511 wait complete. 2024-04-25T10:14:22.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb42af0000 20 client.8775 trim_cache size 0 max 16384 2024-04-25T10:14:22.660 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:14:22.660 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "0ea93863-a079-45c4-abed-6e83540ea9fe", 2024-04-25T10:14:22.660 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T10:14:22.660 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 8631, 2024-04-25T10:14:22.660 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T10:14:22.660 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T10:14:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T10:14:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 5.8047057300000002, 2024-04-25T10:14:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 77, 2024-04-25T10:14:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 97, 2024-04-25T10:14:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 97, 2024-04-25T10:14:22.661 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 10.871353346999999 2024-04-25T10:14:22.661 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:14:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb3f983640 1 -- 172.21.15.60:0/3003219486 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb1c066f60 msgr2=0x7fdb1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb3f983640 1 --2- 172.21.15.60:0/3003219486 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb1c066f60 0x7fdb1c069420 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdb300ae000 tx=0x7fdb300a7070 comp rx=0 tx=0).stop 2024-04-25T10:14:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb3f983640 1 -- 172.21.15.60:0/3003219486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb38161ce0 msgr2=0x7fdb380c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb3f983640 1 --2- 172.21.15.60:0/3003219486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb38161ce0 0x7fdb380c6ee0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400b440 tx=0x7fdb3400b900 comp rx=0 tx=0).stop 2024-04-25T10:14:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb3f983640 1 -- 172.21.15.60:0/3003219486 shutdown_connections 2024-04-25T10:14:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb3f983640 1 --2- 172.21.15.60:0/3003219486 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb1c066f60 0x7fdb1c069420 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb3f983640 1 --2- 172.21.15.60:0/3003219486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb38161ce0 0x7fdb380c6ee0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb3f983640 1 --2- 172.21.15.60:0/3003219486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb38099ef0 0x7fdb380c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb3f983640 1 --2- 172.21.15.60:0/3003219486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb38099b10 0x7fdb3814f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb3f983640 1 -- 172.21.15.60:0/3003219486 >> 172.21.15.60:0/3003219486 conn(0x7fdb380b3320 msgr2=0x7fdb3815dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb3f983640 1 -- 172.21.15.60:0/3003219486 shutdown_connections 2024-04-25T10:14:22.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.651+0000 7fdb3f983640 1 -- 172.21.15.60:0/3003219486 wait complete. 2024-04-25T10:14:22.671 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:14:22.768 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T10:14:22.763+0000 7fbb92112640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:14:22.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3f0368640 1 Processor -- start 2024-04-25T10:14:22.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3f0368640 1 -- start start 2024-04-25T10:14:22.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3f0368640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff3e80ce7a0 0x7ff3e80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3f0368640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3e80b4660 0x7ff3e80b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3f0368640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e80cf0c0 con 0x7ff3e815bc90 2024-04-25T10:14:22.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3f0368640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e80cc370 con 0x7ff3e80ce7a0 2024-04-25T10:14:22.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3f0368640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e8071ca0 con 0x7ff3e80b4660 2024-04-25T10:14:22.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed903640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff3e815bc90 0x7ff3e80b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55516/0 (socket says 172.21.15.60:55516) 2024-04-25T10:14:22.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed903640 1 -- 172.21.15.60:0/3107297562 learned_addr learned my addr 172.21.15.60:0/3107297562 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:22.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ee104640 1 --2- 172.21.15.60:0/3107297562 >> v2:172.21.15.100:3300/0 conn(0x7ff3e80ce7a0 0x7ff3e80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ee905640 1 --2- 172.21.15.60:0/3107297562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3e80b4660 0x7ff3e80b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed102640 1 -- 172.21.15.60:0/3107297562 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541709035 0 0) 0x7ff3e80cf0c0 con 0x7ff3e815bc90 2024-04-25T10:14:22.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed102640 1 -- 172.21.15.60:0/3107297562 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3d4003530 con 0x7ff3e815bc90 2024-04-25T10:14:22.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed102640 1 -- 172.21.15.60:0/3107297562 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3643996449 0 0) 0x7ff3d4003530 con 0x7ff3e815bc90 2024-04-25T10:14:22.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed102640 1 -- 172.21.15.60:0/3107297562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3e80b4660 msgr2=0x7ff3e80b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed102640 1 --2- 172.21.15.60:0/3107297562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3e80b4660 0x7ff3e80b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed102640 1 -- 172.21.15.60:0/3107297562 >> v2:172.21.15.100:3300/0 conn(0x7ff3e80ce7a0 msgr2=0x7ff3e80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed102640 1 --2- 172.21.15.60:0/3107297562 >> v2:172.21.15.100:3300/0 conn(0x7ff3e80ce7a0 0x7ff3e80ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed102640 1 -- 172.21.15.60:0/3107297562 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e80cca40 con 0x7ff3e815bc90 2024-04-25T10:14:22.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed102640 1 -- 172.21.15.60:0/3107297562 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7ff3d8001680 con 0x7ff3e815bc90 2024-04-25T10:14:22.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed102640 1 -- 172.21.15.60:0/3107297562 >> v1:172.21.15.60:6789/0 conn(0x7ff3e815bc90 legacy=0x7ff3e80b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed102640 1 --2- 172.21.15.60:0/3107297562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3d40039c0 0x7ff3d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed102640 1 --2- 172.21.15.60:0/3107297562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3d4004810 0x7ff3d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed102640 1 --2- 172.21.15.60:0/3107297562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3d40051d0 0x7ff3d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed102640 1 -- 172.21.15.60:0/3107297562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e8071ca0 con 0x7ff3d40039c0 2024-04-25T10:14:22.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed102640 1 -- 172.21.15.60:0/3107297562 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d4004530 con 0x7ff3d4004810 2024-04-25T10:14:22.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed102640 1 -- 172.21.15.60:0/3107297562 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d400bf10 con 0x7ff3d40051d0 2024-04-25T10:14:22.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ee104640 1 --2- 172.21.15.60:0/3107297562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3d40039c0 0x7ff3d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ed903640 1 --2- 172.21.15.60:0/3107297562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3d4004810 0x7ff3d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ee905640 1 --2- 172.21.15.60:0/3107297562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3d40051d0 0x7ff3d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ee104640 1 -- 172.21.15.60:0/3107297562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3d40051d0 msgr2=0x7ff3d400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ee104640 1 --2- 172.21.15.60:0/3107297562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3d40051d0 0x7ff3d400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ee104640 1 -- 172.21.15.60:0/3107297562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3d4004810 msgr2=0x7ff3d4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ee104640 1 --2- 172.21.15.60:0/3107297562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3d4004810 0x7ff3d4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ee104640 1 -- 172.21.15.60:0/3107297562 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d4003530 con 0x7ff3d40039c0 2024-04-25T10:14:22.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.783+0000 7ff3ee905640 1 --2- 172.21.15.60:0/3107297562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3d40051d0 0x7ff3d400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:22.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.787+0000 7ff3ee104640 1 --2- 172.21.15.60:0/3107297562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3d40039c0 0x7ff3d4003de0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009b10 tx=0x7ff3e401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e534659417321 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.787+0000 7ff3ed102640 1 -- 172.21.15.60:0/3107297562 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40096b0 con 0x7ff3d40039c0 2024-04-25T10:14:22.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.787+0000 7ff3ed102640 1 -- 172.21.15.60:0/3107297562 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e40a6490 con 0x7ff3d40039c0 2024-04-25T10:14:22.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.787+0000 7ff3ed102640 1 -- 172.21.15.60:0/3107297562 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40a6880 con 0x7ff3d40039c0 2024-04-25T10:14:22.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.787+0000 7ff3f0368640 1 -- 172.21.15.60:0/3107297562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3d40039c0 msgr2=0x7ff3d4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.787+0000 7ff3f0368640 1 --2- 172.21.15.60:0/3107297562 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3d40039c0 0x7ff3d4003de0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009b10 tx=0x7ff3e401cab0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.787+0000 7ff3ee905640 1 -- 172.21.15.60:0/3107297562 reap_dead start 2024-04-25T10:14:22.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.787+0000 7ff3f0368640 1 -- 172.21.15.60:0/3107297562 shutdown_connections 2024-04-25T10:14:22.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.787+0000 7ff3f0368640 1 -- 172.21.15.60:0/3107297562 >> 172.21.15.60:0/3107297562 conn(0x7ff3e80b12d0 msgr2=0x7ff3e80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.787+0000 7ff3f0368640 1 -- 172.21.15.60:0/3107297562 shutdown_connections 2024-04-25T10:14:22.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.787+0000 7ff3f0368640 1 -- 172.21.15.60:0/3107297562 wait complete. 2024-04-25T10:14:22.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.787+0000 7ff3f0368640 1 Processor -- start 2024-04-25T10:14:22.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.787+0000 7ff3f0368640 1 -- start start 2024-04-25T10:14:22.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3f0368640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3d40039c0 0x7ff3e8158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3f0368640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3e8151060 0x7ff3e814d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3f0368640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3e80af640 0x7ff3e80afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3f0368640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80b4280 con 0x7ff3e8151060 2024-04-25T10:14:22.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3f0368640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80b3db0 con 0x7ff3e80af640 2024-04-25T10:14:22.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3f0368640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80b4400 con 0x7ff3d40039c0 2024-04-25T10:14:22.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3ed903640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3e8151060 0x7ff3e814d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3ed903640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3e8151060 0x7ff3e814d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47554/0 (socket says 172.21.15.60:47554) 2024-04-25T10:14:22.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3ed903640 1 -- 172.21.15.60:0/336664033 learned_addr learned my addr 172.21.15.60:0/336664033 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:22.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3ee905640 1 --2- 172.21.15.60:0/336664033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3e80af640 0x7ff3e80afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3ee104640 1 --2- 172.21.15.60:0/336664033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3d40039c0 0x7ff3e8158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3ed903640 1 -- 172.21.15.60:0/336664033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3d40039c0 msgr2=0x7ff3e8158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3ed903640 1 --2- 172.21.15.60:0/336664033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3d40039c0 0x7ff3e8158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3ed903640 1 -- 172.21.15.60:0/336664033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3e80af640 msgr2=0x7ff3e80afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3ed903640 1 --2- 172.21.15.60:0/336664033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3e80af640 0x7ff3e80afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3ed903640 1 -- 172.21.15.60:0/336664033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e814e9f0 con 0x7ff3e8151060 2024-04-25T10:14:22.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3ed903640 1 --2- 172.21.15.60:0/336664033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3e8151060 0x7ff3e814d040 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8004f90 tx=0x7ff3d80074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3deffd640 1 -- 172.21.15.60:0/336664033 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d801a070 con 0x7ff3e8151060 2024-04-25T10:14:22.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3deffd640 1 -- 172.21.15.60:0/336664033 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d800f750 con 0x7ff3e8151060 2024-04-25T10:14:22.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3f0368640 1 -- 172.21.15.60:0/336664033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e8158ab0 con 0x7ff3e8151060 2024-04-25T10:14:22.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3deffd640 1 -- 172.21.15.60:0/336664033 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d800fb40 con 0x7ff3e8151060 2024-04-25T10:14:22.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3f0368640 1 -- 172.21.15.60:0/336664033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e8151870 con 0x7ff3e8151060 2024-04-25T10:14:22.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3f0368640 1 -- 172.21.15.60:0/336664033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e80b0000 con 0x7ff3e8151060 2024-04-25T10:14:22.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3deffd640 1 -- 172.21.15.60:0/336664033 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3d801d070 con 0x7ff3e8151060 2024-04-25T10:14:22.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3deffd640 1 --2- 172.21.15.60:0/336664033 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3cc066f60 0x7ff3cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3deffd640 1 -- 172.21.15.60:0/336664033 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3d8084e40 con 0x7ff3e8151060 2024-04-25T10:14:22.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.791+0000 7ff3ee104640 1 --2- 172.21.15.60:0/336664033 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3cc066f60 0x7ff3cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.795+0000 7ff3ee104640 1 --2- 172.21.15.60:0/336664033 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3cc066f60 0x7ff3cc069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009b10 tx=0x7ff3e40046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.795+0000 7ff3deffd640 1 -- 172.21.15.60:0/336664033 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3d8055ea0 con 0x7ff3e8151060 2024-04-25T10:14:22.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.987+0000 7ff3f0368640 1 -- 172.21.15.60:0/336664033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff3e814ebc0 con 0x7ff3e8151060 2024-04-25T10:14:22.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.987+0000 7ff3deffd640 1 -- 172.21.15.60:0/336664033 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff3d80557a0 con 0x7ff3e8151060 2024-04-25T10:14:22.994 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:14:22.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.991+0000 7ff3f0368640 1 -- 172.21.15.60:0/336664033 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3cc066f60 msgr2=0x7ff3cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.991+0000 7ff3f0368640 1 --2- 172.21.15.60:0/336664033 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3cc066f60 0x7ff3cc069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009b10 tx=0x7ff3e40046d0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.991+0000 7ff3f0368640 1 -- 172.21.15.60:0/336664033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3e8151060 msgr2=0x7ff3e814d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.991+0000 7ff3f0368640 1 --2- 172.21.15.60:0/336664033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3e8151060 0x7ff3e814d040 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8004f90 tx=0x7ff3d80074f0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.991+0000 7ff3f0368640 1 -- 172.21.15.60:0/336664033 shutdown_connections 2024-04-25T10:14:22.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.991+0000 7ff3f0368640 1 --2- 172.21.15.60:0/336664033 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3cc066f60 0x7ff3cc069420 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.991+0000 7ff3f0368640 1 --2- 172.21.15.60:0/336664033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3e80af640 0x7ff3e80afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.991+0000 7ff3f0368640 1 --2- 172.21.15.60:0/336664033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3e8151060 0x7ff3e814d040 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.991+0000 7ff3f0368640 1 --2- 172.21.15.60:0/336664033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3d40039c0 0x7ff3e8158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.991+0000 7ff3f0368640 1 -- 172.21.15.60:0/336664033 >> 172.21.15.60:0/336664033 conn(0x7ff3e80b12d0 msgr2=0x7ff3e80b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.991+0000 7ff3f0368640 1 -- 172.21.15.60:0/336664033 shutdown_connections 2024-04-25T10:14:22.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:22.991+0000 7ff3f0368640 1 -- 172.21.15.60:0/336664033 wait complete. 2024-04-25T10:14:23.011 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:14:23.011 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:14:23.011 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:14:23.011 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:14:23.011 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:23.011 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:14:23.011 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:14:23.011 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:14:23.011 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:23.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:23.012 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:14:23.018 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:14:23.018 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:14:23.018 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:14:23.018 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:23.018 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:14:23.018 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:23.019 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:14:23.019 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:23.019 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:14:23.019 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:14:23.019 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:23.019 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:14:23.019 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:14:23.019 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:23.019 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:14:23.019 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:14:23.019 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41994sec preferred_lft 41994sec 2024-04-25T10:14:23.019 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:14:23.019 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:23.019 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:14:23.019 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:23.020 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:14:23.020 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:14:23.020 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:23.020 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:14:23.020 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:23.020 INFO:teuthology.orchestra.run.smithi060.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:14:23.020 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:23.020 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:14:23.020 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:23.020 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:14:23.020 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:23.020 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:23.020 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:23.021 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:14:23.021 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:23.021 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:23.021 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:23.021 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:14:23.021 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:23.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:23.021 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:14:23.072 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:14:23.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:23.074 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:14:23.124 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:14:23.124 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:14:23.124 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:14:23.124 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:14:23.124 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:14:23.124 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:14:23.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:23.125 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:14:23.125 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:14:23.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:14:23.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:14:23.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:14:23.125 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:14:23.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:23.198 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:14:23.246 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:14:23.246 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:14:23.246 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:23.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:23.246 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T10:14:23.303 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:14:23.303 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:23.303 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:23.351 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:23.351 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:23.352 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:23.398 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:14:23.456 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:14:23.505 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:14:23.506 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:14:23.506 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:14:23.506 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:14:23.506 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:14:23.506 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:14:23.506 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:14:23.506 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:14:23.506 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:14:23.506 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:14:23.507 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:14:23.507 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:14:23.507 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:14:23.507 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:14:23.507 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:14:23.507 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:14:23.507 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:14:23.507 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:14:23.507 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:14:23.507 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:14:23.507 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:14:23.507 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:14:23.507 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:23.507 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:23.507 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:23.508 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:23.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:23.508 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:14:23.555 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:14:23.555 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:14:23.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:14:23.614 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:14:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:14:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:14:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:14:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:14:23.663 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:14:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:14:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:14:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:14:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:14:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:14:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:14:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:14:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:14:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:14:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:14:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:14:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:14:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:14:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:14:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:14:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:14:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:14:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:14:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:14:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:14:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:14:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:14:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:14:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:14:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:23.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:23.666 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:14:23.684 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2024-04-25T10:14:23.679+0000 7f3b45a8e580 -1 init, newargv = 0x564f76cd7f50 newargc=15 2024-04-25T10:14:23.684 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[48912]: starting ceph client 2024-04-25T10:14:23.689 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[48912]: starting fuse 2024-04-25T10:14:24.789 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:14:24.807 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:14:24.857 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:14:24.857 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:14:24.857 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:14:24.857 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:14:24.857 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:14:24.857 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:14:24.857 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:14:24.857 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:14:24.858 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:14:24.858 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:14:24.858 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:14:24.858 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:14:24.858 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:14:24.858 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:14:24.858 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:14:24.858 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:14:24.858 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:14:24.858 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:14:24.858 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:14:24.858 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:14:24.858 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:14:24.858 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:14:24.859 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:14:24.859 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:14:24.859 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:14:24.859 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:14:24.859 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:14:24.859 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:14:24.859 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:14:24.859 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:24.859 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:24.859 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:24.859 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:24.859 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:24.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:24.860 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:14:24.907 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:14:24.907 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:14:24.907 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:14:24.907 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:14:24.907 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:14:24.907 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:14:24.907 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:14:24.908 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:14:24.908 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:14:24.908 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:14:24.908 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:14:24.908 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:14:24.908 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:14:24.908 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:14:24.908 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:14:24.908 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:14:24.908 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:14:24.908 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:14:24.908 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:14:24.908 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:14:24.908 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:14:24.908 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:14:24.908 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:14:24.909 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:14:24.909 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:14:24.909 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:14:24.909 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:14:24.909 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2024-04-25T10:14:24.909 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:14:25.034 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.48912.asok 2024-04-25T10:14:25.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:25.046 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48912.asok 2024-04-25T10:14:25.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:25.047 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48912.asok status 2024-04-25T10:14:25.209 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:14:25.209 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:14:25.209 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:14:25.209 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:14:25.210 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2024-04-25T10:14:25.210 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:14:25.210 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:14:25.210 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "48912", 2024-04-25T10:14:25.210 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:14:25.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:25.210 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:14:25.210 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:14:25.210 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 8787, 2024-04-25T10:14:25.210 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:14:25.210 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:14:25.210 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:14:25.211 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 8787 2024-04-25T10:14:25.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:25.211 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:14:25.211 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:14:25.211 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:14:25.211 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2492392587 2024-04-25T10:14:25.211 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:25.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:25.211 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:14:25.211 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:14:25.211 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:14:25.212 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2492392587 2024-04-25T10:14:25.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:25.212 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.8787 192.168.0.1:0/2492392587", 2024-04-25T10:14:25.212 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/2492392587", 2024-04-25T10:14:25.212 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:14:25.212 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 77, 2024-04-25T10:14:25.212 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 97, 2024-04-25T10:14:25.212 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:14:25.212 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:14:25.212 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:14:25.212 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:14:25.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:25.223 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:14:25.223 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:25.230 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:14:25.230 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:25.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:25.230 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:25.292 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:14:25.292 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:25.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:25.339 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:14:25.473 INFO:teuthology.orchestra.run.smithi060.stderr:6+0 records in 2024-04-25T10:14:25.474 INFO:teuthology.orchestra.run.smithi060.stderr:6+0 records out 2024-04-25T10:14:25.474 INFO:teuthology.orchestra.run.smithi060.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0726833 s, 86.6 MB/s 2024-04-25T10:14:25.475 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:14:25.475 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:25.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.651+0000 7f7cdf617640 1 Processor -- start 2024-04-25T10:14:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.651+0000 7f7cdf617640 1 -- start start 2024-04-25T10:14:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.655+0000 7f7cdf617640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7cd80ce7a0 0x7f7cd80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.655+0000 7f7cdf617640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cd815bc90 0x7f7cd80b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.655+0000 7f7cdf617640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cd80cf0c0 con 0x7f7cd80b4660 2024-04-25T10:14:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.655+0000 7f7cdf617640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cd80cc370 con 0x7f7cd80ce7a0 2024-04-25T10:14:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.655+0000 7f7cdf617640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd8071ca0 con 0x7f7cd815bc90 2024-04-25T10:14:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.655+0000 7f7cdcbb2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cd815bc90 0x7f7cd80b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.655+0000 7f7cdd3b3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7cd80ce7a0 0x7f7cd80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.655+0000 7f7cdcbb2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cd815bc90 0x7f7cd80b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:57852/0 (socket says 172.21.15.60:57852) 2024-04-25T10:14:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.655+0000 7f7cdcbb2640 1 -- 172.21.15.60:0/2833287022 learned_addr learned my addr 172.21.15.60:0/2833287022 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.655+0000 7f7cdd3b3640 1 -- 172.21.15.60:0/2833287022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cd815bc90 msgr2=0x7f7cd80b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.655+0000 7f7cdd3b3640 1 --2- 172.21.15.60:0/2833287022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cd815bc90 0x7f7cd80b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.655+0000 7f7cdd3b3640 1 -- 172.21.15.60:0/2833287022 >> v1:172.21.15.100:6789/0 conn(0x7f7cd80b4660 legacy=0x7f7cd80b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.655+0000 7f7cdd3b3640 1 -- 172.21.15.60:0/2833287022 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd80cca40 con 0x7f7cd80ce7a0 2024-04-25T10:14:25.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdd3b3640 1 --2- 172.21.15.60:0/2833287022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cd80ce7a0 0x7f7cd80ceb80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009dc0 tx=0x7f7cd401c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d3ee867a51dcc4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7ccffff640 1 -- 172.21.15.60:0/2833287022 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40b2070 con 0x7f7cd80ce7a0 2024-04-25T10:14:25.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7ccffff640 1 -- 172.21.15.60:0/2833287022 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd40a5810 con 0x7f7cd80ce7a0 2024-04-25T10:14:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7ccffff640 1 -- 172.21.15.60:0/2833287022 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40a5be0 con 0x7f7cd80ce7a0 2024-04-25T10:14:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdf617640 1 -- 172.21.15.60:0/2833287022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cd80ce7a0 msgr2=0x7f7cd80ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdf617640 1 --2- 172.21.15.60:0/2833287022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cd80ce7a0 0x7f7cd80ceb80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009dc0 tx=0x7f7cd401c950 comp rx=0 tx=0).stop 2024-04-25T10:14:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdf617640 1 -- 172.21.15.60:0/2833287022 shutdown_connections 2024-04-25T10:14:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdf617640 1 --2- 172.21.15.60:0/2833287022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cd815bc90 0x7f7cd80b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdf617640 1 --2- 172.21.15.60:0/2833287022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cd80ce7a0 0x7f7cd80ceb80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdf617640 1 -- 172.21.15.60:0/2833287022 >> 172.21.15.60:0/2833287022 conn(0x7f7cd80b12d0 msgr2=0x7f7cd80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdf617640 1 -- 172.21.15.60:0/2833287022 shutdown_connections 2024-04-25T10:14:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdf617640 1 -- 172.21.15.60:0/2833287022 wait complete. 2024-04-25T10:14:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdf617640 1 Processor -- start 2024-04-25T10:14:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdf617640 1 -- start start 2024-04-25T10:14:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdf617640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cd80ce7a0 0x7f7cd8152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdf617640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cd815bc90 0x7f7cd80b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdf617640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cd8158e40 0x7f7cd814ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdf617640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd80b4280 con 0x7f7cd815bc90 2024-04-25T10:14:25.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdf617640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd80b3db0 con 0x7f7cd80ce7a0 2024-04-25T10:14:25.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdf617640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd80b4400 con 0x7f7cd8158e40 2024-04-25T10:14:25.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdcbb2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cd815bc90 0x7f7cd80b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdcbb2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cd815bc90 0x7f7cd80b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47558/0 (socket says 172.21.15.60:47558) 2024-04-25T10:14:25.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdcbb2640 1 -- 172.21.15.60:0/3426705862 learned_addr learned my addr 172.21.15.60:0/3426705862 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:25.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cdd3b3640 1 --2- 172.21.15.60:0/3426705862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cd80ce7a0 0x7f7cd8152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.659+0000 7f7cddbb4640 1 --2- 172.21.15.60:0/3426705862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cd8158e40 0x7f7cd814ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7cdcbb2640 1 -- 172.21.15.60:0/3426705862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cd8158e40 msgr2=0x7f7cd814ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7cdcbb2640 1 --2- 172.21.15.60:0/3426705862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cd8158e40 0x7f7cd814ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7cdcbb2640 1 -- 172.21.15.60:0/3426705862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cd80ce7a0 msgr2=0x7f7cd8152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7cdcbb2640 1 --2- 172.21.15.60:0/3426705862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cd80ce7a0 0x7f7cd8152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7cdcbb2640 1 -- 172.21.15.60:0/3426705862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd815a920 con 0x7f7cd815bc90 2024-04-25T10:14:25.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7cdcbb2640 1 --2- 172.21.15.60:0/3426705862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cd815bc90 0x7f7cd80b23b0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800e890 tx=0x7f7cc800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7ccdffb640 1 -- 172.21.15.60:0/3426705862 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8017070 con 0x7f7cd815bc90 2024-04-25T10:14:25.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7cdf617640 1 -- 172.21.15.60:0/3426705862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd815ac00 con 0x7f7cd815bc90 2024-04-25T10:14:25.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7ccdffb640 1 -- 172.21.15.60:0/3426705862 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc800c940 con 0x7f7cd815bc90 2024-04-25T10:14:25.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7ccdffb640 1 -- 172.21.15.60:0/3426705862 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc800cd30 con 0x7f7cd815bc90 2024-04-25T10:14:25.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7cdf617640 1 -- 172.21.15.60:0/3426705862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd815b1c0 con 0x7f7cd815bc90 2024-04-25T10:14:25.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7cdf617640 1 -- 172.21.15.60:0/3426705862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd8074770 con 0x7f7cd815bc90 2024-04-25T10:14:25.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7ccdffb640 1 -- 172.21.15.60:0/3426705862 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7cc801a070 con 0x7f7cd815bc90 2024-04-25T10:14:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7ccdffb640 1 --2- 172.21.15.60:0/3426705862 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7cbc066f90 0x7f7cbc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7ccdffb640 1 -- 172.21.15.60:0/3426705862 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cc8085270 con 0x7f7cd815bc90 2024-04-25T10:14:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7cdd3b3640 1 --2- 172.21.15.60:0/3426705862 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7cbc066f90 0x7f7cbc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.663+0000 7f7cdd3b3640 1 --2- 172.21.15.60:0/3426705862 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7cbc066f90 0x7f7cbc069450 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40af000 tx=0x7f7cd40a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.667+0000 7f7ccdffb640 1 -- 172.21.15.60:0/3426705862 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7cc8055c40 con 0x7f7cd815bc90 2024-04-25T10:14:25.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.827+0000 7f7cdf617640 1 -- 172.21.15.60:0/3426705862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cd80ceb80 con 0x7f7cd815bc90 2024-04-25T10:14:25.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.831+0000 7f7ccdffb640 1 -- 172.21.15.60:0/3426705862 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f7cc8059ce0 con 0x7f7cd815bc90 2024-04-25T10:14:25.835 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:25.835 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":97,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:14:17.788631+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:14.444411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:15.153386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:25.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.831+0000 7f7cdf617640 1 -- 172.21.15.60:0/3426705862 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7cbc066f90 msgr2=0x7f7cbc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.831+0000 7f7cdf617640 1 --2- 172.21.15.60:0/3426705862 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7cbc066f90 0x7f7cbc069450 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40af000 tx=0x7f7cd40a7070 comp rx=0 tx=0).stop 2024-04-25T10:14:25.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.831+0000 7f7cdf617640 1 -- 172.21.15.60:0/3426705862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cd815bc90 msgr2=0x7f7cd80b23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.831+0000 7f7cdf617640 1 --2- 172.21.15.60:0/3426705862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cd815bc90 0x7f7cd80b23b0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800e890 tx=0x7f7cc800ed50 comp rx=0 tx=0).stop 2024-04-25T10:14:25.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.831+0000 7f7cdf617640 1 -- 172.21.15.60:0/3426705862 shutdown_connections 2024-04-25T10:14:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.831+0000 7f7cdf617640 1 --2- 172.21.15.60:0/3426705862 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7cbc066f90 0x7f7cbc069450 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.831+0000 7f7cdf617640 1 --2- 172.21.15.60:0/3426705862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cd8158e40 0x7f7cd814ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.831+0000 7f7cdf617640 1 --2- 172.21.15.60:0/3426705862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cd815bc90 0x7f7cd80b23b0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.831+0000 7f7cdf617640 1 --2- 172.21.15.60:0/3426705862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cd80ce7a0 0x7f7cd8152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.831+0000 7f7cdf617640 1 -- 172.21.15.60:0/3426705862 >> 172.21.15.60:0/3426705862 conn(0x7f7cd80b12d0 msgr2=0x7f7cd8157ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.831+0000 7f7cdf617640 1 -- 172.21.15.60:0/3426705862 shutdown_connections 2024-04-25T10:14:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.831+0000 7f7cdf617640 1 -- 172.21.15.60:0/3426705862 wait complete. 2024-04-25T10:14:25.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:14:14.444411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:14:15.153386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:25.855 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:25.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.983+0000 7f525e478640 1 Processor -- start 2024-04-25T10:14:25.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.983+0000 7f525e478640 1 -- start start 2024-04-25T10:14:25.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.983+0000 7f525e478640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f525815b6f0 0x7f52581620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.983+0000 7f525e478640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f525815e0d0 0x7f525815e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.983+0000 7f525e478640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525815bad0 con 0x7f52580ce7e0 2024-04-25T10:14:25.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.983+0000 7f525e478640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5258162610 con 0x7f525815e0d0 2024-04-25T10:14:25.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.983+0000 7f525e478640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5258161c40 con 0x7f525815b6f0 2024-04-25T10:14:25.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.983+0000 7f52577fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f525815e0d0 0x7f525815e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.983+0000 7f52577fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f525815e0d0 0x7f525815e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47560/0 (socket says 172.21.15.60:47560) 2024-04-25T10:14:25.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.983+0000 7f52577fe640 1 -- 172.21.15.60:0/2684253726 learned_addr learned my addr 172.21.15.60:0/2684253726 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:25.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.983+0000 7f5257fff640 1 --2- 172.21.15.60:0/2684253726 >> v2:172.21.15.100:3300/0 conn(0x7f525815b6f0 0x7f52581620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.983+0000 7f52577fe640 1 -- 172.21.15.60:0/2684253726 >> v2:172.21.15.100:3300/0 conn(0x7f525815b6f0 msgr2=0x7f52581620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:25.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.983+0000 7f52577fe640 1 --2- 172.21.15.60:0/2684253726 >> v2:172.21.15.100:3300/0 conn(0x7f525815b6f0 0x7f52581620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.983+0000 7f52577fe640 1 -- 172.21.15.60:0/2684253726 >> v1:172.21.15.60:6789/0 conn(0x7f52580ce7e0 legacy=0x7f52580d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.983+0000 7f52577fe640 1 -- 172.21.15.60:0/2684253726 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525815ebe0 con 0x7f525815e0d0 2024-04-25T10:14:25.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.983+0000 7f5256ffd640 1 -- 172.21.15.60:0/2684253726 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773063692 0 0) 0x7f525815bad0 con 0x7f52580ce7e0 2024-04-25T10:14:25.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.987+0000 7f52577fe640 1 --2- 172.21.15.60:0/2684253726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f525815e0d0 0x7f525815e510 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f5258073060 tx=0x7f524c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d29253c7c15758 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.987+0000 7f5256ffd640 1 -- 172.21.15.60:0/2684253726 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c0ab610 con 0x7f525815e0d0 2024-04-25T10:14:25.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.987+0000 7f5256ffd640 1 -- 172.21.15.60:0/2684253726 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c0b0070 con 0x7f525815e0d0 2024-04-25T10:14:25.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.987+0000 7f5256ffd640 1 -- 172.21.15.60:0/2684253726 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c0ab9d0 con 0x7f525815e0d0 2024-04-25T10:14:25.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.987+0000 7f525e478640 1 -- 172.21.15.60:0/2684253726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f525815e0d0 msgr2=0x7f525815e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.987+0000 7f525e478640 1 --2- 172.21.15.60:0/2684253726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f525815e0d0 0x7f525815e510 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f5258073060 tx=0x7f524c01c520 comp rx=0 tx=0).stop 2024-04-25T10:14:25.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.987+0000 7f525e478640 1 -- 172.21.15.60:0/2684253726 shutdown_connections 2024-04-25T10:14:25.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.987+0000 7f525e478640 1 --2- 172.21.15.60:0/2684253726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f525815e0d0 0x7f525815e510 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.987+0000 7f525e478640 1 --2- 172.21.15.60:0/2684253726 >> v2:172.21.15.100:3300/0 conn(0x7f525815b6f0 0x7f52581620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.987+0000 7f525e478640 1 -- 172.21.15.60:0/2684253726 >> 172.21.15.60:0/2684253726 conn(0x7f5258072340 msgr2=0x7f5258072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:25.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.987+0000 7f525e478640 1 -- 172.21.15.60:0/2684253726 shutdown_connections 2024-04-25T10:14:25.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.987+0000 7f525e478640 1 -- 172.21.15.60:0/2684253726 wait complete. 2024-04-25T10:14:25.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.987+0000 7f525e478640 1 Processor -- start 2024-04-25T10:14:25.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.987+0000 7f525e478640 1 -- start start 2024-04-25T10:14:25.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f525e478640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52580ce7e0 0x7f52580cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f525e478640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f525815b6f0 0x7f52581f9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f525e478640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f525815e0d0 0x7f5258150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f525e478640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52580b96d0 con 0x7f525815b6f0 2024-04-25T10:14:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f525e478640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52580b9550 con 0x7f52580ce7e0 2024-04-25T10:14:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f525e478640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52580cf000 con 0x7f525815e0d0 2024-04-25T10:14:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f52577fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f525815b6f0 0x7f52581f9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f52577fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f525815b6f0 0x7f52581f9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47562/0 (socket says 172.21.15.60:47562) 2024-04-25T10:14:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f52577fe640 1 -- 172.21.15.60:0/377505938 learned_addr learned my addr 172.21.15.60:0/377505938 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f5257fff640 1 --2- 172.21.15.60:0/377505938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52580ce7e0 0x7f52580cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f52577fe640 1 -- 172.21.15.60:0/377505938 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f525815e0d0 msgr2=0x7f5258150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f525ca15640 1 --2- 172.21.15.60:0/377505938 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f525815e0d0 0x7f5258150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f52577fe640 1 --2- 172.21.15.60:0/377505938 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f525815e0d0 0x7f5258150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f52577fe640 1 -- 172.21.15.60:0/377505938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52580ce7e0 msgr2=0x7f52580cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f52577fe640 1 --2- 172.21.15.60:0/377505938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52580ce7e0 0x7f52580cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f52577fe640 1 -- 172.21.15.60:0/377505938 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52580cd5d0 con 0x7f525815b6f0 2024-04-25T10:14:25.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f5257fff640 1 --2- 172.21.15.60:0/377505938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52580ce7e0 0x7f52580cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:25.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f525ca15640 1 --2- 172.21.15.60:0/377505938 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f525815e0d0 0x7f5258150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:25.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f52577fe640 1 --2- 172.21.15.60:0/377505938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f525815b6f0 0x7f52581f9f70 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f524c01ca00 tx=0x7f524c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f5254ff9640 1 -- 172.21.15.60:0/377505938 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c0be5f0 con 0x7f525815b6f0 2024-04-25T10:14:25.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f5254ff9640 1 -- 172.21.15.60:0/377505938 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c0be7b0 con 0x7f525815b6f0 2024-04-25T10:14:25.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f525e478640 1 -- 172.21.15.60:0/377505938 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52581f02d0 con 0x7f525815b6f0 2024-04-25T10:14:25.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f5254ff9640 1 -- 172.21.15.60:0/377505938 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c0beba0 con 0x7f525815b6f0 2024-04-25T10:14:25.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f525e478640 1 -- 172.21.15.60:0/377505938 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52581f0890 con 0x7f525815b6f0 2024-04-25T10:14:25.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f525e478640 1 -- 172.21.15.60:0/377505938 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5258074770 con 0x7f525815b6f0 2024-04-25T10:14:25.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f5254ff9640 1 -- 172.21.15.60:0/377505938 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f524c0b43f0 con 0x7f525815b6f0 2024-04-25T10:14:25.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f5254ff9640 1 --2- 172.21.15.60:0/377505938 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5244066f60 0x7f5244069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f5254ff9640 1 -- 172.21.15.60:0/377505938 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f524c11fb30 con 0x7f525815b6f0 2024-04-25T10:14:25.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.991+0000 7f5257fff640 1 --2- 172.21.15.60:0/377505938 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5244066f60 0x7f5244069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.995+0000 7f5257fff640 1 --2- 172.21.15.60:0/377505938 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5244066f60 0x7f5244069420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5248009840 tx=0x7f5248008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:25.995+0000 7f5254ff9640 1 -- 172.21.15.60:0/377505938 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f524c0c4060 con 0x7f525815b6f0 2024-04-25T10:14:26.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.159+0000 7f525e478640 1 -- 172.21.15.60:0/377505938 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5258153fc0 con 0x7f525815b6f0 2024-04-25T10:14:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.159+0000 7f5254ff9640 1 -- 172.21.15.60:0/377505938 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7f524c0a8050 con 0x7f525815b6f0 2024-04-25T10:14:26.164 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:14:26.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.163+0000 7f525e478640 1 -- 172.21.15.60:0/377505938 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5244066f60 msgr2=0x7f5244069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.163+0000 7f525e478640 1 --2- 172.21.15.60:0/377505938 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5244066f60 0x7f5244069420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5248009840 tx=0x7f5248008040 comp rx=0 tx=0).stop 2024-04-25T10:14:26.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.163+0000 7f525e478640 1 -- 172.21.15.60:0/377505938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f525815b6f0 msgr2=0x7f52581f9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.163+0000 7f525e478640 1 --2- 172.21.15.60:0/377505938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f525815b6f0 0x7f52581f9f70 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f524c01ca00 tx=0x7f524c01ad60 comp rx=0 tx=0).stop 2024-04-25T10:14:26.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.163+0000 7f525e478640 1 -- 172.21.15.60:0/377505938 shutdown_connections 2024-04-25T10:14:26.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.163+0000 7f525e478640 1 --2- 172.21.15.60:0/377505938 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5244066f60 0x7f5244069420 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.163+0000 7f525e478640 1 --2- 172.21.15.60:0/377505938 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f525815e0d0 0x7f5258150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.163+0000 7f525e478640 1 --2- 172.21.15.60:0/377505938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f525815b6f0 0x7f52581f9f70 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.163+0000 7f525e478640 1 --2- 172.21.15.60:0/377505938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52580ce7e0 0x7f52580cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.163+0000 7f525e478640 1 -- 172.21.15.60:0/377505938 >> 172.21.15.60:0/377505938 conn(0x7f5258072340 msgr2=0x7f525815f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:26.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.163+0000 7f525e478640 1 -- 172.21.15.60:0/377505938 shutdown_connections 2024-04-25T10:14:26.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.163+0000 7f525e478640 1 -- 172.21.15.60:0/377505938 wait complete. 2024-04-25T10:14:26.184 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:26.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2ffec640 1 Processor -- start 2024-04-25T10:14:26.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2ffec640 1 -- start start 2024-04-25T10:14:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2ffec640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed2815fbf0 0x7fed28166870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2ffec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed28144a40 con 0x7fed280cbcf0 2024-04-25T10:14:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2ffec640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed280cc9f0 con 0x7fed28160080 2024-04-25T10:14:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2ffec640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed280b2dc0 con 0x7fed2815fbf0 2024-04-25T10:14:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2dd88640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fed280cbcf0 0x7fed28071c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55538/0 (socket says 172.21.15.60:55538) 2024-04-25T10:14:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2dd88640 1 -- 172.21.15.60:0/3393301455 learned_addr learned my addr 172.21.15.60:0/3393301455 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2e589640 1 --2- 172.21.15.60:0/3393301455 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed2815fbf0 0x7fed28166870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2cd86640 1 -- 172.21.15.60:0/3393301455 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883590348 0 0) 0x7fed28144a40 con 0x7fed280cbcf0 2024-04-25T10:14:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2cd86640 1 -- 172.21.15.60:0/3393301455 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed14003530 con 0x7fed280cbcf0 2024-04-25T10:14:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2cd86640 1 -- 172.21.15.60:0/3393301455 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1569285426 0 0) 0x7fed14003530 con 0x7fed280cbcf0 2024-04-25T10:14:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2cd86640 1 -- 172.21.15.60:0/3393301455 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed2815fbf0 msgr2=0x7fed28166870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2cd86640 1 --2- 172.21.15.60:0/3393301455 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed2815fbf0 0x7fed28166870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2cd86640 1 -- 172.21.15.60:0/3393301455 >> v1:172.21.15.100:6789/0 conn(0x7fed28160080 legacy=0x7fed2815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2cd86640 1 -- 172.21.15.60:0/3393301455 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed280cccd0 con 0x7fed280cbcf0 2024-04-25T10:14:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2cd86640 1 -- 172.21.15.60:0/3393301455 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fed24001680 con 0x7fed280cbcf0 2024-04-25T10:14:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2cd86640 1 -- 172.21.15.60:0/3393301455 >> v1:172.21.15.60:6789/0 conn(0x7fed280cbcf0 legacy=0x7fed28071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2cd86640 1 --2- 172.21.15.60:0/3393301455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140039f0 0x7fed14003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2cd86640 1 --2- 172.21.15.60:0/3393301455 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed2815fbf0 0x7fed14004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2cd86640 1 --2- 172.21.15.60:0/3393301455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed14004d50 0x7fed140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2cd86640 1 -- 172.21.15.60:0/3393301455 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1400ba80 con 0x7fed140039f0 2024-04-25T10:14:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2cd86640 1 -- 172.21.15.60:0/3393301455 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1400bc30 con 0x7fed14004d50 2024-04-25T10:14:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2cd86640 1 -- 172.21.15.60:0/3393301455 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1400be30 con 0x7fed2815fbf0 2024-04-25T10:14:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2dd88640 1 --2- 172.21.15.60:0/3393301455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140039f0 0x7fed14003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2e589640 1 --2- 172.21.15.60:0/3393301455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed14004d50 0x7fed140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2d587640 1 --2- 172.21.15.60:0/3393301455 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed2815fbf0 0x7fed14004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2dd88640 1 -- 172.21.15.60:0/3393301455 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed2815fbf0 msgr2=0x7fed14004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2dd88640 1 --2- 172.21.15.60:0/3393301455 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed2815fbf0 0x7fed14004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2dd88640 1 -- 172.21.15.60:0/3393301455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed14004d50 msgr2=0x7fed140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2dd88640 1 --2- 172.21.15.60:0/3393301455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed14004d50 0x7fed140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.303+0000 7fed2dd88640 1 -- 172.21.15.60:0/3393301455 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed14003530 con 0x7fed140039f0 2024-04-25T10:14:26.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.307+0000 7fed2dd88640 1 --2- 172.21.15.60:0/3393301455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140039f0 0x7fed14003e10 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fed2400c240 tx=0x7fed2401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32343d1ebe613ec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.307+0000 7fed2cd86640 1 -- 172.21.15.60:0/3393301455 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24007d20 con 0x7fed140039f0 2024-04-25T10:14:26.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.307+0000 7fed2cd86640 1 -- 172.21.15.60:0/3393301455 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed240a9070 con 0x7fed140039f0 2024-04-25T10:14:26.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.307+0000 7fed2cd86640 1 -- 172.21.15.60:0/3393301455 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24005d60 con 0x7fed140039f0 2024-04-25T10:14:26.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 -- 172.21.15.60:0/3393301455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140039f0 msgr2=0x7fed14003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 --2- 172.21.15.60:0/3393301455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140039f0 0x7fed14003e10 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fed2400c240 tx=0x7fed2401cc80 comp rx=0 tx=0).stop 2024-04-25T10:14:26.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2e589640 1 -- 172.21.15.60:0/3393301455 reap_dead start 2024-04-25T10:14:26.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 -- 172.21.15.60:0/3393301455 shutdown_connections 2024-04-25T10:14:26.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 -- 172.21.15.60:0/3393301455 >> 172.21.15.60:0/3393301455 conn(0x7fed280670b0 msgr2=0x7fed2806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:26.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 -- 172.21.15.60:0/3393301455 shutdown_connections 2024-04-25T10:14:26.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 -- 172.21.15.60:0/3393301455 wait complete. 2024-04-25T10:14:26.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 Processor -- start 2024-04-25T10:14:26.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 -- start start 2024-04-25T10:14:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140039f0 0x7fed280ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed281fa8b0 0x7fed280ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed2813daa0 0x7fed280b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed280cfed0 con 0x7fed140039f0 2024-04-25T10:14:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed280ce5c0 con 0x7fed281fa8b0 2024-04-25T10:14:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed280acf30 con 0x7fed2813daa0 2024-04-25T10:14:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2dd88640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140039f0 0x7fed280ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2dd88640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140039f0 0x7fed280ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47576/0 (socket says 172.21.15.60:47576) 2024-04-25T10:14:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2dd88640 1 -- 172.21.15.60:0/3489459829 learned_addr learned my addr 172.21.15.60:0/3489459829 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2e589640 1 --2- 172.21.15.60:0/3489459829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed2813daa0 0x7fed280b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2d587640 1 --2- 172.21.15.60:0/3489459829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed281fa8b0 0x7fed280ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2dd88640 1 -- 172.21.15.60:0/3489459829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed2813daa0 msgr2=0x7fed280b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2dd88640 1 --2- 172.21.15.60:0/3489459829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed2813daa0 0x7fed280b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2dd88640 1 -- 172.21.15.60:0/3489459829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed281fa8b0 msgr2=0x7fed280ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2dd88640 1 --2- 172.21.15.60:0/3489459829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed281fa8b0 0x7fed280ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2dd88640 1 -- 172.21.15.60:0/3489459829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed281f9dd0 con 0x7fed140039f0 2024-04-25T10:14:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2dd88640 1 --2- 172.21.15.60:0/3489459829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140039f0 0x7fed280ac8a0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fed2400c240 tx=0x7fed14005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed1e7fc640 1 -- 172.21.15.60:0/3489459829 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24005d40 con 0x7fed140039f0 2024-04-25T10:14:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed1e7fc640 1 -- 172.21.15.60:0/3489459829 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed240a5850 con 0x7fed140039f0 2024-04-25T10:14:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed1e7fc640 1 -- 172.21.15.60:0/3489459829 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed240a5c00 con 0x7fed140039f0 2024-04-25T10:14:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 -- 172.21.15.60:0/3489459829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed281fa0b0 con 0x7fed140039f0 2024-04-25T10:14:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 -- 172.21.15.60:0/3489459829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed280aeba0 con 0x7fed140039f0 2024-04-25T10:14:26.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.311+0000 7fed2ffec640 1 -- 172.21.15.60:0/3489459829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2806bba0 con 0x7fed140039f0 2024-04-25T10:14:26.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.315+0000 7fed1e7fc640 1 -- 172.21.15.60:0/3489459829 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed240047f0 con 0x7fed140039f0 2024-04-25T10:14:26.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.315+0000 7fed1e7fc640 1 --2- 172.21.15.60:0/3489459829 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed0c066f60 0x7fed0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.315+0000 7fed1e7fc640 1 -- 172.21.15.60:0/3489459829 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed2411e6e0 con 0x7fed140039f0 2024-04-25T10:14:26.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.315+0000 7fed2d587640 1 --2- 172.21.15.60:0/3489459829 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed0c066f60 0x7fed0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.315+0000 7fed2d587640 1 --2- 172.21.15.60:0/3489459829 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed0c066f60 0x7fed0c069420 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fed2806b290 tx=0x7fed18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.319+0000 7fed1e7fc640 1 -- 172.21.15.60:0/3489459829 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed24016990 con 0x7fed140039f0 2024-04-25T10:14:26.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.479+0000 7fed2ffec640 1 -- 172.21.15.60:0/3489459829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed28074770 con 0x7fed140039f0 2024-04-25T10:14:26.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.479+0000 7fed1e7fc640 1 -- 172.21.15.60:0/3489459829 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed240ef6d0 con 0x7fed140039f0 2024-04-25T10:14:26.483 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:14:26.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.479+0000 7fed2ffec640 1 -- 172.21.15.60:0/3489459829 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed0c066f60 msgr2=0x7fed0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.479+0000 7fed2ffec640 1 --2- 172.21.15.60:0/3489459829 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed0c066f60 0x7fed0c069420 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fed2806b290 tx=0x7fed18009040 comp rx=0 tx=0).stop 2024-04-25T10:14:26.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.479+0000 7fed2ffec640 1 -- 172.21.15.60:0/3489459829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140039f0 msgr2=0x7fed280ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.479+0000 7fed2ffec640 1 --2- 172.21.15.60:0/3489459829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140039f0 0x7fed280ac8a0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fed2400c240 tx=0x7fed14005840 comp rx=0 tx=0).stop 2024-04-25T10:14:26.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.479+0000 7fed2ffec640 1 -- 172.21.15.60:0/3489459829 shutdown_connections 2024-04-25T10:14:26.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.479+0000 7fed2ffec640 1 --2- 172.21.15.60:0/3489459829 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed0c066f60 0x7fed0c069420 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.479+0000 7fed2ffec640 1 --2- 172.21.15.60:0/3489459829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed2813daa0 0x7fed280b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.479+0000 7fed2ffec640 1 --2- 172.21.15.60:0/3489459829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed281fa8b0 0x7fed280ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.479+0000 7fed2ffec640 1 --2- 172.21.15.60:0/3489459829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed140039f0 0x7fed280ac8a0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.479+0000 7fed2ffec640 1 -- 172.21.15.60:0/3489459829 >> 172.21.15.60:0/3489459829 conn(0x7fed280670b0 msgr2=0x7fed2815bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:26.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.479+0000 7fed2ffec640 1 -- 172.21.15.60:0/3489459829 shutdown_connections 2024-04-25T10:14:26.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.479+0000 7fed2ffec640 1 -- 172.21.15.60:0/3489459829 wait complete. 2024-04-25T10:14:26.503 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:26.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08ee8ba640 1 Processor -- start 2024-04-25T10:14:26.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08ee8ba640 1 -- start start 2024-04-25T10:14:26.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08ee8ba640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08e815b6f0 0x7f08e81620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08ee8ba640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e815bad0 con 0x7f08e815e0d0 2024-04-25T10:14:26.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08ee8ba640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e815ebe0 con 0x7f08e80ceef0 2024-04-25T10:14:26.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08ee8ba640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e8162610 con 0x7f08e815b6f0 2024-04-25T10:14:26.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e77fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f08e815e0d0 0x7f08e80ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55548/0 (socket says 172.21.15.60:55548) 2024-04-25T10:14:26.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e77fe640 1 -- 172.21.15.60:0/236685532 learned_addr learned my addr 172.21.15.60:0/236685532 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:26.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e7fff640 1 --2- 172.21.15.60:0/236685532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08e815b6f0 0x7f08e81620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e6ffd640 1 -- 172.21.15.60:0/236685532 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192862030 0 0) 0x7f08e815bad0 con 0x7f08e815e0d0 2024-04-25T10:14:26.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e6ffd640 1 -- 172.21.15.60:0/236685532 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08d4003530 con 0x7f08e815e0d0 2024-04-25T10:14:26.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e6ffd640 1 -- 172.21.15.60:0/236685532 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2527332093 0 0) 0x7f08d4003530 con 0x7f08e815e0d0 2024-04-25T10:14:26.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e6ffd640 1 -- 172.21.15.60:0/236685532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08e815b6f0 msgr2=0x7f08e81620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e6ffd640 1 --2- 172.21.15.60:0/236685532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08e815b6f0 0x7f08e81620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e6ffd640 1 -- 172.21.15.60:0/236685532 >> v1:172.21.15.100:6789/0 conn(0x7f08e80ceef0 legacy=0x7f08e80d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e6ffd640 1 -- 172.21.15.60:0/236685532 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e815eec0 con 0x7f08e815e0d0 2024-04-25T10:14:26.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e6ffd640 1 -- 172.21.15.60:0/236685532 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f08d0001680 con 0x7f08e815e0d0 2024-04-25T10:14:26.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e6ffd640 1 -- 172.21.15.60:0/236685532 >> v1:172.21.15.60:6789/0 conn(0x7f08e815e0d0 legacy=0x7f08e80ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e6ffd640 1 --2- 172.21.15.60:0/236685532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08e815b6f0 0x7f08d40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e6ffd640 1 --2- 172.21.15.60:0/236685532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08d4003f70 0x7f08d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e6ffd640 1 --2- 172.21.15.60:0/236685532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08d4004d50 0x7f08d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e6ffd640 1 -- 172.21.15.60:0/236685532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e8162610 con 0x7f08d4003f70 2024-04-25T10:14:26.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e6ffd640 1 -- 172.21.15.60:0/236685532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d400bab0 con 0x7f08d4004d50 2024-04-25T10:14:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e6ffd640 1 -- 172.21.15.60:0/236685532 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d400bcb0 con 0x7f08e815b6f0 2024-04-25T10:14:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e77fe640 1 --2- 172.21.15.60:0/236685532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08d4003f70 0x7f08d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e7fff640 1 --2- 172.21.15.60:0/236685532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08e815b6f0 0x7f08d40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08ece57640 1 --2- 172.21.15.60:0/236685532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08d4004d50 0x7f08d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e77fe640 1 -- 172.21.15.60:0/236685532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08e815b6f0 msgr2=0x7f08d40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e77fe640 1 --2- 172.21.15.60:0/236685532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08e815b6f0 0x7f08d40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e77fe640 1 -- 172.21.15.60:0/236685532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08d4004d50 msgr2=0x7f08d40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e77fe640 1 --2- 172.21.15.60:0/236685532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08d4004d50 0x7f08d40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.619+0000 7f08e77fe640 1 -- 172.21.15.60:0/236685532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d4003530 con 0x7f08d4003f70 2024-04-25T10:14:26.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08e77fe640 1 --2- 172.21.15.60:0/236685532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08d4003f70 0x7f08d4004810 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f08d000c210 tx=0x7f08d001f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a057844321fd3607 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08e6ffd640 1 -- 172.21.15.60:0/236685532 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0007d40 con 0x7f08d4003f70 2024-04-25T10:14:26.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08e6ffd640 1 -- 172.21.15.60:0/236685532 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d0005b00 con 0x7f08d4003f70 2024-04-25T10:14:26.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08e6ffd640 1 -- 172.21.15.60:0/236685532 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0007070 con 0x7f08d4003f70 2024-04-25T10:14:26.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ee8ba640 1 -- 172.21.15.60:0/236685532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08d4003f70 msgr2=0x7f08d4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ee8ba640 1 --2- 172.21.15.60:0/236685532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08d4003f70 0x7f08d4004810 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f08d000c210 tx=0x7f08d001f3f0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ece57640 1 -- 172.21.15.60:0/236685532 reap_dead start 2024-04-25T10:14:26.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ee8ba640 1 -- 172.21.15.60:0/236685532 shutdown_connections 2024-04-25T10:14:26.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ee8ba640 1 -- 172.21.15.60:0/236685532 >> 172.21.15.60:0/236685532 conn(0x7f08e8072340 msgr2=0x7f08e8072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:26.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ee8ba640 1 -- 172.21.15.60:0/236685532 shutdown_connections 2024-04-25T10:14:26.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ee8ba640 1 -- 172.21.15.60:0/236685532 wait complete. 2024-04-25T10:14:26.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ee8ba640 1 Processor -- start 2024-04-25T10:14:26.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ee8ba640 1 -- start start 2024-04-25T10:14:26.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ee8ba640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08d4003f70 0x7f08e814ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ee8ba640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08e8154d80 0x7f08e80c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ee8ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e81f4350 0x7f08e80b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ee8ba640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e80b9550 con 0x7f08e81f4350 2024-04-25T10:14:26.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ee8ba640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e8161c40 con 0x7f08d4003f70 2024-04-25T10:14:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ee8ba640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e80b96d0 con 0x7f08e8154d80 2024-04-25T10:14:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ece57640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e81f4350 0x7f08e80b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.623+0000 7f08ece57640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e81f4350 0x7f08e80b81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47588/0 (socket says 172.21.15.60:47588) 2024-04-25T10:14:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08ece57640 1 -- 172.21.15.60:0/3957584006 learned_addr learned my addr 172.21.15.60:0/3957584006 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08e7fff640 1 --2- 172.21.15.60:0/3957584006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08d4003f70 0x7f08e814ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08e77fe640 1 --2- 172.21.15.60:0/3957584006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08e8154d80 0x7f08e80c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08ece57640 1 -- 172.21.15.60:0/3957584006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08e8154d80 msgr2=0x7f08e80c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08ece57640 1 --2- 172.21.15.60:0/3957584006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08e8154d80 0x7f08e80c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08ece57640 1 -- 172.21.15.60:0/3957584006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08d4003f70 msgr2=0x7f08e814ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08ece57640 1 --2- 172.21.15.60:0/3957584006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08d4003f70 0x7f08e814ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08ece57640 1 -- 172.21.15.60:0/3957584006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e80c8400 con 0x7f08e81f4350 2024-04-25T10:14:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08e77fe640 1 --2- 172.21.15.60:0/3957584006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08e8154d80 0x7f08e80c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08e7fff640 1 --2- 172.21.15.60:0/3957584006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08d4003f70 0x7f08e814ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:26.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08ece57640 1 --2- 172.21.15.60:0/3957584006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e81f4350 0x7f08e80b81a0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f08e80d1690 tx=0x7f08e000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08e4ff9640 1 -- 172.21.15.60:0/3957584006 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0003980 con 0x7f08e81f4350 2024-04-25T10:14:26.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08e4ff9640 1 -- 172.21.15.60:0/3957584006 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e0018070 con 0x7f08e81f4350 2024-04-25T10:14:26.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08ee8ba640 1 -- 172.21.15.60:0/3957584006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e80c7e40 con 0x7f08e81f4350 2024-04-25T10:14:26.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08e4ff9640 1 -- 172.21.15.60:0/3957584006 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0003d60 con 0x7f08e81f4350 2024-04-25T10:14:26.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08ee8ba640 1 -- 172.21.15.60:0/3957584006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e81f3b50 con 0x7f08e81f4350 2024-04-25T10:14:26.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08ee8ba640 1 -- 172.21.15.60:0/3957584006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e80d16e0 con 0x7f08e81f4350 2024-04-25T10:14:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08e4ff9640 1 -- 172.21.15.60:0/3957584006 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08e00050d0 con 0x7f08e81f4350 2024-04-25T10:14:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08e4ff9640 1 --2- 172.21.15.60:0/3957584006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f08c8066f60 0x7f08c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08e4ff9640 1 -- 172.21.15.60:0/3957584006 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08e0084da0 con 0x7f08e81f4350 2024-04-25T10:14:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08e7fff640 1 --2- 172.21.15.60:0/3957584006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f08c8066f60 0x7f08c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.627+0000 7f08e7fff640 1 --2- 172.21.15.60:0/3957584006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f08c8066f60 0x7f08c8069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f08e80cdd10 tx=0x7f08d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.631+0000 7f08e4ff9640 1 -- 172.21.15.60:0/3957584006 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f08e00556c0 con 0x7f08e81f4350 2024-04-25T10:14:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.787+0000 7f08ee8ba640 1 -- 172.21.15.60:0/3957584006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f08e80d2280 con 0x7f08e81f4350 2024-04-25T10:14:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.787+0000 7f08e4ff9640 1 -- 172.21.15.60:0/3957584006 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+11 (secure 0 0 0) 0x7f08e0059760 con 0x7f08e81f4350 2024-04-25T10:14:26.794 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:14:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.791+0000 7f08ee8ba640 1 -- 172.21.15.60:0/3957584006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f08c8066f60 msgr2=0x7f08c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.791+0000 7f08ee8ba640 1 --2- 172.21.15.60:0/3957584006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f08c8066f60 0x7f08c8069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f08e80cdd10 tx=0x7f08d8008040 comp rx=0 tx=0).stop 2024-04-25T10:14:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.791+0000 7f08ee8ba640 1 -- 172.21.15.60:0/3957584006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e81f4350 msgr2=0x7f08e80b81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.791+0000 7f08ee8ba640 1 --2- 172.21.15.60:0/3957584006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e81f4350 0x7f08e80b81a0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f08e80d1690 tx=0x7f08e000ec10 comp rx=0 tx=0).stop 2024-04-25T10:14:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.791+0000 7f08ee8ba640 1 -- 172.21.15.60:0/3957584006 shutdown_connections 2024-04-25T10:14:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.791+0000 7f08ee8ba640 1 --2- 172.21.15.60:0/3957584006 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f08c8066f60 0x7f08c8069420 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.791+0000 7f08ee8ba640 1 --2- 172.21.15.60:0/3957584006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e81f4350 0x7f08e80b81a0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.791+0000 7f08ee8ba640 1 --2- 172.21.15.60:0/3957584006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08e8154d80 0x7f08e80c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.791+0000 7f08ee8ba640 1 --2- 172.21.15.60:0/3957584006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08d4003f70 0x7f08e814ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.791+0000 7f08ee8ba640 1 -- 172.21.15.60:0/3957584006 >> 172.21.15.60:0/3957584006 conn(0x7f08e8072340 msgr2=0x7f08e815a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.791+0000 7f08ee8ba640 1 -- 172.21.15.60:0/3957584006 shutdown_connections 2024-04-25T10:14:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.791+0000 7f08ee8ba640 1 -- 172.21.15.60:0/3957584006 wait complete. 2024-04-25T10:14:26.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:14:26.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.931+0000 7fa14dcc1640 1 Processor -- start 2024-04-25T10:14:26.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.931+0000 7fa14dcc1640 1 -- start start 2024-04-25T10:14:26.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.931+0000 7fa14dcc1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa14815f480 0x7fa14815f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.931+0000 7fa14dcc1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1480cf380 0x7fa1480cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.931+0000 7fa14dcc1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa14815fda0 con 0x7fa1480cd390 2024-04-25T10:14:26.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.931+0000 7fa14dcc1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1480cc5d0 con 0x7fa14815f480 2024-04-25T10:14:26.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.931+0000 7fa14dcc1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1480cc8b0 con 0x7fa1480cf380 2024-04-25T10:14:26.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.931+0000 7fa147fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1480cf380 0x7fa1480cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.931+0000 7fa1477fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa14815f480 0x7fa14815f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa146ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa1480cd390 0x7fa14809a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:44568/0 (socket says 172.21.15.60:44568) 2024-04-25T10:14:26.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa146ffd640 1 -- 172.21.15.60:0/1568216883 learned_addr learned my addr 172.21.15.60:0/1568216883 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:26.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa1477fe640 1 -- 172.21.15.60:0/1568216883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1480cf380 msgr2=0x7fa1480cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa1477fe640 1 --2- 172.21.15.60:0/1568216883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1480cf380 0x7fa1480cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa1477fe640 1 -- 172.21.15.60:0/1568216883 >> v1:172.21.15.100:6789/0 conn(0x7fa1480cd390 legacy=0x7fa14809a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa1477fe640 1 -- 172.21.15.60:0/1568216883 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1480ccb90 con 0x7fa14815f480 2024-04-25T10:14:26.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa1477fe640 1 --2- 172.21.15.60:0/1568216883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa14815f480 0x7fa14815f860 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa1380097c0 tx=0x7fa13801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29f2de8968155e93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa1467fc640 1 -- 172.21.15.60:0/1568216883 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1380aa5f0 con 0x7fa14815f480 2024-04-25T10:14:26.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa1467fc640 1 -- 172.21.15.60:0/1568216883 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1380af070 con 0x7fa14815f480 2024-04-25T10:14:26.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa1467fc640 1 -- 172.21.15.60:0/1568216883 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1380aa9b0 con 0x7fa14815f480 2024-04-25T10:14:26.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa14dcc1640 1 -- 172.21.15.60:0/1568216883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa14815f480 msgr2=0x7fa14815f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa14dcc1640 1 --2- 172.21.15.60:0/1568216883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa14815f480 0x7fa14815f860 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa1380097c0 tx=0x7fa13801c470 comp rx=0 tx=0).stop 2024-04-25T10:14:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa14dcc1640 1 -- 172.21.15.60:0/1568216883 shutdown_connections 2024-04-25T10:14:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa14dcc1640 1 --2- 172.21.15.60:0/1568216883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1480cf380 0x7fa1480cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa14dcc1640 1 --2- 172.21.15.60:0/1568216883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa14815f480 0x7fa14815f860 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa14dcc1640 1 -- 172.21.15.60:0/1568216883 >> 172.21.15.60:0/1568216883 conn(0x7fa1480b32a0 msgr2=0x7fa1480b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa14dcc1640 1 -- 172.21.15.60:0/1568216883 shutdown_connections 2024-04-25T10:14:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.935+0000 7fa14dcc1640 1 -- 172.21.15.60:0/1568216883 wait complete. 2024-04-25T10:14:26.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa14dcc1640 1 Processor -- start 2024-04-25T10:14:26.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa14dcc1640 1 -- start start 2024-04-25T10:14:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa14dcc1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1480cd390 0x7fa1481544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa14dcc1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1480cf380 0x7fa148154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa14dcc1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa14815f480 0x7fa14806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa14dcc1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa148160170 con 0x7fa1480cf380 2024-04-25T10:14:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa14dcc1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1481602f0 con 0x7fa14815f480 2024-04-25T10:14:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa14dcc1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14815cbc0 con 0x7fa1480cd390 2024-04-25T10:14:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa146ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1480cf380 0x7fa148154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa146ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1480cf380 0x7fa148154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47590/0 (socket says 172.21.15.60:47590) 2024-04-25T10:14:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa146ffd640 1 -- 172.21.15.60:0/3473069026 learned_addr learned my addr 172.21.15.60:0/3473069026 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa1477fe640 1 --2- 172.21.15.60:0/3473069026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1480cd390 0x7fa1481544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa147fff640 1 --2- 172.21.15.60:0/3473069026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa14815f480 0x7fa14806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa146ffd640 1 -- 172.21.15.60:0/3473069026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1480cd390 msgr2=0x7fa1481544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa146ffd640 1 --2- 172.21.15.60:0/3473069026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1480cd390 0x7fa1481544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa146ffd640 1 -- 172.21.15.60:0/3473069026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa14815f480 msgr2=0x7fa14806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa146ffd640 1 --2- 172.21.15.60:0/3473069026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa14815f480 0x7fa14806c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa146ffd640 1 -- 172.21.15.60:0/3473069026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa148165550 con 0x7fa1480cf380 2024-04-25T10:14:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa146ffd640 1 --2- 172.21.15.60:0/3473069026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1480cf380 0x7fa148154a20 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00c420 tx=0x7fa13c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa12ffff640 1 -- 172.21.15.60:0/3473069026 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c017070 con 0x7fa1480cf380 2024-04-25T10:14:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa12ffff640 1 -- 172.21.15.60:0/3473069026 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c005cf0 con 0x7fa1480cf380 2024-04-25T10:14:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa14dcc1640 1 -- 172.21.15.60:0/3473069026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa148165830 con 0x7fa1480cf380 2024-04-25T10:14:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa12ffff640 1 -- 172.21.15.60:0/3473069026 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0072f0 con 0x7fa1480cf380 2024-04-25T10:14:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.939+0000 7fa14dcc1640 1 -- 172.21.15.60:0/3473069026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa148165df0 con 0x7fa1480cf380 2024-04-25T10:14:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.943+0000 7fa14dcc1640 1 -- 172.21.15.60:0/3473069026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa148165ff0 con 0x7fa1480cf380 2024-04-25T10:14:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.943+0000 7fa12ffff640 1 -- 172.21.15.60:0/3473069026 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa13c01a070 con 0x7fa1480cf380 2024-04-25T10:14:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.943+0000 7fa12ffff640 1 --2- 172.21.15.60:0/3473069026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa134066f60 0x7fa134069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.943+0000 7fa12ffff640 1 -- 172.21.15.60:0/3473069026 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa13c0842c0 con 0x7fa1480cf380 2024-04-25T10:14:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.943+0000 7fa1477fe640 1 --2- 172.21.15.60:0/3473069026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa134066f60 0x7fa134069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.943+0000 7fa1477fe640 1 --2- 172.21.15.60:0/3473069026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa134066f60 0x7fa134069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa1380ae000 tx=0x7fa1380a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:26.947+0000 7fa12ffff640 1 -- 172.21.15.60:0/3473069026 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa148165ff0 con 0x7fa1480cf380 2024-04-25T10:14:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.119+0000 7fa14dcc1640 1 -- 172.21.15.60:0/3473069026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa148074770 con 0x7fa1480cf380 2024-04-25T10:14:27.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.119+0000 7fa12ffff640 1 -- 172.21.15.60:0/3473069026 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v97) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa13c054d10 con 0x7fa1480cf380 2024-04-25T10:14:27.126 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:14:27.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.123+0000 7fa14dcc1640 1 -- 172.21.15.60:0/3473069026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa134066f60 msgr2=0x7fa134069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.123+0000 7fa14dcc1640 1 --2- 172.21.15.60:0/3473069026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa134066f60 0x7fa134069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa1380ae000 tx=0x7fa1380a7000 comp rx=0 tx=0).stop 2024-04-25T10:14:27.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.123+0000 7fa14dcc1640 1 -- 172.21.15.60:0/3473069026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1480cf380 msgr2=0x7fa148154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.123+0000 7fa14dcc1640 1 --2- 172.21.15.60:0/3473069026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1480cf380 0x7fa148154a20 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00c420 tx=0x7fa13c00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.123+0000 7fa14dcc1640 1 -- 172.21.15.60:0/3473069026 shutdown_connections 2024-04-25T10:14:27.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.123+0000 7fa14dcc1640 1 --2- 172.21.15.60:0/3473069026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa134066f60 0x7fa134069420 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.123+0000 7fa14dcc1640 1 --2- 172.21.15.60:0/3473069026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa14815f480 0x7fa14806c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.123+0000 7fa14dcc1640 1 --2- 172.21.15.60:0/3473069026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1480cf380 0x7fa148154a20 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.123+0000 7fa14dcc1640 1 --2- 172.21.15.60:0/3473069026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1480cd390 0x7fa1481544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.123+0000 7fa14dcc1640 1 -- 172.21.15.60:0/3473069026 >> 172.21.15.60:0/3473069026 conn(0x7fa1480b32a0 msgr2=0x7fa148158400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:27.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.123+0000 7fa14dcc1640 1 -- 172.21.15.60:0/3473069026 shutdown_connections 2024-04-25T10:14:27.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.123+0000 7fa14dcc1640 1 -- 172.21.15.60:0/3473069026 wait complete. 2024-04-25T10:14:27.144 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:14:27.144 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:27.150 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.315+0000 7efe99f10640 1 Processor -- start 2024-04-25T10:14:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.315+0000 7efe99f10640 1 -- start start 2024-04-25T10:14:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.319+0000 7efe99f10640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efe9415deb0 0x7efe940b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.319+0000 7efe99f10640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe940d0340 0x7efe940d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.319+0000 7efe99f10640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9406d200 con 0x7efe940ce2b0 2024-04-25T10:14:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.319+0000 7efe99f10640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe940cf2c0 con 0x7efe9415deb0 2024-04-25T10:14:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.319+0000 7efe99f10640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe940cefe0 con 0x7efe940d0340 2024-04-25T10:14:27.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.319+0000 7efe92ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe940d0340 0x7efe940d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.319+0000 7efe937fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efe9415deb0 0x7efe940b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.319+0000 7efe92ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe940d0340 0x7efe940d07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:57942/0 (socket says 172.21.15.60:57942) 2024-04-25T10:14:27.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.319+0000 7efe92ffd640 1 -- 172.21.15.60:0/1767458182 learned_addr learned my addr 172.21.15.60:0/1767458182 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:27.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.319+0000 7efe92ffd640 1 -- 172.21.15.60:0/1767458182 >> v2:172.21.15.100:3300/0 conn(0x7efe9415deb0 msgr2=0x7efe940b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:27.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.319+0000 7efe92ffd640 1 --2- 172.21.15.60:0/1767458182 >> v2:172.21.15.100:3300/0 conn(0x7efe9415deb0 0x7efe940b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.319+0000 7efe92ffd640 1 -- 172.21.15.60:0/1767458182 >> v1:172.21.15.100:6789/0 conn(0x7efe940ce2b0 legacy=0x7efe940ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.319+0000 7efe92ffd640 1 -- 172.21.15.60:0/1767458182 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe940d1d70 con 0x7efe940d0340 2024-04-25T10:14:27.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.319+0000 7efe92ffd640 1 --2- 172.21.15.60:0/1767458182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe940d0340 0x7efe940d07c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efe88009870 tx=0x7efe8801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6224f305473cf99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe927fc640 1 -- 172.21.15.60:0/1767458182 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe880b2070 con 0x7efe940d0340 2024-04-25T10:14:27.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe927fc640 1 -- 172.21.15.60:0/1767458182 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe880a9070 con 0x7efe940d0340 2024-04-25T10:14:27.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe927fc640 1 -- 172.21.15.60:0/1767458182 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe880ad880 con 0x7efe940d0340 2024-04-25T10:14:27.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe99f10640 1 -- 172.21.15.60:0/1767458182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe940d0340 msgr2=0x7efe940d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe99f10640 1 --2- 172.21.15.60:0/1767458182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe940d0340 0x7efe940d07c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efe88009870 tx=0x7efe8801c560 comp rx=0 tx=0).stop 2024-04-25T10:14:27.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe99f10640 1 -- 172.21.15.60:0/1767458182 shutdown_connections 2024-04-25T10:14:27.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe99f10640 1 --2- 172.21.15.60:0/1767458182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe940d0340 0x7efe940d07c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe99f10640 1 --2- 172.21.15.60:0/1767458182 >> v2:172.21.15.100:3300/0 conn(0x7efe9415deb0 0x7efe940b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe99f10640 1 -- 172.21.15.60:0/1767458182 >> 172.21.15.60:0/1767458182 conn(0x7efe9406c090 msgr2=0x7efe9406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:27.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe99f10640 1 -- 172.21.15.60:0/1767458182 shutdown_connections 2024-04-25T10:14:27.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe99f10640 1 -- 172.21.15.60:0/1767458182 wait complete. 2024-04-25T10:14:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe99f10640 1 Processor -- start 2024-04-25T10:14:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe99f10640 1 -- start start 2024-04-25T10:14:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe99f10640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe940ce2b0 0x7efe941497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe99f10640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe940d0340 0x7efe940c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe99f10640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe9415deb0 0x7efe940c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe99f10640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940d3430 con 0x7efe940d0340 2024-04-25T10:14:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe99f10640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940d32b0 con 0x7efe9415deb0 2024-04-25T10:14:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe99f10640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe940d35b0 con 0x7efe940ce2b0 2024-04-25T10:14:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe92ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe940d0340 0x7efe940c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe92ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe940d0340 0x7efe940c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47602/0 (socket says 172.21.15.60:47602) 2024-04-25T10:14:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe92ffd640 1 -- 172.21.15.60:0/1738477600 learned_addr learned my addr 172.21.15.60:0/1738477600 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:27.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe93fff640 1 --2- 172.21.15.60:0/1738477600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe9415deb0 0x7efe940c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe937fe640 1 --2- 172.21.15.60:0/1738477600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe940ce2b0 0x7efe941497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe92ffd640 1 -- 172.21.15.60:0/1738477600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe940ce2b0 msgr2=0x7efe941497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe92ffd640 1 --2- 172.21.15.60:0/1738477600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe940ce2b0 0x7efe941497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe92ffd640 1 -- 172.21.15.60:0/1738477600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe9415deb0 msgr2=0x7efe940c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe92ffd640 1 --2- 172.21.15.60:0/1738477600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe9415deb0 0x7efe940c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe92ffd640 1 -- 172.21.15.60:0/1738477600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9415ad10 con 0x7efe940d0340 2024-04-25T10:14:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe93fff640 1 --2- 172.21.15.60:0/1738477600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe9415deb0 0x7efe940c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe92ffd640 1 --2- 172.21.15.60:0/1738477600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe940d0340 0x7efe940c6820 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7efe880b0040 tx=0x7efe88019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.323+0000 7efe7bfff640 1 -- 172.21.15.60:0/1738477600 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe880bb070 con 0x7efe940d0340 2024-04-25T10:14:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.327+0000 7efe7bfff640 1 -- 172.21.15.60:0/1738477600 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe880add00 con 0x7efe940d0340 2024-04-25T10:14:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.327+0000 7efe99f10640 1 -- 172.21.15.60:0/1738477600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9415aa30 con 0x7efe940d0340 2024-04-25T10:14:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.327+0000 7efe7bfff640 1 -- 172.21.15.60:0/1738477600 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe880b2040 con 0x7efe940d0340 2024-04-25T10:14:27.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.327+0000 7efe99f10640 1 -- 172.21.15.60:0/1738477600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe94148ce0 con 0x7efe940d0340 2024-04-25T10:14:27.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.327+0000 7efe99f10640 1 -- 172.21.15.60:0/1738477600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe94074770 con 0x7efe940d0340 2024-04-25T10:14:27.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.327+0000 7efe7bfff640 1 -- 172.21.15.60:0/1738477600 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe880be070 con 0x7efe940d0340 2024-04-25T10:14:27.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.327+0000 7efe7bfff640 1 --2- 172.21.15.60:0/1738477600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efe80066f90 0x7efe80069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.327+0000 7efe7bfff640 1 -- 172.21.15.60:0/1738477600 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe8811df80 con 0x7efe940d0340 2024-04-25T10:14:27.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.327+0000 7efe937fe640 1 --2- 172.21.15.60:0/1738477600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efe80066f90 0x7efe80069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.327+0000 7efe937fe640 1 --2- 172.21.15.60:0/1738477600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efe80066f90 0x7efe80069450 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7efe940b8820 tx=0x7efe84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.331+0000 7efe7bfff640 1 -- 172.21.15.60:0/1738477600 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe88016150 con 0x7efe940d0340 2024-04-25T10:14:27.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.491+0000 7efe99f10640 1 -- 172.21.15.60:0/1738477600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe940d17a0 con 0x7efe940d0340 2024-04-25T10:14:27.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.495+0000 7efe7bfff640 1 -- 172.21.15.60:0/1738477600 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16620 (secure 0 0 0) 0x7efe880eef00 con 0x7efe940d0340 2024-04-25T10:14:27.500 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:27.500 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":97,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:14:17.788631+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:14.444411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:15.153386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:27.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.495+0000 7efe99f10640 1 -- 172.21.15.60:0/1738477600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efe80066f90 msgr2=0x7efe80069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.495+0000 7efe99f10640 1 --2- 172.21.15.60:0/1738477600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efe80066f90 0x7efe80069450 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7efe940b8820 tx=0x7efe84008040 comp rx=0 tx=0).stop 2024-04-25T10:14:27.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.495+0000 7efe99f10640 1 -- 172.21.15.60:0/1738477600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe940d0340 msgr2=0x7efe940c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.495+0000 7efe99f10640 1 --2- 172.21.15.60:0/1738477600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe940d0340 0x7efe940c6820 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7efe880b0040 tx=0x7efe88019300 comp rx=0 tx=0).stop 2024-04-25T10:14:27.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.495+0000 7efe99f10640 1 -- 172.21.15.60:0/1738477600 shutdown_connections 2024-04-25T10:14:27.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.495+0000 7efe99f10640 1 --2- 172.21.15.60:0/1738477600 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7efe80066f90 0x7efe80069450 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.495+0000 7efe99f10640 1 --2- 172.21.15.60:0/1738477600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe9415deb0 0x7efe940c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.495+0000 7efe99f10640 1 --2- 172.21.15.60:0/1738477600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe940d0340 0x7efe940c6820 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.495+0000 7efe99f10640 1 --2- 172.21.15.60:0/1738477600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe940ce2b0 0x7efe941497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.495+0000 7efe99f10640 1 -- 172.21.15.60:0/1738477600 >> 172.21.15.60:0/1738477600 conn(0x7efe9406c090 msgr2=0x7efe940d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:27.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.499+0000 7efe99f10640 1 -- 172.21.15.60:0/1738477600 shutdown_connections 2024-04-25T10:14:27.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.499+0000 7efe99f10640 1 -- 172.21.15.60:0/1738477600 wait complete. 2024-04-25T10:14:27.519 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:14:14.444411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:14:15.153386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:27.519 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:27.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.643+0000 7f2f7df16640 1 Processor -- start 2024-04-25T10:14:27.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.643+0000 7f2f7df16640 1 -- start start 2024-04-25T10:14:27.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.643+0000 7f2f7df16640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2f78160080 0x7f2f7815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.643+0000 7f2f7df16640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2f7815fa20 0x7f2f780ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.643+0000 7f2f7df16640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f78144a40 con 0x7f2f780cbcf0 2024-04-25T10:14:27.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.643+0000 7f2f7df16640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f7815fe90 con 0x7f2f7815fa20 2024-04-25T10:14:27.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.643+0000 7f2f7df16640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f780b2dc0 con 0x7f2f78160080 2024-04-25T10:14:27.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.643+0000 7f2f777fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2f780cbcf0 0x7f2f78071c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55560/0 (socket says 172.21.15.60:55560) 2024-04-25T10:14:27.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.643+0000 7f2f777fe640 1 -- 172.21.15.60:0/28235364 learned_addr learned my addr 172.21.15.60:0/28235364 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:27.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.643+0000 7f2f77fff640 1 --2- 172.21.15.60:0/28235364 >> v2:172.21.15.60:3300/0 conn(0x7f2f7815fa20 0x7f2f780ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.643+0000 7f2f76ffd640 1 --2- 172.21.15.60:0/28235364 >> v2:172.21.15.100:3300/0 conn(0x7f2f78160080 0x7f2f7815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.643+0000 7f2f77fff640 1 -- 172.21.15.60:0/28235364 >> v2:172.21.15.100:3300/0 conn(0x7f2f78160080 msgr2=0x7f2f7815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:27.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.643+0000 7f2f77fff640 1 --2- 172.21.15.60:0/28235364 >> v2:172.21.15.100:3300/0 conn(0x7f2f78160080 0x7f2f7815f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.643+0000 7f2f77fff640 1 -- 172.21.15.60:0/28235364 >> v1:172.21.15.60:6789/0 conn(0x7f2f780cbcf0 legacy=0x7f2f78071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.643+0000 7f2f77fff640 1 -- 172.21.15.60:0/28235364 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f780cc9f0 con 0x7f2f7815fa20 2024-04-25T10:14:27.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.647+0000 7f2f77fff640 1 --2- 172.21.15.60:0/28235364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f7815fa20 0x7f2f780ceec0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f2f780d0570 tx=0x7f2f7001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28682a0d7ba1bd26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.647+0000 7f2f767fc640 1 -- 172.21.15.60:0/28235364 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f700b2070 con 0x7f2f7815fa20 2024-04-25T10:14:27.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.647+0000 7f2f767fc640 1 -- 172.21.15.60:0/28235364 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f700a58e0 con 0x7f2f7815fa20 2024-04-25T10:14:27.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.647+0000 7f2f767fc640 1 -- 172.21.15.60:0/28235364 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f700a5cb0 con 0x7f2f7815fa20 2024-04-25T10:14:27.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.647+0000 7f2f7df16640 1 -- 172.21.15.60:0/28235364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f7815fa20 msgr2=0x7f2f780ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.647+0000 7f2f7df16640 1 --2- 172.21.15.60:0/28235364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f7815fa20 0x7f2f780ceec0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f2f780d0570 tx=0x7f2f7001c930 comp rx=0 tx=0).stop 2024-04-25T10:14:27.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.647+0000 7f2f7df16640 1 -- 172.21.15.60:0/28235364 shutdown_connections 2024-04-25T10:14:27.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.647+0000 7f2f7df16640 1 --2- 172.21.15.60:0/28235364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f7815fa20 0x7f2f780ceec0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.647+0000 7f2f7df16640 1 --2- 172.21.15.60:0/28235364 >> v2:172.21.15.100:3300/0 conn(0x7f2f78160080 0x7f2f7815f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.647+0000 7f2f7df16640 1 -- 172.21.15.60:0/28235364 >> 172.21.15.60:0/28235364 conn(0x7f2f780670b0 msgr2=0x7f2f7806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:27.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.647+0000 7f2f7df16640 1 -- 172.21.15.60:0/28235364 shutdown_connections 2024-04-25T10:14:27.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.647+0000 7f2f7df16640 1 -- 172.21.15.60:0/28235364 wait complete. 2024-04-25T10:14:27.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.647+0000 7f2f7df16640 1 Processor -- start 2024-04-25T10:14:27.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.647+0000 7f2f7df16640 1 -- start start 2024-04-25T10:14:27.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f7df16640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f780cbcf0 0x7f2f780b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f7df16640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7815fa20 0x7f2f78140810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f7df16640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f78160080 0x7f2f78158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f7df16640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780ce5c0 con 0x7f2f78160080 2024-04-25T10:14:27.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f7df16640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780cc0d0 con 0x7f2f7815fa20 2024-04-25T10:14:27.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f7df16640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780ce740 con 0x7f2f780cbcf0 2024-04-25T10:14:27.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f77fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f78160080 0x7f2f78158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f76ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7815fa20 0x7f2f78140810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f76ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7815fa20 0x7f2f78140810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37152/0 (socket says 172.21.15.60:37152) 2024-04-25T10:14:27.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f77fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f78160080 0x7f2f78158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47626/0 (socket says 172.21.15.60:47626) 2024-04-25T10:14:27.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f76ffd640 1 -- 172.21.15.60:0/1051504331 learned_addr learned my addr 172.21.15.60:0/1051504331 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:27.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f777fe640 1 --2- 172.21.15.60:0/1051504331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f780cbcf0 0x7f2f780b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f77fff640 1 -- 172.21.15.60:0/1051504331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f780cbcf0 msgr2=0x7f2f780b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f77fff640 1 --2- 172.21.15.60:0/1051504331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f780cbcf0 0x7f2f780b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f77fff640 1 -- 172.21.15.60:0/1051504331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7815fa20 msgr2=0x7f2f78140810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f77fff640 1 --2- 172.21.15.60:0/1051504331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7815fa20 0x7f2f78140810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f77fff640 1 -- 172.21.15.60:0/1051504331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f78159de0 con 0x7f2f78160080 2024-04-25T10:14:27.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f77fff640 1 --2- 172.21.15.60:0/1051504331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f78160080 0x7f2f78158c60 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f2f700ac000 tx=0x7f2f700196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f5ffff640 1 -- 172.21.15.60:0/1051504331 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f700b2050 con 0x7f2f78160080 2024-04-25T10:14:27.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f5ffff640 1 -- 172.21.15.60:0/1051504331 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f700be070 con 0x7f2f78160080 2024-04-25T10:14:27.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f7df16640 1 -- 172.21.15.60:0/1051504331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7815a0c0 con 0x7f2f78160080 2024-04-25T10:14:27.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f5ffff640 1 -- 172.21.15.60:0/1051504331 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f700ada90 con 0x7f2f78160080 2024-04-25T10:14:27.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f7df16640 1 -- 172.21.15.60:0/1051504331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7815a680 con 0x7f2f78160080 2024-04-25T10:14:27.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f7df16640 1 -- 172.21.15.60:0/1051504331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f7806bba0 con 0x7f2f78160080 2024-04-25T10:14:27.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f5ffff640 1 -- 172.21.15.60:0/1051504331 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f700adc30 con 0x7f2f78160080 2024-04-25T10:14:27.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f5ffff640 1 --2- 172.21.15.60:0/1051504331 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2f64066f60 0x7f2f64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.651+0000 7f2f5ffff640 1 -- 172.21.15.60:0/1051504331 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f70120240 con 0x7f2f78160080 2024-04-25T10:14:27.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.655+0000 7f2f777fe640 1 --2- 172.21.15.60:0/1051504331 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2f64066f60 0x7f2f64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.655+0000 7f2f777fe640 1 --2- 172.21.15.60:0/1051504331 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2f64066f60 0x7f2f64069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c000f70 tx=0x7f2f6c007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.659+0000 7f2f5ffff640 1 -- 172.21.15.60:0/1051504331 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f70016520 con 0x7f2f78160080 2024-04-25T10:14:27.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.819+0000 7f2f7df16640 1 -- 172.21.15.60:0/1051504331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f780b2970 con 0x7f2f78160080 2024-04-25T10:14:27.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.823+0000 7f2f5ffff640 1 -- 172.21.15.60:0/1051504331 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f700f0b70 con 0x7f2f78160080 2024-04-25T10:14:27.826 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:14:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.823+0000 7f2f7df16640 1 -- 172.21.15.60:0/1051504331 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2f64066f60 msgr2=0x7f2f64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.823+0000 7f2f7df16640 1 --2- 172.21.15.60:0/1051504331 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2f64066f60 0x7f2f64069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c000f70 tx=0x7f2f6c007e90 comp rx=0 tx=0).stop 2024-04-25T10:14:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.823+0000 7f2f7df16640 1 -- 172.21.15.60:0/1051504331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f78160080 msgr2=0x7f2f78158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.823+0000 7f2f7df16640 1 --2- 172.21.15.60:0/1051504331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f78160080 0x7f2f78158c60 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f2f700ac000 tx=0x7f2f700196d0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.823+0000 7f2f7df16640 1 -- 172.21.15.60:0/1051504331 shutdown_connections 2024-04-25T10:14:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.823+0000 7f2f7df16640 1 --2- 172.21.15.60:0/1051504331 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2f64066f60 0x7f2f64069420 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.823+0000 7f2f7df16640 1 --2- 172.21.15.60:0/1051504331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f78160080 0x7f2f78158c60 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.823+0000 7f2f7df16640 1 --2- 172.21.15.60:0/1051504331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7815fa20 0x7f2f78140810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.823+0000 7f2f7df16640 1 --2- 172.21.15.60:0/1051504331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f780cbcf0 0x7f2f780b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.823+0000 7f2f7df16640 1 -- 172.21.15.60:0/1051504331 >> 172.21.15.60:0/1051504331 conn(0x7f2f780670b0 msgr2=0x7f2f7815d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.823+0000 7f2f7df16640 1 -- 172.21.15.60:0/1051504331 shutdown_connections 2024-04-25T10:14:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.823+0000 7f2f7df16640 1 -- 172.21.15.60:0/1051504331 wait complete. 2024-04-25T10:14:27.846 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.959+0000 7fdf68b53640 1 Processor -- start 2024-04-25T10:14:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.959+0000 7fdf68b53640 1 -- start start 2024-04-25T10:14:27.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.963+0000 7fdf68b53640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdf640cbcf0 0x7fdf64160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.963+0000 7fdf68b53640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf64144a40 con 0x7fdf6415fbf0 2024-04-25T10:14:27.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.963+0000 7fdf68b53640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf640cc9f0 con 0x7fdf64071c30 2024-04-25T10:14:27.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.963+0000 7fdf68b53640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf640b2dc0 con 0x7fdf640cbcf0 2024-04-25T10:14:27.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.963+0000 7fdf6259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdf640cbcf0 0x7fdf64160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.963+0000 7fdf62d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdf6415fbf0 0x7fdf640ceec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55572/0 (socket says 172.21.15.60:55572) 2024-04-25T10:14:27.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.963+0000 7fdf62d9d640 1 -- 172.21.15.60:0/4117366783 learned_addr learned my addr 172.21.15.60:0/4117366783 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:27.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.963+0000 7fdf6259c640 1 -- 172.21.15.60:0/4117366783 >> v1:172.21.15.100:6789/0 conn(0x7fdf64071c30 legacy=0x7fdf6415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:27.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.963+0000 7fdf6259c640 1 -- 172.21.15.60:0/4117366783 >> v1:172.21.15.60:6789/0 conn(0x7fdf6415fbf0 legacy=0x7fdf640ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.963+0000 7fdf6159a640 1 -- 172.21.15.60:0/4117366783 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999326815 0 0) 0x7fdf64144a40 con 0x7fdf6415fbf0 2024-04-25T10:14:27.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.963+0000 7fdf6259c640 1 -- 172.21.15.60:0/4117366783 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf640cccd0 con 0x7fdf640cbcf0 2024-04-25T10:14:27.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.963+0000 7fdf6259c640 1 --2- 172.21.15.60:0/4117366783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf640cbcf0 0x7fdf64160080 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fdf540097c0 tx=0x7fdf5401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5792edcf802b4ce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.963+0000 7fdf6159a640 1 -- 172.21.15.60:0/4117366783 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf540ab430 con 0x7fdf640cbcf0 2024-04-25T10:14:27.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.963+0000 7fdf6159a640 1 -- 172.21.15.60:0/4117366783 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf540b0070 con 0x7fdf640cbcf0 2024-04-25T10:14:27.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.963+0000 7fdf6159a640 1 -- 172.21.15.60:0/4117366783 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf540ab7f0 con 0x7fdf640cbcf0 2024-04-25T10:14:27.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 -- 172.21.15.60:0/4117366783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf640cbcf0 msgr2=0x7fdf64160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 --2- 172.21.15.60:0/4117366783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf640cbcf0 0x7fdf64160080 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fdf540097c0 tx=0x7fdf5401c430 comp rx=0 tx=0).stop 2024-04-25T10:14:27.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 -- 172.21.15.60:0/4117366783 shutdown_connections 2024-04-25T10:14:27.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 --2- 172.21.15.60:0/4117366783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf640cbcf0 0x7fdf64160080 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 -- 172.21.15.60:0/4117366783 >> 172.21.15.60:0/4117366783 conn(0x7fdf640670b0 msgr2=0x7fdf6406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 -- 172.21.15.60:0/4117366783 shutdown_connections 2024-04-25T10:14:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 -- 172.21.15.60:0/4117366783 wait complete. 2024-04-25T10:14:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 Processor -- start 2024-04-25T10:14:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 -- start start 2024-04-25T10:14:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf64071c30 0x7fdf6412fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf640cbcf0 0x7fdf641f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf6415fbf0 0x7fdf640b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640ce5c0 con 0x7fdf6415fbf0 2024-04-25T10:14:27.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640720b0 con 0x7fdf64071c30 2024-04-25T10:14:27.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640ce740 con 0x7fdf640cbcf0 2024-04-25T10:14:27.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf62d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf6415fbf0 0x7fdf640b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf62d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf6415fbf0 0x7fdf640b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47642/0 (socket says 172.21.15.60:47642) 2024-04-25T10:14:27.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf62d9d640 1 -- 172.21.15.60:0/1377441479 learned_addr learned my addr 172.21.15.60:0/1377441479 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:27.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf62d9d640 1 -- 172.21.15.60:0/1377441479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf640cbcf0 msgr2=0x7fdf641f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf62d9d640 1 --2- 172.21.15.60:0/1377441479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf640cbcf0 0x7fdf641f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf62d9d640 1 -- 172.21.15.60:0/1377441479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf64071c30 msgr2=0x7fdf6412fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf62d9d640 1 --2- 172.21.15.60:0/1377441479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf64071c30 0x7fdf6412fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf62d9d640 1 -- 172.21.15.60:0/1377441479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf640ad190 con 0x7fdf6415fbf0 2024-04-25T10:14:27.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf62d9d640 1 --2- 172.21.15.60:0/1377441479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf6415fbf0 0x7fdf640b1b40 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00c700 tx=0x7fdf5c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf52ffd640 1 -- 172.21.15.60:0/1377441479 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c018070 con 0x7fdf6415fbf0 2024-04-25T10:14:27.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf52ffd640 1 -- 172.21.15.60:0/1377441479 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5c00ce10 con 0x7fdf6415fbf0 2024-04-25T10:14:27.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf52ffd640 1 -- 172.21.15.60:0/1377441479 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c00a860 con 0x7fdf6415fbf0 2024-04-25T10:14:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 -- 172.21.15.60:0/1377441479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf640ad470 con 0x7fdf6415fbf0 2024-04-25T10:14:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 -- 172.21.15.60:0/1377441479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf640ada30 con 0x7fdf6415fbf0 2024-04-25T10:14:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.967+0000 7fdf68b53640 1 -- 172.21.15.60:0/1377441479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf6406bba0 con 0x7fdf6415fbf0 2024-04-25T10:14:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.971+0000 7fdf52ffd640 1 -- 172.21.15.60:0/1377441479 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf5c01b070 con 0x7fdf6415fbf0 2024-04-25T10:14:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.971+0000 7fdf52ffd640 1 --2- 172.21.15.60:0/1377441479 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf44066f60 0x7fdf44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.971+0000 7fdf52ffd640 1 -- 172.21.15.60:0/1377441479 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf5c084540 con 0x7fdf6415fbf0 2024-04-25T10:14:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.971+0000 7fdf6259c640 1 --2- 172.21.15.60:0/1377441479 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf44066f60 0x7fdf44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.971+0000 7fdf6259c640 1 --2- 172.21.15.60:0/1377441479 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf44066f60 0x7fdf44069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdf540af000 tx=0x7fdf54004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:27.975+0000 7fdf52ffd640 1 -- 172.21.15.60:0/1377441479 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf5c0555e0 con 0x7fdf6415fbf0 2024-04-25T10:14:28.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.127+0000 7fdf68b53640 1 -- 172.21.15.60:0/1377441479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf64074770 con 0x7fdf6415fbf0 2024-04-25T10:14:28.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.127+0000 7fdf52ffd640 1 -- 172.21.15.60:0/1377441479 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf5c054ee0 con 0x7fdf6415fbf0 2024-04-25T10:14:28.131 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:14:28.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.127+0000 7fdf68b53640 1 -- 172.21.15.60:0/1377441479 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf44066f60 msgr2=0x7fdf44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.127+0000 7fdf68b53640 1 --2- 172.21.15.60:0/1377441479 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf44066f60 0x7fdf44069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdf540af000 tx=0x7fdf54004740 comp rx=0 tx=0).stop 2024-04-25T10:14:28.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.127+0000 7fdf68b53640 1 -- 172.21.15.60:0/1377441479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf6415fbf0 msgr2=0x7fdf640b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.127+0000 7fdf68b53640 1 --2- 172.21.15.60:0/1377441479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf6415fbf0 0x7fdf640b1b40 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00c700 tx=0x7fdf5c00cbc0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.127+0000 7fdf68b53640 1 -- 172.21.15.60:0/1377441479 shutdown_connections 2024-04-25T10:14:28.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.127+0000 7fdf68b53640 1 --2- 172.21.15.60:0/1377441479 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdf44066f60 0x7fdf44069420 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.127+0000 7fdf68b53640 1 --2- 172.21.15.60:0/1377441479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf6415fbf0 0x7fdf640b1b40 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.127+0000 7fdf68b53640 1 --2- 172.21.15.60:0/1377441479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf640cbcf0 0x7fdf641f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.127+0000 7fdf68b53640 1 --2- 172.21.15.60:0/1377441479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf64071c30 0x7fdf6412fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.127+0000 7fdf68b53640 1 -- 172.21.15.60:0/1377441479 >> 172.21.15.60:0/1377441479 conn(0x7fdf640670b0 msgr2=0x7fdf64071490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.127+0000 7fdf68b53640 1 -- 172.21.15.60:0/1377441479 shutdown_connections 2024-04-25T10:14:28.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.127+0000 7fdf68b53640 1 -- 172.21.15.60:0/1377441479 wait complete. 2024-04-25T10:14:28.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:28.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.271+0000 7f0124719640 1 Processor -- start 2024-04-25T10:14:28.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.275+0000 7f0124719640 1 -- start start 2024-04-25T10:14:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.275+0000 7f0124719640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f011c0718a0 0x7f011c071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.275+0000 7f0124719640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011c15ce20 con 0x7f011c15bea0 2024-04-25T10:14:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.275+0000 7f0124719640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011c15c580 con 0x7f011c0cefe0 2024-04-25T10:14:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.275+0000 7f0124719640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011c15c350 con 0x7f011c0718a0 2024-04-25T10:14:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.275+0000 7f0122cb6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f011c0718a0 0x7f011c071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.275+0000 7f0121cb4640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f011c15bea0 0x7f011c0b30a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55584/0 (socket says 172.21.15.60:55584) 2024-04-25T10:14:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.275+0000 7f0122cb6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f011c0718a0 0x7f011c071cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47650/0 (socket says 172.21.15.60:47650) 2024-04-25T10:14:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.275+0000 7f0121cb4640 1 -- 172.21.15.60:0/3280136051 learned_addr learned my addr 172.21.15.60:0/3280136051 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.275+0000 7f0122cb6640 1 -- 172.21.15.60:0/3280136051 >> v1:172.21.15.100:6789/0 conn(0x7f011c0cefe0 legacy=0x7f011c0cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.275+0000 7f0122cb6640 1 -- 172.21.15.60:0/3280136051 >> v1:172.21.15.60:6789/0 conn(0x7f011c15bea0 legacy=0x7f011c0b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.275+0000 7f0122cb6640 1 -- 172.21.15.60:0/3280136051 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c15c860 con 0x7f011c0718a0 2024-04-25T10:14:28.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.275+0000 7f01214b3640 1 -- 172.21.15.60:0/3280136051 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655768375 0 0) 0x7f011c15ce20 con 0x7f011c15bea0 2024-04-25T10:14:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0122cb6640 1 --2- 172.21.15.60:0/3280136051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f011c0718a0 0x7f011c071cb0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f0118009d90 tx=0x7f011801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31946e75496ec49f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f01214b3640 1 -- 172.21.15.60:0/3280136051 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01180b2070 con 0x7f011c0718a0 2024-04-25T10:14:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f01214b3640 1 -- 172.21.15.60:0/3280136051 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01180a57a0 con 0x7f011c0718a0 2024-04-25T10:14:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f01214b3640 1 -- 172.21.15.60:0/3280136051 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01180a5b70 con 0x7f011c0718a0 2024-04-25T10:14:28.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0124719640 1 -- 172.21.15.60:0/3280136051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f011c0718a0 msgr2=0x7f011c071cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0124719640 1 --2- 172.21.15.60:0/3280136051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f011c0718a0 0x7f011c071cb0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f0118009d90 tx=0x7f011801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0124719640 1 -- 172.21.15.60:0/3280136051 shutdown_connections 2024-04-25T10:14:28.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0124719640 1 --2- 172.21.15.60:0/3280136051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f011c0718a0 0x7f011c071cb0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0124719640 1 -- 172.21.15.60:0/3280136051 >> 172.21.15.60:0/3280136051 conn(0x7f011c0670b0 msgr2=0x7f011c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0124719640 1 -- 172.21.15.60:0/3280136051 shutdown_connections 2024-04-25T10:14:28.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0124719640 1 -- 172.21.15.60:0/3280136051 wait complete. 2024-04-25T10:14:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0124719640 1 Processor -- start 2024-04-25T10:14:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0124719640 1 -- start start 2024-04-25T10:14:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0124719640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f011c0718a0 0x7f011c1544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0124719640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f011c0cefe0 0x7f011c14d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0124719640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f011c15bea0 0x7f011c14d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0124719640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c169a30 con 0x7f011c15bea0 2024-04-25T10:14:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0124719640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c071410 con 0x7f011c0718a0 2024-04-25T10:14:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0124719640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011c169bb0 con 0x7f011c0cefe0 2024-04-25T10:14:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0122cb6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f011c15bea0 0x7f011c14d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0122cb6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f011c15bea0 0x7f011c14d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47658/0 (socket says 172.21.15.60:47658) 2024-04-25T10:14:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0122cb6640 1 -- 172.21.15.60:0/746766820 learned_addr learned my addr 172.21.15.60:0/746766820 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:28.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f01224b5640 1 --2- 172.21.15.60:0/746766820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f011c0718a0 0x7f011c1544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0121cb4640 1 --2- 172.21.15.60:0/746766820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f011c0cefe0 0x7f011c14d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0122cb6640 1 -- 172.21.15.60:0/746766820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f011c0cefe0 msgr2=0x7f011c14d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0122cb6640 1 --2- 172.21.15.60:0/746766820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f011c0cefe0 0x7f011c14d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0122cb6640 1 -- 172.21.15.60:0/746766820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f011c0718a0 msgr2=0x7f011c1544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0122cb6640 1 --2- 172.21.15.60:0/746766820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f011c0718a0 0x7f011c1544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0122cb6640 1 -- 172.21.15.60:0/746766820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c146020 con 0x7f011c15bea0 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0121cb4640 1 --2- 172.21.15.60:0/746766820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f011c0cefe0 0x7f011c14d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.279+0000 7f0122cb6640 1 --2- 172.21.15.60:0/746766820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f011c15bea0 0x7f011c14d630 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f01180b0040 tx=0x7f0118019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.283+0000 7f0112ffd640 1 -- 172.21.15.60:0/746766820 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01180b2050 con 0x7f011c15bea0 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.283+0000 7f0112ffd640 1 -- 172.21.15.60:0/746766820 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01180be070 con 0x7f011c15bea0 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.283+0000 7f0124719640 1 -- 172.21.15.60:0/746766820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f011c146300 con 0x7f011c15bea0 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.283+0000 7f0112ffd640 1 -- 172.21.15.60:0/746766820 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01180ada50 con 0x7f011c15bea0 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.283+0000 7f0124719640 1 -- 172.21.15.60:0/746766820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011c1468c0 con 0x7f011c15bea0 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.283+0000 7f0124719640 1 -- 172.21.15.60:0/746766820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011c074770 con 0x7f011c15bea0 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.283+0000 7f0112ffd640 1 -- 172.21.15.60:0/746766820 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01180adbf0 con 0x7f011c15bea0 2024-04-25T10:14:28.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.283+0000 7f0112ffd640 1 --2- 172.21.15.60:0/746766820 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0100066f60 0x7f0100069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.283+0000 7f0112ffd640 1 -- 172.21.15.60:0/746766820 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01181201c0 con 0x7f011c15bea0 2024-04-25T10:14:28.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.283+0000 7f01224b5640 1 --2- 172.21.15.60:0/746766820 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0100066f60 0x7f0100069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.283+0000 7f01224b5640 1 --2- 172.21.15.60:0/746766820 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0100066f60 0x7f0100069420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0114000f70 tx=0x7f0114009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.287+0000 7f0112ffd640 1 -- 172.21.15.60:0/746766820 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0118016560 con 0x7f011c15bea0 2024-04-25T10:14:28.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.447+0000 7f0124719640 1 -- 172.21.15.60:0/746766820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f011c0b2970 con 0x7f011c15bea0 2024-04-25T10:14:28.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.447+0000 7f0112ffd640 1 -- 172.21.15.60:0/746766820 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+11 (secure 0 0 0) 0x7f01180f11d0 con 0x7f011c15bea0 2024-04-25T10:14:28.453 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:14:28.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.451+0000 7f0124719640 1 -- 172.21.15.60:0/746766820 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0100066f60 msgr2=0x7f0100069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.451+0000 7f0124719640 1 --2- 172.21.15.60:0/746766820 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0100066f60 0x7f0100069420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0114000f70 tx=0x7f0114009040 comp rx=0 tx=0).stop 2024-04-25T10:14:28.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.451+0000 7f0124719640 1 -- 172.21.15.60:0/746766820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f011c15bea0 msgr2=0x7f011c14d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.451+0000 7f0124719640 1 --2- 172.21.15.60:0/746766820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f011c15bea0 0x7f011c14d630 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f01180b0040 tx=0x7f0118019680 comp rx=0 tx=0).stop 2024-04-25T10:14:28.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.451+0000 7f0124719640 1 -- 172.21.15.60:0/746766820 shutdown_connections 2024-04-25T10:14:28.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.451+0000 7f0124719640 1 --2- 172.21.15.60:0/746766820 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0100066f60 0x7f0100069420 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.451+0000 7f0124719640 1 --2- 172.21.15.60:0/746766820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f011c15bea0 0x7f011c14d630 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.451+0000 7f0124719640 1 --2- 172.21.15.60:0/746766820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f011c0cefe0 0x7f011c14d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.451+0000 7f0124719640 1 --2- 172.21.15.60:0/746766820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f011c0718a0 0x7f011c1544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.451+0000 7f0124719640 1 -- 172.21.15.60:0/746766820 >> 172.21.15.60:0/746766820 conn(0x7f011c0670b0 msgr2=0x7f011c0b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.451+0000 7f0124719640 1 -- 172.21.15.60:0/746766820 shutdown_connections 2024-04-25T10:14:28.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.451+0000 7f0124719640 1 -- 172.21.15.60:0/746766820 wait complete. 2024-04-25T10:14:28.473 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:14:28.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382891c640 1 Processor -- start 2024-04-25T10:14:28.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382891c640 1 -- start start 2024-04-25T10:14:28.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382891c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f382415bc90 0x7f382415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382891c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3824071ca0 con 0x7f38240ce7a0 2024-04-25T10:14:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382891c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38240cca40 con 0x7f38240b4660 2024-04-25T10:14:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382891c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38240cc370 con 0x7f382415bc90 2024-04-25T10:14:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382259c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f38240ce7a0 0x7f38240ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55594/0 (socket says 172.21.15.60:55594) 2024-04-25T10:14:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382259c640 1 -- 172.21.15.60:0/3697156320 learned_addr learned my addr 172.21.15.60:0/3697156320 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382159a640 1 -- 172.21.15.60:0/3697156320 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3882255421 0 0) 0x7f3824071ca0 con 0x7f38240ce7a0 2024-04-25T10:14:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f3821d9b640 1 --2- 172.21.15.60:0/3697156320 >> v2:172.21.15.100:3300/0 conn(0x7f382415bc90 0x7f382415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382159a640 1 -- 172.21.15.60:0/3697156320 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f380c003530 con 0x7f38240ce7a0 2024-04-25T10:14:28.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382159a640 1 -- 172.21.15.60:0/3697156320 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3421021976 0 0) 0x7f380c003530 con 0x7f38240ce7a0 2024-04-25T10:14:28.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382159a640 1 -- 172.21.15.60:0/3697156320 >> v2:172.21.15.100:3300/0 conn(0x7f382415bc90 msgr2=0x7f382415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382159a640 1 --2- 172.21.15.60:0/3697156320 >> v2:172.21.15.100:3300/0 conn(0x7f382415bc90 0x7f382415c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382159a640 1 -- 172.21.15.60:0/3697156320 >> v1:172.21.15.100:6789/0 conn(0x7f38240b4660 legacy=0x7f382415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382159a640 1 -- 172.21.15.60:0/3697156320 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38240ccd20 con 0x7f38240ce7a0 2024-04-25T10:14:28.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382159a640 1 -- 172.21.15.60:0/3697156320 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f3814001680 con 0x7f38240ce7a0 2024-04-25T10:14:28.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382159a640 1 -- 172.21.15.60:0/3697156320 >> v1:172.21.15.60:6789/0 conn(0x7f38240ce7a0 legacy=0x7f38240ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382159a640 1 --2- 172.21.15.60:0/3697156320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f380c0039c0 0x7f380c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382159a640 1 --2- 172.21.15.60:0/3697156320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f380c004810 0x7f380c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382159a640 1 --2- 172.21.15.60:0/3697156320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f380c0051d0 0x7f380c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382159a640 1 -- 172.21.15.60:0/3697156320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c00c040 con 0x7f380c0051d0 2024-04-25T10:14:28.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382159a640 1 -- 172.21.15.60:0/3697156320 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c00c1f0 con 0x7f380c0039c0 2024-04-25T10:14:28.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f382159a640 1 -- 172.21.15.60:0/3697156320 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380c00c3f0 con 0x7f380c004810 2024-04-25T10:14:28.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.591+0000 7f3822d9d640 1 --2- 172.21.15.60:0/3697156320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f380c0051d0 0x7f380c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f382259c640 1 --2- 172.21.15.60:0/3697156320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f380c0039c0 0x7f380c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f3821d9b640 1 --2- 172.21.15.60:0/3697156320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f380c004810 0x7f380c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f3822d9d640 1 -- 172.21.15.60:0/3697156320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f380c004810 msgr2=0x7f380c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f3822d9d640 1 --2- 172.21.15.60:0/3697156320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f380c004810 0x7f380c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f3822d9d640 1 -- 172.21.15.60:0/3697156320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f380c0039c0 msgr2=0x7f380c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f3822d9d640 1 --2- 172.21.15.60:0/3697156320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f380c0039c0 0x7f380c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f3822d9d640 1 -- 172.21.15.60:0/3697156320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c003530 con 0x7f380c0051d0 2024-04-25T10:14:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f382259c640 1 --2- 172.21.15.60:0/3697156320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f380c0039c0 0x7f380c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f3821d9b640 1 --2- 172.21.15.60:0/3697156320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f380c004810 0x7f380c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:28.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f3822d9d640 1 -- 172.21.15.60:0/3697156320 reap_dead start 2024-04-25T10:14:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f3822d9d640 1 --2- 172.21.15.60:0/3697156320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f380c0051d0 0x7f380c00b9d0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f381c001e40 tx=0x7f381c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89a270dbe8e3e98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f382159a640 1 -- 172.21.15.60:0/3697156320 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c001a90 con 0x7f380c0051d0 2024-04-25T10:14:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f382159a640 1 -- 172.21.15.60:0/3697156320 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381c003b80 con 0x7f380c0051d0 2024-04-25T10:14:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f382159a640 1 -- 172.21.15.60:0/3697156320 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c005020 con 0x7f380c0051d0 2024-04-25T10:14:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f382891c640 1 -- 172.21.15.60:0/3697156320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f380c0051d0 msgr2=0x7f380c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f382891c640 1 --2- 172.21.15.60:0/3697156320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f380c0051d0 0x7f380c00b9d0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f381c001e40 tx=0x7f381c01c760 comp rx=0 tx=0).stop 2024-04-25T10:14:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f382891c640 1 -- 172.21.15.60:0/3697156320 shutdown_connections 2024-04-25T10:14:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f382891c640 1 --2- 172.21.15.60:0/3697156320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f380c0051d0 0x7f380c00b9d0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f382891c640 1 -- 172.21.15.60:0/3697156320 >> 172.21.15.60:0/3697156320 conn(0x7f38240b12d0 msgr2=0x7f38240b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f382891c640 1 -- 172.21.15.60:0/3697156320 shutdown_connections 2024-04-25T10:14:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.595+0000 7f382891c640 1 -- 172.21.15.60:0/3697156320 wait complete. 2024-04-25T10:14:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f382891c640 1 Processor -- start 2024-04-25T10:14:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f382891c640 1 -- start start 2024-04-25T10:14:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f382891c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f380c0051d0 0x7f382415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f382891c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38241e6f70 0x7f3824150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f382891c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38241e57b0 0x7f38241e5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f382891c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38240b4400 con 0x7f38241e6f70 2024-04-25T10:14:28.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f382891c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38240b4280 con 0x7f38241e57b0 2024-04-25T10:14:28.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f382891c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38240b3ba0 con 0x7f380c0051d0 2024-04-25T10:14:28.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f3821d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38241e6f70 0x7f3824150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f3821d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38241e6f70 0x7f3824150980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47682/0 (socket says 172.21.15.60:47682) 2024-04-25T10:14:28.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f3821d9b640 1 -- 172.21.15.60:0/192771262 learned_addr learned my addr 172.21.15.60:0/192771262 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:28.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f3821d9b640 1 -- 172.21.15.60:0/192771262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f380c0051d0 msgr2=0x7f382415ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f3821d9b640 1 --2- 172.21.15.60:0/192771262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f380c0051d0 0x7f382415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f3822d9d640 1 --2- 172.21.15.60:0/192771262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38241e57b0 0x7f38241e5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f3821d9b640 1 -- 172.21.15.60:0/192771262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38241e57b0 msgr2=0x7f38241e5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f3821d9b640 1 --2- 172.21.15.60:0/192771262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38241e57b0 0x7f38241e5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f3821d9b640 1 -- 172.21.15.60:0/192771262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38241e73f0 con 0x7f38241e6f70 2024-04-25T10:14:28.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f3822d9d640 1 --2- 172.21.15.60:0/192771262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38241e57b0 0x7f38241e5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:28.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f3821d9b640 1 --2- 172.21.15.60:0/192771262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38241e6f70 0x7f3824150980 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f3818010900 tx=0x7f3818010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f3812ffd640 1 -- 172.21.15.60:0/192771262 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381800f040 con 0x7f38241e6f70 2024-04-25T10:14:28.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f3812ffd640 1 -- 172.21.15.60:0/192771262 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381800ec20 con 0x7f38241e6f70 2024-04-25T10:14:28.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f382891c640 1 -- 172.21.15.60:0/192771262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38241e8760 con 0x7f38241e6f70 2024-04-25T10:14:28.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f3812ffd640 1 -- 172.21.15.60:0/192771262 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381801c400 con 0x7f38241e6f70 2024-04-25T10:14:28.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f382891c640 1 -- 172.21.15.60:0/192771262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38241e79a0 con 0x7f38241e6f70 2024-04-25T10:14:28.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f382891c640 1 -- 172.21.15.60:0/192771262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3824074770 con 0x7f38241e6f70 2024-04-25T10:14:28.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.599+0000 7f3812ffd640 1 -- 172.21.15.60:0/192771262 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f381801c5a0 con 0x7f38241e6f70 2024-04-25T10:14:28.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.603+0000 7f3812ffd640 1 --2- 172.21.15.60:0/192771262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3804066f60 0x7f3804069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.603+0000 7f3812ffd640 1 -- 172.21.15.60:0/192771262 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3818085830 con 0x7f38241e6f70 2024-04-25T10:14:28.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.603+0000 7f382259c640 1 --2- 172.21.15.60:0/192771262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3804066f60 0x7f3804069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.603+0000 7f382259c640 1 --2- 172.21.15.60:0/192771262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3804066f60 0x7f3804069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3814001ec0 tx=0x7f3814000ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.607+0000 7f3812ffd640 1 -- 172.21.15.60:0/192771262 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3818018030 con 0x7f38241e6f70 2024-04-25T10:14:28.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.775+0000 7f382891c640 1 -- 172.21.15.60:0/192771262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f38240afec0 con 0x7f38241e6f70 2024-04-25T10:14:28.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.775+0000 7f3812ffd640 1 -- 172.21.15.60:0/192771262 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v97) v1 ==== 80+0+0 (secure 0 0 0) 0x7f38180567a0 con 0x7f38241e6f70 2024-04-25T10:14:28.780 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:14:28.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.775+0000 7f382891c640 1 -- 172.21.15.60:0/192771262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3804066f60 msgr2=0x7f3804069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.775+0000 7f382891c640 1 --2- 172.21.15.60:0/192771262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3804066f60 0x7f3804069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3814001ec0 tx=0x7f3814000ee0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.775+0000 7f382891c640 1 -- 172.21.15.60:0/192771262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38241e6f70 msgr2=0x7f3824150980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.775+0000 7f382891c640 1 --2- 172.21.15.60:0/192771262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38241e6f70 0x7f3824150980 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f3818010900 tx=0x7f3818010dc0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.775+0000 7f382891c640 1 -- 172.21.15.60:0/192771262 shutdown_connections 2024-04-25T10:14:28.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.775+0000 7f382891c640 1 --2- 172.21.15.60:0/192771262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3804066f60 0x7f3804069420 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.775+0000 7f382891c640 1 --2- 172.21.15.60:0/192771262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38241e57b0 0x7f38241e5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.775+0000 7f382891c640 1 --2- 172.21.15.60:0/192771262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38241e6f70 0x7f3824150980 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.775+0000 7f382891c640 1 --2- 172.21.15.60:0/192771262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f380c0051d0 0x7f382415ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.779+0000 7f382891c640 1 -- 172.21.15.60:0/192771262 >> 172.21.15.60:0/192771262 conn(0x7f38240b12d0 msgr2=0x7f382415b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.779+0000 7f382891c640 1 -- 172.21.15.60:0/192771262 shutdown_connections 2024-04-25T10:14:28.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:28.779+0000 7f382891c640 1 -- 172.21.15.60:0/192771262 wait complete. 2024-04-25T10:14:28.798 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:14:28.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:28.799 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:28.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[48912]: fuse finished with error 0 and tester_r 0 2024-04-25T10:14:28.811 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:28.876 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:14:28.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:28.877 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:14:28.882 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:29.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.043+0000 7fe8a9a38640 1 Processor -- start 2024-04-25T10:14:29.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a9a38640 1 -- start start 2024-04-25T10:14:29.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a9a38640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe8a40ce7a0 0x7fe8a40ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a9a38640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe8a415bc90 0x7fe8a40b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a9a38640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a40cf0c0 con 0x7fe8a40b4660 2024-04-25T10:14:29.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a9a38640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a40cc370 con 0x7fe8a415bc90 2024-04-25T10:14:29.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a9a38640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a4071ca0 con 0x7fe8a40ce7a0 2024-04-25T10:14:29.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a27fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe8a415bc90 0x7fe8a40b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a27fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe8a415bc90 0x7fe8a40b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40464/0 (socket says 172.21.15.60:40464) 2024-04-25T10:14:29.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a27fc640 1 -- 172.21.15.60:0/3857327094 learned_addr learned my addr 172.21.15.60:0/3857327094 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:29.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe8a40b4660 0x7fe8a40b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53018/0 (socket says 172.21.15.60:53018) 2024-04-25T10:14:29.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a2ffd640 1 --2- 172.21.15.60:0/3857327094 >> v2:172.21.15.100:3300/0 conn(0x7fe8a40ce7a0 0x7fe8a40ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a27fc640 1 -- 172.21.15.60:0/3857327094 >> v2:172.21.15.100:3300/0 conn(0x7fe8a40ce7a0 msgr2=0x7fe8a40ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a27fc640 1 --2- 172.21.15.60:0/3857327094 >> v2:172.21.15.100:3300/0 conn(0x7fe8a40ce7a0 0x7fe8a40ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a27fc640 1 -- 172.21.15.60:0/3857327094 >> v1:172.21.15.60:6789/0 conn(0x7fe8a40b4660 legacy=0x7fe8a40b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a1ffb640 1 -- 172.21.15.60:0/3857327094 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3572487186 0 0) 0x7fe8a40cf0c0 con 0x7fe8a40b4660 2024-04-25T10:14:29.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a27fc640 1 -- 172.21.15.60:0/3857327094 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a40cca40 con 0x7fe8a415bc90 2024-04-25T10:14:29.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.047+0000 7fe8a27fc640 1 --2- 172.21.15.60:0/3857327094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8a415bc90 0x7fe8a40b25d0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fe8a404f5d0 tx=0x7fe89801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d443def6170732 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a1ffb640 1 -- 172.21.15.60:0/3857327094 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8980b0070 con 0x7fe8a415bc90 2024-04-25T10:14:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a1ffb640 1 -- 172.21.15.60:0/3857327094 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8980ab600 con 0x7fe8a415bc90 2024-04-25T10:14:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a1ffb640 1 -- 172.21.15.60:0/3857327094 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8980ab9d0 con 0x7fe8a415bc90 2024-04-25T10:14:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 -- 172.21.15.60:0/3857327094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8a415bc90 msgr2=0x7fe8a40b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 --2- 172.21.15.60:0/3857327094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8a415bc90 0x7fe8a40b25d0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fe8a404f5d0 tx=0x7fe89801c520 comp rx=0 tx=0).stop 2024-04-25T10:14:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 -- 172.21.15.60:0/3857327094 shutdown_connections 2024-04-25T10:14:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 --2- 172.21.15.60:0/3857327094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8a415bc90 0x7fe8a40b25d0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 --2- 172.21.15.60:0/3857327094 >> v2:172.21.15.100:3300/0 conn(0x7fe8a40ce7a0 0x7fe8a40ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 -- 172.21.15.60:0/3857327094 >> 172.21.15.60:0/3857327094 conn(0x7fe8a40b12d0 msgr2=0x7fe8a40b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 -- 172.21.15.60:0/3857327094 shutdown_connections 2024-04-25T10:14:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 -- 172.21.15.60:0/3857327094 wait complete. 2024-04-25T10:14:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 Processor -- start 2024-04-25T10:14:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 -- start start 2024-04-25T10:14:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8a40ce7a0 0x7fe8a414d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8a415bc90 0x7fe8a414e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8a414e100 0x7fe8a4159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40b4400 con 0x7fe8a40ce7a0 2024-04-25T10:14:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40b4280 con 0x7fe8a414e100 2024-04-25T10:14:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40b3220 con 0x7fe8a415bc90 2024-04-25T10:14:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8a40ce7a0 0x7fe8a414d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8a40ce7a0 0x7fe8a414d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40472/0 (socket says 172.21.15.60:40472) 2024-04-25T10:14:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a2ffd640 1 -- 172.21.15.60:0/1301884804 learned_addr learned my addr 172.21.15.60:0/1301884804 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a2ffd640 1 -- 172.21.15.60:0/1301884804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8a415bc90 msgr2=0x7fe8a414e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a2ffd640 1 --2- 172.21.15.60:0/1301884804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8a415bc90 0x7fe8a414e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a37fe640 1 --2- 172.21.15.60:0/1301884804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8a414e100 0x7fe8a4159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a2ffd640 1 -- 172.21.15.60:0/1301884804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8a414e100 msgr2=0x7fe8a4159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a2ffd640 1 --2- 172.21.15.60:0/1301884804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8a414e100 0x7fe8a4159be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a2ffd640 1 -- 172.21.15.60:0/1301884804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a4152000 con 0x7fe8a40ce7a0 2024-04-25T10:14:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a2ffd640 1 --2- 172.21.15.60:0/1301884804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8a40ce7a0 0x7fe8a414d990 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fe89400d800 tx=0x7fe89400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe88bfff640 1 -- 172.21.15.60:0/1301884804 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89400b660 con 0x7fe8a40ce7a0 2024-04-25T10:14:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe88bfff640 1 -- 172.21.15.60:0/1301884804 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe894019070 con 0x7fe8a40ce7a0 2024-04-25T10:14:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 -- 172.21.15.60:0/1301884804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a414f490 con 0x7fe8a40ce7a0 2024-04-25T10:14:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe88bfff640 1 -- 172.21.15.60:0/1301884804 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89400ba00 con 0x7fe8a40ce7a0 2024-04-25T10:14:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.051+0000 7fe8a9a38640 1 -- 172.21.15.60:0/1301884804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a414ef20 con 0x7fe8a40ce7a0 2024-04-25T10:14:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.055+0000 7fe8a9a38640 1 -- 172.21.15.60:0/1301884804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a4074770 con 0x7fe8a40ce7a0 2024-04-25T10:14:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.055+0000 7fe88bfff640 1 -- 172.21.15.60:0/1301884804 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe89400bba0 con 0x7fe8a40ce7a0 2024-04-25T10:14:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.055+0000 7fe88bfff640 1 --2- 172.21.15.60:0/1301884804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe890066f90 0x7fe890069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.055+0000 7fe88bfff640 1 -- 172.21.15.60:0/1301884804 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe894085490 con 0x7fe8a40ce7a0 2024-04-25T10:14:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.055+0000 7fe8a27fc640 1 --2- 172.21.15.60:0/1301884804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe890066f90 0x7fe890069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.055+0000 7fe8a27fc640 1 --2- 172.21.15.60:0/1301884804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe890066f90 0x7fe890069450 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe8980ad000 tx=0x7fe898004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.059+0000 7fe88bfff640 1 -- 172.21.15.60:0/1301884804 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8940564a0 con 0x7fe8a40ce7a0 2024-04-25T10:14:29.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.247+0000 7fe8a9a38640 1 -- 172.21.15.60:0/1301884804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8a4158070 con 0x7fe8a40ce7a0 2024-04-25T10:14:29.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.247+0000 7fe88bfff640 1 -- 172.21.15.60:0/1301884804 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe894055da0 con 0x7fe8a40ce7a0 2024-04-25T10:14:29.252 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:29.252 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1491228634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1491228634},{"type":"v1","addr":"172.21.15.100:6837","nonce":1491228634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/209898872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":209898872},{"type":"v1","addr":"172.21.15.100:6835","nonce":209898872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1793343287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1793343287},{"type":"v1","addr":"172.21.15.60:6833","nonce":1793343287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:16.146481+0000","modified":"2024-04-25T10:14:19.300954+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2002755047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2002755047},{"type":"v1","addr":"172.21.15.60:6835","nonce":2002755047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:14:29.252 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 77 2024-04-25T10:14:29.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.251+0000 7fe8a9a38640 1 -- 172.21.15.60:0/1301884804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe890066f90 msgr2=0x7fe890069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.251+0000 7fe8a9a38640 1 --2- 172.21.15.60:0/1301884804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe890066f90 0x7fe890069450 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe8980ad000 tx=0x7fe898004740 comp rx=0 tx=0).stop 2024-04-25T10:14:29.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.251+0000 7fe8a9a38640 1 -- 172.21.15.60:0/1301884804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8a40ce7a0 msgr2=0x7fe8a414d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.251+0000 7fe8a9a38640 1 --2- 172.21.15.60:0/1301884804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8a40ce7a0 0x7fe8a414d990 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fe89400d800 tx=0x7fe89400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.251+0000 7fe8a9a38640 1 -- 172.21.15.60:0/1301884804 shutdown_connections 2024-04-25T10:14:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.251+0000 7fe8a9a38640 1 --2- 172.21.15.60:0/1301884804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe890066f90 0x7fe890069450 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.251+0000 7fe8a9a38640 1 --2- 172.21.15.60:0/1301884804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8a414e100 0x7fe8a4159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.251+0000 7fe8a9a38640 1 --2- 172.21.15.60:0/1301884804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8a415bc90 0x7fe8a414e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.251+0000 7fe8a9a38640 1 --2- 172.21.15.60:0/1301884804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8a40ce7a0 0x7fe8a414d990 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.251+0000 7fe8a9a38640 1 -- 172.21.15.60:0/1301884804 >> 172.21.15.60:0/1301884804 conn(0x7fe8a40b12d0 msgr2=0x7fe8a415b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.251+0000 7fe8a9a38640 1 -- 172.21.15.60:0/1301884804 shutdown_connections 2024-04-25T10:14:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.251+0000 7fe8a9a38640 1 -- 172.21.15.60:0/1301884804 wait complete. 2024-04-25T10:14:29.271 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:14:29.412 INFO:teuthology.orchestra.run.smithi060.stdout:{"message":"","return_code":0} 2024-04-25T10:14:29.412 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:14:29.413 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:29.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f47872dd640 1 Processor -- start 2024-04-25T10:14:29.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f47872dd640 1 -- start start 2024-04-25T10:14:29.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f47872dd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f478006ec30 0x7f47800cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f47872dd640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478006f130 con 0x7f478015a7a0 2024-04-25T10:14:29.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f47872dd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478006f410 con 0x7f47800b37e0 2024-04-25T10:14:29.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f47872dd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47800cdaa0 con 0x7f478006ec30 2024-04-25T10:14:29.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4785079640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f478015a7a0 0x7f4780071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53028/0 (socket says 172.21.15.60:53028) 2024-04-25T10:14:29.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4785079640 1 -- 172.21.15.60:0/2332843203 learned_addr learned my addr 172.21.15.60:0/2332843203 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4777fff640 1 -- 172.21.15.60:0/2332843203 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214461477 0 0) 0x7f478006f130 con 0x7f478015a7a0 2024-04-25T10:14:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4777fff640 1 -- 172.21.15.60:0/2332843203 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f476c003530 con 0x7f478015a7a0 2024-04-25T10:14:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4784878640 1 --2- 172.21.15.60:0/2332843203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f478006ec30 0x7f47800cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4777fff640 1 -- 172.21.15.60:0/2332843203 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3763501418 0 0) 0x7f476c003530 con 0x7f478015a7a0 2024-04-25T10:14:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4777fff640 1 -- 172.21.15.60:0/2332843203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f478006ec30 msgr2=0x7f47800cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4777fff640 1 --2- 172.21.15.60:0/2332843203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f478006ec30 0x7f47800cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4777fff640 1 -- 172.21.15.60:0/2332843203 >> v1:172.21.15.100:6789/0 conn(0x7f47800b37e0 legacy=0x7f478006dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4777fff640 1 -- 172.21.15.60:0/2332843203 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478006d540 con 0x7f478015a7a0 2024-04-25T10:14:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4777fff640 1 -- 172.21.15.60:0/2332843203 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f477c001680 con 0x7f478015a7a0 2024-04-25T10:14:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4777fff640 1 -- 172.21.15.60:0/2332843203 >> v1:172.21.15.60:6789/0 conn(0x7f478015a7a0 legacy=0x7f4780071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4777fff640 1 --2- 172.21.15.60:0/2332843203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c0039f0 0x7f476c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4777fff640 1 --2- 172.21.15.60:0/2332843203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f476c004810 0x7f476c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4777fff640 1 --2- 172.21.15.60:0/2332843203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f478006ec30 0x7f476c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4777fff640 1 -- 172.21.15.60:0/2332843203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c00bb00 con 0x7f476c004810 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4777fff640 1 -- 172.21.15.60:0/2332843203 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c00bcb0 con 0x7f476c0039f0 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4777fff640 1 -- 172.21.15.60:0/2332843203 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c00e0c0 con 0x7f478006ec30 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4784878640 1 --2- 172.21.15.60:0/2332843203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f476c004810 0x7f476c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4785079640 1 --2- 172.21.15.60:0/2332843203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c0039f0 0x7f476c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f478587a640 1 --2- 172.21.15.60:0/2332843203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f478006ec30 0x7f476c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4784878640 1 -- 172.21.15.60:0/2332843203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f478006ec30 msgr2=0x7f476c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4784878640 1 --2- 172.21.15.60:0/2332843203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f478006ec30 0x7f476c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4784878640 1 -- 172.21.15.60:0/2332843203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c0039f0 msgr2=0x7f476c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4784878640 1 --2- 172.21.15.60:0/2332843203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c0039f0 0x7f476c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4784878640 1 -- 172.21.15.60:0/2332843203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c003530 con 0x7f476c004810 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f478587a640 1 --2- 172.21.15.60:0/2332843203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f478006ec30 0x7f476c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:29.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.531+0000 7f4785079640 1 --2- 172.21.15.60:0/2332843203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c0039f0 0x7f476c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:29.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.535+0000 7f4784878640 1 --2- 172.21.15.60:0/2332843203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f476c004810 0x7f476c004c90 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f477000ba30 tx=0x7f477001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc29322df899e016 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.535+0000 7f4777fff640 1 -- 172.21.15.60:0/2332843203 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47700ae070 con 0x7f476c004810 2024-04-25T10:14:29.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.535+0000 7f4777fff640 1 -- 172.21.15.60:0/2332843203 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47700a9680 con 0x7f476c004810 2024-04-25T10:14:29.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.535+0000 7f4777fff640 1 -- 172.21.15.60:0/2332843203 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477000b600 con 0x7f476c004810 2024-04-25T10:14:29.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.535+0000 7f47872dd640 1 -- 172.21.15.60:0/2332843203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f476c004810 msgr2=0x7f476c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.535+0000 7f47872dd640 1 --2- 172.21.15.60:0/2332843203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f476c004810 0x7f476c004c90 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f477000ba30 tx=0x7f477001c6a0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.535+0000 7f478587a640 1 -- 172.21.15.60:0/2332843203 reap_dead start 2024-04-25T10:14:29.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.535+0000 7f47872dd640 1 -- 172.21.15.60:0/2332843203 shutdown_connections 2024-04-25T10:14:29.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.535+0000 7f47872dd640 1 -- 172.21.15.60:0/2332843203 >> 172.21.15.60:0/2332843203 conn(0x7f47800b12d0 msgr2=0x7f47800b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.535+0000 7f47872dd640 1 -- 172.21.15.60:0/2332843203 shutdown_connections 2024-04-25T10:14:29.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.535+0000 7f47872dd640 1 -- 172.21.15.60:0/2332843203 wait complete. 2024-04-25T10:14:29.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f47872dd640 1 Processor -- start 2024-04-25T10:14:29.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f47872dd640 1 -- start start 2024-04-25T10:14:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f47872dd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f476c004810 0x7f4780154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f47872dd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47801568b0 0x7f47801588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f47872dd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4780156cf0 0x7f4780158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f47872dd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478006d8b0 con 0x7f47801568b0 2024-04-25T10:14:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f47872dd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478006d2f0 con 0x7f4780156cf0 2024-04-25T10:14:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f47872dd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47800cd560 con 0x7f476c004810 2024-04-25T10:14:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f4784878640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47801568b0 0x7f47801588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f4784878640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47801568b0 0x7f47801588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40490/0 (socket says 172.21.15.60:40490) 2024-04-25T10:14:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f4784878640 1 -- 172.21.15.60:0/3172381143 learned_addr learned my addr 172.21.15.60:0/3172381143 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f4784878640 1 -- 172.21.15.60:0/3172381143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f476c004810 msgr2=0x7f4780154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f4784878640 1 --2- 172.21.15.60:0/3172381143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f476c004810 0x7f4780154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f4784878640 1 -- 172.21.15.60:0/3172381143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4780156cf0 msgr2=0x7f4780158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f4784878640 1 --2- 172.21.15.60:0/3172381143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4780156cf0 0x7f4780158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f4784878640 1 -- 172.21.15.60:0/3172381143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47801572c0 con 0x7f47801568b0 2024-04-25T10:14:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f4784878640 1 --2- 172.21.15.60:0/3172381143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47801568b0 0x7f47801588c0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f477001c4d0 tx=0x7f476c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f4775ffb640 1 -- 172.21.15.60:0/3172381143 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47700ae040 con 0x7f47801568b0 2024-04-25T10:14:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f4775ffb640 1 -- 172.21.15.60:0/3172381143 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47700a9d00 con 0x7f47801568b0 2024-04-25T10:14:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f47872dd640 1 -- 172.21.15.60:0/3172381143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47801575a0 con 0x7f47801568b0 2024-04-25T10:14:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f4775ffb640 1 -- 172.21.15.60:0/3172381143 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c001d50 con 0x7f47801568b0 2024-04-25T10:14:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f47872dd640 1 -- 172.21.15.60:0/3172381143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4780157b60 con 0x7f47801568b0 2024-04-25T10:14:29.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f47872dd640 1 -- 172.21.15.60:0/3172381143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4780074770 con 0x7f47801568b0 2024-04-25T10:14:29.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f4775ffb640 1 -- 172.21.15.60:0/3172381143 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f47700c2060 con 0x7f47801568b0 2024-04-25T10:14:29.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f4775ffb640 1 --2- 172.21.15.60:0/3172381143 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4764066f90 0x7f4764069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f4775ffb640 1 -- 172.21.15.60:0/3172381143 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f477011f5a0 con 0x7f47801568b0 2024-04-25T10:14:29.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.539+0000 7f4785079640 1 --2- 172.21.15.60:0/3172381143 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4764066f90 0x7f4764069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.543+0000 7f4785079640 1 --2- 172.21.15.60:0/3172381143 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4764066f90 0x7f4764069450 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f477c0055e0 tx=0x7f477c0011d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.547+0000 7f4775ffb640 1 -- 172.21.15.60:0/3172381143 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f47700163b0 con 0x7f47801568b0 2024-04-25T10:14:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.739+0000 7f47872dd640 1 -- 172.21.15.60:0/3172381143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47801f3750 con 0x7f47801568b0 2024-04-25T10:14:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.739+0000 7f4775ffb640 1 -- 172.21.15.60:0/3172381143 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f47700f05a0 con 0x7f47801568b0 2024-04-25T10:14:29.745 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:29.745 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1491228634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1491228634},{"type":"v1","addr":"172.21.15.100:6837","nonce":1491228634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/209898872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":209898872},{"type":"v1","addr":"172.21.15.100:6835","nonce":209898872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1793343287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1793343287},{"type":"v1","addr":"172.21.15.60:6833","nonce":1793343287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:16.146481+0000","modified":"2024-04-25T10:14:19.300954+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2002755047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2002755047},{"type":"v1","addr":"172.21.15.60:6835","nonce":2002755047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:14:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 77 2024-04-25T10:14:29.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.743+0000 7f47872dd640 1 -- 172.21.15.60:0/3172381143 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4764066f90 msgr2=0x7f4764069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.743+0000 7f47872dd640 1 --2- 172.21.15.60:0/3172381143 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4764066f90 0x7f4764069450 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f477c0055e0 tx=0x7f477c0011d0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.743+0000 7f47872dd640 1 -- 172.21.15.60:0/3172381143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47801568b0 msgr2=0x7f47801588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.743+0000 7f47872dd640 1 --2- 172.21.15.60:0/3172381143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47801568b0 0x7f47801588c0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f477001c4d0 tx=0x7f476c005840 comp rx=0 tx=0).stop 2024-04-25T10:14:29.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.743+0000 7f47872dd640 1 -- 172.21.15.60:0/3172381143 shutdown_connections 2024-04-25T10:14:29.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.743+0000 7f47872dd640 1 --2- 172.21.15.60:0/3172381143 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4764066f90 0x7f4764069450 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.743+0000 7f47872dd640 1 --2- 172.21.15.60:0/3172381143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4780156cf0 0x7f4780158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.743+0000 7f47872dd640 1 --2- 172.21.15.60:0/3172381143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47801568b0 0x7f47801588c0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.743+0000 7f47872dd640 1 --2- 172.21.15.60:0/3172381143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f476c004810 0x7f4780154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.743+0000 7f47872dd640 1 -- 172.21.15.60:0/3172381143 >> 172.21.15.60:0/3172381143 conn(0x7f47800b12d0 msgr2=0x7f47800af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.743+0000 7f47872dd640 1 -- 172.21.15.60:0/3172381143 shutdown_connections 2024-04-25T10:14:29.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.743+0000 7f47872dd640 1 -- 172.21.15.60:0/3172381143 wait complete. 2024-04-25T10:14:29.764 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-04-25T10:14:29.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 Processor -- start 2024-04-25T10:14:29.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 -- start start 2024-04-25T10:14:29.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637d005c340 0x5637d00ff150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637d0101400 0x5637d00fbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637d0101da0 0x5637d00745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d0073e80 con 0x5637d0101400 2024-04-25T10:14:29.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637cffd29f0 con 0x5637d0101da0 2024-04-25T10:14:29.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5637d0074120 con 0x5637d005c340 2024-04-25T10:14:29.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022bd4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637d0101400 0x5637d00fbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022bd4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637d0101400 0x5637d00fbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40498/0 (socket says 172.21.15.60:40498) 2024-04-25T10:14:29.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022bd4c640 1 -- 172.21.15.60:0/3895896775 learned_addr learned my addr 172.21.15.60:0/3895896775 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:29.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022c54d640 1 --2- 172.21.15.60:0/3895896775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637d005c340 0x5637d00ff150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022cd4e640 1 --2- 172.21.15.60:0/3895896775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637d0101da0 0x5637d00745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022bd4c640 1 -- 172.21.15.60:0/3895896775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637d005c340 msgr2=0x5637d00ff150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022bd4c640 1 --2- 172.21.15.60:0/3895896775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637d005c340 0x5637d00ff150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022bd4c640 1 -- 172.21.15.60:0/3895896775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637d0101da0 msgr2=0x5637d00745c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022bd4c640 1 --2- 172.21.15.60:0/3895896775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637d0101da0 0x5637d00745c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022bd4c640 1 -- 172.21.15.60:0/3895896775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637d00ff940 con 0x5637d0101400 2024-04-25T10:14:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022bd4c640 1 --2- 172.21.15.60:0/3895896775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637d0101400 0x5637d00fbc10 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f022000a7e0 tx=0x7f022000aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5782e257482bf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ad4a640 1 -- 172.21.15.60:0/3895896775 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0220008c00 con 0x5637d0101400 2024-04-25T10:14:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ad4a640 1 -- 172.21.15.60:0/3895896775 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0220008da0 con 0x5637d0101400 2024-04-25T10:14:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ad4a640 1 -- 172.21.15.60:0/3895896775 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0220013610 con 0x5637d0101400 2024-04-25T10:14:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 -- 172.21.15.60:0/3895896775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637d0101400 msgr2=0x5637d00fbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 --2- 172.21.15.60:0/3895896775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637d0101400 0x5637d00fbc10 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f022000a7e0 tx=0x7f022000aca0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 -- 172.21.15.60:0/3895896775 shutdown_connections 2024-04-25T10:14:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 --2- 172.21.15.60:0/3895896775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637d0101da0 0x5637d00745c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 --2- 172.21.15.60:0/3895896775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637d0101400 0x5637d00fbc10 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 --2- 172.21.15.60:0/3895896775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637d005c340 0x5637d00ff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 -- 172.21.15.60:0/3895896775 >> 172.21.15.60:0/3895896775 conn(0x5637d0056e80 msgr2=0x5637d001ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 -- 172.21.15.60:0/3895896775 shutdown_connections 2024-04-25T10:14:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 -- 172.21.15.60:0/3895896775 wait complete. 2024-04-25T10:14:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 Processor -- start 2024-04-25T10:14:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 -- start start 2024-04-25T10:14:29.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637d005c340 0x5637d004e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637d0101400 0x5637d0100390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637d0101da0 0x5637d002ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d00fcf20 con 0x5637d0101da0 2024-04-25T10:14:29.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d010ba20 con 0x5637d005c340 2024-04-25T10:14:29.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.791+0000 7f022ea2f080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5637d00fd180 con 0x5637d0101400 2024-04-25T10:14:29.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022cd4e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637d0101da0 0x5637d002ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022cd4e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637d0101da0 0x5637d002ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40500/0 (socket says 172.21.15.60:40500) 2024-04-25T10:14:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022cd4e640 1 -- 172.21.15.60:0/717240990 learned_addr learned my addr 172.21.15.60:0/717240990 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022c54d640 1 --2- 172.21.15.60:0/717240990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637d005c340 0x5637d004e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022cd4e640 1 -- 172.21.15.60:0/717240990 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637d0101400 msgr2=0x5637d0100390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022cd4e640 1 --2- 172.21.15.60:0/717240990 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637d0101400 0x5637d0100390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022cd4e640 1 -- 172.21.15.60:0/717240990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637d005c340 msgr2=0x5637d004e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022cd4e640 1 --2- 172.21.15.60:0/717240990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637d005c340 0x5637d004e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022cd4e640 1 -- 172.21.15.60:0/717240990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637d00725d0 con 0x5637d0101da0 2024-04-25T10:14:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022c54d640 1 --2- 172.21.15.60:0/717240990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637d005c340 0x5637d004e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022cd4e640 1 --2- 172.21.15.60:0/717240990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637d0101da0 0x5637d002ded0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f022401b880 tx=0x7f0224018140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f0229d48640 1 -- 172.21.15.60:0/717240990 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02240af040 con 0x5637d0101da0 2024-04-25T10:14:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f0229d48640 1 -- 172.21.15.60:0/717240990 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02240b5480 con 0x5637d0101da0 2024-04-25T10:14:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f0229d48640 1 -- 172.21.15.60:0/717240990 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02240b5830 con 0x5637d0101da0 2024-04-25T10:14:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022ea2f080 1 -- 172.21.15.60:0/717240990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5637d0072870 con 0x5637d0101da0 2024-04-25T10:14:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022ea2f080 1 -- 172.21.15.60:0/717240990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637d00fae10 con 0x5637d0101da0 2024-04-25T10:14:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f0229d48640 1 -- 172.21.15.60:0/717240990 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f02240b59d0 con 0x5637d0101da0 2024-04-25T10:14:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f0229d48640 1 --2- 172.21.15.60:0/717240990 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0218065970 0x7f0218067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f0229d48640 1 -- 172.21.15.60:0/717240990 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f022411eee0 con 0x5637d0101da0 2024-04-25T10:14:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022ea2f080 1 --2- 172.21.15.60:0/717240990 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x5637d01008d0 0x5637d006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022ea2f080 1 -- 172.21.15.60:0/717240990 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) v9 -- 0x5637d006da30 con 0x5637d01008d0 2024-04-25T10:14:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022c54d640 1 --2- 172.21.15.60:0/717240990 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0218065970 0x7f0218067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022bd4c640 1 --2- 172.21.15.60:0/717240990 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x5637d01008d0 0x5637d006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022bd4c640 1 --2- 172.21.15.60:0/717240990 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x5637d01008d0 0x5637d006ec50 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022c54d640 1 --2- 172.21.15.60:0/717240990 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0218065970 0x7f0218067e30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f021c0025e0 tx=0x7f021c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022bd4c640 1 -- 172.21.15.60:0/717240990 <== osd.2 v2:172.21.15.60:6800/1771147646 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x7f022001f420 con 0x5637d01008d0 2024-04-25T10:14:29.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022ea2f080 1 -- 172.21.15.60:0/717240990 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x5637d01008d0 msgr2=0x5637d006ec50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022ea2f080 1 --2- 172.21.15.60:0/717240990 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x5637d01008d0 0x5637d006ec50 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022ea2f080 1 -- 172.21.15.60:0/717240990 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0218065970 msgr2=0x7f0218067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022ea2f080 1 --2- 172.21.15.60:0/717240990 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0218065970 0x7f0218067e30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f021c0025e0 tx=0x7f021c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:29.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022ea2f080 1 -- 172.21.15.60:0/717240990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637d0101da0 msgr2=0x5637d002ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022ea2f080 1 --2- 172.21.15.60:0/717240990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637d0101da0 0x5637d002ded0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f022401b880 tx=0x7f0224018140 comp rx=0 tx=0).stop 2024-04-25T10:14:29.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022cd4e640 1 -- 172.21.15.60:0/717240990 reap_dead start 2024-04-25T10:14:29.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022ea2f080 1 -- 172.21.15.60:0/717240990 shutdown_connections 2024-04-25T10:14:29.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022ea2f080 1 -- 172.21.15.60:0/717240990 >> 172.21.15.60:0/717240990 conn(0x5637d0056e80 msgr2=0x5637d001ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022ea2f080 1 -- 172.21.15.60:0/717240990 shutdown_connections 2024-04-25T10:14:29.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.795+0000 7f022ea2f080 1 -- 172.21.15.60:0/717240990 wait complete. 2024-04-25T10:14:29.807 INFO:teuthology.orchestra.run.smithi060.stdout:����� 2024-04-25T10:14:29.807 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:14:29.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.959+0000 7ff0aec0a640 1 Processor -- start 2024-04-25T10:14:29.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.959+0000 7ff0aec0a640 1 -- start start 2024-04-25T10:14:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0aec0a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a80cde60 0x7ff0a8164e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0aec0a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a815dc10 con 0x7ff0a806e480 2024-04-25T10:14:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0aec0a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a815def0 con 0x7ff0a8071b20 2024-04-25T10:14:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0aec0a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a815e1d0 con 0x7ff0a80cde60 2024-04-25T10:14:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff0a806e480 0x7ff0a806e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53030/0 (socket says 172.21.15.60:53030) 2024-04-25T10:14:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a7fff640 1 -- 172.21.15.60:0/3835275212 learned_addr learned my addr 172.21.15.60:0/3835275212 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0ad1a7640 1 --2- 172.21.15.60:0/3835275212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a80cde60 0x7ff0a8164e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a77fe640 1 -- 172.21.15.60:0/3835275212 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337849131 0 0) 0x7ff0a815dc10 con 0x7ff0a806e480 2024-04-25T10:14:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a77fe640 1 -- 172.21.15.60:0/3835275212 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff090005740 con 0x7ff0a806e480 2024-04-25T10:14:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a77fe640 1 -- 172.21.15.60:0/3835275212 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 138563088 0 0) 0x7ff090005740 con 0x7ff0a806e480 2024-04-25T10:14:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a77fe640 1 -- 172.21.15.60:0/3835275212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a80cde60 msgr2=0x7ff0a8164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a77fe640 1 --2- 172.21.15.60:0/3835275212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a80cde60 0x7ff0a8164e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a77fe640 1 -- 172.21.15.60:0/3835275212 >> v1:172.21.15.100:6789/0 conn(0x7ff0a8071b20 legacy=0x7ff0a8071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a77fe640 1 -- 172.21.15.60:0/3835275212 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a815e4b0 con 0x7ff0a806e480 2024-04-25T10:14:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a77fe640 1 -- 172.21.15.60:0/3835275212 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7ff09c001680 con 0x7ff0a806e480 2024-04-25T10:14:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a77fe640 1 -- 172.21.15.60:0/3835275212 >> v1:172.21.15.60:6789/0 conn(0x7ff0a806e480 legacy=0x7ff0a806e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a77fe640 1 --2- 172.21.15.60:0/3835275212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff090005c00 0x7ff090005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a77fe640 1 --2- 172.21.15.60:0/3835275212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a80cde60 0x7ff090006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a77fe640 1 --2- 172.21.15.60:0/3835275212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff090006f60 0x7ff0900073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a77fe640 1 -- 172.21.15.60:0/3835275212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a815e1d0 con 0x7ff090006f60 2024-04-25T10:14:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a77fe640 1 -- 172.21.15.60:0/3835275212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09000dc90 con 0x7ff090005c00 2024-04-25T10:14:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a77fe640 1 -- 172.21.15.60:0/3835275212 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09000de90 con 0x7ff0a80cde60 2024-04-25T10:14:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0ad1a7640 1 --2- 172.21.15.60:0/3835275212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff090006f60 0x7ff0900073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0a7fff640 1 --2- 172.21.15.60:0/3835275212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a80cde60 0x7ff090006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0ac9a6640 1 --2- 172.21.15.60:0/3835275212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff090005c00 0x7ff090005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0ad1a7640 1 -- 172.21.15.60:0/3835275212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a80cde60 msgr2=0x7ff090006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0ad1a7640 1 --2- 172.21.15.60:0/3835275212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a80cde60 0x7ff090006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0ad1a7640 1 -- 172.21.15.60:0/3835275212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff090005c00 msgr2=0x7ff090005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0ad1a7640 1 --2- 172.21.15.60:0/3835275212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff090005c00 0x7ff090005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.963+0000 7ff0ad1a7640 1 -- 172.21.15.60:0/3835275212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff090005740 con 0x7ff090006f60 2024-04-25T10:14:29.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0ad1a7640 1 --2- 172.21.15.60:0/3835275212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff090006f60 0x7ff0900073e0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009f70 tx=0x7ff0a001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b454d2b9085c5aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0a77fe640 1 -- 172.21.15.60:0/3835275212 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a00b2070 con 0x7ff090006f60 2024-04-25T10:14:29.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0a77fe640 1 -- 172.21.15.60:0/3835275212 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a00a5cf0 con 0x7ff090006f60 2024-04-25T10:14:29.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0a77fe640 1 -- 172.21.15.60:0/3835275212 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0009bf0 con 0x7ff090006f60 2024-04-25T10:14:29.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0aec0a640 1 -- 172.21.15.60:0/3835275212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff090006f60 msgr2=0x7ff0900073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0aec0a640 1 --2- 172.21.15.60:0/3835275212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff090006f60 0x7ff0900073e0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009f70 tx=0x7ff0a001cc20 comp rx=0 tx=0).stop 2024-04-25T10:14:29.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0ad1a7640 1 -- 172.21.15.60:0/3835275212 reap_dead start 2024-04-25T10:14:29.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0aec0a640 1 -- 172.21.15.60:0/3835275212 shutdown_connections 2024-04-25T10:14:29.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0aec0a640 1 -- 172.21.15.60:0/3835275212 >> 172.21.15.60:0/3835275212 conn(0x7ff0a80b12d0 msgr2=0x7ff0a80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0aec0a640 1 -- 172.21.15.60:0/3835275212 shutdown_connections 2024-04-25T10:14:29.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0aec0a640 1 -- 172.21.15.60:0/3835275212 wait complete. 2024-04-25T10:14:29.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0aec0a640 1 Processor -- start 2024-04-25T10:14:29.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0aec0a640 1 -- start start 2024-04-25T10:14:29.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0aec0a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a8155070 0x7ff0a80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0aec0a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a80b3950 0x7ff0a8140850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0aec0a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a8140d90 0x7ff0a81e38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0aec0a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a80cf070 con 0x7ff0a8155070 2024-04-25T10:14:29.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0aec0a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a80ceef0 con 0x7ff0a8140d90 2024-04-25T10:14:29.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0aec0a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8071290 con 0x7ff0a80b3950 2024-04-25T10:14:29.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0ac9a6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a8155070 0x7ff0a80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0ac9a6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a8155070 0x7ff0a80b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40522/0 (socket says 172.21.15.60:40522) 2024-04-25T10:14:29.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0ac9a6640 1 -- 172.21.15.60:0/3614732486 learned_addr learned my addr 172.21.15.60:0/3614732486 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:29.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0a7fff640 1 --2- 172.21.15.60:0/3614732486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a80b3950 0x7ff0a8140850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0ad1a7640 1 --2- 172.21.15.60:0/3614732486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a8140d90 0x7ff0a81e38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0ac9a6640 1 -- 172.21.15.60:0/3614732486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a80b3950 msgr2=0x7ff0a8140850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0ac9a6640 1 --2- 172.21.15.60:0/3614732486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a80b3950 0x7ff0a8140850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0ac9a6640 1 -- 172.21.15.60:0/3614732486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a8140d90 msgr2=0x7ff0a81e38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0ac9a6640 1 --2- 172.21.15.60:0/3614732486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a8140d90 0x7ff0a81e38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0ac9a6640 1 -- 172.21.15.60:0/3614732486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a8157010 con 0x7ff0a8155070 2024-04-25T10:14:29.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.967+0000 7ff0ac9a6640 1 --2- 172.21.15.60:0/3614732486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a8155070 0x7ff0a80b3410 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7ff098010f70 tx=0x7ff09800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.971+0000 7ff0a57fa640 1 -- 172.21.15.60:0/3614732486 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff098018020 con 0x7ff0a8155070 2024-04-25T10:14:29.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.971+0000 7ff0a57fa640 1 -- 172.21.15.60:0/3614732486 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff098005070 con 0x7ff0a8155070 2024-04-25T10:14:29.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.971+0000 7ff0aec0a640 1 -- 172.21.15.60:0/3614732486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a81415c0 con 0x7ff0a8155070 2024-04-25T10:14:29.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.971+0000 7ff0a57fa640 1 -- 172.21.15.60:0/3614732486 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff098005460 con 0x7ff0a8155070 2024-04-25T10:14:29.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.971+0000 7ff0aec0a640 1 -- 172.21.15.60:0/3614732486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a8141b80 con 0x7ff0a8155070 2024-04-25T10:14:29.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.971+0000 7ff0aec0a640 1 -- 172.21.15.60:0/3614732486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a8074770 con 0x7ff0a8155070 2024-04-25T10:14:29.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.971+0000 7ff0a57fa640 1 -- 172.21.15.60:0/3614732486 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff09801b020 con 0x7ff0a8155070 2024-04-25T10:14:29.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.971+0000 7ff0a57fa640 1 --2- 172.21.15.60:0/3614732486 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff094066f60 0x7ff094069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.971+0000 7ff0a57fa640 1 -- 172.21.15.60:0/3614732486 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff098088860 con 0x7ff0a8155070 2024-04-25T10:14:29.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.971+0000 7ff0a7fff640 1 --2- 172.21.15.60:0/3614732486 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff094066f60 0x7ff094069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.971+0000 7ff0a7fff640 1 --2- 172.21.15.60:0/3614732486 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff094066f60 0x7ff094069420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0010f0 tx=0x7ff09c001160 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:29.975+0000 7ff0a57fa640 1 -- 172.21.15.60:0/3614732486 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff098022020 con 0x7ff0a8155070 2024-04-25T10:14:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:30.167+0000 7ff0aec0a640 1 -- 172.21.15.60:0/3614732486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff0a8143030 con 0x7ff0a8155070 2024-04-25T10:14:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:30.171+0000 7ff0a57fa640 1 -- 172.21.15.60:0/3614732486 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff098059860 con 0x7ff0a8155070 2024-04-25T10:14:30.174 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:14:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:30.171+0000 7ff0aec0a640 1 -- 172.21.15.60:0/3614732486 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff094066f60 msgr2=0x7ff094069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:30.171+0000 7ff0aec0a640 1 --2- 172.21.15.60:0/3614732486 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff094066f60 0x7ff094069420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0010f0 tx=0x7ff09c001160 comp rx=0 tx=0).stop 2024-04-25T10:14:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:30.171+0000 7ff0aec0a640 1 -- 172.21.15.60:0/3614732486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a8155070 msgr2=0x7ff0a80b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:30.171+0000 7ff0aec0a640 1 --2- 172.21.15.60:0/3614732486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a8155070 0x7ff0a80b3410 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7ff098010f70 tx=0x7ff09800e530 comp rx=0 tx=0).stop 2024-04-25T10:14:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:30.171+0000 7ff0aec0a640 1 -- 172.21.15.60:0/3614732486 shutdown_connections 2024-04-25T10:14:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:30.171+0000 7ff0aec0a640 1 --2- 172.21.15.60:0/3614732486 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff094066f60 0x7ff094069420 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:30.171+0000 7ff0aec0a640 1 --2- 172.21.15.60:0/3614732486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a8140d90 0x7ff0a81e38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:30.171+0000 7ff0aec0a640 1 --2- 172.21.15.60:0/3614732486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a80b3950 0x7ff0a8140850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:30.171+0000 7ff0aec0a640 1 --2- 172.21.15.60:0/3614732486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a8155070 0x7ff0a80b3410 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:30.171+0000 7ff0aec0a640 1 -- 172.21.15.60:0/3614732486 >> 172.21.15.60:0/3614732486 conn(0x7ff0a80b12d0 msgr2=0x7ff0a80cb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:30.171+0000 7ff0aec0a640 1 -- 172.21.15.60:0/3614732486 shutdown_connections 2024-04-25T10:14:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:30.171+0000 7ff0aec0a640 1 -- 172.21.15.60:0/3614732486 wait complete. 2024-04-25T10:14:30.193 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:14:30.193 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:14:30.193 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:14:30.193 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:14:30.193 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:30.193 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:14:30.193 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:14:30.193 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:14:30.193 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:30.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:30.194 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:14:30.200 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:14:30.200 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:14:30.200 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:14:30.200 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:30.200 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:14:30.200 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:30.200 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:14:30.200 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:30.201 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:14:30.201 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:14:30.201 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:30.201 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:14:30.201 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:14:30.201 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:30.201 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:14:30.201 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:14:30.201 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41987sec preferred_lft 41987sec 2024-04-25T10:14:30.201 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:14:30.201 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:30.201 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:14:30.201 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:30.202 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:14:30.202 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:14:30.202 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:30.202 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:14:30.202 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:30.202 INFO:teuthology.orchestra.run.smithi060.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:14:30.202 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:30.202 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:14:30.202 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:30.202 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:14:30.202 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:30.202 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:30.203 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:30.203 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:14:30.203 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:30.203 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:30.203 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:30.203 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:14:30.203 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:30.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:30.203 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:14:30.250 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:14:30.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:30.252 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:14:30.299 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:14:30.300 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:14:30.300 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:14:30.300 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:14:30.300 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:14:30.300 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:14:30.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:30.300 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:14:30.300 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:14:30.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:14:30.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:14:30.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:14:30.301 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:14:30.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:30.373 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:14:30.418 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:14:30.418 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:14:30.418 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:30.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:30.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T10:14:30.475 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:14:30.476 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:30.476 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:30.523 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:30.524 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:30.524 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:30.571 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:14:30.632 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:14:30.679 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:14:30.680 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:14:30.680 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:14:30.680 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:14:30.680 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:14:30.680 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:14:30.680 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:14:30.680 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:14:30.680 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:14:30.680 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:14:30.680 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:14:30.680 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:14:30.681 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:14:30.681 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:14:30.681 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:14:30.681 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:14:30.681 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:14:30.681 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:14:30.681 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:14:30.681 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:14:30.681 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:14:30.681 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:14:30.681 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:14:30.681 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:14:30.681 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:14:30.682 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:14:30.682 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:14:30.682 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:14:30.682 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:14:30.682 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:30.682 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:30.682 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:30.682 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:30.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:30.683 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:14:30.731 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:14:30.731 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:14:30.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:14:30.792 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:14:30.842 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:14:30.842 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:14:30.842 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:14:30.842 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:14:30.842 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:14:30.842 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:14:30.842 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:14:30.842 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:14:30.842 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:14:30.842 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:14:30.843 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:14:30.843 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:14:30.843 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:14:30.843 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:14:30.843 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:14:30.843 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:14:30.843 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:14:30.843 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:14:30.843 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:14:30.843 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:14:30.843 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:14:30.843 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:14:30.844 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:14:30.844 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:14:30.844 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:14:30.844 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:14:30.844 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:14:30.844 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:14:30.844 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:14:30.844 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:30.844 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:30.845 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:30.845 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:30.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:30.845 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:14:30.861 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2024-04-25T10:14:30.855+0000 7fc8b9958580 -1 init, newargv = 0x562bbc11fd80 newargc=15 2024-04-25T10:14:30.861 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[49369]: starting ceph client 2024-04-25T10:14:30.867 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[49369]: starting fuse 2024-04-25T10:14:31.967 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:14:31.984 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:14:32.033 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:14:32.036 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:14:32.036 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:14:32.036 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:14:32.036 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:14:32.036 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:32.036 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:32.036 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:32.036 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:32.036 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:32.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.037 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:14:32.083 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:14:32.083 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:14:32.083 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:14:32.083 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:14:32.083 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:14:32.084 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:14:32.084 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:14:32.084 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:14:32.084 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:14:32.084 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:14:32.084 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:14:32.084 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:14:32.084 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:14:32.084 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:14:32.084 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:14:32.084 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:14:32.084 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:14:32.084 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:14:32.084 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:14:32.084 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:14:32.084 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:14:32.085 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:14:32.085 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:14:32.085 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:14:32.085 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:14:32.085 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:14:32.085 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:14:32.085 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2024-04-25T10:14:32.085 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:14:32.210 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.49369.asok 2024-04-25T10:14:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:32.224 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49369.asok 2024-04-25T10:14:32.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.224 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49369.asok status 2024-04-25T10:14:32.383 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:14:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:14:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:14:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:14:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2024-04-25T10:14:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:14:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:14:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "49369", 2024-04-25T10:14:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:14:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:14:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:14:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 8889, 2024-04-25T10:14:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:14:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:14:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:14:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 8889 2024-04-25T10:14:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:14:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:14:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:14:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2506607198 2024-04-25T10:14:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:14:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:14:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:14:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2506607198 2024-04-25T10:14:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:14:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.8889 192.168.0.1:0/2506607198", 2024-04-25T10:14:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/2506607198", 2024-04-25T10:14:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:14:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 77, 2024-04-25T10:14:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 97, 2024-04-25T10:14:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:14:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:14:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:14:32.387 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:14:32.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.398 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:14:32.398 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:32.404 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:14:32.404 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:32.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.405 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:32.458 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:14:32.459 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:32.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.507 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:14:32.636 INFO:teuthology.orchestra.run.smithi060.stderr:6+0 records in 2024-04-25T10:14:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:6+0 records out 2024-04-25T10:14:32.637 INFO:teuthology.orchestra.run.smithi060.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0732529 s, 85.9 MB/s 2024-04-25T10:14:32.638 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:14:32.638 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:32.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.644 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:14:32.759 INFO:teuthology.orchestra.run.smithi060.stderr:6+0 records in 2024-04-25T10:14:32.759 INFO:teuthology.orchestra.run.smithi060.stderr:6+0 records out 2024-04-25T10:14:32.760 INFO:teuthology.orchestra.run.smithi060.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0633596 s, 99.3 MB/s 2024-04-25T10:14:32.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.761 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:14:32.772 INFO:teuthology.orchestra.run.smithi060.stdout:./ 2024-04-25T10:14:32.772 INFO:teuthology.orchestra.run.smithi060.stdout:./file2_sixmegs 2024-04-25T10:14:32.773 INFO:teuthology.orchestra.run.smithi060.stdout:./file3_sixmegs 2024-04-25T10:14:32.773 INFO:teuthology.orchestra.run.smithi060.stdout:./file1_sixmegs 2024-04-25T10:14:32.773 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:14:32.773 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:14:32.773 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:14:32.773 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:14:32.773 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:14:32.773 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:14:32.884 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:14:32.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:32.895 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:14:32.895 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:14:32.895 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:14:32.895 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:14:32.895 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-04-25T10:14:32.895 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:14:32.999 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627777 2024-04-25T10:14:33.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:33.010 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:14:33.010 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:14:33.010 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:14:33.010 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:14:33.010 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-04-25T10:14:33.010 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:14:33.126 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627778 2024-04-25T10:14:33.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:33.136 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:14:33.137 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:14:33.137 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:14:33.137 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:14:33.137 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-04-25T10:14:33.137 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:14:33.242 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627776 2024-04-25T10:14:33.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:33.252 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:14:33.252 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:33.299 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:33.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f31695f2640 1 Processor -- start 2024-04-25T10:14:33.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f31695f2640 1 -- start start 2024-04-25T10:14:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f31695f2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f316415e4b0 0x7f31640d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f31695f2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316415f5a0 con 0x7f3164099b10 2024-04-25T10:14:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f31695f2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316415ea60 con 0x7f316406c0e0 2024-04-25T10:14:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f31695f2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316406c570 con 0x7f316415e4b0 2024-04-25T10:14:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3162ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3164099b10 0x7f3164099ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53040/0 (socket says 172.21.15.60:53040) 2024-04-25T10:14:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3162ffd640 1 -- 172.21.15.60:0/1602277036 learned_addr learned my addr 172.21.15.60:0/1602277036 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f31637fe640 1 --2- 172.21.15.60:0/1602277036 >> v2:172.21.15.100:3300/0 conn(0x7f316415e4b0 0x7f31640d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3161ffb640 1 -- 172.21.15.60:0/1602277036 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962457747 0 0) 0x7f314c001120 con 0x7f3164099b10 2024-04-25T10:14:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3161ffb640 1 -- 172.21.15.60:0/1602277036 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3150003530 con 0x7f3164099b10 2024-04-25T10:14:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3161ffb640 1 -- 172.21.15.60:0/1602277036 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3387366022 0 0) 0x7f3150003530 con 0x7f3164099b10 2024-04-25T10:14:33.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3161ffb640 1 -- 172.21.15.60:0/1602277036 >> v2:172.21.15.100:3300/0 conn(0x7f316415e4b0 msgr2=0x7f31640d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3161ffb640 1 --2- 172.21.15.60:0/1602277036 >> v2:172.21.15.100:3300/0 conn(0x7f316415e4b0 0x7f31640d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3161ffb640 1 -- 172.21.15.60:0/1602277036 >> v1:172.21.15.100:6789/0 conn(0x7f316406c0e0 legacy=0x7f31640d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3161ffb640 1 -- 172.21.15.60:0/1602277036 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316415ed40 con 0x7f3164099b10 2024-04-25T10:14:33.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3161ffb640 1 -- 172.21.15.60:0/1602277036 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f314c001850 con 0x7f3164099b10 2024-04-25T10:14:33.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3161ffb640 1 -- 172.21.15.60:0/1602277036 >> v1:172.21.15.60:6789/0 conn(0x7f3164099b10 legacy=0x7f3164099ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3161ffb640 1 --2- 172.21.15.60:0/1602277036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31500039c0 0x7f3150003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3161ffb640 1 --2- 172.21.15.60:0/1602277036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3150004810 0x7f3150004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3161ffb640 1 --2- 172.21.15.60:0/1602277036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31500051d0 0x7f315000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3161ffb640 1 -- 172.21.15.60:0/1602277036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316406c570 con 0x7f3150004810 2024-04-25T10:14:33.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3161ffb640 1 -- 172.21.15.60:0/1602277036 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315000c170 con 0x7f31500051d0 2024-04-25T10:14:33.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3161ffb640 1 -- 172.21.15.60:0/1602277036 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315000c370 con 0x7f31500039c0 2024-04-25T10:14:33.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f31627fc640 1 --2- 172.21.15.60:0/1602277036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3150004810 0x7f3150004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3162ffd640 1 --2- 172.21.15.60:0/1602277036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31500039c0 0x7f3150003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f31637fe640 1 --2- 172.21.15.60:0/1602277036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31500051d0 0x7f315000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f31627fc640 1 -- 172.21.15.60:0/1602277036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31500039c0 msgr2=0x7f3150003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f31627fc640 1 --2- 172.21.15.60:0/1602277036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31500039c0 0x7f3150003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f31627fc640 1 -- 172.21.15.60:0/1602277036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31500051d0 msgr2=0x7f315000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f31627fc640 1 --2- 172.21.15.60:0/1602277036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31500051d0 0x7f315000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f31627fc640 1 -- 172.21.15.60:0/1602277036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3150003530 con 0x7f3150004810 2024-04-25T10:14:33.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f31637fe640 1 -- 172.21.15.60:0/1602277036 reap_dead start 2024-04-25T10:14:33.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.467+0000 7f3162ffd640 1 --2- 172.21.15.60:0/1602277036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31500039c0 0x7f3150003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:33.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.471+0000 7f31627fc640 1 --2- 172.21.15.60:0/1602277036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3150004810 0x7f3150004c90 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f3154001780 tx=0x7f315401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1f1b0b37d77fee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.471+0000 7f3161ffb640 1 -- 172.21.15.60:0/1602277036 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31540013a0 con 0x7f3150004810 2024-04-25T10:14:33.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.471+0000 7f3161ffb640 1 -- 172.21.15.60:0/1602277036 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31540052f0 con 0x7f3150004810 2024-04-25T10:14:33.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.471+0000 7f3161ffb640 1 -- 172.21.15.60:0/1602277036 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3154003840 con 0x7f3150004810 2024-04-25T10:14:33.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.471+0000 7f31695f2640 1 -- 172.21.15.60:0/1602277036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3150004810 msgr2=0x7f3150004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.471+0000 7f31695f2640 1 --2- 172.21.15.60:0/1602277036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3150004810 0x7f3150004c90 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f3154001780 tx=0x7f315401c3d0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.471+0000 7f31695f2640 1 -- 172.21.15.60:0/1602277036 shutdown_connections 2024-04-25T10:14:33.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.471+0000 7f31695f2640 1 --2- 172.21.15.60:0/1602277036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3150004810 0x7f3150004c90 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.471+0000 7f31695f2640 1 -- 172.21.15.60:0/1602277036 >> 172.21.15.60:0/1602277036 conn(0x7f31640b3320 msgr2=0x7f31640b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.471+0000 7f31695f2640 1 -- 172.21.15.60:0/1602277036 shutdown_connections 2024-04-25T10:14:33.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.471+0000 7f31695f2640 1 -- 172.21.15.60:0/1602277036 wait complete. 2024-04-25T10:14:33.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.471+0000 7f31695f2640 1 Processor -- start 2024-04-25T10:14:33.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.471+0000 7f31695f2640 1 -- start start 2024-04-25T10:14:33.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f31695f2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3150004810 0x7f31641521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f31695f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31640b6740 0x7f31640c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f31695f2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31640ce240 0x7f31640ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f31695f2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316409a480 con 0x7f3150004810 2024-04-25T10:14:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f31695f2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316409a300 con 0x7f31640b6740 2024-04-25T10:14:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f31695f2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3164161390 con 0x7f31640ce240 2024-04-25T10:14:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f3162ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3150004810 0x7f31641521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f3162ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3150004810 0x7f31641521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40538/0 (socket says 172.21.15.60:40538) 2024-04-25T10:14:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f3162ffd640 1 -- 172.21.15.60:0/3266907198 learned_addr learned my addr 172.21.15.60:0/3266907198 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f31627fc640 1 --2- 172.21.15.60:0/3266907198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31640b6740 0x7f31640c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f31637fe640 1 --2- 172.21.15.60:0/3266907198 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31640ce240 0x7f31640ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f3162ffd640 1 -- 172.21.15.60:0/3266907198 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31640ce240 msgr2=0x7f31640ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f3162ffd640 1 --2- 172.21.15.60:0/3266907198 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31640ce240 0x7f31640ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f3162ffd640 1 -- 172.21.15.60:0/3266907198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31640b6740 msgr2=0x7f31640c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f3162ffd640 1 --2- 172.21.15.60:0/3266907198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31640b6740 0x7f31640c5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f3162ffd640 1 -- 172.21.15.60:0/3266907198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31640c5c90 con 0x7f3150004810 2024-04-25T10:14:33.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f31627fc640 1 --2- 172.21.15.60:0/3266907198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31640b6740 0x7f31640c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:33.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f31637fe640 1 --2- 172.21.15.60:0/3266907198 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31640ce240 0x7f31640ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:33.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f3162ffd640 1 --2- 172.21.15.60:0/3266907198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3150004810 0x7f31641521a0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f314c00d7f0 tx=0x7f314c00dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f315b7fe640 1 -- 172.21.15.60:0/3266907198 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c019070 con 0x7f3150004810 2024-04-25T10:14:33.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f315b7fe640 1 -- 172.21.15.60:0/3266907198 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c0074e0 con 0x7f3150004810 2024-04-25T10:14:33.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f31695f2640 1 -- 172.21.15.60:0/3266907198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31640c5f70 con 0x7f3150004810 2024-04-25T10:14:33.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f315b7fe640 1 -- 172.21.15.60:0/3266907198 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c004a10 con 0x7f3150004810 2024-04-25T10:14:33.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f31695f2640 1 -- 172.21.15.60:0/3266907198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31640c6530 con 0x7f3150004810 2024-04-25T10:14:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f31695f2640 1 -- 172.21.15.60:0/3266907198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3164074770 con 0x7f3150004810 2024-04-25T10:14:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f315b7fe640 1 -- 172.21.15.60:0/3266907198 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f314c01c070 con 0x7f3150004810 2024-04-25T10:14:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f315b7fe640 1 --2- 172.21.15.60:0/3266907198 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3144066f90 0x7f3144069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f315b7fe640 1 -- 172.21.15.60:0/3266907198 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f314c084a40 con 0x7f3150004810 2024-04-25T10:14:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.475+0000 7f31627fc640 1 --2- 172.21.15.60:0/3266907198 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3144066f90 0x7f3144069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.479+0000 7f31627fc640 1 --2- 172.21.15.60:0/3266907198 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3144066f90 0x7f3144069450 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3154001750 tx=0x7f31540011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.479+0000 7f315b7fe640 1 -- 172.21.15.60:0/3266907198 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f316415f5a0 con 0x7f3150004810 2024-04-25T10:14:33.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.651+0000 7f31695f2640 1 -- 172.21.15.60:0/3266907198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f316409a640 con 0x7f3150004810 2024-04-25T10:14:33.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.651+0000 7f315b7fe640 1 -- 172.21.15.60:0/3266907198 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f314c084e20 con 0x7f3150004810 2024-04-25T10:14:33.657 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:33.657 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":97,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:14:17.788631+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:14.444411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:15.153386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:33.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.655+0000 7f31695f2640 1 -- 172.21.15.60:0/3266907198 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3144066f90 msgr2=0x7f3144069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.655+0000 7f31695f2640 1 --2- 172.21.15.60:0/3266907198 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3144066f90 0x7f3144069450 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3154001750 tx=0x7f31540011a0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.655+0000 7f31695f2640 1 -- 172.21.15.60:0/3266907198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3150004810 msgr2=0x7f31641521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.655+0000 7f31695f2640 1 --2- 172.21.15.60:0/3266907198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3150004810 0x7f31641521a0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f314c00d7f0 tx=0x7f314c00dcb0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.655+0000 7f31695f2640 1 -- 172.21.15.60:0/3266907198 shutdown_connections 2024-04-25T10:14:33.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.655+0000 7f31695f2640 1 --2- 172.21.15.60:0/3266907198 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3144066f90 0x7f3144069450 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.655+0000 7f31695f2640 1 --2- 172.21.15.60:0/3266907198 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31640ce240 0x7f31640ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.655+0000 7f31695f2640 1 --2- 172.21.15.60:0/3266907198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31640b6740 0x7f31640c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.655+0000 7f31695f2640 1 --2- 172.21.15.60:0/3266907198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3150004810 0x7f31641521a0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.655+0000 7f31695f2640 1 -- 172.21.15.60:0/3266907198 >> 172.21.15.60:0/3266907198 conn(0x7f31640b3320 msgr2=0x7f316409ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.655+0000 7f31695f2640 1 -- 172.21.15.60:0/3266907198 shutdown_connections 2024-04-25T10:14:33.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.655+0000 7f31695f2640 1 -- 172.21.15.60:0/3266907198 wait complete. 2024-04-25T10:14:33.676 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:14:14.444411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:14:15.153386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:33.677 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:33.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5f0fd9640 1 Processor -- start 2024-04-25T10:14:33.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5f0fd9640 1 -- start start 2024-04-25T10:14:33.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5f0fd9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5ec0cbcf0 0x7fc5ec160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5f0fd9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc5ec071c30 0x7fc5ec15f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5f0fd9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc5ec15fa20 0x7fc5ec15bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5f0fd9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ec0b2dc0 con 0x7fc5ec15fa20 2024-04-25T10:14:33.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5f0fd9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ec15fe80 con 0x7fc5ec071c30 2024-04-25T10:14:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5f0fd9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec144a40 con 0x7fc5ec0cbcf0 2024-04-25T10:14:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5ead9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc5ec15fa20 0x7fc5ec15bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5ead9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc5ec15fa20 0x7fc5ec15bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40552/0 (socket says 172.21.15.60:40552) 2024-04-25T10:14:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5ead9d640 1 -- 172.21.15.60:0/3901593001 learned_addr learned my addr 172.21.15.60:0/3901593001 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5ea59c640 1 --2- 172.21.15.60:0/3901593001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5ec0cbcf0 0x7fc5ec160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5e3fff640 1 --2- 172.21.15.60:0/3901593001 >> v2:172.21.15.100:3300/0 conn(0x7fc5ec071c30 0x7fc5ec15f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5ead9d640 1 -- 172.21.15.60:0/3901593001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5ec0cbcf0 msgr2=0x7fc5ec160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5ead9d640 1 --2- 172.21.15.60:0/3901593001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5ec0cbcf0 0x7fc5ec160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5ead9d640 1 -- 172.21.15.60:0/3901593001 >> v2:172.21.15.100:3300/0 conn(0x7fc5ec071c30 msgr2=0x7fc5ec15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5ead9d640 1 --2- 172.21.15.60:0/3901593001 >> v2:172.21.15.100:3300/0 conn(0x7fc5ec071c30 0x7fc5ec15f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.795+0000 7fc5ead9d640 1 -- 172.21.15.60:0/3901593001 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec0cc9f0 con 0x7fc5ec15fa20 2024-04-25T10:14:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5ead9d640 1 --2- 172.21.15.60:0/3901593001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5ec15fa20 0x7fc5ec15bf10 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec06f680 tx=0x7fc5e401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37df875f8918d20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5e9d9b640 1 -- 172.21.15.60:0/3901593001 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e40a5840 con 0x7fc5ec15fa20 2024-04-25T10:14:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5e9d9b640 1 -- 172.21.15.60:0/3901593001 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e40a59e0 con 0x7fc5ec15fa20 2024-04-25T10:14:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5e9d9b640 1 -- 172.21.15.60:0/3901593001 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e40a5db0 con 0x7fc5ec15fa20 2024-04-25T10:14:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 -- 172.21.15.60:0/3901593001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5ec15fa20 msgr2=0x7fc5ec15bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 --2- 172.21.15.60:0/3901593001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5ec15fa20 0x7fc5ec15bf10 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec06f680 tx=0x7fc5e401c9c0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 -- 172.21.15.60:0/3901593001 shutdown_connections 2024-04-25T10:14:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 --2- 172.21.15.60:0/3901593001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5ec15fa20 0x7fc5ec15bf10 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 --2- 172.21.15.60:0/3901593001 >> v2:172.21.15.100:3300/0 conn(0x7fc5ec071c30 0x7fc5ec15f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 --2- 172.21.15.60:0/3901593001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5ec0cbcf0 0x7fc5ec160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 -- 172.21.15.60:0/3901593001 >> 172.21.15.60:0/3901593001 conn(0x7fc5ec0670b0 msgr2=0x7fc5ec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 -- 172.21.15.60:0/3901593001 shutdown_connections 2024-04-25T10:14:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 -- 172.21.15.60:0/3901593001 wait complete. 2024-04-25T10:14:33.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 Processor -- start 2024-04-25T10:14:33.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 -- start start 2024-04-25T10:14:33.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5ec071c30 0x7fc5ec13e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5ec0cbcf0 0x7fc5ec13f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5ec15fa20 0x7fc5ec13fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec0ce740 con 0x7fc5ec0cbcf0 2024-04-25T10:14:33.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec0ce5c0 con 0x7fc5ec071c30 2024-04-25T10:14:33.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5f0fd9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec072010 con 0x7fc5ec15fa20 2024-04-25T10:14:33.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5e3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5ec0cbcf0 0x7fc5ec13f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5e3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5ec0cbcf0 0x7fc5ec13f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40556/0 (socket says 172.21.15.60:40556) 2024-04-25T10:14:33.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5e3fff640 1 -- 172.21.15.60:0/1084063582 learned_addr learned my addr 172.21.15.60:0/1084063582 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:33.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5ea59c640 1 --2- 172.21.15.60:0/1084063582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5ec071c30 0x7fc5ec13e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5e3fff640 1 -- 172.21.15.60:0/1084063582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5ec15fa20 msgr2=0x7fc5ec13fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5ead9d640 1 --2- 172.21.15.60:0/1084063582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5ec15fa20 0x7fc5ec13fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5e3fff640 1 --2- 172.21.15.60:0/1084063582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5ec15fa20 0x7fc5ec13fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5e3fff640 1 -- 172.21.15.60:0/1084063582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5ec071c30 msgr2=0x7fc5ec13e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5e3fff640 1 --2- 172.21.15.60:0/1084063582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5ec071c30 0x7fc5ec13e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5e3fff640 1 -- 172.21.15.60:0/1084063582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec071a60 con 0x7fc5ec0cbcf0 2024-04-25T10:14:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.799+0000 7fc5ea59c640 1 --2- 172.21.15.60:0/1084063582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5ec071c30 0x7fc5ec13e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.803+0000 7fc5e3fff640 1 --2- 172.21.15.60:0/1084063582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5ec0cbcf0 0x7fc5ec13f8d0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400d800 tx=0x7fc5d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.803+0000 7fc5e2ffd640 1 -- 172.21.15.60:0/1084063582 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4018020 con 0x7fc5ec0cbcf0 2024-04-25T10:14:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.803+0000 7fc5e2ffd640 1 -- 172.21.15.60:0/1084063582 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d40096a0 con 0x7fc5ec0cbcf0 2024-04-25T10:14:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.803+0000 7fc5f0fd9640 1 -- 172.21.15.60:0/1084063582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ec13f440 con 0x7fc5ec0cbcf0 2024-04-25T10:14:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.803+0000 7fc5e2ffd640 1 -- 172.21.15.60:0/1084063582 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4009a50 con 0x7fc5ec0cbcf0 2024-04-25T10:14:33.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.803+0000 7fc5f0fd9640 1 -- 172.21.15.60:0/1084063582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ec0afd90 con 0x7fc5ec0cbcf0 2024-04-25T10:14:33.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.803+0000 7fc5f0fd9640 1 -- 172.21.15.60:0/1084063582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ec074770 con 0x7fc5ec0cbcf0 2024-04-25T10:14:33.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.803+0000 7fc5e2ffd640 1 -- 172.21.15.60:0/1084063582 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5d401b020 con 0x7fc5ec0cbcf0 2024-04-25T10:14:33.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.803+0000 7fc5e2ffd640 1 --2- 172.21.15.60:0/1084063582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc5cc066f60 0x7fc5cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.803+0000 7fc5e2ffd640 1 -- 172.21.15.60:0/1084063582 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5d4085610 con 0x7fc5ec0cbcf0 2024-04-25T10:14:33.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.803+0000 7fc5ea59c640 1 --2- 172.21.15.60:0/1084063582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc5cc066f60 0x7fc5cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.803+0000 7fc5ea59c640 1 --2- 172.21.15.60:0/1084063582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc5cc066f60 0x7fc5cc069420 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec15f1a0 tx=0x7fc5dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.807+0000 7fc5e2ffd640 1 -- 172.21.15.60:0/1084063582 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5d4022020 con 0x7fc5ec0cbcf0 2024-04-25T10:14:33.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.967+0000 7fc5f0fd9640 1 -- 172.21.15.60:0/1084063582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5ec15be20 con 0x7fc5ec0cbcf0 2024-04-25T10:14:33.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.967+0000 7fc5e2ffd640 1 -- 172.21.15.60:0/1084063582 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5ec13f440 con 0x7fc5ec0cbcf0 2024-04-25T10:14:33.973 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:14:33.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.971+0000 7fc5f0fd9640 1 -- 172.21.15.60:0/1084063582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc5cc066f60 msgr2=0x7fc5cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.971+0000 7fc5f0fd9640 1 --2- 172.21.15.60:0/1084063582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc5cc066f60 0x7fc5cc069420 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec15f1a0 tx=0x7fc5dc008040 comp rx=0 tx=0).stop 2024-04-25T10:14:33.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.971+0000 7fc5f0fd9640 1 -- 172.21.15.60:0/1084063582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5ec0cbcf0 msgr2=0x7fc5ec13f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.971+0000 7fc5f0fd9640 1 --2- 172.21.15.60:0/1084063582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5ec0cbcf0 0x7fc5ec13f8d0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400d800 tx=0x7fc5d400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.971+0000 7fc5f0fd9640 1 -- 172.21.15.60:0/1084063582 shutdown_connections 2024-04-25T10:14:33.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.971+0000 7fc5f0fd9640 1 --2- 172.21.15.60:0/1084063582 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc5cc066f60 0x7fc5cc069420 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.971+0000 7fc5f0fd9640 1 --2- 172.21.15.60:0/1084063582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5ec15fa20 0x7fc5ec13fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.971+0000 7fc5f0fd9640 1 --2- 172.21.15.60:0/1084063582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5ec0cbcf0 0x7fc5ec13f8d0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.971+0000 7fc5f0fd9640 1 --2- 172.21.15.60:0/1084063582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5ec071c30 0x7fc5ec13e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.971+0000 7fc5f0fd9640 1 -- 172.21.15.60:0/1084063582 >> 172.21.15.60:0/1084063582 conn(0x7fc5ec0670b0 msgr2=0x7fc5ec1436e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.971+0000 7fc5f0fd9640 1 -- 172.21.15.60:0/1084063582 shutdown_connections 2024-04-25T10:14:33.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:33.971+0000 7fc5f0fd9640 1 -- 172.21.15.60:0/1084063582 wait complete. 2024-04-25T10:14:33.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8749f01640 1 Processor -- start 2024-04-25T10:14:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8749f01640 1 -- start start 2024-04-25T10:14:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8749f01640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f874415ef20 0x7f87440b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8749f01640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f87440cfcf0 0x7f87440b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8749f01640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8744099a30 0x7f8744099ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8749f01640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87440cc320 con 0x7f87440cfcf0 2024-04-25T10:14:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8749f01640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87440cc5c0 con 0x7f874415ef20 2024-04-25T10:14:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8749f01640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87440d07b0 con 0x7f8744099a30 2024-04-25T10:14:34.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8742ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f87440cfcf0 0x7f87440b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f87437fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f874415ef20 0x7f87440b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8742ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f87440cfcf0 0x7f87440b4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40566/0 (socket says 172.21.15.60:40566) 2024-04-25T10:14:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8742ffd640 1 -- 172.21.15.60:0/2491143380 learned_addr learned my addr 172.21.15.60:0/2491143380 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f87437fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f874415ef20 0x7f87440b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:45762/0 (socket says 172.21.15.60:45762) 2024-04-25T10:14:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8743fff640 1 --2- 172.21.15.60:0/2491143380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8744099a30 0x7f8744099ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8742ffd640 1 -- 172.21.15.60:0/2491143380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8744099a30 msgr2=0x7f8744099ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8742ffd640 1 --2- 172.21.15.60:0/2491143380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8744099a30 0x7f8744099ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8742ffd640 1 -- 172.21.15.60:0/2491143380 >> v2:172.21.15.100:3300/0 conn(0x7f874415ef20 msgr2=0x7f87440b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8742ffd640 1 --2- 172.21.15.60:0/2491143380 >> v2:172.21.15.100:3300/0 conn(0x7f874415ef20 0x7f87440b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.123+0000 7f8742ffd640 1 -- 172.21.15.60:0/2491143380 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87440d04d0 con 0x7f87440cfcf0 2024-04-25T10:14:34.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8742ffd640 1 --2- 172.21.15.60:0/2491143380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87440cfcf0 0x7f87440b4560 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f87440b3da0 tx=0x7f873801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4485fc4c5f09620 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f87427fc640 1 -- 172.21.15.60:0/2491143380 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87380b0070 con 0x7f87440cfcf0 2024-04-25T10:14:34.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f87427fc640 1 -- 172.21.15.60:0/2491143380 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87380ab600 con 0x7f87440cfcf0 2024-04-25T10:14:34.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f87427fc640 1 -- 172.21.15.60:0/2491143380 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87380ab9d0 con 0x7f87440cfcf0 2024-04-25T10:14:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 -- 172.21.15.60:0/2491143380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87440cfcf0 msgr2=0x7f87440b4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 --2- 172.21.15.60:0/2491143380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87440cfcf0 0x7f87440b4560 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f87440b3da0 tx=0x7f873801c520 comp rx=0 tx=0).stop 2024-04-25T10:14:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 -- 172.21.15.60:0/2491143380 shutdown_connections 2024-04-25T10:14:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 --2- 172.21.15.60:0/2491143380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8744099a30 0x7f8744099ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 --2- 172.21.15.60:0/2491143380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87440cfcf0 0x7f87440b4560 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 --2- 172.21.15.60:0/2491143380 >> v2:172.21.15.100:3300/0 conn(0x7f874415ef20 0x7f87440b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 -- 172.21.15.60:0/2491143380 >> 172.21.15.60:0/2491143380 conn(0x7f87440b3200 msgr2=0x7f87440b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 -- 172.21.15.60:0/2491143380 shutdown_connections 2024-04-25T10:14:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 -- 172.21.15.60:0/2491143380 wait complete. 2024-04-25T10:14:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 Processor -- start 2024-04-25T10:14:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 -- start start 2024-04-25T10:14:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8744099a30 0x7f874415d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87440cfcf0 0x7f87441521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f874415ef20 0x7f87440cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874415f300 con 0x7f8744099a30 2024-04-25T10:14:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87440d0170 con 0x7f874415ef20 2024-04-25T10:14:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8749f01640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874415f480 con 0x7f87440cfcf0 2024-04-25T10:14:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f87437fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8744099a30 0x7f874415d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f87437fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8744099a30 0x7f874415d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40576/0 (socket says 172.21.15.60:40576) 2024-04-25T10:14:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f87437fe640 1 -- 172.21.15.60:0/1504204038 learned_addr learned my addr 172.21.15.60:0/1504204038 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8742ffd640 1 --2- 172.21.15.60:0/1504204038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87440cfcf0 0x7f87441521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f8743fff640 1 --2- 172.21.15.60:0/1504204038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f874415ef20 0x7f87440cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f87437fe640 1 -- 172.21.15.60:0/1504204038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87440cfcf0 msgr2=0x7f87441521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f87437fe640 1 --2- 172.21.15.60:0/1504204038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87440cfcf0 0x7f87441521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f87437fe640 1 -- 172.21.15.60:0/1504204038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f874415ef20 msgr2=0x7f87440cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f87437fe640 1 --2- 172.21.15.60:0/1504204038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f874415ef20 0x7f87440cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.127+0000 7f87437fe640 1 -- 172.21.15.60:0/1504204038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8744154320 con 0x7f8744099a30 2024-04-25T10:14:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.131+0000 7f87437fe640 1 --2- 172.21.15.60:0/1504204038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8744099a30 0x7f874415d920 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f873400e870 tx=0x7f873400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.131+0000 7f872bfff640 1 -- 172.21.15.60:0/1504204038 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8734017070 con 0x7f8744099a30 2024-04-25T10:14:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.131+0000 7f872bfff640 1 -- 172.21.15.60:0/1504204038 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873400c8e0 con 0x7f8744099a30 2024-04-25T10:14:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.131+0000 7f872bfff640 1 -- 172.21.15.60:0/1504204038 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873400cc90 con 0x7f8744099a30 2024-04-25T10:14:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.131+0000 7f8749f01640 1 -- 172.21.15.60:0/1504204038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8744154600 con 0x7f8744099a30 2024-04-25T10:14:34.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.131+0000 7f8749f01640 1 -- 172.21.15.60:0/1504204038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8744154bc0 con 0x7f8744099a30 2024-04-25T10:14:34.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.131+0000 7f8749f01640 1 -- 172.21.15.60:0/1504204038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8744074770 con 0x7f8744099a30 2024-04-25T10:14:34.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.131+0000 7f872bfff640 1 -- 172.21.15.60:0/1504204038 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f873401a070 con 0x7f8744099a30 2024-04-25T10:14:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.131+0000 7f872bfff640 1 --2- 172.21.15.60:0/1504204038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8730066f90 0x7f8730069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.131+0000 7f872bfff640 1 -- 172.21.15.60:0/1504204038 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8734085190 con 0x7f8744099a30 2024-04-25T10:14:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.131+0000 7f8742ffd640 1 --2- 172.21.15.60:0/1504204038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8730066f90 0x7f8730069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.131+0000 7f8742ffd640 1 --2- 172.21.15.60:0/1504204038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8730066f90 0x7f8730069450 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f87380ad000 tx=0x7f8738008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.135+0000 7f872bfff640 1 -- 172.21.15.60:0/1504204038 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8734056230 con 0x7f8744099a30 2024-04-25T10:14:34.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.295+0000 7f8749f01640 1 -- 172.21.15.60:0/1504204038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87440b4070 con 0x7f8744099a30 2024-04-25T10:14:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.295+0000 7f872bfff640 1 -- 172.21.15.60:0/1504204038 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8734055b30 con 0x7f8744099a30 2024-04-25T10:14:34.301 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:14:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.299+0000 7f8749f01640 1 -- 172.21.15.60:0/1504204038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8730066f90 msgr2=0x7f8730069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.299+0000 7f8749f01640 1 --2- 172.21.15.60:0/1504204038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8730066f90 0x7f8730069450 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f87380ad000 tx=0x7f8738008040 comp rx=0 tx=0).stop 2024-04-25T10:14:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.299+0000 7f8749f01640 1 -- 172.21.15.60:0/1504204038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8744099a30 msgr2=0x7f874415d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.299+0000 7f8749f01640 1 --2- 172.21.15.60:0/1504204038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8744099a30 0x7f874415d920 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f873400e870 tx=0x7f873400ec70 comp rx=0 tx=0).stop 2024-04-25T10:14:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.299+0000 7f8749f01640 1 -- 172.21.15.60:0/1504204038 shutdown_connections 2024-04-25T10:14:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.299+0000 7f8749f01640 1 --2- 172.21.15.60:0/1504204038 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8730066f90 0x7f8730069450 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.299+0000 7f8749f01640 1 --2- 172.21.15.60:0/1504204038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f874415ef20 0x7f87440cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.299+0000 7f8749f01640 1 --2- 172.21.15.60:0/1504204038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87440cfcf0 0x7f87441521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.299+0000 7f8749f01640 1 --2- 172.21.15.60:0/1504204038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8744099a30 0x7f874415d920 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.299+0000 7f8749f01640 1 -- 172.21.15.60:0/1504204038 >> 172.21.15.60:0/1504204038 conn(0x7f87440b3200 msgr2=0x7f87440cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.299+0000 7f8749f01640 1 -- 172.21.15.60:0/1504204038 shutdown_connections 2024-04-25T10:14:34.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.299+0000 7f8749f01640 1 -- 172.21.15.60:0/1504204038 wait complete. 2024-04-25T10:14:34.321 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49db3ee640 1 Processor -- start 2024-04-25T10:14:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49db3ee640 1 -- start start 2024-04-25T10:14:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49db3ee640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f49d40cbcf0 0x7f49d4160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49db3ee640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f49d4071c30 0x7f49d415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49db3ee640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d4144a40 con 0x7f49d415fa20 2024-04-25T10:14:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49db3ee640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d40b2dc0 con 0x7f49d4071c30 2024-04-25T10:14:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49db3ee640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d40cc9f0 con 0x7f49d40cbcf0 2024-04-25T10:14:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49d8989640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f49d4071c30 0x7f49d415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49d998b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f49d415fa20 0x7f49d415bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53044/0 (socket says 172.21.15.60:53044) 2024-04-25T10:14:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49d8989640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f49d4071c30 0x7f49d415f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40584/0 (socket says 172.21.15.60:40584) 2024-04-25T10:14:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49d998b640 1 -- 172.21.15.60:0/2287916305 learned_addr learned my addr 172.21.15.60:0/2287916305 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49d918a640 1 --2- 172.21.15.60:0/2287916305 >> v2:172.21.15.100:3300/0 conn(0x7f49d40cbcf0 0x7f49d4160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49d8989640 1 -- 172.21.15.60:0/2287916305 >> v2:172.21.15.100:3300/0 conn(0x7f49d40cbcf0 msgr2=0x7f49d4160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49d8989640 1 --2- 172.21.15.60:0/2287916305 >> v2:172.21.15.100:3300/0 conn(0x7f49d40cbcf0 0x7f49d4160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49d8989640 1 -- 172.21.15.60:0/2287916305 >> v1:172.21.15.60:6789/0 conn(0x7f49d415fa20 legacy=0x7f49d415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49d8989640 1 -- 172.21.15.60:0/2287916305 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d40cccd0 con 0x7f49d4071c30 2024-04-25T10:14:34.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.447+0000 7f49cbfff640 1 -- 172.21.15.60:0/2287916305 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780196528 0 0) 0x7f49d4144a40 con 0x7f49d415fa20 2024-04-25T10:14:34.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.451+0000 7f49d8989640 1 --2- 172.21.15.60:0/2287916305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49d4071c30 0x7f49d415f4e0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f49d40b4c50 tx=0x7f49c401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8957a2140b476e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.451+0000 7f49cbfff640 1 -- 172.21.15.60:0/2287916305 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c40b0070 con 0x7f49d4071c30 2024-04-25T10:14:34.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.451+0000 7f49cbfff640 1 -- 172.21.15.60:0/2287916305 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c40ab600 con 0x7f49d4071c30 2024-04-25T10:14:34.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.451+0000 7f49cbfff640 1 -- 172.21.15.60:0/2287916305 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c40ab9d0 con 0x7f49d4071c30 2024-04-25T10:14:34.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.451+0000 7f49db3ee640 1 -- 172.21.15.60:0/2287916305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49d4071c30 msgr2=0x7f49d415f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.451+0000 7f49db3ee640 1 --2- 172.21.15.60:0/2287916305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49d4071c30 0x7f49d415f4e0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f49d40b4c50 tx=0x7f49c401c480 comp rx=0 tx=0).stop 2024-04-25T10:14:34.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.451+0000 7f49db3ee640 1 -- 172.21.15.60:0/2287916305 shutdown_connections 2024-04-25T10:14:34.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.451+0000 7f49db3ee640 1 --2- 172.21.15.60:0/2287916305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49d4071c30 0x7f49d415f4e0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.451+0000 7f49db3ee640 1 --2- 172.21.15.60:0/2287916305 >> v2:172.21.15.100:3300/0 conn(0x7f49d40cbcf0 0x7f49d4160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.451+0000 7f49db3ee640 1 -- 172.21.15.60:0/2287916305 >> 172.21.15.60:0/2287916305 conn(0x7f49d40670b0 msgr2=0x7f49d406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:34.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.451+0000 7f49db3ee640 1 -- 172.21.15.60:0/2287916305 shutdown_connections 2024-04-25T10:14:34.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.451+0000 7f49db3ee640 1 -- 172.21.15.60:0/2287916305 wait complete. 2024-04-25T10:14:34.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49db3ee640 1 Processor -- start 2024-04-25T10:14:34.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49db3ee640 1 -- start start 2024-04-25T10:14:34.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49db3ee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49d4071c30 0x7f49d40b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49db3ee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d40cbcf0 0x7f49d40aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49db3ee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49d415fa20 0x7f49d415a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49db3ee640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d40cfed0 con 0x7f49d415fa20 2024-04-25T10:14:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49db3ee640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d40720b0 con 0x7f49d40cbcf0 2024-04-25T10:14:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49db3ee640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d413f870 con 0x7f49d4071c30 2024-04-25T10:14:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49d998b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49d415fa20 0x7f49d415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49d998b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49d415fa20 0x7f49d415a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40588/0 (socket says 172.21.15.60:40588) 2024-04-25T10:14:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49d998b640 1 -- 172.21.15.60:0/1133826151 learned_addr learned my addr 172.21.15.60:0/1133826151 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49d918a640 1 --2- 172.21.15.60:0/1133826151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49d4071c30 0x7f49d40b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49d8989640 1 --2- 172.21.15.60:0/1133826151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d40cbcf0 0x7f49d40aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49d998b640 1 -- 172.21.15.60:0/1133826151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49d4071c30 msgr2=0x7f49d40b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49d998b640 1 --2- 172.21.15.60:0/1133826151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49d4071c30 0x7f49d40b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49d998b640 1 -- 172.21.15.60:0/1133826151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d40cbcf0 msgr2=0x7f49d40aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49d998b640 1 --2- 172.21.15.60:0/1133826151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d40cbcf0 0x7f49d40aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49d998b640 1 -- 172.21.15.60:0/1133826151 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d406d3f0 con 0x7f49d415fa20 2024-04-25T10:14:34.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49d998b640 1 --2- 172.21.15.60:0/1133826151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49d415fa20 0x7f49d415a780 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00c7a0 tx=0x7f49cc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49c9ffb640 1 -- 172.21.15.60:0/1133826151 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc018070 con 0x7f49d415fa20 2024-04-25T10:14:34.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49c9ffb640 1 -- 172.21.15.60:0/1133826151 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc00a640 con 0x7f49d415fa20 2024-04-25T10:14:34.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49c9ffb640 1 -- 172.21.15.60:0/1133826151 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc00aab0 con 0x7f49d415fa20 2024-04-25T10:14:34.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49db3ee640 1 -- 172.21.15.60:0/1133826151 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d413f2d0 con 0x7f49d415fa20 2024-04-25T10:14:34.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49db3ee640 1 -- 172.21.15.60:0/1133826151 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d40b3f80 con 0x7f49d415fa20 2024-04-25T10:14:34.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.455+0000 7f49db3ee640 1 -- 172.21.15.60:0/1133826151 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d406bba0 con 0x7f49d415fa20 2024-04-25T10:14:34.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.459+0000 7f49c9ffb640 1 -- 172.21.15.60:0/1133826151 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49cc01b070 con 0x7f49d415fa20 2024-04-25T10:14:34.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.459+0000 7f49c9ffb640 1 --2- 172.21.15.60:0/1133826151 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f49b8066f60 0x7f49b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.459+0000 7f49c9ffb640 1 -- 172.21.15.60:0/1133826151 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49cc084590 con 0x7f49d415fa20 2024-04-25T10:14:34.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.459+0000 7f49d918a640 1 --2- 172.21.15.60:0/1133826151 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f49b8066f60 0x7f49b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.459+0000 7f49d918a640 1 --2- 172.21.15.60:0/1133826151 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f49b8066f60 0x7f49b8069420 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f49d000b9d0 tx=0x7f49d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.463+0000 7f49c9ffb640 1 -- 172.21.15.60:0/1133826151 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f49cc055630 con 0x7f49d415fa20 2024-04-25T10:14:34.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.623+0000 7f49db3ee640 1 -- 172.21.15.60:0/1133826151 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f49d4074770 con 0x7f49d415fa20 2024-04-25T10:14:34.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.623+0000 7f49c9ffb640 1 -- 172.21.15.60:0/1133826151 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+11 (secure 0 0 0) 0x7f49cc054f30 con 0x7f49d415fa20 2024-04-25T10:14:34.627 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:14:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.623+0000 7f49db3ee640 1 -- 172.21.15.60:0/1133826151 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f49b8066f60 msgr2=0x7f49b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.623+0000 7f49db3ee640 1 --2- 172.21.15.60:0/1133826151 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f49b8066f60 0x7f49b8069420 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f49d000b9d0 tx=0x7f49d0008040 comp rx=0 tx=0).stop 2024-04-25T10:14:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.623+0000 7f49db3ee640 1 -- 172.21.15.60:0/1133826151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49d415fa20 msgr2=0x7f49d415a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.623+0000 7f49db3ee640 1 --2- 172.21.15.60:0/1133826151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49d415fa20 0x7f49d415a780 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00c7a0 tx=0x7f49cc00cc60 comp rx=0 tx=0).stop 2024-04-25T10:14:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.623+0000 7f49db3ee640 1 -- 172.21.15.60:0/1133826151 shutdown_connections 2024-04-25T10:14:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.623+0000 7f49db3ee640 1 --2- 172.21.15.60:0/1133826151 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f49b8066f60 0x7f49b8069420 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.623+0000 7f49db3ee640 1 --2- 172.21.15.60:0/1133826151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49d415fa20 0x7f49d415a780 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.623+0000 7f49db3ee640 1 --2- 172.21.15.60:0/1133826151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d40cbcf0 0x7f49d40aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.623+0000 7f49db3ee640 1 --2- 172.21.15.60:0/1133826151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49d4071c30 0x7f49d40b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.623+0000 7f49db3ee640 1 -- 172.21.15.60:0/1133826151 >> 172.21.15.60:0/1133826151 conn(0x7f49d40670b0 msgr2=0x7f49d4071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.623+0000 7f49db3ee640 1 -- 172.21.15.60:0/1133826151 shutdown_connections 2024-04-25T10:14:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.623+0000 7f49db3ee640 1 -- 172.21.15.60:0/1133826151 wait complete. 2024-04-25T10:14:34.649 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:14:34.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd3af317640 1 Processor -- start 2024-04-25T10:14:34.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd3af317640 1 -- start start 2024-04-25T10:14:34.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd3af317640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd3a815a7a0 0x7fd3a80cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd3af317640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3a806ec30 0x7fd3a8071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd3af317640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a80721e0 con 0x7fd3a80b37e0 2024-04-25T10:14:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd3af317640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a80cdaa0 con 0x7fd3a815a7a0 2024-04-25T10:14:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd3af317640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a806d2f0 con 0x7fd3a806ec30 2024-04-25T10:14:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd3ad8b4640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd3a80b37e0 0x7fd3a806dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53056/0 (socket says 172.21.15.60:53056) 2024-04-25T10:14:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd3ad8b4640 1 -- 172.21.15.60:0/47086441 learned_addr learned my addr 172.21.15.60:0/47086441 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd3ad0b3640 1 --2- 172.21.15.60:0/47086441 >> v2:172.21.15.100:3300/0 conn(0x7fd3a815a7a0 0x7fd3a80cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd3ac8b2640 1 --2- 172.21.15.60:0/47086441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3a806ec30 0x7fd3a8071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd39ffff640 1 -- 172.21.15.60:0/47086441 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926627166 0 0) 0x7fd3a80721e0 con 0x7fd3a80b37e0 2024-04-25T10:14:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd39ffff640 1 -- 172.21.15.60:0/47086441 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd394003530 con 0x7fd3a80b37e0 2024-04-25T10:14:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd39ffff640 1 -- 172.21.15.60:0/47086441 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2928611211 0 0) 0x7fd394003530 con 0x7fd3a80b37e0 2024-04-25T10:14:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd39ffff640 1 -- 172.21.15.60:0/47086441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3a806ec30 msgr2=0x7fd3a8071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd39ffff640 1 --2- 172.21.15.60:0/47086441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3a806ec30 0x7fd3a8071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd39ffff640 1 -- 172.21.15.60:0/47086441 >> v2:172.21.15.100:3300/0 conn(0x7fd3a815a7a0 msgr2=0x7fd3a80cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd39ffff640 1 --2- 172.21.15.60:0/47086441 >> v2:172.21.15.100:3300/0 conn(0x7fd3a815a7a0 0x7fd3a80cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd39ffff640 1 -- 172.21.15.60:0/47086441 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a806f130 con 0x7fd3a80b37e0 2024-04-25T10:14:34.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd39ffff640 1 -- 172.21.15.60:0/47086441 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fd3a0001a00 con 0x7fd3a80b37e0 2024-04-25T10:14:34.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd39ffff640 1 -- 172.21.15.60:0/47086441 >> v1:172.21.15.60:6789/0 conn(0x7fd3a80b37e0 legacy=0x7fd3a806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd39ffff640 1 --2- 172.21.15.60:0/47086441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3940039c0 0x7fd394003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd39ffff640 1 --2- 172.21.15.60:0/47086441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3a806ec30 0x7fd394004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd39ffff640 1 --2- 172.21.15.60:0/47086441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd394004d50 0x7fd3940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd39ffff640 1 -- 172.21.15.60:0/47086441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a806d2f0 con 0x7fd3940039c0 2024-04-25T10:14:34.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd39ffff640 1 -- 172.21.15.60:0/47086441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39400b9d0 con 0x7fd394004d50 2024-04-25T10:14:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd39ffff640 1 -- 172.21.15.60:0/47086441 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39400bb50 con 0x7fd3a806ec30 2024-04-25T10:14:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.771+0000 7fd3ad0b3640 1 --2- 172.21.15.60:0/47086441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3940039c0 0x7fd394003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.775+0000 7fd3ad0b3640 1 -- 172.21.15.60:0/47086441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3a806ec30 msgr2=0x7fd394004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.775+0000 7fd3ad0b3640 1 --2- 172.21.15.60:0/47086441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3a806ec30 0x7fd394004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.775+0000 7fd3ad0b3640 1 -- 172.21.15.60:0/47086441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd394004d50 msgr2=0x7fd3940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.775+0000 7fd3ad0b3640 1 --2- 172.21.15.60:0/47086441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd394004d50 0x7fd3940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.775+0000 7fd3ad0b3640 1 -- 172.21.15.60:0/47086441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd394003530 con 0x7fd3940039c0 2024-04-25T10:14:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.775+0000 7fd3ad0b3640 1 --2- 172.21.15.60:0/47086441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3940039c0 0x7fd394003de0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009b10 tx=0x7fd3a401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342738e1a5728e7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.775+0000 7fd39ffff640 1 -- 172.21.15.60:0/47086441 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a40b2070 con 0x7fd3940039c0 2024-04-25T10:14:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.775+0000 7fd39ffff640 1 -- 172.21.15.60:0/47086441 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a40a5590 con 0x7fd3940039c0 2024-04-25T10:14:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.775+0000 7fd39ffff640 1 -- 172.21.15.60:0/47086441 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a40096b0 con 0x7fd3940039c0 2024-04-25T10:14:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.775+0000 7fd3af317640 1 -- 172.21.15.60:0/47086441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3940039c0 msgr2=0x7fd394003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.775+0000 7fd3af317640 1 --2- 172.21.15.60:0/47086441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3940039c0 0x7fd394003de0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009b10 tx=0x7fd3a401cab0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.775+0000 7fd3ad8b4640 1 -- 172.21.15.60:0/47086441 reap_dead start 2024-04-25T10:14:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.775+0000 7fd3af317640 1 -- 172.21.15.60:0/47086441 shutdown_connections 2024-04-25T10:14:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.775+0000 7fd3af317640 1 -- 172.21.15.60:0/47086441 >> 172.21.15.60:0/47086441 conn(0x7fd3a80b12d0 msgr2=0x7fd3a80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.775+0000 7fd3af317640 1 -- 172.21.15.60:0/47086441 shutdown_connections 2024-04-25T10:14:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.775+0000 7fd3af317640 1 -- 172.21.15.60:0/47086441 wait complete. 2024-04-25T10:14:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3af317640 1 Processor -- start 2024-04-25T10:14:34.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3af317640 1 -- start start 2024-04-25T10:14:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3af317640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3940039c0 0x7fd3a814c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3af317640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a815f790 0x7fd3a8154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3af317640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3a813c840 0x7fd3a813ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3af317640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a815ab80 con 0x7fd3a815f790 2024-04-25T10:14:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3af317640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a806d8b0 con 0x7fd3940039c0 2024-04-25T10:14:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3af317640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a815ad00 con 0x7fd3a813c840 2024-04-25T10:14:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3ac8b2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a815f790 0x7fd3a8154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3ac8b2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a815f790 0x7fd3a8154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40604/0 (socket says 172.21.15.60:40604) 2024-04-25T10:14:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3ac8b2640 1 -- 172.21.15.60:0/1384688467 learned_addr learned my addr 172.21.15.60:0/1384688467 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3ad8b4640 1 --2- 172.21.15.60:0/1384688467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3a813c840 0x7fd3a813ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3ad0b3640 1 --2- 172.21.15.60:0/1384688467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3940039c0 0x7fd3a814c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3ac8b2640 1 -- 172.21.15.60:0/1384688467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3a813c840 msgr2=0x7fd3a813ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3ac8b2640 1 --2- 172.21.15.60:0/1384688467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3a813c840 0x7fd3a813ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3ac8b2640 1 -- 172.21.15.60:0/1384688467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3940039c0 msgr2=0x7fd3a814c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3ac8b2640 1 --2- 172.21.15.60:0/1384688467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3940039c0 0x7fd3a814c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3ac8b2640 1 -- 172.21.15.60:0/1384688467 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a815ff00 con 0x7fd3a815f790 2024-04-25T10:14:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3ac8b2640 1 --2- 172.21.15.60:0/1384688467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a815f790 0x7fd3a8154640 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fd39800e7f0 tx=0x7fd39800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd39dffb640 1 -- 172.21.15.60:0/1384688467 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd398017070 con 0x7fd3a815f790 2024-04-25T10:14:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd39dffb640 1 -- 172.21.15.60:0/1384688467 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39800c940 con 0x7fd3a815f790 2024-04-25T10:14:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd39dffb640 1 -- 172.21.15.60:0/1384688467 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39800cd30 con 0x7fd3a815f790 2024-04-25T10:14:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3af317640 1 -- 172.21.15.60:0/1384688467 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a81601e0 con 0x7fd3a815f790 2024-04-25T10:14:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3af317640 1 -- 172.21.15.60:0/1384688467 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a81607a0 con 0x7fd3a815f790 2024-04-25T10:14:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd3af317640 1 -- 172.21.15.60:0/1384688467 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a8074770 con 0x7fd3a815f790 2024-04-25T10:14:34.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.779+0000 7fd39dffb640 1 -- 172.21.15.60:0/1384688467 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd39801a070 con 0x7fd3a815f790 2024-04-25T10:14:34.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.783+0000 7fd39dffb640 1 --2- 172.21.15.60:0/1384688467 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd38c066f90 0x7fd38c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.783+0000 7fd39dffb640 1 -- 172.21.15.60:0/1384688467 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd398085340 con 0x7fd3a815f790 2024-04-25T10:14:34.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.783+0000 7fd3ad0b3640 1 --2- 172.21.15.60:0/1384688467 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd38c066f90 0x7fd38c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.783+0000 7fd3ad0b3640 1 --2- 172.21.15.60:0/1384688467 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd38c066f90 0x7fd38c069450 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40af000 tx=0x7fd3a40046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.787+0000 7fd39dffb640 1 -- 172.21.15.60:0/1384688467 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd3980562b0 con 0x7fd3a815f790 2024-04-25T10:14:34.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.959+0000 7fd3af317640 1 -- 172.21.15.60:0/1384688467 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd3a81612a0 con 0x7fd3a815f790 2024-04-25T10:14:34.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.963+0000 7fd39dffb640 1 -- 172.21.15.60:0/1384688467 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v97) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd398055bb0 con 0x7fd3a815f790 2024-04-25T10:14:34.966 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:14:34.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.963+0000 7fd3af317640 1 -- 172.21.15.60:0/1384688467 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd38c066f90 msgr2=0x7fd38c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.963+0000 7fd3af317640 1 --2- 172.21.15.60:0/1384688467 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd38c066f90 0x7fd38c069450 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40af000 tx=0x7fd3a40046d0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.963+0000 7fd3af317640 1 -- 172.21.15.60:0/1384688467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a815f790 msgr2=0x7fd3a8154640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.963+0000 7fd3af317640 1 --2- 172.21.15.60:0/1384688467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a815f790 0x7fd3a8154640 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fd39800e7f0 tx=0x7fd39800ecb0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.963+0000 7fd3af317640 1 -- 172.21.15.60:0/1384688467 shutdown_connections 2024-04-25T10:14:34.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.963+0000 7fd3af317640 1 --2- 172.21.15.60:0/1384688467 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd38c066f90 0x7fd38c069450 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.963+0000 7fd3af317640 1 --2- 172.21.15.60:0/1384688467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3a813c840 0x7fd3a813ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.963+0000 7fd3af317640 1 --2- 172.21.15.60:0/1384688467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3a815f790 0x7fd3a8154640 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.963+0000 7fd3af317640 1 --2- 172.21.15.60:0/1384688467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3940039c0 0x7fd3a814c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.963+0000 7fd3af317640 1 -- 172.21.15.60:0/1384688467 >> 172.21.15.60:0/1384688467 conn(0x7fd3a80b12d0 msgr2=0x7fd3a80b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:34.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.963+0000 7fd3af317640 1 -- 172.21.15.60:0/1384688467 shutdown_connections 2024-04-25T10:14:34.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:34.963+0000 7fd3af317640 1 -- 172.21.15.60:0/1384688467 wait complete. 2024-04-25T10:14:34.986 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:14:34.986 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:34.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:35.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8b06aba640 1 Processor -- start 2024-04-25T10:14:35.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8b06aba640 1 -- start start 2024-04-25T10:14:35.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8b06aba640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8b000ce7a0 0x7f8b000ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8b06aba640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b000b33f0 con 0x7f8b0015bc90 2024-04-25T10:14:35.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8b06aba640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b000cf0c0 con 0x7f8b000b4660 2024-04-25T10:14:35.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8b06aba640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b000cc370 con 0x7f8b000ce7a0 2024-04-25T10:14:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8afffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8b0015bc90 0x7f8b000b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53058/0 (socket says 172.21.15.60:53058) 2024-04-25T10:14:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8afffff640 1 -- 172.21.15.60:0/481000023 learned_addr learned my addr 172.21.15.60:0/481000023 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8b04856640 1 --2- 172.21.15.60:0/481000023 >> v2:172.21.15.100:3300/0 conn(0x7f8b000ce7a0 0x7f8b000ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8aff7fe640 1 -- 172.21.15.60:0/481000023 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266229666 0 0) 0x7f8b000b33f0 con 0x7f8b0015bc90 2024-04-25T10:14:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8aff7fe640 1 -- 172.21.15.60:0/481000023 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ae8003530 con 0x7f8b0015bc90 2024-04-25T10:14:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8aff7fe640 1 -- 172.21.15.60:0/481000023 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 279061397 0 0) 0x7f8ae8003530 con 0x7f8b0015bc90 2024-04-25T10:14:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8aff7fe640 1 -- 172.21.15.60:0/481000023 >> v2:172.21.15.100:3300/0 conn(0x7f8b000ce7a0 msgr2=0x7f8b000ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8aff7fe640 1 --2- 172.21.15.60:0/481000023 >> v2:172.21.15.100:3300/0 conn(0x7f8b000ce7a0 0x7f8b000ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8aff7fe640 1 -- 172.21.15.60:0/481000023 >> v1:172.21.15.100:6789/0 conn(0x7f8b000b4660 legacy=0x7f8b000b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8aff7fe640 1 -- 172.21.15.60:0/481000023 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b00071ca0 con 0x7f8b0015bc90 2024-04-25T10:14:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8aff7fe640 1 -- 172.21.15.60:0/481000023 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f8af4001680 con 0x7f8b0015bc90 2024-04-25T10:14:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8aff7fe640 1 -- 172.21.15.60:0/481000023 >> v1:172.21.15.60:6789/0 conn(0x7f8b0015bc90 legacy=0x7f8b000b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8aff7fe640 1 --2- 172.21.15.60:0/481000023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ae80039c0 0x7f8ae8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8aff7fe640 1 --2- 172.21.15.60:0/481000023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ae8004810 0x7f8ae8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8aff7fe640 1 --2- 172.21.15.60:0/481000023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ae80051d0 0x7f8ae800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8aff7fe640 1 -- 172.21.15.60:0/481000023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae800c040 con 0x7f8ae80039c0 2024-04-25T10:14:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8aff7fe640 1 -- 172.21.15.60:0/481000023 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae800c1f0 con 0x7f8ae8004810 2024-04-25T10:14:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8aff7fe640 1 -- 172.21.15.60:0/481000023 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae800c3f0 con 0x7f8ae80051d0 2024-04-25T10:14:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8b04856640 1 --2- 172.21.15.60:0/481000023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ae80039c0 0x7f8ae8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8afffff640 1 --2- 172.21.15.60:0/481000023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ae8004810 0x7f8ae8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8b05057640 1 --2- 172.21.15.60:0/481000023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ae80051d0 0x7f8ae800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8b04856640 1 -- 172.21.15.60:0/481000023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ae80051d0 msgr2=0x7f8ae800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8b04856640 1 --2- 172.21.15.60:0/481000023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ae80051d0 0x7f8ae800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8b04856640 1 -- 172.21.15.60:0/481000023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ae8004810 msgr2=0x7f8ae8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8b04856640 1 --2- 172.21.15.60:0/481000023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ae8004810 0x7f8ae8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8b04856640 1 -- 172.21.15.60:0/481000023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae8003530 con 0x7f8ae80039c0 2024-04-25T10:14:35.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8afffff640 1 --2- 172.21.15.60:0/481000023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ae8004810 0x7f8ae8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:35.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8b05057640 1 --2- 172.21.15.60:0/481000023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ae80051d0 0x7f8ae800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:35.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.159+0000 7f8b05057640 1 -- 172.21.15.60:0/481000023 reap_dead start 2024-04-25T10:14:35.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.163+0000 7f8b04856640 1 --2- 172.21.15.60:0/481000023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ae80039c0 0x7f8ae8003de0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009ac0 tx=0x7f8af001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c518689a7698a6d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8aff7fe640 1 -- 172.21.15.60:0/481000023 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0009690 con 0x7f8ae80039c0 2024-04-25T10:14:35.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8aff7fe640 1 -- 172.21.15.60:0/481000023 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af00a8460 con 0x7f8ae80039c0 2024-04-25T10:14:35.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8aff7fe640 1 -- 172.21.15.60:0/481000023 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af00a8830 con 0x7f8ae80039c0 2024-04-25T10:14:35.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b06aba640 1 -- 172.21.15.60:0/481000023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ae80039c0 msgr2=0x7f8ae8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b06aba640 1 --2- 172.21.15.60:0/481000023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ae80039c0 0x7f8ae8003de0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009ac0 tx=0x7f8af001ca50 comp rx=0 tx=0).stop 2024-04-25T10:14:35.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b06aba640 1 -- 172.21.15.60:0/481000023 shutdown_connections 2024-04-25T10:14:35.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b06aba640 1 --2- 172.21.15.60:0/481000023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ae80039c0 0x7f8ae8003de0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b06aba640 1 -- 172.21.15.60:0/481000023 >> 172.21.15.60:0/481000023 conn(0x7f8b000b12d0 msgr2=0x7f8b000b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b06aba640 1 -- 172.21.15.60:0/481000023 shutdown_connections 2024-04-25T10:14:35.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b06aba640 1 -- 172.21.15.60:0/481000023 wait complete. 2024-04-25T10:14:35.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b06aba640 1 Processor -- start 2024-04-25T10:14:35.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b06aba640 1 -- start start 2024-04-25T10:14:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b06aba640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ae80039c0 0x7f8b000c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b06aba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b001f87c0 0x7f8b000a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b06aba640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b000aa4b0 0x7f8b00148c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b06aba640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000b3d20 con 0x7f8b001f87c0 2024-04-25T10:14:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b06aba640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000b3ba0 con 0x7f8ae80039c0 2024-04-25T10:14:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b06aba640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000b4280 con 0x7f8b000aa4b0 2024-04-25T10:14:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8afffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b001f87c0 0x7f8b000a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8afffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b001f87c0 0x7f8b000a9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40626/0 (socket says 172.21.15.60:40626) 2024-04-25T10:14:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8afffff640 1 -- 172.21.15.60:0/3337802681 learned_addr learned my addr 172.21.15.60:0/3337802681 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b04856640 1 --2- 172.21.15.60:0/3337802681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ae80039c0 0x7f8b000c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8b05057640 1 --2- 172.21.15.60:0/3337802681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b000aa4b0 0x7f8b00148c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8afffff640 1 -- 172.21.15.60:0/3337802681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b000aa4b0 msgr2=0x7f8b00148c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8afffff640 1 --2- 172.21.15.60:0/3337802681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b000aa4b0 0x7f8b00148c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8afffff640 1 -- 172.21.15.60:0/3337802681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ae80039c0 msgr2=0x7f8b000c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8afffff640 1 --2- 172.21.15.60:0/3337802681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ae80039c0 0x7f8b000c47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8afffff640 1 -- 172.21.15.60:0/3337802681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b000a8ed0 con 0x7f8b001f87c0 2024-04-25T10:14:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.167+0000 7f8afffff640 1 --2- 172.21.15.60:0/3337802681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b001f87c0 0x7f8b000a9f70 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f8af4005fd0 tx=0x7f8af400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.171+0000 7f8afd7fa640 1 -- 172.21.15.60:0/3337802681 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af400c8d0 con 0x7f8b001f87c0 2024-04-25T10:14:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.171+0000 7f8afd7fa640 1 -- 172.21.15.60:0/3337802681 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af4018070 con 0x7f8b001f87c0 2024-04-25T10:14:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.171+0000 7f8b06aba640 1 -- 172.21.15.60:0/3337802681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b000a91b0 con 0x7f8b001f87c0 2024-04-25T10:14:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.171+0000 7f8afd7fa640 1 -- 172.21.15.60:0/3337802681 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af400cc90 con 0x7f8b001f87c0 2024-04-25T10:14:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.171+0000 7f8b06aba640 1 -- 172.21.15.60:0/3337802681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b000a9770 con 0x7f8b001f87c0 2024-04-25T10:14:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.171+0000 7f8b06aba640 1 -- 172.21.15.60:0/3337802681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b00074770 con 0x7f8b001f87c0 2024-04-25T10:14:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.171+0000 7f8afd7fa640 1 -- 172.21.15.60:0/3337802681 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8af401d460 con 0x7f8b001f87c0 2024-04-25T10:14:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.171+0000 7f8afd7fa640 1 --2- 172.21.15.60:0/3337802681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8aec066f60 0x7f8aec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.171+0000 7f8afd7fa640 1 -- 172.21.15.60:0/3337802681 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8af4085000 con 0x7f8b001f87c0 2024-04-25T10:14:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.171+0000 7f8b04856640 1 --2- 172.21.15.60:0/3337802681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8aec066f60 0x7f8aec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.171+0000 7f8b04856640 1 --2- 172.21.15.60:0/3337802681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8aec066f60 0x7f8aec069420 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8af00af000 tx=0x7f8af00b2040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.175+0000 7f8afd7fa640 1 -- 172.21.15.60:0/3337802681 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8af4055f70 con 0x7f8b001f87c0 2024-04-25T10:14:35.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.335+0000 7f8b06aba640 1 -- 172.21.15.60:0/3337802681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b000b18c0 con 0x7f8b001f87c0 2024-04-25T10:14:35.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.339+0000 7f8afd7fa640 1 -- 172.21.15.60:0/3337802681 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f8af4055870 con 0x7f8b001f87c0 2024-04-25T10:14:35.343 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:35.343 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":97,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:14:17.788631+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:14.444411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:15.153386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:35.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.339+0000 7f8b06aba640 1 -- 172.21.15.60:0/3337802681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8aec066f60 msgr2=0x7f8aec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.339+0000 7f8b06aba640 1 --2- 172.21.15.60:0/3337802681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8aec066f60 0x7f8aec069420 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8af00af000 tx=0x7f8af00b2040 comp rx=0 tx=0).stop 2024-04-25T10:14:35.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.339+0000 7f8b06aba640 1 -- 172.21.15.60:0/3337802681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b001f87c0 msgr2=0x7f8b000a9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.339+0000 7f8b06aba640 1 --2- 172.21.15.60:0/3337802681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b001f87c0 0x7f8b000a9f70 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f8af4005fd0 tx=0x7f8af400ed50 comp rx=0 tx=0).stop 2024-04-25T10:14:35.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.339+0000 7f8b06aba640 1 -- 172.21.15.60:0/3337802681 shutdown_connections 2024-04-25T10:14:35.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.339+0000 7f8b06aba640 1 --2- 172.21.15.60:0/3337802681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8aec066f60 0x7f8aec069420 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.339+0000 7f8b06aba640 1 --2- 172.21.15.60:0/3337802681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b000aa4b0 0x7f8b00148c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.339+0000 7f8b06aba640 1 --2- 172.21.15.60:0/3337802681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b001f87c0 0x7f8b000a9f70 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.339+0000 7f8b06aba640 1 --2- 172.21.15.60:0/3337802681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ae80039c0 0x7f8b000c47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.339+0000 7f8b06aba640 1 -- 172.21.15.60:0/3337802681 >> 172.21.15.60:0/3337802681 conn(0x7f8b000b12d0 msgr2=0x7f8b000b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.339+0000 7f8b06aba640 1 -- 172.21.15.60:0/3337802681 shutdown_connections 2024-04-25T10:14:35.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.339+0000 7f8b06aba640 1 -- 172.21.15.60:0/3337802681 wait complete. 2024-04-25T10:14:35.362 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:14:14.444411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:14:15.153386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:35.362 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:35.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.487+0000 7f13ebd6c640 1 Processor -- start 2024-04-25T10:14:35.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.487+0000 7f13ebd6c640 1 -- start start 2024-04-25T10:14:35.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.487+0000 7f13ebd6c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f13e415ef20 0x7f13e40b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.487+0000 7f13ebd6c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e40cc320 con 0x7f13e40cfcf0 2024-04-25T10:14:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.487+0000 7f13ebd6c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e40cc5c0 con 0x7f13e4099a30 2024-04-25T10:14:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.487+0000 7f13ebd6c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e40d07b0 con 0x7f13e415ef20 2024-04-25T10:14:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.487+0000 7f13e9b08640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f13e415ef20 0x7f13e40b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.487+0000 7f13e9307640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f13e40cfcf0 0x7f13e40b4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53066/0 (socket says 172.21.15.60:53066) 2024-04-25T10:14:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.487+0000 7f13e9307640 1 -- 172.21.15.60:0/531073874 learned_addr learned my addr 172.21.15.60:0/531073874 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.487+0000 7f13e8b06640 1 -- 172.21.15.60:0/531073874 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539209948 0 0) 0x7f13e40cc320 con 0x7f13e40cfcf0 2024-04-25T10:14:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.487+0000 7f13e8b06640 1 -- 172.21.15.60:0/531073874 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d4003530 con 0x7f13e40cfcf0 2024-04-25T10:14:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.487+0000 7f13e9b08640 1 -- 172.21.15.60:0/531073874 >> v1:172.21.15.100:6789/0 conn(0x7f13e4099a30 legacy=0x7f13e40cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.487+0000 7f13e9b08640 1 -- 172.21.15.60:0/531073874 >> v1:172.21.15.60:6789/0 conn(0x7f13e40cfcf0 legacy=0x7f13e40b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.487+0000 7f13e9b08640 1 -- 172.21.15.60:0/531073874 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e40d04d0 con 0x7f13e415ef20 2024-04-25T10:14:35.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.491+0000 7f13e9b08640 1 --2- 172.21.15.60:0/531073874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e415ef20 0x7f13e40b2560 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009d90 tx=0x7f13d801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbae489cef8c2216 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.491+0000 7f13e8b06640 1 -- 172.21.15.60:0/531073874 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d80b2070 con 0x7f13e415ef20 2024-04-25T10:14:35.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.491+0000 7f13e8b06640 1 -- 172.21.15.60:0/531073874 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d80a5830 con 0x7f13e415ef20 2024-04-25T10:14:35.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.491+0000 7f13e8b06640 1 -- 172.21.15.60:0/531073874 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d80a5c00 con 0x7f13e415ef20 2024-04-25T10:14:35.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.491+0000 7f13ebd6c640 1 -- 172.21.15.60:0/531073874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e415ef20 msgr2=0x7f13e40b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.491+0000 7f13ebd6c640 1 --2- 172.21.15.60:0/531073874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e415ef20 0x7f13e40b2560 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009d90 tx=0x7f13d801c970 comp rx=0 tx=0).stop 2024-04-25T10:14:35.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.491+0000 7f13ebd6c640 1 -- 172.21.15.60:0/531073874 shutdown_connections 2024-04-25T10:14:35.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.491+0000 7f13ebd6c640 1 --2- 172.21.15.60:0/531073874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e415ef20 0x7f13e40b2560 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.491+0000 7f13ebd6c640 1 -- 172.21.15.60:0/531073874 >> 172.21.15.60:0/531073874 conn(0x7f13e40b3200 msgr2=0x7f13e40b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.491+0000 7f13ebd6c640 1 -- 172.21.15.60:0/531073874 shutdown_connections 2024-04-25T10:14:35.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.491+0000 7f13ebd6c640 1 -- 172.21.15.60:0/531073874 wait complete. 2024-04-25T10:14:35.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.491+0000 7f13ebd6c640 1 Processor -- start 2024-04-25T10:14:35.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.491+0000 7f13ebd6c640 1 -- start start 2024-04-25T10:14:35.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ebd6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13e4099a30 0x7f13e414c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ebd6c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13e40cfcf0 0x7f13e415d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ebd6c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e415ef20 0x7f13e414c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ebd6c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e40d0270 con 0x7f13e415ef20 2024-04-25T10:14:35.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ebd6c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e40d00f0 con 0x7f13e4099a30 2024-04-25T10:14:35.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ebd6c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e415f300 con 0x7f13e40cfcf0 2024-04-25T10:14:35.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ea309640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e415ef20 0x7f13e414c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ea309640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e415ef20 0x7f13e414c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40644/0 (socket says 172.21.15.60:40644) 2024-04-25T10:14:35.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ea309640 1 -- 172.21.15.60:0/4069713173 learned_addr learned my addr 172.21.15.60:0/4069713173 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:35.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13e9b08640 1 --2- 172.21.15.60:0/4069713173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13e4099a30 0x7f13e414c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13e9307640 1 --2- 172.21.15.60:0/4069713173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13e40cfcf0 0x7f13e415d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ea309640 1 -- 172.21.15.60:0/4069713173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13e40cfcf0 msgr2=0x7f13e415d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ea309640 1 --2- 172.21.15.60:0/4069713173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13e40cfcf0 0x7f13e415d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ea309640 1 -- 172.21.15.60:0/4069713173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13e4099a30 msgr2=0x7f13e414c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ea309640 1 --2- 172.21.15.60:0/4069713173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13e4099a30 0x7f13e414c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ea309640 1 -- 172.21.15.60:0/4069713173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e414c7b0 con 0x7f13e415ef20 2024-04-25T10:14:35.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ea309640 1 --2- 172.21.15.60:0/4069713173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e415ef20 0x7f13e414c270 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00c430 tx=0x7f13dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13e27fc640 1 -- 172.21.15.60:0/4069713173 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc005220 con 0x7f13e415ef20 2024-04-25T10:14:35.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13e27fc640 1 -- 172.21.15.60:0/4069713173 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc0053c0 con 0x7f13e415ef20 2024-04-25T10:14:35.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ebd6c640 1 -- 172.21.15.60:0/4069713173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e414df30 con 0x7f13e415ef20 2024-04-25T10:14:35.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13e27fc640 1 -- 172.21.15.60:0/4069713173 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc007270 con 0x7f13e415ef20 2024-04-25T10:14:35.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ebd6c640 1 -- 172.21.15.60:0/4069713173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e415c690 con 0x7f13e415ef20 2024-04-25T10:14:35.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13ebd6c640 1 -- 172.21.15.60:0/4069713173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e4074770 con 0x7f13e415ef20 2024-04-25T10:14:35.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13e27fc640 1 -- 172.21.15.60:0/4069713173 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13dc019020 con 0x7f13e415ef20 2024-04-25T10:14:35.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13e27fc640 1 --2- 172.21.15.60:0/4069713173 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13c8066f60 0x7f13c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13e27fc640 1 -- 172.21.15.60:0/4069713173 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13dc084450 con 0x7f13e415ef20 2024-04-25T10:14:35.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.495+0000 7f13e9b08640 1 --2- 172.21.15.60:0/4069713173 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13c8066f60 0x7f13c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.499+0000 7f13e9b08640 1 --2- 172.21.15.60:0/4069713173 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13c8066f60 0x7f13c8069420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f13d80af000 tx=0x7f13d80a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.499+0000 7f13e27fc640 1 -- 172.21.15.60:0/4069713173 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f13dc0553b0 con 0x7f13e415ef20 2024-04-25T10:14:35.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.663+0000 7f13ebd6c640 1 -- 172.21.15.60:0/4069713173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13e40b6870 con 0x7f13e415ef20 2024-04-25T10:14:35.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.663+0000 7f13e27fc640 1 -- 172.21.15.60:0/4069713173 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13dc054cb0 con 0x7f13e415ef20 2024-04-25T10:14:35.669 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:14:35.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.667+0000 7f13ebd6c640 1 -- 172.21.15.60:0/4069713173 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13c8066f60 msgr2=0x7f13c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.667+0000 7f13ebd6c640 1 --2- 172.21.15.60:0/4069713173 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13c8066f60 0x7f13c8069420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f13d80af000 tx=0x7f13d80a7070 comp rx=0 tx=0).stop 2024-04-25T10:14:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.667+0000 7f13ebd6c640 1 -- 172.21.15.60:0/4069713173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e415ef20 msgr2=0x7f13e414c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.667+0000 7f13ebd6c640 1 --2- 172.21.15.60:0/4069713173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e415ef20 0x7f13e414c270 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00c430 tx=0x7f13dc00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.667+0000 7f13ebd6c640 1 -- 172.21.15.60:0/4069713173 shutdown_connections 2024-04-25T10:14:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.667+0000 7f13ebd6c640 1 --2- 172.21.15.60:0/4069713173 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f13c8066f60 0x7f13c8069420 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.667+0000 7f13ebd6c640 1 --2- 172.21.15.60:0/4069713173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e415ef20 0x7f13e414c270 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.667+0000 7f13ebd6c640 1 --2- 172.21.15.60:0/4069713173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13e40cfcf0 0x7f13e415d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.667+0000 7f13ebd6c640 1 --2- 172.21.15.60:0/4069713173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13e4099a30 0x7f13e414c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.667+0000 7f13ebd6c640 1 -- 172.21.15.60:0/4069713173 >> 172.21.15.60:0/4069713173 conn(0x7f13e40b3200 msgr2=0x7f13e40b4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.667+0000 7f13ebd6c640 1 -- 172.21.15.60:0/4069713173 shutdown_connections 2024-04-25T10:14:35.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.667+0000 7f13ebd6c640 1 -- 172.21.15.60:0/4069713173 wait complete. 2024-04-25T10:14:35.689 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:35.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44ad45e640 1 Processor -- start 2024-04-25T10:14:35.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44ad45e640 1 -- start start 2024-04-25T10:14:35.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44ad45e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44a80cbcf0 0x7f44a8160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44ad45e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f44a8071c30 0x7f44a815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44ad45e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a8144a40 con 0x7f44a815fa20 2024-04-25T10:14:35.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44ad45e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a80b2dc0 con 0x7f44a8071c30 2024-04-25T10:14:35.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44ad45e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a815fe80 con 0x7f44a80cbcf0 2024-04-25T10:14:35.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a77fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f44a815fa20 0x7f44a815bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53076/0 (socket says 172.21.15.60:53076) 2024-04-25T10:14:35.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a77fe640 1 -- 172.21.15.60:0/1478193077 learned_addr learned my addr 172.21.15.60:0/1478193077 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:35.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a67fc640 1 --2- 172.21.15.60:0/1478193077 >> v2:172.21.15.100:3300/0 conn(0x7f44a8071c30 0x7f44a815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a5ffb640 1 -- 172.21.15.60:0/1478193077 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241212619 0 0) 0x7f44a8144a40 con 0x7f44a815fa20 2024-04-25T10:14:35.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a5ffb640 1 -- 172.21.15.60:0/1478193077 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4490003530 con 0x7f44a815fa20 2024-04-25T10:14:35.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a6ffd640 1 --2- 172.21.15.60:0/1478193077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44a80cbcf0 0x7f44a8160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a5ffb640 1 -- 172.21.15.60:0/1478193077 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2733881585 0 0) 0x7f4490003530 con 0x7f44a815fa20 2024-04-25T10:14:35.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a5ffb640 1 -- 172.21.15.60:0/1478193077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44a80cbcf0 msgr2=0x7f44a8160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a5ffb640 1 --2- 172.21.15.60:0/1478193077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44a80cbcf0 0x7f44a8160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a5ffb640 1 -- 172.21.15.60:0/1478193077 >> v2:172.21.15.100:3300/0 conn(0x7f44a8071c30 msgr2=0x7f44a815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a5ffb640 1 --2- 172.21.15.60:0/1478193077 >> v2:172.21.15.100:3300/0 conn(0x7f44a8071c30 0x7f44a815f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a5ffb640 1 -- 172.21.15.60:0/1478193077 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a80cc9f0 con 0x7f44a815fa20 2024-04-25T10:14:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a5ffb640 1 -- 172.21.15.60:0/1478193077 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f44a0001a00 con 0x7f44a815fa20 2024-04-25T10:14:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a6ffd640 1 --2- 172.21.15.60:0/1478193077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44a80cbcf0 0x7f44a8160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a5ffb640 1 -- 172.21.15.60:0/1478193077 >> v1:172.21.15.60:6789/0 conn(0x7f44a815fa20 legacy=0x7f44a815bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a5ffb640 1 --2- 172.21.15.60:0/1478193077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44900039c0 0x7f4490003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a5ffb640 1 --2- 172.21.15.60:0/1478193077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44a80cbcf0 0x7f4490004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a5ffb640 1 --2- 172.21.15.60:0/1478193077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4490004d50 0x7f44900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a5ffb640 1 -- 172.21.15.60:0/1478193077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449000b9d0 con 0x7f44900039c0 2024-04-25T10:14:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a5ffb640 1 -- 172.21.15.60:0/1478193077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449000bb50 con 0x7f4490004d50 2024-04-25T10:14:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a5ffb640 1 -- 172.21.15.60:0/1478193077 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449000bcd0 con 0x7f44a80cbcf0 2024-04-25T10:14:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a6ffd640 1 --2- 172.21.15.60:0/1478193077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44900039c0 0x7f4490003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a6ffd640 1 -- 172.21.15.60:0/1478193077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44a80cbcf0 msgr2=0x7f4490004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a6ffd640 1 --2- 172.21.15.60:0/1478193077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44a80cbcf0 0x7f4490004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a6ffd640 1 -- 172.21.15.60:0/1478193077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4490004d50 msgr2=0x7f44900051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:35.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a77fe640 1 --2- 172.21.15.60:0/1478193077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4490004d50 0x7f44900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a6ffd640 1 --2- 172.21.15.60:0/1478193077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4490004d50 0x7f44900051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a6ffd640 1 -- 172.21.15.60:0/1478193077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4490003530 con 0x7f44900039c0 2024-04-25T10:14:35.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.811+0000 7f44a77fe640 1 --2- 172.21.15.60:0/1478193077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4490004d50 0x7f44900051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:35.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.815+0000 7f44a6ffd640 1 --2- 172.21.15.60:0/1478193077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44900039c0 0x7f4490003de0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f449c0098d0 tx=0x7f449c01c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8de0ebaf1bdb40e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.815+0000 7f44a5ffb640 1 -- 172.21.15.60:0/1478193077 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c00f040 con 0x7f44900039c0 2024-04-25T10:14:35.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.815+0000 7f44a5ffb640 1 -- 172.21.15.60:0/1478193077 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449c0af070 con 0x7f44900039c0 2024-04-25T10:14:35.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.815+0000 7f44a5ffb640 1 -- 172.21.15.60:0/1478193077 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c0aa880 con 0x7f44900039c0 2024-04-25T10:14:35.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.815+0000 7f44ad45e640 1 -- 172.21.15.60:0/1478193077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44900039c0 msgr2=0x7f4490003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.815+0000 7f44ad45e640 1 --2- 172.21.15.60:0/1478193077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44900039c0 0x7f4490003de0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f449c0098d0 tx=0x7f449c01c5e0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.815+0000 7f44a77fe640 1 -- 172.21.15.60:0/1478193077 reap_dead start 2024-04-25T10:14:35.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.815+0000 7f44ad45e640 1 -- 172.21.15.60:0/1478193077 shutdown_connections 2024-04-25T10:14:35.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.815+0000 7f44ad45e640 1 -- 172.21.15.60:0/1478193077 >> 172.21.15.60:0/1478193077 conn(0x7f44a80670b0 msgr2=0x7f44a806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.815+0000 7f44ad45e640 1 -- 172.21.15.60:0/1478193077 shutdown_connections 2024-04-25T10:14:35.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.815+0000 7f44ad45e640 1 -- 172.21.15.60:0/1478193077 wait complete. 2024-04-25T10:14:35.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.815+0000 7f44ad45e640 1 Processor -- start 2024-04-25T10:14:35.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.815+0000 7f44ad45e640 1 -- start start 2024-04-25T10:14:35.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44ad45e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44900039c0 0x7f44a81402b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44ad45e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44a813e6f0 0x7f44a81595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44ad45e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44a813d350 0x7f44a80ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44ad45e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a80cfed0 con 0x7f44a813e6f0 2024-04-25T10:14:35.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44ad45e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a80cc0d0 con 0x7f44900039c0 2024-04-25T10:14:35.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44ad45e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a813a790 con 0x7f44a813d350 2024-04-25T10:14:35.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44a67fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44a813e6f0 0x7f44a81595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44a67fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44a813e6f0 0x7f44a81595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40662/0 (socket says 172.21.15.60:40662) 2024-04-25T10:14:35.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44a67fc640 1 -- 172.21.15.60:0/1507194052 learned_addr learned my addr 172.21.15.60:0/1507194052 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:35.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44a77fe640 1 --2- 172.21.15.60:0/1507194052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44a813d350 0x7f44a80ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44a6ffd640 1 --2- 172.21.15.60:0/1507194052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44900039c0 0x7f44a81402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44a67fc640 1 -- 172.21.15.60:0/1507194052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44a813d350 msgr2=0x7f44a80ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44a67fc640 1 --2- 172.21.15.60:0/1507194052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44a813d350 0x7f44a80ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44a67fc640 1 -- 172.21.15.60:0/1507194052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44900039c0 msgr2=0x7f44a81402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44a67fc640 1 --2- 172.21.15.60:0/1507194052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44900039c0 0x7f44a81402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44a67fc640 1 -- 172.21.15.60:0/1507194052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a8067670 con 0x7f44a813e6f0 2024-04-25T10:14:35.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44a67fc640 1 --2- 172.21.15.60:0/1507194052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44a813e6f0 0x7f44a81595c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f449800e840 tx=0x7f449800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f448f7fe640 1 -- 172.21.15.60:0/1507194052 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4498017070 con 0x7f44a813e6f0 2024-04-25T10:14:35.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f448f7fe640 1 -- 172.21.15.60:0/1507194052 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449800c940 con 0x7f44a813e6f0 2024-04-25T10:14:35.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44ad45e640 1 -- 172.21.15.60:0/1507194052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a813ef00 con 0x7f44a813e6f0 2024-04-25T10:14:35.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f448f7fe640 1 -- 172.21.15.60:0/1507194052 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449800cd30 con 0x7f44a813e6f0 2024-04-25T10:14:35.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44ad45e640 1 -- 172.21.15.60:0/1507194052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a813ad30 con 0x7f44a813e6f0 2024-04-25T10:14:35.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44ad45e640 1 -- 172.21.15.60:0/1507194052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a806bba0 con 0x7f44a813e6f0 2024-04-25T10:14:35.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f448f7fe640 1 -- 172.21.15.60:0/1507194052 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f449801a070 con 0x7f44a813e6f0 2024-04-25T10:14:35.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f448f7fe640 1 --2- 172.21.15.60:0/1507194052 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4494066f60 0x7f4494069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:35.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f448f7fe640 1 -- 172.21.15.60:0/1507194052 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4498085260 con 0x7f44a813e6f0 2024-04-25T10:14:35.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.819+0000 7f44a6ffd640 1 --2- 172.21.15.60:0/1507194052 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4494066f60 0x7f4494069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:35.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.823+0000 7f44a6ffd640 1 --2- 172.21.15.60:0/1507194052 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4494066f60 0x7f4494069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f449c0ae000 tx=0x7f449c0046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:35.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.823+0000 7f448f7fe640 1 -- 172.21.15.60:0/1507194052 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4498056270 con 0x7f44a813e6f0 2024-04-25T10:14:35.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.979+0000 7f44ad45e640 1 -- 172.21.15.60:0/1507194052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f44a80b2970 con 0x7f44a813e6f0 2024-04-25T10:14:35.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.983+0000 7f448f7fe640 1 -- 172.21.15.60:0/1507194052 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4498055b70 con 0x7f44a813e6f0 2024-04-25T10:14:35.986 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:14:35.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.983+0000 7f44ad45e640 1 -- 172.21.15.60:0/1507194052 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4494066f60 msgr2=0x7f4494069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.983+0000 7f44ad45e640 1 --2- 172.21.15.60:0/1507194052 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4494066f60 0x7f4494069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f449c0ae000 tx=0x7f449c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.983+0000 7f44ad45e640 1 -- 172.21.15.60:0/1507194052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44a813e6f0 msgr2=0x7f44a81595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.983+0000 7f44ad45e640 1 --2- 172.21.15.60:0/1507194052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44a813e6f0 0x7f44a81595c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f449800e840 tx=0x7f449800ed00 comp rx=0 tx=0).stop 2024-04-25T10:14:35.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.983+0000 7f44ad45e640 1 -- 172.21.15.60:0/1507194052 shutdown_connections 2024-04-25T10:14:35.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.983+0000 7f44ad45e640 1 --2- 172.21.15.60:0/1507194052 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4494066f60 0x7f4494069420 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.983+0000 7f44ad45e640 1 --2- 172.21.15.60:0/1507194052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44a813d350 0x7f44a80ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.983+0000 7f44ad45e640 1 --2- 172.21.15.60:0/1507194052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44a813e6f0 0x7f44a81595c0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.983+0000 7f44ad45e640 1 --2- 172.21.15.60:0/1507194052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44900039c0 0x7f44a81402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.983+0000 7f44ad45e640 1 -- 172.21.15.60:0/1507194052 >> 172.21.15.60:0/1507194052 conn(0x7f44a80670b0 msgr2=0x7f44a8144eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.983+0000 7f44ad45e640 1 -- 172.21.15.60:0/1507194052 shutdown_connections 2024-04-25T10:14:35.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:35.983+0000 7f44ad45e640 1 -- 172.21.15.60:0/1507194052 wait complete. 2024-04-25T10:14:36.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:36.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f6430a1e640 1 Processor -- start 2024-04-25T10:14:36.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f6430a1e640 1 -- start start 2024-04-25T10:14:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f6430a1e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f642c0cbcf0 0x7f642c160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f6430a1e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f642c071c30 0x7f642c15f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f6430a1e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f642c15fa20 0x7f642c15bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f6430a1e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f642c0b2dc0 con 0x7f642c0cbcf0 2024-04-25T10:14:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f6430a1e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f642c144a40 con 0x7f642c071c30 2024-04-25T10:14:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f6430a1e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642c0cc9f0 con 0x7f642c15fa20 2024-04-25T10:14:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f642a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f642c0cbcf0 0x7f642c160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f642a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f642c0cbcf0 0x7f642c160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40670/0 (socket says 172.21.15.60:40670) 2024-04-25T10:14:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f642a59c640 1 -- 172.21.15.60:0/2041372712 learned_addr learned my addr 172.21.15.60:0/2041372712 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f6429d9b640 1 --2- 172.21.15.60:0/2041372712 >> v2:172.21.15.100:3300/0 conn(0x7f642c071c30 0x7f642c15f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f642ad9d640 1 --2- 172.21.15.60:0/2041372712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f642c15fa20 0x7f642c15bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f642a59c640 1 -- 172.21.15.60:0/2041372712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f642c15fa20 msgr2=0x7f642c15bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f642a59c640 1 --2- 172.21.15.60:0/2041372712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f642c15fa20 0x7f642c15bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f642a59c640 1 -- 172.21.15.60:0/2041372712 >> v2:172.21.15.100:3300/0 conn(0x7f642c071c30 msgr2=0x7f642c15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f642a59c640 1 --2- 172.21.15.60:0/2041372712 >> v2:172.21.15.100:3300/0 conn(0x7f642c071c30 0x7f642c15f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.115+0000 7f642a59c640 1 -- 172.21.15.60:0/2041372712 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c0cccd0 con 0x7f642c0cbcf0 2024-04-25T10:14:36.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.119+0000 7f642a59c640 1 --2- 172.21.15.60:0/2041372712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c0cbcf0 0x7f642c160080 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f642c0d0570 tx=0x7f641c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a261d0cf7e233e3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:36.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.119+0000 7f642959a640 1 -- 172.21.15.60:0/2041372712 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c0b1070 con 0x7f642c0cbcf0 2024-04-25T10:14:36.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.119+0000 7f642959a640 1 -- 172.21.15.60:0/2041372712 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641c0ac630 con 0x7f642c0cbcf0 2024-04-25T10:14:36.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.119+0000 7f642959a640 1 -- 172.21.15.60:0/2041372712 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c0a8050 con 0x7f642c0cbcf0 2024-04-25T10:14:36.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.119+0000 7f6430a1e640 1 -- 172.21.15.60:0/2041372712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c0cbcf0 msgr2=0x7f642c160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.119+0000 7f6430a1e640 1 --2- 172.21.15.60:0/2041372712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c0cbcf0 0x7f642c160080 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f642c0d0570 tx=0x7f641c01c520 comp rx=0 tx=0).stop 2024-04-25T10:14:36.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.119+0000 7f6430a1e640 1 -- 172.21.15.60:0/2041372712 shutdown_connections 2024-04-25T10:14:36.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.119+0000 7f6430a1e640 1 --2- 172.21.15.60:0/2041372712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f642c15fa20 0x7f642c15bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.119+0000 7f6430a1e640 1 --2- 172.21.15.60:0/2041372712 >> v2:172.21.15.100:3300/0 conn(0x7f642c071c30 0x7f642c15f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.119+0000 7f6430a1e640 1 --2- 172.21.15.60:0/2041372712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c0cbcf0 0x7f642c160080 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.119+0000 7f6430a1e640 1 -- 172.21.15.60:0/2041372712 >> 172.21.15.60:0/2041372712 conn(0x7f642c0670b0 msgr2=0x7f642c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:36.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f6430a1e640 1 -- 172.21.15.60:0/2041372712 shutdown_connections 2024-04-25T10:14:36.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f6430a1e640 1 -- 172.21.15.60:0/2041372712 wait complete. 2024-04-25T10:14:36.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f6430a1e640 1 Processor -- start 2024-04-25T10:14:36.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f6430a1e640 1 -- start start 2024-04-25T10:14:36.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f6430a1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f642c071c30 0x7f642c13cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f6430a1e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f642c0cbcf0 0x7f642c1650b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f6430a1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c15fa20 0x7f642c13fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f6430a1e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c0ce740 con 0x7f642c15fa20 2024-04-25T10:14:36.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f6430a1e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c0ce5c0 con 0x7f642c071c30 2024-04-25T10:14:36.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f6430a1e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642c072010 con 0x7f642c0cbcf0 2024-04-25T10:14:36.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f642ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c15fa20 0x7f642c13fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f642ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c15fa20 0x7f642c13fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40672/0 (socket says 172.21.15.60:40672) 2024-04-25T10:14:36.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f642ad9d640 1 -- 172.21.15.60:0/32863917 learned_addr learned my addr 172.21.15.60:0/32863917 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:36.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f642a59c640 1 --2- 172.21.15.60:0/32863917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f642c071c30 0x7f642c13cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f6429d9b640 1 --2- 172.21.15.60:0/32863917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f642c0cbcf0 0x7f642c1650b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f642ad9d640 1 -- 172.21.15.60:0/32863917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f642c0cbcf0 msgr2=0x7f642c1650b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f642ad9d640 1 --2- 172.21.15.60:0/32863917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f642c0cbcf0 0x7f642c1650b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f642ad9d640 1 -- 172.21.15.60:0/32863917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f642c071c30 msgr2=0x7f642c13cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f642ad9d640 1 --2- 172.21.15.60:0/32863917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f642c071c30 0x7f642c13cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f642ad9d640 1 -- 172.21.15.60:0/32863917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c165a90 con 0x7f642c15fa20 2024-04-25T10:14:36.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f642ad9d640 1 --2- 172.21.15.60:0/32863917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c15fa20 0x7f642c13fa00 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f6424007e40 tx=0x7f642400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:36.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f641affd640 1 -- 172.21.15.60:0/32863917 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424014620 con 0x7f642c15fa20 2024-04-25T10:14:36.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f641affd640 1 -- 172.21.15.60:0/32863917 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6424019070 con 0x7f642c15fa20 2024-04-25T10:14:36.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f6430a1e640 1 -- 172.21.15.60:0/32863917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642c165d70 con 0x7f642c15fa20 2024-04-25T10:14:36.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f641affd640 1 -- 172.21.15.60:0/32863917 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424014a20 con 0x7f642c15fa20 2024-04-25T10:14:36.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f6430a1e640 1 -- 172.21.15.60:0/32863917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642c166330 con 0x7f642c15fa20 2024-04-25T10:14:36.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.123+0000 7f6430a1e640 1 -- 172.21.15.60:0/32863917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642c166670 con 0x7f642c15fa20 2024-04-25T10:14:36.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.127+0000 7f641affd640 1 -- 172.21.15.60:0/32863917 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6424014bc0 con 0x7f642c15fa20 2024-04-25T10:14:36.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.127+0000 7f641affd640 1 --2- 172.21.15.60:0/32863917 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f640c066f90 0x7f640c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.127+0000 7f641affd640 1 -- 172.21.15.60:0/32863917 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64240859d0 con 0x7f642c15fa20 2024-04-25T10:14:36.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.127+0000 7f642a59c640 1 --2- 172.21.15.60:0/32863917 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f640c066f90 0x7f640c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.127+0000 7f642a59c640 1 --2- 172.21.15.60:0/32863917 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f640c066f90 0x7f640c069450 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f641c0ae000 tx=0x7f641c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:36.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.131+0000 7f641affd640 1 -- 172.21.15.60:0/32863917 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64240568b0 con 0x7f642c15fa20 2024-04-25T10:14:36.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.283+0000 7f6430a1e640 1 -- 172.21.15.60:0/32863917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f642c06d250 con 0x7f642c15fa20 2024-04-25T10:14:36.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.283+0000 7f641affd640 1 -- 172.21.15.60:0/32863917 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64240561b0 con 0x7f642c15fa20 2024-04-25T10:14:36.287 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:14:36.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.283+0000 7f6430a1e640 1 -- 172.21.15.60:0/32863917 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f640c066f90 msgr2=0x7f640c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.283+0000 7f6430a1e640 1 --2- 172.21.15.60:0/32863917 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f640c066f90 0x7f640c069450 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f641c0ae000 tx=0x7f641c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:36.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.283+0000 7f6430a1e640 1 -- 172.21.15.60:0/32863917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c15fa20 msgr2=0x7f642c13fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.283+0000 7f6430a1e640 1 --2- 172.21.15.60:0/32863917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c15fa20 0x7f642c13fa00 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f6424007e40 tx=0x7f642400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.283+0000 7f6430a1e640 1 -- 172.21.15.60:0/32863917 shutdown_connections 2024-04-25T10:14:36.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.283+0000 7f6430a1e640 1 --2- 172.21.15.60:0/32863917 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f640c066f90 0x7f640c069450 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.283+0000 7f6430a1e640 1 --2- 172.21.15.60:0/32863917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642c15fa20 0x7f642c13fa00 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.283+0000 7f6430a1e640 1 --2- 172.21.15.60:0/32863917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f642c0cbcf0 0x7f642c1650b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.283+0000 7f6430a1e640 1 --2- 172.21.15.60:0/32863917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f642c071c30 0x7f642c13cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.283+0000 7f6430a1e640 1 -- 172.21.15.60:0/32863917 >> 172.21.15.60:0/32863917 conn(0x7f642c0670b0 msgr2=0x7f642c0b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:36.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.283+0000 7f6430a1e640 1 -- 172.21.15.60:0/32863917 shutdown_connections 2024-04-25T10:14:36.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.283+0000 7f6430a1e640 1 -- 172.21.15.60:0/32863917 wait complete. 2024-04-25T10:14:36.305 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:14:36.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.427+0000 7fcdccfb4640 1 Processor -- start 2024-04-25T10:14:36.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.427+0000 7fcdccfb4640 1 -- start start 2024-04-25T10:14:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.427+0000 7fcdccfb4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcdc815bc90 0x7fcdc815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.427+0000 7fcdccfb4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdc80b4660 0x7fcdc815c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.427+0000 7fcdccfb4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc8071ca0 con 0x7fcdc80ce7a0 2024-04-25T10:14:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.427+0000 7fcdccfb4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc80cc370 con 0x7fcdc815bc90 2024-04-25T10:14:36.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.427+0000 7fcdccfb4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc80cca40 con 0x7fcdc80b4660 2024-04-25T10:14:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.427+0000 7fcdc5d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcdc815bc90 0x7fcdc815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.427+0000 7fcdc5d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcdc815bc90 0x7fcdc815c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40682/0 (socket says 172.21.15.60:40682) 2024-04-25T10:14:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.427+0000 7fcdc5d9b640 1 -- 172.21.15.60:0/1592150491 learned_addr learned my addr 172.21.15.60:0/1592150491 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.427+0000 7fcdc6d9d640 1 --2- 172.21.15.60:0/1592150491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdc80b4660 0x7fcdc815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.427+0000 7fcdc5d9b640 1 -- 172.21.15.60:0/1592150491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdc80b4660 msgr2=0x7fcdc815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.427+0000 7fcdc5d9b640 1 --2- 172.21.15.60:0/1592150491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdc80b4660 0x7fcdc815c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.427+0000 7fcdc5d9b640 1 -- 172.21.15.60:0/1592150491 >> v1:172.21.15.100:6789/0 conn(0x7fcdc80ce7a0 legacy=0x7fcdc80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:36.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.427+0000 7fcdc5d9b640 1 -- 172.21.15.60:0/1592150491 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc80ccd20 con 0x7fcdc815bc90 2024-04-25T10:14:36.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.431+0000 7fcdc5d9b640 1 --2- 172.21.15.60:0/1592150491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc815bc90 0x7fcdc815c220 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009870 tx=0x7fcdb001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57ed0845b5e999 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:36.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.431+0000 7fcdc559a640 1 -- 172.21.15.60:0/1592150491 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb00b0070 con 0x7fcdc815bc90 2024-04-25T10:14:36.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.431+0000 7fcdc559a640 1 -- 172.21.15.60:0/1592150491 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb00ab600 con 0x7fcdc815bc90 2024-04-25T10:14:36.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.431+0000 7fcdc559a640 1 -- 172.21.15.60:0/1592150491 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb00ab9d0 con 0x7fcdc815bc90 2024-04-25T10:14:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.431+0000 7fcdccfb4640 1 -- 172.21.15.60:0/1592150491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc815bc90 msgr2=0x7fcdc815c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.431+0000 7fcdccfb4640 1 --2- 172.21.15.60:0/1592150491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc815bc90 0x7fcdc815c220 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009870 tx=0x7fcdb001c520 comp rx=0 tx=0).stop 2024-04-25T10:14:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.431+0000 7fcdccfb4640 1 -- 172.21.15.60:0/1592150491 shutdown_connections 2024-04-25T10:14:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.431+0000 7fcdccfb4640 1 --2- 172.21.15.60:0/1592150491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc815bc90 0x7fcdc815c220 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.431+0000 7fcdccfb4640 1 --2- 172.21.15.60:0/1592150491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdc80b4660 0x7fcdc815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.431+0000 7fcdccfb4640 1 -- 172.21.15.60:0/1592150491 >> 172.21.15.60:0/1592150491 conn(0x7fcdc80b12d0 msgr2=0x7fcdc80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:36.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.431+0000 7fcdccfb4640 1 -- 172.21.15.60:0/1592150491 shutdown_connections 2024-04-25T10:14:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.431+0000 7fcdccfb4640 1 -- 172.21.15.60:0/1592150491 wait complete. 2024-04-25T10:14:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdccfb4640 1 Processor -- start 2024-04-25T10:14:36.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdccfb4640 1 -- start start 2024-04-25T10:14:36.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdccfb4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80ce7a0 0x7fcdc81511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdccfb4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc8154a00 0x7fcdc8154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdccfb4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdc8156b30 0x7fcdc8156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdccfb4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc81685c0 con 0x7fcdc80ce7a0 2024-04-25T10:14:36.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdccfb4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc80b3db0 con 0x7fcdc8154a00 2024-04-25T10:14:36.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdccfb4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc8168740 con 0x7fcdc8156b30 2024-04-25T10:14:36.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdc659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80ce7a0 0x7fcdc81511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdc659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80ce7a0 0x7fcdc81511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40698/0 (socket says 172.21.15.60:40698) 2024-04-25T10:14:36.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdc659c640 1 -- 172.21.15.60:0/2531844847 learned_addr learned my addr 172.21.15.60:0/2531844847 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:36.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdc5d9b640 1 --2- 172.21.15.60:0/2531844847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc8154a00 0x7fcdc8154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdc6d9d640 1 --2- 172.21.15.60:0/2531844847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdc8156b30 0x7fcdc8156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdc659c640 1 -- 172.21.15.60:0/2531844847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdc8156b30 msgr2=0x7fcdc8156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdc659c640 1 --2- 172.21.15.60:0/2531844847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdc8156b30 0x7fcdc8156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdc659c640 1 -- 172.21.15.60:0/2531844847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc8154a00 msgr2=0x7fcdc8154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdc659c640 1 --2- 172.21.15.60:0/2531844847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc8154a00 0x7fcdc8154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdc659c640 1 -- 172.21.15.60:0/2531844847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc8151710 con 0x7fcdc80ce7a0 2024-04-25T10:14:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdc6d9d640 1 --2- 172.21.15.60:0/2531844847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdc8156b30 0x7fcdc8156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdc659c640 1 --2- 172.21.15.60:0/2531844847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80ce7a0 0x7fcdc81511d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800c450 tx=0x7fcdb800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdbeffd640 1 -- 172.21.15.60:0/2531844847 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8011790 con 0x7fcdc80ce7a0 2024-04-25T10:14:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdbeffd640 1 -- 172.21.15.60:0/2531844847 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb8017070 con 0x7fcdc80ce7a0 2024-04-25T10:14:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdccfb4640 1 -- 172.21.15.60:0/2531844847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc81554c0 con 0x7fcdc80ce7a0 2024-04-25T10:14:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdbeffd640 1 -- 172.21.15.60:0/2531844847 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8011b90 con 0x7fcdc80ce7a0 2024-04-25T10:14:36.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdccfb4640 1 -- 172.21.15.60:0/2531844847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc8155a80 con 0x7fcdc80ce7a0 2024-04-25T10:14:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.435+0000 7fcdccfb4640 1 -- 172.21.15.60:0/2531844847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc8074770 con 0x7fcdc80ce7a0 2024-04-25T10:14:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.439+0000 7fcdbeffd640 1 -- 172.21.15.60:0/2531844847 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcdb8011d30 con 0x7fcdc80ce7a0 2024-04-25T10:14:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.439+0000 7fcdbeffd640 1 --2- 172.21.15.60:0/2531844847 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcda8066f60 0x7fcda8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.439+0000 7fcdbeffd640 1 -- 172.21.15.60:0/2531844847 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdb8084dd0 con 0x7fcdc80ce7a0 2024-04-25T10:14:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.439+0000 7fcdc5d9b640 1 --2- 172.21.15.60:0/2531844847 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcda8066f60 0x7fcda8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.439+0000 7fcdc5d9b640 1 --2- 172.21.15.60:0/2531844847 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcda8066f60 0x7fcda8069420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00ad000 tx=0x7fcdb0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:36.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.443+0000 7fcdbeffd640 1 -- 172.21.15.60:0/2531844847 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcdb8055d50 con 0x7fcdc80ce7a0 2024-04-25T10:14:36.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.607+0000 7fcdccfb4640 1 -- 172.21.15.60:0/2531844847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcdc815b5c0 con 0x7fcdc80ce7a0 2024-04-25T10:14:36.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.607+0000 7fcdbeffd640 1 -- 172.21.15.60:0/2531844847 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v97) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcdb8086030 con 0x7fcdc80ce7a0 2024-04-25T10:14:36.612 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:14:36.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.611+0000 7fcdccfb4640 1 -- 172.21.15.60:0/2531844847 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcda8066f60 msgr2=0x7fcda8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.611+0000 7fcdccfb4640 1 --2- 172.21.15.60:0/2531844847 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcda8066f60 0x7fcda8069420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00ad000 tx=0x7fcdb0008040 comp rx=0 tx=0).stop 2024-04-25T10:14:36.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.611+0000 7fcdccfb4640 1 -- 172.21.15.60:0/2531844847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80ce7a0 msgr2=0x7fcdc81511d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.611+0000 7fcdccfb4640 1 --2- 172.21.15.60:0/2531844847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80ce7a0 0x7fcdc81511d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800c450 tx=0x7fcdb800c910 comp rx=0 tx=0).stop 2024-04-25T10:14:36.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.611+0000 7fcdccfb4640 1 -- 172.21.15.60:0/2531844847 shutdown_connections 2024-04-25T10:14:36.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.611+0000 7fcdccfb4640 1 --2- 172.21.15.60:0/2531844847 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcda8066f60 0x7fcda8069420 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.611+0000 7fcdccfb4640 1 --2- 172.21.15.60:0/2531844847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdc8156b30 0x7fcdc8156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.611+0000 7fcdccfb4640 1 --2- 172.21.15.60:0/2531844847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc8154a00 0x7fcdc8154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.611+0000 7fcdccfb4640 1 --2- 172.21.15.60:0/2531844847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdc80ce7a0 0x7fcdc81511d0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.611+0000 7fcdccfb4640 1 -- 172.21.15.60:0/2531844847 >> 172.21.15.60:0/2531844847 conn(0x7fcdc80b12d0 msgr2=0x7fcdc80adc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:36.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.611+0000 7fcdccfb4640 1 -- 172.21.15.60:0/2531844847 shutdown_connections 2024-04-25T10:14:36.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:36.611+0000 7fcdccfb4640 1 -- 172.21.15.60:0/2531844847 wait complete. 2024-04-25T10:14:36.631 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:14:36.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:36.632 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:36.643 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[49369]: fuse finished with error 0 and tester_r 0 2024-04-25T10:14:36.647 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:42.748 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:14:42.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:42.749 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:14:42.757 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:42.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.927+0000 7f8d8f029640 1 Processor -- start 2024-04-25T10:14:42.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.927+0000 7f8d8f029640 1 -- start start 2024-04-25T10:14:42.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.927+0000 7f8d8f029640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8d8815b8d0 0x7f8d880b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:42.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.927+0000 7f8d8f029640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8d8806bc20 0x7f8d8806c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:42.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.927+0000 7f8d8f029640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d880b3370 con 0x7f8d8806ff50 2024-04-25T10:14:42.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.927+0000 7f8d8f029640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d88070390 con 0x7f8d8815b8d0 2024-04-25T10:14:42.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.927+0000 7f8d8f029640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8815c3c0 con 0x7f8d8806bc20 2024-04-25T10:14:42.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.927+0000 7f8d8cdc5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8d8815b8d0 0x7f8d880b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:42.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.927+0000 7f8d8cdc5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8d8815b8d0 0x7f8d880b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41358/0 (socket says 172.21.15.60:41358) 2024-04-25T10:14:42.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.927+0000 7f8d8cdc5640 1 -- 172.21.15.60:0/2036816132 learned_addr learned my addr 172.21.15.60:0/2036816132 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:42.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.927+0000 7f8d8d5c6640 1 --2- 172.21.15.60:0/2036816132 >> v2:172.21.15.100:3300/0 conn(0x7f8d8806bc20 0x7f8d8806c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:42.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.927+0000 7f8d8cdc5640 1 -- 172.21.15.60:0/2036816132 >> v2:172.21.15.100:3300/0 conn(0x7f8d8806bc20 msgr2=0x7f8d8806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:42.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.927+0000 7f8d8cdc5640 1 --2- 172.21.15.60:0/2036816132 >> v2:172.21.15.100:3300/0 conn(0x7f8d8806bc20 0x7f8d8806c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.927+0000 7f8d8cdc5640 1 -- 172.21.15.60:0/2036816132 >> v1:172.21.15.100:6789/0 conn(0x7f8d8806ff50 legacy=0x7f8d8815bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:42.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.927+0000 7f8d8cdc5640 1 -- 172.21.15.60:0/2036816132 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d880b3650 con 0x7f8d8815b8d0 2024-04-25T10:14:42.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.931+0000 7f8d8cdc5640 1 --2- 172.21.15.60:0/2036816132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8815b8d0 0x7f8d880b2d90 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f8d88071b00 tx=0x7f8d8001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347d36693f0a7023 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:42.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.931+0000 7f8d877fe640 1 -- 172.21.15.60:0/2036816132 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d800a57a0 con 0x7f8d8815b8d0 2024-04-25T10:14:42.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.931+0000 7f8d877fe640 1 -- 172.21.15.60:0/2036816132 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d800a5940 con 0x7f8d8815b8d0 2024-04-25T10:14:42.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.931+0000 7f8d877fe640 1 -- 172.21.15.60:0/2036816132 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d800a5d10 con 0x7f8d8815b8d0 2024-04-25T10:14:42.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.931+0000 7f8d8f029640 1 -- 172.21.15.60:0/2036816132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8815b8d0 msgr2=0x7f8d880b2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:42.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.931+0000 7f8d8f029640 1 --2- 172.21.15.60:0/2036816132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8815b8d0 0x7f8d880b2d90 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f8d88071b00 tx=0x7f8d8001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.931+0000 7f8d8f029640 1 -- 172.21.15.60:0/2036816132 shutdown_connections 2024-04-25T10:14:42.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.931+0000 7f8d8f029640 1 --2- 172.21.15.60:0/2036816132 >> v2:172.21.15.100:3300/0 conn(0x7f8d8806bc20 0x7f8d8806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.931+0000 7f8d8f029640 1 --2- 172.21.15.60:0/2036816132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8815b8d0 0x7f8d880b2d90 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.931+0000 7f8d8f029640 1 -- 172.21.15.60:0/2036816132 >> 172.21.15.60:0/2036816132 conn(0x7f8d88071b70 msgr2=0x7f8d88071f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:42.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.931+0000 7f8d8f029640 1 -- 172.21.15.60:0/2036816132 shutdown_connections 2024-04-25T10:14:42.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.931+0000 7f8d8f029640 1 -- 172.21.15.60:0/2036816132 wait complete. 2024-04-25T10:14:42.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.931+0000 7f8d8f029640 1 Processor -- start 2024-04-25T10:14:42.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.931+0000 7f8d8f029640 1 -- start start 2024-04-25T10:14:42.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8f029640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d8806bc20 0x7f8d8814beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:42.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8f029640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d8806ff50 0x7f8d880b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:42.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8f029640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8815b8d0 0x7f8d88157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:42.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8f029640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d880af2e0 con 0x7f8d8815b8d0 2024-04-25T10:14:42.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8f029640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d880b4c20 con 0x7f8d8806ff50 2024-04-25T10:14:42.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8f029640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d880b1890 con 0x7f8d8806bc20 2024-04-25T10:14:42.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8d5c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8815b8d0 0x7f8d88157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:42.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8d5c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8815b8d0 0x7f8d88157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41370/0 (socket says 172.21.15.60:41370) 2024-04-25T10:14:42.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8d5c6640 1 -- 172.21.15.60:0/2348116215 learned_addr learned my addr 172.21.15.60:0/2348116215 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:42.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d87fff640 1 --2- 172.21.15.60:0/2348116215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d8806ff50 0x7f8d880b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:42.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8cdc5640 1 --2- 172.21.15.60:0/2348116215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d8806bc20 0x7f8d8814beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:42.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8d5c6640 1 -- 172.21.15.60:0/2348116215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d8806bc20 msgr2=0x7f8d8814beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:42.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8d5c6640 1 --2- 172.21.15.60:0/2348116215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d8806bc20 0x7f8d8814beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8d5c6640 1 -- 172.21.15.60:0/2348116215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d8806ff50 msgr2=0x7f8d880b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:42.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8d5c6640 1 --2- 172.21.15.60:0/2348116215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d8806ff50 0x7f8d880b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8d5c6640 1 -- 172.21.15.60:0/2348116215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d880ad920 con 0x7f8d8815b8d0 2024-04-25T10:14:42.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8cdc5640 1 --2- 172.21.15.60:0/2348116215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d8806bc20 0x7f8d8814beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:42.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d87fff640 1 --2- 172.21.15.60:0/2348116215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d8806ff50 0x7f8d880b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:42.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8d5c6640 1 --2- 172.21.15.60:0/2348116215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8815b8d0 0x7f8d88157360 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800d830 tx=0x7f8d7800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:42.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d857fa640 1 -- 172.21.15.60:0/2348116215 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7800b6c0 con 0x7f8d8815b8d0 2024-04-25T10:14:42.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d857fa640 1 -- 172.21.15.60:0/2348116215 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d78019070 con 0x7f8d8815b8d0 2024-04-25T10:14:42.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d857fa640 1 -- 172.21.15.60:0/2348116215 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7800baa0 con 0x7f8d8815b8d0 2024-04-25T10:14:42.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8f029640 1 -- 172.21.15.60:0/2348116215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d880adc00 con 0x7f8d8815b8d0 2024-04-25T10:14:42.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8f029640 1 -- 172.21.15.60:0/2348116215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d88154a90 con 0x7f8d8815b8d0 2024-04-25T10:14:42.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d8f029640 1 -- 172.21.15.60:0/2348116215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d88074770 con 0x7f8d8815b8d0 2024-04-25T10:14:42.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d857fa640 1 -- 172.21.15.60:0/2348116215 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d7801e460 con 0x7f8d8815b8d0 2024-04-25T10:14:42.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d857fa640 1 --2- 172.21.15.60:0/2348116215 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8d74066f60 0x7f8d74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:42.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.935+0000 7f8d857fa640 1 -- 172.21.15.60:0/2348116215 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d78085640 con 0x7f8d8815b8d0 2024-04-25T10:14:42.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.939+0000 7f8d8cdc5640 1 --2- 172.21.15.60:0/2348116215 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8d74066f60 0x7f8d74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:42.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.939+0000 7f8d8cdc5640 1 --2- 172.21.15.60:0/2348116215 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8d74066f60 0x7f8d74069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8d800ae000 tx=0x7f8d800a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:42.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:42.943+0000 7f8d857fa640 1 -- 172.21.15.60:0/2348116215 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d78056640 con 0x7f8d8815b8d0 2024-04-25T10:14:43.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.135+0000 7f8d8f029640 1 -- 172.21.15.60:0/2348116215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d880727c0 con 0x7f8d8815b8d0 2024-04-25T10:14:43.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.135+0000 7f8d857fa640 1 -- 172.21.15.60:0/2348116215 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f8d78055f40 con 0x7f8d8815b8d0 2024-04-25T10:14:43.141 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:43.142 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1491228634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1491228634},{"type":"v1","addr":"172.21.15.100:6837","nonce":1491228634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/209898872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":209898872},{"type":"v1","addr":"172.21.15.100:6835","nonce":209898872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1793343287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1793343287},{"type":"v1","addr":"172.21.15.60:6833","nonce":1793343287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:16.146481+0000","modified":"2024-04-25T10:14:19.300954+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2002755047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2002755047},{"type":"v1","addr":"172.21.15.60:6835","nonce":2002755047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:14:43.142 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 77 2024-04-25T10:14:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.139+0000 7f8d8f029640 1 -- 172.21.15.60:0/2348116215 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8d74066f60 msgr2=0x7f8d74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.139+0000 7f8d8f029640 1 --2- 172.21.15.60:0/2348116215 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8d74066f60 0x7f8d74069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8d800ae000 tx=0x7f8d800a7070 comp rx=0 tx=0).stop 2024-04-25T10:14:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.139+0000 7f8d8f029640 1 -- 172.21.15.60:0/2348116215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8815b8d0 msgr2=0x7f8d88157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.139+0000 7f8d8f029640 1 --2- 172.21.15.60:0/2348116215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8815b8d0 0x7f8d88157360 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800d830 tx=0x7f8d7800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.139+0000 7f8d8f029640 1 -- 172.21.15.60:0/2348116215 shutdown_connections 2024-04-25T10:14:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.139+0000 7f8d8f029640 1 --2- 172.21.15.60:0/2348116215 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8d74066f60 0x7f8d74069420 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.139+0000 7f8d8f029640 1 --2- 172.21.15.60:0/2348116215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8815b8d0 0x7f8d88157360 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.139+0000 7f8d8f029640 1 --2- 172.21.15.60:0/2348116215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d8806ff50 0x7f8d880b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.139+0000 7f8d8f029640 1 --2- 172.21.15.60:0/2348116215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d8806bc20 0x7f8d8814beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.139+0000 7f8d8f029640 1 -- 172.21.15.60:0/2348116215 >> 172.21.15.60:0/2348116215 conn(0x7f8d88071b70 msgr2=0x7f8d88165290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.139+0000 7f8d8f029640 1 -- 172.21.15.60:0/2348116215 shutdown_connections 2024-04-25T10:14:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.139+0000 7f8d8f029640 1 -- 172.21.15.60:0/2348116215 wait complete. 2024-04-25T10:14:43.161 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:14:43.312 INFO:teuthology.orchestra.run.smithi060.stdout:{"message":"","return_code":0} 2024-04-25T10:14:43.313 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:14:43.313 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:14:43.313 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:43.319 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:43.319 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi060.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:43.319 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:14:43.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:43.319 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:14:43.366 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:14:43.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:43.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:14:43.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:43.412 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-25T10:14:43.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.543+0000 7f9079395640 1 Processor -- start 2024-04-25T10:14:43.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.543+0000 7f9079395640 1 -- start start 2024-04-25T10:14:43.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.543+0000 7f9079395640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f907415b680 0x7f907406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.543+0000 7f9079395640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f907406bc20 0x7f907406c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.543+0000 7f9079395640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90740b2f10 con 0x7f90740af110 2024-04-25T10:14:43.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.543+0000 7f9079395640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90740b3840 con 0x7f907415b680 2024-04-25T10:14:43.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.543+0000 7f9079395640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90740b4ce0 con 0x7f907406bc20 2024-04-25T10:14:43.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.543+0000 7f9072ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f907415b680 0x7f907406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.543+0000 7f9072ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f907415b680 0x7f907406fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41376/0 (socket says 172.21.15.60:41376) 2024-04-25T10:14:43.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.543+0000 7f9072ffd640 1 -- 172.21.15.60:0/1932963136 learned_addr learned my addr 172.21.15.60:0/1932963136 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:43.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.543+0000 7f90737fe640 1 --2- 172.21.15.60:0/1932963136 >> v2:172.21.15.100:3300/0 conn(0x7f907406bc20 0x7f907406c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.543+0000 7f9072ffd640 1 -- 172.21.15.60:0/1932963136 >> v2:172.21.15.100:3300/0 conn(0x7f907406bc20 msgr2=0x7f907406c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.543+0000 7f9072ffd640 1 --2- 172.21.15.60:0/1932963136 >> v2:172.21.15.100:3300/0 conn(0x7f907406bc20 0x7f907406c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.543+0000 7f9072ffd640 1 -- 172.21.15.60:0/1932963136 >> v1:172.21.15.100:6789/0 conn(0x7f90740af110 legacy=0x7f907415ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.543+0000 7f9072ffd640 1 -- 172.21.15.60:0/1932963136 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90740b31f0 con 0x7f907415b680 2024-04-25T10:14:43.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9072ffd640 1 --2- 172.21.15.60:0/1932963136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f907415b680 0x7f907406fea0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f907406fa10 tx=0x7f906401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7a4008941c8416 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f90727fc640 1 -- 172.21.15.60:0/1932963136 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90640b0070 con 0x7f907415b680 2024-04-25T10:14:43.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f90727fc640 1 -- 172.21.15.60:0/1932963136 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90640ab600 con 0x7f907415b680 2024-04-25T10:14:43.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f90727fc640 1 -- 172.21.15.60:0/1932963136 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90640ab9d0 con 0x7f907415b680 2024-04-25T10:14:43.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9079395640 1 -- 172.21.15.60:0/1932963136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f907415b680 msgr2=0x7f907406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9079395640 1 --2- 172.21.15.60:0/1932963136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f907415b680 0x7f907406fea0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f907406fa10 tx=0x7f906401c3c0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9079395640 1 -- 172.21.15.60:0/1932963136 shutdown_connections 2024-04-25T10:14:43.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9079395640 1 --2- 172.21.15.60:0/1932963136 >> v2:172.21.15.100:3300/0 conn(0x7f907406bc20 0x7f907406c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9079395640 1 --2- 172.21.15.60:0/1932963136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f907415b680 0x7f907406fea0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9079395640 1 -- 172.21.15.60:0/1932963136 >> 172.21.15.60:0/1932963136 conn(0x7f9074071b00 msgr2=0x7f9074071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9079395640 1 -- 172.21.15.60:0/1932963136 shutdown_connections 2024-04-25T10:14:43.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9079395640 1 -- 172.21.15.60:0/1932963136 wait complete. 2024-04-25T10:14:43.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9079395640 1 Processor -- start 2024-04-25T10:14:43.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9079395640 1 -- start start 2024-04-25T10:14:43.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9079395640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f907406bc20 0x7f90741557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9079395640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90740af110 0x7f907415a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9079395640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f907415b680 0x7f907415a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9079395640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90740b4870 con 0x7f907406bc20 2024-04-25T10:14:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9079395640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90740b46f0 con 0x7f90740af110 2024-04-25T10:14:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9079395640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90740b49f0 con 0x7f907415b680 2024-04-25T10:14:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9072ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f907406bc20 0x7f90741557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9072ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f907406bc20 0x7f90741557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41392/0 (socket says 172.21.15.60:41392) 2024-04-25T10:14:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9072ffd640 1 -- 172.21.15.60:0/3939176504 learned_addr learned my addr 172.21.15.60:0/3939176504 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f906a7fc640 1 --2- 172.21.15.60:0/3939176504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90740af110 0x7f907415a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f90737fe640 1 --2- 172.21.15.60:0/3939176504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f907415b680 0x7f907415a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9072ffd640 1 -- 172.21.15.60:0/3939176504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f907415b680 msgr2=0x7f907415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.547+0000 7f9072ffd640 1 --2- 172.21.15.60:0/3939176504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f907415b680 0x7f907415a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.551+0000 7f9072ffd640 1 -- 172.21.15.60:0/3939176504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90740af110 msgr2=0x7f907415a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.551+0000 7f9072ffd640 1 --2- 172.21.15.60:0/3939176504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90740af110 0x7f907415a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.551+0000 7f9072ffd640 1 -- 172.21.15.60:0/3939176504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90740cee80 con 0x7f907406bc20 2024-04-25T10:14:43.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.551+0000 7f906a7fc640 1 --2- 172.21.15.60:0/3939176504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90740af110 0x7f907415a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:43.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.551+0000 7f9072ffd640 1 --2- 172.21.15.60:0/3939176504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f907406bc20 0x7f90741557d0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f90640ae040 tx=0x7f906401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.551+0000 7f906bfff640 1 -- 172.21.15.60:0/3939176504 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90640ba020 con 0x7f907406bc20 2024-04-25T10:14:43.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.551+0000 7f906bfff640 1 -- 172.21.15.60:0/3939176504 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90640b6420 con 0x7f907406bc20 2024-04-25T10:14:43.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.551+0000 7f9079395640 1 -- 172.21.15.60:0/3939176504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90740cf160 con 0x7f907406bc20 2024-04-25T10:14:43.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.551+0000 7f906bfff640 1 -- 172.21.15.60:0/3939176504 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90640b0040 con 0x7f907406bc20 2024-04-25T10:14:43.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.551+0000 7f9079395640 1 -- 172.21.15.60:0/3939176504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90740cf720 con 0x7f907406bc20 2024-04-25T10:14:43.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.551+0000 7f906bfff640 1 -- 172.21.15.60:0/3939176504 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f90640bd020 con 0x7f907406bc20 2024-04-25T10:14:43.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.551+0000 7f906bfff640 1 --2- 172.21.15.60:0/3939176504 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9054066f90 0x7f9054069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.551+0000 7f906bfff640 1 -- 172.21.15.60:0/3939176504 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f906411f300 con 0x7f907406bc20 2024-04-25T10:14:43.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.551+0000 7f907c502000 1 Processor -- start 2024-04-25T10:14:43.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.551+0000 7f906a7fc640 1 --2- 172.21.15.60:0/3939176504 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9054066f90 0x7f9054069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.551+0000 7f907c502000 1 -- start start 2024-04-25T10:14:43.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e1918c90 0x55f0e1919070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e19195b0 0x55f0e18ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e18cf290 0x55f0e18cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e17b5f40 con 0x55f0e1918c90 2024-04-25T10:14:43.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e171d070 con 0x55f0e19195b0 2024-04-25T10:14:43.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0e16a51a0 con 0x55f0e18cf290 2024-04-25T10:14:43.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f906a7fc640 1 --2- 172.21.15.60:0/3939176504 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9054066f90 0x7f9054069450 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f907415af00 tx=0x7f905c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f9072ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e1918c90 0x55f0e1919070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f9072ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e1918c90 0x55f0e1919070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41402/0 (socket says 172.21.15.60:41402) 2024-04-25T10:14:43.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f9072ffd640 1 -- 172.21.15.60:0/2748346641 learned_addr learned my addr 172.21.15.60:0/2748346641 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:43.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f906a7fc640 1 --2- 172.21.15.60:0/2748346641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e19195b0 0x55f0e18ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f90737fe640 1 --2- 172.21.15.60:0/2748346641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e18cf290 0x55f0e18cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f9072ffd640 1 -- 172.21.15.60:0/2748346641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e18cf290 msgr2=0x55f0e18cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f9072ffd640 1 --2- 172.21.15.60:0/2748346641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e18cf290 0x55f0e18cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f9072ffd640 1 -- 172.21.15.60:0/2748346641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e19195b0 msgr2=0x55f0e18ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f9072ffd640 1 --2- 172.21.15.60:0/2748346641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e19195b0 0x55f0e18ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f9072ffd640 1 -- 172.21.15.60:0/2748346641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0e16a3510 con 0x55f0e1918c90 2024-04-25T10:14:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f9072ffd640 1 --2- 172.21.15.60:0/2748346641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e1918c90 0x55f0e1919070 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f90640ef920 tx=0x7f9064009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c03d5af0f90206 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f9068ff9640 1 -- 172.21.15.60:0/2748346641 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90640f3d70 con 0x55f0e1918c90 2024-04-25T10:14:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f9068ff9640 1 -- 172.21.15.60:0/2748346641 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90640f8330 con 0x55f0e1918c90 2024-04-25T10:14:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f9068ff9640 1 -- 172.21.15.60:0/2748346641 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90640ec290 con 0x55f0e1918c90 2024-04-25T10:14:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 -- 172.21.15.60:0/2748346641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e1918c90 msgr2=0x55f0e1919070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 --2- 172.21.15.60:0/2748346641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e1918c90 0x55f0e1919070 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f90640ef920 tx=0x7f9064009240 comp rx=0 tx=0).stop 2024-04-25T10:14:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 -- 172.21.15.60:0/2748346641 shutdown_connections 2024-04-25T10:14:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 --2- 172.21.15.60:0/2748346641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e18cf290 0x55f0e18cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 --2- 172.21.15.60:0/2748346641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e19195b0 0x55f0e18ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 --2- 172.21.15.60:0/2748346641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e1918c90 0x55f0e1919070 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 -- 172.21.15.60:0/2748346641 >> 172.21.15.60:0/2748346641 conn(0x55f0e18cb240 msgr2=0x55f0e18cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 -- 172.21.15.60:0/2748346641 shutdown_connections 2024-04-25T10:14:43.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 -- 172.21.15.60:0/2748346641 wait complete. 2024-04-25T10:14:43.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 -- start start 2024-04-25T10:14:43.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 Processor -- start 2024-04-25T10:14:43.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e18cf290 0x55f0e18d9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e1918c90 0x55f0e18da370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e19195b0 0x55f0e18da8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e16d76c0 con 0x55f0e1918c90 2024-04-25T10:14:43.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e1719110 con 0x55f0e19195b0 2024-04-25T10:14:43.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0e17b5f40 con 0x55f0e18cf290 2024-04-25T10:14:43.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f907c502000 10 client.0 _ll_register_callbacks cb 0x55f0e18ba570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:43.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f906a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e1918c90 0x55f0e18da370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f906a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e1918c90 0x55f0e18da370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41406/0 (socket says 172.21.15.60:41406) 2024-04-25T10:14:43.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f906a7fc640 1 -- 172.21.15.60:0/3479759085 learned_addr learned my addr 172.21.15.60:0/3479759085 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:43.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.555+0000 7f90737fe640 1 --2- 172.21.15.60:0/3479759085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e19195b0 0x55f0e18da8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f9072ffd640 1 --2- 172.21.15.60:0/3479759085 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e18cf290 0x55f0e18d9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f906a7fc640 1 -- 172.21.15.60:0/3479759085 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e18cf290 msgr2=0x55f0e18d9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f906a7fc640 1 --2- 172.21.15.60:0/3479759085 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e18cf290 0x55f0e18d9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f906a7fc640 1 -- 172.21.15.60:0/3479759085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e19195b0 msgr2=0x55f0e18da8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f906a7fc640 1 --2- 172.21.15.60:0/3479759085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e19195b0 0x55f0e18da8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f906a7fc640 1 -- 172.21.15.60:0/3479759085 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0e16a53b0 con 0x55f0e1918c90 2024-04-25T10:14:43.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f9072ffd640 1 --2- 172.21.15.60:0/3479759085 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e18cf290 0x55f0e18d9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:43.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f90737fe640 1 --2- 172.21.15.60:0/3479759085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e19195b0 0x55f0e18da8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:43.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f906a7fc640 1 --2- 172.21.15.60:0/3479759085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e1918c90 0x55f0e18da370 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f905c006ef0 tx=0x7f905c003770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8366a38d815f01bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f9051ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:14:43.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f9051ffb640 1 -- 172.21.15.60:0/3479759085 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c005030 con 0x55f0e1918c90 2024-04-25T10:14:43.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f9051ffb640 1 -- 172.21.15.60:0/3479759085 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f905c0051d0 con 0x55f0e1918c90 2024-04-25T10:14:43.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 1 -- 172.21.15.60:0/3479759085 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90640f8330 con 0x55f0e1918c90 2024-04-25T10:14:43.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f9051ffb640 1 -- 172.21.15.60:0/3479759085 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f905c00e7b0 con 0x55f0e1918c90 2024-04-25T10:14:43.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 10 client.8973 fetch_fsmap learned FSMap version 77 2024-04-25T10:14:43.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 1 -- 172.21.15.60:0/3479759085 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55f0e179ee00 con 0x55f0e1918c90 2024-04-25T10:14:43.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f9051ffb640 1 -- 172.21.15.60:0/3479759085 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90640f8330 con 0x55f0e1918c90 2024-04-25T10:14:43.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f9051ffb640 1 -- 172.21.15.60:0/3479759085 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f905c02d070 con 0x55f0e1918c90 2024-04-25T10:14:43.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 10 client.8973 fetch_fsmap finished waiting for FSMap version 77 2024-04-25T10:14:43.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 10 client.8973 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:14:43.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 20 client.8973 populate_metadata read hostname 'smithi060' 2024-04-25T10:14:43.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 1 --2- 172.21.15.60:0/3479759085 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x55f0e18db180 0x55f0e192a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 4 client.8973 mds_command: new command op to 8631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:43.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 1 -- 172.21.15.60:0/3479759085 --> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0e16b8d60 con 0x55f0e18db180 2024-04-25T10:14:43.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f9072ffd640 1 --2- 172.21.15.60:0/3479759085 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x55f0e18db180 0x55f0e192a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f9072ffd640 1 --2- 172.21.15.60:0/3479759085 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x55f0e18db180 0x55f0e192a410 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aed43eb12583662 server_cookie=5cc956a6598ba567 in_seq=0 out_seq=0 2024-04-25T10:14:43.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f9051ffb640 10 client.8973 ms_handle_connect on v2:172.21.15.60:6834/2002755047 2024-04-25T10:14:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f9051ffb640 1 -- 172.21.15.60:0/3479759085 <== mds.0 v2:172.21.15.60:6834/2002755047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f0e16b8d60 con 0x55f0e18db180 2024-04-25T10:14:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f9051ffb640 10 client.8973 handle_command_reply: tid=0 2024-04-25T10:14:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 1 client.8973 shutdown 2024-04-25T10:14:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 1 -- 172.21.15.60:0/3479759085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e1918c90 msgr2=0x55f0e18da370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 1 --2- 172.21.15.60:0/3479759085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e1918c90 0x55f0e18da370 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f905c006ef0 tx=0x7f905c003770 comp rx=0 tx=0).stop 2024-04-25T10:14:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 1 -- 172.21.15.60:0/3479759085 shutdown_connections 2024-04-25T10:14:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 1 --2- 172.21.15.60:0/3479759085 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x55f0e18db180 0x55f0e192a410 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 1 --2- 172.21.15.60:0/3479759085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e19195b0 0x55f0e18da8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 1 --2- 172.21.15.60:0/3479759085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e1918c90 0x55f0e18da370 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f9051ffb640 0 client.8973 ms_handle_reset on v2:172.21.15.60:6834/2002755047 2024-04-25T10:14:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 1 --2- 172.21.15.60:0/3479759085 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e18cf290 0x55f0e18d9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 1 -- 172.21.15.60:0/3479759085 >> 172.21.15.60:0/3479759085 conn(0x55f0e18cb240 msgr2=0x55f0e18cdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 1 -- 172.21.15.60:0/3479759085 shutdown_connections 2024-04-25T10:14:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 1 -- 172.21.15.60:0/3479759085 wait complete. 2024-04-25T10:14:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.559+0000 7f907c502000 20 client.8973 trim_cache size 0 max 16384 2024-04-25T10:14:43.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.563+0000 7f4f20793640 1 Processor -- start 2024-04-25T10:14:43.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.563+0000 7f4f20793640 1 -- start start 2024-04-25T10:14:43.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.563+0000 7f4f20793640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4f1815bb00 0x7f4f1809a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.563+0000 7f4f20793640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f18099b60 con 0x7f4f1815f570 2024-04-25T10:14:43.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.563+0000 7f4f20793640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f18099e40 con 0x7f4f180d04c0 2024-04-25T10:14:43.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.563+0000 7f4f20793640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1815bee0 con 0x7f4f1815bb00 2024-04-25T10:14:43.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.563+0000 7f4f1ed30640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4f1815f570 0x7f4f1815f980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40472/0 (socket says 172.21.15.60:40472) 2024-04-25T10:14:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.563+0000 7f4f1ed30640 1 -- 172.21.15.60:0/1936413819 learned_addr learned my addr 172.21.15.60:0/1936413819 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.563+0000 7f4f1e52f640 1 --2- 172.21.15.60:0/1936413819 >> v2:172.21.15.60:3300/0 conn(0x7f4f1815bb00 0x7f4f1809a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.563+0000 7f4f1d52d640 1 -- 172.21.15.60:0/1936413819 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647024599 0 0) 0x7f4f18099b60 con 0x7f4f1815f570 2024-04-25T10:14:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.563+0000 7f4f1d52d640 1 -- 172.21.15.60:0/1936413819 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f04003530 con 0x7f4f1815f570 2024-04-25T10:14:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.563+0000 7f4f1e52f640 1 -- 172.21.15.60:0/1936413819 >> v1:172.21.15.100:6789/0 conn(0x7f4f180d04c0 legacy=0x7f4f180ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.563+0000 7f4f1e52f640 1 -- 172.21.15.60:0/1936413819 >> v1:172.21.15.60:6789/0 conn(0x7f4f1815f570 legacy=0x7f4f1815f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.563+0000 7f4f1e52f640 1 -- 172.21.15.60:0/1936413819 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1809a120 con 0x7f4f1815bb00 2024-04-25T10:14:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f1e52f640 1 --2- 172.21.15.60:0/1936413819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f1815bb00 0x7f4f1809a920 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f4f140097c0 tx=0x7f4f1401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76cae3fa298ee6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f1d52d640 1 -- 172.21.15.60:0/1936413819 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f140b0070 con 0x7f4f1815bb00 2024-04-25T10:14:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f1d52d640 1 -- 172.21.15.60:0/1936413819 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f140ab600 con 0x7f4f1815bb00 2024-04-25T10:14:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f1d52d640 1 -- 172.21.15.60:0/1936413819 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f140ab9d0 con 0x7f4f1815bb00 2024-04-25T10:14:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f20793640 1 -- 172.21.15.60:0/1936413819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f1815bb00 msgr2=0x7f4f1809a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f20793640 1 --2- 172.21.15.60:0/1936413819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f1815bb00 0x7f4f1809a920 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f4f140097c0 tx=0x7f4f1401c380 comp rx=0 tx=0).stop 2024-04-25T10:14:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f20793640 1 -- 172.21.15.60:0/1936413819 shutdown_connections 2024-04-25T10:14:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f20793640 1 --2- 172.21.15.60:0/1936413819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f1815bb00 0x7f4f1809a920 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f20793640 1 -- 172.21.15.60:0/1936413819 >> 172.21.15.60:0/1936413819 conn(0x7f4f180b31f0 msgr2=0x7f4f180b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f20793640 1 -- 172.21.15.60:0/1936413819 shutdown_connections 2024-04-25T10:14:43.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f20793640 1 -- 172.21.15.60:0/1936413819 wait complete. 2024-04-25T10:14:43.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f20793640 1 Processor -- start 2024-04-25T10:14:43.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f20793640 1 -- start start 2024-04-25T10:14:43.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f20793640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f180d04c0 0x7f4f18154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f20793640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f1815bb00 0x7f4f18149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f20793640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f1815f570 0x7f4f180af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f20793640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f180b19f0 con 0x7f4f1815bb00 2024-04-25T10:14:43.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f20793640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f180b1870 con 0x7f4f180d04c0 2024-04-25T10:14:43.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f20793640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f180b1b70 con 0x7f4f1815f570 2024-04-25T10:14:43.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f1dd2e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f1815bb00 0x7f4f18149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f1dd2e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f1815bb00 0x7f4f18149d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41424/0 (socket says 172.21.15.60:41424) 2024-04-25T10:14:43.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f1dd2e640 1 -- 172.21.15.60:0/733567075 learned_addr learned my addr 172.21.15.60:0/733567075 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:43.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f1ed30640 1 --2- 172.21.15.60:0/733567075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f1815f570 0x7f4f180af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f1e52f640 1 --2- 172.21.15.60:0/733567075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f180d04c0 0x7f4f18154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f1dd2e640 1 -- 172.21.15.60:0/733567075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f1815f570 msgr2=0x7f4f180af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f1dd2e640 1 --2- 172.21.15.60:0/733567075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f1815f570 0x7f4f180af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f1dd2e640 1 -- 172.21.15.60:0/733567075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f180d04c0 msgr2=0x7f4f18154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f1dd2e640 1 --2- 172.21.15.60:0/733567075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f180d04c0 0x7f4f18154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f1dd2e640 1 -- 172.21.15.60:0/733567075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f18154930 con 0x7f4f1815bb00 2024-04-25T10:14:43.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f1e52f640 1 --2- 172.21.15.60:0/733567075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f180d04c0 0x7f4f18154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:43.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f1dd2e640 1 --2- 172.21.15.60:0/733567075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f1815bb00 0x7f4f18149d30 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800c3a0 tx=0x7f4f0800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.567+0000 7f4f0effd640 1 -- 172.21.15.60:0/733567075 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f080052d0 con 0x7f4f1815bb00 2024-04-25T10:14:43.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.571+0000 7f4f0effd640 1 -- 172.21.15.60:0/733567075 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f08005470 con 0x7f4f1815bb00 2024-04-25T10:14:43.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.571+0000 7f4f0effd640 1 -- 172.21.15.60:0/733567075 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f08007260 con 0x7f4f1815bb00 2024-04-25T10:14:43.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.571+0000 7f4f20793640 1 -- 172.21.15.60:0/733567075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f18154c10 con 0x7f4f1815bb00 2024-04-25T10:14:43.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.571+0000 7f4f20793640 1 -- 172.21.15.60:0/733567075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f181551d0 con 0x7f4f1815bb00 2024-04-25T10:14:43.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.571+0000 7f4f20793640 1 -- 172.21.15.60:0/733567075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f4f18074770 con 0x7f4f1815bb00 2024-04-25T10:14:43.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.571+0000 7f4f0effd640 1 -- 172.21.15.60:0/733567075 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f08007400 con 0x7f4f1815bb00 2024-04-25T10:14:43.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.571+0000 7f4f0effd640 1 --2- 172.21.15.60:0/733567075 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4efc066f60 0x7f4efc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.571+0000 7f4f0effd640 1 -- 172.21.15.60:0/733567075 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f08084ff0 con 0x7f4f1815bb00 2024-04-25T10:14:43.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.571+0000 7f4f0effd640 1 -- 172.21.15.60:0/733567075 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+511 (secure 0 0 0) 0x7f4f080854a0 con 0x7f4f1815bb00 2024-04-25T10:14:43.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.571+0000 7f4f1e52f640 1 --2- 172.21.15.60:0/733567075 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4efc066f60 0x7f4efc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.571+0000 7f4f20793640 1 -- 172.21.15.60:0/733567075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f4f180b3f30 con 0x7f4f1815bb00 2024-04-25T10:14:43.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.571+0000 7f4f1e52f640 1 --2- 172.21.15.60:0/733567075 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4efc066f60 0x7f4efc069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4f140ad000 tx=0x7f4f14008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.575+0000 7f907c502000 1 Processor -- start 2024-04-25T10:14:43.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.575+0000 7f907c502000 1 -- start start 2024-04-25T10:14:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.575+0000 7f907c502000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e18cf290 0x55f0e18ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.575+0000 7f907c502000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e1918c90 0x55f0e18ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e19195b0 0x55f0e192a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9074200850 con 0x55f0e18cf290 2024-04-25T10:14:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9074153880 con 0x55f0e1918c90 2024-04-25T10:14:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9074152c80 con 0x55f0e19195b0 2024-04-25T10:14:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e18cf290 0x55f0e18ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e18cf290 0x55f0e18ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41436/0 (socket says 172.21.15.60:41436) 2024-04-25T10:14:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 -- 172.21.15.60:0/153133567 learned_addr learned my addr 172.21.15.60:0/153133567 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 -- 172.21.15.60:0/153133567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e19195b0 msgr2=0x55f0e192a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f90737fe640 1 --2- 172.21.15.60:0/153133567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e19195b0 0x55f0e192a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f906a7fc640 1 --2- 172.21.15.60:0/153133567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e1918c90 0x55f0e18ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 --2- 172.21.15.60:0/153133567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e19195b0 0x55f0e192a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 -- 172.21.15.60:0/153133567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e1918c90 msgr2=0x55f0e18ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 --2- 172.21.15.60:0/153133567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e1918c90 0x55f0e18ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 -- 172.21.15.60:0/153133567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0e1774c50 con 0x55f0e18cf290 2024-04-25T10:14:43.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f90737fe640 1 --2- 172.21.15.60:0/153133567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e19195b0 0x55f0e192a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:43.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 --2- 172.21.15.60:0/153133567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e18cf290 0x55f0e18ce640 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f906401c930 tx=0x7f90640f1000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292acd334d611744 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9053fff640 1 -- 172.21.15.60:0/153133567 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90640edf20 con 0x55f0e18cf290 2024-04-25T10:14:43.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9053fff640 1 -- 172.21.15.60:0/153133567 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90640ec290 con 0x55f0e18cf290 2024-04-25T10:14:43.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9053fff640 1 -- 172.21.15.60:0/153133567 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064103de0 con 0x55f0e18cf290 2024-04-25T10:14:43.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 -- 172.21.15.60:0/153133567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e18cf290 msgr2=0x55f0e18ce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 --2- 172.21.15.60:0/153133567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e18cf290 0x55f0e18ce640 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f906401c930 tx=0x7f90640f1000 comp rx=0 tx=0).stop 2024-04-25T10:14:43.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 -- 172.21.15.60:0/153133567 shutdown_connections 2024-04-25T10:14:43.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 --2- 172.21.15.60:0/153133567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e19195b0 0x55f0e192a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 --2- 172.21.15.60:0/153133567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e1918c90 0x55f0e18ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 --2- 172.21.15.60:0/153133567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e18cf290 0x55f0e18ce640 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 -- 172.21.15.60:0/153133567 >> 172.21.15.60:0/153133567 conn(0x55f0e18cb240 msgr2=0x55f0e18d3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 -- 172.21.15.60:0/153133567 shutdown_connections 2024-04-25T10:14:43.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 -- 172.21.15.60:0/153133567 wait complete. 2024-04-25T10:14:43.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 -- start start 2024-04-25T10:14:43.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 Processor -- start 2024-04-25T10:14:43.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e18cf290 0x55f0e197d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e1918c90 0x55f0e197d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e19195b0 0x55f0e19382a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e171c380 con 0x55f0e18cf290 2024-04-25T10:14:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9074153280 con 0x55f0e1918c90 2024-04-25T10:14:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0e18d8200 con 0x55f0e19195b0 2024-04-25T10:14:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f907c502000 10 client.0 _ll_register_callbacks cb 0x7f9064009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e18cf290 0x55f0e197d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e18cf290 0x55f0e197d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41452/0 (socket says 172.21.15.60:41452) 2024-04-25T10:14:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 -- 172.21.15.60:0/1397488281 learned_addr learned my addr 172.21.15.60:0/1397488281 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f90737fe640 1 --2- 172.21.15.60:0/1397488281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e19195b0 0x55f0e19382a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 -- 172.21.15.60:0/1397488281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e19195b0 msgr2=0x55f0e19382a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f906a7fc640 1 --2- 172.21.15.60:0/1397488281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e1918c90 0x55f0e197d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 --2- 172.21.15.60:0/1397488281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e19195b0 0x55f0e19382a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 -- 172.21.15.60:0/1397488281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e1918c90 msgr2=0x55f0e197d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 --2- 172.21.15.60:0/1397488281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e1918c90 0x55f0e197d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 -- 172.21.15.60:0/1397488281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0e16e5120 con 0x55f0e18cf290 2024-04-25T10:14:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9072ffd640 1 --2- 172.21.15.60:0/1397488281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e18cf290 0x55f0e197d460 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f9064097da0 tx=0x7f906411d790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46a29db5a1ee6bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9051ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:14:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9051ffb640 1 -- 172.21.15.60:0/1397488281 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90640ef580 con 0x55f0e18cf290 2024-04-25T10:14:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.579+0000 7f9051ffb640 1 -- 172.21.15.60:0/1397488281 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90640edf20 con 0x55f0e18cf290 2024-04-25T10:14:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 1 -- 172.21.15.60:0/1397488281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90640ec290 con 0x55f0e18cf290 2024-04-25T10:14:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9051ffb640 1 -- 172.21.15.60:0/1397488281 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9064103be0 con 0x55f0e18cf290 2024-04-25T10:14:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 10 client.8988 fetch_fsmap learned FSMap version 77 2024-04-25T10:14:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 1 -- 172.21.15.60:0/1397488281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55f0e16c4170 con 0x55f0e18cf290 2024-04-25T10:14:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9051ffb640 1 -- 172.21.15.60:0/1397488281 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90640ec290 con 0x55f0e18cf290 2024-04-25T10:14:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9051ffb640 1 -- 172.21.15.60:0/1397488281 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f90640b3030 con 0x55f0e18cf290 2024-04-25T10:14:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 10 client.8988 fetch_fsmap finished waiting for FSMap version 77 2024-04-25T10:14:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 10 client.8988 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:14:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 20 client.8988 populate_metadata read hostname 'smithi060' 2024-04-25T10:14:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 1 --2- 172.21.15.60:0/1397488281 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x55f0e1938c90 0x55f0e1939070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 4 client.8988 mds_command: new command op to 8631 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:14:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 1 -- 172.21.15.60:0/1397488281 --> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55f0e172a5b0 con 0x55f0e1938c90 2024-04-25T10:14:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f906a7fc640 1 --2- 172.21.15.60:0/1397488281 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x55f0e1938c90 0x55f0e1939070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9051ffb640 10 client.8988 ms_handle_connect on v2:172.21.15.60:6834/2002755047 2024-04-25T10:14:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f906a7fc640 1 --2- 172.21.15.60:0/1397488281 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x55f0e1938c90 0x55f0e1939070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d69ac2953b9402 server_cookie=14dff533a3369273 in_seq=0 out_seq=0 2024-04-25T10:14:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9051ffb640 1 -- 172.21.15.60:0/1397488281 <== mds.0 v2:172.21.15.60:6834/2002755047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f0e172a5b0 con 0x55f0e1938c90 2024-04-25T10:14:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9051ffb640 10 client.8988 handle_command_reply: tid=0 2024-04-25T10:14:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 1 client.8988 shutdown 2024-04-25T10:14:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 1 -- 172.21.15.60:0/1397488281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e18cf290 msgr2=0x55f0e197d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 1 --2- 172.21.15.60:0/1397488281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e18cf290 0x55f0e197d460 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f9064097da0 tx=0x7f906411d790 comp rx=0 tx=0).stop 2024-04-25T10:14:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 1 -- 172.21.15.60:0/1397488281 shutdown_connections 2024-04-25T10:14:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 1 --2- 172.21.15.60:0/1397488281 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x55f0e1938c90 0x55f0e1939070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 1 --2- 172.21.15.60:0/1397488281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f0e19195b0 0x55f0e19382a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 1 --2- 172.21.15.60:0/1397488281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f0e1918c90 0x55f0e197d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9051ffb640 0 client.8988 ms_handle_reset on v2:172.21.15.60:6834/2002755047 2024-04-25T10:14:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 1 --2- 172.21.15.60:0/1397488281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f0e18cf290 0x55f0e197d460 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 1 -- 172.21.15.60:0/1397488281 >> 172.21.15.60:0/1397488281 conn(0x55f0e18cb240 msgr2=0x55f0e18d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 1 -- 172.21.15.60:0/1397488281 shutdown_connections 2024-04-25T10:14:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 1 -- 172.21.15.60:0/1397488281 wait complete. 2024-04-25T10:14:43.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f907c502000 20 client.8988 trim_cache size 0 max 16384 2024-04-25T10:14:43.597 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:14:43.597 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T10:14:43.597 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "b35a8b3e-a32d-4cb8-a2fb-3477e1fe1098", 2024-04-25T10:14:43.597 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T10:14:43.598 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:14:43.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9079395640 1 -- 172.21.15.60:0/3939176504 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9054066f90 msgr2=0x7f9054069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9079395640 1 --2- 172.21.15.60:0/3939176504 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9054066f90 0x7f9054069450 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f907415af00 tx=0x7f905c009040 comp rx=0 tx=0).stop 2024-04-25T10:14:43.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9079395640 1 -- 172.21.15.60:0/3939176504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f907406bc20 msgr2=0x7f90741557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9079395640 1 --2- 172.21.15.60:0/3939176504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f907406bc20 0x7f90741557d0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f90640ae040 tx=0x7f906401ac00 comp rx=0 tx=0).stop 2024-04-25T10:14:43.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9079395640 1 -- 172.21.15.60:0/3939176504 shutdown_connections 2024-04-25T10:14:43.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9079395640 1 --2- 172.21.15.60:0/3939176504 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9054066f90 0x7f9054069450 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9079395640 1 --2- 172.21.15.60:0/3939176504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f907415b680 0x7f907415a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9079395640 1 --2- 172.21.15.60:0/3939176504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90740af110 0x7f907415a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9079395640 1 --2- 172.21.15.60:0/3939176504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f907406bc20 0x7f90741557d0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9079395640 1 -- 172.21.15.60:0/3939176504 >> 172.21.15.60:0/3939176504 conn(0x7f9074071b00 msgr2=0x7f9074150b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9079395640 1 -- 172.21.15.60:0/3939176504 shutdown_connections 2024-04-25T10:14:43.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.583+0000 7f9079395640 1 -- 172.21.15.60:0/3939176504 wait complete. 2024-04-25T10:14:43.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:43.605 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:14:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.719+0000 7f1c2f113640 1 Processor -- start 2024-04-25T10:14:43.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.719+0000 7f1c2f113640 1 -- start start 2024-04-25T10:14:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.719+0000 7f1c2f113640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1c2815dd30 0x7f1c2815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.719+0000 7f1c2f113640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c280cd890 con 0x7f1c280cd3d0 2024-04-25T10:14:43.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.719+0000 7f1c2f113640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c280cdb70 con 0x7f1c280cf8a0 2024-04-25T10:14:43.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.719+0000 7f1c2f113640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c280ce4a0 con 0x7f1c2815dd30 2024-04-25T10:14:43.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.719+0000 7f1c2ceaf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1c2815dd30 0x7f1c2815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.719+0000 7f1c27fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1c280cd3d0 0x7f1c280cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40480/0 (socket says 172.21.15.60:40480) 2024-04-25T10:14:43.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.719+0000 7f1c27fff640 1 -- 172.21.15.60:0/4098456216 learned_addr learned my addr 172.21.15.60:0/4098456216 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:14:43.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.719+0000 7f1c2ceaf640 1 -- 172.21.15.60:0/4098456216 >> v1:172.21.15.100:6789/0 conn(0x7f1c280cf8a0 legacy=0x7f1c2815be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.719+0000 7f1c2ceaf640 1 -- 172.21.15.60:0/4098456216 >> v1:172.21.15.60:6789/0 conn(0x7f1c280cd3d0 legacy=0x7f1c280cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.719+0000 7f1c277fe640 1 -- 172.21.15.60:0/4098456216 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1554788723 0 0) 0x7f1c280cd890 con 0x7f1c280cd3d0 2024-04-25T10:14:43.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.719+0000 7f1c2ceaf640 1 -- 172.21.15.60:0/4098456216 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c280cde50 con 0x7f1c2815dd30 2024-04-25T10:14:43.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2ceaf640 1 --2- 172.21.15.60:0/4098456216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815dd30 0x7f1c2815ac30 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0097c0 tx=0x7f1c1c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f06007589faf2c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c277fe640 1 -- 172.21.15.60:0/4098456216 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0ab600 con 0x7f1c2815dd30 2024-04-25T10:14:43.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c277fe640 1 -- 172.21.15.60:0/4098456216 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c0b0070 con 0x7f1c2815dd30 2024-04-25T10:14:43.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c277fe640 1 -- 172.21.15.60:0/4098456216 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0ab9c0 con 0x7f1c2815dd30 2024-04-25T10:14:43.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2f113640 1 -- 172.21.15.60:0/4098456216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815dd30 msgr2=0x7f1c2815ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2f113640 1 --2- 172.21.15.60:0/4098456216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815dd30 0x7f1c2815ac30 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0097c0 tx=0x7f1c1c01c3e0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2f113640 1 -- 172.21.15.60:0/4098456216 shutdown_connections 2024-04-25T10:14:43.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2f113640 1 --2- 172.21.15.60:0/4098456216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815dd30 0x7f1c2815ac30 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2f113640 1 -- 172.21.15.60:0/4098456216 >> 172.21.15.60:0/4098456216 conn(0x7f1c280670b0 msgr2=0x7f1c2806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2f113640 1 -- 172.21.15.60:0/4098456216 shutdown_connections 2024-04-25T10:14:43.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2f113640 1 -- 172.21.15.60:0/4098456216 wait complete. 2024-04-25T10:14:43.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2f113640 1 Processor -- start 2024-04-25T10:14:43.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2f113640 1 -- start start 2024-04-25T10:14:43.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2f113640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c280cd3d0 0x7f1c280afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2f113640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c280cf8a0 0x7f1c280cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2f113640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c2815dd30 0x7f1c280af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2f113640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c281682b0 con 0x7f1c280cd3d0 2024-04-25T10:14:43.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2f113640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2815cfd0 con 0x7f1c280cf8a0 2024-04-25T10:14:43.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2f113640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c28168430 con 0x7f1c2815dd30 2024-04-25T10:14:43.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2ceaf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c280cd3d0 0x7f1c280afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2ceaf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c280cd3d0 0x7f1c280afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41464/0 (socket says 172.21.15.60:41464) 2024-04-25T10:14:43.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2ceaf640 1 -- 172.21.15.60:0/3228165158 learned_addr learned my addr 172.21.15.60:0/3228165158 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:43.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c27fff640 1 --2- 172.21.15.60:0/3228165158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c280cf8a0 0x7f1c280cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2d6b0640 1 --2- 172.21.15.60:0/3228165158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c2815dd30 0x7f1c280af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2ceaf640 1 -- 172.21.15.60:0/3228165158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c2815dd30 msgr2=0x7f1c280af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2ceaf640 1 --2- 172.21.15.60:0/3228165158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c2815dd30 0x7f1c280af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2ceaf640 1 -- 172.21.15.60:0/3228165158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c280cf8a0 msgr2=0x7f1c280cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2ceaf640 1 --2- 172.21.15.60:0/3228165158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c280cf8a0 0x7f1c280cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.723+0000 7f1c2ceaf640 1 -- 172.21.15.60:0/3228165158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c281431c0 con 0x7f1c280cd3d0 2024-04-25T10:14:43.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c2ceaf640 1 --2- 172.21.15.60:0/3228165158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c280cd3d0 0x7f1c280afff0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0ad040 tx=0x7f1c1c01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c257fa640 1 -- 172.21.15.60:0/3228165158 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0b5810 con 0x7f1c280cd3d0 2024-04-25T10:14:43.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c257fa640 1 -- 172.21.15.60:0/3228165158 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c0b59b0 con 0x7f1c280cd3d0 2024-04-25T10:14:43.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c257fa640 1 -- 172.21.15.60:0/3228165158 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0b5da0 con 0x7f1c280cd3d0 2024-04-25T10:14:43.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c2f113640 1 -- 172.21.15.60:0/3228165158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c281434a0 con 0x7f1c280cd3d0 2024-04-25T10:14:43.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c2f113640 1 -- 172.21.15.60:0/3228165158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c280b0530 con 0x7f1c280cd3d0 2024-04-25T10:14:43.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c257fa640 1 -- 172.21.15.60:0/3228165158 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c1c0bd020 con 0x7f1c280cd3d0 2024-04-25T10:14:43.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c257fa640 1 --2- 172.21.15.60:0/3228165158 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c14066f60 0x7f1c14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c257fa640 1 -- 172.21.15.60:0/3228165158 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c1c11f950 con 0x7f1c280cd3d0 2024-04-25T10:14:43.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c27fff640 1 --2- 172.21.15.60:0/3228165158 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c14066f60 0x7f1c14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c32280000 1 Processor -- start 2024-04-25T10:14:43.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c32280000 1 -- start start 2024-04-25T10:14:43.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c27fff640 1 --2- 172.21.15.60:0/3228165158 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c14066f60 0x7f1c14069420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1c280b46b0 tx=0x7f1c18007240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c32280000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d6149d0 0x55f12d614db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c32280000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6152f0 0x55f12d5cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c32280000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d5cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c32280000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f12d417ff0 con 0x55f12d5cb0a0 2024-04-25T10:14:43.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c32280000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f12d3a0150 con 0x55f12d6149d0 2024-04-25T10:14:43.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.727+0000 7f1c32280000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f12d414090 con 0x55f12d6152f0 2024-04-25T10:14:43.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2d6b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d5cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2d6b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d5cb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41472/0 (socket says 172.21.15.60:41472) 2024-04-25T10:14:43.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2d6b0640 1 -- 172.21.15.60:0/2950173750 learned_addr learned my addr 172.21.15.60:0/2950173750 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:43.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2ceaf640 1 --2- 172.21.15.60:0/2950173750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d6149d0 0x55f12d614db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c27fff640 1 --2- 172.21.15.60:0/2950173750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6152f0 0x55f12d5cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2d6b0640 1 -- 172.21.15.60:0/2950173750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6152f0 msgr2=0x55f12d5cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2d6b0640 1 --2- 172.21.15.60:0/2950173750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6152f0 0x55f12d5cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2d6b0640 1 -- 172.21.15.60:0/2950173750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d6149d0 msgr2=0x55f12d614db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2d6b0640 1 --2- 172.21.15.60:0/2950173750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d6149d0 0x55f12d614db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2d6b0640 1 -- 172.21.15.60:0/2950173750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f12d5ae0d0 con 0x55f12d5cb0a0 2024-04-25T10:14:43.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2ceaf640 1 --2- 172.21.15.60:0/2950173750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d6149d0 0x55f12d614db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:43.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2d6b0640 1 --2- 172.21.15.60:0/2950173750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d5cb480 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f1c20001bc0 tx=0x7f1c20002080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6992d21c25f6c4a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c0e7fc640 1 -- 172.21.15.60:0/2950173750 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20019070 con 0x55f12d5cb0a0 2024-04-25T10:14:43.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c0e7fc640 1 -- 172.21.15.60:0/2950173750 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c20003b90 con 0x55f12d5cb0a0 2024-04-25T10:14:43.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c0e7fc640 1 -- 172.21.15.60:0/2950173750 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2000e800 con 0x55f12d5cb0a0 2024-04-25T10:14:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 -- 172.21.15.60:0/2950173750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 msgr2=0x55f12d5cb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 --2- 172.21.15.60:0/2950173750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d5cb480 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f1c20001bc0 tx=0x7f1c20002080 comp rx=0 tx=0).stop 2024-04-25T10:14:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 -- 172.21.15.60:0/2950173750 shutdown_connections 2024-04-25T10:14:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 --2- 172.21.15.60:0/2950173750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d5cb480 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 --2- 172.21.15.60:0/2950173750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6152f0 0x55f12d5cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 --2- 172.21.15.60:0/2950173750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d6149d0 0x55f12d614db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 -- 172.21.15.60:0/2950173750 >> 172.21.15.60:0/2950173750 conn(0x55f12d5c7020 msgr2=0x55f12d5c9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 -- 172.21.15.60:0/2950173750 shutdown_connections 2024-04-25T10:14:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 -- 172.21.15.60:0/2950173750 wait complete. 2024-04-25T10:14:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 -- start start 2024-04-25T10:14:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 Processor -- start 2024-04-25T10:14:43.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d5f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6149d0 0x55f12d5f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d6152f0 0x55f12d5fa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f12d3dc250 con 0x55f12d5cb0a0 2024-04-25T10:14:43.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f12d3d2690 con 0x55f12d6152f0 2024-04-25T10:14:43.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f12d3a0a40 con 0x55f12d6149d0 2024-04-25T10:14:43.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c32280000 10 client.0 _ll_register_callbacks cb 0x55f12d5b5e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:43.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2ceaf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d5f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2ceaf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d5f97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41488/0 (socket says 172.21.15.60:41488) 2024-04-25T10:14:43.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2ceaf640 1 -- 172.21.15.60:0/690641266 learned_addr learned my addr 172.21.15.60:0/690641266 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:43.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2d6b0640 1 --2- 172.21.15.60:0/690641266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d6152f0 0x55f12d5fa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c27fff640 1 --2- 172.21.15.60:0/690641266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6149d0 0x55f12d5f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2ceaf640 1 -- 172.21.15.60:0/690641266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6149d0 msgr2=0x55f12d5f9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2ceaf640 1 --2- 172.21.15.60:0/690641266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6149d0 0x55f12d5f9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2ceaf640 1 -- 172.21.15.60:0/690641266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d6152f0 msgr2=0x55f12d5fa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2ceaf640 1 --2- 172.21.15.60:0/690641266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d6152f0 0x55f12d5fa270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2ceaf640 1 -- 172.21.15.60:0/690641266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f12d4c51c0 con 0x55f12d5cb0a0 2024-04-25T10:14:43.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c27fff640 1 --2- 172.21.15.60:0/690641266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6149d0 0x55f12d5f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:43.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c2ceaf640 1 --2- 172.21.15.60:0/690641266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d5f97f0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c018ff0 tx=0x7f1c1c0efd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5caee9804fafd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c077fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:14:43.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c077fe640 1 -- 172.21.15.60:0/690641266 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0f4070 con 0x55f12d5cb0a0 2024-04-25T10:14:43.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.731+0000 7f1c077fe640 1 -- 172.21.15.60:0/690641266 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c1c0ef760 con 0x55f12d5cb0a0 2024-04-25T10:14:43.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 1 -- 172.21.15.60:0/690641266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c20003b90 con 0x55f12d5cb0a0 2024-04-25T10:14:43.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c077fe640 1 -- 172.21.15.60:0/690641266 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c1c10e050 con 0x55f12d5cb0a0 2024-04-25T10:14:43.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 10 client.9003 fetch_fsmap learned FSMap version 77 2024-04-25T10:14:43.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 1 -- 172.21.15.60:0/690641266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55f12d4c1170 con 0x55f12d5cb0a0 2024-04-25T10:14:43.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c077fe640 1 -- 172.21.15.60:0/690641266 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c20003b90 con 0x55f12d5cb0a0 2024-04-25T10:14:43.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c077fe640 1 -- 172.21.15.60:0/690641266 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1c1c0c7030 con 0x55f12d5cb0a0 2024-04-25T10:14:43.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 10 client.9003 fetch_fsmap finished waiting for FSMap version 77 2024-04-25T10:14:43.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 10 client.9003 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:14:43.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 20 client.9003 populate_metadata read hostname 'smithi060' 2024-04-25T10:14:43.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 1 --2- 172.21.15.60:0/690641266 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x55f12d624c80 0x55f12d627070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 4 client.9003 mds_command: new command op to 8631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:43.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 1 -- 172.21.15.60:0/690641266 --> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f12d3b3c60 con 0x55f12d624c80 2024-04-25T10:14:43.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c27fff640 1 --2- 172.21.15.60:0/690641266 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x55f12d624c80 0x55f12d627070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c077fe640 10 client.9003 ms_handle_connect on v2:172.21.15.60:6834/2002755047 2024-04-25T10:14:43.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c27fff640 1 --2- 172.21.15.60:0/690641266 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x55f12d624c80 0x55f12d627070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8809ea2465b0b33 server_cookie=8287b557ad08833a in_seq=0 out_seq=0 2024-04-25T10:14:43.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c077fe640 1 -- 172.21.15.60:0/690641266 <== mds.0 v2:172.21.15.60:6834/2002755047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f12d3b3c60 con 0x55f12d624c80 2024-04-25T10:14:43.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c077fe640 10 client.9003 handle_command_reply: tid=0 2024-04-25T10:14:43.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 1 client.9003 shutdown 2024-04-25T10:14:43.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 1 -- 172.21.15.60:0/690641266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 msgr2=0x55f12d5f97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 1 --2- 172.21.15.60:0/690641266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d5f97f0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c018ff0 tx=0x7f1c1c0efd60 comp rx=0 tx=0).stop 2024-04-25T10:14:43.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 1 -- 172.21.15.60:0/690641266 shutdown_connections 2024-04-25T10:14:43.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 1 --2- 172.21.15.60:0/690641266 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x55f12d624c80 0x55f12d627070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 1 --2- 172.21.15.60:0/690641266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d6152f0 0x55f12d5fa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 1 --2- 172.21.15.60:0/690641266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6149d0 0x55f12d5f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 1 --2- 172.21.15.60:0/690641266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d5f97f0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c077fe640 0 client.9003 ms_handle_reset on v2:172.21.15.60:6834/2002755047 2024-04-25T10:14:43.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 1 -- 172.21.15.60:0/690641266 >> 172.21.15.60:0/690641266 conn(0x55f12d5c7020 msgr2=0x55f12d5c9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 1 -- 172.21.15.60:0/690641266 shutdown_connections 2024-04-25T10:14:43.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 1 -- 172.21.15.60:0/690641266 wait complete. 2024-04-25T10:14:43.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.735+0000 7f1c32280000 20 client.9003 trim_cache size 0 max 16384 2024-04-25T10:14:43.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 Processor -- start 2024-04-25T10:14:43.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 -- start start 2024-04-25T10:14:43.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d5cb0a0 0x55f12d6710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6149d0 0x55f12d671630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d6152f0 0x55f12d671b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c28202990 con 0x55f12d6152f0 2024-04-25T10:14:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c28201d90 con 0x55f12d5cb0a0 2024-04-25T10:14:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c28144240 con 0x55f12d6149d0 2024-04-25T10:14:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c2d6b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d6152f0 0x55f12d671b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c2d6b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d6152f0 0x55f12d671b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41504/0 (socket says 172.21.15.60:41504) 2024-04-25T10:14:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c2d6b0640 1 -- 172.21.15.60:0/1634369126 learned_addr learned my addr 172.21.15.60:0/1634369126 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c2ceaf640 1 --2- 172.21.15.60:0/1634369126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d5cb0a0 0x55f12d6710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c2d6b0640 1 -- 172.21.15.60:0/1634369126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6149d0 msgr2=0x55f12d671630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c27fff640 1 --2- 172.21.15.60:0/1634369126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6149d0 0x55f12d671630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c2d6b0640 1 --2- 172.21.15.60:0/1634369126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6149d0 0x55f12d671630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c2d6b0640 1 -- 172.21.15.60:0/1634369126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d5cb0a0 msgr2=0x55f12d6710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c2d6b0640 1 --2- 172.21.15.60:0/1634369126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d5cb0a0 0x55f12d6710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c2d6b0640 1 -- 172.21.15.60:0/1634369126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f12d45ede0 con 0x55f12d6152f0 2024-04-25T10:14:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c2ceaf640 1 --2- 172.21.15.60:0/1634369126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d5cb0a0 0x55f12d6710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c2d6b0640 1 --2- 172.21.15.60:0/1634369126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d6152f0 0x55f12d671b70 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f1c200012f0 tx=0x7f1c2000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b76e61ac820bfd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c0dffb640 1 -- 172.21.15.60:0/1634369126 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20019040 con 0x55f12d6152f0 2024-04-25T10:14:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c0dffb640 1 -- 172.21.15.60:0/1634369126 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c20023070 con 0x55f12d6152f0 2024-04-25T10:14:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c0dffb640 1 -- 172.21.15.60:0/1634369126 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20003e50 con 0x55f12d6152f0 2024-04-25T10:14:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 -- 172.21.15.60:0/1634369126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d6152f0 msgr2=0x55f12d671b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 --2- 172.21.15.60:0/1634369126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d6152f0 0x55f12d671b70 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f1c200012f0 tx=0x7f1c2000a000 comp rx=0 tx=0).stop 2024-04-25T10:14:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 -- 172.21.15.60:0/1634369126 shutdown_connections 2024-04-25T10:14:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 --2- 172.21.15.60:0/1634369126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d6152f0 0x55f12d671b70 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 --2- 172.21.15.60:0/1634369126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6149d0 0x55f12d671630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 --2- 172.21.15.60:0/1634369126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d5cb0a0 0x55f12d6710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 -- 172.21.15.60:0/1634369126 >> 172.21.15.60:0/1634369126 conn(0x55f12d5c7020 msgr2=0x55f12d6130a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 -- 172.21.15.60:0/1634369126 shutdown_connections 2024-04-25T10:14:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 -- 172.21.15.60:0/1634369126 wait complete. 2024-04-25T10:14:43.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.751+0000 7f1c32280000 1 -- start start 2024-04-25T10:14:43.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 Processor -- start 2024-04-25T10:14:43.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d627bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6149d0 0x55f12d628110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d6152f0 0x55f12d628650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f12d3b2e00 con 0x55f12d5cb0a0 2024-04-25T10:14:43.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c28144840 con 0x55f12d6152f0 2024-04-25T10:14:43.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f12d417300 con 0x55f12d6149d0 2024-04-25T10:14:43.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 10 client.0 _ll_register_callbacks cb 0x55f12d596040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:43.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2ceaf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d627bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2ceaf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d627bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41510/0 (socket says 172.21.15.60:41510) 2024-04-25T10:14:43.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2ceaf640 1 -- 172.21.15.60:0/3131826472 learned_addr learned my addr 172.21.15.60:0/3131826472 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:14:43.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2d6b0640 1 --2- 172.21.15.60:0/3131826472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d6152f0 0x55f12d628650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2ceaf640 1 -- 172.21.15.60:0/3131826472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6149d0 msgr2=0x55f12d628110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2ceaf640 1 --2- 172.21.15.60:0/3131826472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6149d0 0x55f12d628110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2ceaf640 1 -- 172.21.15.60:0/3131826472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d6152f0 msgr2=0x55f12d628650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2ceaf640 1 --2- 172.21.15.60:0/3131826472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d6152f0 0x55f12d628650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2ceaf640 1 -- 172.21.15.60:0/3131826472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f12d3b98e0 con 0x55f12d5cb0a0 2024-04-25T10:14:43.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2ceaf640 1 --2- 172.21.15.60:0/3131826472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d627bd0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c01c210 tx=0x7f1c1c11dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffb96835cc23257 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c077fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:14:43.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c077fe640 1 -- 172.21.15.60:0/3131826472 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0efc70 con 0x55f12d5cb0a0 2024-04-25T10:14:43.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c077fe640 1 -- 172.21.15.60:0/3131826472 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c1c11c040 con 0x55f12d5cb0a0 2024-04-25T10:14:43.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 -- 172.21.15.60:0/3131826472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c20023070 con 0x55f12d5cb0a0 2024-04-25T10:14:43.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c077fe640 1 -- 172.21.15.60:0/3131826472 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c1c11ddc0 con 0x55f12d5cb0a0 2024-04-25T10:14:43.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 10 client.9009 fetch_fsmap learned FSMap version 77 2024-04-25T10:14:43.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 -- 172.21.15.60:0/3131826472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55f12d4718f0 con 0x55f12d5cb0a0 2024-04-25T10:14:43.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c077fe640 1 -- 172.21.15.60:0/3131826472 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c20023070 con 0x55f12d5cb0a0 2024-04-25T10:14:43.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c077fe640 1 -- 172.21.15.60:0/3131826472 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1c1c115070 con 0x55f12d5cb0a0 2024-04-25T10:14:43.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 10 client.9009 fetch_fsmap finished waiting for FSMap version 77 2024-04-25T10:14:43.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 10 client.9009 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:14:43.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 20 client.9009 populate_metadata read hostname 'smithi060' 2024-04-25T10:14:43.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 --2- 172.21.15.60:0/3131826472 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x55f12d67af50 0x55f12d67d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 4 client.9009 mds_command: new command op to 8631 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:14:43.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 -- 172.21.15.60:0/3131826472 --> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f12d425530 con 0x55f12d67af50 2024-04-25T10:14:43.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c27fff640 1 --2- 172.21.15.60:0/3131826472 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x55f12d67af50 0x55f12d67d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c27fff640 1 --2- 172.21.15.60:0/3131826472 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x55f12d67af50 0x55f12d67d340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc31d00ee3e1a629 server_cookie=b383f09f6847f1ef in_seq=0 out_seq=0 2024-04-25T10:14:43.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c077fe640 10 client.9009 ms_handle_connect on v2:172.21.15.60:6834/2002755047 2024-04-25T10:14:43.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c077fe640 1 -- 172.21.15.60:0/3131826472 <== mds.0 v2:172.21.15.60:6834/2002755047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55f12d425530 con 0x55f12d67af50 2024-04-25T10:14:43.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c077fe640 10 client.9009 handle_command_reply: tid=0 2024-04-25T10:14:43.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 client.9009 shutdown 2024-04-25T10:14:43.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 -- 172.21.15.60:0/3131826472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 msgr2=0x55f12d627bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 --2- 172.21.15.60:0/3131826472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d627bd0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c01c210 tx=0x7f1c1c11dd50 comp rx=0 tx=0).stop 2024-04-25T10:14:43.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 -- 172.21.15.60:0/3131826472 shutdown_connections 2024-04-25T10:14:43.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 --2- 172.21.15.60:0/3131826472 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x55f12d67af50 0x55f12d67d340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 --2- 172.21.15.60:0/3131826472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f12d6152f0 0x55f12d628650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 --2- 172.21.15.60:0/3131826472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f12d6149d0 0x55f12d628110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 --2- 172.21.15.60:0/3131826472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f12d5cb0a0 0x55f12d627bd0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c077fe640 0 client.9009 ms_handle_reset on v2:172.21.15.60:6834/2002755047 2024-04-25T10:14:43.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 -- 172.21.15.60:0/3131826472 >> 172.21.15.60:0/3131826472 conn(0x55f12d5c7020 msgr2=0x55f12d613930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 -- 172.21.15.60:0/3131826472 shutdown_connections 2024-04-25T10:14:43.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 1 -- 172.21.15.60:0/3131826472 wait complete. 2024-04-25T10:14:43.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c32280000 20 client.9009 trim_cache size 0 max 16384 2024-04-25T10:14:43.767 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:14:43.767 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:14:43.767 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T10:14:43.767 INFO:teuthology.orchestra.run.smithi060.stdout: "b35a8b3e-a32d-4cb8-a2fb-3477e1fe1098": { 2024-04-25T10:14:43.768 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-04-25T10:14:43.768 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "b35a8b3e-a32d-4cb8-a2fb-3477e1fe1098", 2024-04-25T10:14:43.768 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,repair" 2024-04-25T10:14:43.768 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:43.768 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:14:43.768 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:14:43.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2f113640 1 -- 172.21.15.60:0/3228165158 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c14066f60 msgr2=0x7f1c14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2f113640 1 --2- 172.21.15.60:0/3228165158 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c14066f60 0x7f1c14069420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1c280b46b0 tx=0x7f1c18007240 comp rx=0 tx=0).stop 2024-04-25T10:14:43.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2f113640 1 -- 172.21.15.60:0/3228165158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c280cd3d0 msgr2=0x7f1c280afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2f113640 1 --2- 172.21.15.60:0/3228165158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c280cd3d0 0x7f1c280afff0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0ad040 tx=0x7f1c1c01ac20 comp rx=0 tx=0).stop 2024-04-25T10:14:43.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2f113640 1 -- 172.21.15.60:0/3228165158 shutdown_connections 2024-04-25T10:14:43.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2f113640 1 --2- 172.21.15.60:0/3228165158 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c14066f60 0x7f1c14069420 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2f113640 1 --2- 172.21.15.60:0/3228165158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c2815dd30 0x7f1c280af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2f113640 1 --2- 172.21.15.60:0/3228165158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c280cf8a0 0x7f1c280cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2f113640 1 --2- 172.21.15.60:0/3228165158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c280cd3d0 0x7f1c280afff0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2f113640 1 -- 172.21.15.60:0/3228165158 >> 172.21.15.60:0/3228165158 conn(0x7f1c280670b0 msgr2=0x7f1c280b0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2f113640 1 -- 172.21.15.60:0/3228165158 shutdown_connections 2024-04-25T10:14:43.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:43.755+0000 7f1c2f113640 1 -- 172.21.15.60:0/3228165158 wait complete. 2024-04-25T10:14:43.778 INFO:tasks.cephfs.filesystem:scrub status for tag:b35a8b3e-a32d-4cb8-a2fb-3477e1fe1098 - {'path': '/', 'tag': 'b35a8b3e-a32d-4cb8-a2fb-3477e1fe1098', 'options': 'recursive,repair'} 2024-04-25T10:14:44.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:44.663+0000 7f4f0effd640 1 -- 172.21.15.60:0/733567075 <== mon.0 v2:172.21.15.60:3300/0 7 ==== log(4 entries from seq 1 at 2024-04-25T10:14:43.585174+0000) v1 ==== 695+0+0 (secure 0 0 0) 0x7f4f08083630 con 0x7f4f1815bb00 2024-04-25T10:14:44.670 INFO:teuthology.orchestra.run.smithi060.stdout: cluster: 2024-04-25T10:14:44.670 INFO:teuthology.orchestra.run.smithi060.stdout: id: 0ea93863-a079-45c4-abed-6e83540ea9fe 2024-04-25T10:14:44.670 INFO:teuthology.orchestra.run.smithi060.stdout: health: HEALTH_OK 2024-04-25T10:14:44.670 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:44.670 INFO:teuthology.orchestra.run.smithi060.stdout: services: 2024-04-25T10:14:44.670 INFO:teuthology.orchestra.run.smithi060.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-04-25T10:14:44.670 INFO:teuthology.orchestra.run.smithi060.stdout: mgr: x(active, since 9m), standbys: y 2024-04-25T10:14:44.670 INFO:teuthology.orchestra.run.smithi060.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:14:44.671 INFO:teuthology.orchestra.run.smithi060.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-04-25T10:14:44.671 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:44.671 INFO:teuthology.orchestra.run.smithi060.stdout: data: 2024-04-25T10:14:44.671 INFO:teuthology.orchestra.run.smithi060.stdout: volumes: 1/1 healthy 2024-04-25T10:14:44.671 INFO:teuthology.orchestra.run.smithi060.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:14:44.671 INFO:teuthology.orchestra.run.smithi060.stdout: objects: 30 objects, 19 MiB 2024-04-25T10:14:44.671 INFO:teuthology.orchestra.run.smithi060.stdout: usage: 306 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:14:44.671 INFO:teuthology.orchestra.run.smithi060.stdout: pgs: 97 active+clean 2024-04-25T10:14:44.671 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:44.671 INFO:teuthology.orchestra.run.smithi060.stdout: io: 2024-04-25T10:14:44.671 INFO:teuthology.orchestra.run.smithi060.stdout: client: 85 B/s rd, 1.0 MiB/s wr, 0 op/s rd, 2 op/s wr 2024-04-25T10:14:44.671 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:44.671 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:14:44.672 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:14:43.585174+0000 mds.a [INF] scrub queued for path: / 2024-04-25T10:14:44.672 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:14:43.585179+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:14:44.672 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:14:43.585185+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-25T10:14:44.672 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:14:43.586738+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:14:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:14:47.679+0000 7f4f0effd640 1 -- 172.21.15.60:0/733567075 <== mon.0 v2:172.21.15.60:3300/0 8 ==== log(1 entries from seq 5 at 2024-04-25T10:14:46.783757+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f4f080539a0 con 0x7f4f1815bb00 2024-04-25T10:14:47.684 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:14:46.783757+0000 mds.a [INF] scrub summary: idle 2024-04-25T10:14:47.685 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:47.687 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:13.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:13.779 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:15:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.910+0000 7f801d46b640 1 Processor -- start 2024-04-25T10:15:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.910+0000 7f801d46b640 1 -- start start 2024-04-25T10:15:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.914+0000 7f801d46b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f801815b6f0 0x7f80181620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.914+0000 7f801d46b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801815e0d0 0x7f801815e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.914+0000 7f801d46b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f801815bad0 con 0x7f80180ce7e0 2024-04-25T10:15:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.914+0000 7f801d46b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8018162610 con 0x7f801815b6f0 2024-04-25T10:15:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.914+0000 7f801d46b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8018161c40 con 0x7f801815e0d0 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.914+0000 7f80167fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801815e0d0 0x7f801815e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.914+0000 7f8016ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f801815b6f0 0x7f80181620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.914+0000 7f80167fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801815e0d0 0x7f801815e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:38132/0 (socket says 172.21.15.60:38132) 2024-04-25T10:15:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.914+0000 7f80167fc640 1 -- 172.21.15.60:0/3806397826 learned_addr learned my addr 172.21.15.60:0/3806397826 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.914+0000 7f8015ffb640 1 -- 172.21.15.60:0/3806397826 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460588287 0 0) 0x7f801815bad0 con 0x7f80180ce7e0 2024-04-25T10:15:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.914+0000 7f8015ffb640 1 -- 172.21.15.60:0/3806397826 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8004003530 con 0x7f80180ce7e0 2024-04-25T10:15:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.914+0000 7f80167fc640 1 -- 172.21.15.60:0/3806397826 >> v2:172.21.15.100:3300/0 conn(0x7f801815b6f0 msgr2=0x7f80181620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.914+0000 7f80167fc640 1 --2- 172.21.15.60:0/3806397826 >> v2:172.21.15.100:3300/0 conn(0x7f801815b6f0 0x7f80181620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.914+0000 7f80167fc640 1 -- 172.21.15.60:0/3806397826 >> v1:172.21.15.100:6789/0 conn(0x7f80180ce7e0 legacy=0x7f80180d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.914+0000 7f80167fc640 1 -- 172.21.15.60:0/3806397826 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801815ebe0 con 0x7f801815e0d0 2024-04-25T10:15:13.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f80167fc640 1 --2- 172.21.15.60:0/3806397826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801815e0d0 0x7f801815e510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8000009e50 tx=0x7f800001ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1d8ce37fa359b84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f8015ffb640 1 -- 172.21.15.60:0/3806397826 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000a57f0 con 0x7f801815e0d0 2024-04-25T10:15:13.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f8015ffb640 1 -- 172.21.15.60:0/3806397826 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80000a5990 con 0x7f801815e0d0 2024-04-25T10:15:13.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f8015ffb640 1 -- 172.21.15.60:0/3806397826 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000a5d60 con 0x7f801815e0d0 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f801d46b640 1 -- 172.21.15.60:0/3806397826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801815e0d0 msgr2=0x7f801815e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f801d46b640 1 --2- 172.21.15.60:0/3806397826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801815e0d0 0x7f801815e510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8000009e50 tx=0x7f800001ca30 comp rx=0 tx=0).stop 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f801d46b640 1 -- 172.21.15.60:0/3806397826 shutdown_connections 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f801d46b640 1 --2- 172.21.15.60:0/3806397826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801815e0d0 0x7f801815e510 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f801d46b640 1 --2- 172.21.15.60:0/3806397826 >> v2:172.21.15.100:3300/0 conn(0x7f801815b6f0 0x7f80181620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f801d46b640 1 -- 172.21.15.60:0/3806397826 >> 172.21.15.60:0/3806397826 conn(0x7f8018072340 msgr2=0x7f8018072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f801d46b640 1 -- 172.21.15.60:0/3806397826 shutdown_connections 2024-04-25T10:15:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f801d46b640 1 -- 172.21.15.60:0/3806397826 wait complete. 2024-04-25T10:15:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f801d46b640 1 Processor -- start 2024-04-25T10:15:13.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f801d46b640 1 -- start start 2024-04-25T10:15:13.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f801d46b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80180ce7e0 0x7f801815a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f801d46b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801815b6f0 0x7f80181fa630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f801d46b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f801815e0d0 0x7f80181fab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f801d46b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80180b96d0 con 0x7f80180ce7e0 2024-04-25T10:15:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f801d46b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80180b9550 con 0x7f801815e0d0 2024-04-25T10:15:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.918+0000 7f801d46b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80180cf000 con 0x7f801815b6f0 2024-04-25T10:15:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f8016ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80180ce7e0 0x7f801815a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f8016ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80180ce7e0 0x7f801815a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58058/0 (socket says 172.21.15.60:58058) 2024-04-25T10:15:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f8016ffd640 1 -- 172.21.15.60:0/3066535291 learned_addr learned my addr 172.21.15.60:0/3066535291 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f80177fe640 1 --2- 172.21.15.60:0/3066535291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f801815e0d0 0x7f80181fab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f80167fc640 1 --2- 172.21.15.60:0/3066535291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801815b6f0 0x7f80181fa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f8016ffd640 1 -- 172.21.15.60:0/3066535291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801815b6f0 msgr2=0x7f80181fa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f8016ffd640 1 --2- 172.21.15.60:0/3066535291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801815b6f0 0x7f80181fa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f8016ffd640 1 -- 172.21.15.60:0/3066535291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f801815e0d0 msgr2=0x7f80181fab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f8016ffd640 1 --2- 172.21.15.60:0/3066535291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f801815e0d0 0x7f80181fab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f8016ffd640 1 -- 172.21.15.60:0/3066535291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80180ca1f0 con 0x7f80180ce7e0 2024-04-25T10:15:13.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f8016ffd640 1 --2- 172.21.15.60:0/3066535291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80180ce7e0 0x7f801815a320 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f800800e960 tx=0x7f800800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f800f7fe640 1 -- 172.21.15.60:0/3066535291 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008017070 con 0x7f80180ce7e0 2024-04-25T10:15:13.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f800f7fe640 1 -- 172.21.15.60:0/3066535291 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800800ca00 con 0x7f80180ce7e0 2024-04-25T10:15:13.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f801d46b640 1 -- 172.21.15.60:0/3066535291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80180b40d0 con 0x7f80180ce7e0 2024-04-25T10:15:13.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f800f7fe640 1 -- 172.21.15.60:0/3066535291 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800800cdb0 con 0x7f80180ce7e0 2024-04-25T10:15:13.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f801d46b640 1 -- 172.21.15.60:0/3066535291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801814d820 con 0x7f80180ce7e0 2024-04-25T10:15:13.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f800f7fe640 1 -- 172.21.15.60:0/3066535291 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f800801a070 con 0x7f80180ce7e0 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f800f7fe640 1 --2- 172.21.15.60:0/3066535291 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ff8066f60 0x7f7ff8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f800f7fe640 1 -- 172.21.15.60:0/3066535291 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8008085450 con 0x7f80180ce7e0 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.922+0000 7f80167fc640 1 --2- 172.21.15.60:0/3066535291 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ff8066f60 0x7f7ff8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 Processor -- start 2024-04-25T10:15:13.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 -- start start 2024-04-25T10:15:13.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80167fc640 1 --2- 172.21.15.60:0/3066535291 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ff8066f60 0x7f7ff8069420 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f80000ae000 tx=0x7f80000a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d59d0 0x556ed00d5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed00d62f0 0x556ed008bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed008c0a0 0x556ed008c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ecfed8ff0 con 0x556ed00d62f0 2024-04-25T10:15:13.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ecfe61150 con 0x556ed00d59d0 2024-04-25T10:15:13.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ecfed5090 con 0x556ed008c0a0 2024-04-25T10:15:13.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80167fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed00d62f0 0x556ed008bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80167fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed00d62f0 0x556ed008bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58066/0 (socket says 172.21.15.60:58066) 2024-04-25T10:15:13.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80167fc640 1 -- 172.21.15.60:0/1674352378 learned_addr learned my addr 172.21.15.60:0/1674352378 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:13.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80177fe640 1 --2- 172.21.15.60:0/1674352378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed008c0a0 0x556ed008c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f8016ffd640 1 --2- 172.21.15.60:0/1674352378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d59d0 0x556ed00d5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80167fc640 1 -- 172.21.15.60:0/1674352378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed008c0a0 msgr2=0x556ed008c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80167fc640 1 --2- 172.21.15.60:0/1674352378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed008c0a0 0x556ed008c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80167fc640 1 -- 172.21.15.60:0/1674352378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d59d0 msgr2=0x556ed00d5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80167fc640 1 --2- 172.21.15.60:0/1674352378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d59d0 0x556ed00d5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80167fc640 1 -- 172.21.15.60:0/1674352378 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ed006f0d0 con 0x556ed00d62f0 2024-04-25T10:15:13.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f8016ffd640 1 --2- 172.21.15.60:0/1674352378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d59d0 0x556ed00d5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:13.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80167fc640 1 --2- 172.21.15.60:0/1674352378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed00d62f0 0x556ed008bb60 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f800001a660 tx=0x7f8000004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce0e041abc35ad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f800cff9640 1 -- 172.21.15.60:0/1674352378 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000c7070 con 0x556ed00d62f0 2024-04-25T10:15:13.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f800cff9640 1 -- 172.21.15.60:0/1674352378 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80000b6620 con 0x556ed00d62f0 2024-04-25T10:15:13.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f800cff9640 1 -- 172.21.15.60:0/1674352378 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000b69d0 con 0x556ed00d62f0 2024-04-25T10:15:13.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 -- 172.21.15.60:0/1674352378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed00d62f0 msgr2=0x556ed008bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 --2- 172.21.15.60:0/1674352378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed00d62f0 0x556ed008bb60 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f800001a660 tx=0x7f8000004ac0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 -- 172.21.15.60:0/1674352378 shutdown_connections 2024-04-25T10:15:13.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 --2- 172.21.15.60:0/1674352378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed008c0a0 0x556ed008c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 --2- 172.21.15.60:0/1674352378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed00d62f0 0x556ed008bb60 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 --2- 172.21.15.60:0/1674352378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d59d0 0x556ed00d5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 -- 172.21.15.60:0/1674352378 >> 172.21.15.60:0/1674352378 conn(0x556ed0088020 msgr2=0x556ed008a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 -- 172.21.15.60:0/1674352378 shutdown_connections 2024-04-25T10:15:13.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 -- 172.21.15.60:0/1674352378 wait complete. 2024-04-25T10:15:13.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 -- start start 2024-04-25T10:15:13.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.926+0000 7f80205d8000 1 Processor -- start 2024-04-25T10:15:13.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed00ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d59d0 0x556ed00bad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed00d62f0 0x556ed00bb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ecfe9d250 con 0x556ed008c0a0 2024-04-25T10:15:13.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ecfe93690 con 0x556ed00d59d0 2024-04-25T10:15:13.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ecfe61a40 con 0x556ed00d62f0 2024-04-25T10:15:13.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 10 client.0 _ll_register_callbacks cb 0x556ed0076e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f8016ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed00ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f8016ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed00ba7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58076/0 (socket says 172.21.15.60:58076) 2024-04-25T10:15:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f8016ffd640 1 -- 172.21.15.60:0/1674809831 learned_addr learned my addr 172.21.15.60:0/1674809831 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80167fc640 1 --2- 172.21.15.60:0/1674809831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d59d0 0x556ed00bad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f8016ffd640 1 -- 172.21.15.60:0/1674809831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed00d62f0 msgr2=0x556ed00bb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f8016ffd640 1 --2- 172.21.15.60:0/1674809831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed00d62f0 0x556ed00bb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f8016ffd640 1 -- 172.21.15.60:0/1674809831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d59d0 msgr2=0x556ed00bad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f8016ffd640 1 --2- 172.21.15.60:0/1674809831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d59d0 0x556ed00bad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f8016ffd640 1 -- 172.21.15.60:0/1674809831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ecff861c0 con 0x556ed008c0a0 2024-04-25T10:15:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f8016ffd640 1 --2- 172.21.15.60:0/1674809831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed00ba7f0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f8008000c10 tx=0x7f8008055d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e5c65260e9fcf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f7ff5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:15:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f7ff5ffb640 1 -- 172.21.15.60:0/1674809831 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008021040 con 0x556ed008c0a0 2024-04-25T10:15:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f7ff5ffb640 1 -- 172.21.15.60:0/1674809831 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8008085070 con 0x556ed008c0a0 2024-04-25T10:15:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 -- 172.21.15.60:0/1674809831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80000b6620 con 0x556ed008c0a0 2024-04-25T10:15:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f7ff5ffb640 1 -- 172.21.15.60:0/1674809831 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8008085290 con 0x556ed008c0a0 2024-04-25T10:15:13.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 10 client.9018 fetch_fsmap learned FSMap version 77 2024-04-25T10:15:13.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 -- 172.21.15.60:0/1674809831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x556ecff82170 con 0x556ed008c0a0 2024-04-25T10:15:13.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f7ff5ffb640 1 -- 172.21.15.60:0/1674809831 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80000b6620 con 0x556ed008c0a0 2024-04-25T10:15:13.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f7ff5ffb640 1 -- 172.21.15.60:0/1674809831 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f80080551a0 con 0x556ed008c0a0 2024-04-25T10:15:13.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 10 client.9018 fetch_fsmap finished waiting for FSMap version 77 2024-04-25T10:15:13.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 10 client.9018 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:15:13.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 20 client.9018 populate_metadata read hostname 'smithi060' 2024-04-25T10:15:13.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 --2- 172.21.15.60:0/1674809831 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x556ed00e5c80 0x556ed00e8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 4 client.9018 mds_command: new command op to 8631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:13.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 -- 172.21.15.60:0/1674809831 --> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ecfe74c60 con 0x556ed00e5c80 2024-04-25T10:15:13.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80167fc640 1 --2- 172.21.15.60:0/1674809831 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x556ed00e5c80 0x556ed00e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f7ff5ffb640 10 client.9018 ms_handle_connect on v2:172.21.15.60:6834/2002755047 2024-04-25T10:15:13.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80167fc640 1 --2- 172.21.15.60:0/1674809831 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x556ed00e5c80 0x556ed00e8070 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf3020ad04a55e05 server_cookie=ad09e2199391268e in_seq=0 out_seq=0 2024-04-25T10:15:13.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f7ff5ffb640 1 -- 172.21.15.60:0/1674809831 <== mds.0 v2:172.21.15.60:6834/2002755047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556ecfe74c60 con 0x556ed00e5c80 2024-04-25T10:15:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f7ff5ffb640 10 client.9018 handle_command_reply: tid=0 2024-04-25T10:15:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 client.9018 shutdown 2024-04-25T10:15:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 -- 172.21.15.60:0/1674809831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 msgr2=0x556ed00ba7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 --2- 172.21.15.60:0/1674809831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed00ba7f0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f8008000c10 tx=0x7f8008055d50 comp rx=0 tx=0).stop 2024-04-25T10:15:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 -- 172.21.15.60:0/1674809831 shutdown_connections 2024-04-25T10:15:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 --2- 172.21.15.60:0/1674809831 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x556ed00e5c80 0x556ed00e8070 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 --2- 172.21.15.60:0/1674809831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed00d62f0 0x556ed00bb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 --2- 172.21.15.60:0/1674809831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d59d0 0x556ed00bad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 --2- 172.21.15.60:0/1674809831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed00ba7f0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f7ff5ffb640 0 client.9018 ms_handle_reset on v2:172.21.15.60:6834/2002755047 2024-04-25T10:15:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 -- 172.21.15.60:0/1674809831 >> 172.21.15.60:0/1674809831 conn(0x556ed0088020 msgr2=0x556ed008aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 -- 172.21.15.60:0/1674809831 shutdown_connections 2024-04-25T10:15:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 1 -- 172.21.15.60:0/1674809831 wait complete. 2024-04-25T10:15:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.930+0000 7f80205d8000 20 client.9018 trim_cache size 0 max 16384 2024-04-25T10:15:13.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f80205d8000 1 Processor -- start 2024-04-25T10:15:13.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f80205d8000 1 -- start start 2024-04-25T10:15:13.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f80205d8000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed01320f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f80205d8000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d59d0 0x556ed0132630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f80205d8000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed00d62f0 0x556ed0132b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f80205d8000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8018202ed0 con 0x556ed008c0a0 2024-04-25T10:15:13.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f80205d8000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80182020b0 con 0x556ed00d59d0 2024-04-25T10:15:13.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f80205d8000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80181f7f40 con 0x556ed00d62f0 2024-04-25T10:15:13.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f8016ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed01320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f8016ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed01320f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58080/0 (socket says 172.21.15.60:58080) 2024-04-25T10:15:13.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f8016ffd640 1 -- 172.21.15.60:0/3557318712 learned_addr learned my addr 172.21.15.60:0/3557318712 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:13.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f8016ffd640 1 -- 172.21.15.60:0/3557318712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed00d62f0 msgr2=0x556ed0132b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f8016ffd640 1 --2- 172.21.15.60:0/3557318712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed00d62f0 0x556ed0132b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f8016ffd640 1 -- 172.21.15.60:0/3557318712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d59d0 msgr2=0x556ed0132630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f8016ffd640 1 --2- 172.21.15.60:0/3557318712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d59d0 0x556ed0132630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f8016ffd640 1 -- 172.21.15.60:0/3557318712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ecff1fde0 con 0x556ed008c0a0 2024-04-25T10:15:13.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f8016ffd640 1 --2- 172.21.15.60:0/3557318712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed01320f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f800805c010 tx=0x7f8008052120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a1121474d2729d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f7ff7fff640 1 -- 172.21.15.60:0/3557318712 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008021040 con 0x556ed008c0a0 2024-04-25T10:15:13.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f7ff7fff640 1 -- 172.21.15.60:0/3557318712 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80080551a0 con 0x556ed008c0a0 2024-04-25T10:15:13.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.946+0000 7f7ff7fff640 1 -- 172.21.15.60:0/3557318712 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008051cc0 con 0x556ed008c0a0 2024-04-25T10:15:13.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 -- 172.21.15.60:0/3557318712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 msgr2=0x556ed01320f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 --2- 172.21.15.60:0/3557318712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed01320f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f800805c010 tx=0x7f8008052120 comp rx=0 tx=0).stop 2024-04-25T10:15:13.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 -- 172.21.15.60:0/3557318712 shutdown_connections 2024-04-25T10:15:13.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 --2- 172.21.15.60:0/3557318712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed00d62f0 0x556ed0132b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 --2- 172.21.15.60:0/3557318712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d59d0 0x556ed0132630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 --2- 172.21.15.60:0/3557318712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed01320f0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 -- 172.21.15.60:0/3557318712 >> 172.21.15.60:0/3557318712 conn(0x556ed0088020 msgr2=0x556ed00d40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 -- 172.21.15.60:0/3557318712 shutdown_connections 2024-04-25T10:15:13.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 -- 172.21.15.60:0/3557318712 wait complete. 2024-04-25T10:15:13.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 -- start start 2024-04-25T10:15:13.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 Processor -- start 2024-04-25T10:15:13.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed0136520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed00d59d0 0x556ed0136a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d62f0 0x556ed00e87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ecfe73e00 con 0x556ed008c0a0 2024-04-25T10:15:13.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8018201ae0 con 0x556ed00d62f0 2024-04-25T10:15:13.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ecfed8300 con 0x556ed00d59d0 2024-04-25T10:15:13.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 10 client.0 _ll_register_callbacks cb 0x556ed0057040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:13.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f8016ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed0136520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f8016ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed0136520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58086/0 (socket says 172.21.15.60:58086) 2024-04-25T10:15:13.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f8016ffd640 1 -- 172.21.15.60:0/3095587929 learned_addr learned my addr 172.21.15.60:0/3095587929 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:13.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80167fc640 1 --2- 172.21.15.60:0/3095587929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed00d59d0 0x556ed0136a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f8016ffd640 1 -- 172.21.15.60:0/3095587929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed00d59d0 msgr2=0x556ed0136a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f8016ffd640 1 --2- 172.21.15.60:0/3095587929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed00d59d0 0x556ed0136a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f8016ffd640 1 -- 172.21.15.60:0/3095587929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d62f0 msgr2=0x556ed00e87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f8016ffd640 1 --2- 172.21.15.60:0/3095587929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d62f0 0x556ed00e87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f8016ffd640 1 -- 172.21.15.60:0/3095587929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ecfe7a8e0 con 0x556ed008c0a0 2024-04-25T10:15:13.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f8016ffd640 1 --2- 172.21.15.60:0/3095587929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed0136520 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f800805c010 tx=0x7f8008083840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef651909ef345ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f7ff5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:15:13.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f7ff5ffb640 1 -- 172.21.15.60:0/3095587929 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008021040 con 0x556ed008c0a0 2024-04-25T10:15:13.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f7ff5ffb640 1 -- 172.21.15.60:0/3095587929 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8008085070 con 0x556ed008c0a0 2024-04-25T10:15:13.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 -- 172.21.15.60:0/3095587929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80080551a0 con 0x556ed008c0a0 2024-04-25T10:15:13.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f7ff5ffb640 1 -- 172.21.15.60:0/3095587929 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80080556b0 con 0x556ed008c0a0 2024-04-25T10:15:13.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 10 client.9024 fetch_fsmap learned FSMap version 77 2024-04-25T10:15:13.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 -- 172.21.15.60:0/3095587929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x556ecff328f0 con 0x556ed008c0a0 2024-04-25T10:15:13.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f7ff5ffb640 1 -- 172.21.15.60:0/3095587929 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80080551a0 con 0x556ed008c0a0 2024-04-25T10:15:13.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f7ff5ffb640 1 -- 172.21.15.60:0/3095587929 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8008054e20 con 0x556ed008c0a0 2024-04-25T10:15:13.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 10 client.9024 fetch_fsmap finished waiting for FSMap version 77 2024-04-25T10:15:13.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 10 client.9024 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:15:13.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 20 client.9024 populate_metadata read hostname 'smithi060' 2024-04-25T10:15:13.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 --2- 172.21.15.60:0/3095587929 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x556ed00e9170 0x556ed00e9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 4 client.9024 mds_command: new command op to 8631 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:15:13.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 -- 172.21.15.60:0/3095587929 --> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556ecfee6530 con 0x556ed00e9170 2024-04-25T10:15:13.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80167fc640 1 --2- 172.21.15.60:0/3095587929 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x556ed00e9170 0x556ed00e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80167fc640 1 --2- 172.21.15.60:0/3095587929 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x556ed00e9170 0x556ed00e9580 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=318eb5fdda624b8e server_cookie=4722b462525799e4 in_seq=0 out_seq=0 2024-04-25T10:15:13.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f7ff5ffb640 10 client.9024 ms_handle_connect on v2:172.21.15.60:6834/2002755047 2024-04-25T10:15:13.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f7ff5ffb640 1 -- 172.21.15.60:0/3095587929 <== mds.0 v2:172.21.15.60:6834/2002755047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556ecfee6530 con 0x556ed00e9170 2024-04-25T10:15:13.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f7ff5ffb640 10 client.9024 handle_command_reply: tid=0 2024-04-25T10:15:13.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 client.9024 shutdown 2024-04-25T10:15:13.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 -- 172.21.15.60:0/3095587929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 msgr2=0x556ed0136520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 --2- 172.21.15.60:0/3095587929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed0136520 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f800805c010 tx=0x7f8008083840 comp rx=0 tx=0).stop 2024-04-25T10:15:13.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 -- 172.21.15.60:0/3095587929 shutdown_connections 2024-04-25T10:15:13.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.950+0000 7f80205d8000 1 --2- 172.21.15.60:0/3095587929 >> [v2:172.21.15.60:6834/2002755047,v1:172.21.15.60:6835/2002755047] conn(0x556ed00e9170 0x556ed00e9580 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f80205d8000 1 --2- 172.21.15.60:0/3095587929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ed00d62f0 0x556ed00e87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f7ff5ffb640 0 client.9024 ms_handle_reset on v2:172.21.15.60:6834/2002755047 2024-04-25T10:15:13.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f80205d8000 1 --2- 172.21.15.60:0/3095587929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556ed00d59d0 0x556ed0136a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f80205d8000 1 --2- 172.21.15.60:0/3095587929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ed008c0a0 0x556ed0136520 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f80205d8000 1 -- 172.21.15.60:0/3095587929 >> 172.21.15.60:0/3095587929 conn(0x556ed0088020 msgr2=0x556ed00d4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f80205d8000 1 -- 172.21.15.60:0/3095587929 shutdown_connections 2024-04-25T10:15:13.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f80205d8000 1 -- 172.21.15.60:0/3095587929 wait complete. 2024-04-25T10:15:13.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f80205d8000 20 client.9024 trim_cache size 0 max 16384 2024-04-25T10:15:13.963 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:15:13.963 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T10:15:13.963 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T10:15:13.963 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:15:13.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f801d46b640 1 -- 172.21.15.60:0/3066535291 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ff8066f60 msgr2=0x7f7ff8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f801d46b640 1 --2- 172.21.15.60:0/3066535291 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ff8066f60 0x7f7ff8069420 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f80000ae000 tx=0x7f80000a7070 comp rx=0 tx=0).stop 2024-04-25T10:15:13.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f801d46b640 1 -- 172.21.15.60:0/3066535291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80180ce7e0 msgr2=0x7f801815a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f801d46b640 1 --2- 172.21.15.60:0/3066535291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80180ce7e0 0x7f801815a320 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f800800e960 tx=0x7f800800ee20 comp rx=0 tx=0).stop 2024-04-25T10:15:13.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f801d46b640 1 -- 172.21.15.60:0/3066535291 shutdown_connections 2024-04-25T10:15:13.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f801d46b640 1 --2- 172.21.15.60:0/3066535291 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ff8066f60 0x7f7ff8069420 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f801d46b640 1 --2- 172.21.15.60:0/3066535291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f801815e0d0 0x7f80181fab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f801d46b640 1 --2- 172.21.15.60:0/3066535291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801815b6f0 0x7f80181fa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f801d46b640 1 --2- 172.21.15.60:0/3066535291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80180ce7e0 0x7f801815a320 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f801d46b640 1 -- 172.21.15.60:0/3066535291 >> 172.21.15.60:0/3066535291 conn(0x7f8018072340 msgr2=0x7f80180cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f801d46b640 1 -- 172.21.15.60:0/3066535291 shutdown_connections 2024-04-25T10:15:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:13.954+0000 7f801d46b640 1 -- 172.21.15.60:0/3066535291 wait complete. 2024-04-25T10:15:13.975 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:15:14.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:15:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.194+0000 7ff52d837640 1 Processor -- start 2024-04-25T10:15:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.194+0000 7ff52d837640 1 -- start start 2024-04-25T10:15:14.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.194+0000 7ff52d837640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff5280cee00 0x7ff52815d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.194+0000 7ff52d837640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff52815da40 0x7ff5280d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.194+0000 7ff52d837640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff528162070 con 0x7ff5280cea20 2024-04-25T10:15:14.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.194+0000 7ff52d837640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff528162350 con 0x7ff5280cee00 2024-04-25T10:15:14.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.194+0000 7ff52d837640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5280d1cb0 con 0x7ff52815da40 2024-04-25T10:15:14.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.194+0000 7ff526ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff5280cea20 0x7ff528161640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43424/0 (socket says 172.21.15.60:43424) 2024-04-25T10:15:14.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.194+0000 7ff5267fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff5280cee00 0x7ff52815d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.194+0000 7ff526ffd640 1 -- 172.21.15.60:0/2390793385 learned_addr learned my addr 172.21.15.60:0/2390793385 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:15:14.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.194+0000 7ff5277fe640 1 --2- 172.21.15.60:0/2390793385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff52815da40 0x7ff5280d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff5267fc640 1 -- 172.21.15.60:0/2390793385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff52815da40 msgr2=0x7ff5280d24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff5267fc640 1 --2- 172.21.15.60:0/2390793385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff52815da40 0x7ff5280d24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff5267fc640 1 -- 172.21.15.60:0/2390793385 >> v1:172.21.15.60:6789/0 conn(0x7ff5280cea20 legacy=0x7ff528161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff525ffb640 1 -- 172.21.15.60:0/2390793385 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654584918 0 0) 0x7ff528162070 con 0x7ff5280cea20 2024-04-25T10:15:14.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff5267fc640 1 -- 172.21.15.60:0/2390793385 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5280d1f90 con 0x7ff5280cee00 2024-04-25T10:15:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff5277fe640 1 --2- 172.21.15.60:0/2390793385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff52815da40 0x7ff5280d24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff5267fc640 1 --2- 172.21.15.60:0/2390793385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5280cee00 0x7ff52815d500 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0097c0 tx=0x7ff51c01cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c051cc297321549c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff525ffb640 1 -- 172.21.15.60:0/2390793385 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c0aa620 con 0x7ff5280cee00 2024-04-25T10:15:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff525ffb640 1 -- 172.21.15.60:0/2390793385 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51c0af070 con 0x7ff5280cee00 2024-04-25T10:15:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff525ffb640 1 -- 172.21.15.60:0/2390793385 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c0aa9e0 con 0x7ff5280cee00 2024-04-25T10:15:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff52d837640 1 -- 172.21.15.60:0/2390793385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5280cee00 msgr2=0x7ff52815d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff52d837640 1 --2- 172.21.15.60:0/2390793385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5280cee00 0x7ff52815d500 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0097c0 tx=0x7ff51c01cec0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff52d837640 1 -- 172.21.15.60:0/2390793385 shutdown_connections 2024-04-25T10:15:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff52d837640 1 --2- 172.21.15.60:0/2390793385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff52815da40 0x7ff5280d24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff52d837640 1 --2- 172.21.15.60:0/2390793385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5280cee00 0x7ff52815d500 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff52d837640 1 -- 172.21.15.60:0/2390793385 >> 172.21.15.60:0/2390793385 conn(0x7ff528072340 msgr2=0x7ff528072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff52d837640 1 -- 172.21.15.60:0/2390793385 shutdown_connections 2024-04-25T10:15:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.198+0000 7ff52d837640 1 -- 172.21.15.60:0/2390793385 wait complete. 2024-04-25T10:15:14.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff52d837640 1 Processor -- start 2024-04-25T10:15:14.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff52d837640 1 -- start start 2024-04-25T10:15:14.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff52d837640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5280cea20 0x7ff52815e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff52d837640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280cee00 0x7ff528158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff52d837640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff52815da40 0x7ff528153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff52d837640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5280d3380 con 0x7ff5280cea20 2024-04-25T10:15:14.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff52d837640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5280d3200 con 0x7ff5280cee00 2024-04-25T10:15:14.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff52d837640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5280d3500 con 0x7ff52815da40 2024-04-25T10:15:14.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff526ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5280cea20 0x7ff52815e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff526ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5280cea20 0x7ff52815e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58102/0 (socket says 172.21.15.60:58102) 2024-04-25T10:15:14.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff526ffd640 1 -- 172.21.15.60:0/947141626 learned_addr learned my addr 172.21.15.60:0/947141626 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:14.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff5277fe640 1 --2- 172.21.15.60:0/947141626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff52815da40 0x7ff528153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff5267fc640 1 --2- 172.21.15.60:0/947141626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280cee00 0x7ff528158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff526ffd640 1 -- 172.21.15.60:0/947141626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff52815da40 msgr2=0x7ff528153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff526ffd640 1 --2- 172.21.15.60:0/947141626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff52815da40 0x7ff528153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff526ffd640 1 -- 172.21.15.60:0/947141626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280cee00 msgr2=0x7ff528158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff526ffd640 1 --2- 172.21.15.60:0/947141626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280cee00 0x7ff528158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff526ffd640 1 -- 172.21.15.60:0/947141626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52815eb50 con 0x7ff5280cea20 2024-04-25T10:15:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff526ffd640 1 --2- 172.21.15.60:0/947141626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5280cea20 0x7ff52815e080 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7ff518001570 tx=0x7ff51800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff50ffff640 1 -- 172.21.15.60:0/947141626 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff518018070 con 0x7ff5280cea20 2024-04-25T10:15:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff50ffff640 1 -- 172.21.15.60:0/947141626 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51800a640 con 0x7ff5280cea20 2024-04-25T10:15:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff52d837640 1 -- 172.21.15.60:0/947141626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52815ee30 con 0x7ff5280cea20 2024-04-25T10:15:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff50ffff640 1 -- 172.21.15.60:0/947141626 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51800aa30 con 0x7ff5280cea20 2024-04-25T10:15:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff52d837640 1 -- 172.21.15.60:0/947141626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52815f3f0 con 0x7ff5280cea20 2024-04-25T10:15:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff52d837640 1 -- 172.21.15.60:0/947141626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52815f8e0 con 0x7ff5280cea20 2024-04-25T10:15:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.202+0000 7ff50ffff640 1 -- 172.21.15.60:0/947141626 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff51801b070 con 0x7ff5280cea20 2024-04-25T10:15:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.206+0000 7ff50ffff640 1 --2- 172.21.15.60:0/947141626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff514066f60 0x7ff514069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.206+0000 7ff50ffff640 1 -- 172.21.15.60:0/947141626 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff518084760 con 0x7ff5280cea20 2024-04-25T10:15:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.206+0000 7ff5267fc640 1 --2- 172.21.15.60:0/947141626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff514066f60 0x7ff514069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.206+0000 7ff5267fc640 1 --2- 172.21.15.60:0/947141626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff514066f60 0x7ff514069420 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0ae000 tx=0x7ff51c004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.210+0000 7ff50ffff640 1 -- 172.21.15.60:0/947141626 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff518055760 con 0x7ff5280cea20 2024-04-25T10:15:14.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.402+0000 7ff52d837640 1 -- 172.21.15.60:0/947141626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff52815e620 con 0x7ff5280cea20 2024-04-25T10:15:14.410 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:15:14.406+0000 7f3a792d6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:15:14.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.406+0000 7ff50ffff640 1 -- 172.21.15.60:0/947141626 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff52815e620 con 0x7ff5280cea20 2024-04-25T10:15:14.412 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:15:14.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.410+0000 7ff52d837640 1 -- 172.21.15.60:0/947141626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff514066f60 msgr2=0x7ff514069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.410+0000 7ff52d837640 1 --2- 172.21.15.60:0/947141626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff514066f60 0x7ff514069420 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0ae000 tx=0x7ff51c004740 comp rx=0 tx=0).stop 2024-04-25T10:15:14.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.410+0000 7ff52d837640 1 -- 172.21.15.60:0/947141626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5280cea20 msgr2=0x7ff52815e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.410+0000 7ff52d837640 1 --2- 172.21.15.60:0/947141626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5280cea20 0x7ff52815e080 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7ff518001570 tx=0x7ff51800cd20 comp rx=0 tx=0).stop 2024-04-25T10:15:14.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.410+0000 7ff52d837640 1 -- 172.21.15.60:0/947141626 shutdown_connections 2024-04-25T10:15:14.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.410+0000 7ff52d837640 1 --2- 172.21.15.60:0/947141626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff514066f60 0x7ff514069420 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.410+0000 7ff52d837640 1 --2- 172.21.15.60:0/947141626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff52815da40 0x7ff528153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.410+0000 7ff52d837640 1 --2- 172.21.15.60:0/947141626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5280cee00 0x7ff528158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.410+0000 7ff52d837640 1 --2- 172.21.15.60:0/947141626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5280cea20 0x7ff52815e080 secure :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7ff518001570 tx=0x7ff51800cd20 comp rx=0 tx=0).stop 2024-04-25T10:15:14.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.410+0000 7ff52d837640 1 -- 172.21.15.60:0/947141626 >> 172.21.15.60:0/947141626 conn(0x7ff528072340 msgr2=0x7ff5280b82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.410+0000 7ff52d837640 1 -- 172.21.15.60:0/947141626 shutdown_connections 2024-04-25T10:15:14.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.410+0000 7ff52d837640 1 -- 172.21.15.60:0/947141626 wait complete. 2024-04-25T10:15:14.423 INFO:tasks.ceph.mds.a.smithi060.stderr: -6092> 2024-04-25T10:14:22.763+0000 7fbb92112640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:15:14.433 DEBUG:teuthology.orchestra.run.smithi060:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-04-25T10:15:14.465 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T10:15:14.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 1 -- start start 2024-04-25T10:15:14.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 1 Processor -- start 2024-04-25T10:15:14.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639de5c1800 0x5639de77a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5639de5c1400 0x5639de77ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639de5c1000 0x5639de77b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639de656180 con 0x5639de5c1000 2024-04-25T10:15:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639de656300 con 0x5639de5c1800 2024-04-25T10:15:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639de656480 con 0x5639de5c1400 2024-04-25T10:15:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f6022161640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639de5c1000 0x5639de77b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f6022161640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639de5c1000 0x5639de77b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58132/0 (socket says 172.21.15.60:58132) 2024-04-25T10:15:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f6022161640 1 -- 172.21.15.60:0/2053147675 learned_addr learned my addr 172.21.15.60:0/2053147675 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f6021960640 1 --2- 172.21.15.60:0/2053147675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639de5c1800 0x5639de77a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f602115f640 1 --2- 172.21.15.60:0/2053147675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5639de5c1400 0x5639de77ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f6022161640 1 -- 172.21.15.60:0/2053147675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5639de5c1400 msgr2=0x5639de77ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f6022161640 1 --2- 172.21.15.60:0/2053147675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5639de5c1400 0x5639de77ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f6022161640 1 -- 172.21.15.60:0/2053147675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639de5c1800 msgr2=0x5639de77a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f6022161640 1 --2- 172.21.15.60:0/2053147675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639de5c1800 0x5639de77a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f6022161640 1 -- 172.21.15.60:0/2053147675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639de6b4960 con 0x5639de5c1000 2024-04-25T10:15:14.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f602115f640 1 --2- 172.21.15.60:0/2053147675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5639de5c1400 0x5639de77ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:14.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f6022161640 1 --2- 172.21.15.60:0/2053147675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639de5c1000 0x5639de77b080 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x5639df42a9c0 tx=0x5639df390000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26740caf800a23f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f601f15b640 1 -- 172.21.15.60:0/2053147675 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5639de5f69c0 con 0x5639de5c1000 2024-04-25T10:15:14.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f601f15b640 1 -- 172.21.15.60:0/2053147675 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5639de60ae00 con 0x5639de5c1000 2024-04-25T10:15:14.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f601f15b640 1 -- 172.21.15.60:0/2053147675 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5639de5f64e0 con 0x5639de5c1000 2024-04-25T10:15:14.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 1 -- 172.21.15.60:0/2053147675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639de6b4d20 con 0x5639de5c1000 2024-04-25T10:15:14.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 1 -- 172.21.15.60:0/2053147675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5639de6b4f00 con 0x5639de5c1000 2024-04-25T10:15:14.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 4 waiting for MDS map... 2024-04-25T10:15:14.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f601f15b640 1 -- 172.21.15.60:0/2053147675 <== mon.0 v2:172.21.15.60:3300/0 4 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x5639df4d2200 con 0x5639de5c1000 2024-04-25T10:15:14.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f601f15b640 1 -- 172.21.15.60:0/2053147675 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 78) v1 ==== 2723+0+0 (secure 0 0 0) 0x5639de617180 con 0x5639de5c1000 2024-04-25T10:15:14.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 4 Got MDS map 78 2024-04-25T10:15:14.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 10 main: JournalTool::main 2024-04-25T10:15:14.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:15:14.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 1 Processor -- start 2024-04-25T10:15:14.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 1 -- start start 2024-04-25T10:15:14.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639df39d800 0x5639de77b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.462+0000 7f60231a1400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639df39dc00 0x5639de77bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5639df532000 0x5639de77c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639de656600 con 0x5639df39dc00 2024-04-25T10:15:14.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639de656780 con 0x5639df39d800 2024-04-25T10:15:14.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639de656900 con 0x5639df532000 2024-04-25T10:15:14.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f6021960640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639df39dc00 0x5639de77bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f6021960640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639df39dc00 0x5639de77bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58144/0 (socket says 172.21.15.60:58144) 2024-04-25T10:15:14.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f6021960640 1 -- 172.21.15.60:0/2514058736 learned_addr learned my addr 172.21.15.60:0/2514058736 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:14.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f602115f640 1 --2- 172.21.15.60:0/2514058736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5639df532000 0x5639de77c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f6021960640 1 -- 172.21.15.60:0/2514058736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5639df532000 msgr2=0x5639de77c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f6021960640 1 --2- 172.21.15.60:0/2514058736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5639df532000 0x5639de77c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f6021960640 1 -- 172.21.15.60:0/2514058736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639df39d800 msgr2=0x5639de77b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f6021960640 1 --2- 172.21.15.60:0/2514058736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639df39d800 0x5639de77b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f6021960640 1 -- 172.21.15.60:0/2514058736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639de6b41e0 con 0x5639df39dc00 2024-04-25T10:15:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f602115f640 1 --2- 172.21.15.60:0/2514058736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5639df532000 0x5639de77c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f6021960640 1 --2- 172.21.15.60:0/2514058736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639df39dc00 0x5639de77bb80 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x5639df52a7b0 tx=0x5639df391e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0173dcaab5a3408 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f601d157640 1 -- 172.21.15.60:0/2514058736 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5639de5f69c0 con 0x5639df39dc00 2024-04-25T10:15:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f601d157640 1 -- 172.21.15.60:0/2514058736 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5639de60aa80 con 0x5639df39dc00 2024-04-25T10:15:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f601d157640 1 -- 172.21.15.60:0/2514058736 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5639de5f64e0 con 0x5639df39dc00 2024-04-25T10:15:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 -- 172.21.15.60:0/2514058736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639df39dc00 msgr2=0x5639de77bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 --2- 172.21.15.60:0/2514058736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639df39dc00 0x5639de77bb80 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x5639df52a7b0 tx=0x5639df391e00 comp rx=0 tx=0).stop 2024-04-25T10:15:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 -- 172.21.15.60:0/2514058736 shutdown_connections 2024-04-25T10:15:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 --2- 172.21.15.60:0/2514058736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5639df532000 0x5639de77c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 --2- 172.21.15.60:0/2514058736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639df39dc00 0x5639de77bb80 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 --2- 172.21.15.60:0/2514058736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639df39d800 0x5639de77b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 -- 172.21.15.60:0/2514058736 >> 172.21.15.60:0/2514058736 conn(0x5639de5c0c00 msgr2=0x5639de5ef800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 -- 172.21.15.60:0/2514058736 shutdown_connections 2024-04-25T10:15:14.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 -- 172.21.15.60:0/2514058736 wait complete. 2024-04-25T10:15:14.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 Processor -- start 2024-04-25T10:15:14.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 -- start start 2024-04-25T10:15:14.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639df39dc00 0x5639de77d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5639df39d800 0x5639de77cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639df533800 0x5639de77c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639de657c80 con 0x5639df533800 2024-04-25T10:15:14.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639de657b00 con 0x5639df39dc00 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639de657980 con 0x5639df39d800 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f602115f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639df533800 0x5639de77c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f602115f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639df533800 0x5639de77c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58160/0 (socket says 172.21.15.60:58160) 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f602115f640 1 -- 172.21.15.60:0/1977530595 learned_addr learned my addr 172.21.15.60:0/1977530595 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f602115f640 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639df39dc00 0x5639de77d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f6021960640 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5639df39d800 0x5639de77cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f602115f640 1 -- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5639df39d800 msgr2=0x5639de77cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f602115f640 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5639df39d800 0x5639de77cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f602115f640 1 -- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639df39dc00 msgr2=0x5639de77d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f602115f640 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639df39dc00 0x5639de77d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f602115f640 1 -- 172.21.15.60:0/1977530595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639de6b5a40 con 0x5639df533800 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f602115f640 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639df533800 0x5639de77c680 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x5639df52b410 tx=0x5639de8ec060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f601b954640 1 -- 172.21.15.60:0/1977530595 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5639de5f69c0 con 0x5639df533800 2024-04-25T10:15:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f601b954640 1 -- 172.21.15.60:0/1977530595 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5639de60a540 con 0x5639df533800 2024-04-25T10:15:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f601b954640 1 -- 172.21.15.60:0/1977530595 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5639de5f64e0 con 0x5639df533800 2024-04-25T10:15:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 -- 172.21.15.60:0/1977530595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5639de6b5c20 con 0x5639df533800 2024-04-25T10:15:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 1 -- 172.21.15.60:0/1977530595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639de8ee000 con 0x5639df533800 2024-04-25T10:15:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.466+0000 7f60231a1400 4 main: JournalTool: resolving pool 14 2024-04-25T10:15:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.470+0000 7f601b954640 1 -- 172.21.15.60:0/1977530595 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x5639df52cc00 con 0x5639df533800 2024-04-25T10:15:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.470+0000 7f601b954640 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5639df541400 0x5639de77d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.470+0000 7f601b954640 1 -- 172.21.15.60:0/1977530595 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x5639de8f2600 con 0x5639df533800 2024-04-25T10:15:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.470+0000 7f60231a1400 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:15:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.470+0000 7f60231a1400 4 main: Executing for rank 0 2024-04-25T10:15:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.470+0000 7f60231a1400 4 Filtering by inode '1099511627777' 2024-04-25T10:15:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.470+0000 7f60231a1400 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5639df533400 0x5639de77c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.470+0000 7f60231a1400 1 -- 172.21.15.60:0/1977530595 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v9 -- 0x5639df533000 con 0x5639df533400 2024-04-25T10:15:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.470+0000 7f6021960640 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5639df541400 0x5639de77d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.470+0000 7f602115f640 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5639df533400 0x5639de77c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.470+0000 7f6021960640 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5639df541400 0x5639de77d700 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x5639df52a7b0 tx=0x5639df390120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.470+0000 7f602115f640 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5639df533400 0x5639de77c100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.474+0000 7f602115f640 1 -- 172.21.15.60:0/1977530595 <== osd.7 v2:172.21.15.100:6816/2771485684 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x5639de73c000 con 0x5639df533400 2024-04-25T10:15:14.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.474+0000 7f60231a1400 4 Pointer 400.00000000 is readable 2024-04-25T10:15:14.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.474+0000 7f60231a1400 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-25T10:15:14.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.474+0000 7f60231a1400 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x5639df532c00 0x5639de77bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.474+0000 7f60231a1400 1 -- 172.21.15.60:0/1977530595 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v9 -- 0x5639df532800 con 0x5639df532c00 2024-04-25T10:15:14.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.474+0000 7f6022161640 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x5639df532c00 0x5639de77bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f6022161640 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x5639df532c00 0x5639de77bb80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f6022161640 1 -- 172.21.15.60:0/1977530595 <== osd.4 v2:172.21.15.100:6808/3965541654 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv30 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x5639de73c500 con 0x5639df532c00 2024-04-25T10:15:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 10 Header 0x400000 0x405f0e 0x40628a 2024-04-25T10:15:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 10 Starting journal scan from offset 0x405f0e 2024-04-25T10:15:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 -- 172.21.15.60:0/1977530595 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v9 -- 0x5639df532400 con 0x5639df532c00 2024-04-25T10:15:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f6022161640 1 -- 172.21.15.60:0/1977530595 <== osd.4 v2:172.21.15.100:6808/3965541654 2 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv27 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x5639de73c500 con 0x5639df532c00 2024-04-25T10:15:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-25T10:15:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 10 Parsing data, 0x37c bytes available 2024-04-25T10:15:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 10 Attempting decode at 0x405f0e 2024-04-25T10:15:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 10 Consumed 0x37c bytes 2024-04-25T10:15:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 10 Valid entry at 0x405f0e 2024-04-25T10:15:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5639df532000 0x5639de77b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 -- 172.21.15.60:0/1977530595 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v9 -- 0x5639df541800 con 0x5639df532000 2024-04-25T10:15:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f6021960640 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5639df532000 0x5639de77b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f6021960640 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5639df532000 0x5639de77b600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f6021960640 1 -- 172.21.15.60:0/1977530595 <== osd.1 v2:172.21.15.60:6824/3313552458 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5639de73c780 con 0x5639df532000 2024-04-25T10:15:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 4 Reached end of journal objects 2024-04-25T10:15:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 4 Scanned objects, 0 missing, 1 valid 2024-04-25T10:15:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 4 Events scanned, 0 gaps 2024-04-25T10:15:14.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 4 Found 1 valid events 2024-04-25T10:15:14.487 INFO:teuthology.orchestra.run.smithi060.stdout:Events by type: 2024-04-25T10:15:14.487 INFO:teuthology.orchestra.run.smithi060.stdout:Errors: 0 2024-04-25T10:15:14.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 4 Selected 0 events events for processing 2024-04-25T10:15:14.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 4 0 events 2024-04-25T10:15:14.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 -- 172.21.15.60:0/1977530595 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5639df532000 msgr2=0x5639de77b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5639df532000 0x5639de77b600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 -- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x5639df532c00 msgr2=0x5639de77bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x5639df532c00 0x5639de77bb80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 -- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5639df533400 msgr2=0x5639de77c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5639df533400 0x5639de77c100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f6021960640 1 -- 172.21.15.60:0/1977530595 reap_dead start 2024-04-25T10:15:14.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 -- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5639df541400 msgr2=0x5639de77d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5639df541400 0x5639de77d700 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x5639df52a7b0 tx=0x5639df390120 comp rx=0 tx=0).stop 2024-04-25T10:15:14.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 -- 172.21.15.60:0/1977530595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639df533800 msgr2=0x5639de77c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639df533800 0x5639de77c680 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x5639df52b410 tx=0x5639de8ec060 comp rx=0 tx=0).stop 2024-04-25T10:15:14.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 -- 172.21.15.60:0/1977530595 shutdown_connections 2024-04-25T10:15:14.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5639df541400 0x5639de77d700 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 --2- 172.21.15.60:0/1977530595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639df533800 0x5639de77c680 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 -- 172.21.15.60:0/1977530595 >> 172.21.15.60:0/1977530595 conn(0x5639de5c0c00 msgr2=0x5639de5ef800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 -- 172.21.15.60:0/1977530595 shutdown_connections 2024-04-25T10:15:14.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.478+0000 7f60231a1400 1 -- 172.21.15.60:0/1977530595 wait complete. 2024-04-25T10:15:14.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.482+0000 7f60231a1400 1 -- 172.21.15.60:0/2053147675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639de5c1000 msgr2=0x5639de77b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.482+0000 7f60231a1400 1 --2- 172.21.15.60:0/2053147675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639de5c1000 0x5639de77b080 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x5639df42a9c0 tx=0x5639df390000 comp rx=0 tx=0).stop 2024-04-25T10:15:14.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.482+0000 7f60231a1400 1 -- 172.21.15.60:0/2053147675 shutdown_connections 2024-04-25T10:15:14.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.482+0000 7f60231a1400 1 --2- 172.21.15.60:0/2053147675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639de5c1000 0x5639de77b080 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.482+0000 7f60231a1400 1 --2- 172.21.15.60:0/2053147675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5639de5c1400 0x5639de77ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.482+0000 7f60231a1400 1 --2- 172.21.15.60:0/2053147675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5639de5c1800 0x5639de77a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.482+0000 7f60231a1400 1 -- 172.21.15.60:0/2053147675 >> 172.21.15.60:0/2053147675 conn(0x5639de5c1c00 msgr2=0x5639de5ef000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.482+0000 7f60231a1400 1 -- 172.21.15.60:0/2053147675 shutdown_connections 2024-04-25T10:15:14.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.482+0000 7f60231a1400 1 -- 172.21.15.60:0/2053147675 wait complete. 2024-04-25T10:15:14.491 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.058115, result: Events by type: Errors: 0 2024-04-25T10:15:14.492 DEBUG:teuthology.orchestra.run.smithi060:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-04-25T10:15:14.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 -- start start 2024-04-25T10:15:14.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 Processor -- start 2024-04-25T10:15:14.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6eef9f800 0x55a6ef158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a6eef9f400 0x55a6ef158b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6eef9f000 0x55a6ef159080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6efe9a180 con 0x55a6eef9f000 2024-04-25T10:15:14.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6efe9a300 con 0x55a6eef9f800 2024-04-25T10:15:14.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6efe9a480 con 0x55a6eef9f400 2024-04-25T10:15:14.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa81c0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6eef9f000 0x55a6ef159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa81c0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6eef9f000 0x55a6ef159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58186/0 (socket says 172.21.15.60:58186) 2024-04-25T10:15:14.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa81c0640 1 -- 172.21.15.60:0/3036616851 learned_addr learned my addr 172.21.15.60:0/3036616851 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:14.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa71be640 1 --2- 172.21.15.60:0/3036616851 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a6eef9f400 0x55a6ef158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa79bf640 1 --2- 172.21.15.60:0/3036616851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6eef9f800 0x55a6ef158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa81c0640 1 -- 172.21.15.60:0/3036616851 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a6eef9f400 msgr2=0x55a6ef158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa81c0640 1 --2- 172.21.15.60:0/3036616851 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a6eef9f400 0x55a6ef158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa81c0640 1 -- 172.21.15.60:0/3036616851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6eef9f800 msgr2=0x55a6ef158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa81c0640 1 --2- 172.21.15.60:0/3036616851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6eef9f800 0x55a6ef158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa81c0640 1 -- 172.21.15.60:0/3036616851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6ef036b40 con 0x55a6eef9f000 2024-04-25T10:15:14.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa79bf640 1 --2- 172.21.15.60:0/3036616851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6eef9f800 0x55a6ef158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:14.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa81c0640 1 --2- 172.21.15.60:0/3036616851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6eef9f000 0x55a6ef159080 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x55a6efdfe9c0 tx=0x55a6ef092000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455a71c7545f0642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa51ba640 1 -- 172.21.15.60:0/3036616851 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a6eefd49c0 con 0x55a6eef9f000 2024-04-25T10:15:14.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa51ba640 1 -- 172.21.15.60:0/3036616851 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a6eefe8e00 con 0x55a6eef9f000 2024-04-25T10:15:14.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa51ba640 1 -- 172.21.15.60:0/3036616851 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a6eefd44e0 con 0x55a6eef9f000 2024-04-25T10:15:14.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 -- 172.21.15.60:0/3036616851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6ef036d20 con 0x55a6eef9f000 2024-04-25T10:15:14.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 -- 172.21.15.60:0/3036616851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55a6ef036f00 con 0x55a6eef9f000 2024-04-25T10:15:14.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 4 waiting for MDS map... 2024-04-25T10:15:14.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa51ba640 1 -- 172.21.15.60:0/3036616851 <== mon.0 v2:172.21.15.60:3300/0 4 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x55a6efeb0200 con 0x55a6eef9f000 2024-04-25T10:15:14.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa51ba640 1 -- 172.21.15.60:0/3036616851 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 78) v1 ==== 2723+0+0 (secure 0 0 0) 0x55a6eeff5180 con 0x55a6eef9f000 2024-04-25T10:15:14.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 4 Got MDS map 78 2024-04-25T10:15:14.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 10 main: JournalTool::main 2024-04-25T10:15:14.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:15:14.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 Processor -- start 2024-04-25T10:15:14.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 -- start start 2024-04-25T10:15:14.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6efd71800 0x55a6ef159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6efd71c00 0x55a6ef159b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a6eff0a000 0x55a6ef15a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6efe9a600 con 0x55a6efd71800 2024-04-25T10:15:14.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6efe9a780 con 0x55a6efd71c00 2024-04-25T10:15:14.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa9200400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6efe9a900 con 0x55a6eff0a000 2024-04-25T10:15:14.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa71be640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6efd71800 0x55a6ef159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa71be640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6efd71800 0x55a6ef159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58188/0 (socket says 172.21.15.60:58188) 2024-04-25T10:15:14.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa71be640 1 -- 172.21.15.60:0/3851479606 learned_addr learned my addr 172.21.15.60:0/3851479606 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa79bf640 1 --2- 172.21.15.60:0/3851479606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6efd71c00 0x55a6ef159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa71be640 1 --2- 172.21.15.60:0/3851479606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a6eff0a000 0x55a6ef15a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa71be640 1 -- 172.21.15.60:0/3851479606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a6eff0a000 msgr2=0x55a6ef15a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa71be640 1 --2- 172.21.15.60:0/3851479606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a6eff0a000 0x55a6ef15a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa71be640 1 -- 172.21.15.60:0/3851479606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6efd71c00 msgr2=0x55a6ef159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa71be640 1 --2- 172.21.15.60:0/3851479606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6efd71c00 0x55a6ef159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa71be640 1 -- 172.21.15.60:0/3851479606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6ef0361e0 con 0x55a6efd71800 2024-04-25T10:15:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.562+0000 7fefa79bf640 1 --2- 172.21.15.60:0/3851479606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6efd71c00 0x55a6ef159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa71be640 1 --2- 172.21.15.60:0/3851479606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6efd71800 0x55a6ef159600 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x55a6eff026f0 tx=0x55a6ef093e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0bfeb89ad052aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa31b6640 1 -- 172.21.15.60:0/3851479606 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a6eefd49c0 con 0x55a6efd71800 2024-04-25T10:15:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa31b6640 1 -- 172.21.15.60:0/3851479606 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a6eefe8a80 con 0x55a6efd71800 2024-04-25T10:15:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa31b6640 1 -- 172.21.15.60:0/3851479606 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a6eefd44e0 con 0x55a6efd71800 2024-04-25T10:15:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 -- 172.21.15.60:0/3851479606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6efd71800 msgr2=0x55a6ef159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 --2- 172.21.15.60:0/3851479606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6efd71800 0x55a6ef159600 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x55a6eff026f0 tx=0x55a6ef093e00 comp rx=0 tx=0).stop 2024-04-25T10:15:14.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 -- 172.21.15.60:0/3851479606 shutdown_connections 2024-04-25T10:15:14.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 --2- 172.21.15.60:0/3851479606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a6eff0a000 0x55a6ef15a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 --2- 172.21.15.60:0/3851479606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6efd71c00 0x55a6ef159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 --2- 172.21.15.60:0/3851479606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6efd71800 0x55a6ef159600 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 -- 172.21.15.60:0/3851479606 >> 172.21.15.60:0/3851479606 conn(0x55a6eef9ec00 msgr2=0x55a6eefcd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 -- 172.21.15.60:0/3851479606 shutdown_connections 2024-04-25T10:15:14.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 -- 172.21.15.60:0/3851479606 wait complete. 2024-04-25T10:15:14.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 Processor -- start 2024-04-25T10:15:14.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 -- start start 2024-04-25T10:15:14.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6efd71c00 0x55a6ef15b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6efd71800 0x55a6ef15ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a6eff0b800 0x55a6ef15a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6efe9bc80 con 0x55a6efd71800 2024-04-25T10:15:14.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6efe9bb00 con 0x55a6efd71c00 2024-04-25T10:15:14.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6efe9b980 con 0x55a6eff0b800 2024-04-25T10:15:14.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa79bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6efd71800 0x55a6ef15ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa79bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6efd71800 0x55a6ef15ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58196/0 (socket says 172.21.15.60:58196) 2024-04-25T10:15:14.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa79bf640 1 -- 172.21.15.60:0/756802588 learned_addr learned my addr 172.21.15.60:0/756802588 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:14.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa71be640 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6efd71c00 0x55a6ef15b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa79bf640 1 -- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a6eff0b800 msgr2=0x55a6ef15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa79bf640 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a6eff0b800 0x55a6ef15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa79bf640 1 -- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6efd71c00 msgr2=0x55a6ef15b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa79bf640 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6efd71c00 0x55a6ef15b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa79bf640 1 -- 172.21.15.60:0/756802588 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6ef037a40 con 0x55a6efd71800 2024-04-25T10:15:14.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa79bf640 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6efd71800 0x55a6ef15ac00 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x55a6eff033e0 tx=0x55a6ef2cc000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa19b3640 1 -- 172.21.15.60:0/756802588 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a6eefd49c0 con 0x55a6efd71800 2024-04-25T10:15:14.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa19b3640 1 -- 172.21.15.60:0/756802588 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a6eefe8540 con 0x55a6efd71800 2024-04-25T10:15:14.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 -- 172.21.15.60:0/756802588 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a6ef037c20 con 0x55a6efd71800 2024-04-25T10:15:14.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa19b3640 1 -- 172.21.15.60:0/756802588 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a6eefd44e0 con 0x55a6efd71800 2024-04-25T10:15:14.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 1 -- 172.21.15.60:0/756802588 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6ef037a40 con 0x55a6efd71800 2024-04-25T10:15:14.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa9200400 4 main: JournalTool: resolving pool 14 2024-04-25T10:15:14.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.566+0000 7fefa19b3640 1 -- 172.21.15.60:0/756802588 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x55a6eff04c00 con 0x55a6efd71800 2024-04-25T10:15:14.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa19b3640 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55a6eff19400 0x55a6ef15b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa19b3640 1 -- 172.21.15.60:0/756802588 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x55a6ef2caa00 con 0x55a6efd71800 2024-04-25T10:15:14.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa9200400 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:15:14.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa9200400 4 main: Executing for rank 0 2024-04-25T10:15:14.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa9200400 4 Filtering by inode '1099511627778' 2024-04-25T10:15:14.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa9200400 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x55a6eff0b400 0x55a6ef15a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa9200400 1 -- 172.21.15.60:0/756802588 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v9 -- 0x55a6eff0b000 con 0x55a6eff0b400 2024-04-25T10:15:14.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa71be640 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55a6eff19400 0x55a6ef15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa71be640 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x55a6eff0b400 0x55a6ef15a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa71be640 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x55a6eff0b400 0x55a6ef15a100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa71be640 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55a6eff19400 0x55a6ef15b700 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x55a6eff02840 tx=0x55a6ef2cc360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa71be640 1 -- 172.21.15.60:0/756802588 <== osd.7 v2:172.21.15.100:6816/2771485684 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x55a6ef11a000 con 0x55a6eff0b400 2024-04-25T10:15:14.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa9200400 4 Pointer 400.00000000 is readable 2024-04-25T10:15:14.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa9200400 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-25T10:15:14.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa9200400 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55a6eff0ac00 0x55a6ef159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa9200400 1 -- 172.21.15.60:0/756802588 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v9 -- 0x55a6eff0a800 con 0x55a6eff0ac00 2024-04-25T10:15:14.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa81c0640 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55a6eff0ac00 0x55a6ef159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa81c0640 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55a6eff0ac00 0x55a6ef159b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa81c0640 1 -- 172.21.15.60:0/756802588 <== osd.4 v2:172.21.15.100:6808/3965541654 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv30 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x55a6ef11a500 con 0x55a6eff0ac00 2024-04-25T10:15:14.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa9200400 10 Header 0x400000 0x405f0e 0x40628a 2024-04-25T10:15:14.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa9200400 10 Starting journal scan from offset 0x405f0e 2024-04-25T10:15:14.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.570+0000 7fefa9200400 1 -- 172.21.15.60:0/756802588 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v9 -- 0x55a6eff0a400 con 0x55a6eff0ac00 2024-04-25T10:15:14.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa81c0640 1 -- 172.21.15.60:0/756802588 <== osd.4 v2:172.21.15.100:6808/3965541654 2 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv27 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x55a6ef11a500 con 0x55a6eff0ac00 2024-04-25T10:15:14.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-25T10:15:14.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 10 Parsing data, 0x37c bytes available 2024-04-25T10:15:14.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 10 Attempting decode at 0x405f0e 2024-04-25T10:15:14.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 10 Consumed 0x37c bytes 2024-04-25T10:15:14.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 10 Valid entry at 0x405f0e 2024-04-25T10:15:14.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55a6eff0a000 0x55a6ef159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 -- 172.21.15.60:0/756802588 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v9 -- 0x55a6eff19800 con 0x55a6eff0a000 2024-04-25T10:15:14.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa79bf640 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55a6eff0a000 0x55a6ef159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa79bf640 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55a6eff0a000 0x55a6ef159600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa79bf640 1 -- 172.21.15.60:0/756802588 <== osd.1 v2:172.21.15.60:6824/3313552458 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a6ef11a780 con 0x55a6eff0a000 2024-04-25T10:15:14.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 4 Reached end of journal objects 2024-04-25T10:15:14.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 4 Scanned objects, 0 missing, 1 valid 2024-04-25T10:15:14.587 INFO:teuthology.orchestra.run.smithi060.stdout:Events by type: 2024-04-25T10:15:14.587 INFO:teuthology.orchestra.run.smithi060.stdout:Errors: 0 2024-04-25T10:15:14.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 4 Events scanned, 0 gaps 2024-04-25T10:15:14.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 4 Found 1 valid events 2024-04-25T10:15:14.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 4 Selected 0 events events for processing 2024-04-25T10:15:14.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 4 0 events 2024-04-25T10:15:14.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 -- 172.21.15.60:0/756802588 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55a6eff0a000 msgr2=0x55a6ef159600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55a6eff0a000 0x55a6ef159600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 -- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55a6eff0ac00 msgr2=0x55a6ef159b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55a6eff0ac00 0x55a6ef159b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 -- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x55a6eff0b400 msgr2=0x55a6ef15a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x55a6eff0b400 0x55a6ef15a100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa79bf640 1 -- 172.21.15.60:0/756802588 reap_dead start 2024-04-25T10:15:14.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 -- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55a6eff19400 msgr2=0x55a6ef15b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55a6eff19400 0x55a6ef15b700 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x55a6eff02840 tx=0x55a6ef2cc360 comp rx=0 tx=0).stop 2024-04-25T10:15:14.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 -- 172.21.15.60:0/756802588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6efd71800 msgr2=0x55a6ef15ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6efd71800 0x55a6ef15ac00 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x55a6eff033e0 tx=0x55a6ef2cc000 comp rx=0 tx=0).stop 2024-04-25T10:15:14.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 -- 172.21.15.60:0/756802588 shutdown_connections 2024-04-25T10:15:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55a6eff19400 0x55a6ef15b700 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 --2- 172.21.15.60:0/756802588 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6efd71800 0x55a6ef15ac00 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 -- 172.21.15.60:0/756802588 >> 172.21.15.60:0/756802588 conn(0x55a6eef9ec00 msgr2=0x55a6eefcd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 -- 172.21.15.60:0/756802588 shutdown_connections 2024-04-25T10:15:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 -- 172.21.15.60:0/756802588 wait complete. 2024-04-25T10:15:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 -- 172.21.15.60:0/3036616851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6eef9f000 msgr2=0x55a6ef159080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 --2- 172.21.15.60:0/3036616851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6eef9f000 0x55a6ef159080 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x55a6efdfe9c0 tx=0x55a6ef092000 comp rx=0 tx=0).stop 2024-04-25T10:15:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 -- 172.21.15.60:0/3036616851 shutdown_connections 2024-04-25T10:15:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 --2- 172.21.15.60:0/3036616851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a6eef9f000 0x55a6ef159080 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 --2- 172.21.15.60:0/3036616851 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a6eef9f400 0x55a6ef158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 --2- 172.21.15.60:0/3036616851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6eef9f800 0x55a6ef158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 -- 172.21.15.60:0/3036616851 >> 172.21.15.60:0/3036616851 conn(0x55a6eef9fc00 msgr2=0x55a6eefcd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 -- 172.21.15.60:0/3036616851 shutdown_connections 2024-04-25T10:15:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.574+0000 7fefa9200400 1 -- 172.21.15.60:0/3036616851 wait complete. 2024-04-25T10:15:14.593 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.101289, result: Events by type: Errors: 0 2024-04-25T10:15:14.593 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-25T10:15:14.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.658+0000 7ff40cfeb080 1 Processor -- start 2024-04-25T10:15:14.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.658+0000 7ff40cfeb080 1 -- start start 2024-04-25T10:15:14.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.658+0000 7ff40cfeb080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fb5c74ec70 0x55fb5c6bc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.658+0000 7ff40cfeb080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fb5c74f050 0x55fb5c6be800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.658+0000 7ff40cfeb080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5c6a9e20 0x55fb5c6aa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.658+0000 7ff40cfeb080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c6229f0 con 0x55fb5c74ec70 2024-04-25T10:15:14.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.658+0000 7ff40cfeb080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c626950 con 0x55fb5c6a9e20 2024-04-25T10:15:14.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.658+0000 7ff40cfeb080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c6ac4c0 con 0x55fb5c74f050 2024-04-25T10:15:14.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.658+0000 7ff40ab09640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fb5c74ec70 0x55fb5c6bc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.658+0000 7ff40ab09640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fb5c74ec70 0x55fb5c6bc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58214/0 (socket says 172.21.15.60:58214) 2024-04-25T10:15:14.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.658+0000 7ff40ab09640 1 -- 172.21.15.60:0/3453984879 learned_addr learned my addr 172.21.15.60:0/3453984879 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:14.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40b30a640 1 --2- 172.21.15.60:0/3453984879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5c6a9e20 0x55fb5c6aa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40ab09640 1 -- 172.21.15.60:0/3453984879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fb5c74f050 msgr2=0x55fb5c6be800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40ab09640 1 --2- 172.21.15.60:0/3453984879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fb5c74f050 0x55fb5c6be800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40ab09640 1 -- 172.21.15.60:0/3453984879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5c6a9e20 msgr2=0x55fb5c6aa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40ab09640 1 --2- 172.21.15.60:0/3453984879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5c6a9e20 0x55fb5c6aa200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40ab09640 1 -- 172.21.15.60:0/3453984879 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb5c6bee30 con 0x55fb5c74ec70 2024-04-25T10:15:14.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40b30a640 1 --2- 172.21.15.60:0/3453984879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5c6a9e20 0x55fb5c6aa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:14.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40ab09640 1 --2- 172.21.15.60:0/3453984879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fb5c74ec70 0x55fb5c6bc2b0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ff40400f400 tx=0x7ff40400f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66f9f555ad09e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff409306640 1 -- 172.21.15.60:0/3453984879 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff404013600 con 0x55fb5c74ec70 2024-04-25T10:15:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff409306640 1 -- 172.21.15.60:0/3453984879 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff404018070 con 0x55fb5c74ec70 2024-04-25T10:15:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff409306640 1 -- 172.21.15.60:0/3453984879 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4040139e0 con 0x55fb5c74ec70 2024-04-25T10:15:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 -- 172.21.15.60:0/3453984879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fb5c74ec70 msgr2=0x55fb5c6bc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 --2- 172.21.15.60:0/3453984879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fb5c74ec70 0x55fb5c6bc2b0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ff40400f400 tx=0x7ff40400f8c0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 -- 172.21.15.60:0/3453984879 shutdown_connections 2024-04-25T10:15:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 --2- 172.21.15.60:0/3453984879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5c6a9e20 0x55fb5c6aa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 --2- 172.21.15.60:0/3453984879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fb5c74f050 0x55fb5c6be800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 --2- 172.21.15.60:0/3453984879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fb5c74ec70 0x55fb5c6bc2b0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 -- 172.21.15.60:0/3453984879 >> 172.21.15.60:0/3453984879 conn(0x55fb5c6a6e80 msgr2=0x55fb5c67ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 -- 172.21.15.60:0/3453984879 shutdown_connections 2024-04-25T10:15:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 -- 172.21.15.60:0/3453984879 wait complete. 2024-04-25T10:15:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 Processor -- start 2024-04-25T10:15:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 -- start start 2024-04-25T10:15:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fb5c6a9e20 0x55fb5c6be190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5c74ec70 0x55fb5c751e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fb5c74f050 0x55fb5c6bd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c74f5b0 con 0x55fb5c74f050 2024-04-25T10:15:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c74f850 con 0x55fb5c74ec70 2024-04-25T10:15:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb5c6c02f0 con 0x55fb5c6a9e20 2024-04-25T10:15:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40b30a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fb5c74f050 0x55fb5c6bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40b30a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fb5c74f050 0x55fb5c6bd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58222/0 (socket says 172.21.15.60:58222) 2024-04-25T10:15:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40b30a640 1 -- 172.21.15.60:0/2832240848 learned_addr learned my addr 172.21.15.60:0/2832240848 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40a308640 1 --2- 172.21.15.60:0/2832240848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5c74ec70 0x55fb5c751e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40ab09640 1 --2- 172.21.15.60:0/2832240848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fb5c6a9e20 0x55fb5c6be190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40b30a640 1 -- 172.21.15.60:0/2832240848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fb5c6a9e20 msgr2=0x55fb5c6be190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40b30a640 1 --2- 172.21.15.60:0/2832240848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fb5c6a9e20 0x55fb5c6be190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40b30a640 1 -- 172.21.15.60:0/2832240848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5c74ec70 msgr2=0x55fb5c751e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40b30a640 1 --2- 172.21.15.60:0/2832240848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5c74ec70 0x55fb5c751e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40b30a640 1 -- 172.21.15.60:0/2832240848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb5c6c0720 con 0x55fb5c74f050 2024-04-25T10:15:14.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40a308640 1 --2- 172.21.15.60:0/2832240848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb5c74ec70 0x55fb5c751e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:14.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40b30a640 1 --2- 172.21.15.60:0/2832240848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fb5c74f050 0x55fb5c6bd060 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc003040 tx=0x7ff3f4004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff3f3fff640 1 -- 172.21.15.60:0/2832240848 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc0014f0 con 0x55fb5c74f050 2024-04-25T10:15:14.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff3f3fff640 1 -- 172.21.15.60:0/2832240848 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3fc0b9070 con 0x55fb5c74f050 2024-04-25T10:15:14.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 -- 172.21.15.60:0/2832240848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fb5c6c09c0 con 0x55fb5c74f050 2024-04-25T10:15:14.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff3f3fff640 1 -- 172.21.15.60:0/2832240848 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc000f60 con 0x55fb5c74f050 2024-04-25T10:15:14.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.662+0000 7ff40cfeb080 1 -- 172.21.15.60:0/2832240848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb5c6c0f00 con 0x55fb5c74f050 2024-04-25T10:15:14.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.666+0000 7ff3f3fff640 1 -- 172.21.15.60:0/2832240848 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3fc0b3070 con 0x55fb5c74f050 2024-04-25T10:15:14.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.666+0000 7ff3f3fff640 1 --2- 172.21.15.60:0/2832240848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3f8065970 0x7ff3f8067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.666+0000 7ff3f3fff640 1 -- 172.21.15.60:0/2832240848 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff3fc11d910 con 0x55fb5c74f050 2024-04-25T10:15:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.666+0000 7ff40cfeb080 1 --2- 172.21.15.60:0/2832240848 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x55fb5c7523b0 0x55fb5c6bdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.666+0000 7ff40cfeb080 1 -- 172.21.15.60:0/2832240848 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v9 -- 0x55fb5c6be6d0 con 0x55fb5c7523b0 2024-04-25T10:15:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.666+0000 7ff40ab09640 1 --2- 172.21.15.60:0/2832240848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3f8065970 0x7ff3f8067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.666+0000 7ff40a308640 1 --2- 172.21.15.60:0/2832240848 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x55fb5c7523b0 0x55fb5c6bdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.666+0000 7ff40a308640 1 --2- 172.21.15.60:0/2832240848 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x55fb5c7523b0 0x55fb5c6bdab0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.666+0000 7ff40ab09640 1 --2- 172.21.15.60:0/2832240848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3f8065970 0x7ff3f8067e30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff404001d00 tx=0x7ff40400d000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.670+0000 7ff40a308640 1 -- 172.21.15.60:0/2832240848 <== osd.2 v2:172.21.15.60:6800/1771147646 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v98'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff4000097b0 con 0x55fb5c7523b0 2024-04-25T10:15:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.670+0000 7ff40cfeb080 1 -- 172.21.15.60:0/2832240848 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x55fb5c7523b0 msgr2=0x55fb5c6bdab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.670+0000 7ff40cfeb080 1 --2- 172.21.15.60:0/2832240848 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x55fb5c7523b0 0x55fb5c6bdab0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.670+0000 7ff40cfeb080 1 -- 172.21.15.60:0/2832240848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3f8065970 msgr2=0x7ff3f8067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.670+0000 7ff40cfeb080 1 --2- 172.21.15.60:0/2832240848 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3f8065970 0x7ff3f8067e30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff404001d00 tx=0x7ff40400d000 comp rx=0 tx=0).stop 2024-04-25T10:15:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.670+0000 7ff40cfeb080 1 -- 172.21.15.60:0/2832240848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fb5c74f050 msgr2=0x55fb5c6bd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.670+0000 7ff40cfeb080 1 --2- 172.21.15.60:0/2832240848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fb5c74f050 0x55fb5c6bd060 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc003040 tx=0x7ff3f4004130 comp rx=0 tx=0).stop 2024-04-25T10:15:14.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.670+0000 7ff40b30a640 1 -- 172.21.15.60:0/2832240848 reap_dead start 2024-04-25T10:15:14.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.670+0000 7ff40cfeb080 1 -- 172.21.15.60:0/2832240848 shutdown_connections 2024-04-25T10:15:14.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.670+0000 7ff40cfeb080 1 -- 172.21.15.60:0/2832240848 >> 172.21.15.60:0/2832240848 conn(0x55fb5c6a6e80 msgr2=0x55fb5c74e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.670+0000 7ff40cfeb080 1 -- 172.21.15.60:0/2832240848 shutdown_connections 2024-04-25T10:15:14.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.670+0000 7ff40cfeb080 1 -- 172.21.15.60:0/2832240848 wait complete. 2024-04-25T10:15:14.679 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:15:14.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5f918640 1 Processor -- start 2024-04-25T10:15:14.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5f918640 1 -- start start 2024-04-25T10:15:14.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5f918640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf580afc10 0x7fbf58070990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5f918640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf58140f50 0x7fbf5815c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5f918640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbf5815cb70 0x7fbf5806ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5f918640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf580c57c0 con 0x7fbf58140f50 2024-04-25T10:15:14.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5f918640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5815c1a0 con 0x7fbf5815cb70 2024-04-25T10:15:14.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5f918640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5815be50 con 0x7fbf580afc10 2024-04-25T10:15:14.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5ceb3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf58140f50 0x7fbf5815c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5ceb3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf58140f50 0x7fbf5815c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58232/0 (socket says 172.21.15.60:58232) 2024-04-25T10:15:14.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5ceb3640 1 -- 172.21.15.60:0/2534261583 learned_addr learned my addr 172.21.15.60:0/2534261583 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:14.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5deb5640 1 --2- 172.21.15.60:0/2534261583 >> v2:172.21.15.100:3300/0 conn(0x7fbf5815cb70 0x7fbf5806ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5ceb3640 1 -- 172.21.15.60:0/2534261583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf580afc10 msgr2=0x7fbf58070990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5d6b4640 1 --2- 172.21.15.60:0/2534261583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf580afc10 0x7fbf58070990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5ceb3640 1 --2- 172.21.15.60:0/2534261583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf580afc10 0x7fbf58070990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5ceb3640 1 -- 172.21.15.60:0/2534261583 >> v2:172.21.15.100:3300/0 conn(0x7fbf5815cb70 msgr2=0x7fbf5806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5ceb3640 1 --2- 172.21.15.60:0/2534261583 >> v2:172.21.15.100:3300/0 conn(0x7fbf5815cb70 0x7fbf5806ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5ceb3640 1 -- 172.21.15.60:0/2534261583 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf58141730 con 0x7fbf58140f50 2024-04-25T10:15:14.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5deb5640 1 --2- 172.21.15.60:0/2534261583 >> v2:172.21.15.100:3300/0 conn(0x7fbf5815cb70 0x7fbf5806ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:14.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.838+0000 7fbf5d6b4640 1 --2- 172.21.15.60:0/2534261583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf580afc10 0x7fbf58070990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5ceb3640 1 --2- 172.21.15.60:0/2534261583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf58140f50 0x7fbf5815c630 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fbf5806fca0 tx=0x7fbf4801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3979b937e733f23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf4ffff640 1 -- 172.21.15.60:0/2534261583 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf480b1070 con 0x7fbf58140f50 2024-04-25T10:15:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf4ffff640 1 -- 172.21.15.60:0/2534261583 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf480ac620 con 0x7fbf58140f50 2024-04-25T10:15:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf4ffff640 1 -- 172.21.15.60:0/2534261583 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf480ac9f0 con 0x7fbf58140f50 2024-04-25T10:15:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 -- 172.21.15.60:0/2534261583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf58140f50 msgr2=0x7fbf5815c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 --2- 172.21.15.60:0/2534261583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf58140f50 0x7fbf5815c630 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fbf5806fca0 tx=0x7fbf4801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 -- 172.21.15.60:0/2534261583 shutdown_connections 2024-04-25T10:15:14.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 --2- 172.21.15.60:0/2534261583 >> v2:172.21.15.100:3300/0 conn(0x7fbf5815cb70 0x7fbf5806ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 --2- 172.21.15.60:0/2534261583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf58140f50 0x7fbf5815c630 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 --2- 172.21.15.60:0/2534261583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf580afc10 0x7fbf58070990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 -- 172.21.15.60:0/2534261583 >> 172.21.15.60:0/2534261583 conn(0x7fbf5806e400 msgr2=0x7fbf5806e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 -- 172.21.15.60:0/2534261583 shutdown_connections 2024-04-25T10:15:14.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 -- 172.21.15.60:0/2534261583 wait complete. 2024-04-25T10:15:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 Processor -- start 2024-04-25T10:15:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 -- start start 2024-04-25T10:15:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf580afc10 0x7fbf5814f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf58140f50 0x7fbf580b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf580b41f0 0x7fbf58072370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf580b0170 con 0x7fbf580afc10 2024-04-25T10:15:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf580afff0 con 0x7fbf58140f50 2024-04-25T10:15:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5f918640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf580ce5f0 con 0x7fbf580b41f0 2024-04-25T10:15:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5d6b4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf580afc10 0x7fbf5814f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5d6b4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf580afc10 0x7fbf5814f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58246/0 (socket says 172.21.15.60:58246) 2024-04-25T10:15:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5d6b4640 1 -- 172.21.15.60:0/892446986 learned_addr learned my addr 172.21.15.60:0/892446986 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5deb5640 1 --2- 172.21.15.60:0/892446986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf580b41f0 0x7fbf58072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5ceb3640 1 --2- 172.21.15.60:0/892446986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf58140f50 0x7fbf580b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5d6b4640 1 -- 172.21.15.60:0/892446986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf580b41f0 msgr2=0x7fbf58072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5d6b4640 1 --2- 172.21.15.60:0/892446986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf580b41f0 0x7fbf58072370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5d6b4640 1 -- 172.21.15.60:0/892446986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf58140f50 msgr2=0x7fbf580b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5d6b4640 1 --2- 172.21.15.60:0/892446986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf58140f50 0x7fbf580b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.842+0000 7fbf5d6b4640 1 -- 172.21.15.60:0/892446986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf580b4fb0 con 0x7fbf580afc10 2024-04-25T10:15:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.846+0000 7fbf5d6b4640 1 --2- 172.21.15.60:0/892446986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf580afc10 0x7fbf5814f650 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400c7f0 tx=0x7fbf5400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.846+0000 7fbf4dffb640 1 -- 172.21.15.60:0/892446986 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54015070 con 0x7fbf580afc10 2024-04-25T10:15:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.846+0000 7fbf4dffb640 1 -- 172.21.15.60:0/892446986 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf54007a30 con 0x7fbf580afc10 2024-04-25T10:15:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.846+0000 7fbf5f918640 1 -- 172.21.15.60:0/892446986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf580b0540 con 0x7fbf580afc10 2024-04-25T10:15:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.846+0000 7fbf4dffb640 1 -- 172.21.15.60:0/892446986 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54007e00 con 0x7fbf580afc10 2024-04-25T10:15:14.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.846+0000 7fbf5f918640 1 -- 172.21.15.60:0/892446986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5814fb90 con 0x7fbf580afc10 2024-04-25T10:15:14.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.846+0000 7fbf5f918640 1 -- 172.21.15.60:0/892446986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf58074770 con 0x7fbf580afc10 2024-04-25T10:15:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.846+0000 7fbf4dffb640 1 -- 172.21.15.60:0/892446986 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf54018070 con 0x7fbf580afc10 2024-04-25T10:15:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.846+0000 7fbf4dffb640 1 --2- 172.21.15.60:0/892446986 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbf3c066f60 0x7fbf3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.846+0000 7fbf4dffb640 1 -- 172.21.15.60:0/892446986 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbf54084fd0 con 0x7fbf580afc10 2024-04-25T10:15:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.846+0000 7fbf5ceb3640 1 --2- 172.21.15.60:0/892446986 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbf3c066f60 0x7fbf3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.846+0000 7fbf5ceb3640 1 --2- 172.21.15.60:0/892446986 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbf3c066f60 0x7fbf3c069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbf480ae000 tx=0x7fbf48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:14.850+0000 7fbf4dffb640 1 -- 172.21.15.60:0/892446986 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbf54055f80 con 0x7fbf580afc10 2024-04-25T10:15:15.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.046+0000 7fbf5f918640 1 -- 172.21.15.60:0/892446986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fbf580ce1c0 con 0x7fbf580afc10 2024-04-25T10:15:15.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.414+0000 7fbf4dffb640 1 -- 172.21.15.60:0/892446986 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v79) v1 ==== 162+0+0 (secure 0 0 0) 0x7fbf54055880 con 0x7fbf580afc10 2024-04-25T10:15:15.421 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:15:15.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.418+0000 7fbf5f918640 1 -- 172.21.15.60:0/892446986 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbf3c066f60 msgr2=0x7fbf3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.418+0000 7fbf5f918640 1 --2- 172.21.15.60:0/892446986 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbf3c066f60 0x7fbf3c069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbf480ae000 tx=0x7fbf48008040 comp rx=0 tx=0).stop 2024-04-25T10:15:15.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.418+0000 7fbf5f918640 1 -- 172.21.15.60:0/892446986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf580afc10 msgr2=0x7fbf5814f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.418+0000 7fbf5f918640 1 --2- 172.21.15.60:0/892446986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf580afc10 0x7fbf5814f650 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400c7f0 tx=0x7fbf5400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.418+0000 7fbf5f918640 1 -- 172.21.15.60:0/892446986 shutdown_connections 2024-04-25T10:15:15.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.418+0000 7fbf5f918640 1 --2- 172.21.15.60:0/892446986 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbf3c066f60 0x7fbf3c069420 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.418+0000 7fbf5f918640 1 --2- 172.21.15.60:0/892446986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf580b41f0 0x7fbf58072370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.418+0000 7fbf5f918640 1 --2- 172.21.15.60:0/892446986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf58140f50 0x7fbf580b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.418+0000 7fbf5f918640 1 --2- 172.21.15.60:0/892446986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf580afc10 0x7fbf5814f650 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.418+0000 7fbf5f918640 1 -- 172.21.15.60:0/892446986 >> 172.21.15.60:0/892446986 conn(0x7fbf5806e400 msgr2=0x7fbf5815ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:15.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.418+0000 7fbf5f918640 1 -- 172.21.15.60:0/892446986 shutdown_connections 2024-04-25T10:15:15.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.418+0000 7fbf5f918640 1 -- 172.21.15.60:0/892446986 wait complete. 2024-04-25T10:15:15.446 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:15.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.558+0000 7ff405a1c640 1 Processor -- start 2024-04-25T10:15:15.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.558+0000 7ff405a1c640 1 -- start start 2024-04-25T10:15:15.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.558+0000 7ff405a1c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff40015a7a0 0x7ff4000cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.558+0000 7ff405a1c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40006f130 con 0x7ff4000b37e0 2024-04-25T10:15:15.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.558+0000 7ff405a1c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40006f410 con 0x7ff40006ec30 2024-04-25T10:15:15.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.558+0000 7ff405a1c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4000cdaa0 con 0x7ff40015a7a0 2024-04-25T10:15:15.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.558+0000 7ff3feffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff40015a7a0 0x7ff4000cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.558+0000 7ff3feffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff40015a7a0 0x7ff4000cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58262/0 (socket says 172.21.15.60:58262) 2024-04-25T10:15:15.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.558+0000 7ff3feffd640 1 -- 172.21.15.60:0/3281468550 learned_addr learned my addr 172.21.15.60:0/3281468550 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:15.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.558+0000 7ff3ff7fe640 1 --1- 172.21.15.60:0/3281468550 >> v1:172.21.15.60:6789/0 conn(0x7ff4000b37e0 0x7ff40006dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43444/0 (socket says 172.21.15.60:43444) 2024-04-25T10:15:15.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.562+0000 7ff3feffd640 1 -- 172.21.15.60:0/3281468550 >> v1:172.21.15.100:6789/0 conn(0x7ff40006ec30 legacy=0x7ff400071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.562+0000 7ff3feffd640 1 -- 172.21.15.60:0/3281468550 >> v1:172.21.15.60:6789/0 conn(0x7ff4000b37e0 legacy=0x7ff40006dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.562+0000 7ff3feffd640 1 -- 172.21.15.60:0/3281468550 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40006d540 con 0x7ff40015a7a0 2024-04-25T10:15:15.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.562+0000 7ff3fdffb640 1 -- 172.21.15.60:0/3281468550 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3619862030 0 0) 0x7ff40006f130 con 0x7ff4000b37e0 2024-04-25T10:15:15.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.562+0000 7ff3feffd640 1 --2- 172.21.15.60:0/3281468550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40015a7a0 0x7ff4000cd560 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00098a0 tx=0x7ff3f001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed49ebf66b211e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.562+0000 7ff3fdffb640 1 -- 172.21.15.60:0/3281468550 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f00ab650 con 0x7ff40015a7a0 2024-04-25T10:15:15.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.562+0000 7ff3fdffb640 1 -- 172.21.15.60:0/3281468550 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f00b0070 con 0x7ff40015a7a0 2024-04-25T10:15:15.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.562+0000 7ff3fdffb640 1 -- 172.21.15.60:0/3281468550 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f00aba10 con 0x7ff40015a7a0 2024-04-25T10:15:15.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.562+0000 7ff405a1c640 1 -- 172.21.15.60:0/3281468550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40015a7a0 msgr2=0x7ff4000cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.562+0000 7ff405a1c640 1 --2- 172.21.15.60:0/3281468550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40015a7a0 0x7ff4000cd560 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00098a0 tx=0x7ff3f001c540 comp rx=0 tx=0).stop 2024-04-25T10:15:15.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.562+0000 7ff405a1c640 1 -- 172.21.15.60:0/3281468550 shutdown_connections 2024-04-25T10:15:15.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.562+0000 7ff405a1c640 1 --2- 172.21.15.60:0/3281468550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40015a7a0 0x7ff4000cd560 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.562+0000 7ff405a1c640 1 -- 172.21.15.60:0/3281468550 >> 172.21.15.60:0/3281468550 conn(0x7ff4000b12d0 msgr2=0x7ff4000b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:15.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.562+0000 7ff405a1c640 1 -- 172.21.15.60:0/3281468550 shutdown_connections 2024-04-25T10:15:15.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.562+0000 7ff405a1c640 1 -- 172.21.15.60:0/3281468550 wait complete. 2024-04-25T10:15:15.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff405a1c640 1 Processor -- start 2024-04-25T10:15:15.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff405a1c640 1 -- start start 2024-04-25T10:15:15.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff405a1c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff40006ec30 0x7ff40014f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff405a1c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4000b37e0 0x7ff400153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff405a1c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40015a7a0 0x7ff4001595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff405a1c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40006d8b0 con 0x7ff40015a7a0 2024-04-25T10:15:15.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff405a1c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40006d2f0 con 0x7ff40006ec30 2024-04-25T10:15:15.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff405a1c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff400168d10 con 0x7ff4000b37e0 2024-04-25T10:15:15.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3ff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40015a7a0 0x7ff4001595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3ff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40015a7a0 0x7ff4001595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58276/0 (socket says 172.21.15.60:58276) 2024-04-25T10:15:15.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3ff7fe640 1 -- 172.21.15.60:0/943959262 learned_addr learned my addr 172.21.15.60:0/943959262 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:15.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3feffd640 1 --2- 172.21.15.60:0/943959262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff40006ec30 0x7ff40014f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3fe7fc640 1 --2- 172.21.15.60:0/943959262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4000b37e0 0x7ff400153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3ff7fe640 1 -- 172.21.15.60:0/943959262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4000b37e0 msgr2=0x7ff400153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3ff7fe640 1 --2- 172.21.15.60:0/943959262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4000b37e0 0x7ff400153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3ff7fe640 1 -- 172.21.15.60:0/943959262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff40006ec30 msgr2=0x7ff40014f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3ff7fe640 1 --2- 172.21.15.60:0/943959262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff40006ec30 0x7ff40014f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3ff7fe640 1 -- 172.21.15.60:0/943959262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40015de90 con 0x7ff40015a7a0 2024-04-25T10:15:15.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3feffd640 1 --2- 172.21.15.60:0/943959262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff40006ec30 0x7ff40014f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:15.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3ff7fe640 1 --2- 172.21.15.60:0/943959262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40015a7a0 0x7ff4001595d0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800c7a0 tx=0x7ff3f800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3e7fff640 1 -- 172.21.15.60:0/943959262 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8018070 con 0x7ff40015a7a0 2024-04-25T10:15:15.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3e7fff640 1 -- 172.21.15.60:0/943959262 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f800a640 con 0x7ff40015a7a0 2024-04-25T10:15:15.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3e7fff640 1 -- 172.21.15.60:0/943959262 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f800aab0 con 0x7ff40015a7a0 2024-04-25T10:15:15.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff405a1c640 1 -- 172.21.15.60:0/943959262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff400154530 con 0x7ff40015a7a0 2024-04-25T10:15:15.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff405a1c640 1 -- 172.21.15.60:0/943959262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40013dc40 con 0x7ff40015a7a0 2024-04-25T10:15:15.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff405a1c640 1 -- 172.21.15.60:0/943959262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff400074770 con 0x7ff40015a7a0 2024-04-25T10:15:15.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3e7fff640 1 -- 172.21.15.60:0/943959262 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3f801b070 con 0x7ff40015a7a0 2024-04-25T10:15:15.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3e7fff640 1 --2- 172.21.15.60:0/943959262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3ec066f60 0x7ff3ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.566+0000 7ff3e7fff640 1 -- 172.21.15.60:0/943959262 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff3f8084680 con 0x7ff40015a7a0 2024-04-25T10:15:15.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.570+0000 7ff3feffd640 1 --2- 172.21.15.60:0/943959262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3ec066f60 0x7ff3ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.570+0000 7ff3feffd640 1 --2- 172.21.15.60:0/943959262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3ec066f60 0x7ff3ec069420 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00af000 tx=0x7ff3f00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.574+0000 7ff3e7fff640 1 -- 172.21.15.60:0/943959262 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3f80556c0 con 0x7ff40015a7a0 2024-04-25T10:15:15.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.758+0000 7ff405a1c640 1 -- 172.21.15.60:0/943959262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4000cedf0 con 0x7ff40015a7a0 2024-04-25T10:15:15.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.758+0000 7ff3e7fff640 1 -- 172.21.15.60:0/943959262 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4944 (secure 0 0 0) 0x7ff3f8054fc0 con 0x7ff40015a7a0 2024-04-25T10:15:15.764 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:15:15.764 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":80,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1491228634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1491228634},{"type":"v1","addr":"172.21.15.100:6837","nonce":1491228634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/209898872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":209898872},{"type":"v1","addr":"172.21.15.100:6835","nonce":209898872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1793343287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1793343287},{"type":"v1","addr":"172.21.15.60:6833","nonce":1793343287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:16.146481+0000","modified":"2024-04-25T10:15:15.423690+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9048":{"gid":9048,"name":"a","rank":0,"incarnation":80,"state":"up:replay","state_seq":1,"addr":"172.21.15.60:6835/4149609651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":4149609651},{"type":"v1","addr":"172.21.15.60:6835","nonce":4149609651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:15:15.764 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 80 2024-04-25T10:15:15.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.758+0000 7ff405a1c640 1 -- 172.21.15.60:0/943959262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3ec066f60 msgr2=0x7ff3ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.758+0000 7ff405a1c640 1 --2- 172.21.15.60:0/943959262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3ec066f60 0x7ff3ec069420 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00af000 tx=0x7ff3f00a7000 comp rx=0 tx=0).stop 2024-04-25T10:15:15.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.758+0000 7ff405a1c640 1 -- 172.21.15.60:0/943959262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40015a7a0 msgr2=0x7ff4001595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.758+0000 7ff405a1c640 1 --2- 172.21.15.60:0/943959262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40015a7a0 0x7ff4001595d0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800c7a0 tx=0x7ff3f800cc60 comp rx=0 tx=0).stop 2024-04-25T10:15:15.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.758+0000 7ff405a1c640 1 -- 172.21.15.60:0/943959262 shutdown_connections 2024-04-25T10:15:15.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.758+0000 7ff405a1c640 1 --2- 172.21.15.60:0/943959262 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff3ec066f60 0x7ff3ec069420 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.758+0000 7ff405a1c640 1 --2- 172.21.15.60:0/943959262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff40015a7a0 0x7ff4001595d0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.758+0000 7ff405a1c640 1 --2- 172.21.15.60:0/943959262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4000b37e0 0x7ff400153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.758+0000 7ff405a1c640 1 --2- 172.21.15.60:0/943959262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff40006ec30 0x7ff40014f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.758+0000 7ff405a1c640 1 -- 172.21.15.60:0/943959262 >> 172.21.15.60:0/943959262 conn(0x7ff4000b12d0 msgr2=0x7ff40015d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:15.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.758+0000 7ff405a1c640 1 -- 172.21.15.60:0/943959262 shutdown_connections 2024-04-25T10:15:15.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:15.758+0000 7ff405a1c640 1 -- 172.21.15.60:0/943959262 wait complete. 2024-04-25T10:15:16.184 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:14:16.146481+0000', 'modified': '2024-04-25T10:15:15.423690+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9048}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9048': {'gid': 9048, 'name': 'a', 'rank': 0, 'incarnation': 80, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.60:6835/4149609651', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 4149609651}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 4149609651}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:15:16.184 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9048:up:replay 2024-04-25T10:15:17.185 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:17.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.310+0000 7f398c046640 1 Processor -- start 2024-04-25T10:15:17.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.310+0000 7f398c046640 1 -- start start 2024-04-25T10:15:17.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.314+0000 7f398c046640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39840cdc30 0x7f398406ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.314+0000 7f398c046640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39840cff30 0x7f39840cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.314+0000 7f398c046640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39840d03b0 con 0x7f39840cfae0 2024-04-25T10:15:17.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.314+0000 7f398c046640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39840d0690 con 0x7f39840cdc30 2024-04-25T10:15:17.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.314+0000 7f398c046640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39840d0970 con 0x7f39840cff30 2024-04-25T10:15:17.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.314+0000 7f3989de2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39840cdc30 0x7f398406ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.314+0000 7f3989de2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39840cdc30 0x7f398406ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58278/0 (socket says 172.21.15.60:58278) 2024-04-25T10:15:17.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.314+0000 7f3989de2640 1 -- 172.21.15.60:0/2814591784 learned_addr learned my addr 172.21.15.60:0/2814591784 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.314+0000 7f398a5e3640 1 --2- 172.21.15.60:0/2814591784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39840cff30 0x7f39840cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.314+0000 7f3989de2640 1 -- 172.21.15.60:0/2814591784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39840cff30 msgr2=0x7f39840cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.314+0000 7f3989de2640 1 --2- 172.21.15.60:0/2814591784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39840cff30 0x7f39840cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.314+0000 7f3989de2640 1 -- 172.21.15.60:0/2814591784 >> v1:172.21.15.100:6789/0 conn(0x7f39840cfae0 legacy=0x7f3984099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.314+0000 7f3989de2640 1 -- 172.21.15.60:0/2814591784 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39840d0c50 con 0x7f39840cdc30 2024-04-25T10:15:17.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f3989de2640 1 --2- 172.21.15.60:0/2814591784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39840cdc30 0x7f398406ce10 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f39800098a0 tx=0x7f398001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cd11753138001 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f3988de0640 1 -- 172.21.15.60:0/2814591784 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39800b0070 con 0x7f39840cdc30 2024-04-25T10:15:17.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f3988de0640 1 -- 172.21.15.60:0/2814591784 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39800ab600 con 0x7f39840cdc30 2024-04-25T10:15:17.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f3988de0640 1 -- 172.21.15.60:0/2814591784 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39800ab9d0 con 0x7f39840cdc30 2024-04-25T10:15:17.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398c046640 1 -- 172.21.15.60:0/2814591784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39840cdc30 msgr2=0x7f398406ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398c046640 1 --2- 172.21.15.60:0/2814591784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39840cdc30 0x7f398406ce10 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f39800098a0 tx=0x7f398001c540 comp rx=0 tx=0).stop 2024-04-25T10:15:17.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398c046640 1 -- 172.21.15.60:0/2814591784 shutdown_connections 2024-04-25T10:15:17.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398c046640 1 --2- 172.21.15.60:0/2814591784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39840cff30 0x7f39840cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398c046640 1 --2- 172.21.15.60:0/2814591784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39840cdc30 0x7f398406ce10 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398c046640 1 -- 172.21.15.60:0/2814591784 >> 172.21.15.60:0/2814591784 conn(0x7f39840b32a0 msgr2=0x7f39840b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398c046640 1 -- 172.21.15.60:0/2814591784 shutdown_connections 2024-04-25T10:15:17.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398c046640 1 -- 172.21.15.60:0/2814591784 wait complete. 2024-04-25T10:15:17.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398c046640 1 Processor -- start 2024-04-25T10:15:17.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398c046640 1 -- start start 2024-04-25T10:15:17.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398c046640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39840cdc30 0x7f398409f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398c046640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39840cfae0 0x7f398415b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398c046640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39840cff30 0x7f398406f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398c046640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39840ccb50 con 0x7f39840cdc30 2024-04-25T10:15:17.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398c046640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398409a350 con 0x7f39840cfae0 2024-04-25T10:15:17.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398c046640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39840cccd0 con 0x7f39840cff30 2024-04-25T10:15:17.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f3989de2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39840cdc30 0x7f398409f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f3989de2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39840cdc30 0x7f398409f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58280/0 (socket says 172.21.15.60:58280) 2024-04-25T10:15:17.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f3989de2640 1 -- 172.21.15.60:0/3141196358 learned_addr learned my addr 172.21.15.60:0/3141196358 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:17.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f39895e1640 1 --2- 172.21.15.60:0/3141196358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39840cfae0 0x7f398415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f398a5e3640 1 --2- 172.21.15.60:0/3141196358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39840cff30 0x7f398406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f3989de2640 1 -- 172.21.15.60:0/3141196358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39840cff30 msgr2=0x7f398406f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f3989de2640 1 --2- 172.21.15.60:0/3141196358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39840cff30 0x7f398406f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f3989de2640 1 -- 172.21.15.60:0/3141196358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39840cfae0 msgr2=0x7f398415b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f3989de2640 1 --2- 172.21.15.60:0/3141196358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39840cfae0 0x7f398415b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.318+0000 7f3989de2640 1 -- 172.21.15.60:0/3141196358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398415b990 con 0x7f39840cdc30 2024-04-25T10:15:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.322+0000 7f3989de2640 1 --2- 172.21.15.60:0/3141196358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39840cdc30 0x7f398409f210 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f398001a170 tx=0x7f39800192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.322+0000 7f397a7fc640 1 -- 172.21.15.60:0/3141196358 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39800ba070 con 0x7f39840cdc30 2024-04-25T10:15:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.322+0000 7f397a7fc640 1 -- 172.21.15.60:0/3141196358 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39800b4600 con 0x7f39840cdc30 2024-04-25T10:15:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.322+0000 7f398c046640 1 -- 172.21.15.60:0/3141196358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398415c780 con 0x7f39840cdc30 2024-04-25T10:15:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.322+0000 7f397a7fc640 1 -- 172.21.15.60:0/3141196358 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39800b0040 con 0x7f39840cdc30 2024-04-25T10:15:17.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.322+0000 7f398c046640 1 -- 172.21.15.60:0/3141196358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398415b660 con 0x7f39840cdc30 2024-04-25T10:15:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.322+0000 7f398c046640 1 -- 172.21.15.60:0/3141196358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3984074770 con 0x7f39840cdc30 2024-04-25T10:15:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.322+0000 7f397a7fc640 1 -- 172.21.15.60:0/3141196358 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f39800bd070 con 0x7f39840cdc30 2024-04-25T10:15:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.322+0000 7f397a7fc640 1 --2- 172.21.15.60:0/3141196358 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3968066f60 0x7f3968069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.322+0000 7f397a7fc640 1 -- 172.21.15.60:0/3141196358 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f398011e4c0 con 0x7f39840cdc30 2024-04-25T10:15:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.322+0000 7f39895e1640 1 --2- 172.21.15.60:0/3141196358 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3968066f60 0x7f3968069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.322+0000 7f39895e1640 1 --2- 172.21.15.60:0/3141196358 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3968066f60 0x7f3968069420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f39840b6390 tx=0x7f3974009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.326+0000 7f397a7fc640 1 -- 172.21.15.60:0/3141196358 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3980016130 con 0x7f39840cdc30 2024-04-25T10:15:17.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.522+0000 7f398c046640 1 -- 172.21.15.60:0/3141196358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39840b7730 con 0x7f39840cdc30 2024-04-25T10:15:17.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.522+0000 7f397a7fc640 1 -- 172.21.15.60:0/3141196358 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f39800ef500 con 0x7f39840cdc30 2024-04-25T10:15:17.527 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:15:17.527 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1491228634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1491228634},{"type":"v1","addr":"172.21.15.100:6837","nonce":1491228634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/209898872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":209898872},{"type":"v1","addr":"172.21.15.100:6835","nonce":209898872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1793343287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1793343287},{"type":"v1","addr":"172.21.15.60:6833","nonce":1793343287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:16.146481+0000","modified":"2024-04-25T10:15:16.431846+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9048":{"gid":9048,"name":"a","rank":0,"incarnation":80,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.60:6835/4149609651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":4149609651},{"type":"v1","addr":"172.21.15.60:6835","nonce":4149609651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:15:17.527 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 82 2024-04-25T10:15:17.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.522+0000 7f398c046640 1 -- 172.21.15.60:0/3141196358 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3968066f60 msgr2=0x7f3968069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.522+0000 7f398c046640 1 --2- 172.21.15.60:0/3141196358 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3968066f60 0x7f3968069420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f39840b6390 tx=0x7f3974009040 comp rx=0 tx=0).stop 2024-04-25T10:15:17.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.522+0000 7f398c046640 1 -- 172.21.15.60:0/3141196358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39840cdc30 msgr2=0x7f398409f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.522+0000 7f398c046640 1 --2- 172.21.15.60:0/3141196358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39840cdc30 0x7f398409f210 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f398001a170 tx=0x7f39800192e0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.522+0000 7f398c046640 1 -- 172.21.15.60:0/3141196358 shutdown_connections 2024-04-25T10:15:17.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.522+0000 7f398c046640 1 --2- 172.21.15.60:0/3141196358 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3968066f60 0x7f3968069420 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.522+0000 7f398c046640 1 --2- 172.21.15.60:0/3141196358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39840cff30 0x7f398406f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.522+0000 7f398c046640 1 --2- 172.21.15.60:0/3141196358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39840cfae0 0x7f398415b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.522+0000 7f398c046640 1 --2- 172.21.15.60:0/3141196358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39840cdc30 0x7f398409f210 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.522+0000 7f398c046640 1 -- 172.21.15.60:0/3141196358 >> 172.21.15.60:0/3141196358 conn(0x7f39840b32a0 msgr2=0x7f3984161e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.522+0000 7f398c046640 1 -- 172.21.15.60:0/3141196358 shutdown_connections 2024-04-25T10:15:17.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:17.522+0000 7f398c046640 1 -- 172.21.15.60:0/3141196358 wait complete. 2024-04-25T10:15:17.824 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:17.827 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:17.948 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:14:16.146481+0000', 'modified': '2024-04-25T10:15:16.431846+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9048}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9048': {'gid': 9048, 'name': 'a', 'rank': 0, 'incarnation': 80, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.60:6835/4149609651', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 4149609651}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 4149609651}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:15:17.948 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9048:up:rejoin 2024-04-25T10:15:18.949 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:19.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.078+0000 7f499a45a640 1 Processor -- start 2024-04-25T10:15:19.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.078+0000 7f499a45a640 1 -- start start 2024-04-25T10:15:19.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.078+0000 7f499a45a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f49940719a0 0x7f49940ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.078+0000 7f499a45a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4994071fd0 con 0x7f499415dc90 2024-04-25T10:15:19.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.078+0000 7f499a45a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49940b2ce0 con 0x7f49940ae7c0 2024-04-25T10:15:19.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.078+0000 7f499a45a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49940accc0 con 0x7f49940719a0 2024-04-25T10:15:19.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.082+0000 7f49989f7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f49940719a0 0x7f49940ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.082+0000 7f4993fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f499415dc90 0x7f49940b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36684/0 (socket says 172.21.15.60:36684) 2024-04-25T10:15:19.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.082+0000 7f4993fff640 1 -- 172.21.15.60:0/1338746069 learned_addr learned my addr 172.21.15.60:0/1338746069 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:15:19.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.082+0000 7f49989f7640 1 -- 172.21.15.60:0/1338746069 >> v1:172.21.15.100:6789/0 conn(0x7f49940ae7c0 legacy=0x7f4994071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.082+0000 7f49989f7640 1 -- 172.21.15.60:0/1338746069 >> v1:172.21.15.60:6789/0 conn(0x7f499415dc90 legacy=0x7f49940b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.082+0000 7f49989f7640 1 -- 172.21.15.60:0/1338746069 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4994072290 con 0x7f49940719a0 2024-04-25T10:15:19.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.082+0000 7f49989f7640 1 --2- 172.21.15.60:0/1338746069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49940719a0 0x7f49940ad6f0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f498c009d90 tx=0x7f498c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640443f818053c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:19.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.082+0000 7f4992ffd640 1 -- 172.21.15.60:0/1338746069 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c0b2070 con 0x7f49940719a0 2024-04-25T10:15:19.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.082+0000 7f4992ffd640 1 -- 172.21.15.60:0/1338746069 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498c0a5940 con 0x7f49940719a0 2024-04-25T10:15:19.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.082+0000 7f4992ffd640 1 -- 172.21.15.60:0/1338746069 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c0a5d10 con 0x7f49940719a0 2024-04-25T10:15:19.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 -- 172.21.15.60:0/1338746069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49940719a0 msgr2=0x7f49940ad6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 --2- 172.21.15.60:0/1338746069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49940719a0 0x7f49940ad6f0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f498c009d90 tx=0x7f498c01c970 comp rx=0 tx=0).stop 2024-04-25T10:15:19.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 -- 172.21.15.60:0/1338746069 shutdown_connections 2024-04-25T10:15:19.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 --2- 172.21.15.60:0/1338746069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49940719a0 0x7f49940ad6f0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 -- 172.21.15.60:0/1338746069 >> 172.21.15.60:0/1338746069 conn(0x7f49940b12d0 msgr2=0x7f49940b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:19.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 -- 172.21.15.60:0/1338746069 shutdown_connections 2024-04-25T10:15:19.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 -- 172.21.15.60:0/1338746069 wait complete. 2024-04-25T10:15:19.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 Processor -- start 2024-04-25T10:15:19.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 -- start start 2024-04-25T10:15:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49940719a0 0x7f49941466d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49940ae7c0 0x7f4994142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f499415dc90 0x7f4994145850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49940ad260 con 0x7f49940719a0 2024-04-25T10:15:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49940b2220 con 0x7f499415dc90 2024-04-25T10:15:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499416c000 con 0x7f49940ae7c0 2024-04-25T10:15:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f4993fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49940719a0 0x7f49941466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f4993fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49940719a0 0x7f49941466d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54972/0 (socket says 172.21.15.60:54972) 2024-04-25T10:15:19.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f4993fff640 1 -- 172.21.15.60:0/532631703 learned_addr learned my addr 172.21.15.60:0/532631703 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f49937fe640 1 --2- 172.21.15.60:0/532631703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49940ae7c0 0x7f4994142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f4993fff640 1 -- 172.21.15.60:0/532631703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49940ae7c0 msgr2=0x7f4994142970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f4993fff640 1 --2- 172.21.15.60:0/532631703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49940ae7c0 0x7f4994142970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f49989f7640 1 --2- 172.21.15.60:0/532631703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f499415dc90 0x7f4994145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f4993fff640 1 -- 172.21.15.60:0/532631703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f499415dc90 msgr2=0x7f4994145850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f4993fff640 1 --2- 172.21.15.60:0/532631703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f499415dc90 0x7f4994145850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f4993fff640 1 -- 172.21.15.60:0/532631703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4994146c10 con 0x7f49940719a0 2024-04-25T10:15:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f4993fff640 1 --2- 172.21.15.60:0/532631703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49940719a0 0x7f49941466d0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f498400c400 tx=0x7f498400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f4990ff9640 1 -- 172.21.15.60:0/532631703 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984017070 con 0x7f49940719a0 2024-04-25T10:15:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f4990ff9640 1 -- 172.21.15.60:0/532631703 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4984005cf0 con 0x7f49940719a0 2024-04-25T10:15:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 -- 172.21.15.60:0/532631703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499414a3e0 con 0x7f49940719a0 2024-04-25T10:15:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f4990ff9640 1 -- 172.21.15.60:0/532631703 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49840072d0 con 0x7f49940719a0 2024-04-25T10:15:19.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.086+0000 7f499a45a640 1 -- 172.21.15.60:0/532631703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4994150280 con 0x7f49940719a0 2024-04-25T10:15:19.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.090+0000 7f499a45a640 1 -- 172.21.15.60:0/532631703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4994074770 con 0x7f49940719a0 2024-04-25T10:15:19.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.090+0000 7f4990ff9640 1 -- 172.21.15.60:0/532631703 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f498401a070 con 0x7f49940719a0 2024-04-25T10:15:19.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.090+0000 7f4990ff9640 1 --2- 172.21.15.60:0/532631703 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4980066f90 0x7f4980069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.090+0000 7f4990ff9640 1 -- 172.21.15.60:0/532631703 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f49840846f0 con 0x7f49940719a0 2024-04-25T10:15:19.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.090+0000 7f49937fe640 1 --2- 172.21.15.60:0/532631703 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4980066f90 0x7f4980069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.090+0000 7f49937fe640 1 --2- 172.21.15.60:0/532631703 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4980066f90 0x7f4980069450 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4988007fd0 tx=0x7f4988009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.094+0000 7f4990ff9640 1 -- 172.21.15.60:0/532631703 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4984055130 con 0x7f49940719a0 2024-04-25T10:15:19.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.282+0000 7f499a45a640 1 -- 172.21.15.60:0/532631703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4994071d80 con 0x7f49940719a0 2024-04-25T10:15:19.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.286+0000 7f4990ff9640 1 -- 172.21.15.60:0/532631703 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f49840591d0 con 0x7f49940719a0 2024-04-25T10:15:19.290 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:15:19.290 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1491228634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1491228634},{"type":"v1","addr":"172.21.15.100:6837","nonce":1491228634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/209898872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":209898872},{"type":"v1","addr":"172.21.15.100:6835","nonce":209898872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1793343287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1793343287},{"type":"v1","addr":"172.21.15.60:6833","nonce":1793343287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:16.146481+0000","modified":"2024-04-25T10:15:17.435090+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9048":{"gid":9048,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/4149609651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":4149609651},{"type":"v1","addr":"172.21.15.60:6835","nonce":4149609651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:15:19.291 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 83 2024-04-25T10:15:19.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.286+0000 7f499a45a640 1 -- 172.21.15.60:0/532631703 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4980066f90 msgr2=0x7f4980069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.286+0000 7f499a45a640 1 --2- 172.21.15.60:0/532631703 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4980066f90 0x7f4980069450 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4988007fd0 tx=0x7f4988009040 comp rx=0 tx=0).stop 2024-04-25T10:15:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.286+0000 7f499a45a640 1 -- 172.21.15.60:0/532631703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49940719a0 msgr2=0x7f49941466d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.286+0000 7f499a45a640 1 --2- 172.21.15.60:0/532631703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49940719a0 0x7f49941466d0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f498400c400 tx=0x7f498400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.286+0000 7f499a45a640 1 -- 172.21.15.60:0/532631703 shutdown_connections 2024-04-25T10:15:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.286+0000 7f499a45a640 1 --2- 172.21.15.60:0/532631703 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4980066f90 0x7f4980069450 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.286+0000 7f499a45a640 1 --2- 172.21.15.60:0/532631703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f499415dc90 0x7f4994145850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.286+0000 7f499a45a640 1 --2- 172.21.15.60:0/532631703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49940ae7c0 0x7f4994142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.286+0000 7f499a45a640 1 --2- 172.21.15.60:0/532631703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49940719a0 0x7f49941466d0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.286+0000 7f499a45a640 1 -- 172.21.15.60:0/532631703 >> 172.21.15.60:0/532631703 conn(0x7f49940b12d0 msgr2=0x7f49940b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.286+0000 7f499a45a640 1 -- 172.21.15.60:0/532631703 shutdown_connections 2024-04-25T10:15:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.286+0000 7f499a45a640 1 -- 172.21.15.60:0/532631703 wait complete. 2024-04-25T10:15:19.710 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:14:16.146481+0000', 'modified': '2024-04-25T10:15:17.435090+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9048}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9048': {'gid': 9048, 'name': 'a', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.60:6835/4149609651', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 4149609651}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 4149609651}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:15:19.711 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:15:19.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:15:19.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e8cb3640 1 Processor -- start 2024-04-25T10:15:19.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e8cb3640 1 -- start start 2024-04-25T10:15:19.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e8cb3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54e415bc90 0x7f54e415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e8cb3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e4071ca0 con 0x7f54e40b4660 2024-04-25T10:15:19.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e8cb3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e40cca40 con 0x7f54e40ce7a0 2024-04-25T10:15:19.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e8cb3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40cc370 con 0x7f54e415bc90 2024-04-25T10:15:19.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e2d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f54e40b4660 0x7f54e415c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36696/0 (socket says 172.21.15.60:36696) 2024-04-25T10:15:19.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e2d9d640 1 -- 172.21.15.60:0/1054563586 learned_addr learned my addr 172.21.15.60:0/1054563586 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:15:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e1d9b640 1 --2- 172.21.15.60:0/1054563586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54e415bc90 0x7f54e415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 -- 172.21.15.60:0/1054563586 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024999499 0 0) 0x7f54e4071ca0 con 0x7f54e40b4660 2024-04-25T10:15:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 -- 172.21.15.60:0/1054563586 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54cc003530 con 0x7f54e40b4660 2024-04-25T10:15:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 -- 172.21.15.60:0/1054563586 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861610928 0 0) 0x7f54e40cca40 con 0x7f54e40ce7a0 2024-04-25T10:15:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 -- 172.21.15.60:0/1054563586 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54e4071ca0 con 0x7f54e40ce7a0 2024-04-25T10:15:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 -- 172.21.15.60:0/1054563586 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4065641215 0 0) 0x7f54cc003530 con 0x7f54e40b4660 2024-04-25T10:15:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 -- 172.21.15.60:0/1054563586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54e415bc90 msgr2=0x7f54e415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 --2- 172.21.15.60:0/1054563586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54e415bc90 0x7f54e415c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 -- 172.21.15.60:0/1054563586 >> v1:172.21.15.100:6789/0 conn(0x7f54e40ce7a0 legacy=0x7f54e40ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 -- 172.21.15.60:0/1054563586 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40ccd20 con 0x7f54e40b4660 2024-04-25T10:15:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 -- 172.21.15.60:0/1054563586 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f54dc001a00 con 0x7f54e40b4660 2024-04-25T10:15:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 -- 172.21.15.60:0/1054563586 >> v1:172.21.15.60:6789/0 conn(0x7f54e40b4660 legacy=0x7f54e415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 --2- 172.21.15.60:0/1054563586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54cc003b80 0x7f54cc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 --2- 172.21.15.60:0/1054563586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54cc004810 0x7f54cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 --2- 172.21.15.60:0/1054563586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54cc0051d0 0x7f54cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 -- 172.21.15.60:0/1054563586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc00c0d0 con 0x7f54cc003b80 2024-04-25T10:15:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 -- 172.21.15.60:0/1054563586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc00c280 con 0x7f54cc0051d0 2024-04-25T10:15:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e159a640 1 -- 172.21.15.60:0/1054563586 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc00c480 con 0x7f54cc004810 2024-04-25T10:15:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e1d9b640 1 --2- 172.21.15.60:0/1054563586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54e415bc90 0x7f54e415c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e259c640 1 --2- 172.21.15.60:0/1054563586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54cc003b80 0x7f54cc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e259c640 1 -- 172.21.15.60:0/1054563586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54cc004810 msgr2=0x7f54cc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e259c640 1 --2- 172.21.15.60:0/1054563586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54cc004810 0x7f54cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e259c640 1 -- 172.21.15.60:0/1054563586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54cc0051d0 msgr2=0x7f54cc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e2d9d640 1 --2- 172.21.15.60:0/1054563586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54cc0051d0 0x7f54cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e259c640 1 --2- 172.21.15.60:0/1054563586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54cc0051d0 0x7f54cc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e259c640 1 -- 172.21.15.60:0/1054563586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc003530 con 0x7f54cc003b80 2024-04-25T10:15:19.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.846+0000 7f54e2d9d640 1 --2- 172.21.15.60:0/1054563586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54cc0051d0 0x7f54cc00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:19.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.850+0000 7f54e259c640 1 --2- 172.21.15.60:0/1054563586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54cc003b80 0x7f54cc003f80 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f54d40018f0 tx=0x7f54e415d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0ad972086bd47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:19.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.850+0000 7f54e159a640 1 -- 172.21.15.60:0/1054563586 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d4001510 con 0x7f54cc003b80 2024-04-25T10:15:19.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.850+0000 7f54e159a640 1 -- 172.21.15.60:0/1054563586 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d40052d0 con 0x7f54cc003b80 2024-04-25T10:15:19.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.850+0000 7f54e159a640 1 -- 172.21.15.60:0/1054563586 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d40054c0 con 0x7f54cc003b80 2024-04-25T10:15:19.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.850+0000 7f54e8cb3640 1 -- 172.21.15.60:0/1054563586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54cc003b80 msgr2=0x7f54cc003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.850+0000 7f54e8cb3640 1 --2- 172.21.15.60:0/1054563586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54cc003b80 0x7f54cc003f80 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f54d40018f0 tx=0x7f54e415d680 comp rx=0 tx=0).stop 2024-04-25T10:15:19.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.850+0000 7f54e2d9d640 1 -- 172.21.15.60:0/1054563586 reap_dead start 2024-04-25T10:15:19.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.850+0000 7f54e8cb3640 1 -- 172.21.15.60:0/1054563586 shutdown_connections 2024-04-25T10:15:19.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.850+0000 7f54e8cb3640 1 -- 172.21.15.60:0/1054563586 >> 172.21.15.60:0/1054563586 conn(0x7f54e40b12d0 msgr2=0x7f54e40b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:19.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.850+0000 7f54e8cb3640 1 -- 172.21.15.60:0/1054563586 shutdown_connections 2024-04-25T10:15:19.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.850+0000 7f54e8cb3640 1 -- 172.21.15.60:0/1054563586 wait complete. 2024-04-25T10:15:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.850+0000 7f54e8cb3640 1 Processor -- start 2024-04-25T10:15:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.850+0000 7f54e8cb3640 1 -- start start 2024-04-25T10:15:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e8cb3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54cc003b80 0x7f54e4152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e8cb3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54e4153320 0x7f54e4160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e8cb3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e4160a80 0x7f54e40af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e8cb3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40b3db0 con 0x7f54e4160a80 2024-04-25T10:15:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e8cb3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40b4490 con 0x7f54e4153320 2024-04-25T10:15:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e8cb3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e41675c0 con 0x7f54cc003b80 2024-04-25T10:15:19.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e4160a80 0x7f54e40af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e4160a80 0x7f54e40af9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54976/0 (socket says 172.21.15.60:54976) 2024-04-25T10:15:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e2d9d640 1 -- 172.21.15.60:0/1805174332 learned_addr learned my addr 172.21.15.60:0/1805174332 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e1d9b640 1 --2- 172.21.15.60:0/1805174332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54e4153320 0x7f54e4160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e259c640 1 --2- 172.21.15.60:0/1805174332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54cc003b80 0x7f54e4152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e2d9d640 1 -- 172.21.15.60:0/1805174332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54cc003b80 msgr2=0x7f54e4152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e2d9d640 1 --2- 172.21.15.60:0/1805174332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54cc003b80 0x7f54e4152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e2d9d640 1 -- 172.21.15.60:0/1805174332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54e4153320 msgr2=0x7f54e4160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e2d9d640 1 --2- 172.21.15.60:0/1805174332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54e4153320 0x7f54e4160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e2d9d640 1 -- 172.21.15.60:0/1805174332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e4153090 con 0x7f54e4160a80 2024-04-25T10:15:19.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e259c640 1 --2- 172.21.15.60:0/1805174332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54cc003b80 0x7f54e4152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:19.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e2d9d640 1 --2- 172.21.15.60:0/1805174332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e4160a80 0x7f54e40af9e0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f54e4072c10 tx=0x7f54dc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:19.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54d2ffd640 1 -- 172.21.15.60:0/1805174332 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc004d70 con 0x7f54e4160a80 2024-04-25T10:15:19.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54d2ffd640 1 -- 172.21.15.60:0/1805174332 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc00bb20 con 0x7f54e4160a80 2024-04-25T10:15:19.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e8cb3640 1 -- 172.21.15.60:0/1805174332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e4160f40 con 0x7f54e4160a80 2024-04-25T10:15:19.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54d2ffd640 1 -- 172.21.15.60:0/1805174332 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc007110 con 0x7f54e4160a80 2024-04-25T10:15:19.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e8cb3640 1 -- 172.21.15.60:0/1805174332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e4161500 con 0x7f54e4160a80 2024-04-25T10:15:19.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54d2ffd640 1 -- 172.21.15.60:0/1805174332 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54dc01b020 con 0x7f54e4160a80 2024-04-25T10:15:19.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54d2ffd640 1 --2- 172.21.15.60:0/1805174332 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f54c4066f60 0x7f54c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54d2ffd640 1 -- 172.21.15.60:0/1805174332 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f54dc085260 con 0x7f54e4160a80 2024-04-25T10:15:19.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.854+0000 7f54e259c640 1 --2- 172.21.15.60:0/1805174332 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f54c4066f60 0x7f54c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 Processor -- start 2024-04-25T10:15:19.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 -- start start 2024-04-25T10:15:19.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54e259c640 1 --2- 172.21.15.60:0/1805174332 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f54c4066f60 0x7f54c4069420 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f54d40018c0 tx=0x7f54d40012e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:19.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bde6920 0x56024bde6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde7240 0x56024bd9c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bd9cf30 0x56024bd9d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56024bc845f0 con 0x56024bd9cf30 2024-04-25T10:15:19.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56024bbebda0 con 0x56024bde7240 2024-04-25T10:15:19.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56024bb74150 con 0x56024bde6920 2024-04-25T10:15:19.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bd9cf30 0x56024bd9d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bd9cf30 0x56024bd9d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54992/0 (socket says 172.21.15.60:54992) 2024-04-25T10:15:19.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54e2d9d640 1 -- 172.21.15.60:0/3650500445 learned_addr learned my addr 172.21.15.60:0/3650500445 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:19.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54e259c640 1 --2- 172.21.15.60:0/3650500445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bde6920 0x56024bde6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54e1d9b640 1 --2- 172.21.15.60:0/3650500445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde7240 0x56024bd9c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54e2d9d640 1 -- 172.21.15.60:0/3650500445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bde6920 msgr2=0x56024bde6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:19.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54e2d9d640 1 --2- 172.21.15.60:0/3650500445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bde6920 0x56024bde6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54e2d9d640 1 -- 172.21.15.60:0/3650500445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde7240 msgr2=0x56024bd9c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:19.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54e2d9d640 1 --2- 172.21.15.60:0/3650500445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde7240 0x56024bd9c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54e2d9d640 1 -- 172.21.15.60:0/3650500445 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56024bb8dca0 con 0x56024bd9cf30 2024-04-25T10:15:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54e2d9d640 1 --2- 172.21.15.60:0/3650500445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bd9cf30 0x56024bd9d310 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f54dc004fd0 tx=0x7f54dc051ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa11f2865888e6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54c3fff640 1 -- 172.21.15.60:0/3650500445 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc059c50 con 0x56024bd9cf30 2024-04-25T10:15:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54c3fff640 1 -- 172.21.15.60:0/3650500445 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc05e210 con 0x56024bd9cf30 2024-04-25T10:15:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54c3fff640 1 -- 172.21.15.60:0/3650500445 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc052170 con 0x56024bd9cf30 2024-04-25T10:15:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 -- 172.21.15.60:0/3650500445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bd9cf30 msgr2=0x56024bd9d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 --2- 172.21.15.60:0/3650500445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bd9cf30 0x56024bd9d310 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f54dc004fd0 tx=0x7f54dc051ed0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 -- 172.21.15.60:0/3650500445 shutdown_connections 2024-04-25T10:15:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 --2- 172.21.15.60:0/3650500445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bd9cf30 0x56024bd9d310 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 --2- 172.21.15.60:0/3650500445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde7240 0x56024bd9c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 --2- 172.21.15.60:0/3650500445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bde6920 0x56024bde6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 -- 172.21.15.60:0/3650500445 >> 172.21.15.60:0/3650500445 conn(0x56024bda32a0 msgr2=0x56024bd9b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 -- 172.21.15.60:0/3650500445 shutdown_connections 2024-04-25T10:15:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 -- 172.21.15.60:0/3650500445 wait complete. 2024-04-25T10:15:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 -- start start 2024-04-25T10:15:19.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.858+0000 7f54ebe20000 1 Processor -- start 2024-04-25T10:15:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bd9cf30 0x56024bdede60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde6920 0x56024bdee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde7240 0x56024bda0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56024bba6990 con 0x56024bde6920 2024-04-25T10:15:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56024bbe7e40 con 0x56024bde7240 2024-04-25T10:15:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56024bb77900 con 0x56024bd9cf30 2024-04-25T10:15:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 10 client.0 _ll_register_callbacks cb 0x56024bc9f1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54e1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde6920 0x56024bdee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54e1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde6920 0x56024bdee3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55002/0 (socket says 172.21.15.60:55002) 2024-04-25T10:15:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54e1d9b640 1 -- 172.21.15.60:0/3275285639 learned_addr learned my addr 172.21.15.60:0/3275285639 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54e259c640 1 --2- 172.21.15.60:0/3275285639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bd9cf30 0x56024bdede60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54e2d9d640 1 --2- 172.21.15.60:0/3275285639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde7240 0x56024bda0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54e1d9b640 1 -- 172.21.15.60:0/3275285639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bd9cf30 msgr2=0x56024bdede60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54e1d9b640 1 --2- 172.21.15.60:0/3275285639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bd9cf30 0x56024bdede60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54e1d9b640 1 -- 172.21.15.60:0/3275285639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde7240 msgr2=0x56024bda0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54e1d9b640 1 --2- 172.21.15.60:0/3275285639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde7240 0x56024bda0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54e1d9b640 1 -- 172.21.15.60:0/3275285639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56024bba4630 con 0x56024bde6920 2024-04-25T10:15:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54e1d9b640 1 --2- 172.21.15.60:0/3275285639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde6920 0x56024bdee3a0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f54d8012320 tx=0x7f54d80127e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137b73a34ddc6d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:15:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54c17fa640 1 -- 172.21.15.60:0/3275285639 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d8019070 con 0x56024bde6920 2024-04-25T10:15:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54c17fa640 1 -- 172.21.15.60:0/3275285639 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f54d8004240 con 0x56024bde6920 2024-04-25T10:15:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 -- 172.21.15.60:0/3275285639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54dc05e210 con 0x56024bde6920 2024-04-25T10:15:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54c17fa640 1 -- 172.21.15.60:0/3275285639 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54d8024050 con 0x56024bde6920 2024-04-25T10:15:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 10 client.9117 fetch_fsmap learned FSMap version 83 2024-04-25T10:15:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 -- 172.21.15.60:0/3275285639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x56024bb743a0 con 0x56024bde6920 2024-04-25T10:15:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54c17fa640 1 -- 172.21.15.60:0/3275285639 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc05e210 con 0x56024bde6920 2024-04-25T10:15:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54c17fa640 1 -- 172.21.15.60:0/3275285639 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f54d8027020 con 0x56024bde6920 2024-04-25T10:15:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 10 client.9117 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:15:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 10 client.9117 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:15:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 20 client.9117 populate_metadata read hostname 'smithi060' 2024-04-25T10:15:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 --2- 172.21.15.60:0/3275285639 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x56024bda2160 0x56024bdf7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 4 client.9117 mds_command: new command op to 9048 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 -- 172.21.15.60:0/3275285639 --> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56024bb87a50 con 0x56024bda2160 2024-04-25T10:15:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54e259c640 1 --2- 172.21.15.60:0/3275285639 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x56024bda2160 0x56024bdf7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54c17fa640 10 client.9117 ms_handle_connect on v2:172.21.15.60:6834/4149609651 2024-04-25T10:15:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54e259c640 1 --2- 172.21.15.60:0/3275285639 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x56024bda2160 0x56024bdf7ec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a98b5f4f541a6d0 server_cookie=d597c6efe8afbcf5 in_seq=0 out_seq=0 2024-04-25T10:15:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54c17fa640 1 -- 172.21.15.60:0/3275285639 <== mds.0 v2:172.21.15.60:6834/4149609651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56024bb87a50 con 0x56024bda2160 2024-04-25T10:15:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54c17fa640 10 client.9117 handle_command_reply: tid=0 2024-04-25T10:15:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 client.9117 shutdown 2024-04-25T10:15:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 -- 172.21.15.60:0/3275285639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde6920 msgr2=0x56024bdee3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 --2- 172.21.15.60:0/3275285639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde6920 0x56024bdee3a0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f54d8012320 tx=0x7f54d80127e0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 -- 172.21.15.60:0/3275285639 shutdown_connections 2024-04-25T10:15:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 --2- 172.21.15.60:0/3275285639 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x56024bda2160 0x56024bdf7ec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 --2- 172.21.15.60:0/3275285639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde7240 0x56024bda0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 --2- 172.21.15.60:0/3275285639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde6920 0x56024bdee3a0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 --2- 172.21.15.60:0/3275285639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bd9cf30 0x56024bdede60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54c17fa640 0 client.9117 ms_handle_reset on v2:172.21.15.60:6834/4149609651 2024-04-25T10:15:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 -- 172.21.15.60:0/3275285639 >> 172.21.15.60:0/3275285639 conn(0x56024bda32a0 msgr2=0x56024bd9b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 -- 172.21.15.60:0/3275285639 shutdown_connections 2024-04-25T10:15:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 1 -- 172.21.15.60:0/3275285639 wait complete. 2024-04-25T10:15:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.862+0000 7f54ebe20000 20 client.9117 trim_cache size 0 max 16384 2024-04-25T10:15:19.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54ebe20000 1 Processor -- start 2024-04-25T10:15:19.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54ebe20000 1 -- start start 2024-04-25T10:15:19.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54ebe20000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bd9cf30 0x56024bda02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54ebe20000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde6920 0x56024bda0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54ebe20000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde7240 0x56024bdefa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54ebe20000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e4200570 con 0x56024bde7240 2024-04-25T10:15:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54ebe20000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e41658d0 con 0x56024bde6920 2024-04-25T10:15:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54ebe20000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e4200010 con 0x56024bd9cf30 2024-04-25T10:15:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde7240 0x56024bdefa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde7240 0x56024bdefa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55012/0 (socket says 172.21.15.60:55012) 2024-04-25T10:15:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54e2d9d640 1 -- 172.21.15.60:0/2728473151 learned_addr learned my addr 172.21.15.60:0/2728473151 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54e1d9b640 1 --2- 172.21.15.60:0/2728473151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde6920 0x56024bda0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54e2d9d640 1 -- 172.21.15.60:0/2728473151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bd9cf30 msgr2=0x56024bda02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54e2d9d640 1 --2- 172.21.15.60:0/2728473151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bd9cf30 0x56024bda02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54e2d9d640 1 -- 172.21.15.60:0/2728473151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde6920 msgr2=0x56024bda0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54e2d9d640 1 --2- 172.21.15.60:0/2728473151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde6920 0x56024bda0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54e2d9d640 1 -- 172.21.15.60:0/2728473151 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56024bde7070 con 0x56024bde7240 2024-04-25T10:15:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54e1d9b640 1 --2- 172.21.15.60:0/2728473151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde6920 0x56024bda0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.878+0000 7f54e2d9d640 1 --2- 172.21.15.60:0/2728473151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde7240 0x56024bdefa60 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f54dc05c0b0 tx=0x7f54dc057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6232aa3453b9ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:19.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54c37fe640 1 -- 172.21.15.60:0/2728473151 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc083410 con 0x56024bde7240 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54c37fe640 1 -- 172.21.15.60:0/2728473151 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc059c50 con 0x56024bde7240 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54c37fe640 1 -- 172.21.15.60:0/2728473151 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc052170 con 0x56024bde7240 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 -- 172.21.15.60:0/2728473151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde7240 msgr2=0x56024bdefa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 --2- 172.21.15.60:0/2728473151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde7240 0x56024bdefa60 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f54dc05c0b0 tx=0x7f54dc057000 comp rx=0 tx=0).stop 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 -- 172.21.15.60:0/2728473151 shutdown_connections 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 --2- 172.21.15.60:0/2728473151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde7240 0x56024bdefa60 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 --2- 172.21.15.60:0/2728473151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde6920 0x56024bda0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 --2- 172.21.15.60:0/2728473151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bd9cf30 0x56024bda02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 -- 172.21.15.60:0/2728473151 >> 172.21.15.60:0/2728473151 conn(0x56024bda32a0 msgr2=0x56024bd9fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 -- 172.21.15.60:0/2728473151 shutdown_connections 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 -- 172.21.15.60:0/2728473151 wait complete. 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 -- start start 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 Processor -- start 2024-04-25T10:15:19.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bd9cf30 0x56024bdf3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde6920 0x56024be2da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde7240 0x56024be2dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56024bbf03d0 con 0x56024bde7240 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56024bca7d50 con 0x56024bde6920 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56024bb86980 con 0x56024bd9cf30 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 10 client.0 _ll_register_callbacks cb 0x7f54dc051da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde7240 0x56024be2dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde7240 0x56024be2dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55016/0 (socket says 172.21.15.60:55016) 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54e2d9d640 1 -- 172.21.15.60:0/2031545225 learned_addr learned my addr 172.21.15.60:0/2031545225 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54e2d9d640 1 -- 172.21.15.60:0/2031545225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bd9cf30 msgr2=0x56024bdf3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54e2d9d640 1 --2- 172.21.15.60:0/2031545225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bd9cf30 0x56024bdf3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54e2d9d640 1 -- 172.21.15.60:0/2031545225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde6920 msgr2=0x56024be2da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54e1d9b640 1 --2- 172.21.15.60:0/2031545225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde6920 0x56024be2da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54e2d9d640 1 --2- 172.21.15.60:0/2031545225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde6920 0x56024be2da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54e2d9d640 1 -- 172.21.15.60:0/2031545225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56024bc6d600 con 0x56024bde7240 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54e1d9b640 1 --2- 172.21.15.60:0/2031545225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde6920 0x56024be2da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54e2d9d640 1 --2- 172.21.15.60:0/2031545225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde7240 0x56024be2dfc0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0557b0 tx=0x7f54dc01f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7344788f5b7e0676 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:15:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54c17fa640 1 -- 172.21.15.60:0/2031545225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc05df80 con 0x56024bde7240 2024-04-25T10:15:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54c17fa640 1 -- 172.21.15.60:0/2031545225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f54dc05ab70 con 0x56024bde7240 2024-04-25T10:15:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 -- 172.21.15.60:0/2031545225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54dc059c50 con 0x56024bde7240 2024-04-25T10:15:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54c17fa640 1 -- 172.21.15.60:0/2031545225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54dc07c050 con 0x56024bde7240 2024-04-25T10:15:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 10 client.9123 fetch_fsmap learned FSMap version 83 2024-04-25T10:15:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 -- 172.21.15.60:0/2031545225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x56024bb809b0 con 0x56024bde7240 2024-04-25T10:15:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54c17fa640 1 -- 172.21.15.60:0/2031545225 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc059c50 con 0x56024bde7240 2024-04-25T10:15:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54c17fa640 1 -- 172.21.15.60:0/2031545225 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f54dc054f50 con 0x56024bde7240 2024-04-25T10:15:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 10 client.9123 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:15:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 10 client.9123 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:15:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 20 client.9123 populate_metadata read hostname 'smithi060' 2024-04-25T10:15:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 --2- 172.21.15.60:0/2031545225 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x56024be01940 0x56024be01d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 4 client.9123 mds_command: new command op to 9048 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54ebe20000 1 -- 172.21.15.60:0/2031545225 --> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56024bbf92e0 con 0x56024be01940 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54e259c640 1 --2- 172.21.15.60:0/2031545225 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x56024be01940 0x56024be01d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54c17fa640 10 client.9123 ms_handle_connect on v2:172.21.15.60:6834/4149609651 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.882+0000 7f54e259c640 1 --2- 172.21.15.60:0/2031545225 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x56024be01940 0x56024be01d20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e66ed195516f37c server_cookie=134128556879be9e in_seq=0 out_seq=0 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54c17fa640 1 -- 172.21.15.60:0/2031545225 <== mds.0 v2:172.21.15.60:6834/4149609651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+317 (crc 0 0 0) 0x56024bbf92e0 con 0x56024be01940 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54c17fa640 10 client.9123 handle_command_reply: tid=0 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54ebe20000 1 client.9123 shutdown 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54ebe20000 1 -- 172.21.15.60:0/2031545225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde7240 msgr2=0x56024be2dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54ebe20000 1 --2- 172.21.15.60:0/2031545225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde7240 0x56024be2dfc0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0557b0 tx=0x7f54dc01f070 comp rx=0 tx=0).stop 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54ebe20000 1 -- 172.21.15.60:0/2031545225 shutdown_connections 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54ebe20000 1 --2- 172.21.15.60:0/2031545225 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x56024be01940 0x56024be01d20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54ebe20000 1 --2- 172.21.15.60:0/2031545225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56024bde7240 0x56024be2dfc0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54ebe20000 1 --2- 172.21.15.60:0/2031545225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56024bde6920 0x56024be2da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54c17fa640 0 client.9123 ms_handle_reset on v2:172.21.15.60:6834/4149609651 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54ebe20000 1 --2- 172.21.15.60:0/2031545225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56024bd9cf30 0x56024bdf3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54ebe20000 1 -- 172.21.15.60:0/2031545225 >> 172.21.15.60:0/2031545225 conn(0x56024bda32a0 msgr2=0x56024bda02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:19.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54ebe20000 1 -- 172.21.15.60:0/2031545225 shutdown_connections 2024-04-25T10:15:19.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54ebe20000 1 -- 172.21.15.60:0/2031545225 wait complete. 2024-04-25T10:15:19.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54ebe20000 20 client.9123 trim_cache size 0 max 16384 2024-04-25T10:15:19.896 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:15:19.896 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "0ea93863-a079-45c4-abed-6e83540ea9fe", 2024-04-25T10:15:19.896 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T10:15:19.896 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9048, 2024-04-25T10:15:19.896 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T10:15:19.896 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T10:15:19.897 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T10:15:19.897 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 4.458155595, 2024-04-25T10:15:19.897 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 83, 2024-04-25T10:15:19.897 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 98, 2024-04-25T10:15:19.897 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 98, 2024-04-25T10:15:19.897 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 5.423362472 2024-04-25T10:15:19.897 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:15:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54e8cb3640 1 -- 172.21.15.60:0/1805174332 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f54c4066f60 msgr2=0x7f54c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54e8cb3640 1 --2- 172.21.15.60:0/1805174332 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f54c4066f60 0x7f54c4069420 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f54d40018c0 tx=0x7f54d40012e0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54e8cb3640 1 -- 172.21.15.60:0/1805174332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e4160a80 msgr2=0x7f54e40af9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54e8cb3640 1 --2- 172.21.15.60:0/1805174332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e4160a80 0x7f54e40af9e0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f54e4072c10 tx=0x7f54dc00b4a0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54e8cb3640 1 -- 172.21.15.60:0/1805174332 shutdown_connections 2024-04-25T10:15:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54e8cb3640 1 --2- 172.21.15.60:0/1805174332 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f54c4066f60 0x7f54c4069420 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54e8cb3640 1 --2- 172.21.15.60:0/1805174332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e4160a80 0x7f54e40af9e0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54e8cb3640 1 --2- 172.21.15.60:0/1805174332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54e4153320 0x7f54e4160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54e8cb3640 1 --2- 172.21.15.60:0/1805174332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54cc003b80 0x7f54e4152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54e8cb3640 1 -- 172.21.15.60:0/1805174332 >> 172.21.15.60:0/1805174332 conn(0x7f54e40b12d0 msgr2=0x7f54e4155d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54e8cb3640 1 -- 172.21.15.60:0/1805174332 shutdown_connections 2024-04-25T10:15:19.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:19.886+0000 7f54e8cb3640 1 -- 172.21.15.60:0/1805174332 wait complete. 2024-04-25T10:15:19.907 DEBUG:teuthology.orchestra.run.smithi060:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:15:19.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:19.909 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-25T10:15:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.026+0000 7f06eb294640 1 Processor -- start 2024-04-25T10:15:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.026+0000 7f06eb294640 1 -- start start 2024-04-25T10:15:20.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.026+0000 7f06eb294640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06e415a790 0x7f06e40b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.026+0000 7f06eb294640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06e4072620 0x7f06e406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.026+0000 7f06eb294640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e415ab70 con 0x7f06e406f280 2024-04-25T10:15:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.026+0000 7f06eb294640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e40b4580 con 0x7f06e415a790 2024-04-25T10:15:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.026+0000 7f06eb294640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e40b4860 con 0x7f06e4072620 2024-04-25T10:15:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.026+0000 7f06e9030640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06e415a790 0x7f06e40b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.026+0000 7f06e9030640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06e415a790 0x7f06e40b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55018/0 (socket says 172.21.15.60:55018) 2024-04-25T10:15:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.026+0000 7f06e9030640 1 -- 172.21.15.60:0/1505049489 learned_addr learned my addr 172.21.15.60:0/1505049489 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.026+0000 7f06e882f640 1 --2- 172.21.15.60:0/1505049489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06e4072620 0x7f06e406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.026+0000 7f06e9030640 1 -- 172.21.15.60:0/1505049489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06e4072620 msgr2=0x7f06e406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.026+0000 7f06e9030640 1 --2- 172.21.15.60:0/1505049489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06e4072620 0x7f06e406ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.026+0000 7f06e9030640 1 -- 172.21.15.60:0/1505049489 >> v1:172.21.15.100:6789/0 conn(0x7f06e406f280 legacy=0x7f06e406bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.030+0000 7f06e9030640 1 -- 172.21.15.60:0/1505049489 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e40b4b40 con 0x7f06e415a790 2024-04-25T10:15:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.030+0000 7f06e882f640 1 --2- 172.21.15.60:0/1505049489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06e4072620 0x7f06e406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.030+0000 7f06e9030640 1 --2- 172.21.15.60:0/1505049489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06e415a790 0x7f06e40b3fa0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009870 tx=0x7f06e001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa4fd63935ee0cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.030+0000 7f06dbfff640 1 -- 172.21.15.60:0/1505049489 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e00a9610 con 0x7f06e415a790 2024-04-25T10:15:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.030+0000 7f06dbfff640 1 -- 172.21.15.60:0/1505049489 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e00a97b0 con 0x7f06e415a790 2024-04-25T10:15:20.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.030+0000 7f06dbfff640 1 -- 172.21.15.60:0/1505049489 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e00a9b80 con 0x7f06e415a790 2024-04-25T10:15:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.030+0000 7f06eb294640 1 -- 172.21.15.60:0/1505049489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06e415a790 msgr2=0x7f06e40b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.030+0000 7f06eb294640 1 --2- 172.21.15.60:0/1505049489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06e415a790 0x7f06e40b3fa0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009870 tx=0x7f06e001c530 comp rx=0 tx=0).stop 2024-04-25T10:15:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.030+0000 7f06eb294640 1 -- 172.21.15.60:0/1505049489 shutdown_connections 2024-04-25T10:15:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.030+0000 7f06eb294640 1 --2- 172.21.15.60:0/1505049489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06e4072620 0x7f06e406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.030+0000 7f06eb294640 1 --2- 172.21.15.60:0/1505049489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06e415a790 0x7f06e40b3fa0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.030+0000 7f06eb294640 1 -- 172.21.15.60:0/1505049489 >> 172.21.15.60:0/1505049489 conn(0x7f06e4071b00 msgr2=0x7f06e4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.030+0000 7f06eb294640 1 -- 172.21.15.60:0/1505049489 shutdown_connections 2024-04-25T10:15:20.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.030+0000 7f06eb294640 1 -- 172.21.15.60:0/1505049489 wait complete. 2024-04-25T10:15:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06eb294640 1 Processor -- start 2024-04-25T10:15:20.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06eb294640 1 -- start start 2024-04-25T10:15:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06eb294640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06e406f280 0x7f06e414a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06eb294640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06e4072620 0x7f06e414e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06eb294640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06e415a790 0x7f06e40b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06eb294640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e415f240 con 0x7f06e4072620 2024-04-25T10:15:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06eb294640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e415f0c0 con 0x7f06e415a790 2024-04-25T10:15:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06eb294640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e406c330 con 0x7f06e406f280 2024-04-25T10:15:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06e882f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06e4072620 0x7f06e414e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06e882f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06e4072620 0x7f06e414e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55020/0 (socket says 172.21.15.60:55020) 2024-04-25T10:15:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06e882f640 1 -- 172.21.15.60:0/3315939107 learned_addr learned my addr 172.21.15.60:0/3315939107 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:20.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06e9030640 1 --2- 172.21.15.60:0/3315939107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06e406f280 0x7f06e414a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06e9831640 1 --2- 172.21.15.60:0/3315939107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06e415a790 0x7f06e40b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06e882f640 1 -- 172.21.15.60:0/3315939107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06e406f280 msgr2=0x7f06e414a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06e882f640 1 --2- 172.21.15.60:0/3315939107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06e406f280 0x7f06e414a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06e882f640 1 -- 172.21.15.60:0/3315939107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06e415a790 msgr2=0x7f06e40b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06e882f640 1 --2- 172.21.15.60:0/3315939107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06e415a790 0x7f06e40b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06e882f640 1 -- 172.21.15.60:0/3315939107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e414a920 con 0x7f06e4072620 2024-04-25T10:15:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06e882f640 1 --2- 172.21.15.60:0/3315939107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06e4072620 0x7f06e414e370 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f06d400c7c0 tx=0x7f06d400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06d9ffb640 1 -- 172.21.15.60:0/3315939107 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4007a30 con 0x7f06e4072620 2024-04-25T10:15:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06d9ffb640 1 -- 172.21.15.60:0/3315939107 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d4007bd0 con 0x7f06e4072620 2024-04-25T10:15:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06eb294640 1 -- 172.21.15.60:0/3315939107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e414ec70 con 0x7f06e4072620 2024-04-25T10:15:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06d9ffb640 1 -- 172.21.15.60:0/3315939107 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d40113f0 con 0x7f06e4072620 2024-04-25T10:15:20.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06eb294640 1 -- 172.21.15.60:0/3315939107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e414f230 con 0x7f06e4072620 2024-04-25T10:15:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f06d9ffb640 1 -- 172.21.15.60:0/3315939107 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06d4011590 con 0x7f06e4072620 2024-04-25T10:15:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.034+0000 7f5a3f122640 1 Processor -- start 2024-04-25T10:15:20.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f5a3f122640 1 -- start start 2024-04-25T10:15:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06d9ffb640 1 --2- 172.21.15.60:0/3315939107 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f06c8066f60 0x7f06c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06d9ffb640 1 -- 172.21.15.60:0/3315939107 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f06d40852e0 con 0x7f06e4072620 2024-04-25T10:15:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06e9030640 1 --2- 172.21.15.60:0/3315939107 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f06c8066f60 0x7f06c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f5a3f122640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a3815ee60 0x7f5a380cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f5a3f122640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a3815f240 0x7f5a3809a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f5a3f122640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a3809ae00 con 0x7f5a380cffc0 2024-04-25T10:15:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f5a3f122640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a380cfb30 con 0x7f5a3815f240 2024-04-25T10:15:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f5a3f122640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a380cd4e0 con 0x7f5a3815ee60 2024-04-25T10:15:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 Processor -- start 2024-04-25T10:15:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 -- start start 2024-04-25T10:15:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f5a37fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a3815f240 0x7f5a3809a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f5a37fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a3815f240 0x7f5a3809a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55024/0 (socket says 172.21.15.60:55024) 2024-04-25T10:15:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f5a37fff640 1 -- 172.21.15.60:0/4284140619 learned_addr learned my addr 172.21.15.60:0/4284140619 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f5a3cebe640 1 --2- 172.21.15.60:0/4284140619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a3815ee60 0x7f5a380cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f5a37fff640 1 -- 172.21.15.60:0/4284140619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a3815ee60 msgr2=0x7f5a380cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f5a37fff640 1 --2- 172.21.15.60:0/4284140619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a3815ee60 0x7f5a380cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f5a37fff640 1 -- 172.21.15.60:0/4284140619 >> v1:172.21.15.100:6789/0 conn(0x7f5a380cffc0 legacy=0x7f5a380c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f5a37fff640 1 -- 172.21.15.60:0/4284140619 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a380cf4e0 con 0x7f5a3815f240 2024-04-25T10:15:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06e9030640 1 --2- 172.21.15.60:0/3315939107 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f06c8066f60 0x7f06c8069420 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f06e00ae000 tx=0x7f06e00b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f3b3c90 0x556b1f3b4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f3b45b0 0x556b1f369d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f36a290 0x556b1f36a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b1f250f40 con 0x556b1f36a290 2024-04-25T10:15:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b1f1b8070 con 0x556b1f3b3c90 2024-04-25T10:15:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b1f1401a0 con 0x556b1f3b45b0 2024-04-25T10:15:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06e9831640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f36a290 0x556b1f36a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06e9831640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f36a290 0x556b1f36a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55032/0 (socket says 172.21.15.60:55032) 2024-04-25T10:15:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06e9831640 1 -- 172.21.15.60:0/3352874693 learned_addr learned my addr 172.21.15.60:0/3352874693 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06e882f640 1 --2- 172.21.15.60:0/3352874693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f3b45b0 0x556b1f369d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06e9030640 1 --2- 172.21.15.60:0/3352874693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f3b3c90 0x556b1f3b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06e9831640 1 -- 172.21.15.60:0/3352874693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f3b45b0 msgr2=0x556b1f369d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06e9831640 1 --2- 172.21.15.60:0/3352874693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f3b45b0 0x556b1f369d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06e9831640 1 -- 172.21.15.60:0/3352874693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f3b3c90 msgr2=0x556b1f3b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06e9831640 1 --2- 172.21.15.60:0/3352874693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f3b3c90 0x556b1f3b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06e9831640 1 -- 172.21.15.60:0/3352874693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b1f13e510 con 0x556b1f36a290 2024-04-25T10:15:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06e9831640 1 --2- 172.21.15.60:0/3352874693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f36a290 0x556b1f36a670 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f06dc001ca0 tx=0x7f06dc002160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6ee00d3eb512d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ceffd640 1 -- 172.21.15.60:0/3352874693 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc019070 con 0x556b1f36a290 2024-04-25T10:15:20.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ceffd640 1 -- 172.21.15.60:0/3352874693 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06dc003b90 con 0x556b1f36a290 2024-04-25T10:15:20.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ceffd640 1 -- 172.21.15.60:0/3352874693 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc00e8c0 con 0x556b1f36a290 2024-04-25T10:15:20.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 -- 172.21.15.60:0/3352874693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f36a290 msgr2=0x556b1f36a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 --2- 172.21.15.60:0/3352874693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f36a290 0x556b1f36a670 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f06dc001ca0 tx=0x7f06dc002160 comp rx=0 tx=0).stop 2024-04-25T10:15:20.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 -- 172.21.15.60:0/3352874693 shutdown_connections 2024-04-25T10:15:20.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 --2- 172.21.15.60:0/3352874693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f36a290 0x556b1f36a670 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 --2- 172.21.15.60:0/3352874693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f3b45b0 0x556b1f369d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 --2- 172.21.15.60:0/3352874693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f3b3c90 0x556b1f3b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 -- 172.21.15.60:0/3352874693 >> 172.21.15.60:0/3352874693 conn(0x556b1f366240 msgr2=0x556b1f368630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:20.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 -- 172.21.15.60:0/3352874693 shutdown_connections 2024-04-25T10:15:20.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f06ee401000 1 -- 172.21.15.60:0/3352874693 wait complete. 2024-04-25T10:15:20.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.038+0000 7f5a37fff640 1 --2- 172.21.15.60:0/4284140619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a3815f240 0x7f5a3809a8c0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0098a0 tx=0x7f5a2c01ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f628f2c61ffca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a377fe640 1 -- 172.21.15.60:0/4284140619 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c0b5070 con 0x7f5a3815f240 2024-04-25T10:15:20.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a377fe640 1 -- 172.21.15.60:0/4284140619 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a2c0a8a30 con 0x7f5a3815f240 2024-04-25T10:15:20.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a377fe640 1 -- 172.21.15.60:0/4284140619 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c0a8e00 con 0x7f5a3815f240 2024-04-25T10:15:20.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 1 -- start start 2024-04-25T10:15:20.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 1 Processor -- start 2024-04-25T10:15:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3f122640 1 -- 172.21.15.60:0/4284140619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a3815f240 msgr2=0x7f5a3809a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3f122640 1 --2- 172.21.15.60:0/4284140619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a3815f240 0x7f5a3809a8c0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0098a0 tx=0x7f5a2c01ed30 comp rx=0 tx=0).stop 2024-04-25T10:15:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3f122640 1 -- 172.21.15.60:0/4284140619 shutdown_connections 2024-04-25T10:15:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3f122640 1 --2- 172.21.15.60:0/4284140619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a3815f240 0x7f5a3809a8c0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3f122640 1 --2- 172.21.15.60:0/4284140619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a3815ee60 0x7f5a380cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3f122640 1 -- 172.21.15.60:0/4284140619 >> 172.21.15.60:0/4284140619 conn(0x7f5a380b3260 msgr2=0x7f5a380b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3f122640 1 -- 172.21.15.60:0/4284140619 shutdown_connections 2024-04-25T10:15:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3f122640 1 -- 172.21.15.60:0/4284140619 wait complete. 2024-04-25T10:15:20.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f36a290 0x556b1f374e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f3b3c90 0x556b1f375370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f3b45b0 0x556b1f3758b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b1f1726c0 con 0x556b1f3b3c90 2024-04-25T10:15:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b1f1b4110 con 0x556b1f3b45b0 2024-04-25T10:15:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b1f26ef20 con 0x556b1f36a290 2024-04-25T10:15:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 10 client.0 _ll_register_callbacks cb 0x556b1f355570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06e882f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f3b3c90 0x556b1f375370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06e882f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f3b3c90 0x556b1f375370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55040/0 (socket says 172.21.15.60:55040) 2024-04-25T10:15:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06e882f640 1 -- 172.21.15.60:0/370714689 learned_addr learned my addr 172.21.15.60:0/370714689 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06e9030640 1 --2- 172.21.15.60:0/370714689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f36a290 0x556b1f374e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06e9831640 1 --2- 172.21.15.60:0/370714689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f3b45b0 0x556b1f3758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06e882f640 1 -- 172.21.15.60:0/370714689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f36a290 msgr2=0x556b1f374e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06e882f640 1 --2- 172.21.15.60:0/370714689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f36a290 0x556b1f374e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06e882f640 1 -- 172.21.15.60:0/370714689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f3b45b0 msgr2=0x556b1f3758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06e882f640 1 --2- 172.21.15.60:0/370714689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f3b45b0 0x556b1f3758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06e882f640 1 -- 172.21.15.60:0/370714689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b1f1403b0 con 0x556b1f3b3c90 2024-04-25T10:15:20.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06e9831640 1 --2- 172.21.15.60:0/370714689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f3b45b0 0x556b1f3758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06e882f640 1 --2- 172.21.15.60:0/370714689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f3b3c90 0x556b1f375370 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f06d4083710 tx=0x7f06d4059c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b284dd3f97a58ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:15:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06c3fff640 1 -- 172.21.15.60:0/370714689 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d401e040 con 0x556b1f3b3c90 2024-04-25T10:15:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06c3fff640 1 -- 172.21.15.60:0/370714689 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f06d4083d30 con 0x556b1f3b3c90 2024-04-25T10:15:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 1 -- 172.21.15.60:0/370714689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06dc003b90 con 0x556b1f3b3c90 2024-04-25T10:15:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06c3fff640 1 -- 172.21.15.60:0/370714689 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06d4085110 con 0x556b1f3b3c90 2024-04-25T10:15:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3f122640 1 Processor -- start 2024-04-25T10:15:20.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 10 client.9138 fetch_fsmap learned FSMap version 83 2024-04-25T10:15:20.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 1 -- 172.21.15.60:0/370714689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x556b1f239e00 con 0x556b1f3b3c90 2024-04-25T10:15:20.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3f122640 1 -- start start 2024-04-25T10:15:20.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06c3fff640 1 -- 172.21.15.60:0/370714689 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06dc003b90 con 0x556b1f3b3c90 2024-04-25T10:15:20.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06c3fff640 1 -- 172.21.15.60:0/370714689 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f06d4054f70 con 0x556b1f3b3c90 2024-04-25T10:15:20.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 10 client.9138 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:15:20.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 10 client.9138 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:15:20.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 20 client.9138 populate_metadata read hostname 'smithi060' 2024-04-25T10:15:20.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 1 --2- 172.21.15.60:0/370714689 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x556b1f376180 0x556b1f3c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 4 client.9138 mds_command: new command op to 9048 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:20.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06ee401000 1 -- 172.21.15.60:0/370714689 --> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b1f153d60 con 0x556b1f376180 2024-04-25T10:15:20.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06e9030640 1 --2- 172.21.15.60:0/370714689 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x556b1f376180 0x556b1f3c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3f122640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a380cffc0 0x7f5a380c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3f122640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a3815ee60 0x7f5a3814fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3f122640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a3815f240 0x7f5a380cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3f122640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a380c59d0 con 0x7f5a3815ee60 2024-04-25T10:15:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3f122640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a380cf7c0 con 0x7f5a380cffc0 2024-04-25T10:15:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3f122640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a380c5b50 con 0x7f5a3815f240 2024-04-25T10:15:20.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06e9030640 1 --2- 172.21.15.60:0/370714689 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x556b1f376180 0x556b1f3c5410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=961f7570fe332b32 server_cookie=35a40da47467d95f in_seq=0 out_seq=0 2024-04-25T10:15:20.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f06c3fff640 10 client.9138 ms_handle_connect on v2:172.21.15.60:6834/4149609651 2024-04-25T10:15:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a37fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a3815ee60 0x7f5a3814fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a37fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a3815ee60 0x7f5a3814fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55054/0 (socket says 172.21.15.60:55054) 2024-04-25T10:15:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a37fff640 1 -- 172.21.15.60:0/1510715055 learned_addr learned my addr 172.21.15.60:0/1510715055 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3cebe640 1 --2- 172.21.15.60:0/1510715055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a380cffc0 0x7f5a380c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a3d6bf640 1 --2- 172.21.15.60:0/1510715055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a3815f240 0x7f5a380cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a37fff640 1 -- 172.21.15.60:0/1510715055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a3815f240 msgr2=0x7f5a380cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a37fff640 1 --2- 172.21.15.60:0/1510715055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a3815f240 0x7f5a380cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a37fff640 1 -- 172.21.15.60:0/1510715055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a380cffc0 msgr2=0x7f5a380c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a37fff640 1 --2- 172.21.15.60:0/1510715055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a380cffc0 0x7f5a380c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.042+0000 7f5a37fff640 1 -- 172.21.15.60:0/1510715055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a380c9740 con 0x7f5a3815ee60 2024-04-25T10:15:20.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f06c3fff640 1 -- 172.21.15.60:0/370714689 <== mds.0 v2:172.21.15.60:6834/4149609651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556b1f153d60 con 0x556b1f376180 2024-04-25T10:15:20.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f06c3fff640 10 client.9138 handle_command_reply: tid=0 2024-04-25T10:15:20.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f5a37fff640 1 --2- 172.21.15.60:0/1510715055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a3815ee60 0x7f5a3814fb70 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0b3040 tx=0x7f5a2c01bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f06ee401000 1 client.9138 shutdown 2024-04-25T10:15:20.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f06ee401000 1 -- 172.21.15.60:0/370714689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f3b3c90 msgr2=0x556b1f375370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f06ee401000 1 --2- 172.21.15.60:0/370714689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f3b3c90 0x556b1f375370 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f06d4083710 tx=0x7f06d4059c80 comp rx=0 tx=0).stop 2024-04-25T10:15:20.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f5a357fa640 1 -- 172.21.15.60:0/1510715055 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c0b5070 con 0x7f5a3815ee60 2024-04-25T10:15:20.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f5a357fa640 1 -- 172.21.15.60:0/1510715055 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a2c0b0b60 con 0x7f5a3815ee60 2024-04-25T10:15:20.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f5a3f122640 1 -- 172.21.15.60:0/1510715055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a380c9dc0 con 0x7f5a3815ee60 2024-04-25T10:15:20.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f5a357fa640 1 -- 172.21.15.60:0/1510715055 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c0bb410 con 0x7f5a3815ee60 2024-04-25T10:15:20.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f5a3f122640 1 -- 172.21.15.60:0/1510715055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a380ca380 con 0x7f5a3815ee60 2024-04-25T10:15:20.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f06ee401000 1 -- 172.21.15.60:0/370714689 shutdown_connections 2024-04-25T10:15:20.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f06ee401000 1 --2- 172.21.15.60:0/370714689 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x556b1f376180 0x556b1f3c5410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f06ee401000 1 --2- 172.21.15.60:0/370714689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f3b45b0 0x556b1f3758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f06ee401000 1 --2- 172.21.15.60:0/370714689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f3b3c90 0x556b1f375370 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f06c3fff640 0 client.9138 ms_handle_reset on v2:172.21.15.60:6834/4149609651 2024-04-25T10:15:20.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f06ee401000 1 --2- 172.21.15.60:0/370714689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f36a290 0x556b1f374e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f06ee401000 1 -- 172.21.15.60:0/370714689 >> 172.21.15.60:0/370714689 conn(0x556b1f366240 msgr2=0x556b1f368ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:20.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f06ee401000 1 -- 172.21.15.60:0/370714689 shutdown_connections 2024-04-25T10:15:20.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f06ee401000 1 -- 172.21.15.60:0/370714689 wait complete. 2024-04-25T10:15:20.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f06ee401000 20 client.9138 trim_cache size 0 max 16384 2024-04-25T10:15:20.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f5a3f122640 1 -- 172.21.15.60:0/1510715055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f5a38074770 con 0x7f5a3815ee60 2024-04-25T10:15:20.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f5a357fa640 1 -- 172.21.15.60:0/1510715055 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a2c0c2060 con 0x7f5a3815ee60 2024-04-25T10:15:20.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f5a357fa640 1 --2- 172.21.15.60:0/1510715055 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5a1c066f60 0x7f5a1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f5a357fa640 1 -- 172.21.15.60:0/1510715055 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5a2c122720 con 0x7f5a3815ee60 2024-04-25T10:15:20.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f5a357fa640 1 -- 172.21.15.60:0/1510715055 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+516 (secure 0 0 0) 0x7f5a2c0ca020 con 0x7f5a3815ee60 2024-04-25T10:15:20.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f5a3cebe640 1 --2- 172.21.15.60:0/1510715055 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5a1c066f60 0x7f5a1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f5a3f122640 1 -- 172.21.15.60:0/1510715055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f5a380b4070 con 0x7f5a3815ee60 2024-04-25T10:15:20.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.046+0000 7f5a3cebe640 1 --2- 172.21.15.60:0/1510715055 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5a1c066f60 0x7f5a1c069420 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5a3815e450 tx=0x7f5a24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.058+0000 7f06ee401000 1 Processor -- start 2024-04-25T10:15:20.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 -- start start 2024-04-25T10:15:20.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f36a290 0x556b1f369640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f3b3c90 0x556b1f369b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f3b45b0 0x556b1f3c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e42009e0 con 0x556b1f3b45b0 2024-04-25T10:15:20.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e41ffba0 con 0x556b1f36a290 2024-04-25T10:15:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e40cd500 con 0x556b1f3b3c90 2024-04-25T10:15:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06e9831640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f3b45b0 0x556b1f3c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06e9831640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f3b45b0 0x556b1f3c5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55066/0 (socket says 172.21.15.60:55066) 2024-04-25T10:15:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06e9831640 1 -- 172.21.15.60:0/3866903779 learned_addr learned my addr 172.21.15.60:0/3866903779 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06e9030640 1 --2- 172.21.15.60:0/3866903779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f36a290 0x556b1f369640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06e9831640 1 -- 172.21.15.60:0/3866903779 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f3b3c90 msgr2=0x556b1f369b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06e9831640 1 --2- 172.21.15.60:0/3866903779 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f3b3c90 0x556b1f369b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06e9831640 1 -- 172.21.15.60:0/3866903779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f36a290 msgr2=0x556b1f369640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06e9831640 1 --2- 172.21.15.60:0/3866903779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f36a290 0x556b1f369640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06e9831640 1 -- 172.21.15.60:0/3866903779 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b1f20fc50 con 0x556b1f3b45b0 2024-04-25T10:15:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06e9831640 1 --2- 172.21.15.60:0/3866903779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f3b45b0 0x556b1f3c5410 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f06dc013000 tx=0x7f06dc00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03e80fd6e16ca58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ce7fc640 1 -- 172.21.15.60:0/3866903779 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc023070 con 0x556b1f3b45b0 2024-04-25T10:15:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ce7fc640 1 -- 172.21.15.60:0/3866903779 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06dc003c50 con 0x556b1f3b45b0 2024-04-25T10:15:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ce7fc640 1 -- 172.21.15.60:0/3866903779 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc019040 con 0x556b1f3b45b0 2024-04-25T10:15:20.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 -- 172.21.15.60:0/3866903779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f3b45b0 msgr2=0x556b1f3c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 --2- 172.21.15.60:0/3866903779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f3b45b0 0x556b1f3c5410 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f06dc013000 tx=0x7f06dc00a000 comp rx=0 tx=0).stop 2024-04-25T10:15:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 -- 172.21.15.60:0/3866903779 shutdown_connections 2024-04-25T10:15:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 --2- 172.21.15.60:0/3866903779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f3b45b0 0x556b1f3c5410 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 --2- 172.21.15.60:0/3866903779 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f3b3c90 0x556b1f369b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 --2- 172.21.15.60:0/3866903779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f36a290 0x556b1f369640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 -- 172.21.15.60:0/3866903779 >> 172.21.15.60:0/3866903779 conn(0x556b1f366240 msgr2=0x556b1f36ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 -- 172.21.15.60:0/3866903779 shutdown_connections 2024-04-25T10:15:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 -- 172.21.15.60:0/3866903779 wait complete. 2024-04-25T10:15:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 -- start start 2024-04-25T10:15:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 Processor -- start 2024-04-25T10:15:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f36a290 0x556b1f414260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f3b3c90 0x556b1f4147a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f3b45b0 0x556b1f3cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b1f15abd0 con 0x556b1f36a290 2024-04-25T10:15:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e41ff5a0 con 0x556b1f3b45b0 2024-04-25T10:15:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b1f1b4110 con 0x556b1f3b3c90 2024-04-25T10:15:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06ee401000 10 client.0 _ll_register_callbacks cb 0x7f06dc001710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06e9030640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f36a290 0x556b1f414260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06e9030640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f36a290 0x556b1f414260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55078/0 (socket says 172.21.15.60:55078) 2024-04-25T10:15:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.062+0000 7f06e9030640 1 -- 172.21.15.60:0/3353662070 learned_addr learned my addr 172.21.15.60:0/3353662070 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06e9831640 1 --2- 172.21.15.60:0/3353662070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f3b45b0 0x556b1f3cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06e882f640 1 --2- 172.21.15.60:0/3353662070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f3b3c90 0x556b1f4147a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06e9030640 1 -- 172.21.15.60:0/3353662070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f3b3c90 msgr2=0x556b1f4147a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06e9030640 1 --2- 172.21.15.60:0/3353662070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f3b3c90 0x556b1f4147a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06e9030640 1 -- 172.21.15.60:0/3353662070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f3b45b0 msgr2=0x556b1f3cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06e9030640 1 --2- 172.21.15.60:0/3353662070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f3b45b0 0x556b1f3cee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06e9030640 1 -- 172.21.15.60:0/3353662070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b1f180120 con 0x556b1f36a290 2024-04-25T10:15:20.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06e9030640 1 --2- 172.21.15.60:0/3353662070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f36a290 0x556b1f414260 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f06e001abe0 tx=0x7f06e00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e877e4fb0cd86225 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:15:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06c3fff640 1 -- 172.21.15.60:0/3353662070 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e00bcb40 con 0x556b1f36a290 2024-04-25T10:15:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06c3fff640 1 -- 172.21.15.60:0/3353662070 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f06e00bcce0 con 0x556b1f36a290 2024-04-25T10:15:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 1 -- 172.21.15.60:0/3353662070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06dc003c50 con 0x556b1f36a290 2024-04-25T10:15:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06c3fff640 1 -- 172.21.15.60:0/3353662070 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06e00dc050 con 0x556b1f36a290 2024-04-25T10:15:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 10 client.9147 fetch_fsmap learned FSMap version 83 2024-04-25T10:15:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 1 -- 172.21.15.60:0/3353662070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x556b1f15f170 con 0x556b1f36a290 2024-04-25T10:15:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06c3fff640 1 -- 172.21.15.60:0/3353662070 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06dc003c50 con 0x556b1f36a290 2024-04-25T10:15:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06c3fff640 1 -- 172.21.15.60:0/3353662070 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f06e00df020 con 0x556b1f36a290 2024-04-25T10:15:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 10 client.9147 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:15:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 10 client.9147 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:15:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 20 client.9147 populate_metadata read hostname 'smithi060' 2024-04-25T10:15:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 1 --2- 172.21.15.60:0/3353662070 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x556b1f3cfb10 0x556b1f3cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 4 client.9147 mds_command: new command op to 9048 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:15:20.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 1 -- 172.21.15.60:0/3353662070 --> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x556b1f1c55b0 con 0x556b1f3cfb10 2024-04-25T10:15:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06e882f640 1 --2- 172.21.15.60:0/3353662070 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x556b1f3cfb10 0x556b1f3cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06c3fff640 10 client.9147 ms_handle_connect on v2:172.21.15.60:6834/4149609651 2024-04-25T10:15:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06e882f640 1 --2- 172.21.15.60:0/3353662070 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x556b1f3cfb10 0x556b1f3cff20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40e171c91e660840 server_cookie=eb114679ca124ee0 in_seq=0 out_seq=0 2024-04-25T10:15:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06c3fff640 1 -- 172.21.15.60:0/3353662070 <== mds.0 v2:172.21.15.60:6834/4149609651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556b1f1c55b0 con 0x556b1f3cfb10 2024-04-25T10:15:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06c3fff640 10 client.9147 handle_command_reply: tid=0 2024-04-25T10:15:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 1 client.9147 shutdown 2024-04-25T10:15:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 1 -- 172.21.15.60:0/3353662070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f36a290 msgr2=0x556b1f414260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 1 --2- 172.21.15.60:0/3353662070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f36a290 0x556b1f414260 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f06e001abe0 tx=0x7f06e00046d0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.078 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f1b7b02b640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:15:20.078 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f1b7b02b640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-25T10:15:20.078 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f1b7b02b640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:15:20.078 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f1b7b02b640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-25T10:15:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 1 -- 172.21.15.60:0/3353662070 shutdown_connections 2024-04-25T10:15:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 1 --2- 172.21.15.60:0/3353662070 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x556b1f3cfb10 0x556b1f3cff20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 1 --2- 172.21.15.60:0/3353662070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556b1f3b45b0 0x556b1f3cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06c3fff640 0 client.9147 ms_handle_reset on v2:172.21.15.60:6834/4149609651 2024-04-25T10:15:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 1 --2- 172.21.15.60:0/3353662070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556b1f3b3c90 0x556b1f4147a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 1 --2- 172.21.15.60:0/3353662070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556b1f36a290 0x556b1f414260 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 1 -- 172.21.15.60:0/3353662070 >> 172.21.15.60:0/3353662070 conn(0x556b1f366240 msgr2=0x556b1f36f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 1 -- 172.21.15.60:0/3353662070 shutdown_connections 2024-04-25T10:15:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 1 -- 172.21.15.60:0/3353662070 wait complete. 2024-04-25T10:15:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.066+0000 7f06ee401000 20 client.9147 trim_cache size 0 max 16384 2024-04-25T10:15:20.080 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:15:20.080 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T10:15:20.080 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "b44fa99b-1975-4459-ba7b-62e271f9d75f", 2024-04-25T10:15:20.080 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T10:15:20.080 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:15:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.070+0000 7f06eb294640 1 -- 172.21.15.60:0/3315939107 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f06c8066f60 msgr2=0x7f06c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.070+0000 7f06eb294640 1 --2- 172.21.15.60:0/3315939107 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f06c8066f60 0x7f06c8069420 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f06e00ae000 tx=0x7f06e00b3040 comp rx=0 tx=0).stop 2024-04-25T10:15:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.070+0000 7f06eb294640 1 -- 172.21.15.60:0/3315939107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06e4072620 msgr2=0x7f06e414e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.070+0000 7f06eb294640 1 --2- 172.21.15.60:0/3315939107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06e4072620 0x7f06e414e370 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f06d400c7c0 tx=0x7f06d400cc80 comp rx=0 tx=0).stop 2024-04-25T10:15:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.070+0000 7f06eb294640 1 -- 172.21.15.60:0/3315939107 shutdown_connections 2024-04-25T10:15:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.070+0000 7f06eb294640 1 --2- 172.21.15.60:0/3315939107 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f06c8066f60 0x7f06c8069420 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.070+0000 7f06eb294640 1 --2- 172.21.15.60:0/3315939107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06e415a790 0x7f06e40b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.070+0000 7f06eb294640 1 --2- 172.21.15.60:0/3315939107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06e4072620 0x7f06e414e370 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.070+0000 7f06eb294640 1 --2- 172.21.15.60:0/3315939107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06e406f280 0x7f06e414a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.070+0000 7f06eb294640 1 -- 172.21.15.60:0/3315939107 >> 172.21.15.60:0/3315939107 conn(0x7f06e4071b00 msgr2=0x7f06e41502e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:20.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.070+0000 7f06eb294640 1 -- 172.21.15.60:0/3315939107 shutdown_connections 2024-04-25T10:15:20.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.070+0000 7f06eb294640 1 -- 172.21.15.60:0/3315939107 wait complete. 2024-04-25T10:15:20.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:20.090 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:15:20.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f99067e2640 1 Processor -- start 2024-04-25T10:15:20.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f99067e2640 1 -- start start 2024-04-25T10:15:20.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f99067e2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99000acde0 0x7f990015c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f99067e2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9900160280 con 0x7f99000b4e40 2024-04-25T10:15:20.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f99067e2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990015bc20 con 0x7f99000af0d0 2024-04-25T10:15:20.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f99067e2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99000af550 con 0x7f99000acde0 2024-04-25T10:15:20.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98ff7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f99000b4e40 0x7f99000ad1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36702/0 (socket says 172.21.15.60:36702) 2024-04-25T10:15:20.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98ff7fe640 1 -- 172.21.15.60:0/1451378832 learned_addr learned my addr 172.21.15.60:0/1451378832 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:15:20.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98fffff640 1 --2- 172.21.15.60:0/1451378832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99000acde0 0x7f990015c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98feffd640 1 -- 172.21.15.60:0/1451378832 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348400755 0 0) 0x7f9900160280 con 0x7f99000b4e40 2024-04-25T10:15:20.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98feffd640 1 -- 172.21.15.60:0/1451378832 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98e8003530 con 0x7f99000b4e40 2024-04-25T10:15:20.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98feffd640 1 -- 172.21.15.60:0/1451378832 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 15510855 0 0) 0x7f98e8003530 con 0x7f99000b4e40 2024-04-25T10:15:20.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98feffd640 1 -- 172.21.15.60:0/1451378832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99000acde0 msgr2=0x7f990015c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98feffd640 1 --2- 172.21.15.60:0/1451378832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99000acde0 0x7f990015c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98feffd640 1 -- 172.21.15.60:0/1451378832 >> v1:172.21.15.100:6789/0 conn(0x7f99000af0d0 legacy=0x7f990006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98feffd640 1 -- 172.21.15.60:0/1451378832 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990015ffa0 con 0x7f99000b4e40 2024-04-25T10:15:20.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98feffd640 1 -- 172.21.15.60:0/1451378832 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f98f4001680 con 0x7f99000b4e40 2024-04-25T10:15:20.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98feffd640 1 -- 172.21.15.60:0/1451378832 >> v1:172.21.15.60:6789/0 conn(0x7f99000b4e40 legacy=0x7f99000ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98feffd640 1 --2- 172.21.15.60:0/1451378832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99000acde0 0x7f98e80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98feffd640 1 --2- 172.21.15.60:0/1451378832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e8003f30 0x7f98e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98feffd640 1 --2- 172.21.15.60:0/1451378832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e8004d50 0x7f98e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98feffd640 1 -- 172.21.15.60:0/1451378832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e800ba00 con 0x7f98e8004d50 2024-04-25T10:15:20.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98feffd640 1 -- 172.21.15.60:0/1451378832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e800bbb0 con 0x7f98e8003f30 2024-04-25T10:15:20.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98feffd640 1 -- 172.21.15.60:0/1451378832 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e800bdb0 con 0x7f99000acde0 2024-04-25T10:15:20.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f9904d7f640 1 --2- 172.21.15.60:0/1451378832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e8004d50 0x7f98e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98ff7fe640 1 --2- 172.21.15.60:0/1451378832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e8003f30 0x7f98e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f98fffff640 1 --2- 172.21.15.60:0/1451378832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99000acde0 0x7f98e80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f9904d7f640 1 -- 172.21.15.60:0/1451378832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99000acde0 msgr2=0x7f98e80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f9904d7f640 1 --2- 172.21.15.60:0/1451378832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99000acde0 0x7f98e80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f9904d7f640 1 -- 172.21.15.60:0/1451378832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e8003f30 msgr2=0x7f98e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f9904d7f640 1 --2- 172.21.15.60:0/1451378832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e8003f30 0x7f98e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.206+0000 7f9904d7f640 1 -- 172.21.15.60:0/1451378832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e8003530 con 0x7f98e8004d50 2024-04-25T10:15:20.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.210+0000 7f9904d7f640 1 --2- 172.21.15.60:0/1451378832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e8004d50 0x7f98e80051d0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f99000b52a0 tx=0x7f98f801c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c420d02c7c064c08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.210+0000 7f98feffd640 1 -- 172.21.15.60:0/1451378832 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f8001b60 con 0x7f98e8004d50 2024-04-25T10:15:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.210+0000 7f98feffd640 1 -- 172.21.15.60:0/1451378832 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f80a66b0 con 0x7f98e8004d50 2024-04-25T10:15:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.210+0000 7f98feffd640 1 -- 172.21.15.60:0/1451378832 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f80a6ae0 con 0x7f98e8004d50 2024-04-25T10:15:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.210+0000 7f99067e2640 1 -- 172.21.15.60:0/1451378832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e8004d50 msgr2=0x7f98e80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.210+0000 7f99067e2640 1 --2- 172.21.15.60:0/1451378832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e8004d50 0x7f98e80051d0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f99000b52a0 tx=0x7f98f801c840 comp rx=0 tx=0).stop 2024-04-25T10:15:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.210+0000 7f9904d7f640 1 -- 172.21.15.60:0/1451378832 reap_dead start 2024-04-25T10:15:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.210+0000 7f99067e2640 1 -- 172.21.15.60:0/1451378832 shutdown_connections 2024-04-25T10:15:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.210+0000 7f99067e2640 1 -- 172.21.15.60:0/1451378832 >> 172.21.15.60:0/1451378832 conn(0x7f99000cfdd0 msgr2=0x7f99000d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.210+0000 7f99067e2640 1 -- 172.21.15.60:0/1451378832 shutdown_connections 2024-04-25T10:15:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.210+0000 7f99067e2640 1 -- 172.21.15.60:0/1451378832 wait complete. 2024-04-25T10:15:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f99067e2640 1 Processor -- start 2024-04-25T10:15:20.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f99067e2640 1 -- start start 2024-04-25T10:15:20.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f99067e2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9900145b30 0x7f990006ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f99067e2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99001491e0 0x7f990006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f99067e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9900148c20 0x7f9900158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f99067e2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990015c1d0 con 0x7f9900145b30 2024-04-25T10:15:20.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f99067e2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990015fd50 con 0x7f9900148c20 2024-04-25T10:15:20.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f99067e2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990015c4b0 con 0x7f99001491e0 2024-04-25T10:15:20.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f98fffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9900145b30 0x7f990006ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f98fffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9900145b30 0x7f990006ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55104/0 (socket says 172.21.15.60:55104) 2024-04-25T10:15:20.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f98fffff640 1 -- 172.21.15.60:0/1007600105 learned_addr learned my addr 172.21.15.60:0/1007600105 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:20.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f9904d7f640 1 --2- 172.21.15.60:0/1007600105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9900148c20 0x7f9900158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f98ff7fe640 1 --2- 172.21.15.60:0/1007600105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99001491e0 0x7f990006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f98fffff640 1 -- 172.21.15.60:0/1007600105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99001491e0 msgr2=0x7f990006e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f98fffff640 1 --2- 172.21.15.60:0/1007600105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99001491e0 0x7f990006e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f98fffff640 1 -- 172.21.15.60:0/1007600105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9900148c20 msgr2=0x7f9900158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f98fffff640 1 --2- 172.21.15.60:0/1007600105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9900148c20 0x7f9900158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f98fffff640 1 -- 172.21.15.60:0/1007600105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99001f7b30 con 0x7f9900145b30 2024-04-25T10:15:20.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f98fffff640 1 --2- 172.21.15.60:0/1007600105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9900145b30 0x7f990006ddb0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f98f00115e0 tx=0x7f98f0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f98fcff9640 1 -- 172.21.15.60:0/1007600105 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0015870 con 0x7f9900145b30 2024-04-25T10:15:20.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f98fcff9640 1 -- 172.21.15.60:0/1007600105 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f0015a10 con 0x7f9900145b30 2024-04-25T10:15:20.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f99067e2640 1 -- 172.21.15.60:0/1007600105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99001f6fb0 con 0x7f9900145b30 2024-04-25T10:15:20.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f98fcff9640 1 -- 172.21.15.60:0/1007600105 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0015dc0 con 0x7f9900145b30 2024-04-25T10:15:20.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.214+0000 7f99067e2640 1 -- 172.21.15.60:0/1007600105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99001f7570 con 0x7f9900145b30 2024-04-25T10:15:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 Processor -- start 2024-04-25T10:15:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 -- start start 2024-04-25T10:15:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f98fcff9640 1 -- 172.21.15.60:0/1007600105 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98f000f480 con 0x7f9900145b30 2024-04-25T10:15:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3c9d0 0x5616c8a3cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c8a3d2f0 0x5616c89f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c89f30a0 0x5616c89f3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c883fff0 con 0x5616c8a3d2f0 2024-04-25T10:15:20.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c87c8150 con 0x5616c89f30a0 2024-04-25T10:15:20.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616c883c090 con 0x5616c8a3c9d0 2024-04-25T10:15:20.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f98fcff9640 1 --2- 172.21.15.60:0/1007600105 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f98ec066f90 0x7f98ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f98fcff9640 1 -- 172.21.15.60:0/1007600105 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f98f0085860 con 0x7f9900145b30 2024-04-25T10:15:20.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f98fffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c8a3d2f0 0x5616c89f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f98fffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c8a3d2f0 0x5616c89f2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55108/0 (socket says 172.21.15.60:55108) 2024-04-25T10:15:20.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f98fffff640 1 -- 172.21.15.60:0/1640704401 learned_addr learned my addr 172.21.15.60:0/1640704401 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:20.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f9904d7f640 1 --2- 172.21.15.60:0/1640704401 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3c9d0 0x5616c8a3cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f9904d7f640 1 --2- 172.21.15.60:0/1007600105 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f98ec066f90 0x7f98ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f98ff7fe640 1 --2- 172.21.15.60:0/1640704401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c89f30a0 0x5616c89f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f98fffff640 1 -- 172.21.15.60:0/1640704401 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3c9d0 msgr2=0x5616c8a3cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f98fffff640 1 --2- 172.21.15.60:0/1640704401 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3c9d0 0x5616c8a3cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f98fffff640 1 -- 172.21.15.60:0/1640704401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c89f30a0 msgr2=0x5616c89f3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f98fffff640 1 --2- 172.21.15.60:0/1640704401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c89f30a0 0x5616c89f3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f98fffff640 1 -- 172.21.15.60:0/1640704401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616c89d60d0 con 0x5616c8a3d2f0 2024-04-25T10:15:20.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f98fffff640 1 --2- 172.21.15.60:0/1640704401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c8a3d2f0 0x5616c89f2b60 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f98f0083d60 tx=0x7f98f0055e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fc072cc887da40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f98e5ffb640 1 -- 172.21.15.60:0/1640704401 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0082070 con 0x5616c8a3d2f0 2024-04-25T10:15:20.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f98e5ffb640 1 -- 172.21.15.60:0/1640704401 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f00544a0 con 0x5616c8a3d2f0 2024-04-25T10:15:20.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f98e5ffb640 1 -- 172.21.15.60:0/1640704401 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f005e420 con 0x5616c8a3d2f0 2024-04-25T10:15:20.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f9904d7f640 1 --2- 172.21.15.60:0/1007600105 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f98ec066f90 0x7f98ec069450 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f98f80af000 tx=0x7f98f80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 -- 172.21.15.60:0/1640704401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c8a3d2f0 msgr2=0x5616c89f2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 --2- 172.21.15.60:0/1640704401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c8a3d2f0 0x5616c89f2b60 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f98f0083d60 tx=0x7f98f0055e10 comp rx=0 tx=0).stop 2024-04-25T10:15:20.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 -- 172.21.15.60:0/1640704401 shutdown_connections 2024-04-25T10:15:20.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 --2- 172.21.15.60:0/1640704401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c89f30a0 0x5616c89f3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 --2- 172.21.15.60:0/1640704401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c8a3d2f0 0x5616c89f2b60 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 --2- 172.21.15.60:0/1640704401 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3c9d0 0x5616c8a3cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 -- 172.21.15.60:0/1640704401 >> 172.21.15.60:0/1640704401 conn(0x5616c89ef020 msgr2=0x5616c89f1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 -- 172.21.15.60:0/1640704401 shutdown_connections 2024-04-25T10:15:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.218+0000 7f990994f000 1 -- 172.21.15.60:0/1640704401 wait complete. 2024-04-25T10:15:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 -- start start 2024-04-25T10:15:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 Processor -- start 2024-04-25T10:15:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c89f30a0 0x5616c8a217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3c9d0 0x5616c8a21d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c8a3d2f0 0x5616c8a22270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c8804250 con 0x5616c89f30a0 2024-04-25T10:15:20.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c87fa690 con 0x5616c8a3d2f0 2024-04-25T10:15:20.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616c87c8a40 con 0x5616c8a3c9d0 2024-04-25T10:15:20.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 10 client.0 _ll_register_callbacks cb 0x5616c89dde90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:20.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98fffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c89f30a0 0x5616c8a217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98fffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c89f30a0 0x5616c8a217f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55116/0 (socket says 172.21.15.60:55116) 2024-04-25T10:15:20.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98fffff640 1 -- 172.21.15.60:0/1693953741 learned_addr learned my addr 172.21.15.60:0/1693953741 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:20.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f9904d7f640 1 --2- 172.21.15.60:0/1693953741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c8a3d2f0 0x5616c8a22270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98fffff640 1 -- 172.21.15.60:0/1693953741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3c9d0 msgr2=0x5616c8a21d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98ff7fe640 1 --2- 172.21.15.60:0/1693953741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3c9d0 0x5616c8a21d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98fffff640 1 --2- 172.21.15.60:0/1693953741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3c9d0 0x5616c8a21d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98fffff640 1 -- 172.21.15.60:0/1693953741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c8a3d2f0 msgr2=0x5616c8a22270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98fffff640 1 --2- 172.21.15.60:0/1693953741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c8a3d2f0 0x5616c8a22270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98fffff640 1 -- 172.21.15.60:0/1693953741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616c88ed1c0 con 0x5616c89f30a0 2024-04-25T10:15:20.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98fffff640 1 --2- 172.21.15.60:0/1693953741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c89f30a0 0x5616c8a217f0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f98f0012010 tx=0x7f98f0011fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79be756f2727c5db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98deffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:15:20.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98deffd640 1 -- 172.21.15.60:0/1693953741 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0082040 con 0x5616c89f30a0 2024-04-25T10:15:20.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98deffd640 1 -- 172.21.15.60:0/1693953741 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f98f0075cf0 con 0x5616c89f30a0 2024-04-25T10:15:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 -- 172.21.15.60:0/1693953741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98f00544a0 con 0x5616c89f30a0 2024-04-25T10:15:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98deffd640 1 -- 172.21.15.60:0/1693953741 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98f0052440 con 0x5616c89f30a0 2024-04-25T10:15:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 10 client.9159 fetch_fsmap learned FSMap version 83 2024-04-25T10:15:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 -- 172.21.15.60:0/1693953741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x5616c88e9170 con 0x5616c89f30a0 2024-04-25T10:15:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98deffd640 1 -- 172.21.15.60:0/1693953741 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f00544a0 con 0x5616c89f30a0 2024-04-25T10:15:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98deffd640 1 -- 172.21.15.60:0/1693953741 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f98f005e420 con 0x5616c89f30a0 2024-04-25T10:15:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 10 client.9159 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:15:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 10 client.9159 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:15:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 20 client.9159 populate_metadata read hostname 'smithi060' 2024-04-25T10:15:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 --2- 172.21.15.60:0/1693953741 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x5616c8a4cc80 0x5616c8a4f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 4 client.9159 mds_command: new command op to 9048 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:20.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 -- 172.21.15.60:0/1693953741 --> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616c87dbc60 con 0x5616c8a4cc80 2024-04-25T10:15:20.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98ff7fe640 1 --2- 172.21.15.60:0/1693953741 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x5616c8a4cc80 0x5616c8a4f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98ff7fe640 1 --2- 172.21.15.60:0/1693953741 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x5616c8a4cc80 0x5616c8a4f070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab963e597b5ef8d server_cookie=d3695aa185d84087 in_seq=0 out_seq=0 2024-04-25T10:15:20.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98deffd640 10 client.9159 ms_handle_connect on v2:172.21.15.60:6834/4149609651 2024-04-25T10:15:20.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98deffd640 1 -- 172.21.15.60:0/1693953741 <== mds.0 v2:172.21.15.60:6834/4149609651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5616c87dbc60 con 0x5616c8a4cc80 2024-04-25T10:15:20.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98deffd640 10 client.9159 handle_command_reply: tid=0 2024-04-25T10:15:20.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 client.9159 shutdown 2024-04-25T10:15:20.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 -- 172.21.15.60:0/1693953741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c89f30a0 msgr2=0x5616c8a217f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 --2- 172.21.15.60:0/1693953741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c89f30a0 0x5616c8a217f0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f98f0012010 tx=0x7f98f0011fa0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 -- 172.21.15.60:0/1693953741 shutdown_connections 2024-04-25T10:15:20.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 --2- 172.21.15.60:0/1693953741 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x5616c8a4cc80 0x5616c8a4f070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 --2- 172.21.15.60:0/1693953741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c8a3d2f0 0x5616c8a22270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 --2- 172.21.15.60:0/1693953741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3c9d0 0x5616c8a21d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f98deffd640 0 client.9159 ms_handle_reset on v2:172.21.15.60:6834/4149609651 2024-04-25T10:15:20.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 --2- 172.21.15.60:0/1693953741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c89f30a0 0x5616c8a217f0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 -- 172.21.15.60:0/1693953741 >> 172.21.15.60:0/1693953741 conn(0x5616c89ef020 msgr2=0x5616c89f1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:20.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 -- 172.21.15.60:0/1693953741 shutdown_connections 2024-04-25T10:15:20.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 1 -- 172.21.15.60:0/1693953741 wait complete. 2024-04-25T10:15:20.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.222+0000 7f990994f000 20 client.9159 trim_cache size 0 max 16384 2024-04-25T10:15:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.238+0000 7f990994f000 1 Processor -- start 2024-04-25T10:15:20.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.238+0000 7f990994f000 1 -- start start 2024-04-25T10:15:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.238+0000 7f990994f000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c89f30a0 0x5616c8a98ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.238+0000 7f990994f000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c8a3c9d0 0x5616c8a99420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.238+0000 7f990994f000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3d2f0 0x5616c8a99960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.238+0000 7f990994f000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9900200620 con 0x5616c89f30a0 2024-04-25T10:15:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.238+0000 7f990994f000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99001ff7a0 con 0x5616c8a3c9d0 2024-04-25T10:15:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.238+0000 7f990994f000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9900155b50 con 0x5616c8a3d2f0 2024-04-25T10:15:20.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.238+0000 7f98fffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c89f30a0 0x5616c8a98ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.238+0000 7f98fffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c89f30a0 0x5616c8a98ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55118/0 (socket says 172.21.15.60:55118) 2024-04-25T10:15:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.238+0000 7f98fffff640 1 -- 172.21.15.60:0/921093705 learned_addr learned my addr 172.21.15.60:0/921093705 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.238+0000 7f98ff7fe640 1 --2- 172.21.15.60:0/921093705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c8a3c9d0 0x5616c8a99420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.238+0000 7f98fffff640 1 -- 172.21.15.60:0/921093705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3d2f0 msgr2=0x5616c8a99960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98fffff640 1 --2- 172.21.15.60:0/921093705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3d2f0 0x5616c8a99960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98fffff640 1 -- 172.21.15.60:0/921093705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c8a3c9d0 msgr2=0x5616c8a99420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98fffff640 1 --2- 172.21.15.60:0/921093705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c8a3c9d0 0x5616c8a99420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98fffff640 1 -- 172.21.15.60:0/921093705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616c8886de0 con 0x5616c89f30a0 2024-04-25T10:15:20.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98ff7fe640 1 --2- 172.21.15.60:0/921093705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c8a3c9d0 0x5616c8a99420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:20.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98fffff640 1 --2- 172.21.15.60:0/921093705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c89f30a0 0x5616c8a98ee0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f98f0008010 tx=0x7f98f0083000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab9583f31be7b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98e57fa640 1 -- 172.21.15.60:0/921093705 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0082070 con 0x5616c89f30a0 2024-04-25T10:15:20.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98e57fa640 1 -- 172.21.15.60:0/921093705 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f007d4c0 con 0x5616c89f30a0 2024-04-25T10:15:20.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98e57fa640 1 -- 172.21.15.60:0/921093705 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f00544a0 con 0x5616c89f30a0 2024-04-25T10:15:20.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 -- 172.21.15.60:0/921093705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c89f30a0 msgr2=0x5616c8a98ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 --2- 172.21.15.60:0/921093705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c89f30a0 0x5616c8a98ee0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f98f0008010 tx=0x7f98f0083000 comp rx=0 tx=0).stop 2024-04-25T10:15:20.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 -- 172.21.15.60:0/921093705 shutdown_connections 2024-04-25T10:15:20.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 --2- 172.21.15.60:0/921093705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3d2f0 0x5616c8a99960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 --2- 172.21.15.60:0/921093705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c8a3c9d0 0x5616c8a99420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 --2- 172.21.15.60:0/921093705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c89f30a0 0x5616c8a98ee0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 -- 172.21.15.60:0/921093705 >> 172.21.15.60:0/921093705 conn(0x5616c89ef020 msgr2=0x5616c8a49fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:20.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 -- 172.21.15.60:0/921093705 shutdown_connections 2024-04-25T10:15:20.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 -- 172.21.15.60:0/921093705 wait complete. 2024-04-25T10:15:20.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 -- start start 2024-04-25T10:15:20.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 Processor -- start 2024-04-25T10:15:20.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c89f30a0 0x5616c8a9cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3c9d0 0x5616c8a9d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c8a3d2f0 0x5616c8a4efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616c87dae00 con 0x5616c8a3d2f0 2024-04-25T10:15:20.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99001ff1a0 con 0x5616c89f30a0 2024-04-25T10:15:20.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616c883f300 con 0x5616c8a3c9d0 2024-04-25T10:15:20.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 10 client.0 _ll_register_callbacks cb 0x5616c89be040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:20.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f9904d7f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c8a3d2f0 0x5616c8a4efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f9904d7f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c8a3d2f0 0x5616c8a4efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55120/0 (socket says 172.21.15.60:55120) 2024-04-25T10:15:20.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f9904d7f640 1 -- 172.21.15.60:0/4209745011 learned_addr learned my addr 172.21.15.60:0/4209745011 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:20.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98fffff640 1 --2- 172.21.15.60:0/4209745011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c89f30a0 0x5616c8a9cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98ff7fe640 1 --2- 172.21.15.60:0/4209745011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3c9d0 0x5616c8a9d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f9904d7f640 1 -- 172.21.15.60:0/4209745011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3c9d0 msgr2=0x5616c8a9d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f9904d7f640 1 --2- 172.21.15.60:0/4209745011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3c9d0 0x5616c8a9d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f9904d7f640 1 -- 172.21.15.60:0/4209745011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c89f30a0 msgr2=0x5616c8a9cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f9904d7f640 1 --2- 172.21.15.60:0/4209745011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c89f30a0 0x5616c8a9cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f9904d7f640 1 -- 172.21.15.60:0/4209745011 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616c87e18e0 con 0x5616c8a3d2f0 2024-04-25T10:15:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98fffff640 1 --2- 172.21.15.60:0/4209745011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c89f30a0 0x5616c8a9cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f9904d7f640 1 --2- 172.21.15.60:0/4209745011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c8a3d2f0 0x5616c8a4efc0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f98f80b0040 tx=0x7f98f80195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9811308640cf41e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98deffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:15:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98deffd640 1 -- 172.21.15.60:0/4209745011 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f8003da0 con 0x5616c8a3d2f0 2024-04-25T10:15:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98deffd640 1 -- 172.21.15.60:0/4209745011 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f98f8001b40 con 0x5616c8a3d2f0 2024-04-25T10:15:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 -- 172.21.15.60:0/4209745011 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98f007d4c0 con 0x5616c8a3d2f0 2024-04-25T10:15:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98deffd640 1 -- 172.21.15.60:0/4209745011 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98f80a64b0 con 0x5616c8a3d2f0 2024-04-25T10:15:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 10 client.9165 fetch_fsmap learned FSMap version 83 2024-04-25T10:15:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 -- 172.21.15.60:0/4209745011 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x5616c88998f0 con 0x5616c8a3d2f0 2024-04-25T10:15:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98deffd640 1 -- 172.21.15.60:0/4209745011 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f007d4c0 con 0x5616c8a3d2f0 2024-04-25T10:15:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98deffd640 1 -- 172.21.15.60:0/4209745011 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f98f80a6650 con 0x5616c8a3d2f0 2024-04-25T10:15:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 10 client.9165 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:15:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 10 client.9165 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:15:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 20 client.9165 populate_metadata read hostname 'smithi060' 2024-04-25T10:15:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 --2- 172.21.15.60:0/4209745011 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x5616c8a4fb40 0x5616c8a4ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 4 client.9165 mds_command: new command op to 9048 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:15:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f990994f000 1 -- 172.21.15.60:0/4209745011 --> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5616c884d530 con 0x5616c8a4fb40 2024-04-25T10:15:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.242+0000 7f98fffff640 1 --2- 172.21.15.60:0/4209745011 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x5616c8a4fb40 0x5616c8a4ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f98fffff640 1 --2- 172.21.15.60:0/4209745011 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x5616c8a4fb40 0x5616c8a4ff50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=374dd2834f38e08a server_cookie=d9f997353a89860b in_seq=0 out_seq=0 2024-04-25T10:15:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f98deffd640 10 client.9165 ms_handle_connect on v2:172.21.15.60:6834/4149609651 2024-04-25T10:15:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f98deffd640 1 -- 172.21.15.60:0/4209745011 <== mds.0 v2:172.21.15.60:6834/4149609651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x5616c884d530 con 0x5616c8a4fb40 2024-04-25T10:15:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f98deffd640 10 client.9165 handle_command_reply: tid=0 2024-04-25T10:15:20.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f990994f000 1 client.9165 shutdown 2024-04-25T10:15:20.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f990994f000 1 -- 172.21.15.60:0/4209745011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c8a3d2f0 msgr2=0x5616c8a4efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f990994f000 1 --2- 172.21.15.60:0/4209745011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c8a3d2f0 0x5616c8a4efc0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f98f80b0040 tx=0x7f98f80195e0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f990994f000 1 -- 172.21.15.60:0/4209745011 shutdown_connections 2024-04-25T10:15:20.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f990994f000 1 --2- 172.21.15.60:0/4209745011 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x5616c8a4fb40 0x5616c8a4ff50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f990994f000 1 --2- 172.21.15.60:0/4209745011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5616c8a3d2f0 0x5616c8a4efc0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f990994f000 1 --2- 172.21.15.60:0/4209745011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5616c8a3c9d0 0x5616c8a9d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f98deffd640 0 client.9165 ms_handle_reset on v2:172.21.15.60:6834/4149609651 2024-04-25T10:15:20.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f990994f000 1 --2- 172.21.15.60:0/4209745011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616c89f30a0 0x5616c8a9cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f990994f000 1 -- 172.21.15.60:0/4209745011 >> 172.21.15.60:0/4209745011 conn(0x5616c89ef020 msgr2=0x5616c8a3b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:20.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f990994f000 1 -- 172.21.15.60:0/4209745011 shutdown_connections 2024-04-25T10:15:20.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f990994f000 1 -- 172.21.15.60:0/4209745011 wait complete. 2024-04-25T10:15:20.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f990994f000 20 client.9165 trim_cache size 0 max 16384 2024-04-25T10:15:20.258 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:15:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:15:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T10:15:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "b44fa99b-1975-4459-ba7b-62e271f9d75f": { 2024-04-25T10:15:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-04-25T10:15:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "b44fa99b-1975-4459-ba7b-62e271f9d75f", 2024-04-25T10:15:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,repair" 2024-04-25T10:15:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:15:20.259 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:15:20.259 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:15:20.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f99067e2640 1 -- 172.21.15.60:0/1007600105 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f98ec066f90 msgr2=0x7f98ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f99067e2640 1 --2- 172.21.15.60:0/1007600105 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f98ec066f90 0x7f98ec069450 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f98f80af000 tx=0x7f98f80a7000 comp rx=0 tx=0).stop 2024-04-25T10:15:20.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f99067e2640 1 -- 172.21.15.60:0/1007600105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9900145b30 msgr2=0x7f990006ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f99067e2640 1 --2- 172.21.15.60:0/1007600105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9900145b30 0x7f990006ddb0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f98f00115e0 tx=0x7f98f0011aa0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f99067e2640 1 -- 172.21.15.60:0/1007600105 shutdown_connections 2024-04-25T10:15:20.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f99067e2640 1 --2- 172.21.15.60:0/1007600105 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f98ec066f90 0x7f98ec069450 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f99067e2640 1 --2- 172.21.15.60:0/1007600105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9900148c20 0x7f9900158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f99067e2640 1 --2- 172.21.15.60:0/1007600105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99001491e0 0x7f990006e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f99067e2640 1 --2- 172.21.15.60:0/1007600105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9900145b30 0x7f990006ddb0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f99067e2640 1 -- 172.21.15.60:0/1007600105 >> 172.21.15.60:0/1007600105 conn(0x7f99000cfdd0 msgr2=0x7f99000cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:20.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f99067e2640 1 -- 172.21.15.60:0/1007600105 shutdown_connections 2024-04-25T10:15:20.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.246+0000 7f99067e2640 1 -- 172.21.15.60:0/1007600105 wait complete. 2024-04-25T10:15:20.267 INFO:tasks.cephfs.filesystem:scrub status for tag:b44fa99b-1975-4459-ba7b-62e271f9d75f - {'path': '/', 'tag': 'b44fa99b-1975-4459-ba7b-62e271f9d75f', 'options': 'recursive,repair'} 2024-04-25T10:15:20.433 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T10:15:20.426+0000 7f1b7f033640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:15:20.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:20.750+0000 7f5a357fa640 1 -- 172.21.15.60:0/1510715055 <== mon.0 v2:172.21.15.60:3300/0 7 ==== log(8 entries from seq 1 at 2024-04-25T10:15:20.069640+0000) v1 ==== 1407+0+0 (secure 0 0 0) 0x7f5a2c0a89d0 con 0x7f5a3815ee60 2024-04-25T10:15:20.755 INFO:teuthology.orchestra.run.smithi060.stdout: cluster: 2024-04-25T10:15:20.755 INFO:teuthology.orchestra.run.smithi060.stdout: id: 0ea93863-a079-45c4-abed-6e83540ea9fe 2024-04-25T10:15:20.755 INFO:teuthology.orchestra.run.smithi060.stdout: health: HEALTH_OK 2024-04-25T10:15:20.755 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:15:20.755 INFO:teuthology.orchestra.run.smithi060.stdout: services: 2024-04-25T10:15:20.755 INFO:teuthology.orchestra.run.smithi060.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-04-25T10:15:20.755 INFO:teuthology.orchestra.run.smithi060.stdout: mgr: x(active, since 10m), standbys: y 2024-04-25T10:15:20.755 INFO:teuthology.orchestra.run.smithi060.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:15:20.755 INFO:teuthology.orchestra.run.smithi060.stdout: osd: 8 osds: 8 up (since 10m), 8 in (since 10m) 2024-04-25T10:15:20.755 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:15:20.755 INFO:teuthology.orchestra.run.smithi060.stdout: data: 2024-04-25T10:15:20.756 INFO:teuthology.orchestra.run.smithi060.stdout: volumes: 1/1 healthy 2024-04-25T10:15:20.756 INFO:teuthology.orchestra.run.smithi060.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:15:20.756 INFO:teuthology.orchestra.run.smithi060.stdout: objects: 30 objects, 19 MiB 2024-04-25T10:15:20.756 INFO:teuthology.orchestra.run.smithi060.stdout: usage: 306 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:15:20.756 INFO:teuthology.orchestra.run.smithi060.stdout: pgs: 97 active+clean 2024-04-25T10:15:20.756 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:15:20.756 INFO:teuthology.orchestra.run.smithi060.stdout: io: 2024-04-25T10:15:20.756 INFO:teuthology.orchestra.run.smithi060.stdout: client: 3.6 KiB/s rd, 102 B/s wr, 3 op/s rd, 0 op/s wr 2024-04-25T10:15:20.756 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:15:20.756 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:15:20.756 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:15:20.069640+0000 mds.a [INF] scrub queued for path: / 2024-04-25T10:15:20.756 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:15:20.069643+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:15:20.756 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:15:20.069647+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-25T10:15:20.756 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:15:20.070284+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:15:20.757 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:15:20.070292+0000 mds.a [ERR] inode table repaired for inode: 0x10000000001 2024-04-25T10:15:20.757 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:15:20.070365+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:15:20.757 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:15:20.070371+0000 mds.a [ERR] inode table repaired for inode: 0x10000000002 2024-04-25T10:15:20.757 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:15:20.071431+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:15:25.772 INFO:teuthology.orchestra.run.smithi060.stdout:2024-04-25T10:15:24.466893+0000 mds.a [INF] scrub summary: idle 2024-04-25T10:15:25.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:25.766+0000 7f5a357fa640 1 -- 172.21.15.60:0/1510715055 <== mon.0 v2:172.21.15.60:3300/0 8 ==== log(1 entries from seq 9 at 2024-04-25T10:15:24.466893+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f5a2c120d00 con 0x7f5a3815ee60 2024-04-25T10:15:47.870 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:47.873 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:50.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:50.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:15:50.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.402+0000 7f9dd3647640 1 Processor -- start 2024-04-25T10:15:50.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.402+0000 7f9dd3647640 1 -- start start 2024-04-25T10:15:50.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.402+0000 7f9dd3647640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9dcc15dd30 0x7f9dcc15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.402+0000 7f9dd3647640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9dcc0cf8a0 0x7f9dcc15be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.402+0000 7f9dd3647640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dcc0cd890 con 0x7f9dcc0cd3d0 2024-04-25T10:15:50.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.402+0000 7f9dd3647640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dcc0ce4a0 con 0x7f9dcc0cf8a0 2024-04-25T10:15:50.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.402+0000 7f9dd3647640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dcc0cdb70 con 0x7f9dcc15dd30 2024-04-25T10:15:50.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.402+0000 7f9dd1be4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9dcc0cf8a0 0x7f9dcc15be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.402+0000 7f9dd1be4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9dcc0cf8a0 0x7f9dcc15be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41538/0 (socket says 172.21.15.60:41538) 2024-04-25T10:15:50.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.402+0000 7f9dd1be4640 1 -- 172.21.15.60:0/1610037681 learned_addr learned my addr 172.21.15.60:0/1610037681 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:50.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.402+0000 7f9dd13e3640 1 --2- 172.21.15.60:0/1610037681 >> v2:172.21.15.100:3300/0 conn(0x7f9dcc15dd30 0x7f9dcc15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.402+0000 7f9dd1be4640 1 -- 172.21.15.60:0/1610037681 >> v2:172.21.15.100:3300/0 conn(0x7f9dcc15dd30 msgr2=0x7f9dcc15ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.402+0000 7f9dd1be4640 1 --2- 172.21.15.60:0/1610037681 >> v2:172.21.15.100:3300/0 conn(0x7f9dcc15dd30 0x7f9dcc15ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.402+0000 7f9dd1be4640 1 -- 172.21.15.60:0/1610037681 >> v1:172.21.15.100:6789/0 conn(0x7f9dcc0cd3d0 legacy=0x7f9dcc0cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.402+0000 7f9dd1be4640 1 -- 172.21.15.60:0/1610037681 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dcc0cde50 con 0x7f9dcc0cf8a0 2024-04-25T10:15:50.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.402+0000 7f9dd13e3640 1 --2- 172.21.15.60:0/1610037681 >> v2:172.21.15.100:3300/0 conn(0x7f9dcc15dd30 0x7f9dcc15ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:50.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd1be4640 1 --2- 172.21.15.60:0/1610037681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc0cf8a0 0x7f9dcc15be20 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc06bbf0 tx=0x7f9dc401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c4cbf6cd441651 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dc3fff640 1 -- 172.21.15.60:0/1610037681 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40a57d0 con 0x7f9dcc0cf8a0 2024-04-25T10:15:50.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dc3fff640 1 -- 172.21.15.60:0/1610037681 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc40a5970 con 0x7f9dcc0cf8a0 2024-04-25T10:15:50.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dc3fff640 1 -- 172.21.15.60:0/1610037681 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40a5d40 con 0x7f9dcc0cf8a0 2024-04-25T10:15:50.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd3647640 1 -- 172.21.15.60:0/1610037681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc0cf8a0 msgr2=0x7f9dcc15be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd3647640 1 --2- 172.21.15.60:0/1610037681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc0cf8a0 0x7f9dcc15be20 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc06bbf0 tx=0x7f9dc401c8f0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd3647640 1 -- 172.21.15.60:0/1610037681 shutdown_connections 2024-04-25T10:15:50.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd3647640 1 --2- 172.21.15.60:0/1610037681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc0cf8a0 0x7f9dcc15be20 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd3647640 1 --2- 172.21.15.60:0/1610037681 >> v2:172.21.15.100:3300/0 conn(0x7f9dcc15dd30 0x7f9dcc15ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd3647640 1 -- 172.21.15.60:0/1610037681 >> 172.21.15.60:0/1610037681 conn(0x7f9dcc0670b0 msgr2=0x7f9dcc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd3647640 1 -- 172.21.15.60:0/1610037681 shutdown_connections 2024-04-25T10:15:50.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd3647640 1 -- 172.21.15.60:0/1610037681 wait complete. 2024-04-25T10:15:50.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd3647640 1 Processor -- start 2024-04-25T10:15:50.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd3647640 1 -- start start 2024-04-25T10:15:50.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd3647640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dcc0cd3d0 0x7f9dcc159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd3647640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc0cf8a0 0x7f9dcc14aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd3647640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dcc15dd30 0x7f9dcc14b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd3647640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc15cf00 con 0x7f9dcc0cf8a0 2024-04-25T10:15:50.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd3647640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc15cd80 con 0x7f9dcc0cd3d0 2024-04-25T10:15:50.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd3647640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc15d080 con 0x7f9dcc15dd30 2024-04-25T10:15:50.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd0be2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc0cf8a0 0x7f9dcc14aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd0be2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc0cf8a0 0x7f9dcc14aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41540/0 (socket says 172.21.15.60:41540) 2024-04-25T10:15:50.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd0be2640 1 -- 172.21.15.60:0/2661294784 learned_addr learned my addr 172.21.15.60:0/2661294784 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:50.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd13e3640 1 --2- 172.21.15.60:0/2661294784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dcc0cd3d0 0x7f9dcc159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd0be2640 1 -- 172.21.15.60:0/2661294784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dcc15dd30 msgr2=0x7f9dcc14b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd0be2640 1 --2- 172.21.15.60:0/2661294784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dcc15dd30 0x7f9dcc14b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd0be2640 1 -- 172.21.15.60:0/2661294784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dcc0cd3d0 msgr2=0x7f9dcc159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd0be2640 1 --2- 172.21.15.60:0/2661294784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dcc0cd3d0 0x7f9dcc159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd0be2640 1 -- 172.21.15.60:0/2661294784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dcc14ca00 con 0x7f9dcc0cf8a0 2024-04-25T10:15:50.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd13e3640 1 --2- 172.21.15.60:0/2661294784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dcc0cd3d0 0x7f9dcc159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:50.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dd0be2640 1 --2- 172.21.15.60:0/2661294784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc0cf8a0 0x7f9dcc14aba0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00c480 tx=0x7f9dbc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.406+0000 7f9dc1ffb640 1 -- 172.21.15.60:0/2661294784 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc012830 con 0x7f9dcc0cf8a0 2024-04-25T10:15:50.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.410+0000 7f9dc1ffb640 1 -- 172.21.15.60:0/2661294784 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc017070 con 0x7f9dcc0cf8a0 2024-04-25T10:15:50.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.410+0000 7f9dc1ffb640 1 -- 172.21.15.60:0/2661294784 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc012c30 con 0x7f9dcc0cf8a0 2024-04-25T10:15:50.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.410+0000 7f9dd3647640 1 -- 172.21.15.60:0/2661294784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dcc14cce0 con 0x7f9dcc0cf8a0 2024-04-25T10:15:50.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.410+0000 7f9dd3647640 1 -- 172.21.15.60:0/2661294784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dcc14d2a0 con 0x7f9dcc0cf8a0 2024-04-25T10:15:50.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.410+0000 7f9dc1ffb640 1 -- 172.21.15.60:0/2661294784 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9dbc01c460 con 0x7f9dcc0cf8a0 2024-04-25T10:15:50.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.410+0000 7f9dc1ffb640 1 --2- 172.21.15.60:0/2661294784 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9db0066f60 0x7f9db0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.410+0000 7f9dc1ffb640 1 -- 172.21.15.60:0/2661294784 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9dbc084ed0 con 0x7f9dcc0cf8a0 2024-04-25T10:15:50.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.410+0000 7f9dd13e3640 1 --2- 172.21.15.60:0/2661294784 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9db0066f60 0x7f9db0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.410+0000 7f9dd67b4000 1 Processor -- start 2024-04-25T10:15:50.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.410+0000 7f9dd13e3640 1 --2- 172.21.15.60:0/2661294784 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9db0066f60 0x7f9db0069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc158200 tx=0x7f9dc8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.410+0000 7f9dd67b4000 1 -- start start 2024-04-25T10:15:50.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4f39d0 0x56474e4f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e4a9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4aa0a0 0x56474e4aa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474e2f6ff0 con 0x56474e4f42f0 2024-04-25T10:15:50.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474e27f150 con 0x56474e4f39d0 2024-04-25T10:15:50.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56474e2f3090 con 0x56474e4aa0a0 2024-04-25T10:15:50.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd0be2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e4a9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd0be2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e4a9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41546/0 (socket says 172.21.15.60:41546) 2024-04-25T10:15:50.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd0be2640 1 -- 172.21.15.60:0/483001617 learned_addr learned my addr 172.21.15.60:0/483001617 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:50.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd13e3640 1 --2- 172.21.15.60:0/483001617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4f39d0 0x56474e4f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd0be2640 1 -- 172.21.15.60:0/483001617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4aa0a0 msgr2=0x56474e4aa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd1be4640 1 --2- 172.21.15.60:0/483001617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4aa0a0 0x56474e4aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd0be2640 1 --2- 172.21.15.60:0/483001617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4aa0a0 0x56474e4aa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd0be2640 1 -- 172.21.15.60:0/483001617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4f39d0 msgr2=0x56474e4f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd0be2640 1 --2- 172.21.15.60:0/483001617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4f39d0 0x56474e4f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd0be2640 1 -- 172.21.15.60:0/483001617 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56474e48d0d0 con 0x56474e4f42f0 2024-04-25T10:15:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd13e3640 1 --2- 172.21.15.60:0/483001617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4f39d0 0x56474e4f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd0be2640 1 --2- 172.21.15.60:0/483001617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e4a9b60 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0554f0 tx=0x7f9dbc055fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3ef909b9a58f7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9daeffd640 1 -- 172.21.15.60:0/483001617 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc06d070 con 0x56474e4f42f0 2024-04-25T10:15:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9daeffd640 1 -- 172.21.15.60:0/483001617 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc05df00 con 0x56474e4f42f0 2024-04-25T10:15:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9daeffd640 1 -- 172.21.15.60:0/483001617 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc06d210 con 0x56474e4f42f0 2024-04-25T10:15:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 -- 172.21.15.60:0/483001617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 msgr2=0x56474e4a9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/483001617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e4a9b60 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0554f0 tx=0x7f9dbc055fb0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 -- 172.21.15.60:0/483001617 shutdown_connections 2024-04-25T10:15:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/483001617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4aa0a0 0x56474e4aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/483001617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e4a9b60 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/483001617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4f39d0 0x56474e4f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 -- 172.21.15.60:0/483001617 >> 172.21.15.60:0/483001617 conn(0x56474e4a6020 msgr2=0x56474e4a8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 -- 172.21.15.60:0/483001617 shutdown_connections 2024-04-25T10:15:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 -- 172.21.15.60:0/483001617 wait complete. 2024-04-25T10:15:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 -- start start 2024-04-25T10:15:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 Processor -- start 2024-04-25T10:15:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4aa0a0 0x56474e4d87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4f39d0 0x56474e4d8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4f42f0 0x56474e4d9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474e2bb250 con 0x56474e4aa0a0 2024-04-25T10:15:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474e2b1690 con 0x56474e4f42f0 2024-04-25T10:15:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56474e27fa40 con 0x56474e4f39d0 2024-04-25T10:15:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd67b4000 10 client.0 _ll_register_callbacks cb 0x56474e494e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd13e3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4aa0a0 0x56474e4d87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd13e3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4aa0a0 0x56474e4d87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41548/0 (socket says 172.21.15.60:41548) 2024-04-25T10:15:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd13e3640 1 -- 172.21.15.60:0/3459892196 learned_addr learned my addr 172.21.15.60:0/3459892196 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:50.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd1be4640 1 --2- 172.21.15.60:0/3459892196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4f42f0 0x56474e4d9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd13e3640 1 -- 172.21.15.60:0/3459892196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4f39d0 msgr2=0x56474e4d8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd13e3640 1 --2- 172.21.15.60:0/3459892196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4f39d0 0x56474e4d8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd13e3640 1 -- 172.21.15.60:0/3459892196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4f42f0 msgr2=0x56474e4d9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd13e3640 1 --2- 172.21.15.60:0/3459892196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4f42f0 0x56474e4d9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd13e3640 1 -- 172.21.15.60:0/3459892196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56474e3a41c0 con 0x56474e4aa0a0 2024-04-25T10:15:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9dd13e3640 1 --2- 172.21.15.60:0/3459892196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4aa0a0 0x56474e4d87f0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80082c0 tx=0x7f9dc8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dcb72a11879e287 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.414+0000 7f9da7fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:15:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9da7fff640 1 -- 172.21.15.60:0/3459892196 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8023020 con 0x56474e4aa0a0 2024-04-25T10:15:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9da7fff640 1 -- 172.21.15.60:0/3459892196 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9dc801f620 con 0x56474e4aa0a0 2024-04-25T10:15:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 1 -- 172.21.15.60:0/3459892196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9dbc05df00 con 0x56474e4aa0a0 2024-04-25T10:15:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9da7fff640 1 -- 172.21.15.60:0/3459892196 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dc802d050 con 0x56474e4aa0a0 2024-04-25T10:15:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 10 client.9177 fetch_fsmap learned FSMap version 83 2024-04-25T10:15:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 1 -- 172.21.15.60:0/3459892196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x56474e3a0170 con 0x56474e4aa0a0 2024-04-25T10:15:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9da7fff640 1 -- 172.21.15.60:0/3459892196 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc05df00 con 0x56474e4aa0a0 2024-04-25T10:15:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9da7fff640 1 -- 172.21.15.60:0/3459892196 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9dc8030020 con 0x56474e4aa0a0 2024-04-25T10:15:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 10 client.9177 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:15:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 10 client.9177 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:15:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 20 client.9177 populate_metadata read hostname 'smithi060' 2024-04-25T10:15:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/3459892196 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x56474e503c80 0x56474e506070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 4 client.9177 mds_command: new command op to 9048 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 1 -- 172.21.15.60:0/3459892196 --> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56474e292c60 con 0x56474e503c80 2024-04-25T10:15:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd0be2640 1 --2- 172.21.15.60:0/3459892196 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x56474e503c80 0x56474e506070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd0be2640 1 --2- 172.21.15.60:0/3459892196 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x56474e503c80 0x56474e506070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e712842944ffb30 server_cookie=e5b1f92b77f706d2 in_seq=0 out_seq=0 2024-04-25T10:15:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9da7fff640 10 client.9177 ms_handle_connect on v2:172.21.15.60:6834/4149609651 2024-04-25T10:15:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9da7fff640 1 -- 172.21.15.60:0/3459892196 <== mds.0 v2:172.21.15.60:6834/4149609651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56474e292c60 con 0x56474e503c80 2024-04-25T10:15:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9da7fff640 10 client.9177 handle_command_reply: tid=0 2024-04-25T10:15:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 1 client.9177 shutdown 2024-04-25T10:15:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 1 -- 172.21.15.60:0/3459892196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4aa0a0 msgr2=0x56474e4d87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/3459892196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4aa0a0 0x56474e4d87f0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80082c0 tx=0x7f9dc8004740 comp rx=0 tx=0).stop 2024-04-25T10:15:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 1 -- 172.21.15.60:0/3459892196 shutdown_connections 2024-04-25T10:15:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/3459892196 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x56474e503c80 0x56474e506070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/3459892196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4f42f0 0x56474e4d9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/3459892196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4f39d0 0x56474e4d8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/3459892196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4aa0a0 0x56474e4d87f0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 1 -- 172.21.15.60:0/3459892196 >> 172.21.15.60:0/3459892196 conn(0x56474e4a6020 msgr2=0x56474e4a8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9da7fff640 0 client.9177 ms_handle_reset on v2:172.21.15.60:6834/4149609651 2024-04-25T10:15:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 1 -- 172.21.15.60:0/3459892196 shutdown_connections 2024-04-25T10:15:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 1 -- 172.21.15.60:0/3459892196 wait complete. 2024-04-25T10:15:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.418+0000 7f9dd67b4000 20 client.9177 trim_cache size 0 max 16384 2024-04-25T10:15:50.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 Processor -- start 2024-04-25T10:15:50.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 -- start start 2024-04-25T10:15:50.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4aa0a0 0x56474e4a8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4f39d0 0x56474e4a8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e4a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc200a20 con 0x56474e4f42f0 2024-04-25T10:15:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc1ffc50 con 0x56474e4aa0a0 2024-04-25T10:15:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc150f10 con 0x56474e4f39d0 2024-04-25T10:15:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd1be4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e4a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd1be4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e4a8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41550/0 (socket says 172.21.15.60:41550) 2024-04-25T10:15:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd1be4640 1 -- 172.21.15.60:0/3978606131 learned_addr learned my addr 172.21.15.60:0/3978606131 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd13e3640 1 --2- 172.21.15.60:0/3978606131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4aa0a0 0x56474e4a8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd1be4640 1 -- 172.21.15.60:0/3978606131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4f39d0 msgr2=0x56474e4a8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd0be2640 1 --2- 172.21.15.60:0/3978606131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4f39d0 0x56474e4a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd1be4640 1 --2- 172.21.15.60:0/3978606131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4f39d0 0x56474e4a8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd1be4640 1 -- 172.21.15.60:0/3978606131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4aa0a0 msgr2=0x56474e4a8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd1be4640 1 --2- 172.21.15.60:0/3978606131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4aa0a0 0x56474e4a8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd1be4640 1 -- 172.21.15.60:0/3978606131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56474e33dde0 con 0x56474e4f42f0 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd13e3640 1 --2- 172.21.15.60:0/3978606131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4aa0a0 0x56474e4a8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd1be4640 1 --2- 172.21.15.60:0/3978606131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e4a8e90 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f9dc401a520 tx=0x7f9dc4004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22ba8f4b817ae4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dae7fc640 1 -- 172.21.15.60:0/3978606131 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40bd070 con 0x56474e4f42f0 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dae7fc640 1 -- 172.21.15.60:0/3978606131 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc40a57d0 con 0x56474e4f42f0 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dae7fc640 1 -- 172.21.15.60:0/3978606131 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40a5bc0 con 0x56474e4f42f0 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 -- 172.21.15.60:0/3978606131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 msgr2=0x56474e4a8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/3978606131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e4a8e90 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f9dc401a520 tx=0x7f9dc4004a50 comp rx=0 tx=0).stop 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 -- 172.21.15.60:0/3978606131 shutdown_connections 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/3978606131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e4a8e90 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/3978606131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4f39d0 0x56474e4a8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/3978606131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4aa0a0 0x56474e4a8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 -- 172.21.15.60:0/3978606131 >> 172.21.15.60:0/3978606131 conn(0x56474e4a6020 msgr2=0x56474e500fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 -- 172.21.15.60:0/3978606131 shutdown_connections 2024-04-25T10:15:50.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.434+0000 7f9dd67b4000 1 -- 172.21.15.60:0/3978606131 wait complete. 2024-04-25T10:15:50.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 -- start start 2024-04-25T10:15:50.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 Processor -- start 2024-04-25T10:15:50.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4aa0a0 0x56474e505fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4f39d0 0x56474e5064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e506a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474e291e00 con 0x56474e4f42f0 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc151510 con 0x56474e4aa0a0 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56474e2f6300 con 0x56474e4f39d0 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 10 client.0 _ll_register_callbacks cb 0x56474e475040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd1be4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e506a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd1be4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e506a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41558/0 (socket says 172.21.15.60:41558) 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd1be4640 1 -- 172.21.15.60:0/1028576574 learned_addr learned my addr 172.21.15.60:0/1028576574 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd13e3640 1 --2- 172.21.15.60:0/1028576574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4aa0a0 0x56474e505fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd1be4640 1 -- 172.21.15.60:0/1028576574 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4f39d0 msgr2=0x56474e5064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd1be4640 1 --2- 172.21.15.60:0/1028576574 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4f39d0 0x56474e5064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd1be4640 1 -- 172.21.15.60:0/1028576574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4aa0a0 msgr2=0x56474e505fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd1be4640 1 --2- 172.21.15.60:0/1028576574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4aa0a0 0x56474e505fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd1be4640 1 -- 172.21.15.60:0/1028576574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56474e2988e0 con 0x56474e4f42f0 2024-04-25T10:15:50.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd1be4640 1 --2- 172.21.15.60:0/1028576574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e506a20 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f9dc401d190 tx=0x7f9dc401b130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6290e582a3f655 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9da7fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:15:50.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9da7fff640 1 -- 172.21.15.60:0/1028576574 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40c6070 con 0x56474e4f42f0 2024-04-25T10:15:50.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9da7fff640 1 -- 172.21.15.60:0/1028576574 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9dc4003ba0 con 0x56474e4f42f0 2024-04-25T10:15:50.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 -- 172.21.15.60:0/1028576574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9dc40a57d0 con 0x56474e4f42f0 2024-04-25T10:15:50.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9da7fff640 1 -- 172.21.15.60:0/1028576574 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dc40d0050 con 0x56474e4f42f0 2024-04-25T10:15:50.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 10 client.9183 fetch_fsmap learned FSMap version 83 2024-04-25T10:15:50.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 -- 172.21.15.60:0/1028576574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x56474e3508f0 con 0x56474e4f42f0 2024-04-25T10:15:50.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9da7fff640 1 -- 172.21.15.60:0/1028576574 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc40a57d0 con 0x56474e4f42f0 2024-04-25T10:15:50.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9da7fff640 1 -- 172.21.15.60:0/1028576574 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9dc40d3020 con 0x56474e4f42f0 2024-04-25T10:15:50.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 10 client.9183 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:15:50.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 10 client.9183 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:15:50.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 20 client.9183 populate_metadata read hostname 'smithi060' 2024-04-25T10:15:50.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/1028576574 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x56474e507320 0x56474e564770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 4 client.9183 mds_command: new command op to 9048 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:15:50.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 -- 172.21.15.60:0/1028576574 --> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56474e304530 con 0x56474e507320 2024-04-25T10:15:50.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd13e3640 1 --2- 172.21.15.60:0/1028576574 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x56474e507320 0x56474e564770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9da7fff640 10 client.9183 ms_handle_connect on v2:172.21.15.60:6834/4149609651 2024-04-25T10:15:50.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd13e3640 1 --2- 172.21.15.60:0/1028576574 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x56474e507320 0x56474e564770 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=313641c2fd568125 server_cookie=36df3c0c98427cb2 in_seq=0 out_seq=0 2024-04-25T10:15:50.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9da7fff640 1 -- 172.21.15.60:0/1028576574 <== mds.0 v2:172.21.15.60:6834/4149609651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56474e304530 con 0x56474e507320 2024-04-25T10:15:50.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9da7fff640 10 client.9183 handle_command_reply: tid=0 2024-04-25T10:15:50.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 client.9183 shutdown 2024-04-25T10:15:50.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 -- 172.21.15.60:0/1028576574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 msgr2=0x56474e506a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/1028576574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e506a20 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f9dc401d190 tx=0x7f9dc401b130 comp rx=0 tx=0).stop 2024-04-25T10:15:50.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 -- 172.21.15.60:0/1028576574 shutdown_connections 2024-04-25T10:15:50.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/1028576574 >> [v2:172.21.15.60:6834/4149609651,v1:172.21.15.60:6835/4149609651] conn(0x56474e507320 0x56474e564770 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/1028576574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56474e4f42f0 0x56474e506a20 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/1028576574 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56474e4f39d0 0x56474e5064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 --2- 172.21.15.60:0/1028576574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56474e4aa0a0 0x56474e505fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9da7fff640 0 client.9183 ms_handle_reset on v2:172.21.15.60:6834/4149609651 2024-04-25T10:15:50.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 -- 172.21.15.60:0/1028576574 >> 172.21.15.60:0/1028576574 conn(0x56474e4a6020 msgr2=0x56474e4f20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 -- 172.21.15.60:0/1028576574 shutdown_connections 2024-04-25T10:15:50.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 1 -- 172.21.15.60:0/1028576574 wait complete. 2024-04-25T10:15:50.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.438+0000 7f9dd67b4000 20 client.9183 trim_cache size 0 max 16384 2024-04-25T10:15:50.451 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:15:50.451 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.442+0000 7f9dd3647640 1 -- 172.21.15.60:0/2661294784 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9db0066f60 msgr2=0x7f9db0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.442+0000 7f9dd3647640 1 --2- 172.21.15.60:0/2661294784 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9db0066f60 0x7f9db0069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc158200 tx=0x7f9dc8008040 comp rx=0 tx=0).stop 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.442+0000 7f9dd3647640 1 -- 172.21.15.60:0/2661294784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc0cf8a0 msgr2=0x7f9dcc14aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.442+0000 7f9dd3647640 1 --2- 172.21.15.60:0/2661294784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc0cf8a0 0x7f9dcc14aba0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00c480 tx=0x7f9dbc00c940 comp rx=0 tx=0).stop 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.442+0000 7f9dd3647640 1 -- 172.21.15.60:0/2661294784 shutdown_connections 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.442+0000 7f9dd3647640 1 --2- 172.21.15.60:0/2661294784 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9db0066f60 0x7f9db0069420 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.442+0000 7f9dd3647640 1 --2- 172.21.15.60:0/2661294784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dcc15dd30 0x7f9dcc14b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.442+0000 7f9dd3647640 1 --2- 172.21.15.60:0/2661294784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dcc0cf8a0 0x7f9dcc14aba0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.442+0000 7f9dd3647640 1 --2- 172.21.15.60:0/2661294784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dcc0cd3d0 0x7f9dcc159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.442+0000 7f9dd3647640 1 -- 172.21.15.60:0/2661294784 >> 172.21.15.60:0/2661294784 conn(0x7f9dcc0670b0 msgr2=0x7f9dcc15d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.442+0000 7f9dd3647640 1 -- 172.21.15.60:0/2661294784 shutdown_connections 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.442+0000 7f9dd3647640 1 -- 172.21.15.60:0/2661294784 wait complete. 2024-04-25T10:15:50.462 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:15:50.566 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:15:50.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.702+0000 7fe54e478640 1 Processor -- start 2024-04-25T10:15:50.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.702+0000 7fe54e478640 1 -- start start 2024-04-25T10:15:50.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.702+0000 7fe54e478640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe5480afac0 0x7fe5480cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.702+0000 7fe54e478640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe5480cf010 0x7fe548072750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.702+0000 7fe54e478640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe54815e7f0 con 0x7fe54815aa30 2024-04-25T10:15:50.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.702+0000 7fe54e478640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54815ead0 con 0x7fe5480cf010 2024-04-25T10:15:50.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.702+0000 7fe54e478640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54815edb0 con 0x7fe5480afac0 2024-04-25T10:15:50.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe54ca15640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe5480cf010 0x7fe548072750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe54ca15640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe5480cf010 0x7fe548072750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41572/0 (socket says 172.21.15.60:41572) 2024-04-25T10:15:50.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe54ca15640 1 -- 172.21.15.60:0/1838488406 learned_addr learned my addr 172.21.15.60:0/1838488406 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:50.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe5477fe640 1 --2- 172.21.15.60:0/1838488406 >> v2:172.21.15.100:3300/0 conn(0x7fe5480afac0 0x7fe5480cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe54ca15640 1 -- 172.21.15.60:0/1838488406 >> v2:172.21.15.100:3300/0 conn(0x7fe5480afac0 msgr2=0x7fe5480cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe54ca15640 1 --2- 172.21.15.60:0/1838488406 >> v2:172.21.15.100:3300/0 conn(0x7fe5480afac0 0x7fe5480cead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe54ca15640 1 -- 172.21.15.60:0/1838488406 >> v1:172.21.15.100:6789/0 conn(0x7fe54815aa30 legacy=0x7fe5480b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe54ca15640 1 -- 172.21.15.60:0/1838488406 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54815f090 con 0x7fe5480cf010 2024-04-25T10:15:50.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe5477fe640 1 --2- 172.21.15.60:0/1838488406 >> v2:172.21.15.100:3300/0 conn(0x7fe5480afac0 0x7fe5480cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:50.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe54ca15640 1 --2- 172.21.15.60:0/1838488406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480cf010 0x7fe548072750 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fe54806d0a0 tx=0x7fe54001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3dbc5eab3d1e0bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe546ffd640 1 -- 172.21.15.60:0/1838488406 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5400a5590 con 0x7fe5480cf010 2024-04-25T10:15:50.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe546ffd640 1 -- 172.21.15.60:0/1838488406 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5400a5730 con 0x7fe5480cf010 2024-04-25T10:15:50.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe546ffd640 1 -- 172.21.15.60:0/1838488406 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5400a5b00 con 0x7fe5480cf010 2024-04-25T10:15:50.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe54e478640 1 -- 172.21.15.60:0/1838488406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480cf010 msgr2=0x7fe548072750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe54e478640 1 --2- 172.21.15.60:0/1838488406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480cf010 0x7fe548072750 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fe54806d0a0 tx=0x7fe54001c8d0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe54e478640 1 -- 172.21.15.60:0/1838488406 shutdown_connections 2024-04-25T10:15:50.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe54e478640 1 --2- 172.21.15.60:0/1838488406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480cf010 0x7fe548072750 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe54e478640 1 --2- 172.21.15.60:0/1838488406 >> v2:172.21.15.100:3300/0 conn(0x7fe5480afac0 0x7fe5480cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe54e478640 1 -- 172.21.15.60:0/1838488406 >> 172.21.15.60:0/1838488406 conn(0x7fe5480b2360 msgr2=0x7fe5480b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe54e478640 1 -- 172.21.15.60:0/1838488406 shutdown_connections 2024-04-25T10:15:50.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.706+0000 7fe54e478640 1 -- 172.21.15.60:0/1838488406 wait complete. 2024-04-25T10:15:50.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe54e478640 1 Processor -- start 2024-04-25T10:15:50.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe54e478640 1 -- start start 2024-04-25T10:15:50.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe54e478640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480afac0 0x7fe54815d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe54e478640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5480cf010 0x7fe5481419c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe54e478640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe54815aa30 0x7fe548148170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe54e478640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe548165a80 con 0x7fe5480afac0 2024-04-25T10:15:50.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe54e478640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe548165900 con 0x7fe5480cf010 2024-04-25T10:15:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe54e478640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe548165c00 con 0x7fe54815aa30 2024-04-25T10:15:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe547fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480afac0 0x7fe54815d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe547fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480afac0 0x7fe54815d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41584/0 (socket says 172.21.15.60:41584) 2024-04-25T10:15:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe547fff640 1 -- 172.21.15.60:0/71689946 learned_addr learned my addr 172.21.15.60:0/71689946 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe5477fe640 1 --2- 172.21.15.60:0/71689946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5480cf010 0x7fe5481419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe54ca15640 1 --2- 172.21.15.60:0/71689946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe54815aa30 0x7fe548148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe547fff640 1 -- 172.21.15.60:0/71689946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe54815aa30 msgr2=0x7fe548148170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe547fff640 1 --2- 172.21.15.60:0/71689946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe54815aa30 0x7fe548148170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe547fff640 1 -- 172.21.15.60:0/71689946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5480cf010 msgr2=0x7fe5481419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe547fff640 1 --2- 172.21.15.60:0/71689946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5480cf010 0x7fe5481419c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe547fff640 1 -- 172.21.15.60:0/71689946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54815c4a0 con 0x7fe5480afac0 2024-04-25T10:15:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe5477fe640 1 --2- 172.21.15.60:0/71689946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5480cf010 0x7fe5481419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe547fff640 1 --2- 172.21.15.60:0/71689946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480afac0 0x7fe54815d260 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c370 tx=0x7fe53800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe544ff9640 1 -- 172.21.15.60:0/71689946 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5380052d0 con 0x7fe5480afac0 2024-04-25T10:15:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe544ff9640 1 -- 172.21.15.60:0/71689946 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe538005470 con 0x7fe5480afac0 2024-04-25T10:15:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe54e478640 1 -- 172.21.15.60:0/71689946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54815c780 con 0x7fe5480afac0 2024-04-25T10:15:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe544ff9640 1 -- 172.21.15.60:0/71689946 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe538007260 con 0x7fe5480afac0 2024-04-25T10:15:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe54e478640 1 -- 172.21.15.60:0/71689946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe548148bc0 con 0x7fe5480afac0 2024-04-25T10:15:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.710+0000 7fe54e478640 1 -- 172.21.15.60:0/71689946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54806ccb0 con 0x7fe5480afac0 2024-04-25T10:15:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.714+0000 7fe544ff9640 1 -- 172.21.15.60:0/71689946 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe538007400 con 0x7fe5480afac0 2024-04-25T10:15:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.714+0000 7fe544ff9640 1 --2- 172.21.15.60:0/71689946 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe534066f60 0x7fe534069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.714+0000 7fe544ff9640 1 -- 172.21.15.60:0/71689946 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe538084580 con 0x7fe5480afac0 2024-04-25T10:15:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.714+0000 7fe5477fe640 1 --2- 172.21.15.60:0/71689946 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe534066f60 0x7fe534069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.714+0000 7fe5477fe640 1 --2- 172.21.15.60:0/71689946 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe534066f60 0x7fe534069420 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe5480b22c0 tx=0x7fe53c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.718+0000 7fe544ff9640 1 -- 172.21.15.60:0/71689946 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5380554a0 con 0x7fe5480afac0 2024-04-25T10:15:50.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.910+0000 7fe54e478640 1 -- 172.21.15.60:0/71689946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe548074770 con 0x7fe5480afac0 2024-04-25T10:15:50.917 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:15:50.910+0000 7f3a792d6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:15:50.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.914+0000 7fe544ff9640 1 -- 172.21.15.60:0/71689946 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe538054da0 con 0x7fe5480afac0 2024-04-25T10:15:50.919 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:15:50.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.918+0000 7fe54e478640 1 -- 172.21.15.60:0/71689946 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe534066f60 msgr2=0x7fe534069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.918+0000 7fe54e478640 1 --2- 172.21.15.60:0/71689946 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe534066f60 0x7fe534069420 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe5480b22c0 tx=0x7fe53c008040 comp rx=0 tx=0).stop 2024-04-25T10:15:50.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.918+0000 7fe54e478640 1 -- 172.21.15.60:0/71689946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480afac0 msgr2=0x7fe54815d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.918+0000 7fe54e478640 1 --2- 172.21.15.60:0/71689946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480afac0 0x7fe54815d260 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c370 tx=0x7fe53800c830 comp rx=0 tx=0).stop 2024-04-25T10:15:50.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.918+0000 7fe54e478640 1 -- 172.21.15.60:0/71689946 shutdown_connections 2024-04-25T10:15:50.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.918+0000 7fe54e478640 1 --2- 172.21.15.60:0/71689946 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe534066f60 0x7fe534069420 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.918+0000 7fe54e478640 1 --2- 172.21.15.60:0/71689946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe54815aa30 0x7fe548148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.918+0000 7fe54e478640 1 --2- 172.21.15.60:0/71689946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5480cf010 0x7fe5481419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.918+0000 7fe54e478640 1 --2- 172.21.15.60:0/71689946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480afac0 0x7fe54815d260 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.918+0000 7fe54e478640 1 -- 172.21.15.60:0/71689946 >> 172.21.15.60:0/71689946 conn(0x7fe5480b2360 msgr2=0x7fe54806b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.918+0000 7fe54e478640 1 -- 172.21.15.60:0/71689946 shutdown_connections 2024-04-25T10:15:50.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.918+0000 7fe54e478640 1 -- 172.21.15.60:0/71689946 wait complete. 2024-04-25T10:15:50.929 INFO:tasks.ceph.mds.a.smithi060.stderr: -1148> 2024-04-25T10:15:20.066+0000 7f1b7b02b640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:15:50.929 INFO:tasks.ceph.mds.a.smithi060.stderr: -1145> 2024-04-25T10:15:20.066+0000 7f1b7b02b640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-25T10:15:50.929 INFO:tasks.ceph.mds.a.smithi060.stderr: -1129> 2024-04-25T10:15:20.066+0000 7f1b7b02b640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:15:50.929 INFO:tasks.ceph.mds.a.smithi060.stderr: -1126> 2024-04-25T10:15:20.066+0000 7f1b7b02b640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-25T10:15:50.930 INFO:tasks.ceph.mds.a.smithi060.stderr: -1000> 2024-04-25T10:15:20.426+0000 7f1b7f033640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:15:50.940 DEBUG:teuthology.orchestra.run.smithi060:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-04-25T10:15:50.966 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T10:15:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.962+0000 7f7c837e3400 1 -- start start 2024-04-25T10:15:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.962+0000 7f7c837e3400 1 Processor -- start 2024-04-25T10:15:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559a5b2b5800 0x559a5b46e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5b2b5400 0x559a5b46eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a5b2b5000 0x559a5b46f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5c080180 con 0x559a5b2b5400 2024-04-25T10:15:50.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5c080300 con 0x559a5b2b5000 2024-04-25T10:15:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a5c080480 con 0x559a5b2b5800 2024-04-25T10:15:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5b2b5400 0x559a5b46eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5b2b5400 0x559a5b46eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41620/0 (socket says 172.21.15.60:41620) 2024-04-25T10:15:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 -- 172.21.15.60:0/1904512953 learned_addr learned my addr 172.21.15.60:0/1904512953 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 -- 172.21.15.60:0/1904512953 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559a5b2b5800 msgr2=0x559a5b46e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 --2- 172.21.15.60:0/1904512953 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559a5b2b5800 0x559a5b46e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 -- 172.21.15.60:0/1904512953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a5b2b5000 msgr2=0x559a5b46f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 --2- 172.21.15.60:0/1904512953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a5b2b5000 0x559a5b46f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 -- 172.21.15.60:0/1904512953 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a5c076960 con 0x559a5b2b5400 2024-04-25T10:15:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 --2- 172.21.15.60:0/1904512953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5b2b5400 0x559a5b46eb00 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x559a5c1271d0 tx=0x559a5b36cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5f98170a771ef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c7f79d640 1 -- 172.21.15.60:0/1904512953 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559a5b2ea4e0 con 0x559a5b2b5400 2024-04-25T10:15:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c7f79d640 1 -- 172.21.15.60:0/1904512953 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559a5b2fe540 con 0x559a5b2b5400 2024-04-25T10:15:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c7f79d640 1 -- 172.21.15.60:0/1904512953 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559a5b2eab60 con 0x559a5b2b5400 2024-04-25T10:15:50.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 1 -- 172.21.15.60:0/1904512953 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a5c076d20 con 0x559a5b2b5400 2024-04-25T10:15:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 1 -- 172.21.15.60:0/1904512953 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x559a5c076f00 con 0x559a5b2b5400 2024-04-25T10:15:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 4 waiting for MDS map... 2024-04-25T10:15:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c7f79d640 1 -- 172.21.15.60:0/1904512953 <== mon.0 v2:172.21.15.60:3300/0 4 ==== osd_map(99..99 src has 1..99) v4 ==== 5881+0+0 (secure 0 0 0) 0x559a5c1be600 con 0x559a5b2b5400 2024-04-25T10:15:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c7f79d640 1 -- 172.21.15.60:0/1904512953 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 84) v1 ==== 2723+0+0 (secure 0 0 0) 0x559a5b30b180 con 0x559a5b2b5400 2024-04-25T10:15:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 4 Got MDS map 84 2024-04-25T10:15:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 10 main: main 2024-04-25T10:15:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 4 main: connecting to RADOS... 2024-04-25T10:15:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 1 Processor -- start 2024-04-25T10:15:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 1 -- start start 2024-04-25T10:15:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559a5c1c2c00 0x559a5b46f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a5c1c3000 0x559a5b46fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5c1c3400 0x559a5b470100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5c080600 con 0x559a5c1c3400 2024-04-25T10:15:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5c080780 con 0x559a5c1c3000 2024-04-25T10:15:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c837e3400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a5c080900 con 0x559a5c1c2c00 2024-04-25T10:15:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5c1c3400 0x559a5b470100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5c1c3400 0x559a5b470100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41636/0 (socket says 172.21.15.60:41636) 2024-04-25T10:15:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 -- 172.21.15.60:0/401454714 learned_addr learned my addr 172.21.15.60:0/401454714 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c81fa2640 1 --2- 172.21.15.60:0/401454714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a5c1c3000 0x559a5b46fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c827a3640 1 --2- 172.21.15.60:0/401454714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559a5c1c2c00 0x559a5b46f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 -- 172.21.15.60:0/401454714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559a5c1c2c00 msgr2=0x559a5b46f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 --2- 172.21.15.60:0/401454714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559a5c1c2c00 0x559a5b46f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 -- 172.21.15.60:0/401454714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a5c1c3000 msgr2=0x559a5b46fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 --2- 172.21.15.60:0/401454714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a5c1c3000 0x559a5b46fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 -- 172.21.15.60:0/401454714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a5c076d20 con 0x559a5c1c3400 2024-04-25T10:15:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.966+0000 7f7c817a1640 1 --2- 172.21.15.60:0/401454714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5c1c3400 0x559a5b470100 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x559a5c228060 tx=0x559a5b3771a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e8ba0e0a7dbce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c7d799640 1 -- 172.21.15.60:0/401454714 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559a5b2ea340 con 0x559a5c1c3400 2024-04-25T10:15:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c7d799640 1 -- 172.21.15.60:0/401454714 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559a5b2fee00 con 0x559a5c1c3400 2024-04-25T10:15:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c7d799640 1 -- 172.21.15.60:0/401454714 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559a5b2ea4e0 con 0x559a5c1c3400 2024-04-25T10:15:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 -- 172.21.15.60:0/401454714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5c1c3400 msgr2=0x559a5b470100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 --2- 172.21.15.60:0/401454714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5c1c3400 0x559a5b470100 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x559a5c228060 tx=0x559a5b3771a0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 -- 172.21.15.60:0/401454714 shutdown_connections 2024-04-25T10:15:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 --2- 172.21.15.60:0/401454714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5c1c3400 0x559a5b470100 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 --2- 172.21.15.60:0/401454714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a5c1c3000 0x559a5b46fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 --2- 172.21.15.60:0/401454714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559a5c1c2c00 0x559a5b46f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 -- 172.21.15.60:0/401454714 >> 172.21.15.60:0/401454714 conn(0x559a5b2b4c00 msgr2=0x559a5b2e3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 -- 172.21.15.60:0/401454714 shutdown_connections 2024-04-25T10:15:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 -- 172.21.15.60:0/401454714 wait complete. 2024-04-25T10:15:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 Processor -- start 2024-04-25T10:15:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 -- start start 2024-04-25T10:15:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a5c1c2c00 0x559a5b471180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559a5c1c3000 0x559a5b470c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5c1c3400 0x559a5b470680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5c081c80 con 0x559a5c1c3400 2024-04-25T10:15:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5c081b00 con 0x559a5c1c2c00 2024-04-25T10:15:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a5c081980 con 0x559a5c1c3000 2024-04-25T10:15:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c817a1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5c1c3400 0x559a5b470680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c817a1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5c1c3400 0x559a5b470680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41638/0 (socket says 172.21.15.60:41638) 2024-04-25T10:15:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c817a1640 1 -- 172.21.15.60:0/2163534538 learned_addr learned my addr 172.21.15.60:0/2163534538 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c827a3640 1 --2- 172.21.15.60:0/2163534538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a5c1c2c00 0x559a5b471180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c81fa2640 1 --2- 172.21.15.60:0/2163534538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559a5c1c3000 0x559a5b470c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c817a1640 1 -- 172.21.15.60:0/2163534538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559a5c1c3000 msgr2=0x559a5b470c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c817a1640 1 --2- 172.21.15.60:0/2163534538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559a5c1c3000 0x559a5b470c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c817a1640 1 -- 172.21.15.60:0/2163534538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a5c1c2c00 msgr2=0x559a5b471180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c817a1640 1 --2- 172.21.15.60:0/2163534538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a5c1c2c00 0x559a5b471180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c817a1640 1 -- 172.21.15.60:0/2163534538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a5c077a40 con 0x559a5c1c3400 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c817a1640 1 --2- 172.21.15.60:0/2163534538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5c1c3400 0x559a5b470680 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x559a5c2282d0 tx=0x559a5c234300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c7bf96640 1 -- 172.21.15.60:0/2163534538 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559a5b2eab60 con 0x559a5c1c3400 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c7bf96640 1 -- 172.21.15.60:0/2163534538 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559a5b2fec40 con 0x559a5c1c3400 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c7bf96640 1 -- 172.21.15.60:0/2163534538 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559a5b2ea000 con 0x559a5c1c3400 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 -- 172.21.15.60:0/2163534538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559a5c077c20 con 0x559a5c1c3400 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 1 -- 172.21.15.60:0/2163534538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a5b5e2000 con 0x559a5c1c3400 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c837e3400 4 main: resolving pool 14 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.970+0000 7f7c7bf96640 1 -- 172.21.15.60:0/2163534538 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x559a5c236c00 con 0x559a5c1c3400 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c7bf96640 1 --2- 172.21.15.60:0/2163534538 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x559a5b5df000 0x559a5b471700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c7bf96640 1 -- 172.21.15.60:0/2163534538 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5881+0+0 (secure 0 0 0) 0x559a5b5dc600 con 0x559a5c1c3400 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 4 main: creating IoCtx.. 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 --2- 172.21.15.60:0/2163534538 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x559a5c1c3800 0x559a5b470100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 -- 172.21.15.60:0/2163534538 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x559a5c1c3c00 con 0x559a5c1c3800 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c81fa2640 1 --2- 172.21.15.60:0/2163534538 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x559a5c1c3800 0x559a5b470100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c827a3640 1 --2- 172.21.15.60:0/2163534538 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x559a5b5df000 0x559a5b471700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c81fa2640 1 --2- 172.21.15.60:0/2163534538 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x559a5c1c3800 0x559a5b470100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c827a3640 1 --2- 172.21.15.60:0/2163534538 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x559a5b5df000 0x559a5b471700 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x559a5b305920 tx=0x559a5c084000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c81fa2640 1 -- 172.21.15.60:0/2163534538 <== osd.2 v2:172.21.15.60:6800/1771147646 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x559a5b3c0000 con 0x559a5c1c3800 2024-04-25T10:15:50.985 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:15:50.986 INFO:teuthology.orchestra.run.smithi060.stdout: "0": { 2024-04-25T10:15:50.986 INFO:teuthology.orchestra.run.smithi060.stdout: "data": { 2024-04-25T10:15:50.986 INFO:teuthology.orchestra.run.smithi060.stdout: "version": 5, 2024-04-25T10:15:50.986 INFO:teuthology.orchestra.run.smithi060.stdout: "inotable": { 2024-04-25T10:15:50.986 INFO:teuthology.orchestra.run.smithi060.stdout: "projected_free": [ 2024-04-25T10:15:50.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:15:50.986 INFO:teuthology.orchestra.run.smithi060.stdout: "start": 1099511627779, 2024-04-25T10:15:50.986 INFO:teuthology.orchestra.run.smithi060.stdout: "len": 1099511627773 2024-04-25T10:15:50.986 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:15:50.986 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-04-25T10:15:50.986 INFO:teuthology.orchestra.run.smithi060.stdout: "free": [ 2024-04-25T10:15:50.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:15:50.986 INFO:teuthology.orchestra.run.smithi060.stdout: "start": 1099511627779, 2024-04-25T10:15:50.987 INFO:teuthology.orchestra.run.smithi060.stdout: "len": 1099511627773 2024-04-25T10:15:50.987 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:15:50.987 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T10:15:50.987 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:15:50.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:15:50.987 INFO:teuthology.orchestra.run.smithi060.stdout: "result": 0 2024-04-25T10:15:50.987 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:15:50.987 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:15:50.987 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 -- 172.21.15.60:0/2163534538 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x559a5c1c3800 msgr2=0x559a5b470100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 --2- 172.21.15.60:0/2163534538 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x559a5c1c3800 0x559a5b470100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 -- 172.21.15.60:0/2163534538 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x559a5b5df000 msgr2=0x559a5b471700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 --2- 172.21.15.60:0/2163534538 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x559a5b5df000 0x559a5b471700 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x559a5b305920 tx=0x559a5c084000 comp rx=0 tx=0).stop 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 -- 172.21.15.60:0/2163534538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5c1c3400 msgr2=0x559a5b470680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 --2- 172.21.15.60:0/2163534538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5c1c3400 0x559a5b470680 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x559a5c2282d0 tx=0x559a5c234300 comp rx=0 tx=0).stop 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c81fa2640 1 -- 172.21.15.60:0/2163534538 reap_dead start 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 -- 172.21.15.60:0/2163534538 shutdown_connections 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 -- 172.21.15.60:0/2163534538 >> 172.21.15.60:0/2163534538 conn(0x559a5b2b4c00 msgr2=0x559a5b2e3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 -- 172.21.15.60:0/2163534538 shutdown_connections 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 -- 172.21.15.60:0/2163534538 wait complete. 2024-04-25T10:15:50.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 -- 172.21.15.60:0/1904512953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5b2b5400 msgr2=0x559a5b46eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 --2- 172.21.15.60:0/1904512953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5b2b5400 0x559a5b46eb00 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x559a5c1271d0 tx=0x559a5b36cc00 comp rx=0 tx=0).stop 2024-04-25T10:15:50.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 -- 172.21.15.60:0/1904512953 shutdown_connections 2024-04-25T10:15:50.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 --2- 172.21.15.60:0/1904512953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559a5b2b5000 0x559a5b46f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 --2- 172.21.15.60:0/1904512953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559a5b2b5400 0x559a5b46eb00 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 --2- 172.21.15.60:0/1904512953 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559a5b2b5800 0x559a5b46e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 -- 172.21.15.60:0/1904512953 >> 172.21.15.60:0/1904512953 conn(0x559a5b2b5c00 msgr2=0x559a5b2e3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 -- 172.21.15.60:0/1904512953 shutdown_connections 2024-04-25T10:15:50.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:50.974+0000 7f7c837e3400 1 -- 172.21.15.60:0/1904512953 wait complete. 2024-04-25T10:15:50.990 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.049972, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-04-25T10:15:50.991 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:15:50.991 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:51.075 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:51.139 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:15:51.140 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:15:51.140 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:51.145 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:51.145 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:15:51.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:51.145 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:15:51.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:15:51.194 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:15:51.195 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:15:51.195 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:51.201 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:51.201 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:15:51.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:51.201 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:15:51.250 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:15:51.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:15:51.251 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:15:51.251 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:51.257 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:51.257 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:15:51.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:51.257 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:15:51.306 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:15:51.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:15:51.307 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:15:51.307 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:51.313 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:51.313 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:15:51.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:51.313 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:15:51.361 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:15:51.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:15:51.363 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:51.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.494+0000 7fea67d35640 1 Processor -- start 2024-04-25T10:15:51.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.494+0000 7fea67d35640 1 -- start start 2024-04-25T10:15:51.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.494+0000 7fea67d35640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea600b4660 0x7fea6015c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.494+0000 7fea67d35640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea60071ca0 con 0x7fea600ce7a0 2024-04-25T10:15:51.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.494+0000 7fea67d35640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea600cca40 con 0x7fea6015bc90 2024-04-25T10:15:51.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.494+0000 7fea67d35640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea600cc370 con 0x7fea600b4660 2024-04-25T10:15:51.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.494+0000 7fea65ad1640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fea600ce7a0 0x7fea600ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49208/0 (socket says 172.21.15.60:49208) 2024-04-25T10:15:51.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.494+0000 7fea65ad1640 1 -- 172.21.15.60:0/230281499 learned_addr learned my addr 172.21.15.60:0/230281499 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:15:51.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.498+0000 7fea662d2640 1 --2- 172.21.15.60:0/230281499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea600b4660 0x7fea6015c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.498+0000 7fea64acf640 1 -- 172.21.15.60:0/230281499 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547689447 0 0) 0x7fea60071ca0 con 0x7fea600ce7a0 2024-04-25T10:15:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.498+0000 7fea64acf640 1 -- 172.21.15.60:0/230281499 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea48003530 con 0x7fea600ce7a0 2024-04-25T10:15:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.498+0000 7fea662d2640 1 -- 172.21.15.60:0/230281499 >> v1:172.21.15.100:6789/0 conn(0x7fea6015bc90 legacy=0x7fea6015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.498+0000 7fea64acf640 1 -- 172.21.15.60:0/230281499 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142499146 0 0) 0x7fea600cca40 con 0x7fea6015bc90 2024-04-25T10:15:51.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.498+0000 7fea662d2640 1 -- 172.21.15.60:0/230281499 >> v1:172.21.15.60:6789/0 conn(0x7fea600ce7a0 legacy=0x7fea600ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.498+0000 7fea662d2640 1 -- 172.21.15.60:0/230281499 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea600ccd20 con 0x7fea600b4660 2024-04-25T10:15:51.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.498+0000 7fea662d2640 1 --2- 172.21.15.60:0/230281499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea600b4660 0x7fea6015c930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea50009dc0 tx=0x7fea5001c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b273c993d5a1c505 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.498+0000 7fea64acf640 1 -- 172.21.15.60:0/230281499 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea500ac070 con 0x7fea600b4660 2024-04-25T10:15:51.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.498+0000 7fea64acf640 1 -- 172.21.15.60:0/230281499 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea500a6460 con 0x7fea600b4660 2024-04-25T10:15:51.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.498+0000 7fea64acf640 1 -- 172.21.15.60:0/230281499 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58001680 con 0x7fea600b4660 2024-04-25T10:15:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea67d35640 1 -- 172.21.15.60:0/230281499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea600b4660 msgr2=0x7fea6015c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea67d35640 1 --2- 172.21.15.60:0/230281499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea600b4660 0x7fea6015c930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea50009dc0 tx=0x7fea5001c7d0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea67d35640 1 -- 172.21.15.60:0/230281499 shutdown_connections 2024-04-25T10:15:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea67d35640 1 --2- 172.21.15.60:0/230281499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea600b4660 0x7fea6015c930 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea67d35640 1 -- 172.21.15.60:0/230281499 >> 172.21.15.60:0/230281499 conn(0x7fea600b12d0 msgr2=0x7fea600b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea67d35640 1 -- 172.21.15.60:0/230281499 shutdown_connections 2024-04-25T10:15:51.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea67d35640 1 -- 172.21.15.60:0/230281499 wait complete. 2024-04-25T10:15:51.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea67d35640 1 Processor -- start 2024-04-25T10:15:51.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea67d35640 1 -- start start 2024-04-25T10:15:51.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea67d35640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea600ce7a0 0x7fea601580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea67d35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea6015bc90 0x7fea600714d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea67d35640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea60159ce0 0x7fea6014d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea67d35640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea601695d0 con 0x7fea600ce7a0 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea67d35640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea600b3db0 con 0x7fea6015bc90 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea67d35640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea60169750 con 0x7fea60159ce0 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea65ad1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea600ce7a0 0x7fea601580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea65ad1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea600ce7a0 0x7fea601580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41640/0 (socket says 172.21.15.60:41640) 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea65ad1640 1 -- 172.21.15.60:0/3059264483 learned_addr learned my addr 172.21.15.60:0/3059264483 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea662d2640 1 --2- 172.21.15.60:0/3059264483 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea60159ce0 0x7fea6014d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea652d0640 1 --2- 172.21.15.60:0/3059264483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea6015bc90 0x7fea600714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea65ad1640 1 -- 172.21.15.60:0/3059264483 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea60159ce0 msgr2=0x7fea6014d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea65ad1640 1 --2- 172.21.15.60:0/3059264483 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea60159ce0 0x7fea6014d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea65ad1640 1 -- 172.21.15.60:0/3059264483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea6015bc90 msgr2=0x7fea600714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea65ad1640 1 --2- 172.21.15.60:0/3059264483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea6015bc90 0x7fea600714d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea65ad1640 1 -- 172.21.15.60:0/3059264483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6014eeb0 con 0x7fea600ce7a0 2024-04-25T10:15:51.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.502+0000 7fea662d2640 1 --2- 172.21.15.60:0/3059264483 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea60159ce0 0x7fea6014d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:51.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.506+0000 7fea65ad1640 1 --2- 172.21.15.60:0/3059264483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea600ce7a0 0x7fea601580d0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fea5800c760 tx=0x7fea5800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.506+0000 7fea5e7fc640 1 -- 172.21.15.60:0/3059264483 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58018070 con 0x7fea600ce7a0 2024-04-25T10:15:51.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.506+0000 7fea5e7fc640 1 -- 172.21.15.60:0/3059264483 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea58007500 con 0x7fea600ce7a0 2024-04-25T10:15:51.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.506+0000 7fea67d35640 1 -- 172.21.15.60:0/3059264483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea601587e0 con 0x7fea600ce7a0 2024-04-25T10:15:51.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.506+0000 7fea5e7fc640 1 -- 172.21.15.60:0/3059264483 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58004a30 con 0x7fea600ce7a0 2024-04-25T10:15:51.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.506+0000 7fea67d35640 1 -- 172.21.15.60:0/3059264483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea60158da0 con 0x7fea600ce7a0 2024-04-25T10:15:51.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.506+0000 7fea67d35640 1 -- 172.21.15.60:0/3059264483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea60074770 con 0x7fea600ce7a0 2024-04-25T10:15:51.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.506+0000 7fea5e7fc640 1 -- 172.21.15.60:0/3059264483 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fea5801b070 con 0x7fea600ce7a0 2024-04-25T10:15:51.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.506+0000 7fea5e7fc640 1 --2- 172.21.15.60:0/3059264483 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fea4c066f60 0x7fea4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.506+0000 7fea5e7fc640 1 -- 172.21.15.60:0/3059264483 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fea58084660 con 0x7fea600ce7a0 2024-04-25T10:15:51.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.506+0000 7fea652d0640 1 --2- 172.21.15.60:0/3059264483 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fea4c066f60 0x7fea4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.506+0000 7fea652d0640 1 --2- 172.21.15.60:0/3059264483 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fea4c066f60 0x7fea4c069420 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fea600cb7c0 tx=0x7fea54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.510+0000 7fea5e7fc640 1 -- 172.21.15.60:0/3059264483 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fea580555c0 con 0x7fea600ce7a0 2024-04-25T10:15:51.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.698+0000 7fea67d35640 1 -- 172.21.15.60:0/3059264483 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea60153750 con 0x7fea600ce7a0 2024-04-25T10:15:51.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.698+0000 7fea5e7fc640 1 -- 172.21.15.60:0/3059264483 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4145 (secure 0 0 0) 0x7fea58054ec0 con 0x7fea600ce7a0 2024-04-25T10:15:51.703 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:15:51.703 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1491228634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1491228634},{"type":"v1","addr":"172.21.15.100:6837","nonce":1491228634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/209898872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":209898872},{"type":"v1","addr":"172.21.15.100:6835","nonce":209898872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1793343287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1793343287},{"type":"v1","addr":"172.21.15.60:6833","nonce":1793343287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":84,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:16.146481+0000","modified":"2024-04-25T10:15:50.915413+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:15:51.704 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 84 2024-04-25T10:15:51.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.698+0000 7fea67d35640 1 -- 172.21.15.60:0/3059264483 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fea4c066f60 msgr2=0x7fea4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.698+0000 7fea67d35640 1 --2- 172.21.15.60:0/3059264483 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fea4c066f60 0x7fea4c069420 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fea600cb7c0 tx=0x7fea54009040 comp rx=0 tx=0).stop 2024-04-25T10:15:51.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.698+0000 7fea67d35640 1 -- 172.21.15.60:0/3059264483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea600ce7a0 msgr2=0x7fea601580d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.698+0000 7fea67d35640 1 --2- 172.21.15.60:0/3059264483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea600ce7a0 0x7fea601580d0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fea5800c760 tx=0x7fea5800cc20 comp rx=0 tx=0).stop 2024-04-25T10:15:51.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.698+0000 7fea67d35640 1 -- 172.21.15.60:0/3059264483 shutdown_connections 2024-04-25T10:15:51.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.702+0000 7fea67d35640 1 --2- 172.21.15.60:0/3059264483 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fea4c066f60 0x7fea4c069420 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.702+0000 7fea67d35640 1 --2- 172.21.15.60:0/3059264483 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea60159ce0 0x7fea6014d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.702+0000 7fea67d35640 1 --2- 172.21.15.60:0/3059264483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea6015bc90 0x7fea600714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.702+0000 7fea67d35640 1 --2- 172.21.15.60:0/3059264483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea600ce7a0 0x7fea601580d0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.702+0000 7fea67d35640 1 -- 172.21.15.60:0/3059264483 >> 172.21.15.60:0/3059264483 conn(0x7fea600b12d0 msgr2=0x7fea60153f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:51.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.702+0000 7fea67d35640 1 -- 172.21.15.60:0/3059264483 shutdown_connections 2024-04-25T10:15:51.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.702+0000 7fea67d35640 1 -- 172.21.15.60:0/3059264483 wait complete. 2024-04-25T10:15:51.722 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:51.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.838+0000 7fed42fba640 1 Processor -- start 2024-04-25T10:15:51.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.838+0000 7fed42fba640 1 -- start start 2024-04-25T10:15:51.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed42fba640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed3c06ac10 0x7fed3c06b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed42fba640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed3c06b570 con 0x7fed3c145280 2024-04-25T10:15:51.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed42fba640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed3c145660 con 0x7fed3c15bf10 2024-04-25T10:15:51.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed42fba640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0b46a0 con 0x7fed3c06ac10 2024-04-25T10:15:51.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed40d56640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fed3c145280 0x7fed3c0b3f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49216/0 (socket says 172.21.15.60:49216) 2024-04-25T10:15:51.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed40d56640 1 -- 172.21.15.60:0/2803511743 learned_addr learned my addr 172.21.15.60:0/2803511743 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:15:51.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3b7fe640 1 -- 172.21.15.60:0/2803511743 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004049833 0 0) 0x7fed3c06b570 con 0x7fed3c145280 2024-04-25T10:15:51.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3b7fe640 1 -- 172.21.15.60:0/2803511743 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed24003530 con 0x7fed3c145280 2024-04-25T10:15:51.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3bfff640 1 --2- 172.21.15.60:0/2803511743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed3c06ac10 0x7fed3c06b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3b7fe640 1 -- 172.21.15.60:0/2803511743 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4169942487 0 0) 0x7fed24003530 con 0x7fed3c145280 2024-04-25T10:15:51.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3b7fe640 1 -- 172.21.15.60:0/2803511743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed3c06ac10 msgr2=0x7fed3c06b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:51.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3b7fe640 1 --2- 172.21.15.60:0/2803511743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed3c06ac10 0x7fed3c06b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3b7fe640 1 -- 172.21.15.60:0/2803511743 >> v1:172.21.15.100:6789/0 conn(0x7fed3c15bf10 legacy=0x7fed3c15c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3b7fe640 1 -- 172.21.15.60:0/2803511743 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c0b4950 con 0x7fed3c145280 2024-04-25T10:15:51.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3b7fe640 1 -- 172.21.15.60:0/2803511743 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fed2c001a00 con 0x7fed3c145280 2024-04-25T10:15:51.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3b7fe640 1 -- 172.21.15.60:0/2803511743 >> v1:172.21.15.60:6789/0 conn(0x7fed3c145280 legacy=0x7fed3c0b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3b7fe640 1 --2- 172.21.15.60:0/2803511743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed3c06ac10 0x7fed240039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3b7fe640 1 --2- 172.21.15.60:0/2803511743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed24003f70 0x7fed24004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3b7fe640 1 --2- 172.21.15.60:0/2803511743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed24004d50 0x7fed240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3b7fe640 1 -- 172.21.15.60:0/2803511743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0b46a0 con 0x7fed24004d50 2024-04-25T10:15:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3b7fe640 1 -- 172.21.15.60:0/2803511743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2400bab0 con 0x7fed24003f70 2024-04-25T10:15:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3b7fe640 1 -- 172.21.15.60:0/2803511743 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2400bcb0 con 0x7fed3c06ac10 2024-04-25T10:15:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed41557640 1 --2- 172.21.15.60:0/2803511743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed24004d50 0x7fed240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed40d56640 1 --2- 172.21.15.60:0/2803511743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed3c06ac10 0x7fed240039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed41557640 1 -- 172.21.15.60:0/2803511743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed3c06ac10 msgr2=0x7fed240039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed41557640 1 --2- 172.21.15.60:0/2803511743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed3c06ac10 0x7fed240039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed41557640 1 -- 172.21.15.60:0/2803511743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed24003f70 msgr2=0x7fed24004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3bfff640 1 --2- 172.21.15.60:0/2803511743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed24003f70 0x7fed24004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed41557640 1 --2- 172.21.15.60:0/2803511743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed24003f70 0x7fed24004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed41557640 1 -- 172.21.15.60:0/2803511743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed24003530 con 0x7fed24004d50 2024-04-25T10:15:51.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.842+0000 7fed3bfff640 1 --2- 172.21.15.60:0/2803511743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed24003f70 0x7fed24004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:51.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed41557640 1 --2- 172.21.15.60:0/2803511743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed24004d50 0x7fed240051d0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fed34001780 tx=0x7fed3401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3843d2457835159a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed3b7fe640 1 -- 172.21.15.60:0/2803511743 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed340013a0 con 0x7fed24004d50 2024-04-25T10:15:51.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed3b7fe640 1 -- 172.21.15.60:0/2803511743 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed340a9070 con 0x7fed24004d50 2024-04-25T10:15:51.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed3b7fe640 1 -- 172.21.15.60:0/2803511743 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed34003600 con 0x7fed24004d50 2024-04-25T10:15:51.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed42fba640 1 -- 172.21.15.60:0/2803511743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed24004d50 msgr2=0x7fed240051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed42fba640 1 --2- 172.21.15.60:0/2803511743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed24004d50 0x7fed240051d0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fed34001780 tx=0x7fed3401c420 comp rx=0 tx=0).stop 2024-04-25T10:15:51.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed41557640 1 -- 172.21.15.60:0/2803511743 reap_dead start 2024-04-25T10:15:51.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed42fba640 1 -- 172.21.15.60:0/2803511743 shutdown_connections 2024-04-25T10:15:51.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed42fba640 1 -- 172.21.15.60:0/2803511743 >> 172.21.15.60:0/2803511743 conn(0x7fed3c0b5aa0 msgr2=0x7fed3c0b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:51.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed42fba640 1 -- 172.21.15.60:0/2803511743 shutdown_connections 2024-04-25T10:15:51.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed42fba640 1 -- 172.21.15.60:0/2803511743 wait complete. 2024-04-25T10:15:51.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed42fba640 1 Processor -- start 2024-04-25T10:15:51.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed42fba640 1 -- start start 2024-04-25T10:15:51.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed42fba640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed3c140920 0x7fed3c0d0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed42fba640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed3c13f380 0x7fed3c13f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed42fba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3c13fd00 0x7fed3c14d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed42fba640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0b51f0 con 0x7fed3c13fd00 2024-04-25T10:15:51.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed42fba640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0b4f10 con 0x7fed3c140920 2024-04-25T10:15:51.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.846+0000 7fed42fba640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0b54d0 con 0x7fed3c13f380 2024-04-25T10:15:51.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed41557640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3c13fd00 0x7fed3c14d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed41557640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3c13fd00 0x7fed3c14d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41652/0 (socket says 172.21.15.60:41652) 2024-04-25T10:15:51.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed41557640 1 -- 172.21.15.60:0/576813430 learned_addr learned my addr 172.21.15.60:0/576813430 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:51.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed40d56640 1 --2- 172.21.15.60:0/576813430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed3c140920 0x7fed3c0d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed3bfff640 1 --2- 172.21.15.60:0/576813430 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed3c13f380 0x7fed3c13f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed41557640 1 -- 172.21.15.60:0/576813430 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed3c13f380 msgr2=0x7fed3c13f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed41557640 1 --2- 172.21.15.60:0/576813430 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed3c13f380 0x7fed3c13f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed41557640 1 -- 172.21.15.60:0/576813430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed3c140920 msgr2=0x7fed3c0d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed41557640 1 --2- 172.21.15.60:0/576813430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed3c140920 0x7fed3c0d0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed41557640 1 -- 172.21.15.60:0/576813430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c14c360 con 0x7fed3c13fd00 2024-04-25T10:15:51.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed41557640 1 --2- 172.21.15.60:0/576813430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3c13fd00 0x7fed3c14d0e0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fed34005040 tx=0x7fed24005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed397fa640 1 -- 172.21.15.60:0/576813430 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed340bb070 con 0x7fed3c13fd00 2024-04-25T10:15:51.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed397fa640 1 -- 172.21.15.60:0/576813430 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed34007500 con 0x7fed3c13fd00 2024-04-25T10:15:51.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed42fba640 1 -- 172.21.15.60:0/576813430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3c140180 con 0x7fed3c13fd00 2024-04-25T10:15:51.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed397fa640 1 -- 172.21.15.60:0/576813430 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed340013a0 con 0x7fed3c13fd00 2024-04-25T10:15:51.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed42fba640 1 -- 172.21.15.60:0/576813430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c14c8e0 con 0x7fed3c13fd00 2024-04-25T10:15:51.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed42fba640 1 -- 172.21.15.60:0/576813430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3c074770 con 0x7fed3c13fd00 2024-04-25T10:15:51.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed397fa640 1 -- 172.21.15.60:0/576813430 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed340be070 con 0x7fed3c13fd00 2024-04-25T10:15:51.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed397fa640 1 --2- 172.21.15.60:0/576813430 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed28066f90 0x7fed28069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed397fa640 1 -- 172.21.15.60:0/576813430 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fed3411e3a0 con 0x7fed3c13fd00 2024-04-25T10:15:51.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.850+0000 7fed40d56640 1 --2- 172.21.15.60:0/576813430 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed28066f90 0x7fed28069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.854+0000 7fed40d56640 1 --2- 172.21.15.60:0/576813430 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed28066f90 0x7fed28069450 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0013e0 tx=0x7fed2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:51.854+0000 7fed397fa640 1 -- 172.21.15.60:0/576813430 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed340c4060 con 0x7fed3c13fd00 2024-04-25T10:15:52.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.014+0000 7fed42fba640 1 -- 172.21.15.60:0/576813430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed3c072710 con 0x7fed3c13fd00 2024-04-25T10:15:52.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.014+0000 7fed397fa640 1 -- 172.21.15.60:0/576813430 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16879 (secure 0 0 0) 0x7fed34016130 con 0x7fed3c13fd00 2024-04-25T10:15:52.021 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:15:52.022 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":99,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:15:50.915440+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:14.444411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:15.153386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/4149609651":"2024-04-26T10:15:50.915390+0000","172.21.15.60:6834/4149609651":"2024-04-26T10:15:50.915390+0000","172.21.15.60:6835/2002755047":"2024-04-26T10:15:14.408136+0000","172.21.15.60:6834/2002755047":"2024-04-26T10:15:14.408136+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:52.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.018+0000 7fed42fba640 1 -- 172.21.15.60:0/576813430 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed28066f90 msgr2=0x7fed28069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.018+0000 7fed42fba640 1 --2- 172.21.15.60:0/576813430 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed28066f90 0x7fed28069450 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0013e0 tx=0x7fed2c009040 comp rx=0 tx=0).stop 2024-04-25T10:15:52.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.018+0000 7fed42fba640 1 -- 172.21.15.60:0/576813430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3c13fd00 msgr2=0x7fed3c14d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.018+0000 7fed42fba640 1 --2- 172.21.15.60:0/576813430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3c13fd00 0x7fed3c14d0e0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fed34005040 tx=0x7fed24005840 comp rx=0 tx=0).stop 2024-04-25T10:15:52.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.018+0000 7fed42fba640 1 -- 172.21.15.60:0/576813430 shutdown_connections 2024-04-25T10:15:52.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.018+0000 7fed42fba640 1 --2- 172.21.15.60:0/576813430 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fed28066f90 0x7fed28069450 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.018+0000 7fed42fba640 1 --2- 172.21.15.60:0/576813430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3c13fd00 0x7fed3c14d0e0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.018+0000 7fed42fba640 1 --2- 172.21.15.60:0/576813430 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed3c13f380 0x7fed3c13f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.018+0000 7fed42fba640 1 --2- 172.21.15.60:0/576813430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed3c140920 0x7fed3c0d0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.018+0000 7fed42fba640 1 -- 172.21.15.60:0/576813430 >> 172.21.15.60:0/576813430 conn(0x7fed3c0b5aa0 msgr2=0x7fed3c15e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.018+0000 7fed42fba640 1 -- 172.21.15.60:0/576813430 shutdown_connections 2024-04-25T10:15:52.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.018+0000 7fed42fba640 1 -- 172.21.15.60:0/576813430 wait complete. 2024-04-25T10:15:52.041 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:14:14.444411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:14:15.153386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:52.042 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:15:52.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.162+0000 7fe2ff5ac640 1 Processor -- start 2024-04-25T10:15:52.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.162+0000 7fe2ff5ac640 1 -- start start 2024-04-25T10:15:52.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2ff5ac640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe2f815db10 0x7fe2f80b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2ff5ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f80ae140 con 0x7fe2f80cfcf0 2024-04-25T10:15:52.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2ff5ac640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f815def0 con 0x7fe2f80aec10 2024-04-25T10:15:52.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2ff5ac640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f80accc0 con 0x7fe2f815db10 2024-04-25T10:15:52.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2fdb49640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe2f80cfcf0 0x7fe2f8161d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49226/0 (socket says 172.21.15.60:49226) 2024-04-25T10:15:52.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2fdb49640 1 -- 172.21.15.60:0/2177895056 learned_addr learned my addr 172.21.15.60:0/2177895056 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:15:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2fd348640 1 --2- 172.21.15.60:0/2177895056 >> v2:172.21.15.100:3300/0 conn(0x7fe2f815db10 0x7fe2f80b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2effff640 1 -- 172.21.15.60:0/2177895056 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330929135 0 0) 0x7fe2f00014a0 con 0x7fe2f80cfcf0 2024-04-25T10:15:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2effff640 1 -- 172.21.15.60:0/2177895056 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2e4003530 con 0x7fe2f80cfcf0 2024-04-25T10:15:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2effff640 1 -- 172.21.15.60:0/2177895056 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3485748533 0 0) 0x7fe2e4003530 con 0x7fe2f80cfcf0 2024-04-25T10:15:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2effff640 1 -- 172.21.15.60:0/2177895056 >> v2:172.21.15.100:3300/0 conn(0x7fe2f815db10 msgr2=0x7fe2f80b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2effff640 1 --2- 172.21.15.60:0/2177895056 >> v2:172.21.15.100:3300/0 conn(0x7fe2f815db10 0x7fe2f80b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2effff640 1 -- 172.21.15.60:0/2177895056 >> v1:172.21.15.100:6789/0 conn(0x7fe2f80aec10 legacy=0x7fe2f8161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2effff640 1 -- 172.21.15.60:0/2177895056 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f8162480 con 0x7fe2f80cfcf0 2024-04-25T10:15:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2effff640 1 -- 172.21.15.60:0/2177895056 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fe2f0001bd0 con 0x7fe2f80cfcf0 2024-04-25T10:15:52.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2effff640 1 -- 172.21.15.60:0/2177895056 >> v1:172.21.15.60:6789/0 conn(0x7fe2f80cfcf0 legacy=0x7fe2f8161d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2effff640 1 --2- 172.21.15.60:0/2177895056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2e40039c0 0x7fe2e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2effff640 1 --2- 172.21.15.60:0/2177895056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2e4004810 0x7fe2e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2effff640 1 --2- 172.21.15.60:0/2177895056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2e40051d0 0x7fe2e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2effff640 1 -- 172.21.15.60:0/2177895056 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e400c040 con 0x7fe2e40051d0 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2effff640 1 -- 172.21.15.60:0/2177895056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e400c1f0 con 0x7fe2e40039c0 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2effff640 1 -- 172.21.15.60:0/2177895056 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e400c3f0 con 0x7fe2e4004810 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2fdb49640 1 --2- 172.21.15.60:0/2177895056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2e40051d0 0x7fe2e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2fcb47640 1 --2- 172.21.15.60:0/2177895056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2e4004810 0x7fe2e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2fd348640 1 --2- 172.21.15.60:0/2177895056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2e40039c0 0x7fe2e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2fdb49640 1 -- 172.21.15.60:0/2177895056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2e4004810 msgr2=0x7fe2e4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2fdb49640 1 --2- 172.21.15.60:0/2177895056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2e4004810 0x7fe2e4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2fdb49640 1 -- 172.21.15.60:0/2177895056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2e40039c0 msgr2=0x7fe2e4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2fdb49640 1 --2- 172.21.15.60:0/2177895056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2e40039c0 0x7fe2e4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2fdb49640 1 -- 172.21.15.60:0/2177895056 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e4003530 con 0x7fe2e40051d0 2024-04-25T10:15:52.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2fcb47640 1 --2- 172.21.15.60:0/2177895056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2e4004810 0x7fe2e4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:52.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2fd348640 1 --2- 172.21.15.60:0/2177895056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2e40039c0 0x7fe2e4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2fdb49640 1 -- 172.21.15.60:0/2177895056 reap_dead start 2024-04-25T10:15:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.166+0000 7fe2fdb49640 1 --2- 172.21.15.60:0/2177895056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2e40051d0 0x7fe2e400b9d0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000aa30 tx=0x7fe2f001d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbf7654c6e97ba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2effff640 1 -- 172.21.15.60:0/2177895056 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f000a650 con 0x7fe2e40051d0 2024-04-25T10:15:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2effff640 1 -- 172.21.15.60:0/2177895056 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f00ad650 con 0x7fe2e40051d0 2024-04-25T10:15:52.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2effff640 1 -- 172.21.15.60:0/2177895056 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f00ada40 con 0x7fe2e40051d0 2024-04-25T10:15:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 -- 172.21.15.60:0/2177895056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2e40051d0 msgr2=0x7fe2e400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 --2- 172.21.15.60:0/2177895056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2e40051d0 0x7fe2e400b9d0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000aa30 tx=0x7fe2f001d380 comp rx=0 tx=0).stop 2024-04-25T10:15:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 -- 172.21.15.60:0/2177895056 shutdown_connections 2024-04-25T10:15:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 --2- 172.21.15.60:0/2177895056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2e40051d0 0x7fe2e400b9d0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 -- 172.21.15.60:0/2177895056 >> 172.21.15.60:0/2177895056 conn(0x7fe2f80670b0 msgr2=0x7fe2f806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 -- 172.21.15.60:0/2177895056 shutdown_connections 2024-04-25T10:15:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 -- 172.21.15.60:0/2177895056 wait complete. 2024-04-25T10:15:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 Processor -- start 2024-04-25T10:15:52.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 -- start start 2024-04-25T10:15:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2e40051d0 0x7fe2f8151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2f80ceae0 0x7fe2f814f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f81f6c10 0x7fe2f81f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f8071ed0 con 0x7fe2f81f6c10 2024-04-25T10:15:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f8071bf0 con 0x7fe2f80ceae0 2024-04-25T10:15:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f806db70 con 0x7fe2e40051d0 2024-04-25T10:15:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2fdb49640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f81f6c10 0x7fe2f81f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2fdb49640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f81f6c10 0x7fe2f81f7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41678/0 (socket says 172.21.15.60:41678) 2024-04-25T10:15:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2fdb49640 1 -- 172.21.15.60:0/2777141116 learned_addr learned my addr 172.21.15.60:0/2777141116 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2fcb47640 1 --2- 172.21.15.60:0/2777141116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2f80ceae0 0x7fe2f814f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2fd348640 1 --2- 172.21.15.60:0/2777141116 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2e40051d0 0x7fe2f8151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2fdb49640 1 -- 172.21.15.60:0/2777141116 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2e40051d0 msgr2=0x7fe2f8151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2fdb49640 1 --2- 172.21.15.60:0/2777141116 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2e40051d0 0x7fe2f8151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2fdb49640 1 -- 172.21.15.60:0/2777141116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2f80ceae0 msgr2=0x7fe2f814f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2fdb49640 1 --2- 172.21.15.60:0/2777141116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2f80ceae0 0x7fe2f814f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2fdb49640 1 -- 172.21.15.60:0/2777141116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f81520b0 con 0x7fe2f81f6c10 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2fdb49640 1 --2- 172.21.15.60:0/2777141116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f81f6c10 0x7fe2f81f7090 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fe2f001dc20 tx=0x7fe2e4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2edffb640 1 -- 172.21.15.60:0/2777141116 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0001b10 con 0x7fe2f81f6c10 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2edffb640 1 -- 172.21.15.60:0/2777141116 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f00b4070 con 0x7fe2f81f6c10 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 -- 172.21.15.60:0/2777141116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f8152390 con 0x7fe2f81f6c10 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2edffb640 1 -- 172.21.15.60:0/2777141116 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0001260 con 0x7fe2f81f6c10 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.170+0000 7fe2ff5ac640 1 -- 172.21.15.60:0/2777141116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f814fa40 con 0x7fe2f81f6c10 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.174+0000 7fe2ff5ac640 1 -- 172.21.15.60:0/2777141116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f806bba0 con 0x7fe2f81f6c10 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.174+0000 7fe2edffb640 1 -- 172.21.15.60:0/2777141116 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2f00bf060 con 0x7fe2f81f6c10 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.174+0000 7fe2edffb640 1 --2- 172.21.15.60:0/2777141116 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe2dc067130 0x7fe2dc0695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.174+0000 7fe2edffb640 1 -- 172.21.15.60:0/2777141116 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fe2f011ee40 con 0x7fe2f81f6c10 2024-04-25T10:15:52.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.174+0000 7fe2fd348640 1 --2- 172.21.15.60:0/2777141116 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe2dc067130 0x7fe2dc0695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.174+0000 7fe2fd348640 1 --2- 172.21.15.60:0/2777141116 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe2dc067130 0x7fe2dc0695f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009a20 tx=0x7fe2e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.178+0000 7fe2edffb640 1 -- 172.21.15.60:0/2777141116 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2f80ae140 con 0x7fe2f81f6c10 2024-04-25T10:15:52.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.338+0000 7fe2ff5ac640 1 -- 172.21.15.60:0/2777141116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2f8074770 con 0x7fe2f81f6c10 2024-04-25T10:15:52.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.338+0000 7fe2edffb640 1 -- 172.21.15.60:0/2777141116 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2f00efd00 con 0x7fe2f81f6c10 2024-04-25T10:15:52.343 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:15:52.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.338+0000 7fe2ff5ac640 1 -- 172.21.15.60:0/2777141116 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe2dc067130 msgr2=0x7fe2dc0695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.338+0000 7fe2ff5ac640 1 --2- 172.21.15.60:0/2777141116 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe2dc067130 0x7fe2dc0695f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009a20 tx=0x7fe2e8008040 comp rx=0 tx=0).stop 2024-04-25T10:15:52.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.338+0000 7fe2ff5ac640 1 -- 172.21.15.60:0/2777141116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f81f6c10 msgr2=0x7fe2f81f7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.338+0000 7fe2ff5ac640 1 --2- 172.21.15.60:0/2777141116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f81f6c10 0x7fe2f81f7090 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fe2f001dc20 tx=0x7fe2e4005840 comp rx=0 tx=0).stop 2024-04-25T10:15:52.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.338+0000 7fe2ff5ac640 1 -- 172.21.15.60:0/2777141116 shutdown_connections 2024-04-25T10:15:52.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.338+0000 7fe2ff5ac640 1 --2- 172.21.15.60:0/2777141116 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe2dc067130 0x7fe2dc0695f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.338+0000 7fe2ff5ac640 1 --2- 172.21.15.60:0/2777141116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2f81f6c10 0x7fe2f81f7090 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.338+0000 7fe2ff5ac640 1 --2- 172.21.15.60:0/2777141116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2f80ceae0 0x7fe2f814f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.338+0000 7fe2ff5ac640 1 --2- 172.21.15.60:0/2777141116 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2e40051d0 0x7fe2f8151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.338+0000 7fe2ff5ac640 1 -- 172.21.15.60:0/2777141116 >> 172.21.15.60:0/2777141116 conn(0x7fe2f80670b0 msgr2=0x7fe2f8158210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.338+0000 7fe2ff5ac640 1 -- 172.21.15.60:0/2777141116 shutdown_connections 2024-04-25T10:15:52.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.338+0000 7fe2ff5ac640 1 -- 172.21.15.60:0/2777141116 wait complete. 2024-04-25T10:15:52.362 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:15:52.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.474+0000 7f652db7a640 1 Processor -- start 2024-04-25T10:15:52.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.474+0000 7f652db7a640 1 -- start start 2024-04-25T10:15:52.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.474+0000 7f652db7a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f65280b3b20 0x7f652815c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.474+0000 7f652db7a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f652815c630 0x7f65280c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.474+0000 7f652db7a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652815f5c0 con 0x7f65281595a0 2024-04-25T10:15:52.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.474+0000 7f652db7a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6528160380 con 0x7f652815c630 2024-04-25T10:15:52.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.474+0000 7f652db7a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f652815f8a0 con 0x7f65280b3b20 2024-04-25T10:15:52.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.474+0000 7f6527fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f652815c630 0x7f65280c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.474+0000 7f6527fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f652815c630 0x7f65280c6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41692/0 (socket says 172.21.15.60:41692) 2024-04-25T10:15:52.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.474+0000 7f6527fff640 1 -- 172.21.15.60:0/194399895 learned_addr learned my addr 172.21.15.60:0/194399895 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:52.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.474+0000 7f6526ffd640 1 --2- 172.21.15.60:0/194399895 >> v2:172.21.15.100:3300/0 conn(0x7f65280b3b20 0x7f652815c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.474+0000 7f6527fff640 1 -- 172.21.15.60:0/194399895 >> v2:172.21.15.100:3300/0 conn(0x7f65280b3b20 msgr2=0x7f652815c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.474+0000 7f6527fff640 1 --2- 172.21.15.60:0/194399895 >> v2:172.21.15.100:3300/0 conn(0x7f65280b3b20 0x7f652815c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.474+0000 7f6527fff640 1 -- 172.21.15.60:0/194399895 >> v1:172.21.15.100:6789/0 conn(0x7f65281595a0 legacy=0x7f65280b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.474+0000 7f6527fff640 1 -- 172.21.15.60:0/194399895 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652815fb80 con 0x7f652815c630 2024-04-25T10:15:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f6527fff640 1 --2- 172.21.15.60:0/194399895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f652815c630 0x7f65280c6a60 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f65280b40a0 tx=0x7f652001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892ea53506083894 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f65267fc640 1 -- 172.21.15.60:0/194399895 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65200a5780 con 0x7f652815c630 2024-04-25T10:15:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f65267fc640 1 -- 172.21.15.60:0/194399895 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65200a5920 con 0x7f652815c630 2024-04-25T10:15:52.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f65267fc640 1 -- 172.21.15.60:0/194399895 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65200a5cf0 con 0x7f652815c630 2024-04-25T10:15:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f652db7a640 1 -- 172.21.15.60:0/194399895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f652815c630 msgr2=0x7f65280c6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f652db7a640 1 --2- 172.21.15.60:0/194399895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f652815c630 0x7f65280c6a60 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f65280b40a0 tx=0x7f652001c9b0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f652db7a640 1 -- 172.21.15.60:0/194399895 shutdown_connections 2024-04-25T10:15:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f652db7a640 1 --2- 172.21.15.60:0/194399895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f652815c630 0x7f65280c6a60 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f652db7a640 1 --2- 172.21.15.60:0/194399895 >> v2:172.21.15.100:3300/0 conn(0x7f65280b3b20 0x7f652815c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f652db7a640 1 -- 172.21.15.60:0/194399895 >> 172.21.15.60:0/194399895 conn(0x7f65280669a0 msgr2=0x7f6528066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f652db7a640 1 -- 172.21.15.60:0/194399895 shutdown_connections 2024-04-25T10:15:52.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f652db7a640 1 -- 172.21.15.60:0/194399895 wait complete. 2024-04-25T10:15:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f652db7a640 1 Processor -- start 2024-04-25T10:15:52.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f652db7a640 1 -- start start 2024-04-25T10:15:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f652db7a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65280b3b20 0x7f6528149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f652db7a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65281595a0 0x7f6528154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f652db7a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f652815c630 0x7f6528143900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f652db7a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65280c7b80 con 0x7f65280b3b20 2024-04-25T10:15:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f652db7a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65280c7a00 con 0x7f652815c630 2024-04-25T10:15:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f652db7a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6528168750 con 0x7f65281595a0 2024-04-25T10:15:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f65277fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65280b3b20 0x7f6528149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f65277fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65280b3b20 0x7f6528149140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41704/0 (socket says 172.21.15.60:41704) 2024-04-25T10:15:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f65277fe640 1 -- 172.21.15.60:0/3212007896 learned_addr learned my addr 172.21.15.60:0/3212007896 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f6526ffd640 1 --2- 172.21.15.60:0/3212007896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65281595a0 0x7f6528154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f6527fff640 1 --2- 172.21.15.60:0/3212007896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f652815c630 0x7f6528143900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f65277fe640 1 -- 172.21.15.60:0/3212007896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65281595a0 msgr2=0x7f6528154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f65277fe640 1 --2- 172.21.15.60:0/3212007896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65281595a0 0x7f6528154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f65277fe640 1 -- 172.21.15.60:0/3212007896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f652815c630 msgr2=0x7f6528143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f65277fe640 1 --2- 172.21.15.60:0/3212007896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f652815c630 0x7f6528143900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.478+0000 7f65277fe640 1 -- 172.21.15.60:0/3212007896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6528148380 con 0x7f65280b3b20 2024-04-25T10:15:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.482+0000 7f65277fe640 1 --2- 172.21.15.60:0/3212007896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65280b3b20 0x7f6528149140 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f651800b440 tx=0x7f651800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.482+0000 7f650ffff640 1 -- 172.21.15.60:0/3212007896 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518015020 con 0x7f65280b3b20 2024-04-25T10:15:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.482+0000 7f650ffff640 1 -- 172.21.15.60:0/3212007896 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6518011890 con 0x7f65280b3b20 2024-04-25T10:15:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.482+0000 7f652db7a640 1 -- 172.21.15.60:0/3212007896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6528148660 con 0x7f65280b3b20 2024-04-25T10:15:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.482+0000 7f650ffff640 1 -- 172.21.15.60:0/3212007896 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518011c80 con 0x7f65280b3b20 2024-04-25T10:15:52.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.482+0000 7f652db7a640 1 -- 172.21.15.60:0/3212007896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65281445a0 con 0x7f65280b3b20 2024-04-25T10:15:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.482+0000 7f652db7a640 1 -- 172.21.15.60:0/3212007896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6528074770 con 0x7f65280b3b20 2024-04-25T10:15:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.482+0000 7f650ffff640 1 -- 172.21.15.60:0/3212007896 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6518018020 con 0x7f65280b3b20 2024-04-25T10:15:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.482+0000 7f650ffff640 1 --2- 172.21.15.60:0/3212007896 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6514066f90 0x7f6514069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.482+0000 7f650ffff640 1 -- 172.21.15.60:0/3212007896 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f6518084f30 con 0x7f65280b3b20 2024-04-25T10:15:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.482+0000 7f6526ffd640 1 --2- 172.21.15.60:0/3212007896 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6514066f90 0x7f6514069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.482+0000 7f6526ffd640 1 --2- 172.21.15.60:0/3212007896 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6514066f90 0x7f6514069450 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f65280b78b0 tx=0x7f651c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.486+0000 7f650ffff640 1 -- 172.21.15.60:0/3212007896 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f651801f030 con 0x7f65280b3b20 2024-04-25T10:15:52.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.638+0000 7f652db7a640 1 -- 172.21.15.60:0/3212007896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65280cef70 con 0x7f65280b3b20 2024-04-25T10:15:52.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.638+0000 7f650ffff640 1 -- 172.21.15.60:0/3212007896 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f651800d040 con 0x7f65280b3b20 2024-04-25T10:15:52.644 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:15:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.642+0000 7f652db7a640 1 -- 172.21.15.60:0/3212007896 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6514066f90 msgr2=0x7f6514069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.642+0000 7f652db7a640 1 --2- 172.21.15.60:0/3212007896 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6514066f90 0x7f6514069450 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f65280b78b0 tx=0x7f651c008040 comp rx=0 tx=0).stop 2024-04-25T10:15:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.642+0000 7f652db7a640 1 -- 172.21.15.60:0/3212007896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65280b3b20 msgr2=0x7f6528149140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.642+0000 7f652db7a640 1 --2- 172.21.15.60:0/3212007896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65280b3b20 0x7f6528149140 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f651800b440 tx=0x7f651800b900 comp rx=0 tx=0).stop 2024-04-25T10:15:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.642+0000 7f652db7a640 1 -- 172.21.15.60:0/3212007896 shutdown_connections 2024-04-25T10:15:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.642+0000 7f652db7a640 1 --2- 172.21.15.60:0/3212007896 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6514066f90 0x7f6514069450 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.642+0000 7f652db7a640 1 --2- 172.21.15.60:0/3212007896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f652815c630 0x7f6528143900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.642+0000 7f652db7a640 1 --2- 172.21.15.60:0/3212007896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65281595a0 0x7f6528154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.642+0000 7f652db7a640 1 --2- 172.21.15.60:0/3212007896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65280b3b20 0x7f6528149140 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.642+0000 7f652db7a640 1 -- 172.21.15.60:0/3212007896 >> 172.21.15.60:0/3212007896 conn(0x7f65280669a0 msgr2=0x7f6528066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.642+0000 7f652db7a640 1 -- 172.21.15.60:0/3212007896 shutdown_connections 2024-04-25T10:15:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.642+0000 7f652db7a640 1 -- 172.21.15.60:0/3212007896 wait complete. 2024-04-25T10:15:52.662 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:15:52.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.786+0000 7f5e91d7b640 1 Processor -- start 2024-04-25T10:15:52.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.786+0000 7f5e91d7b640 1 -- start start 2024-04-25T10:15:52.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e91d7b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5e8c0b2560 0x7f5e8c0b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e91d7b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e8c099a30 0x7f5e8c099ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e91d7b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8c0cc320 con 0x7f5e8c15ef20 2024-04-25T10:15:52.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e91d7b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e8c0cc5c0 con 0x7f5e8c0b2560 2024-04-25T10:15:52.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e91d7b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c0d07b0 con 0x7f5e8c099a30 2024-04-25T10:15:52.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5e8c15ef20 0x7f5e8c0cfcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49238/0 (socket says 172.21.15.60:49238) 2024-04-25T10:15:52.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8b7fe640 1 -- 172.21.15.60:0/748922407 learned_addr learned my addr 172.21.15.60:0/748922407 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:15:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8affd640 1 --2- 172.21.15.60:0/748922407 >> v2:172.21.15.100:3300/0 conn(0x7f5e8c0b2560 0x7f5e8c0b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8bfff640 1 --2- 172.21.15.60:0/748922407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e8c099a30 0x7f5e8c099ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8a7fc640 1 -- 172.21.15.60:0/748922407 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586221616 0 0) 0x7f5e8c0cc320 con 0x7f5e8c15ef20 2024-04-25T10:15:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8a7fc640 1 -- 172.21.15.60:0/748922407 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e74003530 con 0x7f5e8c15ef20 2024-04-25T10:15:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8a7fc640 1 -- 172.21.15.60:0/748922407 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4280438437 0 0) 0x7f5e74003530 con 0x7f5e8c15ef20 2024-04-25T10:15:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8a7fc640 1 -- 172.21.15.60:0/748922407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e8c099a30 msgr2=0x7f5e8c099ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8a7fc640 1 --2- 172.21.15.60:0/748922407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e8c099a30 0x7f5e8c099ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8a7fc640 1 -- 172.21.15.60:0/748922407 >> v2:172.21.15.100:3300/0 conn(0x7f5e8c0b2560 msgr2=0x7f5e8c0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8a7fc640 1 --2- 172.21.15.60:0/748922407 >> v2:172.21.15.100:3300/0 conn(0x7f5e8c0b2560 0x7f5e8c0b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8a7fc640 1 -- 172.21.15.60:0/748922407 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c0d04d0 con 0x7f5e8c15ef20 2024-04-25T10:15:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8a7fc640 1 -- 172.21.15.60:0/748922407 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f5e7c001680 con 0x7f5e8c15ef20 2024-04-25T10:15:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8a7fc640 1 -- 172.21.15.60:0/748922407 >> v1:172.21.15.60:6789/0 conn(0x7f5e8c15ef20 legacy=0x7f5e8c0cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8a7fc640 1 --2- 172.21.15.60:0/748922407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e740039c0 0x7f5e74003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8a7fc640 1 --2- 172.21.15.60:0/748922407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74004810 0x7f5e74004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8a7fc640 1 --2- 172.21.15.60:0/748922407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e8c099a30 0x7f5e740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8a7fc640 1 -- 172.21.15.60:0/748922407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c0cc5c0 con 0x7f5e740039c0 2024-04-25T10:15:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8a7fc640 1 -- 172.21.15.60:0/748922407 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c0d07b0 con 0x7f5e74004810 2024-04-25T10:15:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8a7fc640 1 -- 172.21.15.60:0/748922407 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7400b9d0 con 0x7f5e8c099a30 2024-04-25T10:15:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8b7fe640 1 --2- 172.21.15.60:0/748922407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e740039c0 0x7f5e74003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8affd640 1 --2- 172.21.15.60:0/748922407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74004810 0x7f5e74004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8bfff640 1 --2- 172.21.15.60:0/748922407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e8c099a30 0x7f5e740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8b7fe640 1 -- 172.21.15.60:0/748922407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e8c099a30 msgr2=0x7f5e740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8b7fe640 1 --2- 172.21.15.60:0/748922407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e8c099a30 0x7f5e740051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8b7fe640 1 -- 172.21.15.60:0/748922407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74004810 msgr2=0x7f5e74004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8b7fe640 1 --2- 172.21.15.60:0/748922407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74004810 0x7f5e74004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8b7fe640 1 -- 172.21.15.60:0/748922407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e74003530 con 0x7f5e740039c0 2024-04-25T10:15:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.790+0000 7f5e8affd640 1 --2- 172.21.15.60:0/748922407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74004810 0x7f5e74004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:52.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e8b7fe640 1 --2- 172.21.15.60:0/748922407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e740039c0 0x7f5e74003de0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00c2c0 tx=0x7f5e7c01cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9befcee6ca5f429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e8a7fc640 1 -- 172.21.15.60:0/748922407 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c007d60 con 0x7f5e740039c0 2024-04-25T10:15:52.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e8a7fc640 1 -- 172.21.15.60:0/748922407 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7c0a9020 con 0x7f5e740039c0 2024-04-25T10:15:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e8a7fc640 1 -- 172.21.15.60:0/748922407 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c005b00 con 0x7f5e740039c0 2024-04-25T10:15:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e91d7b640 1 -- 172.21.15.60:0/748922407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e740039c0 msgr2=0x7f5e74003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e91d7b640 1 --2- 172.21.15.60:0/748922407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e740039c0 0x7f5e74003de0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00c2c0 tx=0x7f5e7c01cd40 comp rx=0 tx=0).stop 2024-04-25T10:15:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e8bfff640 1 -- 172.21.15.60:0/748922407 reap_dead start 2024-04-25T10:15:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e91d7b640 1 -- 172.21.15.60:0/748922407 shutdown_connections 2024-04-25T10:15:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e91d7b640 1 -- 172.21.15.60:0/748922407 >> 172.21.15.60:0/748922407 conn(0x7f5e8c0b3200 msgr2=0x7f5e8c0b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e91d7b640 1 -- 172.21.15.60:0/748922407 shutdown_connections 2024-04-25T10:15:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e91d7b640 1 -- 172.21.15.60:0/748922407 wait complete. 2024-04-25T10:15:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e91d7b640 1 Processor -- start 2024-04-25T10:15:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e91d7b640 1 -- start start 2024-04-25T10:15:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e91d7b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e740039c0 0x7f5e8c1534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e91d7b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e8c1f8ff0 0x7f5e8c0ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e91d7b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e8c0ca930 0x7f5e8c1f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e91d7b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c09a250 con 0x7f5e8c0ca930 2024-04-25T10:15:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e91d7b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c15f300 con 0x7f5e8c1f8ff0 2024-04-25T10:15:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e91d7b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c0c9f60 con 0x7f5e740039c0 2024-04-25T10:15:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e8bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e8c0ca930 0x7f5e8c1f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e8bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e8c0ca930 0x7f5e8c1f83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41730/0 (socket says 172.21.15.60:41730) 2024-04-25T10:15:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e8bfff640 1 -- 172.21.15.60:0/2328434036 learned_addr learned my addr 172.21.15.60:0/2328434036 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e8b7fe640 1 --2- 172.21.15.60:0/2328434036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e740039c0 0x7f5e8c1534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e8affd640 1 --2- 172.21.15.60:0/2328434036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e8c1f8ff0 0x7f5e8c0ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e8bfff640 1 -- 172.21.15.60:0/2328434036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e740039c0 msgr2=0x7f5e8c1534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e8bfff640 1 --2- 172.21.15.60:0/2328434036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e740039c0 0x7f5e8c1534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e8bfff640 1 -- 172.21.15.60:0/2328434036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e8c1f8ff0 msgr2=0x7f5e8c0ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e8bfff640 1 --2- 172.21.15.60:0/2328434036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e8c1f8ff0 0x7f5e8c0ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e8bfff640 1 -- 172.21.15.60:0/2328434036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c153a30 con 0x7f5e8c0ca930 2024-04-25T10:15:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.794+0000 7f5e8bfff640 1 --2- 172.21.15.60:0/2328434036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e8c0ca930 0x7f5e8c1f83a0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c0d16e0 tx=0x7f5e8400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.798+0000 7f5e73fff640 1 -- 172.21.15.60:0/2328434036 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8401c070 con 0x7f5e8c0ca930 2024-04-25T10:15:52.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.798+0000 7f5e73fff640 1 -- 172.21.15.60:0/2328434036 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e84017400 con 0x7f5e8c0ca930 2024-04-25T10:15:52.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.798+0000 7f5e91d7b640 1 -- 172.21.15.60:0/2328434036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8c0c9c30 con 0x7f5e8c0ca930 2024-04-25T10:15:52.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.798+0000 7f5e73fff640 1 -- 172.21.15.60:0/2328434036 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e840177f0 con 0x7f5e8c0ca930 2024-04-25T10:15:52.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.798+0000 7f5e91d7b640 1 -- 172.21.15.60:0/2328434036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8c14d920 con 0x7f5e8c0ca930 2024-04-25T10:15:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.798+0000 7f5e91d7b640 1 -- 172.21.15.60:0/2328434036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8c074770 con 0x7f5e8c0ca930 2024-04-25T10:15:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.798+0000 7f5e73fff640 1 -- 172.21.15.60:0/2328434036 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e8401f070 con 0x7f5e8c0ca930 2024-04-25T10:15:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.798+0000 7f5e73fff640 1 --2- 172.21.15.60:0/2328434036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e78066f60 0x7f5e78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.798+0000 7f5e73fff640 1 -- 172.21.15.60:0/2328434036 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f5e840851e0 con 0x7f5e8c0ca930 2024-04-25T10:15:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.798+0000 7f5e8b7fe640 1 --2- 172.21.15.60:0/2328434036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e78066f60 0x7f5e78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.798+0000 7f5e8b7fe640 1 --2- 172.21.15.60:0/2328434036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e78066f60 0x7f5e78069420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0aa000 tx=0x7f5e7c0b0040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.802+0000 7f5e73fff640 1 -- 172.21.15.60:0/2328434036 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5e8c153a30 con 0x7f5e8c0ca930 2024-04-25T10:15:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.966+0000 7f5e91d7b640 1 -- 172.21.15.60:0/2328434036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e8c0b4070 con 0x7f5e8c0ca930 2024-04-25T10:15:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.970+0000 7f5e73fff640 1 -- 172.21.15.60:0/2328434036 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5e84012070 con 0x7f5e8c0ca930 2024-04-25T10:15:52.974 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:15:52.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.970+0000 7f5e91d7b640 1 -- 172.21.15.60:0/2328434036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e78066f60 msgr2=0x7f5e78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.970+0000 7f5e91d7b640 1 --2- 172.21.15.60:0/2328434036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e78066f60 0x7f5e78069420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0aa000 tx=0x7f5e7c0b0040 comp rx=0 tx=0).stop 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.970+0000 7f5e91d7b640 1 -- 172.21.15.60:0/2328434036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e8c0ca930 msgr2=0x7f5e8c1f83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.970+0000 7f5e91d7b640 1 --2- 172.21.15.60:0/2328434036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e8c0ca930 0x7f5e8c1f83a0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c0d16e0 tx=0x7f5e8400f4a0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.970+0000 7f5e91d7b640 1 -- 172.21.15.60:0/2328434036 shutdown_connections 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.970+0000 7f5e91d7b640 1 --2- 172.21.15.60:0/2328434036 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5e78066f60 0x7f5e78069420 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.970+0000 7f5e91d7b640 1 --2- 172.21.15.60:0/2328434036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e8c0ca930 0x7f5e8c1f83a0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.970+0000 7f5e91d7b640 1 --2- 172.21.15.60:0/2328434036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e8c1f8ff0 0x7f5e8c0ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.970+0000 7f5e91d7b640 1 --2- 172.21.15.60:0/2328434036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e740039c0 0x7f5e8c1534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.970+0000 7f5e91d7b640 1 -- 172.21.15.60:0/2328434036 >> 172.21.15.60:0/2328434036 conn(0x7f5e8c0b3200 msgr2=0x7f5e8c09ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.970+0000 7f5e91d7b640 1 -- 172.21.15.60:0/2328434036 shutdown_connections 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:52.970+0000 7f5e91d7b640 1 -- 172.21.15.60:0/2328434036 wait complete. 2024-04-25T10:15:52.994 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:53.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19ffd36640 1 Processor -- start 2024-04-25T10:15:53.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19ffd36640 1 -- start start 2024-04-25T10:15:53.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19ffd36640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f19f815deb0 0x7f19f80b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19ffd36640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19f80ce9c0 0x7f19f80d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19ffd36640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f806d200 con 0x7f19f80d0340 2024-04-25T10:15:53.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19ffd36640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f80cf2c0 con 0x7f19f815deb0 2024-04-25T10:15:53.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19ffd36640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80cefe0 con 0x7f19f80ce9c0 2024-04-25T10:15:53.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fd2d1640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f19f80d0340 0x7f19f80ce2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49248/0 (socket says 172.21.15.60:49248) 2024-04-25T10:15:53.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fd2d1640 1 -- 172.21.15.60:0/1317258899 learned_addr learned my addr 172.21.15.60:0/1317258899 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:15:53.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fdad2640 1 --2- 172.21.15.60:0/1317258899 >> v2:172.21.15.100:3300/0 conn(0x7f19f815deb0 0x7f19f80b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fe2d3640 1 --2- 172.21.15.60:0/1317258899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19f80ce9c0 0x7f19f80d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fcad0640 1 -- 172.21.15.60:0/1317258899 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707758990 0 0) 0x7f19f806d200 con 0x7f19f80d0340 2024-04-25T10:15:53.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fcad0640 1 -- 172.21.15.60:0/1317258899 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e4003530 con 0x7f19f80d0340 2024-04-25T10:15:53.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fcad0640 1 -- 172.21.15.60:0/1317258899 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3850673409 0 0) 0x7f19e4003530 con 0x7f19f80d0340 2024-04-25T10:15:53.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fcad0640 1 -- 172.21.15.60:0/1317258899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19f80ce9c0 msgr2=0x7f19f80d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fcad0640 1 --2- 172.21.15.60:0/1317258899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19f80ce9c0 0x7f19f80d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fcad0640 1 -- 172.21.15.60:0/1317258899 >> v2:172.21.15.100:3300/0 conn(0x7f19f815deb0 msgr2=0x7f19f80b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fcad0640 1 --2- 172.21.15.60:0/1317258899 >> v2:172.21.15.100:3300/0 conn(0x7f19f815deb0 0x7f19f80b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fcad0640 1 -- 172.21.15.60:0/1317258899 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f80d1d70 con 0x7f19f80d0340 2024-04-25T10:15:53.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fcad0640 1 -- 172.21.15.60:0/1317258899 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f19f0001680 con 0x7f19f80d0340 2024-04-25T10:15:53.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fcad0640 1 -- 172.21.15.60:0/1317258899 >> v1:172.21.15.60:6789/0 conn(0x7f19f80d0340 legacy=0x7f19f80ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fcad0640 1 --2- 172.21.15.60:0/1317258899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e40039c0 0x7f19e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fcad0640 1 --2- 172.21.15.60:0/1317258899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19f80ce9c0 0x7f19e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fcad0640 1 --2- 172.21.15.60:0/1317258899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e4004d50 0x7f19e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fcad0640 1 -- 172.21.15.60:0/1317258899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80cf2c0 con 0x7f19e40039c0 2024-04-25T10:15:53.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fcad0640 1 -- 172.21.15.60:0/1317258899 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80cefe0 con 0x7f19e4004d50 2024-04-25T10:15:53.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fcad0640 1 -- 172.21.15.60:0/1317258899 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e400b9d0 con 0x7f19f80ce9c0 2024-04-25T10:15:53.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fdad2640 1 --2- 172.21.15.60:0/1317258899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e40039c0 0x7f19e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fd2d1640 1 --2- 172.21.15.60:0/1317258899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19f80ce9c0 0x7f19e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fdad2640 1 -- 172.21.15.60:0/1317258899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19f80ce9c0 msgr2=0x7f19e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fdad2640 1 --2- 172.21.15.60:0/1317258899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19f80ce9c0 0x7f19e4004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fdad2640 1 -- 172.21.15.60:0/1317258899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e4004d50 msgr2=0x7f19e40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fdad2640 1 --2- 172.21.15.60:0/1317258899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e4004d50 0x7f19e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.110+0000 7f19fdad2640 1 -- 172.21.15.60:0/1317258899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e4003530 con 0x7f19e40039c0 2024-04-25T10:15:53.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19fdad2640 1 --2- 172.21.15.60:0/1317258899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e40039c0 0x7f19e4003de0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009fc0 tx=0x7f19ec01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f18fec6879e9aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19fcad0640 1 -- 172.21.15.60:0/1317258899 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0b2070 con 0x7f19e40039c0 2024-04-25T10:15:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19fcad0640 1 -- 172.21.15.60:0/1317258899 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec0a5ab0 con 0x7f19e40039c0 2024-04-25T10:15:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19fcad0640 1 -- 172.21.15.60:0/1317258899 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec009c10 con 0x7f19e40039c0 2024-04-25T10:15:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19ffd36640 1 -- 172.21.15.60:0/1317258899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e40039c0 msgr2=0x7f19e4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19ffd36640 1 --2- 172.21.15.60:0/1317258899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19e40039c0 0x7f19e4003de0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009fc0 tx=0x7f19ec01ccc0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19fe2d3640 1 -- 172.21.15.60:0/1317258899 reap_dead start 2024-04-25T10:15:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19ffd36640 1 -- 172.21.15.60:0/1317258899 shutdown_connections 2024-04-25T10:15:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19ffd36640 1 -- 172.21.15.60:0/1317258899 >> 172.21.15.60:0/1317258899 conn(0x7f19f806c090 msgr2=0x7f19f806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19ffd36640 1 -- 172.21.15.60:0/1317258899 shutdown_connections 2024-04-25T10:15:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19ffd36640 1 -- 172.21.15.60:0/1317258899 wait complete. 2024-04-25T10:15:53.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19ffd36640 1 Processor -- start 2024-04-25T10:15:53.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19ffd36640 1 -- start start 2024-04-25T10:15:53.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19ffd36640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e40039c0 0x7f19f80cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19ffd36640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19f814ba40 0x7f19f8164410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19ffd36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8164c60 0x7f19f81650e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19ffd36640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80d31e0 con 0x7f19f8164c60 2024-04-25T10:15:53.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19ffd36640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80d3060 con 0x7f19e40039c0 2024-04-25T10:15:53.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.114+0000 7f19ffd36640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80d3360 con 0x7f19f814ba40 2024-04-25T10:15:53.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19fe2d3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8164c60 0x7f19f81650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19fe2d3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8164c60 0x7f19f81650e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41740/0 (socket says 172.21.15.60:41740) 2024-04-25T10:15:53.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19fe2d3640 1 -- 172.21.15.60:0/3277819767 learned_addr learned my addr 172.21.15.60:0/3277819767 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:53.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19fdad2640 1 --2- 172.21.15.60:0/3277819767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e40039c0 0x7f19f80cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19fd2d1640 1 --2- 172.21.15.60:0/3277819767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19f814ba40 0x7f19f8164410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19fe2d3640 1 -- 172.21.15.60:0/3277819767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19f814ba40 msgr2=0x7f19f8164410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19fe2d3640 1 --2- 172.21.15.60:0/3277819767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19f814ba40 0x7f19f8164410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19fe2d3640 1 -- 172.21.15.60:0/3277819767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e40039c0 msgr2=0x7f19f80cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19fe2d3640 1 --2- 172.21.15.60:0/3277819767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e40039c0 0x7f19f80cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19fe2d3640 1 -- 172.21.15.60:0/3277819767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f80cd9f0 con 0x7f19f8164c60 2024-04-25T10:15:53.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19fe2d3640 1 --2- 172.21.15.60:0/3277819767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8164c60 0x7f19f81650e0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f19f80b6520 tx=0x7f19f400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19ea7fc640 1 -- 172.21.15.60:0/3277819767 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f400c620 con 0x7f19f8164c60 2024-04-25T10:15:53.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19ea7fc640 1 -- 172.21.15.60:0/3277819767 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f401a070 con 0x7f19f8164c60 2024-04-25T10:15:53.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19ffd36640 1 -- 172.21.15.60:0/3277819767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f80cb540 con 0x7f19f8164c60 2024-04-25T10:15:53.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19ea7fc640 1 -- 172.21.15.60:0/3277819767 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f400ca20 con 0x7f19f8164c60 2024-04-25T10:15:53.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19ffd36640 1 -- 172.21.15.60:0/3277819767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f80cbb00 con 0x7f19f8164c60 2024-04-25T10:15:53.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19ffd36640 1 -- 172.21.15.60:0/3277819767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f8074770 con 0x7f19f8164c60 2024-04-25T10:15:53.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19ea7fc640 1 -- 172.21.15.60:0/3277819767 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19f400cbc0 con 0x7f19f8164c60 2024-04-25T10:15:53.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19ea7fc640 1 --2- 172.21.15.60:0/3277819767 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f19dc066f60 0x7f19dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19ea7fc640 1 -- 172.21.15.60:0/3277819767 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f19f4084900 con 0x7f19f8164c60 2024-04-25T10:15:53.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19fdad2640 1 --2- 172.21.15.60:0/3277819767 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f19dc066f60 0x7f19dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.118+0000 7f19fdad2640 1 --2- 172.21.15.60:0/3277819767 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f19dc066f60 0x7f19dc069420 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0af000 tx=0x7f19ec0a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.122+0000 7f19ea7fc640 1 -- 172.21.15.60:0/3277819767 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f19f4023020 con 0x7f19f8164c60 2024-04-25T10:15:53.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.318+0000 7f19ffd36640 1 -- 172.21.15.60:0/3277819767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19f80d17a0 con 0x7f19f8164c60 2024-04-25T10:15:53.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.318+0000 7f19ea7fc640 1 -- 172.21.15.60:0/3277819767 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4936 (secure 0 0 0) 0x7f19f4055860 con 0x7f19f8164c60 2024-04-25T10:15:53.323 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:15:53.323 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1491228634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1491228634},{"type":"v1","addr":"172.21.15.100:6837","nonce":1491228634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/209898872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":209898872},{"type":"v1","addr":"172.21.15.100:6835","nonce":209898872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1793343287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1793343287},{"type":"v1","addr":"172.21.15.60:6833","nonce":1793343287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":9198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1365405879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1365405879},{"type":"v1","addr":"172.21.15.60:6835","nonce":1365405879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":84,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:16.146481+0000","modified":"2024-04-25T10:15:50.915413+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:15:53.323 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 85 2024-04-25T10:15:53.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.318+0000 7f19ffd36640 1 -- 172.21.15.60:0/3277819767 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f19dc066f60 msgr2=0x7f19dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.318+0000 7f19ffd36640 1 --2- 172.21.15.60:0/3277819767 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f19dc066f60 0x7f19dc069420 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0af000 tx=0x7f19ec0a7040 comp rx=0 tx=0).stop 2024-04-25T10:15:53.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.318+0000 7f19ffd36640 1 -- 172.21.15.60:0/3277819767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8164c60 msgr2=0x7f19f81650e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.318+0000 7f19ffd36640 1 --2- 172.21.15.60:0/3277819767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8164c60 0x7f19f81650e0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f19f80b6520 tx=0x7f19f400ec00 comp rx=0 tx=0).stop 2024-04-25T10:15:53.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.318+0000 7f19ffd36640 1 -- 172.21.15.60:0/3277819767 shutdown_connections 2024-04-25T10:15:53.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.318+0000 7f19ffd36640 1 --2- 172.21.15.60:0/3277819767 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f19dc066f60 0x7f19dc069420 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.318+0000 7f19ffd36640 1 --2- 172.21.15.60:0/3277819767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19f8164c60 0x7f19f81650e0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.318+0000 7f19ffd36640 1 --2- 172.21.15.60:0/3277819767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19f814ba40 0x7f19f8164410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.318+0000 7f19ffd36640 1 --2- 172.21.15.60:0/3277819767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19e40039c0 0x7f19f80cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.318+0000 7f19ffd36640 1 -- 172.21.15.60:0/3277819767 >> 172.21.15.60:0/3277819767 conn(0x7f19f806c090 msgr2=0x7f19f8150c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.318+0000 7f19ffd36640 1 -- 172.21.15.60:0/3277819767 shutdown_connections 2024-04-25T10:15:53.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.318+0000 7f19ffd36640 1 -- 172.21.15.60:0/3277819767 wait complete. 2024-04-25T10:15:53.343 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:53.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.466+0000 7f5294d76640 1 Processor -- start 2024-04-25T10:15:53.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.466+0000 7f5294d76640 1 -- start start 2024-04-25T10:15:53.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.466+0000 7f5294d76640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5290161290 0x7f52900b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.466+0000 7f5294d76640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f52900b6fb0 0x7f52900b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.466+0000 7f5294d76640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52900b9900 con 0x7f5290160eb0 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.466+0000 7f5294d76640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52900ce9c0 con 0x7f52900b6fb0 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.466+0000 7f5294d76640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52900ceca0 con 0x7f5290161290 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.466+0000 7f528ed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f52900b6fb0 0x7f52900b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.466+0000 7f528e59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5290160eb0 0x7f52900b91f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49256/0 (socket says 172.21.15.60:49256) 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.466+0000 7f528e59c640 1 -- 172.21.15.60:0/2388040439 learned_addr learned my addr 172.21.15.60:0/2388040439 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.466+0000 7f528dd9b640 1 --2- 172.21.15.60:0/2388040439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5290161290 0x7f52900b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.466+0000 7f528ed9d640 1 -- 172.21.15.60:0/2388040439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5290161290 msgr2=0x7f52900b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.466+0000 7f528ed9d640 1 --2- 172.21.15.60:0/2388040439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5290161290 0x7f52900b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.466+0000 7f528ed9d640 1 -- 172.21.15.60:0/2388040439 >> v1:172.21.15.60:6789/0 conn(0x7f5290160eb0 legacy=0x7f52900b91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.466+0000 7f528d59a640 1 -- 172.21.15.60:0/2388040439 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229434206 0 0) 0x7f52900b9900 con 0x7f5290160eb0 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.466+0000 7f528ed9d640 1 -- 172.21.15.60:0/2388040439 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52900d2490 con 0x7f52900b6fb0 2024-04-25T10:15:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f528ed9d640 1 --2- 172.21.15.60:0/2388040439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52900b6fb0 0x7f52900b53f0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f5288009d90 tx=0x7f528801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02ab51da70a2b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f528d59a640 1 -- 172.21.15.60:0/2388040439 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52880b4070 con 0x7f52900b6fb0 2024-04-25T10:15:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f528d59a640 1 -- 172.21.15.60:0/2388040439 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52880af600 con 0x7f52900b6fb0 2024-04-25T10:15:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f528d59a640 1 -- 172.21.15.60:0/2388040439 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52880af9d0 con 0x7f52900b6fb0 2024-04-25T10:15:53.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f5294d76640 1 -- 172.21.15.60:0/2388040439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52900b6fb0 msgr2=0x7f52900b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f5294d76640 1 --2- 172.21.15.60:0/2388040439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52900b6fb0 0x7f52900b53f0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f5288009d90 tx=0x7f528801f150 comp rx=0 tx=0).stop 2024-04-25T10:15:53.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f5294d76640 1 -- 172.21.15.60:0/2388040439 shutdown_connections 2024-04-25T10:15:53.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f5294d76640 1 --2- 172.21.15.60:0/2388040439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52900b6fb0 0x7f52900b53f0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f5294d76640 1 --2- 172.21.15.60:0/2388040439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5290161290 0x7f52900b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f5294d76640 1 -- 172.21.15.60:0/2388040439 >> 172.21.15.60:0/2388040439 conn(0x7f52900d05c0 msgr2=0x7f52900d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f5294d76640 1 -- 172.21.15.60:0/2388040439 shutdown_connections 2024-04-25T10:15:53.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f5294d76640 1 -- 172.21.15.60:0/2388040439 wait complete. 2024-04-25T10:15:53.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f5294d76640 1 Processor -- start 2024-04-25T10:15:53.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f5294d76640 1 -- start start 2024-04-25T10:15:53.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f5294d76640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52900b6fb0 0x7f52901f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f5294d76640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5290160eb0 0x7f52900ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f5294d76640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5290161290 0x7f5290155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f5294d76640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52900cef80 con 0x7f5290160eb0 2024-04-25T10:15:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f5294d76640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52900b2e70 con 0x7f5290161290 2024-04-25T10:15:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f5294d76640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52900b5930 con 0x7f52900b6fb0 2024-04-25T10:15:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f528dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5290160eb0 0x7f52900ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f528dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5290160eb0 0x7f52900ac710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41764/0 (socket says 172.21.15.60:41764) 2024-04-25T10:15:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f528dd9b640 1 -- 172.21.15.60:0/3751415745 learned_addr learned my addr 172.21.15.60:0/3751415745 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f528ed9d640 1 --2- 172.21.15.60:0/3751415745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5290161290 0x7f5290155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f528e59c640 1 --2- 172.21.15.60:0/3751415745 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52900b6fb0 0x7f52901f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f528dd9b640 1 -- 172.21.15.60:0/3751415745 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52900b6fb0 msgr2=0x7f52901f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f528dd9b640 1 --2- 172.21.15.60:0/3751415745 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52900b6fb0 0x7f52901f77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f528dd9b640 1 -- 172.21.15.60:0/3751415745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5290161290 msgr2=0x7f5290155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f528dd9b640 1 --2- 172.21.15.60:0/3751415745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5290161290 0x7f5290155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f528dd9b640 1 -- 172.21.15.60:0/3751415745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52900ab670 con 0x7f5290160eb0 2024-04-25T10:15:53.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.470+0000 7f528dd9b640 1 --2- 172.21.15.60:0/3751415745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5290160eb0 0x7f52900ac710 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f528400d5f0 tx=0x7f528400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.474+0000 7f527effd640 1 -- 172.21.15.60:0/3751415745 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52840125f0 con 0x7f5290160eb0 2024-04-25T10:15:53.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.474+0000 7f527effd640 1 -- 172.21.15.60:0/3751415745 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5284019070 con 0x7f5290160eb0 2024-04-25T10:15:53.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.474+0000 7f5294d76640 1 -- 172.21.15.60:0/3751415745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52900ab950 con 0x7f5290160eb0 2024-04-25T10:15:53.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.474+0000 7f527effd640 1 -- 172.21.15.60:0/3751415745 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52840129b0 con 0x7f5290160eb0 2024-04-25T10:15:53.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.474+0000 7f5294d76640 1 -- 172.21.15.60:0/3751415745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52900abf10 con 0x7f5290160eb0 2024-04-25T10:15:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.474+0000 7f5294d76640 1 -- 172.21.15.60:0/3751415745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52900ac400 con 0x7f5290160eb0 2024-04-25T10:15:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.474+0000 7f527effd640 1 -- 172.21.15.60:0/3751415745 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5284012b50 con 0x7f5290160eb0 2024-04-25T10:15:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.474+0000 7f527effd640 1 --2- 172.21.15.60:0/3751415745 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5270066f60 0x7f5270069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.474+0000 7f527effd640 1 -- 172.21.15.60:0/3751415745 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f5284085470 con 0x7f5290160eb0 2024-04-25T10:15:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.474+0000 7f528e59c640 1 --2- 172.21.15.60:0/3751415745 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5270066f60 0x7f5270069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.474+0000 7f528e59c640 1 --2- 172.21.15.60:0/3751415745 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5270066f60 0x7f5270069420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5280000f70 tx=0x7f5280007b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.478+0000 7f527effd640 1 -- 172.21.15.60:0/3751415745 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5284056460 con 0x7f5290160eb0 2024-04-25T10:15:53.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.638+0000 7f5294d76640 1 -- 172.21.15.60:0/3751415745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5290074770 con 0x7f5290160eb0 2024-04-25T10:15:53.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.642+0000 7f527effd640 1 -- 172.21.15.60:0/3751415745 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16879 (secure 0 0 0) 0x7f5284055d60 con 0x7f5290160eb0 2024-04-25T10:15:53.646 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:15:53.646 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":99,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:15:50.915440+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:14.444411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:15.153386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/4149609651":"2024-04-26T10:15:50.915390+0000","172.21.15.60:6834/4149609651":"2024-04-26T10:15:50.915390+0000","172.21.15.60:6835/2002755047":"2024-04-26T10:15:14.408136+0000","172.21.15.60:6834/2002755047":"2024-04-26T10:15:14.408136+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:53.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.642+0000 7f5294d76640 1 -- 172.21.15.60:0/3751415745 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5270066f60 msgr2=0x7f5270069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.642+0000 7f5294d76640 1 --2- 172.21.15.60:0/3751415745 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5270066f60 0x7f5270069420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5280000f70 tx=0x7f5280007b60 comp rx=0 tx=0).stop 2024-04-25T10:15:53.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.642+0000 7f5294d76640 1 -- 172.21.15.60:0/3751415745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5290160eb0 msgr2=0x7f52900ac710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.642+0000 7f5294d76640 1 --2- 172.21.15.60:0/3751415745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5290160eb0 0x7f52900ac710 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f528400d5f0 tx=0x7f528400dab0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.642+0000 7f5294d76640 1 -- 172.21.15.60:0/3751415745 shutdown_connections 2024-04-25T10:15:53.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.642+0000 7f5294d76640 1 --2- 172.21.15.60:0/3751415745 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5270066f60 0x7f5270069420 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.642+0000 7f5294d76640 1 --2- 172.21.15.60:0/3751415745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5290161290 0x7f5290155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.642+0000 7f5294d76640 1 --2- 172.21.15.60:0/3751415745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5290160eb0 0x7f52900ac710 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.642+0000 7f5294d76640 1 --2- 172.21.15.60:0/3751415745 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52900b6fb0 0x7f52901f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.642+0000 7f5294d76640 1 -- 172.21.15.60:0/3751415745 >> 172.21.15.60:0/3751415745 conn(0x7f52900d05c0 msgr2=0x7f52900d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.642+0000 7f5294d76640 1 -- 172.21.15.60:0/3751415745 shutdown_connections 2024-04-25T10:15:53.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.642+0000 7f5294d76640 1 -- 172.21.15.60:0/3751415745 wait complete. 2024-04-25T10:15:53.666 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:15:53.666 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:53.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.786+0000 7fd401ff6640 1 Processor -- start 2024-04-25T10:15:53.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.786+0000 7fd401ff6640 1 -- start start 2024-04-25T10:15:53.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.786+0000 7fd401ff6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd3fc0ce7a0 0x7fd3fc0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.786+0000 7fd401ff6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd3fc15bc90 0x7fd3fc0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.786+0000 7fd401ff6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3fc0cf0c0 con 0x7fd3fc0b4660 2024-04-25T10:15:53.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.786+0000 7fd401ff6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3fc0cc370 con 0x7fd3fc15bc90 2024-04-25T10:15:53.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.786+0000 7fd401ff6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3fc071ca0 con 0x7fd3fc0ce7a0 2024-04-25T10:15:53.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.786+0000 7fd3faffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd3fc15bc90 0x7fd3fc0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.786+0000 7fd3faffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd3fc15bc90 0x7fd3fc0b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41772/0 (socket says 172.21.15.60:41772) 2024-04-25T10:15:53.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.786+0000 7fd3faffd640 1 -- 172.21.15.60:0/4188013809 learned_addr learned my addr 172.21.15.60:0/4188013809 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:53.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.786+0000 7fd3fb7fe640 1 --2- 172.21.15.60:0/4188013809 >> v2:172.21.15.100:3300/0 conn(0x7fd3fc0ce7a0 0x7fd3fc0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.786+0000 7fd3faffd640 1 -- 172.21.15.60:0/4188013809 >> v2:172.21.15.100:3300/0 conn(0x7fd3fc0ce7a0 msgr2=0x7fd3fc0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.786+0000 7fd3faffd640 1 --2- 172.21.15.60:0/4188013809 >> v2:172.21.15.100:3300/0 conn(0x7fd3fc0ce7a0 0x7fd3fc0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.786+0000 7fd3faffd640 1 -- 172.21.15.60:0/4188013809 >> v1:172.21.15.100:6789/0 conn(0x7fd3fc0b4660 legacy=0x7fd3fc0b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.786+0000 7fd3faffd640 1 -- 172.21.15.60:0/4188013809 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3fc0cca40 con 0x7fd3fc15bc90 2024-04-25T10:15:53.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd3faffd640 1 --2- 172.21.15.60:0/4188013809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc15bc90 0x7fd3fc0b25d0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc04f5d0 tx=0x7fd3f001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bf08c6d2b6caff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd3fa7fc640 1 -- 172.21.15.60:0/4188013809 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f00b0070 con 0x7fd3fc15bc90 2024-04-25T10:15:53.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd3fa7fc640 1 -- 172.21.15.60:0/4188013809 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f00ab600 con 0x7fd3fc15bc90 2024-04-25T10:15:53.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd3fa7fc640 1 -- 172.21.15.60:0/4188013809 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f00ab9d0 con 0x7fd3fc15bc90 2024-04-25T10:15:53.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd401ff6640 1 -- 172.21.15.60:0/4188013809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc15bc90 msgr2=0x7fd3fc0b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd401ff6640 1 --2- 172.21.15.60:0/4188013809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc15bc90 0x7fd3fc0b25d0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc04f5d0 tx=0x7fd3f001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd401ff6640 1 -- 172.21.15.60:0/4188013809 shutdown_connections 2024-04-25T10:15:53.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd401ff6640 1 --2- 172.21.15.60:0/4188013809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc15bc90 0x7fd3fc0b25d0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd401ff6640 1 --2- 172.21.15.60:0/4188013809 >> v2:172.21.15.100:3300/0 conn(0x7fd3fc0ce7a0 0x7fd3fc0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd401ff6640 1 -- 172.21.15.60:0/4188013809 >> 172.21.15.60:0/4188013809 conn(0x7fd3fc0b12d0 msgr2=0x7fd3fc0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd401ff6640 1 -- 172.21.15.60:0/4188013809 shutdown_connections 2024-04-25T10:15:53.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd401ff6640 1 -- 172.21.15.60:0/4188013809 wait complete. 2024-04-25T10:15:53.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd401ff6640 1 Processor -- start 2024-04-25T10:15:53.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd401ff6640 1 -- start start 2024-04-25T10:15:53.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd401ff6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc0ce7a0 0x7fd3fc15a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd401ff6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3fc15bc90 0x7fd3fc159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd401ff6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3fc159ab0 0x7fd3fc0b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd401ff6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc0b4400 con 0x7fd3fc0ce7a0 2024-04-25T10:15:53.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd401ff6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc0b4280 con 0x7fd3fc159ab0 2024-04-25T10:15:53.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd401ff6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc0b3220 con 0x7fd3fc15bc90 2024-04-25T10:15:53.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd3fb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc0ce7a0 0x7fd3fc15a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd3fb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc0ce7a0 0x7fd3fc15a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41780/0 (socket says 172.21.15.60:41780) 2024-04-25T10:15:53.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.790+0000 7fd3fb7fe640 1 -- 172.21.15.60:0/146764823 learned_addr learned my addr 172.21.15.60:0/146764823 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:53.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd3fbfff640 1 --2- 172.21.15.60:0/146764823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3fc159ab0 0x7fd3fc0b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd3faffd640 1 --2- 172.21.15.60:0/146764823 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3fc15bc90 0x7fd3fc159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd3fb7fe640 1 -- 172.21.15.60:0/146764823 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3fc15bc90 msgr2=0x7fd3fc159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd3fb7fe640 1 --2- 172.21.15.60:0/146764823 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3fc15bc90 0x7fd3fc159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd3fb7fe640 1 -- 172.21.15.60:0/146764823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3fc159ab0 msgr2=0x7fd3fc0b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd3fb7fe640 1 --2- 172.21.15.60:0/146764823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3fc159ab0 0x7fd3fc0b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd3fb7fe640 1 -- 172.21.15.60:0/146764823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3fc06e130 con 0x7fd3fc0ce7a0 2024-04-25T10:15:53.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd3fb7fe640 1 --2- 172.21.15.60:0/146764823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc0ce7a0 0x7fd3fc15a530 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00d830 tx=0x7fd3ec00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd3e3fff640 1 -- 172.21.15.60:0/146764823 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec00b6c0 con 0x7fd3fc0ce7a0 2024-04-25T10:15:53.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd3e3fff640 1 -- 172.21.15.60:0/146764823 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ec019070 con 0x7fd3fc0ce7a0 2024-04-25T10:15:53.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd401ff6640 1 -- 172.21.15.60:0/146764823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3fc06e410 con 0x7fd3fc0ce7a0 2024-04-25T10:15:53.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd3e3fff640 1 -- 172.21.15.60:0/146764823 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec00ba60 con 0x7fd3fc0ce7a0 2024-04-25T10:15:53.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd401ff6640 1 -- 172.21.15.60:0/146764823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3fc06e9d0 con 0x7fd3fc0ce7a0 2024-04-25T10:15:53.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd401ff6640 1 -- 172.21.15.60:0/146764823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3fc074770 con 0x7fd3fc0ce7a0 2024-04-25T10:15:53.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd3e3fff640 1 -- 172.21.15.60:0/146764823 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd3ec01e460 con 0x7fd3fc0ce7a0 2024-04-25T10:15:53.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd3e3fff640 1 --2- 172.21.15.60:0/146764823 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd3e8066f90 0x7fd3e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd3e3fff640 1 -- 172.21.15.60:0/146764823 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fd3ec0860e0 con 0x7fd3fc0ce7a0 2024-04-25T10:15:53.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.794+0000 7fd3faffd640 1 --2- 172.21.15.60:0/146764823 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd3e8066f90 0x7fd3e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.798+0000 7fd3faffd640 1 --2- 172.21.15.60:0/146764823 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd3e8066f90 0x7fd3e8069450 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00ad000 tx=0x7fd3f00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.798+0000 7fd3e3fff640 1 -- 172.21.15.60:0/146764823 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd3ec0571b0 con 0x7fd3fc0ce7a0 2024-04-25T10:15:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.998+0000 7fd401ff6640 1 -- 172.21.15.60:0/146764823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3fc0b37a0 con 0x7fd3fc0ce7a0 2024-04-25T10:15:54.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:53.998+0000 7fd3e3fff640 1 -- 172.21.15.60:0/146764823 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4936 (secure 0 0 0) 0x7fd3ec056ab0 con 0x7fd3fc0ce7a0 2024-04-25T10:15:54.004 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:15:54.004 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1491228634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1491228634},{"type":"v1","addr":"172.21.15.100:6837","nonce":1491228634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/209898872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":209898872},{"type":"v1","addr":"172.21.15.100:6835","nonce":209898872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1793343287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1793343287},{"type":"v1","addr":"172.21.15.60:6833","nonce":1793343287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":9198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1365405879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1365405879},{"type":"v1","addr":"172.21.15.60:6835","nonce":1365405879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":84,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:16.146481+0000","modified":"2024-04-25T10:15:50.915413+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:15:54.005 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 85 2024-04-25T10:15:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.002+0000 7fd401ff6640 1 -- 172.21.15.60:0/146764823 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd3e8066f90 msgr2=0x7fd3e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.002+0000 7fd401ff6640 1 --2- 172.21.15.60:0/146764823 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd3e8066f90 0x7fd3e8069450 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00ad000 tx=0x7fd3f00a6000 comp rx=0 tx=0).stop 2024-04-25T10:15:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.002+0000 7fd401ff6640 1 -- 172.21.15.60:0/146764823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc0ce7a0 msgr2=0x7fd3fc15a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.002+0000 7fd401ff6640 1 --2- 172.21.15.60:0/146764823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc0ce7a0 0x7fd3fc15a530 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00d830 tx=0x7fd3ec00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.002+0000 7fd401ff6640 1 -- 172.21.15.60:0/146764823 shutdown_connections 2024-04-25T10:15:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.002+0000 7fd401ff6640 1 --2- 172.21.15.60:0/146764823 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd3e8066f90 0x7fd3e8069450 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.002+0000 7fd401ff6640 1 --2- 172.21.15.60:0/146764823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3fc159ab0 0x7fd3fc0b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.002+0000 7fd401ff6640 1 --2- 172.21.15.60:0/146764823 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3fc15bc90 0x7fd3fc159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.002+0000 7fd401ff6640 1 --2- 172.21.15.60:0/146764823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3fc0ce7a0 0x7fd3fc15a530 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.002+0000 7fd401ff6640 1 -- 172.21.15.60:0/146764823 >> 172.21.15.60:0/146764823 conn(0x7fd3fc0b12d0 msgr2=0x7fd3fc15b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.002+0000 7fd401ff6640 1 -- 172.21.15.60:0/146764823 shutdown_connections 2024-04-25T10:15:54.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.002+0000 7fd401ff6640 1 -- 172.21.15.60:0/146764823 wait complete. 2024-04-25T10:15:54.024 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:54.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.138+0000 7f4c4d8a5640 1 Processor -- start 2024-04-25T10:15:54.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.138+0000 7f4c4d8a5640 1 -- start start 2024-04-25T10:15:54.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c4d8a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c4806ec30 0x7f4c480cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c4d8a5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4c480b37e0 0x7f4c480b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c4d8a5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4806f130 con 0x7f4c4815a7a0 2024-04-25T10:15:54.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c4d8a5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c480cdaa0 con 0x7f4c480b37e0 2024-04-25T10:15:54.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c4d8a5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4806d2f0 con 0x7f4c4806ec30 2024-04-25T10:15:54.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c477fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4c480b37e0 0x7f4c480b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c477fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4c480b37e0 0x7f4c480b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41792/0 (socket says 172.21.15.60:41792) 2024-04-25T10:15:54.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c477fe640 1 -- 172.21.15.60:0/2191120547 learned_addr learned my addr 172.21.15.60:0/2191120547 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:54.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c467fc640 1 --2- 172.21.15.60:0/2191120547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c4806ec30 0x7f4c480cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c477fe640 1 -- 172.21.15.60:0/2191120547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c4806ec30 msgr2=0x7f4c480cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:54.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c477fe640 1 --2- 172.21.15.60:0/2191120547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c4806ec30 0x7f4c480cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c477fe640 1 -- 172.21.15.60:0/2191120547 >> v1:172.21.15.100:6789/0 conn(0x7f4c4815a7a0 legacy=0x7f4c48071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c477fe640 1 -- 172.21.15.60:0/2191120547 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4806f410 con 0x7f4c480b37e0 2024-04-25T10:15:54.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c477fe640 1 --2- 172.21.15.60:0/2191120547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c480b37e0 0x7f4c480b3c90 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009e50 tx=0x7f4c4001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4116584cbfa5f4f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c45ffb640 1 -- 172.21.15.60:0/2191120547 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c400a5810 con 0x7f4c480b37e0 2024-04-25T10:15:54.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c45ffb640 1 -- 172.21.15.60:0/2191120547 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c400a59b0 con 0x7f4c480b37e0 2024-04-25T10:15:54.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.142+0000 7f4c45ffb640 1 -- 172.21.15.60:0/2191120547 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c400a5d80 con 0x7f4c480b37e0 2024-04-25T10:15:54.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 -- 172.21.15.60:0/2191120547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c480b37e0 msgr2=0x7f4c480b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 --2- 172.21.15.60:0/2191120547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c480b37e0 0x7f4c480b3c90 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009e50 tx=0x7f4c4001ca30 comp rx=0 tx=0).stop 2024-04-25T10:15:54.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 -- 172.21.15.60:0/2191120547 shutdown_connections 2024-04-25T10:15:54.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 --2- 172.21.15.60:0/2191120547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c480b37e0 0x7f4c480b3c90 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 --2- 172.21.15.60:0/2191120547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c4806ec30 0x7f4c480cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 -- 172.21.15.60:0/2191120547 >> 172.21.15.60:0/2191120547 conn(0x7f4c480b12d0 msgr2=0x7f4c480b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:54.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 -- 172.21.15.60:0/2191120547 shutdown_connections 2024-04-25T10:15:54.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 -- 172.21.15.60:0/2191120547 wait complete. 2024-04-25T10:15:54.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 Processor -- start 2024-04-25T10:15:54.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 -- start start 2024-04-25T10:15:54.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c4806ec30 0x7f4c4815d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c480b37e0 0x7f4c48151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c4815a7a0 0x7f4c480ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4806e8c0 con 0x7f4c4815a7a0 2024-04-25T10:15:54.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4806e740 con 0x7f4c480b37e0 2024-04-25T10:15:54.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4806d8b0 con 0x7f4c4806ec30 2024-04-25T10:15:54.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c477fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c4815a7a0 0x7f4c480ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c477fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c4815a7a0 0x7f4c480ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41802/0 (socket says 172.21.15.60:41802) 2024-04-25T10:15:54.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c477fe640 1 -- 172.21.15.60:0/3328626749 learned_addr learned my addr 172.21.15.60:0/3328626749 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:54.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c467fc640 1 --2- 172.21.15.60:0/3328626749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c480b37e0 0x7f4c48151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c46ffd640 1 --2- 172.21.15.60:0/3328626749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c4806ec30 0x7f4c4815d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c477fe640 1 -- 172.21.15.60:0/3328626749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c4806ec30 msgr2=0x7f4c4815d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c477fe640 1 --2- 172.21.15.60:0/3328626749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c4806ec30 0x7f4c4815d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c477fe640 1 -- 172.21.15.60:0/3328626749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c480b37e0 msgr2=0x7f4c48151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c477fe640 1 --2- 172.21.15.60:0/3328626749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c480b37e0 0x7f4c48151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c477fe640 1 -- 172.21.15.60:0/3328626749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4815d830 con 0x7f4c4815a7a0 2024-04-25T10:15:54.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c477fe640 1 --2- 172.21.15.60:0/3328626749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c4815a7a0 0x7f4c480ce780 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f4c400af040 tx=0x7f4c400197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c2ffff640 1 -- 172.21.15.60:0/3328626749 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c400b8670 con 0x7f4c4815a7a0 2024-04-25T10:15:54.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c2ffff640 1 -- 172.21.15.60:0/3328626749 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c400be070 con 0x7f4c4815a7a0 2024-04-25T10:15:54.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 -- 172.21.15.60:0/3328626749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4815db10 con 0x7f4c4815a7a0 2024-04-25T10:15:54.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c2ffff640 1 -- 172.21.15.60:0/3328626749 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c400b8a90 con 0x7f4c4815a7a0 2024-04-25T10:15:54.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.146+0000 7f4c4d8a5640 1 -- 172.21.15.60:0/3328626749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4814f790 con 0x7f4c4815a7a0 2024-04-25T10:15:54.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.150+0000 7f4c4d8a5640 1 -- 172.21.15.60:0/3328626749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c48074770 con 0x7f4c4815a7a0 2024-04-25T10:15:54.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.150+0000 7f4c2ffff640 1 -- 172.21.15.60:0/3328626749 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4c400b8c30 con 0x7f4c4815a7a0 2024-04-25T10:15:54.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.150+0000 7f4c2ffff640 1 --2- 172.21.15.60:0/3328626749 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4c34066f90 0x7f4c34069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.150+0000 7f4c2ffff640 1 -- 172.21.15.60:0/3328626749 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f4c4011fe50 con 0x7f4c4815a7a0 2024-04-25T10:15:54.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.150+0000 7f4c46ffd640 1 --2- 172.21.15.60:0/3328626749 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4c34066f90 0x7f4c34069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.150+0000 7f4c46ffd640 1 --2- 172.21.15.60:0/3328626749 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4c34066f90 0x7f4c34069450 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4c380015b0 tx=0x7f4c38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.154+0000 7f4c2ffff640 1 -- 172.21.15.60:0/3328626749 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4c40016620 con 0x7f4c4815a7a0 2024-04-25T10:15:54.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.346+0000 7f4c4d8a5640 1 -- 172.21.15.60:0/3328626749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c4815e6c0 con 0x7f4c4815a7a0 2024-04-25T10:15:54.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.346+0000 7f4c2ffff640 1 -- 172.21.15.60:0/3328626749 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4936 (secure 0 0 0) 0x7f4c400f0d10 con 0x7f4c4815a7a0 2024-04-25T10:15:54.352 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:15:54.353 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1491228634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1491228634},{"type":"v1","addr":"172.21.15.100:6837","nonce":1491228634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/209898872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":209898872},{"type":"v1","addr":"172.21.15.100:6835","nonce":209898872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1793343287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1793343287},{"type":"v1","addr":"172.21.15.60:6833","nonce":1793343287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":9198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1365405879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1365405879},{"type":"v1","addr":"172.21.15.60:6835","nonce":1365405879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":84,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:16.146481+0000","modified":"2024-04-25T10:15:50.915413+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:15:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 85 2024-04-25T10:15:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.350+0000 7f4c4d8a5640 1 -- 172.21.15.60:0/3328626749 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4c34066f90 msgr2=0x7f4c34069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.350+0000 7f4c4d8a5640 1 --2- 172.21.15.60:0/3328626749 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4c34066f90 0x7f4c34069450 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4c380015b0 tx=0x7f4c38009040 comp rx=0 tx=0).stop 2024-04-25T10:15:54.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.350+0000 7f4c4d8a5640 1 -- 172.21.15.60:0/3328626749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c4815a7a0 msgr2=0x7f4c480ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.350+0000 7f4c4d8a5640 1 --2- 172.21.15.60:0/3328626749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c4815a7a0 0x7f4c480ce780 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f4c400af040 tx=0x7f4c400197d0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.350+0000 7f4c4d8a5640 1 -- 172.21.15.60:0/3328626749 shutdown_connections 2024-04-25T10:15:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.350+0000 7f4c4d8a5640 1 --2- 172.21.15.60:0/3328626749 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4c34066f90 0x7f4c34069450 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.350+0000 7f4c4d8a5640 1 --2- 172.21.15.60:0/3328626749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c4815a7a0 0x7f4c480ce780 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.350+0000 7f4c4d8a5640 1 --2- 172.21.15.60:0/3328626749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c480b37e0 0x7f4c48151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.350+0000 7f4c4d8a5640 1 --2- 172.21.15.60:0/3328626749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c4806ec30 0x7f4c4815d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.350+0000 7f4c4d8a5640 1 -- 172.21.15.60:0/3328626749 >> 172.21.15.60:0/3328626749 conn(0x7f4c480b12d0 msgr2=0x7f4c481648a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.350+0000 7f4c4d8a5640 1 -- 172.21.15.60:0/3328626749 shutdown_connections 2024-04-25T10:15:54.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.350+0000 7f4c4d8a5640 1 -- 172.21.15.60:0/3328626749 wait complete. 2024-04-25T10:15:54.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:54.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.482+0000 7fe8e047e640 1 Processor -- start 2024-04-25T10:15:54.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.482+0000 7fe8e047e640 1 -- start start 2024-04-25T10:15:54.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.486+0000 7fe8e047e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe8d80d0a50 0x7fe8d80b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.486+0000 7fe8e047e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe8d80ce2b0 0x7fe8d80ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.486+0000 7fe8e047e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d806d200 con 0x7fe8d815deb0 2024-04-25T10:15:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.486+0000 7fe8e047e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d80cf2c0 con 0x7fe8d80d0a50 2024-04-25T10:15:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.486+0000 7fe8e047e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d80cefe0 con 0x7fe8d80ce2b0 2024-04-25T10:15:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.486+0000 7fe8dda19640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe8d80d0a50 0x7fe8d80b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.486+0000 7fe8de21a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe8d815deb0 0x7fe8d80d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49272/0 (socket says 172.21.15.60:49272) 2024-04-25T10:15:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.486+0000 7fe8de21a640 1 -- 172.21.15.60:0/2617117774 learned_addr learned my addr 172.21.15.60:0/2617117774 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:15:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.486+0000 7fe8dea1b640 1 --2- 172.21.15.60:0/2617117774 >> v2:172.21.15.100:3300/0 conn(0x7fe8d80ce2b0 0x7fe8d80ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.486+0000 7fe8dda19640 1 -- 172.21.15.60:0/2617117774 >> v2:172.21.15.100:3300/0 conn(0x7fe8d80ce2b0 msgr2=0x7fe8d80ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.486+0000 7fe8dda19640 1 --2- 172.21.15.60:0/2617117774 >> v2:172.21.15.100:3300/0 conn(0x7fe8d80ce2b0 0x7fe8d80ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.486+0000 7fe8dda19640 1 -- 172.21.15.60:0/2617117774 >> v1:172.21.15.60:6789/0 conn(0x7fe8d815deb0 legacy=0x7fe8d80d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.486+0000 7fe8dd218640 1 -- 172.21.15.60:0/2617117774 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268562136 0 0) 0x7fe8d806d200 con 0x7fe8d815deb0 2024-04-25T10:15:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.486+0000 7fe8dda19640 1 -- 172.21.15.60:0/2617117774 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d80d1d70 con 0x7fe8d80d0a50 2024-04-25T10:15:54.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.486+0000 7fe8dda19640 1 --2- 172.21.15.60:0/2617117774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d80d0a50 0x7fe8d80b6c10 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80d1590 tx=0x7fe8c801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f72b7db6b7d7c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.486+0000 7fe8dd218640 1 -- 172.21.15.60:0/2617117774 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80b0070 con 0x7fe8d80d0a50 2024-04-25T10:15:54.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8dd218640 1 -- 172.21.15.60:0/2617117774 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c80ab600 con 0x7fe8d80d0a50 2024-04-25T10:15:54.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8dd218640 1 -- 172.21.15.60:0/2617117774 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80ab9d0 con 0x7fe8d80d0a50 2024-04-25T10:15:54.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8e047e640 1 -- 172.21.15.60:0/2617117774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d80d0a50 msgr2=0x7fe8d80b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8e047e640 1 --2- 172.21.15.60:0/2617117774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d80d0a50 0x7fe8d80b6c10 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80d1590 tx=0x7fe8c801c480 comp rx=0 tx=0).stop 2024-04-25T10:15:54.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8e047e640 1 -- 172.21.15.60:0/2617117774 shutdown_connections 2024-04-25T10:15:54.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8e047e640 1 --2- 172.21.15.60:0/2617117774 >> v2:172.21.15.100:3300/0 conn(0x7fe8d80ce2b0 0x7fe8d80ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8e047e640 1 --2- 172.21.15.60:0/2617117774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d80d0a50 0x7fe8d80b6c10 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8e047e640 1 -- 172.21.15.60:0/2617117774 >> 172.21.15.60:0/2617117774 conn(0x7fe8d806c090 msgr2=0x7fe8d806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:54.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8e047e640 1 -- 172.21.15.60:0/2617117774 shutdown_connections 2024-04-25T10:15:54.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8e047e640 1 -- 172.21.15.60:0/2617117774 wait complete. 2024-04-25T10:15:54.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8e047e640 1 Processor -- start 2024-04-25T10:15:54.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8e047e640 1 -- start start 2024-04-25T10:15:54.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8e047e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d80ce2b0 0x7fe8d81556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8e047e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8d80d0a50 0x7fe8d8156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8e047e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d815deb0 0x7fe8d814b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8e047e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80d31e0 con 0x7fe8d815deb0 2024-04-25T10:15:54.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8e047e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80d3060 con 0x7fe8d80ce2b0 2024-04-25T10:15:54.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8e047e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80d3360 con 0x7fe8d80d0a50 2024-04-25T10:15:54.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8de21a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d80ce2b0 0x7fe8d81556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8de21a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d80ce2b0 0x7fe8d81556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:60632/0 (socket says 172.21.15.60:60632) 2024-04-25T10:15:54.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8de21a640 1 -- 172.21.15.60:0/2099217896 learned_addr learned my addr 172.21.15.60:0/2099217896 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:54.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8dea1b640 1 --2- 172.21.15.60:0/2099217896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d815deb0 0x7fe8d814b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8dda19640 1 --2- 172.21.15.60:0/2099217896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8d80d0a50 0x7fe8d8156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8dea1b640 1 -- 172.21.15.60:0/2099217896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8d80d0a50 msgr2=0x7fe8d8156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8dea1b640 1 --2- 172.21.15.60:0/2099217896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8d80d0a50 0x7fe8d8156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8dea1b640 1 -- 172.21.15.60:0/2099217896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d80ce2b0 msgr2=0x7fe8d81556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8dea1b640 1 --2- 172.21.15.60:0/2099217896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d80ce2b0 0x7fe8d81556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8dea1b640 1 -- 172.21.15.60:0/2099217896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d80ca150 con 0x7fe8d815deb0 2024-04-25T10:15:54.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8dea1b640 1 --2- 172.21.15.60:0/2099217896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d815deb0 0x7fe8d814b700 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0007dc0 tx=0x7fe8d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.490+0000 7fe8ceffd640 1 -- 172.21.15.60:0/2099217896 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d000edd0 con 0x7fe8d815deb0 2024-04-25T10:15:54.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.494+0000 7fe8ceffd640 1 -- 172.21.15.60:0/2099217896 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d00039a0 con 0x7fe8d815deb0 2024-04-25T10:15:54.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.494+0000 7fe8e047e640 1 -- 172.21.15.60:0/2099217896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d8154c00 con 0x7fe8d815deb0 2024-04-25T10:15:54.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.494+0000 7fe8ceffd640 1 -- 172.21.15.60:0/2099217896 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0003d90 con 0x7fe8d815deb0 2024-04-25T10:15:54.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.494+0000 7fe8e047e640 1 -- 172.21.15.60:0/2099217896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d81551c0 con 0x7fe8d815deb0 2024-04-25T10:15:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.494+0000 7fe8e047e640 1 -- 172.21.15.60:0/2099217896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d8074770 con 0x7fe8d815deb0 2024-04-25T10:15:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.494+0000 7fe8ceffd640 1 -- 172.21.15.60:0/2099217896 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8d001b020 con 0x7fe8d815deb0 2024-04-25T10:15:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.494+0000 7fe8ceffd640 1 --2- 172.21.15.60:0/2099217896 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8bc066f60 0x7fe8bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.494+0000 7fe8ceffd640 1 -- 172.21.15.60:0/2099217896 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fe8d0085140 con 0x7fe8d815deb0 2024-04-25T10:15:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.494+0000 7fe8de21a640 1 --2- 172.21.15.60:0/2099217896 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8bc066f60 0x7fe8bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.494+0000 7fe8de21a640 1 --2- 172.21.15.60:0/2099217896 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8bc066f60 0x7fe8bc069420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4000f70 tx=0x7fe8d4007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.498+0000 7fe8ceffd640 1 -- 172.21.15.60:0/2099217896 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8d0055a50 con 0x7fe8d815deb0 2024-04-25T10:15:54.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.658+0000 7fe8e047e640 1 -- 172.21.15.60:0/2099217896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8d80d17a0 con 0x7fe8d815deb0 2024-04-25T10:15:54.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.662+0000 7fe8ceffd640 1 -- 172.21.15.60:0/2099217896 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16879 (secure 0 0 0) 0x7fe8d0059af0 con 0x7fe8d815deb0 2024-04-25T10:15:54.666 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:15:54.666 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":99,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:15:50.915440+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:14.444411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:15.153386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/4149609651":"2024-04-26T10:15:50.915390+0000","172.21.15.60:6834/4149609651":"2024-04-26T10:15:50.915390+0000","172.21.15.60:6835/2002755047":"2024-04-26T10:15:14.408136+0000","172.21.15.60:6834/2002755047":"2024-04-26T10:15:14.408136+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:54.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.662+0000 7fe8e047e640 1 -- 172.21.15.60:0/2099217896 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8bc066f60 msgr2=0x7fe8bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.662+0000 7fe8e047e640 1 --2- 172.21.15.60:0/2099217896 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8bc066f60 0x7fe8bc069420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4000f70 tx=0x7fe8d4007eb0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.662+0000 7fe8e047e640 1 -- 172.21.15.60:0/2099217896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d815deb0 msgr2=0x7fe8d814b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.662+0000 7fe8e047e640 1 --2- 172.21.15.60:0/2099217896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d815deb0 0x7fe8d814b700 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0007dc0 tx=0x7fe8d000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.662+0000 7fe8e047e640 1 -- 172.21.15.60:0/2099217896 shutdown_connections 2024-04-25T10:15:54.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.662+0000 7fe8e047e640 1 --2- 172.21.15.60:0/2099217896 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe8bc066f60 0x7fe8bc069420 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.662+0000 7fe8e047e640 1 --2- 172.21.15.60:0/2099217896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d815deb0 0x7fe8d814b700 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.662+0000 7fe8e047e640 1 --2- 172.21.15.60:0/2099217896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8d80d0a50 0x7fe8d8156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.662+0000 7fe8e047e640 1 --2- 172.21.15.60:0/2099217896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d80ce2b0 0x7fe8d81556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.662+0000 7fe8e047e640 1 -- 172.21.15.60:0/2099217896 >> 172.21.15.60:0/2099217896 conn(0x7fe8d806c090 msgr2=0x7fe8d8158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:54.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.662+0000 7fe8e047e640 1 -- 172.21.15.60:0/2099217896 shutdown_connections 2024-04-25T10:15:54.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.662+0000 7fe8e047e640 1 -- 172.21.15.60:0/2099217896 wait complete. 2024-04-25T10:15:54.685 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:15:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7facf36a3640 1 Processor -- start 2024-04-25T10:15:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7facf36a3640 1 -- start start 2024-04-25T10:15:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7facf36a3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7facec06ac10 0x7facec06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7facf36a3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7facec0b54d0 0x7facec0af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7facf36a3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facec06b530 con 0x7facec0b3610 2024-04-25T10:15:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7facf36a3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facec0b3e60 con 0x7facec0b54d0 2024-04-25T10:15:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7facf36a3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facec0b3b80 con 0x7facec06ac10 2024-04-25T10:15:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7facf0c3e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7facec0b3610 0x7facec0aec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49288/0 (socket says 172.21.15.60:49288) 2024-04-25T10:15:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7facf0c3e640 1 -- 172.21.15.60:0/3614001472 learned_addr learned my addr 172.21.15.60:0/3614001472 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:15:54.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7facf1c40640 1 --2- 172.21.15.60:0/3614001472 >> v2:172.21.15.60:3300/0 conn(0x7facec0b54d0 0x7facec0af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7facf143f640 1 --2- 172.21.15.60:0/3614001472 >> v2:172.21.15.100:3300/0 conn(0x7facec06ac10 0x7facec06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7face3fff640 1 -- 172.21.15.60:0/3614001472 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091078451 0 0) 0x7facec06b530 con 0x7facec0b3610 2024-04-25T10:15:54.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7face3fff640 1 -- 172.21.15.60:0/3614001472 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facd8003530 con 0x7facec0b3610 2024-04-25T10:15:54.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7facf1c40640 1 -- 172.21.15.60:0/3614001472 >> v2:172.21.15.100:3300/0 conn(0x7facec06ac10 msgr2=0x7facec06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:54.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7facf1c40640 1 --2- 172.21.15.60:0/3614001472 >> v2:172.21.15.100:3300/0 conn(0x7facec06ac10 0x7facec06aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7facf1c40640 1 -- 172.21.15.60:0/3614001472 >> v1:172.21.15.60:6789/0 conn(0x7facec0b3610 legacy=0x7facec0aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.794+0000 7facf1c40640 1 -- 172.21.15.60:0/3614001472 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec0ae130 con 0x7facec0b54d0 2024-04-25T10:15:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf1c40640 1 --2- 172.21.15.60:0/3614001472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facec0b54d0 0x7facec0af5d0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7facec071c00 tx=0x7face401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc923de1f7f3eba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7face3fff640 1 -- 172.21.15.60:0/3614001472 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40ab650 con 0x7facec0b54d0 2024-04-25T10:15:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7face3fff640 1 -- 172.21.15.60:0/3614001472 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face40b0070 con 0x7facec0b54d0 2024-04-25T10:15:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7face3fff640 1 -- 172.21.15.60:0/3614001472 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40ab9f0 con 0x7facec0b54d0 2024-04-25T10:15:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf36a3640 1 -- 172.21.15.60:0/3614001472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facec0b54d0 msgr2=0x7facec0af5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf36a3640 1 --2- 172.21.15.60:0/3614001472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facec0b54d0 0x7facec0af5d0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7facec071c00 tx=0x7face401c450 comp rx=0 tx=0).stop 2024-04-25T10:15:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf36a3640 1 -- 172.21.15.60:0/3614001472 shutdown_connections 2024-04-25T10:15:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf36a3640 1 --2- 172.21.15.60:0/3614001472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facec0b54d0 0x7facec0af5d0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf36a3640 1 --2- 172.21.15.60:0/3614001472 >> v2:172.21.15.100:3300/0 conn(0x7facec06ac10 0x7facec06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf36a3640 1 -- 172.21.15.60:0/3614001472 >> 172.21.15.60:0/3614001472 conn(0x7facec072920 msgr2=0x7facec099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf36a3640 1 -- 172.21.15.60:0/3614001472 shutdown_connections 2024-04-25T10:15:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf36a3640 1 -- 172.21.15.60:0/3614001472 wait complete. 2024-04-25T10:15:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf36a3640 1 Processor -- start 2024-04-25T10:15:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf36a3640 1 -- start start 2024-04-25T10:15:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf36a3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facec06ac10 0x7facec145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf36a3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7facec0b3610 0x7facec14f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf36a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facec0b54d0 0x7facec157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf36a3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec0b5aa0 con 0x7facec0b54d0 2024-04-25T10:15:54.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf36a3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec0b5920 con 0x7facec06ac10 2024-04-25T10:15:54.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf36a3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facec0aff20 con 0x7facec0b3610 2024-04-25T10:15:54.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf1c40640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facec0b54d0 0x7facec157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf1c40640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facec0b54d0 0x7facec157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41832/0 (socket says 172.21.15.60:41832) 2024-04-25T10:15:54.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.798+0000 7facf1c40640 1 -- 172.21.15.60:0/1329382106 learned_addr learned my addr 172.21.15.60:0/1329382106 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:54.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7facf143f640 1 --2- 172.21.15.60:0/1329382106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facec06ac10 0x7facec145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7facf0c3e640 1 --2- 172.21.15.60:0/1329382106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7facec0b3610 0x7facec14f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7facf1c40640 1 -- 172.21.15.60:0/1329382106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7facec0b3610 msgr2=0x7facec14f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7facf1c40640 1 --2- 172.21.15.60:0/1329382106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7facec0b3610 0x7facec14f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7facf1c40640 1 -- 172.21.15.60:0/1329382106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facec06ac10 msgr2=0x7facec145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7facf1c40640 1 --2- 172.21.15.60:0/1329382106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facec06ac10 0x7facec145750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7facf1c40640 1 -- 172.21.15.60:0/1329382106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec0b2ed0 con 0x7facec0b54d0 2024-04-25T10:15:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7facf1c40640 1 --2- 172.21.15.60:0/1329382106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facec0b54d0 0x7facec157e00 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7face401ab00 tx=0x7face40191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7face1ffb640 1 -- 172.21.15.60:0/1329382106 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4003820 con 0x7facec0b54d0 2024-04-25T10:15:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7face1ffb640 1 -- 172.21.15.60:0/1329382106 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face40039c0 con 0x7facec0b54d0 2024-04-25T10:15:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7facf36a3640 1 -- 172.21.15.60:0/1329382106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facec147580 con 0x7facec0b54d0 2024-04-25T10:15:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7face1ffb640 1 -- 172.21.15.60:0/1329382106 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4003d70 con 0x7facec0b54d0 2024-04-25T10:15:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7facf36a3640 1 -- 172.21.15.60:0/1329382106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facec14e870 con 0x7facec0b54d0 2024-04-25T10:15:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7facf36a3640 1 -- 172.21.15.60:0/1329382106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facec074770 con 0x7facec0b54d0 2024-04-25T10:15:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7face1ffb640 1 -- 172.21.15.60:0/1329382106 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7face40c3060 con 0x7facec0b54d0 2024-04-25T10:15:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7face1ffb640 1 --2- 172.21.15.60:0/1329382106 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7facd0066fb0 0x7facd0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7face1ffb640 1 -- 172.21.15.60:0/1329382106 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5881+0+0 (secure 0 0 0) 0x7face411ee20 con 0x7facec0b54d0 2024-04-25T10:15:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.802+0000 7facf143f640 1 --2- 172.21.15.60:0/1329382106 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7facd0066fb0 0x7facd0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.806+0000 7facf143f640 1 --2- 172.21.15.60:0/1329382106 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7facd0066fb0 0x7facd0069470 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7face80073f0 tx=0x7face800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.806+0000 7face1ffb640 1 -- 172.21.15.60:0/1329382106 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7face4016070 con 0x7facec0b54d0 2024-04-25T10:15:55.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:54.998+0000 7facf36a3640 1 -- 172.21.15.60:0/1329382106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7facec099ad0 con 0x7facec0b54d0 2024-04-25T10:15:55.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:55.942+0000 7face1ffb640 1 -- 172.21.15.60:0/1329382106 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) v1 ==== 161+0+0 (secure 0 0 0) 0x7face40b2310 con 0x7facec0b54d0 2024-04-25T10:15:55.949 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:15:55.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:55.946+0000 7facf36a3640 1 -- 172.21.15.60:0/1329382106 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7facd0066fb0 msgr2=0x7facd0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:55.946+0000 7facf36a3640 1 --2- 172.21.15.60:0/1329382106 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7facd0066fb0 0x7facd0069470 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7face80073f0 tx=0x7face800b040 comp rx=0 tx=0).stop 2024-04-25T10:15:55.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:55.946+0000 7facf36a3640 1 -- 172.21.15.60:0/1329382106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facec0b54d0 msgr2=0x7facec157e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:55.946+0000 7facf36a3640 1 --2- 172.21.15.60:0/1329382106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facec0b54d0 0x7facec157e00 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7face401ab00 tx=0x7face40191f0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:55.946+0000 7facf36a3640 1 -- 172.21.15.60:0/1329382106 shutdown_connections 2024-04-25T10:15:55.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:55.946+0000 7facf36a3640 1 --2- 172.21.15.60:0/1329382106 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7facd0066fb0 0x7facd0069470 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:55.946+0000 7facf36a3640 1 --2- 172.21.15.60:0/1329382106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7facec0b54d0 0x7facec157e00 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:55.946+0000 7facf36a3640 1 --2- 172.21.15.60:0/1329382106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7facec0b3610 0x7facec14f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:55.946+0000 7facf36a3640 1 --2- 172.21.15.60:0/1329382106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facec06ac10 0x7facec145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:55.946+0000 7facf36a3640 1 -- 172.21.15.60:0/1329382106 >> 172.21.15.60:0/1329382106 conn(0x7facec072920 msgr2=0x7facec1575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:55.946+0000 7facf36a3640 1 -- 172.21.15.60:0/1329382106 shutdown_connections 2024-04-25T10:15:55.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:55.946+0000 7facf36a3640 1 -- 172.21.15.60:0/1329382106 wait complete. 2024-04-25T10:15:55.972 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:15:56.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.094+0000 7f61caab2640 1 Processor -- start 2024-04-25T10:15:56.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.094+0000 7f61caab2640 1 -- start start 2024-04-25T10:15:56.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61caab2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f61c4160080 0x7f61c415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61caab2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f61c415fa20 0x7f61c40ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61caab2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c4144a40 con 0x7f61c40cbcf0 2024-04-25T10:15:56.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61caab2640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c415fe90 con 0x7f61c415fa20 2024-04-25T10:15:56.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61caab2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c40b2dc0 con 0x7f61c4160080 2024-04-25T10:15:56.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61c904f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f61c415fa20 0x7f61c40ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61c904f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f61c415fa20 0x7f61c40ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41848/0 (socket says 172.21.15.60:41848) 2024-04-25T10:15:56.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61c904f640 1 -- 172.21.15.60:0/1238193780 learned_addr learned my addr 172.21.15.60:0/1238193780 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:56.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61c3fff640 1 --2- 172.21.15.60:0/1238193780 >> v2:172.21.15.100:3300/0 conn(0x7f61c4160080 0x7f61c415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61c904f640 1 -- 172.21.15.60:0/1238193780 >> v2:172.21.15.100:3300/0 conn(0x7f61c4160080 msgr2=0x7f61c415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61c904f640 1 --2- 172.21.15.60:0/1238193780 >> v2:172.21.15.100:3300/0 conn(0x7f61c4160080 0x7f61c415f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61c904f640 1 -- 172.21.15.60:0/1238193780 >> v1:172.21.15.100:6789/0 conn(0x7f61c40cbcf0 legacy=0x7f61c4071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61c904f640 1 -- 172.21.15.60:0/1238193780 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c40cc9f0 con 0x7f61c415fa20 2024-04-25T10:15:56.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61c3fff640 1 --2- 172.21.15.60:0/1238193780 >> v2:172.21.15.100:3300/0 conn(0x7f61c4160080 0x7f61c415f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:56.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61c904f640 1 --2- 172.21.15.60:0/1238193780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c415fa20 0x7f61c40ceec0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f61c40d0570 tx=0x7f61bc01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24125b6ad2169495 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61c37fe640 1 -- 172.21.15.60:0/1238193780 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc0a5760 con 0x7f61c415fa20 2024-04-25T10:15:56.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61c37fe640 1 -- 172.21.15.60:0/1238193780 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61bc0a5900 con 0x7f61c415fa20 2024-04-25T10:15:56.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.098+0000 7f61c37fe640 1 -- 172.21.15.60:0/1238193780 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc0a5cd0 con 0x7f61c415fa20 2024-04-25T10:15:56.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 -- 172.21.15.60:0/1238193780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c415fa20 msgr2=0x7f61c40ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 --2- 172.21.15.60:0/1238193780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c415fa20 0x7f61c40ceec0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f61c40d0570 tx=0x7f61bc01c9b0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 -- 172.21.15.60:0/1238193780 shutdown_connections 2024-04-25T10:15:56.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 --2- 172.21.15.60:0/1238193780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c415fa20 0x7f61c40ceec0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 --2- 172.21.15.60:0/1238193780 >> v2:172.21.15.100:3300/0 conn(0x7f61c4160080 0x7f61c415f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 -- 172.21.15.60:0/1238193780 >> 172.21.15.60:0/1238193780 conn(0x7f61c40670b0 msgr2=0x7f61c406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:56.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 -- 172.21.15.60:0/1238193780 shutdown_connections 2024-04-25T10:15:56.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 -- 172.21.15.60:0/1238193780 wait complete. 2024-04-25T10:15:56.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 Processor -- start 2024-04-25T10:15:56.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 -- start start 2024-04-25T10:15:56.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c40cbcf0 0x7f61c4142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61c415fa20 0x7f61c4139340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61c4160080 0x7f61c40ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40ce5c0 con 0x7f61c40cbcf0 2024-04-25T10:15:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40cc0d0 con 0x7f61c4160080 2024-04-25T10:15:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40ce740 con 0x7f61c415fa20 2024-04-25T10:15:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61c884e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c40cbcf0 0x7f61c4142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61c884e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c40cbcf0 0x7f61c4142fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41850/0 (socket says 172.21.15.60:41850) 2024-04-25T10:15:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61c884e640 1 -- 172.21.15.60:0/3742433197 learned_addr learned my addr 172.21.15.60:0/3742433197 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61c904f640 1 --2- 172.21.15.60:0/3742433197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61c4160080 0x7f61c40ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61c3fff640 1 --2- 172.21.15.60:0/3742433197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61c415fa20 0x7f61c4139340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61c884e640 1 -- 172.21.15.60:0/3742433197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61c415fa20 msgr2=0x7f61c4139340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61c884e640 1 --2- 172.21.15.60:0/3742433197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61c415fa20 0x7f61c4139340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61c884e640 1 -- 172.21.15.60:0/3742433197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61c4160080 msgr2=0x7f61c40ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61c884e640 1 --2- 172.21.15.60:0/3742433197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61c4160080 0x7f61c40ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61c884e640 1 -- 172.21.15.60:0/3742433197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c406e100 con 0x7f61c40cbcf0 2024-04-25T10:15:56.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61c884e640 1 --2- 172.21.15.60:0/3742433197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c40cbcf0 0x7f61c4142fc0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f61b400c480 tx=0x7f61b400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61c17fa640 1 -- 172.21.15.60:0/3742433197 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4012830 con 0x7f61c40cbcf0 2024-04-25T10:15:56.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61c17fa640 1 -- 172.21.15.60:0/3742433197 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b4017070 con 0x7f61c40cbcf0 2024-04-25T10:15:56.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 -- 172.21.15.60:0/3742433197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c406e3e0 con 0x7f61c40cbcf0 2024-04-25T10:15:56.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61c17fa640 1 -- 172.21.15.60:0/3742433197 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4012c30 con 0x7f61c40cbcf0 2024-04-25T10:15:56.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.102+0000 7f61caab2640 1 -- 172.21.15.60:0/3742433197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c406e9a0 con 0x7f61c40cbcf0 2024-04-25T10:15:56.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.106+0000 7f61caab2640 1 -- 172.21.15.60:0/3742433197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c406bba0 con 0x7f61c40cbcf0 2024-04-25T10:15:56.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.106+0000 7f61c17fa640 1 -- 172.21.15.60:0/3742433197 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61b401c460 con 0x7f61c40cbcf0 2024-04-25T10:15:56.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.106+0000 7f61c17fa640 1 --2- 172.21.15.60:0/3742433197 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f61b0066f90 0x7f61b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.106+0000 7f61c17fa640 1 -- 172.21.15.60:0/3742433197 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f61b4084f20 con 0x7f61c40cbcf0 2024-04-25T10:15:56.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.106+0000 7f61c3fff640 1 --2- 172.21.15.60:0/3742433197 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f61b0066f90 0x7f61b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.106+0000 7f61c3fff640 1 --2- 172.21.15.60:0/3742433197 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f61b0066f90 0x7f61b0069450 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f61b8006fd0 tx=0x7f61b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.110+0000 7f61c17fa640 1 -- 172.21.15.60:0/3742433197 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61b4055de0 con 0x7f61c40cbcf0 2024-04-25T10:15:56.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.306+0000 7f61caab2640 1 -- 172.21.15.60:0/3742433197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f61c4074770 con 0x7f61c40cbcf0 2024-04-25T10:15:56.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.954+0000 7f61c17fa640 1 -- 172.21.15.60:0/3742433197 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v87) v1 ==== 104+0+0 (secure 0 0 0) 0x7f61b40556e0 con 0x7f61c40cbcf0 2024-04-25T10:15:56.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.958+0000 7f61caab2640 1 -- 172.21.15.60:0/3742433197 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f61b0066f90 msgr2=0x7f61b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.958+0000 7f61caab2640 1 --2- 172.21.15.60:0/3742433197 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f61b0066f90 0x7f61b0069450 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f61b8006fd0 tx=0x7f61b8008040 comp rx=0 tx=0).stop 2024-04-25T10:15:56.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.958+0000 7f61caab2640 1 -- 172.21.15.60:0/3742433197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c40cbcf0 msgr2=0x7f61c4142fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.958+0000 7f61caab2640 1 --2- 172.21.15.60:0/3742433197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c40cbcf0 0x7f61c4142fc0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f61b400c480 tx=0x7f61b400c940 comp rx=0 tx=0).stop 2024-04-25T10:15:56.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.958+0000 7f61caab2640 1 -- 172.21.15.60:0/3742433197 shutdown_connections 2024-04-25T10:15:56.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.958+0000 7f61caab2640 1 --2- 172.21.15.60:0/3742433197 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f61b0066f90 0x7f61b0069450 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.958+0000 7f61caab2640 1 --2- 172.21.15.60:0/3742433197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61c4160080 0x7f61c40ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.958+0000 7f61caab2640 1 --2- 172.21.15.60:0/3742433197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61c415fa20 0x7f61c4139340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.958+0000 7f61caab2640 1 --2- 172.21.15.60:0/3742433197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61c40cbcf0 0x7f61c4142fc0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.958+0000 7f61caab2640 1 -- 172.21.15.60:0/3742433197 >> 172.21.15.60:0/3742433197 conn(0x7f61c40670b0 msgr2=0x7f61c4073ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:56.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.958+0000 7f61caab2640 1 -- 172.21.15.60:0/3742433197 shutdown_connections 2024-04-25T10:15:56.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:56.958+0000 7f61caab2640 1 -- 172.21.15.60:0/3742433197 wait complete. 2024-04-25T10:15:56.987 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:15:57.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.118+0000 7f6f9ff4e640 1 Processor -- start 2024-04-25T10:15:57.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.118+0000 7f6f9ff4e640 1 -- start start 2024-04-25T10:15:57.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.118+0000 7f6f9ff4e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6f98071980 0x7f6f980ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.118+0000 7f6f9ff4e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f980cf0a0 0x7f6f980b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.118+0000 7f6f9ff4e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f980cf810 con 0x7f6f98144e40 2024-04-25T10:15:57.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.118+0000 7f6f9ff4e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f980b3060 con 0x7f6f98071980 2024-04-25T10:15:57.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.118+0000 7f6f9ff4e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f980cfaf0 con 0x7f6f980cf0a0 2024-04-25T10:15:57.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.118+0000 7f6f9d4e9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6f98071980 0x7f6f980ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.118+0000 7f6f9d4e9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6f98071980 0x7f6f980ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41858/0 (socket says 172.21.15.60:41858) 2024-04-25T10:15:57.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.118+0000 7f6f9d4e9640 1 -- 172.21.15.60:0/3435505512 learned_addr learned my addr 172.21.15.60:0/3435505512 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:57.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.118+0000 7f6f9dcea640 1 --1- 172.21.15.60:0/3435505512 >> v1:172.21.15.60:6789/0 conn(0x7f6f98144e40 0x7f6f98071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49294/0 (socket says 172.21.15.60:49294) 2024-04-25T10:15:57.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9e4eb640 1 --2- 172.21.15.60:0/3435505512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f980cf0a0 0x7f6f980b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9d4e9640 1 -- 172.21.15.60:0/3435505512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f980cf0a0 msgr2=0x7f6f980b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9d4e9640 1 --2- 172.21.15.60:0/3435505512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f980cf0a0 0x7f6f980b34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9cce8640 1 -- 172.21.15.60:0/3435505512 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3781027316 0 0) 0x7f6f980cf810 con 0x7f6f98144e40 2024-04-25T10:15:57.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9d4e9640 1 -- 172.21.15.60:0/3435505512 >> v1:172.21.15.60:6789/0 conn(0x7f6f98144e40 legacy=0x7f6f98071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9d4e9640 1 -- 172.21.15.60:0/3435505512 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f980cfdd0 con 0x7f6f98071980 2024-04-25T10:15:57.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9d4e9640 1 --2- 172.21.15.60:0/3435505512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f98071980 0x7f6f980ccc60 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009870 tx=0x7f6f9401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788a5a2fa42665c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9cce8640 1 -- 172.21.15.60:0/3435505512 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f940a9610 con 0x7f6f98071980 2024-04-25T10:15:57.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9cce8640 1 -- 172.21.15.60:0/3435505512 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f98145430 con 0x7f6f98071980 2024-04-25T10:15:57.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9cce8640 1 -- 172.21.15.60:0/3435505512 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f940a99d0 con 0x7f6f98071980 2024-04-25T10:15:57.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9ff4e640 1 -- 172.21.15.60:0/3435505512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f98071980 msgr2=0x7f6f980ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9ff4e640 1 --2- 172.21.15.60:0/3435505512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f98071980 0x7f6f980ccc60 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009870 tx=0x7f6f9401c530 comp rx=0 tx=0).stop 2024-04-25T10:15:57.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9ff4e640 1 -- 172.21.15.60:0/3435505512 shutdown_connections 2024-04-25T10:15:57.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9ff4e640 1 --2- 172.21.15.60:0/3435505512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f980cf0a0 0x7f6f980b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9ff4e640 1 --2- 172.21.15.60:0/3435505512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f98071980 0x7f6f980ccc60 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9ff4e640 1 -- 172.21.15.60:0/3435505512 >> 172.21.15.60:0/3435505512 conn(0x7f6f98066fa0 msgr2=0x7f6f9806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9ff4e640 1 -- 172.21.15.60:0/3435505512 shutdown_connections 2024-04-25T10:15:57.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.122+0000 7f6f9ff4e640 1 -- 172.21.15.60:0/3435505512 wait complete. 2024-04-25T10:15:57.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9ff4e640 1 Processor -- start 2024-04-25T10:15:57.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9ff4e640 1 -- start start 2024-04-25T10:15:57.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9ff4e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f98071980 0x7f6f98165c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9ff4e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980cf0a0 0x7f6f9806ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9ff4e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f98144e40 0x7f6f98149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9ff4e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f980b4650 con 0x7f6f980cf0a0 2024-04-25T10:15:57.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9ff4e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f980b44d0 con 0x7f6f98144e40 2024-04-25T10:15:57.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9ff4e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f980b47d0 con 0x7f6f98071980 2024-04-25T10:15:57.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9d4e9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980cf0a0 0x7f6f9806ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9d4e9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980cf0a0 0x7f6f9806ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41874/0 (socket says 172.21.15.60:41874) 2024-04-25T10:15:57.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9d4e9640 1 -- 172.21.15.60:0/738785941 learned_addr learned my addr 172.21.15.60:0/738785941 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:57.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9d4e9640 1 -- 172.21.15.60:0/738785941 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f98071980 msgr2=0x7f6f98165c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9dcea640 1 --2- 172.21.15.60:0/738785941 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f98071980 0x7f6f98165c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9e4eb640 1 --2- 172.21.15.60:0/738785941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f98144e40 0x7f6f98149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9d4e9640 1 --2- 172.21.15.60:0/738785941 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f98071980 0x7f6f98165c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9d4e9640 1 -- 172.21.15.60:0/738785941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f98144e40 msgr2=0x7f6f98149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9d4e9640 1 --2- 172.21.15.60:0/738785941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f98144e40 0x7f6f98149a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9d4e9640 1 -- 172.21.15.60:0/738785941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9806e4f0 con 0x7f6f980cf0a0 2024-04-25T10:15:57.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9e4eb640 1 --2- 172.21.15.60:0/738785941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f98144e40 0x7f6f98149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:57.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9d4e9640 1 --2- 172.21.15.60:0/738785941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980cf0a0 0x7f6f9806ecd0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f6f9401a160 tx=0x7f6f9401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f8e7fc640 1 -- 172.21.15.60:0/738785941 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f940bc450 con 0x7f6f980cf0a0 2024-04-25T10:15:57.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f8e7fc640 1 -- 172.21.15.60:0/738785941 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f940bc610 con 0x7f6f980cf0a0 2024-04-25T10:15:57.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9ff4e640 1 -- 172.21.15.60:0/738785941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9814a4d0 con 0x7f6f980cf0a0 2024-04-25T10:15:57.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f8e7fc640 1 -- 172.21.15.60:0/738785941 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f940bca00 con 0x7f6f980cf0a0 2024-04-25T10:15:57.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9ff4e640 1 -- 172.21.15.60:0/738785941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9806e7b0 con 0x7f6f980cf0a0 2024-04-25T10:15:57.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.126+0000 7f6f9ff4e640 1 -- 172.21.15.60:0/738785941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f98074770 con 0x7f6f980cf0a0 2024-04-25T10:15:57.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.130+0000 7f6f8e7fc640 1 -- 172.21.15.60:0/738785941 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6f940c53f0 con 0x7f6f980cf0a0 2024-04-25T10:15:57.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.130+0000 7f6f8e7fc640 1 --2- 172.21.15.60:0/738785941 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f7c066f60 0x7f6f7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.130+0000 7f6f8e7fc640 1 -- 172.21.15.60:0/738785941 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f6f9411fd50 con 0x7f6f980cf0a0 2024-04-25T10:15:57.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.130+0000 7f6f9dcea640 1 --2- 172.21.15.60:0/738785941 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f7c066f60 0x7f6f7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.130+0000 7f6f9dcea640 1 --2- 172.21.15.60:0/738785941 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f7c066f60 0x7f6f7c069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6f88000f70 tx=0x7f6f88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.134+0000 7f6f8e7fc640 1 -- 172.21.15.60:0/738785941 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6f940ad020 con 0x7f6f980cf0a0 2024-04-25T10:15:57.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.294+0000 7f6f9ff4e640 1 -- 172.21.15.60:0/738785941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6f980ada80 con 0x7f6f980cf0a0 2024-04-25T10:15:57.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.962+0000 7f6f8e7fc640 1 -- 172.21.15.60:0/738785941 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6f94016150 con 0x7f6f980cf0a0 2024-04-25T10:15:57.968 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:15:57.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.970+0000 7f6f9ff4e640 1 -- 172.21.15.60:0/738785941 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f7c066f60 msgr2=0x7f6f7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.970+0000 7f6f9ff4e640 1 --2- 172.21.15.60:0/738785941 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f7c066f60 0x7f6f7c069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6f88000f70 tx=0x7f6f88009040 comp rx=0 tx=0).stop 2024-04-25T10:15:57.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.970+0000 7f6f9ff4e640 1 -- 172.21.15.60:0/738785941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980cf0a0 msgr2=0x7f6f9806ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.970+0000 7f6f9ff4e640 1 --2- 172.21.15.60:0/738785941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980cf0a0 0x7f6f9806ecd0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f6f9401a160 tx=0x7f6f9401ad70 comp rx=0 tx=0).stop 2024-04-25T10:15:57.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.970+0000 7f6f9ff4e640 1 -- 172.21.15.60:0/738785941 shutdown_connections 2024-04-25T10:15:57.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.970+0000 7f6f9ff4e640 1 --2- 172.21.15.60:0/738785941 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6f7c066f60 0x7f6f7c069420 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.970+0000 7f6f9ff4e640 1 --2- 172.21.15.60:0/738785941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f98144e40 0x7f6f98149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.970+0000 7f6f9ff4e640 1 --2- 172.21.15.60:0/738785941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980cf0a0 0x7f6f9806ecd0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.970+0000 7f6f9ff4e640 1 --2- 172.21.15.60:0/738785941 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f98071980 0x7f6f98165c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.970+0000 7f6f9ff4e640 1 -- 172.21.15.60:0/738785941 >> 172.21.15.60:0/738785941 conn(0x7f6f98066fa0 msgr2=0x7f6f981413a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.970+0000 7f6f9ff4e640 1 -- 172.21.15.60:0/738785941 shutdown_connections 2024-04-25T10:15:57.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:57.970+0000 7f6f9ff4e640 1 -- 172.21.15.60:0/738785941 wait complete. 2024-04-25T10:15:57.995 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:15:58.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.122+0000 7fc41cd9f640 1 Processor -- start 2024-04-25T10:15:58.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.122+0000 7fc41cd9f640 1 -- start start 2024-04-25T10:15:58.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.122+0000 7fc41cd9f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc41806f140 0x7fc418071270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.122+0000 7fc41cd9f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc41806d2d0 0x7fc4180717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.122+0000 7fc41cd9f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4180cd2f0 con 0x7fc418159f90 2024-04-25T10:15:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.122+0000 7fc41cd9f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41815a370 con 0x7fc41806d2d0 2024-04-25T10:15:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.122+0000 7fc41cd9f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41806d7d0 con 0x7fc41806f140 2024-04-25T10:15:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.122+0000 7fc415d9b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc41806f140 0x7fc418071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.122+0000 7fc415d9b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc41806f140 0x7fc418071270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:59776/0 (socket says 172.21.15.60:59776) 2024-04-25T10:15:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.122+0000 7fc415d9b640 1 -- 172.21.15.60:0/3118093114 learned_addr learned my addr 172.21.15.60:0/3118093114 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.122+0000 7fc416d9d640 1 --2- 172.21.15.60:0/3118093114 >> v2:172.21.15.100:3300/0 conn(0x7fc41806d2d0 0x7fc4180717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.122+0000 7fc415d9b640 1 -- 172.21.15.60:0/3118093114 >> v2:172.21.15.100:3300/0 conn(0x7fc41806d2d0 msgr2=0x7fc4180717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.122+0000 7fc415d9b640 1 --2- 172.21.15.60:0/3118093114 >> v2:172.21.15.100:3300/0 conn(0x7fc41806d2d0 0x7fc4180717b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.122+0000 7fc415d9b640 1 -- 172.21.15.60:0/3118093114 >> v1:172.21.15.100:6789/0 conn(0x7fc418159f90 legacy=0x7fc4180ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.122+0000 7fc415d9b640 1 -- 172.21.15.60:0/3118093114 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41806dab0 con 0x7fc41806f140 2024-04-25T10:15:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.122+0000 7fc416d9d640 1 --2- 172.21.15.60:0/3118093114 >> v2:172.21.15.100:3300/0 conn(0x7fc41806d2d0 0x7fc4180717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:58.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.126+0000 7fc415d9b640 1 --2- 172.21.15.60:0/3118093114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc41806f140 0x7fc418071270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc40c006fd0 tx=0x7fc40c01e9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd4ce50ad17249ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.126+0000 7fc41559a640 1 -- 172.21.15.60:0/3118093114 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c0ad070 con 0x7fc41806f140 2024-04-25T10:15:58.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.126+0000 7fc41559a640 1 -- 172.21.15.60:0/3118093114 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c004230 con 0x7fc41806f140 2024-04-25T10:15:58.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.126+0000 7fc41559a640 1 -- 172.21.15.60:0/3118093114 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c0a7730 con 0x7fc41806f140 2024-04-25T10:15:58.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.126+0000 7fc41cd9f640 1 -- 172.21.15.60:0/3118093114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc41806f140 msgr2=0x7fc418071270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.126+0000 7fc41cd9f640 1 --2- 172.21.15.60:0/3118093114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc41806f140 0x7fc418071270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc40c006fd0 tx=0x7fc40c01e9f0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.126+0000 7fc41cd9f640 1 -- 172.21.15.60:0/3118093114 shutdown_connections 2024-04-25T10:15:58.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.126+0000 7fc41cd9f640 1 --2- 172.21.15.60:0/3118093114 >> v2:172.21.15.100:3300/0 conn(0x7fc41806d2d0 0x7fc4180717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.126+0000 7fc41cd9f640 1 --2- 172.21.15.60:0/3118093114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc41806f140 0x7fc418071270 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.126+0000 7fc41cd9f640 1 -- 172.21.15.60:0/3118093114 >> 172.21.15.60:0/3118093114 conn(0x7fc418066fa0 msgr2=0x7fc41806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.126+0000 7fc41cd9f640 1 -- 172.21.15.60:0/3118093114 shutdown_connections 2024-04-25T10:15:58.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.126+0000 7fc41cd9f640 1 -- 172.21.15.60:0/3118093114 wait complete. 2024-04-25T10:15:58.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41cd9f640 1 Processor -- start 2024-04-25T10:15:58.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41cd9f640 1 -- start start 2024-04-25T10:15:58.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41cd9f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc41806d2d0 0x7fc41814ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41cd9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc41806f140 0x7fc418151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41cd9f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc418159f90 0x7fc418151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41cd9f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4180b2db0 con 0x7fc41806d2d0 2024-04-25T10:15:58.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41cd9f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4180b2c30 con 0x7fc41806f140 2024-04-25T10:15:58.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41cd9f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4180b2f30 con 0x7fc418159f90 2024-04-25T10:15:58.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc41806d2d0 0x7fc41814ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc41806d2d0 0x7fc41814ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41882/0 (socket says 172.21.15.60:41882) 2024-04-25T10:15:58.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41659c640 1 -- 172.21.15.60:0/3707471448 learned_addr learned my addr 172.21.15.60:0/3707471448 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:58.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc415d9b640 1 --2- 172.21.15.60:0/3707471448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc41806f140 0x7fc418151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc416d9d640 1 --2- 172.21.15.60:0/3707471448 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc418159f90 0x7fc418151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41659c640 1 -- 172.21.15.60:0/3707471448 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc418159f90 msgr2=0x7fc418151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41659c640 1 --2- 172.21.15.60:0/3707471448 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc418159f90 0x7fc418151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41659c640 1 -- 172.21.15.60:0/3707471448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc41806f140 msgr2=0x7fc418151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41659c640 1 --2- 172.21.15.60:0/3707471448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc41806f140 0x7fc418151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41659c640 1 -- 172.21.15.60:0/3707471448 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4181567c0 con 0x7fc41806d2d0 2024-04-25T10:15:58.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41659c640 1 --2- 172.21.15.60:0/3707471448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc41806d2d0 0x7fc41814ca40 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fc40800c3c0 tx=0x7fc40800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc406ffd640 1 -- 172.21.15.60:0/3707471448 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408017070 con 0x7fc41806d2d0 2024-04-25T10:15:58.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc406ffd640 1 -- 172.21.15.60:0/3707471448 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4080052d0 con 0x7fc41806d2d0 2024-04-25T10:15:58.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc406ffd640 1 -- 172.21.15.60:0/3707471448 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408003840 con 0x7fc41806d2d0 2024-04-25T10:15:58.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41cd9f640 1 -- 172.21.15.60:0/3707471448 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41814bc80 con 0x7fc41806d2d0 2024-04-25T10:15:58.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41cd9f640 1 -- 172.21.15.60:0/3707471448 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41814c240 con 0x7fc41806d2d0 2024-04-25T10:15:58.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.130+0000 7fc41cd9f640 1 -- 172.21.15.60:0/3707471448 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc418074770 con 0x7fc41806d2d0 2024-04-25T10:15:58.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.134+0000 7fc406ffd640 1 -- 172.21.15.60:0/3707471448 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc40801a070 con 0x7fc41806d2d0 2024-04-25T10:15:58.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.134+0000 7fc406ffd640 1 --2- 172.21.15.60:0/3707471448 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc3f8066f60 0x7fc3f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.134+0000 7fc406ffd640 1 -- 172.21.15.60:0/3707471448 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5443+0+0 (secure 0 0 0) 0x7fc4080841b0 con 0x7fc41806d2d0 2024-04-25T10:15:58.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.134+0000 7fc415d9b640 1 --2- 172.21.15.60:0/3707471448 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc3f8066f60 0x7fc3f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.134+0000 7fc415d9b640 1 --2- 172.21.15.60:0/3707471448 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc3f8066f60 0x7fc3f8069420 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0b1000 tx=0x7fc40c0a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.138+0000 7fc406ffd640 1 -- 172.21.15.60:0/3707471448 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4180cd2f0 con 0x7fc41806d2d0 2024-04-25T10:15:58.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.298+0000 7fc41cd9f640 1 -- 172.21.15.60:0/3707471448 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc4180aefc0 con 0x7fc41806d2d0 2024-04-25T10:15:58.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.966+0000 7fc406ffd640 1 -- 172.21.15.60:0/3707471448 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc408005480 con 0x7fc41806d2d0 2024-04-25T10:15:58.971 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T10:15:58.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.970+0000 7fc41cd9f640 1 -- 172.21.15.60:0/3707471448 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc3f8066f60 msgr2=0x7fc3f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.970+0000 7fc41cd9f640 1 --2- 172.21.15.60:0/3707471448 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc3f8066f60 0x7fc3f8069420 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0b1000 tx=0x7fc40c0a9070 comp rx=0 tx=0).stop 2024-04-25T10:15:58.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.970+0000 7fc41cd9f640 1 -- 172.21.15.60:0/3707471448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc41806d2d0 msgr2=0x7fc41814ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.970+0000 7fc41cd9f640 1 --2- 172.21.15.60:0/3707471448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc41806d2d0 0x7fc41814ca40 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fc40800c3c0 tx=0x7fc40800c880 comp rx=0 tx=0).stop 2024-04-25T10:15:58.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.970+0000 7fc41cd9f640 1 -- 172.21.15.60:0/3707471448 shutdown_connections 2024-04-25T10:15:58.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.970+0000 7fc41cd9f640 1 --2- 172.21.15.60:0/3707471448 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc3f8066f60 0x7fc3f8069420 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.970+0000 7fc41cd9f640 1 --2- 172.21.15.60:0/3707471448 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc418159f90 0x7fc418151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.970+0000 7fc41cd9f640 1 --2- 172.21.15.60:0/3707471448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc41806f140 0x7fc418151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.970+0000 7fc41cd9f640 1 --2- 172.21.15.60:0/3707471448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc41806d2d0 0x7fc41814ca40 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.970+0000 7fc41cd9f640 1 -- 172.21.15.60:0/3707471448 >> 172.21.15.60:0/3707471448 conn(0x7fc418066fa0 msgr2=0x7fc4180ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.974+0000 7fc41cd9f640 1 -- 172.21.15.60:0/3707471448 shutdown_connections 2024-04-25T10:15:58.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:58.974+0000 7fc41cd9f640 1 -- 172.21.15.60:0/3707471448 wait complete. 2024-04-25T10:15:59.006 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:15:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0cb05df640 1 Processor -- start 2024-04-25T10:15:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0cb05df640 1 -- start start 2024-04-25T10:15:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0cb05df640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ca81620d0 0x7f0ca80ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0cb05df640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0ca80ced20 0x7f0ca80d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0cb05df640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca815bad0 con 0x7f0ca815b6f0 2024-04-25T10:15:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0cb05df640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca8161c40 con 0x7f0ca80ced20 2024-04-25T10:15:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0cb05df640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca815ebe0 con 0x7f0ca81620d0 2024-04-25T10:15:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0cae37b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0ca815b6f0 0x7f0ca815e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46800/0 (socket says 172.21.15.60:46800) 2024-04-25T10:15:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0caeb7c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0ca80ced20 0x7f0ca80d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0cae37b640 1 -- 172.21.15.60:0/16993148 learned_addr learned my addr 172.21.15.60:0/16993148 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:15:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0cad379640 1 -- 172.21.15.60:0/16993148 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659660456 0 0) 0x7f0ca815bad0 con 0x7f0ca815b6f0 2024-04-25T10:15:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0cad379640 1 -- 172.21.15.60:0/16993148 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c94003530 con 0x7f0ca815b6f0 2024-04-25T10:15:59.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0cadb7a640 1 --2- 172.21.15.60:0/16993148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ca81620d0 0x7f0ca80ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0caeb7c640 1 -- 172.21.15.60:0/16993148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ca81620d0 msgr2=0x7f0ca80ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:59.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0caeb7c640 1 --2- 172.21.15.60:0/16993148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ca81620d0 0x7f0ca80ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0caeb7c640 1 -- 172.21.15.60:0/16993148 >> v1:172.21.15.60:6789/0 conn(0x7f0ca815b6f0 legacy=0x7f0ca815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.134+0000 7f0caeb7c640 1 -- 172.21.15.60:0/16993148 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca815eec0 con 0x7f0ca80ced20 2024-04-25T10:15:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.138+0000 7f0caeb7c640 1 --2- 172.21.15.60:0/16993148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca80ced20 0x7f0ca80d1b30 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009d90 tx=0x7f0ca001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86b541cfb1495dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.138+0000 7f0cad379640 1 -- 172.21.15.60:0/16993148 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00b2070 con 0x7f0ca80ced20 2024-04-25T10:15:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.138+0000 7f0cad379640 1 -- 172.21.15.60:0/16993148 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca00a57f0 con 0x7f0ca80ced20 2024-04-25T10:15:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.138+0000 7f0cad379640 1 -- 172.21.15.60:0/16993148 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00a5bc0 con 0x7f0ca80ced20 2024-04-25T10:15:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.138+0000 7f0cb05df640 1 -- 172.21.15.60:0/16993148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca80ced20 msgr2=0x7f0ca80d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.138+0000 7f0cb05df640 1 --2- 172.21.15.60:0/16993148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca80ced20 0x7f0ca80d1b30 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009d90 tx=0x7f0ca001c970 comp rx=0 tx=0).stop 2024-04-25T10:15:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.138+0000 7f0cb05df640 1 -- 172.21.15.60:0/16993148 shutdown_connections 2024-04-25T10:15:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.138+0000 7f0cb05df640 1 --2- 172.21.15.60:0/16993148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca80ced20 0x7f0ca80d1b30 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.138+0000 7f0cb05df640 1 --2- 172.21.15.60:0/16993148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ca81620d0 0x7f0ca80ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.138+0000 7f0cb05df640 1 -- 172.21.15.60:0/16993148 >> 172.21.15.60:0/16993148 conn(0x7f0ca8072340 msgr2=0x7f0ca8072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.138+0000 7f0cb05df640 1 -- 172.21.15.60:0/16993148 shutdown_connections 2024-04-25T10:15:59.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.138+0000 7f0cb05df640 1 -- 172.21.15.60:0/16993148 wait complete. 2024-04-25T10:15:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cb05df640 1 Processor -- start 2024-04-25T10:15:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cb05df640 1 -- start start 2024-04-25T10:15:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cb05df640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ca80ced20 0x7f0ca80c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cb05df640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca815b6f0 0x7f0ca80c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cb05df640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca81620d0 0x7f0ca814f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cb05df640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca815e960 con 0x7f0ca815b6f0 2024-04-25T10:15:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cb05df640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca815e7e0 con 0x7f0ca81620d0 2024-04-25T10:15:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cb05df640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca816aa60 con 0x7f0ca80ced20 2024-04-25T10:15:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cadb7a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca815b6f0 0x7f0ca80c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cadb7a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca815b6f0 0x7f0ca80c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39382/0 (socket says 172.21.15.60:39382) 2024-04-25T10:15:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cadb7a640 1 -- 172.21.15.60:0/1827637317 learned_addr learned my addr 172.21.15.60:0/1827637317 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cae37b640 1 --2- 172.21.15.60:0/1827637317 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ca80ced20 0x7f0ca80c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0caeb7c640 1 --2- 172.21.15.60:0/1827637317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca81620d0 0x7f0ca814f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cadb7a640 1 -- 172.21.15.60:0/1827637317 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ca80ced20 msgr2=0x7f0ca80c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cadb7a640 1 --2- 172.21.15.60:0/1827637317 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ca80ced20 0x7f0ca80c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cadb7a640 1 -- 172.21.15.60:0/1827637317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca81620d0 msgr2=0x7f0ca814f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cadb7a640 1 --2- 172.21.15.60:0/1827637317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca81620d0 0x7f0ca814f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cadb7a640 1 -- 172.21.15.60:0/1827637317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca80b36c0 con 0x7f0ca815b6f0 2024-04-25T10:15:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cadb7a640 1 --2- 172.21.15.60:0/1827637317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca815b6f0 0x7f0ca80c9c50 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800d620 tx=0x7f0c9800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0c9effd640 1 -- 172.21.15.60:0/1827637317 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800b620 con 0x7f0ca815b6f0 2024-04-25T10:15:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0c9effd640 1 -- 172.21.15.60:0/1827637317 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c98019070 con 0x7f0ca815b6f0 2024-04-25T10:15:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0c9effd640 1 -- 172.21.15.60:0/1827637317 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800b9e0 con 0x7f0ca815b6f0 2024-04-25T10:15:59.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cb05df640 1 -- 172.21.15.60:0/1827637317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca80b39a0 con 0x7f0ca815b6f0 2024-04-25T10:15:59.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cb05df640 1 -- 172.21.15.60:0/1827637317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca80b3f60 con 0x7f0ca815b6f0 2024-04-25T10:15:59.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.142+0000 7f0cb05df640 1 -- 172.21.15.60:0/1827637317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca8074770 con 0x7f0ca815b6f0 2024-04-25T10:15:59.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.146+0000 7f0c9effd640 1 -- 172.21.15.60:0/1827637317 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c9800bb80 con 0x7f0ca815b6f0 2024-04-25T10:15:59.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.146+0000 7f0c9effd640 1 --2- 172.21.15.60:0/1827637317 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0c8c066f60 0x7f0c8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.146+0000 7f0c9effd640 1 -- 172.21.15.60:0/1827637317 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f0c98085050 con 0x7f0ca815b6f0 2024-04-25T10:15:59.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.146+0000 7f0cae37b640 1 --2- 172.21.15.60:0/1827637317 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0c8c066f60 0x7f0c8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.146+0000 7f0cae37b640 1 --2- 172.21.15.60:0/1827637317 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0c8c066f60 0x7f0c8c069420 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4007fd0 tx=0x7f0ca4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.150+0000 7f0c9effd640 1 -- 172.21.15.60:0/1827637317 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0c98055cb0 con 0x7f0ca815b6f0 2024-04-25T10:15:59.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.318+0000 7f0cb05df640 1 -- 172.21.15.60:0/1827637317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0ca815db10 con 0x7f0ca815b6f0 2024-04-25T10:15:59.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.326+0000 7f0c9effd640 1 -- 172.21.15.60:0/1827637317 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0c98059d50 con 0x7f0ca815b6f0 2024-04-25T10:15:59.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.326+0000 7f0cb05df640 1 -- 172.21.15.60:0/1827637317 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0c8c066f60 msgr2=0x7f0c8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.326+0000 7f0cb05df640 1 --2- 172.21.15.60:0/1827637317 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0c8c066f60 0x7f0c8c069420 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4007fd0 tx=0x7f0ca4009040 comp rx=0 tx=0).stop 2024-04-25T10:15:59.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.326+0000 7f0cb05df640 1 -- 172.21.15.60:0/1827637317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca815b6f0 msgr2=0x7f0ca80c9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.326+0000 7f0cb05df640 1 --2- 172.21.15.60:0/1827637317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca815b6f0 0x7f0ca80c9c50 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800d620 tx=0x7f0c9800dae0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.326+0000 7f0cb05df640 1 -- 172.21.15.60:0/1827637317 shutdown_connections 2024-04-25T10:15:59.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.326+0000 7f0cb05df640 1 --2- 172.21.15.60:0/1827637317 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0c8c066f60 0x7f0c8c069420 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.326+0000 7f0cb05df640 1 --2- 172.21.15.60:0/1827637317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca81620d0 0x7f0ca814f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.326+0000 7f0cb05df640 1 --2- 172.21.15.60:0/1827637317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca815b6f0 0x7f0ca80c9c50 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.326+0000 7f0cb05df640 1 --2- 172.21.15.60:0/1827637317 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ca80ced20 0x7f0ca80c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.330+0000 7f0cb05df640 1 -- 172.21.15.60:0/1827637317 >> 172.21.15.60:0/1827637317 conn(0x7f0ca8072340 msgr2=0x7f0ca80d3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:59.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.330+0000 7f0cb05df640 1 -- 172.21.15.60:0/1827637317 shutdown_connections 2024-04-25T10:15:59.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.330+0000 7f0cb05df640 1 -- 172.21.15.60:0/1827637317 wait complete. 2024-04-25T10:15:59.351 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-25T10:15:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fd00a5c6640 1 Processor -- start 2024-04-25T10:15:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fd00a5c6640 1 -- start start 2024-04-25T10:15:59.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fd00a5c6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd00415bc90 0x7fd00415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fd00a5c6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0040b4660 0x7fd00415c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fd00a5c6640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd004071ca0 con 0x7fd0040ce7a0 2024-04-25T10:15:59.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fd00a5c6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0040cc370 con 0x7fd0040b4660 2024-04-25T10:15:59.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fd00a5c6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040cca40 con 0x7fd00415bc90 2024-04-25T10:15:59.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fd008b63640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0040b4660 0x7fd00415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fd003fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd0040ce7a0 0x7fd0040ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46814/0 (socket says 172.21.15.60:46814) 2024-04-25T10:15:59.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fd003fff640 1 -- 172.21.15.60:0/1719733302 learned_addr learned my addr 172.21.15.60:0/1719733302 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:15:59.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fcffbfff640 1 --2- 172.21.15.60:0/1719733302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd00415bc90 0x7fd00415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fd008b63640 1 -- 172.21.15.60:0/1719733302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd00415bc90 msgr2=0x7fd00415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:59.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fd008b63640 1 --2- 172.21.15.60:0/1719733302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd00415bc90 0x7fd00415c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fd008b63640 1 -- 172.21.15.60:0/1719733302 >> v1:172.21.15.60:6789/0 conn(0x7fd0040ce7a0 legacy=0x7fd0040ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fd008b63640 1 -- 172.21.15.60:0/1719733302 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0040ccd20 con 0x7fd0040b4660 2024-04-25T10:15:59.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fd0037fe640 1 -- 172.21.15.60:0/1719733302 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788094222 0 0) 0x7fd004071ca0 con 0x7fd0040ce7a0 2024-04-25T10:15:59.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.474+0000 7fcffbfff640 1 --2- 172.21.15.60:0/1719733302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd00415bc90 0x7fd00415c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:59.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd008b63640 1 --2- 172.21.15.60:0/1719733302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0040b4660 0x7fd00415c760 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009d90 tx=0x7fcffc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b090ae2b92f55d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:59.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd0037fe640 1 -- 172.21.15.60:0/1719733302 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc0b2070 con 0x7fd0040b4660 2024-04-25T10:15:59.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd0037fe640 1 -- 172.21.15.60:0/1719733302 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcffc0a5830 con 0x7fd0040b4660 2024-04-25T10:15:59.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd0037fe640 1 -- 172.21.15.60:0/1719733302 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc0a5c00 con 0x7fd0040b4660 2024-04-25T10:15:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd00a5c6640 1 -- 172.21.15.60:0/1719733302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0040b4660 msgr2=0x7fd00415c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd00a5c6640 1 --2- 172.21.15.60:0/1719733302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0040b4660 0x7fd00415c760 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009d90 tx=0x7fcffc01c970 comp rx=0 tx=0).stop 2024-04-25T10:15:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd00a5c6640 1 -- 172.21.15.60:0/1719733302 shutdown_connections 2024-04-25T10:15:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd00a5c6640 1 --2- 172.21.15.60:0/1719733302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0040b4660 0x7fd00415c760 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd00a5c6640 1 --2- 172.21.15.60:0/1719733302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd00415bc90 0x7fd00415c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd00a5c6640 1 -- 172.21.15.60:0/1719733302 >> 172.21.15.60:0/1719733302 conn(0x7fd0040b12d0 msgr2=0x7fd0040b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd00a5c6640 1 -- 172.21.15.60:0/1719733302 shutdown_connections 2024-04-25T10:15:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd00a5c6640 1 -- 172.21.15.60:0/1719733302 wait complete. 2024-04-25T10:15:59.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd00a5c6640 1 Processor -- start 2024-04-25T10:15:59.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd00a5c6640 1 -- start start 2024-04-25T10:15:59.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd00a5c6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0040ce7a0 0x7fd0040adef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd00a5c6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00415bc90 0x7fd00406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd00a5c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00415d950 0x7fd004159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd00a5c6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040b3db0 con 0x7fd00415d950 2024-04-25T10:15:59.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd00a5c6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040b4490 con 0x7fd00415bc90 2024-04-25T10:15:59.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.478+0000 7fd00a5c6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0041685c0 con 0x7fd0040ce7a0 2024-04-25T10:15:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd008b63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00415d950 0x7fd004159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd008b63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00415d950 0x7fd004159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39406/0 (socket says 172.21.15.60:39406) 2024-04-25T10:15:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fcffbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00415bc90 0x7fd00406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd008b63640 1 -- 172.21.15.60:0/935400576 learned_addr learned my addr 172.21.15.60:0/935400576 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:15:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd003fff640 1 --2- 172.21.15.60:0/935400576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0040ce7a0 0x7fd0040adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd008b63640 1 -- 172.21.15.60:0/935400576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0040ce7a0 msgr2=0x7fd0040adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd008b63640 1 --2- 172.21.15.60:0/935400576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0040ce7a0 0x7fd0040adef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd008b63640 1 -- 172.21.15.60:0/935400576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00415bc90 msgr2=0x7fd00406e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd008b63640 1 --2- 172.21.15.60:0/935400576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00415bc90 0x7fd00406e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd008b63640 1 -- 172.21.15.60:0/935400576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00406d2f0 con 0x7fd00415d950 2024-04-25T10:15:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd008b63640 1 --2- 172.21.15.60:0/935400576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00415d950 0x7fd004159020 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fcffc0b0040 tx=0x7fcffc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd0017fa640 1 -- 172.21.15.60:0/935400576 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc0b2050 con 0x7fd00415d950 2024-04-25T10:15:59.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd0017fa640 1 -- 172.21.15.60:0/935400576 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcffc0b3040 con 0x7fd00415d950 2024-04-25T10:15:59.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd00a5c6640 1 -- 172.21.15.60:0/935400576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00406d5b0 con 0x7fd00415d950 2024-04-25T10:15:59.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd0017fa640 1 -- 172.21.15.60:0/935400576 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc0ad850 con 0x7fd00415d950 2024-04-25T10:15:59.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd00a5c6640 1 -- 172.21.15.60:0/935400576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00406db70 con 0x7fd00415d950 2024-04-25T10:15:59.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd00a5c6640 1 -- 172.21.15.60:0/935400576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd004074770 con 0x7fd00415d950 2024-04-25T10:15:59.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd0017fa640 1 -- 172.21.15.60:0/935400576 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcffc0c0020 con 0x7fd00415d950 2024-04-25T10:15:59.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd0017fa640 1 --2- 172.21.15.60:0/935400576 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcfe4066f90 0x7fcfe4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd0017fa640 1 -- 172.21.15.60:0/935400576 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fcffc11f530 con 0x7fd00415d950 2024-04-25T10:15:59.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.482+0000 7fd003fff640 1 --2- 172.21.15.60:0/935400576 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcfe4066f90 0x7fcfe4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.486+0000 7fd003fff640 1 --2- 172.21.15.60:0/935400576 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcfe4066f90 0x7fcfe4069450 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcff4000f70 tx=0x7fcff4007df0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:59.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.486+0000 7fd0017fa640 1 -- 172.21.15.60:0/935400576 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcffc016560 con 0x7fd00415d950 2024-04-25T10:15:59.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.722+0000 7fd00a5c6640 1 -- 172.21.15.60:0/935400576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7fd004154cb0 con 0x7fd00415d950 2024-04-25T10:15:59.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.986+0000 7fd0017fa640 1 -- 172.21.15.60:0/935400576 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v481) v1 ==== 146+0+0 (secure 0 0 0) 0x7fcffc0f02b0 con 0x7fd00415d950 2024-04-25T10:15:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.990+0000 7fd00a5c6640 1 -- 172.21.15.60:0/935400576 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcfe4066f90 msgr2=0x7fcfe4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.990+0000 7fd00a5c6640 1 --2- 172.21.15.60:0/935400576 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcfe4066f90 0x7fcfe4069450 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcff4000f70 tx=0x7fcff4007df0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.990+0000 7fd00a5c6640 1 -- 172.21.15.60:0/935400576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00415d950 msgr2=0x7fd004159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.990+0000 7fd00a5c6640 1 --2- 172.21.15.60:0/935400576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00415d950 0x7fd004159020 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fcffc0b0040 tx=0x7fcffc019710 comp rx=0 tx=0).stop 2024-04-25T10:15:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.990+0000 7fd00a5c6640 1 -- 172.21.15.60:0/935400576 shutdown_connections 2024-04-25T10:15:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.990+0000 7fd00a5c6640 1 --2- 172.21.15.60:0/935400576 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fcfe4066f90 0x7fcfe4069450 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.990+0000 7fd00a5c6640 1 --2- 172.21.15.60:0/935400576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00415d950 0x7fd004159020 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.990+0000 7fd00a5c6640 1 --2- 172.21.15.60:0/935400576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00415bc90 0x7fd00406e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.990+0000 7fd00a5c6640 1 --2- 172.21.15.60:0/935400576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0040ce7a0 0x7fd0040adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.990+0000 7fd00a5c6640 1 -- 172.21.15.60:0/935400576 >> 172.21.15.60:0/935400576 conn(0x7fd0040b12d0 msgr2=0x7fd0040ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.990+0000 7fd00a5c6640 1 -- 172.21.15.60:0/935400576 shutdown_connections 2024-04-25T10:15:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:15:59.990+0000 7fd00a5c6640 1 -- 172.21.15.60:0/935400576 wait complete. 2024-04-25T10:16:00.017 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:16:00.018 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:16:00.018 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:00.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.150+0000 7f6f15a19640 1 Processor -- start 2024-04-25T10:16:00.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.150+0000 7f6f15a19640 1 -- start start 2024-04-25T10:16:00.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.150+0000 7f6f15a19640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6f1006ec30 0x7f6f100cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.150+0000 7f6f15a19640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1006f130 con 0x7f6f1015a7a0 2024-04-25T10:16:00.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.150+0000 7f6f15a19640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1006f410 con 0x7f6f100b37e0 2024-04-25T10:16:00.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.150+0000 7f6f15a19640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f100cdaa0 con 0x7f6f1006ec30 2024-04-25T10:16:00.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.150+0000 7f6f0effd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6f1015a7a0 0x7f6f10071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46828/0 (socket says 172.21.15.60:46828) 2024-04-25T10:16:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.150+0000 7f6f0effd640 1 -- 172.21.15.60:0/3971390136 learned_addr learned my addr 172.21.15.60:0/3971390136 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.150+0000 7f6f07fff640 1 --2- 172.21.15.60:0/3971390136 >> v2:172.21.15.60:3300/0 conn(0x7f6f1006ec30 0x7f6f100cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.150+0000 7f6f0e7fc640 1 -- 172.21.15.60:0/3971390136 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794661668 0 0) 0x7f6f1006f130 con 0x7f6f1015a7a0 2024-04-25T10:16:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.150+0000 7f6f0e7fc640 1 -- 172.21.15.60:0/3971390136 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6efc003530 con 0x7f6f1015a7a0 2024-04-25T10:16:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.150+0000 7f6f07fff640 1 -- 172.21.15.60:0/3971390136 >> v1:172.21.15.100:6789/0 conn(0x7f6f100b37e0 legacy=0x7f6f1006dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.150+0000 7f6f07fff640 1 -- 172.21.15.60:0/3971390136 >> v1:172.21.15.60:6789/0 conn(0x7f6f1015a7a0 legacy=0x7f6f10071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.150+0000 7f6f0e7fc640 1 -- 172.21.15.60:0/3971390136 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884279690 0 0) 0x7f6f1006f410 con 0x7f6f100b37e0 2024-04-25T10:16:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.150+0000 7f6f07fff640 1 -- 172.21.15.60:0/3971390136 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1006d540 con 0x7f6f1006ec30 2024-04-25T10:16:00.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f07fff640 1 --2- 172.21.15.60:0/3971390136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f1006ec30 0x7f6f100cd560 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80097f0 tx=0x7f6ef801c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e5b082b1bbdf49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f0e7fc640 1 -- 172.21.15.60:0/3971390136 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef80b0070 con 0x7f6f1006ec30 2024-04-25T10:16:00.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f0e7fc640 1 -- 172.21.15.60:0/3971390136 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef80ab400 con 0x7f6f1006ec30 2024-04-25T10:16:00.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f0e7fc640 1 -- 172.21.15.60:0/3971390136 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f00001680 con 0x7f6f1006ec30 2024-04-25T10:16:00.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f15a19640 1 -- 172.21.15.60:0/3971390136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f1006ec30 msgr2=0x7f6f100cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f15a19640 1 --2- 172.21.15.60:0/3971390136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f1006ec30 0x7f6f100cd560 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80097f0 tx=0x7f6ef801c320 comp rx=0 tx=0).stop 2024-04-25T10:16:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f15a19640 1 -- 172.21.15.60:0/3971390136 shutdown_connections 2024-04-25T10:16:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f15a19640 1 --2- 172.21.15.60:0/3971390136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f1006ec30 0x7f6f100cd560 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f15a19640 1 -- 172.21.15.60:0/3971390136 >> 172.21.15.60:0/3971390136 conn(0x7f6f100b12d0 msgr2=0x7f6f100b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f15a19640 1 -- 172.21.15.60:0/3971390136 shutdown_connections 2024-04-25T10:16:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f15a19640 1 -- 172.21.15.60:0/3971390136 wait complete. 2024-04-25T10:16:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f15a19640 1 Processor -- start 2024-04-25T10:16:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f15a19640 1 -- start start 2024-04-25T10:16:00.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f15a19640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1006ec30 0x7f6f10150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f15a19640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f100b37e0 0x7f6f10151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f15a19640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f1015a7a0 0x7f6f101f9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f15a19640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1006d8b0 con 0x7f6f100b37e0 2024-04-25T10:16:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f15a19640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1006d2f0 con 0x7f6f1006ec30 2024-04-25T10:16:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f15a19640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f10168d10 con 0x7f6f1015a7a0 2024-04-25T10:16:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f07fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f100b37e0 0x7f6f10151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f07fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f100b37e0 0x7f6f10151180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39428/0 (socket says 172.21.15.60:39428) 2024-04-25T10:16:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f07fff640 1 -- 172.21.15.60:0/2592427152 learned_addr learned my addr 172.21.15.60:0/2592427152 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f0effd640 1 --2- 172.21.15.60:0/2592427152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1006ec30 0x7f6f10150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f0f7fe640 1 --2- 172.21.15.60:0/2592427152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f1015a7a0 0x7f6f101f9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f07fff640 1 -- 172.21.15.60:0/2592427152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f1015a7a0 msgr2=0x7f6f101f9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f07fff640 1 --2- 172.21.15.60:0/2592427152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f1015a7a0 0x7f6f101f9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f07fff640 1 -- 172.21.15.60:0/2592427152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1006ec30 msgr2=0x7f6f10150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f07fff640 1 --2- 172.21.15.60:0/2592427152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1006ec30 0x7f6f10150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f07fff640 1 -- 172.21.15.60:0/2592427152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f101fa970 con 0x7f6f100b37e0 2024-04-25T10:16:00.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.154+0000 7f6f0f7fe640 1 --2- 172.21.15.60:0/2592427152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f1015a7a0 0x7f6f101f9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:00.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.158+0000 7f6f07fff640 1 --2- 172.21.15.60:0/2592427152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f100b37e0 0x7f6f10151180 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80097f0 tx=0x7f6ef801ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.158+0000 7f6f0cff9640 1 -- 172.21.15.60:0/2592427152 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef80bb020 con 0x7f6f100b37e0 2024-04-25T10:16:00.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.158+0000 7f6f0cff9640 1 -- 172.21.15.60:0/2592427152 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef80b1040 con 0x7f6f100b37e0 2024-04-25T10:16:00.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.158+0000 7f6f15a19640 1 -- 172.21.15.60:0/2592427152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f101fac50 con 0x7f6f100b37e0 2024-04-25T10:16:00.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.158+0000 7f6f0cff9640 1 -- 172.21.15.60:0/2592427152 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef80b0050 con 0x7f6f100b37e0 2024-04-25T10:16:00.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.158+0000 7f6f15a19640 1 -- 172.21.15.60:0/2592427152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1014fc30 con 0x7f6f100b37e0 2024-04-25T10:16:00.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.158+0000 7f6f15a19640 1 -- 172.21.15.60:0/2592427152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f10074770 con 0x7f6f100b37e0 2024-04-25T10:16:00.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.158+0000 7f6f0cff9640 1 -- 172.21.15.60:0/2592427152 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ef80be070 con 0x7f6f100b37e0 2024-04-25T10:16:00.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.158+0000 7f6f0cff9640 1 --2- 172.21.15.60:0/2592427152 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6ef0066f60 0x7f6ef0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.158+0000 7f6f0cff9640 1 -- 172.21.15.60:0/2592427152 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f6ef811e300 con 0x7f6f100b37e0 2024-04-25T10:16:00.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.158+0000 7f6f0effd640 1 --2- 172.21.15.60:0/2592427152 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6ef0066f60 0x7f6ef0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.158+0000 7f6f0effd640 1 --2- 172.21.15.60:0/2592427152 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6ef0066f60 0x7f6ef0069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6f00001c50 tx=0x7f6f00001be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.162+0000 7f6f0cff9640 1 -- 172.21.15.60:0/2592427152 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ef8015fd0 con 0x7f6f100b37e0 2024-04-25T10:16:00.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.326+0000 7f6f15a19640 1 -- 172.21.15.60:0/2592427152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f101fae20 con 0x7f6f100b37e0 2024-04-25T10:16:00.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.326+0000 7f6f0cff9640 1 -- 172.21.15.60:0/2592427152 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+13164 (secure 0 0 0) 0x7f6ef80ef5b0 con 0x7f6f100b37e0 2024-04-25T10:16:00.332 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:00.332 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":101,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:15:58.966156+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/4149609651":"2024-04-26T10:15:50.915390+0000","172.21.15.60:6834/4149609651":"2024-04-26T10:15:50.915390+0000","172.21.15.60:6835/2002755047":"2024-04-26T10:15:14.408136+0000","172.21.15.60:6834/2002755047":"2024-04-26T10:15:14.408136+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:00.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.330+0000 7f6f15a19640 1 -- 172.21.15.60:0/2592427152 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6ef0066f60 msgr2=0x7f6ef0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.330+0000 7f6f15a19640 1 --2- 172.21.15.60:0/2592427152 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6ef0066f60 0x7f6ef0069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6f00001c50 tx=0x7f6f00001be0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.330+0000 7f6f15a19640 1 -- 172.21.15.60:0/2592427152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f100b37e0 msgr2=0x7f6f10151180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.330+0000 7f6f15a19640 1 --2- 172.21.15.60:0/2592427152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f100b37e0 0x7f6f10151180 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80097f0 tx=0x7f6ef801ab60 comp rx=0 tx=0).stop 2024-04-25T10:16:00.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.330+0000 7f6f15a19640 1 -- 172.21.15.60:0/2592427152 shutdown_connections 2024-04-25T10:16:00.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.330+0000 7f6f15a19640 1 --2- 172.21.15.60:0/2592427152 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6ef0066f60 0x7f6ef0069420 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.330+0000 7f6f15a19640 1 --2- 172.21.15.60:0/2592427152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f1015a7a0 0x7f6f101f9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.330+0000 7f6f15a19640 1 --2- 172.21.15.60:0/2592427152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f100b37e0 0x7f6f10151180 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.330+0000 7f6f15a19640 1 --2- 172.21.15.60:0/2592427152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1006ec30 0x7f6f10150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.330+0000 7f6f15a19640 1 -- 172.21.15.60:0/2592427152 >> 172.21.15.60:0/2592427152 conn(0x7f6f100b12d0 msgr2=0x7f6f1006dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:00.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.330+0000 7f6f15a19640 1 -- 172.21.15.60:0/2592427152 shutdown_connections 2024-04-25T10:16:00.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.330+0000 7f6f15a19640 1 -- 172.21.15.60:0/2592427152 wait complete. 2024-04-25T10:16:00.353 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:00.353 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:00.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.482+0000 7f09162bb640 1 Processor -- start 2024-04-25T10:16:00.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.482+0000 7f09162bb640 1 -- start start 2024-04-25T10:16:00.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.482+0000 7f09162bb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f091015dd30 0x7f091015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.482+0000 7f09162bb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f09100cf8a0 0x7f091015be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.482+0000 7f09162bb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09100cd890 con 0x7f09100cd3d0 2024-04-25T10:16:00.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.482+0000 7f09162bb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09100ce4a0 con 0x7f091015dd30 2024-04-25T10:16:00.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.482+0000 7f09162bb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09100cdb70 con 0x7f09100cf8a0 2024-04-25T10:16:00.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.482+0000 7f090f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f09100cd3d0 0x7f09100cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46830/0 (socket says 172.21.15.60:46830) 2024-04-25T10:16:00.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.482+0000 7f090ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f091015dd30 0x7f091015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.482+0000 7f090f7fe640 1 -- 172.21.15.60:0/3299792845 learned_addr learned my addr 172.21.15.60:0/3299792845 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:00.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.482+0000 7f0914858640 1 --2- 172.21.15.60:0/3299792845 >> v2:172.21.15.100:3300/0 conn(0x7f09100cf8a0 0x7f091015be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.482+0000 7f090ffff640 1 -- 172.21.15.60:0/3299792845 >> v2:172.21.15.100:3300/0 conn(0x7f09100cf8a0 msgr2=0x7f091015be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:00.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.482+0000 7f090ffff640 1 --2- 172.21.15.60:0/3299792845 >> v2:172.21.15.100:3300/0 conn(0x7f09100cf8a0 0x7f091015be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.482+0000 7f090ffff640 1 -- 172.21.15.60:0/3299792845 >> v1:172.21.15.60:6789/0 conn(0x7f09100cd3d0 legacy=0x7f09100cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.482+0000 7f090effd640 1 -- 172.21.15.60:0/3299792845 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472541930 0 0) 0x7f09100cd890 con 0x7f09100cd3d0 2024-04-25T10:16:00.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.482+0000 7f090ffff640 1 -- 172.21.15.60:0/3299792845 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09100cde50 con 0x7f091015dd30 2024-04-25T10:16:00.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.486+0000 7f090ffff640 1 --2- 172.21.15.60:0/3299792845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f091015dd30 0x7f091015ac30 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f091006eab0 tx=0x7f090001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbcc3be30bd7de9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.486+0000 7f090effd640 1 -- 172.21.15.60:0/3299792845 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09000b2070 con 0x7f091015dd30 2024-04-25T10:16:00.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.486+0000 7f090effd640 1 -- 172.21.15.60:0/3299792845 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09000a5950 con 0x7f091015dd30 2024-04-25T10:16:00.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.486+0000 7f090effd640 1 -- 172.21.15.60:0/3299792845 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09000a5d20 con 0x7f091015dd30 2024-04-25T10:16:00.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.486+0000 7f09162bb640 1 -- 172.21.15.60:0/3299792845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f091015dd30 msgr2=0x7f091015ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.486+0000 7f09162bb640 1 --2- 172.21.15.60:0/3299792845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f091015dd30 0x7f091015ac30 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f091006eab0 tx=0x7f090001c870 comp rx=0 tx=0).stop 2024-04-25T10:16:00.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.486+0000 7f09162bb640 1 -- 172.21.15.60:0/3299792845 shutdown_connections 2024-04-25T10:16:00.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.486+0000 7f09162bb640 1 --2- 172.21.15.60:0/3299792845 >> v2:172.21.15.100:3300/0 conn(0x7f09100cf8a0 0x7f091015be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.486+0000 7f09162bb640 1 --2- 172.21.15.60:0/3299792845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f091015dd30 0x7f091015ac30 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.486+0000 7f09162bb640 1 -- 172.21.15.60:0/3299792845 >> 172.21.15.60:0/3299792845 conn(0x7f09100670b0 msgr2=0x7f091006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.486+0000 7f09162bb640 1 -- 172.21.15.60:0/3299792845 shutdown_connections 2024-04-25T10:16:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.486+0000 7f09162bb640 1 -- 172.21.15.60:0/3299792845 wait complete. 2024-04-25T10:16:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.486+0000 7f09162bb640 1 Processor -- start 2024-04-25T10:16:00.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.486+0000 7f09162bb640 1 -- start start 2024-04-25T10:16:00.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f09162bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09100cd3d0 0x7f09101566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f09162bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09100cf8a0 0x7f091014a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f09162bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f091015dd30 0x7f091014bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f09162bb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091015cf00 con 0x7f09100cf8a0 2024-04-25T10:16:00.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f09162bb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091015cd80 con 0x7f09100cd3d0 2024-04-25T10:16:00.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f09162bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091015d080 con 0x7f091015dd30 2024-04-25T10:16:00.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09100cf8a0 0x7f091014a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09100cf8a0 0x7f091014a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39448/0 (socket says 172.21.15.60:39448) 2024-04-25T10:16:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090f7fe640 1 -- 172.21.15.60:0/3245775064 learned_addr learned my addr 172.21.15.60:0/3245775064 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090ffff640 1 --2- 172.21.15.60:0/3245775064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09100cd3d0 0x7f09101566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090f7fe640 1 -- 172.21.15.60:0/3245775064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f091015dd30 msgr2=0x7f091014bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090f7fe640 1 --2- 172.21.15.60:0/3245775064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f091015dd30 0x7f091014bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090f7fe640 1 -- 172.21.15.60:0/3245775064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09100cd3d0 msgr2=0x7f09101566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090f7fe640 1 --2- 172.21.15.60:0/3245775064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09100cd3d0 0x7f09101566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090f7fe640 1 -- 172.21.15.60:0/3245775064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091014c770 con 0x7f09100cf8a0 2024-04-25T10:16:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090ffff640 1 --2- 172.21.15.60:0/3245775064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09100cd3d0 0x7f09101566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090f7fe640 1 --2- 172.21.15.60:0/3245775064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09100cf8a0 0x7f091014a940 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f090400c480 tx=0x7f090400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090cff9640 1 -- 172.21.15.60:0/3245775064 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904018070 con 0x7f09100cf8a0 2024-04-25T10:16:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090cff9640 1 -- 172.21.15.60:0/3245775064 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0904007500 con 0x7f09100cf8a0 2024-04-25T10:16:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090cff9640 1 -- 172.21.15.60:0/3245775064 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904004a30 con 0x7f09100cf8a0 2024-04-25T10:16:00.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f09162bb640 1 -- 172.21.15.60:0/3245775064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091014ca50 con 0x7f09100cf8a0 2024-04-25T10:16:00.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f09162bb640 1 -- 172.21.15.60:0/3245775064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091014d010 con 0x7f09100cf8a0 2024-04-25T10:16:00.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f09162bb640 1 -- 172.21.15.60:0/3245775064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0910074770 con 0x7f09100cf8a0 2024-04-25T10:16:00.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090cff9640 1 -- 172.21.15.60:0/3245775064 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0904004bd0 con 0x7f09100cf8a0 2024-04-25T10:16:00.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090cff9640 1 --2- 172.21.15.60:0/3245775064 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f08fc066f60 0x7f08fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090cff9640 1 -- 172.21.15.60:0/3245775064 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f0904083f90 con 0x7f09100cf8a0 2024-04-25T10:16:00.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.490+0000 7f090ffff640 1 --2- 172.21.15.60:0/3245775064 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f08fc066f60 0x7f08fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.494+0000 7f090ffff640 1 --2- 172.21.15.60:0/3245775064 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f08fc066f60 0x7f08fc069420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f09000af000 tx=0x7f09000a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.494+0000 7f090cff9640 1 -- 172.21.15.60:0/3245775064 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f09040552d0 con 0x7f09100cf8a0 2024-04-25T10:16:00.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.654+0000 7f09162bb640 1 -- 172.21.15.60:0/3245775064 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09100ceb40 con 0x7f09100cf8a0 2024-04-25T10:16:00.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.654+0000 7f090cff9640 1 -- 172.21.15.60:0/3245775064 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0904054bd0 con 0x7f09100cf8a0 2024-04-25T10:16:00.661 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:16:00.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.658+0000 7f09162bb640 1 -- 172.21.15.60:0/3245775064 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f08fc066f60 msgr2=0x7f08fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.658+0000 7f09162bb640 1 --2- 172.21.15.60:0/3245775064 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f08fc066f60 0x7f08fc069420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f09000af000 tx=0x7f09000a7070 comp rx=0 tx=0).stop 2024-04-25T10:16:00.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.658+0000 7f09162bb640 1 -- 172.21.15.60:0/3245775064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09100cf8a0 msgr2=0x7f091014a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.658+0000 7f09162bb640 1 --2- 172.21.15.60:0/3245775064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09100cf8a0 0x7f091014a940 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f090400c480 tx=0x7f090400c940 comp rx=0 tx=0).stop 2024-04-25T10:16:00.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.658+0000 7f09162bb640 1 -- 172.21.15.60:0/3245775064 shutdown_connections 2024-04-25T10:16:00.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.658+0000 7f09162bb640 1 --2- 172.21.15.60:0/3245775064 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f08fc066f60 0x7f08fc069420 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.658+0000 7f09162bb640 1 --2- 172.21.15.60:0/3245775064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f091015dd30 0x7f091014bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.658+0000 7f09162bb640 1 --2- 172.21.15.60:0/3245775064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09100cf8a0 0x7f091014a940 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.658+0000 7f09162bb640 1 --2- 172.21.15.60:0/3245775064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09100cd3d0 0x7f09101566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.658+0000 7f09162bb640 1 -- 172.21.15.60:0/3245775064 >> 172.21.15.60:0/3245775064 conn(0x7f09100670b0 msgr2=0x7f0910151580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:00.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.658+0000 7f09162bb640 1 -- 172.21.15.60:0/3245775064 shutdown_connections 2024-04-25T10:16:00.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.658+0000 7f09162bb640 1 -- 172.21.15.60:0/3245775064 wait complete. 2024-04-25T10:16:00.682 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-25T10:16:00.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.798+0000 7f33fcdc6640 1 Processor -- start 2024-04-25T10:16:00.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.798+0000 7f33fcdc6640 1 -- start start 2024-04-25T10:16:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.798+0000 7f33fcdc6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33f80ce7a0 0x7f33f80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.798+0000 7f33fcdc6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f33f815bc90 0x7f33f80b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.798+0000 7f33fcdc6640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f80cf0c0 con 0x7f33f80b4660 2024-04-25T10:16:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.798+0000 7f33fcdc6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f80cc370 con 0x7f33f815bc90 2024-04-25T10:16:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.798+0000 7f33fcdc6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f8071ca0 con 0x7f33f80ce7a0 2024-04-25T10:16:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.798+0000 7f33edd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f33f815bc90 0x7f33f80b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.798+0000 7f33f6d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f33f80b4660 0x7f33f80b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46844/0 (socket says 172.21.15.60:46844) 2024-04-25T10:16:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.798+0000 7f33f6d9d640 1 -- 172.21.15.60:0/3661874570 learned_addr learned my addr 172.21.15.60:0/3661874570 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.798+0000 7f33f659c640 1 --2- 172.21.15.60:0/3661874570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33f80ce7a0 0x7f33f80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.798+0000 7f33edd9b640 1 -- 172.21.15.60:0/3661874570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33f80ce7a0 msgr2=0x7f33f80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.798+0000 7f33edd9b640 1 --2- 172.21.15.60:0/3661874570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33f80ce7a0 0x7f33f80ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.798+0000 7f33edd9b640 1 -- 172.21.15.60:0/3661874570 >> v1:172.21.15.60:6789/0 conn(0x7f33f80b4660 legacy=0x7f33f80b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.798+0000 7f33edd9b640 1 -- 172.21.15.60:0/3661874570 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f80cca40 con 0x7f33f815bc90 2024-04-25T10:16:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.798+0000 7f33f659c640 1 --2- 172.21.15.60:0/3661874570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33f80ce7a0 0x7f33f80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33edd9b640 1 --2- 172.21.15.60:0/3661874570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33f815bc90 0x7f33f80b25d0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f33e00097f0 tx=0x7f33e001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52071391d510b8f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33f5d9b640 1 -- 172.21.15.60:0/3661874570 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e00b0070 con 0x7f33f815bc90 2024-04-25T10:16:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33f5d9b640 1 -- 172.21.15.60:0/3661874570 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e00ab600 con 0x7f33f815bc90 2024-04-25T10:16:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33f5d9b640 1 -- 172.21.15.60:0/3661874570 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e00ab9d0 con 0x7f33f815bc90 2024-04-25T10:16:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33fcdc6640 1 -- 172.21.15.60:0/3661874570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33f815bc90 msgr2=0x7f33f80b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33fcdc6640 1 --2- 172.21.15.60:0/3661874570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33f815bc90 0x7f33f80b25d0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f33e00097f0 tx=0x7f33e001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33fcdc6640 1 -- 172.21.15.60:0/3661874570 shutdown_connections 2024-04-25T10:16:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33fcdc6640 1 --2- 172.21.15.60:0/3661874570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33f815bc90 0x7f33f80b25d0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33fcdc6640 1 --2- 172.21.15.60:0/3661874570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33f80ce7a0 0x7f33f80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33fcdc6640 1 -- 172.21.15.60:0/3661874570 >> 172.21.15.60:0/3661874570 conn(0x7f33f80b12d0 msgr2=0x7f33f80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33fcdc6640 1 -- 172.21.15.60:0/3661874570 shutdown_connections 2024-04-25T10:16:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33fcdc6640 1 -- 172.21.15.60:0/3661874570 wait complete. 2024-04-25T10:16:00.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33fcdc6640 1 Processor -- start 2024-04-25T10:16:00.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33fcdc6640 1 -- start start 2024-04-25T10:16:00.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33fcdc6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33f80ce7a0 0x7f33f813b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33fcdc6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33f815bc90 0x7f33f80a8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33fcdc6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33f8150e30 0x7f33f8152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33fcdc6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80b4280 con 0x7f33f8150e30 2024-04-25T10:16:00.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33fcdc6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80b3db0 con 0x7f33f815bc90 2024-04-25T10:16:00.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33fcdc6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80b4400 con 0x7f33f80ce7a0 2024-04-25T10:16:00.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33f6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33f8150e30 0x7f33f8152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33f6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33f8150e30 0x7f33f8152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39474/0 (socket says 172.21.15.60:39474) 2024-04-25T10:16:00.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33f6d9d640 1 -- 172.21.15.60:0/2604819505 learned_addr learned my addr 172.21.15.60:0/2604819505 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:00.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.802+0000 7f33f659c640 1 --2- 172.21.15.60:0/2604819505 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33f80ce7a0 0x7f33f813b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33edd9b640 1 --2- 172.21.15.60:0/2604819505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33f815bc90 0x7f33f80a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33f6d9d640 1 -- 172.21.15.60:0/2604819505 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33f80ce7a0 msgr2=0x7f33f813b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33f6d9d640 1 --2- 172.21.15.60:0/2604819505 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33f80ce7a0 0x7f33f813b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33f6d9d640 1 -- 172.21.15.60:0/2604819505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33f815bc90 msgr2=0x7f33f80a8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33f6d9d640 1 --2- 172.21.15.60:0/2604819505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33f815bc90 0x7f33f80a8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33f6d9d640 1 -- 172.21.15.60:0/2604819505 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f814fd90 con 0x7f33f8150e30 2024-04-25T10:16:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33f6d9d640 1 --2- 172.21.15.60:0/2604819505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33f8150e30 0x7f33f8152060 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f33f000ca50 tx=0x7f33f000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33ef7fe640 1 -- 172.21.15.60:0/2604819505 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0017070 con 0x7f33f8150e30 2024-04-25T10:16:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33ef7fe640 1 -- 172.21.15.60:0/2604819505 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f000abb0 con 0x7f33f8150e30 2024-04-25T10:16:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33fcdc6640 1 -- 172.21.15.60:0/2604819505 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f8150070 con 0x7f33f8150e30 2024-04-25T10:16:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33ef7fe640 1 -- 172.21.15.60:0/2604819505 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0012400 con 0x7f33f8150e30 2024-04-25T10:16:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33fcdc6640 1 -- 172.21.15.60:0/2604819505 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f8150630 con 0x7f33f8150e30 2024-04-25T10:16:00.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33fcdc6640 1 -- 172.21.15.60:0/2604819505 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f8074770 con 0x7f33f8150e30 2024-04-25T10:16:00.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33ef7fe640 1 -- 172.21.15.60:0/2604819505 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33f001a070 con 0x7f33f8150e30 2024-04-25T10:16:00.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33ef7fe640 1 --2- 172.21.15.60:0/2604819505 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f33d8066f60 0x7f33d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33ef7fe640 1 -- 172.21.15.60:0/2604819505 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f33f0084900 con 0x7f33f8150e30 2024-04-25T10:16:00.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33f659c640 1 --2- 172.21.15.60:0/2604819505 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f33d8066f60 0x7f33d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.806+0000 7f33f659c640 1 --2- 172.21.15.60:0/2604819505 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f33d8066f60 0x7f33d8069420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f33e8006fd0 tx=0x7f33e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:00.810+0000 7f33ef7fe640 1 -- 172.21.15.60:0/2604819505 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f33f0055560 con 0x7f33f8150e30 2024-04-25T10:16:01.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:01.034+0000 7f33fcdc6640 1 -- 172.21.15.60:0/2604819505 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7f33f81580d0 con 0x7f33f8150e30 2024-04-25T10:16:02.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:01.994+0000 7f33ef7fe640 1 -- 172.21.15.60:0/2604819505 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v483) v1 ==== 147+0+0 (secure 0 0 0) 0x7f33f0059600 con 0x7f33f8150e30 2024-04-25T10:16:02.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:01.998+0000 7f33fcdc6640 1 -- 172.21.15.60:0/2604819505 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f33d8066f60 msgr2=0x7f33d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:01.998+0000 7f33fcdc6640 1 --2- 172.21.15.60:0/2604819505 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f33d8066f60 0x7f33d8069420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f33e8006fd0 tx=0x7f33e8008040 comp rx=0 tx=0).stop 2024-04-25T10:16:02.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:01.998+0000 7f33fcdc6640 1 -- 172.21.15.60:0/2604819505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33f8150e30 msgr2=0x7f33f8152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:01.998+0000 7f33fcdc6640 1 --2- 172.21.15.60:0/2604819505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33f8150e30 0x7f33f8152060 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f33f000ca50 tx=0x7f33f000cf10 comp rx=0 tx=0).stop 2024-04-25T10:16:02.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:01.998+0000 7f33fcdc6640 1 -- 172.21.15.60:0/2604819505 shutdown_connections 2024-04-25T10:16:02.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:01.998+0000 7f33fcdc6640 1 --2- 172.21.15.60:0/2604819505 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f33d8066f60 0x7f33d8069420 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:01.998+0000 7f33fcdc6640 1 --2- 172.21.15.60:0/2604819505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33f8150e30 0x7f33f8152060 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:01.998+0000 7f33fcdc6640 1 --2- 172.21.15.60:0/2604819505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33f815bc90 0x7f33f80a8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:01.998+0000 7f33fcdc6640 1 --2- 172.21.15.60:0/2604819505 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33f80ce7a0 0x7f33f813b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:01.998+0000 7f33fcdc6640 1 -- 172.21.15.60:0/2604819505 >> 172.21.15.60:0/2604819505 conn(0x7f33f80b12d0 msgr2=0x7f33f8071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:02.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:01.998+0000 7f33fcdc6640 1 -- 172.21.15.60:0/2604819505 shutdown_connections 2024-04-25T10:16:02.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:01.998+0000 7f33fcdc6640 1 -- 172.21.15.60:0/2604819505 wait complete. 2024-04-25T10:16:02.024 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:16:02.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.154+0000 7fa147462640 1 Processor -- start 2024-04-25T10:16:02.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.154+0000 7fa147462640 1 -- start start 2024-04-25T10:16:02.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.154+0000 7fa147462640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1400cbcf0 0x7fa140160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.154+0000 7fa147462640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa14015fbf0 0x7fa1400ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.154+0000 7fa147462640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa140144a40 con 0x7fa140071c30 2024-04-25T10:16:02.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.154+0000 7fa147462640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1400b2dc0 con 0x7fa14015fbf0 2024-04-25T10:16:02.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.154+0000 7fa147462640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400cc9f0 con 0x7fa1400cbcf0 2024-04-25T10:16:02.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.154+0000 7fa1459ff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa14015fbf0 0x7fa1400ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.154+0000 7fa1449fd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa140071c30 0x7fa14015f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46846/0 (socket says 172.21.15.60:46846) 2024-04-25T10:16:02.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.154+0000 7fa1449fd640 1 -- 172.21.15.60:0/210296183 learned_addr learned my addr 172.21.15.60:0/210296183 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:02.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa1451fe640 1 --2- 172.21.15.60:0/210296183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1400cbcf0 0x7fa140160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa1459ff640 1 -- 172.21.15.60:0/210296183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1400cbcf0 msgr2=0x7fa140160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:02.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa1459ff640 1 --2- 172.21.15.60:0/210296183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1400cbcf0 0x7fa140160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa1459ff640 1 -- 172.21.15.60:0/210296183 >> v1:172.21.15.60:6789/0 conn(0x7fa140071c30 legacy=0x7fa14015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa1459ff640 1 -- 172.21.15.60:0/210296183 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1400cccd0 con 0x7fa14015fbf0 2024-04-25T10:16:02.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa137fff640 1 -- 172.21.15.60:0/210296183 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862323008 0 0) 0x7fa140144a40 con 0x7fa140071c30 2024-04-25T10:16:02.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa1451fe640 1 --2- 172.21.15.60:0/210296183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1400cbcf0 0x7fa140160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:02.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa1459ff640 1 --2- 172.21.15.60:0/210296183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa14015fbf0 0x7fa1400ceec0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0097c0 tx=0x7fa13c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3680d3c82d9d8caa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa137fff640 1 -- 172.21.15.60:0/210296183 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0a7070 con 0x7fa14015fbf0 2024-04-25T10:16:02.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa137fff640 1 -- 172.21.15.60:0/210296183 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c0b0070 con 0x7fa14015fbf0 2024-04-25T10:16:02.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa137fff640 1 -- 172.21.15.60:0/210296183 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0ab890 con 0x7fa14015fbf0 2024-04-25T10:16:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa147462640 1 -- 172.21.15.60:0/210296183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa14015fbf0 msgr2=0x7fa1400ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa147462640 1 --2- 172.21.15.60:0/210296183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa14015fbf0 0x7fa1400ceec0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0097c0 tx=0x7fa13c01c470 comp rx=0 tx=0).stop 2024-04-25T10:16:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa147462640 1 -- 172.21.15.60:0/210296183 shutdown_connections 2024-04-25T10:16:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa147462640 1 --2- 172.21.15.60:0/210296183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa14015fbf0 0x7fa1400ceec0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa147462640 1 --2- 172.21.15.60:0/210296183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1400cbcf0 0x7fa140160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa147462640 1 -- 172.21.15.60:0/210296183 >> 172.21.15.60:0/210296183 conn(0x7fa1400670b0 msgr2=0x7fa14006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa147462640 1 -- 172.21.15.60:0/210296183 shutdown_connections 2024-04-25T10:16:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.158+0000 7fa147462640 1 -- 172.21.15.60:0/210296183 wait complete. 2024-04-25T10:16:02.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa147462640 1 Processor -- start 2024-04-25T10:16:02.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa147462640 1 -- start start 2024-04-25T10:16:02.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa147462640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa140071c30 0x7fa14015b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa147462640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1400cbcf0 0x7fa14013c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa147462640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa14015fbf0 0x7fa14015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa147462640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa140072070 con 0x7fa14015fbf0 2024-04-25T10:16:02.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa147462640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400ce5c0 con 0x7fa1400cbcf0 2024-04-25T10:16:02.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa147462640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14013dbf0 con 0x7fa140071c30 2024-04-25T10:16:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa1459ff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa14015fbf0 0x7fa14015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa1459ff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa14015fbf0 0x7fa14015dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39490/0 (socket says 172.21.15.60:39490) 2024-04-25T10:16:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa1459ff640 1 -- 172.21.15.60:0/1290401512 learned_addr learned my addr 172.21.15.60:0/1290401512 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa1449fd640 1 --2- 172.21.15.60:0/1290401512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1400cbcf0 0x7fa14013c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa1451fe640 1 --2- 172.21.15.60:0/1290401512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa140071c30 0x7fa14015b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa1459ff640 1 -- 172.21.15.60:0/1290401512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa140071c30 msgr2=0x7fa14015b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa1459ff640 1 --2- 172.21.15.60:0/1290401512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa140071c30 0x7fa14015b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa1459ff640 1 -- 172.21.15.60:0/1290401512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1400cbcf0 msgr2=0x7fa14013c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa1459ff640 1 --2- 172.21.15.60:0/1290401512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1400cbcf0 0x7fa14013c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa1459ff640 1 -- 172.21.15.60:0/1290401512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14015f000 con 0x7fa14015fbf0 2024-04-25T10:16:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa1449fd640 1 --2- 172.21.15.60:0/1290401512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1400cbcf0 0x7fa14013c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa1451fe640 1 --2- 172.21.15.60:0/1290401512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa140071c30 0x7fa14015b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa1459ff640 1 --2- 172.21.15.60:0/1290401512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa14015fbf0 0x7fa14015dcf0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0ad040 tx=0x7fa13c019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa135ffb640 1 -- 172.21.15.60:0/1290401512 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c004230 con 0x7fa14015fbf0 2024-04-25T10:16:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa135ffb640 1 -- 172.21.15.60:0/1290401512 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c0043d0 con 0x7fa14015fbf0 2024-04-25T10:16:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa147462640 1 -- 172.21.15.60:0/1290401512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14013d350 con 0x7fa14015fbf0 2024-04-25T10:16:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa135ffb640 1 -- 172.21.15.60:0/1290401512 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0abb10 con 0x7fa14015fbf0 2024-04-25T10:16:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa147462640 1 -- 172.21.15.60:0/1290401512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14013d910 con 0x7fa14015fbf0 2024-04-25T10:16:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.162+0000 7fa147462640 1 -- 172.21.15.60:0/1290401512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa14006bba0 con 0x7fa14015fbf0 2024-04-25T10:16:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.166+0000 7fa135ffb640 1 -- 172.21.15.60:0/1290401512 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa13c0c5060 con 0x7fa14015fbf0 2024-04-25T10:16:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.166+0000 7fa135ffb640 1 --2- 172.21.15.60:0/1290401512 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa124066f60 0x7fa124069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.166+0000 7fa135ffb640 1 -- 172.21.15.60:0/1290401512 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fa13c11ec60 con 0x7fa14015fbf0 2024-04-25T10:16:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.166+0000 7fa1451fe640 1 --2- 172.21.15.60:0/1290401512 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa124066f60 0x7fa124069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.166+0000 7fa1451fe640 1 --2- 172.21.15.60:0/1290401512 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa124066f60 0x7fa124069420 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa13800d430 tx=0x7fa138007530 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.170+0000 7fa135ffb640 1 -- 172.21.15.60:0/1290401512 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa13c0b9070 con 0x7fa14015fbf0 2024-04-25T10:16:02.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.330+0000 7fa147462640 1 -- 172.21.15.60:0/1290401512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa140074770 con 0x7fa14015fbf0 2024-04-25T10:16:02.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.338+0000 7fa135ffb640 1 -- 172.21.15.60:0/1290401512 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa13c016090 con 0x7fa14015fbf0 2024-04-25T10:16:02.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.338+0000 7fa147462640 1 -- 172.21.15.60:0/1290401512 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa124066f60 msgr2=0x7fa124069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.338+0000 7fa147462640 1 --2- 172.21.15.60:0/1290401512 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa124066f60 0x7fa124069420 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa13800d430 tx=0x7fa138007530 comp rx=0 tx=0).stop 2024-04-25T10:16:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.338+0000 7fa147462640 1 -- 172.21.15.60:0/1290401512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa14015fbf0 msgr2=0x7fa14015dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.338+0000 7fa147462640 1 --2- 172.21.15.60:0/1290401512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa14015fbf0 0x7fa14015dcf0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0ad040 tx=0x7fa13c019210 comp rx=0 tx=0).stop 2024-04-25T10:16:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.338+0000 7fa147462640 1 -- 172.21.15.60:0/1290401512 shutdown_connections 2024-04-25T10:16:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.338+0000 7fa147462640 1 --2- 172.21.15.60:0/1290401512 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa124066f60 0x7fa124069420 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.338+0000 7fa147462640 1 --2- 172.21.15.60:0/1290401512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa14015fbf0 0x7fa14015dcf0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.338+0000 7fa147462640 1 --2- 172.21.15.60:0/1290401512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1400cbcf0 0x7fa14013c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.338+0000 7fa147462640 1 --2- 172.21.15.60:0/1290401512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa140071c30 0x7fa14015b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.338+0000 7fa147462640 1 -- 172.21.15.60:0/1290401512 >> 172.21.15.60:0/1290401512 conn(0x7fa1400670b0 msgr2=0x7fa1400cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.338+0000 7fa147462640 1 -- 172.21.15.60:0/1290401512 shutdown_connections 2024-04-25T10:16:02.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.338+0000 7fa147462640 1 -- 172.21.15.60:0/1290401512 wait complete. 2024-04-25T10:16:02.361 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:16:02.361 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:16:02.403 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:16:02.463 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:16:02.463 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:16:02.463 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:02.469 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:16:02.469 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:16:02.470 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:02.475 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:16:02.475 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:16:02.475 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:02.518 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:16:02.518 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:16:02.518 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:02.524 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:16:02.524 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:16:02.524 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:02.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.690+0000 7f5ff3721640 1 Processor -- start 2024-04-25T10:16:02.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.690+0000 7f5ff3721640 1 -- start start 2024-04-25T10:16:02.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.690+0000 7f5ff3721640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5fec0cde60 0x7f5fec164e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.690+0000 7f5ff3721640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fec15dc10 con 0x7f5fec071b20 2024-04-25T10:16:02.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.690+0000 7f5ff3721640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fec15def0 con 0x7f5fec06e480 2024-04-25T10:16:02.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.690+0000 7f5ff3721640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fec15e1d0 con 0x7f5fec0cde60 2024-04-25T10:16:02.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.690+0000 7f5ff14bd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5fec071b20 0x7f5fec071f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46858/0 (socket says 172.21.15.60:46858) 2024-04-25T10:16:02.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.690+0000 7f5ff14bd640 1 -- 172.21.15.60:0/1279561108 learned_addr learned my addr 172.21.15.60:0/1279561108 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:02.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.690+0000 7f5ff1cbe640 1 --2- 172.21.15.60:0/1279561108 >> v2:172.21.15.60:3300/0 conn(0x7f5fec0cde60 0x7f5fec164e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.690+0000 7f5fe3fff640 1 -- 172.21.15.60:0/1279561108 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1768984291 0 0) 0x7f5fec15dc10 con 0x7f5fec071b20 2024-04-25T10:16:02.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.690+0000 7f5fe3fff640 1 -- 172.21.15.60:0/1279561108 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fd8003530 con 0x7f5fec071b20 2024-04-25T10:16:02.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.690+0000 7f5ff1cbe640 1 -- 172.21.15.60:0/1279561108 >> v1:172.21.15.100:6789/0 conn(0x7f5fec06e480 legacy=0x7f5fec06e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:02.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.690+0000 7f5ff1cbe640 1 -- 172.21.15.60:0/1279561108 >> v1:172.21.15.60:6789/0 conn(0x7f5fec071b20 legacy=0x7f5fec071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.690+0000 7f5ff1cbe640 1 -- 172.21.15.60:0/1279561108 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec15e4b0 con 0x7f5fec0cde60 2024-04-25T10:16:02.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff1cbe640 1 --2- 172.21.15.60:0/1279561108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec0cde60 0x7f5fec164e50 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400aea0 tx=0x7f5fe401d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b310c38c56ec30fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5fe3fff640 1 -- 172.21.15.60:0/1279561108 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40a6640 con 0x7f5fec0cde60 2024-04-25T10:16:02.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5fe3fff640 1 -- 172.21.15.60:0/1279561108 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe40a67e0 con 0x7f5fec0cde60 2024-04-25T10:16:02.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5fe3fff640 1 -- 172.21.15.60:0/1279561108 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40a6b90 con 0x7f5fec0cde60 2024-04-25T10:16:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff3721640 1 -- 172.21.15.60:0/1279561108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec0cde60 msgr2=0x7f5fec164e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff3721640 1 --2- 172.21.15.60:0/1279561108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec0cde60 0x7f5fec164e50 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400aea0 tx=0x7f5fe401d9e0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff3721640 1 -- 172.21.15.60:0/1279561108 shutdown_connections 2024-04-25T10:16:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff3721640 1 --2- 172.21.15.60:0/1279561108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec0cde60 0x7f5fec164e50 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff3721640 1 -- 172.21.15.60:0/1279561108 >> 172.21.15.60:0/1279561108 conn(0x7f5fec0b12d0 msgr2=0x7f5fec0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff3721640 1 -- 172.21.15.60:0/1279561108 shutdown_connections 2024-04-25T10:16:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff3721640 1 -- 172.21.15.60:0/1279561108 wait complete. 2024-04-25T10:16:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff3721640 1 Processor -- start 2024-04-25T10:16:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff3721640 1 -- start start 2024-04-25T10:16:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff3721640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fec06e480 0x7f5fec158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff3721640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec071b20 0x7f5fec0cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff3721640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fec0cde60 0x7f5fec0cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff3721640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec167730 con 0x7f5fec071b20 2024-04-25T10:16:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff3721640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec1675b0 con 0x7f5fec06e480 2024-04-25T10:16:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff3721640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec1678b0 con 0x7f5fec0cde60 2024-04-25T10:16:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff0cbc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec071b20 0x7f5fec0cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff0cbc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec071b20 0x7f5fec0cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39502/0 (socket says 172.21.15.60:39502) 2024-04-25T10:16:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff0cbc640 1 -- 172.21.15.60:0/1599750290 learned_addr learned my addr 172.21.15.60:0/1599750290 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff14bd640 1 --2- 172.21.15.60:0/1599750290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fec06e480 0x7f5fec158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff1cbe640 1 --2- 172.21.15.60:0/1599750290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fec0cde60 0x7f5fec0cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff0cbc640 1 -- 172.21.15.60:0/1599750290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fec0cde60 msgr2=0x7f5fec0cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff0cbc640 1 --2- 172.21.15.60:0/1599750290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fec0cde60 0x7f5fec0cfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff0cbc640 1 -- 172.21.15.60:0/1599750290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fec06e480 msgr2=0x7f5fec158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff0cbc640 1 --2- 172.21.15.60:0/1599750290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fec06e480 0x7f5fec158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.694+0000 7f5ff0cbc640 1 -- 172.21.15.60:0/1599750290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec15c110 con 0x7f5fec071b20 2024-04-25T10:16:02.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.698+0000 7f5ff0cbc640 1 --2- 172.21.15.60:0/1599750290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec071b20 0x7f5fec0cf520 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f5fe800c450 tx=0x7f5fe800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.698+0000 7f5fe1ffb640 1 -- 172.21.15.60:0/1599750290 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8016020 con 0x7f5fec071b20 2024-04-25T10:16:02.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.698+0000 7f5fe1ffb640 1 -- 172.21.15.60:0/1599750290 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe80052d0 con 0x7f5fec071b20 2024-04-25T10:16:02.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.698+0000 7f5ff3721640 1 -- 172.21.15.60:0/1599750290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fec15c3f0 con 0x7f5fec071b20 2024-04-25T10:16:02.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.698+0000 7f5fe1ffb640 1 -- 172.21.15.60:0/1599750290 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8007260 con 0x7f5fec071b20 2024-04-25T10:16:02.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.698+0000 7f5ff3721640 1 -- 172.21.15.60:0/1599750290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fec15c9b0 con 0x7f5fec071b20 2024-04-25T10:16:02.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.698+0000 7f5ff3721640 1 -- 172.21.15.60:0/1599750290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fec074770 con 0x7f5fec071b20 2024-04-25T10:16:02.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.698+0000 7f5fe1ffb640 1 -- 172.21.15.60:0/1599750290 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5fe8007400 con 0x7f5fec071b20 2024-04-25T10:16:02.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.698+0000 7f5fe1ffb640 1 --2- 172.21.15.60:0/1599750290 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5fd0066f60 0x7f5fd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.698+0000 7f5fe1ffb640 1 -- 172.21.15.60:0/1599750290 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f5fe80846d0 con 0x7f5fec071b20 2024-04-25T10:16:02.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.698+0000 7f5ff14bd640 1 --2- 172.21.15.60:0/1599750290 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5fd0066f60 0x7f5fd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.698+0000 7f5ff14bd640 1 --2- 172.21.15.60:0/1599750290 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5fd0066f60 0x7f5fd0069420 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0011f0 tx=0x7f5fdc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.702+0000 7f5fe1ffb640 1 -- 172.21.15.60:0/1599750290 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5fe8020020 con 0x7f5fec071b20 2024-04-25T10:16:02.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.898+0000 7f5ff3721640 1 -- 172.21.15.60:0/1599750290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fec06ea30 con 0x7f5fec071b20 2024-04-25T10:16:02.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.898+0000 7f5fe1ffb640 1 -- 172.21.15.60:0/1599750290 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f5fe8005480 con 0x7f5fec071b20 2024-04-25T10:16:02.905 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:02.905 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1491228634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":1491228634},{"type":"v1","addr":"172.21.15.100:6837","nonce":1491228634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/209898872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":209898872},{"type":"v1","addr":"172.21.15.100:6835","nonce":209898872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1793343287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1793343287},{"type":"v1","addr":"172.21.15.60:6833","nonce":1793343287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":9198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1365405879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1365405879},{"type":"v1","addr":"172.21.15.60:6835","nonce":1365405879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[]} 2024-04-25T10:16:02.906 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 87 2024-04-25T10:16:02.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.902+0000 7f5ff3721640 1 -- 172.21.15.60:0/1599750290 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5fd0066f60 msgr2=0x7f5fd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.902+0000 7f5ff3721640 1 --2- 172.21.15.60:0/1599750290 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5fd0066f60 0x7f5fd0069420 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0011f0 tx=0x7f5fdc009040 comp rx=0 tx=0).stop 2024-04-25T10:16:02.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.902+0000 7f5ff3721640 1 -- 172.21.15.60:0/1599750290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec071b20 msgr2=0x7f5fec0cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.902+0000 7f5ff3721640 1 --2- 172.21.15.60:0/1599750290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec071b20 0x7f5fec0cf520 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f5fe800c450 tx=0x7f5fe800c910 comp rx=0 tx=0).stop 2024-04-25T10:16:02.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.902+0000 7f5ff3721640 1 -- 172.21.15.60:0/1599750290 shutdown_connections 2024-04-25T10:16:02.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.902+0000 7f5ff3721640 1 --2- 172.21.15.60:0/1599750290 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5fd0066f60 0x7f5fd0069420 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.902+0000 7f5ff3721640 1 --2- 172.21.15.60:0/1599750290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fec0cde60 0x7f5fec0cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.902+0000 7f5ff3721640 1 --2- 172.21.15.60:0/1599750290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec071b20 0x7f5fec0cf520 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.902+0000 7f5ff3721640 1 --2- 172.21.15.60:0/1599750290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fec06e480 0x7f5fec158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.902+0000 7f5ff3721640 1 -- 172.21.15.60:0/1599750290 >> 172.21.15.60:0/1599750290 conn(0x7f5fec0b12d0 msgr2=0x7f5fec0cb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:02.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.902+0000 7f5ff3721640 1 -- 172.21.15.60:0/1599750290 shutdown_connections 2024-04-25T10:16:02.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:02.902+0000 7f5ff3721640 1 -- 172.21.15.60:0/1599750290 wait complete. 2024-04-25T10:16:02.925 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:16:02.926 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:16:02.926 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:16:02.926 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:02.926 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:16:02.926 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:16:02.926 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:16:02.927 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:02.927 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:16:02.927 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:16:02.927 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:16:02.927 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:02.927 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:16:02.927 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:16:02.928 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:16:02.928 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:02.938 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:16:02.938 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:16:02.939 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:16:02.939 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:16:02.940 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:16:02.940 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:16:02.946 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:16:02.947 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:16:02.986 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:16:02.987 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:02.987 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:16:02.987 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:02.989 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:16:02.990 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:02.991 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:16:02.991 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:02.991 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:16:03.057 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T10:16:03.058 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T10:16:03.059 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:16:03.060 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T10:16:03.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.102+0000 7f6cd691d640 1 Processor -- start 2024-04-25T10:16:03.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.102+0000 7f6cd691d640 1 -- start start 2024-04-25T10:16:03.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.102+0000 7f6cd691d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6cd015deb0 0x7f6cd00b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.102+0000 7f6cd691d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6cd00d0340 0x7f6cd00d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.102+0000 7f6cd691d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cd00ce2b0 0x7f6cd00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.102+0000 7f6cd691d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd00cf2c0 con 0x7f6cd00d0340 2024-04-25T10:16:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.102+0000 7f6cd691d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd00cefe0 con 0x7f6cd015deb0 2024-04-25T10:16:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.102+0000 7f6cd691d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00d1d70 con 0x7f6cd00ce2b0 2024-04-25T10:16:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.102+0000 7f6cc7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6cd00d0340 0x7f6cd00d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.102+0000 7f6ccffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6cd015deb0 0x7f6cd00b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.102+0000 7f6cc7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6cd00d0340 0x7f6cd00d07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39544/0 (socket says 172.21.15.60:39544) 2024-04-25T10:16:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.102+0000 7f6cc7fff640 1 -- 172.21.15.60:0/4090640248 learned_addr learned my addr 172.21.15.60:0/4090640248 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.102+0000 7f6cd4eba640 1 --2- 172.21.15.60:0/4090640248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cd00ce2b0 0x7f6cd00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6ccffff640 1 -- 172.21.15.60:0/4090640248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cd00ce2b0 msgr2=0x7f6cd00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6ccffff640 1 --2- 172.21.15.60:0/4090640248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cd00ce2b0 0x7f6cd00ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6ccffff640 1 -- 172.21.15.60:0/4090640248 >> v2:172.21.15.60:3300/0 conn(0x7f6cd00d0340 msgr2=0x7f6cd00d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:03.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6ccffff640 1 --2- 172.21.15.60:0/4090640248 >> v2:172.21.15.60:3300/0 conn(0x7f6cd00d0340 0x7f6cd00d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6ccffff640 1 -- 172.21.15.60:0/4090640248 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd006d200 con 0x7f6cd015deb0 2024-04-25T10:16:03.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6cc7fff640 1 --2- 172.21.15.60:0/4090640248 >> v2:172.21.15.60:3300/0 conn(0x7f6cd00d0340 0x7f6cd00d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:03.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6ccffff640 1 --2- 172.21.15.60:0/4090640248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd015deb0 0x7f6cd00b6c10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6cd008bd20 tx=0x7f6cc001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21a17b6bdad6555e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6ccf7fe640 1 -- 172.21.15.60:0/4090640248 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc00b0070 con 0x7f6cd015deb0 2024-04-25T10:16:03.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6ccf7fe640 1 -- 172.21.15.60:0/4090640248 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc00ab600 con 0x7f6cd015deb0 2024-04-25T10:16:03.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6ccf7fe640 1 -- 172.21.15.60:0/4090640248 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc00ab9d0 con 0x7f6cd015deb0 2024-04-25T10:16:03.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6cd691d640 1 -- 172.21.15.60:0/4090640248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd015deb0 msgr2=0x7f6cd00b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6cd691d640 1 --2- 172.21.15.60:0/4090640248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd015deb0 0x7f6cd00b6c10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6cd008bd20 tx=0x7f6cc001c480 comp rx=0 tx=0).stop 2024-04-25T10:16:03.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6cd691d640 1 -- 172.21.15.60:0/4090640248 shutdown_connections 2024-04-25T10:16:03.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6cd691d640 1 --2- 172.21.15.60:0/4090640248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cd00ce2b0 0x7f6cd00ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6cd691d640 1 --2- 172.21.15.60:0/4090640248 >> v2:172.21.15.60:3300/0 conn(0x7f6cd00d0340 0x7f6cd00d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6cd691d640 1 --2- 172.21.15.60:0/4090640248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd015deb0 0x7f6cd00b6c10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6cd691d640 1 -- 172.21.15.60:0/4090640248 >> 172.21.15.60:0/4090640248 conn(0x7f6cd006c090 msgr2=0x7f6cd006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:03.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6cd691d640 1 -- 172.21.15.60:0/4090640248 shutdown_connections 2024-04-25T10:16:03.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.106+0000 7f6cd691d640 1 -- 172.21.15.60:0/4090640248 wait complete. 2024-04-25T10:16:03.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cd691d640 1 Processor -- start 2024-04-25T10:16:03.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cd691d640 1 -- start start 2024-04-25T10:16:03.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cd691d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cd00ce2b0 0x7f6cd01536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cd691d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cd00d0340 0x7f6cd00cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cd691d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd015deb0 0x7f6cd0162ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cd691d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00b5d40 con 0x7f6cd00d0340 2024-04-25T10:16:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cd691d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00b5bc0 con 0x7f6cd015deb0 2024-04-25T10:16:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cd691d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00b5ec0 con 0x7f6cd00ce2b0 2024-04-25T10:16:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cc7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cd00d0340 0x7f6cd00cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cc7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cd00d0340 0x7f6cd00cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39554/0 (socket says 172.21.15.60:39554) 2024-04-25T10:16:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cc7fff640 1 -- 172.21.15.60:0/1136379738 learned_addr learned my addr 172.21.15.60:0/1136379738 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6ccffff640 1 --2- 172.21.15.60:0/1136379738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cd00ce2b0 0x7f6cd01536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cd4eba640 1 --2- 172.21.15.60:0/1136379738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd015deb0 0x7f6cd0162ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cc7fff640 1 -- 172.21.15.60:0/1136379738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cd00ce2b0 msgr2=0x7f6cd01536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cc7fff640 1 --2- 172.21.15.60:0/1136379738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cd00ce2b0 0x7f6cd01536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cc7fff640 1 -- 172.21.15.60:0/1136379738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd015deb0 msgr2=0x7f6cd0162ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cc7fff640 1 --2- 172.21.15.60:0/1136379738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd015deb0 0x7f6cd0162ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cc7fff640 1 -- 172.21.15.60:0/1136379738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd00ca270 con 0x7f6cd00d0340 2024-04-25T10:16:03.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cc7fff640 1 --2- 172.21.15.60:0/1136379738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cd00d0340 0x7f6cd00cc020 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800e9d0 tx=0x7f6cb800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6ccd7fa640 1 -- 172.21.15.60:0/1136379738 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8017070 con 0x7f6cd00d0340 2024-04-25T10:16:03.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6ccd7fa640 1 -- 172.21.15.60:0/1136379738 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb800ca60 con 0x7f6cd00d0340 2024-04-25T10:16:03.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6ccd7fa640 1 -- 172.21.15.60:0/1136379738 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb800ce10 con 0x7f6cd00d0340 2024-04-25T10:16:03.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cd691d640 1 -- 172.21.15.60:0/1136379738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd00caf80 con 0x7f6cd00d0340 2024-04-25T10:16:03.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cd691d640 1 -- 172.21.15.60:0/1136379738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd00cb540 con 0x7f6cd00d0340 2024-04-25T10:16:03.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6cd691d640 1 -- 172.21.15.60:0/1136379738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd0074770 con 0x7f6cd00d0340 2024-04-25T10:16:03.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.110+0000 7f6ccd7fa640 1 -- 172.21.15.60:0/1136379738 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6cb801a070 con 0x7f6cd00d0340 2024-04-25T10:16:03.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.114+0000 7f6ccd7fa640 1 --2- 172.21.15.60:0/1136379738 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6cb0066f60 0x7f6cb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.114+0000 7f6ccd7fa640 1 -- 172.21.15.60:0/1136379738 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f6cb8084fb0 con 0x7f6cd00d0340 2024-04-25T10:16:03.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.114+0000 7f6ccffff640 1 --2- 172.21.15.60:0/1136379738 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6cb0066f60 0x7f6cb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.114+0000 7f6ccffff640 1 --2- 172.21.15.60:0/1136379738 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6cb0066f60 0x7f6cb0069420 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00ad000 tx=0x7f6cc00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.118+0000 7f6ccd7fa640 1 -- 172.21.15.60:0/1136379738 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6cb8055c20 con 0x7f6cd00d0340 2024-04-25T10:16:03.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:03.282+0000 7f6cd691d640 1 -- 172.21.15.60:0/1136379738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6cd00b8090 con 0x7f6cd00d0340 2024-04-25T10:16:04.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.362+0000 7f6ccd7fa640 1 -- 172.21.15.60:0/1136379738 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v106) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6cb8059cc0 con 0x7f6cd00d0340 2024-04-25T10:16:04.367 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-04-25T10:16:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.366+0000 7f6cd691d640 1 -- 172.21.15.60:0/1136379738 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6cb0066f60 msgr2=0x7f6cb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.366+0000 7f6cd691d640 1 --2- 172.21.15.60:0/1136379738 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6cb0066f60 0x7f6cb0069420 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00ad000 tx=0x7f6cc00a6000 comp rx=0 tx=0).stop 2024-04-25T10:16:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.366+0000 7f6cd691d640 1 -- 172.21.15.60:0/1136379738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cd00d0340 msgr2=0x7f6cd00cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.366+0000 7f6cd691d640 1 --2- 172.21.15.60:0/1136379738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cd00d0340 0x7f6cd00cc020 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800e9d0 tx=0x7f6cb800ee90 comp rx=0 tx=0).stop 2024-04-25T10:16:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.366+0000 7f6cd691d640 1 -- 172.21.15.60:0/1136379738 shutdown_connections 2024-04-25T10:16:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.366+0000 7f6cd691d640 1 --2- 172.21.15.60:0/1136379738 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6cb0066f60 0x7f6cb0069420 secure :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6cc00ad000 tx=0x7f6cc00a6000 comp rx=0 tx=0).stop 2024-04-25T10:16:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.366+0000 7f6cd691d640 1 --2- 172.21.15.60:0/1136379738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd015deb0 0x7f6cd0162ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.366+0000 7f6cd691d640 1 --2- 172.21.15.60:0/1136379738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cd00d0340 0x7f6cd00cc020 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.366+0000 7f6cd691d640 1 --2- 172.21.15.60:0/1136379738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cd00ce2b0 0x7f6cd01536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.366+0000 7f6cd691d640 1 -- 172.21.15.60:0/1136379738 >> 172.21.15.60:0/1136379738 conn(0x7f6cd006c090 msgr2=0x7f6cd01515c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:04.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.366+0000 7f6cd691d640 1 -- 172.21.15.60:0/1136379738 shutdown_connections 2024-04-25T10:16:04.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.366+0000 7f6cd691d640 1 -- 172.21.15.60:0/1136379738 wait complete. 2024-04-25T10:16:04.389 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:16:04.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.510+0000 7fe25670f640 1 Processor -- start 2024-04-25T10:16:04.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.510+0000 7fe25670f640 1 -- start start 2024-04-25T10:16:04.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.514+0000 7fe25670f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe2500ce7a0 0x7fe2500ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.514+0000 7fe25670f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2500b4660 0x7fe2500b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.514+0000 7fe25670f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2500cf0c0 con 0x7fe25015bc90 2024-04-25T10:16:04.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.514+0000 7fe25670f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2500cc370 con 0x7fe2500ce7a0 2024-04-25T10:16:04.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.514+0000 7fe25670f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe250071ca0 con 0x7fe2500b4660 2024-04-25T10:16:04.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.514+0000 7fe254cac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2500b4660 0x7fe2500b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.514+0000 7fe24ffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe2500ce7a0 0x7fe2500ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.514+0000 7fe24f7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe25015bc90 0x7fe2500b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:49740/0 (socket says 172.21.15.60:49740) 2024-04-25T10:16:04.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.514+0000 7fe24f7fe640 1 -- 172.21.15.60:0/453771801 learned_addr learned my addr 172.21.15.60:0/453771801 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:04.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.514+0000 7fe24effd640 1 -- 172.21.15.60:0/453771801 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3727139437 0 0) 0x7fe2500cf0c0 con 0x7fe25015bc90 2024-04-25T10:16:04.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.514+0000 7fe24effd640 1 -- 172.21.15.60:0/453771801 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe23c003530 con 0x7fe25015bc90 2024-04-25T10:16:04.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.514+0000 7fe254cac640 1 -- 172.21.15.60:0/453771801 >> v2:172.21.15.100:3300/0 conn(0x7fe2500ce7a0 msgr2=0x7fe2500ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:04.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.514+0000 7fe254cac640 1 --2- 172.21.15.60:0/453771801 >> v2:172.21.15.100:3300/0 conn(0x7fe2500ce7a0 0x7fe2500ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.514+0000 7fe254cac640 1 -- 172.21.15.60:0/453771801 >> v1:172.21.15.100:6789/0 conn(0x7fe25015bc90 legacy=0x7fe2500b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.514+0000 7fe254cac640 1 -- 172.21.15.60:0/453771801 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2500cca40 con 0x7fe2500b4660 2024-04-25T10:16:04.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.514+0000 7fe24ffff640 1 --2- 172.21.15.60:0/453771801 >> v2:172.21.15.100:3300/0 conn(0x7fe2500ce7a0 0x7fe2500ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:04.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.518+0000 7fe254cac640 1 --2- 172.21.15.60:0/453771801 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2500b4660 0x7fe2500b2ce0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe248009d90 tx=0x7fe24801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e9815794af414b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.518+0000 7fe24effd640 1 -- 172.21.15.60:0/453771801 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2480b2070 con 0x7fe2500b4660 2024-04-25T10:16:04.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.518+0000 7fe24effd640 1 -- 172.21.15.60:0/453771801 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2480a5830 con 0x7fe2500b4660 2024-04-25T10:16:04.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.518+0000 7fe24effd640 1 -- 172.21.15.60:0/453771801 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2480a5be0 con 0x7fe2500b4660 2024-04-25T10:16:04.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.518+0000 7fe25670f640 1 -- 172.21.15.60:0/453771801 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2500b4660 msgr2=0x7fe2500b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.518+0000 7fe25670f640 1 --2- 172.21.15.60:0/453771801 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2500b4660 0x7fe2500b2ce0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe248009d90 tx=0x7fe24801c970 comp rx=0 tx=0).stop 2024-04-25T10:16:04.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.518+0000 7fe25670f640 1 -- 172.21.15.60:0/453771801 shutdown_connections 2024-04-25T10:16:04.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.518+0000 7fe25670f640 1 --2- 172.21.15.60:0/453771801 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2500b4660 0x7fe2500b2ce0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.518+0000 7fe25670f640 1 --2- 172.21.15.60:0/453771801 >> v2:172.21.15.100:3300/0 conn(0x7fe2500ce7a0 0x7fe2500ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.518+0000 7fe25670f640 1 -- 172.21.15.60:0/453771801 >> 172.21.15.60:0/453771801 conn(0x7fe2500b12d0 msgr2=0x7fe2500b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:04.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.518+0000 7fe25670f640 1 -- 172.21.15.60:0/453771801 shutdown_connections 2024-04-25T10:16:04.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.518+0000 7fe25670f640 1 -- 172.21.15.60:0/453771801 wait complete. 2024-04-25T10:16:04.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.518+0000 7fe25670f640 1 Processor -- start 2024-04-25T10:16:04.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.518+0000 7fe25670f640 1 -- start start 2024-04-25T10:16:04.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe25670f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2500ce7a0 0x7fe2501500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe25670f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe25015bc90 0x7fe2501470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe25670f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2501497a0 0x7fe250154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe25670f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2500b4280 con 0x7fe25015bc90 2024-04-25T10:16:04.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe25670f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25006f300 con 0x7fe2501497a0 2024-04-25T10:16:04.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe25670f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2500b4400 con 0x7fe2500ce7a0 2024-04-25T10:16:04.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe25015bc90 0x7fe2501470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe25015bc90 0x7fe2501470b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39562/0 (socket says 172.21.15.60:39562) 2024-04-25T10:16:04.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24f7fe640 1 -- 172.21.15.60:0/2881004541 learned_addr learned my addr 172.21.15.60:0/2881004541 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:04.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24ffff640 1 --2- 172.21.15.60:0/2881004541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2500ce7a0 0x7fe2501500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe254cac640 1 --2- 172.21.15.60:0/2881004541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2501497a0 0x7fe250154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24f7fe640 1 -- 172.21.15.60:0/2881004541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2500ce7a0 msgr2=0x7fe2501500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24f7fe640 1 --2- 172.21.15.60:0/2881004541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2500ce7a0 0x7fe2501500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24f7fe640 1 -- 172.21.15.60:0/2881004541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2501497a0 msgr2=0x7fe250154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24f7fe640 1 --2- 172.21.15.60:0/2881004541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2501497a0 0x7fe250154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24f7fe640 1 -- 172.21.15.60:0/2881004541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe250150760 con 0x7fe25015bc90 2024-04-25T10:16:04.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24f7fe640 1 --2- 172.21.15.60:0/2881004541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe25015bc90 0x7fe2501470b0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fe23800c3c0 tx=0x7fe23800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24cff9640 1 -- 172.21.15.60:0/2881004541 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238017070 con 0x7fe25015bc90 2024-04-25T10:16:04.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe25670f640 1 -- 172.21.15.60:0/2881004541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe250150a40 con 0x7fe25015bc90 2024-04-25T10:16:04.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24cff9640 1 -- 172.21.15.60:0/2881004541 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe238005cf0 con 0x7fe25015bc90 2024-04-25T10:16:04.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24cff9640 1 -- 172.21.15.60:0/2881004541 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2380072f0 con 0x7fe25015bc90 2024-04-25T10:16:04.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe25670f640 1 -- 172.21.15.60:0/2881004541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe250151000 con 0x7fe25015bc90 2024-04-25T10:16:04.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe25670f640 1 -- 172.21.15.60:0/2881004541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe250074770 con 0x7fe25015bc90 2024-04-25T10:16:04.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24cff9640 1 -- 172.21.15.60:0/2881004541 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe23801a070 con 0x7fe25015bc90 2024-04-25T10:16:04.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24cff9640 1 --2- 172.21.15.60:0/2881004541 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe230066f60 0x7fe230069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24cff9640 1 -- 172.21.15.60:0/2881004541 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe238084180 con 0x7fe25015bc90 2024-04-25T10:16:04.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.522+0000 7fe24ffff640 1 --2- 172.21.15.60:0/2881004541 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe230066f60 0x7fe230069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.526+0000 7fe24ffff640 1 --2- 172.21.15.60:0/2881004541 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe230066f60 0x7fe230069420 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe240009a50 tx=0x7fe240008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.526+0000 7fe24cff9640 1 -- 172.21.15.60:0/2881004541 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2380553d0 con 0x7fe25015bc90 2024-04-25T10:16:04.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.742+0000 7fe25670f640 1 -- 172.21.15.60:0/2881004541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe250153f10 con 0x7fe25015bc90 2024-04-25T10:16:04.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.742+0000 7fe24cff9640 1 -- 172.21.15.60:0/2881004541 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe238082e20 con 0x7fe25015bc90 2024-04-25T10:16:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:04.748 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:16:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-04-25T10:16:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-04-25T10:16:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmKr12EhAAnHHZ5VjG0cywMrIv6VElVg==", 2024-04-25T10:16:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:16:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:16:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:16:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:16:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-04-25T10:16:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmU39rIRAAk6Ca0tKG4LpmR4pNxjzv0w==", 2024-04-25T10:16:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:16:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:16:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:16:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:16:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-04-25T10:16:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmVsshGxAAttXv0jYEGKgo0sPhIRTuqQ==", 2024-04-25T10:16:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:16:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:16:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:16:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:16:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-04-25T10:16:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmQtXUKRAAI3Iq02h3KH/PZbWKvJD9Jg==", 2024-04-25T10:16:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:16:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:16:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:16:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:16:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-04-25T10:16:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCkKipmE+UIBRAAc8ctpHoL3PkkbkMRCAJUGw==", 2024-04-25T10:16:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:16:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:16:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-04-25T10:16:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCnKipmbAnYCBAALLsPH2FjUpjoIZwTw+8aOA==", 2024-04-25T10:16:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:16:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:16:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-04-25T10:16:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCqKipmqrNSDxAAZdgFrcyeVwRQMKJJYIM98g==", 2024-04-25T10:16:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:16:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:16:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-04-25T10:16:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCtKipm1L+hFxAAPnPWlCm1Fmi0mN3tyReeQw==", 2024-04-25T10:16:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:16:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:16:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-04-25T10:16:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCzKipm2WyDFRAAmyfg6qu5drE9yMaR2FKd/A==", 2024-04-25T10:16:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:16:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:16:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-04-25T10:16:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC2KipmZLDhGRAAcgaz8YaPGUMIfxOLtWObyA==", 2024-04-25T10:16:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:16:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:16:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-04-25T10:16:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC5Kipm/3VMIBAA+rhGCyA/7d+s8m7ZWTzX/Q==", 2024-04-25T10:16:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:16:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:16:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-04-25T10:16:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8KipmFNCCJxAAv0OSTpAG+xIHhLqYXAZazQ==", 2024-04-25T10:16:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T10:16:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:16:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-04-25T10:16:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmv65dLxAA6/QpPrqE9aT+CtZqkyutog==", 2024-04-25T10:16:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:16:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-04-25T10:16:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:16:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-04-25T10:16:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipm2NiONxAAd3X4zx1JnBXKHJvdZ/zMdQ==", 2024-04-25T10:16:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:16:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:16:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:16:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:16:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-04-25T10:16:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmZRONNBAAa+XTv5YIBJWeUvf5dfoWxw==", 2024-04-25T10:16:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:16:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:16:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:16:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:16:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-04-25T10:16:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCjKipmgvfiABAApLvTC9U6HzeAefCvX4tiSQ==", 2024-04-25T10:16:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T10:16:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T10:16:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T10:16:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T10:16:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-04-25T10:16:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQChKipmumzYNhAAlCRYMpSNaTo7bz8P6x0W9g==", 2024-04-25T10:16:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:16:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T10:16:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T10:16:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:16:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-04-25T10:16:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmhxlLDxAAegBrJWdMNz6ZC3oXT2Qo0A==", 2024-04-25T10:16:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:16:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T10:16:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:16:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-04-25T10:16:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCiKipmpl1cDBAAGqx2kIak9NoWpkcdHczcFA==", 2024-04-25T10:16:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T10:16:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T10:16:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T10:16:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T10:16:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T10:16:04.761 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:16:04.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.746+0000 7fe25670f640 1 -- 172.21.15.60:0/2881004541 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe230066f60 msgr2=0x7fe230069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.746+0000 7fe25670f640 1 --2- 172.21.15.60:0/2881004541 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe230066f60 0x7fe230069420 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe240009a50 tx=0x7fe240008040 comp rx=0 tx=0).stop 2024-04-25T10:16:04.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.746+0000 7fe25670f640 1 -- 172.21.15.60:0/2881004541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe25015bc90 msgr2=0x7fe2501470b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.746+0000 7fe25670f640 1 --2- 172.21.15.60:0/2881004541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe25015bc90 0x7fe2501470b0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fe23800c3c0 tx=0x7fe23800c880 comp rx=0 tx=0).stop 2024-04-25T10:16:04.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.746+0000 7fe25670f640 1 -- 172.21.15.60:0/2881004541 shutdown_connections 2024-04-25T10:16:04.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.746+0000 7fe25670f640 1 --2- 172.21.15.60:0/2881004541 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe230066f60 0x7fe230069420 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.746+0000 7fe25670f640 1 --2- 172.21.15.60:0/2881004541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2501497a0 0x7fe250154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.746+0000 7fe25670f640 1 --2- 172.21.15.60:0/2881004541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe25015bc90 0x7fe2501470b0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.746+0000 7fe25670f640 1 --2- 172.21.15.60:0/2881004541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2500ce7a0 0x7fe2501500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.746+0000 7fe25670f640 1 -- 172.21.15.60:0/2881004541 >> 172.21.15.60:0/2881004541 conn(0x7fe2500b12d0 msgr2=0x7fe2500b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:04.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.746+0000 7fe25670f640 1 -- 172.21.15.60:0/2881004541 shutdown_connections 2024-04-25T10:16:04.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.746+0000 7fe25670f640 1 -- 172.21.15.60:0/2881004541 wait complete. 2024-04-25T10:16:04.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:04.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.870+0000 7f8339d4c640 1 Processor -- start 2024-04-25T10:16:04.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.870+0000 7f8339d4c640 1 -- start start 2024-04-25T10:16:04.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f8339d4c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f83340ce7a0 0x7f83340ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f8339d4c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83340b33f0 con 0x7f83340b4660 2024-04-25T10:16:04.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f8339d4c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83340cf0c0 con 0x7f833415bc90 2024-04-25T10:16:04.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f8339d4c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83340cc370 con 0x7f83340ce7a0 2024-04-25T10:16:04.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f8333fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f83340b4660 0x7f83340b2ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46876/0 (socket says 172.21.15.60:46876) 2024-04-25T10:16:04.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f8333fff640 1 -- 172.21.15.60:0/1079394580 learned_addr learned my addr 172.21.15.60:0/1079394580 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:04.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83337fe640 1 --2- 172.21.15.60:0/1079394580 >> v2:172.21.15.100:3300/0 conn(0x7f83340ce7a0 0x7f83340ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83327fc640 1 -- 172.21.15.60:0/1079394580 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231967579 0 0) 0x7f83340b33f0 con 0x7f83340b4660 2024-04-25T10:16:04.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83327fc640 1 -- 172.21.15.60:0/1079394580 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f831c003530 con 0x7f83340b4660 2024-04-25T10:16:04.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83327fc640 1 -- 172.21.15.60:0/1079394580 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2812234508 0 0) 0x7f831c003530 con 0x7f83340b4660 2024-04-25T10:16:04.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83327fc640 1 -- 172.21.15.60:0/1079394580 >> v2:172.21.15.100:3300/0 conn(0x7f83340ce7a0 msgr2=0x7f83340ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:04.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83327fc640 1 --2- 172.21.15.60:0/1079394580 >> v2:172.21.15.100:3300/0 conn(0x7f83340ce7a0 0x7f83340ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83327fc640 1 -- 172.21.15.60:0/1079394580 >> v1:172.21.15.100:6789/0 conn(0x7f833415bc90 legacy=0x7f83340b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83327fc640 1 -- 172.21.15.60:0/1079394580 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8334071ca0 con 0x7f83340b4660 2024-04-25T10:16:04.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83327fc640 1 -- 172.21.15.60:0/1079394580 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f832c001a00 con 0x7f83340b4660 2024-04-25T10:16:04.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83327fc640 1 -- 172.21.15.60:0/1079394580 >> v1:172.21.15.60:6789/0 conn(0x7f83340b4660 legacy=0x7f83340b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83327fc640 1 --2- 172.21.15.60:0/1079394580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f831c0039c0 0x7f831c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83327fc640 1 --2- 172.21.15.60:0/1079394580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f831c004810 0x7f831c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83327fc640 1 --2- 172.21.15.60:0/1079394580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f831c0051d0 0x7f831c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83327fc640 1 -- 172.21.15.60:0/1079394580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c00c040 con 0x7f831c0051d0 2024-04-25T10:16:04.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83327fc640 1 -- 172.21.15.60:0/1079394580 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c00c1f0 con 0x7f831c004810 2024-04-25T10:16:04.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83327fc640 1 -- 172.21.15.60:0/1079394580 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831c00c3f0 con 0x7f831c0039c0 2024-04-25T10:16:04.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f8333fff640 1 --2- 172.21.15.60:0/1079394580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f831c0051d0 0x7f831c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f8332ffd640 1 --2- 172.21.15.60:0/1079394580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f831c004810 0x7f831c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f83337fe640 1 --2- 172.21.15.60:0/1079394580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f831c0039c0 0x7f831c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f8333fff640 1 -- 172.21.15.60:0/1079394580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f831c0039c0 msgr2=0x7f831c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:04.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f8333fff640 1 --2- 172.21.15.60:0/1079394580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f831c0039c0 0x7f831c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f8333fff640 1 -- 172.21.15.60:0/1079394580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f831c004810 msgr2=0x7f831c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:04.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f8333fff640 1 --2- 172.21.15.60:0/1079394580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f831c004810 0x7f831c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f8333fff640 1 -- 172.21.15.60:0/1079394580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c003530 con 0x7f831c0051d0 2024-04-25T10:16:04.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f8332ffd640 1 --2- 172.21.15.60:0/1079394580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f831c004810 0x7f831c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:04.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.874+0000 7f8333fff640 1 -- 172.21.15.60:0/1079394580 reap_dead start 2024-04-25T10:16:04.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8333fff640 1 --2- 172.21.15.60:0/1079394580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f831c0051d0 0x7f831c00b9d0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f832c00a8d0 tx=0x7f832c01d400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e7cb65941924aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f83327fc640 1 -- 172.21.15.60:0/1079394580 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c00a4a0 con 0x7f831c0051d0 2024-04-25T10:16:04.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f83327fc640 1 -- 172.21.15.60:0/1079394580 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c0ac640 con 0x7f831c0051d0 2024-04-25T10:16:04.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f83327fc640 1 -- 172.21.15.60:0/1079394580 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c0aca30 con 0x7f831c0051d0 2024-04-25T10:16:04.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 -- 172.21.15.60:0/1079394580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f831c0051d0 msgr2=0x7f831c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 --2- 172.21.15.60:0/1079394580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f831c0051d0 0x7f831c00b9d0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f832c00a8d0 tx=0x7f832c01d400 comp rx=0 tx=0).stop 2024-04-25T10:16:04.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 -- 172.21.15.60:0/1079394580 shutdown_connections 2024-04-25T10:16:04.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 --2- 172.21.15.60:0/1079394580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f831c0051d0 0x7f831c00b9d0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 -- 172.21.15.60:0/1079394580 >> 172.21.15.60:0/1079394580 conn(0x7f83340b12d0 msgr2=0x7f83340b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 -- 172.21.15.60:0/1079394580 shutdown_connections 2024-04-25T10:16:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 -- 172.21.15.60:0/1079394580 wait complete. 2024-04-25T10:16:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 Processor -- start 2024-04-25T10:16:04.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 -- start start 2024-04-25T10:16:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f831c0051d0 0x7f83341f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8334153bd0 0x7f83340b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83340b0a00 0x7f83340aca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8334071f80 con 0x7f8334153bd0 2024-04-25T10:16:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83340ccd20 con 0x7f83340b0a00 2024-04-25T10:16:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83340b4280 con 0x7f831c0051d0 2024-04-25T10:16:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8332ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8334153bd0 0x7f83340b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8332ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8334153bd0 0x7f83340b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39578/0 (socket says 172.21.15.60:39578) 2024-04-25T10:16:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8332ffd640 1 -- 172.21.15.60:0/230286095 learned_addr learned my addr 172.21.15.60:0/230286095 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f83337fe640 1 --2- 172.21.15.60:0/230286095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f831c0051d0 0x7f83341f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8333fff640 1 --2- 172.21.15.60:0/230286095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83340b0a00 0x7f83340aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8332ffd640 1 -- 172.21.15.60:0/230286095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f831c0051d0 msgr2=0x7f83341f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8332ffd640 1 --2- 172.21.15.60:0/230286095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f831c0051d0 0x7f83341f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8332ffd640 1 -- 172.21.15.60:0/230286095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83340b0a00 msgr2=0x7f83340aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8332ffd640 1 --2- 172.21.15.60:0/230286095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83340b0a00 0x7f83340aca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8332ffd640 1 -- 172.21.15.60:0/230286095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83340ad790 con 0x7f8334153bd0 2024-04-25T10:16:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f83337fe640 1 --2- 172.21.15.60:0/230286095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f831c0051d0 0x7f83341f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8333fff640 1 --2- 172.21.15.60:0/230286095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83340b0a00 0x7f83340aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8332ffd640 1 --2- 172.21.15.60:0/230286095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8334153bd0 0x7f83340b04c0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f832800e4e0 tx=0x7f832800e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f831bfff640 1 -- 172.21.15.60:0/230286095 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83280052e0 con 0x7f8334153bd0 2024-04-25T10:16:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f831bfff640 1 -- 172.21.15.60:0/230286095 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8328018070 con 0x7f8334153bd0 2024-04-25T10:16:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 -- 172.21.15.60:0/230286095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83341e6970 con 0x7f8334153bd0 2024-04-25T10:16:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f831bfff640 1 -- 172.21.15.60:0/230286095 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328005480 con 0x7f8334153bd0 2024-04-25T10:16:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.878+0000 7f8339d4c640 1 -- 172.21.15.60:0/230286095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83341e6f30 con 0x7f8334153bd0 2024-04-25T10:16:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.882+0000 7f8339d4c640 1 -- 172.21.15.60:0/230286095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8334144800 con 0x7f8334153bd0 2024-04-25T10:16:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.882+0000 7f831bfff640 1 -- 172.21.15.60:0/230286095 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8328003820 con 0x7f8334153bd0 2024-04-25T10:16:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.882+0000 7f831bfff640 1 --2- 172.21.15.60:0/230286095 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8320066f60 0x7f8320069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.882+0000 7f831bfff640 1 -- 172.21.15.60:0/230286095 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f83280849f0 con 0x7f8334153bd0 2024-04-25T10:16:04.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.882+0000 7f83337fe640 1 --2- 172.21.15.60:0/230286095 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8320066f60 0x7f8320069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.882+0000 7f83337fe640 1 --2- 172.21.15.60:0/230286095 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8320066f60 0x7f8320069420 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8324009a70 tx=0x7f8324008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:04.886+0000 7f831bfff640 1 -- 172.21.15.60:0/230286095 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8334144800 con 0x7f8334153bd0 2024-04-25T10:16:05.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.050+0000 7f8339d4c640 1 -- 172.21.15.60:0/230286095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8334074770 con 0x7f8334153bd0 2024-04-25T10:16:05.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.050+0000 7f831bfff640 1 -- 172.21.15.60:0/230286095 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+12905 (secure 0 0 0) 0x7f8328055c30 con 0x7f8334153bd0 2024-04-25T10:16:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:05.056 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":106,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:16:04.362428+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:05.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.054+0000 7f8339d4c640 1 -- 172.21.15.60:0/230286095 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8320066f60 msgr2=0x7f8320069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.054+0000 7f8339d4c640 1 --2- 172.21.15.60:0/230286095 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8320066f60 0x7f8320069420 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8324009a70 tx=0x7f8324008040 comp rx=0 tx=0).stop 2024-04-25T10:16:05.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.054+0000 7f8339d4c640 1 -- 172.21.15.60:0/230286095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8334153bd0 msgr2=0x7f83340b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.054+0000 7f8339d4c640 1 --2- 172.21.15.60:0/230286095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8334153bd0 0x7f83340b04c0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f832800e4e0 tx=0x7f832800e9a0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.054+0000 7f8339d4c640 1 -- 172.21.15.60:0/230286095 shutdown_connections 2024-04-25T10:16:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.054+0000 7f8339d4c640 1 --2- 172.21.15.60:0/230286095 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8320066f60 0x7f8320069420 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.054+0000 7f8339d4c640 1 --2- 172.21.15.60:0/230286095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83340b0a00 0x7f83340aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.054+0000 7f8339d4c640 1 --2- 172.21.15.60:0/230286095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8334153bd0 0x7f83340b04c0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.054+0000 7f8339d4c640 1 --2- 172.21.15.60:0/230286095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f831c0051d0 0x7f83341f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.054+0000 7f8339d4c640 1 -- 172.21.15.60:0/230286095 >> 172.21.15.60:0/230286095 conn(0x7f83340b12d0 msgr2=0x7f83340cbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.054+0000 7f8339d4c640 1 -- 172.21.15.60:0/230286095 shutdown_connections 2024-04-25T10:16:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.054+0000 7f8339d4c640 1 -- 172.21.15.60:0/230286095 wait complete. 2024-04-25T10:16:05.076 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:05.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:05.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f3218756640 1 Processor -- start 2024-04-25T10:16:05.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f3218756640 1 -- start start 2024-04-25T10:16:05.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f3218756640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32100cbcf0 0x7f3210160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f3218756640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3210144a40 con 0x7f321015fbf0 2024-04-25T10:16:05.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f3218756640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32100cc9f0 con 0x7f3210071c30 2024-04-25T10:16:05.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f3218756640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32100b2dc0 con 0x7f32100cbcf0 2024-04-25T10:16:05.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f3216cf3640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f321015fbf0 0x7f32100ceec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46890/0 (socket says 172.21.15.60:46890) 2024-04-25T10:16:05.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f3216cf3640 1 -- 172.21.15.60:0/1966282140 learned_addr learned my addr 172.21.15.60:0/1966282140 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:05.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32164f2640 1 --2- 172.21.15.60:0/1966282140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32100cbcf0 0x7f3210160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32154f0640 1 -- 172.21.15.60:0/1966282140 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405244182 0 0) 0x7f3210144a40 con 0x7f321015fbf0 2024-04-25T10:16:05.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32154f0640 1 -- 172.21.15.60:0/1966282140 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31fc003530 con 0x7f321015fbf0 2024-04-25T10:16:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32154f0640 1 -- 172.21.15.60:0/1966282140 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2132982740 0 0) 0x7f31fc003530 con 0x7f321015fbf0 2024-04-25T10:16:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32154f0640 1 -- 172.21.15.60:0/1966282140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32100cbcf0 msgr2=0x7f3210160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32154f0640 1 --2- 172.21.15.60:0/1966282140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32100cbcf0 0x7f3210160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32154f0640 1 -- 172.21.15.60:0/1966282140 >> v1:172.21.15.100:6789/0 conn(0x7f3210071c30 legacy=0x7f321015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32154f0640 1 -- 172.21.15.60:0/1966282140 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32100cccd0 con 0x7f321015fbf0 2024-04-25T10:16:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32154f0640 1 -- 172.21.15.60:0/1966282140 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f3208001a00 con 0x7f321015fbf0 2024-04-25T10:16:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32154f0640 1 -- 172.21.15.60:0/1966282140 >> v1:172.21.15.60:6789/0 conn(0x7f321015fbf0 legacy=0x7f32100ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32154f0640 1 --2- 172.21.15.60:0/1966282140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31fc0039f0 0x7f31fc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32154f0640 1 --2- 172.21.15.60:0/1966282140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31fc004810 0x7f31fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32154f0640 1 --2- 172.21.15.60:0/1966282140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32100cbcf0 0x7f31fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32154f0640 1 -- 172.21.15.60:0/1966282140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc00ba80 con 0x7f31fc004810 2024-04-25T10:16:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32154f0640 1 -- 172.21.15.60:0/1966282140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc00bc00 con 0x7f31fc0039f0 2024-04-25T10:16:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32154f0640 1 -- 172.21.15.60:0/1966282140 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc00be00 con 0x7f32100cbcf0 2024-04-25T10:16:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f3215cf1640 1 --2- 172.21.15.60:0/1966282140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31fc004810 0x7f31fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f32164f2640 1 --2- 172.21.15.60:0/1966282140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31fc0039f0 0x7f31fc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f3216cf3640 1 --2- 172.21.15.60:0/1966282140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32100cbcf0 0x7f31fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f3215cf1640 1 -- 172.21.15.60:0/1966282140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32100cbcf0 msgr2=0x7f31fc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f3215cf1640 1 --2- 172.21.15.60:0/1966282140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32100cbcf0 0x7f31fc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f3215cf1640 1 -- 172.21.15.60:0/1966282140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31fc0039f0 msgr2=0x7f31fc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f3215cf1640 1 --2- 172.21.15.60:0/1966282140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31fc0039f0 0x7f31fc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f3215cf1640 1 -- 172.21.15.60:0/1966282140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc003530 con 0x7f31fc004810 2024-04-25T10:16:05.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.194+0000 7f3216cf3640 1 --2- 172.21.15.60:0/1966282140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32100cbcf0 0x7f31fc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:05.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.198+0000 7f3215cf1640 1 --2- 172.21.15.60:0/1966282140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31fc004810 0x7f31fc004c90 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f320c001750 tx=0x7f320c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54715b366a729e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:05.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.198+0000 7f32154f0640 1 -- 172.21.15.60:0/1966282140 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c0013a0 con 0x7f31fc004810 2024-04-25T10:16:05.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.198+0000 7f32154f0640 1 -- 172.21.15.60:0/1966282140 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320c0052f0 con 0x7f31fc004810 2024-04-25T10:16:05.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.198+0000 7f32154f0640 1 -- 172.21.15.60:0/1966282140 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32080020d0 con 0x7f31fc004810 2024-04-25T10:16:05.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.198+0000 7f3218756640 1 -- 172.21.15.60:0/1966282140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31fc004810 msgr2=0x7f31fc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.198+0000 7f3218756640 1 --2- 172.21.15.60:0/1966282140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31fc004810 0x7f31fc004c90 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f320c001750 tx=0x7f320c01c3c0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.198+0000 7f3216cf3640 1 -- 172.21.15.60:0/1966282140 reap_dead start 2024-04-25T10:16:05.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.198+0000 7f3218756640 1 -- 172.21.15.60:0/1966282140 shutdown_connections 2024-04-25T10:16:05.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.198+0000 7f3218756640 1 -- 172.21.15.60:0/1966282140 >> 172.21.15.60:0/1966282140 conn(0x7f32100670b0 msgr2=0x7f321006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:05.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.198+0000 7f3218756640 1 -- 172.21.15.60:0/1966282140 shutdown_connections 2024-04-25T10:16:05.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.198+0000 7f3218756640 1 -- 172.21.15.60:0/1966282140 wait complete. 2024-04-25T10:16:05.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.198+0000 7f3218756640 1 Processor -- start 2024-04-25T10:16:05.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.198+0000 7f3218756640 1 -- start start 2024-04-25T10:16:05.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3218756640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f321013f680 0x7f32100ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3218756640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f321006eba0 0x7f321015e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3218756640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f321006f000 0x7f321015e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3218756640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100cc0d0 con 0x7f321013f680 2024-04-25T10:16:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3218756640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100ce9a0 con 0x7f321006eba0 2024-04-25T10:16:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3218756640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32100720b0 con 0x7f321006f000 2024-04-25T10:16:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f32164f2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f321013f680 0x7f32100ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f32164f2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f321013f680 0x7f32100ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39594/0 (socket says 172.21.15.60:39594) 2024-04-25T10:16:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f32164f2640 1 -- 172.21.15.60:0/3456119140 learned_addr learned my addr 172.21.15.60:0/3456119140 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3215cf1640 1 --2- 172.21.15.60:0/3456119140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f321006eba0 0x7f321015e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3216cf3640 1 --2- 172.21.15.60:0/3456119140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f321006f000 0x7f321015e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f32164f2640 1 -- 172.21.15.60:0/3456119140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f321006f000 msgr2=0x7f321015e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f32164f2640 1 --2- 172.21.15.60:0/3456119140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f321006f000 0x7f321015e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f32164f2640 1 -- 172.21.15.60:0/3456119140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f321006eba0 msgr2=0x7f321015e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f32164f2640 1 --2- 172.21.15.60:0/3456119140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f321006eba0 0x7f321015e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f32164f2640 1 -- 172.21.15.60:0/3456119140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321006db00 con 0x7f321013f680 2024-04-25T10:16:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3216cf3640 1 --2- 172.21.15.60:0/3456119140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f321006f000 0x7f321015e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f32164f2640 1 --2- 172.21.15.60:0/3456119140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f321013f680 0x7f32100ac8a0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f32000115e0 tx=0x7f3200011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3206ffd640 1 -- 172.21.15.60:0/3456119140 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320000f8f0 con 0x7f321013f680 2024-04-25T10:16:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3206ffd640 1 -- 172.21.15.60:0/3456119140 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320000fa90 con 0x7f321013f680 2024-04-25T10:16:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3218756640 1 -- 172.21.15.60:0/3456119140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321006dde0 con 0x7f321013f680 2024-04-25T10:16:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3206ffd640 1 -- 172.21.15.60:0/3456119140 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320000fe40 con 0x7f321013f680 2024-04-25T10:16:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3218756640 1 -- 172.21.15.60:0/3456119140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321006e3a0 con 0x7f321013f680 2024-04-25T10:16:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3218756640 1 -- 172.21.15.60:0/3456119140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f321006bba0 con 0x7f321013f680 2024-04-25T10:16:05.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3206ffd640 1 -- 172.21.15.60:0/3456119140 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3200015460 con 0x7f321013f680 2024-04-25T10:16:05.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3206ffd640 1 --2- 172.21.15.60:0/3456119140 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f31f4066f60 0x7f31f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3206ffd640 1 -- 172.21.15.60:0/3456119140 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3200085730 con 0x7f321013f680 2024-04-25T10:16:05.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.202+0000 7f3215cf1640 1 --2- 172.21.15.60:0/3456119140 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f31f4066f60 0x7f31f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.206+0000 7f3215cf1640 1 --2- 172.21.15.60:0/3456119140 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f31f4066f60 0x7f31f4069420 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0057f0 tx=0x7f320c0011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:05.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.206+0000 7f3206ffd640 1 -- 172.21.15.60:0/3456119140 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3200019030 con 0x7f321013f680 2024-04-25T10:16:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.370+0000 7f3218756640 1 -- 172.21.15.60:0/3456119140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32100b2970 con 0x7f321013f680 2024-04-25T10:16:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.370+0000 7f3206ffd640 1 -- 172.21.15.60:0/3456119140 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3200056a00 con 0x7f321013f680 2024-04-25T10:16:05.376 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:16:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.374+0000 7f3218756640 1 -- 172.21.15.60:0/3456119140 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f31f4066f60 msgr2=0x7f31f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.374+0000 7f3218756640 1 --2- 172.21.15.60:0/3456119140 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f31f4066f60 0x7f31f4069420 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0057f0 tx=0x7f320c0011a0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.374+0000 7f3218756640 1 -- 172.21.15.60:0/3456119140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f321013f680 msgr2=0x7f32100ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.374+0000 7f3218756640 1 --2- 172.21.15.60:0/3456119140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f321013f680 0x7f32100ac8a0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f32000115e0 tx=0x7f3200011aa0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.374+0000 7f3218756640 1 -- 172.21.15.60:0/3456119140 shutdown_connections 2024-04-25T10:16:05.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.374+0000 7f3218756640 1 --2- 172.21.15.60:0/3456119140 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f31f4066f60 0x7f31f4069420 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.374+0000 7f3218756640 1 --2- 172.21.15.60:0/3456119140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f321006f000 0x7f321015e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.374+0000 7f3218756640 1 --2- 172.21.15.60:0/3456119140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f321006eba0 0x7f321015e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.374+0000 7f3218756640 1 --2- 172.21.15.60:0/3456119140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f321013f680 0x7f32100ac8a0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.374+0000 7f3218756640 1 -- 172.21.15.60:0/3456119140 >> 172.21.15.60:0/3456119140 conn(0x7f32100670b0 msgr2=0x7f32100cf940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:05.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.374+0000 7f3218756640 1 -- 172.21.15.60:0/3456119140 shutdown_connections 2024-04-25T10:16:05.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.374+0000 7f3218756640 1 -- 172.21.15.60:0/3456119140 wait complete. 2024-04-25T10:16:05.395 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:16:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b0be57640 1 Processor -- start 2024-04-25T10:16:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b0be57640 1 -- start start 2024-04-25T10:16:05.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b0be57640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b0415a7a0 0x7f7b040cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b0be57640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7b0406ec30 0x7f7b04071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b0be57640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7b040b37e0 0x7f7b040b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b0be57640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b040cdaa0 con 0x7f7b0406ec30 2024-04-25T10:16:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b0be57640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0406d2f0 con 0x7f7b040b37e0 2024-04-25T10:16:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b0be57640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0406d8b0 con 0x7f7b0415a7a0 2024-04-25T10:16:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b093f2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7b0406ec30 0x7f7b04071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b093f2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7b0406ec30 0x7f7b04071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39604/0 (socket says 172.21.15.60:39604) 2024-04-25T10:16:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b093f2640 1 -- 172.21.15.60:0/3147324497 learned_addr learned my addr 172.21.15.60:0/3147324497 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b0a3f4640 1 --2- 172.21.15.60:0/3147324497 >> v2:172.21.15.100:3300/0 conn(0x7f7b040b37e0 0x7f7b040b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b093f2640 1 -- 172.21.15.60:0/3147324497 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b0415a7a0 msgr2=0x7f7b040cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b09bf3640 1 --2- 172.21.15.60:0/3147324497 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b0415a7a0 0x7f7b040cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b093f2640 1 --2- 172.21.15.60:0/3147324497 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b0415a7a0 0x7f7b040cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b093f2640 1 -- 172.21.15.60:0/3147324497 >> v2:172.21.15.100:3300/0 conn(0x7f7b040b37e0 msgr2=0x7f7b040b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b093f2640 1 --2- 172.21.15.60:0/3147324497 >> v2:172.21.15.100:3300/0 conn(0x7f7b040b37e0 0x7f7b040b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b093f2640 1 -- 172.21.15.60:0/3147324497 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b040721e0 con 0x7f7b0406ec30 2024-04-25T10:16:05.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b0a3f4640 1 --2- 172.21.15.60:0/3147324497 >> v2:172.21.15.100:3300/0 conn(0x7f7b040b37e0 0x7f7b040b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:05.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.510+0000 7f7b09bf3640 1 --2- 172.21.15.60:0/3147324497 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b0415a7a0 0x7f7b040cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.514+0000 7f7b093f2640 1 --2- 172.21.15.60:0/3147324497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0406ec30 0x7f7b04071ca0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f7b040b2660 tx=0x7f7af401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5e3e62d72d70a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.514+0000 7f7b08bf1640 1 -- 172.21.15.60:0/3147324497 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40ab5f0 con 0x7f7b0406ec30 2024-04-25T10:16:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.514+0000 7f7b08bf1640 1 -- 172.21.15.60:0/3147324497 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af40b0070 con 0x7f7b0406ec30 2024-04-25T10:16:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.514+0000 7f7b08bf1640 1 -- 172.21.15.60:0/3147324497 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40ab9b0 con 0x7f7b0406ec30 2024-04-25T10:16:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.514+0000 7f7b0be57640 1 -- 172.21.15.60:0/3147324497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0406ec30 msgr2=0x7f7b04071ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.514+0000 7f7b0be57640 1 --2- 172.21.15.60:0/3147324497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0406ec30 0x7f7b04071ca0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f7b040b2660 tx=0x7f7af401c520 comp rx=0 tx=0).stop 2024-04-25T10:16:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.514+0000 7f7b0be57640 1 -- 172.21.15.60:0/3147324497 shutdown_connections 2024-04-25T10:16:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.514+0000 7f7b0be57640 1 --2- 172.21.15.60:0/3147324497 >> v2:172.21.15.100:3300/0 conn(0x7f7b040b37e0 0x7f7b040b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.514+0000 7f7b0be57640 1 --2- 172.21.15.60:0/3147324497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0406ec30 0x7f7b04071ca0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.514+0000 7f7b0be57640 1 --2- 172.21.15.60:0/3147324497 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b0415a7a0 0x7f7b040cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.514+0000 7f7b0be57640 1 -- 172.21.15.60:0/3147324497 >> 172.21.15.60:0/3147324497 conn(0x7f7b040b12d0 msgr2=0x7f7b040b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.514+0000 7f7b0be57640 1 -- 172.21.15.60:0/3147324497 shutdown_connections 2024-04-25T10:16:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.514+0000 7f7b0be57640 1 -- 172.21.15.60:0/3147324497 wait complete. 2024-04-25T10:16:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0be57640 1 Processor -- start 2024-04-25T10:16:05.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0be57640 1 -- start start 2024-04-25T10:16:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0be57640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b0406ec30 0x7f7b041642f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0be57640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b040b37e0 0x7f7b0414f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0be57640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0415a7a0 0x7f7b04160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0be57640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0415ad00 con 0x7f7b0415a7a0 2024-04-25T10:16:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0be57640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0415ab80 con 0x7f7b040b37e0 2024-04-25T10:16:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0be57640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0406e530 con 0x7f7b0406ec30 2024-04-25T10:16:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0a3f4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0415a7a0 0x7f7b04160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0a3f4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0415a7a0 0x7f7b04160be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39612/0 (socket says 172.21.15.60:39612) 2024-04-25T10:16:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0a3f4640 1 -- 172.21.15.60:0/4128927607 learned_addr learned my addr 172.21.15.60:0/4128927607 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b093f2640 1 --2- 172.21.15.60:0/4128927607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b040b37e0 0x7f7b0414f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b09bf3640 1 --2- 172.21.15.60:0/4128927607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b0406ec30 0x7f7b041642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0a3f4640 1 -- 172.21.15.60:0/4128927607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b0406ec30 msgr2=0x7f7b041642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0a3f4640 1 --2- 172.21.15.60:0/4128927607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b0406ec30 0x7f7b041642f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0a3f4640 1 -- 172.21.15.60:0/4128927607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b040b37e0 msgr2=0x7f7b0414f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0a3f4640 1 --2- 172.21.15.60:0/4128927607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b040b37e0 0x7f7b0414f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0a3f4640 1 -- 172.21.15.60:0/4128927607 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b04158af0 con 0x7f7b0415a7a0 2024-04-25T10:16:05.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0a3f4640 1 --2- 172.21.15.60:0/4128927607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0415a7a0 0x7f7b04160be0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00cef0 tx=0x7f7afc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:05.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7afa7fc640 1 -- 172.21.15.60:0/4128927607 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc01a070 con 0x7f7b0415a7a0 2024-04-25T10:16:05.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7afa7fc640 1 -- 172.21.15.60:0/4128927607 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc00d860 con 0x7f7b0415a7a0 2024-04-25T10:16:05.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7afa7fc640 1 -- 172.21.15.60:0/4128927607 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00dc50 con 0x7f7b0415a7a0 2024-04-25T10:16:05.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0be57640 1 -- 172.21.15.60:0/4128927607 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b04163810 con 0x7f7b0415a7a0 2024-04-25T10:16:05.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0be57640 1 -- 172.21.15.60:0/4128927607 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b04163dd0 con 0x7f7b0415a7a0 2024-04-25T10:16:05.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7b0be57640 1 -- 172.21.15.60:0/4128927607 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b04074770 con 0x7f7b0415a7a0 2024-04-25T10:16:05.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.518+0000 7f7afa7fc640 1 -- 172.21.15.60:0/4128927607 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7afc01d070 con 0x7f7b0415a7a0 2024-04-25T10:16:05.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.522+0000 7f7afa7fc640 1 --2- 172.21.15.60:0/4128927607 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ae8066f60 0x7f7ae8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.522+0000 7f7afa7fc640 1 -- 172.21.15.60:0/4128927607 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7afc0855a0 con 0x7f7b0415a7a0 2024-04-25T10:16:05.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.522+0000 7f7b09bf3640 1 --2- 172.21.15.60:0/4128927607 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ae8066f60 0x7f7ae8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.522+0000 7f7b09bf3640 1 --2- 172.21.15.60:0/4128927607 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ae8066f60 0x7f7ae8069420 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7b00006fd0 tx=0x7f7b00008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:05.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.526+0000 7f7afa7fc640 1 -- 172.21.15.60:0/4128927607 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7afc0569b0 con 0x7f7b0415a7a0 2024-04-25T10:16:05.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.678+0000 7f7b0be57640 1 -- 172.21.15.60:0/4128927607 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7b04153c50 con 0x7f7b0415a7a0 2024-04-25T10:16:05.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.682+0000 7f7afa7fc640 1 -- 172.21.15.60:0/4128927607 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v106) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7afc0562b0 con 0x7f7b0415a7a0 2024-04-25T10:16:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:05.686 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T10:16:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T10:16:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T10:16:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T10:16:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:05.686 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T10:16:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.682+0000 7f7b0be57640 1 -- 172.21.15.60:0/4128927607 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ae8066f60 msgr2=0x7f7ae8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.682+0000 7f7b0be57640 1 --2- 172.21.15.60:0/4128927607 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ae8066f60 0x7f7ae8069420 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7b00006fd0 tx=0x7f7b00008040 comp rx=0 tx=0).stop 2024-04-25T10:16:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.682+0000 7f7b0be57640 1 -- 172.21.15.60:0/4128927607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0415a7a0 msgr2=0x7f7b04160be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.682+0000 7f7b0be57640 1 --2- 172.21.15.60:0/4128927607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0415a7a0 0x7f7b04160be0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00cef0 tx=0x7f7afc00a710 comp rx=0 tx=0).stop 2024-04-25T10:16:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.682+0000 7f7b0be57640 1 -- 172.21.15.60:0/4128927607 shutdown_connections 2024-04-25T10:16:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.682+0000 7f7b0be57640 1 --2- 172.21.15.60:0/4128927607 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ae8066f60 0x7f7ae8069420 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.682+0000 7f7b0be57640 1 --2- 172.21.15.60:0/4128927607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0415a7a0 0x7f7b04160be0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.682+0000 7f7b0be57640 1 --2- 172.21.15.60:0/4128927607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b040b37e0 0x7f7b0414f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.682+0000 7f7b0be57640 1 --2- 172.21.15.60:0/4128927607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b0406ec30 0x7f7b041642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.682+0000 7f7b0be57640 1 -- 172.21.15.60:0/4128927607 >> 172.21.15.60:0/4128927607 conn(0x7f7b040b12d0 msgr2=0x7f7b040b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:05.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.682+0000 7f7b0be57640 1 -- 172.21.15.60:0/4128927607 shutdown_connections 2024-04-25T10:16:05.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.682+0000 7f7b0be57640 1 -- 172.21.15.60:0/4128927607 wait complete. 2024-04-25T10:16:05.703 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:16:05.704 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:16:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1eeb30c640 1 Processor -- start 2024-04-25T10:16:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1eeb30c640 1 -- start start 2024-04-25T10:16:05.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1eeb30c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1ee4144e40 0x7f1ee40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1eeb30c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1ee4071980 0x7f1ee40b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1eeb30c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee40cf810 con 0x7f1ee40cf0a0 2024-04-25T10:16:05.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1eeb30c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee40b3060 con 0x7f1ee4144e40 2024-04-25T10:16:05.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1eeb30c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee40cfaf0 con 0x7f1ee4071980 2024-04-25T10:16:05.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1ee90a8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1ee4144e40 0x7f1ee40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1ee98a9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1ee4071980 0x7f1ee40b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1ee90a8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1ee4144e40 0x7f1ee40ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:48484/0 (socket says 172.21.15.60:48484) 2024-04-25T10:16:05.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1ee98a9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1ee4071980 0x7f1ee40b34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:52812/0 (socket says 172.21.15.60:52812) 2024-04-25T10:16:05.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1ee90a8640 1 -- 172.21.15.60:0/1068004527 learned_addr learned my addr 172.21.15.60:0/1068004527 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:05.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1ee88a7640 1 --1- 172.21.15.60:0/1068004527 >> v1:172.21.15.100:6789/0 conn(0x7f1ee40cf0a0 0x7f1ee4071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:49758/0 (socket says 172.21.15.60:49758) 2024-04-25T10:16:05.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1ee98a9640 1 -- 172.21.15.60:0/1068004527 >> v2:172.21.15.100:3300/0 conn(0x7f1ee4144e40 msgr2=0x7f1ee40ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:05.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1ee98a9640 1 --2- 172.21.15.60:0/1068004527 >> v2:172.21.15.100:3300/0 conn(0x7f1ee4144e40 0x7f1ee40ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1ee98a9640 1 -- 172.21.15.60:0/1068004527 >> v1:172.21.15.100:6789/0 conn(0x7f1ee40cf0a0 legacy=0x7f1ee4071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.818+0000 7f1ee98a9640 1 -- 172.21.15.60:0/1068004527 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee40cfdd0 con 0x7f1ee4071980 2024-04-25T10:16:05.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.822+0000 7f1ee98a9640 1 --2- 172.21.15.60:0/1068004527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1ee4071980 0x7f1ee40b34f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009d90 tx=0x7f1edc01c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f721c68f3862a39d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.822+0000 7f1edbfff640 1 -- 172.21.15.60:0/1068004527 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0b2070 con 0x7f1ee4071980 2024-04-25T10:16:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1edbfff640 1 -- 172.21.15.60:0/1068004527 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee4145430 con 0x7f1ee4071980 2024-04-25T10:16:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1edbfff640 1 -- 172.21.15.60:0/1068004527 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0a5c10 con 0x7f1ee4071980 2024-04-25T10:16:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1eeb30c640 1 -- 172.21.15.60:0/1068004527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1ee4071980 msgr2=0x7f1ee40b34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1eeb30c640 1 --2- 172.21.15.60:0/1068004527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1ee4071980 0x7f1ee40b34f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009d90 tx=0x7f1edc01c910 comp rx=0 tx=0).stop 2024-04-25T10:16:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1eeb30c640 1 -- 172.21.15.60:0/1068004527 shutdown_connections 2024-04-25T10:16:05.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1eeb30c640 1 --2- 172.21.15.60:0/1068004527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1ee4071980 0x7f1ee40b34f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1eeb30c640 1 --2- 172.21.15.60:0/1068004527 >> v2:172.21.15.100:3300/0 conn(0x7f1ee4144e40 0x7f1ee40ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1eeb30c640 1 -- 172.21.15.60:0/1068004527 >> 172.21.15.60:0/1068004527 conn(0x7f1ee4066fa0 msgr2=0x7f1ee406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:05.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1eeb30c640 1 -- 172.21.15.60:0/1068004527 shutdown_connections 2024-04-25T10:16:05.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1eeb30c640 1 -- 172.21.15.60:0/1068004527 wait complete. 2024-04-25T10:16:05.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1eeb30c640 1 Processor -- start 2024-04-25T10:16:05.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1eeb30c640 1 -- start start 2024-04-25T10:16:05.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1eeb30c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ee4071980 0x7f1ee40a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1eeb30c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1ee40cf0a0 0x7f1ee40aae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1eeb30c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee4144e40 0x7f1ee40cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1eeb30c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee40b4610 con 0x7f1ee4144e40 2024-04-25T10:16:05.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1eeb30c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee40b4490 con 0x7f1ee4071980 2024-04-25T10:16:05.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1eeb30c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee40b4790 con 0x7f1ee40cf0a0 2024-04-25T10:16:05.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1ee98a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee4144e40 0x7f1ee40cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1ee98a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee4144e40 0x7f1ee40cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39618/0 (socket says 172.21.15.60:39618) 2024-04-25T10:16:05.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1ee98a9640 1 -- 172.21.15.60:0/372665384 learned_addr learned my addr 172.21.15.60:0/372665384 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:05.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1ee90a8640 1 --2- 172.21.15.60:0/372665384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ee4071980 0x7f1ee40a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.826+0000 7f1ee88a7640 1 --2- 172.21.15.60:0/372665384 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1ee40cf0a0 0x7f1ee40aae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1ee98a9640 1 -- 172.21.15.60:0/372665384 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1ee40cf0a0 msgr2=0x7f1ee40aae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1ee98a9640 1 --2- 172.21.15.60:0/372665384 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1ee40cf0a0 0x7f1ee40aae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1ee98a9640 1 -- 172.21.15.60:0/372665384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ee4071980 msgr2=0x7f1ee40a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1ee98a9640 1 --2- 172.21.15.60:0/372665384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ee4071980 0x7f1ee40a9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1ee98a9640 1 -- 172.21.15.60:0/372665384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee40a85c0 con 0x7f1ee4144e40 2024-04-25T10:16:05.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1ee90a8640 1 --2- 172.21.15.60:0/372665384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ee4071980 0x7f1ee40a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:05.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1ee98a9640 1 --2- 172.21.15.60:0/372665384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee4144e40 0x7f1ee40cbdf0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0b0040 tx=0x7f1edc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:05.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1ed9ffb640 1 -- 172.21.15.60:0/372665384 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0b2050 con 0x7f1ee4144e40 2024-04-25T10:16:05.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1ed9ffb640 1 -- 172.21.15.60:0/372665384 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc0be070 con 0x7f1ee4144e40 2024-04-25T10:16:05.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1eeb30c640 1 -- 172.21.15.60:0/372665384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee40a88a0 con 0x7f1ee4144e40 2024-04-25T10:16:05.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1ed9ffb640 1 -- 172.21.15.60:0/372665384 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc009450 con 0x7f1ee4144e40 2024-04-25T10:16:05.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1eeb30c640 1 -- 172.21.15.60:0/372665384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee40a8e60 con 0x7f1ee4144e40 2024-04-25T10:16:05.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1eeb30c640 1 -- 172.21.15.60:0/372665384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee4074770 con 0x7f1ee4144e40 2024-04-25T10:16:05.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1ed9ffb640 1 -- 172.21.15.60:0/372665384 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1edc0ada90 con 0x7f1ee4144e40 2024-04-25T10:16:05.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1ed9ffb640 1 --2- 172.21.15.60:0/372665384 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1ec8066f60 0x7f1ec8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1ed9ffb640 1 -- 172.21.15.60:0/372665384 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1edc120920 con 0x7f1ee4144e40 2024-04-25T10:16:05.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1ee90a8640 1 --2- 172.21.15.60:0/372665384 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1ec8066f60 0x7f1ec8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.830+0000 7f1ee90a8640 1 --2- 172.21.15.60:0/372665384 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1ec8066f60 0x7f1ec8069420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009ba0 tx=0x7f1ee0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:05.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.834+0000 7f1ed9ffb640 1 -- 172.21.15.60:0/372665384 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1edc016560 con 0x7f1ee4144e40 2024-04-25T10:16:05.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:05.990+0000 7f1eeb30c640 1 -- 172.21.15.60:0/372665384 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1ee40b29e0 con 0x7f1ee4144e40 2024-04-25T10:16:06.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.366+0000 7f1ed9ffb640 1 -- 172.21.15.60:0/372665384 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v107) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1edc0f1cf0 con 0x7f1ee4144e40 2024-04-25T10:16:06.372 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T10:16:06.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.370+0000 7f1eeb30c640 1 -- 172.21.15.60:0/372665384 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1ec8066f60 msgr2=0x7f1ec8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.370+0000 7f1eeb30c640 1 --2- 172.21.15.60:0/372665384 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1ec8066f60 0x7f1ec8069420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009ba0 tx=0x7f1ee0008040 comp rx=0 tx=0).stop 2024-04-25T10:16:06.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.370+0000 7f1eeb30c640 1 -- 172.21.15.60:0/372665384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee4144e40 msgr2=0x7f1ee40cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.370+0000 7f1eeb30c640 1 --2- 172.21.15.60:0/372665384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee4144e40 0x7f1ee40cbdf0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0b0040 tx=0x7f1edc019710 comp rx=0 tx=0).stop 2024-04-25T10:16:06.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.370+0000 7f1eeb30c640 1 -- 172.21.15.60:0/372665384 shutdown_connections 2024-04-25T10:16:06.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.370+0000 7f1eeb30c640 1 --2- 172.21.15.60:0/372665384 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1ec8066f60 0x7f1ec8069420 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.370+0000 7f1eeb30c640 1 --2- 172.21.15.60:0/372665384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee4144e40 0x7f1ee40cbdf0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.370+0000 7f1eeb30c640 1 --2- 172.21.15.60:0/372665384 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1ee40cf0a0 0x7f1ee40aae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.370+0000 7f1eeb30c640 1 --2- 172.21.15.60:0/372665384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ee4071980 0x7f1ee40a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.370+0000 7f1eeb30c640 1 -- 172.21.15.60:0/372665384 >> 172.21.15.60:0/372665384 conn(0x7f1ee4066fa0 msgr2=0x7f1ee413d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:06.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.370+0000 7f1eeb30c640 1 -- 172.21.15.60:0/372665384 shutdown_connections 2024-04-25T10:16:06.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.370+0000 7f1eeb30c640 1 -- 172.21.15.60:0/372665384 wait complete. 2024-04-25T10:16:06.402 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:16:06.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.530+0000 7f427339f640 1 Processor -- start 2024-04-25T10:16:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.530+0000 7f427339f640 1 -- start start 2024-04-25T10:16:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.530+0000 7f427339f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f426c15ec40 0x7f426c15fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.530+0000 7f427339f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426c0ce950 0x7f426c0cce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.530+0000 7f427339f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f426c0b0c60 con 0x7f426c0cfe10 2024-04-25T10:16:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.530+0000 7f427339f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426c160400 con 0x7f426c15ec40 2024-04-25T10:16:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.530+0000 7f427339f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0d02a0 con 0x7f426c0ce950 2024-04-25T10:16:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.530+0000 7f427093a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f426c0cfe10 0x7f426c0ce240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46898/0 (socket says 172.21.15.60:46898) 2024-04-25T10:16:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.530+0000 7f427093a640 1 -- 172.21.15.60:0/3896332460 learned_addr learned my addr 172.21.15.60:0/3896332460 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f427113b640 1 --2- 172.21.15.60:0/3896332460 >> v2:172.21.15.100:3300/0 conn(0x7f426c15ec40 0x7f426c15fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f427193c640 1 --2- 172.21.15.60:0/3896332460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426c0ce950 0x7f426c0cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 -- 172.21.15.60:0/3896332460 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231341237 0 0) 0x7f426c0b0c60 con 0x7f426c0cfe10 2024-04-25T10:16:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 -- 172.21.15.60:0/3896332460 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4258003530 con 0x7f426c0cfe10 2024-04-25T10:16:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 -- 172.21.15.60:0/3896332460 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2441049691 0 0) 0x7f4258003530 con 0x7f426c0cfe10 2024-04-25T10:16:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 -- 172.21.15.60:0/3896332460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426c0ce950 msgr2=0x7f426c0cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:06.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 --2- 172.21.15.60:0/3896332460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426c0ce950 0x7f426c0cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 -- 172.21.15.60:0/3896332460 >> v2:172.21.15.100:3300/0 conn(0x7f426c15ec40 msgr2=0x7f426c15fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:06.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 --2- 172.21.15.60:0/3896332460 >> v2:172.21.15.100:3300/0 conn(0x7f426c15ec40 0x7f426c15fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 -- 172.21.15.60:0/3896332460 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c0cc340 con 0x7f426c0cfe10 2024-04-25T10:16:06.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 -- 172.21.15.60:0/3896332460 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f425c001680 con 0x7f426c0cfe10 2024-04-25T10:16:06.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 -- 172.21.15.60:0/3896332460 >> v1:172.21.15.60:6789/0 conn(0x7f426c0cfe10 legacy=0x7f426c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 --2- 172.21.15.60:0/3896332460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426c0ce950 0x7f42580039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 --2- 172.21.15.60:0/3896332460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4258003f40 0x7f4258004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 --2- 172.21.15.60:0/3896332460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4258004d50 0x7f42580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 -- 172.21.15.60:0/3896332460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c160400 con 0x7f4258003f40 2024-04-25T10:16:06.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 -- 172.21.15.60:0/3896332460 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0d02a0 con 0x7f4258004d50 2024-04-25T10:16:06.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 -- 172.21.15.60:0/3896332460 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425800ba20 con 0x7f426c0ce950 2024-04-25T10:16:06.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f427093a640 1 --2- 172.21.15.60:0/3896332460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4258003f40 0x7f4258004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f427113b640 1 --2- 172.21.15.60:0/3896332460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426c0ce950 0x7f42580039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f427193c640 1 --2- 172.21.15.60:0/3896332460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4258004d50 0x7f42580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f427093a640 1 -- 172.21.15.60:0/3896332460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426c0ce950 msgr2=0x7f42580039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:06.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f427093a640 1 --2- 172.21.15.60:0/3896332460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426c0ce950 0x7f42580039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f427093a640 1 -- 172.21.15.60:0/3896332460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4258004d50 msgr2=0x7f42580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:06.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f427093a640 1 --2- 172.21.15.60:0/3896332460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4258004d50 0x7f42580051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f427093a640 1 -- 172.21.15.60:0/3896332460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4258003530 con 0x7f4258003f40 2024-04-25T10:16:06.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f427093a640 1 --2- 172.21.15.60:0/3896332460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4258003f40 0x7f4258004810 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f425c00c2f0 tx=0x7f425c01cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf81e30a32eec34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 -- 172.21.15.60:0/3896332460 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c007d60 con 0x7f4258003f40 2024-04-25T10:16:06.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 -- 172.21.15.60:0/3896332460 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c0a9020 con 0x7f4258003f40 2024-04-25T10:16:06.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f4263fff640 1 -- 172.21.15.60:0/3896332460 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c005b00 con 0x7f4258003f40 2024-04-25T10:16:06.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f427339f640 1 -- 172.21.15.60:0/3896332460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4258003f40 msgr2=0x7f4258004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f427339f640 1 --2- 172.21.15.60:0/3896332460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4258003f40 0x7f4258004810 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f425c00c2f0 tx=0x7f425c01cd40 comp rx=0 tx=0).stop 2024-04-25T10:16:06.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.534+0000 7f427193c640 1 -- 172.21.15.60:0/3896332460 reap_dead start 2024-04-25T10:16:06.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427339f640 1 -- 172.21.15.60:0/3896332460 shutdown_connections 2024-04-25T10:16:06.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427339f640 1 -- 172.21.15.60:0/3896332460 >> 172.21.15.60:0/3896332460 conn(0x7f426c0d1830 msgr2=0x7f426c0d1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:06.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427339f640 1 -- 172.21.15.60:0/3896332460 shutdown_connections 2024-04-25T10:16:06.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427339f640 1 -- 172.21.15.60:0/3896332460 wait complete. 2024-04-25T10:16:06.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427339f640 1 Processor -- start 2024-04-25T10:16:06.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427339f640 1 -- start start 2024-04-25T10:16:06.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427339f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4258003f40 0x7f426c158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427339f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f426c06f4f0 0x7f426c06f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427339f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f426c0b5da0 0x7f426c0b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427339f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c15f020 con 0x7f426c06f4f0 2024-04-25T10:16:06.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427339f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0d0580 con 0x7f426c0b5da0 2024-04-25T10:16:06.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427339f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426c15f1a0 con 0x7f4258003f40 2024-04-25T10:16:06.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427093a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f426c06f4f0 0x7f426c06f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427093a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f426c06f4f0 0x7f426c06f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39630/0 (socket says 172.21.15.60:39630) 2024-04-25T10:16:06.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427093a640 1 -- 172.21.15.60:0/4065447951 learned_addr learned my addr 172.21.15.60:0/4065447951 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:06.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427193c640 1 --2- 172.21.15.60:0/4065447951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f426c0b5da0 0x7f426c0b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427113b640 1 --2- 172.21.15.60:0/4065447951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4258003f40 0x7f426c158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427093a640 1 -- 172.21.15.60:0/4065447951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4258003f40 msgr2=0x7f426c158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427093a640 1 --2- 172.21.15.60:0/4065447951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4258003f40 0x7f426c158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427093a640 1 -- 172.21.15.60:0/4065447951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f426c0b5da0 msgr2=0x7f426c0b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427093a640 1 --2- 172.21.15.60:0/4065447951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f426c0b5da0 0x7f426c0b6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427093a640 1 -- 172.21.15.60:0/4065447951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c0b4470 con 0x7f426c06f4f0 2024-04-25T10:16:06.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427093a640 1 --2- 172.21.15.60:0/4065447951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f426c06f4f0 0x7f426c06f950 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f425c00c2f0 tx=0x7f4258005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f4261ffb640 1 -- 172.21.15.60:0/4065447951 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c0049f0 con 0x7f426c06f4f0 2024-04-25T10:16:06.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f4261ffb640 1 -- 172.21.15.60:0/4065447951 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c004b90 con 0x7f426c06f4f0 2024-04-25T10:16:06.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427339f640 1 -- 172.21.15.60:0/4065447951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426c0b4750 con 0x7f426c06f4f0 2024-04-25T10:16:06.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f4261ffb640 1 -- 172.21.15.60:0/4065447951 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c0070d0 con 0x7f426c06f4f0 2024-04-25T10:16:06.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.538+0000 7f427339f640 1 -- 172.21.15.60:0/4065447951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426c0b4d10 con 0x7f426c06f4f0 2024-04-25T10:16:06.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.542+0000 7f427339f640 1 -- 172.21.15.60:0/4065447951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426c0c5aa0 con 0x7f426c06f4f0 2024-04-25T10:16:06.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.542+0000 7f4261ffb640 1 -- 172.21.15.60:0/4065447951 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f425c007270 con 0x7f426c06f4f0 2024-04-25T10:16:06.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.542+0000 7f4261ffb640 1 --2- 172.21.15.60:0/4065447951 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4250066f90 0x7f4250069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.542+0000 7f4261ffb640 1 -- 172.21.15.60:0/4065447951 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f425c11e390 con 0x7f426c06f4f0 2024-04-25T10:16:06.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.542+0000 7f427113b640 1 --2- 172.21.15.60:0/4065447951 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4250066f90 0x7f4250069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.542+0000 7f427113b640 1 --2- 172.21.15.60:0/4065447951 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4250066f90 0x7f4250069450 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4268009a30 tx=0x7f4268008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.546+0000 7f4261ffb640 1 -- 172.21.15.60:0/4065447951 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f426c0c5aa0 con 0x7f426c06f4f0 2024-04-25T10:16:06.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:06.710+0000 7f427339f640 1 -- 172.21.15.60:0/4065447951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f426c074770 con 0x7f426c06f4f0 2024-04-25T10:16:07.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.378+0000 7f4261ffb640 1 -- 172.21.15.60:0/4065447951 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v108) v1 ==== 170+0+0 (secure 0 0 0) 0x7f425c016a50 con 0x7f426c06f4f0 2024-04-25T10:16:07.384 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T10:16:07.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.382+0000 7f427339f640 1 -- 172.21.15.60:0/4065447951 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4250066f90 msgr2=0x7f4250069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.382+0000 7f427339f640 1 --2- 172.21.15.60:0/4065447951 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4250066f90 0x7f4250069450 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4268009a30 tx=0x7f4268008040 comp rx=0 tx=0).stop 2024-04-25T10:16:07.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.382+0000 7f427339f640 1 -- 172.21.15.60:0/4065447951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f426c06f4f0 msgr2=0x7f426c06f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.382+0000 7f427339f640 1 --2- 172.21.15.60:0/4065447951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f426c06f4f0 0x7f426c06f950 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f425c00c2f0 tx=0x7f4258005840 comp rx=0 tx=0).stop 2024-04-25T10:16:07.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.382+0000 7f427339f640 1 -- 172.21.15.60:0/4065447951 shutdown_connections 2024-04-25T10:16:07.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.382+0000 7f427339f640 1 --2- 172.21.15.60:0/4065447951 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4250066f90 0x7f4250069450 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.382+0000 7f427339f640 1 --2- 172.21.15.60:0/4065447951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f426c0b5da0 0x7f426c0b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.382+0000 7f427339f640 1 --2- 172.21.15.60:0/4065447951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f426c06f4f0 0x7f426c06f950 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.382+0000 7f427339f640 1 --2- 172.21.15.60:0/4065447951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4258003f40 0x7f426c158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.382+0000 7f427339f640 1 -- 172.21.15.60:0/4065447951 >> 172.21.15.60:0/4065447951 conn(0x7f426c0d1830 msgr2=0x7f426c070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:07.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.382+0000 7f427339f640 1 -- 172.21.15.60:0/4065447951 shutdown_connections 2024-04-25T10:16:07.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.382+0000 7f427339f640 1 -- 172.21.15.60:0/4065447951 wait complete. 2024-04-25T10:16:07.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:16:07.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd65b3bc640 1 Processor -- start 2024-04-25T10:16:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd65b3bc640 1 -- start start 2024-04-25T10:16:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd65b3bc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd65415b6f0 0x7fd6541620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd65b3bc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6540ceef0 0x7fd65415e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd65b3bc640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65415bad0 con 0x7fd65415e0d0 2024-04-25T10:16:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd65b3bc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd654162610 con 0x7fd65415b6f0 2024-04-25T10:16:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd65b3bc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd654161c40 con 0x7fd6540ceef0 2024-04-25T10:16:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd658957640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd65415e0d0 0x7fd6540ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46904/0 (socket says 172.21.15.60:46904) 2024-04-25T10:16:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd658957640 1 -- 172.21.15.60:0/586105127 learned_addr learned my addr 172.21.15.60:0/586105127 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd659959640 1 --2- 172.21.15.60:0/586105127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6540ceef0 0x7fd65415e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd64bfff640 1 -- 172.21.15.60:0/586105127 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1333334851 0 0) 0x7fd65415bad0 con 0x7fd65415e0d0 2024-04-25T10:16:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd659158640 1 --2- 172.21.15.60:0/586105127 >> v2:172.21.15.100:3300/0 conn(0x7fd65415b6f0 0x7fd6541620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd64bfff640 1 -- 172.21.15.60:0/586105127 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd640003530 con 0x7fd65415e0d0 2024-04-25T10:16:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd64bfff640 1 -- 172.21.15.60:0/586105127 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 987563981 0 0) 0x7fd640003530 con 0x7fd65415e0d0 2024-04-25T10:16:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd64bfff640 1 -- 172.21.15.60:0/586105127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6540ceef0 msgr2=0x7fd65415e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd64bfff640 1 --2- 172.21.15.60:0/586105127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6540ceef0 0x7fd65415e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd64bfff640 1 -- 172.21.15.60:0/586105127 >> v2:172.21.15.100:3300/0 conn(0x7fd65415b6f0 msgr2=0x7fd6541620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd64bfff640 1 --2- 172.21.15.60:0/586105127 >> v2:172.21.15.100:3300/0 conn(0x7fd65415b6f0 0x7fd6541620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd64bfff640 1 -- 172.21.15.60:0/586105127 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65415ebe0 con 0x7fd65415e0d0 2024-04-25T10:16:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd64bfff640 1 -- 172.21.15.60:0/586105127 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fd644001680 con 0x7fd65415e0d0 2024-04-25T10:16:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd64bfff640 1 -- 172.21.15.60:0/586105127 >> v1:172.21.15.60:6789/0 conn(0x7fd65415e0d0 legacy=0x7fd6540ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd64bfff640 1 --2- 172.21.15.60:0/586105127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6400039c0 0x7fd640003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd64bfff640 1 --2- 172.21.15.60:0/586105127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd640004810 0x7fd640004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd64bfff640 1 --2- 172.21.15.60:0/586105127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6540ceef0 0x7fd6400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd64bfff640 1 -- 172.21.15.60:0/586105127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd654162610 con 0x7fd640004810 2024-04-25T10:16:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd64bfff640 1 -- 172.21.15.60:0/586105127 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd654161c40 con 0x7fd6400039c0 2024-04-25T10:16:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd64bfff640 1 -- 172.21.15.60:0/586105127 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64000b9d0 con 0x7fd6540ceef0 2024-04-25T10:16:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd658957640 1 --2- 172.21.15.60:0/586105127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd640004810 0x7fd640004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd659158640 1 --2- 172.21.15.60:0/586105127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6400039c0 0x7fd640003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd659959640 1 --2- 172.21.15.60:0/586105127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6540ceef0 0x7fd6400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd658957640 1 -- 172.21.15.60:0/586105127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6540ceef0 msgr2=0x7fd6400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:07.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd658957640 1 --2- 172.21.15.60:0/586105127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6540ceef0 0x7fd6400051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd658957640 1 -- 172.21.15.60:0/586105127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6400039c0 msgr2=0x7fd640003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd658957640 1 --2- 172.21.15.60:0/586105127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6400039c0 0x7fd640003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd658957640 1 -- 172.21.15.60:0/586105127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd640003530 con 0x7fd640004810 2024-04-25T10:16:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.546+0000 7fd659158640 1 --2- 172.21.15.60:0/586105127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6400039c0 0x7fd640003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:07.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.550+0000 7fd658957640 1 --2- 172.21.15.60:0/586105127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd640004810 0x7fd640004c90 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fd644001060 tx=0x7fd64401f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9fb51b53da3d2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.550+0000 7fd64bfff640 1 -- 172.21.15.60:0/586105127 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd644007d80 con 0x7fd640004810 2024-04-25T10:16:07.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.550+0000 7fd64bfff640 1 -- 172.21.15.60:0/586105127 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd644005b00 con 0x7fd640004810 2024-04-25T10:16:07.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.550+0000 7fd64bfff640 1 -- 172.21.15.60:0/586105127 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd644007070 con 0x7fd640004810 2024-04-25T10:16:07.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.550+0000 7fd65b3bc640 1 -- 172.21.15.60:0/586105127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd640004810 msgr2=0x7fd640004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.550+0000 7fd65b3bc640 1 --2- 172.21.15.60:0/586105127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd640004810 0x7fd640004c90 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fd644001060 tx=0x7fd64401f410 comp rx=0 tx=0).stop 2024-04-25T10:16:07.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.550+0000 7fd659959640 1 -- 172.21.15.60:0/586105127 reap_dead start 2024-04-25T10:16:07.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.550+0000 7fd65b3bc640 1 -- 172.21.15.60:0/586105127 shutdown_connections 2024-04-25T10:16:07.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.550+0000 7fd65b3bc640 1 -- 172.21.15.60:0/586105127 >> 172.21.15.60:0/586105127 conn(0x7fd654072340 msgr2=0x7fd654072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:07.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.550+0000 7fd65b3bc640 1 -- 172.21.15.60:0/586105127 shutdown_connections 2024-04-25T10:16:07.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.550+0000 7fd65b3bc640 1 -- 172.21.15.60:0/586105127 wait complete. 2024-04-25T10:16:07.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.550+0000 7fd65b3bc640 1 Processor -- start 2024-04-25T10:16:07.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.550+0000 7fd65b3bc640 1 -- start start 2024-04-25T10:16:07.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd65b3bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd640004810 0x7fd6540caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd65b3bc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd654152820 0x7fd654152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd65b3bc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6540cc6c0 0x7fd6540ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd65b3bc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6540b9550 con 0x7fd640004810 2024-04-25T10:16:07.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd65b3bc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6541618d0 con 0x7fd654152820 2024-04-25T10:16:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd65b3bc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6540b96d0 con 0x7fd6540cc6c0 2024-04-25T10:16:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd659158640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd640004810 0x7fd6540caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd659158640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd640004810 0x7fd6540caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39648/0 (socket says 172.21.15.60:39648) 2024-04-25T10:16:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd659158640 1 -- 172.21.15.60:0/3899369078 learned_addr learned my addr 172.21.15.60:0/3899369078 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd658957640 1 --2- 172.21.15.60:0/3899369078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd654152820 0x7fd654152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd659959640 1 --2- 172.21.15.60:0/3899369078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6540cc6c0 0x7fd6540ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd659158640 1 -- 172.21.15.60:0/3899369078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6540cc6c0 msgr2=0x7fd6540ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd659158640 1 --2- 172.21.15.60:0/3899369078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6540cc6c0 0x7fd6540ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd659158640 1 -- 172.21.15.60:0/3899369078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd654152820 msgr2=0x7fd654152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd659158640 1 --2- 172.21.15.60:0/3899369078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd654152820 0x7fd654152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd659158640 1 -- 172.21.15.60:0/3899369078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd654156130 con 0x7fd640004810 2024-04-25T10:16:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd658957640 1 --2- 172.21.15.60:0/3899369078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd654152820 0x7fd654152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd659959640 1 --2- 172.21.15.60:0/3899369078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6540cc6c0 0x7fd6540ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd659158640 1 --2- 172.21.15.60:0/3899369078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd640004810 0x7fd6540caae0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fd650007f60 tx=0x7fd650007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd649ffb640 1 -- 172.21.15.60:0/3899369078 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65001a030 con 0x7fd640004810 2024-04-25T10:16:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd649ffb640 1 -- 172.21.15.60:0/3899369078 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6500039a0 con 0x7fd640004810 2024-04-25T10:16:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd65b3bc640 1 -- 172.21.15.60:0/3899369078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd654155370 con 0x7fd640004810 2024-04-25T10:16:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd649ffb640 1 -- 172.21.15.60:0/3899369078 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd650003d70 con 0x7fd640004810 2024-04-25T10:16:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd65b3bc640 1 -- 172.21.15.60:0/3899369078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd654155930 con 0x7fd640004810 2024-04-25T10:16:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd65b3bc640 1 -- 172.21.15.60:0/3899369078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd654074770 con 0x7fd640004810 2024-04-25T10:16:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd649ffb640 1 -- 172.21.15.60:0/3899369078 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd65001d070 con 0x7fd640004810 2024-04-25T10:16:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd649ffb640 1 --2- 172.21.15.60:0/3899369078 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd638066f60 0x7fd638069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.554+0000 7fd649ffb640 1 -- 172.21.15.60:0/3899369078 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fd650085e60 con 0x7fd640004810 2024-04-25T10:16:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.558+0000 7fd658957640 1 --2- 172.21.15.60:0/3899369078 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd638066f60 0x7fd638069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.558+0000 7fd658957640 1 --2- 172.21.15.60:0/3899369078 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd638066f60 0x7fd638069420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd6440ae000 tx=0x7fd644001a50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.562+0000 7fd649ffb640 1 -- 172.21.15.60:0/3899369078 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd650056ed0 con 0x7fd640004810 2024-04-25T10:16:07.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:07.758+0000 7fd65b3bc640 1 -- 172.21.15.60:0/3899369078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd6540b8220 con 0x7fd640004810 2024-04-25T10:16:08.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.382+0000 7fd649ffb640 1 -- 172.21.15.60:0/3899369078 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v89) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd6500567d0 con 0x7fd640004810 2024-04-25T10:16:08.386 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 16 and data pool 17 2024-04-25T10:16:08.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.382+0000 7fd65b3bc640 1 -- 172.21.15.60:0/3899369078 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd638066f60 msgr2=0x7fd638069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.382+0000 7fd65b3bc640 1 --2- 172.21.15.60:0/3899369078 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd638066f60 0x7fd638069420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd6440ae000 tx=0x7fd644001a50 comp rx=0 tx=0).stop 2024-04-25T10:16:08.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.382+0000 7fd65b3bc640 1 -- 172.21.15.60:0/3899369078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd640004810 msgr2=0x7fd6540caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.382+0000 7fd65b3bc640 1 --2- 172.21.15.60:0/3899369078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd640004810 0x7fd6540caae0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fd650007f60 tx=0x7fd650007f90 comp rx=0 tx=0).stop 2024-04-25T10:16:08.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.382+0000 7fd65b3bc640 1 -- 172.21.15.60:0/3899369078 shutdown_connections 2024-04-25T10:16:08.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.382+0000 7fd65b3bc640 1 --2- 172.21.15.60:0/3899369078 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd638066f60 0x7fd638069420 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.382+0000 7fd65b3bc640 1 --2- 172.21.15.60:0/3899369078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6540cc6c0 0x7fd6540ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.382+0000 7fd65b3bc640 1 --2- 172.21.15.60:0/3899369078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd654152820 0x7fd654152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.382+0000 7fd65b3bc640 1 --2- 172.21.15.60:0/3899369078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd640004810 0x7fd6540caae0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.382+0000 7fd65b3bc640 1 -- 172.21.15.60:0/3899369078 >> 172.21.15.60:0/3899369078 conn(0x7fd654072340 msgr2=0x7fd6540d3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:08.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.386+0000 7fd65b3bc640 1 -- 172.21.15.60:0/3899369078 shutdown_connections 2024-04-25T10:16:08.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.386+0000 7fd65b3bc640 1 -- 172.21.15.60:0/3899369078 wait complete. 2024-04-25T10:16:08.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:08.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a4d187640 1 Processor -- start 2024-04-25T10:16:08.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a4d187640 1 -- start start 2024-04-25T10:16:08.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a4d187640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a4815deb0 0x7f4a480b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a4d187640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a4806d200 con 0x7f4a480d0340 2024-04-25T10:16:08.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a4d187640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a480cefe0 con 0x7f4a480ce9c0 2024-04-25T10:16:08.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a4d187640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480cf2c0 con 0x7f4a4815deb0 2024-04-25T10:16:08.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a4659c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4a480d0340 0x7f4a480ce2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46914/0 (socket says 172.21.15.60:46914) 2024-04-25T10:16:08.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a4659c640 1 -- 172.21.15.60:0/1229668287 learned_addr learned my addr 172.21.15.60:0/1229668287 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:08.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a46d9d640 1 --2- 172.21.15.60:0/1229668287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a4815deb0 0x7f4a480b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a45d9b640 1 -- 172.21.15.60:0/1229668287 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186838262 0 0) 0x7f4a4806d200 con 0x7f4a480d0340 2024-04-25T10:16:08.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a45d9b640 1 -- 172.21.15.60:0/1229668287 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a30003530 con 0x7f4a480d0340 2024-04-25T10:16:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a45d9b640 1 -- 172.21.15.60:0/1229668287 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2953171393 0 0) 0x7f4a30003530 con 0x7f4a480d0340 2024-04-25T10:16:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a45d9b640 1 -- 172.21.15.60:0/1229668287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a4815deb0 msgr2=0x7f4a480b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a45d9b640 1 --2- 172.21.15.60:0/1229668287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a4815deb0 0x7f4a480b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a45d9b640 1 -- 172.21.15.60:0/1229668287 >> v1:172.21.15.100:6789/0 conn(0x7f4a480ce9c0 legacy=0x7f4a480d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a45d9b640 1 -- 172.21.15.60:0/1229668287 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a480d1d70 con 0x7f4a480d0340 2024-04-25T10:16:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a45d9b640 1 -- 172.21.15.60:0/1229668287 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f4a3c001680 con 0x7f4a480d0340 2024-04-25T10:16:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a45d9b640 1 -- 172.21.15.60:0/1229668287 >> v1:172.21.15.60:6789/0 conn(0x7f4a480d0340 legacy=0x7f4a480ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a45d9b640 1 --2- 172.21.15.60:0/1229668287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a4815deb0 0x7f4a300039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a45d9b640 1 --2- 172.21.15.60:0/1229668287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a30003f30 0x7f4a30004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a45d9b640 1 --2- 172.21.15.60:0/1229668287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a30004d50 0x7f4a300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a45d9b640 1 -- 172.21.15.60:0/1229668287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480cf2c0 con 0x7f4a30003f30 2024-04-25T10:16:08.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a45d9b640 1 -- 172.21.15.60:0/1229668287 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3000ba80 con 0x7f4a30004d50 2024-04-25T10:16:08.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a45d9b640 1 -- 172.21.15.60:0/1229668287 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3000bc80 con 0x7f4a4815deb0 2024-04-25T10:16:08.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a4659c640 1 --2- 172.21.15.60:0/1229668287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a30003f30 0x7f4a30004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a46d9d640 1 --2- 172.21.15.60:0/1229668287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a4815deb0 0x7f4a300039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a4759e640 1 --2- 172.21.15.60:0/1229668287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a30004d50 0x7f4a300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a4659c640 1 -- 172.21.15.60:0/1229668287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a4815deb0 msgr2=0x7f4a300039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:08.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a4659c640 1 --2- 172.21.15.60:0/1229668287 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a4815deb0 0x7f4a300039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a4659c640 1 -- 172.21.15.60:0/1229668287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a30004d50 msgr2=0x7f4a300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:08.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a4659c640 1 --2- 172.21.15.60:0/1229668287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a30004d50 0x7f4a300051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.534+0000 7f4a4659c640 1 -- 172.21.15.60:0/1229668287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a30003530 con 0x7f4a30003f30 2024-04-25T10:16:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.538+0000 7f4a4659c640 1 --2- 172.21.15.60:0/1229668287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a30003f30 0x7f4a30004810 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00c160 tx=0x7f4a3c01cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb57c2500ff8f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.538+0000 7f4a45d9b640 1 -- 172.21.15.60:0/1229668287 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c007c90 con 0x7f4a30003f30 2024-04-25T10:16:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.538+0000 7f4a45d9b640 1 -- 172.21.15.60:0/1229668287 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c0a9070 con 0x7f4a30003f30 2024-04-25T10:16:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.538+0000 7f4a45d9b640 1 -- 172.21.15.60:0/1229668287 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c005d40 con 0x7f4a30003f30 2024-04-25T10:16:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.538+0000 7f4a4d187640 1 -- 172.21.15.60:0/1229668287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a30003f30 msgr2=0x7f4a30004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.538+0000 7f4a4d187640 1 --2- 172.21.15.60:0/1229668287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a30003f30 0x7f4a30004810 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00c160 tx=0x7f4a3c01cbc0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.538+0000 7f4a4759e640 1 -- 172.21.15.60:0/1229668287 reap_dead start 2024-04-25T10:16:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.538+0000 7f4a4d187640 1 -- 172.21.15.60:0/1229668287 shutdown_connections 2024-04-25T10:16:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.538+0000 7f4a4d187640 1 -- 172.21.15.60:0/1229668287 >> 172.21.15.60:0/1229668287 conn(0x7f4a4806c090 msgr2=0x7f4a4806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.538+0000 7f4a4d187640 1 -- 172.21.15.60:0/1229668287 shutdown_connections 2024-04-25T10:16:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.538+0000 7f4a4d187640 1 -- 172.21.15.60:0/1229668287 wait complete. 2024-04-25T10:16:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.538+0000 7f4a4d187640 1 Processor -- start 2024-04-25T10:16:08.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.538+0000 7f4a4d187640 1 -- start start 2024-04-25T10:16:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4d187640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a30003f30 0x7f4a480c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4d187640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a48150a00 0x7f4a480cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4d187640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a48150e80 0x7f4a480cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4d187640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a48169c30 con 0x7f4a48150e80 2024-04-25T10:16:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4d187640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480d3770 con 0x7f4a48150a00 2024-04-25T10:16:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4d187640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a48169db0 con 0x7f4a30003f30 2024-04-25T10:16:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a48150e80 0x7f4a480cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a48150e80 0x7f4a480cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39668/0 (socket says 172.21.15.60:39668) 2024-04-25T10:16:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4759e640 1 -- 172.21.15.60:0/1461231515 learned_addr learned my addr 172.21.15.60:0/1461231515 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4659c640 1 --2- 172.21.15.60:0/1461231515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a48150a00 0x7f4a480cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a46d9d640 1 --2- 172.21.15.60:0/1461231515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a30003f30 0x7f4a480c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4759e640 1 -- 172.21.15.60:0/1461231515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a30003f30 msgr2=0x7f4a480c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4759e640 1 --2- 172.21.15.60:0/1461231515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a30003f30 0x7f4a480c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4759e640 1 -- 172.21.15.60:0/1461231515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a48150a00 msgr2=0x7f4a480cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4759e640 1 --2- 172.21.15.60:0/1461231515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a48150a00 0x7f4a480cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4759e640 1 -- 172.21.15.60:0/1461231515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4814f960 con 0x7f4a48150e80 2024-04-25T10:16:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4659c640 1 --2- 172.21.15.60:0/1461231515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a48150a00 0x7f4a480cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4759e640 1 --2- 172.21.15.60:0/1461231515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a48150e80 0x7f4a480cce00 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f4a480d1590 tx=0x7f4a4000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a2f7fe640 1 -- 172.21.15.60:0/1461231515 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40003980 con 0x7f4a48150e80 2024-04-25T10:16:08.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a2f7fe640 1 -- 172.21.15.60:0/1461231515 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a40018070 con 0x7f4a48150e80 2024-04-25T10:16:08.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a2f7fe640 1 -- 172.21.15.60:0/1461231515 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40003d60 con 0x7f4a48150e80 2024-04-25T10:16:08.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4d187640 1 -- 172.21.15.60:0/1461231515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4814fc40 con 0x7f4a48150e80 2024-04-25T10:16:08.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4d187640 1 -- 172.21.15.60:0/1461231515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a48150200 con 0x7f4a48150e80 2024-04-25T10:16:08.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a4d187640 1 -- 172.21.15.60:0/1461231515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a48074770 con 0x7f4a48150e80 2024-04-25T10:16:08.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a2f7fe640 1 -- 172.21.15.60:0/1461231515 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a400050d0 con 0x7f4a48150e80 2024-04-25T10:16:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a2f7fe640 1 --2- 172.21.15.60:0/1461231515 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a34066f90 0x7f4a34069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a2f7fe640 1 -- 172.21.15.60:0/1461231515 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a40084db0 con 0x7f4a48150e80 2024-04-25T10:16:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.542+0000 7f4a46d9d640 1 --2- 172.21.15.60:0/1461231515 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a34066f90 0x7f4a34069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.546+0000 7f4a46d9d640 1 --2- 172.21.15.60:0/1461231515 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a34066f90 0x7f4a34069450 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4a480b7990 tx=0x7f4a38008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.550+0000 7f4a2f7fe640 1 -- 172.21.15.60:0/1461231515 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4a40055dc0 con 0x7f4a48150e80 2024-04-25T10:16:08.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.706+0000 7f4a4d187640 1 -- 172.21.15.60:0/1461231515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a480b7270 con 0x7f4a48150e80 2024-04-25T10:16:08.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.710+0000 7f4a2f7fe640 1 -- 172.21.15.60:0/1461231515 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16652 (secure 0 0 0) 0x7f4a400556c0 con 0x7f4a48150e80 2024-04-25T10:16:08.715 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:08.715 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":109,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:16:07.762597+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:06.033387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:06.753421+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.710+0000 7f4a4d187640 1 -- 172.21.15.60:0/1461231515 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a34066f90 msgr2=0x7f4a34069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.710+0000 7f4a4d187640 1 --2- 172.21.15.60:0/1461231515 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a34066f90 0x7f4a34069450 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4a480b7990 tx=0x7f4a38008040 comp rx=0 tx=0).stop 2024-04-25T10:16:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.710+0000 7f4a4d187640 1 -- 172.21.15.60:0/1461231515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a48150e80 msgr2=0x7f4a480cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.710+0000 7f4a4d187640 1 --2- 172.21.15.60:0/1461231515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a48150e80 0x7f4a480cce00 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f4a480d1590 tx=0x7f4a4000eb20 comp rx=0 tx=0).stop 2024-04-25T10:16:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.710+0000 7f4a4d187640 1 -- 172.21.15.60:0/1461231515 shutdown_connections 2024-04-25T10:16:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.710+0000 7f4a4d187640 1 --2- 172.21.15.60:0/1461231515 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4a34066f90 0x7f4a34069450 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.710+0000 7f4a4d187640 1 --2- 172.21.15.60:0/1461231515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a48150e80 0x7f4a480cce00 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.710+0000 7f4a4d187640 1 --2- 172.21.15.60:0/1461231515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a48150a00 0x7f4a480cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.710+0000 7f4a4d187640 1 --2- 172.21.15.60:0/1461231515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a30003f30 0x7f4a480c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.710+0000 7f4a4d187640 1 -- 172.21.15.60:0/1461231515 >> 172.21.15.60:0/1461231515 conn(0x7f4a4806c090 msgr2=0x7f4a48160140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:08.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.710+0000 7f4a4d187640 1 -- 172.21.15.60:0/1461231515 shutdown_connections 2024-04-25T10:16:08.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.710+0000 7f4a4d187640 1 -- 172.21.15.60:0/1461231515 wait complete. 2024-04-25T10:16:08.735 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:08.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.862+0000 7ff20d89c640 1 Processor -- start 2024-04-25T10:16:08.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.862+0000 7ff20d89c640 1 -- start start 2024-04-25T10:16:08.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.862+0000 7ff20d89c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff208160eb0 0x7ff2080b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.862+0000 7ff20d89c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2080b9730 0x7ff2080b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.862+0000 7ff20d89c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2080ce9c0 con 0x7ff208161290 2024-04-25T10:16:08.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.862+0000 7ff20d89c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2080ceca0 con 0x7ff208160eb0 2024-04-25T10:16:08.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.862+0000 7ff20d89c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080d2490 con 0x7ff2080b9730 2024-04-25T10:16:08.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.862+0000 7ff206ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff208160eb0 0x7ff2080b91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.862+0000 7ff206ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff208160eb0 0x7ff2080b91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39670/0 (socket says 172.21.15.60:39670) 2024-04-25T10:16:08.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.862+0000 7ff206ffd640 1 -- 172.21.15.60:0/2731515525 learned_addr learned my addr 172.21.15.60:0/2731515525 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:08.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.862+0000 7ff2077fe640 1 --2- 172.21.15.60:0/2731515525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2080b9730 0x7ff2080b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.862+0000 7ff206ffd640 1 -- 172.21.15.60:0/2731515525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2080b9730 msgr2=0x7ff2080b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:08.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.862+0000 7ff206ffd640 1 --2- 172.21.15.60:0/2731515525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2080b9730 0x7ff2080b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.862+0000 7ff206ffd640 1 -- 172.21.15.60:0/2731515525 >> v1:172.21.15.100:6789/0 conn(0x7ff208161290 legacy=0x7ff2080b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:08.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.862+0000 7ff206ffd640 1 -- 172.21.15.60:0/2731515525 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2080d2770 con 0x7ff208160eb0 2024-04-25T10:16:08.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.862+0000 7ff2077fe640 1 --2- 172.21.15.60:0/2731515525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2080b9730 0x7ff2080b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:08.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff206ffd640 1 --2- 172.21.15.60:0/2731515525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff208160eb0 0x7ff2080b91f0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009870 tx=0x7ff1f801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cde93b0be0f0d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff205ffb640 1 -- 172.21.15.60:0/2731515525 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f80b5070 con 0x7ff208160eb0 2024-04-25T10:16:08.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff205ffb640 1 -- 172.21.15.60:0/2731515525 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f80a89d0 con 0x7ff208160eb0 2024-04-25T10:16:08.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff205ffb640 1 -- 172.21.15.60:0/2731515525 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f80a8da0 con 0x7ff208160eb0 2024-04-25T10:16:08.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff20d89c640 1 -- 172.21.15.60:0/2731515525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff208160eb0 msgr2=0x7ff2080b91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff20d89c640 1 --2- 172.21.15.60:0/2731515525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff208160eb0 0x7ff2080b91f0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009870 tx=0x7ff1f801ed10 comp rx=0 tx=0).stop 2024-04-25T10:16:08.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff20d89c640 1 -- 172.21.15.60:0/2731515525 shutdown_connections 2024-04-25T10:16:08.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff20d89c640 1 --2- 172.21.15.60:0/2731515525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2080b9730 0x7ff2080b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff20d89c640 1 --2- 172.21.15.60:0/2731515525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff208160eb0 0x7ff2080b91f0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff20d89c640 1 -- 172.21.15.60:0/2731515525 >> 172.21.15.60:0/2731515525 conn(0x7ff2080d05c0 msgr2=0x7ff2080d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:08.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff20d89c640 1 -- 172.21.15.60:0/2731515525 shutdown_connections 2024-04-25T10:16:08.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff20d89c640 1 -- 172.21.15.60:0/2731515525 wait complete. 2024-04-25T10:16:08.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff20d89c640 1 Processor -- start 2024-04-25T10:16:08.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff20d89c640 1 -- start start 2024-04-25T10:16:08.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff20d89c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2080b9730 0x7ff2080cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff20d89c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff208160eb0 0x7ff2081f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff20d89c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff208161290 0x7ff20815c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff20d89c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080cef80 con 0x7ff208161290 2024-04-25T10:16:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff20d89c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080b7180 con 0x7ff2080b9730 2024-04-25T10:16:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff20d89c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2080b5930 con 0x7ff208160eb0 2024-04-25T10:16:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff2077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff208161290 0x7ff20815c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff2077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff208161290 0x7ff20815c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39672/0 (socket says 172.21.15.60:39672) 2024-04-25T10:16:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff2077fe640 1 -- 172.21.15.60:0/2226300159 learned_addr learned my addr 172.21.15.60:0/2226300159 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff2067fc640 1 --2- 172.21.15.60:0/2226300159 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff208160eb0 0x7ff2081f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff206ffd640 1 --2- 172.21.15.60:0/2226300159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2080b9730 0x7ff2080cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff2077fe640 1 -- 172.21.15.60:0/2226300159 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff208160eb0 msgr2=0x7ff2081f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff2077fe640 1 --2- 172.21.15.60:0/2226300159 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff208160eb0 0x7ff2081f2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff2077fe640 1 -- 172.21.15.60:0/2226300159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2080b9730 msgr2=0x7ff2080cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff2077fe640 1 --2- 172.21.15.60:0/2226300159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2080b9730 0x7ff2080cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff2077fe640 1 -- 172.21.15.60:0/2226300159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2080cc050 con 0x7ff208161290 2024-04-25T10:16:08.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.866+0000 7ff2077fe640 1 --2- 172.21.15.60:0/2226300159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff208161290 0x7ff20815c430 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7ff20000ceb0 tx=0x7ff20000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.870+0000 7ff1effff640 1 -- 172.21.15.60:0/2226300159 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20001a070 con 0x7ff208161290 2024-04-25T10:16:08.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.870+0000 7ff1effff640 1 -- 172.21.15.60:0/2226300159 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20000d840 con 0x7ff208161290 2024-04-25T10:16:08.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.870+0000 7ff20d89c640 1 -- 172.21.15.60:0/2226300159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2080cc330 con 0x7ff208161290 2024-04-25T10:16:08.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.870+0000 7ff1effff640 1 -- 172.21.15.60:0/2226300159 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20000dc30 con 0x7ff208161290 2024-04-25T10:16:08.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.870+0000 7ff20d89c640 1 -- 172.21.15.60:0/2226300159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2080cc8f0 con 0x7ff208161290 2024-04-25T10:16:08.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.870+0000 7ff20d89c640 1 -- 172.21.15.60:0/2226300159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff208074770 con 0x7ff208161290 2024-04-25T10:16:08.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.870+0000 7ff1effff640 1 -- 172.21.15.60:0/2226300159 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff20001d070 con 0x7ff208161290 2024-04-25T10:16:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.870+0000 7ff1effff640 1 --2- 172.21.15.60:0/2226300159 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff1f4066f60 0x7ff1f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.870+0000 7ff1effff640 1 -- 172.21.15.60:0/2226300159 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff200085bf0 con 0x7ff208161290 2024-04-25T10:16:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.870+0000 7ff206ffd640 1 --2- 172.21.15.60:0/2226300159 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff1f4066f60 0x7ff1f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.870+0000 7ff206ffd640 1 --2- 172.21.15.60:0/2226300159 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff1f4066f60 0x7ff1f4069420 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff1f80b2000 tx=0x7ff1f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:08.874+0000 7ff1effff640 1 -- 172.21.15.60:0/2226300159 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff200024020 con 0x7ff208161290 2024-04-25T10:16:09.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.034+0000 7ff20d89c640 1 -- 172.21.15.60:0/2226300159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2080d15b0 con 0x7ff208161290 2024-04-25T10:16:09.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.038+0000 7ff1effff640 1 -- 172.21.15.60:0/2226300159 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16652 (secure 0 0 0) 0x7ff200056cf0 con 0x7ff208161290 2024-04-25T10:16:09.042 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:09.042 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":109,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:16:07.762597+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:06.033387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:06.753421+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:09.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.038+0000 7ff20d89c640 1 -- 172.21.15.60:0/2226300159 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff1f4066f60 msgr2=0x7ff1f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.038+0000 7ff20d89c640 1 --2- 172.21.15.60:0/2226300159 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff1f4066f60 0x7ff1f4069420 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff1f80b2000 tx=0x7ff1f8008040 comp rx=0 tx=0).stop 2024-04-25T10:16:09.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.038+0000 7ff20d89c640 1 -- 172.21.15.60:0/2226300159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff208161290 msgr2=0x7ff20815c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.038+0000 7ff20d89c640 1 --2- 172.21.15.60:0/2226300159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff208161290 0x7ff20815c430 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7ff20000ceb0 tx=0x7ff20000a710 comp rx=0 tx=0).stop 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.038+0000 7ff20d89c640 1 -- 172.21.15.60:0/2226300159 shutdown_connections 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.038+0000 7ff20d89c640 1 --2- 172.21.15.60:0/2226300159 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff1f4066f60 0x7ff1f4069420 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.038+0000 7ff20d89c640 1 --2- 172.21.15.60:0/2226300159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff208161290 0x7ff20815c430 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.038+0000 7ff20d89c640 1 --2- 172.21.15.60:0/2226300159 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff208160eb0 0x7ff2081f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.038+0000 7ff20d89c640 1 --2- 172.21.15.60:0/2226300159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2080b9730 0x7ff2080cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.038+0000 7ff20d89c640 1 -- 172.21.15.60:0/2226300159 >> 172.21.15.60:0/2226300159 conn(0x7ff2080d05c0 msgr2=0x7ff2080b81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.038+0000 7ff20d89c640 1 -- 172.21.15.60:0/2226300159 shutdown_connections 2024-04-25T10:16:09.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.038+0000 7ff20d89c640 1 -- 172.21.15.60:0/2226300159 wait complete. 2024-04-25T10:16:09.064 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:16:09.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.182+0000 7f95e5ac1640 1 Processor -- start 2024-04-25T10:16:09.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.182+0000 7f95e5ac1640 1 -- start start 2024-04-25T10:16:09.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.186+0000 7f95e5ac1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95e00d05a0 0x7f95e00b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.186+0000 7f95e5ac1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f95e006ac10 0x7f95e0070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.186+0000 7f95e5ac1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e006fb70 con 0x7f95e00b4dd0 2024-04-25T10:16:09.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.186+0000 7f95e5ac1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e00b56f0 con 0x7f95e006ac10 2024-04-25T10:16:09.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.186+0000 7f95e5ac1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e006b1d0 con 0x7f95e00d05a0 2024-04-25T10:16:09.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.186+0000 7f95deffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95e00d05a0 0x7f95e00b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.186+0000 7f95dffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f95e006ac10 0x7f95e0070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.186+0000 7f95dffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f95e006ac10 0x7f95e0070020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49750/0 (socket says 172.21.15.60:49750) 2024-04-25T10:16:09.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.186+0000 7f95dffff640 1 -- 172.21.15.60:0/2286125464 learned_addr learned my addr 172.21.15.60:0/2286125464 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:09.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.186+0000 7f95df7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f95e00b4dd0 0x7f95e015be30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:55308/0 (socket says 172.21.15.60:55308) 2024-04-25T10:16:09.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.186+0000 7f95deffd640 1 -- 172.21.15.60:0/2286125464 >> v2:172.21.15.100:3300/0 conn(0x7f95e006ac10 msgr2=0x7f95e0070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.186+0000 7f95deffd640 1 --2- 172.21.15.60:0/2286125464 >> v2:172.21.15.100:3300/0 conn(0x7f95e006ac10 0x7f95e0070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.186+0000 7f95deffd640 1 -- 172.21.15.60:0/2286125464 >> v1:172.21.15.100:6789/0 conn(0x7f95e00b4dd0 legacy=0x7f95e015be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.186+0000 7f95deffd640 1 -- 172.21.15.60:0/2286125464 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e006b4b0 con 0x7f95e00d05a0 2024-04-25T10:16:09.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.186+0000 7f95dffff640 1 --2- 172.21.15.60:0/2286125464 >> v2:172.21.15.100:3300/0 conn(0x7f95e006ac10 0x7f95e0070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:09.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95deffd640 1 --2- 172.21.15.60:0/2286125464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95e00d05a0 0x7f95e00b51b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f95c80097c0 tx=0x7f95c801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cca96787d8f5e7bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95de7fc640 1 -- 172.21.15.60:0/2286125464 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c80ab600 con 0x7f95e00d05a0 2024-04-25T10:16:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95de7fc640 1 -- 172.21.15.60:0/2286125464 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c80b0070 con 0x7f95e00d05a0 2024-04-25T10:16:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95de7fc640 1 -- 172.21.15.60:0/2286125464 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c80ab9c0 con 0x7f95e00d05a0 2024-04-25T10:16:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95e5ac1640 1 -- 172.21.15.60:0/2286125464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95e00d05a0 msgr2=0x7f95e00b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95e5ac1640 1 --2- 172.21.15.60:0/2286125464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95e00d05a0 0x7f95e00b51b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f95c80097c0 tx=0x7f95c801c3e0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95e5ac1640 1 -- 172.21.15.60:0/2286125464 shutdown_connections 2024-04-25T10:16:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95e5ac1640 1 --2- 172.21.15.60:0/2286125464 >> v2:172.21.15.100:3300/0 conn(0x7f95e006ac10 0x7f95e0070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95e5ac1640 1 --2- 172.21.15.60:0/2286125464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95e00d05a0 0x7f95e00b51b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95e5ac1640 1 -- 172.21.15.60:0/2286125464 >> 172.21.15.60:0/2286125464 conn(0x7f95e00b2360 msgr2=0x7f95e00b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95e5ac1640 1 -- 172.21.15.60:0/2286125464 shutdown_connections 2024-04-25T10:16:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95e5ac1640 1 -- 172.21.15.60:0/2286125464 wait complete. 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95e5ac1640 1 Processor -- start 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95e5ac1640 1 -- start start 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95e5ac1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95e006ac10 0x7f95e0159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95e5ac1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e00b4dd0 0x7f95e013a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95e5ac1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95e00d05a0 0x7f95e00b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95e5ac1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e0145510 con 0x7f95e006ac10 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95e5ac1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e0145390 con 0x7f95e00b4dd0 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95e5ac1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e00ce1e0 con 0x7f95e00d05a0 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95df7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95e006ac10 0x7f95e0159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95df7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95e006ac10 0x7f95e0159580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35172/0 (socket says 172.21.15.60:35172) 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95df7fe640 1 -- 172.21.15.60:0/2679923472 learned_addr learned my addr 172.21.15.60:0/2679923472 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95dffff640 1 --2- 172.21.15.60:0/2679923472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95e00d05a0 0x7f95e00b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95deffd640 1 --2- 172.21.15.60:0/2679923472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e00b4dd0 0x7f95e013a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95df7fe640 1 -- 172.21.15.60:0/2679923472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95e00d05a0 msgr2=0x7f95e00b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95df7fe640 1 --2- 172.21.15.60:0/2679923472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95e00d05a0 0x7f95e00b1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95df7fe640 1 -- 172.21.15.60:0/2679923472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e00b4dd0 msgr2=0x7f95e013a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95df7fe640 1 --2- 172.21.15.60:0/2679923472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e00b4dd0 0x7f95e013a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.190+0000 7f95df7fe640 1 -- 172.21.15.60:0/2679923472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e015c5c0 con 0x7f95e006ac10 2024-04-25T10:16:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.194+0000 7f95df7fe640 1 --2- 172.21.15.60:0/2679923472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95e006ac10 0x7f95e0159580 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f95d000c490 tx=0x7f95d000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.194+0000 7f95d7fff640 1 -- 172.21.15.60:0/2679923472 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0018070 con 0x7f95e006ac10 2024-04-25T10:16:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.194+0000 7f95d7fff640 1 -- 172.21.15.60:0/2679923472 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d0007480 con 0x7f95e006ac10 2024-04-25T10:16:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.194+0000 7f95e5ac1640 1 -- 172.21.15.60:0/2679923472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e015c8a0 con 0x7f95e006ac10 2024-04-25T10:16:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.194+0000 7f95d7fff640 1 -- 172.21.15.60:0/2679923472 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0004a10 con 0x7f95e006ac10 2024-04-25T10:16:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.194+0000 7f95e5ac1640 1 -- 172.21.15.60:0/2679923472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e015ce60 con 0x7f95e006ac10 2024-04-25T10:16:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.194+0000 7f95e5ac1640 1 -- 172.21.15.60:0/2679923472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e0074770 con 0x7f95e006ac10 2024-04-25T10:16:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.194+0000 7f95d7fff640 1 -- 172.21.15.60:0/2679923472 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95d001b070 con 0x7f95e006ac10 2024-04-25T10:16:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.194+0000 7f95d7fff640 1 --2- 172.21.15.60:0/2679923472 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f95c0066f60 0x7f95c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.194+0000 7f95d7fff640 1 -- 172.21.15.60:0/2679923472 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95d00842b0 con 0x7f95e006ac10 2024-04-25T10:16:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.194+0000 7f95deffd640 1 --2- 172.21.15.60:0/2679923472 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f95c0066f60 0x7f95c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.194+0000 7f95deffd640 1 --2- 172.21.15.60:0/2679923472 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f95c0066f60 0x7f95c0069420 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f95c80af000 tx=0x7f95c80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.198+0000 7f95d7fff640 1 -- 172.21.15.60:0/2679923472 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95d00552c0 con 0x7f95e006ac10 2024-04-25T10:16:09.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:09.390+0000 7f95e5ac1640 1 -- 172.21.15.60:0/2679923472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f95e015ed90 con 0x7f95e006ac10 2024-04-25T10:16:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.398+0000 7f95d7fff640 1 -- 172.21.15.60:0/2679923472 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v92) v1 ==== 118+0+0 (secure 0 0 0) 0x7f95d0054bc0 con 0x7f95e006ac10 2024-04-25T10:16:10.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.402+0000 7f95e5ac1640 1 -- 172.21.15.60:0/2679923472 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f95c0066f60 msgr2=0x7f95c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.402+0000 7f95e5ac1640 1 --2- 172.21.15.60:0/2679923472 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f95c0066f60 0x7f95c0069420 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f95c80af000 tx=0x7f95c80a7000 comp rx=0 tx=0).stop 2024-04-25T10:16:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.402+0000 7f95e5ac1640 1 -- 172.21.15.60:0/2679923472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95e006ac10 msgr2=0x7f95e0159580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.402+0000 7f95e5ac1640 1 --2- 172.21.15.60:0/2679923472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95e006ac10 0x7f95e0159580 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f95d000c490 tx=0x7f95d000c950 comp rx=0 tx=0).stop 2024-04-25T10:16:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.402+0000 7f95e5ac1640 1 -- 172.21.15.60:0/2679923472 shutdown_connections 2024-04-25T10:16:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.402+0000 7f95e5ac1640 1 --2- 172.21.15.60:0/2679923472 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f95c0066f60 0x7f95c0069420 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.402+0000 7f95e5ac1640 1 --2- 172.21.15.60:0/2679923472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95e00d05a0 0x7f95e00b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.402+0000 7f95e5ac1640 1 --2- 172.21.15.60:0/2679923472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e00b4dd0 0x7f95e013a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.402+0000 7f95e5ac1640 1 --2- 172.21.15.60:0/2679923472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95e006ac10 0x7f95e0159580 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.402+0000 7f95e5ac1640 1 -- 172.21.15.60:0/2679923472 >> 172.21.15.60:0/2679923472 conn(0x7f95e00b2360 msgr2=0x7f95e0072c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.402+0000 7f95e5ac1640 1 -- 172.21.15.60:0/2679923472 shutdown_connections 2024-04-25T10:16:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.402+0000 7f95e5ac1640 1 -- 172.21.15.60:0/2679923472 wait complete. 2024-04-25T10:16:10.427 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:16:10.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:16:10.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd57e901640 1 Processor -- start 2024-04-25T10:16:10.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd57e901640 1 -- start start 2024-04-25T10:16:10.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd57e901640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd5780cbcf0 0x7fd578160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd57e901640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd578071c30 0x7fd57815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd57e901640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd57815fa20 0x7fd57815bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd57e901640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5780b2dc0 con 0x7fd578071c30 2024-04-25T10:16:10.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd57e901640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd578144a40 con 0x7fd5780cbcf0 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd57e901640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780cc9f0 con 0x7fd57815fa20 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd5777fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd578071c30 0x7fd57815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd5777fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd578071c30 0x7fd57815f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35176/0 (socket says 172.21.15.60:35176) 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd5777fe640 1 -- 172.21.15.60:0/1304976746 learned_addr learned my addr 172.21.15.60:0/1304976746 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd577fff640 1 --2- 172.21.15.60:0/1304976746 >> v2:172.21.15.100:3300/0 conn(0x7fd5780cbcf0 0x7fd578160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd57ce9e640 1 --2- 172.21.15.60:0/1304976746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd57815fa20 0x7fd57815bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd5777fe640 1 -- 172.21.15.60:0/1304976746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd57815fa20 msgr2=0x7fd57815bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd5777fe640 1 --2- 172.21.15.60:0/1304976746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd57815fa20 0x7fd57815bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd5777fe640 1 -- 172.21.15.60:0/1304976746 >> v2:172.21.15.100:3300/0 conn(0x7fd5780cbcf0 msgr2=0x7fd578160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd5777fe640 1 --2- 172.21.15.60:0/1304976746 >> v2:172.21.15.100:3300/0 conn(0x7fd5780cbcf0 0x7fd578160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd5777fe640 1 -- 172.21.15.60:0/1304976746 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5780cccd0 con 0x7fd578071c30 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.554+0000 7fd577fff640 1 --2- 172.21.15.60:0/1304976746 >> v2:172.21.15.100:3300/0 conn(0x7fd5780cbcf0 0x7fd578160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:10.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd5777fe640 1 --2- 172.21.15.60:0/1304976746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd578071c30 0x7fd57815f4e0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fd5780b4c50 tx=0x7fd56c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7237da759927616 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd576ffd640 1 -- 172.21.15.60:0/1304976746 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c0b1070 con 0x7fd578071c30 2024-04-25T10:16:10.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd576ffd640 1 -- 172.21.15.60:0/1304976746 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56c0ac630 con 0x7fd578071c30 2024-04-25T10:16:10.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd576ffd640 1 -- 172.21.15.60:0/1304976746 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c0a8050 con 0x7fd578071c30 2024-04-25T10:16:10.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 -- 172.21.15.60:0/1304976746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd578071c30 msgr2=0x7fd57815f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 --2- 172.21.15.60:0/1304976746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd578071c30 0x7fd57815f4e0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fd5780b4c50 tx=0x7fd56c01c520 comp rx=0 tx=0).stop 2024-04-25T10:16:10.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 -- 172.21.15.60:0/1304976746 shutdown_connections 2024-04-25T10:16:10.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 --2- 172.21.15.60:0/1304976746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd57815fa20 0x7fd57815bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 --2- 172.21.15.60:0/1304976746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd578071c30 0x7fd57815f4e0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 --2- 172.21.15.60:0/1304976746 >> v2:172.21.15.100:3300/0 conn(0x7fd5780cbcf0 0x7fd578160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 -- 172.21.15.60:0/1304976746 >> 172.21.15.60:0/1304976746 conn(0x7fd5780670b0 msgr2=0x7fd57806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 -- 172.21.15.60:0/1304976746 shutdown_connections 2024-04-25T10:16:10.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 -- 172.21.15.60:0/1304976746 wait complete. 2024-04-25T10:16:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 Processor -- start 2024-04-25T10:16:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 -- start start 2024-04-25T10:16:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd578071c30 0x7fd578160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5780cbcf0 0x7fd57813d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd57815fa20 0x7fd57813f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780ce5c0 con 0x7fd578071c30 2024-04-25T10:16:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780720b0 con 0x7fd57815fa20 2024-04-25T10:16:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd57e901640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780ce740 con 0x7fd5780cbcf0 2024-04-25T10:16:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd577fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd578071c30 0x7fd578160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd577fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd578071c30 0x7fd578160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35190/0 (socket says 172.21.15.60:35190) 2024-04-25T10:16:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.558+0000 7fd577fff640 1 -- 172.21.15.60:0/2329427279 learned_addr learned my addr 172.21.15.60:0/2329427279 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd5777fe640 1 --2- 172.21.15.60:0/2329427279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5780cbcf0 0x7fd57813d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd57ce9e640 1 --2- 172.21.15.60:0/2329427279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd57815fa20 0x7fd57813f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd577fff640 1 -- 172.21.15.60:0/2329427279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5780cbcf0 msgr2=0x7fd57813d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd577fff640 1 --2- 172.21.15.60:0/2329427279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5780cbcf0 0x7fd57813d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd577fff640 1 -- 172.21.15.60:0/2329427279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd57815fa20 msgr2=0x7fd57813f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd577fff640 1 --2- 172.21.15.60:0/2329427279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd57815fa20 0x7fd57813f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd577fff640 1 -- 172.21.15.60:0/2329427279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57813fcb0 con 0x7fd578071c30 2024-04-25T10:16:10.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd577fff640 1 --2- 172.21.15.60:0/2329427279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd578071c30 0x7fd578160fe0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fd56800c7f0 tx=0x7fd56800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd574ff9640 1 -- 172.21.15.60:0/2329427279 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568015070 con 0x7fd578071c30 2024-04-25T10:16:10.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd574ff9640 1 -- 172.21.15.60:0/2329427279 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd568007820 con 0x7fd578071c30 2024-04-25T10:16:10.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd574ff9640 1 -- 172.21.15.60:0/2329427279 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568007c10 con 0x7fd578071c30 2024-04-25T10:16:10.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd57e901640 1 -- 172.21.15.60:0/2329427279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57813ff90 con 0x7fd578071c30 2024-04-25T10:16:10.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd57e901640 1 -- 172.21.15.60:0/2329427279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd578140550 con 0x7fd578071c30 2024-04-25T10:16:10.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd57e901640 1 -- 172.21.15.60:0/2329427279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57806bba0 con 0x7fd578071c30 2024-04-25T10:16:10.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd574ff9640 1 -- 172.21.15.60:0/2329427279 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd568018070 con 0x7fd578071c30 2024-04-25T10:16:10.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd574ff9640 1 --2- 172.21.15.60:0/2329427279 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd564066f60 0x7fd564069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd574ff9640 1 -- 172.21.15.60:0/2329427279 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd568085190 con 0x7fd578071c30 2024-04-25T10:16:10.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd5777fe640 1 --2- 172.21.15.60:0/2329427279 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd564066f60 0x7fd564069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.562+0000 7fd5777fe640 1 --2- 172.21.15.60:0/2329427279 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd564066f60 0x7fd564069420 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0ae000 tx=0x7fd56c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.566+0000 7fd574ff9640 1 -- 172.21.15.60:0/2329427279 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd568056110 con 0x7fd578071c30 2024-04-25T10:16:10.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:10.762+0000 7fd57e901640 1 -- 172.21.15.60:0/2329427279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd578074770 con 0x7fd578071c30 2024-04-25T10:16:11.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.406+0000 7fd574ff9640 1 -- 172.21.15.60:0/2329427279 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v93) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd568055a10 con 0x7fd578071c30 2024-04-25T10:16:11.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.410+0000 7fd57e901640 1 -- 172.21.15.60:0/2329427279 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd564066f60 msgr2=0x7fd564069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.410+0000 7fd57e901640 1 --2- 172.21.15.60:0/2329427279 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd564066f60 0x7fd564069420 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0ae000 tx=0x7fd56c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:11.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.410+0000 7fd57e901640 1 -- 172.21.15.60:0/2329427279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd578071c30 msgr2=0x7fd578160fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.410+0000 7fd57e901640 1 --2- 172.21.15.60:0/2329427279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd578071c30 0x7fd578160fe0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fd56800c7f0 tx=0x7fd56800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.410+0000 7fd57e901640 1 -- 172.21.15.60:0/2329427279 shutdown_connections 2024-04-25T10:16:11.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.410+0000 7fd57e901640 1 --2- 172.21.15.60:0/2329427279 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd564066f60 0x7fd564069420 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.410+0000 7fd57e901640 1 --2- 172.21.15.60:0/2329427279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd57815fa20 0x7fd57813f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.410+0000 7fd57e901640 1 --2- 172.21.15.60:0/2329427279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5780cbcf0 0x7fd57813d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.410+0000 7fd57e901640 1 --2- 172.21.15.60:0/2329427279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd578071c30 0x7fd578160fe0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.410+0000 7fd57e901640 1 -- 172.21.15.60:0/2329427279 >> 172.21.15.60:0/2329427279 conn(0x7fd5780670b0 msgr2=0x7fd57815e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:11.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.410+0000 7fd57e901640 1 -- 172.21.15.60:0/2329427279 shutdown_connections 2024-04-25T10:16:11.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.410+0000 7fd57e901640 1 -- 172.21.15.60:0/2329427279 wait complete. 2024-04-25T10:16:11.437 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:16:11.437 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:11.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa9435bb640 1 Processor -- start 2024-04-25T10:16:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa9435bb640 1 -- start start 2024-04-25T10:16:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa9435bb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa93c0b6c10 0x7fa93c0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa9435bb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93c06d200 con 0x7fa93c15deb0 2024-04-25T10:16:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa9435bb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93c0cefe0 con 0x7fa93c0d0a50 2024-04-25T10:16:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa9435bb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa93c0cf2c0 con 0x7fa93c0b6c10 2024-04-25T10:16:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa941357640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa93c15deb0 0x7fa93c0d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48650/0 (socket says 172.21.15.60:48650) 2024-04-25T10:16:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa941357640 1 -- 172.21.15.60:0/4081540349 learned_addr learned my addr 172.21.15.60:0/4081540349 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa941b58640 1 --2- 172.21.15.60:0/4081540349 >> v2:172.21.15.100:3300/0 conn(0x7fa93c0b6c10 0x7fa93c0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa933fff640 1 -- 172.21.15.60:0/4081540349 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134699656 0 0) 0x7fa93c06d200 con 0x7fa93c15deb0 2024-04-25T10:16:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa933fff640 1 -- 172.21.15.60:0/4081540349 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa928003530 con 0x7fa93c15deb0 2024-04-25T10:16:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa933fff640 1 -- 172.21.15.60:0/4081540349 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 491363087 0 0) 0x7fa928003530 con 0x7fa93c15deb0 2024-04-25T10:16:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa933fff640 1 -- 172.21.15.60:0/4081540349 >> v2:172.21.15.100:3300/0 conn(0x7fa93c0b6c10 msgr2=0x7fa93c0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa933fff640 1 --2- 172.21.15.60:0/4081540349 >> v2:172.21.15.100:3300/0 conn(0x7fa93c0b6c10 0x7fa93c0ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa933fff640 1 -- 172.21.15.60:0/4081540349 >> v1:172.21.15.100:6789/0 conn(0x7fa93c0d0a50 legacy=0x7fa93c0ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa933fff640 1 -- 172.21.15.60:0/4081540349 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c0d1d70 con 0x7fa93c15deb0 2024-04-25T10:16:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa933fff640 1 -- 172.21.15.60:0/4081540349 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fa938001680 con 0x7fa93c15deb0 2024-04-25T10:16:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa933fff640 1 -- 172.21.15.60:0/4081540349 >> v1:172.21.15.60:6789/0 conn(0x7fa93c15deb0 legacy=0x7fa93c0d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa933fff640 1 --2- 172.21.15.60:0/4081540349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9280039c0 0x7fa928003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa933fff640 1 --2- 172.21.15.60:0/4081540349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa928004810 0x7fa928004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa933fff640 1 --2- 172.21.15.60:0/4081540349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9280051d0 0x7fa92800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa933fff640 1 -- 172.21.15.60:0/4081540349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0cf2c0 con 0x7fa9280051d0 2024-04-25T10:16:11.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa933fff640 1 -- 172.21.15.60:0/4081540349 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92800c070 con 0x7fa9280039c0 2024-04-25T10:16:11.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa933fff640 1 -- 172.21.15.60:0/4081540349 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92800c270 con 0x7fa928004810 2024-04-25T10:16:11.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa941b58640 1 --2- 172.21.15.60:0/4081540349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9280051d0 0x7fa92800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa941357640 1 --2- 172.21.15.60:0/4081540349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9280039c0 0x7fa928003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa941b58640 1 -- 172.21.15.60:0/4081540349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa928004810 msgr2=0x7fa928004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:11.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa941b58640 1 --2- 172.21.15.60:0/4081540349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa928004810 0x7fa928004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa941b58640 1 -- 172.21.15.60:0/4081540349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9280039c0 msgr2=0x7fa928003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:11.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa941b58640 1 --2- 172.21.15.60:0/4081540349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9280039c0 0x7fa928003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.558+0000 7fa941b58640 1 -- 172.21.15.60:0/4081540349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa928003530 con 0x7fa9280051d0 2024-04-25T10:16:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.562+0000 7fa941b58640 1 -- 172.21.15.60:0/4081540349 reap_dead start 2024-04-25T10:16:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.562+0000 7fa941b58640 1 --2- 172.21.15.60:0/4081540349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9280051d0 0x7fa92800b9d0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fa934009fc0 tx=0x7fa93401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5520e8fc72bdec68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.562+0000 7fa933fff640 1 -- 172.21.15.60:0/4081540349 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934009c10 con 0x7fa9280051d0 2024-04-25T10:16:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.562+0000 7fa933fff640 1 -- 172.21.15.60:0/4081540349 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9340a5d10 con 0x7fa9280051d0 2024-04-25T10:16:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.562+0000 7fa933fff640 1 -- 172.21.15.60:0/4081540349 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934003be0 con 0x7fa9280051d0 2024-04-25T10:16:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.562+0000 7fa9435bb640 1 -- 172.21.15.60:0/4081540349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9280051d0 msgr2=0x7fa92800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.562+0000 7fa9435bb640 1 --2- 172.21.15.60:0/4081540349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9280051d0 0x7fa92800b9d0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fa934009fc0 tx=0x7fa93401ccc0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.562+0000 7fa9435bb640 1 -- 172.21.15.60:0/4081540349 shutdown_connections 2024-04-25T10:16:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.562+0000 7fa9435bb640 1 --2- 172.21.15.60:0/4081540349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9280051d0 0x7fa92800b9d0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.562+0000 7fa9435bb640 1 -- 172.21.15.60:0/4081540349 >> 172.21.15.60:0/4081540349 conn(0x7fa93c06c090 msgr2=0x7fa93c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.562+0000 7fa9435bb640 1 -- 172.21.15.60:0/4081540349 shutdown_connections 2024-04-25T10:16:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.562+0000 7fa9435bb640 1 -- 172.21.15.60:0/4081540349 wait complete. 2024-04-25T10:16:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.562+0000 7fa9435bb640 1 Processor -- start 2024-04-25T10:16:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.562+0000 7fa9435bb640 1 -- start start 2024-04-25T10:16:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa9435bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9280051d0 0x7fa93c154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa9435bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa93c0caa00 0x7fa93c1f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa9435bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa93c0d2570 0x7fa93c0d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa9435bb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0d0340 con 0x7fa9280051d0 2024-04-25T10:16:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa9435bb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0d2330 con 0x7fa93c0d2570 2024-04-25T10:16:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa9435bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0d04c0 con 0x7fa93c0caa00 2024-04-25T10:16:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa941357640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9280051d0 0x7fa93c154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa941357640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9280051d0 0x7fa93c154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35220/0 (socket says 172.21.15.60:35220) 2024-04-25T10:16:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa941357640 1 -- 172.21.15.60:0/2462121842 learned_addr learned my addr 172.21.15.60:0/2462121842 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa941357640 1 -- 172.21.15.60:0/2462121842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa93c0caa00 msgr2=0x7fa93c1f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa941b58640 1 --2- 172.21.15.60:0/2462121842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa93c0d2570 0x7fa93c0d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa941357640 1 --2- 172.21.15.60:0/2462121842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa93c0caa00 0x7fa93c1f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa941357640 1 -- 172.21.15.60:0/2462121842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa93c0d2570 msgr2=0x7fa93c0d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa941357640 1 --2- 172.21.15.60:0/2462121842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa93c0d2570 0x7fa93c0d29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa941357640 1 -- 172.21.15.60:0/2462121842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c0caf50 con 0x7fa9280051d0 2024-04-25T10:16:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa941357640 1 --2- 172.21.15.60:0/2462121842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9280051d0 0x7fa93c154a20 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fa93800e780 tx=0x7fa93800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa931ffb640 1 -- 172.21.15.60:0/2462121842 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93800ee90 con 0x7fa9280051d0 2024-04-25T10:16:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa931ffb640 1 -- 172.21.15.60:0/2462121842 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa938018070 con 0x7fa9280051d0 2024-04-25T10:16:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa9435bb640 1 -- 172.21.15.60:0/2462121842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93c0cb230 con 0x7fa9280051d0 2024-04-25T10:16:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa931ffb640 1 -- 172.21.15.60:0/2462121842 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93800cb10 con 0x7fa9280051d0 2024-04-25T10:16:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa9435bb640 1 -- 172.21.15.60:0/2462121842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa93c0cb7f0 con 0x7fa9280051d0 2024-04-25T10:16:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa9435bb640 1 -- 172.21.15.60:0/2462121842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93c074770 con 0x7fa9280051d0 2024-04-25T10:16:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa931ffb640 1 -- 172.21.15.60:0/2462121842 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa93800ccb0 con 0x7fa9280051d0 2024-04-25T10:16:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa931ffb640 1 --2- 172.21.15.60:0/2462121842 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa920067100 0x7fa9200695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa931ffb640 1 -- 172.21.15.60:0/2462121842 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa938084e50 con 0x7fa9280051d0 2024-04-25T10:16:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.566+0000 7fa940b56640 1 --2- 172.21.15.60:0/2462121842 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa920067100 0x7fa9200695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.570+0000 7fa940b56640 1 --2- 172.21.15.60:0/2462121842 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa920067100 0x7fa9200695c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa92c007fd0 tx=0x7fa92c0079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.570+0000 7fa931ffb640 1 -- 172.21.15.60:0/2462121842 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa938055ef0 con 0x7fa9280051d0 2024-04-25T10:16:11.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.766+0000 7fa9435bb640 1 -- 172.21.15.60:0/2462121842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa93c0d17a0 con 0x7fa9280051d0 2024-04-25T10:16:11.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.766+0000 7fa931ffb640 1 -- 172.21.15.60:0/2462121842 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa9380557f0 con 0x7fa9280051d0 2024-04-25T10:16:11.771 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:11.771 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2440975447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2440975447},{"type":"v1","addr":"172.21.15.100:6835","nonce":2440975447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2398080758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2398080758},{"type":"v1","addr":"172.21.15.60:6833","nonce":2398080758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2759861958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2759861958},{"type":"v1","addr":"172.21.15.100:6837","nonce":2759861958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:07.763322+0000","modified":"2024-04-25T10:16:10.765897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9381":{"gid":9381,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/722171246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":722171246},{"type":"v1","addr":"172.21.15.60:6835","nonce":722171246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:16:11.771 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 93 2024-04-25T10:16:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.766+0000 7fa9435bb640 1 -- 172.21.15.60:0/2462121842 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa920067100 msgr2=0x7fa9200695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.766+0000 7fa9435bb640 1 --2- 172.21.15.60:0/2462121842 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa920067100 0x7fa9200695c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa92c007fd0 tx=0x7fa92c0079c0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.766+0000 7fa9435bb640 1 -- 172.21.15.60:0/2462121842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9280051d0 msgr2=0x7fa93c154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.766+0000 7fa9435bb640 1 --2- 172.21.15.60:0/2462121842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9280051d0 0x7fa93c154a20 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fa93800e780 tx=0x7fa93800ec40 comp rx=0 tx=0).stop 2024-04-25T10:16:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.766+0000 7fa9435bb640 1 -- 172.21.15.60:0/2462121842 shutdown_connections 2024-04-25T10:16:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.766+0000 7fa9435bb640 1 --2- 172.21.15.60:0/2462121842 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa920067100 0x7fa9200695c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.766+0000 7fa9435bb640 1 --2- 172.21.15.60:0/2462121842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa93c0d2570 0x7fa93c0d29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.766+0000 7fa9435bb640 1 --2- 172.21.15.60:0/2462121842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa93c0caa00 0x7fa93c1f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.766+0000 7fa9435bb640 1 --2- 172.21.15.60:0/2462121842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9280051d0 0x7fa93c154a20 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.766+0000 7fa9435bb640 1 -- 172.21.15.60:0/2462121842 >> 172.21.15.60:0/2462121842 conn(0x7fa93c06c090 msgr2=0x7fa93c151ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:11.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.766+0000 7fa9435bb640 1 -- 172.21.15.60:0/2462121842 shutdown_connections 2024-04-25T10:16:11.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.766+0000 7fa9435bb640 1 -- 172.21.15.60:0/2462121842 wait complete. 2024-04-25T10:16:11.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:11.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.902+0000 7fc90c4f7640 1 Processor -- start 2024-04-25T10:16:11.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.906+0000 7fc90c4f7640 1 -- start start 2024-04-25T10:16:11.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.906+0000 7fc90c4f7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc90415a7a0 0x7fc9040cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.906+0000 7fc90c4f7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90406f130 con 0x7fc90406ec30 2024-04-25T10:16:11.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.906+0000 7fc90c4f7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90406f410 con 0x7fc9040b37e0 2024-04-25T10:16:11.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.906+0000 7fc90c4f7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9040cdaa0 con 0x7fc90415a7a0 2024-04-25T10:16:11.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.906+0000 7fc90a293640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc90415a7a0 0x7fc9040cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.906+0000 7fc909a92640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc90406ec30 0x7fc904071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48652/0 (socket says 172.21.15.60:48652) 2024-04-25T10:16:11.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.906+0000 7fc909a92640 1 -- 172.21.15.60:0/2093679391 learned_addr learned my addr 172.21.15.60:0/2093679391 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:11.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.906+0000 7fc90a293640 1 -- 172.21.15.60:0/2093679391 >> v1:172.21.15.100:6789/0 conn(0x7fc9040b37e0 legacy=0x7fc90406dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:11.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.906+0000 7fc90a293640 1 -- 172.21.15.60:0/2093679391 >> v1:172.21.15.60:6789/0 conn(0x7fc90406ec30 legacy=0x7fc904071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.906+0000 7fc90a293640 1 -- 172.21.15.60:0/2093679391 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90406d540 con 0x7fc90415a7a0 2024-04-25T10:16:11.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.906+0000 7fc90a293640 1 --2- 172.21.15.60:0/2093679391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc90415a7a0 0x7fc9040cd560 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fc9000097f0 tx=0x7fc90001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5747e0d906b1ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc909291640 1 -- 172.21.15.60:0/2093679391 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000b0070 con 0x7fc90415a7a0 2024-04-25T10:16:11.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc909291640 1 -- 172.21.15.60:0/2093679391 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9000ab600 con 0x7fc90415a7a0 2024-04-25T10:16:11.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc909291640 1 -- 172.21.15.60:0/2093679391 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000ab9d0 con 0x7fc90415a7a0 2024-04-25T10:16:11.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 -- 172.21.15.60:0/2093679391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc90415a7a0 msgr2=0x7fc9040cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 --2- 172.21.15.60:0/2093679391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc90415a7a0 0x7fc9040cd560 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fc9000097f0 tx=0x7fc90001c460 comp rx=0 tx=0).stop 2024-04-25T10:16:11.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 -- 172.21.15.60:0/2093679391 shutdown_connections 2024-04-25T10:16:11.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 --2- 172.21.15.60:0/2093679391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc90415a7a0 0x7fc9040cd560 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 -- 172.21.15.60:0/2093679391 >> 172.21.15.60:0/2093679391 conn(0x7fc9040b12d0 msgr2=0x7fc9040b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:11.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 -- 172.21.15.60:0/2093679391 shutdown_connections 2024-04-25T10:16:11.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 -- 172.21.15.60:0/2093679391 wait complete. 2024-04-25T10:16:11.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 Processor -- start 2024-04-25T10:16:11.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 -- start start 2024-04-25T10:16:11.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc90406ec30 0x7fc90415e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9040b37e0 0x7fc9041592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc90415a7a0 0x7fc904152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90406d8b0 con 0x7fc90406ec30 2024-04-25T10:16:11.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90406d2f0 con 0x7fc9040b37e0 2024-04-25T10:16:11.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040b2ee0 con 0x7fc90415a7a0 2024-04-25T10:16:11.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90a293640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc90406ec30 0x7fc90415e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90a293640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc90406ec30 0x7fc90415e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35236/0 (socket says 172.21.15.60:35236) 2024-04-25T10:16:11.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90a293640 1 -- 172.21.15.60:0/3313223650 learned_addr learned my addr 172.21.15.60:0/3313223650 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:11.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc909a92640 1 --2- 172.21.15.60:0/3313223650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9040b37e0 0x7fc9041592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90aa94640 1 --2- 172.21.15.60:0/3313223650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc90415a7a0 0x7fc904152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90a293640 1 -- 172.21.15.60:0/3313223650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc90415a7a0 msgr2=0x7fc904152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90a293640 1 --2- 172.21.15.60:0/3313223650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc90415a7a0 0x7fc904152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90a293640 1 -- 172.21.15.60:0/3313223650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9040b37e0 msgr2=0x7fc9041592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90a293640 1 --2- 172.21.15.60:0/3313223650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9040b37e0 0x7fc9041592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90a293640 1 -- 172.21.15.60:0/3313223650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc904158b00 con 0x7fc90406ec30 2024-04-25T10:16:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90a293640 1 --2- 172.21.15.60:0/3313223650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc90406ec30 0x7fc90415e4d0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fc900019070 tx=0x7fc90001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc8faffd640 1 -- 172.21.15.60:0/3313223650 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000b0050 con 0x7fc90406ec30 2024-04-25T10:16:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc8faffd640 1 -- 172.21.15.60:0/3313223650 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9000bc070 con 0x7fc90406ec30 2024-04-25T10:16:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 -- 172.21.15.60:0/3313223650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9040ce6a0 con 0x7fc90406ec30 2024-04-25T10:16:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc8faffd640 1 -- 172.21.15.60:0/3313223650 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000b5a30 con 0x7fc90406ec30 2024-04-25T10:16:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.910+0000 7fc90c4f7640 1 -- 172.21.15.60:0/3313223650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90415ea10 con 0x7fc90406ec30 2024-04-25T10:16:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.914+0000 7fc90c4f7640 1 -- 172.21.15.60:0/3313223650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc904158910 con 0x7fc90406ec30 2024-04-25T10:16:11.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.914+0000 7fc8faffd640 1 -- 172.21.15.60:0/3313223650 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc9000ab4c0 con 0x7fc90406ec30 2024-04-25T10:16:11.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.914+0000 7fc8faffd640 1 --2- 172.21.15.60:0/3313223650 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc8e8066f60 0x7fc8e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.914+0000 7fc8faffd640 1 -- 172.21.15.60:0/3313223650 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc90011f780 con 0x7fc90406ec30 2024-04-25T10:16:11.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.914+0000 7fc909a92640 1 --2- 172.21.15.60:0/3313223650 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc8e8066f60 0x7fc8e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.914+0000 7fc909a92640 1 --2- 172.21.15.60:0/3313223650 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc8e8066f60 0x7fc8e8069420 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc9041517a0 tx=0x7fc8f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:11.918+0000 7fc8faffd640 1 -- 172.21.15.60:0/3313223650 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc900016050 con 0x7fc90406ec30 2024-04-25T10:16:12.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.074+0000 7fc90c4f7640 1 -- 172.21.15.60:0/3313223650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc90406df90 con 0x7fc90406ec30 2024-04-25T10:16:12.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.078+0000 7fc8faffd640 1 -- 172.21.15.60:0/3313223650 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16639 (secure 0 0 0) 0x7fc9000f0820 con 0x7fc90406ec30 2024-04-25T10:16:12.082 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:12.082 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":110,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:16:09.041178+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:06.033387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:06.753421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:12.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.078+0000 7fc90c4f7640 1 -- 172.21.15.60:0/3313223650 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc8e8066f60 msgr2=0x7fc8e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.078+0000 7fc90c4f7640 1 --2- 172.21.15.60:0/3313223650 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc8e8066f60 0x7fc8e8069420 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc9041517a0 tx=0x7fc8f4009040 comp rx=0 tx=0).stop 2024-04-25T10:16:12.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.078+0000 7fc90c4f7640 1 -- 172.21.15.60:0/3313223650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc90406ec30 msgr2=0x7fc90415e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.078+0000 7fc90c4f7640 1 --2- 172.21.15.60:0/3313223650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc90406ec30 0x7fc90415e4d0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fc900019070 tx=0x7fc90001aca0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.078+0000 7fc90c4f7640 1 -- 172.21.15.60:0/3313223650 shutdown_connections 2024-04-25T10:16:12.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.078+0000 7fc90c4f7640 1 --2- 172.21.15.60:0/3313223650 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc8e8066f60 0x7fc8e8069420 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.078+0000 7fc90c4f7640 1 --2- 172.21.15.60:0/3313223650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc90415a7a0 0x7fc904152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.078+0000 7fc90c4f7640 1 --2- 172.21.15.60:0/3313223650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9040b37e0 0x7fc9041592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.078+0000 7fc90c4f7640 1 --2- 172.21.15.60:0/3313223650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc90406ec30 0x7fc90415e4d0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.078+0000 7fc90c4f7640 1 -- 172.21.15.60:0/3313223650 >> 172.21.15.60:0/3313223650 conn(0x7fc9040b12d0 msgr2=0x7fc90415c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.078+0000 7fc90c4f7640 1 -- 172.21.15.60:0/3313223650 shutdown_connections 2024-04-25T10:16:12.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.078+0000 7fc90c4f7640 1 -- 172.21.15.60:0/3313223650 wait complete. 2024-04-25T10:16:12.100 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:16:12.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:16:12.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.218+0000 7ffb38c5d640 1 Processor -- start 2024-04-25T10:16:12.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.218+0000 7ffb38c5d640 1 -- start start 2024-04-25T10:16:12.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb38c5d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffb34145300 0x7ffb3415bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb38c5d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffb34072d80 0x7ffb3406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb38c5d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb340b3880 con 0x7ffb3406b150 2024-04-25T10:16:12.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb38c5d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3406faf0 con 0x7ffb34072d80 2024-04-25T10:16:12.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb38c5d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb340b4cd0 con 0x7ffb34145300 2024-04-25T10:16:12.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb2bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffb34072d80 0x7ffb3406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb2bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffb34072d80 0x7ffb3406ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35240/0 (socket says 172.21.15.60:35240) 2024-04-25T10:16:12.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb2bfff640 1 -- 172.21.15.60:0/978049286 learned_addr learned my addr 172.21.15.60:0/978049286 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:12.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb3259c640 1 --2- 172.21.15.60:0/978049286 >> v2:172.21.15.100:3300/0 conn(0x7ffb34145300 0x7ffb3415bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb2bfff640 1 -- 172.21.15.60:0/978049286 >> v2:172.21.15.100:3300/0 conn(0x7ffb34145300 msgr2=0x7ffb3415bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:12.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb2bfff640 1 --2- 172.21.15.60:0/978049286 >> v2:172.21.15.100:3300/0 conn(0x7ffb34145300 0x7ffb3415bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb2bfff640 1 -- 172.21.15.60:0/978049286 >> v1:172.21.15.100:6789/0 conn(0x7ffb3406b150 legacy=0x7ffb3406ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:12.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb2bfff640 1 -- 172.21.15.60:0/978049286 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb340b4f70 con 0x7ffb34072d80 2024-04-25T10:16:12.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb3259c640 1 --2- 172.21.15.60:0/978049286 >> v2:172.21.15.100:3300/0 conn(0x7ffb34145300 0x7ffb3415bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:12.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb2bfff640 1 --2- 172.21.15.60:0/978049286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb34072d80 0x7ffb3406ac10 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009870 tx=0x7ffb1c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683135834c9d33fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb31d9b640 1 -- 172.21.15.60:0/978049286 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c0b1070 con 0x7ffb34072d80 2024-04-25T10:16:12.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb31d9b640 1 -- 172.21.15.60:0/978049286 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1c0ac600 con 0x7ffb34072d80 2024-04-25T10:16:12.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb31d9b640 1 -- 172.21.15.60:0/978049286 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c0ac9d0 con 0x7ffb34072d80 2024-04-25T10:16:12.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb38c5d640 1 -- 172.21.15.60:0/978049286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb34072d80 msgr2=0x7ffb3406ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb38c5d640 1 --2- 172.21.15.60:0/978049286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb34072d80 0x7ffb3406ac10 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009870 tx=0x7ffb1c01c4b0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb38c5d640 1 -- 172.21.15.60:0/978049286 shutdown_connections 2024-04-25T10:16:12.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb38c5d640 1 --2- 172.21.15.60:0/978049286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb34072d80 0x7ffb3406ac10 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.222+0000 7ffb38c5d640 1 --2- 172.21.15.60:0/978049286 >> v2:172.21.15.100:3300/0 conn(0x7ffb34145300 0x7ffb3415bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb38c5d640 1 -- 172.21.15.60:0/978049286 >> 172.21.15.60:0/978049286 conn(0x7ffb340b2360 msgr2=0x7ffb340b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb38c5d640 1 -- 172.21.15.60:0/978049286 shutdown_connections 2024-04-25T10:16:12.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb38c5d640 1 -- 172.21.15.60:0/978049286 wait complete. 2024-04-25T10:16:12.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb38c5d640 1 Processor -- start 2024-04-25T10:16:12.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb38c5d640 1 -- start start 2024-04-25T10:16:12.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb38c5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb3406b150 0x7ffb3413c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb38c5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb34072d80 0x7ffb340b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb38c5d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb34145300 0x7ffb3413ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb38c5d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340731e0 con 0x7ffb3406b150 2024-04-25T10:16:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb38c5d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3406b950 con 0x7ffb34072d80 2024-04-25T10:16:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb38c5d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3415d030 con 0x7ffb34145300 2024-04-25T10:16:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb3259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb3406b150 0x7ffb3413c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb3259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb3406b150 0x7ffb3413c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35256/0 (socket says 172.21.15.60:35256) 2024-04-25T10:16:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb3259c640 1 -- 172.21.15.60:0/2250096891 learned_addr learned my addr 172.21.15.60:0/2250096891 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb2bfff640 1 --2- 172.21.15.60:0/2250096891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb34072d80 0x7ffb340b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb32d9d640 1 --2- 172.21.15.60:0/2250096891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb34145300 0x7ffb3413ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb3259c640 1 -- 172.21.15.60:0/2250096891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb34145300 msgr2=0x7ffb3413ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb3259c640 1 --2- 172.21.15.60:0/2250096891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb34145300 0x7ffb3413ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb3259c640 1 -- 172.21.15.60:0/2250096891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb34072d80 msgr2=0x7ffb340b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb3259c640 1 --2- 172.21.15.60:0/2250096891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb34072d80 0x7ffb340b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb3259c640 1 -- 172.21.15.60:0/2250096891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3415d280 con 0x7ffb3406b150 2024-04-25T10:16:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb3259c640 1 --2- 172.21.15.60:0/2250096891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb3406b150 0x7ffb3413c280 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400c7f0 tx=0x7ffb2400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb2affd640 1 -- 172.21.15.60:0/2250096891 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24015070 con 0x7ffb3406b150 2024-04-25T10:16:12.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb2affd640 1 -- 172.21.15.60:0/2250096891 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb24007820 con 0x7ffb3406b150 2024-04-25T10:16:12.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb38c5d640 1 -- 172.21.15.60:0/2250096891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3415d560 con 0x7ffb3406b150 2024-04-25T10:16:12.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb2affd640 1 -- 172.21.15.60:0/2250096891 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24007c10 con 0x7ffb3406b150 2024-04-25T10:16:12.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.226+0000 7ffb38c5d640 1 -- 172.21.15.60:0/2250096891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3415db20 con 0x7ffb3406b150 2024-04-25T10:16:12.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.230+0000 7ffb38c5d640 1 -- 172.21.15.60:0/2250096891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3406ccb0 con 0x7ffb3406b150 2024-04-25T10:16:12.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.230+0000 7ffb2affd640 1 -- 172.21.15.60:0/2250096891 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb24018070 con 0x7ffb3406b150 2024-04-25T10:16:12.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.230+0000 7ffb2affd640 1 --2- 172.21.15.60:0/2250096891 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb14066f60 0x7ffb14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.230+0000 7ffb2affd640 1 -- 172.21.15.60:0/2250096891 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb24085140 con 0x7ffb3406b150 2024-04-25T10:16:12.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.230+0000 7ffb2bfff640 1 --2- 172.21.15.60:0/2250096891 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb14066f60 0x7ffb14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.230+0000 7ffb2bfff640 1 --2- 172.21.15.60:0/2250096891 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb14066f60 0x7ffb14069420 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0ae000 tx=0x7ffb1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.234+0000 7ffb2affd640 1 -- 172.21.15.60:0/2250096891 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb24056150 con 0x7ffb3406b150 2024-04-25T10:16:12.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.390+0000 7ffb38c5d640 1 -- 172.21.15.60:0/2250096891 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffb3413f200 con 0x7ffb14066f60 2024-04-25T10:16:12.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.410+0000 7ffb2affd640 1 -- 172.21.15.60:0/2250096891 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248887 (secure 0 0 0) 0x7ffb3413f200 con 0x7ffb14066f60 2024-04-25T10:16:12.418 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:12.420 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:16:12.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.414+0000 7ffb38c5d640 1 -- 172.21.15.60:0/2250096891 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb14066f60 msgr2=0x7ffb14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.414+0000 7ffb38c5d640 1 --2- 172.21.15.60:0/2250096891 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb14066f60 0x7ffb14069420 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0ae000 tx=0x7ffb1c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:12.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.414+0000 7ffb38c5d640 1 -- 172.21.15.60:0/2250096891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb3406b150 msgr2=0x7ffb3413c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.414+0000 7ffb38c5d640 1 --2- 172.21.15.60:0/2250096891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb3406b150 0x7ffb3413c280 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400c7f0 tx=0x7ffb2400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.414+0000 7ffb38c5d640 1 -- 172.21.15.60:0/2250096891 shutdown_connections 2024-04-25T10:16:12.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.414+0000 7ffb38c5d640 1 --2- 172.21.15.60:0/2250096891 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb14066f60 0x7ffb14069420 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.414+0000 7ffb38c5d640 1 --2- 172.21.15.60:0/2250096891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb34145300 0x7ffb3413ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.414+0000 7ffb38c5d640 1 --2- 172.21.15.60:0/2250096891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb34072d80 0x7ffb340b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.414+0000 7ffb38c5d640 1 --2- 172.21.15.60:0/2250096891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb3406b150 0x7ffb3413c280 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.414+0000 7ffb38c5d640 1 -- 172.21.15.60:0/2250096891 >> 172.21.15.60:0/2250096891 conn(0x7ffb340b2360 msgr2=0x7ffb3413f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.414+0000 7ffb38c5d640 1 -- 172.21.15.60:0/2250096891 shutdown_connections 2024-04-25T10:16:12.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.414+0000 7ffb38c5d640 1 -- 172.21.15.60:0/2250096891 wait complete. 2024-04-25T10:16:12.437 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-04-25T10:16:10.615234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":277620,"kb_used_data":11520,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754697100,"statfs":{"total":773094113280,"available":772809830400,"internally_reserved":0,"allocated":11796480,"data_stored":5748646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.237341"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.405662+0000","last_change":"2024-04-25T10:16:07.405662+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.405662+0000","last_undegraded":"2024-04-25T10:16:07.405662+0000","last_fullsized":"2024-04-25T10:16:07.405662+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.077829+0000","last_change":"2024-04-25T10:16:08.065994+0000","last_active":"2024-04-25T10:16:08.077829+0000","last_peered":"2024-04-25T10:16:08.077829+0000","last_clean":"2024-04-25T10:16:08.077829+0000","last_became_active":"2024-04-25T10:16:08.064520+0000","last_became_peered":"2024-04-25T10:16:08.064520+0000","last_unstale":"2024-04-25T10:16:08.077829+0000","last_undegraded":"2024-04-25T10:16:08.077829+0000","last_fullsized":"2024-04-25T10:16:08.077829+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:54.439251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.096803+0000","last_change":"2024-04-25T10:16:08.089945+0000","last_active":"2024-04-25T10:16:08.096803+0000","last_peered":"2024-04-25T10:16:08.096803+0000","last_clean":"2024-04-25T10:16:08.096803+0000","last_became_active":"2024-04-25T10:16:08.086425+0000","last_became_peered":"2024-04-25T10:16:08.086425+0000","last_unstale":"2024-04-25T10:16:08.096803+0000","last_undegraded":"2024-04-25T10:16:08.096803+0000","last_fullsized":"2024-04-25T10:16:08.096803+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:40.325134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.416450+0000","last_change":"2024-04-25T10:16:07.416450+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.416450+0000","last_undegraded":"2024-04-25T10:16:07.416450+0000","last_fullsized":"2024-04-25T10:16:07.416450+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.393521+0000","last_change":"2024-04-25T10:16:07.393521+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.393521+0000","last_undegraded":"2024-04-25T10:16:07.393521+0000","last_fullsized":"2024-04-25T10:16:07.393521+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.392536+0000","last_change":"2024-04-25T10:16:07.392536+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.392536+0000","last_undegraded":"2024-04-25T10:16:07.392536+0000","last_fullsized":"2024-04-25T10:16:07.392536+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.395608+0000","last_change":"2024-04-25T10:16:07.395608+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.395608+0000","last_undegraded":"2024-04-25T10:16:07.395608+0000","last_fullsized":"2024-04-25T10:16:07.395608+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.089390+0000","last_change":"2024-04-25T10:16:08.083106+0000","last_active":"2024-04-25T10:16:08.089390+0000","last_peered":"2024-04-25T10:16:08.089390+0000","last_clean":"2024-04-25T10:16:08.089390+0000","last_became_active":"2024-04-25T10:16:08.082341+0000","last_became_peered":"2024-04-25T10:16:08.082341+0000","last_unstale":"2024-04-25T10:16:08.089390+0000","last_undegraded":"2024-04-25T10:16:08.089390+0000","last_fullsized":"2024-04-25T10:16:08.089390+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:05.885118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.405081+0000","last_change":"2024-04-25T10:16:07.405081+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.405081+0000","last_undegraded":"2024-04-25T10:16:07.405081+0000","last_fullsized":"2024-04-25T10:16:07.405081+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.411363+0000","last_change":"2024-04-25T10:16:07.411363+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.411363+0000","last_undegraded":"2024-04-25T10:16:07.411363+0000","last_fullsized":"2024-04-25T10:16:07.411363+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.081997+0000","last_change":"2024-04-25T10:16:08.072100+0000","last_active":"2024-04-25T10:16:08.081997+0000","last_peered":"2024-04-25T10:16:08.081997+0000","last_clean":"2024-04-25T10:16:08.081997+0000","last_became_active":"2024-04-25T10:16:08.071642+0000","last_became_peered":"2024-04-25T10:16:08.071642+0000","last_unstale":"2024-04-25T10:16:08.081997+0000","last_undegraded":"2024-04-25T10:16:08.081997+0000","last_fullsized":"2024-04-25T10:16:08.081997+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:08.872501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.412558+0000","last_change":"2024-04-25T10:16:07.412558+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.412558+0000","last_undegraded":"2024-04-25T10:16:07.412558+0000","last_fullsized":"2024-04-25T10:16:07.412558+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.096806+0000","last_change":"2024-04-25T10:16:08.091187+0000","last_active":"2024-04-25T10:16:08.096806+0000","last_peered":"2024-04-25T10:16:08.096806+0000","last_clean":"2024-04-25T10:16:08.096806+0000","last_became_active":"2024-04-25T10:16:08.090586+0000","last_became_peered":"2024-04-25T10:16:08.090586+0000","last_unstale":"2024-04-25T10:16:08.096806+0000","last_undegraded":"2024-04-25T10:16:08.096806+0000","last_fullsized":"2024-04-25T10:16:08.096806+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:57.403738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.089839+0000","last_change":"2024-04-25T10:16:08.084680+0000","last_active":"2024-04-25T10:16:08.089839+0000","last_peered":"2024-04-25T10:16:08.089839+0000","last_clean":"2024-04-25T10:16:08.089839+0000","last_became_active":"2024-04-25T10:16:08.082666+0000","last_became_peered":"2024-04-25T10:16:08.082666+0000","last_unstale":"2024-04-25T10:16:08.089839+0000","last_undegraded":"2024-04-25T10:16:08.089839+0000","last_fullsized":"2024-04-25T10:16:08.089839+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:40.403842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.390931+0000","last_change":"2024-04-25T10:16:07.390931+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.390931+0000","last_undegraded":"2024-04-25T10:16:07.390931+0000","last_fullsized":"2024-04-25T10:16:07.390931+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.406375+0000","last_change":"2024-04-25T10:16:07.406375+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.406375+0000","last_undegraded":"2024-04-25T10:16:07.406375+0000","last_fullsized":"2024-04-25T10:16:07.406375+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.082240+0000","last_change":"2024-04-25T10:16:08.070863+0000","last_active":"2024-04-25T10:16:08.082240+0000","last_peered":"2024-04-25T10:16:08.082240+0000","last_clean":"2024-04-25T10:16:08.082240+0000","last_became_active":"2024-04-25T10:16:08.070241+0000","last_became_peered":"2024-04-25T10:16:08.070241+0000","last_unstale":"2024-04-25T10:16:08.082240+0000","last_undegraded":"2024-04-25T10:16:08.082240+0000","last_fullsized":"2024-04-25T10:16:08.082240+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:59.145085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.416928+0000","last_change":"2024-04-25T10:16:07.416928+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.416928+0000","last_undegraded":"2024-04-25T10:16:07.416928+0000","last_fullsized":"2024-04-25T10:16:07.416928+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.096678+0000","last_change":"2024-04-25T10:16:08.086962+0000","last_active":"2024-04-25T10:16:08.096678+0000","last_peered":"2024-04-25T10:16:08.096678+0000","last_clean":"2024-04-25T10:16:08.096678+0000","last_became_active":"2024-04-25T10:16:08.085446+0000","last_became_peered":"2024-04-25T10:16:08.085446+0000","last_unstale":"2024-04-25T10:16:08.096678+0000","last_undegraded":"2024-04-25T10:16:08.096678+0000","last_fullsized":"2024-04-25T10:16:08.096678+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:56.557720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.081622+0000","last_change":"2024-04-25T10:16:08.067883+0000","last_active":"2024-04-25T10:16:08.081622+0000","last_peered":"2024-04-25T10:16:08.081622+0000","last_clean":"2024-04-25T10:16:08.081622+0000","last_became_active":"2024-04-25T10:16:08.067127+0000","last_became_peered":"2024-04-25T10:16:08.067127+0000","last_unstale":"2024-04-25T10:16:08.081622+0000","last_undegraded":"2024-04-25T10:16:08.081622+0000","last_fullsized":"2024-04-25T10:16:08.081622+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:38.208647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.081647+0000","last_change":"2024-04-25T10:16:08.069858+0000","last_active":"2024-04-25T10:16:08.081647+0000","last_peered":"2024-04-25T10:16:08.081647+0000","last_clean":"2024-04-25T10:16:08.081647+0000","last_became_active":"2024-04-25T10:16:08.069397+0000","last_became_peered":"2024-04-25T10:16:08.069397+0000","last_unstale":"2024-04-25T10:16:08.081647+0000","last_undegraded":"2024-04-25T10:16:08.081647+0000","last_fullsized":"2024-04-25T10:16:08.081647+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:27.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.407666+0000","last_change":"2024-04-25T10:16:07.407666+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.407666+0000","last_undegraded":"2024-04-25T10:16:07.407666+0000","last_fullsized":"2024-04-25T10:16:07.407666+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.083408+0000","last_change":"2024-04-25T10:16:08.075287+0000","last_active":"2024-04-25T10:16:08.083408+0000","last_peered":"2024-04-25T10:16:08.083408+0000","last_clean":"2024-04-25T10:16:08.083408+0000","last_became_active":"2024-04-25T10:16:08.071404+0000","last_became_peered":"2024-04-25T10:16:08.071404+0000","last_unstale":"2024-04-25T10:16:08.083408+0000","last_undegraded":"2024-04-25T10:16:08.083408+0000","last_fullsized":"2024-04-25T10:16:08.083408+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:43.193925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.396830+0000","last_change":"2024-04-25T10:16:07.396830+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.396830+0000","last_undegraded":"2024-04-25T10:16:07.396830+0000","last_fullsized":"2024-04-25T10:16:07.396830+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.082212+0000","last_change":"2024-04-25T10:16:08.070333+0000","last_active":"2024-04-25T10:16:08.082212+0000","last_peered":"2024-04-25T10:16:08.082212+0000","last_clean":"2024-04-25T10:16:08.082212+0000","last_became_active":"2024-04-25T10:16:08.069979+0000","last_became_peered":"2024-04-25T10:16:08.069979+0000","last_unstale":"2024-04-25T10:16:08.082212+0000","last_undegraded":"2024-04-25T10:16:08.082212+0000","last_fullsized":"2024-04-25T10:16:08.082212+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:45.689885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.395724+0000","last_change":"2024-04-25T10:16:07.395724+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.395724+0000","last_undegraded":"2024-04-25T10:16:07.395724+0000","last_fullsized":"2024-04-25T10:16:07.395724+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.391312+0000","last_change":"2024-04-25T10:16:07.391312+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.391312+0000","last_undegraded":"2024-04-25T10:16:07.391312+0000","last_fullsized":"2024-04-25T10:16:07.391312+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.405094+0000","last_change":"2024-04-25T10:16:07.405094+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.405094+0000","last_undegraded":"2024-04-25T10:16:07.405094+0000","last_fullsized":"2024-04-25T10:16:07.405094+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.078429+0000","last_change":"2024-04-25T10:16:08.066140+0000","last_active":"2024-04-25T10:16:08.078429+0000","last_peered":"2024-04-25T10:16:08.078429+0000","last_clean":"2024-04-25T10:16:08.078429+0000","last_became_active":"2024-04-25T10:16:08.065191+0000","last_became_peered":"2024-04-25T10:16:08.065191+0000","last_unstale":"2024-04-25T10:16:08.078429+0000","last_undegraded":"2024-04-25T10:16:08.078429+0000","last_fullsized":"2024-04-25T10:16:08.078429+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:27.960973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.399233+0000","last_change":"2024-04-25T10:16:07.399233+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.399233+0000","last_undegraded":"2024-04-25T10:16:07.399233+0000","last_fullsized":"2024-04-25T10:16:07.399233+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.409280+0000","last_change":"2024-04-25T10:16:07.409280+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.409280+0000","last_undegraded":"2024-04-25T10:16:07.409280+0000","last_fullsized":"2024-04-25T10:16:07.409280+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.391233+0000","last_change":"2024-04-25T10:16:07.391233+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.391233+0000","last_undegraded":"2024-04-25T10:16:07.391233+0000","last_fullsized":"2024-04-25T10:16:07.391233+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.390942+0000","last_change":"2024-04-25T10:16:07.390942+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.390942+0000","last_undegraded":"2024-04-25T10:16:07.390942+0000","last_fullsized":"2024-04-25T10:16:07.390942+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.420544+0000","last_change":"2024-04-25T10:16:07.412642+0000","last_active":"2024-04-25T10:16:07.420544+0000","last_peered":"2024-04-25T10:16:07.420544+0000","last_clean":"2024-04-25T10:16:07.420544+0000","last_became_active":"2024-04-25T10:16:07.412025+0000","last_became_peered":"2024-04-25T10:16:07.412025+0000","last_unstale":"2024-04-25T10:16:07.420544+0000","last_undegraded":"2024-04-25T10:16:07.420544+0000","last_fullsized":"2024-04-25T10:16:07.420544+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.277969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.412403+0000","last_change":"2024-04-25T10:16:07.412403+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.412403+0000","last_undegraded":"2024-04-25T10:16:07.412403+0000","last_fullsized":"2024-04-25T10:16:07.412403+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.415355+0000","last_change":"2024-04-25T10:16:07.413012+0000","last_active":"2024-04-25T10:16:08.415355+0000","last_peered":"2024-04-25T10:16:08.415355+0000","last_clean":"2024-04-25T10:16:08.415355+0000","last_became_active":"2024-04-25T10:16:07.412205+0000","last_became_peered":"2024-04-25T10:16:07.412205+0000","last_unstale":"2024-04-25T10:16:08.415355+0000","last_undegraded":"2024-04-25T10:16:08.415355+0000","last_fullsized":"2024-04-25T10:16:08.415355+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:54.134548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.391256+0000","last_change":"2024-04-25T10:16:07.391256+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.391256+0000","last_undegraded":"2024-04-25T10:16:07.391256+0000","last_fullsized":"2024-04-25T10:16:07.391256+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.423329+0000","last_change":"2024-04-25T10:16:07.422712+0000","last_active":"2024-04-25T10:16:07.423329+0000","last_peered":"2024-04-25T10:16:07.423329+0000","last_clean":"2024-04-25T10:16:07.423329+0000","last_became_active":"2024-04-25T10:16:07.422427+0000","last_became_peered":"2024-04-25T10:16:07.422427+0000","last_unstale":"2024-04-25T10:16:07.423329+0000","last_undegraded":"2024-04-25T10:16:07.423329+0000","last_fullsized":"2024-04-25T10:16:07.423329+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:24.548285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.096843+0000","last_change":"2024-04-25T10:16:08.090394+0000","last_active":"2024-04-25T10:16:08.096843+0000","last_peered":"2024-04-25T10:16:08.096843+0000","last_clean":"2024-04-25T10:16:08.096843+0000","last_became_active":"2024-04-25T10:16:08.090054+0000","last_became_peered":"2024-04-25T10:16:08.090054+0000","last_unstale":"2024-04-25T10:16:08.096843+0000","last_undegraded":"2024-04-25T10:16:08.096843+0000","last_fullsized":"2024-04-25T10:16:08.096843+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:32.765726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.420609+0000","last_change":"2024-04-25T10:16:07.418521+0000","last_active":"2024-04-25T10:16:07.420609+0000","last_peered":"2024-04-25T10:16:07.420609+0000","last_clean":"2024-04-25T10:16:07.420609+0000","last_became_active":"2024-04-25T10:16:07.418004+0000","last_became_peered":"2024-04-25T10:16:07.418004+0000","last_unstale":"2024-04-25T10:16:07.420609+0000","last_undegraded":"2024-04-25T10:16:07.420609+0000","last_fullsized":"2024-04-25T10:16:07.420609+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:24.954684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.395168+0000","last_change":"2024-04-25T10:16:07.395168+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.395168+0000","last_undegraded":"2024-04-25T10:16:07.395168+0000","last_fullsized":"2024-04-25T10:16:07.395168+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.426730+0000","last_change":"2024-04-25T10:16:07.423900+0000","last_active":"2024-04-25T10:16:07.426730+0000","last_peered":"2024-04-25T10:16:07.426730+0000","last_clean":"2024-04-25T10:16:07.426730+0000","last_became_active":"2024-04-25T10:16:07.423111+0000","last_became_peered":"2024-04-25T10:16:07.423111+0000","last_unstale":"2024-04-25T10:16:07.426730+0000","last_undegraded":"2024-04-25T10:16:07.426730+0000","last_fullsized":"2024-04-25T10:16:07.426730+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:36.047945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.077789+0000","last_change":"2024-04-25T10:16:08.065852+0000","last_active":"2024-04-25T10:16:08.077789+0000","last_peered":"2024-04-25T10:16:08.077789+0000","last_clean":"2024-04-25T10:16:08.077789+0000","last_became_active":"2024-04-25T10:16:08.065044+0000","last_became_peered":"2024-04-25T10:16:08.065044+0000","last_unstale":"2024-04-25T10:16:08.077789+0000","last_undegraded":"2024-04-25T10:16:08.077789+0000","last_fullsized":"2024-04-25T10:16:08.077789+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:27.548407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.421890+0000","last_change":"2024-04-25T10:16:07.419309+0000","last_active":"2024-04-25T10:16:07.421890+0000","last_peered":"2024-04-25T10:16:07.421890+0000","last_clean":"2024-04-25T10:16:07.421890+0000","last_became_active":"2024-04-25T10:16:07.419014+0000","last_became_peered":"2024-04-25T10:16:07.419014+0000","last_unstale":"2024-04-25T10:16:07.421890+0000","last_undegraded":"2024-04-25T10:16:07.421890+0000","last_fullsized":"2024-04-25T10:16:07.421890+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:55.572136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.405358+0000","last_change":"2024-04-25T10:16:07.405358+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.405358+0000","last_undegraded":"2024-04-25T10:16:07.405358+0000","last_fullsized":"2024-04-25T10:16:07.405358+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.052148+0000","last_change":"2024-04-25T10:16:07.422002+0000","last_active":"2024-04-25T10:16:08.052148+0000","last_peered":"2024-04-25T10:16:08.052148+0000","last_clean":"2024-04-25T10:16:08.052148+0000","last_became_active":"2024-04-25T10:16:07.421210+0000","last_became_peered":"2024-04-25T10:16:07.421210+0000","last_unstale":"2024-04-25T10:16:08.052148+0000","last_undegraded":"2024-04-25T10:16:08.052148+0000","last_fullsized":"2024-04-25T10:16:08.052148+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:32.378200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.396747+0000","last_change":"2024-04-25T10:16:07.396747+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.396747+0000","last_undegraded":"2024-04-25T10:16:07.396747+0000","last_fullsized":"2024-04-25T10:16:07.396747+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.429689+0000","last_change":"2024-04-25T10:16:07.429065+0000","last_active":"2024-04-25T10:16:07.429689+0000","last_peered":"2024-04-25T10:16:07.429689+0000","last_clean":"2024-04-25T10:16:07.429689+0000","last_became_active":"2024-04-25T10:16:07.428582+0000","last_became_peered":"2024-04-25T10:16:07.428582+0000","last_unstale":"2024-04-25T10:16:07.429689+0000","last_undegraded":"2024-04-25T10:16:07.429689+0000","last_fullsized":"2024-04-25T10:16:07.429689+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:41.926025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.394437+0000","last_change":"2024-04-25T10:16:07.394437+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.394437+0000","last_undegraded":"2024-04-25T10:16:07.394437+0000","last_fullsized":"2024-04-25T10:16:07.394437+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.418428+0000","last_change":"2024-04-25T10:16:07.416827+0000","last_active":"2024-04-25T10:16:07.418428+0000","last_peered":"2024-04-25T10:16:07.418428+0000","last_clean":"2024-04-25T10:16:07.418428+0000","last_became_active":"2024-04-25T10:16:07.416558+0000","last_became_peered":"2024-04-25T10:16:07.416558+0000","last_unstale":"2024-04-25T10:16:07.418428+0000","last_undegraded":"2024-04-25T10:16:07.418428+0000","last_fullsized":"2024-04-25T10:16:07.418428+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:56.050776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.411524+0000","last_change":"2024-04-25T10:16:07.411524+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.411524+0000","last_undegraded":"2024-04-25T10:16:07.411524+0000","last_fullsized":"2024-04-25T10:16:07.411524+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.423239+0000","last_change":"2024-04-25T10:16:07.421886+0000","last_active":"2024-04-25T10:16:07.423239+0000","last_peered":"2024-04-25T10:16:07.423239+0000","last_clean":"2024-04-25T10:16:07.423239+0000","last_became_active":"2024-04-25T10:16:07.421532+0000","last_became_peered":"2024-04-25T10:16:07.421532+0000","last_unstale":"2024-04-25T10:16:07.423239+0000","last_undegraded":"2024-04-25T10:16:07.423239+0000","last_fullsized":"2024-04-25T10:16:07.423239+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:39.137374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.404044+0000","last_change":"2024-04-25T10:16:07.404044+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.404044+0000","last_undegraded":"2024-04-25T10:16:07.404044+0000","last_fullsized":"2024-04-25T10:16:07.404044+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.427225+0000","last_change":"2024-04-25T10:16:07.418216+0000","last_active":"2024-04-25T10:16:07.427225+0000","last_peered":"2024-04-25T10:16:07.427225+0000","last_clean":"2024-04-25T10:16:07.427225+0000","last_became_active":"2024-04-25T10:16:07.417942+0000","last_became_peered":"2024-04-25T10:16:07.417942+0000","last_unstale":"2024-04-25T10:16:07.427225+0000","last_undegraded":"2024-04-25T10:16:07.427225+0000","last_fullsized":"2024-04-25T10:16:07.427225+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:41.574226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.394715+0000","last_change":"2024-04-25T10:16:07.394715+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.394715+0000","last_undegraded":"2024-04-25T10:16:07.394715+0000","last_fullsized":"2024-04-25T10:16:07.394715+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.426649+0000","last_change":"2024-04-25T10:16:07.418178+0000","last_active":"2024-04-25T10:16:07.426649+0000","last_peered":"2024-04-25T10:16:07.426649+0000","last_clean":"2024-04-25T10:16:07.426649+0000","last_became_active":"2024-04-25T10:16:07.417805+0000","last_became_peered":"2024-04-25T10:16:07.417805+0000","last_unstale":"2024-04-25T10:16:07.426649+0000","last_undegraded":"2024-04-25T10:16:07.426649+0000","last_fullsized":"2024-04-25T10:16:07.426649+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:47.895635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.406838+0000","last_change":"2024-04-25T10:16:07.406838+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.406838+0000","last_undegraded":"2024-04-25T10:16:07.406838+0000","last_fullsized":"2024-04-25T10:16:07.406838+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.422479+0000","last_change":"2024-04-25T10:16:07.414135+0000","last_active":"2024-04-25T10:16:07.422479+0000","last_peered":"2024-04-25T10:16:07.422479+0000","last_clean":"2024-04-25T10:16:07.422479+0000","last_became_active":"2024-04-25T10:16:07.413293+0000","last_became_peered":"2024-04-25T10:16:07.413293+0000","last_unstale":"2024-04-25T10:16:07.422479+0000","last_undegraded":"2024-04-25T10:16:07.422479+0000","last_fullsized":"2024-04-25T10:16:07.422479+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:50.306119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.406092+0000","last_change":"2024-04-25T10:16:07.406092+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.406092+0000","last_undegraded":"2024-04-25T10:16:07.406092+0000","last_fullsized":"2024-04-25T10:16:07.406092+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.433461+0000","last_change":"2024-04-25T10:16:07.430570+0000","last_active":"2024-04-25T10:16:07.433461+0000","last_peered":"2024-04-25T10:16:07.433461+0000","last_clean":"2024-04-25T10:16:07.433461+0000","last_became_active":"2024-04-25T10:16:07.430098+0000","last_became_peered":"2024-04-25T10:16:07.430098+0000","last_unstale":"2024-04-25T10:16:07.433461+0000","last_undegraded":"2024-04-25T10:16:07.433461+0000","last_fullsized":"2024-04-25T10:16:07.433461+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:57.919496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.397887+0000","last_change":"2024-04-25T10:16:07.397887+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.397887+0000","last_undegraded":"2024-04-25T10:16:07.397887+0000","last_fullsized":"2024-04-25T10:16:07.397887+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.420683+0000","last_change":"2024-04-25T10:16:07.418621+0000","last_active":"2024-04-25T10:16:07.420683+0000","last_peered":"2024-04-25T10:16:07.420683+0000","last_clean":"2024-04-25T10:16:07.420683+0000","last_became_active":"2024-04-25T10:16:07.417941+0000","last_became_peered":"2024-04-25T10:16:07.417941+0000","last_unstale":"2024-04-25T10:16:07.420683+0000","last_undegraded":"2024-04-25T10:16:07.420683+0000","last_fullsized":"2024-04-25T10:16:07.420683+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:02.596736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.392594+0000","last_change":"2024-04-25T10:16:07.392594+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.392594+0000","last_undegraded":"2024-04-25T10:16:07.392594+0000","last_fullsized":"2024-04-25T10:16:07.392594+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.428717+0000","last_change":"2024-04-25T10:16:07.426643+0000","last_active":"2024-04-25T10:16:07.428717+0000","last_peered":"2024-04-25T10:16:07.428717+0000","last_clean":"2024-04-25T10:16:07.428717+0000","last_became_active":"2024-04-25T10:16:07.417621+0000","last_became_peered":"2024-04-25T10:16:07.417621+0000","last_unstale":"2024-04-25T10:16:07.428717+0000","last_undegraded":"2024-04-25T10:16:07.428717+0000","last_fullsized":"2024-04-25T10:16:07.428717+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:07.218543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.397357+0000","last_change":"2024-04-25T10:16:07.397357+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.397357+0000","last_undegraded":"2024-04-25T10:16:07.397357+0000","last_fullsized":"2024-04-25T10:16:07.397357+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.420721+0000","last_change":"2024-04-25T10:16:07.420219+0000","last_active":"2024-04-25T10:16:07.420721+0000","last_peered":"2024-04-25T10:16:07.420721+0000","last_clean":"2024-04-25T10:16:07.420721+0000","last_became_active":"2024-04-25T10:16:07.419584+0000","last_became_peered":"2024-04-25T10:16:07.419584+0000","last_unstale":"2024-04-25T10:16:07.420721+0000","last_undegraded":"2024-04-25T10:16:07.420721+0000","last_fullsized":"2024-04-25T10:16:07.420721+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:24.366732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.099281+0000","last_change":"2024-04-25T10:16:08.098323+0000","last_active":"2024-04-25T10:16:08.099281+0000","last_peered":"2024-04-25T10:16:08.099281+0000","last_clean":"2024-04-25T10:16:08.099281+0000","last_became_active":"2024-04-25T10:16:08.097919+0000","last_became_peered":"2024-04-25T10:16:08.097919+0000","last_unstale":"2024-04-25T10:16:08.099281+0000","last_undegraded":"2024-04-25T10:16:08.099281+0000","last_fullsized":"2024-04-25T10:16:08.099281+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:45.513718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.422930+0000","last_change":"2024-04-25T10:16:07.420743+0000","last_active":"2024-04-25T10:16:07.422930+0000","last_peered":"2024-04-25T10:16:07.422930+0000","last_clean":"2024-04-25T10:16:07.422930+0000","last_became_active":"2024-04-25T10:16:07.419996+0000","last_became_peered":"2024-04-25T10:16:07.419996+0000","last_unstale":"2024-04-25T10:16:07.422930+0000","last_undegraded":"2024-04-25T10:16:07.422930+0000","last_fullsized":"2024-04-25T10:16:07.422930+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:02.778487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.096755+0000","last_change":"2024-04-25T10:16:08.089387+0000","last_active":"2024-04-25T10:16:08.096755+0000","last_peered":"2024-04-25T10:16:08.096755+0000","last_clean":"2024-04-25T10:16:08.096755+0000","last_became_active":"2024-04-25T10:16:08.084835+0000","last_became_peered":"2024-04-25T10:16:08.084835+0000","last_unstale":"2024-04-25T10:16:08.096755+0000","last_undegraded":"2024-04-25T10:16:08.096755+0000","last_fullsized":"2024-04-25T10:16:08.096755+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:27.686840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.434871+0000","last_change":"2024-04-25T10:16:07.432911+0000","last_active":"2024-04-25T10:16:07.434871+0000","last_peered":"2024-04-25T10:16:07.434871+0000","last_clean":"2024-04-25T10:16:07.434871+0000","last_became_active":"2024-04-25T10:16:07.432458+0000","last_became_peered":"2024-04-25T10:16:07.432458+0000","last_unstale":"2024-04-25T10:16:07.434871+0000","last_undegraded":"2024-04-25T10:16:07.434871+0000","last_fullsized":"2024-04-25T10:16:07.434871+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:02.196126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.096742+0000","last_change":"2024-04-25T10:16:08.089355+0000","last_active":"2024-04-25T10:16:08.096742+0000","last_peered":"2024-04-25T10:16:08.096742+0000","last_clean":"2024-04-25T10:16:08.096742+0000","last_became_active":"2024-04-25T10:16:08.086653+0000","last_became_peered":"2024-04-25T10:16:08.086653+0000","last_unstale":"2024-04-25T10:16:08.096742+0000","last_undegraded":"2024-04-25T10:16:08.096742+0000","last_fullsized":"2024-04-25T10:16:08.096742+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:22.394766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.422456+0000","last_change":"2024-04-25T10:16:07.411824+0000","last_active":"2024-04-25T10:16:07.422456+0000","last_peered":"2024-04-25T10:16:07.422456+0000","last_clean":"2024-04-25T10:16:07.422456+0000","last_became_active":"2024-04-25T10:16:07.407193+0000","last_became_peered":"2024-04-25T10:16:07.407193+0000","last_unstale":"2024-04-25T10:16:07.422456+0000","last_undegraded":"2024-04-25T10:16:07.422456+0000","last_fullsized":"2024-04-25T10:16:07.422456+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:39.505392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.395536+0000","last_change":"2024-04-25T10:16:07.395536+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.395536+0000","last_undegraded":"2024-04-25T10:16:07.395536+0000","last_fullsized":"2024-04-25T10:16:07.395536+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.414640+0000","last_change":"2024-04-25T10:16:07.413599+0000","last_active":"2024-04-25T10:16:07.414640+0000","last_peered":"2024-04-25T10:16:07.414640+0000","last_clean":"2024-04-25T10:16:07.414640+0000","last_became_active":"2024-04-25T10:16:07.413293+0000","last_became_peered":"2024-04-25T10:16:07.413293+0000","last_unstale":"2024-04-25T10:16:07.414640+0000","last_undegraded":"2024-04-25T10:16:07.414640+0000","last_fullsized":"2024-04-25T10:16:07.414640+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:01.844673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.096725+0000","last_change":"2024-04-25T10:16:08.089103+0000","last_active":"2024-04-25T10:16:08.096725+0000","last_peered":"2024-04-25T10:16:08.096725+0000","last_clean":"2024-04-25T10:16:08.096725+0000","last_became_active":"2024-04-25T10:16:08.085453+0000","last_became_peered":"2024-04-25T10:16:08.085453+0000","last_unstale":"2024-04-25T10:16:08.096725+0000","last_undegraded":"2024-04-25T10:16:08.096725+0000","last_fullsized":"2024-04-25T10:16:08.096725+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:59.638430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.422038+0000","last_change":"2024-04-25T10:16:07.412591+0000","last_active":"2024-04-25T10:16:07.422038+0000","last_peered":"2024-04-25T10:16:07.422038+0000","last_clean":"2024-04-25T10:16:07.422038+0000","last_became_active":"2024-04-25T10:16:07.411412+0000","last_became_peered":"2024-04-25T10:16:07.411412+0000","last_unstale":"2024-04-25T10:16:07.422038+0000","last_undegraded":"2024-04-25T10:16:07.422038+0000","last_fullsized":"2024-04-25T10:16:07.422038+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:11.307881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.397115+0000","last_change":"2024-04-25T10:16:07.397115+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.397115+0000","last_undegraded":"2024-04-25T10:16:07.397115+0000","last_fullsized":"2024-04-25T10:16:07.397115+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.429308+0000","last_change":"2024-04-25T10:16:07.427610+0000","last_active":"2024-04-25T10:16:07.429308+0000","last_peered":"2024-04-25T10:16:07.429308+0000","last_clean":"2024-04-25T10:16:07.429308+0000","last_became_active":"2024-04-25T10:16:07.427304+0000","last_became_peered":"2024-04-25T10:16:07.427304+0000","last_unstale":"2024-04-25T10:16:07.429308+0000","last_undegraded":"2024-04-25T10:16:07.429308+0000","last_fullsized":"2024-04-25T10:16:07.429308+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:39.821292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.415394+0000","last_change":"2024-04-25T10:16:07.415394+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.415394+0000","last_undegraded":"2024-04-25T10:16:07.415394+0000","last_fullsized":"2024-04-25T10:16:07.415394+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.430485+0000","last_change":"2024-04-25T10:16:07.429884+0000","last_active":"2024-04-25T10:16:07.430485+0000","last_peered":"2024-04-25T10:16:07.430485+0000","last_clean":"2024-04-25T10:16:07.430485+0000","last_became_active":"2024-04-25T10:16:07.429534+0000","last_became_peered":"2024-04-25T10:16:07.429534+0000","last_unstale":"2024-04-25T10:16:07.430485+0000","last_undegraded":"2024-04-25T10:16:07.430485+0000","last_fullsized":"2024-04-25T10:16:07.430485+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:23.681463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.404505+0000","last_change":"2024-04-25T10:16:07.404505+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.404505+0000","last_undegraded":"2024-04-25T10:16:07.404505+0000","last_fullsized":"2024-04-25T10:16:07.404505+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.425719+0000","last_change":"2024-04-25T10:16:07.423602+0000","last_active":"2024-04-25T10:16:07.425719+0000","last_peered":"2024-04-25T10:16:07.425719+0000","last_clean":"2024-04-25T10:16:07.425719+0000","last_became_active":"2024-04-25T10:16:07.422889+0000","last_became_peered":"2024-04-25T10:16:07.422889+0000","last_unstale":"2024-04-25T10:16:07.425719+0000","last_undegraded":"2024-04-25T10:16:07.425719+0000","last_fullsized":"2024-04-25T10:16:07.425719+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:08.862874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.395730+0000","last_change":"2024-04-25T10:16:07.395730+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.395730+0000","last_undegraded":"2024-04-25T10:16:07.395730+0000","last_fullsized":"2024-04-25T10:16:07.395730+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.428910+0000","last_change":"2024-04-25T10:16:07.427399+0000","last_active":"2024-04-25T10:16:07.428910+0000","last_peered":"2024-04-25T10:16:07.428910+0000","last_clean":"2024-04-25T10:16:07.428910+0000","last_became_active":"2024-04-25T10:16:07.426945+0000","last_became_peered":"2024-04-25T10:16:07.426945+0000","last_unstale":"2024-04-25T10:16:07.428910+0000","last_undegraded":"2024-04-25T10:16:07.428910+0000","last_fullsized":"2024-04-25T10:16:07.428910+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:26.657752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.413935+0000","last_change":"2024-04-25T10:16:07.413935+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.413935+0000","last_undegraded":"2024-04-25T10:16:07.413935+0000","last_fullsized":"2024-04-25T10:16:07.413935+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.423512+0000","last_change":"2024-04-25T10:16:07.423090+0000","last_active":"2024-04-25T10:16:07.423512+0000","last_peered":"2024-04-25T10:16:07.423512+0000","last_clean":"2024-04-25T10:16:07.423512+0000","last_became_active":"2024-04-25T10:16:07.422770+0000","last_became_peered":"2024-04-25T10:16:07.422770+0000","last_unstale":"2024-04-25T10:16:07.423512+0000","last_undegraded":"2024-04-25T10:16:07.423512+0000","last_fullsized":"2024-04-25T10:16:07.423512+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:28.612242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.078443+0000","last_change":"2024-04-25T10:16:08.060603+0000","last_active":"2024-04-25T10:16:08.078443+0000","last_peered":"2024-04-25T10:16:08.078443+0000","last_clean":"2024-04-25T10:16:08.078443+0000","last_became_active":"2024-04-25T10:16:08.059770+0000","last_became_peered":"2024-04-25T10:16:08.059770+0000","last_unstale":"2024-04-25T10:16:08.078443+0000","last_undegraded":"2024-04-25T10:16:08.078443+0000","last_fullsized":"2024-04-25T10:16:08.078443+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:37.996501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.427811+0000","last_change":"2024-04-25T10:16:07.425514+0000","last_active":"2024-04-25T10:16:07.427811+0000","last_peered":"2024-04-25T10:16:07.427811+0000","last_clean":"2024-04-25T10:16:07.427811+0000","last_became_active":"2024-04-25T10:16:07.424995+0000","last_became_peered":"2024-04-25T10:16:07.424995+0000","last_unstale":"2024-04-25T10:16:07.427811+0000","last_undegraded":"2024-04-25T10:16:07.427811+0000","last_fullsized":"2024-04-25T10:16:07.427811+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:13.223432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.083495+0000","last_change":"2024-04-25T10:16:08.078197+0000","last_active":"2024-04-25T10:16:08.083495+0000","last_peered":"2024-04-25T10:16:08.083495+0000","last_clean":"2024-04-25T10:16:08.083495+0000","last_became_active":"2024-04-25T10:16:08.077680+0000","last_became_peered":"2024-04-25T10:16:08.077680+0000","last_unstale":"2024-04-25T10:16:08.083495+0000","last_undegraded":"2024-04-25T10:16:08.083495+0000","last_fullsized":"2024-04-25T10:16:08.083495+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:06.032932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.052504+0000","last_change":"2024-04-25T10:16:07.428578+0000","last_active":"2024-04-25T10:16:08.052504+0000","last_peered":"2024-04-25T10:16:08.052504+0000","last_clean":"2024-04-25T10:16:08.052504+0000","last_became_active":"2024-04-25T10:16:07.427915+0000","last_became_peered":"2024-04-25T10:16:07.427915+0000","last_unstale":"2024-04-25T10:16:08.052504+0000","last_undegraded":"2024-04-25T10:16:08.052504+0000","last_fullsized":"2024-04-25T10:16:08.052504+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:29.819106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.096683+0000","last_change":"2024-04-25T10:16:08.086860+0000","last_active":"2024-04-25T10:16:08.096683+0000","last_peered":"2024-04-25T10:16:08.096683+0000","last_clean":"2024-04-25T10:16:08.096683+0000","last_became_active":"2024-04-25T10:16:08.085415+0000","last_became_peered":"2024-04-25T10:16:08.085415+0000","last_unstale":"2024-04-25T10:16:08.096683+0000","last_undegraded":"2024-04-25T10:16:08.096683+0000","last_fullsized":"2024-04-25T10:16:08.096683+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:37.647431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.406674+0000","last_change":"2024-04-25T10:16:07.402713+0000","last_active":"2024-04-25T10:16:07.406674+0000","last_peered":"2024-04-25T10:16:07.406674+0000","last_clean":"2024-04-25T10:16:07.406674+0000","last_became_active":"2024-04-25T10:16:07.402415+0000","last_became_peered":"2024-04-25T10:16:07.402415+0000","last_unstale":"2024-04-25T10:16:07.406674+0000","last_undegraded":"2024-04-25T10:16:07.406674+0000","last_fullsized":"2024-04-25T10:16:07.406674+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:43.140323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.394586+0000","last_change":"2024-04-25T10:16:07.394586+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.394586+0000","last_undegraded":"2024-04-25T10:16:07.394586+0000","last_fullsized":"2024-04-25T10:16:07.394586+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.428963+0000","last_change":"2024-04-25T10:16:07.426844+0000","last_active":"2024-04-25T10:16:07.428963+0000","last_peered":"2024-04-25T10:16:07.428963+0000","last_clean":"2024-04-25T10:16:07.428963+0000","last_became_active":"2024-04-25T10:16:07.424170+0000","last_became_peered":"2024-04-25T10:16:07.424170+0000","last_unstale":"2024-04-25T10:16:07.428963+0000","last_undegraded":"2024-04-25T10:16:07.428963+0000","last_fullsized":"2024-04-25T10:16:07.428963+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:17.171914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T10:16:07.392340+0000","last_change":"2024-04-25T10:16:07.392340+0000","last_active":"2024-04-25T10:16:07.370666+0000","last_peered":"2024-04-25T10:16:07.370666+0000","last_clean":"2024-04-25T10:16:07.370666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:07.392340+0000","last_undegraded":"2024-04-25T10:16:07.392340+0000","last_fullsized":"2024-04-25T10:16:07.392340+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:16:07.434876+0000","last_change":"2024-04-25T10:16:07.432894+0000","last_active":"2024-04-25T10:16:07.434876+0000","last_peered":"2024-04-25T10:16:07.434876+0000","last_clean":"2024-04-25T10:16:07.434876+0000","last_became_active":"2024-04-25T10:16:07.432221+0000","last_became_peered":"2024-04-25T10:16:07.432221+0000","last_unstale":"2024-04-25T10:16:07.434876+0000","last_undegraded":"2024-04-25T10:16:07.434876+0000","last_fullsized":"2024-04-25T10:16:07.434876+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:35.643119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":331,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.073129+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:16:08.073129+0000","last_peered":"2024-04-25T10:16:08.073129+0000","last_clean":"2024-04-25T10:16:08.073129+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:16:08.073129+0000","last_undegraded":"2024-04-25T10:16:08.073129+0000","last_fullsized":"2024-04-25T10:16:08.073129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509576,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1929216,"data_stored":1166063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1335296,"data_stored":575695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542278,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1323008,"data_stored":569420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542279,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1323008,"data_stored":569420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542279,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36636,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335204,"statfs":{"total":96636764160,"available":96599248896,"internally_reserved":0,"allocated":1323008,"data_stored":569420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542279,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36636,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335204,"statfs":{"total":96636764160,"available":96599248896,"internally_reserved":0,"allocated":1323008,"data_stored":569420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542279,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36632,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335208,"statfs":{"total":96636764160,"available":96599252992,"internally_reserved":0,"allocated":1323008,"data_stored":569420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542280,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37212,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334628,"statfs":{"total":96636764160,"available":96598659072,"internally_reserved":0,"allocated":1916928,"data_stored":1159788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:16:12.442 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:16:12.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.598+0000 7f34609f1640 1 Processor -- start 2024-04-25T10:16:12.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.598+0000 7f34609f1640 1 -- start start 2024-04-25T10:16:12.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f34609f1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f345c0cdc30 0x7f345c06ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f34609f1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f345c0cff30 0x7f345c0cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f34609f1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345c0d03b0 con 0x7f345c0cfae0 2024-04-25T10:16:12.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f34609f1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f345c0d0690 con 0x7f345c0cdc30 2024-04-25T10:16:12.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f34609f1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345c0d0970 con 0x7f345c0cff30 2024-04-25T10:16:12.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3453fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f345c0cfae0 0x7f345c099a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48654/0 (socket says 172.21.15.60:48654) 2024-04-25T10:16:12.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3453fff640 1 -- 172.21.15.60:0/2932960224 learned_addr learned my addr 172.21.15.60:0/2932960224 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f345a59c640 1 --2- 172.21.15.60:0/2932960224 >> v2:172.21.15.100:3300/0 conn(0x7f345c0cdc30 0x7f345c06ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f345ad9d640 1 --2- 172.21.15.60:0/2932960224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f345c0cff30 0x7f345c0cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 -- 172.21.15.60:0/2932960224 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028489041 0 0) 0x7f345c0d03b0 con 0x7f345c0cfae0 2024-04-25T10:16:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 -- 172.21.15.60:0/2932960224 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3448003530 con 0x7f345c0cfae0 2024-04-25T10:16:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 -- 172.21.15.60:0/2932960224 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1245816933 0 0) 0x7f3448003530 con 0x7f345c0cfae0 2024-04-25T10:16:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 -- 172.21.15.60:0/2932960224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f345c0cff30 msgr2=0x7f345c0cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 --2- 172.21.15.60:0/2932960224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f345c0cff30 0x7f345c0cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 -- 172.21.15.60:0/2932960224 >> v2:172.21.15.100:3300/0 conn(0x7f345c0cdc30 msgr2=0x7f345c06ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 --2- 172.21.15.60:0/2932960224 >> v2:172.21.15.100:3300/0 conn(0x7f345c0cdc30 0x7f345c06ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 -- 172.21.15.60:0/2932960224 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c0d0c50 con 0x7f345c0cfae0 2024-04-25T10:16:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 -- 172.21.15.60:0/2932960224 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f3444001680 con 0x7f345c0cfae0 2024-04-25T10:16:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 -- 172.21.15.60:0/2932960224 >> v1:172.21.15.60:6789/0 conn(0x7f345c0cfae0 legacy=0x7f345c099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 --2- 172.21.15.60:0/2932960224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34480039c0 0x7f3448003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 --2- 172.21.15.60:0/2932960224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3448004810 0x7f3448004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 --2- 172.21.15.60:0/2932960224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f345c0cff30 0x7f34480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 -- 172.21.15.60:0/2932960224 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c0d0690 con 0x7f3448004810 2024-04-25T10:16:12.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 -- 172.21.15.60:0/2932960224 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c0d0970 con 0x7f34480039c0 2024-04-25T10:16:12.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 -- 172.21.15.60:0/2932960224 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344800b9d0 con 0x7f345c0cff30 2024-04-25T10:16:12.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3453fff640 1 --2- 172.21.15.60:0/2932960224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3448004810 0x7f3448004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f345a59c640 1 --2- 172.21.15.60:0/2932960224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34480039c0 0x7f3448003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f345ad9d640 1 --2- 172.21.15.60:0/2932960224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f345c0cff30 0x7f34480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3453fff640 1 -- 172.21.15.60:0/2932960224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f345c0cff30 msgr2=0x7f34480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:12.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3453fff640 1 --2- 172.21.15.60:0/2932960224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f345c0cff30 0x7f34480051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3453fff640 1 -- 172.21.15.60:0/2932960224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34480039c0 msgr2=0x7f3448003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:12.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3453fff640 1 --2- 172.21.15.60:0/2932960224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34480039c0 0x7f3448003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3453fff640 1 -- 172.21.15.60:0/2932960224 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3448003530 con 0x7f3448004810 2024-04-25T10:16:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3453fff640 1 --2- 172.21.15.60:0/2932960224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3448004810 0x7f3448004c90 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f344400c210 tx=0x7f344401f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22a090f02cc0f47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 -- 172.21.15.60:0/2932960224 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444007d40 con 0x7f3448004810 2024-04-25T10:16:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 -- 172.21.15.60:0/2932960224 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3444005b00 con 0x7f3448004810 2024-04-25T10:16:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.602+0000 7f3459d9b640 1 -- 172.21.15.60:0/2932960224 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444007070 con 0x7f3448004810 2024-04-25T10:16:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f34609f1640 1 -- 172.21.15.60:0/2932960224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3448004810 msgr2=0x7f3448004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f34609f1640 1 --2- 172.21.15.60:0/2932960224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3448004810 0x7f3448004c90 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f344400c210 tx=0x7f344401f3f0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f345ad9d640 1 -- 172.21.15.60:0/2932960224 reap_dead start 2024-04-25T10:16:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f34609f1640 1 -- 172.21.15.60:0/2932960224 shutdown_connections 2024-04-25T10:16:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f34609f1640 1 -- 172.21.15.60:0/2932960224 >> 172.21.15.60:0/2932960224 conn(0x7f345c0b32a0 msgr2=0x7f345c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f34609f1640 1 -- 172.21.15.60:0/2932960224 shutdown_connections 2024-04-25T10:16:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f34609f1640 1 -- 172.21.15.60:0/2932960224 wait complete. 2024-04-25T10:16:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f34609f1640 1 Processor -- start 2024-04-25T10:16:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f34609f1640 1 -- start start 2024-04-25T10:16:12.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f34609f1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3448004810 0x7f345c09cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f34609f1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f345c09c450 0x7f345c1f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f34609f1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f345c1f2d40 0x7f345c1f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f34609f1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c09a2c0 con 0x7f345c1f2d40 2024-04-25T10:16:12.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f34609f1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c09a140 con 0x7f345c09c450 2024-04-25T10:16:12.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f34609f1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345c0cbe00 con 0x7f3448004810 2024-04-25T10:16:12.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f345ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f345c1f2d40 0x7f345c1f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f345ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f345c1f2d40 0x7f345c1f3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35278/0 (socket says 172.21.15.60:35278) 2024-04-25T10:16:12.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f345ad9d640 1 -- 172.21.15.60:0/2686258004 learned_addr learned my addr 172.21.15.60:0/2686258004 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:12.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f345a59c640 1 --2- 172.21.15.60:0/2686258004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3448004810 0x7f345c09cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f3453fff640 1 --2- 172.21.15.60:0/2686258004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f345c09c450 0x7f345c1f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f345ad9d640 1 -- 172.21.15.60:0/2686258004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3448004810 msgr2=0x7f345c09cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f345ad9d640 1 --2- 172.21.15.60:0/2686258004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3448004810 0x7f345c09cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f345ad9d640 1 -- 172.21.15.60:0/2686258004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f345c09c450 msgr2=0x7f345c1f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f345ad9d640 1 --2- 172.21.15.60:0/2686258004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f345c09c450 0x7f345c1f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f345ad9d640 1 -- 172.21.15.60:0/2686258004 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c09b690 con 0x7f345c1f2d40 2024-04-25T10:16:12.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f345ad9d640 1 --2- 172.21.15.60:0/2686258004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f345c1f2d40 0x7f345c1f3520 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f345c072f10 tx=0x7f34540115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f3452ffd640 1 -- 172.21.15.60:0/2686258004 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34540039a0 con 0x7f345c1f2d40 2024-04-25T10:16:12.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f3452ffd640 1 -- 172.21.15.60:0/2686258004 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345401b070 con 0x7f345c1f2d40 2024-04-25T10:16:12.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f34609f1640 1 -- 172.21.15.60:0/2686258004 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345c09b970 con 0x7f345c1f2d40 2024-04-25T10:16:12.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f3452ffd640 1 -- 172.21.15.60:0/2686258004 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3454003b80 con 0x7f345c1f2d40 2024-04-25T10:16:12.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.606+0000 7f34609f1640 1 -- 172.21.15.60:0/2686258004 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f345c1f4290 con 0x7f345c1f2d40 2024-04-25T10:16:12.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.610+0000 7f34609f1640 1 -- 172.21.15.60:0/2686258004 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345c074770 con 0x7f345c1f2d40 2024-04-25T10:16:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.610+0000 7f3452ffd640 1 -- 172.21.15.60:0/2686258004 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3454004030 con 0x7f345c1f2d40 2024-04-25T10:16:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.610+0000 7f3452ffd640 1 --2- 172.21.15.60:0/2686258004 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f343c066f60 0x7f343c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.610+0000 7f3452ffd640 1 -- 172.21.15.60:0/2686258004 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3454085810 con 0x7f345c1f2d40 2024-04-25T10:16:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.610+0000 7f345a59c640 1 --2- 172.21.15.60:0/2686258004 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f343c066f60 0x7f343c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.610+0000 7f345a59c640 1 --2- 172.21.15.60:0/2686258004 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f343c066f60 0x7f343c069420 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f345c0b62d0 tx=0x7f344c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.614+0000 7f3452ffd640 1 -- 172.21.15.60:0/2686258004 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3454025020 con 0x7f345c1f2d40 2024-04-25T10:16:12.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.770+0000 7f34609f1640 1 -- 172.21.15.60:0/2686258004 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f345c099b60 con 0x7f343c066f60 2024-04-25T10:16:12.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.786+0000 7f3452ffd640 1 -- 172.21.15.60:0/2686258004 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252460 (secure 0 0 0) 0x7f345c099b60 con 0x7f343c066f60 2024-04-25T10:16:12.793 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:12.796 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:16:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.790+0000 7f34609f1640 1 -- 172.21.15.60:0/2686258004 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f343c066f60 msgr2=0x7f343c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.790+0000 7f34609f1640 1 --2- 172.21.15.60:0/2686258004 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f343c066f60 0x7f343c069420 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f345c0b62d0 tx=0x7f344c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.790+0000 7f34609f1640 1 -- 172.21.15.60:0/2686258004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f345c1f2d40 msgr2=0x7f345c1f3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.790+0000 7f34609f1640 1 --2- 172.21.15.60:0/2686258004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f345c1f2d40 0x7f345c1f3520 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f345c072f10 tx=0x7f34540115a0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.790+0000 7f34609f1640 1 -- 172.21.15.60:0/2686258004 shutdown_connections 2024-04-25T10:16:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.790+0000 7f34609f1640 1 --2- 172.21.15.60:0/2686258004 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f343c066f60 0x7f343c069420 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.790+0000 7f34609f1640 1 --2- 172.21.15.60:0/2686258004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f345c1f2d40 0x7f345c1f3520 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.790+0000 7f34609f1640 1 --2- 172.21.15.60:0/2686258004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f345c09c450 0x7f345c1f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.790+0000 7f34609f1640 1 --2- 172.21.15.60:0/2686258004 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3448004810 0x7f345c09cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.790+0000 7f34609f1640 1 -- 172.21.15.60:0/2686258004 >> 172.21.15.60:0/2686258004 conn(0x7f345c0b32a0 msgr2=0x7f345c06c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.790+0000 7f34609f1640 1 -- 172.21.15.60:0/2686258004 shutdown_connections 2024-04-25T10:16:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.790+0000 7f34609f1640 1 -- 172.21.15.60:0/2686258004 wait complete. 2024-04-25T10:16:12.813 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-04-25T10:16:12.615779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":277908,"kb_used_data":11792,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754696812,"statfs":{"total":773094113280,"available":772809535488,"internally_reserved":0,"allocated":12075008,"data_stored":5868950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001757"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192500+0000","last_change":"2024-04-25T10:16:08.087098+0000","last_active":"2024-04-25T10:16:09.192500+0000","last_peered":"2024-04-25T10:16:09.192500+0000","last_clean":"2024-04-25T10:16:09.192500+0000","last_became_active":"2024-04-25T10:16:08.081195+0000","last_became_peered":"2024-04-25T10:16:08.081195+0000","last_unstale":"2024-04-25T10:16:09.192500+0000","last_undegraded":"2024-04-25T10:16:09.192500+0000","last_fullsized":"2024-04-25T10:16:09.192500+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:14.186023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.077829+0000","last_change":"2024-04-25T10:16:08.065994+0000","last_active":"2024-04-25T10:16:08.077829+0000","last_peered":"2024-04-25T10:16:08.077829+0000","last_clean":"2024-04-25T10:16:08.077829+0000","last_became_active":"2024-04-25T10:16:08.064520+0000","last_became_peered":"2024-04-25T10:16:08.064520+0000","last_unstale":"2024-04-25T10:16:08.077829+0000","last_undegraded":"2024-04-25T10:16:08.077829+0000","last_fullsized":"2024-04-25T10:16:08.077829+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:54.439251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.096803+0000","last_change":"2024-04-25T10:16:08.089945+0000","last_active":"2024-04-25T10:16:08.096803+0000","last_peered":"2024-04-25T10:16:08.096803+0000","last_clean":"2024-04-25T10:16:08.096803+0000","last_became_active":"2024-04-25T10:16:08.086425+0000","last_became_peered":"2024-04-25T10:16:08.086425+0000","last_unstale":"2024-04-25T10:16:08.096803+0000","last_undegraded":"2024-04-25T10:16:08.096803+0000","last_fullsized":"2024-04-25T10:16:08.096803+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:40.325134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192677+0000","last_change":"2024-04-25T10:16:08.093100+0000","last_active":"2024-04-25T10:16:09.192677+0000","last_peered":"2024-04-25T10:16:09.192677+0000","last_clean":"2024-04-25T10:16:09.192677+0000","last_became_active":"2024-04-25T10:16:08.087083+0000","last_became_peered":"2024-04-25T10:16:08.087083+0000","last_unstale":"2024-04-25T10:16:09.192677+0000","last_undegraded":"2024-04-25T10:16:09.192677+0000","last_fullsized":"2024-04-25T10:16:09.192677+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:10.729196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.056068+0000","last_change":"2024-04-25T10:16:08.062193+0000","last_active":"2024-04-25T10:16:09.056068+0000","last_peered":"2024-04-25T10:16:09.056068+0000","last_clean":"2024-04-25T10:16:09.056068+0000","last_became_active":"2024-04-25T10:16:08.061601+0000","last_became_peered":"2024-04-25T10:16:08.061601+0000","last_unstale":"2024-04-25T10:16:09.056068+0000","last_undegraded":"2024-04-25T10:16:09.056068+0000","last_fullsized":"2024-04-25T10:16:09.056068+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:37.732452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.294847+0000","last_change":"2024-04-25T10:16:08.091786+0000","last_active":"2024-04-25T10:16:10.294847+0000","last_peered":"2024-04-25T10:16:10.294847+0000","last_clean":"2024-04-25T10:16:10.294847+0000","last_became_active":"2024-04-25T10:16:08.091055+0000","last_became_peered":"2024-04-25T10:16:08.091055+0000","last_unstale":"2024-04-25T10:16:10.294847+0000","last_undegraded":"2024-04-25T10:16:10.294847+0000","last_fullsized":"2024-04-25T10:16:10.294847+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:19.891442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.064297+0000","last_change":"2024-04-25T10:16:08.076555+0000","last_active":"2024-04-25T10:16:09.064297+0000","last_peered":"2024-04-25T10:16:09.064297+0000","last_clean":"2024-04-25T10:16:09.064297+0000","last_became_active":"2024-04-25T10:16:08.069960+0000","last_became_peered":"2024-04-25T10:16:08.069960+0000","last_unstale":"2024-04-25T10:16:09.064297+0000","last_undegraded":"2024-04-25T10:16:09.064297+0000","last_fullsized":"2024-04-25T10:16:09.064297+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:58.923322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.089390+0000","last_change":"2024-04-25T10:16:08.083106+0000","last_active":"2024-04-25T10:16:08.089390+0000","last_peered":"2024-04-25T10:16:08.089390+0000","last_clean":"2024-04-25T10:16:08.089390+0000","last_became_active":"2024-04-25T10:16:08.082341+0000","last_became_peered":"2024-04-25T10:16:08.082341+0000","last_unstale":"2024-04-25T10:16:08.089390+0000","last_undegraded":"2024-04-25T10:16:08.089390+0000","last_fullsized":"2024-04-25T10:16:08.089390+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:05.885118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192522+0000","last_change":"2024-04-25T10:16:08.094016+0000","last_active":"2024-04-25T10:16:09.192522+0000","last_peered":"2024-04-25T10:16:09.192522+0000","last_clean":"2024-04-25T10:16:09.192522+0000","last_became_active":"2024-04-25T10:16:08.092949+0000","last_became_peered":"2024-04-25T10:16:08.092949+0000","last_unstale":"2024-04-25T10:16:09.192522+0000","last_undegraded":"2024-04-25T10:16:09.192522+0000","last_fullsized":"2024-04-25T10:16:09.192522+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:16.261614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192303+0000","last_change":"2024-04-25T10:16:08.094473+0000","last_active":"2024-04-25T10:16:09.192303+0000","last_peered":"2024-04-25T10:16:09.192303+0000","last_clean":"2024-04-25T10:16:09.192303+0000","last_became_active":"2024-04-25T10:16:08.093753+0000","last_became_peered":"2024-04-25T10:16:08.093753+0000","last_unstale":"2024-04-25T10:16:09.192303+0000","last_undegraded":"2024-04-25T10:16:09.192303+0000","last_fullsized":"2024-04-25T10:16:09.192303+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:21.203240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.081997+0000","last_change":"2024-04-25T10:16:08.072100+0000","last_active":"2024-04-25T10:16:08.081997+0000","last_peered":"2024-04-25T10:16:08.081997+0000","last_clean":"2024-04-25T10:16:08.081997+0000","last_became_active":"2024-04-25T10:16:08.071642+0000","last_became_peered":"2024-04-25T10:16:08.071642+0000","last_unstale":"2024-04-25T10:16:08.081997+0000","last_undegraded":"2024-04-25T10:16:08.081997+0000","last_fullsized":"2024-04-25T10:16:08.081997+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:08.872501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192781+0000","last_change":"2024-04-25T10:16:08.098666+0000","last_active":"2024-04-25T10:16:09.192781+0000","last_peered":"2024-04-25T10:16:09.192781+0000","last_clean":"2024-04-25T10:16:09.192781+0000","last_became_active":"2024-04-25T10:16:08.094678+0000","last_became_peered":"2024-04-25T10:16:08.094678+0000","last_unstale":"2024-04-25T10:16:09.192781+0000","last_undegraded":"2024-04-25T10:16:09.192781+0000","last_fullsized":"2024-04-25T10:16:09.192781+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:04.087341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.096806+0000","last_change":"2024-04-25T10:16:08.091187+0000","last_active":"2024-04-25T10:16:08.096806+0000","last_peered":"2024-04-25T10:16:08.096806+0000","last_clean":"2024-04-25T10:16:08.096806+0000","last_became_active":"2024-04-25T10:16:08.090586+0000","last_became_peered":"2024-04-25T10:16:08.090586+0000","last_unstale":"2024-04-25T10:16:08.096806+0000","last_undegraded":"2024-04-25T10:16:08.096806+0000","last_fullsized":"2024-04-25T10:16:08.096806+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:57.403738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.089839+0000","last_change":"2024-04-25T10:16:08.084680+0000","last_active":"2024-04-25T10:16:08.089839+0000","last_peered":"2024-04-25T10:16:08.089839+0000","last_clean":"2024-04-25T10:16:08.089839+0000","last_became_active":"2024-04-25T10:16:08.082666+0000","last_became_peered":"2024-04-25T10:16:08.082666+0000","last_unstale":"2024-04-25T10:16:08.089839+0000","last_undegraded":"2024-04-25T10:16:08.089839+0000","last_fullsized":"2024-04-25T10:16:08.089839+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:40.403842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.294899+0000","last_change":"2024-04-25T10:16:08.077029+0000","last_active":"2024-04-25T10:16:10.294899+0000","last_peered":"2024-04-25T10:16:10.294899+0000","last_clean":"2024-04-25T10:16:10.294899+0000","last_became_active":"2024-04-25T10:16:08.076511+0000","last_became_peered":"2024-04-25T10:16:08.076511+0000","last_unstale":"2024-04-25T10:16:10.294899+0000","last_undegraded":"2024-04-25T10:16:10.294899+0000","last_fullsized":"2024-04-25T10:16:10.294899+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:53.567326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.191886+0000","last_change":"2024-04-25T10:16:08.091490+0000","last_active":"2024-04-25T10:16:09.191886+0000","last_peered":"2024-04-25T10:16:09.191886+0000","last_clean":"2024-04-25T10:16:09.191886+0000","last_became_active":"2024-04-25T10:16:08.087029+0000","last_became_peered":"2024-04-25T10:16:08.087029+0000","last_unstale":"2024-04-25T10:16:09.191886+0000","last_undegraded":"2024-04-25T10:16:09.191886+0000","last_fullsized":"2024-04-25T10:16:09.191886+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:37.675608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.082240+0000","last_change":"2024-04-25T10:16:08.070863+0000","last_active":"2024-04-25T10:16:08.082240+0000","last_peered":"2024-04-25T10:16:08.082240+0000","last_clean":"2024-04-25T10:16:08.082240+0000","last_became_active":"2024-04-25T10:16:08.070241+0000","last_became_peered":"2024-04-25T10:16:08.070241+0000","last_unstale":"2024-04-25T10:16:08.082240+0000","last_undegraded":"2024-04-25T10:16:08.082240+0000","last_fullsized":"2024-04-25T10:16:08.082240+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:59.145085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.300119+0000","last_change":"2024-04-25T10:16:08.079511+0000","last_active":"2024-04-25T10:16:10.300119+0000","last_peered":"2024-04-25T10:16:10.300119+0000","last_clean":"2024-04-25T10:16:10.300119+0000","last_became_active":"2024-04-25T10:16:08.079182+0000","last_became_peered":"2024-04-25T10:16:08.079182+0000","last_unstale":"2024-04-25T10:16:10.300119+0000","last_undegraded":"2024-04-25T10:16:10.300119+0000","last_fullsized":"2024-04-25T10:16:10.300119+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:29.481955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.096678+0000","last_change":"2024-04-25T10:16:08.086962+0000","last_active":"2024-04-25T10:16:08.096678+0000","last_peered":"2024-04-25T10:16:08.096678+0000","last_clean":"2024-04-25T10:16:08.096678+0000","last_became_active":"2024-04-25T10:16:08.085446+0000","last_became_peered":"2024-04-25T10:16:08.085446+0000","last_unstale":"2024-04-25T10:16:08.096678+0000","last_undegraded":"2024-04-25T10:16:08.096678+0000","last_fullsized":"2024-04-25T10:16:08.096678+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:56.557720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.081622+0000","last_change":"2024-04-25T10:16:08.067883+0000","last_active":"2024-04-25T10:16:08.081622+0000","last_peered":"2024-04-25T10:16:08.081622+0000","last_clean":"2024-04-25T10:16:08.081622+0000","last_became_active":"2024-04-25T10:16:08.067127+0000","last_became_peered":"2024-04-25T10:16:08.067127+0000","last_unstale":"2024-04-25T10:16:08.081622+0000","last_undegraded":"2024-04-25T10:16:08.081622+0000","last_fullsized":"2024-04-25T10:16:08.081622+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:38.208647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.081647+0000","last_change":"2024-04-25T10:16:08.069858+0000","last_active":"2024-04-25T10:16:08.081647+0000","last_peered":"2024-04-25T10:16:08.081647+0000","last_clean":"2024-04-25T10:16:08.081647+0000","last_became_active":"2024-04-25T10:16:08.069397+0000","last_became_peered":"2024-04-25T10:16:08.069397+0000","last_unstale":"2024-04-25T10:16:08.081647+0000","last_undegraded":"2024-04-25T10:16:08.081647+0000","last_fullsized":"2024-04-25T10:16:08.081647+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:27.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.193466+0000","last_change":"2024-04-25T10:16:08.071307+0000","last_active":"2024-04-25T10:16:10.193466+0000","last_peered":"2024-04-25T10:16:10.193466+0000","last_clean":"2024-04-25T10:16:10.193466+0000","last_became_active":"2024-04-25T10:16:08.070450+0000","last_became_peered":"2024-04-25T10:16:08.070450+0000","last_unstale":"2024-04-25T10:16:10.193466+0000","last_undegraded":"2024-04-25T10:16:10.193466+0000","last_fullsized":"2024-04-25T10:16:10.193466+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:41.387726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.083408+0000","last_change":"2024-04-25T10:16:08.075287+0000","last_active":"2024-04-25T10:16:08.083408+0000","last_peered":"2024-04-25T10:16:08.083408+0000","last_clean":"2024-04-25T10:16:08.083408+0000","last_became_active":"2024-04-25T10:16:08.071404+0000","last_became_peered":"2024-04-25T10:16:08.071404+0000","last_unstale":"2024-04-25T10:16:08.083408+0000","last_undegraded":"2024-04-25T10:16:08.083408+0000","last_fullsized":"2024-04-25T10:16:08.083408+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:43.193925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.193192+0000","last_change":"2024-04-25T10:16:08.078036+0000","last_active":"2024-04-25T10:16:10.193192+0000","last_peered":"2024-04-25T10:16:10.193192+0000","last_clean":"2024-04-25T10:16:10.193192+0000","last_became_active":"2024-04-25T10:16:08.071630+0000","last_became_peered":"2024-04-25T10:16:08.071630+0000","last_unstale":"2024-04-25T10:16:10.193192+0000","last_undegraded":"2024-04-25T10:16:10.193192+0000","last_fullsized":"2024-04-25T10:16:10.193192+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:03.913858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.082212+0000","last_change":"2024-04-25T10:16:08.070333+0000","last_active":"2024-04-25T10:16:08.082212+0000","last_peered":"2024-04-25T10:16:08.082212+0000","last_clean":"2024-04-25T10:16:08.082212+0000","last_became_active":"2024-04-25T10:16:08.069979+0000","last_became_peered":"2024-04-25T10:16:08.069979+0000","last_unstale":"2024-04-25T10:16:08.082212+0000","last_undegraded":"2024-04-25T10:16:08.082212+0000","last_fullsized":"2024-04-25T10:16:08.082212+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:45.689885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192429+0000","last_change":"2024-04-25T10:16:08.081129+0000","last_active":"2024-04-25T10:16:09.192429+0000","last_peered":"2024-04-25T10:16:09.192429+0000","last_clean":"2024-04-25T10:16:09.192429+0000","last_became_active":"2024-04-25T10:16:08.079928+0000","last_became_peered":"2024-04-25T10:16:08.079928+0000","last_unstale":"2024-04-25T10:16:09.192429+0000","last_undegraded":"2024-04-25T10:16:09.192429+0000","last_fullsized":"2024-04-25T10:16:09.192429+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:11.981763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.295425+0000","last_change":"2024-04-25T10:16:08.077108+0000","last_active":"2024-04-25T10:16:10.295425+0000","last_peered":"2024-04-25T10:16:10.295425+0000","last_clean":"2024-04-25T10:16:10.295425+0000","last_became_active":"2024-04-25T10:16:08.076296+0000","last_became_peered":"2024-04-25T10:16:08.076296+0000","last_unstale":"2024-04-25T10:16:10.295425+0000","last_undegraded":"2024-04-25T10:16:10.295425+0000","last_fullsized":"2024-04-25T10:16:10.295425+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:37.996501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.193751+0000","last_change":"2024-04-25T10:16:08.091843+0000","last_active":"2024-04-25T10:16:10.193751+0000","last_peered":"2024-04-25T10:16:10.193751+0000","last_clean":"2024-04-25T10:16:10.193751+0000","last_became_active":"2024-04-25T10:16:08.091095+0000","last_became_peered":"2024-04-25T10:16:08.091095+0000","last_unstale":"2024-04-25T10:16:10.193751+0000","last_undegraded":"2024-04-25T10:16:10.193751+0000","last_fullsized":"2024-04-25T10:16:10.193751+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:47.250605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.078429+0000","last_change":"2024-04-25T10:16:08.066140+0000","last_active":"2024-04-25T10:16:08.078429+0000","last_peered":"2024-04-25T10:16:08.078429+0000","last_clean":"2024-04-25T10:16:08.078429+0000","last_became_active":"2024-04-25T10:16:08.065191+0000","last_became_peered":"2024-04-25T10:16:08.065191+0000","last_unstale":"2024-04-25T10:16:08.078429+0000","last_undegraded":"2024-04-25T10:16:08.078429+0000","last_fullsized":"2024-04-25T10:16:08.078429+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:27.960973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.064278+0000","last_change":"2024-04-25T10:16:08.076665+0000","last_active":"2024-04-25T10:16:09.064278+0000","last_peered":"2024-04-25T10:16:09.064278+0000","last_clean":"2024-04-25T10:16:09.064278+0000","last_became_active":"2024-04-25T10:16:08.076274+0000","last_became_peered":"2024-04-25T10:16:08.076274+0000","last_unstale":"2024-04-25T10:16:09.064278+0000","last_undegraded":"2024-04-25T10:16:09.064278+0000","last_fullsized":"2024-04-25T10:16:09.064278+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:09.866700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192039+0000","last_change":"2024-04-25T10:16:08.081986+0000","last_active":"2024-04-25T10:16:09.192039+0000","last_peered":"2024-04-25T10:16:09.192039+0000","last_clean":"2024-04-25T10:16:09.192039+0000","last_became_active":"2024-04-25T10:16:08.081140+0000","last_became_peered":"2024-04-25T10:16:08.081140+0000","last_unstale":"2024-04-25T10:16:09.192039+0000","last_undegraded":"2024-04-25T10:16:09.192039+0000","last_fullsized":"2024-04-25T10:16:09.192039+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:00.912328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.294607+0000","last_change":"2024-04-25T10:16:08.091185+0000","last_active":"2024-04-25T10:16:10.294607+0000","last_peered":"2024-04-25T10:16:10.294607+0000","last_clean":"2024-04-25T10:16:10.294607+0000","last_became_active":"2024-04-25T10:16:08.090706+0000","last_became_peered":"2024-04-25T10:16:08.090706+0000","last_unstale":"2024-04-25T10:16:10.294607+0000","last_undegraded":"2024-04-25T10:16:10.294607+0000","last_fullsized":"2024-04-25T10:16:10.294607+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:03.193773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.295276+0000","last_change":"2024-04-25T10:16:08.075203+0000","last_active":"2024-04-25T10:16:10.295276+0000","last_peered":"2024-04-25T10:16:10.295276+0000","last_clean":"2024-04-25T10:16:10.295276+0000","last_became_active":"2024-04-25T10:16:08.074529+0000","last_became_peered":"2024-04-25T10:16:08.074529+0000","last_unstale":"2024-04-25T10:16:10.295276+0000","last_undegraded":"2024-04-25T10:16:10.295276+0000","last_fullsized":"2024-04-25T10:16:10.295276+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:33.382025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"109'4","reported_seq":18,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.064819+0000","last_change":"2024-04-25T10:16:07.412642+0000","last_active":"2024-04-25T10:16:09.064819+0000","last_peered":"2024-04-25T10:16:09.064819+0000","last_clean":"2024-04-25T10:16:09.064819+0000","last_became_active":"2024-04-25T10:16:07.412025+0000","last_became_peered":"2024-04-25T10:16:07.412025+0000","last_unstale":"2024-04-25T10:16:09.064819+0000","last_undegraded":"2024-04-25T10:16:09.064819+0000","last_fullsized":"2024-04-25T10:16:09.064819+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:57.557432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.300552+0000","last_change":"2024-04-25T10:16:08.070525+0000","last_active":"2024-04-25T10:16:10.300552+0000","last_peered":"2024-04-25T10:16:10.300552+0000","last_clean":"2024-04-25T10:16:10.300552+0000","last_became_active":"2024-04-25T10:16:08.069821+0000","last_became_peered":"2024-04-25T10:16:08.069821+0000","last_unstale":"2024-04-25T10:16:10.300552+0000","last_undegraded":"2024-04-25T10:16:10.300552+0000","last_fullsized":"2024-04-25T10:16:10.300552+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:19.218376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.415355+0000","last_change":"2024-04-25T10:16:07.413012+0000","last_active":"2024-04-25T10:16:08.415355+0000","last_peered":"2024-04-25T10:16:08.415355+0000","last_clean":"2024-04-25T10:16:08.415355+0000","last_became_active":"2024-04-25T10:16:07.412205+0000","last_became_peered":"2024-04-25T10:16:07.412205+0000","last_unstale":"2024-04-25T10:16:08.415355+0000","last_undegraded":"2024-04-25T10:16:08.415355+0000","last_fullsized":"2024-04-25T10:16:08.415355+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:54.134548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.295553+0000","last_change":"2024-04-25T10:16:08.078592+0000","last_active":"2024-04-25T10:16:10.295553+0000","last_peered":"2024-04-25T10:16:10.295553+0000","last_clean":"2024-04-25T10:16:10.295553+0000","last_became_active":"2024-04-25T10:16:08.077787+0000","last_became_peered":"2024-04-25T10:16:08.077787+0000","last_unstale":"2024-04-25T10:16:10.295553+0000","last_undegraded":"2024-04-25T10:16:10.295553+0000","last_fullsized":"2024-04-25T10:16:10.295553+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:27.548407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.055913+0000","last_change":"2024-04-25T10:16:07.422712+0000","last_active":"2024-04-25T10:16:09.055913+0000","last_peered":"2024-04-25T10:16:09.055913+0000","last_clean":"2024-04-25T10:16:09.055913+0000","last_became_active":"2024-04-25T10:16:07.422427+0000","last_became_peered":"2024-04-25T10:16:07.422427+0000","last_unstale":"2024-04-25T10:16:09.055913+0000","last_undegraded":"2024-04-25T10:16:09.055913+0000","last_fullsized":"2024-04-25T10:16:09.055913+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:04.957872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.096843+0000","last_change":"2024-04-25T10:16:08.090394+0000","last_active":"2024-04-25T10:16:08.096843+0000","last_peered":"2024-04-25T10:16:08.096843+0000","last_clean":"2024-04-25T10:16:08.096843+0000","last_became_active":"2024-04-25T10:16:08.090054+0000","last_became_peered":"2024-04-25T10:16:08.090054+0000","last_unstale":"2024-04-25T10:16:08.096843+0000","last_undegraded":"2024-04-25T10:16:08.096843+0000","last_fullsized":"2024-04-25T10:16:08.096843+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:32.765726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.055628+0000","last_change":"2024-04-25T10:16:07.418521+0000","last_active":"2024-04-25T10:16:09.055628+0000","last_peered":"2024-04-25T10:16:09.055628+0000","last_clean":"2024-04-25T10:16:09.055628+0000","last_became_active":"2024-04-25T10:16:07.418004+0000","last_became_peered":"2024-04-25T10:16:07.418004+0000","last_unstale":"2024-04-25T10:16:09.055628+0000","last_undegraded":"2024-04-25T10:16:09.055628+0000","last_fullsized":"2024-04-25T10:16:09.055628+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:05.653698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192069+0000","last_change":"2024-04-25T10:16:08.097718+0000","last_active":"2024-04-25T10:16:09.192069+0000","last_peered":"2024-04-25T10:16:09.192069+0000","last_clean":"2024-04-25T10:16:09.192069+0000","last_became_active":"2024-04-25T10:16:08.097048+0000","last_became_peered":"2024-04-25T10:16:08.097048+0000","last_unstale":"2024-04-25T10:16:09.192069+0000","last_undegraded":"2024-04-25T10:16:09.192069+0000","last_fullsized":"2024-04-25T10:16:09.192069+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:38.527331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.064124+0000","last_change":"2024-04-25T10:16:07.423900+0000","last_active":"2024-04-25T10:16:09.064124+0000","last_peered":"2024-04-25T10:16:09.064124+0000","last_clean":"2024-04-25T10:16:09.064124+0000","last_became_active":"2024-04-25T10:16:07.423111+0000","last_became_peered":"2024-04-25T10:16:07.423111+0000","last_unstale":"2024-04-25T10:16:09.064124+0000","last_undegraded":"2024-04-25T10:16:09.064124+0000","last_fullsized":"2024-04-25T10:16:09.064124+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:31.120952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.077789+0000","last_change":"2024-04-25T10:16:08.065852+0000","last_active":"2024-04-25T10:16:08.077789+0000","last_peered":"2024-04-25T10:16:08.077789+0000","last_clean":"2024-04-25T10:16:08.077789+0000","last_became_active":"2024-04-25T10:16:08.065044+0000","last_became_peered":"2024-04-25T10:16:08.065044+0000","last_unstale":"2024-04-25T10:16:08.077789+0000","last_undegraded":"2024-04-25T10:16:08.077789+0000","last_fullsized":"2024-04-25T10:16:08.077789+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:27.548407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.193761+0000","last_change":"2024-04-25T10:16:07.419309+0000","last_active":"2024-04-25T10:16:10.193761+0000","last_peered":"2024-04-25T10:16:10.193761+0000","last_clean":"2024-04-25T10:16:10.193761+0000","last_became_active":"2024-04-25T10:16:07.419014+0000","last_became_peered":"2024-04-25T10:16:07.419014+0000","last_unstale":"2024-04-25T10:16:10.193761+0000","last_undegraded":"2024-04-25T10:16:10.193761+0000","last_fullsized":"2024-04-25T10:16:10.193761+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:58.952027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.191887+0000","last_change":"2024-04-25T10:16:08.091496+0000","last_active":"2024-04-25T10:16:09.191887+0000","last_peered":"2024-04-25T10:16:09.191887+0000","last_clean":"2024-04-25T10:16:09.191887+0000","last_became_active":"2024-04-25T10:16:08.087304+0000","last_became_peered":"2024-04-25T10:16:08.087304+0000","last_unstale":"2024-04-25T10:16:09.191887+0000","last_undegraded":"2024-04-25T10:16:09.191887+0000","last_fullsized":"2024-04-25T10:16:09.191887+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:25.704962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.052148+0000","last_change":"2024-04-25T10:16:07.422002+0000","last_active":"2024-04-25T10:16:08.052148+0000","last_peered":"2024-04-25T10:16:08.052148+0000","last_clean":"2024-04-25T10:16:08.052148+0000","last_became_active":"2024-04-25T10:16:07.421210+0000","last_became_peered":"2024-04-25T10:16:07.421210+0000","last_unstale":"2024-04-25T10:16:08.052148+0000","last_undegraded":"2024-04-25T10:16:08.052148+0000","last_fullsized":"2024-04-25T10:16:08.052148+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:32.378200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.064071+0000","last_change":"2024-04-25T10:16:08.077768+0000","last_active":"2024-04-25T10:16:09.064071+0000","last_peered":"2024-04-25T10:16:09.064071+0000","last_clean":"2024-04-25T10:16:09.064071+0000","last_became_active":"2024-04-25T10:16:08.077323+0000","last_became_peered":"2024-04-25T10:16:08.077323+0000","last_unstale":"2024-04-25T10:16:09.064071+0000","last_undegraded":"2024-04-25T10:16:09.064071+0000","last_fullsized":"2024-04-25T10:16:09.064071+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:03.782312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192053+0000","last_change":"2024-04-25T10:16:07.429065+0000","last_active":"2024-04-25T10:16:09.192053+0000","last_peered":"2024-04-25T10:16:09.192053+0000","last_clean":"2024-04-25T10:16:09.192053+0000","last_became_active":"2024-04-25T10:16:07.428582+0000","last_became_peered":"2024-04-25T10:16:07.428582+0000","last_unstale":"2024-04-25T10:16:09.192053+0000","last_undegraded":"2024-04-25T10:16:09.192053+0000","last_fullsized":"2024-04-25T10:16:09.192053+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:01.425214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.065122+0000","last_change":"2024-04-25T10:16:08.078397+0000","last_active":"2024-04-25T10:16:09.065122+0000","last_peered":"2024-04-25T10:16:09.065122+0000","last_clean":"2024-04-25T10:16:09.065122+0000","last_became_active":"2024-04-25T10:16:08.077753+0000","last_became_peered":"2024-04-25T10:16:08.077753+0000","last_unstale":"2024-04-25T10:16:09.065122+0000","last_undegraded":"2024-04-25T10:16:09.065122+0000","last_fullsized":"2024-04-25T10:16:09.065122+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:26.713626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.055582+0000","last_change":"2024-04-25T10:16:07.416827+0000","last_active":"2024-04-25T10:16:09.055582+0000","last_peered":"2024-04-25T10:16:09.055582+0000","last_clean":"2024-04-25T10:16:09.055582+0000","last_became_active":"2024-04-25T10:16:07.416558+0000","last_became_peered":"2024-04-25T10:16:07.416558+0000","last_unstale":"2024-04-25T10:16:09.055582+0000","last_undegraded":"2024-04-25T10:16:09.055582+0000","last_fullsized":"2024-04-25T10:16:09.055582+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:53.782139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.300961+0000","last_change":"2024-04-25T10:16:08.099653+0000","last_active":"2024-04-25T10:16:10.300961+0000","last_peered":"2024-04-25T10:16:10.300961+0000","last_clean":"2024-04-25T10:16:10.300961+0000","last_became_active":"2024-04-25T10:16:08.099226+0000","last_became_peered":"2024-04-25T10:16:08.099226+0000","last_unstale":"2024-04-25T10:16:10.300961+0000","last_undegraded":"2024-04-25T10:16:10.300961+0000","last_fullsized":"2024-04-25T10:16:10.300961+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:12.393084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.194103+0000","last_change":"2024-04-25T10:16:07.421886+0000","last_active":"2024-04-25T10:16:10.194103+0000","last_peered":"2024-04-25T10:16:10.194103+0000","last_clean":"2024-04-25T10:16:10.194103+0000","last_became_active":"2024-04-25T10:16:07.421532+0000","last_became_peered":"2024-04-25T10:16:07.421532+0000","last_unstale":"2024-04-25T10:16:10.194103+0000","last_undegraded":"2024-04-25T10:16:10.194103+0000","last_fullsized":"2024-04-25T10:16:10.194103+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:28.478129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.193895+0000","last_change":"2024-04-25T10:16:08.076815+0000","last_active":"2024-04-25T10:16:10.193895+0000","last_peered":"2024-04-25T10:16:10.193895+0000","last_clean":"2024-04-25T10:16:10.193895+0000","last_became_active":"2024-04-25T10:16:08.076064+0000","last_became_peered":"2024-04-25T10:16:08.076064+0000","last_unstale":"2024-04-25T10:16:10.193895+0000","last_undegraded":"2024-04-25T10:16:10.193895+0000","last_fullsized":"2024-04-25T10:16:10.193895+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:20.195078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"109'3","reported_seq":17,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192245+0000","last_change":"2024-04-25T10:16:07.418216+0000","last_active":"2024-04-25T10:16:09.192245+0000","last_peered":"2024-04-25T10:16:09.192245+0000","last_clean":"2024-04-25T10:16:09.192245+0000","last_became_active":"2024-04-25T10:16:07.417942+0000","last_became_peered":"2024-04-25T10:16:07.417942+0000","last_unstale":"2024-04-25T10:16:09.192245+0000","last_undegraded":"2024-04-25T10:16:09.192245+0000","last_fullsized":"2024-04-25T10:16:09.192245+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:40.383900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.064693+0000","last_change":"2024-04-25T10:16:08.068349+0000","last_active":"2024-04-25T10:16:09.064693+0000","last_peered":"2024-04-25T10:16:09.064693+0000","last_clean":"2024-04-25T10:16:09.064693+0000","last_became_active":"2024-04-25T10:16:08.067886+0000","last_became_peered":"2024-04-25T10:16:08.067886+0000","last_unstale":"2024-04-25T10:16:09.064693+0000","last_undegraded":"2024-04-25T10:16:09.064693+0000","last_fullsized":"2024-04-25T10:16:09.064693+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:51.197611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"109'6","reported_seq":20,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192906+0000","last_change":"2024-04-25T10:16:07.418178+0000","last_active":"2024-04-25T10:16:09.192906+0000","last_peered":"2024-04-25T10:16:09.192906+0000","last_clean":"2024-04-25T10:16:09.192906+0000","last_became_active":"2024-04-25T10:16:07.417805+0000","last_became_peered":"2024-04-25T10:16:07.417805+0000","last_unstale":"2024-04-25T10:16:09.192906+0000","last_undegraded":"2024-04-25T10:16:09.192906+0000","last_fullsized":"2024-04-25T10:16:09.192906+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:48.346969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.193637+0000","last_change":"2024-04-25T10:16:08.077015+0000","last_active":"2024-04-25T10:16:10.193637+0000","last_peered":"2024-04-25T10:16:10.193637+0000","last_clean":"2024-04-25T10:16:10.193637+0000","last_became_active":"2024-04-25T10:16:08.076264+0000","last_became_peered":"2024-04-25T10:16:08.076264+0000","last_unstale":"2024-04-25T10:16:10.193637+0000","last_undegraded":"2024-04-25T10:16:10.193637+0000","last_fullsized":"2024-04-25T10:16:10.193637+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:23.783432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.294664+0000","last_change":"2024-04-25T10:16:07.414135+0000","last_active":"2024-04-25T10:16:10.294664+0000","last_peered":"2024-04-25T10:16:10.294664+0000","last_clean":"2024-04-25T10:16:10.294664+0000","last_became_active":"2024-04-25T10:16:07.413293+0000","last_became_peered":"2024-04-25T10:16:07.413293+0000","last_unstale":"2024-04-25T10:16:10.294664+0000","last_undegraded":"2024-04-25T10:16:10.294664+0000","last_fullsized":"2024-04-25T10:16:10.294664+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:48.875918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.193742+0000","last_change":"2024-04-25T10:16:08.077580+0000","last_active":"2024-04-25T10:16:10.193742+0000","last_peered":"2024-04-25T10:16:10.193742+0000","last_clean":"2024-04-25T10:16:10.193742+0000","last_became_active":"2024-04-25T10:16:08.077100+0000","last_became_peered":"2024-04-25T10:16:08.077100+0000","last_unstale":"2024-04-25T10:16:10.193742+0000","last_undegraded":"2024-04-25T10:16:10.193742+0000","last_fullsized":"2024-04-25T10:16:10.193742+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:49.350794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.300001+0000","last_change":"2024-04-25T10:16:07.430570+0000","last_active":"2024-04-25T10:16:10.300001+0000","last_peered":"2024-04-25T10:16:10.300001+0000","last_clean":"2024-04-25T10:16:10.300001+0000","last_became_active":"2024-04-25T10:16:07.430098+0000","last_became_peered":"2024-04-25T10:16:07.430098+0000","last_unstale":"2024-04-25T10:16:10.300001+0000","last_undegraded":"2024-04-25T10:16:10.300001+0000","last_fullsized":"2024-04-25T10:16:10.300001+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:50.193785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.193260+0000","last_change":"2024-04-25T10:16:08.071688+0000","last_active":"2024-04-25T10:16:10.193260+0000","last_peered":"2024-04-25T10:16:10.193260+0000","last_clean":"2024-04-25T10:16:10.193260+0000","last_became_active":"2024-04-25T10:16:08.071124+0000","last_became_peered":"2024-04-25T10:16:08.071124+0000","last_unstale":"2024-04-25T10:16:10.193260+0000","last_undegraded":"2024-04-25T10:16:10.193260+0000","last_fullsized":"2024-04-25T10:16:10.193260+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:55.196394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"109'8","reported_seq":22,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.055413+0000","last_change":"2024-04-25T10:16:07.418621+0000","last_active":"2024-04-25T10:16:09.055413+0000","last_peered":"2024-04-25T10:16:09.055413+0000","last_clean":"2024-04-25T10:16:09.055413+0000","last_became_active":"2024-04-25T10:16:07.417941+0000","last_became_peered":"2024-04-25T10:16:07.417941+0000","last_unstale":"2024-04-25T10:16:09.055413+0000","last_undegraded":"2024-04-25T10:16:09.055413+0000","last_fullsized":"2024-04-25T10:16:09.055413+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:16.808663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.056050+0000","last_change":"2024-04-25T10:16:08.092687+0000","last_active":"2024-04-25T10:16:09.056050+0000","last_peered":"2024-04-25T10:16:09.056050+0000","last_clean":"2024-04-25T10:16:09.056050+0000","last_became_active":"2024-04-25T10:16:08.092207+0000","last_became_peered":"2024-04-25T10:16:08.092207+0000","last_unstale":"2024-04-25T10:16:09.056050+0000","last_undegraded":"2024-04-25T10:16:09.056050+0000","last_fullsized":"2024-04-25T10:16:09.056050+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:20.798239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192634+0000","last_change":"2024-04-25T10:16:07.426643+0000","last_active":"2024-04-25T10:16:09.192634+0000","last_peered":"2024-04-25T10:16:09.192634+0000","last_clean":"2024-04-25T10:16:09.192634+0000","last_became_active":"2024-04-25T10:16:07.417621+0000","last_became_peered":"2024-04-25T10:16:07.417621+0000","last_unstale":"2024-04-25T10:16:09.192634+0000","last_undegraded":"2024-04-25T10:16:09.192634+0000","last_fullsized":"2024-04-25T10:16:09.192634+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:40.856308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.295365+0000","last_change":"2024-04-25T10:16:08.096151+0000","last_active":"2024-04-25T10:16:10.295365+0000","last_peered":"2024-04-25T10:16:10.295365+0000","last_clean":"2024-04-25T10:16:10.295365+0000","last_became_active":"2024-04-25T10:16:08.095645+0000","last_became_peered":"2024-04-25T10:16:08.095645+0000","last_unstale":"2024-04-25T10:16:10.295365+0000","last_undegraded":"2024-04-25T10:16:10.295365+0000","last_fullsized":"2024-04-25T10:16:10.295365+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:30.822932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.056017+0000","last_change":"2024-04-25T10:16:07.420219+0000","last_active":"2024-04-25T10:16:09.056017+0000","last_peered":"2024-04-25T10:16:09.056017+0000","last_clean":"2024-04-25T10:16:09.056017+0000","last_became_active":"2024-04-25T10:16:07.419584+0000","last_became_peered":"2024-04-25T10:16:07.419584+0000","last_unstale":"2024-04-25T10:16:09.056017+0000","last_undegraded":"2024-04-25T10:16:09.056017+0000","last_fullsized":"2024-04-25T10:16:09.056017+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:49.956448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.099281+0000","last_change":"2024-04-25T10:16:08.098323+0000","last_active":"2024-04-25T10:16:08.099281+0000","last_peered":"2024-04-25T10:16:08.099281+0000","last_clean":"2024-04-25T10:16:08.099281+0000","last_became_active":"2024-04-25T10:16:08.097919+0000","last_became_peered":"2024-04-25T10:16:08.097919+0000","last_unstale":"2024-04-25T10:16:08.099281+0000","last_undegraded":"2024-04-25T10:16:08.099281+0000","last_fullsized":"2024-04-25T10:16:08.099281+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:45.513718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.299976+0000","last_change":"2024-04-25T10:16:07.420743+0000","last_active":"2024-04-25T10:16:10.299976+0000","last_peered":"2024-04-25T10:16:10.299976+0000","last_clean":"2024-04-25T10:16:10.299976+0000","last_became_active":"2024-04-25T10:16:07.419996+0000","last_became_peered":"2024-04-25T10:16:07.419996+0000","last_unstale":"2024-04-25T10:16:10.299976+0000","last_undegraded":"2024-04-25T10:16:10.299976+0000","last_fullsized":"2024-04-25T10:16:10.299976+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:25.709800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.096755+0000","last_change":"2024-04-25T10:16:08.089387+0000","last_active":"2024-04-25T10:16:08.096755+0000","last_peered":"2024-04-25T10:16:08.096755+0000","last_clean":"2024-04-25T10:16:08.096755+0000","last_became_active":"2024-04-25T10:16:08.084835+0000","last_became_peered":"2024-04-25T10:16:08.084835+0000","last_unstale":"2024-04-25T10:16:08.096755+0000","last_undegraded":"2024-04-25T10:16:08.096755+0000","last_fullsized":"2024-04-25T10:16:08.096755+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:27.686840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.064141+0000","last_change":"2024-04-25T10:16:07.432911+0000","last_active":"2024-04-25T10:16:09.064141+0000","last_peered":"2024-04-25T10:16:09.064141+0000","last_clean":"2024-04-25T10:16:09.064141+0000","last_became_active":"2024-04-25T10:16:07.432458+0000","last_became_peered":"2024-04-25T10:16:07.432458+0000","last_unstale":"2024-04-25T10:16:09.064141+0000","last_undegraded":"2024-04-25T10:16:09.064141+0000","last_fullsized":"2024-04-25T10:16:09.064141+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:38.969589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.096742+0000","last_change":"2024-04-25T10:16:08.089355+0000","last_active":"2024-04-25T10:16:08.096742+0000","last_peered":"2024-04-25T10:16:08.096742+0000","last_clean":"2024-04-25T10:16:08.096742+0000","last_became_active":"2024-04-25T10:16:08.086653+0000","last_became_peered":"2024-04-25T10:16:08.086653+0000","last_unstale":"2024-04-25T10:16:08.096742+0000","last_undegraded":"2024-04-25T10:16:08.096742+0000","last_fullsized":"2024-04-25T10:16:08.096742+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:22.394766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.294780+0000","last_change":"2024-04-25T10:16:07.411824+0000","last_active":"2024-04-25T10:16:10.294780+0000","last_peered":"2024-04-25T10:16:10.294780+0000","last_clean":"2024-04-25T10:16:10.294780+0000","last_became_active":"2024-04-25T10:16:07.407193+0000","last_became_peered":"2024-04-25T10:16:07.407193+0000","last_unstale":"2024-04-25T10:16:10.294780+0000","last_undegraded":"2024-04-25T10:16:10.294780+0000","last_fullsized":"2024-04-25T10:16:10.294780+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:57.039442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.191994+0000","last_change":"2024-04-25T10:16:08.087872+0000","last_active":"2024-04-25T10:16:09.191994+0000","last_peered":"2024-04-25T10:16:09.191994+0000","last_clean":"2024-04-25T10:16:09.191994+0000","last_became_active":"2024-04-25T10:16:08.086790+0000","last_became_peered":"2024-04-25T10:16:08.086790+0000","last_unstale":"2024-04-25T10:16:09.191994+0000","last_undegraded":"2024-04-25T10:16:09.191994+0000","last_fullsized":"2024-04-25T10:16:09.191994+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:27.821765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.055427+0000","last_change":"2024-04-25T10:16:07.413599+0000","last_active":"2024-04-25T10:16:09.055427+0000","last_peered":"2024-04-25T10:16:09.055427+0000","last_clean":"2024-04-25T10:16:09.055427+0000","last_became_active":"2024-04-25T10:16:07.413293+0000","last_became_peered":"2024-04-25T10:16:07.413293+0000","last_unstale":"2024-04-25T10:16:09.055427+0000","last_undegraded":"2024-04-25T10:16:09.055427+0000","last_fullsized":"2024-04-25T10:16:09.055427+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:05.252407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.096725+0000","last_change":"2024-04-25T10:16:08.089103+0000","last_active":"2024-04-25T10:16:08.096725+0000","last_peered":"2024-04-25T10:16:08.096725+0000","last_clean":"2024-04-25T10:16:08.096725+0000","last_became_active":"2024-04-25T10:16:08.085453+0000","last_became_peered":"2024-04-25T10:16:08.085453+0000","last_unstale":"2024-04-25T10:16:08.096725+0000","last_undegraded":"2024-04-25T10:16:08.096725+0000","last_fullsized":"2024-04-25T10:16:08.096725+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:59.638430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.294695+0000","last_change":"2024-04-25T10:16:07.412591+0000","last_active":"2024-04-25T10:16:10.294695+0000","last_peered":"2024-04-25T10:16:10.294695+0000","last_clean":"2024-04-25T10:16:10.294695+0000","last_became_active":"2024-04-25T10:16:07.411412+0000","last_became_peered":"2024-04-25T10:16:07.411412+0000","last_unstale":"2024-04-25T10:16:10.294695+0000","last_undegraded":"2024-04-25T10:16:10.294695+0000","last_fullsized":"2024-04-25T10:16:10.294695+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:07.599218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.193181+0000","last_change":"2024-04-25T10:16:08.095223+0000","last_active":"2024-04-25T10:16:10.193181+0000","last_peered":"2024-04-25T10:16:10.193181+0000","last_clean":"2024-04-25T10:16:10.193181+0000","last_became_active":"2024-04-25T10:16:08.093014+0000","last_became_peered":"2024-04-25T10:16:08.093014+0000","last_unstale":"2024-04-25T10:16:10.193181+0000","last_undegraded":"2024-04-25T10:16:10.193181+0000","last_fullsized":"2024-04-25T10:16:10.193181+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:32.015709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192273+0000","last_change":"2024-04-25T10:16:07.427610+0000","last_active":"2024-04-25T10:16:09.192273+0000","last_peered":"2024-04-25T10:16:09.192273+0000","last_clean":"2024-04-25T10:16:09.192273+0000","last_became_active":"2024-04-25T10:16:07.427304+0000","last_became_peered":"2024-04-25T10:16:07.427304+0000","last_unstale":"2024-04-25T10:16:09.192273+0000","last_undegraded":"2024-04-25T10:16:09.192273+0000","last_fullsized":"2024-04-25T10:16:09.192273+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:31.011884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.300806+0000","last_change":"2024-04-25T10:16:08.099939+0000","last_active":"2024-04-25T10:16:10.300806+0000","last_peered":"2024-04-25T10:16:10.300806+0000","last_clean":"2024-04-25T10:16:10.300806+0000","last_became_active":"2024-04-25T10:16:08.099328+0000","last_became_peered":"2024-04-25T10:16:08.099328+0000","last_unstale":"2024-04-25T10:16:10.300806+0000","last_undegraded":"2024-04-25T10:16:10.300806+0000","last_fullsized":"2024-04-25T10:16:10.300806+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:35.721440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.194075+0000","last_change":"2024-04-25T10:16:07.429884+0000","last_active":"2024-04-25T10:16:10.194075+0000","last_peered":"2024-04-25T10:16:10.194075+0000","last_clean":"2024-04-25T10:16:10.194075+0000","last_became_active":"2024-04-25T10:16:07.429534+0000","last_became_peered":"2024-04-25T10:16:07.429534+0000","last_unstale":"2024-04-25T10:16:10.194075+0000","last_undegraded":"2024-04-25T10:16:10.194075+0000","last_fullsized":"2024-04-25T10:16:10.194075+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:18.214551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192657+0000","last_change":"2024-04-25T10:16:08.091576+0000","last_active":"2024-04-25T10:16:09.192657+0000","last_peered":"2024-04-25T10:16:09.192657+0000","last_clean":"2024-04-25T10:16:09.192657+0000","last_became_active":"2024-04-25T10:16:08.085951+0000","last_became_peered":"2024-04-25T10:16:08.085951+0000","last_unstale":"2024-04-25T10:16:09.192657+0000","last_undegraded":"2024-04-25T10:16:09.192657+0000","last_fullsized":"2024-04-25T10:16:09.192657+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:36.626806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"110'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.301151+0000","last_change":"2024-04-25T10:16:07.423602+0000","last_active":"2024-04-25T10:16:10.301151+0000","last_peered":"2024-04-25T10:16:10.301151+0000","last_clean":"2024-04-25T10:16:10.301151+0000","last_became_active":"2024-04-25T10:16:07.422889+0000","last_became_peered":"2024-04-25T10:16:07.422889+0000","last_unstale":"2024-04-25T10:16:10.301151+0000","last_undegraded":"2024-04-25T10:16:10.301151+0000","last_fullsized":"2024-04-25T10:16:10.301151+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:58.250456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192855+0000","last_change":"2024-04-25T10:16:08.080323+0000","last_active":"2024-04-25T10:16:09.192855+0000","last_peered":"2024-04-25T10:16:09.192855+0000","last_clean":"2024-04-25T10:16:09.192855+0000","last_became_active":"2024-04-25T10:16:08.074431+0000","last_became_peered":"2024-04-25T10:16:08.074431+0000","last_unstale":"2024-04-25T10:16:09.192855+0000","last_undegraded":"2024-04-25T10:16:09.192855+0000","last_fullsized":"2024-04-25T10:16:09.192855+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:10.643846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192968+0000","last_change":"2024-04-25T10:16:07.427399+0000","last_active":"2024-04-25T10:16:09.192968+0000","last_peered":"2024-04-25T10:16:09.192968+0000","last_clean":"2024-04-25T10:16:09.192968+0000","last_became_active":"2024-04-25T10:16:07.426945+0000","last_became_peered":"2024-04-25T10:16:07.426945+0000","last_unstale":"2024-04-25T10:16:09.192968+0000","last_undegraded":"2024-04-25T10:16:09.192968+0000","last_fullsized":"2024-04-25T10:16:09.192968+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:46.246780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.300067+0000","last_change":"2024-04-25T10:16:08.076805+0000","last_active":"2024-04-25T10:16:10.300067+0000","last_peered":"2024-04-25T10:16:10.300067+0000","last_clean":"2024-04-25T10:16:10.300067+0000","last_became_active":"2024-04-25T10:16:08.076201+0000","last_became_peered":"2024-04-25T10:16:08.076201+0000","last_unstale":"2024-04-25T10:16:10.300067+0000","last_undegraded":"2024-04-25T10:16:10.300067+0000","last_fullsized":"2024-04-25T10:16:10.300067+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:09.692407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.194056+0000","last_change":"2024-04-25T10:16:07.423090+0000","last_active":"2024-04-25T10:16:10.194056+0000","last_peered":"2024-04-25T10:16:10.194056+0000","last_clean":"2024-04-25T10:16:10.194056+0000","last_became_active":"2024-04-25T10:16:07.422770+0000","last_became_peered":"2024-04-25T10:16:07.422770+0000","last_unstale":"2024-04-25T10:16:10.194056+0000","last_undegraded":"2024-04-25T10:16:10.194056+0000","last_fullsized":"2024-04-25T10:16:10.194056+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:08.688581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.078443+0000","last_change":"2024-04-25T10:16:08.060603+0000","last_active":"2024-04-25T10:16:08.078443+0000","last_peered":"2024-04-25T10:16:08.078443+0000","last_clean":"2024-04-25T10:16:08.078443+0000","last_became_active":"2024-04-25T10:16:08.059770+0000","last_became_peered":"2024-04-25T10:16:08.059770+0000","last_unstale":"2024-04-25T10:16:08.078443+0000","last_undegraded":"2024-04-25T10:16:08.078443+0000","last_fullsized":"2024-04-25T10:16:08.078443+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:37.996501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.064362+0000","last_change":"2024-04-25T10:16:07.425514+0000","last_active":"2024-04-25T10:16:09.064362+0000","last_peered":"2024-04-25T10:16:09.064362+0000","last_clean":"2024-04-25T10:16:09.064362+0000","last_became_active":"2024-04-25T10:16:07.424995+0000","last_became_peered":"2024-04-25T10:16:07.424995+0000","last_unstale":"2024-04-25T10:16:09.064362+0000","last_undegraded":"2024-04-25T10:16:09.064362+0000","last_fullsized":"2024-04-25T10:16:09.064362+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:41.134322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.083495+0000","last_change":"2024-04-25T10:16:08.078197+0000","last_active":"2024-04-25T10:16:08.083495+0000","last_peered":"2024-04-25T10:16:08.083495+0000","last_clean":"2024-04-25T10:16:08.083495+0000","last_became_active":"2024-04-25T10:16:08.077680+0000","last_became_peered":"2024-04-25T10:16:08.077680+0000","last_unstale":"2024-04-25T10:16:08.083495+0000","last_undegraded":"2024-04-25T10:16:08.083495+0000","last_fullsized":"2024-04-25T10:16:08.083495+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:06.032932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.052504+0000","last_change":"2024-04-25T10:16:07.428578+0000","last_active":"2024-04-25T10:16:08.052504+0000","last_peered":"2024-04-25T10:16:08.052504+0000","last_clean":"2024-04-25T10:16:08.052504+0000","last_became_active":"2024-04-25T10:16:07.427915+0000","last_became_peered":"2024-04-25T10:16:07.427915+0000","last_unstale":"2024-04-25T10:16:08.052504+0000","last_undegraded":"2024-04-25T10:16:08.052504+0000","last_fullsized":"2024-04-25T10:16:08.052504+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:29.819106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.096683+0000","last_change":"2024-04-25T10:16:08.086860+0000","last_active":"2024-04-25T10:16:08.096683+0000","last_peered":"2024-04-25T10:16:08.096683+0000","last_clean":"2024-04-25T10:16:08.096683+0000","last_became_active":"2024-04-25T10:16:08.085415+0000","last_became_peered":"2024-04-25T10:16:08.085415+0000","last_unstale":"2024-04-25T10:16:08.096683+0000","last_undegraded":"2024-04-25T10:16:08.096683+0000","last_fullsized":"2024-04-25T10:16:08.096683+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:37.647431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.055473+0000","last_change":"2024-04-25T10:16:07.402713+0000","last_active":"2024-04-25T10:16:09.055473+0000","last_peered":"2024-04-25T10:16:09.055473+0000","last_clean":"2024-04-25T10:16:09.055473+0000","last_became_active":"2024-04-25T10:16:07.402415+0000","last_became_peered":"2024-04-25T10:16:07.402415+0000","last_unstale":"2024-04-25T10:16:09.055473+0000","last_undegraded":"2024-04-25T10:16:09.055473+0000","last_fullsized":"2024-04-25T10:16:09.055473+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:42.908913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192178+0000","last_change":"2024-04-25T10:16:08.085375+0000","last_active":"2024-04-25T10:16:09.192178+0000","last_peered":"2024-04-25T10:16:09.192178+0000","last_clean":"2024-04-25T10:16:09.192178+0000","last_became_active":"2024-04-25T10:16:08.081301+0000","last_became_peered":"2024-04-25T10:16:08.081301+0000","last_unstale":"2024-04-25T10:16:09.192178+0000","last_undegraded":"2024-04-25T10:16:09.192178+0000","last_fullsized":"2024-04-25T10:16:09.192178+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:38.626839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.192275+0000","last_change":"2024-04-25T10:16:07.426844+0000","last_active":"2024-04-25T10:16:09.192275+0000","last_peered":"2024-04-25T10:16:09.192275+0000","last_clean":"2024-04-25T10:16:09.192275+0000","last_became_active":"2024-04-25T10:16:07.424170+0000","last_became_peered":"2024-04-25T10:16:07.424170+0000","last_unstale":"2024-04-25T10:16:09.192275+0000","last_undegraded":"2024-04-25T10:16:09.192275+0000","last_fullsized":"2024-04-25T10:16:09.192275+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:02.910032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:10.194059+0000","last_change":"2024-04-25T10:16:08.077908+0000","last_active":"2024-04-25T10:16:10.194059+0000","last_peered":"2024-04-25T10:16:10.194059+0000","last_clean":"2024-04-25T10:16:10.194059+0000","last_became_active":"2024-04-25T10:16:08.071078+0000","last_became_peered":"2024-04-25T10:16:08.071078+0000","last_unstale":"2024-04-25T10:16:10.194059+0000","last_undegraded":"2024-04-25T10:16:10.194059+0000","last_fullsized":"2024-04-25T10:16:10.194059+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.370666+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.370666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:49.746639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.065027+0000","last_change":"2024-04-25T10:16:07.432894+0000","last_active":"2024-04-25T10:16:09.065027+0000","last_peered":"2024-04-25T10:16:09.065027+0000","last_clean":"2024-04-25T10:16:09.065027+0000","last_became_active":"2024-04-25T10:16:07.432221+0000","last_became_peered":"2024-04-25T10:16:07.432221+0000","last_unstale":"2024-04-25T10:16:09.065027+0000","last_undegraded":"2024-04-25T10:16:09.065027+0000","last_fullsized":"2024-04-25T10:16:09.065027+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:06.366840+0000","last_clean_scrub_stamp":"2024-04-25T10:16:06.366840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:27.350848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":331,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:16:08.073129+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:16:08.073129+0000","last_peered":"2024-04-25T10:16:08.073129+0000","last_clean":"2024-04-25T10:16:08.073129+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:16:08.073129+0000","last_undegraded":"2024-04-25T10:16:08.073129+0000","last_fullsized":"2024-04-25T10:16:08.073129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509576,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1929216,"data_stored":1166063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1335296,"data_stored":575695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542279,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1363968,"data_stored":588840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1368064,"data_stored":589374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542280,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36692,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1376256,"data_stored":590452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542280,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36676,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335164,"statfs":{"total":96636764160,"available":96599207936,"internally_reserved":0,"allocated":1359872,"data_stored":588770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542280,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36676,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335164,"statfs":{"total":96636764160,"available":96599207936,"internally_reserved":0,"allocated":1368064,"data_stored":588894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542281,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37268,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1974272,"data_stored":1180862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:16:12.817 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:16:12.817 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:16:12.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.986+0000 7fd66e656640 1 Processor -- start 2024-04-25T10:16:12.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.986+0000 7fd66e656640 1 -- start start 2024-04-25T10:16:12.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.986+0000 7fd66e656640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd668161250 0x7fd6680b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.986+0000 7fd66e656640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd6680b5940 0x7fd6680d2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.986+0000 7fd66e656640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd668162200 con 0x7fd668160e70 2024-04-25T10:16:12.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.986+0000 7fd66e656640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6680d1a30 con 0x7fd6680b5940 2024-04-25T10:16:12.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.986+0000 7fd66e656640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6681624e0 con 0x7fd668161250 2024-04-25T10:16:12.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd66cbf3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd6680b5940 0x7fd6680d2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd66cbf3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd6680b5940 0x7fd6680d2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35292/0 (socket says 172.21.15.60:35292) 2024-04-25T10:16:12.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd66cbf3640 1 -- 172.21.15.60:0/2642705326 learned_addr learned my addr 172.21.15.60:0/2642705326 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:12.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd667fff640 1 --1- 172.21.15.60:0/2642705326 >> v1:172.21.15.60:6789/0 conn(0x7fd668160e70 0x7fd668161760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48660/0 (socket says 172.21.15.60:48660) 2024-04-25T10:16:12.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd65ffff640 1 --2- 172.21.15.60:0/2642705326 >> v2:172.21.15.100:3300/0 conn(0x7fd668161250 0x7fd6680b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd66cbf3640 1 -- 172.21.15.60:0/2642705326 >> v2:172.21.15.100:3300/0 conn(0x7fd668161250 msgr2=0x7fd6680b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:12.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd66cbf3640 1 --2- 172.21.15.60:0/2642705326 >> v2:172.21.15.100:3300/0 conn(0x7fd668161250 0x7fd6680b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd66cbf3640 1 -- 172.21.15.60:0/2642705326 >> v1:172.21.15.60:6789/0 conn(0x7fd668160e70 legacy=0x7fd668161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd66cbf3640 1 -- 172.21.15.60:0/2642705326 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6680d1c80 con 0x7fd6680b5940 2024-04-25T10:16:12.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd66cbf3640 1 --2- 172.21.15.60:0/2642705326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680b5940 0x7fd6680d2480 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fd660009e50 tx=0x7fd66001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a8bf5544b37ef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd6677fe640 1 -- 172.21.15.60:0/2642705326 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6600a5730 con 0x7fd6680b5940 2024-04-25T10:16:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd6677fe640 1 -- 172.21.15.60:0/2642705326 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6600a58d0 con 0x7fd6680b5940 2024-04-25T10:16:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd6677fe640 1 -- 172.21.15.60:0/2642705326 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6600a5ca0 con 0x7fd6680b5940 2024-04-25T10:16:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd66e656640 1 -- 172.21.15.60:0/2642705326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680b5940 msgr2=0x7fd6680d2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd66e656640 1 --2- 172.21.15.60:0/2642705326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680b5940 0x7fd6680d2480 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fd660009e50 tx=0x7fd66001c960 comp rx=0 tx=0).stop 2024-04-25T10:16:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd66e656640 1 -- 172.21.15.60:0/2642705326 shutdown_connections 2024-04-25T10:16:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd66e656640 1 --2- 172.21.15.60:0/2642705326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680b5940 0x7fd6680d2480 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd66e656640 1 --2- 172.21.15.60:0/2642705326 >> v2:172.21.15.100:3300/0 conn(0x7fd668161250 0x7fd6680b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd66e656640 1 -- 172.21.15.60:0/2642705326 >> 172.21.15.60:0/2642705326 conn(0x7fd6680d05d0 msgr2=0x7fd6680d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd66e656640 1 -- 172.21.15.60:0/2642705326 shutdown_connections 2024-04-25T10:16:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.990+0000 7fd66e656640 1 -- 172.21.15.60:0/2642705326 wait complete. 2024-04-25T10:16:12.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd66e656640 1 Processor -- start 2024-04-25T10:16:12.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd66e656640 1 -- start start 2024-04-25T10:16:13.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd66e656640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680b5940 0x7fd6680c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd66e656640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd668160e70 0x7fd6681534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd66e656640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd668161250 0x7fd6680cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd66e656640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680d3350 con 0x7fd6680b5940 2024-04-25T10:16:13.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd66e656640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680d31d0 con 0x7fd668160e70 2024-04-25T10:16:13.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd66e656640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680d34d0 con 0x7fd668161250 2024-04-25T10:16:13.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd667fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680b5940 0x7fd6680c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd667fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680b5940 0x7fd6680c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35300/0 (socket says 172.21.15.60:35300) 2024-04-25T10:16:13.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd667fff640 1 -- 172.21.15.60:0/3881683373 learned_addr learned my addr 172.21.15.60:0/3881683373 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:13.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd667fff640 1 -- 172.21.15.60:0/3881683373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd668161250 msgr2=0x7fd6680cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd66cbf3640 1 --2- 172.21.15.60:0/3881683373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd668161250 0x7fd6680cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd65ffff640 1 --2- 172.21.15.60:0/3881683373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd668160e70 0x7fd6681534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd667fff640 1 --2- 172.21.15.60:0/3881683373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd668161250 0x7fd6680cc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd667fff640 1 -- 172.21.15.60:0/3881683373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd668160e70 msgr2=0x7fd6681534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd667fff640 1 --2- 172.21.15.60:0/3881683373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd668160e70 0x7fd6681534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd667fff640 1 -- 172.21.15.60:0/3881683373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6680c82c0 con 0x7fd6680b5940 2024-04-25T10:16:13.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd65ffff640 1 --2- 172.21.15.60:0/3881683373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd668160e70 0x7fd6681534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:13.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd667fff640 1 --2- 172.21.15.60:0/3881683373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680b5940 0x7fd6680c9360 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fd65800c420 tx=0x7fd65800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd6657fa640 1 -- 172.21.15.60:0/3881683373 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6580074e0 con 0x7fd6680b5940 2024-04-25T10:16:13.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd6657fa640 1 -- 172.21.15.60:0/3881683373 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd658007680 con 0x7fd6680b5940 2024-04-25T10:16:13.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd66e656640 1 -- 172.21.15.60:0/3881683373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6680c85a0 con 0x7fd6680b5940 2024-04-25T10:16:13.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd6657fa640 1 -- 172.21.15.60:0/3881683373 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658004a30 con 0x7fd6680b5940 2024-04-25T10:16:13.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd66e656640 1 -- 172.21.15.60:0/3881683373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6680c8b60 con 0x7fd6680b5940 2024-04-25T10:16:13.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.994+0000 7fd66e656640 1 -- 172.21.15.60:0/3881683373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd668074770 con 0x7fd6680b5940 2024-04-25T10:16:13.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.998+0000 7fd6657fa640 1 -- 172.21.15.60:0/3881683373 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd65801a020 con 0x7fd6680b5940 2024-04-25T10:16:13.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.998+0000 7fd6657fa640 1 --2- 172.21.15.60:0/3881683373 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd648066f60 0x7fd648069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.998+0000 7fd6657fa640 1 -- 172.21.15.60:0/3881683373 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6580847f0 con 0x7fd6680b5940 2024-04-25T10:16:13.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.998+0000 7fd65ffff640 1 --2- 172.21.15.60:0/3881683373 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd648066f60 0x7fd648069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:12.998+0000 7fd65ffff640 1 --2- 172.21.15.60:0/3881683373 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd648066f60 0x7fd648069420 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd6680cd710 tx=0x7fd650008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.002+0000 7fd6657fa640 1 -- 172.21.15.60:0/3881683373 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd658055890 con 0x7fd6680b5940 2024-04-25T10:16:13.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.214+0000 7fd66e656640 1 -- 172.21.15.60:0/3881683373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd6680c7250 con 0x7fd6680b5940 2024-04-25T10:16:13.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.218+0000 7fd6657fa640 1 -- 172.21.15.60:0/3881683373 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd658055190 con 0x7fd6680b5940 2024-04-25T10:16:13.225 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T10:16:13.225 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCiKipmv65dLxAA6/QpPrqE9aT+CtZqkyutog== 2024-04-25T10:16:13.225 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-04-25T10:16:13.225 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-04-25T10:16:13.225 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:16:13.226 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-04-25T10:16:13.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.222+0000 7fd66e656640 1 -- 172.21.15.60:0/3881683373 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd648066f60 msgr2=0x7fd648069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.222+0000 7fd66e656640 1 --2- 172.21.15.60:0/3881683373 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd648066f60 0x7fd648069420 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd6680cd710 tx=0x7fd650008040 comp rx=0 tx=0).stop 2024-04-25T10:16:13.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.222+0000 7fd66e656640 1 -- 172.21.15.60:0/3881683373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680b5940 msgr2=0x7fd6680c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.222+0000 7fd66e656640 1 --2- 172.21.15.60:0/3881683373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680b5940 0x7fd6680c9360 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fd65800c420 tx=0x7fd65800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.222+0000 7fd66e656640 1 -- 172.21.15.60:0/3881683373 shutdown_connections 2024-04-25T10:16:13.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.222+0000 7fd66e656640 1 --2- 172.21.15.60:0/3881683373 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd648066f60 0x7fd648069420 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.222+0000 7fd66e656640 1 --2- 172.21.15.60:0/3881683373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd668161250 0x7fd6680cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.222+0000 7fd66e656640 1 --2- 172.21.15.60:0/3881683373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd668160e70 0x7fd6681534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.222+0000 7fd66e656640 1 --2- 172.21.15.60:0/3881683373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6680b5940 0x7fd6680c9360 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.222+0000 7fd66e656640 1 -- 172.21.15.60:0/3881683373 >> 172.21.15.60:0/3881683373 conn(0x7fd6680d05d0 msgr2=0x7fd668160110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:13.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.222+0000 7fd66e656640 1 -- 172.21.15.60:0/3881683373 shutdown_connections 2024-04-25T10:16:13.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.222+0000 7fd66e656640 1 -- 172.21.15.60:0/3881683373 wait complete. 2024-04-25T10:16:13.243 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:13.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.358+0000 7f9a57cb7640 1 Processor -- start 2024-04-25T10:16:13.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.358+0000 7f9a57cb7640 1 -- start start 2024-04-25T10:16:13.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a57cb7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a500ce7a0 0x7f9a500ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a57cb7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a500b4660 0x7f9a500b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a57cb7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a500cf0c0 con 0x7f9a5015bc90 2024-04-25T10:16:13.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a57cb7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a500cc370 con 0x7f9a500ce7a0 2024-04-25T10:16:13.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a57cb7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a50071ca0 con 0x7f9a500b4660 2024-04-25T10:16:13.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a55a53640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a500ce7a0 0x7f9a500ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a55a53640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a500ce7a0 0x7f9a500ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35312/0 (socket says 172.21.15.60:35312) 2024-04-25T10:16:13.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a55a53640 1 -- 172.21.15.60:0/2828634185 learned_addr learned my addr 172.21.15.60:0/2828634185 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:13.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a56254640 1 --2- 172.21.15.60:0/2828634185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a500b4660 0x7f9a500b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a55a53640 1 -- 172.21.15.60:0/2828634185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a500b4660 msgr2=0x7f9a500b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:13.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a55a53640 1 --2- 172.21.15.60:0/2828634185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a500b4660 0x7f9a500b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a55a53640 1 -- 172.21.15.60:0/2828634185 >> v1:172.21.15.100:6789/0 conn(0x7f9a5015bc90 legacy=0x7f9a500b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:13.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a55a53640 1 -- 172.21.15.60:0/2828634185 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a500cca40 con 0x7f9a500ce7a0 2024-04-25T10:16:13.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a55a53640 1 --2- 172.21.15.60:0/2828634185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a500ce7a0 0x7f9a500ceb80 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009e50 tx=0x7f9a4801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a674777fb6f7a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a54a51640 1 -- 172.21.15.60:0/2828634185 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a480a5670 con 0x7f9a500ce7a0 2024-04-25T10:16:13.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a54a51640 1 -- 172.21.15.60:0/2828634185 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a480a5810 con 0x7f9a500ce7a0 2024-04-25T10:16:13.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a54a51640 1 -- 172.21.15.60:0/2828634185 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a480a5be0 con 0x7f9a500ce7a0 2024-04-25T10:16:13.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a57cb7640 1 -- 172.21.15.60:0/2828634185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a500ce7a0 msgr2=0x7f9a500ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a57cb7640 1 --2- 172.21.15.60:0/2828634185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a500ce7a0 0x7f9a500ceb80 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009e50 tx=0x7f9a4801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a57cb7640 1 -- 172.21.15.60:0/2828634185 shutdown_connections 2024-04-25T10:16:13.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a57cb7640 1 --2- 172.21.15.60:0/2828634185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a500b4660 0x7f9a500b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a57cb7640 1 --2- 172.21.15.60:0/2828634185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a500ce7a0 0x7f9a500ceb80 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.362+0000 7f9a57cb7640 1 -- 172.21.15.60:0/2828634185 >> 172.21.15.60:0/2828634185 conn(0x7f9a500b12d0 msgr2=0x7f9a500b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:13.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a57cb7640 1 -- 172.21.15.60:0/2828634185 shutdown_connections 2024-04-25T10:16:13.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a57cb7640 1 -- 172.21.15.60:0/2828634185 wait complete. 2024-04-25T10:16:13.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a57cb7640 1 Processor -- start 2024-04-25T10:16:13.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a57cb7640 1 -- start start 2024-04-25T10:16:13.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a57cb7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a500ce7a0 0x7f9a50154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a57cb7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a5015bc90 0x7f9a500ab440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a57cb7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a50154dd0 0x7f9a500ab980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a57cb7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500b4400 con 0x7f9a50154dd0 2024-04-25T10:16:13.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a57cb7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500b4280 con 0x7f9a500ce7a0 2024-04-25T10:16:13.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a57cb7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500b3220 con 0x7f9a5015bc90 2024-04-25T10:16:13.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a56254640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a50154dd0 0x7f9a500ab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a56254640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a50154dd0 0x7f9a500ab980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35320/0 (socket says 172.21.15.60:35320) 2024-04-25T10:16:13.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a56254640 1 -- 172.21.15.60:0/2319207947 learned_addr learned my addr 172.21.15.60:0/2319207947 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:13.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a55a53640 1 --2- 172.21.15.60:0/2319207947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a500ce7a0 0x7f9a50154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a55252640 1 --2- 172.21.15.60:0/2319207947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a5015bc90 0x7f9a500ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a56254640 1 -- 172.21.15.60:0/2319207947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a5015bc90 msgr2=0x7f9a500ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a56254640 1 --2- 172.21.15.60:0/2319207947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a5015bc90 0x7f9a500ab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a56254640 1 -- 172.21.15.60:0/2319207947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a500ce7a0 msgr2=0x7f9a50154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a56254640 1 --2- 172.21.15.60:0/2319207947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a500ce7a0 0x7f9a50154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a56254640 1 -- 172.21.15.60:0/2319207947 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a500ac1a0 con 0x7f9a50154dd0 2024-04-25T10:16:13.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a56254640 1 --2- 172.21.15.60:0/2319207947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a50154dd0 0x7f9a500ab980 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c5f0 tx=0x7f9a4c00cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a467fc640 1 -- 172.21.15.60:0/2319207947 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c015070 con 0x7f9a50154dd0 2024-04-25T10:16:13.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a467fc640 1 -- 172.21.15.60:0/2319207947 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4c004220 con 0x7f9a50154dd0 2024-04-25T10:16:13.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a467fc640 1 -- 172.21.15.60:0/2319207947 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c003620 con 0x7f9a50154dd0 2024-04-25T10:16:13.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a57cb7640 1 -- 172.21.15.60:0/2319207947 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a500a9ae0 con 0x7f9a50154dd0 2024-04-25T10:16:13.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a57cb7640 1 -- 172.21.15.60:0/2319207947 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a500abec0 con 0x7f9a50154dd0 2024-04-25T10:16:13.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.366+0000 7f9a57cb7640 1 -- 172.21.15.60:0/2319207947 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a50074770 con 0x7f9a50154dd0 2024-04-25T10:16:13.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.370+0000 7f9a467fc640 1 -- 172.21.15.60:0/2319207947 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a4c0037c0 con 0x7f9a50154dd0 2024-04-25T10:16:13.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.370+0000 7f9a467fc640 1 --2- 172.21.15.60:0/2319207947 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a34066f60 0x7f9a34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.370+0000 7f9a467fc640 1 -- 172.21.15.60:0/2319207947 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a4c084e30 con 0x7f9a50154dd0 2024-04-25T10:16:13.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.370+0000 7f9a55a53640 1 --2- 172.21.15.60:0/2319207947 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a34066f60 0x7f9a34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.370+0000 7f9a55a53640 1 --2- 172.21.15.60:0/2319207947 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a34066f60 0x7f9a34069420 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9a480ae000 tx=0x7f9a480a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.374+0000 7f9a467fc640 1 -- 172.21.15.60:0/2319207947 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a4c055e50 con 0x7f9a50154dd0 2024-04-25T10:16:13.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.566+0000 7f9a57cb7640 1 -- 172.21.15.60:0/2319207947 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a500b36e0 con 0x7f9a50154dd0 2024-04-25T10:16:13.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.566+0000 7f9a467fc640 1 -- 172.21.15.60:0/2319207947 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f9a4c055750 con 0x7f9a50154dd0 2024-04-25T10:16:13.571 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:13.571 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2440975447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2440975447},{"type":"v1","addr":"172.21.15.100:6835","nonce":2440975447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2398080758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2398080758},{"type":"v1","addr":"172.21.15.60:6833","nonce":2398080758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2759861958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2759861958},{"type":"v1","addr":"172.21.15.100:6837","nonce":2759861958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:07.763322+0000","modified":"2024-04-25T10:16:10.765897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9381":{"gid":9381,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/722171246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":722171246},{"type":"v1","addr":"172.21.15.60:6835","nonce":722171246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:16:13.572 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 93 2024-04-25T10:16:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.570+0000 7f9a57cb7640 1 -- 172.21.15.60:0/2319207947 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a34066f60 msgr2=0x7f9a34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.570+0000 7f9a57cb7640 1 --2- 172.21.15.60:0/2319207947 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a34066f60 0x7f9a34069420 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9a480ae000 tx=0x7f9a480a7070 comp rx=0 tx=0).stop 2024-04-25T10:16:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.570+0000 7f9a57cb7640 1 -- 172.21.15.60:0/2319207947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a50154dd0 msgr2=0x7f9a500ab980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.570+0000 7f9a57cb7640 1 --2- 172.21.15.60:0/2319207947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a50154dd0 0x7f9a500ab980 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c5f0 tx=0x7f9a4c00cab0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.570+0000 7f9a57cb7640 1 -- 172.21.15.60:0/2319207947 shutdown_connections 2024-04-25T10:16:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.570+0000 7f9a57cb7640 1 --2- 172.21.15.60:0/2319207947 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a34066f60 0x7f9a34069420 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.570+0000 7f9a57cb7640 1 --2- 172.21.15.60:0/2319207947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a50154dd0 0x7f9a500ab980 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.570+0000 7f9a57cb7640 1 --2- 172.21.15.60:0/2319207947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a5015bc90 0x7f9a500ab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.570+0000 7f9a57cb7640 1 --2- 172.21.15.60:0/2319207947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a500ce7a0 0x7f9a50154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.570+0000 7f9a57cb7640 1 -- 172.21.15.60:0/2319207947 >> 172.21.15.60:0/2319207947 conn(0x7f9a500b12d0 msgr2=0x7f9a500b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:13.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.570+0000 7f9a57cb7640 1 -- 172.21.15.60:0/2319207947 shutdown_connections 2024-04-25T10:16:13.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:13.570+0000 7f9a57cb7640 1 -- 172.21.15.60:0/2319207947 wait complete. 2024-04-25T10:16:13.992 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 93, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:16:07.763322+0000', 'modified': '2024-04-25T10:16:10.765897+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9381}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9381': {'gid': 9381, 'name': 'c', 'rank': 0, 'incarnation': 89, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/722171246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 722171246}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 722171246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:13.992 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:16:13.993 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:16:14.021 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-04-25T10:16:14.014+0000 7efda3e5b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:16:14.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd40f17640 1 Processor -- start 2024-04-25T10:16:14.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd40f17640 1 -- start start 2024-04-25T10:16:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd40f17640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbd3c15bc90 0x7fbd3c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd40f17640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd3c0b4660 0x7fbd3c15c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd40f17640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3c071ca0 con 0x7fbd3c0ce7a0 2024-04-25T10:16:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd40f17640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3c0cc370 con 0x7fbd3c0b4660 2024-04-25T10:16:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd40f17640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3c0cca40 con 0x7fbd3c15bc90 2024-04-25T10:16:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd3ad9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd3c0b4660 0x7fbd3c15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd3ad9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd3c0b4660 0x7fbd3c15c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35324/0 (socket says 172.21.15.60:35324) 2024-04-25T10:16:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd3ad9d640 1 -- 172.21.15.60:0/2671584909 learned_addr learned my addr 172.21.15.60:0/2671584909 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd39d9b640 1 --2- 172.21.15.60:0/2671584909 >> v2:172.21.15.100:3300/0 conn(0x7fbd3c15bc90 0x7fbd3c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd3ad9d640 1 -- 172.21.15.60:0/2671584909 >> v2:172.21.15.100:3300/0 conn(0x7fbd3c15bc90 msgr2=0x7fbd3c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd3ad9d640 1 --2- 172.21.15.60:0/2671584909 >> v2:172.21.15.100:3300/0 conn(0x7fbd3c15bc90 0x7fbd3c15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd3ad9d640 1 -- 172.21.15.60:0/2671584909 >> v1:172.21.15.60:6789/0 conn(0x7fbd3c0ce7a0 legacy=0x7fbd3c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd3959a640 1 -- 172.21.15.60:0/2671584909 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1088590967 0 0) 0x7fbd3c071ca0 con 0x7fbd3c0ce7a0 2024-04-25T10:16:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd3ad9d640 1 -- 172.21.15.60:0/2671584909 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c0ccd20 con 0x7fbd3c0b4660 2024-04-25T10:16:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.122+0000 7fbd39d9b640 1 --2- 172.21.15.60:0/2671584909 >> v2:172.21.15.100:3300/0 conn(0x7fbd3c15bc90 0x7fbd3c15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:14.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.126+0000 7fbd3ad9d640 1 --2- 172.21.15.60:0/2671584909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd3c0b4660 0x7fbd3c15c760 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0d0530 tx=0x7fbd3401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c14f40ff6f84c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.126+0000 7fbd3959a640 1 -- 172.21.15.60:0/2671584909 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340a56c0 con 0x7fbd3c0b4660 2024-04-25T10:16:14.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.126+0000 7fbd3959a640 1 -- 172.21.15.60:0/2671584909 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd340a5860 con 0x7fbd3c0b4660 2024-04-25T10:16:14.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.126+0000 7fbd3959a640 1 -- 172.21.15.60:0/2671584909 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340a5c30 con 0x7fbd3c0b4660 2024-04-25T10:16:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.126+0000 7fbd40f17640 1 -- 172.21.15.60:0/2671584909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd3c0b4660 msgr2=0x7fbd3c15c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.126+0000 7fbd40f17640 1 --2- 172.21.15.60:0/2671584909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd3c0b4660 0x7fbd3c15c760 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0d0530 tx=0x7fbd3401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.126+0000 7fbd40f17640 1 -- 172.21.15.60:0/2671584909 shutdown_connections 2024-04-25T10:16:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.126+0000 7fbd40f17640 1 --2- 172.21.15.60:0/2671584909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd3c0b4660 0x7fbd3c15c760 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.126+0000 7fbd40f17640 1 --2- 172.21.15.60:0/2671584909 >> v2:172.21.15.100:3300/0 conn(0x7fbd3c15bc90 0x7fbd3c15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.126+0000 7fbd40f17640 1 -- 172.21.15.60:0/2671584909 >> 172.21.15.60:0/2671584909 conn(0x7fbd3c0b12d0 msgr2=0x7fbd3c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.126+0000 7fbd40f17640 1 -- 172.21.15.60:0/2671584909 shutdown_connections 2024-04-25T10:16:14.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.126+0000 7fbd40f17640 1 -- 172.21.15.60:0/2671584909 wait complete. 2024-04-25T10:16:14.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd40f17640 1 Processor -- start 2024-04-25T10:16:14.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd40f17640 1 -- start start 2024-04-25T10:16:14.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd40f17640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd3c0ce7a0 0x7fbd3c14b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd40f17640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd3c15bc90 0x7fbd3c14e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd40f17640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c14c6a0 0x7fbd3c159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd40f17640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c1685c0 con 0x7fbd3c15bc90 2024-04-25T10:16:14.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd40f17640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c0b3db0 con 0x7fbd3c14c6a0 2024-04-25T10:16:14.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd40f17640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c168740 con 0x7fbd3c0ce7a0 2024-04-25T10:16:14.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd39d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd3c15bc90 0x7fbd3c14e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd39d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd3c15bc90 0x7fbd3c14e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35328/0 (socket says 172.21.15.60:35328) 2024-04-25T10:16:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd39d9b640 1 -- 172.21.15.60:0/1389628667 learned_addr learned my addr 172.21.15.60:0/1389628667 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd39d9b640 1 -- 172.21.15.60:0/1389628667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd3c0ce7a0 msgr2=0x7fbd3c14b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd39d9b640 1 --2- 172.21.15.60:0/1389628667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd3c0ce7a0 0x7fbd3c14b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd39d9b640 1 -- 172.21.15.60:0/1389628667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c14c6a0 msgr2=0x7fbd3c159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd3ad9d640 1 --2- 172.21.15.60:0/1389628667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c14c6a0 0x7fbd3c159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd39d9b640 1 --2- 172.21.15.60:0/1389628667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c14c6a0 0x7fbd3c159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd39d9b640 1 -- 172.21.15.60:0/1389628667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c14aea0 con 0x7fbd3c15bc90 2024-04-25T10:16:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd39d9b640 1 --2- 172.21.15.60:0/1389628667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd3c15bc90 0x7fbd3c14e600 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000c820 tx=0x7fbd3000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd2affd640 1 -- 172.21.15.60:0/1389628667 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd30015070 con 0x7fbd3c15bc90 2024-04-25T10:16:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd2affd640 1 -- 172.21.15.60:0/1389628667 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd30007a30 con 0x7fbd3c15bc90 2024-04-25T10:16:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd2affd640 1 -- 172.21.15.60:0/1389628667 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd30007e40 con 0x7fbd3c15bc90 2024-04-25T10:16:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd40f17640 1 -- 172.21.15.60:0/1389628667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3c153010 con 0x7fbd3c15bc90 2024-04-25T10:16:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd40f17640 1 -- 172.21.15.60:0/1389628667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3c1535d0 con 0x7fbd3c15bc90 2024-04-25T10:16:14.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.130+0000 7fbd2affd640 1 -- 172.21.15.60:0/1389628667 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd30018070 con 0x7fbd3c15bc90 2024-04-25T10:16:14.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd2affd640 1 --2- 172.21.15.60:0/1389628667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd1c066f60 0x7fbd1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd2affd640 1 -- 172.21.15.60:0/1389628667 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd30084f20 con 0x7fbd3c15bc90 2024-04-25T10:16:14.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd3a59c640 1 --2- 172.21.15.60:0/1389628667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd1c066f60 0x7fbd1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 Processor -- start 2024-04-25T10:16:14.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 -- start start 2024-04-25T10:16:14.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd3a59c640 1 --2- 172.21.15.60:0/1389628667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd1c066f60 0x7fbd1c069420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c007fd0 tx=0x7fbd2c007940 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa2678bd80 0x55fa2678c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678c6a0 0x55fa267421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa26742b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa266255f0 con 0x55fa26742720 2024-04-25T10:16:14.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa2658cda0 con 0x55fa2678c6a0 2024-04-25T10:16:14.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fa26515150 con 0x55fa2678bd80 2024-04-25T10:16:14.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd3ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa26742b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd3ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa26742b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35344/0 (socket says 172.21.15.60:35344) 2024-04-25T10:16:14.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd3ad9d640 1 -- 172.21.15.60:0/1133402756 learned_addr learned my addr 172.21.15.60:0/1133402756 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:14.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd3a59c640 1 --2- 172.21.15.60:0/1133402756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa2678bd80 0x55fa2678c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd39d9b640 1 --2- 172.21.15.60:0/1133402756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678c6a0 0x55fa267421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd3ad9d640 1 -- 172.21.15.60:0/1133402756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa2678bd80 msgr2=0x55fa2678c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd3ad9d640 1 --2- 172.21.15.60:0/1133402756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa2678bd80 0x55fa2678c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd3ad9d640 1 -- 172.21.15.60:0/1133402756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678c6a0 msgr2=0x55fa267421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd3ad9d640 1 --2- 172.21.15.60:0/1133402756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678c6a0 0x55fa267421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd3ad9d640 1 -- 172.21.15.60:0/1133402756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa2652eca0 con 0x55fa26742720 2024-04-25T10:16:14.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd39d9b640 1 --2- 172.21.15.60:0/1133402756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678c6a0 0x55fa267421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:14.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd3ad9d640 1 --2- 172.21.15.60:0/1133402756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa26742b00 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fbd34019600 tx=0x7fbd34004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0833714ab4b40a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd1bfff640 1 -- 172.21.15.60:0/1133402756 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340039a0 con 0x55fa26742720 2024-04-25T10:16:14.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd1bfff640 1 -- 172.21.15.60:0/1133402756 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd34003b40 con 0x55fa26742720 2024-04-25T10:16:14.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd1bfff640 1 -- 172.21.15.60:0/1133402756 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340a56c0 con 0x55fa26742720 2024-04-25T10:16:14.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 -- 172.21.15.60:0/1133402756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 msgr2=0x55fa26742b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 --2- 172.21.15.60:0/1133402756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa26742b00 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fbd34019600 tx=0x7fbd34004ac0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 -- 172.21.15.60:0/1133402756 shutdown_connections 2024-04-25T10:16:14.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 --2- 172.21.15.60:0/1133402756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa26742b00 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 --2- 172.21.15.60:0/1133402756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678c6a0 0x55fa267421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 --2- 172.21.15.60:0/1133402756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa2678bd80 0x55fa2678c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 -- 172.21.15.60:0/1133402756 >> 172.21.15.60:0/1133402756 conn(0x55fa2673e6d0 msgr2=0x55fa26740ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 -- 172.21.15.60:0/1133402756 shutdown_connections 2024-04-25T10:16:14.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.134+0000 7fbd44084000 1 -- 172.21.15.60:0/1133402756 wait complete. 2024-04-25T10:16:14.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 1 -- start start 2024-04-25T10:16:14.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 1 Processor -- start 2024-04-25T10:16:14.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa26742720 0x55fa267788e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678bd80 0x55fa2674b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa2678c6a0 0x55fa2674b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa26547990 con 0x55fa2678c6a0 2024-04-25T10:16:14.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa26588e40 con 0x55fa2678bd80 2024-04-25T10:16:14.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fa2658c0b0 con 0x55fa26742720 2024-04-25T10:16:14.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 10 client.0 _ll_register_callbacks cb 0x55fa266401c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:14.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd3ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa2678c6a0 0x55fa2674b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd3ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa2678c6a0 0x55fa2674b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35348/0 (socket says 172.21.15.60:35348) 2024-04-25T10:16:14.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd3ad9d640 1 -- 172.21.15.60:0/2245503647 learned_addr learned my addr 172.21.15.60:0/2245503647 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:14.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd39d9b640 1 --2- 172.21.15.60:0/2245503647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678bd80 0x55fa2674b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd3a59c640 1 --2- 172.21.15.60:0/2245503647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa26742720 0x55fa267788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd3ad9d640 1 -- 172.21.15.60:0/2245503647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa26742720 msgr2=0x55fa267788e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd3ad9d640 1 --2- 172.21.15.60:0/2245503647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa26742720 0x55fa267788e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd3ad9d640 1 -- 172.21.15.60:0/2245503647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678bd80 msgr2=0x55fa2674b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd3ad9d640 1 --2- 172.21.15.60:0/2245503647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678bd80 0x55fa2674b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd3ad9d640 1 -- 172.21.15.60:0/2245503647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa26545630 con 0x55fa2678c6a0 2024-04-25T10:16:14.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd3a59c640 1 --2- 172.21.15.60:0/2245503647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa26742720 0x55fa267788e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:14.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd39d9b640 1 --2- 172.21.15.60:0/2245503647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678bd80 0x55fa2674b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:14.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd3ad9d640 1 --2- 172.21.15.60:0/2245503647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa2678c6a0 0x55fa2674b7b0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fbd34019600 tx=0x7fbd3401b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4e6d944ec27208 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd197fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:16:14.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd197fa640 1 -- 172.21.15.60:0/2245503647 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340039a0 con 0x55fa2678c6a0 2024-04-25T10:16:14.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 1 -- 172.21.15.60:0/2245503647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd34003b40 con 0x55fa2678c6a0 2024-04-25T10:16:14.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd197fa640 1 -- 172.21.15.60:0/2245503647 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd34009150 con 0x55fa2678c6a0 2024-04-25T10:16:14.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd197fa640 1 -- 172.21.15.60:0/2245503647 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd340cf050 con 0x55fa2678c6a0 2024-04-25T10:16:14.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 10 client.9504 fetch_fsmap learned FSMap version 93 2024-04-25T10:16:14.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 1 -- 172.21.15.60:0/2245503647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x55fa265153a0 con 0x55fa2678c6a0 2024-04-25T10:16:14.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd197fa640 1 -- 172.21.15.60:0/2245503647 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd34003b40 con 0x55fa2678c6a0 2024-04-25T10:16:14.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd197fa640 1 -- 172.21.15.60:0/2245503647 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 93) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbd340d2020 con 0x55fa2678c6a0 2024-04-25T10:16:14.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 10 client.9504 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:16:14.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 10 client.9504 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:16:14.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 20 client.9504 populate_metadata read hostname 'smithi060' 2024-04-25T10:16:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 1 --2- 172.21.15.60:0/2245503647 >> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] conn(0x55fa2674c1c0 0x55fa2674c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 4 client.9504 mds_command: new command op to 9381 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 1 -- 172.21.15.60:0/2245503647 --> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa26528a50 con 0x55fa2674c1c0 2024-04-25T10:16:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd3a59c640 1 --2- 172.21.15.60:0/2245503647 >> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] conn(0x55fa2674c1c0 0x55fa2674c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd3a59c640 1 --2- 172.21.15.60:0/2245503647 >> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] conn(0x55fa2674c1c0 0x55fa2674c5a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68e593f94d16e4d1 server_cookie=6a6fadccea39d9ae in_seq=0 out_seq=0 2024-04-25T10:16:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd197fa640 10 client.9504 ms_handle_connect on v2:172.21.15.60:6834/722171246 2024-04-25T10:16:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd197fa640 1 -- 172.21.15.60:0/2245503647 <== mds.0 v2:172.21.15.60:6834/722171246 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fa26528a50 con 0x55fa2674c1c0 2024-04-25T10:16:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd197fa640 10 client.9504 handle_command_reply: tid=0 2024-04-25T10:16:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.138+0000 7fbd44084000 1 client.9504 shutdown 2024-04-25T10:16:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.142+0000 7fbd44084000 1 -- 172.21.15.60:0/2245503647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa2678c6a0 msgr2=0x55fa2674b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.142+0000 7fbd44084000 1 --2- 172.21.15.60:0/2245503647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa2678c6a0 0x55fa2674b7b0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fbd34019600 tx=0x7fbd3401b230 comp rx=0 tx=0).stop 2024-04-25T10:16:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.142+0000 7fbd44084000 1 -- 172.21.15.60:0/2245503647 shutdown_connections 2024-04-25T10:16:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.142+0000 7fbd44084000 1 --2- 172.21.15.60:0/2245503647 >> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] conn(0x55fa2674c1c0 0x55fa2674c5a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.142+0000 7fbd44084000 1 --2- 172.21.15.60:0/2245503647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa2678c6a0 0x55fa2674b7b0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.142+0000 7fbd44084000 1 --2- 172.21.15.60:0/2245503647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678bd80 0x55fa2674b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.142+0000 7fbd44084000 1 --2- 172.21.15.60:0/2245503647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa26742720 0x55fa267788e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.142+0000 7fbd197fa640 0 client.9504 ms_handle_reset on v2:172.21.15.60:6834/722171246 2024-04-25T10:16:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.142+0000 7fbd44084000 1 -- 172.21.15.60:0/2245503647 >> 172.21.15.60:0/2245503647 conn(0x55fa2673e6d0 msgr2=0x55fa26743390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.142+0000 7fbd44084000 1 -- 172.21.15.60:0/2245503647 shutdown_connections 2024-04-25T10:16:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.142+0000 7fbd44084000 1 -- 172.21.15.60:0/2245503647 wait complete. 2024-04-25T10:16:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.142+0000 7fbd44084000 20 client.9504 trim_cache size 0 max 16384 2024-04-25T10:16:14.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.154+0000 7fbd44084000 1 Processor -- start 2024-04-25T10:16:14.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.154+0000 7fbd44084000 1 -- start start 2024-04-25T10:16:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa26741ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678bd80 0x55fa26742010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa2678c6a0 0x55fa2679e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c149a10 con 0x55fa26742720 2024-04-25T10:16:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c06e790 con 0x55fa2678bd80 2024-04-25T10:16:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c06ed90 con 0x55fa2678c6a0 2024-04-25T10:16:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd3a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa26741ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd3a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa26741ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35352/0 (socket says 172.21.15.60:35352) 2024-04-25T10:16:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd3a59c640 1 -- 172.21.15.60:0/365816356 learned_addr learned my addr 172.21.15.60:0/365816356 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd3a59c640 1 -- 172.21.15.60:0/365816356 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa2678c6a0 msgr2=0x55fa2679e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd3a59c640 1 --2- 172.21.15.60:0/365816356 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa2678c6a0 0x55fa2679e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd3a59c640 1 -- 172.21.15.60:0/365816356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678bd80 msgr2=0x55fa26742010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd3a59c640 1 --2- 172.21.15.60:0/365816356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678bd80 0x55fa26742010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd3a59c640 1 -- 172.21.15.60:0/365816356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa2678c4d0 con 0x55fa26742720 2024-04-25T10:16:14.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd3a59c640 1 --2- 172.21.15.60:0/365816356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa26741ad0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x55fa26628680 tx=0x7fbd2c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2c1d5b03c96d2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd1b7fe640 1 -- 172.21.15.60:0/365816356 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c027070 con 0x55fa26742720 2024-04-25T10:16:14.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd1b7fe640 1 -- 172.21.15.60:0/365816356 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2c003c50 con 0x55fa26742720 2024-04-25T10:16:14.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd1b7fe640 1 -- 172.21.15.60:0/365816356 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c0223d0 con 0x55fa26742720 2024-04-25T10:16:14.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 -- 172.21.15.60:0/365816356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 msgr2=0x55fa26741ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 --2- 172.21.15.60:0/365816356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa26741ad0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x55fa26628680 tx=0x7fbd2c00f000 comp rx=0 tx=0).stop 2024-04-25T10:16:14.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 -- 172.21.15.60:0/365816356 shutdown_connections 2024-04-25T10:16:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 --2- 172.21.15.60:0/365816356 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa2678c6a0 0x55fa2679e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 --2- 172.21.15.60:0/365816356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678bd80 0x55fa26742010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 --2- 172.21.15.60:0/365816356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa26741ad0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 -- 172.21.15.60:0/365816356 >> 172.21.15.60:0/365816356 conn(0x55fa2673e6d0 msgr2=0x55fa2678b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 -- 172.21.15.60:0/365816356 shutdown_connections 2024-04-25T10:16:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 -- 172.21.15.60:0/365816356 wait complete. 2024-04-25T10:16:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 -- start start 2024-04-25T10:16:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 Processor -- start 2024-04-25T10:16:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa267cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678bd80 0x55fa267a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa2678c6a0 0x55fa267a1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa26525750 con 0x55fa26742720 2024-04-25T10:16:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa265913d0 con 0x55fa2678bd80 2024-04-25T10:16:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fa265324b0 con 0x55fa2678c6a0 2024-04-25T10:16:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd44084000 10 client.0 _ll_register_callbacks cb 0x7fbd34009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd3a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa267cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd3a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa267cf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35362/0 (socket says 172.21.15.60:35362) 2024-04-25T10:16:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd3a59c640 1 -- 172.21.15.60:0/2160284900 learned_addr learned my addr 172.21.15.60:0/2160284900 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd3a59c640 1 -- 172.21.15.60:0/2160284900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa2678c6a0 msgr2=0x55fa267a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd3a59c640 1 --2- 172.21.15.60:0/2160284900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa2678c6a0 0x55fa267a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd3a59c640 1 -- 172.21.15.60:0/2160284900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678bd80 msgr2=0x55fa267a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.158+0000 7fbd39d9b640 1 --2- 172.21.15.60:0/2160284900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678bd80 0x55fa267a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd3a59c640 1 --2- 172.21.15.60:0/2160284900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678bd80 0x55fa267a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd3a59c640 1 -- 172.21.15.60:0/2160284900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa2660e600 con 0x55fa26742720 2024-04-25T10:16:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd3a59c640 1 --2- 172.21.15.60:0/2160284900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa267cf4b0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c016040 tx=0x7fbd2c001100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ecebab09186dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd197fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:16:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd197fa640 1 -- 172.21.15.60:0/2160284900 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c030060 con 0x55fa26742720 2024-04-25T10:16:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 1 -- 172.21.15.60:0/2160284900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd2c003c50 con 0x55fa26742720 2024-04-25T10:16:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd197fa640 1 -- 172.21.15.60:0/2160284900 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd2c0227e0 con 0x55fa26742720 2024-04-25T10:16:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd197fa640 1 -- 172.21.15.60:0/2160284900 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd2c036050 con 0x55fa26742720 2024-04-25T10:16:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 10 client.9510 fetch_fsmap learned FSMap version 93 2024-04-25T10:16:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 1 -- 172.21.15.60:0/2160284900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x55fa265219b0 con 0x55fa26742720 2024-04-25T10:16:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd197fa640 1 -- 172.21.15.60:0/2160284900 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2c003c50 con 0x55fa26742720 2024-04-25T10:16:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd197fa640 1 -- 172.21.15.60:0/2160284900 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 93) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbd2c039020 con 0x55fa26742720 2024-04-25T10:16:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 10 client.9510 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:16:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 10 client.9510 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:16:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 20 client.9510 populate_metadata read hostname 'smithi060' 2024-04-25T10:16:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 1 --2- 172.21.15.60:0/2160284900 >> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] conn(0x55fa267a17c0 0x55fa267a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 4 client.9510 mds_command: new command op to 9381 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:16:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 1 -- 172.21.15.60:0/2160284900 --> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fa2659a2e0 con 0x55fa267a17c0 2024-04-25T10:16:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd39d9b640 1 --2- 172.21.15.60:0/2160284900 >> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] conn(0x55fa267a17c0 0x55fa267a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd39d9b640 1 --2- 172.21.15.60:0/2160284900 >> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] conn(0x55fa267a17c0 0x55fa267a1ba0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd10909c20d7b8c server_cookie=6132bca18f32d721 in_seq=0 out_seq=0 2024-04-25T10:16:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd197fa640 10 client.9510 ms_handle_connect on v2:172.21.15.60:6834/722171246 2024-04-25T10:16:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd197fa640 1 -- 172.21.15.60:0/2160284900 <== mds.0 v2:172.21.15.60:6834/722171246 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55fa2659a2e0 con 0x55fa267a17c0 2024-04-25T10:16:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd197fa640 10 client.9510 handle_command_reply: tid=0 2024-04-25T10:16:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 1 client.9510 shutdown 2024-04-25T10:16:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 1 -- 172.21.15.60:0/2160284900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 msgr2=0x55fa267cf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 1 --2- 172.21.15.60:0/2160284900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa267cf4b0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c016040 tx=0x7fbd2c001100 comp rx=0 tx=0).stop 2024-04-25T10:16:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 1 -- 172.21.15.60:0/2160284900 shutdown_connections 2024-04-25T10:16:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 1 --2- 172.21.15.60:0/2160284900 >> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] conn(0x55fa267a17c0 0x55fa267a1ba0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 1 --2- 172.21.15.60:0/2160284900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa2678c6a0 0x55fa267a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 1 --2- 172.21.15.60:0/2160284900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa2678bd80 0x55fa267a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd197fa640 0 client.9510 ms_handle_reset on v2:172.21.15.60:6834/722171246 2024-04-25T10:16:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 1 --2- 172.21.15.60:0/2160284900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa26742720 0x55fa267cf4b0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 1 -- 172.21.15.60:0/2160284900 >> 172.21.15.60:0/2160284900 conn(0x55fa2673e6d0 msgr2=0x55fa26740ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 1 -- 172.21.15.60:0/2160284900 shutdown_connections 2024-04-25T10:16:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 1 -- 172.21.15.60:0/2160284900 wait complete. 2024-04-25T10:16:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd44084000 20 client.9510 trim_cache size 0 max 16384 2024-04-25T10:16:14.173 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:16:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "0ea93863-a079-45c4-abed-6e83540ea9fe", 2024-04-25T10:16:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T10:16:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9381, 2024-04-25T10:16:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T10:16:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T10:16:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T10:16:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 5.7800754809999999, 2024-04-25T10:16:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 93, 2024-04-25T10:16:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 110, 2024-04-25T10:16:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 110, 2024-04-25T10:16:14.173 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 11.105899783 2024-04-25T10:16:14.174 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:16:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd40f17640 1 -- 172.21.15.60:0/1389628667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd1c066f60 msgr2=0x7fbd1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd40f17640 1 --2- 172.21.15.60:0/1389628667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd1c066f60 0x7fbd1c069420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c007fd0 tx=0x7fbd2c007940 comp rx=0 tx=0).stop 2024-04-25T10:16:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd40f17640 1 -- 172.21.15.60:0/1389628667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd3c15bc90 msgr2=0x7fbd3c14e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd40f17640 1 --2- 172.21.15.60:0/1389628667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd3c15bc90 0x7fbd3c14e600 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000c820 tx=0x7fbd3000cce0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd40f17640 1 -- 172.21.15.60:0/1389628667 shutdown_connections 2024-04-25T10:16:14.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd40f17640 1 --2- 172.21.15.60:0/1389628667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd1c066f60 0x7fbd1c069420 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd40f17640 1 --2- 172.21.15.60:0/1389628667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd3c14c6a0 0x7fbd3c159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd40f17640 1 --2- 172.21.15.60:0/1389628667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd3c15bc90 0x7fbd3c14e600 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd40f17640 1 --2- 172.21.15.60:0/1389628667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd3c0ce7a0 0x7fbd3c14b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd40f17640 1 -- 172.21.15.60:0/1389628667 >> 172.21.15.60:0/1389628667 conn(0x7fbd3c0b12d0 msgr2=0x7fbd3c0b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd40f17640 1 -- 172.21.15.60:0/1389628667 shutdown_connections 2024-04-25T10:16:14.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.162+0000 7fbd40f17640 1 -- 172.21.15.60:0/1389628667 wait complete. 2024-04-25T10:16:14.184 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:16:14.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.338+0000 7f79ddc89640 1 Processor -- start 2024-04-25T10:16:14.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.338+0000 7f79ddc89640 1 -- start start 2024-04-25T10:16:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.338+0000 7f79ddc89640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79d80ce7a0 0x7f79d80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.338+0000 7f79ddc89640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d815bc90 0x7f79d80b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79ddc89640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d80cf0c0 con 0x7f79d80b4660 2024-04-25T10:16:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79ddc89640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d80cc370 con 0x7f79d80ce7a0 2024-04-25T10:16:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79ddc89640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8071ca0 con 0x7f79d815bc90 2024-04-25T10:16:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79d77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79d80ce7a0 0x7f79d80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79d7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f79d80b4660 0x7f79d80b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48678/0 (socket says 172.21.15.60:48678) 2024-04-25T10:16:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79d7fff640 1 -- 172.21.15.60:0/4111649169 learned_addr learned my addr 172.21.15.60:0/4111649169 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79d77fe640 1 -- 172.21.15.60:0/4111649169 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d815bc90 msgr2=0x7f79d80b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79d6ffd640 1 --2- 172.21.15.60:0/4111649169 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d815bc90 0x7f79d80b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79d67fc640 1 -- 172.21.15.60:0/4111649169 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898191021 0 0) 0x7f79d80cf0c0 con 0x7f79d80b4660 2024-04-25T10:16:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79d77fe640 1 --2- 172.21.15.60:0/4111649169 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d815bc90 0x7f79d80b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79d77fe640 1 -- 172.21.15.60:0/4111649169 >> v1:172.21.15.60:6789/0 conn(0x7f79d80b4660 legacy=0x7f79d80b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79d77fe640 1 -- 172.21.15.60:0/4111649169 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d80cca40 con 0x7f79d80ce7a0 2024-04-25T10:16:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79d6ffd640 1 --2- 172.21.15.60:0/4111649169 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d815bc90 0x7f79d80b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79d77fe640 1 --2- 172.21.15.60:0/4111649169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d80ce7a0 0x7f79d80ceb80 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f79c80097f0 tx=0x7f79c801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8824c9fc2e00ae09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79d67fc640 1 -- 172.21.15.60:0/4111649169 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c80b0070 con 0x7f79d80ce7a0 2024-04-25T10:16:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79d67fc640 1 -- 172.21.15.60:0/4111649169 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c80ab600 con 0x7f79d80ce7a0 2024-04-25T10:16:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79d67fc640 1 -- 172.21.15.60:0/4111649169 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c80ab9d0 con 0x7f79d80ce7a0 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79ddc89640 1 -- 172.21.15.60:0/4111649169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d80ce7a0 msgr2=0x7f79d80ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79ddc89640 1 --2- 172.21.15.60:0/4111649169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d80ce7a0 0x7f79d80ceb80 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f79c80097f0 tx=0x7f79c801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79ddc89640 1 -- 172.21.15.60:0/4111649169 shutdown_connections 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79ddc89640 1 --2- 172.21.15.60:0/4111649169 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d815bc90 0x7f79d80b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79ddc89640 1 --2- 172.21.15.60:0/4111649169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d80ce7a0 0x7f79d80ceb80 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79ddc89640 1 -- 172.21.15.60:0/4111649169 >> 172.21.15.60:0/4111649169 conn(0x7f79d80b12d0 msgr2=0x7f79d80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79ddc89640 1 -- 172.21.15.60:0/4111649169 shutdown_connections 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.342+0000 7f79ddc89640 1 -- 172.21.15.60:0/4111649169 wait complete. 2024-04-25T10:16:14.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79ddc89640 1 Processor -- start 2024-04-25T10:16:14.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79ddc89640 1 -- start start 2024-04-25T10:16:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79ddc89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d80ce7a0 0x7f79d814c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79ddc89640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d815bc90 0x7f79d80b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79ddc89640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d814f590 0x7f79d80b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79ddc89640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80b4280 con 0x7f79d80ce7a0 2024-04-25T10:16:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79ddc89640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80b3db0 con 0x7f79d814f590 2024-04-25T10:16:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79ddc89640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80b4400 con 0x7f79d815bc90 2024-04-25T10:16:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79d77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d80ce7a0 0x7f79d814c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79d77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d80ce7a0 0x7f79d814c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35382/0 (socket says 172.21.15.60:35382) 2024-04-25T10:16:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79d77fe640 1 -- 172.21.15.60:0/1682611605 learned_addr learned my addr 172.21.15.60:0/1682611605 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79d6ffd640 1 --2- 172.21.15.60:0/1682611605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d815bc90 0x7f79d80b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79d77fe640 1 -- 172.21.15.60:0/1682611605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d815bc90 msgr2=0x7f79d80b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79d77fe640 1 --2- 172.21.15.60:0/1682611605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d815bc90 0x7f79d80b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79d77fe640 1 -- 172.21.15.60:0/1682611605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d814f590 msgr2=0x7f79d80b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79d77fe640 1 --2- 172.21.15.60:0/1682611605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d814f590 0x7f79d80b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79d77fe640 1 -- 172.21.15.60:0/1682611605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d8151080 con 0x7f79d80ce7a0 2024-04-25T10:16:14.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79d77fe640 1 --2- 172.21.15.60:0/1682611605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d80ce7a0 0x7f79d814c150 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f79c80ae040 tx=0x7f79c801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79bffff640 1 -- 172.21.15.60:0/1682611605 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c80b0050 con 0x7f79d80ce7a0 2024-04-25T10:16:14.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79bffff640 1 -- 172.21.15.60:0/1682611605 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c80bc070 con 0x7f79d80ce7a0 2024-04-25T10:16:14.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79ddc89640 1 -- 172.21.15.60:0/1682611605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d8151360 con 0x7f79d80ce7a0 2024-04-25T10:16:14.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79bffff640 1 -- 172.21.15.60:0/1682611605 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c80b5a70 con 0x7f79d80ce7a0 2024-04-25T10:16:14.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79ddc89640 1 -- 172.21.15.60:0/1682611605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d814c690 con 0x7f79d80ce7a0 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79ddc89640 1 -- 172.21.15.60:0/1682611605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d8074770 con 0x7f79d80ce7a0 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.346+0000 7f79bffff640 1 -- 172.21.15.60:0/1682611605 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79c80ab4c0 con 0x7f79d80ce7a0 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.350+0000 7f79bffff640 1 --2- 172.21.15.60:0/1682611605 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79c4066f60 0x7f79c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.350+0000 7f79bffff640 1 -- 172.21.15.60:0/1682611605 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79c811f9a0 con 0x7f79d80ce7a0 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.350+0000 7f79d6ffd640 1 --2- 172.21.15.60:0/1682611605 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79c4066f60 0x7f79c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.350+0000 7f79d6ffd640 1 --2- 172.21.15.60:0/1682611605 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79c4066f60 0x7f79c4069420 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f79cc006fd0 tx=0x7f79cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.354+0000 7f79bffff640 1 -- 172.21.15.60:0/1682611605 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79c8016090 con 0x7f79d80ce7a0 2024-04-25T10:16:14.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.566+0000 7f79ddc89640 1 -- 172.21.15.60:0/1682611605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f79d80aeb80 con 0x7f79d80ce7a0 2024-04-25T10:16:14.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.566+0000 7f79bffff640 1 -- 172.21.15.60:0/1682611605 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v93) v1 ==== 53+0+73 (secure 0 0 0) 0x7f79c80f09b0 con 0x7f79d80ce7a0 2024-04-25T10:16:14.571 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:16:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.570+0000 7f79ddc89640 1 -- 172.21.15.60:0/1682611605 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79c4066f60 msgr2=0x7f79c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.570+0000 7f79ddc89640 1 --2- 172.21.15.60:0/1682611605 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79c4066f60 0x7f79c4069420 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f79cc006fd0 tx=0x7f79cc008040 comp rx=0 tx=0).stop 2024-04-25T10:16:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.570+0000 7f79ddc89640 1 -- 172.21.15.60:0/1682611605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d80ce7a0 msgr2=0x7f79d814c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.570+0000 7f79ddc89640 1 --2- 172.21.15.60:0/1682611605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d80ce7a0 0x7f79d814c150 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f79c80ae040 tx=0x7f79c801ace0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.570+0000 7f79ddc89640 1 -- 172.21.15.60:0/1682611605 shutdown_connections 2024-04-25T10:16:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.570+0000 7f79ddc89640 1 --2- 172.21.15.60:0/1682611605 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79c4066f60 0x7f79c4069420 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.570+0000 7f79ddc89640 1 --2- 172.21.15.60:0/1682611605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d814f590 0x7f79d80b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.570+0000 7f79ddc89640 1 --2- 172.21.15.60:0/1682611605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d815bc90 0x7f79d80b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.570+0000 7f79ddc89640 1 --2- 172.21.15.60:0/1682611605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d80ce7a0 0x7f79d814c150 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.570+0000 7f79ddc89640 1 -- 172.21.15.60:0/1682611605 >> 172.21.15.60:0/1682611605 conn(0x7f79d80b12d0 msgr2=0x7f79d815e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.570+0000 7f79ddc89640 1 -- 172.21.15.60:0/1682611605 shutdown_connections 2024-04-25T10:16:14.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:14.570+0000 7f79ddc89640 1 -- 172.21.15.60:0/1682611605 wait complete. 2024-04-25T10:16:14.590 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:16:14.590 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:16:14.590 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:16:14.590 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:16:14.590 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:14.590 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:16:14.590 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:16:14.590 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:16:14.590 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:14.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:14.591 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:16:14.596 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:16:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:16:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:16:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:16:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:14.596 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:16:14.596 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:16:14.597 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:16:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:16:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:16:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41882sec preferred_lft 41882sec 2024-04-25T10:16:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:16:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:14.597 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:16:14.597 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:16:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:16:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:14.598 INFO:teuthology.orchestra.run.smithi060.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:16:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:16:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:16:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:14.598 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:16:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:14.598 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:16:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:14.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:14.599 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:16:14.646 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:16:14.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:14.647 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:16:14.695 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:16:14.696 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:16:14.696 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:16:14.696 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:16:14.696 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:16:14.696 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:16:14.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:14.696 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:16:14.696 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:16:14.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:16:14.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:16:14.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:16:14.697 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:16:14.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:14.774 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:16:14.822 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:16:14.823 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:16:14.823 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:14.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:14.823 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T10:16:14.878 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:16:14.878 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:14.878 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:14.927 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:14.927 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:14.928 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:14.973 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:16:15.031 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:16:15.080 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:16:15.080 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:16:15.080 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:16:15.080 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:16:15.080 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:16:15.080 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:16:15.081 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:16:15.081 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:16:15.081 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:16:15.081 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:16:15.081 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:16:15.081 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:16:15.081 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:16:15.081 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:16:15.081 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:16:15.081 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:16:15.081 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:16:15.081 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:16:15.081 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:16:15.081 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:16:15.082 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:16:15.082 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:16:15.082 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:16:15.082 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:16:15.082 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:16:15.082 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:16:15.082 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:16:15.082 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:16:15.082 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:16:15.082 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:15.082 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:15.082 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:15.082 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:15.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:15.083 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:16:15.130 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:16:15.130 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:16:15.179 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:16:15.193 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:16:15.242 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:16:15.242 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:16:15.242 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:16:15.242 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:16:15.243 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:16:15.243 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:16:15.243 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:16:15.243 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:16:15.243 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:16:15.243 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:16:15.243 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:16:15.243 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:16:15.243 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:16:15.243 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:16:15.243 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:16:15.243 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:16:15.243 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:16:15.243 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:16:15.244 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:16:15.244 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:16:15.244 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:16:15.244 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:16:15.244 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:16:15.244 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:16:15.244 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:16:15.244 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:16:15.244 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:16:15.244 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:16:15.244 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:16:15.244 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:15.244 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:15.244 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:15.245 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:15.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:15.245 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:16:15.264 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2024-04-25T10:16:15.258+0000 7ff1a6867580 -1 init, newargv = 0x5567a4f91eb0 newargc=15 2024-04-25T10:16:15.264 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[51660]: starting ceph client 2024-04-25T10:16:15.269 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[51660]: starting fuse 2024-04-25T10:16:16.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:16:16.387 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:16:16.440 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:16:16.440 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:16:16.441 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:16:16.441 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:16:16.441 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:16:16.441 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:16:16.441 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:16:16.441 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:16:16.441 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:16:16.441 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:16:16.441 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:16:16.441 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:16:16.441 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:16:16.441 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:16:16.441 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:16:16.442 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:16:16.442 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:16:16.442 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:16:16.442 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:16:16.442 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:16:16.442 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:16:16.442 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:16:16.442 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:16:16.442 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:16:16.442 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:16:16.442 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:16:16.442 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:16:16.442 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:16:16.443 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:16:16.443 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:16.443 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:16.443 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:16.443 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:16.443 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:16.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:16.444 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:16:16.491 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:16:16.491 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:16:16.491 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:16:16.491 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:16:16.491 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:16:16.491 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:16.491 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:16:16.491 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:16.491 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:16:16.492 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:16:16.492 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:16:16.492 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:16:16.492 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:16.492 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:16:16.492 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:16:16.492 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:16:16.493 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:16:16.493 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:16.493 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:16:16.493 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:16:16.493 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:16:16.493 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:16:16.493 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:16:16.493 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:16:16.493 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:16:16.493 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:16:16.493 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:16.493 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2024-04-25T10:16:16.493 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:16.618 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.51660.asok 2024-04-25T10:16:16.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:16.632 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51660.asok 2024-04-25T10:16:16.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:16.632 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51660.asok status 2024-04-25T10:16:16.787 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:16:16.788 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:16:16.788 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:16:16.788 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:16:16.788 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2024-04-25T10:16:16.788 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:16:16.788 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:16:16.788 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "51660", 2024-04-25T10:16:16.788 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:16:16.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:16.788 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:16:16.788 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:16:16.788 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9525, 2024-04-25T10:16:16.788 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:16:16.788 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:16:16.788 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:16:16.789 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 9525 2024-04-25T10:16:16.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:16.789 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:16:16.789 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:16:16.789 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:16:16.789 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4235242208 2024-04-25T10:16:16.789 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:16.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:16.789 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:16:16.789 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:16:16.789 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:16:16.789 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4235242208 2024-04-25T10:16:16.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:16.789 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.9525 192.168.0.1:0/4235242208", 2024-04-25T10:16:16.789 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/4235242208", 2024-04-25T10:16:16.789 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:16:16.790 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 93, 2024-04-25T10:16:16.790 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 110, 2024-04-25T10:16:16.790 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:16:16.790 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:16:16.790 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:16:16.790 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:16:16.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:16.801 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:16:16.801 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:16.808 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:16:16.808 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:16.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:16.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:16.861 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:16.861 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:16.861 DEBUG:teuthology.orchestra.run.smithi060:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:16:16.861 DEBUG:teuthology.orchestra.run.smithi060:> parent = os.path.join(mount_point, "parent") 2024-04-25T10:16:16.861 DEBUG:teuthology.orchestra.run.smithi060:> os.mkdir(parent) 2024-04-25T10:16:16.861 DEBUG:teuthology.orchestra.run.smithi060:> flushed = os.path.join(parent, "flushed") 2024-04-25T10:16:16.861 DEBUG:teuthology.orchestra.run.smithi060:> os.mkdir(flushed) 2024-04-25T10:16:16.861 DEBUG:teuthology.orchestra.run.smithi060:> for f in ["alpha", "bravo", "charlie"]: 2024-04-25T10:16:16.861 DEBUG:teuthology.orchestra.run.smithi060:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-04-25T10:16:16.861 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:16.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:17.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:17.004 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:16:17.070 INFO:teuthology.orchestra.run.smithi060.stdout:./ 2024-04-25T10:16:17.070 INFO:teuthology.orchestra.run.smithi060.stdout:./parent 2024-04-25T10:16:17.071 INFO:teuthology.orchestra.run.smithi060.stdout:./parent/flushed 2024-04-25T10:16:17.071 INFO:teuthology.orchestra.run.smithi060.stdout:./parent/flushed/charlie 2024-04-25T10:16:17.071 INFO:teuthology.orchestra.run.smithi060.stdout:./parent/flushed/alpha 2024-04-25T10:16:17.071 INFO:teuthology.orchestra.run.smithi060.stdout:./parent/flushed/bravo 2024-04-25T10:16:17.071 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:17.071 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:17.071 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:17.071 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:17.071 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:16:17.071 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:17.177 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:16:17.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:17.188 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:17.188 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:17.188 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:17.188 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:17.188 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:16:17.188 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:17.298 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627776 2024-04-25T10:16:17.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:17.307 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:17.307 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:17.307 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:17.308 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:17.308 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:16:17.308 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:17.414 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627777 2024-04-25T10:16:17.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:17.424 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:17.424 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:17.424 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:17.424 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:17.424 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:16:17.424 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:17.528 INFO:teuthology.orchestra.run.smithi060.stdout:1099511628280 2024-04-25T10:16:17.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:17.538 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:17.539 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:17.539 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:17.539 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:17.539 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:16:17.539 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:17.643 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627778 2024-04-25T10:16:17.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:17.653 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:17.653 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:17.653 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:17.653 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:17.653 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-25T10:16:17.653 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:17.759 INFO:teuthology.orchestra.run.smithi060.stdout:1099511628279 2024-04-25T10:16:17.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:17.769 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:16:17.769 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:17.819 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:17.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:17.944 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:18.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.994+0000 7f107f9e4640 1 Processor -- start 2024-04-25T10:16:18.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.994+0000 7f107f9e4640 1 -- start start 2024-04-25T10:16:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.994+0000 7f107f9e4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10780d0a50 0x7f10780b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107f9e4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f10780ce2b0 0x7f10780ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107f9e4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107806d200 con 0x7f107815deb0 2024-04-25T10:16:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107f9e4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10780cf2c0 con 0x7f10780ce2b0 2024-04-25T10:16:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107f9e4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10780cefe0 con 0x7f10780d0a50 2024-04-25T10:16:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107df81640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f10780ce2b0 0x7f10780ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107df81640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f10780ce2b0 0x7f10780ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35392/0 (socket says 172.21.15.60:35392) 2024-04-25T10:16:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107df81640 1 -- 172.21.15.60:0/3843837812 learned_addr learned my addr 172.21.15.60:0/3843837812 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107cf7f640 1 --2- 172.21.15.60:0/3843837812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10780d0a50 0x7f10780b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107df81640 1 -- 172.21.15.60:0/3843837812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10780d0a50 msgr2=0x7f10780b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107df81640 1 --2- 172.21.15.60:0/3843837812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10780d0a50 0x7f10780b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107df81640 1 -- 172.21.15.60:0/3843837812 >> v1:172.21.15.100:6789/0 conn(0x7f107815deb0 legacy=0x7f10780d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107df81640 1 -- 172.21.15.60:0/3843837812 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10780d1d70 con 0x7f10780ce2b0 2024-04-25T10:16:18.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107df81640 1 --2- 172.21.15.60:0/3843837812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10780ce2b0 0x7f10780ce760 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f1070009e50 tx=0x7f107001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b52e3205031b3c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f106ffff640 1 -- 172.21.15.60:0/3843837812 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10700a57f0 con 0x7f10780ce2b0 2024-04-25T10:16:18.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f106ffff640 1 -- 172.21.15.60:0/3843837812 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10700a5990 con 0x7f10780ce2b0 2024-04-25T10:16:18.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f106ffff640 1 -- 172.21.15.60:0/3843837812 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10700a5d60 con 0x7f10780ce2b0 2024-04-25T10:16:18.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107f9e4640 1 -- 172.21.15.60:0/3843837812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10780ce2b0 msgr2=0x7f10780ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107f9e4640 1 --2- 172.21.15.60:0/3843837812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10780ce2b0 0x7f10780ce760 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f1070009e50 tx=0x7f107001ca30 comp rx=0 tx=0).stop 2024-04-25T10:16:18.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107f9e4640 1 -- 172.21.15.60:0/3843837812 shutdown_connections 2024-04-25T10:16:18.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107f9e4640 1 --2- 172.21.15.60:0/3843837812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10780ce2b0 0x7f10780ce760 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107f9e4640 1 --2- 172.21.15.60:0/3843837812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10780d0a50 0x7f10780b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107f9e4640 1 -- 172.21.15.60:0/3843837812 >> 172.21.15.60:0/3843837812 conn(0x7f107806c090 msgr2=0x7f107806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107f9e4640 1 -- 172.21.15.60:0/3843837812 shutdown_connections 2024-04-25T10:16:18.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:17.998+0000 7f107f9e4640 1 -- 172.21.15.60:0/3843837812 wait complete. 2024-04-25T10:16:18.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107f9e4640 1 Processor -- start 2024-04-25T10:16:18.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107f9e4640 1 -- start start 2024-04-25T10:16:18.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107f9e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10780ce2b0 0x7f10780c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107f9e4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10780d0a50 0x7f107815d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107f9e4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f107815deb0 0x7f107815f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107f9e4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10780d31e0 con 0x7f10780ce2b0 2024-04-25T10:16:18.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107f9e4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10780d3060 con 0x7f107815deb0 2024-04-25T10:16:18.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107f9e4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10780d3360 con 0x7f10780d0a50 2024-04-25T10:16:18.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107d780640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10780ce2b0 0x7f10780c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107d780640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10780ce2b0 0x7f10780c7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35396/0 (socket says 172.21.15.60:35396) 2024-04-25T10:16:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107d780640 1 -- 172.21.15.60:0/670071599 learned_addr learned my addr 172.21.15.60:0/670071599 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107df81640 1 --2- 172.21.15.60:0/670071599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f107815deb0 0x7f107815f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107cf7f640 1 --2- 172.21.15.60:0/670071599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10780d0a50 0x7f107815d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107d780640 1 -- 172.21.15.60:0/670071599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10780d0a50 msgr2=0x7f107815d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107d780640 1 --2- 172.21.15.60:0/670071599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10780d0a50 0x7f107815d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107d780640 1 -- 172.21.15.60:0/670071599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f107815deb0 msgr2=0x7f107815f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107d780640 1 --2- 172.21.15.60:0/670071599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f107815deb0 0x7f107815f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107d780640 1 -- 172.21.15.60:0/670071599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10780c6740 con 0x7f10780ce2b0 2024-04-25T10:16:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107df81640 1 --2- 172.21.15.60:0/670071599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f107815deb0 0x7f107815f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107d780640 1 --2- 172.21.15.60:0/670071599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10780ce2b0 0x7f10780c7000 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f107400c3f0 tx=0x7f107400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f106dffb640 1 -- 172.21.15.60:0/670071599 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074017070 con 0x7f10780ce2b0 2024-04-25T10:16:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f106dffb640 1 -- 172.21.15.60:0/670071599 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10740052d0 con 0x7f10780ce2b0 2024-04-25T10:16:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107f9e4640 1 -- 172.21.15.60:0/670071599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10780c7540 con 0x7f10780ce2b0 2024-04-25T10:16:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f106dffb640 1 -- 172.21.15.60:0/670071599 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074003840 con 0x7f10780ce2b0 2024-04-25T10:16:18.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107f9e4640 1 -- 172.21.15.60:0/670071599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10780c69b0 con 0x7f10780ce2b0 2024-04-25T10:16:18.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.002+0000 7f107f9e4640 1 -- 172.21.15.60:0/670071599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1078074770 con 0x7f10780ce2b0 2024-04-25T10:16:18.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.006+0000 7f106dffb640 1 -- 172.21.15.60:0/670071599 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f107401a070 con 0x7f10780ce2b0 2024-04-25T10:16:18.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.006+0000 7f106dffb640 1 --2- 172.21.15.60:0/670071599 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f105c066f60 0x7f105c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.006+0000 7f106dffb640 1 -- 172.21.15.60:0/670071599 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1074084470 con 0x7f10780ce2b0 2024-04-25T10:16:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.006+0000 7f107cf7f640 1 --2- 172.21.15.60:0/670071599 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f105c066f60 0x7f105c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.006+0000 7f107cf7f640 1 --2- 172.21.15.60:0/670071599 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f105c066f60 0x7f105c069420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f10780cd650 tx=0x7f1068008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.010+0000 7f106dffb640 1 -- 172.21.15.60:0/670071599 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f10780c7540 con 0x7f10780ce2b0 2024-04-25T10:16:18.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.170+0000 7f107f9e4640 1 -- 172.21.15.60:0/670071599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10780ca200 con 0x7f10780ce2b0 2024-04-25T10:16:18.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.174+0000 7f106dffb640 1 -- 172.21.15.60:0/670071599 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16639 (secure 0 0 0) 0x7f1074005480 con 0x7f10780ce2b0 2024-04-25T10:16:18.177 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:18.178 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":110,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:16:09.041178+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:06.033387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:06.753421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:18.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.174+0000 7f107f9e4640 1 -- 172.21.15.60:0/670071599 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f105c066f60 msgr2=0x7f105c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.174+0000 7f107f9e4640 1 --2- 172.21.15.60:0/670071599 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f105c066f60 0x7f105c069420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f10780cd650 tx=0x7f1068008040 comp rx=0 tx=0).stop 2024-04-25T10:16:18.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.174+0000 7f107f9e4640 1 -- 172.21.15.60:0/670071599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10780ce2b0 msgr2=0x7f10780c7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.174+0000 7f107f9e4640 1 --2- 172.21.15.60:0/670071599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10780ce2b0 0x7f10780c7000 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f107400c3f0 tx=0x7f107400c8b0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.174+0000 7f107f9e4640 1 -- 172.21.15.60:0/670071599 shutdown_connections 2024-04-25T10:16:18.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.174+0000 7f107f9e4640 1 --2- 172.21.15.60:0/670071599 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f105c066f60 0x7f105c069420 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.174+0000 7f107f9e4640 1 --2- 172.21.15.60:0/670071599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f107815deb0 0x7f107815f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.174+0000 7f107f9e4640 1 --2- 172.21.15.60:0/670071599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10780d0a50 0x7f107815d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.174+0000 7f107f9e4640 1 --2- 172.21.15.60:0/670071599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10780ce2b0 0x7f10780c7000 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.174+0000 7f107f9e4640 1 -- 172.21.15.60:0/670071599 >> 172.21.15.60:0/670071599 conn(0x7f107806c090 msgr2=0x7f10780c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.174+0000 7f107f9e4640 1 -- 172.21.15.60:0/670071599 shutdown_connections 2024-04-25T10:16:18.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.174+0000 7f107f9e4640 1 -- 172.21.15.60:0/670071599 wait complete. 2024-04-25T10:16:18.196 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:06.033387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:06.753421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:18.197 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:18.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.354+0000 7f887af2b640 1 Processor -- start 2024-04-25T10:16:18.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.354+0000 7f887af2b640 1 -- start start 2024-04-25T10:16:18.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.354+0000 7f887af2b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f887415dd30 0x7f887415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.354+0000 7f887af2b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f88740cf8a0 0x7f887415be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.354+0000 7f887af2b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88740cd890 con 0x7f88740cd3d0 2024-04-25T10:16:18.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.354+0000 7f887af2b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88740ce4a0 con 0x7f88740cf8a0 2024-04-25T10:16:18.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.354+0000 7f887af2b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88740cdb70 con 0x7f887415dd30 2024-04-25T10:16:18.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.354+0000 7f88794c8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f88740cf8a0 0x7f887415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.354+0000 7f88794c8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f88740cf8a0 0x7f887415be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35404/0 (socket says 172.21.15.60:35404) 2024-04-25T10:16:18.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.354+0000 7f88794c8640 1 -- 172.21.15.60:0/151046613 learned_addr learned my addr 172.21.15.60:0/151046613 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:18.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.354+0000 7f8878cc7640 1 --2- 172.21.15.60:0/151046613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f887415dd30 0x7f887415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.354+0000 7f88794c8640 1 -- 172.21.15.60:0/151046613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f887415dd30 msgr2=0x7f887415ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.354+0000 7f88794c8640 1 --2- 172.21.15.60:0/151046613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f887415dd30 0x7f887415ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.354+0000 7f88794c8640 1 -- 172.21.15.60:0/151046613 >> v1:172.21.15.100:6789/0 conn(0x7f88740cd3d0 legacy=0x7f88740cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.354+0000 7f88794c8640 1 -- 172.21.15.60:0/151046613 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88740cde50 con 0x7f88740cf8a0 2024-04-25T10:16:18.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f88794c8640 1 --2- 172.21.15.60:0/151046613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88740cf8a0 0x7f887415be20 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f886c00bb10 tx=0x7f886c01eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1426256f582f370d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f88737fe640 1 -- 172.21.15.60:0/151046613 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c0b4070 con 0x7f88740cf8a0 2024-04-25T10:16:18.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f88737fe640 1 -- 172.21.15.60:0/151046613 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c0a7be0 con 0x7f88740cf8a0 2024-04-25T10:16:18.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f88737fe640 1 -- 172.21.15.60:0/151046613 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c0af440 con 0x7f88740cf8a0 2024-04-25T10:16:18.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f887af2b640 1 -- 172.21.15.60:0/151046613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88740cf8a0 msgr2=0x7f887415be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f887af2b640 1 --2- 172.21.15.60:0/151046613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88740cf8a0 0x7f887415be20 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f886c00bb10 tx=0x7f886c01eb30 comp rx=0 tx=0).stop 2024-04-25T10:16:18.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f887af2b640 1 -- 172.21.15.60:0/151046613 shutdown_connections 2024-04-25T10:16:18.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f887af2b640 1 --2- 172.21.15.60:0/151046613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88740cf8a0 0x7f887415be20 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f887af2b640 1 --2- 172.21.15.60:0/151046613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f887415dd30 0x7f887415ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f887af2b640 1 -- 172.21.15.60:0/151046613 >> 172.21.15.60:0/151046613 conn(0x7f88740670b0 msgr2=0x7f887406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f887af2b640 1 -- 172.21.15.60:0/151046613 shutdown_connections 2024-04-25T10:16:18.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f887af2b640 1 -- 172.21.15.60:0/151046613 wait complete. 2024-04-25T10:16:18.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f887af2b640 1 Processor -- start 2024-04-25T10:16:18.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f887af2b640 1 -- start start 2024-04-25T10:16:18.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f887af2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88740cd3d0 0x7f887414c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f887af2b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88740cf8a0 0x7f88741490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f887af2b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f887415dd30 0x7f887414a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f887af2b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887415cf00 con 0x7f887415dd30 2024-04-25T10:16:18.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f887af2b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887415cd80 con 0x7f88740cd3d0 2024-04-25T10:16:18.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f887af2b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887415d080 con 0x7f88740cf8a0 2024-04-25T10:16:18.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f88794c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f887415dd30 0x7f887414a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f88794c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f887415dd30 0x7f887414a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35420/0 (socket says 172.21.15.60:35420) 2024-04-25T10:16:18.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f88794c8640 1 -- 172.21.15.60:0/1142185611 learned_addr learned my addr 172.21.15.60:0/1142185611 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:18.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f88794c8640 1 -- 172.21.15.60:0/1142185611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88740cf8a0 msgr2=0x7f88741490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f8873fff640 1 --2- 172.21.15.60:0/1142185611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88740cf8a0 0x7f88741490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f88794c8640 1 --2- 172.21.15.60:0/1142185611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88740cf8a0 0x7f88741490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f88794c8640 1 -- 172.21.15.60:0/1142185611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88740cd3d0 msgr2=0x7f887414c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f88794c8640 1 --2- 172.21.15.60:0/1142185611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88740cd3d0 0x7f887414c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f88794c8640 1 -- 172.21.15.60:0/1142185611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887414b000 con 0x7f887415dd30 2024-04-25T10:16:18.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.358+0000 7f8873fff640 1 --2- 172.21.15.60:0/1142185611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88740cf8a0 0x7f88741490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:18.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.362+0000 7f88794c8640 1 --2- 172.21.15.60:0/1142185611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f887415dd30 0x7f887414a370 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f886c01d1e0 tx=0x7f886c01b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.362+0000 7f88717fa640 1 -- 172.21.15.60:0/1142185611 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c0be020 con 0x7f887415dd30 2024-04-25T10:16:18.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.362+0000 7f88717fa640 1 -- 172.21.15.60:0/1142185611 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c0afe50 con 0x7f887415dd30 2024-04-25T10:16:18.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.362+0000 7f887af2b640 1 -- 172.21.15.60:0/1142185611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887414b2a0 con 0x7f887415dd30 2024-04-25T10:16:18.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.362+0000 7f88717fa640 1 -- 172.21.15.60:0/1142185611 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c0b4040 con 0x7f887415dd30 2024-04-25T10:16:18.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.362+0000 7f887af2b640 1 -- 172.21.15.60:0/1142185611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887414b860 con 0x7f887415dd30 2024-04-25T10:16:18.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.362+0000 7f887af2b640 1 -- 172.21.15.60:0/1142185611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8874074770 con 0x7f887415dd30 2024-04-25T10:16:18.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.362+0000 7f88717fa640 1 -- 172.21.15.60:0/1142185611 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f886c0c1020 con 0x7f887415dd30 2024-04-25T10:16:18.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.362+0000 7f88717fa640 1 --2- 172.21.15.60:0/1142185611 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8860066f60 0x7f8860069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.362+0000 7f88717fa640 1 -- 172.21.15.60:0/1142185611 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f886c122370 con 0x7f887415dd30 2024-04-25T10:16:18.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.362+0000 7f8878cc7640 1 --2- 172.21.15.60:0/1142185611 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8860066f60 0x7f8860069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.362+0000 7f8878cc7640 1 --2- 172.21.15.60:0/1142185611 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8860066f60 0x7f8860069420 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f88640070b0 tx=0x7f8864007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.366+0000 7f88717fa640 1 -- 172.21.15.60:0/1142185611 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f886c0c8030 con 0x7f887415dd30 2024-04-25T10:16:18.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.526+0000 7f887af2b640 1 -- 172.21.15.60:0/1142185611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88740712f0 con 0x7f887415dd30 2024-04-25T10:16:18.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.526+0000 7f88717fa640 1 -- 172.21.15.60:0/1142185611 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f886c0187e0 con 0x7f887415dd30 2024-04-25T10:16:18.532 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:16:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.530+0000 7f887af2b640 1 -- 172.21.15.60:0/1142185611 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8860066f60 msgr2=0x7f8860069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.530+0000 7f887af2b640 1 --2- 172.21.15.60:0/1142185611 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8860066f60 0x7f8860069420 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f88640070b0 tx=0x7f8864007040 comp rx=0 tx=0).stop 2024-04-25T10:16:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.530+0000 7f887af2b640 1 -- 172.21.15.60:0/1142185611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f887415dd30 msgr2=0x7f887414a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.530+0000 7f887af2b640 1 --2- 172.21.15.60:0/1142185611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f887415dd30 0x7f887414a370 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f886c01d1e0 tx=0x7f886c01b8d0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.530+0000 7f887af2b640 1 -- 172.21.15.60:0/1142185611 shutdown_connections 2024-04-25T10:16:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.530+0000 7f887af2b640 1 --2- 172.21.15.60:0/1142185611 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8860066f60 0x7f8860069420 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.530+0000 7f887af2b640 1 --2- 172.21.15.60:0/1142185611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f887415dd30 0x7f887414a370 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.530+0000 7f887af2b640 1 --2- 172.21.15.60:0/1142185611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88740cf8a0 0x7f88741490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.530+0000 7f887af2b640 1 --2- 172.21.15.60:0/1142185611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88740cd3d0 0x7f887414c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.530+0000 7f887af2b640 1 -- 172.21.15.60:0/1142185611 >> 172.21.15.60:0/1142185611 conn(0x7f88740670b0 msgr2=0x7f887415aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.530+0000 7f887af2b640 1 -- 172.21.15.60:0/1142185611 shutdown_connections 2024-04-25T10:16:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.530+0000 7f887af2b640 1 -- 172.21.15.60:0/1142185611 wait complete. 2024-04-25T10:16:18.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:16:18.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.674+0000 7fb0e4ae3640 1 Processor -- start 2024-04-25T10:16:18.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.674+0000 7fb0e4ae3640 1 -- start start 2024-04-25T10:16:18.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.674+0000 7fb0e4ae3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0e01620d0 0x7fb0e00ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.674+0000 7fb0e4ae3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb0e00ced20 0x7fb0e00d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.674+0000 7fb0e4ae3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e015bad0 con 0x7fb0e015b6f0 2024-04-25T10:16:18.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.674+0000 7fb0e4ae3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e0161c40 con 0x7fb0e00ced20 2024-04-25T10:16:18.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.674+0000 7fb0e4ae3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e015ebe0 con 0x7fb0e01620d0 2024-04-25T10:16:18.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.674+0000 7fb0ded9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb0e00ced20 0x7fb0e00d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.674+0000 7fb0ded9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb0e00ced20 0x7fb0e00d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49968/0 (socket says 172.21.15.60:49968) 2024-04-25T10:16:18.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.674+0000 7fb0ded9d640 1 -- 172.21.15.60:0/1165792638 learned_addr learned my addr 172.21.15.60:0/1165792638 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:18.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.674+0000 7fb0ddd9b640 1 --2- 172.21.15.60:0/1165792638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0e01620d0 0x7fb0e00ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.674+0000 7fb0ded9d640 1 -- 172.21.15.60:0/1165792638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0e01620d0 msgr2=0x7fb0e00ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.674+0000 7fb0ded9d640 1 --2- 172.21.15.60:0/1165792638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0e01620d0 0x7fb0e00ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.674+0000 7fb0ded9d640 1 -- 172.21.15.60:0/1165792638 >> v1:172.21.15.100:6789/0 conn(0x7fb0e015b6f0 legacy=0x7fb0e015e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.674+0000 7fb0ded9d640 1 -- 172.21.15.60:0/1165792638 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e015eec0 con 0x7fb0e00ced20 2024-04-25T10:16:18.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.678+0000 7fb0ded9d640 1 --2- 172.21.15.60:0/1165792638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0e00ced20 0x7fb0e00d1b30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009e50 tx=0x7fb0d801ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7326d114a637abc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.678+0000 7fb0dd59a640 1 -- 172.21.15.60:0/1165792638 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80a56c0 con 0x7fb0e00ced20 2024-04-25T10:16:18.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.678+0000 7fb0dd59a640 1 -- 172.21.15.60:0/1165792638 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d80a5860 con 0x7fb0e00ced20 2024-04-25T10:16:18.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.678+0000 7fb0dd59a640 1 -- 172.21.15.60:0/1165792638 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80a5c30 con 0x7fb0e00ced20 2024-04-25T10:16:18.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.678+0000 7fb0e4ae3640 1 -- 172.21.15.60:0/1165792638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0e00ced20 msgr2=0x7fb0e00d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.678+0000 7fb0e4ae3640 1 --2- 172.21.15.60:0/1165792638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0e00ced20 0x7fb0e00d1b30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009e50 tx=0x7fb0d801ca30 comp rx=0 tx=0).stop 2024-04-25T10:16:18.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.678+0000 7fb0e4ae3640 1 -- 172.21.15.60:0/1165792638 shutdown_connections 2024-04-25T10:16:18.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.678+0000 7fb0e4ae3640 1 --2- 172.21.15.60:0/1165792638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0e00ced20 0x7fb0e00d1b30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.678+0000 7fb0e4ae3640 1 --2- 172.21.15.60:0/1165792638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0e01620d0 0x7fb0e00ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.678+0000 7fb0e4ae3640 1 -- 172.21.15.60:0/1165792638 >> 172.21.15.60:0/1165792638 conn(0x7fb0e0072340 msgr2=0x7fb0e0072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.678+0000 7fb0e4ae3640 1 -- 172.21.15.60:0/1165792638 shutdown_connections 2024-04-25T10:16:18.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.678+0000 7fb0e4ae3640 1 -- 172.21.15.60:0/1165792638 wait complete. 2024-04-25T10:16:18.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0e4ae3640 1 Processor -- start 2024-04-25T10:16:18.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0e4ae3640 1 -- start start 2024-04-25T10:16:18.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0e4ae3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0e00ced20 0x7fb0e01501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0e4ae3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0e015b6f0 0x7fb0e00b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0e4ae3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0e01620d0 0x7fb0e00b8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0e4ae3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e015e960 con 0x7fb0e01620d0 2024-04-25T10:16:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0e4ae3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e015e7e0 con 0x7fb0e015b6f0 2024-04-25T10:16:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0e4ae3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e016aa60 con 0x7fb0e00ced20 2024-04-25T10:16:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0ded9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0e01620d0 0x7fb0e00b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0ded9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0e01620d0 0x7fb0e00b8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35434/0 (socket says 172.21.15.60:35434) 2024-04-25T10:16:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0ded9d640 1 -- 172.21.15.60:0/151528877 learned_addr learned my addr 172.21.15.60:0/151528877 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0ddd9b640 1 --2- 172.21.15.60:0/151528877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0e015b6f0 0x7fb0e00b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0de59c640 1 --2- 172.21.15.60:0/151528877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0e00ced20 0x7fb0e01501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0ded9d640 1 -- 172.21.15.60:0/151528877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0e00ced20 msgr2=0x7fb0e01501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0ded9d640 1 --2- 172.21.15.60:0/151528877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0e00ced20 0x7fb0e01501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0ded9d640 1 -- 172.21.15.60:0/151528877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0e015b6f0 msgr2=0x7fb0e00b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0ded9d640 1 --2- 172.21.15.60:0/151528877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0e015b6f0 0x7fb0e00b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0ded9d640 1 -- 172.21.15.60:0/151528877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e00c7f00 con 0x7fb0e01620d0 2024-04-25T10:16:18.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0ded9d640 1 --2- 172.21.15.60:0/151528877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0e01620d0 0x7fb0e00b8140 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80af040 tx=0x7fb0d80197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0ceffd640 1 -- 172.21.15.60:0/151528877 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80bc020 con 0x7fb0e01620d0 2024-04-25T10:16:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0ceffd640 1 -- 172.21.15.60:0/151528877 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d80b8670 con 0x7fb0e01620d0 2024-04-25T10:16:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0e4ae3640 1 -- 172.21.15.60:0/151528877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e00c81a0 con 0x7fb0e01620d0 2024-04-25T10:16:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0ceffd640 1 -- 172.21.15.60:0/151528877 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8009510 con 0x7fb0e01620d0 2024-04-25T10:16:18.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0e4ae3640 1 -- 172.21.15.60:0/151528877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e00c8760 con 0x7fb0e01620d0 2024-04-25T10:16:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0e4ae3640 1 -- 172.21.15.60:0/151528877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e0074770 con 0x7fb0e01620d0 2024-04-25T10:16:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.682+0000 7fb0ceffd640 1 -- 172.21.15.60:0/151528877 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0d80bf020 con 0x7fb0e01620d0 2024-04-25T10:16:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.686+0000 7fb0ceffd640 1 --2- 172.21.15.60:0/151528877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb0c0066f60 0x7fb0c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.686+0000 7fb0ceffd640 1 -- 172.21.15.60:0/151528877 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0d811ffe0 con 0x7fb0e01620d0 2024-04-25T10:16:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.686+0000 7fb0de59c640 1 --2- 172.21.15.60:0/151528877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb0c0066f60 0x7fb0c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.686+0000 7fb0de59c640 1 --2- 172.21.15.60:0/151528877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb0c0066f60 0x7fb0c0069420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00b8950 tx=0x7fb0d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.690+0000 7fb0ceffd640 1 -- 172.21.15.60:0/151528877 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0d80c6030 con 0x7fb0e01620d0 2024-04-25T10:16:18.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.846+0000 7fb0e4ae3640 1 -- 172.21.15.60:0/151528877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb0e00d16e0 con 0x7fb0e01620d0 2024-04-25T10:16:18.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.850+0000 7fb0ceffd640 1 -- 172.21.15.60:0/151528877 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb0d8016620 con 0x7fb0e01620d0 2024-04-25T10:16:18.854 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:16:18.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.850+0000 7fb0e4ae3640 1 -- 172.21.15.60:0/151528877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb0c0066f60 msgr2=0x7fb0c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.850+0000 7fb0e4ae3640 1 --2- 172.21.15.60:0/151528877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb0c0066f60 0x7fb0c0069420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00b8950 tx=0x7fb0d0009040 comp rx=0 tx=0).stop 2024-04-25T10:16:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.850+0000 7fb0e4ae3640 1 -- 172.21.15.60:0/151528877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0e01620d0 msgr2=0x7fb0e00b8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.850+0000 7fb0e4ae3640 1 --2- 172.21.15.60:0/151528877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0e01620d0 0x7fb0e00b8140 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80af040 tx=0x7fb0d80197d0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.850+0000 7fb0e4ae3640 1 -- 172.21.15.60:0/151528877 shutdown_connections 2024-04-25T10:16:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.850+0000 7fb0e4ae3640 1 --2- 172.21.15.60:0/151528877 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb0c0066f60 0x7fb0c0069420 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.850+0000 7fb0e4ae3640 1 --2- 172.21.15.60:0/151528877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0e01620d0 0x7fb0e00b8140 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.850+0000 7fb0e4ae3640 1 --2- 172.21.15.60:0/151528877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0e015b6f0 0x7fb0e00b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.850+0000 7fb0e4ae3640 1 --2- 172.21.15.60:0/151528877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0e00ced20 0x7fb0e01501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.850+0000 7fb0e4ae3640 1 -- 172.21.15.60:0/151528877 >> 172.21.15.60:0/151528877 conn(0x7fb0e0072340 msgr2=0x7fb0e0151190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.850+0000 7fb0e4ae3640 1 -- 172.21.15.60:0/151528877 shutdown_connections 2024-04-25T10:16:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.850+0000 7fb0e4ae3640 1 -- 172.21.15.60:0/151528877 wait complete. 2024-04-25T10:16:18.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:16:19.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.998+0000 7f1c2cb3e640 1 Processor -- start 2024-04-25T10:16:19.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.998+0000 7f1c2cb3e640 1 -- start start 2024-04-25T10:16:19.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.998+0000 7f1c2cb3e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1c2815d8a0 0x7f1c280ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.998+0000 7f1c2cb3e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1c2815dc80 0x7f1c280b5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.998+0000 7f1c2cb3e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c28070b30 con 0x7f1c280afd60 2024-04-25T10:16:19.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.998+0000 7f1c2cb3e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c28161790 con 0x7f1c2815d8a0 2024-04-25T10:16:19.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.998+0000 7f1c2cb3e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c28161a70 con 0x7f1c2815dc80 2024-04-25T10:16:19.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.998+0000 7f1c2659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1c2815d8a0 0x7f1c280ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.998+0000 7f1c2659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1c2815d8a0 0x7f1c280ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60260/0 (socket says 172.21.15.60:60260) 2024-04-25T10:16:19.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.998+0000 7f1c2659c640 1 -- 172.21.15.60:0/3710791640 learned_addr learned my addr 172.21.15.60:0/3710791640 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:19.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.998+0000 7f1c25d9b640 1 --2- 172.21.15.60:0/3710791640 >> v2:172.21.15.100:3300/0 conn(0x7f1c2815dc80 0x7f1c280b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.998+0000 7f1c2659c640 1 -- 172.21.15.60:0/3710791640 >> v2:172.21.15.100:3300/0 conn(0x7f1c2815dc80 msgr2=0x7f1c280b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:19.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.998+0000 7f1c2659c640 1 --2- 172.21.15.60:0/3710791640 >> v2:172.21.15.100:3300/0 conn(0x7f1c2815dc80 0x7f1c280b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.998+0000 7f1c2659c640 1 -- 172.21.15.60:0/3710791640 >> v1:172.21.15.100:6789/0 conn(0x7f1c280afd60 legacy=0x7f1c28072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:18.998+0000 7f1c2659c640 1 -- 172.21.15.60:0/3710791640 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c28070800 con 0x7f1c2815d8a0 2024-04-25T10:16:19.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.002+0000 7f1c2659c640 1 --2- 172.21.15.60:0/3710791640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815d8a0 0x7f1c280ccd00 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f1c280afc80 tx=0x7f1c1801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0b66a6fad3e330 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:19.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.002+0000 7f1c2559a640 1 -- 172.21.15.60:0/3710791640 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c180a5560 con 0x7f1c2815d8a0 2024-04-25T10:16:19.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.002+0000 7f1c2559a640 1 -- 172.21.15.60:0/3710791640 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c180a5700 con 0x7f1c2815d8a0 2024-04-25T10:16:19.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.002+0000 7f1c2559a640 1 -- 172.21.15.60:0/3710791640 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c180a5ad0 con 0x7f1c2815d8a0 2024-04-25T10:16:19.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.002+0000 7f1c2cb3e640 1 -- 172.21.15.60:0/3710791640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815d8a0 msgr2=0x7f1c280ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.002+0000 7f1c2cb3e640 1 --2- 172.21.15.60:0/3710791640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815d8a0 0x7f1c280ccd00 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f1c280afc80 tx=0x7f1c1801c8a0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.002+0000 7f1c2cb3e640 1 -- 172.21.15.60:0/3710791640 shutdown_connections 2024-04-25T10:16:19.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.002+0000 7f1c2cb3e640 1 --2- 172.21.15.60:0/3710791640 >> v2:172.21.15.100:3300/0 conn(0x7f1c2815dc80 0x7f1c280b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.002+0000 7f1c2cb3e640 1 --2- 172.21.15.60:0/3710791640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815d8a0 0x7f1c280ccd00 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.002+0000 7f1c2cb3e640 1 -- 172.21.15.60:0/3710791640 >> 172.21.15.60:0/3710791640 conn(0x7f1c2806e400 msgr2=0x7f1c2806e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:19.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.002+0000 7f1c2cb3e640 1 -- 172.21.15.60:0/3710791640 shutdown_connections 2024-04-25T10:16:19.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.002+0000 7f1c2cb3e640 1 -- 172.21.15.60:0/3710791640 wait complete. 2024-04-25T10:16:19.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c2cb3e640 1 Processor -- start 2024-04-25T10:16:19.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c2cb3e640 1 -- start start 2024-04-25T10:16:19.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c2cb3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c280afd60 0x7f1c28150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c2cb3e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c2815d8a0 0x7f1c28154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c2cb3e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815b7b0 0x7f1c28157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c2cb3e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2816ad20 con 0x7f1c2815b7b0 2024-04-25T10:16:19.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c2cb3e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2816aba0 con 0x7f1c280afd60 2024-04-25T10:16:19.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c2cb3e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2816aea0 con 0x7f1c2815d8a0 2024-04-25T10:16:19.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c26d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815b7b0 0x7f1c28157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c26d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815b7b0 0x7f1c28157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60276/0 (socket says 172.21.15.60:60276) 2024-04-25T10:16:19.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c26d9d640 1 -- 172.21.15.60:0/493116166 learned_addr learned my addr 172.21.15.60:0/493116166 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:19.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c2659c640 1 --2- 172.21.15.60:0/493116166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c280afd60 0x7f1c28150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c25d9b640 1 --2- 172.21.15.60:0/493116166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c2815d8a0 0x7f1c28154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c26d9d640 1 -- 172.21.15.60:0/493116166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c2815d8a0 msgr2=0x7f1c28154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c26d9d640 1 --2- 172.21.15.60:0/493116166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c2815d8a0 0x7f1c28154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c26d9d640 1 -- 172.21.15.60:0/493116166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c280afd60 msgr2=0x7f1c28150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c26d9d640 1 --2- 172.21.15.60:0/493116166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c280afd60 0x7f1c28150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c26d9d640 1 -- 172.21.15.60:0/493116166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c28157a40 con 0x7f1c2815b7b0 2024-04-25T10:16:19.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c2659c640 1 --2- 172.21.15.60:0/493116166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c280afd60 0x7f1c28150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:19.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c26d9d640 1 --2- 172.21.15.60:0/493116166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815b7b0 0x7f1c28157f40 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f1c2000c370 tx=0x7f1c2000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:19.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c1effd640 1 -- 172.21.15.60:0/493116166 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20005220 con 0x7f1c2815b7b0 2024-04-25T10:16:19.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c1effd640 1 -- 172.21.15.60:0/493116166 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c200053c0 con 0x7f1c2815b7b0 2024-04-25T10:16:19.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c1effd640 1 -- 172.21.15.60:0/493116166 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20007270 con 0x7f1c2815b7b0 2024-04-25T10:16:19.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c2cb3e640 1 -- 172.21.15.60:0/493116166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c28154da0 con 0x7f1c2815b7b0 2024-04-25T10:16:19.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c2cb3e640 1 -- 172.21.15.60:0/493116166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c28155350 con 0x7f1c2815b7b0 2024-04-25T10:16:19.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c2cb3e640 1 -- 172.21.15.60:0/493116166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c28074770 con 0x7f1c2815b7b0 2024-04-25T10:16:19.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.006+0000 7f1c1effd640 1 -- 172.21.15.60:0/493116166 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c20019020 con 0x7f1c2815b7b0 2024-04-25T10:16:19.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.010+0000 7f1c1effd640 1 --2- 172.21.15.60:0/493116166 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c08066f60 0x7f1c08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.010+0000 7f1c1effd640 1 -- 172.21.15.60:0/493116166 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c20084310 con 0x7f1c2815b7b0 2024-04-25T10:16:19.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.010+0000 7f1c2659c640 1 --2- 172.21.15.60:0/493116166 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c08066f60 0x7f1c08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.010+0000 7f1c2659c640 1 --2- 172.21.15.60:0/493116166 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c08066f60 0x7f1c08069420 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1c180ae000 tx=0x7f1c180a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:19.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.014+0000 7f1c1effd640 1 -- 172.21.15.60:0/493116166 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c20055320 con 0x7f1c2815b7b0 2024-04-25T10:16:19.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.174+0000 7f1c2cb3e640 1 -- 172.21.15.60:0/493116166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1c280700d0 con 0x7f1c2815b7b0 2024-04-25T10:16:19.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.174+0000 7f1c1effd640 1 -- 172.21.15.60:0/493116166 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1c20054c20 con 0x7f1c2815b7b0 2024-04-25T10:16:19.178 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:16:19.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.174+0000 7f1c2cb3e640 1 -- 172.21.15.60:0/493116166 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c08066f60 msgr2=0x7f1c08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.174+0000 7f1c2cb3e640 1 --2- 172.21.15.60:0/493116166 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c08066f60 0x7f1c08069420 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1c180ae000 tx=0x7f1c180a7070 comp rx=0 tx=0).stop 2024-04-25T10:16:19.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.174+0000 7f1c2cb3e640 1 -- 172.21.15.60:0/493116166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815b7b0 msgr2=0x7f1c28157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.174+0000 7f1c2cb3e640 1 --2- 172.21.15.60:0/493116166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815b7b0 0x7f1c28157f40 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f1c2000c370 tx=0x7f1c2000c830 comp rx=0 tx=0).stop 2024-04-25T10:16:19.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.174+0000 7f1c2cb3e640 1 -- 172.21.15.60:0/493116166 shutdown_connections 2024-04-25T10:16:19.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.174+0000 7f1c2cb3e640 1 --2- 172.21.15.60:0/493116166 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1c08066f60 0x7f1c08069420 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.174+0000 7f1c2cb3e640 1 --2- 172.21.15.60:0/493116166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815b7b0 0x7f1c28157f40 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.174+0000 7f1c2cb3e640 1 --2- 172.21.15.60:0/493116166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c2815d8a0 0x7f1c28154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.174+0000 7f1c2cb3e640 1 --2- 172.21.15.60:0/493116166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c280afd60 0x7f1c28150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.174+0000 7f1c2cb3e640 1 -- 172.21.15.60:0/493116166 >> 172.21.15.60:0/493116166 conn(0x7f1c2806e400 msgr2=0x7f1c280b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:19.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.174+0000 7f1c2cb3e640 1 -- 172.21.15.60:0/493116166 shutdown_connections 2024-04-25T10:16:19.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.174+0000 7f1c2cb3e640 1 -- 172.21.15.60:0/493116166 wait complete. 2024-04-25T10:16:19.198 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:16:19.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.314+0000 7fdc3edab640 1 Processor -- start 2024-04-25T10:16:19.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.314+0000 7fdc3edab640 1 -- start start 2024-04-25T10:16:19.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.314+0000 7fdc3edab640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdc3815deb0 0x7fdc380b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.314+0000 7fdc3edab640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdc380ce9c0 0x7fdc380d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.314+0000 7fdc3edab640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3806d200 con 0x7fdc380d0340 2024-04-25T10:16:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.314+0000 7fdc3edab640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc380cf2c0 con 0x7fdc3815deb0 2024-04-25T10:16:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.314+0000 7fdc3edab640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc380cefe0 con 0x7fdc380ce9c0 2024-04-25T10:16:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.314+0000 7fdc3cb47640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdc3815deb0 0x7fdc380b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.314+0000 7fdc3cb47640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdc3815deb0 0x7fdc380b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60280/0 (socket says 172.21.15.60:60280) 2024-04-25T10:16:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.314+0000 7fdc3cb47640 1 -- 172.21.15.60:0/667352865 learned_addr learned my addr 172.21.15.60:0/667352865 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc3d348640 1 --2- 172.21.15.60:0/667352865 >> v2:172.21.15.100:3300/0 conn(0x7fdc380ce9c0 0x7fdc380d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc3cb47640 1 -- 172.21.15.60:0/667352865 >> v2:172.21.15.100:3300/0 conn(0x7fdc380ce9c0 msgr2=0x7fdc380d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc3cb47640 1 --2- 172.21.15.60:0/667352865 >> v2:172.21.15.100:3300/0 conn(0x7fdc380ce9c0 0x7fdc380d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc3cb47640 1 -- 172.21.15.60:0/667352865 >> v1:172.21.15.100:6789/0 conn(0x7fdc380d0340 legacy=0x7fdc380ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc3cb47640 1 -- 172.21.15.60:0/667352865 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc380d1d70 con 0x7fdc3815deb0 2024-04-25T10:16:19.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc3cb47640 1 --2- 172.21.15.60:0/667352865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc3815deb0 0x7fdc380b6c10 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fdc380b9a00 tx=0x7fdc2801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502e8ec6808631b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:19.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc377fe640 1 -- 172.21.15.60:0/667352865 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc280a5670 con 0x7fdc3815deb0 2024-04-25T10:16:19.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc377fe640 1 -- 172.21.15.60:0/667352865 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc280a5810 con 0x7fdc3815deb0 2024-04-25T10:16:19.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc377fe640 1 -- 172.21.15.60:0/667352865 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc280a5be0 con 0x7fdc3815deb0 2024-04-25T10:16:19.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc3edab640 1 -- 172.21.15.60:0/667352865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc3815deb0 msgr2=0x7fdc380b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc3edab640 1 --2- 172.21.15.60:0/667352865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc3815deb0 0x7fdc380b6c10 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fdc380b9a00 tx=0x7fdc2801c9b0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc3edab640 1 -- 172.21.15.60:0/667352865 shutdown_connections 2024-04-25T10:16:19.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc3edab640 1 --2- 172.21.15.60:0/667352865 >> v2:172.21.15.100:3300/0 conn(0x7fdc380ce9c0 0x7fdc380d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc3edab640 1 --2- 172.21.15.60:0/667352865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc3815deb0 0x7fdc380b6c10 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc3edab640 1 -- 172.21.15.60:0/667352865 >> 172.21.15.60:0/667352865 conn(0x7fdc3806c090 msgr2=0x7fdc3806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:19.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc3edab640 1 -- 172.21.15.60:0/667352865 shutdown_connections 2024-04-25T10:16:19.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.318+0000 7fdc3edab640 1 -- 172.21.15.60:0/667352865 wait complete. 2024-04-25T10:16:19.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3edab640 1 Processor -- start 2024-04-25T10:16:19.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3edab640 1 -- start start 2024-04-25T10:16:19.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3edab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc380ce9c0 0x7fdc380cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3edab640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc380d0340 0x7fdc3815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3edab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc3815deb0 0x7fdc3815cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3edab640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380d3430 con 0x7fdc380ce9c0 2024-04-25T10:16:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3edab640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380d32b0 con 0x7fdc3815deb0 2024-04-25T10:16:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3edab640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380d35b0 con 0x7fdc380d0340 2024-04-25T10:16:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3cb47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc380ce9c0 0x7fdc380cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3cb47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc380ce9c0 0x7fdc380cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60290/0 (socket says 172.21.15.60:60290) 2024-04-25T10:16:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3cb47640 1 -- 172.21.15.60:0/139356376 learned_addr learned my addr 172.21.15.60:0/139356376 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3d348640 1 --2- 172.21.15.60:0/139356376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc3815deb0 0x7fdc3815cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc37fff640 1 --2- 172.21.15.60:0/139356376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc380d0340 0x7fdc3815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3cb47640 1 -- 172.21.15.60:0/139356376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc380d0340 msgr2=0x7fdc3815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3cb47640 1 --2- 172.21.15.60:0/139356376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc380d0340 0x7fdc3815a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3cb47640 1 -- 172.21.15.60:0/139356376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc3815deb0 msgr2=0x7fdc3815cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3cb47640 1 --2- 172.21.15.60:0/139356376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc3815deb0 0x7fdc3815cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3cb47640 1 -- 172.21.15.60:0/139356376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3815d6e0 con 0x7fdc380ce9c0 2024-04-25T10:16:19.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3d348640 1 --2- 172.21.15.60:0/139356376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc3815deb0 0x7fdc3815cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:19.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3cb47640 1 --2- 172.21.15.60:0/139356376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc380ce9c0 0x7fdc380cb380 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fdc280af040 tx=0x7fdc28019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:19.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc357fa640 1 -- 172.21.15.60:0/139356376 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28009510 con 0x7fdc380ce9c0 2024-04-25T10:16:19.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc357fa640 1 -- 172.21.15.60:0/139356376 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc280be070 con 0x7fdc380ce9c0 2024-04-25T10:16:19.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3edab640 1 -- 172.21.15.60:0/139356376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3814ea10 con 0x7fdc380ce9c0 2024-04-25T10:16:19.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc357fa640 1 -- 172.21.15.60:0/139356376 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc280b88d0 con 0x7fdc380ce9c0 2024-04-25T10:16:19.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3edab640 1 -- 172.21.15.60:0/139356376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3814efd0 con 0x7fdc380ce9c0 2024-04-25T10:16:19.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc3edab640 1 -- 172.21.15.60:0/139356376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc38074770 con 0x7fdc380ce9c0 2024-04-25T10:16:19.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.322+0000 7fdc357fa640 1 -- 172.21.15.60:0/139356376 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc280b8a70 con 0x7fdc380ce9c0 2024-04-25T10:16:19.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.326+0000 7fdc357fa640 1 --2- 172.21.15.60:0/139356376 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc24066f60 0x7fdc24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.326+0000 7fdc357fa640 1 -- 172.21.15.60:0/139356376 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc28120090 con 0x7fdc380ce9c0 2024-04-25T10:16:19.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.326+0000 7fdc37fff640 1 --2- 172.21.15.60:0/139356376 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc24066f60 0x7fdc24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.326+0000 7fdc37fff640 1 --2- 172.21.15.60:0/139356376 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc24066f60 0x7fdc24069420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fdc380cad30 tx=0x7fdc2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:19.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.330+0000 7fdc357fa640 1 -- 172.21.15.60:0/139356376 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc280165a0 con 0x7fdc380ce9c0 2024-04-25T10:16:19.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.494+0000 7fdc3edab640 1 -- 172.21.15.60:0/139356376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdc380d17a0 con 0x7fdc380ce9c0 2024-04-25T10:16:19.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.494+0000 7fdc357fa640 1 -- 172.21.15.60:0/139356376 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdc280f11c0 con 0x7fdc380ce9c0 2024-04-25T10:16:19.501 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:16:19.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.498+0000 7fdc3edab640 1 -- 172.21.15.60:0/139356376 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc24066f60 msgr2=0x7fdc24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.498+0000 7fdc3edab640 1 --2- 172.21.15.60:0/139356376 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc24066f60 0x7fdc24069420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fdc380cad30 tx=0x7fdc2c009040 comp rx=0 tx=0).stop 2024-04-25T10:16:19.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.498+0000 7fdc3edab640 1 -- 172.21.15.60:0/139356376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc380ce9c0 msgr2=0x7fdc380cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.498+0000 7fdc3edab640 1 --2- 172.21.15.60:0/139356376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc380ce9c0 0x7fdc380cb380 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fdc280af040 tx=0x7fdc28019750 comp rx=0 tx=0).stop 2024-04-25T10:16:19.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.498+0000 7fdc3edab640 1 -- 172.21.15.60:0/139356376 shutdown_connections 2024-04-25T10:16:19.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.498+0000 7fdc3edab640 1 --2- 172.21.15.60:0/139356376 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdc24066f60 0x7fdc24069420 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.498+0000 7fdc3edab640 1 --2- 172.21.15.60:0/139356376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc3815deb0 0x7fdc3815cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.498+0000 7fdc3edab640 1 --2- 172.21.15.60:0/139356376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc380d0340 0x7fdc3815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.498+0000 7fdc3edab640 1 --2- 172.21.15.60:0/139356376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc380ce9c0 0x7fdc380cb380 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.498+0000 7fdc3edab640 1 -- 172.21.15.60:0/139356376 >> 172.21.15.60:0/139356376 conn(0x7fdc3806c090 msgr2=0x7fdc38160d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:19.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.498+0000 7fdc3edab640 1 -- 172.21.15.60:0/139356376 shutdown_connections 2024-04-25T10:16:19.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.498+0000 7fdc3edab640 1 -- 172.21.15.60:0/139356376 wait complete. 2024-04-25T10:16:19.519 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:16:19.519 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:19.525 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.690+0000 7f82918c7640 1 Processor -- start 2024-04-25T10:16:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.690+0000 7f82918c7640 1 -- start start 2024-04-25T10:16:19.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.690+0000 7f82918c7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f828c15bc90 0x7f828c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.690+0000 7f82918c7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f828c0b4660 0x7f828c15c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.690+0000 7f82918c7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828c071ca0 con 0x7f828c0ce7a0 2024-04-25T10:16:19.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.690+0000 7f82918c7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f828c0cc370 con 0x7f828c15bc90 2024-04-25T10:16:19.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.690+0000 7f82918c7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0cca40 con 0x7f828c0b4660 2024-04-25T10:16:19.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.690+0000 7f828affd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f828c0ce7a0 0x7f828c0ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35082/0 (socket says 172.21.15.60:35082) 2024-04-25T10:16:19.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.690+0000 7f828affd640 1 -- 172.21.15.60:0/1455305178 learned_addr learned my addr 172.21.15.60:0/1455305178 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:19.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f828b7fe640 1 --2- 172.21.15.60:0/1455305178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f828c0b4660 0x7f828c15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f828a7fc640 1 --2- 172.21.15.60:0/1455305178 >> v2:172.21.15.100:3300/0 conn(0x7f828c15bc90 0x7f828c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 -- 172.21.15.60:0/1455305178 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334529758 0 0) 0x7f828c071ca0 con 0x7f828c0ce7a0 2024-04-25T10:16:19.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 -- 172.21.15.60:0/1455305178 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8278003530 con 0x7f828c0ce7a0 2024-04-25T10:16:19.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 -- 172.21.15.60:0/1455305178 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 686505727 0 0) 0x7f8278003530 con 0x7f828c0ce7a0 2024-04-25T10:16:19.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 -- 172.21.15.60:0/1455305178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f828c0b4660 msgr2=0x7f828c15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:19.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 --2- 172.21.15.60:0/1455305178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f828c0b4660 0x7f828c15c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 -- 172.21.15.60:0/1455305178 >> v2:172.21.15.100:3300/0 conn(0x7f828c15bc90 msgr2=0x7f828c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:19.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 --2- 172.21.15.60:0/1455305178 >> v2:172.21.15.100:3300/0 conn(0x7f828c15bc90 0x7f828c15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 -- 172.21.15.60:0/1455305178 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c0ccd20 con 0x7f828c0ce7a0 2024-04-25T10:16:19.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 -- 172.21.15.60:0/1455305178 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f827c001680 con 0x7f828c0ce7a0 2024-04-25T10:16:19.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 -- 172.21.15.60:0/1455305178 >> v1:172.21.15.60:6789/0 conn(0x7f828c0ce7a0 legacy=0x7f828c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 --2- 172.21.15.60:0/1455305178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82780039c0 0x7f8278003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 --2- 172.21.15.60:0/1455305178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8278004810 0x7f8278004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 --2- 172.21.15.60:0/1455305178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8278005190 0x7f827800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 -- 172.21.15.60:0/1455305178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0cca40 con 0x7f82780039c0 2024-04-25T10:16:19.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 -- 172.21.15.60:0/1455305178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8278004590 con 0x7f8278005190 2024-04-25T10:16:19.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 -- 172.21.15.60:0/1455305178 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827800bf10 con 0x7f8278004810 2024-04-25T10:16:19.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f828affd640 1 --2- 172.21.15.60:0/1455305178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82780039c0 0x7f8278003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f828a7fc640 1 --2- 172.21.15.60:0/1455305178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8278004810 0x7f8278004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f828b7fe640 1 --2- 172.21.15.60:0/1455305178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8278005190 0x7f827800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f828affd640 1 -- 172.21.15.60:0/1455305178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8278004810 msgr2=0x7f8278004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:19.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f828affd640 1 --2- 172.21.15.60:0/1455305178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8278004810 0x7f8278004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f828affd640 1 -- 172.21.15.60:0/1455305178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8278005190 msgr2=0x7f827800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:19.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f828affd640 1 --2- 172.21.15.60:0/1455305178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8278005190 0x7f827800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f828affd640 1 -- 172.21.15.60:0/1455305178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8278003530 con 0x7f82780039c0 2024-04-25T10:16:19.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f828affd640 1 --2- 172.21.15.60:0/1455305178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82780039c0 0x7f8278003de0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f827c00c210 tx=0x7f827c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585ae6228e0b8eb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:19.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 -- 172.21.15.60:0/1455305178 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c007d60 con 0x7f82780039c0 2024-04-25T10:16:19.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 -- 172.21.15.60:0/1455305178 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c0a9020 con 0x7f82780039c0 2024-04-25T10:16:19.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f8289ffb640 1 -- 172.21.15.60:0/1455305178 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c005b20 con 0x7f82780039c0 2024-04-25T10:16:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f82918c7640 1 -- 172.21.15.60:0/1455305178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82780039c0 msgr2=0x7f8278003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f82918c7640 1 --2- 172.21.15.60:0/1455305178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82780039c0 0x7f8278003de0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f827c00c210 tx=0x7f827c01cca0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f828b7fe640 1 -- 172.21.15.60:0/1455305178 reap_dead start 2024-04-25T10:16:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f82918c7640 1 -- 172.21.15.60:0/1455305178 shutdown_connections 2024-04-25T10:16:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.694+0000 7f82918c7640 1 -- 172.21.15.60:0/1455305178 >> 172.21.15.60:0/1455305178 conn(0x7f828c0b12d0 msgr2=0x7f828c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f82918c7640 1 -- 172.21.15.60:0/1455305178 shutdown_connections 2024-04-25T10:16:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f82918c7640 1 -- 172.21.15.60:0/1455305178 wait complete. 2024-04-25T10:16:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f82918c7640 1 Processor -- start 2024-04-25T10:16:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f82918c7640 1 -- start start 2024-04-25T10:16:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f82918c7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82780039c0 0x7f828c1f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f82918c7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f828c151a60 0x7f828c15a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f82918c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c14ebc0 0x7f828c159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f82918c7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0b3db0 con 0x7f82780039c0 2024-04-25T10:16:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f82918c7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0b4490 con 0x7f828c14ebc0 2024-04-25T10:16:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f82918c7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c15cf70 con 0x7f828c151a60 2024-04-25T10:16:19.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f828affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82780039c0 0x7f828c1f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f828affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82780039c0 0x7f828c1f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60310/0 (socket says 172.21.15.60:60310) 2024-04-25T10:16:19.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f828affd640 1 -- 172.21.15.60:0/2869716058 learned_addr learned my addr 172.21.15.60:0/2869716058 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:19.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f828b7fe640 1 --2- 172.21.15.60:0/2869716058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c14ebc0 0x7f828c159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f828a7fc640 1 --2- 172.21.15.60:0/2869716058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f828c151a60 0x7f828c15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f828affd640 1 -- 172.21.15.60:0/2869716058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f828c151a60 msgr2=0x7f828c15a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f828affd640 1 --2- 172.21.15.60:0/2869716058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f828c151a60 0x7f828c15a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f828affd640 1 -- 172.21.15.60:0/2869716058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c14ebc0 msgr2=0x7f828c159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f828affd640 1 --2- 172.21.15.60:0/2869716058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c14ebc0 0x7f828c159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f828affd640 1 -- 172.21.15.60:0/2869716058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c1f7e00 con 0x7f82780039c0 2024-04-25T10:16:19.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f828b7fe640 1 --2- 172.21.15.60:0/2869716058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c14ebc0 0x7f828c159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:19.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f828a7fc640 1 --2- 172.21.15.60:0/2869716058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f828c151a60 0x7f828c15a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:19.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f828affd640 1 --2- 172.21.15.60:0/2869716058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82780039c0 0x7f828c1f78c0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f827c0ab040 tx=0x7f8278005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:19.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f8283fff640 1 -- 172.21.15.60:0/2869716058 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c005dc0 con 0x7f82780039c0 2024-04-25T10:16:19.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f8283fff640 1 -- 172.21.15.60:0/2869716058 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c0047f0 con 0x7f82780039c0 2024-04-25T10:16:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f8283fff640 1 -- 172.21.15.60:0/2869716058 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c004bc0 con 0x7f82780039c0 2024-04-25T10:16:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f82918c7640 1 -- 172.21.15.60:0/2869716058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828c152030 con 0x7f82780039c0 2024-04-25T10:16:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f82918c7640 1 -- 172.21.15.60:0/2869716058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c1f70c0 con 0x7f82780039c0 2024-04-25T10:16:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.698+0000 7f82918c7640 1 -- 172.21.15.60:0/2869716058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828c074770 con 0x7f82780039c0 2024-04-25T10:16:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.702+0000 7f8283fff640 1 -- 172.21.15.60:0/2869716058 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f827c0a5780 con 0x7f82780039c0 2024-04-25T10:16:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.702+0000 7f8283fff640 1 --2- 172.21.15.60:0/2869716058 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f826c066f90 0x7f826c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.702+0000 7f8283fff640 1 -- 172.21.15.60:0/2869716058 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f827c11e1f0 con 0x7f82780039c0 2024-04-25T10:16:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.702+0000 7f828a7fc640 1 --2- 172.21.15.60:0/2869716058 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f826c066f90 0x7f826c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.702+0000 7f828a7fc640 1 --2- 172.21.15.60:0/2869716058 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f826c066f90 0x7f826c069450 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f828c15f2d0 tx=0x7f827400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:19.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.706+0000 7f8283fff640 1 -- 172.21.15.60:0/2869716058 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f827c0b4070 con 0x7f82780039c0 2024-04-25T10:16:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.866+0000 7f82918c7640 1 -- 172.21.15.60:0/2869716058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f828c152550 con 0x7f82780039c0 2024-04-25T10:16:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.870+0000 7f8283fff640 1 -- 172.21.15.60:0/2869716058 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16639 (secure 0 0 0) 0x7f827c0169b0 con 0x7f82780039c0 2024-04-25T10:16:19.873 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:19.874 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":110,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:16:09.041178+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:06.033387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:06.753421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:19.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.870+0000 7f82918c7640 1 -- 172.21.15.60:0/2869716058 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f826c066f90 msgr2=0x7f826c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.870+0000 7f82918c7640 1 --2- 172.21.15.60:0/2869716058 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f826c066f90 0x7f826c069450 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f828c15f2d0 tx=0x7f827400b040 comp rx=0 tx=0).stop 2024-04-25T10:16:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.870+0000 7f82918c7640 1 -- 172.21.15.60:0/2869716058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82780039c0 msgr2=0x7f828c1f78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.870+0000 7f82918c7640 1 --2- 172.21.15.60:0/2869716058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82780039c0 0x7f828c1f78c0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f827c0ab040 tx=0x7f8278005840 comp rx=0 tx=0).stop 2024-04-25T10:16:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.870+0000 7f82918c7640 1 -- 172.21.15.60:0/2869716058 shutdown_connections 2024-04-25T10:16:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.870+0000 7f82918c7640 1 --2- 172.21.15.60:0/2869716058 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f826c066f90 0x7f826c069450 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.870+0000 7f82918c7640 1 --2- 172.21.15.60:0/2869716058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f828c14ebc0 0x7f828c159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.870+0000 7f82918c7640 1 --2- 172.21.15.60:0/2869716058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f828c151a60 0x7f828c15a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.870+0000 7f82918c7640 1 --2- 172.21.15.60:0/2869716058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82780039c0 0x7f828c1f78c0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.870+0000 7f82918c7640 1 -- 172.21.15.60:0/2869716058 >> 172.21.15.60:0/2869716058 conn(0x7f828c0b12d0 msgr2=0x7f828c0b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.870+0000 7f82918c7640 1 -- 172.21.15.60:0/2869716058 shutdown_connections 2024-04-25T10:16:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:19.870+0000 7f82918c7640 1 -- 172.21.15.60:0/2869716058 wait complete. 2024-04-25T10:16:19.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:06.033387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:06.753421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:19.893 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:20.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.018+0000 7f6008b94640 1 Processor -- start 2024-04-25T10:16:20.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.018+0000 7f6008b94640 1 -- start start 2024-04-25T10:16:20.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.018+0000 7f6008b94640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f600415b6f0 0x7f60041620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.018+0000 7f6008b94640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f60040ceef0 0x7f600415e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.018+0000 7f6008b94640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600415bad0 con 0x7f600415e0d0 2024-04-25T10:16:20.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.018+0000 7f6008b94640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6004162610 con 0x7f60040ceef0 2024-04-25T10:16:20.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.018+0000 7f6008b94640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6004161c40 con 0x7f600415b6f0 2024-04-25T10:16:20.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.018+0000 7f6002d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f60040ceef0 0x7f600415e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.018+0000 7f6002d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f60040ceef0 0x7f600415e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60318/0 (socket says 172.21.15.60:60318) 2024-04-25T10:16:20.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.018+0000 7f6002d9d640 1 -- 172.21.15.60:0/978052251 learned_addr learned my addr 172.21.15.60:0/978052251 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:20.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.018+0000 7f600259c640 1 --2- 172.21.15.60:0/978052251 >> v2:172.21.15.100:3300/0 conn(0x7f600415b6f0 0x7f60041620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.018+0000 7f6002d9d640 1 -- 172.21.15.60:0/978052251 >> v2:172.21.15.100:3300/0 conn(0x7f600415b6f0 msgr2=0x7f60041620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.018+0000 7f6002d9d640 1 --2- 172.21.15.60:0/978052251 >> v2:172.21.15.100:3300/0 conn(0x7f600415b6f0 0x7f60041620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.018+0000 7f6002d9d640 1 -- 172.21.15.60:0/978052251 >> v1:172.21.15.100:6789/0 conn(0x7f600415e0d0 legacy=0x7f60040ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.018+0000 7f6002d9d640 1 -- 172.21.15.60:0/978052251 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600415ebe0 con 0x7f60040ceef0 2024-04-25T10:16:20.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.018+0000 7f6002d9d640 1 --2- 172.21.15.60:0/978052251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60040ceef0 0x7f600415e510 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f6004073060 tx=0x7f5ffc01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c98118e54f1c025 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6001d9b640 1 -- 172.21.15.60:0/978052251 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc0a5760 con 0x7f60040ceef0 2024-04-25T10:16:20.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6001d9b640 1 -- 172.21.15.60:0/978052251 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc0a5900 con 0x7f60040ceef0 2024-04-25T10:16:20.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6001d9b640 1 -- 172.21.15.60:0/978052251 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc0a5cd0 con 0x7f60040ceef0 2024-04-25T10:16:20.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 -- 172.21.15.60:0/978052251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60040ceef0 msgr2=0x7f600415e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 --2- 172.21.15.60:0/978052251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60040ceef0 0x7f600415e510 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f6004073060 tx=0x7f5ffc01c9b0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 -- 172.21.15.60:0/978052251 shutdown_connections 2024-04-25T10:16:20.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 --2- 172.21.15.60:0/978052251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60040ceef0 0x7f600415e510 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 --2- 172.21.15.60:0/978052251 >> v2:172.21.15.100:3300/0 conn(0x7f600415b6f0 0x7f60041620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 -- 172.21.15.60:0/978052251 >> 172.21.15.60:0/978052251 conn(0x7f6004072340 msgr2=0x7f6004072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 -- 172.21.15.60:0/978052251 shutdown_connections 2024-04-25T10:16:20.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 -- 172.21.15.60:0/978052251 wait complete. 2024-04-25T10:16:20.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 Processor -- start 2024-04-25T10:16:20.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 -- start start 2024-04-25T10:16:20.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60040ceef0 0x7f6004148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600415b6f0 0x7f60040d3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600415e0d0 0x7f60040c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60040d24c0 con 0x7f600415b6f0 2024-04-25T10:16:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60040d2340 con 0x7f60040ceef0 2024-04-25T10:16:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60040d2640 con 0x7f600415e0d0 2024-04-25T10:16:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600415b6f0 0x7f60040d3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600415b6f0 0x7f60040d3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60324/0 (socket says 172.21.15.60:60324) 2024-04-25T10:16:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f5ffbfff640 1 -- 172.21.15.60:0/3713475680 learned_addr learned my addr 172.21.15.60:0/3713475680 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6002d9d640 1 --2- 172.21.15.60:0/3713475680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600415e0d0 0x7f60040c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f600259c640 1 --2- 172.21.15.60:0/3713475680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60040ceef0 0x7f6004148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f5ffbfff640 1 -- 172.21.15.60:0/3713475680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600415e0d0 msgr2=0x7f60040c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f5ffbfff640 1 --2- 172.21.15.60:0/3713475680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600415e0d0 0x7f60040c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f5ffbfff640 1 -- 172.21.15.60:0/3713475680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60040ceef0 msgr2=0x7f6004148190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f5ffbfff640 1 --2- 172.21.15.60:0/3713475680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60040ceef0 0x7f6004148190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f5ffbfff640 1 -- 172.21.15.60:0/3713475680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60040d44f0 con 0x7f600415b6f0 2024-04-25T10:16:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f600259c640 1 --2- 172.21.15.60:0/3713475680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60040ceef0 0x7f6004148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6002d9d640 1 --2- 172.21.15.60:0/3713475680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600415e0d0 0x7f60040c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f5ffbfff640 1 --2- 172.21.15.60:0/3713475680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600415b6f0 0x7f60040d3fb0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00b410 tx=0x7f5fec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f5ffaffd640 1 -- 172.21.15.60:0/3713475680 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec015030 con 0x7f600415b6f0 2024-04-25T10:16:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f5ffaffd640 1 -- 172.21.15.60:0/3713475680 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec004020 con 0x7f600415b6f0 2024-04-25T10:16:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 -- 172.21.15.60:0/3713475680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60040c81a0 con 0x7f600415b6f0 2024-04-25T10:16:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f5ffaffd640 1 -- 172.21.15.60:0/3713475680 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec004450 con 0x7f600415b6f0 2024-04-25T10:16:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.022+0000 7f6008b94640 1 -- 172.21.15.60:0/3713475680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60040c8760 con 0x7f600415b6f0 2024-04-25T10:16:20.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.026+0000 7f6008b94640 1 -- 172.21.15.60:0/3713475680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6004074770 con 0x7f600415b6f0 2024-04-25T10:16:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.026+0000 7f5ffaffd640 1 -- 172.21.15.60:0/3713475680 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5fec018070 con 0x7f600415b6f0 2024-04-25T10:16:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.026+0000 7f5ffaffd640 1 --2- 172.21.15.60:0/3713475680 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5fe4066f60 0x7f5fe4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.026+0000 7f5ffaffd640 1 -- 172.21.15.60:0/3713475680 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fec084b00 con 0x7f600415b6f0 2024-04-25T10:16:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.026+0000 7f600259c640 1 --2- 172.21.15.60:0/3713475680 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5fe4066f60 0x7f5fe4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.026+0000 7f600259c640 1 --2- 172.21.15.60:0/3713475680 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5fe4066f60 0x7f5fe4069420 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4006fd0 tx=0x7f5ff4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.030+0000 7f5ffaffd640 1 -- 172.21.15.60:0/3713475680 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f60040d44f0 con 0x7f600415b6f0 2024-04-25T10:16:20.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.182+0000 7f6008b94640 1 -- 172.21.15.60:0/3713475680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f60040d16e0 con 0x7f600415b6f0 2024-04-25T10:16:20.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.182+0000 7f5ffaffd640 1 -- 172.21.15.60:0/3713475680 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5fec055b10 con 0x7f600415b6f0 2024-04-25T10:16:20.188 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:16:20.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.186+0000 7f6008b94640 1 -- 172.21.15.60:0/3713475680 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5fe4066f60 msgr2=0x7f5fe4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.186+0000 7f6008b94640 1 --2- 172.21.15.60:0/3713475680 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5fe4066f60 0x7f5fe4069420 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4006fd0 tx=0x7f5ff4008040 comp rx=0 tx=0).stop 2024-04-25T10:16:20.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.186+0000 7f6008b94640 1 -- 172.21.15.60:0/3713475680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600415b6f0 msgr2=0x7f60040d3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.186+0000 7f6008b94640 1 --2- 172.21.15.60:0/3713475680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600415b6f0 0x7f60040d3fb0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00b410 tx=0x7f5fec00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.186+0000 7f6008b94640 1 -- 172.21.15.60:0/3713475680 shutdown_connections 2024-04-25T10:16:20.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.186+0000 7f6008b94640 1 --2- 172.21.15.60:0/3713475680 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5fe4066f60 0x7f5fe4069420 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.186+0000 7f6008b94640 1 --2- 172.21.15.60:0/3713475680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600415e0d0 0x7f60040c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.186+0000 7f6008b94640 1 --2- 172.21.15.60:0/3713475680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600415b6f0 0x7f60040d3fb0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.186+0000 7f6008b94640 1 --2- 172.21.15.60:0/3713475680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60040ceef0 0x7f6004148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.186+0000 7f6008b94640 1 -- 172.21.15.60:0/3713475680 >> 172.21.15.60:0/3713475680 conn(0x7f6004072340 msgr2=0x7f600415f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.186+0000 7f6008b94640 1 -- 172.21.15.60:0/3713475680 shutdown_connections 2024-04-25T10:16:20.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.186+0000 7f6008b94640 1 -- 172.21.15.60:0/3713475680 wait complete. 2024-04-25T10:16:20.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:16:20.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.330+0000 7f4706321640 1 Processor -- start 2024-04-25T10:16:20.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.330+0000 7f4706321640 1 -- start start 2024-04-25T10:16:20.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.330+0000 7f4706321640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f470015dd80 0x7f47000b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.330+0000 7f4706321640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47000b2860 0x7f47000b9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.330+0000 7f4706321640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f470015e160 con 0x7f47000b5470 2024-04-25T10:16:20.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.330+0000 7f4706321640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47000b7060 con 0x7f470015dd80 2024-04-25T10:16:20.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.330+0000 7f4706321640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47000d1b20 con 0x7f47000b2860 2024-04-25T10:16:20.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.330+0000 7f46fffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f470015dd80 0x7f47000b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.330+0000 7f46fffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f470015dd80 0x7f47000b6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60334/0 (socket says 172.21.15.60:60334) 2024-04-25T10:16:20.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.330+0000 7f46fffff640 1 -- 172.21.15.60:0/3246056730 learned_addr learned my addr 172.21.15.60:0/3246056730 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:20.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.330+0000 7f46ff7fe640 1 --2- 172.21.15.60:0/3246056730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47000b2860 0x7f47000b9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.330+0000 7f46fffff640 1 -- 172.21.15.60:0/3246056730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47000b2860 msgr2=0x7f47000b9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.330+0000 7f46fffff640 1 --2- 172.21.15.60:0/3246056730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47000b2860 0x7f47000b9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.330+0000 7f46fffff640 1 -- 172.21.15.60:0/3246056730 >> v1:172.21.15.100:6789/0 conn(0x7f47000b5470 legacy=0x7f47000d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.330+0000 7f46fffff640 1 -- 172.21.15.60:0/3246056730 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47000d1e00 con 0x7f470015dd80 2024-04-25T10:16:20.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.334+0000 7f46fffff640 1 --2- 172.21.15.60:0/3246056730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f470015dd80 0x7f47000b6b20 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009870 tx=0x7f46f001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571351100e11fe8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.334+0000 7f46feffd640 1 -- 172.21.15.60:0/3246056730 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f00a95f0 con 0x7f470015dd80 2024-04-25T10:16:20.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.334+0000 7f46feffd640 1 -- 172.21.15.60:0/3246056730 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f00a9790 con 0x7f470015dd80 2024-04-25T10:16:20.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.334+0000 7f46feffd640 1 -- 172.21.15.60:0/3246056730 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f00a9b60 con 0x7f470015dd80 2024-04-25T10:16:20.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.334+0000 7f4706321640 1 -- 172.21.15.60:0/3246056730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f470015dd80 msgr2=0x7f47000b6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.334+0000 7f4706321640 1 --2- 172.21.15.60:0/3246056730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f470015dd80 0x7f47000b6b20 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009870 tx=0x7f46f001c530 comp rx=0 tx=0).stop 2024-04-25T10:16:20.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.334+0000 7f4706321640 1 -- 172.21.15.60:0/3246056730 shutdown_connections 2024-04-25T10:16:20.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.334+0000 7f4706321640 1 --2- 172.21.15.60:0/3246056730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47000b2860 0x7f47000b9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.334+0000 7f4706321640 1 --2- 172.21.15.60:0/3246056730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f470015dd80 0x7f47000b6b20 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.334+0000 7f4706321640 1 -- 172.21.15.60:0/3246056730 >> 172.21.15.60:0/3246056730 conn(0x7f47000d0470 msgr2=0x7f47000d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.334+0000 7f4706321640 1 -- 172.21.15.60:0/3246056730 shutdown_connections 2024-04-25T10:16:20.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.334+0000 7f4706321640 1 -- 172.21.15.60:0/3246056730 wait complete. 2024-04-25T10:16:20.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.334+0000 7f4706321640 1 Processor -- start 2024-04-25T10:16:20.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.334+0000 7f4706321640 1 -- start start 2024-04-25T10:16:20.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f4706321640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47000b2860 0x7f47000c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f4706321640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47000b5470 0x7f47000c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f4706321640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f470015dd80 0x7f47001568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f4706321640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47000b5c90 con 0x7f470015dd80 2024-04-25T10:16:20.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f4706321640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47000d36e0 con 0x7f47000b2860 2024-04-25T10:16:20.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f4706321640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47000c7a60 con 0x7f47000b5470 2024-04-25T10:16:20.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f47048be640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f470015dd80 0x7f47001568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f47048be640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f470015dd80 0x7f47001568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60350/0 (socket says 172.21.15.60:60350) 2024-04-25T10:16:20.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f47048be640 1 -- 172.21.15.60:0/2245771249 learned_addr learned my addr 172.21.15.60:0/2245771249 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:20.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f46fffff640 1 --2- 172.21.15.60:0/2245771249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47000b2860 0x7f47000c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f46ff7fe640 1 --2- 172.21.15.60:0/2245771249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47000b5470 0x7f47000c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f47048be640 1 -- 172.21.15.60:0/2245771249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47000b5470 msgr2=0x7f47000c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f47048be640 1 --2- 172.21.15.60:0/2245771249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47000b5470 0x7f47000c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f47048be640 1 -- 172.21.15.60:0/2245771249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47000b2860 msgr2=0x7f47000c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f47048be640 1 --2- 172.21.15.60:0/2245771249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47000b2860 0x7f47000c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f47048be640 1 -- 172.21.15.60:0/2245771249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47000c7c90 con 0x7f470015dd80 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f47048be640 1 --2- 172.21.15.60:0/2245771249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f470015dd80 0x7f47001568a0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f46f800c710 tx=0x7f46f800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f46fcff9640 1 -- 172.21.15.60:0/2245771249 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8016020 con 0x7f470015dd80 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f46fcff9640 1 -- 172.21.15.60:0/2245771249 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f8006d60 con 0x7f470015dd80 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f4706321640 1 -- 172.21.15.60:0/2245771249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47000c7f70 con 0x7f470015dd80 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f46fcff9640 1 -- 172.21.15.60:0/2245771249 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8003bc0 con 0x7f470015dd80 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f4706321640 1 -- 172.21.15.60:0/2245771249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47000c8530 con 0x7f470015dd80 2024-04-25T10:16:20.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f4706321640 1 -- 172.21.15.60:0/2245771249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4700074770 con 0x7f470015dd80 2024-04-25T10:16:20.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f46fcff9640 1 -- 172.21.15.60:0/2245771249 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46f800a850 con 0x7f470015dd80 2024-04-25T10:16:20.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f46fcff9640 1 --2- 172.21.15.60:0/2245771249 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46ec066f60 0x7f46ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f46fcff9640 1 -- 172.21.15.60:0/2245771249 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46f8084d60 con 0x7f470015dd80 2024-04-25T10:16:20.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.338+0000 7f46fffff640 1 --2- 172.21.15.60:0/2245771249 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46ec066f60 0x7f46ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.342+0000 7f46fffff640 1 --2- 172.21.15.60:0/2245771249 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46ec066f60 0x7f46ec069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f46f00ae000 tx=0x7f46f00b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.342+0000 7f46fcff9640 1 -- 172.21.15.60:0/2245771249 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46f8020020 con 0x7f470015dd80 2024-04-25T10:16:20.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.502+0000 7f4706321640 1 -- 172.21.15.60:0/2245771249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f47000cf520 con 0x7f470015dd80 2024-04-25T10:16:20.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.502+0000 7f46fcff9640 1 -- 172.21.15.60:0/2245771249 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f46f8055d60 con 0x7f470015dd80 2024-04-25T10:16:20.508 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:16:20.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.506+0000 7f4706321640 1 -- 172.21.15.60:0/2245771249 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46ec066f60 msgr2=0x7f46ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.506+0000 7f4706321640 1 --2- 172.21.15.60:0/2245771249 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46ec066f60 0x7f46ec069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f46f00ae000 tx=0x7f46f00b3040 comp rx=0 tx=0).stop 2024-04-25T10:16:20.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.506+0000 7f4706321640 1 -- 172.21.15.60:0/2245771249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f470015dd80 msgr2=0x7f47001568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.506+0000 7f4706321640 1 --2- 172.21.15.60:0/2245771249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f470015dd80 0x7f47001568a0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f46f800c710 tx=0x7f46f800cbd0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.506+0000 7f4706321640 1 -- 172.21.15.60:0/2245771249 shutdown_connections 2024-04-25T10:16:20.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.506+0000 7f4706321640 1 --2- 172.21.15.60:0/2245771249 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46ec066f60 0x7f46ec069420 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.506+0000 7f4706321640 1 --2- 172.21.15.60:0/2245771249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f470015dd80 0x7f47001568a0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.506+0000 7f4706321640 1 --2- 172.21.15.60:0/2245771249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47000b5470 0x7f47000c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.506+0000 7f4706321640 1 --2- 172.21.15.60:0/2245771249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47000b2860 0x7f47000c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.506+0000 7f4706321640 1 -- 172.21.15.60:0/2245771249 >> 172.21.15.60:0/2245771249 conn(0x7f47000d0470 msgr2=0x7f4700159e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.506+0000 7f4706321640 1 -- 172.21.15.60:0/2245771249 shutdown_connections 2024-04-25T10:16:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.506+0000 7f4706321640 1 -- 172.21.15.60:0/2245771249 wait complete. 2024-04-25T10:16:20.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:16:20.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.642+0000 7fb5cf23a640 1 Processor -- start 2024-04-25T10:16:20.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.642+0000 7fb5cf23a640 1 -- start start 2024-04-25T10:16:20.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.642+0000 7fb5cf23a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5c81620d0 0x7fb5c80ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.642+0000 7fb5cf23a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb5c80ced20 0x7fb5c80d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.642+0000 7fb5cf23a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c815bad0 con 0x7fb5c815b6f0 2024-04-25T10:16:20.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.642+0000 7fb5cf23a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c8161c40 con 0x7fb5c80ced20 2024-04-25T10:16:20.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.642+0000 7fb5cf23a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c815ebe0 con 0x7fb5c81620d0 2024-04-25T10:16:20.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.642+0000 7fb5cd7d7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb5c80ced20 0x7fb5c80d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.642+0000 7fb5cd7d7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb5c80ced20 0x7fb5c80d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60354/0 (socket says 172.21.15.60:60354) 2024-04-25T10:16:20.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.642+0000 7fb5cd7d7640 1 -- 172.21.15.60:0/434132187 learned_addr learned my addr 172.21.15.60:0/434132187 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:20.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.642+0000 7fb5c7fff640 1 --2- 172.21.15.60:0/434132187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5c81620d0 0x7fb5c80ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.642+0000 7fb5cd7d7640 1 -- 172.21.15.60:0/434132187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5c81620d0 msgr2=0x7fb5c80ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.642+0000 7fb5cd7d7640 1 --2- 172.21.15.60:0/434132187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5c81620d0 0x7fb5c80ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.642+0000 7fb5cd7d7640 1 -- 172.21.15.60:0/434132187 >> v1:172.21.15.100:6789/0 conn(0x7fb5c815b6f0 legacy=0x7fb5c815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.642+0000 7fb5cd7d7640 1 -- 172.21.15.60:0/434132187 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c815eec0 con 0x7fb5c80ced20 2024-04-25T10:16:20.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.642+0000 7fb5cd7d7640 1 --2- 172.21.15.60:0/434132187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c80ced20 0x7fb5c80d1b30 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009e50 tx=0x7fb5b801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b73a5629d28f70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5c77fe640 1 -- 172.21.15.60:0/434132187 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b80a5630 con 0x7fb5c80ced20 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5c77fe640 1 -- 172.21.15.60:0/434132187 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b80a57d0 con 0x7fb5c80ced20 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5c77fe640 1 -- 172.21.15.60:0/434132187 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b80a5ba0 con 0x7fb5c80ced20 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 -- 172.21.15.60:0/434132187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c80ced20 msgr2=0x7fb5c80d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 --2- 172.21.15.60:0/434132187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c80ced20 0x7fb5c80d1b30 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009e50 tx=0x7fb5b801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 -- 172.21.15.60:0/434132187 shutdown_connections 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 --2- 172.21.15.60:0/434132187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c80ced20 0x7fb5c80d1b30 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 --2- 172.21.15.60:0/434132187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5c81620d0 0x7fb5c80ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 -- 172.21.15.60:0/434132187 >> 172.21.15.60:0/434132187 conn(0x7fb5c8072340 msgr2=0x7fb5c8072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 -- 172.21.15.60:0/434132187 shutdown_connections 2024-04-25T10:16:20.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 -- 172.21.15.60:0/434132187 wait complete. 2024-04-25T10:16:20.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 Processor -- start 2024-04-25T10:16:20.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 -- start start 2024-04-25T10:16:20.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c80ced20 0x7fb5c80c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5c815b6f0 0x7fb5c815a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5c81620d0 0x7fb5c814b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c815e960 con 0x7fb5c80ced20 2024-04-25T10:16:20.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c815e7e0 con 0x7fb5c815b6f0 2024-04-25T10:16:20.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c816aa60 con 0x7fb5c81620d0 2024-04-25T10:16:20.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5ccfd6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c80ced20 0x7fb5c80c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5ccfd6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c80ced20 0x7fb5c80c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60364/0 (socket says 172.21.15.60:60364) 2024-04-25T10:16:20.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5ccfd6640 1 -- 172.21.15.60:0/2743977501 learned_addr learned my addr 172.21.15.60:0/2743977501 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:20.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5c7fff640 1 --2- 172.21.15.60:0/2743977501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5c815b6f0 0x7fb5c815a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cd7d7640 1 --2- 172.21.15.60:0/2743977501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5c81620d0 0x7fb5c814b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5ccfd6640 1 -- 172.21.15.60:0/2743977501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5c81620d0 msgr2=0x7fb5c814b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5ccfd6640 1 --2- 172.21.15.60:0/2743977501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5c81620d0 0x7fb5c814b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5ccfd6640 1 -- 172.21.15.60:0/2743977501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5c815b6f0 msgr2=0x7fb5c815a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5ccfd6640 1 --2- 172.21.15.60:0/2743977501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5c815b6f0 0x7fb5c815a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5ccfd6640 1 -- 172.21.15.60:0/2743977501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c80b6800 con 0x7fb5c80ced20 2024-04-25T10:16:20.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5ccfd6640 1 --2- 172.21.15.60:0/2743977501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c80ced20 0x7fb5c80c65e0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000c480 tx=0x7fb5c000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5c57fa640 1 -- 172.21.15.60:0/2743977501 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0012830 con 0x7fb5c80ced20 2024-04-25T10:16:20.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5c57fa640 1 -- 172.21.15.60:0/2743977501 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c0017070 con 0x7fb5c80ced20 2024-04-25T10:16:20.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 -- 172.21.15.60:0/2743977501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c80c6b20 con 0x7fb5c80ced20 2024-04-25T10:16:20.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5cf23a640 1 -- 172.21.15.60:0/2743977501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c80c79c0 con 0x7fb5c80ced20 2024-04-25T10:16:20.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.646+0000 7fb5c57fa640 1 -- 172.21.15.60:0/2743977501 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0012c30 con 0x7fb5c80ced20 2024-04-25T10:16:20.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.650+0000 7fb5cf23a640 1 -- 172.21.15.60:0/2743977501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c8074770 con 0x7fb5c80ced20 2024-04-25T10:16:20.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.650+0000 7fb5c57fa640 1 -- 172.21.15.60:0/2743977501 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5c001c460 con 0x7fb5c80ced20 2024-04-25T10:16:20.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.650+0000 7fb5c57fa640 1 --2- 172.21.15.60:0/2743977501 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb5a8066f90 0x7fb5a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.650+0000 7fb5c57fa640 1 -- 172.21.15.60:0/2743977501 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5c0084df0 con 0x7fb5c80ced20 2024-04-25T10:16:20.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.650+0000 7fb5c7fff640 1 --2- 172.21.15.60:0/2743977501 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb5a8066f90 0x7fb5a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.650+0000 7fb5c7fff640 1 --2- 172.21.15.60:0/2743977501 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb5a8066f90 0x7fb5a8069450 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80d3900 tx=0x7fb5b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.654+0000 7fb5c57fa640 1 -- 172.21.15.60:0/2743977501 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb5c0055df0 con 0x7fb5c80ced20 2024-04-25T10:16:20.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.806+0000 7fb5cf23a640 1 -- 172.21.15.60:0/2743977501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb5c80d16e0 con 0x7fb5c80ced20 2024-04-25T10:16:20.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.806+0000 7fb5c57fa640 1 -- 172.21.15.60:0/2743977501 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb5c00556f0 con 0x7fb5c80ced20 2024-04-25T10:16:20.813 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:16:20.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.810+0000 7fb5cf23a640 1 -- 172.21.15.60:0/2743977501 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb5a8066f90 msgr2=0x7fb5a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.810+0000 7fb5cf23a640 1 --2- 172.21.15.60:0/2743977501 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb5a8066f90 0x7fb5a8069450 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80d3900 tx=0x7fb5b0008040 comp rx=0 tx=0).stop 2024-04-25T10:16:20.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.810+0000 7fb5cf23a640 1 -- 172.21.15.60:0/2743977501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c80ced20 msgr2=0x7fb5c80c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.810+0000 7fb5cf23a640 1 --2- 172.21.15.60:0/2743977501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c80ced20 0x7fb5c80c65e0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000c480 tx=0x7fb5c000c940 comp rx=0 tx=0).stop 2024-04-25T10:16:20.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.810+0000 7fb5cf23a640 1 -- 172.21.15.60:0/2743977501 shutdown_connections 2024-04-25T10:16:20.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.810+0000 7fb5cf23a640 1 --2- 172.21.15.60:0/2743977501 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb5a8066f90 0x7fb5a8069450 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.810+0000 7fb5cf23a640 1 --2- 172.21.15.60:0/2743977501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5c81620d0 0x7fb5c814b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.810+0000 7fb5cf23a640 1 --2- 172.21.15.60:0/2743977501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5c815b6f0 0x7fb5c815a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.810+0000 7fb5cf23a640 1 --2- 172.21.15.60:0/2743977501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c80ced20 0x7fb5c80c65e0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.810+0000 7fb5cf23a640 1 -- 172.21.15.60:0/2743977501 >> 172.21.15.60:0/2743977501 conn(0x7fb5c8072340 msgr2=0x7fb5c815f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.810+0000 7fb5cf23a640 1 -- 172.21.15.60:0/2743977501 shutdown_connections 2024-04-25T10:16:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.810+0000 7fb5cf23a640 1 -- 172.21.15.60:0/2743977501 wait complete. 2024-04-25T10:16:20.831 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:16:20.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.950+0000 7eff415b1640 1 Processor -- start 2024-04-25T10:16:20.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.950+0000 7eff415b1640 1 -- start start 2024-04-25T10:16:20.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.950+0000 7eff415b1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7eff3c15bc90 0x7eff3c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.950+0000 7eff415b1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff3c071ca0 con 0x7eff3c0ce7a0 2024-04-25T10:16:20.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.950+0000 7eff415b1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff3c0cca40 con 0x7eff3c0b4660 2024-04-25T10:16:20.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.950+0000 7eff415b1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff3c0cc370 con 0x7eff3c15bc90 2024-04-25T10:16:20.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.950+0000 7eff3affd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7eff3c0ce7a0 0x7eff3c0ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35084/0 (socket says 172.21.15.60:35084) 2024-04-25T10:16:20.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.950+0000 7eff3affd640 1 -- 172.21.15.60:0/2365442696 learned_addr learned my addr 172.21.15.60:0/2365442696 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:20.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.950+0000 7eff3a7fc640 1 --2- 172.21.15.60:0/2365442696 >> v2:172.21.15.60:3300/0 conn(0x7eff3c15bc90 0x7eff3c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.950+0000 7eff39ffb640 1 -- 172.21.15.60:0/2365442696 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3602344419 0 0) 0x7eff3c071ca0 con 0x7eff3c0ce7a0 2024-04-25T10:16:20.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.950+0000 7eff39ffb640 1 -- 172.21.15.60:0/2365442696 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff28003530 con 0x7eff3c0ce7a0 2024-04-25T10:16:20.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.950+0000 7eff3a7fc640 1 -- 172.21.15.60:0/2365442696 >> v1:172.21.15.100:6789/0 conn(0x7eff3c0b4660 legacy=0x7eff3c15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.950+0000 7eff3a7fc640 1 -- 172.21.15.60:0/2365442696 >> v1:172.21.15.60:6789/0 conn(0x7eff3c0ce7a0 legacy=0x7eff3c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.950+0000 7eff3a7fc640 1 -- 172.21.15.60:0/2365442696 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3c0ccd20 con 0x7eff3c15bc90 2024-04-25T10:16:20.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.954+0000 7eff3a7fc640 1 --2- 172.21.15.60:0/2365442696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff3c15bc90 0x7eff3c15c220 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7eff240097c0 tx=0x7eff2401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc41ca9f95b84724 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.954+0000 7eff39ffb640 1 -- 172.21.15.60:0/2365442696 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff240b0070 con 0x7eff3c15bc90 2024-04-25T10:16:20.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.954+0000 7eff39ffb640 1 -- 172.21.15.60:0/2365442696 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff240ab600 con 0x7eff3c15bc90 2024-04-25T10:16:20.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.954+0000 7eff39ffb640 1 -- 172.21.15.60:0/2365442696 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff240ab9b0 con 0x7eff3c15bc90 2024-04-25T10:16:20.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.954+0000 7eff415b1640 1 -- 172.21.15.60:0/2365442696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff3c15bc90 msgr2=0x7eff3c15c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.954+0000 7eff415b1640 1 --2- 172.21.15.60:0/2365442696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff3c15bc90 0x7eff3c15c220 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7eff240097c0 tx=0x7eff2401c460 comp rx=0 tx=0).stop 2024-04-25T10:16:20.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.954+0000 7eff415b1640 1 -- 172.21.15.60:0/2365442696 shutdown_connections 2024-04-25T10:16:20.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.954+0000 7eff415b1640 1 --2- 172.21.15.60:0/2365442696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff3c15bc90 0x7eff3c15c220 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.954+0000 7eff415b1640 1 -- 172.21.15.60:0/2365442696 >> 172.21.15.60:0/2365442696 conn(0x7eff3c0b12d0 msgr2=0x7eff3c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.954+0000 7eff415b1640 1 -- 172.21.15.60:0/2365442696 shutdown_connections 2024-04-25T10:16:20.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.954+0000 7eff415b1640 1 -- 172.21.15.60:0/2365442696 wait complete. 2024-04-25T10:16:20.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.954+0000 7eff415b1640 1 Processor -- start 2024-04-25T10:16:20.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff415b1640 1 -- start start 2024-04-25T10:16:20.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff415b1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff3c0ce7a0 0x7eff3c150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff415b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff3c15bc90 0x7eff3c149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff415b1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff3c153860 0x7eff3c153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff415b1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c1695d0 con 0x7eff3c153860 2024-04-25T10:16:20.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff415b1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c0b3db0 con 0x7eff3c15bc90 2024-04-25T10:16:20.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff415b1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c169750 con 0x7eff3c0ce7a0 2024-04-25T10:16:20.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff3b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff3c153860 0x7eff3c153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff3b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff3c153860 0x7eff3c153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60376/0 (socket says 172.21.15.60:60376) 2024-04-25T10:16:20.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff3b7fe640 1 -- 172.21.15.60:0/1908930035 learned_addr learned my addr 172.21.15.60:0/1908930035 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:20.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff3a7fc640 1 --2- 172.21.15.60:0/1908930035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff3c15bc90 0x7eff3c149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff3affd640 1 --2- 172.21.15.60:0/1908930035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff3c0ce7a0 0x7eff3c150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff3b7fe640 1 -- 172.21.15.60:0/1908930035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff3c0ce7a0 msgr2=0x7eff3c150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff3b7fe640 1 --2- 172.21.15.60:0/1908930035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff3c0ce7a0 0x7eff3c150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff3b7fe640 1 -- 172.21.15.60:0/1908930035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff3c15bc90 msgr2=0x7eff3c149710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff3b7fe640 1 --2- 172.21.15.60:0/1908930035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff3c15bc90 0x7eff3c149710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff3b7fe640 1 -- 172.21.15.60:0/1908930035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3c14fd00 con 0x7eff3c153860 2024-04-25T10:16:20.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff3b7fe640 1 --2- 172.21.15.60:0/1908930035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff3c153860 0x7eff3c153cf0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7eff3400b9e0 tx=0x7eff3400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff337fe640 1 -- 172.21.15.60:0/1908930035 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34003b80 con 0x7eff3c153860 2024-04-25T10:16:20.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff337fe640 1 -- 172.21.15.60:0/1908930035 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff34003d20 con 0x7eff3c153860 2024-04-25T10:16:20.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff415b1640 1 -- 172.21.15.60:0/1908930035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3c149c50 con 0x7eff3c153860 2024-04-25T10:16:20.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff337fe640 1 -- 172.21.15.60:0/1908930035 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3400cb60 con 0x7eff3c153860 2024-04-25T10:16:20.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff415b1640 1 -- 172.21.15.60:0/1908930035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c150260 con 0x7eff3c153860 2024-04-25T10:16:20.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff415b1640 1 -- 172.21.15.60:0/1908930035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3c074770 con 0x7eff3c153860 2024-04-25T10:16:20.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff337fe640 1 -- 172.21.15.60:0/1908930035 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff34006d10 con 0x7eff3c153860 2024-04-25T10:16:20.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff337fe640 1 --2- 172.21.15.60:0/1908930035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7eff1c066f60 0x7eff1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.958+0000 7eff337fe640 1 -- 172.21.15.60:0/1908930035 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff34084c00 con 0x7eff3c153860 2024-04-25T10:16:20.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.962+0000 7eff3affd640 1 --2- 172.21.15.60:0/1908930035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7eff1c066f60 0x7eff1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.962+0000 7eff3affd640 1 --2- 172.21.15.60:0/1908930035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7eff1c066f60 0x7eff1c069420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7eff2c007fd0 tx=0x7eff2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:20.966+0000 7eff337fe640 1 -- 172.21.15.60:0/1908930035 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff3c149c50 con 0x7eff3c153860 2024-04-25T10:16:21.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.130+0000 7eff415b1640 1 -- 172.21.15.60:0/1908930035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7eff3c157030 con 0x7eff3c153860 2024-04-25T10:16:21.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.134+0000 7eff337fe640 1 -- 172.21.15.60:0/1908930035 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) v1 ==== 80+0+0 (secure 0 0 0) 0x7eff3c14fd00 con 0x7eff3c153860 2024-04-25T10:16:21.138 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:16:21.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.134+0000 7eff415b1640 1 -- 172.21.15.60:0/1908930035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7eff1c066f60 msgr2=0x7eff1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.134+0000 7eff415b1640 1 --2- 172.21.15.60:0/1908930035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7eff1c066f60 0x7eff1c069420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7eff2c007fd0 tx=0x7eff2c009040 comp rx=0 tx=0).stop 2024-04-25T10:16:21.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.134+0000 7eff415b1640 1 -- 172.21.15.60:0/1908930035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff3c153860 msgr2=0x7eff3c153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.134+0000 7eff415b1640 1 --2- 172.21.15.60:0/1908930035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff3c153860 0x7eff3c153cf0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7eff3400b9e0 tx=0x7eff3400bea0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.134+0000 7eff415b1640 1 -- 172.21.15.60:0/1908930035 shutdown_connections 2024-04-25T10:16:21.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.134+0000 7eff415b1640 1 --2- 172.21.15.60:0/1908930035 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7eff1c066f60 0x7eff1c069420 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.134+0000 7eff415b1640 1 --2- 172.21.15.60:0/1908930035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff3c153860 0x7eff3c153cf0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.134+0000 7eff415b1640 1 --2- 172.21.15.60:0/1908930035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff3c15bc90 0x7eff3c149710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.134+0000 7eff415b1640 1 --2- 172.21.15.60:0/1908930035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff3c0ce7a0 0x7eff3c150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.134+0000 7eff415b1640 1 -- 172.21.15.60:0/1908930035 >> 172.21.15.60:0/1908930035 conn(0x7eff3c0b12d0 msgr2=0x7eff3c0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:21.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.134+0000 7eff415b1640 1 -- 172.21.15.60:0/1908930035 shutdown_connections 2024-04-25T10:16:21.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.134+0000 7eff415b1640 1 -- 172.21.15.60:0/1908930035 wait complete. 2024-04-25T10:16:21.155 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:16:21.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.155 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:21.166 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[51660]: fuse finished with error 0 and tester_r 0 2024-04-25T10:16:21.167 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:21.253 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:16:21.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.254 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:16:21.258 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:16:21.258 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-25T10:16:21.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.406+0000 7fc1e6661640 1 Processor -- start 2024-04-25T10:16:21.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.406+0000 7fc1e6661640 1 -- start start 2024-04-25T10:16:21.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.410+0000 7fc1e6661640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc1e00b3f60 0x7fc1e015a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.410+0000 7fc1e6661640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1e0071250 0x7fc1e00716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.410+0000 7fc1e6661640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e00b3510 con 0x7fc1e006f030 2024-04-25T10:16:21.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.410+0000 7fc1e6661640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e015ad30 con 0x7fc1e00b3f60 2024-04-25T10:16:21.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.410+0000 7fc1e6661640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e015b010 con 0x7fc1e0071250 2024-04-25T10:16:21.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.410+0000 7fc1dffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc1e00b3f60 0x7fc1e015a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.410+0000 7fc1dffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc1e00b3f60 0x7fc1e015a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:50336/0 (socket says 172.21.15.60:50336) 2024-04-25T10:16:21.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.410+0000 7fc1dffff640 1 -- 172.21.15.60:0/1119871313 learned_addr learned my addr 172.21.15.60:0/1119871313 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:21.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.410+0000 7fc1e4bfe640 1 --2- 172.21.15.60:0/1119871313 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1e0071250 0x7fc1e00716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.410+0000 7fc1dffff640 1 -- 172.21.15.60:0/1119871313 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1e0071250 msgr2=0x7fc1e00716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:21.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.410+0000 7fc1dffff640 1 --2- 172.21.15.60:0/1119871313 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1e0071250 0x7fc1e00716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.410+0000 7fc1dffff640 1 -- 172.21.15.60:0/1119871313 >> v1:172.21.15.100:6789/0 conn(0x7fc1e006f030 legacy=0x7fc1e00b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.410+0000 7fc1dffff640 1 -- 172.21.15.60:0/1119871313 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e015b2f0 con 0x7fc1e00b3f60 2024-04-25T10:16:21.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.410+0000 7fc1dffff640 1 --2- 172.21.15.60:0/1119871313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1e00b3f60 0x7fc1e015a3e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0006fd0 tx=0x7fc1d001e960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d67438a976a6977 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:21.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1df7fe640 1 -- 172.21.15.60:0/1119871313 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00ad070 con 0x7fc1e00b3f60 2024-04-25T10:16:21.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1df7fe640 1 -- 172.21.15.60:0/1119871313 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d0004230 con 0x7fc1e00b3f60 2024-04-25T10:16:21.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1df7fe640 1 -- 172.21.15.60:0/1119871313 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0003620 con 0x7fc1e00b3f60 2024-04-25T10:16:21.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1e6661640 1 -- 172.21.15.60:0/1119871313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1e00b3f60 msgr2=0x7fc1e015a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1e6661640 1 --2- 172.21.15.60:0/1119871313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1e00b3f60 0x7fc1e015a3e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0006fd0 tx=0x7fc1d001e960 comp rx=0 tx=0).stop 2024-04-25T10:16:21.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1e6661640 1 -- 172.21.15.60:0/1119871313 shutdown_connections 2024-04-25T10:16:21.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1e6661640 1 --2- 172.21.15.60:0/1119871313 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1e0071250 0x7fc1e00716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1e6661640 1 --2- 172.21.15.60:0/1119871313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1e00b3f60 0x7fc1e015a3e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1e6661640 1 -- 172.21.15.60:0/1119871313 >> 172.21.15.60:0/1119871313 conn(0x7fc1e00670b0 msgr2=0x7fc1e006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:21.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1e6661640 1 -- 172.21.15.60:0/1119871313 shutdown_connections 2024-04-25T10:16:21.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1e6661640 1 -- 172.21.15.60:0/1119871313 wait complete. 2024-04-25T10:16:21.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1e6661640 1 Processor -- start 2024-04-25T10:16:21.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1e6661640 1 -- start start 2024-04-25T10:16:21.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1e6661640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1e006f030 0x7fc1e0142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1e6661640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1e0071250 0x7fc1e014c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1e6661640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1e00b3f60 0x7fc1e014b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1e6661640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e015b720 con 0x7fc1e0071250 2024-04-25T10:16:21.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1e6661640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e015b5a0 con 0x7fc1e00b3f60 2024-04-25T10:16:21.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1e6661640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e015b8a0 con 0x7fc1e006f030 2024-04-25T10:16:21.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1d77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1e0071250 0x7fc1e014c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1d77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1e0071250 0x7fc1e014c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60386/0 (socket says 172.21.15.60:60386) 2024-04-25T10:16:21.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.414+0000 7fc1d77fe640 1 -- 172.21.15.60:0/1546803881 learned_addr learned my addr 172.21.15.60:0/1546803881 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:21.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1dffff640 1 --2- 172.21.15.60:0/1546803881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1e006f030 0x7fc1e0142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1e4bfe640 1 --2- 172.21.15.60:0/1546803881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1e00b3f60 0x7fc1e014b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1d77fe640 1 -- 172.21.15.60:0/1546803881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1e006f030 msgr2=0x7fc1e0142fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1d77fe640 1 --2- 172.21.15.60:0/1546803881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1e006f030 0x7fc1e0142fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1d77fe640 1 -- 172.21.15.60:0/1546803881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1e00b3f60 msgr2=0x7fc1e014b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1d77fe640 1 --2- 172.21.15.60:0/1546803881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1e00b3f60 0x7fc1e014b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1d77fe640 1 -- 172.21.15.60:0/1546803881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e0149c00 con 0x7fc1e0071250 2024-04-25T10:16:21.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1e4bfe640 1 --2- 172.21.15.60:0/1546803881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1e00b3f60 0x7fc1e014b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:21.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1dffff640 1 --2- 172.21.15.60:0/1546803881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1e006f030 0x7fc1e0142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:21.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1d77fe640 1 --2- 172.21.15.60:0/1546803881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1e0071250 0x7fc1e014c830 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800c450 tx=0x7fc1c800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:21.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1dd7fa640 1 -- 172.21.15.60:0/1546803881 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8016020 con 0x7fc1e0071250 2024-04-25T10:16:21.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1dd7fa640 1 -- 172.21.15.60:0/1546803881 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c80052d0 con 0x7fc1e0071250 2024-04-25T10:16:21.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1e6661640 1 -- 172.21.15.60:0/1546803881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e0149ee0 con 0x7fc1e0071250 2024-04-25T10:16:21.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1dd7fa640 1 -- 172.21.15.60:0/1546803881 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8007260 con 0x7fc1e0071250 2024-04-25T10:16:21.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1e6661640 1 -- 172.21.15.60:0/1546803881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e0143780 con 0x7fc1e0071250 2024-04-25T10:16:21.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1dd7fa640 1 -- 172.21.15.60:0/1546803881 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc1c8007400 con 0x7fc1e0071250 2024-04-25T10:16:21.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1dd7fa640 1 --2- 172.21.15.60:0/1546803881 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc1c0066f60 0x7fc1c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1dd7fa640 1 -- 172.21.15.60:0/1546803881 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1c8084a20 con 0x7fc1e0071250 2024-04-25T10:16:21.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1dffff640 1 --2- 172.21.15.60:0/1546803881 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc1c0066f60 0x7fc1c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1e97ce000 1 Processor -- start 2024-04-25T10:16:21.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.418+0000 7fc1e97ce000 1 -- start start 2024-04-25T10:16:21.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 --2- 172.21.15.60:0/1546803881 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc1c0066f60 0x7fc1c0069420 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00b1000 tx=0x7fc1d00a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9f79d0 0x56554a9f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f82f0 0x56554a9adb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9ae0a0 0x56554a9ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554a7faff0 con 0x56554a9f79d0 2024-04-25T10:16:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554a783150 con 0x56554a9f82f0 2024-04-25T10:16:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56554a7f7090 con 0x56554a9ae0a0 2024-04-25T10:16:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9f79d0 0x56554a9f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9f79d0 0x56554a9f7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60388/0 (socket says 172.21.15.60:60388) 2024-04-25T10:16:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 -- 172.21.15.60:0/3172609328 learned_addr learned my addr 172.21.15.60:0/3172609328 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e4bfe640 1 --2- 172.21.15.60:0/3172609328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9ae0a0 0x56554a9ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1d77fe640 1 --2- 172.21.15.60:0/3172609328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f82f0 0x56554a9adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 -- 172.21.15.60:0/3172609328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9ae0a0 msgr2=0x56554a9ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 --2- 172.21.15.60:0/3172609328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9ae0a0 0x56554a9ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 -- 172.21.15.60:0/3172609328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f82f0 msgr2=0x56554a9adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 --2- 172.21.15.60:0/3172609328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f82f0 0x56554a9adb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 -- 172.21.15.60:0/3172609328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56554a9910d0 con 0x56554a9f79d0 2024-04-25T10:16:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 --2- 172.21.15.60:0/3172609328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9f79d0 0x56554a9f7db0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fc1d009a350 tx=0x7fc1d0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683230444d3672c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1d5ffb640 1 -- 172.21.15.60:0/3172609328 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00ad040 con 0x56554a9f79d0 2024-04-25T10:16:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1d5ffb640 1 -- 172.21.15.60:0/3172609328 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d0004390 con 0x56554a9f79d0 2024-04-25T10:16:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1d5ffb640 1 -- 172.21.15.60:0/3172609328 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0003af0 con 0x56554a9f79d0 2024-04-25T10:16:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 -- 172.21.15.60:0/3172609328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9f79d0 msgr2=0x56554a9f7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/3172609328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9f79d0 0x56554a9f7db0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fc1d009a350 tx=0x7fc1d0004740 comp rx=0 tx=0).stop 2024-04-25T10:16:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 -- 172.21.15.60:0/3172609328 shutdown_connections 2024-04-25T10:16:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/3172609328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9ae0a0 0x56554a9ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/3172609328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f82f0 0x56554a9adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/3172609328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9f79d0 0x56554a9f7db0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 -- 172.21.15.60:0/3172609328 >> 172.21.15.60:0/3172609328 conn(0x56554a9aa020 msgr2=0x56554a9ac440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 -- 172.21.15.60:0/3172609328 shutdown_connections 2024-04-25T10:16:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 -- 172.21.15.60:0/3172609328 wait complete. 2024-04-25T10:16:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 -- start start 2024-04-25T10:16:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 Processor -- start 2024-04-25T10:16:21.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9ae0a0 0x56554a9dc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f79d0 0x56554a9dcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9f82f0 0x56554a9dd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554a7bf250 con 0x56554a9ae0a0 2024-04-25T10:16:21.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554a7b5690 con 0x56554a9f79d0 2024-04-25T10:16:21.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56554a783a40 con 0x56554a9f82f0 2024-04-25T10:16:21.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e97ce000 10 client.0 _ll_register_callbacks cb 0x56554a998e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9ae0a0 0x56554a9dc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9ae0a0 0x56554a9dc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60390/0 (socket says 172.21.15.60:60390) 2024-04-25T10:16:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 -- 172.21.15.60:0/414830707 learned_addr learned my addr 172.21.15.60:0/414830707 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1d77fe640 1 --2- 172.21.15.60:0/414830707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f79d0 0x56554a9dcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1e4bfe640 1 --2- 172.21.15.60:0/414830707 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9f82f0 0x56554a9dd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 -- 172.21.15.60:0/414830707 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9f82f0 msgr2=0x56554a9dd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 --2- 172.21.15.60:0/414830707 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9f82f0 0x56554a9dd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 -- 172.21.15.60:0/414830707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f79d0 msgr2=0x56554a9dcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 --2- 172.21.15.60:0/414830707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f79d0 0x56554a9dcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 -- 172.21.15.60:0/414830707 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56554a8a81c0 con 0x56554a9ae0a0 2024-04-25T10:16:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1d77fe640 1 --2- 172.21.15.60:0/414830707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f79d0 0x56554a9dcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1dffff640 1 --2- 172.21.15.60:0/414830707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9ae0a0 0x56554a9dc7f0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0000c10 tx=0x7fc1d001d1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4b0acae0395914 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1beffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:16:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.422+0000 7fc1beffd640 1 -- 172.21.15.60:0/414830707 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00ad040 con 0x56554a9ae0a0 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1beffd640 1 -- 172.21.15.60:0/414830707 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1d00c2040 con 0x56554a9ae0a0 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 1 -- 172.21.15.60:0/414830707 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1d0004390 con 0x56554a9ae0a0 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1beffd640 1 -- 172.21.15.60:0/414830707 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1d0003cb0 con 0x56554a9ae0a0 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 10 client.9594 fetch_fsmap learned FSMap version 93 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 1 -- 172.21.15.60:0/414830707 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x56554a8a4170 con 0x56554a9ae0a0 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1beffd640 1 -- 172.21.15.60:0/414830707 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d0004390 con 0x56554a9ae0a0 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1beffd640 1 -- 172.21.15.60:0/414830707 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 93) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc1d00ba050 con 0x56554a9ae0a0 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 10 client.9594 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 10 client.9594 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 20 client.9594 populate_metadata read hostname 'smithi060' 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/414830707 >> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] conn(0x56554aa07c80 0x56554aa0a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 4 client.9594 mds_command: new command op to 9381 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 1 -- 172.21.15.60:0/414830707 --> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56554a796c60 con 0x56554aa07c80 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1d77fe640 1 --2- 172.21.15.60:0/414830707 >> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] conn(0x56554aa07c80 0x56554aa0a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1d77fe640 1 --2- 172.21.15.60:0/414830707 >> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] conn(0x56554aa07c80 0x56554aa0a070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7b3e68ac8fd229 server_cookie=f71f04f34e83d882 in_seq=0 out_seq=0 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1beffd640 10 client.9594 ms_handle_connect on v2:172.21.15.60:6834/722171246 2024-04-25T10:16:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1beffd640 1 -- 172.21.15.60:0/414830707 <== mds.0 v2:172.21.15.60:6834/722171246 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56554a796c60 con 0x56554aa07c80 2024-04-25T10:16:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1beffd640 10 client.9594 handle_command_reply: tid=0 2024-04-25T10:16:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 1 client.9594 shutdown 2024-04-25T10:16:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 1 -- 172.21.15.60:0/414830707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9ae0a0 msgr2=0x56554a9dc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/414830707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9ae0a0 0x56554a9dc7f0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0000c10 tx=0x7fc1d001d1a0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 1 -- 172.21.15.60:0/414830707 shutdown_connections 2024-04-25T10:16:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/414830707 >> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] conn(0x56554aa07c80 0x56554aa0a070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/414830707 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9f82f0 0x56554a9dd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/414830707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f79d0 0x56554a9dcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1beffd640 0 client.9594 ms_handle_reset on v2:172.21.15.60:6834/722171246 2024-04-25T10:16:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/414830707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9ae0a0 0x56554a9dc7f0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 1 -- 172.21.15.60:0/414830707 >> 172.21.15.60:0/414830707 conn(0x56554a9aa020 msgr2=0x56554a9acca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 1 -- 172.21.15.60:0/414830707 shutdown_connections 2024-04-25T10:16:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 1 -- 172.21.15.60:0/414830707 wait complete. 2024-04-25T10:16:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.426+0000 7fc1e97ce000 20 client.9594 trim_cache size 0 max 16384 2024-04-25T10:16:21.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1e97ce000 1 Processor -- start 2024-04-25T10:16:21.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1e97ce000 1 -- start start 2024-04-25T10:16:21.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1e97ce000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9ae0a0 0x56554aa540f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1e97ce000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f79d0 0x56554aa54630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1e97ce000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9f82f0 0x56554aa54b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1e97ce000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e01fe800 con 0x56554a9ae0a0 2024-04-25T10:16:21.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1e97ce000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e01fdb00 con 0x56554a9f79d0 2024-04-25T10:16:21.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1e97ce000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e0153270 con 0x56554a9f82f0 2024-04-25T10:16:21.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9ae0a0 0x56554aa540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9ae0a0 0x56554aa540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60396/0 (socket says 172.21.15.60:60396) 2024-04-25T10:16:21.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1dffff640 1 -- 172.21.15.60:0/994446437 learned_addr learned my addr 172.21.15.60:0/994446437 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:21.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1dffff640 1 -- 172.21.15.60:0/994446437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9f82f0 msgr2=0x56554aa54b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:21.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1dffff640 1 --2- 172.21.15.60:0/994446437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9f82f0 0x56554aa54b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1dffff640 1 -- 172.21.15.60:0/994446437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f79d0 msgr2=0x56554aa54630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:21.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1dffff640 1 --2- 172.21.15.60:0/994446437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f79d0 0x56554aa54630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1dffff640 1 -- 172.21.15.60:0/994446437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56554a841de0 con 0x56554a9ae0a0 2024-04-25T10:16:21.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.438+0000 7fc1dffff640 1 --2- 172.21.15.60:0/994446437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9ae0a0 0x56554aa540f0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00a7920 tx=0x7fc1d00bf000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81705baf69675de6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:21.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1d57fa640 1 -- 172.21.15.60:0/994446437 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00a7680 con 0x56554a9ae0a0 2024-04-25T10:16:21.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1d57fa640 1 -- 172.21.15.60:0/994446437 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d0004390 con 0x56554a9ae0a0 2024-04-25T10:16:21.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1d57fa640 1 -- 172.21.15.60:0/994446437 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00a7ba0 con 0x56554a9ae0a0 2024-04-25T10:16:21.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 -- 172.21.15.60:0/994446437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9ae0a0 msgr2=0x56554aa540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/994446437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9ae0a0 0x56554aa540f0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00a7920 tx=0x7fc1d00bf000 comp rx=0 tx=0).stop 2024-04-25T10:16:21.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 -- 172.21.15.60:0/994446437 shutdown_connections 2024-04-25T10:16:21.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/994446437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9f82f0 0x56554aa54b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/994446437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f79d0 0x56554aa54630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/994446437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9ae0a0 0x56554aa540f0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 -- 172.21.15.60:0/994446437 >> 172.21.15.60:0/994446437 conn(0x56554a9aa020 msgr2=0x56554aa04fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:21.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 -- 172.21.15.60:0/994446437 shutdown_connections 2024-04-25T10:16:21.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 -- 172.21.15.60:0/994446437 wait complete. 2024-04-25T10:16:21.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 -- start start 2024-04-25T10:16:21.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 Processor -- start 2024-04-25T10:16:21.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9ae0a0 0x56554aa0a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f79d0 0x56554aa0a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9f82f0 0x56554aa0ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554a795e00 con 0x56554a9f82f0 2024-04-25T10:16:21.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e0153870 con 0x56554a9f79d0 2024-04-25T10:16:21.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56554a7fa300 con 0x56554a9ae0a0 2024-04-25T10:16:21.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 10 client.0 _ll_register_callbacks cb 0x56554a979040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:21.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e4bfe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9f82f0 0x56554aa0ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e4bfe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9f82f0 0x56554aa0ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60410/0 (socket says 172.21.15.60:60410) 2024-04-25T10:16:21.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e4bfe640 1 -- 172.21.15.60:0/1432784799 learned_addr learned my addr 172.21.15.60:0/1432784799 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:21.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1d77fe640 1 --2- 172.21.15.60:0/1432784799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f79d0 0x56554aa0a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1dffff640 1 --2- 172.21.15.60:0/1432784799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9ae0a0 0x56554aa0a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e4bfe640 1 -- 172.21.15.60:0/1432784799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9ae0a0 msgr2=0x56554aa0a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:21.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e4bfe640 1 --2- 172.21.15.60:0/1432784799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9ae0a0 0x56554aa0a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e4bfe640 1 -- 172.21.15.60:0/1432784799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f79d0 msgr2=0x56554aa0a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:21.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e4bfe640 1 --2- 172.21.15.60:0/1432784799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f79d0 0x56554aa0a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e4bfe640 1 -- 172.21.15.60:0/1432784799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56554a79c8e0 con 0x56554a9f82f0 2024-04-25T10:16:21.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e4bfe640 1 --2- 172.21.15.60:0/1432784799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9f82f0 0x56554aa0ae00 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8007bd0 tx=0x7fc1d8014630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab370689a642fd0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:21.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1beffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:16:21.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1beffd640 1 -- 172.21.15.60:0/1432784799 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d801c070 con 0x56554a9f82f0 2024-04-25T10:16:21.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1beffd640 1 -- 172.21.15.60:0/1432784799 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1d8018460 con 0x56554a9f82f0 2024-04-25T10:16:21.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 -- 172.21.15.60:0/1432784799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1d0004390 con 0x56554a9f82f0 2024-04-25T10:16:21.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1beffd640 1 -- 172.21.15.60:0/1432784799 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1d8027050 con 0x56554a9f82f0 2024-04-25T10:16:21.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 10 client.9600 fetch_fsmap learned FSMap version 93 2024-04-25T10:16:21.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 -- 172.21.15.60:0/1432784799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x56554a8548f0 con 0x56554a9f82f0 2024-04-25T10:16:21.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1beffd640 1 -- 172.21.15.60:0/1432784799 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d0004390 con 0x56554a9f82f0 2024-04-25T10:16:21.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1beffd640 1 -- 172.21.15.60:0/1432784799 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 93) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc1d802a020 con 0x56554a9f82f0 2024-04-25T10:16:21.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 10 client.9600 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:16:21.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 10 client.9600 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:16:21.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 20 client.9600 populate_metadata read hostname 'smithi060' 2024-04-25T10:16:21.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/1432784799 >> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] conn(0x56554aa0bab0 0x56554aa5ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 4 client.9600 mds_command: new command op to 9381 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:16:21.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1e97ce000 1 -- 172.21.15.60:0/1432784799 --> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56554a808530 con 0x56554aa0bab0 2024-04-25T10:16:21.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1dffff640 1 --2- 172.21.15.60:0/1432784799 >> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] conn(0x56554aa0bab0 0x56554aa5ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1dffff640 1 --2- 172.21.15.60:0/1432784799 >> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] conn(0x56554aa0bab0 0x56554aa5ff60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4412a7bb397260e server_cookie=395b62d958009d81 in_seq=0 out_seq=0 2024-04-25T10:16:21.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.442+0000 7fc1beffd640 10 client.9600 ms_handle_connect on v2:172.21.15.60:6834/722171246 2024-04-25T10:16:21.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1beffd640 1 -- 172.21.15.60:0/1432784799 <== mds.0 v2:172.21.15.60:6834/722171246 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56554a808530 con 0x56554aa0bab0 2024-04-25T10:16:21.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1beffd640 10 client.9600 handle_command_reply: tid=0 2024-04-25T10:16:21.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e97ce000 1 client.9600 shutdown 2024-04-25T10:16:21.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e97ce000 1 -- 172.21.15.60:0/1432784799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9f82f0 msgr2=0x56554aa0ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/1432784799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9f82f0 0x56554aa0ae00 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8007bd0 tx=0x7fc1d8014630 comp rx=0 tx=0).stop 2024-04-25T10:16:21.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e97ce000 1 -- 172.21.15.60:0/1432784799 shutdown_connections 2024-04-25T10:16:21.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/1432784799 >> [v2:172.21.15.60:6834/722171246,v1:172.21.15.60:6835/722171246] conn(0x56554aa0bab0 0x56554aa5ff60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/1432784799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56554a9f82f0 0x56554aa0ae00 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/1432784799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56554a9f79d0 0x56554aa0a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1beffd640 0 client.9600 ms_handle_reset on v2:172.21.15.60:6834/722171246 2024-04-25T10:16:21.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e97ce000 1 --2- 172.21.15.60:0/1432784799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56554a9ae0a0 0x56554aa0a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e97ce000 1 -- 172.21.15.60:0/1432784799 >> 172.21.15.60:0/1432784799 conn(0x56554a9aa020 msgr2=0x56554a9f60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:21.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e97ce000 1 -- 172.21.15.60:0/1432784799 shutdown_connections 2024-04-25T10:16:21.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e97ce000 1 -- 172.21.15.60:0/1432784799 wait complete. 2024-04-25T10:16:21.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e97ce000 20 client.9600 trim_cache size 0 max 16384 2024-04-25T10:16:21.462 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:16:21.462 INFO:teuthology.orchestra.run.smithi060.stdout: "message": "", 2024-04-25T10:16:21.462 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T10:16:21.462 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:16:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e6661640 1 -- 172.21.15.60:0/1546803881 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc1c0066f60 msgr2=0x7fc1c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e6661640 1 --2- 172.21.15.60:0/1546803881 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc1c0066f60 0x7fc1c0069420 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00b1000 tx=0x7fc1d00a9070 comp rx=0 tx=0).stop 2024-04-25T10:16:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e6661640 1 -- 172.21.15.60:0/1546803881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1e0071250 msgr2=0x7fc1e014c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e6661640 1 --2- 172.21.15.60:0/1546803881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1e0071250 0x7fc1e014c830 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800c450 tx=0x7fc1c800c910 comp rx=0 tx=0).stop 2024-04-25T10:16:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e6661640 1 -- 172.21.15.60:0/1546803881 shutdown_connections 2024-04-25T10:16:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e6661640 1 --2- 172.21.15.60:0/1546803881 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc1c0066f60 0x7fc1c0069420 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e6661640 1 --2- 172.21.15.60:0/1546803881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1e00b3f60 0x7fc1e014b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e6661640 1 --2- 172.21.15.60:0/1546803881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1e0071250 0x7fc1e014c830 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e6661640 1 --2- 172.21.15.60:0/1546803881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1e006f030 0x7fc1e0142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e6661640 1 -- 172.21.15.60:0/1546803881 >> 172.21.15.60:0/1546803881 conn(0x7fc1e00670b0 msgr2=0x7fc1e00ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e6661640 1 -- 172.21.15.60:0/1546803881 shutdown_connections 2024-04-25T10:16:21.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.454+0000 7fc1e6661640 1 -- 172.21.15.60:0/1546803881 wait complete. 2024-04-25T10:16:21.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:16:21.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.582+0000 7fef20326640 1 Processor -- start 2024-04-25T10:16:21.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.582+0000 7fef20326640 1 -- start start 2024-04-25T10:16:21.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef20326640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fef1806ce10 0x7fef18099a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef20326640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef180d03b0 con 0x7fef18099f70 2024-04-25T10:16:21.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef20326640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef180d0690 con 0x7fef180cdc30 2024-04-25T10:16:21.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef20326640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef1809a3c0 con 0x7fef1806ce10 2024-04-25T10:16:21.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef1d8c1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fef1806ce10 0x7fef18099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef1e8c3640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fef18099f70 0x7fef180cbe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35088/0 (socket says 172.21.15.60:35088) 2024-04-25T10:16:21.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef1e8c3640 1 -- 172.21.15.60:0/339260784 learned_addr learned my addr 172.21.15.60:0/339260784 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:21.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef1d8c1640 1 -- 172.21.15.60:0/339260784 >> v1:172.21.15.100:6789/0 conn(0x7fef180cdc30 legacy=0x7fef180cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:21.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef1d8c1640 1 -- 172.21.15.60:0/339260784 >> v1:172.21.15.60:6789/0 conn(0x7fef18099f70 legacy=0x7fef180cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef1d0c0640 1 -- 172.21.15.60:0/339260784 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149515207 0 0) 0x7fef180d03b0 con 0x7fef18099f70 2024-04-25T10:16:21.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef1d8c1640 1 -- 172.21.15.60:0/339260784 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef180d0970 con 0x7fef1806ce10 2024-04-25T10:16:21.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef1d8c1640 1 --2- 172.21.15.60:0/339260784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef1806ce10 0x7fef18099a30 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fef080097f0 tx=0x7fef0801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c65040f6648805a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:21.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef1d0c0640 1 -- 172.21.15.60:0/339260784 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef080b1070 con 0x7fef1806ce10 2024-04-25T10:16:21.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef1d0c0640 1 -- 172.21.15.60:0/339260784 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef080ac410 con 0x7fef1806ce10 2024-04-25T10:16:21.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef1d0c0640 1 -- 172.21.15.60:0/339260784 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef080ac7e0 con 0x7fef1806ce10 2024-04-25T10:16:21.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef20326640 1 -- 172.21.15.60:0/339260784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef1806ce10 msgr2=0x7fef18099a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef20326640 1 --2- 172.21.15.60:0/339260784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef1806ce10 0x7fef18099a30 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fef080097f0 tx=0x7fef0801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef20326640 1 -- 172.21.15.60:0/339260784 shutdown_connections 2024-04-25T10:16:21.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef20326640 1 --2- 172.21.15.60:0/339260784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef1806ce10 0x7fef18099a30 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.586+0000 7fef20326640 1 -- 172.21.15.60:0/339260784 >> 172.21.15.60:0/339260784 conn(0x7fef180b32a0 msgr2=0x7fef180b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef20326640 1 -- 172.21.15.60:0/339260784 shutdown_connections 2024-04-25T10:16:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef20326640 1 -- 172.21.15.60:0/339260784 wait complete. 2024-04-25T10:16:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef20326640 1 Processor -- start 2024-04-25T10:16:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef20326640 1 -- start start 2024-04-25T10:16:21.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef20326640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef1806ce10 0x7fef180ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef20326640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef18099f70 0x7fef180c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef20326640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef180cdc30 0x7fef1814cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef20326640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1806d260 con 0x7fef18099f70 2024-04-25T10:16:21.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef20326640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef180ccfb0 con 0x7fef1806ce10 2024-04-25T10:16:21.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef20326640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1816ba70 con 0x7fef180cdc30 2024-04-25T10:16:21.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef1d8c1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef18099f70 0x7fef180c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef1d8c1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef18099f70 0x7fef180c52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60422/0 (socket says 172.21.15.60:60422) 2024-04-25T10:16:21.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef1d8c1640 1 -- 172.21.15.60:0/2629449774 learned_addr learned my addr 172.21.15.60:0/2629449774 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:21.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef1e0c2640 1 --2- 172.21.15.60:0/2629449774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef1806ce10 0x7fef180ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef1e8c3640 1 --2- 172.21.15.60:0/2629449774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef180cdc30 0x7fef1814cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef1d8c1640 1 -- 172.21.15.60:0/2629449774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef180cdc30 msgr2=0x7fef1814cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef1d8c1640 1 --2- 172.21.15.60:0/2629449774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef180cdc30 0x7fef1814cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef1d8c1640 1 -- 172.21.15.60:0/2629449774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef1806ce10 msgr2=0x7fef180ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef1d8c1640 1 --2- 172.21.15.60:0/2629449774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef1806ce10 0x7fef180ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef1d8c1640 1 -- 172.21.15.60:0/2629449774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef180c5a30 con 0x7fef18099f70 2024-04-25T10:16:21.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef1e0c2640 1 --2- 172.21.15.60:0/2629449774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef1806ce10 0x7fef180ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:21.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef1d8c1640 1 --2- 172.21.15.60:0/2629449774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef18099f70 0x7fef180c52e0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fef080190b0 tx=0x7fef08019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:21.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef0effd640 1 -- 172.21.15.60:0/2629449774 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef080b1040 con 0x7fef18099f70 2024-04-25T10:16:21.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef0effd640 1 -- 172.21.15.60:0/2629449774 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef080bb070 con 0x7fef18099f70 2024-04-25T10:16:21.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef20326640 1 -- 172.21.15.60:0/2629449774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef180c5d10 con 0x7fef18099f70 2024-04-25T10:16:21.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef0effd640 1 -- 172.21.15.60:0/2629449774 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef080b5650 con 0x7fef18099f70 2024-04-25T10:16:21.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef20326640 1 -- 172.21.15.60:0/2629449774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef180c62d0 con 0x7fef18099f70 2024-04-25T10:16:21.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.590+0000 7fef20326640 1 -- 172.21.15.60:0/2629449774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef18074770 con 0x7fef18099f70 2024-04-25T10:16:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.594+0000 7fef0effd640 1 -- 172.21.15.60:0/2629449774 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef080b57f0 con 0x7fef18099f70 2024-04-25T10:16:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.594+0000 7fef0effd640 1 --2- 172.21.15.60:0/2629449774 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feefc066f60 0x7feefc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.594+0000 7fef0effd640 1 -- 172.21.15.60:0/2629449774 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef0811f590 con 0x7fef18099f70 2024-04-25T10:16:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.594+0000 7fef1e0c2640 1 --2- 172.21.15.60:0/2629449774 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feefc066f60 0x7feefc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.594+0000 7fef1e0c2640 1 --2- 172.21.15.60:0/2629449774 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feefc066f60 0x7feefc069420 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fef180c7760 tx=0x7fef14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.598+0000 7fef0effd640 1 -- 172.21.15.60:0/2629449774 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef08016090 con 0x7fef18099f70 2024-04-25T10:16:21.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.786+0000 7fef20326640 1 -- 172.21.15.60:0/2629449774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fef1815d680 con 0x7fef18099f70 2024-04-25T10:16:21.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.786+0000 7fef0effd640 1 -- 172.21.15.60:0/2629449774 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v93) v1 ==== 53+0+73 (secure 0 0 0) 0x7fef080f0580 con 0x7fef18099f70 2024-04-25T10:16:21.792 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:16:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.790+0000 7fef20326640 1 -- 172.21.15.60:0/2629449774 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feefc066f60 msgr2=0x7feefc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.790+0000 7fef20326640 1 --2- 172.21.15.60:0/2629449774 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feefc066f60 0x7feefc069420 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fef180c7760 tx=0x7fef14009040 comp rx=0 tx=0).stop 2024-04-25T10:16:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.790+0000 7fef20326640 1 -- 172.21.15.60:0/2629449774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef18099f70 msgr2=0x7fef180c52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.790+0000 7fef20326640 1 --2- 172.21.15.60:0/2629449774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef18099f70 0x7fef180c52e0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fef080190b0 tx=0x7fef08019240 comp rx=0 tx=0).stop 2024-04-25T10:16:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.790+0000 7fef20326640 1 -- 172.21.15.60:0/2629449774 shutdown_connections 2024-04-25T10:16:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.790+0000 7fef20326640 1 --2- 172.21.15.60:0/2629449774 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feefc066f60 0x7feefc069420 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.790+0000 7fef20326640 1 --2- 172.21.15.60:0/2629449774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef180cdc30 0x7fef1814cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.790+0000 7fef20326640 1 --2- 172.21.15.60:0/2629449774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef18099f70 0x7fef180c52e0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.790+0000 7fef20326640 1 --2- 172.21.15.60:0/2629449774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef1806ce10 0x7fef180ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.790+0000 7fef20326640 1 -- 172.21.15.60:0/2629449774 >> 172.21.15.60:0/2629449774 conn(0x7fef180b32a0 msgr2=0x7fef180b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:21.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.790+0000 7fef20326640 1 -- 172.21.15.60:0/2629449774 shutdown_connections 2024-04-25T10:16:21.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:21.790+0000 7fef20326640 1 -- 172.21.15.60:0/2629449774 wait complete. 2024-04-25T10:16:21.809 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:16:21.810 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:16:21.810 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:16:21.810 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:16:21.810 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:21.810 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:16:21.810 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:16:21.810 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:16:21.810 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:21.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.810 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:16:21.817 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:16:21.817 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:16:21.817 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:16:21.817 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:21.818 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:16:21.818 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:21.818 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:21.818 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:21.818 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:16:21.818 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:21.818 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:21.818 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:16:21.818 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:16:21.818 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:21.818 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:16:21.818 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:16:21.818 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41875sec preferred_lft 41875sec 2024-04-25T10:16:21.819 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:16:21.819 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:21.819 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:21.819 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:21.819 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:16:21.819 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:16:21.819 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:21.819 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:16:21.819 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:21.819 INFO:teuthology.orchestra.run.smithi060.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:16:21.819 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:21.819 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:16:21.819 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:21.819 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:16:21.820 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:21.820 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:21.820 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:21.820 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:16:21.820 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:21.820 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:21.820 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:21.820 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:16:21.820 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:21.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.821 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:16:21.870 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:16:21.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.872 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:16:21.919 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:16:21.919 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:16:21.919 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:16:21.920 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:16:21.920 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:16:21.920 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:16:21.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.920 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:16:21.920 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:16:21.920 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:16:21.920 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:16:21.920 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:16:21.920 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:16:21.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:21.993 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:16:22.042 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:16:22.042 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:16:22.042 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:22.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:22.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T10:16:22.099 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:16:22.099 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:22.099 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:22.159 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:22.160 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:22.160 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:22.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:16:22.263 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:16:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:16:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:16:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:16:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:16:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:16:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:16:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:16:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:16:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:16:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:16:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:16:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:16:22.315 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:16:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:16:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:16:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:16:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:16:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:16:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:16:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:16:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:16:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:16:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:16:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:16:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:16:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:16:22.316 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:16:22.317 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:16:22.317 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:16:22.317 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:22.317 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:22.317 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:22.317 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:22.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:22.318 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:16:22.370 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:16:22.370 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:16:22.415 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:16:22.429 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:16:22.478 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:16:22.478 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:16:22.478 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:16:22.478 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:16:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:16:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:16:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:16:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:16:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:16:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:16:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:16:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:16:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:16:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:16:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:16:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:16:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:16:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:16:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:16:22.479 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:16:22.480 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:16:22.480 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:16:22.480 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:16:22.480 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:16:22.480 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:16:22.480 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:16:22.480 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:16:22.480 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:16:22.480 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:16:22.480 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:22.480 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:22.480 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:22.480 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:22.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:22.481 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:16:22.494 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2024-04-25T10:16:22.490+0000 7fcfc7e6c580 -1 init, newargv = 0x5573b945d0c0 newargc=15 2024-04-25T10:16:22.495 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[52106]: starting ceph client 2024-04-25T10:16:22.500 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[52106]: starting fuse 2024-04-25T10:16:23.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:16:23.616 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:16:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:16:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:16:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:16:23.664 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:16:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:16:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:16:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:16:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:16:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:16:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:16:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:16:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:16:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:16:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:16:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:16:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:16:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:16:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:16:23.665 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:16:23.666 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:16:23.666 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:16:23.666 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:16:23.666 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:16:23.666 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:16:23.666 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:16:23.666 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:16:23.666 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:16:23.666 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:16:23.666 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:16:23.666 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:23.666 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:23.666 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:23.666 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:23.666 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:23.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:23.667 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:16:23.714 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:16:23.714 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:16:23.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:16:23.715 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:16:23.715 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:16:23.715 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:23.715 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:16:23.715 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:23.715 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:16:23.715 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:16:23.715 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:16:23.715 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:16:23.715 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:23.715 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:16:23.715 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:16:23.716 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:16:23.716 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:16:23.716 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:23.716 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:16:23.716 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:16:23.716 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:16:23.716 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:16:23.716 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:16:23.716 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:16:23.716 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:16:23.716 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:16:23.716 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:23.716 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2024-04-25T10:16:23.716 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:23.843 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.52106.asok 2024-04-25T10:16:23.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:23.855 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52106.asok 2024-04-25T10:16:23.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:23.855 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52106.asok status 2024-04-25T10:16:24.017 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:16:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:16:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:16:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:16:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2024-04-25T10:16:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:16:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:16:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "52106", 2024-04-25T10:16:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:16:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:16:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:16:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9612, 2024-04-25T10:16:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:16:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:16:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:16:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 9612 2024-04-25T10:16:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:16:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:16:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:16:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 934737889 2024-04-25T10:16:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:16:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:16:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:16:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 934737889 2024-04-25T10:16:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.9612 192.168.0.1:0/934737889", 2024-04-25T10:16:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/934737889", 2024-04-25T10:16:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:16:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 93, 2024-04-25T10:16:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 110, 2024-04-25T10:16:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:16:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:16:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:16:24.019 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:16:24.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:24.032 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:16:24.032 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:24.038 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:16:24.038 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:24.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:24.039 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:24.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:24.091 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-04-25T10:16:24.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:24.145 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-04-25T10:16:24.273 INFO:teuthology.orchestra.run.smithi060.stderr:8+0 records in 2024-04-25T10:16:24.273 INFO:teuthology.orchestra.run.smithi060.stderr:8+0 records out 2024-04-25T10:16:24.273 INFO:teuthology.orchestra.run.smithi060.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0718688 s, 117 MB/s 2024-04-25T10:16:24.274 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:24.275 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:24.275 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:24.275 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:24.275 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:16:24.275 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:24.346 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627779 2024-04-25T10:16:24.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:24.356 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:24.356 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:24.356 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:24.356 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:24.356 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:16:24.356 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:24.458 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627780 2024-04-25T10:16:24.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:24.468 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:16:24.468 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:24.515 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:24.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.678+0000 7fc3545a1640 1 Processor -- start 2024-04-25T10:16:24.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.678+0000 7fc3545a1640 1 -- start start 2024-04-25T10:16:24.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.678+0000 7fc3545a1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc34c0ce7a0 0x7fc34c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:24.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.678+0000 7fc3545a1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc34c15bc90 0x7fc34c0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:24.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.678+0000 7fc3545a1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34c0cf0c0 con 0x7fc34c0b4660 2024-04-25T10:16:24.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.678+0000 7fc3545a1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc34c0cc370 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.678+0000 7fc3545a1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c071ca0 con 0x7fc34c15bc90 2024-04-25T10:16:24.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.678+0000 7fc35233d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc34c0ce7a0 0x7fc34c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:24.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.678+0000 7fc35233d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc34c0ce7a0 0x7fc34c0ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60436/0 (socket says 172.21.15.60:60436) 2024-04-25T10:16:24.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.678+0000 7fc35233d640 1 -- 172.21.15.60:0/3266386323 learned_addr learned my addr 172.21.15.60:0/3266386323 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:24.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.678+0000 7fc351b3c640 1 --2- 172.21.15.60:0/3266386323 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc34c15bc90 0x7fc34c0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:24.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.678+0000 7fc35233d640 1 -- 172.21.15.60:0/3266386323 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc34c15bc90 msgr2=0x7fc34c0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:24.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.678+0000 7fc35233d640 1 --2- 172.21.15.60:0/3266386323 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc34c15bc90 0x7fc34c0b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.678+0000 7fc35233d640 1 -- 172.21.15.60:0/3266386323 >> v1:172.21.15.100:6789/0 conn(0x7fc34c0b4660 legacy=0x7fc34c0b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:24.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.678+0000 7fc35233d640 1 -- 172.21.15.60:0/3266386323 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c0cca40 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.678+0000 7fc351b3c640 1 --2- 172.21.15.60:0/3266386323 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc34c15bc90 0x7fc34c0b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:24.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc35233d640 1 --2- 172.21.15.60:0/3266386323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc34c0ce7a0 0x7fc34c0ceb80 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fc3480098a0 tx=0x7fc34801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a253dc36d5f03ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:24.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc35133b640 1 -- 172.21.15.60:0/3266386323 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3480b0070 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc35133b640 1 -- 172.21.15.60:0/3266386323 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3480ab600 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc35133b640 1 -- 172.21.15.60:0/3266386323 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3480ab9d0 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc3545a1640 1 -- 172.21.15.60:0/3266386323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc34c0ce7a0 msgr2=0x7fc34c0ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:24.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc3545a1640 1 --2- 172.21.15.60:0/3266386323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc34c0ce7a0 0x7fc34c0ceb80 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fc3480098a0 tx=0x7fc34801c500 comp rx=0 tx=0).stop 2024-04-25T10:16:24.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc3545a1640 1 -- 172.21.15.60:0/3266386323 shutdown_connections 2024-04-25T10:16:24.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc3545a1640 1 --2- 172.21.15.60:0/3266386323 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc34c15bc90 0x7fc34c0b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc3545a1640 1 --2- 172.21.15.60:0/3266386323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc34c0ce7a0 0x7fc34c0ceb80 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc3545a1640 1 -- 172.21.15.60:0/3266386323 >> 172.21.15.60:0/3266386323 conn(0x7fc34c0b12d0 msgr2=0x7fc34c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:24.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc3545a1640 1 -- 172.21.15.60:0/3266386323 shutdown_connections 2024-04-25T10:16:24.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc3545a1640 1 -- 172.21.15.60:0/3266386323 wait complete. 2024-04-25T10:16:24.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc3545a1640 1 Processor -- start 2024-04-25T10:16:24.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc3545a1640 1 -- start start 2024-04-25T10:16:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc3545a1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc34c0ce7a0 0x7fc34c140340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc3545a1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc34c15bc90 0x7fc34c0afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc3545a1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc34c0b03b0 0x7fc34c157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc3545a1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0b4280 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc3545a1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0b3db0 con 0x7fc34c15bc90 2024-04-25T10:16:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc3545a1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0b4400 con 0x7fc34c0b03b0 2024-04-25T10:16:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc35233d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc34c0ce7a0 0x7fc34c140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc35233d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc34c0ce7a0 0x7fc34c140340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60440/0 (socket says 172.21.15.60:60440) 2024-04-25T10:16:24.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.682+0000 7fc35233d640 1 -- 172.21.15.60:0/4206968356 learned_addr learned my addr 172.21.15.60:0/4206968356 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc351b3c640 1 --2- 172.21.15.60:0/4206968356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc34c15bc90 0x7fc34c0afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc35233d640 1 -- 172.21.15.60:0/4206968356 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc34c0b03b0 msgr2=0x7fc34c157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc352b3e640 1 --2- 172.21.15.60:0/4206968356 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc34c0b03b0 0x7fc34c157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc35233d640 1 --2- 172.21.15.60:0/4206968356 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc34c0b03b0 0x7fc34c157420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc35233d640 1 -- 172.21.15.60:0/4206968356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc34c15bc90 msgr2=0x7fc34c0afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc35233d640 1 --2- 172.21.15.60:0/4206968356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc34c15bc90 0x7fc34c0afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc35233d640 1 -- 172.21.15.60:0/4206968356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c1581c0 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc351b3c640 1 --2- 172.21.15.60:0/4206968356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc34c15bc90 0x7fc34c0afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc35233d640 1 --2- 172.21.15.60:0/4206968356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc34c0ce7a0 0x7fc34c140340 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fc3480ae040 tx=0x7fc34801ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc342ffd640 1 -- 172.21.15.60:0/4206968356 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3480ba020 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc342ffd640 1 -- 172.21.15.60:0/4206968356 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3480b6640 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc3545a1640 1 -- 172.21.15.60:0/4206968356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34c1584a0 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc342ffd640 1 -- 172.21.15.60:0/4206968356 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3480b0040 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc3545a1640 1 -- 172.21.15.60:0/4206968356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34c140f40 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc3545a1640 1 -- 172.21.15.60:0/4206968356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc34c141140 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc342ffd640 1 -- 172.21.15.60:0/4206968356 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3480bd020 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc342ffd640 1 --2- 172.21.15.60:0/4206968356 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc330066f60 0x7fc330069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:24.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc342ffd640 1 -- 172.21.15.60:0/4206968356 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc34811f190 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc351b3c640 1 --2- 172.21.15.60:0/4206968356 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc330066f60 0x7fc330069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:24.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.686+0000 7fc351b3c640 1 --2- 172.21.15.60:0/4206968356 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc330066f60 0x7fc330069420 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc34c1fa250 tx=0x7fc33c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:24.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.690+0000 7fc342ffd640 1 -- 172.21.15.60:0/4206968356 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc34c141140 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.858+0000 7fc3545a1640 1 -- 172.21.15.60:0/4206968356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc34c074770 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.858+0000 7fc342ffd640 1 -- 172.21.15.60:0/4206968356 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16639 (secure 0 0 0) 0x7fc3480c4030 con 0x7fc34c0ce7a0 2024-04-25T10:16:24.865 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:24.865 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":110,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:16:09.041178+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:06.033387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:06.753421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:24.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.862+0000 7fc3545a1640 1 -- 172.21.15.60:0/4206968356 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc330066f60 msgr2=0x7fc330069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:24.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.862+0000 7fc3545a1640 1 --2- 172.21.15.60:0/4206968356 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc330066f60 0x7fc330069420 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc34c1fa250 tx=0x7fc33c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:24.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.862+0000 7fc3545a1640 1 -- 172.21.15.60:0/4206968356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc34c0ce7a0 msgr2=0x7fc34c140340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:24.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.862+0000 7fc3545a1640 1 --2- 172.21.15.60:0/4206968356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc34c0ce7a0 0x7fc34c140340 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fc3480ae040 tx=0x7fc34801ad40 comp rx=0 tx=0).stop 2024-04-25T10:16:24.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.862+0000 7fc3545a1640 1 -- 172.21.15.60:0/4206968356 shutdown_connections 2024-04-25T10:16:24.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.862+0000 7fc3545a1640 1 --2- 172.21.15.60:0/4206968356 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc330066f60 0x7fc330069420 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.862+0000 7fc3545a1640 1 --2- 172.21.15.60:0/4206968356 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc34c0b03b0 0x7fc34c157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.862+0000 7fc3545a1640 1 --2- 172.21.15.60:0/4206968356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc34c15bc90 0x7fc34c0afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.862+0000 7fc3545a1640 1 --2- 172.21.15.60:0/4206968356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc34c0ce7a0 0x7fc34c140340 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.862+0000 7fc3545a1640 1 -- 172.21.15.60:0/4206968356 >> 172.21.15.60:0/4206968356 conn(0x7fc34c0b12d0 msgr2=0x7fc34c155130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:24.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.862+0000 7fc3545a1640 1 -- 172.21.15.60:0/4206968356 shutdown_connections 2024-04-25T10:16:24.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:24.862+0000 7fc3545a1640 1 -- 172.21.15.60:0/4206968356 wait complete. 2024-04-25T10:16:24.884 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:06.033387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:06.753421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:24.884 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:25.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe01583b640 1 Processor -- start 2024-04-25T10:16:25.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe01583b640 1 -- start start 2024-04-25T10:16:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe01583b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe0100cf140 0x7fe0101668c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe01583b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe010145700 con 0x7fe010145320 2024-04-25T10:16:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe01583b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0100d0bd0 con 0x7fe0100713e0 2024-04-25T10:16:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe01583b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0100b4800 con 0x7fe0100cf140 2024-04-25T10:16:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00effd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe010145320 0x7fe010070cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35090/0 (socket says 172.21.15.60:35090) 2024-04-25T10:16:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00effd640 1 -- 172.21.15.60:0/1001669749 learned_addr learned my addr 172.21.15.60:0/1001669749 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00f7fe640 1 --2- 172.21.15.60:0/1001669749 >> v2:172.21.15.100:3300/0 conn(0x7fe0100cf140 0x7fe0101668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00dffb640 1 -- 172.21.15.60:0/1001669749 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831348698 0 0) 0x7fe010145700 con 0x7fe010145320 2024-04-25T10:16:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00dffb640 1 -- 172.21.15.60:0/1001669749 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdff8003530 con 0x7fe010145320 2024-04-25T10:16:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00dffb640 1 -- 172.21.15.60:0/1001669749 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 507242729 0 0) 0x7fdff8003530 con 0x7fe010145320 2024-04-25T10:16:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00dffb640 1 -- 172.21.15.60:0/1001669749 >> v2:172.21.15.100:3300/0 conn(0x7fe0100cf140 msgr2=0x7fe0101668c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00dffb640 1 --2- 172.21.15.60:0/1001669749 >> v2:172.21.15.100:3300/0 conn(0x7fe0100cf140 0x7fe0101668c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00dffb640 1 -- 172.21.15.60:0/1001669749 >> v1:172.21.15.100:6789/0 conn(0x7fe0100713e0 legacy=0x7fe0100cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00dffb640 1 -- 172.21.15.60:0/1001669749 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0100d0eb0 con 0x7fe010145320 2024-04-25T10:16:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00dffb640 1 -- 172.21.15.60:0/1001669749 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fe008001a00 con 0x7fe010145320 2024-04-25T10:16:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00dffb640 1 -- 172.21.15.60:0/1001669749 >> v1:172.21.15.60:6789/0 conn(0x7fe010145320 legacy=0x7fe010070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00dffb640 1 --2- 172.21.15.60:0/1001669749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8003a10 0x7fdff8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00dffb640 1 --2- 172.21.15.60:0/1001669749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdff8004810 0x7fdff8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00dffb640 1 --2- 172.21.15.60:0/1001669749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff8005220 0x7fdff800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00dffb640 1 -- 172.21.15.60:0/1001669749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100b4800 con 0x7fdff8004810 2024-04-25T10:16:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00dffb640 1 -- 172.21.15.60:0/1001669749 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff800c170 con 0x7fdff8005220 2024-04-25T10:16:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00dffb640 1 -- 172.21.15.60:0/1001669749 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff800c370 con 0x7fdff8003a10 2024-04-25T10:16:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00e7fc640 1 --2- 172.21.15.60:0/1001669749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdff8004810 0x7fdff8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00effd640 1 --2- 172.21.15.60:0/1001669749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8003a10 0x7fdff8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00f7fe640 1 --2- 172.21.15.60:0/1001669749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff8005220 0x7fdff800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00e7fc640 1 -- 172.21.15.60:0/1001669749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8003a10 msgr2=0x7fdff8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00e7fc640 1 --2- 172.21.15.60:0/1001669749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8003a10 0x7fdff8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00e7fc640 1 -- 172.21.15.60:0/1001669749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff8005220 msgr2=0x7fdff800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:25.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00e7fc640 1 --2- 172.21.15.60:0/1001669749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdff8005220 0x7fdff800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00e7fc640 1 -- 172.21.15.60:0/1001669749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff8003530 con 0x7fdff8004810 2024-04-25T10:16:25.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00f7fe640 1 -- 172.21.15.60:0/1001669749 reap_dead start 2024-04-25T10:16:25.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.010+0000 7fe00effd640 1 --2- 172.21.15.60:0/1001669749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8003a10 0x7fdff8003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.014+0000 7fe00e7fc640 1 --2- 172.21.15.60:0/1001669749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdff8004810 0x7fdff8004c90 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fe004001750 tx=0x7fe00401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb39b6dd275c17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.014+0000 7fe00dffb640 1 -- 172.21.15.60:0/1001669749 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0040013a0 con 0x7fdff8004810 2024-04-25T10:16:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.014+0000 7fe00dffb640 1 -- 172.21.15.60:0/1001669749 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0040052f0 con 0x7fdff8004810 2024-04-25T10:16:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.014+0000 7fe00dffb640 1 -- 172.21.15.60:0/1001669749 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004003840 con 0x7fdff8004810 2024-04-25T10:16:25.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.014+0000 7fe01583b640 1 -- 172.21.15.60:0/1001669749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdff8004810 msgr2=0x7fdff8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.014+0000 7fe01583b640 1 --2- 172.21.15.60:0/1001669749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdff8004810 0x7fdff8004c90 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fe004001750 tx=0x7fe00401c390 comp rx=0 tx=0).stop 2024-04-25T10:16:25.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.014+0000 7fe01583b640 1 -- 172.21.15.60:0/1001669749 shutdown_connections 2024-04-25T10:16:25.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.014+0000 7fe01583b640 1 --2- 172.21.15.60:0/1001669749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdff8004810 0x7fdff8004c90 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.014+0000 7fe01583b640 1 -- 172.21.15.60:0/1001669749 >> 172.21.15.60:0/1001669749 conn(0x7fe01006f530 msgr2=0x7fe01006f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:25.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.014+0000 7fe01583b640 1 -- 172.21.15.60:0/1001669749 shutdown_connections 2024-04-25T10:16:25.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.014+0000 7fe01583b640 1 -- 172.21.15.60:0/1001669749 wait complete. 2024-04-25T10:16:25.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe01583b640 1 Processor -- start 2024-04-25T10:16:25.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe01583b640 1 -- start start 2024-04-25T10:16:25.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe01583b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8004810 0x7fe01006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe01583b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe01006bd20 0x7fe01015ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe01583b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01013acb0 0x7fe0101413d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe01583b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe010071260 con 0x7fe01013acb0 2024-04-25T10:16:25.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe01583b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100710e0 con 0x7fe01006bd20 2024-04-25T10:16:25.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe01583b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01015fda0 con 0x7fdff8004810 2024-04-25T10:16:25.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe00f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01013acb0 0x7fe0101413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe00f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01013acb0 0x7fe0101413d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60472/0 (socket says 172.21.15.60:60472) 2024-04-25T10:16:25.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe00f7fe640 1 -- 172.21.15.60:0/1916696013 learned_addr learned my addr 172.21.15.60:0/1916696013 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:25.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe00e7fc640 1 --2- 172.21.15.60:0/1916696013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe01006bd20 0x7fe01015ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe00effd640 1 --2- 172.21.15.60:0/1916696013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8004810 0x7fe01006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe00f7fe640 1 -- 172.21.15.60:0/1916696013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8004810 msgr2=0x7fe01006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe00f7fe640 1 --2- 172.21.15.60:0/1916696013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8004810 0x7fe01006d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe00f7fe640 1 -- 172.21.15.60:0/1916696013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe01006bd20 msgr2=0x7fe01015ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe00f7fe640 1 --2- 172.21.15.60:0/1916696013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe01006bd20 0x7fe01015ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe00f7fe640 1 -- 172.21.15.60:0/1916696013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01006c200 con 0x7fe01013acb0 2024-04-25T10:16:25.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe00f7fe640 1 --2- 172.21.15.60:0/1916696013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01013acb0 0x7fe0101413d0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fe0100705a0 tx=0x7fe000011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fdff7fff640 1 -- 172.21.15.60:0/1916696013 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000019020 con 0x7fe01013acb0 2024-04-25T10:16:25.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fdff7fff640 1 -- 172.21.15.60:0/1916696013 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe000004460 con 0x7fe01013acb0 2024-04-25T10:16:25.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fdff7fff640 1 -- 172.21.15.60:0/1916696013 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000003860 con 0x7fe01013acb0 2024-04-25T10:16:25.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe01583b640 1 -- 172.21.15.60:0/1916696013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01006c4e0 con 0x7fe01013acb0 2024-04-25T10:16:25.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe01583b640 1 -- 172.21.15.60:0/1916696013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01006caa0 con 0x7fe01013acb0 2024-04-25T10:16:25.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fe01583b640 1 -- 172.21.15.60:0/1916696013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe010074770 con 0x7fe01013acb0 2024-04-25T10:16:25.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.018+0000 7fdff7fff640 1 -- 172.21.15.60:0/1916696013 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe00001c020 con 0x7fe01013acb0 2024-04-25T10:16:25.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.022+0000 7fdff7fff640 1 --2- 172.21.15.60:0/1916696013 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdffc066f90 0x7fdffc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.022+0000 7fdff7fff640 1 -- 172.21.15.60:0/1916696013 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0000855d0 con 0x7fe01013acb0 2024-04-25T10:16:25.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.022+0000 7fe00effd640 1 --2- 172.21.15.60:0/1916696013 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdffc066f90 0x7fdffc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.022+0000 7fe00effd640 1 --2- 172.21.15.60:0/1916696013 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdffc066f90 0x7fdffc069450 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe008001340 tx=0x7fe008009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.026+0000 7fdff7fff640 1 -- 172.21.15.60:0/1916696013 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe000023020 con 0x7fe01013acb0 2024-04-25T10:16:25.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.186+0000 7fe01583b640 1 -- 172.21.15.60:0/1916696013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0100b5e60 con 0x7fe01013acb0 2024-04-25T10:16:25.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.186+0000 7fdff7fff640 1 -- 172.21.15.60:0/1916696013 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe000056670 con 0x7fe01013acb0 2024-04-25T10:16:25.190 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:16:25.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.186+0000 7fe01583b640 1 -- 172.21.15.60:0/1916696013 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdffc066f90 msgr2=0x7fdffc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.186+0000 7fe01583b640 1 --2- 172.21.15.60:0/1916696013 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdffc066f90 0x7fdffc069450 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe008001340 tx=0x7fe008009040 comp rx=0 tx=0).stop 2024-04-25T10:16:25.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.186+0000 7fe01583b640 1 -- 172.21.15.60:0/1916696013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01013acb0 msgr2=0x7fe0101413d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.186+0000 7fe01583b640 1 --2- 172.21.15.60:0/1916696013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01013acb0 0x7fe0101413d0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fe0100705a0 tx=0x7fe000011920 comp rx=0 tx=0).stop 2024-04-25T10:16:25.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.186+0000 7fe01583b640 1 -- 172.21.15.60:0/1916696013 shutdown_connections 2024-04-25T10:16:25.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.186+0000 7fe01583b640 1 --2- 172.21.15.60:0/1916696013 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdffc066f90 0x7fdffc069450 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.186+0000 7fe01583b640 1 --2- 172.21.15.60:0/1916696013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe01013acb0 0x7fe0101413d0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.186+0000 7fe01583b640 1 --2- 172.21.15.60:0/1916696013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe01006bd20 0x7fe01015ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.186+0000 7fe01583b640 1 --2- 172.21.15.60:0/1916696013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdff8004810 0x7fe01006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.186+0000 7fe01583b640 1 -- 172.21.15.60:0/1916696013 >> 172.21.15.60:0/1916696013 conn(0x7fe01006f530 msgr2=0x7fe0100b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:25.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.186+0000 7fe01583b640 1 -- 172.21.15.60:0/1916696013 shutdown_connections 2024-04-25T10:16:25.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.186+0000 7fe01583b640 1 -- 172.21.15.60:0/1916696013 wait complete. 2024-04-25T10:16:25.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:16:25.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff6a9a640 1 Processor -- start 2024-04-25T10:16:25.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff6a9a640 1 -- start start 2024-04-25T10:16:25.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff6a9a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbff015de90 0x7fbff00b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff6a9a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbff00d0320 0x7fbff00d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff6a9a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff00ce290 0x7fbff00ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff6a9a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff00cf2a0 con 0x7fbff015de90 2024-04-25T10:16:25.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff6a9a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff00cefc0 con 0x7fbff00d0320 2024-04-25T10:16:25.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff6a9a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00d1d50 con 0x7fbff00ce290 2024-04-25T10:16:25.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff4836640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbff015de90 0x7fbff00b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff4836640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbff015de90 0x7fbff00b6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60478/0 (socket says 172.21.15.60:60478) 2024-04-25T10:16:25.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff4836640 1 -- 172.21.15.60:0/2821253408 learned_addr learned my addr 172.21.15.60:0/2821253408 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:25.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbfeffff640 1 --2- 172.21.15.60:0/2821253408 >> v2:172.21.15.100:3300/0 conn(0x7fbff00d0320 0x7fbff00d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff5037640 1 --2- 172.21.15.60:0/2821253408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff00ce290 0x7fbff00ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff4836640 1 -- 172.21.15.60:0/2821253408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff00ce290 msgr2=0x7fbff00ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:25.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff4836640 1 --2- 172.21.15.60:0/2821253408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff00ce290 0x7fbff00ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff4836640 1 -- 172.21.15.60:0/2821253408 >> v2:172.21.15.100:3300/0 conn(0x7fbff00d0320 msgr2=0x7fbff00d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:25.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff4836640 1 --2- 172.21.15.60:0/2821253408 >> v2:172.21.15.100:3300/0 conn(0x7fbff00d0320 0x7fbff00d07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff4836640 1 -- 172.21.15.60:0/2821253408 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff006d200 con 0x7fbff015de90 2024-04-25T10:16:25.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbff5037640 1 --2- 172.21.15.60:0/2821253408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff00ce290 0x7fbff00ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:25.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.326+0000 7fbfeffff640 1 --2- 172.21.15.60:0/2821253408 >> v2:172.21.15.100:3300/0 conn(0x7fbff00d0320 0x7fbff00d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:25.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff4836640 1 --2- 172.21.15.60:0/2821253408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff015de90 0x7fbff00b6bf0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fbff00b6470 tx=0x7fbfe001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23dfaba7d3b934 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbfef7fe640 1 -- 172.21.15.60:0/2821253408 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe00b0070 con 0x7fbff015de90 2024-04-25T10:16:25.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbfef7fe640 1 -- 172.21.15.60:0/2821253408 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe00ab600 con 0x7fbff015de90 2024-04-25T10:16:25.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbfef7fe640 1 -- 172.21.15.60:0/2821253408 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe00ab9d0 con 0x7fbff015de90 2024-04-25T10:16:25.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 -- 172.21.15.60:0/2821253408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff015de90 msgr2=0x7fbff00b6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 --2- 172.21.15.60:0/2821253408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff015de90 0x7fbff00b6bf0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fbff00b6470 tx=0x7fbfe001c520 comp rx=0 tx=0).stop 2024-04-25T10:16:25.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 -- 172.21.15.60:0/2821253408 shutdown_connections 2024-04-25T10:16:25.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 --2- 172.21.15.60:0/2821253408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff00ce290 0x7fbff00ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 --2- 172.21.15.60:0/2821253408 >> v2:172.21.15.100:3300/0 conn(0x7fbff00d0320 0x7fbff00d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 --2- 172.21.15.60:0/2821253408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff015de90 0x7fbff00b6bf0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 -- 172.21.15.60:0/2821253408 >> 172.21.15.60:0/2821253408 conn(0x7fbff006c090 msgr2=0x7fbff006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:25.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 -- 172.21.15.60:0/2821253408 shutdown_connections 2024-04-25T10:16:25.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 -- 172.21.15.60:0/2821253408 wait complete. 2024-04-25T10:16:25.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 Processor -- start 2024-04-25T10:16:25.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 -- start start 2024-04-25T10:16:25.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff00ce290 0x7fbff00c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff00d0320 0x7fbff014b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff015de90 0x7fbff014be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00b5d20 con 0x7fbff015de90 2024-04-25T10:16:25.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00b5ba0 con 0x7fbff00d0320 2024-04-25T10:16:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff6a9a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00b5ea0 con 0x7fbff00ce290 2024-04-25T10:16:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.330+0000 7fbff5037640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff015de90 0x7fbff014be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbff5037640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff015de90 0x7fbff014be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60480/0 (socket says 172.21.15.60:60480) 2024-04-25T10:16:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbff5037640 1 -- 172.21.15.60:0/169349642 learned_addr learned my addr 172.21.15.60:0/169349642 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbff4836640 1 --2- 172.21.15.60:0/169349642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff00ce290 0x7fbff00c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbfeffff640 1 --2- 172.21.15.60:0/169349642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff00d0320 0x7fbff014b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbff5037640 1 -- 172.21.15.60:0/169349642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff00ce290 msgr2=0x7fbff00c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbff5037640 1 --2- 172.21.15.60:0/169349642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff00ce290 0x7fbff00c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbff5037640 1 -- 172.21.15.60:0/169349642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff00d0320 msgr2=0x7fbff014b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbff5037640 1 --2- 172.21.15.60:0/169349642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff00d0320 0x7fbff014b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbff5037640 1 -- 172.21.15.60:0/169349642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff00cc820 con 0x7fbff015de90 2024-04-25T10:16:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbff4836640 1 --2- 172.21.15.60:0/169349642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff00ce290 0x7fbff00c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbff5037640 1 --2- 172.21.15.60:0/169349642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff015de90 0x7fbff014be10 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800cf20 tx=0x7fbfe800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbfed7fa640 1 -- 172.21.15.60:0/169349642 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe801a070 con 0x7fbff015de90 2024-04-25T10:16:25.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbfed7fa640 1 -- 172.21.15.60:0/169349642 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe800d840 con 0x7fbff015de90 2024-04-25T10:16:25.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbff6a9a640 1 -- 172.21.15.60:0/169349642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff00ccb00 con 0x7fbff015de90 2024-04-25T10:16:25.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbfed7fa640 1 -- 172.21.15.60:0/169349642 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe800dc30 con 0x7fbff015de90 2024-04-25T10:16:25.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbff6a9a640 1 -- 172.21.15.60:0/169349642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff00cd0c0 con 0x7fbff015de90 2024-04-25T10:16:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbff6a9a640 1 -- 172.21.15.60:0/169349642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff0074770 con 0x7fbff015de90 2024-04-25T10:16:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbfed7fa640 1 -- 172.21.15.60:0/169349642 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbfe801d070 con 0x7fbff015de90 2024-04-25T10:16:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbfed7fa640 1 --2- 172.21.15.60:0/169349642 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbfdc066f60 0x7fbfdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbfed7fa640 1 -- 172.21.15.60:0/169349642 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfe8085800 con 0x7fbff015de90 2024-04-25T10:16:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbff4836640 1 --2- 172.21.15.60:0/169349642 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbfdc066f60 0x7fbfdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.334+0000 7fbff4836640 1 --2- 172.21.15.60:0/169349642 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbfdc066f60 0x7fbfdc069420 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00ad000 tx=0x7fbfe00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.338+0000 7fbfed7fa640 1 -- 172.21.15.60:0/169349642 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbfe80568a0 con 0x7fbff015de90 2024-04-25T10:16:25.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.502+0000 7fbff6a9a640 1 -- 172.21.15.60:0/169349642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbff00d1780 con 0x7fbff015de90 2024-04-25T10:16:25.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.502+0000 7fbfed7fa640 1 -- 172.21.15.60:0/169349642 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbfe80561a0 con 0x7fbff015de90 2024-04-25T10:16:25.507 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:16:25.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.502+0000 7fbff6a9a640 1 -- 172.21.15.60:0/169349642 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbfdc066f60 msgr2=0x7fbfdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.502+0000 7fbff6a9a640 1 --2- 172.21.15.60:0/169349642 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbfdc066f60 0x7fbfdc069420 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00ad000 tx=0x7fbfe00a6000 comp rx=0 tx=0).stop 2024-04-25T10:16:25.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.502+0000 7fbff6a9a640 1 -- 172.21.15.60:0/169349642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff015de90 msgr2=0x7fbff014be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.502+0000 7fbff6a9a640 1 --2- 172.21.15.60:0/169349642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff015de90 0x7fbff014be10 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800cf20 tx=0x7fbfe800a710 comp rx=0 tx=0).stop 2024-04-25T10:16:25.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.502+0000 7fbff6a9a640 1 -- 172.21.15.60:0/169349642 shutdown_connections 2024-04-25T10:16:25.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.502+0000 7fbff6a9a640 1 --2- 172.21.15.60:0/169349642 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbfdc066f60 0x7fbfdc069420 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.502+0000 7fbff6a9a640 1 --2- 172.21.15.60:0/169349642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff015de90 0x7fbff014be10 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.502+0000 7fbff6a9a640 1 --2- 172.21.15.60:0/169349642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff00d0320 0x7fbff014b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.502+0000 7fbff6a9a640 1 --2- 172.21.15.60:0/169349642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff00ce290 0x7fbff00c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.502+0000 7fbff6a9a640 1 -- 172.21.15.60:0/169349642 >> 172.21.15.60:0/169349642 conn(0x7fbff006c090 msgr2=0x7fbff00b8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:25.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.502+0000 7fbff6a9a640 1 -- 172.21.15.60:0/169349642 shutdown_connections 2024-04-25T10:16:25.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.502+0000 7fbff6a9a640 1 -- 172.21.15.60:0/169349642 wait complete. 2024-04-25T10:16:25.525 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:16:25.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd0611a1640 1 Processor -- start 2024-04-25T10:16:25.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd0611a1640 1 -- start start 2024-04-25T10:16:25.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd0611a1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd05c1620d0 0x7fd05c0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd0611a1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd05c15bad0 con 0x7fd05c15b6f0 2024-04-25T10:16:25.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd0611a1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd05c15ebe0 con 0x7fd05c0ced20 2024-04-25T10:16:25.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd0611a1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c161c40 con 0x7fd05c1620d0 2024-04-25T10:16:25.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd05ad9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd05c15b6f0 0x7fd05c15e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35094/0 (socket says 172.21.15.60:35094) 2024-04-25T10:16:25.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd05ad9d640 1 -- 172.21.15.60:0/3810666139 learned_addr learned my addr 172.21.15.60:0/3810666139 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:25.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd05a59c640 1 --2- 172.21.15.60:0/3810666139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd05c1620d0 0x7fd05c0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd059d9b640 1 -- 172.21.15.60:0/3810666139 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920592831 0 0) 0x7fd05c15bad0 con 0x7fd05c15b6f0 2024-04-25T10:16:25.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd059d9b640 1 -- 172.21.15.60:0/3810666139 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd044003530 con 0x7fd05c15b6f0 2024-04-25T10:16:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd059d9b640 1 -- 172.21.15.60:0/3810666139 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188887159 0 0) 0x7fd05c15ebe0 con 0x7fd05c0ced20 2024-04-25T10:16:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd059d9b640 1 -- 172.21.15.60:0/3810666139 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd05c15bad0 con 0x7fd05c0ced20 2024-04-25T10:16:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd05a59c640 1 -- 172.21.15.60:0/3810666139 >> v1:172.21.15.100:6789/0 conn(0x7fd05c0ced20 legacy=0x7fd05c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd059d9b640 1 -- 172.21.15.60:0/3810666139 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4076817616 0 0) 0x7fd044003530 con 0x7fd05c15b6f0 2024-04-25T10:16:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd05a59c640 1 -- 172.21.15.60:0/3810666139 >> v1:172.21.15.60:6789/0 conn(0x7fd05c15b6f0 legacy=0x7fd05c15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.650+0000 7fd05a59c640 1 -- 172.21.15.60:0/3810666139 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05c15eec0 con 0x7fd05c1620d0 2024-04-25T10:16:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.654+0000 7fd05a59c640 1 --2- 172.21.15.60:0/3810666139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd05c1620d0 0x7fd05c0ce7e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd0500097f0 tx=0x7fd05001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=889593bb85d7fe58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.654+0000 7fd059d9b640 1 -- 172.21.15.60:0/3810666139 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0500b0070 con 0x7fd05c1620d0 2024-04-25T10:16:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.654+0000 7fd059d9b640 1 -- 172.21.15.60:0/3810666139 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0500ab400 con 0x7fd05c1620d0 2024-04-25T10:16:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.654+0000 7fd059d9b640 1 -- 172.21.15.60:0/3810666139 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c001680 con 0x7fd05c1620d0 2024-04-25T10:16:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.654+0000 7fd0611a1640 1 -- 172.21.15.60:0/3810666139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd05c1620d0 msgr2=0x7fd05c0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.654+0000 7fd0611a1640 1 --2- 172.21.15.60:0/3810666139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd05c1620d0 0x7fd05c0ce7e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd0500097f0 tx=0x7fd05001c400 comp rx=0 tx=0).stop 2024-04-25T10:16:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.654+0000 7fd0611a1640 1 -- 172.21.15.60:0/3810666139 shutdown_connections 2024-04-25T10:16:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.654+0000 7fd0611a1640 1 --2- 172.21.15.60:0/3810666139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd05c1620d0 0x7fd05c0ce7e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.654+0000 7fd0611a1640 1 -- 172.21.15.60:0/3810666139 >> 172.21.15.60:0/3810666139 conn(0x7fd05c072340 msgr2=0x7fd05c072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.654+0000 7fd0611a1640 1 -- 172.21.15.60:0/3810666139 shutdown_connections 2024-04-25T10:16:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.654+0000 7fd0611a1640 1 -- 172.21.15.60:0/3810666139 wait complete. 2024-04-25T10:16:25.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.654+0000 7fd0611a1640 1 Processor -- start 2024-04-25T10:16:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.654+0000 7fd0611a1640 1 -- start start 2024-04-25T10:16:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd0611a1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd05c0ced20 0x7fd05c14e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd0611a1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd05c15b6f0 0x7fd05c14eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd0611a1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd05c1620d0 0x7fd05c148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd0611a1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c0b96d0 con 0x7fd05c15b6f0 2024-04-25T10:16:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd0611a1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c0b9550 con 0x7fd05c1620d0 2024-04-25T10:16:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd0611a1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c15e7e0 con 0x7fd05c0ced20 2024-04-25T10:16:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd05a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd05c15b6f0 0x7fd05c14eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd05a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd05c15b6f0 0x7fd05c14eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60488/0 (socket says 172.21.15.60:60488) 2024-04-25T10:16:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd05a59c640 1 -- 172.21.15.60:0/912980621 learned_addr learned my addr 172.21.15.60:0/912980621 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd05ad9d640 1 --2- 172.21.15.60:0/912980621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd05c0ced20 0x7fd05c14e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd05b59e640 1 --2- 172.21.15.60:0/912980621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd05c1620d0 0x7fd05c148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd05a59c640 1 -- 172.21.15.60:0/912980621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd05c0ced20 msgr2=0x7fd05c14e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd05a59c640 1 --2- 172.21.15.60:0/912980621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd05c0ced20 0x7fd05c14e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd05a59c640 1 -- 172.21.15.60:0/912980621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd05c1620d0 msgr2=0x7fd05c148c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd05a59c640 1 --2- 172.21.15.60:0/912980621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd05c1620d0 0x7fd05c148c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd05a59c640 1 -- 172.21.15.60:0/912980621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05c149af0 con 0x7fd05c15b6f0 2024-04-25T10:16:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd05a59c640 1 --2- 172.21.15.60:0/912980621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd05c15b6f0 0x7fd05c14eb80 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fd05001a030 tx=0x7fd0500191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd0437fe640 1 -- 172.21.15.60:0/912980621 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0500b0040 con 0x7fd05c15b6f0 2024-04-25T10:16:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd0437fe640 1 -- 172.21.15.60:0/912980621 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd050003620 con 0x7fd05c15b6f0 2024-04-25T10:16:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd0611a1640 1 -- 172.21.15.60:0/912980621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd05c152380 con 0x7fd05c15b6f0 2024-04-25T10:16:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd0437fe640 1 -- 172.21.15.60:0/912980621 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0500039d0 con 0x7fd05c15b6f0 2024-04-25T10:16:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd0611a1640 1 -- 172.21.15.60:0/912980621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05c14cbc0 con 0x7fd05c15b6f0 2024-04-25T10:16:25.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd0611a1640 1 -- 172.21.15.60:0/912980621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05c074770 con 0x7fd05c15b6f0 2024-04-25T10:16:25.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd0437fe640 1 -- 172.21.15.60:0/912980621 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0500bd020 con 0x7fd05c15b6f0 2024-04-25T10:16:25.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd0437fe640 1 --2- 172.21.15.60:0/912980621 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd048066f60 0x7fd048069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd0437fe640 1 -- 172.21.15.60:0/912980621 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd05011f540 con 0x7fd05c15b6f0 2024-04-25T10:16:25.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.658+0000 7fd05ad9d640 1 --2- 172.21.15.60:0/912980621 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd048066f60 0x7fd048069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.662+0000 7fd05ad9d640 1 --2- 172.21.15.60:0/912980621 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd048066f60 0x7fd048069420 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0011f0 tx=0x7fd04c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.662+0000 7fd0437fe640 1 -- 172.21.15.60:0/912980621 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd050015ff0 con 0x7fd05c15b6f0 2024-04-25T10:16:25.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.822+0000 7fd0611a1640 1 -- 172.21.15.60:0/912980621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd05c149cc0 con 0x7fd05c15b6f0 2024-04-25T10:16:25.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.822+0000 7fd0437fe640 1 -- 172.21.15.60:0/912980621 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd0500f04e0 con 0x7fd05c15b6f0 2024-04-25T10:16:25.827 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:16:25.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.822+0000 7fd0611a1640 1 -- 172.21.15.60:0/912980621 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd048066f60 msgr2=0x7fd048069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.822+0000 7fd0611a1640 1 --2- 172.21.15.60:0/912980621 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd048066f60 0x7fd048069420 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0011f0 tx=0x7fd04c009040 comp rx=0 tx=0).stop 2024-04-25T10:16:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.826+0000 7fd0611a1640 1 -- 172.21.15.60:0/912980621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd05c15b6f0 msgr2=0x7fd05c14eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.826+0000 7fd0611a1640 1 --2- 172.21.15.60:0/912980621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd05c15b6f0 0x7fd05c14eb80 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fd05001a030 tx=0x7fd0500191a0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.826+0000 7fd0611a1640 1 -- 172.21.15.60:0/912980621 shutdown_connections 2024-04-25T10:16:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.826+0000 7fd0611a1640 1 --2- 172.21.15.60:0/912980621 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd048066f60 0x7fd048069420 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.826+0000 7fd0611a1640 1 --2- 172.21.15.60:0/912980621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd05c1620d0 0x7fd05c148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.826+0000 7fd0611a1640 1 --2- 172.21.15.60:0/912980621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd05c15b6f0 0x7fd05c14eb80 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.826+0000 7fd0611a1640 1 --2- 172.21.15.60:0/912980621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd05c0ced20 0x7fd05c14e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.826+0000 7fd0611a1640 1 -- 172.21.15.60:0/912980621 >> 172.21.15.60:0/912980621 conn(0x7fd05c072340 msgr2=0x7fd05c0d28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.826+0000 7fd0611a1640 1 -- 172.21.15.60:0/912980621 shutdown_connections 2024-04-25T10:16:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.826+0000 7fd0611a1640 1 -- 172.21.15.60:0/912980621 wait complete. 2024-04-25T10:16:25.844 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:16:25.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa589247640 1 Processor -- start 2024-04-25T10:16:25.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa589247640 1 -- start start 2024-04-25T10:16:25.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa589247640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5840ce7a0 0x7fa5840ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa589247640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa58415bc90 0x7fa5840b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa589247640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa5840b4660 0x7fa5840b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa589247640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5840cc370 con 0x7fa5840b4660 2024-04-25T10:16:25.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa589247640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5840cf0c0 con 0x7fa58415bc90 2024-04-25T10:16:25.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa589247640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa584071ca0 con 0x7fa5840ce7a0 2024-04-25T10:16:25.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa58359e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa5840b4660 0x7fa5840b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa58359e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa5840b4660 0x7fa5840b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60502/0 (socket says 172.21.15.60:60502) 2024-04-25T10:16:25.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa58359e640 1 -- 172.21.15.60:0/643036760 learned_addr learned my addr 172.21.15.60:0/643036760 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:25.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa582d9d640 1 --2- 172.21.15.60:0/643036760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5840ce7a0 0x7fa5840ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa57bfff640 1 --2- 172.21.15.60:0/643036760 >> v2:172.21.15.100:3300/0 conn(0x7fa58415bc90 0x7fa5840b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa58359e640 1 -- 172.21.15.60:0/643036760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5840ce7a0 msgr2=0x7fa5840ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:25.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa58359e640 1 --2- 172.21.15.60:0/643036760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5840ce7a0 0x7fa5840ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa58359e640 1 -- 172.21.15.60:0/643036760 >> v2:172.21.15.100:3300/0 conn(0x7fa58415bc90 msgr2=0x7fa5840b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:25.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa58359e640 1 --2- 172.21.15.60:0/643036760 >> v2:172.21.15.100:3300/0 conn(0x7fa58415bc90 0x7fa5840b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.954+0000 7fa58359e640 1 -- 172.21.15.60:0/643036760 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5840cca40 con 0x7fa5840b4660 2024-04-25T10:16:25.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa58359e640 1 --2- 172.21.15.60:0/643036760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5840b4660 0x7fa5840b2b10 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009e50 tx=0x7fa57c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad29ea3800fb1b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa58259c640 1 -- 172.21.15.60:0/643036760 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0a5670 con 0x7fa5840b4660 2024-04-25T10:16:25.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa58259c640 1 -- 172.21.15.60:0/643036760 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57c0a5810 con 0x7fa5840b4660 2024-04-25T10:16:25.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa58259c640 1 -- 172.21.15.60:0/643036760 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0a5be0 con 0x7fa5840b4660 2024-04-25T10:16:25.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 -- 172.21.15.60:0/643036760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5840b4660 msgr2=0x7fa5840b2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 --2- 172.21.15.60:0/643036760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5840b4660 0x7fa5840b2b10 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009e50 tx=0x7fa57c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 -- 172.21.15.60:0/643036760 shutdown_connections 2024-04-25T10:16:25.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 --2- 172.21.15.60:0/643036760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5840b4660 0x7fa5840b2b10 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 --2- 172.21.15.60:0/643036760 >> v2:172.21.15.100:3300/0 conn(0x7fa58415bc90 0x7fa5840b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 --2- 172.21.15.60:0/643036760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5840ce7a0 0x7fa5840ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 -- 172.21.15.60:0/643036760 >> 172.21.15.60:0/643036760 conn(0x7fa5840b12d0 msgr2=0x7fa5840b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:25.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 -- 172.21.15.60:0/643036760 shutdown_connections 2024-04-25T10:16:25.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 -- 172.21.15.60:0/643036760 wait complete. 2024-04-25T10:16:25.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 Processor -- start 2024-04-25T10:16:25.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 -- start start 2024-04-25T10:16:25.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5840ce7a0 0x7fa584139f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa58415bc90 0x7fa58415b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa584149460 0x7fa5840afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840b3db0 con 0x7fa584149460 2024-04-25T10:16:25.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840b3460 con 0x7fa58415bc90 2024-04-25T10:16:25.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa589247640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840b4280 con 0x7fa5840ce7a0 2024-04-25T10:16:25.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa58359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa584149460 0x7fa5840afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa58359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa584149460 0x7fa5840afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60508/0 (socket says 172.21.15.60:60508) 2024-04-25T10:16:25.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa58359e640 1 -- 172.21.15.60:0/248164211 learned_addr learned my addr 172.21.15.60:0/248164211 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:25.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa582d9d640 1 --2- 172.21.15.60:0/248164211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5840ce7a0 0x7fa584139f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa57bfff640 1 --2- 172.21.15.60:0/248164211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa58415bc90 0x7fa58415b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa58359e640 1 -- 172.21.15.60:0/248164211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5840ce7a0 msgr2=0x7fa584139f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa58359e640 1 --2- 172.21.15.60:0/248164211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5840ce7a0 0x7fa584139f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa58359e640 1 -- 172.21.15.60:0/248164211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa58415bc90 msgr2=0x7fa58415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa58359e640 1 --2- 172.21.15.60:0/248164211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa58415bc90 0x7fa58415b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa58359e640 1 -- 172.21.15.60:0/248164211 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58415b7b0 con 0x7fa584149460 2024-04-25T10:16:25.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.958+0000 7fa58359e640 1 --2- 172.21.15.60:0/248164211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa584149460 0x7fa5840afb70 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0af040 tx=0x7fa57c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.962+0000 7fa57b7fe640 1 -- 172.21.15.60:0/248164211 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c009510 con 0x7fa584149460 2024-04-25T10:16:25.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.962+0000 7fa57b7fe640 1 -- 172.21.15.60:0/248164211 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57c0be070 con 0x7fa584149460 2024-04-25T10:16:25.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.962+0000 7fa589247640 1 -- 172.21.15.60:0/248164211 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa584148050 con 0x7fa584149460 2024-04-25T10:16:25.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.962+0000 7fa57b7fe640 1 -- 172.21.15.60:0/248164211 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0b88b0 con 0x7fa584149460 2024-04-25T10:16:25.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.962+0000 7fa589247640 1 -- 172.21.15.60:0/248164211 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa584148610 con 0x7fa584149460 2024-04-25T10:16:25.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.962+0000 7fa589247640 1 -- 172.21.15.60:0/248164211 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa584074770 con 0x7fa584149460 2024-04-25T10:16:25.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.962+0000 7fa57b7fe640 1 -- 172.21.15.60:0/248164211 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa57c0b8a50 con 0x7fa584149460 2024-04-25T10:16:25.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.962+0000 7fa57b7fe640 1 --2- 172.21.15.60:0/248164211 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 0x7fa564069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.962+0000 7fa57b7fe640 1 -- 172.21.15.60:0/248164211 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa57c120220 con 0x7fa584149460 2024-04-25T10:16:25.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.962+0000 7fa582d9d640 1 --2- 172.21.15.60:0/248164211 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 0x7fa564069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.962+0000 7fa582d9d640 1 --2- 172.21.15.60:0/248164211 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 0x7fa564069420 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa58415ebb0 tx=0x7fa574008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:25.966+0000 7fa57b7fe640 1 -- 172.21.15.60:0/248164211 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa57c0165e0 con 0x7fa584149460 2024-04-25T10:16:26.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.130+0000 7fa589247640 1 -- 172.21.15.60:0/248164211 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa5840aed70 con 0x7fa584149460 2024-04-25T10:16:26.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.130+0000 7fa57b7fe640 1 -- 172.21.15.60:0/248164211 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa57c0f1410 con 0x7fa584149460 2024-04-25T10:16:26.137 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:16:26.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.134+0000 7fa589247640 1 -- 172.21.15.60:0/248164211 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 msgr2=0x7fa564069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.134+0000 7fa589247640 1 --2- 172.21.15.60:0/248164211 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 0x7fa564069420 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa58415ebb0 tx=0x7fa574008040 comp rx=0 tx=0).stop 2024-04-25T10:16:26.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.134+0000 7fa589247640 1 -- 172.21.15.60:0/248164211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa584149460 msgr2=0x7fa5840afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.134+0000 7fa589247640 1 --2- 172.21.15.60:0/248164211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa584149460 0x7fa5840afb70 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0af040 tx=0x7fa57c019790 comp rx=0 tx=0).stop 2024-04-25T10:16:26.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.134+0000 7fa589247640 1 -- 172.21.15.60:0/248164211 shutdown_connections 2024-04-25T10:16:26.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.134+0000 7fa589247640 1 --2- 172.21.15.60:0/248164211 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa564066f60 0x7fa564069420 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.134+0000 7fa589247640 1 --2- 172.21.15.60:0/248164211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa584149460 0x7fa5840afb70 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.134+0000 7fa589247640 1 --2- 172.21.15.60:0/248164211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa58415bc90 0x7fa58415b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.134+0000 7fa589247640 1 --2- 172.21.15.60:0/248164211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5840ce7a0 0x7fa584139f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.134+0000 7fa589247640 1 -- 172.21.15.60:0/248164211 >> 172.21.15.60:0/248164211 conn(0x7fa5840b12d0 msgr2=0x7fa584154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.134+0000 7fa589247640 1 -- 172.21.15.60:0/248164211 shutdown_connections 2024-04-25T10:16:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.134+0000 7fa589247640 1 -- 172.21.15.60:0/248164211 wait complete. 2024-04-25T10:16:26.154 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:16:26.154 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:26.159 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.318+0000 7fd06bb5a640 1 Processor -- start 2024-04-25T10:16:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.318+0000 7fd06bb5a640 1 -- start start 2024-04-25T10:16:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.318+0000 7fd06bb5a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd06415bc90 0x7fd06415c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.318+0000 7fd06bb5a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd064071ca0 con 0x7fd0640b4660 2024-04-25T10:16:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.318+0000 7fd06bb5a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0640cca40 con 0x7fd0640ce7a0 2024-04-25T10:16:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.318+0000 7fd06bb5a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0640cc370 con 0x7fd06415bc90 2024-04-25T10:16:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.318+0000 7fd0690f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd06415bc90 0x7fd06415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.318+0000 7fd0690f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd06415bc90 0x7fd06415c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60524/0 (socket says 172.21.15.60:60524) 2024-04-25T10:16:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.318+0000 7fd06a0f7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd0640b4660 0x7fd06415c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35106/0 (socket says 172.21.15.60:35106) 2024-04-25T10:16:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.318+0000 7fd0690f5640 1 -- 172.21.15.60:0/3712212569 learned_addr learned my addr 172.21.15.60:0/3712212569 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:26.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.318+0000 7fd0690f5640 1 -- 172.21.15.60:0/3712212569 >> v1:172.21.15.100:6789/0 conn(0x7fd0640ce7a0 legacy=0x7fd0640ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:26.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.318+0000 7fd0690f5640 1 -- 172.21.15.60:0/3712212569 >> v1:172.21.15.60:6789/0 conn(0x7fd0640b4660 legacy=0x7fd06415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.318+0000 7fd0688f4640 1 -- 172.21.15.60:0/3712212569 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645729851 0 0) 0x7fd064071ca0 con 0x7fd0640b4660 2024-04-25T10:16:26.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.318+0000 7fd0690f5640 1 -- 172.21.15.60:0/3712212569 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0640ccd20 con 0x7fd06415bc90 2024-04-25T10:16:26.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.322+0000 7fd0690f5640 1 --2- 172.21.15.60:0/3712212569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd06415bc90 0x7fd06415c220 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fd054009870 tx=0x7fd05401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77fa2963b1b80f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.322+0000 7fd0688f4640 1 -- 172.21.15.60:0/3712212569 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0540b0070 con 0x7fd06415bc90 2024-04-25T10:16:26.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.322+0000 7fd0688f4640 1 -- 172.21.15.60:0/3712212569 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0540ab600 con 0x7fd06415bc90 2024-04-25T10:16:26.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.322+0000 7fd0688f4640 1 -- 172.21.15.60:0/3712212569 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0540ab9d0 con 0x7fd06415bc90 2024-04-25T10:16:26.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.322+0000 7fd06bb5a640 1 -- 172.21.15.60:0/3712212569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd06415bc90 msgr2=0x7fd06415c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.322+0000 7fd06bb5a640 1 --2- 172.21.15.60:0/3712212569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd06415bc90 0x7fd06415c220 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fd054009870 tx=0x7fd05401c520 comp rx=0 tx=0).stop 2024-04-25T10:16:26.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.322+0000 7fd06bb5a640 1 -- 172.21.15.60:0/3712212569 shutdown_connections 2024-04-25T10:16:26.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.322+0000 7fd06bb5a640 1 --2- 172.21.15.60:0/3712212569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd06415bc90 0x7fd06415c220 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.322+0000 7fd06bb5a640 1 -- 172.21.15.60:0/3712212569 >> 172.21.15.60:0/3712212569 conn(0x7fd0640b12d0 msgr2=0x7fd0640b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:26.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.322+0000 7fd06bb5a640 1 -- 172.21.15.60:0/3712212569 shutdown_connections 2024-04-25T10:16:26.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.322+0000 7fd06bb5a640 1 -- 172.21.15.60:0/3712212569 wait complete. 2024-04-25T10:16:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.322+0000 7fd06bb5a640 1 Processor -- start 2024-04-25T10:16:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06bb5a640 1 -- start start 2024-04-25T10:16:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06bb5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0640ce7a0 0x7fd0641480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06bb5a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06415bc90 0x7fd064155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06bb5a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0641399a0 0x7fd064139e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06bb5a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0641695d0 con 0x7fd0641399a0 2024-04-25T10:16:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06bb5a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640b3db0 con 0x7fd0640ce7a0 2024-04-25T10:16:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06bb5a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd064169750 con 0x7fd06415bc90 2024-04-25T10:16:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06a0f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0641399a0 0x7fd064139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06a0f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0641399a0 0x7fd064139e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60538/0 (socket says 172.21.15.60:60538) 2024-04-25T10:16:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06a0f7640 1 -- 172.21.15.60:0/1598772321 learned_addr learned my addr 172.21.15.60:0/1598772321 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06a0f7640 1 -- 172.21.15.60:0/1598772321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06415bc90 msgr2=0x7fd064155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06a0f7640 1 --2- 172.21.15.60:0/1598772321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06415bc90 0x7fd064155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06a0f7640 1 -- 172.21.15.60:0/1598772321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0640ce7a0 msgr2=0x7fd0641480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd0698f6640 1 --2- 172.21.15.60:0/1598772321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0640ce7a0 0x7fd0641480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06a0f7640 1 --2- 172.21.15.60:0/1598772321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0640ce7a0 0x7fd0641480e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06a0f7640 1 -- 172.21.15.60:0/1598772321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06413a370 con 0x7fd0641399a0 2024-04-25T10:16:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06a0f7640 1 --2- 172.21.15.60:0/1598772321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0641399a0 0x7fd064139e30 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00c6d0 tx=0x7fd05c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd05a7fc640 1 -- 172.21.15.60:0/1598772321 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c018070 con 0x7fd0641399a0 2024-04-25T10:16:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd05a7fc640 1 -- 172.21.15.60:0/1598772321 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c004d70 con 0x7fd0641399a0 2024-04-25T10:16:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06bb5a640 1 -- 172.21.15.60:0/1598772321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06414aa40 con 0x7fd0641399a0 2024-04-25T10:16:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd05a7fc640 1 -- 172.21.15.60:0/1598772321 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c00a880 con 0x7fd0641399a0 2024-04-25T10:16:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06bb5a640 1 -- 172.21.15.60:0/1598772321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06414b000 con 0x7fd0641399a0 2024-04-25T10:16:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd06bb5a640 1 -- 172.21.15.60:0/1598772321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd064074770 con 0x7fd0641399a0 2024-04-25T10:16:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd05a7fc640 1 -- 172.21.15.60:0/1598772321 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd05c01b070 con 0x7fd0641399a0 2024-04-25T10:16:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd05a7fc640 1 --2- 172.21.15.60:0/1598772321 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd048066f60 0x7fd048069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd05a7fc640 1 -- 172.21.15.60:0/1598772321 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd05c084790 con 0x7fd0641399a0 2024-04-25T10:16:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.326+0000 7fd0698f6640 1 --2- 172.21.15.60:0/1598772321 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd048066f60 0x7fd048069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.330+0000 7fd0698f6640 1 --2- 172.21.15.60:0/1598772321 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd048066f60 0x7fd048069420 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd06000a750 tx=0x7fd060008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.330+0000 7fd05a7fc640 1 -- 172.21.15.60:0/1598772321 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd05c055780 con 0x7fd0641399a0 2024-04-25T10:16:26.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.490+0000 7fd06bb5a640 1 -- 172.21.15.60:0/1598772321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd06415e030 con 0x7fd0641399a0 2024-04-25T10:16:26.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.494+0000 7fd05a7fc640 1 -- 172.21.15.60:0/1598772321 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16639 (secure 0 0 0) 0x7fd05c055080 con 0x7fd0641399a0 2024-04-25T10:16:26.500 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:26.500 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":110,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:16:09.041178+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:06.033387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:06.753421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.498+0000 7fd06bb5a640 1 -- 172.21.15.60:0/1598772321 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd048066f60 msgr2=0x7fd048069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.498+0000 7fd06bb5a640 1 --2- 172.21.15.60:0/1598772321 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd048066f60 0x7fd048069420 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd06000a750 tx=0x7fd060008040 comp rx=0 tx=0).stop 2024-04-25T10:16:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.498+0000 7fd06bb5a640 1 -- 172.21.15.60:0/1598772321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0641399a0 msgr2=0x7fd064139e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.498+0000 7fd06bb5a640 1 --2- 172.21.15.60:0/1598772321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0641399a0 0x7fd064139e30 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00c6d0 tx=0x7fd05c00cb90 comp rx=0 tx=0).stop 2024-04-25T10:16:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.498+0000 7fd06bb5a640 1 -- 172.21.15.60:0/1598772321 shutdown_connections 2024-04-25T10:16:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.498+0000 7fd06bb5a640 1 --2- 172.21.15.60:0/1598772321 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd048066f60 0x7fd048069420 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.498+0000 7fd06bb5a640 1 --2- 172.21.15.60:0/1598772321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0641399a0 0x7fd064139e30 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.498+0000 7fd06bb5a640 1 --2- 172.21.15.60:0/1598772321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd06415bc90 0x7fd064155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.498+0000 7fd06bb5a640 1 --2- 172.21.15.60:0/1598772321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0640ce7a0 0x7fd0641480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.498+0000 7fd06bb5a640 1 -- 172.21.15.60:0/1598772321 >> 172.21.15.60:0/1598772321 conn(0x7fd0640b12d0 msgr2=0x7fd0640b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:26.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.498+0000 7fd06bb5a640 1 -- 172.21.15.60:0/1598772321 shutdown_connections 2024-04-25T10:16:26.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.498+0000 7fd06bb5a640 1 -- 172.21.15.60:0/1598772321 wait complete. 2024-04-25T10:16:26.520 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:06.033387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:06.753421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:26.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:26.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.650+0000 7f8b95b14640 1 Processor -- start 2024-04-25T10:16:26.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.650+0000 7f8b95b14640 1 -- start start 2024-04-25T10:16:26.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.650+0000 7f8b95b14640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8b9015dd30 0x7f8b9015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.650+0000 7f8b95b14640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8b900cf8a0 0x7f8b9015be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.650+0000 7f8b95b14640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b900cd890 con 0x7f8b900cd3d0 2024-04-25T10:16:26.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.650+0000 7f8b95b14640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b900ce4a0 con 0x7f8b900cf8a0 2024-04-25T10:16:26.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.650+0000 7f8b95b14640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b900cdb70 con 0x7f8b9015dd30 2024-04-25T10:16:26.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.650+0000 7f8b8ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8b900cf8a0 0x7f8b9015be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.650+0000 7f8b8ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8b900cf8a0 0x7f8b9015be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60540/0 (socket says 172.21.15.60:60540) 2024-04-25T10:16:26.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.650+0000 7f8b8ffff640 1 -- 172.21.15.60:0/1553615882 learned_addr learned my addr 172.21.15.60:0/1553615882 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:26.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.650+0000 7f8b8f7fe640 1 --2- 172.21.15.60:0/1553615882 >> v2:172.21.15.100:3300/0 conn(0x7f8b9015dd30 0x7f8b9015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.650+0000 7f8b8ffff640 1 -- 172.21.15.60:0/1553615882 >> v2:172.21.15.100:3300/0 conn(0x7f8b9015dd30 msgr2=0x7f8b9015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:26.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.650+0000 7f8b8ffff640 1 --2- 172.21.15.60:0/1553615882 >> v2:172.21.15.100:3300/0 conn(0x7f8b9015dd30 0x7f8b9015ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.650+0000 7f8b8ffff640 1 -- 172.21.15.60:0/1553615882 >> v1:172.21.15.100:6789/0 conn(0x7f8b900cd3d0 legacy=0x7f8b900cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:26.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.650+0000 7f8b8ffff640 1 -- 172.21.15.60:0/1553615882 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b900cde50 con 0x7f8b900cf8a0 2024-04-25T10:16:26.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b8ffff640 1 --2- 172.21.15.60:0/1553615882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b900cf8a0 0x7f8b9015be20 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f8b9006bbf0 tx=0x7f8b8001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a739c94ae8d99fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b8e7fc640 1 -- 172.21.15.60:0/1553615882 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b800ab640 con 0x7f8b900cf8a0 2024-04-25T10:16:26.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b8e7fc640 1 -- 172.21.15.60:0/1553615882 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b800b0070 con 0x7f8b900cf8a0 2024-04-25T10:16:26.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b8e7fc640 1 -- 172.21.15.60:0/1553615882 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b800aba00 con 0x7f8b900cf8a0 2024-04-25T10:16:26.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b95b14640 1 -- 172.21.15.60:0/1553615882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b900cf8a0 msgr2=0x7f8b9015be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b95b14640 1 --2- 172.21.15.60:0/1553615882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b900cf8a0 0x7f8b9015be20 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f8b9006bbf0 tx=0x7f8b8001c420 comp rx=0 tx=0).stop 2024-04-25T10:16:26.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b95b14640 1 -- 172.21.15.60:0/1553615882 shutdown_connections 2024-04-25T10:16:26.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b95b14640 1 --2- 172.21.15.60:0/1553615882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b900cf8a0 0x7f8b9015be20 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b95b14640 1 --2- 172.21.15.60:0/1553615882 >> v2:172.21.15.100:3300/0 conn(0x7f8b9015dd30 0x7f8b9015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b95b14640 1 -- 172.21.15.60:0/1553615882 >> 172.21.15.60:0/1553615882 conn(0x7f8b900670b0 msgr2=0x7f8b9006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:26.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b95b14640 1 -- 172.21.15.60:0/1553615882 shutdown_connections 2024-04-25T10:16:26.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b95b14640 1 -- 172.21.15.60:0/1553615882 wait complete. 2024-04-25T10:16:26.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b95b14640 1 Processor -- start 2024-04-25T10:16:26.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b95b14640 1 -- start start 2024-04-25T10:16:26.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b95b14640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b900cd3d0 0x7f8b9013f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b95b14640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b900cf8a0 0x7f8b900a8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b95b14640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b9015dd30 0x7f8b900a92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b95b14640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9015cf00 con 0x7f8b900cf8a0 2024-04-25T10:16:26.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b95b14640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9015cd80 con 0x7f8b900cd3d0 2024-04-25T10:16:26.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b95b14640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9015d080 con 0x7f8b9015dd30 2024-04-25T10:16:26.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b8effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b900cf8a0 0x7f8b900a8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b8effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b900cf8a0 0x7f8b900a8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60552/0 (socket says 172.21.15.60:60552) 2024-04-25T10:16:26.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b8effd640 1 -- 172.21.15.60:0/1186052626 learned_addr learned my addr 172.21.15.60:0/1186052626 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:26.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b8f7fe640 1 --2- 172.21.15.60:0/1186052626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b900cd3d0 0x7f8b9013f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b8ffff640 1 --2- 172.21.15.60:0/1186052626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b9015dd30 0x7f8b900a92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b8effd640 1 -- 172.21.15.60:0/1186052626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b9015dd30 msgr2=0x7f8b900a92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b8effd640 1 --2- 172.21.15.60:0/1186052626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b9015dd30 0x7f8b900a92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b8effd640 1 -- 172.21.15.60:0/1186052626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b900cd3d0 msgr2=0x7f8b9013f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b8effd640 1 --2- 172.21.15.60:0/1186052626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b900cd3d0 0x7f8b9013f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.654+0000 7f8b8effd640 1 -- 172.21.15.60:0/1186052626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9013f540 con 0x7f8b900cf8a0 2024-04-25T10:16:26.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.658+0000 7f8b8effd640 1 --2- 172.21.15.60:0/1186052626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b900cf8a0 0x7f8b900a8da0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800c450 tx=0x7f8b7800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.658+0000 7f8b87fff640 1 -- 172.21.15.60:0/1186052626 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78012830 con 0x7f8b900cf8a0 2024-04-25T10:16:26.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.658+0000 7f8b87fff640 1 -- 172.21.15.60:0/1186052626 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b78017070 con 0x7f8b900cf8a0 2024-04-25T10:16:26.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.658+0000 7f8b95b14640 1 -- 172.21.15.60:0/1186052626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9014f530 con 0x7f8b900cf8a0 2024-04-25T10:16:26.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.658+0000 7f8b87fff640 1 -- 172.21.15.60:0/1186052626 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78012c30 con 0x7f8b900cf8a0 2024-04-25T10:16:26.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.658+0000 7f8b95b14640 1 -- 172.21.15.60:0/1186052626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9014faf0 con 0x7f8b900cf8a0 2024-04-25T10:16:26.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.658+0000 7f8b95b14640 1 -- 172.21.15.60:0/1186052626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b901502c0 con 0x7f8b900cf8a0 2024-04-25T10:16:26.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.658+0000 7f8b87fff640 1 -- 172.21.15.60:0/1186052626 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b7801c460 con 0x7f8b900cf8a0 2024-04-25T10:16:26.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.658+0000 7f8b87fff640 1 --2- 172.21.15.60:0/1186052626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8b70066f60 0x7f8b70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.658+0000 7f8b87fff640 1 -- 172.21.15.60:0/1186052626 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b78084d50 con 0x7f8b900cf8a0 2024-04-25T10:16:26.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.658+0000 7f8b8f7fe640 1 --2- 172.21.15.60:0/1186052626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8b70066f60 0x7f8b70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.658+0000 7f8b8f7fe640 1 --2- 172.21.15.60:0/1186052626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8b70066f60 0x7f8b70069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8b880070b0 tx=0x7f8b88007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.662+0000 7f8b87fff640 1 -- 172.21.15.60:0/1186052626 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b78088030 con 0x7f8b900cf8a0 2024-04-25T10:16:26.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.822+0000 7f8b95b14640 1 -- 172.21.15.60:0/1186052626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b900cec50 con 0x7f8b900cf8a0 2024-04-25T10:16:26.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.826+0000 7f8b87fff640 1 -- 172.21.15.60:0/1186052626 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b78055df0 con 0x7f8b900cf8a0 2024-04-25T10:16:26.830 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:16:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.826+0000 7f8b95b14640 1 -- 172.21.15.60:0/1186052626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8b70066f60 msgr2=0x7f8b70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.826+0000 7f8b95b14640 1 --2- 172.21.15.60:0/1186052626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8b70066f60 0x7f8b70069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8b880070b0 tx=0x7f8b88007040 comp rx=0 tx=0).stop 2024-04-25T10:16:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.826+0000 7f8b95b14640 1 -- 172.21.15.60:0/1186052626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b900cf8a0 msgr2=0x7f8b900a8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.826+0000 7f8b95b14640 1 --2- 172.21.15.60:0/1186052626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b900cf8a0 0x7f8b900a8da0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800c450 tx=0x7f8b7800c910 comp rx=0 tx=0).stop 2024-04-25T10:16:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.826+0000 7f8b95b14640 1 -- 172.21.15.60:0/1186052626 shutdown_connections 2024-04-25T10:16:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.826+0000 7f8b95b14640 1 --2- 172.21.15.60:0/1186052626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f8b70066f60 0x7f8b70069420 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.826+0000 7f8b95b14640 1 --2- 172.21.15.60:0/1186052626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b9015dd30 0x7f8b900a92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.826+0000 7f8b95b14640 1 --2- 172.21.15.60:0/1186052626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b900cf8a0 0x7f8b900a8da0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.826+0000 7f8b95b14640 1 --2- 172.21.15.60:0/1186052626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b900cd3d0 0x7f8b9013f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.826+0000 7f8b95b14640 1 -- 172.21.15.60:0/1186052626 >> 172.21.15.60:0/1186052626 conn(0x7f8b900670b0 msgr2=0x7f8b90152240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:26.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.826+0000 7f8b95b14640 1 -- 172.21.15.60:0/1186052626 shutdown_connections 2024-04-25T10:16:26.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.826+0000 7f8b95b14640 1 -- 172.21.15.60:0/1186052626 wait complete. 2024-04-25T10:16:26.849 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:16:26.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.978+0000 7f84e6d71640 1 Processor -- start 2024-04-25T10:16:26.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.978+0000 7f84e6d71640 1 -- start start 2024-04-25T10:16:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.978+0000 7f84e6d71640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f84e015db10 0x7f84e00b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.978+0000 7f84e6d71640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84e00aec10 0x7f84e00b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.978+0000 7f84e6d71640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e00ae140 con 0x7f84e00cfcf0 2024-04-25T10:16:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.978+0000 7f84e6d71640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e00accc0 con 0x7f84e00aec10 2024-04-25T10:16:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.978+0000 7f84e6d71640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e015def0 con 0x7f84e015db10 2024-04-25T10:16:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.978+0000 7f84dffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84e00aec10 0x7f84e00b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.978+0000 7f84dffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84e00aec10 0x7f84e00b3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60560/0 (socket says 172.21.15.60:60560) 2024-04-25T10:16:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.978+0000 7f84e530e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f84e00cfcf0 0x7f84e0161660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35114/0 (socket says 172.21.15.60:35114) 2024-04-25T10:16:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.978+0000 7f84dffff640 1 -- 172.21.15.60:0/259331178 learned_addr learned my addr 172.21.15.60:0/259331178 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.978+0000 7f84e4b0d640 1 --2- 172.21.15.60:0/259331178 >> v2:172.21.15.100:3300/0 conn(0x7f84e015db10 0x7f84e00b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.978+0000 7f84dffff640 1 -- 172.21.15.60:0/259331178 >> v2:172.21.15.100:3300/0 conn(0x7f84e015db10 msgr2=0x7f84e00b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.978+0000 7f84dffff640 1 --2- 172.21.15.60:0/259331178 >> v2:172.21.15.100:3300/0 conn(0x7f84e015db10 0x7f84e00b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.978+0000 7f84dffff640 1 -- 172.21.15.60:0/259331178 >> v1:172.21.15.60:6789/0 conn(0x7f84e00cfcf0 legacy=0x7f84e0161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.978+0000 7f84dffff640 1 -- 172.21.15.60:0/259331178 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e0071630 con 0x7f84e00aec10 2024-04-25T10:16:26.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84dffff640 1 --2- 172.21.15.60:0/259331178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84e00aec10 0x7f84e00b3420 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f84e006f5d0 tx=0x7f84d401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b6f6bc3d27b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84df7fe640 1 -- 172.21.15.60:0/259331178 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d40ab640 con 0x7f84e00aec10 2024-04-25T10:16:26.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84df7fe640 1 -- 172.21.15.60:0/259331178 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d40b0070 con 0x7f84e00aec10 2024-04-25T10:16:26.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84df7fe640 1 -- 172.21.15.60:0/259331178 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d40aba00 con 0x7f84e00aec10 2024-04-25T10:16:26.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e6d71640 1 -- 172.21.15.60:0/259331178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84e00aec10 msgr2=0x7f84e00b3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e6d71640 1 --2- 172.21.15.60:0/259331178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84e00aec10 0x7f84e00b3420 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f84e006f5d0 tx=0x7f84d401c520 comp rx=0 tx=0).stop 2024-04-25T10:16:26.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e6d71640 1 -- 172.21.15.60:0/259331178 shutdown_connections 2024-04-25T10:16:26.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e6d71640 1 --2- 172.21.15.60:0/259331178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84e00aec10 0x7f84e00b3420 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e6d71640 1 --2- 172.21.15.60:0/259331178 >> v2:172.21.15.100:3300/0 conn(0x7f84e015db10 0x7f84e00b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e6d71640 1 -- 172.21.15.60:0/259331178 >> 172.21.15.60:0/259331178 conn(0x7f84e00670b0 msgr2=0x7f84e006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:26.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e6d71640 1 -- 172.21.15.60:0/259331178 shutdown_connections 2024-04-25T10:16:26.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e6d71640 1 -- 172.21.15.60:0/259331178 wait complete. 2024-04-25T10:16:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e6d71640 1 Processor -- start 2024-04-25T10:16:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e6d71640 1 -- start start 2024-04-25T10:16:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e6d71640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84e00aec10 0x7f84e00b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e6d71640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e00cfcf0 0x7f84e014fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e6d71640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84e015db10 0x7f84e015ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e6d71640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e006dcf0 con 0x7f84e015db10 2024-04-25T10:16:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e6d71640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e006db70 con 0x7f84e00cfcf0 2024-04-25T10:16:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e6d71640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e0071250 con 0x7f84e00aec10 2024-04-25T10:16:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e530e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84e015db10 0x7f84e015ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e530e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84e015db10 0x7f84e015ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60570/0 (socket says 172.21.15.60:60570) 2024-04-25T10:16:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e530e640 1 -- 172.21.15.60:0/1041611874 learned_addr learned my addr 172.21.15.60:0/1041611874 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e530e640 1 -- 172.21.15.60:0/1041611874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84e00aec10 msgr2=0x7f84e00b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e530e640 1 --2- 172.21.15.60:0/1041611874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84e00aec10 0x7f84e00b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e530e640 1 -- 172.21.15.60:0/1041611874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e00cfcf0 msgr2=0x7f84e014fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84dffff640 1 --2- 172.21.15.60:0/1041611874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e00cfcf0 0x7f84e014fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.982+0000 7f84e530e640 1 --2- 172.21.15.60:0/1041611874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e00cfcf0 0x7f84e014fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.986+0000 7f84e530e640 1 -- 172.21.15.60:0/1041611874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e014ebc0 con 0x7f84e015db10 2024-04-25T10:16:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.986+0000 7f84dffff640 1 --2- 172.21.15.60:0/1041611874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e00cfcf0 0x7f84e014fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.986+0000 7f84e530e640 1 --2- 172.21.15.60:0/1041611874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84e015db10 0x7f84e015ee10 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f84d800c740 tx=0x7f84d800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.986+0000 7f84dd7fa640 1 -- 172.21.15.60:0/1041611874 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8017070 con 0x7f84e015db10 2024-04-25T10:16:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.986+0000 7f84dd7fa640 1 -- 172.21.15.60:0/1041611874 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d800ce50 con 0x7f84e015db10 2024-04-25T10:16:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.986+0000 7f84e6d71640 1 -- 172.21.15.60:0/1041611874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e014eea0 con 0x7f84e015db10 2024-04-25T10:16:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.986+0000 7f84dd7fa640 1 -- 172.21.15.60:0/1041611874 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d800a860 con 0x7f84e015db10 2024-04-25T10:16:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.986+0000 7f84e6d71640 1 -- 172.21.15.60:0/1041611874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e014f460 con 0x7f84e015db10 2024-04-25T10:16:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.986+0000 7f84e6d71640 1 -- 172.21.15.60:0/1041611874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e006bba0 con 0x7f84e015db10 2024-04-25T10:16:26.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.986+0000 7f84dd7fa640 1 -- 172.21.15.60:0/1041611874 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f84d801a070 con 0x7f84e015db10 2024-04-25T10:16:26.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.986+0000 7f84dd7fa640 1 --2- 172.21.15.60:0/1041611874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84cc066f60 0x7f84cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.986+0000 7f84dd7fa640 1 -- 172.21.15.60:0/1041611874 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84d8084620 con 0x7f84e015db10 2024-04-25T10:16:26.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.986+0000 7f84e4b0d640 1 --2- 172.21.15.60:0/1041611874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84cc066f60 0x7f84cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.986+0000 7f84e4b0d640 1 --2- 172.21.15.60:0/1041611874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84cc066f60 0x7f84cc069420 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f84e015ca70 tx=0x7f84d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:26.990+0000 7f84dd7fa640 1 -- 172.21.15.60:0/1041611874 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f84e00ae140 con 0x7f84e015db10 2024-04-25T10:16:27.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.146+0000 7f84e6d71640 1 -- 172.21.15.60:0/1041611874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f84e006d560 con 0x7f84e015db10 2024-04-25T10:16:27.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.150+0000 7f84dd7fa640 1 -- 172.21.15.60:0/1041611874 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f84d8055620 con 0x7f84e015db10 2024-04-25T10:16:27.154 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:16:27.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.150+0000 7f84e6d71640 1 -- 172.21.15.60:0/1041611874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84cc066f60 msgr2=0x7f84cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.150+0000 7f84e6d71640 1 --2- 172.21.15.60:0/1041611874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84cc066f60 0x7f84cc069420 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f84e015ca70 tx=0x7f84d0008040 comp rx=0 tx=0).stop 2024-04-25T10:16:27.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.150+0000 7f84e6d71640 1 -- 172.21.15.60:0/1041611874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84e015db10 msgr2=0x7f84e015ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.150+0000 7f84e6d71640 1 --2- 172.21.15.60:0/1041611874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84e015db10 0x7f84e015ee10 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f84d800c740 tx=0x7f84d800cc00 comp rx=0 tx=0).stop 2024-04-25T10:16:27.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.150+0000 7f84e6d71640 1 -- 172.21.15.60:0/1041611874 shutdown_connections 2024-04-25T10:16:27.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.150+0000 7f84e6d71640 1 --2- 172.21.15.60:0/1041611874 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84cc066f60 0x7f84cc069420 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.150+0000 7f84e6d71640 1 --2- 172.21.15.60:0/1041611874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84e015db10 0x7f84e015ee10 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.150+0000 7f84e6d71640 1 --2- 172.21.15.60:0/1041611874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e00cfcf0 0x7f84e014fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.150+0000 7f84e6d71640 1 --2- 172.21.15.60:0/1041611874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84e00aec10 0x7f84e00b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.150+0000 7f84e6d71640 1 -- 172.21.15.60:0/1041611874 >> 172.21.15.60:0/1041611874 conn(0x7f84e00670b0 msgr2=0x7f84e0156220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:27.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.150+0000 7f84e6d71640 1 -- 172.21.15.60:0/1041611874 shutdown_connections 2024-04-25T10:16:27.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.150+0000 7f84e6d71640 1 -- 172.21.15.60:0/1041611874 wait complete. 2024-04-25T10:16:27.172 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:16:27.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f023c3fe640 1 Processor -- start 2024-04-25T10:16:27.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f023c3fe640 1 -- start start 2024-04-25T10:16:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f023c3fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f023406b320 0x7f02340af510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f023c3fe640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f023406b900 con 0x7f02340b3750 2024-04-25T10:16:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f023c3fe640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f023406bbe0 con 0x7f02340b3370 2024-04-25T10:16:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f023c3fe640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f023415ef20 con 0x7f023406b320 2024-04-25T10:16:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239999640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f02340b3750 0x7f02340aee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35120/0 (socket says 172.21.15.60:35120) 2024-04-25T10:16:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239999640 1 -- 172.21.15.60:0/2844320137 learned_addr learned my addr 172.21.15.60:0/2844320137 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f023a99b640 1 --2- 172.21.15.60:0/2844320137 >> v2:172.21.15.100:3300/0 conn(0x7f023406b320 0x7f02340af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239198640 1 -- 172.21.15.60:0/2844320137 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403231049 0 0) 0x7f023406b900 con 0x7f02340b3750 2024-04-25T10:16:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239198640 1 -- 172.21.15.60:0/2844320137 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0220003530 con 0x7f02340b3750 2024-04-25T10:16:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239198640 1 -- 172.21.15.60:0/2844320137 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3994057286 0 0) 0x7f0220003530 con 0x7f02340b3750 2024-04-25T10:16:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239198640 1 -- 172.21.15.60:0/2844320137 >> v2:172.21.15.100:3300/0 conn(0x7f023406b320 msgr2=0x7f02340af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239198640 1 --2- 172.21.15.60:0/2844320137 >> v2:172.21.15.100:3300/0 conn(0x7f023406b320 0x7f02340af510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239198640 1 -- 172.21.15.60:0/2844320137 >> v1:172.21.15.100:6789/0 conn(0x7f02340b3370 legacy=0x7f023406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239198640 1 -- 172.21.15.60:0/2844320137 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023406bec0 con 0x7f02340b3750 2024-04-25T10:16:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239198640 1 -- 172.21.15.60:0/2844320137 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f0224001680 con 0x7f02340b3750 2024-04-25T10:16:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239198640 1 -- 172.21.15.60:0/2844320137 >> v1:172.21.15.60:6789/0 conn(0x7f02340b3750 legacy=0x7f02340aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239198640 1 --2- 172.21.15.60:0/2844320137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02200039c0 0x7f0220003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239198640 1 --2- 172.21.15.60:0/2844320137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0220004810 0x7f0220004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239198640 1 --2- 172.21.15.60:0/2844320137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02200051d0 0x7f022000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239198640 1 -- 172.21.15.60:0/2844320137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022000c040 con 0x7f0220004810 2024-04-25T10:16:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239198640 1 -- 172.21.15.60:0/2844320137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022000c1f0 con 0x7f02200051d0 2024-04-25T10:16:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239198640 1 -- 172.21.15.60:0/2844320137 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022000c3f0 con 0x7f02200039c0 2024-04-25T10:16:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f0239999640 1 --2- 172.21.15.60:0/2844320137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0220004810 0x7f0220004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.294+0000 7f023a19a640 1 --2- 172.21.15.60:0/2844320137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02200039c0 0x7f0220003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f023a99b640 1 --2- 172.21.15.60:0/2844320137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02200051d0 0x7f022000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f0239999640 1 -- 172.21.15.60:0/2844320137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02200039c0 msgr2=0x7f0220003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f0239999640 1 --2- 172.21.15.60:0/2844320137 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02200039c0 0x7f0220003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f0239999640 1 -- 172.21.15.60:0/2844320137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02200051d0 msgr2=0x7f022000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f0239999640 1 --2- 172.21.15.60:0/2844320137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02200051d0 0x7f022000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f0239999640 1 -- 172.21.15.60:0/2844320137 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0220003530 con 0x7f0220004810 2024-04-25T10:16:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f023a99b640 1 -- 172.21.15.60:0/2844320137 reap_dead start 2024-04-25T10:16:27.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f0239999640 1 --2- 172.21.15.60:0/2844320137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0220004810 0x7f0220004c90 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f022400c1c0 tx=0x7f022401f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304142d2d80ffeca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:27.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f0239198640 1 -- 172.21.15.60:0/2844320137 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224007d00 con 0x7f0220004810 2024-04-25T10:16:27.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f0239198640 1 -- 172.21.15.60:0/2844320137 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0224005b00 con 0x7f0220004810 2024-04-25T10:16:27.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f0239198640 1 -- 172.21.15.60:0/2844320137 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224007070 con 0x7f0220004810 2024-04-25T10:16:27.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f023c3fe640 1 -- 172.21.15.60:0/2844320137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0220004810 msgr2=0x7f0220004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f023c3fe640 1 --2- 172.21.15.60:0/2844320137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0220004810 0x7f0220004c90 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f022400c1c0 tx=0x7f022401f390 comp rx=0 tx=0).stop 2024-04-25T10:16:27.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f023c3fe640 1 -- 172.21.15.60:0/2844320137 shutdown_connections 2024-04-25T10:16:27.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f023c3fe640 1 --2- 172.21.15.60:0/2844320137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0220004810 0x7f0220004c90 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f023c3fe640 1 -- 172.21.15.60:0/2844320137 >> 172.21.15.60:0/2844320137 conn(0x7f02340cfdd0 msgr2=0x7f02340d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:27.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f023c3fe640 1 -- 172.21.15.60:0/2844320137 shutdown_connections 2024-04-25T10:16:27.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.298+0000 7f023c3fe640 1 -- 172.21.15.60:0/2844320137 wait complete. 2024-04-25T10:16:27.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023c3fe640 1 Processor -- start 2024-04-25T10:16:27.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023c3fe640 1 -- start start 2024-04-25T10:16:27.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023c3fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0220004810 0x7f0234153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023c3fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f023415b400 0x7f023415fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023c3fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0234159b80 0x7f0234157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023c3fe640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023406ac10 con 0x7f0220004810 2024-04-25T10:16:27.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023c3fe640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023406c510 con 0x7f0234159b80 2024-04-25T10:16:27.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023c3fe640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023406ad90 con 0x7f023415b400 2024-04-25T10:16:27.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023a19a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0220004810 0x7f0234153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023a19a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0220004810 0x7f0234153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60592/0 (socket says 172.21.15.60:60592) 2024-04-25T10:16:27.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023a19a640 1 -- 172.21.15.60:0/3122486101 learned_addr learned my addr 172.21.15.60:0/3122486101 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:27.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023a99b640 1 --2- 172.21.15.60:0/3122486101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0234159b80 0x7f0234157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f0239999640 1 --2- 172.21.15.60:0/3122486101 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f023415b400 0x7f023415fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023a19a640 1 -- 172.21.15.60:0/3122486101 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f023415b400 msgr2=0x7f023415fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023a19a640 1 --2- 172.21.15.60:0/3122486101 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f023415b400 0x7f023415fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023a19a640 1 -- 172.21.15.60:0/3122486101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0234159b80 msgr2=0x7f0234157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023a19a640 1 --2- 172.21.15.60:0/3122486101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0234159b80 0x7f0234157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023a19a640 1 -- 172.21.15.60:0/3122486101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023415b860 con 0x7f0220004810 2024-04-25T10:16:27.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023a99b640 1 --2- 172.21.15.60:0/3122486101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0234159b80 0x7f0234157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:27.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023a19a640 1 --2- 172.21.15.60:0/3122486101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0220004810 0x7f0234153d30 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f022c00e950 tx=0x7f022c00ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:27.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f022affd640 1 -- 172.21.15.60:0/3122486101 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c003b90 con 0x7f0220004810 2024-04-25T10:16:27.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f022affd640 1 -- 172.21.15.60:0/3122486101 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022c018070 con 0x7f0220004810 2024-04-25T10:16:27.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f022affd640 1 -- 172.21.15.60:0/3122486101 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c0050d0 con 0x7f0220004810 2024-04-25T10:16:27.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023c3fe640 1 -- 172.21.15.60:0/3122486101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f023415a360 con 0x7f0220004810 2024-04-25T10:16:27.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023c3fe640 1 -- 172.21.15.60:0/3122486101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f023415a920 con 0x7f0220004810 2024-04-25T10:16:27.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f023c3fe640 1 -- 172.21.15.60:0/3122486101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0234074770 con 0x7f0220004810 2024-04-25T10:16:27.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.302+0000 7f022affd640 1 -- 172.21.15.60:0/3122486101 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f022c005270 con 0x7f0220004810 2024-04-25T10:16:27.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.306+0000 7f022affd640 1 --2- 172.21.15.60:0/3122486101 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0218066f90 0x7f0218069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.306+0000 7f022affd640 1 -- 172.21.15.60:0/3122486101 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f022c084f10 con 0x7f0220004810 2024-04-25T10:16:27.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.306+0000 7f0239999640 1 --2- 172.21.15.60:0/3122486101 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0218066f90 0x7f0218069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.306+0000 7f0239999640 1 --2- 172.21.15.60:0/3122486101 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0218066f90 0x7f0218069450 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f02240ae000 tx=0x7f0224007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:27.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.310+0000 7f022affd640 1 -- 172.21.15.60:0/3122486101 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f022c055fb0 con 0x7f0220004810 2024-04-25T10:16:27.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.466+0000 7f023c3fe640 1 -- 172.21.15.60:0/3122486101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f023415afe0 con 0x7f0220004810 2024-04-25T10:16:27.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.470+0000 7f022affd640 1 -- 172.21.15.60:0/3122486101 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f022c01be40 con 0x7f0220004810 2024-04-25T10:16:27.473 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:16:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.470+0000 7f023c3fe640 1 -- 172.21.15.60:0/3122486101 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0218066f90 msgr2=0x7f0218069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.470+0000 7f023c3fe640 1 --2- 172.21.15.60:0/3122486101 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0218066f90 0x7f0218069450 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f02240ae000 tx=0x7f0224007040 comp rx=0 tx=0).stop 2024-04-25T10:16:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.470+0000 7f023c3fe640 1 -- 172.21.15.60:0/3122486101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0220004810 msgr2=0x7f0234153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.470+0000 7f023c3fe640 1 --2- 172.21.15.60:0/3122486101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0220004810 0x7f0234153d30 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f022c00e950 tx=0x7f022c00ee10 comp rx=0 tx=0).stop 2024-04-25T10:16:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.470+0000 7f023c3fe640 1 -- 172.21.15.60:0/3122486101 shutdown_connections 2024-04-25T10:16:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.470+0000 7f023c3fe640 1 --2- 172.21.15.60:0/3122486101 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0218066f90 0x7f0218069450 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.470+0000 7f023c3fe640 1 --2- 172.21.15.60:0/3122486101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0234159b80 0x7f0234157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.470+0000 7f023c3fe640 1 --2- 172.21.15.60:0/3122486101 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f023415b400 0x7f023415fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.470+0000 7f023c3fe640 1 --2- 172.21.15.60:0/3122486101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0220004810 0x7f0234153d30 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.470+0000 7f023c3fe640 1 -- 172.21.15.60:0/3122486101 >> 172.21.15.60:0/3122486101 conn(0x7f02340cfdd0 msgr2=0x7f0234161470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.470+0000 7f023c3fe640 1 -- 172.21.15.60:0/3122486101 shutdown_connections 2024-04-25T10:16:27.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.470+0000 7f023c3fe640 1 -- 172.21.15.60:0/3122486101 wait complete. 2024-04-25T10:16:27.492 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:16:27.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.606+0000 7f143e54e640 1 Processor -- start 2024-04-25T10:16:27.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.606+0000 7f143e54e640 1 -- start start 2024-04-25T10:16:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.606+0000 7f143e54e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1438071290 0x7f14380716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.606+0000 7f143e54e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1438071c30 0x7f14380720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.606+0000 7f143e54e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f143806ef70 con 0x7f143806e480 2024-04-25T10:16:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.606+0000 7f143e54e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f143815dc10 con 0x7f1438071c30 2024-04-25T10:16:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.606+0000 7f143e54e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143815def0 con 0x7f1438071290 2024-04-25T10:16:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.606+0000 7f1437fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f143806e480 0x7f143806e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35124/0 (socket says 172.21.15.60:35124) 2024-04-25T10:16:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.606+0000 7f143caeb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1438071c30 0x7f14380720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.606+0000 7f1437fff640 1 -- 172.21.15.60:0/4036295075 learned_addr learned my addr 172.21.15.60:0/4036295075 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.606+0000 7f14377fe640 1 --2- 172.21.15.60:0/4036295075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1438071290 0x7f14380716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.606+0000 7f143caeb640 1 -- 172.21.15.60:0/4036295075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1438071290 msgr2=0x7f14380716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.606+0000 7f143caeb640 1 --2- 172.21.15.60:0/4036295075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1438071290 0x7f14380716f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.606+0000 7f143caeb640 1 -- 172.21.15.60:0/4036295075 >> v1:172.21.15.60:6789/0 conn(0x7f143806e480 legacy=0x7f143806e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.606+0000 7f143caeb640 1 -- 172.21.15.60:0/4036295075 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143815e1d0 con 0x7f1438071c30 2024-04-25T10:16:27.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.606+0000 7f14377fe640 1 --2- 172.21.15.60:0/4036295075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1438071290 0x7f14380716f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:27.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143caeb640 1 --2- 172.21.15.60:0/4036295075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1438071c30 0x7f14380720a0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f1428009d90 tx=0x7f142801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e0fe652b399c73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:27.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f1436ffd640 1 -- 172.21.15.60:0/4036295075 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14280b2070 con 0x7f1438071c30 2024-04-25T10:16:27.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f1436ffd640 1 -- 172.21.15.60:0/4036295075 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14280a5890 con 0x7f1438071c30 2024-04-25T10:16:27.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f1436ffd640 1 -- 172.21.15.60:0/4036295075 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14280a5c60 con 0x7f1438071c30 2024-04-25T10:16:27.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143e54e640 1 -- 172.21.15.60:0/4036295075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1438071c30 msgr2=0x7f14380720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143e54e640 1 --2- 172.21.15.60:0/4036295075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1438071c30 0x7f14380720a0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f1428009d90 tx=0x7f142801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143e54e640 1 -- 172.21.15.60:0/4036295075 shutdown_connections 2024-04-25T10:16:27.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143e54e640 1 --2- 172.21.15.60:0/4036295075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1438071c30 0x7f14380720a0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143e54e640 1 --2- 172.21.15.60:0/4036295075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1438071290 0x7f14380716f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143e54e640 1 -- 172.21.15.60:0/4036295075 >> 172.21.15.60:0/4036295075 conn(0x7f14380b12d0 msgr2=0x7f14380b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:27.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143e54e640 1 -- 172.21.15.60:0/4036295075 shutdown_connections 2024-04-25T10:16:27.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143e54e640 1 -- 172.21.15.60:0/4036295075 wait complete. 2024-04-25T10:16:27.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143e54e640 1 Processor -- start 2024-04-25T10:16:27.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143e54e640 1 -- start start 2024-04-25T10:16:27.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143e54e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f143806e480 0x7f143812f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143e54e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1438071290 0x7f143814ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143e54e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1438071c30 0x7f14381559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143e54e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14380ccfd0 con 0x7f143806e480 2024-04-25T10:16:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143e54e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14380cce50 con 0x7f1438071c30 2024-04-25T10:16:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f143e54e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14380cd150 con 0x7f1438071290 2024-04-25T10:16:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f1437fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f143806e480 0x7f143812f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f1437fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f143806e480 0x7f143812f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60608/0 (socket says 172.21.15.60:60608) 2024-04-25T10:16:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f1437fff640 1 -- 172.21.15.60:0/136467403 learned_addr learned my addr 172.21.15.60:0/136467403 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f1437fff640 1 -- 172.21.15.60:0/136467403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1438071290 msgr2=0x7f143814ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f1437fff640 1 --2- 172.21.15.60:0/136467403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1438071290 0x7f143814ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f1437fff640 1 -- 172.21.15.60:0/136467403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1438071c30 msgr2=0x7f14381559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f1437fff640 1 --2- 172.21.15.60:0/136467403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1438071c30 0x7f14381559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.610+0000 7f1437fff640 1 -- 172.21.15.60:0/136467403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1438128670 con 0x7f143806e480 2024-04-25T10:16:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.614+0000 7f1437fff640 1 --2- 172.21.15.60:0/136467403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f143806e480 0x7f143812f700 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f143000c450 tx=0x7f143000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.614+0000 7f1434ff9640 1 -- 172.21.15.60:0/136467403 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430017070 con 0x7f143806e480 2024-04-25T10:16:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.614+0000 7f1434ff9640 1 -- 172.21.15.60:0/136467403 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1430005cf0 con 0x7f143806e480 2024-04-25T10:16:27.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.614+0000 7f1434ff9640 1 -- 172.21.15.60:0/136467403 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430007270 con 0x7f143806e480 2024-04-25T10:16:27.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.614+0000 7f143e54e640 1 -- 172.21.15.60:0/136467403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1438128950 con 0x7f143806e480 2024-04-25T10:16:27.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.614+0000 7f143e54e640 1 -- 172.21.15.60:0/136467403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1438128f10 con 0x7f143806e480 2024-04-25T10:16:27.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.614+0000 7f143e54e640 1 -- 172.21.15.60:0/136467403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1438074770 con 0x7f143806e480 2024-04-25T10:16:27.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.614+0000 7f1434ff9640 1 -- 172.21.15.60:0/136467403 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f143001a070 con 0x7f143806e480 2024-04-25T10:16:27.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.614+0000 7f1434ff9640 1 --2- 172.21.15.60:0/136467403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1418066f60 0x7f1418069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.614+0000 7f1434ff9640 1 -- 172.21.15.60:0/136467403 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1430084200 con 0x7f143806e480 2024-04-25T10:16:27.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.614+0000 7f14377fe640 1 --2- 172.21.15.60:0/136467403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1418066f60 0x7f1418069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.614+0000 7f14377fe640 1 --2- 172.21.15.60:0/136467403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1418066f60 0x7f1418069420 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f14380b1eb0 tx=0x7f1420008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:27.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.618+0000 7f1434ff9640 1 -- 172.21.15.60:0/136467403 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1430054bc0 con 0x7f143806e480 2024-04-25T10:16:27.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.782+0000 7f143e54e640 1 -- 172.21.15.60:0/136467403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1438143030 con 0x7f143806e480 2024-04-25T10:16:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.782+0000 7f1434ff9640 1 -- 172.21.15.60:0/136467403 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1430058c60 con 0x7f143806e480 2024-04-25T10:16:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T10:16:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.786+0000 7f143e54e640 1 -- 172.21.15.60:0/136467403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1418066f60 msgr2=0x7f1418069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.786+0000 7f143e54e640 1 --2- 172.21.15.60:0/136467403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1418066f60 0x7f1418069420 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f14380b1eb0 tx=0x7f1420008040 comp rx=0 tx=0).stop 2024-04-25T10:16:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.786+0000 7f143e54e640 1 -- 172.21.15.60:0/136467403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f143806e480 msgr2=0x7f143812f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.786+0000 7f143e54e640 1 --2- 172.21.15.60:0/136467403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f143806e480 0x7f143812f700 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f143000c450 tx=0x7f143000c910 comp rx=0 tx=0).stop 2024-04-25T10:16:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.786+0000 7f143e54e640 1 -- 172.21.15.60:0/136467403 shutdown_connections 2024-04-25T10:16:27.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.786+0000 7f143e54e640 1 --2- 172.21.15.60:0/136467403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1418066f60 0x7f1418069420 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.786+0000 7f143e54e640 1 --2- 172.21.15.60:0/136467403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1438071c30 0x7f14381559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.786+0000 7f143e54e640 1 --2- 172.21.15.60:0/136467403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1438071290 0x7f143814ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.786+0000 7f143e54e640 1 --2- 172.21.15.60:0/136467403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f143806e480 0x7f143812f700 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.786+0000 7f143e54e640 1 -- 172.21.15.60:0/136467403 >> 172.21.15.60:0/136467403 conn(0x7f14380b12d0 msgr2=0x7f1438159790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:27.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.786+0000 7f143e54e640 1 -- 172.21.15.60:0/136467403 shutdown_connections 2024-04-25T10:16:27.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:27.786+0000 7f143e54e640 1 -- 172.21.15.60:0/136467403 wait complete. 2024-04-25T10:16:27.807 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:16:27.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:27.807 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:27.818 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[52106]: fuse finished with error 0 and tester_r 0 2024-04-25T10:16:27.820 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:27.884 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:16:27.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:27.885 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:16:27.889 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:16:27.889 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:27.937 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:16:27.937 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi060.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:27.938 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:16:27.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:27.938 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:16:27.985 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:16:27.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:16:27.987 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:16:28.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4915c640 1 Processor -- start 2024-04-25T10:16:28.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4915c640 1 -- start start 2024-04-25T10:16:28.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4915c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9e4415a7a0 0x7f9e440cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4915c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4406f130 con 0x7f9e4406ec30 2024-04-25T10:16:28.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4915c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4406f410 con 0x7f9e440b37e0 2024-04-25T10:16:28.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4915c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e440cdaa0 con 0x7f9e4415a7a0 2024-04-25T10:16:28.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e3bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9e4406ec30 0x7f9e44071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35138/0 (socket says 172.21.15.60:35138) 2024-04-25T10:16:28.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e3bfff640 1 -- 172.21.15.60:0/3285631041 learned_addr learned my addr 172.21.15.60:0/3285631041 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:28.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e42d9d640 1 --2- 172.21.15.60:0/3285631041 >> v2:172.21.15.100:3300/0 conn(0x7f9e4415a7a0 0x7f9e440cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4259c640 1 -- 172.21.15.60:0/3285631041 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035592379 0 0) 0x7f9e4406f130 con 0x7f9e4406ec30 2024-04-25T10:16:28.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4259c640 1 -- 172.21.15.60:0/3285631041 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e30003530 con 0x7f9e4406ec30 2024-04-25T10:16:28.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4259c640 1 -- 172.21.15.60:0/3285631041 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 285177851 0 0) 0x7f9e30003530 con 0x7f9e4406ec30 2024-04-25T10:16:28.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4259c640 1 -- 172.21.15.60:0/3285631041 >> v2:172.21.15.100:3300/0 conn(0x7f9e4415a7a0 msgr2=0x7f9e440cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:28.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4259c640 1 --2- 172.21.15.60:0/3285631041 >> v2:172.21.15.100:3300/0 conn(0x7f9e4415a7a0 0x7f9e440cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4259c640 1 -- 172.21.15.60:0/3285631041 >> v1:172.21.15.100:6789/0 conn(0x7f9e440b37e0 legacy=0x7f9e4406dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4259c640 1 -- 172.21.15.60:0/3285631041 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4406d540 con 0x7f9e4406ec30 2024-04-25T10:16:28.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4259c640 1 -- 172.21.15.60:0/3285631041 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f9e2c001680 con 0x7f9e4406ec30 2024-04-25T10:16:28.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4259c640 1 -- 172.21.15.60:0/3285631041 >> v1:172.21.15.60:6789/0 conn(0x7f9e4406ec30 legacy=0x7f9e44071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4259c640 1 --2- 172.21.15.60:0/3285631041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e300039c0 0x7f9e30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4259c640 1 --2- 172.21.15.60:0/3285631041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e30004810 0x7f9e30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4259c640 1 --2- 172.21.15.60:0/3285631041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e300051d0 0x7f9e3000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4259c640 1 -- 172.21.15.60:0/3285631041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3000c040 con 0x7f9e300051d0 2024-04-25T10:16:28.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4259c640 1 -- 172.21.15.60:0/3285631041 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3000c1f0 con 0x7f9e30004810 2024-04-25T10:16:28.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.150+0000 7f9e4259c640 1 -- 172.21.15.60:0/3285631041 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3000c3f0 con 0x7f9e300039c0 2024-04-25T10:16:28.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4359e640 1 --2- 172.21.15.60:0/3285631041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e300051d0 0x7f9e3000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e42d9d640 1 --2- 172.21.15.60:0/3285631041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e300039c0 0x7f9e30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e3bfff640 1 --2- 172.21.15.60:0/3285631041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e30004810 0x7f9e30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4359e640 1 -- 172.21.15.60:0/3285631041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e300039c0 msgr2=0x7f9e30003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:28.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4359e640 1 --2- 172.21.15.60:0/3285631041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e300039c0 0x7f9e30003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4359e640 1 -- 172.21.15.60:0/3285631041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e30004810 msgr2=0x7f9e30004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:28.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4359e640 1 --2- 172.21.15.60:0/3285631041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e30004810 0x7f9e30004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4359e640 1 -- 172.21.15.60:0/3285631041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e30003530 con 0x7f9e300051d0 2024-04-25T10:16:28.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e42d9d640 1 --2- 172.21.15.60:0/3285631041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e300039c0 0x7f9e30003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:28.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e3bfff640 1 --2- 172.21.15.60:0/3285631041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e30004810 0x7f9e30004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:28.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4359e640 1 -- 172.21.15.60:0/3285631041 reap_dead start 2024-04-25T10:16:28.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4359e640 1 --2- 172.21.15.60:0/3285631041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e300051d0 0x7f9e3000b9d0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c001e70 tx=0x7f9e3c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256cb66472c2d03d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:28.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4259c640 1 -- 172.21.15.60:0/3285631041 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c001a90 con 0x7f9e300051d0 2024-04-25T10:16:28.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4259c640 1 -- 172.21.15.60:0/3285631041 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c003bc0 con 0x7f9e300051d0 2024-04-25T10:16:28.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4259c640 1 -- 172.21.15.60:0/3285631041 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c005020 con 0x7f9e300051d0 2024-04-25T10:16:28.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4915c640 1 -- 172.21.15.60:0/3285631041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e300051d0 msgr2=0x7f9e3000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4915c640 1 --2- 172.21.15.60:0/3285631041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e300051d0 0x7f9e3000b9d0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c001e70 tx=0x7f9e3c01c780 comp rx=0 tx=0).stop 2024-04-25T10:16:28.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4915c640 1 -- 172.21.15.60:0/3285631041 shutdown_connections 2024-04-25T10:16:28.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4915c640 1 --2- 172.21.15.60:0/3285631041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e300051d0 0x7f9e3000b9d0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4915c640 1 -- 172.21.15.60:0/3285631041 >> 172.21.15.60:0/3285631041 conn(0x7f9e440b12d0 msgr2=0x7f9e440b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:28.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4915c640 1 -- 172.21.15.60:0/3285631041 shutdown_connections 2024-04-25T10:16:28.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.154+0000 7f9e4915c640 1 -- 172.21.15.60:0/3285631041 wait complete. 2024-04-25T10:16:28.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4915c640 1 Processor -- start 2024-04-25T10:16:28.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4915c640 1 -- start start 2024-04-25T10:16:28.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4915c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e300051d0 0x7f9e4414a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4915c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e4415b010 0x7f9e4414d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4915c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e4415c650 0x7f9e440ce710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4915c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4406d8b0 con 0x7f9e4415c650 2024-04-25T10:16:28.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4915c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4406d2f0 con 0x7f9e300051d0 2024-04-25T10:16:28.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4915c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440cd560 con 0x7f9e4415b010 2024-04-25T10:16:28.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e4415c650 0x7f9e440ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e4415c650 0x7f9e440ce710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60618/0 (socket says 172.21.15.60:60618) 2024-04-25T10:16:28.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4359e640 1 -- 172.21.15.60:0/2800243193 learned_addr learned my addr 172.21.15.60:0/2800243193 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:28.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e3bfff640 1 --2- 172.21.15.60:0/2800243193 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e4415b010 0x7f9e4414d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e42d9d640 1 --2- 172.21.15.60:0/2800243193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e300051d0 0x7f9e4414a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4359e640 1 -- 172.21.15.60:0/2800243193 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e4415b010 msgr2=0x7f9e4414d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4359e640 1 --2- 172.21.15.60:0/2800243193 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e4415b010 0x7f9e4414d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4359e640 1 -- 172.21.15.60:0/2800243193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e300051d0 msgr2=0x7f9e4414a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4359e640 1 --2- 172.21.15.60:0/2800243193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e300051d0 0x7f9e4414a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4359e640 1 -- 172.21.15.60:0/2800243193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4415b5b0 con 0x7f9e4415c650 2024-04-25T10:16:28.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4359e640 1 --2- 172.21.15.60:0/2800243193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e4415c650 0x7f9e440ce710 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c01c5b0 tx=0x7f9e30005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:28.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e3b7fe640 1 -- 172.21.15.60:0/2800243193 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c001a90 con 0x7f9e4415c650 2024-04-25T10:16:28.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e3b7fe640 1 -- 172.21.15.60:0/2800243193 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c001260 con 0x7f9e4415c650 2024-04-25T10:16:28.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4915c640 1 -- 172.21.15.60:0/2800243193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4415b890 con 0x7f9e4415c650 2024-04-25T10:16:28.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e3b7fe640 1 -- 172.21.15.60:0/2800243193 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c005800 con 0x7f9e4415c650 2024-04-25T10:16:28.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4915c640 1 -- 172.21.15.60:0/2800243193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4415be50 con 0x7f9e4415c650 2024-04-25T10:16:28.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e4915c640 1 -- 172.21.15.60:0/2800243193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e44074770 con 0x7f9e4415c650 2024-04-25T10:16:28.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.158+0000 7f9e3b7fe640 1 -- 172.21.15.60:0/2800243193 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e3c0a6480 con 0x7f9e4415c650 2024-04-25T10:16:28.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.162+0000 7f9e3b7fe640 1 --2- 172.21.15.60:0/2800243193 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e24066f90 0x7f9e24069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.162+0000 7f9e3b7fe640 1 -- 172.21.15.60:0/2800243193 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e3c11e530 con 0x7f9e4415c650 2024-04-25T10:16:28.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.162+0000 7f9e42d9d640 1 --2- 172.21.15.60:0/2800243193 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e24066f90 0x7f9e24069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.162+0000 7f9e42d9d640 1 --2- 172.21.15.60:0/2800243193 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e24066f90 0x7f9e24069450 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009ac0 tx=0x7f9e34008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:28.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.166+0000 7f9e3b7fe640 1 -- 172.21.15.60:0/2800243193 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e3c0b8060 con 0x7f9e4415c650 2024-04-25T10:16:28.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.350+0000 7f9e4915c640 1 -- 172.21.15.60:0/2800243193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9e44155790 con 0x7f9e4415c650 2024-04-25T10:16:28.357 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:16:28.354+0000 7f3a792d6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:16:28.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.354+0000 7f9e3b7fe640 1 -- 172.21.15.60:0/2800243193 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v94) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9e3c016490 con 0x7f9e4415c650 2024-04-25T10:16:28.360 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:16:28.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.358+0000 7f9e4915c640 1 -- 172.21.15.60:0/2800243193 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e24066f90 msgr2=0x7f9e24069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.358+0000 7f9e4915c640 1 --2- 172.21.15.60:0/2800243193 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e24066f90 0x7f9e24069450 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009ac0 tx=0x7f9e34008040 comp rx=0 tx=0).stop 2024-04-25T10:16:28.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.358+0000 7f9e4915c640 1 -- 172.21.15.60:0/2800243193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e4415c650 msgr2=0x7f9e440ce710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.358+0000 7f9e4915c640 1 --2- 172.21.15.60:0/2800243193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e4415c650 0x7f9e440ce710 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c01c5b0 tx=0x7f9e30005840 comp rx=0 tx=0).stop 2024-04-25T10:16:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.358+0000 7f9e4915c640 1 -- 172.21.15.60:0/2800243193 shutdown_connections 2024-04-25T10:16:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.358+0000 7f9e4915c640 1 --2- 172.21.15.60:0/2800243193 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9e24066f90 0x7f9e24069450 secure :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009ac0 tx=0x7f9e34008040 comp rx=0 tx=0).stop 2024-04-25T10:16:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.358+0000 7f9e4915c640 1 --2- 172.21.15.60:0/2800243193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e4415c650 0x7f9e440ce710 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.358+0000 7f9e4915c640 1 --2- 172.21.15.60:0/2800243193 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e4415b010 0x7f9e4414d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.358+0000 7f9e4915c640 1 --2- 172.21.15.60:0/2800243193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e300051d0 0x7f9e4414a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.358+0000 7f9e4915c640 1 -- 172.21.15.60:0/2800243193 >> 172.21.15.60:0/2800243193 conn(0x7f9e440b12d0 msgr2=0x7f9e44155080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:28.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.358+0000 7f9e4915c640 1 -- 172.21.15.60:0/2800243193 shutdown_connections 2024-04-25T10:16:28.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.358+0000 7f9e4915c640 1 -- 172.21.15.60:0/2800243193 wait complete. 2024-04-25T10:16:28.375 INFO:tasks.ceph.mds.c.smithi060.stderr: -7353> 2024-04-25T10:16:14.014+0000 7efda3e5b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:16:28.381 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:16:28.383 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:16:28.406 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2024-04-25T10:16:28.407 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2024-04-25T10:16:28.407 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:16:28.407 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2024-04-25T10:16:28.407 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.407 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2024-04-25T10:16:28.407 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.407 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2024-04-25T10:16:28.407 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.407 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2024-04-25T10:16:28.407 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2024-04-25T10:16:28.407 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.407 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2024-04-25T10:16:28.407 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2024-04-25T10:16:28.407 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.407 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:16:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2024-04-25T10:16:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:16:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:16:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2024-04-25T10:16:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2024-04-25T10:16:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2024-04-25T10:16:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:16:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:16:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2024-04-25T10:16:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:16:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2024-04-25T10:16:28.408 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:16:28.409 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:16:28.409 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2024-04-25T10:16:28.409 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.409 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:16:28.409 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.409 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2024-04-25T10:16:28.409 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.409 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2024-04-25T10:16:28.409 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.409 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2024-04-25T10:16:28.409 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2024-04-25T10:16:28.410 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2024-04-25T10:16:28.410 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = "172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2024-04-25T10:16:28.410 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = 0ea93863-a079-45c4-abed-6e83540ea9fe 2024-04-25T10:16:28.410 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.410 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2024-04-25T10:16:28.410 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2024-04-25T10:16:28.410 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.410 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2024-04-25T10:16:28.410 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2024-04-25T10:16:28.410 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:16:28.410 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.410 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2024-04-25T10:16:28.411 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2024-04-25T10:16:28.411 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.411 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2024-04-25T10:16:28.411 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2024-04-25T10:16:28.411 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2024-04-25T10:16:28.411 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug trim objects = true 2024-04-25T10:16:28.411 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.411 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2024-04-25T10:16:28.411 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2024-04-25T10:16:28.411 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.411 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:16:28.411 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.411 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2024-04-25T10:16:28.411 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.411 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2024-04-25T10:16:28.412 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.412 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:16:28.412 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2024-04-25T10:16:28.412 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.412 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2024-04-25T10:16:28.412 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2024-04-25T10:16:28.412 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2024-04-25T10:16:28.412 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2024-04-25T10:16:28.412 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2024-04-25T10:16:28.412 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2024-04-25T10:16:28.412 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2024-04-25T10:16:28.412 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2024-04-25T10:16:28.413 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2024-04-25T10:16:28.413 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:28.413 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2024-04-25T10:16:28.413 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2024-04-25T10:16:28.413 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:16:28.413 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2024-04-25T10:16:28.413 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:16:28.413 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:16:28.413 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2024-04-25T10:16:28.413 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2024-04-25T10:16:28.413 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.413 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2024-04-25T10:16:28.413 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:28.414 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2024-04-25T10:16:28.414 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-04-25T10:16:28.414 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-04-25T10:16:28.414 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:28.414 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:28.414 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2024-04-25T10:16:28.414 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.414 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2024-04-25T10:16:28.414 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:28.414 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-04-25T10:16:28.414 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2024-04-25T10:16:28.414 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-04-25T10:16:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2024-04-25T10:16:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2024-04-25T10:16:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2024-04-25T10:16:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2024-04-25T10:16:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:16:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:16:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:16:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:16:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:16:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:16:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2024-04-25T10:16:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:16:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:16:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: mon op complaint time = 120 2024-04-25T10:16:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.416 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2024-04-25T10:16:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2024-04-25T10:16:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2024-04-25T10:16:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2024-04-25T10:16:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:16:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:16:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: client mount timeout = 600 2024-04-25T10:16:28.417 INFO:teuthology.orchestra.run.smithi060.stdout: debug client = 20 2024-04-25T10:16:28.417 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:28.417 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = "" 2024-04-25T10:16:28.417 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2024-04-25T10:16:28.417 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2024-04-25T10:16:28.417 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2024-04-25T10:16:28.417 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.b] 2024-04-25T10:16:28.417 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.c] 2024-04-25T10:16:28.417 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2024-04-25T10:16:28.417 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2024-04-25T10:16:28.417 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2024-04-25T10:16:28.417 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2024-04-25T10:16:28.417 INFO:teuthology.orchestra.run.smithi060.stdout:[mds] 2024-04-25T10:16:28.417 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds = 20 2024-04-25T10:16:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds balancer = 20 2024-04-25T10:16:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug frag = True 2024-04-25T10:16:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug scatterstat = True 2024-04-25T10:16:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: mds op complaint time = 180 2024-04-25T10:16:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: mds verify scatter = False 2024-04-25T10:16:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2024-04-25T10:16:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2024-04-25T10:16:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2024-04-25T10:16:28.419 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-04-25T10:16:28.419 INFO:teuthology.orchestra.run.smithi100.stdout: chdir = "" 2024-04-25T10:16:28.419 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:16:28.419 INFO:teuthology.orchestra.run.smithi100.stdout: auth supported = cephx 2024-04-25T10:16:28.419 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.419 INFO:teuthology.orchestra.run.smithi100.stdout: filestore xattr use omap = true 2024-04-25T10:16:28.419 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.419 INFO:teuthology.orchestra.run.smithi100.stdout: mon clock drift allowed = 1.000 2024-04-25T10:16:28.419 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.420 INFO:teuthology.orchestra.run.smithi100.stdout: osd crush chooseleaf type = 0 2024-04-25T10:16:28.420 INFO:teuthology.orchestra.run.smithi100.stdout: auth debug = true 2024-04-25T10:16:28.420 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.420 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on old message = true 2024-04-25T10:16:28.420 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on bug = true 2024-04-25T10:16:28.420 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.420 INFO:teuthology.orchestra.run.smithi100.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:16:28.420 INFO:teuthology.orchestra.run.smithi100.stdout: mon pg warn max object skew = 0 2024-04-25T10:16:28.420 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.420 INFO:teuthology.orchestra.run.smithi100.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:16:28.420 INFO:teuthology.orchestra.run.smithi100.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:16:28.420 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.420 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default size = 2 2024-04-25T10:16:28.420 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.420 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow primary affinity = true 2024-04-25T10:16:28.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow pg remap = true 2024-04-25T10:16:28.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:16:28.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:16:28.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on no sortbitwise = false 2024-04-25T10:16:28.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:16:28.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on too few osds = false 2024-04-25T10:16:28.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:16:28.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:16:28.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon_allow_pool_size_one = true 2024-04-25T10:16:28.421 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.421 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:16:28.421 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.421 INFO:teuthology.orchestra.run.smithi100.stdout: osd default data pool replay window = 5 2024-04-25T10:16:28.421 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.421 INFO:teuthology.orchestra.run.smithi100.stdout: mon allow pool delete = true 2024-04-25T10:16:28.422 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.422 INFO:teuthology.orchestra.run.smithi100.stdout: mon cluster log file level = debug 2024-04-25T10:16:28.422 INFO:teuthology.orchestra.run.smithi100.stdout: debug asserts on shutdown = true 2024-04-25T10:16:28.422 INFO:teuthology.orchestra.run.smithi100.stdout: mon health detail to clog = false 2024-04-25T10:16:28.422 INFO:teuthology.orchestra.run.smithi100.stdout: mon host = "172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2024-04-25T10:16:28.422 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = 0ea93863-a079-45c4-abed-6e83540ea9fe 2024-04-25T10:16:28.422 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.422 INFO:teuthology.orchestra.run.smithi100.stdout:[osd] 2024-04-25T10:16:28.422 INFO:teuthology.orchestra.run.smithi100.stdout: osd journal size = 100 2024-04-25T10:16:28.422 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.422 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub load threshold = 5.0 2024-04-25T10:16:28.422 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub max interval = 600 2024-04-25T10:16:28.422 INFO:teuthology.orchestra.run.smithi100.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:16:28.422 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd recover clone overlap = true 2024-04-25T10:16:28.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd recovery max chunk = 1048576 2024-04-25T10:16:28.423 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug shutdown = true 2024-04-25T10:16:28.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug op order = true 2024-04-25T10:16:28.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify stray on activate = true 2024-04-25T10:16:28.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug trim objects = true 2024-04-25T10:16:28.423 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd open classes on start = true 2024-04-25T10:16:28.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug pg log writeout = true 2024-04-25T10:16:28.423 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:16:28.423 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.423 INFO:teuthology.orchestra.run.smithi100.stdout: osd map max advance = 10 2024-04-25T10:16:28.423 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.424 INFO:teuthology.orchestra.run.smithi100.stdout: journal zero on create = true 2024-04-25T10:16:28.424 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.424 INFO:teuthology.orchestra.run.smithi100.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:16:28.424 INFO:teuthology.orchestra.run.smithi100.stdout: filestore apply finisher threads = 3 2024-04-25T10:16:28.424 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.424 INFO:teuthology.orchestra.run.smithi100.stdout: bdev debug aio = true 2024-04-25T10:16:28.424 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug misdirected ops = true 2024-04-25T10:16:28.424 INFO:teuthology.orchestra.run.smithi100.stdout: bdev async discard = True 2024-04-25T10:16:28.424 INFO:teuthology.orchestra.run.smithi100.stdout: bdev enable discard = True 2024-04-25T10:16:28.424 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore allocator = bitmap 2024-04-25T10:16:28.424 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore block size = 96636764160 2024-04-25T10:16:28.424 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore fsck on mount = True 2024-04-25T10:16:28.424 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluefs = 1/20 2024-04-25T10:16:28.424 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluestore = 1/20 2024-04-25T10:16:28.424 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:28.424 INFO:teuthology.orchestra.run.smithi100.stdout: debug osd = 20 2024-04-25T10:16:28.425 INFO:teuthology.orchestra.run.smithi100.stdout: debug rocksdb = 4/10 2024-04-25T10:16:28.425 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:16:28.425 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd full ratio = 0.9 2024-04-25T10:16:28.425 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:16:28.425 INFO:teuthology.orchestra.run.smithi100.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:16:28.425 INFO:teuthology.orchestra.run.smithi100.stdout: osd objectstore = bluestore 2024-04-25T10:16:28.425 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-04-25T10:16:28.425 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.425 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr] 2024-04-25T10:16:28.425 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:28.425 INFO:teuthology.orchestra.run.smithi100.stdout: debug mgr = 20 2024-04-25T10:16:28.425 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-04-25T10:16:28.425 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-04-25T10:16:28.425 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:28.426 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:28.426 INFO:teuthology.orchestra.run.smithi100.stdout: mgr/telemetry/nag = false 2024-04-25T10:16:28.426 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.426 INFO:teuthology.orchestra.run.smithi100.stdout:[mon] 2024-04-25T10:16:28.426 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:28.426 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-04-25T10:16:28.426 INFO:teuthology.orchestra.run.smithi100.stdout: debug paxos = 20 2024-04-25T10:16:28.426 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-04-25T10:16:28.426 INFO:teuthology.orchestra.run.smithi100.stdout: mon data avail warn = 5 2024-04-25T10:16:28.426 INFO:teuthology.orchestra.run.smithi100.stdout: mon mgr mkfs grace = 240 2024-04-25T10:16:28.426 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:28.426 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd reporter subtree level = osd 2024-04-25T10:16:28.426 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd prime pg temp = true 2024-04-25T10:16:28.426 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:28.426 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.427 INFO:teuthology.orchestra.run.smithi100.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:16:28.427 INFO:teuthology.orchestra.run.smithi100.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:16:28.427 INFO:teuthology.orchestra.run.smithi100.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:16:28.427 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.427 INFO:teuthology.orchestra.run.smithi100.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:16:28.427 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:16:28.427 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:16:28.427 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.427 INFO:teuthology.orchestra.run.smithi100.stdout: # 1m isn't quite enough 2024-04-25T10:16:28.427 INFO:teuthology.orchestra.run.smithi100.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:16:28.427 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.427 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:16:28.427 INFO:teuthology.orchestra.run.smithi100.stdout: mon op complaint time = 120 2024-04-25T10:16:28.427 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.427 INFO:teuthology.orchestra.run.smithi100.stdout:[client] 2024-04-25T10:16:28.428 INFO:teuthology.orchestra.run.smithi100.stdout: rgw cache enabled = true 2024-04-25T10:16:28.428 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable ops log = true 2024-04-25T10:16:28.428 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable usage log = true 2024-04-25T10:16:28.428 INFO:teuthology.orchestra.run.smithi100.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:16:28.428 INFO:teuthology.orchestra.run.smithi100.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:16:28.428 INFO:teuthology.orchestra.run.smithi100.stdout: client mount timeout = 600 2024-04-25T10:16:28.428 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2024-04-25T10:16:28.428 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:28.428 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = "" 2024-04-25T10:16:28.428 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-04-25T10:16:28.428 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-04-25T10:16:28.428 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.a] 2024-04-25T10:16:28.428 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.b] 2024-04-25T10:16:28.428 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.c] 2024-04-25T10:16:28.429 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-04-25T10:16:28.429 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-04-25T10:16:28.429 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-04-25T10:16:28.429 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-04-25T10:16:28.429 INFO:teuthology.orchestra.run.smithi100.stdout:[mds] 2024-04-25T10:16:28.429 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds = 20 2024-04-25T10:16:28.429 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds balancer = 20 2024-04-25T10:16:28.429 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:28.429 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug frag = True 2024-04-25T10:16:28.429 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug scatterstat = True 2024-04-25T10:16:28.429 INFO:teuthology.orchestra.run.smithi100.stdout: mds op complaint time = 180 2024-04-25T10:16:28.429 INFO:teuthology.orchestra.run.smithi100.stdout: mds verify scatter = False 2024-04-25T10:16:28.429 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-04-25T10:16:28.429 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-04-25T10:16:28.429 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-04-25T10:16:28.430 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:16:28.432 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:16:28.474 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:16:28.493 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2024-04-25T10:16:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2024-04-25T10:16:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:16:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2024-04-25T10:16:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2024-04-25T10:16:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2024-04-25T10:16:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2024-04-25T10:16:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2024-04-25T10:16:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2024-04-25T10:16:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2024-04-25T10:16:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:16:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2024-04-25T10:16:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:16:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:16:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2024-04-25T10:16:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2024-04-25T10:16:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2024-04-25T10:16:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:16:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:16:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2024-04-25T10:16:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:16:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2024-04-25T10:16:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:16:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:16:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2024-04-25T10:16:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:16:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2024-04-25T10:16:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2024-04-25T10:16:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2024-04-25T10:16:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2024-04-25T10:16:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2024-04-25T10:16:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = "172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2024-04-25T10:16:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = 0ea93863-a079-45c4-abed-6e83540ea9fe 2024-04-25T10:16:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.498 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2024-04-25T10:16:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2024-04-25T10:16:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2024-04-25T10:16:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2024-04-25T10:16:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:16:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2024-04-25T10:16:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2024-04-25T10:16:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2024-04-25T10:16:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2024-04-25T10:16:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2024-04-25T10:16:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug trim objects = true 2024-04-25T10:16:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2024-04-25T10:16:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2024-04-25T10:16:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:16:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2024-04-25T10:16:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2024-04-25T10:16:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:16:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2024-04-25T10:16:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2024-04-25T10:16:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2024-04-25T10:16:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2024-04-25T10:16:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2024-04-25T10:16:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2024-04-25T10:16:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2024-04-25T10:16:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2024-04-25T10:16:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2024-04-25T10:16:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2024-04-25T10:16:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2024-04-25T10:16:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2024-04-25T10:16:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:16:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2024-04-25T10:16:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:16:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:16:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2024-04-25T10:16:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2024-04-25T10:16:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.502 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2024-04-25T10:16:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2024-04-25T10:16:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-04-25T10:16:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-04-25T10:16:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2024-04-25T10:16:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.502 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2024-04-25T10:16:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-04-25T10:16:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2024-04-25T10:16:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-04-25T10:16:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2024-04-25T10:16:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2024-04-25T10:16:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2024-04-25T10:16:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2024-04-25T10:16:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:16:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:16:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:16:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:16:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:16:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:16:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2024-04-25T10:16:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:16:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:16:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: mon op complaint time = 120 2024-04-25T10:16:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:28.504 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2024-04-25T10:16:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2024-04-25T10:16:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2024-04-25T10:16:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2024-04-25T10:16:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:16:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:16:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: client mount timeout = 600 2024-04-25T10:16:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: debug client = 20 2024-04-25T10:16:28.505 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:28.505 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = "" 2024-04-25T10:16:28.505 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2024-04-25T10:16:28.505 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2024-04-25T10:16:28.505 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2024-04-25T10:16:28.505 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.b] 2024-04-25T10:16:28.505 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.c] 2024-04-25T10:16:28.505 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2024-04-25T10:16:28.505 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2024-04-25T10:16:28.505 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2024-04-25T10:16:28.505 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2024-04-25T10:16:28.506 INFO:teuthology.orchestra.run.smithi060.stdout:[mds] 2024-04-25T10:16:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds = 20 2024-04-25T10:16:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds balancer = 20 2024-04-25T10:16:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug frag = True 2024-04-25T10:16:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug scatterstat = False 2024-04-25T10:16:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: mds op complaint time = 180 2024-04-25T10:16:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: mds verify scatter = False 2024-04-25T10:16:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2024-04-25T10:16:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2024-04-25T10:16:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2024-04-25T10:16:28.507 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-04-25T10:16:28.507 INFO:teuthology.orchestra.run.smithi100.stdout: chdir = "" 2024-04-25T10:16:28.507 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:16:28.507 INFO:teuthology.orchestra.run.smithi100.stdout: auth supported = cephx 2024-04-25T10:16:28.507 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.508 INFO:teuthology.orchestra.run.smithi100.stdout: filestore xattr use omap = true 2024-04-25T10:16:28.508 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.508 INFO:teuthology.orchestra.run.smithi100.stdout: mon clock drift allowed = 1.000 2024-04-25T10:16:28.508 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.508 INFO:teuthology.orchestra.run.smithi100.stdout: osd crush chooseleaf type = 0 2024-04-25T10:16:28.508 INFO:teuthology.orchestra.run.smithi100.stdout: auth debug = true 2024-04-25T10:16:28.508 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.508 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on old message = true 2024-04-25T10:16:28.508 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on bug = true 2024-04-25T10:16:28.508 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.508 INFO:teuthology.orchestra.run.smithi100.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:16:28.508 INFO:teuthology.orchestra.run.smithi100.stdout: mon pg warn max object skew = 0 2024-04-25T10:16:28.508 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.508 INFO:teuthology.orchestra.run.smithi100.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:16:28.509 INFO:teuthology.orchestra.run.smithi100.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:16:28.509 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.509 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default size = 2 2024-04-25T10:16:28.509 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.509 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow primary affinity = true 2024-04-25T10:16:28.509 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow pg remap = true 2024-04-25T10:16:28.509 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:16:28.509 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:16:28.509 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on no sortbitwise = false 2024-04-25T10:16:28.509 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:16:28.509 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on too few osds = false 2024-04-25T10:16:28.509 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:16:28.509 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:16:28.510 INFO:teuthology.orchestra.run.smithi100.stdout: mon_allow_pool_size_one = true 2024-04-25T10:16:28.510 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.510 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:16:28.510 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.510 INFO:teuthology.orchestra.run.smithi100.stdout: osd default data pool replay window = 5 2024-04-25T10:16:28.510 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.510 INFO:teuthology.orchestra.run.smithi100.stdout: mon allow pool delete = true 2024-04-25T10:16:28.510 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.510 INFO:teuthology.orchestra.run.smithi100.stdout: mon cluster log file level = debug 2024-04-25T10:16:28.510 INFO:teuthology.orchestra.run.smithi100.stdout: debug asserts on shutdown = true 2024-04-25T10:16:28.510 INFO:teuthology.orchestra.run.smithi100.stdout: mon health detail to clog = false 2024-04-25T10:16:28.510 INFO:teuthology.orchestra.run.smithi100.stdout: mon host = "172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2024-04-25T10:16:28.510 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = 0ea93863-a079-45c4-abed-6e83540ea9fe 2024-04-25T10:16:28.511 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.511 INFO:teuthology.orchestra.run.smithi100.stdout:[osd] 2024-04-25T10:16:28.511 INFO:teuthology.orchestra.run.smithi100.stdout: osd journal size = 100 2024-04-25T10:16:28.511 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.511 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub load threshold = 5.0 2024-04-25T10:16:28.511 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub max interval = 600 2024-04-25T10:16:28.511 INFO:teuthology.orchestra.run.smithi100.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:16:28.511 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.511 INFO:teuthology.orchestra.run.smithi100.stdout: osd recover clone overlap = true 2024-04-25T10:16:28.511 INFO:teuthology.orchestra.run.smithi100.stdout: osd recovery max chunk = 1048576 2024-04-25T10:16:28.511 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.511 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug shutdown = true 2024-04-25T10:16:28.511 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug op order = true 2024-04-25T10:16:28.511 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify stray on activate = true 2024-04-25T10:16:28.511 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug trim objects = true 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi100.stdout: osd open classes on start = true 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug pg log writeout = true 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi100.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi100.stdout: osd map max advance = 10 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi100.stdout: journal zero on create = true 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi100.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi100.stdout: filestore apply finisher threads = 3 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi100.stdout: bdev debug aio = true 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug misdirected ops = true 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi100.stdout: bdev async discard = True 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi100.stdout: bdev enable discard = True 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore allocator = bitmap 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore block size = 96636764160 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore fsck on mount = True 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluefs = 1/20 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluestore = 1/20 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi100.stdout: debug osd = 20 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi100.stdout: debug rocksdb = 4/10 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd full ratio = 0.9 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:16:28.514 INFO:teuthology.orchestra.run.smithi100.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:16:28.514 INFO:teuthology.orchestra.run.smithi100.stdout: osd objectstore = bluestore 2024-04-25T10:16:28.514 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-04-25T10:16:28.514 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.514 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr] 2024-04-25T10:16:28.514 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:28.514 INFO:teuthology.orchestra.run.smithi100.stdout: debug mgr = 20 2024-04-25T10:16:28.514 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-04-25T10:16:28.514 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-04-25T10:16:28.514 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:28.514 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:28.515 INFO:teuthology.orchestra.run.smithi100.stdout: mgr/telemetry/nag = false 2024-04-25T10:16:28.515 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.515 INFO:teuthology.orchestra.run.smithi100.stdout:[mon] 2024-04-25T10:16:28.515 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:28.515 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-04-25T10:16:28.515 INFO:teuthology.orchestra.run.smithi100.stdout: debug paxos = 20 2024-04-25T10:16:28.515 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-04-25T10:16:28.515 INFO:teuthology.orchestra.run.smithi100.stdout: mon data avail warn = 5 2024-04-25T10:16:28.515 INFO:teuthology.orchestra.run.smithi100.stdout: mon mgr mkfs grace = 240 2024-04-25T10:16:28.515 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:28.515 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd reporter subtree level = osd 2024-04-25T10:16:28.515 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd prime pg temp = true 2024-04-25T10:16:28.515 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:28.515 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.516 INFO:teuthology.orchestra.run.smithi100.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:16:28.516 INFO:teuthology.orchestra.run.smithi100.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:16:28.516 INFO:teuthology.orchestra.run.smithi100.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:16:28.516 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.516 INFO:teuthology.orchestra.run.smithi100.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:16:28.516 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:16:28.516 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:16:28.516 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.516 INFO:teuthology.orchestra.run.smithi100.stdout: # 1m isn't quite enough 2024-04-25T10:16:28.516 INFO:teuthology.orchestra.run.smithi100.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:16:28.516 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.516 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:16:28.516 INFO:teuthology.orchestra.run.smithi100.stdout: mon op complaint time = 120 2024-04-25T10:16:28.516 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:28.516 INFO:teuthology.orchestra.run.smithi100.stdout:[client] 2024-04-25T10:16:28.517 INFO:teuthology.orchestra.run.smithi100.stdout: rgw cache enabled = true 2024-04-25T10:16:28.517 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable ops log = true 2024-04-25T10:16:28.517 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable usage log = true 2024-04-25T10:16:28.517 INFO:teuthology.orchestra.run.smithi100.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:16:28.517 INFO:teuthology.orchestra.run.smithi100.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:16:28.517 INFO:teuthology.orchestra.run.smithi100.stdout: client mount timeout = 600 2024-04-25T10:16:28.517 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2024-04-25T10:16:28.517 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:28.517 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = "" 2024-04-25T10:16:28.517 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-04-25T10:16:28.517 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-04-25T10:16:28.517 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.a] 2024-04-25T10:16:28.517 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.b] 2024-04-25T10:16:28.517 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.c] 2024-04-25T10:16:28.517 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-04-25T10:16:28.518 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-04-25T10:16:28.518 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-04-25T10:16:28.518 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-04-25T10:16:28.518 INFO:teuthology.orchestra.run.smithi100.stdout:[mds] 2024-04-25T10:16:28.518 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds = 20 2024-04-25T10:16:28.518 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds balancer = 20 2024-04-25T10:16:28.518 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:28.518 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug frag = True 2024-04-25T10:16:28.518 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug scatterstat = False 2024-04-25T10:16:28.518 INFO:teuthology.orchestra.run.smithi100.stdout: mds op complaint time = 180 2024-04-25T10:16:28.518 INFO:teuthology.orchestra.run.smithi100.stdout: mds verify scatter = False 2024-04-25T10:16:28.518 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-04-25T10:16:28.518 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-04-25T10:16:28.518 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-04-25T10:16:28.520 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-04-25T10:16:28.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.590+0000 7fae15f9d080 1 Processor -- start 2024-04-25T10:16:28.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.590+0000 7fae15f9d080 1 -- start start 2024-04-25T10:16:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.590+0000 7fae15f9d080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556301ce1790 0x556301cb1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.590+0000 7fae15f9d080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556301cfa320 0x556301cddd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.590+0000 7fae15f9d080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556301d861e0 0x556301d82390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.590+0000 7fae15f9d080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556301ce2220 con 0x556301ce1790 2024-04-25T10:16:28.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.590+0000 7fae15f9d080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556301c589f0 con 0x556301d861e0 2024-04-25T10:16:28.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.590+0000 7fae15f9d080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556301ce24c0 con 0x556301cfa320 2024-04-25T10:16:28.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556301ce1790 0x556301cb1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556301ce1790 0x556301cb1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60666/0 (socket says 172.21.15.60:60666) 2024-04-25T10:16:28.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 -- 172.21.15.60:0/2883653059 learned_addr learned my addr 172.21.15.60:0/2883653059 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:28.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae132ba640 1 --2- 172.21.15.60:0/2883653059 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556301cfa320 0x556301cddd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae142bc640 1 --2- 172.21.15.60:0/2883653059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556301d861e0 0x556301d82390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 -- 172.21.15.60:0/2883653059 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556301cfa320 msgr2=0x556301cddd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:28.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 --2- 172.21.15.60:0/2883653059 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556301cfa320 0x556301cddd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 -- 172.21.15.60:0/2883653059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556301d861e0 msgr2=0x556301d82390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:28.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 --2- 172.21.15.60:0/2883653059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556301d861e0 0x556301d82390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 -- 172.21.15.60:0/2883653059 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556301d8c420 con 0x556301ce1790 2024-04-25T10:16:28.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae142bc640 1 --2- 172.21.15.60:0/2883653059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556301d861e0 0x556301d82390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 --2- 172.21.15.60:0/2883653059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556301ce1790 0x556301cb1180 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00a4b0 tx=0x7fae0c0033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e08fafdc708e7e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae122b8640 1 -- 172.21.15.60:0/2883653059 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c0ac7b0 con 0x556301ce1790 2024-04-25T10:16:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae122b8640 1 -- 172.21.15.60:0/2883653059 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c0ac950 con 0x556301ce1790 2024-04-25T10:16:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae122b8640 1 -- 172.21.15.60:0/2883653059 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c0acd40 con 0x556301ce1790 2024-04-25T10:16:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 -- 172.21.15.60:0/2883653059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556301ce1790 msgr2=0x556301cb1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 --2- 172.21.15.60:0/2883653059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556301ce1790 0x556301cb1180 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00a4b0 tx=0x7fae0c0033b0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 -- 172.21.15.60:0/2883653059 shutdown_connections 2024-04-25T10:16:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 --2- 172.21.15.60:0/2883653059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556301d861e0 0x556301d82390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 --2- 172.21.15.60:0/2883653059 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556301cfa320 0x556301cddd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 --2- 172.21.15.60:0/2883653059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556301ce1790 0x556301cb1180 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 -- 172.21.15.60:0/2883653059 >> 172.21.15.60:0/2883653059 conn(0x556301cdce80 msgr2=0x556301cb4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 -- 172.21.15.60:0/2883653059 shutdown_connections 2024-04-25T10:16:28.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 -- 172.21.15.60:0/2883653059 wait complete. 2024-04-25T10:16:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 Processor -- start 2024-04-25T10:16:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 -- start start 2024-04-25T10:16:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556301ce1790 0x556301cf4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556301cfa320 0x556301d84020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556301d861e0 0x556301d84560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556301cdd370 con 0x556301ce1790 2024-04-25T10:16:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556301d8c180 con 0x556301d861e0 2024-04-25T10:16:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556301cdd5d0 con 0x556301cfa320 2024-04-25T10:16:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556301ce1790 0x556301cf4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556301ce1790 0x556301cf4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60670/0 (socket says 172.21.15.60:60670) 2024-04-25T10:16:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 -- 172.21.15.60:0/1648219622 learned_addr learned my addr 172.21.15.60:0/1648219622 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae132ba640 1 --2- 172.21.15.60:0/1648219622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556301cfa320 0x556301d84020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae142bc640 1 --2- 172.21.15.60:0/1648219622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556301d861e0 0x556301d84560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 -- 172.21.15.60:0/1648219622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556301cfa320 msgr2=0x556301d84020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 --2- 172.21.15.60:0/1648219622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556301cfa320 0x556301d84020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 -- 172.21.15.60:0/1648219622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556301d861e0 msgr2=0x556301d84560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 --2- 172.21.15.60:0/1648219622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556301d861e0 0x556301d84560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 -- 172.21.15.60:0/1648219622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556301cf3830 con 0x556301ce1790 2024-04-25T10:16:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae142bc640 1 --2- 172.21.15.60:0/1648219622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556301d861e0 0x556301d84560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae13abb640 1 --2- 172.21.15.60:0/1648219622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556301ce1790 0x556301cf4160 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0b5040 tx=0x7fae0c017eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae112b6640 1 -- 172.21.15.60:0/1648219622 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c0b0040 con 0x556301ce1790 2024-04-25T10:16:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae112b6640 1 -- 172.21.15.60:0/1648219622 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c0c3070 con 0x556301ce1790 2024-04-25T10:16:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 -- 172.21.15.60:0/1648219622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556301d84ba0 con 0x556301ce1790 2024-04-25T10:16:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae112b6640 1 -- 172.21.15.60:0/1648219622 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c0bd8a0 con 0x556301ce1790 2024-04-25T10:16:28.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.594+0000 7fae15f9d080 1 -- 172.21.15.60:0/1648219622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556301d850e0 con 0x556301ce1790 2024-04-25T10:16:28.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.598+0000 7fae112b6640 1 -- 172.21.15.60:0/1648219622 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae0c0a9070 con 0x556301ce1790 2024-04-25T10:16:28.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.598+0000 7fae112b6640 1 --2- 172.21.15.60:0/1648219622 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae00065970 0x7fae00067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.598+0000 7fae112b6640 1 -- 172.21.15.60:0/1648219622 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fae0c11f6a0 con 0x556301ce1790 2024-04-25T10:16:28.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.598+0000 7fae15f9d080 1 --2- 172.21.15.60:0/1648219622 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x556301cf1fa0 0x556301cf2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.598+0000 7fae15f9d080 1 -- 172.21.15.60:0/1648219622 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v9 -- 0x556301cf28c0 con 0x556301cf1fa0 2024-04-25T10:16:28.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.598+0000 7fae132ba640 1 --2- 172.21.15.60:0/1648219622 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae00065970 0x7fae00067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.598+0000 7fae142bc640 1 --2- 172.21.15.60:0/1648219622 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x556301cf1fa0 0x556301cf2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.598+0000 7fae142bc640 1 --2- 172.21.15.60:0/1648219622 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x556301cf1fa0 0x556301cf2380 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:28.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.598+0000 7fae132ba640 1 --2- 172.21.15.60:0/1648219622 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae00065970 0x7fae00067e30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fae0800a730 tx=0x7fae0800c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:28.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.602+0000 7fae142bc640 1 -- 172.21.15.60:0/1648219622 <== osd.3 v2:172.21.15.60:6814/1867957156 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v111'5 uv5 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fae04002040 con 0x556301cf1fa0 2024-04-25T10:16:28.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.602+0000 7fae15f9d080 1 -- 172.21.15.60:0/1648219622 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x556301cf1fa0 msgr2=0x556301cf2380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.602+0000 7fae15f9d080 1 --2- 172.21.15.60:0/1648219622 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x556301cf1fa0 0x556301cf2380 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.602+0000 7fae15f9d080 1 -- 172.21.15.60:0/1648219622 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae00065970 msgr2=0x7fae00067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.602+0000 7fae15f9d080 1 --2- 172.21.15.60:0/1648219622 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fae00065970 0x7fae00067e30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fae0800a730 tx=0x7fae0800c040 comp rx=0 tx=0).stop 2024-04-25T10:16:28.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.602+0000 7fae15f9d080 1 -- 172.21.15.60:0/1648219622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556301ce1790 msgr2=0x556301cf4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.602+0000 7fae15f9d080 1 --2- 172.21.15.60:0/1648219622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556301ce1790 0x556301cf4160 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0b5040 tx=0x7fae0c017eb0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.602+0000 7fae142bc640 1 -- 172.21.15.60:0/1648219622 reap_dead start 2024-04-25T10:16:28.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.602+0000 7fae15f9d080 1 -- 172.21.15.60:0/1648219622 shutdown_connections 2024-04-25T10:16:28.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.602+0000 7fae15f9d080 1 -- 172.21.15.60:0/1648219622 >> 172.21.15.60:0/1648219622 conn(0x556301cdce80 msgr2=0x556301d90260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:28.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.602+0000 7fae15f9d080 1 -- 172.21.15.60:0/1648219622 shutdown_connections 2024-04-25T10:16:28.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.602+0000 7fae15f9d080 1 -- 172.21.15.60:0/1648219622 wait complete. 2024-04-25T10:16:28.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:16:28.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8fdc3640 1 Processor -- start 2024-04-25T10:16:28.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8fdc3640 1 -- start start 2024-04-25T10:16:28.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8fdc3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f880cbcf0 0x7f3f88160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8fdc3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3f8815fbf0 0x7f3f880ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8fdc3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f88144a40 con 0x7f3f88071c30 2024-04-25T10:16:28.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8fdc3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f880b2dc0 con 0x7f3f8815fbf0 2024-04-25T10:16:28.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8fdc3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880cc9f0 con 0x7f3f880cbcf0 2024-04-25T10:16:28.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8e360640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3f8815fbf0 0x7f3f880ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8d35e640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f3f88071c30 0x7f3f8815f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:47054/0 (socket says 172.21.15.60:47054) 2024-04-25T10:16:28.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8d35e640 1 -- 172.21.15.60:0/3906918805 learned_addr learned my addr 172.21.15.60:0/3906918805 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:28.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8db5f640 1 --2- 172.21.15.60:0/3906918805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f880cbcf0 0x7f3f88160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8cb5d640 1 -- 172.21.15.60:0/3906918805 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214056366 0 0) 0x7f3f88144a40 con 0x7f3f88071c30 2024-04-25T10:16:28.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8cb5d640 1 -- 172.21.15.60:0/3906918805 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f74003530 con 0x7f3f88071c30 2024-04-25T10:16:28.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8db5f640 1 -- 172.21.15.60:0/3906918805 >> v2:172.21.15.100:3300/0 conn(0x7f3f8815fbf0 msgr2=0x7f3f880ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:28.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8db5f640 1 --2- 172.21.15.60:0/3906918805 >> v2:172.21.15.100:3300/0 conn(0x7f3f8815fbf0 0x7f3f880ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8db5f640 1 -- 172.21.15.60:0/3906918805 >> v1:172.21.15.100:6789/0 conn(0x7f3f88071c30 legacy=0x7f3f8815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.774+0000 7f3f8db5f640 1 -- 172.21.15.60:0/3906918805 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f880cccd0 con 0x7f3f880cbcf0 2024-04-25T10:16:28.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.778+0000 7f3f8db5f640 1 --2- 172.21.15.60:0/3906918805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f880cbcf0 0x7f3f88160080 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3f840097c0 tx=0x7f3f8401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f26daa89b1fb730d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:28.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.778+0000 7f3f8cb5d640 1 -- 172.21.15.60:0/3906918805 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f840a7070 con 0x7f3f880cbcf0 2024-04-25T10:16:28.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.778+0000 7f3f8cb5d640 1 -- 172.21.15.60:0/3906918805 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f840b0070 con 0x7f3f880cbcf0 2024-04-25T10:16:28.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.778+0000 7f3f8cb5d640 1 -- 172.21.15.60:0/3906918805 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f840ab890 con 0x7f3f880cbcf0 2024-04-25T10:16:28.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.778+0000 7f3f8fdc3640 1 -- 172.21.15.60:0/3906918805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f880cbcf0 msgr2=0x7f3f88160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.778+0000 7f3f8fdc3640 1 --2- 172.21.15.60:0/3906918805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f880cbcf0 0x7f3f88160080 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3f840097c0 tx=0x7f3f8401c470 comp rx=0 tx=0).stop 2024-04-25T10:16:28.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.778+0000 7f3f8fdc3640 1 -- 172.21.15.60:0/3906918805 shutdown_connections 2024-04-25T10:16:28.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.778+0000 7f3f8fdc3640 1 --2- 172.21.15.60:0/3906918805 >> v2:172.21.15.100:3300/0 conn(0x7f3f8815fbf0 0x7f3f880ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.778+0000 7f3f8fdc3640 1 --2- 172.21.15.60:0/3906918805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f880cbcf0 0x7f3f88160080 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.778+0000 7f3f8fdc3640 1 -- 172.21.15.60:0/3906918805 >> 172.21.15.60:0/3906918805 conn(0x7f3f880670b0 msgr2=0x7f3f8806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:28.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.778+0000 7f3f8fdc3640 1 -- 172.21.15.60:0/3906918805 shutdown_connections 2024-04-25T10:16:28.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.778+0000 7f3f8fdc3640 1 -- 172.21.15.60:0/3906918805 wait complete. 2024-04-25T10:16:28.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.778+0000 7f3f8fdc3640 1 Processor -- start 2024-04-25T10:16:28.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.778+0000 7f3f8fdc3640 1 -- start start 2024-04-25T10:16:28.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8fdc3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f88071c30 0x7f3f88166290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8fdc3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f880cbcf0 0x7f3f880b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8fdc3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8815fbf0 0x7f3f8815a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8fdc3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880ce740 con 0x7f3f880cbcf0 2024-04-25T10:16:28.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8fdc3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880ce5c0 con 0x7f3f8815fbf0 2024-04-25T10:16:28.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8fdc3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f88072030 con 0x7f3f88071c30 2024-04-25T10:16:28.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8d35e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f880cbcf0 0x7f3f880b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8d35e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f880cbcf0 0x7f3f880b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60676/0 (socket says 172.21.15.60:60676) 2024-04-25T10:16:28.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8d35e640 1 -- 172.21.15.60:0/231454422 learned_addr learned my addr 172.21.15.60:0/231454422 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:28.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8db5f640 1 --2- 172.21.15.60:0/231454422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f88071c30 0x7f3f88166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8e360640 1 --2- 172.21.15.60:0/231454422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8815fbf0 0x7f3f8815a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8d35e640 1 -- 172.21.15.60:0/231454422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f88071c30 msgr2=0x7f3f88166290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8d35e640 1 --2- 172.21.15.60:0/231454422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f88071c30 0x7f3f88166290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8d35e640 1 -- 172.21.15.60:0/231454422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8815fbf0 msgr2=0x7f3f8815a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8d35e640 1 --2- 172.21.15.60:0/231454422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8815fbf0 0x7f3f8815a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8d35e640 1 -- 172.21.15.60:0/231454422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8815a810 con 0x7f3f880cbcf0 2024-04-25T10:16:28.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8d35e640 1 --2- 172.21.15.60:0/231454422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f880cbcf0 0x7f3f880b19f0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c3f0 tx=0x7f3f7800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:28.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f7e7fc640 1 -- 172.21.15.60:0/231454422 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78017070 con 0x7f3f880cbcf0 2024-04-25T10:16:28.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f7e7fc640 1 -- 172.21.15.60:0/231454422 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f78005cf0 con 0x7f3f880cbcf0 2024-04-25T10:16:28.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f7e7fc640 1 -- 172.21.15.60:0/231454422 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f780072f0 con 0x7f3f880cbcf0 2024-04-25T10:16:28.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8fdc3640 1 -- 172.21.15.60:0/231454422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8815aaf0 con 0x7f3f880cbcf0 2024-04-25T10:16:28.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8fdc3640 1 -- 172.21.15.60:0/231454422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8815b0b0 con 0x7f3f880cbcf0 2024-04-25T10:16:28.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8fdc3640 1 -- 172.21.15.60:0/231454422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8806bba0 con 0x7f3f880cbcf0 2024-04-25T10:16:28.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f7e7fc640 1 -- 172.21.15.60:0/231454422 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f7801a070 con 0x7f3f880cbcf0 2024-04-25T10:16:28.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f7e7fc640 1 --2- 172.21.15.60:0/231454422 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3f6c066f60 0x7f3f6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f7e7fc640 1 -- 172.21.15.60:0/231454422 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3f78084570 con 0x7f3f880cbcf0 2024-04-25T10:16:28.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.782+0000 7f3f8db5f640 1 --2- 172.21.15.60:0/231454422 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3f6c066f60 0x7f3f6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.786+0000 7f3f8db5f640 1 --2- 172.21.15.60:0/231454422 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3f6c066f60 0x7f3f6c069420 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3f840af000 tx=0x7f3f840a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:28.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.790+0000 7f3f7e7fc640 1 -- 172.21.15.60:0/231454422 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f780554a0 con 0x7f3f880cbcf0 2024-04-25T10:16:28.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:28.986+0000 7f3f8fdc3640 1 -- 172.21.15.60:0/231454422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f3f88074770 con 0x7f3f880cbcf0 2024-04-25T10:16:29.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.362+0000 7f3f7e7fc640 1 -- 172.21.15.60:0/231454422 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v95) v1 ==== 162+0+0 (secure 0 0 0) 0x7f3f78054da0 con 0x7f3f880cbcf0 2024-04-25T10:16:29.369 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:16:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.366+0000 7f3f8fdc3640 1 -- 172.21.15.60:0/231454422 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3f6c066f60 msgr2=0x7f3f6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.366+0000 7f3f8fdc3640 1 --2- 172.21.15.60:0/231454422 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3f6c066f60 0x7f3f6c069420 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3f840af000 tx=0x7f3f840a7000 comp rx=0 tx=0).stop 2024-04-25T10:16:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.366+0000 7f3f8fdc3640 1 -- 172.21.15.60:0/231454422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f880cbcf0 msgr2=0x7f3f880b19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.366+0000 7f3f8fdc3640 1 --2- 172.21.15.60:0/231454422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f880cbcf0 0x7f3f880b19f0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c3f0 tx=0x7f3f7800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.366+0000 7f3f8fdc3640 1 -- 172.21.15.60:0/231454422 shutdown_connections 2024-04-25T10:16:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.366+0000 7f3f8fdc3640 1 --2- 172.21.15.60:0/231454422 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3f6c066f60 0x7f3f6c069420 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.366+0000 7f3f8fdc3640 1 --2- 172.21.15.60:0/231454422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8815fbf0 0x7f3f8815a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.366+0000 7f3f8fdc3640 1 --2- 172.21.15.60:0/231454422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f880cbcf0 0x7f3f880b19f0 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.366+0000 7f3f8fdc3640 1 --2- 172.21.15.60:0/231454422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f88071c30 0x7f3f88166290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.366+0000 7f3f8fdc3640 1 -- 172.21.15.60:0/231454422 >> 172.21.15.60:0/231454422 conn(0x7f3f880670b0 msgr2=0x7f3f8815f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:29.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.370+0000 7f3f8fdc3640 1 -- 172.21.15.60:0/231454422 shutdown_connections 2024-04-25T10:16:29.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.370+0000 7f3f8fdc3640 1 -- 172.21.15.60:0/231454422 wait complete. 2024-04-25T10:16:29.402 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:29.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd141a62640 1 Processor -- start 2024-04-25T10:16:29.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd141a62640 1 -- start start 2024-04-25T10:16:29.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd141a62640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd13c15bc90 0x7fd13c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd141a62640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd13c071ca0 con 0x7fd13c0ce7a0 2024-04-25T10:16:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd141a62640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd13c0cca40 con 0x7fd13c0b4660 2024-04-25T10:16:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd141a62640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd13c0cc370 con 0x7fd13c15bc90 2024-04-25T10:16:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd13c0ce7a0 0x7fd13c0ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56074/0 (socket says 172.21.15.60:56074) 2024-04-25T10:16:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13b7fe640 1 -- 172.21.15.60:0/2002146606 learned_addr learned my addr 172.21.15.60:0/2002146606 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13a7fc640 1 -- 172.21.15.60:0/2002146606 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532649472 0 0) 0x7fd13c071ca0 con 0x7fd13c0ce7a0 2024-04-25T10:16:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13a7fc640 1 -- 172.21.15.60:0/2002146606 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd124003530 con 0x7fd13c0ce7a0 2024-04-25T10:16:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13affd640 1 --2- 172.21.15.60:0/2002146606 >> v2:172.21.15.100:3300/0 conn(0x7fd13c15bc90 0x7fd13c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13a7fc640 1 -- 172.21.15.60:0/2002146606 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3147791061 0 0) 0x7fd124003530 con 0x7fd13c0ce7a0 2024-04-25T10:16:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13a7fc640 1 -- 172.21.15.60:0/2002146606 >> v2:172.21.15.100:3300/0 conn(0x7fd13c15bc90 msgr2=0x7fd13c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13a7fc640 1 --2- 172.21.15.60:0/2002146606 >> v2:172.21.15.100:3300/0 conn(0x7fd13c15bc90 0x7fd13c15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13a7fc640 1 -- 172.21.15.60:0/2002146606 >> v1:172.21.15.100:6789/0 conn(0x7fd13c0b4660 legacy=0x7fd13c15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13a7fc640 1 -- 172.21.15.60:0/2002146606 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c0ccd20 con 0x7fd13c0ce7a0 2024-04-25T10:16:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13a7fc640 1 -- 172.21.15.60:0/2002146606 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fd12c001680 con 0x7fd13c0ce7a0 2024-04-25T10:16:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13a7fc640 1 -- 172.21.15.60:0/2002146606 >> v1:172.21.15.60:6789/0 conn(0x7fd13c0ce7a0 legacy=0x7fd13c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13a7fc640 1 --2- 172.21.15.60:0/2002146606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd124003a10 0x7fd124003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13a7fc640 1 --2- 172.21.15.60:0/2002146606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd124004810 0x7fd124004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13a7fc640 1 --2- 172.21.15.60:0/2002146606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1240051d0 0x7fd12400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13a7fc640 1 -- 172.21.15.60:0/2002146606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12400c0a0 con 0x7fd124003a10 2024-04-25T10:16:29.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13a7fc640 1 -- 172.21.15.60:0/2002146606 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12400c250 con 0x7fd1240051d0 2024-04-25T10:16:29.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13a7fc640 1 -- 172.21.15.60:0/2002146606 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12400c450 con 0x7fd124004810 2024-04-25T10:16:29.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13b7fe640 1 --2- 172.21.15.60:0/2002146606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd124003a10 0x7fd124003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13affd640 1 --2- 172.21.15.60:0/2002146606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd124004810 0x7fd124004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13bfff640 1 --2- 172.21.15.60:0/2002146606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1240051d0 0x7fd12400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13b7fe640 1 -- 172.21.15.60:0/2002146606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd124004810 msgr2=0x7fd124004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:29.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13b7fe640 1 --2- 172.21.15.60:0/2002146606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd124004810 0x7fd124004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13b7fe640 1 -- 172.21.15.60:0/2002146606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1240051d0 msgr2=0x7fd12400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:29.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13b7fe640 1 --2- 172.21.15.60:0/2002146606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1240051d0 0x7fd12400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13bfff640 1 -- 172.21.15.60:0/2002146606 reap_dead start 2024-04-25T10:16:29.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.522+0000 7fd13b7fe640 1 -- 172.21.15.60:0/2002146606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd124003530 con 0x7fd124003a10 2024-04-25T10:16:29.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.526+0000 7fd13b7fe640 1 --2- 172.21.15.60:0/2002146606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd124003a10 0x7fd124003e30 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00c110 tx=0x7fd12c01cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc479c22319be9db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:29.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.526+0000 7fd13a7fc640 1 -- 172.21.15.60:0/2002146606 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c007c50 con 0x7fd124003a10 2024-04-25T10:16:29.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.526+0000 7fd13a7fc640 1 -- 172.21.15.60:0/2002146606 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12c0a9070 con 0x7fd124003a10 2024-04-25T10:16:29.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.526+0000 7fd13a7fc640 1 -- 172.21.15.60:0/2002146606 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c005d80 con 0x7fd124003a10 2024-04-25T10:16:29.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.526+0000 7fd141a62640 1 -- 172.21.15.60:0/2002146606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd124003a10 msgr2=0x7fd124003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.526+0000 7fd141a62640 1 --2- 172.21.15.60:0/2002146606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd124003a10 0x7fd124003e30 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00c110 tx=0x7fd12c01cbc0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.526+0000 7fd141a62640 1 -- 172.21.15.60:0/2002146606 shutdown_connections 2024-04-25T10:16:29.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.526+0000 7fd141a62640 1 --2- 172.21.15.60:0/2002146606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd124003a10 0x7fd124003e30 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.526+0000 7fd141a62640 1 -- 172.21.15.60:0/2002146606 >> 172.21.15.60:0/2002146606 conn(0x7fd13c0b12d0 msgr2=0x7fd13c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:29.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.526+0000 7fd141a62640 1 -- 172.21.15.60:0/2002146606 shutdown_connections 2024-04-25T10:16:29.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.526+0000 7fd141a62640 1 -- 172.21.15.60:0/2002146606 wait complete. 2024-04-25T10:16:29.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.526+0000 7fd141a62640 1 Processor -- start 2024-04-25T10:16:29.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.526+0000 7fd141a62640 1 -- start start 2024-04-25T10:16:29.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd141a62640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd124003a10 0x7fd13c156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd141a62640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd13c157f40 0x7fd13c0b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd141a62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd13c1fad10 0x7fd13c1fb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd141a62640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c0b4280 con 0x7fd124003a10 2024-04-25T10:16:29.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd141a62640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c0b3db0 con 0x7fd13c1fad10 2024-04-25T10:16:29.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd141a62640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c0b4400 con 0x7fd13c157f40 2024-04-25T10:16:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd13b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd124003a10 0x7fd13c156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd13b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd124003a10 0x7fd13c156610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39376/0 (socket says 172.21.15.60:39376) 2024-04-25T10:16:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd13b7fe640 1 -- 172.21.15.60:0/2536741403 learned_addr learned my addr 172.21.15.60:0/2536741403 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd13affd640 1 --2- 172.21.15.60:0/2536741403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd13c157f40 0x7fd13c0b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd13bfff640 1 --2- 172.21.15.60:0/2536741403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd13c1fad10 0x7fd13c1fb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd13b7fe640 1 -- 172.21.15.60:0/2536741403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd13c157f40 msgr2=0x7fd13c0b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd13b7fe640 1 --2- 172.21.15.60:0/2536741403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd13c157f40 0x7fd13c0b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd13b7fe640 1 -- 172.21.15.60:0/2536741403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd13c1fad10 msgr2=0x7fd13c1fb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd13b7fe640 1 --2- 172.21.15.60:0/2536741403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd13c1fad10 0x7fd13c1fb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd13b7fe640 1 -- 172.21.15.60:0/2536741403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c06da50 con 0x7fd124003a10 2024-04-25T10:16:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd13b7fe640 1 --2- 172.21.15.60:0/2536741403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd124003a10 0x7fd13c156610 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fd12c01c9f0 tx=0x7fd124005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd123fff640 1 -- 172.21.15.60:0/2536741403 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c007140 con 0x7fd124003a10 2024-04-25T10:16:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd123fff640 1 -- 172.21.15.60:0/2536741403 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12c0072e0 con 0x7fd124003a10 2024-04-25T10:16:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd123fff640 1 -- 172.21.15.60:0/2536741403 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c007690 con 0x7fd124003a10 2024-04-25T10:16:29.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd141a62640 1 -- 172.21.15.60:0/2536741403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13c06dd30 con 0x7fd124003a10 2024-04-25T10:16:29.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd141a62640 1 -- 172.21.15.60:0/2536741403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13c0b0d00 con 0x7fd124003a10 2024-04-25T10:16:29.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd141a62640 1 -- 172.21.15.60:0/2536741403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd13c074770 con 0x7fd124003a10 2024-04-25T10:16:29.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd123fff640 1 -- 172.21.15.60:0/2536741403 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd12c0b6030 con 0x7fd124003a10 2024-04-25T10:16:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd123fff640 1 --2- 172.21.15.60:0/2536741403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd128066f60 0x7fd128069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd123fff640 1 -- 172.21.15.60:0/2536741403 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd12c11e1d0 con 0x7fd124003a10 2024-04-25T10:16:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.530+0000 7fd13affd640 1 --2- 172.21.15.60:0/2536741403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd128066f60 0x7fd128069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.534+0000 7fd13affd640 1 --2- 172.21.15.60:0/2536741403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd128066f60 0x7fd128069420 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0ad9f0 tx=0x7fd130008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:29.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.534+0000 7fd123fff640 1 -- 172.21.15.60:0/2536741403 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd12c0168d0 con 0x7fd124003a10 2024-04-25T10:16:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.730+0000 7fd141a62640 1 -- 172.21.15.60:0/2536741403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd13c0cbfc0 con 0x7fd124003a10 2024-04-25T10:16:29.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.730+0000 7fd123fff640 1 -- 172.21.15.60:0/2536741403 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4945 (secure 0 0 0) 0x7fd12c0ef190 con 0x7fd124003a10 2024-04-25T10:16:29.737 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:29.737 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":96,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2440975447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2440975447},{"type":"v1","addr":"172.21.15.100:6835","nonce":2440975447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2398080758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2398080758},{"type":"v1","addr":"172.21.15.60:6833","nonce":2398080758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2759861958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2759861958},{"type":"v1","addr":"172.21.15.100:6837","nonce":2759861958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:07.763322+0000","modified":"2024-04-25T10:16:29.371519+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9693},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9693":{"gid":9693,"name":"c","rank":0,"incarnation":96,"state":"up:replay","state_seq":1,"addr":"172.21.15.60:6835/178428701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":178428701},{"type":"v1","addr":"172.21.15.60:6835","nonce":178428701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:16:29.737 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 96 2024-04-25T10:16:29.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.734+0000 7fd141a62640 1 -- 172.21.15.60:0/2536741403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd128066f60 msgr2=0x7fd128069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.734+0000 7fd141a62640 1 --2- 172.21.15.60:0/2536741403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd128066f60 0x7fd128069420 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0ad9f0 tx=0x7fd130008040 comp rx=0 tx=0).stop 2024-04-25T10:16:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.734+0000 7fd141a62640 1 -- 172.21.15.60:0/2536741403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd124003a10 msgr2=0x7fd13c156610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.734+0000 7fd141a62640 1 --2- 172.21.15.60:0/2536741403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd124003a10 0x7fd13c156610 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fd12c01c9f0 tx=0x7fd124005840 comp rx=0 tx=0).stop 2024-04-25T10:16:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.734+0000 7fd141a62640 1 -- 172.21.15.60:0/2536741403 shutdown_connections 2024-04-25T10:16:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.734+0000 7fd141a62640 1 --2- 172.21.15.60:0/2536741403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd128066f60 0x7fd128069420 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.734+0000 7fd141a62640 1 --2- 172.21.15.60:0/2536741403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd13c1fad10 0x7fd13c1fb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.734+0000 7fd141a62640 1 --2- 172.21.15.60:0/2536741403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd13c157f40 0x7fd13c0b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.734+0000 7fd141a62640 1 --2- 172.21.15.60:0/2536741403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd124003a10 0x7fd13c156610 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.734+0000 7fd141a62640 1 -- 172.21.15.60:0/2536741403 >> 172.21.15.60:0/2536741403 conn(0x7fd13c0b12d0 msgr2=0x7fd13c15e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.734+0000 7fd141a62640 1 -- 172.21.15.60:0/2536741403 shutdown_connections 2024-04-25T10:16:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:29.734+0000 7fd141a62640 1 -- 172.21.15.60:0/2536741403 wait complete. 2024-04-25T10:16:30.159 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:16:07.763322+0000', 'modified': '2024-04-25T10:16:29.371519+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 111, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9693}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9693': {'gid': 9693, 'name': 'c', 'rank': 0, 'incarnation': 96, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.60:6835/178428701', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 178428701}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 178428701}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:30.159 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9693:up:replay 2024-04-25T10:16:31.161 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:31.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.294+0000 7f3c4d1cc640 1 Processor -- start 2024-04-25T10:16:31.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.294+0000 7f3c4d1cc640 1 -- start start 2024-04-25T10:16:31.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.294+0000 7f3c4d1cc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3c4815deb0 0x7f3c480b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:31.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.294+0000 7f3c4d1cc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3c480d0340 0x7f3c480d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:31.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.294+0000 7f3c4d1cc640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c4806d200 con 0x7f3c480ce2b0 2024-04-25T10:16:31.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.294+0000 7f3c4d1cc640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c480cf2c0 con 0x7f3c4815deb0 2024-04-25T10:16:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.294+0000 7f3c4d1cc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c480cefe0 con 0x7f3c480d0340 2024-04-25T10:16:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.294+0000 7f3c46d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3c4815deb0 0x7f3c480b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.294+0000 7f3c4759e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3c480ce2b0 0x7f3c480ce720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56076/0 (socket says 172.21.15.60:56076) 2024-04-25T10:16:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.294+0000 7f3c4759e640 1 -- 172.21.15.60:0/1148301565 learned_addr learned my addr 172.21.15.60:0/1148301565 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.294+0000 7f3c4659c640 1 --2- 172.21.15.60:0/1148301565 >> v2:172.21.15.100:3300/0 conn(0x7f3c480d0340 0x7f3c480d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.294+0000 7f3c46d9d640 1 -- 172.21.15.60:0/1148301565 >> v2:172.21.15.100:3300/0 conn(0x7f3c480d0340 msgr2=0x7f3c480d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.294+0000 7f3c46d9d640 1 --2- 172.21.15.60:0/1148301565 >> v2:172.21.15.100:3300/0 conn(0x7f3c480d0340 0x7f3c480d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.294+0000 7f3c46d9d640 1 -- 172.21.15.60:0/1148301565 >> v1:172.21.15.60:6789/0 conn(0x7f3c480ce2b0 legacy=0x7f3c480ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.294+0000 7f3c46d9d640 1 -- 172.21.15.60:0/1148301565 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c480d1d70 con 0x7f3c4815deb0 2024-04-25T10:16:31.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c46d9d640 1 --2- 172.21.15.60:0/1148301565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c4815deb0 0x7f3c480b6c10 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f3c480b6490 tx=0x7f3c3801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ea13e6fdda4416 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:31.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c45d9b640 1 -- 172.21.15.60:0/1148301565 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c380b0070 con 0x7f3c4815deb0 2024-04-25T10:16:31.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c45d9b640 1 -- 172.21.15.60:0/1148301565 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c380ab600 con 0x7f3c4815deb0 2024-04-25T10:16:31.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c45d9b640 1 -- 172.21.15.60:0/1148301565 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c380ab9d0 con 0x7f3c4815deb0 2024-04-25T10:16:31.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c4d1cc640 1 -- 172.21.15.60:0/1148301565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c4815deb0 msgr2=0x7f3c480b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:31.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c4d1cc640 1 --2- 172.21.15.60:0/1148301565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c4815deb0 0x7f3c480b6c10 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f3c480b6490 tx=0x7f3c3801c480 comp rx=0 tx=0).stop 2024-04-25T10:16:31.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c4d1cc640 1 -- 172.21.15.60:0/1148301565 shutdown_connections 2024-04-25T10:16:31.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c4d1cc640 1 --2- 172.21.15.60:0/1148301565 >> v2:172.21.15.100:3300/0 conn(0x7f3c480d0340 0x7f3c480d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c4d1cc640 1 --2- 172.21.15.60:0/1148301565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c4815deb0 0x7f3c480b6c10 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c4d1cc640 1 -- 172.21.15.60:0/1148301565 >> 172.21.15.60:0/1148301565 conn(0x7f3c4806c090 msgr2=0x7f3c4806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:31.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c4d1cc640 1 -- 172.21.15.60:0/1148301565 shutdown_connections 2024-04-25T10:16:31.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c4d1cc640 1 -- 172.21.15.60:0/1148301565 wait complete. 2024-04-25T10:16:31.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c4d1cc640 1 Processor -- start 2024-04-25T10:16:31.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c4d1cc640 1 -- start start 2024-04-25T10:16:31.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c4d1cc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c480ce2b0 0x7f3c4814cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:31.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c4d1cc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c480d0340 0x7f3c4814d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:31.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c4d1cc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c4815deb0 0x7f3c48153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:31.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c4d1cc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c480d3430 con 0x7f3c480ce2b0 2024-04-25T10:16:31.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c4d1cc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c480d32b0 con 0x7f3c4815deb0 2024-04-25T10:16:31.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c4d1cc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c480d35b0 con 0x7f3c480d0340 2024-04-25T10:16:31.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.298+0000 7f3c46d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c480ce2b0 0x7f3c4814cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:31.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c46d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c480ce2b0 0x7f3c4814cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39390/0 (socket says 172.21.15.60:39390) 2024-04-25T10:16:31.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c46d9d640 1 -- 172.21.15.60:0/1047274626 learned_addr learned my addr 172.21.15.60:0/1047274626 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:31.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c4659c640 1 --2- 172.21.15.60:0/1047274626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c480d0340 0x7f3c4814d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:31.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c4759e640 1 --2- 172.21.15.60:0/1047274626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c4815deb0 0x7f3c48153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:31.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c46d9d640 1 -- 172.21.15.60:0/1047274626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c480d0340 msgr2=0x7f3c4814d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:31.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c46d9d640 1 --2- 172.21.15.60:0/1047274626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c480d0340 0x7f3c4814d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c46d9d640 1 -- 172.21.15.60:0/1047274626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c4815deb0 msgr2=0x7f3c48153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:31.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c46d9d640 1 --2- 172.21.15.60:0/1047274626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c4815deb0 0x7f3c48153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c46d9d640 1 -- 172.21.15.60:0/1047274626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4814ddf0 con 0x7f3c480ce2b0 2024-04-25T10:16:31.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c4759e640 1 --2- 172.21.15.60:0/1047274626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c4815deb0 0x7f3c48153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:31.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c46d9d640 1 --2- 172.21.15.60:0/1047274626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c480ce2b0 0x7f3c4814cb70 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f3c380ae040 tx=0x7f3c3801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:31.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c2f7fe640 1 -- 172.21.15.60:0/1047274626 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c380b0050 con 0x7f3c480ce2b0 2024-04-25T10:16:31.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c2f7fe640 1 -- 172.21.15.60:0/1047274626 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c380bc070 con 0x7f3c480ce2b0 2024-04-25T10:16:31.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c4d1cc640 1 -- 172.21.15.60:0/1047274626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c481517f0 con 0x7f3c480ce2b0 2024-04-25T10:16:31.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c2f7fe640 1 -- 172.21.15.60:0/1047274626 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c380b5810 con 0x7f3c480ce2b0 2024-04-25T10:16:31.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c4d1cc640 1 -- 172.21.15.60:0/1047274626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4815c290 con 0x7f3c480ce2b0 2024-04-25T10:16:31.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c4d1cc640 1 -- 172.21.15.60:0/1047274626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c48074770 con 0x7f3c480ce2b0 2024-04-25T10:16:31.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c2f7fe640 1 -- 172.21.15.60:0/1047274626 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c380b59b0 con 0x7f3c480ce2b0 2024-04-25T10:16:31.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c2f7fe640 1 --2- 172.21.15.60:0/1047274626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c34066f60 0x7f3c34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:31.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c2f7fe640 1 -- 172.21.15.60:0/1047274626 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3c3811f8a0 con 0x7f3c480ce2b0 2024-04-25T10:16:31.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c4659c640 1 --2- 172.21.15.60:0/1047274626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c34066f60 0x7f3c34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:31.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.302+0000 7f3c4659c640 1 --2- 172.21.15.60:0/1047274626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c34066f60 0x7f3c34069420 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3c4815b8f0 tx=0x7f3c3c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:31.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.306+0000 7f3c2f7fe640 1 -- 172.21.15.60:0/1047274626 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c38016070 con 0x7f3c480ce2b0 2024-04-25T10:16:31.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.502+0000 7f3c4d1cc640 1 -- 172.21.15.60:0/1047274626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c480d17a0 con 0x7f3c480ce2b0 2024-04-25T10:16:31.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.502+0000 7f3c2f7fe640 1 -- 172.21.15.60:0/1047274626 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f3c380f08e0 con 0x7f3c480ce2b0 2024-04-25T10:16:31.508 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:31.508 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2440975447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2440975447},{"type":"v1","addr":"172.21.15.100:6835","nonce":2440975447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2398080758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2398080758},{"type":"v1","addr":"172.21.15.60:6833","nonce":2398080758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2759861958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2759861958},{"type":"v1","addr":"172.21.15.100:6837","nonce":2759861958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:07.763322+0000","modified":"2024-04-25T10:16:30.381046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9693},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9693":{"gid":9693,"name":"c","rank":0,"incarnation":96,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.60:6835/178428701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":178428701},{"type":"v1","addr":"172.21.15.60:6835","nonce":178428701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:16:31.508 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 98 2024-04-25T10:16:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.506+0000 7f3c4d1cc640 1 -- 172.21.15.60:0/1047274626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c34066f60 msgr2=0x7f3c34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.506+0000 7f3c4d1cc640 1 --2- 172.21.15.60:0/1047274626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c34066f60 0x7f3c34069420 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3c4815b8f0 tx=0x7f3c3c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.506+0000 7f3c4d1cc640 1 -- 172.21.15.60:0/1047274626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c480ce2b0 msgr2=0x7f3c4814cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.506+0000 7f3c4d1cc640 1 --2- 172.21.15.60:0/1047274626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c480ce2b0 0x7f3c4814cb70 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f3c380ae040 tx=0x7f3c3801acc0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.506+0000 7f3c4d1cc640 1 -- 172.21.15.60:0/1047274626 shutdown_connections 2024-04-25T10:16:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.506+0000 7f3c4d1cc640 1 --2- 172.21.15.60:0/1047274626 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c34066f60 0x7f3c34069420 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.506+0000 7f3c4d1cc640 1 --2- 172.21.15.60:0/1047274626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c4815deb0 0x7f3c48153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.506+0000 7f3c4d1cc640 1 --2- 172.21.15.60:0/1047274626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c480d0340 0x7f3c4814d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.506+0000 7f3c4d1cc640 1 --2- 172.21.15.60:0/1047274626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c480ce2b0 0x7f3c4814cb70 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.506+0000 7f3c4d1cc640 1 -- 172.21.15.60:0/1047274626 >> 172.21.15.60:0/1047274626 conn(0x7f3c4806c090 msgr2=0x7f3c480ce720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:31.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.506+0000 7f3c4d1cc640 1 -- 172.21.15.60:0/1047274626 shutdown_connections 2024-04-25T10:16:31.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:31.506+0000 7f3c4d1cc640 1 -- 172.21.15.60:0/1047274626 wait complete. 2024-04-25T10:16:31.928 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 98, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:16:07.763322+0000', 'modified': '2024-04-25T10:16:30.381046+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 111, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9693}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9693': {'gid': 9693, 'name': 'c', 'rank': 0, 'incarnation': 96, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.60:6835/178428701', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 178428701}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 178428701}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:31.929 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9693:up:rejoin 2024-04-25T10:16:32.930 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:33.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.062+0000 7f4b3c2dc640 1 Processor -- start 2024-04-25T10:16:33.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.062+0000 7f4b3c2dc640 1 -- start start 2024-04-25T10:16:33.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.062+0000 7f4b3c2dc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3415dea0 0x7f4b3406c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.062+0000 7f4b3c2dc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b340ce4f0 0x7f4b340d2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.062+0000 7f4b3c2dc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b3406c680 con 0x7f4b340d3110 2024-04-25T10:16:33.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.062+0000 7f4b3c2dc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b3415e280 con 0x7f4b340ce4f0 2024-04-25T10:16:33.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.062+0000 7f4b3c2dc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3406c930 con 0x7f4b3415dea0 2024-04-25T10:16:33.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.062+0000 7f4b39877640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b340ce4f0 0x7f4b340d2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.062+0000 7f4b3a078640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3415dea0 0x7f4b3406c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.062+0000 7f4b3a879640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4b340d3110 0x7f4b340ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:59786/0 (socket says 172.21.15.60:59786) 2024-04-25T10:16:33.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.062+0000 7f4b3a078640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3415dea0 0x7f4b3406c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:48886/0 (socket says 172.21.15.60:48886) 2024-04-25T10:16:33.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.062+0000 7f4b3a879640 1 -- 172.21.15.60:0/1403620945 learned_addr learned my addr 172.21.15.60:0/1403620945 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:33.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.062+0000 7f4b3a078640 1 -- 172.21.15.60:0/1403620945 >> v2:172.21.15.100:3300/0 conn(0x7f4b340ce4f0 msgr2=0x7f4b340d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:33.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.062+0000 7f4b3a078640 1 --2- 172.21.15.60:0/1403620945 >> v2:172.21.15.100:3300/0 conn(0x7f4b340ce4f0 0x7f4b340d2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.062+0000 7f4b3a078640 1 -- 172.21.15.60:0/1403620945 >> v1:172.21.15.100:6789/0 conn(0x7f4b340d3110 legacy=0x7f4b340ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.062+0000 7f4b3a078640 1 -- 172.21.15.60:0/1403620945 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b340d20f0 con 0x7f4b3415dea0 2024-04-25T10:16:33.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.066+0000 7f4b3a078640 1 --2- 172.21.15.60:0/1403620945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3415dea0 0x7f4b3406c140 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4b300097c0 tx=0x7f4b3001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f1e75ac1506b188 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.066+0000 7f4b39076640 1 -- 172.21.15.60:0/1403620945 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b300b2070 con 0x7f4b3415dea0 2024-04-25T10:16:33.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.066+0000 7f4b39076640 1 -- 172.21.15.60:0/1403620945 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b300ad640 con 0x7f4b3415dea0 2024-04-25T10:16:33.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.066+0000 7f4b39076640 1 -- 172.21.15.60:0/1403620945 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b300ada10 con 0x7f4b3415dea0 2024-04-25T10:16:33.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.066+0000 7f4b3c2dc640 1 -- 172.21.15.60:0/1403620945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3415dea0 msgr2=0x7f4b3406c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.066+0000 7f4b3c2dc640 1 --2- 172.21.15.60:0/1403620945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3415dea0 0x7f4b3406c140 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4b300097c0 tx=0x7f4b3001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.066+0000 7f4b3c2dc640 1 -- 172.21.15.60:0/1403620945 shutdown_connections 2024-04-25T10:16:33.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.066+0000 7f4b3c2dc640 1 --2- 172.21.15.60:0/1403620945 >> v2:172.21.15.100:3300/0 conn(0x7f4b340ce4f0 0x7f4b340d2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.066+0000 7f4b3c2dc640 1 --2- 172.21.15.60:0/1403620945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3415dea0 0x7f4b3406c140 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.066+0000 7f4b3c2dc640 1 -- 172.21.15.60:0/1403620945 >> 172.21.15.60:0/1403620945 conn(0x7f4b340d0480 msgr2=0x7f4b340d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.066+0000 7f4b3c2dc640 1 -- 172.21.15.60:0/1403620945 shutdown_connections 2024-04-25T10:16:33.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.066+0000 7f4b3c2dc640 1 -- 172.21.15.60:0/1403620945 wait complete. 2024-04-25T10:16:33.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.066+0000 7f4b3c2dc640 1 Processor -- start 2024-04-25T10:16:33.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.066+0000 7f4b3c2dc640 1 -- start start 2024-04-25T10:16:33.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b3c2dc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b340ce4f0 0x7f4b340c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b3c2dc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b340d3110 0x7f4b34154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b3c2dc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3415dea0 0x7f4b34153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b3c2dc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b340b7620 con 0x7f4b340d3110 2024-04-25T10:16:33.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b3c2dc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b340cf2f0 con 0x7f4b340ce4f0 2024-04-25T10:16:33.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b3c2dc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b340b77a0 con 0x7f4b3415dea0 2024-04-25T10:16:33.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b39877640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b340d3110 0x7f4b34154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b39877640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b340d3110 0x7f4b34154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39408/0 (socket says 172.21.15.60:39408) 2024-04-25T10:16:33.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b39877640 1 -- 172.21.15.60:0/3882537711 learned_addr learned my addr 172.21.15.60:0/3882537711 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:33.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b3a078640 1 --2- 172.21.15.60:0/3882537711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b340ce4f0 0x7f4b340c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b3a879640 1 --2- 172.21.15.60:0/3882537711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3415dea0 0x7f4b34153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b39877640 1 -- 172.21.15.60:0/3882537711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3415dea0 msgr2=0x7f4b34153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b39877640 1 --2- 172.21.15.60:0/3882537711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3415dea0 0x7f4b34153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b39877640 1 -- 172.21.15.60:0/3882537711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b340ce4f0 msgr2=0x7f4b340c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b39877640 1 --2- 172.21.15.60:0/3882537711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b340ce4f0 0x7f4b340c7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b39877640 1 -- 172.21.15.60:0/3882537711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b340c7a60 con 0x7f4b340d3110 2024-04-25T10:16:33.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b3a879640 1 --2- 172.21.15.60:0/3882537711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3415dea0 0x7f4b34153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:33.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b3a078640 1 --2- 172.21.15.60:0/3882537711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b340ce4f0 0x7f4b340c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:33.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b39877640 1 --2- 172.21.15.60:0/3882537711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b340d3110 0x7f4b34154ef0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400e870 tx=0x7f4b2400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b2affd640 1 -- 172.21.15.60:0/3882537711 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b24017070 con 0x7f4b340d3110 2024-04-25T10:16:33.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b2affd640 1 -- 172.21.15.60:0/3882537711 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2400c940 con 0x7f4b340d3110 2024-04-25T10:16:33.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b3c2dc640 1 -- 172.21.15.60:0/3882537711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b340c7d40 con 0x7f4b340d3110 2024-04-25T10:16:33.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b2affd640 1 -- 172.21.15.60:0/3882537711 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2400ccf0 con 0x7f4b340d3110 2024-04-25T10:16:33.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b3c2dc640 1 -- 172.21.15.60:0/3882537711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b341f9ca0 con 0x7f4b340d3110 2024-04-25T10:16:33.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b3c2dc640 1 -- 172.21.15.60:0/3882537711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b34074770 con 0x7f4b340d3110 2024-04-25T10:16:33.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b2affd640 1 -- 172.21.15.60:0/3882537711 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b2401a070 con 0x7f4b340d3110 2024-04-25T10:16:33.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b2affd640 1 --2- 172.21.15.60:0/3882537711 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b18066f60 0x7f4b18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b2affd640 1 -- 172.21.15.60:0/3882537711 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4b24085320 con 0x7f4b340d3110 2024-04-25T10:16:33.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.070+0000 7f4b3a078640 1 --2- 172.21.15.60:0/3882537711 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b18066f60 0x7f4b18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.074+0000 7f4b3a078640 1 --2- 172.21.15.60:0/3882537711 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b18066f60 0x7f4b18069420 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4b300af000 tx=0x7f4b30008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.074+0000 7f4b2affd640 1 -- 172.21.15.60:0/3882537711 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b240562d0 con 0x7f4b340d3110 2024-04-25T10:16:33.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.270+0000 7f4b3c2dc640 1 -- 172.21.15.60:0/3882537711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b340686e0 con 0x7f4b340d3110 2024-04-25T10:16:33.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.270+0000 7f4b2affd640 1 -- 172.21.15.60:0/3882537711 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f4b24055bd0 con 0x7f4b340d3110 2024-04-25T10:16:33.275 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:33.275 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2440975447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2440975447},{"type":"v1","addr":"172.21.15.100:6835","nonce":2440975447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2398080758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2398080758},{"type":"v1","addr":"172.21.15.60:6833","nonce":2398080758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2759861958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2759861958},{"type":"v1","addr":"172.21.15.100:6837","nonce":2759861958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:07.763322+0000","modified":"2024-04-25T10:16:31.387513+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9693},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9693":{"gid":9693,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/178428701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":178428701},{"type":"v1","addr":"172.21.15.60:6835","nonce":178428701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:16:33.276 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 99 2024-04-25T10:16:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.270+0000 7f4b3c2dc640 1 -- 172.21.15.60:0/3882537711 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b18066f60 msgr2=0x7f4b18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.270+0000 7f4b3c2dc640 1 --2- 172.21.15.60:0/3882537711 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b18066f60 0x7f4b18069420 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4b300af000 tx=0x7f4b30008040 comp rx=0 tx=0).stop 2024-04-25T10:16:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.274+0000 7f4b3c2dc640 1 -- 172.21.15.60:0/3882537711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b340d3110 msgr2=0x7f4b34154ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.274+0000 7f4b3c2dc640 1 --2- 172.21.15.60:0/3882537711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b340d3110 0x7f4b34154ef0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400e870 tx=0x7f4b2400ec70 comp rx=0 tx=0).stop 2024-04-25T10:16:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.274+0000 7f4b3c2dc640 1 -- 172.21.15.60:0/3882537711 shutdown_connections 2024-04-25T10:16:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.274+0000 7f4b3c2dc640 1 --2- 172.21.15.60:0/3882537711 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b18066f60 0x7f4b18069420 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.274+0000 7f4b3c2dc640 1 --2- 172.21.15.60:0/3882537711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3415dea0 0x7f4b34153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.274+0000 7f4b3c2dc640 1 --2- 172.21.15.60:0/3882537711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b340d3110 0x7f4b34154ef0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.274+0000 7f4b3c2dc640 1 --2- 172.21.15.60:0/3882537711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b340ce4f0 0x7f4b340c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.274+0000 7f4b3c2dc640 1 -- 172.21.15.60:0/3882537711 >> 172.21.15.60:0/3882537711 conn(0x7f4b340d0480 msgr2=0x7f4b341607a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.274+0000 7f4b3c2dc640 1 -- 172.21.15.60:0/3882537711 shutdown_connections 2024-04-25T10:16:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.274+0000 7f4b3c2dc640 1 -- 172.21.15.60:0/3882537711 wait complete. 2024-04-25T10:16:33.697 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:16:07.763322+0000', 'modified': '2024-04-25T10:16:31.387513+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 111, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9693}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9693': {'gid': 9693, 'name': 'c', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.60:6835/178428701', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 178428701}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 178428701}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:33.697 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:16:33.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:16:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.830+0000 7fc321491640 1 Processor -- start 2024-04-25T10:16:33.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.830+0000 7fc321491640 1 -- start start 2024-04-25T10:16:33.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.830+0000 7fc321491640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc31c15a7a0 0x7fc31c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.830+0000 7fc321491640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc31c0b37e0 0x7fc31c0b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.830+0000 7fc321491640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc31c06f130 con 0x7fc31c06ec30 2024-04-25T10:16:33.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.830+0000 7fc321491640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc31c0b4190 con 0x7fc31c0b37e0 2024-04-25T10:16:33.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.830+0000 7fc321491640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc31c0cdaa0 con 0x7fc31c15a7a0 2024-04-25T10:16:33.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.830+0000 7fc31b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc31c0b37e0 0x7fc31c0b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.830+0000 7fc31b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc31c0b37e0 0x7fc31c0b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39420/0 (socket says 172.21.15.60:39420) 2024-04-25T10:16:33.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.830+0000 7fc31b7fe640 1 -- 172.21.15.60:0/3335550308 learned_addr learned my addr 172.21.15.60:0/3335550308 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:33.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.830+0000 7fc31affd640 1 --2- 172.21.15.60:0/3335550308 >> v2:172.21.15.100:3300/0 conn(0x7fc31c15a7a0 0x7fc31c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.830+0000 7fc31b7fe640 1 -- 172.21.15.60:0/3335550308 >> v2:172.21.15.100:3300/0 conn(0x7fc31c15a7a0 msgr2=0x7fc31c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:33.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.830+0000 7fc31b7fe640 1 --2- 172.21.15.60:0/3335550308 >> v2:172.21.15.100:3300/0 conn(0x7fc31c15a7a0 0x7fc31c0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.830+0000 7fc31b7fe640 1 -- 172.21.15.60:0/3335550308 >> v1:172.21.15.100:6789/0 conn(0x7fc31c06ec30 legacy=0x7fc31c071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:33.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.830+0000 7fc31b7fe640 1 -- 172.21.15.60:0/3335550308 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31c06f410 con 0x7fc31c0b37e0 2024-04-25T10:16:33.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.834+0000 7fc31b7fe640 1 --2- 172.21.15.60:0/3335550308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c0b37e0 0x7fc31c0b3c50 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fc314009870 tx=0x7fc31401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edb910da6b32de3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.834+0000 7fc319ffb640 1 -- 172.21.15.60:0/3335550308 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3140ab430 con 0x7fc31c0b37e0 2024-04-25T10:16:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.834+0000 7fc319ffb640 1 -- 172.21.15.60:0/3335550308 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3140b0070 con 0x7fc31c0b37e0 2024-04-25T10:16:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.834+0000 7fc319ffb640 1 -- 172.21.15.60:0/3335550308 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3140ab7f0 con 0x7fc31c0b37e0 2024-04-25T10:16:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.834+0000 7fc321491640 1 -- 172.21.15.60:0/3335550308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c0b37e0 msgr2=0x7fc31c0b3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.834+0000 7fc321491640 1 --2- 172.21.15.60:0/3335550308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c0b37e0 0x7fc31c0b3c50 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fc314009870 tx=0x7fc31401c520 comp rx=0 tx=0).stop 2024-04-25T10:16:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.834+0000 7fc321491640 1 -- 172.21.15.60:0/3335550308 shutdown_connections 2024-04-25T10:16:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.834+0000 7fc321491640 1 --2- 172.21.15.60:0/3335550308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c0b37e0 0x7fc31c0b3c50 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.834+0000 7fc321491640 1 --2- 172.21.15.60:0/3335550308 >> v2:172.21.15.100:3300/0 conn(0x7fc31c15a7a0 0x7fc31c0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.834+0000 7fc321491640 1 -- 172.21.15.60:0/3335550308 >> 172.21.15.60:0/3335550308 conn(0x7fc31c0b12d0 msgr2=0x7fc31c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.834+0000 7fc321491640 1 -- 172.21.15.60:0/3335550308 shutdown_connections 2024-04-25T10:16:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.834+0000 7fc321491640 1 -- 172.21.15.60:0/3335550308 wait complete. 2024-04-25T10:16:33.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.834+0000 7fc321491640 1 Processor -- start 2024-04-25T10:16:33.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.834+0000 7fc321491640 1 -- start start 2024-04-25T10:16:33.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc321491640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc31c06ec30 0x7fc31c150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc321491640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c0b37e0 0x7fc31c15fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc321491640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc31c15a7a0 0x7fc31c162d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc321491640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c06e8c0 con 0x7fc31c0b37e0 2024-04-25T10:16:33.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc321491640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c06e740 con 0x7fc31c06ec30 2024-04-25T10:16:33.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc321491640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c06d8b0 con 0x7fc31c15a7a0 2024-04-25T10:16:33.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc31a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c0b37e0 0x7fc31c15fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc31a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c0b37e0 0x7fc31c15fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39430/0 (socket says 172.21.15.60:39430) 2024-04-25T10:16:33.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc31a7fc640 1 -- 172.21.15.60:0/2473173235 learned_addr learned my addr 172.21.15.60:0/2473173235 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:33.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc31b7fe640 1 --2- 172.21.15.60:0/2473173235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc31c15a7a0 0x7fc31c162d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc31affd640 1 --2- 172.21.15.60:0/2473173235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc31c06ec30 0x7fc31c150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc31a7fc640 1 -- 172.21.15.60:0/2473173235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc31c15a7a0 msgr2=0x7fc31c162d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc31a7fc640 1 --2- 172.21.15.60:0/2473173235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc31c15a7a0 0x7fc31c162d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc31a7fc640 1 -- 172.21.15.60:0/2473173235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc31c06ec30 msgr2=0x7fc31c150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc31a7fc640 1 --2- 172.21.15.60:0/2473173235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc31c06ec30 0x7fc31c150070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc31a7fc640 1 -- 172.21.15.60:0/2473173235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31c1505b0 con 0x7fc31c0b37e0 2024-04-25T10:16:33.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc31a7fc640 1 --2- 172.21.15.60:0/2473173235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c0b37e0 0x7fc31c15fe50 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fc31000c460 tx=0x7fc31000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc3037fe640 1 -- 172.21.15.60:0/2473173235 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310016020 con 0x7fc31c0b37e0 2024-04-25T10:16:33.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc3037fe640 1 -- 172.21.15.60:0/2473173235 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3100052d0 con 0x7fc31c0b37e0 2024-04-25T10:16:33.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc3037fe640 1 -- 172.21.15.60:0/2473173235 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310007260 con 0x7fc31c0b37e0 2024-04-25T10:16:33.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc321491640 1 -- 172.21.15.60:0/2473173235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31c160880 con 0x7fc31c0b37e0 2024-04-25T10:16:33.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc321491640 1 -- 172.21.15.60:0/2473173235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31c160e40 con 0x7fc31c0b37e0 2024-04-25T10:16:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.838+0000 7fc3037fe640 1 -- 172.21.15.60:0/2473173235 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc310007400 con 0x7fc31c0b37e0 2024-04-25T10:16:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3037fe640 1 --2- 172.21.15.60:0/2473173235 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc308066f90 0x7fc308069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3037fe640 1 -- 172.21.15.60:0/2473173235 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc310084950 con 0x7fc31c0b37e0 2024-04-25T10:16:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc31affd640 1 --2- 172.21.15.60:0/2473173235 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc308066f90 0x7fc308069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 Processor -- start 2024-04-25T10:16:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 -- start start 2024-04-25T10:16:33.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc31affd640 1 --2- 172.21.15.60:0/2473173235 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc308066f90 0x7fc308069450 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009e90 tx=0x7fc30c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e7920 0x5574153e7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e8240 0x55741539d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55741539df30 0x55741539e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574152855f0 con 0x5574153e8240 2024-04-25T10:16:33.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574151ecda0 con 0x55741539df30 2024-04-25T10:16:33.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x557415175150 con 0x5574153e7920 2024-04-25T10:16:33.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc31a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e8240 0x55741539d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc31a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e8240 0x55741539d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39432/0 (socket says 172.21.15.60:39432) 2024-04-25T10:16:33.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc31a7fc640 1 -- 172.21.15.60:0/3026447803 learned_addr learned my addr 172.21.15.60:0/3026447803 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:33.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc31b7fe640 1 --2- 172.21.15.60:0/3026447803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55741539df30 0x55741539e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc31a7fc640 1 -- 172.21.15.60:0/3026447803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e7920 msgr2=0x5574153e7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:33.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc31affd640 1 --2- 172.21.15.60:0/3026447803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e7920 0x5574153e7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc31a7fc640 1 --2- 172.21.15.60:0/3026447803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e7920 0x5574153e7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc31a7fc640 1 -- 172.21.15.60:0/3026447803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55741539df30 msgr2=0x55741539e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:33.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc31a7fc640 1 --2- 172.21.15.60:0/3026447803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55741539df30 0x55741539e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc31a7fc640 1 -- 172.21.15.60:0/3026447803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55741518eca0 con 0x5574153e8240 2024-04-25T10:16:33.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc31b7fe640 1 --2- 172.21.15.60:0/3026447803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55741539df30 0x55741539e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:33.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc31affd640 1 --2- 172.21.15.60:0/3026447803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e7920 0x5574153e7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:33.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc31a7fc640 1 --2- 172.21.15.60:0/3026447803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e8240 0x55741539d9f0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fc310082e50 tx=0x7fc310000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8eaa47ee63e3f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc300ff9640 1 -- 172.21.15.60:0/3026447803 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3100593e0 con 0x5574153e8240 2024-04-25T10:16:33.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc300ff9640 1 -- 172.21.15.60:0/3026447803 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31005d9a0 con 0x5574153e8240 2024-04-25T10:16:33.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc300ff9640 1 -- 172.21.15.60:0/3026447803 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310019df0 con 0x5574153e8240 2024-04-25T10:16:33.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 -- 172.21.15.60:0/3026447803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e8240 msgr2=0x55741539d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 --2- 172.21.15.60:0/3026447803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e8240 0x55741539d9f0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fc310082e50 tx=0x7fc310000bd0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 -- 172.21.15.60:0/3026447803 shutdown_connections 2024-04-25T10:16:33.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 --2- 172.21.15.60:0/3026447803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55741539df30 0x55741539e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 --2- 172.21.15.60:0/3026447803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e8240 0x55741539d9f0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 --2- 172.21.15.60:0/3026447803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e7920 0x5574153e7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 -- 172.21.15.60:0/3026447803 >> 172.21.15.60:0/3026447803 conn(0x5574153a42a0 msgr2=0x55741539c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 -- 172.21.15.60:0/3026447803 shutdown_connections 2024-04-25T10:16:33.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 -- 172.21.15.60:0/3026447803 wait complete. 2024-04-25T10:16:33.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 -- start start 2024-04-25T10:16:33.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.842+0000 7fc3245fe000 1 Processor -- start 2024-04-25T10:16:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55741539df30 0x5574153eedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574153e7920 0x5574153ef310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e8240 0x5574153a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574151a7990 con 0x55741539df30 2024-04-25T10:16:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574151e8e40 con 0x5574153e7920 2024-04-25T10:16:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x557415178900 con 0x5574153e8240 2024-04-25T10:16:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 10 client.0 _ll_register_callbacks cb 0x5574152a01c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc31affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55741539df30 0x5574153eedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc31affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55741539df30 0x5574153eedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39434/0 (socket says 172.21.15.60:39434) 2024-04-25T10:16:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc31affd640 1 -- 172.21.15.60:0/4205171594 learned_addr learned my addr 172.21.15.60:0/4205171594 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc31a7fc640 1 --2- 172.21.15.60:0/4205171594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574153e7920 0x5574153ef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc31b7fe640 1 --2- 172.21.15.60:0/4205171594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e8240 0x5574153a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc31affd640 1 -- 172.21.15.60:0/4205171594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e8240 msgr2=0x5574153a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc31affd640 1 --2- 172.21.15.60:0/4205171594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e8240 0x5574153a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc31affd640 1 -- 172.21.15.60:0/4205171594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574153e7920 msgr2=0x5574153ef310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc31affd640 1 --2- 172.21.15.60:0/4205171594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574153e7920 0x5574153ef310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc31affd640 1 -- 172.21.15.60:0/4205171594 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574151a5630 con 0x55741539df30 2024-04-25T10:16:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc31b7fe640 1 --2- 172.21.15.60:0/4205171594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e8240 0x5574153a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc31affd640 1 --2- 172.21.15.60:0/4205171594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55741539df30 0x5574153eedd0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fc30c016a50 tx=0x7fc30c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3c1fd2d7aceae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc2f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:16:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc2f9ffb640 1 -- 172.21.15.60:0/4205171594 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c014d60 con 0x55741539df30 2024-04-25T10:16:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc2f9ffb640 1 -- 172.21.15.60:0/4205171594 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc30c021410 con 0x55741539df30 2024-04-25T10:16:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 -- 172.21.15.60:0/4205171594 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc31005d9a0 con 0x55741539df30 2024-04-25T10:16:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc2f9ffb640 1 -- 172.21.15.60:0/4205171594 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc30c02e050 con 0x55741539df30 2024-04-25T10:16:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 10 client.9735 fetch_fsmap learned FSMap version 99 2024-04-25T10:16:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 -- 172.21.15.60:0/4205171594 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5574151753a0 con 0x55741539df30 2024-04-25T10:16:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc2f9ffb640 1 -- 172.21.15.60:0/4205171594 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31005d9a0 con 0x55741539df30 2024-04-25T10:16:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc2f9ffb640 1 -- 172.21.15.60:0/4205171594 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc30c031020 con 0x55741539df30 2024-04-25T10:16:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 10 client.9735 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:16:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 10 client.9735 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:16:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 20 client.9735 populate_metadata read hostname 'smithi060' 2024-04-25T10:16:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 --2- 172.21.15.60:0/4205171594 >> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] conn(0x5574153a24b0 0x5574153a2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 4 client.9735 mds_command: new command op to 9693 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:33.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 -- 172.21.15.60:0/4205171594 --> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557415188a50 con 0x5574153a24b0 2024-04-25T10:16:33.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc31a7fc640 1 --2- 172.21.15.60:0/4205171594 >> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] conn(0x5574153a24b0 0x5574153a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc31a7fc640 1 --2- 172.21.15.60:0/4205171594 >> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] conn(0x5574153a24b0 0x5574153a2890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81377bccbe65bec server_cookie=bd4c5b3ccac565ff in_seq=0 out_seq=0 2024-04-25T10:16:33.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc2f9ffb640 10 client.9735 ms_handle_connect on v2:172.21.15.60:6834/178428701 2024-04-25T10:16:33.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc2f9ffb640 1 -- 172.21.15.60:0/4205171594 <== mds.0 v2:172.21.15.60:6834/178428701 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557415188a50 con 0x5574153a24b0 2024-04-25T10:16:33.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc2f9ffb640 10 client.9735 handle_command_reply: tid=0 2024-04-25T10:16:33.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 client.9735 shutdown 2024-04-25T10:16:33.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 -- 172.21.15.60:0/4205171594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55741539df30 msgr2=0x5574153eedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 --2- 172.21.15.60:0/4205171594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55741539df30 0x5574153eedd0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fc30c016a50 tx=0x7fc30c004ac0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 -- 172.21.15.60:0/4205171594 shutdown_connections 2024-04-25T10:16:33.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 --2- 172.21.15.60:0/4205171594 >> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] conn(0x5574153a24b0 0x5574153a2890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 --2- 172.21.15.60:0/4205171594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e8240 0x5574153a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 --2- 172.21.15.60:0/4205171594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574153e7920 0x5574153ef310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc2f9ffb640 0 client.9735 ms_handle_reset on v2:172.21.15.60:6834/178428701 2024-04-25T10:16:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 --2- 172.21.15.60:0/4205171594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55741539df30 0x5574153eedd0 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 -- 172.21.15.60:0/4205171594 >> 172.21.15.60:0/4205171594 conn(0x5574153a42a0 msgr2=0x55741539c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 -- 172.21.15.60:0/4205171594 shutdown_connections 2024-04-25T10:16:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 1 -- 172.21.15.60:0/4205171594 wait complete. 2024-04-25T10:16:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.846+0000 7fc3245fe000 20 client.9735 trim_cache size 0 max 16384 2024-04-25T10:16:33.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.862+0000 7fc3245fe000 1 Processor -- start 2024-04-25T10:16:33.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.862+0000 7fc3245fe000 1 -- start start 2024-04-25T10:16:33.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.862+0000 7fc3245fe000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55741539df30 0x557415443120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.862+0000 7fc3245fe000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e7920 0x557415443660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.862+0000 7fc3245fe000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574153e8240 0x557415443ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.862+0000 7fc3245fe000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c1ffd50 con 0x5574153e7920 2024-04-25T10:16:33.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.862+0000 7fc3245fe000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c157250 con 0x5574153e8240 2024-04-25T10:16:33.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.862+0000 7fc3245fe000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c157850 con 0x55741539df30 2024-04-25T10:16:33.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e7920 0x557415443660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e7920 0x557415443660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39450/0 (socket says 172.21.15.60:39450) 2024-04-25T10:16:33.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 -- 172.21.15.60:0/268282728 learned_addr learned my addr 172.21.15.60:0/268282728 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:33.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31affd640 1 --2- 172.21.15.60:0/268282728 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55741539df30 0x557415443120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31b7fe640 1 --2- 172.21.15.60:0/268282728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574153e8240 0x557415443ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 -- 172.21.15.60:0/268282728 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55741539df30 msgr2=0x557415443120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:33.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 --2- 172.21.15.60:0/268282728 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55741539df30 0x557415443120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 -- 172.21.15.60:0/268282728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574153e8240 msgr2=0x557415443ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:33.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 --2- 172.21.15.60:0/268282728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574153e8240 0x557415443ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 -- 172.21.15.60:0/268282728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574153e8070 con 0x5574153e7920 2024-04-25T10:16:33.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 --2- 172.21.15.60:0/268282728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e7920 0x557415443660 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fc310084c70 tx=0x7fc310057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5f2e89c6578781 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc2fbfff640 1 -- 172.21.15.60:0/268282728 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310071040 con 0x5574153e7920 2024-04-25T10:16:33.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc2fbfff640 1 -- 172.21.15.60:0/268282728 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc310077030 con 0x5574153e7920 2024-04-25T10:16:33.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc2fbfff640 1 -- 172.21.15.60:0/268282728 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31005d9a0 con 0x5574153e7920 2024-04-25T10:16:33.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 -- 172.21.15.60:0/268282728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e7920 msgr2=0x557415443660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 --2- 172.21.15.60:0/268282728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e7920 0x557415443660 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fc310084c70 tx=0x7fc310057000 comp rx=0 tx=0).stop 2024-04-25T10:16:33.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 -- 172.21.15.60:0/268282728 shutdown_connections 2024-04-25T10:16:33.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 --2- 172.21.15.60:0/268282728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5574153e8240 0x557415443ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 --2- 172.21.15.60:0/268282728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e7920 0x557415443660 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 --2- 172.21.15.60:0/268282728 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55741539df30 0x557415443120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 -- 172.21.15.60:0/268282728 >> 172.21.15.60:0/268282728 conn(0x5574153a42a0 msgr2=0x5574153a1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 -- 172.21.15.60:0/268282728 shutdown_connections 2024-04-25T10:16:33.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 -- 172.21.15.60:0/268282728 wait complete. 2024-04-25T10:16:33.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 -- start start 2024-04-25T10:16:33.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 Processor -- start 2024-04-25T10:16:33.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55741539df30 0x5574153f4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e7920 0x5574153f4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e8240 0x557415444430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574151f13d0 con 0x5574153e7920 2024-04-25T10:16:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574152a8d50 con 0x55741539df30 2024-04-25T10:16:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x557415187980 con 0x5574153e8240 2024-04-25T10:16:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 10 client.0 _ll_register_callbacks cb 0x7fc310051530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e7920 0x5574153f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e7920 0x5574153f4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39462/0 (socket says 172.21.15.60:39462) 2024-04-25T10:16:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 -- 172.21.15.60:0/241784676 learned_addr learned my addr 172.21.15.60:0/241784676 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31b7fe640 1 --2- 172.21.15.60:0/241784676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e8240 0x557415444430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31affd640 1 --2- 172.21.15.60:0/241784676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55741539df30 0x5574153f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 -- 172.21.15.60:0/241784676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e8240 msgr2=0x557415444430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 --2- 172.21.15.60:0/241784676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e8240 0x557415444430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 -- 172.21.15.60:0/241784676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55741539df30 msgr2=0x5574153f4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 --2- 172.21.15.60:0/241784676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55741539df30 0x5574153f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 -- 172.21.15.60:0/241784676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55741526e600 con 0x5574153e7920 2024-04-25T10:16:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc31a7fc640 1 --2- 172.21.15.60:0/241784676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e7920 0x5574153f4850 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fc310055220 tx=0x7fc310000f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c44ee2531cbcf0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc2f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:16:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc2f9ffb640 1 -- 172.21.15.60:0/241784676 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310071040 con 0x5574153e7920 2024-04-25T10:16:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc2f9ffb640 1 -- 172.21.15.60:0/241784676 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc310053330 con 0x5574153e7920 2024-04-25T10:16:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 -- 172.21.15.60:0/241784676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc310077030 con 0x5574153e7920 2024-04-25T10:16:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc2f9ffb640 1 -- 172.21.15.60:0/241784676 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc31005d9a0 con 0x5574153e7920 2024-04-25T10:16:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 10 client.9741 fetch_fsmap learned FSMap version 99 2024-04-25T10:16:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc3245fe000 1 -- 172.21.15.60:0/241784676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5574151819b0 con 0x5574153e7920 2024-04-25T10:16:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.866+0000 7fc2f9ffb640 1 -- 172.21.15.60:0/241784676 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc310077030 con 0x5574153e7920 2024-04-25T10:16:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc2f9ffb640 1 -- 172.21.15.60:0/241784676 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc31006b070 con 0x5574153e7920 2024-04-25T10:16:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 10 client.9741 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:16:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 10 client.9741 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:16:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 20 client.9741 populate_metadata read hostname 'smithi060' 2024-04-25T10:16:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 1 --2- 172.21.15.60:0/241784676 >> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] conn(0x557415444970 0x557415444d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 4 client.9741 mds_command: new command op to 9693 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:16:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 1 -- 172.21.15.60:0/241784676 --> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5574153e75b0 con 0x557415444970 2024-04-25T10:16:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc31affd640 1 --2- 172.21.15.60:0/241784676 >> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] conn(0x557415444970 0x557415444d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc2f9ffb640 10 client.9741 ms_handle_connect on v2:172.21.15.60:6834/178428701 2024-04-25T10:16:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc31affd640 1 --2- 172.21.15.60:0/241784676 >> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] conn(0x557415444970 0x557415444d50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f21f01ffdcaa51 server_cookie=ed88ed4dbb0706b0 in_seq=0 out_seq=0 2024-04-25T10:16:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc2f9ffb640 1 -- 172.21.15.60:0/241784676 <== mds.0 v2:172.21.15.60:6834/178428701 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5574153e75b0 con 0x557415444970 2024-04-25T10:16:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc2f9ffb640 10 client.9741 handle_command_reply: tid=0 2024-04-25T10:16:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 1 client.9741 shutdown 2024-04-25T10:16:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 1 -- 172.21.15.60:0/241784676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e7920 msgr2=0x5574153f4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 1 --2- 172.21.15.60:0/241784676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e7920 0x5574153f4850 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fc310055220 tx=0x7fc310000f80 comp rx=0 tx=0).stop 2024-04-25T10:16:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 1 -- 172.21.15.60:0/241784676 shutdown_connections 2024-04-25T10:16:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 1 --2- 172.21.15.60:0/241784676 >> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] conn(0x557415444970 0x557415444d50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 1 --2- 172.21.15.60:0/241784676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5574153e8240 0x557415444430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 1 --2- 172.21.15.60:0/241784676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5574153e7920 0x5574153f4850 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc2f9ffb640 0 client.9741 ms_handle_reset on v2:172.21.15.60:6834/178428701 2024-04-25T10:16:33.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 1 --2- 172.21.15.60:0/241784676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55741539df30 0x5574153f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 1 -- 172.21.15.60:0/241784676 >> 172.21.15.60:0/241784676 conn(0x5574153a42a0 msgr2=0x5574153ef9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 1 -- 172.21.15.60:0/241784676 shutdown_connections 2024-04-25T10:16:33.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 1 -- 172.21.15.60:0/241784676 wait complete. 2024-04-25T10:16:33.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc3245fe000 20 client.9741 trim_cache size 0 max 16384 2024-04-25T10:16:33.880 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:16:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "0ea93863-a079-45c4-abed-6e83540ea9fe", 2024-04-25T10:16:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T10:16:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9693, 2024-04-25T10:16:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T10:16:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T10:16:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T10:16:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 4.493009625, 2024-04-25T10:16:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 99, 2024-04-25T10:16:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 111, 2024-04-25T10:16:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 111, 2024-04-25T10:16:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 5.4529833730000004 2024-04-25T10:16:33.881 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:16:33.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc321491640 1 -- 172.21.15.60:0/2473173235 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc308066f90 msgr2=0x7fc308069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc321491640 1 --2- 172.21.15.60:0/2473173235 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc308066f90 0x7fc308069450 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009e90 tx=0x7fc30c00b040 comp rx=0 tx=0).stop 2024-04-25T10:16:33.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc321491640 1 -- 172.21.15.60:0/2473173235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c0b37e0 msgr2=0x7fc31c15fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc321491640 1 --2- 172.21.15.60:0/2473173235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c0b37e0 0x7fc31c15fe50 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fc31000c460 tx=0x7fc31000c920 comp rx=0 tx=0).stop 2024-04-25T10:16:33.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc321491640 1 -- 172.21.15.60:0/2473173235 shutdown_connections 2024-04-25T10:16:33.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc321491640 1 --2- 172.21.15.60:0/2473173235 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc308066f90 0x7fc308069450 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc321491640 1 --2- 172.21.15.60:0/2473173235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc31c15a7a0 0x7fc31c162d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc321491640 1 --2- 172.21.15.60:0/2473173235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc31c0b37e0 0x7fc31c15fe50 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc321491640 1 --2- 172.21.15.60:0/2473173235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc31c06ec30 0x7fc31c150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc321491640 1 -- 172.21.15.60:0/2473173235 >> 172.21.15.60:0/2473173235 conn(0x7fc31c0b12d0 msgr2=0x7fc31c0b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc321491640 1 -- 172.21.15.60:0/2473173235 shutdown_connections 2024-04-25T10:16:33.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:33.870+0000 7fc321491640 1 -- 172.21.15.60:0/2473173235 wait complete. 2024-04-25T10:16:33.891 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:16:34.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.002+0000 7f7a039b8640 1 Processor -- start 2024-04-25T10:16:34.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.002+0000 7f7a039b8640 1 -- start start 2024-04-25T10:16:34.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.002+0000 7f7a039b8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79fc0b3610 0x7f79fc0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.002+0000 7f7a039b8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79fc0b3e60 con 0x7f79fc06ac10 2024-04-25T10:16:34.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.002+0000 7f7a039b8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79fc0b3b80 con 0x7f79fc0b54d0 2024-04-25T10:16:34.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.002+0000 7f7a039b8640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc0ae130 con 0x7f79fc0b3610 2024-04-25T10:16:34.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.002+0000 7f7a00f53640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79fc0b3610 0x7f79fc0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.002+0000 7f7a01754640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f79fc06ac10 0x7f79fc06aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56088/0 (socket says 172.21.15.60:56088) 2024-04-25T10:16:34.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.002+0000 7f7a01754640 1 -- 172.21.15.60:0/401833485 learned_addr learned my addr 172.21.15.60:0/401833485 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:34.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.002+0000 7f7a00f53640 1 -- 172.21.15.60:0/401833485 >> v1:172.21.15.100:6789/0 conn(0x7f79fc0b54d0 legacy=0x7f79fc0af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:34.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.002+0000 7f7a00f53640 1 -- 172.21.15.60:0/401833485 >> v1:172.21.15.60:6789/0 conn(0x7f79fc06ac10 legacy=0x7f79fc06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.002+0000 7f79f3fff640 1 -- 172.21.15.60:0/401833485 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026057324 0 0) 0x7f79fc0b3e60 con 0x7f79fc06ac10 2024-04-25T10:16:34.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.002+0000 7f7a00f53640 1 -- 172.21.15.60:0/401833485 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc0ae410 con 0x7f79fc0b3610 2024-04-25T10:16:34.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a00f53640 1 --2- 172.21.15.60:0/401833485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0b3610 0x7f79fc0aec10 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0097c0 tx=0x7f79ec01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e2f345d4177f6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:34.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f79f3fff640 1 -- 172.21.15.60:0/401833485 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0ab650 con 0x7f79fc0b3610 2024-04-25T10:16:34.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f79f3fff640 1 -- 172.21.15.60:0/401833485 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec0b0070 con 0x7f79fc0b3610 2024-04-25T10:16:34.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f79f3fff640 1 -- 172.21.15.60:0/401833485 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0aba10 con 0x7f79fc0b3610 2024-04-25T10:16:34.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a039b8640 1 -- 172.21.15.60:0/401833485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0b3610 msgr2=0x7f79fc0aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a039b8640 1 --2- 172.21.15.60:0/401833485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0b3610 0x7f79fc0aec10 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0097c0 tx=0x7f79ec01c430 comp rx=0 tx=0).stop 2024-04-25T10:16:34.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a039b8640 1 -- 172.21.15.60:0/401833485 shutdown_connections 2024-04-25T10:16:34.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a039b8640 1 --2- 172.21.15.60:0/401833485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0b3610 0x7f79fc0aec10 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a039b8640 1 -- 172.21.15.60:0/401833485 >> 172.21.15.60:0/401833485 conn(0x7f79fc072920 msgr2=0x7f79fc099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:34.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a039b8640 1 -- 172.21.15.60:0/401833485 shutdown_connections 2024-04-25T10:16:34.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a039b8640 1 -- 172.21.15.60:0/401833485 wait complete. 2024-04-25T10:16:34.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a039b8640 1 Processor -- start 2024-04-25T10:16:34.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a039b8640 1 -- start start 2024-04-25T10:16:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a039b8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79fc06ac10 0x7f79fc14fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a039b8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0b54d0 0x7f79fc14ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a039b8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79fc150110 0x7f79fc155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a039b8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0b5ac0 con 0x7f79fc0b54d0 2024-04-25T10:16:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a039b8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0b5940 con 0x7f79fc150110 2024-04-25T10:16:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a039b8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc15dd80 con 0x7f79fc06ac10 2024-04-25T10:16:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a00f53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0b54d0 0x7f79fc14ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a00f53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0b54d0 0x7f79fc14ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39486/0 (socket says 172.21.15.60:39486) 2024-04-25T10:16:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.006+0000 7f7a00f53640 1 -- 172.21.15.60:0/170314495 learned_addr learned my addr 172.21.15.60:0/170314495 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f7a01f55640 1 --2- 172.21.15.60:0/170314495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79fc150110 0x7f79fc155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f7a01754640 1 --2- 172.21.15.60:0/170314495 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79fc06ac10 0x7f79fc14fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f7a00f53640 1 -- 172.21.15.60:0/170314495 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79fc06ac10 msgr2=0x7f79fc14fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f7a00f53640 1 --2- 172.21.15.60:0/170314495 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79fc06ac10 0x7f79fc14fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f7a00f53640 1 -- 172.21.15.60:0/170314495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79fc150110 msgr2=0x7f79fc155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f7a00f53640 1 --2- 172.21.15.60:0/170314495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79fc150110 0x7f79fc155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f7a00f53640 1 -- 172.21.15.60:0/170314495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc15ee70 con 0x7f79fc0b54d0 2024-04-25T10:16:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f7a01f55640 1 --2- 172.21.15.60:0/170314495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79fc150110 0x7f79fc155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f7a00f53640 1 --2- 172.21.15.60:0/170314495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0b54d0 0x7f79fc14ceb0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0ad040 tx=0x7f79ec01ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f79f1ffb640 1 -- 172.21.15.60:0/170314495 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec004220 con 0x7f79fc0b54d0 2024-04-25T10:16:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f79f1ffb640 1 -- 172.21.15.60:0/170314495 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec0043c0 con 0x7f79fc0b54d0 2024-04-25T10:16:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f79f1ffb640 1 -- 172.21.15.60:0/170314495 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec003860 con 0x7f79fc0b54d0 2024-04-25T10:16:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f7a039b8640 1 -- 172.21.15.60:0/170314495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79fc15f150 con 0x7f79fc0b54d0 2024-04-25T10:16:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f7a039b8640 1 -- 172.21.15.60:0/170314495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79fc160010 con 0x7f79fc0b54d0 2024-04-25T10:16:34.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f7a039b8640 1 -- 172.21.15.60:0/170314495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79fc074770 con 0x7f79fc0b54d0 2024-04-25T10:16:34.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f79f1ffb640 1 -- 172.21.15.60:0/170314495 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79ec0bd070 con 0x7f79fc0b54d0 2024-04-25T10:16:34.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f79f1ffb640 1 --2- 172.21.15.60:0/170314495 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79e0066f60 0x7f79e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f79f1ffb640 1 -- 172.21.15.60:0/170314495 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f79ec11f680 con 0x7f79fc0b54d0 2024-04-25T10:16:34.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f7a01754640 1 --2- 172.21.15.60:0/170314495 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79e0066f60 0x7f79e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.010+0000 7f7a01754640 1 --2- 172.21.15.60:0/170314495 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79e0066f60 0x7f79e0069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f79f8000f70 tx=0x7f79f8007b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:34.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.014+0000 7f79f1ffb640 1 -- 172.21.15.60:0/170314495 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79ec016050 con 0x7f79fc0b54d0 2024-04-25T10:16:34.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.214+0000 7f7a039b8640 1 -- 172.21.15.60:0/170314495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f79fc14d3f0 con 0x7f79fc0b54d0 2024-04-25T10:16:34.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.214+0000 7f79f1ffb640 1 -- 172.21.15.60:0/170314495 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) v1 ==== 53+0+73 (secure 0 0 0) 0x7f79ec0effe0 con 0x7f79fc0b54d0 2024-04-25T10:16:34.218 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:16:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.214+0000 7f7a039b8640 1 -- 172.21.15.60:0/170314495 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79e0066f60 msgr2=0x7f79e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.214+0000 7f7a039b8640 1 --2- 172.21.15.60:0/170314495 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79e0066f60 0x7f79e0069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f79f8000f70 tx=0x7f79f8007b60 comp rx=0 tx=0).stop 2024-04-25T10:16:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.214+0000 7f7a039b8640 1 -- 172.21.15.60:0/170314495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0b54d0 msgr2=0x7f79fc14ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.214+0000 7f7a039b8640 1 --2- 172.21.15.60:0/170314495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0b54d0 0x7f79fc14ceb0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0ad040 tx=0x7f79ec01ac70 comp rx=0 tx=0).stop 2024-04-25T10:16:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.214+0000 7f7a039b8640 1 -- 172.21.15.60:0/170314495 shutdown_connections 2024-04-25T10:16:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.214+0000 7f7a039b8640 1 --2- 172.21.15.60:0/170314495 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f79e0066f60 0x7f79e0069420 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.214+0000 7f7a039b8640 1 --2- 172.21.15.60:0/170314495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79fc150110 0x7f79fc155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.214+0000 7f7a039b8640 1 --2- 172.21.15.60:0/170314495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79fc0b54d0 0x7f79fc14ceb0 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.214+0000 7f7a039b8640 1 --2- 172.21.15.60:0/170314495 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79fc06ac10 0x7f79fc14fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.214+0000 7f7a039b8640 1 -- 172.21.15.60:0/170314495 >> 172.21.15.60:0/170314495 conn(0x7f79fc072920 msgr2=0x7f79fc0ad580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.214+0000 7f7a039b8640 1 -- 172.21.15.60:0/170314495 shutdown_connections 2024-04-25T10:16:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:34.214+0000 7f7a039b8640 1 -- 172.21.15.60:0/170314495 wait complete. 2024-04-25T10:16:34.238 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:16:34.238 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:16:34.238 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:16:34.238 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:16:34.238 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:34.238 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:16:34.238 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:16:34.238 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:16:34.238 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:34.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:34.239 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:16:34.245 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:16:34.245 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:16:34.245 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:16:34.245 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:34.245 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:16:34.245 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:34.245 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:34.245 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:34.246 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:16:34.246 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:34.246 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:34.246 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:16:34.246 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:16:34.246 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:34.246 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:16:34.246 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:16:34.246 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41863sec preferred_lft 41863sec 2024-04-25T10:16:34.246 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:16:34.246 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:34.246 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:34.246 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:34.246 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:16:34.247 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:16:34.247 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:34.247 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:16:34.247 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:34.247 INFO:teuthology.orchestra.run.smithi060.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:16:34.247 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:34.247 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:16:34.247 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:34.247 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:16:34.247 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:34.247 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:34.247 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:34.247 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:16:34.247 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:34.248 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:34.248 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:34.248 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:16:34.248 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:34.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:34.248 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:16:34.299 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:16:34.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:34.301 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:16:34.355 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:16:34.355 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:16:34.355 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:16:34.356 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:16:34.356 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:16:34.356 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:16:34.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:34.356 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:16:34.356 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:16:34.356 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:16:34.356 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:16:34.356 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:16:34.356 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:16:34.382 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-04-25T10:16:34.378+0000 7f7af8fe6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:16:34.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:34.416 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:16:34.462 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:16:34.462 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:16:34.463 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:34.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:34.463 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T10:16:34.519 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:16:34.519 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:34.519 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:34.566 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:34.566 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:34.566 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:34.614 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:16:34.672 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:16:34.720 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:16:34.720 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:16:34.720 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:16:34.720 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:16:34.720 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:16:34.720 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:16:34.720 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:16:34.721 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:16:34.721 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:16:34.721 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:16:34.721 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:16:34.721 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:16:34.721 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:16:34.721 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:16:34.721 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:16:34.721 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:16:34.721 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:16:34.721 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:16:34.721 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:16:34.721 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:16:34.721 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:16:34.722 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:16:34.722 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:16:34.722 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:16:34.722 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:16:34.722 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:16:34.722 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:16:34.722 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:16:34.722 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:16:34.722 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:34.722 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:34.722 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:34.722 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:34.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:34.723 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:16:34.770 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:16:34.770 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:16:34.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:16:34.831 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:16:34.878 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:16:34.878 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:16:34.878 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:16:34.878 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:16:34.878 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:16:34.879 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:16:34.879 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:16:34.879 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:16:34.879 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:16:34.879 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:16:34.879 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:16:34.879 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:16:34.879 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:16:34.879 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:16:34.879 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:16:34.879 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:16:34.879 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:16:34.880 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:16:34.880 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:16:34.880 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:16:34.880 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:16:34.880 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:16:34.880 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:16:34.880 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:16:34.880 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:16:34.880 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:16:34.880 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:16:34.880 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:16:34.880 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:16:34.880 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:34.880 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:34.880 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:34.881 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:34.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:34.882 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:16:34.894 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2024-04-25T10:16:34.890+0000 7f2a05330580 -1 init, newargv = 0x55bd0b6b9320 newargc=15 2024-04-25T10:16:34.895 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[52745]: starting ceph client 2024-04-25T10:16:34.900 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[52745]: starting fuse 2024-04-25T10:16:36.000 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:16:36.016 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:16:36.069 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:16:36.069 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:16:36.069 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:16:36.069 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:16:36.069 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:16:36.069 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:16:36.069 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:16:36.069 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:16:36.069 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:16:36.069 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:16:36.069 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:16:36.069 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:16:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:16:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:16:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:16:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:16:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:16:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:16:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:16:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:16:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:16:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:16:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:16:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:16:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:16:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:16:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:16:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:16:36.071 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:16:36.071 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:36.071 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:36.071 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:36.071 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:36.071 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:36.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:36.072 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:16:36.118 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:16:36.118 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:16:36.118 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:16:36.118 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:16:36.118 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:16:36.118 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:36.118 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:16:36.119 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:36.119 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:16:36.119 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:16:36.119 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:16:36.119 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:16:36.119 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:36.119 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:16:36.119 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:16:36.119 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:16:36.119 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:16:36.119 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:36.119 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:16:36.119 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:16:36.119 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:16:36.119 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:16:36.119 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:16:36.120 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:16:36.120 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:16:36.120 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:16:36.120 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:36.120 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2024-04-25T10:16:36.120 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:36.246 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.52745.asok 2024-04-25T10:16:36.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:36.259 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52745.asok 2024-04-25T10:16:36.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:36.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52745.asok status 2024-04-25T10:16:36.418 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:16:36.418 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:16:36.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:16:36.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:16:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2024-04-25T10:16:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:16:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:16:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "52745", 2024-04-25T10:16:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:16:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:16:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:16:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9756, 2024-04-25T10:16:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:16:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:16:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:16:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 9756 2024-04-25T10:16:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:36.419 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 879790120 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 879790120 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.9756 192.168.0.1:0/879790120", 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/879790120", 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 99, 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 111, 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:16:36.420 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:16:36.421 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:16:36.421 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:16:36.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:36.434 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:16:36.434 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:36.441 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:16:36.441 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:36.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:36.441 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:36.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:36.497 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:16:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:./ 2024-04-25T10:16:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:./parent 2024-04-25T10:16:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:./parent/flushed 2024-04-25T10:16:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:./parent/flushed/charlie 2024-04-25T10:16:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:./parent/flushed/alpha 2024-04-25T10:16:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:./parent/unflushed 2024-04-25T10:16:36.560 INFO:teuthology.orchestra.run.smithi060.stdout:./parent/unflushed/jfile 2024-04-25T10:16:36.561 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:36.561 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:36.561 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:36.562 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:36.562 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:16:36.562 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:36.673 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:16:36.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:36.684 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:36.684 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:36.684 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:36.684 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:36.684 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:16:36.684 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:36.788 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627776 2024-04-25T10:16:36.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:36.799 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:36.799 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:36.799 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:36.799 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:36.799 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:16:36.799 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:36.909 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627777 2024-04-25T10:16:36.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:36.919 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:36.919 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:36.919 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:36.919 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:36.920 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:16:36.920 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:37.027 INFO:teuthology.orchestra.run.smithi060.stdout:1099511628280 2024-04-25T10:16:37.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:37.038 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:37.038 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:37.038 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:37.038 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:37.038 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:16:37.038 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:37.144 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627778 2024-04-25T10:16:37.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:37.154 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:37.154 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:37.155 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:37.155 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:37.155 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:16:37.155 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:37.265 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627779 2024-04-25T10:16:37.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:37.277 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:37.277 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:37.277 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:37.277 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:37.277 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:16:37.277 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:37.381 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627780 2024-04-25T10:16:37.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:37.391 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:16:37.391 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:37.438 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:37.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.606+0000 7fce7340e640 1 Processor -- start 2024-04-25T10:16:37.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.606+0000 7fce7340e640 1 -- start start 2024-04-25T10:16:37.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce7340e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce6c06ec30 0x7fce6c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce7340e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6c06f130 con 0x7fce6c15a7a0 2024-04-25T10:16:37.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce7340e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6c06f410 con 0x7fce6c0b37e0 2024-04-25T10:16:37.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce7340e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0cdaa0 con 0x7fce6c06ec30 2024-04-25T10:16:37.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce711aa640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fce6c15a7a0 0x7fce6c071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56104/0 (socket says 172.21.15.60:56104) 2024-04-25T10:16:37.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce711aa640 1 -- 172.21.15.60:0/2308853003 learned_addr learned my addr 172.21.15.60:0/2308853003 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:37.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce709a9640 1 --2- 172.21.15.60:0/2308853003 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce6c06ec30 0x7fce6c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce63fff640 1 -- 172.21.15.60:0/2308853003 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256258470 0 0) 0x7fce6c06f130 con 0x7fce6c15a7a0 2024-04-25T10:16:37.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce63fff640 1 -- 172.21.15.60:0/2308853003 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce58003530 con 0x7fce6c15a7a0 2024-04-25T10:16:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce63fff640 1 -- 172.21.15.60:0/2308853003 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 644433731 0 0) 0x7fce58003530 con 0x7fce6c15a7a0 2024-04-25T10:16:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce63fff640 1 -- 172.21.15.60:0/2308853003 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce6c06ec30 msgr2=0x7fce6c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce63fff640 1 --2- 172.21.15.60:0/2308853003 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce6c06ec30 0x7fce6c0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce63fff640 1 -- 172.21.15.60:0/2308853003 >> v1:172.21.15.100:6789/0 conn(0x7fce6c0b37e0 legacy=0x7fce6c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce63fff640 1 -- 172.21.15.60:0/2308853003 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c06d540 con 0x7fce6c15a7a0 2024-04-25T10:16:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce63fff640 1 -- 172.21.15.60:0/2308853003 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fce68001680 con 0x7fce6c15a7a0 2024-04-25T10:16:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce63fff640 1 -- 172.21.15.60:0/2308853003 >> v1:172.21.15.60:6789/0 conn(0x7fce6c15a7a0 legacy=0x7fce6c071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce63fff640 1 --2- 172.21.15.60:0/2308853003 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce6c06ec30 0x7fce580039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce63fff640 1 --2- 172.21.15.60:0/2308853003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce58003f70 0x7fce58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce63fff640 1 --2- 172.21.15.60:0/2308853003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce58004d50 0x7fce580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce63fff640 1 -- 172.21.15.60:0/2308853003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5800ba80 con 0x7fce58004d50 2024-04-25T10:16:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce63fff640 1 -- 172.21.15.60:0/2308853003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5800bc30 con 0x7fce58003f70 2024-04-25T10:16:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce63fff640 1 -- 172.21.15.60:0/2308853003 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5800be30 con 0x7fce6c06ec30 2024-04-25T10:16:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce719ab640 1 --2- 172.21.15.60:0/2308853003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce58004d50 0x7fce580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce711aa640 1 --2- 172.21.15.60:0/2308853003 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce6c06ec30 0x7fce580039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce709a9640 1 --2- 172.21.15.60:0/2308853003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce58003f70 0x7fce58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce719ab640 1 -- 172.21.15.60:0/2308853003 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce6c06ec30 msgr2=0x7fce580039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce719ab640 1 --2- 172.21.15.60:0/2308853003 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce6c06ec30 0x7fce580039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce719ab640 1 -- 172.21.15.60:0/2308853003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce58003f70 msgr2=0x7fce58004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce719ab640 1 --2- 172.21.15.60:0/2308853003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce58003f70 0x7fce58004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.610+0000 7fce719ab640 1 -- 172.21.15.60:0/2308853003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce58003530 con 0x7fce58004d50 2024-04-25T10:16:37.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce719ab640 1 --2- 172.21.15.60:0/2308853003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce58004d50 0x7fce580051d0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fce64001e70 tx=0x7fce6401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c2c46b54d2e345 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce63fff640 1 -- 172.21.15.60:0/2308853003 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64001a90 con 0x7fce58004d50 2024-04-25T10:16:37.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce63fff640 1 -- 172.21.15.60:0/2308853003 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce64003b80 con 0x7fce58004d50 2024-04-25T10:16:37.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce63fff640 1 -- 172.21.15.60:0/2308853003 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64005080 con 0x7fce58004d50 2024-04-25T10:16:37.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce7340e640 1 -- 172.21.15.60:0/2308853003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce58004d50 msgr2=0x7fce580051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce7340e640 1 --2- 172.21.15.60:0/2308853003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce58004d50 0x7fce580051d0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fce64001e70 tx=0x7fce6401c780 comp rx=0 tx=0).stop 2024-04-25T10:16:37.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce719ab640 1 -- 172.21.15.60:0/2308853003 reap_dead start 2024-04-25T10:16:37.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce7340e640 1 -- 172.21.15.60:0/2308853003 shutdown_connections 2024-04-25T10:16:37.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce7340e640 1 -- 172.21.15.60:0/2308853003 >> 172.21.15.60:0/2308853003 conn(0x7fce6c0b12d0 msgr2=0x7fce6c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:37.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce7340e640 1 -- 172.21.15.60:0/2308853003 shutdown_connections 2024-04-25T10:16:37.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce7340e640 1 -- 172.21.15.60:0/2308853003 wait complete. 2024-04-25T10:16:37.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce7340e640 1 Processor -- start 2024-04-25T10:16:37.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce7340e640 1 -- start start 2024-04-25T10:16:37.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce7340e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce6c155000 0x7fce6c0c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce7340e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce6c0c4a80 0x7fce6c1f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce7340e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce6c1553e0 0x7fce6c1f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce7340e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c06d8b0 con 0x7fce6c1553e0 2024-04-25T10:16:37.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce7340e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c06d2f0 con 0x7fce6c0c4a80 2024-04-25T10:16:37.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce7340e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0cd560 con 0x7fce6c155000 2024-04-25T10:16:37.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce719ab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce6c1553e0 0x7fce6c1f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce719ab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce6c1553e0 0x7fce6c1f6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39506/0 (socket says 172.21.15.60:39506) 2024-04-25T10:16:37.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce719ab640 1 -- 172.21.15.60:0/4221886396 learned_addr learned my addr 172.21.15.60:0/4221886396 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:37.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce711aa640 1 --2- 172.21.15.60:0/4221886396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce6c155000 0x7fce6c0c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce709a9640 1 --2- 172.21.15.60:0/4221886396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce6c0c4a80 0x7fce6c1f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce719ab640 1 -- 172.21.15.60:0/4221886396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce6c155000 msgr2=0x7fce6c0c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce719ab640 1 --2- 172.21.15.60:0/4221886396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce6c155000 0x7fce6c0c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce719ab640 1 -- 172.21.15.60:0/4221886396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce6c0c4a80 msgr2=0x7fce6c1f6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce719ab640 1 --2- 172.21.15.60:0/4221886396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce6c0c4a80 0x7fce6c1f6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.614+0000 7fce719ab640 1 -- 172.21.15.60:0/4221886396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c15cac0 con 0x7fce6c1553e0 2024-04-25T10:16:37.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.618+0000 7fce719ab640 1 --2- 172.21.15.60:0/4221886396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce6c1553e0 0x7fce6c1f6cc0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fce6401cc60 tx=0x7fce58005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.618+0000 7fce61ffb640 1 -- 172.21.15.60:0/4221886396 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce640b8070 con 0x7fce6c1553e0 2024-04-25T10:16:37.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.618+0000 7fce61ffb640 1 -- 172.21.15.60:0/4221886396 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce64005820 con 0x7fce6c1553e0 2024-04-25T10:16:37.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.618+0000 7fce7340e640 1 -- 172.21.15.60:0/4221886396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6c15cda0 con 0x7fce6c1553e0 2024-04-25T10:16:37.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.618+0000 7fce61ffb640 1 -- 172.21.15.60:0/4221886396 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64001a90 con 0x7fce6c1553e0 2024-04-25T10:16:37.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.618+0000 7fce7340e640 1 -- 172.21.15.60:0/4221886396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6c15d360 con 0x7fce6c1553e0 2024-04-25T10:16:37.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.618+0000 7fce7340e640 1 -- 172.21.15.60:0/4221886396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6c074770 con 0x7fce6c1553e0 2024-04-25T10:16:37.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.618+0000 7fce61ffb640 1 -- 172.21.15.60:0/4221886396 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce640bb070 con 0x7fce6c1553e0 2024-04-25T10:16:37.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.618+0000 7fce61ffb640 1 --2- 172.21.15.60:0/4221886396 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fce50066f60 0x7fce50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.618+0000 7fce61ffb640 1 -- 172.21.15.60:0/4221886396 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fce6411e320 con 0x7fce6c1553e0 2024-04-25T10:16:37.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.618+0000 7fce711aa640 1 --2- 172.21.15.60:0/4221886396 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fce50066f60 0x7fce50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.618+0000 7fce711aa640 1 --2- 172.21.15.60:0/4221886396 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fce50066f60 0x7fce50069420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fce68000c10 tx=0x7fce68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.622+0000 7fce61ffb640 1 -- 172.21.15.60:0/4221886396 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce640ac020 con 0x7fce6c1553e0 2024-04-25T10:16:37.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.786+0000 7fce7340e640 1 -- 172.21.15.60:0/4221886396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce6c074220 con 0x7fce6c1553e0 2024-04-25T10:16:37.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.786+0000 7fce61ffb640 1 -- 172.21.15.60:0/4221886396 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16766 (secure 0 0 0) 0x7fce64016490 con 0x7fce6c1553e0 2024-04-25T10:16:37.793 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:37.793 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":111,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:16:28.355822+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:06.033387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:06.753421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/722171246":"2024-04-26T10:16:28.355776+0000","172.21.15.60:6834/722171246":"2024-04-26T10:16:28.355776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:37.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.790+0000 7fce7340e640 1 -- 172.21.15.60:0/4221886396 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fce50066f60 msgr2=0x7fce50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.790+0000 7fce7340e640 1 --2- 172.21.15.60:0/4221886396 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fce50066f60 0x7fce50069420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fce68000c10 tx=0x7fce68009040 comp rx=0 tx=0).stop 2024-04-25T10:16:37.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.790+0000 7fce7340e640 1 -- 172.21.15.60:0/4221886396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce6c1553e0 msgr2=0x7fce6c1f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.790+0000 7fce7340e640 1 --2- 172.21.15.60:0/4221886396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce6c1553e0 0x7fce6c1f6cc0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fce6401cc60 tx=0x7fce58005840 comp rx=0 tx=0).stop 2024-04-25T10:16:37.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.790+0000 7fce7340e640 1 -- 172.21.15.60:0/4221886396 shutdown_connections 2024-04-25T10:16:37.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.790+0000 7fce7340e640 1 --2- 172.21.15.60:0/4221886396 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fce50066f60 0x7fce50069420 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.790+0000 7fce7340e640 1 --2- 172.21.15.60:0/4221886396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce6c1553e0 0x7fce6c1f6cc0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.790+0000 7fce7340e640 1 --2- 172.21.15.60:0/4221886396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce6c0c4a80 0x7fce6c1f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.790+0000 7fce7340e640 1 --2- 172.21.15.60:0/4221886396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce6c155000 0x7fce6c0c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.790+0000 7fce7340e640 1 -- 172.21.15.60:0/4221886396 >> 172.21.15.60:0/4221886396 conn(0x7fce6c0b12d0 msgr2=0x7fce6c0cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:37.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.790+0000 7fce7340e640 1 -- 172.21.15.60:0/4221886396 shutdown_connections 2024-04-25T10:16:37.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.790+0000 7fce7340e640 1 -- 172.21.15.60:0/4221886396 wait complete. 2024-04-25T10:16:37.812 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:06.033387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:06.753421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:37.813 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:37.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.938+0000 7eff2f6b3640 1 Processor -- start 2024-04-25T10:16:37.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.938+0000 7eff2f6b3640 1 -- start start 2024-04-25T10:16:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.938+0000 7eff2f6b3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7eff2815dd30 0x7eff2815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.938+0000 7eff2f6b3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7eff280cd3d0 0x7eff280cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.938+0000 7eff2f6b3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff280cd890 con 0x7eff2806f310 2024-04-25T10:16:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.938+0000 7eff2f6b3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff280ce4a0 con 0x7eff2815dd30 2024-04-25T10:16:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.938+0000 7eff2f6b3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff280cdb70 con 0x7eff280cd3d0 2024-04-25T10:16:37.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.938+0000 7eff2d44f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7eff2815dd30 0x7eff2815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.938+0000 7eff2d44f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7eff2815dd30 0x7eff2815ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39522/0 (socket says 172.21.15.60:39522) 2024-04-25T10:16:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.938+0000 7eff2d44f640 1 -- 172.21.15.60:0/2242340853 learned_addr learned my addr 172.21.15.60:0/2242340853 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.938+0000 7eff2cc4e640 1 --2- 172.21.15.60:0/2242340853 >> v2:172.21.15.100:3300/0 conn(0x7eff280cd3d0 0x7eff280cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.938+0000 7eff2d44f640 1 -- 172.21.15.60:0/2242340853 >> v2:172.21.15.100:3300/0 conn(0x7eff280cd3d0 msgr2=0x7eff280cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.938+0000 7eff2d44f640 1 --2- 172.21.15.60:0/2242340853 >> v2:172.21.15.100:3300/0 conn(0x7eff280cd3d0 0x7eff280cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.938+0000 7eff2d44f640 1 -- 172.21.15.60:0/2242340853 >> v1:172.21.15.100:6789/0 conn(0x7eff2806f310 legacy=0x7eff2806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.938+0000 7eff2d44f640 1 -- 172.21.15.60:0/2242340853 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff280cde50 con 0x7eff2815dd30 2024-04-25T10:16:37.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.938+0000 7eff2cc4e640 1 --2- 172.21.15.60:0/2242340853 >> v2:172.21.15.100:3300/0 conn(0x7eff280cd3d0 0x7eff280cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:37.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2d44f640 1 --2- 172.21.15.60:0/2242340853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff2815dd30 0x7eff2815ac30 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7eff28072550 tx=0x7eff2401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c1482c3dd11c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff1ffff640 1 -- 172.21.15.60:0/2242340853 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff240ab640 con 0x7eff2815dd30 2024-04-25T10:16:37.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff1ffff640 1 -- 172.21.15.60:0/2242340853 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff240b0070 con 0x7eff2815dd30 2024-04-25T10:16:37.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff1ffff640 1 -- 172.21.15.60:0/2242340853 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff240aba00 con 0x7eff2815dd30 2024-04-25T10:16:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2f6b3640 1 -- 172.21.15.60:0/2242340853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff2815dd30 msgr2=0x7eff2815ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2f6b3640 1 --2- 172.21.15.60:0/2242340853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff2815dd30 0x7eff2815ac30 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7eff28072550 tx=0x7eff2401c420 comp rx=0 tx=0).stop 2024-04-25T10:16:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2f6b3640 1 -- 172.21.15.60:0/2242340853 shutdown_connections 2024-04-25T10:16:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2f6b3640 1 --2- 172.21.15.60:0/2242340853 >> v2:172.21.15.100:3300/0 conn(0x7eff280cd3d0 0x7eff280cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2f6b3640 1 --2- 172.21.15.60:0/2242340853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff2815dd30 0x7eff2815ac30 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2f6b3640 1 -- 172.21.15.60:0/2242340853 >> 172.21.15.60:0/2242340853 conn(0x7eff280670b0 msgr2=0x7eff2806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2f6b3640 1 -- 172.21.15.60:0/2242340853 shutdown_connections 2024-04-25T10:16:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2f6b3640 1 -- 172.21.15.60:0/2242340853 wait complete. 2024-04-25T10:16:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2f6b3640 1 Processor -- start 2024-04-25T10:16:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2f6b3640 1 -- start start 2024-04-25T10:16:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2f6b3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff2806f310 0x7eff28161000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2f6b3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff280cd3d0 0x7eff2815f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2f6b3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff2815dd30 0x7eff2815fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2f6b3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff281674b0 con 0x7eff2815dd30 2024-04-25T10:16:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2f6b3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff280cfae0 con 0x7eff2806f310 2024-04-25T10:16:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2f6b3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff28167630 con 0x7eff280cd3d0 2024-04-25T10:16:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2dc50640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff2815dd30 0x7eff2815fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2dc50640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff2815dd30 0x7eff2815fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39538/0 (socket says 172.21.15.60:39538) 2024-04-25T10:16:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2dc50640 1 -- 172.21.15.60:0/361891170 learned_addr learned my addr 172.21.15.60:0/361891170 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2d44f640 1 --2- 172.21.15.60:0/361891170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff2806f310 0x7eff28161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2cc4e640 1 --2- 172.21.15.60:0/361891170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff280cd3d0 0x7eff2815f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2dc50640 1 -- 172.21.15.60:0/361891170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff280cd3d0 msgr2=0x7eff2815f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2dc50640 1 --2- 172.21.15.60:0/361891170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff280cd3d0 0x7eff2815f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2dc50640 1 -- 172.21.15.60:0/361891170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff2806f310 msgr2=0x7eff28161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2dc50640 1 --2- 172.21.15.60:0/361891170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff2806f310 0x7eff28161000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.942+0000 7eff2dc50640 1 -- 172.21.15.60:0/361891170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff280c44d0 con 0x7eff2815dd30 2024-04-25T10:16:37.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.946+0000 7eff2dc50640 1 --2- 172.21.15.60:0/361891170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff2815dd30 0x7eff2815fea0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7eff2000b9e0 tx=0x7eff2000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.946+0000 7eff1dffb640 1 -- 172.21.15.60:0/361891170 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20003b80 con 0x7eff2815dd30 2024-04-25T10:16:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.946+0000 7eff1dffb640 1 -- 172.21.15.60:0/361891170 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff20003d20 con 0x7eff2815dd30 2024-04-25T10:16:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.946+0000 7eff2f6b3640 1 -- 172.21.15.60:0/361891170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff281628b0 con 0x7eff2815dd30 2024-04-25T10:16:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.946+0000 7eff1dffb640 1 -- 172.21.15.60:0/361891170 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2000cb60 con 0x7eff2815dd30 2024-04-25T10:16:37.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.946+0000 7eff2f6b3640 1 -- 172.21.15.60:0/361891170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff280c4a50 con 0x7eff2815dd30 2024-04-25T10:16:37.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.946+0000 7eff2f6b3640 1 -- 172.21.15.60:0/361891170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff28074770 con 0x7eff2815dd30 2024-04-25T10:16:37.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.946+0000 7eff1dffb640 1 -- 172.21.15.60:0/361891170 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff20006d10 con 0x7eff2815dd30 2024-04-25T10:16:37.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.946+0000 7eff1dffb640 1 --2- 172.21.15.60:0/361891170 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7eff0c066f60 0x7eff0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.946+0000 7eff1dffb640 1 -- 172.21.15.60:0/361891170 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7eff20084be0 con 0x7eff2815dd30 2024-04-25T10:16:37.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.946+0000 7eff2d44f640 1 --2- 172.21.15.60:0/361891170 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7eff0c066f60 0x7eff0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.946+0000 7eff2d44f640 1 --2- 172.21.15.60:0/361891170 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7eff0c066f60 0x7eff0c069420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7eff240af000 tx=0x7eff24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:37.950+0000 7eff1dffb640 1 -- 172.21.15.60:0/361891170 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff20055b70 con 0x7eff2815dd30 2024-04-25T10:16:38.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.110+0000 7eff2f6b3640 1 -- 172.21.15.60:0/361891170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7eff28158ae0 con 0x7eff2815dd30 2024-04-25T10:16:38.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.110+0000 7eff1dffb640 1 -- 172.21.15.60:0/361891170 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7eff20055470 con 0x7eff2815dd30 2024-04-25T10:16:38.117 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:16:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.114+0000 7eff2f6b3640 1 -- 172.21.15.60:0/361891170 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7eff0c066f60 msgr2=0x7eff0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.114+0000 7eff2f6b3640 1 --2- 172.21.15.60:0/361891170 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7eff0c066f60 0x7eff0c069420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7eff240af000 tx=0x7eff24008040 comp rx=0 tx=0).stop 2024-04-25T10:16:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.114+0000 7eff2f6b3640 1 -- 172.21.15.60:0/361891170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff2815dd30 msgr2=0x7eff2815fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.114+0000 7eff2f6b3640 1 --2- 172.21.15.60:0/361891170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff2815dd30 0x7eff2815fea0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7eff2000b9e0 tx=0x7eff2000bea0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.114+0000 7eff2f6b3640 1 -- 172.21.15.60:0/361891170 shutdown_connections 2024-04-25T10:16:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.114+0000 7eff2f6b3640 1 --2- 172.21.15.60:0/361891170 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7eff0c066f60 0x7eff0c069420 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.114+0000 7eff2f6b3640 1 --2- 172.21.15.60:0/361891170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff2815dd30 0x7eff2815fea0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.114+0000 7eff2f6b3640 1 --2- 172.21.15.60:0/361891170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff280cd3d0 0x7eff2815f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.114+0000 7eff2f6b3640 1 --2- 172.21.15.60:0/361891170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff2806f310 0x7eff28161000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.114+0000 7eff2f6b3640 1 -- 172.21.15.60:0/361891170 >> 172.21.15.60:0/361891170 conn(0x7eff280670b0 msgr2=0x7eff28154ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.114+0000 7eff2f6b3640 1 -- 172.21.15.60:0/361891170 shutdown_connections 2024-04-25T10:16:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.114+0000 7eff2f6b3640 1 -- 172.21.15.60:0/361891170 wait complete. 2024-04-25T10:16:38.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:16:38.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.254+0000 7fe6d4162640 1 Processor -- start 2024-04-25T10:16:38.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.258+0000 7fe6d4162640 1 -- start start 2024-04-25T10:16:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.258+0000 7fe6d4162640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe6cc15dd30 0x7fe6cc15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.258+0000 7fe6d4162640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6cc0cd890 con 0x7fe6cc0cd3d0 2024-04-25T10:16:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.258+0000 7fe6d4162640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6cc0cdb70 con 0x7fe6cc0cf8a0 2024-04-25T10:16:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.258+0000 7fe6d4162640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6cc0ce4a0 con 0x7fe6cc15dd30 2024-04-25T10:16:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.258+0000 7fe6d1efe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe6cc15dd30 0x7fe6cc15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.258+0000 7fe6d16fd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe6cc0cd3d0 0x7fe6cc0cf190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56108/0 (socket says 172.21.15.60:56108) 2024-04-25T10:16:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.258+0000 7fe6d16fd640 1 -- 172.21.15.60:0/706089408 learned_addr learned my addr 172.21.15.60:0/706089408 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:38.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.258+0000 7fe6d1efe640 1 -- 172.21.15.60:0/706089408 >> v1:172.21.15.100:6789/0 conn(0x7fe6cc0cf8a0 legacy=0x7fe6cc15be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:38.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.258+0000 7fe6d1efe640 1 -- 172.21.15.60:0/706089408 >> v1:172.21.15.60:6789/0 conn(0x7fe6cc0cd3d0 legacy=0x7fe6cc0cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.258+0000 7fe6d0efc640 1 -- 172.21.15.60:0/706089408 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825845579 0 0) 0x7fe6cc0cd890 con 0x7fe6cc0cd3d0 2024-04-25T10:16:38.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.258+0000 7fe6d1efe640 1 -- 172.21.15.60:0/706089408 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc0cde50 con 0x7fe6cc15dd30 2024-04-25T10:16:38.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d1efe640 1 --2- 172.21.15.60:0/706089408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6cc15dd30 0x7fe6cc15ac30 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80097c0 tx=0x7fe6c801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc0b8a86c376245 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d0efc640 1 -- 172.21.15.60:0/706089408 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c80ab600 con 0x7fe6cc15dd30 2024-04-25T10:16:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d0efc640 1 -- 172.21.15.60:0/706089408 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c80b0070 con 0x7fe6cc15dd30 2024-04-25T10:16:38.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d0efc640 1 -- 172.21.15.60:0/706089408 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c80ab9c0 con 0x7fe6cc15dd30 2024-04-25T10:16:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d4162640 1 -- 172.21.15.60:0/706089408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6cc15dd30 msgr2=0x7fe6cc15ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d4162640 1 --2- 172.21.15.60:0/706089408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6cc15dd30 0x7fe6cc15ac30 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80097c0 tx=0x7fe6c801c3e0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d4162640 1 -- 172.21.15.60:0/706089408 shutdown_connections 2024-04-25T10:16:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d4162640 1 --2- 172.21.15.60:0/706089408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6cc15dd30 0x7fe6cc15ac30 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d4162640 1 -- 172.21.15.60:0/706089408 >> 172.21.15.60:0/706089408 conn(0x7fe6cc0670b0 msgr2=0x7fe6cc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d4162640 1 -- 172.21.15.60:0/706089408 shutdown_connections 2024-04-25T10:16:38.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d4162640 1 -- 172.21.15.60:0/706089408 wait complete. 2024-04-25T10:16:38.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d4162640 1 Processor -- start 2024-04-25T10:16:38.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d4162640 1 -- start start 2024-04-25T10:16:38.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d4162640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6cc0cd3d0 0x7fe6cc0c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d4162640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6cc0cf8a0 0x7fe6cc155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d4162640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6cc15dd30 0x7fe6cc0afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d4162640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc071250 con 0x7fe6cc0cd3d0 2024-04-25T10:16:38.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d4162640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc15cfd0 con 0x7fe6cc0cf8a0 2024-04-25T10:16:38.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d4162640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc0713d0 con 0x7fe6cc15dd30 2024-04-25T10:16:38.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d1efe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6cc0cd3d0 0x7fe6cc0c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d1efe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6cc0cd3d0 0x7fe6cc0c61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39550/0 (socket says 172.21.15.60:39550) 2024-04-25T10:16:38.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d1efe640 1 -- 172.21.15.60:0/2879996804 learned_addr learned my addr 172.21.15.60:0/2879996804 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d26ff640 1 --2- 172.21.15.60:0/2879996804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6cc15dd30 0x7fe6cc0afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d1efe640 1 -- 172.21.15.60:0/2879996804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6cc15dd30 msgr2=0x7fe6cc0afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d1efe640 1 --2- 172.21.15.60:0/2879996804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6cc15dd30 0x7fe6cc0afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d1efe640 1 -- 172.21.15.60:0/2879996804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6cc0cf8a0 msgr2=0x7fe6cc155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d1efe640 1 --2- 172.21.15.60:0/2879996804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6cc0cf8a0 0x7fe6cc155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d1efe640 1 -- 172.21.15.60:0/2879996804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc0c5120 con 0x7fe6cc0cd3d0 2024-04-25T10:16:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.262+0000 7fe6d1efe640 1 --2- 172.21.15.60:0/2879996804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6cc0cd3d0 0x7fe6cc0c61f0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80ad040 tx=0x7fe6c801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.266+0000 7fe6c27fc640 1 -- 172.21.15.60:0/2879996804 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8004220 con 0x7fe6cc0cd3d0 2024-04-25T10:16:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.266+0000 7fe6c27fc640 1 -- 172.21.15.60:0/2879996804 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c80043c0 con 0x7fe6cc0cd3d0 2024-04-25T10:16:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.266+0000 7fe6d4162640 1 -- 172.21.15.60:0/2879996804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6cc0c5710 con 0x7fe6cc0cd3d0 2024-04-25T10:16:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.266+0000 7fe6c27fc640 1 -- 172.21.15.60:0/2879996804 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8003820 con 0x7fe6cc0cd3d0 2024-04-25T10:16:38.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.266+0000 7fe6d4162640 1 -- 172.21.15.60:0/2879996804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6cc0c5cd0 con 0x7fe6cc0cd3d0 2024-04-25T10:16:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.266+0000 7fe6d4162640 1 -- 172.21.15.60:0/2879996804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6cc074770 con 0x7fe6cc0cd3d0 2024-04-25T10:16:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.266+0000 7fe6c27fc640 1 -- 172.21.15.60:0/2879996804 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6c80aa070 con 0x7fe6cc0cd3d0 2024-04-25T10:16:38.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.266+0000 7fe6c27fc640 1 --2- 172.21.15.60:0/2879996804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe6b0066f60 0x7fe6b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.266+0000 7fe6c27fc640 1 -- 172.21.15.60:0/2879996804 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe6c811fa10 con 0x7fe6cc0cd3d0 2024-04-25T10:16:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.266+0000 7fe6d16fd640 1 --2- 172.21.15.60:0/2879996804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe6b0066f60 0x7fe6b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.270+0000 7fe6d16fd640 1 --2- 172.21.15.60:0/2879996804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe6b0066f60 0x7fe6b0069420 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc000c10 tx=0x7fe6bc007910 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.270+0000 7fe6c27fc640 1 -- 172.21.15.60:0/2879996804 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6c8016090 con 0x7fe6cc0cd3d0 2024-04-25T10:16:38.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.430+0000 7fe6d4162640 1 -- 172.21.15.60:0/2879996804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe6cc0ce9d0 con 0x7fe6cc0cd3d0 2024-04-25T10:16:38.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.430+0000 7fe6c27fc640 1 -- 172.21.15.60:0/2879996804 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe6c80f09a0 con 0x7fe6cc0cd3d0 2024-04-25T10:16:38.437 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:16:38.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.434+0000 7fe6d4162640 1 -- 172.21.15.60:0/2879996804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe6b0066f60 msgr2=0x7fe6b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.434+0000 7fe6d4162640 1 --2- 172.21.15.60:0/2879996804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe6b0066f60 0x7fe6b0069420 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc000c10 tx=0x7fe6bc007910 comp rx=0 tx=0).stop 2024-04-25T10:16:38.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.434+0000 7fe6d4162640 1 -- 172.21.15.60:0/2879996804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6cc0cd3d0 msgr2=0x7fe6cc0c61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.434+0000 7fe6d4162640 1 --2- 172.21.15.60:0/2879996804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6cc0cd3d0 0x7fe6cc0c61f0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80ad040 tx=0x7fe6c801ac20 comp rx=0 tx=0).stop 2024-04-25T10:16:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.434+0000 7fe6d4162640 1 -- 172.21.15.60:0/2879996804 shutdown_connections 2024-04-25T10:16:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.434+0000 7fe6d4162640 1 --2- 172.21.15.60:0/2879996804 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe6b0066f60 0x7fe6b0069420 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.434+0000 7fe6d4162640 1 --2- 172.21.15.60:0/2879996804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6cc15dd30 0x7fe6cc0afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.434+0000 7fe6d4162640 1 --2- 172.21.15.60:0/2879996804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6cc0cf8a0 0x7fe6cc155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.434+0000 7fe6d4162640 1 --2- 172.21.15.60:0/2879996804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6cc0cd3d0 0x7fe6cc0c61f0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.434+0000 7fe6d4162640 1 -- 172.21.15.60:0/2879996804 >> 172.21.15.60:0/2879996804 conn(0x7fe6cc0670b0 msgr2=0x7fe6cc155bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.434+0000 7fe6d4162640 1 -- 172.21.15.60:0/2879996804 shutdown_connections 2024-04-25T10:16:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.434+0000 7fe6d4162640 1 -- 172.21.15.60:0/2879996804 wait complete. 2024-04-25T10:16:38.458 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:16:38.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.570+0000 7f5de48fe640 1 Processor -- start 2024-04-25T10:16:38.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5de48fe640 1 -- start start 2024-04-25T10:16:38.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5de48fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5de0160080 0x7f5de015f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5de48fe640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de0144a40 con 0x7f5de015fa20 2024-04-25T10:16:38.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5de48fe640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de00cc9f0 con 0x7f5de00cbcf0 2024-04-25T10:16:38.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5de48fe640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de015fe90 con 0x7f5de0160080 2024-04-25T10:16:38.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5dded9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5de015fa20 0x7f5de00ceec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56118/0 (socket says 172.21.15.60:56118) 2024-04-25T10:16:38.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5dded9d640 1 -- 172.21.15.60:0/942830155 learned_addr learned my addr 172.21.15.60:0/942830155 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:38.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5dddd9b640 1 --2- 172.21.15.60:0/942830155 >> v2:172.21.15.100:3300/0 conn(0x7f5de0160080 0x7f5de015f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5ddd59a640 1 -- 172.21.15.60:0/942830155 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969403903 0 0) 0x7f5de0144a40 con 0x7f5de015fa20 2024-04-25T10:16:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5ddd59a640 1 -- 172.21.15.60:0/942830155 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dc8003530 con 0x7f5de015fa20 2024-04-25T10:16:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5ddd59a640 1 -- 172.21.15.60:0/942830155 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1604250398 0 0) 0x7f5dc8003530 con 0x7f5de015fa20 2024-04-25T10:16:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5ddd59a640 1 -- 172.21.15.60:0/942830155 >> v2:172.21.15.100:3300/0 conn(0x7f5de0160080 msgr2=0x7f5de015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5ddd59a640 1 --2- 172.21.15.60:0/942830155 >> v2:172.21.15.100:3300/0 conn(0x7f5de0160080 0x7f5de015f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5ddd59a640 1 -- 172.21.15.60:0/942830155 >> v1:172.21.15.100:6789/0 conn(0x7f5de00cbcf0 legacy=0x7f5de0071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5ddd59a640 1 -- 172.21.15.60:0/942830155 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de00cccd0 con 0x7f5de015fa20 2024-04-25T10:16:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5ddd59a640 1 -- 172.21.15.60:0/942830155 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f5dd0001680 con 0x7f5de015fa20 2024-04-25T10:16:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5ddd59a640 1 -- 172.21.15.60:0/942830155 >> v1:172.21.15.60:6789/0 conn(0x7f5de015fa20 legacy=0x7f5de00ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5ddd59a640 1 --2- 172.21.15.60:0/942830155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dc80039c0 0x7f5dc8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5ddd59a640 1 --2- 172.21.15.60:0/942830155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dc8004810 0x7f5dc8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5ddd59a640 1 --2- 172.21.15.60:0/942830155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dc80051d0 0x7f5dc800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5ddd59a640 1 -- 172.21.15.60:0/942830155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de015fe90 con 0x7f5dc80039c0 2024-04-25T10:16:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5ddd59a640 1 -- 172.21.15.60:0/942830155 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc800c070 con 0x7f5dc80051d0 2024-04-25T10:16:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5ddd59a640 1 -- 172.21.15.60:0/942830155 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc800c270 con 0x7f5dc8004810 2024-04-25T10:16:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5dde59c640 1 --2- 172.21.15.60:0/942830155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dc80039c0 0x7f5dc8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5dded9d640 1 --2- 172.21.15.60:0/942830155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dc80051d0 0x7f5dc800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5dddd9b640 1 --2- 172.21.15.60:0/942830155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dc8004810 0x7f5dc8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5dde59c640 1 -- 172.21.15.60:0/942830155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dc8004810 msgr2=0x7f5dc8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5dde59c640 1 --2- 172.21.15.60:0/942830155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dc8004810 0x7f5dc8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5dde59c640 1 -- 172.21.15.60:0/942830155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dc80051d0 msgr2=0x7f5dc800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5dde59c640 1 --2- 172.21.15.60:0/942830155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dc80051d0 0x7f5dc800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5dde59c640 1 -- 172.21.15.60:0/942830155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc8003530 con 0x7f5dc80039c0 2024-04-25T10:16:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5dddd9b640 1 --2- 172.21.15.60:0/942830155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dc8004810 0x7f5dc8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5dded9d640 1 --2- 172.21.15.60:0/942830155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dc80051d0 0x7f5dc800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:38.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.574+0000 7f5dded9d640 1 -- 172.21.15.60:0/942830155 reap_dead start 2024-04-25T10:16:38.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5dde59c640 1 --2- 172.21.15.60:0/942830155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dc80039c0 0x7f5dc8003de0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8001e70 tx=0x7f5dd801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817b94762ab39c78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:38.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5ddd59a640 1 -- 172.21.15.60:0/942830155 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80af070 con 0x7f5dc80039c0 2024-04-25T10:16:38.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5ddd59a640 1 -- 172.21.15.60:0/942830155 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd8003b80 con 0x7f5dc80039c0 2024-04-25T10:16:38.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5ddd59a640 1 -- 172.21.15.60:0/942830155 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8001a90 con 0x7f5dc80039c0 2024-04-25T10:16:38.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5de48fe640 1 -- 172.21.15.60:0/942830155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dc80039c0 msgr2=0x7f5dc8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5de48fe640 1 --2- 172.21.15.60:0/942830155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dc80039c0 0x7f5dc8003de0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8001e70 tx=0x7f5dd801c760 comp rx=0 tx=0).stop 2024-04-25T10:16:38.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5de48fe640 1 -- 172.21.15.60:0/942830155 shutdown_connections 2024-04-25T10:16:38.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5de48fe640 1 --2- 172.21.15.60:0/942830155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dc80039c0 0x7f5dc8003de0 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5de48fe640 1 -- 172.21.15.60:0/942830155 >> 172.21.15.60:0/942830155 conn(0x7f5de00670b0 msgr2=0x7f5de006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:38.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5de48fe640 1 -- 172.21.15.60:0/942830155 shutdown_connections 2024-04-25T10:16:38.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5de48fe640 1 -- 172.21.15.60:0/942830155 wait complete. 2024-04-25T10:16:38.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5de48fe640 1 Processor -- start 2024-04-25T10:16:38.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5de48fe640 1 -- start start 2024-04-25T10:16:38.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5de48fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dc80039c0 0x7f5de014edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5de48fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5de0140660 0x7f5de0140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5de48fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5de013eb00 0x7f5de013ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5de48fe640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de006f260 con 0x7f5de013eb00 2024-04-25T10:16:38.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5de48fe640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de0072040 con 0x7f5dc80039c0 2024-04-25T10:16:38.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5de48fe640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de00721c0 con 0x7f5de0140660 2024-04-25T10:16:38.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5dded9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5de013eb00 0x7f5de013ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5dded9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5de013eb00 0x7f5de013ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39580/0 (socket says 172.21.15.60:39580) 2024-04-25T10:16:38.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.578+0000 7f5dded9d640 1 -- 172.21.15.60:0/571105042 learned_addr learned my addr 172.21.15.60:0/571105042 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:38.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5dddd9b640 1 --2- 172.21.15.60:0/571105042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5de0140660 0x7f5de0140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5dde59c640 1 --2- 172.21.15.60:0/571105042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dc80039c0 0x7f5de014edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5dded9d640 1 -- 172.21.15.60:0/571105042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5de0140660 msgr2=0x7f5de0140ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5dded9d640 1 --2- 172.21.15.60:0/571105042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5de0140660 0x7f5de0140ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5dded9d640 1 -- 172.21.15.60:0/571105042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dc80039c0 msgr2=0x7f5de014edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5dded9d640 1 --2- 172.21.15.60:0/571105042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dc80039c0 0x7f5de014edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5dded9d640 1 -- 172.21.15.60:0/571105042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de013f5c0 con 0x7f5de013eb00 2024-04-25T10:16:38.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5dded9d640 1 --2- 172.21.15.60:0/571105042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5de013eb00 0x7f5de013ef80 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f5de006f720 tx=0x7f5dd000d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5dceffd640 1 -- 172.21.15.60:0/571105042 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00074e0 con 0x7f5de013eb00 2024-04-25T10:16:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5dceffd640 1 -- 172.21.15.60:0/571105042 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd00047f0 con 0x7f5de013eb00 2024-04-25T10:16:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5de48fe640 1 -- 172.21.15.60:0/571105042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de013f8a0 con 0x7f5de013eb00 2024-04-25T10:16:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5dceffd640 1 -- 172.21.15.60:0/571105042 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0007680 con 0x7f5de013eb00 2024-04-25T10:16:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5de48fe640 1 -- 172.21.15.60:0/571105042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de013fe60 con 0x7f5de013eb00 2024-04-25T10:16:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5de48fe640 1 -- 172.21.15.60:0/571105042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de006bba0 con 0x7f5de013eb00 2024-04-25T10:16:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5dceffd640 1 -- 172.21.15.60:0/571105042 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5dd0004be0 con 0x7f5de013eb00 2024-04-25T10:16:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5dceffd640 1 --2- 172.21.15.60:0/571105042 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5dc0066f60 0x7f5dc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5dceffd640 1 -- 172.21.15.60:0/571105042 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5dd0084c70 con 0x7f5de013eb00 2024-04-25T10:16:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5dde59c640 1 --2- 172.21.15.60:0/571105042 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5dc0066f60 0x7f5dc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.582+0000 7f5dde59c640 1 --2- 172.21.15.60:0/571105042 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5dc0066f60 0x7f5dc0069420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80ac000 tx=0x7f5dd8001520 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:38.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.586+0000 7f5dceffd640 1 -- 172.21.15.60:0/571105042 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5dd001a060 con 0x7f5de013eb00 2024-04-25T10:16:38.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.746+0000 7f5de48fe640 1 -- 172.21.15.60:0/571105042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5de00b2970 con 0x7f5de013eb00 2024-04-25T10:16:38.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.746+0000 7f5dceffd640 1 -- 172.21.15.60:0/571105042 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5dd0055c20 con 0x7f5de013eb00 2024-04-25T10:16:38.753 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:16:38.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.750+0000 7f5de48fe640 1 -- 172.21.15.60:0/571105042 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5dc0066f60 msgr2=0x7f5dc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.750+0000 7f5de48fe640 1 --2- 172.21.15.60:0/571105042 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5dc0066f60 0x7f5dc0069420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80ac000 tx=0x7f5dd8001520 comp rx=0 tx=0).stop 2024-04-25T10:16:38.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.750+0000 7f5de48fe640 1 -- 172.21.15.60:0/571105042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5de013eb00 msgr2=0x7f5de013ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.750+0000 7f5de48fe640 1 --2- 172.21.15.60:0/571105042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5de013eb00 0x7f5de013ef80 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f5de006f720 tx=0x7f5dd000d960 comp rx=0 tx=0).stop 2024-04-25T10:16:38.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.750+0000 7f5de48fe640 1 -- 172.21.15.60:0/571105042 shutdown_connections 2024-04-25T10:16:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.750+0000 7f5de48fe640 1 --2- 172.21.15.60:0/571105042 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5dc0066f60 0x7f5dc0069420 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.750+0000 7f5de48fe640 1 --2- 172.21.15.60:0/571105042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5de013eb00 0x7f5de013ef80 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.750+0000 7f5de48fe640 1 --2- 172.21.15.60:0/571105042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5de0140660 0x7f5de0140ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.750+0000 7f5de48fe640 1 --2- 172.21.15.60:0/571105042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dc80039c0 0x7f5de014edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.750+0000 7f5de48fe640 1 -- 172.21.15.60:0/571105042 >> 172.21.15.60:0/571105042 conn(0x7f5de00670b0 msgr2=0x7f5de00b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.750+0000 7f5de48fe640 1 -- 172.21.15.60:0/571105042 shutdown_connections 2024-04-25T10:16:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.750+0000 7f5de48fe640 1 -- 172.21.15.60:0/571105042 wait complete. 2024-04-25T10:16:38.773 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:16:38.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.885+0000 7f2a9d121640 1 Processor -- start 2024-04-25T10:16:38.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.885+0000 7f2a9d121640 1 -- start start 2024-04-25T10:16:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.885+0000 7f2a9d121640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2a980ce7a0 0x7f2a980ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.885+0000 7f2a9d121640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a9815bc90 0x7f2a980b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.885+0000 7f2a9d121640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a980cf0c0 con 0x7f2a980b4660 2024-04-25T10:16:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.885+0000 7f2a9d121640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a980cc370 con 0x7f2a980ce7a0 2024-04-25T10:16:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.885+0000 7f2a9d121640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a98071ca0 con 0x7f2a9815bc90 2024-04-25T10:16:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.885+0000 7f2a9759e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2a980b4660 0x7f2a980b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56120/0 (socket says 172.21.15.60:56120) 2024-04-25T10:16:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a9759e640 1 -- 172.21.15.60:0/802032201 learned_addr learned my addr 172.21.15.60:0/802032201 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a96d9d640 1 --2- 172.21.15.60:0/802032201 >> v2:172.21.15.100:3300/0 conn(0x7f2a980ce7a0 0x7f2a980ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a9659c640 1 --2- 172.21.15.60:0/802032201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a9815bc90 0x7f2a980b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 -- 172.21.15.60:0/802032201 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413132980 0 0) 0x7f2a980cf0c0 con 0x7f2a980b4660 2024-04-25T10:16:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 -- 172.21.15.60:0/802032201 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a80003530 con 0x7f2a980b4660 2024-04-25T10:16:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 -- 172.21.15.60:0/802032201 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 630225074 0 0) 0x7f2a80003530 con 0x7f2a980b4660 2024-04-25T10:16:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 -- 172.21.15.60:0/802032201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a9815bc90 msgr2=0x7f2a980b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 --2- 172.21.15.60:0/802032201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a9815bc90 0x7f2a980b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 -- 172.21.15.60:0/802032201 >> v2:172.21.15.100:3300/0 conn(0x7f2a980ce7a0 msgr2=0x7f2a980ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 --2- 172.21.15.60:0/802032201 >> v2:172.21.15.100:3300/0 conn(0x7f2a980ce7a0 0x7f2a980ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 -- 172.21.15.60:0/802032201 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a980cca40 con 0x7f2a980b4660 2024-04-25T10:16:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 -- 172.21.15.60:0/802032201 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f2a90001a00 con 0x7f2a980b4660 2024-04-25T10:16:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 -- 172.21.15.60:0/802032201 >> v1:172.21.15.60:6789/0 conn(0x7f2a980b4660 legacy=0x7f2a980b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 --2- 172.21.15.60:0/802032201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a800039c0 0x7f2a80003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 --2- 172.21.15.60:0/802032201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a80004810 0x7f2a80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 --2- 172.21.15.60:0/802032201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a9815bc90 0x7f2a800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 -- 172.21.15.60:0/802032201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a98071ca0 con 0x7f2a800039c0 2024-04-25T10:16:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 -- 172.21.15.60:0/802032201 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8000b9d0 con 0x7f2a80004810 2024-04-25T10:16:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 -- 172.21.15.60:0/802032201 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8000bb50 con 0x7f2a9815bc90 2024-04-25T10:16:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a96d9d640 1 --2- 172.21.15.60:0/802032201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a800039c0 0x7f2a80003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a9659c640 1 --2- 172.21.15.60:0/802032201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a80004810 0x7f2a80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a9759e640 1 --2- 172.21.15.60:0/802032201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a9815bc90 0x7f2a800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a96d9d640 1 -- 172.21.15.60:0/802032201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a9815bc90 msgr2=0x7f2a800051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a96d9d640 1 --2- 172.21.15.60:0/802032201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a9815bc90 0x7f2a800051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a96d9d640 1 -- 172.21.15.60:0/802032201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a80004810 msgr2=0x7f2a80004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a96d9d640 1 --2- 172.21.15.60:0/802032201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a80004810 0x7f2a80004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a96d9d640 1 -- 172.21.15.60:0/802032201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a80003530 con 0x7f2a800039c0 2024-04-25T10:16:38.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a96d9d640 1 --2- 172.21.15.60:0/802032201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a800039c0 0x7f2a80003de0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009ae0 tx=0x7f2a8801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35aac99c688a1716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:38.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 -- 172.21.15.60:0/802032201 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a880096b0 con 0x7f2a800039c0 2024-04-25T10:16:38.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 -- 172.21.15.60:0/802032201 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a880a8410 con 0x7f2a800039c0 2024-04-25T10:16:38.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.889+0000 7f2a95d9b640 1 -- 172.21.15.60:0/802032201 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a880a8800 con 0x7f2a800039c0 2024-04-25T10:16:38.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9d121640 1 -- 172.21.15.60:0/802032201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a800039c0 msgr2=0x7f2a80003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9d121640 1 --2- 172.21.15.60:0/802032201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a800039c0 0x7f2a80003de0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009ae0 tx=0x7f2a8801cab0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9759e640 1 -- 172.21.15.60:0/802032201 reap_dead start 2024-04-25T10:16:38.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9d121640 1 -- 172.21.15.60:0/802032201 shutdown_connections 2024-04-25T10:16:38.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9d121640 1 -- 172.21.15.60:0/802032201 >> 172.21.15.60:0/802032201 conn(0x7f2a980b12d0 msgr2=0x7f2a980b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:38.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9d121640 1 -- 172.21.15.60:0/802032201 shutdown_connections 2024-04-25T10:16:38.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9d121640 1 -- 172.21.15.60:0/802032201 wait complete. 2024-04-25T10:16:38.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9d121640 1 Processor -- start 2024-04-25T10:16:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9d121640 1 -- start start 2024-04-25T10:16:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9d121640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a800039c0 0x7f2a9814cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9d121640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a9806dd30 0x7f2a9806e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9d121640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a9815a1f0 0x7f2a9815a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9d121640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a980b4280 con 0x7f2a800039c0 2024-04-25T10:16:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9d121640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a980b3db0 con 0x7f2a9815a1f0 2024-04-25T10:16:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9d121640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a980b4400 con 0x7f2a9806dd30 2024-04-25T10:16:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a96d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a800039c0 0x7f2a9814cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a96d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a800039c0 0x7f2a9814cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54388/0 (socket says 172.21.15.60:54388) 2024-04-25T10:16:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a96d9d640 1 -- 172.21.15.60:0/3400820184 learned_addr learned my addr 172.21.15.60:0/3400820184 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9759e640 1 --2- 172.21.15.60:0/3400820184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a9815a1f0 0x7f2a9815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a96d9d640 1 -- 172.21.15.60:0/3400820184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a9806dd30 msgr2=0x7f2a9806e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a96d9d640 1 --2- 172.21.15.60:0/3400820184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a9806dd30 0x7f2a9806e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a96d9d640 1 -- 172.21.15.60:0/3400820184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a9815a1f0 msgr2=0x7f2a9815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a96d9d640 1 --2- 172.21.15.60:0/3400820184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a9815a1f0 0x7f2a9815a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a96d9d640 1 -- 172.21.15.60:0/3400820184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a98160060 con 0x7f2a800039c0 2024-04-25T10:16:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9759e640 1 --2- 172.21.15.60:0/3400820184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a9815a1f0 0x7f2a9815a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a96d9d640 1 --2- 172.21.15.60:0/3400820184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a800039c0 0x7f2a9814cea0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009ae0 tx=0x7f2a88019850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a7f7fe640 1 -- 172.21.15.60:0/3400820184 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a880a8c40 con 0x7f2a800039c0 2024-04-25T10:16:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a7f7fe640 1 -- 172.21.15.60:0/3400820184 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a880a8de0 con 0x7f2a800039c0 2024-04-25T10:16:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9d121640 1 -- 172.21.15.60:0/3400820184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9806e8a0 con 0x7f2a800039c0 2024-04-25T10:16:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a7f7fe640 1 -- 172.21.15.60:0/3400820184 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a880bc670 con 0x7f2a800039c0 2024-04-25T10:16:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.893+0000 7f2a9d121640 1 -- 172.21.15.60:0/3400820184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9806ee60 con 0x7f2a800039c0 2024-04-25T10:16:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.897+0000 7f2a9d121640 1 -- 172.21.15.60:0/3400820184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a98074770 con 0x7f2a800039c0 2024-04-25T10:16:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.897+0000 7f2a7f7fe640 1 -- 172.21.15.60:0/3400820184 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a880b0400 con 0x7f2a800039c0 2024-04-25T10:16:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.897+0000 7f2a7f7fe640 1 --2- 172.21.15.60:0/3400820184 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2a84066f90 0x7f2a84069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.897+0000 7f2a7f7fe640 1 -- 172.21.15.60:0/3400820184 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2a8811f460 con 0x7f2a800039c0 2024-04-25T10:16:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.897+0000 7f2a9659c640 1 --2- 172.21.15.60:0/3400820184 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2a84066f90 0x7f2a84069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.897+0000 7f2a9659c640 1 --2- 172.21.15.60:0/3400820184 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2a84066f90 0x7f2a84069450 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2a980cffe0 tx=0x7f2a8c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:38.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:38.901+0000 7f2a7f7fe640 1 -- 172.21.15.60:0/3400820184 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a880ac030 con 0x7f2a800039c0 2024-04-25T10:16:39.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.073+0000 7f2a9d121640 1 -- 172.21.15.60:0/3400820184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2a981577f0 con 0x7f2a800039c0 2024-04-25T10:16:39.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.073+0000 7f2a7f7fe640 1 -- 172.21.15.60:0/3400820184 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v111) v1 ==== 80+0+120 (secure 0 0 0) 0x7f2a880167c0 con 0x7f2a800039c0 2024-04-25T10:16:39.080 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6835/722171246 2024-04-26T10:16:28.355776+0000 2024-04-25T10:16:39.080 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6834/722171246 2024-04-26T10:16:28.355776+0000 2024-04-25T10:16:39.080 INFO:teuthology.orchestra.run.smithi060.stderr:listed 2 entries 2024-04-25T10:16:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.077+0000 7f2a9d121640 1 -- 172.21.15.60:0/3400820184 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2a84066f90 msgr2=0x7f2a84069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.077+0000 7f2a9d121640 1 --2- 172.21.15.60:0/3400820184 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2a84066f90 0x7f2a84069450 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2a980cffe0 tx=0x7f2a8c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.077+0000 7f2a9d121640 1 -- 172.21.15.60:0/3400820184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a800039c0 msgr2=0x7f2a9814cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.077+0000 7f2a9d121640 1 --2- 172.21.15.60:0/3400820184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a800039c0 0x7f2a9814cea0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009ae0 tx=0x7f2a88019850 comp rx=0 tx=0).stop 2024-04-25T10:16:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.077+0000 7f2a9d121640 1 -- 172.21.15.60:0/3400820184 shutdown_connections 2024-04-25T10:16:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.077+0000 7f2a9d121640 1 --2- 172.21.15.60:0/3400820184 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2a84066f90 0x7f2a84069450 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.077+0000 7f2a9d121640 1 --2- 172.21.15.60:0/3400820184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a9815a1f0 0x7f2a9815a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.077+0000 7f2a9d121640 1 --2- 172.21.15.60:0/3400820184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a9806dd30 0x7f2a9806e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.077+0000 7f2a9d121640 1 --2- 172.21.15.60:0/3400820184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a800039c0 0x7f2a9814cea0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.077+0000 7f2a9d121640 1 -- 172.21.15.60:0/3400820184 >> 172.21.15.60:0/3400820184 conn(0x7f2a980b12d0 msgr2=0x7f2a98071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:39.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.077+0000 7f2a9d121640 1 -- 172.21.15.60:0/3400820184 shutdown_connections 2024-04-25T10:16:39.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.077+0000 7f2a9d121640 1 -- 172.21.15.60:0/3400820184 wait complete. 2024-04-25T10:16:39.099 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:16:39.099 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:39.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:39.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.257+0000 7f06becdf640 1 Processor -- start 2024-04-25T10:16:39.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.257+0000 7f06becdf640 1 -- start start 2024-04-25T10:16:39.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06becdf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06b806ec30 0x7f06b80cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06becdf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06b80b37e0 0x7f06b80b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06becdf640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b806f130 con 0x7f06b815a7a0 2024-04-25T10:16:39.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06becdf640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b80cdaa0 con 0x7f06b80b37e0 2024-04-25T10:16:39.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06becdf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b806d2f0 con 0x7f06b806ec30 2024-04-25T10:16:39.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06bd27c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06b80b37e0 0x7f06b80b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06bd27c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06b80b37e0 0x7f06b80b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54396/0 (socket says 172.21.15.60:54396) 2024-04-25T10:16:39.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06bd27c640 1 -- 172.21.15.60:0/2410457837 learned_addr learned my addr 172.21.15.60:0/2410457837 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:39.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06b7fff640 1 --2- 172.21.15.60:0/2410457837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06b806ec30 0x7f06b80cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06bd27c640 1 -- 172.21.15.60:0/2410457837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06b806ec30 msgr2=0x7f06b80cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:39.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06bd27c640 1 --2- 172.21.15.60:0/2410457837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06b806ec30 0x7f06b80cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06bd27c640 1 -- 172.21.15.60:0/2410457837 >> v1:172.21.15.100:6789/0 conn(0x7f06b815a7a0 legacy=0x7f06b8071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06bd27c640 1 -- 172.21.15.60:0/2410457837 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b806f410 con 0x7f06b80b37e0 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06bd27c640 1 --2- 172.21.15.60:0/2410457837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b80b37e0 0x7f06b80b3c90 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f06b000d150 tx=0x7f06b001ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f183ae4c35caf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06b77fe640 1 -- 172.21.15.60:0/2410457837 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b000b2d0 con 0x7f06b80b37e0 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06b77fe640 1 -- 172.21.15.60:0/2410457837 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b000b470 con 0x7f06b80b37e0 2024-04-25T10:16:39.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.261+0000 7f06b77fe640 1 -- 172.21.15.60:0/2410457837 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b00a7c30 con 0x7f06b80b37e0 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06becdf640 1 -- 172.21.15.60:0/2410457837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b80b37e0 msgr2=0x7f06b80b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06becdf640 1 --2- 172.21.15.60:0/2410457837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b80b37e0 0x7f06b80b3c90 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f06b000d150 tx=0x7f06b001ec30 comp rx=0 tx=0).stop 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06becdf640 1 -- 172.21.15.60:0/2410457837 shutdown_connections 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06becdf640 1 --2- 172.21.15.60:0/2410457837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b80b37e0 0x7f06b80b3c90 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06becdf640 1 --2- 172.21.15.60:0/2410457837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06b806ec30 0x7f06b80cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06becdf640 1 -- 172.21.15.60:0/2410457837 >> 172.21.15.60:0/2410457837 conn(0x7f06b80b12d0 msgr2=0x7f06b80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06becdf640 1 -- 172.21.15.60:0/2410457837 shutdown_connections 2024-04-25T10:16:39.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06becdf640 1 -- 172.21.15.60:0/2410457837 wait complete. 2024-04-25T10:16:39.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06becdf640 1 Processor -- start 2024-04-25T10:16:39.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06becdf640 1 -- start start 2024-04-25T10:16:39.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06becdf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06b806ec30 0x7f06b8144f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06becdf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06b80b37e0 0x7f06b8157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06becdf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b815a7a0 0x7f06b80cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06becdf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b806e8c0 con 0x7f06b815a7a0 2024-04-25T10:16:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06becdf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b806e740 con 0x7f06b806ec30 2024-04-25T10:16:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06becdf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b806d8b0 con 0x7f06b80b37e0 2024-04-25T10:16:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06bd27c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b815a7a0 0x7f06b80cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06bd27c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b815a7a0 0x7f06b80cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54406/0 (socket says 172.21.15.60:54406) 2024-04-25T10:16:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06bd27c640 1 -- 172.21.15.60:0/638042996 learned_addr learned my addr 172.21.15.60:0/638042996 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06b7fff640 1 --2- 172.21.15.60:0/638042996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06b80b37e0 0x7f06b8157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06bca7b640 1 --2- 172.21.15.60:0/638042996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06b806ec30 0x7f06b8144f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.265+0000 7f06bd27c640 1 -- 172.21.15.60:0/638042996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06b80b37e0 msgr2=0x7f06b8157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.269+0000 7f06bd27c640 1 --2- 172.21.15.60:0/638042996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06b80b37e0 0x7f06b8157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.269+0000 7f06bd27c640 1 -- 172.21.15.60:0/638042996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06b806ec30 msgr2=0x7f06b8144f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.269+0000 7f06bd27c640 1 --2- 172.21.15.60:0/638042996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06b806ec30 0x7f06b8144f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.269+0000 7f06bd27c640 1 -- 172.21.15.60:0/638042996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b81454b0 con 0x7f06b815a7a0 2024-04-25T10:16:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.269+0000 7f06bd27c640 1 --2- 172.21.15.60:0/638042996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b815a7a0 0x7f06b80cded0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f06b00b1040 tx=0x7f06b001b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.269+0000 7f06b57fa640 1 -- 172.21.15.60:0/638042996 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0009800 con 0x7f06b815a7a0 2024-04-25T10:16:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.269+0000 7f06b57fa640 1 -- 172.21.15.60:0/638042996 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b00be070 con 0x7f06b815a7a0 2024-04-25T10:16:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.269+0000 7f06becdf640 1 -- 172.21.15.60:0/638042996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b8157a90 con 0x7f06b815a7a0 2024-04-25T10:16:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.269+0000 7f06b57fa640 1 -- 172.21.15.60:0/638042996 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0009c00 con 0x7f06b815a7a0 2024-04-25T10:16:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.269+0000 7f06becdf640 1 -- 172.21.15.60:0/638042996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b8156a70 con 0x7f06b815a7a0 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.269+0000 7f06becdf640 1 -- 172.21.15.60:0/638042996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b8074770 con 0x7f06b815a7a0 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.269+0000 7f06b57fa640 1 -- 172.21.15.60:0/638042996 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06b00b6070 con 0x7f06b815a7a0 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.269+0000 7f06b57fa640 1 --2- 172.21.15.60:0/638042996 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f06a4066f60 0x7f06a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.269+0000 7f06b57fa640 1 -- 172.21.15.60:0/638042996 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f06b0120110 con 0x7f06b815a7a0 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.269+0000 7f06bca7b640 1 --2- 172.21.15.60:0/638042996 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f06a4066f60 0x7f06a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.269+0000 7f06bca7b640 1 --2- 172.21.15.60:0/638042996 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f06a4066f60 0x7f06a4069420 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0015b0 tx=0x7f06ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.277+0000 7f06b57fa640 1 -- 172.21.15.60:0/638042996 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f06b0018820 con 0x7f06b815a7a0 2024-04-25T10:16:39.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.441+0000 7f06becdf640 1 -- 172.21.15.60:0/638042996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06b80b4730 con 0x7f06b815a7a0 2024-04-25T10:16:39.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.441+0000 7f06b57fa640 1 -- 172.21.15.60:0/638042996 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16766 (secure 0 0 0) 0x7f06b00f1170 con 0x7f06b815a7a0 2024-04-25T10:16:39.449 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:39.449 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":111,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:16:28.355822+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:06.033387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:06.753421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/722171246":"2024-04-26T10:16:28.355776+0000","172.21.15.60:6834/722171246":"2024-04-26T10:16:28.355776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:39.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.445+0000 7f06becdf640 1 -- 172.21.15.60:0/638042996 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f06a4066f60 msgr2=0x7f06a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.445+0000 7f06becdf640 1 --2- 172.21.15.60:0/638042996 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f06a4066f60 0x7f06a4069420 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0015b0 tx=0x7f06ac009040 comp rx=0 tx=0).stop 2024-04-25T10:16:39.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.445+0000 7f06becdf640 1 -- 172.21.15.60:0/638042996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b815a7a0 msgr2=0x7f06b80cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.445+0000 7f06becdf640 1 --2- 172.21.15.60:0/638042996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b815a7a0 0x7f06b80cded0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f06b00b1040 tx=0x7f06b001b9d0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.445+0000 7f06becdf640 1 -- 172.21.15.60:0/638042996 shutdown_connections 2024-04-25T10:16:39.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.445+0000 7f06becdf640 1 --2- 172.21.15.60:0/638042996 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f06a4066f60 0x7f06a4069420 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.445+0000 7f06becdf640 1 --2- 172.21.15.60:0/638042996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b815a7a0 0x7f06b80cded0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.445+0000 7f06becdf640 1 --2- 172.21.15.60:0/638042996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06b80b37e0 0x7f06b8157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.445+0000 7f06becdf640 1 --2- 172.21.15.60:0/638042996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06b806ec30 0x7f06b8144f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.445+0000 7f06becdf640 1 -- 172.21.15.60:0/638042996 >> 172.21.15.60:0/638042996 conn(0x7f06b80b12d0 msgr2=0x7f06b80b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:39.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.445+0000 7f06becdf640 1 -- 172.21.15.60:0/638042996 shutdown_connections 2024-04-25T10:16:39.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.445+0000 7f06becdf640 1 -- 172.21.15.60:0/638042996 wait complete. 2024-04-25T10:16:39.469 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:06.033387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:06.753421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:39.469 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:39.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.593+0000 7fa6529ef640 1 Processor -- start 2024-04-25T10:16:39.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.593+0000 7fa6529ef640 1 -- start start 2024-04-25T10:16:39.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.597+0000 7fa6529ef640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa64c1620d0 0x7fa64c0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.597+0000 7fa6529ef640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa64c0ced20 0x7fa64c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.597+0000 7fa6529ef640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64c15bad0 con 0x7fa64c15b6f0 2024-04-25T10:16:39.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.597+0000 7fa6529ef640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64c161c40 con 0x7fa64c0ced20 2024-04-25T10:16:39.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.597+0000 7fa6529ef640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c15ebe0 con 0x7fa64c1620d0 2024-04-25T10:16:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.597+0000 7fa650f8c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa64c0ced20 0x7fa64c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.597+0000 7fa64b7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa64c1620d0 0x7fa64c0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.597+0000 7fa650f8c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa64c0ced20 0x7fa64c0d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:45392/0 (socket says 172.21.15.60:45392) 2024-04-25T10:16:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.597+0000 7fa64bfff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa64c15b6f0 0x7fa64c15e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:60122/0 (socket says 172.21.15.60:60122) 2024-04-25T10:16:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.597+0000 7fa650f8c640 1 -- 172.21.15.60:0/927232847 learned_addr learned my addr 172.21.15.60:0/927232847 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.597+0000 7fa64b7fe640 1 -- 172.21.15.60:0/927232847 >> v2:172.21.15.100:3300/0 conn(0x7fa64c0ced20 msgr2=0x7fa64c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.597+0000 7fa64b7fe640 1 --2- 172.21.15.60:0/927232847 >> v2:172.21.15.100:3300/0 conn(0x7fa64c0ced20 0x7fa64c0d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.597+0000 7fa64b7fe640 1 -- 172.21.15.60:0/927232847 >> v1:172.21.15.100:6789/0 conn(0x7fa64c15b6f0 legacy=0x7fa64c15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.597+0000 7fa64b7fe640 1 -- 172.21.15.60:0/927232847 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c15eec0 con 0x7fa64c1620d0 2024-04-25T10:16:39.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.601+0000 7fa64b7fe640 1 --2- 172.21.15.60:0/927232847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa64c1620d0 0x7fa64c0ce7e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa6400097c0 tx=0x7fa64001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be8cffcd3baaea8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.601+0000 7fa64affd640 1 -- 172.21.15.60:0/927232847 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6400b0070 con 0x7fa64c1620d0 2024-04-25T10:16:39.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.601+0000 7fa64affd640 1 -- 172.21.15.60:0/927232847 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6400ab600 con 0x7fa64c1620d0 2024-04-25T10:16:39.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.601+0000 7fa64affd640 1 -- 172.21.15.60:0/927232847 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6400ab9d0 con 0x7fa64c1620d0 2024-04-25T10:16:39.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.601+0000 7fa6529ef640 1 -- 172.21.15.60:0/927232847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa64c1620d0 msgr2=0x7fa64c0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.601+0000 7fa6529ef640 1 --2- 172.21.15.60:0/927232847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa64c1620d0 0x7fa64c0ce7e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa6400097c0 tx=0x7fa64001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.601+0000 7fa6529ef640 1 -- 172.21.15.60:0/927232847 shutdown_connections 2024-04-25T10:16:39.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.601+0000 7fa6529ef640 1 --2- 172.21.15.60:0/927232847 >> v2:172.21.15.100:3300/0 conn(0x7fa64c0ced20 0x7fa64c0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.601+0000 7fa6529ef640 1 --2- 172.21.15.60:0/927232847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa64c1620d0 0x7fa64c0ce7e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.601+0000 7fa6529ef640 1 -- 172.21.15.60:0/927232847 >> 172.21.15.60:0/927232847 conn(0x7fa64c072340 msgr2=0x7fa64c072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:39.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.601+0000 7fa6529ef640 1 -- 172.21.15.60:0/927232847 shutdown_connections 2024-04-25T10:16:39.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.601+0000 7fa6529ef640 1 -- 172.21.15.60:0/927232847 wait complete. 2024-04-25T10:16:39.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.601+0000 7fa6529ef640 1 Processor -- start 2024-04-25T10:16:39.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.601+0000 7fa6529ef640 1 -- start start 2024-04-25T10:16:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa6529ef640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa64c0ced20 0x7fa64c0b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa6529ef640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa64c15b6f0 0x7fa64c0b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa6529ef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c1620d0 0x7fa64c1539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa6529ef640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c15e960 con 0x7fa64c1620d0 2024-04-25T10:16:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa6529ef640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c15e7e0 con 0x7fa64c0ced20 2024-04-25T10:16:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa6529ef640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c16aa60 con 0x7fa64c15b6f0 2024-04-25T10:16:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa650f8c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c1620d0 0x7fa64c1539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa650f8c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c1620d0 0x7fa64c1539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54410/0 (socket says 172.21.15.60:54410) 2024-04-25T10:16:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa650f8c640 1 -- 172.21.15.60:0/2724031701 learned_addr learned my addr 172.21.15.60:0/2724031701 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa64bfff640 1 --2- 172.21.15.60:0/2724031701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa64c0ced20 0x7fa64c0b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa64b7fe640 1 --2- 172.21.15.60:0/2724031701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa64c15b6f0 0x7fa64c0b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa650f8c640 1 -- 172.21.15.60:0/2724031701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa64c15b6f0 msgr2=0x7fa64c0b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa650f8c640 1 --2- 172.21.15.60:0/2724031701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa64c15b6f0 0x7fa64c0b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa650f8c640 1 -- 172.21.15.60:0/2724031701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa64c0ced20 msgr2=0x7fa64c0b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa650f8c640 1 --2- 172.21.15.60:0/2724031701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa64c0ced20 0x7fa64c0b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa650f8c640 1 -- 172.21.15.60:0/2724031701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c147960 con 0x7fa64c1620d0 2024-04-25T10:16:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa64b7fe640 1 --2- 172.21.15.60:0/2724031701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa64c15b6f0 0x7fa64c0b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa650f8c640 1 --2- 172.21.15.60:0/2724031701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c1620d0 0x7fa64c1539b0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fa64400efc0 tx=0x7fa64400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa648ff9640 1 -- 172.21.15.60:0/2724031701 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64400cd20 con 0x7fa64c1620d0 2024-04-25T10:16:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa648ff9640 1 -- 172.21.15.60:0/2724031701 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6440039a0 con 0x7fa64c1620d0 2024-04-25T10:16:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa648ff9640 1 -- 172.21.15.60:0/2724031701 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644003d90 con 0x7fa64c1620d0 2024-04-25T10:16:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa6529ef640 1 -- 172.21.15.60:0/2724031701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64c1531b0 con 0x7fa64c1620d0 2024-04-25T10:16:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa6529ef640 1 -- 172.21.15.60:0/2724031701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c0c7dc0 con 0x7fa64c1620d0 2024-04-25T10:16:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa6529ef640 1 -- 172.21.15.60:0/2724031701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64c074770 con 0x7fa64c1620d0 2024-04-25T10:16:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa648ff9640 1 -- 172.21.15.60:0/2724031701 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa644012050 con 0x7fa64c1620d0 2024-04-25T10:16:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa648ff9640 1 --2- 172.21.15.60:0/2724031701 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa638066f60 0x7fa638069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.605+0000 7fa648ff9640 1 -- 172.21.15.60:0/2724031701 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa644085460 con 0x7fa64c1620d0 2024-04-25T10:16:39.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.609+0000 7fa64bfff640 1 --2- 172.21.15.60:0/2724031701 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa638066f60 0x7fa638069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.609+0000 7fa64bfff640 1 --2- 172.21.15.60:0/2724031701 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa638066f60 0x7fa638069420 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa64c168a20 tx=0x7fa63c007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.613+0000 7fa648ff9640 1 -- 172.21.15.60:0/2724031701 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa644010040 con 0x7fa64c1620d0 2024-04-25T10:16:39.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.773+0000 7fa6529ef640 1 -- 172.21.15.60:0/2724031701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa64c0d16e0 con 0x7fa64c1620d0 2024-04-25T10:16:39.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.773+0000 7fa648ff9640 1 -- 172.21.15.60:0/2724031701 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6440564a0 con 0x7fa64c1620d0 2024-04-25T10:16:39.781 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:16:39.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.777+0000 7fa6529ef640 1 -- 172.21.15.60:0/2724031701 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa638066f60 msgr2=0x7fa638069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.777+0000 7fa6529ef640 1 --2- 172.21.15.60:0/2724031701 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa638066f60 0x7fa638069420 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa64c168a20 tx=0x7fa63c007e90 comp rx=0 tx=0).stop 2024-04-25T10:16:39.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.777+0000 7fa6529ef640 1 -- 172.21.15.60:0/2724031701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c1620d0 msgr2=0x7fa64c1539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.777+0000 7fa6529ef640 1 --2- 172.21.15.60:0/2724031701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c1620d0 0x7fa64c1539b0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fa64400efc0 tx=0x7fa64400c490 comp rx=0 tx=0).stop 2024-04-25T10:16:39.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.777+0000 7fa6529ef640 1 -- 172.21.15.60:0/2724031701 shutdown_connections 2024-04-25T10:16:39.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.777+0000 7fa6529ef640 1 --2- 172.21.15.60:0/2724031701 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa638066f60 0x7fa638069420 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.777+0000 7fa6529ef640 1 --2- 172.21.15.60:0/2724031701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa64c1620d0 0x7fa64c1539b0 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.777+0000 7fa6529ef640 1 --2- 172.21.15.60:0/2724031701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa64c15b6f0 0x7fa64c0b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.777+0000 7fa6529ef640 1 --2- 172.21.15.60:0/2724031701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa64c0ced20 0x7fa64c0b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.777+0000 7fa6529ef640 1 -- 172.21.15.60:0/2724031701 >> 172.21.15.60:0/2724031701 conn(0x7fa64c072340 msgr2=0x7fa64c15f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:39.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.777+0000 7fa6529ef640 1 -- 172.21.15.60:0/2724031701 shutdown_connections 2024-04-25T10:16:39.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.777+0000 7fa6529ef640 1 -- 172.21.15.60:0/2724031701 wait complete. 2024-04-25T10:16:39.801 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:16:39.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.921+0000 7f0404744640 1 Processor -- start 2024-04-25T10:16:39.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.921+0000 7f0404744640 1 -- start start 2024-04-25T10:16:39.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.921+0000 7f0404744640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f03fc15f0d0 0x7f03fc0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.921+0000 7f0404744640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f03fc0cdb00 0x7f03fc099b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.921+0000 7f0404744640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03fc0cca60 con 0x7f03fc15f850 2024-04-25T10:16:39.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.921+0000 7f0404744640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03fc0cc830 con 0x7f03fc15f0d0 2024-04-25T10:16:39.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.921+0000 7f0404744640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03fc0ce2b0 con 0x7f03fc0cdb00 2024-04-25T10:16:39.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.921+0000 7f04024e0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f03fc15f0d0 0x7f03fc0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.921+0000 7f04024e0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f03fc15f0d0 0x7f03fc0cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54416/0 (socket says 172.21.15.60:54416) 2024-04-25T10:16:39.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.921+0000 7f04024e0640 1 -- 172.21.15.60:0/3417433714 learned_addr learned my addr 172.21.15.60:0/3417433714 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:39.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.921+0000 7f0401cdf640 1 --2- 172.21.15.60:0/3417433714 >> v2:172.21.15.100:3300/0 conn(0x7f03fc0cdb00 0x7f03fc099b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.921+0000 7f04024e0640 1 -- 172.21.15.60:0/3417433714 >> v2:172.21.15.100:3300/0 conn(0x7f03fc0cdb00 msgr2=0x7f03fc099b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:39.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.921+0000 7f04024e0640 1 --2- 172.21.15.60:0/3417433714 >> v2:172.21.15.100:3300/0 conn(0x7f03fc0cdb00 0x7f03fc099b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.921+0000 7f04024e0640 1 -- 172.21.15.60:0/3417433714 >> v1:172.21.15.100:6789/0 conn(0x7f03fc15f850 legacy=0x7f03fc0b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.921+0000 7f04024e0640 1 -- 172.21.15.60:0/3417433714 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc0ccd40 con 0x7f03fc15f0d0 2024-04-25T10:16:39.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.925+0000 7f04024e0640 1 --2- 172.21.15.60:0/3417433714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03fc15f0d0 0x7f03fc0cc2f0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0d1090 tx=0x7f03f801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807b6a154af7b867 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.925+0000 7f04014de640 1 -- 172.21.15.60:0/3417433714 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f80b0070 con 0x7f03fc15f0d0 2024-04-25T10:16:39.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.925+0000 7f04014de640 1 -- 172.21.15.60:0/3417433714 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f80ab600 con 0x7f03fc15f0d0 2024-04-25T10:16:39.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.925+0000 7f04014de640 1 -- 172.21.15.60:0/3417433714 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f80ab9d0 con 0x7f03fc15f0d0 2024-04-25T10:16:39.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.925+0000 7f0404744640 1 -- 172.21.15.60:0/3417433714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03fc15f0d0 msgr2=0x7f03fc0cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.925+0000 7f0404744640 1 --2- 172.21.15.60:0/3417433714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03fc15f0d0 0x7f03fc0cc2f0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0d1090 tx=0x7f03f801c490 comp rx=0 tx=0).stop 2024-04-25T10:16:39.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.925+0000 7f0404744640 1 -- 172.21.15.60:0/3417433714 shutdown_connections 2024-04-25T10:16:39.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.925+0000 7f0404744640 1 --2- 172.21.15.60:0/3417433714 >> v2:172.21.15.100:3300/0 conn(0x7f03fc0cdb00 0x7f03fc099b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.925+0000 7f0404744640 1 --2- 172.21.15.60:0/3417433714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03fc15f0d0 0x7f03fc0cc2f0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.925+0000 7f0404744640 1 -- 172.21.15.60:0/3417433714 >> 172.21.15.60:0/3417433714 conn(0x7f03fc0b34d0 msgr2=0x7f03fc0b38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:39.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.925+0000 7f0404744640 1 -- 172.21.15.60:0/3417433714 shutdown_connections 2024-04-25T10:16:39.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.925+0000 7f0404744640 1 -- 172.21.15.60:0/3417433714 wait complete. 2024-04-25T10:16:39.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.925+0000 7f0404744640 1 Processor -- start 2024-04-25T10:16:39.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0404744640 1 -- start start 2024-04-25T10:16:39.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0404744640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03fc0cdb00 0x7f03fc06d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0404744640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03fc15f0d0 0x7f03fc1570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0404744640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03fc15f850 0x7f03fc06bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0404744640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc168d10 con 0x7f03fc15f850 2024-04-25T10:16:39.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0404744640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc168b90 con 0x7f03fc15f0d0 2024-04-25T10:16:39.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0404744640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc168e90 con 0x7f03fc0cdb00 2024-04-25T10:16:39.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0402ce1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03fc15f850 0x7f03fc06bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0402ce1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03fc15f850 0x7f03fc06bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54420/0 (socket says 172.21.15.60:54420) 2024-04-25T10:16:39.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0402ce1640 1 -- 172.21.15.60:0/2840505428 learned_addr learned my addr 172.21.15.60:0/2840505428 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:39.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f04024e0640 1 --2- 172.21.15.60:0/2840505428 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03fc0cdb00 0x7f03fc06d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0401cdf640 1 --2- 172.21.15.60:0/2840505428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03fc15f0d0 0x7f03fc1570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0402ce1640 1 -- 172.21.15.60:0/2840505428 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03fc0cdb00 msgr2=0x7f03fc06d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0402ce1640 1 --2- 172.21.15.60:0/2840505428 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03fc0cdb00 0x7f03fc06d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0402ce1640 1 -- 172.21.15.60:0/2840505428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03fc15f0d0 msgr2=0x7f03fc1570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0402ce1640 1 --2- 172.21.15.60:0/2840505428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03fc15f0d0 0x7f03fc1570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0402ce1640 1 -- 172.21.15.60:0/2840505428 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc06dc50 con 0x7f03fc15f850 2024-04-25T10:16:39.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0402ce1640 1 --2- 172.21.15.60:0/2840505428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03fc15f850 0x7f03fc06bd20 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f03f400c730 tx=0x7f03f400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f03f2ffd640 1 -- 172.21.15.60:0/2840505428 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4016020 con 0x7f03fc15f850 2024-04-25T10:16:39.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f03f2ffd640 1 -- 172.21.15.60:0/2840505428 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f4006d60 con 0x7f03fc15f850 2024-04-25T10:16:39.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0404744640 1 -- 172.21.15.60:0/2840505428 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03fc06c670 con 0x7f03fc15f850 2024-04-25T10:16:39.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f03f2ffd640 1 -- 172.21.15.60:0/2840505428 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4003bc0 con 0x7f03fc15f850 2024-04-25T10:16:39.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0404744640 1 -- 172.21.15.60:0/2840505428 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc06cf10 con 0x7f03fc15f850 2024-04-25T10:16:39.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f0404744640 1 -- 172.21.15.60:0/2840505428 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03fc074770 con 0x7f03fc15f850 2024-04-25T10:16:39.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f03f2ffd640 1 -- 172.21.15.60:0/2840505428 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03f40050d0 con 0x7f03fc15f850 2024-04-25T10:16:39.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f03f2ffd640 1 --2- 172.21.15.60:0/2840505428 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f03e0066f60 0x7f03e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.929+0000 7f03f2ffd640 1 -- 172.21.15.60:0/2840505428 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f03f4084c60 con 0x7f03fc15f850 2024-04-25T10:16:39.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.933+0000 7f04024e0640 1 --2- 172.21.15.60:0/2840505428 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f03e0066f60 0x7f03e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.933+0000 7f04024e0640 1 --2- 172.21.15.60:0/2840505428 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f03e0066f60 0x7f03e0069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f03f80ad000 tx=0x7f03f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:39.937+0000 7f03f2ffd640 1 -- 172.21.15.60:0/2840505428 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f03f4020020 con 0x7f03fc15f850 2024-04-25T10:16:40.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.101+0000 7f0404744640 1 -- 172.21.15.60:0/2840505428 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03fc0b7960 con 0x7f03fc15f850 2024-04-25T10:16:40.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.101+0000 7f03f2ffd640 1 -- 172.21.15.60:0/2840505428 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03f4055c00 con 0x7f03fc15f850 2024-04-25T10:16:40.108 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:16:40.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.105+0000 7f0404744640 1 -- 172.21.15.60:0/2840505428 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f03e0066f60 msgr2=0x7f03e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.105+0000 7f0404744640 1 --2- 172.21.15.60:0/2840505428 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f03e0066f60 0x7f03e0069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f03f80ad000 tx=0x7f03f8008040 comp rx=0 tx=0).stop 2024-04-25T10:16:40.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.105+0000 7f0404744640 1 -- 172.21.15.60:0/2840505428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03fc15f850 msgr2=0x7f03fc06bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.105+0000 7f0404744640 1 --2- 172.21.15.60:0/2840505428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03fc15f850 0x7f03fc06bd20 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f03f400c730 tx=0x7f03f400cb30 comp rx=0 tx=0).stop 2024-04-25T10:16:40.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.105+0000 7f0404744640 1 -- 172.21.15.60:0/2840505428 shutdown_connections 2024-04-25T10:16:40.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.105+0000 7f0404744640 1 --2- 172.21.15.60:0/2840505428 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f03e0066f60 0x7f03e0069420 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.105+0000 7f0404744640 1 --2- 172.21.15.60:0/2840505428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03fc15f850 0x7f03fc06bd20 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.105+0000 7f0404744640 1 --2- 172.21.15.60:0/2840505428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03fc15f0d0 0x7f03fc1570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.105+0000 7f0404744640 1 --2- 172.21.15.60:0/2840505428 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03fc0cdb00 0x7f03fc06d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.105+0000 7f0404744640 1 -- 172.21.15.60:0/2840505428 >> 172.21.15.60:0/2840505428 conn(0x7f03fc0b34d0 msgr2=0x7f03fc0ae7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:40.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.105+0000 7f0404744640 1 -- 172.21.15.60:0/2840505428 shutdown_connections 2024-04-25T10:16:40.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.105+0000 7f0404744640 1 -- 172.21.15.60:0/2840505428 wait complete. 2024-04-25T10:16:40.128 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:16:40.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd5834f640 1 Processor -- start 2024-04-25T10:16:40.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd5834f640 1 -- start start 2024-04-25T10:16:40.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd5834f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd500cbcf0 0x7fbd50160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd5834f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbd50071c30 0x7fbd5015f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd5834f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd5015fa20 0x7fbd5015bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd5834f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd500b2dc0 con 0x7fbd500cbcf0 2024-04-25T10:16:40.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd5834f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd50144a40 con 0x7fbd50071c30 2024-04-25T10:16:40.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd5834f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd500cc9f0 con 0x7fbd5015fa20 2024-04-25T10:16:40.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd560eb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd500cbcf0 0x7fbd50160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd560eb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd500cbcf0 0x7fbd50160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54436/0 (socket says 172.21.15.60:54436) 2024-04-25T10:16:40.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd560eb640 1 -- 172.21.15.60:0/1058947197 learned_addr learned my addr 172.21.15.60:0/1058947197 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:40.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd558ea640 1 --2- 172.21.15.60:0/1058947197 >> v2:172.21.15.100:3300/0 conn(0x7fbd50071c30 0x7fbd5015f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd568ec640 1 --2- 172.21.15.60:0/1058947197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd5015fa20 0x7fbd5015bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd560eb640 1 -- 172.21.15.60:0/1058947197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd5015fa20 msgr2=0x7fbd5015bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:40.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd560eb640 1 --2- 172.21.15.60:0/1058947197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd5015fa20 0x7fbd5015bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd560eb640 1 -- 172.21.15.60:0/1058947197 >> v2:172.21.15.100:3300/0 conn(0x7fbd50071c30 msgr2=0x7fbd5015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:40.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd560eb640 1 --2- 172.21.15.60:0/1058947197 >> v2:172.21.15.100:3300/0 conn(0x7fbd50071c30 0x7fbd5015f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.245+0000 7fbd560eb640 1 -- 172.21.15.60:0/1058947197 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd500cccd0 con 0x7fbd500cbcf0 2024-04-25T10:16:40.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd560eb640 1 --2- 172.21.15.60:0/1058947197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd500cbcf0 0x7fbd50160080 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fbd500d0570 tx=0x7fbd4c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e78fa9e175e552 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:40.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd550e9640 1 -- 172.21.15.60:0/1058947197 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c0b1070 con 0x7fbd500cbcf0 2024-04-25T10:16:40.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd550e9640 1 -- 172.21.15.60:0/1058947197 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c0ac630 con 0x7fbd500cbcf0 2024-04-25T10:16:40.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd550e9640 1 -- 172.21.15.60:0/1058947197 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c0a8050 con 0x7fbd500cbcf0 2024-04-25T10:16:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 -- 172.21.15.60:0/1058947197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd500cbcf0 msgr2=0x7fbd50160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 --2- 172.21.15.60:0/1058947197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd500cbcf0 0x7fbd50160080 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fbd500d0570 tx=0x7fbd4c01c520 comp rx=0 tx=0).stop 2024-04-25T10:16:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 -- 172.21.15.60:0/1058947197 shutdown_connections 2024-04-25T10:16:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 --2- 172.21.15.60:0/1058947197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd5015fa20 0x7fbd5015bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 --2- 172.21.15.60:0/1058947197 >> v2:172.21.15.100:3300/0 conn(0x7fbd50071c30 0x7fbd5015f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 --2- 172.21.15.60:0/1058947197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd500cbcf0 0x7fbd50160080 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 -- 172.21.15.60:0/1058947197 >> 172.21.15.60:0/1058947197 conn(0x7fbd500670b0 msgr2=0x7fbd5006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 -- 172.21.15.60:0/1058947197 shutdown_connections 2024-04-25T10:16:40.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 -- 172.21.15.60:0/1058947197 wait complete. 2024-04-25T10:16:40.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 Processor -- start 2024-04-25T10:16:40.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 -- start start 2024-04-25T10:16:40.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd50071c30 0x7fbd500af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd500cbcf0 0x7fbd5013ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd5015fa20 0x7fbd500ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd500ce740 con 0x7fbd500cbcf0 2024-04-25T10:16:40.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd500ce5c0 con 0x7fbd5015fa20 2024-04-25T10:16:40.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.249+0000 7fbd5834f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd50072010 con 0x7fbd50071c30 2024-04-25T10:16:40.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd558ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd500cbcf0 0x7fbd5013ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd558ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd500cbcf0 0x7fbd5013ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54446/0 (socket says 172.21.15.60:54446) 2024-04-25T10:16:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd558ea640 1 -- 172.21.15.60:0/378524968 learned_addr learned my addr 172.21.15.60:0/378524968 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd568ec640 1 --2- 172.21.15.60:0/378524968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd5015fa20 0x7fbd500ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd560eb640 1 --2- 172.21.15.60:0/378524968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd50071c30 0x7fbd500af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd558ea640 1 -- 172.21.15.60:0/378524968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd50071c30 msgr2=0x7fbd500af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd558ea640 1 --2- 172.21.15.60:0/378524968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd50071c30 0x7fbd500af5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd558ea640 1 -- 172.21.15.60:0/378524968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd5015fa20 msgr2=0x7fbd500ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd558ea640 1 --2- 172.21.15.60:0/378524968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd5015fa20 0x7fbd500ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd558ea640 1 -- 172.21.15.60:0/378524968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd500afef0 con 0x7fbd500cbcf0 2024-04-25T10:16:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd558ea640 1 --2- 172.21.15.60:0/378524968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd500cbcf0 0x7fbd5013ec90 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000d800 tx=0x7fbd4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd46ffd640 1 -- 172.21.15.60:0/378524968 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd40018030 con 0x7fbd500cbcf0 2024-04-25T10:16:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd46ffd640 1 -- 172.21.15.60:0/378524968 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd400096a0 con 0x7fbd500cbcf0 2024-04-25T10:16:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd46ffd640 1 -- 172.21.15.60:0/378524968 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd40009870 con 0x7fbd500cbcf0 2024-04-25T10:16:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd5834f640 1 -- 172.21.15.60:0/378524968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd500b01d0 con 0x7fbd500cbcf0 2024-04-25T10:16:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd5834f640 1 -- 172.21.15.60:0/378524968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd500b04b0 con 0x7fbd500cbcf0 2024-04-25T10:16:40.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd5834f640 1 -- 172.21.15.60:0/378524968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5006bba0 con 0x7fbd500cbcf0 2024-04-25T10:16:40.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd46ffd640 1 -- 172.21.15.60:0/378524968 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd4001b070 con 0x7fbd500cbcf0 2024-04-25T10:16:40.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd46ffd640 1 --2- 172.21.15.60:0/378524968 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd34066f60 0x7fbd34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd46ffd640 1 -- 172.21.15.60:0/378524968 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbd40085260 con 0x7fbd500cbcf0 2024-04-25T10:16:40.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.253+0000 7fbd560eb640 1 --2- 172.21.15.60:0/378524968 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd34066f60 0x7fbd34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.257+0000 7fbd560eb640 1 --2- 172.21.15.60:0/378524968 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd34066f60 0x7fbd34069420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0ae000 tx=0x7fbd4c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:40.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.257+0000 7fbd46ffd640 1 -- 172.21.15.60:0/378524968 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd40055b30 con 0x7fbd500cbcf0 2024-04-25T10:16:40.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.417+0000 7fbd5834f640 1 -- 172.21.15.60:0/378524968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbd50166650 con 0x7fbd500cbcf0 2024-04-25T10:16:40.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.417+0000 7fbd46ffd640 1 -- 172.21.15.60:0/378524968 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbd40059bd0 con 0x7fbd500cbcf0 2024-04-25T10:16:40.423 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:16:40.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.421+0000 7fbd5834f640 1 -- 172.21.15.60:0/378524968 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd34066f60 msgr2=0x7fbd34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.421+0000 7fbd5834f640 1 --2- 172.21.15.60:0/378524968 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd34066f60 0x7fbd34069420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0ae000 tx=0x7fbd4c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:40.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.421+0000 7fbd5834f640 1 -- 172.21.15.60:0/378524968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd500cbcf0 msgr2=0x7fbd5013ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.421+0000 7fbd5834f640 1 --2- 172.21.15.60:0/378524968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd500cbcf0 0x7fbd5013ec90 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000d800 tx=0x7fbd4000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.421+0000 7fbd5834f640 1 -- 172.21.15.60:0/378524968 shutdown_connections 2024-04-25T10:16:40.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.421+0000 7fbd5834f640 1 --2- 172.21.15.60:0/378524968 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd34066f60 0x7fbd34069420 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.421+0000 7fbd5834f640 1 --2- 172.21.15.60:0/378524968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd5015fa20 0x7fbd500ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.421+0000 7fbd5834f640 1 --2- 172.21.15.60:0/378524968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd500cbcf0 0x7fbd5013ec90 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.421+0000 7fbd5834f640 1 --2- 172.21.15.60:0/378524968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd50071c30 0x7fbd500af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.421+0000 7fbd5834f640 1 -- 172.21.15.60:0/378524968 >> 172.21.15.60:0/378524968 conn(0x7fbd500670b0 msgr2=0x7fbd500b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:40.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.421+0000 7fbd5834f640 1 -- 172.21.15.60:0/378524968 shutdown_connections 2024-04-25T10:16:40.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.421+0000 7fbd5834f640 1 -- 172.21.15.60:0/378524968 wait complete. 2024-04-25T10:16:40.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:16:40.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.565+0000 7fe6b3df7640 1 Processor -- start 2024-04-25T10:16:40.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.565+0000 7fe6b3df7640 1 -- start start 2024-04-25T10:16:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.565+0000 7fe6b3df7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe6ac0ce7a0 0x7fe6ac0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.565+0000 7fe6b3df7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe6ac15bc90 0x7fe6ac0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.565+0000 7fe6b3df7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6ac0cf0c0 con 0x7fe6ac0b4660 2024-04-25T10:16:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.565+0000 7fe6b3df7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6ac0cc370 con 0x7fe6ac15bc90 2024-04-25T10:16:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.565+0000 7fe6b3df7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6ac071ca0 con 0x7fe6ac0ce7a0 2024-04-25T10:16:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.565+0000 7fe6b1392640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe6ac15bc90 0x7fe6ac0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.565+0000 7fe6b1392640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe6ac15bc90 0x7fe6ac0b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54456/0 (socket says 172.21.15.60:54456) 2024-04-25T10:16:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.565+0000 7fe6b1392640 1 -- 172.21.15.60:0/660882678 learned_addr learned my addr 172.21.15.60:0/660882678 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.565+0000 7fe6b1392640 1 -- 172.21.15.60:0/660882678 >> v2:172.21.15.100:3300/0 conn(0x7fe6ac0ce7a0 msgr2=0x7fe6ac0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.565+0000 7fe6b1392640 1 --2- 172.21.15.60:0/660882678 >> v2:172.21.15.100:3300/0 conn(0x7fe6ac0ce7a0 0x7fe6ac0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.565+0000 7fe6b1392640 1 -- 172.21.15.60:0/660882678 >> v1:172.21.15.100:6789/0 conn(0x7fe6ac0b4660 legacy=0x7fe6ac0b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.565+0000 7fe6b1392640 1 -- 172.21.15.60:0/660882678 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac0cca40 con 0x7fe6ac15bc90 2024-04-25T10:16:40.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.569+0000 7fe6b1392640 1 --2- 172.21.15.60:0/660882678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ac15bc90 0x7fe6ac0b25d0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac04f5d0 tx=0x7fe69c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f8aa5d05123fcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:40.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.569+0000 7fe6b0b91640 1 -- 172.21.15.60:0/660882678 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0b0070 con 0x7fe6ac15bc90 2024-04-25T10:16:40.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.569+0000 7fe6b0b91640 1 -- 172.21.15.60:0/660882678 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c0ab600 con 0x7fe6ac15bc90 2024-04-25T10:16:40.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.569+0000 7fe6b0b91640 1 -- 172.21.15.60:0/660882678 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0ab9d0 con 0x7fe6ac15bc90 2024-04-25T10:16:40.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.569+0000 7fe6b3df7640 1 -- 172.21.15.60:0/660882678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ac15bc90 msgr2=0x7fe6ac0b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.569+0000 7fe6b3df7640 1 --2- 172.21.15.60:0/660882678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ac15bc90 0x7fe6ac0b25d0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac04f5d0 tx=0x7fe69c01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.569+0000 7fe6b3df7640 1 -- 172.21.15.60:0/660882678 shutdown_connections 2024-04-25T10:16:40.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.569+0000 7fe6b3df7640 1 --2- 172.21.15.60:0/660882678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ac15bc90 0x7fe6ac0b25d0 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.569+0000 7fe6b3df7640 1 --2- 172.21.15.60:0/660882678 >> v2:172.21.15.100:3300/0 conn(0x7fe6ac0ce7a0 0x7fe6ac0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.569+0000 7fe6b3df7640 1 -- 172.21.15.60:0/660882678 >> 172.21.15.60:0/660882678 conn(0x7fe6ac0b12d0 msgr2=0x7fe6ac0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:40.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.569+0000 7fe6b3df7640 1 -- 172.21.15.60:0/660882678 shutdown_connections 2024-04-25T10:16:40.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.569+0000 7fe6b3df7640 1 -- 172.21.15.60:0/660882678 wait complete. 2024-04-25T10:16:40.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b3df7640 1 Processor -- start 2024-04-25T10:16:40.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b3df7640 1 -- start start 2024-04-25T10:16:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b3df7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6ac0ce7a0 0x7fe6ac142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b3df7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6ac15bc90 0x7fe6ac06ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b3df7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ac150eb0 0x7fe6ac1521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b3df7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac0b4400 con 0x7fe6ac150eb0 2024-04-25T10:16:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b3df7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac0b4280 con 0x7fe6ac15bc90 2024-04-25T10:16:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b3df7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac0b3220 con 0x7fe6ac0ce7a0 2024-04-25T10:16:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b2394640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ac150eb0 0x7fe6ac1521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b2394640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ac150eb0 0x7fe6ac1521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54460/0 (socket says 172.21.15.60:54460) 2024-04-25T10:16:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b2394640 1 -- 172.21.15.60:0/2316810485 learned_addr learned my addr 172.21.15.60:0/2316810485 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b1392640 1 --2- 172.21.15.60:0/2316810485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6ac15bc90 0x7fe6ac06ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b1b93640 1 --2- 172.21.15.60:0/2316810485 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6ac0ce7a0 0x7fe6ac142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b2394640 1 -- 172.21.15.60:0/2316810485 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6ac0ce7a0 msgr2=0x7fe6ac142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b2394640 1 --2- 172.21.15.60:0/2316810485 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6ac0ce7a0 0x7fe6ac142060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b2394640 1 -- 172.21.15.60:0/2316810485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6ac15bc90 msgr2=0x7fe6ac06ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b2394640 1 --2- 172.21.15.60:0/2316810485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6ac15bc90 0x7fe6ac06ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b2394640 1 -- 172.21.15.60:0/2316810485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac067640 con 0x7fe6ac150eb0 2024-04-25T10:16:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b2394640 1 --2- 172.21.15.60:0/2316810485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ac150eb0 0x7fe6ac1521c0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400c780 tx=0x7fe6a400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6a27fc640 1 -- 172.21.15.60:0/2316810485 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4017070 con 0x7fe6ac150eb0 2024-04-25T10:16:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6a27fc640 1 -- 172.21.15.60:0/2316810485 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a4003db0 con 0x7fe6ac150eb0 2024-04-25T10:16:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6a27fc640 1 -- 172.21.15.60:0/2316810485 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40052b0 con 0x7fe6ac150eb0 2024-04-25T10:16:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b3df7640 1 -- 172.21.15.60:0/2316810485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ac151400 con 0x7fe6ac150eb0 2024-04-25T10:16:40.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b3df7640 1 -- 172.21.15.60:0/2316810485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ac1519c0 con 0x7fe6ac150eb0 2024-04-25T10:16:40.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b3df7640 1 -- 172.21.15.60:0/2316810485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ac074770 con 0x7fe6ac150eb0 2024-04-25T10:16:40.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6a27fc640 1 -- 172.21.15.60:0/2316810485 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6a401a070 con 0x7fe6ac150eb0 2024-04-25T10:16:40.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6a27fc640 1 --2- 172.21.15.60:0/2316810485 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe690066f60 0x7fe690069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6a27fc640 1 -- 172.21.15.60:0/2316810485 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe6a4084700 con 0x7fe6ac150eb0 2024-04-25T10:16:40.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.573+0000 7fe6b1b93640 1 --2- 172.21.15.60:0/2316810485 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe690066f60 0x7fe690069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.577+0000 7fe6b1b93640 1 --2- 172.21.15.60:0/2316810485 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe690066f60 0x7fe690069420 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac158840 tx=0x7fe6a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:40.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.581+0000 7fe6a27fc640 1 -- 172.21.15.60:0/2316810485 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6a40556b0 con 0x7fe6ac150eb0 2024-04-25T10:16:40.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.745+0000 7fe6b3df7640 1 -- 172.21.15.60:0/2316810485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe6ac158400 con 0x7fe6ac150eb0 2024-04-25T10:16:40.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.745+0000 7fe6a27fc640 1 -- 172.21.15.60:0/2316810485 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v111) v1 ==== 80+0+120 (secure 0 0 0) 0x7fe6a4054fb0 con 0x7fe6ac150eb0 2024-04-25T10:16:40.752 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6835/722171246 2024-04-26T10:16:28.355776+0000 2024-04-25T10:16:40.753 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6834/722171246 2024-04-26T10:16:28.355776+0000 2024-04-25T10:16:40.753 INFO:teuthology.orchestra.run.smithi060.stderr:listed 2 entries 2024-04-25T10:16:40.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.749+0000 7fe6b3df7640 1 -- 172.21.15.60:0/2316810485 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe690066f60 msgr2=0x7fe690069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.749+0000 7fe6b3df7640 1 --2- 172.21.15.60:0/2316810485 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe690066f60 0x7fe690069420 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac158840 tx=0x7fe6a8008040 comp rx=0 tx=0).stop 2024-04-25T10:16:40.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.749+0000 7fe6b3df7640 1 -- 172.21.15.60:0/2316810485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ac150eb0 msgr2=0x7fe6ac1521c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.749+0000 7fe6b3df7640 1 --2- 172.21.15.60:0/2316810485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ac150eb0 0x7fe6ac1521c0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400c780 tx=0x7fe6a400cc40 comp rx=0 tx=0).stop 2024-04-25T10:16:40.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.749+0000 7fe6b3df7640 1 -- 172.21.15.60:0/2316810485 shutdown_connections 2024-04-25T10:16:40.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.749+0000 7fe6b3df7640 1 --2- 172.21.15.60:0/2316810485 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe690066f60 0x7fe690069420 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.749+0000 7fe6b3df7640 1 --2- 172.21.15.60:0/2316810485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6ac150eb0 0x7fe6ac1521c0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.749+0000 7fe6b3df7640 1 --2- 172.21.15.60:0/2316810485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6ac15bc90 0x7fe6ac06ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.749+0000 7fe6b3df7640 1 --2- 172.21.15.60:0/2316810485 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6ac0ce7a0 0x7fe6ac142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.749+0000 7fe6b3df7640 1 -- 172.21.15.60:0/2316810485 >> 172.21.15.60:0/2316810485 conn(0x7fe6ac0b12d0 msgr2=0x7fe6ac0b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:40.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.749+0000 7fe6b3df7640 1 -- 172.21.15.60:0/2316810485 shutdown_connections 2024-04-25T10:16:40.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:40.749+0000 7fe6b3df7640 1 -- 172.21.15.60:0/2316810485 wait complete. 2024-04-25T10:16:40.770 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:16:40.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:40.771 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:40.779 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[52745]: fuse finished with error 0 and tester_r 0 2024-04-25T10:16:40.780 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:46.882 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:16:46.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:46.883 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:16:46.890 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:47.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.065+0000 7f2dae0f4640 1 Processor -- start 2024-04-25T10:16:47.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.065+0000 7f2dae0f4640 1 -- start start 2024-04-25T10:16:47.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.065+0000 7f2dae0f4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2da80ce7a0 0x7f2da80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.065+0000 7f2dae0f4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2da80b4660 0x7f2da80b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.065+0000 7f2dae0f4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da80cf0c0 con 0x7f2da815bc90 2024-04-25T10:16:47.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.065+0000 7f2dae0f4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da80cc370 con 0x7f2da80ce7a0 2024-04-25T10:16:47.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.065+0000 7f2dae0f4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da8071ca0 con 0x7f2da80b4660 2024-04-25T10:16:47.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.065+0000 7f2da77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2da80ce7a0 0x7f2da80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.065+0000 7f2d9ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2da815bc90 0x7f2da80b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51158/0 (socket says 172.21.15.60:51158) 2024-04-25T10:16:47.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.065+0000 7f2d9ffff640 1 -- 172.21.15.60:0/336445156 learned_addr learned my addr 172.21.15.60:0/336445156 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:47.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.065+0000 7f2da7fff640 1 --2- 172.21.15.60:0/336445156 >> v2:172.21.15.100:3300/0 conn(0x7f2da80b4660 0x7f2da80b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.065+0000 7f2da77fe640 1 -- 172.21.15.60:0/336445156 >> v2:172.21.15.100:3300/0 conn(0x7f2da80b4660 msgr2=0x7f2da80b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.065+0000 7f2da77fe640 1 --2- 172.21.15.60:0/336445156 >> v2:172.21.15.100:3300/0 conn(0x7f2da80b4660 0x7f2da80b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.065+0000 7f2da77fe640 1 -- 172.21.15.60:0/336445156 >> v1:172.21.15.60:6789/0 conn(0x7f2da815bc90 legacy=0x7f2da80b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.065+0000 7f2da77fe640 1 -- 172.21.15.60:0/336445156 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da80cca40 con 0x7f2da80ce7a0 2024-04-25T10:16:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2da77fe640 1 --2- 172.21.15.60:0/336445156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da80ce7a0 0x7f2da80ceb80 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f2da806cf50 tx=0x7f2d9801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a08f84f5dd407a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2da6ffd640 1 -- 172.21.15.60:0/336445156 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d980b0070 con 0x7f2da80ce7a0 2024-04-25T10:16:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2da6ffd640 1 -- 172.21.15.60:0/336445156 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d980ab600 con 0x7f2da80ce7a0 2024-04-25T10:16:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2da6ffd640 1 -- 172.21.15.60:0/336445156 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d980ab9d0 con 0x7f2da80ce7a0 2024-04-25T10:16:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2dae0f4640 1 -- 172.21.15.60:0/336445156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da80ce7a0 msgr2=0x7f2da80ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2dae0f4640 1 --2- 172.21.15.60:0/336445156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da80ce7a0 0x7f2da80ceb80 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f2da806cf50 tx=0x7f2d9801c480 comp rx=0 tx=0).stop 2024-04-25T10:16:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2dae0f4640 1 -- 172.21.15.60:0/336445156 shutdown_connections 2024-04-25T10:16:47.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2dae0f4640 1 --2- 172.21.15.60:0/336445156 >> v2:172.21.15.100:3300/0 conn(0x7f2da80b4660 0x7f2da80b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2dae0f4640 1 --2- 172.21.15.60:0/336445156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da80ce7a0 0x7f2da80ceb80 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2dae0f4640 1 -- 172.21.15.60:0/336445156 >> 172.21.15.60:0/336445156 conn(0x7f2da80b12d0 msgr2=0x7f2da80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2dae0f4640 1 -- 172.21.15.60:0/336445156 shutdown_connections 2024-04-25T10:16:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2dae0f4640 1 -- 172.21.15.60:0/336445156 wait complete. 2024-04-25T10:16:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2dae0f4640 1 Processor -- start 2024-04-25T10:16:47.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2dae0f4640 1 -- start start 2024-04-25T10:16:47.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2dae0f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da80ce7a0 0x7f2da8142000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2dae0f4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da815bc90 0x7f2da80aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2dae0f4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2da8154c20 0x7f2da80b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2dae0f4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da80b4400 con 0x7f2da815bc90 2024-04-25T10:16:47.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2dae0f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da80b4280 con 0x7f2da80ce7a0 2024-04-25T10:16:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2dae0f4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da80b3220 con 0x7f2da8154c20 2024-04-25T10:16:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2d9ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da815bc90 0x7f2da80aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2d9ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da815bc90 0x7f2da80aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54482/0 (socket says 172.21.15.60:54482) 2024-04-25T10:16:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2d9ffff640 1 -- 172.21.15.60:0/4175320238 learned_addr learned my addr 172.21.15.60:0/4175320238 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2da7fff640 1 --2- 172.21.15.60:0/4175320238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2da8154c20 0x7f2da80b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2da77fe640 1 --2- 172.21.15.60:0/4175320238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da80ce7a0 0x7f2da8142000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2d9ffff640 1 -- 172.21.15.60:0/4175320238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2da8154c20 msgr2=0x7f2da80b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2d9ffff640 1 --2- 172.21.15.60:0/4175320238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2da8154c20 0x7f2da80b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2d9ffff640 1 -- 172.21.15.60:0/4175320238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da80ce7a0 msgr2=0x7f2da8142000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2d9ffff640 1 --2- 172.21.15.60:0/4175320238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da80ce7a0 0x7f2da8142000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2d9ffff640 1 -- 172.21.15.60:0/4175320238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da815ea10 con 0x7f2da815bc90 2024-04-25T10:16:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2da77fe640 1 --2- 172.21.15.60:0/4175320238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da80ce7a0 0x7f2da8142000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.069+0000 7f2d9ffff640 1 --2- 172.21.15.60:0/4175320238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da815bc90 0x7f2da80aff00 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000c370 tx=0x7f2d9000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.073+0000 7f2da4ff9640 1 -- 172.21.15.60:0/4175320238 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90017070 con 0x7f2da815bc90 2024-04-25T10:16:47.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.073+0000 7f2da4ff9640 1 -- 172.21.15.60:0/4175320238 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d90005cf0 con 0x7f2da815bc90 2024-04-25T10:16:47.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.073+0000 7f2dae0f4640 1 -- 172.21.15.60:0/4175320238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da815ecf0 con 0x7f2da815bc90 2024-04-25T10:16:47.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.073+0000 7f2da4ff9640 1 -- 172.21.15.60:0/4175320238 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d900072f0 con 0x7f2da815bc90 2024-04-25T10:16:47.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.073+0000 7f2dae0f4640 1 -- 172.21.15.60:0/4175320238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da8142540 con 0x7f2da815bc90 2024-04-25T10:16:47.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.073+0000 7f2dae0f4640 1 -- 172.21.15.60:0/4175320238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da815eec0 con 0x7f2da815bc90 2024-04-25T10:16:47.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.073+0000 7f2da4ff9640 1 -- 172.21.15.60:0/4175320238 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d9001a070 con 0x7f2da815bc90 2024-04-25T10:16:47.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.073+0000 7f2da4ff9640 1 --2- 172.21.15.60:0/4175320238 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d88066f60 0x7f2d88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.073+0000 7f2da4ff9640 1 -- 172.21.15.60:0/4175320238 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2d900843a0 con 0x7f2da815bc90 2024-04-25T10:16:47.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.073+0000 7f2da77fe640 1 --2- 172.21.15.60:0/4175320238 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d88066f60 0x7f2d88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.073+0000 7f2da77fe640 1 --2- 172.21.15.60:0/4175320238 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d88066f60 0x7f2d88069420 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2d980ad000 tx=0x7f2d98008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.077+0000 7f2da4ff9640 1 -- 172.21.15.60:0/4175320238 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d900553e0 con 0x7f2da815bc90 2024-04-25T10:16:47.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.273+0000 7f2dae0f4640 1 -- 172.21.15.60:0/4175320238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2da8074770 con 0x7f2da815bc90 2024-04-25T10:16:47.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.273+0000 7f2da4ff9640 1 -- 172.21.15.60:0/4175320238 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f2d90054ce0 con 0x7f2da815bc90 2024-04-25T10:16:47.279 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:47.279 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2440975447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2440975447},{"type":"v1","addr":"172.21.15.100:6835","nonce":2440975447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2398080758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2398080758},{"type":"v1","addr":"172.21.15.60:6833","nonce":2398080758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2759861958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2759861958},{"type":"v1","addr":"172.21.15.100:6837","nonce":2759861958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:07.763322+0000","modified":"2024-04-25T10:16:31.387513+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9693},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9693":{"gid":9693,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/178428701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":178428701},{"type":"v1","addr":"172.21.15.60:6835","nonce":178428701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:16:47.279 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 99 2024-04-25T10:16:47.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.277+0000 7f2dae0f4640 1 -- 172.21.15.60:0/4175320238 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d88066f60 msgr2=0x7f2d88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.277+0000 7f2dae0f4640 1 --2- 172.21.15.60:0/4175320238 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d88066f60 0x7f2d88069420 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2d980ad000 tx=0x7f2d98008040 comp rx=0 tx=0).stop 2024-04-25T10:16:47.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.277+0000 7f2dae0f4640 1 -- 172.21.15.60:0/4175320238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da815bc90 msgr2=0x7f2da80aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.277+0000 7f2dae0f4640 1 --2- 172.21.15.60:0/4175320238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da815bc90 0x7f2da80aff00 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000c370 tx=0x7f2d9000c830 comp rx=0 tx=0).stop 2024-04-25T10:16:47.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.277+0000 7f2dae0f4640 1 -- 172.21.15.60:0/4175320238 shutdown_connections 2024-04-25T10:16:47.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.277+0000 7f2dae0f4640 1 --2- 172.21.15.60:0/4175320238 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2d88066f60 0x7f2d88069420 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.277+0000 7f2dae0f4640 1 --2- 172.21.15.60:0/4175320238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2da8154c20 0x7f2da80b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.277+0000 7f2dae0f4640 1 --2- 172.21.15.60:0/4175320238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2da815bc90 0x7f2da80aff00 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.277+0000 7f2dae0f4640 1 --2- 172.21.15.60:0/4175320238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2da80ce7a0 0x7f2da8142000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.277+0000 7f2dae0f4640 1 -- 172.21.15.60:0/4175320238 >> 172.21.15.60:0/4175320238 conn(0x7f2da80b12d0 msgr2=0x7f2da80b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.277+0000 7f2dae0f4640 1 -- 172.21.15.60:0/4175320238 shutdown_connections 2024-04-25T10:16:47.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.277+0000 7f2dae0f4640 1 -- 172.21.15.60:0/4175320238 wait complete. 2024-04-25T10:16:47.297 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parent mytag123 2024-04-25T10:16:47.430 INFO:teuthology.orchestra.run.smithi060.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-04-25T10:16:47.430 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-04-25T10:16:47.430 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-25T10:16:47.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f39010bf080 1 Processor -- start 2024-04-25T10:16:47.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f39010bf080 1 -- start start 2024-04-25T10:16:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f39010bf080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563d02ec340 0x5563d02eb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f39010bf080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5563d0391400 0x5563d038c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f39010bf080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5563d0391da0 0x5563d02e5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f39010bf080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d0304120 con 0x5563d0391400 2024-04-25T10:16:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f39010bf080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d02629f0 con 0x5563d02ec340 2024-04-25T10:16:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f39010bf080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563d03043c0 con 0x5563d0391da0 2024-04-25T10:16:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f38fe3dc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5563d0391400 0x5563d038c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f38fe3dc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5563d0391400 0x5563d038c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54498/0 (socket says 172.21.15.60:54498) 2024-04-25T10:16:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f38fe3dc640 1 -- 172.21.15.60:0/2229511402 learned_addr learned my addr 172.21.15.60:0/2229511402 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:47.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f38febdd640 1 --2- 172.21.15.60:0/2229511402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563d02ec340 0x5563d02eb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f38fe3dc640 1 -- 172.21.15.60:0/2229511402 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5563d0391da0 msgr2=0x5563d02e5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f38fe3dc640 1 --2- 172.21.15.60:0/2229511402 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5563d0391da0 0x5563d02e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f38fe3dc640 1 -- 172.21.15.60:0/2229511402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563d02ec340 msgr2=0x5563d02eb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f38fe3dc640 1 --2- 172.21.15.60:0/2229511402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563d02ec340 0x5563d02eb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f38fe3dc640 1 -- 172.21.15.60:0/2229511402 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563d02fe030 con 0x5563d0391400 2024-04-25T10:16:47.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f38febdd640 1 --2- 172.21.15.60:0/2229511402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563d02ec340 0x5563d02eb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f38fe3dc640 1 --2- 172.21.15.60:0/2229511402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5563d0391400 0x5563d038c140 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009640 tx=0x7f38f4009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8b376f339d793b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f38fd3da640 1 -- 172.21.15.60:0/2229511402 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4017070 con 0x5563d0391400 2024-04-25T10:16:47.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f38fd3da640 1 -- 172.21.15.60:0/2229511402 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f4006660 con 0x5563d0391400 2024-04-25T10:16:47.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.461+0000 7f38fd3da640 1 -- 172.21.15.60:0/2229511402 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4006a10 con 0x5563d0391400 2024-04-25T10:16:47.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 -- 172.21.15.60:0/2229511402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5563d0391400 msgr2=0x5563d038c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 --2- 172.21.15.60:0/2229511402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5563d0391400 0x5563d038c140 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009640 tx=0x7f38f4009b00 comp rx=0 tx=0).stop 2024-04-25T10:16:47.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 -- 172.21.15.60:0/2229511402 shutdown_connections 2024-04-25T10:16:47.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 --2- 172.21.15.60:0/2229511402 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5563d0391da0 0x5563d02e5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 --2- 172.21.15.60:0/2229511402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5563d0391400 0x5563d038c140 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 --2- 172.21.15.60:0/2229511402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563d02ec340 0x5563d02eb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 -- 172.21.15.60:0/2229511402 >> 172.21.15.60:0/2229511402 conn(0x5563d02e6e80 msgr2=0x5563d03045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 -- 172.21.15.60:0/2229511402 shutdown_connections 2024-04-25T10:16:47.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 -- 172.21.15.60:0/2229511402 wait complete. 2024-04-25T10:16:47.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 Processor -- start 2024-04-25T10:16:47.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 -- start start 2024-04-25T10:16:47.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5563d02ec340 0x5563d042c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563d0391400 0x5563d038bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5563d0391da0 0x5563d02e25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d038c680 con 0x5563d02ec340 2024-04-25T10:16:47.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d02eab40 con 0x5563d0391400 2024-04-25T10:16:47.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563d0303290 con 0x5563d0391da0 2024-04-25T10:16:47.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f38febdd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5563d02ec340 0x5563d042c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f38febdd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5563d02ec340 0x5563d042c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54512/0 (socket says 172.21.15.60:54512) 2024-04-25T10:16:47.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f38febdd640 1 -- 172.21.15.60:0/4249643111 learned_addr learned my addr 172.21.15.60:0/4249643111 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:47.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f38febdd640 1 -- 172.21.15.60:0/4249643111 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5563d0391da0 msgr2=0x5563d02e25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f38febdd640 1 --2- 172.21.15.60:0/4249643111 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5563d0391da0 0x5563d02e25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f38febdd640 1 -- 172.21.15.60:0/4249643111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563d0391400 msgr2=0x5563d038bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f38febdd640 1 --2- 172.21.15.60:0/4249643111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563d0391400 0x5563d038bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f38febdd640 1 -- 172.21.15.60:0/4249643111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563d02e51f0 con 0x5563d02ec340 2024-04-25T10:16:47.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f38febdd640 1 --2- 172.21.15.60:0/4249643111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5563d02ec340 0x5563d042c520 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f38f000cfa0 tx=0x7f38f000a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f38e7fff640 1 -- 172.21.15.60:0/4249643111 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0010680 con 0x5563d02ec340 2024-04-25T10:16:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f38e7fff640 1 -- 172.21.15.60:0/4249643111 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f0010820 con 0x5563d02ec340 2024-04-25T10:16:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 -- 172.21.15.60:0/4249643111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5563d039b640 con 0x5563d02ec340 2024-04-25T10:16:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f38e7fff640 1 -- 172.21.15.60:0/4249643111 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0010c10 con 0x5563d02ec340 2024-04-25T10:16:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f39010bf080 1 -- 172.21.15.60:0/4249643111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563d0304a60 con 0x5563d02ec340 2024-04-25T10:16:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.465+0000 7f38e7fff640 1 -- 172.21.15.60:0/4249643111 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38f001b470 con 0x5563d02ec340 2024-04-25T10:16:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f38e7fff640 1 --2- 172.21.15.60:0/4249643111 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f38ec065970 0x7f38ec067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f38e7fff640 1 -- 172.21.15.60:0/4249643111 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f38f0083e50 con 0x5563d02ec340 2024-04-25T10:16:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f39010bf080 1 --2- 172.21.15.60:0/4249643111 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5563d038c050 0x5563d038e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f39010bf080 1 -- 172.21.15.60:0/4249643111 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:1 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) v9 -- 0x5563d042ca60 con 0x5563d038c050 2024-04-25T10:16:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f38ff3de640 1 --2- 172.21.15.60:0/4249643111 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5563d038c050 0x5563d038e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f38fe3dc640 1 --2- 172.21.15.60:0/4249643111 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f38ec065970 0x7f38ec067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f38ff3de640 1 --2- 172.21.15.60:0/4249643111 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5563d038c050 0x5563d038e2a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f38ff3de640 1 -- 172.21.15.60:0/4249643111 <== osd.0 v2:172.21.15.60:6801/4056167511 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f38f80ac4c0 con 0x5563d038c050 2024-04-25T10:16:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-04-25T10:16:47.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f39010bf080 1 -- 172.21.15.60:0/4249643111 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5563d038c050 msgr2=0x5563d038e2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f39010bf080 1 --2- 172.21.15.60:0/4249643111 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5563d038c050 0x5563d038e2a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f39010bf080 1 -- 172.21.15.60:0/4249643111 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f38ec065970 msgr2=0x7f38ec067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f39010bf080 1 --2- 172.21.15.60:0/4249643111 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f38ec065970 0x7f38ec067e30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f38f4015040 tx=0x7f38f400e000 comp rx=0 tx=0).stop 2024-04-25T10:16:47.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f39010bf080 1 -- 172.21.15.60:0/4249643111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5563d02ec340 msgr2=0x5563d042c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f39010bf080 1 --2- 172.21.15.60:0/4249643111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5563d02ec340 0x5563d042c520 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f38f000cfa0 tx=0x7f38f000a5b0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f38ff3de640 1 -- 172.21.15.60:0/4249643111 reap_dead start 2024-04-25T10:16:47.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f39010bf080 1 -- 172.21.15.60:0/4249643111 shutdown_connections 2024-04-25T10:16:47.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f39010bf080 1 -- 172.21.15.60:0/4249643111 >> 172.21.15.60:0/4249643111 conn(0x5563d02e6e80 msgr2=0x5563d02e8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f39010bf080 1 -- 172.21.15.60:0/4249643111 shutdown_connections 2024-04-25T10:16:47.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.469+0000 7f39010bf080 1 -- 172.21.15.60:0/4249643111 wait complete. 2024-04-25T10:16:47.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:16:47.479 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-25T10:16:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.553+0000 7f4d5da7a080 1 Processor -- start 2024-04-25T10:16:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.553+0000 7f4d5da7a080 1 -- start start 2024-04-25T10:16:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.553+0000 7f4d5da7a080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d48003100 0x55c622e0a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.553+0000 7f4d5da7a080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c622d38920 0x55c622d38d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.553+0000 7f4d5da7a080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c622e9ab30 0x55c622e9af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.553+0000 7f4d5da7a080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c622cdd9f0 con 0x55c622e9ab30 2024-04-25T10:16:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.553+0000 7f4d5da7a080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c622ce1950 con 0x55c622d38920 2024-04-25T10:16:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.553+0000 7f4d5da7a080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c622e0ccf0 con 0x7f4d48003100 2024-04-25T10:16:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.553+0000 7f4d5bd99640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c622e9ab30 0x55c622e9af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.553+0000 7f4d5bd99640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c622e9ab30 0x55c622e9af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54520/0 (socket says 172.21.15.60:54520) 2024-04-25T10:16:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.553+0000 7f4d5bd99640 1 -- 172.21.15.60:0/3182539124 learned_addr learned my addr 172.21.15.60:0/3182539124 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5b598640 1 --2- 172.21.15.60:0/3182539124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d48003100 0x55c622e0a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5ad97640 1 --2- 172.21.15.60:0/3182539124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c622d38920 0x55c622d38d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5bd99640 1 -- 172.21.15.60:0/3182539124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d48003100 msgr2=0x55c622e0a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5bd99640 1 --2- 172.21.15.60:0/3182539124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d48003100 0x55c622e0a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5bd99640 1 -- 172.21.15.60:0/3182539124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c622d38920 msgr2=0x55c622d38d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5bd99640 1 --2- 172.21.15.60:0/3182539124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c622d38920 0x55c622d38d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5bd99640 1 -- 172.21.15.60:0/3182539124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c622d3a2a0 con 0x55c622e9ab30 2024-04-25T10:16:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5b598640 1 --2- 172.21.15.60:0/3182539124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d48003100 0x55c622e0a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5bd99640 1 --2- 172.21.15.60:0/3182539124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c622e9ab30 0x55c622e9af40 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x55c622d23920 tx=0x7f4d4c003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca08c2f0e285771 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d59d95640 1 -- 172.21.15.60:0/3182539124 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c003480 con 0x55c622e9ab30 2024-04-25T10:16:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d59d95640 1 -- 172.21.15.60:0/3182539124 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4c001620 con 0x55c622e9ab30 2024-04-25T10:16:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d59d95640 1 -- 172.21.15.60:0/3182539124 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c0019f0 con 0x55c622e9ab30 2024-04-25T10:16:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 -- 172.21.15.60:0/3182539124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c622e9ab30 msgr2=0x55c622e9af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 --2- 172.21.15.60:0/3182539124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c622e9ab30 0x55c622e9af40 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x55c622d23920 tx=0x7f4d4c003a10 comp rx=0 tx=0).stop 2024-04-25T10:16:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 -- 172.21.15.60:0/3182539124 shutdown_connections 2024-04-25T10:16:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 --2- 172.21.15.60:0/3182539124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c622e9ab30 0x55c622e9af40 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 --2- 172.21.15.60:0/3182539124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c622d38920 0x55c622d38d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 --2- 172.21.15.60:0/3182539124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d48003100 0x55c622e0a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 -- 172.21.15.60:0/3182539124 >> 172.21.15.60:0/3182539124 conn(0x55c622d61e80 msgr2=0x55c622e10f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 -- 172.21.15.60:0/3182539124 shutdown_connections 2024-04-25T10:16:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 -- 172.21.15.60:0/3182539124 wait complete. 2024-04-25T10:16:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 Processor -- start 2024-04-25T10:16:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 -- start start 2024-04-25T10:16:47.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c622d38920 0x55c622d7cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c622e9ab30 0x55c622e9b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d48003100 0x55c622e08a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c622d39620 con 0x55c622e9ab30 2024-04-25T10:16:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c622d39380 con 0x7f4d48003100 2024-04-25T10:16:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c622d398c0 con 0x55c622d38920 2024-04-25T10:16:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5ad97640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c622e9ab30 0x55c622e9b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5ad97640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c622e9ab30 0x55c622e9b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54532/0 (socket says 172.21.15.60:54532) 2024-04-25T10:16:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5ad97640 1 -- 172.21.15.60:0/3553413011 learned_addr learned my addr 172.21.15.60:0/3553413011 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5bd99640 1 --2- 172.21.15.60:0/3553413011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d48003100 0x55c622e08a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5b598640 1 --2- 172.21.15.60:0/3553413011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c622d38920 0x55c622d7cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5ad97640 1 -- 172.21.15.60:0/3553413011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c622d38920 msgr2=0x55c622d7cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5ad97640 1 --2- 172.21.15.60:0/3553413011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c622d38920 0x55c622d7cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5ad97640 1 -- 172.21.15.60:0/3553413011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d48003100 msgr2=0x55c622e08a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5ad97640 1 --2- 172.21.15.60:0/3553413011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d48003100 0x55c622e08a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5ad97640 1 -- 172.21.15.60:0/3553413011 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c622d7d000 con 0x55c622e9ab30 2024-04-25T10:16:47.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5bd99640 1 --2- 172.21.15.60:0/3553413011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d48003100 0x55c622e08a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5b598640 1 --2- 172.21.15.60:0/3553413011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c622d38920 0x55c622d7cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5ad97640 1 --2- 172.21.15.60:0/3553413011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c622e9ab30 0x55c622e9b120 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f4d4401b640 tx=0x7f4d48004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d58d93640 1 -- 172.21.15.60:0/3553413011 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d440b8070 con 0x55c622e9ab30 2024-04-25T10:16:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d58d93640 1 -- 172.21.15.60:0/3553413011 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d440a79f0 con 0x55c622e9ab30 2024-04-25T10:16:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 -- 172.21.15.60:0/3553413011 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c622d7bba0 con 0x55c622e9ab30 2024-04-25T10:16:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d58d93640 1 -- 172.21.15.60:0/3553413011 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d440a7de0 con 0x55c622e9ab30 2024-04-25T10:16:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.557+0000 7f4d5da7a080 1 -- 172.21.15.60:0/3553413011 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c622d7c0e0 con 0x55c622e9ab30 2024-04-25T10:16:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d58d93640 1 -- 172.21.15.60:0/3553413011 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d440b2430 con 0x55c622e9ab30 2024-04-25T10:16:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d58d93640 1 --2- 172.21.15.60:0/3553413011 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4d3c065970 0x7f4d3c067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d58d93640 1 -- 172.21.15.60:0/3553413011 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4d4411e950 con 0x55c622e9ab30 2024-04-25T10:16:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5da7a080 1 --2- 172.21.15.60:0/3553413011 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55c622e9b660 0x55c622ea5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5da7a080 1 -- 172.21.15.60:0/3553413011 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:1 17.12 17:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) v9 -- 0x55c622d7b6c0 con 0x55c622e9b660 2024-04-25T10:16:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5b598640 1 --2- 172.21.15.60:0/3553413011 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4d3c065970 0x7f4d3c067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5bd99640 1 --2- 172.21.15.60:0/3553413011 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55c622e9b660 0x55c622ea5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5bd99640 1 --2- 172.21.15.60:0/3553413011 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55c622e9b660 0x55c622ea5b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5bd99640 1 -- 172.21.15.60:0/3553413011 <== osd.3 v2:172.21.15.60:6814/1867957156 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f4d4c0017d0 con 0x55c622e9b660 2024-04-25T10:16:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5b598640 1 --2- 172.21.15.60:0/3553413011 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4d3c065970 0x7f4d3c067e30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400e710 tx=0x7f4d54001480 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5da7a080 1 -- 172.21.15.60:0/3553413011 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55c622e9b660 msgr2=0x55c622ea5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5da7a080 1 --2- 172.21.15.60:0/3553413011 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55c622e9b660 0x55c622ea5b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5da7a080 1 -- 172.21.15.60:0/3553413011 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4d3c065970 msgr2=0x7f4d3c067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5da7a080 1 --2- 172.21.15.60:0/3553413011 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4d3c065970 0x7f4d3c067e30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400e710 tx=0x7f4d54001480 comp rx=0 tx=0).stop 2024-04-25T10:16:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5da7a080 1 -- 172.21.15.60:0/3553413011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c622e9ab30 msgr2=0x55c622e9b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5da7a080 1 --2- 172.21.15.60:0/3553413011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c622e9ab30 0x55c622e9b120 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f4d4401b640 tx=0x7f4d48004130 comp rx=0 tx=0).stop 2024-04-25T10:16:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5bd99640 1 -- 172.21.15.60:0/3553413011 reap_dead start 2024-04-25T10:16:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5da7a080 1 -- 172.21.15.60:0/3553413011 shutdown_connections 2024-04-25T10:16:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5da7a080 1 -- 172.21.15.60:0/3553413011 >> 172.21.15.60:0/3553413011 conn(0x55c622d61e80 msgr2=0x55c622d77660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5da7a080 1 -- 172.21.15.60:0/3553413011 shutdown_connections 2024-04-25T10:16:47.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.561+0000 7f4d5da7a080 1 -- 172.21.15.60:0/3553413011 wait complete. 2024-04-25T10:16:47.572 INFO:teuthology.orchestra.run.smithi060.stdout:mytag123 2024-04-25T10:16:47.573 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-25T10:16:47.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f79874eb080 1 Processor -- start 2024-04-25T10:16:47.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f79874eb080 1 -- start start 2024-04-25T10:16:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f79874eb080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7970003100 0x5589c6ab4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f79874eb080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5589c6b6b350 0x5589c6af53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f79874eb080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5589c6b87ba0 0x5589c6b87fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f79874eb080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589c6a569f0 con 0x5589c6b87ba0 2024-04-25T10:16:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f79874eb080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589c6a5a950 con 0x7f7970003100 2024-04-25T10:16:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f79874eb080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589c6b87a20 con 0x5589c6b6b350 2024-04-25T10:16:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f798580a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5589c6b87ba0 0x5589c6b87fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f798580a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5589c6b87ba0 0x5589c6b87fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54544/0 (socket says 172.21.15.60:54544) 2024-04-25T10:16:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f798580a640 1 -- 172.21.15.60:0/3131462149 learned_addr learned my addr 172.21.15.60:0/3131462149 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f7984808640 1 --2- 172.21.15.60:0/3131462149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5589c6b6b350 0x5589c6af53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f7985009640 1 --2- 172.21.15.60:0/3131462149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7970003100 0x5589c6ab4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f798580a640 1 -- 172.21.15.60:0/3131462149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5589c6b6b350 msgr2=0x5589c6af53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f798580a640 1 --2- 172.21.15.60:0/3131462149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5589c6b6b350 0x5589c6af53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f798580a640 1 -- 172.21.15.60:0/3131462149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7970003100 msgr2=0x5589c6ab4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f798580a640 1 --2- 172.21.15.60:0/3131462149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7970003100 0x5589c6ab4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f798580a640 1 -- 172.21.15.60:0/3131462149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589c6b884f0 con 0x5589c6b87ba0 2024-04-25T10:16:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f7984808640 1 --2- 172.21.15.60:0/3131462149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5589c6b6b350 0x5589c6af53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f7985009640 1 --2- 172.21.15.60:0/3131462149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7970003100 0x5589c6ab4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f798580a640 1 --2- 172.21.15.60:0/3131462149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5589c6b87ba0 0x5589c6b87fb0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x5589c6ab5330 tx=0x7f797c00b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4b6faf91f6bf2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f79777fe640 1 -- 172.21.15.60:0/3131462149 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c00a040 con 0x5589c6b87ba0 2024-04-25T10:16:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f79777fe640 1 -- 172.21.15.60:0/3131462149 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797c014070 con 0x5589c6b87ba0 2024-04-25T10:16:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.641+0000 7f79777fe640 1 -- 172.21.15.60:0/3131462149 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c00f870 con 0x5589c6b87ba0 2024-04-25T10:16:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 -- 172.21.15.60:0/3131462149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5589c6b87ba0 msgr2=0x5589c6b87fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 --2- 172.21.15.60:0/3131462149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5589c6b87ba0 0x5589c6b87fb0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x5589c6ab5330 tx=0x7f797c00b5c0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 -- 172.21.15.60:0/3131462149 shutdown_connections 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 --2- 172.21.15.60:0/3131462149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5589c6b87ba0 0x5589c6b87fb0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 --2- 172.21.15.60:0/3131462149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5589c6b6b350 0x5589c6af53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 --2- 172.21.15.60:0/3131462149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7970003100 0x5589c6ab4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 -- 172.21.15.60:0/3131462149 >> 172.21.15.60:0/3131462149 conn(0x5589c6adc0e0 msgr2=0x5589c6ae1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 -- 172.21.15.60:0/3131462149 shutdown_connections 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 -- 172.21.15.60:0/3131462149 wait complete. 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 Processor -- start 2024-04-25T10:16:47.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 -- start start 2024-04-25T10:16:47.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5589c6b6b350 0x5589c6b6aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5589c6b87ba0 0x5589c6b6d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7970003100 0x5589c6ad4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589c6b89010 con 0x7f7970003100 2024-04-25T10:16:47.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589c6af5a90 con 0x5589c6b6b350 2024-04-25T10:16:47.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589c6b892b0 con 0x5589c6b87ba0 2024-04-25T10:16:47.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f798580a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7970003100 0x5589c6ad4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f798580a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7970003100 0x5589c6ad4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54558/0 (socket says 172.21.15.60:54558) 2024-04-25T10:16:47.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f798580a640 1 -- 172.21.15.60:0/3275955176 learned_addr learned my addr 172.21.15.60:0/3275955176 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:47.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f7985009640 1 --2- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5589c6b6b350 0x5589c6b6aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f7984808640 1 --2- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5589c6b87ba0 0x5589c6b6d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f798580a640 1 -- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5589c6b87ba0 msgr2=0x5589c6b6d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f798580a640 1 --2- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5589c6b87ba0 0x5589c6b6d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f798580a640 1 -- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5589c6b6b350 msgr2=0x5589c6b6aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f798580a640 1 --2- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5589c6b6b350 0x5589c6b6aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f798580a640 1 -- 172.21.15.60:0/3275955176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589c6b6e2e0 con 0x7f7970003100 2024-04-25T10:16:47.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f7984808640 1 --2- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5589c6b87ba0 0x5589c6b6d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f7985009640 1 --2- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5589c6b6b350 0x5589c6b6aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f798580a640 1 --2- 172.21.15.60:0/3275955176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7970003100 0x5589c6ad4c30 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f797c010000 tx=0x7f797c001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79767fc640 1 -- 172.21.15.60:0/3275955176 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c018830 con 0x7f7970003100 2024-04-25T10:16:47.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79767fc640 1 -- 172.21.15.60:0/3275955176 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797c0189d0 con 0x7f7970003100 2024-04-25T10:16:47.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 -- 172.21.15.60:0/3275955176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5589c6b6e580 con 0x7f7970003100 2024-04-25T10:16:47.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79767fc640 1 -- 172.21.15.60:0/3275955176 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c018dc0 con 0x7f7970003100 2024-04-25T10:16:47.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79874eb080 1 -- 172.21.15.60:0/3275955176 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589c6b6eac0 con 0x7f7970003100 2024-04-25T10:16:47.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.645+0000 7f79767fc640 1 -- 172.21.15.60:0/3275955176 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f797c00f410 con 0x7f7970003100 2024-04-25T10:16:47.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f79767fc640 1 --2- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7968065940 0x7f7968067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f79767fc640 1 -- 172.21.15.60:0/3275955176 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f797c0859f0 con 0x7f7970003100 2024-04-25T10:16:47.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f79874eb080 1 --2- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x5589c6b7b6f0 0x5589c6b7bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f79874eb080 1 -- 172.21.15.60:0/3275955176 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:1 17.19 17:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) v9 -- 0x5589c6b7c010 con 0x5589c6b7b6f0 2024-04-25T10:16:47.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f7985009640 1 --2- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7968065940 0x7f7968067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f7984808640 1 --2- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x5589c6b7b6f0 0x5589c6b7bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f7985009640 1 --2- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7968065940 0x7f7968067e00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f7980002260 tx=0x7f7980008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f7984808640 1 --2- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x5589c6b7b6f0 0x5589c6b7bad0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f7984808640 1 -- 172.21.15.60:0/3275955176 <== osd.5 v2:172.21.15.100:6800/3542120824 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f79780b4040 con 0x5589c6b7b6f0 2024-04-25T10:16:47.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f79874eb080 1 -- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x5589c6b7b6f0 msgr2=0x5589c6b7bad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f79874eb080 1 --2- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x5589c6b7b6f0 0x5589c6b7bad0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f79874eb080 1 -- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7968065940 msgr2=0x7f7968067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f79874eb080 1 --2- 172.21.15.60:0/3275955176 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7968065940 0x7f7968067e00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f7980002260 tx=0x7f7980008040 comp rx=0 tx=0).stop 2024-04-25T10:16:47.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f79874eb080 1 -- 172.21.15.60:0/3275955176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7970003100 msgr2=0x5589c6ad4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f79874eb080 1 --2- 172.21.15.60:0/3275955176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7970003100 0x5589c6ad4c30 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f797c010000 tx=0x7f797c001130 comp rx=0 tx=0).stop 2024-04-25T10:16:47.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f798580a640 1 -- 172.21.15.60:0/3275955176 reap_dead start 2024-04-25T10:16:47.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f79874eb080 1 -- 172.21.15.60:0/3275955176 shutdown_connections 2024-04-25T10:16:47.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f79874eb080 1 -- 172.21.15.60:0/3275955176 >> 172.21.15.60:0/3275955176 conn(0x5589c6adc0e0 msgr2=0x5589c6b89430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f79874eb080 1 -- 172.21.15.60:0/3275955176 shutdown_connections 2024-04-25T10:16:47.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.649+0000 7f79874eb080 1 -- 172.21.15.60:0/3275955176 wait complete. 2024-04-25T10:16:47.662 INFO:teuthology.orchestra.run.smithi060.stdout:mytag123 2024-04-25T10:16:47.662 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-04-25T10:16:47.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.729+0000 7f8010186080 1 Processor -- start 2024-04-25T10:16:47.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.729+0000 7f8010186080 1 -- start start 2024-04-25T10:16:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.729+0000 7f8010186080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5581df6a3d60 0x5581df77fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.729+0000 7f8010186080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581df6d5c90 0x5581df785180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.729+0000 7f8010186080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581df780970 0x5581df7856c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.729+0000 7f8010186080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581df6d4310 con 0x5581df6d5c90 2024-04-25T10:16:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.729+0000 7f8010186080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581df6d51a0 con 0x5581df780970 2024-04-25T10:16:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.729+0000 7f8010186080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581df6d45b0 con 0x5581df6a3d60 2024-04-25T10:16:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800d4a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581df6d5c90 0x5581df785180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800d4a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581df6d5c90 0x5581df785180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54580/0 (socket says 172.21.15.60:54580) 2024-04-25T10:16:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800d4a3640 1 -- 172.21.15.60:0/168412569 learned_addr learned my addr 172.21.15.60:0/168412569 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800e4a5640 1 --2- 172.21.15.60:0/168412569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581df780970 0x5581df7856c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800dca4640 1 --2- 172.21.15.60:0/168412569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5581df6a3d60 0x5581df77fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800d4a3640 1 -- 172.21.15.60:0/168412569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5581df6a3d60 msgr2=0x5581df77fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800d4a3640 1 --2- 172.21.15.60:0/168412569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5581df6a3d60 0x5581df77fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800d4a3640 1 -- 172.21.15.60:0/168412569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581df780970 msgr2=0x5581df7856c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800d4a3640 1 --2- 172.21.15.60:0/168412569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581df780970 0x5581df7856c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800d4a3640 1 -- 172.21.15.60:0/168412569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581df812270 con 0x5581df6d5c90 2024-04-25T10:16:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800e4a5640 1 --2- 172.21.15.60:0/168412569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581df780970 0x5581df7856c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800d4a3640 1 --2- 172.21.15.60:0/168412569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581df6d5c90 0x5581df785180 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f800400a590 tx=0x7f8004003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bc317f43baf096 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f7ffffff640 1 -- 172.21.15.60:0/168412569 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040b0040 con 0x5581df6d5c90 2024-04-25T10:16:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f7ffffff640 1 -- 172.21.15.60:0/168412569 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80040abb30 con 0x5581df6d5c90 2024-04-25T10:16:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f7ffffff640 1 -- 172.21.15.60:0/168412569 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040b7460 con 0x5581df6d5c90 2024-04-25T10:16:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 -- 172.21.15.60:0/168412569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581df6d5c90 msgr2=0x5581df785180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 --2- 172.21.15.60:0/168412569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581df6d5c90 0x5581df785180 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f800400a590 tx=0x7f8004003030 comp rx=0 tx=0).stop 2024-04-25T10:16:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 -- 172.21.15.60:0/168412569 shutdown_connections 2024-04-25T10:16:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 --2- 172.21.15.60:0/168412569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581df780970 0x5581df7856c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 --2- 172.21.15.60:0/168412569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581df6d5c90 0x5581df785180 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 --2- 172.21.15.60:0/168412569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5581df6a3d60 0x5581df77fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 -- 172.21.15.60:0/168412569 >> 172.21.15.60:0/168412569 conn(0x5581df6cfe80 msgr2=0x5581df78ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 -- 172.21.15.60:0/168412569 shutdown_connections 2024-04-25T10:16:47.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 -- 172.21.15.60:0/168412569 wait complete. 2024-04-25T10:16:47.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 Processor -- start 2024-04-25T10:16:47.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 -- start start 2024-04-25T10:16:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581df6a3d60 0x5581df78ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5581df6d5c90 0x5581df7693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581df780970 0x5581df769900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581df7866a0 con 0x5581df6a3d60 2024-04-25T10:16:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581df786400 con 0x5581df780970 2024-04-25T10:16:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581df788850 con 0x5581df6d5c90 2024-04-25T10:16:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800dca4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581df6a3d60 0x5581df78ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800dca4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581df6a3d60 0x5581df78ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54588/0 (socket says 172.21.15.60:54588) 2024-04-25T10:16:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800dca4640 1 -- 172.21.15.60:0/1991375190 learned_addr learned my addr 172.21.15.60:0/1991375190 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:47.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800d4a3640 1 --2- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5581df6d5c90 0x5581df7693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800e4a5640 1 --2- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581df780970 0x5581df769900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800dca4640 1 -- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5581df6d5c90 msgr2=0x5581df7693c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800dca4640 1 --2- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5581df6d5c90 0x5581df7693c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800dca4640 1 -- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581df780970 msgr2=0x5581df769900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800dca4640 1 --2- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581df780970 0x5581df769900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800dca4640 1 -- 172.21.15.60:0/1991375190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581df781060 con 0x5581df6a3d60 2024-04-25T10:16:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800e4a5640 1 --2- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581df780970 0x5581df769900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f800dca4640 1 --2- 172.21.15.60:0/1991375190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581df6a3d60 0x5581df78ab10 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f8008008200 tx=0x7f80080086c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f7ffeffd640 1 -- 172.21.15.60:0/1991375190 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008016030 con 0x5581df6a3d60 2024-04-25T10:16:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f7ffeffd640 1 -- 172.21.15.60:0/1991375190 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8008012410 con 0x5581df6a3d60 2024-04-25T10:16:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f7ffeffd640 1 -- 172.21.15.60:0/1991375190 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800800e040 con 0x5581df6a3d60 2024-04-25T10:16:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 -- 172.21.15.60:0/1991375190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5581df781300 con 0x5581df6a3d60 2024-04-25T10:16:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.733+0000 7f8010186080 1 -- 172.21.15.60:0/1991375190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581df781840 con 0x5581df6a3d60 2024-04-25T10:16:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f7ffeffd640 1 -- 172.21.15.60:0/1991375190 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8008019070 con 0x5581df6a3d60 2024-04-25T10:16:47.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f7ffeffd640 1 --2- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ff0065940 0x7f7ff0067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f7ffeffd640 1 -- 172.21.15.60:0/1991375190 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8008083930 con 0x5581df6a3d60 2024-04-25T10:16:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f8010186080 1 --2- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x5581df778c80 0x5581df77ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f800d4a3640 1 --2- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ff0065940 0x7f7ff0067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f8010186080 1 -- 172.21.15.60:0/1991375190 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:1 17.5 17:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) v9 -- 0x5581df78b050 con 0x5581df778c80 2024-04-25T10:16:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f800e4a5640 1 --2- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x5581df778c80 0x5581df77ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f800d4a3640 1 --2- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ff0065940 0x7f7ff0067e00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f80040bc000 tx=0x7f80040a8000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f800e4a5640 1 --2- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x5581df778c80 0x5581df77ba70 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f800e4a5640 1 -- 172.21.15.60:0/1991375190 <== osd.6 v2:172.21.15.100:6824/4086390550 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f8000009740 con 0x5581df778c80 2024-04-25T10:16:47.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f8010186080 1 -- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x5581df778c80 msgr2=0x5581df77ba70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f8010186080 1 --2- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x5581df778c80 0x5581df77ba70 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f8010186080 1 -- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ff0065940 msgr2=0x7f7ff0067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f8010186080 1 --2- 172.21.15.60:0/1991375190 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ff0065940 0x7f7ff0067e00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f80040bc000 tx=0x7f80040a8000 comp rx=0 tx=0).stop 2024-04-25T10:16:47.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f8010186080 1 -- 172.21.15.60:0/1991375190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581df6a3d60 msgr2=0x5581df78ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f8010186080 1 --2- 172.21.15.60:0/1991375190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5581df6a3d60 0x5581df78ab10 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f8008008200 tx=0x7f80080086c0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f800e4a5640 1 -- 172.21.15.60:0/1991375190 reap_dead start 2024-04-25T10:16:47.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f8010186080 1 -- 172.21.15.60:0/1991375190 shutdown_connections 2024-04-25T10:16:47.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f8010186080 1 -- 172.21.15.60:0/1991375190 >> 172.21.15.60:0/1991375190 conn(0x5581df6cfe80 msgr2=0x5581df77f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f8010186080 1 -- 172.21.15.60:0/1991375190 shutdown_connections 2024-04-25T10:16:47.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.737+0000 7f8010186080 1 -- 172.21.15.60:0/1991375190 wait complete. 2024-04-25T10:16:47.749 INFO:teuthology.orchestra.run.smithi060.stdout:mytag123 2024-04-25T10:16:47.749 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:16:47.749 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-25T10:16:47.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.913+0000 7fdbb2872640 1 Processor -- start 2024-04-25T10:16:47.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.913+0000 7fdbb2872640 1 -- start start 2024-04-25T10:16:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.913+0000 7fdbb2872640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbac0713e0 0x7fdbac0cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.913+0000 7fdbb2872640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdbac0cef70 0x7fdbac0b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.913+0000 7fdbb2872640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbac145700 con 0x7fdbac145320 2024-04-25T10:16:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.913+0000 7fdbb2872640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbac0cf3d0 con 0x7fdbac0cef70 2024-04-25T10:16:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.913+0000 7fdbb2872640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0b4800 con 0x7fdbac0713e0 2024-04-25T10:16:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.913+0000 7fdbb0e0f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdbac0cef70 0x7fdbac0b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.913+0000 7fdbabfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdbac145320 0x7fdbac070cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51194/0 (socket says 172.21.15.60:51194) 2024-04-25T10:16:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.913+0000 7fdbabfff640 1 -- 172.21.15.60:0/1956197228 learned_addr learned my addr 172.21.15.60:0/1956197228 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.913+0000 7fdbab7fe640 1 --2- 172.21.15.60:0/1956197228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbac0713e0 0x7fdbac0cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.913+0000 7fdbb0e0f640 1 -- 172.21.15.60:0/1956197228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbac0713e0 msgr2=0x7fdbac0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.913+0000 7fdbb0e0f640 1 --2- 172.21.15.60:0/1956197228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbac0713e0 0x7fdbac0cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.913+0000 7fdbb0e0f640 1 -- 172.21.15.60:0/1956197228 >> v1:172.21.15.60:6789/0 conn(0x7fdbac145320 legacy=0x7fdbac070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.913+0000 7fdbb0e0f640 1 -- 172.21.15.60:0/1956197228 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac0d0bd0 con 0x7fdbac0cef70 2024-04-25T10:16:47.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.917+0000 7fdbb0e0f640 1 --2- 172.21.15.60:0/1956197228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbac0cef70 0x7fdbac0b4c90 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0097c0 tx=0x7fdb9c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3bc7e16c93fe7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.917+0000 7fdbaaffd640 1 -- 172.21.15.60:0/1956197228 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c0b1070 con 0x7fdbac0cef70 2024-04-25T10:16:47.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.917+0000 7fdbaaffd640 1 -- 172.21.15.60:0/1956197228 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9c0a8050 con 0x7fdbac0cef70 2024-04-25T10:16:47.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.917+0000 7fdbaaffd640 1 -- 172.21.15.60:0/1956197228 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c0ac860 con 0x7fdbac0cef70 2024-04-25T10:16:47.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.917+0000 7fdbb2872640 1 -- 172.21.15.60:0/1956197228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbac0cef70 msgr2=0x7fdbac0b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.917+0000 7fdbb2872640 1 --2- 172.21.15.60:0/1956197228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbac0cef70 0x7fdbac0b4c90 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0097c0 tx=0x7fdb9c01c3e0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.917+0000 7fdbb2872640 1 -- 172.21.15.60:0/1956197228 shutdown_connections 2024-04-25T10:16:47.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.917+0000 7fdbb2872640 1 --2- 172.21.15.60:0/1956197228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbac0cef70 0x7fdbac0b4c90 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.917+0000 7fdbb2872640 1 --2- 172.21.15.60:0/1956197228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbac0713e0 0x7fdbac0cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.917+0000 7fdbb2872640 1 -- 172.21.15.60:0/1956197228 >> 172.21.15.60:0/1956197228 conn(0x7fdbac06f530 msgr2=0x7fdbac06f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.917+0000 7fdbb2872640 1 -- 172.21.15.60:0/1956197228 shutdown_connections 2024-04-25T10:16:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.917+0000 7fdbb2872640 1 -- 172.21.15.60:0/1956197228 wait complete. 2024-04-25T10:16:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.917+0000 7fdbb2872640 1 Processor -- start 2024-04-25T10:16:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.917+0000 7fdbb2872640 1 -- start start 2024-04-25T10:16:47.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbb2872640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbac0713e0 0x7fdbac135750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbb2872640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbac0cef70 0x7fdbac1361d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbb2872640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbac145320 0x7fdbac0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbb2872640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0d0970 con 0x7fdbac0cef70 2024-04-25T10:16:47.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbb2872640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0d07f0 con 0x7fdbac145320 2024-04-25T10:16:47.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbb2872640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0b59e0 con 0x7fdbac0713e0 2024-04-25T10:16:47.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbab7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbac0cef70 0x7fdbac1361d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbab7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbac0cef70 0x7fdbac1361d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54616/0 (socket says 172.21.15.60:54616) 2024-04-25T10:16:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbab7fe640 1 -- 172.21.15.60:0/491258855 learned_addr learned my addr 172.21.15.60:0/491258855 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbabfff640 1 --2- 172.21.15.60:0/491258855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbac0713e0 0x7fdbac135750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbb0e0f640 1 --2- 172.21.15.60:0/491258855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbac145320 0x7fdbac0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbab7fe640 1 -- 172.21.15.60:0/491258855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbac0713e0 msgr2=0x7fdbac135750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbab7fe640 1 --2- 172.21.15.60:0/491258855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbac0713e0 0x7fdbac135750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbab7fe640 1 -- 172.21.15.60:0/491258855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbac145320 msgr2=0x7fdbac0b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbab7fe640 1 --2- 172.21.15.60:0/491258855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbac145320 0x7fdbac0b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbab7fe640 1 -- 172.21.15.60:0/491258855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac136db0 con 0x7fdbac0cef70 2024-04-25T10:16:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbab7fe640 1 --2- 172.21.15.60:0/491258855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbac0cef70 0x7fdbac1361d0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400d800 tx=0x7fdb9400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdba8ff9640 1 -- 172.21.15.60:0/491258855 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9400b6c0 con 0x7fdbac0cef70 2024-04-25T10:16:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdba8ff9640 1 -- 172.21.15.60:0/491258855 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb94019070 con 0x7fdbac0cef70 2024-04-25T10:16:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdba8ff9640 1 -- 172.21.15.60:0/491258855 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9400ba60 con 0x7fdbac0cef70 2024-04-25T10:16:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbb2872640 1 -- 172.21.15.60:0/491258855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbac136ad0 con 0x7fdbac0cef70 2024-04-25T10:16:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbb2872640 1 -- 172.21.15.60:0/491258855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbac134f50 con 0x7fdbac0cef70 2024-04-25T10:16:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdba8ff9640 1 -- 172.21.15.60:0/491258855 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdb9400bc00 con 0x7fdbac0cef70 2024-04-25T10:16:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdba8ff9640 1 --2- 172.21.15.60:0/491258855 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb8c066f60 0x7fdb8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdba8ff9640 1 -- 172.21.15.60:0/491258855 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fdb94085500 con 0x7fdbac0cef70 2024-04-25T10:16:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.921+0000 7fdbabfff640 1 --2- 172.21.15.60:0/491258855 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb8c066f60 0x7fdb8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 Processor -- start 2024-04-25T10:16:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 -- start start 2024-04-25T10:16:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbabfff640 1 --2- 172.21.15.60:0/491258855 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb8c066f60 0x7fdb8c069420 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fdba40012f0 tx=0x7fdba4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4db6e30 0x55e0d4db7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4db7750 0x55e0d4d6cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4d6d4b0 0x55e0d4d6d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d4bb5ff0 con 0x55e0d4db7750 2024-04-25T10:16:47.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d4b3e150 con 0x55e0d4db6e30 2024-04-25T10:16:47.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0d4bb2090 con 0x55e0d4d6d4b0 2024-04-25T10:16:47.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbab7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4db7750 0x55e0d4d6cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbab7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4db7750 0x55e0d4d6cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54626/0 (socket says 172.21.15.60:54626) 2024-04-25T10:16:47.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbab7fe640 1 -- 172.21.15.60:0/3702819921 learned_addr learned my addr 172.21.15.60:0/3702819921 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:47.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb0e0f640 1 --2- 172.21.15.60:0/3702819921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4d6d4b0 0x55e0d4d6d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbabfff640 1 --2- 172.21.15.60:0/3702819921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4db6e30 0x55e0d4db7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbab7fe640 1 -- 172.21.15.60:0/3702819921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4d6d4b0 msgr2=0x55e0d4d6d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbab7fe640 1 --2- 172.21.15.60:0/3702819921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4d6d4b0 0x55e0d4d6d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbab7fe640 1 -- 172.21.15.60:0/3702819921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4db6e30 msgr2=0x55e0d4db7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbab7fe640 1 --2- 172.21.15.60:0/3702819921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4db6e30 0x55e0d4db7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbab7fe640 1 -- 172.21.15.60:0/3702819921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0d4d4c0d0 con 0x55e0d4db7750 2024-04-25T10:16:47.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb0e0f640 1 --2- 172.21.15.60:0/3702819921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4d6d4b0 0x55e0d4d6d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbab7fe640 1 --2- 172.21.15.60:0/3702819921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4db7750 0x55e0d4d6cf70 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fdb94052200 tx=0x7fdb94055a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4b0f613692cc5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdba1ffb640 1 -- 172.21.15.60:0/3702819921 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94059030 con 0x55e0d4db7750 2024-04-25T10:16:47.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdba1ffb640 1 -- 172.21.15.60:0/3702819921 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9405e4b0 con 0x55e0d4db7750 2024-04-25T10:16:47.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdba1ffb640 1 -- 172.21.15.60:0/3702819921 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9405c0c0 con 0x55e0d4db7750 2024-04-25T10:16:47.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 -- 172.21.15.60:0/3702819921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4db7750 msgr2=0x55e0d4d6cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 --2- 172.21.15.60:0/3702819921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4db7750 0x55e0d4d6cf70 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fdb94052200 tx=0x7fdb94055a10 comp rx=0 tx=0).stop 2024-04-25T10:16:47.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 -- 172.21.15.60:0/3702819921 shutdown_connections 2024-04-25T10:16:47.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 --2- 172.21.15.60:0/3702819921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4d6d4b0 0x55e0d4d6d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 --2- 172.21.15.60:0/3702819921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4db7750 0x55e0d4d6cf70 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 --2- 172.21.15.60:0/3702819921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4db6e30 0x55e0d4db7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 -- 172.21.15.60:0/3702819921 >> 172.21.15.60:0/3702819921 conn(0x55e0d4d65020 msgr2=0x55e0d4d67440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 -- 172.21.15.60:0/3702819921 shutdown_connections 2024-04-25T10:16:47.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 -- 172.21.15.60:0/3702819921 wait complete. 2024-04-25T10:16:47.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 -- start start 2024-04-25T10:16:47.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.925+0000 7fdbb59df000 1 Processor -- start 2024-04-25T10:16:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4d6d4b0 0x55e0d4db6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4db6e30 0x55e0d4d73360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4db7750 0x55e0d4d738a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d4bba620 con 0x55e0d4db6e30 2024-04-25T10:16:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d4b70690 con 0x55e0d4d6d4b0 2024-04-25T10:16:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0d4b3ea40 con 0x55e0d4db7750 2024-04-25T10:16:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 10 client.0 _ll_register_callbacks cb 0x55e0d4d53e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbab7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4db6e30 0x55e0d4d73360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbab7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4db6e30 0x55e0d4d73360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54628/0 (socket says 172.21.15.60:54628) 2024-04-25T10:16:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbab7fe640 1 -- 172.21.15.60:0/221862231 learned_addr learned my addr 172.21.15.60:0/221862231 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbabfff640 1 --2- 172.21.15.60:0/221862231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4d6d4b0 0x55e0d4db6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb0e0f640 1 --2- 172.21.15.60:0/221862231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4db7750 0x55e0d4d738a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbab7fe640 1 -- 172.21.15.60:0/221862231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4db7750 msgr2=0x55e0d4d738a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbab7fe640 1 --2- 172.21.15.60:0/221862231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4db7750 0x55e0d4d738a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbab7fe640 1 -- 172.21.15.60:0/221862231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4d6d4b0 msgr2=0x55e0d4db6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbab7fe640 1 --2- 172.21.15.60:0/221862231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4d6d4b0 0x55e0d4db6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbab7fe640 1 -- 172.21.15.60:0/221862231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0d4c631c0 con 0x55e0d4db6e30 2024-04-25T10:16:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb0e0f640 1 --2- 172.21.15.60:0/221862231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4db7750 0x55e0d4d738a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbab7fe640 1 --2- 172.21.15.60:0/221862231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4db6e30 0x55e0d4d73360 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fdb9406b040 tx=0x7fdb94085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0897947a0d00b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdb8affd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:16:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdb8affd640 1 -- 172.21.15.60:0/221862231 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94072020 con 0x55e0d4db6e30 2024-04-25T10:16:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdb8affd640 1 -- 172.21.15.60:0/221862231 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fdb94052410 con 0x55e0d4db6e30 2024-04-25T10:16:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 -- 172.21.15.60:0/221862231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb9405e4b0 con 0x55e0d4db6e30 2024-04-25T10:16:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdb8affd640 1 -- 172.21.15.60:0/221862231 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb9407d050 con 0x55e0d4db6e30 2024-04-25T10:16:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 10 client.9879 fetch_fsmap learned FSMap version 99 2024-04-25T10:16:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 -- 172.21.15.60:0/221862231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55e0d4c5f170 con 0x55e0d4db6e30 2024-04-25T10:16:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdb8affd640 1 -- 172.21.15.60:0/221862231 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9405e4b0 con 0x55e0d4db6e30 2024-04-25T10:16:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdb8affd640 1 -- 172.21.15.60:0/221862231 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdb94013070 con 0x55e0d4db6e30 2024-04-25T10:16:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 10 client.9879 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:16:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 10 client.9879 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:16:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 20 client.9879 populate_metadata read hostname 'smithi060' 2024-04-25T10:16:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 --2- 172.21.15.60:0/221862231 >> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] conn(0x55e0d4d74640 0x55e0d4dc90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 4 client.9879 mds_command: new command op to 9693 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 -- 172.21.15.60:0/221862231 --> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0d4b51c60 con 0x55e0d4d74640 2024-04-25T10:16:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbabfff640 1 --2- 172.21.15.60:0/221862231 >> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] conn(0x55e0d4d74640 0x55e0d4dc90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdb8affd640 10 client.9879 ms_handle_connect on v2:172.21.15.60:6834/178428701 2024-04-25T10:16:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbabfff640 1 --2- 172.21.15.60:0/221862231 >> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] conn(0x55e0d4d74640 0x55e0d4dc90f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a38ac8934adafcd server_cookie=52165a04aa87ad99 in_seq=0 out_seq=0 2024-04-25T10:16:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdb8affd640 1 -- 172.21.15.60:0/221862231 <== mds.0 v2:172.21.15.60:6834/178428701 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e0d4b51c60 con 0x55e0d4d74640 2024-04-25T10:16:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdb8affd640 10 client.9879 handle_command_reply: tid=0 2024-04-25T10:16:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 client.9879 shutdown 2024-04-25T10:16:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 -- 172.21.15.60:0/221862231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4db6e30 msgr2=0x55e0d4d73360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 --2- 172.21.15.60:0/221862231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4db6e30 0x55e0d4d73360 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fdb9406b040 tx=0x7fdb94085040 comp rx=0 tx=0).stop 2024-04-25T10:16:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 -- 172.21.15.60:0/221862231 shutdown_connections 2024-04-25T10:16:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 --2- 172.21.15.60:0/221862231 >> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] conn(0x55e0d4d74640 0x55e0d4dc90f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 --2- 172.21.15.60:0/221862231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4db7750 0x55e0d4d738a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 --2- 172.21.15.60:0/221862231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4db6e30 0x55e0d4d73360 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdb8affd640 0 client.9879 ms_handle_reset on v2:172.21.15.60:6834/178428701 2024-04-25T10:16:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 --2- 172.21.15.60:0/221862231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4d6d4b0 0x55e0d4db6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 -- 172.21.15.60:0/221862231 >> 172.21.15.60:0/221862231 conn(0x55e0d4d65020 msgr2=0x55e0d4d6e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 -- 172.21.15.60:0/221862231 shutdown_connections 2024-04-25T10:16:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 1 -- 172.21.15.60:0/221862231 wait complete. 2024-04-25T10:16:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.929+0000 7fdbb59df000 20 client.9879 trim_cache size 0 max 16384 2024-04-25T10:16:47.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.941+0000 7fdbb59df000 1 Processor -- start 2024-04-25T10:16:47.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.941+0000 7fdbb59df000 1 -- start start 2024-04-25T10:16:47.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4d6d4b0 0x55e0d4d95f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4d74640 0x55e0d4dcb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4db6e30 0x55e0d4dcb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac06dd30 con 0x55e0d4d74640 2024-04-25T10:16:47.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac06cf30 con 0x55e0d4db6e30 2024-04-25T10:16:47.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0b1100 con 0x55e0d4d6d4b0 2024-04-25T10:16:47.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbab7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4d74640 0x55e0d4dcb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbab7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4d74640 0x55e0d4dcb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54636/0 (socket says 172.21.15.60:54636) 2024-04-25T10:16:47.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbab7fe640 1 -- 172.21.15.60:0/1583159290 learned_addr learned my addr 172.21.15.60:0/1583159290 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:47.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbab7fe640 1 -- 172.21.15.60:0/1583159290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4d6d4b0 msgr2=0x55e0d4d95f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbabfff640 1 --2- 172.21.15.60:0/1583159290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4d6d4b0 0x55e0d4d95f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbab7fe640 1 --2- 172.21.15.60:0/1583159290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4d6d4b0 0x55e0d4d95f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb0e0f640 1 --2- 172.21.15.60:0/1583159290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4db6e30 0x55e0d4dcb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbab7fe640 1 -- 172.21.15.60:0/1583159290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4db6e30 msgr2=0x55e0d4dcb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbab7fe640 1 --2- 172.21.15.60:0/1583159290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4db6e30 0x55e0d4dcb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbab7fe640 1 -- 172.21.15.60:0/1583159290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0d4bfcde0 con 0x55e0d4d74640 2024-04-25T10:16:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbab7fe640 1 --2- 172.21.15.60:0/1583159290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4d74640 0x55e0d4dcb110 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fdb9405df80 tx=0x7fdb94081000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0262629d090f5f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdba17fa640 1 -- 172.21.15.60:0/1583159290 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94059070 con 0x55e0d4d74640 2024-04-25T10:16:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdba17fa640 1 -- 172.21.15.60:0/1583159290 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb94083070 con 0x55e0d4d74640 2024-04-25T10:16:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdba17fa640 1 -- 172.21.15.60:0/1583159290 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb940850b0 con 0x55e0d4d74640 2024-04-25T10:16:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 -- 172.21.15.60:0/1583159290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4d74640 msgr2=0x55e0d4dcb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 --2- 172.21.15.60:0/1583159290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4d74640 0x55e0d4dcb110 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fdb9405df80 tx=0x7fdb94081000 comp rx=0 tx=0).stop 2024-04-25T10:16:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 -- 172.21.15.60:0/1583159290 shutdown_connections 2024-04-25T10:16:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 --2- 172.21.15.60:0/1583159290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4db6e30 0x55e0d4dcb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 --2- 172.21.15.60:0/1583159290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4d74640 0x55e0d4dcb110 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 --2- 172.21.15.60:0/1583159290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4d6d4b0 0x55e0d4d95f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 -- 172.21.15.60:0/1583159290 >> 172.21.15.60:0/1583159290 conn(0x55e0d4d65020 msgr2=0x55e0d4dc3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 -- 172.21.15.60:0/1583159290 shutdown_connections 2024-04-25T10:16:47.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 -- 172.21.15.60:0/1583159290 wait complete. 2024-04-25T10:16:47.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 -- start start 2024-04-25T10:16:47.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 Processor -- start 2024-04-25T10:16:47.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4d6d4b0 0x55e0d4dc90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4d74640 0x55e0d4dc9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4db6e30 0x55e0d4dc9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0d4b50e00 con 0x55e0d4d6d4b0 2024-04-25T10:16:47.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0b1700 con 0x55e0d4db6e30 2024-04-25T10:16:47.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0d4bb5300 con 0x55e0d4d74640 2024-04-25T10:16:47.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb59df000 10 client.0 _ll_register_callbacks cb 0x55e0d4d34040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:47.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbabfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4d6d4b0 0x55e0d4dc90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbabfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4d6d4b0 0x55e0d4dc90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54648/0 (socket says 172.21.15.60:54648) 2024-04-25T10:16:47.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbabfff640 1 -- 172.21.15.60:0/1120570132 learned_addr learned my addr 172.21.15.60:0/1120570132 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:47.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbab7fe640 1 --2- 172.21.15.60:0/1120570132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4d74640 0x55e0d4dc9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb0e0f640 1 --2- 172.21.15.60:0/1120570132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4db6e30 0x55e0d4dc9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbabfff640 1 -- 172.21.15.60:0/1120570132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4d74640 msgr2=0x55e0d4dc9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbabfff640 1 --2- 172.21.15.60:0/1120570132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4d74640 0x55e0d4dc9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbabfff640 1 -- 172.21.15.60:0/1120570132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4db6e30 msgr2=0x55e0d4dc9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbabfff640 1 --2- 172.21.15.60:0/1120570132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4db6e30 0x55e0d4dc9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbabfff640 1 -- 172.21.15.60:0/1120570132 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0d4b578e0 con 0x55e0d4d6d4b0 2024-04-25T10:16:47.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.945+0000 7fdbb0e0f640 1 --2- 172.21.15.60:0/1120570132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4db6e30 0x55e0d4dc9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdbabfff640 1 --2- 172.21.15.60:0/1120570132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4d6d4b0 0x55e0d4dc90f0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fdba4024410 tx=0x7fdba4000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bcadf0618cbf54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdb8affd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:16:47.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdb8affd640 1 -- 172.21.15.60:0/1120570132 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4007920 con 0x55e0d4d6d4b0 2024-04-25T10:16:47.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdb8affd640 1 -- 172.21.15.60:0/1120570132 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fdba4010040 con 0x55e0d4d6d4b0 2024-04-25T10:16:47.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdbb59df000 1 -- 172.21.15.60:0/1120570132 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb94083070 con 0x55e0d4d6d4b0 2024-04-25T10:16:47.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdb8affd640 1 -- 172.21.15.60:0/1120570132 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdba4035050 con 0x55e0d4d6d4b0 2024-04-25T10:16:47.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdbb59df000 10 client.9885 fetch_fsmap learned FSMap version 99 2024-04-25T10:16:47.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdbb59df000 1 -- 172.21.15.60:0/1120570132 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55e0d4c0f8f0 con 0x55e0d4d6d4b0 2024-04-25T10:16:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdb8affd640 1 -- 172.21.15.60:0/1120570132 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb94083070 con 0x55e0d4d6d4b0 2024-04-25T10:16:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdb8affd640 1 -- 172.21.15.60:0/1120570132 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdba402b020 con 0x55e0d4d6d4b0 2024-04-25T10:16:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdbb59df000 10 client.9885 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:16:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdbb59df000 10 client.9885 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:16:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdbb59df000 20 client.9885 populate_metadata read hostname 'smithi060' 2024-04-25T10:16:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdbb59df000 1 --2- 172.21.15.60:0/1120570132 >> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] conn(0x55e0d4db7750 0x55e0d4e1efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdbb59df000 4 client.9885 mds_command: new command op to 9693 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:16:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdbb59df000 1 -- 172.21.15.60:0/1120570132 --> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e0d4bc3530 con 0x55e0d4db7750 2024-04-25T10:16:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdbab7fe640 1 --2- 172.21.15.60:0/1120570132 >> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] conn(0x55e0d4db7750 0x55e0d4e1efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdb8affd640 10 client.9885 ms_handle_connect on v2:172.21.15.60:6834/178428701 2024-04-25T10:16:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.949+0000 7fdbab7fe640 1 --2- 172.21.15.60:0/1120570132 >> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] conn(0x55e0d4db7750 0x55e0d4e1efe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b1e210ee1d7486 server_cookie=221abbd489d8af35 in_seq=0 out_seq=0 2024-04-25T10:16:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.957+0000 7fdb8affd640 1 -- 172.21.15.60:0/1120570132 <== mds.0 v2:172.21.15.60:6834/178428701 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e0d4bc3530 con 0x55e0d4db7750 2024-04-25T10:16:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.957+0000 7fdb8affd640 10 client.9885 handle_command_reply: tid=0 2024-04-25T10:16:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.957+0000 7fdbb59df000 1 client.9885 shutdown 2024-04-25T10:16:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.957+0000 7fdbb59df000 1 -- 172.21.15.60:0/1120570132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4d6d4b0 msgr2=0x55e0d4dc90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.957+0000 7fdbb59df000 1 --2- 172.21.15.60:0/1120570132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4d6d4b0 0x55e0d4dc90f0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fdba4024410 tx=0x7fdba4000f50 comp rx=0 tx=0).stop 2024-04-25T10:16:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.957+0000 7fdbb59df000 1 -- 172.21.15.60:0/1120570132 shutdown_connections 2024-04-25T10:16:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.957+0000 7fdbb59df000 1 --2- 172.21.15.60:0/1120570132 >> [v2:172.21.15.60:6834/178428701,v1:172.21.15.60:6835/178428701] conn(0x55e0d4db7750 0x55e0d4e1efe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.957+0000 7fdbb59df000 1 --2- 172.21.15.60:0/1120570132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e0d4db6e30 0x55e0d4dc9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.957+0000 7fdb8affd640 0 client.9885 ms_handle_reset on v2:172.21.15.60:6834/178428701 2024-04-25T10:16:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.957+0000 7fdbb59df000 1 --2- 172.21.15.60:0/1120570132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e0d4d74640 0x55e0d4dc9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.957+0000 7fdbb59df000 1 --2- 172.21.15.60:0/1120570132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e0d4d6d4b0 0x55e0d4dc90f0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.957+0000 7fdbb59df000 1 -- 172.21.15.60:0/1120570132 >> 172.21.15.60:0/1120570132 conn(0x55e0d4d65020 msgr2=0x55e0d4dc4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.957+0000 7fdbb59df000 1 -- 172.21.15.60:0/1120570132 shutdown_connections 2024-04-25T10:16:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.957+0000 7fdbb59df000 1 -- 172.21.15.60:0/1120570132 wait complete. 2024-04-25T10:16:47.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.961+0000 7fdbb59df000 20 client.9885 trim_cache size 0 max 16384 2024-04-25T10:16:47.966 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:16:47.967 INFO:teuthology.orchestra.run.smithi060.stdout: "message": "", 2024-04-25T10:16:47.967 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T10:16:47.967 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:16:47.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.961+0000 7fdbb2872640 1 -- 172.21.15.60:0/491258855 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb8c066f60 msgr2=0x7fdb8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.961+0000 7fdbb2872640 1 --2- 172.21.15.60:0/491258855 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb8c066f60 0x7fdb8c069420 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fdba40012f0 tx=0x7fdba4009040 comp rx=0 tx=0).stop 2024-04-25T10:16:47.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.961+0000 7fdbb2872640 1 -- 172.21.15.60:0/491258855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbac0cef70 msgr2=0x7fdbac1361d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.961+0000 7fdbb2872640 1 --2- 172.21.15.60:0/491258855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbac0cef70 0x7fdbac1361d0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400d800 tx=0x7fdb9400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.961+0000 7fdbb2872640 1 -- 172.21.15.60:0/491258855 shutdown_connections 2024-04-25T10:16:47.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.961+0000 7fdbb2872640 1 --2- 172.21.15.60:0/491258855 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdb8c066f60 0x7fdb8c069420 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.961+0000 7fdbb2872640 1 --2- 172.21.15.60:0/491258855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbac145320 0x7fdbac0b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.961+0000 7fdbb2872640 1 --2- 172.21.15.60:0/491258855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbac0cef70 0x7fdbac1361d0 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.961+0000 7fdbb2872640 1 --2- 172.21.15.60:0/491258855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbac0713e0 0x7fdbac135750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.961+0000 7fdbb2872640 1 -- 172.21.15.60:0/491258855 >> 172.21.15.60:0/491258855 conn(0x7fdbac06f530 msgr2=0x7fdbac0cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.961+0000 7fdbb2872640 1 -- 172.21.15.60:0/491258855 shutdown_connections 2024-04-25T10:16:47.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:47.961+0000 7fdbb2872640 1 -- 172.21.15.60:0/491258855 wait complete. 2024-04-25T10:16:47.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:16:48.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f6695386640 1 Processor -- start 2024-04-25T10:16:48.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f6695386640 1 -- start start 2024-04-25T10:16:48.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f6695386640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669006ac10 0x7f66900af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f6695386640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f66900b0fd0 0x7f66900b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f6695386640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66900aff30 con 0x7f669006aff0 2024-04-25T10:16:48.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f6695386640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f669015bf10 con 0x7f66900b0fd0 2024-04-25T10:16:48.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f6695386640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669006bb80 con 0x7f669006ac10 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f668effd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669006ac10 0x7f66900af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f668effd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669006ac10 0x7f66900af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:36166/0 (socket says 172.21.15.60:36166) 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f668effd640 1 -- 172.21.15.60:0/396797789 learned_addr learned my addr 172.21.15.60:0/396797789 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f6687fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f669006aff0 0x7f669006b470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:60166/0 (socket says 172.21.15.60:60166) 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f668f7fe640 1 --2- 172.21.15.60:0/396797789 >> v2:172.21.15.100:3300/0 conn(0x7f66900b0fd0 0x7f66900b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f668f7fe640 1 -- 172.21.15.60:0/396797789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669006ac10 msgr2=0x7f66900af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f668f7fe640 1 --2- 172.21.15.60:0/396797789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669006ac10 0x7f66900af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f668f7fe640 1 -- 172.21.15.60:0/396797789 >> v1:172.21.15.100:6789/0 conn(0x7f669006aff0 legacy=0x7f669006b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f668f7fe640 1 -- 172.21.15.60:0/396797789 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66900b0210 con 0x7f66900b0fd0 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.093+0000 7f668effd640 1 --2- 172.21.15.60:0/396797789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669006ac10 0x7f66900af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:48.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f668f7fe640 1 --2- 172.21.15.60:0/396797789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66900b0fd0 0x7f66900b1430 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6688009d90 tx=0x7f668801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df6b632588ca90a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f668e7fc640 1 -- 172.21.15.60:0/396797789 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880b2070 con 0x7f66900b0fd0 2024-04-25T10:16:48.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f668e7fc640 1 -- 172.21.15.60:0/396797789 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66880a5920 con 0x7f66900b0fd0 2024-04-25T10:16:48.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f668e7fc640 1 -- 172.21.15.60:0/396797789 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880a5cf0 con 0x7f66900b0fd0 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f6695386640 1 -- 172.21.15.60:0/396797789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66900b0fd0 msgr2=0x7f66900b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f6695386640 1 --2- 172.21.15.60:0/396797789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66900b0fd0 0x7f66900b1430 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6688009d90 tx=0x7f668801c970 comp rx=0 tx=0).stop 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f6695386640 1 -- 172.21.15.60:0/396797789 shutdown_connections 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f6695386640 1 --2- 172.21.15.60:0/396797789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66900b0fd0 0x7f66900b1430 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f6695386640 1 --2- 172.21.15.60:0/396797789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669006ac10 0x7f66900af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f6695386640 1 -- 172.21.15.60:0/396797789 >> 172.21.15.60:0/396797789 conn(0x7f6690072920 msgr2=0x7f6690099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f6695386640 1 -- 172.21.15.60:0/396797789 shutdown_connections 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f6695386640 1 -- 172.21.15.60:0/396797789 wait complete. 2024-04-25T10:16:48.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f6695386640 1 Processor -- start 2024-04-25T10:16:48.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f6695386640 1 -- start start 2024-04-25T10:16:48.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f6695386640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669006ac10 0x7f669013a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f6695386640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669006aff0 0x7f66901427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f6695386640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b0fd0 0x7f6690138910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f6695386640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6690168460 con 0x7f66900b0fd0 2024-04-25T10:16:48.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f6695386640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6690145510 con 0x7f669006ac10 2024-04-25T10:16:48.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f6695386640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66901685e0 con 0x7f669006aff0 2024-04-25T10:16:48.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.097+0000 7f668f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b0fd0 0x7f6690138910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f668f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b0fd0 0x7f6690138910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54658/0 (socket says 172.21.15.60:54658) 2024-04-25T10:16:48.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f668f7fe640 1 -- 172.21.15.60:0/865809391 learned_addr learned my addr 172.21.15.60:0/865809391 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:48.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f668effd640 1 --2- 172.21.15.60:0/865809391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669006ac10 0x7f669013a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f6687fff640 1 --2- 172.21.15.60:0/865809391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669006aff0 0x7f66901427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f668f7fe640 1 -- 172.21.15.60:0/865809391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669006aff0 msgr2=0x7f66901427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f668f7fe640 1 --2- 172.21.15.60:0/865809391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669006aff0 0x7f66901427d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f668f7fe640 1 -- 172.21.15.60:0/865809391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669006ac10 msgr2=0x7f669013a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f668f7fe640 1 --2- 172.21.15.60:0/865809391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669006ac10 0x7f669013a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f668f7fe640 1 -- 172.21.15.60:0/865809391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669013abb0 con 0x7f66900b0fd0 2024-04-25T10:16:48.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f668f7fe640 1 --2- 172.21.15.60:0/865809391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b0fd0 0x7f6690138910 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f66880b0040 tx=0x7f6688019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f66877fe640 1 -- 172.21.15.60:0/865809391 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880b2050 con 0x7f66900b0fd0 2024-04-25T10:16:48.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f66877fe640 1 -- 172.21.15.60:0/865809391 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66880be070 con 0x7f66900b0fd0 2024-04-25T10:16:48.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f6695386640 1 -- 172.21.15.60:0/865809391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669013ae90 con 0x7f66900b0fd0 2024-04-25T10:16:48.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f66877fe640 1 -- 172.21.15.60:0/865809391 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880ada90 con 0x7f66900b0fd0 2024-04-25T10:16:48.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f6695386640 1 -- 172.21.15.60:0/865809391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66901548b0 con 0x7f66900b0fd0 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f6695386640 1 -- 172.21.15.60:0/865809391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6690074770 con 0x7f66900b0fd0 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f66877fe640 1 -- 172.21.15.60:0/865809391 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f66880adc30 con 0x7f66900b0fd0 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f66877fe640 1 --2- 172.21.15.60:0/865809391 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6670066f60 0x7f6670069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f66877fe640 1 -- 172.21.15.60:0/865809391 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f66881200b0 con 0x7f66900b0fd0 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.101+0000 7f668effd640 1 --2- 172.21.15.60:0/865809391 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6670066f60 0x7f6670069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.105+0000 7f668effd640 1 --2- 172.21.15.60:0/865809391 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6670066f60 0x7f6670069420 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6680006cb0 tx=0x7f6680009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.105+0000 7f66877fe640 1 -- 172.21.15.60:0/865809391 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6688016560 con 0x7f66900b0fd0 2024-04-25T10:16:48.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:48.205 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:48.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.297+0000 7f6695386640 1 -- 172.21.15.60:0/865809391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f66900af8c0 con 0x7f66900b0fd0 2024-04-25T10:16:48.305 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:16:48.297+0000 7f3a792d6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:16:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.301+0000 7f66877fe640 1 -- 172.21.15.60:0/865809391 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) v1 ==== 161+0+0 (secure 0 0 0) 0x7f66880f0aa0 con 0x7f66900b0fd0 2024-04-25T10:16:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:16:48.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.309+0000 7f6695386640 1 -- 172.21.15.60:0/865809391 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6670066f60 msgr2=0x7f6670069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.309+0000 7f6695386640 1 --2- 172.21.15.60:0/865809391 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6670066f60 0x7f6670069420 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6680006cb0 tx=0x7f6680009240 comp rx=0 tx=0).stop 2024-04-25T10:16:48.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.309+0000 7f6695386640 1 -- 172.21.15.60:0/865809391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b0fd0 msgr2=0x7f6690138910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.309+0000 7f6695386640 1 --2- 172.21.15.60:0/865809391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b0fd0 0x7f6690138910 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f66880b0040 tx=0x7f6688019710 comp rx=0 tx=0).stop 2024-04-25T10:16:48.313 INFO:tasks.ceph.mds.c.smithi060.stderr: -2314> 2024-04-25T10:16:34.378+0000 7f7af8fe6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:16:48.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.309+0000 7f6695386640 1 -- 172.21.15.60:0/865809391 shutdown_connections 2024-04-25T10:16:48.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.309+0000 7f6695386640 1 --2- 172.21.15.60:0/865809391 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6670066f60 0x7f6670069420 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.309+0000 7f6695386640 1 --2- 172.21.15.60:0/865809391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66900b0fd0 0x7f6690138910 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.309+0000 7f6695386640 1 --2- 172.21.15.60:0/865809391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f669006aff0 0x7f66901427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.309+0000 7f6695386640 1 --2- 172.21.15.60:0/865809391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f669006ac10 0x7f669013a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.309+0000 7f6695386640 1 -- 172.21.15.60:0/865809391 >> 172.21.15.60:0/865809391 conn(0x7f6690072920 msgr2=0x7f669006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.309+0000 7f6695386640 1 -- 172.21.15.60:0/865809391 shutdown_connections 2024-04-25T10:16:48.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.309+0000 7f6695386640 1 -- 172.21.15.60:0/865809391 wait complete. 2024-04-25T10:16:48.331 DEBUG:teuthology.orchestra.run.smithi060:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-04-25T10:16:48.342 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:16:48.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787c9f3400 1 -- start start 2024-04-25T10:16:48.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787c9f3400 1 Processor -- start 2024-04-25T10:16:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787c9f3400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a5e50000 0x5620a5249080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787c9f3400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a508fc00 0x5620a5248b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787c9f3400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a508f800 0x5620a5249b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787c9f3400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a5f98000 con 0x5620a508f800 2024-04-25T10:16:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787c9f3400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a5f98180 con 0x5620a508fc00 2024-04-25T10:16:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787c9f3400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620a5f98300 con 0x5620a5e50000 2024-04-25T10:16:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787b9b3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a508f800 0x5620a5249b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787b9b3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a508f800 0x5620a5249b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54684/0 (socket says 172.21.15.60:54684) 2024-04-25T10:16:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787b9b3640 1 -- 172.21.15.60:0/4286437084 learned_addr learned my addr 172.21.15.60:0/4286437084 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:48.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787b1b2640 1 --2- 172.21.15.60:0/4286437084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a5e50000 0x5620a5249080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787a9b1640 1 --2- 172.21.15.60:0/4286437084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a508fc00 0x5620a5248b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787b9b3640 1 -- 172.21.15.60:0/4286437084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a5e50000 msgr2=0x5620a5249080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787b9b3640 1 --2- 172.21.15.60:0/4286437084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a5e50000 0x5620a5249080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787b9b3640 1 -- 172.21.15.60:0/4286437084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a508fc00 msgr2=0x5620a5248b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787b9b3640 1 --2- 172.21.15.60:0/4286437084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a508fc00 0x5620a5248b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787b9b3640 1 -- 172.21.15.60:0/4286437084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620a514eb40 con 0x5620a508f800 2024-04-25T10:16:48.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787b9b3640 1 --2- 172.21.15.60:0/4286437084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a508f800 0x5620a5249b80 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x5620a5eeb140 tx=0x5620a51512c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c9c1aee870f1da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f78789ad640 1 -- 172.21.15.60:0/4286437084 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5620a50c41a0 con 0x5620a508f800 2024-04-25T10:16:48.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f78789ad640 1 -- 172.21.15.60:0/4286437084 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5620a50d8c40 con 0x5620a508f800 2024-04-25T10:16:48.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f78789ad640 1 -- 172.21.15.60:0/4286437084 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5620a50c44e0 con 0x5620a508f800 2024-04-25T10:16:48.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787c9f3400 1 -- 172.21.15.60:0/4286437084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620a514f0e0 con 0x5620a508f800 2024-04-25T10:16:48.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787c9f3400 1 -- 172.21.15.60:0/4286437084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5620a514f2c0 con 0x5620a508f800 2024-04-25T10:16:48.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787c9f3400 4 waiting for MDS map... 2024-04-25T10:16:48.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f78789ad640 1 -- 172.21.15.60:0/4286437084 <== mon.0 v2:172.21.15.60:3300/0 4 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x5620a5e5c400 con 0x5620a508f800 2024-04-25T10:16:48.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f78789ad640 1 -- 172.21.15.60:0/4286437084 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 100) v1 ==== 2723+0+0 (secure 0 0 0) 0x5620a50e5180 con 0x5620a508f800 2024-04-25T10:16:48.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787c9f3400 4 Got MDS map 100 2024-04-25T10:16:48.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787c9f3400 10 main: JournalTool::main 2024-04-25T10:16:48.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.357+0000 7f787c9f3400 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:16:48.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 Processor -- start 2024-04-25T10:16:48.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 -- start start 2024-04-25T10:16:48.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e50400 0x5620a5249600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a5e64800 0x5620a5248580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a5e64c00 0x5620a524a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a5f98480 con 0x5620a5e50400 2024-04-25T10:16:48.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a5f98600 con 0x5620a5e64c00 2024-04-25T10:16:48.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620a5f98780 con 0x5620a5e64800 2024-04-25T10:16:48.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e50400 0x5620a5249600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e50400 0x5620a5249600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54688/0 (socket says 172.21.15.60:54688) 2024-04-25T10:16:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 -- 172.21.15.60:0/1364342439 learned_addr learned my addr 172.21.15.60:0/1364342439 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 -- 172.21.15.60:0/1364342439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a5e64800 msgr2=0x5620a5248580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 --2- 172.21.15.60:0/1364342439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a5e64800 0x5620a5248580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 -- 172.21.15.60:0/1364342439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a5e64c00 msgr2=0x5620a524a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 --2- 172.21.15.60:0/1364342439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a5e64c00 0x5620a524a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 -- 172.21.15.60:0/1364342439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620a514e1e0 con 0x5620a5e50400 2024-04-25T10:16:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 --2- 172.21.15.60:0/1364342439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e50400 0x5620a5249600 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x5620a5f9c720 tx=0x5620a6016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e0e0450dcaa6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f78769a9640 1 -- 172.21.15.60:0/1364342439 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5620a50c41a0 con 0x5620a5e50400 2024-04-25T10:16:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f78769a9640 1 -- 172.21.15.60:0/1364342439 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5620a50d81c0 con 0x5620a5e50400 2024-04-25T10:16:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f78769a9640 1 -- 172.21.15.60:0/1364342439 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5620a50c44e0 con 0x5620a5e50400 2024-04-25T10:16:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 -- 172.21.15.60:0/1364342439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e50400 msgr2=0x5620a5249600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 --2- 172.21.15.60:0/1364342439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e50400 0x5620a5249600 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x5620a5f9c720 tx=0x5620a6016000 comp rx=0 tx=0).stop 2024-04-25T10:16:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 -- 172.21.15.60:0/1364342439 shutdown_connections 2024-04-25T10:16:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 --2- 172.21.15.60:0/1364342439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a5e64c00 0x5620a524a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 --2- 172.21.15.60:0/1364342439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a5e64800 0x5620a5248580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 --2- 172.21.15.60:0/1364342439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e50400 0x5620a5249600 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 -- 172.21.15.60:0/1364342439 >> 172.21.15.60:0/1364342439 conn(0x5620a5e50800 msgr2=0x5620a50bb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 -- 172.21.15.60:0/1364342439 shutdown_connections 2024-04-25T10:16:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 -- 172.21.15.60:0/1364342439 wait complete. 2024-04-25T10:16:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 Processor -- start 2024-04-25T10:16:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 -- start start 2024-04-25T10:16:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e50800 0x5620a524b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a5e65800 0x5620a524ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a5e65400 0x5620a524a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a5f99e00 con 0x5620a5e50800 2024-04-25T10:16:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a5f99c80 con 0x5620a5e65800 2024-04-25T10:16:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620a5f99b00 con 0x5620a5e65400 2024-04-25T10:16:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e50800 0x5620a524b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e50800 0x5620a524b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54694/0 (socket says 172.21.15.60:54694) 2024-04-25T10:16:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 -- 172.21.15.60:0/1345351169 learned_addr learned my addr 172.21.15.60:0/1345351169 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 -- 172.21.15.60:0/1345351169 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a5e65400 msgr2=0x5620a524a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 --2- 172.21.15.60:0/1345351169 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a5e65400 0x5620a524a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 -- 172.21.15.60:0/1345351169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a5e65800 msgr2=0x5620a524ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 --2- 172.21.15.60:0/1345351169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a5e65800 0x5620a524ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 -- 172.21.15.60:0/1345351169 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620a514fc20 con 0x5620a5e50800 2024-04-25T10:16:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787a9b1640 1 --2- 172.21.15.60:0/1345351169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e50800 0x5620a524b180 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x5620a5f9d3e0 tx=0x5620a51513e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f78751a6640 1 -- 172.21.15.60:0/1345351169 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5620a50c4340 con 0x5620a5e50800 2024-04-25T10:16:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f78751a6640 1 -- 172.21.15.60:0/1345351169 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5620a50d9340 con 0x5620a5e50800 2024-04-25T10:16:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 -- 172.21.15.60:0/1345351169 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5620a514fe00 con 0x5620a5e50800 2024-04-25T10:16:48.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f78751a6640 1 -- 172.21.15.60:0/1345351169 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5620a50c41a0 con 0x5620a5e50800 2024-04-25T10:16:48.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 1 -- 172.21.15.60:0/1345351169 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620a514fc20 con 0x5620a5e50800 2024-04-25T10:16:48.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.361+0000 7f787c9f3400 4 main: JournalTool: resolving pool 16 2024-04-25T10:16:48.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f78751a6640 1 -- 172.21.15.60:0/1345351169 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x5620a5ff6c00 con 0x5620a5e50800 2024-04-25T10:16:48.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f78751a6640 1 --2- 172.21.15.60:0/1345351169 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5620a53b6800 0x5620a524b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f78751a6640 1 -- 172.21.15.60:0/1345351169 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x5620a53c0400 con 0x5620a5e50800 2024-04-25T10:16:48.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787c9f3400 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:16:48.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787c9f3400 4 main: Executing for rank 0 2024-04-25T10:16:48.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787b1b2640 1 --2- 172.21.15.60:0/1345351169 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5620a53b6800 0x5620a524b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787c9f3400 1 -- start start 2024-04-25T10:16:48.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787c9f3400 1 Processor -- start 2024-04-25T10:16:48.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787b1b2640 1 --2- 172.21.15.60:0/1345351169 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5620a53b6800 0x5620a524b700 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x5620a519cb10 tx=0x5620a5e54060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787c9f3400 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a5e64c00 0x5620a524a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787c9f3400 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e64800 0x5620a5248580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787c9f3400 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a53b6c00 0x5620a5249600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787c9f3400 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a5f99800 con 0x5620a5e64800 2024-04-25T10:16:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787c9f3400 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a5f99680 con 0x5620a5e64c00 2024-04-25T10:16:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787c9f3400 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620a5f98480 con 0x5620a53b6c00 2024-04-25T10:16:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787b1b2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e64800 0x5620a5248580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787b1b2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e64800 0x5620a5248580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54698/0 (socket says 172.21.15.60:54698) 2024-04-25T10:16:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787b1b2640 1 -- 172.21.15.60:0/126831862 learned_addr learned my addr 172.21.15.60:0/126831862 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787a9b1640 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a53b6c00 0x5620a5249600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787b9b3640 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a5e64c00 0x5620a524a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787b1b2640 1 -- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a53b6c00 msgr2=0x5620a5249600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787b1b2640 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a53b6c00 0x5620a5249600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787b1b2640 1 -- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a5e64c00 msgr2=0x5620a524a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787b1b2640 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a5e64c00 0x5620a524a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787b1b2640 1 -- 172.21.15.60:0/126831862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620a54f4960 con 0x5620a5e64800 2024-04-25T10:16:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787b1b2640 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e64800 0x5620a5248580 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x5620a5469560 tx=0x5620a5e54360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4243c1acd3b28bee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f78721a0640 1 -- 172.21.15.60:0/126831862 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5620a50c4b60 con 0x5620a5e64800 2024-04-25T10:16:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f78721a0640 1 -- 172.21.15.60:0/126831862 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5620a50d9500 con 0x5620a5e64800 2024-04-25T10:16:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f78721a0640 1 -- 172.21.15.60:0/126831862 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5620a53f2000 con 0x5620a5e64800 2024-04-25T10:16:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620a54f4b40 con 0x5620a5e64800 2024-04-25T10:16:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5620a54f4d20 con 0x5620a5e64800 2024-04-25T10:16:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.365+0000 7f787c9f3400 4 waiting for MDS map... 2024-04-25T10:16:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.369+0000 7f78721a0640 1 -- 172.21.15.60:0/126831862 <== mon.0 v2:172.21.15.60:3300/0 4 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x5620a5506400 con 0x5620a5e64800 2024-04-25T10:16:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.369+0000 7f78721a0640 1 -- 172.21.15.60:0/126831862 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 100) v1 ==== 2723+0+0 (secure 0 0 0) 0x5620a5514000 con 0x5620a5e64800 2024-04-25T10:16:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.369+0000 7f787c9f3400 4 Got MDS map 100 2024-04-25T10:16:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.369+0000 7f787c9f3400 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5620a53b7000 0x5620a5520000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.369+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:1 16.f 16:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x5620a53b7400 con 0x5620a53b7000 2024-04-25T10:16:48.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.369+0000 7f787b9b3640 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5620a53b7000 0x5620a5520000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.369+0000 7f787b9b3640 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5620a53b7000 0x5620a5520000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=eb352e533417a064 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787b9b3640 1 -- 172.21.15.60:0/126831862 <== osd.0 v2:172.21.15.60:6801/4056167511 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v112'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5620a520a000 con 0x5620a53b7000 2024-04-25T10:16:48.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:2 16.14 16:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x5620a53b7800 con 0x5620a53b7000 2024-04-25T10:16:48.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787b9b3640 1 -- 172.21.15.60:0/126831862 <== osd.0 v2:172.21.15.60:6801/4056167511 2 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v112'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5620a520a000 con 0x5620a53b7000 2024-04-25T10:16:48.382 INFO:teuthology.orchestra.run.smithi060.stdout:writing EResetJournal entry 2024-04-25T10:16:48.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:3 16.14 16:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x5620a53b7c00 con 0x5620a53b7000 2024-04-25T10:16:48.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787c9f3400 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x5620a5509400 0x5620a5520580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:4 16.1b 16:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x5620a5509800 con 0x5620a5509400 2024-04-25T10:16:48.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787c9f3400 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5620a5509c00 0x5620a5520b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:5 16.b 16:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x5620a53b7400 con 0x5620a5509c00 2024-04-25T10:16:48.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:6 16.1f 16:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x5620a53b7800 con 0x5620a5509400 2024-04-25T10:16:48.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787c9f3400 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x5620a53b7c00 0x5620a5521080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:7 16.1a 16:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x5620a5e64400 con 0x5620a53b7c00 2024-04-25T10:16:48.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787c9f3400 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5620a5516000 0x5620a5521600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:8 16.1e 16:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x5620a5516400 con 0x5620a5516000 2024-04-25T10:16:48.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787b1b2640 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x5620a53b7c00 0x5620a5521080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787b9b3640 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5620a5509c00 0x5620a5520b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787a9b1640 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x5620a5509400 0x5620a5520580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787a9b1640 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5620a5516000 0x5620a5521600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.373+0000 7f787b9b3640 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5620a5509c00 0x5620a5520b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=9e34a569abdf7c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.377+0000 7f787b1b2640 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x5620a53b7c00 0x5620a5521080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=475fd285aa6e7ae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.377+0000 7f787a9b1640 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x5620a5509400 0x5620a5520580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=de8483a121ad25ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.377+0000 7f787a9b1640 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5620a5516000 0x5620a5521600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=5aefa6cd6ba2c3c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.377+0000 7f787b9b3640 1 -- 172.21.15.60:0/126831862 <== osd.0 v2:172.21.15.60:6801/4056167511 3 ==== osd_op_reply(3 200.00000001 [delete] v112'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5620a520a000 con 0x5620a53b7000 2024-04-25T10:16:48.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.377+0000 7f787b1b2640 1 -- 172.21.15.60:0/126831862 <== osd.3 v2:172.21.15.60:6814/1867957156 1 ==== osd_op_reply(7 200.00000005 [delete] v112'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5620a520a500 con 0x5620a53b7c00 2024-04-25T10:16:48.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.377+0000 7f787b9b3640 1 -- 172.21.15.60:0/126831862 <== osd.1 v2:172.21.15.60:6824/3313552458 1 ==== osd_op_reply(5 200.00000003 [delete] v112'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5620a520a000 con 0x5620a5509c00 2024-04-25T10:16:48.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.381+0000 7f787a9b1640 1 -- 172.21.15.60:0/126831862 <== osd.5 v2:172.21.15.100:6800/3542120824 1 ==== osd_op_reply(4 200.00000002 [delete] v112'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5620a520a780 con 0x5620a5509400 2024-04-25T10:16:48.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.381+0000 7f787099d640 1 -- 172.21.15.60:0/126831862 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:9 16.14 16:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x5620a5542000 con 0x5620a53b7000 2024-04-25T10:16:48.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.381+0000 7f787a9b1640 1 -- 172.21.15.60:0/126831862 <== osd.5 v2:172.21.15.100:6800/3542120824 2 ==== osd_op_reply(6 200.00000004 [delete] v112'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5620a520a780 con 0x5620a5509400 2024-04-25T10:16:48.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.381+0000 7f787b9b3640 1 -- 172.21.15.60:0/126831862 <== osd.0 v2:172.21.15.60:6801/4056167511 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v112'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5620a520a000 con 0x5620a53b7000 2024-04-25T10:16:48.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787a9b1640 1 -- 172.21.15.60:0/126831862 <== osd.7 v2:172.21.15.100:6816/2771485684 1 ==== osd_op_reply(8 200.00000006 [delete] v112'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5620a520a780 con 0x5620a5516000 2024-04-25T10:16:48.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 4 Successfully wrote new journal pointer and header for rank 8:0 2024-04-25T10:16:48.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5620a53b7000 msgr2=0x5620a5520000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x5620a53b7000 0x5620a5520000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5620a5509c00 msgr2=0x5620a5520b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x5620a5509c00 0x5620a5520b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x5620a53b7c00 msgr2=0x5620a5521080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x5620a53b7c00 0x5620a5521080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x5620a5509400 msgr2=0x5620a5520580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x5620a5509400 0x5620a5520580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787a9b1640 1 -- 172.21.15.60:0/126831862 reap_dead start 2024-04-25T10:16:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787a9b1640 1 -- 172.21.15.60:0/126831862 reap_dead start 2024-04-25T10:16:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5620a5516000 msgr2=0x5620a5521600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5620a5516000 0x5620a5521600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e64800 msgr2=0x5620a5248580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e64800 0x5620a5248580 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x5620a5469560 tx=0x5620a5e54360 comp rx=0 tx=0).stop 2024-04-25T10:16:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 shutdown_connections 2024-04-25T10:16:48.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5620a5516000 0x5620a5521600 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/126831862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e64800 0x5620a5248580 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 >> 172.21.15.60:0/126831862 conn(0x5620a5e65000 msgr2=0x5620a50bb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 shutdown_connections 2024-04-25T10:16:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/126831862 wait complete. 2024-04-25T10:16:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/1345351169 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5620a53b6800 msgr2=0x5620a524b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/1345351169 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5620a53b6800 0x5620a524b700 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x5620a519cb10 tx=0x5620a5e54060 comp rx=0 tx=0).stop 2024-04-25T10:16:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/1345351169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e50800 msgr2=0x5620a524b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/1345351169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e50800 0x5620a524b180 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x5620a5f9d3e0 tx=0x5620a51513e0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/1345351169 shutdown_connections 2024-04-25T10:16:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/1345351169 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x5620a53b6800 0x5620a524b700 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/1345351169 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a5e65400 0x5620a524a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/1345351169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a5e65800 0x5620a524ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/1345351169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a5e50800 0x5620a524b180 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/1345351169 >> 172.21.15.60:0/1345351169 conn(0x5620a5e50400 msgr2=0x5620a50bb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/1345351169 shutdown_connections 2024-04-25T10:16:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/1345351169 wait complete. 2024-04-25T10:16:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/4286437084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a508f800 msgr2=0x5620a5249b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/4286437084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a508f800 0x5620a5249b80 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x5620a5eeb140 tx=0x5620a51512c0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/4286437084 shutdown_connections 2024-04-25T10:16:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/4286437084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5620a508f800 0x5620a5249b80 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/4286437084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5620a508fc00 0x5620a5248b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 --2- 172.21.15.60:0/4286437084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5620a5e50000 0x5620a5249080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/4286437084 >> 172.21.15.60:0/4286437084 conn(0x5620a508ec00 msgr2=0x5620a50bc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/4286437084 shutdown_connections 2024-04-25T10:16:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.389+0000 7f787c9f3400 1 -- 172.21.15.60:0/4286437084 wait complete. 2024-04-25T10:16:48.399 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.068169, result: writing EResetJournal entry 2024-04-25T10:16:48.399 DEBUG:teuthology.orchestra.run.smithi060:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-25T10:16:48.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.461+0000 7f4e47c2f8c0 1 -- start start 2024-04-25T10:16:48.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.461+0000 7f4e47c2f8c0 1 Processor -- start 2024-04-25T10:16:48.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c45f21f800 0x55c45f3d8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c45f21f400 0x55c45f3d8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45f21f000 0x55c45f3d9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c460118180 con 0x55c45f21f400 2024-04-25T10:16:48.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c460118300 con 0x55c45f21f000 2024-04-25T10:16:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c460118480 con 0x55c45f21f800 2024-04-25T10:16:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e45bed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c45f21f400 0x55c45f3d8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e45bed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c45f21f400 0x55c45f3d8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54710/0 (socket says 172.21.15.60:54710) 2024-04-25T10:16:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e45bed640 1 -- 172.21.15.60:0/368670001 learned_addr learned my addr 172.21.15.60:0/368670001 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e45bed640 1 -- 172.21.15.60:0/368670001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c45f21f800 msgr2=0x55c45f3d8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e45bed640 1 --2- 172.21.15.60:0/368670001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c45f21f800 0x55c45f3d8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e45bed640 1 -- 172.21.15.60:0/368670001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45f21f000 msgr2=0x55c45f3d9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e46bef640 1 --2- 172.21.15.60:0/368670001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45f21f000 0x55c45f3d9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e45bed640 1 --2- 172.21.15.60:0/368670001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45f21f000 0x55c45f3d9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e45bed640 1 -- 172.21.15.60:0/368670001 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c45f2441e0 con 0x55c45f21f400 2024-04-25T10:16:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e45bed640 1 --2- 172.21.15.60:0/368670001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c45f21f400 0x55c45f3d8b00 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x55c4600610e0 tx=0x55c45f31c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6273cd0a3767101f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e43be9640 1 -- 172.21.15.60:0/368670001 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c45f2549c0 con 0x55c45f21f400 2024-04-25T10:16:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e43be9640 1 -- 172.21.15.60:0/368670001 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c45f268e00 con 0x55c45f21f400 2024-04-25T10:16:48.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e43be9640 1 -- 172.21.15.60:0/368670001 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c45f2544e0 con 0x55c45f21f400 2024-04-25T10:16:48.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/368670001 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c45f2452c0 con 0x55c45f21f400 2024-04-25T10:16:48.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/368670001 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55c45f2443c0 con 0x55c45f21f400 2024-04-25T10:16:48.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 4 waiting for MDS map... 2024-04-25T10:16:48.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e43be9640 1 -- 172.21.15.60:0/368670001 <== mon.0 v2:172.21.15.60:3300/0 4 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x55c460128200 con 0x55c45f21f400 2024-04-25T10:16:48.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e43be9640 1 -- 172.21.15.60:0/368670001 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 100) v1 ==== 2723+0+0 (secure 0 0 0) 0x55c45f275180 con 0x55c45f21f400 2024-04-25T10:16:48.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 4 Got MDS map 100 2024-04-25T10:16:48.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 4 datascan.main: Using metadata pool output 2024-04-25T10:16:48.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 4 datascan.main: connecting to RADOS... 2024-04-25T10:16:48.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 1 Processor -- start 2024-04-25T10:16:48.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 1 -- start start 2024-04-25T10:16:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c45ffe1800 0x55c45f3d9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c45ffe1c00 0x55c45f3d9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c460196000 0x55c45f3da100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c460118600 con 0x55c45ffe1800 2024-04-25T10:16:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c460118780 con 0x55c460196000 2024-04-25T10:16:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e47c2f8c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c460118900 con 0x55c45ffe1c00 2024-04-25T10:16:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e46bef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c45ffe1800 0x55c45f3d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e46bef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c45ffe1800 0x55c45f3d9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54724/0 (socket says 172.21.15.60:54724) 2024-04-25T10:16:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e46bef640 1 -- 172.21.15.60:0/763680170 learned_addr learned my addr 172.21.15.60:0/763680170 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e45bed640 1 --2- 172.21.15.60:0/763680170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c460196000 0x55c45f3da100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e463ee640 1 --2- 172.21.15.60:0/763680170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c45ffe1c00 0x55c45f3d9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e46bef640 1 -- 172.21.15.60:0/763680170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c45ffe1c00 msgr2=0x55c45f3d9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e46bef640 1 --2- 172.21.15.60:0/763680170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c45ffe1c00 0x55c45f3d9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e46bef640 1 -- 172.21.15.60:0/763680170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c460196000 msgr2=0x55c45f3da100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e46bef640 1 --2- 172.21.15.60:0/763680170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c460196000 0x55c45f3da100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e46bef640 1 -- 172.21.15.60:0/763680170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c45f2441e0 con 0x55c45ffe1800 2024-04-25T10:16:48.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e463ee640 1 --2- 172.21.15.60:0/763680170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c45ffe1c00 0x55c45f3d9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:48.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e46bef640 1 --2- 172.21.15.60:0/763680170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c45ffe1800 0x55c45f3d9600 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x55c45ffeac30 tx=0x55c45f31de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c05a7803e5af41b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e41be5640 1 -- 172.21.15.60:0/763680170 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c45f2549c0 con 0x55c45ffe1800 2024-04-25T10:16:48.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e41be5640 1 -- 172.21.15.60:0/763680170 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c45f268540 con 0x55c45ffe1800 2024-04-25T10:16:48.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.465+0000 7f4e41be5640 1 -- 172.21.15.60:0/763680170 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c45f2544e0 con 0x55c45ffe1800 2024-04-25T10:16:48.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/763680170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c45ffe1800 msgr2=0x55c45f3d9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/763680170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c45ffe1800 0x55c45f3d9600 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x55c45ffeac30 tx=0x55c45f31de00 comp rx=0 tx=0).stop 2024-04-25T10:16:48.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/763680170 shutdown_connections 2024-04-25T10:16:48.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/763680170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c460196000 0x55c45f3da100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/763680170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c45ffe1c00 0x55c45f3d9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/763680170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c45ffe1800 0x55c45f3d9600 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/763680170 >> 172.21.15.60:0/763680170 conn(0x55c45f21ec00 msgr2=0x55c45f24d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/763680170 shutdown_connections 2024-04-25T10:16:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/763680170 wait complete. 2024-04-25T10:16:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 Processor -- start 2024-04-25T10:16:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 -- start start 2024-04-25T10:16:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c45ffe1c00 0x55c45f3db180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45ffe1800 0x55c45f3dac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c460197800 0x55c45f3da680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c460119c80 con 0x55c460197800 2024-04-25T10:16:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c460119b00 con 0x55c45ffe1800 2024-04-25T10:16:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c460119980 con 0x55c45ffe1c00 2024-04-25T10:16:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e45bed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c460197800 0x55c45f3da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e45bed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c460197800 0x55c45f3da680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54732/0 (socket says 172.21.15.60:54732) 2024-04-25T10:16:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 learned_addr learned my addr 172.21.15.60:0/276555391 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e46bef640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c45ffe1c00 0x55c45f3db180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e463ee640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45ffe1800 0x55c45f3dac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c45ffe1c00 msgr2=0x55c45f3db180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e45bed640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c45ffe1c00 0x55c45f3db180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45ffe1800 msgr2=0x55c45f3dac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e45bed640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45ffe1800 0x55c45f3dac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c45f2454a0 con 0x55c460197800 2024-04-25T10:16:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e463ee640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45ffe1800 0x55c45f3dac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e45bed640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c460197800 0x55c45f3da680 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x55c45ffeb860 tx=0x55c45f31c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e403e2640 1 -- 172.21.15.60:0/276555391 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c45f254b60 con 0x55c460197800 2024-04-25T10:16:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e403e2640 1 -- 172.21.15.60:0/276555391 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c45f268c40 con 0x55c460197800 2024-04-25T10:16:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c45f245a40 con 0x55c460197800 2024-04-25T10:16:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e403e2640 1 -- 172.21.15.60:0/276555391 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c45f254340 con 0x55c460197800 2024-04-25T10:16:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c45f245c20 con 0x55c460197800 2024-04-25T10:16:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 4 datascan.init: resolving metadata pool 16 2024-04-25T10:16:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e403e2640 1 -- 172.21.15.60:0/276555391 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x55c45f54cc00 con 0x55c460197800 2024-04-25T10:16:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e403e2640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55c4601a7800 0x55c45f3db700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e403e2640 1 -- 172.21.15.60:0/276555391 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x55c460199200 con 0x55c460197800 2024-04-25T10:16:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:16:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-04-25T10:16:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:16:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x55c460197400 0x55c45f3da100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460197000 con 0x55c460197400 2024-04-25T10:16:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.469+0000 7f4e46bef640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55c4601a7800 0x55c45f3db700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e463ee640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x55c460197400 0x55c45f3da100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e46bef640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55c4601a7800 0x55c45f3db700 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x55c45ffeac30 tx=0x55c45f31dd40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e463ee640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x55c460197400 0x55c45f3da100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.7 v2:172.21.15.100:6816/2771485684 1 ==== osd_op_reply(1 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c460197400 2024-04-25T10:16:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e413e4640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55c4601a7c00 0x55c45f64a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:2 17.20 17:04000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64e000 con 0x55c4601a7c00 2024-04-25T10:16:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e46bef640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55c4601a7c00 0x55c45f64a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e46bef640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55c4601a7c00 0x55c45f64a000 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.1 v2:172.21.15.60:6824/3313552458 1 ==== osd_op_reply(2 [pgnls start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c4601a7c00 2024-04-25T10:16:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e41be5640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55c45f64e800 0x55c45f64a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:3 17.10 17:08000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64ec00 con 0x55c45f64e800 2024-04-25T10:16:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e463ee640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55c45f64e800 0x55c45f64a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e463ee640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55c45f64e800 0x55c45f64a580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.3 v2:172.21.15.60:6814/1867957156 1 ==== osd_op_reply(3 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64e800 2024-04-25T10:16:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e413e4640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x55c45f64e400 0x55c45f64ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:4 17.30 17:0c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64ec00 con 0x55c45f64e400 2024-04-25T10:16:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e45bed640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x55c45f64e400 0x55c45f64ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e45bed640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x55c45f64e400 0x55c45f64ab00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.0 v2:172.21.15.60:6801/4056167511 1 ==== osd_op_reply(4 [pgnls start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c45f64e400 2024-04-25T10:16:48.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.473+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:5 17.8 17:10000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64f000 con 0x55c460197400 2024-04-25T10:16:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.7 v2:172.21.15.100:6816/2771485684 2 ==== osd_op_reply(5 [pgnls start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c460197400 2024-04-25T10:16:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:6 17.28 17:14000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64f400 con 0x55c45f64e800 2024-04-25T10:16:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.3 v2:172.21.15.60:6814/1867957156 2 ==== osd_op_reply(6 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64e800 2024-04-25T10:16:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:7 17.18 17:18000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64f800 con 0x55c45f64e400 2024-04-25T10:16:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.0 v2:172.21.15.60:6801/4056167511 2 ==== osd_op_reply(7 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c45f64e400 2024-04-25T10:16:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:8 17.38 17:1c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460197000 con 0x55c460197400 2024-04-25T10:16:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.7 v2:172.21.15.100:6816/2771485684 3 ==== osd_op_reply(8 [pgnls start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c460197400 2024-04-25T10:16:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:9 17.4 17:20000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64fc00 con 0x55c45f64e400 2024-04-25T10:16:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.0 v2:172.21.15.60:6801/4056167511 3 ==== osd_op_reply(9 [pgnls start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c45f64e400 2024-04-25T10:16:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:10 17.24 17:24000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460197000 con 0x55c45f64e800 2024-04-25T10:16:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.3 v2:172.21.15.60:6814/1867957156 3 ==== osd_op_reply(10 [pgnls start_epoch 112 out=49b] v110'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64e800 2024-04-25T10:16:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:11 17.14 17:28000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65e000 con 0x55c45f64e800 2024-04-25T10:16:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.3 v2:172.21.15.60:6814/1867957156 4 ==== osd_op_reply(11 [pgnls start_epoch 112 out=49b] v110'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64e800 2024-04-25T10:16:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:12 17.34 17:2c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64f000 con 0x55c45f64e400 2024-04-25T10:16:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.0 v2:172.21.15.60:6801/4056167511 4 ==== osd_op_reply(12 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c45f64e400 2024-04-25T10:16:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:13 17.c 17:30000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65e400 con 0x55c460197400 2024-04-25T10:16:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.7 v2:172.21.15.100:6816/2771485684 4 ==== osd_op_reply(13 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c460197400 2024-04-25T10:16:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e413e4640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55c45f64f400 0x55c45f64b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:14 17.2c 17:34000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64ec00 con 0x55c45f64f400 2024-04-25T10:16:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.477+0000 7f4e46bef640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55c45f64f400 0x55c45f64b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e46bef640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55c45f64f400 0x55c45f64b080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.4 v2:172.21.15.100:6808/3965541654 1 ==== osd_op_reply(14 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c45f64f400 2024-04-25T10:16:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:15 17.1c 17:38000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65e800 con 0x55c4601a7c00 2024-04-25T10:16:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.1 v2:172.21.15.60:6824/3313552458 2 ==== osd_op_reply(15 [pgnls start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c4601a7c00 2024-04-25T10:16:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:16 17.3c 17:3c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64f800 con 0x55c45f64e400 2024-04-25T10:16:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.0 v2:172.21.15.60:6801/4056167511 5 ==== osd_op_reply(16 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c45f64e400 2024-04-25T10:16:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:17 17.2 17:40000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65ec00 con 0x55c45f64f400 2024-04-25T10:16:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.4 v2:172.21.15.100:6808/3965541654 2 ==== osd_op_reply(17 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c45f64f400 2024-04-25T10:16:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e413e4640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55c45f64fc00 0x55c45f64b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:18 17.22 17:44000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c4601a7400 con 0x55c45f64fc00 2024-04-25T10:16:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e463ee640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55c45f64fc00 0x55c45f64b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e463ee640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55c45f64fc00 0x55c45f64b600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.5 v2:172.21.15.100:6800/3542120824 1 ==== osd_op_reply(18 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64fc00 2024-04-25T10:16:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:19 17.12 17:48000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64e000 con 0x55c45f64e800 2024-04-25T10:16:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.3 v2:172.21.15.60:6814/1867957156 5 ==== osd_op_reply(19 [pgnls start_epoch 112 out=81b] v111'22 uv22 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64e800 2024-04-25T10:16:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:20 17.12 17:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460196c00 con 0x55c45f64e800 2024-04-25T10:16:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.3 v2:172.21.15.60:6814/1867957156 6 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64e800 2024-04-25T10:16:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.481+0000 7f4e47c2f8c0 10 datascan.operator(): handling object 1099511627778.0 2024-04-25T10:16:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.485+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:21 17.12 17:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460196800 con 0x55c45f64e800 2024-04-25T10:16:48.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.485+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.3 v2:172.21.15.60:6814/1867957156 7 ==== osd_op_reply(21 10000000002.00000000 [call] v112'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64e800 2024-04-25T10:16:48.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.485+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:22 17.32 17:4c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460196400 con 0x55c460197400 2024-04-25T10:16:48.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.485+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.7 v2:172.21.15.100:6816/2771485684 5 ==== osd_op_reply(22 [pgnls start_epoch 112 out=49b] v110'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c460197400 2024-04-25T10:16:48.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.485+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:23 17.a 17:50000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64ec00 con 0x55c460197400 2024-04-25T10:16:48.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.485+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.7 v2:172.21.15.100:6816/2771485684 6 ==== osd_op_reply(23 [pgnls start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c460197400 2024-04-25T10:16:48.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.485+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:24 17.2a 17:54000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c4601a7000 con 0x55c45f64e800 2024-04-25T10:16:48.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.485+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.3 v2:172.21.15.60:6814/1867957156 8 ==== osd_op_reply(24 [pgnls start_epoch 112 out=49b] v110'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64e800 2024-04-25T10:16:48.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.485+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:25 17.1a 17:58000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c4601a7400 con 0x55c45f64e400 2024-04-25T10:16:48.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.0 v2:172.21.15.60:6801/4056167511 6 ==== osd_op_reply(25 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c45f64e400 2024-04-25T10:16:48.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:26 17.3a 17:5c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64e000 con 0x55c4601a7c00 2024-04-25T10:16:48.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.1 v2:172.21.15.60:6824/3313552458 3 ==== osd_op_reply(26 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c4601a7c00 2024-04-25T10:16:48.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e41be5640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x55c4601a6400 0x55c45f64c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:27 17.6 17:60000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460197000 con 0x55c4601a6400 2024-04-25T10:16:48.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e45bed640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x55c4601a6400 0x55c45f64c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e45bed640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x55c4601a6400 0x55c45f64c680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.6 v2:172.21.15.100:6824/4086390550 1 ==== osd_op_reply(27 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c4601a6400 2024-04-25T10:16:48.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:28 17.26 17:64000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64ec00 con 0x55c45f64e400 2024-04-25T10:16:48.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.0 v2:172.21.15.60:6801/4056167511 7 ==== osd_op_reply(28 [pgnls start_epoch 112 out=49b] v110'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c45f64e400 2024-04-25T10:16:48.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:29 17.16 17:68000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65e000 con 0x55c45f64fc00 2024-04-25T10:16:48.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.5 v2:172.21.15.100:6800/3542120824 2 ==== osd_op_reply(29 [pgnls start_epoch 112 out=49b] v110'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64fc00 2024-04-25T10:16:48.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:30 17.36 17:6c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64f000 con 0x55c45f64e400 2024-04-25T10:16:48.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.0 v2:172.21.15.60:6801/4056167511 8 ==== osd_op_reply(30 [pgnls start_epoch 112 out=81b] v110'33 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c45f2d2780 con 0x55c45f64e400 2024-04-25T10:16:48.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:31 17.36 17:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460196000 con 0x55c45f64e400 2024-04-25T10:16:48.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.0 v2:172.21.15.60:6801/4056167511 9 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c45f2d2780 con 0x55c45f64e400 2024-04-25T10:16:48.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e47c2f8c0 10 datascan.operator(): handling object 1099511628279.0 2024-04-25T10:16:48.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.489+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:32 17.36 17:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65fc00 con 0x55c45f64e400 2024-04-25T10:16:48.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.0 v2:172.21.15.60:6801/4056167511 10 ==== osd_op_reply(32 100000001f7.00000000 [call] v112'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c45f2d2780 con 0x55c45f64e400 2024-04-25T10:16:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x55c45f65fc00 0x55c45f3d9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:33 17.e 17:70000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460197000 con 0x55c45f65fc00 2024-04-25T10:16:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e46bef640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x55c45f65fc00 0x55c45f3d9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e46bef640 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x55c45f65fc00 0x55c45f3d9b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.2 v2:172.21.15.60:6800/1771147646 1 ==== osd_op_reply(33 [pgnls start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c45f65fc00 2024-04-25T10:16:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:34 17.2e 17:74000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460196c00 con 0x55c4601a6400 2024-04-25T10:16:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.6 v2:172.21.15.100:6824/4086390550 2 ==== osd_op_reply(34 [pgnls start_epoch 112 out=49b] v110'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c4601a6400 2024-04-25T10:16:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:35 17.1e 17:78000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65e400 con 0x55c4601a7c00 2024-04-25T10:16:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.1 v2:172.21.15.60:6824/3313552458 4 ==== osd_op_reply(35 [pgnls start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c4601a7c00 2024-04-25T10:16:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:36 17.3e 17:7c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460196800 con 0x55c45f64f400 2024-04-25T10:16:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.4 v2:172.21.15.100:6808/3965541654 3 ==== osd_op_reply(36 [pgnls start_epoch 112 out=49b] v110'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c45f64f400 2024-04-25T10:16:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:37 17.1 17:80000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65e800 con 0x55c45f64e400 2024-04-25T10:16:48.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.0 v2:172.21.15.60:6801/4056167511 11 ==== osd_op_reply(37 [pgnls start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c45f64e400 2024-04-25T10:16:48.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:38 17.21 17:84000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460196400 con 0x55c45f64e400 2024-04-25T10:16:48.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.0 v2:172.21.15.60:6801/4056167511 12 ==== osd_op_reply(38 [pgnls start_epoch 112 out=49b] v110'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c45f64e400 2024-04-25T10:16:48.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:39 17.11 17:88000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65f000 con 0x55c45f64e800 2024-04-25T10:16:48.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.3 v2:172.21.15.60:6814/1867957156 9 ==== osd_op_reply(39 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64e800 2024-04-25T10:16:48.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.493+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:40 17.31 17:8c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460197000 con 0x55c4601a7c00 2024-04-25T10:16:48.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.497+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.1 v2:172.21.15.60:6824/3313552458 5 ==== osd_op_reply(40 [pgnls start_epoch 112 out=81b] v110'15 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c45f2d2500 con 0x55c4601a7c00 2024-04-25T10:16:48.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.497+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:41 17.31 17:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460196000 con 0x55c4601a7c00 2024-04-25T10:16:48.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.497+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.1 v2:172.21.15.60:6824/3313552458 6 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c45f2d2500 con 0x55c4601a7c00 2024-04-25T10:16:48.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.497+0000 7f4e47c2f8c0 10 datascan.operator(): handling object 1099511627780.1 2024-04-25T10:16:48.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.497+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:42 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64f800 con 0x55c4601a6400 2024-04-25T10:16:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.497+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.6 v2:172.21.15.100:6824/4086390550 3 ==== osd_op_reply(42 10000000004.00000000 [call] v112'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c45f2d2780 con 0x55c4601a6400 2024-04-25T10:16:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.497+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:43 17.9 17:90000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64ec00 con 0x55c45f64e800 2024-04-25T10:16:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.497+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.3 v2:172.21.15.60:6814/1867957156 10 ==== osd_op_reply(43 [pgnls start_epoch 112 out=49b] v110'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64e800 2024-04-25T10:16:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.497+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:44 17.29 17:94000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460196800 con 0x55c45f64f400 2024-04-25T10:16:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.497+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.4 v2:172.21.15.100:6808/3965541654 4 ==== osd_op_reply(44 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c45f64f400 2024-04-25T10:16:48.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.497+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:45 17.19 17:98000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65f400 con 0x55c45f64fc00 2024-04-25T10:16:48.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.501+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.5 v2:172.21.15.100:6800/3542120824 3 ==== osd_op_reply(45 [pgnls start_epoch 112 out=81b] v111'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64fc00 2024-04-25T10:16:48.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.501+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:46 17.19 17:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64f000 con 0x55c45f64fc00 2024-04-25T10:16:48.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.501+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.5 v2:172.21.15.100:6800/3542120824 4 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64fc00 2024-04-25T10:16:48.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.501+0000 7f4e47c2f8c0 10 datascan.operator(): handling object 1099511628280.0 2024-04-25T10:16:48.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.501+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:47 17.19 17:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c4601a7400 con 0x55c45f64fc00 2024-04-25T10:16:48.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.505+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.5 v2:172.21.15.100:6800/3542120824 5 ==== osd_op_reply(47 100000001f8.00000000 [call] v112'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64fc00 2024-04-25T10:16:48.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.505+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:48 17.39 17:9c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64ec00 con 0x55c45f64fc00 2024-04-25T10:16:48.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.505+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.5 v2:172.21.15.100:6800/3542120824 6 ==== osd_op_reply(48 [pgnls start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64fc00 2024-04-25T10:16:48.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.505+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:49 17.5 17:a0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65f800 con 0x55c4601a6400 2024-04-25T10:16:48.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.505+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.6 v2:172.21.15.100:6824/4086390550 4 ==== osd_op_reply(49 [pgnls start_epoch 112 out=81b] v112'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c45f2d2780 con 0x55c4601a6400 2024-04-25T10:16:48.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.505+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:50 17.5 17:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64f000 con 0x55c4601a6400 2024-04-25T10:16:48.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.505+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.6 v2:172.21.15.100:6824/4086390550 5 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c45f2d2780 con 0x55c4601a6400 2024-04-25T10:16:48.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.505+0000 7f4e47c2f8c0 10 datascan.operator(): handling object 1099511627780.0 2024-04-25T10:16:48.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.505+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:51 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65e000 con 0x55c4601a6400 2024-04-25T10:16:48.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.509+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.6 v2:172.21.15.100:6824/4086390550 6 ==== osd_op_reply(51 10000000004.00000000 [call] v112'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c45f2d2780 con 0x55c4601a6400 2024-04-25T10:16:48.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.509+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:52 17.25 17:a4000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65f000 con 0x55c4601a7c00 2024-04-25T10:16:48.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.509+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.1 v2:172.21.15.60:6824/3313552458 7 ==== osd_op_reply(52 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c4601a7c00 2024-04-25T10:16:48.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.509+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:53 17.15 17:a8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f682000 con 0x55c45f64fc00 2024-04-25T10:16:48.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.509+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.5 v2:172.21.15.100:6800/3542120824 7 ==== osd_op_reply(53 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64fc00 2024-04-25T10:16:48.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.509+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:54 17.35 17:ac000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65ec00 con 0x55c45f64f400 2024-04-25T10:16:48.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.509+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.4 v2:172.21.15.100:6808/3965541654 5 ==== osd_op_reply(54 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c45f64f400 2024-04-25T10:16:48.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.509+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:55 17.d 17:b0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f682400 con 0x55c460197400 2024-04-25T10:16:48.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.509+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.7 v2:172.21.15.100:6816/2771485684 7 ==== osd_op_reply(55 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c460197400 2024-04-25T10:16:48.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.509+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:56 17.2d 17:b4000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65e400 con 0x55c460197400 2024-04-25T10:16:48.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.513+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.7 v2:172.21.15.100:6816/2771485684 8 ==== osd_op_reply(56 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c460197400 2024-04-25T10:16:48.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.513+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:57 17.1d 17:b8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f682800 con 0x55c460197400 2024-04-25T10:16:48.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.513+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.7 v2:172.21.15.100:6816/2771485684 9 ==== osd_op_reply(57 [pgnls start_epoch 112 out=49b] v110'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c460197400 2024-04-25T10:16:48.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.513+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:58 17.3d 17:bc000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64e000 con 0x55c460197400 2024-04-25T10:16:48.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.513+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.7 v2:172.21.15.100:6816/2771485684 10 ==== osd_op_reply(58 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c460197400 2024-04-25T10:16:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.513+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:59 17.3 17:c0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f682c00 con 0x55c45f64f400 2024-04-25T10:16:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.513+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.4 v2:172.21.15.100:6808/3965541654 6 ==== osd_op_reply(59 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c45f64f400 2024-04-25T10:16:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.513+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:60 17.23 17:c4000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64ec00 con 0x55c45f64f400 2024-04-25T10:16:48.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.513+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.4 v2:172.21.15.100:6808/3965541654 7 ==== osd_op_reply(60 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c45f64f400 2024-04-25T10:16:48.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.513+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:61 17.13 17:c8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f683000 con 0x55c45f64e800 2024-04-25T10:16:48.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.513+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.3 v2:172.21.15.60:6814/1867957156 11 ==== osd_op_reply(61 [pgnls start_epoch 112 out=49b] v110'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c45f64e800 2024-04-25T10:16:48.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.513+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:62 17.33 17:cc000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65f000 con 0x55c460197400 2024-04-25T10:16:48.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 <== osd.7 v2:172.21.15.100:6816/2771485684 11 ==== osd_op_reply(62 [pgnls start_epoch 112 out=49b] v110'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2000 con 0x55c460197400 2024-04-25T10:16:48.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:63 17.b 17:d0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f683400 con 0x55c45f64e400 2024-04-25T10:16:48.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.0 v2:172.21.15.60:6801/4056167511 13 ==== osd_op_reply(63 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c45f64e400 2024-04-25T10:16:48.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:64 17.2b 17:d4000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65ec00 con 0x55c45f64f400 2024-04-25T10:16:48.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.4 v2:172.21.15.100:6808/3965541654 8 ==== osd_op_reply(64 [pgnls start_epoch 112 out=49b] v110'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c45f64f400 2024-04-25T10:16:48.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:65 17.1b 17:d8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f683800 con 0x55c45f64f400 2024-04-25T10:16:48.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.4 v2:172.21.15.100:6808/3965541654 9 ==== osd_op_reply(65 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c45f64f400 2024-04-25T10:16:48.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:66 17.3b 17:dc000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65e400 con 0x55c45f65fc00 2024-04-25T10:16:48.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.2 v2:172.21.15.60:6800/1771147646 2 ==== osd_op_reply(66 [pgnls start_epoch 112 out=49b] v110'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c45f65fc00 2024-04-25T10:16:48.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:67 17.7 17:e0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64f800 con 0x55c45f64e400 2024-04-25T10:16:48.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.0 v2:172.21.15.60:6801/4056167511 14 ==== osd_op_reply(67 [pgnls start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c45f64e400 2024-04-25T10:16:48.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:68 17.27 17:e4000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c4601a7400 con 0x55c45f64f400 2024-04-25T10:16:48.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.4 v2:172.21.15.100:6808/3965541654 10 ==== osd_op_reply(68 [pgnls start_epoch 112 out=49b] v110'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c45f64f400 2024-04-25T10:16:48.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:69 17.17 17:e8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64f000 con 0x55c4601a6400 2024-04-25T10:16:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.6 v2:172.21.15.100:6824/4086390550 7 ==== osd_op_reply(69 [pgnls start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c4601a6400 2024-04-25T10:16:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:70 17.37 17:ec000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460197000 con 0x55c45f64e400 2024-04-25T10:16:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.0 v2:172.21.15.60:6801/4056167511 15 ==== osd_op_reply(70 [pgnls start_epoch 112 out=49b] v110'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c45f64e400 2024-04-25T10:16:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.517+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:71 17.f 17:f0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f64e000 con 0x55c4601a7c00 2024-04-25T10:16:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.1 v2:172.21.15.60:6824/3313552458 8 ==== osd_op_reply(71 [pgnls start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c4601a7c00 2024-04-25T10:16:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:72 17.2f 17:f4000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460196000 con 0x55c45f64f400 2024-04-25T10:16:48.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e46bef640 1 -- 172.21.15.60:0/276555391 <== osd.4 v2:172.21.15.100:6808/3965541654 11 ==== osd_op_reply(72 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2500 con 0x55c45f64f400 2024-04-25T10:16:48.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e41be5640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:73 17.1f 17:f8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c45f65e800 con 0x55c4601a6400 2024-04-25T10:16:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.6 v2:172.21.15.100:6824/4086390550 8 ==== osd_op_reply(73 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c4601a6400 2024-04-25T10:16:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e413e4640 1 -- 172.21.15.60:0/276555391 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:74 17.3f 17:fc000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55c460196800 con 0x55c45f64e400 2024-04-25T10:16:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e45bed640 1 -- 172.21.15.60:0/276555391 <== osd.0 v2:172.21.15.60:6801/4056167511 16 ==== osd_op_reply(74 [pgnls start_epoch 112 out=49b] v110'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c45f2d2780 con 0x55c45f64e400 2024-04-25T10:16:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x55c45f64e400 msgr2=0x55c45f64ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x55c45f64e400 0x55c45f64ab00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55c4601a7c00 msgr2=0x55c45f64a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x55c4601a7c00 0x55c45f64a000 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x55c45f65fc00 msgr2=0x55c45f3d9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x55c45f65fc00 0x55c45f3d9b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55c45f64e800 msgr2=0x55c45f64a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x55c45f64e800 0x55c45f64a580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 reap_dead start 2024-04-25T10:16:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55c45f64f400 msgr2=0x55c45f64b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x55c45f64f400 0x55c45f64b080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 reap_dead start 2024-04-25T10:16:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55c45f64fc00 msgr2=0x55c45f64b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x55c45f64fc00 0x55c45f64b600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x55c4601a6400 msgr2=0x55c45f64c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x55c4601a6400 0x55c45f64c680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x55c460197400 msgr2=0x55c45f3da100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x55c460197400 0x55c45f3da100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55c4601a7800 msgr2=0x55c45f3db700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x55c4601a7800 0x55c45f3db700 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x55c45ffeac30 tx=0x55c45f31dd40 comp rx=0 tx=0).stop 2024-04-25T10:16:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c460197800 msgr2=0x55c45f3da680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/276555391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c460197800 0x55c45f3da680 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x55c45ffeb860 tx=0x55c45f31c300 comp rx=0 tx=0).stop 2024-04-25T10:16:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e463ee640 1 -- 172.21.15.60:0/276555391 reap_dead start 2024-04-25T10:16:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 shutdown_connections 2024-04-25T10:16:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 >> 172.21.15.60:0/276555391 conn(0x55c45f21ec00 msgr2=0x55c45f24d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 shutdown_connections 2024-04-25T10:16:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/276555391 wait complete. 2024-04-25T10:16:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/368670001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c45f21f400 msgr2=0x55c45f3d8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/368670001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c45f21f400 0x55c45f3d8b00 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x55c4600610e0 tx=0x55c45f31c000 comp rx=0 tx=0).stop 2024-04-25T10:16:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/368670001 shutdown_connections 2024-04-25T10:16:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/368670001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45f21f000 0x55c45f3d9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/368670001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c45f21f400 0x55c45f3d8b00 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 --2- 172.21.15.60:0/368670001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c45f21f800 0x55c45f3d8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/368670001 >> 172.21.15.60:0/368670001 conn(0x55c45f21fc00 msgr2=0x55c45f24d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/368670001 shutdown_connections 2024-04-25T10:16:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.521+0000 7f4e47c2f8c0 1 -- 172.21.15.60:0/368670001 wait complete. 2024-04-25T10:16:48.532 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.132607, result: 2024-04-25T10:16:48.532 DEBUG:teuthology.orchestra.run.smithi060:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-04-25T10:16:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 -- start start 2024-04-25T10:16:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 Processor -- start 2024-04-25T10:16:48.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a14444400 0x563a1383d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563a13683c00 0x563a1383cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a14444800 0x563a1383db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14584000 con 0x563a14444800 2024-04-25T10:16:48.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14584180 con 0x563a14444400 2024-04-25T10:16:48.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a14584300 con 0x563a13683c00 2024-04-25T10:16:48.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5260e2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a14444800 0x563a1383db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5260e2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a14444800 0x563a1383db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54750/0 (socket says 172.21.15.60:54750) 2024-04-25T10:16:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5260e2640 1 -- 172.21.15.60:0/1305551525 learned_addr learned my addr 172.21.15.60:0/1305551525 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5258e1640 1 --2- 172.21.15.60:0/1305551525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a14444400 0x563a1383d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5260e2640 1 -- 172.21.15.60:0/1305551525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563a13683c00 msgr2=0x563a1383cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5250e0640 1 --2- 172.21.15.60:0/1305551525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563a13683c00 0x563a1383cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5260e2640 1 --2- 172.21.15.60:0/1305551525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563a13683c00 0x563a1383cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5260e2640 1 -- 172.21.15.60:0/1305551525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a14444400 msgr2=0x563a1383d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5260e2640 1 --2- 172.21.15.60:0/1305551525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a14444400 0x563a1383d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5260e2640 1 -- 172.21.15.60:0/1305551525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a136a81e0 con 0x563a14444800 2024-04-25T10:16:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5258e1640 1 --2- 172.21.15.60:0/1305551525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a14444400 0x563a1383d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5260e2640 1 --2- 172.21.15.60:0/1305551525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a14444800 0x563a1383db80 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x563a1375e6f0 tx=0x563a144420c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3108ebc9bdc526 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5230dc640 1 -- 172.21.15.60:0/1305551525 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563a136b84e0 con 0x563a14444800 2024-04-25T10:16:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5230dc640 1 -- 172.21.15.60:0/1305551525 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a136ccc40 con 0x563a14444800 2024-04-25T10:16:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5230dc640 1 -- 172.21.15.60:0/1305551525 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563a136b89c0 con 0x563a14444800 2024-04-25T10:16:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 -- 172.21.15.60:0/1305551525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a136a9c20 con 0x563a14444800 2024-04-25T10:16:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 -- 172.21.15.60:0/1305551525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563a136a94a0 con 0x563a14444800 2024-04-25T10:16:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 4 waiting for MDS map... 2024-04-25T10:16:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5230dc640 1 -- 172.21.15.60:0/1305551525 <== mon.0 v2:172.21.15.60:3300/0 4 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x563a14590200 con 0x563a14444800 2024-04-25T10:16:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5230dc640 1 -- 172.21.15.60:0/1305551525 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 100) v1 ==== 2723+0+0 (secure 0 0 0) 0x563a136d9180 con 0x563a14444800 2024-04-25T10:16:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 4 Got MDS map 100 2024-04-25T10:16:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-04-25T10:16:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 4 datascan.main: Using metadata pool output 2024-04-25T10:16:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 4 datascan.main: connecting to RADOS... 2024-04-25T10:16:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 Processor -- start 2024-04-25T10:16:48.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 -- start start 2024-04-25T10:16:48.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a14445400 0x563a1383d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563a14445800 0x563a1383c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a145f6000 0x563a1383e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14584480 con 0x563a145f6000 2024-04-25T10:16:48.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14584600 con 0x563a14445400 2024-04-25T10:16:48.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5271228c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a14584780 con 0x563a14445800 2024-04-25T10:16:48.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5250e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a145f6000 0x563a1383e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5250e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a145f6000 0x563a1383e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54754/0 (socket says 172.21.15.60:54754) 2024-04-25T10:16:48.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5250e0640 1 -- 172.21.15.60:0/4178621370 learned_addr learned my addr 172.21.15.60:0/4178621370 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:48.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5258e1640 1 --2- 172.21.15.60:0/4178621370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563a14445800 0x563a1383c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5250e0640 1 -- 172.21.15.60:0/4178621370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563a14445800 msgr2=0x563a1383c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5250e0640 1 --2- 172.21.15.60:0/4178621370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563a14445800 0x563a1383c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5250e0640 1 -- 172.21.15.60:0/4178621370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a14445400 msgr2=0x563a1383d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.605+0000 7fd5250e0640 1 --2- 172.21.15.60:0/4178621370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a14445400 0x563a1383d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5250e0640 1 -- 172.21.15.60:0/4178621370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a136a81e0 con 0x563a145f6000 2024-04-25T10:16:48.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5258e1640 1 --2- 172.21.15.60:0/4178621370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563a14445800 0x563a1383c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:48.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5250e0640 1 --2- 172.21.15.60:0/4178621370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a145f6000 0x563a1383e100 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x563a144c9d40 tx=0x563a146000c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c8553c6f87430b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5210d8640 1 -- 172.21.15.60:0/4178621370 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563a136b84e0 con 0x563a145f6000 2024-04-25T10:16:48.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5210d8640 1 -- 172.21.15.60:0/4178621370 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a136cd340 con 0x563a145f6000 2024-04-25T10:16:48.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5210d8640 1 -- 172.21.15.60:0/4178621370 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563a136b89c0 con 0x563a145f6000 2024-04-25T10:16:48.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 -- 172.21.15.60:0/4178621370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a145f6000 msgr2=0x563a1383e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 --2- 172.21.15.60:0/4178621370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a145f6000 0x563a1383e100 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x563a144c9d40 tx=0x563a146000c0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 -- 172.21.15.60:0/4178621370 shutdown_connections 2024-04-25T10:16:48.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 --2- 172.21.15.60:0/4178621370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a145f6000 0x563a1383e100 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 --2- 172.21.15.60:0/4178621370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563a14445800 0x563a1383c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 --2- 172.21.15.60:0/4178621370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a14445400 0x563a1383d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 -- 172.21.15.60:0/4178621370 >> 172.21.15.60:0/4178621370 conn(0x563a14445c00 msgr2=0x563a145ec000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 -- 172.21.15.60:0/4178621370 shutdown_connections 2024-04-25T10:16:48.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 -- 172.21.15.60:0/4178621370 wait complete. 2024-04-25T10:16:48.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 Processor -- start 2024-04-25T10:16:48.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 -- start start 2024-04-25T10:16:48.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a14445800 0x563a1383f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a14445400 0x563a1383ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563a145f7000 0x563a1383e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14585e00 con 0x563a14445400 2024-04-25T10:16:48.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14585c80 con 0x563a14445800 2024-04-25T10:16:48.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a14585b00 con 0x563a145f7000 2024-04-25T10:16:48.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5258e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a14445400 0x563a1383ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5258e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a14445400 0x563a1383ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54768/0 (socket says 172.21.15.60:54768) 2024-04-25T10:16:48.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 learned_addr learned my addr 172.21.15.60:0/1261603403 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:48.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5250e0640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563a145f7000 0x563a1383e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563a145f7000 msgr2=0x563a1383e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5258e1640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563a145f7000 0x563a1383e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a14445800 msgr2=0x563a1383f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5258e1640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a14445800 0x563a1383f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a136a92c0 con 0x563a14445400 2024-04-25T10:16:48.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5250e0640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563a145f7000 0x563a1383e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:48.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5258e1640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a14445400 0x563a1383ec00 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x563a1459dad0 tx=0x563a146002a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd51f8d5640 1 -- 172.21.15.60:0/1261603403 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563a136b84e0 con 0x563a14445400 2024-04-25T10:16:48.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd51f8d5640 1 -- 172.21.15.60:0/1261603403 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a136cc1c0 con 0x563a14445400 2024-04-25T10:16:48.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563a136a9e00 con 0x563a14445400 2024-04-25T10:16:48.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd51f8d5640 1 -- 172.21.15.60:0/1261603403 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563a136b89c0 con 0x563a14445400 2024-04-25T10:16:48.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a136a92c0 con 0x563a14445400 2024-04-25T10:16:48.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd5271228c0 4 datascan.init: resolving metadata pool 16 2024-04-25T10:16:48.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.609+0000 7fd51f8d5640 1 -- 172.21.15.60:0/1261603403 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x563a14608c00 con 0x563a14445400 2024-04-25T10:16:48.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd51f8d5640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x563a139bec00 0x563a1383f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd51f8d5640 1 -- 172.21.15.60:0/1261603403 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x563a14606c00 con 0x563a14445400 2024-04-25T10:16:48.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5271228c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:16:48.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5271228c0 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-04-25T10:16:48.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5271228c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:16:48.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x563a145f6c00 0x563a1383e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:1 16.14 16:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a145f6800 con 0x563a145f6c00 2024-04-25T10:16:48.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5250e0640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x563a145f6c00 0x563a1383e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5250e0640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x563a139bec00 0x563a1383f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5250e0640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x563a145f6c00 0x563a1383e100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv34 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x563a145f6400 0x563a1383c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:2 16.1f 16:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a145f6000 con 0x563a145f6400 2024-04-25T10:16:48.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5250e0640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x563a139bec00 0x563a1383f700 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x563a144c9bf0 tx=0x563a14600120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5260e2640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x563a145f6400 0x563a1383c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5260e2640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x563a145f6400 0x563a1383c580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.5 v2:172.21.15.100:6800/3542120824 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x563a13792500 con 0x563a145f6400 2024-04-25T10:16:48.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x563a139bf000 0x563a1383d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:3 17.0 17:00000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a139bf400 con 0x563a139bf000 2024-04-25T10:16:48.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5258e1640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x563a139bf000 0x563a1383d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.613+0000 7fd5258e1640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x563a139bf000 0x563a1383d080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.7 v2:172.21.15.100:6816/2771485684 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a139bf000 2024-04-25T10:16:48.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5208d7640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x563a13ac0000 0x563a13ac6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:4 17.20 17:04000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac0400 con 0x563a13ac0000 2024-04-25T10:16:48.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5250e0640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x563a13ac0000 0x563a13ac6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5250e0640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x563a13ac0000 0x563a13ac6000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.1 v2:172.21.15.60:6824/3313552458 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a13ac0000 2024-04-25T10:16:48.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5210d8640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x563a13ac0c00 0x563a13ac6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:5 17.10 17:08000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1000 con 0x563a13ac0c00 2024-04-25T10:16:48.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5260e2640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x563a13ac0c00 0x563a13ac6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5260e2640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x563a13ac0c00 0x563a13ac6580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:6 17.30 17:0c000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac0800 con 0x563a145f6c00 2024-04-25T10:16:48.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:7 17.8 17:10000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1400 con 0x563a139bf000 2024-04-25T10:16:48.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.7 v2:172.21.15.100:6816/2771485684 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a139bf000 2024-04-25T10:16:48.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:8 17.28 17:14000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1800 con 0x563a13ac0c00 2024-04-25T10:16:48.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:9 17.18 17:18000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ace000 con 0x563a145f6c00 2024-04-25T10:16:48.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 3 ==== osd_op_reply(9 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:10 17.38 17:1c000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1c00 con 0x563a139bf000 2024-04-25T10:16:48.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.7 v2:172.21.15.100:6816/2771485684 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a139bf000 2024-04-25T10:16:48.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.617+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:11 17.4 17:20000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ace400 con 0x563a145f6c00 2024-04-25T10:16:48.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:12 17.24 17:24000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac0400 con 0x563a13ac0c00 2024-04-25T10:16:48.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 112 out=49b] v110'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:13 17.14 17:28000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ace800 con 0x563a13ac0c00 2024-04-25T10:16:48.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 4 ==== osd_op_reply(13 [pgnls-filter start_epoch 112 out=49b] v110'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:14 17.34 17:2c000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac0800 con 0x563a145f6c00 2024-04-25T10:16:48.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:15 17.c 17:30000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ace000 con 0x563a139bf000 2024-04-25T10:16:48.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.7 v2:172.21.15.100:6816/2771485684 4 ==== osd_op_reply(15 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a139bf000 2024-04-25T10:16:48.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5208d7640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x563a145f6800 0x563a13ac6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:16 17.2c 17:34000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a145f7800 con 0x563a145f6800 2024-04-25T10:16:48.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5258e1640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x563a145f6800 0x563a13ac6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5258e1640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x563a145f6800 0x563a13ac6b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.4 v2:172.21.15.100:6808/3965541654 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a145f6800 2024-04-25T10:16:48.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:17 17.1c 17:38000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ace400 con 0x563a13ac0000 2024-04-25T10:16:48.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.1 v2:172.21.15.60:6824/3313552458 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a13ac0000 2024-04-25T10:16:48.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:18 17.3c 17:3c000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ace000 con 0x563a145f6c00 2024-04-25T10:16:48.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 6 ==== osd_op_reply(18 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.621+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:19 17.2 17:40000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13acec00 con 0x563a145f6800 2024-04-25T10:16:48.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.4 v2:172.21.15.100:6808/3965541654 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a145f6800 2024-04-25T10:16:48.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:20 17.22 17:44000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13acf400 con 0x563a145f6400 2024-04-25T10:16:48.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.5 v2:172.21.15.100:6800/3542120824 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a145f6400 2024-04-25T10:16:48.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:21 17.12 17:48000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13acf000 con 0x563a13ac0c00 2024-04-25T10:16:48.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 112 out=81b] v112'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:22 17.0 17:00000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a139bf800 con 0x563a139bf000 2024-04-25T10:16:48.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.7 v2:172.21.15.100:6816/2771485684 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a139bf000 2024-04-25T10:16:48.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:23 17.20 17:04000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1000 con 0x563a13ac0000 2024-04-25T10:16:48.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.1 v2:172.21.15.60:6824/3313552458 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a13ac0000 2024-04-25T10:16:48.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:24 17.10 17:08000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1400 con 0x563a13ac0c00 2024-04-25T10:16:48.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 6 ==== osd_op_reply(24 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:25 17.30 17:0c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1800 con 0x563a145f6c00 2024-04-25T10:16:48.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:26 17.8 17:10000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1c00 con 0x563a139bf000 2024-04-25T10:16:48.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.7 v2:172.21.15.100:6816/2771485684 6 ==== osd_op_reply(26 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a139bf000 2024-04-25T10:16:48.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:27 17.28 17:14000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac0400 con 0x563a13ac0c00 2024-04-25T10:16:48.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.625+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 7 ==== osd_op_reply(27 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:28 17.18 17:18000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a139bf400 con 0x563a145f6c00 2024-04-25T10:16:48.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 8 ==== osd_op_reply(28 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:29 17.38 17:1c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a145f6000 con 0x563a139bf000 2024-04-25T10:16:48.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.7 v2:172.21.15.100:6816/2771485684 7 ==== osd_op_reply(29 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a139bf000 2024-04-25T10:16:48.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:30 17.4 17:20000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13acec00 con 0x563a145f6c00 2024-04-25T10:16:48.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 9 ==== osd_op_reply(30 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:31 17.24 17:24000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a14444000 con 0x563a13ac0c00 2024-04-25T10:16:48.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 8 ==== osd_op_reply(31 [pgnls-filter start_epoch 112 out=49b] v110'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:32 17.14 17:28000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ace800 con 0x563a13ac0c00 2024-04-25T10:16:48.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 9 ==== osd_op_reply(32 [pgnls-filter start_epoch 112 out=49b] v110'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:33 17.34 17:2c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ace400 con 0x563a145f6c00 2024-04-25T10:16:48.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 10 ==== osd_op_reply(33 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:34 17.c 17:30000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a145f7800 con 0x563a139bf000 2024-04-25T10:16:48.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.7 v2:172.21.15.100:6816/2771485684 8 ==== osd_op_reply(34 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a139bf000 2024-04-25T10:16:48.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:35 17.2c 17:34000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ace000 con 0x563a145f6800 2024-04-25T10:16:48.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.4 v2:172.21.15.100:6808/3965541654 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a145f6800 2024-04-25T10:16:48.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:36 17.1c 17:38000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a145f6000 con 0x563a13ac0000 2024-04-25T10:16:48.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.1 v2:172.21.15.60:6824/3313552458 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a13ac0000 2024-04-25T10:16:48.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:37 17.3c 17:3c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a139bf400 con 0x563a145f6c00 2024-04-25T10:16:48.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 11 ==== osd_op_reply(37 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.629+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:38 17.2 17:40000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1c00 con 0x563a145f6800 2024-04-25T10:16:48.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.633+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.4 v2:172.21.15.100:6808/3965541654 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a145f6800 2024-04-25T10:16:48.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.633+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:39 17.22 17:44000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac0800 con 0x563a145f6400 2024-04-25T10:16:48.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.633+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.5 v2:172.21.15.100:6800/3542120824 3 ==== osd_op_reply(39 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a145f6400 2024-04-25T10:16:48.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.633+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:40 17.12 17:48000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a139bf800 con 0x563a13ac0c00 2024-04-25T10:16:48.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.633+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 10 ==== osd_op_reply(40 [pgnls-filter start_epoch 112 out=49b] v112'23 uv23 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.633+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:41 17.32 17:4c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1000 con 0x563a139bf000 2024-04-25T10:16:48.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.633+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.7 v2:172.21.15.100:6816/2771485684 9 ==== osd_op_reply(41 [pgnls-filter start_epoch 112 out=49b] v110'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a139bf000 2024-04-25T10:16:48.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.633+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:42 17.a 17:50000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13acec00 con 0x563a139bf000 2024-04-25T10:16:48.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.633+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.7 v2:172.21.15.100:6816/2771485684 10 ==== osd_op_reply(42 [pgnls-filter start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a139bf000 2024-04-25T10:16:48.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.633+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:43 17.2a 17:54000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1800 con 0x563a13ac0c00 2024-04-25T10:16:48.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.633+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 11 ==== osd_op_reply(43 [pgnls-filter start_epoch 112 out=49b] v110'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.633+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:44 17.1a 17:58000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13acf400 con 0x563a145f6c00 2024-04-25T10:16:48.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.633+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 12 ==== osd_op_reply(44 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.633+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:45 17.3a 17:5c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1400 con 0x563a13ac0000 2024-04-25T10:16:48.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.633+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.1 v2:172.21.15.60:6824/3313552458 5 ==== osd_op_reply(45 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a13ac0000 2024-04-25T10:16:48.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5208d7640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x563a13acf800 0x563a13ac7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:46 17.6 17:60000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13acfc00 con 0x563a13acf800 2024-04-25T10:16:48.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5250e0640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x563a13acf800 0x563a13ac7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5250e0640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x563a13acf800 0x563a13ac7080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.6 v2:172.21.15.100:6824/4086390550 1 ==== osd_op_reply(46 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a13acf800 2024-04-25T10:16:48.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:47 17.26 17:64000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac0400 con 0x563a145f6c00 2024-04-25T10:16:48.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 13 ==== osd_op_reply(47 [pgnls-filter start_epoch 112 out=49b] v110'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:48 17.16 17:68000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1800 con 0x563a145f6400 2024-04-25T10:16:48.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.5 v2:172.21.15.100:6800/3542120824 4 ==== osd_op_reply(48 [pgnls-filter start_epoch 112 out=49b] v110'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a145f6400 2024-04-25T10:16:48.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:49 17.36 17:6c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac0800 con 0x563a145f6c00 2024-04-25T10:16:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 14 ==== osd_op_reply(49 [pgnls-filter start_epoch 112 out=81b] v112'34 uv34 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5271228c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-04-25T10:16:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5271228c0 10 datascan.operator(): handling object 100000001f7.0 2024-04-25T10:16:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:50 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1400 con 0x563a145f6c00 2024-04-25T10:16:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 15 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) v8 ==== 416+0+194 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5271228c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-04-25T10:16:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5271228c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-04-25T10:16:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5271228c0 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-04-25T10:16:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:51 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac0400 con 0x563a13ac0c00 2024-04-25T10:16:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 12 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5271228c0 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-04-25T10:16:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x563a13ac0800 0x563a13ac7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:52 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ace400 con 0x563a13ac0800 2024-04-25T10:16:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.637+0000 7fd5260e2640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x563a13ac0800 0x563a13ac7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5260e2640 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x563a13ac0800 0x563a13ac7600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.2 v2:172.21.15.60:6800/1771147646 1 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+524 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0800 2024-04-25T10:16:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5271228c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-04-25T10:16:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5271228c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-04-25T10:16:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5271228c0 20 datascan.get_frag_of: frag is * 2024-04-25T10:16:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:53 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13acf400 con 0x563a13ac0c00 2024-04-25T10:16:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 13 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5271228c0 10 datascan.find_or_create_dirfrag: read_version = 8 2024-04-25T10:16:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5271228c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-04-25T10:16:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:54 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13acfc00 con 0x563a13ac0c00 2024-04-25T10:16:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 14 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5271228c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-25T10:16:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5271228c0 20 datascan.read_dentry: bravo_head not found in result 2024-04-25T10:16:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5271228c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-04-25T10:16:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:55 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a139bf400 con 0x563a13ac0c00 2024-04-25T10:16:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 15 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v112'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5271228c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-25T10:16:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:56 17.e 17:70000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a145f6000 con 0x563a13ac0800 2024-04-25T10:16:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.2 v2:172.21.15.60:6800/1771147646 2 ==== osd_op_reply(56 [pgnls-filter start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0800 2024-04-25T10:16:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:57 17.2e 17:74000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a139bf800 con 0x563a13acf800 2024-04-25T10:16:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.6 v2:172.21.15.100:6824/4086390550 2 ==== osd_op_reply(57 [pgnls-filter start_epoch 112 out=49b] v110'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a13acf800 2024-04-25T10:16:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:58 17.1e 17:78000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13af4000 con 0x563a13ac0000 2024-04-25T10:16:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.1 v2:172.21.15.60:6824/3313552458 6 ==== osd_op_reply(58 [pgnls-filter start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a13ac0000 2024-04-25T10:16:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:59 17.3e 17:7c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13acf000 con 0x563a145f6800 2024-04-25T10:16:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.4 v2:172.21.15.100:6808/3965541654 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 112 out=49b] v110'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a145f6800 2024-04-25T10:16:48.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.641+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:60 17.1 17:80000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13af4400 con 0x563a145f6c00 2024-04-25T10:16:48.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.645+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 16 ==== osd_op_reply(60 [pgnls-filter start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.645+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:61 17.21 17:84000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ace800 con 0x563a145f6c00 2024-04-25T10:16:48.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.645+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 17 ==== osd_op_reply(61 [pgnls-filter start_epoch 112 out=49b] v110'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.645+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:62 17.11 17:88000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13af4800 con 0x563a13ac0c00 2024-04-25T10:16:48.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.645+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 16 ==== osd_op_reply(62 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.645+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:63 17.31 17:8c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a14444000 con 0x563a13ac0000 2024-04-25T10:16:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.645+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.1 v2:172.21.15.60:6824/3313552458 7 ==== osd_op_reply(63 [pgnls-filter start_epoch 112 out=49b] v110'15 uv8 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a13ac0000 2024-04-25T10:16:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.645+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:64 17.9 17:90000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13af4c00 con 0x563a13ac0c00 2024-04-25T10:16:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.645+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 17 ==== osd_op_reply(64 [pgnls-filter start_epoch 112 out=49b] v110'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.645+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:65 17.29 17:94000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1800 con 0x563a145f6800 2024-04-25T10:16:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.645+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.4 v2:172.21.15.100:6808/3965541654 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a145f6800 2024-04-25T10:16:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.645+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:66 17.19 17:98000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13af5000 con 0x563a145f6400 2024-04-25T10:16:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.645+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.5 v2:172.21.15.100:6800/3542120824 5 ==== osd_op_reply(66 [pgnls-filter start_epoch 112 out=49b] v112'24 uv24 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a145f6400 2024-04-25T10:16:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.645+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:67 17.39 17:9c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac0400 con 0x563a145f6400 2024-04-25T10:16:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.645+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.5 v2:172.21.15.100:6800/3542120824 6 ==== osd_op_reply(67 [pgnls-filter start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a145f6400 2024-04-25T10:16:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.645+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13af5400 con 0x563a13acf800 2024-04-25T10:16:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.649+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.6 v2:172.21.15.100:6824/4086390550 3 ==== osd_op_reply(68 [pgnls-filter start_epoch 112 out=49b] v112'15 uv15 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a13acf800 2024-04-25T10:16:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.649+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:69 17.25 17:a4000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1c00 con 0x563a13ac0000 2024-04-25T10:16:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.649+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.1 v2:172.21.15.60:6824/3313552458 8 ==== osd_op_reply(69 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a13ac0000 2024-04-25T10:16:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.649+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:70 17.15 17:a8000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13af5800 con 0x563a145f6400 2024-04-25T10:16:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.649+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.5 v2:172.21.15.100:6800/3542120824 7 ==== osd_op_reply(70 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a145f6400 2024-04-25T10:16:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.649+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:71 17.35 17:ac000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a145f6000 con 0x563a145f6800 2024-04-25T10:16:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.649+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.4 v2:172.21.15.100:6808/3965541654 7 ==== osd_op_reply(71 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a145f6800 2024-04-25T10:16:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.649+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:72 17.d 17:b0000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13af5c00 con 0x563a139bf000 2024-04-25T10:16:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.649+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.7 v2:172.21.15.100:6816/2771485684 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a139bf000 2024-04-25T10:16:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.649+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:73 17.2d 17:b4000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a145f7800 con 0x563a139bf000 2024-04-25T10:16:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.649+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.7 v2:172.21.15.100:6816/2771485684 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a139bf000 2024-04-25T10:16:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.649+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:74 17.1d 17:b8000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ace800 con 0x563a139bf000 2024-04-25T10:16:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.649+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.7 v2:172.21.15.100:6816/2771485684 13 ==== osd_op_reply(74 [pgnls-filter start_epoch 112 out=49b] v110'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a139bf000 2024-04-25T10:16:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.649+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:75 17.3d 17:bc000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13af4800 con 0x563a139bf000 2024-04-25T10:16:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.653+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.7 v2:172.21.15.100:6816/2771485684 14 ==== osd_op_reply(75 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a139bf000 2024-04-25T10:16:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.653+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:76 17.3 17:c0000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1400 con 0x563a145f6800 2024-04-25T10:16:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.653+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.4 v2:172.21.15.100:6808/3965541654 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a145f6800 2024-04-25T10:16:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.653+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:77 17.23 17:c4000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13af4c00 con 0x563a145f6800 2024-04-25T10:16:48.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.653+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.4 v2:172.21.15.100:6808/3965541654 9 ==== osd_op_reply(77 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a145f6800 2024-04-25T10:16:48.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.653+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:78 17.13 17:c8000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1c00 con 0x563a13ac0c00 2024-04-25T10:16:48.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.653+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.3 v2:172.21.15.60:6814/1867957156 18 ==== osd_op_reply(78 [pgnls-filter start_epoch 112 out=49b] v110'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0c00 2024-04-25T10:16:48.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.653+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:79 17.33 17:cc000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a139bf400 con 0x563a139bf000 2024-04-25T10:16:48.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.653+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.7 v2:172.21.15.100:6816/2771485684 15 ==== osd_op_reply(79 [pgnls-filter start_epoch 112 out=49b] v110'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a139bf000 2024-04-25T10:16:48.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.653+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:80 17.b 17:d0000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1000 con 0x563a145f6c00 2024-04-25T10:16:48.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.653+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 18 ==== osd_op_reply(80 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.653+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:81 17.2b 17:d4000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ace800 con 0x563a145f6800 2024-04-25T10:16:48.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.657+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.4 v2:172.21.15.100:6808/3965541654 10 ==== osd_op_reply(81 [pgnls-filter start_epoch 112 out=49b] v110'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a145f6800 2024-04-25T10:16:48.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.657+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:82 17.1b 17:d8000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13af4000 con 0x563a145f6800 2024-04-25T10:16:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.657+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.4 v2:172.21.15.100:6808/3965541654 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a145f6800 2024-04-25T10:16:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.657+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:83 17.3b 17:dc000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ace400 con 0x563a13ac0800 2024-04-25T10:16:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.657+0000 7fd5260e2640 1 -- 172.21.15.60:0/1261603403 <== osd.2 v2:172.21.15.60:6800/1771147646 3 ==== osd_op_reply(83 [pgnls-filter start_epoch 112 out=49b] v110'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792500 con 0x563a13ac0800 2024-04-25T10:16:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.657+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:84 17.7 17:e0000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13af4400 con 0x563a145f6c00 2024-04-25T10:16:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.657+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 19 ==== osd_op_reply(84 [pgnls-filter start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.657+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:85 17.27 17:e4000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13acf400 con 0x563a145f6800 2024-04-25T10:16:48.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.657+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.4 v2:172.21.15.100:6808/3965541654 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 112 out=49b] v110'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a145f6800 2024-04-25T10:16:48.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.657+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:86 17.17 17:e8000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13af5400 con 0x563a13acf800 2024-04-25T10:16:48.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.657+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.6 v2:172.21.15.100:6824/4086390550 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a13acf800 2024-04-25T10:16:48.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.657+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:87 17.37 17:ec000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1800 con 0x563a145f6c00 2024-04-25T10:16:48.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 20 ==== osd_op_reply(87 [pgnls-filter start_epoch 112 out=49b] v110'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:88 17.f 17:f0000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a14444000 con 0x563a13ac0000 2024-04-25T10:16:48.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.1 v2:172.21.15.60:6824/3313552458 9 ==== osd_op_reply(88 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a13ac0000 2024-04-25T10:16:48.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:89 17.2f 17:f4000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13ac1400 con 0x563a145f6800 2024-04-25T10:16:48.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 <== osd.4 v2:172.21.15.100:6808/3965541654 13 ==== osd_op_reply(89 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792780 con 0x563a145f6800 2024-04-25T10:16:48.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5208d7640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:90 17.1f 17:f8000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a139bfc00 con 0x563a13acf800 2024-04-25T10:16:48.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.6 v2:172.21.15.100:6824/4086390550 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a13acf800 2024-04-25T10:16:48.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5210d8640 1 -- 172.21.15.60:0/1261603403 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:91 17.3f 17:fc000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x563a13af5c00 con 0x563a145f6c00 2024-04-25T10:16:48.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5250e0640 1 -- 172.21.15.60:0/1261603403 <== osd.0 v2:172.21.15.60:6801/4056167511 21 ==== osd_op_reply(91 [pgnls-filter start_epoch 112 out=49b] v110'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a13792000 con 0x563a145f6c00 2024-04-25T10:16:48.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x563a145f6c00 msgr2=0x563a1383e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x563a145f6c00 0x563a1383e100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x563a13ac0000 msgr2=0x563a13ac6000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x563a13ac0000 0x563a13ac6000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x563a13ac0800 msgr2=0x563a13ac7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x563a13ac0800 0x563a13ac7600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x563a13ac0c00 msgr2=0x563a13ac6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x563a13ac0c00 0x563a13ac6580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 reap_dead start 2024-04-25T10:16:48.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x563a145f6800 msgr2=0x563a13ac6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x563a145f6800 0x563a13ac6b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 reap_dead start 2024-04-25T10:16:48.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x563a145f6400 msgr2=0x563a1383c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x563a145f6400 0x563a1383c580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x563a13acf800 msgr2=0x563a13ac7080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x563a13acf800 0x563a13ac7080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x563a139bf000 msgr2=0x563a1383d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x563a139bf000 0x563a1383d080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x563a139bec00 msgr2=0x563a1383f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x563a139bec00 0x563a1383f700 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x563a144c9bf0 tx=0x563a14600120 comp rx=0 tx=0).stop 2024-04-25T10:16:48.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a14445400 msgr2=0x563a1383ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1261603403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a14445400 0x563a1383ec00 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x563a1459dad0 tx=0x563a146002a0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 reap_dead start 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5258e1640 1 -- 172.21.15.60:0/1261603403 reap_dead start 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 shutdown_connections 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 >> 172.21.15.60:0/1261603403 conn(0x563a14445c00 msgr2=0x563a145ec000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 shutdown_connections 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1261603403 wait complete. 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1305551525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a14444800 msgr2=0x563a1383db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1305551525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a14444800 0x563a1383db80 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x563a1375e6f0 tx=0x563a144420c0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1305551525 shutdown_connections 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1305551525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563a14444800 0x563a1383db80 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1305551525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563a13683c00 0x563a1383cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 --2- 172.21.15.60:0/1305551525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a14444400 0x563a1383d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1305551525 >> 172.21.15.60:0/1305551525 conn(0x563a13682c00 msgr2=0x563a136b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1305551525 shutdown_connections 2024-04-25T10:16:48.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.661+0000 7fd5271228c0 1 -- 172.21.15.60:0/1305551525 wait complete. 2024-04-25T10:16:48.672 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.140517, result: 2024-04-25T10:16:48.673 DEBUG:teuthology.orchestra.run.smithi060:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-25T10:16:48.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05c7b78c0 1 -- start start 2024-04-25T10:16:48.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05c7b78c0 1 Processor -- start 2024-04-25T10:16:48.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05c7b78c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5591805f3800 0x5591807ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05c7b78c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591805f3400 0x5591807acb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05c7b78c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591805f3000 0x5591807ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05c7b78c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591813ca600 con 0x5591805f3400 2024-04-25T10:16:48.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05c7b78c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591813ca480 con 0x5591805f3000 2024-04-25T10:16:48.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05c7b78c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591813ca300 con 0x5591805f3800 2024-04-25T10:16:48.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05a775640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591805f3400 0x5591807acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05a775640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591805f3400 0x5591807acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54778/0 (socket says 172.21.15.60:54778) 2024-04-25T10:16:48.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05a775640 1 -- 172.21.15.60:0/2607230886 learned_addr learned my addr 172.21.15.60:0/2607230886 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:48.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05b777640 1 --2- 172.21.15.60:0/2607230886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591805f3000 0x5591807ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05a775640 1 -- 172.21.15.60:0/2607230886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5591805f3800 msgr2=0x5591807ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05a775640 1 --2- 172.21.15.60:0/2607230886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5591805f3800 0x5591807ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05a775640 1 -- 172.21.15.60:0/2607230886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591805f3000 msgr2=0x5591807ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05a775640 1 --2- 172.21.15.60:0/2607230886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591805f3000 0x5591807ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05a775640 1 -- 172.21.15.60:0/2607230886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591806181e0 con 0x5591805f3400 2024-04-25T10:16:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05a775640 1 --2- 172.21.15.60:0/2607230886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591805f3400 0x5591807acb00 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x55918143d8f0 tx=0x5591806ad020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c6a6d906cd9fc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe058771640 1 -- 172.21.15.60:0/2607230886 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5591806284e0 con 0x5591805f3400 2024-04-25T10:16:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe058771640 1 -- 172.21.15.60:0/2607230886 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55918063c540 con 0x5591805f3400 2024-04-25T10:16:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe058771640 1 -- 172.21.15.60:0/2607230886 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559180628b60 con 0x5591805f3400 2024-04-25T10:16:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2607230886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591806194a0 con 0x5591805f3400 2024-04-25T10:16:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2607230886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x559180619a40 con 0x5591805f3400 2024-04-25T10:16:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05c7b78c0 4 waiting for MDS map... 2024-04-25T10:16:48.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe058771640 1 -- 172.21.15.60:0/2607230886 <== mon.0 v2:172.21.15.60:3300/0 4 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x5591814fe600 con 0x5591805f3400 2024-04-25T10:16:48.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe058771640 1 -- 172.21.15.60:0/2607230886 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 100) v1 ==== 2723+0+0 (secure 0 0 0) 0x559180649180 con 0x5591805f3400 2024-04-25T10:16:48.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05c7b78c0 4 Got MDS map 100 2024-04-25T10:16:48.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05c7b78c0 4 datascan.main: Using metadata pool output 2024-04-25T10:16:48.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05c7b78c0 4 datascan.main: connecting to RADOS... 2024-04-25T10:16:48.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05c7b78c0 1 Processor -- start 2024-04-25T10:16:48.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.745+0000 7fe05c7b78c0 1 -- start start 2024-04-25T10:16:48.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559181502c00 0x5591807ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559181503000 0x5591807adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559181503400 0x5591807ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591813ca180 con 0x559181503000 2024-04-25T10:16:48.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559180652480 con 0x559181503400 2024-04-25T10:16:48.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559180652900 con 0x559181502c00 2024-04-25T10:16:48.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05af76640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559181503000 0x5591807adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05af76640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559181503000 0x5591807adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54780/0 (socket says 172.21.15.60:54780) 2024-04-25T10:16:48.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05af76640 1 -- 172.21.15.60:0/2355400288 learned_addr learned my addr 172.21.15.60:0/2355400288 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:48.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05a775640 1 --2- 172.21.15.60:0/2355400288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559181503400 0x5591807ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05af76640 1 -- 172.21.15.60:0/2355400288 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559181502c00 msgr2=0x5591807ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05af76640 1 --2- 172.21.15.60:0/2355400288 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559181502c00 0x5591807ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05af76640 1 -- 172.21.15.60:0/2355400288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559181503400 msgr2=0x5591807ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05af76640 1 --2- 172.21.15.60:0/2355400288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559181503400 0x5591807ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05af76640 1 -- 172.21.15.60:0/2355400288 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559180619c20 con 0x559181503000 2024-04-25T10:16:48.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05af76640 1 --2- 172.21.15.60:0/2355400288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559181503000 0x5591807adb80 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x559181514c00 tx=0x559181572000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9615da74911d28ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05676d640 1 -- 172.21.15.60:0/2355400288 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5591806284e0 con 0x559181503000 2024-04-25T10:16:48.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05676d640 1 -- 172.21.15.60:0/2355400288 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55918063cc40 con 0x559181503000 2024-04-25T10:16:48.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05676d640 1 -- 172.21.15.60:0/2355400288 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559180628b60 con 0x559181503000 2024-04-25T10:16:48.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2355400288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559181503000 msgr2=0x5591807adb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2355400288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559181503000 0x5591807adb80 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x559181514c00 tx=0x559181572000 comp rx=0 tx=0).stop 2024-04-25T10:16:48.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2355400288 shutdown_connections 2024-04-25T10:16:48.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2355400288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559181503400 0x5591807ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2355400288 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559181503000 0x5591807adb80 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2355400288 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559181502c00 0x5591807ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2355400288 >> 172.21.15.60:0/2355400288 conn(0x5591805f2c00 msgr2=0x55918061f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2355400288 shutdown_connections 2024-04-25T10:16:48.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2355400288 wait complete. 2024-04-25T10:16:48.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 Processor -- start 2024-04-25T10:16:48.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 -- start start 2024-04-25T10:16:48.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559181502c00 0x5591807af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559181503000 0x5591807aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559181503400 0x5591807ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591813cb380 con 0x559181503400 2024-04-25T10:16:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591813ca180 con 0x559181503000 2024-04-25T10:16:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591813cb500 con 0x559181502c00 2024-04-25T10:16:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05a775640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559181503400 0x5591807ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05a775640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559181503400 0x5591807ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54788/0 (socket says 172.21.15.60:54788) 2024-04-25T10:16:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 learned_addr learned my addr 172.21.15.60:0/2530668225 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05b777640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559181502c00 0x5591807af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05af76640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559181503000 0x5591807aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559181502c00 msgr2=0x5591807af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05a775640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559181502c00 0x5591807af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559181503000 msgr2=0x5591807aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05a775640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559181503000 0x5591807aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591806181e0 con 0x559181503400 2024-04-25T10:16:48.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05a775640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559181503400 0x5591807ae680 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x5591815158c0 tx=0x5591806accc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe054f6a640 1 -- 172.21.15.60:0/2530668225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559180628340 con 0x559181503400 2024-04-25T10:16:48.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe054f6a640 1 -- 172.21.15.60:0/2530668225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55918063ca80 con 0x559181503400 2024-04-25T10:16:48.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe054f6a640 1 -- 172.21.15.60:0/2530668225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5591806284e0 con 0x559181503400 2024-04-25T10:16:48.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5591806194a0 con 0x559181503400 2024-04-25T10:16:48.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559180619a40 con 0x559181503400 2024-04-25T10:16:48.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.749+0000 7fe05c7b78c0 4 datascan.init: resolving metadata pool 16 2024-04-25T10:16:48.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe054f6a640 1 -- 172.21.15.60:0/2530668225 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x55918157ac00 con 0x559181503400 2024-04-25T10:16:48.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe054f6a640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x559181571c00 0x5591807af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe054f6a640 1 -- 172.21.15.60:0/2530668225 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x559181578c00 con 0x559181503400 2024-04-25T10:16:48.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05c7b78c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:16:48.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05c7b78c0 4 datascan.main: resolving metadata pool 16 2024-04-25T10:16:48.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x559181503800 0x5591807ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:1 16.0 16:00000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181503c00 con 0x559181503800 2024-04-25T10:16:48.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05b777640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x559181571c00 0x5591807af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05af76640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x559181503800 0x5591807ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05af76640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x559181503800 0x5591807ae100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 1 ==== osd_op_reply(1 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05af76640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x559181503c00 0x559180a0e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:2 16.10 16:08000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181570800 con 0x559181503c00 2024-04-25T10:16:48.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05b777640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x559181571c00 0x5591807af700 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x5591806ce1e0 tx=0x5591806f0000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05b777640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x559181503c00 0x559180a0e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05b777640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x559181503c00 0x559180a0e000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 1 ==== osd_op_reply(2 [pgnls start_epoch 112 out=177b] v111'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:3 16.8 16:10000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181570800 con 0x559181503800 2024-04-25T10:16:48.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 2 ==== osd_op_reply(3 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:4 16.18 16:18000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181570800 con 0x559181503800 2024-04-25T10:16:48.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 3 ==== osd_op_reply(4 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05af76640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x559181570800 0x559180a0e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:5 16.4 16:20000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181570000 con 0x559181570800 2024-04-25T10:16:48.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.753+0000 7fe05af76640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x559181570800 0x559180a0e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.757+0000 7fe05af76640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x559181570800 0x559180a0e580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.757+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.3 v2:172.21.15.60:6814/1867957156 1 ==== osd_op_reply(5 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702000 con 0x559181570800 2024-04-25T10:16:48.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.757+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:6 16.14 16:28000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181570000 con 0x559181503800 2024-04-25T10:16:48.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.757+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 4 ==== osd_op_reply(6 [pgnls start_epoch 112 out=149b] v112'41 uv41 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.757+0000 7fe05af76640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x559181570000 0x559180a0eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.757+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:7 16.c 16:30000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181570400 con 0x559181570000 2024-04-25T10:16:48.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.757+0000 7fe05a775640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x559181570000 0x559180a0eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.757+0000 7fe05a775640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x559181570000 0x559180a0eb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.757+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 <== osd.5 v2:172.21.15.100:6800/3542120824 1 ==== osd_op_reply(7 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702780 con 0x559181570000 2024-04-25T10:16:48.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.757+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:8 16.1c 16:38000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181570400 con 0x559181503c00 2024-04-25T10:16:48.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.757+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 2 ==== osd_op_reply(8 [pgnls start_epoch 112 out=73b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.757+0000 7fe05b777640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x559181570400 0x559180a0f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.757+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:9 16.2 16:40000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5591813cd000 con 0x559181570400 2024-04-25T10:16:48.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.757+0000 7fe05b777640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x559181570400 0x559180a0f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.757+0000 7fe05b777640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x559181570400 0x559180a0f080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.4 v2:172.21.15.100:6808/3965541654 1 ==== osd_op_reply(9 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702500 con 0x559181570400 2024-04-25T10:16:48.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05b777640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x5591813cd000 0x559180a0f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:10 16.12 16:48000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5591813cd400 con 0x5591813cd000 2024-04-25T10:16:48.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05af76640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x5591813cd000 0x559180a0f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05af76640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x5591813cd000 0x559180a0f600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.6 v2:172.21.15.100:6824/4086390550 1 ==== osd_op_reply(10 [pgnls start_epoch 112 out=81b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x559180702000 con 0x5591813cd000 2024-04-25T10:16:48.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:11 16.a 16:50000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5591813cd400 con 0x559181503c00 2024-04-25T10:16:48.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 3 ==== osd_op_reply(11 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:12 16.1a 16:58000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5591813cd400 con 0x559181570800 2024-04-25T10:16:48.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.3 v2:172.21.15.60:6814/1867957156 2 ==== osd_op_reply(12 [pgnls start_epoch 112 out=49b] v112'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702000 con 0x559181570800 2024-04-25T10:16:48.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:13 16.6 16:60000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5591813cd400 con 0x5591813cd000 2024-04-25T10:16:48.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.6 v2:172.21.15.100:6824/4086390550 2 ==== osd_op_reply(13 [pgnls start_epoch 112 out=77b] v111'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x559180702000 con 0x5591813cd000 2024-04-25T10:16:48.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:14 16.16 16:68000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5591813cd400 con 0x559181570800 2024-04-25T10:16:48.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.3 v2:172.21.15.60:6814/1867957156 3 ==== osd_op_reply(14 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702000 con 0x559181570800 2024-04-25T10:16:48.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:15 16.e 16:70000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5591813cd400 con 0x559181503c00 2024-04-25T10:16:48.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 4 ==== osd_op_reply(15 [pgnls start_epoch 112 out=73b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05b777640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5591813cd400 0x559180a0fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:16 16.1e 16:78000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a30000 con 0x5591813cd400 2024-04-25T10:16:48.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.761+0000 7fe05a775640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5591813cd400 0x559180a0fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05a775640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5591813cd400 0x559180a0fb80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 <== osd.7 v2:172.21.15.100:6816/2771485684 1 ==== osd_op_reply(16 [pgnls start_epoch 112 out=49b] v112'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702780 con 0x5591813cd400 2024-04-25T10:16:48.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:17 16.1 16:80000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a30000 con 0x559181503c00 2024-04-25T10:16:48.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 5 ==== osd_op_reply(17 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:18 16.11 16:88000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a30000 con 0x559181570000 2024-04-25T10:16:48.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 <== osd.5 v2:172.21.15.100:6800/3542120824 2 ==== osd_op_reply(18 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702780 con 0x559181570000 2024-04-25T10:16:48.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05a775640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x559180a30000 0x559180a10100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:19 16.9 16:90000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181571000 con 0x559180a30000 2024-04-25T10:16:48.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05b777640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x559180a30000 0x559180a10100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05b777640 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x559180a30000 0x559180a10100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.1 v2:172.21.15.60:6824/3313552458 1 ==== osd_op_reply(19 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702500 con 0x559180a30000 2024-04-25T10:16:48.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:20 16.19 16:98000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181571000 con 0x559181570400 2024-04-25T10:16:48.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.4 v2:172.21.15.100:6808/3965541654 2 ==== osd_op_reply(20 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702500 con 0x559181570400 2024-04-25T10:16:48.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:21 16.5 16:a0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181571000 con 0x559181503800 2024-04-25T10:16:48.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 5 ==== osd_op_reply(21 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:22 16.15 16:a8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181571000 con 0x559181503800 2024-04-25T10:16:48.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 6 ==== osd_op_reply(22 [pgnls start_epoch 112 out=97b] v111'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.765+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:23 16.d 16:b0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181571000 con 0x5591813cd000 2024-04-25T10:16:48.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.6 v2:172.21.15.100:6824/4086390550 3 ==== osd_op_reply(23 [pgnls start_epoch 112 out=74b] v109'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x559180702000 con 0x5591813cd000 2024-04-25T10:16:48.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:24 16.1d 16:b8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181571000 con 0x559181503c00 2024-04-25T10:16:48.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 6 ==== osd_op_reply(24 [pgnls start_epoch 112 out=73b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:25 16.3 16:c0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181571000 con 0x559181570000 2024-04-25T10:16:48.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 <== osd.5 v2:172.21.15.100:6800/3542120824 3 ==== osd_op_reply(25 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702780 con 0x559181570000 2024-04-25T10:16:48.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:26 16.13 16:c8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181571000 con 0x559180a30000 2024-04-25T10:16:48.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.1 v2:172.21.15.60:6824/3313552458 2 ==== osd_op_reply(26 [pgnls start_epoch 112 out=98b] v111'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x559180702500 con 0x559180a30000 2024-04-25T10:16:48.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:27 16.b 16:d0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181571000 con 0x559180a30000 2024-04-25T10:16:48.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.1 v2:172.21.15.60:6824/3313552458 3 ==== osd_op_reply(27 [pgnls start_epoch 112 out=76b] v112'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x559180702500 con 0x559180a30000 2024-04-25T10:16:48.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:28 16.1b 16:d8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181571000 con 0x559181570000 2024-04-25T10:16:48.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 <== osd.5 v2:172.21.15.100:6800/3542120824 4 ==== osd_op_reply(28 [pgnls start_epoch 112 out=49b] v112'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702780 con 0x559181570000 2024-04-25T10:16:48.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:29 16.7 16:e0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181571000 con 0x559181570800 2024-04-25T10:16:48.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.3 v2:172.21.15.60:6814/1867957156 4 ==== osd_op_reply(29 [pgnls start_epoch 112 out=105b] v112'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x559180702000 con 0x559181570800 2024-04-25T10:16:48.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:30 16.17 16:e8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181571000 con 0x559181503c00 2024-04-25T10:16:48.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 7 ==== osd_op_reply(30 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:31 16.f 16:f0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181571000 con 0x559181503800 2024-04-25T10:16:48.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 7 ==== osd_op_reply(31 [pgnls start_epoch 112 out=73b] v112'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.769+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:32 16.1f 16:f8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559181571000 con 0x559181570000 2024-04-25T10:16:48.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 <== osd.5 v2:172.21.15.100:6800/3542120824 5 ==== osd_op_reply(32 [pgnls start_epoch 112 out=101b] v112'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x559180702780 con 0x559181570000 2024-04-25T10:16:48.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-25T10:16:48.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:33 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a31000 con 0x559181503c00 2024-04-25T10:16:48.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-25T10:16:48.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:34 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a31400 con 0x559181503c00 2024-04-25T10:16:48.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T10:16:48.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:35 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a31800 con 0x559181503c00 2024-04-25T10:16:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-25T10:16:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:36 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a31c00 con 0x559181503c00 2024-04-25T10:16:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-25T10:16:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:37 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a30c00 con 0x559181503c00 2024-04-25T10:16:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T10:16:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:16:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-25T10:16:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:16:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-25T10:16:48.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:16:48.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-25T10:16:48.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:38 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a30800 con 0x559181503800 2024-04-25T10:16:48.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 8 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-25T10:16:48.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:39 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a30400 con 0x559181503c00 2024-04-25T10:16:48.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-04-25T10:16:48.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:40 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a31000 con 0x5591813cd000 2024-04-25T10:16:48.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.6 v2:172.21.15.100:6824/4086390550 4 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+523 (crc 0 0 0) 0x559180702000 con 0x5591813cd000 2024-04-25T10:16:48.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-25T10:16:48.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:16:48.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-25T10:16:48.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:41 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a31400 con 0x559181503c00 2024-04-25T10:16:48.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-25T10:16:48.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.773+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:42 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a31800 con 0x559181503800 2024-04-25T10:16:48.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-25T10:16:48.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:16:48.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T10:16:48.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:16:48.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-25T10:16:48.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:43 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181503c00 2024-04-25T10:16:48.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T10:16:48.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:16:48.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T10:16:48.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:44 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40400 con 0x559180a30000 2024-04-25T10:16:48.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.1 v2:172.21.15.60:6824/3313552458 4 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x559180702500 con 0x559180a30000 2024-04-25T10:16:48.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T10:16:48.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:16:48.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-25T10:16:48.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:45 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40800 con 0x559181570800 2024-04-25T10:16:48.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.3 v2:172.21.15.60:6814/1867957156 5 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702000 con 0x559181570800 2024-04-25T10:16:48.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-04-25T10:16:48.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:46 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40c00 con 0x559181570800 2024-04-25T10:16:48.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.3 v2:172.21.15.60:6814/1867957156 6 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1545 (crc 0 0 0) 0x559180702000 con 0x559181570800 2024-04-25T10:16:48.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 20 datascan.scan_links: injected ino detected 2024-04-25T10:16:48.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-25T10:16:48.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:16:48.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T10:16:48.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:16:48.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T10:16:48.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:47 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a41000 con 0x559181570000 2024-04-25T10:16:48.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 <== osd.5 v2:172.21.15.100:6800/3542120824 6 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x559180702780 con 0x559181570000 2024-04-25T10:16:48.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:48 16.0 16:00000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a41400 con 0x559181503800 2024-04-25T10:16:48.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 10 ==== osd_op_reply(48 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:49 16.10 16:08000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40800 con 0x559181503c00 2024-04-25T10:16:48.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 16 ==== osd_op_reply(49 [pgnls start_epoch 112 out=177b] v111'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:50 16.8 16:10000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181503800 2024-04-25T10:16:48.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 11 ==== osd_op_reply(50 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:51 16.18 16:18000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181503800 2024-04-25T10:16:48.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 12 ==== osd_op_reply(51 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:52 16.4 16:20000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181570800 2024-04-25T10:16:48.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.3 v2:172.21.15.60:6814/1867957156 7 ==== osd_op_reply(52 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702000 con 0x559181570800 2024-04-25T10:16:48.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.777+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:53 16.14 16:28000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181503800 2024-04-25T10:16:48.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.781+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 13 ==== osd_op_reply(53 [pgnls start_epoch 112 out=149b] v112'41 uv41 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.781+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:54 16.c 16:30000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181570000 2024-04-25T10:16:48.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.781+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 <== osd.5 v2:172.21.15.100:6800/3542120824 7 ==== osd_op_reply(54 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702780 con 0x559181570000 2024-04-25T10:16:48.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.781+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:55 16.1c 16:38000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181503c00 2024-04-25T10:16:48.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.781+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 17 ==== osd_op_reply(55 [pgnls start_epoch 112 out=73b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.781+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:56 16.2 16:40000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181570400 2024-04-25T10:16:48.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.781+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.4 v2:172.21.15.100:6808/3965541654 3 ==== osd_op_reply(56 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702500 con 0x559181570400 2024-04-25T10:16:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.781+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:57 16.12 16:48000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x5591813cd000 2024-04-25T10:16:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.781+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.6 v2:172.21.15.100:6824/4086390550 5 ==== osd_op_reply(57 [pgnls start_epoch 112 out=81b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x559180702000 con 0x5591813cd000 2024-04-25T10:16:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.781+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:58 16.a 16:50000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181503c00 2024-04-25T10:16:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.781+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 18 ==== osd_op_reply(58 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.781+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:59 16.1a 16:58000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181570800 2024-04-25T10:16:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.781+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.3 v2:172.21.15.60:6814/1867957156 8 ==== osd_op_reply(59 [pgnls start_epoch 112 out=49b] v112'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702000 con 0x559181570800 2024-04-25T10:16:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.781+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:60 16.6 16:60000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x5591813cd000 2024-04-25T10:16:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.6 v2:172.21.15.100:6824/4086390550 6 ==== osd_op_reply(60 [pgnls start_epoch 112 out=77b] v111'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x559180702000 con 0x5591813cd000 2024-04-25T10:16:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:61 16.16 16:68000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181570800 2024-04-25T10:16:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.3 v2:172.21.15.60:6814/1867957156 9 ==== osd_op_reply(61 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702000 con 0x559181570800 2024-04-25T10:16:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:62 16.e 16:70000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181503c00 2024-04-25T10:16:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 19 ==== osd_op_reply(62 [pgnls start_epoch 112 out=73b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- osd_op(unknown.0.0:63 16.1e 16:78000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x5591813cd400 2024-04-25T10:16:48.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 <== osd.7 v2:172.21.15.100:6816/2771485684 2 ==== osd_op_reply(63 [pgnls start_epoch 112 out=49b] v112'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702780 con 0x5591813cd400 2024-04-25T10:16:48.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:64 16.1 16:80000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181503c00 2024-04-25T10:16:48.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 20 ==== osd_op_reply(64 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:65 16.11 16:88000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181570000 2024-04-25T10:16:48.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 <== osd.5 v2:172.21.15.100:6800/3542120824 8 ==== osd_op_reply(65 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702780 con 0x559181570000 2024-04-25T10:16:48.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:66 16.9 16:90000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559180a30000 2024-04-25T10:16:48.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.1 v2:172.21.15.60:6824/3313552458 5 ==== osd_op_reply(66 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702500 con 0x559180a30000 2024-04-25T10:16:48.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- osd_op(unknown.0.0:67 16.19 16:98000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181570400 2024-04-25T10:16:48.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.4 v2:172.21.15.100:6808/3965541654 4 ==== osd_op_reply(67 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702500 con 0x559181570400 2024-04-25T10:16:48.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:68 16.5 16:a0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181503800 2024-04-25T10:16:48.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 14 ==== osd_op_reply(68 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:69 16.15 16:a8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181503800 2024-04-25T10:16:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 15 ==== osd_op_reply(69 [pgnls start_epoch 112 out=97b] v111'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.785+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:70 16.d 16:b0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x5591813cd000 2024-04-25T10:16:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.6 v2:172.21.15.100:6824/4086390550 7 ==== osd_op_reply(70 [pgnls start_epoch 112 out=74b] v109'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x559180702000 con 0x5591813cd000 2024-04-25T10:16:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:71 16.1d 16:b8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181503c00 2024-04-25T10:16:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 21 ==== osd_op_reply(71 [pgnls start_epoch 112 out=73b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:72 16.3 16:c0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181570000 2024-04-25T10:16:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 <== osd.5 v2:172.21.15.100:6800/3542120824 9 ==== osd_op_reply(72 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702780 con 0x559181570000 2024-04-25T10:16:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:73 16.13 16:c8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559180a30000 2024-04-25T10:16:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.1 v2:172.21.15.60:6824/3313552458 6 ==== osd_op_reply(73 [pgnls start_epoch 112 out=98b] v111'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x559180702500 con 0x559180a30000 2024-04-25T10:16:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:74 16.b 16:d0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559180a30000 2024-04-25T10:16:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.1 v2:172.21.15.60:6824/3313552458 7 ==== osd_op_reply(74 [pgnls start_epoch 112 out=76b] v112'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x559180702500 con 0x559180a30000 2024-04-25T10:16:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:75 16.1b 16:d8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181570000 2024-04-25T10:16:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 <== osd.5 v2:172.21.15.100:6800/3542120824 10 ==== osd_op_reply(75 [pgnls start_epoch 112 out=49b] v112'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702780 con 0x559181570000 2024-04-25T10:16:48.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:76 16.7 16:e0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181570800 2024-04-25T10:16:48.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.3 v2:172.21.15.60:6814/1867957156 10 ==== osd_op_reply(76 [pgnls start_epoch 112 out=105b] v112'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x559180702000 con 0x559181570800 2024-04-25T10:16:48.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:77 16.17 16:e8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181503c00 2024-04-25T10:16:48.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 22 ==== osd_op_reply(77 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:78 16.f 16:f0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181503800 2024-04-25T10:16:48.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 16 ==== osd_op_reply(78 [pgnls start_epoch 112 out=73b] v112'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.789+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:79 16.1f 16:f8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40000 con 0x559181570000 2024-04-25T10:16:48.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 <== osd.5 v2:172.21.15.100:6800/3542120824 11 ==== osd_op_reply(79 [pgnls start_epoch 112 out=101b] v112'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x559180702780 con 0x559181570000 2024-04-25T10:16:48.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-25T10:16:48.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:80 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a41800 con 0x559181503c00 2024-04-25T10:16:48.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 23 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-25T10:16:48.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:81 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a41c00 con 0x559181503c00 2024-04-25T10:16:48.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 24 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T10:16:48.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:82 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a42000 con 0x559181503c00 2024-04-25T10:16:48.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 25 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-25T10:16:48.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:83 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a42400 con 0x559181503c00 2024-04-25T10:16:48.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 26 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-25T10:16:48.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:84 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a42800 con 0x559181503c00 2024-04-25T10:16:48.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 27 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T10:16:48.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:16:48.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-25T10:16:48.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:16:48.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-25T10:16:48.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:16:48.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-25T10:16:48.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:85 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a42c00 con 0x559181503800 2024-04-25T10:16:48.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 17 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-25T10:16:48.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:86 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a43000 con 0x559181503c00 2024-04-25T10:16:48.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 28 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-04-25T10:16:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:87 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a43400 con 0x5591813cd000 2024-04-25T10:16:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.6 v2:172.21.15.100:6824/4086390550 8 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+523 (crc 0 0 0) 0x559180702000 con 0x5591813cd000 2024-04-25T10:16:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-25T10:16:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:16:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-25T10:16:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:88 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a43800 con 0x559181503c00 2024-04-25T10:16:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 29 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-25T10:16:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- osd_op(unknown.0.0:89 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a43c00 con 0x559181503800 2024-04-25T10:16:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.0 v2:172.21.15.60:6801/4056167511 18 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702000 con 0x559181503800 2024-04-25T10:16:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-25T10:16:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:16:48.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T10:16:48.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:16:48.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-25T10:16:48.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.793+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- osd_op(unknown.0.0:90 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a41400 con 0x559181503c00 2024-04-25T10:16:48.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.2 v2:172.21.15.60:6800/1771147646 30 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702500 con 0x559181503c00 2024-04-25T10:16:48.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T10:16:48.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:16:48.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T10:16:48.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:91 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40c00 con 0x559180a30000 2024-04-25T10:16:48.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.1 v2:172.21.15.60:6824/3313552458 8 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x559180702500 con 0x559180a30000 2024-04-25T10:16:48.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T10:16:48.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:16:48.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-25T10:16:48.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:92 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40800 con 0x559181570800 2024-04-25T10:16:48.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.3 v2:172.21.15.60:6814/1867957156 11 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x559180702000 con 0x559181570800 2024-04-25T10:16:48.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-04-25T10:16:48.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:93 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a40400 con 0x559181570800 2024-04-25T10:16:48.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.3 v2:172.21.15.60:6814/1867957156 12 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1545 (crc 0 0 0) 0x559180702000 con 0x559181570800 2024-04-25T10:16:48.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 20 datascan.scan_links: injected ino detected 2024-04-25T10:16:48.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-04-25T10:16:48.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-25T10:16:48.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:16:48.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T10:16:48.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:16:48.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T10:16:48.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- osd_op(unknown.0.0:94 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a31800 con 0x559181570000 2024-04-25T10:16:48.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05a775640 1 -- 172.21.15.60:0/2530668225 <== osd.5 v2:172.21.15.100:6800/3542120824 12 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x559180702780 con 0x559181570000 2024-04-25T10:16:48.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-04-25T10:16:48.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T10:16:48.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-04-25T10:16:48.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: processing 1 injected_inos 2024-04-25T10:16:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-04-25T10:16:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-04-25T10:16:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:95 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a31000 con 0x559181570800 2024-04-25T10:16:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.3 v2:172.21.15.60:6814/1867957156 13 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x559180702000 con 0x559181570800 2024-04-25T10:16:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-25T10:16:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 20 datascan.scan_links: first is now 2 2024-04-25T10:16:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- osd_op(unknown.0.0:96 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a30800 con 0x559181570800 2024-04-25T10:16:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.3 v2:172.21.15.60:6814/1867957156 14 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v112'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x559180702000 con 0x559181570800 2024-04-25T10:16:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-25T10:16:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 10 datascan.scan_links: updating inotable 2024-04-25T10:16:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.797+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:97 16.13 16:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a31400 con 0x559180a30000 2024-04-25T10:16:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.1 v2:172.21.15.60:6824/3313552458 9 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x559180702500 con 0x559180a30000 2024-04-25T10:16:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-04-25T10:16:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-04-25T10:16:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- osd_op(unknown.0.0:98 16.13 16:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a30400 con 0x559180a30000 2024-04-25T10:16:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05b777640 1 -- 172.21.15.60:0/2530668225 <== osd.1 v2:172.21.15.60:6824/3313552458 10 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v112'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x559180702500 con 0x559180a30000 2024-04-25T10:16:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 10 datascan.scan_links: updating snaptable 2024-04-25T10:16:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- osd_op(unknown.0.0:99 16.d 16:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x559180a30c00 con 0x5591813cd000 2024-04-25T10:16:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 <== osd.6 v2:172.21.15.100:6824/4086390550 9 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x559180702000 con 0x5591813cd000 2024-04-25T10:16:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x559181503800 msgr2=0x5591807ae100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x559181503800 0x5591807ae100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x559180a30000 msgr2=0x559180a10100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x559180a30000 0x559180a10100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x559181503c00 msgr2=0x559180a0e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x559181503c00 0x559180a0e000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x559181570800 msgr2=0x559180a0e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x559181570800 0x559180a0e580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 reap_dead start 2024-04-25T10:16:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x559181570400 msgr2=0x559180a0f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x559181570400 0x559180a0f080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 reap_dead start 2024-04-25T10:16:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x559181570000 msgr2=0x559180a0eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x559181570000 0x559180a0eb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x5591813cd000 msgr2=0x559180a0f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x5591813cd000 0x559180a0f600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5591813cd400 msgr2=0x559180a0fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x5591813cd400 0x559180a0fb80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x559181571c00 msgr2=0x5591807af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x559181571c00 0x5591807af700 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x5591806ce1e0 tx=0x5591806f0000 comp rx=0 tx=0).stop 2024-04-25T10:16:48.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559181503400 msgr2=0x5591807ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2530668225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559181503400 0x5591807ae680 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x5591815158c0 tx=0x5591806accc0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 reap_dead start 2024-04-25T10:16:48.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05af76640 1 -- 172.21.15.60:0/2530668225 reap_dead start 2024-04-25T10:16:48.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 shutdown_connections 2024-04-25T10:16:48.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.801+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 >> 172.21.15.60:0/2530668225 conn(0x5591805f2c00 msgr2=0x55918061f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.805+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 shutdown_connections 2024-04-25T10:16:48.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.805+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2530668225 wait complete. 2024-04-25T10:16:48.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.805+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2607230886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591805f3400 msgr2=0x5591807acb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.805+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2607230886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591805f3400 0x5591807acb00 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x55918143d8f0 tx=0x5591806ad020 comp rx=0 tx=0).stop 2024-04-25T10:16:48.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.805+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2607230886 shutdown_connections 2024-04-25T10:16:48.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.805+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2607230886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591805f3000 0x5591807ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.805+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2607230886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591805f3400 0x5591807acb00 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.805+0000 7fe05c7b78c0 1 --2- 172.21.15.60:0/2607230886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5591805f3800 0x5591807ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.805+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2607230886 >> 172.21.15.60:0/2607230886 conn(0x5591805f3c00 msgr2=0x55918061f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.805+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2607230886 shutdown_connections 2024-04-25T10:16:48.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:48.805+0000 7fe05c7b78c0 1 -- 172.21.15.60:0/2607230886 wait complete. 2024-04-25T10:16:48.821 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.148141, result: 2024-04-25T10:16:48.822 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:16:48.858 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:16:48.884 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2024-04-25T10:16:48.884 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2024-04-25T10:16:48.885 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:16:48.885 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2024-04-25T10:16:48.885 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.885 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2024-04-25T10:16:48.885 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.885 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2024-04-25T10:16:48.885 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.885 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2024-04-25T10:16:48.885 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2024-04-25T10:16:48.885 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.885 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2024-04-25T10:16:48.885 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2024-04-25T10:16:48.885 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.886 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:16:48.886 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2024-04-25T10:16:48.886 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.886 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:16:48.886 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:16:48.886 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.886 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2024-04-25T10:16:48.886 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.886 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2024-04-25T10:16:48.886 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2024-04-25T10:16:48.886 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:16:48.886 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:16:48.887 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2024-04-25T10:16:48.887 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:16:48.887 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2024-04-25T10:16:48.887 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:16:48.887 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:16:48.887 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2024-04-25T10:16:48.887 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.887 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:16:48.887 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.887 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2024-04-25T10:16:48.887 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.887 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2024-04-25T10:16:48.887 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.887 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2024-04-25T10:16:48.888 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2024-04-25T10:16:48.888 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2024-04-25T10:16:48.888 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = "172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2024-04-25T10:16:48.888 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = 0ea93863-a079-45c4-abed-6e83540ea9fe 2024-04-25T10:16:48.888 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.888 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2024-04-25T10:16:48.888 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2024-04-25T10:16:48.888 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.888 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2024-04-25T10:16:48.888 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2024-04-25T10:16:48.888 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:16:48.888 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.888 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2024-04-25T10:16:48.888 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2024-04-25T10:16:48.889 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.889 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2024-04-25T10:16:48.889 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2024-04-25T10:16:48.889 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2024-04-25T10:16:48.889 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug trim objects = true 2024-04-25T10:16:48.889 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.889 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2024-04-25T10:16:48.889 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2024-04-25T10:16:48.889 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.889 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:16:48.889 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.889 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2024-04-25T10:16:48.889 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.889 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2024-04-25T10:16:48.890 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.890 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:16:48.890 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2024-04-25T10:16:48.890 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.890 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2024-04-25T10:16:48.890 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2024-04-25T10:16:48.890 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2024-04-25T10:16:48.890 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2024-04-25T10:16:48.890 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2024-04-25T10:16:48.890 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2024-04-25T10:16:48.890 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2024-04-25T10:16:48.890 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2024-04-25T10:16:48.890 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2024-04-25T10:16:48.891 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:48.891 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2024-04-25T10:16:48.891 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2024-04-25T10:16:48.891 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:16:48.891 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2024-04-25T10:16:48.891 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:16:48.891 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:16:48.891 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2024-04-25T10:16:48.891 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2024-04-25T10:16:48.891 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.891 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2024-04-25T10:16:48.891 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:48.891 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2024-04-25T10:16:48.891 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-04-25T10:16:48.891 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-04-25T10:16:48.891 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:48.892 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:48.892 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2024-04-25T10:16:48.892 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.892 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2024-04-25T10:16:48.892 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:48.892 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-04-25T10:16:48.892 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2024-04-25T10:16:48.892 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-04-25T10:16:48.892 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2024-04-25T10:16:48.892 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2024-04-25T10:16:48.892 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:48.892 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2024-04-25T10:16:48.892 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2024-04-25T10:16:48.892 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:48.893 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.893 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:16:48.893 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:16:48.893 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:16:48.893 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.893 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:16:48.893 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:16:48.893 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:16:48.893 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.893 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2024-04-25T10:16:48.893 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:16:48.893 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.893 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:16:48.894 INFO:teuthology.orchestra.run.smithi060.stdout: mon op complaint time = 120 2024-04-25T10:16:48.894 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.894 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2024-04-25T10:16:48.894 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2024-04-25T10:16:48.894 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2024-04-25T10:16:48.894 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2024-04-25T10:16:48.894 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:16:48.894 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:16:48.894 INFO:teuthology.orchestra.run.smithi060.stdout: client mount timeout = 600 2024-04-25T10:16:48.894 INFO:teuthology.orchestra.run.smithi060.stdout: debug client = 20 2024-04-25T10:16:48.894 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:48.894 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = "" 2024-04-25T10:16:48.895 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2024-04-25T10:16:48.895 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2024-04-25T10:16:48.895 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2024-04-25T10:16:48.895 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.b] 2024-04-25T10:16:48.895 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.c] 2024-04-25T10:16:48.895 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2024-04-25T10:16:48.895 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2024-04-25T10:16:48.895 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2024-04-25T10:16:48.895 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2024-04-25T10:16:48.895 INFO:teuthology.orchestra.run.smithi060.stdout:[mds] 2024-04-25T10:16:48.895 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds = 20 2024-04-25T10:16:48.895 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds balancer = 20 2024-04-25T10:16:48.895 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:48.896 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug frag = True 2024-04-25T10:16:48.896 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug scatterstat = False 2024-04-25T10:16:48.896 INFO:teuthology.orchestra.run.smithi060.stdout: mds op complaint time = 180 2024-04-25T10:16:48.896 INFO:teuthology.orchestra.run.smithi060.stdout: mds verify scatter = True 2024-04-25T10:16:48.896 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2024-04-25T10:16:48.896 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2024-04-25T10:16:48.896 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2024-04-25T10:16:48.897 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-04-25T10:16:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: chdir = "" 2024-04-25T10:16:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:16:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: auth supported = cephx 2024-04-25T10:16:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: filestore xattr use omap = true 2024-04-25T10:16:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: mon clock drift allowed = 1.000 2024-04-25T10:16:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: osd crush chooseleaf type = 0 2024-04-25T10:16:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: auth debug = true 2024-04-25T10:16:48.897 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on old message = true 2024-04-25T10:16:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on bug = true 2024-04-25T10:16:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:16:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: mon pg warn max object skew = 0 2024-04-25T10:16:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:16:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:16:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default size = 2 2024-04-25T10:16:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow primary affinity = true 2024-04-25T10:16:48.898 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow pg remap = true 2024-04-25T10:16:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:16:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:16:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on no sortbitwise = false 2024-04-25T10:16:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:16:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on too few osds = false 2024-04-25T10:16:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:16:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:16:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: mon_allow_pool_size_one = true 2024-04-25T10:16:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:16:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: osd default data pool replay window = 5 2024-04-25T10:16:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.899 INFO:teuthology.orchestra.run.smithi100.stdout: mon allow pool delete = true 2024-04-25T10:16:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: mon cluster log file level = debug 2024-04-25T10:16:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: debug asserts on shutdown = true 2024-04-25T10:16:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: mon health detail to clog = false 2024-04-25T10:16:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: mon host = "172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2024-04-25T10:16:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = 0ea93863-a079-45c4-abed-6e83540ea9fe 2024-04-25T10:16:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.900 INFO:teuthology.orchestra.run.smithi100.stdout:[osd] 2024-04-25T10:16:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: osd journal size = 100 2024-04-25T10:16:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub load threshold = 5.0 2024-04-25T10:16:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub max interval = 600 2024-04-25T10:16:48.900 INFO:teuthology.orchestra.run.smithi100.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: osd recover clone overlap = true 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: osd recovery max chunk = 1048576 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug shutdown = true 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug op order = true 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify stray on activate = true 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug trim objects = true 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: osd open classes on start = true 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug pg log writeout = true 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi100.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: osd map max advance = 10 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: journal zero on create = true 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: filestore apply finisher threads = 3 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: bdev debug aio = true 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug misdirected ops = true 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: bdev async discard = True 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: bdev enable discard = True 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore allocator = bitmap 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore block size = 96636764160 2024-04-25T10:16:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore fsck on mount = True 2024-04-25T10:16:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluefs = 1/20 2024-04-25T10:16:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluestore = 1/20 2024-04-25T10:16:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: debug osd = 20 2024-04-25T10:16:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: debug rocksdb = 4/10 2024-04-25T10:16:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:16:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd full ratio = 0.9 2024-04-25T10:16:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:16:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:16:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: osd objectstore = bluestore 2024-04-25T10:16:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-04-25T10:16:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.904 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr] 2024-04-25T10:16:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: debug mgr = 20 2024-04-25T10:16:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-04-25T10:16:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-04-25T10:16:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: mgr/telemetry/nag = false 2024-04-25T10:16:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.904 INFO:teuthology.orchestra.run.smithi100.stdout:[mon] 2024-04-25T10:16:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-04-25T10:16:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: debug paxos = 20 2024-04-25T10:16:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-04-25T10:16:48.904 INFO:teuthology.orchestra.run.smithi100.stdout: mon data avail warn = 5 2024-04-25T10:16:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: mon mgr mkfs grace = 240 2024-04-25T10:16:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd reporter subtree level = osd 2024-04-25T10:16:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd prime pg temp = true 2024-04-25T10:16:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:16:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:16:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:16:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:16:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:16:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:16:48.905 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: # 1m isn't quite enough 2024-04-25T10:16:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:16:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:16:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: mon op complaint time = 120 2024-04-25T10:16:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.906 INFO:teuthology.orchestra.run.smithi100.stdout:[client] 2024-04-25T10:16:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: rgw cache enabled = true 2024-04-25T10:16:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable ops log = true 2024-04-25T10:16:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable usage log = true 2024-04-25T10:16:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:16:48.906 INFO:teuthology.orchestra.run.smithi100.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:16:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: client mount timeout = 600 2024-04-25T10:16:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2024-04-25T10:16:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = "" 2024-04-25T10:16:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-04-25T10:16:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-04-25T10:16:48.907 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.a] 2024-04-25T10:16:48.907 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.b] 2024-04-25T10:16:48.907 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.c] 2024-04-25T10:16:48.907 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-04-25T10:16:48.907 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-04-25T10:16:48.907 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-04-25T10:16:48.907 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-04-25T10:16:48.907 INFO:teuthology.orchestra.run.smithi100.stdout:[mds] 2024-04-25T10:16:48.907 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds = 20 2024-04-25T10:16:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds balancer = 20 2024-04-25T10:16:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug frag = True 2024-04-25T10:16:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug scatterstat = False 2024-04-25T10:16:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: mds op complaint time = 180 2024-04-25T10:16:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: mds verify scatter = True 2024-04-25T10:16:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-04-25T10:16:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-04-25T10:16:48.908 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-04-25T10:16:48.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:16:48.955 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:16:48.979 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2024-04-25T10:16:48.980 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2024-04-25T10:16:48.980 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:16:48.980 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2024-04-25T10:16:48.980 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.980 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2024-04-25T10:16:48.980 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.980 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2024-04-25T10:16:48.980 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.980 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2024-04-25T10:16:48.980 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2024-04-25T10:16:48.980 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.980 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2024-04-25T10:16:48.980 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2024-04-25T10:16:48.980 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.980 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:16:48.981 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2024-04-25T10:16:48.981 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.981 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:16:48.981 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:16:48.981 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.981 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2024-04-25T10:16:48.981 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.981 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2024-04-25T10:16:48.981 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2024-04-25T10:16:48.981 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:16:48.981 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:16:48.981 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2024-04-25T10:16:48.981 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:16:48.981 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2024-04-25T10:16:48.981 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:16:48.982 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:16:48.982 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2024-04-25T10:16:48.982 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.982 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:16:48.982 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.982 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2024-04-25T10:16:48.982 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.982 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2024-04-25T10:16:48.982 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.982 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2024-04-25T10:16:48.982 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2024-04-25T10:16:48.982 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2024-04-25T10:16:48.983 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = "172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2024-04-25T10:16:48.983 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = 0ea93863-a079-45c4-abed-6e83540ea9fe 2024-04-25T10:16:48.983 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.983 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2024-04-25T10:16:48.983 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2024-04-25T10:16:48.983 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.983 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2024-04-25T10:16:48.983 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2024-04-25T10:16:48.983 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:16:48.983 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.983 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2024-04-25T10:16:48.983 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2024-04-25T10:16:48.984 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.984 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2024-04-25T10:16:48.984 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2024-04-25T10:16:48.984 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2024-04-25T10:16:48.984 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug trim objects = true 2024-04-25T10:16:48.984 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.984 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2024-04-25T10:16:48.984 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2024-04-25T10:16:48.984 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.984 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:16:48.984 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.984 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2024-04-25T10:16:48.984 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.984 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2024-04-25T10:16:48.984 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.985 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:16:48.985 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2024-04-25T10:16:48.985 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.985 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2024-04-25T10:16:48.985 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2024-04-25T10:16:48.985 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2024-04-25T10:16:48.985 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2024-04-25T10:16:48.985 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2024-04-25T10:16:48.985 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2024-04-25T10:16:48.985 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2024-04-25T10:16:48.985 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2024-04-25T10:16:48.985 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2024-04-25T10:16:48.985 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:48.985 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2024-04-25T10:16:48.985 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2024-04-25T10:16:48.986 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:16:48.986 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2024-04-25T10:16:48.986 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:16:48.986 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:16:48.986 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2024-04-25T10:16:48.986 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2024-04-25T10:16:48.986 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.986 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2024-04-25T10:16:48.986 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:48.986 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2024-04-25T10:16:48.986 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-04-25T10:16:48.986 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-04-25T10:16:48.986 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:48.987 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:48.987 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2024-04-25T10:16:48.987 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2024-04-25T10:16:48.987 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:48.987 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-04-25T10:16:48.987 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2024-04-25T10:16:48.987 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-04-25T10:16:48.987 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2024-04-25T10:16:48.987 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2024-04-25T10:16:48.987 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:48.987 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2024-04-25T10:16:48.987 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2024-04-25T10:16:48.987 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:48.987 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.988 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:16:48.988 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:16:48.988 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:16:48.988 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.988 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:16:48.988 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:16:48.988 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:16:48.988 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.988 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2024-04-25T10:16:48.988 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:16:48.988 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.988 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:16:48.988 INFO:teuthology.orchestra.run.smithi060.stdout: mon op complaint time = 120 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout: client mount timeout = 600 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout: debug client = 20 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = "" 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.b] 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.c] 2024-04-25T10:16:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2024-04-25T10:16:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2024-04-25T10:16:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2024-04-25T10:16:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2024-04-25T10:16:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:[mds] 2024-04-25T10:16:48.990 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds = 20 2024-04-25T10:16:48.990 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds balancer = 20 2024-04-25T10:16:48.990 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T10:16:48.990 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug frag = True 2024-04-25T10:16:48.990 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug scatterstat = True 2024-04-25T10:16:48.990 INFO:teuthology.orchestra.run.smithi060.stdout: mds op complaint time = 180 2024-04-25T10:16:48.990 INFO:teuthology.orchestra.run.smithi060.stdout: mds verify scatter = True 2024-04-25T10:16:48.990 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2024-04-25T10:16:48.990 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2024-04-25T10:16:48.990 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2024-04-25T10:16:48.991 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-04-25T10:16:48.991 INFO:teuthology.orchestra.run.smithi100.stdout: chdir = "" 2024-04-25T10:16:48.991 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:16:48.991 INFO:teuthology.orchestra.run.smithi100.stdout: auth supported = cephx 2024-04-25T10:16:48.991 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.991 INFO:teuthology.orchestra.run.smithi100.stdout: filestore xattr use omap = true 2024-04-25T10:16:48.992 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.992 INFO:teuthology.orchestra.run.smithi100.stdout: mon clock drift allowed = 1.000 2024-04-25T10:16:48.992 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.992 INFO:teuthology.orchestra.run.smithi100.stdout: osd crush chooseleaf type = 0 2024-04-25T10:16:48.992 INFO:teuthology.orchestra.run.smithi100.stdout: auth debug = true 2024-04-25T10:16:48.992 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.992 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on old message = true 2024-04-25T10:16:48.992 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on bug = true 2024-04-25T10:16:48.992 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.992 INFO:teuthology.orchestra.run.smithi100.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:16:48.992 INFO:teuthology.orchestra.run.smithi100.stdout: mon pg warn max object skew = 0 2024-04-25T10:16:48.992 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.992 INFO:teuthology.orchestra.run.smithi100.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:16:48.992 INFO:teuthology.orchestra.run.smithi100.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:16:48.992 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.993 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default size = 2 2024-04-25T10:16:48.993 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.993 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow primary affinity = true 2024-04-25T10:16:48.993 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow pg remap = true 2024-04-25T10:16:48.993 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:16:48.993 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:16:48.993 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on no sortbitwise = false 2024-04-25T10:16:48.993 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:16:48.993 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on too few osds = false 2024-04-25T10:16:48.993 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:16:48.993 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:16:48.993 INFO:teuthology.orchestra.run.smithi100.stdout: mon_allow_pool_size_one = true 2024-04-25T10:16:48.993 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.993 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:16:48.994 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.994 INFO:teuthology.orchestra.run.smithi100.stdout: osd default data pool replay window = 5 2024-04-25T10:16:48.994 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.994 INFO:teuthology.orchestra.run.smithi100.stdout: mon allow pool delete = true 2024-04-25T10:16:48.994 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.994 INFO:teuthology.orchestra.run.smithi100.stdout: mon cluster log file level = debug 2024-04-25T10:16:48.994 INFO:teuthology.orchestra.run.smithi100.stdout: debug asserts on shutdown = true 2024-04-25T10:16:48.994 INFO:teuthology.orchestra.run.smithi100.stdout: mon health detail to clog = false 2024-04-25T10:16:48.994 INFO:teuthology.orchestra.run.smithi100.stdout: mon host = "172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2024-04-25T10:16:48.994 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = 0ea93863-a079-45c4-abed-6e83540ea9fe 2024-04-25T10:16:48.994 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.994 INFO:teuthology.orchestra.run.smithi100.stdout:[osd] 2024-04-25T10:16:48.994 INFO:teuthology.orchestra.run.smithi100.stdout: osd journal size = 100 2024-04-25T10:16:48.995 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.995 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub load threshold = 5.0 2024-04-25T10:16:48.995 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub max interval = 600 2024-04-25T10:16:48.995 INFO:teuthology.orchestra.run.smithi100.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:16:48.995 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.995 INFO:teuthology.orchestra.run.smithi100.stdout: osd recover clone overlap = true 2024-04-25T10:16:48.995 INFO:teuthology.orchestra.run.smithi100.stdout: osd recovery max chunk = 1048576 2024-04-25T10:16:48.995 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.995 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug shutdown = true 2024-04-25T10:16:48.995 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug op order = true 2024-04-25T10:16:48.995 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify stray on activate = true 2024-04-25T10:16:48.995 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug trim objects = true 2024-04-25T10:16:48.996 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.996 INFO:teuthology.orchestra.run.smithi100.stdout: osd open classes on start = true 2024-04-25T10:16:48.996 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug pg log writeout = true 2024-04-25T10:16:48.996 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.996 INFO:teuthology.orchestra.run.smithi100.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:16:48.996 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.996 INFO:teuthology.orchestra.run.smithi100.stdout: osd map max advance = 10 2024-04-25T10:16:48.996 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.996 INFO:teuthology.orchestra.run.smithi100.stdout: journal zero on create = true 2024-04-25T10:16:48.996 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.996 INFO:teuthology.orchestra.run.smithi100.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:16:48.996 INFO:teuthology.orchestra.run.smithi100.stdout: filestore apply finisher threads = 3 2024-04-25T10:16:48.996 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.996 INFO:teuthology.orchestra.run.smithi100.stdout: bdev debug aio = true 2024-04-25T10:16:48.997 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug misdirected ops = true 2024-04-25T10:16:48.997 INFO:teuthology.orchestra.run.smithi100.stdout: bdev async discard = True 2024-04-25T10:16:48.997 INFO:teuthology.orchestra.run.smithi100.stdout: bdev enable discard = True 2024-04-25T10:16:48.997 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore allocator = bitmap 2024-04-25T10:16:48.997 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore block size = 96636764160 2024-04-25T10:16:48.997 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore fsck on mount = True 2024-04-25T10:16:48.997 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluefs = 1/20 2024-04-25T10:16:48.997 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluestore = 1/20 2024-04-25T10:16:48.997 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:48.997 INFO:teuthology.orchestra.run.smithi100.stdout: debug osd = 20 2024-04-25T10:16:48.997 INFO:teuthology.orchestra.run.smithi100.stdout: debug rocksdb = 4/10 2024-04-25T10:16:48.997 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:16:48.997 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd full ratio = 0.9 2024-04-25T10:16:48.997 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:16:48.997 INFO:teuthology.orchestra.run.smithi100.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:16:48.997 INFO:teuthology.orchestra.run.smithi100.stdout: osd objectstore = bluestore 2024-04-25T10:16:48.998 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-04-25T10:16:48.998 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.998 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr] 2024-04-25T10:16:48.998 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:48.998 INFO:teuthology.orchestra.run.smithi100.stdout: debug mgr = 20 2024-04-25T10:16:48.998 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-04-25T10:16:48.998 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-04-25T10:16:48.998 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:48.998 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:48.998 INFO:teuthology.orchestra.run.smithi100.stdout: mgr/telemetry/nag = false 2024-04-25T10:16:48.998 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.998 INFO:teuthology.orchestra.run.smithi100.stdout:[mon] 2024-04-25T10:16:48.999 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:48.999 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-04-25T10:16:48.999 INFO:teuthology.orchestra.run.smithi100.stdout: debug paxos = 20 2024-04-25T10:16:48.999 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-04-25T10:16:48.999 INFO:teuthology.orchestra.run.smithi100.stdout: mon data avail warn = 5 2024-04-25T10:16:48.999 INFO:teuthology.orchestra.run.smithi100.stdout: mon mgr mkfs grace = 240 2024-04-25T10:16:48.999 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:48.999 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd reporter subtree level = osd 2024-04-25T10:16:48.999 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd prime pg temp = true 2024-04-25T10:16:48.999 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:48.999 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:48.999 INFO:teuthology.orchestra.run.smithi100.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:16:48.999 INFO:teuthology.orchestra.run.smithi100.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:16:48.999 INFO:teuthology.orchestra.run.smithi100.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:16:49.000 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:49.000 INFO:teuthology.orchestra.run.smithi100.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:16:49.000 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:16:49.000 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:16:49.000 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:49.000 INFO:teuthology.orchestra.run.smithi100.stdout: # 1m isn't quite enough 2024-04-25T10:16:49.000 INFO:teuthology.orchestra.run.smithi100.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:16:49.000 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:49.000 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:16:49.000 INFO:teuthology.orchestra.run.smithi100.stdout: mon op complaint time = 120 2024-04-25T10:16:49.000 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T10:16:49.000 INFO:teuthology.orchestra.run.smithi100.stdout:[client] 2024-04-25T10:16:49.000 INFO:teuthology.orchestra.run.smithi100.stdout: rgw cache enabled = true 2024-04-25T10:16:49.000 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable ops log = true 2024-04-25T10:16:49.000 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable usage log = true 2024-04-25T10:16:49.001 INFO:teuthology.orchestra.run.smithi100.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:16:49.001 INFO:teuthology.orchestra.run.smithi100.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:16:49.001 INFO:teuthology.orchestra.run.smithi100.stdout: client mount timeout = 600 2024-04-25T10:16:49.001 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2024-04-25T10:16:49.001 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:49.001 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = "" 2024-04-25T10:16:49.001 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-04-25T10:16:49.001 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-04-25T10:16:49.001 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.a] 2024-04-25T10:16:49.001 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.b] 2024-04-25T10:16:49.001 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.c] 2024-04-25T10:16:49.001 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-04-25T10:16:49.001 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-04-25T10:16:49.001 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-04-25T10:16:49.001 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-04-25T10:16:49.001 INFO:teuthology.orchestra.run.smithi100.stdout:[mds] 2024-04-25T10:16:49.002 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds = 20 2024-04-25T10:16:49.002 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds balancer = 20 2024-04-25T10:16:49.002 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T10:16:49.002 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug frag = True 2024-04-25T10:16:49.002 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug scatterstat = True 2024-04-25T10:16:49.002 INFO:teuthology.orchestra.run.smithi100.stdout: mds op complaint time = 180 2024-04-25T10:16:49.002 INFO:teuthology.orchestra.run.smithi100.stdout: mds verify scatter = True 2024-04-25T10:16:49.002 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-04-25T10:16:49.002 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-04-25T10:16:49.002 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-04-25T10:16:49.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:16:49.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.173+0000 7f7f17280640 1 Processor -- start 2024-04-25T10:16:49.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.173+0000 7f7f17280640 1 -- start start 2024-04-25T10:16:49.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.173+0000 7f7f17280640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f1015b6f0 0x7f7f101620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.173+0000 7f7f17280640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7f100ceef0 0x7f7f1015e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.173+0000 7f7f17280640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1015bad0 con 0x7f7f1015e0d0 2024-04-25T10:16:49.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.173+0000 7f7f17280640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f10162610 con 0x7f7f1015b6f0 2024-04-25T10:16:49.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.173+0000 7f7f17280640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f10161c40 con 0x7f7f100ceef0 2024-04-25T10:16:49.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.173+0000 7f7f1501c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f1015b6f0 0x7f7f101620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.173+0000 7f7f1501c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f1015b6f0 0x7f7f101620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33308/0 (socket says 172.21.15.60:33308) 2024-04-25T10:16:49.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.173+0000 7f7f1501c640 1 -- 172.21.15.60:0/2283861158 learned_addr learned my addr 172.21.15.60:0/2283861158 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:49.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.173+0000 7f7f1581d640 1 --2- 172.21.15.60:0/2283861158 >> v2:172.21.15.100:3300/0 conn(0x7f7f100ceef0 0x7f7f1015e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.173+0000 7f7f1501c640 1 -- 172.21.15.60:0/2283861158 >> v2:172.21.15.100:3300/0 conn(0x7f7f100ceef0 msgr2=0x7f7f1015e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:49.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.173+0000 7f7f1501c640 1 --2- 172.21.15.60:0/2283861158 >> v2:172.21.15.100:3300/0 conn(0x7f7f100ceef0 0x7f7f1015e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.173+0000 7f7f1501c640 1 -- 172.21.15.60:0/2283861158 >> v1:172.21.15.100:6789/0 conn(0x7f7f1015e0d0 legacy=0x7f7f100ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:49.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.173+0000 7f7f1501c640 1 -- 172.21.15.60:0/2283861158 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1015ebe0 con 0x7f7f1015b6f0 2024-04-25T10:16:49.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.173+0000 7f7f1581d640 1 --2- 172.21.15.60:0/2283861158 >> v2:172.21.15.100:3300/0 conn(0x7f7f100ceef0 0x7f7f1015e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:49.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.177+0000 7f7f1501c640 1 --2- 172.21.15.60:0/2283861158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f1015b6f0 0x7f7f101620d0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f7f10073010 tx=0x7f7f0c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f0491602dcc015 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.177+0000 7f7f03fff640 1 -- 172.21.15.60:0/2283861158 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0b1070 con 0x7f7f1015b6f0 2024-04-25T10:16:49.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.177+0000 7f7f03fff640 1 -- 172.21.15.60:0/2283861158 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c0ac630 con 0x7f7f1015b6f0 2024-04-25T10:16:49.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.177+0000 7f7f03fff640 1 -- 172.21.15.60:0/2283861158 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0a8050 con 0x7f7f1015b6f0 2024-04-25T10:16:49.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.177+0000 7f7f17280640 1 -- 172.21.15.60:0/2283861158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f1015b6f0 msgr2=0x7f7f101620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.177+0000 7f7f17280640 1 --2- 172.21.15.60:0/2283861158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f1015b6f0 0x7f7f101620d0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f7f10073010 tx=0x7f7f0c01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.177+0000 7f7f17280640 1 -- 172.21.15.60:0/2283861158 shutdown_connections 2024-04-25T10:16:49.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.177+0000 7f7f17280640 1 --2- 172.21.15.60:0/2283861158 >> v2:172.21.15.100:3300/0 conn(0x7f7f100ceef0 0x7f7f1015e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.177+0000 7f7f17280640 1 --2- 172.21.15.60:0/2283861158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f1015b6f0 0x7f7f101620d0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.177+0000 7f7f17280640 1 -- 172.21.15.60:0/2283861158 >> 172.21.15.60:0/2283861158 conn(0x7f7f10072340 msgr2=0x7f7f10072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:49.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.177+0000 7f7f17280640 1 -- 172.21.15.60:0/2283861158 shutdown_connections 2024-04-25T10:16:49.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.177+0000 7f7f17280640 1 -- 172.21.15.60:0/2283861158 wait complete. 2024-04-25T10:16:49.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.177+0000 7f7f17280640 1 Processor -- start 2024-04-25T10:16:49.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.177+0000 7f7f17280640 1 -- start start 2024-04-25T10:16:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.177+0000 7f7f17280640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f100ceef0 0x7f7f100cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.177+0000 7f7f17280640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1015b6f0 0x7f7f100c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f17280640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f1015e0d0 0x7f7f100c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f17280640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100d24c0 con 0x7f7f100ceef0 2024-04-25T10:16:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f17280640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100d2340 con 0x7f7f1015b6f0 2024-04-25T10:16:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f17280640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100d2640 con 0x7f7f1015e0d0 2024-04-25T10:16:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f1501c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f100ceef0 0x7f7f100cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f1501c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f100ceef0 0x7f7f100cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33322/0 (socket says 172.21.15.60:33322) 2024-04-25T10:16:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f1501c640 1 -- 172.21.15.60:0/3400021680 learned_addr learned my addr 172.21.15.60:0/3400021680 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f1481b640 1 --2- 172.21.15.60:0/3400021680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1015b6f0 0x7f7f100c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f1581d640 1 --2- 172.21.15.60:0/3400021680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f1015e0d0 0x7f7f100c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f1501c640 1 -- 172.21.15.60:0/3400021680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f1015e0d0 msgr2=0x7f7f100c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f1501c640 1 --2- 172.21.15.60:0/3400021680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f1015e0d0 0x7f7f100c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f1501c640 1 -- 172.21.15.60:0/3400021680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1015b6f0 msgr2=0x7f7f100c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f1501c640 1 --2- 172.21.15.60:0/3400021680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1015b6f0 0x7f7f100c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f1501c640 1 -- 172.21.15.60:0/3400021680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f100cac80 con 0x7f7f100ceef0 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f1581d640 1 --2- 172.21.15.60:0/3400021680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f1015e0d0 0x7f7f100c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f1501c640 1 --2- 172.21.15.60:0/3400021680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f100ceef0 0x7f7f100cb4d0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0af040 tx=0x7f7f0c019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f01ffb640 1 -- 172.21.15.60:0/3400021680 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0bc070 con 0x7f7f100ceef0 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f01ffb640 1 -- 172.21.15.60:0/3400021680 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c0ab800 con 0x7f7f100ceef0 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f17280640 1 -- 172.21.15.60:0/3400021680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f100caf60 con 0x7f7f100ceef0 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f01ffb640 1 -- 172.21.15.60:0/3400021680 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0b1070 con 0x7f7f100ceef0 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f17280640 1 -- 172.21.15.60:0/3400021680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f10153530 con 0x7f7f100ceef0 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f17280640 1 -- 172.21.15.60:0/3400021680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f10074770 con 0x7f7f100ceef0 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f01ffb640 1 -- 172.21.15.60:0/3400021680 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f0c0bf070 con 0x7f7f100ceef0 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f01ffb640 1 --2- 172.21.15.60:0/3400021680 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ef4066f60 0x7f7ef4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f01ffb640 1 -- 172.21.15.60:0/3400021680 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f7f0c11ea80 con 0x7f7f100ceef0 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.181+0000 7f7f1481b640 1 --2- 172.21.15.60:0/3400021680 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ef4066f60 0x7f7ef4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.185+0000 7f7f1481b640 1 --2- 172.21.15.60:0/3400021680 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ef4066f60 0x7f7ef4069420 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7f100c6cc0 tx=0x7f7f040094f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.185+0000 7f7f01ffb640 1 -- 172.21.15.60:0/3400021680 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f0c0160d0 con 0x7f7f100ceef0 2024-04-25T10:16:49.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:49.377+0000 7f7f17280640 1 -- 172.21.15.60:0/3400021680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f7f100d16e0 con 0x7f7f100ceef0 2024-04-25T10:16:50.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.313+0000 7f7f01ffb640 1 -- 172.21.15.60:0/3400021680 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v102) v1 ==== 162+0+0 (secure 0 0 0) 0x7f7f0c0ef9e0 con 0x7f7f100ceef0 2024-04-25T10:16:50.320 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:16:50.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.317+0000 7f7f17280640 1 -- 172.21.15.60:0/3400021680 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ef4066f60 msgr2=0x7f7ef4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.317+0000 7f7f17280640 1 --2- 172.21.15.60:0/3400021680 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ef4066f60 0x7f7ef4069420 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7f100c6cc0 tx=0x7f7f040094f0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.317+0000 7f7f17280640 1 -- 172.21.15.60:0/3400021680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f100ceef0 msgr2=0x7f7f100cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.317+0000 7f7f17280640 1 --2- 172.21.15.60:0/3400021680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f100ceef0 0x7f7f100cb4d0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0af040 tx=0x7f7f0c019280 comp rx=0 tx=0).stop 2024-04-25T10:16:50.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.317+0000 7f7f17280640 1 -- 172.21.15.60:0/3400021680 shutdown_connections 2024-04-25T10:16:50.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.317+0000 7f7f17280640 1 --2- 172.21.15.60:0/3400021680 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f7ef4066f60 0x7f7ef4069420 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.317+0000 7f7f17280640 1 --2- 172.21.15.60:0/3400021680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f1015e0d0 0x7f7f100c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.317+0000 7f7f17280640 1 --2- 172.21.15.60:0/3400021680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1015b6f0 0x7f7f100c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.317+0000 7f7f17280640 1 --2- 172.21.15.60:0/3400021680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f100ceef0 0x7f7f100cb4d0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.317+0000 7f7f17280640 1 -- 172.21.15.60:0/3400021680 >> 172.21.15.60:0/3400021680 conn(0x7f7f10072340 msgr2=0x7f7f100d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:50.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.317+0000 7f7f17280640 1 -- 172.21.15.60:0/3400021680 shutdown_connections 2024-04-25T10:16:50.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.317+0000 7f7f17280640 1 -- 172.21.15.60:0/3400021680 wait complete. 2024-04-25T10:16:50.347 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:50.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.457+0000 7f9a1c5b0640 1 Processor -- start 2024-04-25T10:16:50.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.461+0000 7f9a1c5b0640 1 -- start start 2024-04-25T10:16:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.461+0000 7f9a1c5b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a140b3ef0 0x7f9a140cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.461+0000 7f9a1c5b0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1406f130 con 0x7f9a1415a7a0 2024-04-25T10:16:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.461+0000 7f9a1c5b0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1406f410 con 0x7f9a1406ec30 2024-04-25T10:16:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.461+0000 7f9a1c5b0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a140cdaa0 con 0x7f9a140b3ef0 2024-04-25T10:16:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.461+0000 7f9a1a34c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9a1415a7a0 0x7f9a14071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52650/0 (socket says 172.21.15.60:52650) 2024-04-25T10:16:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.461+0000 7f9a1a34c640 1 -- 172.21.15.60:0/1642555030 learned_addr learned my addr 172.21.15.60:0/1642555030 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.461+0000 7f9a1ab4d640 1 --2- 172.21.15.60:0/1642555030 >> v2:172.21.15.60:3300/0 conn(0x7f9a140b3ef0 0x7f9a140cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.461+0000 7f9a1934a640 1 -- 172.21.15.60:0/1642555030 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595725237 0 0) 0x7f9a1406f130 con 0x7f9a1415a7a0 2024-04-25T10:16:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.461+0000 7f9a1934a640 1 -- 172.21.15.60:0/1642555030 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99fc005740 con 0x7f9a1415a7a0 2024-04-25T10:16:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.461+0000 7f9a1ab4d640 1 -- 172.21.15.60:0/1642555030 >> v1:172.21.15.100:6789/0 conn(0x7f9a1406ec30 legacy=0x7f9a140b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.461+0000 7f9a1ab4d640 1 -- 172.21.15.60:0/1642555030 >> v1:172.21.15.60:6789/0 conn(0x7f9a1415a7a0 legacy=0x7f9a14071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.461+0000 7f9a1ab4d640 1 -- 172.21.15.60:0/1642555030 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1406d540 con 0x7f9a140b3ef0 2024-04-25T10:16:50.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1ab4d640 1 --2- 172.21.15.60:0/1642555030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a140b3ef0 0x7f9a140cd560 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f9a040097f0 tx=0x7f9a0401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84dc406f1ccf8da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1934a640 1 -- 172.21.15.60:0/1642555030 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040b0070 con 0x7f9a140b3ef0 2024-04-25T10:16:50.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1934a640 1 -- 172.21.15.60:0/1642555030 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a040ab600 con 0x7f9a140b3ef0 2024-04-25T10:16:50.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1934a640 1 -- 172.21.15.60:0/1642555030 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040ab9d0 con 0x7f9a140b3ef0 2024-04-25T10:16:50.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1c5b0640 1 -- 172.21.15.60:0/1642555030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a140b3ef0 msgr2=0x7f9a140cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1c5b0640 1 --2- 172.21.15.60:0/1642555030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a140b3ef0 0x7f9a140cd560 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f9a040097f0 tx=0x7f9a0401c460 comp rx=0 tx=0).stop 2024-04-25T10:16:50.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1c5b0640 1 -- 172.21.15.60:0/1642555030 shutdown_connections 2024-04-25T10:16:50.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1c5b0640 1 --2- 172.21.15.60:0/1642555030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a140b3ef0 0x7f9a140cd560 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1c5b0640 1 -- 172.21.15.60:0/1642555030 >> 172.21.15.60:0/1642555030 conn(0x7f9a140b12d0 msgr2=0x7f9a140b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:50.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1c5b0640 1 -- 172.21.15.60:0/1642555030 shutdown_connections 2024-04-25T10:16:50.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1c5b0640 1 -- 172.21.15.60:0/1642555030 wait complete. 2024-04-25T10:16:50.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1c5b0640 1 Processor -- start 2024-04-25T10:16:50.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1c5b0640 1 -- start start 2024-04-25T10:16:50.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1c5b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a1406ec30 0x7f9a1414c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1c5b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a140b3ef0 0x7f9a140cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1c5b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1415a7a0 0x7f9a1415c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1c5b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1406e8c0 con 0x7f9a1415a7a0 2024-04-25T10:16:50.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1c5b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1406e740 con 0x7f9a140b3ef0 2024-04-25T10:16:50.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1c5b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1406d2f0 con 0x7f9a1406ec30 2024-04-25T10:16:50.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1ab4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1415a7a0 0x7f9a1415c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1ab4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1415a7a0 0x7f9a1415c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33340/0 (socket says 172.21.15.60:33340) 2024-04-25T10:16:50.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1ab4d640 1 -- 172.21.15.60:0/761420243 learned_addr learned my addr 172.21.15.60:0/761420243 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:50.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1a34c640 1 --2- 172.21.15.60:0/761420243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a1406ec30 0x7f9a1414c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a19b4b640 1 --2- 172.21.15.60:0/761420243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a140b3ef0 0x7f9a140cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1ab4d640 1 -- 172.21.15.60:0/761420243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a1406ec30 msgr2=0x7f9a1414c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1ab4d640 1 --2- 172.21.15.60:0/761420243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a1406ec30 0x7f9a1414c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1ab4d640 1 -- 172.21.15.60:0/761420243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a140b3ef0 msgr2=0x7f9a140cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1ab4d640 1 --2- 172.21.15.60:0/761420243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a140b3ef0 0x7f9a140cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1ab4d640 1 -- 172.21.15.60:0/761420243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1415da70 con 0x7f9a1415a7a0 2024-04-25T10:16:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.465+0000 7f9a1ab4d640 1 --2- 172.21.15.60:0/761420243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1415a7a0 0x7f9a1415c7e0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f9a040ae040 tx=0x7f9a0401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.469+0000 7f9a12ffd640 1 -- 172.21.15.60:0/761420243 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040b0050 con 0x7f9a1415a7a0 2024-04-25T10:16:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.469+0000 7f9a12ffd640 1 -- 172.21.15.60:0/761420243 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a040bc070 con 0x7f9a1415a7a0 2024-04-25T10:16:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.469+0000 7f9a1c5b0640 1 -- 172.21.15.60:0/761420243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1415dd50 con 0x7f9a1415a7a0 2024-04-25T10:16:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.469+0000 7f9a12ffd640 1 -- 172.21.15.60:0/761420243 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040b5a10 con 0x7f9a1415a7a0 2024-04-25T10:16:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.469+0000 7f9a1c5b0640 1 -- 172.21.15.60:0/761420243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1415f770 con 0x7f9a1415a7a0 2024-04-25T10:16:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.469+0000 7f9a1c5b0640 1 -- 172.21.15.60:0/761420243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a14074770 con 0x7f9a1415a7a0 2024-04-25T10:16:50.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.469+0000 7f9a12ffd640 1 -- 172.21.15.60:0/761420243 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a040ab480 con 0x7f9a1415a7a0 2024-04-25T10:16:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.469+0000 7f9a12ffd640 1 --2- 172.21.15.60:0/761420243 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a00066f60 0x7f9a00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.469+0000 7f9a12ffd640 1 -- 172.21.15.60:0/761420243 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f9a0411f920 con 0x7f9a1415a7a0 2024-04-25T10:16:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.469+0000 7f9a1a34c640 1 --2- 172.21.15.60:0/761420243 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a00066f60 0x7f9a00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.469+0000 7f9a1a34c640 1 --2- 172.21.15.60:0/761420243 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a00066f60 0x7f9a00069420 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c005c10 tx=0x7f9a0c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.473+0000 7f9a12ffd640 1 -- 172.21.15.60:0/761420243 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a04016050 con 0x7f9a1415a7a0 2024-04-25T10:16:50.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.661+0000 7f9a1c5b0640 1 -- 172.21.15.60:0/761420243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a141508f0 con 0x7f9a1415a7a0 2024-04-25T10:16:50.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.661+0000 7f9a12ffd640 1 -- 172.21.15.60:0/761420243 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f9a040f0890 con 0x7f9a1415a7a0 2024-04-25T10:16:50.668 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:50.669 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":103,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2440975447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2440975447},{"type":"v1","addr":"172.21.15.100:6835","nonce":2440975447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2398080758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2398080758},{"type":"v1","addr":"172.21.15.60:6833","nonce":2398080758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2759861958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2759861958},{"type":"v1","addr":"172.21.15.100:6837","nonce":2759861958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:07.763322+0000","modified":"2024-04-25T10:16:50.322670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9894":{"gid":9894,"name":"c","rank":0,"incarnation":103,"state":"up:replay","state_seq":1,"addr":"172.21.15.60:6835/3461876440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3461876440},{"type":"v1","addr":"172.21.15.60:6835","nonce":3461876440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:16:50.669 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 103 2024-04-25T10:16:50.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.665+0000 7f9a1c5b0640 1 -- 172.21.15.60:0/761420243 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a00066f60 msgr2=0x7f9a00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.665+0000 7f9a1c5b0640 1 --2- 172.21.15.60:0/761420243 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a00066f60 0x7f9a00069420 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c005c10 tx=0x7f9a0c009040 comp rx=0 tx=0).stop 2024-04-25T10:16:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.665+0000 7f9a1c5b0640 1 -- 172.21.15.60:0/761420243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1415a7a0 msgr2=0x7f9a1415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.665+0000 7f9a1c5b0640 1 --2- 172.21.15.60:0/761420243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1415a7a0 0x7f9a1415c7e0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f9a040ae040 tx=0x7f9a0401aca0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.665+0000 7f9a1c5b0640 1 -- 172.21.15.60:0/761420243 shutdown_connections 2024-04-25T10:16:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.665+0000 7f9a1c5b0640 1 --2- 172.21.15.60:0/761420243 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f9a00066f60 0x7f9a00069420 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.665+0000 7f9a1c5b0640 1 --2- 172.21.15.60:0/761420243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a1415a7a0 0x7f9a1415c7e0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.665+0000 7f9a1c5b0640 1 --2- 172.21.15.60:0/761420243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a140b3ef0 0x7f9a140cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.665+0000 7f9a1c5b0640 1 --2- 172.21.15.60:0/761420243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a1406ec30 0x7f9a1414c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.665+0000 7f9a1c5b0640 1 -- 172.21.15.60:0/761420243 >> 172.21.15.60:0/761420243 conn(0x7f9a140b12d0 msgr2=0x7f9a140cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.665+0000 7f9a1c5b0640 1 -- 172.21.15.60:0/761420243 shutdown_connections 2024-04-25T10:16:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:50.665+0000 7f9a1c5b0640 1 -- 172.21.15.60:0/761420243 wait complete. 2024-04-25T10:16:51.090 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:16:07.763322+0000', 'modified': '2024-04-25T10:16:50.322670+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9894}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9894': {'gid': 9894, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.60:6835/3461876440', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 3461876440}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 3461876440}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:51.090 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9894:up:replay 2024-04-25T10:16:52.091 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:52.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407fa85640 1 Processor -- start 2024-04-25T10:16:52.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407fa85640 1 -- start start 2024-04-25T10:16:52.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407fa85640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f407806ec30 0x7f40780cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407fa85640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40780b37e0 0x7f40780b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407fa85640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407806f130 con 0x7f407815a7a0 2024-04-25T10:16:52.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407fa85640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40780cdaa0 con 0x7f407806ec30 2024-04-25T10:16:52.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407fa85640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407806d2f0 con 0x7f40780b37e0 2024-04-25T10:16:52.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407d020640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f407806ec30 0x7f40780cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407e022640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40780b37e0 0x7f40780b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407e022640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40780b37e0 0x7f40780b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:52572/0 (socket says 172.21.15.60:52572) 2024-04-25T10:16:52.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407d020640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f407806ec30 0x7f40780cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49210/0 (socket says 172.21.15.60:49210) 2024-04-25T10:16:52.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407e022640 1 -- 172.21.15.60:0/2742755544 learned_addr learned my addr 172.21.15.60:0/2742755544 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:52.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407d821640 1 --1- 172.21.15.60:0/2742755544 >> v1:172.21.15.100:6789/0 conn(0x7f407815a7a0 0x7f4078071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:44644/0 (socket says 172.21.15.60:44644) 2024-04-25T10:16:52.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407d020640 1 -- 172.21.15.60:0/2742755544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40780b37e0 msgr2=0x7f40780b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:52.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407d020640 1 --2- 172.21.15.60:0/2742755544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40780b37e0 0x7f40780b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407d020640 1 -- 172.21.15.60:0/2742755544 >> v1:172.21.15.100:6789/0 conn(0x7f407815a7a0 legacy=0x7f4078071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.221+0000 7f407d020640 1 -- 172.21.15.60:0/2742755544 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407806f410 con 0x7f407806ec30 2024-04-25T10:16:52.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407d020640 1 --2- 172.21.15.60:0/2742755544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f407806ec30 0x7f40780cd560 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f40680097f0 tx=0x7f406801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e17c493ac769d3d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:52.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407c81f640 1 -- 172.21.15.60:0/2742755544 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40680b1070 con 0x7f407806ec30 2024-04-25T10:16:52.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407c81f640 1 -- 172.21.15.60:0/2742755544 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40680ac410 con 0x7f407806ec30 2024-04-25T10:16:52.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407c81f640 1 -- 172.21.15.60:0/2742755544 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40680ac7e0 con 0x7f407806ec30 2024-04-25T10:16:52.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407fa85640 1 -- 172.21.15.60:0/2742755544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f407806ec30 msgr2=0x7f40780cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407fa85640 1 --2- 172.21.15.60:0/2742755544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f407806ec30 0x7f40780cd560 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f40680097f0 tx=0x7f406801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407fa85640 1 -- 172.21.15.60:0/2742755544 shutdown_connections 2024-04-25T10:16:52.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407fa85640 1 --2- 172.21.15.60:0/2742755544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f407806ec30 0x7f40780cd560 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407fa85640 1 --2- 172.21.15.60:0/2742755544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40780b37e0 0x7f40780b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407fa85640 1 -- 172.21.15.60:0/2742755544 >> 172.21.15.60:0/2742755544 conn(0x7f40780b12d0 msgr2=0x7f40780b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:52.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407fa85640 1 -- 172.21.15.60:0/2742755544 shutdown_connections 2024-04-25T10:16:52.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407fa85640 1 -- 172.21.15.60:0/2742755544 wait complete. 2024-04-25T10:16:52.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407fa85640 1 Processor -- start 2024-04-25T10:16:52.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407fa85640 1 -- start start 2024-04-25T10:16:52.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407fa85640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407806ec30 0x7f407814d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407fa85640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40780b37e0 0x7f407814fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407fa85640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f407815a7a0 0x7f407814ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407fa85640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407806e8c0 con 0x7f407806ec30 2024-04-25T10:16:52.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407fa85640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407806e740 con 0x7f407815a7a0 2024-04-25T10:16:52.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407fa85640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407806d8b0 con 0x7f40780b37e0 2024-04-25T10:16:52.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407d821640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407806ec30 0x7f407814d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407d821640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407806ec30 0x7f407814d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33346/0 (socket says 172.21.15.60:33346) 2024-04-25T10:16:52.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.225+0000 7f407d821640 1 -- 172.21.15.60:0/2131810681 learned_addr learned my addr 172.21.15.60:0/2131810681 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:52.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f407e022640 1 --2- 172.21.15.60:0/2131810681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f407815a7a0 0x7f407814ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f407d020640 1 --2- 172.21.15.60:0/2131810681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40780b37e0 0x7f407814fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f407d821640 1 -- 172.21.15.60:0/2131810681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40780b37e0 msgr2=0x7f407814fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f407d821640 1 --2- 172.21.15.60:0/2131810681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40780b37e0 0x7f407814fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f407d821640 1 -- 172.21.15.60:0/2131810681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f407815a7a0 msgr2=0x7f407814ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f407d821640 1 --2- 172.21.15.60:0/2131810681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f407815a7a0 0x7f407814ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f407d821640 1 -- 172.21.15.60:0/2131810681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4078158500 con 0x7f407806ec30 2024-04-25T10:16:52.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f407d821640 1 --2- 172.21.15.60:0/2131810681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407806ec30 0x7f407814d6d0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f407400c370 tx=0x7f407400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:52.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f406e7fc640 1 -- 172.21.15.60:0/2131810681 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40740074e0 con 0x7f407806ec30 2024-04-25T10:16:52.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f406e7fc640 1 -- 172.21.15.60:0/2131810681 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4074007680 con 0x7f407806ec30 2024-04-25T10:16:52.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f407fa85640 1 -- 172.21.15.60:0/2131810681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4078157460 con 0x7f407806ec30 2024-04-25T10:16:52.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f406e7fc640 1 -- 172.21.15.60:0/2131810681 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074004a30 con 0x7f407806ec30 2024-04-25T10:16:52.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f407fa85640 1 -- 172.21.15.60:0/2131810681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4078157a20 con 0x7f407806ec30 2024-04-25T10:16:52.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f407fa85640 1 -- 172.21.15.60:0/2131810681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4078074770 con 0x7f407806ec30 2024-04-25T10:16:52.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f406e7fc640 1 -- 172.21.15.60:0/2131810681 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f407401a020 con 0x7f407806ec30 2024-04-25T10:16:52.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f406e7fc640 1 --2- 172.21.15.60:0/2131810681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f405c066f60 0x7f405c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f406e7fc640 1 -- 172.21.15.60:0/2131810681 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f40740849d0 con 0x7f407806ec30 2024-04-25T10:16:52.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.229+0000 7f407d020640 1 --2- 172.21.15.60:0/2131810681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f405c066f60 0x7f405c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.233+0000 7f407d020640 1 --2- 172.21.15.60:0/2131810681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f405c066f60 0x7f405c069420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f40680ae000 tx=0x7f4068008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:52.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.233+0000 7f406e7fc640 1 -- 172.21.15.60:0/2131810681 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4074055800 con 0x7f407806ec30 2024-04-25T10:16:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.429+0000 7f407fa85640 1 -- 172.21.15.60:0/2131810681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40780ced60 con 0x7f407806ec30 2024-04-25T10:16:52.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.429+0000 7f406e7fc640 1 -- 172.21.15.60:0/2131810681 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f4074055100 con 0x7f407806ec30 2024-04-25T10:16:52.437 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:52.437 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2440975447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2440975447},{"type":"v1","addr":"172.21.15.100:6835","nonce":2440975447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2398080758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2398080758},{"type":"v1","addr":"172.21.15.60:6833","nonce":2398080758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2759861958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2759861958},{"type":"v1","addr":"172.21.15.100:6837","nonce":2759861958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:07.763322+0000","modified":"2024-04-25T10:16:51.340531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9894":{"gid":9894,"name":"c","rank":0,"incarnation":103,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.60:6835/3461876440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3461876440},{"type":"v1","addr":"172.21.15.60:6835","nonce":3461876440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:16:52.437 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 105 2024-04-25T10:16:52.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.433+0000 7f407fa85640 1 -- 172.21.15.60:0/2131810681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f405c066f60 msgr2=0x7f405c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.433+0000 7f407fa85640 1 --2- 172.21.15.60:0/2131810681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f405c066f60 0x7f405c069420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f40680ae000 tx=0x7f4068008040 comp rx=0 tx=0).stop 2024-04-25T10:16:52.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.433+0000 7f407fa85640 1 -- 172.21.15.60:0/2131810681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407806ec30 msgr2=0x7f407814d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.433+0000 7f407fa85640 1 --2- 172.21.15.60:0/2131810681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407806ec30 0x7f407814d6d0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f407400c370 tx=0x7f407400c830 comp rx=0 tx=0).stop 2024-04-25T10:16:52.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.433+0000 7f407fa85640 1 -- 172.21.15.60:0/2131810681 shutdown_connections 2024-04-25T10:16:52.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.433+0000 7f407fa85640 1 --2- 172.21.15.60:0/2131810681 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f405c066f60 0x7f405c069420 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.433+0000 7f407fa85640 1 --2- 172.21.15.60:0/2131810681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f407815a7a0 0x7f407814ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.433+0000 7f407fa85640 1 --2- 172.21.15.60:0/2131810681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40780b37e0 0x7f407814fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.433+0000 7f407fa85640 1 --2- 172.21.15.60:0/2131810681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407806ec30 0x7f407814d6d0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.433+0000 7f407fa85640 1 -- 172.21.15.60:0/2131810681 >> 172.21.15.60:0/2131810681 conn(0x7f40780b12d0 msgr2=0x7f40780b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:52.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.433+0000 7f407fa85640 1 -- 172.21.15.60:0/2131810681 shutdown_connections 2024-04-25T10:16:52.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:52.433+0000 7f407fa85640 1 -- 172.21.15.60:0/2131810681 wait complete. 2024-04-25T10:16:52.858 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:16:07.763322+0000', 'modified': '2024-04-25T10:16:51.340531+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9894}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9894': {'gid': 9894, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.60:6835/3461876440', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 3461876440}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 3461876440}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:52.858 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9894:up:rejoin 2024-04-25T10:16:53.859 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:53.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.989+0000 7f0bdae43640 1 Processor -- start 2024-04-25T10:16:53.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.989+0000 7f0bdae43640 1 -- start start 2024-04-25T10:16:53.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.989+0000 7f0bdae43640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0bd415f480 0x7f0bd415f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.989+0000 7f0bdae43640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd40cd390 0x7f0bd409a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.989+0000 7f0bdae43640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd415fda0 con 0x7f0bd409a930 2024-04-25T10:16:53.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.989+0000 7f0bdae43640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bd40cc5d0 con 0x7f0bd415f480 2024-04-25T10:16:53.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.989+0000 7f0bdae43640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd40cc8b0 con 0x7f0bd40cd390 2024-04-25T10:16:53.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.989+0000 7f0bd8bdf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0bd415f480 0x7f0bd415f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.989+0000 7f0bd8bdf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0bd415f480 0x7f0bd415f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33350/0 (socket says 172.21.15.60:33350) 2024-04-25T10:16:53.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.989+0000 7f0bd8bdf640 1 -- 172.21.15.60:0/809653563 learned_addr learned my addr 172.21.15.60:0/809653563 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:53.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.989+0000 7f0bd3fff640 1 --2- 172.21.15.60:0/809653563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd40cd390 0x7f0bd409a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.989+0000 7f0bd8bdf640 1 -- 172.21.15.60:0/809653563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd40cd390 msgr2=0x7f0bd409a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:53.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.989+0000 7f0bd8bdf640 1 --2- 172.21.15.60:0/809653563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd40cd390 0x7f0bd409a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.989+0000 7f0bd8bdf640 1 -- 172.21.15.60:0/809653563 >> v1:172.21.15.100:6789/0 conn(0x7f0bd409a930 legacy=0x7f0bd40cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:53.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.989+0000 7f0bd8bdf640 1 -- 172.21.15.60:0/809653563 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd40ccb90 con 0x7f0bd415f480 2024-04-25T10:16:53.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.989+0000 7f0bd3fff640 1 --2- 172.21.15.60:0/809653563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd40cd390 0x7f0bd409a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:53.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.993+0000 7f0bd8bdf640 1 --2- 172.21.15.60:0/809653563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd415f480 0x7f0bd415f860 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009870 tx=0x7f0bc401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fd0ffc22574e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.993+0000 7f0bd37fe640 1 -- 172.21.15.60:0/809653563 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40aa670 con 0x7f0bd415f480 2024-04-25T10:16:53.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.993+0000 7f0bd37fe640 1 -- 172.21.15.60:0/809653563 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc40af070 con 0x7f0bd415f480 2024-04-25T10:16:53.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.993+0000 7f0bd37fe640 1 -- 172.21.15.60:0/809653563 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40aaa30 con 0x7f0bd415f480 2024-04-25T10:16:53.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.993+0000 7f0bdae43640 1 -- 172.21.15.60:0/809653563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd415f480 msgr2=0x7f0bd415f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.993+0000 7f0bdae43640 1 --2- 172.21.15.60:0/809653563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd415f480 0x7f0bd415f860 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009870 tx=0x7f0bc401c530 comp rx=0 tx=0).stop 2024-04-25T10:16:53.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.993+0000 7f0bdae43640 1 -- 172.21.15.60:0/809653563 shutdown_connections 2024-04-25T10:16:53.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.993+0000 7f0bdae43640 1 --2- 172.21.15.60:0/809653563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd40cd390 0x7f0bd409a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.993+0000 7f0bdae43640 1 --2- 172.21.15.60:0/809653563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd415f480 0x7f0bd415f860 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.993+0000 7f0bdae43640 1 -- 172.21.15.60:0/809653563 >> 172.21.15.60:0/809653563 conn(0x7f0bd40b32a0 msgr2=0x7f0bd40b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.993+0000 7f0bdae43640 1 -- 172.21.15.60:0/809653563 shutdown_connections 2024-04-25T10:16:53.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.993+0000 7f0bdae43640 1 -- 172.21.15.60:0/809653563 wait complete. 2024-04-25T10:16:54.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.993+0000 7f0bdae43640 1 Processor -- start 2024-04-25T10:16:54.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.993+0000 7f0bdae43640 1 -- start start 2024-04-25T10:16:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bdae43640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd409a930 0x7f0bd415a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bdae43640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd40cd390 0x7f0bd40b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bdae43640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd415f480 0x7f0bd40b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bdae43640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd415cd40 con 0x7f0bd40cd390 2024-04-25T10:16:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bdae43640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd415cbc0 con 0x7f0bd415f480 2024-04-25T10:16:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bdae43640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd41687c0 con 0x7f0bd409a930 2024-04-25T10:16:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd40cd390 0x7f0bd40b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd40cd390 0x7f0bd40b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33366/0 (socket says 172.21.15.60:33366) 2024-04-25T10:16:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd3fff640 1 -- 172.21.15.60:0/6210095 learned_addr learned my addr 172.21.15.60:0/6210095 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd8bdf640 1 --2- 172.21.15.60:0/6210095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd409a930 0x7f0bd415a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd93e0640 1 --2- 172.21.15.60:0/6210095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd415f480 0x7f0bd40b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd3fff640 1 -- 172.21.15.60:0/6210095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd409a930 msgr2=0x7f0bd415a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd3fff640 1 --2- 172.21.15.60:0/6210095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd409a930 0x7f0bd415a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd3fff640 1 -- 172.21.15.60:0/6210095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd415f480 msgr2=0x7f0bd40b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd3fff640 1 --2- 172.21.15.60:0/6210095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd415f480 0x7f0bd40b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd3fff640 1 -- 172.21.15.60:0/6210095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd414b460 con 0x7f0bd40cd390 2024-04-25T10:16:54.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd93e0640 1 --2- 172.21.15.60:0/6210095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd415f480 0x7f0bd40b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:54.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd3fff640 1 --2- 172.21.15.60:0/6210095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd40cd390 0x7f0bd40b4600 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800c7c0 tx=0x7f0bc800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd17fa640 1 -- 172.21.15.60:0/6210095 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8007a30 con 0x7f0bd40cd390 2024-04-25T10:16:54.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd17fa640 1 -- 172.21.15.60:0/6210095 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc8007bd0 con 0x7f0bd40cd390 2024-04-25T10:16:54.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd17fa640 1 -- 172.21.15.60:0/6210095 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc80113f0 con 0x7f0bd40cd390 2024-04-25T10:16:54.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bdae43640 1 -- 172.21.15.60:0/6210095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd40b7220 con 0x7f0bd40cd390 2024-04-25T10:16:54.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bdae43640 1 -- 172.21.15.60:0/6210095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd40c6fc0 con 0x7f0bd40cd390 2024-04-25T10:16:54.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bdae43640 1 -- 172.21.15.60:0/6210095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd4074770 con 0x7f0bd40cd390 2024-04-25T10:16:54.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd17fa640 1 -- 172.21.15.60:0/6210095 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0bc8011590 con 0x7f0bd40cd390 2024-04-25T10:16:54.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:53.997+0000 7f0bd17fa640 1 --2- 172.21.15.60:0/6210095 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0bc0066f60 0x7f0bc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.001+0000 7f0bd17fa640 1 -- 172.21.15.60:0/6210095 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f0bc8085180 con 0x7f0bd40cd390 2024-04-25T10:16:54.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.001+0000 7f0bd8bdf640 1 --2- 172.21.15.60:0/6210095 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0bc0066f60 0x7f0bc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.001+0000 7f0bd8bdf640 1 --2- 172.21.15.60:0/6210095 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0bc0066f60 0x7f0bc0069420 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40ae000 tx=0x7f0bc40a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.005+0000 7f0bd17fa640 1 -- 172.21.15.60:0/6210095 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0bc8015020 con 0x7f0bd40cd390 2024-04-25T10:16:54.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.197+0000 7f0bdae43640 1 -- 172.21.15.60:0/6210095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bd40b40a0 con 0x7f0bd40cd390 2024-04-25T10:16:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.197+0000 7f0bd17fa640 1 -- 172.21.15.60:0/6210095 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f0bc8055b20 con 0x7f0bd40cd390 2024-04-25T10:16:54.203 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:54.203 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":106,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2440975447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2440975447},{"type":"v1","addr":"172.21.15.100:6835","nonce":2440975447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2398080758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2398080758},{"type":"v1","addr":"172.21.15.60:6833","nonce":2398080758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2759861958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2759861958},{"type":"v1","addr":"172.21.15.100:6837","nonce":2759861958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:07.763322+0000","modified":"2024-04-25T10:16:52.349719+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9894":{"gid":9894,"name":"c","rank":0,"incarnation":103,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/3461876440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3461876440},{"type":"v1","addr":"172.21.15.60:6835","nonce":3461876440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:16:54.203 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 106 2024-04-25T10:16:54.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.197+0000 7f0bdae43640 1 -- 172.21.15.60:0/6210095 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0bc0066f60 msgr2=0x7f0bc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.197+0000 7f0bdae43640 1 --2- 172.21.15.60:0/6210095 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0bc0066f60 0x7f0bc0069420 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40ae000 tx=0x7f0bc40a7000 comp rx=0 tx=0).stop 2024-04-25T10:16:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.197+0000 7f0bdae43640 1 -- 172.21.15.60:0/6210095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd40cd390 msgr2=0x7f0bd40b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.197+0000 7f0bdae43640 1 --2- 172.21.15.60:0/6210095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd40cd390 0x7f0bd40b4600 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800c7c0 tx=0x7f0bc800cc80 comp rx=0 tx=0).stop 2024-04-25T10:16:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.197+0000 7f0bdae43640 1 -- 172.21.15.60:0/6210095 shutdown_connections 2024-04-25T10:16:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.197+0000 7f0bdae43640 1 --2- 172.21.15.60:0/6210095 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f0bc0066f60 0x7f0bc0069420 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.197+0000 7f0bdae43640 1 --2- 172.21.15.60:0/6210095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd415f480 0x7f0bd40b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.197+0000 7f0bdae43640 1 --2- 172.21.15.60:0/6210095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd40cd390 0x7f0bd40b4600 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.197+0000 7f0bdae43640 1 --2- 172.21.15.60:0/6210095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd409a930 0x7f0bd415a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.197+0000 7f0bdae43640 1 -- 172.21.15.60:0/6210095 >> 172.21.15.60:0/6210095 conn(0x7f0bd40b32a0 msgr2=0x7f0bd4154ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.197+0000 7f0bdae43640 1 -- 172.21.15.60:0/6210095 shutdown_connections 2024-04-25T10:16:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.197+0000 7f0bdae43640 1 -- 172.21.15.60:0/6210095 wait complete. 2024-04-25T10:16:54.326 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-04-25T10:16:54.321+0000 7f51866cb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:16:54.623 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 106, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:16:07.763322+0000', 'modified': '2024-04-25T10:16:52.349719+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9894}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9894': {'gid': 9894, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.60:6835/3461876440', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 3461876440}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 3461876440}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:54.624 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:16:54.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:16:54.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.753+0000 7f08b7e12640 1 Processor -- start 2024-04-25T10:16:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.753+0000 7f08b7e12640 1 -- start start 2024-04-25T10:16:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.753+0000 7f08b7e12640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08b015deb0 0x7f08b00b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.753+0000 7f08b7e12640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08b00d0340 0x7f08b00d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.753+0000 7f08b7e12640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f08b00ce2b0 0x7f08b00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.753+0000 7f08b7e12640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b00cf2c0 con 0x7f08b015deb0 2024-04-25T10:16:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.753+0000 7f08b7e12640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b00cefe0 con 0x7f08b00ce2b0 2024-04-25T10:16:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.753+0000 7f08b7e12640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b00d1d70 con 0x7f08b00d0340 2024-04-25T10:16:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.757+0000 7f08b5bae640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08b015deb0 0x7f08b00b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.757+0000 7f08b5bae640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08b015deb0 0x7f08b00b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33382/0 (socket says 172.21.15.60:33382) 2024-04-25T10:16:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.757+0000 7f08b5bae640 1 -- 172.21.15.60:0/3169314967 learned_addr learned my addr 172.21.15.60:0/3169314967 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.757+0000 7f08b63af640 1 --2- 172.21.15.60:0/3169314967 >> v2:172.21.15.100:3300/0 conn(0x7f08b00ce2b0 0x7f08b00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.757+0000 7f08b53ad640 1 --2- 172.21.15.60:0/3169314967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08b00d0340 0x7f08b00d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.757+0000 7f08b5bae640 1 -- 172.21.15.60:0/3169314967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08b00d0340 msgr2=0x7f08b00d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.757+0000 7f08b5bae640 1 --2- 172.21.15.60:0/3169314967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08b00d0340 0x7f08b00d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.757+0000 7f08b5bae640 1 -- 172.21.15.60:0/3169314967 >> v2:172.21.15.100:3300/0 conn(0x7f08b00ce2b0 msgr2=0x7f08b00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.757+0000 7f08b5bae640 1 --2- 172.21.15.60:0/3169314967 >> v2:172.21.15.100:3300/0 conn(0x7f08b00ce2b0 0x7f08b00ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.757+0000 7f08b5bae640 1 -- 172.21.15.60:0/3169314967 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b006d200 con 0x7f08b015deb0 2024-04-25T10:16:54.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.757+0000 7f08b5bae640 1 --2- 172.21.15.60:0/3169314967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08b015deb0 0x7f08b00b6c10 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f08b00d1490 tx=0x7f08a001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f65df4694f0746 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.757+0000 7f08b4bac640 1 -- 172.21.15.60:0/3169314967 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a00a5630 con 0x7f08b015deb0 2024-04-25T10:16:54.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.757+0000 7f08b4bac640 1 -- 172.21.15.60:0/3169314967 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a00a57d0 con 0x7f08b015deb0 2024-04-25T10:16:54.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.757+0000 7f08b4bac640 1 -- 172.21.15.60:0/3169314967 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a00a5ba0 con 0x7f08b015deb0 2024-04-25T10:16:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 -- 172.21.15.60:0/3169314967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08b015deb0 msgr2=0x7f08b00b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 --2- 172.21.15.60:0/3169314967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08b015deb0 0x7f08b00b6c10 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f08b00d1490 tx=0x7f08a001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 -- 172.21.15.60:0/3169314967 shutdown_connections 2024-04-25T10:16:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 --2- 172.21.15.60:0/3169314967 >> v2:172.21.15.100:3300/0 conn(0x7f08b00ce2b0 0x7f08b00ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 --2- 172.21.15.60:0/3169314967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08b00d0340 0x7f08b00d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 --2- 172.21.15.60:0/3169314967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08b015deb0 0x7f08b00b6c10 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 -- 172.21.15.60:0/3169314967 >> 172.21.15.60:0/3169314967 conn(0x7f08b006c090 msgr2=0x7f08b006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 -- 172.21.15.60:0/3169314967 shutdown_connections 2024-04-25T10:16:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 -- 172.21.15.60:0/3169314967 wait complete. 2024-04-25T10:16:54.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 Processor -- start 2024-04-25T10:16:54.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 -- start start 2024-04-25T10:16:54.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08b00ce2b0 0x7f08b00cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08b00d0340 0x7f08b014ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08b015deb0 0x7f08b01608d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b00b5d40 con 0x7f08b00d0340 2024-04-25T10:16:54.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b00b5bc0 con 0x7f08b015deb0 2024-04-25T10:16:54.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b00b5ec0 con 0x7f08b00ce2b0 2024-04-25T10:16:54.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b53ad640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08b00d0340 0x7f08b014ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b53ad640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08b00d0340 0x7f08b014ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33392/0 (socket says 172.21.15.60:33392) 2024-04-25T10:16:54.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b53ad640 1 -- 172.21.15.60:0/2457167787 learned_addr learned my addr 172.21.15.60:0/2457167787 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:54.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b5bae640 1 --2- 172.21.15.60:0/2457167787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08b00ce2b0 0x7f08b00cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b63af640 1 --2- 172.21.15.60:0/2457167787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08b015deb0 0x7f08b01608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b53ad640 1 -- 172.21.15.60:0/2457167787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08b00ce2b0 msgr2=0x7f08b00cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b53ad640 1 --2- 172.21.15.60:0/2457167787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08b00ce2b0 0x7f08b00cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b53ad640 1 -- 172.21.15.60:0/2457167787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08b015deb0 msgr2=0x7f08b01608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b53ad640 1 --2- 172.21.15.60:0/2457167787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08b015deb0 0x7f08b01608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b53ad640 1 -- 172.21.15.60:0/2457167787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b00c6270 con 0x7f08b00d0340 2024-04-25T10:16:54.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b53ad640 1 --2- 172.21.15.60:0/2457167787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08b00d0340 0x7f08b014ce50 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f08a400d800 tx=0x7f08a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08ae7fc640 1 -- 172.21.15.60:0/2457167787 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400b6c0 con 0x7f08b00d0340 2024-04-25T10:16:54.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08ae7fc640 1 -- 172.21.15.60:0/2457167787 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a4019070 con 0x7f08b00d0340 2024-04-25T10:16:54.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 -- 172.21.15.60:0/2457167787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b00c6550 con 0x7f08b00d0340 2024-04-25T10:16:54.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08ae7fc640 1 -- 172.21.15.60:0/2457167787 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400ba60 con 0x7f08b00d0340 2024-04-25T10:16:54.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.761+0000 7f08b7e12640 1 -- 172.21.15.60:0/2457167787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b00ca240 con 0x7f08b00d0340 2024-04-25T10:16:54.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08ae7fc640 1 -- 172.21.15.60:0/2457167787 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08a400bc00 con 0x7f08b00d0340 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08ae7fc640 1 --2- 172.21.15.60:0/2457167787 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f089c066f60 0x7f089c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08ae7fc640 1 -- 172.21.15.60:0/2457167787 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f08a4085600 con 0x7f08b00d0340 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08b5bae640 1 --2- 172.21.15.60:0/2457167787 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f089c066f60 0x7f089c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08baf7f000 1 Processor -- start 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08baf7f000 1 -- start start 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08b5bae640 1 --2- 172.21.15.60:0/2457167787 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f089c066f60 0x7f089c069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f08a00ae000 tx=0x7f08a00a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08baf7f000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ba0f920 0x559f6ba0fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08baf7f000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba10240 0x559f6b9c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08baf7f000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6b9c5f30 0x559f6b9c6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08baf7f000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6b8ad5f0 con 0x559f6ba10240 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08baf7f000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6b814da0 con 0x559f6ba0f920 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08baf7f000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f6b79d150 con 0x559f6b9c5f30 2024-04-25T10:16:54.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08b53ad640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba10240 0x559f6b9c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08b53ad640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba10240 0x559f6b9c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33404/0 (socket says 172.21.15.60:33404) 2024-04-25T10:16:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08b53ad640 1 -- 172.21.15.60:0/3762726061 learned_addr learned my addr 172.21.15.60:0/3762726061 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08b5bae640 1 --2- 172.21.15.60:0/3762726061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ba0f920 0x559f6ba0fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08b63af640 1 --2- 172.21.15.60:0/3762726061 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6b9c5f30 0x559f6b9c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08b53ad640 1 -- 172.21.15.60:0/3762726061 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6b9c5f30 msgr2=0x559f6b9c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08b53ad640 1 --2- 172.21.15.60:0/3762726061 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6b9c5f30 0x559f6b9c6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08b53ad640 1 -- 172.21.15.60:0/3762726061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ba0f920 msgr2=0x559f6ba0fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08b53ad640 1 --2- 172.21.15.60:0/3762726061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ba0f920 0x559f6ba0fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08b53ad640 1 -- 172.21.15.60:0/3762726061 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f6b7b6ca0 con 0x559f6ba10240 2024-04-25T10:16:54.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08b53ad640 1 --2- 172.21.15.60:0/3762726061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba10240 0x559f6b9c59f0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f08a40522b0 tx=0x7f08a40558e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5205db36b50b75e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08977fe640 1 -- 172.21.15.60:0/3762726061 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a406d020 con 0x559f6ba10240 2024-04-25T10:16:54.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08977fe640 1 -- 172.21.15.60:0/3762726061 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a4058050 con 0x559f6ba10240 2024-04-25T10:16:54.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.765+0000 7f08977fe640 1 -- 172.21.15.60:0/3762726061 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a405e560 con 0x559f6ba10240 2024-04-25T10:16:54.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 -- 172.21.15.60:0/3762726061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba10240 msgr2=0x559f6b9c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 --2- 172.21.15.60:0/3762726061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba10240 0x559f6b9c59f0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f08a40522b0 tx=0x7f08a40558e0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 -- 172.21.15.60:0/3762726061 shutdown_connections 2024-04-25T10:16:54.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 --2- 172.21.15.60:0/3762726061 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6b9c5f30 0x559f6b9c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 --2- 172.21.15.60:0/3762726061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba10240 0x559f6b9c59f0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 --2- 172.21.15.60:0/3762726061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ba0f920 0x559f6ba0fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 -- 172.21.15.60:0/3762726061 >> 172.21.15.60:0/3762726061 conn(0x559f6b9cc2a0 msgr2=0x559f6b9c42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 -- 172.21.15.60:0/3762726061 shutdown_connections 2024-04-25T10:16:54.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 -- 172.21.15.60:0/3762726061 wait complete. 2024-04-25T10:16:54.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 -- start start 2024-04-25T10:16:54.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 Processor -- start 2024-04-25T10:16:54.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6b9c5f30 0x559f6ba16c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6ba0f920 0x559f6ba171c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba10240 0x559f6b9c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6b7cf990 con 0x559f6ba10240 2024-04-25T10:16:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6b810e40 con 0x559f6b9c5f30 2024-04-25T10:16:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f6b7a0900 con 0x559f6ba0f920 2024-04-25T10:16:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 10 client.0 _ll_register_callbacks cb 0x559f6b8c81c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08b63af640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba10240 0x559f6b9c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08b63af640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba10240 0x559f6b9c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33416/0 (socket says 172.21.15.60:33416) 2024-04-25T10:16:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08b63af640 1 -- 172.21.15.60:0/3358381168 learned_addr learned my addr 172.21.15.60:0/3358381168 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08b5bae640 1 --2- 172.21.15.60:0/3358381168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6b9c5f30 0x559f6ba16c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08b53ad640 1 --2- 172.21.15.60:0/3358381168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6ba0f920 0x559f6ba171c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08b63af640 1 -- 172.21.15.60:0/3358381168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6ba0f920 msgr2=0x559f6ba171c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08b63af640 1 --2- 172.21.15.60:0/3358381168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6ba0f920 0x559f6ba171c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08b63af640 1 -- 172.21.15.60:0/3358381168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6b9c5f30 msgr2=0x559f6ba16c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08b63af640 1 --2- 172.21.15.60:0/3358381168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6b9c5f30 0x559f6ba16c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08b63af640 1 -- 172.21.15.60:0/3358381168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f6b7cd630 con 0x559f6ba10240 2024-04-25T10:16:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08b5bae640 1 --2- 172.21.15.60:0/3358381168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6b9c5f30 0x559f6ba16c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08b63af640 1 --2- 172.21.15.60:0/3358381168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba10240 0x559f6b9c9aa0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f08a80102f0 tx=0x7f08a80107b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ad1925bfe87572 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f0894ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:16:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f0894ff9640 1 -- 172.21.15.60:0/3358381168 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a800f040 con 0x559f6ba10240 2024-04-25T10:16:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f0894ff9640 1 -- 172.21.15.60:0/3358381168 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f08a8004230 con 0x559f6ba10240 2024-04-25T10:16:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 -- 172.21.15.60:0/3358381168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08a4058050 con 0x559f6ba10240 2024-04-25T10:16:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f0894ff9640 1 -- 172.21.15.60:0/3358381168 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08a8023050 con 0x559f6ba10240 2024-04-25T10:16:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 10 client.9987 fetch_fsmap learned FSMap version 106 2024-04-25T10:16:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 -- 172.21.15.60:0/3358381168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x559f6b79d3a0 con 0x559f6ba10240 2024-04-25T10:16:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f0894ff9640 1 -- 172.21.15.60:0/3358381168 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a4058050 con 0x559f6ba10240 2024-04-25T10:16:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f0894ff9640 1 -- 172.21.15.60:0/3358381168 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 106) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f08a8026020 con 0x559f6ba10240 2024-04-25T10:16:54.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 10 client.9987 fetch_fsmap finished waiting for FSMap version 106 2024-04-25T10:16:54.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 10 client.9987 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:16:54.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 20 client.9987 populate_metadata read hostname 'smithi060' 2024-04-25T10:16:54.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 --2- 172.21.15.60:0/3358381168 >> [v2:172.21.15.60:6834/3461876440,v1:172.21.15.60:6835/3461876440] conn(0x559f6b9cb160 0x559f6ba20ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 4 client.9987 mds_command: new command op to 9894 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:54.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08baf7f000 1 -- 172.21.15.60:0/3358381168 --> [v2:172.21.15.60:6834/3461876440,v1:172.21.15.60:6835/3461876440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f6b7b0a50 con 0x559f6b9cb160 2024-04-25T10:16:54.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08b5bae640 1 --2- 172.21.15.60:0/3358381168 >> [v2:172.21.15.60:6834/3461876440,v1:172.21.15.60:6835/3461876440] conn(0x559f6b9cb160 0x559f6ba20ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f0894ff9640 10 client.9987 ms_handle_connect on v2:172.21.15.60:6834/3461876440 2024-04-25T10:16:54.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.769+0000 7f08b5bae640 1 --2- 172.21.15.60:0/3358381168 >> [v2:172.21.15.60:6834/3461876440,v1:172.21.15.60:6835/3461876440] conn(0x559f6b9cb160 0x559f6ba20ce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb6b2301401c76f server_cookie=cc7aaa49e09c79a0 in_seq=0 out_seq=0 2024-04-25T10:16:54.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.773+0000 7f0894ff9640 1 -- 172.21.15.60:0/3358381168 <== mds.0 v2:172.21.15.60:6834/3461876440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559f6b7b0a50 con 0x559f6b9cb160 2024-04-25T10:16:54.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.773+0000 7f0894ff9640 10 client.9987 handle_command_reply: tid=0 2024-04-25T10:16:54.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.773+0000 7f08baf7f000 1 client.9987 shutdown 2024-04-25T10:16:54.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.773+0000 7f08baf7f000 1 -- 172.21.15.60:0/3358381168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba10240 msgr2=0x559f6b9c9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.773+0000 7f08baf7f000 1 --2- 172.21.15.60:0/3358381168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba10240 0x559f6b9c9aa0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f08a80102f0 tx=0x7f08a80107b0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.773+0000 7f08baf7f000 1 -- 172.21.15.60:0/3358381168 shutdown_connections 2024-04-25T10:16:54.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.773+0000 7f08baf7f000 1 --2- 172.21.15.60:0/3358381168 >> [v2:172.21.15.60:6834/3461876440,v1:172.21.15.60:6835/3461876440] conn(0x559f6b9cb160 0x559f6ba20ce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.773+0000 7f08baf7f000 1 --2- 172.21.15.60:0/3358381168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba10240 0x559f6b9c9aa0 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.773+0000 7f08baf7f000 1 --2- 172.21.15.60:0/3358381168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6ba0f920 0x559f6ba171c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.773+0000 7f08baf7f000 1 --2- 172.21.15.60:0/3358381168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6b9c5f30 0x559f6ba16c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.773+0000 7f0894ff9640 0 client.9987 ms_handle_reset on v2:172.21.15.60:6834/3461876440 2024-04-25T10:16:54.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.773+0000 7f08baf7f000 1 -- 172.21.15.60:0/3358381168 >> 172.21.15.60:0/3358381168 conn(0x559f6b9cc2a0 msgr2=0x559f6b9c49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.773+0000 7f08baf7f000 1 -- 172.21.15.60:0/3358381168 shutdown_connections 2024-04-25T10:16:54.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.773+0000 7f08baf7f000 1 -- 172.21.15.60:0/3358381168 wait complete. 2024-04-25T10:16:54.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.773+0000 7f08baf7f000 20 client.9987 trim_cache size 0 max 16384 2024-04-25T10:16:54.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 Processor -- start 2024-04-25T10:16:54.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 -- start start 2024-04-25T10:16:54.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6b9c5f30 0x559f6ba6b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ba0f920 0x559f6ba6b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6ba10240 0x559f6ba6be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b0202090 con 0x559f6b9c5f30 2024-04-25T10:16:54.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b00cd1b0 con 0x559f6ba0f920 2024-04-25T10:16:54.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b00c8900 con 0x559f6ba10240 2024-04-25T10:16:54.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08b5bae640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6b9c5f30 0x559f6ba6b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08b5bae640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6b9c5f30 0x559f6ba6b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33422/0 (socket says 172.21.15.60:33422) 2024-04-25T10:16:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08b5bae640 1 -- 172.21.15.60:0/405259098 learned_addr learned my addr 172.21.15.60:0/405259098 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08b5bae640 1 -- 172.21.15.60:0/405259098 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6ba10240 msgr2=0x559f6ba6be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08b5bae640 1 --2- 172.21.15.60:0/405259098 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6ba10240 0x559f6ba6be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08b5bae640 1 -- 172.21.15.60:0/405259098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ba0f920 msgr2=0x559f6ba6b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08b5bae640 1 --2- 172.21.15.60:0/405259098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ba0f920 0x559f6ba6b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08b5bae640 1 -- 172.21.15.60:0/405259098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f6ba10070 con 0x559f6b9c5f30 2024-04-25T10:16:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08b5bae640 1 --2- 172.21.15.60:0/405259098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6b9c5f30 0x559f6ba6b3a0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f08a00af040 tx=0x7f08a0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e7ac74903b3002 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f0896ffd640 1 -- 172.21.15.60:0/405259098 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a00a5c00 con 0x559f6b9c5f30 2024-04-25T10:16:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f0896ffd640 1 -- 172.21.15.60:0/405259098 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a00c8e50 con 0x559f6b9c5f30 2024-04-25T10:16:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f0896ffd640 1 -- 172.21.15.60:0/405259098 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a00c8150 con 0x559f6b9c5f30 2024-04-25T10:16:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 -- 172.21.15.60:0/405259098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6b9c5f30 msgr2=0x559f6ba6b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 --2- 172.21.15.60:0/405259098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6b9c5f30 0x559f6ba6b3a0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f08a00af040 tx=0x7f08a0004ac0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 -- 172.21.15.60:0/405259098 shutdown_connections 2024-04-25T10:16:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 --2- 172.21.15.60:0/405259098 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6ba10240 0x559f6ba6be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 --2- 172.21.15.60:0/405259098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6ba0f920 0x559f6ba6b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 --2- 172.21.15.60:0/405259098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6b9c5f30 0x559f6ba6b3a0 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 -- 172.21.15.60:0/405259098 >> 172.21.15.60:0/405259098 conn(0x559f6b9cc2a0 msgr2=0x559f6ba0d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 -- 172.21.15.60:0/405259098 shutdown_connections 2024-04-25T10:16:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 -- 172.21.15.60:0/405259098 wait complete. 2024-04-25T10:16:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 -- start start 2024-04-25T10:16:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 Processor -- start 2024-04-25T10:16:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6b9c5f30 0x559f6ba70320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba0f920 0x559f6ba70860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6ba10240 0x559f6ba70da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6b8193d0 con 0x559f6ba0f920 2024-04-25T10:16:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f6b8d0d50 con 0x559f6b9c5f30 2024-04-25T10:16:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.789+0000 7f08baf7f000 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f6b7af980 con 0x559f6ba10240 2024-04-25T10:16:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 10 client.0 _ll_register_callbacks cb 0x7f08a40520f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b53ad640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba0f920 0x559f6ba70860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b53ad640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba0f920 0x559f6ba70860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33434/0 (socket says 172.21.15.60:33434) 2024-04-25T10:16:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b53ad640 1 -- 172.21.15.60:0/1108768679 learned_addr learned my addr 172.21.15.60:0/1108768679 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b5bae640 1 --2- 172.21.15.60:0/1108768679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6b9c5f30 0x559f6ba70320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b63af640 1 --2- 172.21.15.60:0/1108768679 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6ba10240 0x559f6ba70da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b53ad640 1 -- 172.21.15.60:0/1108768679 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6ba10240 msgr2=0x559f6ba70da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b53ad640 1 --2- 172.21.15.60:0/1108768679 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6ba10240 0x559f6ba70da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b53ad640 1 -- 172.21.15.60:0/1108768679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6b9c5f30 msgr2=0x559f6ba70320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b53ad640 1 --2- 172.21.15.60:0/1108768679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6b9c5f30 0x559f6ba70320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b53ad640 1 -- 172.21.15.60:0/1108768679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f6b896600 con 0x559f6ba0f920 2024-04-25T10:16:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b5bae640 1 --2- 172.21.15.60:0/1108768679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6b9c5f30 0x559f6ba70320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b53ad640 1 --2- 172.21.15.60:0/1108768679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba0f920 0x559f6ba70860 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f08a4055b00 tx=0x7f08a4083940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce83c88e21bc075e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f0894ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T10:16:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f0894ff9640 1 -- 172.21.15.60:0/1108768679 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a406d040 con 0x559f6ba0f920 2024-04-25T10:16:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 1 -- 172.21.15.60:0/1108768679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08a00c8e50 con 0x559f6ba0f920 2024-04-25T10:16:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f0894ff9640 1 -- 172.21.15.60:0/1108768679 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f08a40524c0 con 0x559f6ba0f920 2024-04-25T10:16:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f0894ff9640 1 -- 172.21.15.60:0/1108768679 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08a405e560 con 0x559f6ba0f920 2024-04-25T10:16:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 10 client.9993 fetch_fsmap learned FSMap version 106 2024-04-25T10:16:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 1 -- 172.21.15.60:0/1108768679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x559f6b7a99b0 con 0x559f6ba0f920 2024-04-25T10:16:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f0894ff9640 1 -- 172.21.15.60:0/1108768679 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a00c8e50 con 0x559f6ba0f920 2024-04-25T10:16:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f0894ff9640 1 -- 172.21.15.60:0/1108768679 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 106) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f08a4013070 con 0x559f6ba0f920 2024-04-25T10:16:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 10 client.9993 fetch_fsmap finished waiting for FSMap version 106 2024-04-25T10:16:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 10 client.9993 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:16:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 20 client.9993 populate_metadata read hostname 'smithi060' 2024-04-25T10:16:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 1 --2- 172.21.15.60:0/1108768679 >> [v2:172.21.15.60:6834/3461876440,v1:172.21.15.60:6835/3461876440] conn(0x559f6ba6d180 0x559f6ba6d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 4 client.9993 mds_command: new command op to 9894 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:16:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 1 -- 172.21.15.60:0/1108768679 --> [v2:172.21.15.60:6834/3461876440,v1:172.21.15.60:6835/3461876440] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559f6b8222e0 con 0x559f6ba6d180 2024-04-25T10:16:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b5bae640 1 --2- 172.21.15.60:0/1108768679 >> [v2:172.21.15.60:6834/3461876440,v1:172.21.15.60:6835/3461876440] conn(0x559f6ba6d180 0x559f6ba6d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b5bae640 1 --2- 172.21.15.60:0/1108768679 >> [v2:172.21.15.60:6834/3461876440,v1:172.21.15.60:6835/3461876440] conn(0x559f6ba6d180 0x559f6ba6d560 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8169a7aff95b9784 server_cookie=f761dc2b15f77551 in_seq=0 out_seq=0 2024-04-25T10:16:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f0894ff9640 10 client.9993 ms_handle_connect on v2:172.21.15.60:6834/3461876440 2024-04-25T10:16:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f0894ff9640 1 -- 172.21.15.60:0/1108768679 <== mds.0 v2:172.21.15.60:6834/3461876440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x559f6b8222e0 con 0x559f6ba6d180 2024-04-25T10:16:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f0894ff9640 10 client.9993 handle_command_reply: tid=0 2024-04-25T10:16:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 1 client.9993 shutdown 2024-04-25T10:16:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 1 -- 172.21.15.60:0/1108768679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba0f920 msgr2=0x559f6ba70860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 1 --2- 172.21.15.60:0/1108768679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba0f920 0x559f6ba70860 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f08a4055b00 tx=0x7f08a4083940 comp rx=0 tx=0).stop 2024-04-25T10:16:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 1 -- 172.21.15.60:0/1108768679 shutdown_connections 2024-04-25T10:16:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 1 --2- 172.21.15.60:0/1108768679 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559f6ba10240 0x559f6ba70da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 1 --2- 172.21.15.60:0/1108768679 >> [v2:172.21.15.60:6834/3461876440,v1:172.21.15.60:6835/3461876440] conn(0x559f6ba6d180 0x559f6ba6d560 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 1 --2- 172.21.15.60:0/1108768679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559f6ba0f920 0x559f6ba70860 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 1 --2- 172.21.15.60:0/1108768679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f6b9c5f30 0x559f6ba70320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f0894ff9640 0 client.9993 ms_handle_reset on v2:172.21.15.60:6834/3461876440 2024-04-25T10:16:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 1 -- 172.21.15.60:0/1108768679 >> 172.21.15.60:0/1108768679 conn(0x559f6b9cc2a0 msgr2=0x559f6ba0e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 1 -- 172.21.15.60:0/1108768679 shutdown_connections 2024-04-25T10:16:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 1 -- 172.21.15.60:0/1108768679 wait complete. 2024-04-25T10:16:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08baf7f000 20 client.9993 trim_cache size 0 max 16384 2024-04-25T10:16:54.806 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:16:54.806 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "0ea93863-a079-45c4-abed-6e83540ea9fe", 2024-04-25T10:16:54.806 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T10:16:54.806 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9894, 2024-04-25T10:16:54.806 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T10:16:54.806 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T10:16:54.806 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T10:16:54.806 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 4.4710092890000004, 2024-04-25T10:16:54.806 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 106, 2024-04-25T10:16:54.807 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 112, 2024-04-25T10:16:54.807 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 112, 2024-04-25T10:16:54.807 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 6.4550325470000001 2024-04-25T10:16:54.807 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:16:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b7e12640 1 -- 172.21.15.60:0/2457167787 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f089c066f60 msgr2=0x7f089c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b7e12640 1 --2- 172.21.15.60:0/2457167787 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f089c066f60 0x7f089c069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f08a00ae000 tx=0x7f08a00a7070 comp rx=0 tx=0).stop 2024-04-25T10:16:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b7e12640 1 -- 172.21.15.60:0/2457167787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08b00d0340 msgr2=0x7f08b014ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b7e12640 1 --2- 172.21.15.60:0/2457167787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08b00d0340 0x7f08b014ce50 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f08a400d800 tx=0x7f08a400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b7e12640 1 -- 172.21.15.60:0/2457167787 shutdown_connections 2024-04-25T10:16:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b7e12640 1 --2- 172.21.15.60:0/2457167787 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f089c066f60 0x7f089c069420 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b7e12640 1 --2- 172.21.15.60:0/2457167787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08b015deb0 0x7f08b01608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b7e12640 1 --2- 172.21.15.60:0/2457167787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08b00d0340 0x7f08b014ce50 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b7e12640 1 --2- 172.21.15.60:0/2457167787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08b00ce2b0 0x7f08b00cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b7e12640 1 -- 172.21.15.60:0/2457167787 >> 172.21.15.60:0/2457167787 conn(0x7f08b006c090 msgr2=0x7f08b00b8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b7e12640 1 -- 172.21.15.60:0/2457167787 shutdown_connections 2024-04-25T10:16:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.793+0000 7f08b7e12640 1 -- 172.21.15.60:0/2457167787 wait complete. 2024-04-25T10:16:54.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:16:54.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.925+0000 7f3632dfd640 1 Processor -- start 2024-04-25T10:16:54.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.925+0000 7f3632dfd640 1 -- start start 2024-04-25T10:16:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f3632dfd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f362c0ce7a0 0x7f362c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f3632dfd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f362c0b4660 0x7f362c0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f3632dfd640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c0cf0c0 con 0x7f362c15bc90 2024-04-25T10:16:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f3632dfd640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362c0cc370 con 0x7f362c0ce7a0 2024-04-25T10:16:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f3632dfd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362c071ca0 con 0x7f362c0b4660 2024-04-25T10:16:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f3623fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f362c15bc90 0x7f362c0b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52658/0 (socket says 172.21.15.60:52658) 2024-04-25T10:16:54.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f3623fff640 1 -- 172.21.15.60:0/2383857664 learned_addr learned my addr 172.21.15.60:0/2383857664 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:54.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f3630b99640 1 --2- 172.21.15.60:0/2383857664 >> v2:172.21.15.60:3300/0 conn(0x7f362c0ce7a0 0x7f362c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f363139a640 1 --2- 172.21.15.60:0/2383857664 >> v2:172.21.15.100:3300/0 conn(0x7f362c0b4660 0x7f362c0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f36237fe640 1 -- 172.21.15.60:0/2383857664 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437234095 0 0) 0x7f362c0cf0c0 con 0x7f362c15bc90 2024-04-25T10:16:54.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f36237fe640 1 -- 172.21.15.60:0/2383857664 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3614003530 con 0x7f362c15bc90 2024-04-25T10:16:54.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f3630b99640 1 -- 172.21.15.60:0/2383857664 >> v2:172.21.15.100:3300/0 conn(0x7f362c0b4660 msgr2=0x7f362c0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f3630b99640 1 --2- 172.21.15.60:0/2383857664 >> v2:172.21.15.100:3300/0 conn(0x7f362c0b4660 0x7f362c0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f36237fe640 1 -- 172.21.15.60:0/2383857664 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3921588536 0 0) 0x7f3614003530 con 0x7f362c15bc90 2024-04-25T10:16:54.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f3630b99640 1 -- 172.21.15.60:0/2383857664 >> v1:172.21.15.60:6789/0 conn(0x7f362c15bc90 legacy=0x7f362c0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f3630b99640 1 -- 172.21.15.60:0/2383857664 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c0cca40 con 0x7f362c0ce7a0 2024-04-25T10:16:54.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f3630b99640 1 --2- 172.21.15.60:0/2383857664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362c0ce7a0 0x7f362c0ceb80 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f362c06cf50 tx=0x7f361c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4343c2471e369cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f36237fe640 1 -- 172.21.15.60:0/2383857664 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c0b0070 con 0x7f362c0ce7a0 2024-04-25T10:16:54.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f36237fe640 1 -- 172.21.15.60:0/2383857664 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c0ab400 con 0x7f362c0ce7a0 2024-04-25T10:16:54.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.929+0000 7f36237fe640 1 -- 172.21.15.60:0/2383857664 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628001680 con 0x7f362c0ce7a0 2024-04-25T10:16:54.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 -- 172.21.15.60:0/2383857664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362c0ce7a0 msgr2=0x7f362c0ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 --2- 172.21.15.60:0/2383857664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362c0ce7a0 0x7f362c0ceb80 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f362c06cf50 tx=0x7f361c01c3a0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 -- 172.21.15.60:0/2383857664 shutdown_connections 2024-04-25T10:16:54.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 --2- 172.21.15.60:0/2383857664 >> v2:172.21.15.100:3300/0 conn(0x7f362c0b4660 0x7f362c0b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 --2- 172.21.15.60:0/2383857664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362c0ce7a0 0x7f362c0ceb80 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 -- 172.21.15.60:0/2383857664 >> 172.21.15.60:0/2383857664 conn(0x7f362c0b12d0 msgr2=0x7f362c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 -- 172.21.15.60:0/2383857664 shutdown_connections 2024-04-25T10:16:54.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 -- 172.21.15.60:0/2383857664 wait complete. 2024-04-25T10:16:54.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 Processor -- start 2024-04-25T10:16:54.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 -- start start 2024-04-25T10:16:54.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f362c0ce7a0 0x7f362c14e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f362c15bc90 0x7f362c15ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362c0ace30 0x7f362c1544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0b4400 con 0x7f362c0ace30 2024-04-25T10:16:54.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0b4280 con 0x7f362c0ce7a0 2024-04-25T10:16:54.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0b3220 con 0x7f362c15bc90 2024-04-25T10:16:54.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f363139a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362c0ace30 0x7f362c1544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f363139a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362c0ace30 0x7f362c1544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33456/0 (socket says 172.21.15.60:33456) 2024-04-25T10:16:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f363139a640 1 -- 172.21.15.60:0/4213708068 learned_addr learned my addr 172.21.15.60:0/4213708068 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3623fff640 1 --2- 172.21.15.60:0/4213708068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f362c15bc90 0x7f362c15ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3630b99640 1 --2- 172.21.15.60:0/4213708068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f362c0ce7a0 0x7f362c14e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f363139a640 1 -- 172.21.15.60:0/4213708068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f362c15bc90 msgr2=0x7f362c15ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f363139a640 1 --2- 172.21.15.60:0/4213708068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f362c15bc90 0x7f362c15ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f363139a640 1 -- 172.21.15.60:0/4213708068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f362c0ce7a0 msgr2=0x7f362c14e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f363139a640 1 --2- 172.21.15.60:0/4213708068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f362c0ce7a0 0x7f362c14e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f363139a640 1 -- 172.21.15.60:0/4213708068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c14e740 con 0x7f362c0ace30 2024-04-25T10:16:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f363139a640 1 --2- 172.21.15.60:0/4213708068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362c0ace30 0x7f362c1544c0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f3624009dd0 tx=0x7f362400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f36217fa640 1 -- 172.21.15.60:0/4213708068 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3624018020 con 0x7f362c0ace30 2024-04-25T10:16:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f36217fa640 1 -- 172.21.15.60:0/4213708068 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3624014480 con 0x7f362c0ace30 2024-04-25T10:16:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 -- 172.21.15.60:0/4213708068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c159d70 con 0x7f362c0ace30 2024-04-25T10:16:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f36217fa640 1 -- 172.21.15.60:0/4213708068 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3624014870 con 0x7f362c0ace30 2024-04-25T10:16:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.933+0000 7f3632dfd640 1 -- 172.21.15.60:0/4213708068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c15a330 con 0x7f362c0ace30 2024-04-25T10:16:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.937+0000 7f3632dfd640 1 -- 172.21.15.60:0/4213708068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c074770 con 0x7f362c0ace30 2024-04-25T10:16:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.937+0000 7f36217fa640 1 -- 172.21.15.60:0/4213708068 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f362401b020 con 0x7f362c0ace30 2024-04-25T10:16:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.937+0000 7f36217fa640 1 --2- 172.21.15.60:0/4213708068 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3618066f90 0x7f3618069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.937+0000 7f36217fa640 1 -- 172.21.15.60:0/4213708068 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f3624085ed0 con 0x7f362c0ace30 2024-04-25T10:16:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.937+0000 7f3630b99640 1 --2- 172.21.15.60:0/4213708068 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3618066f90 0x7f3618069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.937+0000 7f3630b99640 1 --2- 172.21.15.60:0/4213708068 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3618066f90 0x7f3618069450 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f361c0ad000 tx=0x7f361c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:54.941+0000 7f36217fa640 1 -- 172.21.15.60:0/4213708068 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3624022030 con 0x7f362c0ace30 2024-04-25T10:16:55.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:55.133+0000 7f3632dfd640 1 -- 172.21.15.60:0/4213708068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f362c157c90 con 0x7f362c0ace30 2024-04-25T10:16:55.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:55.133+0000 7f36217fa640 1 -- 172.21.15.60:0/4213708068 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v106) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3624010040 con 0x7f362c0ace30 2024-04-25T10:16:55.138 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:16:55.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:55.133+0000 7f3632dfd640 1 -- 172.21.15.60:0/4213708068 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3618066f90 msgr2=0x7f3618069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:55.133+0000 7f3632dfd640 1 --2- 172.21.15.60:0/4213708068 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3618066f90 0x7f3618069450 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f361c0ad000 tx=0x7f361c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:55.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:55.133+0000 7f3632dfd640 1 -- 172.21.15.60:0/4213708068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362c0ace30 msgr2=0x7f362c1544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:55.133+0000 7f3632dfd640 1 --2- 172.21.15.60:0/4213708068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362c0ace30 0x7f362c1544c0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f3624009dd0 tx=0x7f362400c490 comp rx=0 tx=0).stop 2024-04-25T10:16:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:55.133+0000 7f3632dfd640 1 -- 172.21.15.60:0/4213708068 shutdown_connections 2024-04-25T10:16:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:55.133+0000 7f3632dfd640 1 --2- 172.21.15.60:0/4213708068 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3618066f90 0x7f3618069450 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:55.133+0000 7f3632dfd640 1 --2- 172.21.15.60:0/4213708068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362c0ace30 0x7f362c1544c0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:55.133+0000 7f3632dfd640 1 --2- 172.21.15.60:0/4213708068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f362c15bc90 0x7f362c15ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:55.133+0000 7f3632dfd640 1 --2- 172.21.15.60:0/4213708068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f362c0ce7a0 0x7f362c14e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:55.133+0000 7f3632dfd640 1 -- 172.21.15.60:0/4213708068 >> 172.21.15.60:0/4213708068 conn(0x7f362c0b12d0 msgr2=0x7f362c071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:55.133+0000 7f3632dfd640 1 -- 172.21.15.60:0/4213708068 shutdown_connections 2024-04-25T10:16:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:55.133+0000 7f3632dfd640 1 -- 172.21.15.60:0/4213708068 wait complete. 2024-04-25T10:16:55.156 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:16:55.156 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:16:55.156 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T10:16:55.156 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:16:55.156 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:55.156 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:16:55.156 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:16:55.156 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:16:55.156 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:55.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:55.156 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T10:16:55.162 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:16:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:16:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:16:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 41842sec preferred_lft 41842sec 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-04-25T10:16:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:55.164 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T10:16:55.164 INFO:teuthology.orchestra.run.smithi060.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:16:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:42:72:4c:68:9e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:16:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:55.164 INFO:teuthology.orchestra.run.smithi060.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:16:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:16:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:16:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:55.164 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:16:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:55.165 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:16:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:55.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:55.165 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:16:55.215 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:16:55.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:55.216 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T10:16:55.267 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T10:16:55.267 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:16:55.267 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:16:55.267 INFO:teuthology.orchestra.run.smithi060.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:16:55.268 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:16:55.268 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:16:55.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:55.268 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T10:16:55.268 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T10:16:55.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:16:55.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:16:55.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:16:55.268 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T10:16:55.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:55.343 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:16:55.394 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:16:55.394 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:16:55.395 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:55.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:55.395 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T10:16:55.451 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:16:55.451 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:55.451 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:55.498 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:55.498 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:55.498 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:55.546 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:16:55.605 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:16:55.657 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:16:55.657 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:16:55.657 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:16:55.657 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:16:55.657 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:16:55.657 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:16:55.657 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:16:55.657 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:16:55.657 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:16:55.658 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:16:55.658 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:16:55.658 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:16:55.658 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:16:55.658 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:16:55.658 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:16:55.658 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:16:55.658 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:16:55.658 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:16:55.658 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:16:55.658 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:16:55.658 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:16:55.658 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:16:55.658 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:16:55.659 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:16:55.659 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:16:55.659 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:16:55.659 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:16:55.659 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:16:55.659 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:16:55.659 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:55.659 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:55.659 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:55.659 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:55.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:55.660 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:16:55.706 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:16:55.707 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:16:55.751 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:16:55.767 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:16:55.815 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:16:55.815 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:16:55.815 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:16:55.815 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:16:55.815 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:16:55.815 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:16:55.815 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:16:55.816 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:16:55.816 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:16:55.816 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:16:55.816 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:16:55.816 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:16:55.816 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:16:55.816 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:16:55.816 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:16:55.816 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:16:55.816 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:16:55.816 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:16:55.816 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:16:55.816 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:16:55.816 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:16:55.817 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:16:55.817 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:16:55.817 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:16:55.817 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:16:55.817 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:16:55.817 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:16:55.817 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:16:55.817 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:16:55.817 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:55.817 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:55.817 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:55.817 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:55.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:55.818 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:16:55.834 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2024-04-25T10:16:55.829+0000 7effae3e9580 -1 init, newargv = 0x55b4ab2a20b0 newargc=15 2024-04-25T10:16:55.834 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[53653]: starting ceph client 2024-04-25T10:16:55.839 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[53653]: starting fuse 2024-04-25T10:16:56.939 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T10:16:56.957 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:16:57.008 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T10:16:57.008 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T10:16:57.008 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T10:16:57.008 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T10:16:57.008 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T10:16:57.008 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T10:16:57.009 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T10:16:57.009 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T10:16:57.009 INFO:teuthology.orchestra.run.smithi060.stdout:/run/lock 2024-04-25T10:16:57.009 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T10:16:57.009 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T10:16:57.009 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T10:16:57.009 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:16:57.009 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T10:16:57.009 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T10:16:57.009 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T10:16:57.009 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T10:16:57.009 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/fs/nfsd 2024-04-25T10:16:57.009 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T10:16:57.010 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T10:16:57.010 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:16:57.010 INFO:teuthology.orchestra.run.smithi060.stdout:/run/rpc_pipefs 2024-04-25T10:16:57.010 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T10:16:57.010 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T10:16:57.010 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:16:57.010 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:16:57.010 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:16:57.010 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:16:57.010 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T10:16:57.010 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:57.010 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:57.011 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:57.011 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:57.011 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:57.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:57.012 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T10:16:57.058 INFO:teuthology.orchestra.run.smithi060.stdout:45 2024-04-25T10:16:57.058 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:16:57.058 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:16:57.058 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T10:16:57.058 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T10:16:57.059 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:57.059 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T10:16:57.059 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:57.059 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T10:16:57.059 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:16:57.059 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T10:16:57.059 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:16:57.059 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:57.059 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T10:16:57.059 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T10:16:57.059 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T10:16:57.059 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T10:16:57.059 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:57.059 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T10:16:57.059 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:16:57.059 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:16:57.060 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:16:57.060 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T10:16:57.060 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T10:16:57.060 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T10:16:57.060 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:16:57.060 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:57.060 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2024-04-25T10:16:57.060 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:57.186 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.53653.asok 2024-04-25T10:16:57.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:57.198 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53653.asok 2024-04-25T10:16:57.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:57.199 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53653.asok status 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "53653", 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 10008, 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 10008 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2296178725 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2296178725 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.10008 192.168.0.1:0/2296178725", 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/2296178725", 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 106, 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 112, 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:16:57.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:57.369 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T10:16:57.369 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:57.375 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T10:16:57.375 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:57.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:57.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:57.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:57.432 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:16:57.498 INFO:teuthology.orchestra.run.smithi060.stdout:./ 2024-04-25T10:16:57.499 INFO:teuthology.orchestra.run.smithi060.stdout:./parent 2024-04-25T10:16:57.499 INFO:teuthology.orchestra.run.smithi060.stdout:./parent/flushed 2024-04-25T10:16:57.499 INFO:teuthology.orchestra.run.smithi060.stdout:./parent/flushed/charlie 2024-04-25T10:16:57.499 INFO:teuthology.orchestra.run.smithi060.stdout:./parent/flushed/alpha 2024-04-25T10:16:57.499 INFO:teuthology.orchestra.run.smithi060.stdout:./parent/flushed/bravo 2024-04-25T10:16:57.499 INFO:teuthology.orchestra.run.smithi060.stdout:./parent/unflushed 2024-04-25T10:16:57.499 INFO:teuthology.orchestra.run.smithi060.stdout:./parent/unflushed/jfile 2024-04-25T10:16:57.499 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:57.499 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:57.500 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:57.500 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:57.500 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:16:57.500 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:57.611 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T10:16:57.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:57.621 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:57.621 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:57.621 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:57.621 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:57.621 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:16:57.622 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:57.733 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627776 2024-04-25T10:16:57.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:57.744 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:57.744 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:57.744 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:57.744 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:57.744 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:16:57.744 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:57.845 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627777 2024-04-25T10:16:57.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:57.855 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:57.855 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:57.855 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:57.855 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:57.856 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:16:57.856 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:57.970 INFO:teuthology.orchestra.run.smithi060.stdout:1099511628280 2024-04-25T10:16:57.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:57.981 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:57.981 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:57.981 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:57.982 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:57.982 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:16:57.982 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:58.096 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627778 2024-04-25T10:16:58.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:58.107 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:58.107 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:58.107 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:58.107 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:58.108 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-25T10:16:58.108 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:58.221 INFO:teuthology.orchestra.run.smithi060.stdout:1099511628279 2024-04-25T10:16:58.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:58.232 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:58.232 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:58.232 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:58.232 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:58.232 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:16:58.233 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:58.337 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627779 2024-04-25T10:16:58.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:58.347 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T10:16:58.347 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T10:16:58.347 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T10:16:58.348 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T10:16:58.348 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:16:58.348 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T10:16:58.458 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627780 2024-04-25T10:16:58.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:58.469 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:16:58.469 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:16:58.558 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:16:58.623 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:16:58.623 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:16:58.623 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:58.630 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:58.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.801+0000 7f362e8c0640 1 Processor -- start 2024-04-25T10:16:58.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.801+0000 7f362e8c0640 1 -- start start 2024-04-25T10:16:58.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.801+0000 7f362e8c0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f362815deb0 0x7f36280b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.801+0000 7f362e8c0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f36280ce9c0 0x7f36280d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.801+0000 7f362e8c0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362806d200 con 0x7f36280d0340 2024-04-25T10:16:58.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.801+0000 7f362e8c0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36280cf2c0 con 0x7f362815deb0 2024-04-25T10:16:58.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.801+0000 7f362e8c0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36280cefe0 con 0x7f36280ce9c0 2024-04-25T10:16:58.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.801+0000 7f361ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f36280d0340 0x7f36280ce2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52662/0 (socket says 172.21.15.60:52662) 2024-04-25T10:16:58.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.801+0000 7f361ffff640 1 -- 172.21.15.60:0/2645689604 learned_addr learned my addr 172.21.15.60:0/2645689604 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:58.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.801+0000 7f3627fff640 1 --2- 172.21.15.60:0/2645689604 >> v2:172.21.15.60:3300/0 conn(0x7f362815deb0 0x7f36280b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.801+0000 7f362ce5d640 1 --2- 172.21.15.60:0/2645689604 >> v2:172.21.15.100:3300/0 conn(0x7f36280ce9c0 0x7f36280d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.801+0000 7f3627fff640 1 -- 172.21.15.60:0/2645689604 >> v2:172.21.15.100:3300/0 conn(0x7f36280ce9c0 msgr2=0x7f36280d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:58.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.801+0000 7f3627fff640 1 --2- 172.21.15.60:0/2645689604 >> v2:172.21.15.100:3300/0 conn(0x7f36280ce9c0 0x7f36280d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.801+0000 7f3627fff640 1 -- 172.21.15.60:0/2645689604 >> v1:172.21.15.60:6789/0 conn(0x7f36280d0340 legacy=0x7f36280ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.801+0000 7f3627fff640 1 -- 172.21.15.60:0/2645689604 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36280d1d70 con 0x7f362815deb0 2024-04-25T10:16:58.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f3627fff640 1 --2- 172.21.15.60:0/2645689604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362815deb0 0x7f36280b6c10 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f36280b9a00 tx=0x7f361801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6fc92d2a438616 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:58.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f36277fe640 1 -- 172.21.15.60:0/2645689604 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36180b0070 con 0x7f362815deb0 2024-04-25T10:16:58.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f36277fe640 1 -- 172.21.15.60:0/2645689604 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36180ab600 con 0x7f362815deb0 2024-04-25T10:16:58.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f36277fe640 1 -- 172.21.15.60:0/2645689604 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36180ab9d0 con 0x7f362815deb0 2024-04-25T10:16:58.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362e8c0640 1 -- 172.21.15.60:0/2645689604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362815deb0 msgr2=0x7f36280b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362e8c0640 1 --2- 172.21.15.60:0/2645689604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362815deb0 0x7f36280b6c10 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f36280b9a00 tx=0x7f361801c480 comp rx=0 tx=0).stop 2024-04-25T10:16:58.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362e8c0640 1 -- 172.21.15.60:0/2645689604 shutdown_connections 2024-04-25T10:16:58.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362e8c0640 1 --2- 172.21.15.60:0/2645689604 >> v2:172.21.15.100:3300/0 conn(0x7f36280ce9c0 0x7f36280d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362e8c0640 1 --2- 172.21.15.60:0/2645689604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362815deb0 0x7f36280b6c10 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362e8c0640 1 -- 172.21.15.60:0/2645689604 >> 172.21.15.60:0/2645689604 conn(0x7f362806c090 msgr2=0x7f362806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:58.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362e8c0640 1 -- 172.21.15.60:0/2645689604 shutdown_connections 2024-04-25T10:16:58.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362e8c0640 1 -- 172.21.15.60:0/2645689604 wait complete. 2024-04-25T10:16:58.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362e8c0640 1 Processor -- start 2024-04-25T10:16:58.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362e8c0640 1 -- start start 2024-04-25T10:16:58.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362e8c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36280ce9c0 0x7f362814a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362e8c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36280d0340 0x7f3628149330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362e8c0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362815deb0 0x7f3628147130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362e8c0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36280d3430 con 0x7f362815deb0 2024-04-25T10:16:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362e8c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36280d32b0 con 0x7f36280ce9c0 2024-04-25T10:16:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362e8c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36280d35b0 con 0x7f36280d0340 2024-04-25T10:16:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362ce5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362815deb0 0x7f3628147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362ce5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362815deb0 0x7f3628147130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33468/0 (socket says 172.21.15.60:33468) 2024-04-25T10:16:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362ce5d640 1 -- 172.21.15.60:0/219938090 learned_addr learned my addr 172.21.15.60:0/219938090 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f361ffff640 1 --2- 172.21.15.60:0/219938090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36280d0340 0x7f3628149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f3627fff640 1 --2- 172.21.15.60:0/219938090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36280ce9c0 0x7f362814a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362ce5d640 1 -- 172.21.15.60:0/219938090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36280d0340 msgr2=0x7f3628149330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362ce5d640 1 --2- 172.21.15.60:0/219938090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36280d0340 0x7f3628149330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362ce5d640 1 -- 172.21.15.60:0/219938090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36280ce9c0 msgr2=0x7f362814a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362ce5d640 1 --2- 172.21.15.60:0/219938090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36280ce9c0 0x7f362814a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362ce5d640 1 -- 172.21.15.60:0/219938090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3628149a60 con 0x7f362815deb0 2024-04-25T10:16:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f361ffff640 1 --2- 172.21.15.60:0/219938090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36280d0340 0x7f3628149330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f3627fff640 1 --2- 172.21.15.60:0/219938090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36280ce9c0 0x7f362814a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.805+0000 7f362ce5d640 1 --2- 172.21.15.60:0/219938090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362815deb0 0x7f3628147130 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f3620007e40 tx=0x7f362000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.809+0000 7f36257fa640 1 -- 172.21.15.60:0/219938090 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620014620 con 0x7f362815deb0 2024-04-25T10:16:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.809+0000 7f36257fa640 1 -- 172.21.15.60:0/219938090 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3620019070 con 0x7f362815deb0 2024-04-25T10:16:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.809+0000 7f362e8c0640 1 -- 172.21.15.60:0/219938090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3628149d40 con 0x7f362815deb0 2024-04-25T10:16:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.809+0000 7f36257fa640 1 -- 172.21.15.60:0/219938090 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620014a20 con 0x7f362815deb0 2024-04-25T10:16:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.809+0000 7f362e8c0640 1 -- 172.21.15.60:0/219938090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36280b71f0 con 0x7f362815deb0 2024-04-25T10:16:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.809+0000 7f362e8c0640 1 -- 172.21.15.60:0/219938090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3628074770 con 0x7f362815deb0 2024-04-25T10:16:58.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.809+0000 7f36257fa640 1 -- 172.21.15.60:0/219938090 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3620014bc0 con 0x7f362815deb0 2024-04-25T10:16:58.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.809+0000 7f36257fa640 1 --2- 172.21.15.60:0/219938090 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3608066f60 0x7f3608069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.809+0000 7f36257fa640 1 -- 172.21.15.60:0/219938090 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f3620085930 con 0x7f362815deb0 2024-04-25T10:16:58.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.809+0000 7f3627fff640 1 --2- 172.21.15.60:0/219938090 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3608066f60 0x7f3608069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.809+0000 7f3627fff640 1 --2- 172.21.15.60:0/219938090 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3608066f60 0x7f3608069420 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f36180ad000 tx=0x7f3618008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:58.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.813+0000 7f36257fa640 1 -- 172.21.15.60:0/219938090 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3620056920 con 0x7f362815deb0 2024-04-25T10:16:58.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.977+0000 7f362e8c0640 1 -- 172.21.15.60:0/219938090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36280d17a0 con 0x7f362815deb0 2024-04-25T10:16:58.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.977+0000 7f36257fa640 1 -- 172.21.15.60:0/219938090 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16894 (secure 0 0 0) 0x7f3620056220 con 0x7f362815deb0 2024-04-25T10:16:58.983 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:16:58.983 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":112,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:16:48.303329+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:06.033387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:06.753421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/178428701":"2024-04-26T10:16:48.303277+0000","172.21.15.60:6834/178428701":"2024-04-26T10:16:48.303277+0000","172.21.15.60:6835/722171246":"2024-04-26T10:16:28.355776+0000","172.21.15.60:6834/722171246":"2024-04-26T10:16:28.355776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.981+0000 7f362e8c0640 1 -- 172.21.15.60:0/219938090 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3608066f60 msgr2=0x7f3608069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.981+0000 7f362e8c0640 1 --2- 172.21.15.60:0/219938090 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3608066f60 0x7f3608069420 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f36180ad000 tx=0x7f3618008040 comp rx=0 tx=0).stop 2024-04-25T10:16:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.981+0000 7f362e8c0640 1 -- 172.21.15.60:0/219938090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362815deb0 msgr2=0x7f3628147130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.981+0000 7f362e8c0640 1 --2- 172.21.15.60:0/219938090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362815deb0 0x7f3628147130 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f3620007e40 tx=0x7f362000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.981+0000 7f362e8c0640 1 -- 172.21.15.60:0/219938090 shutdown_connections 2024-04-25T10:16:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.981+0000 7f362e8c0640 1 --2- 172.21.15.60:0/219938090 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3608066f60 0x7f3608069420 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.981+0000 7f362e8c0640 1 --2- 172.21.15.60:0/219938090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f362815deb0 0x7f3628147130 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.981+0000 7f362e8c0640 1 --2- 172.21.15.60:0/219938090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36280d0340 0x7f3628149330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.981+0000 7f362e8c0640 1 --2- 172.21.15.60:0/219938090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36280ce9c0 0x7f362814a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.981+0000 7f362e8c0640 1 -- 172.21.15.60:0/219938090 >> 172.21.15.60:0/219938090 conn(0x7f362806c090 msgr2=0x7f36280d0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.981+0000 7f362e8c0640 1 -- 172.21.15.60:0/219938090 shutdown_connections 2024-04-25T10:16:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:58.981+0000 7f362e8c0640 1 -- 172.21.15.60:0/219938090 wait complete. 2024-04-25T10:16:59.002 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:06.033387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:06.753421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:59.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:59.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.129+0000 7fd0a07d1640 1 Processor -- start 2024-04-25T10:16:59.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.129+0000 7fd0a07d1640 1 -- start start 2024-04-25T10:16:59.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd0a07d1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd09815dd80 0x7fd0980b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd0a07d1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd09815e160 con 0x7fd0980b5470 2024-04-25T10:16:59.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd0a07d1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0980b7060 con 0x7fd0980b2860 2024-04-25T10:16:59.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd0a07d1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0980d1b20 con 0x7fd09815dd80 2024-04-25T10:16:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09ed6e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd0980b5470 0x7fd0980d2bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57490/0 (socket says 172.21.15.60:57490) 2024-04-25T10:16:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09ed6e640 1 -- 172.21.15.60:0/3292422500 learned_addr learned my addr 172.21.15.60:0/3292422500 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09e56d640 1 --2- 172.21.15.60:0/3292422500 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd09815dd80 0x7fd0980b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09d56b640 1 -- 172.21.15.60:0/3292422500 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785994543 0 0) 0x7fd09815e160 con 0x7fd0980b5470 2024-04-25T10:16:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09d56b640 1 -- 172.21.15.60:0/3292422500 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd084003530 con 0x7fd0980b5470 2024-04-25T10:16:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09d56b640 1 -- 172.21.15.60:0/3292422500 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 277199363 0 0) 0x7fd084003530 con 0x7fd0980b5470 2024-04-25T10:16:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09d56b640 1 -- 172.21.15.60:0/3292422500 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd09815dd80 msgr2=0x7fd0980b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09d56b640 1 --2- 172.21.15.60:0/3292422500 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd09815dd80 0x7fd0980b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09d56b640 1 -- 172.21.15.60:0/3292422500 >> v1:172.21.15.100:6789/0 conn(0x7fd0980b2860 legacy=0x7fd0980b9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09d56b640 1 -- 172.21.15.60:0/3292422500 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0980d1e00 con 0x7fd0980b5470 2024-04-25T10:16:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09d56b640 1 -- 172.21.15.60:0/3292422500 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fd090001a00 con 0x7fd0980b5470 2024-04-25T10:16:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09d56b640 1 -- 172.21.15.60:0/3292422500 >> v1:172.21.15.60:6789/0 conn(0x7fd0980b5470 legacy=0x7fd0980d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09d56b640 1 --2- 172.21.15.60:0/3292422500 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd09815dd80 0x7fd0840039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09d56b640 1 --2- 172.21.15.60:0/3292422500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd084003f30 0x7fd084004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09d56b640 1 --2- 172.21.15.60:0/3292422500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd084004d50 0x7fd0840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09d56b640 1 -- 172.21.15.60:0/3292422500 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0980d1b20 con 0x7fd084004d50 2024-04-25T10:16:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09d56b640 1 -- 172.21.15.60:0/3292422500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08400ba30 con 0x7fd084003f30 2024-04-25T10:16:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09d56b640 1 -- 172.21.15.60:0/3292422500 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08400bc30 con 0x7fd09815dd80 2024-04-25T10:16:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09ed6e640 1 --2- 172.21.15.60:0/3292422500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd084004d50 0x7fd0840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09e56d640 1 --2- 172.21.15.60:0/3292422500 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd09815dd80 0x7fd0840039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09dd6c640 1 --2- 172.21.15.60:0/3292422500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd084003f30 0x7fd084004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09ed6e640 1 -- 172.21.15.60:0/3292422500 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd09815dd80 msgr2=0x7fd0840039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09ed6e640 1 --2- 172.21.15.60:0/3292422500 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd09815dd80 0x7fd0840039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09ed6e640 1 -- 172.21.15.60:0/3292422500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd084003f30 msgr2=0x7fd084004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09ed6e640 1 --2- 172.21.15.60:0/3292422500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd084003f30 0x7fd084004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.133+0000 7fd09ed6e640 1 -- 172.21.15.60:0/3292422500 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd084003530 con 0x7fd084004d50 2024-04-25T10:16:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd09ed6e640 1 --2- 172.21.15.60:0/3292422500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd084004d50 0x7fd0840051d0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fd09000a810 tx=0x7fd09001fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c60eb2ecc78d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd09d56b640 1 -- 172.21.15.60:0/3292422500 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09000a430 con 0x7fd084004d50 2024-04-25T10:16:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd09d56b640 1 -- 172.21.15.60:0/3292422500 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0900aa6b0 con 0x7fd084004d50 2024-04-25T10:16:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd09d56b640 1 -- 172.21.15.60:0/3292422500 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0900aaae0 con 0x7fd084004d50 2024-04-25T10:16:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd0a07d1640 1 -- 172.21.15.60:0/3292422500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd084004d50 msgr2=0x7fd0840051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd0a07d1640 1 --2- 172.21.15.60:0/3292422500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd084004d50 0x7fd0840051d0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fd09000a810 tx=0x7fd09001fa80 comp rx=0 tx=0).stop 2024-04-25T10:16:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd09ed6e640 1 -- 172.21.15.60:0/3292422500 reap_dead start 2024-04-25T10:16:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd0a07d1640 1 -- 172.21.15.60:0/3292422500 shutdown_connections 2024-04-25T10:16:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd0a07d1640 1 -- 172.21.15.60:0/3292422500 >> 172.21.15.60:0/3292422500 conn(0x7fd0980d0470 msgr2=0x7fd0980d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd0a07d1640 1 -- 172.21.15.60:0/3292422500 shutdown_connections 2024-04-25T10:16:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd0a07d1640 1 -- 172.21.15.60:0/3292422500 wait complete. 2024-04-25T10:16:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd0a07d1640 1 Processor -- start 2024-04-25T10:16:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd0a07d1640 1 -- start start 2024-04-25T10:16:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd0a07d1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd084004d50 0x7fd098142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd0a07d1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0981f6b10 0x7fd0980c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd0a07d1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0980ca1c0 0x7fd09815e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd0a07d1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0980d26a0 con 0x7fd084004d50 2024-04-25T10:16:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd0a07d1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0980d23c0 con 0x7fd0980ca1c0 2024-04-25T10:16:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd0a07d1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0980b6b20 con 0x7fd0981f6b10 2024-04-25T10:16:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd09e56d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd084004d50 0x7fd098142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd09e56d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd084004d50 0x7fd098142350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52900/0 (socket says 172.21.15.60:52900) 2024-04-25T10:16:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd09e56d640 1 -- 172.21.15.60:0/1343121173 learned_addr learned my addr 172.21.15.60:0/1343121173 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd09ed6e640 1 --2- 172.21.15.60:0/1343121173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0980ca1c0 0x7fd09815e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.137+0000 7fd09dd6c640 1 --2- 172.21.15.60:0/1343121173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0981f6b10 0x7fd0980c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd09e56d640 1 -- 172.21.15.60:0/1343121173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0981f6b10 msgr2=0x7fd0980c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd09e56d640 1 --2- 172.21.15.60:0/1343121173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0981f6b10 0x7fd0980c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd09e56d640 1 -- 172.21.15.60:0/1343121173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0980ca1c0 msgr2=0x7fd09815e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd09e56d640 1 --2- 172.21.15.60:0/1343121173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0980ca1c0 0x7fd09815e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd09e56d640 1 -- 172.21.15.60:0/1343121173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd098142890 con 0x7fd084004d50 2024-04-25T10:16:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd09e56d640 1 --2- 172.21.15.60:0/1343121173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd084004d50 0x7fd098142350 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fd094011640 tx=0x7fd094011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd08effd640 1 -- 172.21.15.60:0/1343121173 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09400f920 con 0x7fd084004d50 2024-04-25T10:16:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd08effd640 1 -- 172.21.15.60:0/1343121173 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09400fac0 con 0x7fd084004d50 2024-04-25T10:16:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd0a07d1640 1 -- 172.21.15.60:0/1343121173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0981f70d0 con 0x7fd084004d50 2024-04-25T10:16:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd08effd640 1 -- 172.21.15.60:0/1343121173 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09400fe70 con 0x7fd084004d50 2024-04-25T10:16:59.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd0a07d1640 1 -- 172.21.15.60:0/1343121173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0981f7690 con 0x7fd084004d50 2024-04-25T10:16:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd0a07d1640 1 -- 172.21.15.60:0/1343121173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd098074770 con 0x7fd084004d50 2024-04-25T10:16:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd08effd640 1 -- 172.21.15.60:0/1343121173 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd094015460 con 0x7fd084004d50 2024-04-25T10:16:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd08effd640 1 --2- 172.21.15.60:0/1343121173 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd07c066f90 0x7fd07c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd08effd640 1 -- 172.21.15.60:0/1343121173 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fd094085810 con 0x7fd084004d50 2024-04-25T10:16:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd09dd6c640 1 --2- 172.21.15.60:0/1343121173 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd07c066f90 0x7fd07c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.141+0000 7fd09dd6c640 1 --2- 172.21.15.60:0/1343121173 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd07c066f90 0x7fd07c069450 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd088007fd0 tx=0x7fd088007ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.145+0000 7fd08effd640 1 -- 172.21.15.60:0/1343121173 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd098142890 con 0x7fd084004d50 2024-04-25T10:16:59.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.305+0000 7fd0a07d1640 1 -- 172.21.15.60:0/1343121173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd0980b5100 con 0x7fd084004d50 2024-04-25T10:16:59.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.305+0000 7fd08effd640 1 -- 172.21.15.60:0/1343121173 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd094019030 con 0x7fd084004d50 2024-04-25T10:16:59.312 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:16:59.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.309+0000 7fd0a07d1640 1 -- 172.21.15.60:0/1343121173 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd07c066f90 msgr2=0x7fd07c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.309+0000 7fd0a07d1640 1 --2- 172.21.15.60:0/1343121173 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd07c066f90 0x7fd07c069450 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd088007fd0 tx=0x7fd088007ed0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.309+0000 7fd0a07d1640 1 -- 172.21.15.60:0/1343121173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd084004d50 msgr2=0x7fd098142350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.309+0000 7fd0a07d1640 1 --2- 172.21.15.60:0/1343121173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd084004d50 0x7fd098142350 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fd094011640 tx=0x7fd094011b00 comp rx=0 tx=0).stop 2024-04-25T10:16:59.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.309+0000 7fd0a07d1640 1 -- 172.21.15.60:0/1343121173 shutdown_connections 2024-04-25T10:16:59.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.309+0000 7fd0a07d1640 1 --2- 172.21.15.60:0/1343121173 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd07c066f90 0x7fd07c069450 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.309+0000 7fd0a07d1640 1 --2- 172.21.15.60:0/1343121173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0980ca1c0 0x7fd09815e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.309+0000 7fd0a07d1640 1 --2- 172.21.15.60:0/1343121173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0981f6b10 0x7fd0980c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.309+0000 7fd0a07d1640 1 --2- 172.21.15.60:0/1343121173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd084004d50 0x7fd098142350 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.309+0000 7fd0a07d1640 1 -- 172.21.15.60:0/1343121173 >> 172.21.15.60:0/1343121173 conn(0x7fd0980d0470 msgr2=0x7fd098152fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:59.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.309+0000 7fd0a07d1640 1 -- 172.21.15.60:0/1343121173 shutdown_connections 2024-04-25T10:16:59.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.309+0000 7fd0a07d1640 1 -- 172.21.15.60:0/1343121173 wait complete. 2024-04-25T10:16:59.335 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:16:59.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f165b31c640 1 Processor -- start 2024-04-25T10:16:59.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f165b31c640 1 -- start start 2024-04-25T10:16:59.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f165b31c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f165415dd30 0x7f165415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f165b31c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f16540cd3d0 0x7f16540cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f165b31c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16540cd890 con 0x7f165406f310 2024-04-25T10:16:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f165b31c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16540ce4a0 con 0x7f16540cd3d0 2024-04-25T10:16:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f165b31c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16540cdb70 con 0x7f165415dd30 2024-04-25T10:16:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f16598b9640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f165406f310 0x7f165406d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57494/0 (socket says 172.21.15.60:57494) 2024-04-25T10:16:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f16598b9640 1 -- 172.21.15.60:0/3048857009 learned_addr learned my addr 172.21.15.60:0/3048857009 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f16590b8640 1 --2- 172.21.15.60:0/3048857009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f165415dd30 0x7f165415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f16588b7640 1 --2- 172.21.15.60:0/3048857009 >> v2:172.21.15.100:3300/0 conn(0x7f16540cd3d0 0x7f16540cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f164bfff640 1 -- 172.21.15.60:0/3048857009 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2738937358 0 0) 0x7f16540cd890 con 0x7f165406f310 2024-04-25T10:16:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f164bfff640 1 -- 172.21.15.60:0/3048857009 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1640003530 con 0x7f165406f310 2024-04-25T10:16:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f164bfff640 1 -- 172.21.15.60:0/3048857009 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2049175900 0 0) 0x7f1640003530 con 0x7f165406f310 2024-04-25T10:16:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f164bfff640 1 -- 172.21.15.60:0/3048857009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f165415dd30 msgr2=0x7f165415ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f164bfff640 1 --2- 172.21.15.60:0/3048857009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f165415dd30 0x7f165415ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f164bfff640 1 -- 172.21.15.60:0/3048857009 >> v2:172.21.15.100:3300/0 conn(0x7f16540cd3d0 msgr2=0x7f16540cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f164bfff640 1 --2- 172.21.15.60:0/3048857009 >> v2:172.21.15.100:3300/0 conn(0x7f16540cd3d0 0x7f16540cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f164bfff640 1 -- 172.21.15.60:0/3048857009 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16540cde50 con 0x7f165406f310 2024-04-25T10:16:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f164bfff640 1 -- 172.21.15.60:0/3048857009 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f164c001a00 con 0x7f165406f310 2024-04-25T10:16:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f164bfff640 1 -- 172.21.15.60:0/3048857009 >> v1:172.21.15.60:6789/0 conn(0x7f165406f310 legacy=0x7f165406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f164bfff640 1 --2- 172.21.15.60:0/3048857009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16400039c0 0x7f1640003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f164bfff640 1 --2- 172.21.15.60:0/3048857009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1640004810 0x7f1640004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f164bfff640 1 --2- 172.21.15.60:0/3048857009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f165415dd30 0x7f16400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f164bfff640 1 -- 172.21.15.60:0/3048857009 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16540ce4a0 con 0x7f1640004810 2024-04-25T10:16:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f164bfff640 1 -- 172.21.15.60:0/3048857009 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16540cdb70 con 0x7f16400039c0 2024-04-25T10:16:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f164bfff640 1 -- 172.21.15.60:0/3048857009 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164000ba00 con 0x7f165415dd30 2024-04-25T10:16:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f16588b7640 1 --2- 172.21.15.60:0/3048857009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1640004810 0x7f1640004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f16590b8640 1 --2- 172.21.15.60:0/3048857009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16400039c0 0x7f1640003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f16598b9640 1 --2- 172.21.15.60:0/3048857009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f165415dd30 0x7f16400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f16588b7640 1 -- 172.21.15.60:0/3048857009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f165415dd30 msgr2=0x7f16400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f16588b7640 1 --2- 172.21.15.60:0/3048857009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f165415dd30 0x7f16400051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f16588b7640 1 -- 172.21.15.60:0/3048857009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16400039c0 msgr2=0x7f1640003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f16588b7640 1 --2- 172.21.15.60:0/3048857009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16400039c0 0x7f1640003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.461+0000 7f16588b7640 1 -- 172.21.15.60:0/3048857009 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1640003530 con 0x7f1640004810 2024-04-25T10:16:59.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.465+0000 7f16588b7640 1 --2- 172.21.15.60:0/3048857009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1640004810 0x7f1640004c90 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f1644009ae0 tx=0x7f164401ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8592d4f33f6c3bcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.465+0000 7f164bfff640 1 -- 172.21.15.60:0/3048857009 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16440b2070 con 0x7f1640004810 2024-04-25T10:16:59.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.465+0000 7f164bfff640 1 -- 172.21.15.60:0/3048857009 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16440a5590 con 0x7f1640004810 2024-04-25T10:16:59.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.465+0000 7f164bfff640 1 -- 172.21.15.60:0/3048857009 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16440096b0 con 0x7f1640004810 2024-04-25T10:16:59.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.465+0000 7f165b31c640 1 -- 172.21.15.60:0/3048857009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1640004810 msgr2=0x7f1640004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.465+0000 7f165b31c640 1 --2- 172.21.15.60:0/3048857009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1640004810 0x7f1640004c90 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f1644009ae0 tx=0x7f164401ca80 comp rx=0 tx=0).stop 2024-04-25T10:16:59.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.465+0000 7f16598b9640 1 -- 172.21.15.60:0/3048857009 reap_dead start 2024-04-25T10:16:59.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.465+0000 7f165b31c640 1 -- 172.21.15.60:0/3048857009 shutdown_connections 2024-04-25T10:16:59.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.465+0000 7f165b31c640 1 -- 172.21.15.60:0/3048857009 >> 172.21.15.60:0/3048857009 conn(0x7f16540670b0 msgr2=0x7f165406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:59.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.465+0000 7f165b31c640 1 -- 172.21.15.60:0/3048857009 shutdown_connections 2024-04-25T10:16:59.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.465+0000 7f165b31c640 1 -- 172.21.15.60:0/3048857009 wait complete. 2024-04-25T10:16:59.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f165b31c640 1 Processor -- start 2024-04-25T10:16:59.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f165b31c640 1 -- start start 2024-04-25T10:16:59.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f165b31c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1640004810 0x7f1654167e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f165b31c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1654156490 0x7f165415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f165b31c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1654163440 0x7f16541638c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f165b31c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16540cfa60 con 0x7f1654156490 2024-04-25T10:16:59.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f165b31c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16540cf8e0 con 0x7f1654163440 2024-04-25T10:16:59.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f165b31c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1654165290 con 0x7f1640004810 2024-04-25T10:16:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f16588b7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1654156490 0x7f165415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f16588b7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1654156490 0x7f165415a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52920/0 (socket says 172.21.15.60:52920) 2024-04-25T10:16:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f16588b7640 1 -- 172.21.15.60:0/3317360711 learned_addr learned my addr 172.21.15.60:0/3317360711 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f16588b7640 1 -- 172.21.15.60:0/3317360711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1640004810 msgr2=0x7f1654167e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f16590b8640 1 --2- 172.21.15.60:0/3317360711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1640004810 0x7f1654167e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f16598b9640 1 --2- 172.21.15.60:0/3317360711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1654163440 0x7f16541638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f16588b7640 1 --2- 172.21.15.60:0/3317360711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1640004810 0x7f1654167e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f16588b7640 1 -- 172.21.15.60:0/3317360711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1654163440 msgr2=0x7f16541638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f16588b7640 1 --2- 172.21.15.60:0/3317360711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1654163440 0x7f16541638c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f16588b7640 1 -- 172.21.15.60:0/3317360711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16540ac940 con 0x7f1654156490 2024-04-25T10:16:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f16588b7640 1 --2- 172.21.15.60:0/3317360711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1654156490 0x7f165415a070 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f16440b0040 tx=0x7f1640005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f1649ffb640 1 -- 172.21.15.60:0/3317360711 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16440bd020 con 0x7f1654156490 2024-04-25T10:16:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f1649ffb640 1 -- 172.21.15.60:0/3317360711 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16440a5d60 con 0x7f1654156490 2024-04-25T10:16:59.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f165b31c640 1 -- 172.21.15.60:0/3317360711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f165415a5b0 con 0x7f1654156490 2024-04-25T10:16:59.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f1649ffb640 1 -- 172.21.15.60:0/3317360711 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16440b2040 con 0x7f1654156490 2024-04-25T10:16:59.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f165b31c640 1 -- 172.21.15.60:0/3317360711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1654166580 con 0x7f1654156490 2024-04-25T10:16:59.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f165b31c640 1 -- 172.21.15.60:0/3317360711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1654163e00 con 0x7f1654156490 2024-04-25T10:16:59.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.469+0000 7f1649ffb640 1 -- 172.21.15.60:0/3317360711 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1644004260 con 0x7f1654156490 2024-04-25T10:16:59.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.473+0000 7f1649ffb640 1 --2- 172.21.15.60:0/3317360711 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1638066f60 0x7f1638069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.473+0000 7f1649ffb640 1 -- 172.21.15.60:0/3317360711 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f164411ed70 con 0x7f1654156490 2024-04-25T10:16:59.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.473+0000 7f16590b8640 1 --2- 172.21.15.60:0/3317360711 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1638066f60 0x7f1638069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.473+0000 7f16590b8640 1 --2- 172.21.15.60:0/3317360711 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1638066f60 0x7f1638069420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1650009940 tx=0x7f1650008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.477+0000 7f1649ffb640 1 -- 172.21.15.60:0/3317360711 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16440167c0 con 0x7f1654156490 2024-04-25T10:16:59.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.637+0000 7f165b31c640 1 -- 172.21.15.60:0/3317360711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1654074770 con 0x7f1654156490 2024-04-25T10:16:59.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.637+0000 7f1649ffb640 1 -- 172.21.15.60:0/3317360711 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f16440efcd0 con 0x7f1654156490 2024-04-25T10:16:59.642 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:16:59.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.637+0000 7f165b31c640 1 -- 172.21.15.60:0/3317360711 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1638066f60 msgr2=0x7f1638069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.637+0000 7f165b31c640 1 --2- 172.21.15.60:0/3317360711 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1638066f60 0x7f1638069420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1650009940 tx=0x7f1650008040 comp rx=0 tx=0).stop 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.637+0000 7f165b31c640 1 -- 172.21.15.60:0/3317360711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1654156490 msgr2=0x7f165415a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.637+0000 7f165b31c640 1 --2- 172.21.15.60:0/3317360711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1654156490 0x7f165415a070 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f16440b0040 tx=0x7f1640005840 comp rx=0 tx=0).stop 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.637+0000 7f165b31c640 1 -- 172.21.15.60:0/3317360711 shutdown_connections 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.637+0000 7f165b31c640 1 --2- 172.21.15.60:0/3317360711 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1638066f60 0x7f1638069420 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.637+0000 7f165b31c640 1 --2- 172.21.15.60:0/3317360711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1654163440 0x7f16541638c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.637+0000 7f165b31c640 1 --2- 172.21.15.60:0/3317360711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1654156490 0x7f165415a070 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.637+0000 7f165b31c640 1 --2- 172.21.15.60:0/3317360711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1640004810 0x7f1654167e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.637+0000 7f165b31c640 1 -- 172.21.15.60:0/3317360711 >> 172.21.15.60:0/3317360711 conn(0x7f16540670b0 msgr2=0x7f1654071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.637+0000 7f165b31c640 1 -- 172.21.15.60:0/3317360711 shutdown_connections 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.637+0000 7f165b31c640 1 -- 172.21.15.60:0/3317360711 wait complete. 2024-04-25T10:16:59.661 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:16:59.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb93921640 1 Processor -- start 2024-04-25T10:16:59.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb93921640 1 -- start start 2024-04-25T10:16:59.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb93921640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffb8c15dcd0 0x7ffb8c06c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb93921640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb8c0cbd60 0x7ffb8c0cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb93921640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c06d220 con 0x7ffb8c0b2860 2024-04-25T10:16:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb93921640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8c0cfa40 con 0x7ffb8c15dcd0 2024-04-25T10:16:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb93921640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0cfce0 con 0x7ffb8c0cbd60 2024-04-25T10:16:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb90ebc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ffb8c0b2860 0x7ffb8c0d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57506/0 (socket says 172.21.15.60:57506) 2024-04-25T10:16:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb90ebc640 1 -- 172.21.15.60:0/3883988852 learned_addr learned my addr 172.21.15.60:0/3883988852 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:16:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb91ebe640 1 --2- 172.21.15.60:0/3883988852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb8c0cbd60 0x7ffb8c0cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb916bd640 1 --2- 172.21.15.60:0/3883988852 >> v2:172.21.15.100:3300/0 conn(0x7ffb8c15dcd0 0x7ffb8c06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb7ffff640 1 -- 172.21.15.60:0/3883988852 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693974287 0 0) 0x7ffb8c06d220 con 0x7ffb8c0b2860 2024-04-25T10:16:59.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb7ffff640 1 -- 172.21.15.60:0/3883988852 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb78003530 con 0x7ffb8c0b2860 2024-04-25T10:16:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb7ffff640 1 -- 172.21.15.60:0/3883988852 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3079386501 0 0) 0x7ffb78003530 con 0x7ffb8c0b2860 2024-04-25T10:16:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb7ffff640 1 -- 172.21.15.60:0/3883988852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb8c0cbd60 msgr2=0x7ffb8c0cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb7ffff640 1 --2- 172.21.15.60:0/3883988852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb8c0cbd60 0x7ffb8c0cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb7ffff640 1 -- 172.21.15.60:0/3883988852 >> v2:172.21.15.100:3300/0 conn(0x7ffb8c15dcd0 msgr2=0x7ffb8c06c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb7ffff640 1 --2- 172.21.15.60:0/3883988852 >> v2:172.21.15.100:3300/0 conn(0x7ffb8c15dcd0 0x7ffb8c06c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb7ffff640 1 -- 172.21.15.60:0/3883988852 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0cffc0 con 0x7ffb8c0b2860 2024-04-25T10:16:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb7ffff640 1 -- 172.21.15.60:0/3883988852 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7ffb88001a00 con 0x7ffb8c0b2860 2024-04-25T10:16:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb7ffff640 1 -- 172.21.15.60:0/3883988852 >> v1:172.21.15.60:6789/0 conn(0x7ffb8c0b2860 legacy=0x7ffb8c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb7ffff640 1 --2- 172.21.15.60:0/3883988852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb8c0cbd60 0x7ffb780039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb7ffff640 1 --2- 172.21.15.60:0/3883988852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb78003f90 0x7ffb78004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb7ffff640 1 --2- 172.21.15.60:0/3883988852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb78004d50 0x7ffb780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb7ffff640 1 -- 172.21.15.60:0/3883988852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0cfa40 con 0x7ffb78003f90 2024-04-25T10:16:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb7ffff640 1 -- 172.21.15.60:0/3883988852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0cfce0 con 0x7ffb78004d50 2024-04-25T10:16:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb7ffff640 1 -- 172.21.15.60:0/3883988852 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7800ba80 con 0x7ffb8c0cbd60 2024-04-25T10:16:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb90ebc640 1 --2- 172.21.15.60:0/3883988852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb78003f90 0x7ffb78004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb916bd640 1 --2- 172.21.15.60:0/3883988852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb8c0cbd60 0x7ffb780039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb91ebe640 1 --2- 172.21.15.60:0/3883988852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb78004d50 0x7ffb780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb90ebc640 1 -- 172.21.15.60:0/3883988852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb8c0cbd60 msgr2=0x7ffb780039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb90ebc640 1 --2- 172.21.15.60:0/3883988852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb8c0cbd60 0x7ffb780039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb90ebc640 1 -- 172.21.15.60:0/3883988852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb78004d50 msgr2=0x7ffb780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb90ebc640 1 --2- 172.21.15.60:0/3883988852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb78004d50 0x7ffb780051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.781+0000 7ffb90ebc640 1 -- 172.21.15.60:0/3883988852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb78003530 con 0x7ffb78003f90 2024-04-25T10:16:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.785+0000 7ffb90ebc640 1 --2- 172.21.15.60:0/3883988852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb78003f90 0x7ffb78004810 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0b2cf0 tx=0x7ffb8801d270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628ef505a4b6516e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.785+0000 7ffb7ffff640 1 -- 172.21.15.60:0/3883988852 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8800a3f0 con 0x7ffb78003f90 2024-04-25T10:16:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.785+0000 7ffb7ffff640 1 -- 172.21.15.60:0/3883988852 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb880ad650 con 0x7ffb78003f90 2024-04-25T10:16:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.785+0000 7ffb7ffff640 1 -- 172.21.15.60:0/3883988852 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb880ada40 con 0x7ffb78003f90 2024-04-25T10:16:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.785+0000 7ffb93921640 1 -- 172.21.15.60:0/3883988852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb78003f90 msgr2=0x7ffb78004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.785+0000 7ffb93921640 1 --2- 172.21.15.60:0/3883988852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb78003f90 0x7ffb78004810 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0b2cf0 tx=0x7ffb8801d270 comp rx=0 tx=0).stop 2024-04-25T10:16:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.785+0000 7ffb91ebe640 1 -- 172.21.15.60:0/3883988852 reap_dead start 2024-04-25T10:16:59.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.785+0000 7ffb93921640 1 -- 172.21.15.60:0/3883988852 shutdown_connections 2024-04-25T10:16:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.785+0000 7ffb93921640 1 -- 172.21.15.60:0/3883988852 >> 172.21.15.60:0/3883988852 conn(0x7ffb8c0b3200 msgr2=0x7ffb8c0b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.785+0000 7ffb93921640 1 -- 172.21.15.60:0/3883988852 shutdown_connections 2024-04-25T10:16:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.785+0000 7ffb93921640 1 -- 172.21.15.60:0/3883988852 wait complete. 2024-04-25T10:16:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.785+0000 7ffb93921640 1 Processor -- start 2024-04-25T10:16:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.785+0000 7ffb93921640 1 -- start start 2024-04-25T10:16:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.785+0000 7ffb93921640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb8c0cd4e0 0x7ffb8c0b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.785+0000 7ffb93921640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb8c0b6630 0x7ffb8c148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.785+0000 7ffb93921640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb8c0cd8c0 0x7ffb8c148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb93921640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c06ce80 con 0x7ffb8c0b6630 2024-04-25T10:16:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb93921640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0d11b0 con 0x7ffb8c0cd8c0 2024-04-25T10:16:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb93921640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c06d000 con 0x7ffb8c0cd4e0 2024-04-25T10:16:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb90ebc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb8c0b6630 0x7ffb8c148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb90ebc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb8c0b6630 0x7ffb8c148710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52934/0 (socket says 172.21.15.60:52934) 2024-04-25T10:16:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb90ebc640 1 -- 172.21.15.60:0/3924938837 learned_addr learned my addr 172.21.15.60:0/3924938837 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:16:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb916bd640 1 --2- 172.21.15.60:0/3924938837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb8c0cd4e0 0x7ffb8c0b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb91ebe640 1 --2- 172.21.15.60:0/3924938837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb8c0cd8c0 0x7ffb8c148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb90ebc640 1 -- 172.21.15.60:0/3924938837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb8c0cd4e0 msgr2=0x7ffb8c0b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb90ebc640 1 --2- 172.21.15.60:0/3924938837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb8c0cd4e0 0x7ffb8c0b5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb90ebc640 1 -- 172.21.15.60:0/3924938837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb8c0cd8c0 msgr2=0x7ffb8c148c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb90ebc640 1 --2- 172.21.15.60:0/3924938837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb8c0cd8c0 0x7ffb8c148c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb90ebc640 1 -- 172.21.15.60:0/3924938837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0b6ba0 con 0x7ffb8c0b6630 2024-04-25T10:16:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb91ebe640 1 --2- 172.21.15.60:0/3924938837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb8c0cd8c0 0x7ffb8c148c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb90ebc640 1 --2- 172.21.15.60:0/3924938837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb8c0b6630 0x7ffb8c148710 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7ffb8801cce0 tx=0x7ffb78005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb7dffb640 1 -- 172.21.15.60:0/3924938837 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb880bb070 con 0x7ffb8c0b6630 2024-04-25T10:16:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb7dffb640 1 -- 172.21.15.60:0/3924938837 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb880017c0 con 0x7ffb8c0b6630 2024-04-25T10:16:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb7dffb640 1 -- 172.21.15.60:0/3924938837 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb880ad800 con 0x7ffb8c0b6630 2024-04-25T10:16:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb93921640 1 -- 172.21.15.60:0/3924938837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8c0c5170 con 0x7ffb8c0b6630 2024-04-25T10:16:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb93921640 1 -- 172.21.15.60:0/3924938837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8c0c5730 con 0x7ffb8c0b6630 2024-04-25T10:16:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb93921640 1 -- 172.21.15.60:0/3924938837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c0b4070 con 0x7ffb8c0b6630 2024-04-25T10:16:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb7dffb640 1 -- 172.21.15.60:0/3924938837 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb880be070 con 0x7ffb8c0b6630 2024-04-25T10:16:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb7dffb640 1 --2- 172.21.15.60:0/3924938837 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb70066f60 0x7ffb70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb7dffb640 1 -- 172.21.15.60:0/3924938837 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7ffb8811f2c0 con 0x7ffb8c0b6630 2024-04-25T10:16:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb916bd640 1 --2- 172.21.15.60:0/3924938837 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb70066f60 0x7ffb70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.789+0000 7ffb916bd640 1 --2- 172.21.15.60:0/3924938837 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb70066f60 0x7ffb70069420 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009ab0 tx=0x7ffb84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.793+0000 7ffb7dffb640 1 -- 172.21.15.60:0/3924938837 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb88016fb0 con 0x7ffb8c0b6630 2024-04-25T10:16:59.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.953+0000 7ffb93921640 1 -- 172.21.15.60:0/3924938837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb8c06e410 con 0x7ffb8c0b6630 2024-04-25T10:16:59.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.957+0000 7ffb7dffb640 1 -- 172.21.15.60:0/3924938837 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb880efc60 con 0x7ffb8c0b6630 2024-04-25T10:16:59.961 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:16:59.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.957+0000 7ffb93921640 1 -- 172.21.15.60:0/3924938837 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb70066f60 msgr2=0x7ffb70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.957+0000 7ffb93921640 1 --2- 172.21.15.60:0/3924938837 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb70066f60 0x7ffb70069420 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009ab0 tx=0x7ffb84008040 comp rx=0 tx=0).stop 2024-04-25T10:16:59.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.957+0000 7ffb93921640 1 -- 172.21.15.60:0/3924938837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb8c0b6630 msgr2=0x7ffb8c148710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.957+0000 7ffb93921640 1 --2- 172.21.15.60:0/3924938837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb8c0b6630 0x7ffb8c148710 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7ffb8801cce0 tx=0x7ffb78005840 comp rx=0 tx=0).stop 2024-04-25T10:16:59.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.957+0000 7ffb93921640 1 -- 172.21.15.60:0/3924938837 shutdown_connections 2024-04-25T10:16:59.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.957+0000 7ffb93921640 1 --2- 172.21.15.60:0/3924938837 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ffb70066f60 0x7ffb70069420 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.957+0000 7ffb93921640 1 --2- 172.21.15.60:0/3924938837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb8c0cd8c0 0x7ffb8c148c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.957+0000 7ffb93921640 1 --2- 172.21.15.60:0/3924938837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb8c0b6630 0x7ffb8c148710 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.957+0000 7ffb93921640 1 --2- 172.21.15.60:0/3924938837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb8c0cd4e0 0x7ffb8c0b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.957+0000 7ffb93921640 1 -- 172.21.15.60:0/3924938837 >> 172.21.15.60:0/3924938837 conn(0x7ffb8c0b3200 msgr2=0x7ffb8c0d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:59.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.957+0000 7ffb93921640 1 -- 172.21.15.60:0/3924938837 shutdown_connections 2024-04-25T10:16:59.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:16:59.957+0000 7ffb93921640 1 -- 172.21.15.60:0/3924938837 wait complete. 2024-04-25T10:16:59.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:17:00.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.097+0000 7fddc5789640 1 Processor -- start 2024-04-25T10:17:00.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.097+0000 7fddc5789640 1 -- start start 2024-04-25T10:17:00.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.097+0000 7fddc5789640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fddc00afac0 0x7fddc00cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.097+0000 7fddc5789640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fddc00cf010 0x7fddc0072750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.097+0000 7fddc5789640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc015e7f0 con 0x7fddc015aa30 2024-04-25T10:17:00.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.097+0000 7fddc5789640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc015ead0 con 0x7fddc00cf010 2024-04-25T10:17:00.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.097+0000 7fddc5789640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc015edb0 con 0x7fddc00afac0 2024-04-25T10:17:00.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.097+0000 7fddbf7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fddc00cf010 0x7fddc0072750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.097+0000 7fddbeffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fddc015aa30 0x7fddc00b3c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57510/0 (socket says 172.21.15.60:57510) 2024-04-25T10:17:00.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.097+0000 7fddbeffd640 1 -- 172.21.15.60:0/2343776565 learned_addr learned my addr 172.21.15.60:0/2343776565 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:00.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddb7fff640 1 --2- 172.21.15.60:0/2343776565 >> v2:172.21.15.100:3300/0 conn(0x7fddc00afac0 0x7fddc00cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddbf7fe640 1 -- 172.21.15.60:0/2343776565 >> v2:172.21.15.100:3300/0 conn(0x7fddc00afac0 msgr2=0x7fddc00cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:00.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddbf7fe640 1 --2- 172.21.15.60:0/2343776565 >> v2:172.21.15.100:3300/0 conn(0x7fddc00afac0 0x7fddc00cead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddbf7fe640 1 -- 172.21.15.60:0/2343776565 >> v1:172.21.15.60:6789/0 conn(0x7fddc015aa30 legacy=0x7fddc00b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddbf7fe640 1 -- 172.21.15.60:0/2343776565 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc015f090 con 0x7fddc00cf010 2024-04-25T10:17:00.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddbe7fc640 1 -- 172.21.15.60:0/2343776565 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1607744889 0 0) 0x7fddc015e7f0 con 0x7fddc015aa30 2024-04-25T10:17:00.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddbf7fe640 1 --2- 172.21.15.60:0/2343776565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc00cf010 0x7fddc0072750 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fddc006d0a0 tx=0x7fddb801c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e48854e3d2fd009 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:00.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddbe7fc640 1 -- 172.21.15.60:0/2343776565 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb80a66e0 con 0x7fddc00cf010 2024-04-25T10:17:00.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddbe7fc640 1 -- 172.21.15.60:0/2343776565 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb80a6880 con 0x7fddc00cf010 2024-04-25T10:17:00.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddbe7fc640 1 -- 172.21.15.60:0/2343776565 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb80a6c50 con 0x7fddc00cf010 2024-04-25T10:17:00.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddc5789640 1 -- 172.21.15.60:0/2343776565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc00cf010 msgr2=0x7fddc0072750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddc5789640 1 --2- 172.21.15.60:0/2343776565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc00cf010 0x7fddc0072750 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fddc006d0a0 tx=0x7fddb801c810 comp rx=0 tx=0).stop 2024-04-25T10:17:00.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddc5789640 1 -- 172.21.15.60:0/2343776565 shutdown_connections 2024-04-25T10:17:00.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddc5789640 1 --2- 172.21.15.60:0/2343776565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc00cf010 0x7fddc0072750 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddc5789640 1 --2- 172.21.15.60:0/2343776565 >> v2:172.21.15.100:3300/0 conn(0x7fddc00afac0 0x7fddc00cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddc5789640 1 -- 172.21.15.60:0/2343776565 >> 172.21.15.60:0/2343776565 conn(0x7fddc00b2360 msgr2=0x7fddc00b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:00.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddc5789640 1 -- 172.21.15.60:0/2343776565 shutdown_connections 2024-04-25T10:17:00.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.101+0000 7fddc5789640 1 -- 172.21.15.60:0/2343776565 wait complete. 2024-04-25T10:17:00.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddc5789640 1 Processor -- start 2024-04-25T10:17:00.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddc5789640 1 -- start start 2024-04-25T10:17:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddc5789640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc00afac0 0x7fddc014eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddc5789640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc00cf010 0x7fddc014dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddc5789640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddc015aa30 0x7fddc00b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddc5789640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc0167910 con 0x7fddc00cf010 2024-04-25T10:17:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddc5789640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc015e5b0 con 0x7fddc00afac0 2024-04-25T10:17:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddc5789640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc0167a90 con 0x7fddc015aa30 2024-04-25T10:17:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddb7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc00cf010 0x7fddc014dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddb7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc00cf010 0x7fddc014dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52954/0 (socket says 172.21.15.60:52954) 2024-04-25T10:17:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddb7fff640 1 -- 172.21.15.60:0/463670519 learned_addr learned my addr 172.21.15.60:0/463670519 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddbf7fe640 1 --2- 172.21.15.60:0/463670519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddc015aa30 0x7fddc00b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddb7fff640 1 -- 172.21.15.60:0/463670519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddc015aa30 msgr2=0x7fddc00b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddbeffd640 1 --2- 172.21.15.60:0/463670519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc00afac0 0x7fddc014eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddb7fff640 1 --2- 172.21.15.60:0/463670519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddc015aa30 0x7fddc00b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddb7fff640 1 -- 172.21.15.60:0/463670519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc00afac0 msgr2=0x7fddc014eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddb7fff640 1 --2- 172.21.15.60:0/463670519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc00afac0 0x7fddc014eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddb7fff640 1 -- 172.21.15.60:0/463670519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc00b1570 con 0x7fddc00cf010 2024-04-25T10:17:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddbf7fe640 1 --2- 172.21.15.60:0/463670519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddc015aa30 0x7fddc00b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddbeffd640 1 --2- 172.21.15.60:0/463670519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc00afac0 0x7fddc014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddb7fff640 1 --2- 172.21.15.60:0/463670519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc00cf010 0x7fddc014dd20 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fddb000d620 tx=0x7fddb000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddb77fe640 1 -- 172.21.15.60:0/463670519 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb000b620 con 0x7fddc00cf010 2024-04-25T10:17:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddb77fe640 1 -- 172.21.15.60:0/463670519 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb0019070 con 0x7fddc00cf010 2024-04-25T10:17:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddc5789640 1 -- 172.21.15.60:0/463670519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc01567f0 con 0x7fddc00cf010 2024-04-25T10:17:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddb77fe640 1 -- 172.21.15.60:0/463670519 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb000b9e0 con 0x7fddc00cf010 2024-04-25T10:17:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddc5789640 1 -- 172.21.15.60:0/463670519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc014e5e0 con 0x7fddc00cf010 2024-04-25T10:17:00.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddc5789640 1 -- 172.21.15.60:0/463670519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc006ccb0 con 0x7fddc00cf010 2024-04-25T10:17:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.105+0000 7fddb77fe640 1 -- 172.21.15.60:0/463670519 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fddb000bb80 con 0x7fddc00cf010 2024-04-25T10:17:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.109+0000 7fddb77fe640 1 --2- 172.21.15.60:0/463670519 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdda0066f90 0x7fdda0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.109+0000 7fddb77fe640 1 -- 172.21.15.60:0/463670519 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fddb00854d0 con 0x7fddc00cf010 2024-04-25T10:17:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.109+0000 7fddbeffd640 1 --2- 172.21.15.60:0/463670519 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdda0066f90 0x7fdda0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.109+0000 7fddbeffd640 1 --2- 172.21.15.60:0/463670519 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdda0066f90 0x7fdda0069450 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdda8000f70 tx=0x7fdda8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:00.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.113+0000 7fddb77fe640 1 -- 172.21.15.60:0/463670519 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fddb0056410 con 0x7fddc00cf010 2024-04-25T10:17:00.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.281+0000 7fddc5789640 1 -- 172.21.15.60:0/463670519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fddc0074770 con 0x7fddc00cf010 2024-04-25T10:17:00.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.285+0000 7fddb77fe640 1 -- 172.21.15.60:0/463670519 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v112) v1 ==== 80+0+240 (secure 0 0 0) 0x7fddb0055d10 con 0x7fddc00cf010 2024-04-25T10:17:00.290 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6835/178428701 2024-04-26T10:16:48.303277+0000 2024-04-25T10:17:00.290 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6834/178428701 2024-04-26T10:16:48.303277+0000 2024-04-25T10:17:00.290 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6835/722171246 2024-04-26T10:16:28.355776+0000 2024-04-25T10:17:00.290 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6834/722171246 2024-04-26T10:16:28.355776+0000 2024-04-25T10:17:00.290 INFO:teuthology.orchestra.run.smithi060.stderr:listed 4 entries 2024-04-25T10:17:00.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.289+0000 7fddc5789640 1 -- 172.21.15.60:0/463670519 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdda0066f90 msgr2=0x7fdda0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.289+0000 7fddc5789640 1 --2- 172.21.15.60:0/463670519 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdda0066f90 0x7fdda0069450 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdda8000f70 tx=0x7fdda8009040 comp rx=0 tx=0).stop 2024-04-25T10:17:00.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.289+0000 7fddc5789640 1 -- 172.21.15.60:0/463670519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc00cf010 msgr2=0x7fddc014dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.289+0000 7fddc5789640 1 --2- 172.21.15.60:0/463670519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc00cf010 0x7fddc014dd20 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fddb000d620 tx=0x7fddb000dae0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.289+0000 7fddc5789640 1 -- 172.21.15.60:0/463670519 shutdown_connections 2024-04-25T10:17:00.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.289+0000 7fddc5789640 1 --2- 172.21.15.60:0/463670519 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fdda0066f90 0x7fdda0069450 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.289+0000 7fddc5789640 1 --2- 172.21.15.60:0/463670519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddc015aa30 0x7fddc00b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.289+0000 7fddc5789640 1 --2- 172.21.15.60:0/463670519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc00cf010 0x7fddc014dd20 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.289+0000 7fddc5789640 1 --2- 172.21.15.60:0/463670519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc00afac0 0x7fddc014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.289+0000 7fddc5789640 1 -- 172.21.15.60:0/463670519 >> 172.21.15.60:0/463670519 conn(0x7fddc00b2360 msgr2=0x7fddc00b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:00.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.289+0000 7fddc5789640 1 -- 172.21.15.60:0/463670519 shutdown_connections 2024-04-25T10:17:00.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.289+0000 7fddc5789640 1 -- 172.21.15.60:0/463670519 wait complete. 2024-04-25T10:17:00.310 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T10:17:00.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:00.311 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:00.320 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[53653]: fuse finished with error 0 and tester_r 0 2024-04-25T10:17:00.321 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:17:00.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:00.322 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:17:00.426 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:17:00.426 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:00.434 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:00.434 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:17:00.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:00.434 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:17:00.482 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:17:00.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:17:00.483 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:17:00.483 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:00.489 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:17:00.490 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:17:00.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:00.490 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:17:00.537 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:17:00.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:17:00.539 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:17:00.539 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:00.545 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:00.545 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:17:00.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:00.545 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:17:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:17:00.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:17:00.595 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:00.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585ead1640 1 Processor -- start 2024-04-25T10:17:00.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585ead1640 1 -- start start 2024-04-25T10:17:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585ead1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f585815f480 0x7f585815f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585ead1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58580cd390 0x7f585809a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585ead1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f585809a930 0x7f58580cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585ead1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f585815fda0 con 0x7f585809a930 2024-04-25T10:17:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585ead1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58580cc5d0 con 0x7f585815f480 2024-04-25T10:17:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585ead1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58580cc8b0 con 0x7f58580cd390 2024-04-25T10:17:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585d06e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f585809a930 0x7f58580cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585d06e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f585809a930 0x7f58580cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52968/0 (socket says 172.21.15.60:52968) 2024-04-25T10:17:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585d06e640 1 -- 172.21.15.60:0/1717357862 learned_addr learned my addr 172.21.15.60:0/1717357862 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f5857fff640 1 --2- 172.21.15.60:0/1717357862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58580cd390 0x7f585809a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585d06e640 1 -- 172.21.15.60:0/1717357862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58580cd390 msgr2=0x7f585809a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585d06e640 1 --2- 172.21.15.60:0/1717357862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58580cd390 0x7f585809a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585c86d640 1 --2- 172.21.15.60:0/1717357862 >> v2:172.21.15.100:3300/0 conn(0x7f585815f480 0x7f585815f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585d06e640 1 -- 172.21.15.60:0/1717357862 >> v2:172.21.15.100:3300/0 conn(0x7f585815f480 msgr2=0x7f585815f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585d06e640 1 --2- 172.21.15.60:0/1717357862 >> v2:172.21.15.100:3300/0 conn(0x7f585815f480 0x7f585815f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585d06e640 1 -- 172.21.15.60:0/1717357862 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58580ccb90 con 0x7f585809a930 2024-04-25T10:17:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f5857fff640 1 --2- 172.21.15.60:0/1717357862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58580cd390 0x7f585809a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.721+0000 7f585c86d640 1 --2- 172.21.15.60:0/1717357862 >> v2:172.21.15.100:3300/0 conn(0x7f585815f480 0x7f585815f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.725+0000 7f585d06e640 1 --2- 172.21.15.60:0/1717357862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f585809a930 0x7f58580cf380 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f585806b860 tx=0x7f585001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f2b19781098902 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.725+0000 7f58577fe640 1 -- 172.21.15.60:0/1717357862 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58500a5860 con 0x7f585809a930 2024-04-25T10:17:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.725+0000 7f58577fe640 1 -- 172.21.15.60:0/1717357862 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58500a5a00 con 0x7f585809a930 2024-04-25T10:17:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.725+0000 7f58577fe640 1 -- 172.21.15.60:0/1717357862 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58500a5dd0 con 0x7f585809a930 2024-04-25T10:17:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.725+0000 7f585ead1640 1 -- 172.21.15.60:0/1717357862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f585809a930 msgr2=0x7f58580cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.725+0000 7f585ead1640 1 --2- 172.21.15.60:0/1717357862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f585809a930 0x7f58580cf380 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f585806b860 tx=0x7f585001c9c0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.725+0000 7f585ead1640 1 -- 172.21.15.60:0/1717357862 shutdown_connections 2024-04-25T10:17:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.725+0000 7f585ead1640 1 --2- 172.21.15.60:0/1717357862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f585809a930 0x7f58580cf380 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.725+0000 7f585ead1640 1 --2- 172.21.15.60:0/1717357862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58580cd390 0x7f585809a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.725+0000 7f585ead1640 1 --2- 172.21.15.60:0/1717357862 >> v2:172.21.15.100:3300/0 conn(0x7f585815f480 0x7f585815f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.725+0000 7f585ead1640 1 -- 172.21.15.60:0/1717357862 >> 172.21.15.60:0/1717357862 conn(0x7f58580b32a0 msgr2=0x7f58580b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.725+0000 7f585ead1640 1 -- 172.21.15.60:0/1717357862 shutdown_connections 2024-04-25T10:17:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.725+0000 7f585ead1640 1 -- 172.21.15.60:0/1717357862 wait complete. 2024-04-25T10:17:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.725+0000 7f585ead1640 1 Processor -- start 2024-04-25T10:17:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.725+0000 7f585ead1640 1 -- start start 2024-04-25T10:17:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585ead1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f585809a930 0x7f58580b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585ead1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58580cd390 0x7f5858156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585ead1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f585815f480 0x7f585815e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585ead1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585815bd30 con 0x7f585809a930 2024-04-25T10:17:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585ead1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585815bbb0 con 0x7f585815f480 2024-04-25T10:17:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585ead1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585815beb0 con 0x7f58580cd390 2024-04-25T10:17:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585c86d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f585809a930 0x7f58580b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585c86d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f585809a930 0x7f58580b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52984/0 (socket says 172.21.15.60:52984) 2024-04-25T10:17:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585c86d640 1 -- 172.21.15.60:0/229891306 learned_addr learned my addr 172.21.15.60:0/229891306 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f5857fff640 1 --2- 172.21.15.60:0/229891306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58580cd390 0x7f5858156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585d06e640 1 --2- 172.21.15.60:0/229891306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f585815f480 0x7f585815e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585c86d640 1 -- 172.21.15.60:0/229891306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58580cd390 msgr2=0x7f5858156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585c86d640 1 --2- 172.21.15.60:0/229891306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58580cd390 0x7f5858156270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585c86d640 1 -- 172.21.15.60:0/229891306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f585815f480 msgr2=0x7f585815e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585c86d640 1 --2- 172.21.15.60:0/229891306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f585815f480 0x7f585815e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585c86d640 1 -- 172.21.15.60:0/229891306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58580caee0 con 0x7f585809a930 2024-04-25T10:17:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585c86d640 1 --2- 172.21.15.60:0/229891306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f585809a930 0x7f58580b0130 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f584800c820 tx=0x7f584800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f58557fa640 1 -- 172.21.15.60:0/229891306 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848015070 con 0x7f585809a930 2024-04-25T10:17:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f58557fa640 1 -- 172.21.15.60:0/229891306 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5848007a30 con 0x7f585809a930 2024-04-25T10:17:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585ead1640 1 -- 172.21.15.60:0/229891306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58580cb1c0 con 0x7f585809a930 2024-04-25T10:17:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f58557fa640 1 -- 172.21.15.60:0/229891306 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848007e40 con 0x7f585809a930 2024-04-25T10:17:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585ead1640 1 -- 172.21.15.60:0/229891306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58580c7fc0 con 0x7f585809a930 2024-04-25T10:17:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f585ead1640 1 -- 172.21.15.60:0/229891306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58580cb6e0 con 0x7f585809a930 2024-04-25T10:17:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f58557fa640 1 -- 172.21.15.60:0/229891306 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5848018070 con 0x7f585809a930 2024-04-25T10:17:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f58557fa640 1 --2- 172.21.15.60:0/229891306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5844066f60 0x7f5844069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f58557fa640 1 -- 172.21.15.60:0/229891306 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f5848084fe0 con 0x7f585809a930 2024-04-25T10:17:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.729+0000 7f5857fff640 1 --2- 172.21.15.60:0/229891306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5844066f60 0x7f5844069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.733+0000 7f5857fff640 1 --2- 172.21.15.60:0/229891306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5844066f60 0x7f5844069420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f58580c7910 tx=0x7f584c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:00.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.733+0000 7f58557fa640 1 -- 172.21.15.60:0/229891306 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5848055f00 con 0x7f585809a930 2024-04-25T10:17:00.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.937+0000 7f585ead1640 1 -- 172.21.15.60:0/229891306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5858074770 con 0x7f585809a930 2024-04-25T10:17:00.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.937+0000 7f58557fa640 1 -- 172.21.15.60:0/229891306 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f5848055800 con 0x7f585809a930 2024-04-25T10:17:00.943 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:17:00.943 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":106,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2440975447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2440975447},{"type":"v1","addr":"172.21.15.100:6835","nonce":2440975447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2398080758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2398080758},{"type":"v1","addr":"172.21.15.60:6833","nonce":2398080758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2759861958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2759861958},{"type":"v1","addr":"172.21.15.100:6837","nonce":2759861958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:07.763322+0000","modified":"2024-04-25T10:16:52.349719+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9894":{"gid":9894,"name":"c","rank":0,"incarnation":103,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/3461876440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3461876440},{"type":"v1","addr":"172.21.15.60:6835","nonce":3461876440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:17:00.944 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 106 2024-04-25T10:17:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.941+0000 7f585ead1640 1 -- 172.21.15.60:0/229891306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5844066f60 msgr2=0x7f5844069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.941+0000 7f585ead1640 1 --2- 172.21.15.60:0/229891306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5844066f60 0x7f5844069420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f58580c7910 tx=0x7f584c008040 comp rx=0 tx=0).stop 2024-04-25T10:17:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.941+0000 7f585ead1640 1 -- 172.21.15.60:0/229891306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f585809a930 msgr2=0x7f58580b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.941+0000 7f585ead1640 1 --2- 172.21.15.60:0/229891306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f585809a930 0x7f58580b0130 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f584800c820 tx=0x7f584800cce0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.941+0000 7f585ead1640 1 -- 172.21.15.60:0/229891306 shutdown_connections 2024-04-25T10:17:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.941+0000 7f585ead1640 1 --2- 172.21.15.60:0/229891306 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5844066f60 0x7f5844069420 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.941+0000 7f585ead1640 1 --2- 172.21.15.60:0/229891306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f585815f480 0x7f585815e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.941+0000 7f585ead1640 1 --2- 172.21.15.60:0/229891306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58580cd390 0x7f5858156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.941+0000 7f585ead1640 1 --2- 172.21.15.60:0/229891306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f585809a930 0x7f58580b0130 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.941+0000 7f585ead1640 1 -- 172.21.15.60:0/229891306 >> 172.21.15.60:0/229891306 conn(0x7f58580b32a0 msgr2=0x7f58581582f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.941+0000 7f585ead1640 1 -- 172.21.15.60:0/229891306 shutdown_connections 2024-04-25T10:17:00.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:00.941+0000 7f585ead1640 1 -- 172.21.15.60:0/229891306 wait complete. 2024-04-25T10:17:00.964 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.081+0000 7f774f5ea640 1 Processor -- start 2024-04-25T10:17:01.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.081+0000 7f774f5ea640 1 -- start start 2024-04-25T10:17:01.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f774f5ea640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7748160eb0 0x7f77480b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f774f5ea640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7748161290 0x7f77480b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f774f5ea640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77480ce9c0 con 0x7f77480b9730 2024-04-25T10:17:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f774f5ea640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77480ceca0 con 0x7f7748160eb0 2024-04-25T10:17:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f774f5ea640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77480d2490 con 0x7f7748161290 2024-04-25T10:17:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f774d386640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7748160eb0 0x7f77480b91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f774db87640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f77480b9730 0x7f77480b53f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57520/0 (socket says 172.21.15.60:57520) 2024-04-25T10:17:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f774db87640 1 -- 172.21.15.60:0/801133821 learned_addr learned my addr 172.21.15.60:0/801133821 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f774cb85640 1 --2- 172.21.15.60:0/801133821 >> v2:172.21.15.100:3300/0 conn(0x7f7748161290 0x7f77480b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f774d386640 1 -- 172.21.15.60:0/801133821 >> v2:172.21.15.100:3300/0 conn(0x7f7748161290 msgr2=0x7f77480b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f774d386640 1 --2- 172.21.15.60:0/801133821 >> v2:172.21.15.100:3300/0 conn(0x7f7748161290 0x7f77480b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f774d386640 1 -- 172.21.15.60:0/801133821 >> v1:172.21.15.60:6789/0 conn(0x7f77480b9730 legacy=0x7f77480b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f774d386640 1 -- 172.21.15.60:0/801133821 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77480d2770 con 0x7f7748160eb0 2024-04-25T10:17:01.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f774d386640 1 --2- 172.21.15.60:0/801133821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7748160eb0 0x7f77480b91f0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f7748073280 tx=0x7f774401f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504b60b1484620c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:01.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f773ffff640 1 -- 172.21.15.60:0/801133821 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77440b4070 con 0x7f7748160eb0 2024-04-25T10:17:01.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f773ffff640 1 -- 172.21.15.60:0/801133821 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77440af600 con 0x7f7748160eb0 2024-04-25T10:17:01.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.085+0000 7f773ffff640 1 -- 172.21.15.60:0/801133821 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77440af9d0 con 0x7f7748160eb0 2024-04-25T10:17:01.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 -- 172.21.15.60:0/801133821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7748160eb0 msgr2=0x7f77480b91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 --2- 172.21.15.60:0/801133821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7748160eb0 0x7f77480b91f0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f7748073280 tx=0x7f774401f110 comp rx=0 tx=0).stop 2024-04-25T10:17:01.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 -- 172.21.15.60:0/801133821 shutdown_connections 2024-04-25T10:17:01.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 --2- 172.21.15.60:0/801133821 >> v2:172.21.15.100:3300/0 conn(0x7f7748161290 0x7f77480b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 --2- 172.21.15.60:0/801133821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7748160eb0 0x7f77480b91f0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 -- 172.21.15.60:0/801133821 >> 172.21.15.60:0/801133821 conn(0x7f77480d05c0 msgr2=0x7f77480d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:01.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 -- 172.21.15.60:0/801133821 shutdown_connections 2024-04-25T10:17:01.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 -- 172.21.15.60:0/801133821 wait complete. 2024-04-25T10:17:01.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 Processor -- start 2024-04-25T10:17:01.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 -- start start 2024-04-25T10:17:01.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77480b9730 0x7f77480c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7748160eb0 0x7f774815d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7748161290 0x7f774815a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77480b5b00 con 0x7f77480b9730 2024-04-25T10:17:01.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77480cef80 con 0x7f7748161290 2024-04-25T10:17:01.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77480b5c80 con 0x7f7748160eb0 2024-04-25T10:17:01.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774d386640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77480b9730 0x7f77480c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774d386640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77480b9730 0x7f77480c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53006/0 (socket says 172.21.15.60:53006) 2024-04-25T10:17:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774d386640 1 -- 172.21.15.60:0/1393299897 learned_addr learned my addr 172.21.15.60:0/1393299897 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774cb85640 1 --2- 172.21.15.60:0/1393299897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7748160eb0 0x7f774815d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774db87640 1 --2- 172.21.15.60:0/1393299897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7748161290 0x7f774815a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774d386640 1 -- 172.21.15.60:0/1393299897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7748160eb0 msgr2=0x7f774815d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774d386640 1 --2- 172.21.15.60:0/1393299897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7748160eb0 0x7f774815d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774d386640 1 -- 172.21.15.60:0/1393299897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7748161290 msgr2=0x7f774815a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774d386640 1 --2- 172.21.15.60:0/1393299897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7748161290 0x7f774815a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774d386640 1 -- 172.21.15.60:0/1393299897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77480ca150 con 0x7f77480b9730 2024-04-25T10:17:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774d386640 1 --2- 172.21.15.60:0/1393299897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77480b9730 0x7f77480c9c10 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f77440b2040 tx=0x7f774401beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f773dffb640 1 -- 172.21.15.60:0/1393299897 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77440b4040 con 0x7f77480b9730 2024-04-25T10:17:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f773dffb640 1 -- 172.21.15.60:0/1393299897 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77440c1070 con 0x7f77480b9730 2024-04-25T10:17:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 -- 172.21.15.60:0/1393299897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77480cc000 con 0x7f77480b9730 2024-04-25T10:17:01.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f773dffb640 1 -- 172.21.15.60:0/1393299897 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77440ba690 con 0x7f77480b9730 2024-04-25T10:17:01.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.089+0000 7f774f5ea640 1 -- 172.21.15.60:0/1393299897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77480cc5c0 con 0x7f77480b9730 2024-04-25T10:17:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.093+0000 7f774f5ea640 1 -- 172.21.15.60:0/1393299897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7748074770 con 0x7f77480b9730 2024-04-25T10:17:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.093+0000 7f773dffb640 1 -- 172.21.15.60:0/1393299897 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77440ba830 con 0x7f77480b9730 2024-04-25T10:17:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.093+0000 7f773dffb640 1 --2- 172.21.15.60:0/1393299897 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f772c066f60 0x7f772c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.093+0000 7f773dffb640 1 -- 172.21.15.60:0/1393299897 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f7744123f00 con 0x7f77480b9730 2024-04-25T10:17:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.093+0000 7f774cb85640 1 --2- 172.21.15.60:0/1393299897 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f772c066f60 0x7f772c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.093+0000 7f774cb85640 1 --2- 172.21.15.60:0/1393299897 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f772c066f60 0x7f772c069420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7748159dc0 tx=0x7f7738008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:01.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.097+0000 7f773dffb640 1 -- 172.21.15.60:0/1393299897 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7744018c70 con 0x7f77480b9730 2024-04-25T10:17:01.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.261+0000 7f774f5ea640 1 -- 172.21.15.60:0/1393299897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77480d15b0 con 0x7f77480b9730 2024-04-25T10:17:01.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.265+0000 7f773dffb640 1 -- 172.21.15.60:0/1393299897 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16894 (secure 0 0 0) 0x7f77440f50d0 con 0x7f77480b9730 2024-04-25T10:17:01.270 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:17:01.270 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":112,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:16:48.303329+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:06.033387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:06.753421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/178428701":"2024-04-26T10:16:48.303277+0000","172.21.15.60:6834/178428701":"2024-04-26T10:16:48.303277+0000","172.21.15.60:6835/722171246":"2024-04-26T10:16:28.355776+0000","172.21.15.60:6834/722171246":"2024-04-26T10:16:28.355776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.265+0000 7f774f5ea640 1 -- 172.21.15.60:0/1393299897 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f772c066f60 msgr2=0x7f772c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.265+0000 7f774f5ea640 1 --2- 172.21.15.60:0/1393299897 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f772c066f60 0x7f772c069420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7748159dc0 tx=0x7f7738008040 comp rx=0 tx=0).stop 2024-04-25T10:17:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.265+0000 7f774f5ea640 1 -- 172.21.15.60:0/1393299897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77480b9730 msgr2=0x7f77480c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.265+0000 7f774f5ea640 1 --2- 172.21.15.60:0/1393299897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77480b9730 0x7f77480c9c10 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f77440b2040 tx=0x7f774401beb0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.265+0000 7f774f5ea640 1 -- 172.21.15.60:0/1393299897 shutdown_connections 2024-04-25T10:17:01.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.265+0000 7f774f5ea640 1 --2- 172.21.15.60:0/1393299897 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f772c066f60 0x7f772c069420 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.265+0000 7f774f5ea640 1 --2- 172.21.15.60:0/1393299897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7748161290 0x7f774815a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.265+0000 7f774f5ea640 1 --2- 172.21.15.60:0/1393299897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7748160eb0 0x7f774815d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.265+0000 7f774f5ea640 1 --2- 172.21.15.60:0/1393299897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77480b9730 0x7f77480c9c10 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.265+0000 7f774f5ea640 1 -- 172.21.15.60:0/1393299897 >> 172.21.15.60:0/1393299897 conn(0x7f77480d05c0 msgr2=0x7f77480b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:01.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.265+0000 7f774f5ea640 1 -- 172.21.15.60:0/1393299897 shutdown_connections 2024-04-25T10:17:01.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.265+0000 7f774f5ea640 1 -- 172.21.15.60:0/1393299897 wait complete. 2024-04-25T10:17:01.290 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:05:08.461484+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:06.033387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:06.753421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:01.291 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:01.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.409+0000 7f2b347b4640 1 Processor -- start 2024-04-25T10:17:01.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.409+0000 7f2b347b4640 1 -- start start 2024-04-25T10:17:01.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.409+0000 7f2b347b4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b2c0b2560 0x7f2b2c0cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.409+0000 7f2b347b4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2b2c0cf8d0 0x7f2b2c06e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.409+0000 7f2b347b4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2c0cfd40 con 0x7f2b2c154c10 2024-04-25T10:17:01.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.409+0000 7f2b347b4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c15e410 con 0x7f2b2c0cf8d0 2024-04-25T10:17:01.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.409+0000 7f2b347b4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c15e6f0 con 0x7f2b2c0b2560 2024-04-25T10:17:01.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.409+0000 7f2b32d51640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2b2c0cf8d0 0x7f2b2c06e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.409+0000 7f2b32d51640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2b2c0cf8d0 0x7f2b2c06e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:35806/0 (socket says 172.21.15.60:35806) 2024-04-25T10:17:01.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.409+0000 7f2b32550640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2b2c154c10 0x7f2b2c0b42b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:38622/0 (socket says 172.21.15.60:38622) 2024-04-25T10:17:01.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.409+0000 7f2b31d4f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b2c0b2560 0x7f2b2c0cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.409+0000 7f2b32d51640 1 -- 172.21.15.60:0/3846443701 learned_addr learned my addr 172.21.15.60:0/3846443701 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:01.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.409+0000 7f2b32d51640 1 -- 172.21.15.60:0/3846443701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b2c0b2560 msgr2=0x7f2b2c0cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:01.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.409+0000 7f2b32d51640 1 --2- 172.21.15.60:0/3846443701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b2c0b2560 0x7f2b2c0cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.409+0000 7f2b32d51640 1 -- 172.21.15.60:0/3846443701 >> v1:172.21.15.100:6789/0 conn(0x7f2b2c154c10 legacy=0x7f2b2c0b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.409+0000 7f2b32d51640 1 -- 172.21.15.60:0/3846443701 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c15e9d0 con 0x7f2b2c0cf8d0 2024-04-25T10:17:01.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.413+0000 7f2b32d51640 1 --2- 172.21.15.60:0/3846443701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c0cf8d0 0x7f2b2c06e730 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400d150 tx=0x7f2b2401eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe7d3b95a5b4d28a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:01.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.413+0000 7f2b3154e640 1 -- 172.21.15.60:0/3846443701 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2400b2d0 con 0x7f2b2c0cf8d0 2024-04-25T10:17:01.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.413+0000 7f2b3154e640 1 -- 172.21.15.60:0/3846443701 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2400b470 con 0x7f2b2c0cf8d0 2024-04-25T10:17:01.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.413+0000 7f2b3154e640 1 -- 172.21.15.60:0/3846443701 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b240a7ba0 con 0x7f2b2c0cf8d0 2024-04-25T10:17:01.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.413+0000 7f2b347b4640 1 -- 172.21.15.60:0/3846443701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c0cf8d0 msgr2=0x7f2b2c06e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.413+0000 7f2b347b4640 1 --2- 172.21.15.60:0/3846443701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c0cf8d0 0x7f2b2c06e730 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400d150 tx=0x7f2b2401eba0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.413+0000 7f2b347b4640 1 -- 172.21.15.60:0/3846443701 shutdown_connections 2024-04-25T10:17:01.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.413+0000 7f2b347b4640 1 --2- 172.21.15.60:0/3846443701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c0cf8d0 0x7f2b2c06e730 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.413+0000 7f2b347b4640 1 --2- 172.21.15.60:0/3846443701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b2c0b2560 0x7f2b2c0cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.413+0000 7f2b347b4640 1 -- 172.21.15.60:0/3846443701 >> 172.21.15.60:0/3846443701 conn(0x7f2b2c0670b0 msgr2=0x7f2b2c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:01.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.413+0000 7f2b347b4640 1 -- 172.21.15.60:0/3846443701 shutdown_connections 2024-04-25T10:17:01.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.413+0000 7f2b347b4640 1 -- 172.21.15.60:0/3846443701 wait complete. 2024-04-25T10:17:01.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b347b4640 1 Processor -- start 2024-04-25T10:17:01.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b347b4640 1 -- start start 2024-04-25T10:17:01.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b347b4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b2c0b2560 0x7f2b2c151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b347b4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c0cf8d0 0x7f2b2c14b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b347b4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b2c154c10 0x7f2b2c14bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b347b4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c155170 con 0x7f2b2c0b2560 2024-04-25T10:17:01.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b347b4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c154ff0 con 0x7f2b2c0cf8d0 2024-04-25T10:17:01.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b347b4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c071250 con 0x7f2b2c154c10 2024-04-25T10:17:01.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b32550640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b2c0b2560 0x7f2b2c151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b32550640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b2c0b2560 0x7f2b2c151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53020/0 (socket says 172.21.15.60:53020) 2024-04-25T10:17:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b32550640 1 -- 172.21.15.60:0/3623884046 learned_addr learned my addr 172.21.15.60:0/3623884046 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b31d4f640 1 --2- 172.21.15.60:0/3623884046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c0cf8d0 0x7f2b2c14b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b32d51640 1 --2- 172.21.15.60:0/3623884046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b2c154c10 0x7f2b2c14bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b32550640 1 -- 172.21.15.60:0/3623884046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b2c154c10 msgr2=0x7f2b2c14bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b32550640 1 --2- 172.21.15.60:0/3623884046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b2c154c10 0x7f2b2c14bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b32550640 1 -- 172.21.15.60:0/3623884046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c0cf8d0 msgr2=0x7f2b2c14b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b32550640 1 --2- 172.21.15.60:0/3623884046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c0cf8d0 0x7f2b2c14b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b32550640 1 -- 172.21.15.60:0/3623884046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c152030 con 0x7f2b2c0b2560 2024-04-25T10:17:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b31d4f640 1 --2- 172.21.15.60:0/3623884046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c0cf8d0 0x7f2b2c14b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b32d51640 1 --2- 172.21.15.60:0/3623884046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b2c154c10 0x7f2b2c14bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b32550640 1 --2- 172.21.15.60:0/3623884046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b2c0b2560 0x7f2b2c151af0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800c370 tx=0x7f2b2800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b22ffd640 1 -- 172.21.15.60:0/3623884046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b280074e0 con 0x7f2b2c0b2560 2024-04-25T10:17:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b22ffd640 1 -- 172.21.15.60:0/3623884046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b28007680 con 0x7f2b2c0b2560 2024-04-25T10:17:01.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b22ffd640 1 -- 172.21.15.60:0/3623884046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b28004a30 con 0x7f2b2c0b2560 2024-04-25T10:17:01.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b347b4640 1 -- 172.21.15.60:0/3623884046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2c1542c0 con 0x7f2b2c0b2560 2024-04-25T10:17:01.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b347b4640 1 -- 172.21.15.60:0/3623884046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2c133c80 con 0x7f2b2c0b2560 2024-04-25T10:17:01.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.417+0000 7f2b347b4640 1 -- 172.21.15.60:0/3623884046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2c1547e0 con 0x7f2b2c0b2560 2024-04-25T10:17:01.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.421+0000 7f2b22ffd640 1 -- 172.21.15.60:0/3623884046 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2b2801a020 con 0x7f2b2c0b2560 2024-04-25T10:17:01.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.421+0000 7f2b22ffd640 1 --2- 172.21.15.60:0/3623884046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b10066f60 0x7f2b10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.421+0000 7f2b22ffd640 1 -- 172.21.15.60:0/3623884046 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f2b28084930 con 0x7f2b2c0b2560 2024-04-25T10:17:01.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.421+0000 7f2b31d4f640 1 --2- 172.21.15.60:0/3623884046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b10066f60 0x7f2b10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.421+0000 7f2b31d4f640 1 --2- 172.21.15.60:0/3623884046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b10066f60 0x7f2b10069420 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c1f5a30 tx=0x7f2b1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.425+0000 7f2b22ffd640 1 -- 172.21.15.60:0/3623884046 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2b28055800 con 0x7f2b2c0b2560 2024-04-25T10:17:01.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.585+0000 7f2b347b4640 1 -- 172.21.15.60:0/3623884046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2b2c06bba0 con 0x7f2b2c0b2560 2024-04-25T10:17:01.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.589+0000 7f2b22ffd640 1 -- 172.21.15.60:0/3623884046 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b28055100 con 0x7f2b2c0b2560 2024-04-25T10:17:01.593 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T10:17:01.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.589+0000 7f2b347b4640 1 -- 172.21.15.60:0/3623884046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b10066f60 msgr2=0x7f2b10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.589+0000 7f2b347b4640 1 --2- 172.21.15.60:0/3623884046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b10066f60 0x7f2b10069420 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c1f5a30 tx=0x7f2b1c008040 comp rx=0 tx=0).stop 2024-04-25T10:17:01.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.589+0000 7f2b347b4640 1 -- 172.21.15.60:0/3623884046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b2c0b2560 msgr2=0x7f2b2c151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.589+0000 7f2b347b4640 1 --2- 172.21.15.60:0/3623884046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b2c0b2560 0x7f2b2c151af0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800c370 tx=0x7f2b2800c830 comp rx=0 tx=0).stop 2024-04-25T10:17:01.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.589+0000 7f2b347b4640 1 -- 172.21.15.60:0/3623884046 shutdown_connections 2024-04-25T10:17:01.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.589+0000 7f2b347b4640 1 --2- 172.21.15.60:0/3623884046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2b10066f60 0x7f2b10069420 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.589+0000 7f2b347b4640 1 --2- 172.21.15.60:0/3623884046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b2c154c10 0x7f2b2c14bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.589+0000 7f2b347b4640 1 --2- 172.21.15.60:0/3623884046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b2c0cf8d0 0x7f2b2c14b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.589+0000 7f2b347b4640 1 --2- 172.21.15.60:0/3623884046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b2c0b2560 0x7f2b2c151af0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.589+0000 7f2b347b4640 1 -- 172.21.15.60:0/3623884046 >> 172.21.15.60:0/3623884046 conn(0x7f2b2c0670b0 msgr2=0x7f2b2c0cdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:01.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.589+0000 7f2b347b4640 1 -- 172.21.15.60:0/3623884046 shutdown_connections 2024-04-25T10:17:01.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.589+0000 7f2b347b4640 1 -- 172.21.15.60:0/3623884046 wait complete. 2024-04-25T10:17:01.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:17:01.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.733+0000 7feed5410640 1 Processor -- start 2024-04-25T10:17:01.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.733+0000 7feed5410640 1 -- start start 2024-04-25T10:17:01.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.733+0000 7feed5410640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feed0160080 0x7feed015f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.733+0000 7feed5410640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed015fa20 0x7feed00ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.733+0000 7feed5410640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed0144a40 con 0x7feed00cbcf0 2024-04-25T10:17:01.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.733+0000 7feed5410640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed015fe90 con 0x7feed0160080 2024-04-25T10:17:01.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.733+0000 7feed5410640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed00b2dc0 con 0x7feed015fa20 2024-04-25T10:17:01.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.733+0000 7feece7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feed0160080 0x7feed015f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.733+0000 7feece7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feed0160080 0x7feed015f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53026/0 (socket says 172.21.15.60:53026) 2024-04-25T10:17:01.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.733+0000 7feece7fc640 1 -- 172.21.15.60:0/162773170 learned_addr learned my addr 172.21.15.60:0/162773170 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:01.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.733+0000 7feecf7fe640 1 --2- 172.21.15.60:0/162773170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed015fa20 0x7feed00ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.733+0000 7feece7fc640 1 -- 172.21.15.60:0/162773170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed015fa20 msgr2=0x7feed00ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:01.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.733+0000 7feece7fc640 1 --2- 172.21.15.60:0/162773170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed015fa20 0x7feed00ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.733+0000 7feece7fc640 1 -- 172.21.15.60:0/162773170 >> v1:172.21.15.100:6789/0 conn(0x7feed00cbcf0 legacy=0x7feed0071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:01.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.733+0000 7feece7fc640 1 -- 172.21.15.60:0/162773170 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed00cc9f0 con 0x7feed0160080 2024-04-25T10:17:01.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.733+0000 7feecf7fe640 1 --2- 172.21.15.60:0/162773170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed015fa20 0x7feed00ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:01.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feece7fc640 1 --2- 172.21.15.60:0/162773170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed0160080 0x7feed015f4e0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7feed00d0570 tx=0x7feec401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197bb6d02c4fe590 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:01.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feecdffb640 1 -- 172.21.15.60:0/162773170 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40b2070 con 0x7feed0160080 2024-04-25T10:17:01.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feecdffb640 1 -- 172.21.15.60:0/162773170 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec40ad640 con 0x7feed0160080 2024-04-25T10:17:01.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feecdffb640 1 -- 172.21.15.60:0/162773170 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40ada10 con 0x7feed0160080 2024-04-25T10:17:01.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feed5410640 1 -- 172.21.15.60:0/162773170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed0160080 msgr2=0x7feed015f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feed5410640 1 --2- 172.21.15.60:0/162773170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed0160080 0x7feed015f4e0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7feed00d0570 tx=0x7feec401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feed5410640 1 -- 172.21.15.60:0/162773170 shutdown_connections 2024-04-25T10:17:01.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feed5410640 1 --2- 172.21.15.60:0/162773170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed0160080 0x7feed015f4e0 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feed5410640 1 --2- 172.21.15.60:0/162773170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed015fa20 0x7feed00ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feed5410640 1 -- 172.21.15.60:0/162773170 >> 172.21.15.60:0/162773170 conn(0x7feed00670b0 msgr2=0x7feed006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:01.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feed5410640 1 -- 172.21.15.60:0/162773170 shutdown_connections 2024-04-25T10:17:01.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feed5410640 1 -- 172.21.15.60:0/162773170 wait complete. 2024-04-25T10:17:01.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feed5410640 1 Processor -- start 2024-04-25T10:17:01.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feed5410640 1 -- start start 2024-04-25T10:17:01.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feed5410640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed00cbcf0 0x7feed00aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feed5410640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed015fa20 0x7feed013df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feed5410640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed0160080 0x7feed0159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feed5410640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed00ce5c0 con 0x7feed00cbcf0 2024-04-25T10:17:01.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feed5410640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed00cc0d0 con 0x7feed015fa20 2024-04-25T10:17:01.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.737+0000 7feed5410640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed00ce740 con 0x7feed0160080 2024-04-25T10:17:01.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feeceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed00cbcf0 0x7feed00aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feece7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed015fa20 0x7feed013df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feecf7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed0160080 0x7feed0159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feeceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed00cbcf0 0x7feed00aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53028/0 (socket says 172.21.15.60:53028) 2024-04-25T10:17:01.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feece7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed015fa20 0x7feed013df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:35816/0 (socket says 172.21.15.60:35816) 2024-04-25T10:17:01.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feeceffd640 1 -- 172.21.15.60:0/4260784525 learned_addr learned my addr 172.21.15.60:0/4260784525 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:01.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feeceffd640 1 -- 172.21.15.60:0/4260784525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed0160080 msgr2=0x7feed0159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feeceffd640 1 --2- 172.21.15.60:0/4260784525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed0160080 0x7feed0159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feeceffd640 1 -- 172.21.15.60:0/4260784525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed015fa20 msgr2=0x7feed013df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feeceffd640 1 --2- 172.21.15.60:0/4260784525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed015fa20 0x7feed013df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feeceffd640 1 -- 172.21.15.60:0/4260784525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed013e4a0 con 0x7feed00cbcf0 2024-04-25T10:17:01.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feeceffd640 1 --2- 172.21.15.60:0/4260784525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed00cbcf0 0x7feed00aefe0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7feec000c450 tx=0x7feec000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:01.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feeb77fe640 1 -- 172.21.15.60:0/4260784525 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0012830 con 0x7feed00cbcf0 2024-04-25T10:17:01.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feeb77fe640 1 -- 172.21.15.60:0/4260784525 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec0017070 con 0x7feed00cbcf0 2024-04-25T10:17:01.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feed5410640 1 -- 172.21.15.60:0/4260784525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed015a7e0 con 0x7feed00cbcf0 2024-04-25T10:17:01.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feeb77fe640 1 -- 172.21.15.60:0/4260784525 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0012c30 con 0x7feed00cbcf0 2024-04-25T10:17:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feed5410640 1 -- 172.21.15.60:0/4260784525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed015ada0 con 0x7feed00cbcf0 2024-04-25T10:17:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feed5410640 1 -- 172.21.15.60:0/4260784525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed0074770 con 0x7feed00cbcf0 2024-04-25T10:17:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feeb77fe640 1 -- 172.21.15.60:0/4260784525 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feec001c460 con 0x7feed00cbcf0 2024-04-25T10:17:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feeb77fe640 1 --2- 172.21.15.60:0/4260784525 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feebc066f60 0x7feebc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feeb77fe640 1 -- 172.21.15.60:0/4260784525 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7feec0084d90 con 0x7feed00cbcf0 2024-04-25T10:17:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.741+0000 7feece7fc640 1 --2- 172.21.15.60:0/4260784525 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feebc066f60 0x7feebc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.745+0000 7feece7fc640 1 --2- 172.21.15.60:0/4260784525 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feebc066f60 0x7feebc069420 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7feec40af000 tx=0x7feec40a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:01.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.745+0000 7feeb77fe640 1 -- 172.21.15.60:0/4260784525 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feec0055690 con 0x7feed00cbcf0 2024-04-25T10:17:01.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.905+0000 7feed5410640 1 -- 172.21.15.60:0/4260784525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feed015dfd0 con 0x7feed00cbcf0 2024-04-25T10:17:01.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.909+0000 7feeb77fe640 1 -- 172.21.15.60:0/4260784525 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7feec0059730 con 0x7feed00cbcf0 2024-04-25T10:17:01.914 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T10:17:01.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.909+0000 7feed5410640 1 -- 172.21.15.60:0/4260784525 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feebc066f60 msgr2=0x7feebc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.909+0000 7feed5410640 1 --2- 172.21.15.60:0/4260784525 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feebc066f60 0x7feebc069420 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7feec40af000 tx=0x7feec40a7070 comp rx=0 tx=0).stop 2024-04-25T10:17:01.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.909+0000 7feed5410640 1 -- 172.21.15.60:0/4260784525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed00cbcf0 msgr2=0x7feed00aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.909+0000 7feed5410640 1 --2- 172.21.15.60:0/4260784525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed00cbcf0 0x7feed00aefe0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7feec000c450 tx=0x7feec000c910 comp rx=0 tx=0).stop 2024-04-25T10:17:01.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.909+0000 7feed5410640 1 -- 172.21.15.60:0/4260784525 shutdown_connections 2024-04-25T10:17:01.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.909+0000 7feed5410640 1 --2- 172.21.15.60:0/4260784525 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7feebc066f60 0x7feebc069420 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.909+0000 7feed5410640 1 --2- 172.21.15.60:0/4260784525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed0160080 0x7feed0159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.909+0000 7feed5410640 1 --2- 172.21.15.60:0/4260784525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed015fa20 0x7feed013df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.909+0000 7feed5410640 1 --2- 172.21.15.60:0/4260784525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed00cbcf0 0x7feed00aefe0 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.909+0000 7feed5410640 1 -- 172.21.15.60:0/4260784525 >> 172.21.15.60:0/4260784525 conn(0x7feed00670b0 msgr2=0x7feed015e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:01.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.909+0000 7feed5410640 1 -- 172.21.15.60:0/4260784525 shutdown_connections 2024-04-25T10:17:01.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:01.909+0000 7feed5410640 1 -- 172.21.15.60:0/4260784525 wait complete. 2024-04-25T10:17:01.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:17:02.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.049+0000 7f5750947640 1 Processor -- start 2024-04-25T10:17:02.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f5750947640 1 -- start start 2024-04-25T10:17:02.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f5750947640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f574c15b6f0 0x7f574c1620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f5750947640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574c15bad0 con 0x7f574c0ceef0 2024-04-25T10:17:02.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f5750947640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574c15ebe0 con 0x7f574c15e0d0 2024-04-25T10:17:02.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f5750947640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c162610 con 0x7f574c15b6f0 2024-04-25T10:17:02.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574ad9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f574c0ceef0 0x7f574c0d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57530/0 (socket says 172.21.15.60:57530) 2024-04-25T10:17:02.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574ad9d640 1 -- 172.21.15.60:0/4110347028 learned_addr learned my addr 172.21.15.60:0/4110347028 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:02.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574a59c640 1 --2- 172.21.15.60:0/4110347028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f574c15b6f0 0x7f574c1620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574959a640 1 -- 172.21.15.60:0/4110347028 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112154495 0 0) 0x7f574c15bad0 con 0x7f574c0ceef0 2024-04-25T10:17:02.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574959a640 1 -- 172.21.15.60:0/4110347028 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5734003530 con 0x7f574c0ceef0 2024-04-25T10:17:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574959a640 1 -- 172.21.15.60:0/4110347028 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1649402627 0 0) 0x7f5734003530 con 0x7f574c0ceef0 2024-04-25T10:17:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574959a640 1 -- 172.21.15.60:0/4110347028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f574c15b6f0 msgr2=0x7f574c1620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574959a640 1 --2- 172.21.15.60:0/4110347028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f574c15b6f0 0x7f574c1620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574959a640 1 -- 172.21.15.60:0/4110347028 >> v1:172.21.15.100:6789/0 conn(0x7f574c15e0d0 legacy=0x7f574c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574959a640 1 -- 172.21.15.60:0/4110347028 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c15eec0 con 0x7f574c0ceef0 2024-04-25T10:17:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574959a640 1 -- 172.21.15.60:0/4110347028 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f5744001a00 con 0x7f574c0ceef0 2024-04-25T10:17:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574959a640 1 -- 172.21.15.60:0/4110347028 >> v1:172.21.15.60:6789/0 conn(0x7f574c0ceef0 legacy=0x7f574c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574959a640 1 --2- 172.21.15.60:0/4110347028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57340039f0 0x7f5734003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574959a640 1 --2- 172.21.15.60:0/4110347028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f574c15b6f0 0x7f5734004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574959a640 1 --2- 172.21.15.60:0/4110347028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5734004d50 0x7f57340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574959a640 1 -- 172.21.15.60:0/4110347028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c162610 con 0x7f5734004d50 2024-04-25T10:17:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574959a640 1 -- 172.21.15.60:0/4110347028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573400bab0 con 0x7f57340039f0 2024-04-25T10:17:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574959a640 1 -- 172.21.15.60:0/4110347028 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573400bcb0 con 0x7f574c15b6f0 2024-04-25T10:17:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574ad9d640 1 --2- 172.21.15.60:0/4110347028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5734004d50 0x7f57340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574a59c640 1 --2- 172.21.15.60:0/4110347028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57340039f0 0x7f5734003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f5749d9b640 1 --2- 172.21.15.60:0/4110347028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f574c15b6f0 0x7f5734004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574ad9d640 1 -- 172.21.15.60:0/4110347028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f574c15b6f0 msgr2=0x7f5734004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:02.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574ad9d640 1 --2- 172.21.15.60:0/4110347028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f574c15b6f0 0x7f5734004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574ad9d640 1 -- 172.21.15.60:0/4110347028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57340039f0 msgr2=0x7f5734003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:02.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574ad9d640 1 --2- 172.21.15.60:0/4110347028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57340039f0 0x7f5734003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.053+0000 7f574ad9d640 1 -- 172.21.15.60:0/4110347028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5734003530 con 0x7f5734004d50 2024-04-25T10:17:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f574ad9d640 1 --2- 172.21.15.60:0/4110347028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5734004d50 0x7f57340051d0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f574c0d1690 tx=0x7f574401d240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24374551407324dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f574959a640 1 -- 172.21.15.60:0/4110347028 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574400a3d0 con 0x7f5734004d50 2024-04-25T10:17:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f574959a640 1 -- 172.21.15.60:0/4110347028 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57440ad650 con 0x7f5734004d50 2024-04-25T10:17:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f574959a640 1 -- 172.21.15.60:0/4110347028 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57440ada80 con 0x7f5734004d50 2024-04-25T10:17:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f5750947640 1 -- 172.21.15.60:0/4110347028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5734004d50 msgr2=0x7f57340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f5750947640 1 --2- 172.21.15.60:0/4110347028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5734004d50 0x7f57340051d0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f574c0d1690 tx=0x7f574401d240 comp rx=0 tx=0).stop 2024-04-25T10:17:02.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f574ad9d640 1 -- 172.21.15.60:0/4110347028 reap_dead start 2024-04-25T10:17:02.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f5750947640 1 -- 172.21.15.60:0/4110347028 shutdown_connections 2024-04-25T10:17:02.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f5750947640 1 -- 172.21.15.60:0/4110347028 >> 172.21.15.60:0/4110347028 conn(0x7f574c072340 msgr2=0x7f574c072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:02.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f5750947640 1 -- 172.21.15.60:0/4110347028 shutdown_connections 2024-04-25T10:17:02.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f5750947640 1 -- 172.21.15.60:0/4110347028 wait complete. 2024-04-25T10:17:02.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f5750947640 1 Processor -- start 2024-04-25T10:17:02.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f5750947640 1 -- start start 2024-04-25T10:17:02.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f5750947640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5734004d50 0x7f574c159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f5750947640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f574c1f79b0 0x7f574c0c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f5750947640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f574c1f36d0 0x7f574c15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f5750947640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0b9550 con 0x7f5734004d50 2024-04-25T10:17:02.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f5750947640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c161c40 con 0x7f574c1f79b0 2024-04-25T10:17:02.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f5750947640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0b96d0 con 0x7f574c1f36d0 2024-04-25T10:17:02.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f574a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5734004d50 0x7f574c159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f574a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5734004d50 0x7f574c159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53050/0 (socket says 172.21.15.60:53050) 2024-04-25T10:17:02.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.057+0000 7f574a59c640 1 -- 172.21.15.60:0/2916440415 learned_addr learned my addr 172.21.15.60:0/2916440415 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:02.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f574ad9d640 1 --2- 172.21.15.60:0/2916440415 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f574c1f36d0 0x7f574c15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f5749d9b640 1 --2- 172.21.15.60:0/2916440415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f574c1f79b0 0x7f574c0c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f574a59c640 1 -- 172.21.15.60:0/2916440415 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f574c1f36d0 msgr2=0x7f574c15cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f574a59c640 1 --2- 172.21.15.60:0/2916440415 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f574c1f36d0 0x7f574c15cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f574a59c640 1 -- 172.21.15.60:0/2916440415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f574c1f79b0 msgr2=0x7f574c0c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f574a59c640 1 --2- 172.21.15.60:0/2916440415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f574c1f79b0 0x7f574c0c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f574a59c640 1 -- 172.21.15.60:0/2916440415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c1f40b0 con 0x7f5734004d50 2024-04-25T10:17:02.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f574ad9d640 1 --2- 172.21.15.60:0/2916440415 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f574c1f36d0 0x7f574c15cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:02.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f5749d9b640 1 --2- 172.21.15.60:0/2916440415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f574c1f79b0 0x7f574c0c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:02.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f574a59c640 1 --2- 172.21.15.60:0/2916440415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5734004d50 0x7f574c159bd0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f573c0115e0 tx=0x7f573c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:02.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f573affd640 1 -- 172.21.15.60:0/2916440415 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c00f8f0 con 0x7f5734004d50 2024-04-25T10:17:02.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f573affd640 1 -- 172.21.15.60:0/2916440415 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573c00fa90 con 0x7f5734004d50 2024-04-25T10:17:02.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f5750947640 1 -- 172.21.15.60:0/2916440415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c1f4390 con 0x7f5734004d50 2024-04-25T10:17:02.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f573affd640 1 -- 172.21.15.60:0/2916440415 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c00fe40 con 0x7f5734004d50 2024-04-25T10:17:02.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f5750947640 1 -- 172.21.15.60:0/2916440415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c0c7950 con 0x7f5734004d50 2024-04-25T10:17:02.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f5750947640 1 -- 172.21.15.60:0/2916440415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c074770 con 0x7f5734004d50 2024-04-25T10:17:02.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f573affd640 1 -- 172.21.15.60:0/2916440415 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f573c015460 con 0x7f5734004d50 2024-04-25T10:17:02.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f573affd640 1 --2- 172.21.15.60:0/2916440415 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f572c066f60 0x7f572c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f573affd640 1 -- 172.21.15.60:0/2916440415 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f573c085a10 con 0x7f5734004d50 2024-04-25T10:17:02.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f5749d9b640 1 --2- 172.21.15.60:0/2916440415 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f572c066f60 0x7f572c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.061+0000 7f5749d9b640 1 --2- 172.21.15.60:0/2916440415 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f572c066f60 0x7f572c069420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5740000c10 tx=0x7f5740009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:02.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.065+0000 7f573affd640 1 -- 172.21.15.60:0/2916440415 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f573c019030 con 0x7f5734004d50 2024-04-25T10:17:02.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.225+0000 7f5750947640 1 -- 172.21.15.60:0/2916440415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f574c0d16e0 con 0x7f5734004d50 2024-04-25T10:17:02.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.225+0000 7f573affd640 1 -- 172.21.15.60:0/2916440415 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7f573c056a00 con 0x7f5734004d50 2024-04-25T10:17:02.233 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T10:17:02.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.229+0000 7f5750947640 1 -- 172.21.15.60:0/2916440415 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f572c066f60 msgr2=0x7f572c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.229+0000 7f5750947640 1 --2- 172.21.15.60:0/2916440415 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f572c066f60 0x7f572c069420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5740000c10 tx=0x7f5740009040 comp rx=0 tx=0).stop 2024-04-25T10:17:02.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.229+0000 7f5750947640 1 -- 172.21.15.60:0/2916440415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5734004d50 msgr2=0x7f574c159bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.229+0000 7f5750947640 1 --2- 172.21.15.60:0/2916440415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5734004d50 0x7f574c159bd0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f573c0115e0 tx=0x7f573c011aa0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.229+0000 7f5750947640 1 -- 172.21.15.60:0/2916440415 shutdown_connections 2024-04-25T10:17:02.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.229+0000 7f5750947640 1 --2- 172.21.15.60:0/2916440415 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f572c066f60 0x7f572c069420 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.229+0000 7f5750947640 1 --2- 172.21.15.60:0/2916440415 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f574c1f36d0 0x7f574c15cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.229+0000 7f5750947640 1 --2- 172.21.15.60:0/2916440415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f574c1f79b0 0x7f574c0c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.229+0000 7f5750947640 1 --2- 172.21.15.60:0/2916440415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5734004d50 0x7f574c159bd0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.229+0000 7f5750947640 1 -- 172.21.15.60:0/2916440415 >> 172.21.15.60:0/2916440415 conn(0x7f574c072340 msgr2=0x7f574c0b72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:02.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.229+0000 7f5750947640 1 -- 172.21.15.60:0/2916440415 shutdown_connections 2024-04-25T10:17:02.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.229+0000 7f5750947640 1 -- 172.21.15.60:0/2916440415 wait complete. 2024-04-25T10:17:02.252 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:02.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.357+0000 7fc2f1f56640 1 Processor -- start 2024-04-25T10:17:02.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.357+0000 7fc2f1f56640 1 -- start start 2024-04-25T10:17:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.357+0000 7fc2f1f56640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc2ec0ce7a0 0x7fc2ec0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.357+0000 7fc2f1f56640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2ec0b4660 0x7fc2ec0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.357+0000 7fc2f1f56640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2ec0cf0c0 con 0x7fc2ec15bc90 2024-04-25T10:17:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.357+0000 7fc2f1f56640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2ec0cc370 con 0x7fc2ec0ce7a0 2024-04-25T10:17:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.357+0000 7fc2f1f56640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec071ca0 con 0x7fc2ec0b4660 2024-04-25T10:17:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2e3fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc2ec15bc90 0x7fc2ec0b25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57538/0 (socket says 172.21.15.60:57538) 2024-04-25T10:17:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2e3fff640 1 -- 172.21.15.60:0/1790245206 learned_addr learned my addr 172.21.15.60:0/1790245206 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2ebfff640 1 --2- 172.21.15.60:0/1790245206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2ec0b4660 0x7fc2ec0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eb7fe640 1 --2- 172.21.15.60:0/1790245206 >> v2:172.21.15.100:3300/0 conn(0x7fc2ec0ce7a0 0x7fc2ec0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 -- 172.21.15.60:0/1790245206 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832178246 0 0) 0x7fc2ec0cf0c0 con 0x7fc2ec15bc90 2024-04-25T10:17:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 -- 172.21.15.60:0/1790245206 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2d8003530 con 0x7fc2ec15bc90 2024-04-25T10:17:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 -- 172.21.15.60:0/1790245206 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 732294542 0 0) 0x7fc2d8003530 con 0x7fc2ec15bc90 2024-04-25T10:17:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 -- 172.21.15.60:0/1790245206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2ec0b4660 msgr2=0x7fc2ec0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 --2- 172.21.15.60:0/1790245206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2ec0b4660 0x7fc2ec0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 -- 172.21.15.60:0/1790245206 >> v2:172.21.15.100:3300/0 conn(0x7fc2ec0ce7a0 msgr2=0x7fc2ec0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 --2- 172.21.15.60:0/1790245206 >> v2:172.21.15.100:3300/0 conn(0x7fc2ec0ce7a0 0x7fc2ec0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 -- 172.21.15.60:0/1790245206 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec0cca40 con 0x7fc2ec15bc90 2024-04-25T10:17:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 -- 172.21.15.60:0/1790245206 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fc2d4001680 con 0x7fc2ec15bc90 2024-04-25T10:17:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 -- 172.21.15.60:0/1790245206 >> v1:172.21.15.60:6789/0 conn(0x7fc2ec15bc90 legacy=0x7fc2ec0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 --2- 172.21.15.60:0/1790245206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2d80039c0 0x7fc2d8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 --2- 172.21.15.60:0/1790245206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2d8004320 0x7fc2d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 --2- 172.21.15.60:0/1790245206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2d8004e60 0x7fc2d80052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 -- 172.21.15.60:0/1790245206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec071ca0 con 0x7fc2d8004e60 2024-04-25T10:17:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 -- 172.21.15.60:0/1790245206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d800bcd0 con 0x7fc2d8004320 2024-04-25T10:17:02.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 -- 172.21.15.60:0/1790245206 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d800bed0 con 0x7fc2d80039c0 2024-04-25T10:17:02.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2ebfff640 1 --2- 172.21.15.60:0/1790245206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2d8004e60 0x7fc2d80052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eb7fe640 1 --2- 172.21.15.60:0/1790245206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2d80039c0 0x7fc2d8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2ebfff640 1 -- 172.21.15.60:0/1790245206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2d80039c0 msgr2=0x7fc2d8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:02.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2ebfff640 1 --2- 172.21.15.60:0/1790245206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2d80039c0 0x7fc2d8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2ebfff640 1 -- 172.21.15.60:0/1790245206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2d8004320 msgr2=0x7fc2d8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:02.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2e3fff640 1 --2- 172.21.15.60:0/1790245206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2d8004320 0x7fc2d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2ebfff640 1 --2- 172.21.15.60:0/1790245206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2d8004320 0x7fc2d8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2ebfff640 1 -- 172.21.15.60:0/1790245206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d8003530 con 0x7fc2d8004e60 2024-04-25T10:17:02.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eb7fe640 1 --2- 172.21.15.60:0/1790245206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2d80039c0 0x7fc2d8003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:02.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2e3fff640 1 --2- 172.21.15.60:0/1790245206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2d8004320 0x7fc2d8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:02.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2ebfff640 1 --2- 172.21.15.60:0/1790245206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2d8004e60 0x7fc2d80052e0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4009f90 tx=0x7fc2e401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b9d1d580173d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:02.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 -- 172.21.15.60:0/1790245206 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e40b2070 con 0x7fc2d8004e60 2024-04-25T10:17:02.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 -- 172.21.15.60:0/1790245206 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e40a5c50 con 0x7fc2d8004e60 2024-04-25T10:17:02.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.361+0000 7fc2eaffd640 1 -- 172.21.15.60:0/1790245206 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4009c10 con 0x7fc2d8004e60 2024-04-25T10:17:02.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 -- 172.21.15.60:0/1790245206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2d8004e60 msgr2=0x7fc2d80052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 --2- 172.21.15.60:0/1790245206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2d8004e60 0x7fc2d80052e0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4009f90 tx=0x7fc2e401cc40 comp rx=0 tx=0).stop 2024-04-25T10:17:02.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2ebfff640 1 -- 172.21.15.60:0/1790245206 reap_dead start 2024-04-25T10:17:02.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 -- 172.21.15.60:0/1790245206 shutdown_connections 2024-04-25T10:17:02.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 -- 172.21.15.60:0/1790245206 >> 172.21.15.60:0/1790245206 conn(0x7fc2ec0b12d0 msgr2=0x7fc2ec0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:02.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 -- 172.21.15.60:0/1790245206 shutdown_connections 2024-04-25T10:17:02.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 -- 172.21.15.60:0/1790245206 wait complete. 2024-04-25T10:17:02.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 Processor -- start 2024-04-25T10:17:02.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 -- start start 2024-04-25T10:17:02.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2d8004e60 0x7fc2ec0cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2ec157580 0x7fc2ec157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2ec06e630 0x7fc2ec06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec0b3ba0 con 0x7fc2ec157580 2024-04-25T10:17:02.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec071f80 con 0x7fc2d8004e60 2024-04-25T10:17:02.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec0b3d20 con 0x7fc2ec06e630 2024-04-25T10:17:02.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2e3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2ec157580 0x7fc2ec157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2e3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2ec157580 0x7fc2ec157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53056/0 (socket says 172.21.15.60:53056) 2024-04-25T10:17:02.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2e3fff640 1 -- 172.21.15.60:0/3433931667 learned_addr learned my addr 172.21.15.60:0/3433931667 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:02.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2ebfff640 1 --2- 172.21.15.60:0/3433931667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2ec06e630 0x7fc2ec06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2eb7fe640 1 --2- 172.21.15.60:0/3433931667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2d8004e60 0x7fc2ec0cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2e3fff640 1 -- 172.21.15.60:0/3433931667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2ec06e630 msgr2=0x7fc2ec06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2e3fff640 1 --2- 172.21.15.60:0/3433931667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2ec06e630 0x7fc2ec06eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2e3fff640 1 -- 172.21.15.60:0/3433931667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2d8004e60 msgr2=0x7fc2ec0cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2e3fff640 1 --2- 172.21.15.60:0/3433931667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2d8004e60 0x7fc2ec0cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2e3fff640 1 -- 172.21.15.60:0/3433931667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec0cb350 con 0x7fc2ec157580 2024-04-25T10:17:02.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2ebfff640 1 --2- 172.21.15.60:0/3433931667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2ec06e630 0x7fc2ec06eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:02.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2e3fff640 1 --2- 172.21.15.60:0/3433931667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2ec157580 0x7fc2ec157a00 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fc2d400dae0 tx=0x7fc2d4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:02.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2e8ff9640 1 -- 172.21.15.60:0/3433931667 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4019070 con 0x7fc2ec157580 2024-04-25T10:17:02.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2e8ff9640 1 -- 172.21.15.60:0/3433931667 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d40074e0 con 0x7fc2ec157580 2024-04-25T10:17:02.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 -- 172.21.15.60:0/3433931667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ec158020 con 0x7fc2ec157580 2024-04-25T10:17:02.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2e8ff9640 1 -- 172.21.15.60:0/3433931667 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d40049f0 con 0x7fc2ec157580 2024-04-25T10:17:02.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 -- 172.21.15.60:0/3433931667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ec1585e0 con 0x7fc2ec157580 2024-04-25T10:17:02.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.365+0000 7fc2f1f56640 1 -- 172.21.15.60:0/3433931667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ec074770 con 0x7fc2ec157580 2024-04-25T10:17:02.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.369+0000 7fc2e8ff9640 1 -- 172.21.15.60:0/3433931667 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc2d401c070 con 0x7fc2ec157580 2024-04-25T10:17:02.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.369+0000 7fc2e8ff9640 1 --2- 172.21.15.60:0/3433931667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2cc066f60 0x7fc2cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.369+0000 7fc2e8ff9640 1 -- 172.21.15.60:0/3433931667 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fc2d4084c20 con 0x7fc2ec157580 2024-04-25T10:17:02.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.369+0000 7fc2eb7fe640 1 --2- 172.21.15.60:0/3433931667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2cc066f60 0x7fc2cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.369+0000 7fc2eb7fe640 1 --2- 172.21.15.60:0/3433931667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2cc066f60 0x7fc2cc069420 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009b40 tx=0x7fc2dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:02.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.373+0000 7fc2e8ff9640 1 -- 172.21.15.60:0/3433931667 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc2d4055cd0 con 0x7fc2ec157580 2024-04-25T10:17:02.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.561+0000 7fc2f1f56640 1 -- 172.21.15.60:0/3433931667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2ec0adac0 con 0x7fc2ec157580 2024-04-25T10:17:02.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.561+0000 7fc2e8ff9640 1 -- 172.21.15.60:0/3433931667 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+4951 (secure 0 0 0) 0x7fc2d40555d0 con 0x7fc2ec157580 2024-04-25T10:17:02.567 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:17:02.567 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":106,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2440975447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2440975447},{"type":"v1","addr":"172.21.15.100:6835","nonce":2440975447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2398080758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2398080758},{"type":"v1","addr":"172.21.15.60:6833","nonce":2398080758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2759861958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2759861958},{"type":"v1","addr":"172.21.15.100:6837","nonce":2759861958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:07.763322+0000","modified":"2024-04-25T10:16:52.349719+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9894":{"gid":9894,"name":"c","rank":0,"incarnation":103,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/3461876440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3461876440},{"type":"v1","addr":"172.21.15.60:6835","nonce":3461876440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:17:02.567 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 106 2024-04-25T10:17:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.561+0000 7fc2f1f56640 1 -- 172.21.15.60:0/3433931667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2cc066f60 msgr2=0x7fc2cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.561+0000 7fc2f1f56640 1 --2- 172.21.15.60:0/3433931667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2cc066f60 0x7fc2cc069420 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009b40 tx=0x7fc2dc008040 comp rx=0 tx=0).stop 2024-04-25T10:17:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.561+0000 7fc2f1f56640 1 -- 172.21.15.60:0/3433931667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2ec157580 msgr2=0x7fc2ec157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.561+0000 7fc2f1f56640 1 --2- 172.21.15.60:0/3433931667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2ec157580 0x7fc2ec157a00 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fc2d400dae0 tx=0x7fc2d4007e00 comp rx=0 tx=0).stop 2024-04-25T10:17:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.561+0000 7fc2f1f56640 1 -- 172.21.15.60:0/3433931667 shutdown_connections 2024-04-25T10:17:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.561+0000 7fc2f1f56640 1 --2- 172.21.15.60:0/3433931667 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fc2cc066f60 0x7fc2cc069420 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.561+0000 7fc2f1f56640 1 --2- 172.21.15.60:0/3433931667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2ec06e630 0x7fc2ec06eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.561+0000 7fc2f1f56640 1 --2- 172.21.15.60:0/3433931667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2ec157580 0x7fc2ec157a00 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.561+0000 7fc2f1f56640 1 --2- 172.21.15.60:0/3433931667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2d8004e60 0x7fc2ec0cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.565+0000 7fc2f1f56640 1 -- 172.21.15.60:0/3433931667 >> 172.21.15.60:0/3433931667 conn(0x7fc2ec0b12d0 msgr2=0x7fc2ec156240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.565+0000 7fc2f1f56640 1 -- 172.21.15.60:0/3433931667 shutdown_connections 2024-04-25T10:17:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.565+0000 7fc2f1f56640 1 -- 172.21.15.60:0/3433931667 wait complete. 2024-04-25T10:17:02.585 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:02.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.697+0000 7fa15db61640 1 Processor -- start 2024-04-25T10:17:02.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.697+0000 7fa15db61640 1 -- start start 2024-04-25T10:17:02.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.701+0000 7fa15db61640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa158155f50 0x7fa1580af130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.701+0000 7fa15db61640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15806dc60 con 0x7fa1580701b0 2024-04-25T10:17:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.701+0000 7fa15db61640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15806df40 con 0x7fa15806c330 2024-04-25T10:17:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.701+0000 7fa15db61640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15806e870 con 0x7fa158155f50 2024-04-25T10:17:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.701+0000 7fa156ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa1580701b0 0x7fa15806bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57552/0 (socket says 172.21.15.60:57552) 2024-04-25T10:17:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.701+0000 7fa156ffd640 1 -- 172.21.15.60:0/3305086239 learned_addr learned my addr 172.21.15.60:0/3305086239 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.701+0000 7fa1577fe640 1 --2- 172.21.15.60:0/3305086239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa158155f50 0x7fa1580af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.701+0000 7fa1567fc640 1 -- 172.21.15.60:0/3305086239 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016780823 0 0) 0x7fa15806dc60 con 0x7fa1580701b0 2024-04-25T10:17:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.701+0000 7fa1567fc640 1 -- 172.21.15.60:0/3305086239 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa140003530 con 0x7fa1580701b0 2024-04-25T10:17:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.701+0000 7fa1577fe640 1 -- 172.21.15.60:0/3305086239 >> v1:172.21.15.100:6789/0 conn(0x7fa15806c330 legacy=0x7fa1580b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.701+0000 7fa1577fe640 1 -- 172.21.15.60:0/3305086239 >> v1:172.21.15.60:6789/0 conn(0x7fa1580701b0 legacy=0x7fa15806bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.701+0000 7fa1577fe640 1 -- 172.21.15.60:0/3305086239 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15806e220 con 0x7fa158155f50 2024-04-25T10:17:02.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.705+0000 7fa1577fe640 1 --2- 172.21.15.60:0/3305086239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa158155f50 0x7fa1580af130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0097f0 tx=0x7fa14c01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3275270888d75903 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:02.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.705+0000 7fa1567fc640 1 -- 172.21.15.60:0/3305086239 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c0aa5f0 con 0x7fa158155f50 2024-04-25T10:17:02.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.705+0000 7fa1567fc640 1 -- 172.21.15.60:0/3305086239 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14c0af070 con 0x7fa158155f50 2024-04-25T10:17:02.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.705+0000 7fa1567fc640 1 -- 172.21.15.60:0/3305086239 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c0aa990 con 0x7fa158155f50 2024-04-25T10:17:02.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.705+0000 7fa15db61640 1 -- 172.21.15.60:0/3305086239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa158155f50 msgr2=0x7fa1580af130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.705+0000 7fa15db61640 1 --2- 172.21.15.60:0/3305086239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa158155f50 0x7fa1580af130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0097f0 tx=0x7fa14c01c470 comp rx=0 tx=0).stop 2024-04-25T10:17:02.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.705+0000 7fa15db61640 1 -- 172.21.15.60:0/3305086239 shutdown_connections 2024-04-25T10:17:02.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.705+0000 7fa15db61640 1 --2- 172.21.15.60:0/3305086239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa158155f50 0x7fa1580af130 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.705+0000 7fa15db61640 1 -- 172.21.15.60:0/3305086239 >> 172.21.15.60:0/3305086239 conn(0x7fa158071b90 msgr2=0x7fa158071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:02.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.705+0000 7fa15db61640 1 -- 172.21.15.60:0/3305086239 shutdown_connections 2024-04-25T10:17:02.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.705+0000 7fa15db61640 1 -- 172.21.15.60:0/3305086239 wait complete. 2024-04-25T10:17:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.705+0000 7fa15db61640 1 Processor -- start 2024-04-25T10:17:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.705+0000 7fa15db61640 1 -- start start 2024-04-25T10:17:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa15db61640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa15806c330 0x7fa158146780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa15db61640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1580701b0 0x7fa158147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa15db61640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa158149400 0x7fa158158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa15db61640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580b40b0 con 0x7fa1580701b0 2024-04-25T10:17:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa15db61640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580b3f30 con 0x7fa158149400 2024-04-25T10:17:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa15db61640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa158169410 con 0x7fa15806c330 2024-04-25T10:17:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa156ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1580701b0 0x7fa158147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa156ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1580701b0 0x7fa158147890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53064/0 (socket says 172.21.15.60:53064) 2024-04-25T10:17:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa156ffd640 1 -- 172.21.15.60:0/2652463632 learned_addr learned my addr 172.21.15.60:0/2652463632 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa157fff640 1 --2- 172.21.15.60:0/2652463632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa158149400 0x7fa158158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa1577fe640 1 --2- 172.21.15.60:0/2652463632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa15806c330 0x7fa158146780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa156ffd640 1 -- 172.21.15.60:0/2652463632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa15806c330 msgr2=0x7fa158146780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa156ffd640 1 --2- 172.21.15.60:0/2652463632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa15806c330 0x7fa158146780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa156ffd640 1 -- 172.21.15.60:0/2652463632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa158149400 msgr2=0x7fa158158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa156ffd640 1 --2- 172.21.15.60:0/2652463632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa158149400 0x7fa158158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa156ffd640 1 -- 172.21.15.60:0/2652463632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa158146cc0 con 0x7fa1580701b0 2024-04-25T10:17:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa157fff640 1 --2- 172.21.15.60:0/2652463632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa158149400 0x7fa158158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa156ffd640 1 --2- 172.21.15.60:0/2652463632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1580701b0 0x7fa158147890 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fa1480010f0 tx=0x7fa14800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa13ffff640 1 -- 172.21.15.60:0/2652463632 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148017070 con 0x7fa1580701b0 2024-04-25T10:17:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa13ffff640 1 -- 172.21.15.60:0/2652463632 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa148005d10 con 0x7fa1580701b0 2024-04-25T10:17:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa13ffff640 1 -- 172.21.15.60:0/2652463632 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148007310 con 0x7fa1580701b0 2024-04-25T10:17:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa15db61640 1 -- 172.21.15.60:0/2652463632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1581491b0 con 0x7fa1580701b0 2024-04-25T10:17:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa15db61640 1 -- 172.21.15.60:0/2652463632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1581483f0 con 0x7fa1580701b0 2024-04-25T10:17:02.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa15db61640 1 -- 172.21.15.60:0/2652463632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa158074770 con 0x7fa1580701b0 2024-04-25T10:17:02.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.709+0000 7fa13ffff640 1 -- 172.21.15.60:0/2652463632 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa14801a070 con 0x7fa1580701b0 2024-04-25T10:17:02.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.713+0000 7fa13ffff640 1 --2- 172.21.15.60:0/2652463632 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa144066f90 0x7fa144069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.713+0000 7fa13ffff640 1 -- 172.21.15.60:0/2652463632 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fa148084930 con 0x7fa1580701b0 2024-04-25T10:17:02.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.713+0000 7fa1577fe640 1 --2- 172.21.15.60:0/2652463632 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa144066f90 0x7fa144069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.713+0000 7fa1577fe640 1 --2- 172.21.15.60:0/2652463632 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa144066f90 0x7fa144069450 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0ae000 tx=0x7fa14c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:02.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.717+0000 7fa13ffff640 1 -- 172.21.15.60:0/2652463632 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa148021020 con 0x7fa1580701b0 2024-04-25T10:17:02.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.869+0000 7fa15db61640 1 -- 172.21.15.60:0/2652463632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1580afba0 con 0x7fa1580701b0 2024-04-25T10:17:02.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.869+0000 7fa13ffff640 1 -- 172.21.15.60:0/2652463632 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17048 (secure 0 0 0) 0x7fa148055930 con 0x7fa1580701b0 2024-04-25T10:17:02.876 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:17:02.876 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":113,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:17:02.691740+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:06.033387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:06.753421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.25","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/178428701":"2024-04-26T10:16:48.303277+0000","172.21.15.60:6834/178428701":"2024-04-26T10:16:48.303277+0000","172.21.15.60:6835/722171246":"2024-04-26T10:16:28.355776+0000","172.21.15.60:6834/722171246":"2024-04-26T10:16:28.355776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.873+0000 7fa15db61640 1 -- 172.21.15.60:0/2652463632 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa144066f90 msgr2=0x7fa144069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.873+0000 7fa15db61640 1 --2- 172.21.15.60:0/2652463632 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa144066f90 0x7fa144069450 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0ae000 tx=0x7fa14c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:17:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.873+0000 7fa15db61640 1 -- 172.21.15.60:0/2652463632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1580701b0 msgr2=0x7fa158147890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.873+0000 7fa15db61640 1 --2- 172.21.15.60:0/2652463632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1580701b0 0x7fa158147890 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fa1480010f0 tx=0x7fa14800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.873+0000 7fa15db61640 1 -- 172.21.15.60:0/2652463632 shutdown_connections 2024-04-25T10:17:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.873+0000 7fa15db61640 1 --2- 172.21.15.60:0/2652463632 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa144066f90 0x7fa144069450 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.873+0000 7fa15db61640 1 --2- 172.21.15.60:0/2652463632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa158149400 0x7fa158158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.873+0000 7fa15db61640 1 --2- 172.21.15.60:0/2652463632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1580701b0 0x7fa158147890 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.873+0000 7fa15db61640 1 --2- 172.21.15.60:0/2652463632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa15806c330 0x7fa158146780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.873+0000 7fa15db61640 1 -- 172.21.15.60:0/2652463632 >> 172.21.15.60:0/2652463632 conn(0x7fa158071b90 msgr2=0x7fa15806b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:02.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.873+0000 7fa15db61640 1 -- 172.21.15.60:0/2652463632 shutdown_connections 2024-04-25T10:17:02.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.873+0000 7fa15db61640 1 -- 172.21.15.60:0/2652463632 wait complete. 2024-04-25T10:17:02.894 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:17:02.894 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:03.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:02.997+0000 7f46eebf8640 1 Processor -- start 2024-04-25T10:17:03.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46eebf8640 1 -- start start 2024-04-25T10:17:03.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46eebf8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f46e80ce7a0 0x7f46e80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46eebf8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f46e815bc90 0x7f46e80b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46eebf8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e80b4660 0x7f46e80b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46eebf8640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e80cc370 con 0x7f46e815bc90 2024-04-25T10:17:03.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46eebf8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e80cf0c0 con 0x7f46e80ce7a0 2024-04-25T10:17:03.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46eebf8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e8071ca0 con 0x7f46e80b4660 2024-04-25T10:17:03.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46e7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f46e815bc90 0x7f46e80b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46ec994640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f46e80ce7a0 0x7f46e80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46e7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f46e815bc90 0x7f46e80b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53076/0 (socket says 172.21.15.60:53076) 2024-04-25T10:17:03.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46e7fff640 1 -- 172.21.15.60:0/1955446728 learned_addr learned my addr 172.21.15.60:0/1955446728 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:03.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46ed195640 1 --2- 172.21.15.60:0/1955446728 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e80b4660 0x7f46e80b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46e7fff640 1 -- 172.21.15.60:0/1955446728 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e80b4660 msgr2=0x7f46e80b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:03.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46e7fff640 1 --2- 172.21.15.60:0/1955446728 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e80b4660 0x7f46e80b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46e7fff640 1 -- 172.21.15.60:0/1955446728 >> v2:172.21.15.100:3300/0 conn(0x7f46e80ce7a0 msgr2=0x7f46e80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:03.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46e7fff640 1 --2- 172.21.15.60:0/1955446728 >> v2:172.21.15.100:3300/0 conn(0x7f46e80ce7a0 0x7f46e80ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.001+0000 7f46e7fff640 1 -- 172.21.15.60:0/1955446728 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e80cca40 con 0x7f46e815bc90 2024-04-25T10:17:03.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46e7fff640 1 --2- 172.21.15.60:0/1955446728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e815bc90 0x7f46e80b25d0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f46e804f5d0 tx=0x7f46dc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3b59dd4626b0fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46e77fe640 1 -- 172.21.15.60:0/1955446728 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc0b0070 con 0x7f46e815bc90 2024-04-25T10:17:03.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46e77fe640 1 -- 172.21.15.60:0/1955446728 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46dc0ab600 con 0x7f46e815bc90 2024-04-25T10:17:03.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46e77fe640 1 -- 172.21.15.60:0/1955446728 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc0ab9d0 con 0x7f46e815bc90 2024-04-25T10:17:03.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 -- 172.21.15.60:0/1955446728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e815bc90 msgr2=0x7f46e80b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 --2- 172.21.15.60:0/1955446728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e815bc90 0x7f46e80b25d0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f46e804f5d0 tx=0x7f46dc01c520 comp rx=0 tx=0).stop 2024-04-25T10:17:03.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 -- 172.21.15.60:0/1955446728 shutdown_connections 2024-04-25T10:17:03.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 --2- 172.21.15.60:0/1955446728 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e80b4660 0x7f46e80b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 --2- 172.21.15.60:0/1955446728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e815bc90 0x7f46e80b25d0 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 --2- 172.21.15.60:0/1955446728 >> v2:172.21.15.100:3300/0 conn(0x7f46e80ce7a0 0x7f46e80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 -- 172.21.15.60:0/1955446728 >> 172.21.15.60:0/1955446728 conn(0x7f46e80b12d0 msgr2=0x7f46e80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:03.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 -- 172.21.15.60:0/1955446728 shutdown_connections 2024-04-25T10:17:03.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 -- 172.21.15.60:0/1955446728 wait complete. 2024-04-25T10:17:03.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 Processor -- start 2024-04-25T10:17:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 -- start start 2024-04-25T10:17:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46e80ce7a0 0x7f46e8152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e815bc90 0x7f46e81fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e8148730 0x7f46e806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e80b4280 con 0x7f46e8148730 2024-04-25T10:17:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e80b3460 con 0x7f46e80ce7a0 2024-04-25T10:17:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46eebf8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e80b4400 con 0x7f46e815bc90 2024-04-25T10:17:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46ed195640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e8148730 0x7f46e806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46ed195640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e8148730 0x7f46e806e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53090/0 (socket says 172.21.15.60:53090) 2024-04-25T10:17:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46ed195640 1 -- 172.21.15.60:0/3460849583 learned_addr learned my addr 172.21.15.60:0/3460849583 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46e7fff640 1 --2- 172.21.15.60:0/3460849583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e815bc90 0x7f46e81fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46ec994640 1 --2- 172.21.15.60:0/3460849583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46e80ce7a0 0x7f46e8152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46ed195640 1 -- 172.21.15.60:0/3460849583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e815bc90 msgr2=0x7f46e81fb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46ed195640 1 --2- 172.21.15.60:0/3460849583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e815bc90 0x7f46e81fb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46ed195640 1 -- 172.21.15.60:0/3460849583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46e80ce7a0 msgr2=0x7f46e8152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46ed195640 1 --2- 172.21.15.60:0/3460849583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46e80ce7a0 0x7f46e8152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46ed195640 1 -- 172.21.15.60:0/3460849583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e8152e80 con 0x7f46e8148730 2024-04-25T10:17:03.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46e7fff640 1 --2- 172.21.15.60:0/3460849583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e815bc90 0x7f46e81fb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:03.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46ec994640 1 --2- 172.21.15.60:0/3460849583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46e80ce7a0 0x7f46e8152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:03.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.005+0000 7f46ed195640 1 --2- 172.21.15.60:0/3460849583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e8148730 0x7f46e806e340 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f46e000ef00 tx=0x7f46e000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.009+0000 7f46e57fa640 1 -- 172.21.15.60:0/3460849583 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0018020 con 0x7f46e8148730 2024-04-25T10:17:03.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.009+0000 7f46e57fa640 1 -- 172.21.15.60:0/3460849583 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e000bd30 con 0x7f46e8148730 2024-04-25T10:17:03.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.009+0000 7f46e57fa640 1 -- 172.21.15.60:0/3460849583 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0014640 con 0x7f46e8148730 2024-04-25T10:17:03.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.009+0000 7f46eebf8640 1 -- 172.21.15.60:0/3460849583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e8153160 con 0x7f46e8148730 2024-04-25T10:17:03.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.009+0000 7f46eebf8640 1 -- 172.21.15.60:0/3460849583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e8153720 con 0x7f46e8148730 2024-04-25T10:17:03.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.009+0000 7f46eebf8640 1 -- 172.21.15.60:0/3460849583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e8074770 con 0x7f46e8148730 2024-04-25T10:17:03.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.009+0000 7f46e57fa640 1 -- 172.21.15.60:0/3460849583 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46e001b020 con 0x7f46e8148730 2024-04-25T10:17:03.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.009+0000 7f46e57fa640 1 --2- 172.21.15.60:0/3460849583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46d4066f60 0x7f46d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.009+0000 7f46e57fa640 1 -- 172.21.15.60:0/3460849583 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f46e00859e0 con 0x7f46e8148730 2024-04-25T10:17:03.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.009+0000 7f46ec994640 1 --2- 172.21.15.60:0/3460849583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46d4066f60 0x7f46d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.009+0000 7f46ec994640 1 --2- 172.21.15.60:0/3460849583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46d4066f60 0x7f46d4069420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f46e8156d20 tx=0x7f46d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.013+0000 7f46e57fa640 1 -- 172.21.15.60:0/3460849583 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46e0022030 con 0x7f46e8148730 2024-04-25T10:17:03.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.201+0000 7f46eebf8640 1 -- 172.21.15.60:0/3460849583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46e815b240 con 0x7f46e8148730 2024-04-25T10:17:03.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.201+0000 7f46e57fa640 1 -- 172.21.15.60:0/3460849583 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f46e00568f0 con 0x7f46e8148730 2024-04-25T10:17:03.209 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:17:03.209 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":106,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2440975447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2440975447},{"type":"v1","addr":"172.21.15.100:6835","nonce":2440975447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2398080758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2398080758},{"type":"v1","addr":"172.21.15.60:6833","nonce":2398080758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2759861958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2759861958},{"type":"v1","addr":"172.21.15.100:6837","nonce":2759861958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:07.763322+0000","modified":"2024-04-25T10:16:52.349719+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9894":{"gid":9894,"name":"c","rank":0,"incarnation":103,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/3461876440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3461876440},{"type":"v1","addr":"172.21.15.60:6835","nonce":3461876440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:17:03.209 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 106 2024-04-25T10:17:03.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.205+0000 7f46eebf8640 1 -- 172.21.15.60:0/3460849583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46d4066f60 msgr2=0x7f46d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.205+0000 7f46eebf8640 1 --2- 172.21.15.60:0/3460849583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46d4066f60 0x7f46d4069420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f46e8156d20 tx=0x7f46d8008040 comp rx=0 tx=0).stop 2024-04-25T10:17:03.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.205+0000 7f46eebf8640 1 -- 172.21.15.60:0/3460849583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e8148730 msgr2=0x7f46e806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.205+0000 7f46eebf8640 1 --2- 172.21.15.60:0/3460849583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e8148730 0x7f46e806e340 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f46e000ef00 tx=0x7f46e000b480 comp rx=0 tx=0).stop 2024-04-25T10:17:03.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.205+0000 7f46eebf8640 1 -- 172.21.15.60:0/3460849583 shutdown_connections 2024-04-25T10:17:03.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.205+0000 7f46eebf8640 1 --2- 172.21.15.60:0/3460849583 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46d4066f60 0x7f46d4069420 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.205+0000 7f46eebf8640 1 --2- 172.21.15.60:0/3460849583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46e8148730 0x7f46e806e340 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.205+0000 7f46eebf8640 1 --2- 172.21.15.60:0/3460849583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e815bc90 0x7f46e81fb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.205+0000 7f46eebf8640 1 --2- 172.21.15.60:0/3460849583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46e80ce7a0 0x7f46e8152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.205+0000 7f46eebf8640 1 -- 172.21.15.60:0/3460849583 >> 172.21.15.60:0/3460849583 conn(0x7f46e80b12d0 msgr2=0x7f46e80af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:03.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.205+0000 7f46eebf8640 1 -- 172.21.15.60:0/3460849583 shutdown_connections 2024-04-25T10:17:03.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.205+0000 7f46eebf8640 1 -- 172.21.15.60:0/3460849583 wait complete. 2024-04-25T10:17:03.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:03.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46f6bf2640 1 Processor -- start 2024-04-25T10:17:03.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46f6bf2640 1 -- start start 2024-04-25T10:17:03.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46f6bf2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f46f015a7a0 0x7f46f00cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46f6bf2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f46f006ec30 0x7f46f0071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46f6bf2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f00b37e0 0x7f46f00b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46f6bf2640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f00cdaa0 con 0x7f46f006ec30 2024-04-25T10:17:03.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46f6bf2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f006d2f0 con 0x7f46f015a7a0 2024-04-25T10:17:03.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46f6bf2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f006d8b0 con 0x7f46f00b37e0 2024-04-25T10:17:03.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46effff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f46f006ec30 0x7f46f0071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46effff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f46f006ec30 0x7f46f0071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53092/0 (socket says 172.21.15.60:53092) 2024-04-25T10:17:03.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46effff640 1 -- 172.21.15.60:0/719143238 learned_addr learned my addr 172.21.15.60:0/719143238 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:03.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46f518f640 1 --2- 172.21.15.60:0/719143238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f00b37e0 0x7f46f00b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46f498e640 1 --2- 172.21.15.60:0/719143238 >> v2:172.21.15.100:3300/0 conn(0x7f46f015a7a0 0x7f46f00cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46effff640 1 -- 172.21.15.60:0/719143238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f00b37e0 msgr2=0x7f46f00b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:03.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46effff640 1 --2- 172.21.15.60:0/719143238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f00b37e0 0x7f46f00b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46effff640 1 -- 172.21.15.60:0/719143238 >> v2:172.21.15.100:3300/0 conn(0x7f46f015a7a0 msgr2=0x7f46f00cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:03.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46effff640 1 --2- 172.21.15.60:0/719143238 >> v2:172.21.15.100:3300/0 conn(0x7f46f015a7a0 0x7f46f00cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.333+0000 7f46effff640 1 -- 172.21.15.60:0/719143238 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f00721e0 con 0x7f46f006ec30 2024-04-25T10:17:03.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46effff640 1 --2- 172.21.15.60:0/719143238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f006ec30 0x7f46f0071ca0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f46f004f5d0 tx=0x7f46dc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1754e8133186db0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46ef7fe640 1 -- 172.21.15.60:0/719143238 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc0ab5f0 con 0x7f46f006ec30 2024-04-25T10:17:03.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46ef7fe640 1 -- 172.21.15.60:0/719143238 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46dc0b0070 con 0x7f46f006ec30 2024-04-25T10:17:03.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46ef7fe640 1 -- 172.21.15.60:0/719143238 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc0ab9b0 con 0x7f46f006ec30 2024-04-25T10:17:03.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 -- 172.21.15.60:0/719143238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f006ec30 msgr2=0x7f46f0071ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 --2- 172.21.15.60:0/719143238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f006ec30 0x7f46f0071ca0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f46f004f5d0 tx=0x7f46dc01c520 comp rx=0 tx=0).stop 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 -- 172.21.15.60:0/719143238 shutdown_connections 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 --2- 172.21.15.60:0/719143238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f00b37e0 0x7f46f00b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 --2- 172.21.15.60:0/719143238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f006ec30 0x7f46f0071ca0 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 --2- 172.21.15.60:0/719143238 >> v2:172.21.15.100:3300/0 conn(0x7f46f015a7a0 0x7f46f00cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 -- 172.21.15.60:0/719143238 >> 172.21.15.60:0/719143238 conn(0x7f46f00b12d0 msgr2=0x7f46f00b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 -- 172.21.15.60:0/719143238 shutdown_connections 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 -- 172.21.15.60:0/719143238 wait complete. 2024-04-25T10:17:03.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 Processor -- start 2024-04-25T10:17:03.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 -- start start 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f006ec30 0x7f46f014ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f00b37e0 0x7f46f0159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f015a7a0 0x7f46f01598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f015ad00 con 0x7f46f006ec30 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f015ab80 con 0x7f46f015a7a0 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f6bf2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f006e530 con 0x7f46f00b37e0 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f498e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f006ec30 0x7f46f014ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f498e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f006ec30 0x7f46f014ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53102/0 (socket says 172.21.15.60:53102) 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f498e640 1 -- 172.21.15.60:0/3167312800 learned_addr learned my addr 172.21.15.60:0/3167312800 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46effff640 1 --2- 172.21.15.60:0/3167312800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f00b37e0 0x7f46f0159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f518f640 1 --2- 172.21.15.60:0/3167312800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f015a7a0 0x7f46f01598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f498e640 1 -- 172.21.15.60:0/3167312800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f00b37e0 msgr2=0x7f46f0159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f498e640 1 --2- 172.21.15.60:0/3167312800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f00b37e0 0x7f46f0159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f498e640 1 -- 172.21.15.60:0/3167312800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f015a7a0 msgr2=0x7f46f01598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f498e640 1 --2- 172.21.15.60:0/3167312800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f015a7a0 0x7f46f01598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f498e640 1 -- 172.21.15.60:0/3167312800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f014f760 con 0x7f46f006ec30 2024-04-25T10:17:03.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46effff640 1 --2- 172.21.15.60:0/3167312800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f00b37e0 0x7f46f0159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:03.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.337+0000 7f46f498e640 1 --2- 172.21.15.60:0/3167312800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f006ec30 0x7f46f014ece0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f46e400d5f0 tx=0x7f46e400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.341+0000 7f46ed7fa640 1 -- 172.21.15.60:0/3167312800 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e40125f0 con 0x7f46f006ec30 2024-04-25T10:17:03.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.341+0000 7f46ed7fa640 1 -- 172.21.15.60:0/3167312800 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e4019070 con 0x7f46f006ec30 2024-04-25T10:17:03.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.341+0000 7f46ed7fa640 1 -- 172.21.15.60:0/3167312800 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e40129b0 con 0x7f46f006ec30 2024-04-25T10:17:03.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.341+0000 7f46f6bf2640 1 -- 172.21.15.60:0/3167312800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f014fa40 con 0x7f46f006ec30 2024-04-25T10:17:03.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.341+0000 7f46f6bf2640 1 -- 172.21.15.60:0/3167312800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f0150000 con 0x7f46f006ec30 2024-04-25T10:17:03.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.341+0000 7f46f6bf2640 1 -- 172.21.15.60:0/3167312800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f0074770 con 0x7f46f006ec30 2024-04-25T10:17:03.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.341+0000 7f46ed7fa640 1 -- 172.21.15.60:0/3167312800 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46e4012b50 con 0x7f46f006ec30 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.341+0000 7f46ed7fa640 1 --2- 172.21.15.60:0/3167312800 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46d4066f60 0x7f46d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.341+0000 7f46ed7fa640 1 -- 172.21.15.60:0/3167312800 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f46e40854f0 con 0x7f46f006ec30 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.341+0000 7f46effff640 1 --2- 172.21.15.60:0/3167312800 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46d4066f60 0x7f46d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.341+0000 7f46effff640 1 --2- 172.21.15.60:0/3167312800 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46d4066f60 0x7f46d4069420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0af000 tx=0x7f46dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.345+0000 7f46ed7fa640 1 -- 172.21.15.60:0/3167312800 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46e40563f0 con 0x7f46f006ec30 2024-04-25T10:17:03.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.545+0000 7f46f6bf2640 1 -- 172.21.15.60:0/3167312800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46f00cf730 con 0x7f46f006ec30 2024-04-25T10:17:03.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.545+0000 7f46ed7fa640 1 -- 172.21.15.60:0/3167312800 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f46e4055cf0 con 0x7f46f006ec30 2024-04-25T10:17:03.552 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:17:03.552 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":106,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2440975447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2440975447},{"type":"v1","addr":"172.21.15.100:6835","nonce":2440975447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2398080758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2398080758},{"type":"v1","addr":"172.21.15.60:6833","nonce":2398080758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2759861958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2759861958},{"type":"v1","addr":"172.21.15.100:6837","nonce":2759861958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:07.763322+0000","modified":"2024-04-25T10:16:52.349719+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9894":{"gid":9894,"name":"c","rank":0,"incarnation":103,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/3461876440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3461876440},{"type":"v1","addr":"172.21.15.60:6835","nonce":3461876440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:17:03.552 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 106 2024-04-25T10:17:03.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.549+0000 7f46f6bf2640 1 -- 172.21.15.60:0/3167312800 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46d4066f60 msgr2=0x7f46d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.549+0000 7f46f6bf2640 1 --2- 172.21.15.60:0/3167312800 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46d4066f60 0x7f46d4069420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0af000 tx=0x7f46dc008040 comp rx=0 tx=0).stop 2024-04-25T10:17:03.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.549+0000 7f46f6bf2640 1 -- 172.21.15.60:0/3167312800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f006ec30 msgr2=0x7f46f014ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.549+0000 7f46f6bf2640 1 --2- 172.21.15.60:0/3167312800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f006ec30 0x7f46f014ece0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f46e400d5f0 tx=0x7f46e400dab0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.549+0000 7f46f6bf2640 1 -- 172.21.15.60:0/3167312800 shutdown_connections 2024-04-25T10:17:03.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.549+0000 7f46f6bf2640 1 --2- 172.21.15.60:0/3167312800 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f46d4066f60 0x7f46d4069420 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.549+0000 7f46f6bf2640 1 --2- 172.21.15.60:0/3167312800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f015a7a0 0x7f46f01598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.549+0000 7f46f6bf2640 1 --2- 172.21.15.60:0/3167312800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f00b37e0 0x7f46f0159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.549+0000 7f46f6bf2640 1 --2- 172.21.15.60:0/3167312800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f006ec30 0x7f46f014ece0 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.549+0000 7f46f6bf2640 1 -- 172.21.15.60:0/3167312800 >> 172.21.15.60:0/3167312800 conn(0x7f46f00b12d0 msgr2=0x7f46f00cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:03.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.549+0000 7f46f6bf2640 1 -- 172.21.15.60:0/3167312800 shutdown_connections 2024-04-25T10:17:03.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.549+0000 7f46f6bf2640 1 -- 172.21.15.60:0/3167312800 wait complete. 2024-04-25T10:17:03.570 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:03.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.673+0000 7f677acd7640 1 Processor -- start 2024-04-25T10:17:03.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.673+0000 7f677acd7640 1 -- start start 2024-04-25T10:17:03.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f677acd7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67740b4660 0x7f677415c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f677acd7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6774071ca0 con 0x7f67740ce7a0 2024-04-25T10:17:03.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f677acd7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67740cca40 con 0x7f677415bc90 2024-04-25T10:17:03.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f677acd7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67740cc370 con 0x7f67740b4660 2024-04-25T10:17:03.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f6778a73640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f67740ce7a0 0x7f67740ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57566/0 (socket says 172.21.15.60:57566) 2024-04-25T10:17:03.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f6778a73640 1 -- 172.21.15.60:0/3638383371 learned_addr learned my addr 172.21.15.60:0/3638383371 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:03.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f67737fe640 1 -- 172.21.15.60:0/3638383371 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016594450 0 0) 0x7f6774071ca0 con 0x7f67740ce7a0 2024-04-25T10:17:03.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f67737fe640 1 -- 172.21.15.60:0/3638383371 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6764003530 con 0x7f67740ce7a0 2024-04-25T10:17:03.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f6779274640 1 --2- 172.21.15.60:0/3638383371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67740b4660 0x7f677415c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f67737fe640 1 -- 172.21.15.60:0/3638383371 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2075261777 0 0) 0x7f6764003530 con 0x7f67740ce7a0 2024-04-25T10:17:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f67737fe640 1 -- 172.21.15.60:0/3638383371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67740b4660 msgr2=0x7f677415c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f67737fe640 1 --2- 172.21.15.60:0/3638383371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67740b4660 0x7f677415c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f67737fe640 1 -- 172.21.15.60:0/3638383371 >> v1:172.21.15.100:6789/0 conn(0x7f677415bc90 legacy=0x7f677415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f67737fe640 1 -- 172.21.15.60:0/3638383371 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67740ccd20 con 0x7f67740ce7a0 2024-04-25T10:17:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f67737fe640 1 -- 172.21.15.60:0/3638383371 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f6768001680 con 0x7f67740ce7a0 2024-04-25T10:17:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f67737fe640 1 -- 172.21.15.60:0/3638383371 >> v1:172.21.15.60:6789/0 conn(0x7f67740ce7a0 legacy=0x7f67740ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f67737fe640 1 --2- 172.21.15.60:0/3638383371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67640039f0 0x7f6764003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f67737fe640 1 --2- 172.21.15.60:0/3638383371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67640043a0 0x7f6764004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f67737fe640 1 --2- 172.21.15.60:0/3638383371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6764005070 0x7f676400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f67737fe640 1 -- 172.21.15.60:0/3638383371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676400bf10 con 0x7f67640043a0 2024-04-25T10:17:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f67737fe640 1 -- 172.21.15.60:0/3638383371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676400c090 con 0x7f6764005070 2024-04-25T10:17:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f67737fe640 1 -- 172.21.15.60:0/3638383371 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676400c260 con 0x7f67640039f0 2024-04-25T10:17:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f6773fff640 1 --2- 172.21.15.60:0/3638383371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67640043a0 0x7f6764004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f6778a73640 1 --2- 172.21.15.60:0/3638383371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67640039f0 0x7f6764003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f6779274640 1 --2- 172.21.15.60:0/3638383371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6764005070 0x7f676400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f6773fff640 1 -- 172.21.15.60:0/3638383371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67640039f0 msgr2=0x7f6764003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:03.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f6773fff640 1 --2- 172.21.15.60:0/3638383371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67640039f0 0x7f6764003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f6773fff640 1 -- 172.21.15.60:0/3638383371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6764005070 msgr2=0x7f676400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:03.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f6773fff640 1 --2- 172.21.15.60:0/3638383371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6764005070 0x7f676400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f6773fff640 1 -- 172.21.15.60:0/3638383371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6764003530 con 0x7f67640043a0 2024-04-25T10:17:03.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.677+0000 7f6779274640 1 --2- 172.21.15.60:0/3638383371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6764005070 0x7f676400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:03.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f6773fff640 1 --2- 172.21.15.60:0/3638383371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67640043a0 0x7f6764004870 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f6760001780 tx=0x7f676001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1d9253006f2356 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f67737fe640 1 -- 172.21.15.60:0/3638383371 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67600013a0 con 0x7f67640043a0 2024-04-25T10:17:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f67737fe640 1 -- 172.21.15.60:0/3638383371 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67600052f0 con 0x7f67640043a0 2024-04-25T10:17:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f67737fe640 1 -- 172.21.15.60:0/3638383371 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760003840 con 0x7f67640043a0 2024-04-25T10:17:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f677acd7640 1 -- 172.21.15.60:0/3638383371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67640043a0 msgr2=0x7f6764004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f677acd7640 1 --2- 172.21.15.60:0/3638383371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67640043a0 0x7f6764004870 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f6760001780 tx=0x7f676001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f6779274640 1 -- 172.21.15.60:0/3638383371 reap_dead start 2024-04-25T10:17:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f677acd7640 1 -- 172.21.15.60:0/3638383371 shutdown_connections 2024-04-25T10:17:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f677acd7640 1 -- 172.21.15.60:0/3638383371 >> 172.21.15.60:0/3638383371 conn(0x7f67740b12d0 msgr2=0x7f67740b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f677acd7640 1 -- 172.21.15.60:0/3638383371 shutdown_connections 2024-04-25T10:17:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f677acd7640 1 -- 172.21.15.60:0/3638383371 wait complete. 2024-04-25T10:17:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f677acd7640 1 Processor -- start 2024-04-25T10:17:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f677acd7640 1 -- start start 2024-04-25T10:17:03.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f677acd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67640043a0 0x7f67741f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f677acd7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67740ae000 0x7f677406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f677acd7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67741f37a0 0x7f67741f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f677acd7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67740b3d20 con 0x7f67740ae000 2024-04-25T10:17:03.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f677acd7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67740b3ba0 con 0x7f67640043a0 2024-04-25T10:17:03.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f677acd7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67740b4280 con 0x7f67741f37a0 2024-04-25T10:17:03.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f6773fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67740ae000 0x7f677406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f6773fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67740ae000 0x7f677406e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53118/0 (socket says 172.21.15.60:53118) 2024-04-25T10:17:03.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f6773fff640 1 -- 172.21.15.60:0/1322299861 learned_addr learned my addr 172.21.15.60:0/1322299861 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:03.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f6778a73640 1 --2- 172.21.15.60:0/1322299861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67640043a0 0x7f67741f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f6779274640 1 --2- 172.21.15.60:0/1322299861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67741f37a0 0x7f67741f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f6773fff640 1 -- 172.21.15.60:0/1322299861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67741f37a0 msgr2=0x7f67741f3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f6773fff640 1 --2- 172.21.15.60:0/1322299861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67741f37a0 0x7f67741f3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f6773fff640 1 -- 172.21.15.60:0/1322299861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67640043a0 msgr2=0x7f67741f6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f6773fff640 1 --2- 172.21.15.60:0/1322299861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67640043a0 0x7f67741f6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f6773fff640 1 -- 172.21.15.60:0/1322299861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67741f75b0 con 0x7f67740ae000 2024-04-25T10:17:03.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.681+0000 7f6778a73640 1 --2- 172.21.15.60:0/1322299861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67640043a0 0x7f67741f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:03.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.685+0000 7f6773fff640 1 --2- 172.21.15.60:0/1322299861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67740ae000 0x7f677406e920 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f6760001780 tx=0x7f6764005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.685+0000 7f67717fa640 1 -- 172.21.15.60:0/1322299861 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760000ee0 con 0x7f67740ae000 2024-04-25T10:17:03.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.685+0000 7f67717fa640 1 -- 172.21.15.60:0/1322299861 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67600bb070 con 0x7f67740ae000 2024-04-25T10:17:03.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.685+0000 7f677acd7640 1 -- 172.21.15.60:0/1322299861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67741f7890 con 0x7f67740ae000 2024-04-25T10:17:03.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.685+0000 7f67717fa640 1 -- 172.21.15.60:0/1322299861 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67600054a0 con 0x7f67740ae000 2024-04-25T10:17:03.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.685+0000 7f677acd7640 1 -- 172.21.15.60:0/1322299861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6774153ab0 con 0x7f67740ae000 2024-04-25T10:17:03.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.685+0000 7f677acd7640 1 -- 172.21.15.60:0/1322299861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6774074770 con 0x7f67740ae000 2024-04-25T10:17:03.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.685+0000 7f67717fa640 1 -- 172.21.15.60:0/1322299861 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67600b4040 con 0x7f67740ae000 2024-04-25T10:17:03.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.685+0000 7f67717fa640 1 --2- 172.21.15.60:0/1322299861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6758067100 0x7f67580695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.685+0000 7f67717fa640 1 -- 172.21.15.60:0/1322299861 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f676011f4d0 con 0x7f67740ae000 2024-04-25T10:17:03.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.685+0000 7f6778a73640 1 --2- 172.21.15.60:0/1322299861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6758067100 0x7f67580695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.685+0000 7f6778a73640 1 --2- 172.21.15.60:0/1322299861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6758067100 0x7f67580695c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6768001060 tx=0x7f6768009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.689+0000 7f67717fa640 1 -- 172.21.15.60:0/1322299861 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6760016130 con 0x7f67740ae000 2024-04-25T10:17:03.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.857+0000 7f677acd7640 1 -- 172.21.15.60:0/1322299861 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67740b3980 con 0x7f67740ae000 2024-04-25T10:17:03.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.857+0000 7f67717fa640 1 -- 172.21.15.60:0/1322299861 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+17048 (secure 0 0 0) 0x7f67600f0510 con 0x7f67740ae000 2024-04-25T10:17:03.864 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:17:03.865 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":114,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:17:03.699971+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:06.033387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:06.753421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.25","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/178428701":"2024-04-26T10:16:48.303277+0000","172.21.15.60:6834/178428701":"2024-04-26T10:16:48.303277+0000","172.21.15.60:6835/722171246":"2024-04-26T10:16:28.355776+0000","172.21.15.60:6834/722171246":"2024-04-26T10:16:28.355776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:03.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.861+0000 7f677acd7640 1 -- 172.21.15.60:0/1322299861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6758067100 msgr2=0x7f67580695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.861+0000 7f677acd7640 1 --2- 172.21.15.60:0/1322299861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6758067100 0x7f67580695c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6768001060 tx=0x7f6768009040 comp rx=0 tx=0).stop 2024-04-25T10:17:03.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.861+0000 7f677acd7640 1 -- 172.21.15.60:0/1322299861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67740ae000 msgr2=0x7f677406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.861+0000 7f677acd7640 1 --2- 172.21.15.60:0/1322299861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67740ae000 0x7f677406e920 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f6760001780 tx=0x7f6764005840 comp rx=0 tx=0).stop 2024-04-25T10:17:03.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.861+0000 7f677acd7640 1 -- 172.21.15.60:0/1322299861 shutdown_connections 2024-04-25T10:17:03.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.861+0000 7f677acd7640 1 --2- 172.21.15.60:0/1322299861 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6758067100 0x7f67580695c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.861+0000 7f677acd7640 1 --2- 172.21.15.60:0/1322299861 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67741f37a0 0x7f67741f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.861+0000 7f677acd7640 1 --2- 172.21.15.60:0/1322299861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67740ae000 0x7f677406e920 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.861+0000 7f677acd7640 1 --2- 172.21.15.60:0/1322299861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67640043a0 0x7f67741f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.861+0000 7f677acd7640 1 -- 172.21.15.60:0/1322299861 >> 172.21.15.60:0/1322299861 conn(0x7f67740b12d0 msgr2=0x7f67740b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:03.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.861+0000 7f677acd7640 1 -- 172.21.15.60:0/1322299861 shutdown_connections 2024-04-25T10:17:03.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:03.861+0000 7f677acd7640 1 -- 172.21.15.60:0/1322299861 wait complete. 2024-04-25T10:17:03.883 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:17:04.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.001+0000 7f851cfa4640 1 Processor -- start 2024-04-25T10:17:04.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.001+0000 7f851cfa4640 1 -- start start 2024-04-25T10:17:04.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.001+0000 7f851cfa4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851806ac10 0x7f851806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.001+0000 7f851cfa4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85180b5310 con 0x7f851815dcc0 2024-04-25T10:17:04.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.001+0000 7f851cfa4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851806b530 con 0x7f85180b1cc0 2024-04-25T10:17:04.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.001+0000 7f851cfa4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85180b5b70 con 0x7f851806ac10 2024-04-25T10:17:04.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f8515d9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f851815dcc0 0x7f85180adf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57572/0 (socket says 172.21.15.60:57572) 2024-04-25T10:17:04.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f8515d9b640 1 -- 172.21.15.60:0/2116010297 learned_addr learned my addr 172.21.15.60:0/2116010297 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:04.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851659c640 1 --2- 172.21.15.60:0/2116010297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851806ac10 0x7f851806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 -- 172.21.15.60:0/2116010297 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43874315 0 0) 0x7f85180b5310 con 0x7f851815dcc0 2024-04-25T10:17:04.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 -- 172.21.15.60:0/2116010297 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8504003530 con 0x7f851815dcc0 2024-04-25T10:17:04.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 -- 172.21.15.60:0/2116010297 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 502281981 0 0) 0x7f8504003530 con 0x7f851815dcc0 2024-04-25T10:17:04.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 -- 172.21.15.60:0/2116010297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851806ac10 msgr2=0x7f851806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:04.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 --2- 172.21.15.60:0/2116010297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851806ac10 0x7f851806aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 -- 172.21.15.60:0/2116010297 >> v1:172.21.15.100:6789/0 conn(0x7f85180b1cc0 legacy=0x7f85180ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 -- 172.21.15.60:0/2116010297 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85180b55b0 con 0x7f851815dcc0 2024-04-25T10:17:04.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 -- 172.21.15.60:0/2116010297 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f85000016a0 con 0x7f851815dcc0 2024-04-25T10:17:04.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 -- 172.21.15.60:0/2116010297 >> v1:172.21.15.60:6789/0 conn(0x7f851815dcc0 legacy=0x7f85180adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 --2- 172.21.15.60:0/2116010297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851806ac10 0x7f85040039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 --2- 172.21.15.60:0/2116010297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8504003f70 0x7f8504004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 --2- 172.21.15.60:0/2116010297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8504004d50 0x7f85040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 -- 172.21.15.60:0/2116010297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85180b5b70 con 0x7f8504004d50 2024-04-25T10:17:04.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 -- 172.21.15.60:0/2116010297 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850400bab0 con 0x7f8504003f70 2024-04-25T10:17:04.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 -- 172.21.15.60:0/2116010297 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850400bcb0 con 0x7f851806ac10 2024-04-25T10:17:04.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f8516d9d640 1 --2- 172.21.15.60:0/2116010297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8504004d50 0x7f85040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851659c640 1 --2- 172.21.15.60:0/2116010297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851806ac10 0x7f85040039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f8515d9b640 1 --2- 172.21.15.60:0/2116010297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8504003f70 0x7f8504004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f8516d9d640 1 -- 172.21.15.60:0/2116010297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851806ac10 msgr2=0x7f85040039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:04.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f8516d9d640 1 --2- 172.21.15.60:0/2116010297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851806ac10 0x7f85040039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f8516d9d640 1 -- 172.21.15.60:0/2116010297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8504003f70 msgr2=0x7f8504004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:04.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f8516d9d640 1 --2- 172.21.15.60:0/2116010297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8504003f70 0x7f8504004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f8516d9d640 1 -- 172.21.15.60:0/2116010297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8504003530 con 0x7f8504004d50 2024-04-25T10:17:04.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f8516d9d640 1 --2- 172.21.15.60:0/2116010297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8504004d50 0x7f85040051d0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f85180b20d0 tx=0x7f851001c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00ada40f0d51b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:04.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 -- 172.21.15.60:0/2116010297 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510001a90 con 0x7f8504004d50 2024-04-25T10:17:04.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 -- 172.21.15.60:0/2116010297 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8510003b80 con 0x7f8504004d50 2024-04-25T10:17:04.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851559a640 1 -- 172.21.15.60:0/2116010297 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85100050d0 con 0x7f8504004d50 2024-04-25T10:17:04.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851cfa4640 1 -- 172.21.15.60:0/2116010297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8504004d50 msgr2=0x7f85040051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.005+0000 7f851cfa4640 1 --2- 172.21.15.60:0/2116010297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8504004d50 0x7f85040051d0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f85180b20d0 tx=0x7f851001c700 comp rx=0 tx=0).stop 2024-04-25T10:17:04.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f8516d9d640 1 -- 172.21.15.60:0/2116010297 reap_dead start 2024-04-25T10:17:04.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f851cfa4640 1 -- 172.21.15.60:0/2116010297 shutdown_connections 2024-04-25T10:17:04.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f851cfa4640 1 -- 172.21.15.60:0/2116010297 >> 172.21.15.60:0/2116010297 conn(0x7f8518072920 msgr2=0x7f8518099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:04.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f851cfa4640 1 -- 172.21.15.60:0/2116010297 shutdown_connections 2024-04-25T10:17:04.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f851cfa4640 1 -- 172.21.15.60:0/2116010297 wait complete. 2024-04-25T10:17:04.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f851cfa4640 1 Processor -- start 2024-04-25T10:17:04.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f851cfa4640 1 -- start start 2024-04-25T10:17:04.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f851cfa4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851813fd00 0x7f8518158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f851cfa4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8518159360 0x7f8518155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f851cfa4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8518156110 0x7f8518154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f851cfa4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851806b840 con 0x7f8518159360 2024-04-25T10:17:04.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f851cfa4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85180b37b0 con 0x7f8518156110 2024-04-25T10:17:04.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f851cfa4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851806b9c0 con 0x7f851813fd00 2024-04-25T10:17:04.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f8515d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8518159360 0x7f8518155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f8515d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8518159360 0x7f8518155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53142/0 (socket says 172.21.15.60:53142) 2024-04-25T10:17:04.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f8515d9b640 1 -- 172.21.15.60:0/1501222106 learned_addr learned my addr 172.21.15.60:0/1501222106 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:04.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f851659c640 1 --2- 172.21.15.60:0/1501222106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851813fd00 0x7f8518158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f8516d9d640 1 --2- 172.21.15.60:0/1501222106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8518156110 0x7f8518154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f8515d9b640 1 -- 172.21.15.60:0/1501222106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851813fd00 msgr2=0x7f8518158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f8515d9b640 1 --2- 172.21.15.60:0/1501222106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851813fd00 0x7f8518158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f8515d9b640 1 -- 172.21.15.60:0/1501222106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8518156110 msgr2=0x7f8518154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f8515d9b640 1 --2- 172.21.15.60:0/1501222106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8518156110 0x7f8518154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f8515d9b640 1 -- 172.21.15.60:0/1501222106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85181565c0 con 0x7f8518159360 2024-04-25T10:17:04.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f851659c640 1 --2- 172.21.15.60:0/1501222106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851813fd00 0x7f8518158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:04.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f8515d9b640 1 --2- 172.21.15.60:0/1501222106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8518159360 0x7f8518155bd0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f850000dae0 tx=0x7f8500001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:04.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f850effd640 1 -- 172.21.15.60:0/1501222106 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8500019070 con 0x7f8518159360 2024-04-25T10:17:04.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f850effd640 1 -- 172.21.15.60:0/1501222106 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8500004cc0 con 0x7f8518159360 2024-04-25T10:17:04.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f851cfa4640 1 -- 172.21.15.60:0/1501222106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8518142730 con 0x7f8518159360 2024-04-25T10:17:04.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f850effd640 1 -- 172.21.15.60:0/1501222106 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85000072f0 con 0x7f8518159360 2024-04-25T10:17:04.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.009+0000 7f851cfa4640 1 -- 172.21.15.60:0/1501222106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8518142cf0 con 0x7f8518159360 2024-04-25T10:17:04.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.013+0000 7f851cfa4640 1 -- 172.21.15.60:0/1501222106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8518074770 con 0x7f8518159360 2024-04-25T10:17:04.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.013+0000 7f850effd640 1 -- 172.21.15.60:0/1501222106 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f850001c070 con 0x7f8518159360 2024-04-25T10:17:04.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.013+0000 7f850effd640 1 --2- 172.21.15.60:0/1501222106 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84f8066f90 0x7f84f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.013+0000 7f850effd640 1 -- 172.21.15.60:0/1501222106 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f8500084ac0 con 0x7f8518159360 2024-04-25T10:17:04.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.013+0000 7f851659c640 1 --2- 172.21.15.60:0/1501222106 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84f8066f90 0x7f84f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.013+0000 7f851659c640 1 --2- 172.21.15.60:0/1501222106 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84f8066f90 0x7f84f8069450 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8518154570 tx=0x7f8508008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:04.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.017+0000 7f850effd640 1 -- 172.21.15.60:0/1501222106 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8500055a70 con 0x7f8518159360 2024-04-25T10:17:04.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.209+0000 7f851cfa4640 1 -- 172.21.15.60:0/1501222106 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f85180b2e80 con 0x7f8518159360 2024-04-25T10:17:04.218 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T10:17:04.213+0000 7f3a792d6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:17:04.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.213+0000 7f850effd640 1 -- 172.21.15.60:0/1501222106 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v107) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8500055370 con 0x7f8518159360 2024-04-25T10:17:04.220 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:17:04.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.217+0000 7f851cfa4640 1 -- 172.21.15.60:0/1501222106 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84f8066f90 msgr2=0x7f84f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.217+0000 7f851cfa4640 1 --2- 172.21.15.60:0/1501222106 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84f8066f90 0x7f84f8069450 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8518154570 tx=0x7f8508008040 comp rx=0 tx=0).stop 2024-04-25T10:17:04.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.217+0000 7f851cfa4640 1 -- 172.21.15.60:0/1501222106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8518159360 msgr2=0x7f8518155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.217+0000 7f851cfa4640 1 --2- 172.21.15.60:0/1501222106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8518159360 0x7f8518155bd0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f850000dae0 tx=0x7f8500001d90 comp rx=0 tx=0).stop 2024-04-25T10:17:04.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.217+0000 7f851cfa4640 1 -- 172.21.15.60:0/1501222106 shutdown_connections 2024-04-25T10:17:04.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.217+0000 7f851cfa4640 1 --2- 172.21.15.60:0/1501222106 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f84f8066f90 0x7f84f8069450 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.217+0000 7f851cfa4640 1 --2- 172.21.15.60:0/1501222106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8518156110 0x7f8518154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.217+0000 7f851cfa4640 1 --2- 172.21.15.60:0/1501222106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8518159360 0x7f8518155bd0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.217+0000 7f851cfa4640 1 --2- 172.21.15.60:0/1501222106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851813fd00 0x7f8518158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.217+0000 7f851cfa4640 1 -- 172.21.15.60:0/1501222106 >> 172.21.15.60:0/1501222106 conn(0x7f8518072920 msgr2=0x7f851806e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:04.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.217+0000 7f851cfa4640 1 -- 172.21.15.60:0/1501222106 shutdown_connections 2024-04-25T10:17:04.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.217+0000 7f851cfa4640 1 -- 172.21.15.60:0/1501222106 wait complete. 2024-04-25T10:17:04.228 INFO:tasks.ceph.mds.c.smithi060.stderr: -1477> 2024-04-25T10:16:54.321+0000 7f51866cb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:17:04.241 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:17:04.262 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T10:17:04.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f22895e7640 1 Processor -- start 2024-04-25T10:17:04.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f22895e7640 1 -- start start 2024-04-25T10:17:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f22895e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22840b3f60 0x7f228415a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f22895e7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22840b3510 con 0x7f2284071250 2024-04-25T10:17:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f22895e7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228415ad30 con 0x7f228406f030 2024-04-25T10:17:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f22895e7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228415b010 con 0x7f22840b3f60 2024-04-25T10:17:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f22837fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2284071250 0x7f228415b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57580/0 (socket says 172.21.15.60:57580) 2024-04-25T10:17:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f22837fe640 1 -- 172.21.15.60:0/605810788 learned_addr learned my addr 172.21.15.60:0/605810788 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2281ffb640 1 -- 172.21.15.60:0/605810788 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401730709 0 0) 0x7f22840b3510 con 0x7f2284071250 2024-04-25T10:17:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2281ffb640 1 -- 172.21.15.60:0/605810788 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f226c003530 con 0x7f2284071250 2024-04-25T10:17:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2282ffd640 1 --2- 172.21.15.60:0/605810788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22840b3f60 0x7f228415a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2281ffb640 1 -- 172.21.15.60:0/605810788 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1372134353 0 0) 0x7f226c003530 con 0x7f2284071250 2024-04-25T10:17:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2281ffb640 1 -- 172.21.15.60:0/605810788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22840b3f60 msgr2=0x7f228415a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2281ffb640 1 --2- 172.21.15.60:0/605810788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22840b3f60 0x7f228415a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2281ffb640 1 -- 172.21.15.60:0/605810788 >> v1:172.21.15.100:6789/0 conn(0x7f228406f030 legacy=0x7f22840b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2281ffb640 1 -- 172.21.15.60:0/605810788 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228415b2f0 con 0x7f2284071250 2024-04-25T10:17:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2281ffb640 1 -- 172.21.15.60:0/605810788 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f227c001a00 con 0x7f2284071250 2024-04-25T10:17:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2281ffb640 1 -- 172.21.15.60:0/605810788 >> v1:172.21.15.60:6789/0 conn(0x7f2284071250 legacy=0x7f228415b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2281ffb640 1 --2- 172.21.15.60:0/605810788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c0039f0 0x7f226c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2281ffb640 1 --2- 172.21.15.60:0/605810788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f226c004810 0x7f226c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2281ffb640 1 --2- 172.21.15.60:0/605810788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22840b3f60 0x7f226c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2281ffb640 1 -- 172.21.15.60:0/605810788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228415b010 con 0x7f226c004810 2024-04-25T10:17:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2281ffb640 1 -- 172.21.15.60:0/605810788 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c005530 con 0x7f226c0039f0 2024-04-25T10:17:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2281ffb640 1 -- 172.21.15.60:0/605810788 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c00e120 con 0x7f22840b3f60 2024-04-25T10:17:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f22827fc640 1 --2- 172.21.15.60:0/605810788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f226c004810 0x7f226c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2282ffd640 1 --2- 172.21.15.60:0/605810788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c0039f0 0x7f226c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f22837fe640 1 --2- 172.21.15.60:0/605810788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22840b3f60 0x7f226c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f22827fc640 1 -- 172.21.15.60:0/605810788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22840b3f60 msgr2=0x7f226c00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f22827fc640 1 --2- 172.21.15.60:0/605810788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22840b3f60 0x7f226c00dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f22827fc640 1 -- 172.21.15.60:0/605810788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c0039f0 msgr2=0x7f226c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f22827fc640 1 --2- 172.21.15.60:0/605810788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c0039f0 0x7f226c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f22827fc640 1 -- 172.21.15.60:0/605810788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c003530 con 0x7f226c004810 2024-04-25T10:17:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.353+0000 7f2282ffd640 1 --2- 172.21.15.60:0/605810788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f226c0039f0 0x7f226c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.357+0000 7f22827fc640 1 --2- 172.21.15.60:0/605810788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f226c004810 0x7f226c004c90 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f2274001750 tx=0x7f227401ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4b993373430468 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.357+0000 7f2281ffb640 1 -- 172.21.15.60:0/605810788 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22740013a0 con 0x7f226c004810 2024-04-25T10:17:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.357+0000 7f2281ffb640 1 -- 172.21.15.60:0/605810788 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22740052f0 con 0x7f226c004810 2024-04-25T10:17:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.357+0000 7f2281ffb640 1 -- 172.21.15.60:0/605810788 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274003840 con 0x7f226c004810 2024-04-25T10:17:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.357+0000 7f22895e7640 1 -- 172.21.15.60:0/605810788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f226c004810 msgr2=0x7f226c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.357+0000 7f22895e7640 1 --2- 172.21.15.60:0/605810788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f226c004810 0x7f226c004c90 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f2274001750 tx=0x7f227401ce40 comp rx=0 tx=0).stop 2024-04-25T10:17:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.357+0000 7f22837fe640 1 -- 172.21.15.60:0/605810788 reap_dead start 2024-04-25T10:17:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.357+0000 7f22895e7640 1 -- 172.21.15.60:0/605810788 shutdown_connections 2024-04-25T10:17:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.357+0000 7f22895e7640 1 -- 172.21.15.60:0/605810788 >> 172.21.15.60:0/605810788 conn(0x7f22840670b0 msgr2=0x7f228406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.357+0000 7f22895e7640 1 -- 172.21.15.60:0/605810788 shutdown_connections 2024-04-25T10:17:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.357+0000 7f22895e7640 1 -- 172.21.15.60:0/605810788 wait complete. 2024-04-25T10:17:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.357+0000 7f22895e7640 1 Processor -- start 2024-04-25T10:17:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.357+0000 7f22895e7640 1 -- start start 2024-04-25T10:17:04.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22895e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f226c004810 0x7f228414f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22895e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284156760 0x7f2284144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22895e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284156ba0 0x7f22841451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22895e7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228415a560 con 0x7f2284156760 2024-04-25T10:17:04.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22895e7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228415a3e0 con 0x7f2284156ba0 2024-04-25T10:17:04.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22895e7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228415a6e0 con 0x7f226c004810 2024-04-25T10:17:04.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22827fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284156760 0x7f2284144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22827fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284156760 0x7f2284144c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53178/0 (socket says 172.21.15.60:53178) 2024-04-25T10:17:04.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22827fc640 1 -- 172.21.15.60:0/2362046788 learned_addr learned my addr 172.21.15.60:0/2362046788 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:04.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f2282ffd640 1 --2- 172.21.15.60:0/2362046788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f226c004810 0x7f228414f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22837fe640 1 --2- 172.21.15.60:0/2362046788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284156ba0 0x7f22841451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22827fc640 1 -- 172.21.15.60:0/2362046788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f226c004810 msgr2=0x7f228414f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22827fc640 1 --2- 172.21.15.60:0/2362046788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f226c004810 0x7f228414f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22827fc640 1 -- 172.21.15.60:0/2362046788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284156ba0 msgr2=0x7f22841451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22827fc640 1 --2- 172.21.15.60:0/2362046788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284156ba0 0x7f22841451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22827fc640 1 -- 172.21.15.60:0/2362046788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228414f900 con 0x7f2284156760 2024-04-25T10:17:04.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22827fc640 1 --2- 172.21.15.60:0/2362046788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284156760 0x7f2284144c90 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f2274005040 tx=0x7f226c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:04.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f226b7fe640 1 -- 172.21.15.60:0/2362046788 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274007a40 con 0x7f2284156760 2024-04-25T10:17:04.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f226b7fe640 1 -- 172.21.15.60:0/2362046788 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22740bb070 con 0x7f2284156760 2024-04-25T10:17:04.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22895e7640 1 -- 172.21.15.60:0/2362046788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228414fbb0 con 0x7f2284156760 2024-04-25T10:17:04.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f226b7fe640 1 -- 172.21.15.60:0/2362046788 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22740013a0 con 0x7f2284156760 2024-04-25T10:17:04.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22895e7640 1 -- 172.21.15.60:0/2362046788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2284150150 con 0x7f2284156760 2024-04-25T10:17:04.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f22895e7640 1 -- 172.21.15.60:0/2362046788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2284074770 con 0x7f2284156760 2024-04-25T10:17:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f226b7fe640 1 -- 172.21.15.60:0/2362046788 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22740b4040 con 0x7f2284156760 2024-04-25T10:17:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f226b7fe640 1 --2- 172.21.15.60:0/2362046788 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2270066f60 0x7f2270069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f226b7fe640 1 -- 172.21.15.60:0/2362046788 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6025+0+0 (secure 0 0 0) 0x7f227411ef80 con 0x7f2284156760 2024-04-25T10:17:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.361+0000 7f2282ffd640 1 --2- 172.21.15.60:0/2362046788 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2270066f60 0x7f2270069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.365+0000 7f2282ffd640 1 --2- 172.21.15.60:0/2362046788 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2270066f60 0x7f2270069420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f227800c890 tx=0x7f2278008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:04.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.365+0000 7f226b7fe640 1 -- 172.21.15.60:0/2362046788 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22740160f0 con 0x7f2284156760 2024-04-25T10:17:04.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:04.553+0000 7f22895e7640 1 -- 172.21.15.60:0/2362046788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2284067c90 con 0x7f2284156760 2024-04-25T10:17:05.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.225+0000 7f226b7fe640 1 -- 172.21.15.60:0/2362046788 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v108) v1 ==== 104+0+0 (secure 0 0 0) 0x7f22740f0000 con 0x7f2284156760 2024-04-25T10:17:05.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.225+0000 7f22895e7640 1 -- 172.21.15.60:0/2362046788 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2270066f60 msgr2=0x7f2270069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:05.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.225+0000 7f22895e7640 1 --2- 172.21.15.60:0/2362046788 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2270066f60 0x7f2270069420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f227800c890 tx=0x7f2278008040 comp rx=0 tx=0).stop 2024-04-25T10:17:05.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.225+0000 7f22895e7640 1 -- 172.21.15.60:0/2362046788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284156760 msgr2=0x7f2284144c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:05.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.225+0000 7f22895e7640 1 --2- 172.21.15.60:0/2362046788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284156760 0x7f2284144c90 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f2274005040 tx=0x7f226c005840 comp rx=0 tx=0).stop 2024-04-25T10:17:05.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.225+0000 7f22895e7640 1 -- 172.21.15.60:0/2362046788 shutdown_connections 2024-04-25T10:17:05.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.225+0000 7f22895e7640 1 --2- 172.21.15.60:0/2362046788 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2270066f60 0x7f2270069420 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.225+0000 7f22895e7640 1 --2- 172.21.15.60:0/2362046788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2284156ba0 0x7f22841451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.225+0000 7f22895e7640 1 --2- 172.21.15.60:0/2362046788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284156760 0x7f2284144c90 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.225+0000 7f22895e7640 1 --2- 172.21.15.60:0/2362046788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f226c004810 0x7f228414f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.225+0000 7f22895e7640 1 -- 172.21.15.60:0/2362046788 >> 172.21.15.60:0/2362046788 conn(0x7f22840670b0 msgr2=0x7f22840ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:05.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.225+0000 7f22895e7640 1 -- 172.21.15.60:0/2362046788 shutdown_connections 2024-04-25T10:17:05.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.225+0000 7f22895e7640 1 -- 172.21.15.60:0/2362046788 wait complete. 2024-04-25T10:17:05.256 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:17:05.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f298b4b6640 1 Processor -- start 2024-04-25T10:17:05.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f298b4b6640 1 -- start start 2024-04-25T10:17:05.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f298b4b6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f298415db10 0x7f298406c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:05.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f298b4b6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29840ced80 0x7f29840d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:05.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f298b4b6640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298406c600 con 0x7f29840b93a0 2024-04-25T10:17:05.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f298b4b6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29840d1b10 con 0x7f298415db10 2024-04-25T10:17:05.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f298b4b6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298406c8d0 con 0x7f29840ced80 2024-04-25T10:17:05.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f2989252640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f298415db10 0x7f298406c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:05.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f2988a51640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f29840b93a0 0x7f29840ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57588/0 (socket says 172.21.15.60:57588) 2024-04-25T10:17:05.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f2989252640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f298415db10 0x7f298406c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53188/0 (socket says 172.21.15.60:53188) 2024-04-25T10:17:05.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f2988a51640 1 -- 172.21.15.60:0/2333130541 learned_addr learned my addr 172.21.15.60:0/2333130541 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:05.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f2989a53640 1 --2- 172.21.15.60:0/2333130541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29840ced80 0x7f29840d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:05.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f2989252640 1 -- 172.21.15.60:0/2333130541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29840ced80 msgr2=0x7f29840d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:05.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f2989252640 1 --2- 172.21.15.60:0/2333130541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29840ced80 0x7f29840d2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f2989252640 1 -- 172.21.15.60:0/2333130541 >> v1:172.21.15.60:6789/0 conn(0x7f29840b93a0 legacy=0x7f29840ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:05.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f297bfff640 1 -- 172.21.15.60:0/2333130541 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058639826 0 0) 0x7f298406c600 con 0x7f29840b93a0 2024-04-25T10:17:05.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.429+0000 7f2989252640 1 -- 172.21.15.60:0/2333130541 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298406cbb0 con 0x7f298415db10 2024-04-25T10:17:05.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f2989252640 1 --2- 172.21.15.60:0/2333130541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f298415db10 0x7f298406c0c0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f29800097c0 tx=0x7f298001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd75bddc84480ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:05.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f297bfff640 1 -- 172.21.15.60:0/2333130541 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29800b1070 con 0x7f298415db10 2024-04-25T10:17:05.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f297bfff640 1 -- 172.21.15.60:0/2333130541 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29800a8050 con 0x7f298415db10 2024-04-25T10:17:05.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f297bfff640 1 -- 172.21.15.60:0/2333130541 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29800ac860 con 0x7f298415db10 2024-04-25T10:17:05.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f298b4b6640 1 -- 172.21.15.60:0/2333130541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f298415db10 msgr2=0x7f298406c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:05.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f298b4b6640 1 --2- 172.21.15.60:0/2333130541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f298415db10 0x7f298406c0c0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f29800097c0 tx=0x7f298001c3e0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f298b4b6640 1 -- 172.21.15.60:0/2333130541 shutdown_connections 2024-04-25T10:17:05.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f298b4b6640 1 --2- 172.21.15.60:0/2333130541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29840ced80 0x7f29840d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f298b4b6640 1 --2- 172.21.15.60:0/2333130541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f298415db10 0x7f298406c0c0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f298b4b6640 1 -- 172.21.15.60:0/2333130541 >> 172.21.15.60:0/2333130541 conn(0x7f29840d0420 msgr2=0x7f29840d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:05.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f298b4b6640 1 -- 172.21.15.60:0/2333130541 shutdown_connections 2024-04-25T10:17:05.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f298b4b6640 1 -- 172.21.15.60:0/2333130541 wait complete. 2024-04-25T10:17:05.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f298b4b6640 1 Processor -- start 2024-04-25T10:17:05.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f298b4b6640 1 -- start start 2024-04-25T10:17:05.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f298b4b6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29840b93a0 0x7f29840cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:05.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f298b4b6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29840ced80 0x7f2984159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:05.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f298b4b6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f298415db10 0x7f29840cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:05.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f298b4b6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29840d3130 con 0x7f298415db10 2024-04-25T10:17:05.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f298b4b6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29840d2fb0 con 0x7f29840b93a0 2024-04-25T10:17:05.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f298b4b6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29840d32b0 con 0x7f29840ced80 2024-04-25T10:17:05.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f2989a53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f298415db10 0x7f29840cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:05.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f2989a53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f298415db10 0x7f29840cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53194/0 (socket says 172.21.15.60:53194) 2024-04-25T10:17:05.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.433+0000 7f2989a53640 1 -- 172.21.15.60:0/1763722635 learned_addr learned my addr 172.21.15.60:0/1763722635 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:05.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f2989252640 1 --2- 172.21.15.60:0/1763722635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29840b93a0 0x7f29840cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:05.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f2988a51640 1 --2- 172.21.15.60:0/1763722635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29840ced80 0x7f2984159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:05.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f2989a53640 1 -- 172.21.15.60:0/1763722635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29840ced80 msgr2=0x7f2984159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:05.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f2989a53640 1 --2- 172.21.15.60:0/1763722635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29840ced80 0x7f2984159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f2989a53640 1 -- 172.21.15.60:0/1763722635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29840b93a0 msgr2=0x7f29840cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:05.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f2989a53640 1 --2- 172.21.15.60:0/1763722635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29840b93a0 0x7f29840cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f2989a53640 1 -- 172.21.15.60:0/1763722635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298415a340 con 0x7f298415db10 2024-04-25T10:17:05.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f2988a51640 1 --2- 172.21.15.60:0/1763722635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29840ced80 0x7f2984159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:05.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f2989a53640 1 --2- 172.21.15.60:0/1763722635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f298415db10 0x7f29840cb450 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f297c007e60 tx=0x7f297c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:05.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f2979ffb640 1 -- 172.21.15.60:0/1763722635 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c014420 con 0x7f298415db10 2024-04-25T10:17:05.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f2979ffb640 1 -- 172.21.15.60:0/1763722635 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c019070 con 0x7f298415db10 2024-04-25T10:17:05.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f2979ffb640 1 -- 172.21.15.60:0/1763722635 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c014800 con 0x7f298415db10 2024-04-25T10:17:05.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f298b4b6640 1 -- 172.21.15.60:0/1763722635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29840cbcd0 con 0x7f298415db10 2024-04-25T10:17:05.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f298b4b6640 1 -- 172.21.15.60:0/1763722635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29840cc290 con 0x7f298415db10 2024-04-25T10:17:05.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f298b4b6640 1 -- 172.21.15.60:0/1763722635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2984074770 con 0x7f298415db10 2024-04-25T10:17:05.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f2979ffb640 1 -- 172.21.15.60:0/1763722635 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f297c0149a0 con 0x7f298415db10 2024-04-25T10:17:05.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f2979ffb640 1 --2- 172.21.15.60:0/1763722635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2968066f60 0x7f2968069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:05.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f2979ffb640 1 -- 172.21.15.60:0/1763722635 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6025+0+0 (secure 0 0 0) 0x7f297c085a50 con 0x7f298415db10 2024-04-25T10:17:05.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.437+0000 7f2989252640 1 --2- 172.21.15.60:0/1763722635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2968066f60 0x7f2968069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:05.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.441+0000 7f2989252640 1 --2- 172.21.15.60:0/1763722635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2968066f60 0x7f2968069420 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f29800ae000 tx=0x7f2980008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:05.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.441+0000 7f2979ffb640 1 -- 172.21.15.60:0/1763722635 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f297c056920 con 0x7f298415db10 2024-04-25T10:17:05.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:05.609+0000 7f298b4b6640 1 -- 172.21.15.60:0/1763722635 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f298406b040 con 0x7f298415db10 2024-04-25T10:17:06.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.225+0000 7f2979ffb640 1 -- 172.21.15.60:0/1763722635 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v116) v1 ==== 181+0+0 (secure 0 0 0) 0x7f297c056220 con 0x7f298415db10 2024-04-25T10:17:06.232 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:17:06.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.229+0000 7f298b4b6640 1 -- 172.21.15.60:0/1763722635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2968066f60 msgr2=0x7f2968069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.229+0000 7f298b4b6640 1 --2- 172.21.15.60:0/1763722635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2968066f60 0x7f2968069420 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f29800ae000 tx=0x7f2980008040 comp rx=0 tx=0).stop 2024-04-25T10:17:06.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.229+0000 7f298b4b6640 1 -- 172.21.15.60:0/1763722635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f298415db10 msgr2=0x7f29840cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.229+0000 7f298b4b6640 1 --2- 172.21.15.60:0/1763722635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f298415db10 0x7f29840cb450 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f297c007e60 tx=0x7f297c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.229+0000 7f298b4b6640 1 -- 172.21.15.60:0/1763722635 shutdown_connections 2024-04-25T10:17:06.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.229+0000 7f298b4b6640 1 --2- 172.21.15.60:0/1763722635 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2968066f60 0x7f2968069420 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.229+0000 7f298b4b6640 1 --2- 172.21.15.60:0/1763722635 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f298415db10 0x7f29840cb450 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.229+0000 7f298b4b6640 1 --2- 172.21.15.60:0/1763722635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29840ced80 0x7f2984159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.229+0000 7f298b4b6640 1 --2- 172.21.15.60:0/1763722635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29840b93a0 0x7f29840cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.229+0000 7f298b4b6640 1 -- 172.21.15.60:0/1763722635 >> 172.21.15.60:0/1763722635 conn(0x7f29840d0420 msgr2=0x7f29840b8520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:06.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.233+0000 7f298b4b6640 1 -- 172.21.15.60:0/1763722635 shutdown_connections 2024-04-25T10:17:06.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.233+0000 7f298b4b6640 1 -- 172.21.15.60:0/1763722635 wait complete. 2024-04-25T10:17:06.258 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:17:06.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.381+0000 7f3466723640 1 Processor -- start 2024-04-25T10:17:06.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.381+0000 7f3466723640 1 -- start start 2024-04-25T10:17:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.381+0000 7f3466723640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f346015dc80 0x7f34600ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.381+0000 7f3466723640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3460070b30 con 0x7f34600b5870 2024-04-25T10:17:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.381+0000 7f3466723640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3460161790 con 0x7f346015d8a0 2024-04-25T10:17:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.381+0000 7f3466723640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3460161a70 con 0x7f346015dc80 2024-04-25T10:17:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f3464cc0640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f34600b5870 0x7f34600ce5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57600/0 (socket says 172.21.15.60:57600) 2024-04-25T10:17:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f3464cc0640 1 -- 172.21.15.60:0/3416502206 learned_addr learned my addr 172.21.15.60:0/3416502206 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345f7fe640 1 --2- 172.21.15.60:0/3416502206 >> v2:172.21.15.100:3300/0 conn(0x7f346015dc80 0x7f34600ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 -- 172.21.15.60:0/3416502206 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 475766943 0 0) 0x7f3460070b30 con 0x7f34600b5870 2024-04-25T10:17:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 -- 172.21.15.60:0/3416502206 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f344c003530 con 0x7f34600b5870 2024-04-25T10:17:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 -- 172.21.15.60:0/3416502206 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2356568049 0 0) 0x7f344c003530 con 0x7f34600b5870 2024-04-25T10:17:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 -- 172.21.15.60:0/3416502206 >> v2:172.21.15.100:3300/0 conn(0x7f346015dc80 msgr2=0x7f34600ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 --2- 172.21.15.60:0/3416502206 >> v2:172.21.15.100:3300/0 conn(0x7f346015dc80 0x7f34600ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 -- 172.21.15.60:0/3416502206 >> v1:172.21.15.100:6789/0 conn(0x7f346015d8a0 legacy=0x7f34600b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 -- 172.21.15.60:0/3416502206 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3460070800 con 0x7f34600b5870 2024-04-25T10:17:06.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 -- 172.21.15.60:0/3416502206 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f3458001a00 con 0x7f34600b5870 2024-04-25T10:17:06.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 -- 172.21.15.60:0/3416502206 >> v1:172.21.15.60:6789/0 conn(0x7f34600b5870 legacy=0x7f34600ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 --2- 172.21.15.60:0/3416502206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c0039c0 0x7f344c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 --2- 172.21.15.60:0/3416502206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f344c004810 0x7f344c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 --2- 172.21.15.60:0/3416502206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c0051d0 0x7f344c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 -- 172.21.15.60:0/3416502206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3460161a70 con 0x7f344c0039c0 2024-04-25T10:17:06.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 -- 172.21.15.60:0/3416502206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c00c170 con 0x7f344c0051d0 2024-04-25T10:17:06.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 -- 172.21.15.60:0/3416502206 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344c00c370 con 0x7f344c004810 2024-04-25T10:17:06.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345ffff640 1 --2- 172.21.15.60:0/3416502206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c0039c0 0x7f344c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345f7fe640 1 --2- 172.21.15.60:0/3416502206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f344c004810 0x7f344c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f3464cc0640 1 --2- 172.21.15.60:0/3416502206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c0051d0 0x7f344c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345ffff640 1 -- 172.21.15.60:0/3416502206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f344c004810 msgr2=0x7f344c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:06.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345ffff640 1 --2- 172.21.15.60:0/3416502206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f344c004810 0x7f344c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345ffff640 1 -- 172.21.15.60:0/3416502206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c0051d0 msgr2=0x7f344c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:06.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345ffff640 1 --2- 172.21.15.60:0/3416502206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c0051d0 0x7f344c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345ffff640 1 -- 172.21.15.60:0/3416502206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c003530 con 0x7f344c0039c0 2024-04-25T10:17:06.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f3464cc0640 1 -- 172.21.15.60:0/3416502206 reap_dead start 2024-04-25T10:17:06.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345ffff640 1 --2- 172.21.15.60:0/3416502206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c0039c0 0x7f344c003de0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f3448001780 tx=0x7f344801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d94909492f65d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:06.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 -- 172.21.15.60:0/3416502206 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34480013a0 con 0x7f344c0039c0 2024-04-25T10:17:06.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 -- 172.21.15.60:0/3416502206 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34480052f0 con 0x7f344c0039c0 2024-04-25T10:17:06.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.385+0000 7f345effd640 1 -- 172.21.15.60:0/3416502206 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448003840 con 0x7f344c0039c0 2024-04-25T10:17:06.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 -- 172.21.15.60:0/3416502206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c0039c0 msgr2=0x7f344c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 --2- 172.21.15.60:0/3416502206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c0039c0 0x7f344c003de0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f3448001780 tx=0x7f344801c3a0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 -- 172.21.15.60:0/3416502206 shutdown_connections 2024-04-25T10:17:06.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 --2- 172.21.15.60:0/3416502206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c0039c0 0x7f344c003de0 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 -- 172.21.15.60:0/3416502206 >> 172.21.15.60:0/3416502206 conn(0x7f346006e400 msgr2=0x7f346006e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:06.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 -- 172.21.15.60:0/3416502206 shutdown_connections 2024-04-25T10:17:06.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 -- 172.21.15.60:0/3416502206 wait complete. 2024-04-25T10:17:06.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 Processor -- start 2024-04-25T10:17:06.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 -- start start 2024-04-25T10:17:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c0039c0 0x7f346014c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34600cc290 0x7f34600cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34600cf7c0 0x7f34600c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34600cd090 con 0x7f34600cf7c0 2024-04-25T10:17:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34600ccf10 con 0x7f344c0039c0 2024-04-25T10:17:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34600b5160 con 0x7f34600cc290 2024-04-25T10:17:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3464cc0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34600cf7c0 0x7f34600c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3464cc0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34600cf7c0 0x7f34600c9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53218/0 (socket says 172.21.15.60:53218) 2024-04-25T10:17:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3464cc0640 1 -- 172.21.15.60:0/2361865248 learned_addr learned my addr 172.21.15.60:0/2361865248 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:06.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f345ffff640 1 --2- 172.21.15.60:0/2361865248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c0039c0 0x7f346014c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f345f7fe640 1 --2- 172.21.15.60:0/2361865248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34600cc290 0x7f34600cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3464cc0640 1 -- 172.21.15.60:0/2361865248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34600cc290 msgr2=0x7f34600cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3464cc0640 1 --2- 172.21.15.60:0/2361865248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34600cc290 0x7f34600cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3464cc0640 1 -- 172.21.15.60:0/2361865248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c0039c0 msgr2=0x7f346014c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3464cc0640 1 --2- 172.21.15.60:0/2361865248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c0039c0 0x7f346014c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3464cc0640 1 -- 172.21.15.60:0/2361865248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346015f2b0 con 0x7f34600cf7c0 2024-04-25T10:17:06.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f345f7fe640 1 --2- 172.21.15.60:0/2361865248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34600cc290 0x7f34600cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:06.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3464cc0640 1 --2- 172.21.15.60:0/2361865248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34600cf7c0 0x7f34600c9ac0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f34600afc50 tx=0x7f345800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:06.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f345cff9640 1 -- 172.21.15.60:0/2361865248 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458004d70 con 0x7f34600cf7c0 2024-04-25T10:17:06.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f345cff9640 1 -- 172.21.15.60:0/2361865248 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345800bb80 con 0x7f34600cf7c0 2024-04-25T10:17:06.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 -- 172.21.15.60:0/2361865248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346015e210 con 0x7f34600cf7c0 2024-04-25T10:17:06.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f345cff9640 1 -- 172.21.15.60:0/2361865248 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458007070 con 0x7f34600cf7c0 2024-04-25T10:17:06.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.389+0000 7f3466723640 1 -- 172.21.15.60:0/2361865248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346015e7d0 con 0x7f34600cf7c0 2024-04-25T10:17:06.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.393+0000 7f3466723640 1 -- 172.21.15.60:0/2361865248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3460074770 con 0x7f34600cf7c0 2024-04-25T10:17:06.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.393+0000 7f345cff9640 1 -- 172.21.15.60:0/2361865248 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f345801b020 con 0x7f34600cf7c0 2024-04-25T10:17:06.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.393+0000 7f345cff9640 1 --2- 172.21.15.60:0/2361865248 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3440066f90 0x7f3440069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.393+0000 7f345cff9640 1 -- 172.21.15.60:0/2361865248 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5587+0+0 (secure 0 0 0) 0x7f34580851d0 con 0x7f34600cf7c0 2024-04-25T10:17:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.393+0000 7f345ffff640 1 --2- 172.21.15.60:0/2361865248 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3440066f90 0x7f3440069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.393+0000 7f345ffff640 1 --2- 172.21.15.60:0/2361865248 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3440066f90 0x7f3440069450 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3448001750 tx=0x7f34480011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:06.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.397+0000 7f345cff9640 1 -- 172.21.15.60:0/2361865248 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3458056250 con 0x7f34600cf7c0 2024-04-25T10:17:06.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:06.557+0000 7f3466723640 1 -- 172.21.15.60:0/2361865248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3460154840 con 0x7f34600cf7c0 2024-04-25T10:17:07.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.229+0000 7f345cff9640 1 -- 172.21.15.60:0/2361865248 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v117) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3458055b50 con 0x7f34600cf7c0 2024-04-25T10:17:07.235 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T10:17:07.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.233+0000 7f3466723640 1 -- 172.21.15.60:0/2361865248 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3440066f90 msgr2=0x7f3440069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.233+0000 7f3466723640 1 --2- 172.21.15.60:0/2361865248 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3440066f90 0x7f3440069450 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3448001750 tx=0x7f34480011a0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.233+0000 7f3466723640 1 -- 172.21.15.60:0/2361865248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34600cf7c0 msgr2=0x7f34600c9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.233+0000 7f3466723640 1 --2- 172.21.15.60:0/2361865248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34600cf7c0 0x7f34600c9ac0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f34600afc50 tx=0x7f345800b4a0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.233+0000 7f3466723640 1 -- 172.21.15.60:0/2361865248 shutdown_connections 2024-04-25T10:17:07.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.233+0000 7f3466723640 1 --2- 172.21.15.60:0/2361865248 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3440066f90 0x7f3440069450 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.233+0000 7f3466723640 1 --2- 172.21.15.60:0/2361865248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34600cf7c0 0x7f34600c9ac0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.233+0000 7f3466723640 1 --2- 172.21.15.60:0/2361865248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34600cc290 0x7f34600cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.233+0000 7f3466723640 1 --2- 172.21.15.60:0/2361865248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c0039c0 0x7f346014c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.233+0000 7f3466723640 1 -- 172.21.15.60:0/2361865248 >> 172.21.15.60:0/2361865248 conn(0x7f346006e400 msgr2=0x7f34600b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:07.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.233+0000 7f3466723640 1 -- 172.21.15.60:0/2361865248 shutdown_connections 2024-04-25T10:17:07.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.233+0000 7f3466723640 1 -- 172.21.15.60:0/2361865248 wait complete. 2024-04-25T10:17:07.264 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:17:07.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.385+0000 7f6a96272640 1 Processor -- start 2024-04-25T10:17:07.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.385+0000 7f6a96272640 1 -- start start 2024-04-25T10:17:07.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a96272640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6a9015fbf0 0x7f6a90166870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a96272640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a90144a40 con 0x7f6a90160080 2024-04-25T10:17:07.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a96272640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a900cc9f0 con 0x7f6a900cbcf0 2024-04-25T10:17:07.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a96272640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a900b2dc0 con 0x7f6a9015fbf0 2024-04-25T10:17:07.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6a90160080 0x7f6a9015f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57606/0 (socket says 172.21.15.60:57606) 2024-04-25T10:17:07.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8f7fe640 1 -- 172.21.15.60:0/3538646765 learned_addr learned my addr 172.21.15.60:0/3538646765 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:07.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a9480f640 1 --2- 172.21.15.60:0/3538646765 >> v2:172.21.15.100:3300/0 conn(0x7f6a9015fbf0 0x7f6a90166870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8effd640 1 -- 172.21.15.60:0/3538646765 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393930393 0 0) 0x7f6a90144a40 con 0x7f6a90160080 2024-04-25T10:17:07.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8effd640 1 -- 172.21.15.60:0/3538646765 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a7c003530 con 0x7f6a90160080 2024-04-25T10:17:07.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8effd640 1 -- 172.21.15.60:0/3538646765 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2101474761 0 0) 0x7f6a7c003530 con 0x7f6a90160080 2024-04-25T10:17:07.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8effd640 1 -- 172.21.15.60:0/3538646765 >> v2:172.21.15.100:3300/0 conn(0x7f6a9015fbf0 msgr2=0x7f6a90166870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:07.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8effd640 1 --2- 172.21.15.60:0/3538646765 >> v2:172.21.15.100:3300/0 conn(0x7f6a9015fbf0 0x7f6a90166870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8effd640 1 -- 172.21.15.60:0/3538646765 >> v1:172.21.15.100:6789/0 conn(0x7f6a900cbcf0 legacy=0x7f6a90071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8effd640 1 -- 172.21.15.60:0/3538646765 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a900cccd0 con 0x7f6a90160080 2024-04-25T10:17:07.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8effd640 1 -- 172.21.15.60:0/3538646765 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f6a80001a00 con 0x7f6a90160080 2024-04-25T10:17:07.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8effd640 1 -- 172.21.15.60:0/3538646765 >> v1:172.21.15.60:6789/0 conn(0x7f6a90160080 legacy=0x7f6a9015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8effd640 1 --2- 172.21.15.60:0/3538646765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c0039c0 0x7f6a7c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8effd640 1 --2- 172.21.15.60:0/3538646765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a7c004810 0x7f6a7c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8effd640 1 --2- 172.21.15.60:0/3538646765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a7c0051d0 0x7f6a7c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8effd640 1 -- 172.21.15.60:0/3538646765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c00c040 con 0x7f6a7c0039c0 2024-04-25T10:17:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8effd640 1 -- 172.21.15.60:0/3538646765 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c00c1f0 con 0x7f6a7c004810 2024-04-25T10:17:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8effd640 1 -- 172.21.15.60:0/3538646765 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c00c3f0 con 0x7f6a7c0051d0 2024-04-25T10:17:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8ffff640 1 --2- 172.21.15.60:0/3538646765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c0039c0 0x7f6a7c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8f7fe640 1 --2- 172.21.15.60:0/3538646765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a7c004810 0x7f6a7c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a9480f640 1 --2- 172.21.15.60:0/3538646765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a7c0051d0 0x7f6a7c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8ffff640 1 -- 172.21.15.60:0/3538646765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a7c0051d0 msgr2=0x7f6a7c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8ffff640 1 --2- 172.21.15.60:0/3538646765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a7c0051d0 0x7f6a7c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8ffff640 1 -- 172.21.15.60:0/3538646765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a7c004810 msgr2=0x7f6a7c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8ffff640 1 --2- 172.21.15.60:0/3538646765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a7c004810 0x7f6a7c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a8ffff640 1 -- 172.21.15.60:0/3538646765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c003530 con 0x7f6a7c0039c0 2024-04-25T10:17:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a9480f640 1 --2- 172.21.15.60:0/3538646765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a7c0051d0 0x7f6a7c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:07.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.389+0000 7f6a9480f640 1 -- 172.21.15.60:0/3538646765 reap_dead start 2024-04-25T10:17:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a8ffff640 1 --2- 172.21.15.60:0/3538646765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c0039c0 0x7f6a7c003de0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f6a78001780 tx=0x7f6a7801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc97d508d622fca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a8effd640 1 -- 172.21.15.60:0/3538646765 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a780013a0 con 0x7f6a7c0039c0 2024-04-25T10:17:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a8effd640 1 -- 172.21.15.60:0/3538646765 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a780052f0 con 0x7f6a7c0039c0 2024-04-25T10:17:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a8effd640 1 -- 172.21.15.60:0/3538646765 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78003840 con 0x7f6a7c0039c0 2024-04-25T10:17:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a96272640 1 -- 172.21.15.60:0/3538646765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c0039c0 msgr2=0x7f6a7c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a96272640 1 --2- 172.21.15.60:0/3538646765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c0039c0 0x7f6a7c003de0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f6a78001780 tx=0x7f6a7801c400 comp rx=0 tx=0).stop 2024-04-25T10:17:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a96272640 1 -- 172.21.15.60:0/3538646765 shutdown_connections 2024-04-25T10:17:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a96272640 1 --2- 172.21.15.60:0/3538646765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c0039c0 0x7f6a7c003de0 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a96272640 1 -- 172.21.15.60:0/3538646765 >> 172.21.15.60:0/3538646765 conn(0x7f6a900670b0 msgr2=0x7f6a9006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a96272640 1 -- 172.21.15.60:0/3538646765 shutdown_connections 2024-04-25T10:17:07.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a96272640 1 -- 172.21.15.60:0/3538646765 wait complete. 2024-04-25T10:17:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a96272640 1 Processor -- start 2024-04-25T10:17:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a96272640 1 -- start start 2024-04-25T10:17:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a96272640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c0039c0 0x7f6a900acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a96272640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a900ae150 0x7f6a9015b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a96272640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a901e8f30 0x7f6a901e93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a96272640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9015fa70 con 0x7f6a7c0039c0 2024-04-25T10:17:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a96272640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9015f8f0 con 0x7f6a901e8f30 2024-04-25T10:17:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a96272640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a90072040 con 0x7f6a900ae150 2024-04-25T10:17:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a8ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c0039c0 0x7f6a900acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a8ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c0039c0 0x7f6a900acf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53244/0 (socket says 172.21.15.60:53244) 2024-04-25T10:17:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a8ffff640 1 -- 172.21.15.60:0/1434856653 learned_addr learned my addr 172.21.15.60:0/1434856653 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a9480f640 1 --2- 172.21.15.60:0/1434856653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a901e8f30 0x7f6a901e93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a8f7fe640 1 --2- 172.21.15.60:0/1434856653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a900ae150 0x7f6a9015b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a8ffff640 1 -- 172.21.15.60:0/1434856653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a900ae150 msgr2=0x7f6a9015b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a8ffff640 1 --2- 172.21.15.60:0/1434856653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a900ae150 0x7f6a9015b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a8ffff640 1 -- 172.21.15.60:0/1434856653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a901e8f30 msgr2=0x7f6a901e93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a8ffff640 1 --2- 172.21.15.60:0/1434856653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a901e8f30 0x7f6a901e93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a8ffff640 1 -- 172.21.15.60:0/1434856653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9006d2f0 con 0x7f6a7c0039c0 2024-04-25T10:17:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.393+0000 7f6a8f7fe640 1 --2- 172.21.15.60:0/1434856653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a900ae150 0x7f6a9015b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.397+0000 7f6a8ffff640 1 --2- 172.21.15.60:0/1434856653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c0039c0 0x7f6a900acf20 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f6a7801c230 tx=0x7f6a7c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.397+0000 7f6a8cff9640 1 -- 172.21.15.60:0/1434856653 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78007a40 con 0x7f6a7c0039c0 2024-04-25T10:17:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.397+0000 7f6a8cff9640 1 -- 172.21.15.60:0/1434856653 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a780ba070 con 0x7f6a7c0039c0 2024-04-25T10:17:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.397+0000 7f6a96272640 1 -- 172.21.15.60:0/1434856653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a900ae6d0 con 0x7f6a7c0039c0 2024-04-25T10:17:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.397+0000 7f6a8cff9640 1 -- 172.21.15.60:0/1434856653 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a780054a0 con 0x7f6a7c0039c0 2024-04-25T10:17:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.397+0000 7f6a96272640 1 -- 172.21.15.60:0/1434856653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9006d590 con 0x7f6a7c0039c0 2024-04-25T10:17:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.397+0000 7f6a96272640 1 -- 172.21.15.60:0/1434856653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9006bba0 con 0x7f6a7c0039c0 2024-04-25T10:17:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.397+0000 7f6a8cff9640 1 -- 172.21.15.60:0/1434856653 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a78007030 con 0x7f6a7c0039c0 2024-04-25T10:17:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.397+0000 7f6a8cff9640 1 --2- 172.21.15.60:0/1434856653 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6a70066f60 0x7f6a70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.397+0000 7f6a8cff9640 1 -- 172.21.15.60:0/1434856653 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f6a7811edf0 con 0x7f6a7c0039c0 2024-04-25T10:17:07.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.397+0000 7f6a8f7fe640 1 --2- 172.21.15.60:0/1434856653 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6a70066f60 0x7f6a70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.401+0000 7f6a8f7fe640 1 --2- 172.21.15.60:0/1434856653 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6a70066f60 0x7f6a70069420 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6a901434c0 tx=0x7f6a80001260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:07.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.405+0000 7f6a8cff9640 1 -- 172.21.15.60:0/1434856653 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a78016110 con 0x7f6a7c0039c0 2024-04-25T10:17:07.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.573+0000 7f6a96272640 1 -- 172.21.15.60:0/1434856653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6a90074770 con 0x7f6a7c0039c0 2024-04-25T10:17:07.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.577+0000 7f6a8cff9640 1 -- 172.21.15.60:0/1434856653 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6a780f0050 con 0x7f6a7c0039c0 2024-04-25T10:17:07.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.577+0000 7f6a96272640 1 -- 172.21.15.60:0/1434856653 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6a70066f60 msgr2=0x7f6a70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.577+0000 7f6a96272640 1 --2- 172.21.15.60:0/1434856653 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6a70066f60 0x7f6a70069420 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6a901434c0 tx=0x7f6a80001260 comp rx=0 tx=0).stop 2024-04-25T10:17:07.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.577+0000 7f6a96272640 1 -- 172.21.15.60:0/1434856653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c0039c0 msgr2=0x7f6a900acf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.577+0000 7f6a96272640 1 --2- 172.21.15.60:0/1434856653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c0039c0 0x7f6a900acf20 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f6a7801c230 tx=0x7f6a7c005840 comp rx=0 tx=0).stop 2024-04-25T10:17:07.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.577+0000 7f6a96272640 1 -- 172.21.15.60:0/1434856653 shutdown_connections 2024-04-25T10:17:07.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.577+0000 7f6a96272640 1 --2- 172.21.15.60:0/1434856653 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f6a70066f60 0x7f6a70069420 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.577+0000 7f6a96272640 1 --2- 172.21.15.60:0/1434856653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a901e8f30 0x7f6a901e93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.577+0000 7f6a96272640 1 --2- 172.21.15.60:0/1434856653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a900ae150 0x7f6a9015b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.577+0000 7f6a96272640 1 --2- 172.21.15.60:0/1434856653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c0039c0 0x7f6a900acf20 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.577+0000 7f6a96272640 1 -- 172.21.15.60:0/1434856653 >> 172.21.15.60:0/1434856653 conn(0x7f6a900670b0 msgr2=0x7f6a9015e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:07.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.577+0000 7f6a96272640 1 -- 172.21.15.60:0/1434856653 shutdown_connections 2024-04-25T10:17:07.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.577+0000 7f6a96272640 1 -- 172.21.15.60:0/1434856653 wait complete. 2024-04-25T10:17:07.602 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-25T10:17:07.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.721+0000 7f09c0325640 1 Processor -- start 2024-04-25T10:17:07.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.721+0000 7f09c0325640 1 -- start start 2024-04-25T10:17:07.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.721+0000 7f09c0325640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f09b806ec30 0x7f09b80cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.721+0000 7f09c0325640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09b80b37e0 0x7f09b80b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.721+0000 7f09c0325640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b806f130 con 0x7f09b815a7a0 2024-04-25T10:17:07.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.721+0000 7f09c0325640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b80cdaa0 con 0x7f09b806ec30 2024-04-25T10:17:07.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.721+0000 7f09c0325640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b806d2f0 con 0x7f09b80b37e0 2024-04-25T10:17:07.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.721+0000 7f09bd8c0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f09b806ec30 0x7f09b80cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.721+0000 7f09be8c2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09b80b37e0 0x7f09b80b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.721+0000 7f09bd8c0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f09b806ec30 0x7f09b80cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:36062/0 (socket says 172.21.15.60:36062) 2024-04-25T10:17:07.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.721+0000 7f09be0c1640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f09b815a7a0 0x7f09b8071ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:38688/0 (socket says 172.21.15.60:38688) 2024-04-25T10:17:07.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.721+0000 7f09bd8c0640 1 -- 172.21.15.60:0/1499339917 learned_addr learned my addr 172.21.15.60:0/1499339917 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:07.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.721+0000 7f09bd8c0640 1 -- 172.21.15.60:0/1499339917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09b80b37e0 msgr2=0x7f09b80b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:07.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.721+0000 7f09bd8c0640 1 --2- 172.21.15.60:0/1499339917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09b80b37e0 0x7f09b80b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.721+0000 7f09bd8c0640 1 -- 172.21.15.60:0/1499339917 >> v1:172.21.15.100:6789/0 conn(0x7f09b815a7a0 legacy=0x7f09b8071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.721+0000 7f09bd8c0640 1 -- 172.21.15.60:0/1499339917 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b806f410 con 0x7f09b806ec30 2024-04-25T10:17:07.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.725+0000 7f09bd8c0640 1 --2- 172.21.15.60:0/1499339917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09b806ec30 0x7f09b80cd560 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f09a800bb40 tx=0x7f09a801ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=777cfce97f8e5ace server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:07.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.725+0000 7f09bd0bf640 1 -- 172.21.15.60:0/1499339917 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a80b4070 con 0x7f09b806ec30 2024-04-25T10:17:07.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.725+0000 7f09bd0bf640 1 -- 172.21.15.60:0/1499339917 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a80a7bf0 con 0x7f09b806ec30 2024-04-25T10:17:07.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.725+0000 7f09bd0bf640 1 -- 172.21.15.60:0/1499339917 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a80af440 con 0x7f09b806ec30 2024-04-25T10:17:07.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.725+0000 7f09c0325640 1 -- 172.21.15.60:0/1499339917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09b806ec30 msgr2=0x7f09b80cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.725+0000 7f09c0325640 1 --2- 172.21.15.60:0/1499339917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09b806ec30 0x7f09b80cd560 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f09a800bb40 tx=0x7f09a801ec10 comp rx=0 tx=0).stop 2024-04-25T10:17:07.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.725+0000 7f09c0325640 1 -- 172.21.15.60:0/1499339917 shutdown_connections 2024-04-25T10:17:07.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.725+0000 7f09c0325640 1 --2- 172.21.15.60:0/1499339917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09b806ec30 0x7f09b80cd560 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.725+0000 7f09c0325640 1 --2- 172.21.15.60:0/1499339917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09b80b37e0 0x7f09b80b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.725+0000 7f09c0325640 1 -- 172.21.15.60:0/1499339917 >> 172.21.15.60:0/1499339917 conn(0x7f09b80b12d0 msgr2=0x7f09b80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:07.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.725+0000 7f09c0325640 1 -- 172.21.15.60:0/1499339917 shutdown_connections 2024-04-25T10:17:07.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.725+0000 7f09c0325640 1 -- 172.21.15.60:0/1499339917 wait complete. 2024-04-25T10:17:07.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09c0325640 1 Processor -- start 2024-04-25T10:17:07.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09c0325640 1 -- start start 2024-04-25T10:17:07.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09c0325640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09b806ec30 0x7f09b80b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09c0325640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09b80b37e0 0x7f09b8162160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09c0325640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09b815a7a0 0x7f09b81626a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09c0325640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b806e8c0 con 0x7f09b815a7a0 2024-04-25T10:17:07.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09c0325640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b806e740 con 0x7f09b80b37e0 2024-04-25T10:17:07.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09c0325640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b806d8b0 con 0x7f09b806ec30 2024-04-25T10:17:07.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09be8c2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09b815a7a0 0x7f09b81626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09be8c2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09b815a7a0 0x7f09b81626a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53248/0 (socket says 172.21.15.60:53248) 2024-04-25T10:17:07.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09be8c2640 1 -- 172.21.15.60:0/3860839338 learned_addr learned my addr 172.21.15.60:0/3860839338 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:07.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09bd8c0640 1 --2- 172.21.15.60:0/3860839338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09b80b37e0 0x7f09b8162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09be0c1640 1 --2- 172.21.15.60:0/3860839338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09b806ec30 0x7f09b80b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09be8c2640 1 -- 172.21.15.60:0/3860839338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09b806ec30 msgr2=0x7f09b80b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09be8c2640 1 --2- 172.21.15.60:0/3860839338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09b806ec30 0x7f09b80b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09be8c2640 1 -- 172.21.15.60:0/3860839338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09b80b37e0 msgr2=0x7f09b8162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09be8c2640 1 --2- 172.21.15.60:0/3860839338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09b80b37e0 0x7f09b8162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09be8c2640 1 -- 172.21.15.60:0/3860839338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b8159600 con 0x7f09b815a7a0 2024-04-25T10:17:07.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09be8c2640 1 --2- 172.21.15.60:0/3860839338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09b815a7a0 0x7f09b81626a0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f09b000e930 tx=0x7f09b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:07.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09aeffd640 1 -- 172.21.15.60:0/3860839338 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0017070 con 0x7f09b815a7a0 2024-04-25T10:17:07.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09aeffd640 1 -- 172.21.15.60:0/3860839338 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b000c9a0 con 0x7f09b815a7a0 2024-04-25T10:17:07.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09aeffd640 1 -- 172.21.15.60:0/3860839338 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b000cd70 con 0x7f09b815a7a0 2024-04-25T10:17:07.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09c0325640 1 -- 172.21.15.60:0/3860839338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b81598e0 con 0x7f09b815a7a0 2024-04-25T10:17:07.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09c0325640 1 -- 172.21.15.60:0/3860839338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b813ddf0 con 0x7f09b815a7a0 2024-04-25T10:17:07.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.729+0000 7f09c0325640 1 -- 172.21.15.60:0/3860839338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b813d790 con 0x7f09b815a7a0 2024-04-25T10:17:07.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.733+0000 7f09aeffd640 1 -- 172.21.15.60:0/3860839338 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f09b001a070 con 0x7f09b815a7a0 2024-04-25T10:17:07.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.733+0000 7f09aeffd640 1 --2- 172.21.15.60:0/3860839338 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f099c066f60 0x7f099c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.733+0000 7f09aeffd640 1 -- 172.21.15.60:0/3860839338 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f09b0084ea0 con 0x7f09b815a7a0 2024-04-25T10:17:07.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.733+0000 7f09be0c1640 1 --2- 172.21.15.60:0/3860839338 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f099c066f60 0x7f099c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.733+0000 7f09be0c1640 1 --2- 172.21.15.60:0/3860839338 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f099c066f60 0x7f099c069420 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005fd0 tx=0x7f09b4005ba0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:07.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.737+0000 7f09aeffd640 1 -- 172.21.15.60:0/3860839338 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f09b813d790 con 0x7f09b815a7a0 2024-04-25T10:17:07.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:07.957+0000 7f09c0325640 1 -- 172.21.15.60:0/3860839338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7f09b815b6d0 con 0x7f09b815a7a0 2024-04-25T10:17:08.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:08.245+0000 7f09aeffd640 1 -- 172.21.15.60:0/3860839338 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v539) v1 ==== 144+0+0 (secure 0 0 0) 0x7f09b0056120 con 0x7f09b815a7a0 2024-04-25T10:17:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:08.249+0000 7f09c0325640 1 -- 172.21.15.60:0/3860839338 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f099c066f60 msgr2=0x7f099c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:08.249+0000 7f09c0325640 1 --2- 172.21.15.60:0/3860839338 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f099c066f60 0x7f099c069420 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005fd0 tx=0x7f09b4005ba0 comp rx=0 tx=0).stop 2024-04-25T10:17:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:08.249+0000 7f09c0325640 1 -- 172.21.15.60:0/3860839338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09b815a7a0 msgr2=0x7f09b81626a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:08.249+0000 7f09c0325640 1 --2- 172.21.15.60:0/3860839338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09b815a7a0 0x7f09b81626a0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f09b000e930 tx=0x7f09b000edf0 comp rx=0 tx=0).stop 2024-04-25T10:17:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:08.249+0000 7f09c0325640 1 -- 172.21.15.60:0/3860839338 shutdown_connections 2024-04-25T10:17:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:08.249+0000 7f09c0325640 1 --2- 172.21.15.60:0/3860839338 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f099c066f60 0x7f099c069420 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:08.249+0000 7f09c0325640 1 --2- 172.21.15.60:0/3860839338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09b815a7a0 0x7f09b81626a0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:08.249+0000 7f09c0325640 1 --2- 172.21.15.60:0/3860839338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09b80b37e0 0x7f09b8162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:08.249+0000 7f09c0325640 1 --2- 172.21.15.60:0/3860839338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09b806ec30 0x7f09b80b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:08.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:08.249+0000 7f09c0325640 1 -- 172.21.15.60:0/3860839338 >> 172.21.15.60:0/3860839338 conn(0x7f09b80b12d0 msgr2=0x7f09b80ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:08.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:08.249+0000 7f09c0325640 1 -- 172.21.15.60:0/3860839338 shutdown_connections 2024-04-25T10:17:08.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:08.249+0000 7f09c0325640 1 -- 172.21.15.60:0/3860839338 wait complete. 2024-04-25T10:17:08.278 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:17:08.278 INFO:tasks.cephfs_test_runner: 2024-04-25T10:17:08.278 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T10:17:08.279 INFO:tasks.cephfs_test_runner:Ran 7 tests in 676.189s 2024-04-25T10:17:08.279 INFO:tasks.cephfs_test_runner: 2024-04-25T10:17:08.279 INFO:tasks.cephfs_test_runner:OK 2024-04-25T10:17:08.279 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T10:17:08.291 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T10:17:08.299 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T10:17:08.299 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:17:08.300 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:08.306 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:17:08.306 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T10:17:08.306 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:08.354 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16142 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:17:08.354 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:17:08.354 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:08.361 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:08.361 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T10:17:08.361 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:08.410 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336344k,nr_inodes=4084086,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19932 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274652k,nr_inodes=818663,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274656k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:08.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:08.410 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T10:17:08.416 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:17:08.416 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:17:08.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:08.417 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:08.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:08.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:08.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:08.535 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-04-25T10:17:08.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:08.711 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-25T10:17:08.717 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T10:17:08.718 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:17:08.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:08.718 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:17:08.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:08.774 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:17:08.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:08.867 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-04-25T10:17:09.003 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T10:17:09.012 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:17:09.013 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:17:09.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.141+0000 7f1f95586640 1 Processor -- start 2024-04-25T10:17:09.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.141+0000 7f1f95586640 1 -- start start 2024-04-25T10:17:09.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.141+0000 7f1f95586640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1f9006c330 0x7f1f900701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.141+0000 7f1f95586640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1f9015b940 0x7f1f9006dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.141+0000 7f1f95586640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9006e470 con 0x7f1f9015b560 2024-04-25T10:17:09.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.141+0000 7f1f95586640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900b31a0 con 0x7f1f9015b940 2024-04-25T10:17:09.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.141+0000 7f1f95586640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9006e210 con 0x7f1f9006c330 2024-04-25T10:17:09.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.141+0000 7f1f8f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1f9015b940 0x7f1f9006dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.141+0000 7f1f8f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1f9015b940 0x7f1f9006dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46324/0 (socket says 172.21.15.60:46324) 2024-04-25T10:17:09.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.141+0000 7f1f8f7fe640 1 -- 172.21.15.60:0/2758295393 learned_addr learned my addr 172.21.15.60:0/2758295393 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:09.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.141+0000 7f1f8e7fc640 1 --2- 172.21.15.60:0/2758295393 >> v2:172.21.15.100:3300/0 conn(0x7f1f9006c330 0x7f1f900701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.141+0000 7f1f8f7fe640 1 -- 172.21.15.60:0/2758295393 >> v2:172.21.15.100:3300/0 conn(0x7f1f9006c330 msgr2=0x7f1f900701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:09.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.141+0000 7f1f8f7fe640 1 --2- 172.21.15.60:0/2758295393 >> v2:172.21.15.100:3300/0 conn(0x7f1f9006c330 0x7f1f900701e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.141+0000 7f1f8f7fe640 1 -- 172.21.15.60:0/2758295393 >> v1:172.21.15.100:6789/0 conn(0x7f1f9015b560 legacy=0x7f1f9006bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:09.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.141+0000 7f1f8f7fe640 1 -- 172.21.15.60:0/2758295393 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9006e750 con 0x7f1f9015b940 2024-04-25T10:17:09.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.145+0000 7f1f8f7fe640 1 --2- 172.21.15.60:0/2758295393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9015b940 0x7f1f9006dcd0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f1f90067420 tx=0x7f1f8001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7334a00e3b1f6e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:09.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.145+0000 7f1f8dffb640 1 -- 172.21.15.60:0/2758295393 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f800aa430 con 0x7f1f9015b940 2024-04-25T10:17:09.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.145+0000 7f1f8dffb640 1 -- 172.21.15.60:0/2758295393 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f800af070 con 0x7f1f9015b940 2024-04-25T10:17:09.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.145+0000 7f1f8dffb640 1 -- 172.21.15.60:0/2758295393 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f800aa7f0 con 0x7f1f9015b940 2024-04-25T10:17:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.145+0000 7f1f95586640 1 -- 172.21.15.60:0/2758295393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9015b940 msgr2=0x7f1f9006dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.145+0000 7f1f95586640 1 --2- 172.21.15.60:0/2758295393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9015b940 0x7f1f9006dcd0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f1f90067420 tx=0x7f1f8001c4b0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.145+0000 7f1f95586640 1 -- 172.21.15.60:0/2758295393 shutdown_connections 2024-04-25T10:17:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.145+0000 7f1f95586640 1 --2- 172.21.15.60:0/2758295393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9015b940 0x7f1f9006dcd0 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.145+0000 7f1f95586640 1 --2- 172.21.15.60:0/2758295393 >> v2:172.21.15.100:3300/0 conn(0x7f1f9006c330 0x7f1f900701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.145+0000 7f1f95586640 1 -- 172.21.15.60:0/2758295393 >> 172.21.15.60:0/2758295393 conn(0x7f1f90071b90 msgr2=0x7f1f90071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.145+0000 7f1f95586640 1 -- 172.21.15.60:0/2758295393 shutdown_connections 2024-04-25T10:17:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.145+0000 7f1f95586640 1 -- 172.21.15.60:0/2758295393 wait complete. 2024-04-25T10:17:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f95586640 1 Processor -- start 2024-04-25T10:17:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f95586640 1 -- start start 2024-04-25T10:17:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f95586640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9006c330 0x7f1f90154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f95586640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f9015b560 0x7f1f900b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f95586640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f9015b940 0x7f1f90152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f95586640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900b41d0 con 0x7f1f9006c330 2024-04-25T10:17:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f95586640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900b4050 con 0x7f1f9015b560 2024-04-25T10:17:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f95586640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900b4350 con 0x7f1f9015b940 2024-04-25T10:17:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f8effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9006c330 0x7f1f90154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f8effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9006c330 0x7f1f90154c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46334/0 (socket says 172.21.15.60:46334) 2024-04-25T10:17:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f8effd640 1 -- 172.21.15.60:0/3967698405 learned_addr learned my addr 172.21.15.60:0/3967698405 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f8e7fc640 1 --2- 172.21.15.60:0/3967698405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f9015b560 0x7f1f900b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f8f7fe640 1 --2- 172.21.15.60:0/3967698405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f9015b940 0x7f1f90152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f8effd640 1 -- 172.21.15.60:0/3967698405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f9015b940 msgr2=0x7f1f90152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f8effd640 1 --2- 172.21.15.60:0/3967698405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f9015b940 0x7f1f90152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f8effd640 1 -- 172.21.15.60:0/3967698405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f9015b560 msgr2=0x7f1f900b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f8effd640 1 --2- 172.21.15.60:0/3967698405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f9015b560 0x7f1f900b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f8effd640 1 -- 172.21.15.60:0/3967698405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f90155570 con 0x7f1f9006c330 2024-04-25T10:17:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f8effd640 1 --2- 172.21.15.60:0/3967698405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9006c330 0x7f1f90154c40 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800c7c0 tx=0x7f1f8800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f877fe640 1 -- 172.21.15.60:0/3967698405 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f88016020 con 0x7f1f9006c330 2024-04-25T10:17:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f877fe640 1 -- 172.21.15.60:0/3967698405 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8800aa50 con 0x7f1f9006c330 2024-04-25T10:17:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f95586640 1 -- 172.21.15.60:0/3967698405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f90155850 con 0x7f1f9006c330 2024-04-25T10:17:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f877fe640 1 -- 172.21.15.60:0/3967698405 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f88012420 con 0x7f1f9006c330 2024-04-25T10:17:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.149+0000 7f1f95586640 1 -- 172.21.15.60:0/3967698405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f90155e10 con 0x7f1f9006c330 2024-04-25T10:17:09.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.153+0000 7f1f95586640 1 -- 172.21.15.60:0/3967698405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f90074410 con 0x7f1f9006c330 2024-04-25T10:17:09.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.153+0000 7f1f877fe640 1 -- 172.21.15.60:0/3967698405 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f88019020 con 0x7f1f9006c330 2024-04-25T10:17:09.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.153+0000 7f1f877fe640 1 --2- 172.21.15.60:0/3967698405 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f70066f60 0x7f1f70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.153+0000 7f1f877fe640 1 -- 172.21.15.60:0/3967698405 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f1f88084ec0 con 0x7f1f9006c330 2024-04-25T10:17:09.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.153+0000 7f1f8e7fc640 1 --2- 172.21.15.60:0/3967698405 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f70066f60 0x7f1f70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.153+0000 7f1f8e7fc640 1 --2- 172.21.15.60:0/3967698405 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f70066f60 0x7f1f70069420 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1f901f77a0 tx=0x7f1f78008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:09.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.157+0000 7f1f877fe640 1 -- 172.21.15.60:0/3967698405 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f88020030 con 0x7f1f9006c330 2024-04-25T10:17:09.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.317+0000 7f1f95586640 1 -- 172.21.15.60:0/3967698405 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f90074130 con 0x7f1f70066f60 2024-04-25T10:17:09.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.317+0000 7f1f877fe640 1 -- 172.21.15.60:0/3967698405 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13520 (secure 0 0 0) 0x7f1f90074130 con 0x7f1f70066f60 2024-04-25T10:17:09.324 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:17:09.324 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:17:09.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.321+0000 7f1f95586640 1 -- 172.21.15.60:0/3967698405 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f70066f60 msgr2=0x7f1f70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.321+0000 7f1f95586640 1 --2- 172.21.15.60:0/3967698405 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f70066f60 0x7f1f70069420 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1f901f77a0 tx=0x7f1f78008040 comp rx=0 tx=0).stop 2024-04-25T10:17:09.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.321+0000 7f1f95586640 1 -- 172.21.15.60:0/3967698405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9006c330 msgr2=0x7f1f90154c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.321+0000 7f1f95586640 1 --2- 172.21.15.60:0/3967698405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9006c330 0x7f1f90154c40 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800c7c0 tx=0x7f1f8800cc80 comp rx=0 tx=0).stop 2024-04-25T10:17:09.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.321+0000 7f1f95586640 1 -- 172.21.15.60:0/3967698405 shutdown_connections 2024-04-25T10:17:09.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.321+0000 7f1f95586640 1 --2- 172.21.15.60:0/3967698405 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1f70066f60 0x7f1f70069420 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.321+0000 7f1f95586640 1 --2- 172.21.15.60:0/3967698405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f9015b940 0x7f1f90152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.321+0000 7f1f95586640 1 --2- 172.21.15.60:0/3967698405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f9015b560 0x7f1f900b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.321+0000 7f1f95586640 1 --2- 172.21.15.60:0/3967698405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9006c330 0x7f1f90154c40 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.321+0000 7f1f95586640 1 -- 172.21.15.60:0/3967698405 >> 172.21.15.60:0/3967698405 conn(0x7f1f90071b90 msgr2=0x7f1f9015be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:09.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.321+0000 7f1f95586640 1 -- 172.21.15.60:0/3967698405 shutdown_connections 2024-04-25T10:17:09.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.321+0000 7f1f95586640 1 -- 172.21.15.60:0/3967698405 wait complete. 2024-04-25T10:17:09.343 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-04-25T10:17:08.636598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":295012,"kb_used_data":24756,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754679708,"statfs":{"total":773094113280,"available":772792020992,"internally_reserved":0,"allocated":25350144,"data_stored":18794061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390725,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-265,"num_write":-175,"num_write_kb":-8291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.918917"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":347,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.421218+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:17:07.421218+0000","last_peered":"2024-04-25T10:17:07.421218+0000","last_clean":"2024-04-25T10:17:07.421218+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:17:07.421218+0000","last_undegraded":"2024-04-25T10:17:07.421218+0000","last_fullsized":"2024-04-25T10:17:07.421218+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509588,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2048000,"data_stored":1222053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509587,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542290,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":5504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":5636096,"data_stored":4820225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542291,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542291,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40844,"kb_used_data":5504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330996,"statfs":{"total":96636764160,"available":96594939904,"internally_reserved":0,"allocated":5636096,"data_stored":4820223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542291,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36748,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":1441792,"data_stored":625919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542291,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45000,"kb_used_data":5500,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326840,"statfs":{"total":96636764160,"available":96590684160,"internally_reserved":0,"allocated":5632000,"data_stored":4820218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542292,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37344,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334496,"statfs":{"total":96636764160,"available":96598523904,"internally_reserved":0,"allocated":2048000,"data_stored":1222053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:17:09.343 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:17:09.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.461+0000 7fe2ba851640 1 Processor -- start 2024-04-25T10:17:09.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.461+0000 7fe2ba851640 1 -- start start 2024-04-25T10:17:09.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.465+0000 7fe2ba851640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe2b40b4660 0x7fe2b415c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.465+0000 7fe2ba851640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b4071ca0 con 0x7fe2b415bc90 2024-04-25T10:17:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.465+0000 7fe2ba851640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b40cca40 con 0x7fe2b40ce7a0 2024-04-25T10:17:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.465+0000 7fe2ba851640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b40cc370 con 0x7fe2b40b4660 2024-04-25T10:17:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.465+0000 7fe2b8dee640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe2b40b4660 0x7fe2b415c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.465+0000 7fe2b8dee640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe2b40b4660 0x7fe2b415c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46336/0 (socket says 172.21.15.60:46336) 2024-04-25T10:17:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.465+0000 7fe2b8dee640 1 -- 172.21.15.60:0/2912745999 learned_addr learned my addr 172.21.15.60:0/2912745999 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.465+0000 7fe2b37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe2b415bc90 0x7fe2b415c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40818/0 (socket says 172.21.15.60:40818) 2024-04-25T10:17:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.465+0000 7fe2b8dee640 1 -- 172.21.15.60:0/2912745999 >> v1:172.21.15.100:6789/0 conn(0x7fe2b40ce7a0 legacy=0x7fe2b40ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.465+0000 7fe2b8dee640 1 -- 172.21.15.60:0/2912745999 >> v1:172.21.15.60:6789/0 conn(0x7fe2b415bc90 legacy=0x7fe2b415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.465+0000 7fe2b8dee640 1 -- 172.21.15.60:0/2912745999 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b40ccd20 con 0x7fe2b40b4660 2024-04-25T10:17:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.465+0000 7fe2b2ffd640 1 -- 172.21.15.60:0/2912745999 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621233633 0 0) 0x7fe2b4071ca0 con 0x7fe2b415bc90 2024-04-25T10:17:09.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.465+0000 7fe2b8dee640 1 --2- 172.21.15.60:0/2912745999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b40b4660 0x7fe2b415c930 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009e50 tx=0x7fe2ac01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47df2741f7d3e74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:09.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b2ffd640 1 -- 172.21.15.60:0/2912745999 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac0a56c0 con 0x7fe2b40b4660 2024-04-25T10:17:09.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b2ffd640 1 -- 172.21.15.60:0/2912745999 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ac0a5860 con 0x7fe2b40b4660 2024-04-25T10:17:09.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b2ffd640 1 -- 172.21.15.60:0/2912745999 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac0a5c30 con 0x7fe2b40b4660 2024-04-25T10:17:09.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2ba851640 1 -- 172.21.15.60:0/2912745999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b40b4660 msgr2=0x7fe2b415c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2ba851640 1 --2- 172.21.15.60:0/2912745999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b40b4660 0x7fe2b415c930 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009e50 tx=0x7fe2ac01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2ba851640 1 -- 172.21.15.60:0/2912745999 shutdown_connections 2024-04-25T10:17:09.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2ba851640 1 --2- 172.21.15.60:0/2912745999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b40b4660 0x7fe2b415c930 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2ba851640 1 -- 172.21.15.60:0/2912745999 >> 172.21.15.60:0/2912745999 conn(0x7fe2b40b12d0 msgr2=0x7fe2b40b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:09.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2ba851640 1 -- 172.21.15.60:0/2912745999 shutdown_connections 2024-04-25T10:17:09.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2ba851640 1 -- 172.21.15.60:0/2912745999 wait complete. 2024-04-25T10:17:09.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2ba851640 1 Processor -- start 2024-04-25T10:17:09.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2ba851640 1 -- start start 2024-04-25T10:17:09.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2ba851640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2b40ce7a0 0x7fe2b406ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2ba851640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2b415bc90 0x7fe2b4158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2ba851640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b413bdb0 0x7fe2b4152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2ba851640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b41695d0 con 0x7fe2b413bdb0 2024-04-25T10:17:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2ba851640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b40b3db0 con 0x7fe2b415bc90 2024-04-25T10:17:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2ba851640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b4169750 con 0x7fe2b40ce7a0 2024-04-25T10:17:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b8dee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b413bdb0 0x7fe2b4152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b8dee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b413bdb0 0x7fe2b4152590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46344/0 (socket says 172.21.15.60:46344) 2024-04-25T10:17:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b8dee640 1 -- 172.21.15.60:0/1674764449 learned_addr learned my addr 172.21.15.60:0/1674764449 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b37fe640 1 --2- 172.21.15.60:0/1674764449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2b415bc90 0x7fe2b4158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b3fff640 1 --2- 172.21.15.60:0/1674764449 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2b40ce7a0 0x7fe2b406ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b8dee640 1 -- 172.21.15.60:0/1674764449 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2b40ce7a0 msgr2=0x7fe2b406ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b8dee640 1 --2- 172.21.15.60:0/1674764449 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2b40ce7a0 0x7fe2b406ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b8dee640 1 -- 172.21.15.60:0/1674764449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2b415bc90 msgr2=0x7fe2b4158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b8dee640 1 --2- 172.21.15.60:0/1674764449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2b415bc90 0x7fe2b4158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b8dee640 1 -- 172.21.15.60:0/1674764449 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b4159630 con 0x7fe2b413bdb0 2024-04-25T10:17:09.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b37fe640 1 --2- 172.21.15.60:0/1674764449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2b415bc90 0x7fe2b4158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:09.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b8dee640 1 --2- 172.21.15.60:0/1674764449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b413bdb0 0x7fe2b4152590 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac019600 tx=0x7fe2b415d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:09.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b0ff9640 1 -- 172.21.15.60:0/1674764449 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac0b8670 con 0x7fe2b413bdb0 2024-04-25T10:17:09.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b0ff9640 1 -- 172.21.15.60:0/1674764449 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ac0be070 con 0x7fe2b413bdb0 2024-04-25T10:17:09.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.469+0000 7fe2b0ff9640 1 -- 172.21.15.60:0/1674764449 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac0b8a30 con 0x7fe2b413bdb0 2024-04-25T10:17:09.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.473+0000 7fe2ba851640 1 -- 172.21.15.60:0/1674764449 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b4159910 con 0x7fe2b413bdb0 2024-04-25T10:17:09.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.473+0000 7fe2ba851640 1 -- 172.21.15.60:0/1674764449 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b4159ed0 con 0x7fe2b413bdb0 2024-04-25T10:17:09.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.473+0000 7fe2ba851640 1 -- 172.21.15.60:0/1674764449 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b4074770 con 0x7fe2b413bdb0 2024-04-25T10:17:09.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.473+0000 7fe2b0ff9640 1 -- 172.21.15.60:0/1674764449 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2ac0b8bd0 con 0x7fe2b413bdb0 2024-04-25T10:17:09.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.473+0000 7fe2b0ff9640 1 --2- 172.21.15.60:0/1674764449 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe2a0066f60 0x7fe2a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.473+0000 7fe2b0ff9640 1 -- 172.21.15.60:0/1674764449 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fe2ac11ff20 con 0x7fe2b413bdb0 2024-04-25T10:17:09.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.473+0000 7fe2b3fff640 1 --2- 172.21.15.60:0/1674764449 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe2a0066f60 0x7fe2a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.473+0000 7fe2b3fff640 1 --2- 172.21.15.60:0/1674764449 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe2a0066f60 0x7fe2a0069420 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400a750 tx=0x7fe2a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:09.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.477+0000 7fe2b0ff9640 1 -- 172.21.15.60:0/1674764449 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2ac0165e0 con 0x7fe2b413bdb0 2024-04-25T10:17:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.633+0000 7fe2ba851640 1 -- 172.21.15.60:0/1674764449 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2b4157660 con 0x7fe2a0066f60 2024-04-25T10:17:09.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.633+0000 7fe2b0ff9640 1 -- 172.21.15.60:0/1674764449 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13520 (secure 0 0 0) 0x7fe2b4157660 con 0x7fe2a0066f60 2024-04-25T10:17:09.640 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:17:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:17:09.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.637+0000 7fe2ba851640 1 -- 172.21.15.60:0/1674764449 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe2a0066f60 msgr2=0x7fe2a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.637+0000 7fe2ba851640 1 --2- 172.21.15.60:0/1674764449 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe2a0066f60 0x7fe2a0069420 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400a750 tx=0x7fe2a4008040 comp rx=0 tx=0).stop 2024-04-25T10:17:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.637+0000 7fe2ba851640 1 -- 172.21.15.60:0/1674764449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b413bdb0 msgr2=0x7fe2b4152590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.637+0000 7fe2ba851640 1 --2- 172.21.15.60:0/1674764449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b413bdb0 0x7fe2b4152590 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac019600 tx=0x7fe2b415d680 comp rx=0 tx=0).stop 2024-04-25T10:17:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.637+0000 7fe2ba851640 1 -- 172.21.15.60:0/1674764449 shutdown_connections 2024-04-25T10:17:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.637+0000 7fe2ba851640 1 --2- 172.21.15.60:0/1674764449 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fe2a0066f60 0x7fe2a0069420 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.637+0000 7fe2ba851640 1 --2- 172.21.15.60:0/1674764449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b413bdb0 0x7fe2b4152590 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.637+0000 7fe2ba851640 1 --2- 172.21.15.60:0/1674764449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2b415bc90 0x7fe2b4158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.637+0000 7fe2ba851640 1 --2- 172.21.15.60:0/1674764449 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2b40ce7a0 0x7fe2b406ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.637+0000 7fe2ba851640 1 -- 172.21.15.60:0/1674764449 >> 172.21.15.60:0/1674764449 conn(0x7fe2b40b12d0 msgr2=0x7fe2b40b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.637+0000 7fe2ba851640 1 -- 172.21.15.60:0/1674764449 shutdown_connections 2024-04-25T10:17:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.637+0000 7fe2ba851640 1 -- 172.21.15.60:0/1674764449 wait complete. 2024-04-25T10:17:09.659 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-04-25T10:17:08.636598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":295012,"kb_used_data":24756,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754679708,"statfs":{"total":773094113280,"available":772792020992,"internally_reserved":0,"allocated":25350144,"data_stored":18794061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390725,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-265,"num_write":-175,"num_write_kb":-8291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.918917"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":347,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.421218+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:17:07.421218+0000","last_peered":"2024-04-25T10:17:07.421218+0000","last_clean":"2024-04-25T10:17:07.421218+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:17:07.421218+0000","last_undegraded":"2024-04-25T10:17:07.421218+0000","last_fullsized":"2024-04-25T10:17:07.421218+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509588,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2048000,"data_stored":1222053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509587,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542290,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":5504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":5636096,"data_stored":4820225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542291,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542291,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40844,"kb_used_data":5504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330996,"statfs":{"total":96636764160,"available":96594939904,"internally_reserved":0,"allocated":5636096,"data_stored":4820223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542291,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36748,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":1441792,"data_stored":625919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542291,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45000,"kb_used_data":5500,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326840,"statfs":{"total":96636764160,"available":96590684160,"internally_reserved":0,"allocated":5632000,"data_stored":4820218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542292,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37344,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334496,"statfs":{"total":96636764160,"available":96598523904,"internally_reserved":0,"allocated":2048000,"data_stored":1222053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:17:09.659 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:17:09.660 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:17:09.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f6600250640 1 Processor -- start 2024-04-25T10:17:09.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f6600250640 1 -- start start 2024-04-25T10:17:09.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f6600250640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f65f815bc90 0x7f65f815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f6600250640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f80b4660 0x7f65f815c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f6600250640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f8071ca0 con 0x7f65f80ce7a0 2024-04-25T10:17:09.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f6600250640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f80cc370 con 0x7f65f815bc90 2024-04-25T10:17:09.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f6600250640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80cca40 con 0x7f65f80b4660 2024-04-25T10:17:09.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f65fdfec640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f65f80ce7a0 0x7f65f80ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:48030/0 (socket says 172.21.15.60:48030) 2024-04-25T10:17:09.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f65fdfec640 1 -- 172.21.15.60:0/688031822 learned_addr learned my addr 172.21.15.60:0/688031822 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:09.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f65fd7eb640 1 --2- 172.21.15.60:0/688031822 >> v2:172.21.15.100:3300/0 conn(0x7f65f815bc90 0x7f65f815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f65fe7ed640 1 --2- 172.21.15.60:0/688031822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f80b4660 0x7f65f815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f65fcfea640 1 -- 172.21.15.60:0/688031822 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150642718 0 0) 0x7f65f8071ca0 con 0x7f65f80ce7a0 2024-04-25T10:17:09.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f65fcfea640 1 -- 172.21.15.60:0/688031822 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e4003530 con 0x7f65f80ce7a0 2024-04-25T10:17:09.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f65fe7ed640 1 -- 172.21.15.60:0/688031822 >> v2:172.21.15.100:3300/0 conn(0x7f65f815bc90 msgr2=0x7f65f815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:09.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f65fe7ed640 1 --2- 172.21.15.60:0/688031822 >> v2:172.21.15.100:3300/0 conn(0x7f65f815bc90 0x7f65f815c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f65fe7ed640 1 -- 172.21.15.60:0/688031822 >> v1:172.21.15.100:6789/0 conn(0x7f65f80ce7a0 legacy=0x7f65f80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.773+0000 7f65fe7ed640 1 -- 172.21.15.60:0/688031822 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f80ccd20 con 0x7f65f80b4660 2024-04-25T10:17:09.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.777+0000 7f65fe7ed640 1 --2- 172.21.15.60:0/688031822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f80b4660 0x7f65f815c760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009d90 tx=0x7f65f001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d5d5344de9712c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:09.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.777+0000 7f65fcfea640 1 -- 172.21.15.60:0/688031822 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f00b2070 con 0x7f65f80b4660 2024-04-25T10:17:09.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.777+0000 7f65fcfea640 1 -- 172.21.15.60:0/688031822 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f00a5830 con 0x7f65f80b4660 2024-04-25T10:17:09.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.777+0000 7f65fcfea640 1 -- 172.21.15.60:0/688031822 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f00a5be0 con 0x7f65f80b4660 2024-04-25T10:17:09.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.777+0000 7f6600250640 1 -- 172.21.15.60:0/688031822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f80b4660 msgr2=0x7f65f815c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.777+0000 7f6600250640 1 --2- 172.21.15.60:0/688031822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f80b4660 0x7f65f815c760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009d90 tx=0x7f65f001c970 comp rx=0 tx=0).stop 2024-04-25T10:17:09.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.777+0000 7f6600250640 1 -- 172.21.15.60:0/688031822 shutdown_connections 2024-04-25T10:17:09.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.777+0000 7f6600250640 1 --2- 172.21.15.60:0/688031822 >> v2:172.21.15.100:3300/0 conn(0x7f65f815bc90 0x7f65f815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.777+0000 7f6600250640 1 --2- 172.21.15.60:0/688031822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f80b4660 0x7f65f815c760 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.777+0000 7f6600250640 1 -- 172.21.15.60:0/688031822 >> 172.21.15.60:0/688031822 conn(0x7f65f80b12d0 msgr2=0x7f65f80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:09.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.777+0000 7f6600250640 1 -- 172.21.15.60:0/688031822 shutdown_connections 2024-04-25T10:17:09.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.777+0000 7f6600250640 1 -- 172.21.15.60:0/688031822 wait complete. 2024-04-25T10:17:09.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.777+0000 7f6600250640 1 Processor -- start 2024-04-25T10:17:09.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.777+0000 7f6600250640 1 -- start start 2024-04-25T10:17:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f6600250640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f80ce7a0 0x7f65f81493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f6600250640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f815bc90 0x7f65f814e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f6600250640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f814e740 0x7f65f8153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f6600250640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f81685c0 con 0x7f65f815bc90 2024-04-25T10:17:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f6600250640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80b3db0 con 0x7f65f814e740 2024-04-25T10:17:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f6600250640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f8168740 con 0x7f65f80ce7a0 2024-04-25T10:17:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65fd7eb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f815bc90 0x7f65f814e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65fd7eb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f815bc90 0x7f65f814e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46350/0 (socket says 172.21.15.60:46350) 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65fd7eb640 1 -- 172.21.15.60:0/2118906084 learned_addr learned my addr 172.21.15.60:0/2118906084 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65fdfec640 1 --2- 172.21.15.60:0/2118906084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f80ce7a0 0x7f65f81493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65fe7ed640 1 --2- 172.21.15.60:0/2118906084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f814e740 0x7f65f8153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65fd7eb640 1 -- 172.21.15.60:0/2118906084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f80ce7a0 msgr2=0x7f65f81493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65fd7eb640 1 --2- 172.21.15.60:0/2118906084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f80ce7a0 0x7f65f81493d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65fd7eb640 1 -- 172.21.15.60:0/2118906084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f814e740 msgr2=0x7f65f8153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65fd7eb640 1 --2- 172.21.15.60:0/2118906084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f814e740 0x7f65f8153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65fd7eb640 1 -- 172.21.15.60:0/2118906084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f814b470 con 0x7f65f815bc90 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65fd7eb640 1 --2- 172.21.15.60:0/2118906084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f815bc90 0x7f65f814e200 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f65e800e930 tx=0x7f65e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65ee7fc640 1 -- 172.21.15.60:0/2118906084 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8017070 con 0x7f65f815bc90 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65ee7fc640 1 -- 172.21.15.60:0/2118906084 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e800ca00 con 0x7f65f815bc90 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f6600250640 1 -- 172.21.15.60:0/2118906084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f8149d70 con 0x7f65f815bc90 2024-04-25T10:17:09.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65ee7fc640 1 -- 172.21.15.60:0/2118906084 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e800cdb0 con 0x7f65f815bc90 2024-04-25T10:17:09.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f6600250640 1 -- 172.21.15.60:0/2118906084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f814a330 con 0x7f65f815bc90 2024-04-25T10:17:09.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f6600250640 1 -- 172.21.15.60:0/2118906084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f8074770 con 0x7f65f815bc90 2024-04-25T10:17:09.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65ee7fc640 1 -- 172.21.15.60:0/2118906084 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65e801a070 con 0x7f65f815bc90 2024-04-25T10:17:09.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65ee7fc640 1 --2- 172.21.15.60:0/2118906084 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65dc066f60 0x7f65dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65ee7fc640 1 -- 172.21.15.60:0/2118906084 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f65e8084d70 con 0x7f65f815bc90 2024-04-25T10:17:09.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.781+0000 7f65fdfec640 1 --2- 172.21.15.60:0/2118906084 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65dc066f60 0x7f65dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.785+0000 7f65fdfec640 1 --2- 172.21.15.60:0/2118906084 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65dc066f60 0x7f65dc069420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f65f4005890 tx=0x7f65f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.785+0000 7f65ee7fc640 1 -- 172.21.15.60:0/2118906084 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65e8056120 con 0x7f65f815bc90 2024-04-25T10:17:09.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.937+0000 7f6600250640 1 -- 172.21.15.60:0/2118906084 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f65f8154d40 con 0x7f65dc066f60 2024-04-25T10:17:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.941+0000 7f65ee7fc640 1 -- 172.21.15.60:0/2118906084 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13520 (secure 0 0 0) 0x7f65f8154d40 con 0x7f65dc066f60 2024-04-25T10:17:09.947 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:17:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:17:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.941+0000 7f6600250640 1 -- 172.21.15.60:0/2118906084 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65dc066f60 msgr2=0x7f65dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.941+0000 7f6600250640 1 --2- 172.21.15.60:0/2118906084 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65dc066f60 0x7f65dc069420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f65f4005890 tx=0x7f65f4009040 comp rx=0 tx=0).stop 2024-04-25T10:17:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.941+0000 7f6600250640 1 -- 172.21.15.60:0/2118906084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f815bc90 msgr2=0x7f65f814e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.941+0000 7f6600250640 1 --2- 172.21.15.60:0/2118906084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f815bc90 0x7f65f814e200 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f65e800e930 tx=0x7f65e800edf0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.941+0000 7f6600250640 1 -- 172.21.15.60:0/2118906084 shutdown_connections 2024-04-25T10:17:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.941+0000 7f6600250640 1 --2- 172.21.15.60:0/2118906084 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f65dc066f60 0x7f65dc069420 secure :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f65f4005890 tx=0x7f65f4009040 comp rx=0 tx=0).stop 2024-04-25T10:17:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.941+0000 7f6600250640 1 --2- 172.21.15.60:0/2118906084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f814e740 0x7f65f8153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.941+0000 7f6600250640 1 --2- 172.21.15.60:0/2118906084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f815bc90 0x7f65f814e200 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.941+0000 7f6600250640 1 --2- 172.21.15.60:0/2118906084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f80ce7a0 0x7f65f81493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.941+0000 7f6600250640 1 -- 172.21.15.60:0/2118906084 >> 172.21.15.60:0/2118906084 conn(0x7f65f80b12d0 msgr2=0x7f65f8071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:09.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.941+0000 7f6600250640 1 -- 172.21.15.60:0/2118906084 shutdown_connections 2024-04-25T10:17:09.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:09.945+0000 7f6600250640 1 -- 172.21.15.60:0/2118906084 wait complete. 2024-04-25T10:17:09.964 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-04-25T10:17:08.636598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":295012,"kb_used_data":24756,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754679708,"statfs":{"total":773094113280,"available":772792020992,"internally_reserved":0,"allocated":25350144,"data_stored":18794061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390725,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-265,"num_write":-175,"num_write_kb":-8291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.918917"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":347,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.421218+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:17:07.421218+0000","last_peered":"2024-04-25T10:17:07.421218+0000","last_clean":"2024-04-25T10:17:07.421218+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:17:07.421218+0000","last_undegraded":"2024-04-25T10:17:07.421218+0000","last_fullsized":"2024-04-25T10:17:07.421218+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509588,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2048000,"data_stored":1222053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509587,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542290,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":5504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":5636096,"data_stored":4820225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542291,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542291,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40844,"kb_used_data":5504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330996,"statfs":{"total":96636764160,"available":96594939904,"internally_reserved":0,"allocated":5636096,"data_stored":4820223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542291,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36748,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":1441792,"data_stored":625919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542291,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45000,"kb_used_data":5500,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326840,"statfs":{"total":96636764160,"available":96590684160,"internally_reserved":0,"allocated":5632000,"data_stored":4820218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542292,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37344,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334496,"statfs":{"total":96636764160,"available":96598523904,"internally_reserved":0,"allocated":2048000,"data_stored":1222053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:17:09.965 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:10.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.069+0000 7fbd89bc0640 1 Processor -- start 2024-04-25T10:17:10.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.069+0000 7fbd89bc0640 1 -- start start 2024-04-25T10:17:10.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd89bc0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbd840d0a50 0x7fbd840b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd89bc0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd840ce2b0 0x7fbd840ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd89bc0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8406d200 con 0x7fbd8415deb0 2024-04-25T10:17:10.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd89bc0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd840cf2c0 con 0x7fbd840d0a50 2024-04-25T10:17:10.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd89bc0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840cefe0 con 0x7fbd840ce2b0 2024-04-25T10:17:10.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd837fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbd8415deb0 0x7fbd840d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40830/0 (socket says 172.21.15.60:40830) 2024-04-25T10:17:10.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd837fe640 1 -- 172.21.15.60:0/265473914 learned_addr learned my addr 172.21.15.60:0/265473914 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:10.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd827fc640 1 -- 172.21.15.60:0/265473914 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684093809 0 0) 0x7fbd8406d200 con 0x7fbd8415deb0 2024-04-25T10:17:10.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd82ffd640 1 --2- 172.21.15.60:0/265473914 >> v2:172.21.15.100:3300/0 conn(0x7fbd840d0a50 0x7fbd840b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd827fc640 1 -- 172.21.15.60:0/265473914 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd6c003530 con 0x7fbd8415deb0 2024-04-25T10:17:10.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd83fff640 1 --2- 172.21.15.60:0/265473914 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd840ce2b0 0x7fbd840ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd827fc640 1 -- 172.21.15.60:0/265473914 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 949984715 0 0) 0x7fbd6c003530 con 0x7fbd8415deb0 2024-04-25T10:17:10.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd827fc640 1 -- 172.21.15.60:0/265473914 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd840ce2b0 msgr2=0x7fbd840ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:10.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd827fc640 1 --2- 172.21.15.60:0/265473914 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd840ce2b0 0x7fbd840ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd827fc640 1 -- 172.21.15.60:0/265473914 >> v2:172.21.15.100:3300/0 conn(0x7fbd840d0a50 msgr2=0x7fbd840b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:10.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd827fc640 1 --2- 172.21.15.60:0/265473914 >> v2:172.21.15.100:3300/0 conn(0x7fbd840d0a50 0x7fbd840b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd827fc640 1 -- 172.21.15.60:0/265473914 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd840d1d70 con 0x7fbd8415deb0 2024-04-25T10:17:10.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd827fc640 1 -- 172.21.15.60:0/265473914 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fbd78001680 con 0x7fbd8415deb0 2024-04-25T10:17:10.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd827fc640 1 -- 172.21.15.60:0/265473914 >> v1:172.21.15.60:6789/0 conn(0x7fbd8415deb0 legacy=0x7fbd840d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd827fc640 1 --2- 172.21.15.60:0/265473914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd6c0039c0 0x7fbd6c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd827fc640 1 --2- 172.21.15.60:0/265473914 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd840ce2b0 0x7fbd6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd827fc640 1 --2- 172.21.15.60:0/265473914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6c004d50 0x7fbd6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd827fc640 1 -- 172.21.15.60:0/265473914 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840cf2c0 con 0x7fbd6c004d50 2024-04-25T10:17:10.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd827fc640 1 -- 172.21.15.60:0/265473914 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840cefe0 con 0x7fbd6c0039c0 2024-04-25T10:17:10.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd827fc640 1 -- 172.21.15.60:0/265473914 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c00ba50 con 0x7fbd840ce2b0 2024-04-25T10:17:10.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd83fff640 1 --2- 172.21.15.60:0/265473914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6c004d50 0x7fbd6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd82ffd640 1 --2- 172.21.15.60:0/265473914 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd840ce2b0 0x7fbd6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd837fe640 1 --2- 172.21.15.60:0/265473914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd6c0039c0 0x7fbd6c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd83fff640 1 -- 172.21.15.60:0/265473914 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd840ce2b0 msgr2=0x7fbd6c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:10.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd83fff640 1 --2- 172.21.15.60:0/265473914 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd840ce2b0 0x7fbd6c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd83fff640 1 -- 172.21.15.60:0/265473914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd6c0039c0 msgr2=0x7fbd6c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:10.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd83fff640 1 --2- 172.21.15.60:0/265473914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd6c0039c0 0x7fbd6c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd83fff640 1 -- 172.21.15.60:0/265473914 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c003530 con 0x7fbd6c004d50 2024-04-25T10:17:10.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd82ffd640 1 --2- 172.21.15.60:0/265473914 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd840ce2b0 0x7fbd6c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:10.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd837fe640 1 --2- 172.21.15.60:0/265473914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd6c0039c0 0x7fbd6c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:10.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.073+0000 7fbd83fff640 1 --2- 172.21.15.60:0/265473914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6c004d50 0x7fbd6c0051d0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c007360 tx=0x7fbd7c01ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d87bd71ff52a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:10.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd827fc640 1 -- 172.21.15.60:0/265473914 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0b2070 con 0x7fbd6c004d50 2024-04-25T10:17:10.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd827fc640 1 -- 172.21.15.60:0/265473914 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7c0a5d40 con 0x7fbd6c004d50 2024-04-25T10:17:10.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd827fc640 1 -- 172.21.15.60:0/265473914 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c009be0 con 0x7fbd6c004d50 2024-04-25T10:17:10.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd89bc0640 1 -- 172.21.15.60:0/265473914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6c004d50 msgr2=0x7fbd6c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd89bc0640 1 --2- 172.21.15.60:0/265473914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6c004d50 0x7fbd6c0051d0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c007360 tx=0x7fbd7c01ce40 comp rx=0 tx=0).stop 2024-04-25T10:17:10.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd83fff640 1 -- 172.21.15.60:0/265473914 reap_dead start 2024-04-25T10:17:10.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd89bc0640 1 -- 172.21.15.60:0/265473914 shutdown_connections 2024-04-25T10:17:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd89bc0640 1 -- 172.21.15.60:0/265473914 >> 172.21.15.60:0/265473914 conn(0x7fbd8406c090 msgr2=0x7fbd8406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd89bc0640 1 -- 172.21.15.60:0/265473914 shutdown_connections 2024-04-25T10:17:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd89bc0640 1 -- 172.21.15.60:0/265473914 wait complete. 2024-04-25T10:17:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd89bc0640 1 Processor -- start 2024-04-25T10:17:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd89bc0640 1 -- start start 2024-04-25T10:17:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd89bc0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd6c004d50 0x7fbd840cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd89bc0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd841549b0 0x7fbd8415f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd89bc0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd841f6d60 0x7fbd8414f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd89bc0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840ce760 con 0x7fbd841549b0 2024-04-25T10:17:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd89bc0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840d2330 con 0x7fbd841f6d60 2024-04-25T10:17:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd89bc0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840ce8e0 con 0x7fbd6c004d50 2024-04-25T10:17:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd82ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd841549b0 0x7fbd8415f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd82ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd841549b0 0x7fbd8415f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46368/0 (socket says 172.21.15.60:46368) 2024-04-25T10:17:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd82ffd640 1 -- 172.21.15.60:0/1597453026 learned_addr learned my addr 172.21.15.60:0/1597453026 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd83fff640 1 --2- 172.21.15.60:0/1597453026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd841f6d60 0x7fbd8414f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd837fe640 1 --2- 172.21.15.60:0/1597453026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd6c004d50 0x7fbd840cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd82ffd640 1 -- 172.21.15.60:0/1597453026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd6c004d50 msgr2=0x7fbd840cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd82ffd640 1 --2- 172.21.15.60:0/1597453026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd6c004d50 0x7fbd840cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd82ffd640 1 -- 172.21.15.60:0/1597453026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd841f6d60 msgr2=0x7fbd8414f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd82ffd640 1 --2- 172.21.15.60:0/1597453026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd841f6d60 0x7fbd8414f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd82ffd640 1 -- 172.21.15.60:0/1597453026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd84157510 con 0x7fbd841549b0 2024-04-25T10:17:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd82ffd640 1 --2- 172.21.15.60:0/1597453026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd841549b0 0x7fbd8415f9f0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fbd74010950 tx=0x7fbd74010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:10.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd6bfff640 1 -- 172.21.15.60:0/1597453026 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7400f040 con 0x7fbd841549b0 2024-04-25T10:17:10.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd6bfff640 1 -- 172.21.15.60:0/1597453026 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7400ec20 con 0x7fbd841549b0 2024-04-25T10:17:10.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd89bc0640 1 -- 172.21.15.60:0/1597453026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd841577f0 con 0x7fbd841549b0 2024-04-25T10:17:10.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd6bfff640 1 -- 172.21.15.60:0/1597453026 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd74004020 con 0x7fbd841549b0 2024-04-25T10:17:10.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.077+0000 7fbd89bc0640 1 -- 172.21.15.60:0/1597453026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8415f4d0 con 0x7fbd841549b0 2024-04-25T10:17:10.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.081+0000 7fbd89bc0640 1 -- 172.21.15.60:0/1597453026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd84074770 con 0x7fbd841549b0 2024-04-25T10:17:10.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.081+0000 7fbd6bfff640 1 -- 172.21.15.60:0/1597453026 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd740041c0 con 0x7fbd841549b0 2024-04-25T10:17:10.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.081+0000 7fbd6bfff640 1 --2- 172.21.15.60:0/1597453026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd70066f90 0x7fbd70069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.081+0000 7fbd6bfff640 1 -- 172.21.15.60:0/1597453026 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fbd740852c0 con 0x7fbd841549b0 2024-04-25T10:17:10.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.081+0000 7fbd837fe640 1 --2- 172.21.15.60:0/1597453026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd70066f90 0x7fbd70069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.081+0000 7fbd837fe640 1 --2- 172.21.15.60:0/1597453026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd70066f90 0x7fbd70069450 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbd78007960 tx=0x7fbd78009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:10.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.085+0000 7fbd6bfff640 1 -- 172.21.15.60:0/1597453026 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd74018030 con 0x7fbd841549b0 2024-04-25T10:17:10.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.237+0000 7fbd89bc0640 1 -- 172.21.15.60:0/1597453026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd840d4490 con 0x7fbd841549b0 2024-04-25T10:17:10.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.241+0000 7fbd6bfff640 1 -- 172.21.15.60:0/1597453026 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+13298 (secure 0 0 0) 0x7fbd740044b0 con 0x7fbd841549b0 2024-04-25T10:17:10.246 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:17:10.246 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":117,"fsid":"0ea93863-a079-45c4-abed-6e83540ea9fe","created":"2024-04-25T10:04:55.753075+0000","modified":"2024-04-25T10:17:07.230790+0000","last_up_change":"2024-04-25T10:05:07.819950+0000","last_in_change":"2024-04-25T10:04:58.818141+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:05:08.461484+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e461b596-4289-46c7-baac-b1127e3f8b0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6803","nonce":4056167511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6806","nonce":4056167511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6812","nonce":4056167511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":4056167511},{"type":"v1","addr":"172.21.15.60:6810","nonce":4056167511}]},"public_addr":"172.21.15.60:6803/4056167511","cluster_addr":"172.21.15.60:6806/4056167511","heartbeat_back_addr":"172.21.15.60:6812/4056167511","heartbeat_front_addr":"172.21.15.60:6810/4056167511","state":["exists","up"]},{"osd":1,"uuid":"4a2d4b54-a6f8-41f5-a25c-f06886b279dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6825","nonce":3313552458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6827","nonce":3313552458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6831","nonce":3313552458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3313552458},{"type":"v1","addr":"172.21.15.60:6829","nonce":3313552458}]},"public_addr":"172.21.15.60:6825/3313552458","cluster_addr":"172.21.15.60:6827/3313552458","heartbeat_back_addr":"172.21.15.60:6831/3313552458","heartbeat_front_addr":"172.21.15.60:6829/3313552458","state":["exists","up"]},{"osd":2,"uuid":"316be201-bf30-46fc-a685-ddd534dec41d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6802","nonce":1771147646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6807","nonce":1771147646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6818","nonce":1771147646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":1771147646},{"type":"v1","addr":"172.21.15.60:6813","nonce":1771147646}]},"public_addr":"172.21.15.60:6802/1771147646","cluster_addr":"172.21.15.60:6807/1771147646","heartbeat_back_addr":"172.21.15.60:6818/1771147646","heartbeat_front_addr":"172.21.15.60:6813/1771147646","state":["exists","up"]},{"osd":3,"uuid":"db5b8962-b524-423a-8c08-c644f9598b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6815","nonce":1867957156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6819","nonce":1867957156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6823","nonce":1867957156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":1867957156},{"type":"v1","addr":"172.21.15.60:6821","nonce":1867957156}]},"public_addr":"172.21.15.60:6815/1867957156","cluster_addr":"172.21.15.60:6819/1867957156","heartbeat_back_addr":"172.21.15.60:6823/1867957156","heartbeat_front_addr":"172.21.15.60:6821/1867957156","state":["exists","up"]},{"osd":4,"uuid":"37f3785c-f929-4fb1-8d93-cf297404e837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6809","nonce":3965541654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6811","nonce":3965541654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6815","nonce":3965541654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3965541654},{"type":"v1","addr":"172.21.15.100:6813","nonce":3965541654}]},"public_addr":"172.21.15.100:6809/3965541654","cluster_addr":"172.21.15.100:6811/3965541654","heartbeat_back_addr":"172.21.15.100:6815/3965541654","heartbeat_front_addr":"172.21.15.100:6813/3965541654","state":["exists","up"]},{"osd":5,"uuid":"452236ce-e60a-4cc1-8a06-3a550d19c0d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6801","nonce":3542120824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6803","nonce":3542120824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6807","nonce":3542120824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3542120824},{"type":"v1","addr":"172.21.15.100:6805","nonce":3542120824}]},"public_addr":"172.21.15.100:6801/3542120824","cluster_addr":"172.21.15.100:6803/3542120824","heartbeat_back_addr":"172.21.15.100:6807/3542120824","heartbeat_front_addr":"172.21.15.100:6805/3542120824","state":["exists","up"]},{"osd":6,"uuid":"99bd25b7-66bc-4987-ab26-8cdef85f643f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6825","nonce":4086390550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6827","nonce":4086390550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6831","nonce":4086390550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":4086390550},{"type":"v1","addr":"172.21.15.100:6829","nonce":4086390550}]},"public_addr":"172.21.15.100:6825/4086390550","cluster_addr":"172.21.15.100:6827/4086390550","heartbeat_back_addr":"172.21.15.100:6831/4086390550","heartbeat_front_addr":"172.21.15.100:6829/4086390550","state":["exists","up"]},{"osd":7,"uuid":"ffd96246-26fa-4837-a994-a1df9369b36e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6817","nonce":2771485684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6819","nonce":2771485684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6823","nonce":2771485684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2771485684},{"type":"v1","addr":"172.21.15.100:6821","nonce":2771485684}]},"public_addr":"172.21.15.100:6817/2771485684","cluster_addr":"172.21.15.100:6819/2771485684","heartbeat_back_addr":"172.21.15.100:6823/2771485684","heartbeat_front_addr":"172.21.15.100:6821/2771485684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.325840+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466772+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.344690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.329057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.341624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.262712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.324489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:05:04.466636+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/3461876440":"2024-04-26T10:17:04.216249+0000","172.21.15.60:6834/3461876440":"2024-04-26T10:17:04.216249+0000","172.21.15.60:6835/178428701":"2024-04-26T10:16:48.303277+0000","172.21.15.60:6834/178428701":"2024-04-26T10:16:48.303277+0000","172.21.15.60:6835/722171246":"2024-04-26T10:16:28.355776+0000","172.21.15.60:6834/722171246":"2024-04-26T10:16:28.355776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.241+0000 7fbd89bc0640 1 -- 172.21.15.60:0/1597453026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd70066f90 msgr2=0x7fbd70069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.241+0000 7fbd89bc0640 1 --2- 172.21.15.60:0/1597453026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd70066f90 0x7fbd70069450 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbd78007960 tx=0x7fbd78009040 comp rx=0 tx=0).stop 2024-04-25T10:17:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.241+0000 7fbd89bc0640 1 -- 172.21.15.60:0/1597453026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd841549b0 msgr2=0x7fbd8415f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.241+0000 7fbd89bc0640 1 --2- 172.21.15.60:0/1597453026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd841549b0 0x7fbd8415f9f0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fbd74010950 tx=0x7fbd74010e10 comp rx=0 tx=0).stop 2024-04-25T10:17:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.241+0000 7fbd89bc0640 1 -- 172.21.15.60:0/1597453026 shutdown_connections 2024-04-25T10:17:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.241+0000 7fbd89bc0640 1 --2- 172.21.15.60:0/1597453026 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fbd70066f90 0x7fbd70069450 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.241+0000 7fbd89bc0640 1 --2- 172.21.15.60:0/1597453026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd841f6d60 0x7fbd8414f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.241+0000 7fbd89bc0640 1 --2- 172.21.15.60:0/1597453026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd841549b0 0x7fbd8415f9f0 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.241+0000 7fbd89bc0640 1 --2- 172.21.15.60:0/1597453026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd6c004d50 0x7fbd840cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.241+0000 7fbd89bc0640 1 -- 172.21.15.60:0/1597453026 >> 172.21.15.60:0/1597453026 conn(0x7fbd8406c090 msgr2=0x7fbd840b8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:10.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.241+0000 7fbd89bc0640 1 -- 172.21.15.60:0/1597453026 shutdown_connections 2024-04-25T10:17:10.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:10.241+0000 7fbd89bc0640 1 -- 172.21.15.60:0/1597453026 wait complete. 2024-04-25T10:17:11.264 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T10:17:11.264 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:17:11.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.397+0000 7fb9b95f3640 1 Processor -- start 2024-04-25T10:17:11.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.397+0000 7fb9b95f3640 1 -- start start 2024-04-25T10:17:11.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.401+0000 7fb9b95f3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb9b4070990 0x7fb9b415c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.401+0000 7fb9b95f3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9b415cb70 0x7fb9b406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.401+0000 7fb9b95f3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b415be50 con 0x7fb9b40afc10 2024-04-25T10:17:11.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.401+0000 7fb9b95f3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b40c57c0 con 0x7fb9b4070990 2024-04-25T10:17:11.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.401+0000 7fb9b95f3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b415c1a0 con 0x7fb9b415cb70 2024-04-25T10:17:11.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.401+0000 7fb9b27fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb9b4070990 0x7fb9b415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.401+0000 7fb9b2ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb9b40afc10 0x7fb9b4140f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:48038/0 (socket says 172.21.15.60:48038) 2024-04-25T10:17:11.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.401+0000 7fb9b2ffd640 1 -- 172.21.15.60:0/3661387496 learned_addr learned my addr 172.21.15.60:0/3661387496 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:11.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.401+0000 7fb9b27fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb9b4070990 0x7fb9b415c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:32848/0 (socket says 172.21.15.60:32848) 2024-04-25T10:17:11.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.401+0000 7fb9b37fe640 1 --2- 172.21.15.60:0/3661387496 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9b415cb70 0x7fb9b406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.401+0000 7fb9b27fc640 1 -- 172.21.15.60:0/3661387496 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9b415cb70 msgr2=0x7fb9b406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.401+0000 7fb9b27fc640 1 --2- 172.21.15.60:0/3661387496 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9b415cb70 0x7fb9b406ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.401+0000 7fb9b27fc640 1 -- 172.21.15.60:0/3661387496 >> v1:172.21.15.100:6789/0 conn(0x7fb9b40afc10 legacy=0x7fb9b4140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.401+0000 7fb9b27fc640 1 -- 172.21.15.60:0/3661387496 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b4141730 con 0x7fb9b4070990 2024-04-25T10:17:11.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.401+0000 7fb9b37fe640 1 --2- 172.21.15.60:0/3661387496 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9b415cb70 0x7fb9b406ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:11.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.401+0000 7fb9b27fc640 1 --2- 172.21.15.60:0/3661387496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b4070990 0x7fb9b415c630 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80097c0 tx=0x7fb9a801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fb48472e9f78131 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b1ffb640 1 -- 172.21.15.60:0/3661387496 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a80b2070 con 0x7fb9b4070990 2024-04-25T10:17:11.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b1ffb640 1 -- 172.21.15.60:0/3661387496 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a80ad680 con 0x7fb9b4070990 2024-04-25T10:17:11.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b1ffb640 1 -- 172.21.15.60:0/3661387496 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a80ada50 con 0x7fb9b4070990 2024-04-25T10:17:11.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b95f3640 1 -- 172.21.15.60:0/3661387496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b4070990 msgr2=0x7fb9b415c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b95f3640 1 --2- 172.21.15.60:0/3661387496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b4070990 0x7fb9b415c630 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80097c0 tx=0x7fb9a801c470 comp rx=0 tx=0).stop 2024-04-25T10:17:11.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b95f3640 1 -- 172.21.15.60:0/3661387496 shutdown_connections 2024-04-25T10:17:11.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b95f3640 1 --2- 172.21.15.60:0/3661387496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b4070990 0x7fb9b415c630 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b95f3640 1 --2- 172.21.15.60:0/3661387496 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9b415cb70 0x7fb9b406ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b95f3640 1 -- 172.21.15.60:0/3661387496 >> 172.21.15.60:0/3661387496 conn(0x7fb9b406e400 msgr2=0x7fb9b406e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:11.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b95f3640 1 -- 172.21.15.60:0/3661387496 shutdown_connections 2024-04-25T10:17:11.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b95f3640 1 -- 172.21.15.60:0/3661387496 wait complete. 2024-04-25T10:17:11.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b95f3640 1 Processor -- start 2024-04-25T10:17:11.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b95f3640 1 -- start start 2024-04-25T10:17:11.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b95f3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b4070990 0x7fb9b415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b95f3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b40afc10 0x7fb9b4136460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b95f3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9b4155bc0 0x7fb9b41393e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b95f3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b40ce770 con 0x7fb9b4070990 2024-04-25T10:17:11.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b95f3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b40ce5f0 con 0x7fb9b40afc10 2024-04-25T10:17:11.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b95f3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b40ce8f0 con 0x7fb9b4155bc0 2024-04-25T10:17:11.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b4070990 0x7fb9b415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b4070990 0x7fb9b415dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46382/0 (socket says 172.21.15.60:46382) 2024-04-25T10:17:11.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b2ffd640 1 -- 172.21.15.60:0/3738936695 learned_addr learned my addr 172.21.15.60:0/3738936695 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:11.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b27fc640 1 --2- 172.21.15.60:0/3738936695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b40afc10 0x7fb9b4136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b37fe640 1 --2- 172.21.15.60:0/3738936695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9b4155bc0 0x7fb9b41393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b2ffd640 1 -- 172.21.15.60:0/3738936695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9b4155bc0 msgr2=0x7fb9b41393e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b2ffd640 1 --2- 172.21.15.60:0/3738936695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9b4155bc0 0x7fb9b41393e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b2ffd640 1 -- 172.21.15.60:0/3738936695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b40afc10 msgr2=0x7fb9b4136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b2ffd640 1 --2- 172.21.15.60:0/3738936695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9b40afc10 0x7fb9b4136460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b2ffd640 1 -- 172.21.15.60:0/3738936695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b41353c0 con 0x7fb9b4070990 2024-04-25T10:17:11.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b37fe640 1 --2- 172.21.15.60:0/3738936695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9b4155bc0 0x7fb9b41393e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:11.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.405+0000 7fb9b2ffd640 1 --2- 172.21.15.60:0/3738936695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b4070990 0x7fb9b415dba0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400c420 tx=0x7fb9a400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb99b7fe640 1 -- 172.21.15.60:0/3738936695 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4017070 con 0x7fb9b4070990 2024-04-25T10:17:11.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb99b7fe640 1 -- 172.21.15.60:0/3738936695 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a4005cf0 con 0x7fb9b4070990 2024-04-25T10:17:11.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb9b95f3640 1 -- 172.21.15.60:0/3738936695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b41356a0 con 0x7fb9b4070990 2024-04-25T10:17:11.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb99b7fe640 1 -- 172.21.15.60:0/3738936695 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a40072f0 con 0x7fb9b4070990 2024-04-25T10:17:11.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb9b95f3640 1 -- 172.21.15.60:0/3738936695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b4135c60 con 0x7fb9b4070990 2024-04-25T10:17:11.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb9b95f3640 1 -- 172.21.15.60:0/3738936695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb9b4074770 con 0x7fb9b4070990 2024-04-25T10:17:11.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb99b7fe640 1 -- 172.21.15.60:0/3738936695 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9a401a070 con 0x7fb9b4070990 2024-04-25T10:17:11.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb99b7fe640 1 --2- 172.21.15.60:0/3738936695 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9a0066f60 0x7fb9a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb99b7fe640 1 -- 172.21.15.60:0/3738936695 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fb9a4084100 con 0x7fb9b4070990 2024-04-25T10:17:11.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb99b7fe640 1 --2- 172.21.15.60:0/3738936695 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x7fb9a006cf60 0x7fb9a006f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb99b7fe640 1 -- 172.21.15.60:0/3738936695 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9a4016ec0 con 0x7fb9a006cf60 2024-04-25T10:17:11.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb99b7fe640 1 -- 172.21.15.60:0/3738936695 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9a408b050 con 0x7fb9b4070990 2024-04-25T10:17:11.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb9b27fc640 1 --2- 172.21.15.60:0/3738936695 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9a0066f60 0x7fb9a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb9b37fe640 1 --2- 172.21.15.60:0/3738936695 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x7fb9a006cf60 0x7fb9a006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb9b37fe640 1 --2- 172.21.15.60:0/3738936695 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x7fb9a006cf60 0x7fb9a006f380 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb9b27fc640 1 --2- 172.21.15.60:0/3738936695 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9a0066f60 0x7fb9a0069420 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80af000 tx=0x7fb9a80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.409+0000 7fb99b7fe640 1 -- 172.21.15.60:0/3738936695 <== osd.0 v2:172.21.15.60:6801/4056167511 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb9a4016ec0 con 0x7fb9a006cf60 2024-04-25T10:17:11.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.425+0000 7fb9b95f3640 1 -- 172.21.15.60:0/3738936695 --> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb9b40ce1c0 con 0x7fb9a006cf60 2024-04-25T10:17:11.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.425+0000 7fb99b7fe640 1 -- 172.21.15.60:0/3738936695 <== osd.0 v2:172.21.15.60:6801/4056167511 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb9b40ce1c0 con 0x7fb9a006cf60 2024-04-25T10:17:11.430 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:17:11.431 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:17:11.431 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:17:11.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.425+0000 7fb9b95f3640 1 -- 172.21.15.60:0/3738936695 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x7fb9a006cf60 msgr2=0x7fb9a006f380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.425+0000 7fb9b95f3640 1 --2- 172.21.15.60:0/3738936695 >> [v2:172.21.15.60:6801/4056167511,v1:172.21.15.60:6803/4056167511] conn(0x7fb9a006cf60 0x7fb9a006f380 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.425+0000 7fb9b95f3640 1 -- 172.21.15.60:0/3738936695 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9a0066f60 msgr2=0x7fb9a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.425+0000 7fb9b95f3640 1 --2- 172.21.15.60:0/3738936695 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb9a0066f60 0x7fb9a0069420 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80af000 tx=0x7fb9a80a7000 comp rx=0 tx=0).stop 2024-04-25T10:17:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.425+0000 7fb9b95f3640 1 -- 172.21.15.60:0/3738936695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b4070990 msgr2=0x7fb9b415dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.425+0000 7fb9b95f3640 1 --2- 172.21.15.60:0/3738936695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9b4070990 0x7fb9b415dba0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400c420 tx=0x7fb9a400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.425+0000 7fb9b37fe640 1 -- 172.21.15.60:0/3738936695 reap_dead start 2024-04-25T10:17:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.425+0000 7fb9b95f3640 1 -- 172.21.15.60:0/3738936695 shutdown_connections 2024-04-25T10:17:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.425+0000 7fb9b95f3640 1 -- 172.21.15.60:0/3738936695 >> 172.21.15.60:0/3738936695 conn(0x7fb9b406e400 msgr2=0x7fb9b40b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.425+0000 7fb9b95f3640 1 -- 172.21.15.60:0/3738936695 shutdown_connections 2024-04-25T10:17:11.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.425+0000 7fb9b95f3640 1 -- 172.21.15.60:0/3738936695 wait complete. 2024-04-25T10:17:11.447 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T10:17:11.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f1673b49640 1 Processor -- start 2024-04-25T10:17:11.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f1673b49640 1 -- start start 2024-04-25T10:17:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f1673b49640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f166c15bc90 0x7f166c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f1673b49640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f166c0b4660 0x7f166c15c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f1673b49640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f166c071ca0 con 0x7f166c0ce7a0 2024-04-25T10:17:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f1673b49640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166c0cc370 con 0x7f166c0b4660 2024-04-25T10:17:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f1673b49640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c0cca40 con 0x7f166c15bc90 2024-04-25T10:17:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16718e5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f166c0ce7a0 0x7f166c0ceb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40840/0 (socket says 172.21.15.60:40840) 2024-04-25T10:17:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16718e5640 1 -- 172.21.15.60:0/1397895825 learned_addr learned my addr 172.21.15.60:0/1397895825 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16710e4640 1 --2- 172.21.15.60:0/1397895825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f166c15bc90 0x7f166c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16708e3640 1 -- 172.21.15.60:0/1397895825 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637236938 0 0) 0x7f166c071ca0 con 0x7f166c0ce7a0 2024-04-25T10:17:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16708e3640 1 -- 172.21.15.60:0/1397895825 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1658003530 con 0x7f166c0ce7a0 2024-04-25T10:17:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16720e6640 1 --2- 172.21.15.60:0/1397895825 >> v2:172.21.15.100:3300/0 conn(0x7f166c0b4660 0x7f166c15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16708e3640 1 -- 172.21.15.60:0/1397895825 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 182710876 0 0) 0x7f1658003530 con 0x7f166c0ce7a0 2024-04-25T10:17:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16708e3640 1 -- 172.21.15.60:0/1397895825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f166c15bc90 msgr2=0x7f166c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16708e3640 1 --2- 172.21.15.60:0/1397895825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f166c15bc90 0x7f166c15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16708e3640 1 -- 172.21.15.60:0/1397895825 >> v2:172.21.15.100:3300/0 conn(0x7f166c0b4660 msgr2=0x7f166c15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16708e3640 1 --2- 172.21.15.60:0/1397895825 >> v2:172.21.15.100:3300/0 conn(0x7f166c0b4660 0x7f166c15c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16708e3640 1 -- 172.21.15.60:0/1397895825 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c0ccd20 con 0x7f166c0ce7a0 2024-04-25T10:17:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16708e3640 1 -- 172.21.15.60:0/1397895825 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f1668001680 con 0x7f166c0ce7a0 2024-04-25T10:17:11.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16708e3640 1 -- 172.21.15.60:0/1397895825 >> v1:172.21.15.60:6789/0 conn(0x7f166c0ce7a0 legacy=0x7f166c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16708e3640 1 --2- 172.21.15.60:0/1397895825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16580039c0 0x7f1658003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16708e3640 1 --2- 172.21.15.60:0/1397895825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f166c15bc90 0x7f1658004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16708e3640 1 --2- 172.21.15.60:0/1397895825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1658004d50 0x7f16580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16708e3640 1 -- 172.21.15.60:0/1397895825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c0cca40 con 0x7f16580039c0 2024-04-25T10:17:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16708e3640 1 -- 172.21.15.60:0/1397895825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165800b9d0 con 0x7f1658004d50 2024-04-25T10:17:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16708e3640 1 -- 172.21.15.60:0/1397895825 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165800bb50 con 0x7f166c15bc90 2024-04-25T10:17:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16718e5640 1 --2- 172.21.15.60:0/1397895825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16580039c0 0x7f1658003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16710e4640 1 --2- 172.21.15.60:0/1397895825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f166c15bc90 0x7f1658004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16720e6640 1 --2- 172.21.15.60:0/1397895825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1658004d50 0x7f16580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16718e5640 1 -- 172.21.15.60:0/1397895825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f166c15bc90 msgr2=0x7f1658004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16718e5640 1 --2- 172.21.15.60:0/1397895825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f166c15bc90 0x7f1658004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16718e5640 1 -- 172.21.15.60:0/1397895825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1658004d50 msgr2=0x7f16580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16718e5640 1 --2- 172.21.15.60:0/1397895825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1658004d50 0x7f16580051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16718e5640 1 -- 172.21.15.60:0/1397895825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1658003530 con 0x7f16580039c0 2024-04-25T10:17:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.557+0000 7f16710e4640 1 --2- 172.21.15.60:0/1397895825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f166c15bc90 0x7f1658004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.561+0000 7f16718e5640 1 --2- 172.21.15.60:0/1397895825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16580039c0 0x7f1658003de0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f166800c190 tx=0x7f166801cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540258467bdaf4a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.561+0000 7f16708e3640 1 -- 172.21.15.60:0/1397895825 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668007c90 con 0x7f16580039c0 2024-04-25T10:17:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.561+0000 7f16708e3640 1 -- 172.21.15.60:0/1397895825 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16680a9020 con 0x7f16580039c0 2024-04-25T10:17:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.561+0000 7f16708e3640 1 -- 172.21.15.60:0/1397895825 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668005b00 con 0x7f16580039c0 2024-04-25T10:17:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.561+0000 7f1673b49640 1 -- 172.21.15.60:0/1397895825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16580039c0 msgr2=0x7f1658003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.561+0000 7f1673b49640 1 --2- 172.21.15.60:0/1397895825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16580039c0 0x7f1658003de0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f166800c190 tx=0x7f166801cbc0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.561+0000 7f16720e6640 1 -- 172.21.15.60:0/1397895825 reap_dead start 2024-04-25T10:17:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.561+0000 7f1673b49640 1 -- 172.21.15.60:0/1397895825 shutdown_connections 2024-04-25T10:17:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.561+0000 7f1673b49640 1 -- 172.21.15.60:0/1397895825 >> 172.21.15.60:0/1397895825 conn(0x7f166c0b12d0 msgr2=0x7f166c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.561+0000 7f1673b49640 1 -- 172.21.15.60:0/1397895825 shutdown_connections 2024-04-25T10:17:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.561+0000 7f1673b49640 1 -- 172.21.15.60:0/1397895825 wait complete. 2024-04-25T10:17:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.561+0000 7f1673b49640 1 Processor -- start 2024-04-25T10:17:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.561+0000 7f1673b49640 1 -- start start 2024-04-25T10:17:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f1673b49640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16580039c0 0x7f166c155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f1673b49640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f166c155e00 0x7f166c06e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f1673b49640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c06d2f0 0x7f166c1645a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f1673b49640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c0b3db0 con 0x7f166c155e00 2024-04-25T10:17:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f1673b49640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c0b4490 con 0x7f166c06d2f0 2024-04-25T10:17:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f1673b49640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c1665b0 con 0x7f16580039c0 2024-04-25T10:17:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f16710e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f166c155e00 0x7f166c06e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f16710e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f166c155e00 0x7f166c06e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46404/0 (socket says 172.21.15.60:46404) 2024-04-25T10:17:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f16710e4640 1 -- 172.21.15.60:0/1001135775 learned_addr learned my addr 172.21.15.60:0/1001135775 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f16718e5640 1 --2- 172.21.15.60:0/1001135775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16580039c0 0x7f166c155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f16720e6640 1 --2- 172.21.15.60:0/1001135775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c06d2f0 0x7f166c1645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f16710e4640 1 -- 172.21.15.60:0/1001135775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16580039c0 msgr2=0x7f166c155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f16710e4640 1 --2- 172.21.15.60:0/1001135775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16580039c0 0x7f166c155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f16710e4640 1 -- 172.21.15.60:0/1001135775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c06d2f0 msgr2=0x7f166c1645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f16710e4640 1 --2- 172.21.15.60:0/1001135775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c06d2f0 0x7f166c1645a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f16710e4640 1 -- 172.21.15.60:0/1001135775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c06d8b0 con 0x7f166c155e00 2024-04-25T10:17:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f16720e6640 1 --2- 172.21.15.60:0/1001135775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c06d2f0 0x7f166c1645a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f16710e4640 1 --2- 172.21.15.60:0/1001135775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f166c155e00 0x7f166c06e950 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f1660011650 tx=0x7f1660011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f165e7fc640 1 -- 172.21.15.60:0/1001135775 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660004220 con 0x7f166c155e00 2024-04-25T10:17:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f165e7fc640 1 -- 172.21.15.60:0/1001135775 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16600043c0 con 0x7f166c155e00 2024-04-25T10:17:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f1673b49640 1 -- 172.21.15.60:0/1001135775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166c06db90 con 0x7f166c155e00 2024-04-25T10:17:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f165e7fc640 1 -- 172.21.15.60:0/1001135775 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166000f960 con 0x7f166c155e00 2024-04-25T10:17:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f1673b49640 1 -- 172.21.15.60:0/1001135775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166c06e150 con 0x7f166c155e00 2024-04-25T10:17:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f1673b49640 1 -- 172.21.15.60:0/1001135775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f166c074770 con 0x7f166c155e00 2024-04-25T10:17:11.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f165e7fc640 1 -- 172.21.15.60:0/1001135775 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f166000fb00 con 0x7f166c155e00 2024-04-25T10:17:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f165e7fc640 1 --2- 172.21.15.60:0/1001135775 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1650066f60 0x7f1650069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f165e7fc640 1 -- 172.21.15.60:0/1001135775 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f1660085170 con 0x7f166c155e00 2024-04-25T10:17:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.565+0000 7f16718e5640 1 --2- 172.21.15.60:0/1001135775 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1650066f60 0x7f1650069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.569+0000 7f16718e5640 1 --2- 172.21.15.60:0/1001135775 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1650066f60 0x7f1650069420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f16680aa000 tx=0x7f16680b0040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.569+0000 7f165e7fc640 1 -- 172.21.15.60:0/1001135775 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1660019030 con 0x7f166c155e00 2024-04-25T10:17:11.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.729+0000 7f1673b49640 1 -- 172.21.15.60:0/1001135775 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f166c155b50 con 0x7f1650066f60 2024-04-25T10:17:11.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.729+0000 7f165e7fc640 1 -- 172.21.15.60:0/1001135775 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f166c155b50 con 0x7f1650066f60 2024-04-25T10:17:11.735 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T10:17:11.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.733+0000 7f1673b49640 1 -- 172.21.15.60:0/1001135775 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1650066f60 msgr2=0x7f1650069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.733+0000 7f1673b49640 1 --2- 172.21.15.60:0/1001135775 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1650066f60 0x7f1650069420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f16680aa000 tx=0x7f16680b0040 comp rx=0 tx=0).stop 2024-04-25T10:17:11.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.733+0000 7f1673b49640 1 -- 172.21.15.60:0/1001135775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f166c155e00 msgr2=0x7f166c06e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.733+0000 7f1673b49640 1 --2- 172.21.15.60:0/1001135775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f166c155e00 0x7f166c06e950 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f1660011650 tx=0x7f1660011b10 comp rx=0 tx=0).stop 2024-04-25T10:17:11.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.733+0000 7f1673b49640 1 -- 172.21.15.60:0/1001135775 shutdown_connections 2024-04-25T10:17:11.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.733+0000 7f1673b49640 1 --2- 172.21.15.60:0/1001135775 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1650066f60 0x7f1650069420 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.733+0000 7f1673b49640 1 --2- 172.21.15.60:0/1001135775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f166c06d2f0 0x7f166c1645a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.733+0000 7f1673b49640 1 --2- 172.21.15.60:0/1001135775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f166c155e00 0x7f166c06e950 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.733+0000 7f1673b49640 1 --2- 172.21.15.60:0/1001135775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16580039c0 0x7f166c155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.733+0000 7f1673b49640 1 -- 172.21.15.60:0/1001135775 >> 172.21.15.60:0/1001135775 conn(0x7f166c0b12d0 msgr2=0x7f166c165ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:11.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.733+0000 7f1673b49640 1 -- 172.21.15.60:0/1001135775 shutdown_connections 2024-04-25T10:17:11.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.733+0000 7f1673b49640 1 -- 172.21.15.60:0/1001135775 wait complete. 2024-04-25T10:17:11.754 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T10:17:11.754 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:17:11.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.869+0000 7fb69a8d3640 1 Processor -- start 2024-04-25T10:17:11.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.869+0000 7fb69a8d3640 1 -- start start 2024-04-25T10:17:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.869+0000 7fb69a8d3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb69415da40 0x7fb6940b95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.869+0000 7fb69a8d3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb69415de20 0x7fb6940ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.869+0000 7fb69a8d3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6940d1a70 con 0x7fb69415e2b0 2024-04-25T10:17:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.869+0000 7fb69a8d3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69415e7f0 con 0x7fb69415de20 2024-04-25T10:17:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.869+0000 7fb69a8d3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69415ead0 con 0x7fb69415da40 2024-04-25T10:17:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.869+0000 7fb6937fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb69415de20 0x7fb6940ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.869+0000 7fb698e70640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb69415e2b0 0x7fb694072340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40854/0 (socket says 172.21.15.60:40854) 2024-04-25T10:17:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.869+0000 7fb698e70640 1 -- 172.21.15.60:0/361866292 learned_addr learned my addr 172.21.15.60:0/361866292 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.869+0000 7fb693fff640 1 --2- 172.21.15.60:0/361866292 >> v2:172.21.15.100:3300/0 conn(0x7fb69415da40 0x7fb6940b95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.869+0000 7fb6937fe640 1 -- 172.21.15.60:0/361866292 >> v2:172.21.15.100:3300/0 conn(0x7fb69415da40 msgr2=0x7fb6940b95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.869+0000 7fb6937fe640 1 --2- 172.21.15.60:0/361866292 >> v2:172.21.15.100:3300/0 conn(0x7fb69415da40 0x7fb6940b95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.869+0000 7fb6937fe640 1 -- 172.21.15.60:0/361866292 >> v1:172.21.15.60:6789/0 conn(0x7fb69415e2b0 legacy=0x7fb694072340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.869+0000 7fb6937fe640 1 -- 172.21.15.60:0/361866292 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69415edb0 con 0x7fb69415de20 2024-04-25T10:17:11.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb6937fe640 1 --2- 172.21.15.60:0/361866292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69415de20 0x7fb6940ce6c0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fb694068880 tx=0x7fb68801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4cd3d95e9fe4d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb692ffd640 1 -- 172.21.15.60:0/361866292 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6880a7070 con 0x7fb69415de20 2024-04-25T10:17:11.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb692ffd640 1 -- 172.21.15.60:0/361866292 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6880b0070 con 0x7fb69415de20 2024-04-25T10:17:11.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb692ffd640 1 -- 172.21.15.60:0/361866292 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6880ab890 con 0x7fb69415de20 2024-04-25T10:17:11.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb69a8d3640 1 -- 172.21.15.60:0/361866292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69415de20 msgr2=0x7fb6940ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb69a8d3640 1 --2- 172.21.15.60:0/361866292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69415de20 0x7fb6940ce6c0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fb694068880 tx=0x7fb68801c450 comp rx=0 tx=0).stop 2024-04-25T10:17:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb69a8d3640 1 -- 172.21.15.60:0/361866292 shutdown_connections 2024-04-25T10:17:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb69a8d3640 1 --2- 172.21.15.60:0/361866292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69415de20 0x7fb6940ce6c0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb69a8d3640 1 --2- 172.21.15.60:0/361866292 >> v2:172.21.15.100:3300/0 conn(0x7fb69415da40 0x7fb6940b95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb69a8d3640 1 -- 172.21.15.60:0/361866292 >> 172.21.15.60:0/361866292 conn(0x7fb6940d0470 msgr2=0x7fb6940d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb69a8d3640 1 -- 172.21.15.60:0/361866292 shutdown_connections 2024-04-25T10:17:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb69a8d3640 1 -- 172.21.15.60:0/361866292 wait complete. 2024-04-25T10:17:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb69a8d3640 1 Processor -- start 2024-04-25T10:17:11.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb69a8d3640 1 -- start start 2024-04-25T10:17:11.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb69a8d3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69415da40 0x7fb6940c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb69a8d3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb69415de20 0x7fb6941533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb69a8d3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb69415e2b0 0x7fb694153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb69a8d3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69416a900 con 0x7fb69415da40 2024-04-25T10:17:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb69a8d3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69416a780 con 0x7fb69415e2b0 2024-04-25T10:17:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb69a8d3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69416aa80 con 0x7fb69415de20 2024-04-25T10:17:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb693fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69415da40 0x7fb6940c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb693fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69415da40 0x7fb6940c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46432/0 (socket says 172.21.15.60:46432) 2024-04-25T10:17:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb693fff640 1 -- 172.21.15.60:0/2228205771 learned_addr learned my addr 172.21.15.60:0/2228205771 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb693fff640 1 -- 172.21.15.60:0/2228205771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb69415de20 msgr2=0x7fb6941533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb693fff640 1 --2- 172.21.15.60:0/2228205771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb69415de20 0x7fb6941533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb693fff640 1 -- 172.21.15.60:0/2228205771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb69415e2b0 msgr2=0x7fb694153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb693fff640 1 --2- 172.21.15.60:0/2228205771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb69415e2b0 0x7fb694153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb693fff640 1 -- 172.21.15.60:0/2228205771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6940b8760 con 0x7fb69415da40 2024-04-25T10:17:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.873+0000 7fb693fff640 1 --2- 172.21.15.60:0/2228205771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69415da40 0x7fb6940c7d90 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fb68400c7f0 tx=0x7fb68400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb690ff9640 1 -- 172.21.15.60:0/2228205771 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684015070 con 0x7fb69415da40 2024-04-25T10:17:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb690ff9640 1 -- 172.21.15.60:0/2228205771 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb684004240 con 0x7fb69415da40 2024-04-25T10:17:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb69a8d3640 1 -- 172.21.15.60:0/2228205771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6940c7460 con 0x7fb69415da40 2024-04-25T10:17:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb690ff9640 1 -- 172.21.15.60:0/2228205771 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684003620 con 0x7fb69415da40 2024-04-25T10:17:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb69a8d3640 1 -- 172.21.15.60:0/2228205771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb694152a90 con 0x7fb69415da40 2024-04-25T10:17:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb69a8d3640 1 -- 172.21.15.60:0/2228205771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb694074770 con 0x7fb69415da40 2024-04-25T10:17:11.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb690ff9640 1 -- 172.21.15.60:0/2228205771 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6840039d0 con 0x7fb69415da40 2024-04-25T10:17:11.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb690ff9640 1 --2- 172.21.15.60:0/2228205771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb680066f60 0x7fb680069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb690ff9640 1 -- 172.21.15.60:0/2228205771 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fb6840853e0 con 0x7fb69415da40 2024-04-25T10:17:11.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb690ff9640 1 --2- 172.21.15.60:0/2228205771 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x7fb68006cf60 0x7fb68006f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb690ff9640 1 -- 172.21.15.60:0/2228205771 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb68006fa30 con 0x7fb68006cf60 2024-04-25T10:17:11.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb690ff9640 1 -- 172.21.15.60:0/2228205771 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb684085870 con 0x7fb69415da40 2024-04-25T10:17:11.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb698e70640 1 --2- 172.21.15.60:0/2228205771 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x7fb68006cf60 0x7fb68006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb6937fe640 1 --2- 172.21.15.60:0/2228205771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb680066f60 0x7fb680069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb698e70640 1 --2- 172.21.15.60:0/2228205771 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x7fb68006cf60 0x7fb68006f380 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb6937fe640 1 --2- 172.21.15.60:0/2228205771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb680066f60 0x7fb680069420 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb6880af000 tx=0x7fb688004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.877+0000 7fb690ff9640 1 -- 172.21.15.60:0/2228205771 <== osd.1 v2:172.21.15.60:6824/3313552458 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb68006fa30 con 0x7fb68006cf60 2024-04-25T10:17:11.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.889+0000 7fb69a8d3640 1 -- 172.21.15.60:0/2228205771 --> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb6940c6c20 con 0x7fb68006cf60 2024-04-25T10:17:11.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.893+0000 7fb690ff9640 1 -- 172.21.15.60:0/2228205771 <== osd.1 v2:172.21.15.60:6824/3313552458 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb6940c6c20 con 0x7fb68006cf60 2024-04-25T10:17:11.897 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:17:11.897 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:17:11.897 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:17:11.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.893+0000 7fb69a8d3640 1 -- 172.21.15.60:0/2228205771 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x7fb68006cf60 msgr2=0x7fb68006f380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.893+0000 7fb69a8d3640 1 --2- 172.21.15.60:0/2228205771 >> [v2:172.21.15.60:6824/3313552458,v1:172.21.15.60:6825/3313552458] conn(0x7fb68006cf60 0x7fb68006f380 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.893+0000 7fb69a8d3640 1 -- 172.21.15.60:0/2228205771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb680066f60 msgr2=0x7fb680069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.893+0000 7fb69a8d3640 1 --2- 172.21.15.60:0/2228205771 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fb680066f60 0x7fb680069420 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb6880af000 tx=0x7fb688004740 comp rx=0 tx=0).stop 2024-04-25T10:17:11.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.893+0000 7fb69a8d3640 1 -- 172.21.15.60:0/2228205771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69415da40 msgr2=0x7fb6940c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.893+0000 7fb69a8d3640 1 --2- 172.21.15.60:0/2228205771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69415da40 0x7fb6940c7d90 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fb68400c7f0 tx=0x7fb68400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.893+0000 7fb698e70640 1 -- 172.21.15.60:0/2228205771 reap_dead start 2024-04-25T10:17:11.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.893+0000 7fb69a8d3640 1 -- 172.21.15.60:0/2228205771 shutdown_connections 2024-04-25T10:17:11.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.893+0000 7fb69a8d3640 1 -- 172.21.15.60:0/2228205771 >> 172.21.15.60:0/2228205771 conn(0x7fb6940d0470 msgr2=0x7fb69415f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:11.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.893+0000 7fb69a8d3640 1 -- 172.21.15.60:0/2228205771 shutdown_connections 2024-04-25T10:17:11.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:11.893+0000 7fb69a8d3640 1 -- 172.21.15.60:0/2228205771 wait complete. 2024-04-25T10:17:11.911 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T10:17:12.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.021+0000 7f3c264dd640 1 Processor -- start 2024-04-25T10:17:12.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.021+0000 7f3c264dd640 1 -- start start 2024-04-25T10:17:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.021+0000 7f3c264dd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3c200ce7a0 0x7f3c200ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.021+0000 7f3c264dd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c2015bc90 0x7f3c200b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.021+0000 7f3c264dd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c200cf0c0 con 0x7f3c200b4660 2024-04-25T10:17:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.021+0000 7f3c264dd640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c200cc370 con 0x7f3c200ce7a0 2024-04-25T10:17:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.021+0000 7f3c264dd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c20071ca0 con 0x7f3c2015bc90 2024-04-25T10:17:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.021+0000 7f3c1ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3c200ce7a0 0x7f3c200ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.021+0000 7f3c1ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3c200ce7a0 0x7f3c200ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46448/0 (socket says 172.21.15.60:46448) 2024-04-25T10:17:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.021+0000 7f3c1ffff640 1 -- 172.21.15.60:0/922045865 learned_addr learned my addr 172.21.15.60:0/922045865 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.021+0000 7f3c1f7fe640 1 --2- 172.21.15.60:0/922045865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c2015bc90 0x7f3c200b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.021+0000 7f3c1ffff640 1 -- 172.21.15.60:0/922045865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c2015bc90 msgr2=0x7f3c200b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.021+0000 7f3c1ffff640 1 --2- 172.21.15.60:0/922045865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c2015bc90 0x7f3c200b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.021+0000 7f3c1ffff640 1 -- 172.21.15.60:0/922045865 >> v1:172.21.15.100:6789/0 conn(0x7f3c200b4660 legacy=0x7f3c200b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.021+0000 7f3c1ffff640 1 -- 172.21.15.60:0/922045865 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c200cca40 con 0x7f3c200ce7a0 2024-04-25T10:17:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.025+0000 7f3c1ffff640 1 --2- 172.21.15.60:0/922045865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c200ce7a0 0x7f3c200ceb80 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f3c100098a0 tx=0x7f3c1001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6d5d5b9a6473f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.025+0000 7f3c1effd640 1 -- 172.21.15.60:0/922045865 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c100b0070 con 0x7f3c200ce7a0 2024-04-25T10:17:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.025+0000 7f3c1effd640 1 -- 172.21.15.60:0/922045865 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c100ab600 con 0x7f3c200ce7a0 2024-04-25T10:17:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.025+0000 7f3c1effd640 1 -- 172.21.15.60:0/922045865 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c100ab9d0 con 0x7f3c200ce7a0 2024-04-25T10:17:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.025+0000 7f3c264dd640 1 -- 172.21.15.60:0/922045865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c200ce7a0 msgr2=0x7f3c200ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.025+0000 7f3c264dd640 1 --2- 172.21.15.60:0/922045865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c200ce7a0 0x7f3c200ceb80 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f3c100098a0 tx=0x7f3c1001c500 comp rx=0 tx=0).stop 2024-04-25T10:17:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.025+0000 7f3c264dd640 1 -- 172.21.15.60:0/922045865 shutdown_connections 2024-04-25T10:17:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.025+0000 7f3c264dd640 1 --2- 172.21.15.60:0/922045865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c2015bc90 0x7f3c200b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.025+0000 7f3c264dd640 1 --2- 172.21.15.60:0/922045865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c200ce7a0 0x7f3c200ceb80 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.025+0000 7f3c264dd640 1 -- 172.21.15.60:0/922045865 >> 172.21.15.60:0/922045865 conn(0x7f3c200b12d0 msgr2=0x7f3c200b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.025+0000 7f3c264dd640 1 -- 172.21.15.60:0/922045865 shutdown_connections 2024-04-25T10:17:12.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.025+0000 7f3c264dd640 1 -- 172.21.15.60:0/922045865 wait complete. 2024-04-25T10:17:12.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.025+0000 7f3c264dd640 1 Processor -- start 2024-04-25T10:17:12.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.025+0000 7f3c264dd640 1 -- start start 2024-04-25T10:17:12.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.025+0000 7f3c264dd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c200ce7a0 0x7f3c2015f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c264dd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c2015bc90 0x7f3c2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c264dd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c200acb00 0x7f3c2015a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c264dd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200b4280 con 0x7f3c200ce7a0 2024-04-25T10:17:12.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c264dd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200b3db0 con 0x7f3c200acb00 2024-04-25T10:17:12.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c264dd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200b4400 con 0x7f3c2015bc90 2024-04-25T10:17:12.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c200ce7a0 0x7f3c2015f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c200ce7a0 0x7f3c2015f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46464/0 (socket says 172.21.15.60:46464) 2024-04-25T10:17:12.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1ffff640 1 -- 172.21.15.60:0/2986532294 learned_addr learned my addr 172.21.15.60:0/2986532294 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:12.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c24a7a640 1 --2- 172.21.15.60:0/2986532294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c200acb00 0x7f3c2015a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1f7fe640 1 --2- 172.21.15.60:0/2986532294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c2015bc90 0x7f3c2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1ffff640 1 -- 172.21.15.60:0/2986532294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c2015bc90 msgr2=0x7f3c2006edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1ffff640 1 --2- 172.21.15.60:0/2986532294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c2015bc90 0x7f3c2006edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1ffff640 1 -- 172.21.15.60:0/2986532294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c200acb00 msgr2=0x7f3c2015a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1ffff640 1 --2- 172.21.15.60:0/2986532294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c200acb00 0x7f3c2015a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1ffff640 1 -- 172.21.15.60:0/2986532294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2006f2f0 con 0x7f3c200ce7a0 2024-04-25T10:17:12.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1f7fe640 1 --2- 172.21.15.60:0/2986532294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c2015bc90 0x7f3c2006edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:12.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1ffff640 1 --2- 172.21.15.60:0/2986532294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c200ce7a0 0x7f3c2015f570 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f3c100ae040 tx=0x7f3c1001ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1cff9640 1 -- 172.21.15.60:0/2986532294 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c100b0040 con 0x7f3c200ce7a0 2024-04-25T10:17:12.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1cff9640 1 -- 172.21.15.60:0/2986532294 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c100bc070 con 0x7f3c200ce7a0 2024-04-25T10:17:12.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c264dd640 1 -- 172.21.15.60:0/2986532294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2006dda0 con 0x7f3c200ce7a0 2024-04-25T10:17:12.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1cff9640 1 -- 172.21.15.60:0/2986532294 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c100b6640 con 0x7f3c200ce7a0 2024-04-25T10:17:12.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c264dd640 1 -- 172.21.15.60:0/2986532294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2006e320 con 0x7f3c200ce7a0 2024-04-25T10:17:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c264dd640 1 -- 172.21.15.60:0/2986532294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c20074770 con 0x7f3c200ce7a0 2024-04-25T10:17:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1cff9640 1 -- 172.21.15.60:0/2986532294 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c100b67e0 con 0x7f3c200ce7a0 2024-04-25T10:17:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1cff9640 1 --2- 172.21.15.60:0/2986532294 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c0c066f60 0x7f3c0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1cff9640 1 -- 172.21.15.60:0/2986532294 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f3c1011f100 con 0x7f3c200ce7a0 2024-04-25T10:17:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.029+0000 7f3c1f7fe640 1 --2- 172.21.15.60:0/2986532294 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c0c066f60 0x7f3c0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.033+0000 7f3c1f7fe640 1 --2- 172.21.15.60:0/2986532294 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c0c066f60 0x7f3c0c069420 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3c20154ef0 tx=0x7f3c14008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.033+0000 7f3c1cff9640 1 -- 172.21.15.60:0/2986532294 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c2006f2f0 con 0x7f3c200ce7a0 2024-04-25T10:17:12.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.193+0000 7f3c264dd640 1 -- 172.21.15.60:0/2986532294 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c200cbfc0 con 0x7f3c0c066f60 2024-04-25T10:17:12.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.197+0000 7f3c1cff9640 1 -- 172.21.15.60:0/2986532294 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3c200cbfc0 con 0x7f3c0c066f60 2024-04-25T10:17:12.201 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T10:17:12.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.197+0000 7f3c264dd640 1 -- 172.21.15.60:0/2986532294 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c0c066f60 msgr2=0x7f3c0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.197+0000 7f3c264dd640 1 --2- 172.21.15.60:0/2986532294 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c0c066f60 0x7f3c0c069420 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3c20154ef0 tx=0x7f3c14008040 comp rx=0 tx=0).stop 2024-04-25T10:17:12.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.197+0000 7f3c264dd640 1 -- 172.21.15.60:0/2986532294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c200ce7a0 msgr2=0x7f3c2015f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.197+0000 7f3c264dd640 1 --2- 172.21.15.60:0/2986532294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c200ce7a0 0x7f3c2015f570 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f3c100ae040 tx=0x7f3c1001ad40 comp rx=0 tx=0).stop 2024-04-25T10:17:12.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.197+0000 7f3c264dd640 1 -- 172.21.15.60:0/2986532294 shutdown_connections 2024-04-25T10:17:12.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.197+0000 7f3c264dd640 1 --2- 172.21.15.60:0/2986532294 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3c0c066f60 0x7f3c0c069420 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.197+0000 7f3c264dd640 1 --2- 172.21.15.60:0/2986532294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c200acb00 0x7f3c2015a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.197+0000 7f3c264dd640 1 --2- 172.21.15.60:0/2986532294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c2015bc90 0x7f3c2006edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.197+0000 7f3c264dd640 1 --2- 172.21.15.60:0/2986532294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c200ce7a0 0x7f3c2015f570 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.197+0000 7f3c264dd640 1 -- 172.21.15.60:0/2986532294 >> 172.21.15.60:0/2986532294 conn(0x7f3c200b12d0 msgr2=0x7f3c20155160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:12.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.197+0000 7f3c264dd640 1 -- 172.21.15.60:0/2986532294 shutdown_connections 2024-04-25T10:17:12.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.197+0000 7f3c264dd640 1 -- 172.21.15.60:0/2986532294 wait complete. 2024-04-25T10:17:12.218 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T10:17:12.218 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:17:12.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.329+0000 7ff976a79640 1 Processor -- start 2024-04-25T10:17:12.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.329+0000 7ff976a79640 1 -- start start 2024-04-25T10:17:12.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.329+0000 7ff976a79640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff970140f50 0x7ff970070920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.329+0000 7ff976a79640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff97015c510 0x7ff97015c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.329+0000 7ff976a79640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff970141610 con 0x7ff97013e670 2024-04-25T10:17:12.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.329+0000 7ff976a79640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9700c57c0 con 0x7ff970140f50 2024-04-25T10:17:12.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.329+0000 7ff976a79640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff970141eb0 con 0x7ff97015c510 2024-04-25T10:17:12.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff974815640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff97013e670 0x7ff97015f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40866/0 (socket says 172.21.15.60:40866) 2024-04-25T10:17:12.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff974815640 1 -- 172.21.15.60:0/3751113216 learned_addr learned my addr 172.21.15.60:0/3751113216 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:12.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff96ffff640 1 --2- 172.21.15.60:0/3751113216 >> v2:172.21.15.60:3300/0 conn(0x7ff970140f50 0x7ff970070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff975016640 1 --2- 172.21.15.60:0/3751113216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff97015c510 0x7ff97015c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff96f7fe640 1 -- 172.21.15.60:0/3751113216 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282348352 0 0) 0x7ff970141610 con 0x7ff97013e670 2024-04-25T10:17:12.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff96ffff640 1 -- 172.21.15.60:0/3751113216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff97015c510 msgr2=0x7ff97015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:12.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff96ffff640 1 --2- 172.21.15.60:0/3751113216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff97015c510 0x7ff97015c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff96ffff640 1 -- 172.21.15.60:0/3751113216 >> v1:172.21.15.60:6789/0 conn(0x7ff97013e670 legacy=0x7ff97015f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff96ffff640 1 -- 172.21.15.60:0/3751113216 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9701418f0 con 0x7ff970140f50 2024-04-25T10:17:12.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff96ffff640 1 --2- 172.21.15.60:0/3751113216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff970140f50 0x7ff970070920 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7ff9580097c0 tx=0x7ff95801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c63e6df94072497 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff96f7fe640 1 -- 172.21.15.60:0/3751113216 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9580b1070 con 0x7ff970140f50 2024-04-25T10:17:12.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff96f7fe640 1 -- 172.21.15.60:0/3751113216 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9580ac410 con 0x7ff970140f50 2024-04-25T10:17:12.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff96f7fe640 1 -- 172.21.15.60:0/3751113216 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9580ac7e0 con 0x7ff970140f50 2024-04-25T10:17:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff976a79640 1 -- 172.21.15.60:0/3751113216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff970140f50 msgr2=0x7ff970070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff976a79640 1 --2- 172.21.15.60:0/3751113216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff970140f50 0x7ff970070920 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7ff9580097c0 tx=0x7ff95801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff976a79640 1 -- 172.21.15.60:0/3751113216 shutdown_connections 2024-04-25T10:17:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff976a79640 1 --2- 172.21.15.60:0/3751113216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff97015c510 0x7ff97015c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff976a79640 1 --2- 172.21.15.60:0/3751113216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff970140f50 0x7ff970070920 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff976a79640 1 -- 172.21.15.60:0/3751113216 >> 172.21.15.60:0/3751113216 conn(0x7ff97006e400 msgr2=0x7ff97006e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:12.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff976a79640 1 -- 172.21.15.60:0/3751113216 shutdown_connections 2024-04-25T10:17:12.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.333+0000 7ff976a79640 1 -- 172.21.15.60:0/3751113216 wait complete. 2024-04-25T10:17:12.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff976a79640 1 Processor -- start 2024-04-25T10:17:12.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff976a79640 1 -- start start 2024-04-25T10:17:12.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff976a79640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff97013e670 0x7ff970151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff976a79640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff970140f50 0x7ff97013ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff976a79640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff97015c510 0x7ff9700b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff976a79640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97013ebd0 con 0x7ff970140f50 2024-04-25T10:17:12.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff976a79640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97013ea50 con 0x7ff97013e670 2024-04-25T10:17:12.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff976a79640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9700af500 con 0x7ff97015c510 2024-04-25T10:17:12.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff96ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff970140f50 0x7ff97013ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff96ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff970140f50 0x7ff97013ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46476/0 (socket says 172.21.15.60:46476) 2024-04-25T10:17:12.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff96ffff640 1 -- 172.21.15.60:0/1309265046 learned_addr learned my addr 172.21.15.60:0/1309265046 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:12.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff975016640 1 --2- 172.21.15.60:0/1309265046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff97015c510 0x7ff9700b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff974815640 1 --2- 172.21.15.60:0/1309265046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff97013e670 0x7ff970151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff96ffff640 1 -- 172.21.15.60:0/1309265046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff97015c510 msgr2=0x7ff9700b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff96ffff640 1 --2- 172.21.15.60:0/1309265046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff97015c510 0x7ff9700b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff96ffff640 1 -- 172.21.15.60:0/1309265046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff97013e670 msgr2=0x7ff970151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff96ffff640 1 --2- 172.21.15.60:0/1309265046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff97013e670 0x7ff970151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff96ffff640 1 -- 172.21.15.60:0/1309265046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9700b4400 con 0x7ff970140f50 2024-04-25T10:17:12.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff974815640 1 --2- 172.21.15.60:0/1309265046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff97013e670 0x7ff970151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:12.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff96ffff640 1 --2- 172.21.15.60:0/1309265046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff970140f50 0x7ff97013ac60 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7ff95801a0d0 tx=0x7ff958019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff96d7fa640 1 -- 172.21.15.60:0/1309265046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9580b1040 con 0x7ff970140f50 2024-04-25T10:17:12.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff96d7fa640 1 -- 172.21.15.60:0/1309265046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9580bb070 con 0x7ff970140f50 2024-04-25T10:17:12.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff976a79640 1 -- 172.21.15.60:0/1309265046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff97013a180 con 0x7ff970140f50 2024-04-25T10:17:12.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff96d7fa640 1 -- 172.21.15.60:0/1309265046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9580b5630 con 0x7ff970140f50 2024-04-25T10:17:12.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff976a79640 1 -- 172.21.15.60:0/1309265046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97013b1a0 con 0x7ff970140f50 2024-04-25T10:17:12.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff976a79640 1 -- 172.21.15.60:0/1309265046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff9700b46e0 con 0x7ff970140f50 2024-04-25T10:17:12.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff96d7fa640 1 -- 172.21.15.60:0/1309265046 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9580b57d0 con 0x7ff970140f50 2024-04-25T10:17:12.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.337+0000 7ff96d7fa640 1 --2- 172.21.15.60:0/1309265046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff950066f60 0x7ff950069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.341+0000 7ff96d7fa640 1 -- 172.21.15.60:0/1309265046 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7ff95811f290 con 0x7ff970140f50 2024-04-25T10:17:12.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.341+0000 7ff96d7fa640 1 --2- 172.21.15.60:0/1309265046 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x7ff95006cf60 0x7ff95006f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.341+0000 7ff96d7fa640 1 -- 172.21.15.60:0/1309265046 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9580aeec0 con 0x7ff95006cf60 2024-04-25T10:17:12.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.341+0000 7ff96d7fa640 1 -- 172.21.15.60:0/1309265046 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff95811f720 con 0x7ff970140f50 2024-04-25T10:17:12.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.341+0000 7ff975016640 1 --2- 172.21.15.60:0/1309265046 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x7ff95006cf60 0x7ff95006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.341+0000 7ff974815640 1 --2- 172.21.15.60:0/1309265046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff950066f60 0x7ff950069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.341+0000 7ff975016640 1 --2- 172.21.15.60:0/1309265046 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x7ff95006cf60 0x7ff95006f380 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.341+0000 7ff974815640 1 --2- 172.21.15.60:0/1309265046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff950066f60 0x7ff950069420 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff960000f70 tx=0x7ff960009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.341+0000 7ff96d7fa640 1 -- 172.21.15.60:0/1309265046 <== osd.2 v2:172.21.15.60:6800/1771147646 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff9580aeec0 con 0x7ff95006cf60 2024-04-25T10:17:12.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.353+0000 7ff976a79640 1 -- 172.21.15.60:0/1309265046 --> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff97013aa20 con 0x7ff95006cf60 2024-04-25T10:17:12.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.353+0000 7ff96d7fa640 1 -- 172.21.15.60:0/1309265046 <== osd.2 v2:172.21.15.60:6800/1771147646 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff97013aa20 con 0x7ff95006cf60 2024-04-25T10:17:12.359 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:17:12.359 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:17:12.359 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:17:12.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.353+0000 7ff976a79640 1 -- 172.21.15.60:0/1309265046 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x7ff95006cf60 msgr2=0x7ff95006f380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.353+0000 7ff976a79640 1 --2- 172.21.15.60:0/1309265046 >> [v2:172.21.15.60:6800/1771147646,v1:172.21.15.60:6802/1771147646] conn(0x7ff95006cf60 0x7ff95006f380 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.353+0000 7ff976a79640 1 -- 172.21.15.60:0/1309265046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff950066f60 msgr2=0x7ff950069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.353+0000 7ff976a79640 1 --2- 172.21.15.60:0/1309265046 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff950066f60 0x7ff950069420 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff960000f70 tx=0x7ff960009040 comp rx=0 tx=0).stop 2024-04-25T10:17:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.353+0000 7ff976a79640 1 -- 172.21.15.60:0/1309265046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff970140f50 msgr2=0x7ff97013ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.353+0000 7ff976a79640 1 --2- 172.21.15.60:0/1309265046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff970140f50 0x7ff97013ac60 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7ff95801a0d0 tx=0x7ff958019240 comp rx=0 tx=0).stop 2024-04-25T10:17:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.353+0000 7ff975016640 1 -- 172.21.15.60:0/1309265046 reap_dead start 2024-04-25T10:17:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.353+0000 7ff976a79640 1 -- 172.21.15.60:0/1309265046 shutdown_connections 2024-04-25T10:17:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.353+0000 7ff976a79640 1 -- 172.21.15.60:0/1309265046 >> 172.21.15.60:0/1309265046 conn(0x7ff97006e400 msgr2=0x7ff9700cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.353+0000 7ff976a79640 1 -- 172.21.15.60:0/1309265046 shutdown_connections 2024-04-25T10:17:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.353+0000 7ff976a79640 1 -- 172.21.15.60:0/1309265046 wait complete. 2024-04-25T10:17:12.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T10:17:12.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.477+0000 7f280b7d5640 1 Processor -- start 2024-04-25T10:17:12.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.477+0000 7f280b7d5640 1 -- start start 2024-04-25T10:17:12.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f280b7d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f280415a700 0x7f28040b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f280b7d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f28040d0360 0x7f28040b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f280b7d5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28040d0860 con 0x7f28040b19e0 2024-04-25T10:17:12.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f280b7d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28040d0b40 con 0x7f28040d0360 2024-04-25T10:17:12.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f280b7d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28040d0e20 con 0x7f280415a700 2024-04-25T10:17:12.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f2809d72640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f28040b19e0 0x7f280415e430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40876/0 (socket says 172.21.15.60:40876) 2024-04-25T10:17:12.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f2809d72640 1 -- 172.21.15.60:0/865363466 learned_addr learned my addr 172.21.15.60:0/865363466 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:12.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f2809571640 1 --2- 172.21.15.60:0/865363466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f280415a700 0x7f28040b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f2808d70640 1 --2- 172.21.15.60:0/865363466 >> v2:172.21.15.100:3300/0 conn(0x7f28040d0360 0x7f28040b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f27fbfff640 1 -- 172.21.15.60:0/865363466 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4003078845 0 0) 0x7f28040d0860 con 0x7f28040b19e0 2024-04-25T10:17:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f27fbfff640 1 -- 172.21.15.60:0/865363466 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27f0005740 con 0x7f28040b19e0 2024-04-25T10:17:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f27fbfff640 1 -- 172.21.15.60:0/865363466 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2384170363 0 0) 0x7f27f0005740 con 0x7f28040b19e0 2024-04-25T10:17:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f27fbfff640 1 -- 172.21.15.60:0/865363466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f280415a700 msgr2=0x7f28040b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f27fbfff640 1 --2- 172.21.15.60:0/865363466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f280415a700 0x7f28040b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f27fbfff640 1 -- 172.21.15.60:0/865363466 >> v2:172.21.15.100:3300/0 conn(0x7f28040d0360 msgr2=0x7f28040b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f27fbfff640 1 --2- 172.21.15.60:0/865363466 >> v2:172.21.15.100:3300/0 conn(0x7f28040d0360 0x7f28040b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f27fbfff640 1 -- 172.21.15.60:0/865363466 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28040d1100 con 0x7f28040b19e0 2024-04-25T10:17:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f27fbfff640 1 -- 172.21.15.60:0/865363466 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f27fc001a00 con 0x7f28040b19e0 2024-04-25T10:17:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f27fbfff640 1 -- 172.21.15.60:0/865363466 >> v1:172.21.15.60:6789/0 conn(0x7f28040b19e0 legacy=0x7f280415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f27fbfff640 1 --2- 172.21.15.60:0/865363466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f280415a700 0x7f27f0005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f27fbfff640 1 --2- 172.21.15.60:0/865363466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27f0006150 0x7f27f0006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f27fbfff640 1 --2- 172.21.15.60:0/865363466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27f0006f60 0x7f27f00073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f27fbfff640 1 -- 172.21.15.60:0/865363466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040d0b40 con 0x7f27f0006f60 2024-04-25T10:17:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f27fbfff640 1 -- 172.21.15.60:0/865363466 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040d0e20 con 0x7f27f0006150 2024-04-25T10:17:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f27fbfff640 1 -- 172.21.15.60:0/865363466 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f0003530 con 0x7f280415a700 2024-04-25T10:17:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f2809d72640 1 --2- 172.21.15.60:0/865363466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27f0006f60 0x7f27f00073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f2809571640 1 --2- 172.21.15.60:0/865363466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f280415a700 0x7f27f0005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f2808d70640 1 --2- 172.21.15.60:0/865363466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27f0006150 0x7f27f0006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f2809d72640 1 -- 172.21.15.60:0/865363466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f280415a700 msgr2=0x7f27f0005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f2809d72640 1 --2- 172.21.15.60:0/865363466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f280415a700 0x7f27f0005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f2809d72640 1 -- 172.21.15.60:0/865363466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27f0006150 msgr2=0x7f27f0006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f2809d72640 1 --2- 172.21.15.60:0/865363466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27f0006150 0x7f27f0006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f2809d72640 1 -- 172.21.15.60:0/865363466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f0005740 con 0x7f27f0006f60 2024-04-25T10:17:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.481+0000 7f2808d70640 1 --2- 172.21.15.60:0/865363466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27f0006150 0x7f27f0006a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f2809d72640 1 --2- 172.21.15.60:0/865363466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27f0006f60 0x7f27f00073e0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00a870 tx=0x7f27fc01d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359ed3ea465afb96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f27fbfff640 1 -- 172.21.15.60:0/865363466 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc00a4c0 con 0x7f27f0006f60 2024-04-25T10:17:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f27fbfff640 1 -- 172.21.15.60:0/865363466 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc0072d0 con 0x7f27f0006f60 2024-04-25T10:17:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f27fbfff640 1 -- 172.21.15.60:0/865363466 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0076c0 con 0x7f27f0006f60 2024-04-25T10:17:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f280b7d5640 1 -- 172.21.15.60:0/865363466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27f0006f60 msgr2=0x7f27f00073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f280b7d5640 1 --2- 172.21.15.60:0/865363466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27f0006f60 0x7f27f00073e0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00a870 tx=0x7f27fc01d3a0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f2809d72640 1 -- 172.21.15.60:0/865363466 reap_dead start 2024-04-25T10:17:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f280b7d5640 1 -- 172.21.15.60:0/865363466 shutdown_connections 2024-04-25T10:17:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f280b7d5640 1 -- 172.21.15.60:0/865363466 >> 172.21.15.60:0/865363466 conn(0x7f2804072920 msgr2=0x7f2804099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f280b7d5640 1 -- 172.21.15.60:0/865363466 shutdown_connections 2024-04-25T10:17:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f280b7d5640 1 -- 172.21.15.60:0/865363466 wait complete. 2024-04-25T10:17:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f280b7d5640 1 Processor -- start 2024-04-25T10:17:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f280b7d5640 1 -- start start 2024-04-25T10:17:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f280b7d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27f0006f60 0x7f280414d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f280b7d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28041563d0 0x7f28041efb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f280b7d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804157cc0 0x7f2804158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f280b7d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280415ac60 con 0x7f28041563d0 2024-04-25T10:17:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f280b7d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280415aae0 con 0x7f2804157cc0 2024-04-25T10:17:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f280b7d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28040ce270 con 0x7f27f0006f60 2024-04-25T10:17:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f2808d70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28041563d0 0x7f28041efb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f2808d70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28041563d0 0x7f28041efb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46482/0 (socket says 172.21.15.60:46482) 2024-04-25T10:17:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f2808d70640 1 -- 172.21.15.60:0/4266378838 learned_addr learned my addr 172.21.15.60:0/4266378838 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f2809d72640 1 --2- 172.21.15.60:0/4266378838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804157cc0 0x7f2804158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f2809571640 1 --2- 172.21.15.60:0/4266378838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27f0006f60 0x7f280414d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f2808d70640 1 -- 172.21.15.60:0/4266378838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27f0006f60 msgr2=0x7f280414d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f2808d70640 1 --2- 172.21.15.60:0/4266378838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27f0006f60 0x7f280414d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f2808d70640 1 -- 172.21.15.60:0/4266378838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804157cc0 msgr2=0x7f2804158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f2808d70640 1 --2- 172.21.15.60:0/4266378838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804157cc0 0x7f2804158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f2808d70640 1 -- 172.21.15.60:0/4266378838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28041f00a0 con 0x7f28041563d0 2024-04-25T10:17:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f2809d72640 1 --2- 172.21.15.60:0/4266378838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804157cc0 0x7f2804158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.485+0000 7f2808d70640 1 --2- 172.21.15.60:0/4266378838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28041563d0 0x7f28041efb60 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f27f4011620 tx=0x7f27f4011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.489+0000 7f27f9ffb640 1 -- 172.21.15.60:0/4266378838 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4004220 con 0x7f28041563d0 2024-04-25T10:17:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.489+0000 7f27f9ffb640 1 -- 172.21.15.60:0/4266378838 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f40043c0 con 0x7f28041563d0 2024-04-25T10:17:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.489+0000 7f280b7d5640 1 -- 172.21.15.60:0/4266378838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28040b1670 con 0x7f28041563d0 2024-04-25T10:17:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.489+0000 7f27f9ffb640 1 -- 172.21.15.60:0/4266378838 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4015890 con 0x7f28041563d0 2024-04-25T10:17:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.489+0000 7f280b7d5640 1 -- 172.21.15.60:0/4266378838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2804156b70 con 0x7f28041563d0 2024-04-25T10:17:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.489+0000 7f280b7d5640 1 -- 172.21.15.60:0/4266378838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f280406ccb0 con 0x7f28041563d0 2024-04-25T10:17:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.489+0000 7f27f9ffb640 1 -- 172.21.15.60:0/4266378838 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27f4015a30 con 0x7f28041563d0 2024-04-25T10:17:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.489+0000 7f27f9ffb640 1 --2- 172.21.15.60:0/4266378838 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f27e8066f90 0x7f27e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.489+0000 7f27f9ffb640 1 -- 172.21.15.60:0/4266378838 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f27f4085850 con 0x7f28041563d0 2024-04-25T10:17:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.489+0000 7f2809571640 1 --2- 172.21.15.60:0/4266378838 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f27e8066f90 0x7f27e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.489+0000 7f2809571640 1 --2- 172.21.15.60:0/4266378838 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f27e8066f90 0x7f27e8069450 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2804153ce0 tx=0x7f2800008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.493+0000 7f27f9ffb640 1 -- 172.21.15.60:0/4266378838 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28041f00a0 con 0x7f28041563d0 2024-04-25T10:17:12.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.649+0000 7f280b7d5640 1 -- 172.21.15.60:0/4266378838 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f28040cce70 con 0x7f27e8066f90 2024-04-25T10:17:12.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.653+0000 7f27f9ffb640 1 -- 172.21.15.60:0/4266378838 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f28040cce70 con 0x7f27e8066f90 2024-04-25T10:17:12.657 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T10:17:12.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.653+0000 7f280b7d5640 1 -- 172.21.15.60:0/4266378838 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f27e8066f90 msgr2=0x7f27e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.653+0000 7f280b7d5640 1 --2- 172.21.15.60:0/4266378838 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f27e8066f90 0x7f27e8069450 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2804153ce0 tx=0x7f2800008040 comp rx=0 tx=0).stop 2024-04-25T10:17:12.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.653+0000 7f280b7d5640 1 -- 172.21.15.60:0/4266378838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28041563d0 msgr2=0x7f28041efb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.653+0000 7f280b7d5640 1 --2- 172.21.15.60:0/4266378838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28041563d0 0x7f28041efb60 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f27f4011620 tx=0x7f27f4011ae0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.653+0000 7f280b7d5640 1 -- 172.21.15.60:0/4266378838 shutdown_connections 2024-04-25T10:17:12.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.653+0000 7f280b7d5640 1 --2- 172.21.15.60:0/4266378838 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f27e8066f90 0x7f27e8069450 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.653+0000 7f280b7d5640 1 --2- 172.21.15.60:0/4266378838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804157cc0 0x7f2804158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.653+0000 7f280b7d5640 1 --2- 172.21.15.60:0/4266378838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28041563d0 0x7f28041efb60 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.653+0000 7f280b7d5640 1 --2- 172.21.15.60:0/4266378838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27f0006f60 0x7f280414d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.653+0000 7f280b7d5640 1 -- 172.21.15.60:0/4266378838 >> 172.21.15.60:0/4266378838 conn(0x7f2804072920 msgr2=0x7f28040cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:12.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.653+0000 7f280b7d5640 1 -- 172.21.15.60:0/4266378838 shutdown_connections 2024-04-25T10:17:12.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.653+0000 7f280b7d5640 1 -- 172.21.15.60:0/4266378838 wait complete. 2024-04-25T10:17:12.675 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T10:17:12.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:17:12.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.797+0000 7ff31be2d640 1 Processor -- start 2024-04-25T10:17:12.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.797+0000 7ff31be2d640 1 -- start start 2024-04-25T10:17:12.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff31be2d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff314070990 0x7ff31415c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff31be2d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31415be50 con 0x7ff3140afc10 2024-04-25T10:17:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff31be2d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff314141730 con 0x7ff31415cb70 2024-04-25T10:17:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff31be2d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140c57c0 con 0x7ff314070990 2024-04-25T10:17:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff319bc9640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff3140afc10 0x7ff314140f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40892/0 (socket says 172.21.15.60:40892) 2024-04-25T10:17:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff319bc9640 1 -- 172.21.15.60:0/1876004742 learned_addr learned my addr 172.21.15.60:0/1876004742 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff3193c8640 1 --2- 172.21.15.60:0/1876004742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff314070990 0x7ff31415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff318bc7640 1 -- 172.21.15.60:0/1876004742 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440266682 0 0) 0x7ff31415be50 con 0x7ff3140afc10 2024-04-25T10:17:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff318bc7640 1 -- 172.21.15.60:0/1876004742 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff300003530 con 0x7ff3140afc10 2024-04-25T10:17:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff318bc7640 1 -- 172.21.15.60:0/1876004742 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3637555369 0 0) 0x7ff300003530 con 0x7ff3140afc10 2024-04-25T10:17:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff318bc7640 1 -- 172.21.15.60:0/1876004742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff314070990 msgr2=0x7ff31415c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff318bc7640 1 --2- 172.21.15.60:0/1876004742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff314070990 0x7ff31415c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff318bc7640 1 -- 172.21.15.60:0/1876004742 >> v1:172.21.15.100:6789/0 conn(0x7ff31415cb70 legacy=0x7ff31406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff318bc7640 1 -- 172.21.15.60:0/1876004742 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff314141a10 con 0x7ff3140afc10 2024-04-25T10:17:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff318bc7640 1 -- 172.21.15.60:0/1876004742 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7ff310001680 con 0x7ff3140afc10 2024-04-25T10:17:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff318bc7640 1 -- 172.21.15.60:0/1876004742 >> v1:172.21.15.60:6789/0 conn(0x7ff3140afc10 legacy=0x7ff314140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff318bc7640 1 --2- 172.21.15.60:0/1876004742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000039f0 0x7ff300003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff318bc7640 1 --2- 172.21.15.60:0/1876004742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff300004810 0x7ff300004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff318bc7640 1 --2- 172.21.15.60:0/1876004742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff314070990 0x7ff3000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff318bc7640 1 -- 172.21.15.60:0/1876004742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30000ba80 con 0x7ff3000039f0 2024-04-25T10:17:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff318bc7640 1 -- 172.21.15.60:0/1876004742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30000bc00 con 0x7ff300004810 2024-04-25T10:17:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff318bc7640 1 -- 172.21.15.60:0/1876004742 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30000be00 con 0x7ff314070990 2024-04-25T10:17:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff319bc9640 1 --2- 172.21.15.60:0/1876004742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000039f0 0x7ff300003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff3193c8640 1 --2- 172.21.15.60:0/1876004742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff300004810 0x7ff300004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff31a3ca640 1 --2- 172.21.15.60:0/1876004742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff314070990 0x7ff3000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff319bc9640 1 -- 172.21.15.60:0/1876004742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff314070990 msgr2=0x7ff3000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff319bc9640 1 --2- 172.21.15.60:0/1876004742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff314070990 0x7ff3000051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff319bc9640 1 -- 172.21.15.60:0/1876004742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff300004810 msgr2=0x7ff300004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff319bc9640 1 --2- 172.21.15.60:0/1876004742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff300004810 0x7ff300004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff319bc9640 1 -- 172.21.15.60:0/1876004742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff300003530 con 0x7ff3000039f0 2024-04-25T10:17:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.801+0000 7ff31a3ca640 1 --2- 172.21.15.60:0/1876004742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff314070990 0x7ff3000051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff319bc9640 1 --2- 172.21.15.60:0/1876004742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000039f0 0x7ff300003e10 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7ff31000c210 tx=0x7ff31001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ed2e892362e4fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff318bc7640 1 -- 172.21.15.60:0/1876004742 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310007d20 con 0x7ff3000039f0 2024-04-25T10:17:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff318bc7640 1 -- 172.21.15.60:0/1876004742 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3100a9070 con 0x7ff3000039f0 2024-04-25T10:17:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff318bc7640 1 -- 172.21.15.60:0/1876004742 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310005d60 con 0x7ff3000039f0 2024-04-25T10:17:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31be2d640 1 -- 172.21.15.60:0/1876004742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000039f0 msgr2=0x7ff300003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31be2d640 1 --2- 172.21.15.60:0/1876004742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000039f0 0x7ff300003e10 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7ff31000c210 tx=0x7ff31001cc40 comp rx=0 tx=0).stop 2024-04-25T10:17:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31a3ca640 1 -- 172.21.15.60:0/1876004742 reap_dead start 2024-04-25T10:17:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31be2d640 1 -- 172.21.15.60:0/1876004742 shutdown_connections 2024-04-25T10:17:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31be2d640 1 -- 172.21.15.60:0/1876004742 >> 172.21.15.60:0/1876004742 conn(0x7ff31406e400 msgr2=0x7ff31406e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31be2d640 1 -- 172.21.15.60:0/1876004742 shutdown_connections 2024-04-25T10:17:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31be2d640 1 -- 172.21.15.60:0/1876004742 wait complete. 2024-04-25T10:17:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31be2d640 1 Processor -- start 2024-04-25T10:17:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31be2d640 1 -- start start 2024-04-25T10:17:12.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31be2d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000039f0 0x7ff314131cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31be2d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3140b2b00 0x7ff31413d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31be2d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3140b2f80 0x7ff31413d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31be2d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140b0200 con 0x7ff3000039f0 2024-04-25T10:17:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31be2d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31415fe60 con 0x7ff3140b2b00 2024-04-25T10:17:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31be2d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31406b320 con 0x7ff3140b2f80 2024-04-25T10:17:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff319bc9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000039f0 0x7ff314131cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff319bc9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000039f0 0x7ff314131cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46504/0 (socket says 172.21.15.60:46504) 2024-04-25T10:17:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff319bc9640 1 -- 172.21.15.60:0/491951856 learned_addr learned my addr 172.21.15.60:0/491951856 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff3193c8640 1 --2- 172.21.15.60:0/491951856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3140b2b00 0x7ff31413d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31a3ca640 1 --2- 172.21.15.60:0/491951856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3140b2f80 0x7ff31413d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff319bc9640 1 -- 172.21.15.60:0/491951856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3140b2f80 msgr2=0x7ff31413d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff319bc9640 1 --2- 172.21.15.60:0/491951856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3140b2f80 0x7ff31413d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff319bc9640 1 -- 172.21.15.60:0/491951856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3140b2b00 msgr2=0x7ff31413d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff319bc9640 1 --2- 172.21.15.60:0/491951856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3140b2b00 0x7ff31413d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff319bc9640 1 -- 172.21.15.60:0/491951856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff314160550 con 0x7ff3000039f0 2024-04-25T10:17:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff31a3ca640 1 --2- 172.21.15.60:0/491951856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3140b2f80 0x7ff31413d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.805+0000 7ff319bc9640 1 --2- 172.21.15.60:0/491951856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000039f0 0x7ff314131cc0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7ff31001ca70 tx=0x7ff300005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff30a7fc640 1 -- 172.21.15.60:0/491951856 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310005d40 con 0x7ff3000039f0 2024-04-25T10:17:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff30a7fc640 1 -- 172.21.15.60:0/491951856 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3100047f0 con 0x7ff3000039f0 2024-04-25T10:17:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff31be2d640 1 -- 172.21.15.60:0/491951856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff314160830 con 0x7ff3000039f0 2024-04-25T10:17:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff31be2d640 1 -- 172.21.15.60:0/491951856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff314160df0 con 0x7ff3000039f0 2024-04-25T10:17:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff30a7fc640 1 -- 172.21.15.60:0/491951856 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310004be0 con 0x7ff3000039f0 2024-04-25T10:17:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff31be2d640 1 -- 172.21.15.60:0/491951856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff3141614b0 con 0x7ff3000039f0 2024-04-25T10:17:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff30a7fc640 1 -- 172.21.15.60:0/491951856 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff310007040 con 0x7ff3000039f0 2024-04-25T10:17:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff30a7fc640 1 --2- 172.21.15.60:0/491951856 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff2f8066f90 0x7ff2f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff30a7fc640 1 -- 172.21.15.60:0/491951856 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7ff31011e360 con 0x7ff3000039f0 2024-04-25T10:17:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff30a7fc640 1 --2- 172.21.15.60:0/491951856 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x7ff2f806cf90 0x7ff2f806f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff30a7fc640 1 -- 172.21.15.60:0/491951856 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff2f806fa60 con 0x7ff2f806cf90 2024-04-25T10:17:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff30a7fc640 1 -- 172.21.15.60:0/491951856 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff310125050 con 0x7ff3000039f0 2024-04-25T10:17:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff3193c8640 1 --2- 172.21.15.60:0/491951856 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff2f8066f90 0x7ff2f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff31a3ca640 1 --2- 172.21.15.60:0/491951856 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x7ff2f806cf90 0x7ff2f806f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff31a3ca640 1 --2- 172.21.15.60:0/491951856 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x7ff2f806cf90 0x7ff2f806f3b0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff3193c8640 1 --2- 172.21.15.60:0/491951856 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff2f8066f90 0x7ff2f8069450 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff3141fa150 tx=0x7ff304008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.809+0000 7ff30a7fc640 1 -- 172.21.15.60:0/491951856 <== osd.3 v2:172.21.15.60:6814/1867957156 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff2f806fa60 con 0x7ff2f806cf90 2024-04-25T10:17:12.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.825+0000 7ff31be2d640 1 -- 172.21.15.60:0/491951856 --> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff314074770 con 0x7ff2f806cf90 2024-04-25T10:17:12.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.825+0000 7ff30a7fc640 1 -- 172.21.15.60:0/491951856 <== osd.3 v2:172.21.15.60:6814/1867957156 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff314074770 con 0x7ff2f806cf90 2024-04-25T10:17:12.830 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:17:12.830 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:17:12.830 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:17:12.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.825+0000 7ff31be2d640 1 -- 172.21.15.60:0/491951856 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x7ff2f806cf90 msgr2=0x7ff2f806f3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.825+0000 7ff31be2d640 1 --2- 172.21.15.60:0/491951856 >> [v2:172.21.15.60:6814/1867957156,v1:172.21.15.60:6815/1867957156] conn(0x7ff2f806cf90 0x7ff2f806f3b0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.825+0000 7ff31be2d640 1 -- 172.21.15.60:0/491951856 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff2f8066f90 msgr2=0x7ff2f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.825+0000 7ff31be2d640 1 --2- 172.21.15.60:0/491951856 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff2f8066f90 0x7ff2f8069450 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff3141fa150 tx=0x7ff304008040 comp rx=0 tx=0).stop 2024-04-25T10:17:12.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.825+0000 7ff31be2d640 1 -- 172.21.15.60:0/491951856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000039f0 msgr2=0x7ff314131cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.825+0000 7ff31be2d640 1 --2- 172.21.15.60:0/491951856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000039f0 0x7ff314131cc0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7ff31001ca70 tx=0x7ff300005840 comp rx=0 tx=0).stop 2024-04-25T10:17:12.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.825+0000 7ff31a3ca640 1 -- 172.21.15.60:0/491951856 reap_dead start 2024-04-25T10:17:12.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.825+0000 7ff31be2d640 1 -- 172.21.15.60:0/491951856 shutdown_connections 2024-04-25T10:17:12.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.825+0000 7ff31be2d640 1 -- 172.21.15.60:0/491951856 >> 172.21.15.60:0/491951856 conn(0x7ff31406e400 msgr2=0x7ff314140d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:12.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.825+0000 7ff31be2d640 1 -- 172.21.15.60:0/491951856 shutdown_connections 2024-04-25T10:17:12.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.825+0000 7ff31be2d640 1 -- 172.21.15.60:0/491951856 wait complete. 2024-04-25T10:17:12.846 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T10:17:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b5a984640 1 Processor -- start 2024-04-25T10:17:12.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b5a984640 1 -- start start 2024-04-25T10:17:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b5a984640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4b5415deb0 0x7f4b540b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b5a984640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b540ce9c0 0x7f4b540d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b5a984640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b5406d200 con 0x7f4b540d0340 2024-04-25T10:17:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b5a984640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b540cf2c0 con 0x7f4b5415deb0 2024-04-25T10:17:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b5a984640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b540cefe0 con 0x7f4b540ce9c0 2024-04-25T10:17:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b537fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4b540d0340 0x7f4b540ce2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40898/0 (socket says 172.21.15.60:40898) 2024-04-25T10:17:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b537fe640 1 -- 172.21.15.60:0/3029158284 learned_addr learned my addr 172.21.15.60:0/3029158284 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b53fff640 1 --2- 172.21.15.60:0/3029158284 >> v2:172.21.15.60:3300/0 conn(0x7f4b5415deb0 0x7f4b540b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b58f21640 1 --2- 172.21.15.60:0/3029158284 >> v2:172.21.15.100:3300/0 conn(0x7f4b540ce9c0 0x7f4b540d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b52ffd640 1 -- 172.21.15.60:0/3029158284 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491640635 0 0) 0x7f4b5406d200 con 0x7f4b540d0340 2024-04-25T10:17:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b52ffd640 1 -- 172.21.15.60:0/3029158284 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b40003530 con 0x7f4b540d0340 2024-04-25T10:17:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b53fff640 1 -- 172.21.15.60:0/3029158284 >> v2:172.21.15.100:3300/0 conn(0x7f4b540ce9c0 msgr2=0x7f4b540d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b53fff640 1 --2- 172.21.15.60:0/3029158284 >> v2:172.21.15.100:3300/0 conn(0x7f4b540ce9c0 0x7f4b540d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b53fff640 1 -- 172.21.15.60:0/3029158284 >> v1:172.21.15.60:6789/0 conn(0x7f4b540d0340 legacy=0x7f4b540ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b53fff640 1 -- 172.21.15.60:0/3029158284 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b540d1d70 con 0x7f4b5415deb0 2024-04-25T10:17:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.961+0000 7f4b58f21640 1 --2- 172.21.15.60:0/3029158284 >> v2:172.21.15.100:3300/0 conn(0x7f4b540ce9c0 0x7f4b540d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.965+0000 7f4b53fff640 1 --2- 172.21.15.60:0/3029158284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b5415deb0 0x7f4b540b6c10 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f4b540b9a00 tx=0x7f4b3c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc6e82d5851cb45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.965+0000 7f4b52ffd640 1 -- 172.21.15.60:0/3029158284 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0b0070 con 0x7f4b5415deb0 2024-04-25T10:17:12.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.965+0000 7f4b52ffd640 1 -- 172.21.15.60:0/3029158284 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c0ab600 con 0x7f4b5415deb0 2024-04-25T10:17:12.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.965+0000 7f4b52ffd640 1 -- 172.21.15.60:0/3029158284 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0ab9d0 con 0x7f4b5415deb0 2024-04-25T10:17:12.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.965+0000 7f4b5a984640 1 -- 172.21.15.60:0/3029158284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b5415deb0 msgr2=0x7f4b540b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.965+0000 7f4b5a984640 1 --2- 172.21.15.60:0/3029158284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b5415deb0 0x7f4b540b6c10 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f4b540b9a00 tx=0x7f4b3c01c480 comp rx=0 tx=0).stop 2024-04-25T10:17:12.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.965+0000 7f4b5a984640 1 -- 172.21.15.60:0/3029158284 shutdown_connections 2024-04-25T10:17:12.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.965+0000 7f4b5a984640 1 --2- 172.21.15.60:0/3029158284 >> v2:172.21.15.100:3300/0 conn(0x7f4b540ce9c0 0x7f4b540d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.965+0000 7f4b5a984640 1 --2- 172.21.15.60:0/3029158284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b5415deb0 0x7f4b540b6c10 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.965+0000 7f4b5a984640 1 -- 172.21.15.60:0/3029158284 >> 172.21.15.60:0/3029158284 conn(0x7f4b5406c090 msgr2=0x7f4b5406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:12.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.965+0000 7f4b5a984640 1 -- 172.21.15.60:0/3029158284 shutdown_connections 2024-04-25T10:17:12.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.965+0000 7f4b5a984640 1 -- 172.21.15.60:0/3029158284 wait complete. 2024-04-25T10:17:12.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.965+0000 7f4b5a984640 1 Processor -- start 2024-04-25T10:17:12.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.965+0000 7f4b5a984640 1 -- start start 2024-04-25T10:17:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b5a984640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b540ce9c0 0x7f4b5414f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b5a984640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b540d0340 0x7f4b540ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b5a984640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b5415deb0 0x7f4b540cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b5a984640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540d3430 con 0x7f4b540ce9c0 2024-04-25T10:17:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b5a984640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540d32b0 con 0x7f4b5415deb0 2024-04-25T10:17:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b5a984640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540d35b0 con 0x7f4b540d0340 2024-04-25T10:17:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b53fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b540ce9c0 0x7f4b5414f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b53fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b540ce9c0 0x7f4b5414f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46536/0 (socket says 172.21.15.60:46536) 2024-04-25T10:17:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b53fff640 1 -- 172.21.15.60:0/412587005 learned_addr learned my addr 172.21.15.60:0/412587005 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b53fff640 1 -- 172.21.15.60:0/412587005 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b540d0340 msgr2=0x7f4b540ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b53fff640 1 --2- 172.21.15.60:0/412587005 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b540d0340 0x7f4b540ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b53fff640 1 -- 172.21.15.60:0/412587005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b5415deb0 msgr2=0x7f4b540cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b53fff640 1 --2- 172.21.15.60:0/412587005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b5415deb0 0x7f4b540cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b53fff640 1 -- 172.21.15.60:0/412587005 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b541526b0 con 0x7f4b540ce9c0 2024-04-25T10:17:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b53fff640 1 --2- 172.21.15.60:0/412587005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b540ce9c0 0x7f4b5414f120 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0ae040 tx=0x7f4b3c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b50ff9640 1 -- 172.21.15.60:0/412587005 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0ba070 con 0x7f4b540ce9c0 2024-04-25T10:17:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b50ff9640 1 -- 172.21.15.60:0/412587005 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c0b43f0 con 0x7f4b540ce9c0 2024-04-25T10:17:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b5a984640 1 -- 172.21.15.60:0/412587005 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b54152990 con 0x7f4b540ce9c0 2024-04-25T10:17:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b50ff9640 1 -- 172.21.15.60:0/412587005 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0b0040 con 0x7f4b540ce9c0 2024-04-25T10:17:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b5a984640 1 -- 172.21.15.60:0/412587005 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b54152f50 con 0x7f4b540ce9c0 2024-04-25T10:17:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b5a984640 1 -- 172.21.15.60:0/412587005 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b54074770 con 0x7f4b540ce9c0 2024-04-25T10:17:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b50ff9640 1 -- 172.21.15.60:0/412587005 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b3c0bd070 con 0x7f4b540ce9c0 2024-04-25T10:17:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b50ff9640 1 --2- 172.21.15.60:0/412587005 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b34066f60 0x7f4b34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b50ff9640 1 -- 172.21.15.60:0/412587005 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f4b3c11de40 con 0x7f4b540ce9c0 2024-04-25T10:17:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.969+0000 7f4b537fe640 1 --2- 172.21.15.60:0/412587005 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b34066f60 0x7f4b34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.973+0000 7f4b537fe640 1 --2- 172.21.15.60:0/412587005 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b34066f60 0x7f4b34069420 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4b440012f0 tx=0x7f4b44001660 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:12.973+0000 7f4b50ff9640 1 -- 172.21.15.60:0/412587005 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b3c016070 con 0x7f4b540ce9c0 2024-04-25T10:17:13.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.129+0000 7f4b5a984640 1 -- 172.21.15.60:0/412587005 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b540cd710 con 0x7f4b34066f60 2024-04-25T10:17:13.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.129+0000 7f4b50ff9640 1 -- 172.21.15.60:0/412587005 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4b540cd710 con 0x7f4b34066f60 2024-04-25T10:17:13.137 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T10:17:13.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.133+0000 7f4b5a984640 1 -- 172.21.15.60:0/412587005 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b34066f60 msgr2=0x7f4b34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.133+0000 7f4b5a984640 1 --2- 172.21.15.60:0/412587005 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b34066f60 0x7f4b34069420 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4b440012f0 tx=0x7f4b44001660 comp rx=0 tx=0).stop 2024-04-25T10:17:13.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.133+0000 7f4b5a984640 1 -- 172.21.15.60:0/412587005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b540ce9c0 msgr2=0x7f4b5414f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.133+0000 7f4b5a984640 1 --2- 172.21.15.60:0/412587005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b540ce9c0 0x7f4b5414f120 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0ae040 tx=0x7f4b3c019220 comp rx=0 tx=0).stop 2024-04-25T10:17:13.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.133+0000 7f4b5a984640 1 -- 172.21.15.60:0/412587005 shutdown_connections 2024-04-25T10:17:13.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.133+0000 7f4b5a984640 1 --2- 172.21.15.60:0/412587005 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f4b34066f60 0x7f4b34069420 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.133+0000 7f4b5a984640 1 --2- 172.21.15.60:0/412587005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b5415deb0 0x7f4b540cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.133+0000 7f4b5a984640 1 --2- 172.21.15.60:0/412587005 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b540d0340 0x7f4b540ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.133+0000 7f4b5a984640 1 --2- 172.21.15.60:0/412587005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b540ce9c0 0x7f4b5414f120 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.133+0000 7f4b5a984640 1 -- 172.21.15.60:0/412587005 >> 172.21.15.60:0/412587005 conn(0x7f4b5406c090 msgr2=0x7f4b54160be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:13.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.133+0000 7f4b5a984640 1 -- 172.21.15.60:0/412587005 shutdown_connections 2024-04-25T10:17:13.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.133+0000 7f4b5a984640 1 -- 172.21.15.60:0/412587005 wait complete. 2024-04-25T10:17:13.155 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T10:17:13.155 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:17:13.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.277+0000 7ff96a9f4640 1 Processor -- start 2024-04-25T10:17:13.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.277+0000 7ff96a9f4640 1 -- start start 2024-04-25T10:17:13.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.277+0000 7ff96a9f4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff9640b3340 0x7ff9640cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.277+0000 7ff96a9f4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff964162490 0x7ff96415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.277+0000 7ff96a9f4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff964162d50 con 0x7ff9640b3720 2024-04-25T10:17:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.277+0000 7ff96a9f4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff964163030 con 0x7ff9640b3340 2024-04-25T10:17:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.277+0000 7ff96a9f4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff964165b20 con 0x7ff964162490 2024-04-25T10:17:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.277+0000 7ff9637fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff9640b3720 0x7ff964166320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40908/0 (socket says 172.21.15.60:40908) 2024-04-25T10:17:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.277+0000 7ff9637fe640 1 -- 172.21.15.60:0/1124165157 learned_addr learned my addr 172.21.15.60:0/1124165157 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.277+0000 7ff963fff640 1 --2- 172.21.15.60:0/1124165157 >> v2:172.21.15.60:3300/0 conn(0x7ff9640b3340 0x7ff9640cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.277+0000 7ff968f91640 1 --2- 172.21.15.60:0/1124165157 >> v2:172.21.15.100:3300/0 conn(0x7ff964162490 0x7ff96415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.277+0000 7ff962ffd640 1 -- 172.21.15.60:0/1124165157 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147343206 0 0) 0x7ff964162d50 con 0x7ff9640b3720 2024-04-25T10:17:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff962ffd640 1 -- 172.21.15.60:0/1124165157 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff94c003530 con 0x7ff9640b3720 2024-04-25T10:17:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff963fff640 1 -- 172.21.15.60:0/1124165157 >> v2:172.21.15.100:3300/0 conn(0x7ff964162490 msgr2=0x7ff96415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff963fff640 1 --2- 172.21.15.60:0/1124165157 >> v2:172.21.15.100:3300/0 conn(0x7ff964162490 0x7ff96415bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff963fff640 1 -- 172.21.15.60:0/1124165157 >> v1:172.21.15.60:6789/0 conn(0x7ff9640b3720 legacy=0x7ff964166320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff963fff640 1 -- 172.21.15.60:0/1124165157 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff964165e00 con 0x7ff9640b3340 2024-04-25T10:17:13.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff963fff640 1 --2- 172.21.15.60:0/1124165157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9640b3340 0x7ff9640cfc00 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7ff964071200 tx=0x7ff95401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0351a453bce3c97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:13.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff962ffd640 1 -- 172.21.15.60:0/1124165157 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9540b1070 con 0x7ff9640b3340 2024-04-25T10:17:13.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff962ffd640 1 -- 172.21.15.60:0/1124165157 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9540ac5f0 con 0x7ff9640b3340 2024-04-25T10:17:13.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff962ffd640 1 -- 172.21.15.60:0/1124165157 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9540ac9c0 con 0x7ff9640b3340 2024-04-25T10:17:13.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff96a9f4640 1 -- 172.21.15.60:0/1124165157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9640b3340 msgr2=0x7ff9640cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff96a9f4640 1 --2- 172.21.15.60:0/1124165157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9640b3340 0x7ff9640cfc00 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7ff964071200 tx=0x7ff95401c480 comp rx=0 tx=0).stop 2024-04-25T10:17:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff96a9f4640 1 -- 172.21.15.60:0/1124165157 shutdown_connections 2024-04-25T10:17:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff96a9f4640 1 --2- 172.21.15.60:0/1124165157 >> v2:172.21.15.100:3300/0 conn(0x7ff964162490 0x7ff96415bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff96a9f4640 1 --2- 172.21.15.60:0/1124165157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9640b3340 0x7ff9640cfc00 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff96a9f4640 1 -- 172.21.15.60:0/1124165157 >> 172.21.15.60:0/1124165157 conn(0x7ff964066fa0 msgr2=0x7ff96406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff96a9f4640 1 -- 172.21.15.60:0/1124165157 shutdown_connections 2024-04-25T10:17:13.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.281+0000 7ff96a9f4640 1 -- 172.21.15.60:0/1124165157 wait complete. 2024-04-25T10:17:13.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff96a9f4640 1 Processor -- start 2024-04-25T10:17:13.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff96a9f4640 1 -- start start 2024-04-25T10:17:13.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff96a9f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9640b3340 0x7ff9641ff190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff96a9f4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9640b3720 0x7ff964156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff96a9f4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff964162490 0x7ff964157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff96a9f4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9640ae4f0 con 0x7ff9640b3720 2024-04-25T10:17:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff96a9f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96406e770 con 0x7ff9640b3340 2024-04-25T10:17:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff96a9f4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9640ae670 con 0x7ff964162490 2024-04-25T10:17:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff9637fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9640b3720 0x7ff964156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff9637fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9640b3720 0x7ff964156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46554/0 (socket says 172.21.15.60:46554) 2024-04-25T10:17:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff9637fe640 1 -- 172.21.15.60:0/4065839524 learned_addr learned my addr 172.21.15.60:0/4065839524 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff963fff640 1 --2- 172.21.15.60:0/4065839524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9640b3340 0x7ff9641ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff9637fe640 1 -- 172.21.15.60:0/4065839524 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff964162490 msgr2=0x7ff964157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff9637fe640 1 --2- 172.21.15.60:0/4065839524 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff964162490 0x7ff964157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff9637fe640 1 -- 172.21.15.60:0/4065839524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9640b3340 msgr2=0x7ff9641ff190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff9637fe640 1 --2- 172.21.15.60:0/4065839524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9640b3340 0x7ff9641ff190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff9637fe640 1 -- 172.21.15.60:0/4065839524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9640b0e50 con 0x7ff9640b3720 2024-04-25T10:17:13.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff9637fe640 1 --2- 172.21.15.60:0/4065839524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9640b3720 0x7ff964156b10 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c450 tx=0x7ff95800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:13.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff960ff9640 1 -- 172.21.15.60:0/4065839524 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff958017070 con 0x7ff9640b3720 2024-04-25T10:17:13.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff960ff9640 1 -- 172.21.15.60:0/4065839524 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff958005cf0 con 0x7ff9640b3720 2024-04-25T10:17:13.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff96a9f4640 1 -- 172.21.15.60:0/4065839524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9640b1130 con 0x7ff9640b3720 2024-04-25T10:17:13.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff960ff9640 1 -- 172.21.15.60:0/4065839524 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff958007290 con 0x7ff9640b3720 2024-04-25T10:17:13.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff96a9f4640 1 -- 172.21.15.60:0/4065839524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9640b16f0 con 0x7ff9640b3720 2024-04-25T10:17:13.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff96a9f4640 1 -- 172.21.15.60:0/4065839524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff964074770 con 0x7ff9640b3720 2024-04-25T10:17:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff960ff9640 1 -- 172.21.15.60:0/4065839524 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff95801a070 con 0x7ff9640b3720 2024-04-25T10:17:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff960ff9640 1 --2- 172.21.15.60:0/4065839524 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff950066f60 0x7ff950069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff960ff9640 1 -- 172.21.15.60:0/4065839524 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7ff9580845c0 con 0x7ff9640b3720 2024-04-25T10:17:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff960ff9640 1 --2- 172.21.15.60:0/4065839524 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x7ff95006cf60 0x7ff95006f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff960ff9640 1 -- 172.21.15.60:0/4065839524 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff95006fa30 con 0x7ff95006cf60 2024-04-25T10:17:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.285+0000 7ff960ff9640 1 -- 172.21.15.60:0/4065839524 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff958084a50 con 0x7ff9640b3720 2024-04-25T10:17:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.289+0000 7ff963fff640 1 --2- 172.21.15.60:0/4065839524 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff950066f60 0x7ff950069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.289+0000 7ff968f91640 1 --2- 172.21.15.60:0/4065839524 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x7ff95006cf60 0x7ff95006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.289+0000 7ff968f91640 1 --2- 172.21.15.60:0/4065839524 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x7ff95006cf60 0x7ff95006f380 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.289+0000 7ff963fff640 1 --2- 172.21.15.60:0/4065839524 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff950066f60 0x7ff950069420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff9540ae000 tx=0x7ff954008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:13.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.289+0000 7ff960ff9640 1 -- 172.21.15.60:0/4065839524 <== osd.4 v2:172.21.15.100:6808/3965541654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff95006fa30 con 0x7ff95006cf60 2024-04-25T10:17:13.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.301+0000 7ff96a9f4640 1 -- 172.21.15.60:0/4065839524 --> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff96406d820 con 0x7ff95006cf60 2024-04-25T10:17:13.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.301+0000 7ff960ff9640 1 -- 172.21.15.60:0/4065839524 <== osd.4 v2:172.21.15.100:6808/3965541654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff96406d820 con 0x7ff95006cf60 2024-04-25T10:17:13.309 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:17:13.309 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:17:13.309 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:17:13.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.305+0000 7ff96a9f4640 1 -- 172.21.15.60:0/4065839524 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x7ff95006cf60 msgr2=0x7ff95006f380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.305+0000 7ff96a9f4640 1 --2- 172.21.15.60:0/4065839524 >> [v2:172.21.15.100:6808/3965541654,v1:172.21.15.100:6809/3965541654] conn(0x7ff95006cf60 0x7ff95006f380 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.305+0000 7ff96a9f4640 1 -- 172.21.15.60:0/4065839524 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff950066f60 msgr2=0x7ff950069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.305+0000 7ff96a9f4640 1 --2- 172.21.15.60:0/4065839524 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7ff950066f60 0x7ff950069420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff9540ae000 tx=0x7ff954008040 comp rx=0 tx=0).stop 2024-04-25T10:17:13.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.305+0000 7ff96a9f4640 1 -- 172.21.15.60:0/4065839524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9640b3720 msgr2=0x7ff964156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.305+0000 7ff96a9f4640 1 --2- 172.21.15.60:0/4065839524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9640b3720 0x7ff964156b10 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c450 tx=0x7ff95800c910 comp rx=0 tx=0).stop 2024-04-25T10:17:13.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.305+0000 7ff968f91640 1 -- 172.21.15.60:0/4065839524 reap_dead start 2024-04-25T10:17:13.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.305+0000 7ff96a9f4640 1 -- 172.21.15.60:0/4065839524 shutdown_connections 2024-04-25T10:17:13.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.305+0000 7ff96a9f4640 1 -- 172.21.15.60:0/4065839524 >> 172.21.15.60:0/4065839524 conn(0x7ff964066fa0 msgr2=0x7ff964161d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:13.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.305+0000 7ff96a9f4640 1 -- 172.21.15.60:0/4065839524 shutdown_connections 2024-04-25T10:17:13.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.305+0000 7ff96a9f4640 1 -- 172.21.15.60:0/4065839524 wait complete. 2024-04-25T10:17:13.324 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T10:17:13.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.421+0000 7fa58f5bb640 1 Processor -- start 2024-04-25T10:17:13.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.421+0000 7fa58f5bb640 1 -- start start 2024-04-25T10:17:13.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58f5bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5880ce7a0 0x7fa5880ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58f5bb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa58815bc90 0x7fa5880b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58f5bb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5880cf0c0 con 0x7fa5880b4660 2024-04-25T10:17:13.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58f5bb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5880cc370 con 0x7fa58815bc90 2024-04-25T10:17:13.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58f5bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa588071ca0 con 0x7fa5880ce7a0 2024-04-25T10:17:13.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58cb56640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa58815bc90 0x7fa5880b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58cb56640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa58815bc90 0x7fa5880b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46566/0 (socket says 172.21.15.60:46566) 2024-04-25T10:17:13.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58cb56640 1 -- 172.21.15.60:0/2152925850 learned_addr learned my addr 172.21.15.60:0/2152925850 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:13.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58d357640 1 --2- 172.21.15.60:0/2152925850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5880ce7a0 0x7fa5880ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58cb56640 1 -- 172.21.15.60:0/2152925850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5880ce7a0 msgr2=0x7fa5880ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:13.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58cb56640 1 --2- 172.21.15.60:0/2152925850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5880ce7a0 0x7fa5880ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58cb56640 1 -- 172.21.15.60:0/2152925850 >> v1:172.21.15.100:6789/0 conn(0x7fa5880b4660 legacy=0x7fa5880b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:13.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58cb56640 1 -- 172.21.15.60:0/2152925850 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5880cca40 con 0x7fa58815bc90 2024-04-25T10:17:13.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58cb56640 1 --2- 172.21.15.60:0/2152925850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa58815bc90 0x7fa5880b25d0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7fa5840098a0 tx=0x7fa58401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8cc8bcd56c86a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:13.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa57ffff640 1 -- 172.21.15.60:0/2152925850 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840b0070 con 0x7fa58815bc90 2024-04-25T10:17:13.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa57ffff640 1 -- 172.21.15.60:0/2152925850 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5840ab600 con 0x7fa58815bc90 2024-04-25T10:17:13.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa57ffff640 1 -- 172.21.15.60:0/2152925850 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840ab9d0 con 0x7fa58815bc90 2024-04-25T10:17:13.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58f5bb640 1 -- 172.21.15.60:0/2152925850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa58815bc90 msgr2=0x7fa5880b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58f5bb640 1 --2- 172.21.15.60:0/2152925850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa58815bc90 0x7fa5880b25d0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7fa5840098a0 tx=0x7fa58401c500 comp rx=0 tx=0).stop 2024-04-25T10:17:13.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58f5bb640 1 -- 172.21.15.60:0/2152925850 shutdown_connections 2024-04-25T10:17:13.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58f5bb640 1 --2- 172.21.15.60:0/2152925850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa58815bc90 0x7fa5880b25d0 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58f5bb640 1 --2- 172.21.15.60:0/2152925850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5880ce7a0 0x7fa5880ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58f5bb640 1 -- 172.21.15.60:0/2152925850 >> 172.21.15.60:0/2152925850 conn(0x7fa5880b12d0 msgr2=0x7fa5880b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:13.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58f5bb640 1 -- 172.21.15.60:0/2152925850 shutdown_connections 2024-04-25T10:17:13.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.425+0000 7fa58f5bb640 1 -- 172.21.15.60:0/2152925850 wait complete. 2024-04-25T10:17:13.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58f5bb640 1 Processor -- start 2024-04-25T10:17:13.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58f5bb640 1 -- start start 2024-04-25T10:17:13.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58f5bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5880ce7a0 0x7fa588138fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58f5bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa58815bc90 0x7fa58814bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58f5bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa58815a650 0x7fa58815ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58f5bb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5880b4280 con 0x7fa58815bc90 2024-04-25T10:17:13.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58f5bb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5880b3db0 con 0x7fa5880ce7a0 2024-04-25T10:17:13.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58f5bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5880b4400 con 0x7fa58815a650 2024-04-25T10:17:13.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58cb56640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa58815bc90 0x7fa58814bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58cb56640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa58815bc90 0x7fa58814bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46572/0 (socket says 172.21.15.60:46572) 2024-04-25T10:17:13.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58cb56640 1 -- 172.21.15.60:0/2613886925 learned_addr learned my addr 172.21.15.60:0/2613886925 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:13.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58d357640 1 --2- 172.21.15.60:0/2613886925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5880ce7a0 0x7fa588138fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58db58640 1 --2- 172.21.15.60:0/2613886925 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa58815a650 0x7fa58815ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58cb56640 1 -- 172.21.15.60:0/2613886925 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa58815a650 msgr2=0x7fa58815ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58cb56640 1 --2- 172.21.15.60:0/2613886925 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa58815a650 0x7fa58815ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58cb56640 1 -- 172.21.15.60:0/2613886925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5880ce7a0 msgr2=0x7fa588138fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58cb56640 1 --2- 172.21.15.60:0/2613886925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5880ce7a0 0x7fa588138fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58cb56640 1 -- 172.21.15.60:0/2613886925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa588158ab0 con 0x7fa58815bc90 2024-04-25T10:17:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58db58640 1 --2- 172.21.15.60:0/2613886925 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa58815a650 0x7fa58815ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58cb56640 1 --2- 172.21.15.60:0/2613886925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa58815bc90 0x7fa58814bb50 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fa58401cda0 tx=0x7fa5840192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa57dffb640 1 -- 172.21.15.60:0/2613886925 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840ba070 con 0x7fa58815bc90 2024-04-25T10:17:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa57dffb640 1 -- 172.21.15.60:0/2613886925 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5840b4640 con 0x7fa58815bc90 2024-04-25T10:17:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58f5bb640 1 -- 172.21.15.60:0/2613886925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5881584f0 con 0x7fa58815bc90 2024-04-25T10:17:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa57dffb640 1 -- 172.21.15.60:0/2613886925 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840b0040 con 0x7fa58815bc90 2024-04-25T10:17:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58f5bb640 1 -- 172.21.15.60:0/2613886925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58814a4d0 con 0x7fa58815bc90 2024-04-25T10:17:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa58f5bb640 1 -- 172.21.15.60:0/2613886925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa588074770 con 0x7fa58815bc90 2024-04-25T10:17:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.429+0000 7fa57dffb640 1 -- 172.21.15.60:0/2613886925 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5840bd070 con 0x7fa58815bc90 2024-04-25T10:17:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.433+0000 7fa57dffb640 1 --2- 172.21.15.60:0/2613886925 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa56c066f90 0x7fa56c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.433+0000 7fa57dffb640 1 -- 172.21.15.60:0/2613886925 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fa58411df60 con 0x7fa58815bc90 2024-04-25T10:17:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.433+0000 7fa58d357640 1 --2- 172.21.15.60:0/2613886925 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa56c066f90 0x7fa56c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.433+0000 7fa58d357640 1 --2- 172.21.15.60:0/2613886925 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa56c066f90 0x7fa56c069450 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa5881f6570 tx=0x7fa578008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.437+0000 7fa57dffb640 1 -- 172.21.15.60:0/2613886925 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5840160f0 con 0x7fa58815bc90 2024-04-25T10:17:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.589+0000 7fa58f5bb640 1 -- 172.21.15.60:0/2613886925 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa588154660 con 0x7fa56c066f90 2024-04-25T10:17:13.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.589+0000 7fa57dffb640 1 -- 172.21.15.60:0/2613886925 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa588154660 con 0x7fa56c066f90 2024-04-25T10:17:13.594 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T10:17:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.589+0000 7fa58f5bb640 1 -- 172.21.15.60:0/2613886925 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa56c066f90 msgr2=0x7fa56c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.589+0000 7fa58f5bb640 1 --2- 172.21.15.60:0/2613886925 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa56c066f90 0x7fa56c069450 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa5881f6570 tx=0x7fa578008040 comp rx=0 tx=0).stop 2024-04-25T10:17:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.589+0000 7fa58f5bb640 1 -- 172.21.15.60:0/2613886925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa58815bc90 msgr2=0x7fa58814bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.589+0000 7fa58f5bb640 1 --2- 172.21.15.60:0/2613886925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa58815bc90 0x7fa58814bb50 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fa58401cda0 tx=0x7fa5840192a0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.589+0000 7fa58f5bb640 1 -- 172.21.15.60:0/2613886925 shutdown_connections 2024-04-25T10:17:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.589+0000 7fa58f5bb640 1 --2- 172.21.15.60:0/2613886925 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa56c066f90 0x7fa56c069450 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.589+0000 7fa58f5bb640 1 --2- 172.21.15.60:0/2613886925 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa58815a650 0x7fa58815ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.589+0000 7fa58f5bb640 1 --2- 172.21.15.60:0/2613886925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa58815bc90 0x7fa58814bb50 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.589+0000 7fa58f5bb640 1 --2- 172.21.15.60:0/2613886925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5880ce7a0 0x7fa588138fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.589+0000 7fa58f5bb640 1 -- 172.21.15.60:0/2613886925 >> 172.21.15.60:0/2613886925 conn(0x7fa5880b12d0 msgr2=0x7fa58815c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.589+0000 7fa58f5bb640 1 -- 172.21.15.60:0/2613886925 shutdown_connections 2024-04-25T10:17:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.589+0000 7fa58f5bb640 1 -- 172.21.15.60:0/2613886925 wait complete. 2024-04-25T10:17:13.611 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T10:17:13.612 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:17:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.721+0000 7fd198293640 1 Processor -- start 2024-04-25T10:17:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.721+0000 7fd198293640 1 -- start start 2024-04-25T10:17:13.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.721+0000 7fd198293640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd190144e40 0x7fd1900ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.721+0000 7fd198293640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1900cf0a0 0x7fd190071270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.721+0000 7fd198293640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1900717b0 0x7fd19015f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.721+0000 7fd198293640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1900b3060 con 0x7fd1900cf0a0 2024-04-25T10:17:13.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.721+0000 7fd198293640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1900cf810 con 0x7fd190144e40 2024-04-25T10:17:13.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.721+0000 7fd198293640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900cfaf0 con 0x7fd1900717b0 2024-04-25T10:17:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd19582e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1900cf0a0 0x7fd190071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd19602f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd190144e40 0x7fd1900ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd19582e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1900cf0a0 0x7fd190071270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46578/0 (socket says 172.21.15.60:46578) 2024-04-25T10:17:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd19582e640 1 -- 172.21.15.60:0/3736973264 learned_addr learned my addr 172.21.15.60:0/3736973264 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd196830640 1 --2- 172.21.15.60:0/3736973264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1900717b0 0x7fd19015f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd19582e640 1 -- 172.21.15.60:0/3736973264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1900717b0 msgr2=0x7fd19015f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd19582e640 1 --2- 172.21.15.60:0/3736973264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1900717b0 0x7fd19015f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd19582e640 1 -- 172.21.15.60:0/3736973264 >> v2:172.21.15.100:3300/0 conn(0x7fd190144e40 msgr2=0x7fd1900ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd19582e640 1 --2- 172.21.15.60:0/3736973264 >> v2:172.21.15.100:3300/0 conn(0x7fd190144e40 0x7fd1900ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd19582e640 1 -- 172.21.15.60:0/3736973264 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1900cfdd0 con 0x7fd1900cf0a0 2024-04-25T10:17:13.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd19582e640 1 --2- 172.21.15.60:0/3736973264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1900cf0a0 0x7fd190071270 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fd19006b8b0 tx=0x7fd18001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d9dde2bb875aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd19502d640 1 -- 172.21.15.60:0/3736973264 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1800b0070 con 0x7fd1900cf0a0 2024-04-25T10:17:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd19502d640 1 -- 172.21.15.60:0/3736973264 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd190145430 con 0x7fd1900cf0a0 2024-04-25T10:17:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd19502d640 1 -- 172.21.15.60:0/3736973264 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1800ab820 con 0x7fd1900cf0a0 2024-04-25T10:17:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd198293640 1 -- 172.21.15.60:0/3736973264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1900cf0a0 msgr2=0x7fd190071270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd198293640 1 --2- 172.21.15.60:0/3736973264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1900cf0a0 0x7fd190071270 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fd19006b8b0 tx=0x7fd18001c260 comp rx=0 tx=0).stop 2024-04-25T10:17:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd198293640 1 -- 172.21.15.60:0/3736973264 shutdown_connections 2024-04-25T10:17:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd198293640 1 --2- 172.21.15.60:0/3736973264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1900717b0 0x7fd19015f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd198293640 1 --2- 172.21.15.60:0/3736973264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1900cf0a0 0x7fd190071270 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd198293640 1 --2- 172.21.15.60:0/3736973264 >> v2:172.21.15.100:3300/0 conn(0x7fd190144e40 0x7fd1900ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd198293640 1 -- 172.21.15.60:0/3736973264 >> 172.21.15.60:0/3736973264 conn(0x7fd190066fa0 msgr2=0x7fd19006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd198293640 1 -- 172.21.15.60:0/3736973264 shutdown_connections 2024-04-25T10:17:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.725+0000 7fd198293640 1 -- 172.21.15.60:0/3736973264 wait complete. 2024-04-25T10:17:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd198293640 1 Processor -- start 2024-04-25T10:17:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd198293640 1 -- start start 2024-04-25T10:17:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd198293640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1900717b0 0x7fd1900b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd198293640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1900cf0a0 0x7fd19013bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd198293640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd190144e40 0x7fd19013c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd198293640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900b38a0 con 0x7fd1900717b0 2024-04-25T10:17:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd198293640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900b3720 con 0x7fd1900cf0a0 2024-04-25T10:17:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd198293640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900b3a20 con 0x7fd190144e40 2024-04-25T10:17:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd19602f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1900717b0 0x7fd1900b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd19602f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1900717b0 0x7fd1900b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46582/0 (socket says 172.21.15.60:46582) 2024-04-25T10:17:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd19602f640 1 -- 172.21.15.60:0/2697157181 learned_addr learned my addr 172.21.15.60:0/2697157181 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd196830640 1 --2- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd190144e40 0x7fd19013c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd19582e640 1 --2- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1900cf0a0 0x7fd19013bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd19602f640 1 -- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd190144e40 msgr2=0x7fd19013c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd19602f640 1 --2- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd190144e40 0x7fd19013c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd19602f640 1 -- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1900cf0a0 msgr2=0x7fd19013bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd19602f640 1 --2- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1900cf0a0 0x7fd19013bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd19602f640 1 -- 172.21.15.60:0/2697157181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19015e720 con 0x7fd1900717b0 2024-04-25T10:17:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd19582e640 1 --2- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1900cf0a0 0x7fd19013bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd19602f640 1 --2- 172.21.15.60:0/2697157181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1900717b0 0x7fd1900b1510 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00e870 tx=0x7fd18c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd186ffd640 1 -- 172.21.15.60:0/2697157181 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c017070 con 0x7fd1900717b0 2024-04-25T10:17:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd186ffd640 1 -- 172.21.15.60:0/2697157181 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c00c940 con 0x7fd1900717b0 2024-04-25T10:17:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd186ffd640 1 -- 172.21.15.60:0/2697157181 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c00ccf0 con 0x7fd1900717b0 2024-04-25T10:17:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd198293640 1 -- 172.21.15.60:0/2697157181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19015ea00 con 0x7fd1900717b0 2024-04-25T10:17:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd198293640 1 -- 172.21.15.60:0/2697157181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19015ece0 con 0x7fd1900717b0 2024-04-25T10:17:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd198293640 1 -- 172.21.15.60:0/2697157181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd190145430 con 0x7fd1900717b0 2024-04-25T10:17:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.729+0000 7fd186ffd640 1 -- 172.21.15.60:0/2697157181 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd18c01a070 con 0x7fd1900717b0 2024-04-25T10:17:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.733+0000 7fd186ffd640 1 --2- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd174066f60 0x7fd174069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.733+0000 7fd186ffd640 1 -- 172.21.15.60:0/2697157181 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fd18c085410 con 0x7fd1900717b0 2024-04-25T10:17:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.733+0000 7fd186ffd640 1 --2- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x7fd17406cf60 0x7fd17406f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.733+0000 7fd186ffd640 1 -- 172.21.15.60:0/2697157181 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd18c001d40 con 0x7fd17406cf60 2024-04-25T10:17:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.733+0000 7fd186ffd640 1 -- 172.21.15.60:0/2697157181 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd18c0858a0 con 0x7fd1900717b0 2024-04-25T10:17:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.733+0000 7fd19582e640 1 --2- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd174066f60 0x7fd174069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.733+0000 7fd196830640 1 --2- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x7fd17406cf60 0x7fd17406f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.733+0000 7fd19582e640 1 --2- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd174066f60 0x7fd174069420 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd1800ad000 tx=0x7fd180008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.733+0000 7fd196830640 1 --2- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x7fd17406cf60 0x7fd17406f380 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:13.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.733+0000 7fd186ffd640 1 -- 172.21.15.60:0/2697157181 <== osd.5 v2:172.21.15.100:6800/3542120824 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd18c001d40 con 0x7fd17406cf60 2024-04-25T10:17:13.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.749+0000 7fd198293640 1 -- 172.21.15.60:0/2697157181 --> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd190074770 con 0x7fd17406cf60 2024-04-25T10:17:13.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.749+0000 7fd186ffd640 1 -- 172.21.15.60:0/2697157181 <== osd.5 v2:172.21.15.100:6800/3542120824 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd190074770 con 0x7fd17406cf60 2024-04-25T10:17:13.754 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:17:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:17:13.754 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:17:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.749+0000 7fd198293640 1 -- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x7fd17406cf60 msgr2=0x7fd17406f380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.749+0000 7fd198293640 1 --2- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:6800/3542120824,v1:172.21.15.100:6801/3542120824] conn(0x7fd17406cf60 0x7fd17406f380 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.749+0000 7fd198293640 1 -- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd174066f60 msgr2=0x7fd174069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.749+0000 7fd198293640 1 --2- 172.21.15.60:0/2697157181 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd174066f60 0x7fd174069420 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd1800ad000 tx=0x7fd180008040 comp rx=0 tx=0).stop 2024-04-25T10:17:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.749+0000 7fd198293640 1 -- 172.21.15.60:0/2697157181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1900717b0 msgr2=0x7fd1900b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.749+0000 7fd198293640 1 --2- 172.21.15.60:0/2697157181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1900717b0 0x7fd1900b1510 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00e870 tx=0x7fd18c00ec70 comp rx=0 tx=0).stop 2024-04-25T10:17:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.749+0000 7fd196830640 1 -- 172.21.15.60:0/2697157181 reap_dead start 2024-04-25T10:17:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.749+0000 7fd198293640 1 -- 172.21.15.60:0/2697157181 shutdown_connections 2024-04-25T10:17:13.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.749+0000 7fd198293640 1 -- 172.21.15.60:0/2697157181 >> 172.21.15.60:0/2697157181 conn(0x7fd190066fa0 msgr2=0x7fd19015bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:13.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.749+0000 7fd198293640 1 -- 172.21.15.60:0/2697157181 shutdown_connections 2024-04-25T10:17:13.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.749+0000 7fd198293640 1 -- 172.21.15.60:0/2697157181 wait complete. 2024-04-25T10:17:13.769 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T10:17:13.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1cf121f640 1 Processor -- start 2024-04-25T10:17:13.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1cf121f640 1 -- start start 2024-04-25T10:17:13.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1cf121f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cec0d0a50 0x7f1cec0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1cf121f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1cec0ce2b0 0x7f1cec0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1cf121f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cec06d200 con 0x7f1cec15deb0 2024-04-25T10:17:13.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1cf121f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cec0cf2c0 con 0x7f1cec0ce2b0 2024-04-25T10:17:13.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1cf121f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec0cefe0 con 0x7f1cec0d0a50 2024-04-25T10:17:13.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1cea59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cec0d0a50 0x7f1cec0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1ceb59e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1cec0ce2b0 0x7f1cec0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1cea59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cec0d0a50 0x7f1cec0b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:32772/0 (socket says 172.21.15.60:32772) 2024-04-25T10:17:13.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1cead9d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1cec15deb0 0x7f1cec0d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:48084/0 (socket says 172.21.15.60:48084) 2024-04-25T10:17:13.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1ceb59e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1cec0ce2b0 0x7f1cec0ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33078/0 (socket says 172.21.15.60:33078) 2024-04-25T10:17:13.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1cea59c640 1 -- 172.21.15.60:0/357163570 learned_addr learned my addr 172.21.15.60:0/357163570 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:13.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1ceb59e640 1 -- 172.21.15.60:0/357163570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cec0d0a50 msgr2=0x7f1cec0b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:13.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1ceb59e640 1 --2- 172.21.15.60:0/357163570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cec0d0a50 0x7f1cec0b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1ceb59e640 1 -- 172.21.15.60:0/357163570 >> v1:172.21.15.100:6789/0 conn(0x7f1cec15deb0 legacy=0x7f1cec0d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.881+0000 7f1ceb59e640 1 -- 172.21.15.60:0/357163570 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec0d1d70 con 0x7f1cec0ce2b0 2024-04-25T10:17:13.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.885+0000 7f1ceb59e640 1 --2- 172.21.15.60:0/357163570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec0ce2b0 0x7f1cec0ce760 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40097c0 tx=0x7f1ce401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=136523c55c8abd23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:13.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.885+0000 7f1ce9d9b640 1 -- 172.21.15.60:0/357163570 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40b2070 con 0x7f1cec0ce2b0 2024-04-25T10:17:13.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.885+0000 7f1ce9d9b640 1 -- 172.21.15.60:0/357163570 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce40ad640 con 0x7f1cec0ce2b0 2024-04-25T10:17:13.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.885+0000 7f1ce9d9b640 1 -- 172.21.15.60:0/357163570 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40ada10 con 0x7f1cec0ce2b0 2024-04-25T10:17:13.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.885+0000 7f1cf121f640 1 -- 172.21.15.60:0/357163570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec0ce2b0 msgr2=0x7f1cec0ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.885+0000 7f1cf121f640 1 --2- 172.21.15.60:0/357163570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec0ce2b0 0x7f1cec0ce760 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40097c0 tx=0x7f1ce401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.885+0000 7f1cf121f640 1 -- 172.21.15.60:0/357163570 shutdown_connections 2024-04-25T10:17:13.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.885+0000 7f1cf121f640 1 --2- 172.21.15.60:0/357163570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec0ce2b0 0x7f1cec0ce760 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.885+0000 7f1cf121f640 1 --2- 172.21.15.60:0/357163570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cec0d0a50 0x7f1cec0b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.885+0000 7f1cf121f640 1 -- 172.21.15.60:0/357163570 >> 172.21.15.60:0/357163570 conn(0x7f1cec06c090 msgr2=0x7f1cec06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:13.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.885+0000 7f1cf121f640 1 -- 172.21.15.60:0/357163570 shutdown_connections 2024-04-25T10:17:13.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.885+0000 7f1cf121f640 1 -- 172.21.15.60:0/357163570 wait complete. 2024-04-25T10:17:13.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cf121f640 1 Processor -- start 2024-04-25T10:17:13.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cf121f640 1 -- start start 2024-04-25T10:17:13.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cf121f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cec0ce2b0 0x7f1cec0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cf121f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec0d0a50 0x7f1cec1606c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cf121f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cec15deb0 0x7f1cec14a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cf121f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec0d31e0 con 0x7f1cec15deb0 2024-04-25T10:17:13.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cf121f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec0d3060 con 0x7f1cec0d0a50 2024-04-25T10:17:13.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cf121f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec0d3360 con 0x7f1cec0ce2b0 2024-04-25T10:17:13.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1ceb59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cec15deb0 0x7f1cec14a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1ceb59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cec15deb0 0x7f1cec14a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46590/0 (socket says 172.21.15.60:46590) 2024-04-25T10:17:13.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1ceb59e640 1 -- 172.21.15.60:0/1734530689 learned_addr learned my addr 172.21.15.60:0/1734530689 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:13.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cead9d640 1 --2- 172.21.15.60:0/1734530689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cec0ce2b0 0x7f1cec0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cea59c640 1 --2- 172.21.15.60:0/1734530689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec0d0a50 0x7f1cec1606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1ceb59e640 1 -- 172.21.15.60:0/1734530689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cec0ce2b0 msgr2=0x7f1cec0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1ceb59e640 1 --2- 172.21.15.60:0/1734530689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cec0ce2b0 0x7f1cec0cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1ceb59e640 1 -- 172.21.15.60:0/1734530689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec0d0a50 msgr2=0x7f1cec1606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:13.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1ceb59e640 1 --2- 172.21.15.60:0/1734530689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec0d0a50 0x7f1cec1606c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:13.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1ceb59e640 1 -- 172.21.15.60:0/1734530689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec151450 con 0x7f1cec15deb0 2024-04-25T10:17:13.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1ceb59e640 1 --2- 172.21.15.60:0/1734530689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cec15deb0 0x7f1cec14a9a0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f1ce401a0d0 tx=0x7f1ce4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:13.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cd37fe640 1 -- 172.21.15.60:0/1734530689 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40b2040 con 0x7f1cec15deb0 2024-04-25T10:17:13.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cd37fe640 1 -- 172.21.15.60:0/1734530689 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce40bd070 con 0x7f1cec15deb0 2024-04-25T10:17:13.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cd37fe640 1 -- 172.21.15.60:0/1734530689 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40b76c0 con 0x7f1cec15deb0 2024-04-25T10:17:13.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cf121f640 1 -- 172.21.15.60:0/1734530689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cec151170 con 0x7f1cec15deb0 2024-04-25T10:17:13.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cf121f640 1 -- 172.21.15.60:0/1734530689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cec0cc8d0 con 0x7f1cec15deb0 2024-04-25T10:17:13.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cf121f640 1 -- 172.21.15.60:0/1734530689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cec074770 con 0x7f1cec15deb0 2024-04-25T10:17:13.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.889+0000 7f1cd37fe640 1 -- 172.21.15.60:0/1734530689 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ce40b5070 con 0x7f1cec15deb0 2024-04-25T10:17:13.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.893+0000 7f1cd37fe640 1 --2- 172.21.15.60:0/1734530689 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cd8066f90 0x7f1cd8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:13.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.893+0000 7f1cd37fe640 1 -- 172.21.15.60:0/1734530689 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f1ce411ec30 con 0x7f1cec15deb0 2024-04-25T10:17:13.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.893+0000 7f1cead9d640 1 --2- 172.21.15.60:0/1734530689 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cd8066f90 0x7f1cd8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:13.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.893+0000 7f1cead9d640 1 --2- 172.21.15.60:0/1734530689 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cd8066f90 0x7f1cd8069450 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc000f70 tx=0x7f1cdc007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:13.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:13.897+0000 7f1cd37fe640 1 -- 172.21.15.60:0/1734530689 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ce4016090 con 0x7f1cec15deb0 2024-04-25T10:17:14.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.053+0000 7f1cf121f640 1 -- 172.21.15.60:0/1734530689 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cec0c9be0 con 0x7f1cd8066f90 2024-04-25T10:17:14.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.053+0000 7f1cd37fe640 1 -- 172.21.15.60:0/1734530689 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1cec0c9be0 con 0x7f1cd8066f90 2024-04-25T10:17:14.060 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T10:17:14.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.057+0000 7f1cf121f640 1 -- 172.21.15.60:0/1734530689 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cd8066f90 msgr2=0x7f1cd8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.057+0000 7f1cf121f640 1 --2- 172.21.15.60:0/1734530689 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cd8066f90 0x7f1cd8069450 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc000f70 tx=0x7f1cdc007c90 comp rx=0 tx=0).stop 2024-04-25T10:17:14.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.057+0000 7f1cf121f640 1 -- 172.21.15.60:0/1734530689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cec15deb0 msgr2=0x7f1cec14a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.057+0000 7f1cf121f640 1 --2- 172.21.15.60:0/1734530689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cec15deb0 0x7f1cec14a9a0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f1ce401a0d0 tx=0x7f1ce4019240 comp rx=0 tx=0).stop 2024-04-25T10:17:14.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.057+0000 7f1cf121f640 1 -- 172.21.15.60:0/1734530689 shutdown_connections 2024-04-25T10:17:14.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.057+0000 7f1cf121f640 1 --2- 172.21.15.60:0/1734530689 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1cd8066f90 0x7f1cd8069450 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.057+0000 7f1cf121f640 1 --2- 172.21.15.60:0/1734530689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cec15deb0 0x7f1cec14a9a0 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.057+0000 7f1cf121f640 1 --2- 172.21.15.60:0/1734530689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cec0d0a50 0x7f1cec1606c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.057+0000 7f1cf121f640 1 --2- 172.21.15.60:0/1734530689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cec0ce2b0 0x7f1cec0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.057+0000 7f1cf121f640 1 -- 172.21.15.60:0/1734530689 >> 172.21.15.60:0/1734530689 conn(0x7f1cec06c090 msgr2=0x7f1cec14d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:14.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.057+0000 7f1cf121f640 1 -- 172.21.15.60:0/1734530689 shutdown_connections 2024-04-25T10:17:14.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.057+0000 7f1cf121f640 1 -- 172.21.15.60:0/1734530689 wait complete. 2024-04-25T10:17:14.078 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T10:17:14.079 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:17:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.193+0000 7fa55ce05640 1 Processor -- start 2024-04-25T10:17:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.193+0000 7fa55ce05640 1 -- start start 2024-04-25T10:17:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55ce05640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa55815a680 0x7fa5580b3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55ce05640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5580cfb90 0x7fa55806d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55ce05640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5580b2670 con 0x7fa5580cbc10 2024-04-25T10:17:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55ce05640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5580b2950 con 0x7fa55815a680 2024-04-25T10:17:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55ce05640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580b2c30 con 0x7fa5580cfb90 2024-04-25T10:17:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa555d9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa5580cbc10 0x7fa5580cf480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40918/0 (socket says 172.21.15.60:40918) 2024-04-25T10:17:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa555d9b640 1 -- 172.21.15.60:0/1276357612 learned_addr learned my addr 172.21.15.60:0/1276357612 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55559a640 1 -- 172.21.15.60:0/1276357612 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688535036 0 0) 0x7fa5580b2670 con 0x7fa5580cbc10 2024-04-25T10:17:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55559a640 1 -- 172.21.15.60:0/1276357612 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa540003530 con 0x7fa5580cbc10 2024-04-25T10:17:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa556d9d640 1 --2- 172.21.15.60:0/1276357612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5580cfb90 0x7fa55806d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55659c640 1 --2- 172.21.15.60:0/1276357612 >> v2:172.21.15.100:3300/0 conn(0x7fa55815a680 0x7fa5580b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55559a640 1 -- 172.21.15.60:0/1276357612 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3081594116 0 0) 0x7fa540003530 con 0x7fa5580cbc10 2024-04-25T10:17:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55559a640 1 -- 172.21.15.60:0/1276357612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5580cfb90 msgr2=0x7fa55806d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55559a640 1 --2- 172.21.15.60:0/1276357612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5580cfb90 0x7fa55806d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55559a640 1 -- 172.21.15.60:0/1276357612 >> v2:172.21.15.100:3300/0 conn(0x7fa55815a680 msgr2=0x7fa5580b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55559a640 1 --2- 172.21.15.60:0/1276357612 >> v2:172.21.15.100:3300/0 conn(0x7fa55815a680 0x7fa5580b3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55559a640 1 -- 172.21.15.60:0/1276357612 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5580b2f10 con 0x7fa5580cbc10 2024-04-25T10:17:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55559a640 1 -- 172.21.15.60:0/1276357612 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7fa54c001680 con 0x7fa5580cbc10 2024-04-25T10:17:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55559a640 1 -- 172.21.15.60:0/1276357612 >> v1:172.21.15.60:6789/0 conn(0x7fa5580cbc10 legacy=0x7fa5580cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55559a640 1 --2- 172.21.15.60:0/1276357612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5400039c0 0x7fa540003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55559a640 1 --2- 172.21.15.60:0/1276357612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa540004810 0x7fa540004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55559a640 1 --2- 172.21.15.60:0/1276357612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5400051d0 0x7fa54000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55559a640 1 -- 172.21.15.60:0/1276357612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580b2950 con 0x7fa5400039c0 2024-04-25T10:17:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55559a640 1 -- 172.21.15.60:0/1276357612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580b2c30 con 0x7fa540004810 2024-04-25T10:17:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55559a640 1 -- 172.21.15.60:0/1276357612 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54000e150 con 0x7fa5400051d0 2024-04-25T10:17:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa556d9d640 1 --2- 172.21.15.60:0/1276357612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5580cfb90 0x7fa55806d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55659c640 1 --2- 172.21.15.60:0/1276357612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5400039c0 0x7fa540003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa555d9b640 1 --2- 172.21.15.60:0/1276357612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa540004810 0x7fa540004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55659c640 1 -- 172.21.15.60:0/1276357612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5400051d0 msgr2=0x7fa54000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa556d9d640 1 --2- 172.21.15.60:0/1276357612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5400051d0 0x7fa54000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55659c640 1 --2- 172.21.15.60:0/1276357612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5400051d0 0x7fa54000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55659c640 1 -- 172.21.15.60:0/1276357612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa540004810 msgr2=0x7fa540004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:14.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55659c640 1 --2- 172.21.15.60:0/1276357612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa540004810 0x7fa540004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55659c640 1 -- 172.21.15.60:0/1276357612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa540003530 con 0x7fa5400039c0 2024-04-25T10:17:14.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa555d9b640 1 --2- 172.21.15.60:0/1276357612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa540004810 0x7fa540004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:14.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.197+0000 7fa55659c640 1 --2- 172.21.15.60:0/1276357612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5400039c0 0x7fa540003de0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fa550009fc0 tx=0x7fa55001cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edefc71e7a4a674 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:14.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55559a640 1 -- 172.21.15.60:0/1276357612 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5500b2070 con 0x7fa5400039c0 2024-04-25T10:17:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55559a640 1 -- 172.21.15.60:0/1276357612 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5500a5b40 con 0x7fa5400039c0 2024-04-25T10:17:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55559a640 1 -- 172.21.15.60:0/1276357612 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550009c10 con 0x7fa5400039c0 2024-04-25T10:17:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55ce05640 1 -- 172.21.15.60:0/1276357612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5400039c0 msgr2=0x7fa540003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55ce05640 1 --2- 172.21.15.60:0/1276357612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5400039c0 0x7fa540003de0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fa550009fc0 tx=0x7fa55001cc30 comp rx=0 tx=0).stop 2024-04-25T10:17:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa556d9d640 1 -- 172.21.15.60:0/1276357612 reap_dead start 2024-04-25T10:17:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55ce05640 1 -- 172.21.15.60:0/1276357612 shutdown_connections 2024-04-25T10:17:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55ce05640 1 -- 172.21.15.60:0/1276357612 >> 172.21.15.60:0/1276357612 conn(0x7fa558066fa0 msgr2=0x7fa55806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55ce05640 1 -- 172.21.15.60:0/1276357612 shutdown_connections 2024-04-25T10:17:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55ce05640 1 -- 172.21.15.60:0/1276357612 wait complete. 2024-04-25T10:17:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55ce05640 1 Processor -- start 2024-04-25T10:17:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55ce05640 1 -- start start 2024-04-25T10:17:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55ce05640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5400039c0 0x7fa55814fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55ce05640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5580c5430 0x7fa5580c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55ce05640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa558157a40 0x7fa558157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55ce05640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580cd670 con 0x7fa5400039c0 2024-04-25T10:17:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55ce05640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580cd4f0 con 0x7fa558157a40 2024-04-25T10:17:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55ce05640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa558071a80 con 0x7fa5580c5430 2024-04-25T10:17:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5400039c0 0x7fa55814fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5400039c0 0x7fa55814fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46610/0 (socket says 172.21.15.60:46610) 2024-04-25T10:17:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55659c640 1 -- 172.21.15.60:0/2876950919 learned_addr learned my addr 172.21.15.60:0/2876950919 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa555d9b640 1 --2- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5580c5430 0x7fa5580c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa556d9d640 1 --2- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa558157a40 0x7fa558157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55659c640 1 -- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5580c5430 msgr2=0x7fa5580c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55659c640 1 --2- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5580c5430 0x7fa5580c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55659c640 1 -- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa558157a40 msgr2=0x7fa558157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55659c640 1 --2- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa558157a40 0x7fa558157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55659c640 1 -- 172.21.15.60:0/2876950919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa558158700 con 0x7fa5400039c0 2024-04-25T10:17:14.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa556d9d640 1 --2- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa558157a40 0x7fa558157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:14.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55659c640 1 --2- 172.21.15.60:0/2876950919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5400039c0 0x7fa55814fbd0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fa5500987c0 tx=0x7fa550009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:14.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa546ffd640 1 -- 172.21.15.60:0/2876950919 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5500b2040 con 0x7fa5400039c0 2024-04-25T10:17:14.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa546ffd640 1 -- 172.21.15.60:0/2876950919 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5500bd070 con 0x7fa5400039c0 2024-04-25T10:17:14.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55ce05640 1 -- 172.21.15.60:0/2876950919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5581589e0 con 0x7fa5400039c0 2024-04-25T10:17:14.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa546ffd640 1 -- 172.21.15.60:0/2876950919 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5500adc20 con 0x7fa5400039c0 2024-04-25T10:17:14.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.201+0000 7fa55ce05640 1 -- 172.21.15.60:0/2876950919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa558158fa0 con 0x7fa5400039c0 2024-04-25T10:17:14.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.205+0000 7fa55ce05640 1 -- 172.21.15.60:0/2876950919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa558074770 con 0x7fa5400039c0 2024-04-25T10:17:14.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.205+0000 7fa546ffd640 1 -- 172.21.15.60:0/2876950919 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5500cb430 con 0x7fa5400039c0 2024-04-25T10:17:14.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.205+0000 7fa546ffd640 1 --2- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa538066f60 0x7fa538069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.205+0000 7fa546ffd640 1 -- 172.21.15.60:0/2876950919 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fa550120140 con 0x7fa5400039c0 2024-04-25T10:17:14.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.205+0000 7fa546ffd640 1 --2- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x7fa53806cf60 0x7fa53806f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.205+0000 7fa546ffd640 1 -- 172.21.15.60:0/2876950919 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5500c0e50 con 0x7fa53806cf60 2024-04-25T10:17:14.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.205+0000 7fa546ffd640 1 -- 172.21.15.60:0/2876950919 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5501205d0 con 0x7fa5400039c0 2024-04-25T10:17:14.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.205+0000 7fa555d9b640 1 --2- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa538066f60 0x7fa538069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.205+0000 7fa556d9d640 1 --2- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x7fa53806cf60 0x7fa53806f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.205+0000 7fa556d9d640 1 --2- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x7fa53806cf60 0x7fa53806f380 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:14.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.205+0000 7fa555d9b640 1 --2- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa538066f60 0x7fa538069420 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa54c001870 tx=0x7fa54c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:14.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.209+0000 7fa546ffd640 1 -- 172.21.15.60:0/2876950919 <== osd.6 v2:172.21.15.100:6824/4086390550 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa5500c0e50 con 0x7fa53806cf60 2024-04-25T10:17:14.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.221+0000 7fa55ce05640 1 -- 172.21.15.60:0/2876950919 --> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa55815dfc0 con 0x7fa53806cf60 2024-04-25T10:17:14.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.221+0000 7fa546ffd640 1 -- 172.21.15.60:0/2876950919 <== osd.6 v2:172.21.15.100:6824/4086390550 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa55815dfc0 con 0x7fa53806cf60 2024-04-25T10:17:14.226 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:17:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:17:14.227 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:17:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.221+0000 7fa55ce05640 1 -- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x7fa53806cf60 msgr2=0x7fa53806f380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.221+0000 7fa55ce05640 1 --2- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:6824/4086390550,v1:172.21.15.100:6825/4086390550] conn(0x7fa53806cf60 0x7fa53806f380 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.221+0000 7fa55ce05640 1 -- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa538066f60 msgr2=0x7fa538069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.221+0000 7fa55ce05640 1 --2- 172.21.15.60:0/2876950919 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fa538066f60 0x7fa538069420 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa54c001870 tx=0x7fa54c009040 comp rx=0 tx=0).stop 2024-04-25T10:17:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.221+0000 7fa55ce05640 1 -- 172.21.15.60:0/2876950919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5400039c0 msgr2=0x7fa55814fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.221+0000 7fa55ce05640 1 --2- 172.21.15.60:0/2876950919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5400039c0 0x7fa55814fbd0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fa5500987c0 tx=0x7fa550009750 comp rx=0 tx=0).stop 2024-04-25T10:17:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.221+0000 7fa556d9d640 1 -- 172.21.15.60:0/2876950919 reap_dead start 2024-04-25T10:17:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.221+0000 7fa55ce05640 1 -- 172.21.15.60:0/2876950919 shutdown_connections 2024-04-25T10:17:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.221+0000 7fa55ce05640 1 -- 172.21.15.60:0/2876950919 >> 172.21.15.60:0/2876950919 conn(0x7fa558066fa0 msgr2=0x7fa55806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.221+0000 7fa55ce05640 1 -- 172.21.15.60:0/2876950919 shutdown_connections 2024-04-25T10:17:14.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.221+0000 7fa55ce05640 1 -- 172.21.15.60:0/2876950919 wait complete. 2024-04-25T10:17:14.241 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T10:17:14.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.349+0000 7f18b7c0c640 1 Processor -- start 2024-04-25T10:17:14.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.349+0000 7f18b7c0c640 1 -- start start 2024-04-25T10:17:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b7c0c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f18b00cd390 0x7f18b009a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b7c0c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b00cc5d0 con 0x7f18b015f480 2024-04-25T10:17:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b7c0c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b00cc8b0 con 0x7f18b009a930 2024-04-25T10:17:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b7c0c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b00ccb90 con 0x7f18b00cd390 2024-04-25T10:17:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b59a8640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f18b015f480 0x7f18b015f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40932/0 (socket says 172.21.15.60:40932) 2024-04-25T10:17:14.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b59a8640 1 -- 172.21.15.60:0/2065273740 learned_addr learned my addr 172.21.15.60:0/2065273740 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:14.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b49a6640 1 -- 172.21.15.60:0/2065273740 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827423051 0 0) 0x7f18b00cc5d0 con 0x7f18b015f480 2024-04-25T10:17:14.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b49a6640 1 -- 172.21.15.60:0/2065273740 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f189c003530 con 0x7f18b015f480 2024-04-25T10:17:14.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b51a7640 1 --2- 172.21.15.60:0/2065273740 >> v2:172.21.15.100:3300/0 conn(0x7f18b00cd390 0x7f18b009a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b49a6640 1 -- 172.21.15.60:0/2065273740 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2062176215 0 0) 0x7f189c003530 con 0x7f18b015f480 2024-04-25T10:17:14.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b49a6640 1 -- 172.21.15.60:0/2065273740 >> v2:172.21.15.100:3300/0 conn(0x7f18b00cd390 msgr2=0x7f18b009a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:14.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b49a6640 1 --2- 172.21.15.60:0/2065273740 >> v2:172.21.15.100:3300/0 conn(0x7f18b00cd390 0x7f18b009a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b49a6640 1 -- 172.21.15.60:0/2065273740 >> v1:172.21.15.100:6789/0 conn(0x7f18b009a930 legacy=0x7f18b00cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b49a6640 1 -- 172.21.15.60:0/2065273740 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b00cce70 con 0x7f18b015f480 2024-04-25T10:17:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b49a6640 1 -- 172.21.15.60:0/2065273740 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f18ac001680 con 0x7f18b015f480 2024-04-25T10:17:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b49a6640 1 -- 172.21.15.60:0/2065273740 >> v1:172.21.15.60:6789/0 conn(0x7f18b015f480 legacy=0x7f18b015f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b49a6640 1 --2- 172.21.15.60:0/2065273740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c003a10 0x7f189c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b49a6640 1 --2- 172.21.15.60:0/2065273740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189c004810 0x7f189c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b49a6640 1 --2- 172.21.15.60:0/2065273740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f189c0051d0 0x7f189c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b49a6640 1 -- 172.21.15.60:0/2065273740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00ccb90 con 0x7f189c0051d0 2024-04-25T10:17:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b49a6640 1 -- 172.21.15.60:0/2065273740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c00c0d0 con 0x7f189c003a10 2024-04-25T10:17:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b49a6640 1 -- 172.21.15.60:0/2065273740 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c00c2d0 con 0x7f189c004810 2024-04-25T10:17:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b61a9640 1 --2- 172.21.15.60:0/2065273740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f189c0051d0 0x7f189c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b59a8640 1 --2- 172.21.15.60:0/2065273740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c003a10 0x7f189c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b51a7640 1 --2- 172.21.15.60:0/2065273740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189c004810 0x7f189c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b61a9640 1 -- 172.21.15.60:0/2065273740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189c004810 msgr2=0x7f189c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b61a9640 1 --2- 172.21.15.60:0/2065273740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189c004810 0x7f189c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b61a9640 1 -- 172.21.15.60:0/2065273740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c003a10 msgr2=0x7f189c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b61a9640 1 --2- 172.21.15.60:0/2065273740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c003a10 0x7f189c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.353+0000 7f18b61a9640 1 -- 172.21.15.60:0/2065273740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c003530 con 0x7f189c0051d0 2024-04-25T10:17:14.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b61a9640 1 -- 172.21.15.60:0/2065273740 reap_dead start 2024-04-25T10:17:14.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b61a9640 1 --2- 172.21.15.60:0/2065273740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f189c0051d0 0x7f189c00b9d0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f18a8001e40 tx=0x7f18a801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f57f7c6933ae384 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b49a6640 1 -- 172.21.15.60:0/2065273740 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8001a90 con 0x7f189c0051d0 2024-04-25T10:17:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b49a6640 1 -- 172.21.15.60:0/2065273740 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a8003b80 con 0x7f189c0051d0 2024-04-25T10:17:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b49a6640 1 -- 172.21.15.60:0/2065273740 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8005020 con 0x7f189c0051d0 2024-04-25T10:17:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b7c0c640 1 -- 172.21.15.60:0/2065273740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f189c0051d0 msgr2=0x7f189c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b7c0c640 1 --2- 172.21.15.60:0/2065273740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f189c0051d0 0x7f189c00b9d0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f18a8001e40 tx=0x7f18a801c760 comp rx=0 tx=0).stop 2024-04-25T10:17:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b7c0c640 1 -- 172.21.15.60:0/2065273740 shutdown_connections 2024-04-25T10:17:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b7c0c640 1 --2- 172.21.15.60:0/2065273740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f189c0051d0 0x7f189c00b9d0 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b7c0c640 1 -- 172.21.15.60:0/2065273740 >> 172.21.15.60:0/2065273740 conn(0x7f18b00b32a0 msgr2=0x7f18b00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b7c0c640 1 -- 172.21.15.60:0/2065273740 shutdown_connections 2024-04-25T10:17:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b7c0c640 1 -- 172.21.15.60:0/2065273740 wait complete. 2024-04-25T10:17:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b7c0c640 1 Processor -- start 2024-04-25T10:17:14.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b7c0c640 1 -- start start 2024-04-25T10:17:14.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b7c0c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189c0051d0 0x7f18b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b7c0c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b00c8880 0x7f18b00c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b7c0c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18b01f6fb0 0x7f18b01f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b7c0c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b01602f0 con 0x7f18b01f6fb0 2024-04-25T10:17:14.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b7c0c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b0160170 con 0x7f18b00c8880 2024-04-25T10:17:14.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b7c0c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b009a3f0 con 0x7f189c0051d0 2024-04-25T10:17:14.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b61a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18b01f6fb0 0x7f18b01f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b61a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18b01f6fb0 0x7f18b01f7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46630/0 (socket says 172.21.15.60:46630) 2024-04-25T10:17:14.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b61a9640 1 -- 172.21.15.60:0/381367461 learned_addr learned my addr 172.21.15.60:0/381367461 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:14.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b59a8640 1 --2- 172.21.15.60:0/381367461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189c0051d0 0x7f18b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b51a7640 1 --2- 172.21.15.60:0/381367461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b00c8880 0x7f18b00c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b61a9640 1 -- 172.21.15.60:0/381367461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189c0051d0 msgr2=0x7f18b006d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b61a9640 1 --2- 172.21.15.60:0/381367461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189c0051d0 0x7f18b006d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b61a9640 1 -- 172.21.15.60:0/381367461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b00c8880 msgr2=0x7f18b00c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b61a9640 1 --2- 172.21.15.60:0/381367461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b00c8880 0x7f18b00c7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b61a9640 1 -- 172.21.15.60:0/381367461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b00d0410 con 0x7f18b01f6fb0 2024-04-25T10:17:14.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.357+0000 7f18b61a9640 1 --2- 172.21.15.60:0/381367461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18b01f6fb0 0x7f18b01f7430 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f18a801d000 tx=0x7f189c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:14.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.361+0000 7f18a67fc640 1 -- 172.21.15.60:0/381367461 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a80b6070 con 0x7f18b01f6fb0 2024-04-25T10:17:14.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.361+0000 7f18a67fc640 1 -- 172.21.15.60:0/381367461 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a80ad070 con 0x7f18b01f6fb0 2024-04-25T10:17:14.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.361+0000 7f18b7c0c640 1 -- 172.21.15.60:0/381367461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b006d6d0 con 0x7f18b01f6fb0 2024-04-25T10:17:14.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.361+0000 7f18a67fc640 1 -- 172.21.15.60:0/381367461 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8001a90 con 0x7f18b01f6fb0 2024-04-25T10:17:14.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.361+0000 7f18b7c0c640 1 -- 172.21.15.60:0/381367461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b006dc50 con 0x7f18b01f6fb0 2024-04-25T10:17:14.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.361+0000 7f18b7c0c640 1 -- 172.21.15.60:0/381367461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b0074770 con 0x7f18b01f6fb0 2024-04-25T10:17:14.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.361+0000 7f18a67fc640 1 -- 172.21.15.60:0/381367461 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18a80b9070 con 0x7f18b01f6fb0 2024-04-25T10:17:14.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.361+0000 7f18a67fc640 1 --2- 172.21.15.60:0/381367461 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1894066f60 0x7f1894069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.361+0000 7f18a67fc640 1 -- 172.21.15.60:0/381367461 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f18a811e180 con 0x7f18b01f6fb0 2024-04-25T10:17:14.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.361+0000 7f18b59a8640 1 --2- 172.21.15.60:0/381367461 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1894066f60 0x7f1894069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.361+0000 7f18b59a8640 1 --2- 172.21.15.60:0/381367461 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1894066f60 0x7f1894069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f18ac001060 tx=0x7f18ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:14.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.365+0000 7f18a67fc640 1 -- 172.21.15.60:0/381367461 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18a8016470 con 0x7f18b01f6fb0 2024-04-25T10:17:14.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.521+0000 7f18b7c0c640 1 -- 172.21.15.60:0/381367461 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f18b00b7730 con 0x7f1894066f60 2024-04-25T10:17:14.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.521+0000 7f18a67fc640 1 -- 172.21.15.60:0/381367461 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f18b00b7730 con 0x7f1894066f60 2024-04-25T10:17:14.527 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T10:17:14.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.525+0000 7f18b7c0c640 1 -- 172.21.15.60:0/381367461 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1894066f60 msgr2=0x7f1894069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.525+0000 7f18b7c0c640 1 --2- 172.21.15.60:0/381367461 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1894066f60 0x7f1894069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f18ac001060 tx=0x7f18ac009040 comp rx=0 tx=0).stop 2024-04-25T10:17:14.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.525+0000 7f18b7c0c640 1 -- 172.21.15.60:0/381367461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18b01f6fb0 msgr2=0x7f18b01f7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.525+0000 7f18b7c0c640 1 --2- 172.21.15.60:0/381367461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18b01f6fb0 0x7f18b01f7430 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f18a801d000 tx=0x7f189c005840 comp rx=0 tx=0).stop 2024-04-25T10:17:14.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.525+0000 7f18b7c0c640 1 -- 172.21.15.60:0/381367461 shutdown_connections 2024-04-25T10:17:14.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.525+0000 7f18b7c0c640 1 --2- 172.21.15.60:0/381367461 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f1894066f60 0x7f1894069420 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.525+0000 7f18b7c0c640 1 --2- 172.21.15.60:0/381367461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18b01f6fb0 0x7f18b01f7430 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.525+0000 7f18b7c0c640 1 --2- 172.21.15.60:0/381367461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18b00c8880 0x7f18b00c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.525+0000 7f18b7c0c640 1 --2- 172.21.15.60:0/381367461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189c0051d0 0x7f18b006d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.525+0000 7f18b7c0c640 1 -- 172.21.15.60:0/381367461 >> 172.21.15.60:0/381367461 conn(0x7f18b00b32a0 msgr2=0x7f18b0159670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:14.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.525+0000 7f18b7c0c640 1 -- 172.21.15.60:0/381367461 shutdown_connections 2024-04-25T10:17:14.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.525+0000 7f18b7c0c640 1 -- 172.21.15.60:0/381367461 wait complete. 2024-04-25T10:17:14.545 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T10:17:14.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:17:14.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.661+0000 7f3fd963c640 1 Processor -- start 2024-04-25T10:17:14.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.661+0000 7f3fd963c640 1 -- start start 2024-04-25T10:17:14.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.661+0000 7f3fd963c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3fd4144e40 0x7f3fd40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.661+0000 7f3fd963c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fd4071980 0x7f3fd40b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.661+0000 7f3fd963c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd40cf810 con 0x7f3fd40cf0a0 2024-04-25T10:17:14.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.661+0000 7f3fd963c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd40b3060 con 0x7f3fd4144e40 2024-04-25T10:17:14.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.661+0000 7f3fd963c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd40cfaf0 con 0x7f3fd4071980 2024-04-25T10:17:14.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.661+0000 7f3fd2ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3fd4144e40 0x7f3fd40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.661+0000 7f3fd27fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3fd40cf0a0 0x7f3fd4071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40942/0 (socket says 172.21.15.60:40942) 2024-04-25T10:17:14.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.661+0000 7f3fd27fc640 1 -- 172.21.15.60:0/175315243 learned_addr learned my addr 172.21.15.60:0/175315243 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:14.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.661+0000 7f3fd37fe640 1 --2- 172.21.15.60:0/175315243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fd4071980 0x7f3fd40b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.661+0000 7f3fd2ffd640 1 -- 172.21.15.60:0/175315243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fd4071980 msgr2=0x7f3fd40b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:14.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.661+0000 7f3fd2ffd640 1 --2- 172.21.15.60:0/175315243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fd4071980 0x7f3fd40b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.661+0000 7f3fd2ffd640 1 -- 172.21.15.60:0/175315243 >> v1:172.21.15.60:6789/0 conn(0x7f3fd40cf0a0 legacy=0x7f3fd4071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.661+0000 7f3fd2ffd640 1 -- 172.21.15.60:0/175315243 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd40cfdd0 con 0x7f3fd4144e40 2024-04-25T10:17:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.665+0000 7f3fd2ffd640 1 --2- 172.21.15.60:0/175315243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4144e40 0x7f3fd40ccc60 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40097c0 tx=0x7f3fc401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99943a03be8b3a3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.665+0000 7f3fd1ffb640 1 -- 172.21.15.60:0/175315243 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40ab660 con 0x7f3fd4144e40 2024-04-25T10:17:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.665+0000 7f3fd1ffb640 1 -- 172.21.15.60:0/175315243 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc40b0070 con 0x7f3fd4144e40 2024-04-25T10:17:14.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.665+0000 7f3fd1ffb640 1 -- 172.21.15.60:0/175315243 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40aba20 con 0x7f3fd4144e40 2024-04-25T10:17:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.665+0000 7f3fd963c640 1 -- 172.21.15.60:0/175315243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4144e40 msgr2=0x7f3fd40ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.665+0000 7f3fd963c640 1 --2- 172.21.15.60:0/175315243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4144e40 0x7f3fd40ccc60 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40097c0 tx=0x7f3fc401c440 comp rx=0 tx=0).stop 2024-04-25T10:17:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.665+0000 7f3fd963c640 1 -- 172.21.15.60:0/175315243 shutdown_connections 2024-04-25T10:17:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.665+0000 7f3fd963c640 1 --2- 172.21.15.60:0/175315243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fd4071980 0x7f3fd40b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.665+0000 7f3fd963c640 1 --2- 172.21.15.60:0/175315243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4144e40 0x7f3fd40ccc60 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.665+0000 7f3fd963c640 1 -- 172.21.15.60:0/175315243 >> 172.21.15.60:0/175315243 conn(0x7f3fd4066fa0 msgr2=0x7f3fd406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.665+0000 7f3fd963c640 1 -- 172.21.15.60:0/175315243 shutdown_connections 2024-04-25T10:17:14.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.665+0000 7f3fd963c640 1 -- 172.21.15.60:0/175315243 wait complete. 2024-04-25T10:17:14.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd963c640 1 Processor -- start 2024-04-25T10:17:14.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd963c640 1 -- start start 2024-04-25T10:17:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd963c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4071980 0x7f3fd406de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd963c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fd40cf0a0 0x7f3fd415a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd963c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fd4144e40 0x7f3fd40ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd963c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd40b4610 con 0x7f3fd4071980 2024-04-25T10:17:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd963c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd40b4490 con 0x7f3fd40cf0a0 2024-04-25T10:17:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd963c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd40b4790 con 0x7f3fd4144e40 2024-04-25T10:17:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4071980 0x7f3fd406de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4071980 0x7f3fd406de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46646/0 (socket says 172.21.15.60:46646) 2024-04-25T10:17:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd2ffd640 1 -- 172.21.15.60:0/4116962853 learned_addr learned my addr 172.21.15.60:0/4116962853 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd2ffd640 1 -- 172.21.15.60:0/4116962853 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fd4144e40 msgr2=0x7f3fd40ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd2ffd640 1 --2- 172.21.15.60:0/4116962853 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fd4144e40 0x7f3fd40ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd27fc640 1 --2- 172.21.15.60:0/4116962853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fd40cf0a0 0x7f3fd415a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd2ffd640 1 -- 172.21.15.60:0/4116962853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fd40cf0a0 msgr2=0x7f3fd415a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd2ffd640 1 --2- 172.21.15.60:0/4116962853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fd40cf0a0 0x7f3fd415a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd2ffd640 1 -- 172.21.15.60:0/4116962853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd406e4f0 con 0x7f3fd4071980 2024-04-25T10:17:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd2ffd640 1 --2- 172.21.15.60:0/4116962853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4071980 0x7f3fd406de10 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f3fc401a0d0 tx=0x7f3fc401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fbb7fe640 1 -- 172.21.15.60:0/4116962853 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40bf610 con 0x7f3fd4071980 2024-04-25T10:17:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fbb7fe640 1 -- 172.21.15.60:0/4116962853 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd4145430 con 0x7f3fd4071980 2024-04-25T10:17:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd963c640 1 -- 172.21.15.60:0/4116962853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd40c9360 con 0x7f3fd4071980 2024-04-25T10:17:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fbb7fe640 1 -- 172.21.15.60:0/4116962853 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40bfa10 con 0x7f3fd4071980 2024-04-25T10:17:14.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd963c640 1 -- 172.21.15.60:0/4116962853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd40c9920 con 0x7f3fd4071980 2024-04-25T10:17:14.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fd963c640 1 -- 172.21.15.60:0/4116962853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3fd415a920 con 0x7f3fd4071980 2024-04-25T10:17:14.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fbb7fe640 1 -- 172.21.15.60:0/4116962853 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3fc40bd020 con 0x7f3fd4071980 2024-04-25T10:17:14.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fbb7fe640 1 --2- 172.21.15.60:0/4116962853 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3fc0066f60 0x7f3fc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fbb7fe640 1 -- 172.21.15.60:0/4116962853 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f3fc411f980 con 0x7f3fd4071980 2024-04-25T10:17:14.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fbb7fe640 1 --2- 172.21.15.60:0/4116962853 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x7f3fc006cf60 0x7f3fc006f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fbb7fe640 1 -- 172.21.15.60:0/4116962853 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3fc40b7e50 con 0x7f3fc006cf60 2024-04-25T10:17:14.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.669+0000 7f3fbb7fe640 1 -- 172.21.15.60:0/4116962853 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fc411fe10 con 0x7f3fd4071980 2024-04-25T10:17:14.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.673+0000 7f3fd27fc640 1 --2- 172.21.15.60:0/4116962853 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3fc0066f60 0x7f3fc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.673+0000 7f3fd37fe640 1 --2- 172.21.15.60:0/4116962853 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x7f3fc006cf60 0x7f3fc006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.673+0000 7f3fd37fe640 1 --2- 172.21.15.60:0/4116962853 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x7f3fc006cf60 0x7f3fc006f380 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:14.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.673+0000 7f3fd27fc640 1 --2- 172.21.15.60:0/4116962853 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3fc0066f60 0x7f3fc0069420 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40cbab0 tx=0x7f3fc8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:14.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.673+0000 7f3fbb7fe640 1 -- 172.21.15.60:0/4116962853 <== osd.7 v2:172.21.15.100:6816/2771485684 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3fc40b7e50 con 0x7f3fc006cf60 2024-04-25T10:17:14.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.689+0000 7f3fd963c640 1 -- 172.21.15.60:0/4116962853 --> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3fd4074770 con 0x7f3fc006cf60 2024-04-25T10:17:14.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.689+0000 7f3fbb7fe640 1 -- 172.21.15.60:0/4116962853 <== osd.7 v2:172.21.15.100:6816/2771485684 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3fd4074770 con 0x7f3fc006cf60 2024-04-25T10:17:14.694 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T10:17:14.694 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:17:14.694 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T10:17:14.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.689+0000 7f3fd963c640 1 -- 172.21.15.60:0/4116962853 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x7f3fc006cf60 msgr2=0x7f3fc006f380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.689+0000 7f3fd963c640 1 --2- 172.21.15.60:0/4116962853 >> [v2:172.21.15.100:6816/2771485684,v1:172.21.15.100:6817/2771485684] conn(0x7f3fc006cf60 0x7f3fc006f380 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.689+0000 7f3fd963c640 1 -- 172.21.15.60:0/4116962853 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3fc0066f60 msgr2=0x7f3fc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.689+0000 7f3fd963c640 1 --2- 172.21.15.60:0/4116962853 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f3fc0066f60 0x7f3fc0069420 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40cbab0 tx=0x7f3fc8009040 comp rx=0 tx=0).stop 2024-04-25T10:17:14.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.689+0000 7f3fd963c640 1 -- 172.21.15.60:0/4116962853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4071980 msgr2=0x7f3fd406de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.689+0000 7f3fd963c640 1 --2- 172.21.15.60:0/4116962853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fd4071980 0x7f3fd406de10 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f3fc401a0d0 tx=0x7f3fc401ace0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.689+0000 7f3fd37fe640 1 -- 172.21.15.60:0/4116962853 reap_dead start 2024-04-25T10:17:14.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.689+0000 7f3fd963c640 1 -- 172.21.15.60:0/4116962853 shutdown_connections 2024-04-25T10:17:14.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.689+0000 7f3fd963c640 1 -- 172.21.15.60:0/4116962853 >> 172.21.15.60:0/4116962853 conn(0x7f3fd4066fa0 msgr2=0x7f3fd4143de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:14.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.689+0000 7f3fd963c640 1 -- 172.21.15.60:0/4116962853 shutdown_connections 2024-04-25T10:17:14.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.689+0000 7f3fd963c640 1 -- 172.21.15.60:0/4116962853 wait complete. 2024-04-25T10:17:14.710 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T10:17:14.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99ce4f8640 1 Processor -- start 2024-04-25T10:17:14.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99ce4f8640 1 -- start start 2024-04-25T10:17:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99ce4f8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99c80d0a50 0x7f99c80b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99ce4f8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c806d200 con 0x7f99c815deb0 2024-04-25T10:17:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99ce4f8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c80cefe0 con 0x7f99c80ce2b0 2024-04-25T10:17:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99ce4f8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80cf2c0 con 0x7f99c80d0a50 2024-04-25T10:17:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f99c815deb0 0x7f99c80d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40948/0 (socket says 172.21.15.60:40948) 2024-04-25T10:17:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c7fff640 1 -- 172.21.15.60:0/1043378087 learned_addr learned my addr 172.21.15.60:0/1043378087 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c77fe640 1 --2- 172.21.15.60:0/1043378087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99c80d0a50 0x7f99c80b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c6ffd640 1 -- 172.21.15.60:0/1043378087 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353112468 0 0) 0x7f99c806d200 con 0x7f99c815deb0 2024-04-25T10:17:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c6ffd640 1 -- 172.21.15.60:0/1043378087 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99b0003530 con 0x7f99c815deb0 2024-04-25T10:17:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c6ffd640 1 -- 172.21.15.60:0/1043378087 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3522669693 0 0) 0x7f99b0003530 con 0x7f99c815deb0 2024-04-25T10:17:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c6ffd640 1 -- 172.21.15.60:0/1043378087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99c80d0a50 msgr2=0x7f99c80b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c6ffd640 1 --2- 172.21.15.60:0/1043378087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99c80d0a50 0x7f99c80b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c6ffd640 1 -- 172.21.15.60:0/1043378087 >> v1:172.21.15.100:6789/0 conn(0x7f99c80ce2b0 legacy=0x7f99c80ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c6ffd640 1 -- 172.21.15.60:0/1043378087 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c80d1d70 con 0x7f99c815deb0 2024-04-25T10:17:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c6ffd640 1 -- 172.21.15.60:0/1043378087 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 796804655 0 0) 0x7f99b8001a00 con 0x7f99c815deb0 2024-04-25T10:17:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c6ffd640 1 -- 172.21.15.60:0/1043378087 >> v1:172.21.15.60:6789/0 conn(0x7f99c815deb0 legacy=0x7f99c80d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c6ffd640 1 --2- 172.21.15.60:0/1043378087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99b00039f0 0x7f99b0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c6ffd640 1 --2- 172.21.15.60:0/1043378087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99c80d0a50 0x7f99b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c6ffd640 1 --2- 172.21.15.60:0/1043378087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99b0004d50 0x7f99b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c6ffd640 1 -- 172.21.15.60:0/1043378087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80cf2c0 con 0x7f99b00039f0 2024-04-25T10:17:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c6ffd640 1 -- 172.21.15.60:0/1043378087 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b000ba80 con 0x7f99b0004d50 2024-04-25T10:17:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c6ffd640 1 -- 172.21.15.60:0/1043378087 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b000bc80 con 0x7f99c80d0a50 2024-04-25T10:17:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c7fff640 1 --2- 172.21.15.60:0/1043378087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99b00039f0 0x7f99b0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c77fe640 1 --2- 172.21.15.60:0/1043378087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99c80d0a50 0x7f99b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99cca95640 1 --2- 172.21.15.60:0/1043378087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99b0004d50 0x7f99b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c7fff640 1 -- 172.21.15.60:0/1043378087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99c80d0a50 msgr2=0x7f99b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c7fff640 1 --2- 172.21.15.60:0/1043378087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99c80d0a50 0x7f99b0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c7fff640 1 -- 172.21.15.60:0/1043378087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99b0004d50 msgr2=0x7f99b00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c7fff640 1 --2- 172.21.15.60:0/1043378087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99b0004d50 0x7f99b00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c7fff640 1 -- 172.21.15.60:0/1043378087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b0003530 con 0x7f99b00039f0 2024-04-25T10:17:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99c77fe640 1 --2- 172.21.15.60:0/1043378087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99c80d0a50 0x7f99b0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.821+0000 7f99cca95640 1 --2- 172.21.15.60:0/1043378087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99b0004d50 0x7f99b00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.825+0000 7f99c7fff640 1 --2- 172.21.15.60:0/1043378087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99b00039f0 0x7f99b0003e10 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f99b8002240 tx=0x7f99b801d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e74a212fca7f9fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.825+0000 7f99c6ffd640 1 -- 172.21.15.60:0/1043378087 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b800a4a0 con 0x7f99b00039f0 2024-04-25T10:17:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.825+0000 7f99c6ffd640 1 -- 172.21.15.60:0/1043378087 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b80ad650 con 0x7f99b00039f0 2024-04-25T10:17:14.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.825+0000 7f99c6ffd640 1 -- 172.21.15.60:0/1043378087 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b80ada40 con 0x7f99b00039f0 2024-04-25T10:17:14.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.825+0000 7f99ce4f8640 1 -- 172.21.15.60:0/1043378087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99b00039f0 msgr2=0x7f99b0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.825+0000 7f99ce4f8640 1 --2- 172.21.15.60:0/1043378087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99b00039f0 0x7f99b0003e10 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f99b8002240 tx=0x7f99b801d360 comp rx=0 tx=0).stop 2024-04-25T10:17:14.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.825+0000 7f99cca95640 1 -- 172.21.15.60:0/1043378087 reap_dead start 2024-04-25T10:17:14.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.825+0000 7f99ce4f8640 1 -- 172.21.15.60:0/1043378087 shutdown_connections 2024-04-25T10:17:14.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.825+0000 7f99ce4f8640 1 -- 172.21.15.60:0/1043378087 >> 172.21.15.60:0/1043378087 conn(0x7f99c806c090 msgr2=0x7f99c806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:14.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.825+0000 7f99ce4f8640 1 -- 172.21.15.60:0/1043378087 shutdown_connections 2024-04-25T10:17:14.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.825+0000 7f99ce4f8640 1 -- 172.21.15.60:0/1043378087 wait complete. 2024-04-25T10:17:14.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.825+0000 7f99ce4f8640 1 Processor -- start 2024-04-25T10:17:14.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.825+0000 7f99ce4f8640 1 -- start start 2024-04-25T10:17:14.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99ce4f8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99c81498d0 0x7f99c80c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99ce4f8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99c8149cb0 0x7f99c80c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99ce4f8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99c80b8f80 0x7f99c80c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99ce4f8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c81697c0 con 0x7f99c81498d0 2024-04-25T10:17:14.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99ce4f8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c8169640 con 0x7f99c8149cb0 2024-04-25T10:17:14.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99ce4f8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c8169940 con 0x7f99c80b8f80 2024-04-25T10:17:14.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99c81498d0 0x7f99c80c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99c81498d0 0x7f99c80c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46658/0 (socket says 172.21.15.60:46658) 2024-04-25T10:17:14.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c7fff640 1 -- 172.21.15.60:0/2239437159 learned_addr learned my addr 172.21.15.60:0/2239437159 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:14.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c7fff640 1 -- 172.21.15.60:0/2239437159 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99c80b8f80 msgr2=0x7f99c80c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c77fe640 1 --2- 172.21.15.60:0/2239437159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99c8149cb0 0x7f99c80c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99cca95640 1 --2- 172.21.15.60:0/2239437159 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99c80b8f80 0x7f99c80c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c7fff640 1 --2- 172.21.15.60:0/2239437159 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99c80b8f80 0x7f99c80c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c7fff640 1 -- 172.21.15.60:0/2239437159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99c8149cb0 msgr2=0x7f99c80c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:14.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c7fff640 1 --2- 172.21.15.60:0/2239437159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99c8149cb0 0x7f99c80c7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:14.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c7fff640 1 -- 172.21.15.60:0/2239437159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c80c9310 con 0x7f99c81498d0 2024-04-25T10:17:14.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c7fff640 1 --2- 172.21.15.60:0/2239437159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99c81498d0 0x7f99c80c7840 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f99b801dc00 tx=0x7f99b0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c4ff9640 1 -- 172.21.15.60:0/2239437159 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b80ad430 con 0x7f99c81498d0 2024-04-25T10:17:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c4ff9640 1 -- 172.21.15.60:0/2239437159 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b80017c0 con 0x7f99c81498d0 2024-04-25T10:17:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c4ff9640 1 -- 172.21.15.60:0/2239437159 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b80019b0 con 0x7f99c81498d0 2024-04-25T10:17:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99ce4f8640 1 -- 172.21.15.60:0/2239437159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c80c9590 con 0x7f99c81498d0 2024-04-25T10:17:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99ce4f8640 1 -- 172.21.15.60:0/2239437159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c80c9b50 con 0x7f99c81498d0 2024-04-25T10:17:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99ce4f8640 1 -- 172.21.15.60:0/2239437159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c8074770 con 0x7f99c81498d0 2024-04-25T10:17:14.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c4ff9640 1 -- 172.21.15.60:0/2239437159 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99b80bd030 con 0x7f99c81498d0 2024-04-25T10:17:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c4ff9640 1 --2- 172.21.15.60:0/2239437159 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f99b4066f60 0x7f99b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c4ff9640 1 -- 172.21.15.60:0/2239437159 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f99b811ef30 con 0x7f99c81498d0 2024-04-25T10:17:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.829+0000 7f99c77fe640 1 --2- 172.21.15.60:0/2239437159 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f99b4066f60 0x7f99b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:14.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.833+0000 7f99c77fe640 1 --2- 172.21.15.60:0/2239437159 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f99b4066f60 0x7f99b4069420 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f99c814a090 tx=0x7f99bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:14.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.833+0000 7f99c4ff9640 1 -- 172.21.15.60:0/2239437159 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99b80c4060 con 0x7f99c81498d0 2024-04-25T10:17:14.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.993+0000 7f99ce4f8640 1 -- 172.21.15.60:0/2239437159 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f99c80d17a0 con 0x7f99b4066f60 2024-04-25T10:17:14.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.993+0000 7f99c4ff9640 1 -- 172.21.15.60:0/2239437159 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f99c80d17a0 con 0x7f99b4066f60 2024-04-25T10:17:14.998 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T10:17:15.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.993+0000 7f99ce4f8640 1 -- 172.21.15.60:0/2239437159 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f99b4066f60 msgr2=0x7f99b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:15.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.993+0000 7f99ce4f8640 1 --2- 172.21.15.60:0/2239437159 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f99b4066f60 0x7f99b4069420 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f99c814a090 tx=0x7f99bc008040 comp rx=0 tx=0).stop 2024-04-25T10:17:15.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.993+0000 7f99ce4f8640 1 -- 172.21.15.60:0/2239437159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99c81498d0 msgr2=0x7f99c80c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:15.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.993+0000 7f99ce4f8640 1 --2- 172.21.15.60:0/2239437159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99c81498d0 0x7f99c80c7840 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f99b801dc00 tx=0x7f99b0005840 comp rx=0 tx=0).stop 2024-04-25T10:17:15.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.993+0000 7f99ce4f8640 1 -- 172.21.15.60:0/2239437159 shutdown_connections 2024-04-25T10:17:15.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.993+0000 7f99ce4f8640 1 --2- 172.21.15.60:0/2239437159 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f99b4066f60 0x7f99b4069420 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:15.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.993+0000 7f99ce4f8640 1 --2- 172.21.15.60:0/2239437159 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99c80b8f80 0x7f99c80c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:15.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.993+0000 7f99ce4f8640 1 --2- 172.21.15.60:0/2239437159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99c8149cb0 0x7f99c80c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:15.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.993+0000 7f99ce4f8640 1 --2- 172.21.15.60:0/2239437159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99c81498d0 0x7f99c80c7840 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:15.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.993+0000 7f99ce4f8640 1 -- 172.21.15.60:0/2239437159 >> 172.21.15.60:0/2239437159 conn(0x7f99c806c090 msgr2=0x7f99c80d2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:15.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.993+0000 7f99ce4f8640 1 -- 172.21.15.60:0/2239437159 shutdown_connections 2024-04-25T10:17:15.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:14.993+0000 7f99ce4f8640 1 -- 172.21.15.60:0/2239437159 wait complete. 2024-04-25T10:17:15.015 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:17:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.129+0000 7f2ae2904640 1 Processor -- start 2024-04-25T10:17:15.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.129+0000 7f2ae2904640 1 -- start start 2024-04-25T10:17:15.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.129+0000 7f2ae2904640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2adc0ae7c0 0x7f2adc071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:15.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.129+0000 7f2ae2904640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2adc06d2f0 0x7f2adc0717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:15.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.129+0000 7f2ae2904640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2adc071fd0 con 0x7f2adc15dc90 2024-04-25T10:17:15.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.129+0000 7f2ae2904640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2adc0accc0 con 0x7f2adc06d2f0 2024-04-25T10:17:15.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.129+0000 7f2ae2904640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2adc0ad260 con 0x7f2adc0ae7c0 2024-04-25T10:17:15.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.129+0000 7f2ae0ea1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2adc06d2f0 0x7f2adc0717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:15.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.129+0000 7f2ae0ea1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2adc06d2f0 0x7f2adc0717d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46662/0 (socket says 172.21.15.60:46662) 2024-04-25T10:17:15.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.129+0000 7f2ae0ea1640 1 -- 172.21.15.60:0/3869865043 learned_addr learned my addr 172.21.15.60:0/3869865043 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:15.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.129+0000 7f2adb7fe640 1 --2- 172.21.15.60:0/3869865043 >> v2:172.21.15.100:3300/0 conn(0x7f2adc0ae7c0 0x7f2adc071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:15.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.129+0000 7f2ae0ea1640 1 -- 172.21.15.60:0/3869865043 >> v2:172.21.15.100:3300/0 conn(0x7f2adc0ae7c0 msgr2=0x7f2adc071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:15.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.129+0000 7f2ae0ea1640 1 --2- 172.21.15.60:0/3869865043 >> v2:172.21.15.100:3300/0 conn(0x7f2adc0ae7c0 0x7f2adc071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:15.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.129+0000 7f2ae0ea1640 1 -- 172.21.15.60:0/3869865043 >> v1:172.21.15.100:6789/0 conn(0x7f2adc15dc90 legacy=0x7f2adc0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:15.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.129+0000 7f2ae0ea1640 1 -- 172.21.15.60:0/3869865043 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc0b2ce0 con 0x7f2adc06d2f0 2024-04-25T10:17:15.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae0ea1640 1 --2- 172.21.15.60:0/3869865043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2adc06d2f0 0x7f2adc0717d0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0d0530 tx=0x7f2acc01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5238a010be72e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:15.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2adaffd640 1 -- 172.21.15.60:0/3869865043 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc0a5780 con 0x7f2adc06d2f0 2024-04-25T10:17:15.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2adaffd640 1 -- 172.21.15.60:0/3869865043 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2acc0a5920 con 0x7f2adc06d2f0 2024-04-25T10:17:15.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2adaffd640 1 -- 172.21.15.60:0/3869865043 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc0a5cf0 con 0x7f2adc06d2f0 2024-04-25T10:17:15.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 -- 172.21.15.60:0/3869865043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2adc06d2f0 msgr2=0x7f2adc0717d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:15.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 --2- 172.21.15.60:0/3869865043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2adc06d2f0 0x7f2adc0717d0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0d0530 tx=0x7f2acc01c9b0 comp rx=0 tx=0).stop 2024-04-25T10:17:15.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 -- 172.21.15.60:0/3869865043 shutdown_connections 2024-04-25T10:17:15.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 --2- 172.21.15.60:0/3869865043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2adc06d2f0 0x7f2adc0717d0 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:15.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 --2- 172.21.15.60:0/3869865043 >> v2:172.21.15.100:3300/0 conn(0x7f2adc0ae7c0 0x7f2adc071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 -- 172.21.15.60:0/3869865043 >> 172.21.15.60:0/3869865043 conn(0x7f2adc0b12d0 msgr2=0x7f2adc0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 -- 172.21.15.60:0/3869865043 shutdown_connections 2024-04-25T10:17:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 -- 172.21.15.60:0/3869865043 wait complete. 2024-04-25T10:17:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 Processor -- start 2024-04-25T10:17:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 -- start start 2024-04-25T10:17:15.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2adc06d2f0 0x7f2adc14d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:15.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc0ae7c0 0x7f2adc152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:15.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2adc15dc90 0x7f2adc145960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:15.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc16a7e0 con 0x7f2adc06d2f0 2024-04-25T10:17:15.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc0b2220 con 0x7f2adc0ae7c0 2024-04-25T10:17:15.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc16a960 con 0x7f2adc15dc90 2024-04-25T10:17:15.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2adbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2adc06d2f0 0x7f2adc14d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:15.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2adbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2adc06d2f0 0x7f2adc14d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46666/0 (socket says 172.21.15.60:46666) 2024-04-25T10:17:15.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2adbfff640 1 -- 172.21.15.60:0/2911241736 learned_addr learned my addr 172.21.15.60:0/2911241736 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:15.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2adbfff640 1 -- 172.21.15.60:0/2911241736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2adc15dc90 msgr2=0x7f2adc145960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:15.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2adbfff640 1 --2- 172.21.15.60:0/2911241736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2adc15dc90 0x7f2adc145960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:15.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2adbfff640 1 -- 172.21.15.60:0/2911241736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc0ae7c0 msgr2=0x7f2adc152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:15.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2adbfff640 1 --2- 172.21.15.60:0/2911241736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc0ae7c0 0x7f2adc152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:15.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2adbfff640 1 -- 172.21.15.60:0/2911241736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc146e50 con 0x7f2adc06d2f0 2024-04-25T10:17:15.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2adbfff640 1 --2- 172.21.15.60:0/2911241736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2adc06d2f0 0x7f2adc14d010 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400c480 tx=0x7f2ad400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:15.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ad8ff9640 1 -- 172.21.15.60:0/2911241736 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4012830 con 0x7f2adc06d2f0 2024-04-25T10:17:15.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ad8ff9640 1 -- 172.21.15.60:0/2911241736 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad4017070 con 0x7f2adc06d2f0 2024-04-25T10:17:15.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 -- 172.21.15.60:0/2911241736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2adc147130 con 0x7f2adc06d2f0 2024-04-25T10:17:15.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ad8ff9640 1 -- 172.21.15.60:0/2911241736 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4012c30 con 0x7f2adc06d2f0 2024-04-25T10:17:15.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.133+0000 7f2ae2904640 1 -- 172.21.15.60:0/2911241736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2adc154b10 con 0x7f2adc06d2f0 2024-04-25T10:17:15.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.137+0000 7f2ae2904640 1 -- 172.21.15.60:0/2911241736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2adc154d70 con 0x7f2adc06d2f0 2024-04-25T10:17:15.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.137+0000 7f2ad8ff9640 1 -- 172.21.15.60:0/2911241736 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ad401c460 con 0x7f2adc06d2f0 2024-04-25T10:17:15.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.137+0000 7f2ad8ff9640 1 --2- 172.21.15.60:0/2911241736 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ac8066f90 0x7f2ac8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:15.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.137+0000 7f2ad8ff9640 1 -- 172.21.15.60:0/2911241736 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f2ad4084c30 con 0x7f2adc06d2f0 2024-04-25T10:17:15.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.137+0000 7f2adb7fe640 1 --2- 172.21.15.60:0/2911241736 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ac8066f90 0x7f2ac8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:15.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.137+0000 7f2adb7fe640 1 --2- 172.21.15.60:0/2911241736 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ac8066f90 0x7f2ac8069450 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2adc06df00 tx=0x7f2ad0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:15.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.141+0000 7f2ad8ff9640 1 -- 172.21.15.60:0/2911241736 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ad4055f20 con 0x7f2adc06d2f0 2024-04-25T10:17:15.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.301+0000 7f2ae2904640 1 -- 172.21.15.60:0/2911241736 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2adc074770 con 0x7f2ac8066f90 2024-04-25T10:17:15.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.301+0000 7f2ad8ff9640 1 -- 172.21.15.60:0/2911241736 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13392 (secure 0 0 0) 0x7f2adc074770 con 0x7f2ac8066f90 2024-04-25T10:17:15.307 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:17:15.308 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:17:15.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.305+0000 7f2ae2904640 1 -- 172.21.15.60:0/2911241736 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ac8066f90 msgr2=0x7f2ac8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:15.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.305+0000 7f2ae2904640 1 --2- 172.21.15.60:0/2911241736 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ac8066f90 0x7f2ac8069450 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2adc06df00 tx=0x7f2ad0008040 comp rx=0 tx=0).stop 2024-04-25T10:17:15.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.305+0000 7f2ae2904640 1 -- 172.21.15.60:0/2911241736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2adc06d2f0 msgr2=0x7f2adc14d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:15.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.305+0000 7f2ae2904640 1 --2- 172.21.15.60:0/2911241736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2adc06d2f0 0x7f2adc14d010 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400c480 tx=0x7f2ad400c940 comp rx=0 tx=0).stop 2024-04-25T10:17:15.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.305+0000 7f2ae2904640 1 -- 172.21.15.60:0/2911241736 shutdown_connections 2024-04-25T10:17:15.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.305+0000 7f2ae2904640 1 --2- 172.21.15.60:0/2911241736 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2ac8066f90 0x7f2ac8069450 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:15.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.305+0000 7f2ae2904640 1 --2- 172.21.15.60:0/2911241736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2adc15dc90 0x7f2adc145960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:15.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.305+0000 7f2ae2904640 1 --2- 172.21.15.60:0/2911241736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc0ae7c0 0x7f2adc152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:15.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.305+0000 7f2ae2904640 1 --2- 172.21.15.60:0/2911241736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2adc06d2f0 0x7f2adc14d010 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:15.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.305+0000 7f2ae2904640 1 -- 172.21.15.60:0/2911241736 >> 172.21.15.60:0/2911241736 conn(0x7f2adc0b12d0 msgr2=0x7f2adc0ce7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:15.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.305+0000 7f2ae2904640 1 -- 172.21.15.60:0/2911241736 shutdown_connections 2024-04-25T10:17:15.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:15.305+0000 7f2ae2904640 1 -- 172.21.15.60:0/2911241736 wait complete. 2024-04-25T10:17:15.327 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2024-04-25T10:17:14.638160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":282792,"kb_used_data":12520,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754691928,"statfs":{"total":773094113280,"available":772804534272,"internally_reserved":0,"allocated":12820480,"data_stored":6234216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.400771"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":348,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.421317+0000","last_change":"2024-04-25T10:05:10.334295+0000","last_active":"2024-04-25T10:17:07.421317+0000","last_peered":"2024-04-25T10:17:07.421317+0000","last_clean":"2024-04-25T10:17:07.421317+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:17:07.421317+0000","last_undegraded":"2024-04-25T10:17:07.421317+0000","last_fullsized":"2024-04-25T10:17:07.421317+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:08.822959+0000","last_clean_scrub_stamp":"2024-04-25T10:05:08.822959+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.375763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2048000,"data_stored":1222053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542291,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542292,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542292,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36764,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542292,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36764,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542292,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40924,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330916,"statfs":{"total":96636764160,"available":96594857984,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542293,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37344,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334496,"statfs":{"total":96636764160,"available":96598523904,"internally_reserved":0,"allocated":2048000,"data_stored":1222053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:17:15.331 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T10:05:08.822959+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=5, tm_sec=8, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=17, tm_sec=10, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:17:15.331 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T10:17:18.320 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:18.322 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:35.332 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:17:35.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.461+0000 7f25444a1640 1 Processor -- start 2024-04-25T10:17:35.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.461+0000 7f25444a1640 1 -- start start 2024-04-25T10:17:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.461+0000 7f25444a1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253c15a7a0 0x7f253c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.461+0000 7f25444a1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f253c06ec30 0x7f253c071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.461+0000 7f25444a1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253c0721e0 con 0x7f253c0b37e0 2024-04-25T10:17:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.461+0000 7f25444a1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253c0cdaa0 con 0x7f253c06ec30 2024-04-25T10:17:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.461+0000 7f25444a1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c06d2f0 con 0x7f253c15a7a0 2024-04-25T10:17:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.461+0000 7f2541a3c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f253c06ec30 0x7f253c071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.461+0000 7f254223d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253c15a7a0 0x7f253c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.461+0000 7f254223d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253c15a7a0 0x7f253c0cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:50876/0 (socket says 172.21.15.60:50876) 2024-04-25T10:17:35.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.461+0000 7f2541a3c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f253c06ec30 0x7f253c071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56408/0 (socket says 172.21.15.60:56408) 2024-04-25T10:17:35.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.461+0000 7f254223d640 1 -- 172.21.15.60:0/2544994856 learned_addr learned my addr 172.21.15.60:0/2544994856 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:35.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.465+0000 7f254223d640 1 -- 172.21.15.60:0/2544994856 >> v2:172.21.15.100:3300/0 conn(0x7f253c06ec30 msgr2=0x7f253c071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:35.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.465+0000 7f254223d640 1 --2- 172.21.15.60:0/2544994856 >> v2:172.21.15.100:3300/0 conn(0x7f253c06ec30 0x7f253c071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.465+0000 7f254223d640 1 -- 172.21.15.60:0/2544994856 >> v1:172.21.15.100:6789/0 conn(0x7f253c0b37e0 legacy=0x7f253c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.465+0000 7f254223d640 1 -- 172.21.15.60:0/2544994856 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c06f130 con 0x7f253c15a7a0 2024-04-25T10:17:35.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.465+0000 7f254223d640 1 --2- 172.21.15.60:0/2544994856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253c15a7a0 0x7f253c0cd560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2538009870 tx=0x7f253801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae29358e6e542687 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.465+0000 7f254123b640 1 -- 172.21.15.60:0/2544994856 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25380a7070 con 0x7f253c15a7a0 2024-04-25T10:17:35.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.465+0000 7f254123b640 1 -- 172.21.15.60:0/2544994856 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25380b0070 con 0x7f253c15a7a0 2024-04-25T10:17:35.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.465+0000 7f254123b640 1 -- 172.21.15.60:0/2544994856 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25380ab890 con 0x7f253c15a7a0 2024-04-25T10:17:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 -- 172.21.15.60:0/2544994856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253c15a7a0 msgr2=0x7f253c0cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 --2- 172.21.15.60:0/2544994856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253c15a7a0 0x7f253c0cd560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2538009870 tx=0x7f253801c560 comp rx=0 tx=0).stop 2024-04-25T10:17:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 -- 172.21.15.60:0/2544994856 shutdown_connections 2024-04-25T10:17:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 --2- 172.21.15.60:0/2544994856 >> v2:172.21.15.100:3300/0 conn(0x7f253c06ec30 0x7f253c071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 --2- 172.21.15.60:0/2544994856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253c15a7a0 0x7f253c0cd560 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 -- 172.21.15.60:0/2544994856 >> 172.21.15.60:0/2544994856 conn(0x7f253c0b12d0 msgr2=0x7f253c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 -- 172.21.15.60:0/2544994856 shutdown_connections 2024-04-25T10:17:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 -- 172.21.15.60:0/2544994856 wait complete. 2024-04-25T10:17:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 Processor -- start 2024-04-25T10:17:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 -- start start 2024-04-25T10:17:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253c06ec30 0x7f253c147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f253c0b37e0 0x7f253c162050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f253c15a7a0 0x7f253c163ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c15ad00 con 0x7f253c15a7a0 2024-04-25T10:17:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c15ab80 con 0x7f253c0b37e0 2024-04-25T10:17:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c06e860 con 0x7f253c06ec30 2024-04-25T10:17:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f2542a3e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f253c15a7a0 0x7f253c163ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f2542a3e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f253c15a7a0 0x7f253c163ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32842/0 (socket says 172.21.15.60:32842) 2024-04-25T10:17:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f2542a3e640 1 -- 172.21.15.60:0/3524818569 learned_addr learned my addr 172.21.15.60:0/3524818569 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f254223d640 1 --2- 172.21.15.60:0/3524818569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253c06ec30 0x7f253c147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f2541a3c640 1 --2- 172.21.15.60:0/3524818569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f253c0b37e0 0x7f253c162050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f2542a3e640 1 -- 172.21.15.60:0/3524818569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253c06ec30 msgr2=0x7f253c147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f2542a3e640 1 --2- 172.21.15.60:0/3524818569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253c06ec30 0x7f253c147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f2542a3e640 1 -- 172.21.15.60:0/3524818569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f253c0b37e0 msgr2=0x7f253c162050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f2542a3e640 1 --2- 172.21.15.60:0/3524818569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f253c0b37e0 0x7f253c162050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f2542a3e640 1 -- 172.21.15.60:0/3524818569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c162aa0 con 0x7f253c15a7a0 2024-04-25T10:17:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f2541a3c640 1 --2- 172.21.15.60:0/3524818569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f253c0b37e0 0x7f253c162050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f2542a3e640 1 --2- 172.21.15.60:0/3524818569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f253c15a7a0 0x7f253c163ff0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f253400c770 tx=0x7f253400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f2532ffd640 1 -- 172.21.15.60:0/3524818569 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534003b80 con 0x7f253c15a7a0 2024-04-25T10:17:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f2532ffd640 1 -- 172.21.15.60:0/3524818569 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2534003d20 con 0x7f253c15a7a0 2024-04-25T10:17:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 -- 172.21.15.60:0/3524818569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c162d80 con 0x7f253c15a7a0 2024-04-25T10:17:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f2532ffd640 1 -- 172.21.15.60:0/3524818569 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253400aa90 con 0x7f253c15a7a0 2024-04-25T10:17:35.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.469+0000 7f25444a1640 1 -- 172.21.15.60:0/3524818569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c163340 con 0x7f253c15a7a0 2024-04-25T10:17:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.473+0000 7f25444a1640 1 -- 172.21.15.60:0/3524818569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253c074770 con 0x7f253c15a7a0 2024-04-25T10:17:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.473+0000 7f2532ffd640 1 -- 172.21.15.60:0/3524818569 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f253400ac30 con 0x7f253c15a7a0 2024-04-25T10:17:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.473+0000 7f2532ffd640 1 --2- 172.21.15.60:0/3524818569 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2520066f60 0x7f2520069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.473+0000 7f2532ffd640 1 -- 172.21.15.60:0/3524818569 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f2534084cf0 con 0x7f253c15a7a0 2024-04-25T10:17:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.473+0000 7f254223d640 1 --2- 172.21.15.60:0/3524818569 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2520066f60 0x7f2520069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.473+0000 7f254223d640 1 --2- 172.21.15.60:0/3524818569 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2520066f60 0x7f2520069420 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f25380af000 tx=0x7f25380a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.477+0000 7f2532ffd640 1 -- 172.21.15.60:0/3524818569 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2534089070 con 0x7f253c15a7a0 2024-04-25T10:17:35.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.637+0000 7f25444a1640 1 -- 172.21.15.60:0/3524818569 --> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f253c067f90 con 0x7f2520066f60 2024-04-25T10:17:35.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.637+0000 7f2532ffd640 1 -- 172.21.15.60:0/3524818569 <== mgr.4100 v2:172.21.15.100:6832/3699412842 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13379 (secure 0 0 0) 0x7f253c067f90 con 0x7f2520066f60 2024-04-25T10:17:35.644 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:17:35.645 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T10:17:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.641+0000 7f25444a1640 1 -- 172.21.15.60:0/3524818569 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2520066f60 msgr2=0x7f2520069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.641+0000 7f25444a1640 1 --2- 172.21.15.60:0/3524818569 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2520066f60 0x7f2520069420 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f25380af000 tx=0x7f25380a7000 comp rx=0 tx=0).stop 2024-04-25T10:17:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.641+0000 7f25444a1640 1 -- 172.21.15.60:0/3524818569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f253c15a7a0 msgr2=0x7f253c163ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.641+0000 7f25444a1640 1 --2- 172.21.15.60:0/3524818569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f253c15a7a0 0x7f253c163ff0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f253400c770 tx=0x7f253400cc30 comp rx=0 tx=0).stop 2024-04-25T10:17:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.641+0000 7f25444a1640 1 -- 172.21.15.60:0/3524818569 shutdown_connections 2024-04-25T10:17:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.641+0000 7f25444a1640 1 --2- 172.21.15.60:0/3524818569 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f2520066f60 0x7f2520069420 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.641+0000 7f25444a1640 1 --2- 172.21.15.60:0/3524818569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f253c15a7a0 0x7f253c163ff0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.641+0000 7f25444a1640 1 --2- 172.21.15.60:0/3524818569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f253c0b37e0 0x7f253c162050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.641+0000 7f25444a1640 1 --2- 172.21.15.60:0/3524818569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f253c06ec30 0x7f253c147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.641+0000 7f25444a1640 1 -- 172.21.15.60:0/3524818569 >> 172.21.15.60:0/3524818569 conn(0x7f253c0b12d0 msgr2=0x7f253c0cdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.641+0000 7f25444a1640 1 -- 172.21.15.60:0/3524818569 shutdown_connections 2024-04-25T10:17:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.641+0000 7f25444a1640 1 -- 172.21.15.60:0/3524818569 wait complete. 2024-04-25T10:17:35.664 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2024-04-25T10:17:34.643665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":282776,"kb_used_data":12520,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754691944,"statfs":{"total":773094113280,"available":772804550656,"internally_reserved":0,"allocated":12820480,"data_stored":6234216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003466"},"pg_stats":[{"pgid":"1.0","version":"117'119","reported_seq":359,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:17:15.525865+0000","last_change":"2024-04-25T10:17:15.525938+0000","last_active":"2024-04-25T10:17:15.525865+0000","last_peered":"2024-04-25T10:17:15.525865+0000","last_clean":"2024-04-25T10:17:15.525865+0000","last_became_active":"2024-04-25T10:05:10.333505+0000","last_became_peered":"2024-04-25T10:05:10.333505+0000","last_unstale":"2024-04-25T10:17:15.525865+0000","last_undegraded":"2024-04-25T10:17:15.525865+0000","last_fullsized":"2024-04-25T10:17:15.525865+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'119","last_scrub_stamp":"2024-04-25T10:17:15.525672+0000","last_deep_scrub":"117'119","last_deep_scrub_stamp":"2024-04-25T10:17:15.525672+0000","last_clean_scrub_stamp":"2024-04-25T10:17:15.525672+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:00.403718+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509593,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33176,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338664,"statfs":{"total":96636764160,"available":96602791936,"internally_reserved":0,"allocated":2048000,"data_stored":1222053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542295,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542296,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542296,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36764,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542296,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36764,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542296,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40924,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330916,"statfs":{"total":96636764160,"available":96594857984,"internally_reserved":0,"allocated":1454080,"data_stored":631685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542297,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37336,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334504,"statfs":{"total":96636764160,"available":96598532096,"internally_reserved":0,"allocated":2048000,"data_stored":1222053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:17:35.665 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T10:17:35.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f507c499640 1 Processor -- start 2024-04-25T10:17:35.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f507c499640 1 -- start start 2024-04-25T10:17:35.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f507c499640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50740d08e0 0x7f50740ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f507c499640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5074070990 0x7f50740ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f507c499640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507415ae80 con 0x7f5074158320 2024-04-25T10:17:35.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f507c499640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507415bc40 con 0x7f5074070990 2024-04-25T10:17:35.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f507c499640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5074158700 con 0x7f50740d08e0 2024-04-25T10:17:35.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f507aa36640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5074070990 0x7f50740ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f5079a34640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50740d08e0 0x7f50740ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f507aa36640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5074070990 0x7f50740ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56418/0 (socket says 172.21.15.60:56418) 2024-04-25T10:17:35.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f5079a34640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50740d08e0 0x7f50740ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:50894/0 (socket says 172.21.15.60:50894) 2024-04-25T10:17:35.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f507a235640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f5074158320 0x7f50740d01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:38204/0 (socket says 172.21.15.60:38204) 2024-04-25T10:17:35.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f507aa36640 1 -- 172.21.15.60:0/1575236635 learned_addr learned my addr 172.21.15.60:0/1575236635 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:35.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f5079a34640 1 -- 172.21.15.60:0/1575236635 >> v2:172.21.15.100:3300/0 conn(0x7f5074070990 msgr2=0x7f50740ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:35.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f5079a34640 1 --2- 172.21.15.60:0/1575236635 >> v2:172.21.15.100:3300/0 conn(0x7f5074070990 0x7f50740ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f5079a34640 1 -- 172.21.15.60:0/1575236635 >> v1:172.21.15.100:6789/0 conn(0x7f5074158320 legacy=0x7f50740d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f5079a34640 1 -- 172.21.15.60:0/1575236635 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507415b160 con 0x7f50740d08e0 2024-04-25T10:17:35.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.781+0000 7f507aa36640 1 --2- 172.21.15.60:0/1575236635 >> v2:172.21.15.100:3300/0 conn(0x7f5074070990 0x7f50740ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:35.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.785+0000 7f5079a34640 1 --2- 172.21.15.60:0/1575236635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50740d08e0 0x7f50740ce4a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f50680097c0 tx=0x7f506801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ed9ccfbfedae833 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.785+0000 7f5079233640 1 -- 172.21.15.60:0/1575236635 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50680b0070 con 0x7f50740d08e0 2024-04-25T10:17:35.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.785+0000 7f5079233640 1 -- 172.21.15.60:0/1575236635 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50680ab600 con 0x7f50740d08e0 2024-04-25T10:17:35.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.785+0000 7f5079233640 1 -- 172.21.15.60:0/1575236635 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50680ab9d0 con 0x7f50740d08e0 2024-04-25T10:17:35.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.785+0000 7f507c499640 1 -- 172.21.15.60:0/1575236635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50740d08e0 msgr2=0x7f50740ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.785+0000 7f507c499640 1 --2- 172.21.15.60:0/1575236635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50740d08e0 0x7f50740ce4a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f50680097c0 tx=0x7f506801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.785+0000 7f507c499640 1 -- 172.21.15.60:0/1575236635 shutdown_connections 2024-04-25T10:17:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.785+0000 7f507c499640 1 --2- 172.21.15.60:0/1575236635 >> v2:172.21.15.100:3300/0 conn(0x7f5074070990 0x7f50740ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.785+0000 7f507c499640 1 --2- 172.21.15.60:0/1575236635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50740d08e0 0x7f50740ce4a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.785+0000 7f507c499640 1 -- 172.21.15.60:0/1575236635 >> 172.21.15.60:0/1575236635 conn(0x7f507406c6a0 msgr2=0x7f507406e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.785+0000 7f507c499640 1 -- 172.21.15.60:0/1575236635 shutdown_connections 2024-04-25T10:17:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.785+0000 7f507c499640 1 -- 172.21.15.60:0/1575236635 wait complete. 2024-04-25T10:17:35.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.785+0000 7f507c499640 1 Processor -- start 2024-04-25T10:17:35.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.785+0000 7f507c499640 1 -- start start 2024-04-25T10:17:35.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507c499640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074070990 0x7f5074153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507c499640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50740d08e0 0x7f50740b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507c499640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5074158320 0x7f507414d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507c499640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5074167810 con 0x7f5074158320 2024-04-25T10:17:35.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507c499640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507415ac20 con 0x7f5074070990 2024-04-25T10:17:35.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507c499640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5074167990 con 0x7f50740d08e0 2024-04-25T10:17:35.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507aa36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5074158320 0x7f507414d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507aa36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5074158320 0x7f507414d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32852/0 (socket says 172.21.15.60:32852) 2024-04-25T10:17:35.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507aa36640 1 -- 172.21.15.60:0/3937105690 learned_addr learned my addr 172.21.15.60:0/3937105690 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f5079a34640 1 --2- 172.21.15.60:0/3937105690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50740d08e0 0x7f50740b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507a235640 1 --2- 172.21.15.60:0/3937105690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074070990 0x7f5074153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507aa36640 1 -- 172.21.15.60:0/3937105690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50740d08e0 msgr2=0x7f50740b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507aa36640 1 --2- 172.21.15.60:0/3937105690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50740d08e0 0x7f50740b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507aa36640 1 -- 172.21.15.60:0/3937105690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074070990 msgr2=0x7f5074153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507aa36640 1 --2- 172.21.15.60:0/3937105690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074070990 0x7f5074153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507aa36640 1 -- 172.21.15.60:0/3937105690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50740b3c10 con 0x7f5074158320 2024-04-25T10:17:35.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507aa36640 1 --2- 172.21.15.60:0/3937105690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5074158320 0x7f507414d2e0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f506400efc0 tx=0x7f506400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f5072ffd640 1 -- 172.21.15.60:0/3937105690 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506400cbe0 con 0x7f5074158320 2024-04-25T10:17:35.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f5072ffd640 1 -- 172.21.15.60:0/3937105690 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506400cd80 con 0x7f5074158320 2024-04-25T10:17:35.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507c499640 1 -- 172.21.15.60:0/3937105690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50740b41d0 con 0x7f5074158320 2024-04-25T10:17:35.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f5072ffd640 1 -- 172.21.15.60:0/3937105690 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064003bc0 con 0x7f5074158320 2024-04-25T10:17:35.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507c499640 1 -- 172.21.15.60:0/3937105690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50740b3900 con 0x7f5074158320 2024-04-25T10:17:35.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f507c499640 1 -- 172.21.15.60:0/3937105690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5074074770 con 0x7f5074158320 2024-04-25T10:17:35.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f5072ffd640 1 -- 172.21.15.60:0/3937105690 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f506401b030 con 0x7f5074158320 2024-04-25T10:17:35.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f5072ffd640 1 --2- 172.21.15.60:0/3937105690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5060066f60 0x7f5060069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.789+0000 7f5072ffd640 1 -- 172.21.15.60:0/3937105690 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f5064085ee0 con 0x7f5074158320 2024-04-25T10:17:35.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.793+0000 7f507a235640 1 --2- 172.21.15.60:0/3937105690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5060066f60 0x7f5060069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.793+0000 7f507a235640 1 --2- 172.21.15.60:0/3937105690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5060066f60 0x7f5060069420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f506c000f70 tx=0x7f506c007cf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.793+0000 7f5072ffd640 1 -- 172.21.15.60:0/3937105690 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5064022020 con 0x7f5074158320 2024-04-25T10:17:35.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.957+0000 7f507c499640 1 -- 172.21.15.60:0/3937105690 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f5074070010 con 0x7f5074158320 2024-04-25T10:17:35.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.965+0000 7f5072ffd640 1 -- 172.21.15.60:0/3937105690 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) v1 ==== 123+0+0 (secure 0 0 0) 0x7f5064010040 con 0x7f5074158320 2024-04-25T10:17:35.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.965+0000 7f5072ffd640 1 -- 172.21.15.60:0/3937105690 <== mon.0 v2:172.21.15.60:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f50640572b0 con 0x7f5074158320 2024-04-25T10:17:35.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.969+0000 7f507c499640 1 -- 172.21.15.60:0/3937105690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5060066f60 msgr2=0x7f5060069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.969+0000 7f507c499640 1 --2- 172.21.15.60:0/3937105690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5060066f60 0x7f5060069420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f506c000f70 tx=0x7f506c007cf0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.969+0000 7f507c499640 1 -- 172.21.15.60:0/3937105690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5074158320 msgr2=0x7f507414d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.969+0000 7f507c499640 1 --2- 172.21.15.60:0/3937105690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5074158320 0x7f507414d2e0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f506400efc0 tx=0x7f506400c490 comp rx=0 tx=0).stop 2024-04-25T10:17:35.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.969+0000 7f507c499640 1 -- 172.21.15.60:0/3937105690 shutdown_connections 2024-04-25T10:17:35.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.969+0000 7f507c499640 1 --2- 172.21.15.60:0/3937105690 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7f5060066f60 0x7f5060069420 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.969+0000 7f507c499640 1 --2- 172.21.15.60:0/3937105690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5074158320 0x7f507414d2e0 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.969+0000 7f507c499640 1 --2- 172.21.15.60:0/3937105690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50740d08e0 0x7f50740b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.969+0000 7f507c499640 1 --2- 172.21.15.60:0/3937105690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5074070990 0x7f5074153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.969+0000 7f507c499640 1 -- 172.21.15.60:0/3937105690 >> 172.21.15.60:0/3937105690 conn(0x7f507406c6a0 msgr2=0x7f50740b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:35.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.969+0000 7f507c499640 1 -- 172.21.15.60:0/3937105690 shutdown_connections 2024-04-25T10:17:35.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:35.969+0000 7f507c499640 1 -- 172.21.15.60:0/3937105690 wait complete. 2024-04-25T10:17:35.992 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:17:35.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:36.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.105+0000 7fd592b2b640 1 Processor -- start 2024-04-25T10:17:36.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.105+0000 7fd592b2b640 1 -- start start 2024-04-25T10:17:36.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.105+0000 7fd592b2b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd58c0d0a50 0x7fd58c0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.105+0000 7fd592b2b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd58c0ce2b0 0x7fd58c0ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.105+0000 7fd592b2b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd58c06d200 con 0x7fd58c15deb0 2024-04-25T10:17:36.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.105+0000 7fd592b2b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd58c0cf2c0 con 0x7fd58c0d0a50 2024-04-25T10:17:36.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.105+0000 7fd592b2b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd58c0cefe0 con 0x7fd58c0ce2b0 2024-04-25T10:17:36.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.105+0000 7fd5908c7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd58c15deb0 0x7fd58c0d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60396/0 (socket says 172.21.15.60:60396) 2024-04-25T10:17:36.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.105+0000 7fd5908c7640 1 -- 172.21.15.60:0/466148624 learned_addr learned my addr 172.21.15.60:0/466148624 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T10:17:36.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.105+0000 7fd58bfff640 1 --2- 172.21.15.60:0/466148624 >> v2:172.21.15.60:3300/0 conn(0x7fd58c0d0a50 0x7fd58c0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.105+0000 7fd58bfff640 1 -- 172.21.15.60:0/466148624 >> v2:172.21.15.100:3300/0 conn(0x7fd58c0ce2b0 msgr2=0x7fd58c0ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:36.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.105+0000 7fd5910c8640 1 --2- 172.21.15.60:0/466148624 >> v2:172.21.15.100:3300/0 conn(0x7fd58c0ce2b0 0x7fd58c0ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.105+0000 7fd58bfff640 1 --2- 172.21.15.60:0/466148624 >> v2:172.21.15.100:3300/0 conn(0x7fd58c0ce2b0 0x7fd58c0ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.105+0000 7fd58bfff640 1 -- 172.21.15.60:0/466148624 >> v1:172.21.15.60:6789/0 conn(0x7fd58c15deb0 legacy=0x7fd58c0d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.105+0000 7fd58bfff640 1 -- 172.21.15.60:0/466148624 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c0d1d70 con 0x7fd58c0d0a50 2024-04-25T10:17:36.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.105+0000 7fd58b7fe640 1 -- 172.21.15.60:0/466148624 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412199053 0 0) 0x7fd58c06d200 con 0x7fd58c15deb0 2024-04-25T10:17:36.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.109+0000 7fd58bfff640 1 --2- 172.21.15.60:0/466148624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd58c0d0a50 0x7fd58c0b6c10 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0d1590 tx=0x7fd57401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceafa1779e6a032 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.109+0000 7fd58b7fe640 1 -- 172.21.15.60:0/466148624 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5740b0070 con 0x7fd58c0d0a50 2024-04-25T10:17:36.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.109+0000 7fd58b7fe640 1 -- 172.21.15.60:0/466148624 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd5740ab600 con 0x7fd58c0d0a50 2024-04-25T10:17:36.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.109+0000 7fd58b7fe640 1 -- 172.21.15.60:0/466148624 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5740ab9f0 con 0x7fd58c0d0a50 2024-04-25T10:17:36.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.109+0000 7fd592b2b640 1 -- 172.21.15.60:0/466148624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd58c0d0a50 msgr2=0x7fd58c0b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.109+0000 7fd592b2b640 1 --2- 172.21.15.60:0/466148624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd58c0d0a50 0x7fd58c0b6c10 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0d1590 tx=0x7fd57401c480 comp rx=0 tx=0).stop 2024-04-25T10:17:36.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.109+0000 7fd592b2b640 1 -- 172.21.15.60:0/466148624 shutdown_connections 2024-04-25T10:17:36.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.109+0000 7fd592b2b640 1 --2- 172.21.15.60:0/466148624 >> v2:172.21.15.100:3300/0 conn(0x7fd58c0ce2b0 0x7fd58c0ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.109+0000 7fd592b2b640 1 --2- 172.21.15.60:0/466148624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd58c0d0a50 0x7fd58c0b6c10 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.109+0000 7fd592b2b640 1 -- 172.21.15.60:0/466148624 >> 172.21.15.60:0/466148624 conn(0x7fd58c06c090 msgr2=0x7fd58c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:36.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.109+0000 7fd592b2b640 1 -- 172.21.15.60:0/466148624 shutdown_connections 2024-04-25T10:17:36.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.109+0000 7fd592b2b640 1 -- 172.21.15.60:0/466148624 wait complete. 2024-04-25T10:17:36.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.109+0000 7fd592b2b640 1 Processor -- start 2024-04-25T10:17:36.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd592b2b640 1 -- start start 2024-04-25T10:17:36.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd592b2b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58c0ce2b0 0x7fd58c159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd592b2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd58c0d0a50 0x7fd58c14fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd592b2b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd58c15deb0 0x7fd58c0c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd592b2b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c0d31e0 con 0x7fd58c15deb0 2024-04-25T10:17:36.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd592b2b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c0d3060 con 0x7fd58c0d0a50 2024-04-25T10:17:36.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd592b2b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c0d3360 con 0x7fd58c0ce2b0 2024-04-25T10:17:36.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd5910c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd58c15deb0 0x7fd58c0c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd5910c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd58c15deb0 0x7fd58c0c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32872/0 (socket says 172.21.15.60:32872) 2024-04-25T10:17:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd5910c8640 1 -- 172.21.15.60:0/3114419918 learned_addr learned my addr 172.21.15.60:0/3114419918 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T10:17:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd58bfff640 1 --2- 172.21.15.60:0/3114419918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd58c0d0a50 0x7fd58c14fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd5908c7640 1 --2- 172.21.15.60:0/3114419918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58c0ce2b0 0x7fd58c159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd5910c8640 1 -- 172.21.15.60:0/3114419918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58c0ce2b0 msgr2=0x7fd58c159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd5910c8640 1 --2- 172.21.15.60:0/3114419918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58c0ce2b0 0x7fd58c159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd5910c8640 1 -- 172.21.15.60:0/3114419918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd58c0d0a50 msgr2=0x7fd58c14fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd5910c8640 1 --2- 172.21.15.60:0/3114419918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd58c0d0a50 0x7fd58c14fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd5910c8640 1 -- 172.21.15.60:0/3114419918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c150180 con 0x7fd58c15deb0 2024-04-25T10:17:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd5910c8640 1 --2- 172.21.15.60:0/3114419918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd58c15deb0 0x7fd58c0c9960 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7fd58400cda0 tx=0x7fd58400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd5897fa640 1 -- 172.21.15.60:0/3114419918 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584003ba0 con 0x7fd58c15deb0 2024-04-25T10:17:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd5897fa640 1 -- 172.21.15.60:0/3114419918 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd584003d40 con 0x7fd58c15deb0 2024-04-25T10:17:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd592b2b640 1 -- 172.21.15.60:0/3114419918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58c159d10 con 0x7fd58c15deb0 2024-04-25T10:17:36.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd5897fa640 1 -- 172.21.15.60:0/3114419918 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584004260 con 0x7fd58c15deb0 2024-04-25T10:17:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd592b2b640 1 -- 172.21.15.60:0/3114419918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58c15a2d0 con 0x7fd58c15deb0 2024-04-25T10:17:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd592b2b640 1 -- 172.21.15.60:0/3114419918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58c074770 con 0x7fd58c15deb0 2024-04-25T10:17:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd5897fa640 1 -- 172.21.15.60:0/3114419918 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd584004400 con 0x7fd58c15deb0 2024-04-25T10:17:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd5897fa640 1 --2- 172.21.15.60:0/3114419918 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd56c066f60 0x7fd56c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.113+0000 7fd5897fa640 1 -- 172.21.15.60:0/3114419918 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fd584085980 con 0x7fd58c15deb0 2024-04-25T10:17:36.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.117+0000 7fd5908c7640 1 --2- 172.21.15.60:0/3114419918 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd56c066f60 0x7fd56c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.117+0000 7fd5908c7640 1 --2- 172.21.15.60:0/3114419918 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd56c066f60 0x7fd56c069420 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd57c000f70 tx=0x7fd57c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.117+0000 7fd5897fa640 1 -- 172.21.15.60:0/3114419918 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd584056b50 con 0x7fd58c15deb0 2024-04-25T10:17:36.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.309+0000 7fd592b2b640 1 -- 172.21.15.60:0/3114419918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd58c0d17a0 con 0x7fd58c15deb0 2024-04-25T10:17:36.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.313+0000 7fd5897fa640 1 -- 172.21.15.60:0/3114419918 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+3708 (secure 0 0 0) 0x7fd584056450 con 0x7fd58c15deb0 2024-04-25T10:17:36.317 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T10:17:36.317 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":108,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2440975447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2440975447},{"type":"v1","addr":"172.21.15.100:6835","nonce":2440975447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2398080758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2398080758},{"type":"v1","addr":"172.21.15.60:6833","nonce":2398080758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2759861958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2759861958},{"type":"v1","addr":"172.21.15.100:6837","nonce":2759861958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":10119,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3794012225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3794012225},{"type":"v1","addr":"172.21.15.60:6835","nonce":3794012225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[]} 2024-04-25T10:17:36.317 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 108 2024-04-25T10:17:36.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.313+0000 7fd592b2b640 1 -- 172.21.15.60:0/3114419918 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd56c066f60 msgr2=0x7fd56c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.313+0000 7fd592b2b640 1 --2- 172.21.15.60:0/3114419918 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd56c066f60 0x7fd56c069420 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd57c000f70 tx=0x7fd57c009040 comp rx=0 tx=0).stop 2024-04-25T10:17:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.313+0000 7fd592b2b640 1 -- 172.21.15.60:0/3114419918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd58c15deb0 msgr2=0x7fd58c0c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.313+0000 7fd592b2b640 1 --2- 172.21.15.60:0/3114419918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd58c15deb0 0x7fd58c0c9960 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7fd58400cda0 tx=0x7fd58400a660 comp rx=0 tx=0).stop 2024-04-25T10:17:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.313+0000 7fd592b2b640 1 -- 172.21.15.60:0/3114419918 shutdown_connections 2024-04-25T10:17:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.313+0000 7fd592b2b640 1 --2- 172.21.15.60:0/3114419918 >> [v2:172.21.15.100:6832/3699412842,v1:172.21.15.100:6833/3699412842] conn(0x7fd56c066f60 0x7fd56c069420 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.313+0000 7fd592b2b640 1 --2- 172.21.15.60:0/3114419918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd58c15deb0 0x7fd58c0c9960 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.313+0000 7fd592b2b640 1 --2- 172.21.15.60:0/3114419918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd58c0d0a50 0x7fd58c14fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.313+0000 7fd592b2b640 1 --2- 172.21.15.60:0/3114419918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd58c0ce2b0 0x7fd58c159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.313+0000 7fd592b2b640 1 -- 172.21.15.60:0/3114419918 >> 172.21.15.60:0/3114419918 conn(0x7fd58c06c090 msgr2=0x7fd58c0d0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.313+0000 7fd592b2b640 1 -- 172.21.15.60:0/3114419918 shutdown_connections 2024-04-25T10:17:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T10:17:36.313+0000 7fd592b2b640 1 -- 172.21.15.60:0/3114419918 wait complete. 2024-04-25T10:17:36.335 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T10:17:36.335 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T10:17:36.335 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:17:36.335 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:36.347 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:17:36.347 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:17:36.347 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:36.404 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:17:36.405 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:17:36.405 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:36.418 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:17:36.418 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:17:36.418 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:36.474 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:17:36.474 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T10:17:36.474 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T10:17:36.474 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:36.534 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T10:17:36.534 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T10:17:36.534 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:36.634 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T10:17:36.634 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T10:17:36.635 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:36.720 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T10:17:36.720 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T10:17:36.720 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:36.819 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T10:17:36.819 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T10:17:36.819 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:36.886 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T10:17:36.887 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T10:17:36.887 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:36.965 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T10:17:36.965 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T10:17:36.965 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:37.033 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T10:17:37.033 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T10:17:37.033 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:37.127 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T10:17:37.128 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T10:17:37.128 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T10:17:37.128 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:37.156 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T10:17:37.156 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T10:17:37.156 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:37.191 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T10:17:37.191 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T10:17:37.191 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T10:17:37.191 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:37.216 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T10:17:37.216 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T10:17:37.216 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:37.249 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T10:17:37.249 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T10:17:37.249 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:37.314 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T10:17:37.314 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T10:17:37.314 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-25T10:17:37.333 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:17:37.333 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T10:17:37.648 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:17:37.648 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T10:17:37.878 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:17:37.878 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T10:17:38.060 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:17:38.061 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T10:17:38.269 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:17:38.269 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T10:17:38.527 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:17:38.527 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T10:17:38.727 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:17:38.727 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T10:17:38.915 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:17:38.915 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T10:17:39.069 INFO:tasks.ceph:Archiving mon data... 2024-04-25T10:17:39.070 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673046/data/mon.a.tgz 2024-04-25T10:17:39.070 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2024-04-25T10:17:39.076 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.I4NvLmWscF 2024-04-25T10:17:39.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.I4NvLmWscF 2024-04-25T10:17:39.227 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.I4NvLmWscF 2024-04-25T10:17:39.289 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.I4NvLmWscF is 426KB 2024-04-25T10:17:39.351 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.I4NvLmWscF 2024-04-25T10:17:39.357 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673046/data/mon.b.tgz 2024-04-25T10:17:39.357 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-04-25T10:17:39.363 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.Qsdu4sHLMk 2024-04-25T10:17:39.363 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Qsdu4sHLMk 2024-04-25T10:17:39.508 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.Qsdu4sHLMk 2024-04-25T10:17:39.577 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.Qsdu4sHLMk is 437KB 2024-04-25T10:17:39.679 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.Qsdu4sHLMk 2024-04-25T10:17:39.684 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673046/data/mon.c.tgz 2024-04-25T10:17:39.684 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-04-25T10:17:39.733 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.XlFIoVarQk 2024-04-25T10:17:39.733 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.XlFIoVarQk 2024-04-25T10:17:39.875 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.XlFIoVarQk 2024-04-25T10:17:39.941 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.XlFIoVarQk is 437KB 2024-04-25T10:17:40.002 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.XlFIoVarQk 2024-04-25T10:17:40.007 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T10:17:40.008 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:17:40.010 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:17:40.069 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:17:40.109 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:17:40.109 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T10:17:40.110 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673046/remote/smithi060/crash 2024-04-25T10:17:40.111 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:17:40.126 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673046/remote/smithi100/crash 2024-04-25T10:17:40.127 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:17:40.164 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T10:17:40.164 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:40.174 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:40.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:40.266 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:40.304 INFO:tasks.ceph:Compressing logs... 2024-04-25T10:17:40.305 DEBUG:teuthology.orchestra.run.smithi060:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:17:40.310 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:17:40.333 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31614.log 2024-04-25T10:17:40.333 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35240.log 2024-04-25T10:17:40.334 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.31614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45222.log 2024-04-25T10:17:40.335 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31614.log.gz 2024-04-25T10:17:40.335 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34529.log 2024-04-25T10:17:40.335 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35240.log.gz 2024-04-25T10:17:40.335 INFO:teuthology.orchestra.run.smithi060.stderr:gzip/var/log/ceph/ceph-client.admin.45222.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38136.log 2024-04-25T10:17:40.336 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38644.log 2024-04-25T10:17:40.336 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45222.log.gz 2024-04-25T10:17:40.336 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38136.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34529.log.gz 2024-04-25T10:17:40.336 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43817.log 2024-04-25T10:17:40.336 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38136.log.gz 2024-04-25T10:17:40.337 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54880.log 2024-04-25T10:17:40.337 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43817.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38644.log.gz 2024-04-25T10:17:40.337 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42543.log 2024-04-25T10:17:40.338 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53277.log 2024-04-25T10:17:40.338 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43817.log.gz 2024-04-25T10:17:40.339 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32712.log 2024-04-25T10:17:40.339 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42543.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54880.log.gz 2024-04-25T10:17:40.339 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47754.log 2024-04-25T10:17:40.339 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32712.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42543.log.gz 2024-04-25T10:17:40.339 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46925.log 2024-04-25T10:17:40.339 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32712.log.gz 2024-04-25T10:17:40.339 INFO:teuthology.orchestra.run.smithi060.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.53277.log.gz 2024-04-25T10:17:40.339 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42491.log 2024-04-25T10:17:40.342 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47754.log: /var/log/ceph/ceph-client.admin.46925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38190.log 2024-04-25T10:17:40.342 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47754.log.gz 2024-04-25T10:17:40.342 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40768.log 2024-04-25T10:17:40.342 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38190.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42491.log.gz 2024-04-25T10:17:40.342 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32114.log 2024-04-25T10:17:40.342 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38190.log.gz 2024-04-25T10:17:40.342 INFO:teuthology.orchestra.run.smithi060.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.46925.log.gz 2024-04-25T10:17:40.342 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31956.log 2024-04-25T10:17:40.342 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40768.log: /var/log/ceph/ceph-client.admin.32114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47702.log 2024-04-25T10:17:40.343 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40768.log.gz 2024-04-25T10:17:40.343 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.31956.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32114.log.gz 2024-04-25T10:17:40.343 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37056.log 2024-04-25T10:17:40.343 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31956.log.gz 2024-04-25T10:17:40.343 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36453.log 2024-04-25T10:17:40.344 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47702.log.gz 2024-04-25T10:17:40.344 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37705.log 2024-04-25T10:17:40.345 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37056.log.gz 2024-04-25T10:17:40.345 INFO:teuthology.orchestra.run.smithi060.stderr:gzip/var/log/ceph/ceph-client.admin.36453.log: -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T10:17:40.345 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45803.log 2024-04-25T10:17:40.345 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36453.log.gz 2024-04-25T10:17:40.345 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mgr.y.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37705.log.gz 2024-04-25T10:17:40.345 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41298.log 2024-04-25T10:17:40.345 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32218.log 2024-04-25T10:17:40.345 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46221.log 2024-04-25T10:17:40.347 INFO:teuthology.orchestra.run.smithi060.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.45803.log.gz 2024-04-25T10:17:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37966.log 2024-04-25T10:17:40.348 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41298.log.gz 2024-04-25T10:17:40.348 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32218.log.gz 2024-04-25T10:17:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40012.log 2024-04-25T10:17:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46221.log: /var/log/ceph/ceph-client.admin.37966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51031.log 2024-04-25T10:17:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40012.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37966.log.gz 2024-04-25T10:17:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22705.log 2024-04-25T10:17:40.348 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40012.log.gz 2024-04-25T10:17:40.348 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.46221.log.gz 2024-04-25T10:17:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46347.log 2024-04-25T10:17:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51031.log: /var/log/ceph/ceph.tmp-client.admin.22705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44002.log 2024-04-25T10:17:40.348 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22705.log.gz 2024-04-25T10:17:40.348 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51031.log.gz 2024-04-25T10:17:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52640.log 2024-04-25T10:17:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44002.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46347.log.gz 2024-04-25T10:17:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54748.log 2024-04-25T10:17:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52640.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.44002.log.gz 2024-04-25T10:17:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34026.log 2024-04-25T10:17:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48479.log 2024-04-25T10:17:40.353 INFO:teuthology.orchestra.run.smithi060.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52640.log.gz 2024-04-25T10:17:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34026.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54748.log.gz 2024-04-25T10:17:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52560.log 2024-04-25T10:17:40.353 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34026.log.gz 2024-04-25T10:17:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33258.log 2024-04-25T10:17:40.353 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48479.log.gz 2024-04-25T10:17:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41246.log 2024-04-25T10:17:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33258.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.52560.log.gz 2024-04-25T10:17:40.353 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35188.log 2024-04-25T10:17:40.354 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33258.log.gz 2024-04-25T10:17:40.354 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47590.log 2024-04-25T10:17:40.354 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41246.log.gz 2024-04-25T10:17:40.354 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35896.log 2024-04-25T10:17:40.354 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32734.log 89.0% 2024-04-25T10:17:40.354 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35188.log.gz 2024-04-25T10:17:40.354 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47590.log.gz 2024-04-25T10:17:40.354 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50078.log 2024-04-25T10:17:40.354 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35896.log.gz 2024-04-25T10:17:40.354 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34496.log 2024-04-25T10:17:40.361 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32525.log 2024-04-25T10:17:40.361 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32734.log.gz 2024-04-25T10:17:40.361 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50078.log.gz 2024-04-25T10:17:40.361 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39527.log 2024-04-25T10:17:40.361 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34496.log.gz 2024-04-25T10:17:40.361 INFO:teuthology.orchestra.run.smithi060.stderr:gzip/var/log/ceph/ceph-client.admin.32525.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49598.log 2024-04-25T10:17:40.361 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39527.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.32525.log.gz --verbose 2024-04-25T10:17:40.362 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-client.admin.39032.log 2024-04-25T10:17:40.362 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39527.log.gz 2024-04-25T10:17:40.362 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48996.log 2024-04-25T10:17:40.362 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49598.log.gz 2024-04-25T10:17:40.362 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5/var/log/ceph/ceph-client.admin.39032.log: --verbose -- /var/log/ceph/ceph-client.admin.53924.log 2024-04-25T10:17:40.362 INFO:teuthology.orchestra.run.smithi060.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T10:17:40.362 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48996.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39032.log.gz 2024-04-25T10:17:40.362 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49314.log 2024-04-25T10:17:40.362 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48996.log.gz 2024-04-25T10:17:40.362 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54331.log 2024-04-25T10:17:40.362 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53924.log.gz 2024-04-25T10:17:40.362 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41056.log 2024-04-25T10:17:40.362 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44300.log 89.1% 2024-04-25T10:17:40.362 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49314.log.gz 2024-04-25T10:17:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41056.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54331.log.gz 2024-04-25T10:17:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41655.log 2024-04-25T10:17:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44300.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41056.log.gz 2024-04-25T10:17:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32920.log 2024-04-25T10:17:40.363 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44300.log.gz 2024-04-25T10:17:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45036.log 2024-04-25T10:17:40.363 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41655.log.gz 2024-04-25T10:17:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42465.log 2024-04-25T10:17:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45036.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.32920.log.gz -5 2024-04-25T10:17:40.363 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46813.log 2024-04-25T10:17:40.363 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45036.log.gz 2024-04-25T10:17:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:gzip/var/log/ceph/ceph-client.admin.42465.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46525.log 2024-04-25T10:17:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46813.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42465.log.gz 2024-04-25T10:17:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32947.log 2024-04-25T10:17:40.363 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46813.log.gz 2024-04-25T10:17:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46655.log 2024-04-25T10:17:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32947.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46525.log.gz 2024-04-25T10:17:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44947.log 2024-04-25T10:17:40.364 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32947.log.gz 2024-04-25T10:17:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.46655.log: /var/log/ceph/ceph-client.admin.49022.log 2024-04-25T10:17:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51839.log 2024-04-25T10:17:40.364 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46655.log.gz 2024-04-25T10:17:40.364 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44947.log.gz 2024-04-25T10:17:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40898.log 2024-04-25T10:17:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38922.log 2024-04-25T10:17:40.373 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49022.log.gz 2024-04-25T10:17:40.373 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51839.log.gz 2024-04-25T10:17:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45627.log 2024-04-25T10:17:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40898.log: /var/log/ceph/ceph-client.admin.38922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37757.log 2024-04-25T10:17:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51425.log 2024-04-25T10:17:40.373 INFO:teuthology.orchestra.run.smithi060.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.38922.log.gz 2024-04-25T10:17:40.373 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40898.log.gz 2024-04-25T10:17:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37757.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45627.log.gz 2024-04-25T10:17:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32305.log 2024-04-25T10:17:40.374 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37757.log.gz 2024-04-25T10:17:40.374 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.51425.log: /var/log/ceph/ceph-client.admin.39092.log 2024-04-25T10:17:40.374 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32305.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T10:17:40.374 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51425.log.gz 2024-04-25T10:17:40.374 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39092.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32305.log.gz 2024-04-25T10:17:40.374 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42732.log 2024-04-25T10:17:40.374 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39092.log.gz 2024-04-25T10:17:40.374 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32934.log 2024-04-25T10:17:40.374 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44866.log 2024-04-25T10:17:40.374 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42732.log: /var/log/ceph/ceph-client.admin.32934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47119.log 2024-04-25T10:17:40.374 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42732.log.gz 2024-04-25T10:17:40.374 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32934.log.gz 2024-04-25T10:17:40.374 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34712.log 2024-04-25T10:17:40.374 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44866.log.gz 2024-04-25T10:17:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36046.log 2024-04-25T10:17:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47936.log 2024-04-25T10:17:40.375 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47119.log.gz 2024-04-25T10:17:40.375 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34712.log.gz 2024-04-25T10:17:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50940.log 2024-04-25T10:17:40.375 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36046.log.gz 2024-04-25T10:17:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41134.log 2024-04-25T10:17:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47936.log: /var/log/ceph/ceph-client.admin.50940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43893.log 2024-04-25T10:17:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.44579.log 2024-04-25T10:17:40.375 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50940.log.gz 2024-04-25T10:17:40.375 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47936.log.gz 2024-04-25T10:17:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41134.log: /var/log/ceph/ceph-client.admin.43893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33974.log 2024-04-25T10:17:40.375 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41134.log.gz 2024-04-25T10:17:40.375 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43893.log.gz 2024-04-25T10:17:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44326.log 2024-04-25T10:17:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32315.log 89.3% 2024-04-25T10:17:40.376 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44579.log.gz 2024-04-25T10:17:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33974.log: /var/log/ceph/ceph-client.admin.44326.log: 89.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.33974.log.gz 2024-04-25T10:17:40.376 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36685.log 2024-04-25T10:17:40.376 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44326.log.gz 2024-04-25T10:17:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44501.log 2024-04-25T10:17:40.376 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32315.log.gz 2024-04-25T10:17:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37117.log 2024-04-25T10:17:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36685.log: /var/log/ceph/ceph-client.admin.44501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34925.log 2024-04-25T10:17:40.376 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36685.log.gz 2024-04-25T10:17:40.376 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44501.log.gz 2024-04-25T10:17:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52614.log 2024-04-25T10:17:40.376 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37117.log.gz 2024-04-25T10:17:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37168.log 2024-04-25T10:17:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34925.log: /var/log/ceph/ceph-client.admin.52614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39638.log 2024-04-25T10:17:40.377 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34925.log.gz 2024-04-25T10:17:40.377 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52614.log.gz 2024-04-25T10:17:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53548.log 2024-04-25T10:17:40.377 INFO:teuthology.orchestra.run.smithi060.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.37168.log.gz 2024-04-25T10:17:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5/var/log/ceph/ceph-client.admin.39638.log: --verbose -- /var/log/ceph/ceph-client.admin.45324.log 2024-04-25T10:17:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53548.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39638.log.gz 2024-04-25T10:17:40.377 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40974.log 2024-04-25T10:17:40.392 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27841.log 2024-04-25T10:17:40.392 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T10:17:40.392 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.27841.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T10:17:40.392 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27841.log.gzgzip 2024-04-25T10:17:40.392 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.27769.log 2024-04-25T10:17:40.392 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27793.log 2024-04-25T10:17:40.392 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T10:17:40.392 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.27769.log: /var/log/ceph/ceph-client.admin.27793.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T10:17:40.393 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27793.log.gz 2024-04-25T10:17:40.393 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.5.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27769.log.gz 2024-04-25T10:17:40.393 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27721.log 2024-04-25T10:17:40.393 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:17:40.393 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.27721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27872.log 2024-04-25T10:17:40.393 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27721.log.gz 2024-04-25T10:17:40.393 INFO:teuthology.orchestra.run.smithi100.stderr:gzip/var/log/ceph/ceph.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27745.log 2024-04-25T10:17:40.393 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.27872.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T10:17:40.393 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5%/var/log/ceph/ceph-client.admin.27745.log: -- replaced with /var/log/ceph/ceph-client.admin.27872.log.gz 2024-04-25T10:17:40.393 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:17:40.393 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27745.log.gz 2024-04-25T10:17:40.393 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33337.log 2024-04-25T10:17:40.393 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27817.log 2024-04-25T10:17:40.393 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.3.33337.log: 90.6% -- replaced with /var/log/ceph/ceph-client.3.33337.log.gz 2024-04-25T10:17:40.394 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33220.log 2024-04-25T10:17:40.394 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.27817.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27817.log.gz 2024-04-25T10:17:40.394 INFO:teuthology.orchestra.run.smithi100.stderr: 92.5%gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T10:17:40.394 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:17:40.394 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.33220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27896.log 2024-04-25T10:17:40.394 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.d.log: 90.0% -- replaced with /var/log/ceph/ceph-client.1.33220.log.gz 2024-04-25T10:17:40.394 INFO:teuthology.orchestra.run.smithi100.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:17:40.394 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T10:17:40.394 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T10:17:40.394 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.27896.log: /var/log/ceph/ceph-mds.b.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27896.log.gz 2024-04-25T10:17:40.394 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T10:17:40.394 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mon.b.log: 92.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T10:17:40.424 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.4.log: 91.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T10:17:40.468 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54206.log 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi060.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.53548.log.gz 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45324.log.gz 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49369.log 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54722.log 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.49369.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54206.log.gz 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38162.log 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40974.log.gz 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37426.log 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54722.log: /var/log/ceph/ceph-client.admin.38162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32010.log 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54722.log.gz 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38162.log.gz 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43948.log 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi060.stderr:gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.37426.log.gz 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.33492.log 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32010.log: /var/log/ceph/ceph-client.0.43948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36821.log 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32010.log.gz 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32737.log 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4%/var/log/ceph/ceph-client.admin.36821.log: -- replaced with /var/log/ceph/ceph-client.admin.33492.log.gz 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35349.log 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36100.log 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36821.log.gz 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi060.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.0.43948.log.gz 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32737.log.gz 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45601.log 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36100.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35349.log.gz 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44631.log 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36100.log.gz 2024-04-25T10:17:40.471 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48111.log 2024-04-25T10:17:40.471 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45601.log: /var/log/ceph/ceph-client.admin.44631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54436.log 2024-04-25T10:17:40.471 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48111.log: 88.9% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44631.log.gz 2024-04-25T10:17:40.471 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45601.log.gz 2024-04-25T10:17:40.471 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42700.log 2024-04-25T10:17:40.471 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48111.log.gz 2024-04-25T10:17:40.471 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48451.log 2024-04-25T10:17:40.471 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54436.log: /var/log/ceph/ceph-client.admin.42700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35479.log 2024-04-25T10:17:40.471 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54436.log.gz 2024-04-25T10:17:40.471 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48451.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42700.log.gz 2024-04-25T10:17:40.471 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48728.log 2024-04-25T10:17:40.471 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48451.log.gz 2024-04-25T10:17:40.471 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54057.log 2024-04-25T10:17:40.471 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35479.log: 94.5% -- replaced with /var/log/ceph/ceph-client.0.49369.log.gz 2024-04-25T10:17:40.471 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39449.log 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48728.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35479.log.gz 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47406.log 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48728.log.gz 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39449.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54057.log.gz 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41942.log 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39449.log.gz 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48507.log 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32322.log 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41942.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.47406.log.gz 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48507.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41942.log.gz 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33466.log 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48507.log.gz 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44657.log 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33466.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32322.log.gz 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45903.log 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44657.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33466.log.gz 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45763.log 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44657.log.gz 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32321.log 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45903.log.gz 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37992.log 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54135.log 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi060.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.45763.log.gz 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37992.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32321.log.gz 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45248.log 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37992.log.gz 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44605.log 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52169.log 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45248.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54135.log.gz 2024-04-25T10:17:40.474 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44605.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45248.log.gz 2024-04-25T10:17:40.474 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40350.log 2024-04-25T10:17:40.474 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44605.log.gz 2024-04-25T10:17:40.474 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37731.log 2024-04-25T10:17:40.474 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52169.log.gz 2024-04-25T10:17:40.474 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50628.log 2024-04-25T10:17:40.474 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37731.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40350.log.gz 2024-04-25T10:17:40.474 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51097.log 2024-04-25T10:17:40.474 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50628.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37731.log.gz 2024-04-25T10:17:40.474 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51918.log 2024-04-25T10:17:40.474 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50628.log.gz 2024-04-25T10:17:40.474 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43867.log 2024-04-25T10:17:40.474 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51097.log.gz 2024-04-25T10:17:40.475 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38583.log 2024-04-25T10:17:40.475 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43867.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51918.log.gz 2024-04-25T10:17:40.475 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46168.log 2024-04-25T10:17:40.475 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.38583.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43867.log.gz 2024-04-25T10:17:40.475 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38694.log 2024-04-25T10:17:40.475 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54826.log 2024-04-25T10:17:40.475 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53468.log 2024-04-25T10:17:40.475 INFO:teuthology.orchestra.run.smithi060.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.38694.log.gz 2024-04-25T10:17:40.475 INFO:teuthology.orchestra.run.smithi060.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.46168.log.gz 2024-04-25T10:17:40.475 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35870.log 2024-04-25T10:17:40.475 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53838.log 2024-04-25T10:17:40.475 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54826.log.gz 2024-04-25T10:17:40.475 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53468.log.gz 2024-04-25T10:17:40.475 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38722.log 2024-04-25T10:17:40.476 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35870.log.gz 2024-04-25T10:17:40.476 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35766.log 2024-04-25T10:17:40.476 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53838.log: /var/log/ceph/ceph-client.admin.38722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37146.log 2024-04-25T10:17:40.476 INFO:teuthology.orchestra.run.smithi060.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.38583.log.gz 2024-04-25T10:17:40.476 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38722.log.gz 2024-04-25T10:17:40.476 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53838.log.gz 2024-04-25T10:17:40.476 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38954.log 2024-04-25T10:17:40.476 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37146.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35766.log.gz 2024-04-25T10:17:40.476 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33414.log 2024-04-25T10:17:40.476 INFO:teuthology.orchestra.run.smithi060.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.37146.log.gz 2024-04-25T10:17:40.476 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42141.log 2024-04-25T10:17:40.476 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36256.log 2024-04-25T10:17:40.476 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38954.log.gz 2024-04-25T10:17:40.477 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42141.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33414.log.gz 2024-04-25T10:17:40.477 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34392.log 2024-04-25T10:17:40.477 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42141.log.gz 2024-04-25T10:17:40.477 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46681.log 2024-04-25T10:17:40.477 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34392.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36256.log.gz 2024-04-25T10:17:40.477 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40718.log 2024-04-25T10:17:40.477 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46681.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34392.log.gz 2024-04-25T10:17:40.477 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T10:17:40.477 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40718.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46681.log.gz 2024-04-25T10:17:40.477 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35818.log 2024-04-25T10:17:40.477 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46373.log 2024-04-25T10:17:40.477 INFO:teuthology.orchestra.run.smithi060.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.40718.log.gz 2024-04-25T10:17:40.477 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49048.log 2024-04-25T10:17:40.477 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37452.log 2024-04-25T10:17:40.478 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46373.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35818.log.gz 2024-04-25T10:17:40.478 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41970.log 2024-04-25T10:17:40.478 INFO:teuthology.orchestra.run.smithi060.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.46373.log.gz 2024-04-25T10:17:40.478 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49048.log.gz 2024-04-25T10:17:40.478 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46761.log 2024-04-25T10:17:40.478 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47806.log 2024-04-25T10:17:40.478 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37452.log.gz 2024-04-25T10:17:40.478 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41970.log.gz 2024-04-25T10:17:40.478 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51813.log 2024-04-25T10:17:40.478 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45931.log 2024-04-25T10:17:40.478 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47806.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46761.log.gz 2024-04-25T10:17:40.478 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51813.log: gzip -5 89.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.47806.log.gz 2024-04-25T10:17:40.478 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.42347.log 2024-04-25T10:17:40.478 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51813.log.gz 2024-04-25T10:17:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39553.log 2024-04-25T10:17:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40090.log 2024-04-25T10:17:40.479 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45931.log.gz 2024-04-25T10:17:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39553.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42347.log.gz 2024-04-25T10:17:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39579.log 2024-04-25T10:17:40.479 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39553.log.gz 2024-04-25T10:17:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40090.log: /var/log/ceph/ceph-client.0.46062.log 2024-04-25T10:17:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41863.log 2024-04-25T10:17:40.479 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40090.log.gz 2024-04-25T10:17:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.46062.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39579.log.gz 2024-04-25T10:17:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52438.log 2024-04-25T10:17:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36952.log 2024-04-25T10:17:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41863.log: /var/log/ceph/ceph-client.admin.52438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42866.log 2024-04-25T10:17:40.479 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41863.log.gz 2024-04-25T10:17:40.480 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36952.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52438.log.gz 2024-04-25T10:17:40.480 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46629.log 2024-04-25T10:17:40.480 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42866.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36952.log.gz 2024-04-25T10:17:40.480 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37601.log 2024-04-25T10:17:40.480 INFO:teuthology.orchestra.run.smithi060.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.0.46062.log.gz 2024-04-25T10:17:40.480 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42866.log.gz 2024-04-25T10:17:40.480 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34899.log 2024-04-25T10:17:40.480 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49870.log 2024-04-25T10:17:40.480 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46629.log.gz 2024-04-25T10:17:40.480 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37601.log.gz 2024-04-25T10:17:40.480 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53355.log 2024-04-25T10:17:40.480 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53084.log/var/log/ceph/ceph-client.admin.49870.log: 2024-04-25T10:17:40.480 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34899.log.gz 2024-04-25T10:17:40.480 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38478.log 2024-04-25T10:17:40.480 INFO:teuthology.orchestra.run.smithi060.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49870.log.gz 2024-04-25T10:17:40.481 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49572.log 2024-04-25T10:17:40.481 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39932.log 2024-04-25T10:17:40.481 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53084.log.gz 2024-04-25T10:17:40.481 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38478.log: /var/log/ceph/ceph-client.admin.49572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34314.log 2024-04-25T10:17:40.481 INFO:teuthology.orchestra.run.smithi060.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.53355.log.gz 2024-04-25T10:17:40.481 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39932.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49572.log.gz 2024-04-25T10:17:40.481 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32140.log 2024-04-25T10:17:40.481 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38478.log.gz 2024-04-25T10:17:40.481 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54800.log 2024-04-25T10:17:40.481 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39932.log.gz 2024-04-25T10:17:40.481 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34314.log.gz 2024-04-25T10:17:40.481 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54696.log 2024-04-25T10:17:40.481 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51970.log 2024-04-25T10:17:40.481 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54800.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32140.log.gz 2024-04-25T10:17:40.482 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54696.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54800.log.gz 2024-04-25T10:17:40.482 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44814.log 2024-04-25T10:17:40.482 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54696.log.gz 2024-04-25T10:17:40.482 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38074.log 2024-04-25T10:17:40.482 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51970.log: /var/log/ceph/ceph-client.admin.44814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40222.log 2024-04-25T10:17:40.482 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51970.log.gz 2024-04-25T10:17:40.482 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44814.log.gz 2024-04-25T10:17:40.482 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40540.log 2024-04-25T10:17:40.482 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40222.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38074.log.gz 2024-04-25T10:17:40.482 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46997.log 2024-04-25T10:17:40.482 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40222.log.gz 2024-04-25T10:17:40.482 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.40540.log: -- /var/log/ceph/ceph-client.admin.37348.log 2024-04-25T10:17:40.482 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40116.log 2024-04-25T10:17:40.482 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37348.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T10:17:40.483 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40540.log.gz 2024-04-25T10:17:40.483 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37348.log.gz 2024-04-25T10:17:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39787.log 2024-04-25T10:17:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40405.log 2024-04-25T10:17:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mon.a.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40116.log.gz 2024-04-25T10:17:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35453.log 2024-04-25T10:17:40.483 INFO:teuthology.orchestra.run.smithi060.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.46997.log.gz 2024-04-25T10:17:40.483 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39787.log.gz 2024-04-25T10:17:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.40405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37322.log 2024-04-25T10:17:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52896.log 2024-04-25T10:17:40.483 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35453.log.gz 2024-04-25T10:17:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32736.log 2024-04-25T10:17:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54284.log 2024-04-25T10:17:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52896.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37322.log.gz 2024-04-25T10:17:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32738.log 2024-04-25T10:17:40.484 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52896.log.gz 2024-04-25T10:17:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31671.log 2024-04-25T10:17:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54284.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32736.log.gz 2024-04-25T10:17:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32738.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54284.log.gz 2024-04-25T10:17:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51201.log 2024-04-25T10:17:40.484 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32738.log.gz 2024-04-25T10:17:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50914.log 2024-04-25T10:17:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51201.log: /var/log/ceph/ceph-client.admin.31671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41350.log 2024-04-25T10:17:40.484 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51201.log.gz/var/log/ceph/ceph-client.admin.50914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40794.log 2024-04-25T10:17:40.484 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:40.484 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31671.log.gz 2024-04-25T10:17:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41350.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50914.log.gz 2024-04-25T10:17:40.484 INFO:teuthology.orchestra.run.smithi060.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.40405.log.gz 2024-04-25T10:17:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41551.log 2024-04-25T10:17:40.485 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41350.log.gz 2024-04-25T10:17:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47304.log 2024-04-25T10:17:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41551.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.40794.log.gz 2024-04-25T10:17:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51761.log 2024-04-25T10:17:40.485 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41551.log.gz 2024-04-25T10:17:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34738.log 2024-04-25T10:17:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47304.log: /var/log/ceph/ceph-client.admin.51761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37556.log 2024-04-25T10:17:40.485 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51761.log.gz 2024-04-25T10:17:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54592.log 2024-04-25T10:17:40.485 INFO:teuthology.orchestra.run.smithi060.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.47304.log.gz 2024-04-25T10:17:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31902.log 2024-04-25T10:17:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37556.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34738.log.gz 2024-04-25T10:17:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54592.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37556.log.gz 2024-04-25T10:17:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48806.log 2024-04-25T10:17:40.485 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54592.log.gz 2024-04-25T10:17:40.486 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49988.log 2024-04-25T10:17:40.486 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.31902.log: /var/log/ceph/ceph-client.admin.48806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49707.log 2024-04-25T10:17:40.486 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31902.log.gz 2024-04-25T10:17:40.486 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48676.log 2024-04-25T10:17:40.486 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49988.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.48806.log.gz 2024-04-25T10:17:40.486 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39735.log 2024-04-25T10:17:40.486 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48676.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.49988.log.gz 2024-04-25T10:17:40.486 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49707.log.gz 2024-04-25T10:17:40.486 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49814.log 2024-04-25T10:17:40.486 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48676.log.gz 2024-04-25T10:17:40.486 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.39735.log: /var/log/ceph/ceph-client.admin.51605.log 2024-04-25T10:17:40.486 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40168.log 2024-04-25T10:17:40.486 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39735.log.gz 2024-04-25T10:17:40.486 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41681.log 2024-04-25T10:17:40.487 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51605.log.gz 2024-04-25T10:17:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42517.log 2024-04-25T10:17:40.487 INFO:teuthology.orchestra.run.smithi060.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49814.log.gz 2024-04-25T10:17:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41681.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40168.log.gz 2024-04-25T10:17:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53133.log 2024-04-25T10:17:40.487 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41681.log.gz 2024-04-25T10:17:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52051.log 2024-04-25T10:17:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42517.log: /var/log/ceph/ceph-client.admin.53133.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42517.log.gz 2024-04-25T10:17:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45274.log 2024-04-25T10:17:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45496.log 2024-04-25T10:17:40.487 INFO:teuthology.orchestra.run.smithi060.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.53133.log.gz 2024-04-25T10:17:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52051.log: /var/log/ceph/ceph-client.admin.45274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38018.log 2024-04-25T10:17:40.487 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52051.log.gz 2024-04-25T10:17:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50966.log 2024-04-25T10:17:40.488 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.45274.log.gz 2024-04-25T10:17:40.488 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45496.log.gz 2024-04-25T10:17:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46297.log 2024-04-25T10:17:40.488 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38018.log.gz 2024-04-25T10:17:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50706.log 2024-04-25T10:17:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50966.log: /var/log/ceph/ceph-client.admin.46297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52949.log 2024-04-25T10:17:40.488 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50966.log.gz 2024-04-25T10:17:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40844.log 2024-04-25T10:17:40.488 INFO:teuthology.orchestra.run.smithi060.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.46297.log.gz 2024-04-25T10:17:40.488 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2%/var/log/ceph/ceph-client.admin.52949.log: -- replaced with /var/log/ceph/ceph-client.admin.50706.log.gz 2024-04-25T10:17:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45877.log 2024-04-25T10:17:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40844.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52949.log.gz 2024-04-25T10:17:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39352.log 2024-04-25T10:17:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50410.log 2024-04-25T10:17:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45877.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40844.log.gz 2024-04-25T10:17:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45575.log 2024-04-25T10:17:40.489 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45877.log.gz 2024-04-25T10:17:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50410.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39352.log.gz 2024-04-25T10:17:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50680.log 2024-04-25T10:17:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45064.log 2024-04-25T10:17:40.489 INFO:teuthology.orchestra.run.smithi060.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.50410.log.gz 2024-04-25T10:17:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45575.log: /var/log/ceph/ceph-client.admin.50680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48624.log 2024-04-25T10:17:40.489 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45575.log.gz 2024-04-25T10:17:40.489 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50680.log.gz 2024-04-25T10:17:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33746.log 2024-04-25T10:17:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40274.log 2024-04-25T10:17:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48624.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45064.log.gz 2024-04-25T10:17:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33746.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48624.log.gz 2024-04-25T10:17:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36399.log 2024-04-25T10:17:40.490 INFO:teuthology.orchestra.run.smithi060.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.33746.log.gz 2024-04-25T10:17:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36557.log 2024-04-25T10:17:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40274.log: /var/log/ceph/ceph-client.admin.36399.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40274.log.gz 2024-04-25T10:17:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52273.log 2024-04-25T10:17:40.490 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36399.log.gz 2024-04-25T10:17:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32532.log 2024-04-25T10:17:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36557.log: /var/log/ceph/ceph-client.admin.52273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39058.log 2024-04-25T10:17:40.490 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36557.log.gz 2024-04-25T10:17:40.490 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52273.log.gz 2024-04-25T10:17:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45653.log 2024-04-25T10:17:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35323.log 2024-04-25T10:17:40.490 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.32532.log.gz 2024-04-25T10:17:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39058.log: /var/log/ceph/ceph-client.admin.45653.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39058.log.gz 2024-04-25T10:17:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51866.log 2024-04-25T10:17:40.491 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45653.log.gz 2024-04-25T10:17:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34555.log 2024-04-25T10:17:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35323.log: /var/log/ceph/ceph-client.admin.51866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42297.log 2024-04-25T10:17:40.491 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35323.log.gz 2024-04-25T10:17:40.491 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51866.log.gz 2024-04-25T10:17:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43271.log 2024-04-25T10:17:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48372.log 2024-04-25T10:17:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42297.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34555.log.gz 2024-04-25T10:17:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48033.log 2024-04-25T10:17:40.491 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43271.log.gz 2024-04-25T10:17:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54109.log 2024-04-25T10:17:40.491 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42297.log.gz 2024-04-25T10:17:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33362.log 2024-04-25T10:17:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48033.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48372.log.gz 2024-04-25T10:17:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53522.log 2024-04-25T10:17:40.492 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48033.log.gz 2024-04-25T10:17:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33362.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37400.log 2024-04-25T10:17:40.492 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54109.log.gz 2024-04-25T10:17:40.492 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33362.log.gz 2024-04-25T10:17:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46602.log 2024-04-25T10:17:40.492 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53522.log.gz 2024-04-25T10:17:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38845.log 2024-04-25T10:17:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35792.log 2024-04-25T10:17:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46602.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37400.log.gz 2024-04-25T10:17:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38845.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46602.log.gz 2024-04-25T10:17:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49624.log 2024-04-25T10:17:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46475.log 2024-04-25T10:17:40.493 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35792.log: /var/log/ceph/ceph-client.admin.49624.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38845.log.gz 2024-04-25T10:17:40.493 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34977.log 2024-04-25T10:17:40.493 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35792.log.gz 2024-04-25T10:17:40.493 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49624.log.gz 2024-04-25T10:17:40.493 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51399.log 2024-04-25T10:17:40.493 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45379.log 2024-04-25T10:17:40.493 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34977.log: /var/log/ceph/ceph-client.admin.51399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41577.log 2024-04-25T10:17:40.493 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46475.log.gz 2024-04-25T10:17:40.493 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34977.log.gz 2024-04-25T10:17:40.493 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.45379.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39840.log 2024-04-25T10:17:40.493 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51399.log.gz 2024-04-25T10:17:40.493 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49650.log 2024-04-25T10:17:40.493 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41577.log: /var/log/ceph/ceph-client.admin.39840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35954.log 2024-04-25T10:17:40.493 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41577.log.gz 2024-04-25T10:17:40.493 INFO:teuthology.orchestra.run.smithi060.stderr:gzip 88.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.39840.log.gz --verbose 2024-04-25T10:17:40.494 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-client.admin.45740.log 2024-04-25T10:17:40.494 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49650.log: /var/log/ceph/ceph-client.admin.35954.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35003.log 2024-04-25T10:17:40.494 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49650.log.gz 2024-04-25T10:17:40.494 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35954.log.gz 2024-04-25T10:17:40.494 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36310.log 2024-04-25T10:17:40.494 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35557.log 2024-04-25T10:17:40.494 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45740.log.gz/var/log/ceph/ceph-client.admin.35003.log: 2024-04-25T10:17:40.494 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50160.log 2024-04-25T10:17:40.494 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35003.log.gz 2024-04-25T10:17:40.494 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35135.log 2024-04-25T10:17:40.494 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36310.log.gz 2024-04-25T10:17:40.494 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35557.log: /var/log/ceph/ceph-client.admin.50160.log: 92.6% -- replaced with /var/log/ceph/ceph-client.0.45379.log.gz 2024-04-25T10:17:40.494 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35557.log.gz 2024-04-25T10:17:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40692.log 2024-04-25T10:17:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35135.log: gzip -5 --verbose -- 91.1% /var/log/ceph/ceph-client.admin.50836.log -- replaced with /var/log/ceph/ceph-client.admin.50160.log.gz 2024-04-25T10:17:40.495 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:40.495 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35135.log.gz 2024-04-25T10:17:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52818.log 2024-04-25T10:17:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46787.log 2024-04-25T10:17:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50836.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40692.log.gz 2024-04-25T10:17:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52818.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50836.log.gz 2024-04-25T10:17:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34581.log 2024-04-25T10:17:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44248.log 2024-04-25T10:17:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46787.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52818.log.gz 2024-04-25T10:17:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34795.log 2024-04-25T10:17:40.495 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46787.log.gz 2024-04-25T10:17:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35109.log 2024-04-25T10:17:40.496 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34581.log.gz 2024-04-25T10:17:40.496 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44248.log.gz 2024-04-25T10:17:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49545.log 2024-04-25T10:17:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T10:17:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35109.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34795.log.gz 2024-04-25T10:17:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49545.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35109.log.gzgzip 2024-04-25T10:17:40.496 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42271.log 2024-04-25T10:17:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.3.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49545.log.gz 2024-04-25T10:17:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35688.log 2024-04-25T10:17:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53976.log 2024-04-25T10:17:40.496 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42271.log.gz 2024-04-25T10:17:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44137.log 2024-04-25T10:17:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39300.log 2024-04-25T10:17:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53976.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35688.log.gz 2024-04-25T10:17:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41454.log 2024-04-25T10:17:40.497 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53976.log.gz 2024-04-25T10:17:40.497 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44137.log.gz 2024-04-25T10:17:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39300.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39300.log.gz 2024-04-25T10:17:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45118.log 2024-04-25T10:17:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32945.log 2024-04-25T10:17:40.497 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41454.log.gz 2024-04-25T10:17:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39670.log 2024-04-25T10:17:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:gzip/var/log/ceph/ceph-client.admin.32945.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45710.log 2024-04-25T10:17:40.497 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45118.log.gz 2024-04-25T10:17:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39670.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32945.log.gz -5 2024-04-25T10:17:40.497 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37679.log 2024-04-25T10:17:40.497 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39670.log.gz 2024-04-25T10:17:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49289.log 2024-04-25T10:17:40.498 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45710.log.gz 2024-04-25T10:17:40.498 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36230.log 2024-04-25T10:17:40.498 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49289.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37679.log.gz 2024-04-25T10:17:40.498 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34633.log 2024-04-25T10:17:40.498 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48059.log 2024-04-25T10:17:40.498 INFO:teuthology.orchestra.run.smithi060.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.49289.log.gz 2024-04-25T10:17:40.498 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36230.log.gz 2024-04-25T10:17:40.498 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42596.log 2024-04-25T10:17:40.498 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48568.log 2024-04-25T10:17:40.498 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48059.log: -- replaced with /var/log/ceph/ceph-client.admin.34633.log.gz 2024-04-25T10:17:40.498 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35662.log 2024-04-25T10:17:40.498 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48059.log.gz 2024-04-25T10:17:40.498 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51451.log 2024-04-25T10:17:40.498 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42596.log.gz 2024-04-25T10:17:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48568.log: /var/log/ceph/ceph-client.admin.35662.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48568.log.gz 2024-04-25T10:17:40.499 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35662.log.gz 2024-04-25T10:17:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42402.log 2024-04-25T10:17:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42892.log 2024-04-25T10:17:40.499 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51451.log.gz 2024-04-25T10:17:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.42402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52195.log 2024-04-25T10:17:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36152.log 2024-04-25T10:17:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42892.log: /var/log/ceph/ceph-client.admin.52195.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34104.log 2024-04-25T10:17:40.499 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42892.log.gz 2024-04-25T10:17:40.499 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52195.log.gz 2024-04-25T10:17:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36152.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36152.log.gz 2024-04-25T10:17:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46423.log 2024-04-25T10:17:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46118.log 2024-04-25T10:17:40.499 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2%/var/log/ceph/ceph-client.admin.46423.log: -- replaced with /var/log/ceph/ceph-client.admin.34104.log.gz 2024-04-25T10:17:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35844.log 2024-04-25T10:17:40.500 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46118.log: 92.7% -- replaced with /var/log/ceph/ceph-client.0.42402.log.gz 2024-04-25T10:17:40.500 INFO:teuthology.orchestra.run.smithi060.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46423.log.gz 2024-04-25T10:17:40.500 INFO:teuthology.orchestra.run.smithi060.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.46118.log.gz 2024-04-25T10:17:40.500 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35029.log 2024-04-25T10:17:40.500 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47091.log 2024-04-25T10:17:40.500 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35844.log.gz 2024-04-25T10:17:40.500 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40300.log 2024-04-25T10:17:40.500 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5/var/log/ceph/ceph-client.admin.47091.log: --verbose -- /var/log/ceph/ceph-client.admin.47668.log 2024-04-25T10:17:40.500 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35029.log.gz 2024-04-25T10:17:40.500 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40300.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47091.log.gz 2024-04-25T10:17:40.500 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32936.log 2024-04-25T10:17:40.500 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32524.log 2024-04-25T10:17:40.500 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2%/var/log/ceph/ceph-client.admin.32936.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.40300.log.gz 2024-04-25T10:17:40.501 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47668.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47027.log 2024-04-25T10:17:40.501 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:40.501 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32936.log.gz 2024-04-25T10:17:40.501 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32524.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32524.log.gz 2024-04-25T10:17:40.501 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36740.log 2024-04-25T10:17:40.501 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44170.log 2024-04-25T10:17:40.501 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.36740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44430.log 2024-04-25T10:17:40.501 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47027.log.gz 2024-04-25T10:17:40.501 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44840.log 2024-04-25T10:17:40.501 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44170.log.gz/var/log/ceph/ceph-client.admin.44430.log: 2024-04-25T10:17:40.501 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52844.log 2024-04-25T10:17:40.501 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39170.log 2024-04-25T10:17:40.501 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44430.log.gz 2024-04-25T10:17:40.501 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44840.log: /var/log/ceph/ceph-client.admin.52844.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44840.log.gz 2024-04-25T10:17:40.501 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52844.log.gz 2024-04-25T10:17:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41402.log 2024-04-25T10:17:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52404.log 2024-04-25T10:17:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41402.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50284.log 2024-04-25T10:17:40.502 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39170.log.gz 2024-04-25T10:17:40.502 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41402.log.gz 2024-04-25T10:17:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52404.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52404.log.gz 2024-04-25T10:17:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34444.log 2024-04-25T10:17:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31850.log 2024-04-25T10:17:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31795.log 2024-04-25T10:17:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34444.log: /var/log/ceph/ceph-client.admin.31850.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34444.log.gz 2024-04-25T10:17:40.502 INFO:teuthology.orchestra.run.smithi060.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.36740.log.gz 2024-04-25T10:17:40.502 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31850.log.gz 2024-04-25T10:17:40.502 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50284.log.gz 2024-04-25T10:17:40.503 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43192.log 2024-04-25T10:17:40.503 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.31795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45090.log 2024-04-25T10:17:40.503 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41272.log 2024-04-25T10:17:40.503 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31795.log.gz 2024-04-25T10:17:40.503 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32530.log 2024-04-25T10:17:40.503 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43192.log.gz 2024-04-25T10:17:40.503 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33440.log 2024-04-25T10:17:40.503 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45090.log.gz 2024-04-25T10:17:40.503 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41272.log: /var/log/ceph/ceph-client.admin.32530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38426.log 2024-04-25T10:17:40.503 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41272.log.gz 2024-04-25T10:17:40.503 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1%/var/log/ceph/ceph-client.admin.33440.log: -- replaced with /var/log/ceph/ceph-client.admin.32530.log.gz 2024-04-25T10:17:40.511 INFO:teuthology.orchestra.run.smithi100.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T10:17:40.553 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39984.log 2024-04-25T10:17:40.554 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33440.log.gz 2024-04-25T10:17:40.554 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52001.log 2024-04-25T10:17:40.554 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39984.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38426.log.gz 2024-04-25T10:17:40.554 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33913.log 2024-04-25T10:17:40.554 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39984.log.gz 2024-04-25T10:17:40.554 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51555.log 2024-04-25T10:17:40.554 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51503.log 2024-04-25T10:17:40.554 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51555.log: gzip -5 --verbose -- 91.2% /var/log/ceph/ceph-client.admin.40194.log 2024-04-25T10:17:40.554 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52001.log.gz 2024-04-25T10:17:40.554 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33913.log.gz 2024-04-25T10:17:40.554 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51503.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51503.log.gz 2024-04-25T10:17:40.554 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.51555.log.gz 2024-04-25T10:17:40.554 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T10:17:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53328.log 2024-04-25T10:17:40.555 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40194.log.gz 2024-04-25T10:17:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49676.log 2024-04-25T10:17:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37212.log 2024-04-25T10:17:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50213.log 2024-04-25T10:17:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49676.log: /var/log/ceph/ceph-client.admin.37212.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49676.log.gz 2024-04-25T10:17:40.555 INFO:teuthology.orchestra.run.smithi060.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.53328.log.gz 2024-04-25T10:17:40.555 INFO:teuthology.orchestra.run.smithi060.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.37212.log.gz 2024-04-25T10:17:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45522.log 2024-04-25T10:17:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34847.log 2024-04-25T10:17:40.555 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6%/var/log/ceph/ceph-client.admin.45522.log: -- replaced with /var/log/ceph/ceph-client.admin.50213.log.gz 2024-04-25T10:17:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36072.log 2024-04-25T10:17:40.555 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45522.log.gz 2024-04-25T10:17:40.556 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34847.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34847.log.gz 2024-04-25T10:17:40.556 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44033.log 2024-04-25T10:17:40.556 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41108.log 2024-04-25T10:17:40.556 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0%/var/log/ceph/ceph-client.admin.44033.log: -- replaced with /var/log/ceph/ceph-client.admin.36072.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38242.log 2024-04-25T10:17:40.556 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:40.556 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44033.log.gz 2024-04-25T10:17:40.556 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41108.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41108.log.gz 2024-04-25T10:17:40.556 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38721.log 2024-04-25T10:17:40.556 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52300.log 2024-04-25T10:17:40.556 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4%/var/log/ceph/ceph-client.admin.38721.log: -- replaced with /var/log/ceph/ceph-client.admin.38242.log.gz 2024-04-25T10:17:40.556 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37653.log 2024-04-25T10:17:40.556 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52300.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52300.log.gz 2024-04-25T10:17:40.556 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38721.log.gz 2024-04-25T10:17:40.556 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37848.log 2024-04-25T10:17:40.556 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43052.log 2024-04-25T10:17:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37848.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37653.log.gz 2024-04-25T10:17:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32166.log 2024-04-25T10:17:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46839.log 2024-04-25T10:17:40.557 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37848.log.gz 2024-04-25T10:17:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.43052.log: /var/log/ceph/ceph-client.admin.32166.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32166.log.gz 2024-04-25T10:17:40.557 INFO:teuthology.orchestra.run.smithi060.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.43052.log.gz 2024-04-25T10:17:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49466.log 2024-04-25T10:17:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34873.log 2024-04-25T10:17:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49466.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48164.log 2024-04-25T10:17:40.557 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46839.log.gz 2024-04-25T10:17:40.557 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49466.log.gz 2024-04-25T10:17:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34873.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34873.log.gz 2024-04-25T10:17:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35401.log 2024-04-25T10:17:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43349.log 2024-04-25T10:17:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34686.log 2024-04-25T10:17:40.558 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48164.log.gz 2024-04-25T10:17:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35401.log: /var/log/ceph/ceph-client.admin.43349.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35401.log.gz 2024-04-25T10:17:40.558 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43349.log.gz 2024-04-25T10:17:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34821.log 2024-04-25T10:17:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45144.log 2024-04-25T10:17:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34821.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34686.log.gz 2024-04-25T10:17:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40465.log 2024-04-25T10:17:40.558 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51373.log 2024-04-25T10:17:40.558 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34821.log.gz 2024-04-25T10:17:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45144.log: /var/log/ceph/ceph-client.admin.40465.log: 88.7% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45144.log.gz -- replaced with /var/log/ceph/ceph-client.admin.40465.log.gz 2024-04-25T10:17:40.558 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40590.log 2024-04-25T10:17:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38216.log 2024-04-25T10:17:40.558 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4%/var/log/ceph/ceph-client.admin.40590.log: -- replaced with /var/log/ceph/ceph-client.admin.51373.log.gz 2024-04-25T10:17:40.559 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42085.log 2024-04-25T10:17:40.559 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38216.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38216.log.gz 2024-04-25T10:17:40.559 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40590.log.gz 2024-04-25T10:17:40.559 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54258.log 2024-04-25T10:17:40.559 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52690.log 2024-04-25T10:17:40.559 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54258.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42085.log.gz 2024-04-25T10:17:40.559 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41194.log 2024-04-25T10:17:40.559 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41889.log 2024-04-25T10:17:40.559 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54258.log.gz 2024-04-25T10:17:40.559 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52690.log: /var/log/ceph/ceph-client.admin.41194.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52690.log.gz 2024-04-25T10:17:40.559 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41194.log.gz 2024-04-25T10:17:40.559 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51229.log 2024-04-25T10:17:40.559 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50602.log 2024-04-25T10:17:40.559 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5%/var/log/ceph/ceph-client.admin.51229.log: -- replaced with /var/log/ceph/ceph-client.admin.41889.log.gz 2024-04-25T10:17:40.560 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44717.log 2024-04-25T10:17:40.560 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51229.log.gz 2024-04-25T10:17:40.560 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50602.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50602.log.gz 2024-04-25T10:17:40.560 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39761.log 2024-04-25T10:17:40.560 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39423.log 2024-04-25T10:17:40.560 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7%/var/log/ceph/ceph-client.admin.39761.log: -- replaced with /var/log/ceph/ceph-client.admin.44717.log.gz 2024-04-25T10:17:40.560 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48190.log 2024-04-25T10:17:40.560 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39761.log.gz 2024-04-25T10:17:40.560 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39423.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39423.log.gz 2024-04-25T10:17:40.560 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54930.log 2024-04-25T10:17:40.560 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34262.log 2024-04-25T10:17:40.560 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9%/var/log/ceph/ceph-client.admin.54930.log: -- replaced with /var/log/ceph/ceph-client.admin.48190.log.gz 2024-04-25T10:17:40.560 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54852.log 2024-04-25T10:17:40.560 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54930.log.gz 2024-04-25T10:17:40.560 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34262.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34262.log.gz 2024-04-25T10:17:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33128.log 2024-04-25T10:17:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54514.log 2024-04-25T10:17:40.561 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2%/var/log/ceph/ceph-client.admin.33128.log: -- replaced with /var/log/ceph/ceph-client.admin.54852.log.gz 2024-04-25T10:17:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41428.log 2024-04-25T10:17:40.561 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33128.log.gz 2024-04-25T10:17:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54514.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54514.log.gz 2024-04-25T10:17:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43486.log 2024-04-25T10:17:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36018.log 2024-04-25T10:17:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43486.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41428.log.gz 2024-04-25T10:17:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48912.log 2024-04-25T10:17:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47145.log 2024-04-25T10:17:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36018.log: /var/log/ceph/ceph-client.0.48912.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.43486.log.gz 2024-04-25T10:17:40.561 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36018.log.gz 2024-04-25T10:17:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51529.log 2024-04-25T10:17:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34236.log 2024-04-25T10:17:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47145.log: /var/log/ceph/ceph-client.admin.51529.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51529.log.gz 2024-04-25T10:17:40.562 INFO:teuthology.orchestra.run.smithi060.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.47145.log.gz 2024-04-25T10:17:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43741.log 2024-04-25T10:17:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32270.log 2024-04-25T10:17:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34236.log: /var/log/ceph/ceph-client.admin.43741.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34236.log.gz 2024-04-25T10:17:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53898.log 2024-04-25T10:17:40.562 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43741.log.gz 2024-04-25T10:17:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32270.log: 94.1% -- replaced with /var/log/ceph/ceph-client.0.48912.log.gz 2024-04-25T10:17:40.562 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32270.log.gz 2024-04-25T10:17:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47832.log 2024-04-25T10:17:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34156.log 2024-04-25T10:17:40.562 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53898.log.gz 2024-04-25T10:17:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53760.log 2024-04-25T10:17:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34156.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47832.log.gz 2024-04-25T10:17:40.563 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34156.log.gz 2024-04-25T10:17:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34607.log 2024-04-25T10:17:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48216.log 2024-04-25T10:17:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34607.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53760.log.gz 2024-04-25T10:17:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52247.log 2024-04-25T10:17:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47564.log 2024-04-25T10:17:40.563 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34607.log.gz/var/log/ceph/ceph-client.admin.48216.log: 2024-04-25T10:17:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52247.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48216.log.gz 2024-04-25T10:17:40.563 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52247.log.gz 2024-04-25T10:17:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34078.log 2024-04-25T10:17:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54618.log 2024-04-25T10:17:40.563 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8%/var/log/ceph/ceph-client.admin.34078.log: -- replaced with /var/log/ceph/ceph-client.admin.47564.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.2.33837.log 2024-04-25T10:17:40.564 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:40.564 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34078.log.gz 2024-04-25T10:17:40.564 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54618.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54618.log.gz 2024-04-25T10:17:40.564 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41480.log 2024-04-25T10:17:40.564 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.2.33837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44352.log 2024-04-25T10:17:40.564 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38046.log 2024-04-25T10:17:40.564 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41480.log.gz 2024-04-25T10:17:40.564 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44352.log: 91.1% -- replaced with /var/log/ceph/ceph-client.2.33837.log.gz 2024-04-25T10:17:40.564 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44352.log.gz 2024-04-25T10:17:40.564 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35610.log 2024-04-25T10:17:40.564 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52106.log 2024-04-25T10:17:40.564 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35610.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38046.log.gz 2024-04-25T10:17:40.564 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50784.log 2024-04-25T10:17:40.564 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35922.log 2024-04-25T10:17:40.565 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35610.log.gz 2024-04-25T10:17:40.565 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.52106.log: /var/log/ceph/ceph-client.admin.50784.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50784.log.gz 2024-04-25T10:17:40.565 INFO:teuthology.orchestra.run.smithi060.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.0.52106.log.gz 2024-04-25T10:17:40.565 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39475.log 2024-04-25T10:17:40.565 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35922.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:17:40.565 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6%/var/log/ceph/ceph-client.admin.39475.log: -- replaced with /var/log/ceph/ceph-client.admin.35922.log.gz 2024-04-25T10:17:40.565 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32528.log 2024-04-25T10:17:40.565 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39475.log.gz 2024-04-25T10:17:40.565 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35505.log 2024-04-25T10:17:40.565 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.32528.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32528.log.gz 2024-04-25T10:17:40.565 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43432.log 2024-04-25T10:17:40.565 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44683.log 2024-04-25T10:17:40.565 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43432.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35505.log.gz 2024-04-25T10:17:40.565 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39501.log 2024-04-25T10:17:40.566 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43432.log.gz 2024-04-25T10:17:40.566 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44683.log: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:17:40.566 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44683.log.gz 2024-04-25T10:17:40.566 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42167.log 2024-04-25T10:17:40.566 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37004.log 2024-04-25T10:17:40.566 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7%/var/log/ceph/ceph-client.admin.42167.log: -- replaced with /var/log/ceph/ceph-client.admin.39501.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47962.log 2024-04-25T10:17:40.566 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:40.566 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42167.log.gz 2024-04-25T10:17:40.566 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36373.log 2024-04-25T10:17:40.566 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36873.log 2024-04-25T10:17:40.566 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37004.log.gz 2024-04-25T10:17:40.566 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36373.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36126.log 2024-04-25T10:17:40.566 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47962.log.gz 2024-04-25T10:17:40.566 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36373.log.gz 2024-04-25T10:17:40.567 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36873.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36873.log.gz 2024-04-25T10:17:40.567 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35266.log 2024-04-25T10:17:40.567 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31769.log 2024-04-25T10:17:40.567 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35266.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36126.log.gz 2024-04-25T10:17:40.567 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33154.log 2024-04-25T10:17:40.567 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35266.log.gz 2024-04-25T10:17:40.567 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.31769.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31769.log.gz 2024-04-25T10:17:40.567 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36204.log 2024-04-25T10:17:40.567 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38452.log 2024-04-25T10:17:40.567 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36204.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52745.log 2024-04-25T10:17:40.567 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33154.log.gz 2024-04-25T10:17:40.567 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36204.log.gz 2024-04-25T10:17:40.567 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38452.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38452.log.gz 2024-04-25T10:17:40.568 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49519.log 2024-04-25T10:17:40.568 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.52745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42946.log 2024-04-25T10:17:40.568 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34340.log 2024-04-25T10:17:40.568 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2%/var/log/ceph/ceph-client.admin.42946.log: -- replaced with /var/log/ceph/ceph-client.admin.49519.log.gz 2024-04-25T10:17:40.568 INFO:teuthology.orchestra.run.smithi060.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42946.log.gz 2024-04-25T10:17:40.568 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44111.log 2024-04-25T10:17:40.568 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39222.log 2024-04-25T10:17:40.568 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6%/var/log/ceph/ceph-client.admin.44111.log: -- replaced with /var/log/ceph/ceph-client.admin.34340.log.gz 2024-04-25T10:17:40.568 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44111.log.gz 2024-04-25T10:17:40.568 INFO:teuthology.orchestra.run.smithi060.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.0.52745.log.gz 2024-04-25T10:17:40.568 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54488.log 2024-04-25T10:17:40.568 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50040.log 2024-04-25T10:17:40.568 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39222.log: /var/log/ceph/ceph-client.admin.54488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41220.log 2024-04-25T10:17:40.568 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50040.log: 87.8% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39222.log.gz 2024-04-25T10:17:40.568 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54488.log.gz 2024-04-25T10:17:40.569 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50040.log.gz 2024-04-25T10:17:40.569 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50758.log 2024-04-25T10:17:40.569 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49127.log 2024-04-25T10:17:40.569 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41220.log: /var/log/ceph/ceph-client.admin.50758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50550.log 2024-04-25T10:17:40.569 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49127.log: 88.7% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41220.log.gz 2024-04-25T10:17:40.569 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50758.log.gz 2024-04-25T10:17:40.569 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49127.log.gz 2024-04-25T10:17:40.569 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49101.log 2024-04-25T10:17:40.569 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46007.log 2024-04-25T10:17:40.569 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50550.log: /var/log/ceph/ceph-client.admin.49101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49943.log 2024-04-25T10:17:40.569 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50550.log.gz 2024-04-25T10:17:40.569 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7%/var/log/ceph/ceph-client.admin.46007.log: -- replaced with /var/log/ceph/ceph-client.admin.49101.log.gz 2024-04-25T10:17:40.569 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46007.log.gz 2024-04-25T10:17:40.569 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43140.log 2024-04-25T10:17:40.570 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33310.log 2024-04-25T10:17:40.570 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35057.log 2024-04-25T10:17:40.570 INFO:teuthology.orchestra.run.smithi060.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.49943.log.gz 2024-04-25T10:17:40.570 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6%/var/log/ceph/ceph-client.admin.33310.log: -- replaced with /var/log/ceph/ceph-client.admin.43140.log.gz 2024-04-25T10:17:40.570 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33310.log.gz 2024-04-25T10:17:40.570 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54670.log 2024-04-25T10:17:40.570 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51318.log 2024-04-25T10:17:40.570 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54670.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53653.log 2024-04-25T10:17:40.570 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35057.log.gz 2024-04-25T10:17:40.570 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54670.log.gz 2024-04-25T10:17:40.570 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34764.log 2024-04-25T10:17:40.570 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51318.log.gz 2024-04-25T10:17:40.570 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.53653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48856.log 2024-04-25T10:17:40.571 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36479.log 2024-04-25T10:17:40.571 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9%/var/log/ceph/ceph-client.admin.48856.log: -- replaced with /var/log/ceph/ceph-client.admin.34764.log.gz 2024-04-25T10:17:40.571 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48856.log.gz 2024-04-25T10:17:40.571 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54161.log 2024-04-25T10:17:40.571 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44085.log 2024-04-25T10:17:40.571 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5%/var/log/ceph/ceph-client.admin.54161.log: -- replaced with /var/log/ceph/ceph-client.admin.36479.log.gz 2024-04-25T10:17:40.571 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54161.log.gz 2024-04-25T10:17:40.571 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42219.log 2024-04-25T10:17:40.571 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37530.log 2024-04-25T10:17:40.571 INFO:teuthology.orchestra.run.smithi060.stderr: 93.3%/var/log/ceph/ceph-client.admin.44085.log: -- replaced with /var/log/ceph/ceph-client.0.53653.log.gz 2024-04-25T10:17:40.571 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42219.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44085.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40642.log 2024-04-25T10:17:40.571 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42219.log.gz 2024-04-25T10:17:40.571 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:40.571 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49920.log 2024-04-25T10:17:40.572 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37530.log.gz 2024-04-25T10:17:40.572 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48346.log 2024-04-25T10:17:40.572 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40642.log: /var/log/ceph/ceph-client.admin.49920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39248.log 2024-04-25T10:17:40.572 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48346.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49920.log.gz 2024-04-25T10:17:40.572 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48346.log.gz -- replaced with /var/log/ceph/ceph-client.admin.40642.log.gz 2024-04-25T10:17:40.572 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:40.572 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54358.log 2024-04-25T10:17:40.572 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43382.log 2024-04-25T10:17:40.572 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54358.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39248.log.gz 2024-04-25T10:17:40.572 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47065.log 2024-04-25T10:17:40.572 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43382.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54358.log.gz 2024-04-25T10:17:40.572 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43382.log.gz 2024-04-25T10:17:40.572 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53786.log 2024-04-25T10:17:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46707.log 2024-04-25T10:17:40.573 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5%/var/log/ceph/ceph-client.admin.53786.log: -- replaced with /var/log/ceph/ceph-client.admin.47065.log.gz 2024-04-25T10:17:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27794.log 2024-04-25T10:17:40.573 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53786.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49743.log 2024-04-25T10:17:40.573 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46707.log: /var/log/ceph/ceph-client.admin.27794.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46707.log.gz 2024-04-25T10:17:40.573 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27794.log.gz 2024-04-25T10:17:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43114.log 2024-04-25T10:17:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37914.log 2024-04-25T10:17:40.573 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1%/var/log/ceph/ceph-client.admin.43114.log: -- replaced with /var/log/ceph/ceph-client.admin.49743.log.gz 2024-04-25T10:17:40.573 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54566.log 2024-04-25T10:17:40.574 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43114.log.gz 2024-04-25T10:17:40.574 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37914.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37914.log.gz 2024-04-25T10:17:40.574 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51344.log 2024-04-25T10:17:40.603 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52922.log 2024-04-25T10:17:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51344.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41376.log 2024-04-25T10:17:40.604 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54566.log.gz 2024-04-25T10:17:40.604 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51344.log.gz 2024-04-25T10:17:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52922.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52922.log.gz 2024-04-25T10:17:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35740.log 2024-04-25T10:17:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35292.log 2024-04-25T10:17:40.604 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4%/var/log/ceph/ceph-client.admin.35740.log: -- replaced with /var/log/ceph/ceph-client.admin.41376.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54410.log 2024-04-25T10:17:40.604 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:40.604 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35740.log.gz 2024-04-25T10:17:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35292.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35292.log.gz 2024-04-25T10:17:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39605.log 2024-04-25T10:17:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40492.log 2024-04-25T10:17:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39605.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47858.log 2024-04-25T10:17:40.605 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54410.log.gz 2024-04-25T10:17:40.605 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39605.log.gz 2024-04-25T10:17:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40492.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.40492.log.gz 2024-04-25T10:17:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50888.log 2024-04-25T10:17:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45470.log 2024-04-25T10:17:40.605 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6%/var/log/ceph/ceph-client.admin.50888.log: -- replaced with /var/log/ceph/ceph-client.admin.47858.log.gz 2024-04-25T10:17:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34130.log 2024-04-25T10:17:40.605 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50888.log.gz 2024-04-25T10:17:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45470.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45470.log.gz 2024-04-25T10:17:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49236.log 2024-04-25T10:17:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35636.log 2024-04-25T10:17:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49236.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34130.log.gz -5 2024-04-25T10:17:40.605 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41629.log 2024-04-25T10:17:40.605 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49236.log.gz 2024-04-25T10:17:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35636.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35636.log.gz 2024-04-25T10:17:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39814.log 2024-04-25T10:17:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44404.log 2024-04-25T10:17:40.606 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5%/var/log/ceph/ceph-client.admin.39814.log: -- replaced with /var/log/ceph/ceph-client.admin.41629.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46551.log 2024-04-25T10:17:40.606 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:40.606 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39814.log.gz 2024-04-25T10:17:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44404.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44404.log.gz 2024-04-25T10:17:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54462.log 2024-04-25T10:17:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38980.log 2024-04-25T10:17:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42059.log 2024-04-25T10:17:40.606 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.46551.log.gz/var/log/ceph/ceph-client.admin.38980.log: 2024-04-25T10:17:40.606 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54462.log.gz 2024-04-25T10:17:40.606 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38980.log.gz 2024-04-25T10:17:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44749.log 2024-04-25T10:17:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38895.log 2024-04-25T10:17:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44749.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42059.log.gz 2024-04-25T10:17:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35427.log 2024-04-25T10:17:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37940.log 2024-04-25T10:17:40.607 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44749.log.gz 2024-04-25T10:17:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38895.log: /var/log/ceph/ceph-client.admin.35427.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38895.log.gz 2024-04-25T10:17:40.607 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35427.log.gz 2024-04-25T10:17:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47780.log 2024-04-25T10:17:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50337.log 2024-04-25T10:17:40.607 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9%/var/log/ceph/ceph-client.admin.47780.log: -- replaced with /var/log/ceph/ceph-client.admin.37940.log.gz 2024-04-25T10:17:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35531.log 2024-04-25T10:17:40.607 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47780.log.gz 2024-04-25T10:17:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51477.log 2024-04-25T10:17:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32062.log 2024-04-25T10:17:40.607 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50337.log.gz 2024-04-25T10:17:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35531.log: /var/log/ceph/ceph-client.admin.51477.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35531.log.gz 2024-04-25T10:17:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35714.log 2024-04-25T10:17:40.608 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51477.log.gz 2024-04-25T10:17:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38398.log 2024-04-25T10:17:40.608 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47195.log 2024-04-25T10:17:40.608 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32062.log.gz 2024-04-25T10:17:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35714.log: /var/log/ceph/ceph-client.admin.38398.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35714.log.gz 2024-04-25T10:17:40.608 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38398.log.gz 2024-04-25T10:17:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35583.log 2024-04-25T10:17:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51257.log 2024-04-25T10:17:40.608 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5%/var/log/ceph/ceph-client.admin.35583.log: -- replaced with /var/log/ceph/ceph-client.admin.47195.log.gz 2024-04-25T10:17:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49179.log 2024-04-25T10:17:40.608 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35583.log.gz 2024-04-25T10:17:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51257.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51257.log.gz 2024-04-25T10:17:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44378.log 2024-04-25T10:17:40.609 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45548.log 2024-04-25T10:17:40.609 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8%/var/log/ceph/ceph-client.admin.44378.log: -- replaced with /var/log/ceph/ceph-client.admin.49179.log.gz 2024-04-25T10:17:40.609 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50469.log 2024-04-25T10:17:40.609 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44378.log.gz 2024-04-25T10:17:40.609 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45548.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45548.log.gz 2024-04-25T10:17:40.609 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45010.log 2024-04-25T10:17:40.609 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43245.log 2024-04-25T10:17:40.609 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6%/var/log/ceph/ceph-client.admin.45010.log: -- replaced with /var/log/ceph/ceph-client.admin.50469.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34000.log 2024-04-25T10:17:40.609 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:40.609 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45010.log.gz 2024-04-25T10:17:40.609 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43245.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43245.log.gz 2024-04-25T10:17:40.609 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39144.log 2024-04-25T10:17:40.609 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51787.log 2024-04-25T10:17:40.609 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34000.log.gz/var/log/ceph/ceph-client.admin.39144.log: 2024-04-25T10:17:40.610 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48085.log 2024-04-25T10:17:40.610 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39144.log.gz 2024-04-25T10:17:40.610 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51787.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51787.log.gz 2024-04-25T10:17:40.610 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49266.log 2024-04-25T10:17:40.610 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52975.log 2024-04-25T10:17:40.610 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9%/var/log/ceph/ceph-client.admin.49266.log: -- replaced with /var/log/ceph/ceph-client.admin.48085.log.gz 2024-04-25T10:17:40.610 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53202.log 2024-04-25T10:17:40.610 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49266.log.gz 2024-04-25T10:17:40.610 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52975.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52975.log.gz 2024-04-25T10:17:40.610 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31704.log 2024-04-25T10:17:40.610 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47884.log 2024-04-25T10:17:40.610 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.31704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34418.log 2024-04-25T10:17:40.610 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31704.log.gz 2024-04-25T10:17:40.610 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53202.log.gz 2024-04-25T10:17:40.611 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47884.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47884.log.gz 2024-04-25T10:17:40.611 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48650.log 2024-04-25T10:17:40.611 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49492.log 2024-04-25T10:17:40.611 INFO:teuthology.orchestra.run.smithi060.stderr: 89.8%/var/log/ceph/ceph-client.admin.48650.log: -- replaced with /var/log/ceph/ceph-client.admin.34418.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51175.log 2024-04-25T10:17:40.611 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:40.611 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48650.log.gz 2024-04-25T10:17:40.611 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49492.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49492.log.gz 2024-04-25T10:17:40.611 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48970.log 2024-04-25T10:17:40.611 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41024.log 2024-04-25T10:17:40.611 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5%/var/log/ceph/ceph-client.admin.48970.log: -- replaced with /var/log/ceph/ceph-client.admin.51175.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41160.log 2024-04-25T10:17:40.611 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:40.611 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48970.log.gz 2024-04-25T10:17:40.611 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41024.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41024.log.gz 2024-04-25T10:17:40.611 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32323.log 2024-04-25T10:17:40.612 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43166.log 2024-04-25T10:17:40.612 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7%/var/log/ceph/ceph-client.admin.32323.log: -- replaced with /var/log/ceph/ceph-client.admin.41160.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32529.log 2024-04-25T10:17:40.612 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:40.612 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32323.log.gz 2024-04-25T10:17:40.612 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43166.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43166.log.gz 2024-04-25T10:17:40.612 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31824.log 2024-04-25T10:17:40.612 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37504.log 2024-04-25T10:17:40.612 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6%/var/log/ceph/ceph-client.admin.31824.log: -- replaced with /var/log/ceph/ceph-client.admin.32529.log.gzgzip 2024-04-25T10:17:40.612 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34210.log 2024-04-25T10:17:40.612 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31824.log.gz 2024-04-25T10:17:40.612 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37504.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37504.log.gz 2024-04-25T10:17:40.612 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50862.log 2024-04-25T10:17:40.612 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40142.log 2024-04-25T10:17:40.612 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34210.log: /var/log/ceph/ceph-client.admin.50862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43612.log 2024-04-25T10:17:40.612 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34210.log.gz 2024-04-25T10:17:40.613 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50862.log.gz 2024-04-25T10:17:40.613 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40142.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40142.log.gz 2024-04-25T10:17:40.613 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46902.log 2024-04-25T10:17:40.613 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45957.log 2024-04-25T10:17:40.613 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46902.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47482.log 2024-04-25T10:17:40.613 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43612.log.gz 2024-04-25T10:17:40.613 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46902.log.gz 2024-04-25T10:17:40.613 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45957.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.45957.log.gz 2024-04-25T10:17:40.613 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45170.log 2024-04-25T10:17:40.613 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47482.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:17:40.613 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32927.log 2024-04-25T10:17:40.613 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45170.log: 91.1% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45170.log.gz 2024-04-25T10:17:40.613 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47482.log.gz/var/log/ceph/ceph.audit.log: 2024-04-25T10:17:40.613 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:17:40.613 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T10:17:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38109.log 2024-04-25T10:17:40.614 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1%/var/log/ceph/ceph-mds.c.log: -- replaced with /var/log/ceph/ceph-client.admin.32927.log.gz 2024-04-25T10:17:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39958.log 2024-04-25T10:17:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38109.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38109.log.gz 2024-04-25T10:17:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32320.log 2024-04-25T10:17:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52870.log 2024-04-25T10:17:40.614 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0%/var/log/ceph/ceph-client.admin.32320.log: -- replaced with /var/log/ceph/ceph-client.admin.39958.log.gz 2024-04-25T10:17:40.614 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32320.log.gz 2024-04-25T10:17:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43791.log 2024-04-25T10:17:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37030.log 2024-04-25T10:17:40.614 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4%/var/log/ceph/ceph-client.admin.43791.log: -- replaced with /var/log/ceph/ceph-client.admin.52870.log.gz 2024-04-25T10:17:40.614 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43791.log.gz 2024-04-25T10:17:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53177.log 2024-04-25T10:17:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43638.log 2024-04-25T10:17:40.615 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37030.log.gz 2024-04-25T10:17:40.615 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53177.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.53177.log.gz 2024-04-25T10:17:40.615 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54031.log 2024-04-25T10:17:40.615 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43638.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T10:17:40.615 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54031.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54031.log.gz 2024-04-25T10:17:40.615 INFO:teuthology.orchestra.run.smithi060.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.43638.log.gz 2024-04-25T10:17:40.615 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37374.log 2024-04-25T10:17:40.621 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43688.log 2024-04-25T10:17:40.621 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37374.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37374.log.gz 2024-04-25T10:17:40.621 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32244.log 2024-04-25T10:17:40.621 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43688.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43688.log.gz 2024-04-25T10:17:40.621 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37478.log 2024-04-25T10:17:40.621 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32244.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32244.log.gz 2024-04-25T10:17:40.621 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32735.log 2024-04-25T10:17:40.621 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37478.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37478.log.gz 2024-04-25T10:17:40.621 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43536.log 2024-04-25T10:17:40.621 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32735.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32735.log.gz 2024-04-25T10:17:40.621 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44196.log 2024-04-25T10:17:40.621 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43536.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43536.log.gz 2024-04-25T10:17:40.622 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52326.log 2024-04-25T10:17:40.622 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44196.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44196.log.gz 2024-04-25T10:17:40.622 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54904.log 2024-04-25T10:17:40.622 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52326.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52326.log.gz 2024-04-25T10:17:40.622 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42622.log 2024-04-25T10:17:40.622 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54904.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54904.log.gz 2024-04-25T10:17:40.622 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37190.log 2024-04-25T10:17:40.622 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42622.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42622.log.gz 2024-04-25T10:17:40.622 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54774.log 2024-04-25T10:17:40.622 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37190.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.37190.log.gz 2024-04-25T10:17:40.622 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42785.log 2024-04-25T10:17:40.622 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54774.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54774.log.gz 2024-04-25T10:17:40.622 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32531.log 2024-04-25T10:17:40.622 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42785.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.42785.log.gz 2024-04-25T10:17:40.623 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41916.log 2024-04-25T10:17:40.623 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32531.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32531.log.gz 2024-04-25T10:17:40.623 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42033.log 2024-04-25T10:17:40.624 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41916.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41916.log.gz 2024-04-25T10:17:40.624 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41082.log 2024-04-25T10:17:40.624 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42033.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42033.log.gz 2024-04-25T10:17:40.624 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50495.log 2024-04-25T10:17:40.624 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41082.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41082.log.gz 2024-04-25T10:17:40.624 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50212.log 2024-04-25T10:17:40.625 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50495.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50495.log.gz 2024-04-25T10:17:40.625 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48542.log 2024-04-25T10:17:40.626 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50212.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50212.log.gz 2024-04-25T10:17:40.627 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36282.log 2024-04-25T10:17:40.627 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48542.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48542.log.gz 2024-04-25T10:17:40.628 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33887.log 2024-04-25T10:17:40.629 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36282.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36282.log.gz 2024-04-25T10:17:40.629 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49205.log 2024-04-25T10:17:40.630 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33887.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33887.log.gz 2024-04-25T10:17:40.630 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34366.log 2024-04-25T10:17:40.631 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49205.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49205.log.gz 2024-04-25T10:17:40.631 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33336.log 2024-04-25T10:17:40.632 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34366.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34366.log.gz 2024-04-25T10:17:40.633 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44919.log 2024-04-25T10:17:40.633 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33336.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33336.log.gz 2024-04-25T10:17:40.634 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42920.log 2024-04-25T10:17:40.635 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44919.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44919.log.gz 2024-04-25T10:17:40.635 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39869.log 2024-04-25T10:17:40.636 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42920.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42920.log.gz 2024-04-25T10:17:40.636 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47642.log 2024-04-25T10:17:40.637 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39869.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39869.log.gz 2024-04-25T10:17:40.637 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32036.log 2024-04-25T10:17:40.638 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47642.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47642.log.gz 2024-04-25T10:17:40.639 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47910.log 2024-04-25T10:17:40.639 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32036.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32036.log.gz 2024-04-25T10:17:40.640 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49153.log 2024-04-25T10:17:40.640 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47910.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47910.log.gz 2024-04-25T10:17:40.641 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48398.log 2024-04-25T10:17:40.642 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49153.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49153.log.gz 2024-04-25T10:17:40.642 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54540.log 2024-04-25T10:17:40.643 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48398.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48398.log.gz 2024-04-25T10:17:40.643 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38320.log 2024-04-25T10:17:40.644 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54540.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54540.log.gz 2024-04-25T10:17:40.644 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38268.log 2024-04-25T10:17:40.645 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38320.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38320.log.gz 2024-04-25T10:17:40.645 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48754.log 2024-04-25T10:17:40.646 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38268.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38268.log.gz 2024-04-25T10:17:40.647 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53111.log 2024-04-25T10:17:40.647 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48754.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48754.log.gz 2024-04-25T10:17:40.648 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49440.log 2024-04-25T10:17:40.649 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53111.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53111.log.gz 2024-04-25T10:17:40.649 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34660.log 2024-04-25T10:17:40.650 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49440.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49440.log.gz 2024-04-25T10:17:40.650 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39897.log 2024-04-25T10:17:40.651 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34660.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34660.log.gz 2024-04-25T10:17:40.652 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40948.log 2024-04-25T10:17:40.652 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39897.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39897.log.gz 2024-04-25T10:17:40.653 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32324.log 2024-04-25T10:17:40.653 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40948.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40948.log.gz 2024-04-25T10:17:40.654 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41772.log 2024-04-25T10:17:40.655 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32324.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32324.log.gz 2024-04-25T10:17:40.655 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54644.log 2024-04-25T10:17:40.656 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41772.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41772.log.gz 2024-04-25T10:17:40.656 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50576.log 2024-04-25T10:17:40.657 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54644.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54644.log.gz 2024-04-25T10:17:40.657 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48594.log 2024-04-25T10:17:40.658 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50576.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50576.log.gz 2024-04-25T10:17:40.659 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33284.log 2024-04-25T10:17:40.659 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48594.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48594.log.gz 2024-04-25T10:17:40.660 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51733.log 2024-04-25T10:17:40.660 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33284.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33284.log.gz 2024-04-25T10:17:40.661 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32935.log 2024-04-25T10:17:40.662 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51733.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51733.log.gz 2024-04-25T10:17:40.662 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37087.log 2024-04-25T10:17:40.663 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32935.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32935.log.gz 2024-04-25T10:17:40.663 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34052.log 2024-04-25T10:17:40.664 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37087.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37087.log.gz 2024-04-25T10:17:40.665 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33232.log 2024-04-25T10:17:40.666 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34052.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34052.log.gz 2024-04-25T10:17:40.666 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47532.log 2024-04-25T10:17:40.667 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33232.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33232.log.gz 2024-04-25T10:17:40.668 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33206.log 2024-04-25T10:17:40.669 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47532.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47532.log.gz 2024-04-25T10:17:40.669 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53252.log 2024-04-25T10:17:40.670 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33206.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33206.log.gz 2024-04-25T10:17:40.670 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35161.log 2024-04-25T10:17:40.671 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53252.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53252.log.gz 2024-04-25T10:17:40.672 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44553.log 2024-04-25T10:17:40.672 INFO:teuthology.orchestra.run.smithi100.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T10:17:40.673 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35161.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35161.log.gz 2024-04-25T10:17:40.673 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43218.log 2024-04-25T10:17:40.674 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44553.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44553.log.gz 2024-04-25T10:17:40.674 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53950.log 2024-04-25T10:17:40.675 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43218.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43218.log.gz 2024-04-25T10:17:40.676 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52378.log 2024-04-25T10:17:40.677 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53950.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53950.log.gz 2024-04-25T10:17:40.677 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52586.log 2024-04-25T10:17:40.678 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52378.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52378.log.gz 2024-04-25T10:17:40.678 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44975.log 2024-04-25T10:17:40.679 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52586.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52586.log.gz 2024-04-25T10:17:40.680 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37783.log 2024-04-25T10:17:40.681 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44975.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44975.log.gz 2024-04-25T10:17:40.681 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53734.log 2024-04-25T10:17:40.682 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37783.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37783.log.gz 2024-04-25T10:17:40.683 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54384.log 2024-04-25T10:17:40.684 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53734.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53734.log.gz 2024-04-25T10:17:40.684 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36635.log 2024-04-25T10:17:40.685 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54384.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54384.log.gz 2024-04-25T10:17:40.685 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37816.log 2024-04-25T10:17:40.690 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36635.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.36635.log.gz 2024-04-25T10:17:40.691 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53382.log 2024-04-25T10:17:40.691 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37816.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37816.log.gz 2024-04-25T10:17:40.691 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37296.log 2024-04-25T10:17:40.691 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53382.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.53382.log.gz 2024-04-25T10:17:40.691 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38346.log 2024-04-25T10:17:40.692 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37296.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37296.log.gz 2024-04-25T10:17:40.692 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36847.log 2024-04-25T10:17:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38346.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38346.log.gz 2024-04-25T10:17:40.694 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33573.log 2024-04-25T10:17:40.695 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36847.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36847.log.gz 2024-04-25T10:17:40.695 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37234.log 2024-04-25T10:17:40.696 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33573.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33573.log.gz 2024-04-25T10:17:40.696 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42245.log 2024-04-25T10:17:40.698 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37234.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37234.log.gz 2024-04-25T10:17:40.698 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32192.log 2024-04-25T10:17:40.699 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42245.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42245.log.gz 2024-04-25T10:17:40.699 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34288.log 2024-04-25T10:17:40.700 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32192.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32192.log.gz 2024-04-25T10:17:40.700 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36925.log 2024-04-25T10:17:40.701 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34288.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34288.log.gz 2024-04-25T10:17:40.702 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51123.log 2024-04-25T10:17:40.703 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36925.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36925.log.gz 2024-04-25T10:17:40.703 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47250.log 2024-04-25T10:17:40.704 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51123.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51123.log.gz 2024-04-25T10:17:40.705 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44274.log 2024-04-25T10:17:40.706 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.47250.log: 92.4% -- replaced with /var/log/ceph/ceph-client.0.47250.log.gz 2024-04-25T10:17:40.707 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47728.log 2024-04-25T10:17:40.708 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44274.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44274.log.gz 2024-04-25T10:17:40.708 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52352.log 2024-04-25T10:17:40.709 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47728.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47728.log.gz 2024-04-25T10:17:40.709 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50104.log 2024-04-25T10:17:40.710 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52352.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52352.log.gz 2024-04-25T10:17:40.711 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51660.log 2024-04-25T10:17:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50104.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50104.log.gz 2024-04-25T10:17:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38294.log 2024-04-25T10:17:40.719 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.51660.log: 92.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31928.log 2024-04-25T10:17:40.720 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38294.log: 87.7% -- replaced with /var/log/ceph/ceph-client.0.51660.log.gz 2024-04-25T10:17:40.721 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38294.log.gz 2024-04-25T10:17:40.721 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54232.log 2024-04-25T10:17:40.722 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.31928.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31928.log.gz 2024-04-25T10:17:40.722 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42674.log 2024-04-25T10:17:40.723 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54232.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54232.log.gz 2024-04-25T10:17:40.723 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45679.log 2024-04-25T10:17:40.724 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42674.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42674.log.gz 2024-04-25T10:17:40.725 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53430.log 2024-04-25T10:17:40.726 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45679.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45679.log.gz 2024-04-25T10:17:40.726 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47996.log 2024-04-25T10:17:40.727 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53430.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53430.log.gz 2024-04-25T10:17:40.727 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41837.log 2024-04-25T10:17:40.729 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47996.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47996.log.gz 2024-04-25T10:17:40.729 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36899.log 2024-04-25T10:17:40.730 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41837.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41837.log.gz 2024-04-25T10:17:40.730 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32740.log 2024-04-25T10:17:40.731 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36899.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36899.log.gz 2024-04-25T10:17:40.732 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35083.log 2024-04-25T10:17:40.733 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32740.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32740.log.gz 2024-04-25T10:17:40.733 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44527.log 2024-04-25T10:17:40.734 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35083.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35083.log.gz 2024-04-25T10:17:40.734 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42762.log 2024-04-25T10:17:40.735 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44527.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44527.log.gz 2024-04-25T10:17:40.736 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46271.log 2024-04-25T10:17:40.737 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42762.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42762.log.gz 2024-04-25T10:17:40.737 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53001.log 2024-04-25T10:17:40.738 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46271.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46271.log.gz 2024-04-25T10:17:40.738 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42113.log 2024-04-25T10:17:40.739 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53001.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53001.log.gz 2024-04-25T10:17:40.740 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42569.log 2024-04-25T10:17:40.741 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42113.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42113.log.gz 2024-04-25T10:17:40.741 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51292.log 2024-04-25T10:17:40.742 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42569.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42569.log.gz 2024-04-25T10:17:40.742 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38372.log 2024-04-25T10:17:40.743 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51292.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51292.log.gz 2024-04-25T10:17:40.744 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41324.log 2024-04-25T10:17:40.745 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38372.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38372.log.gz 2024-04-25T10:17:40.745 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50654.log 2024-04-25T10:17:40.746 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41324.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41324.log.gz 2024-04-25T10:17:40.746 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45444.log 2024-04-25T10:17:40.747 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50654.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50654.log.gz 2024-04-25T10:17:40.748 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32514.log 2024-04-25T10:17:40.749 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45444.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45444.log.gz 2024-04-25T10:17:40.749 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39118.log 2024-04-25T10:17:40.750 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32514.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32514.log.gz 2024-04-25T10:17:40.750 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37270.log 2024-04-25T10:17:40.751 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39118.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39118.log.gz 2024-04-25T10:17:40.752 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44456.log 2024-04-25T10:17:40.753 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37270.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37270.log.gz 2024-04-25T10:17:40.753 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36505.log 2024-04-25T10:17:40.754 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44456.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44456.log.gz 2024-04-25T10:17:40.755 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49742.log 2024-04-25T10:17:40.756 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36505.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36505.log.gz 2024-04-25T10:17:40.756 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53027.log 2024-04-25T10:17:40.757 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49742.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.49742.log.gz 2024-04-25T10:17:40.758 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33518.log 2024-04-25T10:17:40.759 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53027.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53027.log.gz 2024-04-25T10:17:40.759 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45839.log 2024-04-25T10:17:40.760 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33518.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.33518.log.gz 2024-04-25T10:17:40.761 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38671.log 2024-04-25T10:17:40.762 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45839.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45839.log.gz 2024-04-25T10:17:40.762 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40248.log 2024-04-25T10:17:40.763 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.38671.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38671.log.gz 2024-04-25T10:17:40.763 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44893.log 2024-04-25T10:17:40.764 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40248.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40248.log.gz 2024-04-25T10:17:40.765 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36531.log 2024-04-25T10:17:40.766 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44893.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44893.log.gz 2024-04-25T10:17:40.766 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53155.log 2024-04-25T10:17:40.767 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36531.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36531.log.gz 2024-04-25T10:17:40.767 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37627.log 2024-04-25T10:17:40.769 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53155.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.53155.log.gz 2024-04-25T10:17:40.769 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36583.log 2024-04-25T10:17:40.770 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.37627.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37627.log.gz 2024-04-25T10:17:40.770 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54005.log 2024-04-25T10:17:40.771 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36583.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36583.log.gz 2024-04-25T10:17:40.771 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34470.log 2024-04-25T10:17:40.772 INFO:teuthology.orchestra.run.smithi060.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T10:17:40.772 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54005.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54005.log.gz 2024-04-25T10:17:40.773 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36978.log 2024-04-25T10:17:40.773 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36609.log 2024-04-25T10:17:40.774 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34470.log.gz 2024-04-25T10:17:40.774 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36978.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36978.log.gz 2024-04-25T10:17:40.774 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48425.log 2024-04-25T10:17:40.774 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41525.log 2024-04-25T10:17:40.775 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8%/var/log/ceph/ceph-client.admin.48425.log: -- replaced with /var/log/ceph/ceph-client.admin.36609.log.gz 2024-04-25T10:17:40.775 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48425.log.gz 2024-04-25T10:17:40.775 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31876.log 2024-04-25T10:17:40.776 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53494.log 2024-04-25T10:17:40.776 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.31876.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41525.log.gz 2024-04-25T10:17:40.777 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31876.log.gz 2024-04-25T10:17:40.777 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41740.log 2024-04-25T10:17:40.777 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51892.log 2024-04-25T10:17:40.778 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5%/var/log/ceph/ceph-client.admin.41740.log: -- replaced with /var/log/ceph/ceph-client.admin.53494.log.gz 2024-04-25T10:17:40.778 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41740.log.gz 2024-04-25T10:17:40.779 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43562.log 2024-04-25T10:17:40.779 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33180.log 2024-04-25T10:17:40.779 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43562.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51892.log.gz 2024-04-25T10:17:40.780 INFO:teuthology.orchestra.run.smithi060.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43562.log.gz 2024-04-25T10:17:40.780 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40038.log 2024-04-25T10:17:40.780 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50810.log 2024-04-25T10:17:40.781 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40038.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33180.log.gz 2024-04-25T10:17:40.781 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40038.log.gz 2024-04-25T10:17:40.782 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47354.log 2024-04-25T10:17:40.782 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48249.log 2024-04-25T10:17:40.783 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6%/var/log/ceph/ceph-client.admin.47354.log: -- replaced with /var/log/ceph/ceph-client.admin.50810.log.gz 2024-04-25T10:17:40.783 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47354.log.gz 2024-04-25T10:17:40.783 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42997.log 2024-04-25T10:17:40.783 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41603.log 2024-04-25T10:17:40.784 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7%/var/log/ceph/ceph-client.admin.42997.log: -- replaced with /var/log/ceph/ceph-client.admin.48249.log.gz 2024-04-25T10:17:40.784 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42997.log.gz 2024-04-25T10:17:40.784 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53812.log 2024-04-25T10:17:40.785 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48281.log 2024-04-25T10:17:40.785 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8%/var/log/ceph/ceph-client.admin.53812.log: -- replaced with /var/log/ceph/ceph-client.admin.41603.log.gz 2024-04-25T10:17:40.786 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53812.log.gz 2024-04-25T10:17:40.787 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47616.log 2024-04-25T10:17:40.787 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41998.log 2024-04-25T10:17:40.787 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4%/var/log/ceph/ceph-client.admin.47616.log: -- replaced with /var/log/ceph/ceph-client.admin.48281.log.gz 2024-04-25T10:17:40.788 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47616.log.gz 2024-04-25T10:17:40.788 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36425.log 2024-04-25T10:17:40.788 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53872.log 2024-04-25T10:17:40.788 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0%/var/log/ceph/ceph-client.admin.36425.log: -- replaced with /var/log/ceph/ceph-client.admin.41998.log.gz 2024-04-25T10:17:40.789 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36425.log.gz 2024-04-25T10:17:40.789 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40515.log 2024-04-25T10:17:40.789 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46872.log 2024-04-25T10:17:40.789 INFO:teuthology.orchestra.run.smithi060.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53872.log.gz 2024-04-25T10:17:40.790 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40515.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.40515.log.gz 2024-04-25T10:17:40.790 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36178.log 2024-04-25T10:17:40.791 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51944.log 2024-04-25T10:17:40.791 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7%/var/log/ceph/ceph-client.admin.36178.log: -- replaced with /var/log/ceph/ceph-client.admin.46872.log.gz 2024-04-25T10:17:40.791 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36178.log.gz 2024-04-25T10:17:40.792 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38793.log 2024-04-25T10:17:40.792 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41707.log 2024-04-25T10:17:40.793 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9%/var/log/ceph/ceph-client.admin.38793.log: -- replaced with /var/log/ceph/ceph-client.admin.51944.log.gz 2024-04-25T10:17:40.793 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52522.log 2024-04-25T10:17:40.793 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38793.log.gz 2024-04-25T10:17:40.794 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41707.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41707.log.gz 2024-04-25T10:17:40.794 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50134.log 2024-04-25T10:17:40.794 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39006.log 2024-04-25T10:17:40.795 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9%/var/log/ceph/ceph-client.admin.50134.log: -- replaced with /var/log/ceph/ceph-client.admin.52522.log.gz 2024-04-25T10:17:40.795 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50134.log.gz 2024-04-25T10:17:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40064.log 2024-04-25T10:17:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42648.log 2024-04-25T10:17:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40064.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39006.log.gz 2024-04-25T10:17:40.796 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40064.log.gz 2024-04-25T10:17:40.797 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36338.log 2024-04-25T10:17:40.797 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39326.log 2024-04-25T10:17:40.798 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9%/var/log/ceph/ceph-client.admin.36338.log: -- replaced with /var/log/ceph/ceph-client.admin.42648.log.gz 2024-04-25T10:17:40.798 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36338.log.gz 2024-04-25T10:17:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50999.log 2024-04-25T10:17:40.799 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47456.log 2024-04-25T10:17:40.799 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50999.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39326.log.gz 2024-04-25T10:17:40.799 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50999.log.gz 2024-04-25T10:17:40.801 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35214.log 2024-04-25T10:17:40.801 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39378.log 2024-04-25T10:17:40.801 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2%/var/log/ceph/ceph-client.admin.35214.log: -- replaced with /var/log/ceph/ceph-client.admin.47456.log.gz 2024-04-25T10:17:40.801 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35214.log.gz 2024-04-25T10:17:40.801 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49074.log 2024-04-25T10:17:40.802 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39378.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39378.log.gz 2024-04-25T10:17:40.802 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53598.log 2024-04-25T10:17:40.802 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53053.log 2024-04-25T10:17:40.803 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4%/var/log/ceph/ceph-client.admin.53598.log: -- replaced with /var/log/ceph/ceph-client.admin.49074.log.gz 2024-04-25T10:17:40.803 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53598.log.gz 2024-04-25T10:17:40.803 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34951.log 2024-04-25T10:17:40.804 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52497.log 2024-04-25T10:17:40.804 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34951.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53053.log.gz 2024-04-25T10:17:40.804 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34951.log.gz 2024-04-25T10:17:40.805 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35375.log 2024-04-25T10:17:40.805 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48780.log 2024-04-25T10:17:40.806 INFO:teuthology.orchestra.run.smithi060.stderr: 90.0%/var/log/ceph/ceph-client.admin.35375.log: -- replaced with /var/log/ceph/ceph-client.admin.52497.log.gz 2024-04-25T10:17:40.806 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35375.log.gz 2024-04-25T10:17:40.807 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39274.log 2024-04-25T10:17:40.807 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31984.log 2024-04-25T10:17:40.807 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2%/var/log/ceph/ceph-client.admin.39274.log: -- replaced with /var/log/ceph/ceph-client.admin.48780.log.gz 2024-04-25T10:17:40.807 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39274.log.gz 2024-04-25T10:17:40.808 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46734.log 2024-04-25T10:17:40.808 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.31984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34184.log 2024-04-25T10:17:40.809 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46734.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31984.log.gz 2024-04-25T10:17:40.809 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46734.log.gz 2024-04-25T10:17:40.809 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48702.log 2024-04-25T10:17:40.810 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32088.log 2024-04-25T10:17:40.810 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34184.log.gz 2024-04-25T10:17:40.810 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48702.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48702.log.gz 2024-04-25T10:17:40.811 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32318.log 2024-04-25T10:17:40.811 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44222.log 2024-04-25T10:17:40.811 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32318.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32088.log.gz 2024-04-25T10:17:40.811 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32318.log.gz 2024-04-25T10:17:40.813 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45196.log 2024-04-25T10:17:40.813 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44059.log 2024-04-25T10:17:40.813 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9%/var/log/ceph/ceph-client.admin.45196.log: -- replaced with /var/log/ceph/ceph-client.admin.44222.log.gz 2024-04-25T10:17:40.813 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45196.log.gz 2024-04-25T10:17:40.813 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48137.log 2024-04-25T10:17:40.815 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50015.log 2024-04-25T10:17:40.815 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44059.log: /var/log/ceph/ceph-client.admin.48137.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44059.log.gz 2024-04-25T10:17:40.815 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48137.log.gz 2024-04-25T10:17:40.816 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27558.log 2024-04-25T10:17:40.816 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27678.log 2024-04-25T10:17:40.816 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.27558.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.50015.log.gz 2024-04-25T10:17:40.816 INFO:teuthology.orchestra.run.smithi060.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.27558.log.gz 2024-04-25T10:17:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50387.log 2024-04-25T10:17:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.27678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32739.log 2024-04-25T10:17:40.817 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0%/var/log/ceph/ceph-client.admin.50387.log: -- replaced with /var/log/ceph/ceph-client.admin.27678.log.gz 2024-04-25T10:17:40.817 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50387.log.gz 2024-04-25T10:17:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38528.log 2024-04-25T10:17:40.819 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54083.log 2024-04-25T10:17:40.819 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9%/var/log/ceph/ceph-client.admin.38528.log: -- replaced with /var/log/ceph/ceph-client.admin.32739.log.gz 2024-04-25T10:17:40.819 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38528.log.gz 2024-04-25T10:17:40.819 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50732.log 2024-04-25T10:17:40.819 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43297.log 2024-04-25T10:17:40.820 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50732.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54083.log.gz 2024-04-25T10:17:40.820 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50732.log.gz 2024-04-25T10:17:40.820 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32948.log 2024-04-25T10:17:40.820 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42828.log 2024-04-25T10:17:40.821 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.32948.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43297.log.gz 2024-04-25T10:17:40.822 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32948.log.gz 2024-04-25T10:17:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42193.log 2024-04-25T10:17:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39196.log 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42193.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42828.log.gz 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42193.log.gz 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33948.log 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50524.log 2024-04-25T10:17:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33948.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39196.log.gz 2024-04-25T10:17:40.824 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33948.log.gz 2024-04-25T10:17:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52221.log 2024-04-25T10:17:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33388.log 2024-04-25T10:17:40.825 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50524.log.gz 2024-04-25T10:17:40.825 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52221.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52221.log.gz 2024-04-25T10:17:40.825 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43323.log 2024-04-25T10:17:40.827 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.33388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33696.log 2024-04-25T10:17:40.827 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7%/var/log/ceph/ceph-client.admin.43323.log: -- replaced with /var/log/ceph/ceph-client.admin.33388.log.gz 2024-04-25T10:17:40.827 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43323.log.gz 2024-04-25T10:17:40.827 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46970.log 2024-04-25T10:17:40.827 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.33696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51149.log 2024-04-25T10:17:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46970.log: 90.8% -- replaced with /var/log/ceph/ceph-client.0.33696.log.gz 2024-04-25T10:17:40.829 INFO:teuthology.orchestra.run.smithi060.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.46970.log.gz 2024-04-25T10:17:40.829 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51149.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51149.log.gz 2024-04-25T10:17:40.854 INFO:teuthology.orchestra.run.smithi100.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T10:17:40.920 INFO:teuthology.orchestra.run.smithi060.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T10:17:40.987 INFO:teuthology.orchestra.run.smithi100.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T10:17:41.016 INFO:teuthology.orchestra.run.smithi100.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T10:17:41.039 INFO:teuthology.orchestra.run.smithi100.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T10:17:41.061 INFO:teuthology.orchestra.run.smithi060.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T10:17:41.070 INFO:teuthology.orchestra.run.smithi100.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T10:17:41.070 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-04-25T10:17:41.071 INFO:teuthology.orchestra.run.smithi100.stderr:real 0m0.718s 2024-04-25T10:17:41.071 INFO:teuthology.orchestra.run.smithi100.stderr:user 0m3.438s 2024-04-25T10:17:41.071 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m0.147s 2024-04-25T10:17:41.148 INFO:teuthology.orchestra.run.smithi060.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T10:17:41.166 INFO:teuthology.orchestra.run.smithi060.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T10:17:41.262 INFO:teuthology.orchestra.run.smithi060.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T10:17:41.345 INFO:teuthology.orchestra.run.smithi060.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T10:17:41.347 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T10:17:41.347 INFO:teuthology.orchestra.run.smithi060.stderr:real 0m1.034s 2024-04-25T10:17:41.347 INFO:teuthology.orchestra.run.smithi060.stderr:user 0m5.059s 2024-04-25T10:17:41.347 INFO:teuthology.orchestra.run.smithi060.stderr:sys 0m0.329s 2024-04-25T10:17:41.347 INFO:tasks.ceph:Archiving logs... 2024-04-25T10:17:41.347 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673046/remote/smithi060/log 2024-04-25T10:17:41.348 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:17:42.454 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673046/remote/smithi100/log 2024-04-25T10:17:42.455 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:17:42.821 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T10:17:42.831 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T10:17:42.831 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:17:42.833 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:17:42.906 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T10:17:42.914 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T10:17:42.914 DEBUG:teuthology.orchestra.run.smithi060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:17:42.917 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:17:42.928 INFO:teuthology.orchestra.run.smithi060.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T10:17:42.928 INFO:teuthology.orchestra.run.smithi060.stdout:============================================================================== 2024-04-25T10:17:42.928 INFO:teuthology.orchestra.run.smithi060.stdout:+hv01.front.sepi 67.205.162.81 3 u 49 64 377 0.060 -1.893 0.190 2024-04-25T10:17:42.928 INFO:teuthology.orchestra.run.smithi060.stdout:+hv02.front.sepi 63.231.80.2 3 u 54 64 377 0.092 +0.268 0.185 2024-04-25T10:17:42.928 INFO:teuthology.orchestra.run.smithi060.stdout:*hv03.front.sepi 74.6.168.72 3 u 53 64 377 0.071 -0.462 0.107 2024-04-25T10:17:42.929 INFO:teuthology.orchestra.run.smithi060.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-04-25T10:17:42.930 INFO:teuthology.orchestra.run.smithi100.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T10:17:42.930 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================== 2024-04-25T10:17:42.931 INFO:teuthology.orchestra.run.smithi100.stdout:*hv01.front.sepi 67.205.162.81 3 u 46 64 377 0.086 -7.227 2.964 2024-04-25T10:17:42.931 INFO:teuthology.orchestra.run.smithi100.stdout:+hv02.front.sepi 63.231.80.2 3 u 48 64 377 0.111 -4.537 2.776 2024-04-25T10:17:42.931 INFO:teuthology.orchestra.run.smithi100.stdout:+hv03.front.sepi 74.6.168.72 3 u 50 64 377 0.097 -5.436 2.861 2024-04-25T10:17:42.931 INFO:teuthology.orchestra.run.smithi100.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-04-25T10:17:42.932 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T10:17:42.940 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T10:17:42.941 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T10:17:42.950 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T10:17:42.958 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T10:17:42.966 INFO:teuthology.task.internal:Duration was 1401.275181 seconds 2024-04-25T10:17:42.967 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T10:17:42.975 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T10:17:42.975 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:17:42.977 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:17:43.013 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T10:17:43.013 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T10:17:43.013 DEBUG:teuthology.orchestra.run.smithi060:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:17:43.066 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T10:17:43.066 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:17:43.076 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T10:17:43.076 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:17:43.110 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:17:43.149 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T10:17:43.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:17:43.182 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:17:43.258 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T10:17:43.268 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T10:17:43.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:17:43.281 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:17:43.292 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T10:17:43.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:17:43.326 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:17:43.335 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = core 2024-04-25T10:17:43.350 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-04-25T10:17:43.359 DEBUG:teuthology.orchestra.run.smithi060:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:17:43.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:17:43.397 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:17:43.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:17:43.404 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T10:17:43.415 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T10:17:43.416 DEBUG:teuthology.misc:Transferring archived files from smithi060:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673046/remote/smithi060 2024-04-25T10:17:43.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:17:43.462 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673046/remote/smithi100 2024-04-25T10:17:43.463 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:17:43.484 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T10:17:43.485 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:17:43.506 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:17:43.529 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T10:17:43.541 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T10:17:43.541 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T10:17:43.550 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T10:17:43.550 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:17:43.554 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:17:43.558 INFO:teuthology.orchestra.run.smithi060.stdout: 658649 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:17 /home/ubuntu/cephtest 2024-04-25T10:17:43.577 INFO:teuthology.orchestra.run.smithi100.stdout: 658624 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:17 /home/ubuntu/cephtest 2024-04-25T10:17:43.578 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T10:17:43.589 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T10:17:43.616 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} duration: 1401.2751805782318 flavor: default owner: scheduled_neesingh@teuthology success: true 2024-04-25T10:17:43.616 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:17:43.703 INFO:teuthology.run:pass